2024-03-28T17:39:21.329 INFO:root:teuthology version: 0.0.1.dev262+ge691533 2024-03-28T17:39:21.329 DEBUG:teuthology.run:Teuthology command: teuthology --archive /home/teuthworker/archive/mchangir-2024-03-27_03:45:23-fs-wip-mchangir-testing-main-20240318.032620-testing-default-smithi/7625376 --name mchangir-2024-03-27_03:45:23-fs-wip-mchangir-testing-main-20240318.032620-testing-default-smithi --owner mchangir --description fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate} clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} mount/fuse objectstore-ec/bluestore-bitmap omap_limit/10000 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts session_timeout} ranks/multi/{balancer/automatic export-check n/5 replication/default} standby-replay tasks/{0-subvolume/{with-quota} 1-check-counter 2-scrub/yes 3-snaps/yes 4-flush/no 5-workunit/fs/norstats}} --verbose -- /home/teuthworker/archive/mchangir-2024-03-27_03:45:23-fs-wip-mchangir-testing-main-20240318.032620-testing-default-smithi/7625376/orig.config.yaml 2024-03-28T17:39:21.384 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-03-28T17:39:21.473 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/mchangir-2024-03-27_03:45:23-fs-wip-mchangir-testing-main-20240318.032620-testing-default-smithi/7625376 branch: wip-mchangir-testing-main-20240318.032620 description: fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate} clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} mount/fuse objectstore-ec/bluestore-bitmap omap_limit/10000 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts session_timeout} ranks/multi/{balancer/automatic export-check n/5 replication/default} standby-replay tasks/{0-subvolume/{with-quota} 1-check-counter 2-scrub/yes 3-snaps/yes 4-flush/no 5-workunit/fs/norstats}} email: mchangir@redhat.com first_in_suite: false job_id: '7625376' kernel: flavor: default kdb: true sha1: 1df0c7e8a2354c42d738e5454ba83ea7332deac2 last_in_suite: false log-rotate: ceph-mds: 10G ceph-osd: 10G machine_type: smithi name: mchangir-2024-03-27_03:45:23-fs-wip-mchangir-testing-main-20240318.032620-testing-default-smithi no_nested_subset: false nuke-on-error: true openstack: - volumes: count: 4 size: 20 - machine: disk: 200 os_type: centos os_version: 9.stream overrides: admin_socket: branch: wip-mchangir-testing-main-20240318.032620 ceph: cephfs: max_mds: 5 session_timeout: 300 standby_replay: true conf: client: client dirsize rbytes: false client mount timeout: 600 debug client: 20 debug ms: 1 rados mon op timeout: 900 rados osd op timeout: 900 mds: debug mds: 20 debug mds balancer: 20 debug ms: 1 mds bal fragment size max: 10000 mds bal merge size: 5 mds bal split bits: 3 mds bal split size: 100 mds debug frag: true mds debug scatterstat: true mds op complaint time: 180 mds verify scatter: true osd op complaint time: 180 rados mon op timeout: 900 rados osd op timeout: 900 mgr: cephsqlite lock renewal timeout: 900000 debug cephsqlite: 20 debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon op complaint time: 120 osd: bdev async discard: true bdev enable discard: true bluestore allocator: bitmap bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug ms: 1 debug osd: 20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore osd op complaint time: 180 osd shutdown pgref assert: true osd_max_omap_entries_per_request: 10000 flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - FS_DEGRADED - FS_INLINE_DATA_DEPRECATED - FS_WITH_FAILED_MDS - MDS_ALL_DOWN - MDS_DAMAGE - MDS_DEGRADED - MDS_FAILED - MDS_INSUFFICIENT_STANDBY - MDS_UP_LESS_THAN_MAX - POOL_APP_NOT_ENABLED - overall HEALTH_ - Replacing daemon - deprecated feature inline_data - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding - MON_DOWN - slow metadata IO - SLOW_OPS - slow request sha1: 3be9a5c9bc793e11e2800a8c0c696e8b46742033 subvols: create: 2 subvol_options: --size 25000000000 ceph-deploy: bluestore: true conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} osd: bdev async discard: true bdev enable discard: true bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore fs: xfs ceph-fuse: client.0: mount_subvol_num: 0 check-counter: counters: mds: - mds.exported - mds.imported - mds.root_rsnaps - mds_server.req_mksnap_latency.avgcount - mds_server.req_rmsnap_latency.avgcount install: ceph: flavor: default sha1: 3be9a5c9bc793e11e2800a8c0c696e8b46742033 kclient: client.0: mount_subvol_num: 1 rbytes: false selinux: allowlist: - scontext=system_u:system_r:logrotate_t:s0 - scontext=system_u:system_r:getty_t:s0 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: wip-mchangir-testing-main-20240318.032620 sha1: 3be9a5c9bc793e11e2800a8c0c696e8b46742033 owner: mchangir priority: 100 repo: https://git.ceph.com/ceph-ci.git roles: - - mon.a - mgr.x - mds.a - mds.d - mds.g - mds.j - osd.0 - osd.3 - osd.6 - osd.9 - client.0 - - mon.b - mgr.y - mds.b - mds.e - mds.h - mds.k - osd.1 - osd.4 - osd.7 - osd.10 - - mon.c - mgr.z - mds.c - mds.f - mds.i - mds.l - osd.2 - osd.5 - osd.8 - osd.11 seed: 3645 sha1: 3be9a5c9bc793e11e2800a8c0c696e8b46742033 sleep_before_teardown: 0 subset: 132/256 suite: fs suite_branch: wip-mchangir-testing-main-20240318.032620 suite_path: /home/teuthworker/src/git.ceph.com_ceph-c_3be9a5c9bc793e11e2800a8c0c696e8b46742033/qa suite_relpath: qa suite_repo: https://git.ceph.com/ceph-ci.git suite_sha1: 3be9a5c9bc793e11e2800a8c0c696e8b46742033 syslog: ignorelist: - WARNING*.*check_session_state - WARNING*.*__ceph_remove_cap targets: smithi022.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBOa2HKHhj5Vc8iNUvTZhS/eoOwNQZN69A9U1VwznDG/NepO+6ayTLGZRGYYR0qUKDXyWhUBl/b8889hia3sqIvg= smithi079.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBB7Abtl2sQsB/bRyzMXY9FjKoOuDRyCT0mPM/zdFwu5pSoryJxJv6ZpgMSQgqDzGRBhBII1hcOHEC+aonnKuiuU= smithi134.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBHkuaQm8LDeSqSfdQunMTobHGvxWAeu2rYCX3w/N330hS+vi5cg7uY/QbwDQ3YHoojhMK9gBtp7HaQ1lWuMoupY= tasks: - install: extra_packages: deb: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror rpm: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror extra_system_packages: deb: - bison - flex - libelf-dev - libssl-dev - network-manager - iproute2 - util-linux - dump - indent - libaio-dev - libtool-bin - uuid-dev - xfslibs-dev - postgresql - postgresql-client - postgresql-common - postgresql-contrib rpm: - bison - flex - elfutils-libelf-devel - openssl-devel - NetworkManager - iproute - util-linux - libacl-devel - libaio-devel - libattr-devel - libtool - libuuid-devel - xfsdump - xfsprogs - xfsprogs-devel - libaio-devel - libtool - libuuid-devel - xfsprogs-devel - postgresql - postgresql-server - postgresql-contrib - cephadm: roleless: false - cephadm.shell: mon.a: - ceph orch status - ceph orch ps - ceph orch ls - ceph orch host ls - ceph orch device ls - cephadm.shell: mon.a: - ceph fs dump - ceph osd dump - fs.ready: timeout: 300 - ceph-fuse: null - exec: mon.a: - ceph fs set cephfs balance_automate true - check-counter: {} - fwd_scrub: scrub_timeout: 900 sleep_between_iterations: 1 - exec: mon.a: - ceph mgr module enable snap_schedule - ceph config set mgr mgr/snap_schedule/allow_m_granularity true - ceph config set mgr mgr/snap_schedule/dump_on_update true - ceph fs snap-schedule add --fs=cephfs --path=/ --snap_schedule=1m - ceph fs snap-schedule retention add --fs=cephfs --path=/ --retention-spec-or-period=6m3h - ceph fs snap-schedule status --fs=cephfs --path=/ - ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true - date +%s > START_TIME - full_sequential_finally: - exec: mon.a: - date +%s > END_TIME - START_TIME=$(cat START_TIME); END_TIME=$(cat END_TIME); DIFF_TIME=$((600-(END_TIME-START_TIME))); if [ "$DIFF_TIME" -gt 0 ]; then sleep "$DIFF_TIME"; fi - ceph fs snap-schedule status --fs=cephfs --path=/ - ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true - workunit: clients: all: - fs/norstats teuthology: fragments_dropped: [] meta: {} postmerge: - local function is_kupstream() return false end - local function is_kdistro() return false end - local function is_fuse() return true end - local function syntax_version() return '' end teuthology_branch: main teuthology_sha1: e691533f9cbb33d85b2187bba20d7102f098636d timestamp: 2024-03-27_03:45:23 tube: smithi user: mchangir verbose: false worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.2226885 2024-03-28T17:39:21.474 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/git.ceph.com_ceph-c_3be9a5c9bc793e11e2800a8c0c696e8b46742033/qa; will attempt to use it 2024-03-28T17:39:21.474 INFO:teuthology.run:Found tasks at /home/teuthworker/src/git.ceph.com_ceph-c_3be9a5c9bc793e11e2800a8c0c696e8b46742033/qa/tasks 2024-03-28T17:39:21.475 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-03-28T17:39:21.476 INFO:teuthology.task.internal:Checking packages... 2024-03-28T17:39:21.499 INFO:teuthology.task.internal:Checking packages for os_type 'centos', flavor 'default' and ceph hash '3be9a5c9bc793e11e2800a8c0c696e8b46742033' 2024-03-28T17:39:21.499 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-03-28T17:39:21.499 INFO:teuthology.packaging:ref: None 2024-03-28T17:39:21.500 INFO:teuthology.packaging:tag: None 2024-03-28T17:39:21.500 INFO:teuthology.packaging:branch: wip-mchangir-testing-main-20240318.032620 2024-03-28T17:39:21.500 INFO:teuthology.packaging:sha1: 3be9a5c9bc793e11e2800a8c0c696e8b46742033 2024-03-28T17:39:21.500 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=wip-mchangir-testing-main-20240318.032620 2024-03-28T17:39:21.644 INFO:teuthology.task.internal:Found packages for ceph version 19.0.0-2145.g3be9a5c9 2024-03-28T17:39:21.645 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-03-28T17:39:21.652 INFO:teuthology.task.internal:no buildpackages task found 2024-03-28T17:39:21.652 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-03-28T17:39:21.740 INFO:teuthology.task.internal:Saving configuration 2024-03-28T17:39:21.760 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-03-28T17:39:21.766 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-03-28T17:39:21.787 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi022.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/mchangir-2024-03-27_03:45:23-fs-wip-mchangir-testing-main-20240318.032620-testing-default-smithi/7625376', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-03-28 17:34:15.338782', 'locked_by': 'mchangir', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBOa2HKHhj5Vc8iNUvTZhS/eoOwNQZN69A9U1VwznDG/NepO+6ayTLGZRGYYR0qUKDXyWhUBl/b8889hia3sqIvg='} 2024-03-28T17:39:21.806 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi079.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/mchangir-2024-03-27_03:45:23-fs-wip-mchangir-testing-main-20240318.032620-testing-default-smithi/7625376', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-03-28 17:34:15.341637', 'locked_by': 'mchangir', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBB7Abtl2sQsB/bRyzMXY9FjKoOuDRyCT0mPM/zdFwu5pSoryJxJv6ZpgMSQgqDzGRBhBII1hcOHEC+aonnKuiuU='} 2024-03-28T17:39:21.827 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi134.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/mchangir-2024-03-27_03:45:23-fs-wip-mchangir-testing-main-20240318.032620-testing-default-smithi/7625376', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-03-28 17:34:15.340302', 'locked_by': 'mchangir', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBHkuaQm8LDeSqSfdQunMTobHGvxWAeu2rYCX3w/N330hS+vi5cg7uY/QbwDQ3YHoojhMK9gBtp7HaQ1lWuMoupY='} 2024-03-28T17:39:21.827 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-03-28T17:39:21.833 INFO:teuthology.task.internal:roles: ubuntu@smithi022.front.sepia.ceph.com - ['mon.a', 'mgr.x', 'mds.a', 'mds.d', 'mds.g', 'mds.j', 'osd.0', 'osd.3', 'osd.6', 'osd.9', 'client.0'] 2024-03-28T17:39:21.833 INFO:teuthology.task.internal:roles: ubuntu@smithi079.front.sepia.ceph.com - ['mon.b', 'mgr.y', 'mds.b', 'mds.e', 'mds.h', 'mds.k', 'osd.1', 'osd.4', 'osd.7', 'osd.10'] 2024-03-28T17:39:21.834 INFO:teuthology.task.internal:roles: ubuntu@smithi134.front.sepia.ceph.com - ['mon.c', 'mgr.z', 'mds.c', 'mds.f', 'mds.i', 'mds.l', 'osd.2', 'osd.5', 'osd.8', 'osd.11'] 2024-03-28T17:39:21.834 INFO:teuthology.run_tasks:Running task console_log... 2024-03-28T17:39:21.930 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7fa0a3ddb0d0>, signals=[15]) 2024-03-28T17:39:21.930 INFO:teuthology.run_tasks:Running task internal.connect... 2024-03-28T17:39:21.936 INFO:teuthology.task.internal:Opening connections... 2024-03-28T17:39:21.937 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi022.front.sepia.ceph.com 2024-03-28T17:39:21.939 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi022.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-28T17:39:22.046 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi079.front.sepia.ceph.com 2024-03-28T17:39:22.047 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi079.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-28T17:39:22.136 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi134.front.sepia.ceph.com 2024-03-28T17:39:22.137 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi134.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-28T17:39:22.224 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-03-28T17:39:22.231 DEBUG:teuthology.orchestra.run.smithi022:> uname -m 2024-03-28T17:39:22.249 INFO:teuthology.orchestra.run.smithi022.stdout:x86_64 2024-03-28T17:39:22.249 DEBUG:teuthology.orchestra.run.smithi022:> cat /etc/os-release 2024-03-28T17:39:22.306 INFO:teuthology.orchestra.run.smithi022.stdout:NAME="CentOS Stream" 2024-03-28T17:39:22.306 INFO:teuthology.orchestra.run.smithi022.stdout:VERSION="9" 2024-03-28T17:39:22.306 INFO:teuthology.orchestra.run.smithi022.stdout:ID="centos" 2024-03-28T17:39:22.306 INFO:teuthology.orchestra.run.smithi022.stdout:ID_LIKE="rhel fedora" 2024-03-28T17:39:22.306 INFO:teuthology.orchestra.run.smithi022.stdout:VERSION_ID="9" 2024-03-28T17:39:22.306 INFO:teuthology.orchestra.run.smithi022.stdout:PLATFORM_ID="platform:el9" 2024-03-28T17:39:22.306 INFO:teuthology.orchestra.run.smithi022.stdout:PRETTY_NAME="CentOS Stream 9" 2024-03-28T17:39:22.306 INFO:teuthology.orchestra.run.smithi022.stdout:ANSI_COLOR="0;31" 2024-03-28T17:39:22.306 INFO:teuthology.orchestra.run.smithi022.stdout:LOGO="fedora-logo-icon" 2024-03-28T17:39:22.306 INFO:teuthology.orchestra.run.smithi022.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-03-28T17:39:22.307 INFO:teuthology.orchestra.run.smithi022.stdout:HOME_URL="https://centos.org/" 2024-03-28T17:39:22.307 INFO:teuthology.orchestra.run.smithi022.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-03-28T17:39:22.307 INFO:teuthology.orchestra.run.smithi022.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-03-28T17:39:22.307 INFO:teuthology.orchestra.run.smithi022.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-03-28T17:39:22.307 INFO:teuthology.lock.ops:Updating smithi022.front.sepia.ceph.com on lock server 2024-03-28T17:39:22.333 DEBUG:teuthology.orchestra.run.smithi079:> uname -m 2024-03-28T17:39:22.350 INFO:teuthology.orchestra.run.smithi079.stdout:x86_64 2024-03-28T17:39:22.351 DEBUG:teuthology.orchestra.run.smithi079:> cat /etc/os-release 2024-03-28T17:39:22.407 INFO:teuthology.orchestra.run.smithi079.stdout:NAME="CentOS Stream" 2024-03-28T17:39:22.407 INFO:teuthology.orchestra.run.smithi079.stdout:VERSION="9" 2024-03-28T17:39:22.407 INFO:teuthology.orchestra.run.smithi079.stdout:ID="centos" 2024-03-28T17:39:22.407 INFO:teuthology.orchestra.run.smithi079.stdout:ID_LIKE="rhel fedora" 2024-03-28T17:39:22.407 INFO:teuthology.orchestra.run.smithi079.stdout:VERSION_ID="9" 2024-03-28T17:39:22.407 INFO:teuthology.orchestra.run.smithi079.stdout:PLATFORM_ID="platform:el9" 2024-03-28T17:39:22.407 INFO:teuthology.orchestra.run.smithi079.stdout:PRETTY_NAME="CentOS Stream 9" 2024-03-28T17:39:22.407 INFO:teuthology.orchestra.run.smithi079.stdout:ANSI_COLOR="0;31" 2024-03-28T17:39:22.407 INFO:teuthology.orchestra.run.smithi079.stdout:LOGO="fedora-logo-icon" 2024-03-28T17:39:22.407 INFO:teuthology.orchestra.run.smithi079.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-03-28T17:39:22.408 INFO:teuthology.orchestra.run.smithi079.stdout:HOME_URL="https://centos.org/" 2024-03-28T17:39:22.408 INFO:teuthology.orchestra.run.smithi079.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-03-28T17:39:22.408 INFO:teuthology.orchestra.run.smithi079.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-03-28T17:39:22.408 INFO:teuthology.orchestra.run.smithi079.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-03-28T17:39:22.408 INFO:teuthology.lock.ops:Updating smithi079.front.sepia.ceph.com on lock server 2024-03-28T17:39:22.424 DEBUG:teuthology.orchestra.run.smithi134:> uname -m 2024-03-28T17:39:22.442 INFO:teuthology.orchestra.run.smithi134.stdout:x86_64 2024-03-28T17:39:22.442 DEBUG:teuthology.orchestra.run.smithi134:> cat /etc/os-release 2024-03-28T17:39:22.498 INFO:teuthology.orchestra.run.smithi134.stdout:NAME="CentOS Stream" 2024-03-28T17:39:22.499 INFO:teuthology.orchestra.run.smithi134.stdout:VERSION="9" 2024-03-28T17:39:22.499 INFO:teuthology.orchestra.run.smithi134.stdout:ID="centos" 2024-03-28T17:39:22.499 INFO:teuthology.orchestra.run.smithi134.stdout:ID_LIKE="rhel fedora" 2024-03-28T17:39:22.499 INFO:teuthology.orchestra.run.smithi134.stdout:VERSION_ID="9" 2024-03-28T17:39:22.499 INFO:teuthology.orchestra.run.smithi134.stdout:PLATFORM_ID="platform:el9" 2024-03-28T17:39:22.499 INFO:teuthology.orchestra.run.smithi134.stdout:PRETTY_NAME="CentOS Stream 9" 2024-03-28T17:39:22.499 INFO:teuthology.orchestra.run.smithi134.stdout:ANSI_COLOR="0;31" 2024-03-28T17:39:22.499 INFO:teuthology.orchestra.run.smithi134.stdout:LOGO="fedora-logo-icon" 2024-03-28T17:39:22.499 INFO:teuthology.orchestra.run.smithi134.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-03-28T17:39:22.499 INFO:teuthology.orchestra.run.smithi134.stdout:HOME_URL="https://centos.org/" 2024-03-28T17:39:22.499 INFO:teuthology.orchestra.run.smithi134.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-03-28T17:39:22.499 INFO:teuthology.orchestra.run.smithi134.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-03-28T17:39:22.500 INFO:teuthology.orchestra.run.smithi134.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-03-28T17:39:22.500 INFO:teuthology.lock.ops:Updating smithi134.front.sepia.ceph.com on lock server 2024-03-28T17:39:22.525 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-03-28T17:39:22.535 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-03-28T17:39:22.541 INFO:teuthology.task.internal:Checking for old test directory... 2024-03-28T17:39:22.541 DEBUG:teuthology.orchestra.run.smithi022:> test '!' -e /home/ubuntu/cephtest 2024-03-28T17:39:22.543 DEBUG:teuthology.orchestra.run.smithi079:> test '!' -e /home/ubuntu/cephtest 2024-03-28T17:39:22.546 DEBUG:teuthology.orchestra.run.smithi134:> test '!' -e /home/ubuntu/cephtest 2024-03-28T17:39:22.561 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-03-28T17:39:22.567 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-03-28T17:39:22.567 DEBUG:teuthology.orchestra.run.smithi022:> test -z $(ls -A /var/lib/ceph) 2024-03-28T17:39:22.601 DEBUG:teuthology.orchestra.run.smithi079:> test -z $(ls -A /var/lib/ceph) 2024-03-28T17:39:22.603 DEBUG:teuthology.orchestra.run.smithi134:> test -z $(ls -A /var/lib/ceph) 2024-03-28T17:39:22.875 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-03-28T17:39:22.941 INFO:teuthology.run_tasks:Running task kernel... 2024-03-28T17:39:22.954 INFO:teuthology.task.kernel:normalize config orig: {'flavor': 'default', 'kdb': True, 'sha1': '1df0c7e8a2354c42d738e5454ba83ea7332deac2'} 2024-03-28T17:39:22.954 INFO:teuthology.task.kernel:config {'mon.a': {'flavor': 'default', 'kdb': True, 'sha1': '1df0c7e8a2354c42d738e5454ba83ea7332deac2'}, 'mon.b': {'flavor': 'default', 'kdb': True, 'sha1': '1df0c7e8a2354c42d738e5454ba83ea7332deac2'}, 'mon.c': {'flavor': 'default', 'kdb': True, 'sha1': '1df0c7e8a2354c42d738e5454ba83ea7332deac2'}}, timeout 300 2024-03-28T17:39:22.955 DEBUG:teuthology.orchestra.run.smithi022:> test -f /run/.containerenv -o -f /.dockerenv 2024-03-28T17:39:22.955 DEBUG:teuthology.orchestra.run.smithi079:> test -f /run/.containerenv -o -f /.dockerenv 2024-03-28T17:39:22.955 DEBUG:teuthology.orchestra.run.smithi134:> test -f /run/.containerenv -o -f /.dockerenv 2024-03-28T17:39:22.971 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-28T17:39:22.972 DEBUG:teuthology.task.kernel:sha1 for mon.b is 1df0c7e8a2354c42d738e5454ba83ea7332deac2 2024-03-28T17:39:22.972 INFO:teuthology.task.kernel:Checking kernel version of mon.b, want "1df0c7e8a2354c42d738e5454ba83ea7332deac2"... 2024-03-28T17:39:22.972 DEBUG:teuthology.orchestra.run.smithi079:> uname -r 2024-03-28T17:39:22.973 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-28T17:39:22.973 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-28T17:39:22.974 DEBUG:teuthology.task.kernel:sha1 for mon.a is 1df0c7e8a2354c42d738e5454ba83ea7332deac2 2024-03-28T17:39:22.974 INFO:teuthology.task.kernel:Checking kernel version of mon.a, want "1df0c7e8a2354c42d738e5454ba83ea7332deac2"... 2024-03-28T17:39:22.974 DEBUG:teuthology.orchestra.run.smithi022:> uname -r 2024-03-28T17:39:22.974 DEBUG:teuthology.task.kernel:sha1 for mon.c is 1df0c7e8a2354c42d738e5454ba83ea7332deac2 2024-03-28T17:39:22.974 INFO:teuthology.task.kernel:Checking kernel version of mon.c, want "1df0c7e8a2354c42d738e5454ba83ea7332deac2"... 2024-03-28T17:39:22.974 DEBUG:teuthology.orchestra.run.smithi134:> uname -r 2024-03-28T17:39:23.029 INFO:teuthology.orchestra.run.smithi079.stdout:5.14.0-361.el9.x86_64 2024-03-28T17:39:23.029 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-361.el9.x86_64 vs 1df0c7e8a2354c42d738e5454ba83ea7332deac2 2024-03-28T17:39:23.030 DEBUG:teuthology.task.kernel:failed to parse current kernel version 2024-03-28T17:39:23.030 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F9%2Fx86_64&sha1=1df0c7e8a2354c42d738e5454ba83ea7332deac2 2024-03-28T17:39:23.034 INFO:teuthology.orchestra.run.smithi022.stdout:5.14.0-361.el9.x86_64 2024-03-28T17:39:23.034 INFO:teuthology.orchestra.run.smithi134.stdout:5.14.0-361.el9.x86_64 2024-03-28T17:39:23.034 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-361.el9.x86_64 vs 1df0c7e8a2354c42d738e5454ba83ea7332deac2 2024-03-28T17:39:23.035 DEBUG:teuthology.task.kernel:failed to parse current kernel version 2024-03-28T17:39:23.035 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F9%2Fx86_64&sha1=1df0c7e8a2354c42d738e5454ba83ea7332deac2 2024-03-28T17:39:23.035 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-361.el9.x86_64 vs 1df0c7e8a2354c42d738e5454ba83ea7332deac2 2024-03-28T17:39:23.036 DEBUG:teuthology.task.kernel:failed to parse current kernel version 2024-03-28T17:39:23.036 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F9%2Fx86_64&sha1=1df0c7e8a2354c42d738e5454ba83ea7332deac2 2024-03-28T17:39:23.170 DEBUG:teuthology.orchestra.run.smithi079:> sudo yum upgrade -y linux-firmware 2024-03-28T17:39:23.175 DEBUG:teuthology.orchestra.run.smithi134:> sudo yum upgrade -y linux-firmware 2024-03-28T17:39:23.249 DEBUG:teuthology.orchestra.run.smithi022:> sudo yum upgrade -y linux-firmware 2024-03-28T17:39:25.330 INFO:teuthology.orchestra.run.smithi079.stdout:CentOS-9 - AppStream 13 MB/s | 6.7 MB 00:00 2024-03-28T17:39:25.420 INFO:teuthology.orchestra.run.smithi134.stdout:CentOS-9 - AppStream 10 MB/s | 6.7 MB 00:00 2024-03-28T17:39:25.452 INFO:teuthology.orchestra.run.smithi022.stdout:CentOS-9 - AppStream 12 MB/s | 6.7 MB 00:00 2024-03-28T17:39:26.919 INFO:teuthology.orchestra.run.smithi022.stdout:CentOS-9 - BaseOS 6.5 MB/s | 2.0 MB 00:00 2024-03-28T17:39:26.989 INFO:teuthology.orchestra.run.smithi079.stdout:CentOS-9 - BaseOS 3.3 MB/s | 2.0 MB 00:00 2024-03-28T17:39:27.029 INFO:teuthology.orchestra.run.smithi134.stdout:CentOS-9 - BaseOS 4.0 MB/s | 2.0 MB 00:00 2024-03-28T17:39:27.895 INFO:teuthology.orchestra.run.smithi134.stdout:CentOS Stream 9 - BaseOS 16 MB/s | 8.0 MB 00:00 2024-03-28T17:39:28.396 INFO:teuthology.orchestra.run.smithi079.stdout:CentOS Stream 9 - BaseOS 7.0 MB/s | 8.0 MB 00:01 2024-03-28T17:39:29.762 INFO:teuthology.orchestra.run.smithi134.stdout:CentOS Stream 9 - AppStream 23 MB/s | 19 MB 00:00 2024-03-28T17:39:30.290 INFO:teuthology.orchestra.run.smithi079.stdout:CentOS Stream 9 - AppStream 21 MB/s | 19 MB 00:00 2024-03-28T17:39:33.704 INFO:teuthology.orchestra.run.smithi134.stdout:CentOS Stream 9 - CRB 14 MB/s | 6.1 MB 00:00 2024-03-28T17:39:34.105 INFO:teuthology.orchestra.run.smithi079.stdout:CentOS Stream 9 - CRB 16 MB/s | 6.1 MB 00:00 2024-03-28T17:39:35.139 INFO:teuthology.orchestra.run.smithi134.stdout:CentOS Stream 9 - Extras packages 53 kB/s | 16 kB 00:00 2024-03-28T17:39:35.514 INFO:teuthology.orchestra.run.smithi079.stdout:CentOS Stream 9 - Extras packages 55 kB/s | 16 kB 00:00 2024-03-28T17:39:36.125 INFO:teuthology.orchestra.run.smithi022.stdout:CentOS Stream 9 - BaseOS 928 kB/s | 8.0 MB 00:08 2024-03-28T17:39:36.541 INFO:teuthology.orchestra.run.smithi079.stdout:Extra Packages for Enterprise Linux 23 MB/s | 21 MB 00:00 2024-03-28T17:39:41.482 INFO:teuthology.orchestra.run.smithi079.stdout:lab-extras 11 kB/s | 1.7 kB 00:00 2024-03-28T17:39:43.712 INFO:teuthology.orchestra.run.smithi079.stdout:Dependencies resolved. 2024-03-28T17:39:43.721 INFO:teuthology.orchestra.run.smithi079.stdout:================================================================================ 2024-03-28T17:39:43.721 INFO:teuthology.orchestra.run.smithi079.stdout: Package Arch Version Repository Size 2024-03-28T17:39:43.721 INFO:teuthology.orchestra.run.smithi079.stdout:================================================================================ 2024-03-28T17:39:43.722 INFO:teuthology.orchestra.run.smithi079.stdout:Upgrading: 2024-03-28T17:39:43.722 INFO:teuthology.orchestra.run.smithi079.stdout: linux-firmware noarch 20240219-143.el9 CentOS-BaseOS 375 M 2024-03-28T17:39:43.722 INFO:teuthology.orchestra.run.smithi079.stdout: 2024-03-28T17:39:43.722 INFO:teuthology.orchestra.run.smithi079.stdout:Transaction Summary 2024-03-28T17:39:43.722 INFO:teuthology.orchestra.run.smithi079.stdout:================================================================================ 2024-03-28T17:39:43.722 INFO:teuthology.orchestra.run.smithi079.stdout:Upgrade 1 Package 2024-03-28T17:39:43.722 INFO:teuthology.orchestra.run.smithi079.stdout: 2024-03-28T17:39:43.722 INFO:teuthology.orchestra.run.smithi079.stdout:Total download size: 375 M 2024-03-28T17:39:43.722 INFO:teuthology.orchestra.run.smithi079.stdout:Downloading Packages: 2024-03-28T17:39:52.398 INFO:teuthology.orchestra.run.smithi079.stdout:linux-firmware-20240219-143.el9.noarch.rpm 43 MB/s | 375 MB 00:08 2024-03-28T17:39:52.399 INFO:teuthology.orchestra.run.smithi079.stdout:-------------------------------------------------------------------------------- 2024-03-28T17:39:52.399 INFO:teuthology.orchestra.run.smithi079.stdout:Total 43 MB/s | 375 MB 00:08 2024-03-28T17:39:52.400 INFO:teuthology.orchestra.run.smithi079.stdout:Running transaction check 2024-03-28T17:39:52.491 INFO:teuthology.orchestra.run.smithi079.stdout:Transaction check succeeded. 2024-03-28T17:39:52.492 INFO:teuthology.orchestra.run.smithi079.stdout:Running transaction test 2024-03-28T17:39:54.965 INFO:teuthology.orchestra.run.smithi079.stdout:Transaction test succeeded. 2024-03-28T17:39:54.966 INFO:teuthology.orchestra.run.smithi079.stdout:Running transaction 2024-03-28T17:39:56.475 INFO:teuthology.orchestra.run.smithi134.stdout:Extra Packages for Enterprise Linux 1.0 MB/s | 21 MB 00:21 2024-03-28T17:39:58.765 INFO:teuthology.orchestra.run.smithi022.stdout:CentOS Stream 9 - AppStream 899 kB/s | 19 MB 00:21 2024-03-28T17:40:01.230 INFO:teuthology.orchestra.run.smithi079.stdout: Preparing : 1/1 2024-03-28T17:40:01.445 INFO:teuthology.orchestra.run.smithi134.stdout:lab-extras 13 kB/s | 1.7 kB 00:00 2024-03-28T17:40:01.491 INFO:teuthology.orchestra.run.smithi079.stdout: Upgrading : linux-firmware-20240219-143.el9.noarch 1/2 2024-03-28T17:40:01.580 INFO:teuthology.orchestra.run.smithi079.stdout: Cleanup : linux-firmware-20230814-139.el9.noarch 2/2 2024-03-28T17:40:03.092 INFO:teuthology.orchestra.run.smithi022.stdout:CentOS Stream 9 - CRB 8.7 MB/s | 6.1 MB 00:00 2024-03-28T17:40:03.178 INFO:teuthology.orchestra.run.smithi079.stdout: Running scriptlet: linux-firmware-20230814-139.el9.noarch 2/2 2024-03-28T17:40:03.178 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : linux-firmware-20240219-143.el9.noarch 1/2 2024-03-28T17:40:03.593 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : linux-firmware-20230814-139.el9.noarch 2/2 2024-03-28T17:40:03.593 INFO:teuthology.orchestra.run.smithi079.stdout: 2024-03-28T17:40:03.593 INFO:teuthology.orchestra.run.smithi079.stdout:Upgraded: 2024-03-28T17:40:03.593 INFO:teuthology.orchestra.run.smithi079.stdout: linux-firmware-20240219-143.el9.noarch 2024-03-28T17:40:03.593 INFO:teuthology.orchestra.run.smithi079.stdout: 2024-03-28T17:40:03.593 INFO:teuthology.orchestra.run.smithi079.stdout:Complete! 2024-03-28T17:40:03.705 INFO:teuthology.orchestra.run.smithi134.stdout:Dependencies resolved. 2024-03-28T17:40:03.714 INFO:teuthology.orchestra.run.smithi134.stdout:================================================================================ 2024-03-28T17:40:03.714 INFO:teuthology.orchestra.run.smithi134.stdout: Package Arch Version Repository Size 2024-03-28T17:40:03.714 INFO:teuthology.orchestra.run.smithi134.stdout:================================================================================ 2024-03-28T17:40:03.714 INFO:teuthology.orchestra.run.smithi134.stdout:Upgrading: 2024-03-28T17:40:03.714 INFO:teuthology.orchestra.run.smithi134.stdout: linux-firmware noarch 20240219-143.el9 CentOS-BaseOS 375 M 2024-03-28T17:40:03.714 INFO:teuthology.orchestra.run.smithi134.stdout: 2024-03-28T17:40:03.714 INFO:teuthology.orchestra.run.smithi134.stdout:Transaction Summary 2024-03-28T17:40:03.715 INFO:teuthology.orchestra.run.smithi134.stdout:================================================================================ 2024-03-28T17:40:03.715 INFO:teuthology.orchestra.run.smithi134.stdout:Upgrade 1 Package 2024-03-28T17:40:03.715 INFO:teuthology.orchestra.run.smithi134.stdout: 2024-03-28T17:40:03.715 INFO:teuthology.orchestra.run.smithi134.stdout:Total download size: 375 M 2024-03-28T17:40:03.715 INFO:teuthology.orchestra.run.smithi134.stdout:Downloading Packages: 2024-03-28T17:40:03.908 INFO:teuthology.task.kernel:Downloading kernel 1df0c7e8a2354c42d738e5454ba83ea7332deac2 on mon.b... 2024-03-28T17:40:03.908 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F9%2Fx86_64&sha1=1df0c7e8a2354c42d738e5454ba83ea7332deac2 2024-03-28T17:40:04.052 INFO:teuthology.task.kernel:fetching, builder baseurl is https://3.chacra.ceph.com/r/kernel/testing/1df0c7e8a2354c42d738e5454ba83ea7332deac2/centos/9/flavors/default/x86_64/ 2024-03-28T17:40:04.052 DEBUG:teuthology.orchestra.run.smithi079:> rm -f /tmp/kernel.x86_64.rpm && echo kernel-6.8.0_g1df0c7e8a235-1.x86_64.rpm | wget -nv -O /tmp/kernel.x86_64.rpm --base=https://3.chacra.ceph.com/r/kernel/testing/1df0c7e8a2354c42d738e5454ba83ea7332deac2/centos/9/flavors/default/x86_64/ --input-file=- 2024-03-28T17:40:04.055 DEBUG:teuthology.task.kernel:Waiting for download/copy to ubuntu@smithi079.front.sepia.ceph.com to complete... 2024-03-28T17:40:04.508 INFO:teuthology.orchestra.run.smithi022.stdout:CentOS Stream 9 - Extras packages 76 kB/s | 16 kB 00:00 2024-03-28T17:40:06.667 INFO:teuthology.orchestra.run.smithi022.stdout:Extra Packages for Enterprise Linux 10 MB/s | 21 MB 00:02 2024-03-28T17:40:11.396 INFO:teuthology.orchestra.run.smithi134.stdout:linux-firmware-20240219-143.el9.noarch.rpm 49 MB/s | 375 MB 00:07 2024-03-28T17:40:11.397 INFO:teuthology.orchestra.run.smithi134.stdout:-------------------------------------------------------------------------------- 2024-03-28T17:40:11.398 INFO:teuthology.orchestra.run.smithi134.stdout:Total 49 MB/s | 375 MB 00:07 2024-03-28T17:40:11.398 INFO:teuthology.orchestra.run.smithi134.stdout:Running transaction check 2024-03-28T17:40:11.496 INFO:teuthology.orchestra.run.smithi134.stdout:Transaction check succeeded. 2024-03-28T17:40:11.496 INFO:teuthology.orchestra.run.smithi134.stdout:Running transaction test 2024-03-28T17:40:11.776 INFO:teuthology.orchestra.run.smithi022.stdout:lab-extras 35 kB/s | 1.7 kB 00:00 2024-03-28T17:40:13.881 INFO:teuthology.orchestra.run.smithi134.stdout:Transaction test succeeded. 2024-03-28T17:40:13.881 INFO:teuthology.orchestra.run.smithi134.stdout:Running transaction 2024-03-28T17:40:14.060 INFO:teuthology.orchestra.run.smithi022.stdout:Dependencies resolved. 2024-03-28T17:40:14.071 INFO:teuthology.orchestra.run.smithi022.stdout:================================================================================ 2024-03-28T17:40:14.071 INFO:teuthology.orchestra.run.smithi022.stdout: Package Arch Version Repository Size 2024-03-28T17:40:14.072 INFO:teuthology.orchestra.run.smithi022.stdout:================================================================================ 2024-03-28T17:40:14.072 INFO:teuthology.orchestra.run.smithi022.stdout:Upgrading: 2024-03-28T17:40:14.072 INFO:teuthology.orchestra.run.smithi022.stdout: linux-firmware noarch 20240219-143.el9 CentOS-BaseOS 375 M 2024-03-28T17:40:14.072 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-03-28T17:40:14.072 INFO:teuthology.orchestra.run.smithi022.stdout:Transaction Summary 2024-03-28T17:40:14.072 INFO:teuthology.orchestra.run.smithi022.stdout:================================================================================ 2024-03-28T17:40:14.072 INFO:teuthology.orchestra.run.smithi022.stdout:Upgrade 1 Package 2024-03-28T17:40:14.072 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-03-28T17:40:14.072 INFO:teuthology.orchestra.run.smithi022.stdout:Total download size: 375 M 2024-03-28T17:40:14.072 INFO:teuthology.orchestra.run.smithi022.stdout:Downloading Packages: 2024-03-28T17:40:16.140 INFO:teuthology.orchestra.run.smithi079.stderr:2024-03-28 17:40:16 URL:https://3.chacra.ceph.com/r/kernel/testing/1df0c7e8a2354c42d738e5454ba83ea7332deac2/centos/9/flavors/default/x86_64/kernel-6.8.0_g1df0c7e8a235-1.x86_64.rpm [336317861/336317861] -> "/tmp/kernel.x86_64.rpm" [1] 2024-03-28T17:40:16.140 INFO:teuthology.orchestra.run.smithi079.stderr:FINISHED --2024-03-28 17:40:16-- 2024-03-28T17:40:16.140 INFO:teuthology.orchestra.run.smithi079.stderr:Total wall clock time: 12s 2024-03-28T17:40:16.140 INFO:teuthology.orchestra.run.smithi079.stderr:Downloaded: 1 files, 321M in 12s (27.6 MB/s) 2024-03-28T17:40:16.142 INFO:teuthology.task.kernel:Installing kernel 1df0c7e8a2354c42d738e5454ba83ea7332deac2 on mon.b... 2024-03-28T17:40:16.143 DEBUG:teuthology.orchestra.run.smithi079:> sudo rpm -ivh --oldpackage --replacefiles --replacepkgs /tmp/kernel.x86_64.rpm 2024-03-28T17:40:18.073 INFO:teuthology.orchestra.run.smithi079.stdout:Verifying... ######################################## 2024-03-28T17:40:18.187 INFO:teuthology.orchestra.run.smithi079.stdout:Preparing... ######################################## 2024-03-28T17:40:18.267 INFO:teuthology.orchestra.run.smithi079.stdout:Updating / installing... 2024-03-28T17:40:20.227 INFO:teuthology.orchestra.run.smithi134.stdout: Preparing : 1/1 2024-03-28T17:40:20.432 INFO:teuthology.orchestra.run.smithi134.stdout: Upgrading : linux-firmware-20240219-143.el9.noarch 1/2 2024-03-28T17:40:20.512 INFO:teuthology.orchestra.run.smithi134.stdout: Cleanup : linux-firmware-20230814-139.el9.noarch 2/2 2024-03-28T17:40:21.706 INFO:teuthology.orchestra.run.smithi134.stdout: Running scriptlet: linux-firmware-20230814-139.el9.noarch 2/2 2024-03-28T17:40:21.707 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : linux-firmware-20240219-143.el9.noarch 1/2 2024-03-28T17:40:22.096 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : linux-firmware-20230814-139.el9.noarch 2/2 2024-03-28T17:40:22.096 INFO:teuthology.orchestra.run.smithi134.stdout: 2024-03-28T17:40:22.096 INFO:teuthology.orchestra.run.smithi134.stdout:Upgraded: 2024-03-28T17:40:22.096 INFO:teuthology.orchestra.run.smithi134.stdout: linux-firmware-20240219-143.el9.noarch 2024-03-28T17:40:22.096 INFO:teuthology.orchestra.run.smithi134.stdout: 2024-03-28T17:40:22.096 INFO:teuthology.orchestra.run.smithi134.stdout:Complete! 2024-03-28T17:40:22.398 INFO:teuthology.task.kernel:Downloading kernel 1df0c7e8a2354c42d738e5454ba83ea7332deac2 on mon.c... 2024-03-28T17:40:22.398 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F9%2Fx86_64&sha1=1df0c7e8a2354c42d738e5454ba83ea7332deac2 2024-03-28T17:40:22.735 INFO:teuthology.task.kernel:fetching, builder baseurl is https://3.chacra.ceph.com/r/kernel/testing/1df0c7e8a2354c42d738e5454ba83ea7332deac2/centos/9/flavors/default/x86_64/ 2024-03-28T17:40:22.735 DEBUG:teuthology.orchestra.run.smithi134:> rm -f /tmp/kernel.x86_64.rpm && echo kernel-6.8.0_g1df0c7e8a235-1.x86_64.rpm | wget -nv -O /tmp/kernel.x86_64.rpm --base=https://3.chacra.ceph.com/r/kernel/testing/1df0c7e8a2354c42d738e5454ba83ea7332deac2/centos/9/flavors/default/x86_64/ --input-file=- 2024-03-28T17:40:22.737 DEBUG:teuthology.task.kernel:Waiting for download/copy to ubuntu@smithi134.front.sepia.ceph.com to complete... 2024-03-28T17:40:23.534 INFO:teuthology.orchestra.run.smithi022.stdout:linux-firmware-20240219-143.el9.noarch.rpm 40 MB/s | 375 MB 00:09 2024-03-28T17:40:23.535 INFO:teuthology.orchestra.run.smithi022.stdout:-------------------------------------------------------------------------------- 2024-03-28T17:40:23.536 INFO:teuthology.orchestra.run.smithi022.stdout:Total 40 MB/s | 375 MB 00:09 2024-03-28T17:40:23.536 INFO:teuthology.orchestra.run.smithi022.stdout:Running transaction check 2024-03-28T17:40:23.634 INFO:teuthology.orchestra.run.smithi022.stdout:Transaction check succeeded. 2024-03-28T17:40:23.634 INFO:teuthology.orchestra.run.smithi022.stdout:Running transaction test 2024-03-28T17:40:25.860 INFO:teuthology.orchestra.run.smithi079.stdout:kernel-6.8.0_g1df0c7e8a235-1 ######################################## 2024-03-28T17:40:26.110 INFO:teuthology.orchestra.run.smithi022.stdout:Transaction test succeeded. 2024-03-28T17:40:26.110 INFO:teuthology.orchestra.run.smithi022.stdout:Running transaction 2024-03-28T17:40:32.612 INFO:teuthology.orchestra.run.smithi022.stdout: Preparing : 1/1 2024-03-28T17:40:32.876 INFO:teuthology.orchestra.run.smithi022.stdout: Upgrading : linux-firmware-20240219-143.el9.noarch 1/2 2024-03-28T17:40:32.970 INFO:teuthology.orchestra.run.smithi022.stdout: Cleanup : linux-firmware-20230814-139.el9.noarch 2/2 2024-03-28T17:40:34.132 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-28 17:40:34 URL:https://3.chacra.ceph.com/r/kernel/testing/1df0c7e8a2354c42d738e5454ba83ea7332deac2/centos/9/flavors/default/x86_64/kernel-6.8.0_g1df0c7e8a235-1.x86_64.rpm [336317861/336317861] -> "/tmp/kernel.x86_64.rpm" [1] 2024-03-28T17:40:34.132 INFO:teuthology.orchestra.run.smithi134.stderr:FINISHED --2024-03-28 17:40:34-- 2024-03-28T17:40:34.132 INFO:teuthology.orchestra.run.smithi134.stderr:Total wall clock time: 11s 2024-03-28T17:40:34.132 INFO:teuthology.orchestra.run.smithi134.stderr:Downloaded: 1 files, 321M in 11s (28.9 MB/s) 2024-03-28T17:40:34.135 INFO:teuthology.task.kernel:Installing kernel 1df0c7e8a2354c42d738e5454ba83ea7332deac2 on mon.c... 2024-03-28T17:40:34.135 DEBUG:teuthology.orchestra.run.smithi134:> sudo rpm -ivh --oldpackage --replacefiles --replacepkgs /tmp/kernel.x86_64.rpm 2024-03-28T17:40:34.334 INFO:teuthology.orchestra.run.smithi022.stdout: Running scriptlet: linux-firmware-20230814-139.el9.noarch 2/2 2024-03-28T17:40:34.334 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : linux-firmware-20240219-143.el9.noarch 1/2 2024-03-28T17:40:34.730 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : linux-firmware-20230814-139.el9.noarch 2/2 2024-03-28T17:40:34.730 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-03-28T17:40:34.730 INFO:teuthology.orchestra.run.smithi022.stdout:Upgraded: 2024-03-28T17:40:34.730 INFO:teuthology.orchestra.run.smithi022.stdout: linux-firmware-20240219-143.el9.noarch 2024-03-28T17:40:34.730 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-03-28T17:40:34.731 INFO:teuthology.orchestra.run.smithi022.stdout:Complete! 2024-03-28T17:40:35.020 INFO:teuthology.task.kernel:Downloading kernel 1df0c7e8a2354c42d738e5454ba83ea7332deac2 on mon.a... 2024-03-28T17:40:35.021 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F9%2Fx86_64&sha1=1df0c7e8a2354c42d738e5454ba83ea7332deac2 2024-03-28T17:40:35.238 INFO:teuthology.task.kernel:fetching, builder baseurl is https://3.chacra.ceph.com/r/kernel/testing/1df0c7e8a2354c42d738e5454ba83ea7332deac2/centos/9/flavors/default/x86_64/ 2024-03-28T17:40:35.238 DEBUG:teuthology.orchestra.run.smithi022:> rm -f /tmp/kernel.x86_64.rpm && echo kernel-6.8.0_g1df0c7e8a235-1.x86_64.rpm | wget -nv -O /tmp/kernel.x86_64.rpm --base=https://3.chacra.ceph.com/r/kernel/testing/1df0c7e8a2354c42d738e5454ba83ea7332deac2/centos/9/flavors/default/x86_64/ --input-file=- 2024-03-28T17:40:35.240 DEBUG:teuthology.task.kernel:Waiting for download/copy to ubuntu@smithi022.front.sepia.ceph.com to complete... 2024-03-28T17:40:36.074 INFO:teuthology.orchestra.run.smithi134.stdout:Verifying... ######################################## 2024-03-28T17:40:36.206 INFO:teuthology.orchestra.run.smithi134.stdout:Preparing... ######################################## 2024-03-28T17:40:36.286 INFO:teuthology.orchestra.run.smithi134.stdout:Updating / installing... 2024-03-28T17:40:42.424 INFO:teuthology.orchestra.run.smithi079.stderr:dracut: Disabling early microcode, because kernel does not support it. CONFIG_MICROCODE_[AMD|INTEL]!=y 2024-03-28T17:40:43.794 INFO:teuthology.orchestra.run.smithi134.stdout:kernel-6.8.0_g1df0c7e8a235-1 ######################################## 2024-03-28T17:40:46.863 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28 17:40:46 URL:https://3.chacra.ceph.com/r/kernel/testing/1df0c7e8a2354c42d738e5454ba83ea7332deac2/centos/9/flavors/default/x86_64/kernel-6.8.0_g1df0c7e8a235-1.x86_64.rpm [336317861/336317861] -> "/tmp/kernel.x86_64.rpm" [1] 2024-03-28T17:40:46.863 INFO:teuthology.orchestra.run.smithi022.stderr:FINISHED --2024-03-28 17:40:46-- 2024-03-28T17:40:46.863 INFO:teuthology.orchestra.run.smithi022.stderr:Total wall clock time: 12s 2024-03-28T17:40:46.863 INFO:teuthology.orchestra.run.smithi022.stderr:Downloaded: 1 files, 321M in 11s (28.6 MB/s) 2024-03-28T17:40:46.866 INFO:teuthology.task.kernel:Installing kernel 1df0c7e8a2354c42d738e5454ba83ea7332deac2 on mon.a... 2024-03-28T17:40:46.866 DEBUG:teuthology.orchestra.run.smithi022:> sudo rpm -ivh --oldpackage --replacefiles --replacepkgs /tmp/kernel.x86_64.rpm 2024-03-28T17:40:48.892 INFO:teuthology.orchestra.run.smithi022.stdout:Verifying... ######################################## 2024-03-28T17:40:48.952 INFO:teuthology.orchestra.run.smithi022.stdout:Preparing... ######################################## 2024-03-28T17:40:49.035 INFO:teuthology.orchestra.run.smithi022.stdout:Updating / installing... 2024-03-28T17:40:56.853 INFO:teuthology.orchestra.run.smithi022.stdout:kernel-6.8.0_g1df0c7e8a235-1 ######################################## 2024-03-28T17:40:59.549 INFO:teuthology.orchestra.run.smithi134.stderr:dracut: Disabling early microcode, because kernel does not support it. CONFIG_MICROCODE_[AMD|INTEL]!=y 2024-03-28T17:41:04.987 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi079.front.sepia.ceph.com, path=/tmp/kernel.x86_64.rpm, version=None) 2024-03-28T17:41:04.987 DEBUG:teuthology.orchestra.run.smithi079:> rpm -qlp /tmp/kernel.x86_64.rpm 2024-03-28T17:41:05.063 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235 2024-03-28T17:41:05.063 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/System.map 2024-03-28T17:41:05.064 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/config 2024-03-28T17:41:05.064 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel 2024-03-28T17:41:05.064 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch 2024-03-28T17:41:05.064 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86 2024-03-28T17:41:05.064 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/crypto 2024-03-28T17:41:05.064 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/crypto/blowfish-x86_64.ko 2024-03-28T17:41:05.064 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/crypto/camellia-aesni-avx-x86_64.ko 2024-03-28T17:41:05.064 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/crypto/camellia-aesni-avx2.ko 2024-03-28T17:41:05.064 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/crypto/camellia-x86_64.ko 2024-03-28T17:41:05.064 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/crypto/cast5-avx-x86_64.ko 2024-03-28T17:41:05.064 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/crypto/cast6-avx-x86_64.ko 2024-03-28T17:41:05.064 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/crypto/crc32-pclmul.ko 2024-03-28T17:41:05.064 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/crypto/crc32c-intel.ko 2024-03-28T17:41:05.064 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/crypto/crct10dif-pclmul.ko 2024-03-28T17:41:05.064 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/crypto/des3_ede-x86_64.ko 2024-03-28T17:41:05.065 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/crypto/ghash-clmulni-intel.ko 2024-03-28T17:41:05.065 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/crypto/serpent-avx-x86_64.ko 2024-03-28T17:41:05.065 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/crypto/serpent-avx2.ko 2024-03-28T17:41:05.065 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/crypto/serpent-sse2-x86_64.ko 2024-03-28T17:41:05.065 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/crypto/sha1-ssse3.ko 2024-03-28T17:41:05.065 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/crypto/sha256-ssse3.ko 2024-03-28T17:41:05.065 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/crypto/sha512-ssse3.ko 2024-03-28T17:41:05.065 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/crypto/twofish-avx-x86_64.ko 2024-03-28T17:41:05.065 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/crypto/twofish-x86_64-3way.ko 2024-03-28T17:41:05.065 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/crypto/twofish-x86_64.ko 2024-03-28T17:41:05.065 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/kernel 2024-03-28T17:41:05.065 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/kernel/cpu 2024-03-28T17:41:05.065 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/kernel/cpu/mce 2024-03-28T17:41:05.065 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/kernel/cpu/mce/mce-inject.ko 2024-03-28T17:41:05.065 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/kvm 2024-03-28T17:41:05.065 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/kvm/kvm-amd.ko 2024-03-28T17:41:05.066 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/kvm/kvm-intel.ko 2024-03-28T17:41:05.066 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/kvm/kvm.ko 2024-03-28T17:41:05.066 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto 2024-03-28T17:41:05.066 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/ansi_cprng.ko 2024-03-28T17:41:05.066 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/anubis.ko 2024-03-28T17:41:05.066 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/arc4.ko 2024-03-28T17:41:05.066 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/async_tx 2024-03-28T17:41:05.066 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/async_tx/async_memcpy.ko 2024-03-28T17:41:05.066 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/async_tx/async_pq.ko 2024-03-28T17:41:05.066 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/async_tx/async_raid6_recov.ko 2024-03-28T17:41:05.066 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/async_tx/async_tx.ko 2024-03-28T17:41:05.066 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/async_tx/async_xor.ko 2024-03-28T17:41:05.066 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/async_tx/raid6test.ko 2024-03-28T17:41:05.066 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/authenc.ko 2024-03-28T17:41:05.067 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/authencesn.ko 2024-03-28T17:41:05.067 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/blake2b_generic.ko 2024-03-28T17:41:05.067 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/blowfish_common.ko 2024-03-28T17:41:05.067 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/blowfish_generic.ko 2024-03-28T17:41:05.067 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/camellia_generic.ko 2024-03-28T17:41:05.067 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/cast5_generic.ko 2024-03-28T17:41:05.067 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/cast6_generic.ko 2024-03-28T17:41:05.067 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/cast_common.ko 2024-03-28T17:41:05.067 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/ccm.ko 2024-03-28T17:41:05.067 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/cmac.ko 2024-03-28T17:41:05.067 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/crc32_generic.ko 2024-03-28T17:41:05.067 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/crypto_engine.ko 2024-03-28T17:41:05.067 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/crypto_user.ko 2024-03-28T17:41:05.067 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/cts.ko 2024-03-28T17:41:05.067 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/des_generic.ko 2024-03-28T17:41:05.067 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/dh_generic.ko 2024-03-28T17:41:05.068 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/echainiv.ko 2024-03-28T17:41:05.068 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/essiv.ko 2024-03-28T17:41:05.068 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/fcrypt.ko 2024-03-28T17:41:05.068 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/khazad.ko 2024-03-28T17:41:05.068 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/lz4.ko 2024-03-28T17:41:05.068 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/lz4hc.ko 2024-03-28T17:41:05.068 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/md4.ko 2024-03-28T17:41:05.068 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/michael_mic.ko 2024-03-28T17:41:05.068 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/pcbc.ko 2024-03-28T17:41:05.068 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/pcrypt.ko 2024-03-28T17:41:05.068 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/rmd160.ko 2024-03-28T17:41:05.068 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/seed.ko 2024-03-28T17:41:05.068 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/serpent_generic.ko 2024-03-28T17:41:05.068 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/tcrypt.ko 2024-03-28T17:41:05.068 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/tea.ko 2024-03-28T17:41:05.068 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/twofish_common.ko 2024-03-28T17:41:05.069 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/twofish_generic.ko 2024-03-28T17:41:05.069 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/vmac.ko 2024-03-28T17:41:05.069 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/wp512.ko 2024-03-28T17:41:05.069 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/xcbc.ko 2024-03-28T17:41:05.069 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/xor.ko 2024-03-28T17:41:05.069 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/xxhash_generic.ko 2024-03-28T17:41:05.069 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers 2024-03-28T17:41:05.069 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/acpi 2024-03-28T17:41:05.069 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/acpi/acpi_ipmi.ko 2024-03-28T17:41:05.069 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/acpi/acpi_pad.ko 2024-03-28T17:41:05.069 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/acpi/custom_method.ko 2024-03-28T17:41:05.069 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/acpi/ec_sys.ko 2024-03-28T17:41:05.069 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/acpi/platform_profile.ko 2024-03-28T17:41:05.069 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/acpi/sbs.ko 2024-03-28T17:41:05.069 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/acpi/sbshc.ko 2024-03-28T17:41:05.070 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/acpi/video.ko 2024-03-28T17:41:05.070 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata 2024-03-28T17:41:05.070 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/acard-ahci.ko 2024-03-28T17:41:05.070 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/ahci_platform.ko 2024-03-28T17:41:05.070 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/ata_generic.ko 2024-03-28T17:41:05.070 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/libahci_platform.ko 2024-03-28T17:41:05.070 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_acpi.ko 2024-03-28T17:41:05.070 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_ali.ko 2024-03-28T17:41:05.070 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_amd.ko 2024-03-28T17:41:05.070 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_artop.ko 2024-03-28T17:41:05.070 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_atiixp.ko 2024-03-28T17:41:05.070 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_atp867x.ko 2024-03-28T17:41:05.070 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_cmd640.ko 2024-03-28T17:41:05.070 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_cmd64x.ko 2024-03-28T17:41:05.071 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_cypress.ko 2024-03-28T17:41:05.071 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_efar.ko 2024-03-28T17:41:05.071 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_hpt366.ko 2024-03-28T17:41:05.071 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_hpt37x.ko 2024-03-28T17:41:05.071 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_hpt3x2n.ko 2024-03-28T17:41:05.071 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_hpt3x3.ko 2024-03-28T17:41:05.071 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_it8213.ko 2024-03-28T17:41:05.071 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_it821x.ko 2024-03-28T17:41:05.071 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_jmicron.ko 2024-03-28T17:41:05.071 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_marvell.ko 2024-03-28T17:41:05.071 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_mpiix.ko 2024-03-28T17:41:05.071 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_netcell.ko 2024-03-28T17:41:05.071 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_ninja32.ko 2024-03-28T17:41:05.071 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_ns87410.ko 2024-03-28T17:41:05.071 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_ns87415.ko 2024-03-28T17:41:05.071 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_oldpiix.ko 2024-03-28T17:41:05.072 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_opti.ko 2024-03-28T17:41:05.072 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_optidma.ko 2024-03-28T17:41:05.072 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_pcmcia.ko 2024-03-28T17:41:05.072 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_pdc2027x.ko 2024-03-28T17:41:05.072 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_pdc202xx_old.ko 2024-03-28T17:41:05.072 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_piccolo.ko 2024-03-28T17:41:05.072 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_rdc.ko 2024-03-28T17:41:05.072 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_sch.ko 2024-03-28T17:41:05.072 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_serverworks.ko 2024-03-28T17:41:05.072 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_sil680.ko 2024-03-28T17:41:05.072 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_sis.ko 2024-03-28T17:41:05.072 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_sl82c105.ko 2024-03-28T17:41:05.072 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_triflex.ko 2024-03-28T17:41:05.072 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_via.ko 2024-03-28T17:41:05.072 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pdc_adma.ko 2024-03-28T17:41:05.073 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/sata_inic162x.ko 2024-03-28T17:41:05.073 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/sata_mv.ko 2024-03-28T17:41:05.073 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/sata_nv.ko 2024-03-28T17:41:05.073 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/sata_promise.ko 2024-03-28T17:41:05.073 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/sata_qstor.ko 2024-03-28T17:41:05.073 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/sata_sil.ko 2024-03-28T17:41:05.073 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/sata_sil24.ko 2024-03-28T17:41:05.073 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/sata_sis.ko 2024-03-28T17:41:05.073 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/sata_svw.ko 2024-03-28T17:41:05.073 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/sata_sx4.ko 2024-03-28T17:41:05.073 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/sata_uli.ko 2024-03-28T17:41:05.073 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/sata_via.ko 2024-03-28T17:41:05.073 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/sata_vsc.ko 2024-03-28T17:41:05.073 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/auxdisplay 2024-03-28T17:41:05.073 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/auxdisplay/ks0108.ko 2024-03-28T17:41:05.073 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/base 2024-03-28T17:41:05.074 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/base/regmap 2024-03-28T17:41:05.074 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/base/regmap/regmap-i2c.ko 2024-03-28T17:41:05.074 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/base/regmap/regmap-mmio.ko 2024-03-28T17:41:05.074 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/bcma 2024-03-28T17:41:05.074 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/bcma/bcma.ko 2024-03-28T17:41:05.074 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/block 2024-03-28T17:41:05.074 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/block/aoe 2024-03-28T17:41:05.074 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/block/aoe/aoe.ko 2024-03-28T17:41:05.074 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/block/brd.ko 2024-03-28T17:41:05.074 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/block/drbd 2024-03-28T17:41:05.074 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/block/drbd/drbd.ko 2024-03-28T17:41:05.074 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/block/floppy.ko 2024-03-28T17:41:05.074 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/block/loop.ko 2024-03-28T17:41:05.074 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/block/mtip32xx 2024-03-28T17:41:05.074 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/block/mtip32xx/mtip32xx.ko 2024-03-28T17:41:05.075 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/block/nbd.ko 2024-03-28T17:41:05.075 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/block/null_blk 2024-03-28T17:41:05.075 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/block/null_blk/null_blk.ko 2024-03-28T17:41:05.075 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/block/pktcdvd.ko 2024-03-28T17:41:05.075 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/block/rbd.ko 2024-03-28T17:41:05.075 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/block/virtio_blk.ko 2024-03-28T17:41:05.075 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/block/xen-blkback 2024-03-28T17:41:05.075 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/block/xen-blkback/xen-blkback.ko 2024-03-28T17:41:05.075 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/block/xen-blkfront.ko 2024-03-28T17:41:05.075 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/block/zram 2024-03-28T17:41:05.075 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/block/zram/zram.ko 2024-03-28T17:41:05.075 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char 2024-03-28T17:41:05.075 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/hangcheck-timer.ko 2024-03-28T17:41:05.075 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/hw_random 2024-03-28T17:41:05.075 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/hw_random/amd-rng.ko 2024-03-28T17:41:05.075 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/hw_random/intel-rng.ko 2024-03-28T17:41:05.076 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/hw_random/timeriomem-rng.ko 2024-03-28T17:41:05.076 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/hw_random/via-rng.ko 2024-03-28T17:41:05.076 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/hw_random/virtio-rng.ko 2024-03-28T17:41:05.076 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/ipmi 2024-03-28T17:41:05.076 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/ipmi/ipmi_devintf.ko 2024-03-28T17:41:05.076 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/ipmi/ipmi_msghandler.ko 2024-03-28T17:41:05.076 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/ipmi/ipmi_poweroff.ko 2024-03-28T17:41:05.076 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/ipmi/ipmi_si.ko 2024-03-28T17:41:05.076 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/ipmi/ipmi_ssif.ko 2024-03-28T17:41:05.076 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/ipmi/ipmi_watchdog.ko 2024-03-28T17:41:05.076 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/lp.ko 2024-03-28T17:41:05.076 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/mwave 2024-03-28T17:41:05.076 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/mwave/mwave.ko 2024-03-28T17:41:05.076 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/ppdev.ko 2024-03-28T17:41:05.076 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/tlclk.ko 2024-03-28T17:41:05.076 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/tpm 2024-03-28T17:41:05.076 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/tpm/tpm.ko 2024-03-28T17:41:05.077 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/tpm/tpm_atmel.ko 2024-03-28T17:41:05.077 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/tpm/tpm_infineon.ko 2024-03-28T17:41:05.077 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/tpm/tpm_nsc.ko 2024-03-28T17:41:05.077 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/tpm/tpm_tis.ko 2024-03-28T17:41:05.077 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/tpm/tpm_tis_core.ko 2024-03-28T17:41:05.077 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/uv_mmtimer.ko 2024-03-28T17:41:05.077 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/virtio_console.ko 2024-03-28T17:41:05.077 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/cpufreq 2024-03-28T17:41:05.077 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/cpufreq/acpi-cpufreq.ko 2024-03-28T17:41:05.077 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/cpufreq/amd_freq_sensitivity.ko 2024-03-28T17:41:05.077 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/cpufreq/p4-clockmod.ko 2024-03-28T17:41:05.077 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/cpufreq/pcc-cpufreq.ko 2024-03-28T17:41:05.077 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/cpufreq/powernow-k8.ko 2024-03-28T17:41:05.077 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/cpufreq/speedstep-lib.ko 2024-03-28T17:41:05.077 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/crypto 2024-03-28T17:41:05.078 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/crypto/ccp 2024-03-28T17:41:05.078 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/crypto/ccp/ccp-crypto.ko 2024-03-28T17:41:05.078 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/crypto/ccp/ccp.ko 2024-03-28T17:41:05.078 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/crypto/intel 2024-03-28T17:41:05.078 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/crypto/intel/qat 2024-03-28T17:41:05.078 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/crypto/intel/qat/qat_common 2024-03-28T17:41:05.078 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/crypto/intel/qat/qat_common/intel_qat.ko 2024-03-28T17:41:05.078 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/crypto/intel/qat/qat_dh895xcc 2024-03-28T17:41:05.078 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/crypto/intel/qat/qat_dh895xcc/qat_dh895xcc.ko 2024-03-28T17:41:05.078 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/crypto/padlock-aes.ko 2024-03-28T17:41:05.078 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/crypto/padlock-sha.ko 2024-03-28T17:41:05.078 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/crypto/virtio 2024-03-28T17:41:05.078 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/crypto/virtio/virtio_crypto.ko 2024-03-28T17:41:05.078 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/dca 2024-03-28T17:41:05.078 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/dca/dca.ko 2024-03-28T17:41:05.079 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/devfreq 2024-03-28T17:41:05.079 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/devfreq/governor_simpleondemand.ko 2024-03-28T17:41:05.079 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/dma 2024-03-28T17:41:05.079 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/dma/dw 2024-03-28T17:41:05.079 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/dma/dw/dw_dmac.ko 2024-03-28T17:41:05.079 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/dma/ioat 2024-03-28T17:41:05.079 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/dma/ioat/ioatdma.ko 2024-03-28T17:41:05.079 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/edac 2024-03-28T17:41:05.079 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/edac/amd64_edac.ko 2024-03-28T17:41:05.079 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/edac/e752x_edac.ko 2024-03-28T17:41:05.079 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/edac/edac_mce_amd.ko 2024-03-28T17:41:05.079 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/edac/i3000_edac.ko 2024-03-28T17:41:05.079 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/edac/i3200_edac.ko 2024-03-28T17:41:05.079 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/edac/i5100_edac.ko 2024-03-28T17:41:05.079 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/edac/i5400_edac.ko 2024-03-28T17:41:05.079 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/edac/i7300_edac.ko 2024-03-28T17:41:05.079 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/edac/i7core_edac.ko 2024-03-28T17:41:05.080 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/edac/i82975x_edac.ko 2024-03-28T17:41:05.080 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/edac/ie31200_edac.ko 2024-03-28T17:41:05.080 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/edac/sb_edac.ko 2024-03-28T17:41:05.080 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/edac/x38_edac.ko 2024-03-28T17:41:05.080 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/firmware 2024-03-28T17:41:05.080 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/firmware/edd.ko 2024-03-28T17:41:05.080 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/firmware/iscsi_ibft.ko 2024-03-28T17:41:05.080 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/gpio 2024-03-28T17:41:05.080 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/gpio/gpio-ich.ko 2024-03-28T17:41:05.080 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/gpio/gpio-viperboard.ko 2024-03-28T17:41:05.080 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid 2024-03-28T17:41:05.080 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-appleir.ko 2024-03-28T17:41:05.080 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-aureal.ko 2024-03-28T17:41:05.080 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-axff.ko 2024-03-28T17:41:05.080 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-dr.ko 2024-03-28T17:41:05.081 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-elecom.ko 2024-03-28T17:41:05.081 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-elo.ko 2024-03-28T17:41:05.081 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-emsff.ko 2024-03-28T17:41:05.081 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-gaff.ko 2024-03-28T17:41:05.081 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-gt683r.ko 2024-03-28T17:41:05.081 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-gyration.ko 2024-03-28T17:41:05.081 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-holtek-kbd.ko 2024-03-28T17:41:05.081 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-holtek-mouse.ko 2024-03-28T17:41:05.081 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-holtekff.ko 2024-03-28T17:41:05.081 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-hyperv.ko 2024-03-28T17:41:05.081 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-icade.ko 2024-03-28T17:41:05.081 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-keytouch.ko 2024-03-28T17:41:05.081 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-kye.ko 2024-03-28T17:41:05.081 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-lcpower.ko 2024-03-28T17:41:05.081 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-led.ko 2024-03-28T17:41:05.081 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-lenovo.ko 2024-03-28T17:41:05.082 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-logitech-dj.ko 2024-03-28T17:41:05.082 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-logitech-hidpp.ko 2024-03-28T17:41:05.082 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-multitouch.ko 2024-03-28T17:41:05.082 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-ortek.ko 2024-03-28T17:41:05.082 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-penmount.ko 2024-03-28T17:41:05.082 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-petalynx.ko 2024-03-28T17:41:05.082 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-picolcd.ko 2024-03-28T17:41:05.082 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-pl.ko 2024-03-28T17:41:05.082 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-plantronics.ko 2024-03-28T17:41:05.082 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-primax.ko 2024-03-28T17:41:05.082 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-rmi.ko 2024-03-28T17:41:05.082 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-roccat-arvo.ko 2024-03-28T17:41:05.082 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-roccat-common.ko 2024-03-28T17:41:05.082 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-roccat-isku.ko 2024-03-28T17:41:05.083 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-roccat-kone.ko 2024-03-28T17:41:05.083 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-roccat-koneplus.ko 2024-03-28T17:41:05.083 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-roccat-konepure.ko 2024-03-28T17:41:05.083 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-roccat-kovaplus.ko 2024-03-28T17:41:05.083 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-roccat-lua.ko 2024-03-28T17:41:05.083 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-roccat-pyra.ko 2024-03-28T17:41:05.083 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-roccat-ryos.ko 2024-03-28T17:41:05.083 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-roccat-savu.ko 2024-03-28T17:41:05.083 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-roccat.ko 2024-03-28T17:41:05.083 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-saitek.ko 2024-03-28T17:41:05.083 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-samsung.ko 2024-03-28T17:41:05.083 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-sensor-hub.ko 2024-03-28T17:41:05.083 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-sjoy.ko 2024-03-28T17:41:05.083 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-sony.ko 2024-03-28T17:41:05.083 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-speedlink.ko 2024-03-28T17:41:05.083 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-steelseries.ko 2024-03-28T17:41:05.083 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-sunplus.ko 2024-03-28T17:41:05.084 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-thrustmaster.ko 2024-03-28T17:41:05.084 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-tivo.ko 2024-03-28T17:41:05.084 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-tmff.ko 2024-03-28T17:41:05.084 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-topseed.ko 2024-03-28T17:41:05.084 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-twinhan.ko 2024-03-28T17:41:05.084 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-uclogic.ko 2024-03-28T17:41:05.084 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-waltop.ko 2024-03-28T17:41:05.084 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-wiimote.ko 2024-03-28T17:41:05.084 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-xinmo.ko 2024-03-28T17:41:05.084 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-zpff.ko 2024-03-28T17:41:05.084 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-zydacron.ko 2024-03-28T17:41:05.084 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/uhid.ko 2024-03-28T17:41:05.084 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/wacom.ko 2024-03-28T17:41:05.084 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hv 2024-03-28T17:41:05.084 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hv/hv_balloon.ko 2024-03-28T17:41:05.085 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hv/hv_utils.ko 2024-03-28T17:41:05.085 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hv/hv_vmbus.ko 2024-03-28T17:41:05.085 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon 2024-03-28T17:41:05.085 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/abituguru.ko 2024-03-28T17:41:05.085 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/abituguru3.ko 2024-03-28T17:41:05.085 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/acpi_power_meter.ko 2024-03-28T17:41:05.085 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/ad7414.ko 2024-03-28T17:41:05.085 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/ad7418.ko 2024-03-28T17:41:05.085 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/adc128d818.ko 2024-03-28T17:41:05.085 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/adm1025.ko 2024-03-28T17:41:05.085 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/adm1026.ko 2024-03-28T17:41:05.085 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/adm1029.ko 2024-03-28T17:41:05.085 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/adm1031.ko 2024-03-28T17:41:05.085 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/adm9240.ko 2024-03-28T17:41:05.085 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/ads7828.ko 2024-03-28T17:41:05.085 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/adt7410.ko 2024-03-28T17:41:05.085 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/adt7411.ko 2024-03-28T17:41:05.086 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/adt7462.ko 2024-03-28T17:41:05.086 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/adt7470.ko 2024-03-28T17:41:05.086 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/adt7475.ko 2024-03-28T17:41:05.086 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/adt7x10.ko 2024-03-28T17:41:05.086 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/amc6821.ko 2024-03-28T17:41:05.086 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/applesmc.ko 2024-03-28T17:41:05.086 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/asb100.ko 2024-03-28T17:41:05.086 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/asc7621.ko 2024-03-28T17:41:05.086 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/asus_atk0110.ko 2024-03-28T17:41:05.086 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/atxp1.ko 2024-03-28T17:41:05.086 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/coretemp.ko 2024-03-28T17:41:05.086 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/dell-smm-hwmon.ko 2024-03-28T17:41:05.086 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/dme1737.ko 2024-03-28T17:41:05.086 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/ds1621.ko 2024-03-28T17:41:05.087 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/ds620.ko 2024-03-28T17:41:05.087 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/emc1403.ko 2024-03-28T17:41:05.087 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/emc6w201.ko 2024-03-28T17:41:05.087 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/f71805f.ko 2024-03-28T17:41:05.087 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/f71882fg.ko 2024-03-28T17:41:05.087 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/f75375s.ko 2024-03-28T17:41:05.087 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/fam15h_power.ko 2024-03-28T17:41:05.087 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/fschmd.ko 2024-03-28T17:41:05.087 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/g760a.ko 2024-03-28T17:41:05.087 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/g762.ko 2024-03-28T17:41:05.087 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/gl518sm.ko 2024-03-28T17:41:05.087 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/gl520sm.ko 2024-03-28T17:41:05.087 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/hwmon-vid.ko 2024-03-28T17:41:05.087 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/i5500_temp.ko 2024-03-28T17:41:05.087 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/i5k_amb.ko 2024-03-28T17:41:05.087 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/ibmaem.ko 2024-03-28T17:41:05.087 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/ibmpex.ko 2024-03-28T17:41:05.088 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/ina209.ko 2024-03-28T17:41:05.088 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/ina2xx.ko 2024-03-28T17:41:05.088 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/it87.ko 2024-03-28T17:41:05.088 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/k10temp.ko 2024-03-28T17:41:05.088 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/k8temp.ko 2024-03-28T17:41:05.088 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/lineage-pem.ko 2024-03-28T17:41:05.088 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/lm63.ko 2024-03-28T17:41:05.088 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/lm73.ko 2024-03-28T17:41:05.088 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/lm75.ko 2024-03-28T17:41:05.088 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/lm77.ko 2024-03-28T17:41:05.088 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/lm78.ko 2024-03-28T17:41:05.088 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/lm80.ko 2024-03-28T17:41:05.088 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/lm83.ko 2024-03-28T17:41:05.088 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/lm85.ko 2024-03-28T17:41:05.088 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/lm87.ko 2024-03-28T17:41:05.088 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/lm90.ko 2024-03-28T17:41:05.089 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/lm92.ko 2024-03-28T17:41:05.089 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/lm93.ko 2024-03-28T17:41:05.089 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/lm95234.ko 2024-03-28T17:41:05.089 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/lm95241.ko 2024-03-28T17:41:05.089 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/lm95245.ko 2024-03-28T17:41:05.089 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/ltc2945.ko 2024-03-28T17:41:05.089 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/ltc4151.ko 2024-03-28T17:41:05.089 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/ltc4215.ko 2024-03-28T17:41:05.089 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/ltc4222.ko 2024-03-28T17:41:05.089 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/ltc4245.ko 2024-03-28T17:41:05.089 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/ltc4260.ko 2024-03-28T17:41:05.089 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/ltc4261.ko 2024-03-28T17:41:05.089 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/max16065.ko 2024-03-28T17:41:05.089 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/max1619.ko 2024-03-28T17:41:05.089 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/max1668.ko 2024-03-28T17:41:05.089 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/max197.ko 2024-03-28T17:41:05.089 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/max6639.ko 2024-03-28T17:41:05.090 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/max6650.ko 2024-03-28T17:41:05.090 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/max6697.ko 2024-03-28T17:41:05.090 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/mcp3021.ko 2024-03-28T17:41:05.090 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/nct6683.ko 2024-03-28T17:41:05.090 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/nct6775-core.ko 2024-03-28T17:41:05.090 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/nct6775.ko 2024-03-28T17:41:05.090 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/nct7802.ko 2024-03-28T17:41:05.090 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/pc87360.ko 2024-03-28T17:41:05.090 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/pc87427.ko 2024-03-28T17:41:05.090 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/pcf8591.ko 2024-03-28T17:41:05.090 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/pmbus 2024-03-28T17:41:05.090 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/pmbus/adm1275.ko 2024-03-28T17:41:05.090 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/pmbus/lm25066.ko 2024-03-28T17:41:05.090 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/pmbus/ltc2978.ko 2024-03-28T17:41:05.090 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/pmbus/max16064.ko 2024-03-28T17:41:05.091 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/pmbus/max34440.ko 2024-03-28T17:41:05.091 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/pmbus/max8688.ko 2024-03-28T17:41:05.091 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/pmbus/pmbus.ko 2024-03-28T17:41:05.091 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/pmbus/pmbus_core.ko 2024-03-28T17:41:05.091 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/pmbus/tps40422.ko 2024-03-28T17:41:05.091 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/pmbus/ucd9000.ko 2024-03-28T17:41:05.091 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/pmbus/ucd9200.ko 2024-03-28T17:41:05.091 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/pmbus/zl6100.ko 2024-03-28T17:41:05.091 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/powr1220.ko 2024-03-28T17:41:05.091 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/sch5627.ko 2024-03-28T17:41:05.091 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/sch5636.ko 2024-03-28T17:41:05.091 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/sch56xx-common.ko 2024-03-28T17:41:05.091 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/sht15.ko 2024-03-28T17:41:05.091 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/sht21.ko 2024-03-28T17:41:05.091 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/shtc1.ko 2024-03-28T17:41:05.091 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/sis5595.ko 2024-03-28T17:41:05.092 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/smsc47b397.ko 2024-03-28T17:41:05.092 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/smsc47m1.ko 2024-03-28T17:41:05.092 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/smsc47m192.ko 2024-03-28T17:41:05.092 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/thmc50.ko 2024-03-28T17:41:05.092 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/tmp102.ko 2024-03-28T17:41:05.092 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/tmp103.ko 2024-03-28T17:41:05.092 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/tmp401.ko 2024-03-28T17:41:05.092 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/tmp421.ko 2024-03-28T17:41:05.092 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/via-cputemp.ko 2024-03-28T17:41:05.092 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/via686a.ko 2024-03-28T17:41:05.092 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/vt1211.ko 2024-03-28T17:41:05.092 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/vt8231.ko 2024-03-28T17:41:05.092 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/w83627ehf.ko 2024-03-28T17:41:05.092 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/w83627hf.ko 2024-03-28T17:41:05.092 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/w83781d.ko 2024-03-28T17:41:05.092 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/w83791d.ko 2024-03-28T17:41:05.092 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/w83792d.ko 2024-03-28T17:41:05.093 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/w83793.ko 2024-03-28T17:41:05.093 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/w83795.ko 2024-03-28T17:41:05.093 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/w83l785ts.ko 2024-03-28T17:41:05.093 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/w83l786ng.ko 2024-03-28T17:41:05.093 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c 2024-03-28T17:41:05.093 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/algos 2024-03-28T17:41:05.093 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/algos/i2c-algo-bit.ko 2024-03-28T17:41:05.093 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/algos/i2c-algo-pca.ko 2024-03-28T17:41:05.093 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses 2024-03-28T17:41:05.093 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-amd756-s4882.ko 2024-03-28T17:41:05.093 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-amd756.ko 2024-03-28T17:41:05.093 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-amd8111.ko 2024-03-28T17:41:05.093 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-ccgx-ucsi.ko 2024-03-28T17:41:05.093 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-designware-core.ko 2024-03-28T17:41:05.093 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-designware-pci.ko 2024-03-28T17:41:05.093 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-designware-platform.ko 2024-03-28T17:41:05.094 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-diolan-u2c.ko 2024-03-28T17:41:05.094 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-i801.ko 2024-03-28T17:41:05.094 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-isch.ko 2024-03-28T17:41:05.094 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-ismt.ko 2024-03-28T17:41:05.094 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-nforce2-s4985.ko 2024-03-28T17:41:05.094 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-nforce2.ko 2024-03-28T17:41:05.094 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-parport.ko 2024-03-28T17:41:05.094 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-pca-platform.ko 2024-03-28T17:41:05.094 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-piix4.ko 2024-03-28T17:41:05.094 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-scmi.ko 2024-03-28T17:41:05.094 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-simtec.ko 2024-03-28T17:41:05.094 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-sis96x.ko 2024-03-28T17:41:05.094 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-tiny-usb.ko 2024-03-28T17:41:05.094 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-via.ko 2024-03-28T17:41:05.094 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-viapro.ko 2024-03-28T17:41:05.094 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-viperboard.ko 2024-03-28T17:41:05.095 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/i2c-dev.ko 2024-03-28T17:41:05.095 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/i2c-mux.ko 2024-03-28T17:41:05.095 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/i2c-slave-eeprom.ko 2024-03-28T17:41:05.095 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/i2c-smbus.ko 2024-03-28T17:41:05.095 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/i2c-stub.ko 2024-03-28T17:41:05.095 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband 2024-03-28T17:41:05.095 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/core 2024-03-28T17:41:05.095 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/core/ib_cm.ko 2024-03-28T17:41:05.095 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/core/ib_core.ko 2024-03-28T17:41:05.095 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/core/ib_umad.ko 2024-03-28T17:41:05.095 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/core/ib_uverbs.ko 2024-03-28T17:41:05.095 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/core/iw_cm.ko 2024-03-28T17:41:05.095 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/core/rdma_cm.ko 2024-03-28T17:41:05.095 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/core/rdma_ucm.ko 2024-03-28T17:41:05.096 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/hw 2024-03-28T17:41:05.096 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/hw/cxgb4 2024-03-28T17:41:05.096 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/hw/cxgb4/iw_cxgb4.ko 2024-03-28T17:41:05.096 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/hw/mlx4 2024-03-28T17:41:05.096 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/hw/mlx4/mlx4_ib.ko 2024-03-28T17:41:05.096 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/hw/mlx5 2024-03-28T17:41:05.096 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/hw/mlx5/mlx5_ib.ko 2024-03-28T17:41:05.096 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/hw/mthca 2024-03-28T17:41:05.096 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/hw/mthca/ib_mthca.ko 2024-03-28T17:41:05.096 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/ulp 2024-03-28T17:41:05.096 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/ulp/ipoib 2024-03-28T17:41:05.096 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/ulp/ipoib/ib_ipoib.ko 2024-03-28T17:41:05.096 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/ulp/iser 2024-03-28T17:41:05.097 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/ulp/iser/ib_iser.ko 2024-03-28T17:41:05.097 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/ulp/isert 2024-03-28T17:41:05.097 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/ulp/isert/ib_isert.ko 2024-03-28T17:41:05.097 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/ulp/srp 2024-03-28T17:41:05.097 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/ulp/srp/ib_srp.ko 2024-03-28T17:41:05.097 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/ulp/srpt 2024-03-28T17:41:05.097 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/ulp/srpt/ib_srpt.ko 2024-03-28T17:41:05.097 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input 2024-03-28T17:41:05.097 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/gameport 2024-03-28T17:41:05.097 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/gameport/emu10k1-gp.ko 2024-03-28T17:41:05.097 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/gameport/fm801-gp.ko 2024-03-28T17:41:05.097 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/gameport/gameport.ko 2024-03-28T17:41:05.097 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/joydev.ko 2024-03-28T17:41:05.098 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/keyboard 2024-03-28T17:41:05.098 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/keyboard/gpio_keys.ko 2024-03-28T17:41:05.098 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/mouse 2024-03-28T17:41:05.098 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/mouse/appletouch.ko 2024-03-28T17:41:05.098 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/mouse/bcm5974.ko 2024-03-28T17:41:05.098 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/mouse/cyapatp.ko 2024-03-28T17:41:05.098 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/mouse/elan_i2c.ko 2024-03-28T17:41:05.098 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/mouse/sermouse.ko 2024-03-28T17:41:05.098 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/mouse/synaptics_i2c.ko 2024-03-28T17:41:05.098 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/mouse/synaptics_usb.ko 2024-03-28T17:41:05.098 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/mouse/vsxxxaa.ko 2024-03-28T17:41:05.098 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/rmi4 2024-03-28T17:41:05.098 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/rmi4/rmi_core.ko 2024-03-28T17:41:05.098 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/serio 2024-03-28T17:41:05.099 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/serio/altera_ps2.ko 2024-03-28T17:41:05.099 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/serio/arc_ps2.ko 2024-03-28T17:41:05.099 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/serio/hyperv-keyboard.ko 2024-03-28T17:41:05.099 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/serio/serio_raw.ko 2024-03-28T17:41:05.099 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/sparse-keymap.ko 2024-03-28T17:41:05.099 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/leds 2024-03-28T17:41:05.099 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/leds/leds-blinkm.ko 2024-03-28T17:41:05.099 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/leds/leds-lm3530.ko 2024-03-28T17:41:05.099 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/leds/leds-lp3944.ko 2024-03-28T17:41:05.099 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/leds/leds-ss4200.ko 2024-03-28T17:41:05.099 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/leds/trigger 2024-03-28T17:41:05.099 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/leds/trigger/ledtrig-audio.ko 2024-03-28T17:41:05.099 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/leds/trigger/ledtrig-backlight.ko 2024-03-28T17:41:05.099 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/leds/trigger/ledtrig-camera.ko 2024-03-28T17:41:05.099 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/leds/trigger/ledtrig-default-on.ko 2024-03-28T17:41:05.099 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/leds/trigger/ledtrig-gpio.ko 2024-03-28T17:41:05.100 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/leds/trigger/ledtrig-heartbeat.ko 2024-03-28T17:41:05.100 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/leds/trigger/ledtrig-oneshot.ko 2024-03-28T17:41:05.100 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/leds/trigger/ledtrig-timer.ko 2024-03-28T17:41:05.100 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/leds/trigger/ledtrig-transient.ko 2024-03-28T17:41:05.100 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md 2024-03-28T17:41:05.100 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/bcache 2024-03-28T17:41:05.100 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/bcache/bcache.ko 2024-03-28T17:41:05.100 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/dm-bio-prison.ko 2024-03-28T17:41:05.100 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/dm-cache-smq.ko 2024-03-28T17:41:05.100 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/dm-cache.ko 2024-03-28T17:41:05.100 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/dm-crypt.ko 2024-03-28T17:41:05.100 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/dm-delay.ko 2024-03-28T17:41:05.100 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/dm-flakey.ko 2024-03-28T17:41:05.100 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/dm-log-userspace.ko 2024-03-28T17:41:05.100 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/dm-multipath.ko 2024-03-28T17:41:05.100 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/dm-queue-length.ko 2024-03-28T17:41:05.101 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/dm-raid.ko 2024-03-28T17:41:05.101 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/dm-round-robin.ko 2024-03-28T17:41:05.101 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/dm-service-time.ko 2024-03-28T17:41:05.101 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/dm-switch.ko 2024-03-28T17:41:05.101 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/dm-thin-pool.ko 2024-03-28T17:41:05.101 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/dm-verity.ko 2024-03-28T17:41:05.101 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/persistent-data 2024-03-28T17:41:05.101 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/persistent-data/dm-persistent-data.ko 2024-03-28T17:41:05.101 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/raid0.ko 2024-03-28T17:41:05.101 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/raid1.ko 2024-03-28T17:41:05.101 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/raid10.ko 2024-03-28T17:41:05.101 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/raid456.ko 2024-03-28T17:41:05.101 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/message 2024-03-28T17:41:05.101 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/message/fusion 2024-03-28T17:41:05.101 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/message/fusion/mptbase.ko 2024-03-28T17:41:05.101 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/message/fusion/mptctl.ko 2024-03-28T17:41:05.101 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/message/fusion/mptfc.ko 2024-03-28T17:41:05.102 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/message/fusion/mptlan.ko 2024-03-28T17:41:05.102 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/message/fusion/mptsas.ko 2024-03-28T17:41:05.102 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/message/fusion/mptscsih.ko 2024-03-28T17:41:05.102 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/message/fusion/mptspi.ko 2024-03-28T17:41:05.102 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/mfd 2024-03-28T17:41:05.102 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/mfd/lpc_ich.ko 2024-03-28T17:41:05.102 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/mfd/lpc_sch.ko 2024-03-28T17:41:05.102 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/mfd/mfd-core.ko 2024-03-28T17:41:05.102 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/mfd/sm501.ko 2024-03-28T17:41:05.102 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/mfd/viperboard.ko 2024-03-28T17:41:05.102 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/mfd/vx855.ko 2024-03-28T17:41:05.102 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/mfd/wl1273-core.ko 2024-03-28T17:41:05.102 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc 2024-03-28T17:41:05.102 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/altera-stapl 2024-03-28T17:41:05.102 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/altera-stapl/altera-stapl.ko 2024-03-28T17:41:05.103 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/apds9802als.ko 2024-03-28T17:41:05.103 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/apds990x.ko 2024-03-28T17:41:05.103 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/bh1770glc.ko 2024-03-28T17:41:05.103 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/cb710 2024-03-28T17:41:05.103 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/cb710/cb710.ko 2024-03-28T17:41:05.103 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/eeprom 2024-03-28T17:41:05.103 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/eeprom/at24.ko 2024-03-28T17:41:05.103 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/eeprom/eeprom_93cx6.ko 2024-03-28T17:41:05.103 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/eeprom/max6875.ko 2024-03-28T17:41:05.103 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/enclosure.ko 2024-03-28T17:41:05.103 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/hpilo.ko 2024-03-28T17:41:05.103 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/isl29003.ko 2024-03-28T17:41:05.103 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/isl29020.ko 2024-03-28T17:41:05.103 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/lis3lv02d 2024-03-28T17:41:05.103 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/lis3lv02d/lis3lv02d.ko 2024-03-28T17:41:05.104 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/lis3lv02d/lis3lv02d_i2c.ko 2024-03-28T17:41:05.104 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/mei 2024-03-28T17:41:05.104 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/mei/mei-me.ko 2024-03-28T17:41:05.104 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/mei/mei-txe.ko 2024-03-28T17:41:05.104 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/mei/mei.ko 2024-03-28T17:41:05.104 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/sgi-gru 2024-03-28T17:41:05.104 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/sgi-gru/gru.ko 2024-03-28T17:41:05.104 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/sgi-xp 2024-03-28T17:41:05.104 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/sgi-xp/xp.ko 2024-03-28T17:41:05.104 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/sgi-xp/xpc.ko 2024-03-28T17:41:05.104 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/sgi-xp/xpnet.ko 2024-03-28T17:41:05.104 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/tifm_7xx1.ko 2024-03-28T17:41:05.104 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/tifm_core.ko 2024-03-28T17:41:05.104 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/tsl2550.ko 2024-03-28T17:41:05.104 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/vmw_balloon.ko 2024-03-28T17:41:05.104 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/vmw_vmci 2024-03-28T17:41:05.105 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/vmw_vmci/vmw_vmci.ko 2024-03-28T17:41:05.105 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net 2024-03-28T17:41:05.105 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/bonding 2024-03-28T17:41:05.105 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/bonding/bonding.ko 2024-03-28T17:41:05.105 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/dsa 2024-03-28T17:41:05.105 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/dsa/b53 2024-03-28T17:41:05.105 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/dsa/b53/b53_common.ko 2024-03-28T17:41:05.105 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/dsa/bcm-sf2.ko 2024-03-28T17:41:05.105 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/dsa/mv88e6060.ko 2024-03-28T17:41:05.105 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/dsa/mv88e6xxx 2024-03-28T17:41:05.105 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/dsa/mv88e6xxx/mv88e6xxx.ko 2024-03-28T17:41:05.105 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/dummy.ko 2024-03-28T17:41:05.105 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/eql.ko 2024-03-28T17:41:05.105 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet 2024-03-28T17:41:05.105 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/3com 2024-03-28T17:41:05.105 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/3com/3c574_cs.ko 2024-03-28T17:41:05.106 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/3com/3c589_cs.ko 2024-03-28T17:41:05.106 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/3com/3c59x.ko 2024-03-28T17:41:05.106 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/3com/typhoon.ko 2024-03-28T17:41:05.106 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/8390 2024-03-28T17:41:05.106 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/8390/8390.ko 2024-03-28T17:41:05.106 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/8390/axnet_cs.ko 2024-03-28T17:41:05.106 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/8390/ne2k-pci.ko 2024-03-28T17:41:05.106 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/8390/pcnet_cs.ko 2024-03-28T17:41:05.106 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/adaptec 2024-03-28T17:41:05.106 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/adaptec/starfire.ko 2024-03-28T17:41:05.106 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/agere 2024-03-28T17:41:05.106 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/agere/et131x.ko 2024-03-28T17:41:05.106 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/alteon 2024-03-28T17:41:05.106 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/alteon/acenic.ko 2024-03-28T17:41:05.107 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/altera 2024-03-28T17:41:05.107 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/altera/altera_tse.ko 2024-03-28T17:41:05.107 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/amd 2024-03-28T17:41:05.107 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/amd/amd8111e.ko 2024-03-28T17:41:05.107 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/amd/nmclan_cs.ko 2024-03-28T17:41:05.107 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/amd/pcnet32.ko 2024-03-28T17:41:05.107 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/atheros 2024-03-28T17:41:05.107 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/atheros/alx 2024-03-28T17:41:05.107 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/atheros/alx/alx.ko 2024-03-28T17:41:05.107 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/atheros/atl1c 2024-03-28T17:41:05.107 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/atheros/atl1c/atl1c.ko 2024-03-28T17:41:05.107 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/atheros/atl1e 2024-03-28T17:41:05.107 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/atheros/atl1e/atl1e.ko 2024-03-28T17:41:05.107 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/atheros/atlx 2024-03-28T17:41:05.107 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/atheros/atlx/atl1.ko 2024-03-28T17:41:05.108 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/atheros/atlx/atl2.ko 2024-03-28T17:41:05.108 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/broadcom 2024-03-28T17:41:05.108 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/broadcom/b44.ko 2024-03-28T17:41:05.108 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/broadcom/bnx2.ko 2024-03-28T17:41:05.108 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/broadcom/bnx2x 2024-03-28T17:41:05.108 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/broadcom/bnx2x/bnx2x.ko 2024-03-28T17:41:05.108 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/broadcom/cnic.ko 2024-03-28T17:41:05.108 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/broadcom/genet 2024-03-28T17:41:05.108 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/broadcom/genet/genet.ko 2024-03-28T17:41:05.108 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/broadcom/tg3.ko 2024-03-28T17:41:05.108 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/brocade 2024-03-28T17:41:05.108 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/brocade/bna 2024-03-28T17:41:05.108 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/brocade/bna/bna.ko 2024-03-28T17:41:05.108 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/chelsio 2024-03-28T17:41:05.108 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/chelsio/cxgb 2024-03-28T17:41:05.109 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/chelsio/cxgb/cxgb.ko 2024-03-28T17:41:05.109 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/chelsio/cxgb3 2024-03-28T17:41:05.109 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/chelsio/cxgb3/cxgb3.ko 2024-03-28T17:41:05.109 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/chelsio/cxgb4 2024-03-28T17:41:05.109 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/chelsio/cxgb4/cxgb4.ko 2024-03-28T17:41:05.109 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/chelsio/cxgb4vf 2024-03-28T17:41:05.109 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/chelsio/cxgb4vf/cxgb4vf.ko 2024-03-28T17:41:05.109 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/chelsio/libcxgb 2024-03-28T17:41:05.109 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/chelsio/libcxgb/libcxgb.ko 2024-03-28T17:41:05.109 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/cisco 2024-03-28T17:41:05.109 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/cisco/enic 2024-03-28T17:41:05.109 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/cisco/enic/enic.ko 2024-03-28T17:41:05.109 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/dec 2024-03-28T17:41:05.109 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/dec/tulip 2024-03-28T17:41:05.110 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/dec/tulip/de2104x.ko 2024-03-28T17:41:05.110 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/dec/tulip/dmfe.ko 2024-03-28T17:41:05.110 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/dec/tulip/tulip.ko 2024-03-28T17:41:05.110 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/dec/tulip/uli526x.ko 2024-03-28T17:41:05.110 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/dec/tulip/winbond-840.ko 2024-03-28T17:41:05.110 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/dec/tulip/xircom_cb.ko 2024-03-28T17:41:05.110 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/dlink 2024-03-28T17:41:05.110 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/dlink/dl2k.ko 2024-03-28T17:41:05.110 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/dlink/sundance.ko 2024-03-28T17:41:05.110 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/dnet.ko 2024-03-28T17:41:05.110 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/emulex 2024-03-28T17:41:05.110 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/emulex/benet 2024-03-28T17:41:05.110 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/emulex/benet/be2net.ko 2024-03-28T17:41:05.110 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/ethoc.ko 2024-03-28T17:41:05.111 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/fealnx.ko 2024-03-28T17:41:05.111 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/intel 2024-03-28T17:41:05.111 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/intel/e100.ko 2024-03-28T17:41:05.111 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/intel/e1000 2024-03-28T17:41:05.111 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/intel/e1000/e1000.ko 2024-03-28T17:41:05.111 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/intel/e1000e 2024-03-28T17:41:05.111 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/intel/e1000e/e1000e.ko 2024-03-28T17:41:05.111 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/intel/fm10k 2024-03-28T17:41:05.111 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/intel/fm10k/fm10k.ko 2024-03-28T17:41:05.111 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/intel/i40e 2024-03-28T17:41:05.111 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/intel/i40e/i40e.ko 2024-03-28T17:41:05.111 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/intel/iavf 2024-03-28T17:41:05.111 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/intel/iavf/iavf.ko 2024-03-28T17:41:05.111 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/intel/igb 2024-03-28T17:41:05.111 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/intel/igb/igb.ko 2024-03-28T17:41:05.111 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/intel/igbvf 2024-03-28T17:41:05.111 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/intel/igbvf/igbvf.ko 2024-03-28T17:41:05.112 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/intel/ixgbe 2024-03-28T17:41:05.112 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/intel/ixgbe/ixgbe.ko 2024-03-28T17:41:05.112 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/intel/ixgbevf 2024-03-28T17:41:05.112 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/intel/ixgbevf/ixgbevf.ko 2024-03-28T17:41:05.112 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/jme.ko 2024-03-28T17:41:05.112 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/marvell 2024-03-28T17:41:05.112 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/marvell/mvmdio.ko 2024-03-28T17:41:05.112 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/marvell/skge.ko 2024-03-28T17:41:05.112 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/marvell/sky2.ko 2024-03-28T17:41:05.112 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/mellanox 2024-03-28T17:41:05.112 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/mellanox/mlx4 2024-03-28T17:41:05.112 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_core.ko 2024-03-28T17:41:05.112 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_en.ko 2024-03-28T17:41:05.112 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/mellanox/mlx5 2024-03-28T17:41:05.112 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/mellanox/mlx5/core 2024-03-28T17:41:05.112 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/mellanox/mlx5/core/mlx5_core.ko 2024-03-28T17:41:05.113 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/micrel 2024-03-28T17:41:05.113 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/micrel/ksz884x.ko 2024-03-28T17:41:05.113 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/myricom 2024-03-28T17:41:05.113 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/myricom/myri10ge 2024-03-28T17:41:05.113 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/myricom/myri10ge/myri10ge.ko 2024-03-28T17:41:05.113 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/natsemi 2024-03-28T17:41:05.113 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/natsemi/natsemi.ko 2024-03-28T17:41:05.113 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/natsemi/ns83820.ko 2024-03-28T17:41:05.113 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/neterion 2024-03-28T17:41:05.113 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/neterion/s2io.ko 2024-03-28T17:41:05.113 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/nvidia 2024-03-28T17:41:05.113 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/nvidia/forcedeth.ko 2024-03-28T17:41:05.113 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/packetengines 2024-03-28T17:41:05.113 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/packetengines/hamachi.ko 2024-03-28T17:41:05.113 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/packetengines/yellowfin.ko 2024-03-28T17:41:05.113 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/qlogic 2024-03-28T17:41:05.113 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/qlogic/netxen 2024-03-28T17:41:05.114 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/qlogic/netxen/netxen_nic.ko 2024-03-28T17:41:05.114 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/qlogic/qla3xxx.ko 2024-03-28T17:41:05.114 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/qlogic/qlcnic 2024-03-28T17:41:05.114 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/qlogic/qlcnic/qlcnic.ko 2024-03-28T17:41:05.114 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/rdc 2024-03-28T17:41:05.114 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/rdc/r6040.ko 2024-03-28T17:41:05.114 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/realtek 2024-03-28T17:41:05.114 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/realtek/8139cp.ko 2024-03-28T17:41:05.114 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/realtek/8139too.ko 2024-03-28T17:41:05.114 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/realtek/atp.ko 2024-03-28T17:41:05.114 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/realtek/r8169.ko 2024-03-28T17:41:05.114 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/rocker 2024-03-28T17:41:05.114 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/rocker/rocker.ko 2024-03-28T17:41:05.114 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/sfc 2024-03-28T17:41:05.115 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/sfc/sfc.ko 2024-03-28T17:41:05.115 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/silan 2024-03-28T17:41:05.115 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/silan/sc92031.ko 2024-03-28T17:41:05.115 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/sis 2024-03-28T17:41:05.115 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/sis/sis190.ko 2024-03-28T17:41:05.115 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/sis/sis900.ko 2024-03-28T17:41:05.115 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/smsc 2024-03-28T17:41:05.115 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/smsc/epic100.ko 2024-03-28T17:41:05.115 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/smsc/smc91c92_cs.ko 2024-03-28T17:41:05.115 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/smsc/smsc911x.ko 2024-03-28T17:41:05.115 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/smsc/smsc9420.ko 2024-03-28T17:41:05.115 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/stmicro 2024-03-28T17:41:05.115 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/stmicro/stmmac 2024-03-28T17:41:05.115 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/stmicro/stmmac/dwmac-intel.ko 2024-03-28T17:41:05.115 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/stmicro/stmmac/stmmac.ko 2024-03-28T17:41:05.115 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/sun 2024-03-28T17:41:05.116 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/sun/cassini.ko 2024-03-28T17:41:05.116 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/sun/niu.ko 2024-03-28T17:41:05.116 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/sun/sungem.ko 2024-03-28T17:41:05.116 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/sun/sunhme.ko 2024-03-28T17:41:05.116 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/tehuti 2024-03-28T17:41:05.116 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/tehuti/tehuti.ko 2024-03-28T17:41:05.116 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/ti 2024-03-28T17:41:05.116 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/ti/tlan.ko 2024-03-28T17:41:05.116 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/via 2024-03-28T17:41:05.116 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/via/via-rhine.ko 2024-03-28T17:41:05.116 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/via/via-velocity.ko 2024-03-28T17:41:05.116 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/wiznet 2024-03-28T17:41:05.116 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/wiznet/w5100.ko 2024-03-28T17:41:05.116 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/wiznet/w5300.ko 2024-03-28T17:41:05.116 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/xircom 2024-03-28T17:41:05.116 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/xircom/xirc2ps_cs.ko 2024-03-28T17:41:05.116 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/geneve.ko 2024-03-28T17:41:05.117 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/hyperv 2024-03-28T17:41:05.117 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/hyperv/hv_netvsc.ko 2024-03-28T17:41:05.117 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ieee802154 2024-03-28T17:41:05.117 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ieee802154/fakelb.ko 2024-03-28T17:41:05.117 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ifb.ko 2024-03-28T17:41:05.117 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ipvlan 2024-03-28T17:41:05.117 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ipvlan/ipvlan.ko 2024-03-28T17:41:05.117 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/macvlan.ko 2024-03-28T17:41:05.117 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/macvtap.ko 2024-03-28T17:41:05.117 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/mdio 2024-03-28T17:41:05.117 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/mdio.ko 2024-03-28T17:41:05.117 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/mdio/mdio-bcm-unimac.ko 2024-03-28T17:41:05.117 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/mdio/mdio-bitbang.ko 2024-03-28T17:41:05.117 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/mdio/mdio-regmap.ko 2024-03-28T17:41:05.117 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/mii.ko 2024-03-28T17:41:05.117 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/net_failover.ko 2024-03-28T17:41:05.117 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/netconsole.ko 2024-03-28T17:41:05.118 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/nlmon.ko 2024-03-28T17:41:05.118 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/pcs 2024-03-28T17:41:05.118 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/pcs/pcs-lynx.ko 2024-03-28T17:41:05.118 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/pcs/pcs_xpcs.ko 2024-03-28T17:41:05.118 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/phy 2024-03-28T17:41:05.118 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/phy/amd.ko 2024-03-28T17:41:05.118 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/phy/bcm-phy-lib.ko 2024-03-28T17:41:05.118 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/phy/bcm-phy-ptp.ko 2024-03-28T17:41:05.118 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/phy/bcm7xxx.ko 2024-03-28T17:41:05.118 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/phy/bcm87xx.ko 2024-03-28T17:41:05.118 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/phy/broadcom.ko 2024-03-28T17:41:05.118 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/phy/cicada.ko 2024-03-28T17:41:05.118 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/phy/davicom.ko 2024-03-28T17:41:05.118 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/phy/dp83640.ko 2024-03-28T17:41:05.119 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/phy/et1011c.ko 2024-03-28T17:41:05.119 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/phy/icplus.ko 2024-03-28T17:41:05.119 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/phy/lxt.ko 2024-03-28T17:41:05.119 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/phy/marvell.ko 2024-03-28T17:41:05.119 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/phy/micrel.ko 2024-03-28T17:41:05.119 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/phy/national.ko 2024-03-28T17:41:05.119 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/phy/phylink.ko 2024-03-28T17:41:05.119 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/phy/qsemi.ko 2024-03-28T17:41:05.119 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/phy/realtek.ko 2024-03-28T17:41:05.119 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/phy/smsc.ko 2024-03-28T17:41:05.119 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/phy/ste10Xp.ko 2024-03-28T17:41:05.119 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/phy/vitesse.ko 2024-03-28T17:41:05.119 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ppp 2024-03-28T17:41:05.119 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ppp/bsd_comp.ko 2024-03-28T17:41:05.120 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ppp/ppp_async.ko 2024-03-28T17:41:05.120 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ppp/ppp_deflate.ko 2024-03-28T17:41:05.120 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ppp/ppp_generic.ko 2024-03-28T17:41:05.120 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ppp/ppp_mppe.ko 2024-03-28T17:41:05.120 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ppp/ppp_synctty.ko 2024-03-28T17:41:05.120 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ppp/pppoe.ko 2024-03-28T17:41:05.120 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ppp/pppox.ko 2024-03-28T17:41:05.120 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ppp/pptp.ko 2024-03-28T17:41:05.120 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/slip 2024-03-28T17:41:05.120 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/slip/slhc.ko 2024-03-28T17:41:05.120 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/slip/slip.ko 2024-03-28T17:41:05.120 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/sungem_phy.ko 2024-03-28T17:41:05.120 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/tap.ko 2024-03-28T17:41:05.120 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/team 2024-03-28T17:41:05.120 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/team/team.ko 2024-03-28T17:41:05.121 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/team/team_mode_activebackup.ko 2024-03-28T17:41:05.121 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/team/team_mode_broadcast.ko 2024-03-28T17:41:05.121 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/team/team_mode_loadbalance.ko 2024-03-28T17:41:05.121 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/team/team_mode_random.ko 2024-03-28T17:41:05.121 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/team/team_mode_roundrobin.ko 2024-03-28T17:41:05.121 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/tun.ko 2024-03-28T17:41:05.121 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/veth.ko 2024-03-28T17:41:05.121 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/virtio_net.ko 2024-03-28T17:41:05.121 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/vmxnet3 2024-03-28T17:41:05.121 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/vmxnet3/vmxnet3.ko 2024-03-28T17:41:05.121 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/vxlan 2024-03-28T17:41:05.121 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/vxlan/vxlan.ko 2024-03-28T17:41:05.121 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/xen-netback 2024-03-28T17:41:05.121 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/xen-netback/xen-netback.ko 2024-03-28T17:41:05.122 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/xen-netfront.ko 2024-03-28T17:41:05.122 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ntb 2024-03-28T17:41:05.122 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ntb/ntb.ko 2024-03-28T17:41:05.122 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/nvme 2024-03-28T17:41:05.122 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/nvme/host 2024-03-28T17:41:05.122 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/nvme/host/nvme-core.ko 2024-03-28T17:41:05.122 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/nvme/host/nvme.ko 2024-03-28T17:41:05.122 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/parport 2024-03-28T17:41:05.122 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/parport/parport.ko 2024-03-28T17:41:05.122 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/parport/parport_cs.ko 2024-03-28T17:41:05.122 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/parport/parport_pc.ko 2024-03-28T17:41:05.122 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/parport/parport_serial.ko 2024-03-28T17:41:05.122 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/pci 2024-03-28T17:41:05.122 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/pci/hotplug 2024-03-28T17:41:05.122 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/pci/hotplug/acpiphp_ibm.ko 2024-03-28T17:41:05.123 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/pci/pcie 2024-03-28T17:41:05.123 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/pci/pcie/aer_inject.ko 2024-03-28T17:41:05.123 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/pci/xen-pcifront.ko 2024-03-28T17:41:05.123 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/pcmcia 2024-03-28T17:41:05.123 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/pcmcia/i82092.ko 2024-03-28T17:41:05.123 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/pcmcia/pd6729.ko 2024-03-28T17:41:05.123 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/pcmcia/yenta_socket.ko 2024-03-28T17:41:05.123 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/pinctrl 2024-03-28T17:41:05.123 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/pinctrl/intel 2024-03-28T17:41:05.123 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/pinctrl/intel/pinctrl-cherryview.ko 2024-03-28T17:41:05.123 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform 2024-03-28T17:41:05.123 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86 2024-03-28T17:41:05.123 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/acer-wmi.ko 2024-03-28T17:41:05.123 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/acerhdf.ko 2024-03-28T17:41:05.123 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/apple-gmux.ko 2024-03-28T17:41:05.124 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/asus-laptop.ko 2024-03-28T17:41:05.124 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/asus-nb-wmi.ko 2024-03-28T17:41:05.124 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/asus-wmi.ko 2024-03-28T17:41:05.124 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/classmate-laptop.ko 2024-03-28T17:41:05.124 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/eeepc-laptop.ko 2024-03-28T17:41:05.124 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/eeepc-wmi.ko 2024-03-28T17:41:05.124 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/fujitsu-laptop.ko 2024-03-28T17:41:05.124 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/fujitsu-tablet.ko 2024-03-28T17:41:05.124 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/hdaps.ko 2024-03-28T17:41:05.124 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/intel 2024-03-28T17:41:05.124 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/intel/intel-rst.ko 2024-03-28T17:41:05.124 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/intel_ips.ko 2024-03-28T17:41:05.124 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/msi-wmi.ko 2024-03-28T17:41:05.124 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/mxm-wmi.ko 2024-03-28T17:41:05.124 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/panasonic-laptop.ko 2024-03-28T17:41:05.124 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/samsung-laptop.ko 2024-03-28T17:41:05.124 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/samsung-q10.ko 2024-03-28T17:41:05.125 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/topstar-laptop.ko 2024-03-28T17:41:05.125 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/toshiba_bluetooth.ko 2024-03-28T17:41:05.125 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/toshiba_haps.ko 2024-03-28T17:41:05.125 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/wmi-bmof.ko 2024-03-28T17:41:05.125 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/wmi.ko 2024-03-28T17:41:05.125 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/powercap 2024-03-28T17:41:05.125 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/powercap/intel_rapl_common.ko 2024-03-28T17:41:05.125 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/powercap/intel_rapl_msr.ko 2024-03-28T17:41:05.125 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/pps 2024-03-28T17:41:05.125 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/pps/clients 2024-03-28T17:41:05.125 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/pps/clients/pps-gpio.ko 2024-03-28T17:41:05.125 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/pps/clients/pps-ldisc.ko 2024-03-28T17:41:05.125 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/pps/clients/pps_parport.ko 2024-03-28T17:41:05.125 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/pps/pps_core.ko 2024-03-28T17:41:05.125 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ptp 2024-03-28T17:41:05.125 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ptp/ptp.ko 2024-03-28T17:41:05.126 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ptp/ptp_kvm.ko 2024-03-28T17:41:05.126 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc 2024-03-28T17:41:05.126 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-bq32k.ko 2024-03-28T17:41:05.126 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-ds1286.ko 2024-03-28T17:41:05.126 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-ds1307.ko 2024-03-28T17:41:05.126 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-ds1374.ko 2024-03-28T17:41:05.126 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-ds1511.ko 2024-03-28T17:41:05.126 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-ds1553.ko 2024-03-28T17:41:05.126 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-ds1672.ko 2024-03-28T17:41:05.126 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-ds1742.ko 2024-03-28T17:41:05.126 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-ds2404.ko 2024-03-28T17:41:05.126 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-ds3232.ko 2024-03-28T17:41:05.126 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-em3027.ko 2024-03-28T17:41:05.126 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-fm3130.ko 2024-03-28T17:41:05.126 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-isl12022.ko 2024-03-28T17:41:05.127 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-isl1208.ko 2024-03-28T17:41:05.127 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-m41t80.ko 2024-03-28T17:41:05.127 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-m48t35.ko 2024-03-28T17:41:05.127 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-m48t59.ko 2024-03-28T17:41:05.127 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-max6900.ko 2024-03-28T17:41:05.127 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-msm6242.ko 2024-03-28T17:41:05.127 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-pcf2127.ko 2024-03-28T17:41:05.127 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-pcf85063.ko 2024-03-28T17:41:05.127 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-pcf8523.ko 2024-03-28T17:41:05.127 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-pcf8563.ko 2024-03-28T17:41:05.127 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-pcf8583.ko 2024-03-28T17:41:05.127 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-rp5c01.ko 2024-03-28T17:41:05.127 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-rs5c372.ko 2024-03-28T17:41:05.127 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-rv3029c2.ko 2024-03-28T17:41:05.127 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-rx8025.ko 2024-03-28T17:41:05.128 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-rx8581.ko 2024-03-28T17:41:05.128 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-stk17ta8.ko 2024-03-28T17:41:05.128 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-x1205.ko 2024-03-28T17:41:05.128 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi 2024-03-28T17:41:05.128 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/3w-9xxx.ko 2024-03-28T17:41:05.128 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/3w-sas.ko 2024-03-28T17:41:05.128 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/3w-xxxx.ko 2024-03-28T17:41:05.128 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/BusLogic.ko 2024-03-28T17:41:05.128 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/a100u2w.ko 2024-03-28T17:41:05.128 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/aacraid 2024-03-28T17:41:05.128 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/aacraid/aacraid.ko 2024-03-28T17:41:05.128 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/advansys.ko 2024-03-28T17:41:05.128 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/aic7xxx 2024-03-28T17:41:05.128 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/aic7xxx/aic79xx.ko 2024-03-28T17:41:05.128 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/aic7xxx/aic7xxx.ko 2024-03-28T17:41:05.129 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/am53c974.ko 2024-03-28T17:41:05.129 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/arcmsr 2024-03-28T17:41:05.129 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/arcmsr/arcmsr.ko 2024-03-28T17:41:05.129 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/atp870u.ko 2024-03-28T17:41:05.129 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/be2iscsi 2024-03-28T17:41:05.129 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/be2iscsi/be2iscsi.ko 2024-03-28T17:41:05.129 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/bfa 2024-03-28T17:41:05.129 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/bfa/bfa.ko 2024-03-28T17:41:05.129 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/bnx2fc 2024-03-28T17:41:05.130 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/bnx2fc/bnx2fc.ko 2024-03-28T17:41:05.130 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/bnx2i 2024-03-28T17:41:05.130 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/bnx2i/bnx2i.ko 2024-03-28T17:41:05.130 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/ch.ko 2024-03-28T17:41:05.130 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/csiostor 2024-03-28T17:41:05.130 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/csiostor/csiostor.ko 2024-03-28T17:41:05.130 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/cxgbi 2024-03-28T17:41:05.130 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/cxgbi/cxgb3i 2024-03-28T17:41:05.130 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/cxgbi/cxgb3i/cxgb3i.ko 2024-03-28T17:41:05.130 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/cxgbi/cxgb4i 2024-03-28T17:41:05.130 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/cxgbi/cxgb4i/cxgb4i.ko 2024-03-28T17:41:05.130 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/cxgbi/libcxgbi.ko 2024-03-28T17:41:05.130 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/dc395x.ko 2024-03-28T17:41:05.130 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/device_handler 2024-03-28T17:41:05.131 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/device_handler/scsi_dh_alua.ko 2024-03-28T17:41:05.131 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/device_handler/scsi_dh_emc.ko 2024-03-28T17:41:05.131 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/device_handler/scsi_dh_hp_sw.ko 2024-03-28T17:41:05.131 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/device_handler/scsi_dh_rdac.ko 2024-03-28T17:41:05.131 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/dmx3191d.ko 2024-03-28T17:41:05.131 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/esas2r 2024-03-28T17:41:05.131 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/esas2r/esas2r.ko 2024-03-28T17:41:05.131 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/esp_scsi.ko 2024-03-28T17:41:05.131 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/fcoe 2024-03-28T17:41:05.131 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/fcoe/fcoe.ko 2024-03-28T17:41:05.131 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/fcoe/libfcoe.ko 2024-03-28T17:41:05.131 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/fnic 2024-03-28T17:41:05.131 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/fnic/fnic.ko 2024-03-28T17:41:05.131 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/hpsa.ko 2024-03-28T17:41:05.131 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/hptiop.ko 2024-03-28T17:41:05.132 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/hv_storvsc.ko 2024-03-28T17:41:05.132 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/initio.ko 2024-03-28T17:41:05.132 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/ipr.ko 2024-03-28T17:41:05.132 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/ips.ko 2024-03-28T17:41:05.132 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/isci 2024-03-28T17:41:05.132 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/isci/isci.ko 2024-03-28T17:41:05.132 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/iscsi_boot_sysfs.ko 2024-03-28T17:41:05.132 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/iscsi_tcp.ko 2024-03-28T17:41:05.132 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/libfc 2024-03-28T17:41:05.132 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/libfc/libfc.ko 2024-03-28T17:41:05.132 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/libiscsi.ko 2024-03-28T17:41:05.132 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/libiscsi_tcp.ko 2024-03-28T17:41:05.132 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/libsas 2024-03-28T17:41:05.133 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/libsas/libsas.ko 2024-03-28T17:41:05.133 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/lpfc 2024-03-28T17:41:05.133 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/lpfc/lpfc.ko 2024-03-28T17:41:05.133 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/megaraid 2024-03-28T17:41:05.133 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/megaraid.ko 2024-03-28T17:41:05.133 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/megaraid/megaraid_mbox.ko 2024-03-28T17:41:05.133 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/megaraid/megaraid_mm.ko 2024-03-28T17:41:05.133 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/megaraid/megaraid_sas.ko 2024-03-28T17:41:05.133 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/mpt3sas 2024-03-28T17:41:05.133 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/mpt3sas/mpt3sas.ko 2024-03-28T17:41:05.133 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/mvsas 2024-03-28T17:41:05.133 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/mvsas/mvsas.ko 2024-03-28T17:41:05.133 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/mvumi.ko 2024-03-28T17:41:05.133 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/pm8001 2024-03-28T17:41:05.134 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/pm8001/pm80xx.ko 2024-03-28T17:41:05.134 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/pmcraid.ko 2024-03-28T17:41:05.134 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/qla1280.ko 2024-03-28T17:41:05.134 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/qla2xxx 2024-03-28T17:41:05.134 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/qla2xxx/qla2xxx.ko 2024-03-28T17:41:05.134 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/qla2xxx/tcm_qla2xxx.ko 2024-03-28T17:41:05.134 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/qla4xxx 2024-03-28T17:41:05.134 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/qla4xxx/qla4xxx.ko 2024-03-28T17:41:05.134 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/raid_class.ko 2024-03-28T17:41:05.134 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/scsi_debug.ko 2024-03-28T17:41:05.134 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/scsi_transport_fc.ko 2024-03-28T17:41:05.134 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/scsi_transport_iscsi.ko 2024-03-28T17:41:05.134 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/scsi_transport_sas.ko 2024-03-28T17:41:05.135 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/scsi_transport_spi.ko 2024-03-28T17:41:05.135 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/scsi_transport_srp.ko 2024-03-28T17:41:05.135 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/ses.ko 2024-03-28T17:41:05.135 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/st.ko 2024-03-28T17:41:05.135 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/stex.ko 2024-03-28T17:41:05.135 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/sym53c8xx_2 2024-03-28T17:41:05.135 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/sym53c8xx_2/sym53c8xx.ko 2024-03-28T17:41:05.135 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/virtio_scsi.ko 2024-03-28T17:41:05.135 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/vmw_pvscsi.ko 2024-03-28T17:41:05.135 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/wd719x.ko 2024-03-28T17:41:05.135 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ssb 2024-03-28T17:41:05.135 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ssb/ssb.ko 2024-03-28T17:41:05.135 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/target 2024-03-28T17:41:05.135 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/target/iscsi 2024-03-28T17:41:05.135 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/target/iscsi/iscsi_target_mod.ko 2024-03-28T17:41:05.135 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/target/loopback 2024-03-28T17:41:05.136 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/target/loopback/tcm_loop.ko 2024-03-28T17:41:05.136 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/target/target_core_file.ko 2024-03-28T17:41:05.136 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/target/target_core_iblock.ko 2024-03-28T17:41:05.136 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/target/target_core_mod.ko 2024-03-28T17:41:05.136 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/target/target_core_pscsi.ko 2024-03-28T17:41:05.136 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/target/target_core_user.ko 2024-03-28T17:41:05.136 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/target/tcm_fc 2024-03-28T17:41:05.136 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/target/tcm_fc/tcm_fc.ko 2024-03-28T17:41:05.136 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/thermal 2024-03-28T17:41:05.136 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/thermal/intel 2024-03-28T17:41:05.136 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/thermal/intel/int340x_thermal 2024-03-28T17:41:05.136 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/thermal/intel/int340x_thermal/acpi_thermal_rel.ko 2024-03-28T17:41:05.136 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/thermal/intel/int340x_thermal/int3400_thermal.ko 2024-03-28T17:41:05.136 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/thermal/intel/int340x_thermal/int3401_thermal.ko 2024-03-28T17:41:05.136 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/thermal/intel/int340x_thermal/int3402_thermal.ko 2024-03-28T17:41:05.137 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/thermal/intel/int340x_thermal/int3403_thermal.ko 2024-03-28T17:41:05.137 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/thermal/intel/int340x_thermal/int340x_thermal_zone.ko 2024-03-28T17:41:05.137 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device.ko 2024-03-28T17:41:05.137 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci.ko 2024-03-28T17:41:05.137 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci_legacy.ko 2024-03-28T17:41:05.137 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_mbox.ko 2024-03-28T17:41:05.137 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_power_floor.ko 2024-03-28T17:41:05.137 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rapl.ko 2024-03-28T17:41:05.137 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rfim.ko 2024-03-28T17:41:05.137 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_wt_hint.ko 2024-03-28T17:41:05.137 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_wt_req.ko 2024-03-28T17:41:05.137 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/thermal/intel/intel_soc_dts_iosf.ko 2024-03-28T17:41:05.137 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/thermal/intel/intel_soc_dts_thermal.ko 2024-03-28T17:41:05.137 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/thermal/intel/x86_pkg_temp_thermal.ko 2024-03-28T17:41:05.138 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/tty 2024-03-28T17:41:05.138 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/tty/ipwireless 2024-03-28T17:41:05.138 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/tty/ipwireless/ipwireless.ko 2024-03-28T17:41:05.138 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/tty/n_gsm.ko 2024-03-28T17:41:05.138 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/tty/n_hdlc.ko 2024-03-28T17:41:05.138 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/tty/nozomi.ko 2024-03-28T17:41:05.138 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/tty/serial 2024-03-28T17:41:05.138 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/tty/serial/8250 2024-03-28T17:41:05.138 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/tty/serial/8250/serial_cs.ko 2024-03-28T17:41:05.138 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/tty/serial/arc_uart.ko 2024-03-28T17:41:05.138 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/tty/serial/jsm 2024-03-28T17:41:05.138 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/tty/serial/jsm/jsm.ko 2024-03-28T17:41:05.138 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ufs 2024-03-28T17:41:05.138 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ufs/core 2024-03-28T17:41:05.139 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ufs/core/ufshcd-core.ko 2024-03-28T17:41:05.139 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ufs/host 2024-03-28T17:41:05.139 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ufs/host/ufshcd-pci.ko 2024-03-28T17:41:05.139 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/uio 2024-03-28T17:41:05.139 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/uio/uio.ko 2024-03-28T17:41:05.139 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/uio/uio_aec.ko 2024-03-28T17:41:05.139 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/uio/uio_cif.ko 2024-03-28T17:41:05.139 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/uio/uio_pci_generic.ko 2024-03-28T17:41:05.139 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/uio/uio_sercos3.ko 2024-03-28T17:41:05.139 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb 2024-03-28T17:41:05.139 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/class 2024-03-28T17:41:05.139 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/class/cdc-acm.ko 2024-03-28T17:41:05.139 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/class/cdc-wdm.ko 2024-03-28T17:41:05.139 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/class/usblp.ko 2024-03-28T17:41:05.139 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/class/usbtmc.ko 2024-03-28T17:41:05.139 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/host 2024-03-28T17:41:05.140 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/host/sl811-hcd.ko 2024-03-28T17:41:05.140 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/image 2024-03-28T17:41:05.140 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/image/mdc800.ko 2024-03-28T17:41:05.140 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/image/microtek.ko 2024-03-28T17:41:05.140 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/misc 2024-03-28T17:41:05.140 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/misc/adutux.ko 2024-03-28T17:41:05.140 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/misc/appledisplay.ko 2024-03-28T17:41:05.140 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/misc/emi26.ko 2024-03-28T17:41:05.140 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/misc/emi62.ko 2024-03-28T17:41:05.140 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/misc/ezusb.ko 2024-03-28T17:41:05.140 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/misc/idmouse.ko 2024-03-28T17:41:05.140 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/misc/iowarrior.ko 2024-03-28T17:41:05.140 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/misc/isight_firmware.ko 2024-03-28T17:41:05.141 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/misc/ldusb.ko 2024-03-28T17:41:05.141 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/misc/legousbtower.ko 2024-03-28T17:41:05.141 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/misc/sisusbvga 2024-03-28T17:41:05.141 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/misc/sisusbvga/sisusbvga.ko 2024-03-28T17:41:05.141 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/misc/trancevibrator.ko 2024-03-28T17:41:05.141 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/misc/usb3503.ko 2024-03-28T17:41:05.141 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/misc/usblcd.ko 2024-03-28T17:41:05.141 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/misc/usbsevseg.ko 2024-03-28T17:41:05.141 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/misc/uss720.ko 2024-03-28T17:41:05.141 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/misc/yurex.ko 2024-03-28T17:41:05.141 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/phy 2024-03-28T17:41:05.141 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/phy/phy-generic.ko 2024-03-28T17:41:05.141 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial 2024-03-28T17:41:05.142 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/aircable.ko 2024-03-28T17:41:05.142 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/ark3116.ko 2024-03-28T17:41:05.142 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/belkin_sa.ko 2024-03-28T17:41:05.142 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/ch341.ko 2024-03-28T17:41:05.142 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/cp210x.ko 2024-03-28T17:41:05.142 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/cyberjack.ko 2024-03-28T17:41:05.142 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/cypress_m8.ko 2024-03-28T17:41:05.142 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/digi_acceleport.ko 2024-03-28T17:41:05.142 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/empeg.ko 2024-03-28T17:41:05.142 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/ftdi_sio.ko 2024-03-28T17:41:05.142 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/garmin_gps.ko 2024-03-28T17:41:05.142 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/io_edgeport.ko 2024-03-28T17:41:05.142 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/io_ti.ko 2024-03-28T17:41:05.142 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/ipaq.ko 2024-03-28T17:41:05.143 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/ipw.ko 2024-03-28T17:41:05.143 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/ir-usb.ko 2024-03-28T17:41:05.143 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/iuu_phoenix.ko 2024-03-28T17:41:05.143 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/keyspan.ko 2024-03-28T17:41:05.143 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/keyspan_pda.ko 2024-03-28T17:41:05.143 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/kl5kusb105.ko 2024-03-28T17:41:05.143 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/kobil_sct.ko 2024-03-28T17:41:05.143 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/mct_u232.ko 2024-03-28T17:41:05.143 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/mos7720.ko 2024-03-28T17:41:05.143 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/mos7840.ko 2024-03-28T17:41:05.143 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/navman.ko 2024-03-28T17:41:05.143 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/omninet.ko 2024-03-28T17:41:05.143 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/opticon.ko 2024-03-28T17:41:05.144 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/option.ko 2024-03-28T17:41:05.144 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/oti6858.ko 2024-03-28T17:41:05.144 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/pl2303.ko 2024-03-28T17:41:05.144 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/qcaux.ko 2024-03-28T17:41:05.144 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/qcserial.ko 2024-03-28T17:41:05.144 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/quatech2.ko 2024-03-28T17:41:05.144 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/safe_serial.ko 2024-03-28T17:41:05.144 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/sierra.ko 2024-03-28T17:41:05.144 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/spcp8x5.ko 2024-03-28T17:41:05.144 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/ssu100.ko 2024-03-28T17:41:05.144 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/symbolserial.ko 2024-03-28T17:41:05.144 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/ti_usb_3410_5052.ko 2024-03-28T17:41:05.144 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/usb-serial-simple.ko 2024-03-28T17:41:05.145 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/usb_debug.ko 2024-03-28T17:41:05.145 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/usb_wwan.ko 2024-03-28T17:41:05.145 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/visor.ko 2024-03-28T17:41:05.145 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/whiteheat.ko 2024-03-28T17:41:05.145 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/xsens_mt.ko 2024-03-28T17:41:05.145 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/storage 2024-03-28T17:41:05.145 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/storage/uas.ko 2024-03-28T17:41:05.145 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/storage/ums-alauda.ko 2024-03-28T17:41:05.145 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/storage/ums-cypress.ko 2024-03-28T17:41:05.145 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/storage/ums-datafab.ko 2024-03-28T17:41:05.145 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/storage/ums-eneub6250.ko 2024-03-28T17:41:05.145 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/storage/ums-freecom.ko 2024-03-28T17:41:05.145 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/storage/ums-isd200.ko 2024-03-28T17:41:05.145 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/storage/ums-jumpshot.ko 2024-03-28T17:41:05.146 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/storage/ums-karma.ko 2024-03-28T17:41:05.146 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/storage/ums-onetouch.ko 2024-03-28T17:41:05.146 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/storage/ums-realtek.ko 2024-03-28T17:41:05.146 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/storage/ums-sddr09.ko 2024-03-28T17:41:05.146 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/storage/ums-sddr55.ko 2024-03-28T17:41:05.146 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/storage/ums-usbat.ko 2024-03-28T17:41:05.146 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/storage/usb-storage.ko 2024-03-28T17:41:05.146 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/usbip 2024-03-28T17:41:05.146 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/usbip/usbip-core.ko 2024-03-28T17:41:05.146 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/usbip/usbip-host.ko 2024-03-28T17:41:05.146 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/usbip/vhci-hcd.ko 2024-03-28T17:41:05.146 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/vfio 2024-03-28T17:41:05.146 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/vfio/pci 2024-03-28T17:41:05.146 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/vfio/pci/vfio-pci-core.ko 2024-03-28T17:41:05.146 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/vfio/pci/vfio-pci.ko 2024-03-28T17:41:05.147 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/vfio/vfio.ko 2024-03-28T17:41:05.147 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/vfio/vfio_iommu_type1.ko 2024-03-28T17:41:05.147 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/vhost 2024-03-28T17:41:05.147 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/vhost/vhost.ko 2024-03-28T17:41:05.147 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/vhost/vhost_iotlb.ko 2024-03-28T17:41:05.147 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/vhost/vhost_net.ko 2024-03-28T17:41:05.147 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/vhost/vhost_scsi.ko 2024-03-28T17:41:05.147 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/video 2024-03-28T17:41:05.147 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/video/backlight 2024-03-28T17:41:05.147 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/video/backlight/apple_bl.ko 2024-03-28T17:41:05.147 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/video/backlight/lcd.ko 2024-03-28T17:41:05.147 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/video/backlight/platform_lcd.ko 2024-03-28T17:41:05.147 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/virtio 2024-03-28T17:41:05.147 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/virtio/virtio.ko 2024-03-28T17:41:05.147 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/virtio/virtio_balloon.ko 2024-03-28T17:41:05.148 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/virtio/virtio_mmio.ko 2024-03-28T17:41:05.148 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/virtio/virtio_pci.ko 2024-03-28T17:41:05.148 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/virtio/virtio_pci_legacy_dev.ko 2024-03-28T17:41:05.148 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/virtio/virtio_pci_modern_dev.ko 2024-03-28T17:41:05.148 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/virtio/virtio_ring.ko 2024-03-28T17:41:05.148 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/w1 2024-03-28T17:41:05.148 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/w1/masters 2024-03-28T17:41:05.148 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/w1/masters/ds2482.ko 2024-03-28T17:41:05.148 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/w1/masters/ds2490.ko 2024-03-28T17:41:05.148 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/w1/slaves 2024-03-28T17:41:05.148 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/w1/slaves/w1_ds2406.ko 2024-03-28T17:41:05.148 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/w1/slaves/w1_ds2408.ko 2024-03-28T17:41:05.148 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/w1/slaves/w1_ds2413.ko 2024-03-28T17:41:05.148 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/w1/slaves/w1_ds2423.ko 2024-03-28T17:41:05.148 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/w1/slaves/w1_ds2431.ko 2024-03-28T17:41:05.149 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/w1/slaves/w1_ds2433.ko 2024-03-28T17:41:05.149 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/w1/slaves/w1_ds2780.ko 2024-03-28T17:41:05.149 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/w1/slaves/w1_ds2781.ko 2024-03-28T17:41:05.149 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/w1/slaves/w1_ds28e04.ko 2024-03-28T17:41:05.149 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/w1/slaves/w1_smem.ko 2024-03-28T17:41:05.149 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/w1/slaves/w1_therm.ko 2024-03-28T17:41:05.149 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/w1/wire.ko 2024-03-28T17:41:05.149 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog 2024-03-28T17:41:05.149 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/alim1535_wdt.ko 2024-03-28T17:41:05.149 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/alim7101_wdt.ko 2024-03-28T17:41:05.149 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/f71808e_wdt.ko 2024-03-28T17:41:05.149 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/hpwdt.ko 2024-03-28T17:41:05.149 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/i6300esb.ko 2024-03-28T17:41:05.149 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/iTCO_vendor_support.ko 2024-03-28T17:41:05.150 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/iTCO_wdt.ko 2024-03-28T17:41:05.150 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/ib700wdt.ko 2024-03-28T17:41:05.150 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/ibmasr.ko 2024-03-28T17:41:05.150 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/ie6xx_wdt.ko 2024-03-28T17:41:05.150 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/it8712f_wdt.ko 2024-03-28T17:41:05.150 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/it87_wdt.ko 2024-03-28T17:41:05.150 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/machzwd.ko 2024-03-28T17:41:05.150 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/nv_tco.ko 2024-03-28T17:41:05.150 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/pcwd_pci.ko 2024-03-28T17:41:05.150 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/pcwd_usb.ko 2024-03-28T17:41:05.150 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/sbc_fitpc2_wdt.ko 2024-03-28T17:41:05.150 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/sch311x_wdt.ko 2024-03-28T17:41:05.150 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/softdog.ko 2024-03-28T17:41:05.150 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/sp5100_tco.ko 2024-03-28T17:41:05.151 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/via_wdt.ko 2024-03-28T17:41:05.151 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/w83627hf_wdt.ko 2024-03-28T17:41:05.151 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/w83877f_wdt.ko 2024-03-28T17:41:05.151 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/w83977f_wdt.ko 2024-03-28T17:41:05.151 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/wdt_pci.ko 2024-03-28T17:41:05.151 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/xen_wdt.ko 2024-03-28T17:41:05.151 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/xen 2024-03-28T17:41:05.151 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/xen/xen-acpi-processor.ko 2024-03-28T17:41:05.151 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/xen/xen-evtchn.ko 2024-03-28T17:41:05.151 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/xen/xen-gntalloc.ko 2024-03-28T17:41:05.151 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/xen/xen-gntdev.ko 2024-03-28T17:41:05.151 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/xen/xen-pciback 2024-03-28T17:41:05.151 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/xen/xen-pciback/xen-pciback.ko 2024-03-28T17:41:05.151 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/xen/xen-privcmd.ko 2024-03-28T17:41:05.151 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/xen/xenfs 2024-03-28T17:41:05.152 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/xen/xenfs/xenfs.ko 2024-03-28T17:41:05.152 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs 2024-03-28T17:41:05.152 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/9p 2024-03-28T17:41:05.152 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/9p/9p.ko 2024-03-28T17:41:05.152 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/affs 2024-03-28T17:41:05.152 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/affs/affs.ko 2024-03-28T17:41:05.152 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/befs 2024-03-28T17:41:05.152 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/befs/befs.ko 2024-03-28T17:41:05.152 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/binfmt_misc.ko 2024-03-28T17:41:05.152 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/btrfs 2024-03-28T17:41:05.152 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/btrfs/btrfs.ko 2024-03-28T17:41:05.152 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/cachefiles 2024-03-28T17:41:05.152 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/cachefiles/cachefiles.ko 2024-03-28T17:41:05.152 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/ceph 2024-03-28T17:41:05.152 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/ceph/ceph.ko 2024-03-28T17:41:05.153 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/coda 2024-03-28T17:41:05.153 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/coda/coda.ko 2024-03-28T17:41:05.153 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/cramfs 2024-03-28T17:41:05.153 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/cramfs/cramfs.ko 2024-03-28T17:41:05.153 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/dlm 2024-03-28T17:41:05.153 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/dlm/dlm.ko 2024-03-28T17:41:05.153 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/ecryptfs 2024-03-28T17:41:05.153 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/ecryptfs/ecryptfs.ko 2024-03-28T17:41:05.153 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/f2fs 2024-03-28T17:41:05.153 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/f2fs/f2fs.ko 2024-03-28T17:41:05.153 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/fat 2024-03-28T17:41:05.153 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/fat/fat.ko 2024-03-28T17:41:05.153 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/fat/msdos.ko 2024-03-28T17:41:05.153 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/fat/vfat.ko 2024-03-28T17:41:05.153 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/fuse 2024-03-28T17:41:05.154 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/fuse/cuse.ko 2024-03-28T17:41:05.154 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/fuse/fuse.ko 2024-03-28T17:41:05.154 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/gfs2 2024-03-28T17:41:05.154 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/gfs2/gfs2.ko 2024-03-28T17:41:05.154 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/hfs 2024-03-28T17:41:05.154 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/hfs/hfs.ko 2024-03-28T17:41:05.154 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/hfsplus 2024-03-28T17:41:05.154 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/hfsplus/hfsplus.ko 2024-03-28T17:41:05.154 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/isofs 2024-03-28T17:41:05.154 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/isofs/isofs.ko 2024-03-28T17:41:05.154 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/jfs 2024-03-28T17:41:05.154 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/jfs/jfs.ko 2024-03-28T17:41:05.154 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/lockd 2024-03-28T17:41:05.154 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/lockd/lockd.ko 2024-03-28T17:41:05.155 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/minix 2024-03-28T17:41:05.155 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/minix/minix.ko 2024-03-28T17:41:05.155 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/netfs 2024-03-28T17:41:05.155 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/netfs/netfs.ko 2024-03-28T17:41:05.155 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nfs 2024-03-28T17:41:05.155 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nfs/blocklayout 2024-03-28T17:41:05.155 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nfs/blocklayout/blocklayoutdriver.ko 2024-03-28T17:41:05.155 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nfs/filelayout 2024-03-28T17:41:05.155 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nfs/filelayout/nfs_layout_nfsv41_files.ko 2024-03-28T17:41:05.155 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nfs/flexfilelayout 2024-03-28T17:41:05.155 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nfs/flexfilelayout/nfs_layout_flexfiles.ko 2024-03-28T17:41:05.155 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nfs/nfs.ko 2024-03-28T17:41:05.155 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nfs/nfsv3.ko 2024-03-28T17:41:05.155 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nfs/nfsv4.ko 2024-03-28T17:41:05.155 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nfs_common 2024-03-28T17:41:05.156 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nfs_common/grace.ko 2024-03-28T17:41:05.156 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nfs_common/nfs_acl.ko 2024-03-28T17:41:05.156 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nfsd 2024-03-28T17:41:05.156 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nfsd/nfsd.ko 2024-03-28T17:41:05.156 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nilfs2 2024-03-28T17:41:05.156 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nilfs2/nilfs2.ko 2024-03-28T17:41:05.156 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls 2024-03-28T17:41:05.156 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/mac-celtic.ko 2024-03-28T17:41:05.156 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/mac-centeuro.ko 2024-03-28T17:41:05.156 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/mac-croatian.ko 2024-03-28T17:41:05.156 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/mac-cyrillic.ko 2024-03-28T17:41:05.156 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/mac-gaelic.ko 2024-03-28T17:41:05.156 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/mac-greek.ko 2024-03-28T17:41:05.156 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/mac-iceland.ko 2024-03-28T17:41:05.156 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/mac-inuit.ko 2024-03-28T17:41:05.157 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/mac-roman.ko 2024-03-28T17:41:05.157 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/mac-romanian.ko 2024-03-28T17:41:05.157 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/mac-turkish.ko 2024-03-28T17:41:05.157 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_cp1250.ko 2024-03-28T17:41:05.157 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_cp1251.ko 2024-03-28T17:41:05.157 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_cp1255.ko 2024-03-28T17:41:05.157 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_cp737.ko 2024-03-28T17:41:05.157 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_cp775.ko 2024-03-28T17:41:05.157 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_cp850.ko 2024-03-28T17:41:05.157 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_cp852.ko 2024-03-28T17:41:05.157 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_cp855.ko 2024-03-28T17:41:05.157 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_cp857.ko 2024-03-28T17:41:05.157 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_cp860.ko 2024-03-28T17:41:05.158 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_cp861.ko 2024-03-28T17:41:05.158 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_cp862.ko 2024-03-28T17:41:05.158 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_cp863.ko 2024-03-28T17:41:05.158 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_cp864.ko 2024-03-28T17:41:05.158 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_cp865.ko 2024-03-28T17:41:05.158 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_cp866.ko 2024-03-28T17:41:05.158 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_cp869.ko 2024-03-28T17:41:05.158 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_cp874.ko 2024-03-28T17:41:05.158 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_cp932.ko 2024-03-28T17:41:05.158 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_cp936.ko 2024-03-28T17:41:05.158 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_cp949.ko 2024-03-28T17:41:05.158 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_cp950.ko 2024-03-28T17:41:05.158 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_euc-jp.ko 2024-03-28T17:41:05.158 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_iso8859-1.ko 2024-03-28T17:41:05.159 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_iso8859-13.ko 2024-03-28T17:41:05.159 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_iso8859-14.ko 2024-03-28T17:41:05.159 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_iso8859-15.ko 2024-03-28T17:41:05.159 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_iso8859-2.ko 2024-03-28T17:41:05.159 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_iso8859-3.ko 2024-03-28T17:41:05.159 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_iso8859-4.ko 2024-03-28T17:41:05.159 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_iso8859-5.ko 2024-03-28T17:41:05.159 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_iso8859-6.ko 2024-03-28T17:41:05.159 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_iso8859-7.ko 2024-03-28T17:41:05.159 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_iso8859-9.ko 2024-03-28T17:41:05.159 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_koi8-r.ko 2024-03-28T17:41:05.159 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_koi8-ru.ko 2024-03-28T17:41:05.159 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_koi8-u.ko 2024-03-28T17:41:05.159 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_ucs2_utils.ko 2024-03-28T17:41:05.159 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_utf8.ko 2024-03-28T17:41:05.160 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/ocfs2 2024-03-28T17:41:05.160 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/ocfs2/cluster 2024-03-28T17:41:05.160 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/ocfs2/cluster/ocfs2_nodemanager.ko 2024-03-28T17:41:05.160 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/ocfs2/dlm 2024-03-28T17:41:05.160 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/ocfs2/dlm/ocfs2_dlm.ko 2024-03-28T17:41:05.160 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/ocfs2/dlmfs 2024-03-28T17:41:05.160 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/ocfs2/dlmfs/ocfs2_dlmfs.ko 2024-03-28T17:41:05.160 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/ocfs2/ocfs2.ko 2024-03-28T17:41:05.160 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/ocfs2/ocfs2_stack_o2cb.ko 2024-03-28T17:41:05.160 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/ocfs2/ocfs2_stack_user.ko 2024-03-28T17:41:05.160 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/ocfs2/ocfs2_stackglue.ko 2024-03-28T17:41:05.160 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/overlayfs 2024-03-28T17:41:05.160 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/overlayfs/overlay.ko 2024-03-28T17:41:05.160 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/pstore 2024-03-28T17:41:05.161 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/pstore/ramoops.ko 2024-03-28T17:41:05.161 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/reiserfs 2024-03-28T17:41:05.161 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/reiserfs/reiserfs.ko 2024-03-28T17:41:05.161 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/romfs 2024-03-28T17:41:05.161 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/romfs/romfs.ko 2024-03-28T17:41:05.161 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/smb 2024-03-28T17:41:05.161 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/smb/client 2024-03-28T17:41:05.161 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/smb/client/cifs.ko 2024-03-28T17:41:05.161 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/smb/common 2024-03-28T17:41:05.161 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/smb/common/cifs_arc4.ko 2024-03-28T17:41:05.161 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/smb/common/cifs_md4.ko 2024-03-28T17:41:05.161 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/squashfs 2024-03-28T17:41:05.161 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/squashfs/squashfs.ko 2024-03-28T17:41:05.161 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/sysv 2024-03-28T17:41:05.162 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/sysv/sysv.ko 2024-03-28T17:41:05.162 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/udf 2024-03-28T17:41:05.162 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/udf/udf.ko 2024-03-28T17:41:05.162 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/ufs 2024-03-28T17:41:05.162 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/ufs/ufs.ko 2024-03-28T17:41:05.162 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/xfs 2024-03-28T17:41:05.162 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/xfs/xfs.ko 2024-03-28T17:41:05.162 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/kernel 2024-03-28T17:41:05.162 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/kernel/rcu 2024-03-28T17:41:05.162 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/kernel/rcu/rcutorture.ko 2024-03-28T17:41:05.162 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/kernel/torture.ko 2024-03-28T17:41:05.162 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/kernel/trace 2024-03-28T17:41:05.162 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/kernel/trace/ring_buffer_benchmark.ko 2024-03-28T17:41:05.162 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/lib 2024-03-28T17:41:05.162 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/asn1_encoder.ko 2024-03-28T17:41:05.163 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/crc-itu-t.ko 2024-03-28T17:41:05.163 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/crc8.ko 2024-03-28T17:41:05.163 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/crypto 2024-03-28T17:41:05.163 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/crypto/libarc4.ko 2024-03-28T17:41:05.163 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/crypto/libdes.ko 2024-03-28T17:41:05.163 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/libcrc32c.ko 2024-03-28T17:41:05.163 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/lru_cache.ko 2024-03-28T17:41:05.163 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/lz4 2024-03-28T17:41:05.163 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/lz4/lz4_compress.ko 2024-03-28T17:41:05.163 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/lz4/lz4hc_compress.ko 2024-03-28T17:41:05.163 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/math 2024-03-28T17:41:05.163 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/math/cordic.ko 2024-03-28T17:41:05.163 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/raid6 2024-03-28T17:41:05.163 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/raid6/raid6_pq.ko 2024-03-28T17:41:05.163 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/reed_solomon 2024-03-28T17:41:05.164 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/reed_solomon/reed_solomon.ko 2024-03-28T17:41:05.164 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/ts_bm.ko 2024-03-28T17:41:05.164 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/ts_fsm.ko 2024-03-28T17:41:05.164 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/ts_kmp.ko 2024-03-28T17:41:05.164 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/zstd 2024-03-28T17:41:05.164 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/zstd/zstd_compress.ko 2024-03-28T17:41:05.164 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/mm 2024-03-28T17:41:05.164 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/mm/hwpoison-inject.ko 2024-03-28T17:41:05.164 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net 2024-03-28T17:41:05.164 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/6lowpan 2024-03-28T17:41:05.164 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/6lowpan/6lowpan.ko 2024-03-28T17:41:05.164 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/6lowpan/nhc_dest.ko 2024-03-28T17:41:05.164 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/6lowpan/nhc_fragment.ko 2024-03-28T17:41:05.164 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/6lowpan/nhc_hop.ko 2024-03-28T17:41:05.165 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/6lowpan/nhc_ipv6.ko 2024-03-28T17:41:05.165 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/6lowpan/nhc_mobility.ko 2024-03-28T17:41:05.165 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/6lowpan/nhc_routing.ko 2024-03-28T17:41:05.165 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/6lowpan/nhc_udp.ko 2024-03-28T17:41:05.165 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/802 2024-03-28T17:41:05.165 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/802/garp.ko 2024-03-28T17:41:05.165 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/802/mrp.ko 2024-03-28T17:41:05.165 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/802/p8022.ko 2024-03-28T17:41:05.165 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/802/psnap.ko 2024-03-28T17:41:05.165 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/802/stp.ko 2024-03-28T17:41:05.165 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/8021q 2024-03-28T17:41:05.165 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/8021q/8021q.ko 2024-03-28T17:41:05.165 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/9p 2024-03-28T17:41:05.165 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/9p/9pnet.ko 2024-03-28T17:41:05.165 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/9p/9pnet_fd.ko 2024-03-28T17:41:05.166 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/9p/9pnet_rdma.ko 2024-03-28T17:41:05.166 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/9p/9pnet_virtio.ko 2024-03-28T17:41:05.166 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/batman-adv 2024-03-28T17:41:05.166 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/batman-adv/batman-adv.ko 2024-03-28T17:41:05.166 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge 2024-03-28T17:41:05.166 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/br_netfilter.ko 2024-03-28T17:41:05.166 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/bridge.ko 2024-03-28T17:41:05.166 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter 2024-03-28T17:41:05.166 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/ebt_802_3.ko 2024-03-28T17:41:05.166 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/ebt_among.ko 2024-03-28T17:41:05.166 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/ebt_arp.ko 2024-03-28T17:41:05.166 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/ebt_arpreply.ko 2024-03-28T17:41:05.166 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/ebt_dnat.ko 2024-03-28T17:41:05.166 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/ebt_ip.ko 2024-03-28T17:41:05.166 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/ebt_ip6.ko 2024-03-28T17:41:05.167 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/ebt_limit.ko 2024-03-28T17:41:05.167 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/ebt_log.ko 2024-03-28T17:41:05.167 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/ebt_mark.ko 2024-03-28T17:41:05.167 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/ebt_mark_m.ko 2024-03-28T17:41:05.167 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/ebt_nflog.ko 2024-03-28T17:41:05.167 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/ebt_pkttype.ko 2024-03-28T17:41:05.167 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/ebt_redirect.ko 2024-03-28T17:41:05.167 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/ebt_snat.ko 2024-03-28T17:41:05.167 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/ebt_stp.ko 2024-03-28T17:41:05.167 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/ebt_vlan.ko 2024-03-28T17:41:05.167 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/ebtable_broute.ko 2024-03-28T17:41:05.167 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/ebtable_filter.ko 2024-03-28T17:41:05.167 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/ebtable_nat.ko 2024-03-28T17:41:05.167 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/ebtables.ko 2024-03-28T17:41:05.167 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/nf_conntrack_bridge.ko 2024-03-28T17:41:05.168 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/nft_meta_bridge.ko 2024-03-28T17:41:05.168 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/nft_reject_bridge.ko 2024-03-28T17:41:05.168 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ceph 2024-03-28T17:41:05.168 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ceph/libceph.ko 2024-03-28T17:41:05.168 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/core 2024-03-28T17:41:05.168 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/core/failover.ko 2024-03-28T17:41:05.168 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/core/pktgen.ko 2024-03-28T17:41:05.168 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/dccp 2024-03-28T17:41:05.168 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/dccp/dccp.ko 2024-03-28T17:41:05.168 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/dccp/dccp_diag.ko 2024-03-28T17:41:05.168 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/dccp/dccp_ipv4.ko 2024-03-28T17:41:05.168 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/dccp/dccp_ipv6.ko 2024-03-28T17:41:05.168 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/dns_resolver 2024-03-28T17:41:05.168 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/dns_resolver/dns_resolver.ko 2024-03-28T17:41:05.168 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/dsa 2024-03-28T17:41:05.169 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/dsa/dsa_core.ko 2024-03-28T17:41:05.169 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/dsa/tag_brcm.ko 2024-03-28T17:41:05.169 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/dsa/tag_dsa.ko 2024-03-28T17:41:05.169 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/dsa/tag_none.ko 2024-03-28T17:41:05.169 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/dsa/tag_trailer.ko 2024-03-28T17:41:05.169 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ieee802154 2024-03-28T17:41:05.169 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ieee802154/6lowpan 2024-03-28T17:41:05.169 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ieee802154/6lowpan/ieee802154_6lowpan.ko 2024-03-28T17:41:05.169 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ieee802154/ieee802154.ko 2024-03-28T17:41:05.169 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ieee802154/ieee802154_socket.ko 2024-03-28T17:41:05.169 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4 2024-03-28T17:41:05.169 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/ah4.ko 2024-03-28T17:41:05.169 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/esp4.ko 2024-03-28T17:41:05.169 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/fou.ko 2024-03-28T17:41:05.170 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/gre.ko 2024-03-28T17:41:05.170 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/inet_diag.ko 2024-03-28T17:41:05.170 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/ip_gre.ko 2024-03-28T17:41:05.170 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/ip_tunnel.ko 2024-03-28T17:41:05.170 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/ip_vti.ko 2024-03-28T17:41:05.170 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/ipcomp.ko 2024-03-28T17:41:05.170 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/ipip.ko 2024-03-28T17:41:05.170 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter 2024-03-28T17:41:05.170 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter/arp_tables.ko 2024-03-28T17:41:05.170 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter/arpt_mangle.ko 2024-03-28T17:41:05.170 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter/arptable_filter.ko 2024-03-28T17:41:05.170 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter/ipt_ECN.ko 2024-03-28T17:41:05.170 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter/ipt_SYNPROXY.ko 2024-03-28T17:41:05.170 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter/ipt_ah.ko 2024-03-28T17:41:05.170 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter/ipt_rpfilter.ko 2024-03-28T17:41:05.171 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter/iptable_mangle.ko 2024-03-28T17:41:05.171 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter/iptable_nat.ko 2024-03-28T17:41:05.171 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter/iptable_raw.ko 2024-03-28T17:41:05.171 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter/iptable_security.ko 2024-03-28T17:41:05.171 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter/nf_defrag_ipv4.ko 2024-03-28T17:41:05.171 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter/nf_dup_ipv4.ko 2024-03-28T17:41:05.171 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter/nf_nat_h323.ko 2024-03-28T17:41:05.171 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter/nf_nat_pptp.ko 2024-03-28T17:41:05.171 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter/nf_nat_snmp_basic.ko 2024-03-28T17:41:05.171 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter/nf_socket_ipv4.ko 2024-03-28T17:41:05.171 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter/nf_tproxy_ipv4.ko 2024-03-28T17:41:05.171 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter/nft_dup_ipv4.ko 2024-03-28T17:41:05.171 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter/nft_fib_ipv4.ko 2024-03-28T17:41:05.171 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter/nft_reject_ipv4.ko 2024-03-28T17:41:05.171 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/tcp_bic.ko 2024-03-28T17:41:05.172 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/tcp_dctcp.ko 2024-03-28T17:41:05.172 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/tcp_diag.ko 2024-03-28T17:41:05.172 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/tcp_highspeed.ko 2024-03-28T17:41:05.172 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/tcp_htcp.ko 2024-03-28T17:41:05.172 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/tcp_hybla.ko 2024-03-28T17:41:05.172 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/tcp_illinois.ko 2024-03-28T17:41:05.172 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/tcp_lp.ko 2024-03-28T17:41:05.172 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/tcp_scalable.ko 2024-03-28T17:41:05.172 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/tcp_vegas.ko 2024-03-28T17:41:05.172 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/tcp_veno.ko 2024-03-28T17:41:05.172 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/tcp_westwood.ko 2024-03-28T17:41:05.172 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/tcp_yeah.ko 2024-03-28T17:41:05.172 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/tunnel4.ko 2024-03-28T17:41:05.172 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/udp_diag.ko 2024-03-28T17:41:05.172 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/udp_tunnel.ko 2024-03-28T17:41:05.173 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/xfrm4_tunnel.ko 2024-03-28T17:41:05.173 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6 2024-03-28T17:41:05.173 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/ah6.ko 2024-03-28T17:41:05.173 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/esp6.ko 2024-03-28T17:41:05.173 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/fou6.ko 2024-03-28T17:41:05.173 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/ip6_tunnel.ko 2024-03-28T17:41:05.173 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/ip6_udp_tunnel.ko 2024-03-28T17:41:05.173 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/ip6_vti.ko 2024-03-28T17:41:05.173 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/ipcomp6.ko 2024-03-28T17:41:05.173 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter 2024-03-28T17:41:05.173 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/ip6_tables.ko 2024-03-28T17:41:05.173 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/ip6t_REJECT.ko 2024-03-28T17:41:05.173 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/ip6t_SYNPROXY.ko 2024-03-28T17:41:05.173 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/ip6t_ah.ko 2024-03-28T17:41:05.173 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/ip6t_eui64.ko 2024-03-28T17:41:05.174 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/ip6t_frag.ko 2024-03-28T17:41:05.174 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/ip6t_hbh.ko 2024-03-28T17:41:05.174 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/ip6t_ipv6header.ko 2024-03-28T17:41:05.174 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/ip6t_mh.ko 2024-03-28T17:41:05.174 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/ip6t_rpfilter.ko 2024-03-28T17:41:05.174 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/ip6t_rt.ko 2024-03-28T17:41:05.174 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/ip6table_filter.ko 2024-03-28T17:41:05.174 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/ip6table_mangle.ko 2024-03-28T17:41:05.174 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/ip6table_nat.ko 2024-03-28T17:41:05.174 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/ip6table_raw.ko 2024-03-28T17:41:05.174 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/ip6table_security.ko 2024-03-28T17:41:05.174 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/nf_defrag_ipv6.ko 2024-03-28T17:41:05.174 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/nf_dup_ipv6.ko 2024-03-28T17:41:05.174 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/nf_reject_ipv6.ko 2024-03-28T17:41:05.175 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/nf_socket_ipv6.ko 2024-03-28T17:41:05.175 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/nf_tproxy_ipv6.ko 2024-03-28T17:41:05.175 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/nft_dup_ipv6.ko 2024-03-28T17:41:05.175 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/nft_fib_ipv6.ko 2024-03-28T17:41:05.175 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/nft_reject_ipv6.ko 2024-03-28T17:41:05.175 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/sit.ko 2024-03-28T17:41:05.175 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/tunnel6.ko 2024-03-28T17:41:05.175 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/xfrm6_tunnel.ko 2024-03-28T17:41:05.175 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/key 2024-03-28T17:41:05.175 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/key/af_key.ko 2024-03-28T17:41:05.175 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/l2tp 2024-03-28T17:41:05.175 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/l2tp/l2tp_core.ko 2024-03-28T17:41:05.175 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/l2tp/l2tp_debugfs.ko 2024-03-28T17:41:05.175 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/l2tp/l2tp_eth.ko 2024-03-28T17:41:05.175 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/l2tp/l2tp_ip.ko 2024-03-28T17:41:05.176 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/l2tp/l2tp_ip6.ko 2024-03-28T17:41:05.176 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/l2tp/l2tp_netlink.ko 2024-03-28T17:41:05.176 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/l2tp/l2tp_ppp.ko 2024-03-28T17:41:05.176 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/llc 2024-03-28T17:41:05.176 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/llc/llc.ko 2024-03-28T17:41:05.176 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/mac802154 2024-03-28T17:41:05.176 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/mac802154/mac802154.ko 2024-03-28T17:41:05.176 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/mpls 2024-03-28T17:41:05.176 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/mpls/mpls_gso.ko 2024-03-28T17:41:05.176 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter 2024-03-28T17:41:05.176 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipset 2024-03-28T17:41:05.176 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipset/ip_set.ko 2024-03-28T17:41:05.176 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipset/ip_set_bitmap_ip.ko 2024-03-28T17:41:05.176 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipset/ip_set_bitmap_ipmac.ko 2024-03-28T17:41:05.176 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipset/ip_set_bitmap_port.ko 2024-03-28T17:41:05.176 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipset/ip_set_hash_ip.ko 2024-03-28T17:41:05.177 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipset/ip_set_hash_ipmark.ko 2024-03-28T17:41:05.177 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipset/ip_set_hash_ipport.ko 2024-03-28T17:41:05.177 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipset/ip_set_hash_ipportip.ko 2024-03-28T17:41:05.177 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipset/ip_set_hash_ipportnet.ko 2024-03-28T17:41:05.177 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipset/ip_set_hash_mac.ko 2024-03-28T17:41:05.177 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipset/ip_set_hash_net.ko 2024-03-28T17:41:05.177 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipset/ip_set_hash_netiface.ko 2024-03-28T17:41:05.177 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipset/ip_set_hash_netnet.ko 2024-03-28T17:41:05.177 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipset/ip_set_hash_netport.ko 2024-03-28T17:41:05.177 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipset/ip_set_hash_netportnet.ko 2024-03-28T17:41:05.177 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipset/ip_set_list_set.ko 2024-03-28T17:41:05.177 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipvs 2024-03-28T17:41:05.177 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipvs/ip_vs.ko 2024-03-28T17:41:05.177 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipvs/ip_vs_dh.ko 2024-03-28T17:41:05.177 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipvs/ip_vs_fo.ko 2024-03-28T17:41:05.178 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipvs/ip_vs_ftp.ko 2024-03-28T17:41:05.178 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipvs/ip_vs_lblc.ko 2024-03-28T17:41:05.178 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipvs/ip_vs_lblcr.ko 2024-03-28T17:41:05.178 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipvs/ip_vs_lc.ko 2024-03-28T17:41:05.178 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipvs/ip_vs_nq.ko 2024-03-28T17:41:05.178 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipvs/ip_vs_pe_sip.ko 2024-03-28T17:41:05.178 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipvs/ip_vs_rr.ko 2024-03-28T17:41:05.178 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipvs/ip_vs_sed.ko 2024-03-28T17:41:05.178 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipvs/ip_vs_sh.ko 2024-03-28T17:41:05.178 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipvs/ip_vs_wlc.ko 2024-03-28T17:41:05.178 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipvs/ip_vs_wrr.ko 2024-03-28T17:41:05.178 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_conncount.ko 2024-03-28T17:41:05.178 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_conntrack.ko 2024-03-28T17:41:05.178 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_conntrack_amanda.ko 2024-03-28T17:41:05.179 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_conntrack_broadcast.ko 2024-03-28T17:41:05.179 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_conntrack_ftp.ko 2024-03-28T17:41:05.179 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_conntrack_h323.ko 2024-03-28T17:41:05.179 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_conntrack_irc.ko 2024-03-28T17:41:05.179 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_conntrack_netbios_ns.ko 2024-03-28T17:41:05.179 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_conntrack_netlink.ko 2024-03-28T17:41:05.181 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_conntrack_pptp.ko 2024-03-28T17:41:05.181 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_conntrack_sane.ko 2024-03-28T17:41:05.181 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_conntrack_sip.ko 2024-03-28T17:41:05.181 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_conntrack_snmp.ko 2024-03-28T17:41:05.182 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_conntrack_tftp.ko 2024-03-28T17:41:05.182 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_dup_netdev.ko 2024-03-28T17:41:05.182 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_flow_table.ko 2024-03-28T17:41:05.182 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_flow_table_inet.ko 2024-03-28T17:41:05.182 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_log_syslog.ko 2024-03-28T17:41:05.182 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_nat.ko 2024-03-28T17:41:05.182 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_nat_amanda.ko 2024-03-28T17:41:05.182 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_nat_ftp.ko 2024-03-28T17:41:05.182 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_nat_irc.ko 2024-03-28T17:41:05.182 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_nat_sip.ko 2024-03-28T17:41:05.182 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_nat_tftp.ko 2024-03-28T17:41:05.182 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_synproxy_core.ko 2024-03-28T17:41:05.182 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_tables.ko 2024-03-28T17:41:05.182 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nfnetlink.ko 2024-03-28T17:41:05.183 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nfnetlink_acct.ko 2024-03-28T17:41:05.183 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nfnetlink_log.ko 2024-03-28T17:41:05.183 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nfnetlink_osf.ko 2024-03-28T17:41:05.183 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nfnetlink_queue.ko 2024-03-28T17:41:05.183 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nft_chain_nat.ko 2024-03-28T17:41:05.183 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nft_compat.ko 2024-03-28T17:41:05.183 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nft_ct.ko 2024-03-28T17:41:05.183 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nft_dup_netdev.ko 2024-03-28T17:41:05.183 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nft_fib.ko 2024-03-28T17:41:05.183 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nft_fwd_netdev.ko 2024-03-28T17:41:05.183 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nft_hash.ko 2024-03-28T17:41:05.183 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nft_limit.ko 2024-03-28T17:41:05.183 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nft_log.ko 2024-03-28T17:41:05.183 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nft_masq.ko 2024-03-28T17:41:05.183 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nft_nat.ko 2024-03-28T17:41:05.184 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nft_numgen.ko 2024-03-28T17:41:05.184 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nft_queue.ko 2024-03-28T17:41:05.184 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nft_quota.ko 2024-03-28T17:41:05.184 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nft_redir.ko 2024-03-28T17:41:05.184 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nft_reject.ko 2024-03-28T17:41:05.184 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nft_reject_inet.ko 2024-03-28T17:41:05.184 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nft_socket.ko 2024-03-28T17:41:05.184 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nft_tunnel.ko 2024-03-28T17:41:05.184 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_AUDIT.ko 2024-03-28T17:41:05.184 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_CHECKSUM.ko 2024-03-28T17:41:05.184 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_CLASSIFY.ko 2024-03-28T17:41:05.184 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_CONNSECMARK.ko 2024-03-28T17:41:05.184 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_CT.ko 2024-03-28T17:41:05.184 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_DSCP.ko 2024-03-28T17:41:05.184 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_HL.ko 2024-03-28T17:41:05.184 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_HMARK.ko 2024-03-28T17:41:05.185 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_IDLETIMER.ko 2024-03-28T17:41:05.185 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_LED.ko 2024-03-28T17:41:05.185 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_LOG.ko 2024-03-28T17:41:05.185 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_MASQUERADE.ko 2024-03-28T17:41:05.185 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_NETMAP.ko 2024-03-28T17:41:05.185 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_NFLOG.ko 2024-03-28T17:41:05.185 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_NFQUEUE.ko 2024-03-28T17:41:05.185 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_RATEEST.ko 2024-03-28T17:41:05.185 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_REDIRECT.ko 2024-03-28T17:41:05.185 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_SECMARK.ko 2024-03-28T17:41:05.185 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_TCPMSS.ko 2024-03-28T17:41:05.185 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_TCPOPTSTRIP.ko 2024-03-28T17:41:05.185 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_TEE.ko 2024-03-28T17:41:05.185 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_TPROXY.ko 2024-03-28T17:41:05.185 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_TRACE.ko 2024-03-28T17:41:05.185 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_addrtype.ko 2024-03-28T17:41:05.186 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_bpf.ko 2024-03-28T17:41:05.186 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_cgroup.ko 2024-03-28T17:41:05.186 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_cluster.ko 2024-03-28T17:41:05.186 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_comment.ko 2024-03-28T17:41:05.186 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_connbytes.ko 2024-03-28T17:41:05.186 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_connlabel.ko 2024-03-28T17:41:05.186 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_connlimit.ko 2024-03-28T17:41:05.186 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_connmark.ko 2024-03-28T17:41:05.186 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_conntrack.ko 2024-03-28T17:41:05.186 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_cpu.ko 2024-03-28T17:41:05.186 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_dccp.ko 2024-03-28T17:41:05.186 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_devgroup.ko 2024-03-28T17:41:05.186 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_dscp.ko 2024-03-28T17:41:05.187 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_ecn.ko 2024-03-28T17:41:05.187 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_esp.ko 2024-03-28T17:41:05.187 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_hashlimit.ko 2024-03-28T17:41:05.187 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_helper.ko 2024-03-28T17:41:05.187 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_hl.ko 2024-03-28T17:41:05.187 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_ipcomp.ko 2024-03-28T17:41:05.187 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_iprange.ko 2024-03-28T17:41:05.187 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_ipvs.ko 2024-03-28T17:41:05.187 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_l2tp.ko 2024-03-28T17:41:05.187 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_length.ko 2024-03-28T17:41:05.187 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_limit.ko 2024-03-28T17:41:05.187 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_mac.ko 2024-03-28T17:41:05.187 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_mark.ko 2024-03-28T17:41:05.187 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_multiport.ko 2024-03-28T17:41:05.187 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_nat.ko 2024-03-28T17:41:05.187 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_nfacct.ko 2024-03-28T17:41:05.188 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_osf.ko 2024-03-28T17:41:05.188 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_owner.ko 2024-03-28T17:41:05.188 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_physdev.ko 2024-03-28T17:41:05.188 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_pkttype.ko 2024-03-28T17:41:05.188 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_policy.ko 2024-03-28T17:41:05.188 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_quota.ko 2024-03-28T17:41:05.188 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_rateest.ko 2024-03-28T17:41:05.188 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_realm.ko 2024-03-28T17:41:05.188 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_recent.ko 2024-03-28T17:41:05.188 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_sctp.ko 2024-03-28T17:41:05.188 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_set.ko 2024-03-28T17:41:05.188 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_socket.ko 2024-03-28T17:41:05.188 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_state.ko 2024-03-28T17:41:05.188 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_statistic.ko 2024-03-28T17:41:05.188 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_string.ko 2024-03-28T17:41:05.189 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_tcpmss.ko 2024-03-28T17:41:05.189 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_time.ko 2024-03-28T17:41:05.189 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_u32.ko 2024-03-28T17:41:05.189 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netlink 2024-03-28T17:41:05.189 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netlink/netlink_diag.ko 2024-03-28T17:41:05.189 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/nsh 2024-03-28T17:41:05.189 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/nsh/nsh.ko 2024-03-28T17:41:05.189 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/openvswitch 2024-03-28T17:41:05.189 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/openvswitch/openvswitch.ko 2024-03-28T17:41:05.189 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/openvswitch/vport-geneve.ko 2024-03-28T17:41:05.189 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/openvswitch/vport-gre.ko 2024-03-28T17:41:05.189 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/openvswitch/vport-vxlan.ko 2024-03-28T17:41:05.189 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/packet 2024-03-28T17:41:05.189 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/packet/af_packet_diag.ko 2024-03-28T17:41:05.189 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/rds 2024-03-28T17:41:05.189 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/rds/rds.ko 2024-03-28T17:41:05.190 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/rds/rds_rdma.ko 2024-03-28T17:41:05.190 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/rds/rds_tcp.ko 2024-03-28T17:41:05.190 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched 2024-03-28T17:41:05.190 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/act_csum.ko 2024-03-28T17:41:05.190 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/act_gact.ko 2024-03-28T17:41:05.190 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/act_mirred.ko 2024-03-28T17:41:05.190 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/act_nat.ko 2024-03-28T17:41:05.190 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/act_pedit.ko 2024-03-28T17:41:05.190 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/act_police.ko 2024-03-28T17:41:05.190 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/act_simple.ko 2024-03-28T17:41:05.190 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/act_skbedit.ko 2024-03-28T17:41:05.190 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/act_vlan.ko 2024-03-28T17:41:05.190 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/cls_basic.ko 2024-03-28T17:41:05.190 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/cls_bpf.ko 2024-03-28T17:41:05.190 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/cls_flow.ko 2024-03-28T17:41:05.190 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/cls_fw.ko 2024-03-28T17:41:05.191 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/cls_route.ko 2024-03-28T17:41:05.191 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/cls_u32.ko 2024-03-28T17:41:05.191 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/em_cmp.ko 2024-03-28T17:41:05.191 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/em_ipset.ko 2024-03-28T17:41:05.191 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/em_meta.ko 2024-03-28T17:41:05.191 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/em_nbyte.ko 2024-03-28T17:41:05.191 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/em_text.ko 2024-03-28T17:41:05.191 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/em_u32.ko 2024-03-28T17:41:05.191 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/sch_choke.ko 2024-03-28T17:41:05.191 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/sch_codel.ko 2024-03-28T17:41:05.191 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/sch_drr.ko 2024-03-28T17:41:05.191 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/sch_fq.ko 2024-03-28T17:41:05.191 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/sch_gred.ko 2024-03-28T17:41:05.191 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/sch_hfsc.ko 2024-03-28T17:41:05.191 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/sch_hhf.ko 2024-03-28T17:41:05.191 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/sch_htb.ko 2024-03-28T17:41:05.192 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/sch_ingress.ko 2024-03-28T17:41:05.192 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/sch_mqprio.ko 2024-03-28T17:41:05.192 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/sch_mqprio_lib.ko 2024-03-28T17:41:05.192 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/sch_multiq.ko 2024-03-28T17:41:05.192 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/sch_netem.ko 2024-03-28T17:41:05.192 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/sch_pie.ko 2024-03-28T17:41:05.192 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/sch_plug.ko 2024-03-28T17:41:05.192 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/sch_prio.ko 2024-03-28T17:41:05.192 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/sch_qfq.ko 2024-03-28T17:41:05.192 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/sch_red.ko 2024-03-28T17:41:05.192 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/sch_sfb.ko 2024-03-28T17:41:05.192 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/sch_sfq.ko 2024-03-28T17:41:05.192 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/sch_tbf.ko 2024-03-28T17:41:05.192 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/sch_teql.ko 2024-03-28T17:41:05.192 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sctp 2024-03-28T17:41:05.193 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sctp/sctp.ko 2024-03-28T17:41:05.193 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sctp/sctp_diag.ko 2024-03-28T17:41:05.193 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sunrpc 2024-03-28T17:41:05.193 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sunrpc/auth_gss 2024-03-28T17:41:05.193 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sunrpc/auth_gss/auth_rpcgss.ko 2024-03-28T17:41:05.193 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sunrpc/auth_gss/rpcsec_gss_krb5.ko 2024-03-28T17:41:05.193 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sunrpc/sunrpc.ko 2024-03-28T17:41:05.193 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sunrpc/xprtrdma 2024-03-28T17:41:05.193 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sunrpc/xprtrdma/rpcrdma.ko 2024-03-28T17:41:05.193 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/tipc 2024-03-28T17:41:05.193 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/tipc/diag.ko 2024-03-28T17:41:05.193 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/tipc/tipc.ko 2024-03-28T17:41:05.193 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/unix 2024-03-28T17:41:05.193 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/unix/unix_diag.ko 2024-03-28T17:41:05.193 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/vmw_vsock 2024-03-28T17:41:05.193 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/vmw_vsock/vmw_vsock_virtio_transport_common.ko 2024-03-28T17:41:05.194 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/vmw_vsock/vmw_vsock_vmci_transport.ko 2024-03-28T17:41:05.194 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/vmw_vsock/vsock.ko 2024-03-28T17:41:05.194 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/vmw_vsock/vsock_diag.ko 2024-03-28T17:41:05.194 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/vmw_vsock/vsock_loopback.ko 2024-03-28T17:41:05.194 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/xfrm 2024-03-28T17:41:05.194 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/xfrm/xfrm_ipcomp.ko 2024-03-28T17:41:05.194 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/security 2024-03-28T17:41:05.194 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/security/keys 2024-03-28T17:41:05.194 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/security/keys/encrypted-keys 2024-03-28T17:41:05.194 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/security/keys/encrypted-keys/encrypted-keys.ko 2024-03-28T17:41:05.194 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/security/keys/trusted-keys 2024-03-28T17:41:05.194 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/security/keys/trusted-keys/trusted.ko 2024-03-28T17:41:05.194 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/virt 2024-03-28T17:41:05.194 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/virt/lib 2024-03-28T17:41:05.195 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/virt/lib/irqbypass.ko 2024-03-28T17:41:05.195 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/modules.alias 2024-03-28T17:41:05.195 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/modules.alias.bin 2024-03-28T17:41:05.195 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/modules.builtin 2024-03-28T17:41:05.195 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/modules.builtin.alias.bin 2024-03-28T17:41:05.195 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/modules.builtin.bin 2024-03-28T17:41:05.195 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/modules.builtin.modinfo 2024-03-28T17:41:05.195 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/modules.dep 2024-03-28T17:41:05.195 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/modules.dep.bin 2024-03-28T17:41:05.195 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/modules.devname 2024-03-28T17:41:05.195 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/modules.order 2024-03-28T17:41:05.195 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/modules.softdep 2024-03-28T17:41:05.195 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/modules.symbols 2024-03-28T17:41:05.195 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/modules.symbols.bin 2024-03-28T17:41:05.195 INFO:teuthology.orchestra.run.smithi079.stdout:/lib/modules/6.8.0-g1df0c7e8a235/vmlinuz 2024-03-28T17:41:05.199 DEBUG:teuthology.task.kernel:get_image_version: 6.8.0-g1df0c7e8a235 2024-03-28T17:41:05.200 DEBUG:teuthology.orchestra.run.smithi079:> rpm --scripts -qp /tmp/kernel.x86_64.rpm 2024-03-28T17:41:05.224 INFO:teuthology.orchestra.run.smithi079.stdout:postinstall scriptlet (using /bin/sh): 2024-03-28T17:41:05.224 INFO:teuthology.orchestra.run.smithi079.stdout:if [ -x /usr/bin/kernel-install ]; then 2024-03-28T17:41:05.224 INFO:teuthology.orchestra.run.smithi079.stdout: /usr/bin/kernel-install add 6.8.0-g1df0c7e8a235 /lib/modules/6.8.0-g1df0c7e8a235/vmlinuz 2024-03-28T17:41:05.224 INFO:teuthology.orchestra.run.smithi079.stdout:fi 2024-03-28T17:41:05.224 INFO:teuthology.orchestra.run.smithi079.stdout:for file in vmlinuz System.map config; do 2024-03-28T17:41:05.224 INFO:teuthology.orchestra.run.smithi079.stdout: if ! cmp --silent "/lib/modules/6.8.0-g1df0c7e8a235/${file}" "/boot/${file}-6.8.0-g1df0c7e8a235"; then 2024-03-28T17:41:05.224 INFO:teuthology.orchestra.run.smithi079.stdout: cp "/lib/modules/6.8.0-g1df0c7e8a235/${file}" "/boot/${file}-6.8.0-g1df0c7e8a235" 2024-03-28T17:41:05.224 INFO:teuthology.orchestra.run.smithi079.stdout: fi 2024-03-28T17:41:05.224 INFO:teuthology.orchestra.run.smithi079.stdout:done 2024-03-28T17:41:05.224 INFO:teuthology.orchestra.run.smithi079.stdout:preuninstall scriptlet (using /bin/sh): 2024-03-28T17:41:05.224 INFO:teuthology.orchestra.run.smithi079.stdout:if [ -x /sbin/new-kernel-pkg ]; then 2024-03-28T17:41:05.224 INFO:teuthology.orchestra.run.smithi079.stdout:new-kernel-pkg --remove 6.8.0-g1df0c7e8a235 --rminitrd --initrdfile=/boot/initramfs-6.8.0-g1df0c7e8a235.img 2024-03-28T17:41:05.225 INFO:teuthology.orchestra.run.smithi079.stdout:elif [ -x /usr/bin/kernel-install ]; then 2024-03-28T17:41:05.225 INFO:teuthology.orchestra.run.smithi079.stdout:kernel-install remove 6.8.0-g1df0c7e8a235 2024-03-28T17:41:05.225 INFO:teuthology.orchestra.run.smithi079.stdout:fi 2024-03-28T17:41:05.225 INFO:teuthology.orchestra.run.smithi079.stdout:postuninstall scriptlet (using /bin/sh): 2024-03-28T17:41:05.225 INFO:teuthology.orchestra.run.smithi079.stdout:if [ -x /sbin/update-bootloader ]; then 2024-03-28T17:41:05.225 INFO:teuthology.orchestra.run.smithi079.stdout:/sbin/update-bootloader --remove 6.8.0-g1df0c7e8a235 2024-03-28T17:41:05.225 INFO:teuthology.orchestra.run.smithi079.stdout:fi 2024-03-28T17:41:05.226 DEBUG:teuthology.orchestra.run.smithi079:> sudo rpm -qi grub2-tools 2024-03-28T17:41:05.303 INFO:teuthology.orchestra.run.smithi079.stdout:Name : grub2-tools 2024-03-28T17:41:05.303 INFO:teuthology.orchestra.run.smithi079.stdout:Epoch : 1 2024-03-28T17:41:05.303 INFO:teuthology.orchestra.run.smithi079.stdout:Version : 2.06 2024-03-28T17:41:05.303 INFO:teuthology.orchestra.run.smithi079.stdout:Release : 68.el9 2024-03-28T17:41:05.303 INFO:teuthology.orchestra.run.smithi079.stdout:Architecture: x86_64 2024-03-28T17:41:05.304 INFO:teuthology.orchestra.run.smithi079.stdout:Install Date: Fri 01 Sep 2023 09:14:30 PM UTC 2024-03-28T17:41:05.304 INFO:teuthology.orchestra.run.smithi079.stdout:Group : Unspecified 2024-03-28T17:41:05.304 INFO:teuthology.orchestra.run.smithi079.stdout:Size : 8264254 2024-03-28T17:41:05.304 INFO:teuthology.orchestra.run.smithi079.stdout:License : GPLv3+ 2024-03-28T17:41:05.304 INFO:teuthology.orchestra.run.smithi079.stdout:Signature : RSA/SHA256, Mon 07 Aug 2023 01:57:16 PM UTC, Key ID 05b555b38483c65d 2024-03-28T17:41:05.304 INFO:teuthology.orchestra.run.smithi079.stdout:Source RPM : grub2-2.06-68.el9.src.rpm 2024-03-28T17:41:05.304 INFO:teuthology.orchestra.run.smithi079.stdout:Build Date : Wed 02 Aug 2023 03:15:48 PM UTC 2024-03-28T17:41:05.304 INFO:teuthology.orchestra.run.smithi079.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-03-28T17:41:05.304 INFO:teuthology.orchestra.run.smithi079.stdout:Packager : builder@centos.org 2024-03-28T17:41:05.304 INFO:teuthology.orchestra.run.smithi079.stdout:Vendor : CentOS 2024-03-28T17:41:05.304 INFO:teuthology.orchestra.run.smithi079.stdout:URL : http://www.gnu.org/software/grub/ 2024-03-28T17:41:05.304 INFO:teuthology.orchestra.run.smithi079.stdout:Summary : Support tools for GRUB. 2024-03-28T17:41:05.304 INFO:teuthology.orchestra.run.smithi079.stdout:Description : 2024-03-28T17:41:05.304 INFO:teuthology.orchestra.run.smithi079.stdout: 2024-03-28T17:41:05.304 INFO:teuthology.orchestra.run.smithi079.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-03-28T17:41:05.304 INFO:teuthology.orchestra.run.smithi079.stdout:customizable bootloader with modular architecture. It supports a rich 2024-03-28T17:41:05.305 INFO:teuthology.orchestra.run.smithi079.stdout:variety of kernel formats, file systems, computer architectures and 2024-03-28T17:41:05.305 INFO:teuthology.orchestra.run.smithi079.stdout:hardware devices. 2024-03-28T17:41:05.305 INFO:teuthology.orchestra.run.smithi079.stdout: 2024-03-28T17:41:05.305 INFO:teuthology.orchestra.run.smithi079.stdout:This subpackage provides tools for support of all platforms. 2024-03-28T17:41:05.306 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-03-28T17:41:05.306 INFO:teuthology.task.kernel:Updating grub on smithi079 to boot 6.8.0-g1df0c7e8a235 2024-03-28T17:41:05.306 DEBUG:teuthology.orchestra.run.smithi079:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-03-28T17:41:05.937 INFO:teuthology.orchestra.run.smithi079.stderr:Generating grub configuration file ... 2024-03-28T17:41:07.294 INFO:teuthology.orchestra.run.smithi079.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-03-28T17:41:07.329 INFO:teuthology.orchestra.run.smithi079.stderr:done 2024-03-28T17:41:07.331 DEBUG:teuthology.orchestra.run.smithi079:> mktemp 2024-03-28T17:41:07.346 INFO:teuthology.orchestra.run.smithi079.stdout:/tmp/tmp.f2YcIhVtOe 2024-03-28T17:41:07.346 DEBUG:teuthology.orchestra.run.smithi079:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.f2YcIhVtOe 2024-03-28T17:41:07.432 DEBUG:teuthology.orchestra.run.smithi079:> sudo chmod 0666 /tmp/tmp.f2YcIhVtOe 2024-03-28T17:41:07.530 DEBUG:teuthology.orchestra.remote:smithi079:/tmp/tmp.f2YcIhVtOe is 6KB 2024-03-28T17:41:07.549 DEBUG:teuthology.orchestra.run.smithi079:> rm -fr /tmp/tmp.f2YcIhVtOe 2024-03-28T17:41:07.562 DEBUG:teuthology.orchestra.run.smithi079:> sudo /bin/ls /boot/loader/entries || true 2024-03-28T17:41:07.630 INFO:teuthology.orchestra.run.smithi079.stdout:15949e3c88704300882482eddfd7baae-0-rescue.conf 2024-03-28T17:41:07.630 INFO:teuthology.orchestra.run.smithi079.stdout:15949e3c88704300882482eddfd7baae-5.14.0-239.el9.x86_64.conf 2024-03-28T17:41:07.630 INFO:teuthology.orchestra.run.smithi079.stdout:15949e3c88704300882482eddfd7baae-5.14.0-361.el9.x86_64.conf 2024-03-28T17:41:07.630 INFO:teuthology.orchestra.run.smithi079.stdout:15949e3c88704300882482eddfd7baae-6.8.0-g1df0c7e8a235.conf 2024-03-28T17:41:07.631 DEBUG:teuthology.orchestra.run.smithi079:> sudo grub2-set-default 15949e3c88704300882482eddfd7baae-6.8.0-g1df0c7e8a235 2024-03-28T17:41:07.792 DEBUG:teuthology.orchestra.run.smithi079:> sudo shutdown -r now 2024-03-28T17:41:12.828 INFO:teuthology.orchestra.run.smithi022.stderr:dracut: Disabling early microcode, because kernel does not support it. CONFIG_MICROCODE_[AMD|INTEL]!=y 2024-03-28T17:41:22.105 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi134.front.sepia.ceph.com, path=/tmp/kernel.x86_64.rpm, version=None) 2024-03-28T17:41:22.106 DEBUG:teuthology.orchestra.run.smithi134:> rpm -qlp /tmp/kernel.x86_64.rpm 2024-03-28T17:41:22.177 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235 2024-03-28T17:41:22.177 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/System.map 2024-03-28T17:41:22.177 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/config 2024-03-28T17:41:22.177 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel 2024-03-28T17:41:22.177 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch 2024-03-28T17:41:22.177 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86 2024-03-28T17:41:22.177 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/crypto 2024-03-28T17:41:22.177 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/crypto/blowfish-x86_64.ko 2024-03-28T17:41:22.177 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/crypto/camellia-aesni-avx-x86_64.ko 2024-03-28T17:41:22.177 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/crypto/camellia-aesni-avx2.ko 2024-03-28T17:41:22.177 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/crypto/camellia-x86_64.ko 2024-03-28T17:41:22.177 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/crypto/cast5-avx-x86_64.ko 2024-03-28T17:41:22.177 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/crypto/cast6-avx-x86_64.ko 2024-03-28T17:41:22.177 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/crypto/crc32-pclmul.ko 2024-03-28T17:41:22.178 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/crypto/crc32c-intel.ko 2024-03-28T17:41:22.178 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/crypto/crct10dif-pclmul.ko 2024-03-28T17:41:22.178 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/crypto/des3_ede-x86_64.ko 2024-03-28T17:41:22.178 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/crypto/ghash-clmulni-intel.ko 2024-03-28T17:41:22.178 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/crypto/serpent-avx-x86_64.ko 2024-03-28T17:41:22.178 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/crypto/serpent-avx2.ko 2024-03-28T17:41:22.178 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/crypto/serpent-sse2-x86_64.ko 2024-03-28T17:41:22.178 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/crypto/sha1-ssse3.ko 2024-03-28T17:41:22.178 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/crypto/sha256-ssse3.ko 2024-03-28T17:41:22.178 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/crypto/sha512-ssse3.ko 2024-03-28T17:41:22.178 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/crypto/twofish-avx-x86_64.ko 2024-03-28T17:41:22.178 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/crypto/twofish-x86_64-3way.ko 2024-03-28T17:41:22.178 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/crypto/twofish-x86_64.ko 2024-03-28T17:41:22.179 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/kernel 2024-03-28T17:41:22.179 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/kernel/cpu 2024-03-28T17:41:22.179 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/kernel/cpu/mce 2024-03-28T17:41:22.179 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/kernel/cpu/mce/mce-inject.ko 2024-03-28T17:41:22.179 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/kvm 2024-03-28T17:41:22.179 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/kvm/kvm-amd.ko 2024-03-28T17:41:22.179 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/kvm/kvm-intel.ko 2024-03-28T17:41:22.179 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/kvm/kvm.ko 2024-03-28T17:41:22.179 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto 2024-03-28T17:41:22.179 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/ansi_cprng.ko 2024-03-28T17:41:22.179 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/anubis.ko 2024-03-28T17:41:22.179 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/arc4.ko 2024-03-28T17:41:22.179 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/async_tx 2024-03-28T17:41:22.179 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/async_tx/async_memcpy.ko 2024-03-28T17:41:22.179 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/async_tx/async_pq.ko 2024-03-28T17:41:22.180 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/async_tx/async_raid6_recov.ko 2024-03-28T17:41:22.180 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/async_tx/async_tx.ko 2024-03-28T17:41:22.180 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/async_tx/async_xor.ko 2024-03-28T17:41:22.180 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/async_tx/raid6test.ko 2024-03-28T17:41:22.180 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/authenc.ko 2024-03-28T17:41:22.180 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/authencesn.ko 2024-03-28T17:41:22.180 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/blake2b_generic.ko 2024-03-28T17:41:22.180 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/blowfish_common.ko 2024-03-28T17:41:22.180 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/blowfish_generic.ko 2024-03-28T17:41:22.180 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/camellia_generic.ko 2024-03-28T17:41:22.180 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/cast5_generic.ko 2024-03-28T17:41:22.180 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/cast6_generic.ko 2024-03-28T17:41:22.180 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/cast_common.ko 2024-03-28T17:41:22.180 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/ccm.ko 2024-03-28T17:41:22.180 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/cmac.ko 2024-03-28T17:41:22.181 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/crc32_generic.ko 2024-03-28T17:41:22.181 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/crypto_engine.ko 2024-03-28T17:41:22.181 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/crypto_user.ko 2024-03-28T17:41:22.181 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/cts.ko 2024-03-28T17:41:22.181 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/des_generic.ko 2024-03-28T17:41:22.181 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/dh_generic.ko 2024-03-28T17:41:22.181 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/echainiv.ko 2024-03-28T17:41:22.181 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/essiv.ko 2024-03-28T17:41:22.181 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/fcrypt.ko 2024-03-28T17:41:22.181 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/khazad.ko 2024-03-28T17:41:22.181 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/lz4.ko 2024-03-28T17:41:22.181 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/lz4hc.ko 2024-03-28T17:41:22.181 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/md4.ko 2024-03-28T17:41:22.181 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/michael_mic.ko 2024-03-28T17:41:22.181 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/pcbc.ko 2024-03-28T17:41:22.182 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/pcrypt.ko 2024-03-28T17:41:22.182 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/rmd160.ko 2024-03-28T17:41:22.182 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/seed.ko 2024-03-28T17:41:22.182 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/serpent_generic.ko 2024-03-28T17:41:22.182 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/tcrypt.ko 2024-03-28T17:41:22.182 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/tea.ko 2024-03-28T17:41:22.182 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/twofish_common.ko 2024-03-28T17:41:22.182 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/twofish_generic.ko 2024-03-28T17:41:22.182 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/vmac.ko 2024-03-28T17:41:22.182 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/wp512.ko 2024-03-28T17:41:22.182 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/xcbc.ko 2024-03-28T17:41:22.182 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/xor.ko 2024-03-28T17:41:22.182 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/xxhash_generic.ko 2024-03-28T17:41:22.182 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers 2024-03-28T17:41:22.183 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/acpi 2024-03-28T17:41:22.183 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/acpi/acpi_ipmi.ko 2024-03-28T17:41:22.183 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/acpi/acpi_pad.ko 2024-03-28T17:41:22.183 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/acpi/custom_method.ko 2024-03-28T17:41:22.183 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/acpi/ec_sys.ko 2024-03-28T17:41:22.183 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/acpi/platform_profile.ko 2024-03-28T17:41:22.183 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/acpi/sbs.ko 2024-03-28T17:41:22.183 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/acpi/sbshc.ko 2024-03-28T17:41:22.183 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/acpi/video.ko 2024-03-28T17:41:22.183 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata 2024-03-28T17:41:22.183 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/acard-ahci.ko 2024-03-28T17:41:22.183 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/ahci_platform.ko 2024-03-28T17:41:22.183 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/ata_generic.ko 2024-03-28T17:41:22.183 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/libahci_platform.ko 2024-03-28T17:41:22.184 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_acpi.ko 2024-03-28T17:41:22.184 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_ali.ko 2024-03-28T17:41:22.184 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_amd.ko 2024-03-28T17:41:22.184 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_artop.ko 2024-03-28T17:41:22.184 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_atiixp.ko 2024-03-28T17:41:22.184 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_atp867x.ko 2024-03-28T17:41:22.184 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_cmd640.ko 2024-03-28T17:41:22.184 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_cmd64x.ko 2024-03-28T17:41:22.184 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_cypress.ko 2024-03-28T17:41:22.184 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_efar.ko 2024-03-28T17:41:22.184 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_hpt366.ko 2024-03-28T17:41:22.184 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_hpt37x.ko 2024-03-28T17:41:22.184 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_hpt3x2n.ko 2024-03-28T17:41:22.184 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_hpt3x3.ko 2024-03-28T17:41:22.185 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_it8213.ko 2024-03-28T17:41:22.185 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_it821x.ko 2024-03-28T17:41:22.185 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_jmicron.ko 2024-03-28T17:41:22.185 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_marvell.ko 2024-03-28T17:41:22.185 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_mpiix.ko 2024-03-28T17:41:22.185 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_netcell.ko 2024-03-28T17:41:22.185 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_ninja32.ko 2024-03-28T17:41:22.185 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_ns87410.ko 2024-03-28T17:41:22.185 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_ns87415.ko 2024-03-28T17:41:22.185 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_oldpiix.ko 2024-03-28T17:41:22.185 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_opti.ko 2024-03-28T17:41:22.185 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_optidma.ko 2024-03-28T17:41:22.185 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_pcmcia.ko 2024-03-28T17:41:22.186 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_pdc2027x.ko 2024-03-28T17:41:22.186 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_pdc202xx_old.ko 2024-03-28T17:41:22.186 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_piccolo.ko 2024-03-28T17:41:22.186 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_rdc.ko 2024-03-28T17:41:22.186 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_sch.ko 2024-03-28T17:41:22.186 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_serverworks.ko 2024-03-28T17:41:22.186 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_sil680.ko 2024-03-28T17:41:22.186 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_sis.ko 2024-03-28T17:41:22.186 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_sl82c105.ko 2024-03-28T17:41:22.186 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_triflex.ko 2024-03-28T17:41:22.186 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_via.ko 2024-03-28T17:41:22.186 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pdc_adma.ko 2024-03-28T17:41:22.187 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/sata_inic162x.ko 2024-03-28T17:41:22.187 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/sata_mv.ko 2024-03-28T17:41:22.187 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/sata_nv.ko 2024-03-28T17:41:22.187 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/sata_promise.ko 2024-03-28T17:41:22.187 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/sata_qstor.ko 2024-03-28T17:41:22.187 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/sata_sil.ko 2024-03-28T17:41:22.187 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/sata_sil24.ko 2024-03-28T17:41:22.187 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/sata_sis.ko 2024-03-28T17:41:22.187 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/sata_svw.ko 2024-03-28T17:41:22.187 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/sata_sx4.ko 2024-03-28T17:41:22.187 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/sata_uli.ko 2024-03-28T17:41:22.187 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/sata_via.ko 2024-03-28T17:41:22.187 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/sata_vsc.ko 2024-03-28T17:41:22.187 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/auxdisplay 2024-03-28T17:41:22.188 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/auxdisplay/ks0108.ko 2024-03-28T17:41:22.188 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/base 2024-03-28T17:41:22.188 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/base/regmap 2024-03-28T17:41:22.188 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/base/regmap/regmap-i2c.ko 2024-03-28T17:41:22.188 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/base/regmap/regmap-mmio.ko 2024-03-28T17:41:22.188 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/bcma 2024-03-28T17:41:22.188 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/bcma/bcma.ko 2024-03-28T17:41:22.188 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/block 2024-03-28T17:41:22.188 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/block/aoe 2024-03-28T17:41:22.188 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/block/aoe/aoe.ko 2024-03-28T17:41:22.188 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/block/brd.ko 2024-03-28T17:41:22.188 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/block/drbd 2024-03-28T17:41:22.188 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/block/drbd/drbd.ko 2024-03-28T17:41:22.188 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/block/floppy.ko 2024-03-28T17:41:22.189 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/block/loop.ko 2024-03-28T17:41:22.189 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/block/mtip32xx 2024-03-28T17:41:22.189 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/block/mtip32xx/mtip32xx.ko 2024-03-28T17:41:22.189 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/block/nbd.ko 2024-03-28T17:41:22.189 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/block/null_blk 2024-03-28T17:41:22.189 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/block/null_blk/null_blk.ko 2024-03-28T17:41:22.189 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/block/pktcdvd.ko 2024-03-28T17:41:22.189 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/block/rbd.ko 2024-03-28T17:41:22.189 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/block/virtio_blk.ko 2024-03-28T17:41:22.189 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/block/xen-blkback 2024-03-28T17:41:22.189 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/block/xen-blkback/xen-blkback.ko 2024-03-28T17:41:22.189 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/block/xen-blkfront.ko 2024-03-28T17:41:22.189 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/block/zram 2024-03-28T17:41:22.189 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/block/zram/zram.ko 2024-03-28T17:41:22.190 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char 2024-03-28T17:41:22.190 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/hangcheck-timer.ko 2024-03-28T17:41:22.190 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/hw_random 2024-03-28T17:41:22.190 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/hw_random/amd-rng.ko 2024-03-28T17:41:22.190 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/hw_random/intel-rng.ko 2024-03-28T17:41:22.190 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/hw_random/timeriomem-rng.ko 2024-03-28T17:41:22.190 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/hw_random/via-rng.ko 2024-03-28T17:41:22.190 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/hw_random/virtio-rng.ko 2024-03-28T17:41:22.190 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/ipmi 2024-03-28T17:41:22.190 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/ipmi/ipmi_devintf.ko 2024-03-28T17:41:22.190 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/ipmi/ipmi_msghandler.ko 2024-03-28T17:41:22.190 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/ipmi/ipmi_poweroff.ko 2024-03-28T17:41:22.190 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/ipmi/ipmi_si.ko 2024-03-28T17:41:22.190 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/ipmi/ipmi_ssif.ko 2024-03-28T17:41:22.191 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/ipmi/ipmi_watchdog.ko 2024-03-28T17:41:22.191 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/lp.ko 2024-03-28T17:41:22.191 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/mwave 2024-03-28T17:41:22.191 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/mwave/mwave.ko 2024-03-28T17:41:22.191 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/ppdev.ko 2024-03-28T17:41:22.191 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/tlclk.ko 2024-03-28T17:41:22.191 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/tpm 2024-03-28T17:41:22.191 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/tpm/tpm.ko 2024-03-28T17:41:22.191 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/tpm/tpm_atmel.ko 2024-03-28T17:41:22.191 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/tpm/tpm_infineon.ko 2024-03-28T17:41:22.191 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/tpm/tpm_nsc.ko 2024-03-28T17:41:22.191 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/tpm/tpm_tis.ko 2024-03-28T17:41:22.191 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/tpm/tpm_tis_core.ko 2024-03-28T17:41:22.191 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/uv_mmtimer.ko 2024-03-28T17:41:22.191 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/virtio_console.ko 2024-03-28T17:41:22.192 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/cpufreq 2024-03-28T17:41:22.192 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/cpufreq/acpi-cpufreq.ko 2024-03-28T17:41:22.192 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/cpufreq/amd_freq_sensitivity.ko 2024-03-28T17:41:22.192 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/cpufreq/p4-clockmod.ko 2024-03-28T17:41:22.192 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/cpufreq/pcc-cpufreq.ko 2024-03-28T17:41:22.192 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/cpufreq/powernow-k8.ko 2024-03-28T17:41:22.192 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/cpufreq/speedstep-lib.ko 2024-03-28T17:41:22.192 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/crypto 2024-03-28T17:41:22.192 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/crypto/ccp 2024-03-28T17:41:22.192 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/crypto/ccp/ccp-crypto.ko 2024-03-28T17:41:22.192 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/crypto/ccp/ccp.ko 2024-03-28T17:41:22.192 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/crypto/intel 2024-03-28T17:41:22.192 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/crypto/intel/qat 2024-03-28T17:41:22.192 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/crypto/intel/qat/qat_common 2024-03-28T17:41:22.192 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/crypto/intel/qat/qat_common/intel_qat.ko 2024-03-28T17:41:22.193 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/crypto/intel/qat/qat_dh895xcc 2024-03-28T17:41:22.193 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/crypto/intel/qat/qat_dh895xcc/qat_dh895xcc.ko 2024-03-28T17:41:22.193 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/crypto/padlock-aes.ko 2024-03-28T17:41:22.193 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/crypto/padlock-sha.ko 2024-03-28T17:41:22.193 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/crypto/virtio 2024-03-28T17:41:22.193 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/crypto/virtio/virtio_crypto.ko 2024-03-28T17:41:22.193 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/dca 2024-03-28T17:41:22.193 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/dca/dca.ko 2024-03-28T17:41:22.193 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/devfreq 2024-03-28T17:41:22.193 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/devfreq/governor_simpleondemand.ko 2024-03-28T17:41:22.193 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/dma 2024-03-28T17:41:22.193 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/dma/dw 2024-03-28T17:41:22.193 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/dma/dw/dw_dmac.ko 2024-03-28T17:41:22.193 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/dma/ioat 2024-03-28T17:41:22.193 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/dma/ioat/ioatdma.ko 2024-03-28T17:41:22.193 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/edac 2024-03-28T17:41:22.194 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/edac/amd64_edac.ko 2024-03-28T17:41:22.194 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/edac/e752x_edac.ko 2024-03-28T17:41:22.194 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/edac/edac_mce_amd.ko 2024-03-28T17:41:22.194 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/edac/i3000_edac.ko 2024-03-28T17:41:22.194 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/edac/i3200_edac.ko 2024-03-28T17:41:22.194 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/edac/i5100_edac.ko 2024-03-28T17:41:22.194 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/edac/i5400_edac.ko 2024-03-28T17:41:22.194 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/edac/i7300_edac.ko 2024-03-28T17:41:22.194 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/edac/i7core_edac.ko 2024-03-28T17:41:22.194 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/edac/i82975x_edac.ko 2024-03-28T17:41:22.194 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/edac/ie31200_edac.ko 2024-03-28T17:41:22.194 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/edac/sb_edac.ko 2024-03-28T17:41:22.194 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/edac/x38_edac.ko 2024-03-28T17:41:22.194 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/firmware 2024-03-28T17:41:22.195 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/firmware/edd.ko 2024-03-28T17:41:22.195 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/firmware/iscsi_ibft.ko 2024-03-28T17:41:22.195 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/gpio 2024-03-28T17:41:22.195 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/gpio/gpio-ich.ko 2024-03-28T17:41:22.195 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/gpio/gpio-viperboard.ko 2024-03-28T17:41:22.195 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid 2024-03-28T17:41:22.195 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-appleir.ko 2024-03-28T17:41:22.195 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-aureal.ko 2024-03-28T17:41:22.195 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-axff.ko 2024-03-28T17:41:22.195 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-dr.ko 2024-03-28T17:41:22.195 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-elecom.ko 2024-03-28T17:41:22.195 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-elo.ko 2024-03-28T17:41:22.195 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-emsff.ko 2024-03-28T17:41:22.195 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-gaff.ko 2024-03-28T17:41:22.195 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-gt683r.ko 2024-03-28T17:41:22.196 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-gyration.ko 2024-03-28T17:41:22.196 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-holtek-kbd.ko 2024-03-28T17:41:22.196 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-holtek-mouse.ko 2024-03-28T17:41:22.196 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-holtekff.ko 2024-03-28T17:41:22.196 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-hyperv.ko 2024-03-28T17:41:22.196 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-icade.ko 2024-03-28T17:41:22.196 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-keytouch.ko 2024-03-28T17:41:22.196 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-kye.ko 2024-03-28T17:41:22.196 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-lcpower.ko 2024-03-28T17:41:22.196 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-led.ko 2024-03-28T17:41:22.196 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-lenovo.ko 2024-03-28T17:41:22.196 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-logitech-dj.ko 2024-03-28T17:41:22.196 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-logitech-hidpp.ko 2024-03-28T17:41:22.196 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-multitouch.ko 2024-03-28T17:41:22.196 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-ortek.ko 2024-03-28T17:41:22.197 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-penmount.ko 2024-03-28T17:41:22.197 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-petalynx.ko 2024-03-28T17:41:22.197 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-picolcd.ko 2024-03-28T17:41:22.197 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-pl.ko 2024-03-28T17:41:22.197 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-plantronics.ko 2024-03-28T17:41:22.197 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-primax.ko 2024-03-28T17:41:22.197 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-rmi.ko 2024-03-28T17:41:22.197 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-roccat-arvo.ko 2024-03-28T17:41:22.197 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-roccat-common.ko 2024-03-28T17:41:22.197 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-roccat-isku.ko 2024-03-28T17:41:22.197 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-roccat-kone.ko 2024-03-28T17:41:22.197 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-roccat-koneplus.ko 2024-03-28T17:41:22.197 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-roccat-konepure.ko 2024-03-28T17:41:22.197 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-roccat-kovaplus.ko 2024-03-28T17:41:22.197 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-roccat-lua.ko 2024-03-28T17:41:22.198 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-roccat-pyra.ko 2024-03-28T17:41:22.198 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-roccat-ryos.ko 2024-03-28T17:41:22.198 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-roccat-savu.ko 2024-03-28T17:41:22.198 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-roccat.ko 2024-03-28T17:41:22.198 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-saitek.ko 2024-03-28T17:41:22.198 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-samsung.ko 2024-03-28T17:41:22.198 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-sensor-hub.ko 2024-03-28T17:41:22.198 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-sjoy.ko 2024-03-28T17:41:22.198 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-sony.ko 2024-03-28T17:41:22.198 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-speedlink.ko 2024-03-28T17:41:22.198 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-steelseries.ko 2024-03-28T17:41:22.198 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-sunplus.ko 2024-03-28T17:41:22.198 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-thrustmaster.ko 2024-03-28T17:41:22.198 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-tivo.ko 2024-03-28T17:41:22.198 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-tmff.ko 2024-03-28T17:41:22.199 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-topseed.ko 2024-03-28T17:41:22.199 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-twinhan.ko 2024-03-28T17:41:22.199 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-uclogic.ko 2024-03-28T17:41:22.199 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-waltop.ko 2024-03-28T17:41:22.199 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-wiimote.ko 2024-03-28T17:41:22.199 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-xinmo.ko 2024-03-28T17:41:22.199 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-zpff.ko 2024-03-28T17:41:22.199 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-zydacron.ko 2024-03-28T17:41:22.199 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/uhid.ko 2024-03-28T17:41:22.199 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/wacom.ko 2024-03-28T17:41:22.199 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hv 2024-03-28T17:41:22.199 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hv/hv_balloon.ko 2024-03-28T17:41:22.199 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hv/hv_utils.ko 2024-03-28T17:41:22.199 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hv/hv_vmbus.ko 2024-03-28T17:41:22.199 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon 2024-03-28T17:41:22.200 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/abituguru.ko 2024-03-28T17:41:22.200 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/abituguru3.ko 2024-03-28T17:41:22.200 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/acpi_power_meter.ko 2024-03-28T17:41:22.200 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/ad7414.ko 2024-03-28T17:41:22.200 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/ad7418.ko 2024-03-28T17:41:22.200 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/adc128d818.ko 2024-03-28T17:41:22.200 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/adm1025.ko 2024-03-28T17:41:22.200 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/adm1026.ko 2024-03-28T17:41:22.200 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/adm1029.ko 2024-03-28T17:41:22.200 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/adm1031.ko 2024-03-28T17:41:22.200 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/adm9240.ko 2024-03-28T17:41:22.200 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/ads7828.ko 2024-03-28T17:41:22.200 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/adt7410.ko 2024-03-28T17:41:22.200 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/adt7411.ko 2024-03-28T17:41:22.200 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/adt7462.ko 2024-03-28T17:41:22.200 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/adt7470.ko 2024-03-28T17:41:22.201 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/adt7475.ko 2024-03-28T17:41:22.201 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/adt7x10.ko 2024-03-28T17:41:22.201 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/amc6821.ko 2024-03-28T17:41:22.201 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/applesmc.ko 2024-03-28T17:41:22.201 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/asb100.ko 2024-03-28T17:41:22.201 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/asc7621.ko 2024-03-28T17:41:22.201 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/asus_atk0110.ko 2024-03-28T17:41:22.201 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/atxp1.ko 2024-03-28T17:41:22.201 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/coretemp.ko 2024-03-28T17:41:22.201 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/dell-smm-hwmon.ko 2024-03-28T17:41:22.201 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/dme1737.ko 2024-03-28T17:41:22.201 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/ds1621.ko 2024-03-28T17:41:22.201 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/ds620.ko 2024-03-28T17:41:22.201 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/emc1403.ko 2024-03-28T17:41:22.201 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/emc6w201.ko 2024-03-28T17:41:22.202 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/f71805f.ko 2024-03-28T17:41:22.202 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/f71882fg.ko 2024-03-28T17:41:22.202 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/f75375s.ko 2024-03-28T17:41:22.202 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/fam15h_power.ko 2024-03-28T17:41:22.202 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/fschmd.ko 2024-03-28T17:41:22.202 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/g760a.ko 2024-03-28T17:41:22.202 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/g762.ko 2024-03-28T17:41:22.202 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/gl518sm.ko 2024-03-28T17:41:22.202 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/gl520sm.ko 2024-03-28T17:41:22.202 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/hwmon-vid.ko 2024-03-28T17:41:22.202 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/i5500_temp.ko 2024-03-28T17:41:22.202 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/i5k_amb.ko 2024-03-28T17:41:22.202 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/ibmaem.ko 2024-03-28T17:41:22.202 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/ibmpex.ko 2024-03-28T17:41:22.202 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/ina209.ko 2024-03-28T17:41:22.203 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/ina2xx.ko 2024-03-28T17:41:22.203 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/it87.ko 2024-03-28T17:41:22.203 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/k10temp.ko 2024-03-28T17:41:22.203 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/k8temp.ko 2024-03-28T17:41:22.203 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/lineage-pem.ko 2024-03-28T17:41:22.203 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/lm63.ko 2024-03-28T17:41:22.203 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/lm73.ko 2024-03-28T17:41:22.203 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/lm75.ko 2024-03-28T17:41:22.203 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/lm77.ko 2024-03-28T17:41:22.203 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/lm78.ko 2024-03-28T17:41:22.203 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/lm80.ko 2024-03-28T17:41:22.203 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/lm83.ko 2024-03-28T17:41:22.203 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/lm85.ko 2024-03-28T17:41:22.203 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/lm87.ko 2024-03-28T17:41:22.203 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/lm90.ko 2024-03-28T17:41:22.203 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/lm92.ko 2024-03-28T17:41:22.204 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/lm93.ko 2024-03-28T17:41:22.204 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/lm95234.ko 2024-03-28T17:41:22.204 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/lm95241.ko 2024-03-28T17:41:22.204 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/lm95245.ko 2024-03-28T17:41:22.204 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/ltc2945.ko 2024-03-28T17:41:22.204 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/ltc4151.ko 2024-03-28T17:41:22.204 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/ltc4215.ko 2024-03-28T17:41:22.204 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/ltc4222.ko 2024-03-28T17:41:22.204 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/ltc4245.ko 2024-03-28T17:41:22.204 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/ltc4260.ko 2024-03-28T17:41:22.204 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/ltc4261.ko 2024-03-28T17:41:22.204 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/max16065.ko 2024-03-28T17:41:22.204 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/max1619.ko 2024-03-28T17:41:22.204 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/max1668.ko 2024-03-28T17:41:22.204 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/max197.ko 2024-03-28T17:41:22.205 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/max6639.ko 2024-03-28T17:41:22.205 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/max6650.ko 2024-03-28T17:41:22.205 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/max6697.ko 2024-03-28T17:41:22.205 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/mcp3021.ko 2024-03-28T17:41:22.205 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/nct6683.ko 2024-03-28T17:41:22.205 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/nct6775-core.ko 2024-03-28T17:41:22.205 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/nct6775.ko 2024-03-28T17:41:22.205 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/nct7802.ko 2024-03-28T17:41:22.205 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/pc87360.ko 2024-03-28T17:41:22.205 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/pc87427.ko 2024-03-28T17:41:22.205 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/pcf8591.ko 2024-03-28T17:41:22.205 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/pmbus 2024-03-28T17:41:22.205 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/pmbus/adm1275.ko 2024-03-28T17:41:22.205 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/pmbus/lm25066.ko 2024-03-28T17:41:22.205 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/pmbus/ltc2978.ko 2024-03-28T17:41:22.206 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/pmbus/max16064.ko 2024-03-28T17:41:22.206 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/pmbus/max34440.ko 2024-03-28T17:41:22.206 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/pmbus/max8688.ko 2024-03-28T17:41:22.206 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/pmbus/pmbus.ko 2024-03-28T17:41:22.206 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/pmbus/pmbus_core.ko 2024-03-28T17:41:22.206 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/pmbus/tps40422.ko 2024-03-28T17:41:22.206 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/pmbus/ucd9000.ko 2024-03-28T17:41:22.206 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/pmbus/ucd9200.ko 2024-03-28T17:41:22.206 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/pmbus/zl6100.ko 2024-03-28T17:41:22.206 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/powr1220.ko 2024-03-28T17:41:22.206 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/sch5627.ko 2024-03-28T17:41:22.206 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/sch5636.ko 2024-03-28T17:41:22.206 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/sch56xx-common.ko 2024-03-28T17:41:22.206 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/sht15.ko 2024-03-28T17:41:22.206 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/sht21.ko 2024-03-28T17:41:22.207 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/shtc1.ko 2024-03-28T17:41:22.207 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/sis5595.ko 2024-03-28T17:41:22.207 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/smsc47b397.ko 2024-03-28T17:41:22.207 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/smsc47m1.ko 2024-03-28T17:41:22.207 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/smsc47m192.ko 2024-03-28T17:41:22.207 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/thmc50.ko 2024-03-28T17:41:22.207 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/tmp102.ko 2024-03-28T17:41:22.207 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/tmp103.ko 2024-03-28T17:41:22.207 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/tmp401.ko 2024-03-28T17:41:22.207 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/tmp421.ko 2024-03-28T17:41:22.207 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/via-cputemp.ko 2024-03-28T17:41:22.207 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/via686a.ko 2024-03-28T17:41:22.207 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/vt1211.ko 2024-03-28T17:41:22.207 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/vt8231.ko 2024-03-28T17:41:22.207 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/w83627ehf.ko 2024-03-28T17:41:22.207 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/w83627hf.ko 2024-03-28T17:41:22.208 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/w83781d.ko 2024-03-28T17:41:22.208 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/w83791d.ko 2024-03-28T17:41:22.208 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/w83792d.ko 2024-03-28T17:41:22.208 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/w83793.ko 2024-03-28T17:41:22.208 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/w83795.ko 2024-03-28T17:41:22.208 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/w83l785ts.ko 2024-03-28T17:41:22.208 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/w83l786ng.ko 2024-03-28T17:41:22.208 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c 2024-03-28T17:41:22.208 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/algos 2024-03-28T17:41:22.208 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/algos/i2c-algo-bit.ko 2024-03-28T17:41:22.208 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/algos/i2c-algo-pca.ko 2024-03-28T17:41:22.208 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses 2024-03-28T17:41:22.208 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-amd756-s4882.ko 2024-03-28T17:41:22.208 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-amd756.ko 2024-03-28T17:41:22.208 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-amd8111.ko 2024-03-28T17:41:22.208 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-ccgx-ucsi.ko 2024-03-28T17:41:22.209 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-designware-core.ko 2024-03-28T17:41:22.209 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-designware-pci.ko 2024-03-28T17:41:22.209 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-designware-platform.ko 2024-03-28T17:41:22.209 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-diolan-u2c.ko 2024-03-28T17:41:22.209 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-i801.ko 2024-03-28T17:41:22.209 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-isch.ko 2024-03-28T17:41:22.209 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-ismt.ko 2024-03-28T17:41:22.209 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-nforce2-s4985.ko 2024-03-28T17:41:22.209 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-nforce2.ko 2024-03-28T17:41:22.209 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-parport.ko 2024-03-28T17:41:22.209 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-pca-platform.ko 2024-03-28T17:41:22.209 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-piix4.ko 2024-03-28T17:41:22.209 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-scmi.ko 2024-03-28T17:41:22.209 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-simtec.ko 2024-03-28T17:41:22.209 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-sis96x.ko 2024-03-28T17:41:22.209 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-tiny-usb.ko 2024-03-28T17:41:22.209 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-via.ko 2024-03-28T17:41:22.210 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-viapro.ko 2024-03-28T17:41:22.210 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-viperboard.ko 2024-03-28T17:41:22.210 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/i2c-dev.ko 2024-03-28T17:41:22.210 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/i2c-mux.ko 2024-03-28T17:41:22.210 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/i2c-slave-eeprom.ko 2024-03-28T17:41:22.210 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/i2c-smbus.ko 2024-03-28T17:41:22.210 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/i2c-stub.ko 2024-03-28T17:41:22.210 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband 2024-03-28T17:41:22.210 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/core 2024-03-28T17:41:22.210 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/core/ib_cm.ko 2024-03-28T17:41:22.210 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/core/ib_core.ko 2024-03-28T17:41:22.210 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/core/ib_umad.ko 2024-03-28T17:41:22.210 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/core/ib_uverbs.ko 2024-03-28T17:41:22.210 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/core/iw_cm.ko 2024-03-28T17:41:22.210 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/core/rdma_cm.ko 2024-03-28T17:41:22.211 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/core/rdma_ucm.ko 2024-03-28T17:41:22.211 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/hw 2024-03-28T17:41:22.211 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/hw/cxgb4 2024-03-28T17:41:22.211 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/hw/cxgb4/iw_cxgb4.ko 2024-03-28T17:41:22.211 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/hw/mlx4 2024-03-28T17:41:22.211 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/hw/mlx4/mlx4_ib.ko 2024-03-28T17:41:22.211 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/hw/mlx5 2024-03-28T17:41:22.211 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/hw/mlx5/mlx5_ib.ko 2024-03-28T17:41:22.211 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/hw/mthca 2024-03-28T17:41:22.211 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/hw/mthca/ib_mthca.ko 2024-03-28T17:41:22.211 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/ulp 2024-03-28T17:41:22.211 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/ulp/ipoib 2024-03-28T17:41:22.211 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/ulp/ipoib/ib_ipoib.ko 2024-03-28T17:41:22.211 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/ulp/iser 2024-03-28T17:41:22.211 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/ulp/iser/ib_iser.ko 2024-03-28T17:41:22.211 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/ulp/isert 2024-03-28T17:41:22.211 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/ulp/isert/ib_isert.ko 2024-03-28T17:41:22.212 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/ulp/srp 2024-03-28T17:41:22.212 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/ulp/srp/ib_srp.ko 2024-03-28T17:41:22.212 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/ulp/srpt 2024-03-28T17:41:22.212 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/ulp/srpt/ib_srpt.ko 2024-03-28T17:41:22.212 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input 2024-03-28T17:41:22.212 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/gameport 2024-03-28T17:41:22.212 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/gameport/emu10k1-gp.ko 2024-03-28T17:41:22.212 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/gameport/fm801-gp.ko 2024-03-28T17:41:22.212 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/gameport/gameport.ko 2024-03-28T17:41:22.212 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/joydev.ko 2024-03-28T17:41:22.212 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/keyboard 2024-03-28T17:41:22.212 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/keyboard/gpio_keys.ko 2024-03-28T17:41:22.212 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/mouse 2024-03-28T17:41:22.212 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/mouse/appletouch.ko 2024-03-28T17:41:22.212 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/mouse/bcm5974.ko 2024-03-28T17:41:22.212 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/mouse/cyapatp.ko 2024-03-28T17:41:22.213 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/mouse/elan_i2c.ko 2024-03-28T17:41:22.213 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/mouse/sermouse.ko 2024-03-28T17:41:22.213 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/mouse/synaptics_i2c.ko 2024-03-28T17:41:22.213 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/mouse/synaptics_usb.ko 2024-03-28T17:41:22.213 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/mouse/vsxxxaa.ko 2024-03-28T17:41:22.213 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/rmi4 2024-03-28T17:41:22.213 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/rmi4/rmi_core.ko 2024-03-28T17:41:22.213 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/serio 2024-03-28T17:41:22.213 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/serio/altera_ps2.ko 2024-03-28T17:41:22.213 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/serio/arc_ps2.ko 2024-03-28T17:41:22.213 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/serio/hyperv-keyboard.ko 2024-03-28T17:41:22.213 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/serio/serio_raw.ko 2024-03-28T17:41:22.213 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/sparse-keymap.ko 2024-03-28T17:41:22.213 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/leds 2024-03-28T17:41:22.213 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/leds/leds-blinkm.ko 2024-03-28T17:41:22.213 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/leds/leds-lm3530.ko 2024-03-28T17:41:22.213 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/leds/leds-lp3944.ko 2024-03-28T17:41:22.214 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/leds/leds-ss4200.ko 2024-03-28T17:41:22.214 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/leds/trigger 2024-03-28T17:41:22.214 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/leds/trigger/ledtrig-audio.ko 2024-03-28T17:41:22.214 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/leds/trigger/ledtrig-backlight.ko 2024-03-28T17:41:22.214 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/leds/trigger/ledtrig-camera.ko 2024-03-28T17:41:22.214 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/leds/trigger/ledtrig-default-on.ko 2024-03-28T17:41:22.214 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/leds/trigger/ledtrig-gpio.ko 2024-03-28T17:41:22.214 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/leds/trigger/ledtrig-heartbeat.ko 2024-03-28T17:41:22.214 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/leds/trigger/ledtrig-oneshot.ko 2024-03-28T17:41:22.214 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/leds/trigger/ledtrig-timer.ko 2024-03-28T17:41:22.214 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/leds/trigger/ledtrig-transient.ko 2024-03-28T17:41:22.214 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md 2024-03-28T17:41:22.214 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/bcache 2024-03-28T17:41:22.214 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/bcache/bcache.ko 2024-03-28T17:41:22.214 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/dm-bio-prison.ko 2024-03-28T17:41:22.215 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/dm-cache-smq.ko 2024-03-28T17:41:22.215 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/dm-cache.ko 2024-03-28T17:41:22.215 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/dm-crypt.ko 2024-03-28T17:41:22.215 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/dm-delay.ko 2024-03-28T17:41:22.215 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/dm-flakey.ko 2024-03-28T17:41:22.215 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/dm-log-userspace.ko 2024-03-28T17:41:22.215 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/dm-multipath.ko 2024-03-28T17:41:22.215 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/dm-queue-length.ko 2024-03-28T17:41:22.215 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/dm-raid.ko 2024-03-28T17:41:22.215 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/dm-round-robin.ko 2024-03-28T17:41:22.215 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/dm-service-time.ko 2024-03-28T17:41:22.215 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/dm-switch.ko 2024-03-28T17:41:22.215 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/dm-thin-pool.ko 2024-03-28T17:41:22.215 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/dm-verity.ko 2024-03-28T17:41:22.215 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/persistent-data 2024-03-28T17:41:22.216 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/persistent-data/dm-persistent-data.ko 2024-03-28T17:41:22.216 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/raid0.ko 2024-03-28T17:41:22.216 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/raid1.ko 2024-03-28T17:41:22.216 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/raid10.ko 2024-03-28T17:41:22.216 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/raid456.ko 2024-03-28T17:41:22.216 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/message 2024-03-28T17:41:22.216 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/message/fusion 2024-03-28T17:41:22.216 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/message/fusion/mptbase.ko 2024-03-28T17:41:22.216 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/message/fusion/mptctl.ko 2024-03-28T17:41:22.216 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/message/fusion/mptfc.ko 2024-03-28T17:41:22.216 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/message/fusion/mptlan.ko 2024-03-28T17:41:22.216 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/message/fusion/mptsas.ko 2024-03-28T17:41:22.216 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/message/fusion/mptscsih.ko 2024-03-28T17:41:22.216 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/message/fusion/mptspi.ko 2024-03-28T17:41:22.216 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/mfd 2024-03-28T17:41:22.217 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/mfd/lpc_ich.ko 2024-03-28T17:41:22.217 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/mfd/lpc_sch.ko 2024-03-28T17:41:22.217 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/mfd/mfd-core.ko 2024-03-28T17:41:22.217 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/mfd/sm501.ko 2024-03-28T17:41:22.217 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/mfd/viperboard.ko 2024-03-28T17:41:22.217 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/mfd/vx855.ko 2024-03-28T17:41:22.217 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/mfd/wl1273-core.ko 2024-03-28T17:41:22.217 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc 2024-03-28T17:41:22.217 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/altera-stapl 2024-03-28T17:41:22.217 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/altera-stapl/altera-stapl.ko 2024-03-28T17:41:22.217 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/apds9802als.ko 2024-03-28T17:41:22.217 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/apds990x.ko 2024-03-28T17:41:22.217 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/bh1770glc.ko 2024-03-28T17:41:22.217 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/cb710 2024-03-28T17:41:22.217 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/cb710/cb710.ko 2024-03-28T17:41:22.217 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/eeprom 2024-03-28T17:41:22.218 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/eeprom/at24.ko 2024-03-28T17:41:22.218 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/eeprom/eeprom_93cx6.ko 2024-03-28T17:41:22.218 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/eeprom/max6875.ko 2024-03-28T17:41:22.218 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/enclosure.ko 2024-03-28T17:41:22.218 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/hpilo.ko 2024-03-28T17:41:22.218 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/isl29003.ko 2024-03-28T17:41:22.218 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/isl29020.ko 2024-03-28T17:41:22.218 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/lis3lv02d 2024-03-28T17:41:22.218 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/lis3lv02d/lis3lv02d.ko 2024-03-28T17:41:22.218 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/lis3lv02d/lis3lv02d_i2c.ko 2024-03-28T17:41:22.218 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/mei 2024-03-28T17:41:22.218 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/mei/mei-me.ko 2024-03-28T17:41:22.218 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/mei/mei-txe.ko 2024-03-28T17:41:22.218 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/mei/mei.ko 2024-03-28T17:41:22.218 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/sgi-gru 2024-03-28T17:41:22.219 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/sgi-gru/gru.ko 2024-03-28T17:41:22.219 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/sgi-xp 2024-03-28T17:41:22.219 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/sgi-xp/xp.ko 2024-03-28T17:41:22.219 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/sgi-xp/xpc.ko 2024-03-28T17:41:22.219 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/sgi-xp/xpnet.ko 2024-03-28T17:41:22.219 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/tifm_7xx1.ko 2024-03-28T17:41:22.219 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/tifm_core.ko 2024-03-28T17:41:22.219 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/tsl2550.ko 2024-03-28T17:41:22.219 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/vmw_balloon.ko 2024-03-28T17:41:22.219 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/vmw_vmci 2024-03-28T17:41:22.219 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/vmw_vmci/vmw_vmci.ko 2024-03-28T17:41:22.219 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net 2024-03-28T17:41:22.219 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/bonding 2024-03-28T17:41:22.219 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/bonding/bonding.ko 2024-03-28T17:41:22.219 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/dsa 2024-03-28T17:41:22.219 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/dsa/b53 2024-03-28T17:41:22.220 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/dsa/b53/b53_common.ko 2024-03-28T17:41:22.220 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/dsa/bcm-sf2.ko 2024-03-28T17:41:22.220 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/dsa/mv88e6060.ko 2024-03-28T17:41:22.220 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/dsa/mv88e6xxx 2024-03-28T17:41:22.220 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/dsa/mv88e6xxx/mv88e6xxx.ko 2024-03-28T17:41:22.220 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/dummy.ko 2024-03-28T17:41:22.220 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/eql.ko 2024-03-28T17:41:22.220 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet 2024-03-28T17:41:22.220 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/3com 2024-03-28T17:41:22.220 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/3com/3c574_cs.ko 2024-03-28T17:41:22.220 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/3com/3c589_cs.ko 2024-03-28T17:41:22.220 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/3com/3c59x.ko 2024-03-28T17:41:22.220 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/3com/typhoon.ko 2024-03-28T17:41:22.220 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/8390 2024-03-28T17:41:22.220 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/8390/8390.ko 2024-03-28T17:41:22.220 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/8390/axnet_cs.ko 2024-03-28T17:41:22.221 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/8390/ne2k-pci.ko 2024-03-28T17:41:22.221 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/8390/pcnet_cs.ko 2024-03-28T17:41:22.221 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/adaptec 2024-03-28T17:41:22.221 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/adaptec/starfire.ko 2024-03-28T17:41:22.221 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/agere 2024-03-28T17:41:22.221 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/agere/et131x.ko 2024-03-28T17:41:22.221 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/alteon 2024-03-28T17:41:22.221 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/alteon/acenic.ko 2024-03-28T17:41:22.221 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/altera 2024-03-28T17:41:22.221 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/altera/altera_tse.ko 2024-03-28T17:41:22.221 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/amd 2024-03-28T17:41:22.221 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/amd/amd8111e.ko 2024-03-28T17:41:22.221 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/amd/nmclan_cs.ko 2024-03-28T17:41:22.221 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/amd/pcnet32.ko 2024-03-28T17:41:22.221 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/atheros 2024-03-28T17:41:22.221 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/atheros/alx 2024-03-28T17:41:22.222 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/atheros/alx/alx.ko 2024-03-28T17:41:22.222 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/atheros/atl1c 2024-03-28T17:41:22.222 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/atheros/atl1c/atl1c.ko 2024-03-28T17:41:22.222 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/atheros/atl1e 2024-03-28T17:41:22.222 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/atheros/atl1e/atl1e.ko 2024-03-28T17:41:22.222 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/atheros/atlx 2024-03-28T17:41:22.222 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/atheros/atlx/atl1.ko 2024-03-28T17:41:22.222 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/atheros/atlx/atl2.ko 2024-03-28T17:41:22.222 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/broadcom 2024-03-28T17:41:22.222 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/broadcom/b44.ko 2024-03-28T17:41:22.222 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/broadcom/bnx2.ko 2024-03-28T17:41:22.222 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/broadcom/bnx2x 2024-03-28T17:41:22.222 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/broadcom/bnx2x/bnx2x.ko 2024-03-28T17:41:22.222 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/broadcom/cnic.ko 2024-03-28T17:41:22.222 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/broadcom/genet 2024-03-28T17:41:22.222 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/broadcom/genet/genet.ko 2024-03-28T17:41:22.223 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/broadcom/tg3.ko 2024-03-28T17:41:22.223 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/brocade 2024-03-28T17:41:22.223 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/brocade/bna 2024-03-28T17:41:22.223 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/brocade/bna/bna.ko 2024-03-28T17:41:22.223 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/chelsio 2024-03-28T17:41:22.223 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/chelsio/cxgb 2024-03-28T17:41:22.223 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/chelsio/cxgb/cxgb.ko 2024-03-28T17:41:22.223 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/chelsio/cxgb3 2024-03-28T17:41:22.223 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/chelsio/cxgb3/cxgb3.ko 2024-03-28T17:41:22.223 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/chelsio/cxgb4 2024-03-28T17:41:22.223 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/chelsio/cxgb4/cxgb4.ko 2024-03-28T17:41:22.223 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/chelsio/cxgb4vf 2024-03-28T17:41:22.223 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/chelsio/cxgb4vf/cxgb4vf.ko 2024-03-28T17:41:22.223 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/chelsio/libcxgb 2024-03-28T17:41:22.223 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/chelsio/libcxgb/libcxgb.ko 2024-03-28T17:41:22.223 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/cisco 2024-03-28T17:41:22.224 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/cisco/enic 2024-03-28T17:41:22.224 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/cisco/enic/enic.ko 2024-03-28T17:41:22.224 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/dec 2024-03-28T17:41:22.224 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/dec/tulip 2024-03-28T17:41:22.224 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/dec/tulip/de2104x.ko 2024-03-28T17:41:22.224 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/dec/tulip/dmfe.ko 2024-03-28T17:41:22.224 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/dec/tulip/tulip.ko 2024-03-28T17:41:22.224 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/dec/tulip/uli526x.ko 2024-03-28T17:41:22.224 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/dec/tulip/winbond-840.ko 2024-03-28T17:41:22.224 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/dec/tulip/xircom_cb.ko 2024-03-28T17:41:22.224 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/dlink 2024-03-28T17:41:22.224 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/dlink/dl2k.ko 2024-03-28T17:41:22.224 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/dlink/sundance.ko 2024-03-28T17:41:22.224 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/dnet.ko 2024-03-28T17:41:22.224 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/emulex 2024-03-28T17:41:22.224 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/emulex/benet 2024-03-28T17:41:22.224 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/emulex/benet/be2net.ko 2024-03-28T17:41:22.225 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/ethoc.ko 2024-03-28T17:41:22.225 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/fealnx.ko 2024-03-28T17:41:22.225 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/intel 2024-03-28T17:41:22.225 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/intel/e100.ko 2024-03-28T17:41:22.225 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/intel/e1000 2024-03-28T17:41:22.225 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/intel/e1000/e1000.ko 2024-03-28T17:41:22.225 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/intel/e1000e 2024-03-28T17:41:22.225 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/intel/e1000e/e1000e.ko 2024-03-28T17:41:22.225 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/intel/fm10k 2024-03-28T17:41:22.225 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/intel/fm10k/fm10k.ko 2024-03-28T17:41:22.225 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/intel/i40e 2024-03-28T17:41:22.225 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/intel/i40e/i40e.ko 2024-03-28T17:41:22.225 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/intel/iavf 2024-03-28T17:41:22.225 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/intel/iavf/iavf.ko 2024-03-28T17:41:22.225 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/intel/igb 2024-03-28T17:41:22.225 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/intel/igb/igb.ko 2024-03-28T17:41:22.226 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/intel/igbvf 2024-03-28T17:41:22.226 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/intel/igbvf/igbvf.ko 2024-03-28T17:41:22.226 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/intel/ixgbe 2024-03-28T17:41:22.226 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/intel/ixgbe/ixgbe.ko 2024-03-28T17:41:22.226 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/intel/ixgbevf 2024-03-28T17:41:22.226 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/intel/ixgbevf/ixgbevf.ko 2024-03-28T17:41:22.226 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/jme.ko 2024-03-28T17:41:22.226 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/marvell 2024-03-28T17:41:22.226 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/marvell/mvmdio.ko 2024-03-28T17:41:22.226 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/marvell/skge.ko 2024-03-28T17:41:22.226 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/marvell/sky2.ko 2024-03-28T17:41:22.226 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/mellanox 2024-03-28T17:41:22.226 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/mellanox/mlx4 2024-03-28T17:41:22.226 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_core.ko 2024-03-28T17:41:22.226 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_en.ko 2024-03-28T17:41:22.227 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/mellanox/mlx5 2024-03-28T17:41:22.227 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/mellanox/mlx5/core 2024-03-28T17:41:22.227 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/mellanox/mlx5/core/mlx5_core.ko 2024-03-28T17:41:22.227 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/micrel 2024-03-28T17:41:22.227 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/micrel/ksz884x.ko 2024-03-28T17:41:22.227 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/myricom 2024-03-28T17:41:22.227 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/myricom/myri10ge 2024-03-28T17:41:22.227 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/myricom/myri10ge/myri10ge.ko 2024-03-28T17:41:22.227 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/natsemi 2024-03-28T17:41:22.227 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/natsemi/natsemi.ko 2024-03-28T17:41:22.227 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/natsemi/ns83820.ko 2024-03-28T17:41:22.227 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/neterion 2024-03-28T17:41:22.227 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/neterion/s2io.ko 2024-03-28T17:41:22.227 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/nvidia 2024-03-28T17:41:22.227 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/nvidia/forcedeth.ko 2024-03-28T17:41:22.227 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/packetengines 2024-03-28T17:41:22.227 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/packetengines/hamachi.ko 2024-03-28T17:41:22.228 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/packetengines/yellowfin.ko 2024-03-28T17:41:22.228 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/qlogic 2024-03-28T17:41:22.228 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/qlogic/netxen 2024-03-28T17:41:22.228 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/qlogic/netxen/netxen_nic.ko 2024-03-28T17:41:22.228 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/qlogic/qla3xxx.ko 2024-03-28T17:41:22.228 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/qlogic/qlcnic 2024-03-28T17:41:22.228 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/qlogic/qlcnic/qlcnic.ko 2024-03-28T17:41:22.228 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/rdc 2024-03-28T17:41:22.228 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/rdc/r6040.ko 2024-03-28T17:41:22.228 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/realtek 2024-03-28T17:41:22.228 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/realtek/8139cp.ko 2024-03-28T17:41:22.228 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/realtek/8139too.ko 2024-03-28T17:41:22.228 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/realtek/atp.ko 2024-03-28T17:41:22.228 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/realtek/r8169.ko 2024-03-28T17:41:22.228 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/rocker 2024-03-28T17:41:22.228 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/rocker/rocker.ko 2024-03-28T17:41:22.229 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/sfc 2024-03-28T17:41:22.229 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/sfc/sfc.ko 2024-03-28T17:41:22.229 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/silan 2024-03-28T17:41:22.229 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/silan/sc92031.ko 2024-03-28T17:41:22.229 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/sis 2024-03-28T17:41:22.229 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/sis/sis190.ko 2024-03-28T17:41:22.229 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/sis/sis900.ko 2024-03-28T17:41:22.229 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/smsc 2024-03-28T17:41:22.229 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/smsc/epic100.ko 2024-03-28T17:41:22.229 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/smsc/smc91c92_cs.ko 2024-03-28T17:41:22.229 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/smsc/smsc911x.ko 2024-03-28T17:41:22.229 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/smsc/smsc9420.ko 2024-03-28T17:41:22.229 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/stmicro 2024-03-28T17:41:22.229 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/stmicro/stmmac 2024-03-28T17:41:22.229 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/stmicro/stmmac/dwmac-intel.ko 2024-03-28T17:41:22.229 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/stmicro/stmmac/stmmac.ko 2024-03-28T17:41:22.230 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/sun 2024-03-28T17:41:22.230 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/sun/cassini.ko 2024-03-28T17:41:22.230 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/sun/niu.ko 2024-03-28T17:41:22.230 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/sun/sungem.ko 2024-03-28T17:41:22.230 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/sun/sunhme.ko 2024-03-28T17:41:22.230 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/tehuti 2024-03-28T17:41:22.230 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/tehuti/tehuti.ko 2024-03-28T17:41:22.230 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/ti 2024-03-28T17:41:22.230 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/ti/tlan.ko 2024-03-28T17:41:22.230 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/via 2024-03-28T17:41:22.230 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/via/via-rhine.ko 2024-03-28T17:41:22.230 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/via/via-velocity.ko 2024-03-28T17:41:22.230 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/wiznet 2024-03-28T17:41:22.230 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/wiznet/w5100.ko 2024-03-28T17:41:22.230 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/wiznet/w5300.ko 2024-03-28T17:41:22.230 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/xircom 2024-03-28T17:41:22.231 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/xircom/xirc2ps_cs.ko 2024-03-28T17:41:22.231 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/geneve.ko 2024-03-28T17:41:22.231 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/hyperv 2024-03-28T17:41:22.231 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/hyperv/hv_netvsc.ko 2024-03-28T17:41:22.231 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ieee802154 2024-03-28T17:41:22.231 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ieee802154/fakelb.ko 2024-03-28T17:41:22.231 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ifb.ko 2024-03-28T17:41:22.231 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ipvlan 2024-03-28T17:41:22.231 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ipvlan/ipvlan.ko 2024-03-28T17:41:22.231 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/macvlan.ko 2024-03-28T17:41:22.231 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/macvtap.ko 2024-03-28T17:41:22.231 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/mdio 2024-03-28T17:41:22.231 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/mdio.ko 2024-03-28T17:41:22.231 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/mdio/mdio-bcm-unimac.ko 2024-03-28T17:41:22.231 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/mdio/mdio-bitbang.ko 2024-03-28T17:41:22.231 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/mdio/mdio-regmap.ko 2024-03-28T17:41:22.232 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/mii.ko 2024-03-28T17:41:22.232 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/net_failover.ko 2024-03-28T17:41:22.232 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/netconsole.ko 2024-03-28T17:41:22.232 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/nlmon.ko 2024-03-28T17:41:22.232 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/pcs 2024-03-28T17:41:22.232 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/pcs/pcs-lynx.ko 2024-03-28T17:41:22.232 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/pcs/pcs_xpcs.ko 2024-03-28T17:41:22.232 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/phy 2024-03-28T17:41:22.232 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/phy/amd.ko 2024-03-28T17:41:22.232 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/phy/bcm-phy-lib.ko 2024-03-28T17:41:22.232 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/phy/bcm-phy-ptp.ko 2024-03-28T17:41:22.232 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/phy/bcm7xxx.ko 2024-03-28T17:41:22.232 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/phy/bcm87xx.ko 2024-03-28T17:41:22.232 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/phy/broadcom.ko 2024-03-28T17:41:22.232 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/phy/cicada.ko 2024-03-28T17:41:22.232 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/phy/davicom.ko 2024-03-28T17:41:22.233 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/phy/dp83640.ko 2024-03-28T17:41:22.233 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/phy/et1011c.ko 2024-03-28T17:41:22.233 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/phy/icplus.ko 2024-03-28T17:41:22.233 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/phy/lxt.ko 2024-03-28T17:41:22.233 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/phy/marvell.ko 2024-03-28T17:41:22.233 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/phy/micrel.ko 2024-03-28T17:41:22.233 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/phy/national.ko 2024-03-28T17:41:22.233 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/phy/phylink.ko 2024-03-28T17:41:22.233 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/phy/qsemi.ko 2024-03-28T17:41:22.233 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/phy/realtek.ko 2024-03-28T17:41:22.233 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/phy/smsc.ko 2024-03-28T17:41:22.233 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/phy/ste10Xp.ko 2024-03-28T17:41:22.233 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/phy/vitesse.ko 2024-03-28T17:41:22.233 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ppp 2024-03-28T17:41:22.233 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ppp/bsd_comp.ko 2024-03-28T17:41:22.233 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ppp/ppp_async.ko 2024-03-28T17:41:22.234 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ppp/ppp_deflate.ko 2024-03-28T17:41:22.234 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ppp/ppp_generic.ko 2024-03-28T17:41:22.234 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ppp/ppp_mppe.ko 2024-03-28T17:41:22.234 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ppp/ppp_synctty.ko 2024-03-28T17:41:22.234 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ppp/pppoe.ko 2024-03-28T17:41:22.234 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ppp/pppox.ko 2024-03-28T17:41:22.234 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ppp/pptp.ko 2024-03-28T17:41:22.234 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/slip 2024-03-28T17:41:22.234 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/slip/slhc.ko 2024-03-28T17:41:22.234 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/slip/slip.ko 2024-03-28T17:41:22.234 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/sungem_phy.ko 2024-03-28T17:41:22.234 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/tap.ko 2024-03-28T17:41:22.234 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/team 2024-03-28T17:41:22.235 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/team/team.ko 2024-03-28T17:41:22.235 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/team/team_mode_activebackup.ko 2024-03-28T17:41:22.235 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/team/team_mode_broadcast.ko 2024-03-28T17:41:22.235 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/team/team_mode_loadbalance.ko 2024-03-28T17:41:22.235 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/team/team_mode_random.ko 2024-03-28T17:41:22.235 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/team/team_mode_roundrobin.ko 2024-03-28T17:41:22.235 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/tun.ko 2024-03-28T17:41:22.235 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/veth.ko 2024-03-28T17:41:22.235 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/virtio_net.ko 2024-03-28T17:41:22.235 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/vmxnet3 2024-03-28T17:41:22.235 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/vmxnet3/vmxnet3.ko 2024-03-28T17:41:22.235 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/vxlan 2024-03-28T17:41:22.235 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/vxlan/vxlan.ko 2024-03-28T17:41:22.235 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/xen-netback 2024-03-28T17:41:22.235 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/xen-netback/xen-netback.ko 2024-03-28T17:41:22.235 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/xen-netfront.ko 2024-03-28T17:41:22.236 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ntb 2024-03-28T17:41:22.236 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ntb/ntb.ko 2024-03-28T17:41:22.236 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/nvme 2024-03-28T17:41:22.236 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/nvme/host 2024-03-28T17:41:22.236 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/nvme/host/nvme-core.ko 2024-03-28T17:41:22.236 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/nvme/host/nvme.ko 2024-03-28T17:41:22.236 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/parport 2024-03-28T17:41:22.236 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/parport/parport.ko 2024-03-28T17:41:22.236 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/parport/parport_cs.ko 2024-03-28T17:41:22.236 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/parport/parport_pc.ko 2024-03-28T17:41:22.236 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/parport/parport_serial.ko 2024-03-28T17:41:22.236 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/pci 2024-03-28T17:41:22.236 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/pci/hotplug 2024-03-28T17:41:22.236 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/pci/hotplug/acpiphp_ibm.ko 2024-03-28T17:41:22.236 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/pci/pcie 2024-03-28T17:41:22.236 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/pci/pcie/aer_inject.ko 2024-03-28T17:41:22.237 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/pci/xen-pcifront.ko 2024-03-28T17:41:22.237 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/pcmcia 2024-03-28T17:41:22.237 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/pcmcia/i82092.ko 2024-03-28T17:41:22.237 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/pcmcia/pd6729.ko 2024-03-28T17:41:22.237 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/pcmcia/yenta_socket.ko 2024-03-28T17:41:22.237 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/pinctrl 2024-03-28T17:41:22.237 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/pinctrl/intel 2024-03-28T17:41:22.237 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/pinctrl/intel/pinctrl-cherryview.ko 2024-03-28T17:41:22.237 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform 2024-03-28T17:41:22.237 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86 2024-03-28T17:41:22.237 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/acer-wmi.ko 2024-03-28T17:41:22.237 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/acerhdf.ko 2024-03-28T17:41:22.237 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/apple-gmux.ko 2024-03-28T17:41:22.237 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/asus-laptop.ko 2024-03-28T17:41:22.237 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/asus-nb-wmi.ko 2024-03-28T17:41:22.237 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/asus-wmi.ko 2024-03-28T17:41:22.238 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/classmate-laptop.ko 2024-03-28T17:41:22.238 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/eeepc-laptop.ko 2024-03-28T17:41:22.238 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/eeepc-wmi.ko 2024-03-28T17:41:22.238 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/fujitsu-laptop.ko 2024-03-28T17:41:22.238 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/fujitsu-tablet.ko 2024-03-28T17:41:22.238 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/hdaps.ko 2024-03-28T17:41:22.238 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/intel 2024-03-28T17:41:22.238 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/intel/intel-rst.ko 2024-03-28T17:41:22.238 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/intel_ips.ko 2024-03-28T17:41:22.238 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/msi-wmi.ko 2024-03-28T17:41:22.238 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/mxm-wmi.ko 2024-03-28T17:41:22.238 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/panasonic-laptop.ko 2024-03-28T17:41:22.238 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/samsung-laptop.ko 2024-03-28T17:41:22.238 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/samsung-q10.ko 2024-03-28T17:41:22.238 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/topstar-laptop.ko 2024-03-28T17:41:22.239 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/toshiba_bluetooth.ko 2024-03-28T17:41:22.239 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/toshiba_haps.ko 2024-03-28T17:41:22.239 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/wmi-bmof.ko 2024-03-28T17:41:22.239 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/wmi.ko 2024-03-28T17:41:22.239 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/powercap 2024-03-28T17:41:22.239 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/powercap/intel_rapl_common.ko 2024-03-28T17:41:22.239 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/powercap/intel_rapl_msr.ko 2024-03-28T17:41:22.239 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/pps 2024-03-28T17:41:22.239 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/pps/clients 2024-03-28T17:41:22.239 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/pps/clients/pps-gpio.ko 2024-03-28T17:41:22.239 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/pps/clients/pps-ldisc.ko 2024-03-28T17:41:22.239 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/pps/clients/pps_parport.ko 2024-03-28T17:41:22.239 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/pps/pps_core.ko 2024-03-28T17:41:22.239 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ptp 2024-03-28T17:41:22.240 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ptp/ptp.ko 2024-03-28T17:41:22.240 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ptp/ptp_kvm.ko 2024-03-28T17:41:22.240 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc 2024-03-28T17:41:22.240 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-bq32k.ko 2024-03-28T17:41:22.240 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-ds1286.ko 2024-03-28T17:41:22.240 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-ds1307.ko 2024-03-28T17:41:22.240 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-ds1374.ko 2024-03-28T17:41:22.240 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-ds1511.ko 2024-03-28T17:41:22.240 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-ds1553.ko 2024-03-28T17:41:22.240 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-ds1672.ko 2024-03-28T17:41:22.240 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-ds1742.ko 2024-03-28T17:41:22.240 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-ds2404.ko 2024-03-28T17:41:22.240 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-ds3232.ko 2024-03-28T17:41:22.240 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-em3027.ko 2024-03-28T17:41:22.240 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-fm3130.ko 2024-03-28T17:41:22.240 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-isl12022.ko 2024-03-28T17:41:22.241 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-isl1208.ko 2024-03-28T17:41:22.241 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-m41t80.ko 2024-03-28T17:41:22.241 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-m48t35.ko 2024-03-28T17:41:22.241 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-m48t59.ko 2024-03-28T17:41:22.241 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-max6900.ko 2024-03-28T17:41:22.241 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-msm6242.ko 2024-03-28T17:41:22.241 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-pcf2127.ko 2024-03-28T17:41:22.241 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-pcf85063.ko 2024-03-28T17:41:22.241 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-pcf8523.ko 2024-03-28T17:41:22.241 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-pcf8563.ko 2024-03-28T17:41:22.241 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-pcf8583.ko 2024-03-28T17:41:22.241 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-rp5c01.ko 2024-03-28T17:41:22.241 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-rs5c372.ko 2024-03-28T17:41:22.241 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-rv3029c2.ko 2024-03-28T17:41:22.241 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-rx8025.ko 2024-03-28T17:41:22.241 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-rx8581.ko 2024-03-28T17:41:22.242 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-stk17ta8.ko 2024-03-28T17:41:22.242 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-x1205.ko 2024-03-28T17:41:22.242 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi 2024-03-28T17:41:22.242 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/3w-9xxx.ko 2024-03-28T17:41:22.242 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/3w-sas.ko 2024-03-28T17:41:22.242 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/3w-xxxx.ko 2024-03-28T17:41:22.242 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/BusLogic.ko 2024-03-28T17:41:22.242 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/a100u2w.ko 2024-03-28T17:41:22.242 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/aacraid 2024-03-28T17:41:22.242 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/aacraid/aacraid.ko 2024-03-28T17:41:22.242 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/advansys.ko 2024-03-28T17:41:22.242 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/aic7xxx 2024-03-28T17:41:22.242 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/aic7xxx/aic79xx.ko 2024-03-28T17:41:22.242 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/aic7xxx/aic7xxx.ko 2024-03-28T17:41:22.243 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/am53c974.ko 2024-03-28T17:41:22.243 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/arcmsr 2024-03-28T17:41:22.243 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/arcmsr/arcmsr.ko 2024-03-28T17:41:22.243 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/atp870u.ko 2024-03-28T17:41:22.243 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/be2iscsi 2024-03-28T17:41:22.243 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/be2iscsi/be2iscsi.ko 2024-03-28T17:41:22.243 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/bfa 2024-03-28T17:41:22.243 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/bfa/bfa.ko 2024-03-28T17:41:22.243 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/bnx2fc 2024-03-28T17:41:22.243 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/bnx2fc/bnx2fc.ko 2024-03-28T17:41:22.243 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/bnx2i 2024-03-28T17:41:22.243 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/bnx2i/bnx2i.ko 2024-03-28T17:41:22.243 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/ch.ko 2024-03-28T17:41:22.243 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/csiostor 2024-03-28T17:41:22.243 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/csiostor/csiostor.ko 2024-03-28T17:41:22.244 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/cxgbi 2024-03-28T17:41:22.244 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/cxgbi/cxgb3i 2024-03-28T17:41:22.244 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/cxgbi/cxgb3i/cxgb3i.ko 2024-03-28T17:41:22.244 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/cxgbi/cxgb4i 2024-03-28T17:41:22.244 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/cxgbi/cxgb4i/cxgb4i.ko 2024-03-28T17:41:22.244 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/cxgbi/libcxgbi.ko 2024-03-28T17:41:22.244 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/dc395x.ko 2024-03-28T17:41:22.244 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/device_handler 2024-03-28T17:41:22.244 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/device_handler/scsi_dh_alua.ko 2024-03-28T17:41:22.244 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/device_handler/scsi_dh_emc.ko 2024-03-28T17:41:22.244 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/device_handler/scsi_dh_hp_sw.ko 2024-03-28T17:41:22.244 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/device_handler/scsi_dh_rdac.ko 2024-03-28T17:41:22.244 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/dmx3191d.ko 2024-03-28T17:41:22.244 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/esas2r 2024-03-28T17:41:22.244 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/esas2r/esas2r.ko 2024-03-28T17:41:22.244 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/esp_scsi.ko 2024-03-28T17:41:22.245 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/fcoe 2024-03-28T17:41:22.245 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/fcoe/fcoe.ko 2024-03-28T17:41:22.245 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/fcoe/libfcoe.ko 2024-03-28T17:41:22.245 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/fnic 2024-03-28T17:41:22.245 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/fnic/fnic.ko 2024-03-28T17:41:22.245 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/hpsa.ko 2024-03-28T17:41:22.245 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/hptiop.ko 2024-03-28T17:41:22.245 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/hv_storvsc.ko 2024-03-28T17:41:22.245 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/initio.ko 2024-03-28T17:41:22.245 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/ipr.ko 2024-03-28T17:41:22.245 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/ips.ko 2024-03-28T17:41:22.245 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/isci 2024-03-28T17:41:22.245 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/isci/isci.ko 2024-03-28T17:41:22.245 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/iscsi_boot_sysfs.ko 2024-03-28T17:41:22.245 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/iscsi_tcp.ko 2024-03-28T17:41:22.245 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/libfc 2024-03-28T17:41:22.246 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/libfc/libfc.ko 2024-03-28T17:41:22.246 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/libiscsi.ko 2024-03-28T17:41:22.246 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/libiscsi_tcp.ko 2024-03-28T17:41:22.246 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/libsas 2024-03-28T17:41:22.246 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/libsas/libsas.ko 2024-03-28T17:41:22.246 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/lpfc 2024-03-28T17:41:22.246 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/lpfc/lpfc.ko 2024-03-28T17:41:22.246 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/megaraid 2024-03-28T17:41:22.246 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/megaraid.ko 2024-03-28T17:41:22.246 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/megaraid/megaraid_mbox.ko 2024-03-28T17:41:22.246 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/megaraid/megaraid_mm.ko 2024-03-28T17:41:22.246 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/megaraid/megaraid_sas.ko 2024-03-28T17:41:22.246 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/mpt3sas 2024-03-28T17:41:22.246 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/mpt3sas/mpt3sas.ko 2024-03-28T17:41:22.246 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/mvsas 2024-03-28T17:41:22.247 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/mvsas/mvsas.ko 2024-03-28T17:41:22.247 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/mvumi.ko 2024-03-28T17:41:22.247 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/pm8001 2024-03-28T17:41:22.247 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/pm8001/pm80xx.ko 2024-03-28T17:41:22.247 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/pmcraid.ko 2024-03-28T17:41:22.247 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/qla1280.ko 2024-03-28T17:41:22.247 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/qla2xxx 2024-03-28T17:41:22.247 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/qla2xxx/qla2xxx.ko 2024-03-28T17:41:22.247 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/qla2xxx/tcm_qla2xxx.ko 2024-03-28T17:41:22.247 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/qla4xxx 2024-03-28T17:41:22.247 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/qla4xxx/qla4xxx.ko 2024-03-28T17:41:22.247 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/raid_class.ko 2024-03-28T17:41:22.247 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/scsi_debug.ko 2024-03-28T17:41:22.247 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/scsi_transport_fc.ko 2024-03-28T17:41:22.247 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/scsi_transport_iscsi.ko 2024-03-28T17:41:22.248 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/scsi_transport_sas.ko 2024-03-28T17:41:22.248 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/scsi_transport_spi.ko 2024-03-28T17:41:22.248 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/scsi_transport_srp.ko 2024-03-28T17:41:22.248 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/ses.ko 2024-03-28T17:41:22.248 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/st.ko 2024-03-28T17:41:22.248 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/stex.ko 2024-03-28T17:41:22.248 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/sym53c8xx_2 2024-03-28T17:41:22.248 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/sym53c8xx_2/sym53c8xx.ko 2024-03-28T17:41:22.248 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/virtio_scsi.ko 2024-03-28T17:41:22.248 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/vmw_pvscsi.ko 2024-03-28T17:41:22.248 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/wd719x.ko 2024-03-28T17:41:22.248 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ssb 2024-03-28T17:41:22.248 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ssb/ssb.ko 2024-03-28T17:41:22.248 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/target 2024-03-28T17:41:22.248 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/target/iscsi 2024-03-28T17:41:22.249 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/target/iscsi/iscsi_target_mod.ko 2024-03-28T17:41:22.249 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/target/loopback 2024-03-28T17:41:22.249 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/target/loopback/tcm_loop.ko 2024-03-28T17:41:22.249 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/target/target_core_file.ko 2024-03-28T17:41:22.249 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/target/target_core_iblock.ko 2024-03-28T17:41:22.249 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/target/target_core_mod.ko 2024-03-28T17:41:22.249 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/target/target_core_pscsi.ko 2024-03-28T17:41:22.249 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/target/target_core_user.ko 2024-03-28T17:41:22.249 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/target/tcm_fc 2024-03-28T17:41:22.249 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/target/tcm_fc/tcm_fc.ko 2024-03-28T17:41:22.249 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/thermal 2024-03-28T17:41:22.249 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/thermal/intel 2024-03-28T17:41:22.249 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/thermal/intel/int340x_thermal 2024-03-28T17:41:22.249 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/thermal/intel/int340x_thermal/acpi_thermal_rel.ko 2024-03-28T17:41:22.249 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/thermal/intel/int340x_thermal/int3400_thermal.ko 2024-03-28T17:41:22.250 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/thermal/intel/int340x_thermal/int3401_thermal.ko 2024-03-28T17:41:22.250 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/thermal/intel/int340x_thermal/int3402_thermal.ko 2024-03-28T17:41:22.250 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/thermal/intel/int340x_thermal/int3403_thermal.ko 2024-03-28T17:41:22.250 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/thermal/intel/int340x_thermal/int340x_thermal_zone.ko 2024-03-28T17:41:22.250 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device.ko 2024-03-28T17:41:22.250 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci.ko 2024-03-28T17:41:22.250 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci_legacy.ko 2024-03-28T17:41:22.250 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_mbox.ko 2024-03-28T17:41:22.250 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_power_floor.ko 2024-03-28T17:41:22.250 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rapl.ko 2024-03-28T17:41:22.250 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rfim.ko 2024-03-28T17:41:22.250 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_wt_hint.ko 2024-03-28T17:41:22.250 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_wt_req.ko 2024-03-28T17:41:22.251 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/thermal/intel/intel_soc_dts_iosf.ko 2024-03-28T17:41:22.251 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/thermal/intel/intel_soc_dts_thermal.ko 2024-03-28T17:41:22.251 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/thermal/intel/x86_pkg_temp_thermal.ko 2024-03-28T17:41:22.251 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/tty 2024-03-28T17:41:22.251 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/tty/ipwireless 2024-03-28T17:41:22.251 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/tty/ipwireless/ipwireless.ko 2024-03-28T17:41:22.251 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/tty/n_gsm.ko 2024-03-28T17:41:22.251 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/tty/n_hdlc.ko 2024-03-28T17:41:22.251 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/tty/nozomi.ko 2024-03-28T17:41:22.251 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/tty/serial 2024-03-28T17:41:22.251 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/tty/serial/8250 2024-03-28T17:41:22.251 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/tty/serial/8250/serial_cs.ko 2024-03-28T17:41:22.251 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/tty/serial/arc_uart.ko 2024-03-28T17:41:22.251 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/tty/serial/jsm 2024-03-28T17:41:22.251 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/tty/serial/jsm/jsm.ko 2024-03-28T17:41:22.252 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ufs 2024-03-28T17:41:22.252 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ufs/core 2024-03-28T17:41:22.252 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ufs/core/ufshcd-core.ko 2024-03-28T17:41:22.252 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ufs/host 2024-03-28T17:41:22.252 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ufs/host/ufshcd-pci.ko 2024-03-28T17:41:22.252 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/uio 2024-03-28T17:41:22.252 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/uio/uio.ko 2024-03-28T17:41:22.252 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/uio/uio_aec.ko 2024-03-28T17:41:22.252 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/uio/uio_cif.ko 2024-03-28T17:41:22.252 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/uio/uio_pci_generic.ko 2024-03-28T17:41:22.252 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/uio/uio_sercos3.ko 2024-03-28T17:41:22.252 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb 2024-03-28T17:41:22.252 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/class 2024-03-28T17:41:22.252 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/class/cdc-acm.ko 2024-03-28T17:41:22.252 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/class/cdc-wdm.ko 2024-03-28T17:41:22.252 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/class/usblp.ko 2024-03-28T17:41:22.253 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/class/usbtmc.ko 2024-03-28T17:41:22.253 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/host 2024-03-28T17:41:22.253 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/host/sl811-hcd.ko 2024-03-28T17:41:22.253 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/image 2024-03-28T17:41:22.253 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/image/mdc800.ko 2024-03-28T17:41:22.253 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/image/microtek.ko 2024-03-28T17:41:22.253 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/misc 2024-03-28T17:41:22.253 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/misc/adutux.ko 2024-03-28T17:41:22.253 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/misc/appledisplay.ko 2024-03-28T17:41:22.253 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/misc/emi26.ko 2024-03-28T17:41:22.253 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/misc/emi62.ko 2024-03-28T17:41:22.253 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/misc/ezusb.ko 2024-03-28T17:41:22.253 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/misc/idmouse.ko 2024-03-28T17:41:22.254 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/misc/iowarrior.ko 2024-03-28T17:41:22.254 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/misc/isight_firmware.ko 2024-03-28T17:41:22.254 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/misc/ldusb.ko 2024-03-28T17:41:22.254 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/misc/legousbtower.ko 2024-03-28T17:41:22.254 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/misc/sisusbvga 2024-03-28T17:41:22.254 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/misc/sisusbvga/sisusbvga.ko 2024-03-28T17:41:22.254 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/misc/trancevibrator.ko 2024-03-28T17:41:22.254 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/misc/usb3503.ko 2024-03-28T17:41:22.254 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/misc/usblcd.ko 2024-03-28T17:41:22.254 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/misc/usbsevseg.ko 2024-03-28T17:41:22.254 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/misc/uss720.ko 2024-03-28T17:41:22.254 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/misc/yurex.ko 2024-03-28T17:41:22.254 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/phy 2024-03-28T17:41:22.254 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/phy/phy-generic.ko 2024-03-28T17:41:22.254 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial 2024-03-28T17:41:22.255 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/aircable.ko 2024-03-28T17:41:22.255 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/ark3116.ko 2024-03-28T17:41:22.255 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/belkin_sa.ko 2024-03-28T17:41:22.255 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/ch341.ko 2024-03-28T17:41:22.255 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/cp210x.ko 2024-03-28T17:41:22.255 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/cyberjack.ko 2024-03-28T17:41:22.255 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/cypress_m8.ko 2024-03-28T17:41:22.255 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/digi_acceleport.ko 2024-03-28T17:41:22.255 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/empeg.ko 2024-03-28T17:41:22.255 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/ftdi_sio.ko 2024-03-28T17:41:22.255 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/garmin_gps.ko 2024-03-28T17:41:22.255 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/io_edgeport.ko 2024-03-28T17:41:22.255 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/io_ti.ko 2024-03-28T17:41:22.255 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/ipaq.ko 2024-03-28T17:41:22.255 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/ipw.ko 2024-03-28T17:41:22.256 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/ir-usb.ko 2024-03-28T17:41:22.256 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/iuu_phoenix.ko 2024-03-28T17:41:22.256 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/keyspan.ko 2024-03-28T17:41:22.256 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/keyspan_pda.ko 2024-03-28T17:41:22.256 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/kl5kusb105.ko 2024-03-28T17:41:22.256 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/kobil_sct.ko 2024-03-28T17:41:22.256 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/mct_u232.ko 2024-03-28T17:41:22.256 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/mos7720.ko 2024-03-28T17:41:22.256 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/mos7840.ko 2024-03-28T17:41:22.256 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/navman.ko 2024-03-28T17:41:22.256 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/omninet.ko 2024-03-28T17:41:22.256 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/opticon.ko 2024-03-28T17:41:22.256 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/option.ko 2024-03-28T17:41:22.256 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/oti6858.ko 2024-03-28T17:41:22.256 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/pl2303.ko 2024-03-28T17:41:22.256 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/qcaux.ko 2024-03-28T17:41:22.257 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/qcserial.ko 2024-03-28T17:41:22.257 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/quatech2.ko 2024-03-28T17:41:22.257 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/safe_serial.ko 2024-03-28T17:41:22.257 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/sierra.ko 2024-03-28T17:41:22.257 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/spcp8x5.ko 2024-03-28T17:41:22.257 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/ssu100.ko 2024-03-28T17:41:22.257 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/symbolserial.ko 2024-03-28T17:41:22.257 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/ti_usb_3410_5052.ko 2024-03-28T17:41:22.257 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/usb-serial-simple.ko 2024-03-28T17:41:22.257 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/usb_debug.ko 2024-03-28T17:41:22.257 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/usb_wwan.ko 2024-03-28T17:41:22.257 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/visor.ko 2024-03-28T17:41:22.257 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/whiteheat.ko 2024-03-28T17:41:22.257 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/xsens_mt.ko 2024-03-28T17:41:22.257 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/storage 2024-03-28T17:41:22.257 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/storage/uas.ko 2024-03-28T17:41:22.258 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/storage/ums-alauda.ko 2024-03-28T17:41:22.258 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/storage/ums-cypress.ko 2024-03-28T17:41:22.258 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/storage/ums-datafab.ko 2024-03-28T17:41:22.258 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/storage/ums-eneub6250.ko 2024-03-28T17:41:22.258 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/storage/ums-freecom.ko 2024-03-28T17:41:22.258 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/storage/ums-isd200.ko 2024-03-28T17:41:22.258 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/storage/ums-jumpshot.ko 2024-03-28T17:41:22.258 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/storage/ums-karma.ko 2024-03-28T17:41:22.258 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/storage/ums-onetouch.ko 2024-03-28T17:41:22.258 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/storage/ums-realtek.ko 2024-03-28T17:41:22.258 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/storage/ums-sddr09.ko 2024-03-28T17:41:22.258 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/storage/ums-sddr55.ko 2024-03-28T17:41:22.258 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/storage/ums-usbat.ko 2024-03-28T17:41:22.258 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/storage/usb-storage.ko 2024-03-28T17:41:22.258 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/usbip 2024-03-28T17:41:22.259 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/usbip/usbip-core.ko 2024-03-28T17:41:22.259 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/usbip/usbip-host.ko 2024-03-28T17:41:22.259 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/usbip/vhci-hcd.ko 2024-03-28T17:41:22.259 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/vfio 2024-03-28T17:41:22.259 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/vfio/pci 2024-03-28T17:41:22.259 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/vfio/pci/vfio-pci-core.ko 2024-03-28T17:41:22.259 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/vfio/pci/vfio-pci.ko 2024-03-28T17:41:22.259 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/vfio/vfio.ko 2024-03-28T17:41:22.259 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/vfio/vfio_iommu_type1.ko 2024-03-28T17:41:22.259 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/vhost 2024-03-28T17:41:22.259 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/vhost/vhost.ko 2024-03-28T17:41:22.259 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/vhost/vhost_iotlb.ko 2024-03-28T17:41:22.259 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/vhost/vhost_net.ko 2024-03-28T17:41:22.259 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/vhost/vhost_scsi.ko 2024-03-28T17:41:22.259 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/video 2024-03-28T17:41:22.259 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/video/backlight 2024-03-28T17:41:22.260 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/video/backlight/apple_bl.ko 2024-03-28T17:41:22.260 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/video/backlight/lcd.ko 2024-03-28T17:41:22.260 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/video/backlight/platform_lcd.ko 2024-03-28T17:41:22.260 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/virtio 2024-03-28T17:41:22.260 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/virtio/virtio.ko 2024-03-28T17:41:22.260 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/virtio/virtio_balloon.ko 2024-03-28T17:41:22.260 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/virtio/virtio_mmio.ko 2024-03-28T17:41:22.260 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/virtio/virtio_pci.ko 2024-03-28T17:41:22.260 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/virtio/virtio_pci_legacy_dev.ko 2024-03-28T17:41:22.260 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/virtio/virtio_pci_modern_dev.ko 2024-03-28T17:41:22.260 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/virtio/virtio_ring.ko 2024-03-28T17:41:22.260 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/w1 2024-03-28T17:41:22.260 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/w1/masters 2024-03-28T17:41:22.260 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/w1/masters/ds2482.ko 2024-03-28T17:41:22.260 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/w1/masters/ds2490.ko 2024-03-28T17:41:22.260 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/w1/slaves 2024-03-28T17:41:22.261 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/w1/slaves/w1_ds2406.ko 2024-03-28T17:41:22.261 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/w1/slaves/w1_ds2408.ko 2024-03-28T17:41:22.261 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/w1/slaves/w1_ds2413.ko 2024-03-28T17:41:22.261 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/w1/slaves/w1_ds2423.ko 2024-03-28T17:41:22.261 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/w1/slaves/w1_ds2431.ko 2024-03-28T17:41:22.261 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/w1/slaves/w1_ds2433.ko 2024-03-28T17:41:22.261 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/w1/slaves/w1_ds2780.ko 2024-03-28T17:41:22.261 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/w1/slaves/w1_ds2781.ko 2024-03-28T17:41:22.261 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/w1/slaves/w1_ds28e04.ko 2024-03-28T17:41:22.261 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/w1/slaves/w1_smem.ko 2024-03-28T17:41:22.261 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/w1/slaves/w1_therm.ko 2024-03-28T17:41:22.261 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/w1/wire.ko 2024-03-28T17:41:22.261 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog 2024-03-28T17:41:22.261 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/alim1535_wdt.ko 2024-03-28T17:41:22.261 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/alim7101_wdt.ko 2024-03-28T17:41:22.261 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/f71808e_wdt.ko 2024-03-28T17:41:22.261 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/hpwdt.ko 2024-03-28T17:41:22.262 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/i6300esb.ko 2024-03-28T17:41:22.262 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/iTCO_vendor_support.ko 2024-03-28T17:41:22.262 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/iTCO_wdt.ko 2024-03-28T17:41:22.262 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/ib700wdt.ko 2024-03-28T17:41:22.262 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/ibmasr.ko 2024-03-28T17:41:22.262 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/ie6xx_wdt.ko 2024-03-28T17:41:22.262 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/it8712f_wdt.ko 2024-03-28T17:41:22.262 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/it87_wdt.ko 2024-03-28T17:41:22.262 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/machzwd.ko 2024-03-28T17:41:22.262 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/nv_tco.ko 2024-03-28T17:41:22.262 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/pcwd_pci.ko 2024-03-28T17:41:22.262 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/pcwd_usb.ko 2024-03-28T17:41:22.262 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/sbc_fitpc2_wdt.ko 2024-03-28T17:41:22.262 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/sch311x_wdt.ko 2024-03-28T17:41:22.263 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/softdog.ko 2024-03-28T17:41:22.263 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/sp5100_tco.ko 2024-03-28T17:41:22.263 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/via_wdt.ko 2024-03-28T17:41:22.263 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/w83627hf_wdt.ko 2024-03-28T17:41:22.263 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/w83877f_wdt.ko 2024-03-28T17:41:22.263 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/w83977f_wdt.ko 2024-03-28T17:41:22.263 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/wdt_pci.ko 2024-03-28T17:41:22.263 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/xen_wdt.ko 2024-03-28T17:41:22.263 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/xen 2024-03-28T17:41:22.263 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/xen/xen-acpi-processor.ko 2024-03-28T17:41:22.263 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/xen/xen-evtchn.ko 2024-03-28T17:41:22.263 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/xen/xen-gntalloc.ko 2024-03-28T17:41:22.263 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/xen/xen-gntdev.ko 2024-03-28T17:41:22.263 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/xen/xen-pciback 2024-03-28T17:41:22.264 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/xen/xen-pciback/xen-pciback.ko 2024-03-28T17:41:22.264 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/xen/xen-privcmd.ko 2024-03-28T17:41:22.264 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/xen/xenfs 2024-03-28T17:41:22.264 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/xen/xenfs/xenfs.ko 2024-03-28T17:41:22.264 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs 2024-03-28T17:41:22.264 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/9p 2024-03-28T17:41:22.264 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/9p/9p.ko 2024-03-28T17:41:22.264 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/affs 2024-03-28T17:41:22.264 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/affs/affs.ko 2024-03-28T17:41:22.264 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/befs 2024-03-28T17:41:22.264 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/befs/befs.ko 2024-03-28T17:41:22.264 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/binfmt_misc.ko 2024-03-28T17:41:22.264 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/btrfs 2024-03-28T17:41:22.264 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/btrfs/btrfs.ko 2024-03-28T17:41:22.264 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/cachefiles 2024-03-28T17:41:22.265 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/cachefiles/cachefiles.ko 2024-03-28T17:41:22.265 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/ceph 2024-03-28T17:41:22.265 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/ceph/ceph.ko 2024-03-28T17:41:22.265 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/coda 2024-03-28T17:41:22.265 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/coda/coda.ko 2024-03-28T17:41:22.265 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/cramfs 2024-03-28T17:41:22.265 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/cramfs/cramfs.ko 2024-03-28T17:41:22.265 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/dlm 2024-03-28T17:41:22.265 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/dlm/dlm.ko 2024-03-28T17:41:22.265 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/ecryptfs 2024-03-28T17:41:22.265 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/ecryptfs/ecryptfs.ko 2024-03-28T17:41:22.265 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/f2fs 2024-03-28T17:41:22.265 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/f2fs/f2fs.ko 2024-03-28T17:41:22.265 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/fat 2024-03-28T17:41:22.265 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/fat/fat.ko 2024-03-28T17:41:22.265 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/fat/msdos.ko 2024-03-28T17:41:22.266 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/fat/vfat.ko 2024-03-28T17:41:22.266 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/fuse 2024-03-28T17:41:22.266 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/fuse/cuse.ko 2024-03-28T17:41:22.266 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/fuse/fuse.ko 2024-03-28T17:41:22.266 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/gfs2 2024-03-28T17:41:22.266 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/gfs2/gfs2.ko 2024-03-28T17:41:22.266 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/hfs 2024-03-28T17:41:22.266 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/hfs/hfs.ko 2024-03-28T17:41:22.266 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/hfsplus 2024-03-28T17:41:22.266 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/hfsplus/hfsplus.ko 2024-03-28T17:41:22.266 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/isofs 2024-03-28T17:41:22.266 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/isofs/isofs.ko 2024-03-28T17:41:22.266 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/jfs 2024-03-28T17:41:22.266 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/jfs/jfs.ko 2024-03-28T17:41:22.266 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/lockd 2024-03-28T17:41:22.267 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/lockd/lockd.ko 2024-03-28T17:41:22.267 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/minix 2024-03-28T17:41:22.267 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/minix/minix.ko 2024-03-28T17:41:22.267 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/netfs 2024-03-28T17:41:22.267 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/netfs/netfs.ko 2024-03-28T17:41:22.267 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nfs 2024-03-28T17:41:22.267 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nfs/blocklayout 2024-03-28T17:41:22.267 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nfs/blocklayout/blocklayoutdriver.ko 2024-03-28T17:41:22.267 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nfs/filelayout 2024-03-28T17:41:22.267 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nfs/filelayout/nfs_layout_nfsv41_files.ko 2024-03-28T17:41:22.267 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nfs/flexfilelayout 2024-03-28T17:41:22.267 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nfs/flexfilelayout/nfs_layout_flexfiles.ko 2024-03-28T17:41:22.267 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nfs/nfs.ko 2024-03-28T17:41:22.267 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nfs/nfsv3.ko 2024-03-28T17:41:22.268 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nfs/nfsv4.ko 2024-03-28T17:41:22.268 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nfs_common 2024-03-28T17:41:22.268 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nfs_common/grace.ko 2024-03-28T17:41:22.268 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nfs_common/nfs_acl.ko 2024-03-28T17:41:22.268 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nfsd 2024-03-28T17:41:22.268 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nfsd/nfsd.ko 2024-03-28T17:41:22.268 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nilfs2 2024-03-28T17:41:22.268 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nilfs2/nilfs2.ko 2024-03-28T17:41:22.268 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls 2024-03-28T17:41:22.268 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/mac-celtic.ko 2024-03-28T17:41:22.268 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/mac-centeuro.ko 2024-03-28T17:41:22.268 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/mac-croatian.ko 2024-03-28T17:41:22.268 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/mac-cyrillic.ko 2024-03-28T17:41:22.268 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/mac-gaelic.ko 2024-03-28T17:41:22.268 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/mac-greek.ko 2024-03-28T17:41:22.269 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/mac-iceland.ko 2024-03-28T17:41:22.269 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/mac-inuit.ko 2024-03-28T17:41:22.269 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/mac-roman.ko 2024-03-28T17:41:22.269 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/mac-romanian.ko 2024-03-28T17:41:22.269 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/mac-turkish.ko 2024-03-28T17:41:22.269 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_cp1250.ko 2024-03-28T17:41:22.269 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_cp1251.ko 2024-03-28T17:41:22.269 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_cp1255.ko 2024-03-28T17:41:22.269 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_cp737.ko 2024-03-28T17:41:22.269 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_cp775.ko 2024-03-28T17:41:22.269 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_cp850.ko 2024-03-28T17:41:22.269 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_cp852.ko 2024-03-28T17:41:22.269 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_cp855.ko 2024-03-28T17:41:22.269 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_cp857.ko 2024-03-28T17:41:22.269 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_cp860.ko 2024-03-28T17:41:22.270 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_cp861.ko 2024-03-28T17:41:22.270 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_cp862.ko 2024-03-28T17:41:22.270 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_cp863.ko 2024-03-28T17:41:22.270 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_cp864.ko 2024-03-28T17:41:22.270 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_cp865.ko 2024-03-28T17:41:22.270 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_cp866.ko 2024-03-28T17:41:22.270 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_cp869.ko 2024-03-28T17:41:22.270 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_cp874.ko 2024-03-28T17:41:22.270 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_cp932.ko 2024-03-28T17:41:22.270 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_cp936.ko 2024-03-28T17:41:22.270 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_cp949.ko 2024-03-28T17:41:22.270 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_cp950.ko 2024-03-28T17:41:22.270 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_euc-jp.ko 2024-03-28T17:41:22.270 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_iso8859-1.ko 2024-03-28T17:41:22.270 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_iso8859-13.ko 2024-03-28T17:41:22.271 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_iso8859-14.ko 2024-03-28T17:41:22.271 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_iso8859-15.ko 2024-03-28T17:41:22.271 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_iso8859-2.ko 2024-03-28T17:41:22.271 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_iso8859-3.ko 2024-03-28T17:41:22.271 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_iso8859-4.ko 2024-03-28T17:41:22.271 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_iso8859-5.ko 2024-03-28T17:41:22.271 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_iso8859-6.ko 2024-03-28T17:41:22.271 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_iso8859-7.ko 2024-03-28T17:41:22.271 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_iso8859-9.ko 2024-03-28T17:41:22.271 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_koi8-r.ko 2024-03-28T17:41:22.271 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_koi8-ru.ko 2024-03-28T17:41:22.271 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_koi8-u.ko 2024-03-28T17:41:22.271 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_ucs2_utils.ko 2024-03-28T17:41:22.271 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_utf8.ko 2024-03-28T17:41:22.271 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/ocfs2 2024-03-28T17:41:22.271 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/ocfs2/cluster 2024-03-28T17:41:22.272 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/ocfs2/cluster/ocfs2_nodemanager.ko 2024-03-28T17:41:22.272 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/ocfs2/dlm 2024-03-28T17:41:22.272 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/ocfs2/dlm/ocfs2_dlm.ko 2024-03-28T17:41:22.272 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/ocfs2/dlmfs 2024-03-28T17:41:22.272 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/ocfs2/dlmfs/ocfs2_dlmfs.ko 2024-03-28T17:41:22.272 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/ocfs2/ocfs2.ko 2024-03-28T17:41:22.272 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/ocfs2/ocfs2_stack_o2cb.ko 2024-03-28T17:41:22.272 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/ocfs2/ocfs2_stack_user.ko 2024-03-28T17:41:22.272 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/ocfs2/ocfs2_stackglue.ko 2024-03-28T17:41:22.272 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/overlayfs 2024-03-28T17:41:22.272 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/overlayfs/overlay.ko 2024-03-28T17:41:22.272 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/pstore 2024-03-28T17:41:22.272 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/pstore/ramoops.ko 2024-03-28T17:41:22.272 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/reiserfs 2024-03-28T17:41:22.272 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/reiserfs/reiserfs.ko 2024-03-28T17:41:22.273 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/romfs 2024-03-28T17:41:22.273 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/romfs/romfs.ko 2024-03-28T17:41:22.273 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/smb 2024-03-28T17:41:22.273 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/smb/client 2024-03-28T17:41:22.273 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/smb/client/cifs.ko 2024-03-28T17:41:22.273 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/smb/common 2024-03-28T17:41:22.273 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/smb/common/cifs_arc4.ko 2024-03-28T17:41:22.273 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/smb/common/cifs_md4.ko 2024-03-28T17:41:22.273 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/squashfs 2024-03-28T17:41:22.273 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/squashfs/squashfs.ko 2024-03-28T17:41:22.273 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/sysv 2024-03-28T17:41:22.273 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/sysv/sysv.ko 2024-03-28T17:41:22.273 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/udf 2024-03-28T17:41:22.274 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/udf/udf.ko 2024-03-28T17:41:22.274 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/ufs 2024-03-28T17:41:22.274 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/ufs/ufs.ko 2024-03-28T17:41:22.274 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/xfs 2024-03-28T17:41:22.274 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/xfs/xfs.ko 2024-03-28T17:41:22.274 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/kernel 2024-03-28T17:41:22.274 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/kernel/rcu 2024-03-28T17:41:22.274 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/kernel/rcu/rcutorture.ko 2024-03-28T17:41:22.274 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/kernel/torture.ko 2024-03-28T17:41:22.274 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/kernel/trace 2024-03-28T17:41:22.274 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/kernel/trace/ring_buffer_benchmark.ko 2024-03-28T17:41:22.274 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/lib 2024-03-28T17:41:22.274 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/asn1_encoder.ko 2024-03-28T17:41:22.275 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/crc-itu-t.ko 2024-03-28T17:41:22.275 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/crc8.ko 2024-03-28T17:41:22.275 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/crypto 2024-03-28T17:41:22.275 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/crypto/libarc4.ko 2024-03-28T17:41:22.275 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/crypto/libdes.ko 2024-03-28T17:41:22.275 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/libcrc32c.ko 2024-03-28T17:41:22.275 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/lru_cache.ko 2024-03-28T17:41:22.275 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/lz4 2024-03-28T17:41:22.275 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/lz4/lz4_compress.ko 2024-03-28T17:41:22.275 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/lz4/lz4hc_compress.ko 2024-03-28T17:41:22.275 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/math 2024-03-28T17:41:22.275 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/math/cordic.ko 2024-03-28T17:41:22.275 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/raid6 2024-03-28T17:41:22.275 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/raid6/raid6_pq.ko 2024-03-28T17:41:22.276 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/reed_solomon 2024-03-28T17:41:22.276 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/reed_solomon/reed_solomon.ko 2024-03-28T17:41:22.276 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/ts_bm.ko 2024-03-28T17:41:22.276 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/ts_fsm.ko 2024-03-28T17:41:22.276 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/ts_kmp.ko 2024-03-28T17:41:22.276 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/zstd 2024-03-28T17:41:22.276 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/zstd/zstd_compress.ko 2024-03-28T17:41:22.276 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/mm 2024-03-28T17:41:22.276 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/mm/hwpoison-inject.ko 2024-03-28T17:41:22.276 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net 2024-03-28T17:41:22.276 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/6lowpan 2024-03-28T17:41:22.276 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/6lowpan/6lowpan.ko 2024-03-28T17:41:22.276 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/6lowpan/nhc_dest.ko 2024-03-28T17:41:22.276 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/6lowpan/nhc_fragment.ko 2024-03-28T17:41:22.277 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/6lowpan/nhc_hop.ko 2024-03-28T17:41:22.277 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/6lowpan/nhc_ipv6.ko 2024-03-28T17:41:22.277 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/6lowpan/nhc_mobility.ko 2024-03-28T17:41:22.277 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/6lowpan/nhc_routing.ko 2024-03-28T17:41:22.277 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/6lowpan/nhc_udp.ko 2024-03-28T17:41:22.277 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/802 2024-03-28T17:41:22.277 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/802/garp.ko 2024-03-28T17:41:22.277 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/802/mrp.ko 2024-03-28T17:41:22.277 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/802/p8022.ko 2024-03-28T17:41:22.277 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/802/psnap.ko 2024-03-28T17:41:22.277 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/802/stp.ko 2024-03-28T17:41:22.277 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/8021q 2024-03-28T17:41:22.277 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/8021q/8021q.ko 2024-03-28T17:41:22.277 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/9p 2024-03-28T17:41:22.278 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/9p/9pnet.ko 2024-03-28T17:41:22.278 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/9p/9pnet_fd.ko 2024-03-28T17:41:22.278 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/9p/9pnet_rdma.ko 2024-03-28T17:41:22.278 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/9p/9pnet_virtio.ko 2024-03-28T17:41:22.278 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/batman-adv 2024-03-28T17:41:22.278 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/batman-adv/batman-adv.ko 2024-03-28T17:41:22.278 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge 2024-03-28T17:41:22.278 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/br_netfilter.ko 2024-03-28T17:41:22.278 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/bridge.ko 2024-03-28T17:41:22.278 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter 2024-03-28T17:41:22.278 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/ebt_802_3.ko 2024-03-28T17:41:22.278 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/ebt_among.ko 2024-03-28T17:41:22.278 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/ebt_arp.ko 2024-03-28T17:41:22.279 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/ebt_arpreply.ko 2024-03-28T17:41:22.279 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/ebt_dnat.ko 2024-03-28T17:41:22.279 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/ebt_ip.ko 2024-03-28T17:41:22.279 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/ebt_ip6.ko 2024-03-28T17:41:22.279 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/ebt_limit.ko 2024-03-28T17:41:22.279 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/ebt_log.ko 2024-03-28T17:41:22.279 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/ebt_mark.ko 2024-03-28T17:41:22.279 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/ebt_mark_m.ko 2024-03-28T17:41:22.279 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/ebt_nflog.ko 2024-03-28T17:41:22.279 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/ebt_pkttype.ko 2024-03-28T17:41:22.279 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/ebt_redirect.ko 2024-03-28T17:41:22.279 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/ebt_snat.ko 2024-03-28T17:41:22.279 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/ebt_stp.ko 2024-03-28T17:41:22.279 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/ebt_vlan.ko 2024-03-28T17:41:22.279 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/ebtable_broute.ko 2024-03-28T17:41:22.280 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/ebtable_filter.ko 2024-03-28T17:41:22.280 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/ebtable_nat.ko 2024-03-28T17:41:22.280 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/ebtables.ko 2024-03-28T17:41:22.280 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/nf_conntrack_bridge.ko 2024-03-28T17:41:22.280 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/nft_meta_bridge.ko 2024-03-28T17:41:22.280 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/nft_reject_bridge.ko 2024-03-28T17:41:22.280 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ceph 2024-03-28T17:41:22.280 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ceph/libceph.ko 2024-03-28T17:41:22.280 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/core 2024-03-28T17:41:22.280 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/core/failover.ko 2024-03-28T17:41:22.280 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/core/pktgen.ko 2024-03-28T17:41:22.280 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/dccp 2024-03-28T17:41:22.280 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/dccp/dccp.ko 2024-03-28T17:41:22.280 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/dccp/dccp_diag.ko 2024-03-28T17:41:22.280 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/dccp/dccp_ipv4.ko 2024-03-28T17:41:22.280 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/dccp/dccp_ipv6.ko 2024-03-28T17:41:22.281 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/dns_resolver 2024-03-28T17:41:22.281 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/dns_resolver/dns_resolver.ko 2024-03-28T17:41:22.281 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/dsa 2024-03-28T17:41:22.281 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/dsa/dsa_core.ko 2024-03-28T17:41:22.281 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/dsa/tag_brcm.ko 2024-03-28T17:41:22.281 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/dsa/tag_dsa.ko 2024-03-28T17:41:22.281 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/dsa/tag_none.ko 2024-03-28T17:41:22.281 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/dsa/tag_trailer.ko 2024-03-28T17:41:22.281 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ieee802154 2024-03-28T17:41:22.281 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ieee802154/6lowpan 2024-03-28T17:41:22.281 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ieee802154/6lowpan/ieee802154_6lowpan.ko 2024-03-28T17:41:22.281 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ieee802154/ieee802154.ko 2024-03-28T17:41:22.281 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ieee802154/ieee802154_socket.ko 2024-03-28T17:41:22.281 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4 2024-03-28T17:41:22.281 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/ah4.ko 2024-03-28T17:41:22.281 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/esp4.ko 2024-03-28T17:41:22.282 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/fou.ko 2024-03-28T17:41:22.282 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/gre.ko 2024-03-28T17:41:22.282 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/inet_diag.ko 2024-03-28T17:41:22.282 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/ip_gre.ko 2024-03-28T17:41:22.282 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/ip_tunnel.ko 2024-03-28T17:41:22.282 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/ip_vti.ko 2024-03-28T17:41:22.282 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/ipcomp.ko 2024-03-28T17:41:22.282 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/ipip.ko 2024-03-28T17:41:22.282 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter 2024-03-28T17:41:22.282 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter/arp_tables.ko 2024-03-28T17:41:22.282 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter/arpt_mangle.ko 2024-03-28T17:41:22.282 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter/arptable_filter.ko 2024-03-28T17:41:22.282 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter/ipt_ECN.ko 2024-03-28T17:41:22.282 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter/ipt_SYNPROXY.ko 2024-03-28T17:41:22.283 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter/ipt_ah.ko 2024-03-28T17:41:22.283 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter/ipt_rpfilter.ko 2024-03-28T17:41:22.283 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter/iptable_mangle.ko 2024-03-28T17:41:22.283 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter/iptable_nat.ko 2024-03-28T17:41:22.283 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter/iptable_raw.ko 2024-03-28T17:41:22.283 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter/iptable_security.ko 2024-03-28T17:41:22.283 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter/nf_defrag_ipv4.ko 2024-03-28T17:41:22.283 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter/nf_dup_ipv4.ko 2024-03-28T17:41:22.283 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter/nf_nat_h323.ko 2024-03-28T17:41:22.283 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter/nf_nat_pptp.ko 2024-03-28T17:41:22.283 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter/nf_nat_snmp_basic.ko 2024-03-28T17:41:22.283 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter/nf_socket_ipv4.ko 2024-03-28T17:41:22.283 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter/nf_tproxy_ipv4.ko 2024-03-28T17:41:22.283 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter/nft_dup_ipv4.ko 2024-03-28T17:41:22.283 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter/nft_fib_ipv4.ko 2024-03-28T17:41:22.283 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter/nft_reject_ipv4.ko 2024-03-28T17:41:22.284 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/tcp_bic.ko 2024-03-28T17:41:22.284 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/tcp_dctcp.ko 2024-03-28T17:41:22.284 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/tcp_diag.ko 2024-03-28T17:41:22.284 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/tcp_highspeed.ko 2024-03-28T17:41:22.284 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/tcp_htcp.ko 2024-03-28T17:41:22.284 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/tcp_hybla.ko 2024-03-28T17:41:22.284 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/tcp_illinois.ko 2024-03-28T17:41:22.284 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/tcp_lp.ko 2024-03-28T17:41:22.284 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/tcp_scalable.ko 2024-03-28T17:41:22.284 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/tcp_vegas.ko 2024-03-28T17:41:22.284 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/tcp_veno.ko 2024-03-28T17:41:22.284 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/tcp_westwood.ko 2024-03-28T17:41:22.284 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/tcp_yeah.ko 2024-03-28T17:41:22.284 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/tunnel4.ko 2024-03-28T17:41:22.284 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/udp_diag.ko 2024-03-28T17:41:22.284 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/udp_tunnel.ko 2024-03-28T17:41:22.285 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/xfrm4_tunnel.ko 2024-03-28T17:41:22.285 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6 2024-03-28T17:41:22.285 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/ah6.ko 2024-03-28T17:41:22.285 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/esp6.ko 2024-03-28T17:41:22.285 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/fou6.ko 2024-03-28T17:41:22.285 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/ip6_tunnel.ko 2024-03-28T17:41:22.285 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/ip6_udp_tunnel.ko 2024-03-28T17:41:22.285 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/ip6_vti.ko 2024-03-28T17:41:22.285 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/ipcomp6.ko 2024-03-28T17:41:22.285 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter 2024-03-28T17:41:22.285 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/ip6_tables.ko 2024-03-28T17:41:22.285 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/ip6t_REJECT.ko 2024-03-28T17:41:22.285 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/ip6t_SYNPROXY.ko 2024-03-28T17:41:22.285 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/ip6t_ah.ko 2024-03-28T17:41:22.286 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/ip6t_eui64.ko 2024-03-28T17:41:22.286 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/ip6t_frag.ko 2024-03-28T17:41:22.286 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/ip6t_hbh.ko 2024-03-28T17:41:22.286 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/ip6t_ipv6header.ko 2024-03-28T17:41:22.286 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/ip6t_mh.ko 2024-03-28T17:41:22.286 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/ip6t_rpfilter.ko 2024-03-28T17:41:22.286 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/ip6t_rt.ko 2024-03-28T17:41:22.286 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/ip6table_filter.ko 2024-03-28T17:41:22.286 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/ip6table_mangle.ko 2024-03-28T17:41:22.286 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/ip6table_nat.ko 2024-03-28T17:41:22.286 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/ip6table_raw.ko 2024-03-28T17:41:22.286 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/ip6table_security.ko 2024-03-28T17:41:22.286 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/nf_defrag_ipv6.ko 2024-03-28T17:41:22.286 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/nf_dup_ipv6.ko 2024-03-28T17:41:22.286 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/nf_reject_ipv6.ko 2024-03-28T17:41:22.287 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/nf_socket_ipv6.ko 2024-03-28T17:41:22.287 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/nf_tproxy_ipv6.ko 2024-03-28T17:41:22.287 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/nft_dup_ipv6.ko 2024-03-28T17:41:22.287 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/nft_fib_ipv6.ko 2024-03-28T17:41:22.287 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/nft_reject_ipv6.ko 2024-03-28T17:41:22.287 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/sit.ko 2024-03-28T17:41:22.287 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/tunnel6.ko 2024-03-28T17:41:22.287 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/xfrm6_tunnel.ko 2024-03-28T17:41:22.287 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/key 2024-03-28T17:41:22.287 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/key/af_key.ko 2024-03-28T17:41:22.287 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/l2tp 2024-03-28T17:41:22.287 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/l2tp/l2tp_core.ko 2024-03-28T17:41:22.287 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/l2tp/l2tp_debugfs.ko 2024-03-28T17:41:22.287 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/l2tp/l2tp_eth.ko 2024-03-28T17:41:22.287 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/l2tp/l2tp_ip.ko 2024-03-28T17:41:22.288 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/l2tp/l2tp_ip6.ko 2024-03-28T17:41:22.288 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/l2tp/l2tp_netlink.ko 2024-03-28T17:41:22.288 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/l2tp/l2tp_ppp.ko 2024-03-28T17:41:22.288 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/llc 2024-03-28T17:41:22.288 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/llc/llc.ko 2024-03-28T17:41:22.288 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/mac802154 2024-03-28T17:41:22.288 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/mac802154/mac802154.ko 2024-03-28T17:41:22.288 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/mpls 2024-03-28T17:41:22.288 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/mpls/mpls_gso.ko 2024-03-28T17:41:22.288 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter 2024-03-28T17:41:22.288 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipset 2024-03-28T17:41:22.288 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipset/ip_set.ko 2024-03-28T17:41:22.288 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipset/ip_set_bitmap_ip.ko 2024-03-28T17:41:22.288 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipset/ip_set_bitmap_ipmac.ko 2024-03-28T17:41:22.289 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipset/ip_set_bitmap_port.ko 2024-03-28T17:41:22.289 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipset/ip_set_hash_ip.ko 2024-03-28T17:41:22.289 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipset/ip_set_hash_ipmark.ko 2024-03-28T17:41:22.289 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipset/ip_set_hash_ipport.ko 2024-03-28T17:41:22.289 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipset/ip_set_hash_ipportip.ko 2024-03-28T17:41:22.289 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipset/ip_set_hash_ipportnet.ko 2024-03-28T17:41:22.289 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipset/ip_set_hash_mac.ko 2024-03-28T17:41:22.289 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipset/ip_set_hash_net.ko 2024-03-28T17:41:22.289 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipset/ip_set_hash_netiface.ko 2024-03-28T17:41:22.289 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipset/ip_set_hash_netnet.ko 2024-03-28T17:41:22.289 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipset/ip_set_hash_netport.ko 2024-03-28T17:41:22.289 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipset/ip_set_hash_netportnet.ko 2024-03-28T17:41:22.289 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipset/ip_set_list_set.ko 2024-03-28T17:41:22.289 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipvs 2024-03-28T17:41:22.289 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipvs/ip_vs.ko 2024-03-28T17:41:22.290 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipvs/ip_vs_dh.ko 2024-03-28T17:41:22.290 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipvs/ip_vs_fo.ko 2024-03-28T17:41:22.290 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipvs/ip_vs_ftp.ko 2024-03-28T17:41:22.290 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipvs/ip_vs_lblc.ko 2024-03-28T17:41:22.290 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipvs/ip_vs_lblcr.ko 2024-03-28T17:41:22.290 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipvs/ip_vs_lc.ko 2024-03-28T17:41:22.290 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipvs/ip_vs_nq.ko 2024-03-28T17:41:22.290 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipvs/ip_vs_pe_sip.ko 2024-03-28T17:41:22.290 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipvs/ip_vs_rr.ko 2024-03-28T17:41:22.290 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipvs/ip_vs_sed.ko 2024-03-28T17:41:22.290 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipvs/ip_vs_sh.ko 2024-03-28T17:41:22.290 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipvs/ip_vs_wlc.ko 2024-03-28T17:41:22.290 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipvs/ip_vs_wrr.ko 2024-03-28T17:41:22.290 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_conncount.ko 2024-03-28T17:41:22.291 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_conntrack.ko 2024-03-28T17:41:22.291 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_conntrack_amanda.ko 2024-03-28T17:41:22.291 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_conntrack_broadcast.ko 2024-03-28T17:41:22.291 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_conntrack_ftp.ko 2024-03-28T17:41:22.291 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_conntrack_h323.ko 2024-03-28T17:41:22.291 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_conntrack_irc.ko 2024-03-28T17:41:22.291 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_conntrack_netbios_ns.ko 2024-03-28T17:41:22.291 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_conntrack_netlink.ko 2024-03-28T17:41:22.291 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_conntrack_pptp.ko 2024-03-28T17:41:22.291 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_conntrack_sane.ko 2024-03-28T17:41:22.291 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_conntrack_sip.ko 2024-03-28T17:41:22.291 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_conntrack_snmp.ko 2024-03-28T17:41:22.291 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_conntrack_tftp.ko 2024-03-28T17:41:22.291 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_dup_netdev.ko 2024-03-28T17:41:22.291 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_flow_table.ko 2024-03-28T17:41:22.291 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_flow_table_inet.ko 2024-03-28T17:41:22.292 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_log_syslog.ko 2024-03-28T17:41:22.292 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_nat.ko 2024-03-28T17:41:22.292 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_nat_amanda.ko 2024-03-28T17:41:22.292 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_nat_ftp.ko 2024-03-28T17:41:22.292 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_nat_irc.ko 2024-03-28T17:41:22.292 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_nat_sip.ko 2024-03-28T17:41:22.292 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_nat_tftp.ko 2024-03-28T17:41:22.292 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_synproxy_core.ko 2024-03-28T17:41:22.292 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_tables.ko 2024-03-28T17:41:22.292 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nfnetlink.ko 2024-03-28T17:41:22.292 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nfnetlink_acct.ko 2024-03-28T17:41:22.292 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nfnetlink_log.ko 2024-03-28T17:41:22.292 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nfnetlink_osf.ko 2024-03-28T17:41:22.292 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nfnetlink_queue.ko 2024-03-28T17:41:22.292 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nft_chain_nat.ko 2024-03-28T17:41:22.292 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nft_compat.ko 2024-03-28T17:41:22.293 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nft_ct.ko 2024-03-28T17:41:22.293 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nft_dup_netdev.ko 2024-03-28T17:41:22.293 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nft_fib.ko 2024-03-28T17:41:22.293 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nft_fwd_netdev.ko 2024-03-28T17:41:22.293 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nft_hash.ko 2024-03-28T17:41:22.293 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nft_limit.ko 2024-03-28T17:41:22.293 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nft_log.ko 2024-03-28T17:41:22.293 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nft_masq.ko 2024-03-28T17:41:22.293 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nft_nat.ko 2024-03-28T17:41:22.293 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nft_numgen.ko 2024-03-28T17:41:22.293 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nft_queue.ko 2024-03-28T17:41:22.293 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nft_quota.ko 2024-03-28T17:41:22.293 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nft_redir.ko 2024-03-28T17:41:22.293 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nft_reject.ko 2024-03-28T17:41:22.293 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nft_reject_inet.ko 2024-03-28T17:41:22.294 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nft_socket.ko 2024-03-28T17:41:22.294 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nft_tunnel.ko 2024-03-28T17:41:22.294 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_AUDIT.ko 2024-03-28T17:41:22.294 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_CHECKSUM.ko 2024-03-28T17:41:22.294 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_CLASSIFY.ko 2024-03-28T17:41:22.294 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_CONNSECMARK.ko 2024-03-28T17:41:22.294 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_CT.ko 2024-03-28T17:41:22.294 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_DSCP.ko 2024-03-28T17:41:22.294 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_HL.ko 2024-03-28T17:41:22.294 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_HMARK.ko 2024-03-28T17:41:22.294 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_IDLETIMER.ko 2024-03-28T17:41:22.294 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_LED.ko 2024-03-28T17:41:22.294 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_LOG.ko 2024-03-28T17:41:22.294 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_MASQUERADE.ko 2024-03-28T17:41:22.294 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_NETMAP.ko 2024-03-28T17:41:22.295 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_NFLOG.ko 2024-03-28T17:41:22.295 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_NFQUEUE.ko 2024-03-28T17:41:22.295 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_RATEEST.ko 2024-03-28T17:41:22.295 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_REDIRECT.ko 2024-03-28T17:41:22.295 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_SECMARK.ko 2024-03-28T17:41:22.295 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_TCPMSS.ko 2024-03-28T17:41:22.295 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_TCPOPTSTRIP.ko 2024-03-28T17:41:22.295 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_TEE.ko 2024-03-28T17:41:22.295 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_TPROXY.ko 2024-03-28T17:41:22.295 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_TRACE.ko 2024-03-28T17:41:22.295 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_addrtype.ko 2024-03-28T17:41:22.295 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_bpf.ko 2024-03-28T17:41:22.295 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_cgroup.ko 2024-03-28T17:41:22.295 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_cluster.ko 2024-03-28T17:41:22.295 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_comment.ko 2024-03-28T17:41:22.296 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_connbytes.ko 2024-03-28T17:41:22.296 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_connlabel.ko 2024-03-28T17:41:22.296 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_connlimit.ko 2024-03-28T17:41:22.296 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_connmark.ko 2024-03-28T17:41:22.296 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_conntrack.ko 2024-03-28T17:41:22.296 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_cpu.ko 2024-03-28T17:41:22.296 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_dccp.ko 2024-03-28T17:41:22.296 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_devgroup.ko 2024-03-28T17:41:22.296 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_dscp.ko 2024-03-28T17:41:22.296 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_ecn.ko 2024-03-28T17:41:22.296 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_esp.ko 2024-03-28T17:41:22.296 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_hashlimit.ko 2024-03-28T17:41:22.296 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_helper.ko 2024-03-28T17:41:22.296 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_hl.ko 2024-03-28T17:41:22.296 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_ipcomp.ko 2024-03-28T17:41:22.296 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_iprange.ko 2024-03-28T17:41:22.297 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_ipvs.ko 2024-03-28T17:41:22.297 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_l2tp.ko 2024-03-28T17:41:22.297 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_length.ko 2024-03-28T17:41:22.297 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_limit.ko 2024-03-28T17:41:22.297 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_mac.ko 2024-03-28T17:41:22.297 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_mark.ko 2024-03-28T17:41:22.297 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_multiport.ko 2024-03-28T17:41:22.297 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_nat.ko 2024-03-28T17:41:22.297 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_nfacct.ko 2024-03-28T17:41:22.297 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_osf.ko 2024-03-28T17:41:22.297 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_owner.ko 2024-03-28T17:41:22.297 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_physdev.ko 2024-03-28T17:41:22.297 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_pkttype.ko 2024-03-28T17:41:22.297 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_policy.ko 2024-03-28T17:41:22.298 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_quota.ko 2024-03-28T17:41:22.298 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_rateest.ko 2024-03-28T17:41:22.298 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_realm.ko 2024-03-28T17:41:22.298 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_recent.ko 2024-03-28T17:41:22.298 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_sctp.ko 2024-03-28T17:41:22.298 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_set.ko 2024-03-28T17:41:22.298 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_socket.ko 2024-03-28T17:41:22.298 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_state.ko 2024-03-28T17:41:22.298 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_statistic.ko 2024-03-28T17:41:22.298 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_string.ko 2024-03-28T17:41:22.298 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_tcpmss.ko 2024-03-28T17:41:22.298 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_time.ko 2024-03-28T17:41:22.298 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_u32.ko 2024-03-28T17:41:22.298 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netlink 2024-03-28T17:41:22.299 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netlink/netlink_diag.ko 2024-03-28T17:41:22.299 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/nsh 2024-03-28T17:41:22.299 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/nsh/nsh.ko 2024-03-28T17:41:22.299 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/openvswitch 2024-03-28T17:41:22.299 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/openvswitch/openvswitch.ko 2024-03-28T17:41:22.299 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/openvswitch/vport-geneve.ko 2024-03-28T17:41:22.299 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/openvswitch/vport-gre.ko 2024-03-28T17:41:22.299 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/openvswitch/vport-vxlan.ko 2024-03-28T17:41:22.299 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/packet 2024-03-28T17:41:22.299 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/packet/af_packet_diag.ko 2024-03-28T17:41:22.299 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/rds 2024-03-28T17:41:22.299 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/rds/rds.ko 2024-03-28T17:41:22.299 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/rds/rds_rdma.ko 2024-03-28T17:41:22.299 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/rds/rds_tcp.ko 2024-03-28T17:41:22.299 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched 2024-03-28T17:41:22.300 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/act_csum.ko 2024-03-28T17:41:22.300 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/act_gact.ko 2024-03-28T17:41:22.300 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/act_mirred.ko 2024-03-28T17:41:22.300 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/act_nat.ko 2024-03-28T17:41:22.300 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/act_pedit.ko 2024-03-28T17:41:22.300 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/act_police.ko 2024-03-28T17:41:22.300 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/act_simple.ko 2024-03-28T17:41:22.300 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/act_skbedit.ko 2024-03-28T17:41:22.300 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/act_vlan.ko 2024-03-28T17:41:22.300 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/cls_basic.ko 2024-03-28T17:41:22.300 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/cls_bpf.ko 2024-03-28T17:41:22.300 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/cls_flow.ko 2024-03-28T17:41:22.300 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/cls_fw.ko 2024-03-28T17:41:22.300 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/cls_route.ko 2024-03-28T17:41:22.301 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/cls_u32.ko 2024-03-28T17:41:22.301 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/em_cmp.ko 2024-03-28T17:41:22.301 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/em_ipset.ko 2024-03-28T17:41:22.301 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/em_meta.ko 2024-03-28T17:41:22.301 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/em_nbyte.ko 2024-03-28T17:41:22.301 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/em_text.ko 2024-03-28T17:41:22.301 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/em_u32.ko 2024-03-28T17:41:22.301 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/sch_choke.ko 2024-03-28T17:41:22.301 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/sch_codel.ko 2024-03-28T17:41:22.301 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/sch_drr.ko 2024-03-28T17:41:22.301 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/sch_fq.ko 2024-03-28T17:41:22.301 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/sch_gred.ko 2024-03-28T17:41:22.301 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/sch_hfsc.ko 2024-03-28T17:41:22.301 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/sch_hhf.ko 2024-03-28T17:41:22.301 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/sch_htb.ko 2024-03-28T17:41:22.301 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/sch_ingress.ko 2024-03-28T17:41:22.302 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/sch_mqprio.ko 2024-03-28T17:41:22.302 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/sch_mqprio_lib.ko 2024-03-28T17:41:22.302 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/sch_multiq.ko 2024-03-28T17:41:22.302 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/sch_netem.ko 2024-03-28T17:41:22.302 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/sch_pie.ko 2024-03-28T17:41:22.302 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/sch_plug.ko 2024-03-28T17:41:22.302 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/sch_prio.ko 2024-03-28T17:41:22.302 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/sch_qfq.ko 2024-03-28T17:41:22.302 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/sch_red.ko 2024-03-28T17:41:22.302 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/sch_sfb.ko 2024-03-28T17:41:22.302 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/sch_sfq.ko 2024-03-28T17:41:22.302 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/sch_tbf.ko 2024-03-28T17:41:22.302 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/sch_teql.ko 2024-03-28T17:41:22.302 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sctp 2024-03-28T17:41:22.303 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sctp/sctp.ko 2024-03-28T17:41:22.303 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sctp/sctp_diag.ko 2024-03-28T17:41:22.303 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sunrpc 2024-03-28T17:41:22.303 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sunrpc/auth_gss 2024-03-28T17:41:22.303 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sunrpc/auth_gss/auth_rpcgss.ko 2024-03-28T17:41:22.303 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sunrpc/auth_gss/rpcsec_gss_krb5.ko 2024-03-28T17:41:22.303 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sunrpc/sunrpc.ko 2024-03-28T17:41:22.303 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sunrpc/xprtrdma 2024-03-28T17:41:22.303 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sunrpc/xprtrdma/rpcrdma.ko 2024-03-28T17:41:22.303 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/tipc 2024-03-28T17:41:22.303 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/tipc/diag.ko 2024-03-28T17:41:22.303 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/tipc/tipc.ko 2024-03-28T17:41:22.303 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/unix 2024-03-28T17:41:22.303 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/unix/unix_diag.ko 2024-03-28T17:41:22.303 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/vmw_vsock 2024-03-28T17:41:22.304 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/vmw_vsock/vmw_vsock_virtio_transport_common.ko 2024-03-28T17:41:22.304 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/vmw_vsock/vmw_vsock_vmci_transport.ko 2024-03-28T17:41:22.304 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/vmw_vsock/vsock.ko 2024-03-28T17:41:22.304 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/vmw_vsock/vsock_diag.ko 2024-03-28T17:41:22.304 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/vmw_vsock/vsock_loopback.ko 2024-03-28T17:41:22.304 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/xfrm 2024-03-28T17:41:22.304 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/xfrm/xfrm_ipcomp.ko 2024-03-28T17:41:22.304 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/security 2024-03-28T17:41:22.304 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/security/keys 2024-03-28T17:41:22.304 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/security/keys/encrypted-keys 2024-03-28T17:41:22.304 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/security/keys/encrypted-keys/encrypted-keys.ko 2024-03-28T17:41:22.304 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/security/keys/trusted-keys 2024-03-28T17:41:22.304 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/security/keys/trusted-keys/trusted.ko 2024-03-28T17:41:22.304 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/virt 2024-03-28T17:41:22.304 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/virt/lib 2024-03-28T17:41:22.304 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/virt/lib/irqbypass.ko 2024-03-28T17:41:22.305 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/modules.alias 2024-03-28T17:41:22.305 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/modules.alias.bin 2024-03-28T17:41:22.305 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/modules.builtin 2024-03-28T17:41:22.305 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/modules.builtin.alias.bin 2024-03-28T17:41:22.305 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/modules.builtin.bin 2024-03-28T17:41:22.305 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/modules.builtin.modinfo 2024-03-28T17:41:22.305 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/modules.dep 2024-03-28T17:41:22.305 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/modules.dep.bin 2024-03-28T17:41:22.305 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/modules.devname 2024-03-28T17:41:22.305 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/modules.order 2024-03-28T17:41:22.305 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/modules.softdep 2024-03-28T17:41:22.305 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/modules.symbols 2024-03-28T17:41:22.305 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/modules.symbols.bin 2024-03-28T17:41:22.305 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-g1df0c7e8a235/vmlinuz 2024-03-28T17:41:22.309 DEBUG:teuthology.task.kernel:get_image_version: 6.8.0-g1df0c7e8a235 2024-03-28T17:41:22.309 DEBUG:teuthology.orchestra.run.smithi134:> rpm --scripts -qp /tmp/kernel.x86_64.rpm 2024-03-28T17:41:22.331 INFO:teuthology.orchestra.run.smithi134.stdout:postinstall scriptlet (using /bin/sh): 2024-03-28T17:41:22.331 INFO:teuthology.orchestra.run.smithi134.stdout:if [ -x /usr/bin/kernel-install ]; then 2024-03-28T17:41:22.331 INFO:teuthology.orchestra.run.smithi134.stdout: /usr/bin/kernel-install add 6.8.0-g1df0c7e8a235 /lib/modules/6.8.0-g1df0c7e8a235/vmlinuz 2024-03-28T17:41:22.331 INFO:teuthology.orchestra.run.smithi134.stdout:fi 2024-03-28T17:41:22.331 INFO:teuthology.orchestra.run.smithi134.stdout:for file in vmlinuz System.map config; do 2024-03-28T17:41:22.331 INFO:teuthology.orchestra.run.smithi134.stdout: if ! cmp --silent "/lib/modules/6.8.0-g1df0c7e8a235/${file}" "/boot/${file}-6.8.0-g1df0c7e8a235"; then 2024-03-28T17:41:22.331 INFO:teuthology.orchestra.run.smithi134.stdout: cp "/lib/modules/6.8.0-g1df0c7e8a235/${file}" "/boot/${file}-6.8.0-g1df0c7e8a235" 2024-03-28T17:41:22.331 INFO:teuthology.orchestra.run.smithi134.stdout: fi 2024-03-28T17:41:22.331 INFO:teuthology.orchestra.run.smithi134.stdout:done 2024-03-28T17:41:22.331 INFO:teuthology.orchestra.run.smithi134.stdout:preuninstall scriptlet (using /bin/sh): 2024-03-28T17:41:22.331 INFO:teuthology.orchestra.run.smithi134.stdout:if [ -x /sbin/new-kernel-pkg ]; then 2024-03-28T17:41:22.331 INFO:teuthology.orchestra.run.smithi134.stdout:new-kernel-pkg --remove 6.8.0-g1df0c7e8a235 --rminitrd --initrdfile=/boot/initramfs-6.8.0-g1df0c7e8a235.img 2024-03-28T17:41:22.332 INFO:teuthology.orchestra.run.smithi134.stdout:elif [ -x /usr/bin/kernel-install ]; then 2024-03-28T17:41:22.332 INFO:teuthology.orchestra.run.smithi134.stdout:kernel-install remove 6.8.0-g1df0c7e8a235 2024-03-28T17:41:22.332 INFO:teuthology.orchestra.run.smithi134.stdout:fi 2024-03-28T17:41:22.332 INFO:teuthology.orchestra.run.smithi134.stdout:postuninstall scriptlet (using /bin/sh): 2024-03-28T17:41:22.332 INFO:teuthology.orchestra.run.smithi134.stdout:if [ -x /sbin/update-bootloader ]; then 2024-03-28T17:41:22.332 INFO:teuthology.orchestra.run.smithi134.stdout:/sbin/update-bootloader --remove 6.8.0-g1df0c7e8a235 2024-03-28T17:41:22.332 INFO:teuthology.orchestra.run.smithi134.stdout:fi 2024-03-28T17:41:22.333 DEBUG:teuthology.orchestra.run.smithi134:> sudo rpm -qi grub2-tools 2024-03-28T17:41:22.407 INFO:teuthology.orchestra.run.smithi134.stdout:Name : grub2-tools 2024-03-28T17:41:22.407 INFO:teuthology.orchestra.run.smithi134.stdout:Epoch : 1 2024-03-28T17:41:22.407 INFO:teuthology.orchestra.run.smithi134.stdout:Version : 2.06 2024-03-28T17:41:22.407 INFO:teuthology.orchestra.run.smithi134.stdout:Release : 68.el9 2024-03-28T17:41:22.407 INFO:teuthology.orchestra.run.smithi134.stdout:Architecture: x86_64 2024-03-28T17:41:22.407 INFO:teuthology.orchestra.run.smithi134.stdout:Install Date: Fri 01 Sep 2023 09:14:30 PM UTC 2024-03-28T17:41:22.407 INFO:teuthology.orchestra.run.smithi134.stdout:Group : Unspecified 2024-03-28T17:41:22.407 INFO:teuthology.orchestra.run.smithi134.stdout:Size : 8264254 2024-03-28T17:41:22.407 INFO:teuthology.orchestra.run.smithi134.stdout:License : GPLv3+ 2024-03-28T17:41:22.407 INFO:teuthology.orchestra.run.smithi134.stdout:Signature : RSA/SHA256, Mon 07 Aug 2023 01:57:16 PM UTC, Key ID 05b555b38483c65d 2024-03-28T17:41:22.408 INFO:teuthology.orchestra.run.smithi134.stdout:Source RPM : grub2-2.06-68.el9.src.rpm 2024-03-28T17:41:22.408 INFO:teuthology.orchestra.run.smithi134.stdout:Build Date : Wed 02 Aug 2023 03:15:48 PM UTC 2024-03-28T17:41:22.408 INFO:teuthology.orchestra.run.smithi134.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-03-28T17:41:22.408 INFO:teuthology.orchestra.run.smithi134.stdout:Packager : builder@centos.org 2024-03-28T17:41:22.408 INFO:teuthology.orchestra.run.smithi134.stdout:Vendor : CentOS 2024-03-28T17:41:22.408 INFO:teuthology.orchestra.run.smithi134.stdout:URL : http://www.gnu.org/software/grub/ 2024-03-28T17:41:22.408 INFO:teuthology.orchestra.run.smithi134.stdout:Summary : Support tools for GRUB. 2024-03-28T17:41:22.408 INFO:teuthology.orchestra.run.smithi134.stdout:Description : 2024-03-28T17:41:22.408 INFO:teuthology.orchestra.run.smithi134.stdout: 2024-03-28T17:41:22.408 INFO:teuthology.orchestra.run.smithi134.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-03-28T17:41:22.408 INFO:teuthology.orchestra.run.smithi134.stdout:customizable bootloader with modular architecture. It supports a rich 2024-03-28T17:41:22.408 INFO:teuthology.orchestra.run.smithi134.stdout:variety of kernel formats, file systems, computer architectures and 2024-03-28T17:41:22.408 INFO:teuthology.orchestra.run.smithi134.stdout:hardware devices. 2024-03-28T17:41:22.408 INFO:teuthology.orchestra.run.smithi134.stdout: 2024-03-28T17:41:22.408 INFO:teuthology.orchestra.run.smithi134.stdout:This subpackage provides tools for support of all platforms. 2024-03-28T17:41:22.409 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-03-28T17:41:22.409 INFO:teuthology.task.kernel:Updating grub on smithi134 to boot 6.8.0-g1df0c7e8a235 2024-03-28T17:41:22.409 DEBUG:teuthology.orchestra.run.smithi134:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-03-28T17:41:22.976 INFO:teuthology.orchestra.run.smithi134.stderr:Generating grub configuration file ... 2024-03-28T17:41:24.426 INFO:teuthology.orchestra.run.smithi134.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-03-28T17:41:24.454 INFO:teuthology.orchestra.run.smithi134.stderr:done 2024-03-28T17:41:24.459 DEBUG:teuthology.orchestra.run.smithi134:> mktemp 2024-03-28T17:41:24.473 INFO:teuthology.orchestra.run.smithi134.stdout:/tmp/tmp.U5ZHJyaWID 2024-03-28T17:41:24.473 DEBUG:teuthology.orchestra.run.smithi134:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.U5ZHJyaWID 2024-03-28T17:41:24.542 DEBUG:teuthology.orchestra.run.smithi134:> sudo chmod 0666 /tmp/tmp.U5ZHJyaWID 2024-03-28T17:41:24.689 DEBUG:teuthology.orchestra.remote:smithi134:/tmp/tmp.U5ZHJyaWID is 6KB 2024-03-28T17:41:24.709 DEBUG:teuthology.orchestra.run.smithi134:> rm -fr /tmp/tmp.U5ZHJyaWID 2024-03-28T17:41:24.723 DEBUG:teuthology.orchestra.run.smithi134:> sudo /bin/ls /boot/loader/entries || true 2024-03-28T17:41:24.787 INFO:teuthology.orchestra.run.smithi134.stdout:15949e3c88704300882482eddfd7baae-0-rescue.conf 2024-03-28T17:41:24.787 INFO:teuthology.orchestra.run.smithi134.stdout:15949e3c88704300882482eddfd7baae-5.14.0-239.el9.x86_64.conf 2024-03-28T17:41:24.787 INFO:teuthology.orchestra.run.smithi134.stdout:15949e3c88704300882482eddfd7baae-5.14.0-361.el9.x86_64.conf 2024-03-28T17:41:24.788 INFO:teuthology.orchestra.run.smithi134.stdout:15949e3c88704300882482eddfd7baae-6.8.0-g1df0c7e8a235.conf 2024-03-28T17:41:24.789 DEBUG:teuthology.orchestra.run.smithi134:> sudo grub2-set-default 15949e3c88704300882482eddfd7baae-6.8.0-g1df0c7e8a235 2024-03-28T17:41:24.937 DEBUG:teuthology.orchestra.run.smithi134:> sudo shutdown -r now 2024-03-28T17:41:35.960 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi022.front.sepia.ceph.com, path=/tmp/kernel.x86_64.rpm, version=None) 2024-03-28T17:41:35.961 DEBUG:teuthology.orchestra.run.smithi022:> rpm -qlp /tmp/kernel.x86_64.rpm 2024-03-28T17:41:36.033 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235 2024-03-28T17:41:36.033 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/System.map 2024-03-28T17:41:36.034 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/config 2024-03-28T17:41:36.034 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel 2024-03-28T17:41:36.034 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch 2024-03-28T17:41:36.034 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86 2024-03-28T17:41:36.034 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/crypto 2024-03-28T17:41:36.034 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/crypto/blowfish-x86_64.ko 2024-03-28T17:41:36.034 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/crypto/camellia-aesni-avx-x86_64.ko 2024-03-28T17:41:36.034 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/crypto/camellia-aesni-avx2.ko 2024-03-28T17:41:36.034 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/crypto/camellia-x86_64.ko 2024-03-28T17:41:36.034 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/crypto/cast5-avx-x86_64.ko 2024-03-28T17:41:36.034 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/crypto/cast6-avx-x86_64.ko 2024-03-28T17:41:36.034 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/crypto/crc32-pclmul.ko 2024-03-28T17:41:36.034 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/crypto/crc32c-intel.ko 2024-03-28T17:41:36.035 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/crypto/crct10dif-pclmul.ko 2024-03-28T17:41:36.035 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/crypto/des3_ede-x86_64.ko 2024-03-28T17:41:36.035 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/crypto/ghash-clmulni-intel.ko 2024-03-28T17:41:36.035 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/crypto/serpent-avx-x86_64.ko 2024-03-28T17:41:36.035 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/crypto/serpent-avx2.ko 2024-03-28T17:41:36.035 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/crypto/serpent-sse2-x86_64.ko 2024-03-28T17:41:36.035 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/crypto/sha1-ssse3.ko 2024-03-28T17:41:36.035 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/crypto/sha256-ssse3.ko 2024-03-28T17:41:36.035 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/crypto/sha512-ssse3.ko 2024-03-28T17:41:36.035 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/crypto/twofish-avx-x86_64.ko 2024-03-28T17:41:36.035 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/crypto/twofish-x86_64-3way.ko 2024-03-28T17:41:36.035 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/crypto/twofish-x86_64.ko 2024-03-28T17:41:36.035 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/kernel 2024-03-28T17:41:36.035 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/kernel/cpu 2024-03-28T17:41:36.035 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/kernel/cpu/mce 2024-03-28T17:41:36.036 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/kernel/cpu/mce/mce-inject.ko 2024-03-28T17:41:36.036 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/kvm 2024-03-28T17:41:36.036 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/kvm/kvm-amd.ko 2024-03-28T17:41:36.036 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/kvm/kvm-intel.ko 2024-03-28T17:41:36.036 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/kvm/kvm.ko 2024-03-28T17:41:36.036 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto 2024-03-28T17:41:36.036 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/ansi_cprng.ko 2024-03-28T17:41:36.036 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/anubis.ko 2024-03-28T17:41:36.036 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/arc4.ko 2024-03-28T17:41:36.036 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/async_tx 2024-03-28T17:41:36.036 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/async_tx/async_memcpy.ko 2024-03-28T17:41:36.036 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/async_tx/async_pq.ko 2024-03-28T17:41:36.036 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/async_tx/async_raid6_recov.ko 2024-03-28T17:41:36.036 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/async_tx/async_tx.ko 2024-03-28T17:41:36.036 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/async_tx/async_xor.ko 2024-03-28T17:41:36.036 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/async_tx/raid6test.ko 2024-03-28T17:41:36.037 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/authenc.ko 2024-03-28T17:41:36.037 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/authencesn.ko 2024-03-28T17:41:36.037 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/blake2b_generic.ko 2024-03-28T17:41:36.037 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/blowfish_common.ko 2024-03-28T17:41:36.037 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/blowfish_generic.ko 2024-03-28T17:41:36.037 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/camellia_generic.ko 2024-03-28T17:41:36.037 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/cast5_generic.ko 2024-03-28T17:41:36.037 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/cast6_generic.ko 2024-03-28T17:41:36.037 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/cast_common.ko 2024-03-28T17:41:36.037 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/ccm.ko 2024-03-28T17:41:36.037 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/cmac.ko 2024-03-28T17:41:36.037 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/crc32_generic.ko 2024-03-28T17:41:36.037 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/crypto_engine.ko 2024-03-28T17:41:36.037 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/crypto_user.ko 2024-03-28T17:41:36.037 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/cts.ko 2024-03-28T17:41:36.037 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/des_generic.ko 2024-03-28T17:41:36.038 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/dh_generic.ko 2024-03-28T17:41:36.038 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/echainiv.ko 2024-03-28T17:41:36.038 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/essiv.ko 2024-03-28T17:41:36.038 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/fcrypt.ko 2024-03-28T17:41:36.038 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/khazad.ko 2024-03-28T17:41:36.038 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/lz4.ko 2024-03-28T17:41:36.038 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/lz4hc.ko 2024-03-28T17:41:36.038 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/md4.ko 2024-03-28T17:41:36.038 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/michael_mic.ko 2024-03-28T17:41:36.038 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/pcbc.ko 2024-03-28T17:41:36.038 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/pcrypt.ko 2024-03-28T17:41:36.038 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/rmd160.ko 2024-03-28T17:41:36.038 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/seed.ko 2024-03-28T17:41:36.038 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/serpent_generic.ko 2024-03-28T17:41:36.038 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/tcrypt.ko 2024-03-28T17:41:36.039 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/tea.ko 2024-03-28T17:41:36.039 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/twofish_common.ko 2024-03-28T17:41:36.039 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/twofish_generic.ko 2024-03-28T17:41:36.039 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/vmac.ko 2024-03-28T17:41:36.039 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/wp512.ko 2024-03-28T17:41:36.039 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/xcbc.ko 2024-03-28T17:41:36.039 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/xor.ko 2024-03-28T17:41:36.039 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/xxhash_generic.ko 2024-03-28T17:41:36.039 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers 2024-03-28T17:41:36.039 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/acpi 2024-03-28T17:41:36.039 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/acpi/acpi_ipmi.ko 2024-03-28T17:41:36.039 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/acpi/acpi_pad.ko 2024-03-28T17:41:36.039 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/acpi/custom_method.ko 2024-03-28T17:41:36.039 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/acpi/ec_sys.ko 2024-03-28T17:41:36.039 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/acpi/platform_profile.ko 2024-03-28T17:41:36.039 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/acpi/sbs.ko 2024-03-28T17:41:36.040 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/acpi/sbshc.ko 2024-03-28T17:41:36.040 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/acpi/video.ko 2024-03-28T17:41:36.040 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata 2024-03-28T17:41:36.040 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/acard-ahci.ko 2024-03-28T17:41:36.040 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/ahci_platform.ko 2024-03-28T17:41:36.040 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/ata_generic.ko 2024-03-28T17:41:36.040 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/libahci_platform.ko 2024-03-28T17:41:36.040 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_acpi.ko 2024-03-28T17:41:36.040 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_ali.ko 2024-03-28T17:41:36.040 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_amd.ko 2024-03-28T17:41:36.040 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_artop.ko 2024-03-28T17:41:36.040 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_atiixp.ko 2024-03-28T17:41:36.040 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_atp867x.ko 2024-03-28T17:41:36.040 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_cmd640.ko 2024-03-28T17:41:36.040 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_cmd64x.ko 2024-03-28T17:41:36.040 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_cypress.ko 2024-03-28T17:41:36.041 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_efar.ko 2024-03-28T17:41:36.041 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_hpt366.ko 2024-03-28T17:41:36.041 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_hpt37x.ko 2024-03-28T17:41:36.041 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_hpt3x2n.ko 2024-03-28T17:41:36.041 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_hpt3x3.ko 2024-03-28T17:41:36.041 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_it8213.ko 2024-03-28T17:41:36.041 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_it821x.ko 2024-03-28T17:41:36.041 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_jmicron.ko 2024-03-28T17:41:36.041 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_marvell.ko 2024-03-28T17:41:36.041 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_mpiix.ko 2024-03-28T17:41:36.041 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_netcell.ko 2024-03-28T17:41:36.041 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_ninja32.ko 2024-03-28T17:41:36.041 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_ns87410.ko 2024-03-28T17:41:36.041 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_ns87415.ko 2024-03-28T17:41:36.041 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_oldpiix.ko 2024-03-28T17:41:36.041 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_opti.ko 2024-03-28T17:41:36.041 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_optidma.ko 2024-03-28T17:41:36.042 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_pcmcia.ko 2024-03-28T17:41:36.042 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_pdc2027x.ko 2024-03-28T17:41:36.042 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_pdc202xx_old.ko 2024-03-28T17:41:36.042 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_piccolo.ko 2024-03-28T17:41:36.042 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_rdc.ko 2024-03-28T17:41:36.042 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_sch.ko 2024-03-28T17:41:36.042 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_serverworks.ko 2024-03-28T17:41:36.042 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_sil680.ko 2024-03-28T17:41:36.042 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_sis.ko 2024-03-28T17:41:36.042 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_sl82c105.ko 2024-03-28T17:41:36.042 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_triflex.ko 2024-03-28T17:41:36.042 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_via.ko 2024-03-28T17:41:36.042 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pdc_adma.ko 2024-03-28T17:41:36.042 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/sata_inic162x.ko 2024-03-28T17:41:36.042 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/sata_mv.ko 2024-03-28T17:41:36.042 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/sata_nv.ko 2024-03-28T17:41:36.043 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/sata_promise.ko 2024-03-28T17:41:36.043 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/sata_qstor.ko 2024-03-28T17:41:36.043 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/sata_sil.ko 2024-03-28T17:41:36.043 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/sata_sil24.ko 2024-03-28T17:41:36.043 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/sata_sis.ko 2024-03-28T17:41:36.043 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/sata_svw.ko 2024-03-28T17:41:36.043 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/sata_sx4.ko 2024-03-28T17:41:36.043 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/sata_uli.ko 2024-03-28T17:41:36.043 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/sata_via.ko 2024-03-28T17:41:36.043 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/sata_vsc.ko 2024-03-28T17:41:36.043 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/auxdisplay 2024-03-28T17:41:36.043 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/auxdisplay/ks0108.ko 2024-03-28T17:41:36.043 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/base 2024-03-28T17:41:36.043 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/base/regmap 2024-03-28T17:41:36.043 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/base/regmap/regmap-i2c.ko 2024-03-28T17:41:36.043 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/base/regmap/regmap-mmio.ko 2024-03-28T17:41:36.044 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/bcma 2024-03-28T17:41:36.044 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/bcma/bcma.ko 2024-03-28T17:41:36.044 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/block 2024-03-28T17:41:36.044 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/block/aoe 2024-03-28T17:41:36.044 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/block/aoe/aoe.ko 2024-03-28T17:41:36.044 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/block/brd.ko 2024-03-28T17:41:36.044 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/block/drbd 2024-03-28T17:41:36.044 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/block/drbd/drbd.ko 2024-03-28T17:41:36.044 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/block/floppy.ko 2024-03-28T17:41:36.044 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/block/loop.ko 2024-03-28T17:41:36.044 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/block/mtip32xx 2024-03-28T17:41:36.044 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/block/mtip32xx/mtip32xx.ko 2024-03-28T17:41:36.044 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/block/nbd.ko 2024-03-28T17:41:36.044 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/block/null_blk 2024-03-28T17:41:36.044 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/block/null_blk/null_blk.ko 2024-03-28T17:41:36.044 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/block/pktcdvd.ko 2024-03-28T17:41:36.044 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/block/rbd.ko 2024-03-28T17:41:36.045 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/block/virtio_blk.ko 2024-03-28T17:41:36.045 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/block/xen-blkback 2024-03-28T17:41:36.045 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/block/xen-blkback/xen-blkback.ko 2024-03-28T17:41:36.045 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/block/xen-blkfront.ko 2024-03-28T17:41:36.045 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/block/zram 2024-03-28T17:41:36.045 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/block/zram/zram.ko 2024-03-28T17:41:36.045 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char 2024-03-28T17:41:36.045 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/hangcheck-timer.ko 2024-03-28T17:41:36.045 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/hw_random 2024-03-28T17:41:36.045 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/hw_random/amd-rng.ko 2024-03-28T17:41:36.045 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/hw_random/intel-rng.ko 2024-03-28T17:41:36.045 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/hw_random/timeriomem-rng.ko 2024-03-28T17:41:36.045 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/hw_random/via-rng.ko 2024-03-28T17:41:36.045 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/hw_random/virtio-rng.ko 2024-03-28T17:41:36.045 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/ipmi 2024-03-28T17:41:36.046 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/ipmi/ipmi_devintf.ko 2024-03-28T17:41:36.046 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/ipmi/ipmi_msghandler.ko 2024-03-28T17:41:36.046 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/ipmi/ipmi_poweroff.ko 2024-03-28T17:41:36.046 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/ipmi/ipmi_si.ko 2024-03-28T17:41:36.046 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/ipmi/ipmi_ssif.ko 2024-03-28T17:41:36.046 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/ipmi/ipmi_watchdog.ko 2024-03-28T17:41:36.046 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/lp.ko 2024-03-28T17:41:36.046 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/mwave 2024-03-28T17:41:36.046 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/mwave/mwave.ko 2024-03-28T17:41:36.046 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/ppdev.ko 2024-03-28T17:41:36.046 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/tlclk.ko 2024-03-28T17:41:36.046 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/tpm 2024-03-28T17:41:36.046 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/tpm/tpm.ko 2024-03-28T17:41:36.046 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/tpm/tpm_atmel.ko 2024-03-28T17:41:36.046 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/tpm/tpm_infineon.ko 2024-03-28T17:41:36.046 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/tpm/tpm_nsc.ko 2024-03-28T17:41:36.047 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/tpm/tpm_tis.ko 2024-03-28T17:41:36.047 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/tpm/tpm_tis_core.ko 2024-03-28T17:41:36.047 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/uv_mmtimer.ko 2024-03-28T17:41:36.047 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/virtio_console.ko 2024-03-28T17:41:36.047 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/cpufreq 2024-03-28T17:41:36.047 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/cpufreq/acpi-cpufreq.ko 2024-03-28T17:41:36.047 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/cpufreq/amd_freq_sensitivity.ko 2024-03-28T17:41:36.047 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/cpufreq/p4-clockmod.ko 2024-03-28T17:41:36.047 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/cpufreq/pcc-cpufreq.ko 2024-03-28T17:41:36.047 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/cpufreq/powernow-k8.ko 2024-03-28T17:41:36.047 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/cpufreq/speedstep-lib.ko 2024-03-28T17:41:36.047 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/crypto 2024-03-28T17:41:36.047 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/crypto/ccp 2024-03-28T17:41:36.047 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/crypto/ccp/ccp-crypto.ko 2024-03-28T17:41:36.047 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/crypto/ccp/ccp.ko 2024-03-28T17:41:36.048 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/crypto/intel 2024-03-28T17:41:36.048 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/crypto/intel/qat 2024-03-28T17:41:36.048 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/crypto/intel/qat/qat_common 2024-03-28T17:41:36.048 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/crypto/intel/qat/qat_common/intel_qat.ko 2024-03-28T17:41:36.048 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/crypto/intel/qat/qat_dh895xcc 2024-03-28T17:41:36.048 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/crypto/intel/qat/qat_dh895xcc/qat_dh895xcc.ko 2024-03-28T17:41:36.048 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/crypto/padlock-aes.ko 2024-03-28T17:41:36.048 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/crypto/padlock-sha.ko 2024-03-28T17:41:36.048 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/crypto/virtio 2024-03-28T17:41:36.048 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/crypto/virtio/virtio_crypto.ko 2024-03-28T17:41:36.048 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/dca 2024-03-28T17:41:36.048 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/dca/dca.ko 2024-03-28T17:41:36.048 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/devfreq 2024-03-28T17:41:36.048 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/devfreq/governor_simpleondemand.ko 2024-03-28T17:41:36.048 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/dma 2024-03-28T17:41:36.048 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/dma/dw 2024-03-28T17:41:36.048 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/dma/dw/dw_dmac.ko 2024-03-28T17:41:36.049 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/dma/ioat 2024-03-28T17:41:36.049 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/dma/ioat/ioatdma.ko 2024-03-28T17:41:36.049 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/edac 2024-03-28T17:41:36.049 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/edac/amd64_edac.ko 2024-03-28T17:41:36.049 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/edac/e752x_edac.ko 2024-03-28T17:41:36.049 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/edac/edac_mce_amd.ko 2024-03-28T17:41:36.049 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/edac/i3000_edac.ko 2024-03-28T17:41:36.049 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/edac/i3200_edac.ko 2024-03-28T17:41:36.049 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/edac/i5100_edac.ko 2024-03-28T17:41:36.049 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/edac/i5400_edac.ko 2024-03-28T17:41:36.049 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/edac/i7300_edac.ko 2024-03-28T17:41:36.049 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/edac/i7core_edac.ko 2024-03-28T17:41:36.049 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/edac/i82975x_edac.ko 2024-03-28T17:41:36.049 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/edac/ie31200_edac.ko 2024-03-28T17:41:36.049 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/edac/sb_edac.ko 2024-03-28T17:41:36.049 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/edac/x38_edac.ko 2024-03-28T17:41:36.050 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/firmware 2024-03-28T17:41:36.050 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/firmware/edd.ko 2024-03-28T17:41:36.050 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/firmware/iscsi_ibft.ko 2024-03-28T17:41:36.050 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/gpio 2024-03-28T17:41:36.050 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/gpio/gpio-ich.ko 2024-03-28T17:41:36.050 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/gpio/gpio-viperboard.ko 2024-03-28T17:41:36.050 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid 2024-03-28T17:41:36.050 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-appleir.ko 2024-03-28T17:41:36.050 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-aureal.ko 2024-03-28T17:41:36.050 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-axff.ko 2024-03-28T17:41:36.050 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-dr.ko 2024-03-28T17:41:36.050 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-elecom.ko 2024-03-28T17:41:36.050 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-elo.ko 2024-03-28T17:41:36.050 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-emsff.ko 2024-03-28T17:41:36.050 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-gaff.ko 2024-03-28T17:41:36.050 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-gt683r.ko 2024-03-28T17:41:36.051 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-gyration.ko 2024-03-28T17:41:36.051 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-holtek-kbd.ko 2024-03-28T17:41:36.051 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-holtek-mouse.ko 2024-03-28T17:41:36.051 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-holtekff.ko 2024-03-28T17:41:36.051 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-hyperv.ko 2024-03-28T17:41:36.051 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-icade.ko 2024-03-28T17:41:36.051 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-keytouch.ko 2024-03-28T17:41:36.051 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-kye.ko 2024-03-28T17:41:36.051 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-lcpower.ko 2024-03-28T17:41:36.051 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-led.ko 2024-03-28T17:41:36.051 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-lenovo.ko 2024-03-28T17:41:36.051 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-logitech-dj.ko 2024-03-28T17:41:36.051 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-logitech-hidpp.ko 2024-03-28T17:41:36.051 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-multitouch.ko 2024-03-28T17:41:36.051 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-ortek.ko 2024-03-28T17:41:36.051 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-penmount.ko 2024-03-28T17:41:36.052 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-petalynx.ko 2024-03-28T17:41:36.052 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-picolcd.ko 2024-03-28T17:41:36.052 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-pl.ko 2024-03-28T17:41:36.052 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-plantronics.ko 2024-03-28T17:41:36.052 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-primax.ko 2024-03-28T17:41:36.052 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-rmi.ko 2024-03-28T17:41:36.052 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-roccat-arvo.ko 2024-03-28T17:41:36.052 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-roccat-common.ko 2024-03-28T17:41:36.052 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-roccat-isku.ko 2024-03-28T17:41:36.052 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-roccat-kone.ko 2024-03-28T17:41:36.052 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-roccat-koneplus.ko 2024-03-28T17:41:36.052 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-roccat-konepure.ko 2024-03-28T17:41:36.052 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-roccat-kovaplus.ko 2024-03-28T17:41:36.052 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-roccat-lua.ko 2024-03-28T17:41:36.052 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-roccat-pyra.ko 2024-03-28T17:41:36.052 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-roccat-ryos.ko 2024-03-28T17:41:36.052 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-roccat-savu.ko 2024-03-28T17:41:36.053 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-roccat.ko 2024-03-28T17:41:36.053 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-saitek.ko 2024-03-28T17:41:36.053 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-samsung.ko 2024-03-28T17:41:36.053 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-sensor-hub.ko 2024-03-28T17:41:36.053 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-sjoy.ko 2024-03-28T17:41:36.053 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-sony.ko 2024-03-28T17:41:36.053 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-speedlink.ko 2024-03-28T17:41:36.053 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-steelseries.ko 2024-03-28T17:41:36.053 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-sunplus.ko 2024-03-28T17:41:36.053 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-thrustmaster.ko 2024-03-28T17:41:36.053 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-tivo.ko 2024-03-28T17:41:36.053 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-tmff.ko 2024-03-28T17:41:36.053 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-topseed.ko 2024-03-28T17:41:36.053 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-twinhan.ko 2024-03-28T17:41:36.053 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-uclogic.ko 2024-03-28T17:41:36.054 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-waltop.ko 2024-03-28T17:41:36.054 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-wiimote.ko 2024-03-28T17:41:36.054 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-xinmo.ko 2024-03-28T17:41:36.054 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-zpff.ko 2024-03-28T17:41:36.054 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-zydacron.ko 2024-03-28T17:41:36.054 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/uhid.ko 2024-03-28T17:41:36.054 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/wacom.ko 2024-03-28T17:41:36.054 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hv 2024-03-28T17:41:36.054 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hv/hv_balloon.ko 2024-03-28T17:41:36.054 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hv/hv_utils.ko 2024-03-28T17:41:36.054 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hv/hv_vmbus.ko 2024-03-28T17:41:36.054 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon 2024-03-28T17:41:36.054 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/abituguru.ko 2024-03-28T17:41:36.054 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/abituguru3.ko 2024-03-28T17:41:36.054 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/acpi_power_meter.ko 2024-03-28T17:41:36.054 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/ad7414.ko 2024-03-28T17:41:36.055 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/ad7418.ko 2024-03-28T17:41:36.055 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/adc128d818.ko 2024-03-28T17:41:36.055 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/adm1025.ko 2024-03-28T17:41:36.055 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/adm1026.ko 2024-03-28T17:41:36.055 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/adm1029.ko 2024-03-28T17:41:36.055 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/adm1031.ko 2024-03-28T17:41:36.055 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/adm9240.ko 2024-03-28T17:41:36.055 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/ads7828.ko 2024-03-28T17:41:36.055 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/adt7410.ko 2024-03-28T17:41:36.055 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/adt7411.ko 2024-03-28T17:41:36.055 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/adt7462.ko 2024-03-28T17:41:36.055 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/adt7470.ko 2024-03-28T17:41:36.055 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/adt7475.ko 2024-03-28T17:41:36.055 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/adt7x10.ko 2024-03-28T17:41:36.055 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/amc6821.ko 2024-03-28T17:41:36.055 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/applesmc.ko 2024-03-28T17:41:36.055 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/asb100.ko 2024-03-28T17:41:36.056 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/asc7621.ko 2024-03-28T17:41:36.056 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/asus_atk0110.ko 2024-03-28T17:41:36.056 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/atxp1.ko 2024-03-28T17:41:36.056 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/coretemp.ko 2024-03-28T17:41:36.056 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/dell-smm-hwmon.ko 2024-03-28T17:41:36.056 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/dme1737.ko 2024-03-28T17:41:36.056 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/ds1621.ko 2024-03-28T17:41:36.056 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/ds620.ko 2024-03-28T17:41:36.056 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/emc1403.ko 2024-03-28T17:41:36.056 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/emc6w201.ko 2024-03-28T17:41:36.056 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/f71805f.ko 2024-03-28T17:41:36.056 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/f71882fg.ko 2024-03-28T17:41:36.056 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/f75375s.ko 2024-03-28T17:41:36.056 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/fam15h_power.ko 2024-03-28T17:41:36.056 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/fschmd.ko 2024-03-28T17:41:36.056 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/g760a.ko 2024-03-28T17:41:36.057 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/g762.ko 2024-03-28T17:41:36.057 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/gl518sm.ko 2024-03-28T17:41:36.057 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/gl520sm.ko 2024-03-28T17:41:36.057 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/hwmon-vid.ko 2024-03-28T17:41:36.057 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/i5500_temp.ko 2024-03-28T17:41:36.057 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/i5k_amb.ko 2024-03-28T17:41:36.057 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/ibmaem.ko 2024-03-28T17:41:36.057 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/ibmpex.ko 2024-03-28T17:41:36.057 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/ina209.ko 2024-03-28T17:41:36.057 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/ina2xx.ko 2024-03-28T17:41:36.057 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/it87.ko 2024-03-28T17:41:36.057 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/k10temp.ko 2024-03-28T17:41:36.057 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/k8temp.ko 2024-03-28T17:41:36.057 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/lineage-pem.ko 2024-03-28T17:41:36.057 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/lm63.ko 2024-03-28T17:41:36.057 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/lm73.ko 2024-03-28T17:41:36.058 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/lm75.ko 2024-03-28T17:41:36.058 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/lm77.ko 2024-03-28T17:41:36.058 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/lm78.ko 2024-03-28T17:41:36.058 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/lm80.ko 2024-03-28T17:41:36.058 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/lm83.ko 2024-03-28T17:41:36.058 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/lm85.ko 2024-03-28T17:41:36.058 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/lm87.ko 2024-03-28T17:41:36.058 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/lm90.ko 2024-03-28T17:41:36.058 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/lm92.ko 2024-03-28T17:41:36.058 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/lm93.ko 2024-03-28T17:41:36.058 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/lm95234.ko 2024-03-28T17:41:36.058 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/lm95241.ko 2024-03-28T17:41:36.058 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/lm95245.ko 2024-03-28T17:41:36.058 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/ltc2945.ko 2024-03-28T17:41:36.058 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/ltc4151.ko 2024-03-28T17:41:36.058 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/ltc4215.ko 2024-03-28T17:41:36.059 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/ltc4222.ko 2024-03-28T17:41:36.059 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/ltc4245.ko 2024-03-28T17:41:36.059 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/ltc4260.ko 2024-03-28T17:41:36.059 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/ltc4261.ko 2024-03-28T17:41:36.059 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/max16065.ko 2024-03-28T17:41:36.059 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/max1619.ko 2024-03-28T17:41:36.059 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/max1668.ko 2024-03-28T17:41:36.059 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/max197.ko 2024-03-28T17:41:36.059 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/max6639.ko 2024-03-28T17:41:36.059 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/max6650.ko 2024-03-28T17:41:36.059 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/max6697.ko 2024-03-28T17:41:36.059 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/mcp3021.ko 2024-03-28T17:41:36.059 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/nct6683.ko 2024-03-28T17:41:36.059 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/nct6775-core.ko 2024-03-28T17:41:36.059 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/nct6775.ko 2024-03-28T17:41:36.059 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/nct7802.ko 2024-03-28T17:41:36.059 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/pc87360.ko 2024-03-28T17:41:36.060 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/pc87427.ko 2024-03-28T17:41:36.060 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/pcf8591.ko 2024-03-28T17:41:36.060 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/pmbus 2024-03-28T17:41:36.060 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/pmbus/adm1275.ko 2024-03-28T17:41:36.060 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/pmbus/lm25066.ko 2024-03-28T17:41:36.060 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/pmbus/ltc2978.ko 2024-03-28T17:41:36.060 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/pmbus/max16064.ko 2024-03-28T17:41:36.060 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/pmbus/max34440.ko 2024-03-28T17:41:36.060 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/pmbus/max8688.ko 2024-03-28T17:41:36.060 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/pmbus/pmbus.ko 2024-03-28T17:41:36.060 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/pmbus/pmbus_core.ko 2024-03-28T17:41:36.060 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/pmbus/tps40422.ko 2024-03-28T17:41:36.060 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/pmbus/ucd9000.ko 2024-03-28T17:41:36.060 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/pmbus/ucd9200.ko 2024-03-28T17:41:36.060 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/pmbus/zl6100.ko 2024-03-28T17:41:36.061 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/powr1220.ko 2024-03-28T17:41:36.061 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/sch5627.ko 2024-03-28T17:41:36.061 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/sch5636.ko 2024-03-28T17:41:36.061 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/sch56xx-common.ko 2024-03-28T17:41:36.061 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/sht15.ko 2024-03-28T17:41:36.061 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/sht21.ko 2024-03-28T17:41:36.061 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/shtc1.ko 2024-03-28T17:41:36.061 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/sis5595.ko 2024-03-28T17:41:36.061 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/smsc47b397.ko 2024-03-28T17:41:36.061 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/smsc47m1.ko 2024-03-28T17:41:36.061 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/smsc47m192.ko 2024-03-28T17:41:36.061 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/thmc50.ko 2024-03-28T17:41:36.061 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/tmp102.ko 2024-03-28T17:41:36.061 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/tmp103.ko 2024-03-28T17:41:36.061 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/tmp401.ko 2024-03-28T17:41:36.061 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/tmp421.ko 2024-03-28T17:41:36.061 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/via-cputemp.ko 2024-03-28T17:41:36.062 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/via686a.ko 2024-03-28T17:41:36.062 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/vt1211.ko 2024-03-28T17:41:36.062 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/vt8231.ko 2024-03-28T17:41:36.062 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/w83627ehf.ko 2024-03-28T17:41:36.062 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/w83627hf.ko 2024-03-28T17:41:36.062 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/w83781d.ko 2024-03-28T17:41:36.062 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/w83791d.ko 2024-03-28T17:41:36.062 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/w83792d.ko 2024-03-28T17:41:36.062 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/w83793.ko 2024-03-28T17:41:36.062 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/w83795.ko 2024-03-28T17:41:36.062 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/w83l785ts.ko 2024-03-28T17:41:36.062 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/w83l786ng.ko 2024-03-28T17:41:36.062 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c 2024-03-28T17:41:36.062 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/algos 2024-03-28T17:41:36.062 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/algos/i2c-algo-bit.ko 2024-03-28T17:41:36.062 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/algos/i2c-algo-pca.ko 2024-03-28T17:41:36.063 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses 2024-03-28T17:41:36.063 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-amd756-s4882.ko 2024-03-28T17:41:36.063 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-amd756.ko 2024-03-28T17:41:36.063 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-amd8111.ko 2024-03-28T17:41:36.063 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-ccgx-ucsi.ko 2024-03-28T17:41:36.063 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-designware-core.ko 2024-03-28T17:41:36.063 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-designware-pci.ko 2024-03-28T17:41:36.063 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-designware-platform.ko 2024-03-28T17:41:36.063 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-diolan-u2c.ko 2024-03-28T17:41:36.063 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-i801.ko 2024-03-28T17:41:36.063 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-isch.ko 2024-03-28T17:41:36.063 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-ismt.ko 2024-03-28T17:41:36.063 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-nforce2-s4985.ko 2024-03-28T17:41:36.063 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-nforce2.ko 2024-03-28T17:41:36.063 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-parport.ko 2024-03-28T17:41:36.063 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-pca-platform.ko 2024-03-28T17:41:36.063 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-piix4.ko 2024-03-28T17:41:36.064 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-scmi.ko 2024-03-28T17:41:36.064 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-simtec.ko 2024-03-28T17:41:36.064 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-sis96x.ko 2024-03-28T17:41:36.064 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-tiny-usb.ko 2024-03-28T17:41:36.064 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-via.ko 2024-03-28T17:41:36.064 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-viapro.ko 2024-03-28T17:41:36.064 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-viperboard.ko 2024-03-28T17:41:36.064 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/i2c-dev.ko 2024-03-28T17:41:36.064 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/i2c-mux.ko 2024-03-28T17:41:36.064 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/i2c-slave-eeprom.ko 2024-03-28T17:41:36.064 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/i2c-smbus.ko 2024-03-28T17:41:36.064 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/i2c-stub.ko 2024-03-28T17:41:36.064 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband 2024-03-28T17:41:36.064 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/core 2024-03-28T17:41:36.064 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/core/ib_cm.ko 2024-03-28T17:41:36.064 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/core/ib_core.ko 2024-03-28T17:41:36.065 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/core/ib_umad.ko 2024-03-28T17:41:36.065 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/core/ib_uverbs.ko 2024-03-28T17:41:36.065 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/core/iw_cm.ko 2024-03-28T17:41:36.065 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/core/rdma_cm.ko 2024-03-28T17:41:36.065 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/core/rdma_ucm.ko 2024-03-28T17:41:36.065 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/hw 2024-03-28T17:41:36.065 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/hw/cxgb4 2024-03-28T17:41:36.065 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/hw/cxgb4/iw_cxgb4.ko 2024-03-28T17:41:36.065 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/hw/mlx4 2024-03-28T17:41:36.065 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/hw/mlx4/mlx4_ib.ko 2024-03-28T17:41:36.065 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/hw/mlx5 2024-03-28T17:41:36.065 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/hw/mlx5/mlx5_ib.ko 2024-03-28T17:41:36.065 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/hw/mthca 2024-03-28T17:41:36.065 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/hw/mthca/ib_mthca.ko 2024-03-28T17:41:36.065 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/ulp 2024-03-28T17:41:36.066 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/ulp/ipoib 2024-03-28T17:41:36.066 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/ulp/ipoib/ib_ipoib.ko 2024-03-28T17:41:36.066 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/ulp/iser 2024-03-28T17:41:36.066 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/ulp/iser/ib_iser.ko 2024-03-28T17:41:36.066 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/ulp/isert 2024-03-28T17:41:36.066 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/ulp/isert/ib_isert.ko 2024-03-28T17:41:36.066 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/ulp/srp 2024-03-28T17:41:36.066 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/ulp/srp/ib_srp.ko 2024-03-28T17:41:36.066 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/ulp/srpt 2024-03-28T17:41:36.066 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/ulp/srpt/ib_srpt.ko 2024-03-28T17:41:36.066 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input 2024-03-28T17:41:36.066 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/gameport 2024-03-28T17:41:36.066 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/gameport/emu10k1-gp.ko 2024-03-28T17:41:36.066 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/gameport/fm801-gp.ko 2024-03-28T17:41:36.066 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/gameport/gameport.ko 2024-03-28T17:41:36.067 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/joydev.ko 2024-03-28T17:41:36.067 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/keyboard 2024-03-28T17:41:36.067 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/keyboard/gpio_keys.ko 2024-03-28T17:41:36.067 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/mouse 2024-03-28T17:41:36.067 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/mouse/appletouch.ko 2024-03-28T17:41:36.067 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/mouse/bcm5974.ko 2024-03-28T17:41:36.067 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/mouse/cyapatp.ko 2024-03-28T17:41:36.067 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/mouse/elan_i2c.ko 2024-03-28T17:41:36.067 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/mouse/sermouse.ko 2024-03-28T17:41:36.067 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/mouse/synaptics_i2c.ko 2024-03-28T17:41:36.067 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/mouse/synaptics_usb.ko 2024-03-28T17:41:36.067 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/mouse/vsxxxaa.ko 2024-03-28T17:41:36.067 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/rmi4 2024-03-28T17:41:36.067 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/rmi4/rmi_core.ko 2024-03-28T17:41:36.067 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/serio 2024-03-28T17:41:36.067 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/serio/altera_ps2.ko 2024-03-28T17:41:36.067 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/serio/arc_ps2.ko 2024-03-28T17:41:36.068 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/serio/hyperv-keyboard.ko 2024-03-28T17:41:36.068 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/serio/serio_raw.ko 2024-03-28T17:41:36.068 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/sparse-keymap.ko 2024-03-28T17:41:36.068 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/leds 2024-03-28T17:41:36.068 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/leds/leds-blinkm.ko 2024-03-28T17:41:36.068 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/leds/leds-lm3530.ko 2024-03-28T17:41:36.068 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/leds/leds-lp3944.ko 2024-03-28T17:41:36.068 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/leds/leds-ss4200.ko 2024-03-28T17:41:36.068 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/leds/trigger 2024-03-28T17:41:36.068 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/leds/trigger/ledtrig-audio.ko 2024-03-28T17:41:36.068 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/leds/trigger/ledtrig-backlight.ko 2024-03-28T17:41:36.068 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/leds/trigger/ledtrig-camera.ko 2024-03-28T17:41:36.068 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/leds/trigger/ledtrig-default-on.ko 2024-03-28T17:41:36.068 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/leds/trigger/ledtrig-gpio.ko 2024-03-28T17:41:36.068 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/leds/trigger/ledtrig-heartbeat.ko 2024-03-28T17:41:36.068 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/leds/trigger/ledtrig-oneshot.ko 2024-03-28T17:41:36.069 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/leds/trigger/ledtrig-timer.ko 2024-03-28T17:41:36.069 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/leds/trigger/ledtrig-transient.ko 2024-03-28T17:41:36.069 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md 2024-03-28T17:41:36.069 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/bcache 2024-03-28T17:41:36.069 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/bcache/bcache.ko 2024-03-28T17:41:36.069 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/dm-bio-prison.ko 2024-03-28T17:41:36.069 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/dm-cache-smq.ko 2024-03-28T17:41:36.069 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/dm-cache.ko 2024-03-28T17:41:36.069 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/dm-crypt.ko 2024-03-28T17:41:36.069 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/dm-delay.ko 2024-03-28T17:41:36.069 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/dm-flakey.ko 2024-03-28T17:41:36.069 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/dm-log-userspace.ko 2024-03-28T17:41:36.069 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/dm-multipath.ko 2024-03-28T17:41:36.069 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/dm-queue-length.ko 2024-03-28T17:41:36.069 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/dm-raid.ko 2024-03-28T17:41:36.069 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/dm-round-robin.ko 2024-03-28T17:41:36.069 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/dm-service-time.ko 2024-03-28T17:41:36.070 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/dm-switch.ko 2024-03-28T17:41:36.070 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/dm-thin-pool.ko 2024-03-28T17:41:36.070 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/dm-verity.ko 2024-03-28T17:41:36.070 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/persistent-data 2024-03-28T17:41:36.070 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/persistent-data/dm-persistent-data.ko 2024-03-28T17:41:36.070 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/raid0.ko 2024-03-28T17:41:36.070 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/raid1.ko 2024-03-28T17:41:36.070 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/raid10.ko 2024-03-28T17:41:36.070 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/raid456.ko 2024-03-28T17:41:36.070 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/message 2024-03-28T17:41:36.070 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/message/fusion 2024-03-28T17:41:36.070 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/message/fusion/mptbase.ko 2024-03-28T17:41:36.070 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/message/fusion/mptctl.ko 2024-03-28T17:41:36.070 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/message/fusion/mptfc.ko 2024-03-28T17:41:36.071 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/message/fusion/mptlan.ko 2024-03-28T17:41:36.071 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/message/fusion/mptsas.ko 2024-03-28T17:41:36.071 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/message/fusion/mptscsih.ko 2024-03-28T17:41:36.071 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/message/fusion/mptspi.ko 2024-03-28T17:41:36.071 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/mfd 2024-03-28T17:41:36.071 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/mfd/lpc_ich.ko 2024-03-28T17:41:36.071 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/mfd/lpc_sch.ko 2024-03-28T17:41:36.071 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/mfd/mfd-core.ko 2024-03-28T17:41:36.071 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/mfd/sm501.ko 2024-03-28T17:41:36.071 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/mfd/viperboard.ko 2024-03-28T17:41:36.071 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/mfd/vx855.ko 2024-03-28T17:41:36.071 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/mfd/wl1273-core.ko 2024-03-28T17:41:36.071 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc 2024-03-28T17:41:36.071 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/altera-stapl 2024-03-28T17:41:36.071 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/altera-stapl/altera-stapl.ko 2024-03-28T17:41:36.071 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/apds9802als.ko 2024-03-28T17:41:36.071 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/apds990x.ko 2024-03-28T17:41:36.072 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/bh1770glc.ko 2024-03-28T17:41:36.072 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/cb710 2024-03-28T17:41:36.072 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/cb710/cb710.ko 2024-03-28T17:41:36.072 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/eeprom 2024-03-28T17:41:36.072 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/eeprom/at24.ko 2024-03-28T17:41:36.072 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/eeprom/eeprom_93cx6.ko 2024-03-28T17:41:36.072 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/eeprom/max6875.ko 2024-03-28T17:41:36.072 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/enclosure.ko 2024-03-28T17:41:36.072 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/hpilo.ko 2024-03-28T17:41:36.072 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/isl29003.ko 2024-03-28T17:41:36.072 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/isl29020.ko 2024-03-28T17:41:36.072 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/lis3lv02d 2024-03-28T17:41:36.072 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/lis3lv02d/lis3lv02d.ko 2024-03-28T17:41:36.072 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/lis3lv02d/lis3lv02d_i2c.ko 2024-03-28T17:41:36.072 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/mei 2024-03-28T17:41:36.072 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/mei/mei-me.ko 2024-03-28T17:41:36.073 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/mei/mei-txe.ko 2024-03-28T17:41:36.073 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/mei/mei.ko 2024-03-28T17:41:36.073 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/sgi-gru 2024-03-28T17:41:36.073 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/sgi-gru/gru.ko 2024-03-28T17:41:36.073 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/sgi-xp 2024-03-28T17:41:36.073 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/sgi-xp/xp.ko 2024-03-28T17:41:36.073 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/sgi-xp/xpc.ko 2024-03-28T17:41:36.073 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/sgi-xp/xpnet.ko 2024-03-28T17:41:36.073 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/tifm_7xx1.ko 2024-03-28T17:41:36.073 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/tifm_core.ko 2024-03-28T17:41:36.073 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/tsl2550.ko 2024-03-28T17:41:36.073 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/vmw_balloon.ko 2024-03-28T17:41:36.073 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/vmw_vmci 2024-03-28T17:41:36.073 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/vmw_vmci/vmw_vmci.ko 2024-03-28T17:41:36.073 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net 2024-03-28T17:41:36.073 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/bonding 2024-03-28T17:41:36.074 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/bonding/bonding.ko 2024-03-28T17:41:36.074 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/dsa 2024-03-28T17:41:36.074 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/dsa/b53 2024-03-28T17:41:36.074 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/dsa/b53/b53_common.ko 2024-03-28T17:41:36.074 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/dsa/bcm-sf2.ko 2024-03-28T17:41:36.074 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/dsa/mv88e6060.ko 2024-03-28T17:41:36.074 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/dsa/mv88e6xxx 2024-03-28T17:41:36.074 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/dsa/mv88e6xxx/mv88e6xxx.ko 2024-03-28T17:41:36.074 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/dummy.ko 2024-03-28T17:41:36.074 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/eql.ko 2024-03-28T17:41:36.074 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet 2024-03-28T17:41:36.074 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/3com 2024-03-28T17:41:36.074 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/3com/3c574_cs.ko 2024-03-28T17:41:36.074 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/3com/3c589_cs.ko 2024-03-28T17:41:36.074 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/3com/3c59x.ko 2024-03-28T17:41:36.075 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/3com/typhoon.ko 2024-03-28T17:41:36.075 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/8390 2024-03-28T17:41:36.075 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/8390/8390.ko 2024-03-28T17:41:36.075 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/8390/axnet_cs.ko 2024-03-28T17:41:36.075 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/8390/ne2k-pci.ko 2024-03-28T17:41:36.075 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/8390/pcnet_cs.ko 2024-03-28T17:41:36.075 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/adaptec 2024-03-28T17:41:36.075 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/adaptec/starfire.ko 2024-03-28T17:41:36.075 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/agere 2024-03-28T17:41:36.075 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/agere/et131x.ko 2024-03-28T17:41:36.075 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/alteon 2024-03-28T17:41:36.075 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/alteon/acenic.ko 2024-03-28T17:41:36.075 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/altera 2024-03-28T17:41:36.075 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/altera/altera_tse.ko 2024-03-28T17:41:36.075 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/amd 2024-03-28T17:41:36.075 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/amd/amd8111e.ko 2024-03-28T17:41:36.076 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/amd/nmclan_cs.ko 2024-03-28T17:41:36.076 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/amd/pcnet32.ko 2024-03-28T17:41:36.076 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/atheros 2024-03-28T17:41:36.076 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/atheros/alx 2024-03-28T17:41:36.076 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/atheros/alx/alx.ko 2024-03-28T17:41:36.076 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/atheros/atl1c 2024-03-28T17:41:36.076 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/atheros/atl1c/atl1c.ko 2024-03-28T17:41:36.076 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/atheros/atl1e 2024-03-28T17:41:36.076 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/atheros/atl1e/atl1e.ko 2024-03-28T17:41:36.076 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/atheros/atlx 2024-03-28T17:41:36.076 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/atheros/atlx/atl1.ko 2024-03-28T17:41:36.076 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/atheros/atlx/atl2.ko 2024-03-28T17:41:36.076 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/broadcom 2024-03-28T17:41:36.076 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/broadcom/b44.ko 2024-03-28T17:41:36.076 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/broadcom/bnx2.ko 2024-03-28T17:41:36.077 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/broadcom/bnx2x 2024-03-28T17:41:36.077 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/broadcom/bnx2x/bnx2x.ko 2024-03-28T17:41:36.077 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/broadcom/cnic.ko 2024-03-28T17:41:36.077 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/broadcom/genet 2024-03-28T17:41:36.077 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/broadcom/genet/genet.ko 2024-03-28T17:41:36.077 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/broadcom/tg3.ko 2024-03-28T17:41:36.077 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/brocade 2024-03-28T17:41:36.077 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/brocade/bna 2024-03-28T17:41:36.077 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/brocade/bna/bna.ko 2024-03-28T17:41:36.077 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/chelsio 2024-03-28T17:41:36.077 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/chelsio/cxgb 2024-03-28T17:41:36.077 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/chelsio/cxgb/cxgb.ko 2024-03-28T17:41:36.077 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/chelsio/cxgb3 2024-03-28T17:41:36.077 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/chelsio/cxgb3/cxgb3.ko 2024-03-28T17:41:36.077 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/chelsio/cxgb4 2024-03-28T17:41:36.078 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/chelsio/cxgb4/cxgb4.ko 2024-03-28T17:41:36.078 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/chelsio/cxgb4vf 2024-03-28T17:41:36.078 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/chelsio/cxgb4vf/cxgb4vf.ko 2024-03-28T17:41:36.078 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/chelsio/libcxgb 2024-03-28T17:41:36.078 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/chelsio/libcxgb/libcxgb.ko 2024-03-28T17:41:36.078 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/cisco 2024-03-28T17:41:36.078 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/cisco/enic 2024-03-28T17:41:36.078 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/cisco/enic/enic.ko 2024-03-28T17:41:36.078 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/dec 2024-03-28T17:41:36.078 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/dec/tulip 2024-03-28T17:41:36.078 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/dec/tulip/de2104x.ko 2024-03-28T17:41:36.078 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/dec/tulip/dmfe.ko 2024-03-28T17:41:36.078 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/dec/tulip/tulip.ko 2024-03-28T17:41:36.078 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/dec/tulip/uli526x.ko 2024-03-28T17:41:36.078 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/dec/tulip/winbond-840.ko 2024-03-28T17:41:36.078 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/dec/tulip/xircom_cb.ko 2024-03-28T17:41:36.079 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/dlink 2024-03-28T17:41:36.079 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/dlink/dl2k.ko 2024-03-28T17:41:36.079 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/dlink/sundance.ko 2024-03-28T17:41:36.079 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/dnet.ko 2024-03-28T17:41:36.079 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/emulex 2024-03-28T17:41:36.079 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/emulex/benet 2024-03-28T17:41:36.079 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/emulex/benet/be2net.ko 2024-03-28T17:41:36.079 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/ethoc.ko 2024-03-28T17:41:36.079 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/fealnx.ko 2024-03-28T17:41:36.079 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/intel 2024-03-28T17:41:36.079 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/intel/e100.ko 2024-03-28T17:41:36.079 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/intel/e1000 2024-03-28T17:41:36.079 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/intel/e1000/e1000.ko 2024-03-28T17:41:36.079 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/intel/e1000e 2024-03-28T17:41:36.079 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/intel/e1000e/e1000e.ko 2024-03-28T17:41:36.079 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/intel/fm10k 2024-03-28T17:41:36.080 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/intel/fm10k/fm10k.ko 2024-03-28T17:41:36.080 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/intel/i40e 2024-03-28T17:41:36.080 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/intel/i40e/i40e.ko 2024-03-28T17:41:36.080 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/intel/iavf 2024-03-28T17:41:36.080 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/intel/iavf/iavf.ko 2024-03-28T17:41:36.080 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/intel/igb 2024-03-28T17:41:36.080 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/intel/igb/igb.ko 2024-03-28T17:41:36.080 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/intel/igbvf 2024-03-28T17:41:36.080 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/intel/igbvf/igbvf.ko 2024-03-28T17:41:36.080 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/intel/ixgbe 2024-03-28T17:41:36.080 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/intel/ixgbe/ixgbe.ko 2024-03-28T17:41:36.080 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/intel/ixgbevf 2024-03-28T17:41:36.080 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/intel/ixgbevf/ixgbevf.ko 2024-03-28T17:41:36.080 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/jme.ko 2024-03-28T17:41:36.080 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/marvell 2024-03-28T17:41:36.080 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/marvell/mvmdio.ko 2024-03-28T17:41:36.080 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/marvell/skge.ko 2024-03-28T17:41:36.081 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/marvell/sky2.ko 2024-03-28T17:41:36.081 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/mellanox 2024-03-28T17:41:36.081 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/mellanox/mlx4 2024-03-28T17:41:36.081 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_core.ko 2024-03-28T17:41:36.081 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_en.ko 2024-03-28T17:41:36.081 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/mellanox/mlx5 2024-03-28T17:41:36.081 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/mellanox/mlx5/core 2024-03-28T17:41:36.081 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/mellanox/mlx5/core/mlx5_core.ko 2024-03-28T17:41:36.081 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/micrel 2024-03-28T17:41:36.081 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/micrel/ksz884x.ko 2024-03-28T17:41:36.081 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/myricom 2024-03-28T17:41:36.081 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/myricom/myri10ge 2024-03-28T17:41:36.081 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/myricom/myri10ge/myri10ge.ko 2024-03-28T17:41:36.081 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/natsemi 2024-03-28T17:41:36.081 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/natsemi/natsemi.ko 2024-03-28T17:41:36.081 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/natsemi/ns83820.ko 2024-03-28T17:41:36.082 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/neterion 2024-03-28T17:41:36.082 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/neterion/s2io.ko 2024-03-28T17:41:36.082 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/nvidia 2024-03-28T17:41:36.082 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/nvidia/forcedeth.ko 2024-03-28T17:41:36.082 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/packetengines 2024-03-28T17:41:36.082 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/packetengines/hamachi.ko 2024-03-28T17:41:36.082 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/packetengines/yellowfin.ko 2024-03-28T17:41:36.082 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/qlogic 2024-03-28T17:41:36.082 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/qlogic/netxen 2024-03-28T17:41:36.082 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/qlogic/netxen/netxen_nic.ko 2024-03-28T17:41:36.082 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/qlogic/qla3xxx.ko 2024-03-28T17:41:36.082 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/qlogic/qlcnic 2024-03-28T17:41:36.082 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/qlogic/qlcnic/qlcnic.ko 2024-03-28T17:41:36.082 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/rdc 2024-03-28T17:41:36.082 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/rdc/r6040.ko 2024-03-28T17:41:36.083 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/realtek 2024-03-28T17:41:36.083 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/realtek/8139cp.ko 2024-03-28T17:41:36.083 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/realtek/8139too.ko 2024-03-28T17:41:36.083 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/realtek/atp.ko 2024-03-28T17:41:36.083 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/realtek/r8169.ko 2024-03-28T17:41:36.083 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/rocker 2024-03-28T17:41:36.083 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/rocker/rocker.ko 2024-03-28T17:41:36.083 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/sfc 2024-03-28T17:41:36.083 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/sfc/sfc.ko 2024-03-28T17:41:36.083 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/silan 2024-03-28T17:41:36.083 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/silan/sc92031.ko 2024-03-28T17:41:36.083 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/sis 2024-03-28T17:41:36.083 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/sis/sis190.ko 2024-03-28T17:41:36.083 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/sis/sis900.ko 2024-03-28T17:41:36.083 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/smsc 2024-03-28T17:41:36.084 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/smsc/epic100.ko 2024-03-28T17:41:36.084 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/smsc/smc91c92_cs.ko 2024-03-28T17:41:36.084 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/smsc/smsc911x.ko 2024-03-28T17:41:36.084 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/smsc/smsc9420.ko 2024-03-28T17:41:36.084 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/stmicro 2024-03-28T17:41:36.084 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/stmicro/stmmac 2024-03-28T17:41:36.084 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/stmicro/stmmac/dwmac-intel.ko 2024-03-28T17:41:36.084 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/stmicro/stmmac/stmmac.ko 2024-03-28T17:41:36.084 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/sun 2024-03-28T17:41:36.084 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/sun/cassini.ko 2024-03-28T17:41:36.084 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/sun/niu.ko 2024-03-28T17:41:36.084 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/sun/sungem.ko 2024-03-28T17:41:36.084 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/sun/sunhme.ko 2024-03-28T17:41:36.084 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/tehuti 2024-03-28T17:41:36.084 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/tehuti/tehuti.ko 2024-03-28T17:41:36.084 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/ti 2024-03-28T17:41:36.084 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/ti/tlan.ko 2024-03-28T17:41:36.085 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/via 2024-03-28T17:41:36.085 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/via/via-rhine.ko 2024-03-28T17:41:36.085 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/via/via-velocity.ko 2024-03-28T17:41:36.085 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/wiznet 2024-03-28T17:41:36.085 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/wiznet/w5100.ko 2024-03-28T17:41:36.085 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/wiznet/w5300.ko 2024-03-28T17:41:36.085 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/xircom 2024-03-28T17:41:36.085 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/xircom/xirc2ps_cs.ko 2024-03-28T17:41:36.085 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/geneve.ko 2024-03-28T17:41:36.085 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/hyperv 2024-03-28T17:41:36.085 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/hyperv/hv_netvsc.ko 2024-03-28T17:41:36.085 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ieee802154 2024-03-28T17:41:36.085 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ieee802154/fakelb.ko 2024-03-28T17:41:36.085 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ifb.ko 2024-03-28T17:41:36.085 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ipvlan 2024-03-28T17:41:36.085 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ipvlan/ipvlan.ko 2024-03-28T17:41:36.086 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/macvlan.ko 2024-03-28T17:41:36.086 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/macvtap.ko 2024-03-28T17:41:36.086 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/mdio 2024-03-28T17:41:36.086 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/mdio.ko 2024-03-28T17:41:36.086 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/mdio/mdio-bcm-unimac.ko 2024-03-28T17:41:36.086 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/mdio/mdio-bitbang.ko 2024-03-28T17:41:36.086 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/mdio/mdio-regmap.ko 2024-03-28T17:41:36.086 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/mii.ko 2024-03-28T17:41:36.086 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/net_failover.ko 2024-03-28T17:41:36.086 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/netconsole.ko 2024-03-28T17:41:36.086 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/nlmon.ko 2024-03-28T17:41:36.086 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/pcs 2024-03-28T17:41:36.086 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/pcs/pcs-lynx.ko 2024-03-28T17:41:36.086 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/pcs/pcs_xpcs.ko 2024-03-28T17:41:36.086 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/phy 2024-03-28T17:41:36.086 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/phy/amd.ko 2024-03-28T17:41:36.087 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/phy/bcm-phy-lib.ko 2024-03-28T17:41:36.087 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/phy/bcm-phy-ptp.ko 2024-03-28T17:41:36.087 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/phy/bcm7xxx.ko 2024-03-28T17:41:36.087 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/phy/bcm87xx.ko 2024-03-28T17:41:36.087 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/phy/broadcom.ko 2024-03-28T17:41:36.087 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/phy/cicada.ko 2024-03-28T17:41:36.087 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/phy/davicom.ko 2024-03-28T17:41:36.087 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/phy/dp83640.ko 2024-03-28T17:41:36.087 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/phy/et1011c.ko 2024-03-28T17:41:36.087 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/phy/icplus.ko 2024-03-28T17:41:36.087 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/phy/lxt.ko 2024-03-28T17:41:36.087 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/phy/marvell.ko 2024-03-28T17:41:36.087 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/phy/micrel.ko 2024-03-28T17:41:36.087 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/phy/national.ko 2024-03-28T17:41:36.087 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/phy/phylink.ko 2024-03-28T17:41:36.087 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/phy/qsemi.ko 2024-03-28T17:41:36.087 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/phy/realtek.ko 2024-03-28T17:41:36.088 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/phy/smsc.ko 2024-03-28T17:41:36.088 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/phy/ste10Xp.ko 2024-03-28T17:41:36.088 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/phy/vitesse.ko 2024-03-28T17:41:36.088 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ppp 2024-03-28T17:41:36.088 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ppp/bsd_comp.ko 2024-03-28T17:41:36.088 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ppp/ppp_async.ko 2024-03-28T17:41:36.088 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ppp/ppp_deflate.ko 2024-03-28T17:41:36.088 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ppp/ppp_generic.ko 2024-03-28T17:41:36.088 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ppp/ppp_mppe.ko 2024-03-28T17:41:36.088 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ppp/ppp_synctty.ko 2024-03-28T17:41:36.088 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ppp/pppoe.ko 2024-03-28T17:41:36.088 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ppp/pppox.ko 2024-03-28T17:41:36.088 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ppp/pptp.ko 2024-03-28T17:41:36.088 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/slip 2024-03-28T17:41:36.088 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/slip/slhc.ko 2024-03-28T17:41:36.088 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/slip/slip.ko 2024-03-28T17:41:36.088 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/sungem_phy.ko 2024-03-28T17:41:36.089 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/tap.ko 2024-03-28T17:41:36.089 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/team 2024-03-28T17:41:36.089 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/team/team.ko 2024-03-28T17:41:36.089 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/team/team_mode_activebackup.ko 2024-03-28T17:41:36.089 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/team/team_mode_broadcast.ko 2024-03-28T17:41:36.089 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/team/team_mode_loadbalance.ko 2024-03-28T17:41:36.089 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/team/team_mode_random.ko 2024-03-28T17:41:36.089 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/team/team_mode_roundrobin.ko 2024-03-28T17:41:36.089 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/tun.ko 2024-03-28T17:41:36.089 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/veth.ko 2024-03-28T17:41:36.089 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/virtio_net.ko 2024-03-28T17:41:36.089 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/vmxnet3 2024-03-28T17:41:36.089 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/vmxnet3/vmxnet3.ko 2024-03-28T17:41:36.089 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/vxlan 2024-03-28T17:41:36.089 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/vxlan/vxlan.ko 2024-03-28T17:41:36.090 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/xen-netback 2024-03-28T17:41:36.090 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/xen-netback/xen-netback.ko 2024-03-28T17:41:36.090 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/xen-netfront.ko 2024-03-28T17:41:36.090 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ntb 2024-03-28T17:41:36.090 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ntb/ntb.ko 2024-03-28T17:41:36.090 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/nvme 2024-03-28T17:41:36.090 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/nvme/host 2024-03-28T17:41:36.090 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/nvme/host/nvme-core.ko 2024-03-28T17:41:36.090 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/nvme/host/nvme.ko 2024-03-28T17:41:36.090 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/parport 2024-03-28T17:41:36.090 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/parport/parport.ko 2024-03-28T17:41:36.090 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/parport/parport_cs.ko 2024-03-28T17:41:36.090 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/parport/parport_pc.ko 2024-03-28T17:41:36.090 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/parport/parport_serial.ko 2024-03-28T17:41:36.090 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/pci 2024-03-28T17:41:36.090 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/pci/hotplug 2024-03-28T17:41:36.091 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/pci/hotplug/acpiphp_ibm.ko 2024-03-28T17:41:36.091 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/pci/pcie 2024-03-28T17:41:36.091 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/pci/pcie/aer_inject.ko 2024-03-28T17:41:36.091 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/pci/xen-pcifront.ko 2024-03-28T17:41:36.091 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/pcmcia 2024-03-28T17:41:36.091 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/pcmcia/i82092.ko 2024-03-28T17:41:36.091 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/pcmcia/pd6729.ko 2024-03-28T17:41:36.091 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/pcmcia/yenta_socket.ko 2024-03-28T17:41:36.091 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/pinctrl 2024-03-28T17:41:36.091 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/pinctrl/intel 2024-03-28T17:41:36.091 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/pinctrl/intel/pinctrl-cherryview.ko 2024-03-28T17:41:36.091 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform 2024-03-28T17:41:36.091 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86 2024-03-28T17:41:36.091 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/acer-wmi.ko 2024-03-28T17:41:36.091 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/acerhdf.ko 2024-03-28T17:41:36.091 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/apple-gmux.ko 2024-03-28T17:41:36.092 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/asus-laptop.ko 2024-03-28T17:41:36.092 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/asus-nb-wmi.ko 2024-03-28T17:41:36.092 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/asus-wmi.ko 2024-03-28T17:41:36.092 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/classmate-laptop.ko 2024-03-28T17:41:36.092 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/eeepc-laptop.ko 2024-03-28T17:41:36.092 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/eeepc-wmi.ko 2024-03-28T17:41:36.092 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/fujitsu-laptop.ko 2024-03-28T17:41:36.092 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/fujitsu-tablet.ko 2024-03-28T17:41:36.092 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/hdaps.ko 2024-03-28T17:41:36.092 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/intel 2024-03-28T17:41:36.092 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/intel/intel-rst.ko 2024-03-28T17:41:36.092 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/intel_ips.ko 2024-03-28T17:41:36.092 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/msi-wmi.ko 2024-03-28T17:41:36.092 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/mxm-wmi.ko 2024-03-28T17:41:36.092 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/panasonic-laptop.ko 2024-03-28T17:41:36.092 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/samsung-laptop.ko 2024-03-28T17:41:36.093 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/samsung-q10.ko 2024-03-28T17:41:36.093 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/topstar-laptop.ko 2024-03-28T17:41:36.093 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/toshiba_bluetooth.ko 2024-03-28T17:41:36.093 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/toshiba_haps.ko 2024-03-28T17:41:36.093 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/wmi-bmof.ko 2024-03-28T17:41:36.093 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/wmi.ko 2024-03-28T17:41:36.093 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/powercap 2024-03-28T17:41:36.093 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/powercap/intel_rapl_common.ko 2024-03-28T17:41:36.093 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/powercap/intel_rapl_msr.ko 2024-03-28T17:41:36.093 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/pps 2024-03-28T17:41:36.093 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/pps/clients 2024-03-28T17:41:36.093 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/pps/clients/pps-gpio.ko 2024-03-28T17:41:36.093 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/pps/clients/pps-ldisc.ko 2024-03-28T17:41:36.093 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/pps/clients/pps_parport.ko 2024-03-28T17:41:36.093 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/pps/pps_core.ko 2024-03-28T17:41:36.094 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ptp 2024-03-28T17:41:36.094 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ptp/ptp.ko 2024-03-28T17:41:36.094 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ptp/ptp_kvm.ko 2024-03-28T17:41:36.094 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc 2024-03-28T17:41:36.094 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-bq32k.ko 2024-03-28T17:41:36.094 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-ds1286.ko 2024-03-28T17:41:36.094 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-ds1307.ko 2024-03-28T17:41:36.094 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-ds1374.ko 2024-03-28T17:41:36.094 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-ds1511.ko 2024-03-28T17:41:36.094 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-ds1553.ko 2024-03-28T17:41:36.094 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-ds1672.ko 2024-03-28T17:41:36.094 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-ds1742.ko 2024-03-28T17:41:36.094 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-ds2404.ko 2024-03-28T17:41:36.094 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-ds3232.ko 2024-03-28T17:41:36.094 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-em3027.ko 2024-03-28T17:41:36.095 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-fm3130.ko 2024-03-28T17:41:36.095 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-isl12022.ko 2024-03-28T17:41:36.095 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-isl1208.ko 2024-03-28T17:41:36.095 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-m41t80.ko 2024-03-28T17:41:36.095 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-m48t35.ko 2024-03-28T17:41:36.095 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-m48t59.ko 2024-03-28T17:41:36.095 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-max6900.ko 2024-03-28T17:41:36.095 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-msm6242.ko 2024-03-28T17:41:36.095 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-pcf2127.ko 2024-03-28T17:41:36.095 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-pcf85063.ko 2024-03-28T17:41:36.095 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-pcf8523.ko 2024-03-28T17:41:36.095 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-pcf8563.ko 2024-03-28T17:41:36.095 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-pcf8583.ko 2024-03-28T17:41:36.095 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-rp5c01.ko 2024-03-28T17:41:36.095 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-rs5c372.ko 2024-03-28T17:41:36.096 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-rv3029c2.ko 2024-03-28T17:41:36.096 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-rx8025.ko 2024-03-28T17:41:36.096 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-rx8581.ko 2024-03-28T17:41:36.096 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-stk17ta8.ko 2024-03-28T17:41:36.096 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-x1205.ko 2024-03-28T17:41:36.096 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi 2024-03-28T17:41:36.096 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/3w-9xxx.ko 2024-03-28T17:41:36.096 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/3w-sas.ko 2024-03-28T17:41:36.096 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/3w-xxxx.ko 2024-03-28T17:41:36.096 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/BusLogic.ko 2024-03-28T17:41:36.096 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/a100u2w.ko 2024-03-28T17:41:36.096 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/aacraid 2024-03-28T17:41:36.096 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/aacraid/aacraid.ko 2024-03-28T17:41:36.096 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/advansys.ko 2024-03-28T17:41:36.096 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/aic7xxx 2024-03-28T17:41:36.096 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/aic7xxx/aic79xx.ko 2024-03-28T17:41:36.097 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/aic7xxx/aic7xxx.ko 2024-03-28T17:41:36.097 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/am53c974.ko 2024-03-28T17:41:36.097 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/arcmsr 2024-03-28T17:41:36.097 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/arcmsr/arcmsr.ko 2024-03-28T17:41:36.097 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/atp870u.ko 2024-03-28T17:41:36.097 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/be2iscsi 2024-03-28T17:41:36.097 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/be2iscsi/be2iscsi.ko 2024-03-28T17:41:36.097 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/bfa 2024-03-28T17:41:36.097 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/bfa/bfa.ko 2024-03-28T17:41:36.097 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/bnx2fc 2024-03-28T17:41:36.097 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/bnx2fc/bnx2fc.ko 2024-03-28T17:41:36.097 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/bnx2i 2024-03-28T17:41:36.097 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/bnx2i/bnx2i.ko 2024-03-28T17:41:36.097 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/ch.ko 2024-03-28T17:41:36.097 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/csiostor 2024-03-28T17:41:36.097 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/csiostor/csiostor.ko 2024-03-28T17:41:36.097 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/cxgbi 2024-03-28T17:41:36.098 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/cxgbi/cxgb3i 2024-03-28T17:41:36.098 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/cxgbi/cxgb3i/cxgb3i.ko 2024-03-28T17:41:36.098 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/cxgbi/cxgb4i 2024-03-28T17:41:36.098 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/cxgbi/cxgb4i/cxgb4i.ko 2024-03-28T17:41:36.098 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/cxgbi/libcxgbi.ko 2024-03-28T17:41:36.098 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/dc395x.ko 2024-03-28T17:41:36.098 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/device_handler 2024-03-28T17:41:36.098 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/device_handler/scsi_dh_alua.ko 2024-03-28T17:41:36.098 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/device_handler/scsi_dh_emc.ko 2024-03-28T17:41:36.098 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/device_handler/scsi_dh_hp_sw.ko 2024-03-28T17:41:36.098 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/device_handler/scsi_dh_rdac.ko 2024-03-28T17:41:36.098 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/dmx3191d.ko 2024-03-28T17:41:36.098 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/esas2r 2024-03-28T17:41:36.098 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/esas2r/esas2r.ko 2024-03-28T17:41:36.098 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/esp_scsi.ko 2024-03-28T17:41:36.098 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/fcoe 2024-03-28T17:41:36.099 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/fcoe/fcoe.ko 2024-03-28T17:41:36.099 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/fcoe/libfcoe.ko 2024-03-28T17:41:36.099 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/fnic 2024-03-28T17:41:36.099 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/fnic/fnic.ko 2024-03-28T17:41:36.099 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/hpsa.ko 2024-03-28T17:41:36.099 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/hptiop.ko 2024-03-28T17:41:36.099 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/hv_storvsc.ko 2024-03-28T17:41:36.099 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/initio.ko 2024-03-28T17:41:36.099 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/ipr.ko 2024-03-28T17:41:36.099 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/ips.ko 2024-03-28T17:41:36.099 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/isci 2024-03-28T17:41:36.099 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/isci/isci.ko 2024-03-28T17:41:36.099 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/iscsi_boot_sysfs.ko 2024-03-28T17:41:36.099 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/iscsi_tcp.ko 2024-03-28T17:41:36.099 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/libfc 2024-03-28T17:41:36.099 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/libfc/libfc.ko 2024-03-28T17:41:36.099 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/libiscsi.ko 2024-03-28T17:41:36.100 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/libiscsi_tcp.ko 2024-03-28T17:41:36.100 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/libsas 2024-03-28T17:41:36.100 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/libsas/libsas.ko 2024-03-28T17:41:36.100 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/lpfc 2024-03-28T17:41:36.100 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/lpfc/lpfc.ko 2024-03-28T17:41:36.100 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/megaraid 2024-03-28T17:41:36.100 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/megaraid.ko 2024-03-28T17:41:36.100 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/megaraid/megaraid_mbox.ko 2024-03-28T17:41:36.100 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/megaraid/megaraid_mm.ko 2024-03-28T17:41:36.100 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/megaraid/megaraid_sas.ko 2024-03-28T17:41:36.100 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/mpt3sas 2024-03-28T17:41:36.100 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/mpt3sas/mpt3sas.ko 2024-03-28T17:41:36.100 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/mvsas 2024-03-28T17:41:36.100 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/mvsas/mvsas.ko 2024-03-28T17:41:36.100 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/mvumi.ko 2024-03-28T17:41:36.100 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/pm8001 2024-03-28T17:41:36.100 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/pm8001/pm80xx.ko 2024-03-28T17:41:36.101 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/pmcraid.ko 2024-03-28T17:41:36.101 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/qla1280.ko 2024-03-28T17:41:36.101 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/qla2xxx 2024-03-28T17:41:36.101 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/qla2xxx/qla2xxx.ko 2024-03-28T17:41:36.101 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/qla2xxx/tcm_qla2xxx.ko 2024-03-28T17:41:36.101 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/qla4xxx 2024-03-28T17:41:36.101 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/qla4xxx/qla4xxx.ko 2024-03-28T17:41:36.101 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/raid_class.ko 2024-03-28T17:41:36.101 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/scsi_debug.ko 2024-03-28T17:41:36.101 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/scsi_transport_fc.ko 2024-03-28T17:41:36.101 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/scsi_transport_iscsi.ko 2024-03-28T17:41:36.101 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/scsi_transport_sas.ko 2024-03-28T17:41:36.101 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/scsi_transport_spi.ko 2024-03-28T17:41:36.101 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/scsi_transport_srp.ko 2024-03-28T17:41:36.101 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/ses.ko 2024-03-28T17:41:36.101 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/st.ko 2024-03-28T17:41:36.102 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/stex.ko 2024-03-28T17:41:36.102 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/sym53c8xx_2 2024-03-28T17:41:36.102 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/sym53c8xx_2/sym53c8xx.ko 2024-03-28T17:41:36.102 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/virtio_scsi.ko 2024-03-28T17:41:36.102 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/vmw_pvscsi.ko 2024-03-28T17:41:36.102 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/wd719x.ko 2024-03-28T17:41:36.102 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ssb 2024-03-28T17:41:36.102 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ssb/ssb.ko 2024-03-28T17:41:36.102 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/target 2024-03-28T17:41:36.102 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/target/iscsi 2024-03-28T17:41:36.102 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/target/iscsi/iscsi_target_mod.ko 2024-03-28T17:41:36.102 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/target/loopback 2024-03-28T17:41:36.102 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/target/loopback/tcm_loop.ko 2024-03-28T17:41:36.102 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/target/target_core_file.ko 2024-03-28T17:41:36.102 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/target/target_core_iblock.ko 2024-03-28T17:41:36.103 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/target/target_core_mod.ko 2024-03-28T17:41:36.103 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/target/target_core_pscsi.ko 2024-03-28T17:41:36.103 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/target/target_core_user.ko 2024-03-28T17:41:36.103 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/target/tcm_fc 2024-03-28T17:41:36.103 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/target/tcm_fc/tcm_fc.ko 2024-03-28T17:41:36.103 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/thermal 2024-03-28T17:41:36.103 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/thermal/intel 2024-03-28T17:41:36.103 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/thermal/intel/int340x_thermal 2024-03-28T17:41:36.103 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/thermal/intel/int340x_thermal/acpi_thermal_rel.ko 2024-03-28T17:41:36.103 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/thermal/intel/int340x_thermal/int3400_thermal.ko 2024-03-28T17:41:36.103 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/thermal/intel/int340x_thermal/int3401_thermal.ko 2024-03-28T17:41:36.103 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/thermal/intel/int340x_thermal/int3402_thermal.ko 2024-03-28T17:41:36.103 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/thermal/intel/int340x_thermal/int3403_thermal.ko 2024-03-28T17:41:36.103 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/thermal/intel/int340x_thermal/int340x_thermal_zone.ko 2024-03-28T17:41:36.103 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device.ko 2024-03-28T17:41:36.103 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci.ko 2024-03-28T17:41:36.103 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci_legacy.ko 2024-03-28T17:41:36.104 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_mbox.ko 2024-03-28T17:41:36.104 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_power_floor.ko 2024-03-28T17:41:36.104 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rapl.ko 2024-03-28T17:41:36.104 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rfim.ko 2024-03-28T17:41:36.104 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_wt_hint.ko 2024-03-28T17:41:36.104 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_wt_req.ko 2024-03-28T17:41:36.104 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/thermal/intel/intel_soc_dts_iosf.ko 2024-03-28T17:41:36.104 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/thermal/intel/intel_soc_dts_thermal.ko 2024-03-28T17:41:36.104 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/thermal/intel/x86_pkg_temp_thermal.ko 2024-03-28T17:41:36.104 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/tty 2024-03-28T17:41:36.104 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/tty/ipwireless 2024-03-28T17:41:36.104 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/tty/ipwireless/ipwireless.ko 2024-03-28T17:41:36.104 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/tty/n_gsm.ko 2024-03-28T17:41:36.104 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/tty/n_hdlc.ko 2024-03-28T17:41:36.104 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/tty/nozomi.ko 2024-03-28T17:41:36.104 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/tty/serial 2024-03-28T17:41:36.105 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/tty/serial/8250 2024-03-28T17:41:36.105 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/tty/serial/8250/serial_cs.ko 2024-03-28T17:41:36.105 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/tty/serial/arc_uart.ko 2024-03-28T17:41:36.105 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/tty/serial/jsm 2024-03-28T17:41:36.105 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/tty/serial/jsm/jsm.ko 2024-03-28T17:41:36.105 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ufs 2024-03-28T17:41:36.105 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ufs/core 2024-03-28T17:41:36.105 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ufs/core/ufshcd-core.ko 2024-03-28T17:41:36.105 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ufs/host 2024-03-28T17:41:36.105 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ufs/host/ufshcd-pci.ko 2024-03-28T17:41:36.105 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/uio 2024-03-28T17:41:36.105 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/uio/uio.ko 2024-03-28T17:41:36.105 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/uio/uio_aec.ko 2024-03-28T17:41:36.105 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/uio/uio_cif.ko 2024-03-28T17:41:36.105 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/uio/uio_pci_generic.ko 2024-03-28T17:41:36.106 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/uio/uio_sercos3.ko 2024-03-28T17:41:36.106 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb 2024-03-28T17:41:36.106 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/class 2024-03-28T17:41:36.106 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/class/cdc-acm.ko 2024-03-28T17:41:36.106 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/class/cdc-wdm.ko 2024-03-28T17:41:36.106 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/class/usblp.ko 2024-03-28T17:41:36.106 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/class/usbtmc.ko 2024-03-28T17:41:36.106 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/host 2024-03-28T17:41:36.106 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/host/sl811-hcd.ko 2024-03-28T17:41:36.106 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/image 2024-03-28T17:41:36.106 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/image/mdc800.ko 2024-03-28T17:41:36.106 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/image/microtek.ko 2024-03-28T17:41:36.106 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/misc 2024-03-28T17:41:36.106 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/misc/adutux.ko 2024-03-28T17:41:36.106 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/misc/appledisplay.ko 2024-03-28T17:41:36.106 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/misc/emi26.ko 2024-03-28T17:41:36.107 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/misc/emi62.ko 2024-03-28T17:41:36.107 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/misc/ezusb.ko 2024-03-28T17:41:36.107 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/misc/idmouse.ko 2024-03-28T17:41:36.107 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/misc/iowarrior.ko 2024-03-28T17:41:36.107 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/misc/isight_firmware.ko 2024-03-28T17:41:36.107 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/misc/ldusb.ko 2024-03-28T17:41:36.107 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/misc/legousbtower.ko 2024-03-28T17:41:36.107 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/misc/sisusbvga 2024-03-28T17:41:36.107 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/misc/sisusbvga/sisusbvga.ko 2024-03-28T17:41:36.107 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/misc/trancevibrator.ko 2024-03-28T17:41:36.107 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/misc/usb3503.ko 2024-03-28T17:41:36.107 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/misc/usblcd.ko 2024-03-28T17:41:36.107 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/misc/usbsevseg.ko 2024-03-28T17:41:36.107 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/misc/uss720.ko 2024-03-28T17:41:36.108 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/misc/yurex.ko 2024-03-28T17:41:36.108 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/phy 2024-03-28T17:41:36.108 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/phy/phy-generic.ko 2024-03-28T17:41:36.108 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial 2024-03-28T17:41:36.108 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/aircable.ko 2024-03-28T17:41:36.108 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/ark3116.ko 2024-03-28T17:41:36.108 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/belkin_sa.ko 2024-03-28T17:41:36.108 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/ch341.ko 2024-03-28T17:41:36.108 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/cp210x.ko 2024-03-28T17:41:36.108 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/cyberjack.ko 2024-03-28T17:41:36.108 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/cypress_m8.ko 2024-03-28T17:41:36.108 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/digi_acceleport.ko 2024-03-28T17:41:36.108 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/empeg.ko 2024-03-28T17:41:36.108 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/ftdi_sio.ko 2024-03-28T17:41:36.108 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/garmin_gps.ko 2024-03-28T17:41:36.108 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/io_edgeport.ko 2024-03-28T17:41:36.109 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/io_ti.ko 2024-03-28T17:41:36.109 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/ipaq.ko 2024-03-28T17:41:36.109 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/ipw.ko 2024-03-28T17:41:36.109 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/ir-usb.ko 2024-03-28T17:41:36.109 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/iuu_phoenix.ko 2024-03-28T17:41:36.109 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/keyspan.ko 2024-03-28T17:41:36.109 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/keyspan_pda.ko 2024-03-28T17:41:36.109 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/kl5kusb105.ko 2024-03-28T17:41:36.109 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/kobil_sct.ko 2024-03-28T17:41:36.109 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/mct_u232.ko 2024-03-28T17:41:36.109 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/mos7720.ko 2024-03-28T17:41:36.109 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/mos7840.ko 2024-03-28T17:41:36.109 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/navman.ko 2024-03-28T17:41:36.109 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/omninet.ko 2024-03-28T17:41:36.109 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/opticon.ko 2024-03-28T17:41:36.109 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/option.ko 2024-03-28T17:41:36.109 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/oti6858.ko 2024-03-28T17:41:36.110 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/pl2303.ko 2024-03-28T17:41:36.110 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/qcaux.ko 2024-03-28T17:41:36.110 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/qcserial.ko 2024-03-28T17:41:36.110 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/quatech2.ko 2024-03-28T17:41:36.110 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/safe_serial.ko 2024-03-28T17:41:36.110 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/sierra.ko 2024-03-28T17:41:36.110 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/spcp8x5.ko 2024-03-28T17:41:36.110 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/ssu100.ko 2024-03-28T17:41:36.110 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/symbolserial.ko 2024-03-28T17:41:36.110 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/ti_usb_3410_5052.ko 2024-03-28T17:41:36.110 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/usb-serial-simple.ko 2024-03-28T17:41:36.110 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/usb_debug.ko 2024-03-28T17:41:36.110 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/usb_wwan.ko 2024-03-28T17:41:36.110 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/visor.ko 2024-03-28T17:41:36.110 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/whiteheat.ko 2024-03-28T17:41:36.110 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/xsens_mt.ko 2024-03-28T17:41:36.111 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/storage 2024-03-28T17:41:36.111 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/storage/uas.ko 2024-03-28T17:41:36.111 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/storage/ums-alauda.ko 2024-03-28T17:41:36.111 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/storage/ums-cypress.ko 2024-03-28T17:41:36.111 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/storage/ums-datafab.ko 2024-03-28T17:41:36.111 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/storage/ums-eneub6250.ko 2024-03-28T17:41:36.111 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/storage/ums-freecom.ko 2024-03-28T17:41:36.111 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/storage/ums-isd200.ko 2024-03-28T17:41:36.111 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/storage/ums-jumpshot.ko 2024-03-28T17:41:36.111 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/storage/ums-karma.ko 2024-03-28T17:41:36.111 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/storage/ums-onetouch.ko 2024-03-28T17:41:36.111 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/storage/ums-realtek.ko 2024-03-28T17:41:36.111 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/storage/ums-sddr09.ko 2024-03-28T17:41:36.111 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/storage/ums-sddr55.ko 2024-03-28T17:41:36.111 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/storage/ums-usbat.ko 2024-03-28T17:41:36.111 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/storage/usb-storage.ko 2024-03-28T17:41:36.111 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/usbip 2024-03-28T17:41:36.112 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/usbip/usbip-core.ko 2024-03-28T17:41:36.112 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/usbip/usbip-host.ko 2024-03-28T17:41:36.112 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/usbip/vhci-hcd.ko 2024-03-28T17:41:36.112 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/vfio 2024-03-28T17:41:36.112 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/vfio/pci 2024-03-28T17:41:36.112 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/vfio/pci/vfio-pci-core.ko 2024-03-28T17:41:36.112 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/vfio/pci/vfio-pci.ko 2024-03-28T17:41:36.112 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/vfio/vfio.ko 2024-03-28T17:41:36.112 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/vfio/vfio_iommu_type1.ko 2024-03-28T17:41:36.112 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/vhost 2024-03-28T17:41:36.112 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/vhost/vhost.ko 2024-03-28T17:41:36.112 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/vhost/vhost_iotlb.ko 2024-03-28T17:41:36.112 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/vhost/vhost_net.ko 2024-03-28T17:41:36.112 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/vhost/vhost_scsi.ko 2024-03-28T17:41:36.112 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/video 2024-03-28T17:41:36.112 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/video/backlight 2024-03-28T17:41:36.113 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/video/backlight/apple_bl.ko 2024-03-28T17:41:36.113 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/video/backlight/lcd.ko 2024-03-28T17:41:36.113 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/video/backlight/platform_lcd.ko 2024-03-28T17:41:36.113 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/virtio 2024-03-28T17:41:36.113 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/virtio/virtio.ko 2024-03-28T17:41:36.113 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/virtio/virtio_balloon.ko 2024-03-28T17:41:36.113 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/virtio/virtio_mmio.ko 2024-03-28T17:41:36.113 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/virtio/virtio_pci.ko 2024-03-28T17:41:36.113 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/virtio/virtio_pci_legacy_dev.ko 2024-03-28T17:41:36.113 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/virtio/virtio_pci_modern_dev.ko 2024-03-28T17:41:36.113 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/virtio/virtio_ring.ko 2024-03-28T17:41:36.113 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/w1 2024-03-28T17:41:36.113 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/w1/masters 2024-03-28T17:41:36.113 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/w1/masters/ds2482.ko 2024-03-28T17:41:36.113 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/w1/masters/ds2490.ko 2024-03-28T17:41:36.113 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/w1/slaves 2024-03-28T17:41:36.114 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/w1/slaves/w1_ds2406.ko 2024-03-28T17:41:36.114 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/w1/slaves/w1_ds2408.ko 2024-03-28T17:41:36.114 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/w1/slaves/w1_ds2413.ko 2024-03-28T17:41:36.114 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/w1/slaves/w1_ds2423.ko 2024-03-28T17:41:36.114 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/w1/slaves/w1_ds2431.ko 2024-03-28T17:41:36.114 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/w1/slaves/w1_ds2433.ko 2024-03-28T17:41:36.114 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/w1/slaves/w1_ds2780.ko 2024-03-28T17:41:36.114 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/w1/slaves/w1_ds2781.ko 2024-03-28T17:41:36.114 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/w1/slaves/w1_ds28e04.ko 2024-03-28T17:41:36.114 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/w1/slaves/w1_smem.ko 2024-03-28T17:41:36.114 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/w1/slaves/w1_therm.ko 2024-03-28T17:41:36.114 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/w1/wire.ko 2024-03-28T17:41:36.114 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog 2024-03-28T17:41:36.114 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/alim1535_wdt.ko 2024-03-28T17:41:36.114 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/alim7101_wdt.ko 2024-03-28T17:41:36.114 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/f71808e_wdt.ko 2024-03-28T17:41:36.115 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/hpwdt.ko 2024-03-28T17:41:36.115 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/i6300esb.ko 2024-03-28T17:41:36.115 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/iTCO_vendor_support.ko 2024-03-28T17:41:36.115 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/iTCO_wdt.ko 2024-03-28T17:41:36.115 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/ib700wdt.ko 2024-03-28T17:41:36.115 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/ibmasr.ko 2024-03-28T17:41:36.115 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/ie6xx_wdt.ko 2024-03-28T17:41:36.115 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/it8712f_wdt.ko 2024-03-28T17:41:36.115 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/it87_wdt.ko 2024-03-28T17:41:36.115 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/machzwd.ko 2024-03-28T17:41:36.115 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/nv_tco.ko 2024-03-28T17:41:36.115 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/pcwd_pci.ko 2024-03-28T17:41:36.115 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/pcwd_usb.ko 2024-03-28T17:41:36.115 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/sbc_fitpc2_wdt.ko 2024-03-28T17:41:36.115 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/sch311x_wdt.ko 2024-03-28T17:41:36.116 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/softdog.ko 2024-03-28T17:41:36.116 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/sp5100_tco.ko 2024-03-28T17:41:36.116 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/via_wdt.ko 2024-03-28T17:41:36.116 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/w83627hf_wdt.ko 2024-03-28T17:41:36.116 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/w83877f_wdt.ko 2024-03-28T17:41:36.116 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/w83977f_wdt.ko 2024-03-28T17:41:36.116 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/wdt_pci.ko 2024-03-28T17:41:36.116 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/xen_wdt.ko 2024-03-28T17:41:36.116 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/xen 2024-03-28T17:41:36.116 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/xen/xen-acpi-processor.ko 2024-03-28T17:41:36.116 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/xen/xen-evtchn.ko 2024-03-28T17:41:36.116 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/xen/xen-gntalloc.ko 2024-03-28T17:41:36.116 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/xen/xen-gntdev.ko 2024-03-28T17:41:36.116 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/xen/xen-pciback 2024-03-28T17:41:36.117 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/xen/xen-pciback/xen-pciback.ko 2024-03-28T17:41:36.117 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/xen/xen-privcmd.ko 2024-03-28T17:41:36.117 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/xen/xenfs 2024-03-28T17:41:36.117 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/xen/xenfs/xenfs.ko 2024-03-28T17:41:36.117 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs 2024-03-28T17:41:36.117 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/9p 2024-03-28T17:41:36.117 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/9p/9p.ko 2024-03-28T17:41:36.117 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/affs 2024-03-28T17:41:36.117 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/affs/affs.ko 2024-03-28T17:41:36.117 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/befs 2024-03-28T17:41:36.117 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/befs/befs.ko 2024-03-28T17:41:36.117 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/binfmt_misc.ko 2024-03-28T17:41:36.117 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/btrfs 2024-03-28T17:41:36.117 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/btrfs/btrfs.ko 2024-03-28T17:41:36.117 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/cachefiles 2024-03-28T17:41:36.117 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/cachefiles/cachefiles.ko 2024-03-28T17:41:36.118 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/ceph 2024-03-28T17:41:36.118 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/ceph/ceph.ko 2024-03-28T17:41:36.118 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/coda 2024-03-28T17:41:36.118 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/coda/coda.ko 2024-03-28T17:41:36.118 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/cramfs 2024-03-28T17:41:36.118 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/cramfs/cramfs.ko 2024-03-28T17:41:36.118 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/dlm 2024-03-28T17:41:36.118 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/dlm/dlm.ko 2024-03-28T17:41:36.118 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/ecryptfs 2024-03-28T17:41:36.118 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/ecryptfs/ecryptfs.ko 2024-03-28T17:41:36.118 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/f2fs 2024-03-28T17:41:36.118 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/f2fs/f2fs.ko 2024-03-28T17:41:36.118 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/fat 2024-03-28T17:41:36.118 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/fat/fat.ko 2024-03-28T17:41:36.118 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/fat/msdos.ko 2024-03-28T17:41:36.119 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/fat/vfat.ko 2024-03-28T17:41:36.119 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/fuse 2024-03-28T17:41:36.119 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/fuse/cuse.ko 2024-03-28T17:41:36.119 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/fuse/fuse.ko 2024-03-28T17:41:36.119 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/gfs2 2024-03-28T17:41:36.119 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/gfs2/gfs2.ko 2024-03-28T17:41:36.119 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/hfs 2024-03-28T17:41:36.119 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/hfs/hfs.ko 2024-03-28T17:41:36.119 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/hfsplus 2024-03-28T17:41:36.119 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/hfsplus/hfsplus.ko 2024-03-28T17:41:36.119 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/isofs 2024-03-28T17:41:36.119 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/isofs/isofs.ko 2024-03-28T17:41:36.119 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/jfs 2024-03-28T17:41:36.119 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/jfs/jfs.ko 2024-03-28T17:41:36.119 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/lockd 2024-03-28T17:41:36.119 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/lockd/lockd.ko 2024-03-28T17:41:36.120 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/minix 2024-03-28T17:41:36.120 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/minix/minix.ko 2024-03-28T17:41:36.120 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/netfs 2024-03-28T17:41:36.120 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/netfs/netfs.ko 2024-03-28T17:41:36.120 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nfs 2024-03-28T17:41:36.120 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nfs/blocklayout 2024-03-28T17:41:36.120 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nfs/blocklayout/blocklayoutdriver.ko 2024-03-28T17:41:36.120 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nfs/filelayout 2024-03-28T17:41:36.120 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nfs/filelayout/nfs_layout_nfsv41_files.ko 2024-03-28T17:41:36.120 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nfs/flexfilelayout 2024-03-28T17:41:36.120 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nfs/flexfilelayout/nfs_layout_flexfiles.ko 2024-03-28T17:41:36.120 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nfs/nfs.ko 2024-03-28T17:41:36.120 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nfs/nfsv3.ko 2024-03-28T17:41:36.120 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nfs/nfsv4.ko 2024-03-28T17:41:36.120 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nfs_common 2024-03-28T17:41:36.120 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nfs_common/grace.ko 2024-03-28T17:41:36.120 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nfs_common/nfs_acl.ko 2024-03-28T17:41:36.121 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nfsd 2024-03-28T17:41:36.121 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nfsd/nfsd.ko 2024-03-28T17:41:36.121 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nilfs2 2024-03-28T17:41:36.121 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nilfs2/nilfs2.ko 2024-03-28T17:41:36.121 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls 2024-03-28T17:41:36.121 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/mac-celtic.ko 2024-03-28T17:41:36.121 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/mac-centeuro.ko 2024-03-28T17:41:36.121 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/mac-croatian.ko 2024-03-28T17:41:36.121 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/mac-cyrillic.ko 2024-03-28T17:41:36.121 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/mac-gaelic.ko 2024-03-28T17:41:36.121 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/mac-greek.ko 2024-03-28T17:41:36.121 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/mac-iceland.ko 2024-03-28T17:41:36.121 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/mac-inuit.ko 2024-03-28T17:41:36.121 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/mac-roman.ko 2024-03-28T17:41:36.121 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/mac-romanian.ko 2024-03-28T17:41:36.122 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/mac-turkish.ko 2024-03-28T17:41:36.122 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_cp1250.ko 2024-03-28T17:41:36.122 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_cp1251.ko 2024-03-28T17:41:36.122 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_cp1255.ko 2024-03-28T17:41:36.122 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_cp737.ko 2024-03-28T17:41:36.122 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_cp775.ko 2024-03-28T17:41:36.122 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_cp850.ko 2024-03-28T17:41:36.122 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_cp852.ko 2024-03-28T17:41:36.122 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_cp855.ko 2024-03-28T17:41:36.122 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_cp857.ko 2024-03-28T17:41:36.122 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_cp860.ko 2024-03-28T17:41:36.122 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_cp861.ko 2024-03-28T17:41:36.122 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_cp862.ko 2024-03-28T17:41:36.122 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_cp863.ko 2024-03-28T17:41:36.122 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_cp864.ko 2024-03-28T17:41:36.123 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_cp865.ko 2024-03-28T17:41:36.123 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_cp866.ko 2024-03-28T17:41:36.123 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_cp869.ko 2024-03-28T17:41:36.123 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_cp874.ko 2024-03-28T17:41:36.123 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_cp932.ko 2024-03-28T17:41:36.123 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_cp936.ko 2024-03-28T17:41:36.123 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_cp949.ko 2024-03-28T17:41:36.123 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_cp950.ko 2024-03-28T17:41:36.123 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_euc-jp.ko 2024-03-28T17:41:36.123 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_iso8859-1.ko 2024-03-28T17:41:36.123 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_iso8859-13.ko 2024-03-28T17:41:36.123 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_iso8859-14.ko 2024-03-28T17:41:36.123 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_iso8859-15.ko 2024-03-28T17:41:36.123 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_iso8859-2.ko 2024-03-28T17:41:36.123 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_iso8859-3.ko 2024-03-28T17:41:36.124 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_iso8859-4.ko 2024-03-28T17:41:36.124 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_iso8859-5.ko 2024-03-28T17:41:36.124 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_iso8859-6.ko 2024-03-28T17:41:36.124 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_iso8859-7.ko 2024-03-28T17:41:36.124 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_iso8859-9.ko 2024-03-28T17:41:36.124 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_koi8-r.ko 2024-03-28T17:41:36.124 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_koi8-ru.ko 2024-03-28T17:41:36.124 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_koi8-u.ko 2024-03-28T17:41:36.124 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_ucs2_utils.ko 2024-03-28T17:41:36.124 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_utf8.ko 2024-03-28T17:41:36.124 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/ocfs2 2024-03-28T17:41:36.124 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/ocfs2/cluster 2024-03-28T17:41:36.124 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/ocfs2/cluster/ocfs2_nodemanager.ko 2024-03-28T17:41:36.124 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/ocfs2/dlm 2024-03-28T17:41:36.124 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/ocfs2/dlm/ocfs2_dlm.ko 2024-03-28T17:41:36.124 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/ocfs2/dlmfs 2024-03-28T17:41:36.124 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/ocfs2/dlmfs/ocfs2_dlmfs.ko 2024-03-28T17:41:36.125 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/ocfs2/ocfs2.ko 2024-03-28T17:41:36.125 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/ocfs2/ocfs2_stack_o2cb.ko 2024-03-28T17:41:36.125 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/ocfs2/ocfs2_stack_user.ko 2024-03-28T17:41:36.125 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/ocfs2/ocfs2_stackglue.ko 2024-03-28T17:41:36.125 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/overlayfs 2024-03-28T17:41:36.125 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/overlayfs/overlay.ko 2024-03-28T17:41:36.125 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/pstore 2024-03-28T17:41:36.125 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/pstore/ramoops.ko 2024-03-28T17:41:36.125 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/reiserfs 2024-03-28T17:41:36.125 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/reiserfs/reiserfs.ko 2024-03-28T17:41:36.125 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/romfs 2024-03-28T17:41:36.125 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/romfs/romfs.ko 2024-03-28T17:41:36.125 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/smb 2024-03-28T17:41:36.125 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/smb/client 2024-03-28T17:41:36.125 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/smb/client/cifs.ko 2024-03-28T17:41:36.125 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/smb/common 2024-03-28T17:41:36.125 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/smb/common/cifs_arc4.ko 2024-03-28T17:41:36.126 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/smb/common/cifs_md4.ko 2024-03-28T17:41:36.126 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/squashfs 2024-03-28T17:41:36.126 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/squashfs/squashfs.ko 2024-03-28T17:41:36.126 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/sysv 2024-03-28T17:41:36.126 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/sysv/sysv.ko 2024-03-28T17:41:36.126 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/udf 2024-03-28T17:41:36.126 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/udf/udf.ko 2024-03-28T17:41:36.126 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/ufs 2024-03-28T17:41:36.126 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/ufs/ufs.ko 2024-03-28T17:41:36.126 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/xfs 2024-03-28T17:41:36.126 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/xfs/xfs.ko 2024-03-28T17:41:36.126 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/kernel 2024-03-28T17:41:36.126 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/kernel/rcu 2024-03-28T17:41:36.126 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/kernel/rcu/rcutorture.ko 2024-03-28T17:41:36.126 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/kernel/torture.ko 2024-03-28T17:41:36.126 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/kernel/trace 2024-03-28T17:41:36.127 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/kernel/trace/ring_buffer_benchmark.ko 2024-03-28T17:41:36.127 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/lib 2024-03-28T17:41:36.127 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/asn1_encoder.ko 2024-03-28T17:41:36.127 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/crc-itu-t.ko 2024-03-28T17:41:36.127 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/crc8.ko 2024-03-28T17:41:36.127 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/crypto 2024-03-28T17:41:36.127 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/crypto/libarc4.ko 2024-03-28T17:41:36.127 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/crypto/libdes.ko 2024-03-28T17:41:36.127 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/libcrc32c.ko 2024-03-28T17:41:36.127 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/lru_cache.ko 2024-03-28T17:41:36.127 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/lz4 2024-03-28T17:41:36.127 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/lz4/lz4_compress.ko 2024-03-28T17:41:36.127 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/lz4/lz4hc_compress.ko 2024-03-28T17:41:36.127 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/math 2024-03-28T17:41:36.127 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/math/cordic.ko 2024-03-28T17:41:36.127 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/raid6 2024-03-28T17:41:36.128 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/raid6/raid6_pq.ko 2024-03-28T17:41:36.128 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/reed_solomon 2024-03-28T17:41:36.128 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/reed_solomon/reed_solomon.ko 2024-03-28T17:41:36.128 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/ts_bm.ko 2024-03-28T17:41:36.128 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/ts_fsm.ko 2024-03-28T17:41:36.128 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/ts_kmp.ko 2024-03-28T17:41:36.128 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/zstd 2024-03-28T17:41:36.128 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/zstd/zstd_compress.ko 2024-03-28T17:41:36.128 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/mm 2024-03-28T17:41:36.128 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/mm/hwpoison-inject.ko 2024-03-28T17:41:36.128 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net 2024-03-28T17:41:36.128 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/6lowpan 2024-03-28T17:41:36.128 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/6lowpan/6lowpan.ko 2024-03-28T17:41:36.128 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/6lowpan/nhc_dest.ko 2024-03-28T17:41:36.128 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/6lowpan/nhc_fragment.ko 2024-03-28T17:41:36.128 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/6lowpan/nhc_hop.ko 2024-03-28T17:41:36.129 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/6lowpan/nhc_ipv6.ko 2024-03-28T17:41:36.129 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/6lowpan/nhc_mobility.ko 2024-03-28T17:41:36.129 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/6lowpan/nhc_routing.ko 2024-03-28T17:41:36.129 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/6lowpan/nhc_udp.ko 2024-03-28T17:41:36.129 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/802 2024-03-28T17:41:36.129 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/802/garp.ko 2024-03-28T17:41:36.129 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/802/mrp.ko 2024-03-28T17:41:36.129 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/802/p8022.ko 2024-03-28T17:41:36.129 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/802/psnap.ko 2024-03-28T17:41:36.129 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/802/stp.ko 2024-03-28T17:41:36.129 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/8021q 2024-03-28T17:41:36.129 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/8021q/8021q.ko 2024-03-28T17:41:36.129 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/9p 2024-03-28T17:41:36.129 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/9p/9pnet.ko 2024-03-28T17:41:36.129 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/9p/9pnet_fd.ko 2024-03-28T17:41:36.129 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/9p/9pnet_rdma.ko 2024-03-28T17:41:36.129 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/9p/9pnet_virtio.ko 2024-03-28T17:41:36.130 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/batman-adv 2024-03-28T17:41:36.130 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/batman-adv/batman-adv.ko 2024-03-28T17:41:36.130 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge 2024-03-28T17:41:36.130 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/br_netfilter.ko 2024-03-28T17:41:36.130 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/bridge.ko 2024-03-28T17:41:36.130 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter 2024-03-28T17:41:36.130 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/ebt_802_3.ko 2024-03-28T17:41:36.130 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/ebt_among.ko 2024-03-28T17:41:36.130 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/ebt_arp.ko 2024-03-28T17:41:36.130 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/ebt_arpreply.ko 2024-03-28T17:41:36.130 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/ebt_dnat.ko 2024-03-28T17:41:36.130 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/ebt_ip.ko 2024-03-28T17:41:36.130 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/ebt_ip6.ko 2024-03-28T17:41:36.130 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/ebt_limit.ko 2024-03-28T17:41:36.130 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/ebt_log.ko 2024-03-28T17:41:36.130 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/ebt_mark.ko 2024-03-28T17:41:36.131 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/ebt_mark_m.ko 2024-03-28T17:41:36.131 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/ebt_nflog.ko 2024-03-28T17:41:36.131 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/ebt_pkttype.ko 2024-03-28T17:41:36.131 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/ebt_redirect.ko 2024-03-28T17:41:36.131 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/ebt_snat.ko 2024-03-28T17:41:36.131 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/ebt_stp.ko 2024-03-28T17:41:36.131 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/ebt_vlan.ko 2024-03-28T17:41:36.131 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/ebtable_broute.ko 2024-03-28T17:41:36.131 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/ebtable_filter.ko 2024-03-28T17:41:36.131 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/ebtable_nat.ko 2024-03-28T17:41:36.131 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/ebtables.ko 2024-03-28T17:41:36.131 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/nf_conntrack_bridge.ko 2024-03-28T17:41:36.131 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/nft_meta_bridge.ko 2024-03-28T17:41:36.131 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/nft_reject_bridge.ko 2024-03-28T17:41:36.131 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ceph 2024-03-28T17:41:36.131 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ceph/libceph.ko 2024-03-28T17:41:36.132 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/core 2024-03-28T17:41:36.132 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/core/failover.ko 2024-03-28T17:41:36.132 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/core/pktgen.ko 2024-03-28T17:41:36.132 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/dccp 2024-03-28T17:41:36.132 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/dccp/dccp.ko 2024-03-28T17:41:36.132 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/dccp/dccp_diag.ko 2024-03-28T17:41:36.132 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/dccp/dccp_ipv4.ko 2024-03-28T17:41:36.132 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/dccp/dccp_ipv6.ko 2024-03-28T17:41:36.132 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/dns_resolver 2024-03-28T17:41:36.132 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/dns_resolver/dns_resolver.ko 2024-03-28T17:41:36.132 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/dsa 2024-03-28T17:41:36.132 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/dsa/dsa_core.ko 2024-03-28T17:41:36.132 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/dsa/tag_brcm.ko 2024-03-28T17:41:36.132 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/dsa/tag_dsa.ko 2024-03-28T17:41:36.132 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/dsa/tag_none.ko 2024-03-28T17:41:36.132 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/dsa/tag_trailer.ko 2024-03-28T17:41:36.132 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ieee802154 2024-03-28T17:41:36.133 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ieee802154/6lowpan 2024-03-28T17:41:36.133 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ieee802154/6lowpan/ieee802154_6lowpan.ko 2024-03-28T17:41:36.133 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ieee802154/ieee802154.ko 2024-03-28T17:41:36.133 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ieee802154/ieee802154_socket.ko 2024-03-28T17:41:36.133 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4 2024-03-28T17:41:36.133 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/ah4.ko 2024-03-28T17:41:36.133 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/esp4.ko 2024-03-28T17:41:36.133 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/fou.ko 2024-03-28T17:41:36.133 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/gre.ko 2024-03-28T17:41:36.133 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/inet_diag.ko 2024-03-28T17:41:36.133 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/ip_gre.ko 2024-03-28T17:41:36.133 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/ip_tunnel.ko 2024-03-28T17:41:36.133 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/ip_vti.ko 2024-03-28T17:41:36.133 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/ipcomp.ko 2024-03-28T17:41:36.133 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/ipip.ko 2024-03-28T17:41:36.133 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter 2024-03-28T17:41:36.133 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter/arp_tables.ko 2024-03-28T17:41:36.134 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter/arpt_mangle.ko 2024-03-28T17:41:36.134 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter/arptable_filter.ko 2024-03-28T17:41:36.134 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter/ipt_ECN.ko 2024-03-28T17:41:36.134 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter/ipt_SYNPROXY.ko 2024-03-28T17:41:36.134 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter/ipt_ah.ko 2024-03-28T17:41:36.134 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter/ipt_rpfilter.ko 2024-03-28T17:41:36.134 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter/iptable_mangle.ko 2024-03-28T17:41:36.134 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter/iptable_nat.ko 2024-03-28T17:41:36.134 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter/iptable_raw.ko 2024-03-28T17:41:36.134 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter/iptable_security.ko 2024-03-28T17:41:36.134 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter/nf_defrag_ipv4.ko 2024-03-28T17:41:36.134 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter/nf_dup_ipv4.ko 2024-03-28T17:41:36.134 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter/nf_nat_h323.ko 2024-03-28T17:41:36.134 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter/nf_nat_pptp.ko 2024-03-28T17:41:36.134 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter/nf_nat_snmp_basic.ko 2024-03-28T17:41:36.134 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter/nf_socket_ipv4.ko 2024-03-28T17:41:36.135 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter/nf_tproxy_ipv4.ko 2024-03-28T17:41:36.135 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter/nft_dup_ipv4.ko 2024-03-28T17:41:36.135 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter/nft_fib_ipv4.ko 2024-03-28T17:41:36.135 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter/nft_reject_ipv4.ko 2024-03-28T17:41:36.135 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/tcp_bic.ko 2024-03-28T17:41:36.135 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/tcp_dctcp.ko 2024-03-28T17:41:36.135 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/tcp_diag.ko 2024-03-28T17:41:36.135 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/tcp_highspeed.ko 2024-03-28T17:41:36.135 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/tcp_htcp.ko 2024-03-28T17:41:36.135 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/tcp_hybla.ko 2024-03-28T17:41:36.135 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/tcp_illinois.ko 2024-03-28T17:41:36.135 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/tcp_lp.ko 2024-03-28T17:41:36.135 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/tcp_scalable.ko 2024-03-28T17:41:36.135 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/tcp_vegas.ko 2024-03-28T17:41:36.135 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/tcp_veno.ko 2024-03-28T17:41:36.136 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/tcp_westwood.ko 2024-03-28T17:41:36.136 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/tcp_yeah.ko 2024-03-28T17:41:36.136 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/tunnel4.ko 2024-03-28T17:41:36.136 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/udp_diag.ko 2024-03-28T17:41:36.136 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/udp_tunnel.ko 2024-03-28T17:41:36.136 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/xfrm4_tunnel.ko 2024-03-28T17:41:36.136 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6 2024-03-28T17:41:36.136 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/ah6.ko 2024-03-28T17:41:36.136 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/esp6.ko 2024-03-28T17:41:36.136 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/fou6.ko 2024-03-28T17:41:36.136 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/ip6_tunnel.ko 2024-03-28T17:41:36.136 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/ip6_udp_tunnel.ko 2024-03-28T17:41:36.136 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/ip6_vti.ko 2024-03-28T17:41:36.136 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/ipcomp6.ko 2024-03-28T17:41:36.136 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter 2024-03-28T17:41:36.136 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/ip6_tables.ko 2024-03-28T17:41:36.137 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/ip6t_REJECT.ko 2024-03-28T17:41:36.137 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/ip6t_SYNPROXY.ko 2024-03-28T17:41:36.137 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/ip6t_ah.ko 2024-03-28T17:41:36.137 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/ip6t_eui64.ko 2024-03-28T17:41:36.137 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/ip6t_frag.ko 2024-03-28T17:41:36.137 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/ip6t_hbh.ko 2024-03-28T17:41:36.137 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/ip6t_ipv6header.ko 2024-03-28T17:41:36.137 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/ip6t_mh.ko 2024-03-28T17:41:36.137 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/ip6t_rpfilter.ko 2024-03-28T17:41:36.137 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/ip6t_rt.ko 2024-03-28T17:41:36.137 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/ip6table_filter.ko 2024-03-28T17:41:36.137 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/ip6table_mangle.ko 2024-03-28T17:41:36.137 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/ip6table_nat.ko 2024-03-28T17:41:36.137 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/ip6table_raw.ko 2024-03-28T17:41:36.137 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/ip6table_security.ko 2024-03-28T17:41:36.138 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/nf_defrag_ipv6.ko 2024-03-28T17:41:36.138 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/nf_dup_ipv6.ko 2024-03-28T17:41:36.138 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/nf_reject_ipv6.ko 2024-03-28T17:41:36.138 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/nf_socket_ipv6.ko 2024-03-28T17:41:36.138 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/nf_tproxy_ipv6.ko 2024-03-28T17:41:36.138 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/nft_dup_ipv6.ko 2024-03-28T17:41:36.138 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/nft_fib_ipv6.ko 2024-03-28T17:41:36.138 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/nft_reject_ipv6.ko 2024-03-28T17:41:36.138 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/sit.ko 2024-03-28T17:41:36.138 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/tunnel6.ko 2024-03-28T17:41:36.138 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/xfrm6_tunnel.ko 2024-03-28T17:41:36.138 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/key 2024-03-28T17:41:36.138 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/key/af_key.ko 2024-03-28T17:41:36.138 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/l2tp 2024-03-28T17:41:36.138 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/l2tp/l2tp_core.ko 2024-03-28T17:41:36.138 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/l2tp/l2tp_debugfs.ko 2024-03-28T17:41:36.138 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/l2tp/l2tp_eth.ko 2024-03-28T17:41:36.139 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/l2tp/l2tp_ip.ko 2024-03-28T17:41:36.139 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/l2tp/l2tp_ip6.ko 2024-03-28T17:41:36.139 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/l2tp/l2tp_netlink.ko 2024-03-28T17:41:36.139 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/l2tp/l2tp_ppp.ko 2024-03-28T17:41:36.139 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/llc 2024-03-28T17:41:36.139 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/llc/llc.ko 2024-03-28T17:41:36.139 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/mac802154 2024-03-28T17:41:36.139 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/mac802154/mac802154.ko 2024-03-28T17:41:36.139 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/mpls 2024-03-28T17:41:36.139 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/mpls/mpls_gso.ko 2024-03-28T17:41:36.139 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter 2024-03-28T17:41:36.139 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipset 2024-03-28T17:41:36.139 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipset/ip_set.ko 2024-03-28T17:41:36.139 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipset/ip_set_bitmap_ip.ko 2024-03-28T17:41:36.140 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipset/ip_set_bitmap_ipmac.ko 2024-03-28T17:41:36.140 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipset/ip_set_bitmap_port.ko 2024-03-28T17:41:36.140 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipset/ip_set_hash_ip.ko 2024-03-28T17:41:36.140 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipset/ip_set_hash_ipmark.ko 2024-03-28T17:41:36.140 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipset/ip_set_hash_ipport.ko 2024-03-28T17:41:36.140 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipset/ip_set_hash_ipportip.ko 2024-03-28T17:41:36.140 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipset/ip_set_hash_ipportnet.ko 2024-03-28T17:41:36.140 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipset/ip_set_hash_mac.ko 2024-03-28T17:41:36.140 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipset/ip_set_hash_net.ko 2024-03-28T17:41:36.140 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipset/ip_set_hash_netiface.ko 2024-03-28T17:41:36.140 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipset/ip_set_hash_netnet.ko 2024-03-28T17:41:36.140 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipset/ip_set_hash_netport.ko 2024-03-28T17:41:36.140 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipset/ip_set_hash_netportnet.ko 2024-03-28T17:41:36.140 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipset/ip_set_list_set.ko 2024-03-28T17:41:36.140 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipvs 2024-03-28T17:41:36.140 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipvs/ip_vs.ko 2024-03-28T17:41:36.141 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipvs/ip_vs_dh.ko 2024-03-28T17:41:36.141 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipvs/ip_vs_fo.ko 2024-03-28T17:41:36.141 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipvs/ip_vs_ftp.ko 2024-03-28T17:41:36.141 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipvs/ip_vs_lblc.ko 2024-03-28T17:41:36.141 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipvs/ip_vs_lblcr.ko 2024-03-28T17:41:36.141 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipvs/ip_vs_lc.ko 2024-03-28T17:41:36.141 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipvs/ip_vs_nq.ko 2024-03-28T17:41:36.141 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipvs/ip_vs_pe_sip.ko 2024-03-28T17:41:36.141 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipvs/ip_vs_rr.ko 2024-03-28T17:41:36.141 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipvs/ip_vs_sed.ko 2024-03-28T17:41:36.141 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipvs/ip_vs_sh.ko 2024-03-28T17:41:36.141 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipvs/ip_vs_wlc.ko 2024-03-28T17:41:36.141 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipvs/ip_vs_wrr.ko 2024-03-28T17:41:36.141 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_conncount.ko 2024-03-28T17:41:36.141 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_conntrack.ko 2024-03-28T17:41:36.141 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_conntrack_amanda.ko 2024-03-28T17:41:36.142 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_conntrack_broadcast.ko 2024-03-28T17:41:36.142 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_conntrack_ftp.ko 2024-03-28T17:41:36.142 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_conntrack_h323.ko 2024-03-28T17:41:36.142 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_conntrack_irc.ko 2024-03-28T17:41:36.142 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_conntrack_netbios_ns.ko 2024-03-28T17:41:36.142 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_conntrack_netlink.ko 2024-03-28T17:41:36.142 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_conntrack_pptp.ko 2024-03-28T17:41:36.142 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_conntrack_sane.ko 2024-03-28T17:41:36.142 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_conntrack_sip.ko 2024-03-28T17:41:36.142 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_conntrack_snmp.ko 2024-03-28T17:41:36.142 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_conntrack_tftp.ko 2024-03-28T17:41:36.142 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_dup_netdev.ko 2024-03-28T17:41:36.142 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_flow_table.ko 2024-03-28T17:41:36.142 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_flow_table_inet.ko 2024-03-28T17:41:36.142 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_log_syslog.ko 2024-03-28T17:41:36.142 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_nat.ko 2024-03-28T17:41:36.142 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_nat_amanda.ko 2024-03-28T17:41:36.143 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_nat_ftp.ko 2024-03-28T17:41:36.143 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_nat_irc.ko 2024-03-28T17:41:36.143 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_nat_sip.ko 2024-03-28T17:41:36.143 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_nat_tftp.ko 2024-03-28T17:41:36.143 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_synproxy_core.ko 2024-03-28T17:41:36.143 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_tables.ko 2024-03-28T17:41:36.143 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nfnetlink.ko 2024-03-28T17:41:36.143 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nfnetlink_acct.ko 2024-03-28T17:41:36.143 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nfnetlink_log.ko 2024-03-28T17:41:36.143 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nfnetlink_osf.ko 2024-03-28T17:41:36.143 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nfnetlink_queue.ko 2024-03-28T17:41:36.143 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nft_chain_nat.ko 2024-03-28T17:41:36.143 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nft_compat.ko 2024-03-28T17:41:36.143 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nft_ct.ko 2024-03-28T17:41:36.143 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nft_dup_netdev.ko 2024-03-28T17:41:36.143 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nft_fib.ko 2024-03-28T17:41:36.144 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nft_fwd_netdev.ko 2024-03-28T17:41:36.144 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nft_hash.ko 2024-03-28T17:41:36.144 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nft_limit.ko 2024-03-28T17:41:36.144 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nft_log.ko 2024-03-28T17:41:36.144 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nft_masq.ko 2024-03-28T17:41:36.144 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nft_nat.ko 2024-03-28T17:41:36.144 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nft_numgen.ko 2024-03-28T17:41:36.144 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nft_queue.ko 2024-03-28T17:41:36.144 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nft_quota.ko 2024-03-28T17:41:36.144 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nft_redir.ko 2024-03-28T17:41:36.144 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nft_reject.ko 2024-03-28T17:41:36.144 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nft_reject_inet.ko 2024-03-28T17:41:36.144 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nft_socket.ko 2024-03-28T17:41:36.144 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nft_tunnel.ko 2024-03-28T17:41:36.144 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_AUDIT.ko 2024-03-28T17:41:36.144 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_CHECKSUM.ko 2024-03-28T17:41:36.144 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_CLASSIFY.ko 2024-03-28T17:41:36.145 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_CONNSECMARK.ko 2024-03-28T17:41:36.145 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_CT.ko 2024-03-28T17:41:36.145 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_DSCP.ko 2024-03-28T17:41:36.145 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_HL.ko 2024-03-28T17:41:36.145 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_HMARK.ko 2024-03-28T17:41:36.145 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_IDLETIMER.ko 2024-03-28T17:41:36.145 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_LED.ko 2024-03-28T17:41:36.145 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_LOG.ko 2024-03-28T17:41:36.145 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_MASQUERADE.ko 2024-03-28T17:41:36.145 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_NETMAP.ko 2024-03-28T17:41:36.145 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_NFLOG.ko 2024-03-28T17:41:36.145 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_NFQUEUE.ko 2024-03-28T17:41:36.145 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_RATEEST.ko 2024-03-28T17:41:36.145 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_REDIRECT.ko 2024-03-28T17:41:36.145 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_SECMARK.ko 2024-03-28T17:41:36.146 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_TCPMSS.ko 2024-03-28T17:41:36.146 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_TCPOPTSTRIP.ko 2024-03-28T17:41:36.146 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_TEE.ko 2024-03-28T17:41:36.146 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_TPROXY.ko 2024-03-28T17:41:36.146 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_TRACE.ko 2024-03-28T17:41:36.146 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_addrtype.ko 2024-03-28T17:41:36.146 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_bpf.ko 2024-03-28T17:41:36.146 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_cgroup.ko 2024-03-28T17:41:36.146 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_cluster.ko 2024-03-28T17:41:36.146 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_comment.ko 2024-03-28T17:41:36.146 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_connbytes.ko 2024-03-28T17:41:36.146 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_connlabel.ko 2024-03-28T17:41:36.146 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_connlimit.ko 2024-03-28T17:41:36.146 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_connmark.ko 2024-03-28T17:41:36.147 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_conntrack.ko 2024-03-28T17:41:36.147 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_cpu.ko 2024-03-28T17:41:36.147 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_dccp.ko 2024-03-28T17:41:36.147 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_devgroup.ko 2024-03-28T17:41:36.147 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_dscp.ko 2024-03-28T17:41:36.147 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_ecn.ko 2024-03-28T17:41:36.147 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_esp.ko 2024-03-28T17:41:36.147 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_hashlimit.ko 2024-03-28T17:41:36.147 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_helper.ko 2024-03-28T17:41:36.147 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_hl.ko 2024-03-28T17:41:36.147 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_ipcomp.ko 2024-03-28T17:41:36.147 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_iprange.ko 2024-03-28T17:41:36.147 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_ipvs.ko 2024-03-28T17:41:36.147 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_l2tp.ko 2024-03-28T17:41:36.147 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_length.ko 2024-03-28T17:41:36.147 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_limit.ko 2024-03-28T17:41:36.148 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_mac.ko 2024-03-28T17:41:36.148 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_mark.ko 2024-03-28T17:41:36.148 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_multiport.ko 2024-03-28T17:41:36.148 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_nat.ko 2024-03-28T17:41:36.148 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_nfacct.ko 2024-03-28T17:41:36.148 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_osf.ko 2024-03-28T17:41:36.148 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_owner.ko 2024-03-28T17:41:36.148 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_physdev.ko 2024-03-28T17:41:36.148 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_pkttype.ko 2024-03-28T17:41:36.148 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_policy.ko 2024-03-28T17:41:36.148 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_quota.ko 2024-03-28T17:41:36.148 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_rateest.ko 2024-03-28T17:41:36.148 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_realm.ko 2024-03-28T17:41:36.148 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_recent.ko 2024-03-28T17:41:36.148 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_sctp.ko 2024-03-28T17:41:36.149 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_set.ko 2024-03-28T17:41:36.149 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_socket.ko 2024-03-28T17:41:36.149 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_state.ko 2024-03-28T17:41:36.149 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_statistic.ko 2024-03-28T17:41:36.149 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_string.ko 2024-03-28T17:41:36.149 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_tcpmss.ko 2024-03-28T17:41:36.149 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_time.ko 2024-03-28T17:41:36.149 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_u32.ko 2024-03-28T17:41:36.149 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netlink 2024-03-28T17:41:36.149 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netlink/netlink_diag.ko 2024-03-28T17:41:36.149 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/nsh 2024-03-28T17:41:36.149 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/nsh/nsh.ko 2024-03-28T17:41:36.149 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/openvswitch 2024-03-28T17:41:36.149 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/openvswitch/openvswitch.ko 2024-03-28T17:41:36.149 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/openvswitch/vport-geneve.ko 2024-03-28T17:41:36.150 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/openvswitch/vport-gre.ko 2024-03-28T17:41:36.150 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/openvswitch/vport-vxlan.ko 2024-03-28T17:41:36.150 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/packet 2024-03-28T17:41:36.150 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/packet/af_packet_diag.ko 2024-03-28T17:41:36.150 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/rds 2024-03-28T17:41:36.150 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/rds/rds.ko 2024-03-28T17:41:36.150 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/rds/rds_rdma.ko 2024-03-28T17:41:36.150 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/rds/rds_tcp.ko 2024-03-28T17:41:36.150 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched 2024-03-28T17:41:36.150 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/act_csum.ko 2024-03-28T17:41:36.150 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/act_gact.ko 2024-03-28T17:41:36.150 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/act_mirred.ko 2024-03-28T17:41:36.150 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/act_nat.ko 2024-03-28T17:41:36.150 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/act_pedit.ko 2024-03-28T17:41:36.151 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/act_police.ko 2024-03-28T17:41:36.151 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/act_simple.ko 2024-03-28T17:41:36.151 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/act_skbedit.ko 2024-03-28T17:41:36.151 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/act_vlan.ko 2024-03-28T17:41:36.151 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/cls_basic.ko 2024-03-28T17:41:36.151 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/cls_bpf.ko 2024-03-28T17:41:36.151 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/cls_flow.ko 2024-03-28T17:41:36.151 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/cls_fw.ko 2024-03-28T17:41:36.151 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/cls_route.ko 2024-03-28T17:41:36.151 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/cls_u32.ko 2024-03-28T17:41:36.151 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/em_cmp.ko 2024-03-28T17:41:36.151 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/em_ipset.ko 2024-03-28T17:41:36.151 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/em_meta.ko 2024-03-28T17:41:36.151 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/em_nbyte.ko 2024-03-28T17:41:36.151 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/em_text.ko 2024-03-28T17:41:36.152 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/em_u32.ko 2024-03-28T17:41:36.152 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/sch_choke.ko 2024-03-28T17:41:36.152 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/sch_codel.ko 2024-03-28T17:41:36.152 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/sch_drr.ko 2024-03-28T17:41:36.152 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/sch_fq.ko 2024-03-28T17:41:36.152 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/sch_gred.ko 2024-03-28T17:41:36.152 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/sch_hfsc.ko 2024-03-28T17:41:36.152 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/sch_hhf.ko 2024-03-28T17:41:36.152 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/sch_htb.ko 2024-03-28T17:41:36.152 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/sch_ingress.ko 2024-03-28T17:41:36.152 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/sch_mqprio.ko 2024-03-28T17:41:36.152 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/sch_mqprio_lib.ko 2024-03-28T17:41:36.152 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/sch_multiq.ko 2024-03-28T17:41:36.152 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/sch_netem.ko 2024-03-28T17:41:36.152 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/sch_pie.ko 2024-03-28T17:41:36.153 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/sch_plug.ko 2024-03-28T17:41:36.153 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/sch_prio.ko 2024-03-28T17:41:36.153 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/sch_qfq.ko 2024-03-28T17:41:36.153 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/sch_red.ko 2024-03-28T17:41:36.153 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/sch_sfb.ko 2024-03-28T17:41:36.153 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/sch_sfq.ko 2024-03-28T17:41:36.153 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/sch_tbf.ko 2024-03-28T17:41:36.153 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/sch_teql.ko 2024-03-28T17:41:36.153 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sctp 2024-03-28T17:41:36.153 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sctp/sctp.ko 2024-03-28T17:41:36.153 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sctp/sctp_diag.ko 2024-03-28T17:41:36.153 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sunrpc 2024-03-28T17:41:36.153 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sunrpc/auth_gss 2024-03-28T17:41:36.153 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sunrpc/auth_gss/auth_rpcgss.ko 2024-03-28T17:41:36.153 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sunrpc/auth_gss/rpcsec_gss_krb5.ko 2024-03-28T17:41:36.153 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sunrpc/sunrpc.ko 2024-03-28T17:41:36.154 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sunrpc/xprtrdma 2024-03-28T17:41:36.154 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sunrpc/xprtrdma/rpcrdma.ko 2024-03-28T17:41:36.154 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/tipc 2024-03-28T17:41:36.154 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/tipc/diag.ko 2024-03-28T17:41:36.154 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/tipc/tipc.ko 2024-03-28T17:41:36.154 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/unix 2024-03-28T17:41:36.154 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/unix/unix_diag.ko 2024-03-28T17:41:36.154 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/vmw_vsock 2024-03-28T17:41:36.154 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/vmw_vsock/vmw_vsock_virtio_transport_common.ko 2024-03-28T17:41:36.154 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/vmw_vsock/vmw_vsock_vmci_transport.ko 2024-03-28T17:41:36.154 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/vmw_vsock/vsock.ko 2024-03-28T17:41:36.154 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/vmw_vsock/vsock_diag.ko 2024-03-28T17:41:36.154 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/vmw_vsock/vsock_loopback.ko 2024-03-28T17:41:36.154 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/xfrm 2024-03-28T17:41:36.154 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/xfrm/xfrm_ipcomp.ko 2024-03-28T17:41:36.155 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/security 2024-03-28T17:41:36.155 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/security/keys 2024-03-28T17:41:36.155 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/security/keys/encrypted-keys 2024-03-28T17:41:36.155 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/security/keys/encrypted-keys/encrypted-keys.ko 2024-03-28T17:41:36.155 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/security/keys/trusted-keys 2024-03-28T17:41:36.155 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/security/keys/trusted-keys/trusted.ko 2024-03-28T17:41:36.155 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/virt 2024-03-28T17:41:36.155 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/virt/lib 2024-03-28T17:41:36.155 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/virt/lib/irqbypass.ko 2024-03-28T17:41:36.155 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/modules.alias 2024-03-28T17:41:36.155 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/modules.alias.bin 2024-03-28T17:41:36.155 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/modules.builtin 2024-03-28T17:41:36.155 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/modules.builtin.alias.bin 2024-03-28T17:41:36.155 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/modules.builtin.bin 2024-03-28T17:41:36.155 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/modules.builtin.modinfo 2024-03-28T17:41:36.155 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/modules.dep 2024-03-28T17:41:36.156 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/modules.dep.bin 2024-03-28T17:41:36.156 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/modules.devname 2024-03-28T17:41:36.156 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/modules.order 2024-03-28T17:41:36.156 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/modules.softdep 2024-03-28T17:41:36.156 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/modules.symbols 2024-03-28T17:41:36.156 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/modules.symbols.bin 2024-03-28T17:41:36.156 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.8.0-g1df0c7e8a235/vmlinuz 2024-03-28T17:41:36.159 DEBUG:teuthology.task.kernel:get_image_version: 6.8.0-g1df0c7e8a235 2024-03-28T17:41:36.159 DEBUG:teuthology.orchestra.run.smithi022:> rpm --scripts -qp /tmp/kernel.x86_64.rpm 2024-03-28T17:41:36.183 INFO:teuthology.orchestra.run.smithi022.stdout:postinstall scriptlet (using /bin/sh): 2024-03-28T17:41:36.183 INFO:teuthology.orchestra.run.smithi022.stdout:if [ -x /usr/bin/kernel-install ]; then 2024-03-28T17:41:36.183 INFO:teuthology.orchestra.run.smithi022.stdout: /usr/bin/kernel-install add 6.8.0-g1df0c7e8a235 /lib/modules/6.8.0-g1df0c7e8a235/vmlinuz 2024-03-28T17:41:36.183 INFO:teuthology.orchestra.run.smithi022.stdout:fi 2024-03-28T17:41:36.183 INFO:teuthology.orchestra.run.smithi022.stdout:for file in vmlinuz System.map config; do 2024-03-28T17:41:36.183 INFO:teuthology.orchestra.run.smithi022.stdout: if ! cmp --silent "/lib/modules/6.8.0-g1df0c7e8a235/${file}" "/boot/${file}-6.8.0-g1df0c7e8a235"; then 2024-03-28T17:41:36.183 INFO:teuthology.orchestra.run.smithi022.stdout: cp "/lib/modules/6.8.0-g1df0c7e8a235/${file}" "/boot/${file}-6.8.0-g1df0c7e8a235" 2024-03-28T17:41:36.183 INFO:teuthology.orchestra.run.smithi022.stdout: fi 2024-03-28T17:41:36.183 INFO:teuthology.orchestra.run.smithi022.stdout:done 2024-03-28T17:41:36.183 INFO:teuthology.orchestra.run.smithi022.stdout:preuninstall scriptlet (using /bin/sh): 2024-03-28T17:41:36.183 INFO:teuthology.orchestra.run.smithi022.stdout:if [ -x /sbin/new-kernel-pkg ]; then 2024-03-28T17:41:36.183 INFO:teuthology.orchestra.run.smithi022.stdout:new-kernel-pkg --remove 6.8.0-g1df0c7e8a235 --rminitrd --initrdfile=/boot/initramfs-6.8.0-g1df0c7e8a235.img 2024-03-28T17:41:36.183 INFO:teuthology.orchestra.run.smithi022.stdout:elif [ -x /usr/bin/kernel-install ]; then 2024-03-28T17:41:36.183 INFO:teuthology.orchestra.run.smithi022.stdout:kernel-install remove 6.8.0-g1df0c7e8a235 2024-03-28T17:41:36.184 INFO:teuthology.orchestra.run.smithi022.stdout:fi 2024-03-28T17:41:36.184 INFO:teuthology.orchestra.run.smithi022.stdout:postuninstall scriptlet (using /bin/sh): 2024-03-28T17:41:36.184 INFO:teuthology.orchestra.run.smithi022.stdout:if [ -x /sbin/update-bootloader ]; then 2024-03-28T17:41:36.184 INFO:teuthology.orchestra.run.smithi022.stdout:/sbin/update-bootloader --remove 6.8.0-g1df0c7e8a235 2024-03-28T17:41:36.184 INFO:teuthology.orchestra.run.smithi022.stdout:fi 2024-03-28T17:41:36.185 DEBUG:teuthology.orchestra.run.smithi022:> sudo rpm -qi grub2-tools 2024-03-28T17:41:36.264 INFO:teuthology.orchestra.run.smithi022.stdout:Name : grub2-tools 2024-03-28T17:41:36.264 INFO:teuthology.orchestra.run.smithi022.stdout:Epoch : 1 2024-03-28T17:41:36.264 INFO:teuthology.orchestra.run.smithi022.stdout:Version : 2.06 2024-03-28T17:41:36.264 INFO:teuthology.orchestra.run.smithi022.stdout:Release : 68.el9 2024-03-28T17:41:36.265 INFO:teuthology.orchestra.run.smithi022.stdout:Architecture: x86_64 2024-03-28T17:41:36.265 INFO:teuthology.orchestra.run.smithi022.stdout:Install Date: Fri 01 Sep 2023 09:14:30 PM UTC 2024-03-28T17:41:36.265 INFO:teuthology.orchestra.run.smithi022.stdout:Group : Unspecified 2024-03-28T17:41:36.265 INFO:teuthology.orchestra.run.smithi022.stdout:Size : 8264254 2024-03-28T17:41:36.265 INFO:teuthology.orchestra.run.smithi022.stdout:License : GPLv3+ 2024-03-28T17:41:36.265 INFO:teuthology.orchestra.run.smithi022.stdout:Signature : RSA/SHA256, Mon 07 Aug 2023 01:57:16 PM UTC, Key ID 05b555b38483c65d 2024-03-28T17:41:36.265 INFO:teuthology.orchestra.run.smithi022.stdout:Source RPM : grub2-2.06-68.el9.src.rpm 2024-03-28T17:41:36.265 INFO:teuthology.orchestra.run.smithi022.stdout:Build Date : Wed 02 Aug 2023 03:15:48 PM UTC 2024-03-28T17:41:36.265 INFO:teuthology.orchestra.run.smithi022.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-03-28T17:41:36.265 INFO:teuthology.orchestra.run.smithi022.stdout:Packager : builder@centos.org 2024-03-28T17:41:36.265 INFO:teuthology.orchestra.run.smithi022.stdout:Vendor : CentOS 2024-03-28T17:41:36.265 INFO:teuthology.orchestra.run.smithi022.stdout:URL : http://www.gnu.org/software/grub/ 2024-03-28T17:41:36.265 INFO:teuthology.orchestra.run.smithi022.stdout:Summary : Support tools for GRUB. 2024-03-28T17:41:36.265 INFO:teuthology.orchestra.run.smithi022.stdout:Description : 2024-03-28T17:41:36.265 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-03-28T17:41:36.266 INFO:teuthology.orchestra.run.smithi022.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-03-28T17:41:36.266 INFO:teuthology.orchestra.run.smithi022.stdout:customizable bootloader with modular architecture. It supports a rich 2024-03-28T17:41:36.266 INFO:teuthology.orchestra.run.smithi022.stdout:variety of kernel formats, file systems, computer architectures and 2024-03-28T17:41:36.266 INFO:teuthology.orchestra.run.smithi022.stdout:hardware devices. 2024-03-28T17:41:36.266 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-03-28T17:41:36.266 INFO:teuthology.orchestra.run.smithi022.stdout:This subpackage provides tools for support of all platforms. 2024-03-28T17:41:36.267 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-03-28T17:41:36.267 INFO:teuthology.task.kernel:Updating grub on smithi022 to boot 6.8.0-g1df0c7e8a235 2024-03-28T17:41:36.267 DEBUG:teuthology.orchestra.run.smithi022:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-03-28T17:41:36.864 INFO:teuthology.orchestra.run.smithi022.stderr:Generating grub configuration file ... 2024-03-28T17:41:37.795 INFO:teuthology.task.kernel:Checking client mon.b for new kernel version... 2024-03-28T17:41:37.795 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi079.front.sepia.ceph.com' 2024-03-28T17:41:37.796 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi079.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-28T17:41:38.173 INFO:teuthology.orchestra.run.smithi022.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-03-28T17:41:38.192 INFO:teuthology.orchestra.run.smithi022.stderr:done 2024-03-28T17:41:38.195 DEBUG:teuthology.orchestra.run.smithi022:> mktemp 2024-03-28T17:41:38.210 INFO:teuthology.orchestra.run.smithi022.stdout:/tmp/tmp.UupnkiqjLY 2024-03-28T17:41:38.211 DEBUG:teuthology.orchestra.run.smithi022:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.UupnkiqjLY 2024-03-28T17:41:38.283 DEBUG:teuthology.orchestra.run.smithi022:> sudo chmod 0666 /tmp/tmp.UupnkiqjLY 2024-03-28T17:41:38.422 DEBUG:teuthology.orchestra.remote:smithi022:/tmp/tmp.UupnkiqjLY is 6KB 2024-03-28T17:41:38.470 DEBUG:teuthology.orchestra.run.smithi022:> rm -fr /tmp/tmp.UupnkiqjLY 2024-03-28T17:41:38.484 DEBUG:teuthology.orchestra.run.smithi022:> sudo /bin/ls /boot/loader/entries || true 2024-03-28T17:41:38.552 INFO:teuthology.orchestra.run.smithi022.stdout:15949e3c88704300882482eddfd7baae-0-rescue.conf 2024-03-28T17:41:38.552 INFO:teuthology.orchestra.run.smithi022.stdout:15949e3c88704300882482eddfd7baae-5.14.0-239.el9.x86_64.conf 2024-03-28T17:41:38.553 INFO:teuthology.orchestra.run.smithi022.stdout:15949e3c88704300882482eddfd7baae-5.14.0-361.el9.x86_64.conf 2024-03-28T17:41:38.553 INFO:teuthology.orchestra.run.smithi022.stdout:15949e3c88704300882482eddfd7baae-6.8.0-g1df0c7e8a235.conf 2024-03-28T17:41:38.554 DEBUG:teuthology.orchestra.run.smithi022:> sudo grub2-set-default 15949e3c88704300882482eddfd7baae-6.8.0-g1df0c7e8a235 2024-03-28T17:41:38.703 DEBUG:teuthology.orchestra.run.smithi022:> sudo shutdown -r now 2024-03-28T17:41:54.952 INFO:teuthology.task.kernel:Checking client mon.c for new kernel version... 2024-03-28T17:41:54.952 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi134.front.sepia.ceph.com' 2024-03-28T17:41:54.953 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi134.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-28T17:41:56.126 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.79 2024-03-28T17:42:05.134 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi079.front.sepia.ceph.com' 2024-03-28T17:42:05.135 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi079.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-28T17:42:08.190 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.79 2024-03-28T17:42:08.706 INFO:teuthology.task.kernel:Checking client mon.a for new kernel version... 2024-03-28T17:42:08.707 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi022.front.sepia.ceph.com' 2024-03-28T17:42:08.707 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi022.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-28T17:42:13.278 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.134 2024-03-28T17:42:20.195 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi079.front.sepia.ceph.com' 2024-03-28T17:42:20.196 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi079.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-28T17:42:22.281 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi134.front.sepia.ceph.com' 2024-03-28T17:42:22.282 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi134.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-28T17:42:25.342 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.134 2024-03-28T17:42:27.102 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.22 2024-03-28T17:42:36.110 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi022.front.sepia.ceph.com' 2024-03-28T17:42:36.111 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi022.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-28T17:42:37.345 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi134.front.sepia.ceph.com' 2024-03-28T17:42:37.345 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi134.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-28T17:42:38.622 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.79 2024-03-28T17:42:53.634 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi079.front.sepia.ceph.com' 2024-03-28T17:42:53.635 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi079.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-28T17:42:54.099 DEBUG:teuthology.orchestra.run.smithi079:> true 2024-03-28T17:42:54.724 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi079.front.sepia.ceph.com' 2024-03-28T17:42:54.724 INFO:teuthology.task.kernel:Checking kernel version of mon.b, want "6.8.0-g1df0c7e8a235"... 2024-03-28T17:42:54.724 DEBUG:teuthology.orchestra.run.smithi079:> uname -r 2024-03-28T17:42:54.767 INFO:teuthology.orchestra.run.smithi079.stdout:6.8.0-g1df0c7e8a235 2024-03-28T17:42:54.768 DEBUG:teuthology.task.kernel:current kernel version is 6.8.0-g1df0c7e8a235 vs 6.8.0-g1df0c7e8a235 2024-03-28T17:42:54.768 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-03-28T17:42:54.768 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-03-28T17:42:55.769 INFO:teuthology.task.kernel:Enabling kdb on mon.b... 2024-03-28T17:42:55.769 DEBUG:teuthology.orchestra.run.smithi079:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-03-28T17:42:55.902 INFO:teuthology.orchestra.run.smithi079.stdout:ttyS1 2024-03-28T17:42:55.945 DEBUG:teuthology.parallel:result is None 2024-03-28T17:43:36.114 DEBUG:teuthology.orchestra.remote:timed out 2024-03-28T17:43:37.347 DEBUG:teuthology.orchestra.remote:timed out 2024-03-28T17:43:48.115 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi022.front.sepia.ceph.com' 2024-03-28T17:43:48.116 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi022.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-28T17:43:48.588 DEBUG:teuthology.orchestra.run.smithi022:> true 2024-03-28T17:43:49.314 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi022.front.sepia.ceph.com' 2024-03-28T17:43:49.314 INFO:teuthology.task.kernel:Checking kernel version of mon.a, want "6.8.0-g1df0c7e8a235"... 2024-03-28T17:43:49.315 DEBUG:teuthology.orchestra.run.smithi022:> uname -r 2024-03-28T17:43:49.367 INFO:teuthology.orchestra.run.smithi022.stdout:6.8.0-g1df0c7e8a235 2024-03-28T17:43:49.367 DEBUG:teuthology.task.kernel:current kernel version is 6.8.0-g1df0c7e8a235 vs 6.8.0-g1df0c7e8a235 2024-03-28T17:43:49.367 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-03-28T17:43:49.367 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-03-28T17:43:50.368 INFO:teuthology.task.kernel:Enabling kdb on mon.a... 2024-03-28T17:43:50.368 DEBUG:teuthology.orchestra.run.smithi022:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-03-28T17:43:50.505 INFO:teuthology.orchestra.run.smithi022.stdout:ttyS1 2024-03-28T17:43:50.565 DEBUG:teuthology.parallel:result is None 2024-03-28T17:43:52.348 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi134.front.sepia.ceph.com' 2024-03-28T17:43:52.348 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi134.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-28T17:43:52.821 DEBUG:teuthology.orchestra.run.smithi134:> true 2024-03-28T17:43:53.456 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi134.front.sepia.ceph.com' 2024-03-28T17:43:53.456 INFO:teuthology.task.kernel:Checking kernel version of mon.c, want "6.8.0-g1df0c7e8a235"... 2024-03-28T17:43:53.456 DEBUG:teuthology.orchestra.run.smithi134:> uname -r 2024-03-28T17:43:53.506 INFO:teuthology.orchestra.run.smithi134.stdout:6.8.0-g1df0c7e8a235 2024-03-28T17:43:53.506 DEBUG:teuthology.task.kernel:current kernel version is 6.8.0-g1df0c7e8a235 vs 6.8.0-g1df0c7e8a235 2024-03-28T17:43:53.507 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-03-28T17:43:53.507 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-03-28T17:43:54.508 INFO:teuthology.task.kernel:Enabling kdb on mon.c... 2024-03-28T17:43:54.508 DEBUG:teuthology.orchestra.run.smithi134:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-03-28T17:43:54.646 INFO:teuthology.orchestra.run.smithi134.stdout:ttyS1 2024-03-28T17:43:54.709 DEBUG:teuthology.parallel:result is None 2024-03-28T17:43:54.709 INFO:teuthology.run_tasks:Running task internal.base... 2024-03-28T17:43:54.717 INFO:teuthology.task.internal:Creating test directory... 2024-03-28T17:43:54.717 DEBUG:teuthology.orchestra.run.smithi022:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-03-28T17:43:54.725 DEBUG:teuthology.orchestra.run.smithi079:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-03-28T17:43:54.733 DEBUG:teuthology.orchestra.run.smithi134:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-03-28T17:43:54.788 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-03-28T17:43:54.794 INFO:teuthology.run_tasks:Running task internal.archive... 2024-03-28T17:43:54.801 INFO:teuthology.task.internal:Creating archive directory... 2024-03-28T17:43:54.801 DEBUG:teuthology.orchestra.run.smithi022:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-03-28T17:43:54.809 DEBUG:teuthology.orchestra.run.smithi079:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-03-28T17:43:54.816 DEBUG:teuthology.orchestra.run.smithi134:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-03-28T17:43:54.884 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-03-28T17:43:54.890 INFO:teuthology.task.internal:Enabling coredump saving... 2024-03-28T17:43:54.891 DEBUG:teuthology.orchestra.run.smithi022:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-03-28T17:43:54.898 DEBUG:teuthology.orchestra.run.smithi079:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-03-28T17:43:54.905 DEBUG:teuthology.orchestra.run.smithi134:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-03-28T17:43:54.998 INFO:teuthology.orchestra.run.smithi022.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-03-28T17:43:55.006 INFO:teuthology.orchestra.run.smithi079.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-03-28T17:43:55.006 INFO:teuthology.orchestra.run.smithi134.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-03-28T17:43:55.044 INFO:teuthology.orchestra.run.smithi022.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-03-28T17:43:55.045 INFO:teuthology.orchestra.run.smithi134.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-03-28T17:43:55.046 INFO:teuthology.orchestra.run.smithi079.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-03-28T17:43:55.050 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-03-28T17:43:55.057 INFO:teuthology.task.internal:Configuring sudo... 2024-03-28T17:43:55.057 DEBUG:teuthology.orchestra.run.smithi022:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-03-28T17:43:55.062 DEBUG:teuthology.orchestra.run.smithi079:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-03-28T17:43:55.067 DEBUG:teuthology.orchestra.run.smithi134:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-03-28T17:43:55.159 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-03-28T17:43:55.168 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-03-28T17:43:55.168 DEBUG:teuthology.orchestra.run.smithi022:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-03-28T17:43:55.175 DEBUG:teuthology.orchestra.run.smithi079:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-03-28T17:43:55.182 DEBUG:teuthology.orchestra.run.smithi134:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-03-28T17:43:55.236 DEBUG:teuthology.orchestra.run.smithi022:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-03-28T17:43:55.324 DEBUG:teuthology.orchestra.run.smithi022:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-03-28T17:43:55.460 DEBUG:teuthology.orchestra.run.smithi022:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-03-28T17:43:55.544 DEBUG:teuthology.orchestra.run.smithi022:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-03-28T17:43:55.642 DEBUG:teuthology.orchestra.run.smithi022:> set -ex 2024-03-28T17:43:55.642 DEBUG:teuthology.orchestra.run.smithi022:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-03-28T17:43:55.733 DEBUG:teuthology.orchestra.run.smithi079:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-03-28T17:43:55.820 DEBUG:teuthology.orchestra.run.smithi079:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-03-28T17:43:55.940 DEBUG:teuthology.orchestra.run.smithi079:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-03-28T17:43:56.022 DEBUG:teuthology.orchestra.run.smithi079:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-03-28T17:43:56.114 DEBUG:teuthology.orchestra.run.smithi079:> set -ex 2024-03-28T17:43:56.114 DEBUG:teuthology.orchestra.run.smithi079:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-03-28T17:43:56.208 DEBUG:teuthology.orchestra.run.smithi134:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-03-28T17:43:56.291 DEBUG:teuthology.orchestra.run.smithi134:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-03-28T17:43:56.440 DEBUG:teuthology.orchestra.run.smithi134:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-03-28T17:43:56.521 DEBUG:teuthology.orchestra.run.smithi134:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-03-28T17:43:56.616 DEBUG:teuthology.orchestra.run.smithi134:> set -ex 2024-03-28T17:43:56.616 DEBUG:teuthology.orchestra.run.smithi134:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-03-28T17:43:56.707 DEBUG:teuthology.orchestra.run.smithi022:> sudo service rsyslog restart 2024-03-28T17:43:56.714 DEBUG:teuthology.orchestra.run.smithi079:> sudo service rsyslog restart 2024-03-28T17:43:56.721 DEBUG:teuthology.orchestra.run.smithi134:> sudo service rsyslog restart 2024-03-28T17:43:56.848 INFO:teuthology.orchestra.run.smithi079.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-03-28T17:43:56.850 INFO:teuthology.orchestra.run.smithi022.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-03-28T17:43:56.881 INFO:teuthology.orchestra.run.smithi134.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-03-28T17:43:57.358 INFO:teuthology.run_tasks:Running task internal.timer... 2024-03-28T17:43:57.368 INFO:teuthology.task.internal:Starting timer... 2024-03-28T17:43:57.368 INFO:teuthology.run_tasks:Running task pcp... 2024-03-28T17:43:57.377 INFO:teuthology.run_tasks:Running task selinux... 2024-03-28T17:43:57.385 DEBUG:teuthology.task:Applying overrides for task selinux: {'allowlist': ['scontext=system_u:system_r:logrotate_t:s0', 'scontext=system_u:system_r:getty_t:s0']} 2024-03-28T17:43:57.385 DEBUG:teuthology.orchestra.run.smithi022:> sudo service auditd rotate 2024-03-28T17:43:57.499 INFO:teuthology.orchestra.run.smithi022.stdout:Rotating logs: 2024-03-28T17:43:57.504 DEBUG:teuthology.orchestra.run.smithi079:> sudo service auditd rotate 2024-03-28T17:43:57.654 INFO:teuthology.orchestra.run.smithi079.stdout:Rotating logs: 2024-03-28T17:43:57.658 DEBUG:teuthology.orchestra.run.smithi134:> sudo service auditd rotate 2024-03-28T17:43:57.786 INFO:teuthology.orchestra.run.smithi134.stdout:Rotating logs: 2024-03-28T17:43:57.791 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-03-28T17:43:57.791 DEBUG:teuthology.orchestra.run.smithi022:> /usr/sbin/getenforce 2024-03-28T17:43:57.849 INFO:teuthology.orchestra.run.smithi022.stdout:Permissive 2024-03-28T17:43:57.849 DEBUG:teuthology.orchestra.run.smithi079:> /usr/sbin/getenforce 2024-03-28T17:43:57.905 INFO:teuthology.orchestra.run.smithi079.stdout:Permissive 2024-03-28T17:43:57.905 DEBUG:teuthology.orchestra.run.smithi134:> /usr/sbin/getenforce 2024-03-28T17:43:57.955 INFO:teuthology.orchestra.run.smithi134.stdout:Permissive 2024-03-28T17:43:57.955 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi022.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi079.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi134.front.sepia.ceph.com': 'permissive'} 2024-03-28T17:43:57.956 DEBUG:teuthology.orchestra.run.smithi022:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-03-28T17:43:58.050 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-28T17:43:58.051 DEBUG:teuthology.orchestra.run.smithi079:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-03-28T17:43:58.141 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-28T17:43:58.142 DEBUG:teuthology.orchestra.run.smithi134:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-03-28T17:43:58.223 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-28T17:43:58.224 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-03-28T17:43:58.224 DEBUG:teuthology.orchestra.run.smithi022:> sudo /usr/sbin/setenforce permissive 2024-03-28T17:43:58.327 DEBUG:teuthology.orchestra.run.smithi079:> sudo /usr/sbin/setenforce permissive 2024-03-28T17:43:58.415 DEBUG:teuthology.orchestra.run.smithi134:> sudo /usr/sbin/setenforce permissive 2024-03-28T17:43:58.520 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-03-28T17:43:58.530 DEBUG:teuthology.repo_utils:Setting repo remote to https://git.ceph.com/ceph-cm-ansible.git 2024-03-28T17:43:58.536 INFO:teuthology.repo_utils:Fetching git.ceph.com_ceph-cm-ansible_main from origin 2024-03-28T17:43:58.658 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-03-28T17:43:58.673 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-03-28T17:43:58.743 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi022.front.sepia.ceph.com,smithi079.front.sepia.ceph.com,smithi134.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-03-28T17:50:25.269 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi022.front.sepia.ceph.com'), Remote(name='ubuntu@smithi079.front.sepia.ceph.com'), Remote(name='ubuntu@smithi134.front.sepia.ceph.com')] 2024-03-28T17:50:25.270 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi022.front.sepia.ceph.com' 2024-03-28T17:50:25.271 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi022.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-28T17:50:25.427 DEBUG:teuthology.orchestra.run.smithi022:> true 2024-03-28T17:50:25.579 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi022.front.sepia.ceph.com' 2024-03-28T17:50:25.579 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi079.front.sepia.ceph.com' 2024-03-28T17:50:25.580 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi079.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-28T17:50:25.716 DEBUG:teuthology.orchestra.run.smithi079:> true 2024-03-28T17:50:25.866 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi079.front.sepia.ceph.com' 2024-03-28T17:50:25.867 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi134.front.sepia.ceph.com' 2024-03-28T17:50:25.867 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi134.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-28T17:50:26.011 DEBUG:teuthology.orchestra.run.smithi134:> true 2024-03-28T17:50:26.168 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi134.front.sepia.ceph.com' 2024-03-28T17:50:26.169 INFO:teuthology.run_tasks:Running task clock... 2024-03-28T17:50:26.179 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-03-28T17:50:26.180 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-03-28T17:50:26.180 DEBUG:teuthology.orchestra.run.smithi022:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-03-28T17:50:26.187 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-03-28T17:50:26.188 DEBUG:teuthology.orchestra.run.smithi079:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-03-28T17:50:26.195 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-03-28T17:50:26.195 DEBUG:teuthology.orchestra.run.smithi134:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-03-28T17:50:26.291 INFO:teuthology.orchestra.run.smithi022.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-03-28T17:50:26.291 INFO:teuthology.orchestra.run.smithi079.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-03-28T17:50:26.299 INFO:teuthology.orchestra.run.smithi134.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-03-28T17:50:26.344 INFO:teuthology.orchestra.run.smithi079.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-03-28T17:50:26.346 INFO:teuthology.orchestra.run.smithi022.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-03-28T17:50:26.350 INFO:teuthology.orchestra.run.smithi134.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-03-28T17:50:26.435 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: ntpd: command not found 2024-03-28T17:50:26.442 INFO:teuthology.orchestra.run.smithi134.stderr:sudo: ntpd: command not found 2024-03-28T17:50:26.444 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: ntpd: command not found 2024-03-28T17:50:26.476 INFO:teuthology.orchestra.run.smithi079.stdout:506 Cannot talk to daemon 2024-03-28T17:50:26.485 INFO:teuthology.orchestra.run.smithi134.stdout:506 Cannot talk to daemon 2024-03-28T17:50:26.491 INFO:teuthology.orchestra.run.smithi022.stdout:506 Cannot talk to daemon 2024-03-28T17:50:26.526 INFO:teuthology.orchestra.run.smithi079.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-03-28T17:50:26.536 INFO:teuthology.orchestra.run.smithi134.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-03-28T17:50:26.547 INFO:teuthology.orchestra.run.smithi022.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-03-28T17:50:26.580 INFO:teuthology.orchestra.run.smithi079.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-03-28T17:50:26.589 INFO:teuthology.orchestra.run.smithi134.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-03-28T17:50:26.603 INFO:teuthology.orchestra.run.smithi022.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-03-28T17:50:26.710 INFO:teuthology.orchestra.run.smithi079.stderr:bash: line 1: ntpq: command not found 2024-03-28T17:50:26.717 INFO:teuthology.orchestra.run.smithi134.stderr:bash: line 1: ntpq: command not found 2024-03-28T17:50:26.718 INFO:teuthology.orchestra.run.smithi079.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-03-28T17:50:26.718 INFO:teuthology.orchestra.run.smithi079.stdout:=============================================================================== 2024-03-28T17:50:26.718 INFO:teuthology.orchestra.run.smithi079.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-28T17:50:26.719 INFO:teuthology.orchestra.run.smithi079.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-28T17:50:26.719 INFO:teuthology.orchestra.run.smithi079.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-28T17:50:26.719 INFO:teuthology.orchestra.run.smithi079.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-28T17:50:26.724 INFO:teuthology.orchestra.run.smithi134.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-03-28T17:50:26.724 INFO:teuthology.orchestra.run.smithi134.stdout:=============================================================================== 2024-03-28T17:50:26.724 INFO:teuthology.orchestra.run.smithi134.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-28T17:50:26.725 INFO:teuthology.orchestra.run.smithi134.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-28T17:50:26.725 INFO:teuthology.orchestra.run.smithi134.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-28T17:50:26.725 INFO:teuthology.orchestra.run.smithi134.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-28T17:50:26.739 INFO:teuthology.orchestra.run.smithi022.stderr:bash: line 1: ntpq: command not found 2024-03-28T17:50:26.747 INFO:teuthology.orchestra.run.smithi022.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-03-28T17:50:26.747 INFO:teuthology.orchestra.run.smithi022.stdout:=============================================================================== 2024-03-28T17:50:26.747 INFO:teuthology.orchestra.run.smithi022.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-28T17:50:26.747 INFO:teuthology.orchestra.run.smithi022.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-28T17:50:26.747 INFO:teuthology.orchestra.run.smithi022.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-28T17:50:26.747 INFO:teuthology.orchestra.run.smithi022.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-28T17:50:26.749 INFO:teuthology.run_tasks:Running task install... 2024-03-28T17:50:26.757 DEBUG:teuthology.task.install:project ceph 2024-03-28T17:50:26.757 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': '3be9a5c9bc793e11e2800a8c0c696e8b46742033'}} 2024-03-28T17:50:26.757 DEBUG:teuthology.task.install:config {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': '3be9a5c9bc793e11e2800a8c0c696e8b46742033'} 2024-03-28T17:50:26.758 INFO:teuthology.task.install:Using flavor: default 2024-03-28T17:50:26.763 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2024-03-28T17:50:26.763 INFO:teuthology.task.install:extra packages: {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']} 2024-03-28T17:50:26.764 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '3be9a5c9bc793e11e2800a8c0c696e8b46742033', 'tag': None, 'wait_for_package': False} 2024-03-28T17:50:26.764 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=3be9a5c9bc793e11e2800a8c0c696e8b46742033 2024-03-28T17:50:26.766 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '3be9a5c9bc793e11e2800a8c0c696e8b46742033', 'tag': None, 'wait_for_package': False} 2024-03-28T17:50:26.767 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=3be9a5c9bc793e11e2800a8c0c696e8b46742033 2024-03-28T17:50:26.768 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '3be9a5c9bc793e11e2800a8c0c696e8b46742033', 'tag': None, 'wait_for_package': False} 2024-03-28T17:50:26.768 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=3be9a5c9bc793e11e2800a8c0c696e8b46742033 2024-03-28T17:50:26.908 INFO:teuthology.task.install.rpm:Pulling from https://4.chacra.ceph.com/r/ceph/wip-mchangir-testing-main-20240318.032620/3be9a5c9bc793e11e2800a8c0c696e8b46742033/centos/9/flavors/default/ 2024-03-28T17:50:26.908 INFO:teuthology.task.install.rpm:Package version is 19.0.0-2145.g3be9a5c9 2024-03-28T17:50:26.912 INFO:teuthology.task.install.rpm:Pulling from https://4.chacra.ceph.com/r/ceph/wip-mchangir-testing-main-20240318.032620/3be9a5c9bc793e11e2800a8c0c696e8b46742033/centos/9/flavors/default/ 2024-03-28T17:50:26.913 INFO:teuthology.task.install.rpm:Package version is 19.0.0-2145.g3be9a5c9 2024-03-28T17:50:27.002 INFO:teuthology.task.install.rpm:Pulling from https://4.chacra.ceph.com/r/ceph/wip-mchangir-testing-main-20240318.032620/3be9a5c9bc793e11e2800a8c0c696e8b46742033/centos/9/flavors/default/ 2024-03-28T17:50:27.002 INFO:teuthology.task.install.rpm:Package version is 19.0.0-2145.g3be9a5c9 2024-03-28T17:50:27.035 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://4.chacra.ceph.com/r/ceph/wip-mchangir-testing-main-20240318.032620/3be9a5c9bc793e11e2800a8c0c696e8b46742033/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://4.chacra.ceph.com/r/ceph/wip-mchangir-testing-main-20240318.032620/3be9a5c9bc793e11e2800a8c0c696e8b46742033/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://4.chacra.ceph.com/r/ceph/wip-mchangir-testing-main-20240318.032620/3be9a5c9bc793e11e2800a8c0c696e8b46742033/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-03-28T17:50:27.035 DEBUG:teuthology.orchestra.run.smithi134:> set -ex 2024-03-28T17:50:27.035 DEBUG:teuthology.orchestra.run.smithi134:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-03-28T17:50:27.045 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://4.chacra.ceph.com/r/ceph/wip-mchangir-testing-main-20240318.032620/3be9a5c9bc793e11e2800a8c0c696e8b46742033/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://4.chacra.ceph.com/r/ceph/wip-mchangir-testing-main-20240318.032620/3be9a5c9bc793e11e2800a8c0c696e8b46742033/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://4.chacra.ceph.com/r/ceph/wip-mchangir-testing-main-20240318.032620/3be9a5c9bc793e11e2800a8c0c696e8b46742033/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-03-28T17:50:27.047 DEBUG:teuthology.orchestra.run.smithi022:> set -ex 2024-03-28T17:50:27.047 DEBUG:teuthology.orchestra.run.smithi022:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-03-28T17:50:27.115 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://4.chacra.ceph.com/r/ceph/wip-mchangir-testing-main-20240318.032620/3be9a5c9bc793e11e2800a8c0c696e8b46742033/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://4.chacra.ceph.com/r/ceph/wip-mchangir-testing-main-20240318.032620/3be9a5c9bc793e11e2800a8c0c696e8b46742033/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://4.chacra.ceph.com/r/ceph/wip-mchangir-testing-main-20240318.032620/3be9a5c9bc793e11e2800a8c0c696e8b46742033/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-03-28T17:50:27.115 DEBUG:teuthology.orchestra.run.smithi079:> set -ex 2024-03-28T17:50:27.115 DEBUG:teuthology.orchestra.run.smithi079:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-03-28T17:50:27.133 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-03-28T17:50:27.133 DEBUG:teuthology.orchestra.run.smithi134:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/3be9a5c9bc793e11e2800a8c0c696e8b46742033/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-03-28T17:50:27.156 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-03-28T17:50:27.156 DEBUG:teuthology.orchestra.run.smithi022:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/3be9a5c9bc793e11e2800a8c0c696e8b46742033/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-03-28T17:50:27.214 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-03-28T17:50:27.214 DEBUG:teuthology.orchestra.run.smithi079:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/3be9a5c9bc793e11e2800a8c0c696e8b46742033/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-03-28T17:50:27.271 DEBUG:teuthology.orchestra.run.smithi134:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-03-28T17:50:27.298 DEBUG:teuthology.orchestra.run.smithi022:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-03-28T17:50:27.351 DEBUG:teuthology.orchestra.run.smithi079:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-03-28T17:50:27.398 DEBUG:teuthology.orchestra.run.smithi134:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-03-28T17:50:27.441 DEBUG:teuthology.orchestra.run.smithi022:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-03-28T17:50:27.473 DEBUG:teuthology.orchestra.run.smithi079:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-03-28T17:50:27.488 INFO:teuthology.orchestra.run.smithi134.stdout:check_obsoletes = 1 2024-03-28T17:50:27.492 DEBUG:teuthology.orchestra.run.smithi134:> sudo yum clean all 2024-03-28T17:50:27.530 INFO:teuthology.orchestra.run.smithi022.stdout:check_obsoletes = 1 2024-03-28T17:50:27.534 DEBUG:teuthology.orchestra.run.smithi022:> sudo yum clean all 2024-03-28T17:50:27.556 INFO:teuthology.orchestra.run.smithi079.stdout:check_obsoletes = 1 2024-03-28T17:50:27.561 DEBUG:teuthology.orchestra.run.smithi079:> sudo yum clean all 2024-03-28T17:50:27.845 INFO:teuthology.orchestra.run.smithi134.stdout:74 files removed 2024-03-28T17:50:27.884 DEBUG:teuthology.orchestra.run.smithi134:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-03-28T17:50:27.909 INFO:teuthology.orchestra.run.smithi079.stdout:74 files removed 2024-03-28T17:50:27.910 INFO:teuthology.orchestra.run.smithi022.stdout:74 files removed 2024-03-28T17:50:27.947 DEBUG:teuthology.orchestra.run.smithi079:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-03-28T17:50:27.950 DEBUG:teuthology.orchestra.run.smithi022:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-03-28T17:50:28.860 INFO:teuthology.orchestra.run.smithi134.stdout:ceph packages for x86_64 178 kB/s | 83 kB 00:00 2024-03-28T17:50:28.903 INFO:teuthology.orchestra.run.smithi079.stdout:ceph packages for x86_64 181 kB/s | 83 kB 00:00 2024-03-28T17:50:28.954 INFO:teuthology.orchestra.run.smithi022.stdout:ceph packages for x86_64 171 kB/s | 83 kB 00:00 2024-03-28T17:50:29.395 INFO:teuthology.orchestra.run.smithi079.stdout:ceph noarch packages 25 kB/s | 10 kB 00:00 2024-03-28T17:50:29.410 INFO:teuthology.orchestra.run.smithi134.stdout:ceph noarch packages 22 kB/s | 10 kB 00:00 2024-03-28T17:50:29.538 INFO:teuthology.orchestra.run.smithi022.stdout:ceph noarch packages 21 kB/s | 10 kB 00:00 2024-03-28T17:50:29.888 INFO:teuthology.orchestra.run.smithi079.stdout:ceph source packages 4.6 kB/s | 1.9 kB 00:00 2024-03-28T17:50:29.961 INFO:teuthology.orchestra.run.smithi134.stdout:ceph source packages 4.0 kB/s | 1.9 kB 00:00 2024-03-28T17:50:30.105 INFO:teuthology.orchestra.run.smithi022.stdout:ceph source packages 3.9 kB/s | 1.9 kB 00:00 2024-03-28T17:50:30.607 INFO:teuthology.orchestra.run.smithi079.stdout:CentOS-9 - AppStream 10 MB/s | 6.7 MB 00:00 2024-03-28T17:50:30.814 INFO:teuthology.orchestra.run.smithi134.stdout:CentOS-9 - AppStream 8.6 MB/s | 6.7 MB 00:00 2024-03-28T17:50:30.832 INFO:teuthology.orchestra.run.smithi022.stdout:CentOS-9 - AppStream 10 MB/s | 6.7 MB 00:00 2024-03-28T17:50:32.333 INFO:teuthology.orchestra.run.smithi079.stdout:CentOS-9 - BaseOS 4.5 MB/s | 2.0 MB 00:00 2024-03-28T17:50:32.515 INFO:teuthology.orchestra.run.smithi134.stdout:CentOS-9 - BaseOS 4.2 MB/s | 2.0 MB 00:00 2024-03-28T17:50:32.534 INFO:teuthology.orchestra.run.smithi022.stdout:CentOS-9 - BaseOS 5.9 MB/s | 2.0 MB 00:00 2024-03-28T17:50:33.397 INFO:teuthology.orchestra.run.smithi022.stdout:CentOS Stream 9 - BaseOS 17 MB/s | 8.0 MB 00:00 2024-03-28T17:50:33.469 INFO:teuthology.orchestra.run.smithi134.stdout:CentOS Stream 9 - BaseOS 15 MB/s | 8.0 MB 00:00 2024-03-28T17:50:33.554 INFO:teuthology.orchestra.run.smithi079.stdout:CentOS Stream 9 - BaseOS 9.9 MB/s | 8.0 MB 00:00 2024-03-28T17:50:35.514 INFO:teuthology.orchestra.run.smithi022.stdout:CentOS Stream 9 - AppStream 20 MB/s | 19 MB 00:00 2024-03-28T17:50:35.754 INFO:teuthology.orchestra.run.smithi079.stdout:CentOS Stream 9 - AppStream 19 MB/s | 19 MB 00:01 2024-03-28T17:50:35.800 INFO:teuthology.orchestra.run.smithi134.stdout:CentOS Stream 9 - AppStream 16 MB/s | 19 MB 00:01 2024-03-28T17:50:40.047 INFO:teuthology.orchestra.run.smithi079.stdout:CentOS Stream 9 - CRB 11 MB/s | 6.1 MB 00:00 2024-03-28T17:50:40.096 INFO:teuthology.orchestra.run.smithi134.stdout:CentOS Stream 9 - CRB 11 MB/s | 6.1 MB 00:00 2024-03-28T17:50:40.632 INFO:teuthology.orchestra.run.smithi022.stdout:CentOS Stream 9 - CRB 5.0 MB/s | 6.1 MB 00:01 2024-03-28T17:50:41.688 INFO:teuthology.orchestra.run.smithi134.stdout:CentOS Stream 9 - Extras packages 59 kB/s | 16 kB 00:00 2024-03-28T17:50:42.282 INFO:teuthology.orchestra.run.smithi022.stdout:CentOS Stream 9 - Extras packages 54 kB/s | 16 kB 00:00 2024-03-28T17:50:42.731 INFO:teuthology.orchestra.run.smithi079.stdout:CentOS Stream 9 - Extras packages 12 kB/s | 16 kB 00:01 2024-03-28T17:50:43.079 INFO:teuthology.orchestra.run.smithi134.stdout:Extra Packages for Enterprise Linux 17 MB/s | 21 MB 00:01 2024-03-28T17:50:43.717 INFO:teuthology.orchestra.run.smithi022.stdout:Extra Packages for Enterprise Linux 16 MB/s | 21 MB 00:01 2024-03-28T17:50:46.740 INFO:teuthology.orchestra.run.smithi079.stdout:Extra Packages for Enterprise Linux 5.4 MB/s | 21 MB 00:03 2024-03-28T17:50:49.007 INFO:teuthology.orchestra.run.smithi134.stdout:lab-extras 14 kB/s | 1.7 kB 00:00 2024-03-28T17:50:49.758 INFO:teuthology.orchestra.run.smithi022.stdout:lab-extras 18 kB/s | 1.7 kB 00:00 2024-03-28T17:50:51.571 INFO:teuthology.orchestra.run.smithi134.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-03-28T17:50:51.572 INFO:teuthology.orchestra.run.smithi134.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-03-28T17:50:51.592 INFO:teuthology.orchestra.run.smithi134.stdout:Package NetworkManager-1:1.44.0-2.el9.x86_64 is already installed. 2024-03-28T17:50:51.593 INFO:teuthology.orchestra.run.smithi134.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-03-28T17:50:51.594 INFO:teuthology.orchestra.run.smithi134.stdout:Package util-linux-2.37.4-15.el9.x86_64 is already installed. 2024-03-28T17:50:51.595 INFO:teuthology.orchestra.run.smithi134.stdout:Package libacl-devel-2.3.1-3.el9.x86_64 is already installed. 2024-03-28T17:50:51.598 INFO:teuthology.orchestra.run.smithi134.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-03-28T17:50:51.601 INFO:teuthology.orchestra.run.smithi134.stdout:Package xfsdump-3.1.12-3.el9.x86_64 is already installed. 2024-03-28T17:50:51.602 INFO:teuthology.orchestra.run.smithi134.stdout:Package xfsprogs-5.19.0-4.el9.x86_64 is already installed. 2024-03-28T17:50:51.687 INFO:teuthology.orchestra.run.smithi134.stdout:Dependencies resolved. 2024-03-28T17:50:51.723 INFO:teuthology.orchestra.run.smithi134.stdout:============================================================================================ 2024-03-28T17:50:51.723 INFO:teuthology.orchestra.run.smithi134.stdout: Package Arch Version Repository Size 2024-03-28T17:50:51.724 INFO:teuthology.orchestra.run.smithi134.stdout:============================================================================================ 2024-03-28T17:50:51.724 INFO:teuthology.orchestra.run.smithi134.stdout:Installing: 2024-03-28T17:50:51.724 INFO:teuthology.orchestra.run.smithi134.stdout: bison x86_64 3.7.4-5.el9 CentOS-AppStream 920 k 2024-03-28T17:50:51.724 INFO:teuthology.orchestra.run.smithi134.stdout: ceph x86_64 2:19.0.0-2145.g3be9a5c9.el9 ceph 6.6 k 2024-03-28T17:50:51.724 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-base x86_64 2:19.0.0-2145.g3be9a5c9.el9 ceph 5.4 M 2024-03-28T17:50:51.724 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-fuse x86_64 2:19.0.0-2145.g3be9a5c9.el9 ceph 890 k 2024-03-28T17:50:51.724 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-2145.g3be9a5c9.el9 ceph 145 k 2024-03-28T17:50:51.724 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-mgr x86_64 2:19.0.0-2145.g3be9a5c9.el9 ceph 1.5 M 2024-03-28T17:50:51.724 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-mgr-cephadm noarch 2:19.0.0-2145.g3be9a5c9.el9 ceph-noarch 139 k 2024-03-28T17:50:51.724 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-mgr-dashboard noarch 2:19.0.0-2145.g3be9a5c9.el9 ceph-noarch 1.8 M 2024-03-28T17:50:51.724 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-2145.g3be9a5c9.el9 ceph-noarch 7.4 M 2024-03-28T17:50:51.724 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-mgr-rook noarch 2:19.0.0-2145.g3be9a5c9.el9 ceph-noarch 49 k 2024-03-28T17:50:51.724 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-radosgw x86_64 2:19.0.0-2145.g3be9a5c9.el9 ceph 9.9 M 2024-03-28T17:50:51.724 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-test x86_64 2:19.0.0-2145.g3be9a5c9.el9 ceph 48 M 2024-03-28T17:50:51.725 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-volume noarch 2:19.0.0-2145.g3be9a5c9.el9 ceph-noarch 264 k 2024-03-28T17:50:51.725 INFO:teuthology.orchestra.run.smithi134.stdout: cephadm noarch 2:19.0.0-2145.g3be9a5c9.el9 ceph-noarch 748 k 2024-03-28T17:50:51.725 INFO:teuthology.orchestra.run.smithi134.stdout: cephfs-mirror x86_64 2:19.0.0-2145.g3be9a5c9.el9 ceph 220 k 2024-03-28T17:50:51.725 INFO:teuthology.orchestra.run.smithi134.stdout: cephfs-shell x86_64 2:19.0.0-2145.g3be9a5c9.el9 ceph 27 k 2024-03-28T17:50:51.725 INFO:teuthology.orchestra.run.smithi134.stdout: cephfs-top noarch 2:19.0.0-2145.g3be9a5c9.el9 ceph-noarch 18 k 2024-03-28T17:50:51.725 INFO:teuthology.orchestra.run.smithi134.stdout: elfutils-libelf-devel x86_64 0.190-2.el9 appstream 21 k 2024-03-28T17:50:51.725 INFO:teuthology.orchestra.run.smithi134.stdout: flex x86_64 2.6.4-9.el9 CentOS-AppStream 307 k 2024-03-28T17:50:51.725 INFO:teuthology.orchestra.run.smithi134.stdout: libaio-devel x86_64 0.3.111-13.el9 CentOS-AppStream 10 k 2024-03-28T17:50:51.725 INFO:teuthology.orchestra.run.smithi134.stdout: libcephfs-devel x86_64 2:19.0.0-2145.g3be9a5c9.el9 ceph 31 k 2024-03-28T17:50:51.725 INFO:teuthology.orchestra.run.smithi134.stdout: libcephfs2 x86_64 2:19.0.0-2145.g3be9a5c9.el9 ceph 746 k 2024-03-28T17:50:51.725 INFO:teuthology.orchestra.run.smithi134.stdout: librados-devel x86_64 2:19.0.0-2145.g3be9a5c9.el9 ceph 127 k 2024-03-28T17:50:51.725 INFO:teuthology.orchestra.run.smithi134.stdout: libtool x86_64 2.4.6-45.el9 CentOS-AppStream 578 k 2024-03-28T17:50:51.725 INFO:teuthology.orchestra.run.smithi134.stdout: libuuid-devel x86_64 2.37.4-18.el9 appstream 25 k 2024-03-28T17:50:51.725 INFO:teuthology.orchestra.run.smithi134.stdout: openssl-devel x86_64 1:3.0.7-27.el9 appstream 4.1 M 2024-03-28T17:50:51.726 INFO:teuthology.orchestra.run.smithi134.stdout: postgresql x86_64 13.14-1.el9 appstream 1.6 M 2024-03-28T17:50:51.726 INFO:teuthology.orchestra.run.smithi134.stdout: postgresql-contrib x86_64 13.14-1.el9 appstream 911 k 2024-03-28T17:50:51.726 INFO:teuthology.orchestra.run.smithi134.stdout: postgresql-server x86_64 13.14-1.el9 appstream 5.8 M 2024-03-28T17:50:51.726 INFO:teuthology.orchestra.run.smithi134.stdout: python3-cephfs x86_64 2:19.0.0-2145.g3be9a5c9.el9 ceph 162 k 2024-03-28T17:50:51.726 INFO:teuthology.orchestra.run.smithi134.stdout: python3-rados x86_64 2:19.0.0-2145.g3be9a5c9.el9 ceph 321 k 2024-03-28T17:50:51.726 INFO:teuthology.orchestra.run.smithi134.stdout: python3-rbd x86_64 2:19.0.0-2145.g3be9a5c9.el9 ceph 298 k 2024-03-28T17:50:51.726 INFO:teuthology.orchestra.run.smithi134.stdout: python3-rgw x86_64 2:19.0.0-2145.g3be9a5c9.el9 ceph 99 k 2024-03-28T17:50:51.726 INFO:teuthology.orchestra.run.smithi134.stdout: rbd-fuse x86_64 2:19.0.0-2145.g3be9a5c9.el9 ceph 85 k 2024-03-28T17:50:51.726 INFO:teuthology.orchestra.run.smithi134.stdout: rbd-mirror x86_64 2:19.0.0-2145.g3be9a5c9.el9 ceph 3.1 M 2024-03-28T17:50:51.726 INFO:teuthology.orchestra.run.smithi134.stdout: rbd-nbd x86_64 2:19.0.0-2145.g3be9a5c9.el9 ceph 170 k 2024-03-28T17:50:51.726 INFO:teuthology.orchestra.run.smithi134.stdout: xfsprogs-devel x86_64 6.3.0-1.el9 appstream 94 k 2024-03-28T17:50:51.726 INFO:teuthology.orchestra.run.smithi134.stdout:Upgrading: 2024-03-28T17:50:51.726 INFO:teuthology.orchestra.run.smithi134.stdout: NetworkManager x86_64 1:1.47.2-1.el9 baseos 2.3 M 2024-03-28T17:50:51.727 INFO:teuthology.orchestra.run.smithi134.stdout: NetworkManager-libnm x86_64 1:1.47.2-1.el9 baseos 1.8 M 2024-03-28T17:50:51.727 INFO:teuthology.orchestra.run.smithi134.stdout: NetworkManager-team x86_64 1:1.47.2-1.el9 baseos 42 k 2024-03-28T17:50:51.727 INFO:teuthology.orchestra.run.smithi134.stdout: NetworkManager-tui x86_64 1:1.47.2-1.el9 baseos 248 k 2024-03-28T17:50:51.727 INFO:teuthology.orchestra.run.smithi134.stdout: acl x86_64 2.3.1-4.el9 baseos 71 k 2024-03-28T17:50:51.727 INFO:teuthology.orchestra.run.smithi134.stdout: elfutils-debuginfod-client x86_64 0.190-2.el9 baseos 36 k 2024-03-28T17:50:51.727 INFO:teuthology.orchestra.run.smithi134.stdout: elfutils-libelf x86_64 0.190-2.el9 baseos 192 k 2024-03-28T17:50:51.727 INFO:teuthology.orchestra.run.smithi134.stdout: elfutils-libs x86_64 0.190-2.el9 baseos 256 k 2024-03-28T17:50:51.727 INFO:teuthology.orchestra.run.smithi134.stdout: libacl x86_64 2.3.1-4.el9 baseos 23 k 2024-03-28T17:50:51.727 INFO:teuthology.orchestra.run.smithi134.stdout: libacl-devel x86_64 2.3.1-4.el9 appstream 74 k 2024-03-28T17:50:51.727 INFO:teuthology.orchestra.run.smithi134.stdout: libblkid x86_64 2.37.4-18.el9 baseos 108 k 2024-03-28T17:50:51.727 INFO:teuthology.orchestra.run.smithi134.stdout: libfdisk x86_64 2.37.4-18.el9 baseos 155 k 2024-03-28T17:50:51.727 INFO:teuthology.orchestra.run.smithi134.stdout: libmount x86_64 2.37.4-18.el9 baseos 136 k 2024-03-28T17:50:51.727 INFO:teuthology.orchestra.run.smithi134.stdout: librados2 x86_64 2:19.0.0-2145.g3be9a5c9.el9 ceph 3.4 M 2024-03-28T17:50:51.728 INFO:teuthology.orchestra.run.smithi134.stdout: librbd1 x86_64 2:19.0.0-2145.g3be9a5c9.el9 ceph 3.1 M 2024-03-28T17:50:51.728 INFO:teuthology.orchestra.run.smithi134.stdout: libsmartcols x86_64 2.37.4-18.el9 baseos 64 k 2024-03-28T17:50:51.728 INFO:teuthology.orchestra.run.smithi134.stdout: libuuid x86_64 2.37.4-18.el9 baseos 29 k 2024-03-28T17:50:51.728 INFO:teuthology.orchestra.run.smithi134.stdout: openssl x86_64 1:3.0.7-27.el9 baseos 1.2 M 2024-03-28T17:50:51.728 INFO:teuthology.orchestra.run.smithi134.stdout: openssl-libs x86_64 1:3.0.7-27.el9 baseos 2.2 M 2024-03-28T17:50:51.728 INFO:teuthology.orchestra.run.smithi134.stdout: util-linux x86_64 2.37.4-18.el9 baseos 2.3 M 2024-03-28T17:50:51.728 INFO:teuthology.orchestra.run.smithi134.stdout: util-linux-core x86_64 2.37.4-18.el9 baseos 465 k 2024-03-28T17:50:51.728 INFO:teuthology.orchestra.run.smithi134.stdout: util-linux-user x86_64 2.37.4-18.el9 baseos 31 k 2024-03-28T17:50:51.728 INFO:teuthology.orchestra.run.smithi134.stdout: xfsdump x86_64 3.1.12-4.el9 CentOS-BaseOS 321 k 2024-03-28T17:50:51.728 INFO:teuthology.orchestra.run.smithi134.stdout: xfsprogs x86_64 6.3.0-1.el9 baseos 1.1 M 2024-03-28T17:50:51.728 INFO:teuthology.orchestra.run.smithi134.stdout:Installing dependencies: 2024-03-28T17:50:51.728 INFO:teuthology.orchestra.run.smithi134.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-03-28T17:50:51.728 INFO:teuthology.orchestra.run.smithi134.stdout: automake noarch 1.16.2-8.el9 CentOS-AppStream 662 k 2024-03-28T17:50:51.729 INFO:teuthology.orchestra.run.smithi134.stdout: boost-program-options x86_64 1.75.0-8.el9 CentOS-AppStream 106 k 2024-03-28T17:50:51.729 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-common x86_64 2:19.0.0-2145.g3be9a5c9.el9 ceph 20 M 2024-03-28T17:50:51.729 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-grafana-dashboards noarch 2:19.0.0-2145.g3be9a5c9.el9 ceph-noarch 34 k 2024-03-28T17:50:51.729 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-mds x86_64 2:19.0.0-2145.g3be9a5c9.el9 ceph 2.3 M 2024-03-28T17:50:51.729 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-mgr-modules-core noarch 2:19.0.0-2145.g3be9a5c9.el9 ceph-noarch 248 k 2024-03-28T17:50:51.729 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-mon x86_64 2:19.0.0-2145.g3be9a5c9.el9 ceph 4.7 M 2024-03-28T17:50:51.729 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-osd x86_64 2:19.0.0-2145.g3be9a5c9.el9 ceph 17 M 2024-03-28T17:50:51.729 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-prometheus-alerts noarch 2:19.0.0-2145.g3be9a5c9.el9 ceph-noarch 16 k 2024-03-28T17:50:51.729 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-selinux x86_64 2:19.0.0-2145.g3be9a5c9.el9 ceph 25 k 2024-03-28T17:50:51.729 INFO:teuthology.orchestra.run.smithi134.stdout: flexiblas x86_64 3.0.4-8.el9 CentOS-AppStream 32 k 2024-03-28T17:50:51.729 INFO:teuthology.orchestra.run.smithi134.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 CentOS-AppStream 3.0 M 2024-03-28T17:50:51.729 INFO:teuthology.orchestra.run.smithi134.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 CentOS-AppStream 17 k 2024-03-28T17:50:51.729 INFO:teuthology.orchestra.run.smithi134.stdout: fuse x86_64 2.9.9-15.el9 CentOS-BaseOS 79 k 2024-03-28T17:50:51.729 INFO:teuthology.orchestra.run.smithi134.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-03-28T17:50:51.730 INFO:teuthology.orchestra.run.smithi134.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-03-28T17:50:51.730 INFO:teuthology.orchestra.run.smithi134.stdout: jq x86_64 1.6-16.el9 baseos 187 k 2024-03-28T17:50:51.730 INFO:teuthology.orchestra.run.smithi134.stdout: libarrow x86_64 9.0.0-10.el9 epel 4.4 M 2024-03-28T17:50:51.730 INFO:teuthology.orchestra.run.smithi134.stdout: libarrow-doc noarch 9.0.0-10.el9 epel 26 k 2024-03-28T17:50:51.730 INFO:teuthology.orchestra.run.smithi134.stdout: libcephsqlite x86_64 2:19.0.0-2145.g3be9a5c9.el9 ceph 163 k 2024-03-28T17:50:51.730 INFO:teuthology.orchestra.run.smithi134.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-03-28T17:50:51.730 INFO:teuthology.orchestra.run.smithi134.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-03-28T17:50:51.730 INFO:teuthology.orchestra.run.smithi134.stdout: libpmemobj x86_64 1.12.1-1.el9 CentOS-AppStream 159 k 2024-03-28T17:50:51.730 INFO:teuthology.orchestra.run.smithi134.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-03-28T17:50:51.730 INFO:teuthology.orchestra.run.smithi134.stdout: librabbitmq x86_64 0.11.0-7.el9 CentOS-AppStream 44 k 2024-03-28T17:50:51.730 INFO:teuthology.orchestra.run.smithi134.stdout: libradosstriper1 x86_64 2:19.0.0-2145.g3be9a5c9.el9 ceph 501 k 2024-03-28T17:50:51.730 INFO:teuthology.orchestra.run.smithi134.stdout: librdkafka x86_64 1.6.1-102.el9 CentOS-AppStream 661 k 2024-03-28T17:50:51.730 INFO:teuthology.orchestra.run.smithi134.stdout: librgw2 x86_64 2:19.0.0-2145.g3be9a5c9.el9 ceph 5.1 M 2024-03-28T17:50:51.731 INFO:teuthology.orchestra.run.smithi134.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-03-28T17:50:51.731 INFO:teuthology.orchestra.run.smithi134.stdout: libzstd-devel x86_64 1.5.1-2.el9 CentOS-AppStream 46 k 2024-03-28T17:50:51.731 INFO:teuthology.orchestra.run.smithi134.stdout: lttng-ust x86_64 2.12.0-6.el9 CentOS-AppStream 282 k 2024-03-28T17:50:51.731 INFO:teuthology.orchestra.run.smithi134.stdout: lua x86_64 5.4.4-4.el9 CentOS-AppStream 187 k 2024-03-28T17:50:51.731 INFO:teuthology.orchestra.run.smithi134.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-03-28T17:50:51.731 INFO:teuthology.orchestra.run.smithi134.stdout: luarocks noarch 3.9.2-1.el9 epel 151 k 2024-03-28T17:50:51.731 INFO:teuthology.orchestra.run.smithi134.stdout: oniguruma x86_64 6.9.6-1.el9.5 CentOS-AppStream 217 k 2024-03-28T17:50:51.731 INFO:teuthology.orchestra.run.smithi134.stdout: openblas x86_64 0.3.21-2.el9 CentOS-AppStream 34 k 2024-03-28T17:50:51.731 INFO:teuthology.orchestra.run.smithi134.stdout: openblas-openmp x86_64 0.3.21-2.el9 CentOS-AppStream 4.7 M 2024-03-28T17:50:51.731 INFO:teuthology.orchestra.run.smithi134.stdout: parquet-libs x86_64 9.0.0-10.el9 epel 838 k 2024-03-28T17:50:51.731 INFO:teuthology.orchestra.run.smithi134.stdout: postgresql-private-libs x86_64 13.14-1.el9 appstream 133 k 2024-03-28T17:50:51.731 INFO:teuthology.orchestra.run.smithi134.stdout: protobuf x86_64 3.14.0-13.el9 CentOS-AppStream 1.0 M 2024-03-28T17:50:51.731 INFO:teuthology.orchestra.run.smithi134.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-03-28T17:50:51.732 INFO:teuthology.orchestra.run.smithi134.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-03-28T17:50:51.732 INFO:teuthology.orchestra.run.smithi134.stdout: python3-attrs noarch 20.3.0-7.el9 CentOS-AppStream 78 k 2024-03-28T17:50:51.732 INFO:teuthology.orchestra.run.smithi134.stdout: python3-babel noarch 2.9.1-2.el9 CentOS-AppStream 5.8 M 2024-03-28T17:50:51.732 INFO:teuthology.orchestra.run.smithi134.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-03-28T17:50:51.732 INFO:teuthology.orchestra.run.smithi134.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-03-28T17:50:51.732 INFO:teuthology.orchestra.run.smithi134.stdout: python3-ceph-argparse x86_64 2:19.0.0-2145.g3be9a5c9.el9 ceph 45 k 2024-03-28T17:50:51.732 INFO:teuthology.orchestra.run.smithi134.stdout: python3-ceph-common x86_64 2:19.0.0-2145.g3be9a5c9.el9 ceph 131 k 2024-03-28T17:50:51.732 INFO:teuthology.orchestra.run.smithi134.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-03-28T17:50:51.732 INFO:teuthology.orchestra.run.smithi134.stdout: python3-cffi x86_64 1.14.5-5.el9 CentOS-BaseOS 241 k 2024-03-28T17:50:51.732 INFO:teuthology.orchestra.run.smithi134.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-03-28T17:50:51.732 INFO:teuthology.orchestra.run.smithi134.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-03-28T17:50:51.732 INFO:teuthology.orchestra.run.smithi134.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-03-28T17:50:51.732 INFO:teuthology.orchestra.run.smithi134.stdout: python3-colorama noarch 0.4.4-12.el9 epel 35 k 2024-03-28T17:50:51.732 INFO:teuthology.orchestra.run.smithi134.stdout: python3-cryptography x86_64 36.0.1-4.el9 CentOS-BaseOS 1.1 M 2024-03-28T17:50:51.732 INFO:teuthology.orchestra.run.smithi134.stdout: python3-devel x86_64 3.9.17-2.el9 appstream 245 k 2024-03-28T17:50:51.733 INFO:teuthology.orchestra.run.smithi134.stdout: python3-google-auth noarch 1:2.28.2-1.el9 epel 203 k 2024-03-28T17:50:51.733 INFO:teuthology.orchestra.run.smithi134.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-03-28T17:50:51.733 INFO:teuthology.orchestra.run.smithi134.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-03-28T17:50:51.733 INFO:teuthology.orchestra.run.smithi134.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-03-28T17:50:51.733 INFO:teuthology.orchestra.run.smithi134.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-03-28T17:50:51.733 INFO:teuthology.orchestra.run.smithi134.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-03-28T17:50:51.733 INFO:teuthology.orchestra.run.smithi134.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-03-28T17:50:51.733 INFO:teuthology.orchestra.run.smithi134.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-03-28T17:50:51.733 INFO:teuthology.orchestra.run.smithi134.stdout: python3-jinja2 noarch 2.11.3-5.el9 appstream 248 k 2024-03-28T17:50:51.733 INFO:teuthology.orchestra.run.smithi134.stdout: python3-jsonpatch noarch 1.21-16.el9 CentOS-AppStream 24 k 2024-03-28T17:50:51.733 INFO:teuthology.orchestra.run.smithi134.stdout: python3-jsonpointer noarch 2.0-4.el9 CentOS-AppStream 18 k 2024-03-28T17:50:51.733 INFO:teuthology.orchestra.run.smithi134.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-03-28T17:50:51.733 INFO:teuthology.orchestra.run.smithi134.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-03-28T17:50:51.733 INFO:teuthology.orchestra.run.smithi134.stdout: python3-mako noarch 1.1.4-6.el9 CentOS-AppStream 144 k 2024-03-28T17:50:51.734 INFO:teuthology.orchestra.run.smithi134.stdout: python3-markupsafe x86_64 1.1.1-12.el9 CentOS-AppStream 32 k 2024-03-28T17:50:51.734 INFO:teuthology.orchestra.run.smithi134.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-03-28T17:50:51.734 INFO:teuthology.orchestra.run.smithi134.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-03-28T17:50:51.734 INFO:teuthology.orchestra.run.smithi134.stdout: python3-numpy x86_64 1:1.20.1-5.el9 CentOS-AppStream 4.8 M 2024-03-28T17:50:51.734 INFO:teuthology.orchestra.run.smithi134.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 CentOS-AppStream 217 k 2024-03-28T17:50:51.734 INFO:teuthology.orchestra.run.smithi134.stdout: python3-oauthlib noarch 3.1.1-5.el9 CentOS-AppStream 174 k 2024-03-28T17:50:51.734 INFO:teuthology.orchestra.run.smithi134.stdout: python3-packaging noarch 20.9-5.el9 CentOS-AppStream 69 k 2024-03-28T17:50:51.734 INFO:teuthology.orchestra.run.smithi134.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-03-28T17:50:51.734 INFO:teuthology.orchestra.run.smithi134.stdout: python3-ply noarch 3.11-14.el9 CentOS-BaseOS 103 k 2024-03-28T17:50:51.734 INFO:teuthology.orchestra.run.smithi134.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-03-28T17:50:51.734 INFO:teuthology.orchestra.run.smithi134.stdout: python3-prettytable noarch 0.7.2-27.el9 CentOS-AppStream 41 k 2024-03-28T17:50:51.734 INFO:teuthology.orchestra.run.smithi134.stdout: python3-protobuf noarch 3.14.0-13.el9 CentOS-AppStream 238 k 2024-03-28T17:50:51.734 INFO:teuthology.orchestra.run.smithi134.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-03-28T17:50:51.734 INFO:teuthology.orchestra.run.smithi134.stdout: python3-pyasn1 noarch 0.4.8-6.el9 CentOS-AppStream 132 k 2024-03-28T17:50:51.735 INFO:teuthology.orchestra.run.smithi134.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 CentOS-AppStream 211 k 2024-03-28T17:50:51.735 INFO:teuthology.orchestra.run.smithi134.stdout: python3-pycparser noarch 2.20-6.el9 CentOS-BaseOS 124 k 2024-03-28T17:50:51.735 INFO:teuthology.orchestra.run.smithi134.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-03-28T17:50:51.735 INFO:teuthology.orchestra.run.smithi134.stdout: python3-pytz noarch 2021.1-5.el9 CentOS-AppStream 47 k 2024-03-28T17:50:51.735 INFO:teuthology.orchestra.run.smithi134.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-03-28T17:50:51.735 INFO:teuthology.orchestra.run.smithi134.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-03-28T17:50:51.735 INFO:teuthology.orchestra.run.smithi134.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 CentOS-AppStream 43 k 2024-03-28T17:50:51.735 INFO:teuthology.orchestra.run.smithi134.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-03-28T17:50:51.735 INFO:teuthology.orchestra.run.smithi134.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-03-28T17:50:51.735 INFO:teuthology.orchestra.run.smithi134.stdout: python3-scipy x86_64 1.6.2-8.el9 CentOS-AppStream 16 M 2024-03-28T17:50:51.735 INFO:teuthology.orchestra.run.smithi134.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-03-28T17:50:51.735 INFO:teuthology.orchestra.run.smithi134.stdout: python3-toml noarch 0.10.2-6.el9 CentOS-AppStream 39 k 2024-03-28T17:50:51.735 INFO:teuthology.orchestra.run.smithi134.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-03-28T17:50:51.735 INFO:teuthology.orchestra.run.smithi134.stdout: python3-urllib3 noarch 1.26.5-5.el9 CentOS-BaseOS 187 k 2024-03-28T17:50:51.735 INFO:teuthology.orchestra.run.smithi134.stdout: python3-wcwidth noarch 0.2.5-8.el9 CentOS-AppStream 41 k 2024-03-28T17:50:51.736 INFO:teuthology.orchestra.run.smithi134.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-03-28T17:50:51.736 INFO:teuthology.orchestra.run.smithi134.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-03-28T17:50:51.736 INFO:teuthology.orchestra.run.smithi134.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-03-28T17:50:51.736 INFO:teuthology.orchestra.run.smithi134.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-03-28T17:50:51.736 INFO:teuthology.orchestra.run.smithi134.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-03-28T17:50:51.736 INFO:teuthology.orchestra.run.smithi134.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-03-28T17:50:51.736 INFO:teuthology.orchestra.run.smithi134.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-03-28T17:50:51.736 INFO:teuthology.orchestra.run.smithi134.stdout: socat x86_64 1.7.4.1-5.el9 CentOS-AppStream 300 k 2024-03-28T17:50:51.736 INFO:teuthology.orchestra.run.smithi134.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-03-28T17:50:51.736 INFO:teuthology.orchestra.run.smithi134.stdout: uuid x86_64 1.6.2-55.el9 CentOS-AppStream 56 k 2024-03-28T17:50:51.736 INFO:teuthology.orchestra.run.smithi134.stdout: xmlstarlet x86_64 1.6.1-20.el9 CentOS-AppStream 63 k 2024-03-28T17:50:51.736 INFO:teuthology.orchestra.run.smithi134.stdout: zlib-devel x86_64 1.2.11-40.el9 CentOS-AppStream 44 k 2024-03-28T17:50:51.736 INFO:teuthology.orchestra.run.smithi134.stdout:Installing weak dependencies: 2024-03-28T17:50:51.736 INFO:teuthology.orchestra.run.smithi134.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-03-28T17:50:51.737 INFO:teuthology.orchestra.run.smithi134.stdout: 2024-03-28T17:50:51.737 INFO:teuthology.orchestra.run.smithi134.stdout:Transaction Summary 2024-03-28T17:50:51.737 INFO:teuthology.orchestra.run.smithi134.stdout:============================================================================================ 2024-03-28T17:50:51.737 INFO:teuthology.orchestra.run.smithi134.stdout:Install 150 Packages 2024-03-28T17:50:51.737 INFO:teuthology.orchestra.run.smithi134.stdout:Upgrade 24 Packages 2024-03-28T17:50:51.737 INFO:teuthology.orchestra.run.smithi134.stdout: 2024-03-28T17:50:51.737 INFO:teuthology.orchestra.run.smithi134.stdout:Total download size: 225 M 2024-03-28T17:50:51.737 INFO:teuthology.orchestra.run.smithi134.stdout:Downloading Packages: 2024-03-28T17:50:52.465 INFO:teuthology.orchestra.run.smithi079.stdout:lab-extras 28 kB/s | 1.7 kB 00:00 2024-03-28T17:50:52.516 INFO:teuthology.orchestra.run.smithi022.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-03-28T17:50:52.517 INFO:teuthology.orchestra.run.smithi022.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-03-28T17:50:52.538 INFO:teuthology.orchestra.run.smithi022.stdout:Package NetworkManager-1:1.44.0-2.el9.x86_64 is already installed. 2024-03-28T17:50:52.540 INFO:teuthology.orchestra.run.smithi022.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-03-28T17:50:52.541 INFO:teuthology.orchestra.run.smithi022.stdout:Package util-linux-2.37.4-15.el9.x86_64 is already installed. 2024-03-28T17:50:52.542 INFO:teuthology.orchestra.run.smithi022.stdout:Package libacl-devel-2.3.1-3.el9.x86_64 is already installed. 2024-03-28T17:50:52.545 INFO:teuthology.orchestra.run.smithi022.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-03-28T17:50:52.549 INFO:teuthology.orchestra.run.smithi022.stdout:Package xfsdump-3.1.12-3.el9.x86_64 is already installed. 2024-03-28T17:50:52.550 INFO:teuthology.orchestra.run.smithi022.stdout:Package xfsprogs-5.19.0-4.el9.x86_64 is already installed. 2024-03-28T17:50:52.646 INFO:teuthology.orchestra.run.smithi022.stdout:Dependencies resolved. 2024-03-28T17:50:52.663 INFO:teuthology.orchestra.run.smithi022.stdout:============================================================================================ 2024-03-28T17:50:52.663 INFO:teuthology.orchestra.run.smithi022.stdout: Package Arch Version Repository Size 2024-03-28T17:50:52.663 INFO:teuthology.orchestra.run.smithi022.stdout:============================================================================================ 2024-03-28T17:50:52.663 INFO:teuthology.orchestra.run.smithi022.stdout:Installing: 2024-03-28T17:50:52.663 INFO:teuthology.orchestra.run.smithi022.stdout: bison x86_64 3.7.4-5.el9 CentOS-AppStream 920 k 2024-03-28T17:50:52.663 INFO:teuthology.orchestra.run.smithi022.stdout: ceph x86_64 2:19.0.0-2145.g3be9a5c9.el9 ceph 6.6 k 2024-03-28T17:50:52.663 INFO:teuthology.orchestra.run.smithi022.stdout: ceph-base x86_64 2:19.0.0-2145.g3be9a5c9.el9 ceph 5.4 M 2024-03-28T17:50:52.663 INFO:teuthology.orchestra.run.smithi022.stdout: ceph-fuse x86_64 2:19.0.0-2145.g3be9a5c9.el9 ceph 890 k 2024-03-28T17:50:52.663 INFO:teuthology.orchestra.run.smithi022.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-2145.g3be9a5c9.el9 ceph 145 k 2024-03-28T17:50:52.663 INFO:teuthology.orchestra.run.smithi022.stdout: ceph-mgr x86_64 2:19.0.0-2145.g3be9a5c9.el9 ceph 1.5 M 2024-03-28T17:50:52.663 INFO:teuthology.orchestra.run.smithi022.stdout: ceph-mgr-cephadm noarch 2:19.0.0-2145.g3be9a5c9.el9 ceph-noarch 139 k 2024-03-28T17:50:52.663 INFO:teuthology.orchestra.run.smithi022.stdout: ceph-mgr-dashboard noarch 2:19.0.0-2145.g3be9a5c9.el9 ceph-noarch 1.8 M 2024-03-28T17:50:52.664 INFO:teuthology.orchestra.run.smithi022.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-2145.g3be9a5c9.el9 ceph-noarch 7.4 M 2024-03-28T17:50:52.664 INFO:teuthology.orchestra.run.smithi022.stdout: ceph-mgr-rook noarch 2:19.0.0-2145.g3be9a5c9.el9 ceph-noarch 49 k 2024-03-28T17:50:52.664 INFO:teuthology.orchestra.run.smithi022.stdout: ceph-radosgw x86_64 2:19.0.0-2145.g3be9a5c9.el9 ceph 9.9 M 2024-03-28T17:50:52.664 INFO:teuthology.orchestra.run.smithi022.stdout: ceph-test x86_64 2:19.0.0-2145.g3be9a5c9.el9 ceph 48 M 2024-03-28T17:50:52.664 INFO:teuthology.orchestra.run.smithi022.stdout: ceph-volume noarch 2:19.0.0-2145.g3be9a5c9.el9 ceph-noarch 264 k 2024-03-28T17:50:52.664 INFO:teuthology.orchestra.run.smithi022.stdout: cephadm noarch 2:19.0.0-2145.g3be9a5c9.el9 ceph-noarch 748 k 2024-03-28T17:50:52.664 INFO:teuthology.orchestra.run.smithi022.stdout: cephfs-mirror x86_64 2:19.0.0-2145.g3be9a5c9.el9 ceph 220 k 2024-03-28T17:50:52.664 INFO:teuthology.orchestra.run.smithi022.stdout: cephfs-shell x86_64 2:19.0.0-2145.g3be9a5c9.el9 ceph 27 k 2024-03-28T17:50:52.664 INFO:teuthology.orchestra.run.smithi022.stdout: cephfs-top noarch 2:19.0.0-2145.g3be9a5c9.el9 ceph-noarch 18 k 2024-03-28T17:50:52.664 INFO:teuthology.orchestra.run.smithi022.stdout: elfutils-libelf-devel x86_64 0.190-2.el9 appstream 21 k 2024-03-28T17:50:52.664 INFO:teuthology.orchestra.run.smithi022.stdout: flex x86_64 2.6.4-9.el9 CentOS-AppStream 307 k 2024-03-28T17:50:52.664 INFO:teuthology.orchestra.run.smithi022.stdout: libaio-devel x86_64 0.3.111-13.el9 CentOS-AppStream 10 k 2024-03-28T17:50:52.665 INFO:teuthology.orchestra.run.smithi022.stdout: libcephfs-devel x86_64 2:19.0.0-2145.g3be9a5c9.el9 ceph 31 k 2024-03-28T17:50:52.665 INFO:teuthology.orchestra.run.smithi022.stdout: libcephfs2 x86_64 2:19.0.0-2145.g3be9a5c9.el9 ceph 746 k 2024-03-28T17:50:52.665 INFO:teuthology.orchestra.run.smithi022.stdout: librados-devel x86_64 2:19.0.0-2145.g3be9a5c9.el9 ceph 127 k 2024-03-28T17:50:52.665 INFO:teuthology.orchestra.run.smithi022.stdout: libtool x86_64 2.4.6-45.el9 CentOS-AppStream 578 k 2024-03-28T17:50:52.665 INFO:teuthology.orchestra.run.smithi022.stdout: libuuid-devel x86_64 2.37.4-18.el9 appstream 25 k 2024-03-28T17:50:52.665 INFO:teuthology.orchestra.run.smithi022.stdout: openssl-devel x86_64 1:3.0.7-27.el9 appstream 4.1 M 2024-03-28T17:50:52.665 INFO:teuthology.orchestra.run.smithi022.stdout: postgresql x86_64 13.14-1.el9 appstream 1.6 M 2024-03-28T17:50:52.665 INFO:teuthology.orchestra.run.smithi022.stdout: postgresql-contrib x86_64 13.14-1.el9 appstream 911 k 2024-03-28T17:50:52.665 INFO:teuthology.orchestra.run.smithi022.stdout: postgresql-server x86_64 13.14-1.el9 appstream 5.8 M 2024-03-28T17:50:52.665 INFO:teuthology.orchestra.run.smithi022.stdout: python3-cephfs x86_64 2:19.0.0-2145.g3be9a5c9.el9 ceph 162 k 2024-03-28T17:50:52.665 INFO:teuthology.orchestra.run.smithi022.stdout: python3-rados x86_64 2:19.0.0-2145.g3be9a5c9.el9 ceph 321 k 2024-03-28T17:50:52.665 INFO:teuthology.orchestra.run.smithi022.stdout: python3-rbd x86_64 2:19.0.0-2145.g3be9a5c9.el9 ceph 298 k 2024-03-28T17:50:52.665 INFO:teuthology.orchestra.run.smithi022.stdout: python3-rgw x86_64 2:19.0.0-2145.g3be9a5c9.el9 ceph 99 k 2024-03-28T17:50:52.665 INFO:teuthology.orchestra.run.smithi022.stdout: rbd-fuse x86_64 2:19.0.0-2145.g3be9a5c9.el9 ceph 85 k 2024-03-28T17:50:52.666 INFO:teuthology.orchestra.run.smithi022.stdout: rbd-mirror x86_64 2:19.0.0-2145.g3be9a5c9.el9 ceph 3.1 M 2024-03-28T17:50:52.666 INFO:teuthology.orchestra.run.smithi022.stdout: rbd-nbd x86_64 2:19.0.0-2145.g3be9a5c9.el9 ceph 170 k 2024-03-28T17:50:52.666 INFO:teuthology.orchestra.run.smithi022.stdout: xfsprogs-devel x86_64 6.3.0-1.el9 appstream 94 k 2024-03-28T17:50:52.666 INFO:teuthology.orchestra.run.smithi022.stdout:Upgrading: 2024-03-28T17:50:52.666 INFO:teuthology.orchestra.run.smithi022.stdout: NetworkManager x86_64 1:1.47.2-1.el9 baseos 2.3 M 2024-03-28T17:50:52.666 INFO:teuthology.orchestra.run.smithi022.stdout: NetworkManager-libnm x86_64 1:1.47.2-1.el9 baseos 1.8 M 2024-03-28T17:50:52.666 INFO:teuthology.orchestra.run.smithi022.stdout: NetworkManager-team x86_64 1:1.47.2-1.el9 baseos 42 k 2024-03-28T17:50:52.666 INFO:teuthology.orchestra.run.smithi022.stdout: NetworkManager-tui x86_64 1:1.47.2-1.el9 baseos 248 k 2024-03-28T17:50:52.666 INFO:teuthology.orchestra.run.smithi022.stdout: acl x86_64 2.3.1-4.el9 baseos 71 k 2024-03-28T17:50:52.666 INFO:teuthology.orchestra.run.smithi022.stdout: elfutils-debuginfod-client x86_64 0.190-2.el9 baseos 36 k 2024-03-28T17:50:52.666 INFO:teuthology.orchestra.run.smithi022.stdout: elfutils-libelf x86_64 0.190-2.el9 baseos 192 k 2024-03-28T17:50:52.666 INFO:teuthology.orchestra.run.smithi022.stdout: elfutils-libs x86_64 0.190-2.el9 baseos 256 k 2024-03-28T17:50:52.666 INFO:teuthology.orchestra.run.smithi022.stdout: libacl x86_64 2.3.1-4.el9 baseos 23 k 2024-03-28T17:50:52.667 INFO:teuthology.orchestra.run.smithi022.stdout: libacl-devel x86_64 2.3.1-4.el9 appstream 74 k 2024-03-28T17:50:52.667 INFO:teuthology.orchestra.run.smithi022.stdout: libblkid x86_64 2.37.4-18.el9 baseos 108 k 2024-03-28T17:50:52.667 INFO:teuthology.orchestra.run.smithi022.stdout: libfdisk x86_64 2.37.4-18.el9 baseos 155 k 2024-03-28T17:50:52.667 INFO:teuthology.orchestra.run.smithi022.stdout: libmount x86_64 2.37.4-18.el9 baseos 136 k 2024-03-28T17:50:52.667 INFO:teuthology.orchestra.run.smithi022.stdout: librados2 x86_64 2:19.0.0-2145.g3be9a5c9.el9 ceph 3.4 M 2024-03-28T17:50:52.667 INFO:teuthology.orchestra.run.smithi022.stdout: librbd1 x86_64 2:19.0.0-2145.g3be9a5c9.el9 ceph 3.1 M 2024-03-28T17:50:52.667 INFO:teuthology.orchestra.run.smithi022.stdout: libsmartcols x86_64 2.37.4-18.el9 baseos 64 k 2024-03-28T17:50:52.667 INFO:teuthology.orchestra.run.smithi022.stdout: libuuid x86_64 2.37.4-18.el9 baseos 29 k 2024-03-28T17:50:52.667 INFO:teuthology.orchestra.run.smithi022.stdout: openssl x86_64 1:3.0.7-27.el9 baseos 1.2 M 2024-03-28T17:50:52.667 INFO:teuthology.orchestra.run.smithi022.stdout: openssl-libs x86_64 1:3.0.7-27.el9 baseos 2.2 M 2024-03-28T17:50:52.667 INFO:teuthology.orchestra.run.smithi022.stdout: util-linux x86_64 2.37.4-18.el9 baseos 2.3 M 2024-03-28T17:50:52.667 INFO:teuthology.orchestra.run.smithi022.stdout: util-linux-core x86_64 2.37.4-18.el9 baseos 465 k 2024-03-28T17:50:52.667 INFO:teuthology.orchestra.run.smithi022.stdout: util-linux-user x86_64 2.37.4-18.el9 baseos 31 k 2024-03-28T17:50:52.668 INFO:teuthology.orchestra.run.smithi022.stdout: xfsdump x86_64 3.1.12-4.el9 CentOS-BaseOS 321 k 2024-03-28T17:50:52.668 INFO:teuthology.orchestra.run.smithi022.stdout: xfsprogs x86_64 6.3.0-1.el9 baseos 1.1 M 2024-03-28T17:50:52.668 INFO:teuthology.orchestra.run.smithi022.stdout:Installing dependencies: 2024-03-28T17:50:52.668 INFO:teuthology.orchestra.run.smithi022.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-03-28T17:50:52.668 INFO:teuthology.orchestra.run.smithi022.stdout: automake noarch 1.16.2-8.el9 CentOS-AppStream 662 k 2024-03-28T17:50:52.668 INFO:teuthology.orchestra.run.smithi022.stdout: boost-program-options x86_64 1.75.0-8.el9 CentOS-AppStream 106 k 2024-03-28T17:50:52.668 INFO:teuthology.orchestra.run.smithi022.stdout: ceph-common x86_64 2:19.0.0-2145.g3be9a5c9.el9 ceph 20 M 2024-03-28T17:50:52.668 INFO:teuthology.orchestra.run.smithi022.stdout: ceph-grafana-dashboards noarch 2:19.0.0-2145.g3be9a5c9.el9 ceph-noarch 34 k 2024-03-28T17:50:52.668 INFO:teuthology.orchestra.run.smithi022.stdout: ceph-mds x86_64 2:19.0.0-2145.g3be9a5c9.el9 ceph 2.3 M 2024-03-28T17:50:52.668 INFO:teuthology.orchestra.run.smithi022.stdout: ceph-mgr-modules-core noarch 2:19.0.0-2145.g3be9a5c9.el9 ceph-noarch 248 k 2024-03-28T17:50:52.668 INFO:teuthology.orchestra.run.smithi022.stdout: ceph-mon x86_64 2:19.0.0-2145.g3be9a5c9.el9 ceph 4.7 M 2024-03-28T17:50:52.668 INFO:teuthology.orchestra.run.smithi022.stdout: ceph-osd x86_64 2:19.0.0-2145.g3be9a5c9.el9 ceph 17 M 2024-03-28T17:50:52.668 INFO:teuthology.orchestra.run.smithi022.stdout: ceph-prometheus-alerts noarch 2:19.0.0-2145.g3be9a5c9.el9 ceph-noarch 16 k 2024-03-28T17:50:52.668 INFO:teuthology.orchestra.run.smithi022.stdout: ceph-selinux x86_64 2:19.0.0-2145.g3be9a5c9.el9 ceph 25 k 2024-03-28T17:50:52.668 INFO:teuthology.orchestra.run.smithi022.stdout: flexiblas x86_64 3.0.4-8.el9 CentOS-AppStream 32 k 2024-03-28T17:50:52.669 INFO:teuthology.orchestra.run.smithi022.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 CentOS-AppStream 3.0 M 2024-03-28T17:50:52.669 INFO:teuthology.orchestra.run.smithi022.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 CentOS-AppStream 17 k 2024-03-28T17:50:52.669 INFO:teuthology.orchestra.run.smithi022.stdout: fuse x86_64 2.9.9-15.el9 CentOS-BaseOS 79 k 2024-03-28T17:50:52.669 INFO:teuthology.orchestra.run.smithi022.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-03-28T17:50:52.669 INFO:teuthology.orchestra.run.smithi022.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-03-28T17:50:52.669 INFO:teuthology.orchestra.run.smithi022.stdout: jq x86_64 1.6-16.el9 baseos 187 k 2024-03-28T17:50:52.669 INFO:teuthology.orchestra.run.smithi022.stdout: libarrow x86_64 9.0.0-10.el9 epel 4.4 M 2024-03-28T17:50:52.669 INFO:teuthology.orchestra.run.smithi022.stdout: libarrow-doc noarch 9.0.0-10.el9 epel 26 k 2024-03-28T17:50:52.669 INFO:teuthology.orchestra.run.smithi022.stdout: libcephsqlite x86_64 2:19.0.0-2145.g3be9a5c9.el9 ceph 163 k 2024-03-28T17:50:52.669 INFO:teuthology.orchestra.run.smithi022.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-03-28T17:50:52.669 INFO:teuthology.orchestra.run.smithi022.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-03-28T17:50:52.669 INFO:teuthology.orchestra.run.smithi022.stdout: libpmemobj x86_64 1.12.1-1.el9 CentOS-AppStream 159 k 2024-03-28T17:50:52.669 INFO:teuthology.orchestra.run.smithi022.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-03-28T17:50:52.670 INFO:teuthology.orchestra.run.smithi022.stdout: librabbitmq x86_64 0.11.0-7.el9 CentOS-AppStream 44 k 2024-03-28T17:50:52.670 INFO:teuthology.orchestra.run.smithi022.stdout: libradosstriper1 x86_64 2:19.0.0-2145.g3be9a5c9.el9 ceph 501 k 2024-03-28T17:50:52.670 INFO:teuthology.orchestra.run.smithi022.stdout: librdkafka x86_64 1.6.1-102.el9 CentOS-AppStream 661 k 2024-03-28T17:50:52.670 INFO:teuthology.orchestra.run.smithi022.stdout: librgw2 x86_64 2:19.0.0-2145.g3be9a5c9.el9 ceph 5.1 M 2024-03-28T17:50:52.670 INFO:teuthology.orchestra.run.smithi022.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-03-28T17:50:52.670 INFO:teuthology.orchestra.run.smithi022.stdout: libzstd-devel x86_64 1.5.1-2.el9 CentOS-AppStream 46 k 2024-03-28T17:50:52.670 INFO:teuthology.orchestra.run.smithi022.stdout: lttng-ust x86_64 2.12.0-6.el9 CentOS-AppStream 282 k 2024-03-28T17:50:52.670 INFO:teuthology.orchestra.run.smithi022.stdout: lua x86_64 5.4.4-4.el9 CentOS-AppStream 187 k 2024-03-28T17:50:52.670 INFO:teuthology.orchestra.run.smithi022.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-03-28T17:50:52.670 INFO:teuthology.orchestra.run.smithi022.stdout: luarocks noarch 3.9.2-1.el9 epel 151 k 2024-03-28T17:50:52.670 INFO:teuthology.orchestra.run.smithi022.stdout: oniguruma x86_64 6.9.6-1.el9.5 CentOS-AppStream 217 k 2024-03-28T17:50:52.670 INFO:teuthology.orchestra.run.smithi022.stdout: openblas x86_64 0.3.21-2.el9 CentOS-AppStream 34 k 2024-03-28T17:50:52.670 INFO:teuthology.orchestra.run.smithi022.stdout: openblas-openmp x86_64 0.3.21-2.el9 CentOS-AppStream 4.7 M 2024-03-28T17:50:52.670 INFO:teuthology.orchestra.run.smithi022.stdout: parquet-libs x86_64 9.0.0-10.el9 epel 838 k 2024-03-28T17:50:52.671 INFO:teuthology.orchestra.run.smithi022.stdout: postgresql-private-libs x86_64 13.14-1.el9 appstream 133 k 2024-03-28T17:50:52.671 INFO:teuthology.orchestra.run.smithi022.stdout: protobuf x86_64 3.14.0-13.el9 CentOS-AppStream 1.0 M 2024-03-28T17:50:52.671 INFO:teuthology.orchestra.run.smithi022.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-03-28T17:50:52.671 INFO:teuthology.orchestra.run.smithi022.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-03-28T17:50:52.671 INFO:teuthology.orchestra.run.smithi022.stdout: python3-attrs noarch 20.3.0-7.el9 CentOS-AppStream 78 k 2024-03-28T17:50:52.671 INFO:teuthology.orchestra.run.smithi022.stdout: python3-babel noarch 2.9.1-2.el9 CentOS-AppStream 5.8 M 2024-03-28T17:50:52.671 INFO:teuthology.orchestra.run.smithi022.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-03-28T17:50:52.671 INFO:teuthology.orchestra.run.smithi022.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-03-28T17:50:52.671 INFO:teuthology.orchestra.run.smithi022.stdout: python3-ceph-argparse x86_64 2:19.0.0-2145.g3be9a5c9.el9 ceph 45 k 2024-03-28T17:50:52.671 INFO:teuthology.orchestra.run.smithi022.stdout: python3-ceph-common x86_64 2:19.0.0-2145.g3be9a5c9.el9 ceph 131 k 2024-03-28T17:50:52.671 INFO:teuthology.orchestra.run.smithi022.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-03-28T17:50:52.671 INFO:teuthology.orchestra.run.smithi022.stdout: python3-cffi x86_64 1.14.5-5.el9 CentOS-BaseOS 241 k 2024-03-28T17:50:52.671 INFO:teuthology.orchestra.run.smithi022.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-03-28T17:50:52.671 INFO:teuthology.orchestra.run.smithi022.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-03-28T17:50:52.671 INFO:teuthology.orchestra.run.smithi022.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-03-28T17:50:52.672 INFO:teuthology.orchestra.run.smithi022.stdout: python3-colorama noarch 0.4.4-12.el9 epel 35 k 2024-03-28T17:50:52.672 INFO:teuthology.orchestra.run.smithi022.stdout: python3-cryptography x86_64 36.0.1-4.el9 CentOS-BaseOS 1.1 M 2024-03-28T17:50:52.672 INFO:teuthology.orchestra.run.smithi022.stdout: python3-devel x86_64 3.9.17-2.el9 appstream 245 k 2024-03-28T17:50:52.672 INFO:teuthology.orchestra.run.smithi022.stdout: python3-google-auth noarch 1:2.28.2-1.el9 epel 203 k 2024-03-28T17:50:52.672 INFO:teuthology.orchestra.run.smithi022.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-03-28T17:50:52.672 INFO:teuthology.orchestra.run.smithi022.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-03-28T17:50:52.672 INFO:teuthology.orchestra.run.smithi022.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-03-28T17:50:52.672 INFO:teuthology.orchestra.run.smithi022.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-03-28T17:50:52.672 INFO:teuthology.orchestra.run.smithi022.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-03-28T17:50:52.672 INFO:teuthology.orchestra.run.smithi022.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-03-28T17:50:52.672 INFO:teuthology.orchestra.run.smithi022.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-03-28T17:50:52.672 INFO:teuthology.orchestra.run.smithi022.stdout: python3-jinja2 noarch 2.11.3-5.el9 appstream 248 k 2024-03-28T17:50:52.672 INFO:teuthology.orchestra.run.smithi022.stdout: python3-jsonpatch noarch 1.21-16.el9 CentOS-AppStream 24 k 2024-03-28T17:50:52.673 INFO:teuthology.orchestra.run.smithi022.stdout: python3-jsonpointer noarch 2.0-4.el9 CentOS-AppStream 18 k 2024-03-28T17:50:52.673 INFO:teuthology.orchestra.run.smithi022.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-03-28T17:50:52.673 INFO:teuthology.orchestra.run.smithi022.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-03-28T17:50:52.673 INFO:teuthology.orchestra.run.smithi022.stdout: python3-mako noarch 1.1.4-6.el9 CentOS-AppStream 144 k 2024-03-28T17:50:52.673 INFO:teuthology.orchestra.run.smithi022.stdout: python3-markupsafe x86_64 1.1.1-12.el9 CentOS-AppStream 32 k 2024-03-28T17:50:52.673 INFO:teuthology.orchestra.run.smithi022.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-03-28T17:50:52.673 INFO:teuthology.orchestra.run.smithi022.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-03-28T17:50:52.673 INFO:teuthology.orchestra.run.smithi022.stdout: python3-numpy x86_64 1:1.20.1-5.el9 CentOS-AppStream 4.8 M 2024-03-28T17:50:52.673 INFO:teuthology.orchestra.run.smithi022.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 CentOS-AppStream 217 k 2024-03-28T17:50:52.673 INFO:teuthology.orchestra.run.smithi022.stdout: python3-oauthlib noarch 3.1.1-5.el9 CentOS-AppStream 174 k 2024-03-28T17:50:52.673 INFO:teuthology.orchestra.run.smithi022.stdout: python3-packaging noarch 20.9-5.el9 CentOS-AppStream 69 k 2024-03-28T17:50:52.673 INFO:teuthology.orchestra.run.smithi022.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-03-28T17:50:52.673 INFO:teuthology.orchestra.run.smithi022.stdout: python3-ply noarch 3.11-14.el9 CentOS-BaseOS 103 k 2024-03-28T17:50:52.673 INFO:teuthology.orchestra.run.smithi022.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-03-28T17:50:52.673 INFO:teuthology.orchestra.run.smithi022.stdout: python3-prettytable noarch 0.7.2-27.el9 CentOS-AppStream 41 k 2024-03-28T17:50:52.674 INFO:teuthology.orchestra.run.smithi022.stdout: python3-protobuf noarch 3.14.0-13.el9 CentOS-AppStream 238 k 2024-03-28T17:50:52.674 INFO:teuthology.orchestra.run.smithi022.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-03-28T17:50:52.674 INFO:teuthology.orchestra.run.smithi022.stdout: python3-pyasn1 noarch 0.4.8-6.el9 CentOS-AppStream 132 k 2024-03-28T17:50:52.674 INFO:teuthology.orchestra.run.smithi022.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 CentOS-AppStream 211 k 2024-03-28T17:50:52.674 INFO:teuthology.orchestra.run.smithi022.stdout: python3-pycparser noarch 2.20-6.el9 CentOS-BaseOS 124 k 2024-03-28T17:50:52.674 INFO:teuthology.orchestra.run.smithi022.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-03-28T17:50:52.674 INFO:teuthology.orchestra.run.smithi022.stdout: python3-pytz noarch 2021.1-5.el9 CentOS-AppStream 47 k 2024-03-28T17:50:52.674 INFO:teuthology.orchestra.run.smithi022.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-03-28T17:50:52.674 INFO:teuthology.orchestra.run.smithi022.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-03-28T17:50:52.674 INFO:teuthology.orchestra.run.smithi022.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 CentOS-AppStream 43 k 2024-03-28T17:50:52.674 INFO:teuthology.orchestra.run.smithi022.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-03-28T17:50:52.674 INFO:teuthology.orchestra.run.smithi022.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-03-28T17:50:52.674 INFO:teuthology.orchestra.run.smithi022.stdout: python3-scipy x86_64 1.6.2-8.el9 CentOS-AppStream 16 M 2024-03-28T17:50:52.674 INFO:teuthology.orchestra.run.smithi022.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-03-28T17:50:52.675 INFO:teuthology.orchestra.run.smithi022.stdout: python3-toml noarch 0.10.2-6.el9 CentOS-AppStream 39 k 2024-03-28T17:50:52.675 INFO:teuthology.orchestra.run.smithi022.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-03-28T17:50:52.675 INFO:teuthology.orchestra.run.smithi022.stdout: python3-urllib3 noarch 1.26.5-5.el9 CentOS-BaseOS 187 k 2024-03-28T17:50:52.675 INFO:teuthology.orchestra.run.smithi022.stdout: python3-wcwidth noarch 0.2.5-8.el9 CentOS-AppStream 41 k 2024-03-28T17:50:52.675 INFO:teuthology.orchestra.run.smithi022.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-03-28T17:50:52.675 INFO:teuthology.orchestra.run.smithi022.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-03-28T17:50:52.675 INFO:teuthology.orchestra.run.smithi022.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-03-28T17:50:52.675 INFO:teuthology.orchestra.run.smithi022.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-03-28T17:50:52.675 INFO:teuthology.orchestra.run.smithi022.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-03-28T17:50:52.675 INFO:teuthology.orchestra.run.smithi022.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-03-28T17:50:52.675 INFO:teuthology.orchestra.run.smithi022.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-03-28T17:50:52.675 INFO:teuthology.orchestra.run.smithi022.stdout: socat x86_64 1.7.4.1-5.el9 CentOS-AppStream 300 k 2024-03-28T17:50:52.675 INFO:teuthology.orchestra.run.smithi022.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-03-28T17:50:52.675 INFO:teuthology.orchestra.run.smithi022.stdout: uuid x86_64 1.6.2-55.el9 CentOS-AppStream 56 k 2024-03-28T17:50:52.675 INFO:teuthology.orchestra.run.smithi022.stdout: xmlstarlet x86_64 1.6.1-20.el9 CentOS-AppStream 63 k 2024-03-28T17:50:52.676 INFO:teuthology.orchestra.run.smithi022.stdout: zlib-devel x86_64 1.2.11-40.el9 CentOS-AppStream 44 k 2024-03-28T17:50:52.676 INFO:teuthology.orchestra.run.smithi022.stdout:Installing weak dependencies: 2024-03-28T17:50:52.676 INFO:teuthology.orchestra.run.smithi022.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-03-28T17:50:52.676 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-03-28T17:50:52.676 INFO:teuthology.orchestra.run.smithi022.stdout:Transaction Summary 2024-03-28T17:50:52.676 INFO:teuthology.orchestra.run.smithi022.stdout:============================================================================================ 2024-03-28T17:50:52.676 INFO:teuthology.orchestra.run.smithi022.stdout:Install 150 Packages 2024-03-28T17:50:52.676 INFO:teuthology.orchestra.run.smithi022.stdout:Upgrade 24 Packages 2024-03-28T17:50:52.676 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-03-28T17:50:52.677 INFO:teuthology.orchestra.run.smithi022.stdout:Total download size: 225 M 2024-03-28T17:50:52.677 INFO:teuthology.orchestra.run.smithi022.stdout:Downloading Packages: 2024-03-28T17:50:52.700 INFO:teuthology.orchestra.run.smithi134.stdout:(1/174): ceph-19.0.0-2145.g3be9a5c9.el9.x86_64. 20 kB/s | 6.6 kB 00:00 2024-03-28T17:50:53.025 INFO:teuthology.orchestra.run.smithi134.stdout:(2/174): ceph-base-19.0.0-2145.g3be9a5c9.el9.x8 8.2 MB/s | 5.4 MB 00:00 2024-03-28T17:50:53.075 INFO:teuthology.orchestra.run.smithi134.stdout:(3/174): ceph-immutable-object-cache-19.0.0-214 2.9 MB/s | 145 kB 00:00 2024-03-28T17:50:53.250 INFO:teuthology.orchestra.run.smithi134.stdout:(4/174): ceph-mds-19.0.0-2145.g3be9a5c9.el9.x86 13 MB/s | 2.3 MB 00:00 2024-03-28T17:50:53.292 INFO:teuthology.orchestra.run.smithi134.stdout:(5/174): ceph-fuse-19.0.0-2145.g3be9a5c9.el9.x8 1.5 MB/s | 890 kB 00:00 2024-03-28T17:50:53.390 INFO:teuthology.orchestra.run.smithi134.stdout:(6/174): ceph-mgr-19.0.0-2145.g3be9a5c9.el9.x86 11 MB/s | 1.5 MB 00:00 2024-03-28T17:50:54.219 INFO:teuthology.orchestra.run.smithi022.stdout:(1/174): ceph-19.0.0-2145.g3be9a5c9.el9.x86_64. 17 kB/s | 6.6 kB 00:00 2024-03-28T17:50:54.368 INFO:teuthology.orchestra.run.smithi134.stdout:(7/174): ceph-osd-19.0.0-2145.g3be9a5c9.el9.x86 17 MB/s | 17 MB 00:00 2024-03-28T17:50:54.969 INFO:teuthology.orchestra.run.smithi022.stdout:(2/174): ceph-base-19.0.0-2145.g3be9a5c9.el9.x8 4.7 MB/s | 5.4 MB 00:01 2024-03-28T17:50:55.010 INFO:teuthology.orchestra.run.smithi022.stdout:(3/174): ceph-fuse-19.0.0-2145.g3be9a5c9.el9.x8 1.1 MB/s | 890 kB 00:00 2024-03-28T17:50:55.017 INFO:teuthology.orchestra.run.smithi079.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-03-28T17:50:55.019 INFO:teuthology.orchestra.run.smithi079.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-03-28T17:50:55.038 INFO:teuthology.orchestra.run.smithi079.stdout:Package NetworkManager-1:1.44.0-2.el9.x86_64 is already installed. 2024-03-28T17:50:55.039 INFO:teuthology.orchestra.run.smithi079.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-03-28T17:50:55.040 INFO:teuthology.orchestra.run.smithi079.stdout:Package util-linux-2.37.4-15.el9.x86_64 is already installed. 2024-03-28T17:50:55.041 INFO:teuthology.orchestra.run.smithi079.stdout:Package libacl-devel-2.3.1-3.el9.x86_64 is already installed. 2024-03-28T17:50:55.044 INFO:teuthology.orchestra.run.smithi022.stdout:(4/174): ceph-immutable-object-cache-19.0.0-214 1.9 MB/s | 145 kB 00:00 2024-03-28T17:50:55.044 INFO:teuthology.orchestra.run.smithi079.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-03-28T17:50:55.047 INFO:teuthology.orchestra.run.smithi079.stdout:Package xfsdump-3.1.12-3.el9.x86_64 is already installed. 2024-03-28T17:50:55.048 INFO:teuthology.orchestra.run.smithi079.stdout:Package xfsprogs-5.19.0-4.el9.x86_64 is already installed. 2024-03-28T17:50:55.133 INFO:teuthology.orchestra.run.smithi079.stdout:Dependencies resolved. 2024-03-28T17:50:55.178 INFO:teuthology.orchestra.run.smithi079.stdout:============================================================================================ 2024-03-28T17:50:55.178 INFO:teuthology.orchestra.run.smithi079.stdout: Package Arch Version Repository Size 2024-03-28T17:50:55.178 INFO:teuthology.orchestra.run.smithi079.stdout:============================================================================================ 2024-03-28T17:50:55.178 INFO:teuthology.orchestra.run.smithi079.stdout:Installing: 2024-03-28T17:50:55.179 INFO:teuthology.orchestra.run.smithi079.stdout: bison x86_64 3.7.4-5.el9 CentOS-AppStream 920 k 2024-03-28T17:50:55.179 INFO:teuthology.orchestra.run.smithi079.stdout: ceph x86_64 2:19.0.0-2145.g3be9a5c9.el9 ceph 6.6 k 2024-03-28T17:50:55.179 INFO:teuthology.orchestra.run.smithi079.stdout: ceph-base x86_64 2:19.0.0-2145.g3be9a5c9.el9 ceph 5.4 M 2024-03-28T17:50:55.179 INFO:teuthology.orchestra.run.smithi079.stdout: ceph-fuse x86_64 2:19.0.0-2145.g3be9a5c9.el9 ceph 890 k 2024-03-28T17:50:55.179 INFO:teuthology.orchestra.run.smithi079.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-2145.g3be9a5c9.el9 ceph 145 k 2024-03-28T17:50:55.179 INFO:teuthology.orchestra.run.smithi079.stdout: ceph-mgr x86_64 2:19.0.0-2145.g3be9a5c9.el9 ceph 1.5 M 2024-03-28T17:50:55.179 INFO:teuthology.orchestra.run.smithi079.stdout: ceph-mgr-cephadm noarch 2:19.0.0-2145.g3be9a5c9.el9 ceph-noarch 139 k 2024-03-28T17:50:55.179 INFO:teuthology.orchestra.run.smithi079.stdout: ceph-mgr-dashboard noarch 2:19.0.0-2145.g3be9a5c9.el9 ceph-noarch 1.8 M 2024-03-28T17:50:55.179 INFO:teuthology.orchestra.run.smithi079.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-2145.g3be9a5c9.el9 ceph-noarch 7.4 M 2024-03-28T17:50:55.179 INFO:teuthology.orchestra.run.smithi079.stdout: ceph-mgr-rook noarch 2:19.0.0-2145.g3be9a5c9.el9 ceph-noarch 49 k 2024-03-28T17:50:55.179 INFO:teuthology.orchestra.run.smithi079.stdout: ceph-radosgw x86_64 2:19.0.0-2145.g3be9a5c9.el9 ceph 9.9 M 2024-03-28T17:50:55.179 INFO:teuthology.orchestra.run.smithi079.stdout: ceph-test x86_64 2:19.0.0-2145.g3be9a5c9.el9 ceph 48 M 2024-03-28T17:50:55.179 INFO:teuthology.orchestra.run.smithi079.stdout: ceph-volume noarch 2:19.0.0-2145.g3be9a5c9.el9 ceph-noarch 264 k 2024-03-28T17:50:55.180 INFO:teuthology.orchestra.run.smithi079.stdout: cephadm noarch 2:19.0.0-2145.g3be9a5c9.el9 ceph-noarch 748 k 2024-03-28T17:50:55.180 INFO:teuthology.orchestra.run.smithi079.stdout: cephfs-mirror x86_64 2:19.0.0-2145.g3be9a5c9.el9 ceph 220 k 2024-03-28T17:50:55.180 INFO:teuthology.orchestra.run.smithi079.stdout: cephfs-shell x86_64 2:19.0.0-2145.g3be9a5c9.el9 ceph 27 k 2024-03-28T17:50:55.180 INFO:teuthology.orchestra.run.smithi079.stdout: cephfs-top noarch 2:19.0.0-2145.g3be9a5c9.el9 ceph-noarch 18 k 2024-03-28T17:50:55.180 INFO:teuthology.orchestra.run.smithi079.stdout: elfutils-libelf-devel x86_64 0.190-2.el9 appstream 21 k 2024-03-28T17:50:55.180 INFO:teuthology.orchestra.run.smithi079.stdout: flex x86_64 2.6.4-9.el9 CentOS-AppStream 307 k 2024-03-28T17:50:55.180 INFO:teuthology.orchestra.run.smithi079.stdout: libaio-devel x86_64 0.3.111-13.el9 CentOS-AppStream 10 k 2024-03-28T17:50:55.180 INFO:teuthology.orchestra.run.smithi079.stdout: libcephfs-devel x86_64 2:19.0.0-2145.g3be9a5c9.el9 ceph 31 k 2024-03-28T17:50:55.180 INFO:teuthology.orchestra.run.smithi079.stdout: libcephfs2 x86_64 2:19.0.0-2145.g3be9a5c9.el9 ceph 746 k 2024-03-28T17:50:55.180 INFO:teuthology.orchestra.run.smithi079.stdout: librados-devel x86_64 2:19.0.0-2145.g3be9a5c9.el9 ceph 127 k 2024-03-28T17:50:55.180 INFO:teuthology.orchestra.run.smithi079.stdout: libtool x86_64 2.4.6-45.el9 CentOS-AppStream 578 k 2024-03-28T17:50:55.180 INFO:teuthology.orchestra.run.smithi079.stdout: libuuid-devel x86_64 2.37.4-18.el9 appstream 25 k 2024-03-28T17:50:55.180 INFO:teuthology.orchestra.run.smithi079.stdout: openssl-devel x86_64 1:3.0.7-27.el9 appstream 4.1 M 2024-03-28T17:50:55.181 INFO:teuthology.orchestra.run.smithi079.stdout: postgresql x86_64 13.14-1.el9 appstream 1.6 M 2024-03-28T17:50:55.181 INFO:teuthology.orchestra.run.smithi079.stdout: postgresql-contrib x86_64 13.14-1.el9 appstream 911 k 2024-03-28T17:50:55.181 INFO:teuthology.orchestra.run.smithi079.stdout: postgresql-server x86_64 13.14-1.el9 appstream 5.8 M 2024-03-28T17:50:55.181 INFO:teuthology.orchestra.run.smithi079.stdout: python3-cephfs x86_64 2:19.0.0-2145.g3be9a5c9.el9 ceph 162 k 2024-03-28T17:50:55.181 INFO:teuthology.orchestra.run.smithi079.stdout: python3-rados x86_64 2:19.0.0-2145.g3be9a5c9.el9 ceph 321 k 2024-03-28T17:50:55.181 INFO:teuthology.orchestra.run.smithi079.stdout: python3-rbd x86_64 2:19.0.0-2145.g3be9a5c9.el9 ceph 298 k 2024-03-28T17:50:55.181 INFO:teuthology.orchestra.run.smithi079.stdout: python3-rgw x86_64 2:19.0.0-2145.g3be9a5c9.el9 ceph 99 k 2024-03-28T17:50:55.181 INFO:teuthology.orchestra.run.smithi079.stdout: rbd-fuse x86_64 2:19.0.0-2145.g3be9a5c9.el9 ceph 85 k 2024-03-28T17:50:55.181 INFO:teuthology.orchestra.run.smithi079.stdout: rbd-mirror x86_64 2:19.0.0-2145.g3be9a5c9.el9 ceph 3.1 M 2024-03-28T17:50:55.181 INFO:teuthology.orchestra.run.smithi079.stdout: rbd-nbd x86_64 2:19.0.0-2145.g3be9a5c9.el9 ceph 170 k 2024-03-28T17:50:55.181 INFO:teuthology.orchestra.run.smithi079.stdout: xfsprogs-devel x86_64 6.3.0-1.el9 appstream 94 k 2024-03-28T17:50:55.181 INFO:teuthology.orchestra.run.smithi079.stdout:Upgrading: 2024-03-28T17:50:55.181 INFO:teuthology.orchestra.run.smithi079.stdout: NetworkManager x86_64 1:1.47.2-1.el9 baseos 2.3 M 2024-03-28T17:50:55.181 INFO:teuthology.orchestra.run.smithi079.stdout: NetworkManager-libnm x86_64 1:1.47.2-1.el9 baseos 1.8 M 2024-03-28T17:50:55.182 INFO:teuthology.orchestra.run.smithi079.stdout: NetworkManager-team x86_64 1:1.47.2-1.el9 baseos 42 k 2024-03-28T17:50:55.182 INFO:teuthology.orchestra.run.smithi079.stdout: NetworkManager-tui x86_64 1:1.47.2-1.el9 baseos 248 k 2024-03-28T17:50:55.182 INFO:teuthology.orchestra.run.smithi079.stdout: acl x86_64 2.3.1-4.el9 baseos 71 k 2024-03-28T17:50:55.182 INFO:teuthology.orchestra.run.smithi079.stdout: elfutils-debuginfod-client x86_64 0.190-2.el9 baseos 36 k 2024-03-28T17:50:55.182 INFO:teuthology.orchestra.run.smithi079.stdout: elfutils-libelf x86_64 0.190-2.el9 baseos 192 k 2024-03-28T17:50:55.182 INFO:teuthology.orchestra.run.smithi079.stdout: elfutils-libs x86_64 0.190-2.el9 baseos 256 k 2024-03-28T17:50:55.182 INFO:teuthology.orchestra.run.smithi079.stdout: libacl x86_64 2.3.1-4.el9 baseos 23 k 2024-03-28T17:50:55.182 INFO:teuthology.orchestra.run.smithi079.stdout: libacl-devel x86_64 2.3.1-4.el9 appstream 74 k 2024-03-28T17:50:55.182 INFO:teuthology.orchestra.run.smithi079.stdout: libblkid x86_64 2.37.4-18.el9 baseos 108 k 2024-03-28T17:50:55.182 INFO:teuthology.orchestra.run.smithi079.stdout: libfdisk x86_64 2.37.4-18.el9 baseos 155 k 2024-03-28T17:50:55.182 INFO:teuthology.orchestra.run.smithi079.stdout: libmount x86_64 2.37.4-18.el9 baseos 136 k 2024-03-28T17:50:55.182 INFO:teuthology.orchestra.run.smithi079.stdout: librados2 x86_64 2:19.0.0-2145.g3be9a5c9.el9 ceph 3.4 M 2024-03-28T17:50:55.182 INFO:teuthology.orchestra.run.smithi079.stdout: librbd1 x86_64 2:19.0.0-2145.g3be9a5c9.el9 ceph 3.1 M 2024-03-28T17:50:55.183 INFO:teuthology.orchestra.run.smithi079.stdout: libsmartcols x86_64 2.37.4-18.el9 baseos 64 k 2024-03-28T17:50:55.183 INFO:teuthology.orchestra.run.smithi079.stdout: libuuid x86_64 2.37.4-18.el9 baseos 29 k 2024-03-28T17:50:55.183 INFO:teuthology.orchestra.run.smithi079.stdout: openssl x86_64 1:3.0.7-27.el9 baseos 1.2 M 2024-03-28T17:50:55.183 INFO:teuthology.orchestra.run.smithi079.stdout: openssl-libs x86_64 1:3.0.7-27.el9 baseos 2.2 M 2024-03-28T17:50:55.183 INFO:teuthology.orchestra.run.smithi079.stdout: util-linux x86_64 2.37.4-18.el9 baseos 2.3 M 2024-03-28T17:50:55.183 INFO:teuthology.orchestra.run.smithi079.stdout: util-linux-core x86_64 2.37.4-18.el9 baseos 465 k 2024-03-28T17:50:55.183 INFO:teuthology.orchestra.run.smithi079.stdout: util-linux-user x86_64 2.37.4-18.el9 baseos 31 k 2024-03-28T17:50:55.183 INFO:teuthology.orchestra.run.smithi079.stdout: xfsdump x86_64 3.1.12-4.el9 CentOS-BaseOS 321 k 2024-03-28T17:50:55.183 INFO:teuthology.orchestra.run.smithi079.stdout: xfsprogs x86_64 6.3.0-1.el9 baseos 1.1 M 2024-03-28T17:50:55.183 INFO:teuthology.orchestra.run.smithi079.stdout:Installing dependencies: 2024-03-28T17:50:55.183 INFO:teuthology.orchestra.run.smithi079.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-03-28T17:50:55.183 INFO:teuthology.orchestra.run.smithi079.stdout: automake noarch 1.16.2-8.el9 CentOS-AppStream 662 k 2024-03-28T17:50:55.183 INFO:teuthology.orchestra.run.smithi079.stdout: boost-program-options x86_64 1.75.0-8.el9 CentOS-AppStream 106 k 2024-03-28T17:50:55.183 INFO:teuthology.orchestra.run.smithi079.stdout: ceph-common x86_64 2:19.0.0-2145.g3be9a5c9.el9 ceph 20 M 2024-03-28T17:50:55.183 INFO:teuthology.orchestra.run.smithi079.stdout: ceph-grafana-dashboards noarch 2:19.0.0-2145.g3be9a5c9.el9 ceph-noarch 34 k 2024-03-28T17:50:55.184 INFO:teuthology.orchestra.run.smithi079.stdout: ceph-mds x86_64 2:19.0.0-2145.g3be9a5c9.el9 ceph 2.3 M 2024-03-28T17:50:55.184 INFO:teuthology.orchestra.run.smithi079.stdout: ceph-mgr-modules-core noarch 2:19.0.0-2145.g3be9a5c9.el9 ceph-noarch 248 k 2024-03-28T17:50:55.184 INFO:teuthology.orchestra.run.smithi079.stdout: ceph-mon x86_64 2:19.0.0-2145.g3be9a5c9.el9 ceph 4.7 M 2024-03-28T17:50:55.184 INFO:teuthology.orchestra.run.smithi079.stdout: ceph-osd x86_64 2:19.0.0-2145.g3be9a5c9.el9 ceph 17 M 2024-03-28T17:50:55.184 INFO:teuthology.orchestra.run.smithi079.stdout: ceph-prometheus-alerts noarch 2:19.0.0-2145.g3be9a5c9.el9 ceph-noarch 16 k 2024-03-28T17:50:55.184 INFO:teuthology.orchestra.run.smithi079.stdout: ceph-selinux x86_64 2:19.0.0-2145.g3be9a5c9.el9 ceph 25 k 2024-03-28T17:50:55.184 INFO:teuthology.orchestra.run.smithi079.stdout: flexiblas x86_64 3.0.4-8.el9 CentOS-AppStream 32 k 2024-03-28T17:50:55.184 INFO:teuthology.orchestra.run.smithi079.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 CentOS-AppStream 3.0 M 2024-03-28T17:50:55.184 INFO:teuthology.orchestra.run.smithi079.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 CentOS-AppStream 17 k 2024-03-28T17:50:55.184 INFO:teuthology.orchestra.run.smithi079.stdout: fuse x86_64 2.9.9-15.el9 CentOS-BaseOS 79 k 2024-03-28T17:50:55.184 INFO:teuthology.orchestra.run.smithi079.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-03-28T17:50:55.184 INFO:teuthology.orchestra.run.smithi079.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-03-28T17:50:55.184 INFO:teuthology.orchestra.run.smithi079.stdout: jq x86_64 1.6-16.el9 baseos 187 k 2024-03-28T17:50:55.185 INFO:teuthology.orchestra.run.smithi079.stdout: libarrow x86_64 9.0.0-10.el9 epel 4.4 M 2024-03-28T17:50:55.185 INFO:teuthology.orchestra.run.smithi079.stdout: libarrow-doc noarch 9.0.0-10.el9 epel 26 k 2024-03-28T17:50:55.185 INFO:teuthology.orchestra.run.smithi079.stdout: libcephsqlite x86_64 2:19.0.0-2145.g3be9a5c9.el9 ceph 163 k 2024-03-28T17:50:55.185 INFO:teuthology.orchestra.run.smithi079.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-03-28T17:50:55.185 INFO:teuthology.orchestra.run.smithi079.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-03-28T17:50:55.185 INFO:teuthology.orchestra.run.smithi079.stdout: libpmemobj x86_64 1.12.1-1.el9 CentOS-AppStream 159 k 2024-03-28T17:50:55.185 INFO:teuthology.orchestra.run.smithi079.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-03-28T17:50:55.185 INFO:teuthology.orchestra.run.smithi079.stdout: librabbitmq x86_64 0.11.0-7.el9 CentOS-AppStream 44 k 2024-03-28T17:50:55.185 INFO:teuthology.orchestra.run.smithi079.stdout: libradosstriper1 x86_64 2:19.0.0-2145.g3be9a5c9.el9 ceph 501 k 2024-03-28T17:50:55.185 INFO:teuthology.orchestra.run.smithi079.stdout: librdkafka x86_64 1.6.1-102.el9 CentOS-AppStream 661 k 2024-03-28T17:50:55.185 INFO:teuthology.orchestra.run.smithi079.stdout: librgw2 x86_64 2:19.0.0-2145.g3be9a5c9.el9 ceph 5.1 M 2024-03-28T17:50:55.185 INFO:teuthology.orchestra.run.smithi079.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-03-28T17:50:55.185 INFO:teuthology.orchestra.run.smithi079.stdout: libzstd-devel x86_64 1.5.1-2.el9 CentOS-AppStream 46 k 2024-03-28T17:50:55.185 INFO:teuthology.orchestra.run.smithi079.stdout: lttng-ust x86_64 2.12.0-6.el9 CentOS-AppStream 282 k 2024-03-28T17:50:55.186 INFO:teuthology.orchestra.run.smithi079.stdout: lua x86_64 5.4.4-4.el9 CentOS-AppStream 187 k 2024-03-28T17:50:55.186 INFO:teuthology.orchestra.run.smithi079.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-03-28T17:50:55.186 INFO:teuthology.orchestra.run.smithi079.stdout: luarocks noarch 3.9.2-1.el9 epel 151 k 2024-03-28T17:50:55.186 INFO:teuthology.orchestra.run.smithi079.stdout: oniguruma x86_64 6.9.6-1.el9.5 CentOS-AppStream 217 k 2024-03-28T17:50:55.186 INFO:teuthology.orchestra.run.smithi079.stdout: openblas x86_64 0.3.21-2.el9 CentOS-AppStream 34 k 2024-03-28T17:50:55.186 INFO:teuthology.orchestra.run.smithi079.stdout: openblas-openmp x86_64 0.3.21-2.el9 CentOS-AppStream 4.7 M 2024-03-28T17:50:55.186 INFO:teuthology.orchestra.run.smithi079.stdout: parquet-libs x86_64 9.0.0-10.el9 epel 838 k 2024-03-28T17:50:55.186 INFO:teuthology.orchestra.run.smithi079.stdout: postgresql-private-libs x86_64 13.14-1.el9 appstream 133 k 2024-03-28T17:50:55.186 INFO:teuthology.orchestra.run.smithi079.stdout: protobuf x86_64 3.14.0-13.el9 CentOS-AppStream 1.0 M 2024-03-28T17:50:55.186 INFO:teuthology.orchestra.run.smithi079.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-03-28T17:50:55.186 INFO:teuthology.orchestra.run.smithi079.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-03-28T17:50:55.186 INFO:teuthology.orchestra.run.smithi079.stdout: python3-attrs noarch 20.3.0-7.el9 CentOS-AppStream 78 k 2024-03-28T17:50:55.187 INFO:teuthology.orchestra.run.smithi079.stdout: python3-babel noarch 2.9.1-2.el9 CentOS-AppStream 5.8 M 2024-03-28T17:50:55.187 INFO:teuthology.orchestra.run.smithi079.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-03-28T17:50:55.187 INFO:teuthology.orchestra.run.smithi079.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-03-28T17:50:55.187 INFO:teuthology.orchestra.run.smithi079.stdout: python3-ceph-argparse x86_64 2:19.0.0-2145.g3be9a5c9.el9 ceph 45 k 2024-03-28T17:50:55.187 INFO:teuthology.orchestra.run.smithi079.stdout: python3-ceph-common x86_64 2:19.0.0-2145.g3be9a5c9.el9 ceph 131 k 2024-03-28T17:50:55.187 INFO:teuthology.orchestra.run.smithi079.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-03-28T17:50:55.187 INFO:teuthology.orchestra.run.smithi079.stdout: python3-cffi x86_64 1.14.5-5.el9 CentOS-BaseOS 241 k 2024-03-28T17:50:55.187 INFO:teuthology.orchestra.run.smithi079.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-03-28T17:50:55.187 INFO:teuthology.orchestra.run.smithi079.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-03-28T17:50:55.187 INFO:teuthology.orchestra.run.smithi079.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-03-28T17:50:55.187 INFO:teuthology.orchestra.run.smithi079.stdout: python3-colorama noarch 0.4.4-12.el9 epel 35 k 2024-03-28T17:50:55.187 INFO:teuthology.orchestra.run.smithi079.stdout: python3-cryptography x86_64 36.0.1-4.el9 CentOS-BaseOS 1.1 M 2024-03-28T17:50:55.187 INFO:teuthology.orchestra.run.smithi079.stdout: python3-devel x86_64 3.9.17-2.el9 appstream 245 k 2024-03-28T17:50:55.187 INFO:teuthology.orchestra.run.smithi079.stdout: python3-google-auth noarch 1:2.28.2-1.el9 epel 203 k 2024-03-28T17:50:55.188 INFO:teuthology.orchestra.run.smithi079.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-03-28T17:50:55.188 INFO:teuthology.orchestra.run.smithi079.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-03-28T17:50:55.188 INFO:teuthology.orchestra.run.smithi079.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-03-28T17:50:55.188 INFO:teuthology.orchestra.run.smithi079.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-03-28T17:50:55.188 INFO:teuthology.orchestra.run.smithi079.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-03-28T17:50:55.188 INFO:teuthology.orchestra.run.smithi079.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-03-28T17:50:55.188 INFO:teuthology.orchestra.run.smithi079.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-03-28T17:50:55.188 INFO:teuthology.orchestra.run.smithi079.stdout: python3-jinja2 noarch 2.11.3-5.el9 appstream 248 k 2024-03-28T17:50:55.188 INFO:teuthology.orchestra.run.smithi079.stdout: python3-jsonpatch noarch 1.21-16.el9 CentOS-AppStream 24 k 2024-03-28T17:50:55.188 INFO:teuthology.orchestra.run.smithi079.stdout: python3-jsonpointer noarch 2.0-4.el9 CentOS-AppStream 18 k 2024-03-28T17:50:55.188 INFO:teuthology.orchestra.run.smithi079.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-03-28T17:50:55.188 INFO:teuthology.orchestra.run.smithi079.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-03-28T17:50:55.188 INFO:teuthology.orchestra.run.smithi079.stdout: python3-mako noarch 1.1.4-6.el9 CentOS-AppStream 144 k 2024-03-28T17:50:55.188 INFO:teuthology.orchestra.run.smithi079.stdout: python3-markupsafe x86_64 1.1.1-12.el9 CentOS-AppStream 32 k 2024-03-28T17:50:55.189 INFO:teuthology.orchestra.run.smithi079.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-03-28T17:50:55.189 INFO:teuthology.orchestra.run.smithi079.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-03-28T17:50:55.189 INFO:teuthology.orchestra.run.smithi079.stdout: python3-numpy x86_64 1:1.20.1-5.el9 CentOS-AppStream 4.8 M 2024-03-28T17:50:55.189 INFO:teuthology.orchestra.run.smithi079.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 CentOS-AppStream 217 k 2024-03-28T17:50:55.189 INFO:teuthology.orchestra.run.smithi079.stdout: python3-oauthlib noarch 3.1.1-5.el9 CentOS-AppStream 174 k 2024-03-28T17:50:55.189 INFO:teuthology.orchestra.run.smithi079.stdout: python3-packaging noarch 20.9-5.el9 CentOS-AppStream 69 k 2024-03-28T17:50:55.189 INFO:teuthology.orchestra.run.smithi079.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-03-28T17:50:55.189 INFO:teuthology.orchestra.run.smithi079.stdout: python3-ply noarch 3.11-14.el9 CentOS-BaseOS 103 k 2024-03-28T17:50:55.189 INFO:teuthology.orchestra.run.smithi079.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-03-28T17:50:55.189 INFO:teuthology.orchestra.run.smithi079.stdout: python3-prettytable noarch 0.7.2-27.el9 CentOS-AppStream 41 k 2024-03-28T17:50:55.189 INFO:teuthology.orchestra.run.smithi079.stdout: python3-protobuf noarch 3.14.0-13.el9 CentOS-AppStream 238 k 2024-03-28T17:50:55.189 INFO:teuthology.orchestra.run.smithi079.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-03-28T17:50:55.189 INFO:teuthology.orchestra.run.smithi079.stdout: python3-pyasn1 noarch 0.4.8-6.el9 CentOS-AppStream 132 k 2024-03-28T17:50:55.189 INFO:teuthology.orchestra.run.smithi079.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 CentOS-AppStream 211 k 2024-03-28T17:50:55.190 INFO:teuthology.orchestra.run.smithi079.stdout: python3-pycparser noarch 2.20-6.el9 CentOS-BaseOS 124 k 2024-03-28T17:50:55.190 INFO:teuthology.orchestra.run.smithi079.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-03-28T17:50:55.190 INFO:teuthology.orchestra.run.smithi079.stdout: python3-pytz noarch 2021.1-5.el9 CentOS-AppStream 47 k 2024-03-28T17:50:55.190 INFO:teuthology.orchestra.run.smithi079.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-03-28T17:50:55.190 INFO:teuthology.orchestra.run.smithi079.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-03-28T17:50:55.190 INFO:teuthology.orchestra.run.smithi079.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 CentOS-AppStream 43 k 2024-03-28T17:50:55.190 INFO:teuthology.orchestra.run.smithi079.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-03-28T17:50:55.190 INFO:teuthology.orchestra.run.smithi079.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-03-28T17:50:55.190 INFO:teuthology.orchestra.run.smithi079.stdout: python3-scipy x86_64 1.6.2-8.el9 CentOS-AppStream 16 M 2024-03-28T17:50:55.190 INFO:teuthology.orchestra.run.smithi079.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-03-28T17:50:55.190 INFO:teuthology.orchestra.run.smithi079.stdout: python3-toml noarch 0.10.2-6.el9 CentOS-AppStream 39 k 2024-03-28T17:50:55.190 INFO:teuthology.orchestra.run.smithi079.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-03-28T17:50:55.190 INFO:teuthology.orchestra.run.smithi079.stdout: python3-urllib3 noarch 1.26.5-5.el9 CentOS-BaseOS 187 k 2024-03-28T17:50:55.190 INFO:teuthology.orchestra.run.smithi079.stdout: python3-wcwidth noarch 0.2.5-8.el9 CentOS-AppStream 41 k 2024-03-28T17:50:55.191 INFO:teuthology.orchestra.run.smithi079.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-03-28T17:50:55.191 INFO:teuthology.orchestra.run.smithi079.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-03-28T17:50:55.191 INFO:teuthology.orchestra.run.smithi079.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-03-28T17:50:55.191 INFO:teuthology.orchestra.run.smithi079.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-03-28T17:50:55.191 INFO:teuthology.orchestra.run.smithi079.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-03-28T17:50:55.191 INFO:teuthology.orchestra.run.smithi079.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-03-28T17:50:55.191 INFO:teuthology.orchestra.run.smithi079.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-03-28T17:50:55.191 INFO:teuthology.orchestra.run.smithi079.stdout: socat x86_64 1.7.4.1-5.el9 CentOS-AppStream 300 k 2024-03-28T17:50:55.191 INFO:teuthology.orchestra.run.smithi079.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-03-28T17:50:55.191 INFO:teuthology.orchestra.run.smithi079.stdout: uuid x86_64 1.6.2-55.el9 CentOS-AppStream 56 k 2024-03-28T17:50:55.191 INFO:teuthology.orchestra.run.smithi079.stdout: xmlstarlet x86_64 1.6.1-20.el9 CentOS-AppStream 63 k 2024-03-28T17:50:55.191 INFO:teuthology.orchestra.run.smithi079.stdout: zlib-devel x86_64 1.2.11-40.el9 CentOS-AppStream 44 k 2024-03-28T17:50:55.191 INFO:teuthology.orchestra.run.smithi079.stdout:Installing weak dependencies: 2024-03-28T17:50:55.191 INFO:teuthology.orchestra.run.smithi079.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-03-28T17:50:55.192 INFO:teuthology.orchestra.run.smithi079.stdout: 2024-03-28T17:50:55.192 INFO:teuthology.orchestra.run.smithi079.stdout:Transaction Summary 2024-03-28T17:50:55.192 INFO:teuthology.orchestra.run.smithi079.stdout:============================================================================================ 2024-03-28T17:50:55.192 INFO:teuthology.orchestra.run.smithi079.stdout:Install 150 Packages 2024-03-28T17:50:55.192 INFO:teuthology.orchestra.run.smithi079.stdout:Upgrade 24 Packages 2024-03-28T17:50:55.192 INFO:teuthology.orchestra.run.smithi079.stdout: 2024-03-28T17:50:55.192 INFO:teuthology.orchestra.run.smithi079.stdout:Total download size: 225 M 2024-03-28T17:50:55.192 INFO:teuthology.orchestra.run.smithi079.stdout:Downloading Packages: 2024-03-28T17:50:55.209 INFO:teuthology.orchestra.run.smithi134.stdout:(8/174): ceph-mon-19.0.0-2145.g3be9a5c9.el9.x86 2.4 MB/s | 4.7 MB 00:01 2024-03-28T17:50:55.286 INFO:teuthology.orchestra.run.smithi022.stdout:(5/174): ceph-mgr-19.0.0-2145.g3be9a5c9.el9.x86 6.2 MB/s | 1.5 MB 00:00 2024-03-28T17:50:55.318 INFO:teuthology.orchestra.run.smithi134.stdout:(9/174): ceph-selinux-19.0.0-2145.g3be9a5c9.el9 233 kB/s | 25 kB 00:00 2024-03-28T17:50:55.510 INFO:teuthology.orchestra.run.smithi134.stdout:(10/174): ceph-radosgw-19.0.0-2145.g3be9a5c9.el 8.6 MB/s | 9.9 MB 00:01 2024-03-28T17:50:55.594 INFO:teuthology.orchestra.run.smithi134.stdout:(11/174): cephfs-mirror-19.0.0-2145.g3be9a5c9.e 2.6 MB/s | 220 kB 00:00 2024-03-28T17:50:55.669 INFO:teuthology.orchestra.run.smithi134.stdout:(12/174): cephfs-shell-19.0.0-2145.g3be9a5c9.el 365 kB/s | 27 kB 00:00 2024-03-28T17:50:55.736 INFO:teuthology.orchestra.run.smithi134.stdout:(13/174): libcephfs-devel-19.0.0-2145.g3be9a5c9 475 kB/s | 31 kB 00:00 2024-03-28T17:50:55.819 INFO:teuthology.orchestra.run.smithi022.stdout:(6/174): ceph-mon-19.0.0-2145.g3be9a5c9.el9.x86 8.8 MB/s | 4.7 MB 00:00 2024-03-28T17:50:55.836 INFO:teuthology.orchestra.run.smithi134.stdout:(14/174): libcephfs2-19.0.0-2145.g3be9a5c9.el9. 7.4 MB/s | 746 kB 00:00 2024-03-28T17:50:56.087 INFO:teuthology.orchestra.run.smithi134.stdout:(15/174): ceph-common-19.0.0-2145.g3be9a5c9.el9 5.5 MB/s | 20 MB 00:03 2024-03-28T17:50:56.120 INFO:teuthology.orchestra.run.smithi022.stdout:(7/174): ceph-mds-19.0.0-2145.g3be9a5c9.el9.x86 2.1 MB/s | 2.3 MB 00:01 2024-03-28T17:50:56.120 INFO:teuthology.orchestra.run.smithi134.stdout:(16/174): libcephsqlite-19.0.0-2145.g3be9a5c9.e 575 kB/s | 163 kB 00:00 2024-03-28T17:50:56.157 INFO:teuthology.orchestra.run.smithi079.stdout:(1/174): ceph-19.0.0-2145.g3be9a5c9.el9.x86_64. 20 kB/s | 6.6 kB 00:00 2024-03-28T17:50:56.171 INFO:teuthology.orchestra.run.smithi134.stdout:(17/174): librados-devel-19.0.0-2145.g3be9a5c9. 1.5 MB/s | 127 kB 00:00 2024-03-28T17:50:56.229 INFO:teuthology.orchestra.run.smithi134.stdout:(18/174): libradosstriper1-19.0.0-2145.g3be9a5c 4.5 MB/s | 501 kB 00:00 2024-03-28T17:50:56.288 INFO:teuthology.orchestra.run.smithi134.stdout:(19/174): python3-ceph-argparse-19.0.0-2145.g3b 786 kB/s | 45 kB 00:00 2024-03-28T17:50:56.363 INFO:teuthology.orchestra.run.smithi134.stdout:(20/174): python3-ceph-common-19.0.0-2145.g3be9 1.7 MB/s | 131 kB 00:00 2024-03-28T17:50:56.439 INFO:teuthology.orchestra.run.smithi134.stdout:(21/174): python3-cephfs-19.0.0-2145.g3be9a5c9. 2.1 MB/s | 162 kB 00:00 2024-03-28T17:50:56.531 INFO:teuthology.orchestra.run.smithi134.stdout:(22/174): python3-rados-19.0.0-2145.g3be9a5c9.e 3.5 MB/s | 321 kB 00:00 2024-03-28T17:50:56.622 INFO:teuthology.orchestra.run.smithi134.stdout:(23/174): python3-rbd-19.0.0-2145.g3be9a5c9.el9 3.2 MB/s | 298 kB 00:00 2024-03-28T17:50:56.706 INFO:teuthology.orchestra.run.smithi134.stdout:(24/174): python3-rgw-19.0.0-2145.g3be9a5c9.el9 1.2 MB/s | 99 kB 00:00 2024-03-28T17:50:56.790 INFO:teuthology.orchestra.run.smithi134.stdout:(25/174): rbd-fuse-19.0.0-2145.g3be9a5c9.el9.x8 1.0 MB/s | 85 kB 00:00 2024-03-28T17:50:56.973 INFO:teuthology.orchestra.run.smithi134.stdout:(26/174): librgw2-19.0.0-2145.g3be9a5c9.el9.x86 6.4 MB/s | 5.1 MB 00:00 2024-03-28T17:50:57.082 INFO:teuthology.orchestra.run.smithi134.stdout:(27/174): rbd-nbd-19.0.0-2145.g3be9a5c9.el9.x86 1.5 MB/s | 170 kB 00:00 2024-03-28T17:50:57.107 INFO:teuthology.orchestra.run.smithi079.stdout:(2/174): ceph-fuse-19.0.0-2145.g3be9a5c9.el9.x8 938 kB/s | 890 kB 00:00 2024-03-28T17:50:57.224 INFO:teuthology.orchestra.run.smithi134.stdout:(28/174): rbd-mirror-19.0.0-2145.g3be9a5c9.el9. 7.1 MB/s | 3.1 MB 00:00 2024-03-28T17:50:57.249 INFO:teuthology.orchestra.run.smithi134.stdout:(29/174): ceph-grafana-dashboards-19.0.0-2145.g 204 kB/s | 34 kB 00:00 2024-03-28T17:50:57.266 INFO:teuthology.orchestra.run.smithi079.stdout:(3/174): ceph-immutable-object-cache-19.0.0-214 921 kB/s | 145 kB 00:00 2024-03-28T17:50:57.291 INFO:teuthology.orchestra.run.smithi134.stdout:(30/174): ceph-mgr-cephadm-19.0.0-2145.g3be9a5c 2.1 MB/s | 139 kB 00:00 2024-03-28T17:50:57.582 INFO:teuthology.orchestra.run.smithi134.stdout:(31/174): ceph-mgr-dashboard-19.0.0-2145.g3be9a 5.4 MB/s | 1.8 MB 00:00 2024-03-28T17:50:57.692 INFO:teuthology.orchestra.run.smithi134.stdout:(32/174): ceph-mgr-modules-core-19.0.0-2145.g3b 2.2 MB/s | 248 kB 00:00 2024-03-28T17:50:57.775 INFO:teuthology.orchestra.run.smithi134.stdout:(33/174): ceph-mgr-rook-19.0.0-2145.g3be9a5c9.e 596 kB/s | 49 kB 00:00 2024-03-28T17:50:57.842 INFO:teuthology.orchestra.run.smithi134.stdout:(34/174): ceph-prometheus-alerts-19.0.0-2145.g3 250 kB/s | 16 kB 00:00 2024-03-28T17:50:57.911 INFO:teuthology.orchestra.run.smithi022.stdout:(8/174): ceph-osd-19.0.0-2145.g3be9a5c9.el9.x86 8.1 MB/s | 17 MB 00:02 2024-03-28T17:50:57.934 INFO:teuthology.orchestra.run.smithi134.stdout:(35/174): ceph-volume-19.0.0-2145.g3be9a5c9.el9 2.8 MB/s | 264 kB 00:00 2024-03-28T17:50:57.978 INFO:teuthology.orchestra.run.smithi022.stdout:(9/174): ceph-selinux-19.0.0-2145.g3be9a5c9.el9 384 kB/s | 25 kB 00:00 2024-03-28T17:50:58.118 INFO:teuthology.orchestra.run.smithi134.stdout:(36/174): cephadm-19.0.0-2145.g3be9a5c9.el9.noa 4.0 MB/s | 748 kB 00:00 2024-03-28T17:50:58.310 INFO:teuthology.orchestra.run.smithi134.stdout:(37/174): ceph-mgr-diskprediction-local-19.0.0- 7.3 MB/s | 7.4 MB 00:01 2024-03-28T17:50:58.335 INFO:teuthology.orchestra.run.smithi134.stdout:(38/174): cephfs-top-19.0.0-2145.g3be9a5c9.el9. 84 kB/s | 18 kB 00:00 2024-03-28T17:50:58.499 INFO:teuthology.orchestra.run.smithi079.stdout:(4/174): ceph-base-19.0.0-2145.g3be9a5c9.el9.x8 2.0 MB/s | 5.4 MB 00:02 2024-03-28T17:50:58.552 INFO:teuthology.orchestra.run.smithi134.stdout:(39/174): automake-1.16.2-8.el9.noarch.rpm 2.7 MB/s | 662 kB 00:00 2024-03-28T17:50:58.586 INFO:teuthology.orchestra.run.smithi134.stdout:(40/174): bison-3.7.4-5.el9.x86_64.rpm 3.6 MB/s | 920 kB 00:00 2024-03-28T17:50:58.611 INFO:teuthology.orchestra.run.smithi134.stdout:(41/174): boost-program-options-1.75.0-8.el9.x8 1.9 MB/s | 106 kB 00:00 2024-03-28T17:50:58.645 INFO:teuthology.orchestra.run.smithi134.stdout:(42/174): flex-2.6.4-9.el9.x86_64.rpm 5.2 MB/s | 307 kB 00:00 2024-03-28T17:50:58.670 INFO:teuthology.orchestra.run.smithi134.stdout:(43/174): flexiblas-3.0.4-8.el9.x86_64.rpm 569 kB/s | 32 kB 00:00 2024-03-28T17:50:58.779 INFO:teuthology.orchestra.run.smithi134.stdout:(44/174): flexiblas-netlib-3.0.4-8.el9.x86_64.r 23 MB/s | 3.0 MB 00:00 2024-03-28T17:50:58.825 INFO:teuthology.orchestra.run.smithi079.stdout:(5/174): ceph-mgr-19.0.0-2145.g3be9a5c9.el9.x86 4.6 MB/s | 1.5 MB 00:00 2024-03-28T17:50:58.834 INFO:teuthology.orchestra.run.smithi134.stdout:(45/174): flexiblas-openblas-openmp-3.0.4-8.el9 115 kB/s | 17 kB 00:00 2024-03-28T17:50:58.879 INFO:teuthology.orchestra.run.smithi134.stdout:(46/174): libaio-devel-0.3.111-13.el9.x86_64.rp 232 kB/s | 10 kB 00:00 2024-03-28T17:50:58.913 INFO:teuthology.orchestra.run.smithi134.stdout:(47/174): libpmemobj-1.12.1-1.el9.x86_64.rpm 2.0 MB/s | 159 kB 00:00 2024-03-28T17:50:58.933 INFO:teuthology.orchestra.run.smithi079.stdout:(6/174): ceph-mds-19.0.0-2145.g3be9a5c9.el9.x86 1.4 MB/s | 2.3 MB 00:01 2024-03-28T17:50:58.938 INFO:teuthology.orchestra.run.smithi134.stdout:(48/174): librabbitmq-0.11.0-7.el9.x86_64.rpm 787 kB/s | 44 kB 00:00 2024-03-28T17:50:59.005 INFO:teuthology.orchestra.run.smithi134.stdout:(49/174): librdkafka-1.6.1-102.el9.x86_64.rpm 7.1 MB/s | 661 kB 00:00 2024-03-28T17:50:59.055 INFO:teuthology.orchestra.run.smithi134.stdout:(50/174): libtool-2.4.6-45.el9.x86_64.rpm 5.1 MB/s | 578 kB 00:00 2024-03-28T17:50:59.081 INFO:teuthology.orchestra.run.smithi134.stdout:(51/174): libzstd-devel-1.5.1-2.el9.x86_64.rpm 663 kB/s | 46 kB 00:00 2024-03-28T17:50:59.114 INFO:teuthology.orchestra.run.smithi134.stdout:(52/174): lttng-ust-2.12.0-6.el9.x86_64.rpm 4.8 MB/s | 282 kB 00:00 2024-03-28T17:50:59.148 INFO:teuthology.orchestra.run.smithi134.stdout:(53/174): lua-5.4.4-4.el9.x86_64.rpm 2.9 MB/s | 187 kB 00:00 2024-03-28T17:50:59.182 INFO:teuthology.orchestra.run.smithi134.stdout:(54/174): oniguruma-6.9.6-1.el9.5.x86_64.rpm 3.3 MB/s | 217 kB 00:00 2024-03-28T17:50:59.207 INFO:teuthology.orchestra.run.smithi134.stdout:(55/174): openblas-0.3.21-2.el9.x86_64.rpm 610 kB/s | 34 kB 00:00 2024-03-28T17:50:59.341 INFO:teuthology.orchestra.run.smithi134.stdout:(56/174): openblas-openmp-0.3.21-2.el9.x86_64.r 30 MB/s | 4.7 MB 00:00 2024-03-28T17:50:59.382 INFO:teuthology.orchestra.run.smithi134.stdout:(57/174): protobuf-3.14.0-13.el9.x86_64.rpm 6.3 MB/s | 1.0 MB 00:00 2024-03-28T17:50:59.424 INFO:teuthology.orchestra.run.smithi134.stdout:(58/174): python3-attrs-20.3.0-7.el9.noarch.rpm 1.9 MB/s | 78 kB 00:00 2024-03-28T17:50:59.558 INFO:teuthology.orchestra.run.smithi134.stdout:(59/174): python3-babel-2.9.1-2.el9.noarch.rpm 33 MB/s | 5.8 MB 00:00 2024-03-28T17:50:59.575 INFO:teuthology.orchestra.run.smithi079.stdout:(7/174): ceph-mon-19.0.0-2145.g3be9a5c9.el9.x86 6.3 MB/s | 4.7 MB 00:00 2024-03-28T17:50:59.583 INFO:teuthology.orchestra.run.smithi134.stdout:(60/174): python3-jsonpatch-1.21-16.el9.noarch. 170 kB/s | 24 kB 00:00 2024-03-28T17:50:59.617 INFO:teuthology.orchestra.run.smithi134.stdout:(61/174): python3-jsonpointer-2.0-4.el9.noarch. 562 kB/s | 18 kB 00:00 2024-03-28T17:50:59.674 INFO:teuthology.orchestra.run.smithi134.stdout:(62/174): python3-mako-1.1.4-6.el9.noarch.rpm 1.6 MB/s | 144 kB 00:00 2024-03-28T17:50:59.709 INFO:teuthology.orchestra.run.smithi134.stdout:(63/174): python3-markupsafe-1.1.1-12.el9.x86_6 351 kB/s | 32 kB 00:00 2024-03-28T17:50:59.835 INFO:teuthology.orchestra.run.smithi134.stdout:(64/174): python3-numpy-1.20.1-5.el9.x86_64.rpm 30 MB/s | 4.8 MB 00:00 2024-03-28T17:50:59.860 INFO:teuthology.orchestra.run.smithi134.stdout:(65/174): python3-numpy-f2py-1.20.1-5.el9.x86_6 1.6 MB/s | 217 kB 00:00 2024-03-28T17:50:59.893 INFO:teuthology.orchestra.run.smithi134.stdout:(66/174): python3-oauthlib-3.1.1-5.el9.noarch.r 5.3 MB/s | 174 kB 00:00 2024-03-28T17:50:59.918 INFO:teuthology.orchestra.run.smithi134.stdout:(67/174): python3-packaging-20.9-5.el9.noarch.r 1.2 MB/s | 69 kB 00:00 2024-03-28T17:50:59.944 INFO:teuthology.orchestra.run.smithi134.stdout:(68/174): python3-prettytable-0.7.2-27.el9.noar 1.7 MB/s | 41 kB 00:00 2024-03-28T17:50:59.977 INFO:teuthology.orchestra.run.smithi134.stdout:(69/174): python3-protobuf-3.14.0-13.el9.noarch 4.0 MB/s | 238 kB 00:00 2024-03-28T17:51:00.003 INFO:teuthology.orchestra.run.smithi134.stdout:(70/174): python3-pyasn1-0.4.8-6.el9.noarch.rpm 2.3 MB/s | 132 kB 00:00 2024-03-28T17:51:00.036 INFO:teuthology.orchestra.run.smithi134.stdout:(71/174): python3-pyasn1-modules-0.4.8-6.el9.no 3.6 MB/s | 211 kB 00:00 2024-03-28T17:51:00.085 INFO:teuthology.orchestra.run.smithi134.stdout:(72/174): python3-pytz-2021.1-5.el9.noarch.rpm 595 kB/s | 47 kB 00:00 2024-03-28T17:51:00.112 INFO:teuthology.orchestra.run.smithi134.stdout:(73/174): python3-requests-oauthlib-1.3.0-12.el 576 kB/s | 43 kB 00:00 2024-03-28T17:51:00.446 INFO:teuthology.orchestra.run.smithi134.stdout:(74/174): python3-scipy-1.6.2-8.el9.x86_64.rpm 44 MB/s | 16 MB 00:00 2024-03-28T17:51:00.504 INFO:teuthology.orchestra.run.smithi134.stdout:(75/174): python3-toml-0.10.2-6.el9.noarch.rpm 102 kB/s | 39 kB 00:00 2024-03-28T17:51:00.512 INFO:teuthology.orchestra.run.smithi022.stdout:(10/174): ceph-radosgw-19.0.0-2145.g3be9a5c9.el 2.2 MB/s | 9.9 MB 00:04 2024-03-28T17:51:00.538 INFO:teuthology.orchestra.run.smithi134.stdout:(76/174): python3-wcwidth-0.2.5-8.el9.noarch.rp 451 kB/s | 41 kB 00:00 2024-03-28T17:51:00.572 INFO:teuthology.orchestra.run.smithi134.stdout:(77/174): socat-1.7.4.1-5.el9.x86_64.rpm 4.5 MB/s | 300 kB 00:00 2024-03-28T17:51:00.597 INFO:teuthology.orchestra.run.smithi134.stdout:(78/174): uuid-1.6.2-55.el9.x86_64.rpm 1.0 MB/s | 56 kB 00:00 2024-03-28T17:51:00.612 INFO:teuthology.orchestra.run.smithi022.stdout:(11/174): cephfs-mirror-19.0.0-2145.g3be9a5c9.e 2.2 MB/s | 220 kB 00:00 2024-03-28T17:51:00.622 INFO:teuthology.orchestra.run.smithi134.stdout:(79/174): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.2 MB/s | 63 kB 00:00 2024-03-28T17:51:00.647 INFO:teuthology.orchestra.run.smithi134.stdout:(80/174): zlib-devel-1.2.11-40.el9.x86_64.rpm 901 kB/s | 44 kB 00:00 2024-03-28T17:51:00.673 INFO:teuthology.orchestra.run.smithi134.stdout:(81/174): fuse-2.9.9-15.el9.x86_64.rpm 1.6 MB/s | 79 kB 00:00 2024-03-28T17:51:00.675 INFO:teuthology.orchestra.run.smithi079.stdout:(8/174): ceph-radosgw-19.0.0-2145.g3be9a5c9.el9 9.0 MB/s | 9.9 MB 00:01 2024-03-28T17:51:00.706 INFO:teuthology.orchestra.run.smithi134.stdout:(82/174): python3-cffi-1.14.5-5.el9.x86_64.rpm 4.1 MB/s | 241 kB 00:00 2024-03-28T17:51:00.742 INFO:teuthology.orchestra.run.smithi079.stdout:(9/174): ceph-selinux-19.0.0-2145.g3be9a5c9.el9 384 kB/s | 25 kB 00:00 2024-03-28T17:51:00.757 INFO:teuthology.orchestra.run.smithi134.stdout:(83/174): python3-cryptography-36.0.1-4.el9.x86 14 MB/s | 1.1 MB 00:00 2024-03-28T17:51:00.782 INFO:teuthology.orchestra.run.smithi134.stdout:(84/174): python3-ply-3.11-14.el9.noarch.rpm 1.5 MB/s | 103 kB 00:00 2024-03-28T17:51:00.804 INFO:teuthology.orchestra.run.smithi022.stdout:(12/174): cephfs-shell-19.0.0-2145.g3be9a5c9.el 142 kB/s | 27 kB 00:00 2024-03-28T17:51:00.807 INFO:teuthology.orchestra.run.smithi134.stdout:(85/174): python3-pycparser-2.20-6.el9.noarch.r 2.5 MB/s | 124 kB 00:00 2024-03-28T17:51:00.849 INFO:teuthology.orchestra.run.smithi134.stdout:(86/174): python3-urllib3-1.26.5-5.el9.noarch.r 2.8 MB/s | 187 kB 00:00 2024-03-28T17:51:00.929 INFO:teuthology.orchestra.run.smithi022.stdout:(13/174): libcephfs-devel-19.0.0-2145.g3be9a5c9 252 kB/s | 31 kB 00:00 2024-03-28T17:51:01.396 INFO:teuthology.orchestra.run.smithi022.stdout:(14/174): libcephfs2-19.0.0-2145.g3be9a5c9.el9. 1.6 MB/s | 746 kB 00:00 2024-03-28T17:51:01.546 INFO:teuthology.orchestra.run.smithi022.stdout:(15/174): libcephsqlite-19.0.0-2145.g3be9a5c9.e 1.1 MB/s | 163 kB 00:00 2024-03-28T17:51:01.680 INFO:teuthology.orchestra.run.smithi022.stdout:(16/174): librados-devel-19.0.0-2145.g3be9a5c9. 956 kB/s | 127 kB 00:00 2024-03-28T17:51:01.775 INFO:teuthology.orchestra.run.smithi134.stdout:(87/174): jq-1.6-16.el9.x86_64.rpm 194 kB/s | 187 kB 00:00 2024-03-28T17:51:02.013 INFO:teuthology.orchestra.run.smithi022.stdout:(17/174): libradosstriper1-19.0.0-2145.g3be9a5c 1.5 MB/s | 501 kB 00:00 2024-03-28T17:51:02.517 INFO:teuthology.orchestra.run.smithi134.stdout:(88/174): libgfortran-11.4.1-3.el9.x86_64.rpm 482 kB/s | 803 kB 00:01 2024-03-28T17:51:02.651 INFO:teuthology.orchestra.run.smithi134.stdout:(89/174): libquadmath-11.4.1-3.el9.x86_64.rpm 223 kB/s | 195 kB 00:00 2024-03-28T17:51:02.764 INFO:teuthology.orchestra.run.smithi022.stdout:(18/174): ceph-common-19.0.0-2145.g3be9a5c9.el9 2.3 MB/s | 20 MB 00:08 2024-03-28T17:51:02.856 INFO:teuthology.orchestra.run.smithi022.stdout:(19/174): python3-ceph-argparse-19.0.0-2145.g3b 499 kB/s | 45 kB 00:00 2024-03-28T17:51:03.014 INFO:teuthology.orchestra.run.smithi022.stdout:(20/174): python3-ceph-common-19.0.0-2145.g3be9 833 kB/s | 131 kB 00:00 2024-03-28T17:51:03.181 INFO:teuthology.orchestra.run.smithi022.stdout:(21/174): python3-cephfs-19.0.0-2145.g3be9a5c9. 979 kB/s | 162 kB 00:00 2024-03-28T17:51:03.275 INFO:teuthology.orchestra.run.smithi134.stdout:(90/174): elfutils-libelf-devel-0.190-2.el9.x86 34 kB/s | 21 kB 00:00 2024-03-28T17:51:03.423 INFO:teuthology.orchestra.run.smithi022.stdout:(22/174): python3-rados-19.0.0-2145.g3be9a5c9.e 1.3 MB/s | 321 kB 00:00 2024-03-28T17:51:03.442 INFO:teuthology.orchestra.run.smithi134.stdout:(91/174): python3-requests-2.25.1-8.el9.noarch. 135 kB/s | 125 kB 00:00 2024-03-28T17:51:03.623 INFO:teuthology.orchestra.run.smithi022.stdout:(23/174): python3-rbd-19.0.0-2145.g3be9a5c9.el9 1.5 MB/s | 298 kB 00:00 2024-03-28T17:51:03.715 INFO:teuthology.orchestra.run.smithi022.stdout:(24/174): python3-rgw-19.0.0-2145.g3be9a5c9.el9 1.1 MB/s | 99 kB 00:00 2024-03-28T17:51:03.816 INFO:teuthology.orchestra.run.smithi022.stdout:(25/174): rbd-fuse-19.0.0-2145.g3be9a5c9.el9.x8 858 kB/s | 85 kB 00:00 2024-03-28T17:51:03.951 INFO:teuthology.orchestra.run.smithi134.stdout:(92/174): libuuid-devel-2.37.4-18.el9.x86_64.rp 38 kB/s | 25 kB 00:00 2024-03-28T17:51:04.250 INFO:teuthology.orchestra.run.smithi079.stdout:(10/174): ceph-common-19.0.0-2145.g3be9a5c9.el9 2.4 MB/s | 20 MB 00:08 2024-03-28T17:51:04.441 INFO:teuthology.orchestra.run.smithi022.stdout:(26/174): librgw2-19.0.0-2145.g3be9a5c9.el9.x86 2.1 MB/s | 5.1 MB 00:02 2024-03-28T17:51:04.533 INFO:teuthology.orchestra.run.smithi022.stdout:(27/174): rbd-nbd-19.0.0-2145.g3be9a5c9.el9.x86 1.8 MB/s | 170 kB 00:00 2024-03-28T17:51:04.583 INFO:teuthology.orchestra.run.smithi022.stdout:(28/174): ceph-grafana-dashboards-19.0.0-2145.g 694 kB/s | 34 kB 00:00 2024-03-28T17:51:04.642 INFO:teuthology.orchestra.run.smithi022.stdout:(29/174): ceph-mgr-cephadm-19.0.0-2145.g3be9a5c 2.4 MB/s | 139 kB 00:00 2024-03-28T17:51:04.725 INFO:teuthology.orchestra.run.smithi022.stdout:(30/174): rbd-mirror-19.0.0-2145.g3be9a5c9.el9. 3.4 MB/s | 3.1 MB 00:00 2024-03-28T17:51:04.750 INFO:teuthology.orchestra.run.smithi079.stdout:(11/174): ceph-test-19.0.0-2145.g3be9a5c9.el9.x 12 MB/s | 48 MB 00:04 2024-03-28T17:51:04.784 INFO:teuthology.orchestra.run.smithi079.stdout:(12/174): cephfs-mirror-19.0.0-2145.g3be9a5c9.e 414 kB/s | 220 kB 00:00 2024-03-28T17:51:04.826 INFO:teuthology.orchestra.run.smithi079.stdout:(13/174): cephfs-shell-19.0.0-2145.g3be9a5c9.el 363 kB/s | 27 kB 00:00 2024-03-28T17:51:04.860 INFO:teuthology.orchestra.run.smithi079.stdout:(14/174): libcephfs-devel-19.0.0-2145.g3be9a5c9 420 kB/s | 31 kB 00:00 2024-03-28T17:51:04.985 INFO:teuthology.orchestra.run.smithi079.stdout:(15/174): libcephfs2-19.0.0-2145.g3be9a5c9.el9. 4.6 MB/s | 746 kB 00:00 2024-03-28T17:51:05.035 INFO:teuthology.orchestra.run.smithi079.stdout:(16/174): libcephsqlite-19.0.0-2145.g3be9a5c9.e 934 kB/s | 163 kB 00:00 2024-03-28T17:51:05.077 INFO:teuthology.orchestra.run.smithi079.stdout:(17/174): librados-devel-19.0.0-2145.g3be9a5c9. 1.4 MB/s | 127 kB 00:00 2024-03-28T17:51:05.201 INFO:teuthology.orchestra.run.smithi022.stdout:(31/174): ceph-mgr-dashboard-19.0.0-2145.g3be9a 3.2 MB/s | 1.8 MB 00:00 2024-03-28T17:51:05.278 INFO:teuthology.orchestra.run.smithi079.stdout:(18/174): libradosstriper1-19.0.0-2145.g3be9a5c 2.0 MB/s | 501 kB 00:00 2024-03-28T17:51:05.376 INFO:teuthology.orchestra.run.smithi022.stdout:(32/174): ceph-mgr-modules-core-19.0.0-2145.g3b 1.4 MB/s | 248 kB 00:00 2024-03-28T17:51:05.386 INFO:teuthology.orchestra.run.smithi079.stdout:(19/174): python3-ceph-argparse-19.0.0-2145.g3b 419 kB/s | 45 kB 00:00 2024-03-28T17:51:05.451 INFO:teuthology.orchestra.run.smithi022.stdout:(33/174): ceph-mgr-rook-19.0.0-2145.g3be9a5c9.e 666 kB/s | 49 kB 00:00 2024-03-28T17:51:05.586 INFO:teuthology.orchestra.run.smithi079.stdout:(20/174): librgw2-19.0.0-2145.g3be9a5c9.el9.x86 10 MB/s | 5.1 MB 00:00 2024-03-28T17:51:05.620 INFO:teuthology.orchestra.run.smithi079.stdout:(21/174): python3-ceph-common-19.0.0-2145.g3be9 563 kB/s | 131 kB 00:00 2024-03-28T17:51:05.662 INFO:teuthology.orchestra.run.smithi079.stdout:(22/174): python3-cephfs-19.0.0-2145.g3be9a5c9. 2.1 MB/s | 162 kB 00:00 2024-03-28T17:51:05.729 INFO:teuthology.orchestra.run.smithi079.stdout:(23/174): python3-rados-19.0.0-2145.g3be9a5c9.e 2.9 MB/s | 321 kB 00:00 2024-03-28T17:51:05.762 INFO:teuthology.orchestra.run.smithi079.stdout:(24/174): python3-rbd-19.0.0-2145.g3be9a5c9.el9 2.9 MB/s | 298 kB 00:00 2024-03-28T17:51:05.796 INFO:teuthology.orchestra.run.smithi079.stdout:(25/174): python3-rgw-19.0.0-2145.g3be9a5c9.el9 1.5 MB/s | 99 kB 00:00 2024-03-28T17:51:05.830 INFO:teuthology.orchestra.run.smithi079.stdout:(26/174): rbd-fuse-19.0.0-2145.g3be9a5c9.el9.x8 1.3 MB/s | 85 kB 00:00 2024-03-28T17:51:05.905 INFO:teuthology.orchestra.run.smithi079.stdout:(27/174): rbd-nbd-19.0.0-2145.g3be9a5c9.el9.x86 2.2 MB/s | 170 kB 00:00 2024-03-28T17:51:05.972 INFO:teuthology.orchestra.run.smithi079.stdout:(28/174): ceph-grafana-dashboards-19.0.0-2145.g 516 kB/s | 34 kB 00:00 2024-03-28T17:51:05.993 INFO:teuthology.orchestra.run.smithi022.stdout:(34/174): ceph-test-19.0.0-2145.g3be9a5c9.el9.x 5.9 MB/s | 48 MB 00:08 2024-03-28T17:51:06.018 INFO:teuthology.orchestra.run.smithi022.stdout:(35/174): ceph-prometheus-alerts-19.0.0-2145.g3 29 kB/s | 16 kB 00:00 2024-03-28T17:51:06.039 INFO:teuthology.orchestra.run.smithi079.stdout:(29/174): ceph-mgr-cephadm-19.0.0-2145.g3be9a5c 2.1 MB/s | 139 kB 00:00 2024-03-28T17:51:06.085 INFO:teuthology.orchestra.run.smithi022.stdout:(36/174): ceph-volume-19.0.0-2145.g3be9a5c9.el9 2.8 MB/s | 264 kB 00:00 2024-03-28T17:51:06.185 INFO:teuthology.orchestra.run.smithi022.stdout:(37/174): cephfs-top-19.0.0-2145.g3be9a5c9.el9. 184 kB/s | 18 kB 00:00 2024-03-28T17:51:06.231 INFO:teuthology.orchestra.run.smithi079.stdout:(30/174): ceph-mgr-dashboard-19.0.0-2145.g3be9a 9.4 MB/s | 1.8 MB 00:00 2024-03-28T17:51:06.378 INFO:teuthology.orchestra.run.smithi022.stdout:(38/174): automake-1.16.2-8.el9.noarch.rpm 3.4 MB/s | 662 kB 00:00 2024-03-28T17:51:06.453 INFO:teuthology.orchestra.run.smithi022.stdout:(39/174): cephadm-19.0.0-2145.g3be9a5c9.el9.noa 1.7 MB/s | 748 kB 00:00 2024-03-28T17:51:06.494 INFO:teuthology.orchestra.run.smithi022.stdout:(40/174): bison-3.7.4-5.el9.x86_64.rpm 7.8 MB/s | 920 kB 00:00 2024-03-28T17:51:06.520 INFO:teuthology.orchestra.run.smithi022.stdout:(41/174): boost-program-options-1.75.0-8.el9.x8 1.8 MB/s | 106 kB 00:00 2024-03-28T17:51:06.534 INFO:teuthology.orchestra.run.smithi134.stdout:(93/174): postgresql-13.14-1.el9.x86_64.rpm 630 kB/s | 1.6 MB 00:02 2024-03-28T17:51:06.553 INFO:teuthology.orchestra.run.smithi022.stdout:(42/174): flex-2.6.4-9.el9.x86_64.rpm 5.3 MB/s | 307 kB 00:00 2024-03-28T17:51:06.595 INFO:teuthology.orchestra.run.smithi022.stdout:(43/174): flexiblas-3.0.4-8.el9.x86_64.rpm 442 kB/s | 32 kB 00:00 2024-03-28T17:51:06.704 INFO:teuthology.orchestra.run.smithi022.stdout:(44/174): flexiblas-netlib-3.0.4-8.el9.x86_64.r 20 MB/s | 3.0 MB 00:00 2024-03-28T17:51:06.746 INFO:teuthology.orchestra.run.smithi022.stdout:(45/174): flexiblas-openblas-openmp-3.0.4-8.el9 128 kB/s | 17 kB 00:00 2024-03-28T17:51:06.796 INFO:teuthology.orchestra.run.smithi022.stdout:(46/174): libaio-devel-0.3.111-13.el9.x86_64.rp 206 kB/s | 10 kB 00:00 2024-03-28T17:51:06.814 INFO:teuthology.orchestra.run.smithi079.stdout:(31/174): ceph-mgr-diskprediction-local-19.0.0- 13 MB/s | 7.4 MB 00:00 2024-03-28T17:51:06.821 INFO:teuthology.orchestra.run.smithi022.stdout:(47/174): libpmemobj-1.12.1-1.el9.x86_64.rpm 2.1 MB/s | 159 kB 00:00 2024-03-28T17:51:06.863 INFO:teuthology.orchestra.run.smithi022.stdout:(48/174): librabbitmq-0.11.0-7.el9.x86_64.rpm 684 kB/s | 44 kB 00:00 2024-03-28T17:51:06.890 INFO:teuthology.orchestra.run.smithi079.stdout:(32/174): rbd-mirror-19.0.0-2145.g3be9a5c9.el9. 2.8 MB/s | 3.1 MB 00:01 2024-03-28T17:51:06.922 INFO:teuthology.orchestra.run.smithi022.stdout:(49/174): librdkafka-1.6.1-102.el9.x86_64.rpm 6.5 MB/s | 661 kB 00:00 2024-03-28T17:51:06.923 INFO:teuthology.orchestra.run.smithi079.stdout:(33/174): ceph-mgr-modules-core-19.0.0-2145.g3b 2.3 MB/s | 248 kB 00:00 2024-03-28T17:51:06.957 INFO:teuthology.orchestra.run.smithi079.stdout:(34/174): ceph-mgr-rook-19.0.0-2145.g3be9a5c9.e 748 kB/s | 49 kB 00:00 2024-03-28T17:51:06.964 INFO:teuthology.orchestra.run.smithi022.stdout:(50/174): libtool-2.4.6-45.el9.x86_64.rpm 6.0 MB/s | 578 kB 00:00 2024-03-28T17:51:06.991 INFO:teuthology.orchestra.run.smithi079.stdout:(35/174): ceph-prometheus-alerts-19.0.0-2145.g3 248 kB/s | 16 kB 00:00 2024-03-28T17:51:07.006 INFO:teuthology.orchestra.run.smithi022.stdout:(51/174): libzstd-devel-1.5.1-2.el9.x86_64.rpm 598 kB/s | 46 kB 00:00 2024-03-28T17:51:07.039 INFO:teuthology.orchestra.run.smithi022.stdout:(52/174): lttng-ust-2.12.0-6.el9.x86_64.rpm 3.7 MB/s | 282 kB 00:00 2024-03-28T17:51:07.073 INFO:teuthology.orchestra.run.smithi022.stdout:(53/174): lua-5.4.4-4.el9.x86_64.rpm 2.9 MB/s | 187 kB 00:00 2024-03-28T17:51:07.082 INFO:teuthology.orchestra.run.smithi079.stdout:(36/174): cephadm-19.0.0-2145.g3be9a5c9.el9.noa 8.0 MB/s | 748 kB 00:00 2024-03-28T17:51:07.115 INFO:teuthology.orchestra.run.smithi022.stdout:(54/174): oniguruma-6.9.6-1.el9.5.x86_64.rpm 2.9 MB/s | 217 kB 00:00 2024-03-28T17:51:07.124 INFO:teuthology.orchestra.run.smithi079.stdout:(37/174): ceph-volume-19.0.0-2145.g3be9a5c9.el9 1.5 MB/s | 264 kB 00:00 2024-03-28T17:51:07.140 INFO:teuthology.orchestra.run.smithi022.stdout:(55/174): openblas-0.3.21-2.el9.x86_64.rpm 531 kB/s | 34 kB 00:00 2024-03-28T17:51:07.150 INFO:teuthology.orchestra.run.smithi079.stdout:(38/174): cephfs-top-19.0.0-2145.g3be9a5c9.el9. 276 kB/s | 18 kB 00:00 2024-03-28T17:51:07.274 INFO:teuthology.orchestra.run.smithi022.stdout:(56/174): openblas-openmp-0.3.21-2.el9.x86_64.r 30 MB/s | 4.7 MB 00:00 2024-03-28T17:51:07.348 INFO:teuthology.orchestra.run.smithi022.stdout:(57/174): protobuf-3.14.0-13.el9.x86_64.rpm 5.3 MB/s | 1.0 MB 00:00 2024-03-28T17:51:07.375 INFO:teuthology.orchestra.run.smithi079.stdout:(39/174): automake-1.16.2-8.el9.noarch.rpm 2.6 MB/s | 662 kB 00:00 2024-03-28T17:51:07.383 INFO:teuthology.orchestra.run.smithi022.stdout:(58/174): python3-attrs-20.3.0-7.el9.noarch.rpm 775 kB/s | 78 kB 00:00 2024-03-28T17:51:07.418 INFO:teuthology.orchestra.run.smithi134.stdout:(94/174): ceph-test-19.0.0-2145.g3be9a5c9.el9.x 3.9 MB/s | 48 MB 00:12 2024-03-28T17:51:07.426 INFO:teuthology.orchestra.run.smithi079.stdout:(40/174): bison-3.7.4-5.el9.x86_64.rpm 3.3 MB/s | 920 kB 00:00 2024-03-28T17:51:07.451 INFO:teuthology.orchestra.run.smithi079.stdout:(41/174): boost-program-options-1.75.0-8.el9.x8 1.5 MB/s | 106 kB 00:00 2024-03-28T17:51:07.485 INFO:teuthology.orchestra.run.smithi079.stdout:(42/174): flex-2.6.4-9.el9.x86_64.rpm 5.3 MB/s | 307 kB 00:00 2024-03-28T17:51:07.510 INFO:teuthology.orchestra.run.smithi079.stdout:(43/174): flexiblas-3.0.4-8.el9.x86_64.rpm 572 kB/s | 32 kB 00:00 2024-03-28T17:51:07.533 INFO:teuthology.orchestra.run.smithi022.stdout:(59/174): python3-babel-2.9.1-2.el9.noarch.rpm 32 MB/s | 5.8 MB 00:00 2024-03-28T17:51:07.550 INFO:teuthology.orchestra.run.smithi022.stdout:(60/174): python3-jsonpatch-1.21-16.el9.noarch. 163 kB/s | 24 kB 00:00 2024-03-28T17:51:07.584 INFO:teuthology.orchestra.run.smithi022.stdout:(61/174): python3-jsonpointer-2.0-4.el9.noarch. 564 kB/s | 18 kB 00:00 2024-03-28T17:51:07.602 INFO:teuthology.orchestra.run.smithi079.stdout:(44/174): flexiblas-netlib-3.0.4-8.el9.x86_64.r 26 MB/s | 3.0 MB 00:00 2024-03-28T17:51:07.609 INFO:teuthology.orchestra.run.smithi022.stdout:(62/174): python3-mako-1.1.4-6.el9.noarch.rpm 2.5 MB/s | 144 kB 00:00 2024-03-28T17:51:07.627 INFO:teuthology.orchestra.run.smithi079.stdout:(45/174): flexiblas-openblas-openmp-3.0.4-8.el9 167 kB/s | 17 kB 00:00 2024-03-28T17:51:07.661 INFO:teuthology.orchestra.run.smithi079.stdout:(46/174): libaio-devel-0.3.111-13.el9.x86_64.rp 312 kB/s | 10 kB 00:00 2024-03-28T17:51:07.686 INFO:teuthology.orchestra.run.smithi079.stdout:(47/174): libpmemobj-1.12.1-1.el9.x86_64.rpm 2.7 MB/s | 159 kB 00:00 2024-03-28T17:51:07.711 INFO:teuthology.orchestra.run.smithi079.stdout:(48/174): librabbitmq-0.11.0-7.el9.x86_64.rpm 917 kB/s | 44 kB 00:00 2024-03-28T17:51:07.742 INFO:teuthology.orchestra.run.smithi022.stdout:(63/174): ceph-mgr-diskprediction-local-19.0.0- 2.5 MB/s | 7.4 MB 00:03 2024-03-28T17:51:07.761 INFO:teuthology.orchestra.run.smithi079.stdout:(49/174): librdkafka-1.6.1-102.el9.x86_64.rpm 8.7 MB/s | 661 kB 00:00 2024-03-28T17:51:07.795 INFO:teuthology.orchestra.run.smithi079.stdout:(50/174): libtool-2.4.6-45.el9.x86_64.rpm 7.3 MB/s | 578 kB 00:00 2024-03-28T17:51:07.809 INFO:teuthology.orchestra.run.smithi022.stdout:(64/174): python3-markupsafe-1.1.1-12.el9.x86_6 158 kB/s | 32 kB 00:00 2024-03-28T17:51:07.841 INFO:teuthology.orchestra.run.smithi079.stdout:(51/174): libzstd-devel-1.5.1-2.el9.x86_64.rpm 622 kB/s | 46 kB 00:00 2024-03-28T17:51:07.870 INFO:teuthology.orchestra.run.smithi079.stdout:(52/174): lttng-ust-2.12.0-6.el9.x86_64.rpm 3.7 MB/s | 282 kB 00:00 2024-03-28T17:51:07.896 INFO:teuthology.orchestra.run.smithi079.stdout:(53/174): lua-5.4.4-4.el9.x86_64.rpm 3.6 MB/s | 187 kB 00:00 2024-03-28T17:51:07.926 INFO:teuthology.orchestra.run.smithi022.stdout:(65/174): python3-numpy-1.20.1-5.el9.x86_64.rpm 15 MB/s | 4.8 MB 00:00 2024-03-28T17:51:07.929 INFO:teuthology.orchestra.run.smithi079.stdout:(54/174): oniguruma-6.9.6-1.el9.5.x86_64.rpm 3.7 MB/s | 217 kB 00:00 2024-03-28T17:51:07.951 INFO:teuthology.orchestra.run.smithi022.stdout:(66/174): python3-numpy-f2py-1.20.1-5.el9.x86_6 1.0 MB/s | 217 kB 00:00 2024-03-28T17:51:07.964 INFO:teuthology.orchestra.run.smithi079.stdout:(55/174): openblas-0.3.21-2.el9.x86_64.rpm 525 kB/s | 34 kB 00:00 2024-03-28T17:51:07.976 INFO:teuthology.orchestra.run.smithi022.stdout:(67/174): python3-oauthlib-3.1.1-5.el9.noarch.r 1.1 MB/s | 174 kB 00:00 2024-03-28T17:51:08.027 INFO:teuthology.orchestra.run.smithi022.stdout:(68/174): python3-packaging-20.9-5.el9.noarch.r 1.4 MB/s | 69 kB 00:00 2024-03-28T17:51:08.044 INFO:teuthology.orchestra.run.smithi022.stdout:(69/174): python3-prettytable-0.7.2-27.el9.noar 633 kB/s | 41 kB 00:00 2024-03-28T17:51:08.060 INFO:teuthology.orchestra.run.smithi022.stdout:(70/174): python3-protobuf-3.14.0-13.el9.noarch 2.8 MB/s | 238 kB 00:00 2024-03-28T17:51:08.094 INFO:teuthology.orchestra.run.smithi022.stdout:(71/174): python3-pyasn1-0.4.8-6.el9.noarch.rpm 4.1 MB/s | 132 kB 00:00 2024-03-28T17:51:08.097 INFO:teuthology.orchestra.run.smithi079.stdout:(56/174): openblas-openmp-0.3.21-2.el9.x86_64.r 29 MB/s | 4.7 MB 00:00 2024-03-28T17:51:08.111 INFO:teuthology.orchestra.run.smithi022.stdout:(72/174): python3-pytz-2021.1-5.el9.noarch.rpm 978 kB/s | 47 kB 00:00 2024-03-28T17:51:08.136 INFO:teuthology.orchestra.run.smithi022.stdout:(73/174): python3-pyasn1-modules-0.4.8-6.el9.no 2.8 MB/s | 211 kB 00:00 2024-03-28T17:51:08.138 INFO:teuthology.orchestra.run.smithi079.stdout:(57/174): protobuf-3.14.0-13.el9.x86_64.rpm 6.4 MB/s | 1.0 MB 00:00 2024-03-28T17:51:08.170 INFO:teuthology.orchestra.run.smithi022.stdout:(74/174): python3-requests-oauthlib-1.3.0-12.el 766 kB/s | 43 kB 00:00 2024-03-28T17:51:08.181 INFO:teuthology.orchestra.run.smithi079.stdout:(58/174): python3-attrs-20.3.0-7.el9.noarch.rpm 1.9 MB/s | 78 kB 00:00 2024-03-28T17:51:08.268 INFO:teuthology.orchestra.run.smithi134.stdout:(95/174): postgresql-private-libs-13.14-1.el9.x 157 kB/s | 133 kB 00:00 2024-03-28T17:51:08.473 INFO:teuthology.orchestra.run.smithi079.stdout:(59/174): ceph-osd-19.0.0-2145.g3be9a5c9.el9.x8 1.8 MB/s | 17 MB 00:09 2024-03-28T17:51:08.545 INFO:teuthology.orchestra.run.smithi022.stdout:(75/174): python3-scipy-1.6.2-8.el9.x86_64.rpm 36 MB/s | 16 MB 00:00 2024-03-28T17:51:08.562 INFO:teuthology.orchestra.run.smithi022.stdout:(76/174): python3-toml-0.10.2-6.el9.noarch.rpm 95 kB/s | 39 kB 00:00 2024-03-28T17:51:08.579 INFO:teuthology.orchestra.run.smithi022.stdout:(77/174): python3-wcwidth-0.2.5-8.el9.noarch.rp 104 kB/s | 41 kB 00:00 2024-03-28T17:51:08.590 INFO:teuthology.orchestra.run.smithi079.stdout:(60/174): python3-babel-2.9.1-2.el9.noarch.rpm 13 MB/s | 5.8 MB 00:00 2024-03-28T17:51:08.612 INFO:teuthology.orchestra.run.smithi022.stdout:(78/174): uuid-1.6.2-55.el9.x86_64.rpm 1.7 MB/s | 56 kB 00:00 2024-03-28T17:51:08.615 INFO:teuthology.orchestra.run.smithi079.stdout:(61/174): python3-jsonpatch-1.21-16.el9.noarch. 58 kB/s | 24 kB 00:00 2024-03-28T17:51:08.632 INFO:teuthology.orchestra.run.smithi079.stdout:(62/174): python3-jsonpointer-2.0-4.el9.noarch. 120 kB/s | 18 kB 00:00 2024-03-28T17:51:08.665 INFO:teuthology.orchestra.run.smithi079.stdout:(63/174): python3-mako-1.1.4-6.el9.noarch.rpm 2.9 MB/s | 144 kB 00:00 2024-03-28T17:51:08.666 INFO:teuthology.orchestra.run.smithi022.stdout:(79/174): socat-1.7.4.1-5.el9.x86_64.rpm 3.4 MB/s | 300 kB 00:00 2024-03-28T17:51:08.682 INFO:teuthology.orchestra.run.smithi079.stdout:(64/174): python3-markupsafe-1.1.1-12.el9.x86_6 482 kB/s | 32 kB 00:00 2024-03-28T17:51:08.688 INFO:teuthology.orchestra.run.smithi022.stdout:(80/174): xmlstarlet-1.6.1-20.el9.x86_64.rpm 586 kB/s | 63 kB 00:00 2024-03-28T17:51:08.704 INFO:teuthology.orchestra.run.smithi022.stdout:(81/174): zlib-devel-1.2.11-40.el9.x86_64.rpm 502 kB/s | 44 kB 00:00 2024-03-28T17:51:08.738 INFO:teuthology.orchestra.run.smithi022.stdout:(82/174): fuse-2.9.9-15.el9.x86_64.rpm 1.6 MB/s | 79 kB 00:00 2024-03-28T17:51:08.763 INFO:teuthology.orchestra.run.smithi022.stdout:(83/174): python3-cffi-1.14.5-5.el9.x86_64.rpm 3.2 MB/s | 241 kB 00:00 2024-03-28T17:51:08.814 INFO:teuthology.orchestra.run.smithi022.stdout:(84/174): python3-cryptography-36.0.1-4.el9.x86 11 MB/s | 1.1 MB 00:00 2024-03-28T17:51:08.816 INFO:teuthology.orchestra.run.smithi079.stdout:(65/174): python3-numpy-1.20.1-5.el9.x86_64.rpm 26 MB/s | 4.8 MB 00:00 2024-03-28T17:51:08.835 INFO:teuthology.orchestra.run.smithi134.stdout:(96/174): postgresql-contrib-13.14-1.el9.x86_64 396 kB/s | 911 kB 00:02 2024-03-28T17:51:08.839 INFO:teuthology.orchestra.run.smithi022.stdout:(85/174): python3-ply-3.11-14.el9.noarch.rpm 1.6 MB/s | 103 kB 00:00 2024-03-28T17:51:08.841 INFO:teuthology.orchestra.run.smithi079.stdout:(66/174): python3-numpy-f2py-1.20.1-5.el9.x86_6 1.5 MB/s | 217 kB 00:00 2024-03-28T17:51:08.864 INFO:teuthology.orchestra.run.smithi022.stdout:(86/174): python3-pycparser-2.20-6.el9.noarch.r 1.4 MB/s | 124 kB 00:00 2024-03-28T17:51:08.874 INFO:teuthology.orchestra.run.smithi079.stdout:(67/174): python3-oauthlib-3.1.1-5.el9.noarch.r 985 kB/s | 174 kB 00:00 2024-03-28T17:51:08.898 INFO:teuthology.orchestra.run.smithi022.stdout:(87/174): python3-urllib3-1.26.5-5.el9.noarch.r 2.3 MB/s | 187 kB 00:00 2024-03-28T17:51:08.916 INFO:teuthology.orchestra.run.smithi079.stdout:(68/174): python3-packaging-20.9-5.el9.noarch.r 1.7 MB/s | 69 kB 00:00 2024-03-28T17:51:08.959 INFO:teuthology.orchestra.run.smithi079.stdout:(69/174): python3-prettytable-0.7.2-27.el9.noar 501 kB/s | 41 kB 00:00 2024-03-28T17:51:08.992 INFO:teuthology.orchestra.run.smithi079.stdout:(70/174): python3-protobuf-3.14.0-13.el9.noarch 2.0 MB/s | 238 kB 00:00 2024-03-28T17:51:09.034 INFO:teuthology.orchestra.run.smithi079.stdout:(71/174): python3-pytz-2021.1-5.el9.noarch.rpm 1.2 MB/s | 47 kB 00:00 2024-03-28T17:51:09.068 INFO:teuthology.orchestra.run.smithi079.stdout:(72/174): python3-pyasn1-modules-0.4.8-6.el9.no 2.8 MB/s | 211 kB 00:00 2024-03-28T17:51:09.081 INFO:teuthology.orchestra.run.smithi022.stdout:(88/174): jq-1.6-16.el9.x86_64.rpm 875 kB/s | 187 kB 00:00 2024-03-28T17:51:09.093 INFO:teuthology.orchestra.run.smithi079.stdout:(73/174): python3-pyasn1-0.4.8-6.el9.noarch.rpm 1.3 MB/s | 132 kB 00:00 2024-03-28T17:51:09.118 INFO:teuthology.orchestra.run.smithi079.stdout:(74/174): python3-requests-oauthlib-1.3.0-12.el 526 kB/s | 43 kB 00:00 2024-03-28T17:51:09.131 INFO:teuthology.orchestra.run.smithi022.stdout:(89/174): libquadmath-11.4.1-3.el9.x86_64.rpm 840 kB/s | 195 kB 00:00 2024-03-28T17:51:09.174 INFO:teuthology.orchestra.run.smithi022.stdout:(90/174): python3-requests-2.25.1-8.el9.noarch. 1.3 MB/s | 125 kB 00:00 2024-03-28T17:51:09.241 INFO:teuthology.orchestra.run.smithi022.stdout:(91/174): libgfortran-11.4.1-3.el9.x86_64.rpm 2.1 MB/s | 803 kB 00:00 2024-03-28T17:51:09.291 INFO:teuthology.orchestra.run.smithi022.stdout:(92/174): elfutils-libelf-devel-0.190-2.el9.x86 136 kB/s | 21 kB 00:00 2024-03-28T17:51:09.350 INFO:teuthology.orchestra.run.smithi022.stdout:(93/174): libuuid-devel-2.37.4-18.el9.x86_64.rp 146 kB/s | 25 kB 00:00 2024-03-28T17:51:09.443 INFO:teuthology.orchestra.run.smithi079.stdout:(75/174): python3-scipy-1.6.2-8.el9.x86_64.rpm 42 MB/s | 16 MB 00:00 2024-03-28T17:51:09.469 INFO:teuthology.orchestra.run.smithi079.stdout:(76/174): python3-toml-0.10.2-6.el9.noarch.rpm 115 kB/s | 39 kB 00:00 2024-03-28T17:51:09.494 INFO:teuthology.orchestra.run.smithi079.stdout:(77/174): python3-wcwidth-0.2.5-8.el9.noarch.rp 113 kB/s | 41 kB 00:00 2024-03-28T17:51:09.536 INFO:teuthology.orchestra.run.smithi079.stdout:(78/174): uuid-1.6.2-55.el9.x86_64.rpm 1.4 MB/s | 56 kB 00:00 2024-03-28T17:51:09.561 INFO:teuthology.orchestra.run.smithi079.stdout:(79/174): xmlstarlet-1.6.1-20.el9.x86_64.rpm 962 kB/s | 63 kB 00:00 2024-03-28T17:51:09.569 INFO:teuthology.orchestra.run.smithi134.stdout:(97/174): openssl-devel-3.0.7-27.el9.x86_64.rpm 691 kB/s | 4.1 MB 00:06 2024-03-28T17:51:09.591 INFO:teuthology.orchestra.run.smithi022.stdout:(94/174): postgresql-contrib-13.14-1.el9.x86_64 3.7 MB/s | 911 kB 00:00 2024-03-28T17:51:09.595 INFO:teuthology.orchestra.run.smithi079.stdout:(80/174): socat-1.7.4.1-5.el9.x86_64.rpm 3.0 MB/s | 300 kB 00:00 2024-03-28T17:51:09.620 INFO:teuthology.orchestra.run.smithi079.stdout:(81/174): zlib-devel-1.2.11-40.el9.x86_64.rpm 535 kB/s | 44 kB 00:00 2024-03-28T17:51:09.645 INFO:teuthology.orchestra.run.smithi079.stdout:(82/174): fuse-2.9.9-15.el9.x86_64.rpm 980 kB/s | 79 kB 00:00 2024-03-28T17:51:09.650 INFO:teuthology.orchestra.run.smithi022.stdout:(95/174): postgresql-13.14-1.el9.x86_64.rpm 4.4 MB/s | 1.6 MB 00:00 2024-03-28T17:51:09.684 INFO:teuthology.orchestra.run.smithi022.stdout:(96/174): postgresql-private-libs-13.14-1.el9.x 1.4 MB/s | 133 kB 00:00 2024-03-28T17:51:09.695 INFO:teuthology.orchestra.run.smithi079.stdout:(83/174): python3-cffi-1.14.5-5.el9.x86_64.rpm 3.2 MB/s | 241 kB 00:00 2024-03-28T17:51:09.737 INFO:teuthology.orchestra.run.smithi079.stdout:(84/174): python3-cryptography-36.0.1-4.el9.x86 9.9 MB/s | 1.1 MB 00:00 2024-03-28T17:51:09.765 INFO:teuthology.orchestra.run.smithi079.stdout:(85/174): python3-ply-3.11-14.el9.noarch.rpm 991 kB/s | 103 kB 00:00 2024-03-28T17:51:09.796 INFO:teuthology.orchestra.run.smithi079.stdout:(86/174): python3-pycparser-2.20-6.el9.noarch.r 2.1 MB/s | 124 kB 00:00 2024-03-28T17:51:09.800 INFO:teuthology.orchestra.run.smithi022.stdout:(97/174): openssl-devel-3.0.7-27.el9.x86_64.rpm 7.4 MB/s | 4.1 MB 00:00 2024-03-28T17:51:09.802 INFO:teuthology.orchestra.run.smithi134.stdout:(98/174): python3-devel-3.9.17-2.el9.x86_64.rpm 254 kB/s | 245 kB 00:00 2024-03-28T17:51:09.821 INFO:teuthology.orchestra.run.smithi079.stdout:(87/174): python3-urllib3-1.26.5-5.el9.noarch.r 2.2 MB/s | 187 kB 00:00 2024-03-28T17:51:09.834 INFO:teuthology.orchestra.run.smithi022.stdout:(98/174): python3-devel-3.9.17-2.el9.x86_64.rpm 1.6 MB/s | 245 kB 00:00 2024-03-28T17:51:09.901 INFO:teuthology.orchestra.run.smithi022.stdout:(99/174): python3-jinja2-2.11.3-5.el9.noarch.rp 2.4 MB/s | 248 kB 00:00 2024-03-28T17:51:09.905 INFO:teuthology.orchestra.run.smithi079.stdout:(88/174): jq-1.6-16.el9.x86_64.rpm 1.3 MB/s | 187 kB 00:00 2024-03-28T17:51:09.930 INFO:teuthology.orchestra.run.smithi079.stdout:(89/174): libquadmath-11.4.1-3.el9.x86_64.rpm 1.8 MB/s | 195 kB 00:00 2024-03-28T17:51:09.935 INFO:teuthology.orchestra.run.smithi022.stdout:(100/174): qatlib-24.02.0-1.el9.x86_64.rpm 2.2 MB/s | 221 kB 00:00 2024-03-28T17:51:09.997 INFO:teuthology.orchestra.run.smithi079.stdout:(90/174): libgfortran-11.4.1-3.el9.x86_64.rpm 4.5 MB/s | 803 kB 00:00 2024-03-28T17:51:10.010 INFO:teuthology.orchestra.run.smithi022.stdout:(101/174): qatlib-service-24.02.0-1.el9.x86_64. 332 kB/s | 36 kB 00:00 2024-03-28T17:51:10.022 INFO:teuthology.orchestra.run.smithi079.stdout:(91/174): elfutils-libelf-devel-0.190-2.el9.x86 236 kB/s | 21 kB 00:00 2024-03-28T17:51:10.043 INFO:teuthology.orchestra.run.smithi022.stdout:(102/174): qatzip-libs-1.2.0-1.el9.x86_64.rpm 439 kB/s | 47 kB 00:00 2024-03-28T17:51:10.056 INFO:teuthology.orchestra.run.smithi079.stdout:(92/174): python3-requests-2.25.1-8.el9.noarch. 833 kB/s | 125 kB 00:00 2024-03-28T17:51:10.081 INFO:teuthology.orchestra.run.smithi079.stdout:(93/174): libuuid-devel-2.37.4-18.el9.x86_64.rp 306 kB/s | 25 kB 00:00 2024-03-28T17:51:10.168 INFO:teuthology.orchestra.run.smithi022.stdout:(103/174): postgresql-server-13.14-1.el9.x86_64 11 MB/s | 5.8 MB 00:00 2024-03-28T17:51:10.194 INFO:teuthology.orchestra.run.smithi022.stdout:(104/174): xfsprogs-devel-6.3.0-1.el9.x86_64.rp 513 kB/s | 94 kB 00:00 2024-03-28T17:51:10.236 INFO:teuthology.orchestra.run.smithi022.stdout:(105/174): lua-devel-5.4.4-4.el9.x86_64.rpm 116 kB/s | 22 kB 00:00 2024-03-28T17:51:10.273 INFO:teuthology.orchestra.run.smithi079.stdout:(94/174): openssl-devel-3.0.7-27.el9.x86_64.rpm 17 MB/s | 4.1 MB 00:00 2024-03-28T17:51:10.336 INFO:teuthology.orchestra.run.smithi022.stdout:(106/174): gperftools-libs-2.9.1-2.el9.x86_64.r 3.1 MB/s | 309 kB 00:00 2024-03-28T17:51:10.370 INFO:teuthology.orchestra.run.smithi022.stdout:(107/174): abseil-cpp-20211102.0-3.el9.x86_64.r 3.1 MB/s | 548 kB 00:00 2024-03-28T17:51:10.382 INFO:teuthology.orchestra.run.smithi079.stdout:(95/174): postgresql-contrib-13.14-1.el9.x86_64 3.0 MB/s | 911 kB 00:00 2024-03-28T17:51:10.395 INFO:teuthology.orchestra.run.smithi022.stdout:(108/174): grpc-data-1.46.7-10.el9.noarch.rpm 339 kB/s | 19 kB 00:00 2024-03-28T17:51:10.407 INFO:teuthology.orchestra.run.smithi079.stdout:(96/174): postgresql-private-libs-13.14-1.el9.x 1.0 MB/s | 133 kB 00:00 2024-03-28T17:51:10.458 INFO:teuthology.orchestra.run.smithi079.stdout:(97/174): postgresql-13.14-1.el9.x86_64.rpm 4.0 MB/s | 1.6 MB 00:00 2024-03-28T17:51:10.470 INFO:teuthology.orchestra.run.smithi022.stdout:(109/174): protobuf-compiler-3.14.0-13.el9.x86_ 2.8 MB/s | 863 kB 00:00 2024-03-28T17:51:10.495 INFO:teuthology.orchestra.run.smithi022.stdout:(110/174): libarrow-doc-9.0.0-10.el9.noarch.rpm 261 kB/s | 26 kB 00:00 2024-03-28T17:51:10.537 INFO:teuthology.orchestra.run.smithi022.stdout:(111/174): libunwind-1.6.2-1.el9.x86_64.rpm 1.6 MB/s | 67 kB 00:00 2024-03-28T17:51:10.553 INFO:teuthology.orchestra.run.smithi134.stdout:(99/174): python3-jinja2-2.11.3-5.el9.noarch.rp 252 kB/s | 248 kB 00:00 2024-03-28T17:51:10.562 INFO:teuthology.orchestra.run.smithi022.stdout:(112/174): liboath-2.6.7-2.el9.x86_64.rpm 533 kB/s | 49 kB 00:00 2024-03-28T17:51:10.591 INFO:teuthology.orchestra.run.smithi079.stdout:(98/174): python3-jinja2-2.11.3-5.el9.noarch.rp 1.8 MB/s | 248 kB 00:00 2024-03-28T17:51:10.596 INFO:teuthology.orchestra.run.smithi022.stdout:(113/174): luarocks-3.9.2-1.el9.noarch.rpm 2.6 MB/s | 151 kB 00:00 2024-03-28T17:51:10.625 INFO:teuthology.orchestra.run.smithi079.stdout:(99/174): python3-devel-3.9.17-2.el9.x86_64.rpm 1.1 MB/s | 245 kB 00:00 2024-03-28T17:51:10.663 INFO:teuthology.orchestra.run.smithi022.stdout:(114/174): python3-asyncssh-2.13.2-5.el9.noarch 8.1 MB/s | 548 kB 00:00 2024-03-28T17:51:10.692 INFO:teuthology.orchestra.run.smithi079.stdout:(100/174): qatlib-service-24.02.0-1.el9.x86_64. 545 kB/s | 36 kB 00:00 2024-03-28T17:51:10.713 INFO:teuthology.orchestra.run.smithi022.stdout:(115/174): parquet-libs-9.0.0-10.el9.x86_64.rpm 5.5 MB/s | 838 kB 00:00 2024-03-28T17:51:10.738 INFO:teuthology.orchestra.run.smithi022.stdout:(116/174): python3-bcrypt-3.2.2-1.el9.x86_64.rp 584 kB/s | 43 kB 00:00 2024-03-28T17:51:10.764 INFO:teuthology.orchestra.run.smithi022.stdout:(117/174): python3-cachetools-4.2.4-1.el9.noarc 644 kB/s | 32 kB 00:00 2024-03-28T17:51:10.767 INFO:teuthology.orchestra.run.smithi079.stdout:(101/174): qatlib-24.02.0-1.el9.x86_64.rpm 1.2 MB/s | 221 kB 00:00 2024-03-28T17:51:10.778 INFO:teuthology.orchestra.run.smithi134.stdout:(100/174): qatlib-24.02.0-1.el9.x86_64.rpm 244 kB/s | 221 kB 00:00 2024-03-28T17:51:10.789 INFO:teuthology.orchestra.run.smithi022.stdout:(118/174): python3-certifi-2021.10.8-2.el9.noar 304 kB/s | 15 kB 00:00 2024-03-28T17:51:10.814 INFO:teuthology.orchestra.run.smithi022.stdout:(119/174): python3-cheroot-8.6.0-4.el9.noarch.r 3.4 MB/s | 172 kB 00:00 2024-03-28T17:51:10.834 INFO:teuthology.orchestra.run.smithi079.stdout:(102/174): qatzip-libs-1.2.0-1.el9.x86_64.rpm 336 kB/s | 47 kB 00:00 2024-03-28T17:51:10.848 INFO:teuthology.orchestra.run.smithi022.stdout:(120/174): python3-cherrypy-18.6.1-2.el9.noarch 6.1 MB/s | 358 kB 00:00 2024-03-28T17:51:10.859 INFO:teuthology.orchestra.run.smithi079.stdout:(103/174): xfsprogs-devel-6.3.0-1.el9.x86_64.rp 1.0 MB/s | 94 kB 00:00 2024-03-28T17:51:10.965 INFO:teuthology.orchestra.run.smithi022.stdout:(121/174): libarrow-9.0.0-10.el9.x86_64.rpm 7.4 MB/s | 4.4 MB 00:00 2024-03-28T17:51:10.976 INFO:teuthology.orchestra.run.smithi079.stdout:(104/174): postgresql-server-13.14-1.el9.x86_64 9.7 MB/s | 5.8 MB 00:00 2024-03-28T17:51:10.990 INFO:teuthology.orchestra.run.smithi022.stdout:(122/174): python3-cmd2-2.4.2-2.el9.noarch.rpm 1.7 MB/s | 303 kB 00:00 2024-03-28T17:51:11.016 INFO:teuthology.orchestra.run.smithi022.stdout:(123/174): python3-colorama-0.4.4-12.el9.noarch 212 kB/s | 35 kB 00:00 2024-03-28T17:51:11.049 INFO:teuthology.orchestra.run.smithi022.stdout:(124/174): python3-google-auth-2.28.2-1.el9.noa 2.4 MB/s | 203 kB 00:00 2024-03-28T17:51:11.124 INFO:teuthology.orchestra.run.smithi022.stdout:(125/174): python3-grpcio-tools-1.46.7-10.el9.x 1.3 MB/s | 144 kB 00:00 2024-03-28T17:51:11.141 INFO:teuthology.orchestra.run.smithi022.stdout:(126/174): python3-jaraco-8.2.1-3.el9.noarch.rp 117 kB/s | 11 kB 00:00 2024-03-28T17:51:11.158 INFO:teuthology.orchestra.run.smithi022.stdout:(127/174): python3-jaraco-classes-3.2.1-5.el9.n 541 kB/s | 18 kB 00:00 2024-03-28T17:51:11.160 INFO:teuthology.orchestra.run.smithi079.stdout:(105/174): lua-devel-5.4.4-4.el9.x86_64.rpm 69 kB/s | 22 kB 00:00 2024-03-28T17:51:11.175 INFO:teuthology.orchestra.run.smithi022.stdout:(128/174): python3-jaraco-collections-3.0.0-8.e 709 kB/s | 23 kB 00:00 2024-03-28T17:51:11.200 INFO:teuthology.orchestra.run.smithi022.stdout:(129/174): python3-jaraco-functools-3.5.0-2.el9 473 kB/s | 19 kB 00:00 2024-03-28T17:51:11.220 INFO:teuthology.orchestra.run.smithi134.stdout:(101/174): qatlib-service-24.02.0-1.el9.x86_64. 54 kB/s | 36 kB 00:00 2024-03-28T17:51:11.226 INFO:teuthology.orchestra.run.smithi022.stdout:(130/174): python3-jaraco-text-3.2.0-6.el9.noar 397 kB/s | 20 kB 00:00 2024-03-28T17:51:11.268 INFO:teuthology.orchestra.run.smithi022.stdout:(131/174): python3-logutils-0.3.5-21.el9.noarch 1.1 MB/s | 46 kB 00:00 2024-03-28T17:51:11.268 INFO:teuthology.orchestra.run.smithi079.stdout:(106/174): abseil-cpp-20211102.0-3.el9.x86_64.r 1.8 MB/s | 548 kB 00:00 2024-03-28T17:51:11.329 INFO:teuthology.orchestra.run.smithi022.stdout:(132/174): python3-more-itertools-8.12.0-2.el9. 1.3 MB/s | 79 kB 00:00 2024-03-28T17:51:11.335 INFO:teuthology.orchestra.run.smithi079.stdout:(107/174): grpc-data-1.46.7-10.el9.noarch.rpm 294 kB/s | 19 kB 00:00 2024-03-28T17:51:11.377 INFO:teuthology.orchestra.run.smithi022.stdout:(133/174): python3-kubernetes-26.1.0-2.el9.noar 5.8 MB/s | 1.0 MB 00:00 2024-03-28T17:51:11.402 INFO:teuthology.orchestra.run.smithi022.stdout:(134/174): python3-natsort-7.1.1-5.el9.noarch.r 804 kB/s | 58 kB 00:00 2024-03-28T17:51:11.411 INFO:teuthology.orchestra.run.smithi079.stdout:(108/174): gperftools-libs-2.9.1-2.el9.x86_64.r 1.2 MB/s | 309 kB 00:00 2024-03-28T17:51:11.460 INFO:teuthology.orchestra.run.smithi022.stdout:(135/174): python3-grpcio-1.46.7-10.el9.x86_64. 4.3 MB/s | 2.0 MB 00:00 2024-03-28T17:51:11.469 INFO:teuthology.orchestra.run.smithi079.stdout:(109/174): libarrow-doc-9.0.0-10.el9.noarch.rpm 451 kB/s | 26 kB 00:00 2024-03-28T17:51:11.486 INFO:teuthology.orchestra.run.smithi022.stdout:(136/174): python3-portend-3.1.0-2.el9.noarch.r 198 kB/s | 16 kB 00:00 2024-03-28T17:51:11.519 INFO:teuthology.orchestra.run.smithi022.stdout:(137/174): python3-pecan-1.4.2-3.el9.noarch.rpm 1.9 MB/s | 272 kB 00:00 2024-03-28T17:51:11.528 INFO:teuthology.orchestra.run.smithi079.stdout:(110/174): liboath-2.6.7-2.el9.x86_64.rpm 845 kB/s | 49 kB 00:00 2024-03-28T17:51:11.544 INFO:teuthology.orchestra.run.smithi022.stdout:(138/174): python3-pyOpenSSL-21.0.0-1.el9.noarc 1.1 MB/s | 90 kB 00:00 2024-03-28T17:51:11.545 INFO:teuthology.orchestra.run.smithi134.stdout:(102/174): qatzip-libs-1.2.0-1.el9.x86_64.rpm 68 kB/s | 47 kB 00:00 2024-03-28T17:51:11.570 INFO:teuthology.orchestra.run.smithi022.stdout:(139/174): python3-pyperclip-1.8.2-2.el9.noarch 340 kB/s | 28 kB 00:00 2024-03-28T17:51:11.596 INFO:teuthology.orchestra.run.smithi022.stdout:(140/174): python3-repoze-lru-0.7-16.el9.noarch 406 kB/s | 31 kB 00:00 2024-03-28T17:51:11.620 INFO:teuthology.orchestra.run.smithi022.stdout:(141/174): python3-rsa-4.9-2.el9.noarch.rpm 1.2 MB/s | 59 kB 00:00 2024-03-28T17:51:11.628 INFO:teuthology.orchestra.run.smithi079.stdout:(111/174): libarrow-9.0.0-10.el9.x86_64.rpm 15 MB/s | 4.4 MB 00:00 2024-03-28T17:51:11.645 INFO:teuthology.orchestra.run.smithi022.stdout:(142/174): python3-tempora-5.0.0-2.el9.noarch.r 737 kB/s | 36 kB 00:00 2024-03-28T17:51:11.653 INFO:teuthology.orchestra.run.smithi079.stdout:(112/174): libunwind-1.6.2-1.el9.x86_64.rpm 540 kB/s | 67 kB 00:00 2024-03-28T17:51:11.671 INFO:teuthology.orchestra.run.smithi022.stdout:(143/174): python3-routes-2.5.1-5.el9.noarch.rp 1.5 MB/s | 188 kB 00:00 2024-03-28T17:51:11.696 INFO:teuthology.orchestra.run.smithi022.stdout:(144/174): python3-typing-extensions-4.4.0-2.el 684 kB/s | 51 kB 00:00 2024-03-28T17:51:11.704 INFO:teuthology.orchestra.run.smithi134.stdout:(103/174): lua-devel-5.4.4-4.el9.x86_64.rpm 142 kB/s | 22 kB 00:00 2024-03-28T17:51:11.704 INFO:teuthology.orchestra.run.smithi079.stdout:(113/174): protobuf-compiler-3.14.0-13.el9.x86_ 1.0 MB/s | 863 kB 00:00 2024-03-28T17:51:11.730 INFO:teuthology.orchestra.run.smithi022.stdout:(145/174): python3-webob-1.8.7-6.el9.noarch.rpm 2.7 MB/s | 230 kB 00:00 2024-03-28T17:51:11.746 INFO:teuthology.orchestra.run.smithi079.stdout:(114/174): luarocks-3.9.2-1.el9.noarch.rpm 1.3 MB/s | 151 kB 00:00 2024-03-28T17:51:11.773 INFO:teuthology.orchestra.run.smithi022.stdout:(146/174): python3-websocket-client-1.2.3-2.el9 1.2 MB/s | 90 kB 00:00 2024-03-28T17:51:11.796 INFO:teuthology.orchestra.run.smithi079.stdout:(115/174): parquet-libs-9.0.0-10.el9.x86_64.rpm 5.8 MB/s | 838 kB 00:00 2024-03-28T17:51:11.805 INFO:teuthology.orchestra.run.smithi022.stdout:(147/174): python3-zc-lockfile-2.0-10.el9.noarc 269 kB/s | 20 kB 00:00 2024-03-28T17:51:11.830 INFO:teuthology.orchestra.run.smithi079.stdout:(116/174): python3-bcrypt-3.2.2-1.el9.x86_64.rp 524 kB/s | 43 kB 00:00 2024-03-28T17:51:11.839 INFO:teuthology.orchestra.run.smithi022.stdout:(148/174): re2-20211101-3.el9.x86_64.rpm 2.9 MB/s | 192 kB 00:00 2024-03-28T17:51:11.863 INFO:teuthology.orchestra.run.smithi079.stdout:(117/174): python3-cachetools-4.2.4-1.el9.noarc 482 kB/s | 32 kB 00:00 2024-03-28T17:51:11.872 INFO:teuthology.orchestra.run.smithi022.stdout:(149/174): python3-werkzeug-2.0.3-3.el9.1.noarc 2.4 MB/s | 427 kB 00:00 2024-03-28T17:51:11.888 INFO:teuthology.orchestra.run.smithi079.stdout:(118/174): python3-certifi-2021.10.8-2.el9.noar 259 kB/s | 15 kB 00:00 2024-03-28T17:51:11.922 INFO:teuthology.orchestra.run.smithi079.stdout:(119/174): python3-cheroot-8.6.0-4.el9.noarch.r 2.9 MB/s | 172 kB 00:00 2024-03-28T17:51:11.954 INFO:teuthology.orchestra.run.smithi134.stdout:(104/174): protobuf-compiler-3.14.0-13.el9.x86_ 3.4 MB/s | 863 kB 00:00 2024-03-28T17:51:11.964 INFO:teuthology.orchestra.run.smithi079.stdout:(120/174): python3-cherrypy-18.6.1-2.el9.noarch 4.7 MB/s | 358 kB 00:00 2024-03-28T17:51:11.996 INFO:teuthology.orchestra.run.smithi134.stdout:(105/174): xfsprogs-devel-6.3.0-1.el9.x86_64.rp 121 kB/s | 94 kB 00:00 2024-03-28T17:51:12.006 INFO:teuthology.orchestra.run.smithi079.stdout:(121/174): python3-asyncssh-2.13.2-5.el9.noarch 1.8 MB/s | 548 kB 00:00 2024-03-28T17:51:12.056 INFO:teuthology.orchestra.run.smithi022.stdout:(150/174): thrift-0.15.0-2.el9.x86_64.rpm 6.3 MB/s | 1.6 MB 00:00 2024-03-28T17:51:12.069 INFO:teuthology.orchestra.run.smithi079.stdout:(122/174): python3-cmd2-2.4.2-2.el9.noarch.rpm 2.0 MB/s | 303 kB 00:00 2024-03-28T17:51:12.098 INFO:teuthology.orchestra.run.smithi079.stdout:(123/174): python3-colorama-0.4.4-12.el9.noarch 265 kB/s | 35 kB 00:00 2024-03-28T17:51:12.132 INFO:teuthology.orchestra.run.smithi079.stdout:(124/174): python3-google-auth-2.28.2-1.el9.noa 1.6 MB/s | 203 kB 00:00 2024-03-28T17:51:12.165 INFO:teuthology.orchestra.run.smithi079.stdout:(125/174): python3-grpcio-tools-1.46.7-10.el9.x 2.1 MB/s | 144 kB 00:00 2024-03-28T17:51:12.199 INFO:teuthology.orchestra.run.smithi079.stdout:(126/174): python3-jaraco-8.2.1-3.el9.noarch.rp 161 kB/s | 11 kB 00:00 2024-03-28T17:51:12.224 INFO:teuthology.orchestra.run.smithi079.stdout:(127/174): python3-jaraco-classes-3.2.1-5.el9.n 303 kB/s | 18 kB 00:00 2024-03-28T17:51:12.240 INFO:teuthology.orchestra.run.smithi022.stdout:(151/174): xfsdump-3.1.12-4.el9.x86_64.rpm 1.7 MB/s | 321 kB 00:00 2024-03-28T17:51:12.258 INFO:teuthology.orchestra.run.smithi079.stdout:(128/174): python3-jaraco-collections-3.0.0-8.e 400 kB/s | 23 kB 00:00 2024-03-28T17:51:12.283 INFO:teuthology.orchestra.run.smithi079.stdout:(129/174): python3-jaraco-functools-3.5.0-2.el9 335 kB/s | 19 kB 00:00 2024-03-28T17:51:12.325 INFO:teuthology.orchestra.run.smithi079.stdout:(130/174): python3-jaraco-text-3.2.0-6.el9.noar 296 kB/s | 20 kB 00:00 2024-03-28T17:51:12.393 INFO:teuthology.orchestra.run.smithi079.stdout:(131/174): python3-grpcio-1.46.7-10.el9.x86_64. 6.3 MB/s | 2.0 MB 00:00 2024-03-28T17:51:12.418 INFO:teuthology.orchestra.run.smithi079.stdout:(132/174): python3-logutils-0.3.5-21.el9.noarch 509 kB/s | 46 kB 00:00 2024-03-28T17:51:12.468 INFO:teuthology.orchestra.run.smithi079.stdout:(133/174): python3-kubernetes-26.1.0-2.el9.noar 5.6 MB/s | 1.0 MB 00:00 2024-03-28T17:51:12.485 INFO:teuthology.orchestra.run.smithi079.stdout:(134/174): python3-natsort-7.1.1-5.el9.noarch.r 871 kB/s | 58 kB 00:00 2024-03-28T17:51:12.490 INFO:teuthology.orchestra.run.smithi022.stdout:(152/174): librbd1-19.0.0-2145.g3be9a5c9.el9.x8 5.1 MB/s | 3.1 MB 00:00 2024-03-28T17:51:12.510 INFO:teuthology.orchestra.run.smithi079.stdout:(135/174): python3-more-itertools-8.12.0-2.el9. 676 kB/s | 79 kB 00:00 2024-03-28T17:51:12.521 INFO:teuthology.orchestra.run.smithi134.stdout:(106/174): gperftools-libs-2.9.1-2.el9.x86_64.r 590 kB/s | 309 kB 00:00 2024-03-28T17:51:12.544 INFO:teuthology.orchestra.run.smithi079.stdout:(136/174): python3-pecan-1.4.2-3.el9.noarch.rpm 3.6 MB/s | 272 kB 00:00 2024-03-28T17:51:12.561 INFO:teuthology.orchestra.run.smithi079.stdout:(137/174): python3-portend-3.1.0-2.el9.noarch.r 219 kB/s | 16 kB 00:00 2024-03-28T17:51:12.580 INFO:teuthology.orchestra.run.smithi134.stdout:(107/174): abseil-cpp-20211102.0-3.el9.x86_64.r 878 kB/s | 548 kB 00:00 2024-03-28T17:51:12.586 INFO:teuthology.orchestra.run.smithi079.stdout:(138/174): python3-pyOpenSSL-21.0.0-1.el9.noarc 1.2 MB/s | 90 kB 00:00 2024-03-28T17:51:12.599 INFO:teuthology.orchestra.run.smithi022.stdout:(153/174): librados2-19.0.0-2145.g3be9a5c9.el9. 4.5 MB/s | 3.4 MB 00:00 2024-03-28T17:51:12.611 INFO:teuthology.orchestra.run.smithi079.stdout:(139/174): python3-pyperclip-1.8.2-2.el9.noarch 424 kB/s | 28 kB 00:00 2024-03-28T17:51:12.636 INFO:teuthology.orchestra.run.smithi079.stdout:(140/174): python3-repoze-lru-0.7-16.el9.noarch 410 kB/s | 31 kB 00:00 2024-03-28T17:51:12.670 INFO:teuthology.orchestra.run.smithi079.stdout:(141/174): python3-routes-2.5.1-5.el9.noarch.rp 2.2 MB/s | 188 kB 00:00 2024-03-28T17:51:12.695 INFO:teuthology.orchestra.run.smithi079.stdout:(142/174): python3-rsa-4.9-2.el9.noarch.rpm 711 kB/s | 59 kB 00:00 2024-03-28T17:51:12.699 INFO:teuthology.orchestra.run.smithi022.stdout:(154/174): NetworkManager-1.47.2-1.el9.x86_64.r 5.0 MB/s | 2.3 MB 00:00 2024-03-28T17:51:12.721 INFO:teuthology.orchestra.run.smithi079.stdout:(143/174): python3-tempora-5.0.0-2.el9.noarch.r 430 kB/s | 36 kB 00:00 2024-03-28T17:51:12.746 INFO:teuthology.orchestra.run.smithi079.stdout:(144/174): python3-typing-extensions-4.4.0-2.el 681 kB/s | 51 kB 00:00 2024-03-28T17:51:12.757 INFO:teuthology.orchestra.run.smithi022.stdout:(155/174): NetworkManager-tui-1.47.2-1.el9.x86_ 4.2 MB/s | 248 kB 00:00 2024-03-28T17:51:12.778 INFO:teuthology.orchestra.run.smithi079.stdout:(145/174): python3-webob-1.8.7-6.el9.noarch.rpm 2.8 MB/s | 230 kB 00:00 2024-03-28T17:51:12.792 INFO:teuthology.orchestra.run.smithi022.stdout:(156/174): NetworkManager-team-1.47.2-1.el9.x86 219 kB/s | 42 kB 00:00 2024-03-28T17:51:12.809 INFO:teuthology.orchestra.run.smithi022.stdout:(157/174): acl-2.3.1-4.el9.x86_64.rpm 1.4 MB/s | 71 kB 00:00 2024-03-28T17:51:12.811 INFO:teuthology.orchestra.run.smithi079.stdout:(146/174): python3-websocket-client-1.2.3-2.el9 999 kB/s | 90 kB 00:00 2024-03-28T17:51:12.830 INFO:teuthology.orchestra.run.smithi079.stdout:(147/174): python3-zc-lockfile-2.0-10.el9.noarc 409 kB/s | 20 kB 00:00 2024-03-28T17:51:12.855 INFO:teuthology.orchestra.run.smithi134.stdout:(108/174): grpc-data-1.46.7-10.el9.noarch.rpm 58 kB/s | 19 kB 00:00 2024-03-28T17:51:12.867 INFO:teuthology.orchestra.run.smithi022.stdout:(158/174): elfutils-libelf-0.190-2.el9.x86_64.r 3.3 MB/s | 192 kB 00:00 2024-03-28T17:51:12.870 INFO:teuthology.orchestra.run.smithi079.stdout:(148/174): python3-werkzeug-2.0.3-3.el9.1.noarc 3.4 MB/s | 427 kB 00:00 2024-03-28T17:51:12.893 INFO:teuthology.orchestra.run.smithi022.stdout:(159/174): elfutils-debuginfod-client-0.190-2.e 365 kB/s | 36 kB 00:00 2024-03-28T17:51:12.904 INFO:teuthology.orchestra.run.smithi079.stdout:(149/174): re2-20211101-3.el9.x86_64.rpm 2.0 MB/s | 192 kB 00:00 2024-03-28T17:51:12.927 INFO:teuthology.orchestra.run.smithi022.stdout:(160/174): elfutils-libs-0.190-2.el9.x86_64.rpm 4.3 MB/s | 256 kB 00:00 2024-03-28T17:51:12.944 INFO:teuthology.orchestra.run.smithi022.stdout:(161/174): libacl-2.3.1-4.el9.x86_64.rpm 457 kB/s | 23 kB 00:00 2024-03-28T17:51:12.972 INFO:teuthology.orchestra.run.smithi079.stdout:(150/174): thrift-0.15.0-2.el9.x86_64.rpm 11 MB/s | 1.6 MB 00:00 2024-03-28T17:51:13.003 INFO:teuthology.orchestra.run.smithi022.stdout:(162/174): NetworkManager-libnm-1.47.2-1.el9.x8 3.6 MB/s | 1.8 MB 00:00 2024-03-28T17:51:13.022 INFO:teuthology.orchestra.run.smithi079.stdout:(151/174): xfsdump-3.1.12-4.el9.x86_64.rpm 6.4 MB/s | 321 kB 00:00 2024-03-28T17:51:13.028 INFO:teuthology.orchestra.run.smithi022.stdout:(163/174): libblkid-2.37.4-18.el9.x86_64.rpm 1.1 MB/s | 108 kB 00:00 2024-03-28T17:51:13.061 INFO:teuthology.orchestra.run.smithi022.stdout:(164/174): libfdisk-2.37.4-18.el9.x86_64.rpm 1.3 MB/s | 155 kB 00:00 2024-03-28T17:51:13.113 INFO:teuthology.orchestra.run.smithi022.stdout:(165/174): libsmartcols-2.37.4-18.el9.x86_64.rp 761 kB/s | 64 kB 00:00 2024-03-28T17:51:13.137 INFO:teuthology.orchestra.run.smithi022.stdout:(166/174): libuuid-2.37.4-18.el9.x86_64.rpm 384 kB/s | 29 kB 00:00 2024-03-28T17:51:13.162 INFO:teuthology.orchestra.run.smithi022.stdout:(167/174): libmount-2.37.4-18.el9.x86_64.rpm 856 kB/s | 136 kB 00:00 2024-03-28T17:51:13.163 INFO:teuthology.orchestra.run.smithi134.stdout:(109/174): libarrow-doc-9.0.0-10.el9.noarch.rpm 84 kB/s | 26 kB 00:00 2024-03-28T17:51:13.222 INFO:teuthology.orchestra.run.smithi079.stdout:(152/174): NetworkManager-1.47.2-1.el9.x86_64.r 12 MB/s | 2.3 MB 00:00 2024-03-28T17:51:13.329 INFO:teuthology.orchestra.run.smithi022.stdout:(168/174): openssl-3.0.7-27.el9.x86_64.rpm 5.6 MB/s | 1.2 MB 00:00 2024-03-28T17:51:13.439 INFO:teuthology.orchestra.run.smithi079.stdout:(153/174): librbd1-19.0.0-2145.g3be9a5c9.el9.x8 5.9 MB/s | 3.1 MB 00:00 2024-03-28T17:51:13.615 INFO:teuthology.orchestra.run.smithi079.stdout:(154/174): NetworkManager-libnm-1.47.2-1.el9.x8 4.7 MB/s | 1.8 MB 00:00 2024-03-28T17:51:13.629 INFO:teuthology.orchestra.run.smithi022.stdout:(169/174): util-linux-core-2.37.4-18.el9.x86_64 1.5 MB/s | 465 kB 00:00 2024-03-28T17:51:13.640 INFO:teuthology.orchestra.run.smithi079.stdout:(155/174): NetworkManager-team-1.47.2-1.el9.x86 211 kB/s | 42 kB 00:00 2024-03-28T17:51:13.698 INFO:teuthology.orchestra.run.smithi134.stdout:(110/174): liboath-2.6.7-2.el9.x86_64.rpm 91 kB/s | 49 kB 00:00 2024-03-28T17:51:13.698 INFO:teuthology.orchestra.run.smithi079.stdout:(156/174): acl-2.3.1-4.el9.x86_64.rpm 1.2 MB/s | 71 kB 00:00 2024-03-28T17:51:13.705 INFO:teuthology.orchestra.run.smithi022.stdout:(170/174): openssl-libs-3.0.7-27.el9.x86_64.rpm 3.8 MB/s | 2.2 MB 00:00 2024-03-28T17:51:13.730 INFO:teuthology.orchestra.run.smithi022.stdout:(171/174): util-linux-user-2.37.4-18.el9.x86_64 311 kB/s | 31 kB 00:00 2024-03-28T17:51:13.732 INFO:teuthology.orchestra.run.smithi079.stdout:(157/174): NetworkManager-tui-1.47.2-1.el9.x86_ 2.1 MB/s | 248 kB 00:00 2024-03-28T17:51:13.782 INFO:teuthology.orchestra.run.smithi079.stdout:(158/174): elfutils-debuginfod-client-0.190-2.e 441 kB/s | 36 kB 00:00 2024-03-28T17:51:13.805 INFO:teuthology.orchestra.run.smithi022.stdout:(172/174): util-linux-2.37.4-18.el9.x86_64.rpm 3.6 MB/s | 2.3 MB 00:00 2024-03-28T17:51:13.832 INFO:teuthology.orchestra.run.smithi079.stdout:(159/174): elfutils-libelf-0.190-2.el9.x86_64.r 1.9 MB/s | 192 kB 00:00 2024-03-28T17:51:13.883 INFO:teuthology.orchestra.run.smithi079.stdout:(160/174): elfutils-libs-0.190-2.el9.x86_64.rpm 2.5 MB/s | 256 kB 00:00 2024-03-28T17:51:13.888 INFO:teuthology.orchestra.run.smithi022.stdout:(173/174): xfsprogs-6.3.0-1.el9.x86_64.rpm 5.9 MB/s | 1.1 MB 00:00 2024-03-28T17:51:13.908 INFO:teuthology.orchestra.run.smithi079.stdout:(161/174): libacl-2.3.1-4.el9.x86_64.rpm 310 kB/s | 23 kB 00:00 2024-03-28T17:51:13.913 INFO:teuthology.orchestra.run.smithi022.stdout:(174/174): libacl-devel-2.3.1-4.el9.x86_64.rpm 406 kB/s | 74 kB 00:00 2024-03-28T17:51:13.919 INFO:teuthology.orchestra.run.smithi134.stdout:(111/174): libarrow-9.0.0-10.el9.x86_64.rpm 3.3 MB/s | 4.4 MB 00:01 2024-03-28T17:51:13.922 INFO:teuthology.orchestra.run.smithi022.stdout:-------------------------------------------------------------------------------- 2024-03-28T17:51:13.922 INFO:teuthology.orchestra.run.smithi022.stdout:Total 11 MB/s | 225 MB 00:21 2024-03-28T17:51:13.967 INFO:teuthology.orchestra.run.smithi079.stdout:(162/174): libblkid-2.37.4-18.el9.x86_64.rpm 1.3 MB/s | 108 kB 00:00 2024-03-28T17:51:14.000 INFO:teuthology.orchestra.run.smithi079.stdout:(163/174): libfdisk-2.37.4-18.el9.x86_64.rpm 1.7 MB/s | 155 kB 00:00 2024-03-28T17:51:14.048 INFO:teuthology.orchestra.run.smithi134.stdout:(112/174): libunwind-1.6.2-1.el9.x86_64.rpm 193 kB/s | 67 kB 00:00 2024-03-28T17:51:14.084 INFO:teuthology.orchestra.run.smithi079.stdout:(164/174): librados2-19.0.0-2145.g3be9a5c9.el9. 2.8 MB/s | 3.4 MB 00:01 2024-03-28T17:51:14.134 INFO:teuthology.orchestra.run.smithi079.stdout:(165/174): libmount-2.37.4-18.el9.x86_64.rpm 816 kB/s | 136 kB 00:00 2024-03-28T17:51:14.160 INFO:teuthology.orchestra.run.smithi079.stdout:(166/174): libuuid-2.37.4-18.el9.x86_64.rpm 386 kB/s | 29 kB 00:00 2024-03-28T17:51:14.185 INFO:teuthology.orchestra.run.smithi079.stdout:(167/174): libsmartcols-2.37.4-18.el9.x86_64.rp 349 kB/s | 64 kB 00:00 2024-03-28T17:51:14.326 INFO:teuthology.orchestra.run.smithi079.stdout:(168/174): openssl-3.0.7-27.el9.x86_64.rpm 6.3 MB/s | 1.2 MB 00:00 2024-03-28T17:51:14.390 INFO:teuthology.orchestra.run.smithi134.stdout:(113/174): luarocks-3.9.2-1.el9.noarch.rpm 322 kB/s | 151 kB 00:00 2024-03-28T17:51:14.460 INFO:teuthology.orchestra.run.smithi079.stdout:(169/174): openssl-libs-3.0.7-27.el9.x86_64.rpm 7.2 MB/s | 2.2 MB 00:00 2024-03-28T17:51:14.527 INFO:teuthology.orchestra.run.smithi079.stdout:(170/174): util-linux-2.37.4-18.el9.x86_64.rpm 6.7 MB/s | 2.3 MB 00:00 2024-03-28T17:51:14.569 INFO:teuthology.orchestra.run.smithi079.stdout:(171/174): util-linux-core-2.37.4-18.el9.x86_64 1.9 MB/s | 465 kB 00:00 2024-03-28T17:51:14.594 INFO:teuthology.orchestra.run.smithi079.stdout:(172/174): util-linux-user-2.37.4-18.el9.x86_64 234 kB/s | 31 kB 00:00 2024-03-28T17:51:14.703 INFO:teuthology.orchestra.run.smithi079.stdout:(173/174): xfsprogs-6.3.0-1.el9.x86_64.rpm 6.1 MB/s | 1.1 MB 00:00 2024-03-28T17:51:14.728 INFO:teuthology.orchestra.run.smithi079.stdout:(174/174): libacl-devel-2.3.1-4.el9.x86_64.rpm 471 kB/s | 74 kB 00:00 2024-03-28T17:51:14.740 INFO:teuthology.orchestra.run.smithi079.stdout:-------------------------------------------------------------------------------- 2024-03-28T17:51:14.741 INFO:teuthology.orchestra.run.smithi079.stdout:Total 11 MB/s | 225 MB 00:19 2024-03-28T17:51:14.790 INFO:teuthology.orchestra.run.smithi134.stdout:(114/174): parquet-libs-9.0.0-10.el9.x86_64.rpm 1.1 MB/s | 838 kB 00:00 2024-03-28T17:51:15.023 INFO:teuthology.orchestra.run.smithi134.stdout:(115/174): python3-asyncssh-2.13.2-5.el9.noarch 867 kB/s | 548 kB 00:00 2024-03-28T17:51:15.199 INFO:teuthology.orchestra.run.smithi134.stdout:(116/174): python3-bcrypt-3.2.2-1.el9.x86_64.rp 106 kB/s | 43 kB 00:00 2024-03-28T17:51:15.299 INFO:teuthology.orchestra.run.smithi134.stdout:(117/174): python3-cachetools-4.2.4-1.el9.noarc 117 kB/s | 32 kB 00:00 2024-03-28T17:51:15.516 INFO:teuthology.orchestra.run.smithi134.stdout:(118/174): python3-certifi-2021.10.8-2.el9.noar 47 kB/s | 15 kB 00:00 2024-03-28T17:51:15.652 INFO:teuthology.orchestra.run.smithi022.stdout:Running transaction check 2024-03-28T17:51:15.775 INFO:teuthology.orchestra.run.smithi134.stdout:(119/174): python3-cheroot-8.6.0-4.el9.noarch.r 362 kB/s | 172 kB 00:00 2024-03-28T17:51:16.055 INFO:teuthology.orchestra.run.smithi022.stdout:Transaction check succeeded. 2024-03-28T17:51:16.055 INFO:teuthology.orchestra.run.smithi022.stdout:Running transaction test 2024-03-28T17:51:16.092 INFO:teuthology.orchestra.run.smithi134.stdout:(120/174): python3-cherrypy-18.6.1-2.el9.noarch 624 kB/s | 358 kB 00:00 2024-03-28T17:51:16.326 INFO:teuthology.orchestra.run.smithi134.stdout:(121/174): postgresql-server-13.14-1.el9.x86_64 735 kB/s | 5.8 MB 00:08 2024-03-28T17:51:16.359 INFO:teuthology.orchestra.run.smithi134.stdout:(122/174): python3-colorama-0.4.4-12.el9.noarch 132 kB/s | 35 kB 00:00 2024-03-28T17:51:16.386 INFO:teuthology.orchestra.run.smithi079.stdout:Running transaction check 2024-03-28T17:51:16.443 INFO:teuthology.orchestra.run.smithi134.stdout:(123/174): python3-cmd2-2.4.2-2.el9.noarch.rpm 455 kB/s | 303 kB 00:00 2024-03-28T17:51:16.762 INFO:teuthology.orchestra.run.smithi079.stdout:Transaction check succeeded. 2024-03-28T17:51:16.763 INFO:teuthology.orchestra.run.smithi079.stdout:Running transaction test 2024-03-28T17:51:16.876 INFO:teuthology.orchestra.run.smithi134.stdout:(124/174): python3-google-auth-2.28.2-1.el9.noa 369 kB/s | 203 kB 00:00 2024-03-28T17:51:16.902 INFO:teuthology.orchestra.run.smithi134.stdout:(125/174): python3-grpcio-tools-1.46.7-10.el9.x 315 kB/s | 144 kB 00:00 2024-03-28T17:51:17.110 INFO:teuthology.orchestra.run.smithi134.stdout:(126/174): python3-jaraco-8.2.1-3.el9.noarch.rp 46 kB/s | 11 kB 00:00 2024-03-28T17:51:17.211 INFO:teuthology.orchestra.run.smithi134.stdout:(127/174): python3-jaraco-classes-3.2.1-5.el9.n 57 kB/s | 18 kB 00:00 2024-03-28T17:51:17.303 INFO:teuthology.orchestra.run.smithi134.stdout:(128/174): python3-grpcio-1.46.7-10.el9.x86_64. 2.2 MB/s | 2.0 MB 00:00 2024-03-28T17:51:17.386 INFO:teuthology.orchestra.run.smithi134.stdout:(129/174): python3-jaraco-collections-3.0.0-8.e 84 kB/s | 23 kB 00:00 2024-03-28T17:51:17.503 INFO:teuthology.orchestra.run.smithi134.stdout:(130/174): python3-jaraco-functools-3.5.0-2.el9 67 kB/s | 19 kB 00:00 2024-03-28T17:51:17.595 INFO:teuthology.orchestra.run.smithi134.stdout:(131/174): python3-jaraco-text-3.2.0-6.el9.noar 67 kB/s | 20 kB 00:00 2024-03-28T17:51:17.887 INFO:teuthology.orchestra.run.smithi134.stdout:(132/174): python3-logutils-0.3.5-21.el9.noarch 121 kB/s | 46 kB 00:00 2024-03-28T17:51:18.054 INFO:teuthology.orchestra.run.smithi134.stdout:(133/174): python3-more-itertools-8.12.0-2.el9. 172 kB/s | 79 kB 00:00 2024-03-28T17:51:18.113 INFO:teuthology.orchestra.run.smithi134.stdout:(134/174): python3-kubernetes-26.1.0-2.el9.noar 1.4 MB/s | 1.0 MB 00:00 2024-03-28T17:51:18.271 INFO:teuthology.orchestra.run.smithi134.stdout:(135/174): python3-natsort-7.1.1-5.el9.noarch.r 151 kB/s | 58 kB 00:00 2024-03-28T17:51:18.398 INFO:teuthology.orchestra.run.smithi134.stdout:(136/174): python3-portend-3.1.0-2.el9.noarch.r 58 kB/s | 16 kB 00:00 2024-03-28T17:51:18.528 INFO:teuthology.orchestra.run.smithi022.stdout:Transaction test succeeded. 2024-03-28T17:51:18.529 INFO:teuthology.orchestra.run.smithi022.stdout:Running transaction 2024-03-28T17:51:18.599 INFO:teuthology.orchestra.run.smithi134.stdout:(137/174): python3-pecan-1.4.2-3.el9.noarch.rpm 501 kB/s | 272 kB 00:00 2024-03-28T17:51:18.624 INFO:teuthology.orchestra.run.smithi134.stdout:(138/174): python3-pyOpenSSL-21.0.0-1.el9.noarc 256 kB/s | 90 kB 00:00 2024-03-28T17:51:18.808 INFO:teuthology.orchestra.run.smithi134.stdout:(139/174): python3-pyperclip-1.8.2-2.el9.noarch 69 kB/s | 28 kB 00:00 2024-03-28T17:51:18.925 INFO:teuthology.orchestra.run.smithi134.stdout:(140/174): python3-repoze-lru-0.7-16.el9.noarch 94 kB/s | 31 kB 00:00 2024-03-28T17:51:19.192 INFO:teuthology.orchestra.run.smithi134.stdout:(141/174): python3-rsa-4.9-2.el9.noarch.rpm 154 kB/s | 59 kB 00:00 2024-03-28T17:51:19.217 INFO:teuthology.orchestra.run.smithi134.stdout:(142/174): python3-routes-2.5.1-5.el9.noarch.rp 318 kB/s | 188 kB 00:00 2024-03-28T17:51:19.229 INFO:teuthology.orchestra.run.smithi079.stdout:Transaction test succeeded. 2024-03-28T17:51:19.230 INFO:teuthology.orchestra.run.smithi079.stdout:Running transaction 2024-03-28T17:51:19.242 INFO:teuthology.orchestra.run.smithi134.stdout:(143/174): python3-tempora-5.0.0-2.el9.noarch.r 113 kB/s | 36 kB 00:00 2024-03-28T17:51:19.584 INFO:teuthology.orchestra.run.smithi134.stdout:(144/174): python3-typing-extensions-4.4.0-2.el 130 kB/s | 51 kB 00:00 2024-03-28T17:51:19.668 INFO:teuthology.orchestra.run.smithi134.stdout:(145/174): python3-websocket-client-1.2.3-2.el9 211 kB/s | 90 kB 00:00 2024-03-28T17:51:19.776 INFO:teuthology.orchestra.run.smithi134.stdout:(146/174): python3-webob-1.8.7-6.el9.noarch.rpm 411 kB/s | 230 kB 00:00 2024-03-28T17:51:19.968 INFO:teuthology.orchestra.run.smithi134.stdout:(147/174): python3-zc-lockfile-2.0-10.el9.noarc 67 kB/s | 20 kB 00:00 2024-03-28T17:51:20.210 INFO:teuthology.orchestra.run.smithi134.stdout:(148/174): python3-werkzeug-2.0.3-3.el9.1.noarc 684 kB/s | 427 kB 00:00 2024-03-28T17:51:20.286 INFO:teuthology.orchestra.run.smithi134.stdout:(149/174): re2-20211101-3.el9.x86_64.rpm 377 kB/s | 192 kB 00:00 2024-03-28T17:51:20.628 INFO:teuthology.orchestra.run.smithi134.stdout:(150/174): librbd1-19.0.0-2145.g3be9a5c9.el9.x8 9.2 MB/s | 3.1 MB 00:00 2024-03-28T17:51:20.670 INFO:teuthology.orchestra.run.smithi134.stdout:(151/174): xfsdump-3.1.12-4.el9.x86_64.rpm 7.7 MB/s | 321 kB 00:00 2024-03-28T17:51:20.936 INFO:teuthology.orchestra.run.smithi134.stdout:(152/174): librados2-19.0.0-2145.g3be9a5c9.el9. 4.7 MB/s | 3.4 MB 00:00 2024-03-28T17:51:21.053 INFO:teuthology.orchestra.run.smithi134.stdout:(153/174): thrift-0.15.0-2.el9.x86_64.rpm 1.5 MB/s | 1.6 MB 00:01 2024-03-28T17:51:21.720 INFO:teuthology.orchestra.run.smithi134.stdout:(154/174): NetworkManager-team-1.47.2-1.el9.x86 63 kB/s | 42 kB 00:00 2024-03-28T17:51:22.686 INFO:teuthology.orchestra.run.smithi134.stdout:(155/174): NetworkManager-tui-1.47.2-1.el9.x86_ 258 kB/s | 248 kB 00:00 2024-03-28T17:51:23.421 INFO:teuthology.orchestra.run.smithi134.stdout:(156/174): acl-2.3.1-4.el9.x86_64.rpm 97 kB/s | 71 kB 00:00 2024-03-28T17:51:24.030 INFO:teuthology.orchestra.run.smithi134.stdout:(157/174): NetworkManager-libnm-1.47.2-1.el9.x8 620 kB/s | 1.8 MB 00:03 2024-03-28T17:51:24.105 INFO:teuthology.orchestra.run.smithi134.stdout:(158/174): elfutils-debuginfod-client-0.190-2.e 53 kB/s | 36 kB 00:00 2024-03-28T17:51:24.430 INFO:teuthology.orchestra.run.smithi134.stdout:(159/174): NetworkManager-1.47.2-1.el9.x86_64.r 628 kB/s | 2.3 MB 00:03 2024-03-28T17:51:24.981 INFO:teuthology.orchestra.run.smithi134.stdout:(160/174): elfutils-libelf-0.190-2.el9.x86_64.r 203 kB/s | 192 kB 00:00 2024-03-28T17:51:25.047 INFO:teuthology.orchestra.run.smithi134.stdout:(161/174): libacl-2.3.1-4.el9.x86_64.rpm 37 kB/s | 23 kB 00:00 2024-03-28T17:51:25.115 INFO:teuthology.orchestra.run.smithi134.stdout:(162/174): elfutils-libs-0.190-2.el9.x86_64.rpm 253 kB/s | 256 kB 00:01 2024-03-28T17:51:25.798 INFO:teuthology.orchestra.run.smithi134.stdout:(163/174): libblkid-2.37.4-18.el9.x86_64.rpm 133 kB/s | 108 kB 00:00 2024-03-28T17:51:25.840 INFO:teuthology.orchestra.run.smithi134.stdout:(164/174): libfdisk-2.37.4-18.el9.x86_64.rpm 196 kB/s | 155 kB 00:00 2024-03-28T17:51:26.007 INFO:teuthology.orchestra.run.smithi134.stdout:(165/174): libmount-2.37.4-18.el9.x86_64.rpm 152 kB/s | 136 kB 00:00 2024-03-28T17:51:26.181 INFO:teuthology.orchestra.run.smithi022.stdout: Preparing : 1/1 2024-03-28T17:51:26.262 INFO:teuthology.orchestra.run.smithi022.stdout: Upgrading : openssl-libs-1:3.0.7-27.el9.x86_64 1/198 2024-03-28T17:51:26.310 INFO:teuthology.orchestra.run.smithi022.stdout: Upgrading : libuuid-2.37.4-18.el9.x86_64 2/198 2024-03-28T17:51:26.349 INFO:teuthology.orchestra.run.smithi022.stdout: Upgrading : libblkid-2.37.4-18.el9.x86_64 3/198 2024-03-28T17:51:26.390 INFO:teuthology.orchestra.run.smithi022.stdout: Running scriptlet: libblkid-2.37.4-18.el9.x86_64 3/198 2024-03-28T17:51:26.444 INFO:teuthology.orchestra.run.smithi079.stdout: Preparing : 1/1 2024-03-28T17:51:26.475 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 4/198 2024-03-28T17:51:26.532 INFO:teuthology.orchestra.run.smithi134.stdout:(166/174): libsmartcols-2.37.4-18.el9.x86_64.rp 87 kB/s | 64 kB 00:00 2024-03-28T17:51:26.558 INFO:teuthology.orchestra.run.smithi134.stdout:(167/174): libuuid-2.37.4-18.el9.x86_64.rpm 40 kB/s | 29 kB 00:00 2024-03-28T17:51:26.560 INFO:teuthology.orchestra.run.smithi079.stdout: Upgrading : openssl-libs-1:3.0.7-27.el9.x86_64 1/198 2024-03-28T17:51:26.572 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 5/198 2024-03-28T17:51:26.624 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : thrift-0.15.0-2.el9.x86_64 6/198 2024-03-28T17:51:26.638 INFO:teuthology.orchestra.run.smithi079.stdout: Upgrading : libuuid-2.37.4-18.el9.x86_64 2/198 2024-03-28T17:51:26.656 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 7/198 2024-03-28T17:51:26.830 INFO:teuthology.orchestra.run.smithi079.stdout: Upgrading : libblkid-2.37.4-18.el9.x86_64 3/198 2024-03-28T17:51:26.833 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 8/198 2024-03-28T17:51:26.876 INFO:teuthology.orchestra.run.smithi079.stdout: Running scriptlet: libblkid-2.37.4-18.el9.x86_64 3/198 2024-03-28T17:51:26.969 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 4/198 2024-03-28T17:51:27.064 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 5/198 2024-03-28T17:51:27.121 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : thrift-0.15.0-2.el9.x86_64 6/198 2024-03-28T17:51:27.153 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 7/198 2024-03-28T17:51:27.231 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 8/198 2024-03-28T17:51:27.259 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 9/198 2024-03-28T17:51:27.304 INFO:teuthology.orchestra.run.smithi022.stdout: Upgrading : librados2-2:19.0.0-2145.g3be9a5c9.el9.x86_64 10/198 2024-03-28T17:51:27.463 INFO:teuthology.orchestra.run.smithi022.stdout: Running scriptlet: librados2-2:19.0.0-2145.g3be9a5c9.el9.x86_64 10/198 2024-03-28T17:51:27.496 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : libcephfs2-2:19.0.0-2145.g3be9a5c9.el9.x86_64 11/198 2024-03-28T17:51:27.636 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 9/198 2024-03-28T17:51:27.671 INFO:teuthology.orchestra.run.smithi022.stdout: Running scriptlet: libcephfs2-2:19.0.0-2145.g3be9a5c9.el9.x86_64 11/198 2024-03-28T17:51:27.678 INFO:teuthology.orchestra.run.smithi079.stdout: Upgrading : librados2-2:19.0.0-2145.g3be9a5c9.el9.x86_64 10/198 2024-03-28T17:51:27.704 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : python3-rados-2:19.0.0-2145.g3be9a5c9.el9.x86_64 12/198 2024-03-28T17:51:27.771 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : postgresql-private-libs-13.14-1.el9.x86_64 13/198 2024-03-28T17:51:27.811 INFO:teuthology.orchestra.run.smithi079.stdout: Running scriptlet: librados2-2:19.0.0-2145.g3be9a5c9.el9.x86_64 10/198 2024-03-28T17:51:27.846 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : libcephfs2-2:19.0.0-2145.g3be9a5c9.el9.x86_64 11/198 2024-03-28T17:51:27.968 INFO:teuthology.orchestra.run.smithi079.stdout: Running scriptlet: libcephfs2-2:19.0.0-2145.g3be9a5c9.el9.x86_64 11/198 2024-03-28T17:51:28.003 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : python3-rados-2:19.0.0-2145.g3be9a5c9.el9.x86_64 12/198 2024-03-28T17:51:28.069 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : postgresql-private-libs-13.14-1.el9.x86_64 13/198 2024-03-28T17:51:28.100 INFO:teuthology.orchestra.run.smithi022.stdout: Upgrading : elfutils-libelf-0.190-2.el9.x86_64 14/198 2024-03-28T17:51:28.160 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : postgresql-13.14-1.el9.x86_64 15/198 2024-03-28T17:51:28.185 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : libcephsqlite-2:19.0.0-2145.g3be9a5c9.el9.x86_64 16/198 2024-03-28T17:51:28.216 INFO:teuthology.orchestra.run.smithi134.stdout:(168/174): openssl-3.0.7-27.el9.x86_64.rpm 561 kB/s | 1.2 MB 00:02 2024-03-28T17:51:28.368 INFO:teuthology.orchestra.run.smithi022.stdout: Running scriptlet: libcephsqlite-2:19.0.0-2145.g3be9a5c9.el9.x86_64 16/198 2024-03-28T17:51:28.369 INFO:teuthology.orchestra.run.smithi079.stdout: Upgrading : elfutils-libelf-0.190-2.el9.x86_64 14/198 2024-03-28T17:51:28.403 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : libradosstriper1-2:19.0.0-2145.g3be9a5c9.el9.x86 17/198 2024-03-28T17:51:28.427 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : postgresql-13.14-1.el9.x86_64 15/198 2024-03-28T17:51:28.452 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : libcephsqlite-2:19.0.0-2145.g3be9a5c9.el9.x86_64 16/198 2024-03-28T17:51:28.494 INFO:teuthology.orchestra.run.smithi022.stdout: Running scriptlet: libradosstriper1-2:19.0.0-2145.g3be9a5c9.el9.x86 17/198 2024-03-28T17:51:28.556 INFO:teuthology.orchestra.run.smithi022.stdout: Upgrading : libmount-2.37.4-18.el9.x86_64 18/198 2024-03-28T17:51:28.588 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 19/198 2024-03-28T17:51:28.601 INFO:teuthology.orchestra.run.smithi079.stdout: Running scriptlet: libcephsqlite-2:19.0.0-2145.g3be9a5c9.el9.x86_64 16/198 2024-03-28T17:51:28.628 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : libradosstriper1-2:19.0.0-2145.g3be9a5c9.el9.x86 17/198 2024-03-28T17:51:28.657 INFO:teuthology.orchestra.run.smithi022.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 19/198 2024-03-28T17:51:28.701 INFO:teuthology.orchestra.run.smithi079.stdout: Running scriptlet: libradosstriper1-2:19.0.0-2145.g3be9a5c9.el9.x86 17/198 2024-03-28T17:51:28.774 INFO:teuthology.orchestra.run.smithi079.stdout: Upgrading : libmount-2.37.4-18.el9.x86_64 18/198 2024-03-28T17:51:29.078 INFO:teuthology.orchestra.run.smithi022.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 20/198 2024-03-28T17:51:29.087 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 19/198 2024-03-28T17:51:29.153 INFO:teuthology.orchestra.run.smithi079.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 19/198 2024-03-28T17:51:29.155 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 20/198 2024-03-28T17:51:29.196 INFO:teuthology.orchestra.run.smithi022.stdout: Upgrading : libsmartcols-2.37.4-18.el9.x86_64 21/198 2024-03-28T17:51:29.430 INFO:teuthology.orchestra.run.smithi079.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 20/198 2024-03-28T17:51:29.458 INFO:teuthology.orchestra.run.smithi134.stdout:(169/174): util-linux-core-2.37.4-18.el9.x86_64 375 kB/s | 465 kB 00:01 2024-03-28T17:51:29.487 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 20/198 2024-03-28T17:51:29.530 INFO:teuthology.orchestra.run.smithi022.stdout: Upgrading : libacl-2.3.1-4.el9.x86_64 22/198 2024-03-28T17:51:29.530 INFO:teuthology.orchestra.run.smithi079.stdout: Upgrading : libsmartcols-2.37.4-18.el9.x86_64 21/198 2024-03-28T17:51:29.695 INFO:teuthology.orchestra.run.smithi022.stdout: Upgrading : NetworkManager-libnm-1:1.47.2-1.el9.x86_64 23/198 2024-03-28T17:51:29.818 INFO:teuthology.orchestra.run.smithi022.stdout: Running scriptlet: NetworkManager-libnm-1:1.47.2-1.el9.x86_64 23/198 2024-03-28T17:51:29.847 INFO:teuthology.orchestra.run.smithi079.stdout: Upgrading : libacl-2.3.1-4.el9.x86_64 22/198 2024-03-28T17:51:29.900 INFO:teuthology.orchestra.run.smithi134.stdout:(170/174): openssl-libs-3.0.7-27.el9.x86_64.rpm 656 kB/s | 2.2 MB 00:03 2024-03-28T17:51:30.006 INFO:teuthology.orchestra.run.smithi079.stdout: Upgrading : NetworkManager-libnm-1:1.47.2-1.el9.x86_64 23/198 2024-03-28T17:51:30.017 INFO:teuthology.orchestra.run.smithi022.stdout: Running scriptlet: NetworkManager-1:1.47.2-1.el9.x86_64 24/198 2024-03-28T17:51:30.077 INFO:teuthology.orchestra.run.smithi079.stdout: Running scriptlet: NetworkManager-libnm-1:1.47.2-1.el9.x86_64 23/198 2024-03-28T17:51:30.089 INFO:teuthology.orchestra.run.smithi022.stdout: Upgrading : NetworkManager-1:1.47.2-1.el9.x86_64 24/198 2024-03-28T17:51:30.133 INFO:teuthology.orchestra.run.smithi134.stdout:(171/174): util-linux-2.37.4-18.el9.x86_64.rpm 654 kB/s | 2.3 MB 00:03 2024-03-28T17:51:30.234 INFO:teuthology.orchestra.run.smithi134.stdout:(172/174): util-linux-user-2.37.4-18.el9.x86_64 40 kB/s | 31 kB 00:00 2024-03-28T17:51:30.263 INFO:teuthology.orchestra.run.smithi079.stdout: Running scriptlet: NetworkManager-1:1.47.2-1.el9.x86_64 24/198 2024-03-28T17:51:30.351 INFO:teuthology.orchestra.run.smithi079.stdout: Upgrading : NetworkManager-1:1.47.2-1.el9.x86_64 24/198 2024-03-28T17:51:30.431 INFO:teuthology.orchestra.run.smithi022.stdout: Running scriptlet: NetworkManager-1:1.47.2-1.el9.x86_64 24/198 2024-03-28T17:51:30.647 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : re2-1:20211101-3.el9.x86_64 25/198 2024-03-28T17:51:30.702 INFO:teuthology.orchestra.run.smithi079.stdout: Running scriptlet: NetworkManager-1:1.47.2-1.el9.x86_64 24/198 2024-03-28T17:51:30.781 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : libarrow-9.0.0-10.el9.x86_64 26/198 2024-03-28T17:51:30.820 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 27/198 2024-03-28T17:51:30.858 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : liboath-2.6.7-2.el9.x86_64 28/198 2024-03-28T17:51:30.890 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : re2-1:20211101-3.el9.x86_64 25/198 2024-03-28T17:51:30.925 INFO:teuthology.orchestra.run.smithi134.stdout:(173/174): libacl-devel-2.3.1-4.el9.x86_64.rpm 103 kB/s | 74 kB 00:00 2024-03-28T17:51:30.950 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 29/198 2024-03-28T17:51:31.010 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : libarrow-9.0.0-10.el9.x86_64 26/198 2024-03-28T17:51:31.056 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 27/198 2024-03-28T17:51:31.082 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 30/198 2024-03-28T17:51:31.093 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : liboath-2.6.7-2.el9.x86_64 28/198 2024-03-28T17:51:31.148 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 31/198 2024-03-28T17:51:31.154 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 29/198 2024-03-28T17:51:31.197 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 32/198 2024-03-28T17:51:31.267 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 30/198 2024-03-28T17:51:31.292 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 33/198 2024-03-28T17:51:31.332 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 31/198 2024-03-28T17:51:31.337 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 34/198 2024-03-28T17:51:31.373 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 32/198 2024-03-28T17:51:31.434 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 35/198 2024-03-28T17:51:31.463 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 33/198 2024-03-28T17:51:31.483 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 36/198 2024-03-28T17:51:31.514 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 34/198 2024-03-28T17:51:31.521 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : lua-5.4.4-4.el9.x86_64 37/198 2024-03-28T17:51:31.574 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 35/198 2024-03-28T17:51:31.618 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 36/198 2024-03-28T17:51:31.644 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 38/198 2024-03-28T17:51:31.650 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : lua-5.4.4-4.el9.x86_64 37/198 2024-03-28T17:51:31.682 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : python3-ceph-common-2:19.0.0-2145.g3be9a5c9.el9. 39/198 2024-03-28T17:51:31.748 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : python3-ceph-argparse-2:19.0.0-2145.g3be9a5c9.el 40/198 2024-03-28T17:51:31.762 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 38/198 2024-03-28T17:51:31.796 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : python3-ceph-common-2:19.0.0-2145.g3be9a5c9.el9. 39/198 2024-03-28T17:51:31.810 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : python3-cephfs-2:19.0.0-2145.g3be9a5c9.el9.x86_6 41/198 2024-03-28T17:51:31.890 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : python3-ceph-argparse-2:19.0.0-2145.g3be9a5c9.el 40/198 2024-03-28T17:51:31.929 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : python3-cephfs-2:19.0.0-2145.g3be9a5c9.el9.x86_6 41/198 2024-03-28T17:51:31.946 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 42/198 2024-03-28T17:51:32.010 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : luarocks-3.9.2-1.el9.noarch 43/198 2024-03-28T17:51:32.017 INFO:teuthology.orchestra.run.smithi134.stdout:(174/174): xfsprogs-6.3.0-1.el9.x86_64.rpm 519 kB/s | 1.1 MB 00:02 2024-03-28T17:51:32.030 INFO:teuthology.orchestra.run.smithi134.stdout:-------------------------------------------------------------------------------- 2024-03-28T17:51:32.030 INFO:teuthology.orchestra.run.smithi134.stdout:Total 5.6 MB/s | 225 MB 00:40 2024-03-28T17:51:32.050 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 42/198 2024-03-28T17:51:32.107 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : luarocks-3.9.2-1.el9.noarch 43/198 2024-03-28T17:51:32.113 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 44/198 2024-03-28T17:51:32.207 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 44/198 2024-03-28T17:51:32.301 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 45/198 2024-03-28T17:51:32.372 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 46/198 2024-03-28T17:51:32.384 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 45/198 2024-03-28T17:51:32.450 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 46/198 2024-03-28T17:51:33.089 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : python3-rsa-4.9-2.el9.noarch 47/198 2024-03-28T17:51:33.141 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : python3-rsa-4.9-2.el9.noarch 47/198 2024-03-28T17:51:33.206 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 48/198 2024-03-28T17:51:33.238 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 48/198 2024-03-28T17:51:33.341 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 49/198 2024-03-28T17:51:33.356 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 49/198 2024-03-28T17:51:33.391 INFO:teuthology.orchestra.run.smithi079.stdout: Upgrading : util-linux-core-2.37.4-18.el9.x86_64 50/198 2024-03-28T17:51:33.418 INFO:teuthology.orchestra.run.smithi079.stdout: Running scriptlet: util-linux-core-2.37.4-18.el9.x86_64 50/198 2024-03-28T17:51:33.460 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 51/198 2024-03-28T17:51:33.470 INFO:teuthology.orchestra.run.smithi022.stdout: Upgrading : util-linux-core-2.37.4-18.el9.x86_64 50/198 2024-03-28T17:51:33.504 INFO:teuthology.orchestra.run.smithi022.stdout: Running scriptlet: util-linux-core-2.37.4-18.el9.x86_64 50/198 2024-03-28T17:51:33.527 INFO:teuthology.orchestra.run.smithi079.stdout: Upgrading : elfutils-libs-0.190-2.el9.x86_64 52/198 2024-03-28T17:51:33.573 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : librados-devel-2:19.0.0-2145.g3be9a5c9.el9.x86_6 53/198 2024-03-28T17:51:33.614 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 54/198 2024-03-28T17:51:33.644 INFO:teuthology.orchestra.run.smithi134.stdout:Running transaction check 2024-03-28T17:51:33.866 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 51/198 2024-03-28T17:51:33.966 INFO:teuthology.orchestra.run.smithi022.stdout: Upgrading : elfutils-libs-0.190-2.el9.x86_64 52/198 2024-03-28T17:51:34.013 INFO:teuthology.orchestra.run.smithi134.stdout:Transaction check succeeded. 2024-03-28T17:51:34.013 INFO:teuthology.orchestra.run.smithi134.stdout:Running transaction test 2024-03-28T17:51:34.028 INFO:teuthology.orchestra.run.smithi079.stdout: Upgrading : libfdisk-2.37.4-18.el9.x86_64 55/198 2024-03-28T17:51:34.030 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : librados-devel-2:19.0.0-2145.g3be9a5c9.el9.x86_6 53/198 2024-03-28T17:51:34.105 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 54/198 2024-03-28T17:51:34.264 INFO:teuthology.orchestra.run.smithi079.stdout: Upgrading : util-linux-2.37.4-18.el9.x86_64 56/198 2024-03-28T17:51:34.356 INFO:teuthology.orchestra.run.smithi079.stdout: Upgrading : xfsprogs-6.3.0-1.el9.x86_64 57/198 2024-03-28T17:51:34.414 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 58/198 2024-03-28T17:51:34.565 INFO:teuthology.orchestra.run.smithi022.stdout: Upgrading : libfdisk-2.37.4-18.el9.x86_64 55/198 2024-03-28T17:51:34.620 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 59/198 2024-03-28T17:51:34.687 INFO:teuthology.orchestra.run.smithi079.stdout: Upgrading : openssl-1:3.0.7-27.el9.x86_64 60/198 2024-03-28T17:51:34.763 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 61/198 2024-03-28T17:51:34.793 INFO:teuthology.orchestra.run.smithi022.stdout: Upgrading : util-linux-2.37.4-18.el9.x86_64 56/198 2024-03-28T17:51:34.857 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 62/198 2024-03-28T17:51:34.875 INFO:teuthology.orchestra.run.smithi022.stdout: Upgrading : xfsprogs-6.3.0-1.el9.x86_64 57/198 2024-03-28T17:51:34.894 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 63/198 2024-03-28T17:51:34.931 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 58/198 2024-03-28T17:51:34.934 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 64/198 2024-03-28T17:51:35.002 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 65/198 2024-03-28T17:51:35.042 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 66/198 2024-03-28T17:51:35.104 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 67/198 2024-03-28T17:51:35.150 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 59/198 2024-03-28T17:51:35.164 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 68/198 2024-03-28T17:51:35.204 INFO:teuthology.orchestra.run.smithi022.stdout: Upgrading : openssl-1:3.0.7-27.el9.x86_64 60/198 2024-03-28T17:51:35.284 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 61/198 2024-03-28T17:51:35.355 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 69/198 2024-03-28T17:51:35.386 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 62/198 2024-03-28T17:51:35.425 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 70/198 2024-03-28T17:51:35.429 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 63/198 2024-03-28T17:51:35.463 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : python3-colorama-0.4.4-12.el9.noarch 71/198 2024-03-28T17:51:35.469 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 64/198 2024-03-28T17:51:35.522 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 72/198 2024-03-28T17:51:35.539 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 65/198 2024-03-28T17:51:35.586 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 66/198 2024-03-28T17:51:35.652 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 73/198 2024-03-28T17:51:35.660 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 67/198 2024-03-28T17:51:35.694 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : python3-google-auth-1:2.28.2-1.el9.noarch 74/198 2024-03-28T17:51:35.736 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 68/198 2024-03-28T17:51:35.741 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 75/198 2024-03-28T17:51:35.777 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 76/198 2024-03-28T17:51:35.810 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : libarrow-doc-9.0.0-10.el9.noarch 77/198 2024-03-28T17:51:35.939 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 69/198 2024-03-28T17:51:35.968 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 78/198 2024-03-28T17:51:36.000 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 70/198 2024-03-28T17:51:36.038 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : python3-colorama-0.4.4-12.el9.noarch 71/198 2024-03-28T17:51:36.097 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 72/198 2024-03-28T17:51:36.137 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 79/198 2024-03-28T17:51:36.241 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 73/198 2024-03-28T17:51:36.273 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 80/198 2024-03-28T17:51:36.294 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : python3-google-auth-1:2.28.2-1.el9.noarch 74/198 2024-03-28T17:51:36.339 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : python3-devel-3.9.17-2.el9.x86_64 81/198 2024-03-28T17:51:36.341 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 75/198 2024-03-28T17:51:36.372 INFO:teuthology.orchestra.run.smithi134.stdout:Transaction test succeeded. 2024-03-28T17:51:36.373 INFO:teuthology.orchestra.run.smithi134.stdout:Running transaction 2024-03-28T17:51:36.386 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 76/198 2024-03-28T17:51:36.403 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : python3-ply-3.11-14.el9.noarch 82/198 2024-03-28T17:51:36.419 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : libarrow-doc-9.0.0-10.el9.noarch 77/198 2024-03-28T17:51:36.477 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 83/198 2024-03-28T17:51:36.584 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 78/198 2024-03-28T17:51:36.732 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 84/198 2024-03-28T17:51:36.767 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 79/198 2024-03-28T17:51:36.794 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 85/198 2024-03-28T17:51:36.904 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 80/198 2024-03-28T17:51:36.953 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : python3-devel-3.9.17-2.el9.x86_64 81/198 2024-03-28T17:51:36.967 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 86/198 2024-03-28T17:51:37.015 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : python3-ply-3.11-14.el9.noarch 82/198 2024-03-28T17:51:37.025 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 87/198 2024-03-28T17:51:37.065 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 88/198 2024-03-28T17:51:37.090 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 83/198 2024-03-28T17:51:37.103 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : fuse-2.9.9-15.el9.x86_64 89/198 2024-03-28T17:51:37.145 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : zlib-devel-1.2.11-40.el9.x86_64 90/198 2024-03-28T17:51:37.192 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 91/198 2024-03-28T17:51:37.250 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : uuid-1.6.2-55.el9.x86_64 92/198 2024-03-28T17:51:37.311 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 93/198 2024-03-28T17:51:37.351 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 94/198 2024-03-28T17:51:37.373 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 84/198 2024-03-28T17:51:37.433 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 85/198 2024-03-28T17:51:37.466 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 95/198 2024-03-28T17:51:37.505 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 96/198 2024-03-28T17:51:37.540 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 97/198 2024-03-28T17:51:37.594 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 98/198 2024-03-28T17:51:37.628 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 86/198 2024-03-28T17:51:37.635 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 99/198 2024-03-28T17:51:37.684 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 87/198 2024-03-28T17:51:37.737 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 88/198 2024-03-28T17:51:37.779 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 100/198 2024-03-28T17:51:37.785 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : fuse-2.9.9-15.el9.x86_64 89/198 2024-03-28T17:51:37.828 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : zlib-devel-1.2.11-40.el9.x86_64 90/198 2024-03-28T17:51:37.875 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 91/198 2024-03-28T17:51:37.895 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 101/198 2024-03-28T17:51:37.943 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : uuid-1.6.2-55.el9.x86_64 92/198 2024-03-28T17:51:37.957 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 102/198 2024-03-28T17:51:37.993 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 103/198 2024-03-28T17:51:38.000 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 93/198 2024-03-28T17:51:38.052 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 104/198 2024-03-28T17:51:38.075 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 94/198 2024-03-28T17:51:38.190 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : python3-packaging-20.9-5.el9.noarch 105/198 2024-03-28T17:51:38.209 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 95/198 2024-03-28T17:51:38.251 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 106/198 2024-03-28T17:51:38.253 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 96/198 2024-03-28T17:51:38.296 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 97/198 2024-03-28T17:51:38.362 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 98/198 2024-03-28T17:51:38.408 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 99/198 2024-03-28T17:51:38.558 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 100/198 2024-03-28T17:51:38.676 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 101/198 2024-03-28T17:51:38.737 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 102/198 2024-03-28T17:51:38.773 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 103/198 2024-03-28T17:51:38.840 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 104/198 2024-03-28T17:51:38.985 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : python3-packaging-20.9-5.el9.noarch 105/198 2024-03-28T17:51:39.055 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 106/198 2024-03-28T17:51:39.194 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 107/198 2024-03-28T17:51:39.251 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 108/198 2024-03-28T17:51:39.284 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 109/198 2024-03-28T17:51:39.342 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 110/198 2024-03-28T17:51:39.453 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 111/198 2024-03-28T17:51:39.494 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 112/198 2024-03-28T17:51:39.725 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : openblas-0.3.21-2.el9.x86_64 113/198 2024-03-28T17:51:39.758 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 114/198 2024-03-28T17:51:39.850 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 115/198 2024-03-28T17:51:40.036 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 107/198 2024-03-28T17:51:40.088 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 108/198 2024-03-28T17:51:40.121 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 109/198 2024-03-28T17:51:40.178 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 110/198 2024-03-28T17:51:40.292 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 111/198 2024-03-28T17:51:40.339 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 112/198 2024-03-28T17:51:40.583 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : openblas-0.3.21-2.el9.x86_64 113/198 2024-03-28T17:51:40.618 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 114/198 2024-03-28T17:51:40.637 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 116/198 2024-03-28T17:51:40.719 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 115/198 2024-03-28T17:51:40.770 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 117/198 2024-03-28T17:51:41.572 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 116/198 2024-03-28T17:51:41.728 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 117/198 2024-03-28T17:51:42.279 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 118/198 2024-03-28T17:51:42.374 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 119/198 2024-03-28T17:51:42.411 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : oniguruma-6.9.6-1.el9.5.x86_64 120/198 2024-03-28T17:51:42.442 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : jq-1.6-16.el9.x86_64 121/198 2024-03-28T17:51:42.473 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 122/198 2024-03-28T17:51:42.836 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 123/198 2024-03-28T17:51:42.864 INFO:teuthology.orchestra.run.smithi079.stdout: Upgrading : librbd1-2:19.0.0-2145.g3be9a5c9.el9.x86_64 124/198 2024-03-28T17:51:42.985 INFO:teuthology.orchestra.run.smithi079.stdout: Running scriptlet: librbd1-2:19.0.0-2145.g3be9a5c9.el9.x86_64 124/198 2024-03-28T17:51:43.021 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : python3-rbd-2:19.0.0-2145.g3be9a5c9.el9.x86_64 125/198 2024-03-28T17:51:43.075 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 126/198 2024-03-28T17:51:43.332 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 118/198 2024-03-28T17:51:43.423 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 119/198 2024-03-28T17:51:43.462 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : oniguruma-6.9.6-1.el9.5.x86_64 120/198 2024-03-28T17:51:43.491 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : jq-1.6-16.el9.x86_64 121/198 2024-03-28T17:51:43.532 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 122/198 2024-03-28T17:51:43.618 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : parquet-libs-9.0.0-10.el9.x86_64 127/198 2024-03-28T17:51:43.651 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : librgw2-2:19.0.0-2145.g3be9a5c9.el9.x86_64 128/198 2024-03-28T17:51:43.742 INFO:teuthology.orchestra.run.smithi079.stdout: Running scriptlet: librgw2-2:19.0.0-2145.g3be9a5c9.el9.x86_64 128/198 2024-03-28T17:51:43.770 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : python3-rgw-2:19.0.0-2145.g3be9a5c9.el9.x86_64 129/198 2024-03-28T17:51:43.929 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 123/198 2024-03-28T17:51:43.941 INFO:teuthology.orchestra.run.smithi134.stdout: Preparing : 1/1 2024-03-28T17:51:43.972 INFO:teuthology.orchestra.run.smithi022.stdout: Upgrading : librbd1-2:19.0.0-2145.g3be9a5c9.el9.x86_64 124/198 2024-03-28T17:51:44.059 INFO:teuthology.orchestra.run.smithi134.stdout: Upgrading : openssl-libs-1:3.0.7-27.el9.x86_64 1/198 2024-03-28T17:51:44.079 INFO:teuthology.orchestra.run.smithi022.stdout: Running scriptlet: librbd1-2:19.0.0-2145.g3be9a5c9.el9.x86_64 124/198 2024-03-28T17:51:44.095 INFO:teuthology.orchestra.run.smithi134.stdout: Upgrading : libuuid-2.37.4-18.el9.x86_64 2/198 2024-03-28T17:51:44.120 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : python3-rbd-2:19.0.0-2145.g3be9a5c9.el9.x86_64 125/198 2024-03-28T17:51:44.124 INFO:teuthology.orchestra.run.smithi134.stdout: Upgrading : libblkid-2.37.4-18.el9.x86_64 3/198 2024-03-28T17:51:44.167 INFO:teuthology.orchestra.run.smithi134.stdout: Running scriptlet: libblkid-2.37.4-18.el9.x86_64 3/198 2024-03-28T17:51:44.189 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 126/198 2024-03-28T17:51:44.254 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 4/198 2024-03-28T17:51:44.380 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 5/198 2024-03-28T17:51:44.468 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : thrift-0.15.0-2.el9.x86_64 6/198 2024-03-28T17:51:44.506 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 7/198 2024-03-28T17:51:44.598 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 8/198 2024-03-28T17:51:44.759 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : parquet-libs-9.0.0-10.el9.x86_64 127/198 2024-03-28T17:51:44.792 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : librgw2-2:19.0.0-2145.g3be9a5c9.el9.x86_64 128/198 2024-03-28T17:51:44.891 INFO:teuthology.orchestra.run.smithi022.stdout: Running scriptlet: librgw2-2:19.0.0-2145.g3be9a5c9.el9.x86_64 128/198 2024-03-28T17:51:44.919 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : python3-rgw-2:19.0.0-2145.g3be9a5c9.el9.x86_64 129/198 2024-03-28T17:51:45.019 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 9/198 2024-03-28T17:51:45.087 INFO:teuthology.orchestra.run.smithi134.stdout: Upgrading : librados2-2:19.0.0-2145.g3be9a5c9.el9.x86_64 10/198 2024-03-28T17:51:45.240 INFO:teuthology.orchestra.run.smithi134.stdout: Running scriptlet: librados2-2:19.0.0-2145.g3be9a5c9.el9.x86_64 10/198 2024-03-28T17:51:45.272 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : libcephfs2-2:19.0.0-2145.g3be9a5c9.el9.x86_64 11/198 2024-03-28T17:51:45.377 INFO:teuthology.orchestra.run.smithi134.stdout: Running scriptlet: libcephfs2-2:19.0.0-2145.g3be9a5c9.el9.x86_64 11/198 2024-03-28T17:51:45.412 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-rados-2:19.0.0-2145.g3be9a5c9.el9.x86_64 12/198 2024-03-28T17:51:45.481 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : postgresql-private-libs-13.14-1.el9.x86_64 13/198 2024-03-28T17:51:45.770 INFO:teuthology.orchestra.run.smithi134.stdout: Upgrading : elfutils-libelf-0.190-2.el9.x86_64 14/198 2024-03-28T17:51:45.826 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : postgresql-13.14-1.el9.x86_64 15/198 2024-03-28T17:51:45.851 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : libcephsqlite-2:19.0.0-2145.g3be9a5c9.el9.x86_64 16/198 2024-03-28T17:51:46.001 INFO:teuthology.orchestra.run.smithi134.stdout: Running scriptlet: libcephsqlite-2:19.0.0-2145.g3be9a5c9.el9.x86_64 16/198 2024-03-28T17:51:46.051 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : libradosstriper1-2:19.0.0-2145.g3be9a5c9.el9.x86 17/198 2024-03-28T17:51:46.152 INFO:teuthology.orchestra.run.smithi134.stdout: Running scriptlet: libradosstriper1-2:19.0.0-2145.g3be9a5c9.el9.x86 17/198 2024-03-28T17:51:46.197 INFO:teuthology.orchestra.run.smithi134.stdout: Upgrading : libmount-2.37.4-18.el9.x86_64 18/198 2024-03-28T17:51:46.238 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 19/198 2024-03-28T17:51:46.298 INFO:teuthology.orchestra.run.smithi134.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 19/198 2024-03-28T17:51:46.373 INFO:teuthology.orchestra.run.smithi079.stdout: Running scriptlet: ceph-common-2:19.0.0-2145.g3be9a5c9.el9.x86_64 130/198 2024-03-28T17:51:46.409 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : ceph-common-2:19.0.0-2145.g3be9a5c9.el9.x86_64 130/198 2024-03-28T17:51:46.592 INFO:teuthology.orchestra.run.smithi134.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 20/198 2024-03-28T17:51:46.640 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 20/198 2024-03-28T17:51:46.687 INFO:teuthology.orchestra.run.smithi134.stdout: Upgrading : libsmartcols-2.37.4-18.el9.x86_64 21/198 2024-03-28T17:51:47.023 INFO:teuthology.orchestra.run.smithi134.stdout: Upgrading : libacl-2.3.1-4.el9.x86_64 22/198 2024-03-28T17:51:47.041 INFO:teuthology.orchestra.run.smithi079.stdout: Running scriptlet: ceph-common-2:19.0.0-2145.g3be9a5c9.el9.x86_64 130/198 2024-03-28T17:51:47.079 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : ceph-base-2:19.0.0-2145.g3be9a5c9.el9.x86_64 131/198 2024-03-28T17:51:47.211 INFO:teuthology.orchestra.run.smithi134.stdout: Upgrading : NetworkManager-libnm-1:1.47.2-1.el9.x86_64 23/198 2024-03-28T17:51:47.262 INFO:teuthology.orchestra.run.smithi079.stdout: Running scriptlet: ceph-base-2:19.0.0-2145.g3be9a5c9.el9.x86_64 131/198 2024-03-28T17:51:47.262 INFO:teuthology.orchestra.run.smithi079.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-03-28T17:51:47.262 INFO:teuthology.orchestra.run.smithi079.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-03-28T17:51:47.262 INFO:teuthology.orchestra.run.smithi079.stdout: 2024-03-28T17:51:47.325 INFO:teuthology.orchestra.run.smithi134.stdout: Running scriptlet: NetworkManager-libnm-1:1.47.2-1.el9.x86_64 23/198 2024-03-28T17:51:47.327 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : ceph-selinux-2:19.0.0-2145.g3be9a5c9.el9.x86_64 132/198 2024-03-28T17:51:47.522 INFO:teuthology.orchestra.run.smithi134.stdout: Running scriptlet: NetworkManager-1:1.47.2-1.el9.x86_64 24/198 2024-03-28T17:51:47.561 INFO:teuthology.orchestra.run.smithi022.stdout: Running scriptlet: ceph-common-2:19.0.0-2145.g3be9a5c9.el9.x86_64 130/198 2024-03-28T17:51:47.580 INFO:teuthology.orchestra.run.smithi134.stdout: Upgrading : NetworkManager-1:1.47.2-1.el9.x86_64 24/198 2024-03-28T17:51:47.607 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : ceph-common-2:19.0.0-2145.g3be9a5c9.el9.x86_64 130/198 2024-03-28T17:51:47.934 INFO:teuthology.orchestra.run.smithi134.stdout: Running scriptlet: NetworkManager-1:1.47.2-1.el9.x86_64 24/198 2024-03-28T17:51:48.125 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : re2-1:20211101-3.el9.x86_64 25/198 2024-03-28T17:51:48.247 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : libarrow-9.0.0-10.el9.x86_64 26/198 2024-03-28T17:51:48.275 INFO:teuthology.orchestra.run.smithi022.stdout: Running scriptlet: ceph-common-2:19.0.0-2145.g3be9a5c9.el9.x86_64 130/198 2024-03-28T17:51:48.286 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 27/198 2024-03-28T17:51:48.319 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : ceph-base-2:19.0.0-2145.g3be9a5c9.el9.x86_64 131/198 2024-03-28T17:51:48.323 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : liboath-2.6.7-2.el9.x86_64 28/198 2024-03-28T17:51:48.381 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 29/198 2024-03-28T17:51:48.492 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 30/198 2024-03-28T17:51:48.495 INFO:teuthology.orchestra.run.smithi022.stdout: Running scriptlet: ceph-base-2:19.0.0-2145.g3be9a5c9.el9.x86_64 131/198 2024-03-28T17:51:48.495 INFO:teuthology.orchestra.run.smithi022.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-03-28T17:51:48.495 INFO:teuthology.orchestra.run.smithi022.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-03-28T17:51:48.495 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-03-28T17:51:48.536 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : ceph-selinux-2:19.0.0-2145.g3be9a5c9.el9.x86_64 132/198 2024-03-28T17:51:48.563 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 31/198 2024-03-28T17:51:48.620 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 32/198 2024-03-28T17:51:48.712 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 33/198 2024-03-28T17:51:48.759 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 34/198 2024-03-28T17:51:48.820 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 35/198 2024-03-28T17:51:48.862 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 36/198 2024-03-28T17:51:48.895 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : lua-5.4.4-4.el9.x86_64 37/198 2024-03-28T17:51:49.008 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 38/198 2024-03-28T17:51:49.050 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-ceph-common-2:19.0.0-2145.g3be9a5c9.el9. 39/198 2024-03-28T17:51:49.113 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-ceph-argparse-2:19.0.0-2145.g3be9a5c9.el 40/198 2024-03-28T17:51:49.156 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-cephfs-2:19.0.0-2145.g3be9a5c9.el9.x86_6 41/198 2024-03-28T17:51:49.278 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 42/198 2024-03-28T17:51:49.335 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : luarocks-3.9.2-1.el9.noarch 43/198 2024-03-28T17:51:49.475 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 44/198 2024-03-28T17:51:49.655 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 45/198 2024-03-28T17:51:49.719 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 46/198 2024-03-28T17:51:50.421 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-rsa-4.9-2.el9.noarch 47/198 2024-03-28T17:51:50.519 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 48/198 2024-03-28T17:51:50.633 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 49/198 2024-03-28T17:51:50.709 INFO:teuthology.orchestra.run.smithi134.stdout: Upgrading : util-linux-core-2.37.4-18.el9.x86_64 50/198 2024-03-28T17:51:50.736 INFO:teuthology.orchestra.run.smithi134.stdout: Running scriptlet: util-linux-core-2.37.4-18.el9.x86_64 50/198 2024-03-28T17:51:50.781 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 51/198 2024-03-28T17:51:50.850 INFO:teuthology.orchestra.run.smithi134.stdout: Upgrading : elfutils-libs-0.190-2.el9.x86_64 52/198 2024-03-28T17:51:50.910 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : librados-devel-2:19.0.0-2145.g3be9a5c9.el9.x86_6 53/198 2024-03-28T17:51:50.951 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 54/198 2024-03-28T17:51:51.340 INFO:teuthology.orchestra.run.smithi134.stdout: Upgrading : libfdisk-2.37.4-18.el9.x86_64 55/198 2024-03-28T17:51:51.583 INFO:teuthology.orchestra.run.smithi134.stdout: Upgrading : util-linux-2.37.4-18.el9.x86_64 56/198 2024-03-28T17:51:51.670 INFO:teuthology.orchestra.run.smithi134.stdout: Upgrading : xfsprogs-6.3.0-1.el9.x86_64 57/198 2024-03-28T17:51:51.734 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 58/198 2024-03-28T17:51:51.934 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 59/198 2024-03-28T17:51:51.993 INFO:teuthology.orchestra.run.smithi134.stdout: Upgrading : openssl-1:3.0.7-27.el9.x86_64 60/198 2024-03-28T17:51:52.081 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 61/198 2024-03-28T17:51:52.190 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 62/198 2024-03-28T17:51:52.226 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 63/198 2024-03-28T17:51:52.265 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 64/198 2024-03-28T17:51:52.356 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 65/198 2024-03-28T17:51:52.408 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 66/198 2024-03-28T17:51:52.491 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 67/198 2024-03-28T17:51:52.559 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 68/198 2024-03-28T17:51:52.762 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 69/198 2024-03-28T17:51:52.822 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 70/198 2024-03-28T17:51:52.859 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-colorama-0.4.4-12.el9.noarch 71/198 2024-03-28T17:51:52.916 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 72/198 2024-03-28T17:51:53.046 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 73/198 2024-03-28T17:51:53.082 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-google-auth-1:2.28.2-1.el9.noarch 74/198 2024-03-28T17:51:53.127 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 75/198 2024-03-28T17:51:53.173 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 76/198 2024-03-28T17:51:53.215 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : libarrow-doc-9.0.0-10.el9.noarch 77/198 2024-03-28T17:51:53.414 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 78/198 2024-03-28T17:51:53.593 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 79/198 2024-03-28T17:51:53.731 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 80/198 2024-03-28T17:51:53.783 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-devel-3.9.17-2.el9.x86_64 81/198 2024-03-28T17:51:53.846 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-ply-3.11-14.el9.noarch 82/198 2024-03-28T17:51:53.927 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 83/198 2024-03-28T17:51:54.210 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 84/198 2024-03-28T17:51:54.269 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 85/198 2024-03-28T17:51:54.449 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 86/198 2024-03-28T17:51:54.618 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 87/198 2024-03-28T17:51:55.019 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 88/198 2024-03-28T17:51:55.068 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : fuse-2.9.9-15.el9.x86_64 89/198 2024-03-28T17:51:55.120 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : zlib-devel-1.2.11-40.el9.x86_64 90/198 2024-03-28T17:51:55.171 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 91/198 2024-03-28T17:51:55.239 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : uuid-1.6.2-55.el9.x86_64 92/198 2024-03-28T17:51:55.294 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 93/198 2024-03-28T17:51:55.350 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 94/198 2024-03-28T17:51:55.527 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 95/198 2024-03-28T17:51:55.577 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 96/198 2024-03-28T17:51:55.613 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 97/198 2024-03-28T17:51:55.677 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 98/198 2024-03-28T17:51:55.715 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 99/198 2024-03-28T17:51:55.858 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 100/198 2024-03-28T17:51:55.975 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 101/198 2024-03-28T17:51:56.036 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 102/198 2024-03-28T17:51:56.072 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 103/198 2024-03-28T17:51:56.129 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 104/198 2024-03-28T17:51:56.266 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-packaging-20.9-5.el9.noarch 105/198 2024-03-28T17:51:56.329 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 106/198 2024-03-28T17:51:57.283 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 107/198 2024-03-28T17:51:57.336 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 108/198 2024-03-28T17:51:57.370 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 109/198 2024-03-28T17:51:57.426 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 110/198 2024-03-28T17:51:57.537 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 111/198 2024-03-28T17:51:57.579 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 112/198 2024-03-28T17:51:57.810 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : openblas-0.3.21-2.el9.x86_64 113/198 2024-03-28T17:51:57.842 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 114/198 2024-03-28T17:51:57.935 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 115/198 2024-03-28T17:51:58.739 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 116/198 2024-03-28T17:51:58.871 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 117/198 2024-03-28T17:52:00.315 INFO:teuthology.orchestra.run.smithi079.stdout: Running scriptlet: ceph-selinux-2:19.0.0-2145.g3be9a5c9.el9.x86_64 132/198 2024-03-28T17:52:00.315 INFO:teuthology.orchestra.run.smithi079.stdout:skipping the directory /sys 2024-03-28T17:52:00.316 INFO:teuthology.orchestra.run.smithi079.stdout:skipping the directory /proc 2024-03-28T17:52:00.316 INFO:teuthology.orchestra.run.smithi079.stdout:skipping the directory /mnt 2024-03-28T17:52:00.316 INFO:teuthology.orchestra.run.smithi079.stdout:skipping the directory /var/tmp 2024-03-28T17:52:00.316 INFO:teuthology.orchestra.run.smithi079.stdout:skipping the directory /home 2024-03-28T17:52:00.316 INFO:teuthology.orchestra.run.smithi079.stdout:skipping the directory /root 2024-03-28T17:52:00.316 INFO:teuthology.orchestra.run.smithi079.stdout:skipping the directory /tmp 2024-03-28T17:52:00.316 INFO:teuthology.orchestra.run.smithi079.stdout: 2024-03-28T17:52:00.381 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 118/198 2024-03-28T17:52:00.506 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 119/198 2024-03-28T17:52:00.543 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : ceph-volume-2:19.0.0-2145.g3be9a5c9.el9.noarch 133/198 2024-03-28T17:52:00.545 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : oniguruma-6.9.6-1.el9.5.x86_64 120/198 2024-03-28T17:52:00.582 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : jq-1.6-16.el9.x86_64 121/198 2024-03-28T17:52:00.595 INFO:teuthology.orchestra.run.smithi079.stdout: Running scriptlet: ceph-volume-2:19.0.0-2145.g3be9a5c9.el9.noarch 133/198 2024-03-28T17:52:00.595 INFO:teuthology.orchestra.run.smithi079.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-28T17:52:00.596 INFO:teuthology.orchestra.run.smithi079.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-03-28T17:52:00.596 INFO:teuthology.orchestra.run.smithi079.stdout: 2024-03-28T17:52:00.625 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 122/198 2024-03-28T17:52:01.003 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 123/198 2024-03-28T17:52:01.045 INFO:teuthology.orchestra.run.smithi134.stdout: Upgrading : librbd1-2:19.0.0-2145.g3be9a5c9.el9.x86_64 124/198 2024-03-28T17:52:01.164 INFO:teuthology.orchestra.run.smithi134.stdout: Running scriptlet: librbd1-2:19.0.0-2145.g3be9a5c9.el9.x86_64 124/198 2024-03-28T17:52:01.201 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-rbd-2:19.0.0-2145.g3be9a5c9.el9.x86_64 125/198 2024-03-28T17:52:01.278 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 126/198 2024-03-28T17:52:01.804 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : parquet-libs-9.0.0-10.el9.x86_64 127/198 2024-03-28T17:52:01.839 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : librgw2-2:19.0.0-2145.g3be9a5c9.el9.x86_64 128/198 2024-03-28T17:52:01.913 INFO:teuthology.orchestra.run.smithi134.stdout: Running scriptlet: librgw2-2:19.0.0-2145.g3be9a5c9.el9.x86_64 128/198 2024-03-28T17:52:01.941 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-rgw-2:19.0.0-2145.g3be9a5c9.el9.x86_64 129/198 2024-03-28T17:52:01.989 INFO:teuthology.orchestra.run.smithi022.stdout: Running scriptlet: ceph-selinux-2:19.0.0-2145.g3be9a5c9.el9.x86_64 132/198 2024-03-28T17:52:01.990 INFO:teuthology.orchestra.run.smithi022.stdout:skipping the directory /sys 2024-03-28T17:52:01.990 INFO:teuthology.orchestra.run.smithi022.stdout:skipping the directory /proc 2024-03-28T17:52:01.990 INFO:teuthology.orchestra.run.smithi022.stdout:skipping the directory /mnt 2024-03-28T17:52:01.990 INFO:teuthology.orchestra.run.smithi022.stdout:skipping the directory /var/tmp 2024-03-28T17:52:01.990 INFO:teuthology.orchestra.run.smithi022.stdout:skipping the directory /home 2024-03-28T17:52:01.990 INFO:teuthology.orchestra.run.smithi022.stdout:skipping the directory /root 2024-03-28T17:52:01.990 INFO:teuthology.orchestra.run.smithi022.stdout:skipping the directory /tmp 2024-03-28T17:52:01.990 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-03-28T17:52:02.226 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : ceph-volume-2:19.0.0-2145.g3be9a5c9.el9.noarch 133/198 2024-03-28T17:52:02.239 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : ceph-osd-2:19.0.0-2145.g3be9a5c9.el9.x86_64 134/198 2024-03-28T17:52:02.276 INFO:teuthology.orchestra.run.smithi022.stdout: Running scriptlet: ceph-volume-2:19.0.0-2145.g3be9a5c9.el9.noarch 133/198 2024-03-28T17:52:02.276 INFO:teuthology.orchestra.run.smithi022.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-28T17:52:02.277 INFO:teuthology.orchestra.run.smithi022.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-03-28T17:52:02.277 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-03-28T17:52:02.316 INFO:teuthology.orchestra.run.smithi079.stdout: Running scriptlet: ceph-osd-2:19.0.0-2145.g3be9a5c9.el9.x86_64 134/198 2024-03-28T17:52:02.316 INFO:teuthology.orchestra.run.smithi079.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-28T17:52:02.316 INFO:teuthology.orchestra.run.smithi079.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-03-28T17:52:02.316 INFO:teuthology.orchestra.run.smithi079.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-03-28T17:52:02.316 INFO:teuthology.orchestra.run.smithi079.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-03-28T17:52:02.316 INFO:teuthology.orchestra.run.smithi079.stdout: 2024-03-28T17:52:02.564 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : ceph-mds-2:19.0.0-2145.g3be9a5c9.el9.x86_64 135/198 2024-03-28T17:52:02.629 INFO:teuthology.orchestra.run.smithi079.stdout: Running scriptlet: ceph-mds-2:19.0.0-2145.g3be9a5c9.el9.x86_64 135/198 2024-03-28T17:52:02.629 INFO:teuthology.orchestra.run.smithi079.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-28T17:52:02.629 INFO:teuthology.orchestra.run.smithi079.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-03-28T17:52:02.630 INFO:teuthology.orchestra.run.smithi079.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-03-28T17:52:02.630 INFO:teuthology.orchestra.run.smithi079.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-03-28T17:52:02.630 INFO:teuthology.orchestra.run.smithi079.stdout: 2024-03-28T17:52:03.123 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : ceph-mon-2:19.0.0-2145.g3be9a5c9.el9.x86_64 136/198 2024-03-28T17:52:03.194 INFO:teuthology.orchestra.run.smithi079.stdout: Running scriptlet: ceph-mon-2:19.0.0-2145.g3be9a5c9.el9.x86_64 136/198 2024-03-28T17:52:03.194 INFO:teuthology.orchestra.run.smithi079.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-28T17:52:03.194 INFO:teuthology.orchestra.run.smithi079.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-03-28T17:52:03.194 INFO:teuthology.orchestra.run.smithi079.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-03-28T17:52:03.194 INFO:teuthology.orchestra.run.smithi079.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-03-28T17:52:03.194 INFO:teuthology.orchestra.run.smithi079.stdout: 2024-03-28T17:52:03.318 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : automake-1.16.2-8.el9.noarch 137/198 2024-03-28T17:52:03.922 INFO:teuthology.orchestra.run.smithi079.stdout: Running scriptlet: cephadm-2:19.0.0-2145.g3be9a5c9.el9.noarch 138/198 2024-03-28T17:52:03.936 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : ceph-osd-2:19.0.0-2145.g3be9a5c9.el9.x86_64 134/198 2024-03-28T17:52:03.969 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : cephadm-2:19.0.0-2145.g3be9a5c9.el9.noarch 138/198 2024-03-28T17:52:04.023 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-2145.g3be9a5c9.e 139/198 2024-03-28T17:52:04.024 INFO:teuthology.orchestra.run.smithi022.stdout: Running scriptlet: ceph-osd-2:19.0.0-2145.g3be9a5c9.el9.x86_64 134/198 2024-03-28T17:52:04.024 INFO:teuthology.orchestra.run.smithi022.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-28T17:52:04.025 INFO:teuthology.orchestra.run.smithi022.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-03-28T17:52:04.025 INFO:teuthology.orchestra.run.smithi022.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-03-28T17:52:04.025 INFO:teuthology.orchestra.run.smithi022.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-03-28T17:52:04.025 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-03-28T17:52:04.123 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-2145.g3be9a5c9. 140/198 2024-03-28T17:52:04.156 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-2145.g3be9a5c9.el9.noa 141/198 2024-03-28T17:52:04.286 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : ceph-mds-2:19.0.0-2145.g3be9a5c9.el9.x86_64 135/198 2024-03-28T17:52:04.359 INFO:teuthology.orchestra.run.smithi022.stdout: Running scriptlet: ceph-mds-2:19.0.0-2145.g3be9a5c9.el9.x86_64 135/198 2024-03-28T17:52:04.359 INFO:teuthology.orchestra.run.smithi022.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-28T17:52:04.359 INFO:teuthology.orchestra.run.smithi022.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-03-28T17:52:04.359 INFO:teuthology.orchestra.run.smithi022.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-03-28T17:52:04.359 INFO:teuthology.orchestra.run.smithi022.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-03-28T17:52:04.359 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-03-28T17:52:04.486 INFO:teuthology.orchestra.run.smithi079.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-2145.g3be9a5c9.el9.noa 141/198 2024-03-28T17:52:04.524 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-2145.g3be9a5c9.el9.n 142/198 2024-03-28T17:52:04.639 INFO:teuthology.orchestra.run.smithi134.stdout: Running scriptlet: ceph-common-2:19.0.0-2145.g3be9a5c9.el9.x86_64 130/198 2024-03-28T17:52:04.689 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : ceph-common-2:19.0.0-2145.g3be9a5c9.el9.x86_64 130/198 2024-03-28T17:52:04.870 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : ceph-mon-2:19.0.0-2145.g3be9a5c9.el9.x86_64 136/198 2024-03-28T17:52:04.945 INFO:teuthology.orchestra.run.smithi022.stdout: Running scriptlet: ceph-mon-2:19.0.0-2145.g3be9a5c9.el9.x86_64 136/198 2024-03-28T17:52:04.945 INFO:teuthology.orchestra.run.smithi022.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-28T17:52:04.945 INFO:teuthology.orchestra.run.smithi022.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-03-28T17:52:04.945 INFO:teuthology.orchestra.run.smithi022.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-03-28T17:52:04.946 INFO:teuthology.orchestra.run.smithi022.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-03-28T17:52:04.946 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-03-28T17:52:05.073 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : automake-1.16.2-8.el9.noarch 137/198 2024-03-28T17:52:05.322 INFO:teuthology.orchestra.run.smithi134.stdout: Running scriptlet: ceph-common-2:19.0.0-2145.g3be9a5c9.el9.x86_64 130/198 2024-03-28T17:52:05.367 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : ceph-base-2:19.0.0-2145.g3be9a5c9.el9.x86_64 131/198 2024-03-28T17:52:05.528 INFO:teuthology.orchestra.run.smithi134.stdout: Running scriptlet: ceph-base-2:19.0.0-2145.g3be9a5c9.el9.x86_64 131/198 2024-03-28T17:52:05.528 INFO:teuthology.orchestra.run.smithi134.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-03-28T17:52:05.528 INFO:teuthology.orchestra.run.smithi134.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-03-28T17:52:05.528 INFO:teuthology.orchestra.run.smithi134.stdout: 2024-03-28T17:52:05.576 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : ceph-selinux-2:19.0.0-2145.g3be9a5c9.el9.x86_64 132/198 2024-03-28T17:52:05.644 INFO:teuthology.orchestra.run.smithi079.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-2145.g3be9a5c9.el9.n 142/198 2024-03-28T17:52:05.660 INFO:teuthology.orchestra.run.smithi022.stdout: Running scriptlet: cephadm-2:19.0.0-2145.g3be9a5c9.el9.noarch 138/198 2024-03-28T17:52:05.674 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-2145.g3be 143/198 2024-03-28T17:52:05.707 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : cephadm-2:19.0.0-2145.g3be9a5c9.el9.noarch 138/198 2024-03-28T17:52:05.765 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-2145.g3be9a5c9.e 139/198 2024-03-28T17:52:05.814 INFO:teuthology.orchestra.run.smithi079.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-2145.g3be 143/198 2024-03-28T17:52:05.877 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-2145.g3be9a5c9. 140/198 2024-03-28T17:52:05.919 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-2145.g3be9a5c9.el9.noa 141/198 2024-03-28T17:52:05.988 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-2145.g3be9a5c9.el 144/198 2024-03-28T17:52:06.027 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : ceph-mgr-2:19.0.0-2145.g3be9a5c9.el9.x86_64 145/198 2024-03-28T17:52:06.122 INFO:teuthology.orchestra.run.smithi079.stdout: Running scriptlet: ceph-mgr-2:19.0.0-2145.g3be9a5c9.el9.x86_64 145/198 2024-03-28T17:52:06.122 INFO:teuthology.orchestra.run.smithi079.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-28T17:52:06.122 INFO:teuthology.orchestra.run.smithi079.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-03-28T17:52:06.122 INFO:teuthology.orchestra.run.smithi079.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-03-28T17:52:06.122 INFO:teuthology.orchestra.run.smithi079.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-03-28T17:52:06.122 INFO:teuthology.orchestra.run.smithi079.stdout: 2024-03-28T17:52:06.178 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : ceph-mgr-rook-2:19.0.0-2145.g3be9a5c9.el9.noarch 146/198 2024-03-28T17:52:06.217 INFO:teuthology.orchestra.run.smithi079.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-2145.g3be9a5c9.el9.noarch 146/198 2024-03-28T17:52:06.272 INFO:teuthology.orchestra.run.smithi022.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-2145.g3be9a5c9.el9.noa 141/198 2024-03-28T17:52:06.310 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : ceph-2:19.0.0-2145.g3be9a5c9.el9.x86_64 147/198 2024-03-28T17:52:06.311 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-2145.g3be9a5c9.el9.n 142/198 2024-03-28T17:52:07.260 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : libtool-2.4.6-45.el9.x86_64 148/198 2024-03-28T17:52:07.297 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : ceph-radosgw-2:19.0.0-2145.g3be9a5c9.el9.x86_64 149/198 2024-03-28T17:52:07.377 INFO:teuthology.orchestra.run.smithi079.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-2145.g3be9a5c9.el9.x86_64 149/198 2024-03-28T17:52:07.377 INFO:teuthology.orchestra.run.smithi079.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-28T17:52:07.377 INFO:teuthology.orchestra.run.smithi079.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-03-28T17:52:07.377 INFO:teuthology.orchestra.run.smithi079.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-03-28T17:52:07.378 INFO:teuthology.orchestra.run.smithi079.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-03-28T17:52:07.378 INFO:teuthology.orchestra.run.smithi079.stdout: 2024-03-28T17:52:07.423 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-2145.g3be9a 150/198 2024-03-28T17:52:07.489 INFO:teuthology.orchestra.run.smithi022.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-2145.g3be9a5c9.el9.n 142/198 2024-03-28T17:52:07.507 INFO:teuthology.orchestra.run.smithi079.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-2145.g3be9a 150/198 2024-03-28T17:52:07.507 INFO:teuthology.orchestra.run.smithi079.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-28T17:52:07.507 INFO:teuthology.orchestra.run.smithi079.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-03-28T17:52:07.507 INFO:teuthology.orchestra.run.smithi079.stdout: 2024-03-28T17:52:07.520 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-2145.g3be 143/198 2024-03-28T17:52:07.582 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : cephfs-mirror-2:19.0.0-2145.g3be9a5c9.el9.x86_64 151/198 2024-03-28T17:52:07.669 INFO:teuthology.orchestra.run.smithi022.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-2145.g3be 143/198 2024-03-28T17:52:07.677 INFO:teuthology.orchestra.run.smithi079.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-2145.g3be9a5c9.el9.x86_64 151/198 2024-03-28T17:52:07.678 INFO:teuthology.orchestra.run.smithi079.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-28T17:52:07.678 INFO:teuthology.orchestra.run.smithi079.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-03-28T17:52:07.678 INFO:teuthology.orchestra.run.smithi079.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-03-28T17:52:07.678 INFO:teuthology.orchestra.run.smithi079.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-03-28T17:52:07.678 INFO:teuthology.orchestra.run.smithi079.stdout: 2024-03-28T17:52:07.850 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-2145.g3be9a5c9.el 144/198 2024-03-28T17:52:07.906 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : ceph-mgr-2:19.0.0-2145.g3be9a5c9.el9.x86_64 145/198 2024-03-28T17:52:07.975 INFO:teuthology.orchestra.run.smithi022.stdout: Running scriptlet: ceph-mgr-2:19.0.0-2145.g3be9a5c9.el9.x86_64 145/198 2024-03-28T17:52:07.975 INFO:teuthology.orchestra.run.smithi022.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-28T17:52:07.975 INFO:teuthology.orchestra.run.smithi022.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-03-28T17:52:07.975 INFO:teuthology.orchestra.run.smithi022.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-03-28T17:52:07.976 INFO:teuthology.orchestra.run.smithi022.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-03-28T17:52:07.976 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-03-28T17:52:08.031 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : ceph-mgr-rook-2:19.0.0-2145.g3be9a5c9.el9.noarch 146/198 2024-03-28T17:52:08.058 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : rbd-mirror-2:19.0.0-2145.g3be9a5c9.el9.x86_64 152/198 2024-03-28T17:52:08.074 INFO:teuthology.orchestra.run.smithi022.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-2145.g3be9a5c9.el9.noarch 146/198 2024-03-28T17:52:08.134 INFO:teuthology.orchestra.run.smithi079.stdout: Running scriptlet: rbd-mirror-2:19.0.0-2145.g3be9a5c9.el9.x86_64 152/198 2024-03-28T17:52:08.135 INFO:teuthology.orchestra.run.smithi079.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-28T17:52:08.135 INFO:teuthology.orchestra.run.smithi079.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-03-28T17:52:08.135 INFO:teuthology.orchestra.run.smithi079.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-03-28T17:52:08.135 INFO:teuthology.orchestra.run.smithi079.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-03-28T17:52:08.135 INFO:teuthology.orchestra.run.smithi079.stdout: 2024-03-28T17:52:08.170 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : ceph-2:19.0.0-2145.g3be9a5c9.el9.x86_64 147/198 2024-03-28T17:52:09.219 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : libtool-2.4.6-45.el9.x86_64 148/198 2024-03-28T17:52:09.250 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : ceph-radosgw-2:19.0.0-2145.g3be9a5c9.el9.x86_64 149/198 2024-03-28T17:52:09.322 INFO:teuthology.orchestra.run.smithi022.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-2145.g3be9a5c9.el9.x86_64 149/198 2024-03-28T17:52:09.322 INFO:teuthology.orchestra.run.smithi022.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-28T17:52:09.322 INFO:teuthology.orchestra.run.smithi022.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-03-28T17:52:09.322 INFO:teuthology.orchestra.run.smithi022.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-03-28T17:52:09.323 INFO:teuthology.orchestra.run.smithi022.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-03-28T17:52:09.323 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-03-28T17:52:09.384 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-2145.g3be9a 150/198 2024-03-28T17:52:09.457 INFO:teuthology.orchestra.run.smithi022.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-2145.g3be9a 150/198 2024-03-28T17:52:09.457 INFO:teuthology.orchestra.run.smithi022.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-28T17:52:09.457 INFO:teuthology.orchestra.run.smithi022.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-03-28T17:52:09.457 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-03-28T17:52:09.535 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : cephfs-mirror-2:19.0.0-2145.g3be9a5c9.el9.x86_64 151/198 2024-03-28T17:52:09.617 INFO:teuthology.orchestra.run.smithi022.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-2145.g3be9a5c9.el9.x86_64 151/198 2024-03-28T17:52:09.617 INFO:teuthology.orchestra.run.smithi022.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-28T17:52:09.617 INFO:teuthology.orchestra.run.smithi022.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-03-28T17:52:09.617 INFO:teuthology.orchestra.run.smithi022.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-03-28T17:52:09.618 INFO:teuthology.orchestra.run.smithi022.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-03-28T17:52:09.618 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-03-28T17:52:10.002 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : rbd-mirror-2:19.0.0-2145.g3be9a5c9.el9.x86_64 152/198 2024-03-28T17:52:10.079 INFO:teuthology.orchestra.run.smithi022.stdout: Running scriptlet: rbd-mirror-2:19.0.0-2145.g3be9a5c9.el9.x86_64 152/198 2024-03-28T17:52:10.079 INFO:teuthology.orchestra.run.smithi022.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-28T17:52:10.079 INFO:teuthology.orchestra.run.smithi022.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-03-28T17:52:10.080 INFO:teuthology.orchestra.run.smithi022.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-03-28T17:52:10.080 INFO:teuthology.orchestra.run.smithi022.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-03-28T17:52:10.080 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-03-28T17:52:12.930 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : ceph-test-2:19.0.0-2145.g3be9a5c9.el9.x86_64 153/198 2024-03-28T17:52:12.975 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : rbd-fuse-2:19.0.0-2145.g3be9a5c9.el9.x86_64 154/198 2024-03-28T17:52:13.012 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : rbd-nbd-2:19.0.0-2145.g3be9a5c9.el9.x86_64 155/198 2024-03-28T17:52:13.050 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : elfutils-libelf-devel-0.190-2.el9.x86_64 156/198 2024-03-28T17:52:13.323 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : cephfs-shell-2:19.0.0-2145.g3be9a5c9.el9.x86_64 157/198 2024-03-28T17:52:13.453 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : postgresql-contrib-13.14-1.el9.x86_64 158/198 2024-03-28T17:52:13.506 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : ceph-fuse-2:19.0.0-2145.g3be9a5c9.el9.x86_64 159/198 2024-03-28T17:52:13.561 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : xfsprogs-devel-6.3.0-1.el9.x86_64 160/198 2024-03-28T17:52:13.639 INFO:teuthology.orchestra.run.smithi079.stdout: Upgrading : xfsdump-3.1.12-4.el9.x86_64 161/198 2024-03-28T17:52:13.684 INFO:teuthology.orchestra.run.smithi079.stdout: Upgrading : util-linux-user-2.37.4-18.el9.x86_64 162/198 2024-03-28T17:52:13.718 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : libcephfs-devel-2:19.0.0-2145.g3be9a5c9.el9.x86_ 163/198 2024-03-28T17:52:13.756 INFO:teuthology.orchestra.run.smithi079.stdout: Upgrading : elfutils-debuginfod-client-0.190-2.el9.x86_64 164/198 2024-03-28T17:52:13.807 INFO:teuthology.orchestra.run.smithi079.stdout: Upgrading : NetworkManager-team-1:1.47.2-1.el9.x86_64 165/198 2024-03-28T17:52:13.892 INFO:teuthology.orchestra.run.smithi079.stdout: Upgrading : NetworkManager-tui-1:1.47.2-1.el9.x86_64 166/198 2024-03-28T17:52:14.009 INFO:teuthology.orchestra.run.smithi079.stdout: Upgrading : acl-2.3.1-4.el9.x86_64 167/198 2024-03-28T17:52:14.042 INFO:teuthology.orchestra.run.smithi079.stdout: Upgrading : libacl-devel-2.3.1-4.el9.x86_64 168/198 2024-03-28T17:52:14.953 INFO:teuthology.orchestra.run.smithi079.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 169/198 2024-03-28T17:52:14.997 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : postgresql-server-13.14-1.el9.x86_64 169/198 2024-03-28T17:52:15.053 INFO:teuthology.orchestra.run.smithi079.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 169/198 2024-03-28T17:52:15.176 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : ceph-test-2:19.0.0-2145.g3be9a5c9.el9.x86_64 153/198 2024-03-28T17:52:15.231 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : rbd-fuse-2:19.0.0-2145.g3be9a5c9.el9.x86_64 154/198 2024-03-28T17:52:15.266 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : rbd-nbd-2:19.0.0-2145.g3be9a5c9.el9.x86_64 155/198 2024-03-28T17:52:15.304 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : elfutils-libelf-devel-0.190-2.el9.x86_64 156/198 2024-03-28T17:52:15.593 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : cephfs-shell-2:19.0.0-2145.g3be9a5c9.el9.x86_64 157/198 2024-03-28T17:52:15.723 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : postgresql-contrib-13.14-1.el9.x86_64 158/198 2024-03-28T17:52:15.775 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : ceph-fuse-2:19.0.0-2145.g3be9a5c9.el9.x86_64 159/198 2024-03-28T17:52:15.871 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : xfsprogs-devel-6.3.0-1.el9.x86_64 160/198 2024-03-28T17:52:15.951 INFO:teuthology.orchestra.run.smithi022.stdout: Upgrading : xfsdump-3.1.12-4.el9.x86_64 161/198 2024-03-28T17:52:16.006 INFO:teuthology.orchestra.run.smithi022.stdout: Upgrading : util-linux-user-2.37.4-18.el9.x86_64 162/198 2024-03-28T17:52:16.049 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : libcephfs-devel-2:19.0.0-2145.g3be9a5c9.el9.x86_ 163/198 2024-03-28T17:52:16.100 INFO:teuthology.orchestra.run.smithi022.stdout: Upgrading : elfutils-debuginfod-client-0.190-2.el9.x86_64 164/198 2024-03-28T17:52:16.152 INFO:teuthology.orchestra.run.smithi022.stdout: Upgrading : NetworkManager-team-1:1.47.2-1.el9.x86_64 165/198 2024-03-28T17:52:16.242 INFO:teuthology.orchestra.run.smithi022.stdout: Upgrading : NetworkManager-tui-1:1.47.2-1.el9.x86_64 166/198 2024-03-28T17:52:16.369 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : cephfs-top-2:19.0.0-2145.g3be9a5c9.el9.noarch 170/198 2024-03-28T17:52:16.391 INFO:teuthology.orchestra.run.smithi022.stdout: Upgrading : acl-2.3.1-4.el9.x86_64 167/198 2024-03-28T17:52:16.434 INFO:teuthology.orchestra.run.smithi022.stdout: Upgrading : libacl-devel-2.3.1-4.el9.x86_64 168/198 2024-03-28T17:52:16.462 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : openssl-devel-1:3.0.7-27.el9.x86_64 171/198 2024-03-28T17:52:16.511 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 172/198 2024-03-28T17:52:16.639 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : flex-2.6.4-9.el9.x86_64 173/198 2024-03-28T17:52:16.673 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : bison-3.7.4-5.el9.x86_64 174/198 2024-03-28T17:52:16.707 INFO:teuthology.orchestra.run.smithi079.stdout: Cleanup : NetworkManager-tui-1:1.44.0-2.el9.x86_64 175/198 2024-03-28T17:52:16.741 INFO:teuthology.orchestra.run.smithi079.stdout: Cleanup : elfutils-debuginfod-client-0.189-3.el9.x86_64 176/198 2024-03-28T17:52:16.782 INFO:teuthology.orchestra.run.smithi079.stdout: Cleanup : elfutils-libs-0.189-3.el9.x86_64 177/198 2024-03-28T17:52:16.782 INFO:teuthology.orchestra.run.smithi079.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 178/198 2024-03-28T17:52:16.954 INFO:teuthology.orchestra.run.smithi079.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 178/198 2024-03-28T17:52:16.996 INFO:teuthology.orchestra.run.smithi079.stdout: Cleanup : openssl-1:3.0.7-24.el9.x86_64 179/198 2024-03-28T17:52:16.996 INFO:teuthology.orchestra.run.smithi079.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 180/198 2024-03-28T17:52:17.126 INFO:teuthology.orchestra.run.smithi079.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 180/198 2024-03-28T17:52:17.170 INFO:teuthology.orchestra.run.smithi079.stdout: Cleanup : xfsdump-3.1.12-3.el9.x86_64 181/198 2024-03-28T17:52:17.217 INFO:teuthology.orchestra.run.smithi079.stdout: Cleanup : xfsprogs-5.19.0-4.el9.x86_64 182/198 2024-03-28T17:52:17.250 INFO:teuthology.orchestra.run.smithi079.stdout: Cleanup : acl-2.3.1-3.el9.x86_64 183/198 2024-03-28T17:52:17.274 INFO:teuthology.orchestra.run.smithi079.stdout: Cleanup : libacl-devel-2.3.1-3.el9.x86_64 184/198 2024-03-28T17:52:17.342 INFO:teuthology.orchestra.run.smithi079.stdout: Cleanup : util-linux-user-2.37.4-15.el9.x86_64 185/198 2024-03-28T17:52:17.342 INFO:teuthology.orchestra.run.smithi022.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 169/198 2024-03-28T17:52:17.389 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : postgresql-server-13.14-1.el9.x86_64 169/198 2024-03-28T17:52:17.425 INFO:teuthology.orchestra.run.smithi079.stdout: Cleanup : util-linux-2.37.4-15.el9.x86_64 186/198 2024-03-28T17:52:17.455 INFO:teuthology.orchestra.run.smithi022.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 169/198 2024-03-28T17:52:17.471 INFO:teuthology.orchestra.run.smithi079.stdout: Cleanup : util-linux-core-2.37.4-15.el9.x86_64 187/198 2024-03-28T17:52:17.504 INFO:teuthology.orchestra.run.smithi079.stdout: Cleanup : libfdisk-2.37.4-15.el9.x86_64 188/198 2024-03-28T17:52:17.538 INFO:teuthology.orchestra.run.smithi079.stdout: Cleanup : libmount-2.37.4-15.el9.x86_64 189/198 2024-03-28T17:52:17.571 INFO:teuthology.orchestra.run.smithi079.stdout: Cleanup : libblkid-2.37.4-15.el9.x86_64 190/198 2024-03-28T17:52:17.605 INFO:teuthology.orchestra.run.smithi079.stdout: Cleanup : NetworkManager-team-1:1.44.0-2.el9.x86_64 191/198 2024-03-28T17:52:17.642 INFO:teuthology.orchestra.run.smithi079.stdout: Running scriptlet: NetworkManager-1:1.44.0-2.el9.x86_64 192/198 2024-03-28T17:52:17.642 INFO:teuthology.orchestra.run.smithi079.stdout: Cleanup : NetworkManager-1:1.44.0-2.el9.x86_64 192/198 2024-03-28T17:52:17.972 INFO:teuthology.orchestra.run.smithi079.stdout: Running scriptlet: NetworkManager-1:1.44.0-2.el9.x86_64 192/198 2024-03-28T17:52:17.972 INFO:teuthology.orchestra.run.smithi079.stdout: Cleanup : NetworkManager-libnm-1:1.44.0-2.el9.x86_64 193/198 2024-03-28T17:52:18.111 INFO:teuthology.orchestra.run.smithi079.stdout: Running scriptlet: NetworkManager-libnm-1:1.44.0-2.el9.x86_64 193/198 2024-03-28T17:52:18.142 INFO:teuthology.orchestra.run.smithi079.stdout: Cleanup : libuuid-2.37.4-15.el9.x86_64 194/198 2024-03-28T17:52:18.195 INFO:teuthology.orchestra.run.smithi079.stdout: Cleanup : libsmartcols-2.37.4-15.el9.x86_64 195/198 2024-03-28T17:52:18.228 INFO:teuthology.orchestra.run.smithi079.stdout: Cleanup : libacl-2.3.1-3.el9.x86_64 196/198 2024-03-28T17:52:18.268 INFO:teuthology.orchestra.run.smithi079.stdout: Cleanup : openssl-libs-1:3.0.7-24.el9.x86_64 197/198 2024-03-28T17:52:18.387 INFO:teuthology.orchestra.run.smithi079.stdout: Cleanup : elfutils-libelf-0.189-3.el9.x86_64 198/198 2024-03-28T17:52:18.498 INFO:teuthology.orchestra.run.smithi134.stdout: Running scriptlet: ceph-selinux-2:19.0.0-2145.g3be9a5c9.el9.x86_64 132/198 2024-03-28T17:52:18.498 INFO:teuthology.orchestra.run.smithi134.stdout:skipping the directory /sys 2024-03-28T17:52:18.498 INFO:teuthology.orchestra.run.smithi134.stdout:skipping the directory /proc 2024-03-28T17:52:18.498 INFO:teuthology.orchestra.run.smithi134.stdout:skipping the directory /mnt 2024-03-28T17:52:18.498 INFO:teuthology.orchestra.run.smithi134.stdout:skipping the directory /var/tmp 2024-03-28T17:52:18.498 INFO:teuthology.orchestra.run.smithi134.stdout:skipping the directory /home 2024-03-28T17:52:18.498 INFO:teuthology.orchestra.run.smithi134.stdout:skipping the directory /root 2024-03-28T17:52:18.498 INFO:teuthology.orchestra.run.smithi134.stdout:skipping the directory /tmp 2024-03-28T17:52:18.499 INFO:teuthology.orchestra.run.smithi134.stdout: 2024-03-28T17:52:18.652 INFO:teuthology.orchestra.run.smithi079.stdout: Running scriptlet: elfutils-libelf-0.189-3.el9.x86_64 198/198 2024-03-28T17:52:18.653 INFO:teuthology.orchestra.run.smithi079.stdout:Couldn't write '0' to 'kernel/yama/ptrace_scope', ignoring: No such file or directory 2024-03-28T17:52:18.653 INFO:teuthology.orchestra.run.smithi079.stdout: 2024-03-28T17:52:18.720 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : ceph-volume-2:19.0.0-2145.g3be9a5c9.el9.noarch 133/198 2024-03-28T17:52:18.770 INFO:teuthology.orchestra.run.smithi134.stdout: Running scriptlet: ceph-volume-2:19.0.0-2145.g3be9a5c9.el9.noarch 133/198 2024-03-28T17:52:18.771 INFO:teuthology.orchestra.run.smithi134.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-28T17:52:18.771 INFO:teuthology.orchestra.run.smithi134.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-03-28T17:52:18.771 INFO:teuthology.orchestra.run.smithi134.stdout: 2024-03-28T17:52:18.833 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : cephfs-top-2:19.0.0-2145.g3be9a5c9.el9.noarch 170/198 2024-03-28T17:52:18.953 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : openssl-devel-1:3.0.7-27.el9.x86_64 171/198 2024-03-28T17:52:19.014 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 172/198 2024-03-28T17:52:19.147 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : flex-2.6.4-9.el9.x86_64 173/198 2024-03-28T17:52:19.180 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : bison-3.7.4-5.el9.x86_64 174/198 2024-03-28T17:52:19.212 INFO:teuthology.orchestra.run.smithi022.stdout: Cleanup : NetworkManager-tui-1:1.44.0-2.el9.x86_64 175/198 2024-03-28T17:52:19.247 INFO:teuthology.orchestra.run.smithi022.stdout: Cleanup : elfutils-debuginfod-client-0.189-3.el9.x86_64 176/198 2024-03-28T17:52:19.291 INFO:teuthology.orchestra.run.smithi022.stdout: Cleanup : elfutils-libs-0.189-3.el9.x86_64 177/198 2024-03-28T17:52:19.291 INFO:teuthology.orchestra.run.smithi022.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 178/198 2024-03-28T17:52:19.495 INFO:teuthology.orchestra.run.smithi022.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 178/198 2024-03-28T17:52:19.537 INFO:teuthology.orchestra.run.smithi022.stdout: Cleanup : openssl-1:3.0.7-24.el9.x86_64 179/198 2024-03-28T17:52:19.537 INFO:teuthology.orchestra.run.smithi022.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 180/198 2024-03-28T17:52:19.725 INFO:teuthology.orchestra.run.smithi022.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 180/198 2024-03-28T17:52:19.810 INFO:teuthology.orchestra.run.smithi022.stdout: Cleanup : xfsdump-3.1.12-3.el9.x86_64 181/198 2024-03-28T17:52:19.866 INFO:teuthology.orchestra.run.smithi022.stdout: Cleanup : xfsprogs-5.19.0-4.el9.x86_64 182/198 2024-03-28T17:52:19.906 INFO:teuthology.orchestra.run.smithi022.stdout: Cleanup : acl-2.3.1-3.el9.x86_64 183/198 2024-03-28T17:52:19.939 INFO:teuthology.orchestra.run.smithi022.stdout: Cleanup : libacl-devel-2.3.1-3.el9.x86_64 184/198 2024-03-28T17:52:20.013 INFO:teuthology.orchestra.run.smithi022.stdout: Cleanup : util-linux-user-2.37.4-15.el9.x86_64 185/198 2024-03-28T17:52:20.096 INFO:teuthology.orchestra.run.smithi022.stdout: Cleanup : util-linux-2.37.4-15.el9.x86_64 186/198 2024-03-28T17:52:20.145 INFO:teuthology.orchestra.run.smithi022.stdout: Cleanup : util-linux-core-2.37.4-15.el9.x86_64 187/198 2024-03-28T17:52:20.186 INFO:teuthology.orchestra.run.smithi022.stdout: Cleanup : libfdisk-2.37.4-15.el9.x86_64 188/198 2024-03-28T17:52:20.253 INFO:teuthology.orchestra.run.smithi022.stdout: Cleanup : libmount-2.37.4-15.el9.x86_64 189/198 2024-03-28T17:52:20.286 INFO:teuthology.orchestra.run.smithi022.stdout: Cleanup : libblkid-2.37.4-15.el9.x86_64 190/198 2024-03-28T17:52:20.319 INFO:teuthology.orchestra.run.smithi022.stdout: Cleanup : NetworkManager-team-1:1.44.0-2.el9.x86_64 191/198 2024-03-28T17:52:20.357 INFO:teuthology.orchestra.run.smithi022.stdout: Running scriptlet: NetworkManager-1:1.44.0-2.el9.x86_64 192/198 2024-03-28T17:52:20.357 INFO:teuthology.orchestra.run.smithi022.stdout: Cleanup : NetworkManager-1:1.44.0-2.el9.x86_64 192/198 2024-03-28T17:52:20.439 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : ceph-osd-2:19.0.0-2145.g3be9a5c9.el9.x86_64 134/198 2024-03-28T17:52:20.517 INFO:teuthology.orchestra.run.smithi134.stdout: Running scriptlet: ceph-osd-2:19.0.0-2145.g3be9a5c9.el9.x86_64 134/198 2024-03-28T17:52:20.517 INFO:teuthology.orchestra.run.smithi134.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-28T17:52:20.517 INFO:teuthology.orchestra.run.smithi134.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-03-28T17:52:20.517 INFO:teuthology.orchestra.run.smithi134.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-03-28T17:52:20.517 INFO:teuthology.orchestra.run.smithi134.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-03-28T17:52:20.517 INFO:teuthology.orchestra.run.smithi134.stdout: 2024-03-28T17:52:20.703 INFO:teuthology.orchestra.run.smithi022.stdout: Running scriptlet: NetworkManager-1:1.44.0-2.el9.x86_64 192/198 2024-03-28T17:52:20.704 INFO:teuthology.orchestra.run.smithi022.stdout: Cleanup : NetworkManager-libnm-1:1.44.0-2.el9.x86_64 193/198 2024-03-28T17:52:20.773 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : ceph-mds-2:19.0.0-2145.g3be9a5c9.el9.x86_64 135/198 2024-03-28T17:52:20.843 INFO:teuthology.orchestra.run.smithi134.stdout: Running scriptlet: ceph-mds-2:19.0.0-2145.g3be9a5c9.el9.x86_64 135/198 2024-03-28T17:52:20.843 INFO:teuthology.orchestra.run.smithi134.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-28T17:52:20.843 INFO:teuthology.orchestra.run.smithi134.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-03-28T17:52:20.843 INFO:teuthology.orchestra.run.smithi134.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-03-28T17:52:20.843 INFO:teuthology.orchestra.run.smithi134.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-03-28T17:52:20.843 INFO:teuthology.orchestra.run.smithi134.stdout: 2024-03-28T17:52:20.884 INFO:teuthology.orchestra.run.smithi022.stdout: Running scriptlet: NetworkManager-libnm-1:1.44.0-2.el9.x86_64 193/198 2024-03-28T17:52:20.914 INFO:teuthology.orchestra.run.smithi022.stdout: Cleanup : libuuid-2.37.4-15.el9.x86_64 194/198 2024-03-28T17:52:20.970 INFO:teuthology.orchestra.run.smithi022.stdout: Cleanup : libsmartcols-2.37.4-15.el9.x86_64 195/198 2024-03-28T17:52:21.009 INFO:teuthology.orchestra.run.smithi022.stdout: Cleanup : libacl-2.3.1-3.el9.x86_64 196/198 2024-03-28T17:52:21.057 INFO:teuthology.orchestra.run.smithi022.stdout: Cleanup : openssl-libs-1:3.0.7-24.el9.x86_64 197/198 2024-03-28T17:52:21.205 INFO:teuthology.orchestra.run.smithi022.stdout: Cleanup : elfutils-libelf-0.189-3.el9.x86_64 198/198 2024-03-28T17:52:21.332 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : ceph-mon-2:19.0.0-2145.g3be9a5c9.el9.x86_64 136/198 2024-03-28T17:52:21.401 INFO:teuthology.orchestra.run.smithi134.stdout: Running scriptlet: ceph-mon-2:19.0.0-2145.g3be9a5c9.el9.x86_64 136/198 2024-03-28T17:52:21.401 INFO:teuthology.orchestra.run.smithi134.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-28T17:52:21.401 INFO:teuthology.orchestra.run.smithi134.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-03-28T17:52:21.401 INFO:teuthology.orchestra.run.smithi134.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-03-28T17:52:21.401 INFO:teuthology.orchestra.run.smithi134.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-03-28T17:52:21.401 INFO:teuthology.orchestra.run.smithi134.stdout: 2024-03-28T17:52:21.476 INFO:teuthology.orchestra.run.smithi022.stdout: Running scriptlet: elfutils-libelf-0.189-3.el9.x86_64 198/198 2024-03-28T17:52:21.476 INFO:teuthology.orchestra.run.smithi022.stdout:Couldn't write '0' to 'kernel/yama/ptrace_scope', ignoring: No such file or directory 2024-03-28T17:52:21.476 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-03-28T17:52:21.509 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : automake-1.16.2-8.el9.noarch 137/198 2024-03-28T17:52:22.113 INFO:teuthology.orchestra.run.smithi134.stdout: Running scriptlet: cephadm-2:19.0.0-2145.g3be9a5c9.el9.noarch 138/198 2024-03-28T17:52:22.159 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : cephadm-2:19.0.0-2145.g3be9a5c9.el9.noarch 138/198 2024-03-28T17:52:22.209 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-2145.g3be9a5c9.e 139/198 2024-03-28T17:52:22.335 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-2145.g3be9a5c9. 140/198 2024-03-28T17:52:22.372 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-2145.g3be9a5c9.el9.noa 141/198 2024-03-28T17:52:22.702 INFO:teuthology.orchestra.run.smithi134.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-2145.g3be9a5c9.el9.noa 141/198 2024-03-28T17:52:22.739 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-2145.g3be9a5c9.el9.n 142/198 2024-03-28T17:52:23.740 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : ceph-2:19.0.0-2145.g3be9a5c9.el9.x86_64 1/198 2024-03-28T17:52:23.740 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : ceph-base-2:19.0.0-2145.g3be9a5c9.el9.x86_64 2/198 2024-03-28T17:52:23.740 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : ceph-common-2:19.0.0-2145.g3be9a5c9.el9.x86_64 3/198 2024-03-28T17:52:23.741 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : ceph-fuse-2:19.0.0-2145.g3be9a5c9.el9.x86_64 4/198 2024-03-28T17:52:23.741 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-2145.g3be9a 5/198 2024-03-28T17:52:23.741 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : ceph-mds-2:19.0.0-2145.g3be9a5c9.el9.x86_64 6/198 2024-03-28T17:52:23.741 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : ceph-mgr-2:19.0.0-2145.g3be9a5c9.el9.x86_64 7/198 2024-03-28T17:52:23.741 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : ceph-mon-2:19.0.0-2145.g3be9a5c9.el9.x86_64 8/198 2024-03-28T17:52:23.741 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : ceph-osd-2:19.0.0-2145.g3be9a5c9.el9.x86_64 9/198 2024-03-28T17:52:23.741 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : ceph-radosgw-2:19.0.0-2145.g3be9a5c9.el9.x86_64 10/198 2024-03-28T17:52:23.741 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : ceph-selinux-2:19.0.0-2145.g3be9a5c9.el9.x86_64 11/198 2024-03-28T17:52:23.741 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : ceph-test-2:19.0.0-2145.g3be9a5c9.el9.x86_64 12/198 2024-03-28T17:52:23.741 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : cephfs-mirror-2:19.0.0-2145.g3be9a5c9.el9.x86_64 13/198 2024-03-28T17:52:23.741 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : cephfs-shell-2:19.0.0-2145.g3be9a5c9.el9.x86_64 14/198 2024-03-28T17:52:23.741 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : libcephfs-devel-2:19.0.0-2145.g3be9a5c9.el9.x86_ 15/198 2024-03-28T17:52:23.741 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : libcephfs2-2:19.0.0-2145.g3be9a5c9.el9.x86_64 16/198 2024-03-28T17:52:23.742 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : libcephsqlite-2:19.0.0-2145.g3be9a5c9.el9.x86_64 17/198 2024-03-28T17:52:23.742 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : librados-devel-2:19.0.0-2145.g3be9a5c9.el9.x86_6 18/198 2024-03-28T17:52:23.742 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : libradosstriper1-2:19.0.0-2145.g3be9a5c9.el9.x86 19/198 2024-03-28T17:52:23.742 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : librgw2-2:19.0.0-2145.g3be9a5c9.el9.x86_64 20/198 2024-03-28T17:52:23.742 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : python3-ceph-argparse-2:19.0.0-2145.g3be9a5c9.el 21/198 2024-03-28T17:52:23.742 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : python3-ceph-common-2:19.0.0-2145.g3be9a5c9.el9. 22/198 2024-03-28T17:52:23.742 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : python3-cephfs-2:19.0.0-2145.g3be9a5c9.el9.x86_6 23/198 2024-03-28T17:52:23.742 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : python3-rados-2:19.0.0-2145.g3be9a5c9.el9.x86_64 24/198 2024-03-28T17:52:23.742 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : python3-rbd-2:19.0.0-2145.g3be9a5c9.el9.x86_64 25/198 2024-03-28T17:52:23.742 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : python3-rgw-2:19.0.0-2145.g3be9a5c9.el9.x86_64 26/198 2024-03-28T17:52:23.742 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : rbd-fuse-2:19.0.0-2145.g3be9a5c9.el9.x86_64 27/198 2024-03-28T17:52:23.758 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : rbd-mirror-2:19.0.0-2145.g3be9a5c9.el9.x86_64 28/198 2024-03-28T17:52:23.758 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : rbd-nbd-2:19.0.0-2145.g3be9a5c9.el9.x86_64 29/198 2024-03-28T17:52:23.759 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-2145.g3be9a5c9. 30/198 2024-03-28T17:52:23.759 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-2145.g3be9a5c9.el9.noa 31/198 2024-03-28T17:52:23.759 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-2145.g3be9a5c9.el9.n 32/198 2024-03-28T17:52:23.759 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-2145.g3be 33/198 2024-03-28T17:52:23.759 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-2145.g3be9a5c9.el 34/198 2024-03-28T17:52:23.759 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : ceph-mgr-rook-2:19.0.0-2145.g3be9a5c9.el9.noarch 35/198 2024-03-28T17:52:23.759 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-2145.g3be9a5c9.e 36/198 2024-03-28T17:52:23.759 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : ceph-volume-2:19.0.0-2145.g3be9a5c9.el9.noarch 37/198 2024-03-28T17:52:23.759 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : cephadm-2:19.0.0-2145.g3be9a5c9.el9.noarch 38/198 2024-03-28T17:52:23.759 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : cephfs-top-2:19.0.0-2145.g3be9a5c9.el9.noarch 39/198 2024-03-28T17:52:23.759 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : automake-1.16.2-8.el9.noarch 40/198 2024-03-28T17:52:23.759 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : bison-3.7.4-5.el9.x86_64 41/198 2024-03-28T17:52:23.759 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 42/198 2024-03-28T17:52:23.760 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : flex-2.6.4-9.el9.x86_64 43/198 2024-03-28T17:52:23.760 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 44/198 2024-03-28T17:52:23.760 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 45/198 2024-03-28T17:52:23.760 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 46/198 2024-03-28T17:52:23.760 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 47/198 2024-03-28T17:52:23.760 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 48/198 2024-03-28T17:52:23.760 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 49/198 2024-03-28T17:52:23.760 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 50/198 2024-03-28T17:52:23.760 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 51/198 2024-03-28T17:52:23.760 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 52/198 2024-03-28T17:52:23.760 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 53/198 2024-03-28T17:52:23.760 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : lua-5.4.4-4.el9.x86_64 54/198 2024-03-28T17:52:23.760 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : oniguruma-6.9.6-1.el9.5.x86_64 55/198 2024-03-28T17:52:23.761 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 56/198 2024-03-28T17:52:23.761 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 57/198 2024-03-28T17:52:23.761 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 58/198 2024-03-28T17:52:23.761 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 59/198 2024-03-28T17:52:23.761 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 60/198 2024-03-28T17:52:23.761 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 61/198 2024-03-28T17:52:23.761 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 62/198 2024-03-28T17:52:23.761 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 63/198 2024-03-28T17:52:23.761 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 64/198 2024-03-28T17:52:23.761 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 65/198 2024-03-28T17:52:23.761 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 66/198 2024-03-28T17:52:23.761 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 67/198 2024-03-28T17:52:23.761 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 68/198 2024-03-28T17:52:23.762 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 69/198 2024-03-28T17:52:23.762 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 70/198 2024-03-28T17:52:23.762 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 71/198 2024-03-28T17:52:23.762 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 72/198 2024-03-28T17:52:23.762 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 73/198 2024-03-28T17:52:23.762 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 74/198 2024-03-28T17:52:23.762 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 75/198 2024-03-28T17:52:23.762 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 76/198 2024-03-28T17:52:23.762 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 77/198 2024-03-28T17:52:23.762 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 78/198 2024-03-28T17:52:23.762 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 79/198 2024-03-28T17:52:23.762 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 80/198 2024-03-28T17:52:23.762 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : zlib-devel-1.2.11-40.el9.x86_64 81/198 2024-03-28T17:52:23.763 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 82/198 2024-03-28T17:52:23.763 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 83/198 2024-03-28T17:52:23.763 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 84/198 2024-03-28T17:52:23.763 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : python3-ply-3.11-14.el9.noarch 85/198 2024-03-28T17:52:23.763 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 86/198 2024-03-28T17:52:23.763 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 87/198 2024-03-28T17:52:23.763 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : jq-1.6-16.el9.x86_64 88/198 2024-03-28T17:52:23.763 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 89/198 2024-03-28T17:52:23.763 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 90/198 2024-03-28T17:52:23.763 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 91/198 2024-03-28T17:52:23.763 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : elfutils-libelf-devel-0.190-2.el9.x86_64 92/198 2024-03-28T17:52:23.763 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 93/198 2024-03-28T17:52:23.763 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : openssl-devel-1:3.0.7-27.el9.x86_64 94/198 2024-03-28T17:52:23.763 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : postgresql-13.14-1.el9.x86_64 95/198 2024-03-28T17:52:23.763 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : postgresql-contrib-13.14-1.el9.x86_64 96/198 2024-03-28T17:52:23.764 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : postgresql-private-libs-13.14-1.el9.x86_64 97/198 2024-03-28T17:52:23.764 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : postgresql-server-13.14-1.el9.x86_64 98/198 2024-03-28T17:52:23.764 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : python3-devel-3.9.17-2.el9.x86_64 99/198 2024-03-28T17:52:23.764 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 100/198 2024-03-28T17:52:23.764 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 101/198 2024-03-28T17:52:23.764 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 102/198 2024-03-28T17:52:23.764 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 103/198 2024-03-28T17:52:23.764 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : xfsprogs-devel-6.3.0-1.el9.x86_64 104/198 2024-03-28T17:52:23.764 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 105/198 2024-03-28T17:52:23.764 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 106/198 2024-03-28T17:52:23.764 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 107/198 2024-03-28T17:52:23.764 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 108/198 2024-03-28T17:52:23.764 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 109/198 2024-03-28T17:52:23.764 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : libarrow-9.0.0-10.el9.x86_64 110/198 2024-03-28T17:52:23.764 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : libarrow-doc-9.0.0-10.el9.noarch 111/198 2024-03-28T17:52:23.765 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 112/198 2024-03-28T17:52:23.765 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 113/198 2024-03-28T17:52:23.765 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : luarocks-3.9.2-1.el9.noarch 114/198 2024-03-28T17:52:23.765 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : parquet-libs-9.0.0-10.el9.x86_64 115/198 2024-03-28T17:52:23.765 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 116/198 2024-03-28T17:52:23.765 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 117/198 2024-03-28T17:52:23.765 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 118/198 2024-03-28T17:52:23.765 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 119/198 2024-03-28T17:52:23.765 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 120/198 2024-03-28T17:52:23.765 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 121/198 2024-03-28T17:52:23.765 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 122/198 2024-03-28T17:52:23.765 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : python3-colorama-0.4.4-12.el9.noarch 123/198 2024-03-28T17:52:23.765 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : python3-google-auth-1:2.28.2-1.el9.noarch 124/198 2024-03-28T17:52:23.765 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 125/198 2024-03-28T17:52:23.765 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 126/198 2024-03-28T17:52:23.765 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 127/198 2024-03-28T17:52:23.766 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 128/198 2024-03-28T17:52:23.766 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 129/198 2024-03-28T17:52:23.766 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 130/198 2024-03-28T17:52:23.766 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 131/198 2024-03-28T17:52:23.766 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 132/198 2024-03-28T17:52:23.766 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 133/198 2024-03-28T17:52:23.766 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 134/198 2024-03-28T17:52:23.766 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 135/198 2024-03-28T17:52:23.766 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 136/198 2024-03-28T17:52:23.766 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 137/198 2024-03-28T17:52:23.766 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 138/198 2024-03-28T17:52:23.766 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 139/198 2024-03-28T17:52:23.766 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 140/198 2024-03-28T17:52:23.766 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 141/198 2024-03-28T17:52:23.767 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 142/198 2024-03-28T17:52:23.767 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 143/198 2024-03-28T17:52:23.767 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 144/198 2024-03-28T17:52:23.767 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 145/198 2024-03-28T17:52:23.767 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 146/198 2024-03-28T17:52:23.767 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 147/198 2024-03-28T17:52:23.767 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 148/198 2024-03-28T17:52:23.767 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : re2-1:20211101-3.el9.x86_64 149/198 2024-03-28T17:52:23.767 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 150/198 2024-03-28T17:52:23.767 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : librados2-2:19.0.0-2145.g3be9a5c9.el9.x86_64 151/198 2024-03-28T17:52:23.767 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 152/198 2024-03-28T17:52:23.767 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : librbd1-2:19.0.0-2145.g3be9a5c9.el9.x86_64 153/198 2024-03-28T17:52:23.767 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 154/198 2024-03-28T17:52:23.767 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : xfsdump-3.1.12-4.el9.x86_64 155/198 2024-03-28T17:52:23.768 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : xfsdump-3.1.12-3.el9.x86_64 156/198 2024-03-28T17:52:23.768 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : NetworkManager-1:1.47.2-1.el9.x86_64 157/198 2024-03-28T17:52:23.768 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : NetworkManager-1:1.44.0-2.el9.x86_64 158/198 2024-03-28T17:52:23.768 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : NetworkManager-libnm-1:1.47.2-1.el9.x86_64 159/198 2024-03-28T17:52:23.768 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : NetworkManager-libnm-1:1.44.0-2.el9.x86_64 160/198 2024-03-28T17:52:23.768 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : NetworkManager-team-1:1.47.2-1.el9.x86_64 161/198 2024-03-28T17:52:23.768 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : NetworkManager-team-1:1.44.0-2.el9.x86_64 162/198 2024-03-28T17:52:23.768 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : NetworkManager-tui-1:1.47.2-1.el9.x86_64 163/198 2024-03-28T17:52:23.768 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : NetworkManager-tui-1:1.44.0-2.el9.x86_64 164/198 2024-03-28T17:52:23.768 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : acl-2.3.1-4.el9.x86_64 165/198 2024-03-28T17:52:23.768 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : acl-2.3.1-3.el9.x86_64 166/198 2024-03-28T17:52:23.768 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : elfutils-debuginfod-client-0.190-2.el9.x86_64 167/198 2024-03-28T17:52:23.768 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : elfutils-debuginfod-client-0.189-3.el9.x86_64 168/198 2024-03-28T17:52:23.768 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : elfutils-libelf-0.190-2.el9.x86_64 169/198 2024-03-28T17:52:23.768 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : elfutils-libelf-0.189-3.el9.x86_64 170/198 2024-03-28T17:52:23.769 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : elfutils-libs-0.190-2.el9.x86_64 171/198 2024-03-28T17:52:23.769 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : elfutils-libs-0.189-3.el9.x86_64 172/198 2024-03-28T17:52:23.769 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : libacl-2.3.1-4.el9.x86_64 173/198 2024-03-28T17:52:23.769 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : libacl-2.3.1-3.el9.x86_64 174/198 2024-03-28T17:52:23.769 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : libblkid-2.37.4-18.el9.x86_64 175/198 2024-03-28T17:52:23.769 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : libblkid-2.37.4-15.el9.x86_64 176/198 2024-03-28T17:52:23.769 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : libfdisk-2.37.4-18.el9.x86_64 177/198 2024-03-28T17:52:23.769 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : libfdisk-2.37.4-15.el9.x86_64 178/198 2024-03-28T17:52:23.769 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : libmount-2.37.4-18.el9.x86_64 179/198 2024-03-28T17:52:23.769 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : libmount-2.37.4-15.el9.x86_64 180/198 2024-03-28T17:52:23.769 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : libsmartcols-2.37.4-18.el9.x86_64 181/198 2024-03-28T17:52:23.769 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : libsmartcols-2.37.4-15.el9.x86_64 182/198 2024-03-28T17:52:23.769 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : libuuid-2.37.4-18.el9.x86_64 183/198 2024-03-28T17:52:23.769 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : libuuid-2.37.4-15.el9.x86_64 184/198 2024-03-28T17:52:23.769 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : openssl-1:3.0.7-27.el9.x86_64 185/198 2024-03-28T17:52:23.770 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : openssl-1:3.0.7-24.el9.x86_64 186/198 2024-03-28T17:52:23.770 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : openssl-libs-1:3.0.7-27.el9.x86_64 187/198 2024-03-28T17:52:23.770 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : openssl-libs-1:3.0.7-24.el9.x86_64 188/198 2024-03-28T17:52:23.770 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : util-linux-2.37.4-18.el9.x86_64 189/198 2024-03-28T17:52:23.770 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : util-linux-2.37.4-15.el9.x86_64 190/198 2024-03-28T17:52:23.770 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : util-linux-core-2.37.4-18.el9.x86_64 191/198 2024-03-28T17:52:23.770 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : util-linux-core-2.37.4-15.el9.x86_64 192/198 2024-03-28T17:52:23.770 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : util-linux-user-2.37.4-18.el9.x86_64 193/198 2024-03-28T17:52:23.770 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : util-linux-user-2.37.4-15.el9.x86_64 194/198 2024-03-28T17:52:23.770 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : xfsprogs-6.3.0-1.el9.x86_64 195/198 2024-03-28T17:52:23.770 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : xfsprogs-5.19.0-4.el9.x86_64 196/198 2024-03-28T17:52:23.770 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : libacl-devel-2.3.1-4.el9.x86_64 197/198 2024-03-28T17:52:23.839 INFO:teuthology.orchestra.run.smithi134.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-2145.g3be9a5c9.el9.n 142/198 2024-03-28T17:52:23.864 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-2145.g3be 143/198 2024-03-28T17:52:24.002 INFO:teuthology.orchestra.run.smithi134.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-2145.g3be 143/198 2024-03-28T17:52:24.177 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-2145.g3be9a5c9.el 144/198 2024-03-28T17:52:24.225 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : ceph-mgr-2:19.0.0-2145.g3be9a5c9.el9.x86_64 145/198 2024-03-28T17:52:24.294 INFO:teuthology.orchestra.run.smithi134.stdout: Running scriptlet: ceph-mgr-2:19.0.0-2145.g3be9a5c9.el9.x86_64 145/198 2024-03-28T17:52:24.294 INFO:teuthology.orchestra.run.smithi134.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-28T17:52:24.294 INFO:teuthology.orchestra.run.smithi134.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-03-28T17:52:24.295 INFO:teuthology.orchestra.run.smithi134.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-03-28T17:52:24.295 INFO:teuthology.orchestra.run.smithi134.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-03-28T17:52:24.295 INFO:teuthology.orchestra.run.smithi134.stdout: 2024-03-28T17:52:24.384 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : ceph-mgr-rook-2:19.0.0-2145.g3be9a5c9.el9.noarch 146/198 2024-03-28T17:52:24.422 INFO:teuthology.orchestra.run.smithi134.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-2145.g3be9a5c9.el9.noarch 146/198 2024-03-28T17:52:24.521 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : ceph-2:19.0.0-2145.g3be9a5c9.el9.x86_64 147/198 2024-03-28T17:52:25.486 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : libtool-2.4.6-45.el9.x86_64 148/198 2024-03-28T17:52:25.519 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : ceph-radosgw-2:19.0.0-2145.g3be9a5c9.el9.x86_64 149/198 2024-03-28T17:52:25.589 INFO:teuthology.orchestra.run.smithi134.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-2145.g3be9a5c9.el9.x86_64 149/198 2024-03-28T17:52:25.589 INFO:teuthology.orchestra.run.smithi134.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-28T17:52:25.589 INFO:teuthology.orchestra.run.smithi134.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-03-28T17:52:25.589 INFO:teuthology.orchestra.run.smithi134.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-03-28T17:52:25.589 INFO:teuthology.orchestra.run.smithi134.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-03-28T17:52:25.590 INFO:teuthology.orchestra.run.smithi134.stdout: 2024-03-28T17:52:25.645 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-2145.g3be9a 150/198 2024-03-28T17:52:25.713 INFO:teuthology.orchestra.run.smithi134.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-2145.g3be9a 150/198 2024-03-28T17:52:25.713 INFO:teuthology.orchestra.run.smithi134.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-28T17:52:25.713 INFO:teuthology.orchestra.run.smithi134.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-03-28T17:52:25.713 INFO:teuthology.orchestra.run.smithi134.stdout: 2024-03-28T17:52:25.787 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : cephfs-mirror-2:19.0.0-2145.g3be9a5c9.el9.x86_64 151/198 2024-03-28T17:52:25.862 INFO:teuthology.orchestra.run.smithi134.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-2145.g3be9a5c9.el9.x86_64 151/198 2024-03-28T17:52:25.862 INFO:teuthology.orchestra.run.smithi134.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-28T17:52:25.862 INFO:teuthology.orchestra.run.smithi134.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-03-28T17:52:25.862 INFO:teuthology.orchestra.run.smithi134.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-03-28T17:52:25.862 INFO:teuthology.orchestra.run.smithi134.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-03-28T17:52:25.862 INFO:teuthology.orchestra.run.smithi134.stdout: 2024-03-28T17:52:26.213 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : rbd-mirror-2:19.0.0-2145.g3be9a5c9.el9.x86_64 152/198 2024-03-28T17:52:26.292 INFO:teuthology.orchestra.run.smithi134.stdout: Running scriptlet: rbd-mirror-2:19.0.0-2145.g3be9a5c9.el9.x86_64 152/198 2024-03-28T17:52:26.292 INFO:teuthology.orchestra.run.smithi134.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-28T17:52:26.292 INFO:teuthology.orchestra.run.smithi134.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-03-28T17:52:26.292 INFO:teuthology.orchestra.run.smithi134.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-03-28T17:52:26.292 INFO:teuthology.orchestra.run.smithi134.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-03-28T17:52:26.292 INFO:teuthology.orchestra.run.smithi134.stdout: 2024-03-28T17:52:26.327 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : libacl-devel-2.3.1-3.el9.x86_64 198/198 2024-03-28T17:52:26.327 INFO:teuthology.orchestra.run.smithi079.stdout: 2024-03-28T17:52:26.327 INFO:teuthology.orchestra.run.smithi079.stdout:Upgraded: 2024-03-28T17:52:26.327 INFO:teuthology.orchestra.run.smithi079.stdout: NetworkManager-1:1.47.2-1.el9.x86_64 2024-03-28T17:52:26.327 INFO:teuthology.orchestra.run.smithi079.stdout: NetworkManager-libnm-1:1.47.2-1.el9.x86_64 2024-03-28T17:52:26.327 INFO:teuthology.orchestra.run.smithi079.stdout: NetworkManager-team-1:1.47.2-1.el9.x86_64 2024-03-28T17:52:26.327 INFO:teuthology.orchestra.run.smithi079.stdout: NetworkManager-tui-1:1.47.2-1.el9.x86_64 2024-03-28T17:52:26.327 INFO:teuthology.orchestra.run.smithi079.stdout: acl-2.3.1-4.el9.x86_64 2024-03-28T17:52:26.327 INFO:teuthology.orchestra.run.smithi079.stdout: elfutils-debuginfod-client-0.190-2.el9.x86_64 2024-03-28T17:52:26.327 INFO:teuthology.orchestra.run.smithi079.stdout: elfutils-libelf-0.190-2.el9.x86_64 2024-03-28T17:52:26.327 INFO:teuthology.orchestra.run.smithi079.stdout: elfutils-libs-0.190-2.el9.x86_64 2024-03-28T17:52:26.328 INFO:teuthology.orchestra.run.smithi079.stdout: libacl-2.3.1-4.el9.x86_64 2024-03-28T17:52:26.328 INFO:teuthology.orchestra.run.smithi079.stdout: libacl-devel-2.3.1-4.el9.x86_64 2024-03-28T17:52:26.328 INFO:teuthology.orchestra.run.smithi079.stdout: libblkid-2.37.4-18.el9.x86_64 2024-03-28T17:52:26.328 INFO:teuthology.orchestra.run.smithi079.stdout: libfdisk-2.37.4-18.el9.x86_64 2024-03-28T17:52:26.328 INFO:teuthology.orchestra.run.smithi079.stdout: libmount-2.37.4-18.el9.x86_64 2024-03-28T17:52:26.328 INFO:teuthology.orchestra.run.smithi079.stdout: librados2-2:19.0.0-2145.g3be9a5c9.el9.x86_64 2024-03-28T17:52:26.328 INFO:teuthology.orchestra.run.smithi079.stdout: librbd1-2:19.0.0-2145.g3be9a5c9.el9.x86_64 2024-03-28T17:52:26.328 INFO:teuthology.orchestra.run.smithi079.stdout: libsmartcols-2.37.4-18.el9.x86_64 2024-03-28T17:52:26.328 INFO:teuthology.orchestra.run.smithi079.stdout: libuuid-2.37.4-18.el9.x86_64 2024-03-28T17:52:26.328 INFO:teuthology.orchestra.run.smithi079.stdout: openssl-1:3.0.7-27.el9.x86_64 2024-03-28T17:52:26.328 INFO:teuthology.orchestra.run.smithi079.stdout: openssl-libs-1:3.0.7-27.el9.x86_64 2024-03-28T17:52:26.328 INFO:teuthology.orchestra.run.smithi079.stdout: util-linux-2.37.4-18.el9.x86_64 2024-03-28T17:52:26.328 INFO:teuthology.orchestra.run.smithi079.stdout: util-linux-core-2.37.4-18.el9.x86_64 2024-03-28T17:52:26.328 INFO:teuthology.orchestra.run.smithi079.stdout: util-linux-user-2.37.4-18.el9.x86_64 2024-03-28T17:52:26.329 INFO:teuthology.orchestra.run.smithi079.stdout: xfsdump-3.1.12-4.el9.x86_64 2024-03-28T17:52:26.329 INFO:teuthology.orchestra.run.smithi079.stdout: xfsprogs-6.3.0-1.el9.x86_64 2024-03-28T17:52:26.329 INFO:teuthology.orchestra.run.smithi079.stdout:Installed: 2024-03-28T17:52:26.329 INFO:teuthology.orchestra.run.smithi079.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-03-28T17:52:26.329 INFO:teuthology.orchestra.run.smithi079.stdout: automake-1.16.2-8.el9.noarch 2024-03-28T17:52:26.329 INFO:teuthology.orchestra.run.smithi079.stdout: bison-3.7.4-5.el9.x86_64 2024-03-28T17:52:26.329 INFO:teuthology.orchestra.run.smithi079.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-03-28T17:52:26.329 INFO:teuthology.orchestra.run.smithi079.stdout: ceph-2:19.0.0-2145.g3be9a5c9.el9.x86_64 2024-03-28T17:52:26.329 INFO:teuthology.orchestra.run.smithi079.stdout: ceph-base-2:19.0.0-2145.g3be9a5c9.el9.x86_64 2024-03-28T17:52:26.329 INFO:teuthology.orchestra.run.smithi079.stdout: ceph-common-2:19.0.0-2145.g3be9a5c9.el9.x86_64 2024-03-28T17:52:26.329 INFO:teuthology.orchestra.run.smithi079.stdout: ceph-fuse-2:19.0.0-2145.g3be9a5c9.el9.x86_64 2024-03-28T17:52:26.329 INFO:teuthology.orchestra.run.smithi079.stdout: ceph-grafana-dashboards-2:19.0.0-2145.g3be9a5c9.el9.noarch 2024-03-28T17:52:26.329 INFO:teuthology.orchestra.run.smithi079.stdout: ceph-immutable-object-cache-2:19.0.0-2145.g3be9a5c9.el9.x86_64 2024-03-28T17:52:26.329 INFO:teuthology.orchestra.run.smithi079.stdout: ceph-mds-2:19.0.0-2145.g3be9a5c9.el9.x86_64 2024-03-28T17:52:26.329 INFO:teuthology.orchestra.run.smithi079.stdout: ceph-mgr-2:19.0.0-2145.g3be9a5c9.el9.x86_64 2024-03-28T17:52:26.330 INFO:teuthology.orchestra.run.smithi079.stdout: ceph-mgr-cephadm-2:19.0.0-2145.g3be9a5c9.el9.noarch 2024-03-28T17:52:26.330 INFO:teuthology.orchestra.run.smithi079.stdout: ceph-mgr-dashboard-2:19.0.0-2145.g3be9a5c9.el9.noarch 2024-03-28T17:52:26.330 INFO:teuthology.orchestra.run.smithi079.stdout: ceph-mgr-diskprediction-local-2:19.0.0-2145.g3be9a5c9.el9.noarch 2024-03-28T17:52:26.330 INFO:teuthology.orchestra.run.smithi079.stdout: ceph-mgr-modules-core-2:19.0.0-2145.g3be9a5c9.el9.noarch 2024-03-28T17:52:26.330 INFO:teuthology.orchestra.run.smithi079.stdout: ceph-mgr-rook-2:19.0.0-2145.g3be9a5c9.el9.noarch 2024-03-28T17:52:26.330 INFO:teuthology.orchestra.run.smithi079.stdout: ceph-mon-2:19.0.0-2145.g3be9a5c9.el9.x86_64 2024-03-28T17:52:26.330 INFO:teuthology.orchestra.run.smithi079.stdout: ceph-osd-2:19.0.0-2145.g3be9a5c9.el9.x86_64 2024-03-28T17:52:26.330 INFO:teuthology.orchestra.run.smithi079.stdout: ceph-prometheus-alerts-2:19.0.0-2145.g3be9a5c9.el9.noarch 2024-03-28T17:52:26.330 INFO:teuthology.orchestra.run.smithi079.stdout: ceph-radosgw-2:19.0.0-2145.g3be9a5c9.el9.x86_64 2024-03-28T17:52:26.330 INFO:teuthology.orchestra.run.smithi079.stdout: ceph-selinux-2:19.0.0-2145.g3be9a5c9.el9.x86_64 2024-03-28T17:52:26.330 INFO:teuthology.orchestra.run.smithi079.stdout: ceph-test-2:19.0.0-2145.g3be9a5c9.el9.x86_64 2024-03-28T17:52:26.330 INFO:teuthology.orchestra.run.smithi079.stdout: ceph-volume-2:19.0.0-2145.g3be9a5c9.el9.noarch 2024-03-28T17:52:26.330 INFO:teuthology.orchestra.run.smithi079.stdout: cephadm-2:19.0.0-2145.g3be9a5c9.el9.noarch 2024-03-28T17:52:26.330 INFO:teuthology.orchestra.run.smithi079.stdout: cephfs-mirror-2:19.0.0-2145.g3be9a5c9.el9.x86_64 2024-03-28T17:52:26.330 INFO:teuthology.orchestra.run.smithi079.stdout: cephfs-shell-2:19.0.0-2145.g3be9a5c9.el9.x86_64 2024-03-28T17:52:26.331 INFO:teuthology.orchestra.run.smithi079.stdout: cephfs-top-2:19.0.0-2145.g3be9a5c9.el9.noarch 2024-03-28T17:52:26.331 INFO:teuthology.orchestra.run.smithi079.stdout: elfutils-libelf-devel-0.190-2.el9.x86_64 2024-03-28T17:52:26.331 INFO:teuthology.orchestra.run.smithi079.stdout: flex-2.6.4-9.el9.x86_64 2024-03-28T17:52:26.331 INFO:teuthology.orchestra.run.smithi079.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-03-28T17:52:26.331 INFO:teuthology.orchestra.run.smithi079.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-03-28T17:52:26.331 INFO:teuthology.orchestra.run.smithi079.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-03-28T17:52:26.331 INFO:teuthology.orchestra.run.smithi079.stdout: fuse-2.9.9-15.el9.x86_64 2024-03-28T17:52:26.331 INFO:teuthology.orchestra.run.smithi079.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-03-28T17:52:26.331 INFO:teuthology.orchestra.run.smithi079.stdout: grpc-data-1.46.7-10.el9.noarch 2024-03-28T17:52:26.331 INFO:teuthology.orchestra.run.smithi079.stdout: jq-1.6-16.el9.x86_64 2024-03-28T17:52:26.331 INFO:teuthology.orchestra.run.smithi079.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-03-28T17:52:26.331 INFO:teuthology.orchestra.run.smithi079.stdout: libarrow-9.0.0-10.el9.x86_64 2024-03-28T17:52:26.331 INFO:teuthology.orchestra.run.smithi079.stdout: libarrow-doc-9.0.0-10.el9.noarch 2024-03-28T17:52:26.331 INFO:teuthology.orchestra.run.smithi079.stdout: libcephfs-devel-2:19.0.0-2145.g3be9a5c9.el9.x86_64 2024-03-28T17:52:26.332 INFO:teuthology.orchestra.run.smithi079.stdout: libcephfs2-2:19.0.0-2145.g3be9a5c9.el9.x86_64 2024-03-28T17:52:26.332 INFO:teuthology.orchestra.run.smithi079.stdout: libcephsqlite-2:19.0.0-2145.g3be9a5c9.el9.x86_64 2024-03-28T17:52:26.332 INFO:teuthology.orchestra.run.smithi079.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-03-28T17:52:26.332 INFO:teuthology.orchestra.run.smithi079.stdout: liboath-2.6.7-2.el9.x86_64 2024-03-28T17:52:26.332 INFO:teuthology.orchestra.run.smithi079.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-03-28T17:52:26.332 INFO:teuthology.orchestra.run.smithi079.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-03-28T17:52:26.332 INFO:teuthology.orchestra.run.smithi079.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-03-28T17:52:26.332 INFO:teuthology.orchestra.run.smithi079.stdout: librados-devel-2:19.0.0-2145.g3be9a5c9.el9.x86_64 2024-03-28T17:52:26.332 INFO:teuthology.orchestra.run.smithi079.stdout: libradosstriper1-2:19.0.0-2145.g3be9a5c9.el9.x86_64 2024-03-28T17:52:26.332 INFO:teuthology.orchestra.run.smithi079.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-03-28T17:52:26.332 INFO:teuthology.orchestra.run.smithi079.stdout: librgw2-2:19.0.0-2145.g3be9a5c9.el9.x86_64 2024-03-28T17:52:26.332 INFO:teuthology.orchestra.run.smithi079.stdout: libtool-2.4.6-45.el9.x86_64 2024-03-28T17:52:26.332 INFO:teuthology.orchestra.run.smithi079.stdout: libunwind-1.6.2-1.el9.x86_64 2024-03-28T17:52:26.332 INFO:teuthology.orchestra.run.smithi079.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-03-28T17:52:26.332 INFO:teuthology.orchestra.run.smithi079.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-03-28T17:52:26.332 INFO:teuthology.orchestra.run.smithi079.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-03-28T17:52:26.333 INFO:teuthology.orchestra.run.smithi079.stdout: lua-5.4.4-4.el9.x86_64 2024-03-28T17:52:26.333 INFO:teuthology.orchestra.run.smithi079.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-03-28T17:52:26.333 INFO:teuthology.orchestra.run.smithi079.stdout: luarocks-3.9.2-1.el9.noarch 2024-03-28T17:52:26.333 INFO:teuthology.orchestra.run.smithi079.stdout: oniguruma-6.9.6-1.el9.5.x86_64 2024-03-28T17:52:26.333 INFO:teuthology.orchestra.run.smithi079.stdout: openblas-0.3.21-2.el9.x86_64 2024-03-28T17:52:26.333 INFO:teuthology.orchestra.run.smithi079.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-03-28T17:52:26.333 INFO:teuthology.orchestra.run.smithi079.stdout: openssl-devel-1:3.0.7-27.el9.x86_64 2024-03-28T17:52:26.333 INFO:teuthology.orchestra.run.smithi079.stdout: parquet-libs-9.0.0-10.el9.x86_64 2024-03-28T17:52:26.333 INFO:teuthology.orchestra.run.smithi079.stdout: postgresql-13.14-1.el9.x86_64 2024-03-28T17:52:26.333 INFO:teuthology.orchestra.run.smithi079.stdout: postgresql-contrib-13.14-1.el9.x86_64 2024-03-28T17:52:26.333 INFO:teuthology.orchestra.run.smithi079.stdout: postgresql-private-libs-13.14-1.el9.x86_64 2024-03-28T17:52:26.333 INFO:teuthology.orchestra.run.smithi079.stdout: postgresql-server-13.14-1.el9.x86_64 2024-03-28T17:52:26.333 INFO:teuthology.orchestra.run.smithi079.stdout: protobuf-3.14.0-13.el9.x86_64 2024-03-28T17:52:26.333 INFO:teuthology.orchestra.run.smithi079.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-03-28T17:52:26.334 INFO:teuthology.orchestra.run.smithi079.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-03-28T17:52:26.334 INFO:teuthology.orchestra.run.smithi079.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-03-28T17:52:26.334 INFO:teuthology.orchestra.run.smithi079.stdout: python3-babel-2.9.1-2.el9.noarch 2024-03-28T17:52:26.334 INFO:teuthology.orchestra.run.smithi079.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-03-28T17:52:26.334 INFO:teuthology.orchestra.run.smithi079.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-03-28T17:52:26.334 INFO:teuthology.orchestra.run.smithi079.stdout: python3-ceph-argparse-2:19.0.0-2145.g3be9a5c9.el9.x86_64 2024-03-28T17:52:26.334 INFO:teuthology.orchestra.run.smithi079.stdout: python3-ceph-common-2:19.0.0-2145.g3be9a5c9.el9.x86_64 2024-03-28T17:52:26.334 INFO:teuthology.orchestra.run.smithi079.stdout: python3-cephfs-2:19.0.0-2145.g3be9a5c9.el9.x86_64 2024-03-28T17:52:26.334 INFO:teuthology.orchestra.run.smithi079.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-03-28T17:52:26.334 INFO:teuthology.orchestra.run.smithi079.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-03-28T17:52:26.334 INFO:teuthology.orchestra.run.smithi079.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-03-28T17:52:26.334 INFO:teuthology.orchestra.run.smithi079.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-03-28T17:52:26.334 INFO:teuthology.orchestra.run.smithi079.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-03-28T17:52:26.335 INFO:teuthology.orchestra.run.smithi079.stdout: python3-colorama-0.4.4-12.el9.noarch 2024-03-28T17:52:26.335 INFO:teuthology.orchestra.run.smithi079.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-03-28T17:52:26.335 INFO:teuthology.orchestra.run.smithi079.stdout: python3-devel-3.9.17-2.el9.x86_64 2024-03-28T17:52:26.335 INFO:teuthology.orchestra.run.smithi079.stdout: python3-google-auth-1:2.28.2-1.el9.noarch 2024-03-28T17:52:26.335 INFO:teuthology.orchestra.run.smithi079.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-03-28T17:52:26.335 INFO:teuthology.orchestra.run.smithi079.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-03-28T17:52:26.335 INFO:teuthology.orchestra.run.smithi079.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-03-28T17:52:26.335 INFO:teuthology.orchestra.run.smithi079.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-03-28T17:52:26.335 INFO:teuthology.orchestra.run.smithi079.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-03-28T17:52:26.335 INFO:teuthology.orchestra.run.smithi079.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-03-28T17:52:26.335 INFO:teuthology.orchestra.run.smithi079.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-03-28T17:52:26.335 INFO:teuthology.orchestra.run.smithi079.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-03-28T17:52:26.335 INFO:teuthology.orchestra.run.smithi079.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-03-28T17:52:26.335 INFO:teuthology.orchestra.run.smithi079.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-03-28T17:52:26.335 INFO:teuthology.orchestra.run.smithi079.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-03-28T17:52:26.336 INFO:teuthology.orchestra.run.smithi079.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-03-28T17:52:26.336 INFO:teuthology.orchestra.run.smithi079.stdout: python3-mako-1.1.4-6.el9.noarch 2024-03-28T17:52:26.336 INFO:teuthology.orchestra.run.smithi079.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-03-28T17:52:26.336 INFO:teuthology.orchestra.run.smithi079.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-03-28T17:52:26.336 INFO:teuthology.orchestra.run.smithi079.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-03-28T17:52:26.336 INFO:teuthology.orchestra.run.smithi079.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-03-28T17:52:26.336 INFO:teuthology.orchestra.run.smithi079.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-03-28T17:52:26.336 INFO:teuthology.orchestra.run.smithi079.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-03-28T17:52:26.336 INFO:teuthology.orchestra.run.smithi079.stdout: python3-packaging-20.9-5.el9.noarch 2024-03-28T17:52:26.336 INFO:teuthology.orchestra.run.smithi079.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-03-28T17:52:26.336 INFO:teuthology.orchestra.run.smithi079.stdout: python3-ply-3.11-14.el9.noarch 2024-03-28T17:52:26.336 INFO:teuthology.orchestra.run.smithi079.stdout: python3-portend-3.1.0-2.el9.noarch 2024-03-28T17:52:26.336 INFO:teuthology.orchestra.run.smithi079.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-03-28T17:52:26.336 INFO:teuthology.orchestra.run.smithi079.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-03-28T17:52:26.336 INFO:teuthology.orchestra.run.smithi079.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-03-28T17:52:26.337 INFO:teuthology.orchestra.run.smithi079.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-03-28T17:52:26.337 INFO:teuthology.orchestra.run.smithi079.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-03-28T17:52:26.337 INFO:teuthology.orchestra.run.smithi079.stdout: python3-pycparser-2.20-6.el9.noarch 2024-03-28T17:52:26.337 INFO:teuthology.orchestra.run.smithi079.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-03-28T17:52:26.337 INFO:teuthology.orchestra.run.smithi079.stdout: python3-pytz-2021.1-5.el9.noarch 2024-03-28T17:52:26.337 INFO:teuthology.orchestra.run.smithi079.stdout: python3-rados-2:19.0.0-2145.g3be9a5c9.el9.x86_64 2024-03-28T17:52:26.337 INFO:teuthology.orchestra.run.smithi079.stdout: python3-rbd-2:19.0.0-2145.g3be9a5c9.el9.x86_64 2024-03-28T17:52:26.337 INFO:teuthology.orchestra.run.smithi079.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-03-28T17:52:26.337 INFO:teuthology.orchestra.run.smithi079.stdout: python3-requests-2.25.1-8.el9.noarch 2024-03-28T17:52:26.337 INFO:teuthology.orchestra.run.smithi079.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-03-28T17:52:26.337 INFO:teuthology.orchestra.run.smithi079.stdout: python3-rgw-2:19.0.0-2145.g3be9a5c9.el9.x86_64 2024-03-28T17:52:26.337 INFO:teuthology.orchestra.run.smithi079.stdout: python3-routes-2.5.1-5.el9.noarch 2024-03-28T17:52:26.337 INFO:teuthology.orchestra.run.smithi079.stdout: python3-rsa-4.9-2.el9.noarch 2024-03-28T17:52:26.337 INFO:teuthology.orchestra.run.smithi079.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-03-28T17:52:26.337 INFO:teuthology.orchestra.run.smithi079.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-03-28T17:52:26.337 INFO:teuthology.orchestra.run.smithi079.stdout: python3-toml-0.10.2-6.el9.noarch 2024-03-28T17:52:26.338 INFO:teuthology.orchestra.run.smithi079.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-03-28T17:52:26.338 INFO:teuthology.orchestra.run.smithi079.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-03-28T17:52:26.338 INFO:teuthology.orchestra.run.smithi079.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-03-28T17:52:26.338 INFO:teuthology.orchestra.run.smithi079.stdout: python3-webob-1.8.7-6.el9.noarch 2024-03-28T17:52:26.338 INFO:teuthology.orchestra.run.smithi079.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-03-28T17:52:26.338 INFO:teuthology.orchestra.run.smithi079.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-03-28T17:52:26.338 INFO:teuthology.orchestra.run.smithi079.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-03-28T17:52:26.338 INFO:teuthology.orchestra.run.smithi079.stdout: qatlib-24.02.0-1.el9.x86_64 2024-03-28T17:52:26.338 INFO:teuthology.orchestra.run.smithi079.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-03-28T17:52:26.338 INFO:teuthology.orchestra.run.smithi079.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-03-28T17:52:26.338 INFO:teuthology.orchestra.run.smithi079.stdout: rbd-fuse-2:19.0.0-2145.g3be9a5c9.el9.x86_64 2024-03-28T17:52:26.338 INFO:teuthology.orchestra.run.smithi079.stdout: rbd-mirror-2:19.0.0-2145.g3be9a5c9.el9.x86_64 2024-03-28T17:52:26.338 INFO:teuthology.orchestra.run.smithi079.stdout: rbd-nbd-2:19.0.0-2145.g3be9a5c9.el9.x86_64 2024-03-28T17:52:26.339 INFO:teuthology.orchestra.run.smithi079.stdout: re2-1:20211101-3.el9.x86_64 2024-03-28T17:52:26.339 INFO:teuthology.orchestra.run.smithi079.stdout: socat-1.7.4.1-5.el9.x86_64 2024-03-28T17:52:26.339 INFO:teuthology.orchestra.run.smithi079.stdout: thrift-0.15.0-2.el9.x86_64 2024-03-28T17:52:26.339 INFO:teuthology.orchestra.run.smithi079.stdout: uuid-1.6.2-55.el9.x86_64 2024-03-28T17:52:26.339 INFO:teuthology.orchestra.run.smithi079.stdout: xfsprogs-devel-6.3.0-1.el9.x86_64 2024-03-28T17:52:26.339 INFO:teuthology.orchestra.run.smithi079.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-03-28T17:52:26.339 INFO:teuthology.orchestra.run.smithi079.stdout: zlib-devel-1.2.11-40.el9.x86_64 2024-03-28T17:52:26.339 INFO:teuthology.orchestra.run.smithi079.stdout: 2024-03-28T17:52:26.339 INFO:teuthology.orchestra.run.smithi079.stdout:Complete! 2024-03-28T17:52:26.488 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : ceph-2:19.0.0-2145.g3be9a5c9.el9.x86_64 1/198 2024-03-28T17:52:26.488 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : ceph-base-2:19.0.0-2145.g3be9a5c9.el9.x86_64 2/198 2024-03-28T17:52:26.488 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : ceph-common-2:19.0.0-2145.g3be9a5c9.el9.x86_64 3/198 2024-03-28T17:52:26.488 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : ceph-fuse-2:19.0.0-2145.g3be9a5c9.el9.x86_64 4/198 2024-03-28T17:52:26.488 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-2145.g3be9a 5/198 2024-03-28T17:52:26.488 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : ceph-mds-2:19.0.0-2145.g3be9a5c9.el9.x86_64 6/198 2024-03-28T17:52:26.489 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : ceph-mgr-2:19.0.0-2145.g3be9a5c9.el9.x86_64 7/198 2024-03-28T17:52:26.489 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : ceph-mon-2:19.0.0-2145.g3be9a5c9.el9.x86_64 8/198 2024-03-28T17:52:26.489 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : ceph-osd-2:19.0.0-2145.g3be9a5c9.el9.x86_64 9/198 2024-03-28T17:52:26.489 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : ceph-radosgw-2:19.0.0-2145.g3be9a5c9.el9.x86_64 10/198 2024-03-28T17:52:26.489 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : ceph-selinux-2:19.0.0-2145.g3be9a5c9.el9.x86_64 11/198 2024-03-28T17:52:26.489 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : ceph-test-2:19.0.0-2145.g3be9a5c9.el9.x86_64 12/198 2024-03-28T17:52:26.489 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : cephfs-mirror-2:19.0.0-2145.g3be9a5c9.el9.x86_64 13/198 2024-03-28T17:52:26.489 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : cephfs-shell-2:19.0.0-2145.g3be9a5c9.el9.x86_64 14/198 2024-03-28T17:52:26.489 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : libcephfs-devel-2:19.0.0-2145.g3be9a5c9.el9.x86_ 15/198 2024-03-28T17:52:26.489 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : libcephfs2-2:19.0.0-2145.g3be9a5c9.el9.x86_64 16/198 2024-03-28T17:52:26.489 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : libcephsqlite-2:19.0.0-2145.g3be9a5c9.el9.x86_64 17/198 2024-03-28T17:52:26.489 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : librados-devel-2:19.0.0-2145.g3be9a5c9.el9.x86_6 18/198 2024-03-28T17:52:26.489 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : libradosstriper1-2:19.0.0-2145.g3be9a5c9.el9.x86 19/198 2024-03-28T17:52:26.489 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : librgw2-2:19.0.0-2145.g3be9a5c9.el9.x86_64 20/198 2024-03-28T17:52:26.490 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : python3-ceph-argparse-2:19.0.0-2145.g3be9a5c9.el 21/198 2024-03-28T17:52:26.490 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : python3-ceph-common-2:19.0.0-2145.g3be9a5c9.el9. 22/198 2024-03-28T17:52:26.490 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : python3-cephfs-2:19.0.0-2145.g3be9a5c9.el9.x86_6 23/198 2024-03-28T17:52:26.490 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : python3-rados-2:19.0.0-2145.g3be9a5c9.el9.x86_64 24/198 2024-03-28T17:52:26.490 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : python3-rbd-2:19.0.0-2145.g3be9a5c9.el9.x86_64 25/198 2024-03-28T17:52:26.490 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : python3-rgw-2:19.0.0-2145.g3be9a5c9.el9.x86_64 26/198 2024-03-28T17:52:26.490 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : rbd-fuse-2:19.0.0-2145.g3be9a5c9.el9.x86_64 27/198 2024-03-28T17:52:26.490 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : rbd-mirror-2:19.0.0-2145.g3be9a5c9.el9.x86_64 28/198 2024-03-28T17:52:26.490 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : rbd-nbd-2:19.0.0-2145.g3be9a5c9.el9.x86_64 29/198 2024-03-28T17:52:26.506 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-2145.g3be9a5c9. 30/198 2024-03-28T17:52:26.506 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-2145.g3be9a5c9.el9.noa 31/198 2024-03-28T17:52:26.506 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-2145.g3be9a5c9.el9.n 32/198 2024-03-28T17:52:26.506 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-2145.g3be 33/198 2024-03-28T17:52:26.506 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-2145.g3be9a5c9.el 34/198 2024-03-28T17:52:26.507 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : ceph-mgr-rook-2:19.0.0-2145.g3be9a5c9.el9.noarch 35/198 2024-03-28T17:52:26.507 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-2145.g3be9a5c9.e 36/198 2024-03-28T17:52:26.507 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : ceph-volume-2:19.0.0-2145.g3be9a5c9.el9.noarch 37/198 2024-03-28T17:52:26.507 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : cephadm-2:19.0.0-2145.g3be9a5c9.el9.noarch 38/198 2024-03-28T17:52:26.507 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : cephfs-top-2:19.0.0-2145.g3be9a5c9.el9.noarch 39/198 2024-03-28T17:52:26.507 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : automake-1.16.2-8.el9.noarch 40/198 2024-03-28T17:52:26.507 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : bison-3.7.4-5.el9.x86_64 41/198 2024-03-28T17:52:26.507 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 42/198 2024-03-28T17:52:26.507 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : flex-2.6.4-9.el9.x86_64 43/198 2024-03-28T17:52:26.507 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 44/198 2024-03-28T17:52:26.507 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 45/198 2024-03-28T17:52:26.507 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 46/198 2024-03-28T17:52:26.507 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 47/198 2024-03-28T17:52:26.507 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 48/198 2024-03-28T17:52:26.508 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 49/198 2024-03-28T17:52:26.508 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 50/198 2024-03-28T17:52:26.508 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 51/198 2024-03-28T17:52:26.508 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 52/198 2024-03-28T17:52:26.508 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 53/198 2024-03-28T17:52:26.508 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : lua-5.4.4-4.el9.x86_64 54/198 2024-03-28T17:52:26.508 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : oniguruma-6.9.6-1.el9.5.x86_64 55/198 2024-03-28T17:52:26.508 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 56/198 2024-03-28T17:52:26.508 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 57/198 2024-03-28T17:52:26.508 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 58/198 2024-03-28T17:52:26.508 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 59/198 2024-03-28T17:52:26.508 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 60/198 2024-03-28T17:52:26.508 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 61/198 2024-03-28T17:52:26.508 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 62/198 2024-03-28T17:52:26.509 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 63/198 2024-03-28T17:52:26.509 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 64/198 2024-03-28T17:52:26.509 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 65/198 2024-03-28T17:52:26.509 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 66/198 2024-03-28T17:52:26.509 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 67/198 2024-03-28T17:52:26.509 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 68/198 2024-03-28T17:52:26.509 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 69/198 2024-03-28T17:52:26.509 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 70/198 2024-03-28T17:52:26.509 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 71/198 2024-03-28T17:52:26.509 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 72/198 2024-03-28T17:52:26.509 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 73/198 2024-03-28T17:52:26.509 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 74/198 2024-03-28T17:52:26.509 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 75/198 2024-03-28T17:52:26.509 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 76/198 2024-03-28T17:52:26.510 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 77/198 2024-03-28T17:52:26.510 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 78/198 2024-03-28T17:52:26.510 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 79/198 2024-03-28T17:52:26.510 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 80/198 2024-03-28T17:52:26.510 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : zlib-devel-1.2.11-40.el9.x86_64 81/198 2024-03-28T17:52:26.510 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 82/198 2024-03-28T17:52:26.510 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 83/198 2024-03-28T17:52:26.510 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 84/198 2024-03-28T17:52:26.510 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : python3-ply-3.11-14.el9.noarch 85/198 2024-03-28T17:52:26.510 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 86/198 2024-03-28T17:52:26.510 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 87/198 2024-03-28T17:52:26.510 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : jq-1.6-16.el9.x86_64 88/198 2024-03-28T17:52:26.510 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 89/198 2024-03-28T17:52:26.510 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 90/198 2024-03-28T17:52:26.511 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 91/198 2024-03-28T17:52:26.511 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : elfutils-libelf-devel-0.190-2.el9.x86_64 92/198 2024-03-28T17:52:26.511 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 93/198 2024-03-28T17:52:26.511 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : openssl-devel-1:3.0.7-27.el9.x86_64 94/198 2024-03-28T17:52:26.511 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : postgresql-13.14-1.el9.x86_64 95/198 2024-03-28T17:52:26.511 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : postgresql-contrib-13.14-1.el9.x86_64 96/198 2024-03-28T17:52:26.511 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : postgresql-private-libs-13.14-1.el9.x86_64 97/198 2024-03-28T17:52:26.511 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : postgresql-server-13.14-1.el9.x86_64 98/198 2024-03-28T17:52:26.511 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : python3-devel-3.9.17-2.el9.x86_64 99/198 2024-03-28T17:52:26.511 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 100/198 2024-03-28T17:52:26.511 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 101/198 2024-03-28T17:52:26.511 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 102/198 2024-03-28T17:52:26.511 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 103/198 2024-03-28T17:52:26.511 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : xfsprogs-devel-6.3.0-1.el9.x86_64 104/198 2024-03-28T17:52:26.512 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 105/198 2024-03-28T17:52:26.512 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 106/198 2024-03-28T17:52:26.512 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 107/198 2024-03-28T17:52:26.512 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 108/198 2024-03-28T17:52:26.512 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 109/198 2024-03-28T17:52:26.512 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : libarrow-9.0.0-10.el9.x86_64 110/198 2024-03-28T17:52:26.512 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : libarrow-doc-9.0.0-10.el9.noarch 111/198 2024-03-28T17:52:26.512 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 112/198 2024-03-28T17:52:26.512 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 113/198 2024-03-28T17:52:26.512 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : luarocks-3.9.2-1.el9.noarch 114/198 2024-03-28T17:52:26.512 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : parquet-libs-9.0.0-10.el9.x86_64 115/198 2024-03-28T17:52:26.512 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 116/198 2024-03-28T17:52:26.512 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 117/198 2024-03-28T17:52:26.512 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 118/198 2024-03-28T17:52:26.512 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 119/198 2024-03-28T17:52:26.513 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 120/198 2024-03-28T17:52:26.513 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 121/198 2024-03-28T17:52:26.513 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 122/198 2024-03-28T17:52:26.513 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : python3-colorama-0.4.4-12.el9.noarch 123/198 2024-03-28T17:52:26.513 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : python3-google-auth-1:2.28.2-1.el9.noarch 124/198 2024-03-28T17:52:26.513 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 125/198 2024-03-28T17:52:26.513 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 126/198 2024-03-28T17:52:26.513 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 127/198 2024-03-28T17:52:26.513 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 128/198 2024-03-28T17:52:26.513 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 129/198 2024-03-28T17:52:26.513 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 130/198 2024-03-28T17:52:26.513 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 131/198 2024-03-28T17:52:26.513 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 132/198 2024-03-28T17:52:26.513 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 133/198 2024-03-28T17:52:26.514 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 134/198 2024-03-28T17:52:26.514 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 135/198 2024-03-28T17:52:26.514 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 136/198 2024-03-28T17:52:26.514 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 137/198 2024-03-28T17:52:26.514 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 138/198 2024-03-28T17:52:26.514 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 139/198 2024-03-28T17:52:26.514 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 140/198 2024-03-28T17:52:26.514 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 141/198 2024-03-28T17:52:26.514 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 142/198 2024-03-28T17:52:26.514 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 143/198 2024-03-28T17:52:26.514 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 144/198 2024-03-28T17:52:26.514 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 145/198 2024-03-28T17:52:26.514 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 146/198 2024-03-28T17:52:26.514 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 147/198 2024-03-28T17:52:26.515 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 148/198 2024-03-28T17:52:26.515 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : re2-1:20211101-3.el9.x86_64 149/198 2024-03-28T17:52:26.515 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 150/198 2024-03-28T17:52:26.515 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : librados2-2:19.0.0-2145.g3be9a5c9.el9.x86_64 151/198 2024-03-28T17:52:26.515 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 152/198 2024-03-28T17:52:26.515 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : librbd1-2:19.0.0-2145.g3be9a5c9.el9.x86_64 153/198 2024-03-28T17:52:26.515 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 154/198 2024-03-28T17:52:26.515 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : xfsdump-3.1.12-4.el9.x86_64 155/198 2024-03-28T17:52:26.515 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : xfsdump-3.1.12-3.el9.x86_64 156/198 2024-03-28T17:52:26.515 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : NetworkManager-1:1.47.2-1.el9.x86_64 157/198 2024-03-28T17:52:26.515 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : NetworkManager-1:1.44.0-2.el9.x86_64 158/198 2024-03-28T17:52:26.515 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : NetworkManager-libnm-1:1.47.2-1.el9.x86_64 159/198 2024-03-28T17:52:26.515 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : NetworkManager-libnm-1:1.44.0-2.el9.x86_64 160/198 2024-03-28T17:52:26.515 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : NetworkManager-team-1:1.47.2-1.el9.x86_64 161/198 2024-03-28T17:52:26.515 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : NetworkManager-team-1:1.44.0-2.el9.x86_64 162/198 2024-03-28T17:52:26.516 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : NetworkManager-tui-1:1.47.2-1.el9.x86_64 163/198 2024-03-28T17:52:26.516 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : NetworkManager-tui-1:1.44.0-2.el9.x86_64 164/198 2024-03-28T17:52:26.516 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : acl-2.3.1-4.el9.x86_64 165/198 2024-03-28T17:52:26.516 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : acl-2.3.1-3.el9.x86_64 166/198 2024-03-28T17:52:26.516 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : elfutils-debuginfod-client-0.190-2.el9.x86_64 167/198 2024-03-28T17:52:26.516 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : elfutils-debuginfod-client-0.189-3.el9.x86_64 168/198 2024-03-28T17:52:26.516 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : elfutils-libelf-0.190-2.el9.x86_64 169/198 2024-03-28T17:52:26.516 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : elfutils-libelf-0.189-3.el9.x86_64 170/198 2024-03-28T17:52:26.516 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : elfutils-libs-0.190-2.el9.x86_64 171/198 2024-03-28T17:52:26.516 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : elfutils-libs-0.189-3.el9.x86_64 172/198 2024-03-28T17:52:26.516 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : libacl-2.3.1-4.el9.x86_64 173/198 2024-03-28T17:52:26.516 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : libacl-2.3.1-3.el9.x86_64 174/198 2024-03-28T17:52:26.516 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : libblkid-2.37.4-18.el9.x86_64 175/198 2024-03-28T17:52:26.516 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : libblkid-2.37.4-15.el9.x86_64 176/198 2024-03-28T17:52:26.516 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : libfdisk-2.37.4-18.el9.x86_64 177/198 2024-03-28T17:52:26.517 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : libfdisk-2.37.4-15.el9.x86_64 178/198 2024-03-28T17:52:26.517 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : libmount-2.37.4-18.el9.x86_64 179/198 2024-03-28T17:52:26.517 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : libmount-2.37.4-15.el9.x86_64 180/198 2024-03-28T17:52:26.517 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : libsmartcols-2.37.4-18.el9.x86_64 181/198 2024-03-28T17:52:26.517 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : libsmartcols-2.37.4-15.el9.x86_64 182/198 2024-03-28T17:52:26.517 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : libuuid-2.37.4-18.el9.x86_64 183/198 2024-03-28T17:52:26.517 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : libuuid-2.37.4-15.el9.x86_64 184/198 2024-03-28T17:52:26.517 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : openssl-1:3.0.7-27.el9.x86_64 185/198 2024-03-28T17:52:26.517 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : openssl-1:3.0.7-24.el9.x86_64 186/198 2024-03-28T17:52:26.517 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : openssl-libs-1:3.0.7-27.el9.x86_64 187/198 2024-03-28T17:52:26.517 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : openssl-libs-1:3.0.7-24.el9.x86_64 188/198 2024-03-28T17:52:26.517 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : util-linux-2.37.4-18.el9.x86_64 189/198 2024-03-28T17:52:26.517 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : util-linux-2.37.4-15.el9.x86_64 190/198 2024-03-28T17:52:26.517 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : util-linux-core-2.37.4-18.el9.x86_64 191/198 2024-03-28T17:52:26.518 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : util-linux-core-2.37.4-15.el9.x86_64 192/198 2024-03-28T17:52:26.518 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : util-linux-user-2.37.4-18.el9.x86_64 193/198 2024-03-28T17:52:26.518 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : util-linux-user-2.37.4-15.el9.x86_64 194/198 2024-03-28T17:52:26.518 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : xfsprogs-6.3.0-1.el9.x86_64 195/198 2024-03-28T17:52:26.518 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : xfsprogs-5.19.0-4.el9.x86_64 196/198 2024-03-28T17:52:26.518 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : libacl-devel-2.3.1-4.el9.x86_64 197/198 2024-03-28T17:52:26.975 DEBUG:teuthology.parallel:result is None 2024-03-28T17:52:29.102 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : libacl-devel-2.3.1-3.el9.x86_64 198/198 2024-03-28T17:52:29.102 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-03-28T17:52:29.102 INFO:teuthology.orchestra.run.smithi022.stdout:Upgraded: 2024-03-28T17:52:29.102 INFO:teuthology.orchestra.run.smithi022.stdout: NetworkManager-1:1.47.2-1.el9.x86_64 2024-03-28T17:52:29.102 INFO:teuthology.orchestra.run.smithi022.stdout: NetworkManager-libnm-1:1.47.2-1.el9.x86_64 2024-03-28T17:52:29.103 INFO:teuthology.orchestra.run.smithi022.stdout: NetworkManager-team-1:1.47.2-1.el9.x86_64 2024-03-28T17:52:29.103 INFO:teuthology.orchestra.run.smithi022.stdout: NetworkManager-tui-1:1.47.2-1.el9.x86_64 2024-03-28T17:52:29.103 INFO:teuthology.orchestra.run.smithi022.stdout: acl-2.3.1-4.el9.x86_64 2024-03-28T17:52:29.103 INFO:teuthology.orchestra.run.smithi022.stdout: elfutils-debuginfod-client-0.190-2.el9.x86_64 2024-03-28T17:52:29.103 INFO:teuthology.orchestra.run.smithi022.stdout: elfutils-libelf-0.190-2.el9.x86_64 2024-03-28T17:52:29.103 INFO:teuthology.orchestra.run.smithi022.stdout: elfutils-libs-0.190-2.el9.x86_64 2024-03-28T17:52:29.103 INFO:teuthology.orchestra.run.smithi022.stdout: libacl-2.3.1-4.el9.x86_64 2024-03-28T17:52:29.103 INFO:teuthology.orchestra.run.smithi022.stdout: libacl-devel-2.3.1-4.el9.x86_64 2024-03-28T17:52:29.103 INFO:teuthology.orchestra.run.smithi022.stdout: libblkid-2.37.4-18.el9.x86_64 2024-03-28T17:52:29.103 INFO:teuthology.orchestra.run.smithi022.stdout: libfdisk-2.37.4-18.el9.x86_64 2024-03-28T17:52:29.103 INFO:teuthology.orchestra.run.smithi022.stdout: libmount-2.37.4-18.el9.x86_64 2024-03-28T17:52:29.103 INFO:teuthology.orchestra.run.smithi022.stdout: librados2-2:19.0.0-2145.g3be9a5c9.el9.x86_64 2024-03-28T17:52:29.104 INFO:teuthology.orchestra.run.smithi022.stdout: librbd1-2:19.0.0-2145.g3be9a5c9.el9.x86_64 2024-03-28T17:52:29.104 INFO:teuthology.orchestra.run.smithi022.stdout: libsmartcols-2.37.4-18.el9.x86_64 2024-03-28T17:52:29.104 INFO:teuthology.orchestra.run.smithi022.stdout: libuuid-2.37.4-18.el9.x86_64 2024-03-28T17:52:29.104 INFO:teuthology.orchestra.run.smithi022.stdout: openssl-1:3.0.7-27.el9.x86_64 2024-03-28T17:52:29.104 INFO:teuthology.orchestra.run.smithi022.stdout: openssl-libs-1:3.0.7-27.el9.x86_64 2024-03-28T17:52:29.104 INFO:teuthology.orchestra.run.smithi022.stdout: util-linux-2.37.4-18.el9.x86_64 2024-03-28T17:52:29.104 INFO:teuthology.orchestra.run.smithi022.stdout: util-linux-core-2.37.4-18.el9.x86_64 2024-03-28T17:52:29.104 INFO:teuthology.orchestra.run.smithi022.stdout: util-linux-user-2.37.4-18.el9.x86_64 2024-03-28T17:52:29.104 INFO:teuthology.orchestra.run.smithi022.stdout: xfsdump-3.1.12-4.el9.x86_64 2024-03-28T17:52:29.104 INFO:teuthology.orchestra.run.smithi022.stdout: xfsprogs-6.3.0-1.el9.x86_64 2024-03-28T17:52:29.104 INFO:teuthology.orchestra.run.smithi022.stdout:Installed: 2024-03-28T17:52:29.105 INFO:teuthology.orchestra.run.smithi022.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-03-28T17:52:29.105 INFO:teuthology.orchestra.run.smithi022.stdout: automake-1.16.2-8.el9.noarch 2024-03-28T17:52:29.105 INFO:teuthology.orchestra.run.smithi022.stdout: bison-3.7.4-5.el9.x86_64 2024-03-28T17:52:29.105 INFO:teuthology.orchestra.run.smithi022.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-03-28T17:52:29.105 INFO:teuthology.orchestra.run.smithi022.stdout: ceph-2:19.0.0-2145.g3be9a5c9.el9.x86_64 2024-03-28T17:52:29.105 INFO:teuthology.orchestra.run.smithi022.stdout: ceph-base-2:19.0.0-2145.g3be9a5c9.el9.x86_64 2024-03-28T17:52:29.105 INFO:teuthology.orchestra.run.smithi022.stdout: ceph-common-2:19.0.0-2145.g3be9a5c9.el9.x86_64 2024-03-28T17:52:29.105 INFO:teuthology.orchestra.run.smithi022.stdout: ceph-fuse-2:19.0.0-2145.g3be9a5c9.el9.x86_64 2024-03-28T17:52:29.105 INFO:teuthology.orchestra.run.smithi022.stdout: ceph-grafana-dashboards-2:19.0.0-2145.g3be9a5c9.el9.noarch 2024-03-28T17:52:29.105 INFO:teuthology.orchestra.run.smithi022.stdout: ceph-immutable-object-cache-2:19.0.0-2145.g3be9a5c9.el9.x86_64 2024-03-28T17:52:29.106 INFO:teuthology.orchestra.run.smithi022.stdout: ceph-mds-2:19.0.0-2145.g3be9a5c9.el9.x86_64 2024-03-28T17:52:29.106 INFO:teuthology.orchestra.run.smithi022.stdout: ceph-mgr-2:19.0.0-2145.g3be9a5c9.el9.x86_64 2024-03-28T17:52:29.106 INFO:teuthology.orchestra.run.smithi022.stdout: ceph-mgr-cephadm-2:19.0.0-2145.g3be9a5c9.el9.noarch 2024-03-28T17:52:29.106 INFO:teuthology.orchestra.run.smithi022.stdout: ceph-mgr-dashboard-2:19.0.0-2145.g3be9a5c9.el9.noarch 2024-03-28T17:52:29.106 INFO:teuthology.orchestra.run.smithi022.stdout: ceph-mgr-diskprediction-local-2:19.0.0-2145.g3be9a5c9.el9.noarch 2024-03-28T17:52:29.106 INFO:teuthology.orchestra.run.smithi022.stdout: ceph-mgr-modules-core-2:19.0.0-2145.g3be9a5c9.el9.noarch 2024-03-28T17:52:29.106 INFO:teuthology.orchestra.run.smithi022.stdout: ceph-mgr-rook-2:19.0.0-2145.g3be9a5c9.el9.noarch 2024-03-28T17:52:29.106 INFO:teuthology.orchestra.run.smithi022.stdout: ceph-mon-2:19.0.0-2145.g3be9a5c9.el9.x86_64 2024-03-28T17:52:29.106 INFO:teuthology.orchestra.run.smithi022.stdout: ceph-osd-2:19.0.0-2145.g3be9a5c9.el9.x86_64 2024-03-28T17:52:29.106 INFO:teuthology.orchestra.run.smithi022.stdout: ceph-prometheus-alerts-2:19.0.0-2145.g3be9a5c9.el9.noarch 2024-03-28T17:52:29.106 INFO:teuthology.orchestra.run.smithi022.stdout: ceph-radosgw-2:19.0.0-2145.g3be9a5c9.el9.x86_64 2024-03-28T17:52:29.106 INFO:teuthology.orchestra.run.smithi022.stdout: ceph-selinux-2:19.0.0-2145.g3be9a5c9.el9.x86_64 2024-03-28T17:52:29.106 INFO:teuthology.orchestra.run.smithi022.stdout: ceph-test-2:19.0.0-2145.g3be9a5c9.el9.x86_64 2024-03-28T17:52:29.107 INFO:teuthology.orchestra.run.smithi022.stdout: ceph-volume-2:19.0.0-2145.g3be9a5c9.el9.noarch 2024-03-28T17:52:29.107 INFO:teuthology.orchestra.run.smithi022.stdout: cephadm-2:19.0.0-2145.g3be9a5c9.el9.noarch 2024-03-28T17:52:29.107 INFO:teuthology.orchestra.run.smithi022.stdout: cephfs-mirror-2:19.0.0-2145.g3be9a5c9.el9.x86_64 2024-03-28T17:52:29.107 INFO:teuthology.orchestra.run.smithi022.stdout: cephfs-shell-2:19.0.0-2145.g3be9a5c9.el9.x86_64 2024-03-28T17:52:29.107 INFO:teuthology.orchestra.run.smithi022.stdout: cephfs-top-2:19.0.0-2145.g3be9a5c9.el9.noarch 2024-03-28T17:52:29.107 INFO:teuthology.orchestra.run.smithi022.stdout: elfutils-libelf-devel-0.190-2.el9.x86_64 2024-03-28T17:52:29.107 INFO:teuthology.orchestra.run.smithi022.stdout: flex-2.6.4-9.el9.x86_64 2024-03-28T17:52:29.107 INFO:teuthology.orchestra.run.smithi022.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-03-28T17:52:29.107 INFO:teuthology.orchestra.run.smithi022.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-03-28T17:52:29.107 INFO:teuthology.orchestra.run.smithi022.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-03-28T17:52:29.107 INFO:teuthology.orchestra.run.smithi022.stdout: fuse-2.9.9-15.el9.x86_64 2024-03-28T17:52:29.107 INFO:teuthology.orchestra.run.smithi022.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-03-28T17:52:29.107 INFO:teuthology.orchestra.run.smithi022.stdout: grpc-data-1.46.7-10.el9.noarch 2024-03-28T17:52:29.107 INFO:teuthology.orchestra.run.smithi022.stdout: jq-1.6-16.el9.x86_64 2024-03-28T17:52:29.108 INFO:teuthology.orchestra.run.smithi022.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-03-28T17:52:29.108 INFO:teuthology.orchestra.run.smithi022.stdout: libarrow-9.0.0-10.el9.x86_64 2024-03-28T17:52:29.108 INFO:teuthology.orchestra.run.smithi022.stdout: libarrow-doc-9.0.0-10.el9.noarch 2024-03-28T17:52:29.108 INFO:teuthology.orchestra.run.smithi022.stdout: libcephfs-devel-2:19.0.0-2145.g3be9a5c9.el9.x86_64 2024-03-28T17:52:29.108 INFO:teuthology.orchestra.run.smithi022.stdout: libcephfs2-2:19.0.0-2145.g3be9a5c9.el9.x86_64 2024-03-28T17:52:29.108 INFO:teuthology.orchestra.run.smithi022.stdout: libcephsqlite-2:19.0.0-2145.g3be9a5c9.el9.x86_64 2024-03-28T17:52:29.108 INFO:teuthology.orchestra.run.smithi022.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-03-28T17:52:29.108 INFO:teuthology.orchestra.run.smithi022.stdout: liboath-2.6.7-2.el9.x86_64 2024-03-28T17:52:29.108 INFO:teuthology.orchestra.run.smithi022.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-03-28T17:52:29.108 INFO:teuthology.orchestra.run.smithi022.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-03-28T17:52:29.108 INFO:teuthology.orchestra.run.smithi022.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-03-28T17:52:29.108 INFO:teuthology.orchestra.run.smithi022.stdout: librados-devel-2:19.0.0-2145.g3be9a5c9.el9.x86_64 2024-03-28T17:52:29.108 INFO:teuthology.orchestra.run.smithi022.stdout: libradosstriper1-2:19.0.0-2145.g3be9a5c9.el9.x86_64 2024-03-28T17:52:29.109 INFO:teuthology.orchestra.run.smithi022.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-03-28T17:52:29.109 INFO:teuthology.orchestra.run.smithi022.stdout: librgw2-2:19.0.0-2145.g3be9a5c9.el9.x86_64 2024-03-28T17:52:29.109 INFO:teuthology.orchestra.run.smithi022.stdout: libtool-2.4.6-45.el9.x86_64 2024-03-28T17:52:29.109 INFO:teuthology.orchestra.run.smithi022.stdout: libunwind-1.6.2-1.el9.x86_64 2024-03-28T17:52:29.109 INFO:teuthology.orchestra.run.smithi022.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-03-28T17:52:29.109 INFO:teuthology.orchestra.run.smithi022.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-03-28T17:52:29.109 INFO:teuthology.orchestra.run.smithi022.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-03-28T17:52:29.109 INFO:teuthology.orchestra.run.smithi022.stdout: lua-5.4.4-4.el9.x86_64 2024-03-28T17:52:29.109 INFO:teuthology.orchestra.run.smithi022.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-03-28T17:52:29.109 INFO:teuthology.orchestra.run.smithi022.stdout: luarocks-3.9.2-1.el9.noarch 2024-03-28T17:52:29.109 INFO:teuthology.orchestra.run.smithi022.stdout: oniguruma-6.9.6-1.el9.5.x86_64 2024-03-28T17:52:29.109 INFO:teuthology.orchestra.run.smithi022.stdout: openblas-0.3.21-2.el9.x86_64 2024-03-28T17:52:29.110 INFO:teuthology.orchestra.run.smithi022.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-03-28T17:52:29.110 INFO:teuthology.orchestra.run.smithi022.stdout: openssl-devel-1:3.0.7-27.el9.x86_64 2024-03-28T17:52:29.110 INFO:teuthology.orchestra.run.smithi022.stdout: parquet-libs-9.0.0-10.el9.x86_64 2024-03-28T17:52:29.110 INFO:teuthology.orchestra.run.smithi022.stdout: postgresql-13.14-1.el9.x86_64 2024-03-28T17:52:29.110 INFO:teuthology.orchestra.run.smithi022.stdout: postgresql-contrib-13.14-1.el9.x86_64 2024-03-28T17:52:29.110 INFO:teuthology.orchestra.run.smithi022.stdout: postgresql-private-libs-13.14-1.el9.x86_64 2024-03-28T17:52:29.110 INFO:teuthology.orchestra.run.smithi022.stdout: postgresql-server-13.14-1.el9.x86_64 2024-03-28T17:52:29.110 INFO:teuthology.orchestra.run.smithi022.stdout: protobuf-3.14.0-13.el9.x86_64 2024-03-28T17:52:29.110 INFO:teuthology.orchestra.run.smithi022.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-03-28T17:52:29.110 INFO:teuthology.orchestra.run.smithi022.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-03-28T17:52:29.110 INFO:teuthology.orchestra.run.smithi022.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-03-28T17:52:29.110 INFO:teuthology.orchestra.run.smithi022.stdout: python3-babel-2.9.1-2.el9.noarch 2024-03-28T17:52:29.111 INFO:teuthology.orchestra.run.smithi022.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-03-28T17:52:29.111 INFO:teuthology.orchestra.run.smithi022.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-03-28T17:52:29.111 INFO:teuthology.orchestra.run.smithi022.stdout: python3-ceph-argparse-2:19.0.0-2145.g3be9a5c9.el9.x86_64 2024-03-28T17:52:29.111 INFO:teuthology.orchestra.run.smithi022.stdout: python3-ceph-common-2:19.0.0-2145.g3be9a5c9.el9.x86_64 2024-03-28T17:52:29.111 INFO:teuthology.orchestra.run.smithi022.stdout: python3-cephfs-2:19.0.0-2145.g3be9a5c9.el9.x86_64 2024-03-28T17:52:29.111 INFO:teuthology.orchestra.run.smithi022.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-03-28T17:52:29.111 INFO:teuthology.orchestra.run.smithi022.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-03-28T17:52:29.111 INFO:teuthology.orchestra.run.smithi022.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-03-28T17:52:29.111 INFO:teuthology.orchestra.run.smithi022.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-03-28T17:52:29.111 INFO:teuthology.orchestra.run.smithi022.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-03-28T17:52:29.111 INFO:teuthology.orchestra.run.smithi022.stdout: python3-colorama-0.4.4-12.el9.noarch 2024-03-28T17:52:29.111 INFO:teuthology.orchestra.run.smithi022.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-03-28T17:52:29.111 INFO:teuthology.orchestra.run.smithi022.stdout: python3-devel-3.9.17-2.el9.x86_64 2024-03-28T17:52:29.112 INFO:teuthology.orchestra.run.smithi022.stdout: python3-google-auth-1:2.28.2-1.el9.noarch 2024-03-28T17:52:29.112 INFO:teuthology.orchestra.run.smithi022.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-03-28T17:52:29.112 INFO:teuthology.orchestra.run.smithi022.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-03-28T17:52:29.112 INFO:teuthology.orchestra.run.smithi022.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-03-28T17:52:29.112 INFO:teuthology.orchestra.run.smithi022.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-03-28T17:52:29.112 INFO:teuthology.orchestra.run.smithi022.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-03-28T17:52:29.112 INFO:teuthology.orchestra.run.smithi022.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-03-28T17:52:29.112 INFO:teuthology.orchestra.run.smithi022.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-03-28T17:52:29.112 INFO:teuthology.orchestra.run.smithi022.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-03-28T17:52:29.112 INFO:teuthology.orchestra.run.smithi022.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-03-28T17:52:29.112 INFO:teuthology.orchestra.run.smithi022.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-03-28T17:52:29.113 INFO:teuthology.orchestra.run.smithi022.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-03-28T17:52:29.113 INFO:teuthology.orchestra.run.smithi022.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-03-28T17:52:29.113 INFO:teuthology.orchestra.run.smithi022.stdout: python3-mako-1.1.4-6.el9.noarch 2024-03-28T17:52:29.113 INFO:teuthology.orchestra.run.smithi022.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-03-28T17:52:29.113 INFO:teuthology.orchestra.run.smithi022.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-03-28T17:52:29.113 INFO:teuthology.orchestra.run.smithi022.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-03-28T17:52:29.113 INFO:teuthology.orchestra.run.smithi022.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-03-28T17:52:29.113 INFO:teuthology.orchestra.run.smithi022.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-03-28T17:52:29.113 INFO:teuthology.orchestra.run.smithi022.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-03-28T17:52:29.113 INFO:teuthology.orchestra.run.smithi022.stdout: python3-packaging-20.9-5.el9.noarch 2024-03-28T17:52:29.113 INFO:teuthology.orchestra.run.smithi022.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-03-28T17:52:29.113 INFO:teuthology.orchestra.run.smithi022.stdout: python3-ply-3.11-14.el9.noarch 2024-03-28T17:52:29.113 INFO:teuthology.orchestra.run.smithi022.stdout: python3-portend-3.1.0-2.el9.noarch 2024-03-28T17:52:29.113 INFO:teuthology.orchestra.run.smithi022.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-03-28T17:52:29.114 INFO:teuthology.orchestra.run.smithi022.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-03-28T17:52:29.114 INFO:teuthology.orchestra.run.smithi022.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-03-28T17:52:29.114 INFO:teuthology.orchestra.run.smithi022.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-03-28T17:52:29.114 INFO:teuthology.orchestra.run.smithi022.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-03-28T17:52:29.114 INFO:teuthology.orchestra.run.smithi022.stdout: python3-pycparser-2.20-6.el9.noarch 2024-03-28T17:52:29.114 INFO:teuthology.orchestra.run.smithi022.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-03-28T17:52:29.114 INFO:teuthology.orchestra.run.smithi022.stdout: python3-pytz-2021.1-5.el9.noarch 2024-03-28T17:52:29.114 INFO:teuthology.orchestra.run.smithi022.stdout: python3-rados-2:19.0.0-2145.g3be9a5c9.el9.x86_64 2024-03-28T17:52:29.114 INFO:teuthology.orchestra.run.smithi022.stdout: python3-rbd-2:19.0.0-2145.g3be9a5c9.el9.x86_64 2024-03-28T17:52:29.114 INFO:teuthology.orchestra.run.smithi022.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-03-28T17:52:29.114 INFO:teuthology.orchestra.run.smithi022.stdout: python3-requests-2.25.1-8.el9.noarch 2024-03-28T17:52:29.114 INFO:teuthology.orchestra.run.smithi022.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-03-28T17:52:29.115 INFO:teuthology.orchestra.run.smithi022.stdout: python3-rgw-2:19.0.0-2145.g3be9a5c9.el9.x86_64 2024-03-28T17:52:29.115 INFO:teuthology.orchestra.run.smithi022.stdout: python3-routes-2.5.1-5.el9.noarch 2024-03-28T17:52:29.115 INFO:teuthology.orchestra.run.smithi022.stdout: python3-rsa-4.9-2.el9.noarch 2024-03-28T17:52:29.115 INFO:teuthology.orchestra.run.smithi022.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-03-28T17:52:29.115 INFO:teuthology.orchestra.run.smithi022.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-03-28T17:52:29.115 INFO:teuthology.orchestra.run.smithi022.stdout: python3-toml-0.10.2-6.el9.noarch 2024-03-28T17:52:29.115 INFO:teuthology.orchestra.run.smithi022.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-03-28T17:52:29.115 INFO:teuthology.orchestra.run.smithi022.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-03-28T17:52:29.115 INFO:teuthology.orchestra.run.smithi022.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-03-28T17:52:29.115 INFO:teuthology.orchestra.run.smithi022.stdout: python3-webob-1.8.7-6.el9.noarch 2024-03-28T17:52:29.115 INFO:teuthology.orchestra.run.smithi022.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-03-28T17:52:29.115 INFO:teuthology.orchestra.run.smithi022.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-03-28T17:52:29.116 INFO:teuthology.orchestra.run.smithi022.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-03-28T17:52:29.116 INFO:teuthology.orchestra.run.smithi022.stdout: qatlib-24.02.0-1.el9.x86_64 2024-03-28T17:52:29.116 INFO:teuthology.orchestra.run.smithi022.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-03-28T17:52:29.116 INFO:teuthology.orchestra.run.smithi022.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-03-28T17:52:29.116 INFO:teuthology.orchestra.run.smithi022.stdout: rbd-fuse-2:19.0.0-2145.g3be9a5c9.el9.x86_64 2024-03-28T17:52:29.116 INFO:teuthology.orchestra.run.smithi022.stdout: rbd-mirror-2:19.0.0-2145.g3be9a5c9.el9.x86_64 2024-03-28T17:52:29.116 INFO:teuthology.orchestra.run.smithi022.stdout: rbd-nbd-2:19.0.0-2145.g3be9a5c9.el9.x86_64 2024-03-28T17:52:29.116 INFO:teuthology.orchestra.run.smithi022.stdout: re2-1:20211101-3.el9.x86_64 2024-03-28T17:52:29.116 INFO:teuthology.orchestra.run.smithi022.stdout: socat-1.7.4.1-5.el9.x86_64 2024-03-28T17:52:29.116 INFO:teuthology.orchestra.run.smithi022.stdout: thrift-0.15.0-2.el9.x86_64 2024-03-28T17:52:29.116 INFO:teuthology.orchestra.run.smithi022.stdout: uuid-1.6.2-55.el9.x86_64 2024-03-28T17:52:29.116 INFO:teuthology.orchestra.run.smithi022.stdout: xfsprogs-devel-6.3.0-1.el9.x86_64 2024-03-28T17:52:29.116 INFO:teuthology.orchestra.run.smithi022.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-03-28T17:52:29.117 INFO:teuthology.orchestra.run.smithi022.stdout: zlib-devel-1.2.11-40.el9.x86_64 2024-03-28T17:52:29.117 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-03-28T17:52:29.117 INFO:teuthology.orchestra.run.smithi022.stdout:Complete! 2024-03-28T17:52:29.778 DEBUG:teuthology.parallel:result is None 2024-03-28T17:52:31.096 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : ceph-test-2:19.0.0-2145.g3be9a5c9.el9.x86_64 153/198 2024-03-28T17:52:31.162 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : rbd-fuse-2:19.0.0-2145.g3be9a5c9.el9.x86_64 154/198 2024-03-28T17:52:31.194 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : rbd-nbd-2:19.0.0-2145.g3be9a5c9.el9.x86_64 155/198 2024-03-28T17:52:31.233 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : elfutils-libelf-devel-0.190-2.el9.x86_64 156/198 2024-03-28T17:52:31.512 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : cephfs-shell-2:19.0.0-2145.g3be9a5c9.el9.x86_64 157/198 2024-03-28T17:52:31.646 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : postgresql-contrib-13.14-1.el9.x86_64 158/198 2024-03-28T17:52:31.695 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : ceph-fuse-2:19.0.0-2145.g3be9a5c9.el9.x86_64 159/198 2024-03-28T17:52:31.755 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : xfsprogs-devel-6.3.0-1.el9.x86_64 160/198 2024-03-28T17:52:31.867 INFO:teuthology.orchestra.run.smithi134.stdout: Upgrading : xfsdump-3.1.12-4.el9.x86_64 161/198 2024-03-28T17:52:31.915 INFO:teuthology.orchestra.run.smithi134.stdout: Upgrading : util-linux-user-2.37.4-18.el9.x86_64 162/198 2024-03-28T17:52:31.967 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : libcephfs-devel-2:19.0.0-2145.g3be9a5c9.el9.x86_ 163/198 2024-03-28T17:52:32.023 INFO:teuthology.orchestra.run.smithi134.stdout: Upgrading : elfutils-debuginfod-client-0.190-2.el9.x86_64 164/198 2024-03-28T17:52:32.070 INFO:teuthology.orchestra.run.smithi134.stdout: Upgrading : NetworkManager-team-1:1.47.2-1.el9.x86_64 165/198 2024-03-28T17:52:32.157 INFO:teuthology.orchestra.run.smithi134.stdout: Upgrading : NetworkManager-tui-1:1.47.2-1.el9.x86_64 166/198 2024-03-28T17:52:32.271 INFO:teuthology.orchestra.run.smithi134.stdout: Upgrading : acl-2.3.1-4.el9.x86_64 167/198 2024-03-28T17:52:32.328 INFO:teuthology.orchestra.run.smithi134.stdout: Upgrading : libacl-devel-2.3.1-4.el9.x86_64 168/198 2024-03-28T17:52:33.218 INFO:teuthology.orchestra.run.smithi134.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 169/198 2024-03-28T17:52:33.266 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : postgresql-server-13.14-1.el9.x86_64 169/198 2024-03-28T17:52:33.322 INFO:teuthology.orchestra.run.smithi134.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 169/198 2024-03-28T17:52:34.607 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : cephfs-top-2:19.0.0-2145.g3be9a5c9.el9.noarch 170/198 2024-03-28T17:52:34.704 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : openssl-devel-1:3.0.7-27.el9.x86_64 171/198 2024-03-28T17:52:34.762 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 172/198 2024-03-28T17:52:34.938 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : flex-2.6.4-9.el9.x86_64 173/198 2024-03-28T17:52:34.973 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : bison-3.7.4-5.el9.x86_64 174/198 2024-03-28T17:52:35.006 INFO:teuthology.orchestra.run.smithi134.stdout: Cleanup : NetworkManager-tui-1:1.44.0-2.el9.x86_64 175/198 2024-03-28T17:52:35.041 INFO:teuthology.orchestra.run.smithi134.stdout: Cleanup : elfutils-debuginfod-client-0.189-3.el9.x86_64 176/198 2024-03-28T17:52:35.085 INFO:teuthology.orchestra.run.smithi134.stdout: Cleanup : elfutils-libs-0.189-3.el9.x86_64 177/198 2024-03-28T17:52:35.085 INFO:teuthology.orchestra.run.smithi134.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 178/198 2024-03-28T17:52:35.252 INFO:teuthology.orchestra.run.smithi134.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 178/198 2024-03-28T17:52:35.303 INFO:teuthology.orchestra.run.smithi134.stdout: Cleanup : openssl-1:3.0.7-24.el9.x86_64 179/198 2024-03-28T17:52:35.303 INFO:teuthology.orchestra.run.smithi134.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 180/198 2024-03-28T17:52:35.424 INFO:teuthology.orchestra.run.smithi134.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 180/198 2024-03-28T17:52:35.477 INFO:teuthology.orchestra.run.smithi134.stdout: Cleanup : xfsdump-3.1.12-3.el9.x86_64 181/198 2024-03-28T17:52:35.523 INFO:teuthology.orchestra.run.smithi134.stdout: Cleanup : xfsprogs-5.19.0-4.el9.x86_64 182/198 2024-03-28T17:52:35.556 INFO:teuthology.orchestra.run.smithi134.stdout: Cleanup : acl-2.3.1-3.el9.x86_64 183/198 2024-03-28T17:52:35.588 INFO:teuthology.orchestra.run.smithi134.stdout: Cleanup : libacl-devel-2.3.1-3.el9.x86_64 184/198 2024-03-28T17:52:35.681 INFO:teuthology.orchestra.run.smithi134.stdout: Cleanup : util-linux-user-2.37.4-15.el9.x86_64 185/198 2024-03-28T17:52:35.751 INFO:teuthology.orchestra.run.smithi134.stdout: Cleanup : util-linux-2.37.4-15.el9.x86_64 186/198 2024-03-28T17:52:35.793 INFO:teuthology.orchestra.run.smithi134.stdout: Cleanup : util-linux-core-2.37.4-15.el9.x86_64 187/198 2024-03-28T17:52:35.826 INFO:teuthology.orchestra.run.smithi134.stdout: Cleanup : libfdisk-2.37.4-15.el9.x86_64 188/198 2024-03-28T17:52:35.860 INFO:teuthology.orchestra.run.smithi134.stdout: Cleanup : libmount-2.37.4-15.el9.x86_64 189/198 2024-03-28T17:52:35.893 INFO:teuthology.orchestra.run.smithi134.stdout: Cleanup : libblkid-2.37.4-15.el9.x86_64 190/198 2024-03-28T17:52:35.933 INFO:teuthology.orchestra.run.smithi134.stdout: Cleanup : NetworkManager-team-1:1.44.0-2.el9.x86_64 191/198 2024-03-28T17:52:35.970 INFO:teuthology.orchestra.run.smithi134.stdout: Running scriptlet: NetworkManager-1:1.44.0-2.el9.x86_64 192/198 2024-03-28T17:52:35.970 INFO:teuthology.orchestra.run.smithi134.stdout: Cleanup : NetworkManager-1:1.44.0-2.el9.x86_64 192/198 2024-03-28T17:52:36.310 INFO:teuthology.orchestra.run.smithi134.stdout: Running scriptlet: NetworkManager-1:1.44.0-2.el9.x86_64 192/198 2024-03-28T17:52:36.310 INFO:teuthology.orchestra.run.smithi134.stdout: Cleanup : NetworkManager-libnm-1:1.44.0-2.el9.x86_64 193/198 2024-03-28T17:52:36.487 INFO:teuthology.orchestra.run.smithi134.stdout: Running scriptlet: NetworkManager-libnm-1:1.44.0-2.el9.x86_64 193/198 2024-03-28T17:52:36.521 INFO:teuthology.orchestra.run.smithi134.stdout: Cleanup : libuuid-2.37.4-15.el9.x86_64 194/198 2024-03-28T17:52:36.576 INFO:teuthology.orchestra.run.smithi134.stdout: Cleanup : libsmartcols-2.37.4-15.el9.x86_64 195/198 2024-03-28T17:52:36.616 INFO:teuthology.orchestra.run.smithi134.stdout: Cleanup : libacl-2.3.1-3.el9.x86_64 196/198 2024-03-28T17:52:36.656 INFO:teuthology.orchestra.run.smithi134.stdout: Cleanup : openssl-libs-1:3.0.7-24.el9.x86_64 197/198 2024-03-28T17:52:36.785 INFO:teuthology.orchestra.run.smithi134.stdout: Cleanup : elfutils-libelf-0.189-3.el9.x86_64 198/198 2024-03-28T17:52:37.064 INFO:teuthology.orchestra.run.smithi134.stdout: Running scriptlet: elfutils-libelf-0.189-3.el9.x86_64 198/198 2024-03-28T17:52:37.064 INFO:teuthology.orchestra.run.smithi134.stdout:Couldn't write '0' to 'kernel/yama/ptrace_scope', ignoring: No such file or directory 2024-03-28T17:52:37.064 INFO:teuthology.orchestra.run.smithi134.stdout: 2024-03-28T17:52:41.923 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : ceph-2:19.0.0-2145.g3be9a5c9.el9.x86_64 1/198 2024-03-28T17:52:41.923 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : ceph-base-2:19.0.0-2145.g3be9a5c9.el9.x86_64 2/198 2024-03-28T17:52:41.923 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : ceph-common-2:19.0.0-2145.g3be9a5c9.el9.x86_64 3/198 2024-03-28T17:52:41.923 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : ceph-fuse-2:19.0.0-2145.g3be9a5c9.el9.x86_64 4/198 2024-03-28T17:52:41.923 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-2145.g3be9a 5/198 2024-03-28T17:52:41.924 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : ceph-mds-2:19.0.0-2145.g3be9a5c9.el9.x86_64 6/198 2024-03-28T17:52:41.924 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : ceph-mgr-2:19.0.0-2145.g3be9a5c9.el9.x86_64 7/198 2024-03-28T17:52:41.924 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : ceph-mon-2:19.0.0-2145.g3be9a5c9.el9.x86_64 8/198 2024-03-28T17:52:41.924 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : ceph-osd-2:19.0.0-2145.g3be9a5c9.el9.x86_64 9/198 2024-03-28T17:52:41.924 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : ceph-radosgw-2:19.0.0-2145.g3be9a5c9.el9.x86_64 10/198 2024-03-28T17:52:41.924 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : ceph-selinux-2:19.0.0-2145.g3be9a5c9.el9.x86_64 11/198 2024-03-28T17:52:41.924 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : ceph-test-2:19.0.0-2145.g3be9a5c9.el9.x86_64 12/198 2024-03-28T17:52:41.924 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : cephfs-mirror-2:19.0.0-2145.g3be9a5c9.el9.x86_64 13/198 2024-03-28T17:52:41.924 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : cephfs-shell-2:19.0.0-2145.g3be9a5c9.el9.x86_64 14/198 2024-03-28T17:52:41.924 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : libcephfs-devel-2:19.0.0-2145.g3be9a5c9.el9.x86_ 15/198 2024-03-28T17:52:41.924 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : libcephfs2-2:19.0.0-2145.g3be9a5c9.el9.x86_64 16/198 2024-03-28T17:52:41.924 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : libcephsqlite-2:19.0.0-2145.g3be9a5c9.el9.x86_64 17/198 2024-03-28T17:52:41.924 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : librados-devel-2:19.0.0-2145.g3be9a5c9.el9.x86_6 18/198 2024-03-28T17:52:41.924 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : libradosstriper1-2:19.0.0-2145.g3be9a5c9.el9.x86 19/198 2024-03-28T17:52:41.924 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : librgw2-2:19.0.0-2145.g3be9a5c9.el9.x86_64 20/198 2024-03-28T17:52:41.924 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-ceph-argparse-2:19.0.0-2145.g3be9a5c9.el 21/198 2024-03-28T17:52:41.925 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-ceph-common-2:19.0.0-2145.g3be9a5c9.el9. 22/198 2024-03-28T17:52:41.925 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-cephfs-2:19.0.0-2145.g3be9a5c9.el9.x86_6 23/198 2024-03-28T17:52:41.925 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-rados-2:19.0.0-2145.g3be9a5c9.el9.x86_64 24/198 2024-03-28T17:52:41.925 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-rbd-2:19.0.0-2145.g3be9a5c9.el9.x86_64 25/198 2024-03-28T17:52:41.942 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-rgw-2:19.0.0-2145.g3be9a5c9.el9.x86_64 26/198 2024-03-28T17:52:41.942 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : rbd-fuse-2:19.0.0-2145.g3be9a5c9.el9.x86_64 27/198 2024-03-28T17:52:41.942 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : rbd-mirror-2:19.0.0-2145.g3be9a5c9.el9.x86_64 28/198 2024-03-28T17:52:41.942 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : rbd-nbd-2:19.0.0-2145.g3be9a5c9.el9.x86_64 29/198 2024-03-28T17:52:41.942 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-2145.g3be9a5c9. 30/198 2024-03-28T17:52:41.942 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-2145.g3be9a5c9.el9.noa 31/198 2024-03-28T17:52:41.942 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-2145.g3be9a5c9.el9.n 32/198 2024-03-28T17:52:41.942 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-2145.g3be 33/198 2024-03-28T17:52:41.942 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-2145.g3be9a5c9.el 34/198 2024-03-28T17:52:41.942 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : ceph-mgr-rook-2:19.0.0-2145.g3be9a5c9.el9.noarch 35/198 2024-03-28T17:52:41.942 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-2145.g3be9a5c9.e 36/198 2024-03-28T17:52:41.943 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : ceph-volume-2:19.0.0-2145.g3be9a5c9.el9.noarch 37/198 2024-03-28T17:52:41.943 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : cephadm-2:19.0.0-2145.g3be9a5c9.el9.noarch 38/198 2024-03-28T17:52:41.943 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : cephfs-top-2:19.0.0-2145.g3be9a5c9.el9.noarch 39/198 2024-03-28T17:52:41.943 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : automake-1.16.2-8.el9.noarch 40/198 2024-03-28T17:52:41.943 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : bison-3.7.4-5.el9.x86_64 41/198 2024-03-28T17:52:41.943 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 42/198 2024-03-28T17:52:41.943 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : flex-2.6.4-9.el9.x86_64 43/198 2024-03-28T17:52:41.943 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 44/198 2024-03-28T17:52:41.943 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 45/198 2024-03-28T17:52:41.943 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 46/198 2024-03-28T17:52:41.943 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 47/198 2024-03-28T17:52:41.943 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 48/198 2024-03-28T17:52:41.943 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 49/198 2024-03-28T17:52:41.943 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 50/198 2024-03-28T17:52:41.944 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 51/198 2024-03-28T17:52:41.944 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 52/198 2024-03-28T17:52:41.944 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 53/198 2024-03-28T17:52:41.944 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : lua-5.4.4-4.el9.x86_64 54/198 2024-03-28T17:52:41.944 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : oniguruma-6.9.6-1.el9.5.x86_64 55/198 2024-03-28T17:52:41.944 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 56/198 2024-03-28T17:52:41.944 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 57/198 2024-03-28T17:52:41.944 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 58/198 2024-03-28T17:52:41.944 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 59/198 2024-03-28T17:52:41.944 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 60/198 2024-03-28T17:52:41.944 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 61/198 2024-03-28T17:52:41.944 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 62/198 2024-03-28T17:52:41.944 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 63/198 2024-03-28T17:52:41.944 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 64/198 2024-03-28T17:52:41.944 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 65/198 2024-03-28T17:52:41.945 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 66/198 2024-03-28T17:52:41.945 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 67/198 2024-03-28T17:52:41.945 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 68/198 2024-03-28T17:52:41.945 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 69/198 2024-03-28T17:52:41.945 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 70/198 2024-03-28T17:52:41.945 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 71/198 2024-03-28T17:52:41.945 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 72/198 2024-03-28T17:52:41.945 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 73/198 2024-03-28T17:52:41.945 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 74/198 2024-03-28T17:52:41.945 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 75/198 2024-03-28T17:52:41.945 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 76/198 2024-03-28T17:52:41.945 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 77/198 2024-03-28T17:52:41.945 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 78/198 2024-03-28T17:52:41.945 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 79/198 2024-03-28T17:52:41.945 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 80/198 2024-03-28T17:52:41.945 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : zlib-devel-1.2.11-40.el9.x86_64 81/198 2024-03-28T17:52:41.946 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 82/198 2024-03-28T17:52:41.946 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 83/198 2024-03-28T17:52:41.946 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 84/198 2024-03-28T17:52:41.946 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-ply-3.11-14.el9.noarch 85/198 2024-03-28T17:52:41.946 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 86/198 2024-03-28T17:52:41.946 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 87/198 2024-03-28T17:52:41.946 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : jq-1.6-16.el9.x86_64 88/198 2024-03-28T17:52:41.946 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 89/198 2024-03-28T17:52:41.946 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 90/198 2024-03-28T17:52:41.946 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 91/198 2024-03-28T17:52:41.946 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : elfutils-libelf-devel-0.190-2.el9.x86_64 92/198 2024-03-28T17:52:41.946 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 93/198 2024-03-28T17:52:41.946 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : openssl-devel-1:3.0.7-27.el9.x86_64 94/198 2024-03-28T17:52:41.946 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : postgresql-13.14-1.el9.x86_64 95/198 2024-03-28T17:52:41.947 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : postgresql-contrib-13.14-1.el9.x86_64 96/198 2024-03-28T17:52:41.947 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : postgresql-private-libs-13.14-1.el9.x86_64 97/198 2024-03-28T17:52:41.947 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : postgresql-server-13.14-1.el9.x86_64 98/198 2024-03-28T17:52:41.947 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-devel-3.9.17-2.el9.x86_64 99/198 2024-03-28T17:52:41.947 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 100/198 2024-03-28T17:52:41.947 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 101/198 2024-03-28T17:52:41.947 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 102/198 2024-03-28T17:52:41.947 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 103/198 2024-03-28T17:52:41.947 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : xfsprogs-devel-6.3.0-1.el9.x86_64 104/198 2024-03-28T17:52:41.947 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 105/198 2024-03-28T17:52:41.948 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 106/198 2024-03-28T17:52:41.948 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 107/198 2024-03-28T17:52:41.948 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 108/198 2024-03-28T17:52:41.948 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 109/198 2024-03-28T17:52:41.948 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : libarrow-9.0.0-10.el9.x86_64 110/198 2024-03-28T17:52:41.948 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : libarrow-doc-9.0.0-10.el9.noarch 111/198 2024-03-28T17:52:41.948 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 112/198 2024-03-28T17:52:41.948 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 113/198 2024-03-28T17:52:41.948 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : luarocks-3.9.2-1.el9.noarch 114/198 2024-03-28T17:52:41.948 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : parquet-libs-9.0.0-10.el9.x86_64 115/198 2024-03-28T17:52:41.948 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 116/198 2024-03-28T17:52:41.948 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 117/198 2024-03-28T17:52:41.948 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 118/198 2024-03-28T17:52:41.948 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 119/198 2024-03-28T17:52:41.948 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 120/198 2024-03-28T17:52:41.948 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 121/198 2024-03-28T17:52:41.949 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 122/198 2024-03-28T17:52:41.949 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-colorama-0.4.4-12.el9.noarch 123/198 2024-03-28T17:52:41.949 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-google-auth-1:2.28.2-1.el9.noarch 124/198 2024-03-28T17:52:41.949 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 125/198 2024-03-28T17:52:41.949 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 126/198 2024-03-28T17:52:41.949 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 127/198 2024-03-28T17:52:41.949 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 128/198 2024-03-28T17:52:41.949 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 129/198 2024-03-28T17:52:41.949 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 130/198 2024-03-28T17:52:41.949 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 131/198 2024-03-28T17:52:41.949 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 132/198 2024-03-28T17:52:41.949 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 133/198 2024-03-28T17:52:41.949 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 134/198 2024-03-28T17:52:41.949 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 135/198 2024-03-28T17:52:41.949 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 136/198 2024-03-28T17:52:41.950 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 137/198 2024-03-28T17:52:41.950 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 138/198 2024-03-28T17:52:41.950 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 139/198 2024-03-28T17:52:41.950 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 140/198 2024-03-28T17:52:41.950 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 141/198 2024-03-28T17:52:41.950 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 142/198 2024-03-28T17:52:41.950 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 143/198 2024-03-28T17:52:41.950 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 144/198 2024-03-28T17:52:41.950 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 145/198 2024-03-28T17:52:41.950 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 146/198 2024-03-28T17:52:41.950 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 147/198 2024-03-28T17:52:41.950 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 148/198 2024-03-28T17:52:41.950 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : re2-1:20211101-3.el9.x86_64 149/198 2024-03-28T17:52:41.950 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 150/198 2024-03-28T17:52:41.951 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : librados2-2:19.0.0-2145.g3be9a5c9.el9.x86_64 151/198 2024-03-28T17:52:41.951 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 152/198 2024-03-28T17:52:41.951 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : librbd1-2:19.0.0-2145.g3be9a5c9.el9.x86_64 153/198 2024-03-28T17:52:41.951 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 154/198 2024-03-28T17:52:41.951 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : xfsdump-3.1.12-4.el9.x86_64 155/198 2024-03-28T17:52:41.951 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : xfsdump-3.1.12-3.el9.x86_64 156/198 2024-03-28T17:52:41.951 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : NetworkManager-1:1.47.2-1.el9.x86_64 157/198 2024-03-28T17:52:41.951 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : NetworkManager-1:1.44.0-2.el9.x86_64 158/198 2024-03-28T17:52:41.951 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : NetworkManager-libnm-1:1.47.2-1.el9.x86_64 159/198 2024-03-28T17:52:41.951 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : NetworkManager-libnm-1:1.44.0-2.el9.x86_64 160/198 2024-03-28T17:52:41.951 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : NetworkManager-team-1:1.47.2-1.el9.x86_64 161/198 2024-03-28T17:52:41.951 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : NetworkManager-team-1:1.44.0-2.el9.x86_64 162/198 2024-03-28T17:52:41.952 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : NetworkManager-tui-1:1.47.2-1.el9.x86_64 163/198 2024-03-28T17:52:41.952 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : NetworkManager-tui-1:1.44.0-2.el9.x86_64 164/198 2024-03-28T17:52:41.952 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : acl-2.3.1-4.el9.x86_64 165/198 2024-03-28T17:52:41.952 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : acl-2.3.1-3.el9.x86_64 166/198 2024-03-28T17:52:41.952 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : elfutils-debuginfod-client-0.190-2.el9.x86_64 167/198 2024-03-28T17:52:41.952 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : elfutils-debuginfod-client-0.189-3.el9.x86_64 168/198 2024-03-28T17:52:41.952 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : elfutils-libelf-0.190-2.el9.x86_64 169/198 2024-03-28T17:52:41.952 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : elfutils-libelf-0.189-3.el9.x86_64 170/198 2024-03-28T17:52:41.952 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : elfutils-libs-0.190-2.el9.x86_64 171/198 2024-03-28T17:52:41.952 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : elfutils-libs-0.189-3.el9.x86_64 172/198 2024-03-28T17:52:41.952 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : libacl-2.3.1-4.el9.x86_64 173/198 2024-03-28T17:52:41.952 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : libacl-2.3.1-3.el9.x86_64 174/198 2024-03-28T17:52:41.952 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : libblkid-2.37.4-18.el9.x86_64 175/198 2024-03-28T17:52:41.952 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : libblkid-2.37.4-15.el9.x86_64 176/198 2024-03-28T17:52:41.952 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : libfdisk-2.37.4-18.el9.x86_64 177/198 2024-03-28T17:52:41.952 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : libfdisk-2.37.4-15.el9.x86_64 178/198 2024-03-28T17:52:41.953 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : libmount-2.37.4-18.el9.x86_64 179/198 2024-03-28T17:52:41.953 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : libmount-2.37.4-15.el9.x86_64 180/198 2024-03-28T17:52:41.953 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : libsmartcols-2.37.4-18.el9.x86_64 181/198 2024-03-28T17:52:41.953 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : libsmartcols-2.37.4-15.el9.x86_64 182/198 2024-03-28T17:52:41.953 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : libuuid-2.37.4-18.el9.x86_64 183/198 2024-03-28T17:52:41.953 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : libuuid-2.37.4-15.el9.x86_64 184/198 2024-03-28T17:52:41.953 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : openssl-1:3.0.7-27.el9.x86_64 185/198 2024-03-28T17:52:41.953 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : openssl-1:3.0.7-24.el9.x86_64 186/198 2024-03-28T17:52:41.953 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : openssl-libs-1:3.0.7-27.el9.x86_64 187/198 2024-03-28T17:52:41.953 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : openssl-libs-1:3.0.7-24.el9.x86_64 188/198 2024-03-28T17:52:41.953 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : util-linux-2.37.4-18.el9.x86_64 189/198 2024-03-28T17:52:41.953 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : util-linux-2.37.4-15.el9.x86_64 190/198 2024-03-28T17:52:41.953 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : util-linux-core-2.37.4-18.el9.x86_64 191/198 2024-03-28T17:52:41.953 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : util-linux-core-2.37.4-15.el9.x86_64 192/198 2024-03-28T17:52:41.953 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : util-linux-user-2.37.4-18.el9.x86_64 193/198 2024-03-28T17:52:41.953 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : util-linux-user-2.37.4-15.el9.x86_64 194/198 2024-03-28T17:52:41.954 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : xfsprogs-6.3.0-1.el9.x86_64 195/198 2024-03-28T17:52:41.954 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : xfsprogs-5.19.0-4.el9.x86_64 196/198 2024-03-28T17:52:41.954 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : libacl-devel-2.3.1-4.el9.x86_64 197/198 2024-03-28T17:52:44.451 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : libacl-devel-2.3.1-3.el9.x86_64 198/198 2024-03-28T17:52:44.451 INFO:teuthology.orchestra.run.smithi134.stdout: 2024-03-28T17:52:44.451 INFO:teuthology.orchestra.run.smithi134.stdout:Upgraded: 2024-03-28T17:52:44.452 INFO:teuthology.orchestra.run.smithi134.stdout: NetworkManager-1:1.47.2-1.el9.x86_64 2024-03-28T17:52:44.452 INFO:teuthology.orchestra.run.smithi134.stdout: NetworkManager-libnm-1:1.47.2-1.el9.x86_64 2024-03-28T17:52:44.452 INFO:teuthology.orchestra.run.smithi134.stdout: NetworkManager-team-1:1.47.2-1.el9.x86_64 2024-03-28T17:52:44.452 INFO:teuthology.orchestra.run.smithi134.stdout: NetworkManager-tui-1:1.47.2-1.el9.x86_64 2024-03-28T17:52:44.452 INFO:teuthology.orchestra.run.smithi134.stdout: acl-2.3.1-4.el9.x86_64 2024-03-28T17:52:44.452 INFO:teuthology.orchestra.run.smithi134.stdout: elfutils-debuginfod-client-0.190-2.el9.x86_64 2024-03-28T17:52:44.452 INFO:teuthology.orchestra.run.smithi134.stdout: elfutils-libelf-0.190-2.el9.x86_64 2024-03-28T17:52:44.452 INFO:teuthology.orchestra.run.smithi134.stdout: elfutils-libs-0.190-2.el9.x86_64 2024-03-28T17:52:44.452 INFO:teuthology.orchestra.run.smithi134.stdout: libacl-2.3.1-4.el9.x86_64 2024-03-28T17:52:44.452 INFO:teuthology.orchestra.run.smithi134.stdout: libacl-devel-2.3.1-4.el9.x86_64 2024-03-28T17:52:44.452 INFO:teuthology.orchestra.run.smithi134.stdout: libblkid-2.37.4-18.el9.x86_64 2024-03-28T17:52:44.452 INFO:teuthology.orchestra.run.smithi134.stdout: libfdisk-2.37.4-18.el9.x86_64 2024-03-28T17:52:44.452 INFO:teuthology.orchestra.run.smithi134.stdout: libmount-2.37.4-18.el9.x86_64 2024-03-28T17:52:44.453 INFO:teuthology.orchestra.run.smithi134.stdout: librados2-2:19.0.0-2145.g3be9a5c9.el9.x86_64 2024-03-28T17:52:44.453 INFO:teuthology.orchestra.run.smithi134.stdout: librbd1-2:19.0.0-2145.g3be9a5c9.el9.x86_64 2024-03-28T17:52:44.453 INFO:teuthology.orchestra.run.smithi134.stdout: libsmartcols-2.37.4-18.el9.x86_64 2024-03-28T17:52:44.453 INFO:teuthology.orchestra.run.smithi134.stdout: libuuid-2.37.4-18.el9.x86_64 2024-03-28T17:52:44.453 INFO:teuthology.orchestra.run.smithi134.stdout: openssl-1:3.0.7-27.el9.x86_64 2024-03-28T17:52:44.453 INFO:teuthology.orchestra.run.smithi134.stdout: openssl-libs-1:3.0.7-27.el9.x86_64 2024-03-28T17:52:44.453 INFO:teuthology.orchestra.run.smithi134.stdout: util-linux-2.37.4-18.el9.x86_64 2024-03-28T17:52:44.453 INFO:teuthology.orchestra.run.smithi134.stdout: util-linux-core-2.37.4-18.el9.x86_64 2024-03-28T17:52:44.453 INFO:teuthology.orchestra.run.smithi134.stdout: util-linux-user-2.37.4-18.el9.x86_64 2024-03-28T17:52:44.453 INFO:teuthology.orchestra.run.smithi134.stdout: xfsdump-3.1.12-4.el9.x86_64 2024-03-28T17:52:44.453 INFO:teuthology.orchestra.run.smithi134.stdout: xfsprogs-6.3.0-1.el9.x86_64 2024-03-28T17:52:44.453 INFO:teuthology.orchestra.run.smithi134.stdout:Installed: 2024-03-28T17:52:44.453 INFO:teuthology.orchestra.run.smithi134.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-03-28T17:52:44.453 INFO:teuthology.orchestra.run.smithi134.stdout: automake-1.16.2-8.el9.noarch 2024-03-28T17:52:44.453 INFO:teuthology.orchestra.run.smithi134.stdout: bison-3.7.4-5.el9.x86_64 2024-03-28T17:52:44.454 INFO:teuthology.orchestra.run.smithi134.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-03-28T17:52:44.454 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-2:19.0.0-2145.g3be9a5c9.el9.x86_64 2024-03-28T17:52:44.454 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-base-2:19.0.0-2145.g3be9a5c9.el9.x86_64 2024-03-28T17:52:44.454 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-common-2:19.0.0-2145.g3be9a5c9.el9.x86_64 2024-03-28T17:52:44.454 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-fuse-2:19.0.0-2145.g3be9a5c9.el9.x86_64 2024-03-28T17:52:44.454 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-grafana-dashboards-2:19.0.0-2145.g3be9a5c9.el9.noarch 2024-03-28T17:52:44.454 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-immutable-object-cache-2:19.0.0-2145.g3be9a5c9.el9.x86_64 2024-03-28T17:52:44.454 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-mds-2:19.0.0-2145.g3be9a5c9.el9.x86_64 2024-03-28T17:52:44.454 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-mgr-2:19.0.0-2145.g3be9a5c9.el9.x86_64 2024-03-28T17:52:44.454 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-mgr-cephadm-2:19.0.0-2145.g3be9a5c9.el9.noarch 2024-03-28T17:52:44.454 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-mgr-dashboard-2:19.0.0-2145.g3be9a5c9.el9.noarch 2024-03-28T17:52:44.454 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-mgr-diskprediction-local-2:19.0.0-2145.g3be9a5c9.el9.noarch 2024-03-28T17:52:44.454 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-mgr-modules-core-2:19.0.0-2145.g3be9a5c9.el9.noarch 2024-03-28T17:52:44.455 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-mgr-rook-2:19.0.0-2145.g3be9a5c9.el9.noarch 2024-03-28T17:52:44.455 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-mon-2:19.0.0-2145.g3be9a5c9.el9.x86_64 2024-03-28T17:52:44.455 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-osd-2:19.0.0-2145.g3be9a5c9.el9.x86_64 2024-03-28T17:52:44.455 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-prometheus-alerts-2:19.0.0-2145.g3be9a5c9.el9.noarch 2024-03-28T17:52:44.455 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-radosgw-2:19.0.0-2145.g3be9a5c9.el9.x86_64 2024-03-28T17:52:44.455 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-selinux-2:19.0.0-2145.g3be9a5c9.el9.x86_64 2024-03-28T17:52:44.455 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-test-2:19.0.0-2145.g3be9a5c9.el9.x86_64 2024-03-28T17:52:44.455 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-volume-2:19.0.0-2145.g3be9a5c9.el9.noarch 2024-03-28T17:52:44.455 INFO:teuthology.orchestra.run.smithi134.stdout: cephadm-2:19.0.0-2145.g3be9a5c9.el9.noarch 2024-03-28T17:52:44.455 INFO:teuthology.orchestra.run.smithi134.stdout: cephfs-mirror-2:19.0.0-2145.g3be9a5c9.el9.x86_64 2024-03-28T17:52:44.455 INFO:teuthology.orchestra.run.smithi134.stdout: cephfs-shell-2:19.0.0-2145.g3be9a5c9.el9.x86_64 2024-03-28T17:52:44.455 INFO:teuthology.orchestra.run.smithi134.stdout: cephfs-top-2:19.0.0-2145.g3be9a5c9.el9.noarch 2024-03-28T17:52:44.455 INFO:teuthology.orchestra.run.smithi134.stdout: elfutils-libelf-devel-0.190-2.el9.x86_64 2024-03-28T17:52:44.455 INFO:teuthology.orchestra.run.smithi134.stdout: flex-2.6.4-9.el9.x86_64 2024-03-28T17:52:44.456 INFO:teuthology.orchestra.run.smithi134.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-03-28T17:52:44.456 INFO:teuthology.orchestra.run.smithi134.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-03-28T17:52:44.456 INFO:teuthology.orchestra.run.smithi134.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-03-28T17:52:44.456 INFO:teuthology.orchestra.run.smithi134.stdout: fuse-2.9.9-15.el9.x86_64 2024-03-28T17:52:44.456 INFO:teuthology.orchestra.run.smithi134.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-03-28T17:52:44.456 INFO:teuthology.orchestra.run.smithi134.stdout: grpc-data-1.46.7-10.el9.noarch 2024-03-28T17:52:44.456 INFO:teuthology.orchestra.run.smithi134.stdout: jq-1.6-16.el9.x86_64 2024-03-28T17:52:44.456 INFO:teuthology.orchestra.run.smithi134.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-03-28T17:52:44.456 INFO:teuthology.orchestra.run.smithi134.stdout: libarrow-9.0.0-10.el9.x86_64 2024-03-28T17:52:44.456 INFO:teuthology.orchestra.run.smithi134.stdout: libarrow-doc-9.0.0-10.el9.noarch 2024-03-28T17:52:44.456 INFO:teuthology.orchestra.run.smithi134.stdout: libcephfs-devel-2:19.0.0-2145.g3be9a5c9.el9.x86_64 2024-03-28T17:52:44.456 INFO:teuthology.orchestra.run.smithi134.stdout: libcephfs2-2:19.0.0-2145.g3be9a5c9.el9.x86_64 2024-03-28T17:52:44.456 INFO:teuthology.orchestra.run.smithi134.stdout: libcephsqlite-2:19.0.0-2145.g3be9a5c9.el9.x86_64 2024-03-28T17:52:44.456 INFO:teuthology.orchestra.run.smithi134.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-03-28T17:52:44.457 INFO:teuthology.orchestra.run.smithi134.stdout: liboath-2.6.7-2.el9.x86_64 2024-03-28T17:52:44.457 INFO:teuthology.orchestra.run.smithi134.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-03-28T17:52:44.457 INFO:teuthology.orchestra.run.smithi134.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-03-28T17:52:44.457 INFO:teuthology.orchestra.run.smithi134.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-03-28T17:52:44.457 INFO:teuthology.orchestra.run.smithi134.stdout: librados-devel-2:19.0.0-2145.g3be9a5c9.el9.x86_64 2024-03-28T17:52:44.457 INFO:teuthology.orchestra.run.smithi134.stdout: libradosstriper1-2:19.0.0-2145.g3be9a5c9.el9.x86_64 2024-03-28T17:52:44.457 INFO:teuthology.orchestra.run.smithi134.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-03-28T17:52:44.457 INFO:teuthology.orchestra.run.smithi134.stdout: librgw2-2:19.0.0-2145.g3be9a5c9.el9.x86_64 2024-03-28T17:52:44.457 INFO:teuthology.orchestra.run.smithi134.stdout: libtool-2.4.6-45.el9.x86_64 2024-03-28T17:52:44.457 INFO:teuthology.orchestra.run.smithi134.stdout: libunwind-1.6.2-1.el9.x86_64 2024-03-28T17:52:44.457 INFO:teuthology.orchestra.run.smithi134.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-03-28T17:52:44.457 INFO:teuthology.orchestra.run.smithi134.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-03-28T17:52:44.457 INFO:teuthology.orchestra.run.smithi134.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-03-28T17:52:44.457 INFO:teuthology.orchestra.run.smithi134.stdout: lua-5.4.4-4.el9.x86_64 2024-03-28T17:52:44.458 INFO:teuthology.orchestra.run.smithi134.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-03-28T17:52:44.458 INFO:teuthology.orchestra.run.smithi134.stdout: luarocks-3.9.2-1.el9.noarch 2024-03-28T17:52:44.458 INFO:teuthology.orchestra.run.smithi134.stdout: oniguruma-6.9.6-1.el9.5.x86_64 2024-03-28T17:52:44.458 INFO:teuthology.orchestra.run.smithi134.stdout: openblas-0.3.21-2.el9.x86_64 2024-03-28T17:52:44.458 INFO:teuthology.orchestra.run.smithi134.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-03-28T17:52:44.458 INFO:teuthology.orchestra.run.smithi134.stdout: openssl-devel-1:3.0.7-27.el9.x86_64 2024-03-28T17:52:44.458 INFO:teuthology.orchestra.run.smithi134.stdout: parquet-libs-9.0.0-10.el9.x86_64 2024-03-28T17:52:44.458 INFO:teuthology.orchestra.run.smithi134.stdout: postgresql-13.14-1.el9.x86_64 2024-03-28T17:52:44.458 INFO:teuthology.orchestra.run.smithi134.stdout: postgresql-contrib-13.14-1.el9.x86_64 2024-03-28T17:52:44.458 INFO:teuthology.orchestra.run.smithi134.stdout: postgresql-private-libs-13.14-1.el9.x86_64 2024-03-28T17:52:44.458 INFO:teuthology.orchestra.run.smithi134.stdout: postgresql-server-13.14-1.el9.x86_64 2024-03-28T17:52:44.458 INFO:teuthology.orchestra.run.smithi134.stdout: protobuf-3.14.0-13.el9.x86_64 2024-03-28T17:52:44.458 INFO:teuthology.orchestra.run.smithi134.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-03-28T17:52:44.459 INFO:teuthology.orchestra.run.smithi134.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-03-28T17:52:44.459 INFO:teuthology.orchestra.run.smithi134.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-03-28T17:52:44.459 INFO:teuthology.orchestra.run.smithi134.stdout: python3-babel-2.9.1-2.el9.noarch 2024-03-28T17:52:44.459 INFO:teuthology.orchestra.run.smithi134.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-03-28T17:52:44.459 INFO:teuthology.orchestra.run.smithi134.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-03-28T17:52:44.459 INFO:teuthology.orchestra.run.smithi134.stdout: python3-ceph-argparse-2:19.0.0-2145.g3be9a5c9.el9.x86_64 2024-03-28T17:52:44.459 INFO:teuthology.orchestra.run.smithi134.stdout: python3-ceph-common-2:19.0.0-2145.g3be9a5c9.el9.x86_64 2024-03-28T17:52:44.459 INFO:teuthology.orchestra.run.smithi134.stdout: python3-cephfs-2:19.0.0-2145.g3be9a5c9.el9.x86_64 2024-03-28T17:52:44.459 INFO:teuthology.orchestra.run.smithi134.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-03-28T17:52:44.459 INFO:teuthology.orchestra.run.smithi134.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-03-28T17:52:44.459 INFO:teuthology.orchestra.run.smithi134.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-03-28T17:52:44.459 INFO:teuthology.orchestra.run.smithi134.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-03-28T17:52:44.459 INFO:teuthology.orchestra.run.smithi134.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-03-28T17:52:44.460 INFO:teuthology.orchestra.run.smithi134.stdout: python3-colorama-0.4.4-12.el9.noarch 2024-03-28T17:52:44.460 INFO:teuthology.orchestra.run.smithi134.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-03-28T17:52:44.460 INFO:teuthology.orchestra.run.smithi134.stdout: python3-devel-3.9.17-2.el9.x86_64 2024-03-28T17:52:44.460 INFO:teuthology.orchestra.run.smithi134.stdout: python3-google-auth-1:2.28.2-1.el9.noarch 2024-03-28T17:52:44.460 INFO:teuthology.orchestra.run.smithi134.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-03-28T17:52:44.460 INFO:teuthology.orchestra.run.smithi134.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-03-28T17:52:44.460 INFO:teuthology.orchestra.run.smithi134.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-03-28T17:52:44.460 INFO:teuthology.orchestra.run.smithi134.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-03-28T17:52:44.460 INFO:teuthology.orchestra.run.smithi134.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-03-28T17:52:44.460 INFO:teuthology.orchestra.run.smithi134.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-03-28T17:52:44.460 INFO:teuthology.orchestra.run.smithi134.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-03-28T17:52:44.460 INFO:teuthology.orchestra.run.smithi134.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-03-28T17:52:44.460 INFO:teuthology.orchestra.run.smithi134.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-03-28T17:52:44.461 INFO:teuthology.orchestra.run.smithi134.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-03-28T17:52:44.461 INFO:teuthology.orchestra.run.smithi134.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-03-28T17:52:44.461 INFO:teuthology.orchestra.run.smithi134.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-03-28T17:52:44.461 INFO:teuthology.orchestra.run.smithi134.stdout: python3-mako-1.1.4-6.el9.noarch 2024-03-28T17:52:44.461 INFO:teuthology.orchestra.run.smithi134.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-03-28T17:52:44.461 INFO:teuthology.orchestra.run.smithi134.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-03-28T17:52:44.461 INFO:teuthology.orchestra.run.smithi134.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-03-28T17:52:44.461 INFO:teuthology.orchestra.run.smithi134.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-03-28T17:52:44.461 INFO:teuthology.orchestra.run.smithi134.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-03-28T17:52:44.461 INFO:teuthology.orchestra.run.smithi134.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-03-28T17:52:44.461 INFO:teuthology.orchestra.run.smithi134.stdout: python3-packaging-20.9-5.el9.noarch 2024-03-28T17:52:44.461 INFO:teuthology.orchestra.run.smithi134.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-03-28T17:52:44.461 INFO:teuthology.orchestra.run.smithi134.stdout: python3-ply-3.11-14.el9.noarch 2024-03-28T17:52:44.462 INFO:teuthology.orchestra.run.smithi134.stdout: python3-portend-3.1.0-2.el9.noarch 2024-03-28T17:52:44.462 INFO:teuthology.orchestra.run.smithi134.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-03-28T17:52:44.462 INFO:teuthology.orchestra.run.smithi134.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-03-28T17:52:44.462 INFO:teuthology.orchestra.run.smithi134.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-03-28T17:52:44.462 INFO:teuthology.orchestra.run.smithi134.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-03-28T17:52:44.462 INFO:teuthology.orchestra.run.smithi134.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-03-28T17:52:44.462 INFO:teuthology.orchestra.run.smithi134.stdout: python3-pycparser-2.20-6.el9.noarch 2024-03-28T17:52:44.462 INFO:teuthology.orchestra.run.smithi134.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-03-28T17:52:44.462 INFO:teuthology.orchestra.run.smithi134.stdout: python3-pytz-2021.1-5.el9.noarch 2024-03-28T17:52:44.462 INFO:teuthology.orchestra.run.smithi134.stdout: python3-rados-2:19.0.0-2145.g3be9a5c9.el9.x86_64 2024-03-28T17:52:44.462 INFO:teuthology.orchestra.run.smithi134.stdout: python3-rbd-2:19.0.0-2145.g3be9a5c9.el9.x86_64 2024-03-28T17:52:44.462 INFO:teuthology.orchestra.run.smithi134.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-03-28T17:52:44.462 INFO:teuthology.orchestra.run.smithi134.stdout: python3-requests-2.25.1-8.el9.noarch 2024-03-28T17:52:44.463 INFO:teuthology.orchestra.run.smithi134.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-03-28T17:52:44.463 INFO:teuthology.orchestra.run.smithi134.stdout: python3-rgw-2:19.0.0-2145.g3be9a5c9.el9.x86_64 2024-03-28T17:52:44.463 INFO:teuthology.orchestra.run.smithi134.stdout: python3-routes-2.5.1-5.el9.noarch 2024-03-28T17:52:44.463 INFO:teuthology.orchestra.run.smithi134.stdout: python3-rsa-4.9-2.el9.noarch 2024-03-28T17:52:44.463 INFO:teuthology.orchestra.run.smithi134.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-03-28T17:52:44.463 INFO:teuthology.orchestra.run.smithi134.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-03-28T17:52:44.463 INFO:teuthology.orchestra.run.smithi134.stdout: python3-toml-0.10.2-6.el9.noarch 2024-03-28T17:52:44.463 INFO:teuthology.orchestra.run.smithi134.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-03-28T17:52:44.463 INFO:teuthology.orchestra.run.smithi134.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-03-28T17:52:44.463 INFO:teuthology.orchestra.run.smithi134.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-03-28T17:52:44.463 INFO:teuthology.orchestra.run.smithi134.stdout: python3-webob-1.8.7-6.el9.noarch 2024-03-28T17:52:44.463 INFO:teuthology.orchestra.run.smithi134.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-03-28T17:52:44.463 INFO:teuthology.orchestra.run.smithi134.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-03-28T17:52:44.463 INFO:teuthology.orchestra.run.smithi134.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-03-28T17:52:44.464 INFO:teuthology.orchestra.run.smithi134.stdout: qatlib-24.02.0-1.el9.x86_64 2024-03-28T17:52:44.464 INFO:teuthology.orchestra.run.smithi134.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-03-28T17:52:44.464 INFO:teuthology.orchestra.run.smithi134.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-03-28T17:52:44.464 INFO:teuthology.orchestra.run.smithi134.stdout: rbd-fuse-2:19.0.0-2145.g3be9a5c9.el9.x86_64 2024-03-28T17:52:44.464 INFO:teuthology.orchestra.run.smithi134.stdout: rbd-mirror-2:19.0.0-2145.g3be9a5c9.el9.x86_64 2024-03-28T17:52:44.464 INFO:teuthology.orchestra.run.smithi134.stdout: rbd-nbd-2:19.0.0-2145.g3be9a5c9.el9.x86_64 2024-03-28T17:52:44.464 INFO:teuthology.orchestra.run.smithi134.stdout: re2-1:20211101-3.el9.x86_64 2024-03-28T17:52:44.464 INFO:teuthology.orchestra.run.smithi134.stdout: socat-1.7.4.1-5.el9.x86_64 2024-03-28T17:52:44.464 INFO:teuthology.orchestra.run.smithi134.stdout: thrift-0.15.0-2.el9.x86_64 2024-03-28T17:52:44.464 INFO:teuthology.orchestra.run.smithi134.stdout: uuid-1.6.2-55.el9.x86_64 2024-03-28T17:52:44.464 INFO:teuthology.orchestra.run.smithi134.stdout: xfsprogs-devel-6.3.0-1.el9.x86_64 2024-03-28T17:52:44.464 INFO:teuthology.orchestra.run.smithi134.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-03-28T17:52:44.465 INFO:teuthology.orchestra.run.smithi134.stdout: zlib-devel-1.2.11-40.el9.x86_64 2024-03-28T17:52:44.465 INFO:teuthology.orchestra.run.smithi134.stdout: 2024-03-28T17:52:44.465 INFO:teuthology.orchestra.run.smithi134.stdout:Complete! 2024-03-28T17:52:45.144 DEBUG:teuthology.parallel:result is None 2024-03-28T17:52:45.144 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=3be9a5c9bc793e11e2800a8c0c696e8b46742033 2024-03-28T17:52:45.369 DEBUG:teuthology.orchestra.run.smithi022:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-03-28T17:52:45.447 INFO:teuthology.orchestra.run.smithi022.stdout:19.0.0-2145.g3be9a5c9.el9 2024-03-28T17:52:45.448 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-2145.g3be9a5c9.el9 2024-03-28T17:52:45.448 INFO:teuthology.task.install:The correct ceph version 19.0.0-2145.g3be9a5c9 is installed. 2024-03-28T17:52:45.450 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=3be9a5c9bc793e11e2800a8c0c696e8b46742033 2024-03-28T17:52:45.585 DEBUG:teuthology.orchestra.run.smithi079:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-03-28T17:52:45.651 INFO:teuthology.orchestra.run.smithi079.stdout:19.0.0-2145.g3be9a5c9.el9 2024-03-28T17:52:45.652 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-2145.g3be9a5c9.el9 2024-03-28T17:52:45.652 INFO:teuthology.task.install:The correct ceph version 19.0.0-2145.g3be9a5c9 is installed. 2024-03-28T17:52:45.653 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=3be9a5c9bc793e11e2800a8c0c696e8b46742033 2024-03-28T17:52:45.794 DEBUG:teuthology.orchestra.run.smithi134:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-03-28T17:52:45.849 INFO:teuthology.orchestra.run.smithi134.stdout:19.0.0-2145.g3be9a5c9.el9 2024-03-28T17:52:45.849 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-2145.g3be9a5c9.el9 2024-03-28T17:52:45.850 INFO:teuthology.task.install:The correct ceph version 19.0.0-2145.g3be9a5c9 is installed. 2024-03-28T17:52:45.851 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2024-03-28T17:52:45.852 DEBUG:teuthology.orchestra.run.smithi022:> set -ex 2024-03-28T17:52:45.852 DEBUG:teuthology.orchestra.run.smithi022:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-03-28T17:52:45.963 DEBUG:teuthology.orchestra.run.smithi079:> set -ex 2024-03-28T17:52:45.963 DEBUG:teuthology.orchestra.run.smithi079:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-03-28T17:52:46.067 DEBUG:teuthology.orchestra.run.smithi134:> set -ex 2024-03-28T17:52:46.067 DEBUG:teuthology.orchestra.run.smithi134:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-03-28T17:52:46.161 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2024-03-28T17:52:46.161 DEBUG:teuthology.orchestra.run.smithi022:> set -ex 2024-03-28T17:52:46.161 DEBUG:teuthology.orchestra.run.smithi022:> sudo dd of=/usr/bin/daemon-helper 2024-03-28T17:52:46.267 DEBUG:teuthology.orchestra.run.smithi022:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-03-28T17:52:46.372 DEBUG:teuthology.orchestra.run.smithi079:> set -ex 2024-03-28T17:52:46.372 DEBUG:teuthology.orchestra.run.smithi079:> sudo dd of=/usr/bin/daemon-helper 2024-03-28T17:52:46.468 DEBUG:teuthology.orchestra.run.smithi079:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-03-28T17:52:46.607 DEBUG:teuthology.orchestra.run.smithi134:> set -ex 2024-03-28T17:52:46.607 DEBUG:teuthology.orchestra.run.smithi134:> sudo dd of=/usr/bin/daemon-helper 2024-03-28T17:52:46.684 DEBUG:teuthology.orchestra.run.smithi134:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-03-28T17:52:46.803 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2024-03-28T17:52:46.803 DEBUG:teuthology.orchestra.run.smithi022:> set -ex 2024-03-28T17:52:46.803 DEBUG:teuthology.orchestra.run.smithi022:> sudo dd of=/usr/bin/adjust-ulimits 2024-03-28T17:52:46.905 DEBUG:teuthology.orchestra.run.smithi022:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-03-28T17:52:47.043 DEBUG:teuthology.orchestra.run.smithi079:> set -ex 2024-03-28T17:52:47.043 DEBUG:teuthology.orchestra.run.smithi079:> sudo dd of=/usr/bin/adjust-ulimits 2024-03-28T17:52:47.141 DEBUG:teuthology.orchestra.run.smithi079:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-03-28T17:52:47.275 DEBUG:teuthology.orchestra.run.smithi134:> set -ex 2024-03-28T17:52:47.275 DEBUG:teuthology.orchestra.run.smithi134:> sudo dd of=/usr/bin/adjust-ulimits 2024-03-28T17:52:47.343 DEBUG:teuthology.orchestra.run.smithi134:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-03-28T17:52:47.451 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2024-03-28T17:52:47.451 DEBUG:teuthology.orchestra.run.smithi022:> set -ex 2024-03-28T17:52:47.451 DEBUG:teuthology.orchestra.run.smithi022:> sudo dd of=/usr/bin/stdin-killer 2024-03-28T17:52:47.560 DEBUG:teuthology.orchestra.run.smithi022:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-03-28T17:52:47.662 DEBUG:teuthology.orchestra.run.smithi079:> set -ex 2024-03-28T17:52:47.662 DEBUG:teuthology.orchestra.run.smithi079:> sudo dd of=/usr/bin/stdin-killer 2024-03-28T17:52:47.761 DEBUG:teuthology.orchestra.run.smithi079:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-03-28T17:52:47.851 DEBUG:teuthology.orchestra.run.smithi134:> set -ex 2024-03-28T17:52:47.851 DEBUG:teuthology.orchestra.run.smithi134:> sudo dd of=/usr/bin/stdin-killer 2024-03-28T17:52:47.924 DEBUG:teuthology.orchestra.run.smithi134:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-03-28T17:52:48.034 INFO:teuthology.run_tasks:Running task cephadm... 2024-03-28T17:52:48.129 INFO:tasks.cephadm:Config: {'roleless': False, 'cephfs': {'max_mds': 5, 'session_timeout': 300, 'standby_replay': True}, 'conf': {'client': {'client dirsize rbytes': False, 'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds bal fragment size max': 10000, 'mds bal merge size': 5, 'mds bal split bits': 3, 'mds bal split size': 100, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'cephsqlite lock renewal timeout': 900000, 'debug cephsqlite': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon op complaint time': 120}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180, 'osd shutdown pgref assert': True, 'osd_max_omap_entries_per_request': 10000}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'POOL_APP_NOT_ENABLED', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'MON_DOWN', 'slow metadata IO', 'SLOW_OPS', 'slow request'], 'sha1': '3be9a5c9bc793e11e2800a8c0c696e8b46742033', 'subvols': {'create': 2, 'subvol_options': '--size 25000000000'}} 2024-03-28T17:52:48.130 INFO:tasks.cephadm:Cluster image is quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033 2024-03-28T17:52:48.130 INFO:tasks.cephadm:Cluster fsid is fd220748-ed2b-11ee-95d0-87774f69a715 2024-03-28T17:52:48.130 INFO:tasks.cephadm:Choosing monitor IPs and ports... 2024-03-28T17:52:48.131 INFO:tasks.cephadm:Monitor IPs: {'mon.a': '172.21.15.22', 'mon.b': '172.21.15.79', 'mon.c': '172.21.15.134'} 2024-03-28T17:52:48.131 INFO:tasks.cephadm:First mon is mon.a on smithi022 2024-03-28T17:52:48.131 INFO:tasks.cephadm:First mgr is x 2024-03-28T17:52:48.131 INFO:tasks.cephadm:Normalizing hostnames... 2024-03-28T17:52:48.131 DEBUG:teuthology.orchestra.run.smithi022:> sudo hostname $(hostname -s) 2024-03-28T17:52:48.240 DEBUG:teuthology.orchestra.run.smithi079:> sudo hostname $(hostname -s) 2024-03-28T17:52:48.341 DEBUG:teuthology.orchestra.run.smithi134:> sudo hostname $(hostname -s) 2024-03-28T17:52:48.415 INFO:tasks.cephadm:Downloading "compiled" cephadm from cachra 2024-03-28T17:52:48.416 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=3be9a5c9bc793e11e2800a8c0c696e8b46742033 2024-03-28T17:52:48.558 INFO:tasks.cephadm:builder_project result: [{'status': 'ready', 'sha1': '3be9a5c9bc793e11e2800a8c0c696e8b46742033', 'extra': {'build_url': 'https://jenkins.ceph.com/job/ceph-dev-new-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic/77877/', 'root_build_cause': 'SCMTRIGGER', 'version': '19.0.0-2145-g3be9a5c9', 'node_name': '172.21.5.36+adami06', 'job_name': 'ceph-dev-new-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic', 'package_manager_version': '19.0.0-2145.g3be9a5c9'}, 'url': 'https://4.chacra.ceph.com/r/ceph/wip-mchangir-testing-main-20240318.032620/3be9a5c9bc793e11e2800a8c0c696e8b46742033/centos/9/flavors/default/', 'distro_codename': None, 'modified': '2024-03-22 05:51:42.267384', 'distro_version': '9', 'project': 'ceph', 'flavor': 'default', 'ref': 'wip-mchangir-testing-main-20240318.032620', 'chacra_url': 'https://4.chacra.ceph.com/repos/ceph/wip-mchangir-testing-main-20240318.032620/3be9a5c9bc793e11e2800a8c0c696e8b46742033/centos/9/flavors/default/', 'archs': ['source', 'x86_64'], 'distro': 'centos'}] 2024-03-28T17:52:48.701 INFO:tasks.util.chacra:got chacra host 4.chacra.ceph.com, ref wip-mchangir-testing-main-20240318.032620, sha1 3be9a5c9bc793e11e2800a8c0c696e8b46742033 from https://shaman.ceph.com/api/search/?project=ceph&distros=centos%2F9%2Fx86_64&flavor=default&sha1=3be9a5c9bc793e11e2800a8c0c696e8b46742033 2024-03-28T17:52:48.703 INFO:tasks.cephadm:Discovered cachra url: https://4.chacra.ceph.com/binaries/ceph/wip-mchangir-testing-main-20240318.032620/3be9a5c9bc793e11e2800a8c0c696e8b46742033/centos/9/x86_64/flavors/default/cephadm 2024-03-28T17:52:48.703 DEBUG:teuthology.orchestra.run.smithi022:> curl --silent -L https://4.chacra.ceph.com/binaries/ceph/wip-mchangir-testing-main-20240318.032620/3be9a5c9bc793e11e2800a8c0c696e8b46742033/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-03-28T17:52:49.465 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-r--r--. 1 ubuntu ubuntu 767371 Mar 28 17:52 /home/ubuntu/cephtest/cephadm 2024-03-28T17:52:49.465 DEBUG:teuthology.orchestra.run.smithi079:> curl --silent -L https://4.chacra.ceph.com/binaries/ceph/wip-mchangir-testing-main-20240318.032620/3be9a5c9bc793e11e2800a8c0c696e8b46742033/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-03-28T17:52:50.036 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-r--r--. 1 ubuntu ubuntu 767371 Mar 28 17:52 /home/ubuntu/cephtest/cephadm 2024-03-28T17:52:50.036 DEBUG:teuthology.orchestra.run.smithi134:> curl --silent -L https://4.chacra.ceph.com/binaries/ceph/wip-mchangir-testing-main-20240318.032620/3be9a5c9bc793e11e2800a8c0c696e8b46742033/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-03-28T17:52:50.770 INFO:teuthology.orchestra.run.smithi134.stdout:-rw-r--r--. 1 ubuntu ubuntu 767371 Mar 28 17:52 /home/ubuntu/cephtest/cephadm 2024-03-28T17:52:50.770 DEBUG:teuthology.orchestra.run.smithi022:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-03-28T17:52:50.834 DEBUG:teuthology.orchestra.run.smithi079:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-03-28T17:52:50.894 DEBUG:teuthology.orchestra.run.smithi134:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-03-28T17:52:50.967 INFO:tasks.cephadm:Pulling image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033 on all hosts... 2024-03-28T17:52:50.968 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033 pull 2024-03-28T17:52:50.975 DEBUG:teuthology.orchestra.run.smithi079:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033 pull 2024-03-28T17:52:50.982 DEBUG:teuthology.orchestra.run.smithi134:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033 pull 2024-03-28T17:52:52.081 INFO:teuthology.orchestra.run.smithi079.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033... 2024-03-28T17:52:52.198 INFO:teuthology.orchestra.run.smithi022.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033... 2024-03-28T17:52:52.310 INFO:teuthology.orchestra.run.smithi134.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033... 2024-03-28T17:53:31.636 INFO:teuthology.orchestra.run.smithi022.stdout:{ 2024-03-28T17:53:31.636 INFO:teuthology.orchestra.run.smithi022.stdout: "ceph_version": "ceph version 19.0.0-2145-g3be9a5c9 (3be9a5c9bc793e11e2800a8c0c696e8b46742033) squid (dev)", 2024-03-28T17:53:31.636 INFO:teuthology.orchestra.run.smithi022.stdout: "image_id": "c664afe56bb01db4f9afa9532c20598e201e664f2e9ba2be50db0d52674a26d2", 2024-03-28T17:53:31.636 INFO:teuthology.orchestra.run.smithi022.stdout: "repo_digests": [ 2024-03-28T17:53:31.636 INFO:teuthology.orchestra.run.smithi022.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc" 2024-03-28T17:53:31.636 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-03-28T17:53:31.636 INFO:teuthology.orchestra.run.smithi022.stdout:} 2024-03-28T17:53:41.304 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-03-28T17:53:41.304 INFO:teuthology.orchestra.run.smithi079.stdout: "ceph_version": "ceph version 19.0.0-2145-g3be9a5c9 (3be9a5c9bc793e11e2800a8c0c696e8b46742033) squid (dev)", 2024-03-28T17:53:41.304 INFO:teuthology.orchestra.run.smithi079.stdout: "image_id": "c664afe56bb01db4f9afa9532c20598e201e664f2e9ba2be50db0d52674a26d2", 2024-03-28T17:53:41.304 INFO:teuthology.orchestra.run.smithi079.stdout: "repo_digests": [ 2024-03-28T17:53:41.304 INFO:teuthology.orchestra.run.smithi079.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc" 2024-03-28T17:53:41.304 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-03-28T17:53:41.304 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-03-28T17:53:45.624 INFO:teuthology.orchestra.run.smithi134.stdout:{ 2024-03-28T17:53:45.624 INFO:teuthology.orchestra.run.smithi134.stdout: "ceph_version": "ceph version 19.0.0-2145-g3be9a5c9 (3be9a5c9bc793e11e2800a8c0c696e8b46742033) squid (dev)", 2024-03-28T17:53:45.624 INFO:teuthology.orchestra.run.smithi134.stdout: "image_id": "c664afe56bb01db4f9afa9532c20598e201e664f2e9ba2be50db0d52674a26d2", 2024-03-28T17:53:45.625 INFO:teuthology.orchestra.run.smithi134.stdout: "repo_digests": [ 2024-03-28T17:53:45.625 INFO:teuthology.orchestra.run.smithi134.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc" 2024-03-28T17:53:45.625 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-03-28T17:53:45.625 INFO:teuthology.orchestra.run.smithi134.stdout:} 2024-03-28T17:53:45.689 DEBUG:teuthology.orchestra.run.smithi022:> sudo mkdir -p /etc/ceph 2024-03-28T17:53:45.793 DEBUG:teuthology.orchestra.run.smithi079:> sudo mkdir -p /etc/ceph 2024-03-28T17:53:45.896 DEBUG:teuthology.orchestra.run.smithi134:> sudo mkdir -p /etc/ceph 2024-03-28T17:53:45.990 DEBUG:teuthology.orchestra.run.smithi022:> sudo chmod 777 /etc/ceph 2024-03-28T17:53:46.087 DEBUG:teuthology.orchestra.run.smithi079:> sudo chmod 777 /etc/ceph 2024-03-28T17:53:46.186 DEBUG:teuthology.orchestra.run.smithi134:> sudo chmod 777 /etc/ceph 2024-03-28T17:53:46.288 INFO:tasks.cephadm:Writing seed config... 2024-03-28T17:53:46.289 INFO:tasks.cephadm: override: [client] client dirsize rbytes = False 2024-03-28T17:53:46.289 INFO:tasks.cephadm: override: [client] client mount timeout = 600 2024-03-28T17:53:46.289 INFO:tasks.cephadm: override: [client] debug client = 20 2024-03-28T17:53:46.289 INFO:tasks.cephadm: override: [client] debug ms = 1 2024-03-28T17:53:46.289 INFO:tasks.cephadm: override: [client] rados mon op timeout = 900 2024-03-28T17:53:46.289 INFO:tasks.cephadm: override: [client] rados osd op timeout = 900 2024-03-28T17:53:46.289 INFO:tasks.cephadm: override: [mds] debug mds = 20 2024-03-28T17:53:46.289 INFO:tasks.cephadm: override: [mds] debug mds balancer = 20 2024-03-28T17:53:46.290 INFO:tasks.cephadm: override: [mds] debug ms = 1 2024-03-28T17:53:46.290 INFO:tasks.cephadm: override: [mds] mds bal fragment size max = 10000 2024-03-28T17:53:46.290 INFO:tasks.cephadm: override: [mds] mds bal merge size = 5 2024-03-28T17:53:46.290 INFO:tasks.cephadm: override: [mds] mds bal split bits = 3 2024-03-28T17:53:46.290 INFO:tasks.cephadm: override: [mds] mds bal split size = 100 2024-03-28T17:53:46.290 INFO:tasks.cephadm: override: [mds] mds debug frag = True 2024-03-28T17:53:46.290 INFO:tasks.cephadm: override: [mds] mds debug scatterstat = True 2024-03-28T17:53:46.290 INFO:tasks.cephadm: override: [mds] mds op complaint time = 180 2024-03-28T17:53:46.290 INFO:tasks.cephadm: override: [mds] mds verify scatter = True 2024-03-28T17:53:46.290 INFO:tasks.cephadm: override: [mds] osd op complaint time = 180 2024-03-28T17:53:46.290 INFO:tasks.cephadm: override: [mds] rados mon op timeout = 900 2024-03-28T17:53:46.290 INFO:tasks.cephadm: override: [mds] rados osd op timeout = 900 2024-03-28T17:53:46.290 INFO:tasks.cephadm: override: [mgr] cephsqlite lock renewal timeout = 900000 2024-03-28T17:53:46.290 INFO:tasks.cephadm: override: [mgr] debug cephsqlite = 20 2024-03-28T17:53:46.291 INFO:tasks.cephadm: override: [mgr] debug mgr = 20 2024-03-28T17:53:46.291 INFO:tasks.cephadm: override: [mgr] debug ms = 1 2024-03-28T17:53:46.291 INFO:tasks.cephadm: override: [mon] debug mon = 20 2024-03-28T17:53:46.291 INFO:tasks.cephadm: override: [mon] debug ms = 1 2024-03-28T17:53:46.291 INFO:tasks.cephadm: override: [mon] debug paxos = 20 2024-03-28T17:53:46.291 INFO:tasks.cephadm: override: [mon] mon op complaint time = 120 2024-03-28T17:53:46.291 INFO:tasks.cephadm: override: [osd] bdev async discard = True 2024-03-28T17:53:46.291 INFO:tasks.cephadm: override: [osd] bdev enable discard = True 2024-03-28T17:53:46.291 INFO:tasks.cephadm: override: [osd] bluestore allocator = bitmap 2024-03-28T17:53:46.291 INFO:tasks.cephadm: override: [osd] bluestore block size = 96636764160 2024-03-28T17:53:46.291 INFO:tasks.cephadm: override: [osd] bluestore fsck on mount = True 2024-03-28T17:53:46.291 INFO:tasks.cephadm: override: [osd] debug bluefs = 1/20 2024-03-28T17:53:46.291 INFO:tasks.cephadm: override: [osd] debug bluestore = 1/20 2024-03-28T17:53:46.291 INFO:tasks.cephadm: override: [osd] debug ms = 1 2024-03-28T17:53:46.292 INFO:tasks.cephadm: override: [osd] debug osd = 20 2024-03-28T17:53:46.292 INFO:tasks.cephadm: override: [osd] debug rocksdb = 4/10 2024-03-28T17:53:46.292 INFO:tasks.cephadm: override: [osd] mon osd backfillfull_ratio = 0.85 2024-03-28T17:53:46.292 INFO:tasks.cephadm: override: [osd] mon osd full ratio = 0.9 2024-03-28T17:53:46.292 INFO:tasks.cephadm: override: [osd] mon osd nearfull ratio = 0.8 2024-03-28T17:53:46.292 INFO:tasks.cephadm: override: [osd] osd failsafe full ratio = 0.95 2024-03-28T17:53:46.292 INFO:tasks.cephadm: override: [osd] osd objectstore = bluestore 2024-03-28T17:53:46.292 INFO:tasks.cephadm: override: [osd] osd op complaint time = 180 2024-03-28T17:53:46.292 INFO:tasks.cephadm: override: [osd] osd shutdown pgref assert = True 2024-03-28T17:53:46.292 INFO:tasks.cephadm: override: [osd] osd_max_omap_entries_per_request = 10000 2024-03-28T17:53:46.293 DEBUG:teuthology.orchestra.run.smithi022:> set -ex 2024-03-28T17:53:46.293 DEBUG:teuthology.orchestra.run.smithi022:> dd of=/home/ubuntu/cephtest/seed.ceph.conf 2024-03-28T17:53:46.354 DEBUG:tasks.cephadm:Final config: [global] # make logging friendly to teuthology log_to_file = true log_to_stderr = false log to journald = false mon cluster log to file = true mon cluster log file level = debug mon clock drift allowed = 1.000 # replicate across OSDs, not hosts osd crush chooseleaf type = 0 #osd pool default size = 2 osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd # enable some debugging auth debug = true ms die on old message = true ms die on bug = true debug asserts on shutdown = true # adjust warnings mon max pg per osd = 10000# >= luminous mon pg warn max object skew = 0 mon osd allow primary affinity = true mon osd allow pg remap = true mon warn on legacy crush tunables = false mon warn on crush straw calc version zero = false mon warn on no sortbitwise = false mon warn on osd down out interval zero = false mon warn on too few osds = false mon_warn_on_pool_pg_num_not_power_of_two = false # disable pg_autoscaler by default for new pools osd_pool_default_pg_autoscale_mode = off # tests delete pools mon allow pool delete = true fsid = fd220748-ed2b-11ee-95d0-87774f69a715 [osd] osd scrub load threshold = 5.0 osd scrub max interval = 600 osd mclock profile = high_recovery_ops osd recover clone overlap = true osd recovery max chunk = 1048576 osd deep scrub update digest min age = 30 osd map max advance = 10 osd memory target autotune = true # debugging osd debug shutdown = true osd debug op order = true osd debug verify stray on activate = true osd debug pg log writeout = true osd debug verify cached snaps = true osd debug verify missing on start = true osd debug misdirected ops = true osd op queue = debug_random osd op queue cut off = debug_random osd shutdown pgref assert = True bdev debug aio = true osd sloppy crc = true bdev async discard = True bdev enable discard = True bluestore allocator = bitmap bluestore block size = 96636764160 bluestore fsck on mount = True debug bluefs = 1/20 debug bluestore = 1/20 debug ms = 1 debug osd = 20 debug rocksdb = 4/10 mon osd backfillfull_ratio = 0.85 mon osd full ratio = 0.9 mon osd nearfull ratio = 0.8 osd failsafe full ratio = 0.95 osd objectstore = bluestore osd op complaint time = 180 osd_max_omap_entries_per_request = 10000 [mgr] mon reweight min pgs per osd = 4 mon reweight min bytes per osd = 10 mgr/telemetry/nag = false cephsqlite lock renewal timeout = 900000 debug cephsqlite = 20 debug mgr = 20 debug ms = 1 [mon] mon data avail warn = 5 mon mgr mkfs grace = 240 mon reweight min pgs per osd = 4 mon osd reporter subtree level = osd mon osd prime pg temp = true mon reweight min bytes per osd = 10 # rotate auth tickets quickly to exercise renewal paths auth mon ticket ttl = 660# 11m auth service ticket ttl = 240# 4m # don't complain about global id reclaim mon_warn_on_insecure_global_id_reclaim = false mon_warn_on_insecure_global_id_reclaim_allowed = false debug mon = 20 debug ms = 1 debug paxos = 20 mon op complaint time = 120 [client.rgw] rgw cache enabled = true rgw enable ops log = true rgw enable usage log = true [client] client dirsize rbytes = False client mount timeout = 600 debug client = 20 debug ms = 1 rados mon op timeout = 900 rados osd op timeout = 900 [mds] debug mds = 20 debug mds balancer = 20 debug ms = 1 mds bal fragment size max = 10000 mds bal merge size = 5 mds bal split bits = 3 mds bal split size = 100 mds debug frag = True mds debug scatterstat = True mds op complaint time = 180 mds verify scatter = True osd op complaint time = 180 rados mon op timeout = 900 rados osd op timeout = 900 2024-03-28T17:53:46.355 DEBUG:teuthology.orchestra.run.smithi022:mon.a> sudo journalctl -f -n 0 -u ceph-fd220748-ed2b-11ee-95d0-87774f69a715@mon.a.service 2024-03-28T17:53:46.402 DEBUG:teuthology.orchestra.run.smithi022:mgr.x> sudo journalctl -f -n 0 -u ceph-fd220748-ed2b-11ee-95d0-87774f69a715@mgr.x.service 2024-03-28T17:53:46.448 INFO:tasks.cephadm:Bootstrapping... 2024-03-28T17:53:46.448 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033 -v bootstrap --fsid fd220748-ed2b-11ee-95d0-87774f69a715 --config /home/ubuntu/cephtest/seed.ceph.conf --output-config /etc/ceph/ceph.conf --output-keyring /etc/ceph/ceph.client.admin.keyring --output-pub-ssh-key /home/ubuntu/cephtest/ceph.pub --mon-id a --mgr-id x --orphan-initial-daemons --skip-monitoring-stack --mon-ip 172.21.15.22 --skip-admin-label && sudo chmod +r /etc/ceph/ceph.client.admin.keyring 2024-03-28T17:53:46.728 INFO:teuthology.orchestra.run.smithi022.stdout:-------------------------------------------------------------------------------- 2024-03-28T17:53:46.729 INFO:teuthology.orchestra.run.smithi022.stdout:cephadm ['--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033', '-v', 'bootstrap', '--fsid', 'fd220748-ed2b-11ee-95d0-87774f69a715', '--config', '/home/ubuntu/cephtest/seed.ceph.conf', '--output-config', '/etc/ceph/ceph.conf', '--output-keyring', '/etc/ceph/ceph.client.admin.keyring', '--output-pub-ssh-key', '/home/ubuntu/cephtest/ceph.pub', '--mon-id', 'a', '--mgr-id', 'x', '--orphan-initial-daemons', '--skip-monitoring-stack', '--mon-ip', '172.21.15.22', '--skip-admin-label'] 2024-03-28T17:53:46.790 INFO:teuthology.orchestra.run.smithi022.stdout:/bin/podman: stdout 4.9.4-dev 2024-03-28T17:53:46.790 INFO:teuthology.orchestra.run.smithi022.stderr:Specifying an fsid for your cluster offers no advantages and may increase the likelihood of fsid conflicts. 2024-03-28T17:53:46.791 INFO:teuthology.orchestra.run.smithi022.stdout:Verifying podman|docker is present... 2024-03-28T17:53:46.841 INFO:teuthology.orchestra.run.smithi022.stdout:/bin/podman: stdout 4.9.4-dev 2024-03-28T17:53:46.841 INFO:teuthology.orchestra.run.smithi022.stdout:Verifying lvm2 is present... 2024-03-28T17:53:46.841 INFO:teuthology.orchestra.run.smithi022.stdout:Verifying time synchronization is in place... 2024-03-28T17:53:46.858 INFO:teuthology.orchestra.run.smithi022.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-03-28T17:53:46.858 INFO:teuthology.orchestra.run.smithi022.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-03-28T17:53:46.876 INFO:teuthology.orchestra.run.smithi022.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-03-28T17:53:46.876 INFO:teuthology.orchestra.run.smithi022.stdout:systemctl: stdout inactive 2024-03-28T17:53:46.901 INFO:teuthology.orchestra.run.smithi022.stdout:systemctl: stdout enabled 2024-03-28T17:53:46.923 INFO:teuthology.orchestra.run.smithi022.stdout:systemctl: stdout active 2024-03-28T17:53:46.923 INFO:teuthology.orchestra.run.smithi022.stdout:Unit chronyd.service is enabled and running 2024-03-28T17:53:46.923 INFO:teuthology.orchestra.run.smithi022.stdout:Repeating the final host check... 2024-03-28T17:53:46.985 INFO:teuthology.orchestra.run.smithi022.stdout:/bin/podman: stdout 4.9.4-dev 2024-03-28T17:53:46.985 INFO:teuthology.orchestra.run.smithi022.stdout:podman (/bin/podman) version 4.9.4 is present 2024-03-28T17:53:46.985 INFO:teuthology.orchestra.run.smithi022.stdout:systemctl is present 2024-03-28T17:53:46.985 INFO:teuthology.orchestra.run.smithi022.stdout:lvcreate is present 2024-03-28T17:53:47.002 INFO:teuthology.orchestra.run.smithi022.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-03-28T17:53:47.002 INFO:teuthology.orchestra.run.smithi022.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-03-28T17:53:47.022 INFO:teuthology.orchestra.run.smithi022.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-03-28T17:53:47.022 INFO:teuthology.orchestra.run.smithi022.stdout:systemctl: stdout inactive 2024-03-28T17:53:47.044 INFO:teuthology.orchestra.run.smithi022.stdout:systemctl: stdout enabled 2024-03-28T17:53:47.065 INFO:teuthology.orchestra.run.smithi022.stdout:systemctl: stdout active 2024-03-28T17:53:47.065 INFO:teuthology.orchestra.run.smithi022.stdout:Unit chronyd.service is enabled and running 2024-03-28T17:53:47.065 INFO:teuthology.orchestra.run.smithi022.stdout:Host looks OK 2024-03-28T17:53:47.065 INFO:teuthology.orchestra.run.smithi022.stdout:Cluster fsid: fd220748-ed2b-11ee-95d0-87774f69a715 2024-03-28T17:53:47.065 INFO:teuthology.orchestra.run.smithi022.stdout:Acquiring lock 140104724021152 on /run/cephadm/fd220748-ed2b-11ee-95d0-87774f69a715.lock 2024-03-28T17:53:47.065 INFO:teuthology.orchestra.run.smithi022.stdout:Lock 140104724021152 acquired on /run/cephadm/fd220748-ed2b-11ee-95d0-87774f69a715.lock 2024-03-28T17:53:47.066 INFO:teuthology.orchestra.run.smithi022.stdout:Verifying IP 172.21.15.22 port 3300 ... 2024-03-28T17:53:47.066 INFO:teuthology.orchestra.run.smithi022.stdout:Verifying IP 172.21.15.22 port 6789 ... 2024-03-28T17:53:47.067 INFO:teuthology.orchestra.run.smithi022.stdout:Base mon IP(s) is [172.21.15.22:3300, 172.21.15.22:6789], mon addrv is [v2:172.21.15.22:3300,v1:172.21.15.22:6789] 2024-03-28T17:53:47.077 INFO:teuthology.orchestra.run.smithi022.stdout:/sbin/ip: stdout default via 172.21.15.254 dev ens1f0 proto dhcp src 172.21.15.22 metric 100 2024-03-28T17:53:47.077 INFO:teuthology.orchestra.run.smithi022.stdout:/sbin/ip: stdout 172.21.0.0/20 dev ens1f0 proto kernel scope link src 172.21.15.22 metric 100 2024-03-28T17:53:47.087 INFO:teuthology.orchestra.run.smithi022.stdout:/sbin/ip: stdout ::1 dev lo proto kernel metric 256 pref medium 2024-03-28T17:53:47.087 INFO:teuthology.orchestra.run.smithi022.stdout:/sbin/ip: stdout fe80::/64 dev ens1f0 proto kernel metric 1024 pref medium 2024-03-28T17:53:47.087 INFO:teuthology.orchestra.run.smithi022.stdout:/sbin/ip: stdout default via fe80::327c:5e00:6487:71e0 dev ens1f0 proto ra metric 100 pref medium 2024-03-28T17:53:47.100 INFO:teuthology.orchestra.run.smithi022.stdout:/sbin/ip: stdout 1: lo: mtu 65536 state UNKNOWN qlen 1000 2024-03-28T17:53:47.100 INFO:teuthology.orchestra.run.smithi022.stdout:/sbin/ip: stdout inet6 ::1/128 scope host 2024-03-28T17:53:47.100 INFO:teuthology.orchestra.run.smithi022.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-03-28T17:53:47.100 INFO:teuthology.orchestra.run.smithi022.stdout:/sbin/ip: stdout 4: ens1f0: mtu 1500 state UP qlen 1000 2024-03-28T17:53:47.100 INFO:teuthology.orchestra.run.smithi022.stdout:/sbin/ip: stdout inet6 fe80::ec4:7aff:fe59:5de4/64 scope link noprefixroute 2024-03-28T17:53:47.101 INFO:teuthology.orchestra.run.smithi022.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-03-28T17:53:47.104 INFO:teuthology.orchestra.run.smithi022.stdout:Mon IP `172.21.15.22` is in CIDR network `172.21.0.0/20` 2024-03-28T17:53:47.104 INFO:teuthology.orchestra.run.smithi022.stdout:Mon IP `172.21.15.22` is in CIDR network `172.21.0.0/20` 2024-03-28T17:53:47.104 INFO:teuthology.orchestra.run.smithi022.stdout:Inferred mon public CIDR from local network configuration ['172.21.0.0/20', '172.21.0.0/20'] 2024-03-28T17:53:47.108 INFO:teuthology.orchestra.run.smithi022.stdout:Internal network (--cluster-network) has not been provided, OSD replication will default to the public_network 2024-03-28T17:53:47.111 INFO:teuthology.orchestra.run.smithi022.stdout:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033... 2024-03-28T17:53:48.894 INFO:teuthology.orchestra.run.smithi022.stdout:/bin/podman: stdout c664afe56bb01db4f9afa9532c20598e201e664f2e9ba2be50db0d52674a26d2 2024-03-28T17:53:48.894 INFO:teuthology.orchestra.run.smithi022.stdout:/bin/podman: stderr Trying to pull quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033... 2024-03-28T17:53:48.894 INFO:teuthology.orchestra.run.smithi022.stdout:/bin/podman: stderr Getting image source signatures 2024-03-28T17:53:48.895 INFO:teuthology.orchestra.run.smithi022.stdout:/bin/podman: stderr Copying blob sha256:9516588f908cdb14b5e4062a6063db2d9291b355c3f32e2a99864608a4a7b649 2024-03-28T17:53:48.895 INFO:teuthology.orchestra.run.smithi022.stdout:/bin/podman: stderr Copying blob sha256:4dd83c6ee8921fe17396d77c31e205985ae94c87d3f752465b5f04aaf9ae340a 2024-03-28T17:53:48.895 INFO:teuthology.orchestra.run.smithi022.stdout:/bin/podman: stderr Copying config sha256:c664afe56bb01db4f9afa9532c20598e201e664f2e9ba2be50db0d52674a26d2 2024-03-28T17:53:48.895 INFO:teuthology.orchestra.run.smithi022.stdout:/bin/podman: stderr Writing manifest to image destination 2024-03-28T17:53:50.176 INFO:teuthology.orchestra.run.smithi022.stdout:ceph: stdout ceph version 19.0.0-2145-g3be9a5c9 (3be9a5c9bc793e11e2800a8c0c696e8b46742033) squid (dev) 2024-03-28T17:53:50.176 INFO:teuthology.orchestra.run.smithi022.stdout:Ceph version: ceph version 19.0.0-2145-g3be9a5c9 (3be9a5c9bc793e11e2800a8c0c696e8b46742033) squid (dev) 2024-03-28T17:53:50.176 INFO:teuthology.orchestra.run.smithi022.stdout:Extracting ceph user uid/gid from container image... 2024-03-28T17:53:51.337 INFO:teuthology.orchestra.run.smithi022.stdout:stat: stdout 167 167 2024-03-28T17:53:51.338 INFO:teuthology.orchestra.run.smithi022.stdout:Creating initial keys... 2024-03-28T17:53:52.463 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph-authtool: stdout AQCvrgVm2By4LxAAzIDtZgQEzRlZvKyenbnt3g== 2024-03-28T17:53:53.701 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph-authtool: stdout AQCwrgVmgP45NxAAF4qbqP+rrJahoaNE2b6VFg== 2024-03-28T17:53:55.684 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph-authtool: stdout AQCyrgVmZCpRCRAAlW7LXGE2yQIchKD+SKqsyg== 2024-03-28T17:53:55.686 INFO:teuthology.orchestra.run.smithi022.stdout:Creating initial monmap... 2024-03-28T17:53:56.765 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: monmap file /tmp/monmap 2024-03-28T17:53:56.765 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/monmaptool: stdout setting min_mon_release = quincy 2024-03-28T17:53:56.765 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: set fsid to fd220748-ed2b-11ee-95d0-87774f69a715 2024-03-28T17:53:56.765 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-03-28T17:53:56.766 INFO:teuthology.orchestra.run.smithi022.stdout:monmaptool for a [v2:172.21.15.22:3300,v1:172.21.15.22:6789] on /usr/bin/monmaptool: monmap file /tmp/monmap 2024-03-28T17:53:56.766 INFO:teuthology.orchestra.run.smithi022.stdout:setting min_mon_release = quincy 2024-03-28T17:53:56.766 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/monmaptool: set fsid to fd220748-ed2b-11ee-95d0-87774f69a715 2024-03-28T17:53:56.766 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-03-28T17:53:56.766 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-03-28T17:53:56.766 INFO:teuthology.orchestra.run.smithi022.stdout:Creating mon... 2024-03-28T17:53:57.965 INFO:teuthology.orchestra.run.smithi022.stdout:create mon.a on 2024-03-28T17:53:58.340 INFO:teuthology.orchestra.run.smithi022.stdout:systemctl: stderr Removed "/etc/systemd/system/multi-user.target.wants/ceph.target". 2024-03-28T17:53:58.575 INFO:teuthology.orchestra.run.smithi022.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /etc/systemd/system/ceph.target. 2024-03-28T17:53:58.898 INFO:teuthology.orchestra.run.smithi022.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph-fd220748-ed2b-11ee-95d0-87774f69a715.target → /etc/systemd/system/ceph-fd220748-ed2b-11ee-95d0-87774f69a715.target. 2024-03-28T17:53:58.899 INFO:teuthology.orchestra.run.smithi022.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph.target.wants/ceph-fd220748-ed2b-11ee-95d0-87774f69a715.target → /etc/systemd/system/ceph-fd220748-ed2b-11ee-95d0-87774f69a715.target. 2024-03-28T17:53:59.299 INFO:teuthology.orchestra.run.smithi022.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-fd220748-ed2b-11ee-95d0-87774f69a715@mon.a 2024-03-28T17:53:59.300 INFO:teuthology.orchestra.run.smithi022.stdout:systemctl: stderr Failed to reset failed state of unit ceph-fd220748-ed2b-11ee-95d0-87774f69a715@mon.a.service: Unit ceph-fd220748-ed2b-11ee-95d0-87774f69a715@mon.a.service not loaded. 2024-03-28T17:53:59.592 INFO:teuthology.orchestra.run.smithi022.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-fd220748-ed2b-11ee-95d0-87774f69a715.target.wants/ceph-fd220748-ed2b-11ee-95d0-87774f69a715@mon.a.service → /etc/systemd/system/ceph-fd220748-ed2b-11ee-95d0-87774f69a715@.service. 2024-03-28T17:54:00.263 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:54:00 smithi022 podman[31588]: 2024-03-28 17:54:00.104476114 +0000 UTC m=+0.253689046 container create 3d48d7665b0edc6d1031d3d56a3572a0041b6170c7d80f177a20e2df8a0844d0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033, name=ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mon-a, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240319, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_CLEAN=True) 2024-03-28T17:54:00.534 INFO:teuthology.orchestra.run.smithi022.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-03-28T17:54:00.534 INFO:teuthology.orchestra.run.smithi022.stdout:systemctl: stdout disabled 2024-03-28T17:54:00.557 INFO:teuthology.orchestra.run.smithi022.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-03-28T17:54:00.557 INFO:teuthology.orchestra.run.smithi022.stdout:systemctl: stdout inactive 2024-03-28T17:54:00.558 INFO:teuthology.orchestra.run.smithi022.stdout:firewalld.service is not enabled 2024-03-28T17:54:00.558 INFO:teuthology.orchestra.run.smithi022.stdout:Not possible to enable service . firewalld.service is not available 2024-03-28T17:54:00.559 INFO:teuthology.orchestra.run.smithi022.stdout:Waiting for mon to start... 2024-03-28T17:54:00.559 INFO:teuthology.orchestra.run.smithi022.stdout:Waiting for mon... 2024-03-28T17:54:00.720 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:54:00 smithi022 podman[31588]: 2024-03-28 17:54:00.352020248 +0000 UTC m=+0.501233230 container init 3d48d7665b0edc6d1031d3d56a3572a0041b6170c7d80f177a20e2df8a0844d0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033, name=ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mon-a, io.buildah.version=1.33.5, org.label-schema.build-date=20240319, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, ceph=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0) 2024-03-28T17:54:00.720 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:54:00 smithi022 podman[31588]: 2024-03-28 17:54:00.362866497 +0000 UTC m=+0.512079459 container start 3d48d7665b0edc6d1031d3d56a3572a0041b6170c7d80f177a20e2df8a0844d0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033, name=ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mon-a, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, ceph=True, org.label-schema.build-date=20240319, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS) 2024-03-28T17:54:00.720 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:54:00 smithi022 ceph-mon[31606]: mkfs fd220748-ed2b-11ee-95d0-87774f69a715 2024-03-28T17:54:00.720 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:54:00 smithi022 ceph-mon[31606]: mon.a is new leader, mons a in quorum (ranks 0) 2024-03-28T17:54:00.720 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:54:00 smithi022 bash[31588]: 3d48d7665b0edc6d1031d3d56a3572a0041b6170c7d80f177a20e2df8a0844d0 2024-03-28T17:54:00.720 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:54:00 smithi022 systemd[1]: Started Ceph mon.a for fd220748-ed2b-11ee-95d0-87774f69a715. 2024-03-28T17:54:01.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:54:01 smithi022 ceph-mon[31606]: mon.a is new leader, mons a in quorum (ranks 0) 2024-03-28T17:54:01.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:54:01 smithi022 ceph-mon[31606]: monmap e1: 1 mons at {a=[v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-03-28T17:54:01.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:54:01 smithi022 ceph-mon[31606]: fsmap 2024-03-28T17:54:01.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:54:01 smithi022 ceph-mon[31606]: osdmap e1: 0 total, 0 up, 0 in 2024-03-28T17:54:01.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:54:01 smithi022 ceph-mon[31606]: mgrmap e1: no daemons active 2024-03-28T17:54:01.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:54:01 smithi022 ceph-mon[31606]: from='client.? 172.21.15.22:0/1716173519' entity='client.admin' cmd=[{"prefix": "status"}]: dispatch 2024-03-28T17:54:01.938 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout cluster: 2024-03-28T17:54:01.947 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout id: fd220748-ed2b-11ee-95d0-87774f69a715 2024-03-28T17:54:01.947 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout health: HEALTH_OK 2024-03-28T17:54:01.947 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout 2024-03-28T17:54:01.948 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout services: 2024-03-28T17:54:01.948 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout mon: 1 daemons, quorum a (age 0.809584s) 2024-03-28T17:54:01.948 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout mgr: no daemons active 2024-03-28T17:54:01.948 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout osd: 0 osds: 0 up, 0 in 2024-03-28T17:54:01.948 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout 2024-03-28T17:54:01.948 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout data: 2024-03-28T17:54:01.948 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout pools: 0 pools, 0 pgs 2024-03-28T17:54:01.948 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout objects: 0 objects, 0 B 2024-03-28T17:54:01.948 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout usage: 0 B used, 0 B / 0 B avail 2024-03-28T17:54:01.949 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout pgs: 2024-03-28T17:54:01.949 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout 2024-03-28T17:54:01.949 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:01.154+0000 7fa6b2000700 1 Processor -- start 2024-03-28T17:54:01.949 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:01.155+0000 7fa6b2000700 1 -- start start 2024-03-28T17:54:01.949 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:01.155+0000 7fa6b2000700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa6ac109da0 0x7fa6ac10a1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:54:01.949 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:01.155+0000 7fa6b2000700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6ac10a790 con 0x7fa6ac109da0 2024-03-28T17:54:01.949 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:01.156+0000 7fa6ab400700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa6ac109da0 0x7fa6ac10a1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:54:01.949 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:01.157+0000 7fa6ab400700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa6ac109da0 0x7fa6ac10a1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:54392/0 (socket says 172.21.15.22:54392) 2024-03-28T17:54:01.949 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:01.157+0000 7fa6ab400700 1 -- 172.21.15.22:0/1540951044 learned_addr learned my addr 172.21.15.22:0/1540951044 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-28T17:54:01.949 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:01.157+0000 7fa6ab400700 1 -- 172.21.15.22:0/1540951044 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa6ac10af50 con 0x7fa6ac109da0 2024-03-28T17:54:01.950 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:01.158+0000 7fa6ab400700 1 --2- 172.21.15.22:0/1540951044 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa6ac109da0 0x7fa6ac10a1c0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7fa69c0098a0 tx=0x7fa69c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9657776b3766ed55 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T17:54:01.950 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:01.158+0000 7fa6aa000700 1 -- 172.21.15.22:0/1540951044 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fa69c004160 con 0x7fa6ac109da0 2024-03-28T17:54:01.950 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:01.158+0000 7fa6aa000700 1 -- 172.21.15.22:0/1540951044 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa69c004300 con 0x7fa6ac109da0 2024-03-28T17:54:01.950 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:01.158+0000 7fa6aa000700 1 -- 172.21.15.22:0/1540951044 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fa69c0036a0 con 0x7fa6ac109da0 2024-03-28T17:54:01.950 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:01.159+0000 7fa6b2000700 1 -- 172.21.15.22:0/1540951044 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa6ac109da0 msgr2=0x7fa6ac10a1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:54:01.950 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:01.159+0000 7fa6b2000700 1 --2- 172.21.15.22:0/1540951044 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa6ac109da0 0x7fa6ac10a1c0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7fa69c0098a0 tx=0x7fa69c009bb0 comp rx=0 tx=0).stop 2024-03-28T17:54:01.950 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:01.159+0000 7fa6b2000700 1 -- 172.21.15.22:0/1540951044 shutdown_connections 2024-03-28T17:54:01.950 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:01.159+0000 7fa6b2000700 1 --2- 172.21.15.22:0/1540951044 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa6ac109da0 0x7fa6ac10a1c0 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:54:01.950 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:01.159+0000 7fa6b2000700 1 -- 172.21.15.22:0/1540951044 >> 172.21.15.22:0/1540951044 conn(0x7fa6ac1053b0 msgr2=0x7fa6ac1077a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T17:54:01.950 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:01.159+0000 7fa6b2000700 1 -- 172.21.15.22:0/1540951044 shutdown_connections 2024-03-28T17:54:01.951 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:01.159+0000 7fa6b2000700 1 -- 172.21.15.22:0/1540951044 wait complete. 2024-03-28T17:54:01.951 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:01.160+0000 7fa6b2000700 1 Processor -- start 2024-03-28T17:54:01.951 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:01.161+0000 7fa6b2000700 1 -- start start 2024-03-28T17:54:01.951 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:01.161+0000 7fa6b2000700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa6ac109da0 0x7fa6ac19d4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:54:01.951 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:01.161+0000 7fa6b2000700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6ac10b480 con 0x7fa6ac109da0 2024-03-28T17:54:01.951 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:01.161+0000 7fa6ab400700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa6ac109da0 0x7fa6ac19d4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:54:01.951 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:01.161+0000 7fa6ab400700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa6ac109da0 0x7fa6ac19d4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:54404/0 (socket says 172.21.15.22:54404) 2024-03-28T17:54:01.951 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:01.161+0000 7fa6ab400700 1 -- 172.21.15.22:0/1716173519 learned_addr learned my addr 172.21.15.22:0/1716173519 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-28T17:54:01.951 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:01.162+0000 7fa6ab400700 1 -- 172.21.15.22:0/1716173519 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa6ac19da00 con 0x7fa6ac109da0 2024-03-28T17:54:01.951 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:01.162+0000 7fa6ab400700 1 --2- 172.21.15.22:0/1716173519 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa6ac109da0 0x7fa6ac19d4c0 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7fa69c000c00 tx=0x7fa69c004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T17:54:01.952 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:01.162+0000 7fa6a3e00700 1 -- 172.21.15.22:0/1716173519 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fa69c003db0 con 0x7fa6ac109da0 2024-03-28T17:54:01.952 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:01.162+0000 7fa6a3e00700 1 -- 172.21.15.22:0/1716173519 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa69c011630 con 0x7fa6ac109da0 2024-03-28T17:54:01.952 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:01.162+0000 7fa6b2000700 1 -- 172.21.15.22:0/1716173519 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa6ac19dc90 con 0x7fa6ac109da0 2024-03-28T17:54:01.952 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:01.162+0000 7fa6a3e00700 1 -- 172.21.15.22:0/1716173519 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fa69c011950 con 0x7fa6ac109da0 2024-03-28T17:54:01.952 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:01.162+0000 7fa6b2000700 1 -- 172.21.15.22:0/1716173519 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa6ac1a0980 con 0x7fa6ac109da0 2024-03-28T17:54:01.952 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:01.163+0000 7fa6a3e00700 1 -- 172.21.15.22:0/1716173519 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 1) v1 ==== 807+0+0 (secure 0 0 0) 0x7fa69c01a460 con 0x7fa6ac109da0 2024-03-28T17:54:01.952 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:01.163+0000 7fa6a3e00700 1 -- 172.21.15.22:0/1716173519 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7fa69c01b780 con 0x7fa6ac109da0 2024-03-28T17:54:01.952 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:01.164+0000 7fa6b2000700 1 -- 172.21.15.22:0/1716173519 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa6ac0637c0 con 0x7fa6ac109da0 2024-03-28T17:54:01.952 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:01.167+0000 7fa6a3e00700 1 -- 172.21.15.22:0/1716173519 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73141 (secure 0 0 0) 0x7fa69c031020 con 0x7fa6ac109da0 2024-03-28T17:54:01.953 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:01.247+0000 7fa6b2000700 1 -- 172.21.15.22:0/1716173519 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "status"} v 0) v1 -- 0x7fa6ac02dd30 con 0x7fa6ac109da0 2024-03-28T17:54:01.953 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:01.248+0000 7fa6a3e00700 1 -- 172.21.15.22:0/1716173519 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "status"}]=0 v0) v1 ==== 54+0+317 (secure 0 0 0) 0x7fa69c01b980 con 0x7fa6ac109da0 2024-03-28T17:54:01.953 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:01.249+0000 7fa6b2000700 1 -- 172.21.15.22:0/1716173519 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa6ac109da0 msgr2=0x7fa6ac19d4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:54:01.953 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:01.249+0000 7fa6b2000700 1 --2- 172.21.15.22:0/1716173519 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa6ac109da0 0x7fa6ac19d4c0 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7fa69c000c00 tx=0x7fa69c004750 comp rx=0 tx=0).stop 2024-03-28T17:54:01.953 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:01.249+0000 7fa6b2000700 1 -- 172.21.15.22:0/1716173519 shutdown_connections 2024-03-28T17:54:01.953 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:01.249+0000 7fa6b2000700 1 --2- 172.21.15.22:0/1716173519 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa6ac109da0 0x7fa6ac19d4c0 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:54:01.953 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:01.249+0000 7fa6b2000700 1 -- 172.21.15.22:0/1716173519 >> 172.21.15.22:0/1716173519 conn(0x7fa6ac1053b0 msgr2=0x7fa6ac194050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T17:54:01.953 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:01.249+0000 7fa6b2000700 1 -- 172.21.15.22:0/1716173519 shutdown_connections 2024-03-28T17:54:01.953 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:01.249+0000 7fa6b2000700 1 -- 172.21.15.22:0/1716173519 wait complete. 2024-03-28T17:54:01.953 INFO:teuthology.orchestra.run.smithi022.stdout:mon is available 2024-03-28T17:54:01.953 INFO:teuthology.orchestra.run.smithi022.stdout:Assimilating anything we can from ceph.conf... 2024-03-28T17:54:02.989 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:54:02 smithi022 ceph-mon[31606]: from='client.? 172.21.15.22:0/3724273601' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2024-03-28T17:54:02.989 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:54:02 smithi022 ceph-mon[31606]: from='client.? 172.21.15.22:0/3724273601' entity='client.admin' cmd='[{"prefix": "config assimilate-conf"}]': finished 2024-03-28T17:54:03.323 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout 2024-03-28T17:54:03.323 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout [global] 2024-03-28T17:54:03.323 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout fsid = fd220748-ed2b-11ee-95d0-87774f69a715 2024-03-28T17:54:03.324 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout mon_host = [v2:172.21.15.22:3300,v1:172.21.15.22:6789] 2024-03-28T17:54:03.324 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-03-28T17:54:03.324 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-03-28T17:54:03.324 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-03-28T17:54:03.324 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-03-28T17:54:03.324 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout 2024-03-28T17:54:03.324 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout [mgr] 2024-03-28T17:54:03.324 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-03-28T17:54:03.324 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout 2024-03-28T17:54:03.324 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout [osd] 2024-03-28T17:54:03.324 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout bdev_async_discard = True 2024-03-28T17:54:03.324 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2024-03-28T17:54:03.324 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2024-03-28T17:54:03.325 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2024-03-28T17:54:03.325 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-03-28T17:54:03.325 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-03-28T17:54:03.325 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:02.598+0000 7f1dc0400700 1 Processor -- start 2024-03-28T17:54:03.325 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:02.599+0000 7f1dc0400700 1 -- start start 2024-03-28T17:54:03.325 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:02.599+0000 7f1dc0400700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1db8109db0 0x7f1db810a1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:54:03.325 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:02.599+0000 7f1dc0400700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1db810a7a0 con 0x7f1db8109db0 2024-03-28T17:54:03.325 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:02.599+0000 7f1dbda00700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1db8109db0 0x7f1db810a1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:54:03.325 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:02.600+0000 7f1dbda00700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1db8109db0 0x7f1db810a1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:54416/0 (socket says 172.21.15.22:54416) 2024-03-28T17:54:03.325 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:02.600+0000 7f1dbda00700 1 -- 172.21.15.22:0/104654695 learned_addr learned my addr 172.21.15.22:0/104654695 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-28T17:54:03.325 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:02.600+0000 7f1dbda00700 1 -- 172.21.15.22:0/104654695 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1db810af60 con 0x7f1db8109db0 2024-03-28T17:54:03.325 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:02.600+0000 7f1dbda00700 1 --2- 172.21.15.22:0/104654695 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1db8109db0 0x7f1db810a1d0 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7f1dac009e80 tx=0x7f1dac00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6dce0b30ad4ac665 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T17:54:03.325 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:02.601+0000 7f1db7e00700 1 -- 172.21.15.22:0/104654695 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f1dac017070 con 0x7f1db8109db0 2024-03-28T17:54:03.325 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:02.601+0000 7f1db7e00700 1 -- 172.21.15.22:0/104654695 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1dac003a20 con 0x7f1db8109db0 2024-03-28T17:54:03.326 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:02.601+0000 7f1db7e00700 1 -- 172.21.15.22:0/104654695 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f1dac003d20 con 0x7f1db8109db0 2024-03-28T17:54:03.326 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:02.601+0000 7f1dc0400700 1 -- 172.21.15.22:0/104654695 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1db8109db0 msgr2=0x7f1db810a1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:54:03.326 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:02.601+0000 7f1dc0400700 1 --2- 172.21.15.22:0/104654695 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1db8109db0 0x7f1db810a1d0 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7f1dac009e80 tx=0x7f1dac00b330 comp rx=0 tx=0).stop 2024-03-28T17:54:03.326 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:02.601+0000 7f1dc0400700 1 -- 172.21.15.22:0/104654695 shutdown_connections 2024-03-28T17:54:03.326 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:02.601+0000 7f1dc0400700 1 --2- 172.21.15.22:0/104654695 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1db8109db0 0x7f1db810a1d0 unknown :-1 s=CLOSED pgs=3 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:54:03.326 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:02.601+0000 7f1dc0400700 1 -- 172.21.15.22:0/104654695 >> 172.21.15.22:0/104654695 conn(0x7f1db807d150 msgr2=0x7f1db807d5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T17:54:03.326 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:02.602+0000 7f1dc0400700 1 -- 172.21.15.22:0/104654695 shutdown_connections 2024-03-28T17:54:03.326 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:02.602+0000 7f1dc0400700 1 -- 172.21.15.22:0/104654695 wait complete. 2024-03-28T17:54:03.326 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:02.603+0000 7f1dc0400700 1 Processor -- start 2024-03-28T17:54:03.326 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:02.603+0000 7f1dc0400700 1 -- start start 2024-03-28T17:54:03.326 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:02.603+0000 7f1dc0400700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1db8109db0 0x7f1db819d500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:54:03.326 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:02.603+0000 7f1dc0400700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1db810b490 con 0x7f1db8109db0 2024-03-28T17:54:03.326 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:02.604+0000 7f1dbda00700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1db8109db0 0x7f1db819d500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:54:03.326 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:02.604+0000 7f1dbda00700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1db8109db0 0x7f1db819d500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:54422/0 (socket says 172.21.15.22:54422) 2024-03-28T17:54:03.327 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:02.604+0000 7f1dbda00700 1 -- 172.21.15.22:0/3724273601 learned_addr learned my addr 172.21.15.22:0/3724273601 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-28T17:54:03.327 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:02.604+0000 7f1dbda00700 1 -- 172.21.15.22:0/3724273601 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1db819da40 con 0x7f1db8109db0 2024-03-28T17:54:03.327 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:02.604+0000 7f1dbda00700 1 --2- 172.21.15.22:0/3724273601 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1db8109db0 0x7f1db819d500 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7f1dac004000 tx=0x7f1dac004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T17:54:03.327 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:02.605+0000 7f1db6000700 1 -- 172.21.15.22:0/3724273601 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f1dac022020 con 0x7f1db8109db0 2024-03-28T17:54:03.327 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:02.605+0000 7f1db6000700 1 -- 172.21.15.22:0/3724273601 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1dac018040 con 0x7f1db8109db0 2024-03-28T17:54:03.327 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:02.605+0000 7f1db6000700 1 -- 172.21.15.22:0/3724273601 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f1dac017070 con 0x7f1db8109db0 2024-03-28T17:54:03.327 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:02.605+0000 7f1dc0400700 1 -- 172.21.15.22:0/3724273601 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1db819dcd0 con 0x7f1db8109db0 2024-03-28T17:54:03.327 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:02.605+0000 7f1dc0400700 1 -- 172.21.15.22:0/3724273601 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1db81a09c0 con 0x7f1db8109db0 2024-03-28T17:54:03.327 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:02.605+0000 7f1db6000700 1 -- 172.21.15.22:0/3724273601 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 1) v1 ==== 807+0+0 (secure 0 0 0) 0x7f1dac025020 con 0x7f1db8109db0 2024-03-28T17:54:03.327 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:02.605+0000 7f1db6000700 1 -- 172.21.15.22:0/3724273601 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f1dac00f050 con 0x7f1db8109db0 2024-03-28T17:54:03.327 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:02.606+0000 7f1dc0400700 1 -- 172.21.15.22:0/3724273601 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1db80637c0 con 0x7f1db8109db0 2024-03-28T17:54:03.327 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:02.609+0000 7f1db6000700 1 -- 172.21.15.22:0/3724273601 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73141 (secure 0 0 0) 0x7f1dac02c030 con 0x7f1db8109db0 2024-03-28T17:54:03.327 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:02.686+0000 7f1dc0400700 1 -- 172.21.15.22:0/3724273601 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "config assimilate-conf"} v 0) v1 -- 0x7f1db802db70 con 0x7f1db8109db0 2024-03-28T17:54:03.327 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:02.693+0000 7f1db6000700 1 -- 172.21.15.22:0/3724273601 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v2) v1 ==== 70+0+456 (secure 0 0 0) 0x7f1dac01ede0 con 0x7f1db8109db0 2024-03-28T17:54:03.328 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:02.694+0000 7f1db6000700 1 -- 172.21.15.22:0/3724273601 <== mon.0 v2:172.21.15.22:3300/0 8 ==== config(27 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1dac035070 con 0x7f1db8109db0 2024-03-28T17:54:03.328 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:02.696+0000 7f1dc0400700 1 -- 172.21.15.22:0/3724273601 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1db8109db0 msgr2=0x7f1db819d500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:54:03.328 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:02.696+0000 7f1dc0400700 1 --2- 172.21.15.22:0/3724273601 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1db8109db0 0x7f1db819d500 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7f1dac004000 tx=0x7f1dac004ad0 comp rx=0 tx=0).stop 2024-03-28T17:54:03.328 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:02.696+0000 7f1dc0400700 1 -- 172.21.15.22:0/3724273601 shutdown_connections 2024-03-28T17:54:03.328 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:02.696+0000 7f1dc0400700 1 --2- 172.21.15.22:0/3724273601 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1db8109db0 0x7f1db819d500 unknown :-1 s=CLOSED pgs=4 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:54:03.328 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:02.696+0000 7f1dc0400700 1 -- 172.21.15.22:0/3724273601 >> 172.21.15.22:0/3724273601 conn(0x7f1db807d150 msgr2=0x7f1db8194110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T17:54:03.328 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:02.696+0000 7f1dc0400700 1 -- 172.21.15.22:0/3724273601 shutdown_connections 2024-03-28T17:54:03.328 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:02.696+0000 7f1dc0400700 1 -- 172.21.15.22:0/3724273601 wait complete. 2024-03-28T17:54:03.328 INFO:teuthology.orchestra.run.smithi022.stdout:Generating new minimal ceph.conf... 2024-03-28T17:54:04.282 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:54:04 smithi022 ceph-mon[31606]: from='client.? 172.21.15.22:0/3446079865' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T17:54:04.684 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:03.937+0000 7f4f88a00700 1 Processor -- start 2024-03-28T17:54:04.684 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:03.938+0000 7f4f88a00700 1 -- start start 2024-03-28T17:54:04.684 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:03.938+0000 7f4f88a00700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4f84108690 0x7f4f8410aa80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:54:04.684 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:03.938+0000 7f4f88a00700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f840758f0 con 0x7f4f84108690 2024-03-28T17:54:04.684 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:03.938+0000 7f4f81e00700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4f84108690 0x7f4f8410aa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:54:04.684 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:03.939+0000 7f4f81e00700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4f84108690 0x7f4f8410aa80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:54432/0 (socket says 172.21.15.22:54432) 2024-03-28T17:54:04.684 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:03.939+0000 7f4f81e00700 1 -- 172.21.15.22:0/719314367 learned_addr learned my addr 172.21.15.22:0/719314367 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-28T17:54:04.684 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:03.939+0000 7f4f81e00700 1 -- 172.21.15.22:0/719314367 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4f8410afc0 con 0x7f4f84108690 2024-03-28T17:54:04.684 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:03.939+0000 7f4f81e00700 1 --2- 172.21.15.22:0/719314367 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4f84108690 0x7f4f8410aa80 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f4f740098a0 tx=0x7f4f74009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=108b86282dfb922f server_cookie=0 in_seq=0 out_seq=0 2024-03-28T17:54:04.684 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:03.940+0000 7f4f80a00700 1 -- 172.21.15.22:0/719314367 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f4f74004160 con 0x7f4f84108690 2024-03-28T17:54:04.684 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:03.940+0000 7f4f80a00700 1 -- 172.21.15.22:0/719314367 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4f74004300 con 0x7f4f84108690 2024-03-28T17:54:04.685 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:03.940+0000 7f4f80a00700 1 -- 172.21.15.22:0/719314367 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f4f74011b40 con 0x7f4f84108690 2024-03-28T17:54:04.685 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:03.941+0000 7f4f88a00700 1 -- 172.21.15.22:0/719314367 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4f84108690 msgr2=0x7f4f8410aa80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:54:04.685 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:03.941+0000 7f4f88a00700 1 --2- 172.21.15.22:0/719314367 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4f84108690 0x7f4f8410aa80 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f4f740098a0 tx=0x7f4f74009bb0 comp rx=0 tx=0).stop 2024-03-28T17:54:04.685 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:03.941+0000 7f4f88a00700 1 -- 172.21.15.22:0/719314367 shutdown_connections 2024-03-28T17:54:04.685 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:03.941+0000 7f4f88a00700 1 --2- 172.21.15.22:0/719314367 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4f84108690 0x7f4f8410aa80 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:54:04.685 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:03.941+0000 7f4f88a00700 1 -- 172.21.15.22:0/719314367 >> 172.21.15.22:0/719314367 conn(0x7f4f841020a0 msgr2=0x7f4f841044c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T17:54:04.685 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:03.941+0000 7f4f88a00700 1 -- 172.21.15.22:0/719314367 shutdown_connections 2024-03-28T17:54:04.685 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:03.941+0000 7f4f88a00700 1 -- 172.21.15.22:0/719314367 wait complete. 2024-03-28T17:54:04.685 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:03.942+0000 7f4f88a00700 1 Processor -- start 2024-03-28T17:54:04.685 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:03.943+0000 7f4f88a00700 1 -- start start 2024-03-28T17:54:04.685 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:03.943+0000 7f4f88a00700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4f84108690 0x7f4f84107f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:54:04.685 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:03.943+0000 7f4f88a00700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f8410bf80 con 0x7f4f84108690 2024-03-28T17:54:04.685 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:03.943+0000 7f4f81e00700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4f84108690 0x7f4f84107f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:54:04.685 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:03.943+0000 7f4f81e00700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4f84108690 0x7f4f84107f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:54434/0 (socket says 172.21.15.22:54434) 2024-03-28T17:54:04.686 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:03.943+0000 7f4f81e00700 1 -- 172.21.15.22:0/3446079865 learned_addr learned my addr 172.21.15.22:0/3446079865 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-28T17:54:04.686 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:03.944+0000 7f4f81e00700 1 -- 172.21.15.22:0/3446079865 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4f84108450 con 0x7f4f84108690 2024-03-28T17:54:04.686 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:03.944+0000 7f4f81e00700 1 --2- 172.21.15.22:0/3446079865 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4f84108690 0x7f4f84107f10 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f4f74004180 tx=0x7f4f740046a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T17:54:04.686 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:03.944+0000 7f4f7aa00700 1 -- 172.21.15.22:0/3446079865 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f4f740039e0 con 0x7f4f84108690 2024-03-28T17:54:04.686 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:03.944+0000 7f4f7aa00700 1 -- 172.21.15.22:0/3446079865 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4f74020070 con 0x7f4f84108690 2024-03-28T17:54:04.686 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:03.944+0000 7f4f7aa00700 1 -- 172.21.15.22:0/3446079865 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f4f740193c0 con 0x7f4f84108690 2024-03-28T17:54:04.686 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:03.944+0000 7f4f88a00700 1 -- 172.21.15.22:0/3446079865 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4f84106640 con 0x7f4f84108690 2024-03-28T17:54:04.686 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:03.944+0000 7f4f88a00700 1 -- 172.21.15.22:0/3446079865 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4f84106b80 con 0x7f4f84108690 2024-03-28T17:54:04.686 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:03.945+0000 7f4f7aa00700 1 -- 172.21.15.22:0/3446079865 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 1) v1 ==== 807+0+0 (secure 0 0 0) 0x7f4f740199c0 con 0x7f4f84108690 2024-03-28T17:54:04.686 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:03.945+0000 7f4f7aa00700 1 -- 172.21.15.22:0/3446079865 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f4f7400b720 con 0x7f4f84108690 2024-03-28T17:54:04.686 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:03.946+0000 7f4f88a00700 1 -- 172.21.15.22:0/3446079865 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4f840637c0 con 0x7f4f84108690 2024-03-28T17:54:04.686 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:03.951+0000 7f4f7aa00700 1 -- 172.21.15.22:0/3446079865 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73141 (secure 0 0 0) 0x7f4f74003b80 con 0x7f4f84108690 2024-03-28T17:54:04.686 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:04.030+0000 7f4f88a00700 1 -- 172.21.15.22:0/3446079865 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "config generate-minimal-conf"} v 0) v1 -- 0x7f4f841070b0 con 0x7f4f84108690 2024-03-28T17:54:04.686 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:04.030+0000 7f4f7aa00700 1 -- 172.21.15.22:0/3446079865 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v2) v1 ==== 76+0+175 (secure 0 0 0) 0x7f4f74019df0 con 0x7f4f84108690 2024-03-28T17:54:04.687 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:04.032+0000 7f4f88a00700 1 -- 172.21.15.22:0/3446079865 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4f84108690 msgr2=0x7f4f84107f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:54:04.687 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:04.032+0000 7f4f88a00700 1 --2- 172.21.15.22:0/3446079865 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4f84108690 0x7f4f84107f10 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f4f74004180 tx=0x7f4f740046a0 comp rx=0 tx=0).stop 2024-03-28T17:54:04.687 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:04.032+0000 7f4f88a00700 1 -- 172.21.15.22:0/3446079865 shutdown_connections 2024-03-28T17:54:04.687 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:04.032+0000 7f4f88a00700 1 --2- 172.21.15.22:0/3446079865 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4f84108690 0x7f4f84107f10 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:54:04.687 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:04.032+0000 7f4f88a00700 1 -- 172.21.15.22:0/3446079865 >> 172.21.15.22:0/3446079865 conn(0x7f4f841020a0 msgr2=0x7f4f84102bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T17:54:04.687 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:04.032+0000 7f4f88a00700 1 -- 172.21.15.22:0/3446079865 shutdown_connections 2024-03-28T17:54:04.687 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:04.032+0000 7f4f88a00700 1 -- 172.21.15.22:0/3446079865 wait complete. 2024-03-28T17:54:04.687 INFO:teuthology.orchestra.run.smithi022.stdout:Restarting the monitor... 2024-03-28T17:54:05.031 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:54:04 smithi022 systemd[1]: Stopping Ceph mon.a for fd220748-ed2b-11ee-95d0-87774f69a715... 2024-03-28T17:54:05.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:54:05 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mon-a[31602]: 2024-03-28T17:54:05.125+0000 7f1f88600700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-03-28T17:54:05.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:54:05 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mon-a[31602]: 2024-03-28T17:54:05.125+0000 7f1f88600700 -1 mon.a@0(leader) e1 *** Got Signal Terminated *** 2024-03-28T17:54:05.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:54:05 smithi022 podman[31859]: 2024-03-28 17:54:05.248584433 +0000 UTC m=+0.246332225 container died 3d48d7665b0edc6d1031d3d56a3572a0041b6170c7d80f177a20e2df8a0844d0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033, name=ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mon-a, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, CEPH_POINT_RELEASE=, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.build-date=20240319, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c) 2024-03-28T17:54:05.904 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:54:05 smithi022 podman[31859]: 2024-03-28 17:54:05.686351868 +0000 UTC m=+0.684099653 container cleanup 3d48d7665b0edc6d1031d3d56a3572a0041b6170c7d80f177a20e2df8a0844d0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033, name=ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mon-a, GIT_CLEAN=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, ceph=True, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.build-date=20240319, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0) 2024-03-28T17:54:05.904 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:54:05 smithi022 bash[31859]: ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mon-a 2024-03-28T17:54:06.184 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:54:05 smithi022 podman[31873]: 2024-03-28 17:54:05.903695804 +0000 UTC m=+0.639797516 container remove 3d48d7665b0edc6d1031d3d56a3572a0041b6170c7d80f177a20e2df8a0844d0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033, name=ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mon-a, org.label-schema.build-date=20240319, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, ceph=True, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2) 2024-03-28T17:54:06.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:54:06 smithi022 systemd[1]: ceph-fd220748-ed2b-11ee-95d0-87774f69a715@mon.a.service: Deactivated successfully. 2024-03-28T17:54:06.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:54:06 smithi022 systemd[1]: Stopped Ceph mon.a for fd220748-ed2b-11ee-95d0-87774f69a715. 2024-03-28T17:54:06.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:54:06 smithi022 systemd[1]: ceph-fd220748-ed2b-11ee-95d0-87774f69a715@mon.a.service: Consumed 1.915s CPU time. 2024-03-28T17:54:06.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:54:06 smithi022 systemd[1]: Starting Ceph mon.a for fd220748-ed2b-11ee-95d0-87774f69a715... 2024-03-28T17:54:07.133 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:54:06 smithi022 podman[31967]: 2024-03-28 17:54:06.757776277 +0000 UTC m=+0.033508520 image pull c664afe56bb01db4f9afa9532c20598e201e664f2e9ba2be50db0d52674a26d2 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033 2024-03-28T17:54:07.134 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:54:06 smithi022 podman[31967]: 2024-03-28 17:54:06.879679314 +0000 UTC m=+0.155411583 container create c8a13c8d55b2de981388bb3edb0bc18641bbb8c7b95a1814f61fdd39ed21d506 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033, name=ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mon-a, org.label-schema.license=GPLv2, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, ceph=True, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240319, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS) 2024-03-28T17:54:07.134 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:54:07 smithi022 podman[31967]: 2024-03-28 17:54:07.132930709 +0000 UTC m=+0.408663048 container init c8a13c8d55b2de981388bb3edb0bc18641bbb8c7b95a1814f61fdd39ed21d506 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033, name=ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mon-a, org.label-schema.schema-version=1.0, GIT_CLEAN=True, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, org.label-schema.build-date=20240319, org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-28T17:54:07.224 INFO:teuthology.orchestra.run.smithi022.stdout:Setting public_network to 172.21.0.0/20 in mon config section 2024-03-28T17:54:07.487 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:54:07 smithi022 podman[31967]: 2024-03-28 17:54:07.14331098 +0000 UTC m=+0.419043219 container start c8a13c8d55b2de981388bb3edb0bc18641bbb8c7b95a1814f61fdd39ed21d506 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033, name=ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mon-a, org.label-schema.build-date=20240319, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, ceph=True, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_CLEAN=True, CEPH_POINT_RELEASE=) 2024-03-28T17:54:07.487 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:54:07 smithi022 ceph-mon[31985]: set uid:gid to 167:167 (ceph:ceph) 2024-03-28T17:54:07.488 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:54:07 smithi022 ceph-mon[31985]: ceph version 19.0.0-2145-g3be9a5c9 (3be9a5c9bc793e11e2800a8c0c696e8b46742033) squid (dev), process ceph-mon, pid 2 2024-03-28T17:54:07.488 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:54:07 smithi022 ceph-mon[31985]: pidfile_write: ignore empty --pid-file 2024-03-28T17:54:07.488 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:54:07 smithi022 ceph-mon[31985]: load: jerasure load: lrc 2024-03-28T17:54:07.488 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:54:07 smithi022 ceph-mon[31985]: rocksdb: RocksDB version: 7.9.2 2024-03-28T17:54:07.488 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:54:07 smithi022 ceph-mon[31985]: rocksdb: Git sha 0 2024-03-28T17:54:07.488 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:54:07 smithi022 ceph-mon[31985]: rocksdb: Compile date 2024-03-22 04:58:55 2024-03-28T17:54:07.488 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:54:07 smithi022 ceph-mon[31985]: rocksdb: DB SUMMARY 2024-03-28T17:54:07.488 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:54:07 smithi022 ceph-mon[31985]: rocksdb: DB Session ID: 08AM0AGZ8YZCP0FWOO3X 2024-03-28T17:54:07.488 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:54:07 smithi022 ceph-mon[31985]: rocksdb: CURRENT file: CURRENT 2024-03-28T17:54:07.488 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:54:07 smithi022 ceph-mon[31985]: rocksdb: IDENTITY file: IDENTITY 2024-03-28T17:54:07.488 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:54:07 smithi022 ceph-mon[31985]: rocksdb: MANIFEST file: MANIFEST-000010 size: 179 Bytes 2024-03-28T17:54:07.488 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:54:07 smithi022 ceph-mon[31985]: rocksdb: SST files in /var/lib/ceph/mon/ceph-a/store.db dir, Total Num: 1, files: 000008.sst 2024-03-28T17:54:07.488 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:54:07 smithi022 ceph-mon[31985]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-a/store.db: 000009.log size: 100388 ; 2024-03-28T17:54:07.488 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:54:07 smithi022 ceph-mon[31985]: rocksdb: Options.error_if_exists: 0 2024-03-28T17:54:07.489 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:54:07 smithi022 ceph-mon[31985]: rocksdb: Options.create_if_missing: 0 2024-03-28T17:54:07.489 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:54:07 smithi022 ceph-mon[31985]: rocksdb: Options.paranoid_checks: 1 2024-03-28T17:54:07.489 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:54:07 smithi022 ceph-mon[31985]: rocksdb: Options.flush_verify_memtable_count: 1 2024-03-28T17:54:07.489 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:54:07 smithi022 ceph-mon[31985]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-03-28T17:54:07.489 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:54:07 smithi022 ceph-mon[31985]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-03-28T17:54:07.489 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:54:07 smithi022 ceph-mon[31985]: rocksdb: Options.env: 0x555b485b8b20 2024-03-28T17:54:07.489 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:54:07 smithi022 ceph-mon[31985]: rocksdb: Options.fs: PosixFileSystem 2024-03-28T17:54:07.489 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:54:07 smithi022 ceph-mon[31985]: rocksdb: Options.info_log: 0x555b49339960 2024-03-28T17:54:07.489 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:54:07 smithi022 ceph-mon[31985]: rocksdb: Options.max_file_opening_threads: 16 2024-03-28T17:54:07.489 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:54:07 smithi022 ceph-mon[31985]: rocksdb: Options.statistics: (nil) 2024-03-28T17:54:07.489 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:54:07 smithi022 ceph-mon[31985]: rocksdb: Options.use_fsync: 0 2024-03-28T17:54:07.489 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:54:07 smithi022 ceph-mon[31985]: rocksdb: Options.max_log_file_size: 0 2024-03-28T17:54:07.489 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:54:07 smithi022 ceph-mon[31985]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-03-28T17:54:07.489 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:54:07 smithi022 ceph-mon[31985]: rocksdb: Options.log_file_time_to_roll: 0 2024-03-28T17:54:07.490 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:54:07 smithi022 ceph-mon[31985]: rocksdb: Options.keep_log_file_num: 1000 2024-03-28T17:54:07.490 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:54:07 smithi022 ceph-mon[31985]: rocksdb: Options.recycle_log_file_num: 0 2024-03-28T17:54:07.490 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:54:07 smithi022 ceph-mon[31985]: rocksdb: Options.allow_fallocate: 1 2024-03-28T17:54:07.490 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:54:07 smithi022 ceph-mon[31985]: rocksdb: Options.allow_mmap_reads: 0 2024-03-28T17:54:07.490 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:54:07 smithi022 ceph-mon[31985]: rocksdb: Options.allow_mmap_writes: 0 2024-03-28T17:54:07.490 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:54:07 smithi022 ceph-mon[31985]: rocksdb: Options.use_direct_reads: 0 2024-03-28T17:54:07.490 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:54:07 smithi022 ceph-mon[31985]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-03-28T17:54:07.490 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:54:07 smithi022 ceph-mon[31985]: rocksdb: Options.create_missing_column_families: 0 2024-03-28T17:54:07.490 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:54:07 smithi022 ceph-mon[31985]: rocksdb: Options.db_log_dir: 2024-03-28T17:54:07.491 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:54:07 smithi022 ceph-mon[31985]: rocksdb: Options.wal_dir: 2024-03-28T17:54:07.491 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:54:07 smithi022 ceph-mon[31985]: rocksdb: Options.table_cache_numshardbits: 6 2024-03-28T17:54:07.491 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:54:07 smithi022 ceph-mon[31985]: rocksdb: Options.WAL_ttl_seconds: 0 2024-03-28T17:54:07.491 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:54:07 smithi022 ceph-mon[31985]: rocksdb: Options.WAL_size_limit_MB: 0 2024-03-28T17:54:07.491 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:54:07 smithi022 ceph-mon[31985]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-03-28T17:54:07.491 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:54:07 smithi022 ceph-mon[31985]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-03-28T17:54:07.491 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:54:07 smithi022 ceph-mon[31985]: rocksdb: Options.is_fd_close_on_exec: 1 2024-03-28T17:54:07.491 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:54:07 smithi022 ceph-mon[31985]: rocksdb: Options.advise_random_on_open: 1 2024-03-28T17:54:07.491 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:54:07 smithi022 ceph-mon[31985]: rocksdb: Options.db_write_buffer_size: 0 2024-03-28T17:54:07.491 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:54:07 smithi022 ceph-mon[31985]: rocksdb: Options.write_buffer_manager: 0x555b4930c1e0 2024-03-28T17:54:07.491 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:54:07 smithi022 ceph-mon[31985]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-03-28T17:54:07.491 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:54:07 smithi022 ceph-mon[31985]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-03-28T17:54:07.491 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:54:07 smithi022 ceph-mon[31985]: rocksdb: Options.use_adaptive_mutex: 0 2024-03-28T17:54:07.491 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:54:07 smithi022 ceph-mon[31985]: rocksdb: Options.rate_limiter: (nil) 2024-03-28T17:54:07.491 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:54:07 smithi022 ceph-mon[31985]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-03-28T17:54:07.492 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:54:07 smithi022 ceph-mon[31985]: rocksdb: Options.wal_recovery_mode: 2 2024-03-28T17:54:07.492 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:54:07 smithi022 ceph-mon[31985]: rocksdb: Options.enable_thread_tracking: 0 2024-03-28T17:54:07.492 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:54:07 smithi022 ceph-mon[31985]: rocksdb: Options.enable_pipelined_write: 0 2024-03-28T17:54:07.492 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:54:07 smithi022 ceph-mon[31985]: rocksdb: Options.unordered_write: 0 2024-03-28T17:54:07.492 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:54:07 smithi022 ceph-mon[31985]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-03-28T17:54:07.492 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:54:07 smithi022 ceph-mon[31985]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-03-28T17:54:07.492 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:54:07 smithi022 ceph-mon[31985]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-03-28T17:54:07.492 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:54:07 smithi022 ceph-mon[31985]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-03-28T17:54:07.492 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:54:07 smithi022 ceph-mon[31985]: rocksdb: Options.row_cache: None 2024-03-28T17:54:07.492 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:54:07 smithi022 ceph-mon[31985]: rocksdb: Options.wal_filter: None 2024-03-28T17:54:07.492 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:54:07 smithi022 ceph-mon[31985]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-03-28T17:54:07.492 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:54:07 smithi022 ceph-mon[31985]: rocksdb: Options.allow_ingest_behind: 0 2024-03-28T17:54:07.492 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:54:07 smithi022 ceph-mon[31985]: rocksdb: Options.two_write_queues: 0 2024-03-28T17:54:07.492 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:54:07 smithi022 ceph-mon[31985]: rocksdb: Options.manual_wal_flush: 0 2024-03-28T17:54:07.492 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:54:07 smithi022 ceph-mon[31985]: rocksdb: Options.wal_compression: 0 2024-03-28T17:54:07.493 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:54:07 smithi022 ceph-mon[31985]: rocksdb: Options.atomic_flush: 0 2024-03-28T17:54:07.493 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:54:07 smithi022 ceph-mon[31985]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-03-28T17:54:07.493 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:54:07 smithi022 ceph-mon[31985]: rocksdb: Options.persist_stats_to_disk: 0 2024-03-28T17:54:07.493 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:54:07 smithi022 ceph-mon[31985]: rocksdb: Options.write_dbid_to_manifest: 0 2024-03-28T17:54:07.493 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:54:07 smithi022 ceph-mon[31985]: rocksdb: Options.log_readahead_size: 0 2024-03-28T17:54:07.494 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:54:07 smithi022 ceph-mon[31985]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-03-28T17:54:07.494 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:54:07 smithi022 ceph-mon[31985]: rocksdb: Options.best_efforts_recovery: 0 2024-03-28T17:54:07.494 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:54:07 smithi022 ceph-mon[31985]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-03-28T17:54:07.494 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:54:07 smithi022 ceph-mon[31985]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-03-28T17:54:07.494 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:54:07 smithi022 ceph-mon[31985]: rocksdb: Options.allow_data_in_errors: 0 2024-03-28T17:54:07.494 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:54:07 smithi022 ceph-mon[31985]: rocksdb: Options.db_host_id: __hostname__ 2024-03-28T17:54:07.494 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:54:07 smithi022 ceph-mon[31985]: rocksdb: Options.enforce_single_del_contracts: true 2024-03-28T17:54:07.494 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:54:07 smithi022 ceph-mon[31985]: rocksdb: Options.max_background_jobs: 2 2024-03-28T17:54:07.494 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:54:07 smithi022 ceph-mon[31985]: rocksdb: Options.max_background_compactions: -1 2024-03-28T17:54:07.494 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:54:07 smithi022 ceph-mon[31985]: rocksdb: Options.max_subcompactions: 1 2024-03-28T17:54:07.494 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:54:07 smithi022 ceph-mon[31985]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-03-28T17:54:07.494 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:54:07 smithi022 ceph-mon[31985]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-03-28T17:54:07.494 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:54:07 smithi022 ceph-mon[31985]: rocksdb: Options.delayed_write_rate : 16777216 2024-03-28T17:54:07.494 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:54:07 smithi022 ceph-mon[31985]: rocksdb: Options.max_total_wal_size: 0 2024-03-28T17:54:07.495 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:54:07 smithi022 ceph-mon[31985]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-03-28T17:54:07.495 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:54:07 smithi022 ceph-mon[31985]: rocksdb: Options.stats_dump_period_sec: 600 2024-03-28T17:54:07.495 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:54:07 smithi022 ceph-mon[31985]: rocksdb: Options.stats_persist_period_sec: 600 2024-03-28T17:54:07.495 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:54:07 smithi022 ceph-mon[31985]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-03-28T17:54:07.495 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:54:07 smithi022 ceph-mon[31985]: rocksdb: Options.max_open_files: -1 2024-03-28T17:54:07.495 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:54:07 smithi022 ceph-mon[31985]: rocksdb: Options.bytes_per_sync: 0 2024-03-28T17:54:07.495 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:54:07 smithi022 ceph-mon[31985]: rocksdb: Options.wal_bytes_per_sync: 0 2024-03-28T17:54:07.495 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:54:07 smithi022 ceph-mon[31985]: rocksdb: Options.strict_bytes_per_sync: 0 2024-03-28T17:54:07.495 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:54:07 smithi022 ceph-mon[31985]: rocksdb: Options.compaction_readahead_size: 0 2024-03-28T17:54:07.495 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:54:07 smithi022 ceph-mon[31985]: rocksdb: Options.max_background_flushes: -1 2024-03-28T17:54:07.495 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:54:07 smithi022 ceph-mon[31985]: rocksdb: Compression algorithms supported: 2024-03-28T17:54:07.495 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:54:07 smithi022 ceph-mon[31985]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-03-28T17:54:07.495 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:54:07 smithi022 ceph-mon[31985]: rocksdb: kZSTD supported: 0 2024-03-28T17:54:07.495 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:54:07 smithi022 ceph-mon[31985]: rocksdb: kXpressCompression supported: 0 2024-03-28T17:54:07.495 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:54:07 smithi022 ceph-mon[31985]: rocksdb: kLZ4HCCompression supported: 1 2024-03-28T17:54:07.496 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:54:07 smithi022 ceph-mon[31985]: rocksdb: kZlibCompression supported: 1 2024-03-28T17:54:07.496 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:54:07 smithi022 ceph-mon[31985]: rocksdb: kSnappyCompression supported: 1 2024-03-28T17:54:07.496 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:54:07 smithi022 ceph-mon[31985]: rocksdb: kLZ4Compression supported: 1 2024-03-28T17:54:07.496 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:54:07 smithi022 ceph-mon[31985]: rocksdb: kBZip2Compression supported: 0 2024-03-28T17:54:07.496 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:54:07 smithi022 ceph-mon[31985]: rocksdb: Fast CRC32 supported: Supported on x86 2024-03-28T17:54:07.496 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:54:07 smithi022 ceph-mon[31985]: rocksdb: DMutex implementation: pthread_mutex_t 2024-03-28T17:54:07.496 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:54:07 smithi022 ceph-mon[31985]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 2024-03-28T17:54:07.496 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:54:07 smithi022 ceph-mon[31985]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-03-28T17:54:07.496 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:54:07 smithi022 ceph-mon[31985]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-03-28T17:54:07.496 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:54:07 smithi022 ceph-mon[31985]: rocksdb: Options.merge_operator: 2024-03-28T17:54:07.496 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:54:07 smithi022 ceph-mon[31985]: rocksdb: Options.compaction_filter: None 2024-03-28T17:54:07.496 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:54:07 smithi022 ceph-mon[31985]: rocksdb: Options.compaction_filter_factory: None 2024-03-28T17:54:07.496 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:54:07 smithi022 ceph-mon[31985]: rocksdb: Options.sst_partitioner_factory: None 2024-03-28T17:54:07.496 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:54:07 smithi022 ceph-mon[31985]: rocksdb: Options.memtable_factory: SkipListFactory 2024-03-28T17:54:07.496 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:54:07 smithi022 ceph-mon[31985]: rocksdb: Options.table_factory: BlockBasedTable 2024-03-28T17:54:07.497 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:54:07 smithi022 ceph-mon[31985]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x555b49339a60) 2024-03-28T17:54:07.497 INFO:journalctl@ceph.mon.a.smithi022.stdout: cache_index_and_filter_blocks: 1 2024-03-28T17:54:07.497 INFO:journalctl@ceph.mon.a.smithi022.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-03-28T17:54:07.497 INFO:journalctl@ceph.mon.a.smithi022.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-03-28T17:54:07.497 INFO:journalctl@ceph.mon.a.smithi022.stdout: pin_top_level_index_and_filter: 1 2024-03-28T17:54:07.497 INFO:journalctl@ceph.mon.a.smithi022.stdout: index_type: 0 2024-03-28T17:54:07.497 INFO:journalctl@ceph.mon.a.smithi022.stdout: data_block_index_type: 0 2024-03-28T17:54:07.497 INFO:journalctl@ceph.mon.a.smithi022.stdout: index_shortening: 1 2024-03-28T17:54:07.497 INFO:journalctl@ceph.mon.a.smithi022.stdout: data_block_hash_table_util_ratio: 0.750000 2024-03-28T17:54:07.497 INFO:journalctl@ceph.mon.a.smithi022.stdout: checksum: 4 2024-03-28T17:54:07.497 INFO:journalctl@ceph.mon.a.smithi022.stdout: no_block_cache: 0 2024-03-28T17:54:07.497 INFO:journalctl@ceph.mon.a.smithi022.stdout: block_cache: 0x555b4933f090 2024-03-28T17:54:07.497 INFO:journalctl@ceph.mon.a.smithi022.stdout: block_cache_name: BinnedLRUCache 2024-03-28T17:54:07.497 INFO:journalctl@ceph.mon.a.smithi022.stdout: block_cache_options: 2024-03-28T17:54:07.498 INFO:journalctl@ceph.mon.a.smithi022.stdout: capacity : 536870912 2024-03-28T17:54:07.498 INFO:journalctl@ceph.mon.a.smithi022.stdout: num_shard_bits : 4 2024-03-28T17:54:07.498 INFO:journalctl@ceph.mon.a.smithi022.stdout: strict_capacity_limit : 0 2024-03-28T17:54:07.498 INFO:journalctl@ceph.mon.a.smithi022.stdout: high_pri_pool_ratio: 0.000 2024-03-28T17:54:07.498 INFO:journalctl@ceph.mon.a.smithi022.stdout: block_cache_compressed: (nil) 2024-03-28T17:54:07.498 INFO:journalctl@ceph.mon.a.smithi022.stdout: persistent_cache: (nil) 2024-03-28T17:54:07.498 INFO:journalctl@ceph.mon.a.smithi022.stdout: block_size: 4096 2024-03-28T17:54:07.498 INFO:journalctl@ceph.mon.a.smithi022.stdout: block_size_deviation: 10 2024-03-28T17:54:07.498 INFO:journalctl@ceph.mon.a.smithi022.stdout: block_restart_interval: 16 2024-03-28T17:54:07.498 INFO:journalctl@ceph.mon.a.smithi022.stdout: index_block_restart_interval: 1 2024-03-28T17:54:07.498 INFO:journalctl@ceph.mon.a.smithi022.stdout: metadata_block_size: 4096 2024-03-28T17:54:07.498 INFO:journalctl@ceph.mon.a.smithi022.stdout: partition_filters: 0 2024-03-28T17:54:07.498 INFO:journalctl@ceph.mon.a.smithi022.stdout: use_delta_encoding: 1 2024-03-28T17:54:07.498 INFO:journalctl@ceph.mon.a.smithi022.stdout: filter_policy: bloomfilter 2024-03-28T17:54:07.499 INFO:journalctl@ceph.mon.a.smithi022.stdout: whole_key_filtering: 1 2024-03-28T17:54:07.499 INFO:journalctl@ceph.mon.a.smithi022.stdout: verify_compression: 0 2024-03-28T17:54:07.499 INFO:journalctl@ceph.mon.a.smithi022.stdout: read_amp_bytes_per_bit: 0 2024-03-28T17:54:07.499 INFO:journalctl@ceph.mon.a.smithi022.stdout: format_version: 5 2024-03-28T17:54:07.499 INFO:journalctl@ceph.mon.a.smithi022.stdout: enable_index_compression: 1 2024-03-28T17:54:07.499 INFO:journalctl@ceph.mon.a.smithi022.stdout: block_align: 0 2024-03-28T17:54:07.499 INFO:journalctl@ceph.mon.a.smithi022.stdout: max_auto_readahead_size: 262144 2024-03-28T17:54:07.499 INFO:journalctl@ceph.mon.a.smithi022.stdout: prepopulate_block_cache: 0 2024-03-28T17:54:07.499 INFO:journalctl@ceph.mon.a.smithi022.stdout: initial_auto_readahead_size: 8192 2024-03-28T17:54:07.499 INFO:journalctl@ceph.mon.a.smithi022.stdout: num_file_reads_for_auto_readahead: 2 2024-03-28T17:54:07.499 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:54:07 smithi022 ceph-mon[31985]: rocksdb: Options.write_buffer_size: 33554432 2024-03-28T17:54:07.499 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:54:07 smithi022 ceph-mon[31985]: rocksdb: Options.max_write_buffer_number: 2 2024-03-28T17:54:07.499 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:54:07 smithi022 ceph-mon[31985]: rocksdb: Options.compression: NoCompression 2024-03-28T17:54:07.499 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:54:07 smithi022 ceph-mon[31985]: rocksdb: Options.bottommost_compression: Disabled 2024-03-28T17:54:07.499 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:54:07 smithi022 ceph-mon[31985]: rocksdb: Options.prefix_extractor: nullptr 2024-03-28T17:54:07.499 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:54:07 smithi022 ceph-mon[31985]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-03-28T17:54:07.500 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:54:07 smithi022 ceph-mon[31985]: rocksdb: Options.num_levels: 7 2024-03-28T17:54:07.500 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:54:07 smithi022 ceph-mon[31985]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-03-28T17:54:07.500 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:54:07 smithi022 ceph-mon[31985]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-03-28T17:54:07.500 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:54:07 smithi022 ceph-mon[31985]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-03-28T17:54:07.500 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:54:07 smithi022 ceph-mon[31985]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-03-28T17:54:07.500 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:54:07 smithi022 ceph-mon[31985]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-03-28T17:54:07.500 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:54:07 smithi022 ceph-mon[31985]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-03-28T17:54:07.500 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:54:07 smithi022 ceph-mon[31985]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-03-28T17:54:07.500 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:54:07 smithi022 ceph-mon[31985]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-03-28T17:54:07.500 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:54:07 smithi022 ceph-mon[31985]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-03-28T17:54:07.500 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:54:07 smithi022 ceph-mon[31985]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-03-28T17:54:07.500 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:54:07 smithi022 ceph-mon[31985]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-03-28T17:54:07.500 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:54:07 smithi022 ceph-mon[31985]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-03-28T17:54:07.500 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:54:07 smithi022 ceph-mon[31985]: rocksdb: Options.compression_opts.window_bits: -14 2024-03-28T17:54:07.500 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:54:07 smithi022 ceph-mon[31985]: rocksdb: Options.compression_opts.level: 32767 2024-03-28T17:54:07.501 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:54:07 smithi022 ceph-mon[31985]: rocksdb: Options.compression_opts.strategy: 0 2024-03-28T17:54:07.501 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:54:07 smithi022 ceph-mon[31985]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-03-28T17:54:07.501 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:54:07 smithi022 ceph-mon[31985]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-03-28T17:54:07.501 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:54:07 smithi022 ceph-mon[31985]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-03-28T17:54:07.501 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:54:07 smithi022 ceph-mon[31985]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-03-28T17:54:07.501 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:54:07 smithi022 ceph-mon[31985]: rocksdb: Options.compression_opts.enabled: false 2024-03-28T17:54:07.501 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:54:07 smithi022 ceph-mon[31985]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-03-28T17:54:07.501 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:54:07 smithi022 ceph-mon[31985]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-03-28T17:54:07.501 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:54:07 smithi022 ceph-mon[31985]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-03-28T17:54:07.501 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:54:07 smithi022 ceph-mon[31985]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-03-28T17:54:07.501 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:54:07 smithi022 ceph-mon[31985]: rocksdb: Options.target_file_size_base: 67108864 2024-03-28T17:54:07.501 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:54:07 smithi022 ceph-mon[31985]: rocksdb: Options.target_file_size_multiplier: 1 2024-03-28T17:54:07.501 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:54:07 smithi022 ceph-mon[31985]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-03-28T17:54:07.501 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:54:07 smithi022 ceph-mon[31985]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-03-28T17:54:07.502 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:54:07 smithi022 ceph-mon[31985]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-03-28T17:54:07.502 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:54:07 smithi022 ceph-mon[31985]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-03-28T17:54:07.502 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:54:07 smithi022 ceph-mon[31985]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-03-28T17:54:07.502 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:54:07 smithi022 ceph-mon[31985]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-03-28T17:54:07.502 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:54:07 smithi022 ceph-mon[31985]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-03-28T17:54:07.502 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:54:07 smithi022 ceph-mon[31985]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-03-28T17:54:07.502 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:54:07 smithi022 ceph-mon[31985]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-03-28T17:54:07.502 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:54:07 smithi022 ceph-mon[31985]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-03-28T17:54:07.502 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:54:07 smithi022 ceph-mon[31985]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-03-28T17:54:07.502 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:54:07 smithi022 ceph-mon[31985]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-03-28T17:54:07.502 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:54:07 smithi022 ceph-mon[31985]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-03-28T17:54:07.503 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:54:07 smithi022 ceph-mon[31985]: rocksdb: Options.arena_block_size: 1048576 2024-03-28T17:54:07.503 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:54:07 smithi022 ceph-mon[31985]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-03-28T17:54:07.503 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:54:07 smithi022 ceph-mon[31985]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-03-28T17:54:07.503 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:54:07 smithi022 ceph-mon[31985]: rocksdb: Options.disable_auto_compactions: 0 2024-03-28T17:54:07.503 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:54:07 smithi022 ceph-mon[31985]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-03-28T17:54:07.503 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:54:07 smithi022 ceph-mon[31985]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-03-28T17:54:07.503 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:54:07 smithi022 ceph-mon[31985]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-03-28T17:54:07.504 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:54:07 smithi022 ceph-mon[31985]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-03-28T17:54:07.504 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:54:07 smithi022 ceph-mon[31985]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-03-28T17:54:07.504 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:54:07 smithi022 ceph-mon[31985]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-03-28T17:54:07.504 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:54:07 smithi022 ceph-mon[31985]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-03-28T17:54:07.504 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:54:07 smithi022 ceph-mon[31985]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-03-28T17:54:07.504 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:54:07 smithi022 ceph-mon[31985]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-03-28T17:54:07.504 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:54:07 smithi022 ceph-mon[31985]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-03-28T17:54:07.504 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:54:07 smithi022 ceph-mon[31985]: rocksdb: Options.table_properties_collectors: 2024-03-28T17:54:07.504 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:54:07 smithi022 ceph-mon[31985]: rocksdb: Options.inplace_update_support: 0 2024-03-28T17:54:07.504 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:54:07 smithi022 ceph-mon[31985]: rocksdb: Options.inplace_update_num_locks: 10000 2024-03-28T17:54:07.504 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:54:07 smithi022 ceph-mon[31985]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-03-28T17:54:07.504 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:54:07 smithi022 ceph-mon[31985]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-03-28T17:54:07.504 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:54:07 smithi022 ceph-mon[31985]: rocksdb: Options.memtable_huge_page_size: 0 2024-03-28T17:54:07.504 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:54:07 smithi022 ceph-mon[31985]: rocksdb: Options.bloom_locality: 0 2024-03-28T17:54:07.505 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:54:07 smithi022 ceph-mon[31985]: rocksdb: Options.max_successive_merges: 0 2024-03-28T17:54:07.505 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:54:07 smithi022 ceph-mon[31985]: rocksdb: Options.optimize_filters_for_hits: 0 2024-03-28T17:54:07.505 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:54:07 smithi022 ceph-mon[31985]: rocksdb: Options.paranoid_file_checks: 0 2024-03-28T17:54:07.505 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:54:07 smithi022 ceph-mon[31985]: rocksdb: Options.force_consistency_checks: 1 2024-03-28T17:54:07.505 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:54:07 smithi022 ceph-mon[31985]: rocksdb: Options.report_bg_io_stats: 0 2024-03-28T17:54:07.505 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:54:07 smithi022 ceph-mon[31985]: rocksdb: Options.ttl: 2592000 2024-03-28T17:54:07.505 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:54:07 smithi022 ceph-mon[31985]: rocksdb: Options.periodic_compaction_seconds: 0 2024-03-28T17:54:07.505 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:54:07 smithi022 ceph-mon[31985]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-03-28T17:54:07.505 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:54:07 smithi022 ceph-mon[31985]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-03-28T17:54:07.505 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:54:07 smithi022 ceph-mon[31985]: rocksdb: Options.enable_blob_files: false 2024-03-28T17:54:07.505 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:54:07 smithi022 ceph-mon[31985]: rocksdb: Options.min_blob_size: 0 2024-03-28T17:54:07.505 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:54:07 smithi022 ceph-mon[31985]: rocksdb: Options.blob_file_size: 268435456 2024-03-28T17:54:07.505 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:54:07 smithi022 ceph-mon[31985]: rocksdb: Options.blob_compression_type: NoCompression 2024-03-28T17:54:07.505 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:54:07 smithi022 ceph-mon[31985]: rocksdb: Options.enable_blob_garbage_collection: false 2024-03-28T17:54:07.505 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:54:07 smithi022 ceph-mon[31985]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-03-28T17:54:07.506 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:54:07 smithi022 ceph-mon[31985]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-03-28T17:54:07.506 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:54:07 smithi022 ceph-mon[31985]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-03-28T17:54:07.506 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:54:07 smithi022 ceph-mon[31985]: rocksdb: Options.blob_file_starting_level: 0 2024-03-28T17:54:07.506 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:54:07 smithi022 ceph-mon[31985]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-03-28T17:54:07.506 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:54:07 smithi022 ceph-mon[31985]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 succeeded,manifest_file_number is 10, next_file_number is 12, last_sequence is 5, log_number is 5,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 5 2024-03-28T17:54:07.506 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:54:07 smithi022 ceph-mon[31985]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 5 2024-03-28T17:54:07.506 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:54:07 smithi022 ceph-mon[31985]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: 178e3565-951a-4492-8e85-da9fda5410cf 2024-03-28T17:54:07.506 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:54:07 smithi022 ceph-mon[31985]: rocksdb: EVENT_LOG_v1 {"time_micros": 1711648447200637, "job": 1, "event": "recovery_started", "wal_files": [9]} 2024-03-28T17:54:07.506 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:54:07 smithi022 ceph-mon[31985]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #9 mode 2 2024-03-28T17:54:07.506 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:54:07 smithi022 ceph-mon[31985]: rocksdb: EVENT_LOG_v1 {"time_micros": 1711648447202702, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 13, "file_size": 95571, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 8, "largest_seqno": 318, "table_properties": {"data_size": 93564, "index_size": 289, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 709, "raw_key_size": 13866, "raw_average_key_size": 50, "raw_value_size": 85993, "raw_average_value_size": 310, "num_data_blocks": 11, "num_entries": 277, "num_filter_entries": 277, "num_deletions": 3, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1711648447, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "178e3565-951a-4492-8e85-da9fda5410cf", "db_session_id": "08AM0AGZ8YZCP0FWOO3X", "orig_file_number": 13, "seqno_to_time_mapping": "N/A"}} 2024-03-28T17:54:07.506 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:54:07 smithi022 ceph-mon[31985]: rocksdb: EVENT_LOG_v1 {"time_micros": 1711648447202889, "job": 1, "event": "recovery_finished"} 2024-03-28T17:54:07.506 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:54:07 smithi022 ceph-mon[31985]: rocksdb: [db/version_set.cc:5047] Creating manifest 15 2024-03-28T17:54:07.506 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:54:07 smithi022 ceph-mon[31985]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-a/store.db/000009.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-03-28T17:54:07.506 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:54:07 smithi022 ceph-mon[31985]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x555b49432000 2024-03-28T17:54:07.506 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:54:07 smithi022 ceph-mon[31985]: rocksdb: DB pointer 0x555b4941c000 2024-03-28T17:54:07.507 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:54:07 smithi022 ceph-mon[31985]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-03-28T17:54:07.507 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:54:07 smithi022 ceph-mon[31985]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-03-28T17:54:07.507 INFO:journalctl@ceph.mon.a.smithi022.stdout: ** DB Stats ** 2024-03-28T17:54:07.507 INFO:journalctl@ceph.mon.a.smithi022.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-03-28T17:54:07.507 INFO:journalctl@ceph.mon.a.smithi022.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-03-28T17:54:07.507 INFO:journalctl@ceph.mon.a.smithi022.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-03-28T17:54:07.507 INFO:journalctl@ceph.mon.a.smithi022.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-03-28T17:54:07.507 INFO:journalctl@ceph.mon.a.smithi022.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-03-28T17:54:07.507 INFO:journalctl@ceph.mon.a.smithi022.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-03-28T17:54:07.507 INFO:journalctl@ceph.mon.a.smithi022.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-03-28T17:54:07.507 INFO:journalctl@ceph.mon.a.smithi022.stdout: 2024-03-28T17:54:07.507 INFO:journalctl@ceph.mon.a.smithi022.stdout: ** Compaction Stats [default] ** 2024-03-28T17:54:07.507 INFO:journalctl@ceph.mon.a.smithi022.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-03-28T17:54:07.507 INFO:journalctl@ceph.mon.a.smithi022.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-03-28T17:54:07.508 INFO:journalctl@ceph.mon.a.smithi022.stdout: L0 2/0 95.17 KB 0.5 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 67.3 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-03-28T17:54:07.508 INFO:journalctl@ceph.mon.a.smithi022.stdout: Sum 2/0 95.17 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 67.3 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-03-28T17:54:07.509 INFO:journalctl@ceph.mon.a.smithi022.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 67.3 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-03-28T17:54:07.509 INFO:journalctl@ceph.mon.a.smithi022.stdout: 2024-03-28T17:54:07.509 INFO:journalctl@ceph.mon.a.smithi022.stdout: ** Compaction Stats [default] ** 2024-03-28T17:54:07.509 INFO:journalctl@ceph.mon.a.smithi022.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-03-28T17:54:07.509 INFO:journalctl@ceph.mon.a.smithi022.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-03-28T17:54:07.509 INFO:journalctl@ceph.mon.a.smithi022.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 67.3 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-03-28T17:54:07.509 INFO:journalctl@ceph.mon.a.smithi022.stdout: 2024-03-28T17:54:07.510 INFO:journalctl@ceph.mon.a.smithi022.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-03-28T17:54:07.510 INFO:journalctl@ceph.mon.a.smithi022.stdout: 2024-03-28T17:54:07.510 INFO:journalctl@ceph.mon.a.smithi022.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-03-28T17:54:07.510 INFO:journalctl@ceph.mon.a.smithi022.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-03-28T17:54:07.510 INFO:journalctl@ceph.mon.a.smithi022.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-03-28T17:54:07.510 INFO:journalctl@ceph.mon.a.smithi022.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-03-28T17:54:07.510 INFO:journalctl@ceph.mon.a.smithi022.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-03-28T17:54:07.510 INFO:journalctl@ceph.mon.a.smithi022.stdout: AddFile(Keys): cumulative 0, interval 0 2024-03-28T17:54:07.510 INFO:journalctl@ceph.mon.a.smithi022.stdout: Cumulative compaction: 0.00 GB write, 12.96 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-03-28T17:54:07.510 INFO:journalctl@ceph.mon.a.smithi022.stdout: Interval compaction: 0.00 GB write, 12.96 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-03-28T17:54:07.510 INFO:journalctl@ceph.mon.a.smithi022.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-03-28T17:54:07.511 INFO:journalctl@ceph.mon.a.smithi022.stdout: Block cache BinnedLRUCache@0x555b4933f090#2 capacity: 512.00 MB usage: 1.39 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 1.3e-05 secs_since: 0 2024-03-28T17:54:07.511 INFO:journalctl@ceph.mon.a.smithi022.stdout: Block cache entry stats(count,size,portion): FilterBlock(2,0.89 KB,0.000169873%) IndexBlock(2,0.50 KB,9.53674e-05%) Misc(1,0.00 KB,0%) 2024-03-28T17:54:07.511 INFO:journalctl@ceph.mon.a.smithi022.stdout: 2024-03-28T17:54:07.511 INFO:journalctl@ceph.mon.a.smithi022.stdout: ** File Read Latency Histogram By Level [default] ** 2024-03-28T17:54:07.511 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:54:07 smithi022 ceph-mon[31985]: starting mon.a rank 0 at public addrs [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] at bind addrs [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] mon_data /var/lib/ceph/mon/ceph-a fsid fd220748-ed2b-11ee-95d0-87774f69a715 2024-03-28T17:54:07.511 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:54:07 smithi022 ceph-mon[31985]: mon.a@-1(???) e1 preinit fsid fd220748-ed2b-11ee-95d0-87774f69a715 2024-03-28T17:54:07.511 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:54:07 smithi022 ceph-mon[31985]: mon.a@-1(???).mds e1 new map 2024-03-28T17:54:07.511 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:54:07 smithi022 ceph-mon[31985]: mon.a@-1(???).mds e1 print_map 2024-03-28T17:54:07.511 INFO:journalctl@ceph.mon.a.smithi022.stdout: e1 2024-03-28T17:54:07.511 INFO:journalctl@ceph.mon.a.smithi022.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-03-28T17:54:07.511 INFO:journalctl@ceph.mon.a.smithi022.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments} 2024-03-28T17:54:07.511 INFO:journalctl@ceph.mon.a.smithi022.stdout: legacy client fscid: -1 2024-03-28T17:54:07.511 INFO:journalctl@ceph.mon.a.smithi022.stdout: 2024-03-28T17:54:07.511 INFO:journalctl@ceph.mon.a.smithi022.stdout: No filesystems configured 2024-03-28T17:54:07.512 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:54:07 smithi022 ceph-mon[31985]: mon.a@-1(???).osd e1 crush map has features 3314932999778484224, adjusting msgr requires 2024-03-28T17:54:07.512 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:54:07 smithi022 ceph-mon[31985]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-03-28T17:54:07.512 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:54:07 smithi022 ceph-mon[31985]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-03-28T17:54:07.512 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:54:07 smithi022 ceph-mon[31985]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-03-28T17:54:07.512 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:54:07 smithi022 ceph-mon[31985]: mon.a@-1(???).paxosservice(auth 1..2) refresh upgraded, format 0 -> 3 2024-03-28T17:54:07.512 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:54:07 smithi022 bash[31967]: c8a13c8d55b2de981388bb3edb0bc18641bbb8c7b95a1814f61fdd39ed21d506 2024-03-28T17:54:07.512 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:54:07 smithi022 systemd[1]: Started Ceph mon.a for fd220748-ed2b-11ee-95d0-87774f69a715. 2024-03-28T17:54:07.512 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:54:07 smithi022 ceph-mon[31985]: mon.a is new leader, mons a in quorum (ranks 0) 2024-03-28T17:54:07.512 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:54:07 smithi022 ceph-mon[31985]: monmap e1: 1 mons at {a=[v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-03-28T17:54:07.512 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:54:07 smithi022 ceph-mon[31985]: fsmap 2024-03-28T17:54:07.512 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:54:07 smithi022 ceph-mon[31985]: osdmap e1: 0 total, 0 up, 0 in 2024-03-28T17:54:07.512 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:54:07 smithi022 ceph-mon[31985]: mgrmap e1: no daemons active 2024-03-28T17:54:08.568 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:07.820+0000 7f3b33200700 1 Processor -- start 2024-03-28T17:54:08.568 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:07.820+0000 7f3b33200700 1 -- start start 2024-03-28T17:54:08.568 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:07.820+0000 7f3b33200700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3b34109bc0 0x7f3b34109fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:54:08.568 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:07.820+0000 7f3b33200700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b3410a5b0 con 0x7f3b34109bc0 2024-03-28T17:54:08.569 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:07.821+0000 7f3b31e00700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3b34109bc0 0x7f3b34109fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:54:08.569 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:07.821+0000 7f3b31e00700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3b34109bc0 0x7f3b34109fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:35450/0 (socket says 172.21.15.22:35450) 2024-03-28T17:54:08.569 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:07.821+0000 7f3b31e00700 1 -- 172.21.15.22:0/3314636160 learned_addr learned my addr 172.21.15.22:0/3314636160 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-28T17:54:08.569 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:07.821+0000 7f3b31e00700 1 -- 172.21.15.22:0/3314636160 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3b3410ad70 con 0x7f3b34109bc0 2024-03-28T17:54:08.569 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:07.822+0000 7f3b31e00700 1 --2- 172.21.15.22:0/3314636160 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3b34109bc0 0x7f3b34109fe0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f3b240098a0 tx=0x7f3b24009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=83d09101547fc00a server_cookie=0 in_seq=0 out_seq=0 2024-03-28T17:54:08.569 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:07.822+0000 7f3b30a00700 1 -- 172.21.15.22:0/3314636160 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f3b24004160 con 0x7f3b34109bc0 2024-03-28T17:54:08.571 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:07.822+0000 7f3b30a00700 1 -- 172.21.15.22:0/3314636160 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3b24004300 con 0x7f3b34109bc0 2024-03-28T17:54:08.572 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:07.822+0000 7f3b30a00700 1 -- 172.21.15.22:0/3314636160 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f3b24011b40 con 0x7f3b34109bc0 2024-03-28T17:54:08.572 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:07.823+0000 7f3b33200700 1 -- 172.21.15.22:0/3314636160 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3b34109bc0 msgr2=0x7f3b34109fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:54:08.572 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:07.823+0000 7f3b33200700 1 --2- 172.21.15.22:0/3314636160 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3b34109bc0 0x7f3b34109fe0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f3b240098a0 tx=0x7f3b24009bb0 comp rx=0 tx=0).stop 2024-03-28T17:54:08.572 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:07.823+0000 7f3b33200700 1 -- 172.21.15.22:0/3314636160 shutdown_connections 2024-03-28T17:54:08.572 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:07.823+0000 7f3b33200700 1 --2- 172.21.15.22:0/3314636160 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3b34109bc0 0x7f3b34109fe0 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:54:08.572 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:07.823+0000 7f3b33200700 1 -- 172.21.15.22:0/3314636160 >> 172.21.15.22:0/3314636160 conn(0x7f3b3407cfe0 msgr2=0x7f3b3407d410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T17:54:08.573 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:07.823+0000 7f3b33200700 1 -- 172.21.15.22:0/3314636160 shutdown_connections 2024-03-28T17:54:08.573 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:07.823+0000 7f3b33200700 1 -- 172.21.15.22:0/3314636160 wait complete. 2024-03-28T17:54:08.573 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:07.825+0000 7f3b33200700 1 Processor -- start 2024-03-28T17:54:08.573 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:07.825+0000 7f3b33200700 1 -- start start 2024-03-28T17:54:08.573 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:07.825+0000 7f3b33200700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3b34109bc0 0x7f3b3419db90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:54:08.573 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:07.825+0000 7f3b33200700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b3410e120 con 0x7f3b34109bc0 2024-03-28T17:54:08.577 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:07.826+0000 7f3b31e00700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3b34109bc0 0x7f3b3419db90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:54:08.577 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:07.826+0000 7f3b31e00700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3b34109bc0 0x7f3b3419db90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:35454/0 (socket says 172.21.15.22:35454) 2024-03-28T17:54:08.577 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:07.826+0000 7f3b31e00700 1 -- 172.21.15.22:0/793047984 learned_addr learned my addr 172.21.15.22:0/793047984 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-28T17:54:08.577 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:07.826+0000 7f3b31e00700 1 -- 172.21.15.22:0/793047984 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3b3419e0d0 con 0x7f3b34109bc0 2024-03-28T17:54:08.577 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:07.827+0000 7f3b31e00700 1 --2- 172.21.15.22:0/793047984 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3b34109bc0 0x7f3b3419db90 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f3b24011840 tx=0x7f3b24004230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T17:54:08.577 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:07.827+0000 7f3b2aa00700 1 -- 172.21.15.22:0/793047984 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f3b240038b0 con 0x7f3b34109bc0 2024-03-28T17:54:08.577 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:07.827+0000 7f3b2aa00700 1 -- 172.21.15.22:0/793047984 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3b2401a430 con 0x7f3b34109bc0 2024-03-28T17:54:08.577 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:07.827+0000 7f3b33200700 1 -- 172.21.15.22:0/793047984 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3b3419e360 con 0x7f3b34109bc0 2024-03-28T17:54:08.577 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:07.827+0000 7f3b2aa00700 1 -- 172.21.15.22:0/793047984 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f3b24024650 con 0x7f3b34109bc0 2024-03-28T17:54:08.577 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:07.827+0000 7f3b33200700 1 -- 172.21.15.22:0/793047984 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3b341a1050 con 0x7f3b34109bc0 2024-03-28T17:54:08.578 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:07.828+0000 7f3b2aa00700 1 -- 172.21.15.22:0/793047984 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 1) v1 ==== 807+0+0 (secure 0 0 0) 0x7f3b24022020 con 0x7f3b34109bc0 2024-03-28T17:54:08.578 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:07.828+0000 7f3b2aa00700 1 -- 172.21.15.22:0/793047984 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f3b2402dbc0 con 0x7f3b34109bc0 2024-03-28T17:54:08.578 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:07.829+0000 7f3b33200700 1 -- 172.21.15.22:0/793047984 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3b340637c0 con 0x7f3b34109bc0 2024-03-28T17:54:08.578 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:07.832+0000 7f3b2aa00700 1 -- 172.21.15.22:0/793047984 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73141 (secure 0 0 0) 0x7f3b240042a0 con 0x7f3b34109bc0 2024-03-28T17:54:08.578 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:07.909+0000 7f3b33200700 1 -- 172.21.15.22:0/793047984 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command([{prefix=config set, name=public_network}] v 0) v1 -- 0x7f3b341a1220 con 0x7f3b34109bc0 2024-03-28T17:54:08.578 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:07.912+0000 7f3b2aa00700 1 -- 172.21.15.22:0/793047984 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{prefix=config set, name=public_network}]=0 v3)=0 v3) v1 ==== 124+0+0 (secure 0 0 0) 0x7f3b24032030 con 0x7f3b34109bc0 2024-03-28T17:54:08.578 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:07.913+0000 7f3b33200700 1 -- 172.21.15.22:0/793047984 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3b34109bc0 msgr2=0x7f3b3419db90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:54:08.578 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:07.913+0000 7f3b33200700 1 --2- 172.21.15.22:0/793047984 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3b34109bc0 0x7f3b3419db90 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f3b24011840 tx=0x7f3b24004230 comp rx=0 tx=0).stop 2024-03-28T17:54:08.578 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:07.913+0000 7f3b33200700 1 -- 172.21.15.22:0/793047984 shutdown_connections 2024-03-28T17:54:08.578 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:07.913+0000 7f3b33200700 1 --2- 172.21.15.22:0/793047984 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3b34109bc0 0x7f3b3419db90 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:54:08.578 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:07.913+0000 7f3b33200700 1 -- 172.21.15.22:0/793047984 >> 172.21.15.22:0/793047984 conn(0x7f3b3407cfe0 msgr2=0x7f3b34107590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T17:54:08.578 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:07.913+0000 7f3b33200700 1 -- 172.21.15.22:0/793047984 shutdown_connections 2024-03-28T17:54:08.578 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:07.914+0000 7f3b33200700 1 -- 172.21.15.22:0/793047984 wait complete. 2024-03-28T17:54:08.579 INFO:teuthology.orchestra.run.smithi022.stdout:Wrote config to /etc/ceph/ceph.conf 2024-03-28T17:54:08.603 INFO:teuthology.orchestra.run.smithi022.stdout:Wrote keyring to /etc/ceph/ceph.client.admin.keyring 2024-03-28T17:54:08.603 INFO:teuthology.orchestra.run.smithi022.stdout:Creating mgr... 2024-03-28T17:54:08.605 INFO:teuthology.orchestra.run.smithi022.stdout:Verifying port 0.0.0.0:9283 ... 2024-03-28T17:54:08.606 INFO:teuthology.orchestra.run.smithi022.stdout:Verifying port 0.0.0.0:8765 ... 2024-03-28T17:54:09.073 INFO:teuthology.orchestra.run.smithi022.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-fd220748-ed2b-11ee-95d0-87774f69a715@mgr.x 2024-03-28T17:54:09.073 INFO:teuthology.orchestra.run.smithi022.stdout:systemctl: stderr Failed to reset failed state of unit ceph-fd220748-ed2b-11ee-95d0-87774f69a715@mgr.x.service: Unit ceph-fd220748-ed2b-11ee-95d0-87774f69a715@mgr.x.service not loaded. 2024-03-28T17:54:09.282 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:54:08 smithi022 ceph-mon[31985]: from='client.? 172.21.15.22:0/793047984' entity='client.admin' 2024-03-28T17:54:09.351 INFO:teuthology.orchestra.run.smithi022.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-fd220748-ed2b-11ee-95d0-87774f69a715.target.wants/ceph-fd220748-ed2b-11ee-95d0-87774f69a715@mgr.x.service → /etc/systemd/system/ceph-fd220748-ed2b-11ee-95d0-87774f69a715@.service. 2024-03-28T17:54:09.759 INFO:journalctl@ceph.mgr.x.smithi022.stdout:Mar 28 17:54:09 smithi022 systemd[1]: Starting Ceph mgr.x for fd220748-ed2b-11ee-95d0-87774f69a715... 2024-03-28T17:54:10.033 INFO:journalctl@ceph.mgr.x.smithi022.stdout:Mar 28 17:54:09 smithi022 podman[32197]: 2024-03-28 17:54:09.66311102 +0000 UTC m=+0.034596408 image pull c664afe56bb01db4f9afa9532c20598e201e664f2e9ba2be50db0d52674a26d2 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033 2024-03-28T17:54:10.033 INFO:journalctl@ceph.mgr.x.smithi022.stdout:Mar 28 17:54:09 smithi022 podman[32197]: 2024-03-28 17:54:09.814495815 +0000 UTC m=+0.185981194 container create 6c8e21b01d7be33e34ecbbc00c5284acade8e440050796d246c496044cd73590 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033, name=ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mgr-x, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.build-date=20240319, org.label-schema.license=GPLv2, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD) 2024-03-28T17:54:10.155 INFO:teuthology.orchestra.run.smithi022.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-03-28T17:54:10.155 INFO:teuthology.orchestra.run.smithi022.stdout:systemctl: stdout disabled 2024-03-28T17:54:10.171 INFO:teuthology.orchestra.run.smithi022.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-03-28T17:54:10.172 INFO:teuthology.orchestra.run.smithi022.stdout:systemctl: stdout inactive 2024-03-28T17:54:10.172 INFO:teuthology.orchestra.run.smithi022.stdout:firewalld.service is not enabled 2024-03-28T17:54:10.172 INFO:teuthology.orchestra.run.smithi022.stdout:Not possible to enable service . firewalld.service is not available 2024-03-28T17:54:10.193 INFO:teuthology.orchestra.run.smithi022.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-03-28T17:54:10.193 INFO:teuthology.orchestra.run.smithi022.stdout:systemctl: stdout disabled 2024-03-28T17:54:10.211 INFO:teuthology.orchestra.run.smithi022.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-03-28T17:54:10.211 INFO:teuthology.orchestra.run.smithi022.stdout:systemctl: stdout inactive 2024-03-28T17:54:10.211 INFO:teuthology.orchestra.run.smithi022.stdout:firewalld.service is not enabled 2024-03-28T17:54:10.212 INFO:teuthology.orchestra.run.smithi022.stdout:Not possible to open ports <[9283, 8765]>. firewalld.service is not available 2024-03-28T17:54:10.212 INFO:teuthology.orchestra.run.smithi022.stdout:Waiting for mgr to start... 2024-03-28T17:54:10.212 INFO:teuthology.orchestra.run.smithi022.stdout:Waiting for mgr... 2024-03-28T17:54:10.363 INFO:journalctl@ceph.mgr.x.smithi022.stdout:Mar 28 17:54:10 smithi022 podman[32197]: 2024-03-28 17:54:10.042914596 +0000 UTC m=+0.414400047 container init 6c8e21b01d7be33e34ecbbc00c5284acade8e440050796d246c496044cd73590 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033, name=ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mgr-x, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.build-date=20240319, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.vendor=CentOS, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_CLEAN=True) 2024-03-28T17:54:10.363 INFO:journalctl@ceph.mgr.x.smithi022.stdout:Mar 28 17:54:10 smithi022 podman[32197]: 2024-03-28 17:54:10.052868773 +0000 UTC m=+0.424354186 container start 6c8e21b01d7be33e34ecbbc00c5284acade8e440050796d246c496044cd73590 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033, name=ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mgr-x, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.build-date=20240319, ceph=True, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, org.label-schema.schema-version=1.0) 2024-03-28T17:54:10.363 INFO:journalctl@ceph.mgr.x.smithi022.stdout:Mar 28 17:54:10 smithi022 bash[32197]: 6c8e21b01d7be33e34ecbbc00c5284acade8e440050796d246c496044cd73590 2024-03-28T17:54:10.363 INFO:journalctl@ceph.mgr.x.smithi022.stdout:Mar 28 17:54:10 smithi022 systemd[1]: Started Ceph mgr.x for fd220748-ed2b-11ee-95d0-87774f69a715. 2024-03-28T17:54:10.973 INFO:journalctl@ceph.mgr.x.smithi022.stdout:Mar 28 17:54:10 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mgr-x[32211]: 2024-03-28T17:54:10.839+0000 7fa72ebe2200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-03-28T17:54:10.973 INFO:journalctl@ceph.mgr.x.smithi022.stdout:Mar 28 17:54:10 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mgr-x[32211]: 2024-03-28T17:54:10.968+0000 7fa72ebe2200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-03-28T17:54:11.281 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:54:11 smithi022 ceph-mon[31985]: from='client.? 172.21.15.22:0/3027502273' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-03-28T17:54:11.747 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout 2024-03-28T17:54:11.748 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout { 2024-03-28T17:54:11.748 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "fsid": "fd220748-ed2b-11ee-95d0-87774f69a715", 2024-03-28T17:54:11.748 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "health": { 2024-03-28T17:54:11.748 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-03-28T17:54:11.748 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-03-28T17:54:11.748 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-03-28T17:54:11.748 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout }, 2024-03-28T17:54:11.748 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-03-28T17:54:11.748 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-03-28T17:54:11.748 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout 0 2024-03-28T17:54:11.748 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout ], 2024-03-28T17:54:11.748 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-03-28T17:54:11.748 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "a" 2024-03-28T17:54:11.749 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout ], 2024-03-28T17:54:11.749 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "quorum_age": 3, 2024-03-28T17:54:11.749 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "monmap": { 2024-03-28T17:54:11.749 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-28T17:54:11.749 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-03-28T17:54:11.749 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-03-28T17:54:11.749 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout }, 2024-03-28T17:54:11.749 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-03-28T17:54:11.749 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-28T17:54:11.749 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-03-28T17:54:11.749 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-03-28T17:54:11.749 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-03-28T17:54:11.749 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-03-28T17:54:11.749 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-03-28T17:54:11.749 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-03-28T17:54:11.750 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout }, 2024-03-28T17:54:11.750 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-03-28T17:54:11.750 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-03-28T17:54:11.750 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-03-28T17:54:11.750 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-03-28T17:54:11.750 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-03-28T17:54:11.750 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-03-28T17:54:11.750 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-03-28T17:54:11.750 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-03-28T17:54:11.750 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-03-28T17:54:11.750 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout }, 2024-03-28T17:54:11.750 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-03-28T17:54:11.750 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-28T17:54:11.750 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-03-28T17:54:11.750 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-03-28T17:54:11.750 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout }, 2024-03-28T17:54:11.751 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-03-28T17:54:11.751 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "available": false, 2024-03-28T17:54:11.751 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-03-28T17:54:11.751 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "modules": [ 2024-03-28T17:54:11.751 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "iostat", 2024-03-28T17:54:11.751 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "nfs", 2024-03-28T17:54:11.751 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "restful" 2024-03-28T17:54:11.751 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout ], 2024-03-28T17:54:11.751 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "services": {} 2024-03-28T17:54:11.751 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout }, 2024-03-28T17:54:11.751 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-03-28T17:54:11.751 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-28T17:54:11.751 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "modified": "2024-03-28T17:54:00.441111+0000", 2024-03-28T17:54:11.751 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "services": {} 2024-03-28T17:54:11.751 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout }, 2024-03-28T17:54:11.751 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-03-28T17:54:11.751 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout } 2024-03-28T17:54:11.752 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:10.881+0000 7fae50c00700 1 Processor -- start 2024-03-28T17:54:11.752 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:10.881+0000 7fae50c00700 1 -- start start 2024-03-28T17:54:11.752 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:10.882+0000 7fae50c00700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fae4c106480 0x7fae4c1068a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:54:11.752 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:10.882+0000 7fae50c00700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae4c0758f0 con 0x7fae4c106480 2024-03-28T17:54:11.752 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:10.882+0000 7fae49e00700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fae4c106480 0x7fae4c1068a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:54:11.752 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:10.882+0000 7fae49e00700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fae4c106480 0x7fae4c1068a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:35474/0 (socket says 172.21.15.22:35474) 2024-03-28T17:54:11.752 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:10.882+0000 7fae49e00700 1 -- 172.21.15.22:0/1517859031 learned_addr learned my addr 172.21.15.22:0/1517859031 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-28T17:54:11.752 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:10.883+0000 7fae49e00700 1 -- 172.21.15.22:0/1517859031 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae4c106de0 con 0x7fae4c106480 2024-03-28T17:54:11.752 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:10.883+0000 7fae49e00700 1 --2- 172.21.15.22:0/1517859031 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fae4c106480 0x7fae4c1068a0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7fae3c0098a0 tx=0x7fae3c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c160fedf887961f4 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T17:54:11.752 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:10.883+0000 7fae49400700 1 -- 172.21.15.22:0/1517859031 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fae3c004160 con 0x7fae4c106480 2024-03-28T17:54:11.752 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:10.883+0000 7fae49400700 1 -- 172.21.15.22:0/1517859031 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fae3c004300 con 0x7fae4c106480 2024-03-28T17:54:11.752 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:10.883+0000 7fae49400700 1 -- 172.21.15.22:0/1517859031 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fae3c011b40 con 0x7fae4c106480 2024-03-28T17:54:11.752 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:10.884+0000 7fae50c00700 1 -- 172.21.15.22:0/1517859031 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fae4c106480 msgr2=0x7fae4c1068a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:54:11.752 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:10.884+0000 7fae50c00700 1 --2- 172.21.15.22:0/1517859031 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fae4c106480 0x7fae4c1068a0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7fae3c0098a0 tx=0x7fae3c009bb0 comp rx=0 tx=0).stop 2024-03-28T17:54:11.752 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:10.884+0000 7fae50c00700 1 -- 172.21.15.22:0/1517859031 shutdown_connections 2024-03-28T17:54:11.753 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:10.884+0000 7fae50c00700 1 --2- 172.21.15.22:0/1517859031 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fae4c106480 0x7fae4c1068a0 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:54:11.753 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:10.884+0000 7fae50c00700 1 -- 172.21.15.22:0/1517859031 >> 172.21.15.22:0/1517859031 conn(0x7fae4c1020a0 msgr2=0x7fae4c1044c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T17:54:11.753 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:10.884+0000 7fae50c00700 1 -- 172.21.15.22:0/1517859031 shutdown_connections 2024-03-28T17:54:11.753 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:10.884+0000 7fae50c00700 1 -- 172.21.15.22:0/1517859031 wait complete. 2024-03-28T17:54:11.753 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:10.885+0000 7fae50c00700 1 Processor -- start 2024-03-28T17:54:11.753 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:10.886+0000 7fae50c00700 1 -- start start 2024-03-28T17:54:11.753 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:10.886+0000 7fae50c00700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fae4c106480 0x7fae4c19dd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:54:11.753 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:10.886+0000 7fae50c00700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae4c107e70 con 0x7fae4c106480 2024-03-28T17:54:11.753 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:10.886+0000 7fae49e00700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fae4c106480 0x7fae4c19dd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:54:11.753 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:10.886+0000 7fae49e00700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fae4c106480 0x7fae4c19dd90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:35488/0 (socket says 172.21.15.22:35488) 2024-03-28T17:54:11.753 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:10.886+0000 7fae49e00700 1 -- 172.21.15.22:0/3027502273 learned_addr learned my addr 172.21.15.22:0/3027502273 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-28T17:54:11.753 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:10.887+0000 7fae49e00700 1 -- 172.21.15.22:0/3027502273 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae4c19e2d0 con 0x7fae4c106480 2024-03-28T17:54:11.753 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:10.887+0000 7fae49e00700 1 --2- 172.21.15.22:0/3027502273 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fae4c106480 0x7fae4c19dd90 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fae3c004180 tx=0x7fae3c0046a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T17:54:11.753 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:10.887+0000 7fae42a00700 1 -- 172.21.15.22:0/3027502273 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fae3c0039e0 con 0x7fae4c106480 2024-03-28T17:54:11.753 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:10.887+0000 7fae42a00700 1 -- 172.21.15.22:0/3027502273 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fae3c020070 con 0x7fae4c106480 2024-03-28T17:54:11.754 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:10.887+0000 7fae42a00700 1 -- 172.21.15.22:0/3027502273 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fae3c0193c0 con 0x7fae4c106480 2024-03-28T17:54:11.754 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:10.887+0000 7fae50c00700 1 -- 172.21.15.22:0/3027502273 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fae4c19e560 con 0x7fae4c106480 2024-03-28T17:54:11.754 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:10.888+0000 7fae50c00700 1 -- 172.21.15.22:0/3027502273 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fae4c1a1210 con 0x7fae4c106480 2024-03-28T17:54:11.754 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:10.888+0000 7fae42a00700 1 -- 172.21.15.22:0/3027502273 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 1) v1 ==== 807+0+0 (secure 0 0 0) 0x7fae3c0199c0 con 0x7fae4c106480 2024-03-28T17:54:11.754 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:10.888+0000 7fae42a00700 1 -- 172.21.15.22:0/3027502273 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7fae3c00b720 con 0x7fae4c106480 2024-03-28T17:54:11.754 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:10.889+0000 7fae50c00700 1 -- 172.21.15.22:0/3027502273 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fae4c0637c0 con 0x7fae4c106480 2024-03-28T17:54:11.754 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:10.893+0000 7fae42a00700 1 -- 172.21.15.22:0/3027502273 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73141 (secure 0 0 0) 0x7fae3c003b80 con 0x7fae4c106480 2024-03-28T17:54:11.754 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:10.973+0000 7fae50c00700 1 -- 172.21.15.22:0/3027502273 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) v1 -- 0x7fae4c0008d0 con 0x7fae4c106480 2024-03-28T17:54:11.754 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:10.973+0000 7fae42a00700 1 -- 172.21.15.22:0/3027502273 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) v1 ==== 79+0+1239 (secure 0 0 0) 0x7fae3c025460 con 0x7fae4c106480 2024-03-28T17:54:11.754 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:10.975+0000 7fae50c00700 1 -- 172.21.15.22:0/3027502273 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fae4c106480 msgr2=0x7fae4c19dd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:54:11.754 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:10.975+0000 7fae50c00700 1 --2- 172.21.15.22:0/3027502273 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fae4c106480 0x7fae4c19dd90 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fae3c004180 tx=0x7fae3c0046a0 comp rx=0 tx=0).stop 2024-03-28T17:54:11.754 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:10.976+0000 7fae50c00700 1 -- 172.21.15.22:0/3027502273 shutdown_connections 2024-03-28T17:54:11.754 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:10.976+0000 7fae50c00700 1 --2- 172.21.15.22:0/3027502273 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fae4c106480 0x7fae4c19dd90 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:54:11.754 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:10.976+0000 7fae50c00700 1 -- 172.21.15.22:0/3027502273 >> 172.21.15.22:0/3027502273 conn(0x7fae4c1020a0 msgr2=0x7fae4c194990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T17:54:11.754 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:10.976+0000 7fae50c00700 1 -- 172.21.15.22:0/3027502273 shutdown_connections 2024-03-28T17:54:11.754 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:10.976+0000 7fae50c00700 1 -- 172.21.15.22:0/3027502273 wait complete. 2024-03-28T17:54:11.755 INFO:teuthology.orchestra.run.smithi022.stdout:mgr not available, waiting (1/15)... 2024-03-28T17:54:12.032 INFO:journalctl@ceph.mgr.x.smithi022.stdout:Mar 28 17:54:11 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mgr-x[32211]: 2024-03-28T17:54:11.802+0000 7fa72ebe2200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-03-28T17:54:12.032 INFO:journalctl@ceph.mgr.x.smithi022.stdout:Mar 28 17:54:11 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mgr-x[32211]: 2024-03-28T17:54:11.926+0000 7fa72ebe2200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-03-28T17:54:12.532 INFO:journalctl@ceph.mgr.x.smithi022.stdout:Mar 28 17:54:12 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mgr-x[32211]: 2024-03-28T17:54:12.107+0000 7fa72ebe2200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-03-28T17:54:12.532 INFO:journalctl@ceph.mgr.x.smithi022.stdout:Mar 28 17:54:12 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mgr-x[32211]: 2024-03-28T17:54:12.235+0000 7fa72ebe2200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-03-28T17:54:12.532 INFO:journalctl@ceph.mgr.x.smithi022.stdout:Mar 28 17:54:12 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mgr-x[32211]: 2024-03-28T17:54:12.348+0000 7fa72ebe2200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-03-28T17:54:12.949 INFO:journalctl@ceph.mgr.x.smithi022.stdout:Mar 28 17:54:12 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mgr-x[32211]: 2024-03-28T17:54:12.689+0000 7fa72ebe2200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-03-28T17:54:12.950 INFO:journalctl@ceph.mgr.x.smithi022.stdout:Mar 28 17:54:12 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mgr-x[32211]: 2024-03-28T17:54:12.834+0000 7fa72ebe2200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-03-28T17:54:13.282 INFO:journalctl@ceph.mgr.x.smithi022.stdout:Mar 28 17:54:12 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mgr-x[32211]: 2024-03-28T17:54:12.949+0000 7fa72ebe2200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-03-28T17:54:13.282 INFO:journalctl@ceph.mgr.x.smithi022.stdout:Mar 28 17:54:13 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mgr-x[32211]: 2024-03-28T17:54:13.077+0000 7fa72ebe2200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-03-28T17:54:13.781 INFO:journalctl@ceph.mgr.x.smithi022.stdout:Mar 28 17:54:13 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mgr-x[32211]: 2024-03-28T17:54:13.298+0000 7fa72ebe2200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-03-28T17:54:14.729 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:54:14 smithi022 ceph-mon[31985]: from='client.? 172.21.15.22:0/4161462401' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-03-28T17:54:14.729 INFO:journalctl@ceph.mgr.x.smithi022.stdout:Mar 28 17:54:14 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mgr-x[32211]: 2024-03-28T17:54:14.671+0000 7fa72ebe2200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-03-28T17:54:15.032 INFO:journalctl@ceph.mgr.x.smithi022.stdout:Mar 28 17:54:14 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mgr-x[32211]: 2024-03-28T17:54:14.856+0000 7fa72ebe2200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-03-28T17:54:15.162 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout 2024-03-28T17:54:15.163 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout { 2024-03-28T17:54:15.163 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "fsid": "fd220748-ed2b-11ee-95d0-87774f69a715", 2024-03-28T17:54:15.163 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "health": { 2024-03-28T17:54:15.164 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-03-28T17:54:15.164 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-03-28T17:54:15.164 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-03-28T17:54:15.164 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout }, 2024-03-28T17:54:15.164 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-03-28T17:54:15.164 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-03-28T17:54:15.164 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout 0 2024-03-28T17:54:15.164 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout ], 2024-03-28T17:54:15.166 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-03-28T17:54:15.167 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "a" 2024-03-28T17:54:15.167 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout ], 2024-03-28T17:54:15.167 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "quorum_age": 7, 2024-03-28T17:54:15.167 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "monmap": { 2024-03-28T17:54:15.167 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-28T17:54:15.167 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-03-28T17:54:15.167 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-03-28T17:54:15.167 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout }, 2024-03-28T17:54:15.167 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-03-28T17:54:15.167 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-28T17:54:15.167 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-03-28T17:54:15.167 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-03-28T17:54:15.167 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-03-28T17:54:15.167 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-03-28T17:54:15.168 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-03-28T17:54:15.168 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-03-28T17:54:15.168 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout }, 2024-03-28T17:54:15.168 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-03-28T17:54:15.172 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-03-28T17:54:15.172 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-03-28T17:54:15.172 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-03-28T17:54:15.172 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-03-28T17:54:15.172 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-03-28T17:54:15.172 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-03-28T17:54:15.172 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-03-28T17:54:15.172 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-03-28T17:54:15.172 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout }, 2024-03-28T17:54:15.173 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-03-28T17:54:15.173 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-28T17:54:15.173 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-03-28T17:54:15.173 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-03-28T17:54:15.173 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout }, 2024-03-28T17:54:15.173 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-03-28T17:54:15.173 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "available": false, 2024-03-28T17:54:15.173 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-03-28T17:54:15.173 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "modules": [ 2024-03-28T17:54:15.173 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "iostat", 2024-03-28T17:54:15.173 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "nfs", 2024-03-28T17:54:15.173 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "restful" 2024-03-28T17:54:15.173 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout ], 2024-03-28T17:54:15.174 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "services": {} 2024-03-28T17:54:15.174 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout }, 2024-03-28T17:54:15.174 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-03-28T17:54:15.174 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-28T17:54:15.174 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "modified": "2024-03-28T17:54:00.441111+0000", 2024-03-28T17:54:15.174 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "services": {} 2024-03-28T17:54:15.174 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout }, 2024-03-28T17:54:15.174 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-03-28T17:54:15.174 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout } 2024-03-28T17:54:15.174 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:14.333+0000 7f4ded600700 1 Processor -- start 2024-03-28T17:54:15.174 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:14.333+0000 7f4ded600700 1 -- start start 2024-03-28T17:54:15.174 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:14.333+0000 7f4ded600700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4de8107b80 0x7f4de8107fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:54:15.174 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:14.333+0000 7f4ded600700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4de8108570 con 0x7f4de8107b80 2024-03-28T17:54:15.175 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:14.334+0000 7f4de6a00700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4de8107b80 0x7f4de8107fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:54:15.175 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:14.334+0000 7f4de6a00700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4de8107b80 0x7f4de8107fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:35502/0 (socket says 172.21.15.22:35502) 2024-03-28T17:54:15.175 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:14.334+0000 7f4de6a00700 1 -- 172.21.15.22:0/1060110448 learned_addr learned my addr 172.21.15.22:0/1060110448 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-28T17:54:15.175 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:14.334+0000 7f4de6a00700 1 -- 172.21.15.22:0/1060110448 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4de8108d80 con 0x7f4de8107b80 2024-03-28T17:54:15.175 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:14.335+0000 7f4de6a00700 1 --2- 172.21.15.22:0/1060110448 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4de8107b80 0x7f4de8107fa0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f4dd8009e80 tx=0x7f4dd800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4bef19f24d65aa70 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T17:54:15.175 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:14.335+0000 7f4de5600700 1 -- 172.21.15.22:0/1060110448 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f4dd8017070 con 0x7f4de8107b80 2024-03-28T17:54:15.179 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:14.335+0000 7f4de5600700 1 -- 172.21.15.22:0/1060110448 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4dd800b8c0 con 0x7f4de8107b80 2024-03-28T17:54:15.179 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:14.335+0000 7f4de5600700 1 -- 172.21.15.22:0/1060110448 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f4dd80045a0 con 0x7f4de8107b80 2024-03-28T17:54:15.179 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:14.336+0000 7f4ded600700 1 -- 172.21.15.22:0/1060110448 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4de8107b80 msgr2=0x7f4de8107fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:54:15.179 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:14.336+0000 7f4ded600700 1 --2- 172.21.15.22:0/1060110448 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4de8107b80 0x7f4de8107fa0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f4dd8009e80 tx=0x7f4dd800b330 comp rx=0 tx=0).stop 2024-03-28T17:54:15.179 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:14.336+0000 7f4ded600700 1 -- 172.21.15.22:0/1060110448 shutdown_connections 2024-03-28T17:54:15.179 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:14.336+0000 7f4ded600700 1 --2- 172.21.15.22:0/1060110448 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4de8107b80 0x7f4de8107fa0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:54:15.179 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:14.336+0000 7f4ded600700 1 -- 172.21.15.22:0/1060110448 >> 172.21.15.22:0/1060110448 conn(0x7f4de8103100 msgr2=0x7f4de8105560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T17:54:15.179 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:14.336+0000 7f4ded600700 1 -- 172.21.15.22:0/1060110448 shutdown_connections 2024-03-28T17:54:15.180 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:14.336+0000 7f4ded600700 1 -- 172.21.15.22:0/1060110448 wait complete. 2024-03-28T17:54:15.180 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:14.337+0000 7f4ded600700 1 Processor -- start 2024-03-28T17:54:15.180 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:14.337+0000 7f4ded600700 1 -- start start 2024-03-28T17:54:15.180 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:14.338+0000 7f4ded600700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4de8107b80 0x7f4de807ca60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:54:15.180 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:14.338+0000 7f4ded600700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4de810c130 con 0x7f4de8107b80 2024-03-28T17:54:15.180 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:14.338+0000 7f4de6a00700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4de8107b80 0x7f4de807ca60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:54:15.180 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:14.338+0000 7f4de6a00700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4de8107b80 0x7f4de807ca60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:35508/0 (socket says 172.21.15.22:35508) 2024-03-28T17:54:15.180 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:14.338+0000 7f4de6a00700 1 -- 172.21.15.22:0/4161462401 learned_addr learned my addr 172.21.15.22:0/4161462401 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-28T17:54:15.180 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:14.338+0000 7f4de6a00700 1 -- 172.21.15.22:0/4161462401 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4de807cfa0 con 0x7f4de8107b80 2024-03-28T17:54:15.180 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:14.339+0000 7f4de6a00700 1 --2- 172.21.15.22:0/4161462401 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4de8107b80 0x7f4de807ca60 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f4dd800d010 tx=0x7f4dd800bd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T17:54:15.180 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:14.339+0000 7f4ddf400700 1 -- 172.21.15.22:0/4161462401 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f4dd8021020 con 0x7f4de8107b80 2024-03-28T17:54:15.180 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:14.339+0000 7f4ddf400700 1 -- 172.21.15.22:0/4161462401 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4dd801d470 con 0x7f4de8107b80 2024-03-28T17:54:15.180 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:14.339+0000 7f4ddf400700 1 -- 172.21.15.22:0/4161462401 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f4dd8017040 con 0x7f4de8107b80 2024-03-28T17:54:15.181 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:14.339+0000 7f4ded600700 1 -- 172.21.15.22:0/4161462401 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4de807b190 con 0x7f4de8107b80 2024-03-28T17:54:15.181 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:14.339+0000 7f4ded600700 1 -- 172.21.15.22:0/4161462401 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4de807b6d0 con 0x7f4de8107b80 2024-03-28T17:54:15.181 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:14.340+0000 7f4ddf400700 1 -- 172.21.15.22:0/4161462401 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 1) v1 ==== 807+0+0 (secure 0 0 0) 0x7f4dd8024020 con 0x7f4de8107b80 2024-03-28T17:54:15.181 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:14.340+0000 7f4ddf400700 1 -- 172.21.15.22:0/4161462401 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f4dd800f050 con 0x7f4de8107b80 2024-03-28T17:54:15.181 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:14.340+0000 7f4ded600700 1 -- 172.21.15.22:0/4161462401 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4de80637c0 con 0x7f4de8107b80 2024-03-28T17:54:15.181 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:14.343+0000 7f4ddf400700 1 -- 172.21.15.22:0/4161462401 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73141 (secure 0 0 0) 0x7f4dd802b020 con 0x7f4de8107b80 2024-03-28T17:54:15.181 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:14.422+0000 7f4ded600700 1 -- 172.21.15.22:0/4161462401 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) v1 -- 0x7f4de807ba10 con 0x7f4de8107b80 2024-03-28T17:54:15.181 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:14.423+0000 7f4ddf400700 1 -- 172.21.15.22:0/4161462401 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) v1 ==== 79+0+1239 (secure 0 0 0) 0x7f4dd801da90 con 0x7f4de8107b80 2024-03-28T17:54:15.181 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:14.424+0000 7f4ded600700 1 -- 172.21.15.22:0/4161462401 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4de8107b80 msgr2=0x7f4de807ca60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:54:15.181 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:14.424+0000 7f4ded600700 1 --2- 172.21.15.22:0/4161462401 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4de8107b80 0x7f4de807ca60 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f4dd800d010 tx=0x7f4dd800bd50 comp rx=0 tx=0).stop 2024-03-28T17:54:15.181 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:14.424+0000 7f4ded600700 1 -- 172.21.15.22:0/4161462401 shutdown_connections 2024-03-28T17:54:15.181 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:14.424+0000 7f4ded600700 1 --2- 172.21.15.22:0/4161462401 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4de8107b80 0x7f4de807ca60 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:54:15.181 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:14.424+0000 7f4ded600700 1 -- 172.21.15.22:0/4161462401 >> 172.21.15.22:0/4161462401 conn(0x7f4de8103100 msgr2=0x7f4de8103c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T17:54:15.181 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:14.425+0000 7f4ded600700 1 -- 172.21.15.22:0/4161462401 shutdown_connections 2024-03-28T17:54:15.182 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:14.425+0000 7f4ded600700 1 -- 172.21.15.22:0/4161462401 wait complete. 2024-03-28T17:54:15.182 INFO:teuthology.orchestra.run.smithi022.stdout:mgr not available, waiting (2/15)... 2024-03-28T17:54:15.430 INFO:journalctl@ceph.mgr.x.smithi022.stdout:Mar 28 17:54:15 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mgr-x[32211]: 2024-03-28T17:54:15.288+0000 7fa72ebe2200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-03-28T17:54:15.709 INFO:journalctl@ceph.mgr.x.smithi022.stdout:Mar 28 17:54:15 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mgr-x[32211]: 2024-03-28T17:54:15.431+0000 7fa72ebe2200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-03-28T17:54:15.963 INFO:journalctl@ceph.mgr.x.smithi022.stdout:Mar 28 17:54:15 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mgr-x[32211]: 2024-03-28T17:54:15.708+0000 7fa72ebe2200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-03-28T17:54:15.963 INFO:journalctl@ceph.mgr.x.smithi022.stdout:Mar 28 17:54:15 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mgr-x[32211]: 2024-03-28T17:54:15.962+0000 7fa72ebe2200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-03-28T17:54:16.636 INFO:journalctl@ceph.mgr.x.smithi022.stdout:Mar 28 17:54:16 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mgr-x[32211]: 2024-03-28T17:54:16.349+0000 7fa72ebe2200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-03-28T17:54:16.967 INFO:journalctl@ceph.mgr.x.smithi022.stdout:Mar 28 17:54:16 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mgr-x[32211]: 2024-03-28T17:54:16.673+0000 7fa72ebe2200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-03-28T17:54:17.281 INFO:journalctl@ceph.mgr.x.smithi022.stdout:Mar 28 17:54:17 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mgr-x[32211]: 2024-03-28T17:54:17.005+0000 7fa72ebe2200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-03-28T17:54:18.144 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:54:17 smithi022 ceph-mon[31985]: from='client.? 172.21.15.22:0/2050213643' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-03-28T17:54:18.532 INFO:journalctl@ceph.mgr.x.smithi022.stdout:Mar 28 17:54:18 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mgr-x[32211]: 2024-03-28T17:54:18.143+0000 7fa72ebe2200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-03-28T17:54:18.547 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout 2024-03-28T17:54:18.547 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout { 2024-03-28T17:54:18.547 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "fsid": "fd220748-ed2b-11ee-95d0-87774f69a715", 2024-03-28T17:54:18.548 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "health": { 2024-03-28T17:54:18.548 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-03-28T17:54:18.548 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-03-28T17:54:18.548 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-03-28T17:54:18.548 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout }, 2024-03-28T17:54:18.548 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-03-28T17:54:18.548 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-03-28T17:54:18.548 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout 0 2024-03-28T17:54:18.548 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout ], 2024-03-28T17:54:18.548 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-03-28T17:54:18.548 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "a" 2024-03-28T17:54:18.548 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout ], 2024-03-28T17:54:18.548 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "quorum_age": 10, 2024-03-28T17:54:18.548 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "monmap": { 2024-03-28T17:54:18.548 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-28T17:54:18.549 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-03-28T17:54:18.549 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-03-28T17:54:18.549 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout }, 2024-03-28T17:54:18.549 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-03-28T17:54:18.549 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-28T17:54:18.549 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-03-28T17:54:18.549 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-03-28T17:54:18.557 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-03-28T17:54:18.557 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-03-28T17:54:18.557 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-03-28T17:54:18.557 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-03-28T17:54:18.557 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout }, 2024-03-28T17:54:18.557 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-03-28T17:54:18.557 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-03-28T17:54:18.557 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-03-28T17:54:18.558 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-03-28T17:54:18.558 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-03-28T17:54:18.558 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-03-28T17:54:18.558 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-03-28T17:54:18.558 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-03-28T17:54:18.558 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-03-28T17:54:18.558 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout }, 2024-03-28T17:54:18.558 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-03-28T17:54:18.558 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-28T17:54:18.558 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-03-28T17:54:18.558 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-03-28T17:54:18.558 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout }, 2024-03-28T17:54:18.558 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-03-28T17:54:18.559 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "available": false, 2024-03-28T17:54:18.559 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-03-28T17:54:18.559 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "modules": [ 2024-03-28T17:54:18.559 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "iostat", 2024-03-28T17:54:18.559 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "nfs", 2024-03-28T17:54:18.559 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "restful" 2024-03-28T17:54:18.559 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout ], 2024-03-28T17:54:18.559 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "services": {} 2024-03-28T17:54:18.559 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout }, 2024-03-28T17:54:18.559 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-03-28T17:54:18.559 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-28T17:54:18.559 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "modified": "2024-03-28T17:54:00.441111+0000", 2024-03-28T17:54:18.559 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "services": {} 2024-03-28T17:54:18.559 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout }, 2024-03-28T17:54:18.560 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-03-28T17:54:18.560 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout } 2024-03-28T17:54:18.560 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:17.775+0000 7f71b6200700 1 Processor -- start 2024-03-28T17:54:18.560 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:17.775+0000 7f71b6200700 1 -- start start 2024-03-28T17:54:18.560 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:17.776+0000 7f71b6200700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f71b0107b80 0x7f71b0107fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:54:18.560 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:17.776+0000 7f71b6200700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71b0108570 con 0x7f71b0107b80 2024-03-28T17:54:18.560 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:17.776+0000 7f71af400700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f71b0107b80 0x7f71b0107fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:54:18.560 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:17.776+0000 7f71af400700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f71b0107b80 0x7f71b0107fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:55550/0 (socket says 172.21.15.22:55550) 2024-03-28T17:54:18.560 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:17.776+0000 7f71af400700 1 -- 172.21.15.22:0/2171756765 learned_addr learned my addr 172.21.15.22:0/2171756765 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-28T17:54:18.560 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:17.777+0000 7f71af400700 1 -- 172.21.15.22:0/2171756765 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f71b0108d80 con 0x7f71b0107b80 2024-03-28T17:54:18.560 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:17.777+0000 7f71af400700 1 --2- 172.21.15.22:0/2171756765 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f71b0107b80 0x7f71b0107fa0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f71a00098a0 tx=0x7f71a0009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=248251d43d8bcc07 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T17:54:18.560 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:17.777+0000 7f71aea00700 1 -- 172.21.15.22:0/2171756765 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f71a0004160 con 0x7f71b0107b80 2024-03-28T17:54:18.560 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:17.777+0000 7f71aea00700 1 -- 172.21.15.22:0/2171756765 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f71a0004300 con 0x7f71b0107b80 2024-03-28T17:54:18.560 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:17.777+0000 7f71aea00700 1 -- 172.21.15.22:0/2171756765 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f71a0011b40 con 0x7f71b0107b80 2024-03-28T17:54:18.561 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:17.778+0000 7f71b6200700 1 -- 172.21.15.22:0/2171756765 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f71b0107b80 msgr2=0x7f71b0107fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:54:18.561 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:17.778+0000 7f71b6200700 1 --2- 172.21.15.22:0/2171756765 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f71b0107b80 0x7f71b0107fa0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f71a00098a0 tx=0x7f71a0009bb0 comp rx=0 tx=0).stop 2024-03-28T17:54:18.561 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:17.778+0000 7f71b6200700 1 -- 172.21.15.22:0/2171756765 shutdown_connections 2024-03-28T17:54:18.561 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:17.778+0000 7f71b6200700 1 --2- 172.21.15.22:0/2171756765 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f71b0107b80 0x7f71b0107fa0 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:54:18.561 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:17.778+0000 7f71b6200700 1 -- 172.21.15.22:0/2171756765 >> 172.21.15.22:0/2171756765 conn(0x7f71b0103100 msgr2=0x7f71b0105560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T17:54:18.561 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:17.779+0000 7f71b6200700 1 -- 172.21.15.22:0/2171756765 shutdown_connections 2024-03-28T17:54:18.561 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:17.779+0000 7f71b6200700 1 -- 172.21.15.22:0/2171756765 wait complete. 2024-03-28T17:54:18.561 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:17.780+0000 7f71b6200700 1 Processor -- start 2024-03-28T17:54:18.561 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:17.780+0000 7f71b6200700 1 -- start start 2024-03-28T17:54:18.561 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:17.780+0000 7f71b6200700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f71b0107b80 0x7f71b019dda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:54:18.561 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:17.780+0000 7f71b6200700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71b010c130 con 0x7f71b0107b80 2024-03-28T17:54:18.561 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:17.780+0000 7f71af400700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f71b0107b80 0x7f71b019dda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:54:18.561 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:17.781+0000 7f71af400700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f71b0107b80 0x7f71b019dda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:55556/0 (socket says 172.21.15.22:55556) 2024-03-28T17:54:18.561 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:17.781+0000 7f71af400700 1 -- 172.21.15.22:0/2050213643 learned_addr learned my addr 172.21.15.22:0/2050213643 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-28T17:54:18.562 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:17.781+0000 7f71af400700 1 -- 172.21.15.22:0/2050213643 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f71b019e2e0 con 0x7f71b0107b80 2024-03-28T17:54:18.562 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:17.781+0000 7f71af400700 1 --2- 172.21.15.22:0/2050213643 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f71b0107b80 0x7f71b019dda0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f71a0011840 tx=0x7f71a0004230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T17:54:18.562 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:17.782+0000 7f71acc00700 1 -- 172.21.15.22:0/2050213643 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f71a00038b0 con 0x7f71b0107b80 2024-03-28T17:54:18.562 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:17.782+0000 7f71acc00700 1 -- 172.21.15.22:0/2050213643 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f71a001a430 con 0x7f71b0107b80 2024-03-28T17:54:18.562 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:17.782+0000 7f71acc00700 1 -- 172.21.15.22:0/2050213643 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f71a0024650 con 0x7f71b0107b80 2024-03-28T17:54:18.562 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:17.782+0000 7f71b6200700 1 -- 172.21.15.22:0/2050213643 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f71b019e570 con 0x7f71b0107b80 2024-03-28T17:54:18.562 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:17.782+0000 7f71b6200700 1 -- 172.21.15.22:0/2050213643 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f71b01a1260 con 0x7f71b0107b80 2024-03-28T17:54:18.562 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:17.782+0000 7f71acc00700 1 -- 172.21.15.22:0/2050213643 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 1) v1 ==== 807+0+0 (secure 0 0 0) 0x7f71a0022020 con 0x7f71b0107b80 2024-03-28T17:54:18.562 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:17.783+0000 7f71acc00700 1 -- 172.21.15.22:0/2050213643 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f71a0024a40 con 0x7f71b0107b80 2024-03-28T17:54:18.562 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:17.783+0000 7f71b6200700 1 -- 172.21.15.22:0/2050213643 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f71b00637c0 con 0x7f71b0107b80 2024-03-28T17:54:18.562 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:17.786+0000 7f71acc00700 1 -- 172.21.15.22:0/2050213643 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73141 (secure 0 0 0) 0x7f71a001aa50 con 0x7f71b0107b80 2024-03-28T17:54:18.562 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:17.863+0000 7f71b6200700 1 -- 172.21.15.22:0/2050213643 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) v1 -- 0x7f71b01a17a0 con 0x7f71b0107b80 2024-03-28T17:54:18.562 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:17.864+0000 7f71acc00700 1 -- 172.21.15.22:0/2050213643 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) v1 ==== 79+0+1240 (secure 0 0 0) 0x7f71a001a5e0 con 0x7f71b0107b80 2024-03-28T17:54:18.562 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:17.866+0000 7f71b6200700 1 -- 172.21.15.22:0/2050213643 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f71b0107b80 msgr2=0x7f71b019dda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:54:18.563 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:17.866+0000 7f71b6200700 1 --2- 172.21.15.22:0/2050213643 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f71b0107b80 0x7f71b019dda0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f71a0011840 tx=0x7f71a0004230 comp rx=0 tx=0).stop 2024-03-28T17:54:18.563 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:17.866+0000 7f71b6200700 1 -- 172.21.15.22:0/2050213643 shutdown_connections 2024-03-28T17:54:18.563 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:17.866+0000 7f71b6200700 1 --2- 172.21.15.22:0/2050213643 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f71b0107b80 0x7f71b019dda0 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:54:18.563 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:17.866+0000 7f71b6200700 1 -- 172.21.15.22:0/2050213643 >> 172.21.15.22:0/2050213643 conn(0x7f71b0103100 msgr2=0x7f71b0103bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T17:54:18.563 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:17.866+0000 7f71b6200700 1 -- 172.21.15.22:0/2050213643 shutdown_connections 2024-03-28T17:54:18.563 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:17.866+0000 7f71b6200700 1 -- 172.21.15.22:0/2050213643 wait complete. 2024-03-28T17:54:18.563 INFO:teuthology.orchestra.run.smithi022.stdout:mgr not available, waiting (3/15)... 2024-03-28T17:54:19.282 INFO:journalctl@ceph.mgr.x.smithi022.stdout:Mar 28 17:54:18 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mgr-x[32211]: 2024-03-28T17:54:18.866+0000 7fa72ebe2200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-03-28T17:54:19.282 INFO:journalctl@ceph.mgr.x.smithi022.stdout:Mar 28 17:54:19 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mgr-x[32211]: 2024-03-28T17:54:18.999+0000 7fa72ebe2200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-03-28T17:54:19.782 INFO:journalctl@ceph.mgr.x.smithi022.stdout:Mar 28 17:54:19 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mgr-x[32211]: 2024-03-28T17:54:19.286+0000 7fa72ebe2200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-03-28T17:54:19.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:54:19 smithi022 ceph-mon[31985]: Activating manager daemon x 2024-03-28T17:54:19.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:54:19 smithi022 ceph-mon[31985]: mgrmap e2: x(active, starting, since 0.00750925s) 2024-03-28T17:54:19.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:54:19 smithi022 ceph-mon[31985]: from='mgr.14100 172.21.15.22:0/1893540665' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-03-28T17:54:19.783 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:54:19 smithi022 ceph-mon[31985]: from='mgr.14100 172.21.15.22:0/1893540665' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-03-28T17:54:19.783 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:54:19 smithi022 ceph-mon[31985]: from='mgr.14100 172.21.15.22:0/1893540665' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-03-28T17:54:19.783 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:54:19 smithi022 ceph-mon[31985]: from='mgr.14100 172.21.15.22:0/1893540665' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-28T17:54:19.783 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:54:19 smithi022 ceph-mon[31985]: from='mgr.14100 172.21.15.22:0/1893540665' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-03-28T17:54:19.783 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:54:19 smithi022 ceph-mon[31985]: Manager daemon x is now available 2024-03-28T17:54:19.783 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:54:19 smithi022 ceph-mon[31985]: from='mgr.14100 172.21.15.22:0/1893540665' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-03-28T17:54:19.783 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:54:19 smithi022 ceph-mon[31985]: from='mgr.14100 172.21.15.22:0/1893540665' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-03-28T17:54:19.783 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:54:19 smithi022 ceph-mon[31985]: from='mgr.14100 172.21.15.22:0/1893540665' entity='mgr.x' 2024-03-28T17:54:19.783 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:54:19 smithi022 ceph-mon[31985]: from='mgr.14100 172.21.15.22:0/1893540665' entity='mgr.x' 2024-03-28T17:54:20.720 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:54:20 smithi022 ceph-mon[31985]: from='mgr.14100 172.21.15.22:0/1893540665' entity='mgr.x' 2024-03-28T17:54:20.721 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:54:20 smithi022 ceph-mon[31985]: mgrmap e3: x(active, since 1.02329s) 2024-03-28T17:54:22.250 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout 2024-03-28T17:54:22.250 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout { 2024-03-28T17:54:22.250 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "fsid": "fd220748-ed2b-11ee-95d0-87774f69a715", 2024-03-28T17:54:22.250 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "health": { 2024-03-28T17:54:22.250 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-03-28T17:54:22.250 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-03-28T17:54:22.250 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-03-28T17:54:22.254 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout }, 2024-03-28T17:54:22.254 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-03-28T17:54:22.254 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-03-28T17:54:22.254 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout 0 2024-03-28T17:54:22.254 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout ], 2024-03-28T17:54:22.254 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-03-28T17:54:22.254 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "a" 2024-03-28T17:54:22.254 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout ], 2024-03-28T17:54:22.254 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "quorum_age": 14, 2024-03-28T17:54:22.254 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "monmap": { 2024-03-28T17:54:22.255 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-28T17:54:22.255 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-03-28T17:54:22.255 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-03-28T17:54:22.255 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout }, 2024-03-28T17:54:22.255 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-03-28T17:54:22.255 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-28T17:54:22.255 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-03-28T17:54:22.255 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-03-28T17:54:22.255 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-03-28T17:54:22.255 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-03-28T17:54:22.255 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-03-28T17:54:22.255 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-03-28T17:54:22.255 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout }, 2024-03-28T17:54:22.255 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-03-28T17:54:22.256 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-03-28T17:54:22.256 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-03-28T17:54:22.256 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-03-28T17:54:22.256 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-03-28T17:54:22.256 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-03-28T17:54:22.256 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-03-28T17:54:22.256 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-03-28T17:54:22.256 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-03-28T17:54:22.256 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout }, 2024-03-28T17:54:22.256 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-03-28T17:54:22.264 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-28T17:54:22.264 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-03-28T17:54:22.264 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-03-28T17:54:22.264 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout }, 2024-03-28T17:54:22.264 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-03-28T17:54:22.264 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "available": true, 2024-03-28T17:54:22.264 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-03-28T17:54:22.264 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "modules": [ 2024-03-28T17:54:22.264 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "iostat", 2024-03-28T17:54:22.264 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "nfs", 2024-03-28T17:54:22.264 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "restful" 2024-03-28T17:54:22.264 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout ], 2024-03-28T17:54:22.265 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "services": {} 2024-03-28T17:54:22.265 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout }, 2024-03-28T17:54:22.265 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-03-28T17:54:22.265 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-28T17:54:22.265 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "modified": "2024-03-28T17:54:00.441111+0000", 2024-03-28T17:54:22.265 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "services": {} 2024-03-28T17:54:22.265 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout }, 2024-03-28T17:54:22.265 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-03-28T17:54:22.265 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout } 2024-03-28T17:54:22.265 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:21.204+0000 7f31b0600700 1 Processor -- start 2024-03-28T17:54:22.265 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:21.204+0000 7f31b0600700 1 -- start start 2024-03-28T17:54:22.265 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:21.204+0000 7f31b0600700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f31a807e860 0x7f31a807ec80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:54:22.265 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:21.204+0000 7f31b0600700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31a807f250 con 0x7f31a807e860 2024-03-28T17:54:22.265 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:21.205+0000 7f31adc00700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f31a807e860 0x7f31a807ec80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:54:22.266 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:21.205+0000 7f31adc00700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f31a807e860 0x7f31a807ec80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:55646/0 (socket says 172.21.15.22:55646) 2024-03-28T17:54:22.266 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:21.205+0000 7f31adc00700 1 -- 172.21.15.22:0/2882348063 learned_addr learned my addr 172.21.15.22:0/2882348063 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-28T17:54:22.266 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:21.205+0000 7f31adc00700 1 -- 172.21.15.22:0/2882348063 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f31a807fa60 con 0x7f31a807e860 2024-03-28T17:54:22.266 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:21.205+0000 7f31adc00700 1 --2- 172.21.15.22:0/2882348063 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f31a807e860 0x7f31a807ec80 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f319c0098a0 tx=0x7f319c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3c7681a01e975b37 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T17:54:22.266 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:21.206+0000 7f31a7e00700 1 -- 172.21.15.22:0/2882348063 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f319c004160 con 0x7f31a807e860 2024-03-28T17:54:22.266 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:21.206+0000 7f31a7e00700 1 -- 172.21.15.22:0/2882348063 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f319c004300 con 0x7f31a807e860 2024-03-28T17:54:22.266 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:21.206+0000 7f31a7e00700 1 -- 172.21.15.22:0/2882348063 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f319c011b40 con 0x7f31a807e860 2024-03-28T17:54:22.266 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:21.206+0000 7f31b0600700 1 -- 172.21.15.22:0/2882348063 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f31a807e860 msgr2=0x7f31a807ec80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:54:22.266 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:21.206+0000 7f31b0600700 1 --2- 172.21.15.22:0/2882348063 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f31a807e860 0x7f31a807ec80 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f319c0098a0 tx=0x7f319c009bb0 comp rx=0 tx=0).stop 2024-03-28T17:54:22.266 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:21.206+0000 7f31b0600700 1 -- 172.21.15.22:0/2882348063 shutdown_connections 2024-03-28T17:54:22.266 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:21.206+0000 7f31b0600700 1 --2- 172.21.15.22:0/2882348063 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f31a807e860 0x7f31a807ec80 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:54:22.266 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:21.206+0000 7f31b0600700 1 -- 172.21.15.22:0/2882348063 >> 172.21.15.22:0/2882348063 conn(0x7f31a807d150 msgr2=0x7f31a807d5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T17:54:22.266 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:21.207+0000 7f31b0600700 1 -- 172.21.15.22:0/2882348063 shutdown_connections 2024-03-28T17:54:22.266 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:21.207+0000 7f31b0600700 1 -- 172.21.15.22:0/2882348063 wait complete. 2024-03-28T17:54:22.267 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:21.208+0000 7f31b0600700 1 Processor -- start 2024-03-28T17:54:22.267 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:21.208+0000 7f31b0600700 1 -- start start 2024-03-28T17:54:22.267 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:21.208+0000 7f31b0600700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f31a807e860 0x7f31a81a2130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:54:22.267 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:21.208+0000 7f31b0600700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31a8080e20 con 0x7f31a807e860 2024-03-28T17:54:22.267 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:21.208+0000 7f31adc00700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f31a807e860 0x7f31a81a2130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:54:22.267 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:21.208+0000 7f31adc00700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f31a807e860 0x7f31a81a2130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:55658/0 (socket says 172.21.15.22:55658) 2024-03-28T17:54:22.267 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:21.208+0000 7f31adc00700 1 -- 172.21.15.22:0/2266575425 learned_addr learned my addr 172.21.15.22:0/2266575425 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-28T17:54:22.267 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:21.209+0000 7f31adc00700 1 -- 172.21.15.22:0/2266575425 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f31a81a2670 con 0x7f31a807e860 2024-03-28T17:54:22.267 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:21.209+0000 7f31adc00700 1 --2- 172.21.15.22:0/2266575425 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f31a807e860 0x7f31a81a2130 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f319c004180 tx=0x7f319c0046a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T17:54:22.267 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:21.209+0000 7f31a6000700 1 -- 172.21.15.22:0/2266575425 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f319c0039e0 con 0x7f31a807e860 2024-03-28T17:54:22.267 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:21.209+0000 7f31a6000700 1 -- 172.21.15.22:0/2266575425 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f319c020070 con 0x7f31a807e860 2024-03-28T17:54:22.267 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:21.209+0000 7f31a6000700 1 -- 172.21.15.22:0/2266575425 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f319c0193c0 con 0x7f31a807e860 2024-03-28T17:54:22.267 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:21.209+0000 7f31b0600700 1 -- 172.21.15.22:0/2266575425 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f31a81a2900 con 0x7f31a807e860 2024-03-28T17:54:22.267 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:21.209+0000 7f31b0600700 1 -- 172.21.15.22:0/2266575425 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f31a81a55f0 con 0x7f31a807e860 2024-03-28T17:54:22.268 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:21.211+0000 7f31b0600700 1 -- 172.21.15.22:0/2266575425 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f31a80637c0 con 0x7f31a807e860 2024-03-28T17:54:22.268 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:21.211+0000 7f31a6000700 1 -- 172.21.15.22:0/2266575425 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 3) v1 ==== 45989+0+0 (secure 0 0 0) 0x7f319c01a430 con 0x7f31a807e860 2024-03-28T17:54:22.268 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:21.212+0000 7f31a6000700 1 --2- 172.21.15.22:0/2266575425 >> [v2:172.21.15.22:6800/3166828081,v1:172.21.15.22:6801/3166828081] conn(0x7f3190038db0 0x7f319003b270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:54:22.268 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:21.212+0000 7f31a6000700 1 -- 172.21.15.22:0/2266575425 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f319c04cd90 con 0x7f31a807e860 2024-03-28T17:54:22.268 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:21.212+0000 7f31ad200700 1 --2- 172.21.15.22:0/2266575425 >> [v2:172.21.15.22:6800/3166828081,v1:172.21.15.22:6801/3166828081] conn(0x7f3190038db0 0x7f319003b270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:54:22.268 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:21.213+0000 7f31ad200700 1 --2- 172.21.15.22:0/2266575425 >> [v2:172.21.15.22:6800/3166828081,v1:172.21.15.22:6801/3166828081] conn(0x7f3190038db0 0x7f319003b270 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f3194006fd0 tx=0x7f3194006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T17:54:22.268 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:21.218+0000 7f31a6000700 1 -- 172.21.15.22:0/2266575425 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f319c003b80 con 0x7f31a807e860 2024-03-28T17:54:22.268 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:21.351+0000 7f31a6000700 1 -- 172.21.15.22:0/2266575425 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mgrmap(e 4) v1 ==== 46095+0+0 (secure 0 0 0) 0x7f319c02bd00 con 0x7f31a807e860 2024-03-28T17:54:22.268 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:21.529+0000 7f31b0600700 1 -- 172.21.15.22:0/2266575425 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) v1 -- 0x7f31a80008d0 con 0x7f31a807e860 2024-03-28T17:54:22.268 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:21.530+0000 7f31a6000700 1 -- 172.21.15.22:0/2266575425 <== mon.0 v2:172.21.15.22:3300/0 8 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) v1 ==== 79+0+1239 (secure 0 0 0) 0x7f319c022d90 con 0x7f31a807e860 2024-03-28T17:54:22.268 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:21.533+0000 7f31b0600700 1 -- 172.21.15.22:0/2266575425 >> [v2:172.21.15.22:6800/3166828081,v1:172.21.15.22:6801/3166828081] conn(0x7f3190038db0 msgr2=0x7f319003b270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:54:22.268 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:21.533+0000 7f31b0600700 1 --2- 172.21.15.22:0/2266575425 >> [v2:172.21.15.22:6800/3166828081,v1:172.21.15.22:6801/3166828081] conn(0x7f3190038db0 0x7f319003b270 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f3194006fd0 tx=0x7f3194006e40 comp rx=0 tx=0).stop 2024-03-28T17:54:22.268 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:21.533+0000 7f31b0600700 1 -- 172.21.15.22:0/2266575425 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f31a807e860 msgr2=0x7f31a81a2130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:54:22.269 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:21.533+0000 7f31b0600700 1 --2- 172.21.15.22:0/2266575425 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f31a807e860 0x7f31a81a2130 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f319c004180 tx=0x7f319c0046a0 comp rx=0 tx=0).stop 2024-03-28T17:54:22.269 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:21.533+0000 7f31b0600700 1 -- 172.21.15.22:0/2266575425 shutdown_connections 2024-03-28T17:54:22.269 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:21.533+0000 7f31b0600700 1 --2- 172.21.15.22:0/2266575425 >> [v2:172.21.15.22:6800/3166828081,v1:172.21.15.22:6801/3166828081] conn(0x7f3190038db0 0x7f319003b270 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:54:22.269 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:21.533+0000 7f31b0600700 1 --2- 172.21.15.22:0/2266575425 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f31a807e860 0x7f31a81a2130 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:54:22.269 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:21.533+0000 7f31b0600700 1 -- 172.21.15.22:0/2266575425 >> 172.21.15.22:0/2266575425 conn(0x7f31a807d150 msgr2=0x7f31a8198df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T17:54:22.269 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:21.534+0000 7f31b0600700 1 -- 172.21.15.22:0/2266575425 shutdown_connections 2024-03-28T17:54:22.269 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:21.534+0000 7f31b0600700 1 -- 172.21.15.22:0/2266575425 wait complete. 2024-03-28T17:54:22.269 INFO:teuthology.orchestra.run.smithi022.stdout:mgr is available 2024-03-28T17:54:22.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:54:22 smithi022 ceph-mon[31985]: mgrmap e4: x(active, since 2s) 2024-03-28T17:54:22.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:54:22 smithi022 ceph-mon[31985]: from='client.? 172.21.15.22:0/2266575425' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-03-28T17:54:23.527 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:54:23 smithi022 ceph-mon[31985]: from='client.? 172.21.15.22:0/1487115814' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2024-03-28T17:54:23.847 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout 2024-03-28T17:54:23.847 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout [global] 2024-03-28T17:54:23.847 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout fsid = fd220748-ed2b-11ee-95d0-87774f69a715 2024-03-28T17:54:23.847 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-03-28T17:54:23.847 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-03-28T17:54:23.848 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-03-28T17:54:23.848 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-03-28T17:54:23.848 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout 2024-03-28T17:54:23.848 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout [mgr] 2024-03-28T17:54:23.848 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-03-28T17:54:23.848 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout 2024-03-28T17:54:23.848 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout [osd] 2024-03-28T17:54:23.848 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout bdev_async_discard = True 2024-03-28T17:54:23.848 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2024-03-28T17:54:23.848 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2024-03-28T17:54:23.848 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2024-03-28T17:54:23.848 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-03-28T17:54:23.848 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-03-28T17:54:23.849 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:22.854+0000 7f96cc000700 1 Processor -- start 2024-03-28T17:54:23.849 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:22.854+0000 7f96cc000700 1 -- start start 2024-03-28T17:54:23.849 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:22.855+0000 7f96cc000700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f96c4107b80 0x7f96c4107fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:54:23.849 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:22.855+0000 7f96cc000700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96c4108570 con 0x7f96c4107b80 2024-03-28T17:54:23.849 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:22.855+0000 7f96c9600700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f96c4107b80 0x7f96c4107fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:54:23.849 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:22.856+0000 7f96c9600700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f96c4107b80 0x7f96c4107fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:55660/0 (socket says 172.21.15.22:55660) 2024-03-28T17:54:23.854 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:22.856+0000 7f96c9600700 1 -- 172.21.15.22:0/3423767318 learned_addr learned my addr 172.21.15.22:0/3423767318 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-28T17:54:23.854 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:22.856+0000 7f96c9600700 1 -- 172.21.15.22:0/3423767318 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f96c4108d80 con 0x7f96c4107b80 2024-03-28T17:54:23.854 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:22.856+0000 7f96c9600700 1 --2- 172.21.15.22:0/3423767318 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f96c4107b80 0x7f96c4107fa0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f96b80098a0 tx=0x7f96b8009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e75c187c812a88f1 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T17:54:23.855 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:22.857+0000 7f96c3e00700 1 -- 172.21.15.22:0/3423767318 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f96b8004160 con 0x7f96c4107b80 2024-03-28T17:54:23.855 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:22.857+0000 7f96c3e00700 1 -- 172.21.15.22:0/3423767318 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f96b8004300 con 0x7f96c4107b80 2024-03-28T17:54:23.855 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:22.857+0000 7f96c3e00700 1 -- 172.21.15.22:0/3423767318 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f96b8011b40 con 0x7f96c4107b80 2024-03-28T17:54:23.855 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:22.857+0000 7f96cc000700 1 -- 172.21.15.22:0/3423767318 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f96c4107b80 msgr2=0x7f96c4107fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:54:23.855 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:22.857+0000 7f96cc000700 1 --2- 172.21.15.22:0/3423767318 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f96c4107b80 0x7f96c4107fa0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f96b80098a0 tx=0x7f96b8009bb0 comp rx=0 tx=0).stop 2024-03-28T17:54:23.855 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:22.857+0000 7f96cc000700 1 -- 172.21.15.22:0/3423767318 shutdown_connections 2024-03-28T17:54:23.855 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:22.857+0000 7f96cc000700 1 --2- 172.21.15.22:0/3423767318 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f96c4107b80 0x7f96c4107fa0 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:54:23.855 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:22.857+0000 7f96cc000700 1 -- 172.21.15.22:0/3423767318 >> 172.21.15.22:0/3423767318 conn(0x7f96c4103100 msgr2=0x7f96c4105560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T17:54:23.855 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:22.857+0000 7f96cc000700 1 -- 172.21.15.22:0/3423767318 shutdown_connections 2024-03-28T17:54:23.855 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:22.858+0000 7f96cc000700 1 -- 172.21.15.22:0/3423767318 wait complete. 2024-03-28T17:54:23.855 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:22.858+0000 7f96cc000700 1 Processor -- start 2024-03-28T17:54:23.855 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:22.859+0000 7f96cc000700 1 -- start start 2024-03-28T17:54:23.855 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:22.859+0000 7f96cc000700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f96c4107b80 0x7f96c407ca60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:54:23.855 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:22.859+0000 7f96cc000700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96c410c130 con 0x7f96c4107b80 2024-03-28T17:54:23.855 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:22.859+0000 7f96c9600700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f96c4107b80 0x7f96c407ca60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:54:23.855 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:22.859+0000 7f96c9600700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f96c4107b80 0x7f96c407ca60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:55664/0 (socket says 172.21.15.22:55664) 2024-03-28T17:54:23.856 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:22.859+0000 7f96c9600700 1 -- 172.21.15.22:0/1487115814 learned_addr learned my addr 172.21.15.22:0/1487115814 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-28T17:54:23.856 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:22.860+0000 7f96c9600700 1 -- 172.21.15.22:0/1487115814 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f96c407cfa0 con 0x7f96c4107b80 2024-03-28T17:54:23.856 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:22.860+0000 7f96c9600700 1 --2- 172.21.15.22:0/1487115814 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f96c4107b80 0x7f96c407ca60 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f96b8011840 tx=0x7f96b8004230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T17:54:23.856 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:22.860+0000 7f96c2000700 1 -- 172.21.15.22:0/1487115814 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f96b80038b0 con 0x7f96c4107b80 2024-03-28T17:54:23.856 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:22.860+0000 7f96c2000700 1 -- 172.21.15.22:0/1487115814 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f96b801a430 con 0x7f96c4107b80 2024-03-28T17:54:23.856 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:22.860+0000 7f96c2000700 1 -- 172.21.15.22:0/1487115814 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f96b8024650 con 0x7f96c4107b80 2024-03-28T17:54:23.856 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:22.860+0000 7f96cc000700 1 -- 172.21.15.22:0/1487115814 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f96c407b190 con 0x7f96c4107b80 2024-03-28T17:54:23.856 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:22.860+0000 7f96cc000700 1 -- 172.21.15.22:0/1487115814 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f96c407b6d0 con 0x7f96c4107b80 2024-03-28T17:54:23.856 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:22.862+0000 7f96cc000700 1 -- 172.21.15.22:0/1487115814 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f96c40637c0 con 0x7f96c4107b80 2024-03-28T17:54:23.856 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:22.862+0000 7f96c2000700 1 -- 172.21.15.22:0/1487115814 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 4) v1 ==== 46095+0+0 (secure 0 0 0) 0x7f96b8022020 con 0x7f96c4107b80 2024-03-28T17:54:23.856 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:22.862+0000 7f96c2000700 1 --2- 172.21.15.22:0/1487115814 >> [v2:172.21.15.22:6800/3166828081,v1:172.21.15.22:6801/3166828081] conn(0x7f96ac038f20 0x7f96ac03b3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:54:23.856 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:22.863+0000 7f96c2000700 1 -- 172.21.15.22:0/1487115814 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f96b804c910 con 0x7f96c4107b80 2024-03-28T17:54:23.856 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:22.863+0000 7f96c8c00700 1 --2- 172.21.15.22:0/1487115814 >> [v2:172.21.15.22:6800/3166828081,v1:172.21.15.22:6801/3166828081] conn(0x7f96ac038f20 0x7f96ac03b3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:54:23.856 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:22.864+0000 7f96c8c00700 1 --2- 172.21.15.22:0/1487115814 >> [v2:172.21.15.22:6800/3166828081,v1:172.21.15.22:6801/3166828081] conn(0x7f96ac038f20 0x7f96ac03b3e0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f96b0006fd0 tx=0x7f96b0006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T17:54:23.857 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:22.868+0000 7f96c2000700 1 -- 172.21.15.22:0/1487115814 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f96b801ab00 con 0x7f96c4107b80 2024-03-28T17:54:23.857 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:23.094+0000 7f96cc000700 1 -- 172.21.15.22:0/1487115814 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "config assimilate-conf"} v 0) v1 -- 0x7f96c407bec0 con 0x7f96c4107b80 2024-03-28T17:54:23.857 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:23.096+0000 7f96c2000700 1 -- 172.21.15.22:0/1487115814 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v3) v1 ==== 70+0+400 (secure 0 0 0) 0x7f96b80212c0 con 0x7f96c4107b80 2024-03-28T17:54:23.857 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:23.099+0000 7f96cc000700 1 -- 172.21.15.22:0/1487115814 >> [v2:172.21.15.22:6800/3166828081,v1:172.21.15.22:6801/3166828081] conn(0x7f96ac038f20 msgr2=0x7f96ac03b3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:54:23.857 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:23.099+0000 7f96cc000700 1 --2- 172.21.15.22:0/1487115814 >> [v2:172.21.15.22:6800/3166828081,v1:172.21.15.22:6801/3166828081] conn(0x7f96ac038f20 0x7f96ac03b3e0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f96b0006fd0 tx=0x7f96b0006e40 comp rx=0 tx=0).stop 2024-03-28T17:54:23.857 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:23.099+0000 7f96cc000700 1 -- 172.21.15.22:0/1487115814 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f96c4107b80 msgr2=0x7f96c407ca60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:54:23.857 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:23.099+0000 7f96cc000700 1 --2- 172.21.15.22:0/1487115814 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f96c4107b80 0x7f96c407ca60 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f96b8011840 tx=0x7f96b8004230 comp rx=0 tx=0).stop 2024-03-28T17:54:23.857 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:23.099+0000 7f96cc000700 1 -- 172.21.15.22:0/1487115814 shutdown_connections 2024-03-28T17:54:23.857 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:23.099+0000 7f96cc000700 1 --2- 172.21.15.22:0/1487115814 >> [v2:172.21.15.22:6800/3166828081,v1:172.21.15.22:6801/3166828081] conn(0x7f96ac038f20 0x7f96ac03b3e0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:54:23.857 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:23.099+0000 7f96cc000700 1 --2- 172.21.15.22:0/1487115814 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f96c4107b80 0x7f96c407ca60 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:54:23.857 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:23.099+0000 7f96cc000700 1 -- 172.21.15.22:0/1487115814 >> 172.21.15.22:0/1487115814 conn(0x7f96c4103100 msgr2=0x7f96c4103cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T17:54:23.857 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:23.099+0000 7f96cc000700 1 -- 172.21.15.22:0/1487115814 shutdown_connections 2024-03-28T17:54:23.857 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:23.099+0000 7f96cc000700 1 -- 172.21.15.22:0/1487115814 wait complete. 2024-03-28T17:54:23.858 INFO:teuthology.orchestra.run.smithi022.stdout:Enabling cephadm module... 2024-03-28T17:54:25.256 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:54:24 smithi022 ceph-mon[31985]: from='client.? 172.21.15.22:0/2253303838' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "cephadm"}]: dispatch 2024-03-28T17:54:25.256 INFO:journalctl@ceph.mgr.x.smithi022.stdout:Mar 28 17:54:24 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mgr-x[32211]: ignoring --setuser ceph since I am not root 2024-03-28T17:54:25.256 INFO:journalctl@ceph.mgr.x.smithi022.stdout:Mar 28 17:54:24 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mgr-x[32211]: ignoring --setgroup ceph since I am not root 2024-03-28T17:54:25.623 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:24.503+0000 7f0442200700 1 Processor -- start 2024-03-28T17:54:25.623 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:24.503+0000 7f0442200700 1 -- start start 2024-03-28T17:54:25.623 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:24.503+0000 7f0442200700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f043c106a80 0x7f043c106ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:54:25.623 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:24.503+0000 7f0442200700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f043c107470 con 0x7f043c106a80 2024-03-28T17:54:25.623 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:24.504+0000 7f043b400700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f043c106a80 0x7f043c106ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:54:25.623 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:24.504+0000 7f043b400700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f043c106a80 0x7f043c106ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:55670/0 (socket says 172.21.15.22:55670) 2024-03-28T17:54:25.623 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:24.504+0000 7f043b400700 1 -- 172.21.15.22:0/987599805 learned_addr learned my addr 172.21.15.22:0/987599805 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-28T17:54:25.623 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:24.504+0000 7f043b400700 1 -- 172.21.15.22:0/987599805 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f043c107c80 con 0x7f043c106a80 2024-03-28T17:54:25.623 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:24.505+0000 7f043b400700 1 --2- 172.21.15.22:0/987599805 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f043c106a80 0x7f043c106ea0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f042c0098a0 tx=0x7f042c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f185c00e41f4b992 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T17:54:25.623 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:24.505+0000 7f043aa00700 1 -- 172.21.15.22:0/987599805 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f042c004160 con 0x7f043c106a80 2024-03-28T17:54:25.624 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:24.505+0000 7f043aa00700 1 -- 172.21.15.22:0/987599805 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f042c004300 con 0x7f043c106a80 2024-03-28T17:54:25.624 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:24.505+0000 7f043aa00700 1 -- 172.21.15.22:0/987599805 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f042c011b40 con 0x7f043c106a80 2024-03-28T17:54:25.624 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:24.506+0000 7f0442200700 1 -- 172.21.15.22:0/987599805 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f043c106a80 msgr2=0x7f043c106ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:54:25.624 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:24.506+0000 7f0442200700 1 --2- 172.21.15.22:0/987599805 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f043c106a80 0x7f043c106ea0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f042c0098a0 tx=0x7f042c009bb0 comp rx=0 tx=0).stop 2024-03-28T17:54:25.624 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:24.506+0000 7f0442200700 1 -- 172.21.15.22:0/987599805 shutdown_connections 2024-03-28T17:54:25.624 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:24.506+0000 7f0442200700 1 --2- 172.21.15.22:0/987599805 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f043c106a80 0x7f043c106ea0 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:54:25.624 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:24.506+0000 7f0442200700 1 -- 172.21.15.22:0/987599805 >> 172.21.15.22:0/987599805 conn(0x7f043c102000 msgr2=0x7f043c104460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T17:54:25.624 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:24.506+0000 7f0442200700 1 -- 172.21.15.22:0/987599805 shutdown_connections 2024-03-28T17:54:25.624 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:24.507+0000 7f0442200700 1 -- 172.21.15.22:0/987599805 wait complete. 2024-03-28T17:54:25.624 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:24.507+0000 7f0442200700 1 Processor -- start 2024-03-28T17:54:25.624 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:24.508+0000 7f0442200700 1 -- start start 2024-03-28T17:54:25.624 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:24.508+0000 7f0442200700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f043c19df90 0x7f043c19e3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:54:25.624 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:24.508+0000 7f0442200700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f043c10b030 con 0x7f043c19df90 2024-03-28T17:54:25.624 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:24.508+0000 7f043b400700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f043c19df90 0x7f043c19e3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:54:25.624 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:24.508+0000 7f043b400700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f043c19df90 0x7f043c19e3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:55680/0 (socket says 172.21.15.22:55680) 2024-03-28T17:54:25.625 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:24.508+0000 7f043b400700 1 -- 172.21.15.22:0/2253303838 learned_addr learned my addr 172.21.15.22:0/2253303838 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-28T17:54:25.625 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:24.509+0000 7f043b400700 1 -- 172.21.15.22:0/2253303838 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f043c19e8f0 con 0x7f043c19df90 2024-03-28T17:54:25.625 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:24.509+0000 7f043b400700 1 --2- 172.21.15.22:0/2253303838 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f043c19df90 0x7f043c19e3b0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f042c000c00 tx=0x7f042c011920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T17:54:25.625 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:24.509+0000 7f0438c00700 1 -- 172.21.15.22:0/2253303838 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f042c0037d0 con 0x7f043c19df90 2024-03-28T17:54:25.625 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:24.509+0000 7f0438c00700 1 -- 172.21.15.22:0/2253303838 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f042c01f070 con 0x7f043c19df90 2024-03-28T17:54:25.625 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:24.509+0000 7f0438c00700 1 -- 172.21.15.22:0/2253303838 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f042c01a460 con 0x7f043c19df90 2024-03-28T17:54:25.625 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:24.509+0000 7f0442200700 1 -- 172.21.15.22:0/2253303838 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f043c19eb80 con 0x7f043c19df90 2024-03-28T17:54:25.625 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:24.510+0000 7f0442200700 1 -- 172.21.15.22:0/2253303838 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f043c1a1660 con 0x7f043c19df90 2024-03-28T17:54:25.625 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:24.511+0000 7f0442200700 1 -- 172.21.15.22:0/2253303838 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f043c0637c0 con 0x7f043c19df90 2024-03-28T17:54:25.625 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:24.511+0000 7f0438c00700 1 -- 172.21.15.22:0/2253303838 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 4) v1 ==== 46095+0+0 (secure 0 0 0) 0x7f042c024470 con 0x7f043c19df90 2024-03-28T17:54:25.625 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:24.511+0000 7f0438c00700 1 --2- 172.21.15.22:0/2253303838 >> [v2:172.21.15.22:6800/3166828081,v1:172.21.15.22:6801/3166828081] conn(0x7f0420038f20 0x7f042003b3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:54:25.625 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:24.511+0000 7f0438c00700 1 -- 172.21.15.22:0/2253303838 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f042c04d400 con 0x7f043c19df90 2024-03-28T17:54:25.625 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:24.512+0000 7f0432a00700 1 --2- 172.21.15.22:0/2253303838 >> [v2:172.21.15.22:6800/3166828081,v1:172.21.15.22:6801/3166828081] conn(0x7f0420038f20 0x7f042003b3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:54:25.625 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:24.512+0000 7f0432a00700 1 --2- 172.21.15.22:0/2253303838 >> [v2:172.21.15.22:6800/3166828081,v1:172.21.15.22:6801/3166828081] conn(0x7f0420038f20 0x7f042003b3e0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f042400ad30 tx=0x7f04240093f0 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T17:54:25.625 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:24.517+0000 7f0438c00700 1 -- 172.21.15.22:0/2253303838 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f042c0094f0 con 0x7f043c19df90 2024-03-28T17:54:25.625 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:24.806+0000 7f0442200700 1 -- 172.21.15.22:0/2253303838 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "cephadm"} v 0) v1 -- 0x7f043c062da0 con 0x7f043c19df90 2024-03-28T17:54:25.626 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:24.860+0000 7f0438c00700 1 -- 172.21.15.22:0/2253303838 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "cephadm"}]=0 v5) v1 ==== 86+0+0 (secure 0 0 0) 0x7f042c04ce20 con 0x7f043c19df90 2024-03-28T17:54:25.626 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:24.864+0000 7f0442200700 1 -- 172.21.15.22:0/2253303838 >> [v2:172.21.15.22:6800/3166828081,v1:172.21.15.22:6801/3166828081] conn(0x7f0420038f20 msgr2=0x7f042003b3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:54:25.626 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:24.864+0000 7f0442200700 1 --2- 172.21.15.22:0/2253303838 >> [v2:172.21.15.22:6800/3166828081,v1:172.21.15.22:6801/3166828081] conn(0x7f0420038f20 0x7f042003b3e0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f042400ad30 tx=0x7f04240093f0 comp rx=0 tx=0).stop 2024-03-28T17:54:25.626 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:24.864+0000 7f0442200700 1 -- 172.21.15.22:0/2253303838 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f043c19df90 msgr2=0x7f043c19e3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:54:25.626 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:24.865+0000 7f0442200700 1 --2- 172.21.15.22:0/2253303838 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f043c19df90 0x7f043c19e3b0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f042c000c00 tx=0x7f042c011920 comp rx=0 tx=0).stop 2024-03-28T17:54:25.626 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:24.865+0000 7f0438c00700 1 -- 172.21.15.22:0/2253303838 <== mon.0 v2:172.21.15.22:3300/0 8 ==== mgrmap(e 5) v1 ==== 46106+0+0 (secure 0 0 0) 0x7f042c003970 con 0x7f043c19df90 2024-03-28T17:54:25.626 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:24.865+0000 7f0442200700 1 -- 172.21.15.22:0/2253303838 shutdown_connections 2024-03-28T17:54:25.626 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:24.865+0000 7f0442200700 1 --2- 172.21.15.22:0/2253303838 >> [v2:172.21.15.22:6800/3166828081,v1:172.21.15.22:6801/3166828081] conn(0x7f0420038f20 0x7f042003b3e0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:54:25.626 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:24.865+0000 7f0442200700 1 --2- 172.21.15.22:0/2253303838 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f043c19df90 0x7f043c19e3b0 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:54:25.626 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:24.865+0000 7f0442200700 1 -- 172.21.15.22:0/2253303838 >> 172.21.15.22:0/2253303838 conn(0x7f043c102000 msgr2=0x7f043c102f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T17:54:25.626 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:24.865+0000 7f0442200700 1 -- 172.21.15.22:0/2253303838 shutdown_connections 2024-03-28T17:54:25.626 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:24.866+0000 7f0442200700 1 -- 172.21.15.22:0/2253303838 wait complete. 2024-03-28T17:54:25.870 INFO:journalctl@ceph.mgr.x.smithi022.stdout:Mar 28 17:54:25 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mgr-x[32211]: 2024-03-28T17:54:25.697+0000 7fe814147200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-03-28T17:54:25.870 INFO:journalctl@ceph.mgr.x.smithi022.stdout:Mar 28 17:54:25 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mgr-x[32211]: 2024-03-28T17:54:25.824+0000 7fe814147200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-03-28T17:54:26.172 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:54:25 smithi022 ceph-mon[31985]: from='client.? 172.21.15.22:0/2253303838' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "cephadm"}]': finished 2024-03-28T17:54:26.172 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:54:25 smithi022 ceph-mon[31985]: mgrmap e5: x(active, since 5s) 2024-03-28T17:54:26.779 INFO:journalctl@ceph.mgr.x.smithi022.stdout:Mar 28 17:54:26 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mgr-x[32211]: 2024-03-28T17:54:26.653+0000 7fe814147200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-03-28T17:54:26.779 INFO:journalctl@ceph.mgr.x.smithi022.stdout:Mar 28 17:54:26 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mgr-x[32211]: 2024-03-28T17:54:26.778+0000 7fe814147200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-03-28T17:54:27.032 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:54:26 smithi022 ceph-mon[31985]: from='client.? 172.21.15.22:0/488747493' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-03-28T17:54:27.032 INFO:journalctl@ceph.mgr.x.smithi022.stdout:Mar 28 17:54:26 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mgr-x[32211]: 2024-03-28T17:54:26.968+0000 7fe814147200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-03-28T17:54:27.287 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout { 2024-03-28T17:54:27.287 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "epoch": 5, 2024-03-28T17:54:27.287 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "available": true, 2024-03-28T17:54:27.287 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "active_name": "x", 2024-03-28T17:54:27.288 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-03-28T17:54:27.288 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout } 2024-03-28T17:54:27.288 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:26.256+0000 7faee7e00700 1 Processor -- start 2024-03-28T17:54:27.288 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:26.256+0000 7faee7e00700 1 -- start start 2024-03-28T17:54:27.288 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:26.257+0000 7faee7e00700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7faee8109bd0 0x7faee8109ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:54:27.288 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:26.257+0000 7faee7e00700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faee810a5c0 con 0x7faee8109bd0 2024-03-28T17:54:27.288 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:26.257+0000 7faee6a00700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7faee8109bd0 0x7faee8109ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:54:27.288 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:26.257+0000 7faee6a00700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7faee8109bd0 0x7faee8109ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:55704/0 (socket says 172.21.15.22:55704) 2024-03-28T17:54:27.288 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:26.257+0000 7faee6a00700 1 -- 172.21.15.22:0/1705618375 learned_addr learned my addr 172.21.15.22:0/1705618375 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-28T17:54:27.288 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:26.257+0000 7faee6a00700 1 -- 172.21.15.22:0/1705618375 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faee810ad80 con 0x7faee8109bd0 2024-03-28T17:54:27.288 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:26.258+0000 7faee6a00700 1 --2- 172.21.15.22:0/1705618375 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7faee8109bd0 0x7faee8109ff0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7faed80098a0 tx=0x7faed8009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4b18b0e7d7ab2eb1 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T17:54:27.288 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:26.258+0000 7faee5600700 1 -- 172.21.15.22:0/1705618375 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7faed8004160 con 0x7faee8109bd0 2024-03-28T17:54:27.288 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:26.258+0000 7faee5600700 1 -- 172.21.15.22:0/1705618375 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7faed8004300 con 0x7faee8109bd0 2024-03-28T17:54:27.289 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:26.258+0000 7faee5600700 1 -- 172.21.15.22:0/1705618375 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7faed8011b40 con 0x7faee8109bd0 2024-03-28T17:54:27.289 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:26.259+0000 7faee7e00700 1 -- 172.21.15.22:0/1705618375 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7faee8109bd0 msgr2=0x7faee8109ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:54:27.289 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:26.259+0000 7faee7e00700 1 --2- 172.21.15.22:0/1705618375 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7faee8109bd0 0x7faee8109ff0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7faed80098a0 tx=0x7faed8009bb0 comp rx=0 tx=0).stop 2024-03-28T17:54:27.289 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:26.259+0000 7faee7e00700 1 -- 172.21.15.22:0/1705618375 shutdown_connections 2024-03-28T17:54:27.289 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:26.259+0000 7faee7e00700 1 --2- 172.21.15.22:0/1705618375 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7faee8109bd0 0x7faee8109ff0 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:54:27.289 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:26.259+0000 7faee7e00700 1 -- 172.21.15.22:0/1705618375 >> 172.21.15.22:0/1705618375 conn(0x7faee81053b0 msgr2=0x7faee81077a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T17:54:27.289 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:26.259+0000 7faee7e00700 1 -- 172.21.15.22:0/1705618375 shutdown_connections 2024-03-28T17:54:27.289 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:26.259+0000 7faee7e00700 1 -- 172.21.15.22:0/1705618375 wait complete. 2024-03-28T17:54:27.289 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:26.260+0000 7faee7e00700 1 Processor -- start 2024-03-28T17:54:27.289 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:26.260+0000 7faee7e00700 1 -- start start 2024-03-28T17:54:27.289 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:26.261+0000 7faee7e00700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7faee8109bd0 0x7faee819db70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:54:27.289 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:26.261+0000 7faee7e00700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faee810e130 con 0x7faee8109bd0 2024-03-28T17:54:27.289 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:26.261+0000 7faee6a00700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7faee8109bd0 0x7faee819db70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:54:27.289 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:26.261+0000 7faee6a00700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7faee8109bd0 0x7faee819db70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:55718/0 (socket says 172.21.15.22:55718) 2024-03-28T17:54:27.289 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:26.261+0000 7faee6a00700 1 -- 172.21.15.22:0/488747493 learned_addr learned my addr 172.21.15.22:0/488747493 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-28T17:54:27.290 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:26.261+0000 7faee6a00700 1 -- 172.21.15.22:0/488747493 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faee819e0b0 con 0x7faee8109bd0 2024-03-28T17:54:27.290 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:26.262+0000 7faee6a00700 1 --2- 172.21.15.22:0/488747493 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7faee8109bd0 0x7faee819db70 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7faed8011670 tx=0x7faed8011920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T17:54:27.290 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:26.262+0000 7faeeca00700 1 -- 172.21.15.22:0/488747493 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7faed80037d0 con 0x7faee8109bd0 2024-03-28T17:54:27.290 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:26.262+0000 7faeeca00700 1 -- 172.21.15.22:0/488747493 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7faed801a430 con 0x7faee8109bd0 2024-03-28T17:54:27.290 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:26.262+0000 7faee7e00700 1 -- 172.21.15.22:0/488747493 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faee819e340 con 0x7faee8109bd0 2024-03-28T17:54:27.290 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:26.262+0000 7faeeca00700 1 -- 172.21.15.22:0/488747493 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7faed8019550 con 0x7faee8109bd0 2024-03-28T17:54:27.290 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:26.262+0000 7faee7e00700 1 -- 172.21.15.22:0/488747493 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faee81a1030 con 0x7faee8109bd0 2024-03-28T17:54:27.290 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:26.263+0000 7faee7e00700 1 -- 172.21.15.22:0/488747493 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faee80637c0 con 0x7faee8109bd0 2024-03-28T17:54:27.290 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:26.263+0000 7faeeca00700 1 -- 172.21.15.22:0/488747493 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) v1 ==== 46106+0+0 (secure 0 0 0) 0x7faed8022020 con 0x7faee8109bd0 2024-03-28T17:54:27.290 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:26.264+0000 7faeeca00700 1 --2- 172.21.15.22:0/488747493 >> [v2:172.21.15.22:6800/3166828081,v1:172.21.15.22:6801/3166828081] conn(0x7faecc038f20 0x7faecc03b3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:54:27.290 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:26.264+0000 7faeeca00700 1 -- 172.21.15.22:0/488747493 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7faed804caa0 con 0x7faee8109bd0 2024-03-28T17:54:27.290 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:26.264+0000 7faee6000700 1 -- 172.21.15.22:0/488747493 >> [v2:172.21.15.22:6800/3166828081,v1:172.21.15.22:6801/3166828081] conn(0x7faecc038f20 msgr2=0x7faecc03b3e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.22:6800/3166828081 2024-03-28T17:54:27.290 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:26.264+0000 7faee6000700 1 --2- 172.21.15.22:0/488747493 >> [v2:172.21.15.22:6800/3166828081,v1:172.21.15.22:6801/3166828081] conn(0x7faecc038f20 0x7faecc03b3e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-28T17:54:27.291 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:26.269+0000 7faeeca00700 1 -- 172.21.15.22:0/488747493 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7faed804c800 con 0x7faee8109bd0 2024-03-28T17:54:27.291 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:26.465+0000 7faee6000700 1 -- 172.21.15.22:0/488747493 >> [v2:172.21.15.22:6800/3166828081,v1:172.21.15.22:6801/3166828081] conn(0x7faecc038f20 msgr2=0x7faecc03b3e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.22:6800/3166828081 2024-03-28T17:54:27.291 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:26.465+0000 7faee6000700 1 --2- 172.21.15.22:0/488747493 >> [v2:172.21.15.22:6800/3166828081,v1:172.21.15.22:6801/3166828081] conn(0x7faecc038f20 0x7faecc03b3e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-28T17:54:27.291 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:26.550+0000 7faee7e00700 1 -- 172.21.15.22:0/488747493 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "mgr stat"} v 0) v1 -- 0x7faee81a1200 con 0x7faee8109bd0 2024-03-28T17:54:27.291 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:26.550+0000 7faeeca00700 1 -- 172.21.15.22:0/488747493 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v5) v1 ==== 56+0+88 (secure 0 0 0) 0x7faee81a1200 con 0x7faee8109bd0 2024-03-28T17:54:27.291 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:26.553+0000 7faee7e00700 1 -- 172.21.15.22:0/488747493 >> [v2:172.21.15.22:6800/3166828081,v1:172.21.15.22:6801/3166828081] conn(0x7faecc038f20 msgr2=0x7faecc03b3e0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-28T17:54:27.291 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:26.553+0000 7faee7e00700 1 --2- 172.21.15.22:0/488747493 >> [v2:172.21.15.22:6800/3166828081,v1:172.21.15.22:6801/3166828081] conn(0x7faecc038f20 0x7faecc03b3e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:54:27.291 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:26.553+0000 7faee7e00700 1 -- 172.21.15.22:0/488747493 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7faee8109bd0 msgr2=0x7faee819db70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:54:27.291 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:26.553+0000 7faee7e00700 1 --2- 172.21.15.22:0/488747493 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7faee8109bd0 0x7faee819db70 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7faed8011670 tx=0x7faed8011920 comp rx=0 tx=0).stop 2024-03-28T17:54:27.291 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:26.553+0000 7faee7e00700 1 -- 172.21.15.22:0/488747493 shutdown_connections 2024-03-28T17:54:27.291 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:26.553+0000 7faee7e00700 1 --2- 172.21.15.22:0/488747493 >> [v2:172.21.15.22:6800/3166828081,v1:172.21.15.22:6801/3166828081] conn(0x7faecc038f20 0x7faecc03b3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:54:27.291 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:26.553+0000 7faee7e00700 1 --2- 172.21.15.22:0/488747493 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7faee8109bd0 0x7faee819db70 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:54:27.291 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:26.553+0000 7faee7e00700 1 -- 172.21.15.22:0/488747493 >> 172.21.15.22:0/488747493 conn(0x7faee81053b0 msgr2=0x7faee8106ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T17:54:27.291 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:26.553+0000 7faee7e00700 1 -- 172.21.15.22:0/488747493 shutdown_connections 2024-03-28T17:54:27.291 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:26.553+0000 7faee7e00700 1 -- 172.21.15.22:0/488747493 wait complete. 2024-03-28T17:54:27.291 INFO:teuthology.orchestra.run.smithi022.stdout:Waiting for the mgr to restart... 2024-03-28T17:54:27.292 INFO:teuthology.orchestra.run.smithi022.stdout:Waiting for mgr epoch 5... 2024-03-28T17:54:27.442 INFO:journalctl@ceph.mgr.x.smithi022.stdout:Mar 28 17:54:27 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mgr-x[32211]: 2024-03-28T17:54:27.096+0000 7fe814147200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-03-28T17:54:27.442 INFO:journalctl@ceph.mgr.x.smithi022.stdout:Mar 28 17:54:27 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mgr-x[32211]: 2024-03-28T17:54:27.211+0000 7fe814147200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-03-28T17:54:27.698 INFO:journalctl@ceph.mgr.x.smithi022.stdout:Mar 28 17:54:27 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mgr-x[32211]: 2024-03-28T17:54:27.571+0000 7fe814147200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-03-28T17:54:27.969 INFO:journalctl@ceph.mgr.x.smithi022.stdout:Mar 28 17:54:27 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mgr-x[32211]: 2024-03-28T17:54:27.718+0000 7fe814147200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-03-28T17:54:27.969 INFO:journalctl@ceph.mgr.x.smithi022.stdout:Mar 28 17:54:27 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mgr-x[32211]: 2024-03-28T17:54:27.837+0000 7fe814147200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-03-28T17:54:28.281 INFO:journalctl@ceph.mgr.x.smithi022.stdout:Mar 28 17:54:27 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mgr-x[32211]: 2024-03-28T17:54:27.971+0000 7fe814147200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-03-28T17:54:28.282 INFO:journalctl@ceph.mgr.x.smithi022.stdout:Mar 28 17:54:28 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mgr-x[32211]: 2024-03-28T17:54:28.199+0000 7fe814147200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-03-28T17:54:29.694 INFO:journalctl@ceph.mgr.x.smithi022.stdout:Mar 28 17:54:29 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mgr-x[32211]: 2024-03-28T17:54:29.516+0000 7fe814147200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-03-28T17:54:30.031 INFO:journalctl@ceph.mgr.x.smithi022.stdout:Mar 28 17:54:29 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mgr-x[32211]: 2024-03-28T17:54:29.693+0000 7fe814147200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-03-28T17:54:30.512 INFO:journalctl@ceph.mgr.x.smithi022.stdout:Mar 28 17:54:30 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mgr-x[32211]: 2024-03-28T17:54:30.122+0000 7fe814147200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-03-28T17:54:30.512 INFO:journalctl@ceph.mgr.x.smithi022.stdout:Mar 28 17:54:30 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mgr-x[32211]: 2024-03-28T17:54:30.245+0000 7fe814147200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-03-28T17:54:30.766 INFO:journalctl@ceph.mgr.x.smithi022.stdout:Mar 28 17:54:30 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mgr-x[32211]: 2024-03-28T17:54:30.512+0000 7fe814147200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-03-28T17:54:31.032 INFO:journalctl@ceph.mgr.x.smithi022.stdout:Mar 28 17:54:30 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mgr-x[32211]: 2024-03-28T17:54:30.766+0000 7fe814147200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-03-28T17:54:31.479 INFO:journalctl@ceph.mgr.x.smithi022.stdout:Mar 28 17:54:31 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mgr-x[32211]: 2024-03-28T17:54:31.154+0000 7fe814147200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-03-28T17:54:31.479 INFO:journalctl@ceph.mgr.x.smithi022.stdout:Mar 28 17:54:31 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mgr-x[32211]: 2024-03-28T17:54:31.478+0000 7fe814147200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-03-28T17:54:32.282 INFO:journalctl@ceph.mgr.x.smithi022.stdout:Mar 28 17:54:31 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mgr-x[32211]: 2024-03-28T17:54:31.792+0000 7fe814147200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-03-28T17:54:33.282 INFO:journalctl@ceph.mgr.x.smithi022.stdout:Mar 28 17:54:32 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mgr-x[32211]: 2024-03-28T17:54:32.857+0000 7fe814147200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-03-28T17:54:34.000 INFO:journalctl@ceph.mgr.x.smithi022.stdout:Mar 28 17:54:33 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mgr-x[32211]: 2024-03-28T17:54:33.574+0000 7fe814147200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-03-28T17:54:34.000 INFO:journalctl@ceph.mgr.x.smithi022.stdout:Mar 28 17:54:33 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mgr-x[32211]: 2024-03-28T17:54:33.708+0000 7fe814147200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-03-28T17:54:34.282 INFO:journalctl@ceph.mgr.x.smithi022.stdout:Mar 28 17:54:34 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mgr-x[32211]: 2024-03-28T17:54:33.999+0000 7fe814147200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-03-28T17:54:34.282 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:54:34 smithi022 ceph-mon[31985]: Active manager daemon x restarted 2024-03-28T17:54:34.282 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:54:34 smithi022 ceph-mon[31985]: Activating manager daemon x 2024-03-28T17:54:34.282 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:54:34 smithi022 ceph-mon[31985]: osdmap e2: 0 total, 0 up, 0 in 2024-03-28T17:54:34.282 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:54:34 smithi022 ceph-mon[31985]: mgrmap e6: x(active, starting, since 0.010175s) 2024-03-28T17:54:34.282 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:54:34 smithi022 ceph-mon[31985]: from='mgr.14120 172.21.15.22:0/1493984472' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-28T17:54:34.282 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:54:34 smithi022 ceph-mon[31985]: from='mgr.14120 172.21.15.22:0/1493984472' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-03-28T17:54:34.282 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:54:34 smithi022 ceph-mon[31985]: from='mgr.14120 172.21.15.22:0/1493984472' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-03-28T17:54:34.283 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:54:34 smithi022 ceph-mon[31985]: from='mgr.14120 172.21.15.22:0/1493984472' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-03-28T17:54:34.283 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:54:34 smithi022 ceph-mon[31985]: from='mgr.14120 172.21.15.22:0/1493984472' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-03-28T17:54:34.283 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:54:34 smithi022 ceph-mon[31985]: Manager daemon x is now available 2024-03-28T17:54:34.283 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:54:34 smithi022 ceph-mon[31985]: from='mgr.14120 172.21.15.22:0/1493984472' entity='mgr.x' 2024-03-28T17:54:35.282 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:54:35 smithi022 ceph-mon[31985]: Found migration_current of "None". Setting to last migration. 2024-03-28T17:54:35.282 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:54:35 smithi022 ceph-mon[31985]: from='mgr.14120 172.21.15.22:0/1493984472' entity='mgr.x' 2024-03-28T17:54:35.282 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:54:35 smithi022 ceph-mon[31985]: from='mgr.14120 172.21.15.22:0/1493984472' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T17:54:35.283 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:54:35 smithi022 ceph-mon[31985]: from='mgr.14120 172.21.15.22:0/1493984472' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T17:54:35.283 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:54:35 smithi022 ceph-mon[31985]: from='mgr.14120 172.21.15.22:0/1493984472' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T17:54:35.283 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:54:35 smithi022 ceph-mon[31985]: from='mgr.14120 172.21.15.22:0/1493984472' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-03-28T17:54:35.283 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:54:35 smithi022 ceph-mon[31985]: from='mgr.14120 172.21.15.22:0/1493984472' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-03-28T17:54:35.283 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:54:35 smithi022 ceph-mon[31985]: from='mgr.14120 172.21.15.22:0/1493984472' entity='mgr.x' 2024-03-28T17:54:35.283 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:54:35 smithi022 ceph-mon[31985]: from='mgr.14120 172.21.15.22:0/1493984472' entity='mgr.x' 2024-03-28T17:54:35.283 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:54:35 smithi022 ceph-mon[31985]: [28/Mar/2024:17:54:34] ENGINE Bus STARTING 2024-03-28T17:54:35.283 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:54:35 smithi022 ceph-mon[31985]: mgrmap e7: x(active, since 1.01173s) 2024-03-28T17:54:35.725 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout { 2024-03-28T17:54:35.726 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 7, 2024-03-28T17:54:35.726 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "initialized": true 2024-03-28T17:54:35.726 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout } 2024-03-28T17:54:35.726 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:27.962+0000 7fcdbe400700 1 Processor -- start 2024-03-28T17:54:35.727 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:27.962+0000 7fcdbe400700 1 -- start start 2024-03-28T17:54:35.727 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:27.963+0000 7fcdbe400700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fcdb81083f0 0x7fcdb810a7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:54:35.727 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:27.963+0000 7fcdbe400700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcdb8075930 con 0x7fcdb81083f0 2024-03-28T17:54:35.727 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:27.963+0000 7fcdbd000700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fcdb81083f0 0x7fcdb810a7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:54:35.727 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:27.964+0000 7fcdbd000700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fcdb81083f0 0x7fcdb810a7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:57922/0 (socket says 172.21.15.22:57922) 2024-03-28T17:54:35.727 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:27.964+0000 7fcdbd000700 1 -- 172.21.15.22:0/323543412 learned_addr learned my addr 172.21.15.22:0/323543412 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-28T17:54:35.727 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:27.964+0000 7fcdbd000700 1 -- 172.21.15.22:0/323543412 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcdb810ad20 con 0x7fcdb81083f0 2024-03-28T17:54:35.727 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:27.964+0000 7fcdbd000700 1 --2- 172.21.15.22:0/323543412 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fcdb81083f0 0x7fcdb810a7e0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fcdac0098a0 tx=0x7fcdac009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=222f47f4937d908a server_cookie=0 in_seq=0 out_seq=0 2024-03-28T17:54:35.727 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:27.965+0000 7fcdb7400700 1 -- 172.21.15.22:0/323543412 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fcdac004160 con 0x7fcdb81083f0 2024-03-28T17:54:35.729 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:27.965+0000 7fcdb7400700 1 -- 172.21.15.22:0/323543412 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fcdac004300 con 0x7fcdb81083f0 2024-03-28T17:54:35.729 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:27.965+0000 7fcdb7400700 1 -- 172.21.15.22:0/323543412 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fcdac011b40 con 0x7fcdb81083f0 2024-03-28T17:54:35.729 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:27.966+0000 7fcdbe400700 1 -- 172.21.15.22:0/323543412 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fcdb81083f0 msgr2=0x7fcdb810a7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:54:35.730 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:27.966+0000 7fcdbe400700 1 --2- 172.21.15.22:0/323543412 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fcdb81083f0 0x7fcdb810a7e0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fcdac0098a0 tx=0x7fcdac009bb0 comp rx=0 tx=0).stop 2024-03-28T17:54:35.730 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:27.966+0000 7fcdbe400700 1 -- 172.21.15.22:0/323543412 shutdown_connections 2024-03-28T17:54:35.730 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:27.966+0000 7fcdbe400700 1 --2- 172.21.15.22:0/323543412 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fcdb81083f0 0x7fcdb810a7e0 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:54:35.730 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:27.966+0000 7fcdbe400700 1 -- 172.21.15.22:0/323543412 >> 172.21.15.22:0/323543412 conn(0x7fcdb8101f00 msgr2=0x7fcdb8104320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T17:54:35.730 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:27.966+0000 7fcdbe400700 1 -- 172.21.15.22:0/323543412 shutdown_connections 2024-03-28T17:54:35.730 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:27.966+0000 7fcdbe400700 1 -- 172.21.15.22:0/323543412 wait complete. 2024-03-28T17:54:35.730 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:27.967+0000 7fcdbe400700 1 Processor -- start 2024-03-28T17:54:35.730 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:27.967+0000 7fcdbe400700 1 -- start start 2024-03-28T17:54:35.730 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:27.968+0000 7fcdbe400700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fcdb81083f0 0x7fcdb81948d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:54:35.730 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:27.968+0000 7fcdbe400700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcdb810bd40 con 0x7fcdb81083f0 2024-03-28T17:54:35.730 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:27.968+0000 7fcdbd000700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fcdb81083f0 0x7fcdb81948d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:54:35.730 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:27.968+0000 7fcdbd000700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fcdb81083f0 0x7fcdb81948d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:57934/0 (socket says 172.21.15.22:57934) 2024-03-28T17:54:35.730 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:27.968+0000 7fcdbd000700 1 -- 172.21.15.22:0/575294722 learned_addr learned my addr 172.21.15.22:0/575294722 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-28T17:54:35.733 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:27.968+0000 7fcdbd000700 1 -- 172.21.15.22:0/575294722 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcdb8194e10 con 0x7fcdb81083f0 2024-03-28T17:54:35.733 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:27.969+0000 7fcdbd000700 1 --2- 172.21.15.22:0/575294722 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fcdb81083f0 0x7fcdb81948d0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fcdac004180 tx=0x7fcdac0046a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T17:54:35.733 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:27.969+0000 7fcdb5600700 1 -- 172.21.15.22:0/575294722 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fcdac0039e0 con 0x7fcdb81083f0 2024-03-28T17:54:35.733 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:27.969+0000 7fcdb5600700 1 -- 172.21.15.22:0/575294722 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fcdac020070 con 0x7fcdb81083f0 2024-03-28T17:54:35.733 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:27.969+0000 7fcdb5600700 1 -- 172.21.15.22:0/575294722 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fcdac0193c0 con 0x7fcdb81083f0 2024-03-28T17:54:35.733 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:27.969+0000 7fcdbe400700 1 -- 172.21.15.22:0/575294722 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcdb81950a0 con 0x7fcdb81083f0 2024-03-28T17:54:35.733 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:27.970+0000 7fcdbe400700 1 -- 172.21.15.22:0/575294722 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcdb8191660 con 0x7fcdb81083f0 2024-03-28T17:54:35.733 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:27.971+0000 7fcdb5600700 1 -- 172.21.15.22:0/575294722 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) v1 ==== 46106+0+0 (secure 0 0 0) 0x7fcdac01a430 con 0x7fcdb81083f0 2024-03-28T17:54:35.734 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:27.972+0000 7fcdb5600700 1 --2- 172.21.15.22:0/575294722 >> [v2:172.21.15.22:6800/3166828081,v1:172.21.15.22:6801/3166828081] conn(0x7fcda0038f70 0x7fcda003b430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:54:35.734 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:27.972+0000 7fcdb5600700 1 -- 172.21.15.22:0/575294722 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7fcdac04cfd0 con 0x7fcdb81083f0 2024-03-28T17:54:35.734 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:27.972+0000 7fcdbe400700 1 -- 172.21.15.22:0/575294722 --> [v2:172.21.15.22:6800/3166828081,v1:172.21.15.22:6801/3166828081] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x7fcdb80637c0 con 0x7fcda0038f70 2024-03-28T17:54:35.734 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:27.972+0000 7fcdb7e00700 1 -- 172.21.15.22:0/575294722 >> [v2:172.21.15.22:6800/3166828081,v1:172.21.15.22:6801/3166828081] conn(0x7fcda0038f70 msgr2=0x7fcda003b430 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.22:6800/3166828081 2024-03-28T17:54:35.734 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:27.972+0000 7fcdb7e00700 1 --2- 172.21.15.22:0/575294722 >> [v2:172.21.15.22:6800/3166828081,v1:172.21.15.22:6801/3166828081] conn(0x7fcda0038f70 0x7fcda003b430 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-28T17:54:35.734 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:28.173+0000 7fcdb7e00700 1 -- 172.21.15.22:0/575294722 >> [v2:172.21.15.22:6800/3166828081,v1:172.21.15.22:6801/3166828081] conn(0x7fcda0038f70 msgr2=0x7fcda003b430 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.22:6800/3166828081 2024-03-28T17:54:35.734 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:28.173+0000 7fcdb7e00700 1 --2- 172.21.15.22:0/575294722 >> [v2:172.21.15.22:6800/3166828081,v1:172.21.15.22:6801/3166828081] conn(0x7fcda0038f70 0x7fcda003b430 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-28T17:54:35.734 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:28.574+0000 7fcdb7e00700 1 -- 172.21.15.22:0/575294722 >> [v2:172.21.15.22:6800/3166828081,v1:172.21.15.22:6801/3166828081] conn(0x7fcda0038f70 msgr2=0x7fcda003b430 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.22:6800/3166828081 2024-03-28T17:54:35.734 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:28.574+0000 7fcdb7e00700 1 --2- 172.21.15.22:0/575294722 >> [v2:172.21.15.22:6800/3166828081,v1:172.21.15.22:6801/3166828081] conn(0x7fcda0038f70 0x7fcda003b430 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-28T17:54:35.734 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:29.375+0000 7fcdb7e00700 1 -- 172.21.15.22:0/575294722 >> [v2:172.21.15.22:6800/3166828081,v1:172.21.15.22:6801/3166828081] conn(0x7fcda0038f70 msgr2=0x7fcda003b430 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.22:6800/3166828081 2024-03-28T17:54:35.734 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:29.375+0000 7fcdb7e00700 1 --2- 172.21.15.22:0/575294722 >> [v2:172.21.15.22:6800/3166828081,v1:172.21.15.22:6801/3166828081] conn(0x7fcda0038f70 0x7fcda003b430 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-28T17:54:35.734 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:30.977+0000 7fcdb7e00700 1 -- 172.21.15.22:0/575294722 >> [v2:172.21.15.22:6800/3166828081,v1:172.21.15.22:6801/3166828081] conn(0x7fcda0038f70 msgr2=0x7fcda003b430 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.22:6800/3166828081 2024-03-28T17:54:35.734 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:30.977+0000 7fcdb7e00700 1 --2- 172.21.15.22:0/575294722 >> [v2:172.21.15.22:6800/3166828081,v1:172.21.15.22:6801/3166828081] conn(0x7fcda0038f70 0x7fcda003b430 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2024-03-28T17:54:35.735 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:34.011+0000 7fcdb5600700 1 -- 172.21.15.22:0/575294722 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mgrmap(e 6) v1 ==== 45873+0+0 (secure 0 0 0) 0x7fcdac02bd00 con 0x7fcdb81083f0 2024-03-28T17:54:35.735 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:34.011+0000 7fcdb5600700 1 -- 172.21.15.22:0/575294722 >> [v2:172.21.15.22:6800/3166828081,v1:172.21.15.22:6801/3166828081] conn(0x7fcda0038f70 msgr2=0x7fcda003b430 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-28T17:54:35.735 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:34.011+0000 7fcdb5600700 1 --2- 172.21.15.22:0/575294722 >> [v2:172.21.15.22:6800/3166828081,v1:172.21.15.22:6801/3166828081] conn(0x7fcda0038f70 0x7fcda003b430 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:54:35.735 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:35.012+0000 7fcdb5600700 1 -- 172.21.15.22:0/575294722 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mgrmap(e 7) v1 ==== 46000+0+0 (secure 0 0 0) 0x7fcdac022d50 con 0x7fcdb81083f0 2024-03-28T17:54:35.735 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:35.013+0000 7fcdb5600700 1 --2- 172.21.15.22:0/575294722 >> [v2:172.21.15.22:6800/730635939,v1:172.21.15.22:6801/730635939] conn(0x7fcda003c670 0x7fcda003ea60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:54:35.737 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:35.013+0000 7fcdb5600700 1 -- 172.21.15.22:0/575294722 --> [v2:172.21.15.22:6800/730635939,v1:172.21.15.22:6801/730635939] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x7fcdb80637c0 con 0x7fcda003c670 2024-03-28T17:54:35.737 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:35.014+0000 7fcdb7e00700 1 --2- 172.21.15.22:0/575294722 >> [v2:172.21.15.22:6800/730635939,v1:172.21.15.22:6801/730635939] conn(0x7fcda003c670 0x7fcda003ea60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:54:35.737 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:35.016+0000 7fcdb7e00700 1 --2- 172.21.15.22:0/575294722 >> [v2:172.21.15.22:6800/730635939,v1:172.21.15.22:6801/730635939] conn(0x7fcda003c670 0x7fcda003ea60 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7fcda4003a10 tx=0x7fcda40092b0 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T17:54:35.737 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:35.018+0000 7fcdb5600700 1 -- 172.21.15.22:0/575294722 <== mgr.14120 v2:172.21.15.22:6800/730635939 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+7774 (secure 0 0 0) 0x7fcdb80637c0 con 0x7fcda003c670 2024-03-28T17:54:35.737 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:35.025+0000 7fcdbe400700 1 -- 172.21.15.22:0/575294722 --> [v2:172.21.15.22:6800/730635939,v1:172.21.15.22:6801/730635939] -- command(tid 1: {"prefix": "mgr_status"}) v1 -- 0x7fcdb8195270 con 0x7fcda003c670 2024-03-28T17:54:35.737 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:35.026+0000 7fcdb5600700 1 -- 172.21.15.22:0/575294722 <== mgr.14120 v2:172.21.15.22:6800/730635939 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+51 (secure 0 0 0) 0x7fcdb8195270 con 0x7fcda003c670 2024-03-28T17:54:35.737 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:35.026+0000 7fcdbe400700 1 -- 172.21.15.22:0/575294722 >> [v2:172.21.15.22:6800/730635939,v1:172.21.15.22:6801/730635939] conn(0x7fcda003c670 msgr2=0x7fcda003ea60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:54:35.737 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:35.026+0000 7fcdbe400700 1 --2- 172.21.15.22:0/575294722 >> [v2:172.21.15.22:6800/730635939,v1:172.21.15.22:6801/730635939] conn(0x7fcda003c670 0x7fcda003ea60 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7fcda4003a10 tx=0x7fcda40092b0 comp rx=0 tx=0).stop 2024-03-28T17:54:35.737 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:35.026+0000 7fcdbe400700 1 -- 172.21.15.22:0/575294722 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fcdb81083f0 msgr2=0x7fcdb81948d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:54:35.737 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:35.026+0000 7fcdbe400700 1 --2- 172.21.15.22:0/575294722 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fcdb81083f0 0x7fcdb81948d0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fcdac004180 tx=0x7fcdac0046a0 comp rx=0 tx=0).stop 2024-03-28T17:54:35.737 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:35.027+0000 7fcdbe400700 1 -- 172.21.15.22:0/575294722 shutdown_connections 2024-03-28T17:54:35.738 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:35.027+0000 7fcdbe400700 1 --2- 172.21.15.22:0/575294722 >> [v2:172.21.15.22:6800/730635939,v1:172.21.15.22:6801/730635939] conn(0x7fcda003c670 0x7fcda003ea60 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:54:35.738 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:35.027+0000 7fcdbe400700 1 --2- 172.21.15.22:0/575294722 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fcdb81083f0 0x7fcdb81948d0 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:54:35.738 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:35.027+0000 7fcdbe400700 1 --2- 172.21.15.22:0/575294722 >> [v2:172.21.15.22:6800/3166828081,v1:172.21.15.22:6801/3166828081] conn(0x7fcda0038f70 0x7fcda003b430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:54:35.738 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:35.027+0000 7fcdbe400700 1 -- 172.21.15.22:0/575294722 >> 172.21.15.22:0/575294722 conn(0x7fcdb8101f00 msgr2=0x7fcdb8102a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T17:54:35.738 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:35.027+0000 7fcdbe400700 1 -- 172.21.15.22:0/575294722 shutdown_connections 2024-03-28T17:54:35.738 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:35.027+0000 7fcdbe400700 1 -- 172.21.15.22:0/575294722 wait complete. 2024-03-28T17:54:35.738 INFO:teuthology.orchestra.run.smithi022.stdout:mgr epoch 5 is available 2024-03-28T17:54:35.738 INFO:teuthology.orchestra.run.smithi022.stdout:Setting orchestrator backend to cephadm... 2024-03-28T17:54:36.345 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:54:36 smithi022 ceph-mon[31985]: from='client.14124 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2024-03-28T17:54:36.345 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:54:36 smithi022 ceph-mon[31985]: from='client.14124 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2024-03-28T17:54:36.345 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:54:36 smithi022 ceph-mon[31985]: [28/Mar/2024:17:54:35] ENGINE Serving on http://172.21.15.22:8765 2024-03-28T17:54:36.345 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:54:36 smithi022 ceph-mon[31985]: [28/Mar/2024:17:54:35] ENGINE Serving on https://172.21.15.22:7150 2024-03-28T17:54:36.345 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:54:36 smithi022 ceph-mon[31985]: [28/Mar/2024:17:54:35] ENGINE Bus STARTED 2024-03-28T17:54:36.345 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:54:36 smithi022 ceph-mon[31985]: from='mgr.14120 172.21.15.22:0/1493984472' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T17:54:37.228 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:54:37 smithi022 ceph-mon[31985]: mgrmap e8: x(active, since 2s) 2024-03-28T17:54:37.228 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:54:37 smithi022 ceph-mon[31985]: from='client.14132 -' entity='client.admin' cmd=[{"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}]: dispatch 2024-03-28T17:54:37.228 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:54:37 smithi022 ceph-mon[31985]: from='mgr.14120 172.21.15.22:0/1493984472' entity='mgr.x' 2024-03-28T17:54:37.228 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:54:37 smithi022 ceph-mon[31985]: from='mgr.14120 172.21.15.22:0/1493984472' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T17:54:37.504 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:36.601+0000 7fed1f200700 1 Processor -- start 2024-03-28T17:54:37.504 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:36.601+0000 7fed1f200700 1 -- start start 2024-03-28T17:54:37.504 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:36.601+0000 7fed1f200700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fed20109bc0 0x7fed20109fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:54:37.505 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:36.601+0000 7fed1f200700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed2010a5b0 con 0x7fed20109bc0 2024-03-28T17:54:37.505 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:36.602+0000 7fed1de00700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fed20109bc0 0x7fed20109fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:54:37.507 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:36.602+0000 7fed1de00700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fed20109bc0 0x7fed20109fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:57990/0 (socket says 172.21.15.22:57990) 2024-03-28T17:54:37.507 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:36.602+0000 7fed1de00700 1 -- 172.21.15.22:0/1357421930 learned_addr learned my addr 172.21.15.22:0/1357421930 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-28T17:54:37.507 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:36.602+0000 7fed1de00700 1 -- 172.21.15.22:0/1357421930 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fed2010ad70 con 0x7fed20109bc0 2024-03-28T17:54:37.507 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:36.602+0000 7fed1de00700 1 --2- 172.21.15.22:0/1357421930 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fed20109bc0 0x7fed20109fe0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fed100098a0 tx=0x7fed10009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a4363243336585ff server_cookie=0 in_seq=0 out_seq=0 2024-03-28T17:54:37.507 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:36.603+0000 7fed1ca00700 1 -- 172.21.15.22:0/1357421930 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fed10004160 con 0x7fed20109bc0 2024-03-28T17:54:37.507 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:36.603+0000 7fed1ca00700 1 -- 172.21.15.22:0/1357421930 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fed10004300 con 0x7fed20109bc0 2024-03-28T17:54:37.507 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:36.603+0000 7fed1ca00700 1 -- 172.21.15.22:0/1357421930 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fed10011b40 con 0x7fed20109bc0 2024-03-28T17:54:37.507 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:36.603+0000 7fed1f200700 1 -- 172.21.15.22:0/1357421930 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fed20109bc0 msgr2=0x7fed20109fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:54:37.507 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:36.603+0000 7fed1f200700 1 --2- 172.21.15.22:0/1357421930 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fed20109bc0 0x7fed20109fe0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fed100098a0 tx=0x7fed10009bb0 comp rx=0 tx=0).stop 2024-03-28T17:54:37.508 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:36.603+0000 7fed1f200700 1 -- 172.21.15.22:0/1357421930 shutdown_connections 2024-03-28T17:54:37.508 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:36.603+0000 7fed1f200700 1 --2- 172.21.15.22:0/1357421930 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fed20109bc0 0x7fed20109fe0 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:54:37.508 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:36.603+0000 7fed1f200700 1 -- 172.21.15.22:0/1357421930 >> 172.21.15.22:0/1357421930 conn(0x7fed2007cfe0 msgr2=0x7fed2007d410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T17:54:37.508 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:36.604+0000 7fed1f200700 1 -- 172.21.15.22:0/1357421930 shutdown_connections 2024-03-28T17:54:37.508 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:36.604+0000 7fed1f200700 1 -- 172.21.15.22:0/1357421930 wait complete. 2024-03-28T17:54:37.508 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:36.604+0000 7fed1f200700 1 Processor -- start 2024-03-28T17:54:37.508 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:36.605+0000 7fed1f200700 1 -- start start 2024-03-28T17:54:37.508 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:36.605+0000 7fed1f200700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fed20109bc0 0x7fed2019db90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:54:37.508 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:36.605+0000 7fed1f200700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed2010e120 con 0x7fed20109bc0 2024-03-28T17:54:37.511 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:36.605+0000 7fed1de00700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fed20109bc0 0x7fed2019db90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:54:37.511 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:36.605+0000 7fed1de00700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fed20109bc0 0x7fed2019db90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:58004/0 (socket says 172.21.15.22:58004) 2024-03-28T17:54:37.511 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:36.605+0000 7fed1de00700 1 -- 172.21.15.22:0/464287296 learned_addr learned my addr 172.21.15.22:0/464287296 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-28T17:54:37.511 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:36.606+0000 7fed1de00700 1 -- 172.21.15.22:0/464287296 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fed2019e0d0 con 0x7fed20109bc0 2024-03-28T17:54:37.511 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:36.606+0000 7fed1de00700 1 --2- 172.21.15.22:0/464287296 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fed20109bc0 0x7fed2019db90 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fed10011670 tx=0x7fed10011b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T17:54:37.511 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:36.606+0000 7fed16a00700 1 -- 172.21.15.22:0/464287296 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fed100038b0 con 0x7fed20109bc0 2024-03-28T17:54:37.512 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:36.606+0000 7fed16a00700 1 -- 172.21.15.22:0/464287296 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fed1001a430 con 0x7fed20109bc0 2024-03-28T17:54:37.512 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:36.606+0000 7fed1f200700 1 -- 172.21.15.22:0/464287296 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fed2019e360 con 0x7fed20109bc0 2024-03-28T17:54:37.512 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:36.606+0000 7fed16a00700 1 -- 172.21.15.22:0/464287296 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fed10019550 con 0x7fed20109bc0 2024-03-28T17:54:37.512 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:36.606+0000 7fed1f200700 1 -- 172.21.15.22:0/464287296 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fed201a1050 con 0x7fed20109bc0 2024-03-28T17:54:37.512 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:36.607+0000 7fed1f200700 1 -- 172.21.15.22:0/464287296 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fed200637c0 con 0x7fed20109bc0 2024-03-28T17:54:37.512 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:36.607+0000 7fed16a00700 1 -- 172.21.15.22:0/464287296 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 8) v1 ==== 46106+0+0 (secure 0 0 0) 0x7fed1001aa50 con 0x7fed20109bc0 2024-03-28T17:54:37.512 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:36.608+0000 7fed16a00700 1 --2- 172.21.15.22:0/464287296 >> [v2:172.21.15.22:6800/730635939,v1:172.21.15.22:6801/730635939] conn(0x7fed04038f20 0x7fed0403b3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:54:37.512 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:36.608+0000 7fed16a00700 1 -- 172.21.15.22:0/464287296 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7fed1004d0d0 con 0x7fed20109bc0 2024-03-28T17:54:37.512 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:36.608+0000 7fed1d400700 1 --2- 172.21.15.22:0/464287296 >> [v2:172.21.15.22:6800/730635939,v1:172.21.15.22:6801/730635939] conn(0x7fed04038f20 0x7fed0403b3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:54:37.512 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:36.609+0000 7fed1d400700 1 --2- 172.21.15.22:0/464287296 >> [v2:172.21.15.22:6800/730635939,v1:172.21.15.22:6801/730635939] conn(0x7fed04038f20 0x7fed0403b3e0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fed0c006fd0 tx=0x7fed0c006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T17:54:37.512 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:36.613+0000 7fed16a00700 1 -- 172.21.15.22:0/464287296 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fed10023e50 con 0x7fed20109bc0 2024-03-28T17:54:37.512 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:36.856+0000 7fed1f200700 1 -- 172.21.15.22:0/464287296 --> [v2:172.21.15.22:6800/730635939,v1:172.21.15.22:6801/730635939] -- mgr_command(tid 0: {"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}) v1 -- 0x7fed20107ca0 con 0x7fed04038f20 2024-03-28T17:54:37.512 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:36.868+0000 7fed16a00700 1 -- 172.21.15.22:0/464287296 <== mgr.14120 v2:172.21.15.22:6800/730635939 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7fed20107ca0 con 0x7fed04038f20 2024-03-28T17:54:37.512 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:36.871+0000 7fed1f200700 1 -- 172.21.15.22:0/464287296 >> [v2:172.21.15.22:6800/730635939,v1:172.21.15.22:6801/730635939] conn(0x7fed04038f20 msgr2=0x7fed0403b3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:54:37.512 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:36.871+0000 7fed1f200700 1 --2- 172.21.15.22:0/464287296 >> [v2:172.21.15.22:6800/730635939,v1:172.21.15.22:6801/730635939] conn(0x7fed04038f20 0x7fed0403b3e0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fed0c006fd0 tx=0x7fed0c006e40 comp rx=0 tx=0).stop 2024-03-28T17:54:37.513 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:36.871+0000 7fed1f200700 1 -- 172.21.15.22:0/464287296 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fed20109bc0 msgr2=0x7fed2019db90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:54:37.513 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:36.871+0000 7fed1f200700 1 --2- 172.21.15.22:0/464287296 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fed20109bc0 0x7fed2019db90 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fed10011670 tx=0x7fed10011b50 comp rx=0 tx=0).stop 2024-03-28T17:54:37.513 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:36.871+0000 7fed1f200700 1 -- 172.21.15.22:0/464287296 shutdown_connections 2024-03-28T17:54:37.513 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:36.871+0000 7fed1f200700 1 --2- 172.21.15.22:0/464287296 >> [v2:172.21.15.22:6800/730635939,v1:172.21.15.22:6801/730635939] conn(0x7fed04038f20 0x7fed0403b3e0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:54:37.513 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:36.871+0000 7fed1f200700 1 --2- 172.21.15.22:0/464287296 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fed20109bc0 0x7fed2019db90 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:54:37.513 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:36.871+0000 7fed1f200700 1 -- 172.21.15.22:0/464287296 >> 172.21.15.22:0/464287296 conn(0x7fed2007cfe0 msgr2=0x7fed20107590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T17:54:37.513 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:36.872+0000 7fed1f200700 1 -- 172.21.15.22:0/464287296 shutdown_connections 2024-03-28T17:54:37.513 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:36.872+0000 7fed1f200700 1 -- 172.21.15.22:0/464287296 wait complete. 2024-03-28T17:54:39.107 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout value unchanged 2024-03-28T17:54:39.107 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:38.122+0000 7fb64ce00700 1 Processor -- start 2024-03-28T17:54:39.107 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:38.122+0000 7fb64ce00700 1 -- start start 2024-03-28T17:54:39.107 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:38.123+0000 7fb64ce00700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb64807cd70 0x7fb64807b1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:54:39.107 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:38.123+0000 7fb64ce00700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb64807b7a0 con 0x7fb64807cd70 2024-03-28T17:54:39.107 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:38.123+0000 7fb645e00700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb64807cd70 0x7fb64807b1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:54:39.107 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:38.123+0000 7fb645e00700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb64807cd70 0x7fb64807b1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:45830/0 (socket says 172.21.15.22:45830) 2024-03-28T17:54:39.107 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:38.123+0000 7fb645e00700 1 -- 172.21.15.22:0/95641560 learned_addr learned my addr 172.21.15.22:0/95641560 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-28T17:54:39.108 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:38.124+0000 7fb645e00700 1 -- 172.21.15.22:0/95641560 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb64807b920 con 0x7fb64807cd70 2024-03-28T17:54:39.108 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:38.124+0000 7fb645e00700 1 --2- 172.21.15.22:0/95641560 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb64807cd70 0x7fb64807b1d0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fb6380098a0 tx=0x7fb638009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4521e127d203996d server_cookie=0 in_seq=0 out_seq=0 2024-03-28T17:54:39.108 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:38.125+0000 7fb644a00700 1 -- 172.21.15.22:0/95641560 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fb638004160 con 0x7fb64807cd70 2024-03-28T17:54:39.108 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:38.125+0000 7fb644a00700 1 -- 172.21.15.22:0/95641560 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb638004300 con 0x7fb64807cd70 2024-03-28T17:54:39.108 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:38.125+0000 7fb644a00700 1 -- 172.21.15.22:0/95641560 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fb638011b40 con 0x7fb64807cd70 2024-03-28T17:54:39.108 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:38.125+0000 7fb64ce00700 1 -- 172.21.15.22:0/95641560 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb64807cd70 msgr2=0x7fb64807b1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:54:39.108 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:38.125+0000 7fb64ce00700 1 --2- 172.21.15.22:0/95641560 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb64807cd70 0x7fb64807b1d0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fb6380098a0 tx=0x7fb638009bb0 comp rx=0 tx=0).stop 2024-03-28T17:54:39.108 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:38.126+0000 7fb64ce00700 1 -- 172.21.15.22:0/95641560 shutdown_connections 2024-03-28T17:54:39.108 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:38.126+0000 7fb64ce00700 1 --2- 172.21.15.22:0/95641560 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb64807cd70 0x7fb64807b1d0 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:54:39.108 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:38.126+0000 7fb64ce00700 1 -- 172.21.15.22:0/95641560 >> 172.21.15.22:0/95641560 conn(0x7fb648103100 msgr2=0x7fb648105560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T17:54:39.108 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:38.126+0000 7fb64ce00700 1 -- 172.21.15.22:0/95641560 shutdown_connections 2024-03-28T17:54:39.108 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:38.126+0000 7fb64ce00700 1 -- 172.21.15.22:0/95641560 wait complete. 2024-03-28T17:54:39.108 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:38.127+0000 7fb64ce00700 1 Processor -- start 2024-03-28T17:54:39.108 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:38.128+0000 7fb64ce00700 1 -- start start 2024-03-28T17:54:39.109 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:38.128+0000 7fb64ce00700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb64807cd70 0x7fb64819dd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:54:39.109 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:38.128+0000 7fb64ce00700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb64810e1b0 con 0x7fb64807cd70 2024-03-28T17:54:39.109 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:38.128+0000 7fb645e00700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb64807cd70 0x7fb64819dd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:54:39.109 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:38.128+0000 7fb645e00700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb64807cd70 0x7fb64819dd40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:45838/0 (socket says 172.21.15.22:45838) 2024-03-28T17:54:39.109 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:38.128+0000 7fb645e00700 1 -- 172.21.15.22:0/2106700769 learned_addr learned my addr 172.21.15.22:0/2106700769 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-28T17:54:39.109 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:38.129+0000 7fb645e00700 1 -- 172.21.15.22:0/2106700769 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb64819e280 con 0x7fb64807cd70 2024-03-28T17:54:39.110 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:38.129+0000 7fb645e00700 1 --2- 172.21.15.22:0/2106700769 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb64807cd70 0x7fb64819dd40 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fb638004180 tx=0x7fb6380046a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T17:54:39.110 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:38.129+0000 7fb63ea00700 1 -- 172.21.15.22:0/2106700769 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fb6380039e0 con 0x7fb64807cd70 2024-03-28T17:54:39.110 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:38.129+0000 7fb63ea00700 1 -- 172.21.15.22:0/2106700769 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb638020070 con 0x7fb64807cd70 2024-03-28T17:54:39.110 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:38.129+0000 7fb63ea00700 1 -- 172.21.15.22:0/2106700769 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fb6380193c0 con 0x7fb64807cd70 2024-03-28T17:54:39.110 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:38.129+0000 7fb64ce00700 1 -- 172.21.15.22:0/2106700769 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb64819e510 con 0x7fb64807cd70 2024-03-28T17:54:39.110 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:38.129+0000 7fb64ce00700 1 -- 172.21.15.22:0/2106700769 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb6481a1200 con 0x7fb64807cd70 2024-03-28T17:54:39.110 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:38.131+0000 7fb64ce00700 1 -- 172.21.15.22:0/2106700769 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb6480637c0 con 0x7fb64807cd70 2024-03-28T17:54:39.110 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:38.131+0000 7fb63ea00700 1 -- 172.21.15.22:0/2106700769 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 8) v1 ==== 46106+0+0 (secure 0 0 0) 0x7fb63801a430 con 0x7fb64807cd70 2024-03-28T17:54:39.110 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:38.131+0000 7fb63ea00700 1 --2- 172.21.15.22:0/2106700769 >> [v2:172.21.15.22:6800/730635939,v1:172.21.15.22:6801/730635939] conn(0x7fb62c038f20 0x7fb62c03b3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:54:39.110 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:38.131+0000 7fb63ea00700 1 -- 172.21.15.22:0/2106700769 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7fb63804cfe0 con 0x7fb64807cd70 2024-03-28T17:54:39.111 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:38.132+0000 7fb645400700 1 --2- 172.21.15.22:0/2106700769 >> [v2:172.21.15.22:6800/730635939,v1:172.21.15.22:6801/730635939] conn(0x7fb62c038f20 0x7fb62c03b3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:54:39.111 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:38.132+0000 7fb645400700 1 --2- 172.21.15.22:0/2106700769 >> [v2:172.21.15.22:6800/730635939,v1:172.21.15.22:6801/730635939] conn(0x7fb62c038f20 0x7fb62c03b3e0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fb634006fd0 tx=0x7fb634006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T17:54:39.111 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:38.137+0000 7fb63ea00700 1 -- 172.21.15.22:0/2106700769 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fb638021270 con 0x7fb64807cd70 2024-03-28T17:54:39.111 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:38.363+0000 7fb64ce00700 1 -- 172.21.15.22:0/2106700769 --> [v2:172.21.15.22:6800/730635939,v1:172.21.15.22:6801/730635939] -- mgr_command(tid 0: {"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}) v1 -- 0x7fb6481043c0 con 0x7fb62c038f20 2024-03-28T17:54:39.111 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:38.365+0000 7fb63ea00700 1 -- 172.21.15.22:0/2106700769 <== mgr.14120 v2:172.21.15.22:6800/730635939 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+16 (secure 0 0 0) 0x7fb6481043c0 con 0x7fb62c038f20 2024-03-28T17:54:39.111 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:38.368+0000 7fb64ce00700 1 -- 172.21.15.22:0/2106700769 >> [v2:172.21.15.22:6800/730635939,v1:172.21.15.22:6801/730635939] conn(0x7fb62c038f20 msgr2=0x7fb62c03b3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:54:39.111 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:38.368+0000 7fb64ce00700 1 --2- 172.21.15.22:0/2106700769 >> [v2:172.21.15.22:6800/730635939,v1:172.21.15.22:6801/730635939] conn(0x7fb62c038f20 0x7fb62c03b3e0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fb634006fd0 tx=0x7fb634006e40 comp rx=0 tx=0).stop 2024-03-28T17:54:39.111 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:38.368+0000 7fb64ce00700 1 -- 172.21.15.22:0/2106700769 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb64807cd70 msgr2=0x7fb64819dd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:54:39.111 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:38.368+0000 7fb64ce00700 1 --2- 172.21.15.22:0/2106700769 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb64807cd70 0x7fb64819dd40 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fb638004180 tx=0x7fb6380046a0 comp rx=0 tx=0).stop 2024-03-28T17:54:39.111 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:38.369+0000 7fb64ce00700 1 -- 172.21.15.22:0/2106700769 shutdown_connections 2024-03-28T17:54:39.111 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:38.369+0000 7fb64ce00700 1 --2- 172.21.15.22:0/2106700769 >> [v2:172.21.15.22:6800/730635939,v1:172.21.15.22:6801/730635939] conn(0x7fb62c038f20 0x7fb62c03b3e0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:54:39.111 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:38.369+0000 7fb64ce00700 1 --2- 172.21.15.22:0/2106700769 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb64807cd70 0x7fb64819dd40 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:54:39.111 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:38.369+0000 7fb64ce00700 1 -- 172.21.15.22:0/2106700769 >> 172.21.15.22:0/2106700769 conn(0x7fb648103100 msgr2=0x7fb648103cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T17:54:39.111 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:38.369+0000 7fb64ce00700 1 -- 172.21.15.22:0/2106700769 shutdown_connections 2024-03-28T17:54:39.111 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:38.369+0000 7fb64ce00700 1 -- 172.21.15.22:0/2106700769 wait complete. 2024-03-28T17:54:39.112 INFO:teuthology.orchestra.run.smithi022.stdout:Generating ssh key... 2024-03-28T17:54:39.410 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:54:39 smithi022 ceph-mon[31985]: from='client.14134 -' entity='client.admin' cmd=[{"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}]: dispatch 2024-03-28T17:54:40.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:54:40 smithi022 ceph-mon[31985]: from='client.14136 -' entity='client.admin' cmd=[{"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}]: dispatch 2024-03-28T17:54:40.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:54:40 smithi022 ceph-mon[31985]: Generating ssh key... 2024-03-28T17:54:41.033 INFO:journalctl@ceph.mgr.x.smithi022.stdout:Mar 28 17:54:40 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mgr-x[32211]: Generating public/private rsa key pair. 2024-03-28T17:54:41.033 INFO:journalctl@ceph.mgr.x.smithi022.stdout:Mar 28 17:54:40 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mgr-x[32211]: Your identification has been saved in /tmp/tmpt__liez3/key. 2024-03-28T17:54:41.033 INFO:journalctl@ceph.mgr.x.smithi022.stdout:Mar 28 17:54:40 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mgr-x[32211]: Your public key has been saved in /tmp/tmpt__liez3/key.pub. 2024-03-28T17:54:41.033 INFO:journalctl@ceph.mgr.x.smithi022.stdout:Mar 28 17:54:40 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mgr-x[32211]: The key fingerprint is: 2024-03-28T17:54:41.033 INFO:journalctl@ceph.mgr.x.smithi022.stdout:Mar 28 17:54:40 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mgr-x[32211]: SHA256:ZJTnCd/8U6w9RjY1TPpK3f8I+45Vy1bG1nbzjMcIU+E ceph-fd220748-ed2b-11ee-95d0-87774f69a715 2024-03-28T17:54:41.033 INFO:journalctl@ceph.mgr.x.smithi022.stdout:Mar 28 17:54:40 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mgr-x[32211]: The key's randomart image is: 2024-03-28T17:54:41.033 INFO:journalctl@ceph.mgr.x.smithi022.stdout:Mar 28 17:54:40 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mgr-x[32211]: +---[RSA 3072]----+ 2024-03-28T17:54:41.034 INFO:journalctl@ceph.mgr.x.smithi022.stdout:Mar 28 17:54:40 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mgr-x[32211]: | .. +. | 2024-03-28T17:54:41.034 INFO:journalctl@ceph.mgr.x.smithi022.stdout:Mar 28 17:54:40 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mgr-x[32211]: | .o . ..+.| 2024-03-28T17:54:41.034 INFO:journalctl@ceph.mgr.x.smithi022.stdout:Mar 28 17:54:40 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mgr-x[32211]: | o= + .E.o| 2024-03-28T17:54:41.034 INFO:journalctl@ceph.mgr.x.smithi022.stdout:Mar 28 17:54:40 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mgr-x[32211]: | o + o.o==| 2024-03-28T17:54:41.034 INFO:journalctl@ceph.mgr.x.smithi022.stdout:Mar 28 17:54:40 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mgr-x[32211]: | S oooB%| 2024-03-28T17:54:41.034 INFO:journalctl@ceph.mgr.x.smithi022.stdout:Mar 28 17:54:40 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mgr-x[32211]: | .o*%O| 2024-03-28T17:54:41.034 INFO:journalctl@ceph.mgr.x.smithi022.stdout:Mar 28 17:54:40 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mgr-x[32211]: | ..==B| 2024-03-28T17:54:41.034 INFO:journalctl@ceph.mgr.x.smithi022.stdout:Mar 28 17:54:40 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mgr-x[32211]: | =.o.| 2024-03-28T17:54:41.034 INFO:journalctl@ceph.mgr.x.smithi022.stdout:Mar 28 17:54:40 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mgr-x[32211]: | oo+ .| 2024-03-28T17:54:41.034 INFO:journalctl@ceph.mgr.x.smithi022.stdout:Mar 28 17:54:40 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mgr-x[32211]: +----[SHA256]-----+ 2024-03-28T17:54:41.404 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:39.720+0000 7f8430c00700 1 Processor -- start 2024-03-28T17:54:41.406 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:39.720+0000 7f8430c00700 1 -- start start 2024-03-28T17:54:41.406 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:39.721+0000 7f8430c00700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f842c109d70 0x7f842c10a190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:54:41.407 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:39.721+0000 7f8430c00700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f842c10a760 con 0x7f842c109d70 2024-03-28T17:54:41.407 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:39.721+0000 7f8429e00700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f842c109d70 0x7f842c10a190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:54:41.407 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:39.722+0000 7f8429e00700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f842c109d70 0x7f842c10a190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:45854/0 (socket says 172.21.15.22:45854) 2024-03-28T17:54:41.407 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:39.722+0000 7f8429e00700 1 -- 172.21.15.22:0/3846413552 learned_addr learned my addr 172.21.15.22:0/3846413552 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-28T17:54:41.407 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:39.722+0000 7f8429e00700 1 -- 172.21.15.22:0/3846413552 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f842c10af70 con 0x7f842c109d70 2024-03-28T17:54:41.407 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:39.722+0000 7f8429e00700 1 --2- 172.21.15.22:0/3846413552 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f842c109d70 0x7f842c10a190 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f841c0098a0 tx=0x7f841c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3c474c4568a45db8 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T17:54:41.407 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:39.723+0000 7f8428a00700 1 -- 172.21.15.22:0/3846413552 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f841c004160 con 0x7f842c109d70 2024-03-28T17:54:41.407 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:39.723+0000 7f8428a00700 1 -- 172.21.15.22:0/3846413552 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f841c004300 con 0x7f842c109d70 2024-03-28T17:54:41.407 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:39.723+0000 7f8428a00700 1 -- 172.21.15.22:0/3846413552 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f841c011b40 con 0x7f842c109d70 2024-03-28T17:54:41.407 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:39.723+0000 7f8430c00700 1 -- 172.21.15.22:0/3846413552 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f842c109d70 msgr2=0x7f842c10a190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:54:41.411 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:39.723+0000 7f8430c00700 1 --2- 172.21.15.22:0/3846413552 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f842c109d70 0x7f842c10a190 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f841c0098a0 tx=0x7f841c009bb0 comp rx=0 tx=0).stop 2024-03-28T17:54:41.411 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:39.724+0000 7f8430c00700 1 -- 172.21.15.22:0/3846413552 shutdown_connections 2024-03-28T17:54:41.411 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:39.724+0000 7f8430c00700 1 --2- 172.21.15.22:0/3846413552 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f842c109d70 0x7f842c10a190 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:54:41.411 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:39.724+0000 7f8430c00700 1 -- 172.21.15.22:0/3846413552 >> 172.21.15.22:0/3846413552 conn(0x7f842c07d120 msgr2=0x7f842c07d570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T17:54:41.411 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:39.724+0000 7f8430c00700 1 -- 172.21.15.22:0/3846413552 shutdown_connections 2024-03-28T17:54:41.411 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:39.724+0000 7f8430c00700 1 -- 172.21.15.22:0/3846413552 wait complete. 2024-03-28T17:54:41.411 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:39.725+0000 7f8430c00700 1 Processor -- start 2024-03-28T17:54:41.411 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:39.725+0000 7f8430c00700 1 -- start start 2024-03-28T17:54:41.412 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:39.725+0000 7f8430c00700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f842c109d70 0x7f842c19dd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:54:41.412 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:39.725+0000 7f8430c00700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f842c10e320 con 0x7f842c109d70 2024-03-28T17:54:41.412 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:39.726+0000 7f8429e00700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f842c109d70 0x7f842c19dd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:54:41.412 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:39.726+0000 7f8429e00700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f842c109d70 0x7f842c19dd60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:45860/0 (socket says 172.21.15.22:45860) 2024-03-28T17:54:41.412 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:39.726+0000 7f8429e00700 1 -- 172.21.15.22:0/3743260601 learned_addr learned my addr 172.21.15.22:0/3743260601 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-28T17:54:41.412 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:39.726+0000 7f8429e00700 1 -- 172.21.15.22:0/3743260601 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f842c19e2a0 con 0x7f842c109d70 2024-03-28T17:54:41.412 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:39.726+0000 7f8429e00700 1 --2- 172.21.15.22:0/3743260601 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f842c109d70 0x7f842c19dd60 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f841c011840 tx=0x7f841c004230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T17:54:41.412 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:39.727+0000 7f8422a00700 1 -- 172.21.15.22:0/3743260601 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f841c0038b0 con 0x7f842c109d70 2024-03-28T17:54:41.412 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:39.727+0000 7f8422a00700 1 -- 172.21.15.22:0/3743260601 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f841c01a430 con 0x7f842c109d70 2024-03-28T17:54:41.412 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:39.727+0000 7f8430c00700 1 -- 172.21.15.22:0/3743260601 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f842c19e530 con 0x7f842c109d70 2024-03-28T17:54:41.412 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:39.727+0000 7f8422a00700 1 -- 172.21.15.22:0/3743260601 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f841c024650 con 0x7f842c109d70 2024-03-28T17:54:41.412 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:39.727+0000 7f8430c00700 1 -- 172.21.15.22:0/3743260601 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f842c1a1220 con 0x7f842c109d70 2024-03-28T17:54:41.412 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:39.728+0000 7f8422a00700 1 -- 172.21.15.22:0/3743260601 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 8) v1 ==== 46106+0+0 (secure 0 0 0) 0x7f841c022020 con 0x7f842c109d70 2024-03-28T17:54:41.412 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:39.728+0000 7f8430c00700 1 -- 172.21.15.22:0/3743260601 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f842c0637c0 con 0x7f842c109d70 2024-03-28T17:54:41.412 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:39.728+0000 7f8422a00700 1 --2- 172.21.15.22:0/3743260601 >> [v2:172.21.15.22:6800/730635939,v1:172.21.15.22:6801/730635939] conn(0x7f8410038f70 0x7f841003b430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:54:41.412 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:39.728+0000 7f8422a00700 1 -- 172.21.15.22:0/3743260601 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f841c04cc40 con 0x7f842c109d70 2024-03-28T17:54:41.413 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:39.729+0000 7f8429400700 1 --2- 172.21.15.22:0/3743260601 >> [v2:172.21.15.22:6800/730635939,v1:172.21.15.22:6801/730635939] conn(0x7f8410038f70 0x7f841003b430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:54:41.413 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:39.729+0000 7f8429400700 1 --2- 172.21.15.22:0/3743260601 >> [v2:172.21.15.22:6800/730635939,v1:172.21.15.22:6801/730635939] conn(0x7f8410038f70 0x7f841003b430 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f8418006fd0 tx=0x7f8418006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T17:54:41.413 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:39.734+0000 7f8422a00700 1 -- 172.21.15.22:0/3743260601 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f841c00e260 con 0x7f842c109d70 2024-03-28T17:54:41.413 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:39.949+0000 7f8430c00700 1 -- 172.21.15.22:0/3743260601 --> [v2:172.21.15.22:6800/730635939,v1:172.21.15.22:6801/730635939] -- mgr_command(tid 0: {"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}) v1 -- 0x7f842c107c00 con 0x7f8410038f70 2024-03-28T17:54:41.413 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:40.755+0000 7f8422a00700 1 -- 172.21.15.22:0/3743260601 <== mgr.14120 v2:172.21.15.22:6800/730635939 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f842c107c00 con 0x7f8410038f70 2024-03-28T17:54:41.413 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:40.758+0000 7f8430c00700 1 -- 172.21.15.22:0/3743260601 >> [v2:172.21.15.22:6800/730635939,v1:172.21.15.22:6801/730635939] conn(0x7f8410038f70 msgr2=0x7f841003b430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:54:41.413 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:40.758+0000 7f8430c00700 1 --2- 172.21.15.22:0/3743260601 >> [v2:172.21.15.22:6800/730635939,v1:172.21.15.22:6801/730635939] conn(0x7f8410038f70 0x7f841003b430 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f8418006fd0 tx=0x7f8418006e40 comp rx=0 tx=0).stop 2024-03-28T17:54:41.413 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:40.758+0000 7f8430c00700 1 -- 172.21.15.22:0/3743260601 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f842c109d70 msgr2=0x7f842c19dd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:54:41.413 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:40.758+0000 7f8430c00700 1 --2- 172.21.15.22:0/3743260601 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f842c109d70 0x7f842c19dd60 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f841c011840 tx=0x7f841c004230 comp rx=0 tx=0).stop 2024-03-28T17:54:41.413 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:40.758+0000 7f8430c00700 1 -- 172.21.15.22:0/3743260601 shutdown_connections 2024-03-28T17:54:41.413 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:40.758+0000 7f8430c00700 1 --2- 172.21.15.22:0/3743260601 >> [v2:172.21.15.22:6800/730635939,v1:172.21.15.22:6801/730635939] conn(0x7f8410038f70 0x7f841003b430 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:54:41.414 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:40.758+0000 7f8430c00700 1 --2- 172.21.15.22:0/3743260601 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f842c109d70 0x7f842c19dd60 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:54:41.414 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:40.758+0000 7f8430c00700 1 -- 172.21.15.22:0/3743260601 >> 172.21.15.22:0/3743260601 conn(0x7f842c07d120 msgr2=0x7f842c1074f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T17:54:41.414 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:40.758+0000 7f8430c00700 1 -- 172.21.15.22:0/3743260601 shutdown_connections 2024-03-28T17:54:41.414 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:40.758+0000 7f8430c00700 1 -- 172.21.15.22:0/3743260601 wait complete. 2024-03-28T17:54:42.032 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:54:41 smithi022 ceph-mon[31985]: from='mgr.14120 172.21.15.22:0/1493984472' entity='mgr.x' 2024-03-28T17:54:42.032 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:54:41 smithi022 ceph-mon[31985]: from='mgr.14120 172.21.15.22:0/1493984472' entity='mgr.x' 2024-03-28T17:54:43.116 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDUi+jeSl5COOIlXQAk/lFBdgZWXIsZSspbsG9so++bg1fBAczn//xlfwOpRWKV+8NU3pC1kj15TfHrDeimhlKAjxRQFLaEDDFYlaKmiCVcjpdDzFhL4Dmv8fYmfwSKIdB+Y2UjHB3mf3h61Tk7oWagZwenEljWcSjAWDuPjRFYtefJOV/DfIVaUpcbHt+GzkbO96+XI17MnqFBT/Ms2kD4Ipb9IEg+tlEzde3zLCP1ZyIOpyHjFrsfGx5+HktyRHFxTrFonNfvWhRF4bpy9J47Qe7vGjUJD2lXOftS4741TJz7YBVRrFz2ukflvpwLR9volk6Rtava/3j/JI+2qqulshjsHhcMQ3pJ5jqCDfjj1DdK61Hzkb//S0gFWNi/X6zFgwSQ5sw6+m4A6OLFcw90bmac2HV0RwXzg8Svao51Xl9fu85aUVcp9h0CtnhtGrOc1GEfve5SwE/CW423P5vrz1q6XIqXxOVP2zPgc2AyQbz1RhLCWAzyKZwTRlLpOA0= ceph-fd220748-ed2b-11ee-95d0-87774f69a715 2024-03-28T17:54:43.118 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:42.080+0000 7f9fc4000700 1 Processor -- start 2024-03-28T17:54:43.118 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:42.081+0000 7f9fc4000700 1 -- start start 2024-03-28T17:54:43.118 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:42.081+0000 7f9fc4000700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9fbc1086f0 0x7f9fbc10aae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:54:43.118 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:42.081+0000 7f9fc4000700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9fbc075890 con 0x7f9fbc1086f0 2024-03-28T17:54:43.118 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:42.082+0000 7f9fc1600700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9fbc1086f0 0x7f9fbc10aae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:54:43.118 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:42.082+0000 7f9fc1600700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9fbc1086f0 0x7f9fbc10aae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:45874/0 (socket says 172.21.15.22:45874) 2024-03-28T17:54:43.118 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:42.082+0000 7f9fc1600700 1 -- 172.21.15.22:0/3462116138 learned_addr learned my addr 172.21.15.22:0/3462116138 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-28T17:54:43.120 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:42.083+0000 7f9fc1600700 1 -- 172.21.15.22:0/3462116138 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9fbc10b020 con 0x7f9fbc1086f0 2024-03-28T17:54:43.120 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:42.083+0000 7f9fc1600700 1 --2- 172.21.15.22:0/3462116138 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9fbc1086f0 0x7f9fbc10aae0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f9fb00098a0 tx=0x7f9fb0009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e307eec32f7c8861 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T17:54:43.120 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:42.083+0000 7f9fbbe00700 1 -- 172.21.15.22:0/3462116138 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f9fb0004160 con 0x7f9fbc1086f0 2024-03-28T17:54:43.120 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:42.083+0000 7f9fbbe00700 1 -- 172.21.15.22:0/3462116138 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9fb0004300 con 0x7f9fbc1086f0 2024-03-28T17:54:43.120 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:42.083+0000 7f9fbbe00700 1 -- 172.21.15.22:0/3462116138 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f9fb0011b40 con 0x7f9fbc1086f0 2024-03-28T17:54:43.120 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:42.084+0000 7f9fc4000700 1 -- 172.21.15.22:0/3462116138 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9fbc1086f0 msgr2=0x7f9fbc10aae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:54:43.120 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:42.084+0000 7f9fc4000700 1 --2- 172.21.15.22:0/3462116138 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9fbc1086f0 0x7f9fbc10aae0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f9fb00098a0 tx=0x7f9fb0009bb0 comp rx=0 tx=0).stop 2024-03-28T17:54:43.120 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:42.084+0000 7f9fc4000700 1 -- 172.21.15.22:0/3462116138 shutdown_connections 2024-03-28T17:54:43.120 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:42.084+0000 7f9fc4000700 1 --2- 172.21.15.22:0/3462116138 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9fbc1086f0 0x7f9fbc10aae0 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:54:43.121 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:42.084+0000 7f9fc4000700 1 -- 172.21.15.22:0/3462116138 >> 172.21.15.22:0/3462116138 conn(0x7f9fbc1020a0 msgr2=0x7f9fbc1044c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T17:54:43.121 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:42.085+0000 7f9fc4000700 1 -- 172.21.15.22:0/3462116138 shutdown_connections 2024-03-28T17:54:43.121 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:42.085+0000 7f9fc4000700 1 -- 172.21.15.22:0/3462116138 wait complete. 2024-03-28T17:54:43.121 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:42.086+0000 7f9fc4000700 1 Processor -- start 2024-03-28T17:54:43.124 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:42.086+0000 7f9fc4000700 1 -- start start 2024-03-28T17:54:43.125 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:42.086+0000 7f9fc4000700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9fbc1086f0 0x7f9fbc19bb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:54:43.125 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:42.086+0000 7f9fc4000700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9fbc10bf80 con 0x7f9fbc1086f0 2024-03-28T17:54:43.125 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:42.086+0000 7f9fc1600700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9fbc1086f0 0x7f9fbc19bb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:54:43.125 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:42.086+0000 7f9fc1600700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9fbc1086f0 0x7f9fbc19bb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:45876/0 (socket says 172.21.15.22:45876) 2024-03-28T17:54:43.125 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:42.086+0000 7f9fc1600700 1 -- 172.21.15.22:0/1949255521 learned_addr learned my addr 172.21.15.22:0/1949255521 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-28T17:54:43.125 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:42.087+0000 7f9fc1600700 1 -- 172.21.15.22:0/1949255521 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9fbc19c0a0 con 0x7f9fbc1086f0 2024-03-28T17:54:43.125 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:42.087+0000 7f9fc1600700 1 --2- 172.21.15.22:0/1949255521 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9fbc1086f0 0x7f9fbc19bb60 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f9fb0004180 tx=0x7f9fb00046a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T17:54:43.125 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:42.087+0000 7f9fba000700 1 -- 172.21.15.22:0/1949255521 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f9fb00039e0 con 0x7f9fbc1086f0 2024-03-28T17:54:43.125 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:42.087+0000 7f9fba000700 1 -- 172.21.15.22:0/1949255521 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9fb0020070 con 0x7f9fbc1086f0 2024-03-28T17:54:43.125 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:42.087+0000 7f9fc4000700 1 -- 172.21.15.22:0/1949255521 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9fbc19c330 con 0x7f9fbc1086f0 2024-03-28T17:54:43.125 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:42.087+0000 7f9fba000700 1 -- 172.21.15.22:0/1949255521 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f9fb00193c0 con 0x7f9fbc1086f0 2024-03-28T17:54:43.125 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:42.088+0000 7f9fc4000700 1 -- 172.21.15.22:0/1949255521 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9fbc19f020 con 0x7f9fbc1086f0 2024-03-28T17:54:43.125 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:42.089+0000 7f9fba000700 1 -- 172.21.15.22:0/1949255521 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 8) v1 ==== 46106+0+0 (secure 0 0 0) 0x7f9fb001a430 con 0x7f9fbc1086f0 2024-03-28T17:54:43.125 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:42.089+0000 7f9faf400700 1 -- 172.21.15.22:0/1949255521 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9f98005330 con 0x7f9fbc1086f0 2024-03-28T17:54:43.126 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:42.089+0000 7f9fba000700 1 --2- 172.21.15.22:0/1949255521 >> [v2:172.21.15.22:6800/730635939,v1:172.21.15.22:6801/730635939] conn(0x7f9fa4038f70 0x7f9fa403b430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:54:43.126 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:42.089+0000 7f9fba000700 1 -- 172.21.15.22:0/1949255521 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f9fb004d0a0 con 0x7f9fbc1086f0 2024-03-28T17:54:43.126 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:42.090+0000 7f9fc0c00700 1 --2- 172.21.15.22:0/1949255521 >> [v2:172.21.15.22:6800/730635939,v1:172.21.15.22:6801/730635939] conn(0x7f9fa4038f70 0x7f9fa403b430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:54:43.126 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:42.090+0000 7f9fc0c00700 1 --2- 172.21.15.22:0/1949255521 >> [v2:172.21.15.22:6800/730635939,v1:172.21.15.22:6801/730635939] conn(0x7f9fa4038f70 0x7f9fa403b430 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f9fa8006fd0 tx=0x7f9fa8006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T17:54:43.126 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:42.095+0000 7f9fba000700 1 -- 172.21.15.22:0/1949255521 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f9fb0023370 con 0x7f9fbc1086f0 2024-03-28T17:54:43.126 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:42.323+0000 7f9faf400700 1 -- 172.21.15.22:0/1949255521 --> [v2:172.21.15.22:6800/730635939,v1:172.21.15.22:6801/730635939] -- mgr_command(tid 0: {"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}) v1 -- 0x7f9f98000bc0 con 0x7f9fa4038f70 2024-03-28T17:54:43.126 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:42.324+0000 7f9fba000700 1 -- 172.21.15.22:0/1949255521 <== mgr.14120 v2:172.21.15.22:6800/730635939 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+595 (secure 0 0 0) 0x7f9f98000bc0 con 0x7f9fa4038f70 2024-03-28T17:54:43.126 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:42.327+0000 7f9faf400700 1 -- 172.21.15.22:0/1949255521 >> [v2:172.21.15.22:6800/730635939,v1:172.21.15.22:6801/730635939] conn(0x7f9fa4038f70 msgr2=0x7f9fa403b430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:54:43.126 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:42.327+0000 7f9faf400700 1 --2- 172.21.15.22:0/1949255521 >> [v2:172.21.15.22:6800/730635939,v1:172.21.15.22:6801/730635939] conn(0x7f9fa4038f70 0x7f9fa403b430 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f9fa8006fd0 tx=0x7f9fa8006e40 comp rx=0 tx=0).stop 2024-03-28T17:54:43.126 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:42.327+0000 7f9faf400700 1 -- 172.21.15.22:0/1949255521 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9fbc1086f0 msgr2=0x7f9fbc19bb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:54:43.126 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:42.327+0000 7f9faf400700 1 --2- 172.21.15.22:0/1949255521 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9fbc1086f0 0x7f9fbc19bb60 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f9fb0004180 tx=0x7f9fb00046a0 comp rx=0 tx=0).stop 2024-03-28T17:54:43.126 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:42.327+0000 7f9faf400700 1 -- 172.21.15.22:0/1949255521 shutdown_connections 2024-03-28T17:54:43.126 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:42.327+0000 7f9faf400700 1 --2- 172.21.15.22:0/1949255521 >> [v2:172.21.15.22:6800/730635939,v1:172.21.15.22:6801/730635939] conn(0x7f9fa4038f70 0x7f9fa403b430 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:54:43.126 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:42.327+0000 7f9faf400700 1 --2- 172.21.15.22:0/1949255521 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9fbc1086f0 0x7f9fbc19bb60 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:54:43.126 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:42.327+0000 7f9faf400700 1 -- 172.21.15.22:0/1949255521 >> 172.21.15.22:0/1949255521 conn(0x7f9fbc1020a0 msgr2=0x7f9fbc102bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T17:54:43.127 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:42.327+0000 7f9faf400700 1 -- 172.21.15.22:0/1949255521 shutdown_connections 2024-03-28T17:54:43.127 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:42.327+0000 7f9faf400700 1 -- 172.21.15.22:0/1949255521 wait complete. 2024-03-28T17:54:43.127 INFO:teuthology.orchestra.run.smithi022.stdout:Wrote public SSH key to /home/ubuntu/cephtest/ceph.pub 2024-03-28T17:54:43.127 INFO:teuthology.orchestra.run.smithi022.stdout:Adding key to root@localhost authorized_keys... 2024-03-28T17:54:43.127 INFO:teuthology.orchestra.run.smithi022.stdout:Adding host smithi022... 2024-03-28T17:54:43.324 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:54:43 smithi022 ceph-mon[31985]: from='client.14138 -' entity='client.admin' cmd=[{"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}]: dispatch 2024-03-28T17:54:45.489 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:54:45 smithi022 ceph-mon[31985]: from='client.14140 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi022", "addr": "172.21.15.22", "target": ["mon-mgr", ""]}]: dispatch 2024-03-28T17:54:46.196 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:54:46 smithi022 ceph-mon[31985]: Deploying cephadm binary to smithi022 2024-03-28T17:54:51.395 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout Added host 'smithi022' with addr '172.21.15.22' 2024-03-28T17:54:51.395 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:43.739+0000 7f0a7b200700 1 Processor -- start 2024-03-28T17:54:51.395 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:43.739+0000 7f0a7b200700 1 -- start start 2024-03-28T17:54:51.395 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:43.739+0000 7f0a7b200700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0a7c109bc0 0x7f0a7c109fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:54:51.395 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:43.739+0000 7f0a7b200700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a7c10a5b0 con 0x7f0a7c109bc0 2024-03-28T17:54:51.395 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:43.740+0000 7f0a79e00700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0a7c109bc0 0x7f0a7c109fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:54:51.395 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:43.740+0000 7f0a79e00700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0a7c109bc0 0x7f0a7c109fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:45880/0 (socket says 172.21.15.22:45880) 2024-03-28T17:54:51.395 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:43.740+0000 7f0a79e00700 1 -- 172.21.15.22:0/1536798347 learned_addr learned my addr 172.21.15.22:0/1536798347 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-28T17:54:51.396 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:43.740+0000 7f0a79e00700 1 -- 172.21.15.22:0/1536798347 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a7c10ad70 con 0x7f0a7c109bc0 2024-03-28T17:54:51.396 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:43.741+0000 7f0a79e00700 1 --2- 172.21.15.22:0/1536798347 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0a7c109bc0 0x7f0a7c109fe0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f0a6c009e80 tx=0x7f0a6c00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=531e146d6b274c02 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T17:54:51.396 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:43.741+0000 7f0a79400700 1 -- 172.21.15.22:0/1536798347 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f0a6c017070 con 0x7f0a7c109bc0 2024-03-28T17:54:51.396 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:43.741+0000 7f0a79400700 1 -- 172.21.15.22:0/1536798347 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0a6c00b8c0 con 0x7f0a7c109bc0 2024-03-28T17:54:51.396 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:43.741+0000 7f0a79400700 1 -- 172.21.15.22:0/1536798347 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f0a6c0045a0 con 0x7f0a7c109bc0 2024-03-28T17:54:51.396 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:43.742+0000 7f0a7b200700 1 -- 172.21.15.22:0/1536798347 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0a7c109bc0 msgr2=0x7f0a7c109fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:54:51.396 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:43.742+0000 7f0a7b200700 1 --2- 172.21.15.22:0/1536798347 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0a7c109bc0 0x7f0a7c109fe0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f0a6c009e80 tx=0x7f0a6c00b330 comp rx=0 tx=0).stop 2024-03-28T17:54:51.396 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:43.742+0000 7f0a7b200700 1 -- 172.21.15.22:0/1536798347 shutdown_connections 2024-03-28T17:54:51.396 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:43.742+0000 7f0a7b200700 1 --2- 172.21.15.22:0/1536798347 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0a7c109bc0 0x7f0a7c109fe0 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:54:51.396 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:43.742+0000 7f0a7b200700 1 -- 172.21.15.22:0/1536798347 >> 172.21.15.22:0/1536798347 conn(0x7f0a7c07cfe0 msgr2=0x7f0a7c07d410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T17:54:51.396 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:43.742+0000 7f0a7b200700 1 -- 172.21.15.22:0/1536798347 shutdown_connections 2024-03-28T17:54:51.396 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:43.742+0000 7f0a7b200700 1 -- 172.21.15.22:0/1536798347 wait complete. 2024-03-28T17:54:51.397 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:43.743+0000 7f0a7b200700 1 Processor -- start 2024-03-28T17:54:51.397 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:43.744+0000 7f0a7b200700 1 -- start start 2024-03-28T17:54:51.397 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:43.744+0000 7f0a7b200700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0a7c109bc0 0x7f0a7c19db40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:54:51.397 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:43.744+0000 7f0a7b200700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a7c10e120 con 0x7f0a7c109bc0 2024-03-28T17:54:51.397 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:43.744+0000 7f0a79e00700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0a7c109bc0 0x7f0a7c19db40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:54:51.397 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:43.745+0000 7f0a79e00700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0a7c109bc0 0x7f0a7c19db40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:45890/0 (socket says 172.21.15.22:45890) 2024-03-28T17:54:51.397 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:43.745+0000 7f0a79e00700 1 -- 172.21.15.22:0/715716735 learned_addr learned my addr 172.21.15.22:0/715716735 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-28T17:54:51.397 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:43.745+0000 7f0a79e00700 1 -- 172.21.15.22:0/715716735 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a7c19e080 con 0x7f0a7c109bc0 2024-03-28T17:54:51.397 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:43.745+0000 7f0a79e00700 1 --2- 172.21.15.22:0/715716735 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0a7c109bc0 0x7f0a7c19db40 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f0a6c0072a0 tx=0x7f0a6c003d00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T17:54:51.397 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:43.746+0000 7f0a73400700 1 -- 172.21.15.22:0/715716735 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f0a6c021020 con 0x7f0a7c109bc0 2024-03-28T17:54:51.397 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:43.746+0000 7f0a73400700 1 -- 172.21.15.22:0/715716735 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0a6c0048b0 con 0x7f0a7c109bc0 2024-03-28T17:54:51.397 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:43.746+0000 7f0a73400700 1 -- 172.21.15.22:0/715716735 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f0a6c017040 con 0x7f0a7c109bc0 2024-03-28T17:54:51.398 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:43.746+0000 7f0a7b200700 1 -- 172.21.15.22:0/715716735 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0a7c19e310 con 0x7f0a7c109bc0 2024-03-28T17:54:51.398 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:43.746+0000 7f0a7b200700 1 -- 172.21.15.22:0/715716735 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0a7c1a1000 con 0x7f0a7c109bc0 2024-03-28T17:54:51.398 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:43.747+0000 7f0a73400700 1 -- 172.21.15.22:0/715716735 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 8) v1 ==== 46106+0+0 (secure 0 0 0) 0x7f0a6c024020 con 0x7f0a7c109bc0 2024-03-28T17:54:51.398 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:43.747+0000 7f0a7b200700 1 -- 172.21.15.22:0/715716735 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0a7c0637c0 con 0x7f0a7c109bc0 2024-03-28T17:54:51.398 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:43.747+0000 7f0a73400700 1 --2- 172.21.15.22:0/715716735 >> [v2:172.21.15.22:6800/730635939,v1:172.21.15.22:6801/730635939] conn(0x7f0a60038f20 0x7f0a6003b3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:54:51.398 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:43.748+0000 7f0a73400700 1 -- 172.21.15.22:0/715716735 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f0a6c00f050 con 0x7f0a7c109bc0 2024-03-28T17:54:51.398 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:43.748+0000 7f0a71400700 1 --2- 172.21.15.22:0/715716735 >> [v2:172.21.15.22:6800/730635939,v1:172.21.15.22:6801/730635939] conn(0x7f0a60038f20 0x7f0a6003b3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:54:51.398 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:43.748+0000 7f0a71400700 1 --2- 172.21.15.22:0/715716735 >> [v2:172.21.15.22:6800/730635939,v1:172.21.15.22:6801/730635939] conn(0x7f0a60038f20 0x7f0a6003b3e0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f0a64006fd0 tx=0x7f0a64006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T17:54:51.398 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:43.754+0000 7f0a73400700 1 -- 172.21.15.22:0/715716735 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f0a6c02b020 con 0x7f0a7c109bc0 2024-03-28T17:54:51.398 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:43.983+0000 7f0a7b200700 1 -- 172.21.15.22:0/715716735 --> [v2:172.21.15.22:6800/730635939,v1:172.21.15.22:6801/730635939] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi022", "addr": "172.21.15.22", "target": ["mon-mgr", ""]}) v1 -- 0x7f0a7c107c20 con 0x7f0a60038f20 2024-03-28T17:54:51.398 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:50.607+0000 7f0a73400700 1 -- 172.21.15.22:0/715716735 <== mgr.14120 v2:172.21.15.22:6800/730635939 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+48 (secure 0 0 0) 0x7f0a7c107c20 con 0x7f0a60038f20 2024-03-28T17:54:51.399 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:50.611+0000 7f0a7b200700 1 -- 172.21.15.22:0/715716735 >> [v2:172.21.15.22:6800/730635939,v1:172.21.15.22:6801/730635939] conn(0x7f0a60038f20 msgr2=0x7f0a6003b3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:54:51.399 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:50.611+0000 7f0a7b200700 1 --2- 172.21.15.22:0/715716735 >> [v2:172.21.15.22:6800/730635939,v1:172.21.15.22:6801/730635939] conn(0x7f0a60038f20 0x7f0a6003b3e0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f0a64006fd0 tx=0x7f0a64006e40 comp rx=0 tx=0).stop 2024-03-28T17:54:51.399 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:50.611+0000 7f0a7b200700 1 -- 172.21.15.22:0/715716735 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0a7c109bc0 msgr2=0x7f0a7c19db40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:54:51.399 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:50.611+0000 7f0a7b200700 1 --2- 172.21.15.22:0/715716735 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0a7c109bc0 0x7f0a7c19db40 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f0a6c0072a0 tx=0x7f0a6c003d00 comp rx=0 tx=0).stop 2024-03-28T17:54:51.399 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:50.611+0000 7f0a7b200700 1 -- 172.21.15.22:0/715716735 shutdown_connections 2024-03-28T17:54:51.399 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:50.611+0000 7f0a7b200700 1 --2- 172.21.15.22:0/715716735 >> [v2:172.21.15.22:6800/730635939,v1:172.21.15.22:6801/730635939] conn(0x7f0a60038f20 0x7f0a6003b3e0 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:54:51.399 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:50.611+0000 7f0a7b200700 1 --2- 172.21.15.22:0/715716735 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0a7c109bc0 0x7f0a7c19db40 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:54:51.399 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:50.611+0000 7f0a7b200700 1 -- 172.21.15.22:0/715716735 >> 172.21.15.22:0/715716735 conn(0x7f0a7c07cfe0 msgr2=0x7f0a7c107510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T17:54:51.399 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:50.612+0000 7f0a7b200700 1 -- 172.21.15.22:0/715716735 shutdown_connections 2024-03-28T17:54:51.399 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:50.612+0000 7f0a7b200700 1 -- 172.21.15.22:0/715716735 wait complete. 2024-03-28T17:54:51.399 INFO:teuthology.orchestra.run.smithi022.stdout:Deploying unmanaged mon service... 2024-03-28T17:54:51.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:54:51 smithi022 ceph-mon[31985]: from='mgr.14120 172.21.15.22:0/1493984472' entity='mgr.x' 2024-03-28T17:54:51.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:54:51 smithi022 ceph-mon[31985]: Added host smithi022 2024-03-28T17:54:51.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:54:51 smithi022 ceph-mon[31985]: from='mgr.14120 172.21.15.22:0/1493984472' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T17:54:53.719 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:54:53 smithi022 ceph-mon[31985]: from='client.14142 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2024-03-28T17:54:53.719 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:54:53 smithi022 ceph-mon[31985]: Saving service mon spec with placement count:5 2024-03-28T17:54:53.720 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:54:53 smithi022 ceph-mon[31985]: from='mgr.14120 172.21.15.22:0/1493984472' entity='mgr.x' 2024-03-28T17:54:53.732 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout Scheduled mon update... 2024-03-28T17:54:53.732 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:52.248+0000 7fba39a00700 1 Processor -- start 2024-03-28T17:54:53.732 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:52.249+0000 7fba39a00700 1 -- start start 2024-03-28T17:54:53.732 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:52.249+0000 7fba39a00700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fba341083f0 0x7fba3410a7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:54:53.732 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:52.249+0000 7fba39a00700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba34075930 con 0x7fba341083f0 2024-03-28T17:54:53.732 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:52.249+0000 7fba33e00700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fba341083f0 0x7fba3410a7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:54:53.732 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:52.250+0000 7fba33e00700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fba341083f0 0x7fba3410a7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:35622/0 (socket says 172.21.15.22:35622) 2024-03-28T17:54:53.732 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:52.250+0000 7fba33e00700 1 -- 172.21.15.22:0/1365731636 learned_addr learned my addr 172.21.15.22:0/1365731636 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-28T17:54:53.732 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:52.250+0000 7fba33e00700 1 -- 172.21.15.22:0/1365731636 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fba3410ad20 con 0x7fba341083f0 2024-03-28T17:54:53.732 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:52.250+0000 7fba33e00700 1 --2- 172.21.15.22:0/1365731636 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fba341083f0 0x7fba3410a7e0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fba240098a0 tx=0x7fba24009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5865e8ea33a6b48f server_cookie=0 in_seq=0 out_seq=0 2024-03-28T17:54:53.732 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:52.250+0000 7fba32a00700 1 -- 172.21.15.22:0/1365731636 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fba24004160 con 0x7fba341083f0 2024-03-28T17:54:53.733 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:52.251+0000 7fba32a00700 1 -- 172.21.15.22:0/1365731636 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fba24004300 con 0x7fba341083f0 2024-03-28T17:54:53.733 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:52.251+0000 7fba32a00700 1 -- 172.21.15.22:0/1365731636 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fba24011b40 con 0x7fba341083f0 2024-03-28T17:54:53.733 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:52.251+0000 7fba39a00700 1 -- 172.21.15.22:0/1365731636 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fba341083f0 msgr2=0x7fba3410a7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:54:53.733 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:52.251+0000 7fba39a00700 1 --2- 172.21.15.22:0/1365731636 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fba341083f0 0x7fba3410a7e0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fba240098a0 tx=0x7fba24009bb0 comp rx=0 tx=0).stop 2024-03-28T17:54:53.733 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:52.251+0000 7fba39a00700 1 -- 172.21.15.22:0/1365731636 shutdown_connections 2024-03-28T17:54:53.733 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:52.251+0000 7fba39a00700 1 --2- 172.21.15.22:0/1365731636 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fba341083f0 0x7fba3410a7e0 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:54:53.733 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:52.251+0000 7fba39a00700 1 -- 172.21.15.22:0/1365731636 >> 172.21.15.22:0/1365731636 conn(0x7fba34101f00 msgr2=0x7fba34104320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T17:54:53.733 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:52.251+0000 7fba39a00700 1 -- 172.21.15.22:0/1365731636 shutdown_connections 2024-03-28T17:54:53.733 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:52.251+0000 7fba39a00700 1 -- 172.21.15.22:0/1365731636 wait complete. 2024-03-28T17:54:53.733 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:52.252+0000 7fba39a00700 1 Processor -- start 2024-03-28T17:54:53.733 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:52.253+0000 7fba39a00700 1 -- start start 2024-03-28T17:54:53.733 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:52.253+0000 7fba39a00700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fba341083f0 0x7fba3419db90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:54:53.733 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:52.253+0000 7fba39a00700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba3410bd40 con 0x7fba341083f0 2024-03-28T17:54:53.733 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:52.253+0000 7fba33e00700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fba341083f0 0x7fba3419db90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:54:53.733 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:52.253+0000 7fba33e00700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fba341083f0 0x7fba3419db90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:35630/0 (socket says 172.21.15.22:35630) 2024-03-28T17:54:53.734 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:52.253+0000 7fba33e00700 1 -- 172.21.15.22:0/3353336206 learned_addr learned my addr 172.21.15.22:0/3353336206 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-28T17:54:53.734 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:52.254+0000 7fba33e00700 1 -- 172.21.15.22:0/3353336206 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fba3419e0d0 con 0x7fba341083f0 2024-03-28T17:54:53.734 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:52.254+0000 7fba33e00700 1 --2- 172.21.15.22:0/3353336206 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fba341083f0 0x7fba3419db90 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fba24004180 tx=0x7fba240046a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T17:54:53.734 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:52.254+0000 7fba30c00700 1 -- 172.21.15.22:0/3353336206 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fba240039e0 con 0x7fba341083f0 2024-03-28T17:54:53.734 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:52.254+0000 7fba30c00700 1 -- 172.21.15.22:0/3353336206 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fba24020070 con 0x7fba341083f0 2024-03-28T17:54:53.734 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:52.254+0000 7fba30c00700 1 -- 172.21.15.22:0/3353336206 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fba240193c0 con 0x7fba341083f0 2024-03-28T17:54:53.734 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:52.254+0000 7fba39a00700 1 -- 172.21.15.22:0/3353336206 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fba3419e360 con 0x7fba341083f0 2024-03-28T17:54:53.734 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:52.254+0000 7fba39a00700 1 -- 172.21.15.22:0/3353336206 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fba341a1050 con 0x7fba341083f0 2024-03-28T17:54:53.734 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:52.255+0000 7fba30c00700 1 -- 172.21.15.22:0/3353336206 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 8) v1 ==== 46106+0+0 (secure 0 0 0) 0x7fba2401a430 con 0x7fba341083f0 2024-03-28T17:54:53.734 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:52.255+0000 7fba39a00700 1 -- 172.21.15.22:0/3353336206 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fba340637c0 con 0x7fba341083f0 2024-03-28T17:54:53.734 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:52.256+0000 7fba30c00700 1 --2- 172.21.15.22:0/3353336206 >> [v2:172.21.15.22:6800/730635939,v1:172.21.15.22:6801/730635939] conn(0x7fba18038f70 0x7fba1803b430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:54:53.734 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:52.256+0000 7fba30c00700 1 -- 172.21.15.22:0/3353336206 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7fba2404cf70 con 0x7fba341083f0 2024-03-28T17:54:53.734 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:52.256+0000 7fba33400700 1 --2- 172.21.15.22:0/3353336206 >> [v2:172.21.15.22:6800/730635939,v1:172.21.15.22:6801/730635939] conn(0x7fba18038f70 0x7fba1803b430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:54:53.734 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:52.257+0000 7fba33400700 1 --2- 172.21.15.22:0/3353336206 >> [v2:172.21.15.22:6800/730635939,v1:172.21.15.22:6801/730635939] conn(0x7fba18038f70 0x7fba1803b430 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fba20006fd0 tx=0x7fba20006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T17:54:53.735 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:52.262+0000 7fba30c00700 1 -- 172.21.15.22:0/3353336206 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fba24021270 con 0x7fba341083f0 2024-03-28T17:54:53.735 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:52.506+0000 7fba39a00700 1 -- 172.21.15.22:0/3353336206 --> [v2:172.21.15.22:6800/730635939,v1:172.21.15.22:6801/730635939] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "unmanaged": true, "target": ["mon-mgr", ""]}) v1 -- 0x7fba34103100 con 0x7fba18038f70 2024-03-28T17:54:53.735 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:52.517+0000 7fba30c00700 1 -- 172.21.15.22:0/3353336206 <== mgr.14120 v2:172.21.15.22:6800/730635939 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+24 (secure 0 0 0) 0x7fba34103100 con 0x7fba18038f70 2024-03-28T17:54:53.735 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:52.520+0000 7fba39a00700 1 -- 172.21.15.22:0/3353336206 >> [v2:172.21.15.22:6800/730635939,v1:172.21.15.22:6801/730635939] conn(0x7fba18038f70 msgr2=0x7fba1803b430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:54:53.735 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:52.520+0000 7fba39a00700 1 --2- 172.21.15.22:0/3353336206 >> [v2:172.21.15.22:6800/730635939,v1:172.21.15.22:6801/730635939] conn(0x7fba18038f70 0x7fba1803b430 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fba20006fd0 tx=0x7fba20006e40 comp rx=0 tx=0).stop 2024-03-28T17:54:53.735 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:52.520+0000 7fba39a00700 1 -- 172.21.15.22:0/3353336206 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fba341083f0 msgr2=0x7fba3419db90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:54:53.735 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:52.520+0000 7fba39a00700 1 --2- 172.21.15.22:0/3353336206 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fba341083f0 0x7fba3419db90 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fba24004180 tx=0x7fba240046a0 comp rx=0 tx=0).stop 2024-03-28T17:54:53.735 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:52.520+0000 7fba39a00700 1 -- 172.21.15.22:0/3353336206 shutdown_connections 2024-03-28T17:54:53.735 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:52.521+0000 7fba39a00700 1 --2- 172.21.15.22:0/3353336206 >> [v2:172.21.15.22:6800/730635939,v1:172.21.15.22:6801/730635939] conn(0x7fba18038f70 0x7fba1803b430 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:54:53.735 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:52.521+0000 7fba39a00700 1 --2- 172.21.15.22:0/3353336206 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fba341083f0 0x7fba3419db90 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:54:53.735 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:52.521+0000 7fba39a00700 1 -- 172.21.15.22:0/3353336206 >> 172.21.15.22:0/3353336206 conn(0x7fba34101f00 msgr2=0x7fba341029f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T17:54:53.735 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:52.522+0000 7fba39a00700 1 -- 172.21.15.22:0/3353336206 shutdown_connections 2024-03-28T17:54:53.735 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:52.522+0000 7fba39a00700 1 -- 172.21.15.22:0/3353336206 wait complete. 2024-03-28T17:54:53.735 INFO:teuthology.orchestra.run.smithi022.stdout:Deploying unmanaged mgr service... 2024-03-28T17:54:55.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:54:55 smithi022 ceph-mon[31985]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-28T17:54:55.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:54:55 smithi022 ceph-mon[31985]: from='mgr.14120 172.21.15.22:0/1493984472' entity='mgr.x' 2024-03-28T17:54:55.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:54:55 smithi022 ceph-mon[31985]: from='mgr.14120 172.21.15.22:0/1493984472' entity='mgr.x' 2024-03-28T17:54:55.807 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout Scheduled mgr update... 2024-03-28T17:54:55.807 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:54.790+0000 7f72f9600700 1 Processor -- start 2024-03-28T17:54:55.807 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:54.791+0000 7f72f9600700 1 -- start start 2024-03-28T17:54:55.807 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:54.791+0000 7f72f9600700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f72f4109db0 0x7f72f410a1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:54:55.807 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:54.791+0000 7f72f9600700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72f410a7a0 con 0x7f72f4109db0 2024-03-28T17:54:55.807 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:54.791+0000 7f72f2a00700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f72f4109db0 0x7f72f410a1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:54:55.807 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:54.792+0000 7f72f2a00700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f72f4109db0 0x7f72f410a1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:35646/0 (socket says 172.21.15.22:35646) 2024-03-28T17:54:55.807 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:54.792+0000 7f72f2a00700 1 -- 172.21.15.22:0/2515372282 learned_addr learned my addr 172.21.15.22:0/2515372282 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-28T17:54:55.808 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:54.792+0000 7f72f2a00700 1 -- 172.21.15.22:0/2515372282 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f72f410af60 con 0x7f72f4109db0 2024-03-28T17:54:55.808 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:54.792+0000 7f72f2a00700 1 --2- 172.21.15.22:0/2515372282 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f72f4109db0 0x7f72f410a1d0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f72e40098a0 tx=0x7f72e4009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=48ffaf9cf0c76bdc server_cookie=0 in_seq=0 out_seq=0 2024-03-28T17:54:55.808 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:54.793+0000 7f72f1600700 1 -- 172.21.15.22:0/2515372282 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f72e4004160 con 0x7f72f4109db0 2024-03-28T17:54:55.808 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:54.793+0000 7f72f1600700 1 -- 172.21.15.22:0/2515372282 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f72e4004300 con 0x7f72f4109db0 2024-03-28T17:54:55.808 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:54.793+0000 7f72f1600700 1 -- 172.21.15.22:0/2515372282 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f72e4011b10 con 0x7f72f4109db0 2024-03-28T17:54:55.808 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:54.793+0000 7f72f9600700 1 -- 172.21.15.22:0/2515372282 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f72f4109db0 msgr2=0x7f72f410a1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:54:55.808 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:54.793+0000 7f72f9600700 1 --2- 172.21.15.22:0/2515372282 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f72f4109db0 0x7f72f410a1d0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f72e40098a0 tx=0x7f72e4009bb0 comp rx=0 tx=0).stop 2024-03-28T17:54:55.808 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:54.793+0000 7f72f9600700 1 -- 172.21.15.22:0/2515372282 shutdown_connections 2024-03-28T17:54:55.808 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:54.793+0000 7f72f9600700 1 --2- 172.21.15.22:0/2515372282 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f72f4109db0 0x7f72f410a1d0 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:54:55.808 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:54.793+0000 7f72f9600700 1 -- 172.21.15.22:0/2515372282 >> 172.21.15.22:0/2515372282 conn(0x7f72f407d150 msgr2=0x7f72f407d5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T17:54:55.808 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:54.794+0000 7f72f9600700 1 -- 172.21.15.22:0/2515372282 shutdown_connections 2024-03-28T17:54:55.808 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:54.794+0000 7f72f9600700 1 -- 172.21.15.22:0/2515372282 wait complete. 2024-03-28T17:54:55.808 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:54.795+0000 7f72f9600700 1 Processor -- start 2024-03-28T17:54:55.808 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:54.795+0000 7f72f9600700 1 -- start start 2024-03-28T17:54:55.809 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:54.795+0000 7f72f9600700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f72f4109db0 0x7f72f419dda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:54:55.809 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:54.795+0000 7f72f9600700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72f410e310 con 0x7f72f4109db0 2024-03-28T17:54:55.809 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:54.796+0000 7f72f2a00700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f72f4109db0 0x7f72f419dda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:54:55.809 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:54.796+0000 7f72f2a00700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f72f4109db0 0x7f72f419dda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:35648/0 (socket says 172.21.15.22:35648) 2024-03-28T17:54:55.809 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:54.796+0000 7f72f2a00700 1 -- 172.21.15.22:0/626697411 learned_addr learned my addr 172.21.15.22:0/626697411 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-28T17:54:55.809 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:54.796+0000 7f72f2a00700 1 -- 172.21.15.22:0/626697411 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f72f419e2e0 con 0x7f72f4109db0 2024-03-28T17:54:55.809 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:54.796+0000 7f72f2a00700 1 --2- 172.21.15.22:0/626697411 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f72f4109db0 0x7f72f419dda0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f72e4006af0 tx=0x7f72e4011bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T17:54:55.809 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:54.797+0000 7f72eb400700 1 -- 172.21.15.22:0/626697411 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f72e4003af0 con 0x7f72f4109db0 2024-03-28T17:54:55.809 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:54.797+0000 7f72eb400700 1 -- 172.21.15.22:0/626697411 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f72e4003c90 con 0x7f72f4109db0 2024-03-28T17:54:55.809 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:54.797+0000 7f72eb400700 1 -- 172.21.15.22:0/626697411 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f72e4024830 con 0x7f72f4109db0 2024-03-28T17:54:55.809 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:54.797+0000 7f72f9600700 1 -- 172.21.15.22:0/626697411 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f72f419e570 con 0x7f72f4109db0 2024-03-28T17:54:55.809 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:54.797+0000 7f72f9600700 1 -- 172.21.15.22:0/626697411 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f72f41a1260 con 0x7f72f4109db0 2024-03-28T17:54:55.809 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:54.798+0000 7f72f9600700 1 -- 172.21.15.22:0/626697411 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f72f40637c0 con 0x7f72f4109db0 2024-03-28T17:54:55.809 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:54.798+0000 7f72eb400700 1 -- 172.21.15.22:0/626697411 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 8) v1 ==== 46106+0+0 (secure 0 0 0) 0x7f72e4022020 con 0x7f72f4109db0 2024-03-28T17:54:55.810 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:54.799+0000 7f72eb400700 1 --2- 172.21.15.22:0/626697411 >> [v2:172.21.15.22:6800/730635939,v1:172.21.15.22:6801/730635939] conn(0x7f72d8038f70 0x7f72d803b430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:54:55.810 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:54.799+0000 7f72eb400700 1 -- 172.21.15.22:0/626697411 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f72e404ce40 con 0x7f72f4109db0 2024-03-28T17:54:55.810 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:54.799+0000 7f72f2000700 1 --2- 172.21.15.22:0/626697411 >> [v2:172.21.15.22:6800/730635939,v1:172.21.15.22:6801/730635939] conn(0x7f72d8038f70 0x7f72d803b430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:54:55.810 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:54.800+0000 7f72f2000700 1 --2- 172.21.15.22:0/626697411 >> [v2:172.21.15.22:6800/730635939,v1:172.21.15.22:6801/730635939] conn(0x7f72d8038f70 0x7f72d803b430 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f72e0006fd0 tx=0x7f72e0006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T17:54:55.810 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:54.805+0000 7f72eb400700 1 -- 172.21.15.22:0/626697411 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f72e404c790 con 0x7f72f4109db0 2024-03-28T17:54:55.810 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:55.052+0000 7f72f9600700 1 -- 172.21.15.22:0/626697411 --> [v2:172.21.15.22:6800/730635939,v1:172.21.15.22:6801/730635939] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}) v1 -- 0x7f72f4107c70 con 0x7f72d8038f70 2024-03-28T17:54:55.810 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:55.059+0000 7f72eb400700 1 -- 172.21.15.22:0/626697411 <== mgr.14120 v2:172.21.15.22:6800/730635939 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+24 (secure 0 0 0) 0x7f72f4107c70 con 0x7f72d8038f70 2024-03-28T17:54:55.810 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:55.062+0000 7f72f9600700 1 -- 172.21.15.22:0/626697411 >> [v2:172.21.15.22:6800/730635939,v1:172.21.15.22:6801/730635939] conn(0x7f72d8038f70 msgr2=0x7f72d803b430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:54:55.810 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:55.062+0000 7f72f9600700 1 --2- 172.21.15.22:0/626697411 >> [v2:172.21.15.22:6800/730635939,v1:172.21.15.22:6801/730635939] conn(0x7f72d8038f70 0x7f72d803b430 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f72e0006fd0 tx=0x7f72e0006e40 comp rx=0 tx=0).stop 2024-03-28T17:54:55.810 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:55.063+0000 7f72f9600700 1 -- 172.21.15.22:0/626697411 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f72f4109db0 msgr2=0x7f72f419dda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:54:55.810 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:55.063+0000 7f72f9600700 1 --2- 172.21.15.22:0/626697411 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f72f4109db0 0x7f72f419dda0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f72e4006af0 tx=0x7f72e4011bd0 comp rx=0 tx=0).stop 2024-03-28T17:54:55.810 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:55.063+0000 7f72f9600700 1 -- 172.21.15.22:0/626697411 shutdown_connections 2024-03-28T17:54:55.810 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:55.063+0000 7f72f9600700 1 --2- 172.21.15.22:0/626697411 >> [v2:172.21.15.22:6800/730635939,v1:172.21.15.22:6801/730635939] conn(0x7f72d8038f70 0x7f72d803b430 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:54:55.811 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:55.063+0000 7f72f9600700 1 --2- 172.21.15.22:0/626697411 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f72f4109db0 0x7f72f419dda0 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:54:55.811 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:55.063+0000 7f72f9600700 1 -- 172.21.15.22:0/626697411 >> 172.21.15.22:0/626697411 conn(0x7f72f407d150 msgr2=0x7f72f4107560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T17:54:55.811 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:55.063+0000 7f72f9600700 1 -- 172.21.15.22:0/626697411 shutdown_connections 2024-03-28T17:54:55.811 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:55.063+0000 7f72f9600700 1 -- 172.21.15.22:0/626697411 wait complete. 2024-03-28T17:54:56.927 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:54:56 smithi022 ceph-mon[31985]: from='client.14144 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2024-03-28T17:54:56.927 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:54:56 smithi022 ceph-mon[31985]: Saving service mgr spec with placement count:2 2024-03-28T17:54:56.928 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:54:56 smithi022 ceph-mon[31985]: from='mgr.14120 172.21.15.22:0/1493984472' entity='mgr.x' 2024-03-28T17:54:57.991 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:54:57 smithi022 ceph-mon[31985]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-28T17:54:57.991 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:54:57 smithi022 ceph-mon[31985]: from='client.? 172.21.15.22:0/632554144' entity='client.admin' 2024-03-28T17:54:57.999 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:56.468+0000 7f43f6c00700 1 Processor -- start 2024-03-28T17:54:58.001 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:56.469+0000 7f43f6c00700 1 -- start start 2024-03-28T17:54:58.001 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:56.469+0000 7f43f6c00700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f43f0108630 0x7f43f010aa20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:54:58.001 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:56.469+0000 7f43f6c00700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43f007be70 con 0x7f43f0108630 2024-03-28T17:54:58.001 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:56.469+0000 7f43efe00700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f43f0108630 0x7f43f010aa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:54:58.001 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:56.469+0000 7f43efe00700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f43f0108630 0x7f43f010aa20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:35652/0 (socket says 172.21.15.22:35652) 2024-03-28T17:54:58.001 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:56.469+0000 7f43efe00700 1 -- 172.21.15.22:0/2253777199 learned_addr learned my addr 172.21.15.22:0/2253777199 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-28T17:54:58.001 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:56.470+0000 7f43efe00700 1 -- 172.21.15.22:0/2253777199 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f43f010af60 con 0x7f43f0108630 2024-03-28T17:54:58.001 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:56.470+0000 7f43efe00700 1 --2- 172.21.15.22:0/2253777199 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f43f0108630 0x7f43f010aa20 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f43e00098a0 tx=0x7f43e0009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9dd7e78e9be989ca server_cookie=0 in_seq=0 out_seq=0 2024-03-28T17:54:58.001 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:56.470+0000 7f43eea00700 1 -- 172.21.15.22:0/2253777199 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f43e0004160 con 0x7f43f0108630 2024-03-28T17:54:58.002 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:56.471+0000 7f43eea00700 1 -- 172.21.15.22:0/2253777199 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f43e0004300 con 0x7f43f0108630 2024-03-28T17:54:58.002 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:56.471+0000 7f43eea00700 1 -- 172.21.15.22:0/2253777199 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f43e0011b10 con 0x7f43f0108630 2024-03-28T17:54:58.006 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:56.471+0000 7f43f6c00700 1 -- 172.21.15.22:0/2253777199 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f43f0108630 msgr2=0x7f43f010aa20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:54:58.006 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:56.471+0000 7f43f6c00700 1 --2- 172.21.15.22:0/2253777199 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f43f0108630 0x7f43f010aa20 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f43e00098a0 tx=0x7f43e0009bb0 comp rx=0 tx=0).stop 2024-03-28T17:54:58.006 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:56.472+0000 7f43f6c00700 1 -- 172.21.15.22:0/2253777199 shutdown_connections 2024-03-28T17:54:58.006 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:56.472+0000 7f43f6c00700 1 --2- 172.21.15.22:0/2253777199 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f43f0108630 0x7f43f010aa20 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:54:58.006 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:56.472+0000 7f43f6c00700 1 -- 172.21.15.22:0/2253777199 >> 172.21.15.22:0/2253777199 conn(0x7f43f0102000 msgr2=0x7f43f0104460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T17:54:58.006 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:56.472+0000 7f43f6c00700 1 -- 172.21.15.22:0/2253777199 shutdown_connections 2024-03-28T17:54:58.006 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:56.472+0000 7f43f6c00700 1 -- 172.21.15.22:0/2253777199 wait complete. 2024-03-28T17:54:58.006 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:56.473+0000 7f43f6c00700 1 Processor -- start 2024-03-28T17:54:58.006 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:56.473+0000 7f43f6c00700 1 -- start start 2024-03-28T17:54:58.007 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:56.473+0000 7f43f6c00700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f43f0108630 0x7f43f019bb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:54:58.007 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:56.473+0000 7f43f6c00700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43f010bfc0 con 0x7f43f0108630 2024-03-28T17:54:58.007 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:56.474+0000 7f43efe00700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f43f0108630 0x7f43f019bb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:54:58.007 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:56.474+0000 7f43efe00700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f43f0108630 0x7f43f019bb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:35656/0 (socket says 172.21.15.22:35656) 2024-03-28T17:54:58.007 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:56.474+0000 7f43efe00700 1 -- 172.21.15.22:0/632554144 learned_addr learned my addr 172.21.15.22:0/632554144 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-28T17:54:58.007 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:56.474+0000 7f43efe00700 1 -- 172.21.15.22:0/632554144 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f43f019c0b0 con 0x7f43f0108630 2024-03-28T17:54:58.007 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:56.474+0000 7f43efe00700 1 --2- 172.21.15.22:0/632554144 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f43f0108630 0x7f43f019bb70 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f43e0006af0 tx=0x7f43e0011bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T17:54:58.007 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:56.475+0000 7f43ecc00700 1 -- 172.21.15.22:0/632554144 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f43e0003af0 con 0x7f43f0108630 2024-03-28T17:54:58.007 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:56.475+0000 7f43ecc00700 1 -- 172.21.15.22:0/632554144 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f43e0003c90 con 0x7f43f0108630 2024-03-28T17:54:58.007 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:56.475+0000 7f43f6c00700 1 -- 172.21.15.22:0/632554144 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f43f019c340 con 0x7f43f0108630 2024-03-28T17:54:58.007 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:56.475+0000 7f43f6c00700 1 -- 172.21.15.22:0/632554144 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f43f019f030 con 0x7f43f0108630 2024-03-28T17:54:58.007 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:56.475+0000 7f43ecc00700 1 -- 172.21.15.22:0/632554144 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f43e0024830 con 0x7f43f0108630 2024-03-28T17:54:58.007 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:56.476+0000 7f43ecc00700 1 -- 172.21.15.22:0/632554144 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 8) v1 ==== 46106+0+0 (secure 0 0 0) 0x7f43e0022020 con 0x7f43f0108630 2024-03-28T17:54:58.007 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:56.476+0000 7f43f6c00700 1 -- 172.21.15.22:0/632554144 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f43d0005360 con 0x7f43f0108630 2024-03-28T17:54:58.007 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:56.477+0000 7f43ecc00700 1 --2- 172.21.15.22:0/632554144 >> [v2:172.21.15.22:6800/730635939,v1:172.21.15.22:6801/730635939] conn(0x7f43d4038f70 0x7f43d403b430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:54:58.008 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:56.477+0000 7f43ecc00700 1 -- 172.21.15.22:0/632554144 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f43e004ded0 con 0x7f43f0108630 2024-03-28T17:54:58.008 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:56.477+0000 7f43ef400700 1 --2- 172.21.15.22:0/632554144 >> [v2:172.21.15.22:6800/730635939,v1:172.21.15.22:6801/730635939] conn(0x7f43d4038f70 0x7f43d403b430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:54:58.008 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:56.478+0000 7f43ef400700 1 --2- 172.21.15.22:0/632554144 >> [v2:172.21.15.22:6800/730635939,v1:172.21.15.22:6801/730635939] conn(0x7f43d4038f70 0x7f43d403b430 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f43dc006fd0 tx=0x7f43dc006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T17:54:58.008 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:56.482+0000 7f43ecc00700 1 -- 172.21.15.22:0/632554144 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f43e00172b0 con 0x7f43f0108630 2024-03-28T17:54:58.008 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:56.724+0000 7f43f6c00700 1 -- 172.21.15.22:0/632554144 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/container_init}] v 0) v1 -- 0x7f43d0005650 con 0x7f43f0108630 2024-03-28T17:54:58.008 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:56.727+0000 7f43ecc00700 1 -- 172.21.15.22:0/632554144 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/container_init}]=0 v7)=0 v7) v1 ==== 142+0+0 (secure 0 0 0) 0x7f43e000b5a0 con 0x7f43f0108630 2024-03-28T17:54:58.008 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:56.730+0000 7f43f6c00700 1 -- 172.21.15.22:0/632554144 >> [v2:172.21.15.22:6800/730635939,v1:172.21.15.22:6801/730635939] conn(0x7f43d4038f70 msgr2=0x7f43d403b430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:54:58.008 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:56.730+0000 7f43f6c00700 1 --2- 172.21.15.22:0/632554144 >> [v2:172.21.15.22:6800/730635939,v1:172.21.15.22:6801/730635939] conn(0x7f43d4038f70 0x7f43d403b430 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f43dc006fd0 tx=0x7f43dc006e40 comp rx=0 tx=0).stop 2024-03-28T17:54:58.008 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:56.730+0000 7f43f6c00700 1 -- 172.21.15.22:0/632554144 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f43f0108630 msgr2=0x7f43f019bb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:54:58.008 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:56.730+0000 7f43f6c00700 1 --2- 172.21.15.22:0/632554144 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f43f0108630 0x7f43f019bb70 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f43e0006af0 tx=0x7f43e0011bd0 comp rx=0 tx=0).stop 2024-03-28T17:54:58.008 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:56.730+0000 7f43f6c00700 1 -- 172.21.15.22:0/632554144 shutdown_connections 2024-03-28T17:54:58.008 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:56.730+0000 7f43f6c00700 1 --2- 172.21.15.22:0/632554144 >> [v2:172.21.15.22:6800/730635939,v1:172.21.15.22:6801/730635939] conn(0x7f43d4038f70 0x7f43d403b430 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:54:58.008 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:56.730+0000 7f43f6c00700 1 --2- 172.21.15.22:0/632554144 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f43f0108630 0x7f43f019bb70 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:54:58.008 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:56.730+0000 7f43f6c00700 1 -- 172.21.15.22:0/632554144 >> 172.21.15.22:0/632554144 conn(0x7f43f0102000 msgr2=0x7f43f0102bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T17:54:58.008 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:56.731+0000 7f43f6c00700 1 -- 172.21.15.22:0/632554144 shutdown_connections 2024-03-28T17:54:58.008 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:56.731+0000 7f43f6c00700 1 -- 172.21.15.22:0/632554144 wait complete. 2024-03-28T17:54:59.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:54:59 smithi022 ceph-mon[31985]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-28T17:54:59.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:54:59 smithi022 ceph-mon[31985]: from='mgr.14120 172.21.15.22:0/1493984472' entity='mgr.x' 2024-03-28T17:54:59.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:54:59 smithi022 ceph-mon[31985]: from='client.? 172.21.15.22:0/3700899547' entity='client.admin' 2024-03-28T17:54:59.653 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:58.652+0000 7fc4d5000700 1 Processor -- start 2024-03-28T17:54:59.653 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:58.652+0000 7fc4d5000700 1 -- start start 2024-03-28T17:54:59.653 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:58.653+0000 7fc4d5000700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc4d0106400 0x7fc4d0106820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:54:59.653 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:58.653+0000 7fc4d5000700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4d0075970 con 0x7fc4d0106400 2024-03-28T17:54:59.653 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:58.653+0000 7fc4cde00700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc4d0106400 0x7fc4d0106820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:54:59.653 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:58.653+0000 7fc4cde00700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc4d0106400 0x7fc4d0106820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:43992/0 (socket says 172.21.15.22:43992) 2024-03-28T17:54:59.662 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:58.653+0000 7fc4cde00700 1 -- 172.21.15.22:0/3262207880 learned_addr learned my addr 172.21.15.22:0/3262207880 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-28T17:54:59.662 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:58.654+0000 7fc4cde00700 1 -- 172.21.15.22:0/3262207880 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc4d0106d60 con 0x7fc4d0106400 2024-03-28T17:54:59.662 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:58.654+0000 7fc4cde00700 1 --2- 172.21.15.22:0/3262207880 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc4d0106400 0x7fc4d0106820 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fc4c00098a0 tx=0x7fc4c0009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f45ec87823ff277c server_cookie=0 in_seq=0 out_seq=0 2024-03-28T17:54:59.662 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:58.654+0000 7fc4cca00700 1 -- 172.21.15.22:0/3262207880 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fc4c0004160 con 0x7fc4d0106400 2024-03-28T17:54:59.662 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:58.654+0000 7fc4cca00700 1 -- 172.21.15.22:0/3262207880 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7fc4c0004300 con 0x7fc4d0106400 2024-03-28T17:54:59.662 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:58.654+0000 7fc4cca00700 1 -- 172.21.15.22:0/3262207880 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fc4c0011b10 con 0x7fc4d0106400 2024-03-28T17:54:59.662 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:58.655+0000 7fc4d5000700 1 -- 172.21.15.22:0/3262207880 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc4d0106400 msgr2=0x7fc4d0106820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:54:59.662 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:58.655+0000 7fc4d5000700 1 --2- 172.21.15.22:0/3262207880 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc4d0106400 0x7fc4d0106820 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fc4c00098a0 tx=0x7fc4c0009bb0 comp rx=0 tx=0).stop 2024-03-28T17:54:59.663 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:58.655+0000 7fc4d5000700 1 -- 172.21.15.22:0/3262207880 shutdown_connections 2024-03-28T17:54:59.663 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:58.655+0000 7fc4d5000700 1 --2- 172.21.15.22:0/3262207880 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc4d0106400 0x7fc4d0106820 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:54:59.663 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:58.655+0000 7fc4d5000700 1 -- 172.21.15.22:0/3262207880 >> 172.21.15.22:0/3262207880 conn(0x7fc4d01020a0 msgr2=0x7fc4d01044c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T17:54:59.663 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:58.655+0000 7fc4d5000700 1 -- 172.21.15.22:0/3262207880 shutdown_connections 2024-03-28T17:54:59.663 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:58.656+0000 7fc4d5000700 1 -- 172.21.15.22:0/3262207880 wait complete. 2024-03-28T17:54:59.663 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:58.656+0000 7fc4d5000700 1 Processor -- start 2024-03-28T17:54:59.663 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:58.657+0000 7fc4d5000700 1 -- start start 2024-03-28T17:54:59.663 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:58.657+0000 7fc4d5000700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc4d0106400 0x7fc4d019dda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:54:59.663 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:58.657+0000 7fc4d5000700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4d0107e30 con 0x7fc4d0106400 2024-03-28T17:54:59.663 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:58.657+0000 7fc4cde00700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc4d0106400 0x7fc4d019dda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:54:59.663 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:58.657+0000 7fc4cde00700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc4d0106400 0x7fc4d019dda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:44002/0 (socket says 172.21.15.22:44002) 2024-03-28T17:54:59.663 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:58.657+0000 7fc4cde00700 1 -- 172.21.15.22:0/3700899547 learned_addr learned my addr 172.21.15.22:0/3700899547 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-28T17:54:59.663 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:58.658+0000 7fc4cde00700 1 -- 172.21.15.22:0/3700899547 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc4d019e2e0 con 0x7fc4d0106400 2024-03-28T17:54:59.664 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:58.658+0000 7fc4cde00700 1 --2- 172.21.15.22:0/3700899547 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc4d0106400 0x7fc4d019dda0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fc4c0006af0 tx=0x7fc4c0011bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T17:54:59.664 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:58.658+0000 7fc4c6a00700 1 -- 172.21.15.22:0/3700899547 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fc4c0003af0 con 0x7fc4d0106400 2024-03-28T17:54:59.664 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:58.658+0000 7fc4c6a00700 1 -- 172.21.15.22:0/3700899547 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7fc4c0003c90 con 0x7fc4d0106400 2024-03-28T17:54:59.664 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:58.658+0000 7fc4c6a00700 1 -- 172.21.15.22:0/3700899547 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fc4c0024830 con 0x7fc4d0106400 2024-03-28T17:54:59.664 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:58.658+0000 7fc4d5000700 1 -- 172.21.15.22:0/3700899547 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc4d019e570 con 0x7fc4d0106400 2024-03-28T17:54:59.664 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:58.658+0000 7fc4d5000700 1 -- 172.21.15.22:0/3700899547 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc4d01a1220 con 0x7fc4d0106400 2024-03-28T17:54:59.664 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:58.660+0000 7fc4c6a00700 1 -- 172.21.15.22:0/3700899547 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 8) v1 ==== 46106+0+0 (secure 0 0 0) 0x7fc4c0022020 con 0x7fc4d0106400 2024-03-28T17:54:59.664 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:58.660+0000 7fc4d5000700 1 -- 172.21.15.22:0/3700899547 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc4d00637c0 con 0x7fc4d0106400 2024-03-28T17:54:59.664 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:58.660+0000 7fc4c6a00700 1 --2- 172.21.15.22:0/3700899547 >> [v2:172.21.15.22:6800/730635939,v1:172.21.15.22:6801/730635939] conn(0x7fc4b4038f70 0x7fc4b403b430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:54:59.664 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:58.660+0000 7fc4c6a00700 1 -- 172.21.15.22:0/3700899547 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7fc4c004ceb0 con 0x7fc4d0106400 2024-03-28T17:54:59.664 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:58.660+0000 7fc4cd400700 1 --2- 172.21.15.22:0/3700899547 >> [v2:172.21.15.22:6800/730635939,v1:172.21.15.22:6801/730635939] conn(0x7fc4b4038f70 0x7fc4b403b430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:54:59.664 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:58.661+0000 7fc4cd400700 1 --2- 172.21.15.22:0/3700899547 >> [v2:172.21.15.22:6800/730635939,v1:172.21.15.22:6801/730635939] conn(0x7fc4b4038f70 0x7fc4b403b430 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fc4bc006fd0 tx=0x7fc4bc006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T17:54:59.664 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:58.666+0000 7fc4c6a00700 1 -- 172.21.15.22:0/3700899547 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fc4c004c800 con 0x7fc4d0106400 2024-03-28T17:54:59.665 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:58.898+0000 7fc4d5000700 1 -- 172.21.15.22:0/3700899547 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command([{prefix=config set, name=mgr/dashboard/ssl_server_port}] v 0) v1 -- 0x7fc4d00008d0 con 0x7fc4d0106400 2024-03-28T17:54:59.665 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:58.901+0000 7fc4c6a00700 1 -- 172.21.15.22:0/3700899547 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/dashboard/ssl_server_port}]=0 v8)=0 v8) v1 ==== 130+0+0 (secure 0 0 0) 0x7fc4c002d3f0 con 0x7fc4d0106400 2024-03-28T17:54:59.665 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:58.904+0000 7fc4d5000700 1 -- 172.21.15.22:0/3700899547 >> [v2:172.21.15.22:6800/730635939,v1:172.21.15.22:6801/730635939] conn(0x7fc4b4038f70 msgr2=0x7fc4b403b430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:54:59.665 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:58.905+0000 7fc4d5000700 1 --2- 172.21.15.22:0/3700899547 >> [v2:172.21.15.22:6800/730635939,v1:172.21.15.22:6801/730635939] conn(0x7fc4b4038f70 0x7fc4b403b430 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fc4bc006fd0 tx=0x7fc4bc006e40 comp rx=0 tx=0).stop 2024-03-28T17:54:59.665 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:58.905+0000 7fc4d5000700 1 -- 172.21.15.22:0/3700899547 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc4d0106400 msgr2=0x7fc4d019dda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:54:59.665 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:58.905+0000 7fc4d5000700 1 --2- 172.21.15.22:0/3700899547 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc4d0106400 0x7fc4d019dda0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fc4c0006af0 tx=0x7fc4c0011bd0 comp rx=0 tx=0).stop 2024-03-28T17:54:59.665 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:58.905+0000 7fc4d5000700 1 -- 172.21.15.22:0/3700899547 shutdown_connections 2024-03-28T17:54:59.665 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:58.905+0000 7fc4d5000700 1 --2- 172.21.15.22:0/3700899547 >> [v2:172.21.15.22:6800/730635939,v1:172.21.15.22:6801/730635939] conn(0x7fc4b4038f70 0x7fc4b403b430 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:54:59.665 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:58.905+0000 7fc4d5000700 1 --2- 172.21.15.22:0/3700899547 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc4d0106400 0x7fc4d019dda0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:54:59.665 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:58.905+0000 7fc4d5000700 1 -- 172.21.15.22:0/3700899547 >> 172.21.15.22:0/3700899547 conn(0x7fc4d01020a0 msgr2=0x7fc4d01949a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T17:54:59.665 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:58.905+0000 7fc4d5000700 1 -- 172.21.15.22:0/3700899547 shutdown_connections 2024-03-28T17:54:59.665 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:54:58.905+0000 7fc4d5000700 1 -- 172.21.15.22:0/3700899547 wait complete. 2024-03-28T17:54:59.665 INFO:teuthology.orchestra.run.smithi022.stdout:Enabling the dashboard module... 2024-03-28T17:55:01.371 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:55:01 smithi022 ceph-mon[31985]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-28T17:55:01.372 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:55:01 smithi022 ceph-mon[31985]: from='mgr.14120 172.21.15.22:0/1493984472' entity='mgr.x' 2024-03-28T17:55:01.372 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:55:01 smithi022 ceph-mon[31985]: from='client.? 172.21.15.22:0/578344946' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "dashboard"}]: dispatch 2024-03-28T17:55:01.627 INFO:journalctl@ceph.mgr.x.smithi022.stdout:Mar 28 17:55:01 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mgr-x[32211]: ignoring --setuser ceph since I am not root 2024-03-28T17:55:01.628 INFO:journalctl@ceph.mgr.x.smithi022.stdout:Mar 28 17:55:01 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mgr-x[32211]: ignoring --setgroup ceph since I am not root 2024-03-28T17:55:02.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:55:02 smithi022 ceph-mon[31985]: from='client.? 172.21.15.22:0/578344946' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "dashboard"}]': finished 2024-03-28T17:55:02.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:55:02 smithi022 ceph-mon[31985]: mgrmap e9: x(active, since 27s) 2024-03-28T17:55:02.532 INFO:journalctl@ceph.mgr.x.smithi022.stdout:Mar 28 17:55:02 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mgr-x[32211]: 2024-03-28T17:55:02.063+0000 7fd899c5d200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-03-28T17:55:02.532 INFO:journalctl@ceph.mgr.x.smithi022.stdout:Mar 28 17:55:02 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mgr-x[32211]: 2024-03-28T17:55:02.188+0000 7fd899c5d200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-03-28T17:55:02.747 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:55:00.284+0000 7fecdae00700 1 Processor -- start 2024-03-28T17:55:02.749 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:55:00.285+0000 7fecdae00700 1 -- start start 2024-03-28T17:55:02.749 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:55:00.285+0000 7fecdae00700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fecd4106aa0 0x7fecd4106ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:55:02.749 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:55:00.285+0000 7fecdae00700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fecd4107490 con 0x7fecd4106aa0 2024-03-28T17:55:02.749 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:55:00.285+0000 7fecd3e00700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fecd4106aa0 0x7fecd4106ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:55:02.749 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:55:00.285+0000 7fecd3e00700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fecd4106aa0 0x7fecd4106ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:44018/0 (socket says 172.21.15.22:44018) 2024-03-28T17:55:02.749 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:55:00.286+0000 7fecd3e00700 1 -- 172.21.15.22:0/2533624135 learned_addr learned my addr 172.21.15.22:0/2533624135 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-28T17:55:02.749 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:55:00.286+0000 7fecd3e00700 1 -- 172.21.15.22:0/2533624135 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fecd4107ca0 con 0x7fecd4106aa0 2024-03-28T17:55:02.749 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:55:00.286+0000 7fecd3e00700 1 --2- 172.21.15.22:0/2533624135 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fecd4106aa0 0x7fecd4106ec0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fecc40098a0 tx=0x7fecc4009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a581f5225803f8c9 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T17:55:02.749 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:55:00.287+0000 7fecd2a00700 1 -- 172.21.15.22:0/2533624135 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fecc4004160 con 0x7fecd4106aa0 2024-03-28T17:55:02.754 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:55:00.287+0000 7fecd2a00700 1 -- 172.21.15.22:0/2533624135 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7fecc4004300 con 0x7fecd4106aa0 2024-03-28T17:55:02.754 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:55:00.287+0000 7fecd2a00700 1 -- 172.21.15.22:0/2533624135 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fecc4011b10 con 0x7fecd4106aa0 2024-03-28T17:55:02.754 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:55:00.288+0000 7fecdae00700 1 -- 172.21.15.22:0/2533624135 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fecd4106aa0 msgr2=0x7fecd4106ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:55:02.754 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:55:00.288+0000 7fecdae00700 1 --2- 172.21.15.22:0/2533624135 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fecd4106aa0 0x7fecd4106ec0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fecc40098a0 tx=0x7fecc4009bb0 comp rx=0 tx=0).stop 2024-03-28T17:55:02.754 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:55:00.288+0000 7fecdae00700 1 -- 172.21.15.22:0/2533624135 shutdown_connections 2024-03-28T17:55:02.754 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:55:00.288+0000 7fecdae00700 1 --2- 172.21.15.22:0/2533624135 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fecd4106aa0 0x7fecd4106ec0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:55:02.754 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:55:00.288+0000 7fecdae00700 1 -- 172.21.15.22:0/2533624135 >> 172.21.15.22:0/2533624135 conn(0x7fecd41020a0 msgr2=0x7fecd41044c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T17:55:02.754 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:55:00.288+0000 7fecdae00700 1 -- 172.21.15.22:0/2533624135 shutdown_connections 2024-03-28T17:55:02.755 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:55:00.288+0000 7fecdae00700 1 -- 172.21.15.22:0/2533624135 wait complete. 2024-03-28T17:55:02.755 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:55:00.289+0000 7fecdae00700 1 Processor -- start 2024-03-28T17:55:02.755 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:55:00.289+0000 7fecdae00700 1 -- start start 2024-03-28T17:55:02.755 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:55:00.290+0000 7fecdae00700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fecd4106aa0 0x7fecd41a2200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:55:02.755 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:55:00.290+0000 7fecdae00700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fecd410b050 con 0x7fecd4106aa0 2024-03-28T17:55:02.755 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:55:00.290+0000 7fecd3e00700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fecd4106aa0 0x7fecd41a2200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:55:02.755 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:55:00.290+0000 7fecd3e00700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fecd4106aa0 0x7fecd41a2200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:44024/0 (socket says 172.21.15.22:44024) 2024-03-28T17:55:02.755 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:55:00.290+0000 7fecd3e00700 1 -- 172.21.15.22:0/578344946 learned_addr learned my addr 172.21.15.22:0/578344946 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-28T17:55:02.755 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:55:00.290+0000 7fecd3e00700 1 -- 172.21.15.22:0/578344946 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fecd41a2740 con 0x7fecd4106aa0 2024-03-28T17:55:02.755 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:55:00.291+0000 7fecd3e00700 1 --2- 172.21.15.22:0/578344946 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fecd4106aa0 0x7fecd41a2200 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fecc4006af0 tx=0x7fecc4011bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T17:55:02.755 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:55:00.291+0000 7fecd0c00700 1 -- 172.21.15.22:0/578344946 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fecc4003af0 con 0x7fecd4106aa0 2024-03-28T17:55:02.755 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:55:00.291+0000 7fecd0c00700 1 -- 172.21.15.22:0/578344946 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7fecc4003c90 con 0x7fecd4106aa0 2024-03-28T17:55:02.755 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:55:00.291+0000 7fecdae00700 1 -- 172.21.15.22:0/578344946 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fecd41a29d0 con 0x7fecd4106aa0 2024-03-28T17:55:02.755 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:55:00.291+0000 7fecd0c00700 1 -- 172.21.15.22:0/578344946 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fecc4024830 con 0x7fecd4106aa0 2024-03-28T17:55:02.755 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:55:00.291+0000 7fecdae00700 1 -- 172.21.15.22:0/578344946 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fecd41a5670 con 0x7fecd4106aa0 2024-03-28T17:55:02.756 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:55:00.292+0000 7fecd0c00700 1 -- 172.21.15.22:0/578344946 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 8) v1 ==== 46106+0+0 (secure 0 0 0) 0x7fecc4022020 con 0x7fecd4106aa0 2024-03-28T17:55:02.756 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:55:00.293+0000 7fecdae00700 1 -- 172.21.15.22:0/578344946 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fecd40637c0 con 0x7fecd4106aa0 2024-03-28T17:55:02.756 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:55:00.293+0000 7fecd0c00700 1 --2- 172.21.15.22:0/578344946 >> [v2:172.21.15.22:6800/730635939,v1:172.21.15.22:6801/730635939] conn(0x7fecb8038f20 0x7fecb803b3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:55:02.756 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:55:00.293+0000 7fecd0c00700 1 -- 172.21.15.22:0/578344946 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7fecc404cdf0 con 0x7fecd4106aa0 2024-03-28T17:55:02.756 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:55:00.294+0000 7fecd3400700 1 --2- 172.21.15.22:0/578344946 >> [v2:172.21.15.22:6800/730635939,v1:172.21.15.22:6801/730635939] conn(0x7fecb8038f20 0x7fecb803b3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:55:02.756 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:55:00.295+0000 7fecd3400700 1 --2- 172.21.15.22:0/578344946 >> [v2:172.21.15.22:6800/730635939,v1:172.21.15.22:6801/730635939] conn(0x7fecb8038f20 0x7fecb803b3e0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fecc0006fd0 tx=0x7fecc0006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T17:55:02.756 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:55:00.299+0000 7fecd0c00700 1 -- 172.21.15.22:0/578344946 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fecc402d3f0 con 0x7fecd4106aa0 2024-03-28T17:55:02.756 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:55:00.592+0000 7fecdae00700 1 -- 172.21.15.22:0/578344946 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "dashboard"} v 0) v1 -- 0x7fecd402dd00 con 0x7fecd4106aa0 2024-03-28T17:55:02.756 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:55:01.243+0000 7fecd0c00700 1 -- 172.21.15.22:0/578344946 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "dashboard"}]=0 v9) v1 ==== 88+0+0 (secure 0 0 0) 0x7fecc400bc90 con 0x7fecd4106aa0 2024-03-28T17:55:02.756 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:55:01.249+0000 7fecd0c00700 1 -- 172.21.15.22:0/578344946 <== mon.0 v2:172.21.15.22:3300/0 8 ==== mgrmap(e 9) v1 ==== 46119+0+0 (secure 0 0 0) 0x7fecc4017ca0 con 0x7fecd4106aa0 2024-03-28T17:55:02.756 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:55:01.250+0000 7fecdae00700 1 -- 172.21.15.22:0/578344946 >> [v2:172.21.15.22:6800/730635939,v1:172.21.15.22:6801/730635939] conn(0x7fecb8038f20 msgr2=0x7fecb803b3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:55:02.756 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:55:01.250+0000 7fecdae00700 1 --2- 172.21.15.22:0/578344946 >> [v2:172.21.15.22:6800/730635939,v1:172.21.15.22:6801/730635939] conn(0x7fecb8038f20 0x7fecb803b3e0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fecc0006fd0 tx=0x7fecc0006e40 comp rx=0 tx=0).stop 2024-03-28T17:55:02.756 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:55:01.250+0000 7fecdae00700 1 -- 172.21.15.22:0/578344946 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fecd4106aa0 msgr2=0x7fecd41a2200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:55:02.756 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:55:01.250+0000 7fecdae00700 1 --2- 172.21.15.22:0/578344946 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fecd4106aa0 0x7fecd41a2200 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fecc4006af0 tx=0x7fecc4011bd0 comp rx=0 tx=0).stop 2024-03-28T17:55:02.757 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:55:01.250+0000 7fecdae00700 1 -- 172.21.15.22:0/578344946 shutdown_connections 2024-03-28T17:55:02.757 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:55:01.250+0000 7fecdae00700 1 --2- 172.21.15.22:0/578344946 >> [v2:172.21.15.22:6800/730635939,v1:172.21.15.22:6801/730635939] conn(0x7fecb8038f20 0x7fecb803b3e0 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:55:02.757 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:55:01.250+0000 7fecdae00700 1 --2- 172.21.15.22:0/578344946 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fecd4106aa0 0x7fecd41a2200 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:55:02.757 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:55:01.250+0000 7fecdae00700 1 -- 172.21.15.22:0/578344946 >> 172.21.15.22:0/578344946 conn(0x7fecd41020a0 msgr2=0x7fecd4102f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T17:55:02.757 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:55:01.251+0000 7fecdae00700 1 -- 172.21.15.22:0/578344946 shutdown_connections 2024-03-28T17:55:02.757 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:55:01.251+0000 7fecdae00700 1 -- 172.21.15.22:0/578344946 wait complete. 2024-03-28T17:55:03.282 INFO:journalctl@ceph.mgr.x.smithi022.stdout:Mar 28 17:55:03 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mgr-x[32211]: 2024-03-28T17:55:03.021+0000 7fd899c5d200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-03-28T17:55:03.283 INFO:journalctl@ceph.mgr.x.smithi022.stdout:Mar 28 17:55:03 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mgr-x[32211]: 2024-03-28T17:55:03.156+0000 7fd899c5d200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-03-28T17:55:03.594 INFO:journalctl@ceph.mgr.x.smithi022.stdout:Mar 28 17:55:03 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mgr-x[32211]: 2024-03-28T17:55:03.345+0000 7fd899c5d200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-03-28T17:55:03.594 INFO:journalctl@ceph.mgr.x.smithi022.stdout:Mar 28 17:55:03 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mgr-x[32211]: 2024-03-28T17:55:03.477+0000 7fd899c5d200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-03-28T17:55:03.595 INFO:journalctl@ceph.mgr.x.smithi022.stdout:Mar 28 17:55:03 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mgr-x[32211]: 2024-03-28T17:55:03.593+0000 7fd899c5d200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-03-28T17:55:03.962 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:55:03 smithi022 ceph-mon[31985]: from='client.? 172.21.15.22:0/7174653' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-03-28T17:55:04.227 INFO:journalctl@ceph.mgr.x.smithi022.stdout:Mar 28 17:55:03 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mgr-x[32211]: 2024-03-28T17:55:03.961+0000 7fd899c5d200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-03-28T17:55:04.227 INFO:journalctl@ceph.mgr.x.smithi022.stdout:Mar 28 17:55:04 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mgr-x[32211]: 2024-03-28T17:55:04.108+0000 7fd899c5d200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-03-28T17:55:04.227 INFO:journalctl@ceph.mgr.x.smithi022.stdout:Mar 28 17:55:04 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mgr-x[32211]: 2024-03-28T17:55:04.225+0000 7fd899c5d200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-03-28T17:55:04.460 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout { 2024-03-28T17:55:04.460 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "epoch": 9, 2024-03-28T17:55:04.460 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "available": true, 2024-03-28T17:55:04.460 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "active_name": "x", 2024-03-28T17:55:04.460 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-03-28T17:55:04.460 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout } 2024-03-28T17:55:04.461 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:55:03.369+0000 7fe8a4a00700 1 Processor -- start 2024-03-28T17:55:04.461 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:55:03.369+0000 7fe8a4a00700 1 -- start start 2024-03-28T17:55:04.461 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:55:03.370+0000 7fe8a4a00700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe8a0109be0 0x7fe8a010a000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:55:04.461 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:55:03.370+0000 7fe8a4a00700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8a010a5d0 con 0x7fe8a0109be0 2024-03-28T17:55:04.461 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:55:03.370+0000 7fe89e800700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe8a0109be0 0x7fe8a010a000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:55:04.461 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:55:03.371+0000 7fe89e800700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe8a0109be0 0x7fe8a010a000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:44048/0 (socket says 172.21.15.22:44048) 2024-03-28T17:55:04.461 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:55:03.371+0000 7fe89e800700 1 -- 172.21.15.22:0/625128349 learned_addr learned my addr 172.21.15.22:0/625128349 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-28T17:55:04.461 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:55:03.371+0000 7fe89e800700 1 -- 172.21.15.22:0/625128349 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe8a010ad90 con 0x7fe8a0109be0 2024-03-28T17:55:04.461 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:55:03.371+0000 7fe89e800700 1 --2- 172.21.15.22:0/625128349 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe8a0109be0 0x7fe8a010a000 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fe8900098a0 tx=0x7fe890009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3821ff341f6c1de8 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T17:55:04.461 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:55:03.372+0000 7fe89d400700 1 -- 172.21.15.22:0/625128349 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fe890004160 con 0x7fe8a0109be0 2024-03-28T17:55:04.461 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:55:03.372+0000 7fe89d400700 1 -- 172.21.15.22:0/625128349 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7fe890004300 con 0x7fe8a0109be0 2024-03-28T17:55:04.461 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:55:03.372+0000 7fe89d400700 1 -- 172.21.15.22:0/625128349 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fe890011b10 con 0x7fe8a0109be0 2024-03-28T17:55:04.461 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:55:03.372+0000 7fe8a4a00700 1 -- 172.21.15.22:0/625128349 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe8a0109be0 msgr2=0x7fe8a010a000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:55:04.462 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:55:03.372+0000 7fe8a4a00700 1 --2- 172.21.15.22:0/625128349 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe8a0109be0 0x7fe8a010a000 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fe8900098a0 tx=0x7fe890009bb0 comp rx=0 tx=0).stop 2024-03-28T17:55:04.462 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:55:03.373+0000 7fe8a4a00700 1 -- 172.21.15.22:0/625128349 shutdown_connections 2024-03-28T17:55:04.462 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:55:03.373+0000 7fe8a4a00700 1 --2- 172.21.15.22:0/625128349 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe8a0109be0 0x7fe8a010a000 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:55:04.462 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:55:03.373+0000 7fe8a4a00700 1 -- 172.21.15.22:0/625128349 >> 172.21.15.22:0/625128349 conn(0x7fe8a007cfe0 msgr2=0x7fe8a007d3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T17:55:04.462 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:55:03.373+0000 7fe8a4a00700 1 -- 172.21.15.22:0/625128349 shutdown_connections 2024-03-28T17:55:04.462 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:55:03.373+0000 7fe8a4a00700 1 -- 172.21.15.22:0/625128349 wait complete. 2024-03-28T17:55:04.466 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:55:03.374+0000 7fe8a4a00700 1 Processor -- start 2024-03-28T17:55:04.466 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:55:03.374+0000 7fe8a4a00700 1 -- start start 2024-03-28T17:55:04.466 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:55:03.374+0000 7fe8a4a00700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe8a0109be0 0x7fe8a019dbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:55:04.466 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:55:03.374+0000 7fe8a4a00700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8a010e140 con 0x7fe8a0109be0 2024-03-28T17:55:04.466 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:55:03.375+0000 7fe89e800700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe8a0109be0 0x7fe8a019dbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:55:04.466 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:55:03.375+0000 7fe89e800700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe8a0109be0 0x7fe8a019dbd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:44052/0 (socket says 172.21.15.22:44052) 2024-03-28T17:55:04.466 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:55:03.375+0000 7fe89e800700 1 -- 172.21.15.22:0/7174653 learned_addr learned my addr 172.21.15.22:0/7174653 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-28T17:55:04.466 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:55:03.375+0000 7fe89e800700 1 -- 172.21.15.22:0/7174653 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe8a019e110 con 0x7fe8a0109be0 2024-03-28T17:55:04.466 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:55:03.375+0000 7fe89e800700 1 --2- 172.21.15.22:0/7174653 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe8a0109be0 0x7fe8a019dbd0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fe890009fb0 tx=0x7fe890011bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T17:55:04.466 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:55:03.376+0000 7fe897400700 1 -- 172.21.15.22:0/7174653 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fe890003b20 con 0x7fe8a0109be0 2024-03-28T17:55:04.466 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:55:03.376+0000 7fe897400700 1 -- 172.21.15.22:0/7174653 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7fe890003cc0 con 0x7fe8a0109be0 2024-03-28T17:55:04.467 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:55:03.376+0000 7fe897400700 1 -- 172.21.15.22:0/7174653 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fe8900247a0 con 0x7fe8a0109be0 2024-03-28T17:55:04.467 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:55:03.376+0000 7fe8a4a00700 1 -- 172.21.15.22:0/7174653 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe8a019e3a0 con 0x7fe8a0109be0 2024-03-28T17:55:04.467 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:55:03.376+0000 7fe8a4a00700 1 -- 172.21.15.22:0/7174653 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe8a01a1090 con 0x7fe8a0109be0 2024-03-28T17:55:04.467 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:55:03.377+0000 7fe8a4a00700 1 -- 172.21.15.22:0/7174653 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe8a00637c0 con 0x7fe8a0109be0 2024-03-28T17:55:04.467 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:55:03.377+0000 7fe897400700 1 -- 172.21.15.22:0/7174653 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 9) v1 ==== 46119+0+0 (secure 0 0 0) 0x7fe890022020 con 0x7fe8a0109be0 2024-03-28T17:55:04.467 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:55:03.378+0000 7fe897400700 1 --2- 172.21.15.22:0/7174653 >> [v2:172.21.15.22:6800/730635939,v1:172.21.15.22:6801/730635939] conn(0x7fe884038fd0 0x7fe88403b490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:55:04.467 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:55:03.378+0000 7fe897400700 1 -- 172.21.15.22:0/7174653 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7fe89004cfa0 con 0x7fe8a0109be0 2024-03-28T17:55:04.467 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:55:03.378+0000 7fe89de00700 1 -- 172.21.15.22:0/7174653 >> [v2:172.21.15.22:6800/730635939,v1:172.21.15.22:6801/730635939] conn(0x7fe884038fd0 msgr2=0x7fe88403b490 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.22:6800/730635939 2024-03-28T17:55:04.467 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:55:03.378+0000 7fe89de00700 1 --2- 172.21.15.22:0/7174653 >> [v2:172.21.15.22:6800/730635939,v1:172.21.15.22:6801/730635939] conn(0x7fe884038fd0 0x7fe88403b490 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-28T17:55:04.467 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:55:03.383+0000 7fe897400700 1 -- 172.21.15.22:0/7174653 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fe890016260 con 0x7fe8a0109be0 2024-03-28T17:55:04.467 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:55:03.579+0000 7fe89de00700 1 -- 172.21.15.22:0/7174653 >> [v2:172.21.15.22:6800/730635939,v1:172.21.15.22:6801/730635939] conn(0x7fe884038fd0 msgr2=0x7fe88403b490 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.22:6800/730635939 2024-03-28T17:55:04.467 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:55:03.579+0000 7fe89de00700 1 --2- 172.21.15.22:0/7174653 >> [v2:172.21.15.22:6800/730635939,v1:172.21.15.22:6801/730635939] conn(0x7fe884038fd0 0x7fe88403b490 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-28T17:55:04.467 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:55:03.666+0000 7fe8a4a00700 1 -- 172.21.15.22:0/7174653 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "mgr stat"} v 0) v1 -- 0x7fe8a01a1300 con 0x7fe8a0109be0 2024-03-28T17:55:04.467 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:55:03.667+0000 7fe897400700 1 -- 172.21.15.22:0/7174653 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v9) v1 ==== 56+0+88 (secure 0 0 0) 0x7fe89004c8f0 con 0x7fe8a0109be0 2024-03-28T17:55:04.468 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:55:03.671+0000 7fe8a4a00700 1 -- 172.21.15.22:0/7174653 >> [v2:172.21.15.22:6800/730635939,v1:172.21.15.22:6801/730635939] conn(0x7fe884038fd0 msgr2=0x7fe88403b490 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-28T17:55:04.468 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:55:03.671+0000 7fe8a4a00700 1 --2- 172.21.15.22:0/7174653 >> [v2:172.21.15.22:6800/730635939,v1:172.21.15.22:6801/730635939] conn(0x7fe884038fd0 0x7fe88403b490 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:55:04.468 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:55:03.671+0000 7fe8a4a00700 1 -- 172.21.15.22:0/7174653 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe8a0109be0 msgr2=0x7fe8a019dbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:55:04.468 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:55:03.671+0000 7fe8a4a00700 1 --2- 172.21.15.22:0/7174653 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe8a0109be0 0x7fe8a019dbd0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fe890009fb0 tx=0x7fe890011bd0 comp rx=0 tx=0).stop 2024-03-28T17:55:04.468 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:55:03.672+0000 7fe8a4a00700 1 -- 172.21.15.22:0/7174653 shutdown_connections 2024-03-28T17:55:04.468 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:55:03.672+0000 7fe8a4a00700 1 --2- 172.21.15.22:0/7174653 >> [v2:172.21.15.22:6800/730635939,v1:172.21.15.22:6801/730635939] conn(0x7fe884038fd0 0x7fe88403b490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:55:04.468 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:55:03.672+0000 7fe8a4a00700 1 --2- 172.21.15.22:0/7174653 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe8a0109be0 0x7fe8a019dbd0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:55:04.468 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:55:03.672+0000 7fe8a4a00700 1 -- 172.21.15.22:0/7174653 >> 172.21.15.22:0/7174653 conn(0x7fe8a007cfe0 msgr2=0x7fe8a0107600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T17:55:04.468 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:55:03.672+0000 7fe8a4a00700 1 -- 172.21.15.22:0/7174653 shutdown_connections 2024-03-28T17:55:04.468 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:55:03.672+0000 7fe8a4a00700 1 -- 172.21.15.22:0/7174653 wait complete. 2024-03-28T17:55:04.468 INFO:teuthology.orchestra.run.smithi022.stdout:Waiting for the mgr to restart... 2024-03-28T17:55:04.468 INFO:teuthology.orchestra.run.smithi022.stdout:Waiting for mgr epoch 9... 2024-03-28T17:55:04.531 INFO:journalctl@ceph.mgr.x.smithi022.stdout:Mar 28 17:55:04 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mgr-x[32211]: 2024-03-28T17:55:04.355+0000 7fd899c5d200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-03-28T17:55:04.883 INFO:journalctl@ceph.mgr.x.smithi022.stdout:Mar 28 17:55:04 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mgr-x[32211]: 2024-03-28T17:55:04.580+0000 7fd899c5d200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-03-28T17:55:06.282 INFO:journalctl@ceph.mgr.x.smithi022.stdout:Mar 28 17:55:05 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mgr-x[32211]: 2024-03-28T17:55:05.906+0000 7fd899c5d200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-03-28T17:55:06.282 INFO:journalctl@ceph.mgr.x.smithi022.stdout:Mar 28 17:55:06 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mgr-x[32211]: 2024-03-28T17:55:06.083+0000 7fd899c5d200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-03-28T17:55:06.782 INFO:journalctl@ceph.mgr.x.smithi022.stdout:Mar 28 17:55:06 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mgr-x[32211]: 2024-03-28T17:55:06.514+0000 7fd899c5d200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-03-28T17:55:06.782 INFO:journalctl@ceph.mgr.x.smithi022.stdout:Mar 28 17:55:06 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mgr-x[32211]: 2024-03-28T17:55:06.636+0000 7fd899c5d200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-03-28T17:55:07.163 INFO:journalctl@ceph.mgr.x.smithi022.stdout:Mar 28 17:55:06 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mgr-x[32211]: 2024-03-28T17:55:06.904+0000 7fd899c5d200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-03-28T17:55:07.531 INFO:journalctl@ceph.mgr.x.smithi022.stdout:Mar 28 17:55:07 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mgr-x[32211]: 2024-03-28T17:55:07.161+0000 7fd899c5d200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-03-28T17:55:07.879 INFO:journalctl@ceph.mgr.x.smithi022.stdout:Mar 28 17:55:07 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mgr-x[32211]: 2024-03-28T17:55:07.552+0000 7fd899c5d200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-03-28T17:55:08.197 INFO:journalctl@ceph.mgr.x.smithi022.stdout:Mar 28 17:55:07 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mgr-x[32211]: 2024-03-28T17:55:07.877+0000 7fd899c5d200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-03-28T17:55:08.532 INFO:journalctl@ceph.mgr.x.smithi022.stdout:Mar 28 17:55:08 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mgr-x[32211]: 2024-03-28T17:55:08.195+0000 7fd899c5d200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-03-28T17:55:09.532 INFO:journalctl@ceph.mgr.x.smithi022.stdout:Mar 28 17:55:09 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mgr-x[32211]: 2024-03-28T17:55:09.271+0000 7fd899c5d200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-03-28T17:55:10.282 INFO:journalctl@ceph.mgr.x.smithi022.stdout:Mar 28 17:55:09 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mgr-x[32211]: 2024-03-28T17:55:09.992+0000 7fd899c5d200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-03-28T17:55:10.282 INFO:journalctl@ceph.mgr.x.smithi022.stdout:Mar 28 17:55:10 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mgr-x[32211]: 2024-03-28T17:55:10.127+0000 7fd899c5d200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-03-28T17:55:10.782 INFO:journalctl@ceph.mgr.x.smithi022.stdout:Mar 28 17:55:10 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mgr-x[32211]: 2024-03-28T17:55:10.421+0000 7fd899c5d200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-03-28T17:55:10.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:55:10 smithi022 ceph-mon[31985]: Active manager daemon x restarted 2024-03-28T17:55:10.783 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:55:10 smithi022 ceph-mon[31985]: Activating manager daemon x 2024-03-28T17:55:10.783 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:55:10 smithi022 ceph-mon[31985]: osdmap e3: 0 total, 0 up, 0 in 2024-03-28T17:55:10.783 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:55:10 smithi022 ceph-mon[31985]: mgrmap e10: x(active, starting, since 0.00979263s) 2024-03-28T17:55:10.783 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:55:10 smithi022 ceph-mon[31985]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-28T17:55:10.783 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:55:10 smithi022 ceph-mon[31985]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-03-28T17:55:10.783 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:55:10 smithi022 ceph-mon[31985]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-03-28T17:55:10.783 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:55:10 smithi022 ceph-mon[31985]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-03-28T17:55:10.783 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:55:10 smithi022 ceph-mon[31985]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-03-28T17:55:10.783 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:55:10 smithi022 ceph-mon[31985]: Manager daemon x is now available 2024-03-28T17:55:11.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:55:11 smithi022 ceph-mon[31985]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T17:55:11.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:55:11 smithi022 ceph-mon[31985]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T17:55:11.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:55:11 smithi022 ceph-mon[31985]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-03-28T17:55:11.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:55:11 smithi022 ceph-mon[31985]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-03-28T17:55:11.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:55:11 smithi022 ceph-mon[31985]: [28/Mar/2024:17:55:11] ENGINE Bus STARTING 2024-03-28T17:55:11.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:55:11 smithi022 ceph-mon[31985]: [28/Mar/2024:17:55:11] ENGINE Serving on http://172.21.15.22:8765 2024-03-28T17:55:11.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:55:11 smithi022 ceph-mon[31985]: [28/Mar/2024:17:55:11] ENGINE Serving on https://172.21.15.22:7150 2024-03-28T17:55:11.783 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:55:11 smithi022 ceph-mon[31985]: [28/Mar/2024:17:55:11] ENGINE Bus STARTED 2024-03-28T17:55:11.783 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:55:11 smithi022 ceph-mon[31985]: mgrmap e11: x(active, since 1.00952s) 2024-03-28T17:55:12.385 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout { 2024-03-28T17:55:12.385 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 11, 2024-03-28T17:55:12.385 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "initialized": true 2024-03-28T17:55:12.385 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout } 2024-03-28T17:55:12.385 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:55:05.033+0000 7ff7ce000700 1 Processor -- start 2024-03-28T17:55:12.385 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:55:05.034+0000 7ff7ce000700 1 -- start start 2024-03-28T17:55:12.386 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:55:05.034+0000 7ff7ce000700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff7c81086d0 0x7ff7c810aac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:55:12.386 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:55:05.034+0000 7ff7ce000700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7c80758b0 con 0x7ff7c81086d0 2024-03-28T17:55:12.386 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:55:05.034+0000 7ff7c7400700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff7c81086d0 0x7ff7c810aac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:55:12.392 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:55:05.034+0000 7ff7c7400700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff7c81086d0 0x7ff7c810aac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:44054/0 (socket says 172.21.15.22:44054) 2024-03-28T17:55:12.392 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:55:05.034+0000 7ff7c7400700 1 -- 172.21.15.22:0/2378541443 learned_addr learned my addr 172.21.15.22:0/2378541443 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-28T17:55:12.393 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:55:05.035+0000 7ff7c7400700 1 -- 172.21.15.22:0/2378541443 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff7c810b000 con 0x7ff7c81086d0 2024-03-28T17:55:12.393 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:55:05.035+0000 7ff7c7400700 1 --2- 172.21.15.22:0/2378541443 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff7c81086d0 0x7ff7c810aac0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7ff7b80098a0 tx=0x7ff7b8009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b793ff314c60e24e server_cookie=0 in_seq=0 out_seq=0 2024-03-28T17:55:12.393 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:55:05.035+0000 7ff7c6a00700 1 -- 172.21.15.22:0/2378541443 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ff7b8004160 con 0x7ff7c81086d0 2024-03-28T17:55:12.393 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:55:05.036+0000 7ff7c6a00700 1 -- 172.21.15.22:0/2378541443 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7ff7b8004300 con 0x7ff7c81086d0 2024-03-28T17:55:12.393 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:55:05.036+0000 7ff7c6a00700 1 -- 172.21.15.22:0/2378541443 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ff7b8011b10 con 0x7ff7c81086d0 2024-03-28T17:55:12.393 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:55:05.036+0000 7ff7ce000700 1 -- 172.21.15.22:0/2378541443 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff7c81086d0 msgr2=0x7ff7c810aac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:55:12.393 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:55:05.036+0000 7ff7ce000700 1 --2- 172.21.15.22:0/2378541443 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff7c81086d0 0x7ff7c810aac0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7ff7b80098a0 tx=0x7ff7b8009bb0 comp rx=0 tx=0).stop 2024-03-28T17:55:12.393 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:55:05.036+0000 7ff7ce000700 1 -- 172.21.15.22:0/2378541443 shutdown_connections 2024-03-28T17:55:12.393 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:55:05.036+0000 7ff7ce000700 1 --2- 172.21.15.22:0/2378541443 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff7c81086d0 0x7ff7c810aac0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:55:12.393 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:55:05.036+0000 7ff7ce000700 1 -- 172.21.15.22:0/2378541443 >> 172.21.15.22:0/2378541443 conn(0x7ff7c81020a0 msgr2=0x7ff7c81044c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T17:55:12.393 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:55:05.037+0000 7ff7ce000700 1 -- 172.21.15.22:0/2378541443 shutdown_connections 2024-03-28T17:55:12.393 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:55:05.037+0000 7ff7ce000700 1 -- 172.21.15.22:0/2378541443 wait complete. 2024-03-28T17:55:12.393 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:55:05.038+0000 7ff7ce000700 1 Processor -- start 2024-03-28T17:55:12.393 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:55:05.038+0000 7ff7ce000700 1 -- start start 2024-03-28T17:55:12.394 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:55:05.038+0000 7ff7ce000700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff7c81086d0 0x7ff7c8107f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:55:12.394 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:55:05.038+0000 7ff7ce000700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7c810bfb0 con 0x7ff7c81086d0 2024-03-28T17:55:12.394 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:55:05.038+0000 7ff7c7400700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff7c81086d0 0x7ff7c8107f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:55:12.394 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:55:05.038+0000 7ff7c7400700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff7c81086d0 0x7ff7c8107f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:44056/0 (socket says 172.21.15.22:44056) 2024-03-28T17:55:12.394 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:55:05.038+0000 7ff7c7400700 1 -- 172.21.15.22:0/1271160199 learned_addr learned my addr 172.21.15.22:0/1271160199 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-28T17:55:12.394 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:55:05.039+0000 7ff7c7400700 1 -- 172.21.15.22:0/1271160199 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff7c8108490 con 0x7ff7c81086d0 2024-03-28T17:55:12.394 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:55:05.039+0000 7ff7c7400700 1 --2- 172.21.15.22:0/1271160199 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff7c81086d0 0x7ff7c8107f50 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7ff7b8006af0 tx=0x7ff7b8011bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T17:55:12.394 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:55:05.039+0000 7ff7c4c00700 1 -- 172.21.15.22:0/1271160199 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ff7b8003af0 con 0x7ff7c81086d0 2024-03-28T17:55:12.394 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:55:05.039+0000 7ff7c4c00700 1 -- 172.21.15.22:0/1271160199 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7ff7b8003c90 con 0x7ff7c81086d0 2024-03-28T17:55:12.394 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:55:05.039+0000 7ff7c4c00700 1 -- 172.21.15.22:0/1271160199 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ff7b8024830 con 0x7ff7c81086d0 2024-03-28T17:55:12.394 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:55:05.039+0000 7ff7ce000700 1 -- 172.21.15.22:0/1271160199 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff7c8106680 con 0x7ff7c81086d0 2024-03-28T17:55:12.394 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:55:05.040+0000 7ff7ce000700 1 -- 172.21.15.22:0/1271160199 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff7c8106bc0 con 0x7ff7c81086d0 2024-03-28T17:55:12.394 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:55:05.041+0000 7ff7c4c00700 1 -- 172.21.15.22:0/1271160199 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 9) v1 ==== 46119+0+0 (secure 0 0 0) 0x7ff7b8022020 con 0x7ff7c81086d0 2024-03-28T17:55:12.395 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:55:05.041+0000 7ff7c4c00700 1 --2- 172.21.15.22:0/1271160199 >> [v2:172.21.15.22:6800/730635939,v1:172.21.15.22:6801/730635939] conn(0x7ff7ac038fd0 0x7ff7ac03b490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:55:12.395 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:55:05.041+0000 7ff7c4c00700 1 -- 172.21.15.22:0/1271160199 --> [v2:172.21.15.22:6800/730635939,v1:172.21.15.22:6801/730635939] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x7ff7ac03bba0 con 0x7ff7ac038fd0 2024-03-28T17:55:12.395 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:55:05.041+0000 7ff7c4c00700 1 -- 172.21.15.22:0/1271160199 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7ff7b804ce30 con 0x7ff7c81086d0 2024-03-28T17:55:12.395 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:55:05.041+0000 7ff7bfe00700 1 -- 172.21.15.22:0/1271160199 >> [v2:172.21.15.22:6800/730635939,v1:172.21.15.22:6801/730635939] conn(0x7ff7ac038fd0 msgr2=0x7ff7ac03b490 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.22:6800/730635939 2024-03-28T17:55:12.395 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:55:05.041+0000 7ff7bfe00700 1 --2- 172.21.15.22:0/1271160199 >> [v2:172.21.15.22:6800/730635939,v1:172.21.15.22:6801/730635939] conn(0x7ff7ac038fd0 0x7ff7ac03b490 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-28T17:55:12.395 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:55:05.242+0000 7ff7bfe00700 1 -- 172.21.15.22:0/1271160199 >> [v2:172.21.15.22:6800/730635939,v1:172.21.15.22:6801/730635939] conn(0x7ff7ac038fd0 msgr2=0x7ff7ac03b490 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.22:6800/730635939 2024-03-28T17:55:12.395 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:55:05.242+0000 7ff7bfe00700 1 --2- 172.21.15.22:0/1271160199 >> [v2:172.21.15.22:6800/730635939,v1:172.21.15.22:6801/730635939] conn(0x7ff7ac038fd0 0x7ff7ac03b490 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-28T17:55:12.395 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:55:05.643+0000 7ff7bfe00700 1 -- 172.21.15.22:0/1271160199 >> [v2:172.21.15.22:6800/730635939,v1:172.21.15.22:6801/730635939] conn(0x7ff7ac038fd0 msgr2=0x7ff7ac03b490 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.22:6800/730635939 2024-03-28T17:55:12.395 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:55:05.643+0000 7ff7bfe00700 1 --2- 172.21.15.22:0/1271160199 >> [v2:172.21.15.22:6800/730635939,v1:172.21.15.22:6801/730635939] conn(0x7ff7ac038fd0 0x7ff7ac03b490 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-28T17:55:12.395 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:55:06.444+0000 7ff7bfe00700 1 -- 172.21.15.22:0/1271160199 >> [v2:172.21.15.22:6800/730635939,v1:172.21.15.22:6801/730635939] conn(0x7ff7ac038fd0 msgr2=0x7ff7ac03b490 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.22:6800/730635939 2024-03-28T17:55:12.395 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:55:06.444+0000 7ff7bfe00700 1 --2- 172.21.15.22:0/1271160199 >> [v2:172.21.15.22:6800/730635939,v1:172.21.15.22:6801/730635939] conn(0x7ff7ac038fd0 0x7ff7ac03b490 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-28T17:55:12.395 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:55:08.046+0000 7ff7bfe00700 1 -- 172.21.15.22:0/1271160199 >> [v2:172.21.15.22:6800/730635939,v1:172.21.15.22:6801/730635939] conn(0x7ff7ac038fd0 msgr2=0x7ff7ac03b490 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.22:6800/730635939 2024-03-28T17:55:12.395 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:55:08.046+0000 7ff7bfe00700 1 --2- 172.21.15.22:0/1271160199 >> [v2:172.21.15.22:6800/730635939,v1:172.21.15.22:6801/730635939] conn(0x7ff7ac038fd0 0x7ff7ac03b490 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2024-03-28T17:55:12.395 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:55:10.430+0000 7ff7c4c00700 1 -- 172.21.15.22:0/1271160199 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mgrmap(e 10) v1 ==== 45886+0+0 (secure 0 0 0) 0x7ff7b801a430 con 0x7ff7c81086d0 2024-03-28T17:55:12.396 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:55:10.431+0000 7ff7c4c00700 1 -- 172.21.15.22:0/1271160199 >> [v2:172.21.15.22:6800/730635939,v1:172.21.15.22:6801/730635939] conn(0x7ff7ac038fd0 msgr2=0x7ff7ac03b490 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-28T17:55:12.396 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:55:10.431+0000 7ff7c4c00700 1 --2- 172.21.15.22:0/1271160199 >> [v2:172.21.15.22:6800/730635939,v1:172.21.15.22:6801/730635939] conn(0x7ff7ac038fd0 0x7ff7ac03b490 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:55:12.396 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:55:11.430+0000 7ff7c4c00700 1 -- 172.21.15.22:0/1271160199 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mgrmap(e 11) v1 ==== 46013+0+0 (secure 0 0 0) 0x7ff7b8016cc0 con 0x7ff7c81086d0 2024-03-28T17:55:12.396 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:55:11.431+0000 7ff7c4c00700 1 --2- 172.21.15.22:0/1271160199 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7ff7ac03ca80 0x7ff7ac03ee70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:55:12.396 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:55:11.431+0000 7ff7c4c00700 1 -- 172.21.15.22:0/1271160199 --> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x7ff7b800ce90 con 0x7ff7ac03ca80 2024-03-28T17:55:12.396 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:55:11.432+0000 7ff7bfe00700 1 --2- 172.21.15.22:0/1271160199 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7ff7ac03ca80 0x7ff7ac03ee70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:55:12.396 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:55:11.433+0000 7ff7bfe00700 1 --2- 172.21.15.22:0/1271160199 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7ff7ac03ca80 0x7ff7ac03ee70 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7ff7b0003a10 tx=0x7ff7b00092b0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T17:55:12.396 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:55:11.434+0000 7ff7c4c00700 1 -- 172.21.15.22:0/1271160199 <== mgr.14152 v2:172.21.15.22:6800/4201048392 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+7774 (secure 0 0 0) 0x7ff7b800ce90 con 0x7ff7ac03ca80 2024-03-28T17:55:12.396 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:55:11.442+0000 7ff7ce000700 1 -- 172.21.15.22:0/1271160199 --> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] -- command(tid 1: {"prefix": "mgr_status"}) v1 -- 0x7ff7c80637c0 con 0x7ff7ac03ca80 2024-03-28T17:55:12.396 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:55:11.443+0000 7ff7c4c00700 1 -- 172.21.15.22:0/1271160199 <== mgr.14152 v2:172.21.15.22:6800/4201048392 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+52 (secure 0 0 0) 0x7ff7c80637c0 con 0x7ff7ac03ca80 2024-03-28T17:55:12.396 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:55:11.443+0000 7ff7ce000700 1 -- 172.21.15.22:0/1271160199 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7ff7ac03ca80 msgr2=0x7ff7ac03ee70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:55:12.396 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:55:11.443+0000 7ff7ce000700 1 --2- 172.21.15.22:0/1271160199 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7ff7ac03ca80 0x7ff7ac03ee70 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7ff7b0003a10 tx=0x7ff7b00092b0 comp rx=0 tx=0).stop 2024-03-28T17:55:12.396 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:55:11.443+0000 7ff7ce000700 1 -- 172.21.15.22:0/1271160199 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff7c81086d0 msgr2=0x7ff7c8107f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:55:12.396 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:55:11.443+0000 7ff7ce000700 1 --2- 172.21.15.22:0/1271160199 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff7c81086d0 0x7ff7c8107f50 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7ff7b8006af0 tx=0x7ff7b8011bd0 comp rx=0 tx=0).stop 2024-03-28T17:55:12.397 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:55:11.443+0000 7ff7ce000700 1 -- 172.21.15.22:0/1271160199 shutdown_connections 2024-03-28T17:55:12.397 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:55:11.443+0000 7ff7ce000700 1 --2- 172.21.15.22:0/1271160199 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7ff7ac03ca80 0x7ff7ac03ee70 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:55:12.397 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:55:11.443+0000 7ff7ce000700 1 --2- 172.21.15.22:0/1271160199 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff7c81086d0 0x7ff7c8107f50 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:55:12.397 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:55:11.443+0000 7ff7ce000700 1 --2- 172.21.15.22:0/1271160199 >> [v2:172.21.15.22:6800/730635939,v1:172.21.15.22:6801/730635939] conn(0x7ff7ac038fd0 0x7ff7ac03b490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:55:12.397 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:55:11.443+0000 7ff7ce000700 1 -- 172.21.15.22:0/1271160199 >> 172.21.15.22:0/1271160199 conn(0x7ff7c81020a0 msgr2=0x7ff7c8102b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T17:55:12.397 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:55:11.444+0000 7ff7ce000700 1 -- 172.21.15.22:0/1271160199 shutdown_connections 2024-03-28T17:55:12.397 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:55:11.444+0000 7ff7ce000700 1 -- 172.21.15.22:0/1271160199 wait complete. 2024-03-28T17:55:12.397 INFO:teuthology.orchestra.run.smithi022.stdout:mgr epoch 9 is available 2024-03-28T17:55:12.397 INFO:teuthology.orchestra.run.smithi022.stdout:Generating a dashboard self-signed certificate... 2024-03-28T17:55:12.545 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:55:12 smithi022 ceph-mon[31985]: from='client.14156 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2024-03-28T17:55:12.545 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:55:12 smithi022 ceph-mon[31985]: from='client.14156 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2024-03-28T17:55:13.783 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:55:13 smithi022 ceph-mon[31985]: mgrmap e12: x(active, since 2s) 2024-03-28T17:55:13.783 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:55:13 smithi022 ceph-mon[31985]: from='client.14164 -' entity='client.admin' cmd=[{"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}]: dispatch 2024-03-28T17:55:13.783 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:55:13 smithi022 ceph-mon[31985]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T17:55:13.783 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:55:13 smithi022 ceph-mon[31985]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T17:55:14.324 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout Self-signed certificate created 2024-03-28T17:55:14.324 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:55:13.060+0000 7f5d78200700 1 Processor -- start 2024-03-28T17:55:14.324 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:55:13.060+0000 7f5d78200700 1 -- start start 2024-03-28T17:55:14.324 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:55:13.060+0000 7f5d78200700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5d70109db0 0x7f5d7010a1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:55:14.324 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:55:13.060+0000 7f5d78200700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d7010a7a0 con 0x7f5d70109db0 2024-03-28T17:55:14.324 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:55:13.061+0000 7f5d75800700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5d70109db0 0x7f5d7010a1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:55:14.324 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:55:13.061+0000 7f5d75800700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5d70109db0 0x7f5d7010a1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:56442/0 (socket says 172.21.15.22:56442) 2024-03-28T17:55:14.324 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:55:13.061+0000 7f5d75800700 1 -- 172.21.15.22:0/1759461940 learned_addr learned my addr 172.21.15.22:0/1759461940 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-28T17:55:14.324 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:55:13.062+0000 7f5d75800700 1 -- 172.21.15.22:0/1759461940 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d7010af60 con 0x7f5d70109db0 2024-03-28T17:55:14.324 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:55:13.062+0000 7f5d75800700 1 --2- 172.21.15.22:0/1759461940 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5d70109db0 0x7f5d7010a1d0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f5d600098a0 tx=0x7f5d60009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=87f5d9081b222b51 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T17:55:14.325 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:55:13.062+0000 7f5d6fe00700 1 -- 172.21.15.22:0/1759461940 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f5d60004160 con 0x7f5d70109db0 2024-03-28T17:55:14.325 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:55:13.062+0000 7f5d6fe00700 1 -- 172.21.15.22:0/1759461940 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f5d60004300 con 0x7f5d70109db0 2024-03-28T17:55:14.325 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:55:13.062+0000 7f5d6fe00700 1 -- 172.21.15.22:0/1759461940 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f5d60011b10 con 0x7f5d70109db0 2024-03-28T17:55:14.325 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:55:13.063+0000 7f5d78200700 1 -- 172.21.15.22:0/1759461940 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5d70109db0 msgr2=0x7f5d7010a1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:55:14.325 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:55:13.063+0000 7f5d78200700 1 --2- 172.21.15.22:0/1759461940 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5d70109db0 0x7f5d7010a1d0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f5d600098a0 tx=0x7f5d60009bb0 comp rx=0 tx=0).stop 2024-03-28T17:55:14.325 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:55:13.063+0000 7f5d78200700 1 -- 172.21.15.22:0/1759461940 shutdown_connections 2024-03-28T17:55:14.325 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:55:13.063+0000 7f5d78200700 1 --2- 172.21.15.22:0/1759461940 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5d70109db0 0x7f5d7010a1d0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:55:14.325 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:55:13.063+0000 7f5d78200700 1 -- 172.21.15.22:0/1759461940 >> 172.21.15.22:0/1759461940 conn(0x7f5d7007d150 msgr2=0x7f5d7007d5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T17:55:14.325 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:55:13.063+0000 7f5d78200700 1 -- 172.21.15.22:0/1759461940 shutdown_connections 2024-03-28T17:55:14.325 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:55:13.063+0000 7f5d78200700 1 -- 172.21.15.22:0/1759461940 wait complete. 2024-03-28T17:55:14.325 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:55:13.064+0000 7f5d78200700 1 Processor -- start 2024-03-28T17:55:14.325 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:55:13.065+0000 7f5d78200700 1 -- start start 2024-03-28T17:55:14.325 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:55:13.065+0000 7f5d78200700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5d70109db0 0x7f5d7019dda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:55:14.325 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:55:13.065+0000 7f5d78200700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d7010e310 con 0x7f5d70109db0 2024-03-28T17:55:14.325 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:55:13.065+0000 7f5d75800700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5d70109db0 0x7f5d7019dda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:55:14.326 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:55:13.065+0000 7f5d75800700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5d70109db0 0x7f5d7019dda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:56444/0 (socket says 172.21.15.22:56444) 2024-03-28T17:55:14.326 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:55:13.065+0000 7f5d75800700 1 -- 172.21.15.22:0/2233643617 learned_addr learned my addr 172.21.15.22:0/2233643617 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-28T17:55:14.326 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:55:13.066+0000 7f5d75800700 1 -- 172.21.15.22:0/2233643617 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d7019e2e0 con 0x7f5d70109db0 2024-03-28T17:55:14.326 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:55:13.066+0000 7f5d75800700 1 --2- 172.21.15.22:0/2233643617 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5d70109db0 0x7f5d7019dda0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f5d60006af0 tx=0x7f5d60011bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T17:55:14.326 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:55:13.066+0000 7f5d6e000700 1 -- 172.21.15.22:0/2233643617 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f5d60003af0 con 0x7f5d70109db0 2024-03-28T17:55:14.326 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:55:13.066+0000 7f5d6e000700 1 -- 172.21.15.22:0/2233643617 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f5d60003c90 con 0x7f5d70109db0 2024-03-28T17:55:14.326 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:55:13.066+0000 7f5d78200700 1 -- 172.21.15.22:0/2233643617 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5d7019e570 con 0x7f5d70109db0 2024-03-28T17:55:14.326 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:55:13.066+0000 7f5d6e000700 1 -- 172.21.15.22:0/2233643617 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f5d60024830 con 0x7f5d70109db0 2024-03-28T17:55:14.326 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:55:13.067+0000 7f5d78200700 1 -- 172.21.15.22:0/2233643617 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5d701a1260 con 0x7f5d70109db0 2024-03-28T17:55:14.326 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:55:13.068+0000 7f5d78200700 1 -- 172.21.15.22:0/2233643617 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5d700637c0 con 0x7f5d70109db0 2024-03-28T17:55:14.326 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:55:13.068+0000 7f5d6e000700 1 -- 172.21.15.22:0/2233643617 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 12) v1 ==== 46119+0+0 (secure 0 0 0) 0x7f5d60022020 con 0x7f5d70109db0 2024-03-28T17:55:14.326 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:55:13.068+0000 7f5d6e000700 1 --2- 172.21.15.22:0/2233643617 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7f5d54038fd0 0x7f5d5403b490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:55:14.326 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:55:13.068+0000 7f5d6e000700 1 -- 172.21.15.22:0/2233643617 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f5d6004cf80 con 0x7f5d70109db0 2024-03-28T17:55:14.327 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:55:13.069+0000 7f5d74e00700 1 --2- 172.21.15.22:0/2233643617 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7f5d54038fd0 0x7f5d5403b490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:55:14.327 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:55:13.069+0000 7f5d74e00700 1 --2- 172.21.15.22:0/2233643617 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7f5d54038fd0 0x7f5d5403b490 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f5d5c006fd0 tx=0x7f5d5c006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T17:55:14.327 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:55:13.074+0000 7f5d6e000700 1 -- 172.21.15.22:0/2233643617 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f5d6002d3f0 con 0x7f5d70109db0 2024-03-28T17:55:14.327 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:55:13.311+0000 7f5d78200700 1 -- 172.21.15.22:0/2233643617 --> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] -- mgr_command(tid 0: {"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}) v1 -- 0x7f5d70107c70 con 0x7f5d54038fd0 2024-03-28T17:55:14.327 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:55:13.460+0000 7f5d6e000700 1 -- 172.21.15.22:0/2233643617 <== mgr.14152 v2:172.21.15.22:6800/4201048392 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+32 (secure 0 0 0) 0x7f5d70107c70 con 0x7f5d54038fd0 2024-03-28T17:55:14.327 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:55:13.463+0000 7f5d78200700 1 -- 172.21.15.22:0/2233643617 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7f5d54038fd0 msgr2=0x7f5d5403b490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:55:14.327 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:55:13.463+0000 7f5d78200700 1 --2- 172.21.15.22:0/2233643617 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7f5d54038fd0 0x7f5d5403b490 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f5d5c006fd0 tx=0x7f5d5c006e40 comp rx=0 tx=0).stop 2024-03-28T17:55:14.327 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:55:13.463+0000 7f5d78200700 1 -- 172.21.15.22:0/2233643617 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5d70109db0 msgr2=0x7f5d7019dda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:55:14.327 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:55:13.463+0000 7f5d78200700 1 --2- 172.21.15.22:0/2233643617 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5d70109db0 0x7f5d7019dda0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f5d60006af0 tx=0x7f5d60011bd0 comp rx=0 tx=0).stop 2024-03-28T17:55:14.327 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:55:13.463+0000 7f5d78200700 1 -- 172.21.15.22:0/2233643617 shutdown_connections 2024-03-28T17:55:14.327 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:55:13.463+0000 7f5d78200700 1 --2- 172.21.15.22:0/2233643617 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7f5d54038fd0 0x7f5d5403b490 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:55:14.327 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:55:13.463+0000 7f5d78200700 1 --2- 172.21.15.22:0/2233643617 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5d70109db0 0x7f5d7019dda0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:55:14.327 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:55:13.463+0000 7f5d78200700 1 -- 172.21.15.22:0/2233643617 >> 172.21.15.22:0/2233643617 conn(0x7f5d7007d150 msgr2=0x7f5d70107560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T17:55:14.327 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:55:13.464+0000 7f5d78200700 1 -- 172.21.15.22:0/2233643617 shutdown_connections 2024-03-28T17:55:14.327 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:55:13.464+0000 7f5d78200700 1 -- 172.21.15.22:0/2233643617 wait complete. 2024-03-28T17:55:14.327 INFO:teuthology.orchestra.run.smithi022.stdout:Creating initial admin user... 2024-03-28T17:55:15.509 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:55:15 smithi022 ceph-mon[31985]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T17:55:16.377 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:55:16 smithi022 ceph-mon[31985]: from='client.14166 -' entity='client.admin' cmd=[{"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}]: dispatch 2024-03-28T17:55:16.378 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:55:16 smithi022 ceph-mon[31985]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T17:55:16.383 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout {"username": "admin", "password": "$2b$12$PwWG7TVHbQA97hcEWCi7QuSLHXzADI9i0CbjWFek0a4qJVS1Jt9ru", "roles": ["administrator"], "name": null, "email": null, "lastUpdate": 1711648515, "enabled": true, "pwdExpirationDate": null, "pwdUpdateRequired": true} 2024-03-28T17:55:16.384 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:55:14.980+0000 7f8f3be00700 1 Processor -- start 2024-03-28T17:55:16.384 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:55:14.980+0000 7f8f3be00700 1 -- start start 2024-03-28T17:55:16.384 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:55:14.981+0000 7f8f3be00700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8f3c106840 0x7f8f3c106c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:55:16.385 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:55:14.981+0000 7f8f3be00700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f3c107230 con 0x7f8f3c106840 2024-03-28T17:55:16.385 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:55:14.981+0000 7f8f3aa00700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8f3c106840 0x7f8f3c106c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:55:16.385 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:55:14.982+0000 7f8f3aa00700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8f3c106840 0x7f8f3c106c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:56446/0 (socket says 172.21.15.22:56446) 2024-03-28T17:55:16.389 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:55:14.982+0000 7f8f3aa00700 1 -- 172.21.15.22:0/2818866971 learned_addr learned my addr 172.21.15.22:0/2818866971 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-28T17:55:16.389 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:55:14.982+0000 7f8f3aa00700 1 -- 172.21.15.22:0/2818866971 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8f3c107a40 con 0x7f8f3c106840 2024-03-28T17:55:16.389 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:55:14.983+0000 7f8f3aa00700 1 --2- 172.21.15.22:0/2818866971 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8f3c106840 0x7f8f3c106c60 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f8f2c00bb40 tx=0x7f8f2c00be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9eac88f427f7227b server_cookie=0 in_seq=0 out_seq=0 2024-03-28T17:55:16.390 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:55:14.983+0000 7f8f39600700 1 -- 172.21.15.22:0/2818866971 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f8f2c00dbe0 con 0x7f8f3c106840 2024-03-28T17:55:16.390 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:55:14.983+0000 7f8f39600700 1 -- 172.21.15.22:0/2818866971 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f8f2c004500 con 0x7f8f3c106840 2024-03-28T17:55:16.390 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:55:14.983+0000 7f8f39600700 1 -- 172.21.15.22:0/2818866971 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f8f2c010470 con 0x7f8f3c106840 2024-03-28T17:55:16.390 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:55:14.984+0000 7f8f3be00700 1 -- 172.21.15.22:0/2818866971 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8f3c106840 msgr2=0x7f8f3c106c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:55:16.390 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:55:14.984+0000 7f8f3be00700 1 --2- 172.21.15.22:0/2818866971 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8f3c106840 0x7f8f3c106c60 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f8f2c00bb40 tx=0x7f8f2c00be50 comp rx=0 tx=0).stop 2024-03-28T17:55:16.390 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:55:14.984+0000 7f8f3be00700 1 -- 172.21.15.22:0/2818866971 shutdown_connections 2024-03-28T17:55:16.390 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:55:14.984+0000 7f8f3be00700 1 --2- 172.21.15.22:0/2818866971 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8f3c106840 0x7f8f3c106c60 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:55:16.390 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:55:14.984+0000 7f8f3be00700 1 -- 172.21.15.22:0/2818866971 >> 172.21.15.22:0/2818866971 conn(0x7f8f3c101f00 msgr2=0x7f8f3c104320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T17:55:16.390 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:55:14.984+0000 7f8f3be00700 1 -- 172.21.15.22:0/2818866971 shutdown_connections 2024-03-28T17:55:16.390 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:55:14.984+0000 7f8f3be00700 1 -- 172.21.15.22:0/2818866971 wait complete. 2024-03-28T17:55:16.390 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:55:14.985+0000 7f8f3be00700 1 Processor -- start 2024-03-28T17:55:16.390 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:55:14.985+0000 7f8f3be00700 1 -- start start 2024-03-28T17:55:16.390 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:55:14.986+0000 7f8f3be00700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8f3c106840 0x7f8f3c19dbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:55:16.391 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:55:14.986+0000 7f8f3be00700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f3c074a90 con 0x7f8f3c106840 2024-03-28T17:55:16.391 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:55:14.986+0000 7f8f3aa00700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8f3c106840 0x7f8f3c19dbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:55:16.391 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:55:14.986+0000 7f8f3aa00700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8f3c106840 0x7f8f3c19dbd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:56460/0 (socket says 172.21.15.22:56460) 2024-03-28T17:55:16.391 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:55:14.986+0000 7f8f3aa00700 1 -- 172.21.15.22:0/2724980389 learned_addr learned my addr 172.21.15.22:0/2724980389 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-28T17:55:16.391 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:55:14.986+0000 7f8f3aa00700 1 -- 172.21.15.22:0/2724980389 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8f3c19e110 con 0x7f8f3c106840 2024-03-28T17:55:16.391 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:55:14.987+0000 7f8f3aa00700 1 --2- 172.21.15.22:0/2724980389 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8f3c106840 0x7f8f3c19dbd0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f8f2c00b340 tx=0x7f8f2c004070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T17:55:16.391 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:55:14.987+0000 7f8f33400700 1 -- 172.21.15.22:0/2724980389 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f8f2c010720 con 0x7f8f3c106840 2024-03-28T17:55:16.391 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:55:14.987+0000 7f8f33400700 1 -- 172.21.15.22:0/2724980389 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f8f2c01d070 con 0x7f8f3c106840 2024-03-28T17:55:16.391 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:55:14.987+0000 7f8f3be00700 1 -- 172.21.15.22:0/2724980389 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8f3c19e3a0 con 0x7f8f3c106840 2024-03-28T17:55:16.391 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:55:14.987+0000 7f8f33400700 1 -- 172.21.15.22:0/2724980389 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f8f2c021ae0 con 0x7f8f3c106840 2024-03-28T17:55:16.391 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:55:14.987+0000 7f8f3be00700 1 -- 172.21.15.22:0/2724980389 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8f3c1a1090 con 0x7f8f3c106840 2024-03-28T17:55:16.391 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:55:14.988+0000 7f8f33400700 1 -- 172.21.15.22:0/2724980389 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 12) v1 ==== 46119+0+0 (secure 0 0 0) 0x7f8f2c01f030 con 0x7f8f3c106840 2024-03-28T17:55:16.391 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:55:14.989+0000 7f8f33400700 1 --2- 172.21.15.22:0/2724980389 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7f8f20038fd0 0x7f8f2003b490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:55:16.391 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:55:14.989+0000 7f8f33400700 1 -- 172.21.15.22:0/2724980389 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f8f2c04d720 con 0x7f8f3c106840 2024-03-28T17:55:16.391 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:55:14.989+0000 7f8f3be00700 1 -- 172.21.15.22:0/2724980389 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8f3c0637c0 con 0x7f8f3c106840 2024-03-28T17:55:16.391 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:55:14.989+0000 7f8f3a000700 1 --2- 172.21.15.22:0/2724980389 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7f8f20038fd0 0x7f8f2003b490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:55:16.391 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:55:14.990+0000 7f8f3a000700 1 --2- 172.21.15.22:0/2724980389 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7f8f20038fd0 0x7f8f2003b490 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f8f28006fd0 tx=0x7f8f28006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T17:55:16.392 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:55:14.995+0000 7f8f33400700 1 -- 172.21.15.22:0/2724980389 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f8f2c015070 con 0x7f8f3c106840 2024-03-28T17:55:16.392 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:55:15.239+0000 7f8f3be00700 1 -- 172.21.15.22:0/2724980389 --> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] -- mgr_command(tid 0: {"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}) v1 -- 0x7f8f3c1a1320 con 0x7f8f20038fd0 2024-03-28T17:55:16.392 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:55:15.508+0000 7f8f33400700 1 -- 172.21.15.22:0/2724980389 <== mgr.14152 v2:172.21.15.22:6800/4201048392 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+252 (secure 0 0 0) 0x7f8f3c1a1320 con 0x7f8f20038fd0 2024-03-28T17:55:16.392 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:55:15.511+0000 7f8f3be00700 1 -- 172.21.15.22:0/2724980389 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7f8f20038fd0 msgr2=0x7f8f2003b490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:55:16.392 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:55:15.511+0000 7f8f3be00700 1 --2- 172.21.15.22:0/2724980389 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7f8f20038fd0 0x7f8f2003b490 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f8f28006fd0 tx=0x7f8f28006e40 comp rx=0 tx=0).stop 2024-03-28T17:55:16.392 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:55:15.511+0000 7f8f3be00700 1 -- 172.21.15.22:0/2724980389 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8f3c106840 msgr2=0x7f8f3c19dbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:55:16.392 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:55:15.511+0000 7f8f3be00700 1 --2- 172.21.15.22:0/2724980389 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8f3c106840 0x7f8f3c19dbd0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f8f2c00b340 tx=0x7f8f2c004070 comp rx=0 tx=0).stop 2024-03-28T17:55:16.392 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:55:15.512+0000 7f8f3be00700 1 -- 172.21.15.22:0/2724980389 shutdown_connections 2024-03-28T17:55:16.392 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:55:15.512+0000 7f8f3be00700 1 --2- 172.21.15.22:0/2724980389 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7f8f20038fd0 0x7f8f2003b490 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:55:16.392 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:55:15.512+0000 7f8f3be00700 1 --2- 172.21.15.22:0/2724980389 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8f3c106840 0x7f8f3c19dbd0 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:55:16.392 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:55:15.512+0000 7f8f3be00700 1 -- 172.21.15.22:0/2724980389 >> 172.21.15.22:0/2724980389 conn(0x7f8f3c101f00 msgr2=0x7f8f3c194210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T17:55:16.392 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:55:15.512+0000 7f8f3be00700 1 -- 172.21.15.22:0/2724980389 shutdown_connections 2024-03-28T17:55:16.392 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:55:15.512+0000 7f8f3be00700 1 -- 172.21.15.22:0/2724980389 wait complete. 2024-03-28T17:55:16.392 INFO:teuthology.orchestra.run.smithi022.stdout:Fetching dashboard port number... 2024-03-28T17:55:17.673 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:55:17 smithi022 ceph-mon[31985]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T17:55:17.673 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:55:17 smithi022 ceph-mon[31985]: mgrmap e13: x(active, since 6s) 2024-03-28T17:55:17.673 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:55:17 smithi022 ceph-mon[31985]: from='client.? 172.21.15.22:0/2843072876' entity='client.admin' cmd=[{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]: dispatch 2024-03-28T17:55:17.988 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout 8443 2024-03-28T17:55:17.988 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:55:17.018+0000 7f9933600700 1 Processor -- start 2024-03-28T17:55:17.988 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:55:17.018+0000 7f9933600700 1 -- start start 2024-03-28T17:55:17.988 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:55:17.019+0000 7f9933600700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f992c1086f0 0x7f992c10aae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:55:17.988 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:55:17.019+0000 7f9933600700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f992c07bdd0 con 0x7f992c1086f0 2024-03-28T17:55:17.988 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:55:17.019+0000 7f9930c00700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f992c1086f0 0x7f992c10aae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:55:17.988 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:55:17.019+0000 7f9930c00700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f992c1086f0 0x7f992c10aae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:57996/0 (socket says 172.21.15.22:57996) 2024-03-28T17:55:17.988 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:55:17.019+0000 7f9930c00700 1 -- 172.21.15.22:0/1191305231 learned_addr learned my addr 172.21.15.22:0/1191305231 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-28T17:55:17.988 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:55:17.020+0000 7f9930c00700 1 -- 172.21.15.22:0/1191305231 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f992c10b020 con 0x7f992c1086f0 2024-03-28T17:55:17.988 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:55:17.020+0000 7f9930c00700 1 --2- 172.21.15.22:0/1191305231 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f992c1086f0 0x7f992c10aae0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f99200098a0 tx=0x7f9920009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8ffd6e83ced1dd66 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T17:55:17.989 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:55:17.020+0000 7f992b400700 1 -- 172.21.15.22:0/1191305231 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f9920004160 con 0x7f992c1086f0 2024-03-28T17:55:17.993 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:55:17.020+0000 7f992b400700 1 -- 172.21.15.22:0/1191305231 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f9920004300 con 0x7f992c1086f0 2024-03-28T17:55:17.993 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:55:17.021+0000 7f992b400700 1 -- 172.21.15.22:0/1191305231 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f9920011b10 con 0x7f992c1086f0 2024-03-28T17:55:17.994 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:55:17.021+0000 7f9933600700 1 -- 172.21.15.22:0/1191305231 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f992c1086f0 msgr2=0x7f992c10aae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:55:17.994 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:55:17.021+0000 7f9933600700 1 --2- 172.21.15.22:0/1191305231 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f992c1086f0 0x7f992c10aae0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f99200098a0 tx=0x7f9920009bb0 comp rx=0 tx=0).stop 2024-03-28T17:55:17.994 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:55:17.021+0000 7f9933600700 1 -- 172.21.15.22:0/1191305231 shutdown_connections 2024-03-28T17:55:17.994 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:55:17.021+0000 7f9933600700 1 --2- 172.21.15.22:0/1191305231 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f992c1086f0 0x7f992c10aae0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:55:17.994 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:55:17.021+0000 7f9933600700 1 -- 172.21.15.22:0/1191305231 >> 172.21.15.22:0/1191305231 conn(0x7f992c102020 msgr2=0x7f992c104480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T17:55:17.994 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:55:17.022+0000 7f9933600700 1 -- 172.21.15.22:0/1191305231 shutdown_connections 2024-03-28T17:55:17.994 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:55:17.022+0000 7f9933600700 1 -- 172.21.15.22:0/1191305231 wait complete. 2024-03-28T17:55:17.994 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:55:17.023+0000 7f9933600700 1 Processor -- start 2024-03-28T17:55:17.994 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:55:17.023+0000 7f9933600700 1 -- start start 2024-03-28T17:55:17.994 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:55:17.023+0000 7f9933600700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f992c1086f0 0x7f992c19dda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:55:17.994 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:55:17.023+0000 7f9933600700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f992c10bfc0 con 0x7f992c1086f0 2024-03-28T17:55:17.994 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:55:17.024+0000 7f9930c00700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f992c1086f0 0x7f992c19dda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:55:17.994 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:55:17.024+0000 7f9930c00700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f992c1086f0 0x7f992c19dda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:58006/0 (socket says 172.21.15.22:58006) 2024-03-28T17:55:17.994 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:55:17.024+0000 7f9930c00700 1 -- 172.21.15.22:0/2843072876 learned_addr learned my addr 172.21.15.22:0/2843072876 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-28T17:55:17.995 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:55:17.024+0000 7f9930c00700 1 -- 172.21.15.22:0/2843072876 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f992c19e2e0 con 0x7f992c1086f0 2024-03-28T17:55:17.995 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:55:17.024+0000 7f9930c00700 1 --2- 172.21.15.22:0/2843072876 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f992c1086f0 0x7f992c19dda0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f9920006af0 tx=0x7f9920011bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T17:55:17.995 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:55:17.025+0000 7f9929600700 1 -- 172.21.15.22:0/2843072876 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f9920003af0 con 0x7f992c1086f0 2024-03-28T17:55:17.995 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:55:17.025+0000 7f9929600700 1 -- 172.21.15.22:0/2843072876 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f9920003c90 con 0x7f992c1086f0 2024-03-28T17:55:17.995 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:55:17.025+0000 7f9929600700 1 -- 172.21.15.22:0/2843072876 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f9920024830 con 0x7f992c1086f0 2024-03-28T17:55:17.995 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:55:17.025+0000 7f9933600700 1 -- 172.21.15.22:0/2843072876 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f992c19e570 con 0x7f992c1086f0 2024-03-28T17:55:17.995 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:55:17.025+0000 7f9933600700 1 -- 172.21.15.22:0/2843072876 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f992c1a1220 con 0x7f992c1086f0 2024-03-28T17:55:17.995 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:55:17.026+0000 7f9933600700 1 -- 172.21.15.22:0/2843072876 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f992c0637c0 con 0x7f992c1086f0 2024-03-28T17:55:17.995 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:55:17.027+0000 7f9929600700 1 -- 172.21.15.22:0/2843072876 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 12) v1 ==== 46119+0+0 (secure 0 0 0) 0x7f9920022020 con 0x7f992c1086f0 2024-03-28T17:55:17.995 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:55:17.027+0000 7f9929600700 1 --2- 172.21.15.22:0/2843072876 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7f9914038fd0 0x7f991403b490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:55:17.995 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:55:17.027+0000 7f9929600700 1 -- 172.21.15.22:0/2843072876 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f992004cf30 con 0x7f992c1086f0 2024-03-28T17:55:17.995 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:55:17.027+0000 7f992be00700 1 --2- 172.21.15.22:0/2843072876 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7f9914038fd0 0x7f991403b490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:55:17.995 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:55:17.028+0000 7f992be00700 1 --2- 172.21.15.22:0/2843072876 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7f9914038fd0 0x7f991403b490 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f9918006fd0 tx=0x7f9918006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T17:55:17.995 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:55:17.033+0000 7f9929600700 1 -- 172.21.15.22:0/2843072876 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f992002d3f0 con 0x7f992c1086f0 2024-03-28T17:55:17.995 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:55:17.133+0000 7f9929600700 1 -- 172.21.15.22:0/2843072876 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mgrmap(e 13) v1 ==== 46162+0+0 (secure 0 0 0) 0x7f992000bc90 con 0x7f992c1086f0 2024-03-28T17:55:17.995 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:55:17.281+0000 7f9933600700 1 -- 172.21.15.22:0/2843072876 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"} v 0) v1 -- 0x7f992c1a13f0 con 0x7f992c1086f0 2024-03-28T17:55:17.996 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:55:17.282+0000 7f9929600700 1 -- 172.21.15.22:0/2843072876 <== mon.0 v2:172.21.15.22:3300/0 8 ==== mon_command_ack([{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]=0 v8) v1 ==== 112+0+5 (secure 0 0 0) 0x7f992c1a13f0 con 0x7f992c1086f0 2024-03-28T17:55:17.996 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:55:17.285+0000 7f9933600700 1 -- 172.21.15.22:0/2843072876 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7f9914038fd0 msgr2=0x7f991403b490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:55:17.996 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:55:17.285+0000 7f9933600700 1 --2- 172.21.15.22:0/2843072876 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7f9914038fd0 0x7f991403b490 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f9918006fd0 tx=0x7f9918006e40 comp rx=0 tx=0).stop 2024-03-28T17:55:17.996 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:55:17.285+0000 7f9933600700 1 -- 172.21.15.22:0/2843072876 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f992c1086f0 msgr2=0x7f992c19dda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:55:17.996 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:55:17.285+0000 7f9933600700 1 --2- 172.21.15.22:0/2843072876 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f992c1086f0 0x7f992c19dda0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f9920006af0 tx=0x7f9920011bd0 comp rx=0 tx=0).stop 2024-03-28T17:55:17.996 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:55:17.285+0000 7f9933600700 1 -- 172.21.15.22:0/2843072876 shutdown_connections 2024-03-28T17:55:17.996 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:55:17.285+0000 7f9933600700 1 --2- 172.21.15.22:0/2843072876 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7f9914038fd0 0x7f991403b490 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:55:17.996 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:55:17.285+0000 7f9933600700 1 --2- 172.21.15.22:0/2843072876 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f992c1086f0 0x7f992c19dda0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:55:17.996 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:55:17.285+0000 7f9933600700 1 -- 172.21.15.22:0/2843072876 >> 172.21.15.22:0/2843072876 conn(0x7f992c102020 msgr2=0x7f992c102b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T17:55:17.996 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:55:17.285+0000 7f9933600700 1 -- 172.21.15.22:0/2843072876 shutdown_connections 2024-03-28T17:55:17.996 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:55:17.285+0000 7f9933600700 1 -- 172.21.15.22:0/2843072876 wait complete. 2024-03-28T17:55:18.024 INFO:teuthology.orchestra.run.smithi022.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-03-28T17:55:18.024 INFO:teuthology.orchestra.run.smithi022.stdout:systemctl: stdout disabled 2024-03-28T17:55:18.041 INFO:teuthology.orchestra.run.smithi022.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-03-28T17:55:18.041 INFO:teuthology.orchestra.run.smithi022.stdout:systemctl: stdout inactive 2024-03-28T17:55:18.041 INFO:teuthology.orchestra.run.smithi022.stdout:firewalld.service is not enabled 2024-03-28T17:55:18.042 INFO:teuthology.orchestra.run.smithi022.stdout:Not possible to open ports <[8443]>. firewalld.service is not available 2024-03-28T17:55:18.047 INFO:teuthology.orchestra.run.smithi022.stdout:Ceph Dashboard is now available at: 2024-03-28T17:55:18.047 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-03-28T17:55:18.047 INFO:teuthology.orchestra.run.smithi022.stdout: URL: https://smithi022.front.sepia.ceph.com:8443/ 2024-03-28T17:55:18.047 INFO:teuthology.orchestra.run.smithi022.stdout: User: admin 2024-03-28T17:55:18.047 INFO:teuthology.orchestra.run.smithi022.stdout: Password: aemiy2sm9x 2024-03-28T17:55:18.047 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-03-28T17:55:18.047 INFO:teuthology.orchestra.run.smithi022.stdout:Saving cluster configuration to /var/lib/ceph/fd220748-ed2b-11ee-95d0-87774f69a715/config directory 2024-03-28T17:55:18.049 INFO:teuthology.orchestra.run.smithi022.stdout:Enabling autotune for osd_memory_target 2024-03-28T17:55:20.466 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:55:19.341+0000 7ffb43400700 1 Processor -- start 2024-03-28T17:55:20.466 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:55:19.341+0000 7ffb43400700 1 -- start start 2024-03-28T17:55:20.466 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:55:19.342+0000 7ffb43400700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ffb3c107b80 0x7ffb3c107fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:55:20.466 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:55:19.342+0000 7ffb43400700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb3c108570 con 0x7ffb3c107b80 2024-03-28T17:55:20.466 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:55:19.342+0000 7ffb40a00700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ffb3c107b80 0x7ffb3c107fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:55:20.466 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:55:19.342+0000 7ffb40a00700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ffb3c107b80 0x7ffb3c107fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:58018/0 (socket says 172.21.15.22:58018) 2024-03-28T17:55:20.467 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:55:19.342+0000 7ffb40a00700 1 -- 172.21.15.22:0/1882415901 learned_addr learned my addr 172.21.15.22:0/1882415901 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-28T17:55:20.467 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:55:19.343+0000 7ffb40a00700 1 -- 172.21.15.22:0/1882415901 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb3c108d80 con 0x7ffb3c107b80 2024-03-28T17:55:20.468 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:55:19.343+0000 7ffb40a00700 1 --2- 172.21.15.22:0/1882415901 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ffb3c107b80 0x7ffb3c107fa0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7ffb300098a0 tx=0x7ffb30009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d4d193c44335af17 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T17:55:20.468 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:55:19.343+0000 7ffb3b400700 1 -- 172.21.15.22:0/1882415901 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ffb30004160 con 0x7ffb3c107b80 2024-03-28T17:55:20.468 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:55:19.343+0000 7ffb3b400700 1 -- 172.21.15.22:0/1882415901 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7ffb30004300 con 0x7ffb3c107b80 2024-03-28T17:55:20.468 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:55:19.343+0000 7ffb3b400700 1 -- 172.21.15.22:0/1882415901 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ffb30011b10 con 0x7ffb3c107b80 2024-03-28T17:55:20.468 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:55:19.344+0000 7ffb43400700 1 -- 172.21.15.22:0/1882415901 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ffb3c107b80 msgr2=0x7ffb3c107fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:55:20.468 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:55:19.344+0000 7ffb43400700 1 --2- 172.21.15.22:0/1882415901 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ffb3c107b80 0x7ffb3c107fa0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7ffb300098a0 tx=0x7ffb30009bb0 comp rx=0 tx=0).stop 2024-03-28T17:55:20.468 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:55:19.344+0000 7ffb43400700 1 -- 172.21.15.22:0/1882415901 shutdown_connections 2024-03-28T17:55:20.468 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:55:19.344+0000 7ffb43400700 1 --2- 172.21.15.22:0/1882415901 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ffb3c107b80 0x7ffb3c107fa0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:55:20.468 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:55:19.344+0000 7ffb43400700 1 -- 172.21.15.22:0/1882415901 >> 172.21.15.22:0/1882415901 conn(0x7ffb3c103100 msgr2=0x7ffb3c105560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T17:55:20.471 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:55:19.344+0000 7ffb43400700 1 -- 172.21.15.22:0/1882415901 shutdown_connections 2024-03-28T17:55:20.471 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:55:19.345+0000 7ffb43400700 1 -- 172.21.15.22:0/1882415901 wait complete. 2024-03-28T17:55:20.471 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:55:19.346+0000 7ffb43400700 1 Processor -- start 2024-03-28T17:55:20.471 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:55:19.346+0000 7ffb43400700 1 -- start start 2024-03-28T17:55:20.471 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:55:19.346+0000 7ffb43400700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ffb3c107b80 0x7ffb3c19dda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:55:20.471 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:55:19.346+0000 7ffb43400700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb3c10c130 con 0x7ffb3c107b80 2024-03-28T17:55:20.471 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:55:19.346+0000 7ffb40a00700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ffb3c107b80 0x7ffb3c19dda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:55:20.471 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:55:19.346+0000 7ffb40a00700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ffb3c107b80 0x7ffb3c19dda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:58022/0 (socket says 172.21.15.22:58022) 2024-03-28T17:55:20.471 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:55:19.347+0000 7ffb40a00700 1 -- 172.21.15.22:0/1866051904 learned_addr learned my addr 172.21.15.22:0/1866051904 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-28T17:55:20.472 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:55:19.347+0000 7ffb40a00700 1 -- 172.21.15.22:0/1866051904 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb3c19e2e0 con 0x7ffb3c107b80 2024-03-28T17:55:20.472 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:55:19.347+0000 7ffb40a00700 1 --2- 172.21.15.22:0/1866051904 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ffb3c107b80 0x7ffb3c19dda0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7ffb30006af0 tx=0x7ffb30011bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T17:55:20.472 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:55:19.347+0000 7ffb39600700 1 -- 172.21.15.22:0/1866051904 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ffb30003af0 con 0x7ffb3c107b80 2024-03-28T17:55:20.472 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:55:19.347+0000 7ffb39600700 1 -- 172.21.15.22:0/1866051904 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7ffb30003c90 con 0x7ffb3c107b80 2024-03-28T17:55:20.472 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:55:19.347+0000 7ffb39600700 1 -- 172.21.15.22:0/1866051904 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ffb30024830 con 0x7ffb3c107b80 2024-03-28T17:55:20.472 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:55:19.348+0000 7ffb43400700 1 -- 172.21.15.22:0/1866051904 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffb3c19e570 con 0x7ffb3c107b80 2024-03-28T17:55:20.472 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:55:19.348+0000 7ffb43400700 1 -- 172.21.15.22:0/1866051904 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffb3c1a1260 con 0x7ffb3c107b80 2024-03-28T17:55:20.472 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:55:19.349+0000 7ffb39600700 1 -- 172.21.15.22:0/1866051904 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 13) v1 ==== 46162+0+0 (secure 0 0 0) 0x7ffb30022020 con 0x7ffb3c107b80 2024-03-28T17:55:20.472 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:55:19.349+0000 7ffb43400700 1 -- 172.21.15.22:0/1866051904 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffb3c0637c0 con 0x7ffb3c107b80 2024-03-28T17:55:20.472 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:55:19.349+0000 7ffb39600700 1 --2- 172.21.15.22:0/1866051904 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7ffb24039010 0x7ffb2403b4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:55:20.472 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:55:19.349+0000 7ffb39600700 1 -- 172.21.15.22:0/1866051904 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7ffb3004d0a0 con 0x7ffb3c107b80 2024-03-28T17:55:20.472 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:55:19.350+0000 7ffb3be00700 1 --2- 172.21.15.22:0/1866051904 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7ffb24039010 0x7ffb2403b4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:55:20.472 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:55:19.350+0000 7ffb3be00700 1 --2- 172.21.15.22:0/1866051904 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7ffb24039010 0x7ffb2403b4d0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7ffb28006fd0 tx=0x7ffb28006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T17:55:20.472 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:55:19.355+0000 7ffb39600700 1 -- 172.21.15.22:0/1866051904 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7ffb3002d3f0 con 0x7ffb3c107b80 2024-03-28T17:55:20.473 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:55:19.585+0000 7ffb43400700 1 -- 172.21.15.22:0/1866051904 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command([{prefix=config set, name=osd_memory_target_autotune}] v 0) v1 -- 0x7ffb3c1a18c0 con 0x7ffb3c107b80 2024-03-28T17:55:20.474 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:55:19.586+0000 7ffb39600700 1 -- 172.21.15.22:0/1866051904 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{prefix=config set, name=osd_memory_target_autotune}]=0 v8)=0 v8) v1 ==== 127+0+0 (secure 0 0 0) 0x7ffb3000bc90 con 0x7ffb3c107b80 2024-03-28T17:55:20.474 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:55:19.589+0000 7ffb43400700 1 -- 172.21.15.22:0/1866051904 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7ffb24039010 msgr2=0x7ffb2403b4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:55:20.474 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:55:19.589+0000 7ffb43400700 1 --2- 172.21.15.22:0/1866051904 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7ffb24039010 0x7ffb2403b4d0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7ffb28006fd0 tx=0x7ffb28006e40 comp rx=0 tx=0).stop 2024-03-28T17:55:20.474 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:55:19.589+0000 7ffb43400700 1 -- 172.21.15.22:0/1866051904 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ffb3c107b80 msgr2=0x7ffb3c19dda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:55:20.474 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:55:19.589+0000 7ffb43400700 1 --2- 172.21.15.22:0/1866051904 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ffb3c107b80 0x7ffb3c19dda0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7ffb30006af0 tx=0x7ffb30011bd0 comp rx=0 tx=0).stop 2024-03-28T17:55:20.474 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:55:19.589+0000 7ffb43400700 1 -- 172.21.15.22:0/1866051904 shutdown_connections 2024-03-28T17:55:20.474 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:55:19.589+0000 7ffb43400700 1 --2- 172.21.15.22:0/1866051904 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7ffb24039010 0x7ffb2403b4d0 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:55:20.474 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:55:19.589+0000 7ffb43400700 1 --2- 172.21.15.22:0/1866051904 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ffb3c107b80 0x7ffb3c19dda0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:55:20.474 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:55:19.589+0000 7ffb43400700 1 -- 172.21.15.22:0/1866051904 >> 172.21.15.22:0/1866051904 conn(0x7ffb3c103100 msgr2=0x7ffb3c103c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T17:55:20.475 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:55:19.589+0000 7ffb43400700 1 -- 172.21.15.22:0/1866051904 shutdown_connections 2024-03-28T17:55:20.475 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:55:19.590+0000 7ffb43400700 1 -- 172.21.15.22:0/1866051904 wait complete. 2024-03-28T17:55:22.231 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:55:21.124+0000 7f61a1c00700 1 Processor -- start 2024-03-28T17:55:22.232 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:55:21.124+0000 7f61a1c00700 1 -- start start 2024-03-28T17:55:22.232 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:55:21.124+0000 7f61a1c00700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f619c106a80 0x7f619c106ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:55:22.232 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:55:21.124+0000 7f61a1c00700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f619c107470 con 0x7f619c106a80 2024-03-28T17:55:22.233 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:55:21.125+0000 7f619aa00700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f619c106a80 0x7f619c106ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:55:22.233 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:55:21.125+0000 7f619aa00700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f619c106a80 0x7f619c106ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:58026/0 (socket says 172.21.15.22:58026) 2024-03-28T17:55:22.233 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:55:21.125+0000 7f619aa00700 1 -- 172.21.15.22:0/1813619223 learned_addr learned my addr 172.21.15.22:0/1813619223 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-28T17:55:22.233 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:55:21.125+0000 7f619aa00700 1 -- 172.21.15.22:0/1813619223 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f619c107c80 con 0x7f619c106a80 2024-03-28T17:55:22.234 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:55:21.126+0000 7f619aa00700 1 --2- 172.21.15.22:0/1813619223 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f619c106a80 0x7f619c106ea0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f618c0098a0 tx=0x7f618c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d72bc0d596566b6a server_cookie=0 in_seq=0 out_seq=0 2024-03-28T17:55:22.234 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:55:21.126+0000 7f6199600700 1 -- 172.21.15.22:0/1813619223 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f618c004160 con 0x7f619c106a80 2024-03-28T17:55:22.234 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:55:21.126+0000 7f6199600700 1 -- 172.21.15.22:0/1813619223 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f618c004300 con 0x7f619c106a80 2024-03-28T17:55:22.235 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:55:21.126+0000 7f6199600700 1 -- 172.21.15.22:0/1813619223 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f618c011b10 con 0x7f619c106a80 2024-03-28T17:55:22.235 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:55:21.127+0000 7f61a1c00700 1 -- 172.21.15.22:0/1813619223 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f619c106a80 msgr2=0x7f619c106ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:55:22.235 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:55:21.127+0000 7f61a1c00700 1 --2- 172.21.15.22:0/1813619223 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f619c106a80 0x7f619c106ea0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f618c0098a0 tx=0x7f618c009bb0 comp rx=0 tx=0).stop 2024-03-28T17:55:22.235 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:55:21.127+0000 7f61a1c00700 1 -- 172.21.15.22:0/1813619223 shutdown_connections 2024-03-28T17:55:22.236 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:55:21.127+0000 7f61a1c00700 1 --2- 172.21.15.22:0/1813619223 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f619c106a80 0x7f619c106ea0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:55:22.236 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:55:21.127+0000 7f61a1c00700 1 -- 172.21.15.22:0/1813619223 >> 172.21.15.22:0/1813619223 conn(0x7f619c102000 msgr2=0x7f619c104460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T17:55:22.236 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:55:21.127+0000 7f61a1c00700 1 -- 172.21.15.22:0/1813619223 shutdown_connections 2024-03-28T17:55:22.237 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:55:21.127+0000 7f61a1c00700 1 -- 172.21.15.22:0/1813619223 wait complete. 2024-03-28T17:55:22.237 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:55:21.128+0000 7f61a1c00700 1 Processor -- start 2024-03-28T17:55:22.237 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:55:21.128+0000 7f61a1c00700 1 -- start start 2024-03-28T17:55:22.237 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:55:21.129+0000 7f61a1c00700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f619c106a80 0x7f619c19dde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:55:22.238 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:55:21.129+0000 7f61a1c00700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f619c10b030 con 0x7f619c106a80 2024-03-28T17:55:22.238 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:55:21.129+0000 7f619aa00700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f619c106a80 0x7f619c19dde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:55:22.238 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:55:21.129+0000 7f619aa00700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f619c106a80 0x7f619c19dde0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:58042/0 (socket says 172.21.15.22:58042) 2024-03-28T17:55:22.239 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:55:21.129+0000 7f619aa00700 1 -- 172.21.15.22:0/2340285587 learned_addr learned my addr 172.21.15.22:0/2340285587 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-28T17:55:22.239 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:55:21.129+0000 7f619aa00700 1 -- 172.21.15.22:0/2340285587 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f619c19e320 con 0x7f619c106a80 2024-03-28T17:55:22.239 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:55:21.129+0000 7f619aa00700 1 --2- 172.21.15.22:0/2340285587 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f619c106a80 0x7f619c19dde0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f618c006af0 tx=0x7f618c011bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T17:55:22.243 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:55:21.130+0000 7f6193400700 1 -- 172.21.15.22:0/2340285587 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f618c0038b0 con 0x7f619c106a80 2024-03-28T17:55:22.243 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:55:21.130+0000 7f6193400700 1 -- 172.21.15.22:0/2340285587 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f618c01a430 con 0x7f619c106a80 2024-03-28T17:55:22.243 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:55:21.130+0000 7f6193400700 1 -- 172.21.15.22:0/2340285587 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f618c019530 con 0x7f619c106a80 2024-03-28T17:55:22.243 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:55:21.130+0000 7f61a1c00700 1 -- 172.21.15.22:0/2340285587 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f619c19e5b0 con 0x7f619c106a80 2024-03-28T17:55:22.243 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:55:21.130+0000 7f61a1c00700 1 -- 172.21.15.22:0/2340285587 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f619c1a12a0 con 0x7f619c106a80 2024-03-28T17:55:22.244 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:55:21.131+0000 7f6193400700 1 -- 172.21.15.22:0/2340285587 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 13) v1 ==== 46162+0+0 (secure 0 0 0) 0x7f618c01a9e0 con 0x7f619c106a80 2024-03-28T17:55:22.244 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:55:21.131+0000 7f61a1c00700 1 -- 172.21.15.22:0/2340285587 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f619c0637c0 con 0x7f619c106a80 2024-03-28T17:55:22.244 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:55:21.132+0000 7f6193400700 1 --2- 172.21.15.22:0/2340285587 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7f6180039020 0x7f618003b4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:55:22.244 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:55:21.132+0000 7f6193400700 1 -- 172.21.15.22:0/2340285587 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f618c04d3f0 con 0x7f619c106a80 2024-03-28T17:55:22.244 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:55:21.132+0000 7f619a000700 1 --2- 172.21.15.22:0/2340285587 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7f6180039020 0x7f618003b4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:55:22.244 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:55:21.133+0000 7f619a000700 1 --2- 172.21.15.22:0/2340285587 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7f6180039020 0x7f618003b4e0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f6188006fd0 tx=0x7f6188006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T17:55:22.244 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:55:21.137+0000 7f6193400700 1 -- 172.21.15.22:0/2340285587 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f618c01a690 con 0x7f619c106a80 2024-03-28T17:55:22.244 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:55:21.483+0000 7f61a1c00700 1 -- 172.21.15.22:0/2340285587 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command([{prefix=config-key set, key=mgr/dashboard/cluster/status}] v 0) v1 -- 0x7f619c02dc80 con 0x7f619c106a80 2024-03-28T17:55:22.244 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:55:21.487+0000 7f6193400700 1 -- 172.21.15.22:0/2340285587 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{prefix=config-key set, key=mgr/dashboard/cluster/status}]=0 set mgr/dashboard/cluster/status v28)=0 set mgr/dashboard/cluster/status v28) v1 ==== 153+0+0 (secure 0 0 0) 0x7f618c003a50 con 0x7f619c106a80 2024-03-28T17:55:22.244 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:55:21.490+0000 7f61a1c00700 1 -- 172.21.15.22:0/2340285587 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7f6180039020 msgr2=0x7f618003b4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:55:22.244 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:55:21.490+0000 7f61a1c00700 1 --2- 172.21.15.22:0/2340285587 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7f6180039020 0x7f618003b4e0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f6188006fd0 tx=0x7f6188006e40 comp rx=0 tx=0).stop 2024-03-28T17:55:22.244 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:55:21.490+0000 7f61a1c00700 1 -- 172.21.15.22:0/2340285587 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f619c106a80 msgr2=0x7f619c19dde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:55:22.244 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:55:21.490+0000 7f61a1c00700 1 --2- 172.21.15.22:0/2340285587 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f619c106a80 0x7f619c19dde0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f618c006af0 tx=0x7f618c011bd0 comp rx=0 tx=0).stop 2024-03-28T17:55:22.244 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:55:21.490+0000 7f61a1c00700 1 -- 172.21.15.22:0/2340285587 shutdown_connections 2024-03-28T17:55:22.245 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:55:21.490+0000 7f61a1c00700 1 --2- 172.21.15.22:0/2340285587 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7f6180039020 0x7f618003b4e0 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:55:22.245 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:55:21.490+0000 7f61a1c00700 1 --2- 172.21.15.22:0/2340285587 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f619c106a80 0x7f619c19dde0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:55:22.245 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:55:21.490+0000 7f61a1c00700 1 -- 172.21.15.22:0/2340285587 >> 172.21.15.22:0/2340285587 conn(0x7f619c102000 msgr2=0x7f619c102f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T17:55:22.245 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:55:21.491+0000 7f61a1c00700 1 -- 172.21.15.22:0/2340285587 shutdown_connections 2024-03-28T17:55:22.245 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-28T17:55:21.491+0000 7f61a1c00700 1 -- 172.21.15.22:0/2340285587 wait complete. 2024-03-28T17:55:22.245 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr set mgr/dashboard/cluster/status 2024-03-28T17:55:22.245 INFO:teuthology.orchestra.run.smithi022.stdout:You can access the Ceph CLI as following in case of multi-cluster or non-default config: 2024-03-28T17:55:22.245 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-03-28T17:55:22.245 INFO:teuthology.orchestra.run.smithi022.stdout: sudo /home/ubuntu/cephtest/cephadm shell --fsid fd220748-ed2b-11ee-95d0-87774f69a715 -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring 2024-03-28T17:55:22.245 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-03-28T17:55:22.245 INFO:teuthology.orchestra.run.smithi022.stdout:Or, if you are only running a single cluster on this host: 2024-03-28T17:55:22.245 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-03-28T17:55:22.245 INFO:teuthology.orchestra.run.smithi022.stdout: sudo /home/ubuntu/cephtest/cephadm shell 2024-03-28T17:55:22.245 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-03-28T17:55:22.245 INFO:teuthology.orchestra.run.smithi022.stdout:Please consider enabling telemetry to help improve Ceph: 2024-03-28T17:55:22.246 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-03-28T17:55:22.246 INFO:teuthology.orchestra.run.smithi022.stdout: ceph telemetry on 2024-03-28T17:55:22.246 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-03-28T17:55:22.246 INFO:teuthology.orchestra.run.smithi022.stdout:For more information see: 2024-03-28T17:55:22.246 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-03-28T17:55:22.246 INFO:teuthology.orchestra.run.smithi022.stdout: https://docs.ceph.com/en/latest/mgr/telemetry/ 2024-03-28T17:55:22.246 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-03-28T17:55:22.246 INFO:teuthology.orchestra.run.smithi022.stdout:Bootstrap complete. 2024-03-28T17:55:22.359 INFO:tasks.cephadm:Fetching config... 2024-03-28T17:55:22.359 DEBUG:teuthology.orchestra.run.smithi022:> set -ex 2024-03-28T17:55:22.359 DEBUG:teuthology.orchestra.run.smithi022:> dd if=/etc/ceph/ceph.conf of=/dev/stdout 2024-03-28T17:55:22.415 INFO:tasks.cephadm:Fetching client.admin keyring... 2024-03-28T17:55:22.415 DEBUG:teuthology.orchestra.run.smithi022:> set -ex 2024-03-28T17:55:22.415 DEBUG:teuthology.orchestra.run.smithi022:> dd if=/etc/ceph/ceph.client.admin.keyring of=/dev/stdout 2024-03-28T17:55:22.458 INFO:tasks.cephadm:Fetching mon keyring... 2024-03-28T17:55:22.458 DEBUG:teuthology.orchestra.run.smithi022:> set -ex 2024-03-28T17:55:22.458 DEBUG:teuthology.orchestra.run.smithi022:> sudo dd if=/var/lib/ceph/fd220748-ed2b-11ee-95d0-87774f69a715/mon.a/keyring of=/dev/stdout 2024-03-28T17:55:22.586 INFO:tasks.cephadm:Fetching pub ssh key... 2024-03-28T17:55:22.586 DEBUG:teuthology.orchestra.run.smithi022:> set -ex 2024-03-28T17:55:22.586 DEBUG:teuthology.orchestra.run.smithi022:> dd if=/home/ubuntu/cephtest/ceph.pub of=/dev/stdout 2024-03-28T17:55:22.681 INFO:tasks.cephadm:Installing pub ssh key for root users... 2024-03-28T17:55:22.681 DEBUG:teuthology.orchestra.run.smithi022:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDUi+jeSl5COOIlXQAk/lFBdgZWXIsZSspbsG9so++bg1fBAczn//xlfwOpRWKV+8NU3pC1kj15TfHrDeimhlKAjxRQFLaEDDFYlaKmiCVcjpdDzFhL4Dmv8fYmfwSKIdB+Y2UjHB3mf3h61Tk7oWagZwenEljWcSjAWDuPjRFYtefJOV/DfIVaUpcbHt+GzkbO96+XI17MnqFBT/Ms2kD4Ipb9IEg+tlEzde3zLCP1ZyIOpyHjFrsfGx5+HktyRHFxTrFonNfvWhRF4bpy9J47Qe7vGjUJD2lXOftS4741TJz7YBVRrFz2ukflvpwLR9volk6Rtava/3j/JI+2qqulshjsHhcMQ3pJ5jqCDfjj1DdK61Hzkb//S0gFWNi/X6zFgwSQ5sw6+m4A6OLFcw90bmac2HV0RwXzg8Svao51Xl9fu85aUVcp9h0CtnhtGrOc1GEfve5SwE/CW423P5vrz1q6XIqXxOVP2zPgc2AyQbz1RhLCWAzyKZwTRlLpOA0= ceph-fd220748-ed2b-11ee-95d0-87774f69a715' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-03-28T17:55:22.759 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:55:22 smithi022 ceph-mon[31985]: from='client.? 172.21.15.22:0/2340285587' entity='client.admin' 2024-03-28T17:55:22.807 INFO:teuthology.orchestra.run.smithi022.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDUi+jeSl5COOIlXQAk/lFBdgZWXIsZSspbsG9so++bg1fBAczn//xlfwOpRWKV+8NU3pC1kj15TfHrDeimhlKAjxRQFLaEDDFYlaKmiCVcjpdDzFhL4Dmv8fYmfwSKIdB+Y2UjHB3mf3h61Tk7oWagZwenEljWcSjAWDuPjRFYtefJOV/DfIVaUpcbHt+GzkbO96+XI17MnqFBT/Ms2kD4Ipb9IEg+tlEzde3zLCP1ZyIOpyHjFrsfGx5+HktyRHFxTrFonNfvWhRF4bpy9J47Qe7vGjUJD2lXOftS4741TJz7YBVRrFz2ukflvpwLR9volk6Rtava/3j/JI+2qqulshjsHhcMQ3pJ5jqCDfjj1DdK61Hzkb//S0gFWNi/X6zFgwSQ5sw6+m4A6OLFcw90bmac2HV0RwXzg8Svao51Xl9fu85aUVcp9h0CtnhtGrOc1GEfve5SwE/CW423P5vrz1q6XIqXxOVP2zPgc2AyQbz1RhLCWAzyKZwTRlLpOA0= ceph-fd220748-ed2b-11ee-95d0-87774f69a715 2024-03-28T17:55:22.879 DEBUG:teuthology.orchestra.run.smithi079:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDUi+jeSl5COOIlXQAk/lFBdgZWXIsZSspbsG9so++bg1fBAczn//xlfwOpRWKV+8NU3pC1kj15TfHrDeimhlKAjxRQFLaEDDFYlaKmiCVcjpdDzFhL4Dmv8fYmfwSKIdB+Y2UjHB3mf3h61Tk7oWagZwenEljWcSjAWDuPjRFYtefJOV/DfIVaUpcbHt+GzkbO96+XI17MnqFBT/Ms2kD4Ipb9IEg+tlEzde3zLCP1ZyIOpyHjFrsfGx5+HktyRHFxTrFonNfvWhRF4bpy9J47Qe7vGjUJD2lXOftS4741TJz7YBVRrFz2ukflvpwLR9volk6Rtava/3j/JI+2qqulshjsHhcMQ3pJ5jqCDfjj1DdK61Hzkb//S0gFWNi/X6zFgwSQ5sw6+m4A6OLFcw90bmac2HV0RwXzg8Svao51Xl9fu85aUVcp9h0CtnhtGrOc1GEfve5SwE/CW423P5vrz1q6XIqXxOVP2zPgc2AyQbz1RhLCWAzyKZwTRlLpOA0= ceph-fd220748-ed2b-11ee-95d0-87774f69a715' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-03-28T17:55:23.018 INFO:teuthology.orchestra.run.smithi079.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDUi+jeSl5COOIlXQAk/lFBdgZWXIsZSspbsG9so++bg1fBAczn//xlfwOpRWKV+8NU3pC1kj15TfHrDeimhlKAjxRQFLaEDDFYlaKmiCVcjpdDzFhL4Dmv8fYmfwSKIdB+Y2UjHB3mf3h61Tk7oWagZwenEljWcSjAWDuPjRFYtefJOV/DfIVaUpcbHt+GzkbO96+XI17MnqFBT/Ms2kD4Ipb9IEg+tlEzde3zLCP1ZyIOpyHjFrsfGx5+HktyRHFxTrFonNfvWhRF4bpy9J47Qe7vGjUJD2lXOftS4741TJz7YBVRrFz2ukflvpwLR9volk6Rtava/3j/JI+2qqulshjsHhcMQ3pJ5jqCDfjj1DdK61Hzkb//S0gFWNi/X6zFgwSQ5sw6+m4A6OLFcw90bmac2HV0RwXzg8Svao51Xl9fu85aUVcp9h0CtnhtGrOc1GEfve5SwE/CW423P5vrz1q6XIqXxOVP2zPgc2AyQbz1RhLCWAzyKZwTRlLpOA0= ceph-fd220748-ed2b-11ee-95d0-87774f69a715 2024-03-28T17:55:23.085 DEBUG:teuthology.orchestra.run.smithi134:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDUi+jeSl5COOIlXQAk/lFBdgZWXIsZSspbsG9so++bg1fBAczn//xlfwOpRWKV+8NU3pC1kj15TfHrDeimhlKAjxRQFLaEDDFYlaKmiCVcjpdDzFhL4Dmv8fYmfwSKIdB+Y2UjHB3mf3h61Tk7oWagZwenEljWcSjAWDuPjRFYtefJOV/DfIVaUpcbHt+GzkbO96+XI17MnqFBT/Ms2kD4Ipb9IEg+tlEzde3zLCP1ZyIOpyHjFrsfGx5+HktyRHFxTrFonNfvWhRF4bpy9J47Qe7vGjUJD2lXOftS4741TJz7YBVRrFz2ukflvpwLR9volk6Rtava/3j/JI+2qqulshjsHhcMQ3pJ5jqCDfjj1DdK61Hzkb//S0gFWNi/X6zFgwSQ5sw6+m4A6OLFcw90bmac2HV0RwXzg8Svao51Xl9fu85aUVcp9h0CtnhtGrOc1GEfve5SwE/CW423P5vrz1q6XIqXxOVP2zPgc2AyQbz1RhLCWAzyKZwTRlLpOA0= ceph-fd220748-ed2b-11ee-95d0-87774f69a715' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-03-28T17:55:23.221 INFO:teuthology.orchestra.run.smithi134.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDUi+jeSl5COOIlXQAk/lFBdgZWXIsZSspbsG9so++bg1fBAczn//xlfwOpRWKV+8NU3pC1kj15TfHrDeimhlKAjxRQFLaEDDFYlaKmiCVcjpdDzFhL4Dmv8fYmfwSKIdB+Y2UjHB3mf3h61Tk7oWagZwenEljWcSjAWDuPjRFYtefJOV/DfIVaUpcbHt+GzkbO96+XI17MnqFBT/Ms2kD4Ipb9IEg+tlEzde3zLCP1ZyIOpyHjFrsfGx5+HktyRHFxTrFonNfvWhRF4bpy9J47Qe7vGjUJD2lXOftS4741TJz7YBVRrFz2ukflvpwLR9volk6Rtava/3j/JI+2qqulshjsHhcMQ3pJ5jqCDfjj1DdK61Hzkb//S0gFWNi/X6zFgwSQ5sw6+m4A6OLFcw90bmac2HV0RwXzg8Svao51Xl9fu85aUVcp9h0CtnhtGrOc1GEfve5SwE/CW423P5vrz1q6XIqXxOVP2zPgc2AyQbz1RhLCWAzyKZwTRlLpOA0= ceph-fd220748-ed2b-11ee-95d0-87774f69a715 2024-03-28T17:55:23.288 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fd220748-ed2b-11ee-95d0-87774f69a715 -- ceph config set mgr mgr/cephadm/allow_ptrace true 2024-03-28T17:55:23.726 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/fd220748-ed2b-11ee-95d0-87774f69a715/mon.a/config 2024-03-28T17:55:26.328 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T17:55:26.327+0000 7f8b54a00700 1 -- 172.21.15.22:0/3716950603 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8b50103c40 msgr2=0x7f8b50104060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:55:26.328 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T17:55:26.327+0000 7f8b54a00700 1 --2- 172.21.15.22:0/3716950603 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8b50103c40 0x7f8b50104060 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f8b40009910 tx=0x7f8b40009c20 comp rx=0 tx=0).stop 2024-03-28T17:55:26.328 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T17:55:26.327+0000 7f8b54a00700 1 -- 172.21.15.22:0/3716950603 shutdown_connections 2024-03-28T17:55:26.328 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T17:55:26.327+0000 7f8b54a00700 1 --2- 172.21.15.22:0/3716950603 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8b50103c40 0x7f8b50104060 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:55:26.328 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T17:55:26.327+0000 7f8b54a00700 1 -- 172.21.15.22:0/3716950603 >> 172.21.15.22:0/3716950603 conn(0x7f8b500ff1c0 msgr2=0x7f8b50101620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T17:55:26.328 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T17:55:26.327+0000 7f8b54a00700 1 -- 172.21.15.22:0/3716950603 shutdown_connections 2024-03-28T17:55:26.328 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T17:55:26.327+0000 7f8b54a00700 1 -- 172.21.15.22:0/3716950603 wait complete. 2024-03-28T17:55:26.329 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T17:55:26.328+0000 7f8b54a00700 1 Processor -- start 2024-03-28T17:55:26.329 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T17:55:26.329+0000 7f8b54a00700 1 -- start start 2024-03-28T17:55:26.329 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T17:55:26.329+0000 7f8b54a00700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8b50103c40 0x7f8b501992d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:55:26.329 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T17:55:26.329+0000 7f8b54a00700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b50109cc0 con 0x7f8b50103c40 2024-03-28T17:55:26.330 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T17:55:26.329+0000 7f8b4e800700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8b50103c40 0x7f8b501992d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:55:26.330 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T17:55:26.329+0000 7f8b4e800700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8b50103c40 0x7f8b501992d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:58058/0 (socket says 172.21.15.22:58058) 2024-03-28T17:55:26.330 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T17:55:26.329+0000 7f8b4e800700 1 -- 172.21.15.22:0/1398397323 learned_addr learned my addr 172.21.15.22:0/1398397323 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-28T17:55:26.330 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T17:55:26.330+0000 7f8b4e800700 1 -- 172.21.15.22:0/1398397323 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b50199810 con 0x7f8b50103c40 2024-03-28T17:55:26.330 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T17:55:26.330+0000 7f8b4e800700 1 --2- 172.21.15.22:0/1398397323 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8b50103c40 0x7f8b501992d0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f8b40005950 tx=0x7f8b40009f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T17:55:26.331 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T17:55:26.330+0000 7f8b47400700 1 -- 172.21.15.22:0/1398397323 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f8b40027070 con 0x7f8b50103c40 2024-03-28T17:55:26.331 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T17:55:26.330+0000 7f8b47400700 1 -- 172.21.15.22:0/1398397323 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f8b4000be30 con 0x7f8b50103c40 2024-03-28T17:55:26.331 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T17:55:26.330+0000 7f8b54a00700 1 -- 172.21.15.22:0/1398397323 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8b50199aa0 con 0x7f8b50103c40 2024-03-28T17:55:26.331 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T17:55:26.330+0000 7f8b47400700 1 -- 172.21.15.22:0/1398397323 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f8b4000f460 con 0x7f8b50103c40 2024-03-28T17:55:26.331 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T17:55:26.331+0000 7f8b54a00700 1 -- 172.21.15.22:0/1398397323 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8b5019c790 con 0x7f8b50103c40 2024-03-28T17:55:26.331 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T17:55:26.332+0000 7f8b47400700 1 -- 172.21.15.22:0/1398397323 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 13) v1 ==== 46162+0+0 (secure 0 0 0) 0x7f8b4002a070 con 0x7f8b50103c40 2024-03-28T17:55:26.331 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T17:55:26.332+0000 7f8b54a00700 1 -- 172.21.15.22:0/1398397323 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8b5010a440 con 0x7f8b50103c40 2024-03-28T17:55:26.332 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T17:55:26.332+0000 7f8b47400700 1 --2- 172.21.15.22:0/1398397323 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7f8b34038fc0 0x7f8b3403b480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:55:26.332 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T17:55:26.332+0000 7f8b47400700 1 -- 172.21.15.22:0/1398397323 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f8b4001e030 con 0x7f8b50103c40 2024-03-28T17:55:26.332 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T17:55:26.333+0000 7f8b4de00700 1 --2- 172.21.15.22:0/1398397323 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7f8b34038fc0 0x7f8b3403b480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:55:26.333 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T17:55:26.333+0000 7f8b4de00700 1 --2- 172.21.15.22:0/1398397323 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7f8b34038fc0 0x7f8b3403b480 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f8b3c006fd0 tx=0x7f8b3c006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T17:55:26.338 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T17:55:26.338+0000 7f8b47400700 1 -- 172.21.15.22:0/1398397323 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f8b40013070 con 0x7f8b50103c40 2024-03-28T17:55:26.569 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T17:55:26.569+0000 7f8b54a00700 1 -- 172.21.15.22:0/1398397323 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/allow_ptrace}] v 0) v1 -- 0x7f8b5002db40 con 0x7f8b50103c40 2024-03-28T17:55:26.573 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T17:55:26.573+0000 7f8b47400700 1 -- 172.21.15.22:0/1398397323 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/allow_ptrace}]=0 v9)=0 v9) v1 ==== 125+0+0 (secure 0 0 0) 0x7f8b400227d0 con 0x7f8b50103c40 2024-03-28T17:55:26.577 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T17:55:26.576+0000 7f8b54a00700 1 -- 172.21.15.22:0/1398397323 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7f8b34038fc0 msgr2=0x7f8b3403b480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:55:26.577 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T17:55:26.576+0000 7f8b54a00700 1 --2- 172.21.15.22:0/1398397323 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7f8b34038fc0 0x7f8b3403b480 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f8b3c006fd0 tx=0x7f8b3c006e40 comp rx=0 tx=0).stop 2024-03-28T17:55:26.577 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T17:55:26.576+0000 7f8b54a00700 1 -- 172.21.15.22:0/1398397323 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8b50103c40 msgr2=0x7f8b501992d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:55:26.577 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T17:55:26.576+0000 7f8b54a00700 1 --2- 172.21.15.22:0/1398397323 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8b50103c40 0x7f8b501992d0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f8b40005950 tx=0x7f8b40009f10 comp rx=0 tx=0).stop 2024-03-28T17:55:26.577 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T17:55:26.576+0000 7f8b54a00700 1 -- 172.21.15.22:0/1398397323 shutdown_connections 2024-03-28T17:55:26.577 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T17:55:26.576+0000 7f8b54a00700 1 --2- 172.21.15.22:0/1398397323 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7f8b34038fc0 0x7f8b3403b480 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:55:26.577 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T17:55:26.576+0000 7f8b54a00700 1 --2- 172.21.15.22:0/1398397323 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8b50103c40 0x7f8b501992d0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:55:26.577 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T17:55:26.576+0000 7f8b54a00700 1 -- 172.21.15.22:0/1398397323 >> 172.21.15.22:0/1398397323 conn(0x7f8b500ff1c0 msgr2=0x7f8b500ffbd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T17:55:26.577 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T17:55:26.576+0000 7f8b54a00700 1 -- 172.21.15.22:0/1398397323 shutdown_connections 2024-03-28T17:55:26.577 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T17:55:26.577+0000 7f8b54a00700 1 -- 172.21.15.22:0/1398397323 wait complete. 2024-03-28T17:55:27.281 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:55:26 smithi022 ceph-mon[31985]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T17:55:27.282 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:55:26 smithi022 ceph-mon[31985]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T17:55:27.282 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:55:26 smithi022 ceph-mon[31985]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi022", "name": "osd_memory_target"}]: dispatch 2024-03-28T17:55:27.282 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:55:26 smithi022 ceph-mon[31985]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T17:55:27.282 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:55:26 smithi022 ceph-mon[31985]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T17:55:27.282 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:55:26 smithi022 ceph-mon[31985]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T17:55:27.282 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:55:26 smithi022 ceph-mon[31985]: from='client.? 172.21.15.22:0/1398397323' entity='client.admin' 2024-03-28T17:55:27.282 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:55:26 smithi022 ceph-mon[31985]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T17:55:27.282 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:55:26 smithi022 ceph-mon[31985]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T17:55:27.694 INFO:tasks.cephadm:Distributing conf and client.admin keyring to all hosts + 0755 2024-03-28T17:55:27.695 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fd220748-ed2b-11ee-95d0-87774f69a715 -- ceph orch client-keyring set client.admin '*' --mode 0755 2024-03-28T17:55:28.121 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/fd220748-ed2b-11ee-95d0-87774f69a715/mon.a/config 2024-03-28T17:55:31.016 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T17:55:31.015+0000 7efe2a800700 1 -- 172.21.15.22:0/340367718 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7efe241083a0 msgr2=0x7efe241087c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:55:31.016 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T17:55:31.015+0000 7efe2a800700 1 --2- 172.21.15.22:0/340367718 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7efe241083a0 0x7efe241087c0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7efe14009910 tx=0x7efe14009c20 comp rx=0 tx=0).stop 2024-03-28T17:55:31.016 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T17:55:31.015+0000 7efe2a800700 1 -- 172.21.15.22:0/340367718 shutdown_connections 2024-03-28T17:55:31.016 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T17:55:31.015+0000 7efe2a800700 1 --2- 172.21.15.22:0/340367718 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7efe241083a0 0x7efe241087c0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:55:31.016 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T17:55:31.015+0000 7efe2a800700 1 -- 172.21.15.22:0/340367718 >> 172.21.15.22:0/340367718 conn(0x7efe24077280 msgr2=0x7efe24079700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T17:55:31.017 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T17:55:31.016+0000 7efe2a800700 1 -- 172.21.15.22:0/340367718 shutdown_connections 2024-03-28T17:55:31.017 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T17:55:31.016+0000 7efe2a800700 1 -- 172.21.15.22:0/340367718 wait complete. 2024-03-28T17:55:31.017 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T17:55:31.017+0000 7efe2a800700 1 Processor -- start 2024-03-28T17:55:31.017 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T17:55:31.017+0000 7efe2a800700 1 -- start start 2024-03-28T17:55:31.018 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T17:55:31.017+0000 7efe2a800700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7efe241083a0 0x7efe2419d8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:55:31.018 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T17:55:31.017+0000 7efe2a800700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe2410e210 con 0x7efe241083a0 2024-03-28T17:55:31.018 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T17:55:31.018+0000 7efe23400700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7efe241083a0 0x7efe2419d8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:55:31.018 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T17:55:31.018+0000 7efe23400700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7efe241083a0 0x7efe2419d8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:34034/0 (socket says 172.21.15.22:34034) 2024-03-28T17:55:31.018 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T17:55:31.018+0000 7efe23400700 1 -- 172.21.15.22:0/3582271140 learned_addr learned my addr 172.21.15.22:0/3582271140 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-28T17:55:31.019 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T17:55:31.018+0000 7efe23400700 1 -- 172.21.15.22:0/3582271140 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efe2419de00 con 0x7efe241083a0 2024-03-28T17:55:31.019 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T17:55:31.018+0000 7efe23400700 1 --2- 172.21.15.22:0/3582271140 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7efe241083a0 0x7efe2419d8c0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7efe14005950 tx=0x7efe14009f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T17:55:31.019 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T17:55:31.019+0000 7efe1be00700 1 -- 172.21.15.22:0/3582271140 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7efe14027070 con 0x7efe241083a0 2024-03-28T17:55:31.019 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T17:55:31.019+0000 7efe1be00700 1 -- 172.21.15.22:0/3582271140 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7efe14022470 con 0x7efe241083a0 2024-03-28T17:55:31.019 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T17:55:31.019+0000 7efe1be00700 1 -- 172.21.15.22:0/3582271140 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7efe1400f5f0 con 0x7efe241083a0 2024-03-28T17:55:31.019 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T17:55:31.019+0000 7efe2a800700 1 -- 172.21.15.22:0/3582271140 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efe2419e090 con 0x7efe241083a0 2024-03-28T17:55:31.019 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T17:55:31.019+0000 7efe2a800700 1 -- 172.21.15.22:0/3582271140 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efe241a0d80 con 0x7efe241083a0 2024-03-28T17:55:31.020 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T17:55:31.020+0000 7efe1be00700 1 -- 172.21.15.22:0/3582271140 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 13) v1 ==== 46162+0+0 (secure 0 0 0) 0x7efe1402a070 con 0x7efe241083a0 2024-03-28T17:55:31.020 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T17:55:31.020+0000 7efe2a800700 1 -- 172.21.15.22:0/3582271140 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efe2410e990 con 0x7efe241083a0 2024-03-28T17:55:31.020 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T17:55:31.021+0000 7efe1be00700 1 --2- 172.21.15.22:0/3582271140 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7efe08039020 0x7efe0803b4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:55:31.021 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T17:55:31.021+0000 7efe1be00700 1 -- 172.21.15.22:0/3582271140 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7efe1401e030 con 0x7efe241083a0 2024-03-28T17:55:31.021 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T17:55:31.021+0000 7efe22a00700 1 --2- 172.21.15.22:0/3582271140 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7efe08039020 0x7efe0803b4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:55:31.021 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T17:55:31.022+0000 7efe22a00700 1 --2- 172.21.15.22:0/3582271140 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7efe08039020 0x7efe0803b4e0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7efe10006fd0 tx=0x7efe10006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T17:55:31.026 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T17:55:31.026+0000 7efe1be00700 1 -- 172.21.15.22:0/3582271140 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7efe14013070 con 0x7efe241083a0 2024-03-28T17:55:31.256 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T17:55:31.256+0000 7efe2a800700 1 -- 172.21.15.22:0/3582271140 --> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] -- mgr_command(tid 0: {"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}) v1 -- 0x7efe2402db40 con 0x7efe08039020 2024-03-28T17:55:31.266 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T17:55:31.266+0000 7efe1be00700 1 -- 172.21.15.22:0/3582271140 <== mgr.14152 v2:172.21.15.22:6800/4201048392 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7efe2402db40 con 0x7efe08039020 2024-03-28T17:55:31.269 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T17:55:31.269+0000 7efe2a800700 1 -- 172.21.15.22:0/3582271140 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7efe08039020 msgr2=0x7efe0803b4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:55:31.269 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T17:55:31.269+0000 7efe2a800700 1 --2- 172.21.15.22:0/3582271140 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7efe08039020 0x7efe0803b4e0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7efe10006fd0 tx=0x7efe10006e40 comp rx=0 tx=0).stop 2024-03-28T17:55:31.269 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T17:55:31.269+0000 7efe2a800700 1 -- 172.21.15.22:0/3582271140 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7efe241083a0 msgr2=0x7efe2419d8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:55:31.269 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T17:55:31.269+0000 7efe2a800700 1 --2- 172.21.15.22:0/3582271140 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7efe241083a0 0x7efe2419d8c0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7efe14005950 tx=0x7efe14009f10 comp rx=0 tx=0).stop 2024-03-28T17:55:31.270 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T17:55:31.269+0000 7efe2a800700 1 -- 172.21.15.22:0/3582271140 shutdown_connections 2024-03-28T17:55:31.270 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T17:55:31.269+0000 7efe2a800700 1 --2- 172.21.15.22:0/3582271140 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7efe08039020 0x7efe0803b4e0 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:55:31.270 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T17:55:31.269+0000 7efe2a800700 1 --2- 172.21.15.22:0/3582271140 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7efe241083a0 0x7efe2419d8c0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:55:31.270 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T17:55:31.269+0000 7efe2a800700 1 -- 172.21.15.22:0/3582271140 >> 172.21.15.22:0/3582271140 conn(0x7efe24077280 msgr2=0x7efe24077cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T17:55:31.270 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T17:55:31.269+0000 7efe2a800700 1 -- 172.21.15.22:0/3582271140 shutdown_connections 2024-03-28T17:55:31.270 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T17:55:31.269+0000 7efe2a800700 1 -- 172.21.15.22:0/3582271140 wait complete. 2024-03-28T17:55:32.088 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi079 2024-03-28T17:55:32.088 DEBUG:teuthology.orchestra.run.smithi079:> set -ex 2024-03-28T17:55:32.089 DEBUG:teuthology.orchestra.run.smithi079:> dd of=/etc/ceph/ceph.conf 2024-03-28T17:55:32.147 DEBUG:teuthology.orchestra.run.smithi079:> set -ex 2024-03-28T17:55:32.147 DEBUG:teuthology.orchestra.run.smithi079:> dd of=/etc/ceph/ceph.client.admin.keyring 2024-03-28T17:55:32.241 INFO:tasks.cephadm:Adding host smithi079 to orchestrator... 2024-03-28T17:55:32.241 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fd220748-ed2b-11ee-95d0-87774f69a715 -- ceph orch host add smithi079 2024-03-28T17:55:32.281 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:55:32 smithi022 ceph-mon[31985]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-28T17:55:32.281 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:55:32 smithi022 ceph-mon[31985]: from='client.14176 -' entity='client.admin' cmd=[{"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}]: dispatch 2024-03-28T17:55:32.281 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:55:32 smithi022 ceph-mon[31985]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T17:55:32.282 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:55:32 smithi022 ceph-mon[31985]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T17:55:32.282 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:55:32 smithi022 ceph-mon[31985]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T17:55:32.282 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:55:32 smithi022 ceph-mon[31985]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T17:55:32.282 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:55:32 smithi022 ceph-mon[31985]: Updating smithi022:/etc/ceph/ceph.conf 2024-03-28T17:55:32.679 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/fd220748-ed2b-11ee-95d0-87774f69a715/mon.a/config 2024-03-28T17:55:33.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:55:33 smithi022 ceph-mon[31985]: Updating smithi022:/var/lib/ceph/fd220748-ed2b-11ee-95d0-87774f69a715/config/ceph.conf 2024-03-28T17:55:34.533 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:55:34 smithi022 ceph-mon[31985]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-28T17:55:34.533 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:55:34 smithi022 ceph-mon[31985]: Updating smithi022:/etc/ceph/ceph.client.admin.keyring 2024-03-28T17:55:34.533 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:55:34 smithi022 ceph-mon[31985]: Updating smithi022:/var/lib/ceph/fd220748-ed2b-11ee-95d0-87774f69a715/config/ceph.client.admin.keyring 2024-03-28T17:55:34.533 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:55:34 smithi022 ceph-mon[31985]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T17:55:34.533 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:55:34 smithi022 ceph-mon[31985]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T17:55:34.533 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:55:34 smithi022 ceph-mon[31985]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T17:55:34.775 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T17:55:34.774+0000 7fd5bb200700 1 -- 172.21.15.22:0/3253634764 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd5bc0745c0 msgr2=0x7fd5bc0749a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:55:34.775 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T17:55:34.774+0000 7fd5bb200700 1 --2- 172.21.15.22:0/3253634764 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd5bc0745c0 0x7fd5bc0749a0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fd5ac009910 tx=0x7fd5ac009c20 comp rx=0 tx=0).stop 2024-03-28T17:55:34.775 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T17:55:34.774+0000 7fd5bb200700 1 -- 172.21.15.22:0/3253634764 shutdown_connections 2024-03-28T17:55:34.775 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T17:55:34.774+0000 7fd5bb200700 1 --2- 172.21.15.22:0/3253634764 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd5bc0745c0 0x7fd5bc0749a0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:55:34.775 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T17:55:34.774+0000 7fd5bb200700 1 -- 172.21.15.22:0/3253634764 >> 172.21.15.22:0/3253634764 conn(0x7fd5bc0fd4c0 msgr2=0x7fd5bc0ff8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T17:55:34.775 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T17:55:34.774+0000 7fd5bb200700 1 -- 172.21.15.22:0/3253634764 shutdown_connections 2024-03-28T17:55:34.775 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T17:55:34.774+0000 7fd5bb200700 1 -- 172.21.15.22:0/3253634764 wait complete. 2024-03-28T17:55:34.776 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T17:55:34.776+0000 7fd5bb200700 1 Processor -- start 2024-03-28T17:55:34.776 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T17:55:34.776+0000 7fd5bb200700 1 -- start start 2024-03-28T17:55:34.776 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T17:55:34.776+0000 7fd5bb200700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd5bc0745c0 0x7fd5bc19f3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:55:34.776 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T17:55:34.776+0000 7fd5bb200700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5bc10e240 con 0x7fd5bc0745c0 2024-03-28T17:55:34.777 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T17:55:34.777+0000 7fd5b9e00700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd5bc0745c0 0x7fd5bc19f3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:55:34.777 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T17:55:34.777+0000 7fd5b9e00700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd5bc0745c0 0x7fd5bc19f3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:34040/0 (socket says 172.21.15.22:34040) 2024-03-28T17:55:34.777 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T17:55:34.777+0000 7fd5b9e00700 1 -- 172.21.15.22:0/1815745427 learned_addr learned my addr 172.21.15.22:0/1815745427 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-28T17:55:34.777 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T17:55:34.777+0000 7fd5b9e00700 1 -- 172.21.15.22:0/1815745427 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd5bc19f930 con 0x7fd5bc0745c0 2024-03-28T17:55:34.777 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T17:55:34.777+0000 7fd5b9e00700 1 --2- 172.21.15.22:0/1815745427 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd5bc0745c0 0x7fd5bc19f3f0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fd5ac006010 tx=0x7fd5ac005810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T17:55:34.778 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T17:55:34.778+0000 7fd5b3400700 1 -- 172.21.15.22:0/1815745427 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd5ac026070 con 0x7fd5bc0745c0 2024-03-28T17:55:34.778 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T17:55:34.778+0000 7fd5b3400700 1 -- 172.21.15.22:0/1815745427 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7fd5ac00bc20 con 0x7fd5bc0745c0 2024-03-28T17:55:34.778 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T17:55:34.778+0000 7fd5b3400700 1 -- 172.21.15.22:0/1815745427 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd5ac003a50 con 0x7fd5bc0745c0 2024-03-28T17:55:34.778 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T17:55:34.778+0000 7fd5bb200700 1 -- 172.21.15.22:0/1815745427 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd5bc19fbc0 con 0x7fd5bc0745c0 2024-03-28T17:55:34.778 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T17:55:34.778+0000 7fd5bb200700 1 -- 172.21.15.22:0/1815745427 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd5bc1995b0 con 0x7fd5bc0745c0 2024-03-28T17:55:34.779 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T17:55:34.779+0000 7fd5bb200700 1 -- 172.21.15.22:0/1815745427 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd5bc075c70 con 0x7fd5bc0745c0 2024-03-28T17:55:34.779 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T17:55:34.779+0000 7fd5b3400700 1 -- 172.21.15.22:0/1815745427 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 13) v1 ==== 46162+0+0 (secure 0 0 0) 0x7fd5ac029070 con 0x7fd5bc0745c0 2024-03-28T17:55:34.780 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T17:55:34.780+0000 7fd5b3400700 1 --2- 172.21.15.22:0/1815745427 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7fd5a0039020 0x7fd5a003b4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:55:34.780 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T17:55:34.780+0000 7fd5b3400700 1 -- 172.21.15.22:0/1815745427 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7fd5ac01e070 con 0x7fd5bc0745c0 2024-03-28T17:55:34.780 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T17:55:34.780+0000 7fd5b1400700 1 --2- 172.21.15.22:0/1815745427 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7fd5a0039020 0x7fd5a003b4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:55:34.781 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T17:55:34.781+0000 7fd5b1400700 1 --2- 172.21.15.22:0/1815745427 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7fd5a0039020 0x7fd5a003b4e0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fd5a4006fd0 tx=0x7fd5a4006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T17:55:34.785 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T17:55:34.786+0000 7fd5b3400700 1 -- 172.21.15.22:0/1815745427 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fd5ac013070 con 0x7fd5bc0745c0 2024-03-28T17:55:35.017 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T17:55:35.016+0000 7fd5bb200700 1 -- 172.21.15.22:0/1815745427 --> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi079", "target": ["mon-mgr", ""]}) v1 -- 0x7fd5bc19aae0 con 0x7fd5a0039020 2024-03-28T17:55:36.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:55:36 smithi022 ceph-mon[31985]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-28T17:55:36.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:55:36 smithi022 ceph-mon[31985]: from='client.14178 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi079", "target": ["mon-mgr", ""]}]: dispatch 2024-03-28T17:55:38.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:55:38 smithi022 ceph-mon[31985]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-28T17:55:38.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:55:38 smithi022 ceph-mon[31985]: Deploying cephadm binary to smithi079 2024-03-28T17:55:40.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:55:40 smithi022 ceph-mon[31985]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-28T17:55:41.638 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T17:55:41.637+0000 7fd5b3400700 1 -- 172.21.15.22:0/1815745427 <== mgr.14152 v2:172.21.15.22:6800/4201048392 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+48 (secure 0 0 0) 0x7fd5bc19aae0 con 0x7fd5a0039020 2024-03-28T17:55:41.638 INFO:teuthology.orchestra.run.smithi022.stdout:Added host 'smithi079' with addr '172.21.15.79' 2024-03-28T17:55:41.642 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T17:55:41.641+0000 7fd5bb200700 1 -- 172.21.15.22:0/1815745427 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7fd5a0039020 msgr2=0x7fd5a003b4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:55:41.642 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T17:55:41.641+0000 7fd5bb200700 1 --2- 172.21.15.22:0/1815745427 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7fd5a0039020 0x7fd5a003b4e0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fd5a4006fd0 tx=0x7fd5a4006e40 comp rx=0 tx=0).stop 2024-03-28T17:55:41.642 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T17:55:41.642+0000 7fd5bb200700 1 -- 172.21.15.22:0/1815745427 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd5bc0745c0 msgr2=0x7fd5bc19f3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:55:41.642 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T17:55:41.642+0000 7fd5bb200700 1 --2- 172.21.15.22:0/1815745427 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd5bc0745c0 0x7fd5bc19f3f0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fd5ac006010 tx=0x7fd5ac005810 comp rx=0 tx=0).stop 2024-03-28T17:55:41.642 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T17:55:41.642+0000 7fd5bb200700 1 -- 172.21.15.22:0/1815745427 shutdown_connections 2024-03-28T17:55:41.642 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T17:55:41.642+0000 7fd5bb200700 1 --2- 172.21.15.22:0/1815745427 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7fd5a0039020 0x7fd5a003b4e0 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:55:41.642 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T17:55:41.642+0000 7fd5bb200700 1 --2- 172.21.15.22:0/1815745427 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd5bc0745c0 0x7fd5bc19f3f0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:55:41.643 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T17:55:41.642+0000 7fd5bb200700 1 -- 172.21.15.22:0/1815745427 >> 172.21.15.22:0/1815745427 conn(0x7fd5bc0fd4c0 msgr2=0x7fd5bc105c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T17:55:41.643 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T17:55:41.642+0000 7fd5bb200700 1 -- 172.21.15.22:0/1815745427 shutdown_connections 2024-03-28T17:55:41.643 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T17:55:41.642+0000 7fd5bb200700 1 -- 172.21.15.22:0/1815745427 wait complete. 2024-03-28T17:55:42.361 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:55:42 smithi022 ceph-mon[31985]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-28T17:55:42.362 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:55:42 smithi022 ceph-mon[31985]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T17:55:42.362 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:55:42 smithi022 ceph-mon[31985]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T17:55:42.436 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fd220748-ed2b-11ee-95d0-87774f69a715 -- ceph orch host ls --format=json 2024-03-28T17:55:43.056 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/fd220748-ed2b-11ee-95d0-87774f69a715/mon.a/config 2024-03-28T17:55:43.498 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:55:43 smithi022 ceph-mon[31985]: Added host smithi079 2024-03-28T17:55:43.499 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:55:43 smithi022 ceph-mon[31985]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T17:55:44.587 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:55:44 smithi022 ceph-mon[31985]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-28T17:55:44.587 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:55:44 smithi022 ceph-mon[31985]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T17:55:45.063 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T17:55:45.061+0000 7ff9e5a00700 1 -- 172.21.15.22:0/3705971945 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff9e0102d10 msgr2=0x7ff9e01030f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:55:45.063 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T17:55:45.061+0000 7ff9e5a00700 1 --2- 172.21.15.22:0/3705971945 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff9e0102d10 0x7ff9e01030f0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7ff9d0009910 tx=0x7ff9d0009c20 comp rx=0 tx=0).stop 2024-03-28T17:55:45.063 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T17:55:45.061+0000 7ff9e5a00700 1 -- 172.21.15.22:0/3705971945 shutdown_connections 2024-03-28T17:55:45.063 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T17:55:45.061+0000 7ff9e5a00700 1 --2- 172.21.15.22:0/3705971945 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff9e0102d10 0x7ff9e01030f0 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:55:45.063 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T17:55:45.061+0000 7ff9e5a00700 1 -- 172.21.15.22:0/3705971945 >> 172.21.15.22:0/3705971945 conn(0x7ff9e00fc540 msgr2=0x7ff9e00fe960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T17:55:45.063 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T17:55:45.061+0000 7ff9e5a00700 1 -- 172.21.15.22:0/3705971945 shutdown_connections 2024-03-28T17:55:45.063 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T17:55:45.061+0000 7ff9e5a00700 1 -- 172.21.15.22:0/3705971945 wait complete. 2024-03-28T17:55:45.063 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T17:55:45.062+0000 7ff9e5a00700 1 Processor -- start 2024-03-28T17:55:45.063 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T17:55:45.063+0000 7ff9e5a00700 1 -- start start 2024-03-28T17:55:45.064 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T17:55:45.063+0000 7ff9e5a00700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff9e0102d10 0x7ff9e019f650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:55:45.064 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T17:55:45.063+0000 7ff9e5a00700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9e010c1f0 con 0x7ff9e0102d10 2024-03-28T17:55:45.065 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T17:55:45.063+0000 7ff9dea00700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff9e0102d10 0x7ff9e019f650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:55:45.065 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T17:55:45.063+0000 7ff9dea00700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff9e0102d10 0x7ff9e019f650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:46506/0 (socket says 172.21.15.22:46506) 2024-03-28T17:55:45.065 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T17:55:45.063+0000 7ff9dea00700 1 -- 172.21.15.22:0/458271134 learned_addr learned my addr 172.21.15.22:0/458271134 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-28T17:55:45.065 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T17:55:45.064+0000 7ff9dea00700 1 -- 172.21.15.22:0/458271134 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff9e019fb90 con 0x7ff9e0102d10 2024-03-28T17:55:45.065 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T17:55:45.064+0000 7ff9dea00700 1 --2- 172.21.15.22:0/458271134 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff9e0102d10 0x7ff9e019f650 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7ff9d0006010 tx=0x7ff9d0005810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T17:55:45.065 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T17:55:45.064+0000 7ff9d7400700 1 -- 172.21.15.22:0/458271134 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ff9d0026070 con 0x7ff9e0102d10 2024-03-28T17:55:45.065 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T17:55:45.064+0000 7ff9d7400700 1 -- 172.21.15.22:0/458271134 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7ff9d000bc20 con 0x7ff9e0102d10 2024-03-28T17:55:45.065 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T17:55:45.064+0000 7ff9d7400700 1 -- 172.21.15.22:0/458271134 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ff9d0003a50 con 0x7ff9e0102d10 2024-03-28T17:55:45.065 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T17:55:45.064+0000 7ff9e5a00700 1 -- 172.21.15.22:0/458271134 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff9e019fe20 con 0x7ff9e0102d10 2024-03-28T17:55:45.065 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T17:55:45.064+0000 7ff9e5a00700 1 -- 172.21.15.22:0/458271134 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff9e0199720 con 0x7ff9e0102d10 2024-03-28T17:55:45.067 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T17:55:45.065+0000 7ff9d7400700 1 -- 172.21.15.22:0/458271134 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 13) v1 ==== 46162+0+0 (secure 0 0 0) 0x7ff9d0029070 con 0x7ff9e0102d10 2024-03-28T17:55:45.067 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T17:55:45.065+0000 7ff9e5a00700 1 -- 172.21.15.22:0/458271134 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff9e006a6f0 con 0x7ff9e0102d10 2024-03-28T17:55:45.067 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T17:55:45.066+0000 7ff9d7400700 1 --2- 172.21.15.22:0/458271134 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7ff9c4039020 0x7ff9c403b4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:55:45.067 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T17:55:45.066+0000 7ff9d7400700 1 -- 172.21.15.22:0/458271134 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7ff9d001e070 con 0x7ff9e0102d10 2024-03-28T17:55:45.067 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T17:55:45.066+0000 7ff9de000700 1 --2- 172.21.15.22:0/458271134 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7ff9c4039020 0x7ff9c403b4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:55:45.067 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T17:55:45.067+0000 7ff9de000700 1 --2- 172.21.15.22:0/458271134 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7ff9c4039020 0x7ff9c403b4e0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7ff9cc006fd0 tx=0x7ff9cc006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T17:55:45.071 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T17:55:45.072+0000 7ff9d7400700 1 -- 172.21.15.22:0/458271134 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7ff9d0013070 con 0x7ff9e0102d10 2024-03-28T17:55:45.308 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T17:55:45.307+0000 7ff9e5a00700 1 -- 172.21.15.22:0/458271134 --> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7ff9e019ac50 con 0x7ff9c4039020 2024-03-28T17:55:45.310 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T17:55:45.310+0000 7ff9d7400700 1 -- 172.21.15.22:0/458271134 <== mgr.14152 v2:172.21.15.22:6800/4201048392 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+159 (secure 0 0 0) 0x7ff9e019ac50 con 0x7ff9c4039020 2024-03-28T17:55:45.310 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-03-28T17:55:45.310 INFO:teuthology.orchestra.run.smithi022.stdout:[{"addr": "172.21.15.22", "hostname": "smithi022", "labels": [], "status": ""}, {"addr": "172.21.15.79", "hostname": "smithi079", "labels": [], "status": ""}] 2024-03-28T17:55:45.313 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T17:55:45.312+0000 7ff9e5a00700 1 -- 172.21.15.22:0/458271134 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7ff9c4039020 msgr2=0x7ff9c403b4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:55:45.313 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T17:55:45.312+0000 7ff9e5a00700 1 --2- 172.21.15.22:0/458271134 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7ff9c4039020 0x7ff9c403b4e0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7ff9cc006fd0 tx=0x7ff9cc006e40 comp rx=0 tx=0).stop 2024-03-28T17:55:45.313 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T17:55:45.312+0000 7ff9e5a00700 1 -- 172.21.15.22:0/458271134 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff9e0102d10 msgr2=0x7ff9e019f650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:55:45.313 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T17:55:45.312+0000 7ff9e5a00700 1 --2- 172.21.15.22:0/458271134 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff9e0102d10 0x7ff9e019f650 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7ff9d0006010 tx=0x7ff9d0005810 comp rx=0 tx=0).stop 2024-03-28T17:55:45.313 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T17:55:45.313+0000 7ff9e5a00700 1 -- 172.21.15.22:0/458271134 shutdown_connections 2024-03-28T17:55:45.313 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T17:55:45.313+0000 7ff9e5a00700 1 --2- 172.21.15.22:0/458271134 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7ff9c4039020 0x7ff9c403b4e0 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:55:45.313 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T17:55:45.313+0000 7ff9e5a00700 1 --2- 172.21.15.22:0/458271134 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff9e0102d10 0x7ff9e019f650 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:55:45.313 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T17:55:45.313+0000 7ff9e5a00700 1 -- 172.21.15.22:0/458271134 >> 172.21.15.22:0/458271134 conn(0x7ff9e00fc540 msgr2=0x7ff9e0107eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T17:55:45.313 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T17:55:45.313+0000 7ff9e5a00700 1 -- 172.21.15.22:0/458271134 shutdown_connections 2024-03-28T17:55:45.313 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T17:55:45.313+0000 7ff9e5a00700 1 -- 172.21.15.22:0/458271134 wait complete. 2024-03-28T17:55:46.107 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi134 2024-03-28T17:55:46.107 DEBUG:teuthology.orchestra.run.smithi134:> set -ex 2024-03-28T17:55:46.107 DEBUG:teuthology.orchestra.run.smithi134:> dd of=/etc/ceph/ceph.conf 2024-03-28T17:55:46.165 DEBUG:teuthology.orchestra.run.smithi134:> set -ex 2024-03-28T17:55:46.165 DEBUG:teuthology.orchestra.run.smithi134:> dd of=/etc/ceph/ceph.client.admin.keyring 2024-03-28T17:55:46.259 INFO:tasks.cephadm:Adding host smithi134 to orchestrator... 2024-03-28T17:55:46.259 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fd220748-ed2b-11ee-95d0-87774f69a715 -- ceph orch host add smithi134 2024-03-28T17:55:46.732 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/fd220748-ed2b-11ee-95d0-87774f69a715/mon.a/config 2024-03-28T17:55:46.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:55:46 smithi022 ceph-mon[31985]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-28T17:55:46.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:55:46 smithi022 ceph-mon[31985]: from='client.14180 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-28T17:55:46.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:55:46 smithi022 ceph-mon[31985]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T17:55:48.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:55:48 smithi022 ceph-mon[31985]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-28T17:55:48.718 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T17:55:48.716+0000 7fe0e6600700 1 -- 172.21.15.22:0/3861014300 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe0e0101dd0 msgr2=0x7fe0e01021b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:55:48.718 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T17:55:48.716+0000 7fe0e6600700 1 --2- 172.21.15.22:0/3861014300 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe0e0101dd0 0x7fe0e01021b0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fe0d0009910 tx=0x7fe0d0009c20 comp rx=0 tx=0).stop 2024-03-28T17:55:48.718 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T17:55:48.717+0000 7fe0e6600700 1 -- 172.21.15.22:0/3861014300 shutdown_connections 2024-03-28T17:55:48.718 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T17:55:48.717+0000 7fe0e6600700 1 --2- 172.21.15.22:0/3861014300 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe0e0101dd0 0x7fe0e01021b0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:55:48.718 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T17:55:48.717+0000 7fe0e6600700 1 -- 172.21.15.22:0/3861014300 >> 172.21.15.22:0/3861014300 conn(0x7fe0e00fd6a0 msgr2=0x7fe0e00ffac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T17:55:48.718 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T17:55:48.717+0000 7fe0e6600700 1 -- 172.21.15.22:0/3861014300 shutdown_connections 2024-03-28T17:55:48.719 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T17:55:48.717+0000 7fe0e6600700 1 -- 172.21.15.22:0/3861014300 wait complete. 2024-03-28T17:55:48.719 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T17:55:48.718+0000 7fe0e6600700 1 Processor -- start 2024-03-28T17:55:48.719 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T17:55:48.718+0000 7fe0e6600700 1 -- start start 2024-03-28T17:55:48.720 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T17:55:48.719+0000 7fe0e6600700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe0e0101dd0 0x7fe0e019f610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:55:48.720 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T17:55:48.719+0000 7fe0e6600700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0e010c190 con 0x7fe0e0101dd0 2024-03-28T17:55:48.720 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T17:55:48.719+0000 7fe0df400700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe0e0101dd0 0x7fe0e019f610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:55:48.720 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T17:55:48.719+0000 7fe0df400700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe0e0101dd0 0x7fe0e019f610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:45198/0 (socket says 172.21.15.22:45198) 2024-03-28T17:55:48.720 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T17:55:48.719+0000 7fe0df400700 1 -- 172.21.15.22:0/276401313 learned_addr learned my addr 172.21.15.22:0/276401313 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-28T17:55:48.720 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T17:55:48.719+0000 7fe0df400700 1 -- 172.21.15.22:0/276401313 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe0e019fb50 con 0x7fe0e0101dd0 2024-03-28T17:55:48.720 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T17:55:48.720+0000 7fe0df400700 1 --2- 172.21.15.22:0/276401313 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe0e0101dd0 0x7fe0e019f610 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fe0d0006010 tx=0x7fe0d0005810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T17:55:48.720 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T17:55:48.720+0000 7fe0d7e00700 1 -- 172.21.15.22:0/276401313 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fe0d0026070 con 0x7fe0e0101dd0 2024-03-28T17:55:48.720 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T17:55:48.720+0000 7fe0d7e00700 1 -- 172.21.15.22:0/276401313 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7fe0d000bbc0 con 0x7fe0e0101dd0 2024-03-28T17:55:48.720 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T17:55:48.720+0000 7fe0d7e00700 1 -- 172.21.15.22:0/276401313 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fe0d00038a0 con 0x7fe0e0101dd0 2024-03-28T17:55:48.721 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T17:55:48.720+0000 7fe0e6600700 1 -- 172.21.15.22:0/276401313 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe0e019fe40 con 0x7fe0e0101dd0 2024-03-28T17:55:48.721 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T17:55:48.720+0000 7fe0e6600700 1 -- 172.21.15.22:0/276401313 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe0e01998b0 con 0x7fe0e0101dd0 2024-03-28T17:55:48.721 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T17:55:48.721+0000 7fe0d7e00700 1 -- 172.21.15.22:0/276401313 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 13) v1 ==== 46162+0+0 (secure 0 0 0) 0x7fe0d0029070 con 0x7fe0e0101dd0 2024-03-28T17:55:48.721 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T17:55:48.721+0000 7fe0e6600700 1 -- 172.21.15.22:0/276401313 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe0e01032b0 con 0x7fe0e0101dd0 2024-03-28T17:55:48.722 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T17:55:48.722+0000 7fe0d7e00700 1 --2- 172.21.15.22:0/276401313 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7fe0c4039020 0x7fe0c403b4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:55:48.722 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T17:55:48.722+0000 7fe0d7e00700 1 -- 172.21.15.22:0/276401313 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7fe0d001e070 con 0x7fe0e0101dd0 2024-03-28T17:55:48.722 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T17:55:48.722+0000 7fe0dea00700 1 --2- 172.21.15.22:0/276401313 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7fe0c4039020 0x7fe0c403b4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:55:48.722 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T17:55:48.723+0000 7fe0dea00700 1 --2- 172.21.15.22:0/276401313 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7fe0c4039020 0x7fe0c403b4e0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fe0cc006fd0 tx=0x7fe0cc006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T17:55:48.727 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T17:55:48.728+0000 7fe0d7e00700 1 -- 172.21.15.22:0/276401313 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fe0d0013070 con 0x7fe0e0101dd0 2024-03-28T17:55:48.960 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T17:55:48.960+0000 7fe0e6600700 1 -- 172.21.15.22:0/276401313 --> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi134", "target": ["mon-mgr", ""]}) v1 -- 0x7fe0e019ac10 con 0x7fe0c4039020 2024-03-28T17:55:50.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:55:50 smithi022 ceph-mon[31985]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-28T17:55:50.784 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:55:50 smithi022 ceph-mon[31985]: from='client.14182 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi134", "target": ["mon-mgr", ""]}]: dispatch 2024-03-28T17:55:50.784 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:55:50 smithi022 ceph-mon[31985]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T17:55:50.784 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:55:50 smithi022 ceph-mon[31985]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T17:55:50.784 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:55:50 smithi022 ceph-mon[31985]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T17:55:51.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:55:51 smithi022 ceph-mon[31985]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T17:55:51.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:55:51 smithi022 ceph-mon[31985]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi079", "name": "osd_memory_target"}]: dispatch 2024-03-28T17:55:51.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:55:51 smithi022 ceph-mon[31985]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T17:55:51.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:55:51 smithi022 ceph-mon[31985]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T17:55:51.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:55:51 smithi022 ceph-mon[31985]: Updating smithi079:/etc/ceph/ceph.conf 2024-03-28T17:55:52.781 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:55:52 smithi022 ceph-mon[31985]: Deploying cephadm binary to smithi134 2024-03-28T17:55:52.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:55:52 smithi022 ceph-mon[31985]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-28T17:55:52.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:55:52 smithi022 ceph-mon[31985]: Updating smithi079:/var/lib/ceph/fd220748-ed2b-11ee-95d0-87774f69a715/config/ceph.conf 2024-03-28T17:55:54.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:55:54 smithi022 ceph-mon[31985]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-28T17:55:55.613 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T17:55:55.613+0000 7fe0d7e00700 1 -- 172.21.15.22:0/276401313 <== mgr.14152 v2:172.21.15.22:6800/4201048392 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+49 (secure 0 0 0) 0x7fe0e019ac10 con 0x7fe0c4039020 2024-03-28T17:55:55.614 INFO:teuthology.orchestra.run.smithi022.stdout:Added host 'smithi134' with addr '172.21.15.134' 2024-03-28T17:55:55.617 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T17:55:55.616+0000 7fe0e6600700 1 -- 172.21.15.22:0/276401313 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7fe0c4039020 msgr2=0x7fe0c403b4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:55:55.617 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T17:55:55.616+0000 7fe0e6600700 1 --2- 172.21.15.22:0/276401313 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7fe0c4039020 0x7fe0c403b4e0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fe0cc006fd0 tx=0x7fe0cc006e40 comp rx=0 tx=0).stop 2024-03-28T17:55:55.617 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T17:55:55.616+0000 7fe0e6600700 1 -- 172.21.15.22:0/276401313 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe0e0101dd0 msgr2=0x7fe0e019f610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:55:55.617 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T17:55:55.616+0000 7fe0e6600700 1 --2- 172.21.15.22:0/276401313 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe0e0101dd0 0x7fe0e019f610 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fe0d0006010 tx=0x7fe0d0005810 comp rx=0 tx=0).stop 2024-03-28T17:55:55.617 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T17:55:55.616+0000 7fe0e6600700 1 -- 172.21.15.22:0/276401313 shutdown_connections 2024-03-28T17:55:55.617 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T17:55:55.616+0000 7fe0e6600700 1 --2- 172.21.15.22:0/276401313 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7fe0c4039020 0x7fe0c403b4e0 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:55:55.617 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T17:55:55.616+0000 7fe0e6600700 1 --2- 172.21.15.22:0/276401313 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe0e0101dd0 0x7fe0e019f610 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:55:55.617 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T17:55:55.616+0000 7fe0e6600700 1 -- 172.21.15.22:0/276401313 >> 172.21.15.22:0/276401313 conn(0x7fe0e00fd6a0 msgr2=0x7fe0e00743c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T17:55:55.617 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T17:55:55.617+0000 7fe0e6600700 1 -- 172.21.15.22:0/276401313 shutdown_connections 2024-03-28T17:55:55.617 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T17:55:55.617+0000 7fe0e6600700 1 -- 172.21.15.22:0/276401313 wait complete. 2024-03-28T17:55:56.459 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fd220748-ed2b-11ee-95d0-87774f69a715 -- ceph orch host ls --format=json 2024-03-28T17:55:56.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:55:56 smithi022 ceph-mon[31985]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-28T17:55:56.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:55:56 smithi022 ceph-mon[31985]: Updating smithi079:/etc/ceph/ceph.client.admin.keyring 2024-03-28T17:55:56.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:55:56 smithi022 ceph-mon[31985]: Updating smithi079:/var/lib/ceph/fd220748-ed2b-11ee-95d0-87774f69a715/config/ceph.client.admin.keyring 2024-03-28T17:55:56.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:55:56 smithi022 ceph-mon[31985]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T17:55:56.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:55:56 smithi022 ceph-mon[31985]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T17:55:56.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:55:56 smithi022 ceph-mon[31985]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T17:55:56.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:55:56 smithi022 ceph-mon[31985]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T17:55:56.929 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/fd220748-ed2b-11ee-95d0-87774f69a715/mon.a/config 2024-03-28T17:55:57.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:55:57 smithi022 ceph-mon[31985]: Added host smithi134 2024-03-28T17:55:57.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:55:57 smithi022 ceph-mon[31985]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T17:55:58.662 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:55:58 smithi022 ceph-mon[31985]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-28T17:55:58.663 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:55:58 smithi022 ceph-mon[31985]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T17:55:58.663 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:55:58 smithi022 ceph-mon[31985]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T17:55:58.938 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T17:55:58.938+0000 7f010ee00700 1 -- 172.21.15.22:0/3559583377 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f01081079b0 msgr2=0x7f0108107d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:55:58.939 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T17:55:58.938+0000 7f010ee00700 1 --2- 172.21.15.22:0/3559583377 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f01081079b0 0x7f0108107d90 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f00fc009910 tx=0x7f00fc009c20 comp rx=0 tx=0).stop 2024-03-28T17:55:58.939 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T17:55:58.938+0000 7f010ee00700 1 -- 172.21.15.22:0/3559583377 shutdown_connections 2024-03-28T17:55:58.939 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T17:55:58.938+0000 7f010ee00700 1 --2- 172.21.15.22:0/3559583377 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f01081079b0 0x7f0108107d90 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:55:58.939 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T17:55:58.938+0000 7f010ee00700 1 -- 172.21.15.22:0/3559583377 >> 172.21.15.22:0/3559583377 conn(0x7f01080fd4c0 msgr2=0x7f01080ff8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T17:55:58.939 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T17:55:58.938+0000 7f010ee00700 1 -- 172.21.15.22:0/3559583377 shutdown_connections 2024-03-28T17:55:58.939 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T17:55:58.938+0000 7f010ee00700 1 -- 172.21.15.22:0/3559583377 wait complete. 2024-03-28T17:55:58.940 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T17:55:58.939+0000 7f010ee00700 1 Processor -- start 2024-03-28T17:55:58.940 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T17:55:58.940+0000 7f010ee00700 1 -- start start 2024-03-28T17:55:58.940 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T17:55:58.940+0000 7f010ee00700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f01081079b0 0x7f01081975a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:55:58.940 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T17:55:58.940+0000 7f010ee00700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f010810bfe0 con 0x7f01081079b0 2024-03-28T17:55:58.940 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T17:55:58.940+0000 7f010da00700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f01081079b0 0x7f01081975a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:55:58.940 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T17:55:58.940+0000 7f010da00700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f01081079b0 0x7f01081975a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:39322/0 (socket says 172.21.15.22:39322) 2024-03-28T17:55:58.940 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T17:55:58.940+0000 7f010da00700 1 -- 172.21.15.22:0/2033578512 learned_addr learned my addr 172.21.15.22:0/2033578512 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-28T17:55:58.941 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T17:55:58.941+0000 7f010da00700 1 -- 172.21.15.22:0/2033578512 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0108197ae0 con 0x7f01081079b0 2024-03-28T17:55:58.941 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T17:55:58.941+0000 7f010da00700 1 --2- 172.21.15.22:0/2033578512 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f01081079b0 0x7f01081975a0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f00fc006010 tx=0x7f00fc005810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T17:55:58.941 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T17:55:58.941+0000 7f0106000700 1 -- 172.21.15.22:0/2033578512 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f00fc026070 con 0x7f01081079b0 2024-03-28T17:55:58.942 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T17:55:58.941+0000 7f0106000700 1 -- 172.21.15.22:0/2033578512 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f00fc00bbc0 con 0x7f01081079b0 2024-03-28T17:55:58.942 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T17:55:58.941+0000 7f010ee00700 1 -- 172.21.15.22:0/2033578512 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0108197dd0 con 0x7f01081079b0 2024-03-28T17:55:58.942 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T17:55:58.941+0000 7f0106000700 1 -- 172.21.15.22:0/2033578512 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f00fc0038a0 con 0x7f01081079b0 2024-03-28T17:55:58.942 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T17:55:58.942+0000 7f010ee00700 1 -- 172.21.15.22:0/2033578512 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f010819be70 con 0x7f01081079b0 2024-03-28T17:55:58.946 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T17:55:58.943+0000 7f0106000700 1 -- 172.21.15.22:0/2033578512 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 13) v1 ==== 46162+0+0 (secure 0 0 0) 0x7f00fc029070 con 0x7f01081079b0 2024-03-28T17:55:58.946 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T17:55:58.943+0000 7f010ee00700 1 -- 172.21.15.22:0/2033578512 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f01081030f0 con 0x7f01081079b0 2024-03-28T17:55:58.946 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T17:55:58.943+0000 7f0106000700 1 --2- 172.21.15.22:0/2033578512 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7f00f0039020 0x7f00f003b4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:55:58.946 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T17:55:58.943+0000 7f0106000700 1 -- 172.21.15.22:0/2033578512 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f00fc01e070 con 0x7f01081079b0 2024-03-28T17:55:58.946 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T17:55:58.944+0000 7f010d000700 1 --2- 172.21.15.22:0/2033578512 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7f00f0039020 0x7f00f003b4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:55:58.946 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T17:55:58.944+0000 7f010d000700 1 --2- 172.21.15.22:0/2033578512 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7f00f0039020 0x7f00f003b4e0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f00f4006fd0 tx=0x7f00f4006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T17:55:58.949 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T17:55:58.949+0000 7f0106000700 1 -- 172.21.15.22:0/2033578512 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f00fc013070 con 0x7f01081079b0 2024-03-28T17:55:59.182 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T17:55:59.182+0000 7f010ee00700 1 -- 172.21.15.22:0/2033578512 --> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f01080637c0 con 0x7f00f0039020 2024-03-28T17:55:59.185 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T17:55:59.185+0000 7f0106000700 1 -- 172.21.15.22:0/2033578512 <== mgr.14152 v2:172.21.15.22:6800/4201048392 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+239 (secure 0 0 0) 0x7f01080637c0 con 0x7f00f0039020 2024-03-28T17:55:59.185 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-03-28T17:55:59.185 INFO:teuthology.orchestra.run.smithi022.stdout:[{"addr": "172.21.15.22", "hostname": "smithi022", "labels": [], "status": ""}, {"addr": "172.21.15.79", "hostname": "smithi079", "labels": [], "status": ""}, {"addr": "172.21.15.134", "hostname": "smithi134", "labels": [], "status": ""}] 2024-03-28T17:55:59.187 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T17:55:59.187+0000 7f010ee00700 1 -- 172.21.15.22:0/2033578512 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7f00f0039020 msgr2=0x7f00f003b4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:55:59.187 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T17:55:59.187+0000 7f010ee00700 1 --2- 172.21.15.22:0/2033578512 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7f00f0039020 0x7f00f003b4e0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f00f4006fd0 tx=0x7f00f4006e40 comp rx=0 tx=0).stop 2024-03-28T17:55:59.188 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T17:55:59.187+0000 7f010ee00700 1 -- 172.21.15.22:0/2033578512 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f01081079b0 msgr2=0x7f01081975a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:55:59.188 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T17:55:59.187+0000 7f010ee00700 1 --2- 172.21.15.22:0/2033578512 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f01081079b0 0x7f01081975a0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f00fc006010 tx=0x7f00fc005810 comp rx=0 tx=0).stop 2024-03-28T17:55:59.188 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T17:55:59.188+0000 7f010ee00700 1 -- 172.21.15.22:0/2033578512 shutdown_connections 2024-03-28T17:55:59.188 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T17:55:59.188+0000 7f010ee00700 1 --2- 172.21.15.22:0/2033578512 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7f00f0039020 0x7f00f003b4e0 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:55:59.188 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T17:55:59.188+0000 7f010ee00700 1 --2- 172.21.15.22:0/2033578512 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f01081079b0 0x7f01081975a0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:55:59.188 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T17:55:59.188+0000 7f010ee00700 1 -- 172.21.15.22:0/2033578512 >> 172.21.15.22:0/2033578512 conn(0x7f01080fd4c0 msgr2=0x7f01081095a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T17:55:59.188 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T17:55:59.188+0000 7f010ee00700 1 -- 172.21.15.22:0/2033578512 shutdown_connections 2024-03-28T17:55:59.189 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T17:55:59.188+0000 7f010ee00700 1 -- 172.21.15.22:0/2033578512 wait complete. 2024-03-28T17:55:59.953 INFO:tasks.cephadm:Setting crush tunables to default 2024-03-28T17:55:59.954 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fd220748-ed2b-11ee-95d0-87774f69a715 -- ceph osd crush tunables default 2024-03-28T17:56:00.422 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/fd220748-ed2b-11ee-95d0-87774f69a715/mon.a/config 2024-03-28T17:56:00.778 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:56:00 smithi022 ceph-mon[31985]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-28T17:56:00.778 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:56:00 smithi022 ceph-mon[31985]: from='client.14184 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-28T17:56:00.778 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:56:00 smithi022 ceph-mon[31985]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T17:56:01.722 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:56:01 smithi022 ceph-mon[31985]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-28T17:56:02.535 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T17:56:02.534+0000 7f19bb400700 1 -- 172.21.15.22:0/786099210 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f19b4102d20 msgr2=0x7f19b4103100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:56:02.535 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T17:56:02.534+0000 7f19bb400700 1 --2- 172.21.15.22:0/786099210 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f19b4102d20 0x7f19b4103100 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f19a4009910 tx=0x7f19a4009c20 comp rx=0 tx=0).stop 2024-03-28T17:56:02.535 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T17:56:02.534+0000 7f19bb400700 1 -- 172.21.15.22:0/786099210 shutdown_connections 2024-03-28T17:56:02.535 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T17:56:02.534+0000 7f19bb400700 1 --2- 172.21.15.22:0/786099210 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f19b4102d20 0x7f19b4103100 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:56:02.535 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T17:56:02.534+0000 7f19bb400700 1 -- 172.21.15.22:0/786099210 >> 172.21.15.22:0/786099210 conn(0x7f19b40776d0 msgr2=0x7f19b40fe9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T17:56:02.535 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T17:56:02.534+0000 7f19bb400700 1 -- 172.21.15.22:0/786099210 shutdown_connections 2024-03-28T17:56:02.536 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T17:56:02.534+0000 7f19bb400700 1 -- 172.21.15.22:0/786099210 wait complete. 2024-03-28T17:56:02.536 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T17:56:02.536+0000 7f19bb400700 1 Processor -- start 2024-03-28T17:56:02.536 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T17:56:02.536+0000 7f19bb400700 1 -- start start 2024-03-28T17:56:02.537 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T17:56:02.537+0000 7f19bb400700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f19b4102d20 0x7f19b4100980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:56:02.537 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T17:56:02.537+0000 7f19bb400700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19b410c200 con 0x7f19b4102d20 2024-03-28T17:56:02.537 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T17:56:02.537+0000 7f19b8a00700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f19b4102d20 0x7f19b4100980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:56:02.537 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T17:56:02.537+0000 7f19b8a00700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f19b4102d20 0x7f19b4100980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:39326/0 (socket says 172.21.15.22:39326) 2024-03-28T17:56:02.537 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T17:56:02.537+0000 7f19b8a00700 1 -- 172.21.15.22:0/56783544 learned_addr learned my addr 172.21.15.22:0/56783544 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-28T17:56:02.538 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T17:56:02.538+0000 7f19b8a00700 1 -- 172.21.15.22:0/56783544 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f19b4102870 con 0x7f19b4102d20 2024-03-28T17:56:02.538 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T17:56:02.538+0000 7f19b8a00700 1 --2- 172.21.15.22:0/56783544 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f19b4102d20 0x7f19b4100980 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f19a4006010 tx=0x7f19a4005810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T17:56:02.539 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T17:56:02.538+0000 7f19b2000700 1 -- 172.21.15.22:0/56783544 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f19a4026070 con 0x7f19b4102d20 2024-03-28T17:56:02.539 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T17:56:02.538+0000 7f19b2000700 1 -- 172.21.15.22:0/56783544 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f19a400bc20 con 0x7f19b4102d20 2024-03-28T17:56:02.539 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T17:56:02.538+0000 7f19b2000700 1 -- 172.21.15.22:0/56783544 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f19a4003a50 con 0x7f19b4102d20 2024-03-28T17:56:02.539 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T17:56:02.538+0000 7f19bb400700 1 -- 172.21.15.22:0/56783544 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f19b4100ec0 con 0x7f19b4102d20 2024-03-28T17:56:02.539 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T17:56:02.539+0000 7f19bb400700 1 -- 172.21.15.22:0/56783544 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f19b4101380 con 0x7f19b4102d20 2024-03-28T17:56:02.539 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T17:56:02.540+0000 7f19b2000700 1 -- 172.21.15.22:0/56783544 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 13) v1 ==== 46162+0+0 (secure 0 0 0) 0x7f19a4029070 con 0x7f19b4102d20 2024-03-28T17:56:02.540 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T17:56:02.540+0000 7f19bb400700 1 -- 172.21.15.22:0/56783544 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f19b4069e60 con 0x7f19b4102d20 2024-03-28T17:56:02.540 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T17:56:02.540+0000 7f19b2000700 1 --2- 172.21.15.22:0/56783544 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7f1998039020 0x7f199803b4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:56:02.540 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T17:56:02.540+0000 7f19b2000700 1 -- 172.21.15.22:0/56783544 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f19a401e070 con 0x7f19b4102d20 2024-03-28T17:56:02.540 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T17:56:02.541+0000 7f19abe00700 1 --2- 172.21.15.22:0/56783544 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7f1998039020 0x7f199803b4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:56:02.541 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T17:56:02.541+0000 7f19abe00700 1 --2- 172.21.15.22:0/56783544 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7f1998039020 0x7f199803b4e0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f199c006fd0 tx=0x7f199c006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T17:56:02.546 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T17:56:02.546+0000 7f19b2000700 1 -- 172.21.15.22:0/56783544 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f19a4013070 con 0x7f19b4102d20 2024-03-28T17:56:02.776 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T17:56:02.775+0000 7f19bb400700 1 -- 172.21.15.22:0/56783544 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) v1 -- 0x7f19b4101f50 con 0x7f19b4102d20 2024-03-28T17:56:03.511 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T17:56:03.511+0000 7f19b2000700 1 -- 172.21.15.22:0/56783544 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v4) v1 ==== 124+0+0 (secure 0 0 0) 0x7f19a40142b0 con 0x7f19b4102d20 2024-03-28T17:56:03.515 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T17:56:03.514+0000 7f19bb400700 1 -- 172.21.15.22:0/56783544 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7f1998039020 msgr2=0x7f199803b4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:56:03.515 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T17:56:03.514+0000 7f19bb400700 1 --2- 172.21.15.22:0/56783544 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7f1998039020 0x7f199803b4e0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f199c006fd0 tx=0x7f199c006e40 comp rx=0 tx=0).stop 2024-03-28T17:56:03.515 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T17:56:03.514+0000 7f19bb400700 1 -- 172.21.15.22:0/56783544 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f19b4102d20 msgr2=0x7f19b4100980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:56:03.515 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T17:56:03.514+0000 7f19bb400700 1 --2- 172.21.15.22:0/56783544 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f19b4102d20 0x7f19b4100980 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f19a4006010 tx=0x7f19a4005810 comp rx=0 tx=0).stop 2024-03-28T17:56:03.515 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T17:56:03.514+0000 7f19bb400700 1 -- 172.21.15.22:0/56783544 shutdown_connections 2024-03-28T17:56:03.515 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T17:56:03.515+0000 7f19bb400700 1 --2- 172.21.15.22:0/56783544 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7f1998039020 0x7f199803b4e0 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:56:03.515 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T17:56:03.515+0000 7f19bb400700 1 --2- 172.21.15.22:0/56783544 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f19b4102d20 0x7f19b4100980 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:56:03.515 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T17:56:03.515+0000 7f19bb400700 1 -- 172.21.15.22:0/56783544 >> 172.21.15.22:0/56783544 conn(0x7f19b40776d0 msgr2=0x7f19b4103ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T17:56:03.516 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T17:56:03.515+0000 7f19bb400700 1 -- 172.21.15.22:0/56783544 shutdown_connections 2024-03-28T17:56:03.516 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T17:56:03.515+0000 7f19bb400700 1 -- 172.21.15.22:0/56783544 wait complete. 2024-03-28T17:56:03.517 INFO:teuthology.orchestra.run.smithi022.stderr:adjusted tunables profile to default 2024-03-28T17:56:03.781 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:56:03 smithi022 ceph-mon[31985]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-28T17:56:03.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:56:03 smithi022 ceph-mon[31985]: from='client.? 172.21.15.22:0/56783544' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2024-03-28T17:56:04.275 INFO:tasks.cephadm:Adding mon.a on smithi022 2024-03-28T17:56:04.276 INFO:tasks.cephadm:Adding mon.b on smithi079 2024-03-28T17:56:04.276 INFO:tasks.cephadm:Adding mon.c on smithi134 2024-03-28T17:56:04.276 DEBUG:teuthology.orchestra.run.smithi134:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fd220748-ed2b-11ee-95d0-87774f69a715 -- ceph orch apply mon '3;smithi022:172.21.15.22=a;smithi079:172.21.15.79=b;smithi134:172.21.15.134=c' 2024-03-28T17:56:04.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:56:04 smithi022 ceph-mon[31985]: from='client.? 172.21.15.22:0/56783544' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-03-28T17:56:04.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:56:04 smithi022 ceph-mon[31985]: osdmap e4: 0 total, 0 up, 0 in 2024-03-28T17:56:05.171 INFO:teuthology.orchestra.run.smithi134.stderr:Inferring config /etc/ceph/ceph.conf 2024-03-28T17:56:05.272 INFO:teuthology.orchestra.run.smithi134.stderr:Inferring config /etc/ceph/ceph.conf 2024-03-28T17:56:06.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:56:06 smithi022 ceph-mon[31985]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-28T17:56:06.533 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:56:06 smithi022 ceph-mon[31985]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T17:56:06.533 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:56:06 smithi022 ceph-mon[31985]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T17:56:06.533 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:56:06 smithi022 ceph-mon[31985]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T17:56:06.533 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:56:06 smithi022 ceph-mon[31985]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T17:56:06.533 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:56:06 smithi022 ceph-mon[31985]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi134", "name": "osd_memory_target"}]: dispatch 2024-03-28T17:56:06.533 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:56:06 smithi022 ceph-mon[31985]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T17:56:06.533 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:56:06 smithi022 ceph-mon[31985]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T17:56:06.533 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:56:06 smithi022 ceph-mon[31985]: Updating smithi134:/etc/ceph/ceph.conf 2024-03-28T17:56:07.373 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-28T17:56:07.373+0000 7fc142600700 1 -- 172.21.15.134:0/2297366795 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc13c101bc0 msgr2=0x7fc13c101fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:56:07.373 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-28T17:56:07.373+0000 7fc142600700 1 --2- 172.21.15.134:0/2297366795 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc13c101bc0 0x7fc13c101fe0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fc12c009910 tx=0x7fc12c009c20 comp rx=0 tx=0).stop 2024-03-28T17:56:07.373 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-28T17:56:07.373+0000 7fc142600700 1 -- 172.21.15.134:0/2297366795 shutdown_connections 2024-03-28T17:56:07.374 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-28T17:56:07.373+0000 7fc142600700 1 --2- 172.21.15.134:0/2297366795 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc13c101bc0 0x7fc13c101fe0 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:56:07.374 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-28T17:56:07.373+0000 7fc142600700 1 -- 172.21.15.134:0/2297366795 >> 172.21.15.134:0/2297366795 conn(0x7fc13c0fd160 msgr2=0x7fc13c0ff5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T17:56:07.374 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-28T17:56:07.373+0000 7fc142600700 1 -- 172.21.15.134:0/2297366795 shutdown_connections 2024-03-28T17:56:07.374 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-28T17:56:07.373+0000 7fc142600700 1 -- 172.21.15.134:0/2297366795 wait complete. 2024-03-28T17:56:07.374 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-28T17:56:07.374+0000 7fc142600700 1 Processor -- start 2024-03-28T17:56:07.374 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-28T17:56:07.374+0000 7fc142600700 1 -- start start 2024-03-28T17:56:07.375 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-28T17:56:07.374+0000 7fc142600700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc13c101bc0 0x7fc13c075e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:56:07.375 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-28T17:56:07.374+0000 7fc142600700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc13c107c40 con 0x7fc13c101bc0 2024-03-28T17:56:07.375 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-28T17:56:07.375+0000 7fc13b400700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc13c101bc0 0x7fc13c075e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:56:07.375 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-28T17:56:07.375+0000 7fc13b400700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc13c101bc0 0x7fc13c075e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.134:46178/0 (socket says 172.21.15.134:46178) 2024-03-28T17:56:07.375 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-28T17:56:07.375+0000 7fc13b400700 1 -- 172.21.15.134:0/2653984796 learned_addr learned my addr 172.21.15.134:0/2653984796 (peer_addr_for_me v2:172.21.15.134:0/0) 2024-03-28T17:56:07.376 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-28T17:56:07.376+0000 7fc13b400700 1 -- 172.21.15.134:0/2653984796 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc13c076390 con 0x7fc13c101bc0 2024-03-28T17:56:07.376 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-28T17:56:07.376+0000 7fc13b400700 1 --2- 172.21.15.134:0/2653984796 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc13c101bc0 0x7fc13c075e50 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fc12c004f80 tx=0x7fc12c009f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T17:56:07.377 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-28T17:56:07.377+0000 7fc133e00700 1 -- 172.21.15.134:0/2653984796 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fc12c027070 con 0x7fc13c101bc0 2024-03-28T17:56:07.378 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-28T17:56:07.377+0000 7fc142600700 1 -- 172.21.15.134:0/2653984796 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc13c074580 con 0x7fc13c101bc0 2024-03-28T17:56:07.378 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-28T17:56:07.377+0000 7fc133e00700 1 -- 172.21.15.134:0/2653984796 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7fc12c00be30 con 0x7fc13c101bc0 2024-03-28T17:56:07.378 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-28T17:56:07.377+0000 7fc133e00700 1 -- 172.21.15.134:0/2653984796 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fc12c00f460 con 0x7fc13c101bc0 2024-03-28T17:56:07.378 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-28T17:56:07.377+0000 7fc142600700 1 -- 172.21.15.134:0/2653984796 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc13c074ac0 con 0x7fc13c101bc0 2024-03-28T17:56:07.379 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-28T17:56:07.378+0000 7fc142600700 1 -- 172.21.15.134:0/2653984796 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc13c1083c0 con 0x7fc13c101bc0 2024-03-28T17:56:07.379 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-28T17:56:07.379+0000 7fc133e00700 1 -- 172.21.15.134:0/2653984796 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 13) v1 ==== 46162+0+0 (secure 0 0 0) 0x7fc12c02a070 con 0x7fc13c101bc0 2024-03-28T17:56:07.379 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-28T17:56:07.379+0000 7fc133e00700 1 --2- 172.21.15.134:0/2653984796 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7fc120039020 0x7fc12003b4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:56:07.379 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-28T17:56:07.379+0000 7fc133e00700 1 -- 172.21.15.134:0/2653984796 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7fc12c01e030 con 0x7fc13c101bc0 2024-03-28T17:56:07.380 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-28T17:56:07.380+0000 7fc13aa00700 1 --2- 172.21.15.134:0/2653984796 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7fc120039020 0x7fc12003b4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:56:07.381 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-28T17:56:07.381+0000 7fc13aa00700 1 --2- 172.21.15.134:0/2653984796 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7fc120039020 0x7fc12003b4e0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fc128006fd0 tx=0x7fc128006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T17:56:07.395 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-28T17:56:07.395+0000 7fc133e00700 1 -- 172.21.15.134:0/2653984796 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fc12c013070 con 0x7fc13c101bc0 2024-03-28T17:56:07.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:56:07 smithi022 ceph-mon[31985]: Updating smithi134:/var/lib/ceph/fd220748-ed2b-11ee-95d0-87774f69a715/config/ceph.conf 2024-03-28T17:56:07.633 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-28T17:56:07.633+0000 7fc142600700 1 -- 172.21.15.134:0/2653984796 --> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi022:172.21.15.22=a;smithi079:172.21.15.79=b;smithi134:172.21.15.134=c", "target": ["mon-mgr", ""]}) v1 -- 0x7fc13c1906c0 con 0x7fc120039020 2024-03-28T17:56:07.649 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-28T17:56:07.649+0000 7fc133e00700 1 -- 172.21.15.134:0/2653984796 <== mgr.14152 v2:172.21.15.22:6800/4201048392 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+24 (secure 0 0 0) 0x7fc13c1906c0 con 0x7fc120039020 2024-03-28T17:56:07.649 INFO:teuthology.orchestra.run.smithi134.stdout:Scheduled mon update... 2024-03-28T17:56:07.652 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-28T17:56:07.651+0000 7fc142600700 1 -- 172.21.15.134:0/2653984796 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7fc120039020 msgr2=0x7fc12003b4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:56:07.652 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-28T17:56:07.651+0000 7fc142600700 1 --2- 172.21.15.134:0/2653984796 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7fc120039020 0x7fc12003b4e0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fc128006fd0 tx=0x7fc128006e40 comp rx=0 tx=0).stop 2024-03-28T17:56:07.652 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-28T17:56:07.651+0000 7fc142600700 1 -- 172.21.15.134:0/2653984796 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc13c101bc0 msgr2=0x7fc13c075e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:56:07.652 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-28T17:56:07.651+0000 7fc142600700 1 --2- 172.21.15.134:0/2653984796 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc13c101bc0 0x7fc13c075e50 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fc12c004f80 tx=0x7fc12c009f90 comp rx=0 tx=0).stop 2024-03-28T17:56:07.652 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-28T17:56:07.651+0000 7fc142600700 1 -- 172.21.15.134:0/2653984796 shutdown_connections 2024-03-28T17:56:07.652 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-28T17:56:07.651+0000 7fc142600700 1 --2- 172.21.15.134:0/2653984796 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7fc120039020 0x7fc12003b4e0 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:56:07.652 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-28T17:56:07.651+0000 7fc142600700 1 --2- 172.21.15.134:0/2653984796 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc13c101bc0 0x7fc13c075e50 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:56:07.652 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-28T17:56:07.651+0000 7fc142600700 1 -- 172.21.15.134:0/2653984796 >> 172.21.15.134:0/2653984796 conn(0x7fc13c0fd160 msgr2=0x7fc13c0fdbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T17:56:07.652 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-28T17:56:07.652+0000 7fc142600700 1 -- 172.21.15.134:0/2653984796 shutdown_connections 2024-03-28T17:56:07.652 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-28T17:56:07.652+0000 7fc142600700 1 -- 172.21.15.134:0/2653984796 wait complete. 2024-03-28T17:56:08.394 DEBUG:teuthology.orchestra.run.smithi079:mon.b> sudo journalctl -f -n 0 -u ceph-fd220748-ed2b-11ee-95d0-87774f69a715@mon.b.service 2024-03-28T17:56:08.401 DEBUG:teuthology.orchestra.run.smithi134:mon.c> sudo journalctl -f -n 0 -u ceph-fd220748-ed2b-11ee-95d0-87774f69a715@mon.c.service 2024-03-28T17:56:08.406 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-03-28T17:56:08.406 DEBUG:teuthology.orchestra.run.smithi134:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fd220748-ed2b-11ee-95d0-87774f69a715 -- ceph mon dump -f json 2024-03-28T17:56:08.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:56:08 smithi022 ceph-mon[31985]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-28T17:56:08.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:56:08 smithi022 ceph-mon[31985]: Updating smithi134:/etc/ceph/ceph.client.admin.keyring 2024-03-28T17:56:08.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:56:08 smithi022 ceph-mon[31985]: Updating smithi134:/var/lib/ceph/fd220748-ed2b-11ee-95d0-87774f69a715/config/ceph.client.admin.keyring 2024-03-28T17:56:08.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:56:08 smithi022 ceph-mon[31985]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T17:56:08.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:56:08 smithi022 ceph-mon[31985]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T17:56:08.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:56:08 smithi022 ceph-mon[31985]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T17:56:08.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:56:08 smithi022 ceph-mon[31985]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T17:56:08.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:56:08 smithi022 ceph-mon[31985]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-28T17:56:08.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:56:08 smithi022 ceph-mon[31985]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T17:56:09.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:56:09 smithi022 ceph-mon[31985]: from='client.14188 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi022:172.21.15.22=a;smithi079:172.21.15.79=b;smithi134:172.21.15.134=c", "target": ["mon-mgr", ""]}]: dispatch 2024-03-28T17:56:09.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:56:09 smithi022 ceph-mon[31985]: Saving service mon spec with placement smithi022:172.21.15.22=a;smithi079:172.21.15.79=b;smithi134:172.21.15.134=c;count:3 2024-03-28T17:56:09.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:56:09 smithi022 ceph-mon[31985]: Deploying daemon mon.c on smithi134 2024-03-28T17:56:09.866 INFO:teuthology.orchestra.run.smithi134.stderr:Inferring config /var/lib/ceph/fd220748-ed2b-11ee-95d0-87774f69a715/config/ceph.conf 2024-03-28T17:56:10.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:56:10 smithi022 ceph-mon[31985]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-28T17:56:12.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:56:12 smithi022 ceph-mon[31985]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-28T17:56:13.252 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-28T17:56:13.251+0000 7fcf01e00700 1 -- 172.21.15.134:0/497684784 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fcefc1079b0 msgr2=0x7fcefc107d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:56:13.252 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-28T17:56:13.251+0000 7fcf01e00700 1 --2- 172.21.15.134:0/497684784 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fcefc1079b0 0x7fcefc107d90 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fcef0009910 tx=0x7fcef0009c20 comp rx=0 tx=0).stop 2024-03-28T17:56:13.252 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-28T17:56:13.251+0000 7fcf01e00700 1 -- 172.21.15.134:0/497684784 shutdown_connections 2024-03-28T17:56:13.252 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-28T17:56:13.251+0000 7fcf01e00700 1 --2- 172.21.15.134:0/497684784 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fcefc1079b0 0x7fcefc107d90 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:56:13.252 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-28T17:56:13.251+0000 7fcf01e00700 1 -- 172.21.15.134:0/497684784 >> 172.21.15.134:0/497684784 conn(0x7fcefc0fd4c0 msgr2=0x7fcefc0ff8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T17:56:13.252 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-28T17:56:13.251+0000 7fcf01e00700 1 -- 172.21.15.134:0/497684784 shutdown_connections 2024-03-28T17:56:13.252 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-28T17:56:13.251+0000 7fcf01e00700 1 -- 172.21.15.134:0/497684784 wait complete. 2024-03-28T17:56:13.254 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-28T17:56:13.252+0000 7fcf01e00700 1 Processor -- start 2024-03-28T17:56:13.254 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-28T17:56:13.253+0000 7fcf01e00700 1 -- start start 2024-03-28T17:56:13.254 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-28T17:56:13.253+0000 7fcf01e00700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fcefc1079b0 0x7fcefc1996e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:56:13.254 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-28T17:56:13.253+0000 7fcf01e00700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcefc10bfe0 con 0x7fcefc1079b0 2024-03-28T17:56:13.254 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-28T17:56:13.254+0000 7fcf00a00700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fcefc1079b0 0x7fcefc1996e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:56:13.254 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-28T17:56:13.254+0000 7fcf00a00700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fcefc1079b0 0x7fcefc1996e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.134:47204/0 (socket says 172.21.15.134:47204) 2024-03-28T17:56:13.254 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-28T17:56:13.254+0000 7fcf00a00700 1 -- 172.21.15.134:0/4267932649 learned_addr learned my addr 172.21.15.134:0/4267932649 (peer_addr_for_me v2:172.21.15.134:0/0) 2024-03-28T17:56:13.255 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-28T17:56:13.254+0000 7fcf00a00700 1 -- 172.21.15.134:0/4267932649 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcefc199c20 con 0x7fcefc1079b0 2024-03-28T17:56:13.255 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-28T17:56:13.255+0000 7fcf00a00700 1 --2- 172.21.15.134:0/4267932649 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fcefc1079b0 0x7fcefc1996e0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fcef0006010 tx=0x7fcef0005810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T17:56:13.255 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-28T17:56:13.255+0000 7fcef9600700 1 -- 172.21.15.134:0/4267932649 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fcef0026070 con 0x7fcefc1079b0 2024-03-28T17:56:13.255 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-28T17:56:13.255+0000 7fcf01e00700 1 -- 172.21.15.134:0/4267932649 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcefc199f10 con 0x7fcefc1079b0 2024-03-28T17:56:13.255 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-28T17:56:13.255+0000 7fcef9600700 1 -- 172.21.15.134:0/4267932649 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7fcef000bbc0 con 0x7fcefc1079b0 2024-03-28T17:56:13.256 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-28T17:56:13.255+0000 7fcef9600700 1 -- 172.21.15.134:0/4267932649 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fcef00038a0 con 0x7fcefc1079b0 2024-03-28T17:56:13.256 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-28T17:56:13.255+0000 7fcf01e00700 1 -- 172.21.15.134:0/4267932649 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcefc19dfb0 con 0x7fcefc1079b0 2024-03-28T17:56:13.256 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-28T17:56:13.256+0000 7fcf01e00700 1 -- 172.21.15.134:0/4267932649 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcefc1030f0 con 0x7fcefc1079b0 2024-03-28T17:56:13.257 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-28T17:56:13.257+0000 7fcef9600700 1 -- 172.21.15.134:0/4267932649 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 13) v1 ==== 46162+0+0 (secure 0 0 0) 0x7fcef0029070 con 0x7fcefc1079b0 2024-03-28T17:56:13.257 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-28T17:56:13.257+0000 7fcef9600700 1 --2- 172.21.15.134:0/4267932649 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7fcee4039020 0x7fcee403b4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:56:13.257 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-28T17:56:13.257+0000 7fcef9600700 1 -- 172.21.15.134:0/4267932649 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7fcef001e070 con 0x7fcefc1079b0 2024-03-28T17:56:13.258 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-28T17:56:13.258+0000 7fcefbe00700 1 --2- 172.21.15.134:0/4267932649 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7fcee4039020 0x7fcee403b4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:56:13.259 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-28T17:56:13.259+0000 7fcefbe00700 1 --2- 172.21.15.134:0/4267932649 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7fcee4039020 0x7fcee403b4e0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fcee8006fd0 tx=0x7fcee8006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T17:56:13.273 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-28T17:56:13.273+0000 7fcef9600700 1 -- 172.21.15.134:0/4267932649 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fcef0013070 con 0x7fcefc1079b0 2024-03-28T17:56:13.609 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-28T17:56:13.608+0000 7fcf01e00700 1 -- 172.21.15.134:0/4267932649 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7fcefc19aa40 con 0x7fcefc1079b0 2024-03-28T17:56:13.610 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-28T17:56:13.609+0000 7fcef9600700 1 -- 172.21.15.134:0/4267932649 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+745 (secure 0 0 0) 0x7fcef0034090 con 0x7fcefc1079b0 2024-03-28T17:56:13.610 INFO:teuthology.orchestra.run.smithi134.stdout: 2024-03-28T17:56:13.610 INFO:teuthology.orchestra.run.smithi134.stdout:{"epoch":1,"fsid":"fd220748-ed2b-11ee-95d0-87774f69a715","modified":"2024-03-28T17:53:56.145153Z","created":"2024-03-28T17:53:56.145153Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:3300","nonce":0},{"type":"v1","addr":"172.21.15.22:6789","nonce":0}]},"addr":"172.21.15.22:6789/0","public_addr":"172.21.15.22:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-03-28T17:56:13.612 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-28T17:56:13.612+0000 7fcf01e00700 1 -- 172.21.15.134:0/4267932649 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7fcee4039020 msgr2=0x7fcee403b4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:56:13.612 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-28T17:56:13.612+0000 7fcf01e00700 1 --2- 172.21.15.134:0/4267932649 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7fcee4039020 0x7fcee403b4e0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fcee8006fd0 tx=0x7fcee8006e40 comp rx=0 tx=0).stop 2024-03-28T17:56:13.612 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-28T17:56:13.612+0000 7fcf01e00700 1 -- 172.21.15.134:0/4267932649 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fcefc1079b0 msgr2=0x7fcefc1996e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:56:13.612 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-28T17:56:13.612+0000 7fcf01e00700 1 --2- 172.21.15.134:0/4267932649 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fcefc1079b0 0x7fcefc1996e0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fcef0006010 tx=0x7fcef0005810 comp rx=0 tx=0).stop 2024-03-28T17:56:13.613 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-28T17:56:13.612+0000 7fcf01e00700 1 -- 172.21.15.134:0/4267932649 shutdown_connections 2024-03-28T17:56:13.613 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-28T17:56:13.612+0000 7fcf01e00700 1 --2- 172.21.15.134:0/4267932649 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7fcee4039020 0x7fcee403b4e0 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:56:13.613 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-28T17:56:13.612+0000 7fcf01e00700 1 --2- 172.21.15.134:0/4267932649 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fcefc1079b0 0x7fcefc1996e0 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:56:13.613 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-28T17:56:13.612+0000 7fcf01e00700 1 -- 172.21.15.134:0/4267932649 >> 172.21.15.134:0/4267932649 conn(0x7fcefc0fd4c0 msgr2=0x7fcefc1094b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T17:56:13.613 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-28T17:56:13.613+0000 7fcf01e00700 1 -- 172.21.15.134:0/4267932649 shutdown_connections 2024-03-28T17:56:13.613 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-28T17:56:13.613+0000 7fcf01e00700 1 -- 172.21.15.134:0/4267932649 wait complete. 2024-03-28T17:56:13.614 INFO:teuthology.orchestra.run.smithi134.stderr:dumped monmap epoch 1 2024-03-28T17:56:14.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:56:14 smithi022 ceph-mon[31985]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-28T17:56:14.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:56:14 smithi022 ceph-mon[31985]: from='client.? 172.21.15.134:0/4267932649' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-03-28T17:56:14.897 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:56:14 smithi134 podman[35018]: 2024-03-28 17:56:14.896211185 +0000 UTC m=+0.416033092 container init a88e0c158577a3aa450b13d52b9f2c441f5d21c3e53eadd27a331977dc8f37bb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mon-c, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.build-date=20240319, org.label-schema.vendor=CentOS, ceph=True, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_CLEAN=True, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2) 2024-03-28T17:56:15.176 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:56:14 smithi134 podman[35018]: 2024-03-28 17:56:14.906156996 +0000 UTC m=+0.425978863 container start a88e0c158577a3aa450b13d52b9f2c441f5d21c3e53eadd27a331977dc8f37bb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mon-c, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_BRANCH=HEAD, ceph=True, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.build-date=20240319, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2) 2024-03-28T17:56:15.176 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:56:14 smithi134 ceph-mon[35037]: set uid:gid to 167:167 (ceph:ceph) 2024-03-28T17:56:15.176 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:56:14 smithi134 ceph-mon[35037]: ceph version 19.0.0-2145-g3be9a5c9 (3be9a5c9bc793e11e2800a8c0c696e8b46742033) squid (dev), process ceph-mon, pid 2 2024-03-28T17:56:15.176 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:56:14 smithi134 ceph-mon[35037]: pidfile_write: ignore empty --pid-file 2024-03-28T17:56:15.177 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:56:14 smithi134 ceph-mon[35037]: load: jerasure load: lrc 2024-03-28T17:56:15.177 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:56:14 smithi134 ceph-mon[35037]: rocksdb: RocksDB version: 7.9.2 2024-03-28T17:56:15.177 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:56:14 smithi134 ceph-mon[35037]: rocksdb: Git sha 0 2024-03-28T17:56:15.177 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:56:14 smithi134 ceph-mon[35037]: rocksdb: Compile date 2024-03-22 04:58:55 2024-03-28T17:56:15.177 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:56:14 smithi134 ceph-mon[35037]: rocksdb: DB SUMMARY 2024-03-28T17:56:15.177 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:56:14 smithi134 ceph-mon[35037]: rocksdb: DB Session ID: 2Q9GCQFIVM7BPHOBX3BU 2024-03-28T17:56:15.177 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:56:14 smithi134 ceph-mon[35037]: rocksdb: CURRENT file: CURRENT 2024-03-28T17:56:15.177 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:56:14 smithi134 ceph-mon[35037]: rocksdb: IDENTITY file: IDENTITY 2024-03-28T17:56:15.177 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:56:14 smithi134 ceph-mon[35037]: rocksdb: MANIFEST file: MANIFEST-000005 size: 59 Bytes 2024-03-28T17:56:15.177 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:56:14 smithi134 ceph-mon[35037]: rocksdb: SST files in /var/lib/ceph/mon/ceph-c/store.db dir, Total Num: 0, files: 2024-03-28T17:56:15.177 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:56:14 smithi134 ceph-mon[35037]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-c/store.db: 000004.log size: 511 ; 2024-03-28T17:56:15.177 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:56:14 smithi134 ceph-mon[35037]: rocksdb: Options.error_if_exists: 0 2024-03-28T17:56:15.177 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:56:14 smithi134 ceph-mon[35037]: rocksdb: Options.create_if_missing: 0 2024-03-28T17:56:15.177 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:56:14 smithi134 ceph-mon[35037]: rocksdb: Options.paranoid_checks: 1 2024-03-28T17:56:15.178 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:56:14 smithi134 ceph-mon[35037]: rocksdb: Options.flush_verify_memtable_count: 1 2024-03-28T17:56:15.178 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:56:14 smithi134 ceph-mon[35037]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-03-28T17:56:15.178 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:56:14 smithi134 ceph-mon[35037]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-03-28T17:56:15.178 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:56:14 smithi134 ceph-mon[35037]: rocksdb: Options.env: 0x55cbecc30b20 2024-03-28T17:56:15.178 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:56:14 smithi134 ceph-mon[35037]: rocksdb: Options.fs: PosixFileSystem 2024-03-28T17:56:15.178 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:56:14 smithi134 ceph-mon[35037]: rocksdb: Options.info_log: 0x55cbedab1960 2024-03-28T17:56:15.178 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:56:14 smithi134 ceph-mon[35037]: rocksdb: Options.max_file_opening_threads: 16 2024-03-28T17:56:15.178 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:56:14 smithi134 ceph-mon[35037]: rocksdb: Options.statistics: (nil) 2024-03-28T17:56:15.178 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:56:14 smithi134 ceph-mon[35037]: rocksdb: Options.use_fsync: 0 2024-03-28T17:56:15.178 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:56:14 smithi134 ceph-mon[35037]: rocksdb: Options.max_log_file_size: 0 2024-03-28T17:56:15.178 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:56:14 smithi134 ceph-mon[35037]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-03-28T17:56:15.178 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:56:14 smithi134 ceph-mon[35037]: rocksdb: Options.log_file_time_to_roll: 0 2024-03-28T17:56:15.178 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:56:14 smithi134 ceph-mon[35037]: rocksdb: Options.keep_log_file_num: 1000 2024-03-28T17:56:15.178 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:56:14 smithi134 ceph-mon[35037]: rocksdb: Options.recycle_log_file_num: 0 2024-03-28T17:56:15.179 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:56:14 smithi134 ceph-mon[35037]: rocksdb: Options.allow_fallocate: 1 2024-03-28T17:56:15.179 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:56:14 smithi134 ceph-mon[35037]: rocksdb: Options.allow_mmap_reads: 0 2024-03-28T17:56:15.179 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:56:14 smithi134 ceph-mon[35037]: rocksdb: Options.allow_mmap_writes: 0 2024-03-28T17:56:15.179 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:56:14 smithi134 ceph-mon[35037]: rocksdb: Options.use_direct_reads: 0 2024-03-28T17:56:15.179 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:56:14 smithi134 ceph-mon[35037]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-03-28T17:56:15.179 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:56:14 smithi134 ceph-mon[35037]: rocksdb: Options.create_missing_column_families: 0 2024-03-28T17:56:15.179 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:56:14 smithi134 ceph-mon[35037]: rocksdb: Options.db_log_dir: 2024-03-28T17:56:15.179 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:56:14 smithi134 ceph-mon[35037]: rocksdb: Options.wal_dir: 2024-03-28T17:56:15.179 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:56:14 smithi134 ceph-mon[35037]: rocksdb: Options.table_cache_numshardbits: 6 2024-03-28T17:56:15.179 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:56:14 smithi134 ceph-mon[35037]: rocksdb: Options.WAL_ttl_seconds: 0 2024-03-28T17:56:15.180 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:56:14 smithi134 ceph-mon[35037]: rocksdb: Options.WAL_size_limit_MB: 0 2024-03-28T17:56:15.180 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:56:14 smithi134 ceph-mon[35037]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-03-28T17:56:15.180 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:56:14 smithi134 ceph-mon[35037]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-03-28T17:56:15.180 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:56:14 smithi134 ceph-mon[35037]: rocksdb: Options.is_fd_close_on_exec: 1 2024-03-28T17:56:15.180 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:56:14 smithi134 ceph-mon[35037]: rocksdb: Options.advise_random_on_open: 1 2024-03-28T17:56:15.180 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:56:14 smithi134 ceph-mon[35037]: rocksdb: Options.db_write_buffer_size: 0 2024-03-28T17:56:15.180 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:56:14 smithi134 ceph-mon[35037]: rocksdb: Options.write_buffer_manager: 0x55cbeda841e0 2024-03-28T17:56:15.180 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:56:14 smithi134 ceph-mon[35037]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-03-28T17:56:15.180 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:56:14 smithi134 ceph-mon[35037]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-03-28T17:56:15.180 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:56:14 smithi134 ceph-mon[35037]: rocksdb: Options.use_adaptive_mutex: 0 2024-03-28T17:56:15.180 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:56:14 smithi134 ceph-mon[35037]: rocksdb: Options.rate_limiter: (nil) 2024-03-28T17:56:15.180 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:56:14 smithi134 ceph-mon[35037]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-03-28T17:56:15.180 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:56:14 smithi134 ceph-mon[35037]: rocksdb: Options.wal_recovery_mode: 2 2024-03-28T17:56:15.180 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:56:14 smithi134 ceph-mon[35037]: rocksdb: Options.enable_thread_tracking: 0 2024-03-28T17:56:15.180 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:56:14 smithi134 ceph-mon[35037]: rocksdb: Options.enable_pipelined_write: 0 2024-03-28T17:56:15.180 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:56:14 smithi134 ceph-mon[35037]: rocksdb: Options.unordered_write: 0 2024-03-28T17:56:15.181 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:56:14 smithi134 ceph-mon[35037]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-03-28T17:56:15.181 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:56:14 smithi134 ceph-mon[35037]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-03-28T17:56:15.181 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:56:14 smithi134 ceph-mon[35037]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-03-28T17:56:15.181 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:56:14 smithi134 ceph-mon[35037]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-03-28T17:56:15.181 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:56:14 smithi134 ceph-mon[35037]: rocksdb: Options.row_cache: None 2024-03-28T17:56:15.181 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:56:14 smithi134 ceph-mon[35037]: rocksdb: Options.wal_filter: None 2024-03-28T17:56:15.181 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:56:14 smithi134 ceph-mon[35037]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-03-28T17:56:15.181 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:56:14 smithi134 ceph-mon[35037]: rocksdb: Options.allow_ingest_behind: 0 2024-03-28T17:56:15.181 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:56:14 smithi134 ceph-mon[35037]: rocksdb: Options.two_write_queues: 0 2024-03-28T17:56:15.181 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:56:14 smithi134 ceph-mon[35037]: rocksdb: Options.manual_wal_flush: 0 2024-03-28T17:56:15.181 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:56:14 smithi134 ceph-mon[35037]: rocksdb: Options.wal_compression: 0 2024-03-28T17:56:15.181 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:56:14 smithi134 ceph-mon[35037]: rocksdb: Options.atomic_flush: 0 2024-03-28T17:56:15.181 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:56:14 smithi134 ceph-mon[35037]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-03-28T17:56:15.181 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:56:14 smithi134 ceph-mon[35037]: rocksdb: Options.persist_stats_to_disk: 0 2024-03-28T17:56:15.182 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:56:14 smithi134 ceph-mon[35037]: rocksdb: Options.write_dbid_to_manifest: 0 2024-03-28T17:56:15.182 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:56:14 smithi134 ceph-mon[35037]: rocksdb: Options.log_readahead_size: 0 2024-03-28T17:56:15.182 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:56:14 smithi134 ceph-mon[35037]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-03-28T17:56:15.182 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:56:14 smithi134 ceph-mon[35037]: rocksdb: Options.best_efforts_recovery: 0 2024-03-28T17:56:15.182 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:56:14 smithi134 ceph-mon[35037]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-03-28T17:56:15.182 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:56:14 smithi134 ceph-mon[35037]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-03-28T17:56:15.182 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:56:14 smithi134 ceph-mon[35037]: rocksdb: Options.allow_data_in_errors: 0 2024-03-28T17:56:15.183 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:56:14 smithi134 ceph-mon[35037]: rocksdb: Options.db_host_id: __hostname__ 2024-03-28T17:56:15.183 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:56:14 smithi134 ceph-mon[35037]: rocksdb: Options.enforce_single_del_contracts: true 2024-03-28T17:56:15.183 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:56:14 smithi134 ceph-mon[35037]: rocksdb: Options.max_background_jobs: 2 2024-03-28T17:56:15.183 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:56:14 smithi134 ceph-mon[35037]: rocksdb: Options.max_background_compactions: -1 2024-03-28T17:56:15.183 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:56:14 smithi134 ceph-mon[35037]: rocksdb: Options.max_subcompactions: 1 2024-03-28T17:56:15.183 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:56:14 smithi134 ceph-mon[35037]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-03-28T17:56:15.183 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:56:14 smithi134 ceph-mon[35037]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-03-28T17:56:15.183 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:56:14 smithi134 ceph-mon[35037]: rocksdb: Options.delayed_write_rate : 16777216 2024-03-28T17:56:15.183 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:56:14 smithi134 ceph-mon[35037]: rocksdb: Options.max_total_wal_size: 0 2024-03-28T17:56:15.183 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:56:14 smithi134 ceph-mon[35037]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-03-28T17:56:15.183 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:56:14 smithi134 ceph-mon[35037]: rocksdb: Options.stats_dump_period_sec: 600 2024-03-28T17:56:15.183 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:56:14 smithi134 ceph-mon[35037]: rocksdb: Options.stats_persist_period_sec: 600 2024-03-28T17:56:15.183 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:56:14 smithi134 ceph-mon[35037]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-03-28T17:56:15.183 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:56:14 smithi134 ceph-mon[35037]: rocksdb: Options.max_open_files: -1 2024-03-28T17:56:15.183 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:56:14 smithi134 ceph-mon[35037]: rocksdb: Options.bytes_per_sync: 0 2024-03-28T17:56:15.184 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:56:14 smithi134 ceph-mon[35037]: rocksdb: Options.wal_bytes_per_sync: 0 2024-03-28T17:56:15.184 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:56:14 smithi134 ceph-mon[35037]: rocksdb: Options.strict_bytes_per_sync: 0 2024-03-28T17:56:15.184 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:56:14 smithi134 ceph-mon[35037]: rocksdb: Options.compaction_readahead_size: 0 2024-03-28T17:56:15.184 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:56:14 smithi134 ceph-mon[35037]: rocksdb: Options.max_background_flushes: -1 2024-03-28T17:56:15.184 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:56:14 smithi134 ceph-mon[35037]: rocksdb: Compression algorithms supported: 2024-03-28T17:56:15.184 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:56:14 smithi134 ceph-mon[35037]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-03-28T17:56:15.184 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:56:14 smithi134 ceph-mon[35037]: rocksdb: kZSTD supported: 0 2024-03-28T17:56:15.184 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:56:14 smithi134 ceph-mon[35037]: rocksdb: kXpressCompression supported: 0 2024-03-28T17:56:15.184 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:56:14 smithi134 ceph-mon[35037]: rocksdb: kLZ4HCCompression supported: 1 2024-03-28T17:56:15.184 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:56:14 smithi134 ceph-mon[35037]: rocksdb: kZlibCompression supported: 1 2024-03-28T17:56:15.185 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:56:14 smithi134 ceph-mon[35037]: rocksdb: kSnappyCompression supported: 1 2024-03-28T17:56:15.185 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:56:14 smithi134 ceph-mon[35037]: rocksdb: kLZ4Compression supported: 1 2024-03-28T17:56:15.185 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:56:14 smithi134 ceph-mon[35037]: rocksdb: kBZip2Compression supported: 0 2024-03-28T17:56:15.185 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:56:14 smithi134 ceph-mon[35037]: rocksdb: Fast CRC32 supported: Supported on x86 2024-03-28T17:56:15.185 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:56:14 smithi134 ceph-mon[35037]: rocksdb: DMutex implementation: pthread_mutex_t 2024-03-28T17:56:15.185 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:56:14 smithi134 ceph-mon[35037]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-c/store.db/MANIFEST-000005 2024-03-28T17:56:15.185 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:56:14 smithi134 ceph-mon[35037]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-03-28T17:56:15.185 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:56:14 smithi134 ceph-mon[35037]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-03-28T17:56:15.185 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:56:14 smithi134 ceph-mon[35037]: rocksdb: Options.merge_operator: 2024-03-28T17:56:15.185 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:56:14 smithi134 ceph-mon[35037]: rocksdb: Options.compaction_filter: None 2024-03-28T17:56:15.185 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:56:14 smithi134 ceph-mon[35037]: rocksdb: Options.compaction_filter_factory: None 2024-03-28T17:56:15.185 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:56:14 smithi134 ceph-mon[35037]: rocksdb: Options.sst_partitioner_factory: None 2024-03-28T17:56:15.185 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:56:14 smithi134 ceph-mon[35037]: rocksdb: Options.memtable_factory: SkipListFactory 2024-03-28T17:56:15.185 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:56:14 smithi134 ceph-mon[35037]: rocksdb: Options.table_factory: BlockBasedTable 2024-03-28T17:56:15.185 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:56:14 smithi134 ceph-mon[35037]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x55cbedab1a60) 2024-03-28T17:56:15.186 INFO:journalctl@ceph.mon.c.smithi134.stdout: cache_index_and_filter_blocks: 1 2024-03-28T17:56:15.186 INFO:journalctl@ceph.mon.c.smithi134.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-03-28T17:56:15.186 INFO:journalctl@ceph.mon.c.smithi134.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-03-28T17:56:15.186 INFO:journalctl@ceph.mon.c.smithi134.stdout: pin_top_level_index_and_filter: 1 2024-03-28T17:56:15.186 INFO:journalctl@ceph.mon.c.smithi134.stdout: index_type: 0 2024-03-28T17:56:15.186 INFO:journalctl@ceph.mon.c.smithi134.stdout: data_block_index_type: 0 2024-03-28T17:56:15.186 INFO:journalctl@ceph.mon.c.smithi134.stdout: index_shortening: 1 2024-03-28T17:56:15.186 INFO:journalctl@ceph.mon.c.smithi134.stdout: data_block_hash_table_util_ratio: 0.750000 2024-03-28T17:56:15.186 INFO:journalctl@ceph.mon.c.smithi134.stdout: checksum: 4 2024-03-28T17:56:15.186 INFO:journalctl@ceph.mon.c.smithi134.stdout: no_block_cache: 0 2024-03-28T17:56:15.186 INFO:journalctl@ceph.mon.c.smithi134.stdout: block_cache: 0x55cbedab7090 2024-03-28T17:56:15.186 INFO:journalctl@ceph.mon.c.smithi134.stdout: block_cache_name: BinnedLRUCache 2024-03-28T17:56:15.186 INFO:journalctl@ceph.mon.c.smithi134.stdout: block_cache_options: 2024-03-28T17:56:15.186 INFO:journalctl@ceph.mon.c.smithi134.stdout: capacity : 536870912 2024-03-28T17:56:15.186 INFO:journalctl@ceph.mon.c.smithi134.stdout: num_shard_bits : 4 2024-03-28T17:56:15.187 INFO:journalctl@ceph.mon.c.smithi134.stdout: strict_capacity_limit : 0 2024-03-28T17:56:15.187 INFO:journalctl@ceph.mon.c.smithi134.stdout: high_pri_pool_ratio: 0.000 2024-03-28T17:56:15.187 INFO:journalctl@ceph.mon.c.smithi134.stdout: block_cache_compressed: (nil) 2024-03-28T17:56:15.187 INFO:journalctl@ceph.mon.c.smithi134.stdout: persistent_cache: (nil) 2024-03-28T17:56:15.187 INFO:journalctl@ceph.mon.c.smithi134.stdout: block_size: 4096 2024-03-28T17:56:15.187 INFO:journalctl@ceph.mon.c.smithi134.stdout: block_size_deviation: 10 2024-03-28T17:56:15.187 INFO:journalctl@ceph.mon.c.smithi134.stdout: block_restart_interval: 16 2024-03-28T17:56:15.187 INFO:journalctl@ceph.mon.c.smithi134.stdout: index_block_restart_interval: 1 2024-03-28T17:56:15.187 INFO:journalctl@ceph.mon.c.smithi134.stdout: metadata_block_size: 4096 2024-03-28T17:56:15.187 INFO:journalctl@ceph.mon.c.smithi134.stdout: partition_filters: 0 2024-03-28T17:56:15.187 INFO:journalctl@ceph.mon.c.smithi134.stdout: use_delta_encoding: 1 2024-03-28T17:56:15.187 INFO:journalctl@ceph.mon.c.smithi134.stdout: filter_policy: bloomfilter 2024-03-28T17:56:15.187 INFO:journalctl@ceph.mon.c.smithi134.stdout: whole_key_filtering: 1 2024-03-28T17:56:15.187 INFO:journalctl@ceph.mon.c.smithi134.stdout: verify_compression: 0 2024-03-28T17:56:15.187 INFO:journalctl@ceph.mon.c.smithi134.stdout: read_amp_bytes_per_bit: 0 2024-03-28T17:56:15.187 INFO:journalctl@ceph.mon.c.smithi134.stdout: format_version: 5 2024-03-28T17:56:15.188 INFO:journalctl@ceph.mon.c.smithi134.stdout: enable_index_compression: 1 2024-03-28T17:56:15.188 INFO:journalctl@ceph.mon.c.smithi134.stdout: block_align: 0 2024-03-28T17:56:15.188 INFO:journalctl@ceph.mon.c.smithi134.stdout: max_auto_readahead_size: 262144 2024-03-28T17:56:15.188 INFO:journalctl@ceph.mon.c.smithi134.stdout: prepopulate_block_cache: 0 2024-03-28T17:56:15.188 INFO:journalctl@ceph.mon.c.smithi134.stdout: initial_auto_readahead_size: 8192 2024-03-28T17:56:15.188 INFO:journalctl@ceph.mon.c.smithi134.stdout: num_file_reads_for_auto_readahead: 2 2024-03-28T17:56:15.188 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:56:14 smithi134 ceph-mon[35037]: rocksdb: Options.write_buffer_size: 33554432 2024-03-28T17:56:15.188 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:56:14 smithi134 ceph-mon[35037]: rocksdb: Options.max_write_buffer_number: 2 2024-03-28T17:56:15.188 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:56:14 smithi134 ceph-mon[35037]: rocksdb: Options.compression: NoCompression 2024-03-28T17:56:15.188 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:56:14 smithi134 ceph-mon[35037]: rocksdb: Options.bottommost_compression: Disabled 2024-03-28T17:56:15.188 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:56:14 smithi134 ceph-mon[35037]: rocksdb: Options.prefix_extractor: nullptr 2024-03-28T17:56:15.188 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:56:14 smithi134 ceph-mon[35037]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-03-28T17:56:15.188 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:56:14 smithi134 ceph-mon[35037]: rocksdb: Options.num_levels: 7 2024-03-28T17:56:15.188 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:56:14 smithi134 ceph-mon[35037]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-03-28T17:56:15.189 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:56:14 smithi134 ceph-mon[35037]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-03-28T17:56:15.189 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:56:14 smithi134 ceph-mon[35037]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-03-28T17:56:15.189 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:56:14 smithi134 ceph-mon[35037]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-03-28T17:56:15.190 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:56:14 smithi134 ceph-mon[35037]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-03-28T17:56:15.190 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:56:14 smithi134 ceph-mon[35037]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-03-28T17:56:15.190 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:56:14 smithi134 ceph-mon[35037]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-03-28T17:56:15.190 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:56:14 smithi134 ceph-mon[35037]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-03-28T17:56:15.190 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:56:14 smithi134 ceph-mon[35037]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-03-28T17:56:15.190 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:56:14 smithi134 ceph-mon[35037]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-03-28T17:56:15.190 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:56:14 smithi134 ceph-mon[35037]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-03-28T17:56:15.190 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:56:14 smithi134 ceph-mon[35037]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-03-28T17:56:15.190 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:56:14 smithi134 ceph-mon[35037]: rocksdb: Options.compression_opts.window_bits: -14 2024-03-28T17:56:15.190 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:56:14 smithi134 ceph-mon[35037]: rocksdb: Options.compression_opts.level: 32767 2024-03-28T17:56:15.190 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:56:14 smithi134 ceph-mon[35037]: rocksdb: Options.compression_opts.strategy: 0 2024-03-28T17:56:15.190 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:56:14 smithi134 ceph-mon[35037]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-03-28T17:56:15.190 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:56:14 smithi134 ceph-mon[35037]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-03-28T17:56:15.191 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:56:14 smithi134 ceph-mon[35037]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-03-28T17:56:15.191 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:56:14 smithi134 ceph-mon[35037]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-03-28T17:56:15.191 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:56:14 smithi134 ceph-mon[35037]: rocksdb: Options.compression_opts.enabled: false 2024-03-28T17:56:15.191 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:56:14 smithi134 ceph-mon[35037]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-03-28T17:56:15.191 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:56:14 smithi134 ceph-mon[35037]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-03-28T17:56:15.191 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:56:14 smithi134 ceph-mon[35037]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-03-28T17:56:15.191 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:56:14 smithi134 ceph-mon[35037]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-03-28T17:56:15.191 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:56:14 smithi134 ceph-mon[35037]: rocksdb: Options.target_file_size_base: 67108864 2024-03-28T17:56:15.191 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:56:14 smithi134 ceph-mon[35037]: rocksdb: Options.target_file_size_multiplier: 1 2024-03-28T17:56:15.191 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:56:14 smithi134 ceph-mon[35037]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-03-28T17:56:15.191 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:56:14 smithi134 ceph-mon[35037]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-03-28T17:56:15.191 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:56:14 smithi134 ceph-mon[35037]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-03-28T17:56:15.191 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:56:14 smithi134 ceph-mon[35037]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-03-28T17:56:15.192 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:56:14 smithi134 ceph-mon[35037]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-03-28T17:56:15.192 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:56:14 smithi134 ceph-mon[35037]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-03-28T17:56:15.192 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:56:14 smithi134 ceph-mon[35037]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-03-28T17:56:15.192 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:56:14 smithi134 ceph-mon[35037]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-03-28T17:56:15.192 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:56:14 smithi134 ceph-mon[35037]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-03-28T17:56:15.192 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:56:14 smithi134 ceph-mon[35037]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-03-28T17:56:15.192 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:56:14 smithi134 ceph-mon[35037]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-03-28T17:56:15.192 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:56:14 smithi134 ceph-mon[35037]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-03-28T17:56:15.192 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:56:14 smithi134 ceph-mon[35037]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-03-28T17:56:15.192 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:56:14 smithi134 ceph-mon[35037]: rocksdb: Options.arena_block_size: 1048576 2024-03-28T17:56:15.192 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:56:14 smithi134 ceph-mon[35037]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-03-28T17:56:15.192 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:56:14 smithi134 ceph-mon[35037]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-03-28T17:56:15.192 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:56:14 smithi134 ceph-mon[35037]: rocksdb: Options.disable_auto_compactions: 0 2024-03-28T17:56:15.192 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:56:14 smithi134 ceph-mon[35037]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-03-28T17:56:15.192 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:56:14 smithi134 ceph-mon[35037]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-03-28T17:56:15.192 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:56:14 smithi134 ceph-mon[35037]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-03-28T17:56:15.193 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:56:14 smithi134 ceph-mon[35037]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-03-28T17:56:15.193 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:56:14 smithi134 ceph-mon[35037]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-03-28T17:56:15.193 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:56:14 smithi134 ceph-mon[35037]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-03-28T17:56:15.193 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:56:14 smithi134 ceph-mon[35037]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-03-28T17:56:15.193 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:56:14 smithi134 ceph-mon[35037]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-03-28T17:56:15.193 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:56:14 smithi134 ceph-mon[35037]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-03-28T17:56:15.193 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:56:14 smithi134 ceph-mon[35037]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-03-28T17:56:15.193 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:56:14 smithi134 ceph-mon[35037]: rocksdb: Options.table_properties_collectors: 2024-03-28T17:56:15.193 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:56:14 smithi134 ceph-mon[35037]: rocksdb: Options.inplace_update_support: 0 2024-03-28T17:56:15.193 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:56:14 smithi134 ceph-mon[35037]: rocksdb: Options.inplace_update_num_locks: 10000 2024-03-28T17:56:15.193 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:56:14 smithi134 ceph-mon[35037]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-03-28T17:56:15.193 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:56:14 smithi134 ceph-mon[35037]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-03-28T17:56:15.193 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:56:14 smithi134 ceph-mon[35037]: rocksdb: Options.memtable_huge_page_size: 0 2024-03-28T17:56:15.193 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:56:14 smithi134 ceph-mon[35037]: rocksdb: Options.bloom_locality: 0 2024-03-28T17:56:15.193 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:56:14 smithi134 ceph-mon[35037]: rocksdb: Options.max_successive_merges: 0 2024-03-28T17:56:15.194 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:56:14 smithi134 ceph-mon[35037]: rocksdb: Options.optimize_filters_for_hits: 0 2024-03-28T17:56:15.194 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:56:14 smithi134 ceph-mon[35037]: rocksdb: Options.paranoid_file_checks: 0 2024-03-28T17:56:15.194 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:56:14 smithi134 ceph-mon[35037]: rocksdb: Options.force_consistency_checks: 1 2024-03-28T17:56:15.194 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:56:14 smithi134 ceph-mon[35037]: rocksdb: Options.report_bg_io_stats: 0 2024-03-28T17:56:15.194 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:56:14 smithi134 ceph-mon[35037]: rocksdb: Options.ttl: 2592000 2024-03-28T17:56:15.194 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:56:14 smithi134 ceph-mon[35037]: rocksdb: Options.periodic_compaction_seconds: 0 2024-03-28T17:56:15.194 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:56:14 smithi134 ceph-mon[35037]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-03-28T17:56:15.194 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:56:14 smithi134 ceph-mon[35037]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-03-28T17:56:15.194 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:56:14 smithi134 ceph-mon[35037]: rocksdb: Options.enable_blob_files: false 2024-03-28T17:56:15.194 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:56:14 smithi134 ceph-mon[35037]: rocksdb: Options.min_blob_size: 0 2024-03-28T17:56:15.194 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:56:14 smithi134 ceph-mon[35037]: rocksdb: Options.blob_file_size: 268435456 2024-03-28T17:56:15.194 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:56:14 smithi134 ceph-mon[35037]: rocksdb: Options.blob_compression_type: NoCompression 2024-03-28T17:56:15.194 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:56:14 smithi134 ceph-mon[35037]: rocksdb: Options.enable_blob_garbage_collection: false 2024-03-28T17:56:15.194 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:56:14 smithi134 ceph-mon[35037]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-03-28T17:56:15.194 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:56:14 smithi134 ceph-mon[35037]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-03-28T17:56:15.194 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:56:14 smithi134 ceph-mon[35037]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-03-28T17:56:15.195 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:56:14 smithi134 ceph-mon[35037]: rocksdb: Options.blob_file_starting_level: 0 2024-03-28T17:56:15.195 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:56:14 smithi134 ceph-mon[35037]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-03-28T17:56:15.195 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:56:14 smithi134 ceph-mon[35037]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-c/store.db/MANIFEST-000005 succeeded,manifest_file_number is 5, next_file_number is 7, last_sequence is 0, log_number is 0,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 0 2024-03-28T17:56:15.195 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:56:14 smithi134 ceph-mon[35037]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 0 2024-03-28T17:56:15.195 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:56:14 smithi134 ceph-mon[35037]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: e7f894b9-fd6d-433b-9c2f-fe3c14ef5704 2024-03-28T17:56:15.195 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:56:14 smithi134 ceph-mon[35037]: rocksdb: EVENT_LOG_v1 {"time_micros": 1711648574966539, "job": 1, "event": "recovery_started", "wal_files": [4]} 2024-03-28T17:56:15.195 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:56:14 smithi134 ceph-mon[35037]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #4 mode 2 2024-03-28T17:56:15.195 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:56:14 smithi134 ceph-mon[35037]: rocksdb: EVENT_LOG_v1 {"time_micros": 1711648574967819, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 8, "file_size": 1622, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 1, "largest_seqno": 5, "table_properties": {"data_size": 523, "index_size": 31, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 69, "raw_key_size": 115, "raw_average_key_size": 23, "raw_value_size": 401, "raw_average_value_size": 80, "num_data_blocks": 1, "num_entries": 5, "num_filter_entries": 5, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1711648574, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "e7f894b9-fd6d-433b-9c2f-fe3c14ef5704", "db_session_id": "2Q9GCQFIVM7BPHOBX3BU", "orig_file_number": 8, "seqno_to_time_mapping": "N/A"}} 2024-03-28T17:56:15.195 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:56:14 smithi134 ceph-mon[35037]: rocksdb: EVENT_LOG_v1 {"time_micros": 1711648574968153, "job": 1, "event": "recovery_finished"} 2024-03-28T17:56:15.195 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:56:14 smithi134 ceph-mon[35037]: rocksdb: [db/version_set.cc:5047] Creating manifest 10 2024-03-28T17:56:15.195 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:56:14 smithi134 ceph-mon[35037]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-c/store.db/000004.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-03-28T17:56:15.195 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:56:14 smithi134 ceph-mon[35037]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x55cbedbaa000 2024-03-28T17:56:15.195 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:56:14 smithi134 ceph-mon[35037]: rocksdb: DB pointer 0x55cbedb94000 2024-03-28T17:56:15.195 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:56:14 smithi134 ceph-mon[35037]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-03-28T17:56:15.196 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:56:14 smithi134 ceph-mon[35037]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-03-28T17:56:15.196 INFO:journalctl@ceph.mon.c.smithi134.stdout: ** DB Stats ** 2024-03-28T17:56:15.196 INFO:journalctl@ceph.mon.c.smithi134.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-03-28T17:56:15.196 INFO:journalctl@ceph.mon.c.smithi134.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-03-28T17:56:15.196 INFO:journalctl@ceph.mon.c.smithi134.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-03-28T17:56:15.196 INFO:journalctl@ceph.mon.c.smithi134.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-03-28T17:56:15.196 INFO:journalctl@ceph.mon.c.smithi134.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-03-28T17:56:15.196 INFO:journalctl@ceph.mon.c.smithi134.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-03-28T17:56:15.196 INFO:journalctl@ceph.mon.c.smithi134.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-03-28T17:56:15.196 INFO:journalctl@ceph.mon.c.smithi134.stdout: 2024-03-28T17:56:15.196 INFO:journalctl@ceph.mon.c.smithi134.stdout: ** Compaction Stats [default] ** 2024-03-28T17:56:15.196 INFO:journalctl@ceph.mon.c.smithi134.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-03-28T17:56:15.196 INFO:journalctl@ceph.mon.c.smithi134.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-03-28T17:56:15.197 INFO:journalctl@ceph.mon.c.smithi134.stdout: L0 1/0 1.58 KB 0.2 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 1.3 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-03-28T17:56:15.197 INFO:journalctl@ceph.mon.c.smithi134.stdout: Sum 1/0 1.58 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 1.3 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-03-28T17:56:15.197 INFO:journalctl@ceph.mon.c.smithi134.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 1.3 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-03-28T17:56:15.197 INFO:journalctl@ceph.mon.c.smithi134.stdout: 2024-03-28T17:56:15.197 INFO:journalctl@ceph.mon.c.smithi134.stdout: ** Compaction Stats [default] ** 2024-03-28T17:56:15.197 INFO:journalctl@ceph.mon.c.smithi134.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-03-28T17:56:15.197 INFO:journalctl@ceph.mon.c.smithi134.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-03-28T17:56:15.197 INFO:journalctl@ceph.mon.c.smithi134.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.3 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-03-28T17:56:15.197 INFO:journalctl@ceph.mon.c.smithi134.stdout: 2024-03-28T17:56:15.197 INFO:journalctl@ceph.mon.c.smithi134.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-03-28T17:56:15.197 INFO:journalctl@ceph.mon.c.smithi134.stdout: 2024-03-28T17:56:15.197 INFO:journalctl@ceph.mon.c.smithi134.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-03-28T17:56:15.197 INFO:journalctl@ceph.mon.c.smithi134.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-03-28T17:56:15.197 INFO:journalctl@ceph.mon.c.smithi134.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-03-28T17:56:15.197 INFO:journalctl@ceph.mon.c.smithi134.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-03-28T17:56:15.197 INFO:journalctl@ceph.mon.c.smithi134.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-03-28T17:56:15.198 INFO:journalctl@ceph.mon.c.smithi134.stdout: AddFile(Keys): cumulative 0, interval 0 2024-03-28T17:56:15.198 INFO:journalctl@ceph.mon.c.smithi134.stdout: Cumulative compaction: 0.00 GB write, 0.18 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-03-28T17:56:15.198 INFO:journalctl@ceph.mon.c.smithi134.stdout: Interval compaction: 0.00 GB write, 0.18 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-03-28T17:56:15.198 INFO:journalctl@ceph.mon.c.smithi134.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-03-28T17:56:15.198 INFO:journalctl@ceph.mon.c.smithi134.stdout: Block cache BinnedLRUCache@0x55cbedab7090#2 capacity: 512.00 MB usage: 0.22 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 1.1e-05 secs_since: 0 2024-03-28T17:56:15.198 INFO:journalctl@ceph.mon.c.smithi134.stdout: Block cache entry stats(count,size,portion): FilterBlock(1,0.11 KB,2.08616e-05%) IndexBlock(1,0.11 KB,2.08616e-05%) Misc(1,0.00 KB,0%) 2024-03-28T17:56:15.198 INFO:journalctl@ceph.mon.c.smithi134.stdout: 2024-03-28T17:56:15.198 INFO:journalctl@ceph.mon.c.smithi134.stdout: ** File Read Latency Histogram By Level [default] ** 2024-03-28T17:56:15.198 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:56:14 smithi134 ceph-mon[35037]: mon.c does not exist in monmap, will attempt to join an existing cluster 2024-03-28T17:56:15.198 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:56:14 smithi134 ceph-mon[35037]: using public_addr v2:172.21.15.134:0/0 -> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] 2024-03-28T17:56:15.198 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:56:14 smithi134 ceph-mon[35037]: starting mon.c rank -1 at public addrs [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] at bind addrs [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon_data /var/lib/ceph/mon/ceph-c fsid fd220748-ed2b-11ee-95d0-87774f69a715 2024-03-28T17:56:15.198 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:56:14 smithi134 ceph-mon[35037]: mon.c@-1(???) e0 preinit fsid fd220748-ed2b-11ee-95d0-87774f69a715 2024-03-28T17:56:15.198 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:56:14 smithi134 bash[35018]: a88e0c158577a3aa450b13d52b9f2c441f5d21c3e53eadd27a331977dc8f37bb 2024-03-28T17:56:15.198 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:56:14 smithi134 systemd[1]: Started Ceph mon.c for fd220748-ed2b-11ee-95d0-87774f69a715. 2024-03-28T17:56:15.198 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:56:15 smithi134 ceph-mon[35037]: mon.c@-1(synchronizing).mds e1 new map 2024-03-28T17:56:15.199 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:56:15 smithi134 ceph-mon[35037]: mon.c@-1(synchronizing).mds e1 print_map 2024-03-28T17:56:15.199 INFO:journalctl@ceph.mon.c.smithi134.stdout: e1 2024-03-28T17:56:15.199 INFO:journalctl@ceph.mon.c.smithi134.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-03-28T17:56:15.199 INFO:journalctl@ceph.mon.c.smithi134.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments} 2024-03-28T17:56:15.199 INFO:journalctl@ceph.mon.c.smithi134.stdout: legacy client fscid: -1 2024-03-28T17:56:15.199 INFO:journalctl@ceph.mon.c.smithi134.stdout: 2024-03-28T17:56:15.199 INFO:journalctl@ceph.mon.c.smithi134.stdout: No filesystems configured 2024-03-28T17:56:15.199 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:56:15 smithi134 ceph-mon[35037]: mon.c@-1(synchronizing).osd e0 _set_cache_ratios kv ratio 0.25 inc ratio 0.375 full ratio 0.375 2024-03-28T17:56:15.199 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:56:15 smithi134 ceph-mon[35037]: mon.c@-1(synchronizing).osd e0 register_cache_with_pcm pcm target: 2147483648 pcm max: 1020054732 pcm min: 134217728 inc_osd_cache size: 1 2024-03-28T17:56:15.199 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:56:15 smithi134 ceph-mon[35037]: mon.c@-1(synchronizing).osd e1 e1: 0 total, 0 up, 0 in 2024-03-28T17:56:15.199 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:56:15 smithi134 ceph-mon[35037]: mon.c@-1(synchronizing).osd e2 e2: 0 total, 0 up, 0 in 2024-03-28T17:56:15.199 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:56:15 smithi134 ceph-mon[35037]: mon.c@-1(synchronizing).osd e3 e3: 0 total, 0 up, 0 in 2024-03-28T17:56:15.199 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:56:15 smithi134 ceph-mon[35037]: mon.c@-1(synchronizing).osd e4 e4: 0 total, 0 up, 0 in 2024-03-28T17:56:15.199 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:56:15 smithi134 ceph-mon[35037]: mon.c@-1(synchronizing).osd e4 crush map has features 3314932999778484224, adjusting msgr requires 2024-03-28T17:56:15.199 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:56:15 smithi134 ceph-mon[35037]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-03-28T17:56:15.200 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:56:15 smithi134 ceph-mon[35037]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-03-28T17:56:15.200 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:56:15 smithi134 ceph-mon[35037]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-03-28T17:56:15.200 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:56:15 smithi134 ceph-mon[35037]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-28T17:56:15.200 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:56:15 smithi134 ceph-mon[35037]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T17:56:15.200 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:56:15 smithi134 ceph-mon[35037]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T17:56:15.200 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:56:15 smithi134 ceph-mon[35037]: Added host smithi079 2024-03-28T17:56:15.200 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:56:15 smithi134 ceph-mon[35037]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T17:56:15.200 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:56:15 smithi134 ceph-mon[35037]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-28T17:56:15.200 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:56:15 smithi134 ceph-mon[35037]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T17:56:15.200 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:56:15 smithi134 ceph-mon[35037]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-28T17:56:15.200 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:56:15 smithi134 ceph-mon[35037]: from='client.14180 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-28T17:56:15.200 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:56:15 smithi134 ceph-mon[35037]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T17:56:15.200 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:56:15 smithi134 ceph-mon[35037]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-28T17:56:15.200 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:56:15 smithi134 ceph-mon[35037]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-28T17:56:15.200 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:56:15 smithi134 ceph-mon[35037]: from='client.14182 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi134", "target": ["mon-mgr", ""]}]: dispatch 2024-03-28T17:56:15.200 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:56:15 smithi134 ceph-mon[35037]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T17:56:15.201 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:56:15 smithi134 ceph-mon[35037]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T17:56:15.201 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:56:15 smithi134 ceph-mon[35037]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T17:56:15.201 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:56:15 smithi134 ceph-mon[35037]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T17:56:15.201 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:56:15 smithi134 ceph-mon[35037]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi079", "name": "osd_memory_target"}]: dispatch 2024-03-28T17:56:15.201 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:56:15 smithi134 ceph-mon[35037]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T17:56:15.201 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:56:15 smithi134 ceph-mon[35037]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T17:56:15.201 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:56:15 smithi134 ceph-mon[35037]: Updating smithi079:/etc/ceph/ceph.conf 2024-03-28T17:56:15.201 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:56:15 smithi134 ceph-mon[35037]: Deploying cephadm binary to smithi134 2024-03-28T17:56:15.201 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:56:15 smithi134 ceph-mon[35037]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-28T17:56:15.201 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:56:15 smithi134 ceph-mon[35037]: Updating smithi079:/var/lib/ceph/fd220748-ed2b-11ee-95d0-87774f69a715/config/ceph.conf 2024-03-28T17:56:15.201 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:56:15 smithi134 ceph-mon[35037]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-28T17:56:15.201 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:56:15 smithi134 ceph-mon[35037]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-28T17:56:15.201 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:56:15 smithi134 ceph-mon[35037]: Updating smithi079:/etc/ceph/ceph.client.admin.keyring 2024-03-28T17:56:15.201 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:56:15 smithi134 ceph-mon[35037]: Updating smithi079:/var/lib/ceph/fd220748-ed2b-11ee-95d0-87774f69a715/config/ceph.client.admin.keyring 2024-03-28T17:56:15.202 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:56:15 smithi134 ceph-mon[35037]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T17:56:15.202 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:56:15 smithi134 ceph-mon[35037]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T17:56:15.202 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:56:15 smithi134 ceph-mon[35037]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T17:56:15.202 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:56:15 smithi134 ceph-mon[35037]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T17:56:15.202 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:56:15 smithi134 ceph-mon[35037]: Added host smithi134 2024-03-28T17:56:15.202 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:56:15 smithi134 ceph-mon[35037]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T17:56:15.202 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:56:15 smithi134 ceph-mon[35037]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-28T17:56:15.202 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:56:15 smithi134 ceph-mon[35037]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T17:56:15.202 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:56:15 smithi134 ceph-mon[35037]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T17:56:15.202 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:56:15 smithi134 ceph-mon[35037]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-28T17:56:15.202 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:56:15 smithi134 ceph-mon[35037]: from='client.14184 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-28T17:56:15.202 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:56:15 smithi134 ceph-mon[35037]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T17:56:15.202 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:56:15 smithi134 ceph-mon[35037]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-28T17:56:15.202 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:56:15 smithi134 ceph-mon[35037]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-28T17:56:15.202 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:56:15 smithi134 ceph-mon[35037]: from='client.? 172.21.15.22:0/56783544' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2024-03-28T17:56:15.202 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:56:15 smithi134 ceph-mon[35037]: from='client.? 172.21.15.22:0/56783544' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-03-28T17:56:15.203 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:56:15 smithi134 ceph-mon[35037]: osdmap e4: 0 total, 0 up, 0 in 2024-03-28T17:56:15.203 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:56:15 smithi134 ceph-mon[35037]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-28T17:56:15.203 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:56:15 smithi134 ceph-mon[35037]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T17:56:15.203 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:56:15 smithi134 ceph-mon[35037]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T17:56:15.203 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:56:15 smithi134 ceph-mon[35037]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T17:56:15.203 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:56:15 smithi134 ceph-mon[35037]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T17:56:15.203 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:56:15 smithi134 ceph-mon[35037]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi134", "name": "osd_memory_target"}]: dispatch 2024-03-28T17:56:15.203 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:56:15 smithi134 ceph-mon[35037]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T17:56:15.203 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:56:15 smithi134 ceph-mon[35037]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T17:56:15.203 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:56:15 smithi134 ceph-mon[35037]: Updating smithi134:/etc/ceph/ceph.conf 2024-03-28T17:56:15.203 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:56:15 smithi134 ceph-mon[35037]: Updating smithi134:/var/lib/ceph/fd220748-ed2b-11ee-95d0-87774f69a715/config/ceph.conf 2024-03-28T17:56:15.204 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:56:15 smithi134 ceph-mon[35037]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-28T17:56:15.204 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:56:15 smithi134 ceph-mon[35037]: Updating smithi134:/etc/ceph/ceph.client.admin.keyring 2024-03-28T17:56:15.204 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:56:15 smithi134 ceph-mon[35037]: Updating smithi134:/var/lib/ceph/fd220748-ed2b-11ee-95d0-87774f69a715/config/ceph.client.admin.keyring 2024-03-28T17:56:15.204 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:56:15 smithi134 ceph-mon[35037]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T17:56:15.204 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:56:15 smithi134 ceph-mon[35037]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T17:56:15.204 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:56:15 smithi134 ceph-mon[35037]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T17:56:15.204 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:56:15 smithi134 ceph-mon[35037]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T17:56:15.204 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:56:15 smithi134 ceph-mon[35037]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-28T17:56:15.204 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:56:15 smithi134 ceph-mon[35037]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T17:56:15.204 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:56:15 smithi134 ceph-mon[35037]: from='client.14188 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi022:172.21.15.22=a;smithi079:172.21.15.79=b;smithi134:172.21.15.134=c", "target": ["mon-mgr", ""]}]: dispatch 2024-03-28T17:56:15.204 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:56:15 smithi134 ceph-mon[35037]: Saving service mon spec with placement smithi022:172.21.15.22=a;smithi079:172.21.15.79=b;smithi134:172.21.15.134=c;count:3 2024-03-28T17:56:15.204 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:56:15 smithi134 ceph-mon[35037]: Deploying daemon mon.c on smithi134 2024-03-28T17:56:15.204 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:56:15 smithi134 ceph-mon[35037]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-28T17:56:15.204 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:56:15 smithi134 ceph-mon[35037]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-28T17:56:15.204 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:56:15 smithi134 ceph-mon[35037]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-28T17:56:15.204 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:56:15 smithi134 ceph-mon[35037]: from='client.? 172.21.15.134:0/4267932649' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-03-28T17:56:15.205 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:56:15 smithi134 ceph-mon[35037]: mon.c@-1(synchronizing).paxosservice(auth 1..3) refresh upgraded, format 0 -> 3 2024-03-28T17:56:15.371 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-03-28T17:56:15.371 DEBUG:teuthology.orchestra.run.smithi134:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fd220748-ed2b-11ee-95d0-87774f69a715 -- ceph mon dump -f json 2024-03-28T17:56:15.820 INFO:teuthology.orchestra.run.smithi134.stderr:Inferring config /var/lib/ceph/fd220748-ed2b-11ee-95d0-87774f69a715/mon.c/config 2024-03-28T17:56:20.176 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:56:20 smithi134 ceph-mon[35037]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-28T17:56:20.177 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:56:20 smithi134 ceph-mon[35037]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-28T17:56:20.177 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:56:20 smithi134 ceph-mon[35037]: mon.a calling monitor election 2024-03-28T17:56:20.177 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:56:20 smithi134 ceph-mon[35037]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-28T17:56:20.177 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:56:20 smithi134 ceph-mon[35037]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-28T17:56:20.177 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:56:20 smithi134 ceph-mon[35037]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-28T17:56:20.177 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:56:20 smithi134 ceph-mon[35037]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-28T17:56:20.177 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:56:20 smithi134 ceph-mon[35037]: mon.c calling monitor election 2024-03-28T17:56:20.177 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:56:20 smithi134 ceph-mon[35037]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-28T17:56:20.177 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:56:20 smithi134 ceph-mon[35037]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-28T17:56:20.177 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:56:20 smithi134 ceph-mon[35037]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-28T17:56:20.177 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:56:20 smithi134 ceph-mon[35037]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-28T17:56:20.177 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:56:20 smithi134 ceph-mon[35037]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-03-28T17:56:20.177 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:56:20 smithi134 ceph-mon[35037]: monmap e2: 2 mons at {a=[v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0],c=[v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-03-28T17:56:20.178 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:56:20 smithi134 ceph-mon[35037]: fsmap 2024-03-28T17:56:20.178 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:56:20 smithi134 ceph-mon[35037]: osdmap e4: 0 total, 0 up, 0 in 2024-03-28T17:56:20.178 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:56:20 smithi134 ceph-mon[35037]: mgrmap e13: x(active, since 69s) 2024-03-28T17:56:20.178 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:56:20 smithi134 ceph-mon[35037]: overall HEALTH_OK 2024-03-28T17:56:20.178 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:56:20 smithi134 ceph-mon[35037]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T17:56:20.178 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:56:20 smithi134 ceph-mon[35037]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T17:56:20.178 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:56:20 smithi134 ceph-mon[35037]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T17:56:20.178 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:56:20 smithi134 ceph-mon[35037]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-28T17:56:20.178 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:56:20 smithi134 ceph-mon[35037]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T17:56:20.533 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:56:20 smithi022 ceph-mon[31985]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-28T17:56:20.533 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:56:20 smithi022 ceph-mon[31985]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-28T17:56:20.533 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:56:20 smithi022 ceph-mon[31985]: mon.a calling monitor election 2024-03-28T17:56:20.533 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:56:20 smithi022 ceph-mon[31985]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-28T17:56:20.533 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:56:20 smithi022 ceph-mon[31985]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-28T17:56:20.533 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:56:20 smithi022 ceph-mon[31985]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-28T17:56:20.533 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:56:20 smithi022 ceph-mon[31985]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-28T17:56:20.534 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:56:20 smithi022 ceph-mon[31985]: mon.c calling monitor election 2024-03-28T17:56:20.534 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:56:20 smithi022 ceph-mon[31985]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-28T17:56:20.534 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:56:20 smithi022 ceph-mon[31985]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-28T17:56:20.534 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:56:20 smithi022 ceph-mon[31985]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-28T17:56:20.534 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:56:20 smithi022 ceph-mon[31985]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-28T17:56:20.534 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:56:20 smithi022 ceph-mon[31985]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-03-28T17:56:20.534 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:56:20 smithi022 ceph-mon[31985]: monmap e2: 2 mons at {a=[v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0],c=[v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-03-28T17:56:20.534 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:56:20 smithi022 ceph-mon[31985]: fsmap 2024-03-28T17:56:20.534 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:56:20 smithi022 ceph-mon[31985]: osdmap e4: 0 total, 0 up, 0 in 2024-03-28T17:56:20.534 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:56:20 smithi022 ceph-mon[31985]: mgrmap e13: x(active, since 69s) 2024-03-28T17:56:20.534 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:56:20 smithi022 ceph-mon[31985]: overall HEALTH_OK 2024-03-28T17:56:20.534 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:56:20 smithi022 ceph-mon[31985]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T17:56:20.534 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:56:20 smithi022 ceph-mon[31985]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T17:56:20.534 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:56:20 smithi022 ceph-mon[31985]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T17:56:20.534 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:56:20 smithi022 ceph-mon[31985]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-28T17:56:20.535 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:56:20 smithi022 ceph-mon[31985]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T17:56:20.829 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-28T17:56:20.828+0000 7f0ed8400700 1 -- 172.21.15.134:0/1055514013 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0ed01040e0 msgr2=0x7f0eb8005610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:56:20.829 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-28T17:56:20.828+0000 7f0ed8400700 1 --2- 172.21.15.134:0/1055514013 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0ed01040e0 0x7f0eb8005610 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f0ec4005fa0 tx=0x7f0ec400ffc0 comp rx=0 tx=0).stop 2024-03-28T17:56:20.829 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-28T17:56:20.828+0000 7f0ed8400700 1 -- 172.21.15.134:0/1055514013 shutdown_connections 2024-03-28T17:56:20.829 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-28T17:56:20.828+0000 7f0ed8400700 1 --2- 172.21.15.134:0/1055514013 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0ed01040e0 0x7f0eb8005610 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:56:20.830 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-28T17:56:20.828+0000 7f0ed8400700 1 -- 172.21.15.134:0/1055514013 >> 172.21.15.134:0/1055514013 conn(0x7f0ed00ff990 msgr2=0x7f0ed0101db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T17:56:20.830 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-28T17:56:20.829+0000 7f0ed8400700 1 -- 172.21.15.134:0/1055514013 shutdown_connections 2024-03-28T17:56:20.830 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-28T17:56:20.829+0000 7f0ed8400700 1 -- 172.21.15.134:0/1055514013 wait complete. 2024-03-28T17:56:20.832 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-28T17:56:20.832+0000 7f0ed8400700 1 Processor -- start 2024-03-28T17:56:20.832 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-28T17:56:20.832+0000 7f0ed8400700 1 -- start start 2024-03-28T17:56:20.833 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-28T17:56:20.833+0000 7f0ed8400700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0ed01040e0 0x7f0ed019f6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:56:20.833 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-28T17:56:20.833+0000 7f0ed8400700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0ed019fc00 0x7f0ed01998e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:56:20.833 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-28T17:56:20.833+0000 7f0ed8400700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ed010bf00 con 0x7f0ed019fc00 2024-03-28T17:56:20.833 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-28T17:56:20.833+0000 7f0ed8400700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ed010bd80 con 0x7f0ed01040e0 2024-03-28T17:56:20.834 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-28T17:56:20.834+0000 7f0ed5a00700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0ed01040e0 0x7f0ed019f6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:56:20.834 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-28T17:56:20.834+0000 7f0ed5a00700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0ed01040e0 0x7f0ed019f6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.134:38618/0 (socket says 172.21.15.134:38618) 2024-03-28T17:56:20.835 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-28T17:56:20.834+0000 7f0ed5a00700 1 -- 172.21.15.134:0/1304427795 learned_addr learned my addr 172.21.15.134:0/1304427795 (peer_addr_for_me v2:172.21.15.134:0/0) 2024-03-28T17:56:20.835 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-28T17:56:20.834+0000 7f0ed5000700 1 --2- 172.21.15.134:0/1304427795 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0ed019fc00 0x7f0ed01998e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:56:20.835 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-28T17:56:20.834+0000 7f0ed5a00700 1 -- 172.21.15.134:0/1304427795 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0ed01040e0 msgr2=0x7f0ed019f6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 12 2024-03-28T17:56:20.835 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-28T17:56:20.834+0000 7f0ed5a00700 1 -- 172.21.15.134:0/1304427795 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0ed01040e0 msgr2=0x7f0ed019f6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-03-28T17:56:20.835 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-28T17:56:20.834+0000 7f0ed5a00700 1 --2- 172.21.15.134:0/1304427795 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0ed01040e0 0x7f0ed019f6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-28T17:56:20.835 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-28T17:56:20.834+0000 7f0ed5a00700 1 --2- 172.21.15.134:0/1304427795 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0ed01040e0 0x7f0ed019f6c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-28T17:56:20.835 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-28T17:56:20.835+0000 7f0ed5000700 1 -- 172.21.15.134:0/1304427795 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0ed01040e0 msgr2=0x7f0ed019f6c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-28T17:56:20.835 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-28T17:56:20.835+0000 7f0ed5000700 1 --2- 172.21.15.134:0/1304427795 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0ed01040e0 0x7f0ed019f6c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:56:20.835 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-28T17:56:20.835+0000 7f0ed5000700 1 -- 172.21.15.134:0/1304427795 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ed01a0240 con 0x7f0ed019fc00 2024-03-28T17:56:20.836 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-28T17:56:20.835+0000 7f0ed5000700 1 --2- 172.21.15.134:0/1304427795 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0ed019fc00 0x7f0ed01998e0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f0ebc00b4d0 tx=0x7f0ebc00b890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T17:56:20.836 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-28T17:56:20.835+0000 7f0ece000700 1 -- 172.21.15.134:0/1304427795 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 309+0+0 (secure 0 0 0) 0x7f0ebc011630 con 0x7f0ed019fc00 2024-03-28T17:56:20.836 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-28T17:56:20.836+0000 7f0ed8400700 1 -- 172.21.15.134:0/1304427795 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0ed0199e80 con 0x7f0ed019fc00 2024-03-28T17:56:20.836 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-28T17:56:20.836+0000 7f0ece000700 1 -- 172.21.15.134:0/1304427795 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f0ebc011ca0 con 0x7f0ed019fc00 2024-03-28T17:56:20.836 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-28T17:56:20.836+0000 7f0ece000700 1 -- 172.21.15.134:0/1304427795 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 309+0+0 (secure 0 0 0) 0x7f0ebc010d40 con 0x7f0ed019fc00 2024-03-28T17:56:20.836 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-28T17:56:20.836+0000 7f0ed8400700 1 -- 172.21.15.134:0/1304427795 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0ed019a3e0 con 0x7f0ed019fc00 2024-03-28T17:56:20.837 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-28T17:56:20.837+0000 7f0ece000700 1 -- 172.21.15.134:0/1304427795 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 13) v1 ==== 46162+0+0 (secure 0 0 0) 0x7f0ebc018020 con 0x7f0ed019fc00 2024-03-28T17:56:20.837 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-28T17:56:20.837+0000 7f0ed8400700 1 -- 172.21.15.134:0/1304427795 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0ed01055c0 con 0x7f0ed019fc00 2024-03-28T17:56:20.838 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-28T17:56:20.838+0000 7f0ece000700 1 --2- 172.21.15.134:0/1304427795 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7f0eb0038d80 0x7f0eb003b240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:56:20.838 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-28T17:56:20.838+0000 7f0ece000700 1 -- 172.21.15.134:0/1304427795 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f0ebc04b320 con 0x7f0ed019fc00 2024-03-28T17:56:20.838 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-28T17:56:20.839+0000 7f0ed5a00700 1 --2- 172.21.15.134:0/1304427795 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7f0eb0038d80 0x7f0eb003b240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:56:20.839 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-28T17:56:20.840+0000 7f0ed5a00700 1 --2- 172.21.15.134:0/1304427795 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7f0eb0038d80 0x7f0eb003b240 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f0ec400fb20 tx=0x7f0ec401d040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T17:56:20.853 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-28T17:56:20.853+0000 7f0ece000700 1 -- 172.21.15.134:0/1304427795 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f0ebc01f070 con 0x7f0ed019fc00 2024-03-28T17:56:21.177 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-28T17:56:21.177+0000 7f0ed8400700 1 -- 172.21.15.134:0/1304427795 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f0ed019a750 con 0x7f0ed019fc00 2024-03-28T17:56:21.178 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-28T17:56:21.178+0000 7f0ece000700 1 -- 172.21.15.134:0/1304427795 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 2 v2) v1 ==== 95+0+1015 (secure 0 0 0) 0x7f0ebc018e20 con 0x7f0ed019fc00 2024-03-28T17:56:21.178 INFO:teuthology.orchestra.run.smithi134.stdout: 2024-03-28T17:56:21.178 INFO:teuthology.orchestra.run.smithi134.stdout:{"epoch":2,"fsid":"fd220748-ed2b-11ee-95d0-87774f69a715","modified":"2024-03-28T17:56:15.021482Z","created":"2024-03-28T17:53:56.145153Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:3300","nonce":0},{"type":"v1","addr":"172.21.15.22:6789","nonce":0}]},"addr":"172.21.15.22:6789/0","public_addr":"172.21.15.22:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3300","nonce":0},{"type":"v1","addr":"172.21.15.134:6789","nonce":0}]},"addr":"172.21.15.134:6789/0","public_addr":"172.21.15.134:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-03-28T17:56:21.181 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-28T17:56:21.181+0000 7f0ed8400700 1 -- 172.21.15.134:0/1304427795 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7f0eb0038d80 msgr2=0x7f0eb003b240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:56:21.181 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-28T17:56:21.181+0000 7f0ed8400700 1 --2- 172.21.15.134:0/1304427795 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7f0eb0038d80 0x7f0eb003b240 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f0ec400fb20 tx=0x7f0ec401d040 comp rx=0 tx=0).stop 2024-03-28T17:56:21.181 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-28T17:56:21.181+0000 7f0ed8400700 1 -- 172.21.15.134:0/1304427795 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0ed019fc00 msgr2=0x7f0ed01998e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:56:21.181 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-28T17:56:21.181+0000 7f0ed8400700 1 --2- 172.21.15.134:0/1304427795 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0ed019fc00 0x7f0ed01998e0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f0ebc00b4d0 tx=0x7f0ebc00b890 comp rx=0 tx=0).stop 2024-03-28T17:56:21.181 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-28T17:56:21.181+0000 7f0ed8400700 1 -- 172.21.15.134:0/1304427795 shutdown_connections 2024-03-28T17:56:21.181 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-28T17:56:21.181+0000 7f0ed8400700 1 --2- 172.21.15.134:0/1304427795 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7f0eb0038d80 0x7f0eb003b240 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:56:21.181 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-28T17:56:21.181+0000 7f0ed8400700 1 --2- 172.21.15.134:0/1304427795 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0ed01040e0 0x7f0ed019f6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:56:21.181 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-28T17:56:21.181+0000 7f0ed8400700 1 --2- 172.21.15.134:0/1304427795 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0ed019fc00 0x7f0ed01998e0 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:56:21.181 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-28T17:56:21.181+0000 7f0ed8400700 1 -- 172.21.15.134:0/1304427795 >> 172.21.15.134:0/1304427795 conn(0x7f0ed00ff990 msgr2=0x7f0ed0100700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T17:56:21.182 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-28T17:56:21.181+0000 7f0ed8400700 1 -- 172.21.15.134:0/1304427795 shutdown_connections 2024-03-28T17:56:21.182 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-28T17:56:21.181+0000 7f0ed8400700 1 -- 172.21.15.134:0/1304427795 wait complete. 2024-03-28T17:56:21.182 INFO:teuthology.orchestra.run.smithi134.stderr:dumped monmap epoch 2 2024-03-28T17:56:21.424 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:56:21 smithi134 ceph-mon[35037]: Deploying daemon mon.b on smithi079 2024-03-28T17:56:21.425 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:56:21 smithi134 ceph-mon[35037]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-28T17:56:21.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:56:21 smithi022 ceph-mon[31985]: Deploying daemon mon.b on smithi079 2024-03-28T17:56:21.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:56:21 smithi022 ceph-mon[31985]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-28T17:56:22.241 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:56:22 smithi022 ceph-mon[31985]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-28T17:56:22.241 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:56:22 smithi022 ceph-mon[31985]: from='client.? 172.21.15.134:0/1304427795' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-03-28T17:56:22.241 INFO:journalctl@ceph.mgr.x.smithi022.stdout:Mar 28 17:56:22 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mgr-x[32211]: 2024-03-28T17:56:22.022+0000 7fd84ee00700 -1 mgr.server handle_report got status from non-daemon mon.c 2024-03-28T17:56:22.424 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:56:22 smithi134 ceph-mon[35037]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-28T17:56:22.424 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:56:22 smithi134 ceph-mon[35037]: from='client.? 172.21.15.134:0/1304427795' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-03-28T17:56:22.916 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-03-28T17:56:22.917 DEBUG:teuthology.orchestra.run.smithi134:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fd220748-ed2b-11ee-95d0-87774f69a715 -- ceph mon dump -f json 2024-03-28T17:56:23.346 INFO:teuthology.orchestra.run.smithi134.stderr:Inferring config /var/lib/ceph/fd220748-ed2b-11ee-95d0-87774f69a715/mon.c/config 2024-03-28T17:56:24.424 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:56:24 smithi134 ceph-mon[35037]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-28T17:56:24.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:56:24 smithi022 ceph-mon[31985]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-28T17:56:25.271 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-28T17:56:25.270+0000 7f1db8200700 1 -- 172.21.15.134:0/3482667015 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1db0101ee0 msgr2=0x7f1db01022c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:56:25.271 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-28T17:56:25.270+0000 7f1db8200700 1 --2- 172.21.15.134:0/3482667015 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1db0101ee0 0x7f1db01022c0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f1da4009910 tx=0x7f1da4009c20 comp rx=0 tx=0).stop 2024-03-28T17:56:25.271 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-28T17:56:25.270+0000 7f1db8200700 1 -- 172.21.15.134:0/3482667015 shutdown_connections 2024-03-28T17:56:25.271 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-28T17:56:25.270+0000 7f1db8200700 1 --2- 172.21.15.134:0/3482667015 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1db0101ee0 0x7f1db01022c0 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:56:25.272 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-28T17:56:25.270+0000 7f1db8200700 1 -- 172.21.15.134:0/3482667015 >> 172.21.15.134:0/3482667015 conn(0x7f1db00fd790 msgr2=0x7f1db00ffbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T17:56:25.272 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-28T17:56:25.271+0000 7f1db8200700 1 -- 172.21.15.134:0/3482667015 shutdown_connections 2024-03-28T17:56:25.272 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-28T17:56:25.271+0000 7f1db8200700 1 -- 172.21.15.134:0/3482667015 wait complete. 2024-03-28T17:56:25.273 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-28T17:56:25.272+0000 7f1db8200700 1 Processor -- start 2024-03-28T17:56:25.273 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-28T17:56:25.272+0000 7f1db8200700 1 -- start start 2024-03-28T17:56:25.273 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-28T17:56:25.272+0000 7f1db8200700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1db0101ee0 0x7f1db019f7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:56:25.273 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-28T17:56:25.272+0000 7f1db8200700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1db019fce0 0x7f1db0199960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:56:25.273 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-28T17:56:25.272+0000 7f1db8200700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1db010c2a0 con 0x7f1db0101ee0 2024-03-28T17:56:25.273 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-28T17:56:25.272+0000 7f1db8200700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1db010c120 con 0x7f1db019fce0 2024-03-28T17:56:25.273 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-28T17:56:25.273+0000 7f1db4e00700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1db019fce0 0x7f1db0199960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:56:25.273 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-28T17:56:25.273+0000 7f1db4e00700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1db019fce0 0x7f1db0199960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.134:38622/0 (socket says 172.21.15.134:38622) 2024-03-28T17:56:25.273 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-28T17:56:25.273+0000 7f1db4e00700 1 -- 172.21.15.134:0/1503826216 learned_addr learned my addr 172.21.15.134:0/1503826216 (peer_addr_for_me v2:172.21.15.134:0/0) 2024-03-28T17:56:25.273 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-28T17:56:25.273+0000 7f1db5800700 1 --2- 172.21.15.134:0/1503826216 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1db0101ee0 0x7f1db019f7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:56:25.274 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-28T17:56:25.273+0000 7f1db4e00700 1 -- 172.21.15.134:0/1503826216 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1db0101ee0 msgr2=0x7f1db019f7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:56:25.274 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-28T17:56:25.273+0000 7f1db4e00700 1 --2- 172.21.15.134:0/1503826216 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1db0101ee0 0x7f1db019f7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:56:25.274 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-28T17:56:25.273+0000 7f1db4e00700 1 -- 172.21.15.134:0/1503826216 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1db01a0320 con 0x7f1db019fce0 2024-03-28T17:56:25.274 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-28T17:56:25.274+0000 7f1db4e00700 1 --2- 172.21.15.134:0/1503826216 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1db019fce0 0x7f1db0199960 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7f1d9c00b700 tx=0x7f1d9c00bac0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T17:56:25.275 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-28T17:56:25.274+0000 7f1dae000700 1 -- 172.21.15.134:0/1503826216 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 309+0+0 (secure 0 0 0) 0x7f1d9c011630 con 0x7f1db019fce0 2024-03-28T17:56:25.275 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-28T17:56:25.274+0000 7f1dae000700 1 -- 172.21.15.134:0/1503826216 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f1d9c011ca0 con 0x7f1db019fce0 2024-03-28T17:56:25.275 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-28T17:56:25.274+0000 7f1dae000700 1 -- 172.21.15.134:0/1503826216 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 v1 ==== 309+0+0 (secure 0 0 0) 0x7f1d9c010dd0 con 0x7f1db019fce0 2024-03-28T17:56:25.275 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-28T17:56:25.274+0000 7f1db8200700 1 -- 172.21.15.134:0/1503826216 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1db0199ea0 con 0x7f1db019fce0 2024-03-28T17:56:25.275 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-28T17:56:25.274+0000 7f1db8200700 1 -- 172.21.15.134:0/1503826216 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1db019a430 con 0x7f1db019fce0 2024-03-28T17:56:25.275 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-28T17:56:25.275+0000 7f1dae000700 1 -- 172.21.15.134:0/1503826216 <== mon.1 v2:172.21.15.134:3300/0 4 ==== mgrmap(e 13) v1 ==== 46162+0+0 (secure 0 0 0) 0x7f1d9c018020 con 0x7f1db019fce0 2024-03-28T17:56:25.276 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-28T17:56:25.275+0000 7f1db8200700 1 -- 172.21.15.134:0/1503826216 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1db01033c0 con 0x7f1db019fce0 2024-03-28T17:56:25.276 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-28T17:56:25.276+0000 7f1dae000700 1 --2- 172.21.15.134:0/1503826216 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7f1d98038d60 0x7f1d9803b220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:56:25.276 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-28T17:56:25.276+0000 7f1dae000700 1 -- 172.21.15.134:0/1503826216 <== mon.1 v2:172.21.15.134:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f1d9c04bac0 con 0x7f1db019fce0 2024-03-28T17:56:25.276 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-28T17:56:25.276+0000 7f1db5800700 1 --2- 172.21.15.134:0/1503826216 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7f1d98038d60 0x7f1d9803b220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:56:25.277 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-28T17:56:25.278+0000 7f1db5800700 1 --2- 172.21.15.134:0/1503826216 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7f1d98038d60 0x7f1d9803b220 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f1da4009750 tx=0x7f1da400b580 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T17:56:25.282 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-28T17:56:25.282+0000 7f1dae000700 1 -- 172.21.15.134:0/1503826216 <== mon.1 v2:172.21.15.134:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f1d9c01f020 con 0x7f1db019fce0 2024-03-28T17:56:25.588 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-28T17:56:25.588+0000 7f1db8200700 1 -- 172.21.15.134:0/1503826216 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f1db019ae90 con 0x7f1db019fce0 2024-03-28T17:56:25.588 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-28T17:56:25.588+0000 7f1dae000700 1 -- 172.21.15.134:0/1503826216 <== mon.1 v2:172.21.15.134:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 2 v2) v1 ==== 95+0+1015 (secure 0 0 0) 0x7f1d9c018570 con 0x7f1db019fce0 2024-03-28T17:56:25.589 INFO:teuthology.orchestra.run.smithi134.stdout: 2024-03-28T17:56:25.589 INFO:teuthology.orchestra.run.smithi134.stdout:{"epoch":2,"fsid":"fd220748-ed2b-11ee-95d0-87774f69a715","modified":"2024-03-28T17:56:15.021482Z","created":"2024-03-28T17:53:56.145153Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:3300","nonce":0},{"type":"v1","addr":"172.21.15.22:6789","nonce":0}]},"addr":"172.21.15.22:6789/0","public_addr":"172.21.15.22:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3300","nonce":0},{"type":"v1","addr":"172.21.15.134:6789","nonce":0}]},"addr":"172.21.15.134:6789/0","public_addr":"172.21.15.134:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-03-28T17:56:25.592 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-28T17:56:25.591+0000 7f1db8200700 1 -- 172.21.15.134:0/1503826216 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7f1d98038d60 msgr2=0x7f1d9803b220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:56:25.592 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-28T17:56:25.591+0000 7f1db8200700 1 --2- 172.21.15.134:0/1503826216 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7f1d98038d60 0x7f1d9803b220 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f1da4009750 tx=0x7f1da400b580 comp rx=0 tx=0).stop 2024-03-28T17:56:25.592 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-28T17:56:25.591+0000 7f1db8200700 1 -- 172.21.15.134:0/1503826216 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1db019fce0 msgr2=0x7f1db0199960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:56:25.592 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-28T17:56:25.591+0000 7f1db8200700 1 --2- 172.21.15.134:0/1503826216 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1db019fce0 0x7f1db0199960 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7f1d9c00b700 tx=0x7f1d9c00bac0 comp rx=0 tx=0).stop 2024-03-28T17:56:25.592 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-28T17:56:25.592+0000 7f1db8200700 1 -- 172.21.15.134:0/1503826216 shutdown_connections 2024-03-28T17:56:25.592 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-28T17:56:25.592+0000 7f1db8200700 1 --2- 172.21.15.134:0/1503826216 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1db0101ee0 0x7f1db019f7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:56:25.592 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-28T17:56:25.592+0000 7f1db8200700 1 --2- 172.21.15.134:0/1503826216 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7f1d98038d60 0x7f1d9803b220 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:56:25.593 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-28T17:56:25.592+0000 7f1db8200700 1 --2- 172.21.15.134:0/1503826216 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1db019fce0 0x7f1db0199960 unknown :-1 s=CLOSED pgs=3 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:56:25.593 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-28T17:56:25.592+0000 7f1db8200700 1 -- 172.21.15.134:0/1503826216 >> 172.21.15.134:0/1503826216 conn(0x7f1db00fd790 msgr2=0x7f1db0107a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T17:56:25.593 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-28T17:56:25.592+0000 7f1db8200700 1 -- 172.21.15.134:0/1503826216 shutdown_connections 2024-03-28T17:56:25.593 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-28T17:56:25.592+0000 7f1db8200700 1 -- 172.21.15.134:0/1503826216 wait complete. 2024-03-28T17:56:25.593 INFO:teuthology.orchestra.run.smithi134.stderr:dumped monmap epoch 2 2024-03-28T17:56:25.829 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:56:25 smithi079 podman[35293]: 2024-03-28 17:56:25.575194464 +0000 UTC m=+0.135327340 container create 7fcef29b8b90d8090703b144fb4873f845e0766a28bbf6614e93742dc2179326 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mon-b, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.build-date=20240319, maintainer=Guillaume Abrioux ) 2024-03-28T17:56:25.830 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:56:25 smithi079 podman[35293]: 2024-03-28 17:56:25.828809926 +0000 UTC m=+0.388942781 container init 7fcef29b8b90d8090703b144fb4873f845e0766a28bbf6614e93742dc2179326 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mon-b, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, GIT_CLEAN=True, org.label-schema.build-date=20240319, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.license=GPLv2) 2024-03-28T17:56:26.183 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:56:25 smithi079 podman[35293]: 2024-03-28 17:56:25.83909816 +0000 UTC m=+0.399231016 container start 7fcef29b8b90d8090703b144fb4873f845e0766a28bbf6614e93742dc2179326 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mon-b, ceph=True, maintainer=Guillaume Abrioux , GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240319, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2) 2024-03-28T17:56:26.184 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:56:25 smithi079 ceph-mon[35312]: set uid:gid to 167:167 (ceph:ceph) 2024-03-28T17:56:26.184 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:56:25 smithi079 ceph-mon[35312]: ceph version 19.0.0-2145-g3be9a5c9 (3be9a5c9bc793e11e2800a8c0c696e8b46742033) squid (dev), process ceph-mon, pid 2 2024-03-28T17:56:26.184 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:56:25 smithi079 ceph-mon[35312]: pidfile_write: ignore empty --pid-file 2024-03-28T17:56:26.184 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:56:25 smithi079 ceph-mon[35312]: load: jerasure load: lrc 2024-03-28T17:56:26.184 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:56:25 smithi079 ceph-mon[35312]: rocksdb: RocksDB version: 7.9.2 2024-03-28T17:56:26.184 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:56:25 smithi079 ceph-mon[35312]: rocksdb: Git sha 0 2024-03-28T17:56:26.184 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:56:25 smithi079 ceph-mon[35312]: rocksdb: Compile date 2024-03-22 04:58:55 2024-03-28T17:56:26.184 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:56:25 smithi079 ceph-mon[35312]: rocksdb: DB SUMMARY 2024-03-28T17:56:26.184 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:56:25 smithi079 ceph-mon[35312]: rocksdb: DB Session ID: V7EREQZGW2XZ6HMVUPAU 2024-03-28T17:56:26.184 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:56:25 smithi079 ceph-mon[35312]: rocksdb: CURRENT file: CURRENT 2024-03-28T17:56:26.184 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:56:25 smithi079 ceph-mon[35312]: rocksdb: IDENTITY file: IDENTITY 2024-03-28T17:56:26.184 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:56:25 smithi079 ceph-mon[35312]: rocksdb: MANIFEST file: MANIFEST-000005 size: 59 Bytes 2024-03-28T17:56:26.185 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:56:25 smithi079 ceph-mon[35312]: rocksdb: SST files in /var/lib/ceph/mon/ceph-b/store.db dir, Total Num: 0, files: 2024-03-28T17:56:26.185 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:56:25 smithi079 ceph-mon[35312]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-b/store.db: 000004.log size: 636 ; 2024-03-28T17:56:26.185 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:56:25 smithi079 ceph-mon[35312]: rocksdb: Options.error_if_exists: 0 2024-03-28T17:56:26.185 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:56:25 smithi079 ceph-mon[35312]: rocksdb: Options.create_if_missing: 0 2024-03-28T17:56:26.185 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:56:25 smithi079 ceph-mon[35312]: rocksdb: Options.paranoid_checks: 1 2024-03-28T17:56:26.185 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:56:25 smithi079 ceph-mon[35312]: rocksdb: Options.flush_verify_memtable_count: 1 2024-03-28T17:56:26.185 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:56:25 smithi079 ceph-mon[35312]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-03-28T17:56:26.185 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:56:25 smithi079 ceph-mon[35312]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-03-28T17:56:26.185 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:56:25 smithi079 ceph-mon[35312]: rocksdb: Options.env: 0x55d8c29e1b20 2024-03-28T17:56:26.185 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:56:25 smithi079 ceph-mon[35312]: rocksdb: Options.fs: PosixFileSystem 2024-03-28T17:56:26.185 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:56:25 smithi079 ceph-mon[35312]: rocksdb: Options.info_log: 0x55d8c40ff960 2024-03-28T17:56:26.185 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:56:25 smithi079 ceph-mon[35312]: rocksdb: Options.max_file_opening_threads: 16 2024-03-28T17:56:26.185 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:56:25 smithi079 ceph-mon[35312]: rocksdb: Options.statistics: (nil) 2024-03-28T17:56:26.185 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:56:25 smithi079 ceph-mon[35312]: rocksdb: Options.use_fsync: 0 2024-03-28T17:56:26.186 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:56:25 smithi079 ceph-mon[35312]: rocksdb: Options.max_log_file_size: 0 2024-03-28T17:56:26.186 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:56:25 smithi079 ceph-mon[35312]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-03-28T17:56:26.186 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:56:25 smithi079 ceph-mon[35312]: rocksdb: Options.log_file_time_to_roll: 0 2024-03-28T17:56:26.186 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:56:25 smithi079 ceph-mon[35312]: rocksdb: Options.keep_log_file_num: 1000 2024-03-28T17:56:26.186 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:56:25 smithi079 ceph-mon[35312]: rocksdb: Options.recycle_log_file_num: 0 2024-03-28T17:56:26.186 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:56:25 smithi079 ceph-mon[35312]: rocksdb: Options.allow_fallocate: 1 2024-03-28T17:56:26.187 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:56:25 smithi079 ceph-mon[35312]: rocksdb: Options.allow_mmap_reads: 0 2024-03-28T17:56:26.187 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:56:25 smithi079 ceph-mon[35312]: rocksdb: Options.allow_mmap_writes: 0 2024-03-28T17:56:26.187 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:56:25 smithi079 ceph-mon[35312]: rocksdb: Options.use_direct_reads: 0 2024-03-28T17:56:26.187 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:56:25 smithi079 ceph-mon[35312]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-03-28T17:56:26.187 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:56:25 smithi079 ceph-mon[35312]: rocksdb: Options.create_missing_column_families: 0 2024-03-28T17:56:26.187 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:56:25 smithi079 ceph-mon[35312]: rocksdb: Options.db_log_dir: 2024-03-28T17:56:26.187 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:56:25 smithi079 ceph-mon[35312]: rocksdb: Options.wal_dir: 2024-03-28T17:56:26.187 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:56:25 smithi079 ceph-mon[35312]: rocksdb: Options.table_cache_numshardbits: 6 2024-03-28T17:56:26.187 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:56:25 smithi079 ceph-mon[35312]: rocksdb: Options.WAL_ttl_seconds: 0 2024-03-28T17:56:26.187 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:56:25 smithi079 ceph-mon[35312]: rocksdb: Options.WAL_size_limit_MB: 0 2024-03-28T17:56:26.188 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:56:25 smithi079 ceph-mon[35312]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-03-28T17:56:26.188 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:56:25 smithi079 ceph-mon[35312]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-03-28T17:56:26.188 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:56:25 smithi079 ceph-mon[35312]: rocksdb: Options.is_fd_close_on_exec: 1 2024-03-28T17:56:26.188 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:56:25 smithi079 ceph-mon[35312]: rocksdb: Options.advise_random_on_open: 1 2024-03-28T17:56:26.188 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:56:25 smithi079 ceph-mon[35312]: rocksdb: Options.db_write_buffer_size: 0 2024-03-28T17:56:26.188 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:56:25 smithi079 ceph-mon[35312]: rocksdb: Options.write_buffer_manager: 0x55d8c40d21e0 2024-03-28T17:56:26.188 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:56:25 smithi079 ceph-mon[35312]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-03-28T17:56:26.188 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:56:25 smithi079 ceph-mon[35312]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-03-28T17:56:26.188 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:56:25 smithi079 ceph-mon[35312]: rocksdb: Options.use_adaptive_mutex: 0 2024-03-28T17:56:26.188 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:56:25 smithi079 ceph-mon[35312]: rocksdb: Options.rate_limiter: (nil) 2024-03-28T17:56:26.188 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:56:25 smithi079 ceph-mon[35312]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-03-28T17:56:26.188 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:56:25 smithi079 ceph-mon[35312]: rocksdb: Options.wal_recovery_mode: 2 2024-03-28T17:56:26.188 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:56:25 smithi079 ceph-mon[35312]: rocksdb: Options.enable_thread_tracking: 0 2024-03-28T17:56:26.188 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:56:25 smithi079 ceph-mon[35312]: rocksdb: Options.enable_pipelined_write: 0 2024-03-28T17:56:26.189 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:56:25 smithi079 ceph-mon[35312]: rocksdb: Options.unordered_write: 0 2024-03-28T17:56:26.189 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:56:25 smithi079 ceph-mon[35312]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-03-28T17:56:26.189 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:56:25 smithi079 ceph-mon[35312]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-03-28T17:56:26.189 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:56:25 smithi079 ceph-mon[35312]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-03-28T17:56:26.189 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:56:25 smithi079 ceph-mon[35312]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-03-28T17:56:26.189 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:56:25 smithi079 ceph-mon[35312]: rocksdb: Options.row_cache: None 2024-03-28T17:56:26.189 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:56:25 smithi079 ceph-mon[35312]: rocksdb: Options.wal_filter: None 2024-03-28T17:56:26.189 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:56:25 smithi079 ceph-mon[35312]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-03-28T17:56:26.189 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:56:25 smithi079 ceph-mon[35312]: rocksdb: Options.allow_ingest_behind: 0 2024-03-28T17:56:26.189 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:56:25 smithi079 ceph-mon[35312]: rocksdb: Options.two_write_queues: 0 2024-03-28T17:56:26.189 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:56:25 smithi079 ceph-mon[35312]: rocksdb: Options.manual_wal_flush: 0 2024-03-28T17:56:26.189 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:56:25 smithi079 ceph-mon[35312]: rocksdb: Options.wal_compression: 0 2024-03-28T17:56:26.189 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:56:25 smithi079 ceph-mon[35312]: rocksdb: Options.atomic_flush: 0 2024-03-28T17:56:26.189 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:56:25 smithi079 ceph-mon[35312]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-03-28T17:56:26.190 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:56:25 smithi079 ceph-mon[35312]: rocksdb: Options.persist_stats_to_disk: 0 2024-03-28T17:56:26.190 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:56:25 smithi079 ceph-mon[35312]: rocksdb: Options.write_dbid_to_manifest: 0 2024-03-28T17:56:26.190 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:56:25 smithi079 ceph-mon[35312]: rocksdb: Options.log_readahead_size: 0 2024-03-28T17:56:26.190 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:56:25 smithi079 ceph-mon[35312]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-03-28T17:56:26.190 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:56:25 smithi079 ceph-mon[35312]: rocksdb: Options.best_efforts_recovery: 0 2024-03-28T17:56:26.190 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:56:25 smithi079 ceph-mon[35312]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-03-28T17:56:26.190 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:56:25 smithi079 ceph-mon[35312]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-03-28T17:56:26.190 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:56:25 smithi079 ceph-mon[35312]: rocksdb: Options.allow_data_in_errors: 0 2024-03-28T17:56:26.190 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:56:25 smithi079 ceph-mon[35312]: rocksdb: Options.db_host_id: __hostname__ 2024-03-28T17:56:26.190 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:56:25 smithi079 ceph-mon[35312]: rocksdb: Options.enforce_single_del_contracts: true 2024-03-28T17:56:26.190 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:56:25 smithi079 ceph-mon[35312]: rocksdb: Options.max_background_jobs: 2 2024-03-28T17:56:26.190 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:56:25 smithi079 ceph-mon[35312]: rocksdb: Options.max_background_compactions: -1 2024-03-28T17:56:26.190 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:56:25 smithi079 ceph-mon[35312]: rocksdb: Options.max_subcompactions: 1 2024-03-28T17:56:26.190 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:56:25 smithi079 ceph-mon[35312]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-03-28T17:56:26.191 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:56:25 smithi079 ceph-mon[35312]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-03-28T17:56:26.191 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:56:25 smithi079 ceph-mon[35312]: rocksdb: Options.delayed_write_rate : 16777216 2024-03-28T17:56:26.191 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:56:25 smithi079 ceph-mon[35312]: rocksdb: Options.max_total_wal_size: 0 2024-03-28T17:56:26.191 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:56:25 smithi079 ceph-mon[35312]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-03-28T17:56:26.191 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:56:25 smithi079 ceph-mon[35312]: rocksdb: Options.stats_dump_period_sec: 600 2024-03-28T17:56:26.191 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:56:25 smithi079 ceph-mon[35312]: rocksdb: Options.stats_persist_period_sec: 600 2024-03-28T17:56:26.191 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:56:25 smithi079 ceph-mon[35312]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-03-28T17:56:26.191 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:56:25 smithi079 ceph-mon[35312]: rocksdb: Options.max_open_files: -1 2024-03-28T17:56:26.191 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:56:25 smithi079 ceph-mon[35312]: rocksdb: Options.bytes_per_sync: 0 2024-03-28T17:56:26.191 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:56:25 smithi079 ceph-mon[35312]: rocksdb: Options.wal_bytes_per_sync: 0 2024-03-28T17:56:26.191 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:56:25 smithi079 ceph-mon[35312]: rocksdb: Options.strict_bytes_per_sync: 0 2024-03-28T17:56:26.191 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:56:25 smithi079 ceph-mon[35312]: rocksdb: Options.compaction_readahead_size: 0 2024-03-28T17:56:26.191 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:56:25 smithi079 ceph-mon[35312]: rocksdb: Options.max_background_flushes: -1 2024-03-28T17:56:26.191 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:56:25 smithi079 ceph-mon[35312]: rocksdb: Compression algorithms supported: 2024-03-28T17:56:26.192 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:56:25 smithi079 ceph-mon[35312]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-03-28T17:56:26.192 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:56:25 smithi079 ceph-mon[35312]: rocksdb: kZSTD supported: 0 2024-03-28T17:56:26.192 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:56:25 smithi079 ceph-mon[35312]: rocksdb: kXpressCompression supported: 0 2024-03-28T17:56:26.192 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:56:25 smithi079 ceph-mon[35312]: rocksdb: kLZ4HCCompression supported: 1 2024-03-28T17:56:26.192 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:56:25 smithi079 ceph-mon[35312]: rocksdb: kZlibCompression supported: 1 2024-03-28T17:56:26.192 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:56:25 smithi079 ceph-mon[35312]: rocksdb: kSnappyCompression supported: 1 2024-03-28T17:56:26.192 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:56:25 smithi079 ceph-mon[35312]: rocksdb: kLZ4Compression supported: 1 2024-03-28T17:56:26.192 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:56:25 smithi079 ceph-mon[35312]: rocksdb: kBZip2Compression supported: 0 2024-03-28T17:56:26.192 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:56:25 smithi079 ceph-mon[35312]: rocksdb: Fast CRC32 supported: Supported on x86 2024-03-28T17:56:26.192 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:56:25 smithi079 ceph-mon[35312]: rocksdb: DMutex implementation: pthread_mutex_t 2024-03-28T17:56:26.192 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:56:25 smithi079 ceph-mon[35312]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-b/store.db/MANIFEST-000005 2024-03-28T17:56:26.192 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:56:25 smithi079 ceph-mon[35312]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-03-28T17:56:26.192 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:56:25 smithi079 ceph-mon[35312]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-03-28T17:56:26.192 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:56:25 smithi079 ceph-mon[35312]: rocksdb: Options.merge_operator: 2024-03-28T17:56:26.192 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:56:25 smithi079 ceph-mon[35312]: rocksdb: Options.compaction_filter: None 2024-03-28T17:56:26.193 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:56:25 smithi079 ceph-mon[35312]: rocksdb: Options.compaction_filter_factory: None 2024-03-28T17:56:26.193 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:56:25 smithi079 ceph-mon[35312]: rocksdb: Options.sst_partitioner_factory: None 2024-03-28T17:56:26.193 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:56:25 smithi079 ceph-mon[35312]: rocksdb: Options.memtable_factory: SkipListFactory 2024-03-28T17:56:26.193 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:56:25 smithi079 ceph-mon[35312]: rocksdb: Options.table_factory: BlockBasedTable 2024-03-28T17:56:26.193 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:56:25 smithi079 ceph-mon[35312]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x55d8c40ffa60) 2024-03-28T17:56:26.193 INFO:journalctl@ceph.mon.b.smithi079.stdout: cache_index_and_filter_blocks: 1 2024-03-28T17:56:26.193 INFO:journalctl@ceph.mon.b.smithi079.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-03-28T17:56:26.193 INFO:journalctl@ceph.mon.b.smithi079.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-03-28T17:56:26.193 INFO:journalctl@ceph.mon.b.smithi079.stdout: pin_top_level_index_and_filter: 1 2024-03-28T17:56:26.193 INFO:journalctl@ceph.mon.b.smithi079.stdout: index_type: 0 2024-03-28T17:56:26.193 INFO:journalctl@ceph.mon.b.smithi079.stdout: data_block_index_type: 0 2024-03-28T17:56:26.193 INFO:journalctl@ceph.mon.b.smithi079.stdout: index_shortening: 1 2024-03-28T17:56:26.193 INFO:journalctl@ceph.mon.b.smithi079.stdout: data_block_hash_table_util_ratio: 0.750000 2024-03-28T17:56:26.193 INFO:journalctl@ceph.mon.b.smithi079.stdout: checksum: 4 2024-03-28T17:56:26.194 INFO:journalctl@ceph.mon.b.smithi079.stdout: no_block_cache: 0 2024-03-28T17:56:26.194 INFO:journalctl@ceph.mon.b.smithi079.stdout: block_cache: 0x55d8c4105090 2024-03-28T17:56:26.194 INFO:journalctl@ceph.mon.b.smithi079.stdout: block_cache_name: BinnedLRUCache 2024-03-28T17:56:26.194 INFO:journalctl@ceph.mon.b.smithi079.stdout: block_cache_options: 2024-03-28T17:56:26.194 INFO:journalctl@ceph.mon.b.smithi079.stdout: capacity : 536870912 2024-03-28T17:56:26.194 INFO:journalctl@ceph.mon.b.smithi079.stdout: num_shard_bits : 4 2024-03-28T17:56:26.194 INFO:journalctl@ceph.mon.b.smithi079.stdout: strict_capacity_limit : 0 2024-03-28T17:56:26.194 INFO:journalctl@ceph.mon.b.smithi079.stdout: high_pri_pool_ratio: 0.000 2024-03-28T17:56:26.194 INFO:journalctl@ceph.mon.b.smithi079.stdout: block_cache_compressed: (nil) 2024-03-28T17:56:26.194 INFO:journalctl@ceph.mon.b.smithi079.stdout: persistent_cache: (nil) 2024-03-28T17:56:26.194 INFO:journalctl@ceph.mon.b.smithi079.stdout: block_size: 4096 2024-03-28T17:56:26.194 INFO:journalctl@ceph.mon.b.smithi079.stdout: block_size_deviation: 10 2024-03-28T17:56:26.194 INFO:journalctl@ceph.mon.b.smithi079.stdout: block_restart_interval: 16 2024-03-28T17:56:26.194 INFO:journalctl@ceph.mon.b.smithi079.stdout: index_block_restart_interval: 1 2024-03-28T17:56:26.194 INFO:journalctl@ceph.mon.b.smithi079.stdout: metadata_block_size: 4096 2024-03-28T17:56:26.195 INFO:journalctl@ceph.mon.b.smithi079.stdout: partition_filters: 0 2024-03-28T17:56:26.195 INFO:journalctl@ceph.mon.b.smithi079.stdout: use_delta_encoding: 1 2024-03-28T17:56:26.195 INFO:journalctl@ceph.mon.b.smithi079.stdout: filter_policy: bloomfilter 2024-03-28T17:56:26.195 INFO:journalctl@ceph.mon.b.smithi079.stdout: whole_key_filtering: 1 2024-03-28T17:56:26.195 INFO:journalctl@ceph.mon.b.smithi079.stdout: verify_compression: 0 2024-03-28T17:56:26.195 INFO:journalctl@ceph.mon.b.smithi079.stdout: read_amp_bytes_per_bit: 0 2024-03-28T17:56:26.195 INFO:journalctl@ceph.mon.b.smithi079.stdout: format_version: 5 2024-03-28T17:56:26.195 INFO:journalctl@ceph.mon.b.smithi079.stdout: enable_index_compression: 1 2024-03-28T17:56:26.195 INFO:journalctl@ceph.mon.b.smithi079.stdout: block_align: 0 2024-03-28T17:56:26.195 INFO:journalctl@ceph.mon.b.smithi079.stdout: max_auto_readahead_size: 262144 2024-03-28T17:56:26.195 INFO:journalctl@ceph.mon.b.smithi079.stdout: prepopulate_block_cache: 0 2024-03-28T17:56:26.195 INFO:journalctl@ceph.mon.b.smithi079.stdout: initial_auto_readahead_size: 8192 2024-03-28T17:56:26.195 INFO:journalctl@ceph.mon.b.smithi079.stdout: num_file_reads_for_auto_readahead: 2 2024-03-28T17:56:26.195 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:56:25 smithi079 ceph-mon[35312]: rocksdb: Options.write_buffer_size: 33554432 2024-03-28T17:56:26.196 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:56:25 smithi079 ceph-mon[35312]: rocksdb: Options.max_write_buffer_number: 2 2024-03-28T17:56:26.196 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:56:25 smithi079 ceph-mon[35312]: rocksdb: Options.compression: NoCompression 2024-03-28T17:56:26.196 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:56:25 smithi079 ceph-mon[35312]: rocksdb: Options.bottommost_compression: Disabled 2024-03-28T17:56:26.196 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:56:25 smithi079 ceph-mon[35312]: rocksdb: Options.prefix_extractor: nullptr 2024-03-28T17:56:26.196 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:56:25 smithi079 ceph-mon[35312]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-03-28T17:56:26.196 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:56:25 smithi079 ceph-mon[35312]: rocksdb: Options.num_levels: 7 2024-03-28T17:56:26.196 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:56:25 smithi079 ceph-mon[35312]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-03-28T17:56:26.196 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:56:25 smithi079 ceph-mon[35312]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-03-28T17:56:26.196 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:56:25 smithi079 ceph-mon[35312]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-03-28T17:56:26.196 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:56:25 smithi079 ceph-mon[35312]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-03-28T17:56:26.196 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:56:25 smithi079 ceph-mon[35312]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-03-28T17:56:26.196 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:56:25 smithi079 ceph-mon[35312]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-03-28T17:56:26.196 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:56:25 smithi079 ceph-mon[35312]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-03-28T17:56:26.196 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:56:25 smithi079 ceph-mon[35312]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-03-28T17:56:26.197 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:56:25 smithi079 ceph-mon[35312]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-03-28T17:56:26.197 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:56:25 smithi079 ceph-mon[35312]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-03-28T17:56:26.197 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:56:25 smithi079 ceph-mon[35312]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-03-28T17:56:26.197 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:56:25 smithi079 ceph-mon[35312]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-03-28T17:56:26.197 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:56:25 smithi079 ceph-mon[35312]: rocksdb: Options.compression_opts.window_bits: -14 2024-03-28T17:56:26.197 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:56:25 smithi079 ceph-mon[35312]: rocksdb: Options.compression_opts.level: 32767 2024-03-28T17:56:26.197 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:56:25 smithi079 ceph-mon[35312]: rocksdb: Options.compression_opts.strategy: 0 2024-03-28T17:56:26.197 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:56:25 smithi079 ceph-mon[35312]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-03-28T17:56:26.197 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:56:25 smithi079 ceph-mon[35312]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-03-28T17:56:26.197 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:56:25 smithi079 ceph-mon[35312]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-03-28T17:56:26.197 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:56:25 smithi079 ceph-mon[35312]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-03-28T17:56:26.197 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:56:25 smithi079 ceph-mon[35312]: rocksdb: Options.compression_opts.enabled: false 2024-03-28T17:56:26.197 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:56:25 smithi079 ceph-mon[35312]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-03-28T17:56:26.197 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:56:25 smithi079 ceph-mon[35312]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-03-28T17:56:26.197 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:56:25 smithi079 ceph-mon[35312]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-03-28T17:56:26.198 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:56:25 smithi079 ceph-mon[35312]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-03-28T17:56:26.198 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:56:25 smithi079 ceph-mon[35312]: rocksdb: Options.target_file_size_base: 67108864 2024-03-28T17:56:26.198 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:56:25 smithi079 ceph-mon[35312]: rocksdb: Options.target_file_size_multiplier: 1 2024-03-28T17:56:26.198 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:56:25 smithi079 ceph-mon[35312]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-03-28T17:56:26.198 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:56:25 smithi079 ceph-mon[35312]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-03-28T17:56:26.198 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:56:25 smithi079 ceph-mon[35312]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-03-28T17:56:26.198 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:56:25 smithi079 ceph-mon[35312]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-03-28T17:56:26.198 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:56:25 smithi079 ceph-mon[35312]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-03-28T17:56:26.198 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:56:25 smithi079 ceph-mon[35312]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-03-28T17:56:26.198 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:56:25 smithi079 ceph-mon[35312]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-03-28T17:56:26.198 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:56:25 smithi079 ceph-mon[35312]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-03-28T17:56:26.198 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:56:25 smithi079 ceph-mon[35312]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-03-28T17:56:26.198 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:56:25 smithi079 ceph-mon[35312]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-03-28T17:56:26.198 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:56:25 smithi079 ceph-mon[35312]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-03-28T17:56:26.198 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:56:25 smithi079 ceph-mon[35312]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-03-28T17:56:26.199 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:56:25 smithi079 ceph-mon[35312]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-03-28T17:56:26.200 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:56:25 smithi079 ceph-mon[35312]: rocksdb: Options.arena_block_size: 1048576 2024-03-28T17:56:26.200 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:56:25 smithi079 ceph-mon[35312]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-03-28T17:56:26.200 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:56:25 smithi079 ceph-mon[35312]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-03-28T17:56:26.200 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:56:25 smithi079 ceph-mon[35312]: rocksdb: Options.disable_auto_compactions: 0 2024-03-28T17:56:26.200 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:56:25 smithi079 ceph-mon[35312]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-03-28T17:56:26.200 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:56:25 smithi079 ceph-mon[35312]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-03-28T17:56:26.200 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:56:25 smithi079 ceph-mon[35312]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-03-28T17:56:26.200 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:56:25 smithi079 ceph-mon[35312]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-03-28T17:56:26.200 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:56:25 smithi079 ceph-mon[35312]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-03-28T17:56:26.200 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:56:25 smithi079 ceph-mon[35312]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-03-28T17:56:26.200 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:56:25 smithi079 ceph-mon[35312]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-03-28T17:56:26.201 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:56:25 smithi079 ceph-mon[35312]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-03-28T17:56:26.201 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:56:25 smithi079 ceph-mon[35312]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-03-28T17:56:26.201 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:56:25 smithi079 ceph-mon[35312]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-03-28T17:56:26.201 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:56:25 smithi079 ceph-mon[35312]: rocksdb: Options.table_properties_collectors: 2024-03-28T17:56:26.201 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:56:25 smithi079 ceph-mon[35312]: rocksdb: Options.inplace_update_support: 0 2024-03-28T17:56:26.201 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:56:25 smithi079 ceph-mon[35312]: rocksdb: Options.inplace_update_num_locks: 10000 2024-03-28T17:56:26.201 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:56:25 smithi079 ceph-mon[35312]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-03-28T17:56:26.201 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:56:25 smithi079 ceph-mon[35312]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-03-28T17:56:26.201 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:56:25 smithi079 ceph-mon[35312]: rocksdb: Options.memtable_huge_page_size: 0 2024-03-28T17:56:26.201 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:56:25 smithi079 ceph-mon[35312]: rocksdb: Options.bloom_locality: 0 2024-03-28T17:56:26.201 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:56:25 smithi079 ceph-mon[35312]: rocksdb: Options.max_successive_merges: 0 2024-03-28T17:56:26.201 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:56:25 smithi079 ceph-mon[35312]: rocksdb: Options.optimize_filters_for_hits: 0 2024-03-28T17:56:26.201 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:56:25 smithi079 ceph-mon[35312]: rocksdb: Options.paranoid_file_checks: 0 2024-03-28T17:56:26.201 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:56:25 smithi079 ceph-mon[35312]: rocksdb: Options.force_consistency_checks: 1 2024-03-28T17:56:26.201 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:56:25 smithi079 ceph-mon[35312]: rocksdb: Options.report_bg_io_stats: 0 2024-03-28T17:56:26.201 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:56:25 smithi079 ceph-mon[35312]: rocksdb: Options.ttl: 2592000 2024-03-28T17:56:26.202 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:56:25 smithi079 ceph-mon[35312]: rocksdb: Options.periodic_compaction_seconds: 0 2024-03-28T17:56:26.202 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:56:25 smithi079 ceph-mon[35312]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-03-28T17:56:26.202 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:56:25 smithi079 ceph-mon[35312]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-03-28T17:56:26.202 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:56:25 smithi079 ceph-mon[35312]: rocksdb: Options.enable_blob_files: false 2024-03-28T17:56:26.202 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:56:25 smithi079 ceph-mon[35312]: rocksdb: Options.min_blob_size: 0 2024-03-28T17:56:26.202 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:56:25 smithi079 ceph-mon[35312]: rocksdb: Options.blob_file_size: 268435456 2024-03-28T17:56:26.202 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:56:25 smithi079 ceph-mon[35312]: rocksdb: Options.blob_compression_type: NoCompression 2024-03-28T17:56:26.202 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:56:25 smithi079 ceph-mon[35312]: rocksdb: Options.enable_blob_garbage_collection: false 2024-03-28T17:56:26.202 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:56:25 smithi079 ceph-mon[35312]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-03-28T17:56:26.202 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:56:25 smithi079 ceph-mon[35312]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-03-28T17:56:26.202 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:56:25 smithi079 ceph-mon[35312]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-03-28T17:56:26.202 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:56:25 smithi079 ceph-mon[35312]: rocksdb: Options.blob_file_starting_level: 0 2024-03-28T17:56:26.202 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:56:25 smithi079 ceph-mon[35312]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-03-28T17:56:26.202 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:56:25 smithi079 ceph-mon[35312]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-b/store.db/MANIFEST-000005 succeeded,manifest_file_number is 5, next_file_number is 7, last_sequence is 0, log_number is 0,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 0 2024-03-28T17:56:26.203 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:56:25 smithi079 ceph-mon[35312]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 0 2024-03-28T17:56:26.203 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:56:25 smithi079 ceph-mon[35312]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: 51186af8-dadb-445d-9ef8-277fda0efff0 2024-03-28T17:56:26.203 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:56:25 smithi079 ceph-mon[35312]: rocksdb: EVENT_LOG_v1 {"time_micros": 1711648585898968, "job": 1, "event": "recovery_started", "wal_files": [4]} 2024-03-28T17:56:26.203 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:56:25 smithi079 ceph-mon[35312]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #4 mode 2 2024-03-28T17:56:26.203 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:56:25 smithi079 ceph-mon[35312]: rocksdb: EVENT_LOG_v1 {"time_micros": 1711648585899819, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 8, "file_size": 1747, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 1, "largest_seqno": 5, "table_properties": {"data_size": 648, "index_size": 31, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 69, "raw_key_size": 115, "raw_average_key_size": 23, "raw_value_size": 526, "raw_average_value_size": 105, "num_data_blocks": 1, "num_entries": 5, "num_filter_entries": 5, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1711648585, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "51186af8-dadb-445d-9ef8-277fda0efff0", "db_session_id": "V7EREQZGW2XZ6HMVUPAU", "orig_file_number": 8, "seqno_to_time_mapping": "N/A"}} 2024-03-28T17:56:26.203 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:56:25 smithi079 ceph-mon[35312]: rocksdb: EVENT_LOG_v1 {"time_micros": 1711648585900078, "job": 1, "event": "recovery_finished"} 2024-03-28T17:56:26.203 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:56:25 smithi079 ceph-mon[35312]: rocksdb: [db/version_set.cc:5047] Creating manifest 10 2024-03-28T17:56:26.203 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:56:25 smithi079 ceph-mon[35312]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-b/store.db/000004.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-03-28T17:56:26.203 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:56:25 smithi079 ceph-mon[35312]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x55d8c41f8000 2024-03-28T17:56:26.203 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:56:25 smithi079 ceph-mon[35312]: rocksdb: DB pointer 0x55d8c41e2000 2024-03-28T17:56:26.203 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:56:25 smithi079 ceph-mon[35312]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-03-28T17:56:26.203 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:56:25 smithi079 ceph-mon[35312]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-03-28T17:56:26.203 INFO:journalctl@ceph.mon.b.smithi079.stdout: ** DB Stats ** 2024-03-28T17:56:26.203 INFO:journalctl@ceph.mon.b.smithi079.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-03-28T17:56:26.203 INFO:journalctl@ceph.mon.b.smithi079.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-03-28T17:56:26.203 INFO:journalctl@ceph.mon.b.smithi079.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-03-28T17:56:26.203 INFO:journalctl@ceph.mon.b.smithi079.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-03-28T17:56:26.204 INFO:journalctl@ceph.mon.b.smithi079.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-03-28T17:56:26.204 INFO:journalctl@ceph.mon.b.smithi079.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-03-28T17:56:26.204 INFO:journalctl@ceph.mon.b.smithi079.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-03-28T17:56:26.204 INFO:journalctl@ceph.mon.b.smithi079.stdout: 2024-03-28T17:56:26.204 INFO:journalctl@ceph.mon.b.smithi079.stdout: ** Compaction Stats [default] ** 2024-03-28T17:56:26.204 INFO:journalctl@ceph.mon.b.smithi079.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-03-28T17:56:26.204 INFO:journalctl@ceph.mon.b.smithi079.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-03-28T17:56:26.204 INFO:journalctl@ceph.mon.b.smithi079.stdout: L0 1/0 1.71 KB 0.2 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 2.1 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-03-28T17:56:26.204 INFO:journalctl@ceph.mon.b.smithi079.stdout: Sum 1/0 1.71 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 2.1 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-03-28T17:56:26.204 INFO:journalctl@ceph.mon.b.smithi079.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 2.1 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-03-28T17:56:26.204 INFO:journalctl@ceph.mon.b.smithi079.stdout: 2024-03-28T17:56:26.204 INFO:journalctl@ceph.mon.b.smithi079.stdout: ** Compaction Stats [default] ** 2024-03-28T17:56:26.204 INFO:journalctl@ceph.mon.b.smithi079.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-03-28T17:56:26.204 INFO:journalctl@ceph.mon.b.smithi079.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-03-28T17:56:26.204 INFO:journalctl@ceph.mon.b.smithi079.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 2.1 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-03-28T17:56:26.204 INFO:journalctl@ceph.mon.b.smithi079.stdout: 2024-03-28T17:56:26.205 INFO:journalctl@ceph.mon.b.smithi079.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-03-28T17:56:26.205 INFO:journalctl@ceph.mon.b.smithi079.stdout: 2024-03-28T17:56:26.205 INFO:journalctl@ceph.mon.b.smithi079.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-03-28T17:56:26.205 INFO:journalctl@ceph.mon.b.smithi079.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-03-28T17:56:26.205 INFO:journalctl@ceph.mon.b.smithi079.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-03-28T17:56:26.205 INFO:journalctl@ceph.mon.b.smithi079.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-03-28T17:56:26.205 INFO:journalctl@ceph.mon.b.smithi079.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-03-28T17:56:26.205 INFO:journalctl@ceph.mon.b.smithi079.stdout: AddFile(Keys): cumulative 0, interval 0 2024-03-28T17:56:26.205 INFO:journalctl@ceph.mon.b.smithi079.stdout: Cumulative compaction: 0.00 GB write, 0.30 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-03-28T17:56:26.205 INFO:journalctl@ceph.mon.b.smithi079.stdout: Interval compaction: 0.00 GB write, 0.30 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-03-28T17:56:26.205 INFO:journalctl@ceph.mon.b.smithi079.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-03-28T17:56:26.205 INFO:journalctl@ceph.mon.b.smithi079.stdout: Block cache BinnedLRUCache@0x55d8c4105090#2 capacity: 512.00 MB usage: 0.22 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 1.1e-05 secs_since: 0 2024-03-28T17:56:26.205 INFO:journalctl@ceph.mon.b.smithi079.stdout: Block cache entry stats(count,size,portion): FilterBlock(1,0.11 KB,2.08616e-05%) IndexBlock(1,0.11 KB,2.08616e-05%) Misc(1,0.00 KB,0%) 2024-03-28T17:56:26.205 INFO:journalctl@ceph.mon.b.smithi079.stdout: 2024-03-28T17:56:26.205 INFO:journalctl@ceph.mon.b.smithi079.stdout: ** File Read Latency Histogram By Level [default] ** 2024-03-28T17:56:26.205 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:56:25 smithi079 ceph-mon[35312]: mon.b does not exist in monmap, will attempt to join an existing cluster 2024-03-28T17:56:26.206 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:56:25 smithi079 ceph-mon[35312]: using public_addr v2:172.21.15.79:0/0 -> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] 2024-03-28T17:56:26.206 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:56:25 smithi079 ceph-mon[35312]: starting mon.b rank -1 at public addrs [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] at bind addrs [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] mon_data /var/lib/ceph/mon/ceph-b fsid fd220748-ed2b-11ee-95d0-87774f69a715 2024-03-28T17:56:26.206 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:56:25 smithi079 ceph-mon[35312]: mon.b@-1(???) e0 preinit fsid fd220748-ed2b-11ee-95d0-87774f69a715 2024-03-28T17:56:26.206 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:56:25 smithi079 bash[35293]: 7fcef29b8b90d8090703b144fb4873f845e0766a28bbf6614e93742dc2179326 2024-03-28T17:56:26.206 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:56:25 smithi079 systemd[1]: Started Ceph mon.b for fd220748-ed2b-11ee-95d0-87774f69a715. 2024-03-28T17:56:26.206 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:56:25 smithi079 ceph-mon[35312]: mon.b@-1(synchronizing).mds e1 new map 2024-03-28T17:56:26.206 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:56:25 smithi079 ceph-mon[35312]: mon.b@-1(synchronizing).mds e1 print_map 2024-03-28T17:56:26.206 INFO:journalctl@ceph.mon.b.smithi079.stdout: e1 2024-03-28T17:56:26.206 INFO:journalctl@ceph.mon.b.smithi079.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-03-28T17:56:26.206 INFO:journalctl@ceph.mon.b.smithi079.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments} 2024-03-28T17:56:26.206 INFO:journalctl@ceph.mon.b.smithi079.stdout: legacy client fscid: -1 2024-03-28T17:56:26.206 INFO:journalctl@ceph.mon.b.smithi079.stdout: 2024-03-28T17:56:26.206 INFO:journalctl@ceph.mon.b.smithi079.stdout: No filesystems configured 2024-03-28T17:56:26.206 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:56:25 smithi079 ceph-mon[35312]: mon.b@-1(synchronizing).osd e0 _set_cache_ratios kv ratio 0.25 inc ratio 0.375 full ratio 0.375 2024-03-28T17:56:26.206 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:56:25 smithi079 ceph-mon[35312]: mon.b@-1(synchronizing).osd e0 register_cache_with_pcm pcm target: 2147483648 pcm max: 1020054732 pcm min: 134217728 inc_osd_cache size: 1 2024-03-28T17:56:26.207 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:56:25 smithi079 ceph-mon[35312]: mon.b@-1(synchronizing).osd e1 e1: 0 total, 0 up, 0 in 2024-03-28T17:56:26.207 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:56:25 smithi079 ceph-mon[35312]: mon.b@-1(synchronizing).osd e2 e2: 0 total, 0 up, 0 in 2024-03-28T17:56:26.207 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:56:25 smithi079 ceph-mon[35312]: mon.b@-1(synchronizing).osd e3 e3: 0 total, 0 up, 0 in 2024-03-28T17:56:26.207 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:56:25 smithi079 ceph-mon[35312]: mon.b@-1(synchronizing).osd e4 e4: 0 total, 0 up, 0 in 2024-03-28T17:56:26.207 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:56:25 smithi079 ceph-mon[35312]: mon.b@-1(synchronizing).osd e4 crush map has features 3314932999778484224, adjusting msgr requires 2024-03-28T17:56:26.207 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:56:25 smithi079 ceph-mon[35312]: mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-03-28T17:56:26.207 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:56:25 smithi079 ceph-mon[35312]: mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-03-28T17:56:26.207 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:56:25 smithi079 ceph-mon[35312]: mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-03-28T17:56:26.207 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:56:25 smithi079 ceph-mon[35312]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-28T17:56:26.207 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:56:25 smithi079 ceph-mon[35312]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T17:56:26.207 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:56:25 smithi079 ceph-mon[35312]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T17:56:26.207 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:56:25 smithi079 ceph-mon[35312]: Added host smithi079 2024-03-28T17:56:26.207 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:56:25 smithi079 ceph-mon[35312]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T17:56:26.207 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:56:25 smithi079 ceph-mon[35312]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-28T17:56:26.207 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:56:25 smithi079 ceph-mon[35312]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T17:56:26.207 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:56:25 smithi079 ceph-mon[35312]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-28T17:56:26.208 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:56:25 smithi079 ceph-mon[35312]: from='client.14180 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-28T17:56:26.208 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:56:25 smithi079 ceph-mon[35312]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T17:56:26.208 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:56:25 smithi079 ceph-mon[35312]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-28T17:56:26.208 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:56:25 smithi079 ceph-mon[35312]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-28T17:56:26.208 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:56:25 smithi079 ceph-mon[35312]: from='client.14182 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi134", "target": ["mon-mgr", ""]}]: dispatch 2024-03-28T17:56:26.208 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:56:25 smithi079 ceph-mon[35312]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T17:56:26.208 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:56:25 smithi079 ceph-mon[35312]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T17:56:26.208 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:56:25 smithi079 ceph-mon[35312]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T17:56:26.208 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:56:25 smithi079 ceph-mon[35312]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T17:56:26.208 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:56:25 smithi079 ceph-mon[35312]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi079", "name": "osd_memory_target"}]: dispatch 2024-03-28T17:56:26.208 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:56:25 smithi079 ceph-mon[35312]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T17:56:26.208 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:56:25 smithi079 ceph-mon[35312]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T17:56:26.208 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:56:25 smithi079 ceph-mon[35312]: Updating smithi079:/etc/ceph/ceph.conf 2024-03-28T17:56:26.208 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:56:25 smithi079 ceph-mon[35312]: Deploying cephadm binary to smithi134 2024-03-28T17:56:26.209 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:56:25 smithi079 ceph-mon[35312]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-28T17:56:26.209 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:56:25 smithi079 ceph-mon[35312]: Updating smithi079:/var/lib/ceph/fd220748-ed2b-11ee-95d0-87774f69a715/config/ceph.conf 2024-03-28T17:56:26.209 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:56:25 smithi079 ceph-mon[35312]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-28T17:56:26.209 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:56:25 smithi079 ceph-mon[35312]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-28T17:56:26.209 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:56:25 smithi079 ceph-mon[35312]: Updating smithi079:/etc/ceph/ceph.client.admin.keyring 2024-03-28T17:56:26.209 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:56:25 smithi079 ceph-mon[35312]: Updating smithi079:/var/lib/ceph/fd220748-ed2b-11ee-95d0-87774f69a715/config/ceph.client.admin.keyring 2024-03-28T17:56:26.209 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:56:25 smithi079 ceph-mon[35312]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T17:56:26.209 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:56:25 smithi079 ceph-mon[35312]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T17:56:26.209 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:56:25 smithi079 ceph-mon[35312]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T17:56:26.209 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:56:25 smithi079 ceph-mon[35312]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T17:56:26.209 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:56:25 smithi079 ceph-mon[35312]: Added host smithi134 2024-03-28T17:56:26.209 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:56:25 smithi079 ceph-mon[35312]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T17:56:26.209 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:56:25 smithi079 ceph-mon[35312]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-28T17:56:26.209 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:56:25 smithi079 ceph-mon[35312]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T17:56:26.210 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:56:25 smithi079 ceph-mon[35312]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T17:56:26.210 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:56:25 smithi079 ceph-mon[35312]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-28T17:56:26.210 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:56:25 smithi079 ceph-mon[35312]: from='client.14184 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-28T17:56:26.210 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:56:25 smithi079 ceph-mon[35312]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T17:56:26.210 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:56:25 smithi079 ceph-mon[35312]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-28T17:56:26.210 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:56:25 smithi079 ceph-mon[35312]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-28T17:56:26.210 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:56:25 smithi079 ceph-mon[35312]: from='client.? 172.21.15.22:0/56783544' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2024-03-28T17:56:26.210 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:56:25 smithi079 ceph-mon[35312]: from='client.? 172.21.15.22:0/56783544' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-03-28T17:56:26.210 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:56:25 smithi079 ceph-mon[35312]: osdmap e4: 0 total, 0 up, 0 in 2024-03-28T17:56:26.210 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:56:25 smithi079 ceph-mon[35312]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-28T17:56:26.210 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:56:25 smithi079 ceph-mon[35312]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T17:56:26.210 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:56:25 smithi079 ceph-mon[35312]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T17:56:26.210 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:56:25 smithi079 ceph-mon[35312]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T17:56:26.210 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:56:25 smithi079 ceph-mon[35312]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T17:56:26.211 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:56:25 smithi079 ceph-mon[35312]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi134", "name": "osd_memory_target"}]: dispatch 2024-03-28T17:56:26.211 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:56:25 smithi079 ceph-mon[35312]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T17:56:26.211 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:56:25 smithi079 ceph-mon[35312]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T17:56:26.211 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:56:25 smithi079 ceph-mon[35312]: Updating smithi134:/etc/ceph/ceph.conf 2024-03-28T17:56:26.211 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:56:25 smithi079 ceph-mon[35312]: Updating smithi134:/var/lib/ceph/fd220748-ed2b-11ee-95d0-87774f69a715/config/ceph.conf 2024-03-28T17:56:26.211 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:56:25 smithi079 ceph-mon[35312]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-28T17:56:26.211 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:56:25 smithi079 ceph-mon[35312]: Updating smithi134:/etc/ceph/ceph.client.admin.keyring 2024-03-28T17:56:26.211 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:56:25 smithi079 ceph-mon[35312]: Updating smithi134:/var/lib/ceph/fd220748-ed2b-11ee-95d0-87774f69a715/config/ceph.client.admin.keyring 2024-03-28T17:56:26.211 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:56:25 smithi079 ceph-mon[35312]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T17:56:26.211 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:56:25 smithi079 ceph-mon[35312]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T17:56:26.211 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:56:25 smithi079 ceph-mon[35312]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T17:56:26.211 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:56:25 smithi079 ceph-mon[35312]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T17:56:26.211 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:56:25 smithi079 ceph-mon[35312]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-28T17:56:26.211 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:56:25 smithi079 ceph-mon[35312]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T17:56:26.211 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:56:25 smithi079 ceph-mon[35312]: from='client.14188 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi022:172.21.15.22=a;smithi079:172.21.15.79=b;smithi134:172.21.15.134=c", "target": ["mon-mgr", ""]}]: dispatch 2024-03-28T17:56:26.211 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:56:25 smithi079 ceph-mon[35312]: Saving service mon spec with placement smithi022:172.21.15.22=a;smithi079:172.21.15.79=b;smithi134:172.21.15.134=c;count:3 2024-03-28T17:56:26.212 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:56:25 smithi079 ceph-mon[35312]: Deploying daemon mon.c on smithi134 2024-03-28T17:56:26.212 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:56:25 smithi079 ceph-mon[35312]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-28T17:56:26.212 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:56:25 smithi079 ceph-mon[35312]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-28T17:56:26.212 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:56:25 smithi079 ceph-mon[35312]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-28T17:56:26.212 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:56:25 smithi079 ceph-mon[35312]: from='client.? 172.21.15.134:0/4267932649' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-03-28T17:56:26.212 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:56:25 smithi079 ceph-mon[35312]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-28T17:56:26.212 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:56:25 smithi079 ceph-mon[35312]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-28T17:56:26.212 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:56:25 smithi079 ceph-mon[35312]: mon.a calling monitor election 2024-03-28T17:56:26.212 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:56:25 smithi079 ceph-mon[35312]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-28T17:56:26.212 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:56:25 smithi079 ceph-mon[35312]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-28T17:56:26.212 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:56:25 smithi079 ceph-mon[35312]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-28T17:56:26.212 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:56:25 smithi079 ceph-mon[35312]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-28T17:56:26.212 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:56:25 smithi079 ceph-mon[35312]: mon.c calling monitor election 2024-03-28T17:56:26.212 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:56:25 smithi079 ceph-mon[35312]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-28T17:56:26.212 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:56:25 smithi079 ceph-mon[35312]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-28T17:56:26.213 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:56:25 smithi079 ceph-mon[35312]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-28T17:56:26.213 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:56:25 smithi079 ceph-mon[35312]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-28T17:56:26.213 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:56:25 smithi079 ceph-mon[35312]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-03-28T17:56:26.213 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:56:25 smithi079 ceph-mon[35312]: monmap e2: 2 mons at {a=[v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0],c=[v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-03-28T17:56:26.213 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:56:25 smithi079 ceph-mon[35312]: fsmap 2024-03-28T17:56:26.213 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:56:25 smithi079 ceph-mon[35312]: osdmap e4: 0 total, 0 up, 0 in 2024-03-28T17:56:26.213 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:56:25 smithi079 ceph-mon[35312]: mgrmap e13: x(active, since 69s) 2024-03-28T17:56:26.213 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:56:25 smithi079 ceph-mon[35312]: overall HEALTH_OK 2024-03-28T17:56:26.213 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:56:25 smithi079 ceph-mon[35312]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T17:56:26.213 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:56:25 smithi079 ceph-mon[35312]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T17:56:26.213 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:56:25 smithi079 ceph-mon[35312]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T17:56:26.213 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:56:25 smithi079 ceph-mon[35312]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-28T17:56:26.214 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:56:25 smithi079 ceph-mon[35312]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T17:56:26.214 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:56:25 smithi079 ceph-mon[35312]: Deploying daemon mon.b on smithi079 2024-03-28T17:56:26.214 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:56:25 smithi079 ceph-mon[35312]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-28T17:56:26.214 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:56:25 smithi079 ceph-mon[35312]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-28T17:56:26.214 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:56:25 smithi079 ceph-mon[35312]: from='client.? 172.21.15.134:0/1304427795' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-03-28T17:56:26.214 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:56:25 smithi079 ceph-mon[35312]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-28T17:56:26.214 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:56:25 smithi079 ceph-mon[35312]: mon.b@-1(synchronizing).paxosservice(auth 1..4) refresh upgraded, format 0 -> 3 2024-03-28T17:56:27.314 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-03-28T17:56:27.314 DEBUG:teuthology.orchestra.run.smithi134:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fd220748-ed2b-11ee-95d0-87774f69a715 -- ceph mon dump -f json 2024-03-28T17:56:27.736 INFO:teuthology.orchestra.run.smithi134.stderr:Inferring config /var/lib/ceph/fd220748-ed2b-11ee-95d0-87774f69a715/mon.c/config 2024-03-28T17:56:31.283 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:56:31 smithi022 ceph-mon[31985]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-28T17:56:31.283 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:56:31 smithi022 ceph-mon[31985]: mon.a calling monitor election 2024-03-28T17:56:31.283 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:56:31 smithi022 ceph-mon[31985]: mon.c calling monitor election 2024-03-28T17:56:31.283 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:56:31 smithi022 ceph-mon[31985]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-28T17:56:31.283 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:56:31 smithi022 ceph-mon[31985]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-28T17:56:31.283 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:56:31 smithi022 ceph-mon[31985]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-28T17:56:31.283 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:56:31 smithi022 ceph-mon[31985]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-28T17:56:31.283 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:56:31 smithi022 ceph-mon[31985]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-28T17:56:31.283 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:56:31 smithi022 ceph-mon[31985]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-28T17:56:31.283 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:56:31 smithi022 ceph-mon[31985]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-28T17:56:31.284 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:56:31 smithi022 ceph-mon[31985]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-28T17:56:31.284 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:56:31 smithi022 ceph-mon[31985]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-28T17:56:31.284 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:56:31 smithi022 ceph-mon[31985]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-03-28T17:56:31.284 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:56:31 smithi022 ceph-mon[31985]: monmap e3: 3 mons at {a=[v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0],b=[v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0],c=[v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-03-28T17:56:31.284 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:56:31 smithi022 ceph-mon[31985]: fsmap 2024-03-28T17:56:31.284 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:56:31 smithi022 ceph-mon[31985]: osdmap e4: 0 total, 0 up, 0 in 2024-03-28T17:56:31.284 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:56:31 smithi022 ceph-mon[31985]: mgrmap e13: x(active, since 80s) 2024-03-28T17:56:31.284 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:56:31 smithi022 ceph-mon[31985]: Health check failed: 1/3 mons down, quorum a,c (MON_DOWN) 2024-03-28T17:56:31.284 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:56:31 smithi022 ceph-mon[31985]: Health detail: HEALTH_WARN 1/3 mons down, quorum a,c 2024-03-28T17:56:31.284 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:56:31 smithi022 ceph-mon[31985]: [WRN] MON_DOWN: 1/3 mons down, quorum a,c 2024-03-28T17:56:31.284 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:56:31 smithi022 ceph-mon[31985]: mon.b (rank 2) addr [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] is down (out of quorum) 2024-03-28T17:56:31.284 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:56:31 smithi022 ceph-mon[31985]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T17:56:31.284 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:56:31 smithi022 ceph-mon[31985]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T17:56:31.425 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:56:31 smithi134 ceph-mon[35037]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-28T17:56:31.425 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:56:31 smithi134 ceph-mon[35037]: mon.a calling monitor election 2024-03-28T17:56:31.425 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:56:31 smithi134 ceph-mon[35037]: mon.c calling monitor election 2024-03-28T17:56:31.425 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:56:31 smithi134 ceph-mon[35037]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-28T17:56:31.425 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:56:31 smithi134 ceph-mon[35037]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-28T17:56:31.425 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:56:31 smithi134 ceph-mon[35037]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-28T17:56:31.425 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:56:31 smithi134 ceph-mon[35037]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-28T17:56:31.425 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:56:31 smithi134 ceph-mon[35037]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-28T17:56:31.425 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:56:31 smithi134 ceph-mon[35037]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-28T17:56:31.426 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:56:31 smithi134 ceph-mon[35037]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-28T17:56:31.426 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:56:31 smithi134 ceph-mon[35037]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-28T17:56:31.426 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:56:31 smithi134 ceph-mon[35037]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-28T17:56:31.426 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:56:31 smithi134 ceph-mon[35037]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-03-28T17:56:31.426 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:56:31 smithi134 ceph-mon[35037]: monmap e3: 3 mons at {a=[v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0],b=[v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0],c=[v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-03-28T17:56:31.426 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:56:31 smithi134 ceph-mon[35037]: fsmap 2024-03-28T17:56:31.426 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:56:31 smithi134 ceph-mon[35037]: osdmap e4: 0 total, 0 up, 0 in 2024-03-28T17:56:31.426 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:56:31 smithi134 ceph-mon[35037]: mgrmap e13: x(active, since 80s) 2024-03-28T17:56:31.426 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:56:31 smithi134 ceph-mon[35037]: Health check failed: 1/3 mons down, quorum a,c (MON_DOWN) 2024-03-28T17:56:31.426 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:56:31 smithi134 ceph-mon[35037]: Health detail: HEALTH_WARN 1/3 mons down, quorum a,c 2024-03-28T17:56:31.426 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:56:31 smithi134 ceph-mon[35037]: [WRN] MON_DOWN: 1/3 mons down, quorum a,c 2024-03-28T17:56:31.426 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:56:31 smithi134 ceph-mon[35037]: mon.b (rank 2) addr [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] is down (out of quorum) 2024-03-28T17:56:31.426 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:56:31 smithi134 ceph-mon[35037]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T17:56:31.426 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:56:31 smithi134 ceph-mon[35037]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T17:56:32.424 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:56:32 smithi134 ceph-mon[35037]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-28T17:56:32.425 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:56:32 smithi134 ceph-mon[35037]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T17:56:32.425 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:56:32 smithi134 ceph-mon[35037]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T17:56:32.425 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:56:32 smithi134 ceph-mon[35037]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T17:56:32.425 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:56:32 smithi134 ceph-mon[35037]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-28T17:56:32.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:56:32 smithi022 ceph-mon[31985]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-28T17:56:32.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:56:32 smithi022 ceph-mon[31985]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T17:56:32.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:56:32 smithi022 ceph-mon[31985]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T17:56:32.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:56:32 smithi022 ceph-mon[31985]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T17:56:32.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:56:32 smithi022 ceph-mon[31985]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-28T17:56:32.545 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-28T17:56:32.544+0000 7f22ea600700 1 -- 172.21.15.134:0/199926281 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f22e4101ef0 msgr2=0x7f22c8005610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:56:32.545 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-28T17:56:32.544+0000 7f22ea600700 1 --2- 172.21.15.134:0/199926281 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f22e4101ef0 0x7f22c8005610 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f22d4005fa0 tx=0x7f22d400ffc0 comp rx=0 tx=0).stop 2024-03-28T17:56:32.545 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-28T17:56:32.544+0000 7f22ea600700 1 -- 172.21.15.134:0/199926281 shutdown_connections 2024-03-28T17:56:32.545 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-28T17:56:32.544+0000 7f22ea600700 1 --2- 172.21.15.134:0/199926281 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f22e4101ef0 0x7f22c8005610 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:56:32.545 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-28T17:56:32.544+0000 7f22ea600700 1 -- 172.21.15.134:0/199926281 >> 172.21.15.134:0/199926281 conn(0x7f22e40fd720 msgr2=0x7f22e40ffb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T17:56:32.545 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-28T17:56:32.544+0000 7f22ea600700 1 -- 172.21.15.134:0/199926281 shutdown_connections 2024-03-28T17:56:32.545 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-28T17:56:32.544+0000 7f22ea600700 1 -- 172.21.15.134:0/199926281 wait complete. 2024-03-28T17:56:32.546 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-28T17:56:32.546+0000 7f22ea600700 1 Processor -- start 2024-03-28T17:56:32.546 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-28T17:56:32.546+0000 7f22ea600700 1 -- start start 2024-03-28T17:56:32.546 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-28T17:56:32.546+0000 7f22ea600700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f22e4101ef0 0x7f22e4074490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:56:32.546 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-28T17:56:32.546+0000 7f22ea600700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f22e40749d0 0x7f22e4074e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:56:32.547 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-28T17:56:32.546+0000 7f22ea600700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f22e41a3a40 0x7f22e41a5e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:56:32.547 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-28T17:56:32.546+0000 7f22ea600700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22e4109cc0 con 0x7f22e40749d0 2024-03-28T17:56:32.547 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-28T17:56:32.546+0000 7f22ea600700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22e4109b40 con 0x7f22e4101ef0 2024-03-28T17:56:32.547 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-28T17:56:32.546+0000 7f22ea600700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22e4109e40 con 0x7f22e41a3a40 2024-03-28T17:56:32.548 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-28T17:56:32.547+0000 7f22e3e00700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f22e41a3a40 0x7f22e41a5e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:56:32.548 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-28T17:56:32.547+0000 7f22e3400700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f22e4101ef0 0x7f22e4074490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:56:32.548 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-28T17:56:32.547+0000 7f22e3400700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f22e4101ef0 0x7f22e4074490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.134:47626/0 (socket says 172.21.15.134:47626) 2024-03-28T17:56:32.548 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-28T17:56:32.547+0000 7f22e3400700 1 -- 172.21.15.134:0/3133284110 learned_addr learned my addr 172.21.15.134:0/3133284110 (peer_addr_for_me v2:172.21.15.134:0/0) 2024-03-28T17:56:32.548 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-28T17:56:32.547+0000 7f22e2a00700 1 --2- 172.21.15.134:0/3133284110 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f22e40749d0 0x7f22e4074e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:56:32.548 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-28T17:56:32.547+0000 7f22e3400700 1 -- 172.21.15.134:0/3133284110 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f22e4101ef0 msgr2=0x7f22e4074490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 12 2024-03-28T17:56:32.548 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-28T17:56:32.547+0000 7f22e3400700 1 -- 172.21.15.134:0/3133284110 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f22e4101ef0 msgr2=0x7f22e4074490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-03-28T17:56:32.548 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-28T17:56:32.547+0000 7f22e3400700 1 --2- 172.21.15.134:0/3133284110 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f22e4101ef0 0x7f22e4074490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-28T17:56:32.548 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-28T17:56:32.547+0000 7f22e3400700 1 --2- 172.21.15.134:0/3133284110 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f22e4101ef0 0x7f22e4074490 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-28T17:56:32.548 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-28T17:56:32.547+0000 7f22e3e00700 1 -- 172.21.15.134:0/3133284110 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f22e4101ef0 msgr2=0x7f22e4074490 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-28T17:56:32.548 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-28T17:56:32.547+0000 7f22e3e00700 1 --2- 172.21.15.134:0/3133284110 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f22e4101ef0 0x7f22e4074490 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:56:32.549 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-28T17:56:32.547+0000 7f22e3e00700 1 -- 172.21.15.134:0/3133284110 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f22e40749d0 msgr2=0x7f22e4074e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:56:32.549 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-28T17:56:32.547+0000 7f22e3e00700 1 --2- 172.21.15.134:0/3133284110 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f22e40749d0 0x7f22e4074e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:56:32.549 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-28T17:56:32.548+0000 7f22e3e00700 1 -- 172.21.15.134:0/3133284110 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f22e41a6560 con 0x7f22e41a3a40 2024-03-28T17:56:32.549 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-28T17:56:32.548+0000 7f22e3e00700 1 --2- 172.21.15.134:0/3133284110 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f22e41a3a40 0x7f22e41a5e30 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f22dc00bd50 tx=0x7f22dc00bd80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T17:56:32.549 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-28T17:56:32.548+0000 7f22dbe00700 1 -- 172.21.15.134:0/3133284110 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22dc019070 con 0x7f22e41a3a40 2024-03-28T17:56:32.549 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-28T17:56:32.548+0000 7f22dbe00700 1 -- 172.21.15.134:0/3133284110 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f22dc00cc10 con 0x7f22e41a3a40 2024-03-28T17:56:32.549 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-28T17:56:32.548+0000 7f22dbe00700 1 -- 172.21.15.134:0/3133284110 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22dc0126b0 con 0x7f22e41a3a40 2024-03-28T17:56:32.549 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-28T17:56:32.548+0000 7f22ea600700 1 -- 172.21.15.134:0/3133284110 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f22e41a6850 con 0x7f22e41a3a40 2024-03-28T17:56:32.550 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-28T17:56:32.548+0000 7f22ea600700 1 -- 172.21.15.134:0/3133284110 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f22e41a6d40 con 0x7f22e41a3a40 2024-03-28T17:56:32.550 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-28T17:56:32.550+0000 7f22dbe00700 1 -- 172.21.15.134:0/3133284110 <== mon.1 v2:172.21.15.134:3300/0 4 ==== mgrmap(e 13) v1 ==== 46162+0+0 (secure 0 0 0) 0x7f22dc01c070 con 0x7f22e41a3a40 2024-03-28T17:56:32.550 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-28T17:56:32.550+0000 7f22ea600700 1 -- 172.21.15.134:0/3133284110 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f22e40637c0 con 0x7f22e41a3a40 2024-03-28T17:56:32.550 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-28T17:56:32.551+0000 7f22dbe00700 1 --2- 172.21.15.134:0/3133284110 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7f22c4039450 0x7f22c403b910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:56:32.551 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-28T17:56:32.551+0000 7f22dbe00700 1 -- 172.21.15.134:0/3133284110 <== mon.1 v2:172.21.15.134:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f22dc04b720 con 0x7f22e41a3a40 2024-03-28T17:56:32.551 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-28T17:56:32.551+0000 7f22e3400700 1 --2- 172.21.15.134:0/3133284110 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7f22c4039450 0x7f22c403b910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:56:32.552 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-28T17:56:32.552+0000 7f22e3400700 1 --2- 172.21.15.134:0/3133284110 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7f22c4039450 0x7f22c403b910 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f22d400fb20 tx=0x7f22d40058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T17:56:32.556 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-28T17:56:32.556+0000 7f22dbe00700 1 -- 172.21.15.134:0/3133284110 <== mon.1 v2:172.21.15.134:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f22dc014a10 con 0x7f22e41a3a40 2024-03-28T17:56:32.871 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-28T17:56:32.871+0000 7f22ea600700 1 -- 172.21.15.134:0/3133284110 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f22e402dc90 con 0x7f22e41a3a40 2024-03-28T17:56:32.872 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-28T17:56:32.871+0000 7f22dbe00700 1 -- 172.21.15.134:0/3133284110 <== mon.1 v2:172.21.15.134:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 3 v3) v1 ==== 95+0+1279 (secure 0 0 0) 0x7f22dc022170 con 0x7f22e41a3a40 2024-03-28T17:56:32.872 INFO:teuthology.orchestra.run.smithi134.stdout: 2024-03-28T17:56:32.872 INFO:teuthology.orchestra.run.smithi134.stdout:{"epoch":3,"fsid":"fd220748-ed2b-11ee-95d0-87774f69a715","modified":"2024-03-28T17:56:25.947374Z","created":"2024-03-28T17:53:56.145153Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:3300","nonce":0},{"type":"v1","addr":"172.21.15.22:6789","nonce":0}]},"addr":"172.21.15.22:6789/0","public_addr":"172.21.15.22:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3300","nonce":0},{"type":"v1","addr":"172.21.15.134:6789","nonce":0}]},"addr":"172.21.15.134:6789/0","public_addr":"172.21.15.134:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:3300","nonce":0},{"type":"v1","addr":"172.21.15.79:6789","nonce":0}]},"addr":"172.21.15.79:6789/0","public_addr":"172.21.15.79:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-03-28T17:56:32.874 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-28T17:56:32.874+0000 7f22ea600700 1 -- 172.21.15.134:0/3133284110 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7f22c4039450 msgr2=0x7f22c403b910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:56:32.875 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-28T17:56:32.874+0000 7f22ea600700 1 --2- 172.21.15.134:0/3133284110 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7f22c4039450 0x7f22c403b910 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f22d400fb20 tx=0x7f22d40058e0 comp rx=0 tx=0).stop 2024-03-28T17:56:32.875 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-28T17:56:32.874+0000 7f22ea600700 1 -- 172.21.15.134:0/3133284110 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f22e41a3a40 msgr2=0x7f22e41a5e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:56:32.875 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-28T17:56:32.874+0000 7f22ea600700 1 --2- 172.21.15.134:0/3133284110 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f22e41a3a40 0x7f22e41a5e30 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f22dc00bd50 tx=0x7f22dc00bd80 comp rx=0 tx=0).stop 2024-03-28T17:56:32.875 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-28T17:56:32.874+0000 7f22ea600700 1 -- 172.21.15.134:0/3133284110 shutdown_connections 2024-03-28T17:56:32.875 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-28T17:56:32.874+0000 7f22ea600700 1 --2- 172.21.15.134:0/3133284110 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7f22c4039450 0x7f22c403b910 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:56:32.875 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-28T17:56:32.874+0000 7f22ea600700 1 --2- 172.21.15.134:0/3133284110 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f22e41a3a40 0x7f22e41a5e30 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:56:32.875 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-28T17:56:32.874+0000 7f22ea600700 1 --2- 172.21.15.134:0/3133284110 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f22e4101ef0 0x7f22e4074490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:56:32.875 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-28T17:56:32.874+0000 7f22ea600700 1 --2- 172.21.15.134:0/3133284110 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f22e40749d0 0x7f22e4074e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:56:32.875 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-28T17:56:32.874+0000 7f22ea600700 1 -- 172.21.15.134:0/3133284110 >> 172.21.15.134:0/3133284110 conn(0x7f22e40fd720 msgr2=0x7f22e4100b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T17:56:32.875 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-28T17:56:32.874+0000 7f22ea600700 1 -- 172.21.15.134:0/3133284110 shutdown_connections 2024-03-28T17:56:32.875 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-28T17:56:32.874+0000 7f22ea600700 1 -- 172.21.15.134:0/3133284110 wait complete. 2024-03-28T17:56:32.876 INFO:teuthology.orchestra.run.smithi134.stderr:dumped monmap epoch 3 2024-03-28T17:56:33.054 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:56:32 smithi079 ceph-mon[35312]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-28T17:56:33.054 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:56:32 smithi079 ceph-mon[35312]: mon.a calling monitor election 2024-03-28T17:56:33.054 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:56:32 smithi079 ceph-mon[35312]: mon.c calling monitor election 2024-03-28T17:56:33.054 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:56:32 smithi079 ceph-mon[35312]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-28T17:56:33.055 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:56:32 smithi079 ceph-mon[35312]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-28T17:56:33.055 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:56:32 smithi079 ceph-mon[35312]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-28T17:56:33.055 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:56:32 smithi079 ceph-mon[35312]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-28T17:56:33.055 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:56:32 smithi079 ceph-mon[35312]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-28T17:56:33.055 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:56:32 smithi079 ceph-mon[35312]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-28T17:56:33.055 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:56:32 smithi079 ceph-mon[35312]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-28T17:56:33.055 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:56:32 smithi079 ceph-mon[35312]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-28T17:56:33.055 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:56:32 smithi079 ceph-mon[35312]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-28T17:56:33.055 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:56:32 smithi079 ceph-mon[35312]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-03-28T17:56:33.055 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:56:32 smithi079 ceph-mon[35312]: monmap e3: 3 mons at {a=[v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0],b=[v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0],c=[v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-03-28T17:56:33.055 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:56:32 smithi079 ceph-mon[35312]: fsmap 2024-03-28T17:56:33.055 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:56:32 smithi079 ceph-mon[35312]: osdmap e4: 0 total, 0 up, 0 in 2024-03-28T17:56:33.055 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:56:32 smithi079 ceph-mon[35312]: mgrmap e13: x(active, since 80s) 2024-03-28T17:56:33.055 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:56:32 smithi079 ceph-mon[35312]: Health check failed: 1/3 mons down, quorum a,c (MON_DOWN) 2024-03-28T17:56:33.055 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:56:32 smithi079 ceph-mon[35312]: Health detail: HEALTH_WARN 1/3 mons down, quorum a,c 2024-03-28T17:56:33.055 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:56:32 smithi079 ceph-mon[35312]: [WRN] MON_DOWN: 1/3 mons down, quorum a,c 2024-03-28T17:56:33.056 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:56:32 smithi079 ceph-mon[35312]: mon.b (rank 2) addr [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] is down (out of quorum) 2024-03-28T17:56:33.056 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:56:32 smithi079 ceph-mon[35312]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T17:56:33.056 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:56:32 smithi079 ceph-mon[35312]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T17:56:33.056 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:56:32 smithi079 ceph-mon[35312]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-28T17:56:33.056 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:56:32 smithi079 ceph-mon[35312]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T17:56:33.056 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:56:32 smithi079 ceph-mon[35312]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T17:56:33.056 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:56:32 smithi079 ceph-mon[35312]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T17:56:33.056 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:56:32 smithi079 ceph-mon[35312]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-28T17:56:33.056 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:56:33 smithi079 ceph-mon[35312]: mon.b calling monitor election 2024-03-28T17:56:33.056 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:56:33 smithi079 ceph-mon[35312]: mon.b calling monitor election 2024-03-28T17:56:33.056 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:56:33 smithi079 ceph-mon[35312]: mon.c calling monitor election 2024-03-28T17:56:33.056 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:56:33 smithi079 ceph-mon[35312]: mon.a calling monitor election 2024-03-28T17:56:33.056 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:56:33 smithi079 ceph-mon[35312]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-03-28T17:56:33.056 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:56:33 smithi079 ceph-mon[35312]: monmap e3: 3 mons at {a=[v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0],b=[v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0],c=[v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-03-28T17:56:33.056 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:56:33 smithi079 ceph-mon[35312]: fsmap 2024-03-28T17:56:33.332 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:56:33 smithi079 ceph-mon[35312]: osdmap e4: 0 total, 0 up, 0 in 2024-03-28T17:56:33.332 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:56:33 smithi079 ceph-mon[35312]: mgrmap e13: x(active, since 82s) 2024-03-28T17:56:33.332 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:56:33 smithi079 ceph-mon[35312]: Health check cleared: MON_DOWN (was: 1/3 mons down, quorum a,c) 2024-03-28T17:56:33.332 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:56:33 smithi079 ceph-mon[35312]: Cluster is now healthy 2024-03-28T17:56:33.332 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:56:33 smithi079 ceph-mon[35312]: overall HEALTH_OK 2024-03-28T17:56:33.424 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:56:33 smithi134 ceph-mon[35037]: mon.b calling monitor election 2024-03-28T17:56:33.425 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:56:33 smithi134 ceph-mon[35037]: mon.b calling monitor election 2024-03-28T17:56:33.425 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:56:33 smithi134 ceph-mon[35037]: mon.c calling monitor election 2024-03-28T17:56:33.425 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:56:33 smithi134 ceph-mon[35037]: mon.a calling monitor election 2024-03-28T17:56:33.425 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:56:33 smithi134 ceph-mon[35037]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-03-28T17:56:33.425 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:56:33 smithi134 ceph-mon[35037]: monmap e3: 3 mons at {a=[v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0],b=[v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0],c=[v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-03-28T17:56:33.425 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:56:33 smithi134 ceph-mon[35037]: fsmap 2024-03-28T17:56:33.425 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:56:33 smithi134 ceph-mon[35037]: osdmap e4: 0 total, 0 up, 0 in 2024-03-28T17:56:33.425 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:56:33 smithi134 ceph-mon[35037]: mgrmap e13: x(active, since 82s) 2024-03-28T17:56:33.425 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:56:33 smithi134 ceph-mon[35037]: Health check cleared: MON_DOWN (was: 1/3 mons down, quorum a,c) 2024-03-28T17:56:33.425 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:56:33 smithi134 ceph-mon[35037]: Cluster is now healthy 2024-03-28T17:56:33.425 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:56:33 smithi134 ceph-mon[35037]: overall HEALTH_OK 2024-03-28T17:56:33.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:56:33 smithi022 ceph-mon[31985]: mon.b calling monitor election 2024-03-28T17:56:33.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:56:33 smithi022 ceph-mon[31985]: mon.b calling monitor election 2024-03-28T17:56:33.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:56:33 smithi022 ceph-mon[31985]: mon.c calling monitor election 2024-03-28T17:56:33.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:56:33 smithi022 ceph-mon[31985]: mon.a calling monitor election 2024-03-28T17:56:33.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:56:33 smithi022 ceph-mon[31985]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-03-28T17:56:33.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:56:33 smithi022 ceph-mon[31985]: monmap e3: 3 mons at {a=[v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0],b=[v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0],c=[v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-03-28T17:56:33.533 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:56:33 smithi022 ceph-mon[31985]: fsmap 2024-03-28T17:56:33.533 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:56:33 smithi022 ceph-mon[31985]: osdmap e4: 0 total, 0 up, 0 in 2024-03-28T17:56:33.533 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:56:33 smithi022 ceph-mon[31985]: mgrmap e13: x(active, since 82s) 2024-03-28T17:56:33.533 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:56:33 smithi022 ceph-mon[31985]: Health check cleared: MON_DOWN (was: 1/3 mons down, quorum a,c) 2024-03-28T17:56:33.533 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:56:33 smithi022 ceph-mon[31985]: Cluster is now healthy 2024-03-28T17:56:33.533 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:56:33 smithi022 ceph-mon[31985]: overall HEALTH_OK 2024-03-28T17:56:34.076 INFO:tasks.cephadm:Generating final ceph.conf file... 2024-03-28T17:56:34.076 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fd220748-ed2b-11ee-95d0-87774f69a715 -- ceph config generate-minimal-conf 2024-03-28T17:56:34.521 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/fd220748-ed2b-11ee-95d0-87774f69a715/mon.a/config 2024-03-28T17:56:34.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:56:34 smithi022 ceph-mon[31985]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-28T17:56:34.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:56:34 smithi022 ceph-mon[31985]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T17:56:34.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:56:34 smithi022 ceph-mon[31985]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T17:56:34.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:56:34 smithi022 ceph-mon[31985]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T17:56:34.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:56:34 smithi022 ceph-mon[31985]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T17:56:34.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:56:34 smithi022 ceph-mon[31985]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T17:56:34.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:56:34 smithi022 ceph-mon[31985]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T17:56:34.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:56:34 smithi022 ceph-mon[31985]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-28T17:56:34.674 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:56:34 smithi134 ceph-mon[35037]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-28T17:56:34.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:56:34 smithi134 ceph-mon[35037]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T17:56:34.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:56:34 smithi134 ceph-mon[35037]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T17:56:34.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:56:34 smithi134 ceph-mon[35037]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T17:56:34.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:56:34 smithi134 ceph-mon[35037]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T17:56:34.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:56:34 smithi134 ceph-mon[35037]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T17:56:34.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:56:34 smithi134 ceph-mon[35037]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T17:56:34.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:56:34 smithi134 ceph-mon[35037]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-28T17:56:34.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:56:34 smithi079 ceph-mon[35312]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-28T17:56:34.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:56:34 smithi079 ceph-mon[35312]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T17:56:34.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:56:34 smithi079 ceph-mon[35312]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T17:56:34.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:56:34 smithi079 ceph-mon[35312]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T17:56:34.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:56:34 smithi079 ceph-mon[35312]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T17:56:34.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:56:34 smithi079 ceph-mon[35312]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T17:56:34.683 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:56:34 smithi079 ceph-mon[35312]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T17:56:34.683 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:56:34 smithi079 ceph-mon[35312]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-28T17:56:35.264 INFO:journalctl@ceph.mgr.x.smithi022.stdout:Mar 28 17:56:34 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mgr-x[32211]: 2024-03-28T17:56:34.947+0000 7fd84ee00700 -1 mgr.server handle_report got status from non-daemon mon.b 2024-03-28T17:56:35.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:56:35 smithi022 ceph-mon[31985]: Updating smithi022:/etc/ceph/ceph.conf 2024-03-28T17:56:35.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:56:35 smithi022 ceph-mon[31985]: Updating smithi079:/etc/ceph/ceph.conf 2024-03-28T17:56:35.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:56:35 smithi022 ceph-mon[31985]: Updating smithi134:/etc/ceph/ceph.conf 2024-03-28T17:56:35.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:56:35 smithi022 ceph-mon[31985]: Updating smithi079:/var/lib/ceph/fd220748-ed2b-11ee-95d0-87774f69a715/config/ceph.conf 2024-03-28T17:56:35.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:56:35 smithi022 ceph-mon[31985]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T17:56:35.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:56:35 smithi022 ceph-mon[31985]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T17:56:35.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:56:35 smithi022 ceph-mon[31985]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T17:56:35.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:56:35 smithi022 ceph-mon[31985]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T17:56:35.533 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:56:35 smithi022 ceph-mon[31985]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T17:56:35.533 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:56:35 smithi022 ceph-mon[31985]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T17:56:35.533 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:56:35 smithi022 ceph-mon[31985]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T17:56:35.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:56:35 smithi134 ceph-mon[35037]: Updating smithi022:/etc/ceph/ceph.conf 2024-03-28T17:56:35.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:56:35 smithi134 ceph-mon[35037]: Updating smithi079:/etc/ceph/ceph.conf 2024-03-28T17:56:35.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:56:35 smithi134 ceph-mon[35037]: Updating smithi134:/etc/ceph/ceph.conf 2024-03-28T17:56:35.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:56:35 smithi134 ceph-mon[35037]: Updating smithi079:/var/lib/ceph/fd220748-ed2b-11ee-95d0-87774f69a715/config/ceph.conf 2024-03-28T17:56:35.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:56:35 smithi134 ceph-mon[35037]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T17:56:35.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:56:35 smithi134 ceph-mon[35037]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T17:56:35.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:56:35 smithi134 ceph-mon[35037]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T17:56:35.676 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:56:35 smithi134 ceph-mon[35037]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T17:56:35.676 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:56:35 smithi134 ceph-mon[35037]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T17:56:35.676 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:56:35 smithi134 ceph-mon[35037]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T17:56:35.676 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:56:35 smithi134 ceph-mon[35037]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T17:56:35.683 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:56:35 smithi079 ceph-mon[35312]: Updating smithi022:/etc/ceph/ceph.conf 2024-03-28T17:56:35.683 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:56:35 smithi079 ceph-mon[35312]: Updating smithi079:/etc/ceph/ceph.conf 2024-03-28T17:56:35.683 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:56:35 smithi079 ceph-mon[35312]: Updating smithi134:/etc/ceph/ceph.conf 2024-03-28T17:56:35.683 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:56:35 smithi079 ceph-mon[35312]: Updating smithi079:/var/lib/ceph/fd220748-ed2b-11ee-95d0-87774f69a715/config/ceph.conf 2024-03-28T17:56:35.683 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:56:35 smithi079 ceph-mon[35312]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T17:56:35.683 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:56:35 smithi079 ceph-mon[35312]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T17:56:35.683 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:56:35 smithi079 ceph-mon[35312]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T17:56:35.684 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:56:35 smithi079 ceph-mon[35312]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T17:56:35.684 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:56:35 smithi079 ceph-mon[35312]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T17:56:35.684 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:56:35 smithi079 ceph-mon[35312]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T17:56:35.684 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:56:35 smithi079 ceph-mon[35312]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T17:56:36.533 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:56:36 smithi022 ceph-mon[31985]: Updating smithi022:/var/lib/ceph/fd220748-ed2b-11ee-95d0-87774f69a715/config/ceph.conf 2024-03-28T17:56:36.533 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:56:36 smithi022 ceph-mon[31985]: Updating smithi134:/var/lib/ceph/fd220748-ed2b-11ee-95d0-87774f69a715/config/ceph.conf 2024-03-28T17:56:36.533 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:56:36 smithi022 ceph-mon[31985]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-28T17:56:36.533 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:56:36 smithi022 ceph-mon[31985]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T17:56:36.534 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:56:36 smithi022 ceph-mon[31985]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T17:56:36.534 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:56:36 smithi022 ceph-mon[31985]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T17:56:36.534 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:56:36 smithi022 ceph-mon[31985]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T17:56:36.534 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:56:36 smithi022 ceph-mon[31985]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T17:56:36.534 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:56:36 smithi022 ceph-mon[31985]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-28T17:56:36.534 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:56:36 smithi022 ceph-mon[31985]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-03-28T17:56:36.534 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:56:36 smithi022 ceph-mon[31985]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T17:56:36.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:56:36 smithi134 ceph-mon[35037]: Updating smithi022:/var/lib/ceph/fd220748-ed2b-11ee-95d0-87774f69a715/config/ceph.conf 2024-03-28T17:56:36.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:56:36 smithi134 ceph-mon[35037]: Updating smithi134:/var/lib/ceph/fd220748-ed2b-11ee-95d0-87774f69a715/config/ceph.conf 2024-03-28T17:56:36.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:56:36 smithi134 ceph-mon[35037]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-28T17:56:36.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:56:36 smithi134 ceph-mon[35037]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T17:56:36.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:56:36 smithi134 ceph-mon[35037]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T17:56:36.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:56:36 smithi134 ceph-mon[35037]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T17:56:36.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:56:36 smithi134 ceph-mon[35037]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T17:56:36.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:56:36 smithi134 ceph-mon[35037]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T17:56:36.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:56:36 smithi134 ceph-mon[35037]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-28T17:56:36.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:56:36 smithi134 ceph-mon[35037]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-03-28T17:56:36.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:56:36 smithi134 ceph-mon[35037]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T17:56:36.683 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:56:36 smithi079 ceph-mon[35312]: Updating smithi022:/var/lib/ceph/fd220748-ed2b-11ee-95d0-87774f69a715/config/ceph.conf 2024-03-28T17:56:36.683 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:56:36 smithi079 ceph-mon[35312]: Updating smithi134:/var/lib/ceph/fd220748-ed2b-11ee-95d0-87774f69a715/config/ceph.conf 2024-03-28T17:56:36.683 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:56:36 smithi079 ceph-mon[35312]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-28T17:56:36.683 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:56:36 smithi079 ceph-mon[35312]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T17:56:36.683 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:56:36 smithi079 ceph-mon[35312]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T17:56:36.683 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:56:36 smithi079 ceph-mon[35312]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T17:56:36.683 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:56:36 smithi079 ceph-mon[35312]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T17:56:36.683 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:56:36 smithi079 ceph-mon[35312]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T17:56:36.683 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:56:36 smithi079 ceph-mon[35312]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-28T17:56:36.683 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:56:36 smithi079 ceph-mon[35312]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-03-28T17:56:36.683 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:56:36 smithi079 ceph-mon[35312]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T17:56:36.733 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T17:56:36.732+0000 7fbc81c00700 1 -- 172.21.15.22:0/3333378493 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fbc7c101dc0 msgr2=0x7fbc7c1021a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:56:36.733 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T17:56:36.732+0000 7fbc81c00700 1 --2- 172.21.15.22:0/3333378493 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fbc7c101dc0 0x7fbc7c1021a0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fbc6c009910 tx=0x7fbc6c009c20 comp rx=0 tx=0).stop 2024-03-28T17:56:36.733 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T17:56:36.732+0000 7fbc81c00700 1 -- 172.21.15.22:0/3333378493 shutdown_connections 2024-03-28T17:56:36.733 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T17:56:36.732+0000 7fbc81c00700 1 --2- 172.21.15.22:0/3333378493 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fbc7c101dc0 0x7fbc7c1021a0 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:56:36.733 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T17:56:36.732+0000 7fbc81c00700 1 -- 172.21.15.22:0/3333378493 >> 172.21.15.22:0/3333378493 conn(0x7fbc7c0fd670 msgr2=0x7fbc7c0ffa90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T17:56:36.733 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T17:56:36.732+0000 7fbc81c00700 1 -- 172.21.15.22:0/3333378493 shutdown_connections 2024-03-28T17:56:36.733 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T17:56:36.732+0000 7fbc81c00700 1 -- 172.21.15.22:0/3333378493 wait complete. 2024-03-28T17:56:36.734 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T17:56:36.733+0000 7fbc81c00700 1 Processor -- start 2024-03-28T17:56:36.734 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T17:56:36.733+0000 7fbc81c00700 1 -- start start 2024-03-28T17:56:36.734 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T17:56:36.734+0000 7fbc81c00700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fbc7c101dc0 0x7fbc7c076030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:56:36.734 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T17:56:36.734+0000 7fbc81c00700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbc7c074490 0x7fbc7c074910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:56:36.734 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T17:56:36.734+0000 7fbc81c00700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fbc7c074e50 0x7fbc7c1aa340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:56:36.734 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T17:56:36.734+0000 7fbc81c00700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc7c10c180 con 0x7fbc7c074e50 2024-03-28T17:56:36.734 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T17:56:36.734+0000 7fbc81c00700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc7c10c000 con 0x7fbc7c101dc0 2024-03-28T17:56:36.734 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T17:56:36.734+0000 7fbc81c00700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc7c10c300 con 0x7fbc7c074490 2024-03-28T17:56:36.735 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T17:56:36.734+0000 7fbc7b400700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fbc7c074e50 0x7fbc7c1aa340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:56:36.736 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T17:56:36.734+0000 7fbc7b400700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fbc7c074e50 0x7fbc7c1aa340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:41498/0 (socket says 172.21.15.22:41498) 2024-03-28T17:56:36.736 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T17:56:36.734+0000 7fbc7b400700 1 -- 172.21.15.22:0/4154185460 learned_addr learned my addr 172.21.15.22:0/4154185460 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-28T17:56:36.736 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T17:56:36.735+0000 7fbc7a000700 1 --2- 172.21.15.22:0/4154185460 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbc7c074490 0x7fbc7c074910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:56:36.736 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T17:56:36.735+0000 7fbc7aa00700 1 --2- 172.21.15.22:0/4154185460 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fbc7c101dc0 0x7fbc7c076030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:56:36.736 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T17:56:36.735+0000 7fbc7b400700 1 -- 172.21.15.22:0/4154185460 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbc7c074490 msgr2=0x7fbc7c074910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:56:36.736 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T17:56:36.735+0000 7fbc7b400700 1 --2- 172.21.15.22:0/4154185460 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbc7c074490 0x7fbc7c074910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:56:36.736 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T17:56:36.735+0000 7fbc7b400700 1 -- 172.21.15.22:0/4154185460 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fbc7c101dc0 msgr2=0x7fbc7c076030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:56:36.736 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T17:56:36.735+0000 7fbc7b400700 1 --2- 172.21.15.22:0/4154185460 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fbc7c101dc0 0x7fbc7c076030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:56:36.736 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T17:56:36.735+0000 7fbc7b400700 1 -- 172.21.15.22:0/4154185460 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbc7c1aaa10 con 0x7fbc7c074e50 2024-03-28T17:56:36.736 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T17:56:36.735+0000 7fbc7b400700 1 --2- 172.21.15.22:0/4154185460 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fbc7c074e50 0x7fbc7c1aa340 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fbc7400ba50 tx=0x7fbc7400be10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T17:56:36.736 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T17:56:36.735+0000 7fbc73400700 1 -- 172.21.15.22:0/4154185460 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc74019070 con 0x7fbc7c074e50 2024-03-28T17:56:36.736 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T17:56:36.735+0000 7fbc73400700 1 -- 172.21.15.22:0/4154185460 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7fbc74014440 con 0x7fbc7c074e50 2024-03-28T17:56:36.736 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T17:56:36.735+0000 7fbc73400700 1 -- 172.21.15.22:0/4154185460 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc74007590 con 0x7fbc7c074e50 2024-03-28T17:56:36.736 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T17:56:36.736+0000 7fbc81c00700 1 -- 172.21.15.22:0/4154185460 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbc7c1aad00 con 0x7fbc7c074e50 2024-03-28T17:56:36.736 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T17:56:36.736+0000 7fbc81c00700 1 -- 172.21.15.22:0/4154185460 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbc7c1ab190 con 0x7fbc7c074e50 2024-03-28T17:56:36.737 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T17:56:36.737+0000 7fbc81c00700 1 -- 172.21.15.22:0/4154185460 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbc7c1032a0 con 0x7fbc7c074e50 2024-03-28T17:56:36.738 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T17:56:36.738+0000 7fbc73400700 1 -- 172.21.15.22:0/4154185460 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 13) v1 ==== 46162+0+0 (secure 0 0 0) 0x7fbc7401c070 con 0x7fbc7c074e50 2024-03-28T17:56:36.738 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T17:56:36.738+0000 7fbc73400700 1 --2- 172.21.15.22:0/4154185460 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7fbc60039450 0x7fbc6003b910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:56:36.738 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T17:56:36.738+0000 7fbc73400700 1 -- 172.21.15.22:0/4154185460 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7fbc7404b7b0 con 0x7fbc7c074e50 2024-03-28T17:56:36.739 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T17:56:36.739+0000 7fbc7aa00700 1 --2- 172.21.15.22:0/4154185460 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7fbc60039450 0x7fbc6003b910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:56:36.739 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T17:56:36.740+0000 7fbc7aa00700 1 --2- 172.21.15.22:0/4154185460 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7fbc60039450 0x7fbc6003b910 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fbc6c009750 tx=0x7fbc6c00b540 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T17:56:36.744 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T17:56:36.744+0000 7fbc73400700 1 -- 172.21.15.22:0/4154185460 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fbc7401b2c0 con 0x7fbc7c074e50 2024-03-28T17:56:36.977 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T17:56:36.977+0000 7fbc81c00700 1 -- 172.21.15.22:0/4154185460 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "config generate-minimal-conf"} v 0) v1 -- 0x7fbc7c1ab520 con 0x7fbc7c074e50 2024-03-28T17:56:36.977 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T17:56:36.977+0000 7fbc73400700 1 -- 172.21.15.22:0/4154185460 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v9) v1 ==== 76+0+273 (secure 0 0 0) 0x7fbc740145f0 con 0x7fbc7c074e50 2024-03-28T17:56:36.977 INFO:teuthology.orchestra.run.smithi022.stdout:# minimal ceph.conf for fd220748-ed2b-11ee-95d0-87774f69a715 2024-03-28T17:56:36.978 INFO:teuthology.orchestra.run.smithi022.stdout:[global] 2024-03-28T17:56:36.978 INFO:teuthology.orchestra.run.smithi022.stdout: fsid = fd220748-ed2b-11ee-95d0-87774f69a715 2024-03-28T17:56:36.978 INFO:teuthology.orchestra.run.smithi022.stdout: mon_host = [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] 2024-03-28T17:56:36.980 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T17:56:36.980+0000 7fbc81c00700 1 -- 172.21.15.22:0/4154185460 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7fbc60039450 msgr2=0x7fbc6003b910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:56:36.980 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T17:56:36.980+0000 7fbc81c00700 1 --2- 172.21.15.22:0/4154185460 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7fbc60039450 0x7fbc6003b910 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fbc6c009750 tx=0x7fbc6c00b540 comp rx=0 tx=0).stop 2024-03-28T17:56:36.981 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T17:56:36.980+0000 7fbc81c00700 1 -- 172.21.15.22:0/4154185460 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fbc7c074e50 msgr2=0x7fbc7c1aa340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:56:36.981 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T17:56:36.980+0000 7fbc81c00700 1 --2- 172.21.15.22:0/4154185460 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fbc7c074e50 0x7fbc7c1aa340 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fbc7400ba50 tx=0x7fbc7400be10 comp rx=0 tx=0).stop 2024-03-28T17:56:36.981 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T17:56:36.980+0000 7fbc81c00700 1 -- 172.21.15.22:0/4154185460 shutdown_connections 2024-03-28T17:56:36.981 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T17:56:36.980+0000 7fbc81c00700 1 --2- 172.21.15.22:0/4154185460 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fbc7c074e50 0x7fbc7c1aa340 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:56:36.981 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T17:56:36.980+0000 7fbc81c00700 1 --2- 172.21.15.22:0/4154185460 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fbc7c101dc0 0x7fbc7c076030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:56:36.981 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T17:56:36.980+0000 7fbc81c00700 1 --2- 172.21.15.22:0/4154185460 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7fbc60039450 0x7fbc6003b910 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:56:36.981 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T17:56:36.980+0000 7fbc81c00700 1 --2- 172.21.15.22:0/4154185460 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbc7c074490 0x7fbc7c074910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:56:36.982 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T17:56:36.980+0000 7fbc81c00700 1 -- 172.21.15.22:0/4154185460 >> 172.21.15.22:0/4154185460 conn(0x7fbc7c0fd670 msgr2=0x7fbc7c103790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T17:56:36.982 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T17:56:36.981+0000 7fbc81c00700 1 -- 172.21.15.22:0/4154185460 shutdown_connections 2024-03-28T17:56:36.982 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T17:56:36.981+0000 7fbc81c00700 1 -- 172.21.15.22:0/4154185460 wait complete. 2024-03-28T17:56:37.674 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:56:37 smithi134 ceph-mon[35037]: Reconfiguring mon.a (unknown last config time)... 2024-03-28T17:56:37.674 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:56:37 smithi134 ceph-mon[35037]: Reconfiguring daemon mon.a on smithi022 2024-03-28T17:56:37.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:56:37 smithi134 ceph-mon[35037]: from='client.? 172.21.15.22:0/4154185460' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T17:56:37.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:56:37 smithi079 ceph-mon[35312]: Reconfiguring mon.a (unknown last config time)... 2024-03-28T17:56:37.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:56:37 smithi079 ceph-mon[35312]: Reconfiguring daemon mon.a on smithi022 2024-03-28T17:56:37.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:56:37 smithi079 ceph-mon[35312]: from='client.? 172.21.15.22:0/4154185460' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T17:56:37.709 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:56:37 smithi022 ceph-mon[31985]: Reconfiguring mon.a (unknown last config time)... 2024-03-28T17:56:37.709 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:56:37 smithi022 ceph-mon[31985]: Reconfiguring daemon mon.a on smithi022 2024-03-28T17:56:37.709 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:56:37 smithi022 ceph-mon[31985]: from='client.? 172.21.15.22:0/4154185460' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T17:56:38.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:56:38 smithi022 ceph-mon[31985]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-28T17:56:38.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:56:38 smithi022 ceph-mon[31985]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T17:56:38.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:56:38 smithi022 ceph-mon[31985]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T17:56:38.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:56:38 smithi022 ceph-mon[31985]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-28T17:56:38.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:56:38 smithi022 ceph-mon[31985]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-03-28T17:56:38.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:56:38 smithi022 ceph-mon[31985]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T17:56:38.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:56:38 smithi134 ceph-mon[35037]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-28T17:56:38.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:56:38 smithi134 ceph-mon[35037]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T17:56:38.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:56:38 smithi134 ceph-mon[35037]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T17:56:38.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:56:38 smithi134 ceph-mon[35037]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-28T17:56:38.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:56:38 smithi134 ceph-mon[35037]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-03-28T17:56:38.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:56:38 smithi134 ceph-mon[35037]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T17:56:38.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:56:38 smithi079 ceph-mon[35312]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-28T17:56:38.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:56:38 smithi079 ceph-mon[35312]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T17:56:38.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:56:38 smithi079 ceph-mon[35312]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T17:56:38.683 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:56:38 smithi079 ceph-mon[35312]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-28T17:56:38.683 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:56:38 smithi079 ceph-mon[35312]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-03-28T17:56:38.683 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:56:38 smithi079 ceph-mon[35312]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T17:56:38.779 INFO:tasks.cephadm:Distributing (final) config and client.admin keyring... 2024-03-28T17:56:38.779 DEBUG:teuthology.orchestra.run.smithi022:> set -ex 2024-03-28T17:56:38.779 DEBUG:teuthology.orchestra.run.smithi022:> sudo dd of=/etc/ceph/ceph.conf 2024-03-28T17:56:38.888 DEBUG:teuthology.orchestra.run.smithi022:> set -ex 2024-03-28T17:56:38.888 DEBUG:teuthology.orchestra.run.smithi022:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-03-28T17:56:39.034 DEBUG:teuthology.orchestra.run.smithi079:> set -ex 2024-03-28T17:56:39.034 DEBUG:teuthology.orchestra.run.smithi079:> sudo dd of=/etc/ceph/ceph.conf 2024-03-28T17:56:39.113 DEBUG:teuthology.orchestra.run.smithi079:> set -ex 2024-03-28T17:56:39.113 DEBUG:teuthology.orchestra.run.smithi079:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-03-28T17:56:39.239 DEBUG:teuthology.orchestra.run.smithi134:> set -ex 2024-03-28T17:56:39.239 DEBUG:teuthology.orchestra.run.smithi134:> sudo dd of=/etc/ceph/ceph.conf 2024-03-28T17:56:39.339 DEBUG:teuthology.orchestra.run.smithi134:> set -ex 2024-03-28T17:56:39.339 DEBUG:teuthology.orchestra.run.smithi134:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-03-28T17:56:39.476 INFO:tasks.cephadm:Adding mgr.x on smithi022 2024-03-28T17:56:39.477 INFO:tasks.cephadm:Adding mgr.y on smithi079 2024-03-28T17:56:39.477 INFO:tasks.cephadm:Adding mgr.z on smithi134 2024-03-28T17:56:39.477 DEBUG:teuthology.orchestra.run.smithi134:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fd220748-ed2b-11ee-95d0-87774f69a715 -- ceph orch apply mgr '3;smithi022=x;smithi079=y;smithi134=z' 2024-03-28T17:56:39.603 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:56:39 smithi134 ceph-mon[35037]: Reconfiguring mon.b (monmap changed)... 2024-03-28T17:56:39.603 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:56:39 smithi134 ceph-mon[35037]: Reconfiguring daemon mon.b on smithi079 2024-03-28T17:56:39.683 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:56:39 smithi079 ceph-mon[35312]: Reconfiguring mon.b (monmap changed)... 2024-03-28T17:56:39.683 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:56:39 smithi079 ceph-mon[35312]: Reconfiguring daemon mon.b on smithi079 2024-03-28T17:56:39.781 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:56:39 smithi022 ceph-mon[31985]: Reconfiguring mon.b (monmap changed)... 2024-03-28T17:56:39.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:56:39 smithi022 ceph-mon[31985]: Reconfiguring daemon mon.b on smithi079 2024-03-28T17:56:39.953 INFO:teuthology.orchestra.run.smithi134.stderr:Inferring config /var/lib/ceph/fd220748-ed2b-11ee-95d0-87774f69a715/mon.c/config 2024-03-28T17:56:40.674 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:56:40 smithi134 ceph-mon[35037]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-28T17:56:40.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:56:40 smithi079 ceph-mon[35312]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-28T17:56:40.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:56:40 smithi022 ceph-mon[31985]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-28T17:56:41.507 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:56:41 smithi134 ceph-mon[35037]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T17:56:41.507 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:56:41 smithi134 ceph-mon[35037]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T17:56:41.507 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:56:41 smithi134 ceph-mon[35037]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-28T17:56:41.508 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:56:41 smithi134 ceph-mon[35037]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-03-28T17:56:41.508 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:56:41 smithi134 ceph-mon[35037]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T17:56:41.683 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:56:41 smithi079 ceph-mon[35312]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T17:56:41.683 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:56:41 smithi079 ceph-mon[35312]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T17:56:41.683 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:56:41 smithi079 ceph-mon[35312]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-28T17:56:41.683 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:56:41 smithi079 ceph-mon[35312]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-03-28T17:56:41.683 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:56:41 smithi079 ceph-mon[35312]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T17:56:41.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:56:41 smithi022 ceph-mon[31985]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T17:56:41.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:56:41 smithi022 ceph-mon[31985]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T17:56:41.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:56:41 smithi022 ceph-mon[31985]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-28T17:56:41.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:56:41 smithi022 ceph-mon[31985]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-03-28T17:56:41.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:56:41 smithi022 ceph-mon[31985]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T17:56:42.599 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-28T17:56:42.598+0000 7f0112800700 1 -- 172.21.15.134:0/250709612 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f010c105560 msgr2=0x7f010c105980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:56:42.599 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-28T17:56:42.598+0000 7f0112800700 1 --2- 172.21.15.134:0/250709612 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f010c105560 0x7f010c105980 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f00fc009910 tx=0x7f00fc009c20 comp rx=0 tx=0).stop 2024-03-28T17:56:42.599 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-28T17:56:42.598+0000 7f0112800700 1 -- 172.21.15.134:0/250709612 shutdown_connections 2024-03-28T17:56:42.599 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-28T17:56:42.598+0000 7f0112800700 1 --2- 172.21.15.134:0/250709612 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f010c105560 0x7f010c105980 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:56:42.599 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-28T17:56:42.598+0000 7f0112800700 1 -- 172.21.15.134:0/250709612 >> 172.21.15.134:0/250709612 conn(0x7f010c0ff4c0 msgr2=0x7f010c101900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T17:56:42.599 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-28T17:56:42.598+0000 7f0112800700 1 -- 172.21.15.134:0/250709612 shutdown_connections 2024-03-28T17:56:42.599 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-28T17:56:42.598+0000 7f0112800700 1 -- 172.21.15.134:0/250709612 wait complete. 2024-03-28T17:56:42.600 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-28T17:56:42.599+0000 7f0112800700 1 Processor -- start 2024-03-28T17:56:42.600 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-28T17:56:42.600+0000 7f0112800700 1 -- start start 2024-03-28T17:56:42.601 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-28T17:56:42.600+0000 7f0112800700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f010c105560 0x7f010c1997c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:56:42.601 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-28T17:56:42.600+0000 7f0112800700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f010c199d00 0x7f010c19ed70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:56:42.601 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-28T17:56:42.600+0000 7f0112800700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f010c19a180 0x7f010c1a1300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:56:42.601 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-28T17:56:42.600+0000 7f0112800700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f010c10a020 con 0x7f010c105560 2024-03-28T17:56:42.601 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-28T17:56:42.600+0000 7f0112800700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f010c109ea0 con 0x7f010c19a180 2024-03-28T17:56:42.601 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-28T17:56:42.600+0000 7f0112800700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f010c10a1a0 con 0x7f010c199d00 2024-03-28T17:56:42.601 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-28T17:56:42.600+0000 7f010aa00700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f010c199d00 0x7f010c19ed70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:56:42.601 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-28T17:56:42.601+0000 7f010aa00700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f010c199d00 0x7f010c19ed70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.134:56672/0 (socket says 172.21.15.134:56672) 2024-03-28T17:56:42.601 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-28T17:56:42.601+0000 7f010aa00700 1 -- 172.21.15.134:0/4121398022 learned_addr learned my addr 172.21.15.134:0/4121398022 (peer_addr_for_me v2:172.21.15.134:0/0) 2024-03-28T17:56:42.601 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-28T17:56:42.601+0000 7f010b400700 1 --2- 172.21.15.134:0/4121398022 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f010c105560 0x7f010c1997c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:56:42.602 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-28T17:56:42.601+0000 7f010be00700 1 --2- 172.21.15.134:0/4121398022 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f010c19a180 0x7f010c1a1300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:56:42.602 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-28T17:56:42.601+0000 7f010aa00700 1 -- 172.21.15.134:0/4121398022 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f010c19a180 msgr2=0x7f010c1a1300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:56:42.602 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-28T17:56:42.601+0000 7f010aa00700 1 --2- 172.21.15.134:0/4121398022 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f010c19a180 0x7f010c1a1300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:56:42.602 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-28T17:56:42.601+0000 7f010aa00700 1 -- 172.21.15.134:0/4121398022 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f010c105560 msgr2=0x7f010c1997c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:56:42.603 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-28T17:56:42.601+0000 7f010aa00700 1 --2- 172.21.15.134:0/4121398022 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f010c105560 0x7f010c1997c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:56:42.603 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-28T17:56:42.601+0000 7f010aa00700 1 -- 172.21.15.134:0/4121398022 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f010c1a1bd0 con 0x7f010c199d00 2024-03-28T17:56:42.603 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-28T17:56:42.601+0000 7f010aa00700 1 --2- 172.21.15.134:0/4121398022 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f010c199d00 0x7f010c19ed70 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f00f800b780 tx=0x7f00f800bb40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T17:56:42.603 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-28T17:56:42.602+0000 7f0103e00700 1 -- 172.21.15.134:0/4121398022 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00f8004230 con 0x7f010c199d00 2024-03-28T17:56:42.603 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-28T17:56:42.602+0000 7f0103e00700 1 -- 172.21.15.134:0/4121398022 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f00f80043d0 con 0x7f010c199d00 2024-03-28T17:56:42.603 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-28T17:56:42.602+0000 7f0112800700 1 -- 172.21.15.134:0/4121398022 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f010c1a1ec0 con 0x7f010c199d00 2024-03-28T17:56:42.603 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-28T17:56:42.602+0000 7f0103e00700 1 -- 172.21.15.134:0/4121398022 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00f8011c10 con 0x7f010c199d00 2024-03-28T17:56:42.603 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-28T17:56:42.602+0000 7f0112800700 1 -- 172.21.15.134:0/4121398022 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f010c1a2440 con 0x7f010c199d00 2024-03-28T17:56:42.604 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-28T17:56:42.603+0000 7f0103e00700 1 -- 172.21.15.134:0/4121398022 <== mon.1 v2:172.21.15.134:3300/0 4 ==== mgrmap(e 13) v1 ==== 46162+0+0 (secure 0 0 0) 0x7f00f80036a0 con 0x7f010c199d00 2024-03-28T17:56:42.604 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-28T17:56:42.603+0000 7f0112800700 1 -- 172.21.15.134:0/4121398022 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f010c0637c0 con 0x7f010c199d00 2024-03-28T17:56:42.604 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-28T17:56:42.604+0000 7f0103e00700 1 --2- 172.21.15.134:0/4121398022 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7f00f0039450 0x7f00f003b910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:56:42.604 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-28T17:56:42.604+0000 7f0103e00700 1 -- 172.21.15.134:0/4121398022 <== mon.1 v2:172.21.15.134:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f00f804bb90 con 0x7f010c199d00 2024-03-28T17:56:42.604 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-28T17:56:42.604+0000 7f010b400700 1 --2- 172.21.15.134:0/4121398022 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7f00f0039450 0x7f00f003b910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:56:42.605 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-28T17:56:42.605+0000 7f010b400700 1 --2- 172.21.15.134:0/4121398022 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7f00f0039450 0x7f00f003b910 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f00fc005950 tx=0x7f00fc01a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T17:56:42.610 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-28T17:56:42.610+0000 7f0103e00700 1 -- 172.21.15.134:0/4121398022 <== mon.1 v2:172.21.15.134:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f00f8021e20 con 0x7f010c199d00 2024-03-28T17:56:42.662 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:56:42 smithi134 ceph-mon[35037]: Reconfiguring mon.c (monmap changed)... 2024-03-28T17:56:42.662 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:56:42 smithi134 ceph-mon[35037]: Reconfiguring daemon mon.c on smithi134 2024-03-28T17:56:42.662 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:56:42 smithi134 ceph-mon[35037]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-28T17:56:42.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:56:42 smithi079 ceph-mon[35312]: Reconfiguring mon.c (monmap changed)... 2024-03-28T17:56:42.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:56:42 smithi079 ceph-mon[35312]: Reconfiguring daemon mon.c on smithi134 2024-03-28T17:56:42.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:56:42 smithi079 ceph-mon[35312]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-28T17:56:42.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:56:42 smithi022 ceph-mon[31985]: Reconfiguring mon.c (monmap changed)... 2024-03-28T17:56:42.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:56:42 smithi022 ceph-mon[31985]: Reconfiguring daemon mon.c on smithi134 2024-03-28T17:56:42.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:56:42 smithi022 ceph-mon[31985]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-28T17:56:42.852 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-28T17:56:42.852+0000 7f0112800700 1 -- 172.21.15.134:0/4121398022 --> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi022=x;smithi079=y;smithi134=z", "target": ["mon-mgr", ""]}) v1 -- 0x7f010c062720 con 0x7f00f0039450 2024-03-28T17:56:42.866 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-28T17:56:42.866+0000 7f0103e00700 1 -- 172.21.15.134:0/4121398022 <== mgr.14152 v2:172.21.15.22:6800/4201048392 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+24 (secure 0 0 0) 0x7f010c062720 con 0x7f00f0039450 2024-03-28T17:56:42.866 INFO:teuthology.orchestra.run.smithi134.stdout:Scheduled mgr update... 2024-03-28T17:56:42.869 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-28T17:56:42.869+0000 7f0112800700 1 -- 172.21.15.134:0/4121398022 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7f00f0039450 msgr2=0x7f00f003b910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:56:42.869 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-28T17:56:42.869+0000 7f0112800700 1 --2- 172.21.15.134:0/4121398022 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7f00f0039450 0x7f00f003b910 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f00fc005950 tx=0x7f00fc01a040 comp rx=0 tx=0).stop 2024-03-28T17:56:42.869 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-28T17:56:42.869+0000 7f0112800700 1 -- 172.21.15.134:0/4121398022 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f010c199d00 msgr2=0x7f010c19ed70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:56:42.869 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-28T17:56:42.869+0000 7f0112800700 1 --2- 172.21.15.134:0/4121398022 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f010c199d00 0x7f010c19ed70 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f00f800b780 tx=0x7f00f800bb40 comp rx=0 tx=0).stop 2024-03-28T17:56:42.869 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-28T17:56:42.869+0000 7f0112800700 1 -- 172.21.15.134:0/4121398022 shutdown_connections 2024-03-28T17:56:42.870 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-28T17:56:42.869+0000 7f0112800700 1 --2- 172.21.15.134:0/4121398022 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f010c19a180 0x7f010c1a1300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:56:42.870 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-28T17:56:42.869+0000 7f0112800700 1 --2- 172.21.15.134:0/4121398022 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f010c105560 0x7f010c1997c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:56:42.870 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-28T17:56:42.869+0000 7f0112800700 1 --2- 172.21.15.134:0/4121398022 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7f00f0039450 0x7f00f003b910 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:56:42.870 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-28T17:56:42.869+0000 7f0112800700 1 --2- 172.21.15.134:0/4121398022 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f010c199d00 0x7f010c19ed70 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:56:42.870 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-28T17:56:42.869+0000 7f0112800700 1 -- 172.21.15.134:0/4121398022 >> 172.21.15.134:0/4121398022 conn(0x7f010c0ff4c0 msgr2=0x7f010c1000d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T17:56:42.870 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-28T17:56:42.869+0000 7f0112800700 1 -- 172.21.15.134:0/4121398022 shutdown_connections 2024-03-28T17:56:42.870 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-28T17:56:42.869+0000 7f0112800700 1 -- 172.21.15.134:0/4121398022 wait complete. 2024-03-28T17:56:43.809 DEBUG:teuthology.orchestra.run.smithi079:mgr.y> sudo journalctl -f -n 0 -u ceph-fd220748-ed2b-11ee-95d0-87774f69a715@mgr.y.service 2024-03-28T17:56:43.816 DEBUG:teuthology.orchestra.run.smithi134:mgr.z> sudo journalctl -f -n 0 -u ceph-fd220748-ed2b-11ee-95d0-87774f69a715@mgr.z.service 2024-03-28T17:56:43.823 INFO:tasks.cephadm:Deploying OSDs... 2024-03-28T17:56:43.823 DEBUG:teuthology.orchestra.run.smithi022:> set -ex 2024-03-28T17:56:43.823 DEBUG:teuthology.orchestra.run.smithi022:> dd if=/scratch_devs of=/dev/stdout 2024-03-28T17:56:43.883 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-03-28T17:56:43.883 DEBUG:teuthology.orchestra.run.smithi022:> stat /dev/vg_nvme/lv_1 2024-03-28T17:56:43.935 INFO:teuthology.orchestra.run.smithi022.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-03-28T17:56:43.935 INFO:teuthology.orchestra.run.smithi022.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-28T17:56:43.935 INFO:teuthology.orchestra.run.smithi022.stdout:Device: 5h/5d Inode: 632 Links: 1 2024-03-28T17:56:43.935 INFO:teuthology.orchestra.run.smithi022.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-28T17:56:43.935 INFO:teuthology.orchestra.run.smithi022.stdout:Context: system_u:object_r:device_t:s0 2024-03-28T17:56:43.935 INFO:teuthology.orchestra.run.smithi022.stdout:Access: 2024-03-28 17:55:28.017654681 +0000 2024-03-28T17:56:43.935 INFO:teuthology.orchestra.run.smithi022.stdout:Modify: 2024-03-28 17:49:33.575379139 +0000 2024-03-28T17:56:43.935 INFO:teuthology.orchestra.run.smithi022.stdout:Change: 2024-03-28 17:49:33.575379139 +0000 2024-03-28T17:56:43.935 INFO:teuthology.orchestra.run.smithi022.stdout: Birth: 2024-03-28 17:49:33.575379139 +0000 2024-03-28T17:56:43.936 DEBUG:teuthology.orchestra.run.smithi022:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-03-28T17:56:44.025 INFO:teuthology.orchestra.run.smithi022.stderr:1+0 records in 2024-03-28T17:56:44.025 INFO:teuthology.orchestra.run.smithi022.stderr:1+0 records out 2024-03-28T17:56:44.025 INFO:teuthology.orchestra.run.smithi022.stderr:512 bytes copied, 0.00026679 s, 1.9 MB/s 2024-03-28T17:56:44.030 DEBUG:teuthology.orchestra.run.smithi022:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-03-28T17:56:44.090 DEBUG:teuthology.orchestra.run.smithi022:> stat /dev/vg_nvme/lv_2 2024-03-28T17:56:44.103 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:56:43 smithi134 ceph-mon[35037]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-28T17:56:44.103 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:56:43 smithi134 ceph-mon[35037]: from='client.24109 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi022=x;smithi079=y;smithi134=z", "target": ["mon-mgr", ""]}]: dispatch 2024-03-28T17:56:44.103 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:56:43 smithi134 ceph-mon[35037]: Saving service mgr spec with placement smithi022=x;smithi079=y;smithi134=z;count:3 2024-03-28T17:56:44.103 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:56:43 smithi134 ceph-mon[35037]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T17:56:44.103 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:56:43 smithi134 ceph-mon[35037]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T17:56:44.103 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:56:43 smithi134 ceph-mon[35037]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T17:56:44.103 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:56:43 smithi134 ceph-mon[35037]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T17:56:44.104 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:56:43 smithi134 ceph-mon[35037]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T17:56:44.104 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:56:43 smithi134 ceph-mon[35037]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T17:56:44.104 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:56:43 smithi134 ceph-mon[35037]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T17:56:44.104 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:56:43 smithi134 ceph-mon[35037]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-03-28T17:56:44.104 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:56:43 smithi134 ceph-mon[35037]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-03-28T17:56:44.104 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:56:43 smithi134 ceph-mon[35037]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-03-28T17:56:44.104 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:56:43 smithi134 ceph-mon[35037]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T17:56:44.104 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:56:43 smithi134 ceph-mon[35037]: Deploying daemon mgr.z on smithi134 2024-03-28T17:56:44.142 INFO:teuthology.orchestra.run.smithi022.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-03-28T17:56:44.142 INFO:teuthology.orchestra.run.smithi022.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-28T17:56:44.143 INFO:teuthology.orchestra.run.smithi022.stdout:Device: 5h/5d Inode: 650 Links: 1 2024-03-28T17:56:44.143 INFO:teuthology.orchestra.run.smithi022.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-28T17:56:44.143 INFO:teuthology.orchestra.run.smithi022.stdout:Context: system_u:object_r:device_t:s0 2024-03-28T17:56:44.143 INFO:teuthology.orchestra.run.smithi022.stdout:Access: 2024-03-28 17:55:28.065654061 +0000 2024-03-28T17:56:44.143 INFO:teuthology.orchestra.run.smithi022.stdout:Modify: 2024-03-28 17:49:34.185371263 +0000 2024-03-28T17:56:44.143 INFO:teuthology.orchestra.run.smithi022.stdout:Change: 2024-03-28 17:49:34.185371263 +0000 2024-03-28T17:56:44.143 INFO:teuthology.orchestra.run.smithi022.stdout: Birth: 2024-03-28 17:49:34.185371263 +0000 2024-03-28T17:56:44.143 DEBUG:teuthology.orchestra.run.smithi022:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-03-28T17:56:44.183 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:56:43 smithi079 ceph-mon[35312]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-28T17:56:44.183 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:56:43 smithi079 ceph-mon[35312]: from='client.24109 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi022=x;smithi079=y;smithi134=z", "target": ["mon-mgr", ""]}]: dispatch 2024-03-28T17:56:44.183 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:56:43 smithi079 ceph-mon[35312]: Saving service mgr spec with placement smithi022=x;smithi079=y;smithi134=z;count:3 2024-03-28T17:56:44.183 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:56:43 smithi079 ceph-mon[35312]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T17:56:44.183 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:56:43 smithi079 ceph-mon[35312]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T17:56:44.183 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:56:43 smithi079 ceph-mon[35312]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T17:56:44.183 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:56:43 smithi079 ceph-mon[35312]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T17:56:44.184 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:56:43 smithi079 ceph-mon[35312]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T17:56:44.184 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:56:43 smithi079 ceph-mon[35312]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T17:56:44.184 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:56:43 smithi079 ceph-mon[35312]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T17:56:44.184 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:56:43 smithi079 ceph-mon[35312]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-03-28T17:56:44.184 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:56:43 smithi079 ceph-mon[35312]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-03-28T17:56:44.184 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:56:43 smithi079 ceph-mon[35312]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-03-28T17:56:44.184 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:56:43 smithi079 ceph-mon[35312]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T17:56:44.184 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:56:43 smithi079 ceph-mon[35312]: Deploying daemon mgr.z on smithi134 2024-03-28T17:56:44.215 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:56:43 smithi022 ceph-mon[31985]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-28T17:56:44.215 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:56:43 smithi022 ceph-mon[31985]: from='client.24109 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi022=x;smithi079=y;smithi134=z", "target": ["mon-mgr", ""]}]: dispatch 2024-03-28T17:56:44.215 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:56:43 smithi022 ceph-mon[31985]: Saving service mgr spec with placement smithi022=x;smithi079=y;smithi134=z;count:3 2024-03-28T17:56:44.216 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:56:43 smithi022 ceph-mon[31985]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T17:56:44.216 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:56:43 smithi022 ceph-mon[31985]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T17:56:44.216 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:56:43 smithi022 ceph-mon[31985]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T17:56:44.216 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:56:43 smithi022 ceph-mon[31985]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T17:56:44.216 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:56:43 smithi022 ceph-mon[31985]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T17:56:44.216 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:56:43 smithi022 ceph-mon[31985]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T17:56:44.216 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:56:43 smithi022 ceph-mon[31985]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T17:56:44.216 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:56:43 smithi022 ceph-mon[31985]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-03-28T17:56:44.216 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:56:43 smithi022 ceph-mon[31985]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-03-28T17:56:44.216 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:56:43 smithi022 ceph-mon[31985]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-03-28T17:56:44.216 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:56:43 smithi022 ceph-mon[31985]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T17:56:44.216 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:56:43 smithi022 ceph-mon[31985]: Deploying daemon mgr.z on smithi134 2024-03-28T17:56:44.223 INFO:teuthology.orchestra.run.smithi022.stderr:1+0 records in 2024-03-28T17:56:44.223 INFO:teuthology.orchestra.run.smithi022.stderr:1+0 records out 2024-03-28T17:56:44.223 INFO:teuthology.orchestra.run.smithi022.stderr:512 bytes copied, 0.000259986 s, 2.0 MB/s 2024-03-28T17:56:44.227 DEBUG:teuthology.orchestra.run.smithi022:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-03-28T17:56:44.279 DEBUG:teuthology.orchestra.run.smithi022:> stat /dev/vg_nvme/lv_3 2024-03-28T17:56:44.332 INFO:teuthology.orchestra.run.smithi022.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-03-28T17:56:44.333 INFO:teuthology.orchestra.run.smithi022.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-28T17:56:44.333 INFO:teuthology.orchestra.run.smithi022.stdout:Device: 5h/5d Inode: 669 Links: 1 2024-03-28T17:56:44.333 INFO:teuthology.orchestra.run.smithi022.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-28T17:56:44.333 INFO:teuthology.orchestra.run.smithi022.stdout:Context: system_u:object_r:device_t:s0 2024-03-28T17:56:44.333 INFO:teuthology.orchestra.run.smithi022.stdout:Access: 2024-03-28 17:55:28.118653376 +0000 2024-03-28T17:56:44.333 INFO:teuthology.orchestra.run.smithi022.stdout:Modify: 2024-03-28 17:49:34.793363414 +0000 2024-03-28T17:56:44.333 INFO:teuthology.orchestra.run.smithi022.stdout:Change: 2024-03-28 17:49:34.793363414 +0000 2024-03-28T17:56:44.333 INFO:teuthology.orchestra.run.smithi022.stdout: Birth: 2024-03-28 17:49:34.793363414 +0000 2024-03-28T17:56:44.333 DEBUG:teuthology.orchestra.run.smithi022:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-03-28T17:56:44.424 INFO:teuthology.orchestra.run.smithi022.stderr:1+0 records in 2024-03-28T17:56:44.425 INFO:teuthology.orchestra.run.smithi022.stderr:1+0 records out 2024-03-28T17:56:44.425 INFO:teuthology.orchestra.run.smithi022.stderr:512 bytes copied, 0.000256243 s, 2.0 MB/s 2024-03-28T17:56:44.429 DEBUG:teuthology.orchestra.run.smithi022:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-03-28T17:56:44.490 DEBUG:teuthology.orchestra.run.smithi022:> stat /dev/vg_nvme/lv_4 2024-03-28T17:56:44.542 INFO:teuthology.orchestra.run.smithi022.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-03-28T17:56:44.542 INFO:teuthology.orchestra.run.smithi022.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-28T17:56:44.542 INFO:teuthology.orchestra.run.smithi022.stdout:Device: 5h/5d Inode: 691 Links: 1 2024-03-28T17:56:44.542 INFO:teuthology.orchestra.run.smithi022.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-28T17:56:44.542 INFO:teuthology.orchestra.run.smithi022.stdout:Context: system_u:object_r:device_t:s0 2024-03-28T17:56:44.542 INFO:teuthology.orchestra.run.smithi022.stdout:Access: 2024-03-28 17:55:28.164652782 +0000 2024-03-28T17:56:44.542 INFO:teuthology.orchestra.run.smithi022.stdout:Modify: 2024-03-28 17:49:35.359356107 +0000 2024-03-28T17:56:44.542 INFO:teuthology.orchestra.run.smithi022.stdout:Change: 2024-03-28 17:49:35.359356107 +0000 2024-03-28T17:56:44.542 INFO:teuthology.orchestra.run.smithi022.stdout: Birth: 2024-03-28 17:49:35.359356107 +0000 2024-03-28T17:56:44.543 DEBUG:teuthology.orchestra.run.smithi022:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-03-28T17:56:44.635 INFO:teuthology.orchestra.run.smithi022.stderr:1+0 records in 2024-03-28T17:56:44.635 INFO:teuthology.orchestra.run.smithi022.stderr:1+0 records out 2024-03-28T17:56:44.635 INFO:teuthology.orchestra.run.smithi022.stderr:512 bytes copied, 0.000268559 s, 1.9 MB/s 2024-03-28T17:56:44.639 DEBUG:teuthology.orchestra.run.smithi022:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-03-28T17:56:44.698 DEBUG:teuthology.orchestra.run.smithi079:> set -ex 2024-03-28T17:56:44.698 DEBUG:teuthology.orchestra.run.smithi079:> dd if=/scratch_devs of=/dev/stdout 2024-03-28T17:56:44.758 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-03-28T17:56:44.759 DEBUG:teuthology.orchestra.run.smithi079:> stat /dev/vg_nvme/lv_1 2024-03-28T17:56:44.810 INFO:teuthology.orchestra.run.smithi079.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-03-28T17:56:44.810 INFO:teuthology.orchestra.run.smithi079.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-28T17:56:44.810 INFO:teuthology.orchestra.run.smithi079.stdout:Device: 5h/5d Inode: 633 Links: 1 2024-03-28T17:56:44.810 INFO:teuthology.orchestra.run.smithi079.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-28T17:56:44.811 INFO:teuthology.orchestra.run.smithi079.stdout:Context: system_u:object_r:device_t:s0 2024-03-28T17:56:44.811 INFO:teuthology.orchestra.run.smithi079.stdout:Access: 2024-03-28 17:56:25.053904877 +0000 2024-03-28T17:56:44.811 INFO:teuthology.orchestra.run.smithi079.stdout:Modify: 2024-03-28 17:49:21.258083697 +0000 2024-03-28T17:56:44.811 INFO:teuthology.orchestra.run.smithi079.stdout:Change: 2024-03-28 17:49:21.258083697 +0000 2024-03-28T17:56:44.811 INFO:teuthology.orchestra.run.smithi079.stdout: Birth: 2024-03-28 17:49:21.258083697 +0000 2024-03-28T17:56:44.811 DEBUG:teuthology.orchestra.run.smithi079:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-03-28T17:56:44.898 INFO:teuthology.orchestra.run.smithi079.stderr:1+0 records in 2024-03-28T17:56:44.898 INFO:teuthology.orchestra.run.smithi079.stderr:1+0 records out 2024-03-28T17:56:44.898 INFO:teuthology.orchestra.run.smithi079.stderr:512 bytes copied, 0.000371913 s, 1.4 MB/s 2024-03-28T17:56:44.902 DEBUG:teuthology.orchestra.run.smithi079:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-03-28T17:56:44.954 DEBUG:teuthology.orchestra.run.smithi079:> stat /dev/vg_nvme/lv_2 2024-03-28T17:56:45.000 INFO:teuthology.orchestra.run.smithi079.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-03-28T17:56:45.001 INFO:teuthology.orchestra.run.smithi079.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-28T17:56:45.001 INFO:teuthology.orchestra.run.smithi079.stdout:Device: 5h/5d Inode: 653 Links: 1 2024-03-28T17:56:45.001 INFO:teuthology.orchestra.run.smithi079.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-28T17:56:45.001 INFO:teuthology.orchestra.run.smithi079.stdout:Context: system_u:object_r:device_t:s0 2024-03-28T17:56:45.001 INFO:teuthology.orchestra.run.smithi079.stdout:Access: 2024-03-28 17:56:25.053904877 +0000 2024-03-28T17:56:45.001 INFO:teuthology.orchestra.run.smithi079.stdout:Modify: 2024-03-28 17:49:21.874069575 +0000 2024-03-28T17:56:45.001 INFO:teuthology.orchestra.run.smithi079.stdout:Change: 2024-03-28 17:49:21.874069575 +0000 2024-03-28T17:56:45.001 INFO:teuthology.orchestra.run.smithi079.stdout: Birth: 2024-03-28 17:49:21.874069575 +0000 2024-03-28T17:56:45.001 DEBUG:teuthology.orchestra.run.smithi079:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-03-28T17:56:45.131 INFO:teuthology.orchestra.run.smithi079.stderr:1+0 records in 2024-03-28T17:56:45.131 INFO:teuthology.orchestra.run.smithi079.stderr:1+0 records out 2024-03-28T17:56:45.131 INFO:teuthology.orchestra.run.smithi079.stderr:512 bytes copied, 0.000372962 s, 1.4 MB/s 2024-03-28T17:56:45.135 DEBUG:teuthology.orchestra.run.smithi079:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-03-28T17:56:45.190 DEBUG:teuthology.orchestra.run.smithi079:> stat /dev/vg_nvme/lv_3 2024-03-28T17:56:45.238 INFO:teuthology.orchestra.run.smithi079.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-03-28T17:56:45.239 INFO:teuthology.orchestra.run.smithi079.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-28T17:56:45.239 INFO:teuthology.orchestra.run.smithi079.stdout:Device: 5h/5d Inode: 670 Links: 1 2024-03-28T17:56:45.239 INFO:teuthology.orchestra.run.smithi079.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-28T17:56:45.239 INFO:teuthology.orchestra.run.smithi079.stdout:Context: system_u:object_r:device_t:s0 2024-03-28T17:56:45.239 INFO:teuthology.orchestra.run.smithi079.stdout:Access: 2024-03-28 17:56:25.054904854 +0000 2024-03-28T17:56:45.239 INFO:teuthology.orchestra.run.smithi079.stdout:Modify: 2024-03-28 17:49:22.438056644 +0000 2024-03-28T17:56:45.239 INFO:teuthology.orchestra.run.smithi079.stdout:Change: 2024-03-28 17:49:22.438056644 +0000 2024-03-28T17:56:45.239 INFO:teuthology.orchestra.run.smithi079.stdout: Birth: 2024-03-28 17:49:22.438056644 +0000 2024-03-28T17:56:45.239 DEBUG:teuthology.orchestra.run.smithi079:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-03-28T17:56:45.323 INFO:teuthology.orchestra.run.smithi079.stderr:1+0 records in 2024-03-28T17:56:45.323 INFO:teuthology.orchestra.run.smithi079.stderr:1+0 records out 2024-03-28T17:56:45.323 INFO:teuthology.orchestra.run.smithi079.stderr:512 bytes copied, 0.000380021 s, 1.3 MB/s 2024-03-28T17:56:45.326 DEBUG:teuthology.orchestra.run.smithi079:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-03-28T17:56:45.382 DEBUG:teuthology.orchestra.run.smithi079:> stat /dev/vg_nvme/lv_4 2024-03-28T17:56:45.428 INFO:teuthology.orchestra.run.smithi079.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-03-28T17:56:45.428 INFO:teuthology.orchestra.run.smithi079.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-28T17:56:45.428 INFO:teuthology.orchestra.run.smithi079.stdout:Device: 5h/5d Inode: 689 Links: 1 2024-03-28T17:56:45.428 INFO:teuthology.orchestra.run.smithi079.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-28T17:56:45.428 INFO:teuthology.orchestra.run.smithi079.stdout:Context: system_u:object_r:device_t:s0 2024-03-28T17:56:45.428 INFO:teuthology.orchestra.run.smithi079.stdout:Access: 2024-03-28 17:56:25.055904831 +0000 2024-03-28T17:56:45.428 INFO:teuthology.orchestra.run.smithi079.stdout:Modify: 2024-03-28 17:49:22.994043897 +0000 2024-03-28T17:56:45.428 INFO:teuthology.orchestra.run.smithi079.stdout:Change: 2024-03-28 17:49:22.994043897 +0000 2024-03-28T17:56:45.428 INFO:teuthology.orchestra.run.smithi079.stdout: Birth: 2024-03-28 17:49:22.994043897 +0000 2024-03-28T17:56:45.428 DEBUG:teuthology.orchestra.run.smithi079:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-03-28T17:56:45.559 INFO:teuthology.orchestra.run.smithi079.stderr:1+0 records in 2024-03-28T17:56:45.560 INFO:teuthology.orchestra.run.smithi079.stderr:1+0 records out 2024-03-28T17:56:45.560 INFO:teuthology.orchestra.run.smithi079.stderr:512 bytes copied, 0.000321424 s, 1.6 MB/s 2024-03-28T17:56:45.564 DEBUG:teuthology.orchestra.run.smithi079:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-03-28T17:56:45.617 DEBUG:teuthology.orchestra.run.smithi134:> set -ex 2024-03-28T17:56:45.617 DEBUG:teuthology.orchestra.run.smithi134:> dd if=/scratch_devs of=/dev/stdout 2024-03-28T17:56:45.657 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-03-28T17:56:45.657 DEBUG:teuthology.orchestra.run.smithi134:> stat /dev/vg_nvme/lv_1 2024-03-28T17:56:45.746 INFO:teuthology.orchestra.run.smithi134.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-03-28T17:56:45.746 INFO:teuthology.orchestra.run.smithi134.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-28T17:56:45.746 INFO:teuthology.orchestra.run.smithi134.stdout:Device: 5h/5d Inode: 633 Links: 1 2024-03-28T17:56:45.746 INFO:teuthology.orchestra.run.smithi134.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-28T17:56:45.746 INFO:teuthology.orchestra.run.smithi134.stdout:Context: system_u:object_r:device_t:s0 2024-03-28T17:56:45.746 INFO:teuthology.orchestra.run.smithi134.stdout:Access: 2024-03-28 17:56:45.690826198 +0000 2024-03-28T17:56:45.746 INFO:teuthology.orchestra.run.smithi134.stdout:Modify: 2024-03-28 17:49:22.504392062 +0000 2024-03-28T17:56:45.747 INFO:teuthology.orchestra.run.smithi134.stdout:Change: 2024-03-28 17:49:22.504392062 +0000 2024-03-28T17:56:45.747 INFO:teuthology.orchestra.run.smithi134.stdout: Birth: 2024-03-28 17:49:22.504392062 +0000 2024-03-28T17:56:45.747 DEBUG:teuthology.orchestra.run.smithi134:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-03-28T17:56:45.827 INFO:teuthology.orchestra.run.smithi134.stderr:1+0 records in 2024-03-28T17:56:45.827 INFO:teuthology.orchestra.run.smithi134.stderr:1+0 records out 2024-03-28T17:56:45.827 INFO:teuthology.orchestra.run.smithi134.stderr:512 bytes copied, 0.000334595 s, 1.5 MB/s 2024-03-28T17:56:45.832 DEBUG:teuthology.orchestra.run.smithi134:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-03-28T17:56:45.875 DEBUG:teuthology.orchestra.run.smithi134:> stat /dev/vg_nvme/lv_2 2024-03-28T17:56:45.954 INFO:teuthology.orchestra.run.smithi134.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-03-28T17:56:45.954 INFO:teuthology.orchestra.run.smithi134.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-28T17:56:45.954 INFO:teuthology.orchestra.run.smithi134.stdout:Device: 5h/5d Inode: 651 Links: 1 2024-03-28T17:56:45.954 INFO:teuthology.orchestra.run.smithi134.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-28T17:56:45.954 INFO:teuthology.orchestra.run.smithi134.stdout:Context: system_u:object_r:device_t:s0 2024-03-28T17:56:45.954 INFO:teuthology.orchestra.run.smithi134.stdout:Access: 2024-03-28 17:56:45.690826198 +0000 2024-03-28T17:56:45.954 INFO:teuthology.orchestra.run.smithi134.stdout:Modify: 2024-03-28 17:49:23.097377766 +0000 2024-03-28T17:56:45.954 INFO:teuthology.orchestra.run.smithi134.stdout:Change: 2024-03-28 17:49:23.097377766 +0000 2024-03-28T17:56:45.954 INFO:teuthology.orchestra.run.smithi134.stdout: Birth: 2024-03-28 17:49:23.097377766 +0000 2024-03-28T17:56:45.955 DEBUG:teuthology.orchestra.run.smithi134:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-03-28T17:56:46.085 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:56:45 smithi134 ceph-mon[35037]: pgmap v41: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-28T17:56:46.089 INFO:teuthology.orchestra.run.smithi134.stderr:1+0 records in 2024-03-28T17:56:46.089 INFO:teuthology.orchestra.run.smithi134.stderr:1+0 records out 2024-03-28T17:56:46.089 INFO:teuthology.orchestra.run.smithi134.stderr:512 bytes copied, 0.000316343 s, 1.6 MB/s 2024-03-28T17:56:46.094 DEBUG:teuthology.orchestra.run.smithi134:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-03-28T17:56:46.137 DEBUG:teuthology.orchestra.run.smithi134:> stat /dev/vg_nvme/lv_3 2024-03-28T17:56:46.182 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:56:45 smithi079 ceph-mon[35312]: pgmap v41: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-28T17:56:46.216 INFO:teuthology.orchestra.run.smithi134.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-03-28T17:56:46.216 INFO:teuthology.orchestra.run.smithi134.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-28T17:56:46.216 INFO:teuthology.orchestra.run.smithi134.stdout:Device: 5h/5d Inode: 670 Links: 1 2024-03-28T17:56:46.216 INFO:teuthology.orchestra.run.smithi134.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-28T17:56:46.216 INFO:teuthology.orchestra.run.smithi134.stdout:Context: system_u:object_r:device_t:s0 2024-03-28T17:56:46.216 INFO:teuthology.orchestra.run.smithi134.stdout:Access: 2024-03-28 17:56:45.999818834 +0000 2024-03-28T17:56:46.217 INFO:teuthology.orchestra.run.smithi134.stdout:Modify: 2024-03-28 17:49:23.658364242 +0000 2024-03-28T17:56:46.217 INFO:teuthology.orchestra.run.smithi134.stdout:Change: 2024-03-28 17:49:23.658364242 +0000 2024-03-28T17:56:46.217 INFO:teuthology.orchestra.run.smithi134.stdout: Birth: 2024-03-28 17:49:23.658364242 +0000 2024-03-28T17:56:46.217 DEBUG:teuthology.orchestra.run.smithi134:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-03-28T17:56:46.281 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:56:45 smithi022 ceph-mon[31985]: pgmap v41: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-28T17:56:46.334 INFO:teuthology.orchestra.run.smithi134.stderr:1+0 records in 2024-03-28T17:56:46.334 INFO:teuthology.orchestra.run.smithi134.stderr:1+0 records out 2024-03-28T17:56:46.334 INFO:teuthology.orchestra.run.smithi134.stderr:512 bytes copied, 0.000313903 s, 1.6 MB/s 2024-03-28T17:56:46.336 DEBUG:teuthology.orchestra.run.smithi134:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-03-28T17:56:46.375 DEBUG:teuthology.orchestra.run.smithi134:> stat /dev/vg_nvme/lv_4 2024-03-28T17:56:46.461 INFO:teuthology.orchestra.run.smithi134.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-03-28T17:56:46.461 INFO:teuthology.orchestra.run.smithi134.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-28T17:56:46.461 INFO:teuthology.orchestra.run.smithi134.stdout:Device: 5h/5d Inode: 690 Links: 1 2024-03-28T17:56:46.461 INFO:teuthology.orchestra.run.smithi134.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-28T17:56:46.461 INFO:teuthology.orchestra.run.smithi134.stdout:Context: system_u:object_r:device_t:s0 2024-03-28T17:56:46.461 INFO:teuthology.orchestra.run.smithi134.stdout:Access: 2024-03-28 17:56:45.999818834 +0000 2024-03-28T17:56:46.462 INFO:teuthology.orchestra.run.smithi134.stdout:Modify: 2024-03-28 17:49:24.231350429 +0000 2024-03-28T17:56:46.462 INFO:teuthology.orchestra.run.smithi134.stdout:Change: 2024-03-28 17:49:24.231350429 +0000 2024-03-28T17:56:46.462 INFO:teuthology.orchestra.run.smithi134.stdout: Birth: 2024-03-28 17:49:24.231350429 +0000 2024-03-28T17:56:46.462 DEBUG:teuthology.orchestra.run.smithi134:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-03-28T17:56:46.590 INFO:teuthology.orchestra.run.smithi134.stderr:1+0 records in 2024-03-28T17:56:46.590 INFO:teuthology.orchestra.run.smithi134.stderr:1+0 records out 2024-03-28T17:56:46.590 INFO:teuthology.orchestra.run.smithi134.stderr:512 bytes copied, 0.000358141 s, 1.4 MB/s 2024-03-28T17:56:46.593 DEBUG:teuthology.orchestra.run.smithi134:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-03-28T17:56:46.641 INFO:tasks.cephadm:Deploying osd.0 on smithi022 with /dev/vg_nvme/lv_4... 2024-03-28T17:56:46.642 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fd220748-ed2b-11ee-95d0-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-03-28T17:56:47.096 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/fd220748-ed2b-11ee-95d0-87774f69a715/mon.a/config 2024-03-28T17:56:47.174 INFO:journalctl@ceph.mgr.z.smithi134.stdout:Mar 28 17:56:46 smithi134 systemd[1]: Started Ceph mgr.z for fd220748-ed2b-11ee-95d0-87774f69a715. 2024-03-28T17:56:47.925 INFO:journalctl@ceph.mgr.z.smithi134.stdout:Mar 28 17:56:47 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mgr-z[37033]: 2024-03-28T17:56:47.486+0000 7f25838b0200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-03-28T17:56:47.925 INFO:journalctl@ceph.mgr.z.smithi134.stdout:Mar 28 17:56:47 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mgr-z[37033]: 2024-03-28T17:56:47.605+0000 7f25838b0200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-03-28T17:56:48.277 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:56:47 smithi022 ceph-mon[31985]: pgmap v42: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-28T17:56:48.277 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:56:47 smithi022 ceph-mon[31985]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T17:56:48.277 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:56:47 smithi022 ceph-mon[31985]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T17:56:48.277 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:56:47 smithi022 ceph-mon[31985]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T17:56:48.277 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:56:47 smithi022 ceph-mon[31985]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-03-28T17:56:48.277 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:56:47 smithi022 ceph-mon[31985]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-03-28T17:56:48.277 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:56:47 smithi022 ceph-mon[31985]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-03-28T17:56:48.277 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:56:47 smithi022 ceph-mon[31985]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T17:56:48.277 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:56:47 smithi022 ceph-mon[31985]: Deploying daemon mgr.y on smithi079 2024-03-28T17:56:48.317 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:56:47 smithi079 ceph-mon[35312]: pgmap v42: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-28T17:56:48.317 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:56:47 smithi079 ceph-mon[35312]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T17:56:48.317 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:56:47 smithi079 ceph-mon[35312]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T17:56:48.317 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:56:47 smithi079 ceph-mon[35312]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T17:56:48.317 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:56:47 smithi079 ceph-mon[35312]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-03-28T17:56:48.317 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:56:47 smithi079 ceph-mon[35312]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-03-28T17:56:48.317 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:56:47 smithi079 ceph-mon[35312]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-03-28T17:56:48.317 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:56:47 smithi079 ceph-mon[35312]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T17:56:48.317 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:56:47 smithi079 ceph-mon[35312]: Deploying daemon mgr.y on smithi079 2024-03-28T17:56:48.395 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:56:47 smithi134 ceph-mon[35037]: pgmap v42: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-28T17:56:48.395 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:56:47 smithi134 ceph-mon[35037]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T17:56:48.395 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:56:47 smithi134 ceph-mon[35037]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T17:56:48.395 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:56:47 smithi134 ceph-mon[35037]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T17:56:48.395 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:56:47 smithi134 ceph-mon[35037]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-03-28T17:56:48.395 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:56:47 smithi134 ceph-mon[35037]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-03-28T17:56:48.395 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:56:47 smithi134 ceph-mon[35037]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-03-28T17:56:48.396 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:56:47 smithi134 ceph-mon[35037]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T17:56:48.396 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:56:47 smithi134 ceph-mon[35037]: Deploying daemon mgr.y on smithi079 2024-03-28T17:56:48.675 INFO:journalctl@ceph.mgr.z.smithi134.stdout:Mar 28 17:56:48 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mgr-z[37033]: 2024-03-28T17:56:48.394+0000 7f25838b0200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-03-28T17:56:48.675 INFO:journalctl@ceph.mgr.z.smithi134.stdout:Mar 28 17:56:48 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mgr-z[37033]: 2024-03-28T17:56:48.513+0000 7f25838b0200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-03-28T17:56:49.174 INFO:journalctl@ceph.mgr.z.smithi134.stdout:Mar 28 17:56:48 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mgr-z[37033]: 2024-03-28T17:56:48.688+0000 7f25838b0200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-03-28T17:56:49.175 INFO:journalctl@ceph.mgr.z.smithi134.stdout:Mar 28 17:56:48 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mgr-z[37033]: 2024-03-28T17:56:48.809+0000 7f25838b0200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-03-28T17:56:49.175 INFO:journalctl@ceph.mgr.z.smithi134.stdout:Mar 28 17:56:48 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mgr-z[37033]: 2024-03-28T17:56:48.917+0000 7f25838b0200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-03-28T17:56:49.512 INFO:journalctl@ceph.mgr.z.smithi134.stdout:Mar 28 17:56:49 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mgr-z[37033]: 2024-03-28T17:56:49.260+0000 7f25838b0200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-03-28T17:56:49.512 INFO:journalctl@ceph.mgr.z.smithi134.stdout:Mar 28 17:56:49 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mgr-z[37033]: 2024-03-28T17:56:49.400+0000 7f25838b0200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-03-28T17:56:49.512 INFO:journalctl@ceph.mgr.z.smithi134.stdout:Mar 28 17:56:49 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mgr-z[37033]: 2024-03-28T17:56:49.510+0000 7f25838b0200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-03-28T17:56:49.854 INFO:journalctl@ceph.mgr.z.smithi134.stdout:Mar 28 17:56:49 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mgr-z[37033]: 2024-03-28T17:56:49.635+0000 7f25838b0200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-03-28T17:56:49.854 INFO:journalctl@ceph.mgr.z.smithi134.stdout:Mar 28 17:56:49 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mgr-z[37033]: 2024-03-28T17:56:49.854+0000 7f25838b0200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-03-28T17:56:49.936 INFO:teuthology.orchestra.run.smithi022.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-03-28T17:56:49.936 INFO:teuthology.orchestra.run.smithi022.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-03-28T17:56:49.937 INFO:teuthology.orchestra.run.smithi022.stderr: stderr: 10+0 records in 2024-03-28T17:56:49.937 INFO:teuthology.orchestra.run.smithi022.stderr:10+0 records out 2024-03-28T17:56:49.937 INFO:teuthology.orchestra.run.smithi022.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0340814 s, 308 MB/s 2024-03-28T17:56:49.937 INFO:teuthology.orchestra.run.smithi022.stderr:--> Zapping successful for: 2024-03-28T17:56:50.174 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:56:49 smithi134 ceph-mon[35037]: pgmap v43: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-28T17:56:50.182 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:56:49 smithi079 ceph-mon[35312]: pgmap v43: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-28T17:56:50.185 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:56:49 smithi022 ceph-mon[31985]: pgmap v43: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-28T17:56:50.540 INFO:journalctl@ceph.mgr.y.smithi079.stdout:Mar 28 17:56:50 smithi079 podman[36684]: 2024-03-28 17:56:50.529764961 +0000 UTC m=+0.372857642 container init 35ad6cc68d5dd66112a2439ddb89cf214de8b98e0cccdfb9b10e965029496b7e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mgr-y, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240319, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.vendor=CentOS, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-03-28T17:56:50.540 INFO:journalctl@ceph.mgr.y.smithi079.stdout:Mar 28 17:56:50 smithi079 podman[36684]: 2024-03-28 17:56:50.539861406 +0000 UTC m=+0.382954050 container start 35ad6cc68d5dd66112a2439ddb89cf214de8b98e0cccdfb9b10e965029496b7e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mgr-y, org.label-schema.schema-version=1.0, ceph=True, GIT_BRANCH=HEAD, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240319, io.buildah.version=1.33.5, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, org.label-schema.license=GPLv2) 2024-03-28T17:56:50.815 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fd220748-ed2b-11ee-95d0-87774f69a715 -- ceph orch daemon add osd smithi022:vg_nvme/lv_4 2024-03-28T17:56:50.932 INFO:journalctl@ceph.mgr.y.smithi079.stdout:Mar 28 17:56:50 smithi079 ceph-mgr[36703]: -- 172.21.15.79:0/1253848445 <== mon.2 v2:172.21.15.79:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55d7e174a5a0 con 0x55d7e171f000 2024-03-28T17:56:50.933 INFO:journalctl@ceph.mgr.y.smithi079.stdout:Mar 28 17:56:50 smithi079 bash[36684]: 35ad6cc68d5dd66112a2439ddb89cf214de8b98e0cccdfb9b10e965029496b7e 2024-03-28T17:56:50.933 INFO:journalctl@ceph.mgr.y.smithi079.stdout:Mar 28 17:56:50 smithi079 systemd[1]: Started Ceph mgr.y for fd220748-ed2b-11ee-95d0-87774f69a715. 2024-03-28T17:56:51.267 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/fd220748-ed2b-11ee-95d0-87774f69a715/mon.a/config 2024-03-28T17:56:51.424 INFO:journalctl@ceph.mgr.z.smithi134.stdout:Mar 28 17:56:51 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mgr-z[37033]: 2024-03-28T17:56:51.162+0000 7f25838b0200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-03-28T17:56:51.424 INFO:journalctl@ceph.mgr.z.smithi134.stdout:Mar 28 17:56:51 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mgr-z[37033]: 2024-03-28T17:56:51.341+0000 7f25838b0200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-03-28T17:56:51.682 INFO:journalctl@ceph.mgr.y.smithi079.stdout:Mar 28 17:56:51 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mgr-y[36699]: 2024-03-28T17:56:51.303+0000 7fc7c2d22200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-03-28T17:56:51.682 INFO:journalctl@ceph.mgr.y.smithi079.stdout:Mar 28 17:56:51 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mgr-y[36699]: 2024-03-28T17:56:51.430+0000 7fc7c2d22200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-03-28T17:56:52.002 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:56:51 smithi022 ceph-mon[31985]: pgmap v44: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-28T17:56:52.002 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:56:51 smithi022 ceph-mon[31985]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T17:56:52.002 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:56:51 smithi022 ceph-mon[31985]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T17:56:52.002 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:56:51 smithi022 ceph-mon[31985]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T17:56:52.002 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:56:51 smithi022 ceph-mon[31985]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T17:56:52.002 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:56:51 smithi022 ceph-mon[31985]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T17:56:52.174 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:56:51 smithi134 ceph-mon[35037]: pgmap v44: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-28T17:56:52.174 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:56:51 smithi134 ceph-mon[35037]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T17:56:52.175 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:56:51 smithi134 ceph-mon[35037]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T17:56:52.175 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:56:51 smithi134 ceph-mon[35037]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T17:56:52.175 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:56:51 smithi134 ceph-mon[35037]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T17:56:52.175 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:56:51 smithi134 ceph-mon[35037]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T17:56:52.175 INFO:journalctl@ceph.mgr.z.smithi134.stdout:Mar 28 17:56:51 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mgr-z[37033]: 2024-03-28T17:56:51.769+0000 7f25838b0200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-03-28T17:56:52.175 INFO:journalctl@ceph.mgr.z.smithi134.stdout:Mar 28 17:56:51 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mgr-z[37033]: 2024-03-28T17:56:51.891+0000 7f25838b0200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-03-28T17:56:52.182 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:56:51 smithi079 ceph-mon[35312]: pgmap v44: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-28T17:56:52.182 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:56:51 smithi079 ceph-mon[35312]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T17:56:52.182 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:56:51 smithi079 ceph-mon[35312]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T17:56:52.182 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:56:51 smithi079 ceph-mon[35312]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T17:56:52.182 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:56:51 smithi079 ceph-mon[35312]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T17:56:52.182 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:56:51 smithi079 ceph-mon[35312]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T17:56:52.444 INFO:journalctl@ceph.mgr.z.smithi134.stdout:Mar 28 17:56:52 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mgr-z[37033]: 2024-03-28T17:56:52.184+0000 7f25838b0200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-03-28T17:56:52.444 INFO:journalctl@ceph.mgr.z.smithi134.stdout:Mar 28 17:56:52 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mgr-z[37033]: 2024-03-28T17:56:52.443+0000 7f25838b0200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-03-28T17:56:52.578 INFO:journalctl@ceph.mgr.y.smithi079.stdout:Mar 28 17:56:52 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mgr-y[36699]: 2024-03-28T17:56:52.297+0000 7fc7c2d22200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-03-28T17:56:52.578 INFO:journalctl@ceph.mgr.y.smithi079.stdout:Mar 28 17:56:52 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mgr-y[36699]: 2024-03-28T17:56:52.419+0000 7fc7c2d22200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-03-28T17:56:52.854 INFO:journalctl@ceph.mgr.y.smithi079.stdout:Mar 28 17:56:52 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mgr-y[36699]: 2024-03-28T17:56:52.612+0000 7fc7c2d22200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-03-28T17:56:52.854 INFO:journalctl@ceph.mgr.y.smithi079.stdout:Mar 28 17:56:52 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mgr-y[36699]: 2024-03-28T17:56:52.741+0000 7fc7c2d22200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-03-28T17:56:52.854 INFO:journalctl@ceph.mgr.y.smithi079.stdout:Mar 28 17:56:52 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mgr-y[36699]: 2024-03-28T17:56:52.855+0000 7fc7c2d22200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-03-28T17:56:53.173 INFO:journalctl@ceph.mgr.z.smithi134.stdout:Mar 28 17:56:52 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mgr-z[37033]: 2024-03-28T17:56:52.826+0000 7f25838b0200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-03-28T17:56:53.424 INFO:journalctl@ceph.mgr.z.smithi134.stdout:Mar 28 17:56:53 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mgr-z[37033]: 2024-03-28T17:56:53.173+0000 7f25838b0200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-03-28T17:56:53.437 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T17:56:53.437+0000 7f4292800700 1 -- 172.21.15.22:0/1731922919 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f428c1055d0 msgr2=0x7f428c105a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:56:53.437 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T17:56:53.437+0000 7f4292800700 1 --2- 172.21.15.22:0/1731922919 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f428c1055d0 0x7f428c105a30 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f42740099c0 tx=0x7f4274009cd0 comp rx=0 tx=0).stop 2024-03-28T17:56:53.438 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T17:56:53.437+0000 7f4292800700 1 -- 172.21.15.22:0/1731922919 shutdown_connections 2024-03-28T17:56:53.438 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T17:56:53.437+0000 7f4292800700 1 --2- 172.21.15.22:0/1731922919 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f428c105f70 0x7f428c10c820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:56:53.438 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T17:56:53.437+0000 7f4292800700 1 --2- 172.21.15.22:0/1731922919 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f428c1043d0 0x7f428c1047f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:56:53.438 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T17:56:53.437+0000 7f4292800700 1 --2- 172.21.15.22:0/1731922919 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f428c1055d0 0x7f428c105a30 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:56:53.438 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T17:56:53.437+0000 7f4292800700 1 -- 172.21.15.22:0/1731922919 >> 172.21.15.22:0/1731922919 conn(0x7f428c0ff950 msgr2=0x7f428c101db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T17:56:53.438 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T17:56:53.437+0000 7f4292800700 1 -- 172.21.15.22:0/1731922919 shutdown_connections 2024-03-28T17:56:53.438 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T17:56:53.437+0000 7f4292800700 1 -- 172.21.15.22:0/1731922919 wait complete. 2024-03-28T17:56:53.440 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T17:56:53.438+0000 7f4292800700 1 Processor -- start 2024-03-28T17:56:53.440 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T17:56:53.438+0000 7f4292800700 1 -- start start 2024-03-28T17:56:53.440 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T17:56:53.438+0000 7f4292800700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f428c1043d0 0x7f428c19a5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:56:53.440 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T17:56:53.439+0000 7f4292800700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f428c1055d0 0x7f428c19aaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:56:53.440 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T17:56:53.439+0000 7f4292800700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f428c105f70 0x7f428c1a1b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:56:53.440 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T17:56:53.439+0000 7f4292800700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f428c10fff0 con 0x7f428c105f70 2024-03-28T17:56:53.440 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T17:56:53.439+0000 7f4292800700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f428c10fe70 con 0x7f428c1043d0 2024-03-28T17:56:53.440 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T17:56:53.439+0000 7f4292800700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f428c110170 con 0x7f428c1055d0 2024-03-28T17:56:53.440 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T17:56:53.439+0000 7f428be00700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f428c105f70 0x7f428c1a1b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:56:53.441 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T17:56:53.439+0000 7f428be00700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f428c105f70 0x7f428c1a1b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:43994/0 (socket says 172.21.15.22:43994) 2024-03-28T17:56:53.441 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T17:56:53.439+0000 7f428be00700 1 -- 172.21.15.22:0/4067071193 learned_addr learned my addr 172.21.15.22:0/4067071193 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-28T17:56:53.441 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T17:56:53.439+0000 7f428b400700 1 --2- 172.21.15.22:0/4067071193 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f428c1043d0 0x7f428c19a5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:56:53.441 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T17:56:53.439+0000 7f4283e00700 1 --2- 172.21.15.22:0/4067071193 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f428c1055d0 0x7f428c19aaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:56:53.441 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T17:56:53.439+0000 7f428be00700 1 -- 172.21.15.22:0/4067071193 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f428c1055d0 msgr2=0x7f428c19aaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:56:53.441 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T17:56:53.439+0000 7f428be00700 1 --2- 172.21.15.22:0/4067071193 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f428c1055d0 0x7f428c19aaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:56:53.441 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T17:56:53.439+0000 7f428be00700 1 -- 172.21.15.22:0/4067071193 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f428c1043d0 msgr2=0x7f428c19a5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:56:53.441 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T17:56:53.439+0000 7f428be00700 1 --2- 172.21.15.22:0/4067071193 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f428c1043d0 0x7f428c19a5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:56:53.441 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T17:56:53.439+0000 7f428be00700 1 -- 172.21.15.22:0/4067071193 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f428c1a2210 con 0x7f428c105f70 2024-03-28T17:56:53.441 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T17:56:53.440+0000 7f428be00700 1 --2- 172.21.15.22:0/4067071193 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f428c105f70 0x7f428c1a1b70 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f428400ef60 tx=0x7f428400ef90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T17:56:53.441 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T17:56:53.440+0000 7f4288c00700 1 -- 172.21.15.22:0/4067071193 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42840092e0 con 0x7f428c105f70 2024-03-28T17:56:53.441 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T17:56:53.440+0000 7f4288c00700 1 -- 172.21.15.22:0/4067071193 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f4284004bb0 con 0x7f428c105f70 2024-03-28T17:56:53.441 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T17:56:53.440+0000 7f4292800700 1 -- 172.21.15.22:0/4067071193 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f428c1a24a0 con 0x7f428c105f70 2024-03-28T17:56:53.441 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T17:56:53.440+0000 7f4288c00700 1 -- 172.21.15.22:0/4067071193 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4284004280 con 0x7f428c105f70 2024-03-28T17:56:53.442 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T17:56:53.440+0000 7f4292800700 1 -- 172.21.15.22:0/4067071193 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f428c1a2a30 con 0x7f428c105f70 2024-03-28T17:56:53.442 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T17:56:53.442+0000 7f4288c00700 1 -- 172.21.15.22:0/4067071193 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 13) v1 ==== 46162+0+0 (secure 0 0 0) 0x7f428401b030 con 0x7f428c105f70 2024-03-28T17:56:53.442 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T17:56:53.442+0000 7f4292800700 1 -- 172.21.15.22:0/4067071193 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f428c068e50 con 0x7f428c105f70 2024-03-28T17:56:53.442 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T17:56:53.442+0000 7f4288c00700 1 --2- 172.21.15.22:0/4067071193 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7f4270039200 0x7f427003b6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:56:53.443 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T17:56:53.442+0000 7f4288c00700 1 -- 172.21.15.22:0/4067071193 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f428404bb50 con 0x7f428c105f70 2024-03-28T17:56:53.443 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T17:56:53.443+0000 7f428b400700 1 --2- 172.21.15.22:0/4067071193 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7f4270039200 0x7f427003b6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:56:53.443 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T17:56:53.443+0000 7f428b400700 1 --2- 172.21.15.22:0/4067071193 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7f4270039200 0x7f427003b6c0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f428c105430 tx=0x7f427c00b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T17:56:53.448 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T17:56:53.448+0000 7f4288c00700 1 -- 172.21.15.22:0/4067071193 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f4284022020 con 0x7f428c105f70 2024-03-28T17:56:53.463 INFO:journalctl@ceph.mgr.y.smithi079.stdout:Mar 28 17:56:53 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mgr-y[36699]: 2024-03-28T17:56:53.203+0000 7fc7c2d22200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-03-28T17:56:53.463 INFO:journalctl@ceph.mgr.y.smithi079.stdout:Mar 28 17:56:53 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mgr-y[36699]: 2024-03-28T17:56:53.348+0000 7fc7c2d22200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-03-28T17:56:53.687 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T17:56:53.687+0000 7f4292800700 1 -- 172.21.15.22:0/4067071193 --> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi022:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) v1 -- 0x7f428c0625d0 con 0x7f4270039200 2024-03-28T17:56:53.809 INFO:journalctl@ceph.mgr.y.smithi079.stdout:Mar 28 17:56:53 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mgr-y[36699]: 2024-03-28T17:56:53.463+0000 7fc7c2d22200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-03-28T17:56:53.810 INFO:journalctl@ceph.mgr.y.smithi079.stdout:Mar 28 17:56:53 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mgr-y[36699]: 2024-03-28T17:56:53.587+0000 7fc7c2d22200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-03-28T17:56:53.810 INFO:journalctl@ceph.mgr.y.smithi079.stdout:Mar 28 17:56:53 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mgr-y[36699]: 2024-03-28T17:56:53.809+0000 7fc7c2d22200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-03-28T17:56:53.924 INFO:journalctl@ceph.mgr.z.smithi134.stdout:Mar 28 17:56:53 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mgr-z[37033]: 2024-03-28T17:56:53.474+0000 7f25838b0200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-03-28T17:56:54.183 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:56:53 smithi079 ceph-mon[35312]: pgmap v45: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-28T17:56:54.183 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:56:53 smithi079 ceph-mon[35312]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T17:56:54.183 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:56:53 smithi079 ceph-mon[35312]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T17:56:54.183 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:56:53 smithi079 ceph-mon[35312]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T17:56:54.183 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:56:53 smithi079 ceph-mon[35312]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T17:56:54.183 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:56:53 smithi079 ceph-mon[35312]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-28T17:56:54.183 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:56:53 smithi079 ceph-mon[35312]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-28T17:56:54.183 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:56:53 smithi079 ceph-mon[35312]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T17:56:54.282 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:56:53 smithi022 ceph-mon[31985]: pgmap v45: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-28T17:56:54.282 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:56:53 smithi022 ceph-mon[31985]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T17:56:54.282 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:56:53 smithi022 ceph-mon[31985]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T17:56:54.282 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:56:53 smithi022 ceph-mon[31985]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T17:56:54.282 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:56:53 smithi022 ceph-mon[31985]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T17:56:54.282 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:56:53 smithi022 ceph-mon[31985]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-28T17:56:54.282 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:56:53 smithi022 ceph-mon[31985]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-28T17:56:54.282 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:56:53 smithi022 ceph-mon[31985]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T17:56:54.425 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:56:53 smithi134 ceph-mon[35037]: pgmap v45: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-28T17:56:54.425 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:56:53 smithi134 ceph-mon[35037]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T17:56:54.425 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:56:53 smithi134 ceph-mon[35037]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T17:56:54.425 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:56:53 smithi134 ceph-mon[35037]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T17:56:54.425 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:56:53 smithi134 ceph-mon[35037]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T17:56:54.425 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:56:53 smithi134 ceph-mon[35037]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-28T17:56:54.425 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:56:53 smithi134 ceph-mon[35037]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-28T17:56:54.426 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:56:53 smithi134 ceph-mon[35037]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T17:56:54.792 INFO:journalctl@ceph.mgr.z.smithi134.stdout:Mar 28 17:56:54 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mgr-z[37033]: 2024-03-28T17:56:54.496+0000 7f25838b0200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-03-28T17:56:55.094 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:56:54 smithi134 ceph-mon[35037]: from='client.14229 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi022:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-03-28T17:56:55.095 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:56:54 smithi134 ceph-mon[35037]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T17:56:55.095 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:56:54 smithi134 ceph-mon[35037]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T17:56:55.095 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:56:54 smithi134 ceph-mon[35037]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T17:56:55.095 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:56:54 smithi134 ceph-mon[35037]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-03-28T17:56:55.095 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:56:54 smithi134 ceph-mon[35037]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-03-28T17:56:55.095 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:56:54 smithi134 ceph-mon[35037]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T17:56:55.282 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:56:54 smithi022 ceph-mon[31985]: from='client.14229 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi022:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-03-28T17:56:55.282 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:56:54 smithi022 ceph-mon[31985]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T17:56:55.282 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:56:54 smithi022 ceph-mon[31985]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T17:56:55.282 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:56:54 smithi022 ceph-mon[31985]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T17:56:55.282 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:56:54 smithi022 ceph-mon[31985]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-03-28T17:56:55.282 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:56:54 smithi022 ceph-mon[31985]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-03-28T17:56:55.282 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:56:54 smithi022 ceph-mon[31985]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T17:56:55.303 INFO:journalctl@ceph.mgr.y.smithi079.stdout:Mar 28 17:56:55 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mgr-y[36699]: 2024-03-28T17:56:55.133+0000 7fc7c2d22200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-03-28T17:56:55.304 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:56:54 smithi079 ceph-mon[35312]: from='client.14229 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi022:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-03-28T17:56:55.304 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:56:54 smithi079 ceph-mon[35312]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T17:56:55.304 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:56:54 smithi079 ceph-mon[35312]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T17:56:55.304 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:56:54 smithi079 ceph-mon[35312]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T17:56:55.304 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:56:54 smithi079 ceph-mon[35312]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-03-28T17:56:55.304 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:56:54 smithi079 ceph-mon[35312]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-03-28T17:56:55.304 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:56:54 smithi079 ceph-mon[35312]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T17:56:55.363 INFO:journalctl@ceph.mgr.z.smithi134.stdout:Mar 28 17:56:55 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mgr-z[37033]: 2024-03-28T17:56:55.227+0000 7f25838b0200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-03-28T17:56:55.363 INFO:journalctl@ceph.mgr.z.smithi134.stdout:Mar 28 17:56:55 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mgr-z[37033]: 2024-03-28T17:56:55.363+0000 7f25838b0200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-03-28T17:56:55.682 INFO:journalctl@ceph.mgr.y.smithi079.stdout:Mar 28 17:56:55 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mgr-y[36699]: 2024-03-28T17:56:55.303+0000 7fc7c2d22200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-03-28T17:56:55.924 INFO:journalctl@ceph.mgr.z.smithi134.stdout:Mar 28 17:56:55 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mgr-z[37033]: 2024-03-28T17:56:55.649+0000 7f25838b0200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-03-28T17:56:56.086 INFO:journalctl@ceph.mgr.y.smithi079.stdout:Mar 28 17:56:55 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mgr-y[36699]: 2024-03-28T17:56:55.718+0000 7fc7c2d22200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-03-28T17:56:56.087 INFO:journalctl@ceph.mgr.y.smithi079.stdout:Mar 28 17:56:55 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mgr-y[36699]: 2024-03-28T17:56:55.833+0000 7fc7c2d22200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-03-28T17:56:56.087 INFO:journalctl@ceph.mgr.y.smithi079.stdout:Mar 28 17:56:56 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mgr-y[36699]: 2024-03-28T17:56:56.086+0000 7fc7c2d22200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-03-28T17:56:56.425 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:56:56 smithi134 ceph-mon[35037]: pgmap v46: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-28T17:56:56.425 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:56:56 smithi134 ceph-mon[35037]: Reconfiguring mgr.x (unknown last config time)... 2024-03-28T17:56:56.425 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:56:56 smithi134 ceph-mon[35037]: Reconfiguring daemon mgr.x on smithi022 2024-03-28T17:56:56.425 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:56:56 smithi134 ceph-mon[35037]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T17:56:56.425 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:56:56 smithi134 ceph-mon[35037]: Standby manager daemon z started 2024-03-28T17:56:56.425 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:56:56 smithi134 ceph-mon[35037]: from='mgr.? 172.21.15.134:0/3244746657' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-03-28T17:56:56.425 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:56:56 smithi134 ceph-mon[35037]: from='mgr.? 172.21.15.134:0/3244746657' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-03-28T17:56:56.425 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:56:56 smithi134 ceph-mon[35037]: from='mgr.? 172.21.15.134:0/3244746657' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-03-28T17:56:56.425 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:56:56 smithi134 ceph-mon[35037]: from='mgr.? 172.21.15.134:0/3244746657' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-03-28T17:56:56.432 INFO:journalctl@ceph.mgr.y.smithi079.stdout:Mar 28 17:56:56 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mgr-y[36699]: 2024-03-28T17:56:56.327+0000 7fc7c2d22200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-03-28T17:56:56.433 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:56:56 smithi079 ceph-mon[35312]: pgmap v46: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-28T17:56:56.433 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:56:56 smithi079 ceph-mon[35312]: Reconfiguring mgr.x (unknown last config time)... 2024-03-28T17:56:56.433 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:56:56 smithi079 ceph-mon[35312]: Reconfiguring daemon mgr.x on smithi022 2024-03-28T17:56:56.433 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:56:56 smithi079 ceph-mon[35312]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T17:56:56.433 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:56:56 smithi079 ceph-mon[35312]: Standby manager daemon z started 2024-03-28T17:56:56.433 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:56:56 smithi079 ceph-mon[35312]: from='mgr.? 172.21.15.134:0/3244746657' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-03-28T17:56:56.433 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:56:56 smithi079 ceph-mon[35312]: from='mgr.? 172.21.15.134:0/3244746657' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-03-28T17:56:56.433 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:56:56 smithi079 ceph-mon[35312]: from='mgr.? 172.21.15.134:0/3244746657' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-03-28T17:56:56.433 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:56:56 smithi079 ceph-mon[35312]: from='mgr.? 172.21.15.134:0/3244746657' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-03-28T17:56:56.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:56:56 smithi022 ceph-mon[31985]: pgmap v46: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-28T17:56:56.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:56:56 smithi022 ceph-mon[31985]: Reconfiguring mgr.x (unknown last config time)... 2024-03-28T17:56:56.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:56:56 smithi022 ceph-mon[31985]: Reconfiguring daemon mgr.x on smithi022 2024-03-28T17:56:56.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:56:56 smithi022 ceph-mon[31985]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T17:56:56.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:56:56 smithi022 ceph-mon[31985]: Standby manager daemon z started 2024-03-28T17:56:56.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:56:56 smithi022 ceph-mon[31985]: from='mgr.? 172.21.15.134:0/3244746657' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-03-28T17:56:56.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:56:56 smithi022 ceph-mon[31985]: from='mgr.? 172.21.15.134:0/3244746657' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-03-28T17:56:56.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:56:56 smithi022 ceph-mon[31985]: from='mgr.? 172.21.15.134:0/3244746657' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-03-28T17:56:56.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:56:56 smithi022 ceph-mon[31985]: from='mgr.? 172.21.15.134:0/3244746657' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-03-28T17:56:56.536 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T17:56:56.536+0000 7f4288c00700 1 -- 172.21.15.22:0/4067071193 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mgrmap(e 14) v1 ==== 91698+0+0 (secure 0 0 0) 0x7f4284012050 con 0x7f428c105f70 2024-03-28T17:56:57.016 INFO:journalctl@ceph.mgr.y.smithi079.stdout:Mar 28 17:56:56 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mgr-y[36699]: 2024-03-28T17:56:56.694+0000 7fc7c2d22200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-03-28T17:56:57.017 INFO:journalctl@ceph.mgr.y.smithi079.stdout:Mar 28 17:56:57 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mgr-y[36699]: 2024-03-28T17:56:57.017+0000 7fc7c2d22200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-03-28T17:56:57.682 INFO:journalctl@ceph.mgr.y.smithi079.stdout:Mar 28 17:56:57 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mgr-y[36699]: 2024-03-28T17:56:57.316+0000 7fc7c2d22200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-03-28T17:56:57.683 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:56:57 smithi079 ceph-mon[35312]: pgmap v47: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-28T17:56:57.683 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:56:57 smithi079 ceph-mon[35312]: mgrmap e14: x(active, since 106s), standbys: z 2024-03-28T17:56:57.683 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:56:57 smithi079 ceph-mon[35312]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-03-28T17:56:57.683 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:56:57 smithi079 ceph-mon[35312]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T17:56:57.683 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:56:57 smithi079 ceph-mon[35312]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T17:56:57.683 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:56:57 smithi079 ceph-mon[35312]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T17:56:57.683 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:56:57 smithi079 ceph-mon[35312]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T17:56:57.683 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:56:57 smithi079 ceph-mon[35312]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T17:56:57.683 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:56:57 smithi079 ceph-mon[35312]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T17:56:57.781 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:56:57 smithi022 ceph-mon[31985]: pgmap v47: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-28T17:56:57.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:56:57 smithi022 ceph-mon[31985]: mgrmap e14: x(active, since 106s), standbys: z 2024-03-28T17:56:57.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:56:57 smithi022 ceph-mon[31985]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-03-28T17:56:57.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:56:57 smithi022 ceph-mon[31985]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T17:56:57.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:56:57 smithi022 ceph-mon[31985]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T17:56:57.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:56:57 smithi022 ceph-mon[31985]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T17:56:57.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:56:57 smithi022 ceph-mon[31985]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T17:56:57.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:56:57 smithi022 ceph-mon[31985]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T17:56:57.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:56:57 smithi022 ceph-mon[31985]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T17:56:57.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:56:57 smithi134 ceph-mon[35037]: pgmap v47: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-28T17:56:57.925 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:56:57 smithi134 ceph-mon[35037]: mgrmap e14: x(active, since 106s), standbys: z 2024-03-28T17:56:57.925 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:56:57 smithi134 ceph-mon[35037]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-03-28T17:56:57.925 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:56:57 smithi134 ceph-mon[35037]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T17:56:57.925 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:56:57 smithi134 ceph-mon[35037]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T17:56:57.925 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:56:57 smithi134 ceph-mon[35037]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T17:56:57.925 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:56:57 smithi134 ceph-mon[35037]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T17:56:57.925 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:56:57 smithi134 ceph-mon[35037]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T17:56:57.925 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:56:57 smithi134 ceph-mon[35037]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T17:56:58.682 INFO:journalctl@ceph.mgr.y.smithi079.stdout:Mar 28 17:56:58 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mgr-y[36699]: 2024-03-28T17:56:58.331+0000 7fc7c2d22200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-03-28T17:56:59.428 INFO:journalctl@ceph.mgr.y.smithi079.stdout:Mar 28 17:56:59 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mgr-y[36699]: 2024-03-28T17:56:59.024+0000 7fc7c2d22200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-03-28T17:56:59.428 INFO:journalctl@ceph.mgr.y.smithi079.stdout:Mar 28 17:56:59 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mgr-y[36699]: 2024-03-28T17:56:59.153+0000 7fc7c2d22200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-03-28T17:56:59.683 INFO:journalctl@ceph.mgr.y.smithi079.stdout:Mar 28 17:56:59 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mgr-y[36699]: 2024-03-28T17:56:59.426+0000 7fc7c2d22200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-03-28T17:56:59.683 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:56:59 smithi079 ceph-mon[35312]: pgmap v48: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-28T17:56:59.683 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:56:59 smithi079 ceph-mon[35312]: from='client.? 172.21.15.22:0/2728717014' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "5d708e06-f05b-40ba-bbcf-258ddc4f43d7"}]: dispatch 2024-03-28T17:56:59.683 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:56:59 smithi079 ceph-mon[35312]: from='client.? 172.21.15.22:0/2728717014' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "5d708e06-f05b-40ba-bbcf-258ddc4f43d7"}]': finished 2024-03-28T17:56:59.683 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:56:59 smithi079 ceph-mon[35312]: osdmap e5: 1 total, 0 up, 1 in 2024-03-28T17:56:59.683 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:56:59 smithi079 ceph-mon[35312]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-28T17:56:59.683 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:56:59 smithi079 ceph-mon[35312]: Standby manager daemon y started 2024-03-28T17:56:59.683 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:56:59 smithi079 ceph-mon[35312]: from='mgr.? 172.21.15.79:0/1241014677' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-03-28T17:56:59.683 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:56:59 smithi079 ceph-mon[35312]: from='mgr.? 172.21.15.79:0/1241014677' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-03-28T17:56:59.683 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:56:59 smithi079 ceph-mon[35312]: from='mgr.? 172.21.15.79:0/1241014677' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-03-28T17:56:59.683 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:56:59 smithi079 ceph-mon[35312]: from='mgr.? 172.21.15.79:0/1241014677' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-03-28T17:56:59.925 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:56:59 smithi134 ceph-mon[35037]: pgmap v48: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-28T17:56:59.925 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:56:59 smithi134 ceph-mon[35037]: from='client.? 172.21.15.22:0/2728717014' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "5d708e06-f05b-40ba-bbcf-258ddc4f43d7"}]: dispatch 2024-03-28T17:56:59.925 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:56:59 smithi134 ceph-mon[35037]: from='client.? 172.21.15.22:0/2728717014' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "5d708e06-f05b-40ba-bbcf-258ddc4f43d7"}]': finished 2024-03-28T17:56:59.925 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:56:59 smithi134 ceph-mon[35037]: osdmap e5: 1 total, 0 up, 1 in 2024-03-28T17:56:59.925 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:56:59 smithi134 ceph-mon[35037]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-28T17:56:59.925 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:56:59 smithi134 ceph-mon[35037]: Standby manager daemon y started 2024-03-28T17:56:59.925 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:56:59 smithi134 ceph-mon[35037]: from='mgr.? 172.21.15.79:0/1241014677' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-03-28T17:56:59.925 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:56:59 smithi134 ceph-mon[35037]: from='mgr.? 172.21.15.79:0/1241014677' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-03-28T17:56:59.925 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:56:59 smithi134 ceph-mon[35037]: from='mgr.? 172.21.15.79:0/1241014677' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-03-28T17:56:59.925 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:56:59 smithi134 ceph-mon[35037]: from='mgr.? 172.21.15.79:0/1241014677' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-03-28T17:57:00.032 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:56:59 smithi022 ceph-mon[31985]: pgmap v48: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-28T17:57:00.032 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:56:59 smithi022 ceph-mon[31985]: from='client.? 172.21.15.22:0/2728717014' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "5d708e06-f05b-40ba-bbcf-258ddc4f43d7"}]: dispatch 2024-03-28T17:57:00.032 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:56:59 smithi022 ceph-mon[31985]: from='client.? 172.21.15.22:0/2728717014' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "5d708e06-f05b-40ba-bbcf-258ddc4f43d7"}]': finished 2024-03-28T17:57:00.032 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:56:59 smithi022 ceph-mon[31985]: osdmap e5: 1 total, 0 up, 1 in 2024-03-28T17:57:00.032 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:56:59 smithi022 ceph-mon[31985]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-28T17:57:00.032 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:56:59 smithi022 ceph-mon[31985]: Standby manager daemon y started 2024-03-28T17:57:00.033 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:56:59 smithi022 ceph-mon[31985]: from='mgr.? 172.21.15.79:0/1241014677' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-03-28T17:57:00.033 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:56:59 smithi022 ceph-mon[31985]: from='mgr.? 172.21.15.79:0/1241014677' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-03-28T17:57:00.033 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:56:59 smithi022 ceph-mon[31985]: from='mgr.? 172.21.15.79:0/1241014677' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-03-28T17:57:00.033 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:56:59 smithi022 ceph-mon[31985]: from='mgr.? 172.21.15.79:0/1241014677' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-03-28T17:57:00.117 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T17:57:00.117+0000 7f4288c00700 1 -- 172.21.15.22:0/4067071193 <== mon.0 v2:172.21.15.22:3300/0 8 ==== mgrmap(e 15) v1 ==== 137234+0+0 (secure 0 0 0) 0x7f4284003a00 con 0x7f428c105f70 2024-03-28T17:57:00.925 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:57:00 smithi134 ceph-mon[35037]: from='client.? 172.21.15.22:0/4010977881' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-28T17:57:00.925 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:57:00 smithi134 ceph-mon[35037]: mgrmap e15: x(active, since 109s), standbys: z, y 2024-03-28T17:57:00.925 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:57:00 smithi134 ceph-mon[35037]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-03-28T17:57:00.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:57:00 smithi079 ceph-mon[35312]: from='client.? 172.21.15.22:0/4010977881' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-28T17:57:00.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:57:00 smithi079 ceph-mon[35312]: mgrmap e15: x(active, since 109s), standbys: z, y 2024-03-28T17:57:00.933 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:57:00 smithi079 ceph-mon[35312]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-03-28T17:57:01.032 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:57:00 smithi022 ceph-mon[31985]: from='client.? 172.21.15.22:0/4010977881' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-28T17:57:01.032 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:57:00 smithi022 ceph-mon[31985]: mgrmap e15: x(active, since 109s), standbys: z, y 2024-03-28T17:57:01.032 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:57:00 smithi022 ceph-mon[31985]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-03-28T17:57:02.425 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:57:02 smithi134 ceph-mon[35037]: pgmap v50: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-28T17:57:02.432 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:57:02 smithi079 ceph-mon[35312]: pgmap v50: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-28T17:57:02.531 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:57:02 smithi022 ceph-mon[31985]: pgmap v50: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-28T17:57:04.424 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:57:04 smithi134 ceph-mon[35037]: pgmap v51: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-28T17:57:04.432 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:57:04 smithi079 ceph-mon[35312]: pgmap v51: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-28T17:57:04.531 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:57:04 smithi022 ceph-mon[31985]: pgmap v51: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-28T17:57:06.183 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:57:06 smithi079 ceph-mon[35312]: pgmap v52: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-28T17:57:06.282 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:57:06 smithi022 ceph-mon[31985]: pgmap v52: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-28T17:57:06.424 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:57:06 smithi134 ceph-mon[35037]: pgmap v52: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-28T17:57:08.205 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:57:08 smithi022 ceph-mon[31985]: pgmap v53: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-28T17:57:08.424 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:57:08 smithi134 ceph-mon[35037]: pgmap v53: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-28T17:57:08.432 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:57:08 smithi079 ceph-mon[35312]: pgmap v53: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-28T17:57:09.411 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:57:09 smithi022 ceph-mon[31985]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-03-28T17:57:09.411 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:57:09 smithi022 ceph-mon[31985]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T17:57:09.411 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:57:09 smithi022 ceph-mon[31985]: Deploying daemon osd.0 on smithi022 2024-03-28T17:57:09.424 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:57:09 smithi134 ceph-mon[35037]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-03-28T17:57:09.424 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:57:09 smithi134 ceph-mon[35037]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T17:57:09.424 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:57:09 smithi134 ceph-mon[35037]: Deploying daemon osd.0 on smithi022 2024-03-28T17:57:09.433 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:57:09 smithi079 ceph-mon[35312]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-03-28T17:57:09.433 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:57:09 smithi079 ceph-mon[35312]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T17:57:09.433 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:57:09 smithi079 ceph-mon[35312]: Deploying daemon osd.0 on smithi022 2024-03-28T17:57:10.253 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:57:10 smithi022 ceph-mon[31985]: pgmap v54: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-28T17:57:10.424 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:57:10 smithi134 ceph-mon[35037]: pgmap v54: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-28T17:57:10.432 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:57:10 smithi079 ceph-mon[35312]: pgmap v54: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-28T17:57:12.424 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:57:12 smithi134 ceph-mon[35037]: pgmap v55: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-28T17:57:12.432 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:57:12 smithi079 ceph-mon[35312]: pgmap v55: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-28T17:57:12.515 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:57:12 smithi022 ceph-mon[31985]: pgmap v55: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-28T17:57:14.424 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:57:14 smithi134 ceph-mon[35037]: pgmap v56: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-28T17:57:14.432 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:57:14 smithi079 ceph-mon[35312]: pgmap v56: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-28T17:57:14.533 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:57:14 smithi022 ceph-mon[31985]: pgmap v56: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-28T17:57:16.424 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:57:16 smithi134 ceph-mon[35037]: pgmap v57: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-28T17:57:16.432 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:57:16 smithi079 ceph-mon[35312]: pgmap v57: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-28T17:57:16.531 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:57:16 smithi022 ceph-mon[31985]: pgmap v57: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-28T17:57:17.531 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:57:17 smithi022 ceph-mon[31985]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T17:57:17.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:57:17 smithi022 ceph-mon[31985]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T17:57:17.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:57:17 smithi022 ceph-mon[31985]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T17:57:17.674 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:57:17 smithi134 ceph-mon[35037]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T17:57:17.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:57:17 smithi134 ceph-mon[35037]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T17:57:17.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:57:17 smithi134 ceph-mon[35037]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T17:57:17.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:57:17 smithi079 ceph-mon[35312]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T17:57:17.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:57:17 smithi079 ceph-mon[35312]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T17:57:17.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:57:17 smithi079 ceph-mon[35312]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T17:57:18.399 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:57:18 smithi022 ceph-mon[31985]: pgmap v58: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-28T17:57:18.674 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:57:18 smithi134 ceph-mon[35037]: pgmap v58: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-28T17:57:18.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:57:18 smithi079 ceph-mon[35312]: pgmap v58: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-28T17:57:20.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:57:20 smithi134 ceph-mon[35037]: pgmap v59: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-28T17:57:20.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:57:20 smithi134 ceph-mon[35037]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T17:57:20.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:57:20 smithi134 ceph-mon[35037]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T17:57:20.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:57:20 smithi134 ceph-mon[35037]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T17:57:20.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:57:20 smithi134 ceph-mon[35037]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T17:57:20.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:57:20 smithi134 ceph-mon[35037]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T17:57:20.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:57:20 smithi134 ceph-mon[35037]: from='osd.0 [v2:172.21.15.22:6802/2186290846,v1:172.21.15.22:6803/2186290846]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-03-28T17:57:20.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:57:20 smithi079 ceph-mon[35312]: pgmap v59: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-28T17:57:20.683 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:57:20 smithi079 ceph-mon[35312]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T17:57:20.683 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:57:20 smithi079 ceph-mon[35312]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T17:57:20.683 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:57:20 smithi079 ceph-mon[35312]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T17:57:20.683 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:57:20 smithi079 ceph-mon[35312]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T17:57:20.683 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:57:20 smithi079 ceph-mon[35312]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T17:57:20.683 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:57:20 smithi079 ceph-mon[35312]: from='osd.0 [v2:172.21.15.22:6802/2186290846,v1:172.21.15.22:6803/2186290846]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-03-28T17:57:20.748 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:57:20 smithi022 ceph-mon[31985]: pgmap v59: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-28T17:57:20.748 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:57:20 smithi022 ceph-mon[31985]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T17:57:20.748 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:57:20 smithi022 ceph-mon[31985]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T17:57:20.748 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:57:20 smithi022 ceph-mon[31985]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T17:57:20.748 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:57:20 smithi022 ceph-mon[31985]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T17:57:20.748 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:57:20 smithi022 ceph-mon[31985]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T17:57:20.748 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:57:20 smithi022 ceph-mon[31985]: from='osd.0 [v2:172.21.15.22:6802/2186290846,v1:172.21.15.22:6803/2186290846]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-03-28T17:57:21.597 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T17:57:21.596+0000 7f4288c00700 1 -- 172.21.15.22:0/4067071193 <== mgr.14152 v2:172.21.15.22:6800/4201048392 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f428c0625d0 con 0x7f4270039200 2024-03-28T17:57:21.597 INFO:teuthology.orchestra.run.smithi022.stdout:Created osd(s) 0 on host 'smithi022' 2024-03-28T17:57:21.599 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T17:57:21.599+0000 7f4292800700 1 -- 172.21.15.22:0/4067071193 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7f4270039200 msgr2=0x7f427003b6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:57:21.599 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T17:57:21.599+0000 7f4292800700 1 --2- 172.21.15.22:0/4067071193 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7f4270039200 0x7f427003b6c0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f428c105430 tx=0x7f427c00b410 comp rx=0 tx=0).stop 2024-03-28T17:57:21.599 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T17:57:21.599+0000 7f4292800700 1 -- 172.21.15.22:0/4067071193 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f428c105f70 msgr2=0x7f428c1a1b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:57:21.599 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T17:57:21.599+0000 7f4292800700 1 --2- 172.21.15.22:0/4067071193 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f428c105f70 0x7f428c1a1b70 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f428400ef60 tx=0x7f428400ef90 comp rx=0 tx=0).stop 2024-03-28T17:57:21.599 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T17:57:21.599+0000 7f4292800700 1 -- 172.21.15.22:0/4067071193 shutdown_connections 2024-03-28T17:57:21.600 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T17:57:21.599+0000 7f4292800700 1 --2- 172.21.15.22:0/4067071193 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f428c105f70 0x7f428c1a1b70 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:57:21.600 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T17:57:21.599+0000 7f4292800700 1 --2- 172.21.15.22:0/4067071193 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f428c1043d0 0x7f428c19a5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:57:21.600 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T17:57:21.599+0000 7f4292800700 1 --2- 172.21.15.22:0/4067071193 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7f4270039200 0x7f427003b6c0 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:57:21.600 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T17:57:21.599+0000 7f4292800700 1 --2- 172.21.15.22:0/4067071193 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f428c1055d0 0x7f428c19aaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:57:21.600 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T17:57:21.599+0000 7f4292800700 1 -- 172.21.15.22:0/4067071193 >> 172.21.15.22:0/4067071193 conn(0x7f428c0ff950 msgr2=0x7f428c108870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T17:57:21.600 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T17:57:21.599+0000 7f4292800700 1 -- 172.21.15.22:0/4067071193 shutdown_connections 2024-03-28T17:57:21.600 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T17:57:21.599+0000 7f4292800700 1 -- 172.21.15.22:0/4067071193 wait complete. 2024-03-28T17:57:21.674 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:57:21 smithi134 ceph-mon[35037]: from='osd.0 [v2:172.21.15.22:6802/2186290846,v1:172.21.15.22:6803/2186290846]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-03-28T17:57:21.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:57:21 smithi134 ceph-mon[35037]: osdmap e6: 1 total, 0 up, 1 in 2024-03-28T17:57:21.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:57:21 smithi134 ceph-mon[35037]: from='osd.0 [v2:172.21.15.22:6802/2186290846,v1:172.21.15.22:6803/2186290846]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi022", "root=default"]}]: dispatch 2024-03-28T17:57:21.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:57:21 smithi134 ceph-mon[35037]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-28T17:57:21.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:57:21 smithi079 ceph-mon[35312]: from='osd.0 [v2:172.21.15.22:6802/2186290846,v1:172.21.15.22:6803/2186290846]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-03-28T17:57:21.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:57:21 smithi079 ceph-mon[35312]: osdmap e6: 1 total, 0 up, 1 in 2024-03-28T17:57:21.683 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:57:21 smithi079 ceph-mon[35312]: from='osd.0 [v2:172.21.15.22:6802/2186290846,v1:172.21.15.22:6803/2186290846]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi022", "root=default"]}]: dispatch 2024-03-28T17:57:21.683 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:57:21 smithi079 ceph-mon[35312]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-28T17:57:21.712 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:57:21 smithi022 ceph-mon[31985]: from='osd.0 [v2:172.21.15.22:6802/2186290846,v1:172.21.15.22:6803/2186290846]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-03-28T17:57:21.712 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:57:21 smithi022 ceph-mon[31985]: osdmap e6: 1 total, 0 up, 1 in 2024-03-28T17:57:21.712 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:57:21 smithi022 ceph-mon[31985]: from='osd.0 [v2:172.21.15.22:6802/2186290846,v1:172.21.15.22:6803/2186290846]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi022", "root=default"]}]: dispatch 2024-03-28T17:57:21.712 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:57:21 smithi022 ceph-mon[31985]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-28T17:57:22.319 DEBUG:teuthology.orchestra.run.smithi022:osd.0> sudo journalctl -f -n 0 -u ceph-fd220748-ed2b-11ee-95d0-87774f69a715@osd.0.service 2024-03-28T17:57:22.324 INFO:tasks.cephadm:Deploying osd.1 on smithi079 with /dev/vg_nvme/lv_4... 2024-03-28T17:57:22.324 DEBUG:teuthology.orchestra.run.smithi079:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fd220748-ed2b-11ee-95d0-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-03-28T17:57:22.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:57:22 smithi022 ceph-mon[31985]: pgmap v61: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-28T17:57:22.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:57:22 smithi022 ceph-mon[31985]: from='osd.0 [v2:172.21.15.22:6802/2186290846,v1:172.21.15.22:6803/2186290846]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi022", "root=default"]}]': finished 2024-03-28T17:57:22.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:57:22 smithi022 ceph-mon[31985]: osdmap e7: 1 total, 0 up, 1 in 2024-03-28T17:57:22.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:57:22 smithi022 ceph-mon[31985]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-28T17:57:22.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:57:22 smithi022 ceph-mon[31985]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-28T17:57:22.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:57:22 smithi022 ceph-mon[31985]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T17:57:22.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:57:22 smithi022 ceph-mon[31985]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T17:57:22.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:57:22 smithi022 ceph-mon[31985]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T17:57:22.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:57:22 smithi134 ceph-mon[35037]: pgmap v61: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-28T17:57:22.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:57:22 smithi134 ceph-mon[35037]: from='osd.0 [v2:172.21.15.22:6802/2186290846,v1:172.21.15.22:6803/2186290846]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi022", "root=default"]}]': finished 2024-03-28T17:57:22.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:57:22 smithi134 ceph-mon[35037]: osdmap e7: 1 total, 0 up, 1 in 2024-03-28T17:57:22.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:57:22 smithi134 ceph-mon[35037]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-28T17:57:22.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:57:22 smithi134 ceph-mon[35037]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-28T17:57:22.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:57:22 smithi134 ceph-mon[35037]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T17:57:22.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:57:22 smithi134 ceph-mon[35037]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T17:57:22.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:57:22 smithi134 ceph-mon[35037]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T17:57:22.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:57:22 smithi079 ceph-mon[35312]: pgmap v61: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-28T17:57:22.683 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:57:22 smithi079 ceph-mon[35312]: from='osd.0 [v2:172.21.15.22:6802/2186290846,v1:172.21.15.22:6803/2186290846]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi022", "root=default"]}]': finished 2024-03-28T17:57:22.683 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:57:22 smithi079 ceph-mon[35312]: osdmap e7: 1 total, 0 up, 1 in 2024-03-28T17:57:22.683 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:57:22 smithi079 ceph-mon[35312]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-28T17:57:22.683 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:57:22 smithi079 ceph-mon[35312]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-28T17:57:22.683 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:57:22 smithi079 ceph-mon[35312]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T17:57:22.683 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:57:22 smithi079 ceph-mon[35312]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T17:57:22.683 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:57:22 smithi079 ceph-mon[35312]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T17:57:22.783 INFO:teuthology.orchestra.run.smithi079.stderr:Inferring config /var/lib/ceph/fd220748-ed2b-11ee-95d0-87774f69a715/mon.b/config 2024-03-28T17:57:23.683 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:57:23 smithi079 ceph-mon[35312]: purged_snaps scrub starts 2024-03-28T17:57:23.683 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:57:23 smithi079 ceph-mon[35312]: purged_snaps scrub ok 2024-03-28T17:57:23.683 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:57:23 smithi079 ceph-mon[35312]: osd.0 [v2:172.21.15.22:6802/2186290846,v1:172.21.15.22:6803/2186290846] boot 2024-03-28T17:57:23.684 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:57:23 smithi079 ceph-mon[35312]: osdmap e8: 1 total, 1 up, 1 in 2024-03-28T17:57:23.684 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:57:23 smithi079 ceph-mon[35312]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-28T17:57:23.748 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:57:23 smithi022 ceph-mon[31985]: purged_snaps scrub starts 2024-03-28T17:57:23.748 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:57:23 smithi022 ceph-mon[31985]: purged_snaps scrub ok 2024-03-28T17:57:23.748 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:57:23 smithi022 ceph-mon[31985]: osd.0 [v2:172.21.15.22:6802/2186290846,v1:172.21.15.22:6803/2186290846] boot 2024-03-28T17:57:23.748 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:57:23 smithi022 ceph-mon[31985]: osdmap e8: 1 total, 1 up, 1 in 2024-03-28T17:57:23.748 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:57:23 smithi022 ceph-mon[31985]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-28T17:57:23.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:57:23 smithi134 ceph-mon[35037]: purged_snaps scrub starts 2024-03-28T17:57:23.925 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:57:23 smithi134 ceph-mon[35037]: purged_snaps scrub ok 2024-03-28T17:57:23.925 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:57:23 smithi134 ceph-mon[35037]: osd.0 [v2:172.21.15.22:6802/2186290846,v1:172.21.15.22:6803/2186290846] boot 2024-03-28T17:57:23.925 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:57:23 smithi134 ceph-mon[35037]: osdmap e8: 1 total, 1 up, 1 in 2024-03-28T17:57:23.925 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:57:23 smithi134 ceph-mon[35037]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-28T17:57:24.639 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:57:24 smithi079 ceph-mon[35312]: pgmap v64: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-28T17:57:24.640 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:57:24 smithi079 ceph-mon[35312]: osdmap e9: 1 total, 1 up, 1 in 2024-03-28T17:57:24.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:57:24 smithi022 ceph-mon[31985]: pgmap v64: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-28T17:57:24.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:57:24 smithi022 ceph-mon[31985]: osdmap e9: 1 total, 1 up, 1 in 2024-03-28T17:57:24.826 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:57:24 smithi134 ceph-mon[35037]: pgmap v64: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-28T17:57:24.826 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:57:24 smithi134 ceph-mon[35037]: osdmap e9: 1 total, 1 up, 1 in 2024-03-28T17:57:25.500 INFO:teuthology.orchestra.run.smithi079.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-03-28T17:57:25.500 INFO:teuthology.orchestra.run.smithi079.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-03-28T17:57:25.500 INFO:teuthology.orchestra.run.smithi079.stderr: stderr: 10+0 records in 2024-03-28T17:57:25.500 INFO:teuthology.orchestra.run.smithi079.stderr:10+0 records out 2024-03-28T17:57:25.500 INFO:teuthology.orchestra.run.smithi079.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0355962 s, 295 MB/s 2024-03-28T17:57:25.500 INFO:teuthology.orchestra.run.smithi079.stderr:--> Zapping successful for: 2024-03-28T17:57:25.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:57:25 smithi022 ceph-mon[31985]: pgmap v66: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-28T17:57:25.877 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:57:25 smithi079 ceph-mon[35312]: pgmap v66: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-28T17:57:25.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:57:25 smithi134 ceph-mon[35037]: pgmap v66: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-28T17:57:26.347 DEBUG:teuthology.orchestra.run.smithi079:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fd220748-ed2b-11ee-95d0-87774f69a715 -- ceph orch daemon add osd smithi079:vg_nvme/lv_4 2024-03-28T17:57:26.776 INFO:teuthology.orchestra.run.smithi079.stderr:Inferring config /var/lib/ceph/fd220748-ed2b-11ee-95d0-87774f69a715/mon.b/config 2024-03-28T17:57:28.404 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:57:28 smithi079 ceph-mon[35312]: pgmap v67: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-28T17:57:28.404 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:57:28 smithi079 ceph-mon[35312]: Detected new or changed devices on smithi022 2024-03-28T17:57:28.404 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:57:28 smithi079 ceph-mon[35312]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T17:57:28.404 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:57:28 smithi079 ceph-mon[35312]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T17:57:28.404 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:57:28 smithi079 ceph-mon[35312]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-28T17:57:28.404 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:57:28 smithi079 ceph-mon[35312]: Adjusting osd_memory_target on smithi022 to 17095M 2024-03-28T17:57:28.404 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:57:28 smithi079 ceph-mon[35312]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T17:57:28.404 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:57:28 smithi079 ceph-mon[35312]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T17:57:28.404 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:57:28 smithi079 ceph-mon[35312]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T17:57:28.404 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:57:28 smithi079 ceph-mon[35312]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T17:57:28.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:57:28 smithi022 ceph-mon[31985]: pgmap v67: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-28T17:57:28.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:57:28 smithi022 ceph-mon[31985]: Detected new or changed devices on smithi022 2024-03-28T17:57:28.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:57:28 smithi022 ceph-mon[31985]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T17:57:28.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:57:28 smithi022 ceph-mon[31985]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T17:57:28.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:57:28 smithi022 ceph-mon[31985]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-28T17:57:28.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:57:28 smithi022 ceph-mon[31985]: Adjusting osd_memory_target on smithi022 to 17095M 2024-03-28T17:57:28.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:57:28 smithi022 ceph-mon[31985]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T17:57:28.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:57:28 smithi022 ceph-mon[31985]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T17:57:28.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:57:28 smithi022 ceph-mon[31985]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T17:57:28.533 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:57:28 smithi022 ceph-mon[31985]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T17:57:28.674 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:57:28 smithi134 ceph-mon[35037]: pgmap v67: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-28T17:57:28.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:57:28 smithi134 ceph-mon[35037]: Detected new or changed devices on smithi022 2024-03-28T17:57:28.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:57:28 smithi134 ceph-mon[35037]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T17:57:28.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:57:28 smithi134 ceph-mon[35037]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T17:57:28.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:57:28 smithi134 ceph-mon[35037]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-28T17:57:28.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:57:28 smithi134 ceph-mon[35037]: Adjusting osd_memory_target on smithi022 to 17095M 2024-03-28T17:57:28.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:57:28 smithi134 ceph-mon[35037]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T17:57:28.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:57:28 smithi134 ceph-mon[35037]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T17:57:28.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:57:28 smithi134 ceph-mon[35037]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T17:57:28.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:57:28 smithi134 ceph-mon[35037]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T17:57:28.853 INFO:teuthology.orchestra.run.smithi079.stderr:2024-03-28T17:57:28.852+0000 7f7f15000700 1 -- 172.21.15.79:0/1912492691 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f7f10104010 msgr2=0x7f7f10104470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:57:28.853 INFO:teuthology.orchestra.run.smithi079.stderr:2024-03-28T17:57:28.852+0000 7f7f15000700 1 --2- 172.21.15.79:0/1912492691 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f7f10104010 0x7f7f10104470 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f7efc0099c0 tx=0x7f7efc009cd0 comp rx=0 tx=0).stop 2024-03-28T17:57:28.853 INFO:teuthology.orchestra.run.smithi079.stderr:2024-03-28T17:57:28.852+0000 7f7f15000700 1 -- 172.21.15.79:0/1912492691 shutdown_connections 2024-03-28T17:57:28.853 INFO:teuthology.orchestra.run.smithi079.stderr:2024-03-28T17:57:28.852+0000 7f7f15000700 1 --2- 172.21.15.79:0/1912492691 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7f101049b0 0x7f7f1010c840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:57:28.853 INFO:teuthology.orchestra.run.smithi079.stderr:2024-03-28T17:57:28.852+0000 7f7f15000700 1 --2- 172.21.15.79:0/1912492691 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7f10105ad0 0x7f7f10105ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:57:28.853 INFO:teuthology.orchestra.run.smithi079.stderr:2024-03-28T17:57:28.852+0000 7f7f15000700 1 --2- 172.21.15.79:0/1912492691 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f7f10104010 0x7f7f10104470 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:57:28.853 INFO:teuthology.orchestra.run.smithi079.stderr:2024-03-28T17:57:28.852+0000 7f7f15000700 1 -- 172.21.15.79:0/1912492691 >> 172.21.15.79:0/1912492691 conn(0x7f7f100ff9b0 msgr2=0x7f7f10101dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T17:57:28.853 INFO:teuthology.orchestra.run.smithi079.stderr:2024-03-28T17:57:28.852+0000 7f7f15000700 1 -- 172.21.15.79:0/1912492691 shutdown_connections 2024-03-28T17:57:28.853 INFO:teuthology.orchestra.run.smithi079.stderr:2024-03-28T17:57:28.852+0000 7f7f15000700 1 -- 172.21.15.79:0/1912492691 wait complete. 2024-03-28T17:57:28.854 INFO:teuthology.orchestra.run.smithi079.stderr:2024-03-28T17:57:28.854+0000 7f7f15000700 1 Processor -- start 2024-03-28T17:57:28.854 INFO:teuthology.orchestra.run.smithi079.stderr:2024-03-28T17:57:28.854+0000 7f7f15000700 1 -- start start 2024-03-28T17:57:28.856 INFO:teuthology.orchestra.run.smithi079.stderr:2024-03-28T17:57:28.855+0000 7f7f15000700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f7f10104010 0x7f7f1019a7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:57:28.857 INFO:teuthology.orchestra.run.smithi079.stderr:2024-03-28T17:57:28.855+0000 7f7f15000700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7f101049b0 0x7f7f1019ad10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:57:28.857 INFO:teuthology.orchestra.run.smithi079.stderr:2024-03-28T17:57:28.855+0000 7f7f15000700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7f10105ad0 0x7f7f101a1d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:57:28.857 INFO:teuthology.orchestra.run.smithi079.stderr:2024-03-28T17:57:28.855+0000 7f7f15000700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f10110140 con 0x7f7f101049b0 2024-03-28T17:57:28.857 INFO:teuthology.orchestra.run.smithi079.stderr:2024-03-28T17:57:28.855+0000 7f7f15000700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f1010ffc0 con 0x7f7f10104010 2024-03-28T17:57:28.857 INFO:teuthology.orchestra.run.smithi079.stderr:2024-03-28T17:57:28.855+0000 7f7f15000700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f101102c0 con 0x7f7f10105ad0 2024-03-28T17:57:28.857 INFO:teuthology.orchestra.run.smithi079.stderr:2024-03-28T17:57:28.855+0000 7f7f0de00700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f7f10104010 0x7f7f1019a7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:57:28.857 INFO:teuthology.orchestra.run.smithi079.stderr:2024-03-28T17:57:28.855+0000 7f7f0de00700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f7f10104010 0x7f7f1019a7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:59032/0 (socket says 172.21.15.79:59032) 2024-03-28T17:57:28.857 INFO:teuthology.orchestra.run.smithi079.stderr:2024-03-28T17:57:28.855+0000 7f7f0de00700 1 -- 172.21.15.79:0/3676245392 learned_addr learned my addr 172.21.15.79:0/3676245392 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-03-28T17:57:28.857 INFO:teuthology.orchestra.run.smithi079.stderr:2024-03-28T17:57:28.855+0000 7f7f0d400700 1 --2- 172.21.15.79:0/3676245392 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7f101049b0 0x7f7f1019ad10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:57:28.857 INFO:teuthology.orchestra.run.smithi079.stderr:2024-03-28T17:57:28.855+0000 7f7f0e800700 1 --2- 172.21.15.79:0/3676245392 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7f10105ad0 0x7f7f101a1d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:57:28.857 INFO:teuthology.orchestra.run.smithi079.stderr:2024-03-28T17:57:28.855+0000 7f7f0de00700 1 -- 172.21.15.79:0/3676245392 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7f10105ad0 msgr2=0x7f7f101a1d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:57:28.857 INFO:teuthology.orchestra.run.smithi079.stderr:2024-03-28T17:57:28.856+0000 7f7f0de00700 1 --2- 172.21.15.79:0/3676245392 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7f10105ad0 0x7f7f101a1d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:57:28.858 INFO:teuthology.orchestra.run.smithi079.stderr:2024-03-28T17:57:28.856+0000 7f7f0de00700 1 -- 172.21.15.79:0/3676245392 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7f101049b0 msgr2=0x7f7f1019ad10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:57:28.858 INFO:teuthology.orchestra.run.smithi079.stderr:2024-03-28T17:57:28.856+0000 7f7f0de00700 1 --2- 172.21.15.79:0/3676245392 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7f101049b0 0x7f7f1019ad10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:57:28.858 INFO:teuthology.orchestra.run.smithi079.stderr:2024-03-28T17:57:28.856+0000 7f7f0de00700 1 -- 172.21.15.79:0/3676245392 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7f101a2430 con 0x7f7f10104010 2024-03-28T17:57:28.858 INFO:teuthology.orchestra.run.smithi079.stderr:2024-03-28T17:57:28.856+0000 7f7f0d400700 1 --2- 172.21.15.79:0/3676245392 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7f101049b0 0x7f7f1019ad10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T17:57:28.858 INFO:teuthology.orchestra.run.smithi079.stderr:2024-03-28T17:57:28.856+0000 7f7f0e800700 1 --2- 172.21.15.79:0/3676245392 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7f10105ad0 0x7f7f101a1d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T17:57:28.858 INFO:teuthology.orchestra.run.smithi079.stderr:2024-03-28T17:57:28.856+0000 7f7f0de00700 1 --2- 172.21.15.79:0/3676245392 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f7f10104010 0x7f7f1019a7d0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f7f0000b500 tx=0x7f7f0000b8c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T17:57:28.858 INFO:teuthology.orchestra.run.smithi079.stderr:2024-03-28T17:57:28.856+0000 7f7f06a00700 1 -- 172.21.15.79:0/3676245392 <== mon.2 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f00015020 con 0x7f7f10104010 2024-03-28T17:57:28.858 INFO:teuthology.orchestra.run.smithi079.stderr:2024-03-28T17:57:28.856+0000 7f7f06a00700 1 -- 172.21.15.79:0/3676245392 <== mon.2 v2:172.21.15.79:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f7f00011d00 con 0x7f7f10104010 2024-03-28T17:57:28.858 INFO:teuthology.orchestra.run.smithi079.stderr:2024-03-28T17:57:28.856+0000 7f7f06a00700 1 -- 172.21.15.79:0/3676245392 <== mon.2 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f0000f2e0 con 0x7f7f10104010 2024-03-28T17:57:28.858 INFO:teuthology.orchestra.run.smithi079.stderr:2024-03-28T17:57:28.856+0000 7f7f15000700 1 -- 172.21.15.79:0/3676245392 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7f101a2720 con 0x7f7f10104010 2024-03-28T17:57:28.859 INFO:teuthology.orchestra.run.smithi079.stderr:2024-03-28T17:57:28.856+0000 7f7f15000700 1 -- 172.21.15.79:0/3676245392 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7f101a2c90 con 0x7f7f10104010 2024-03-28T17:57:28.859 INFO:teuthology.orchestra.run.smithi079.stderr:2024-03-28T17:57:28.858+0000 7f7f15000700 1 -- 172.21.15.79:0/3676245392 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7f10068e50 con 0x7f7f10104010 2024-03-28T17:57:28.860 INFO:teuthology.orchestra.run.smithi079.stderr:2024-03-28T17:57:28.860+0000 7f7f06a00700 1 -- 172.21.15.79:0/3676245392 <== mon.2 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 15) v1 ==== 137234+0+0 (secure 0 0 0) 0x7f7f00018020 con 0x7f7f10104010 2024-03-28T17:57:28.861 INFO:teuthology.orchestra.run.smithi079.stderr:2024-03-28T17:57:28.861+0000 7f7f06a00700 1 --2- 172.21.15.79:0/3676245392 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7f7ef40a2880 0x7f7ef40a4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:57:28.861 INFO:teuthology.orchestra.run.smithi079.stderr:2024-03-28T17:57:28.861+0000 7f7f06a00700 1 -- 172.21.15.79:0/3676245392 <== mon.2 v2:172.21.15.79:3300/0 5 ==== osd_map(9..9 src has 1..9) v4 ==== 1767+0+0 (secure 0 0 0) 0x7f7f000cc830 con 0x7f7f10104010 2024-03-28T17:57:28.862 INFO:teuthology.orchestra.run.smithi079.stderr:2024-03-28T17:57:28.862+0000 7f7f0d400700 1 --2- 172.21.15.79:0/3676245392 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7f7ef40a2880 0x7f7ef40a4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:57:28.863 INFO:teuthology.orchestra.run.smithi079.stderr:2024-03-28T17:57:28.863+0000 7f7f0d400700 1 --2- 172.21.15.79:0/3676245392 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7f7ef40a2880 0x7f7ef40a4d40 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f7efc003860 tx=0x7f7efc009f10 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T17:57:28.866 INFO:teuthology.orchestra.run.smithi079.stderr:2024-03-28T17:57:28.866+0000 7f7f06a00700 1 -- 172.21.15.79:0/3676245392 <== mon.2 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f7f0001f020 con 0x7f7f10104010 2024-03-28T17:57:29.099 INFO:teuthology.orchestra.run.smithi079.stderr:2024-03-28T17:57:29.098+0000 7f7f15000700 1 -- 172.21.15.79:0/3676245392 --> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi079:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) v1 -- 0x7f7f100625d0 con 0x7f7ef40a2880 2024-03-28T17:57:29.433 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:57:29 smithi079 ceph-mon[35312]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-28T17:57:29.433 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:57:29 smithi079 ceph-mon[35312]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-28T17:57:29.433 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:57:29 smithi079 ceph-mon[35312]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T17:57:29.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:57:29 smithi022 ceph-mon[31985]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-28T17:57:29.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:57:29 smithi022 ceph-mon[31985]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-28T17:57:29.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:57:29 smithi022 ceph-mon[31985]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T17:57:29.674 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:57:29 smithi134 ceph-mon[35037]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-28T17:57:29.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:57:29 smithi134 ceph-mon[35037]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-28T17:57:29.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:57:29 smithi134 ceph-mon[35037]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T17:57:30.363 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:57:30 smithi079 ceph-mon[35312]: pgmap v68: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-28T17:57:30.363 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:57:30 smithi079 ceph-mon[35312]: from='client.34145 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi079:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-03-28T17:57:30.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:57:30 smithi022 ceph-mon[31985]: pgmap v68: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-28T17:57:30.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:57:30 smithi022 ceph-mon[31985]: from='client.34145 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi079:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-03-28T17:57:30.674 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:57:30 smithi134 ceph-mon[35037]: pgmap v68: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-28T17:57:30.674 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:57:30 smithi134 ceph-mon[35037]: from='client.34145 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi079:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-03-28T17:57:32.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:57:32 smithi022 ceph-mon[31985]: pgmap v69: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-28T17:57:32.674 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:57:32 smithi134 ceph-mon[35037]: pgmap v69: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-28T17:57:32.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:57:32 smithi079 ceph-mon[35312]: pgmap v69: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-28T17:57:33.489 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:57:33 smithi079 ceph-mon[35312]: from='client.? 172.21.15.79:0/3180359059' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "68ad09bb-9256-409a-9867-4e14a2cb94aa"}]: dispatch 2024-03-28T17:57:33.489 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:57:33 smithi079 ceph-mon[35312]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "68ad09bb-9256-409a-9867-4e14a2cb94aa"}]: dispatch 2024-03-28T17:57:33.489 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:57:33 smithi079 ceph-mon[35312]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "68ad09bb-9256-409a-9867-4e14a2cb94aa"}]': finished 2024-03-28T17:57:33.490 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:57:33 smithi079 ceph-mon[35312]: osdmap e10: 2 total, 1 up, 2 in 2024-03-28T17:57:33.490 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:57:33 smithi079 ceph-mon[35312]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-28T17:57:33.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:57:33 smithi022 ceph-mon[31985]: from='client.? 172.21.15.79:0/3180359059' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "68ad09bb-9256-409a-9867-4e14a2cb94aa"}]: dispatch 2024-03-28T17:57:33.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:57:33 smithi022 ceph-mon[31985]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "68ad09bb-9256-409a-9867-4e14a2cb94aa"}]: dispatch 2024-03-28T17:57:33.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:57:33 smithi022 ceph-mon[31985]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "68ad09bb-9256-409a-9867-4e14a2cb94aa"}]': finished 2024-03-28T17:57:33.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:57:33 smithi022 ceph-mon[31985]: osdmap e10: 2 total, 1 up, 2 in 2024-03-28T17:57:33.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:57:33 smithi022 ceph-mon[31985]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-28T17:57:33.674 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:57:33 smithi134 ceph-mon[35037]: from='client.? 172.21.15.79:0/3180359059' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "68ad09bb-9256-409a-9867-4e14a2cb94aa"}]: dispatch 2024-03-28T17:57:33.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:57:33 smithi134 ceph-mon[35037]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "68ad09bb-9256-409a-9867-4e14a2cb94aa"}]: dispatch 2024-03-28T17:57:33.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:57:33 smithi134 ceph-mon[35037]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "68ad09bb-9256-409a-9867-4e14a2cb94aa"}]': finished 2024-03-28T17:57:33.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:57:33 smithi134 ceph-mon[35037]: osdmap e10: 2 total, 1 up, 2 in 2024-03-28T17:57:33.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:57:33 smithi134 ceph-mon[35037]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-28T17:57:34.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:57:34 smithi022 ceph-mon[31985]: pgmap v70: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-28T17:57:34.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:57:34 smithi022 ceph-mon[31985]: from='client.? 172.21.15.79:0/2074386713' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-28T17:57:34.674 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:57:34 smithi134 ceph-mon[35037]: pgmap v70: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-28T17:57:34.674 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:57:34 smithi134 ceph-mon[35037]: from='client.? 172.21.15.79:0/2074386713' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-28T17:57:34.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:57:34 smithi079 ceph-mon[35312]: pgmap v70: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-28T17:57:34.683 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:57:34 smithi079 ceph-mon[35312]: from='client.? 172.21.15.79:0/2074386713' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-28T17:57:36.362 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:57:36 smithi079 ceph-mon[35312]: pgmap v72: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-28T17:57:36.531 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:57:36 smithi134 ceph-mon[35037]: pgmap v72: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-28T17:57:36.531 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:57:36 smithi022 ceph-mon[31985]: pgmap v72: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-28T17:57:38.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:57:38 smithi022 ceph-mon[31985]: pgmap v73: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-28T17:57:38.674 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:57:38 smithi134 ceph-mon[35037]: pgmap v73: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-28T17:57:38.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:57:38 smithi079 ceph-mon[35312]: pgmap v73: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-28T17:57:40.428 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:57:40 smithi079 ceph-mon[35312]: pgmap v74: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-28T17:57:40.531 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:57:40 smithi022 ceph-mon[31985]: pgmap v74: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-28T17:57:40.674 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:57:40 smithi134 ceph-mon[35037]: pgmap v74: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-28T17:57:42.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:57:42 smithi022 ceph-mon[31985]: pgmap v75: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-28T17:57:42.674 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:57:42 smithi134 ceph-mon[35037]: pgmap v75: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-28T17:57:42.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:57:42 smithi079 ceph-mon[35312]: pgmap v75: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-28T17:57:43.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:57:43 smithi022 ceph-mon[31985]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-03-28T17:57:43.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:57:43 smithi022 ceph-mon[31985]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T17:57:43.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:57:43 smithi022 ceph-mon[31985]: Deploying daemon osd.1 on smithi079 2024-03-28T17:57:43.570 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:57:43 smithi079 ceph-mon[35312]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-03-28T17:57:43.570 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:57:43 smithi079 ceph-mon[35312]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T17:57:43.570 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:57:43 smithi079 ceph-mon[35312]: Deploying daemon osd.1 on smithi079 2024-03-28T17:57:43.674 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:57:43 smithi134 ceph-mon[35037]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-03-28T17:57:43.674 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:57:43 smithi134 ceph-mon[35037]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T17:57:43.674 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:57:43 smithi134 ceph-mon[35037]: Deploying daemon osd.1 on smithi079 2024-03-28T17:57:44.674 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:57:44 smithi134 ceph-mon[35037]: pgmap v76: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-28T17:57:44.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:57:44 smithi079 ceph-mon[35312]: pgmap v76: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-28T17:57:44.781 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:57:44 smithi022 ceph-mon[31985]: pgmap v76: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-28T17:57:46.433 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:57:46 smithi079 ceph-mon[35312]: pgmap v77: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-28T17:57:46.674 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:57:46 smithi134 ceph-mon[35037]: pgmap v77: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-28T17:57:46.781 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:57:46 smithi022 ceph-mon[31985]: pgmap v77: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-28T17:57:48.674 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:57:48 smithi134 ceph-mon[35037]: pgmap v78: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-28T17:57:48.683 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:57:48 smithi079 ceph-mon[35312]: pgmap v78: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-28T17:57:48.781 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:57:48 smithi022 ceph-mon[31985]: pgmap v78: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-28T17:57:50.585 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:57:50 smithi022 ceph-mon[31985]: pgmap v79: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-28T17:57:50.585 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:57:50 smithi022 ceph-mon[31985]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T17:57:50.585 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:57:50 smithi022 ceph-mon[31985]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T17:57:50.585 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:57:50 smithi022 ceph-mon[31985]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T17:57:50.603 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:57:50 smithi079 ceph-mon[35312]: pgmap v79: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-28T17:57:50.603 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:57:50 smithi079 ceph-mon[35312]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T17:57:50.603 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:57:50 smithi079 ceph-mon[35312]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T17:57:50.603 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:57:50 smithi079 ceph-mon[35312]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T17:57:50.674 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:57:50 smithi134 ceph-mon[35037]: pgmap v79: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-28T17:57:50.674 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:57:50 smithi134 ceph-mon[35037]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T17:57:50.674 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:57:50 smithi134 ceph-mon[35037]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T17:57:50.674 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:57:50 smithi134 ceph-mon[35037]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T17:57:52.432 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:57:52 smithi079 ceph-mon[35312]: pgmap v80: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-28T17:57:52.531 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:57:52 smithi022 ceph-mon[31985]: pgmap v80: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-28T17:57:52.674 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:57:52 smithi134 ceph-mon[35037]: pgmap v80: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-28T17:57:54.433 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:57:54 smithi079 ceph-mon[35312]: pgmap v81: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-28T17:57:54.433 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:57:54 smithi079 ceph-mon[35312]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T17:57:54.433 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:57:54 smithi079 ceph-mon[35312]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T17:57:54.433 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:57:54 smithi079 ceph-mon[35312]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T17:57:54.433 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:57:54 smithi079 ceph-mon[35312]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T17:57:54.434 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:57:54 smithi079 ceph-mon[35312]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T17:57:54.434 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:57:54 smithi079 ceph-mon[35312]: from='osd.1 [v2:172.21.15.79:6800/2835549681,v1:172.21.15.79:6801/2835549681]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-03-28T17:57:54.434 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:57:54 smithi079 ceph-mon[35312]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-03-28T17:57:54.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:57:54 smithi134 ceph-mon[35037]: pgmap v81: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-28T17:57:54.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:57:54 smithi134 ceph-mon[35037]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T17:57:54.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:57:54 smithi134 ceph-mon[35037]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T17:57:54.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:57:54 smithi134 ceph-mon[35037]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T17:57:54.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:57:54 smithi134 ceph-mon[35037]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T17:57:54.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:57:54 smithi134 ceph-mon[35037]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T17:57:54.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:57:54 smithi134 ceph-mon[35037]: from='osd.1 [v2:172.21.15.79:6800/2835549681,v1:172.21.15.79:6801/2835549681]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-03-28T17:57:54.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:57:54 smithi134 ceph-mon[35037]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-03-28T17:57:54.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:57:54 smithi022 ceph-mon[31985]: pgmap v81: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-28T17:57:54.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:57:54 smithi022 ceph-mon[31985]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T17:57:54.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:57:54 smithi022 ceph-mon[31985]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T17:57:54.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:57:54 smithi022 ceph-mon[31985]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T17:57:54.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:57:54 smithi022 ceph-mon[31985]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T17:57:54.783 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:57:54 smithi022 ceph-mon[31985]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T17:57:54.783 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:57:54 smithi022 ceph-mon[31985]: from='osd.1 [v2:172.21.15.79:6800/2835549681,v1:172.21.15.79:6801/2835549681]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-03-28T17:57:54.783 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:57:54 smithi022 ceph-mon[31985]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-03-28T17:57:54.915 INFO:teuthology.orchestra.run.smithi079.stderr:2024-03-28T17:57:54.913+0000 7f7f06a00700 1 -- 172.21.15.79:0/3676245392 <== mgr.14152 v2:172.21.15.22:6800/4201048392 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f7f100625d0 con 0x7f7ef40a2880 2024-03-28T17:57:54.915 INFO:teuthology.orchestra.run.smithi079.stdout:Created osd(s) 1 on host 'smithi079' 2024-03-28T17:57:54.918 INFO:teuthology.orchestra.run.smithi079.stderr:2024-03-28T17:57:54.917+0000 7f7f15000700 1 -- 172.21.15.79:0/3676245392 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7f7ef40a2880 msgr2=0x7f7ef40a4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:57:54.918 INFO:teuthology.orchestra.run.smithi079.stderr:2024-03-28T17:57:54.917+0000 7f7f15000700 1 --2- 172.21.15.79:0/3676245392 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7f7ef40a2880 0x7f7ef40a4d40 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f7efc003860 tx=0x7f7efc009f10 comp rx=0 tx=0).stop 2024-03-28T17:57:54.918 INFO:teuthology.orchestra.run.smithi079.stderr:2024-03-28T17:57:54.917+0000 7f7f15000700 1 -- 172.21.15.79:0/3676245392 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f7f10104010 msgr2=0x7f7f1019a7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:57:54.918 INFO:teuthology.orchestra.run.smithi079.stderr:2024-03-28T17:57:54.917+0000 7f7f15000700 1 --2- 172.21.15.79:0/3676245392 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f7f10104010 0x7f7f1019a7d0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f7f0000b500 tx=0x7f7f0000b8c0 comp rx=0 tx=0).stop 2024-03-28T17:57:54.918 INFO:teuthology.orchestra.run.smithi079.stderr:2024-03-28T17:57:54.917+0000 7f7f15000700 1 -- 172.21.15.79:0/3676245392 shutdown_connections 2024-03-28T17:57:54.919 INFO:teuthology.orchestra.run.smithi079.stderr:2024-03-28T17:57:54.917+0000 7f7f15000700 1 --2- 172.21.15.79:0/3676245392 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7f7ef40a2880 0x7f7ef40a4d40 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:57:54.919 INFO:teuthology.orchestra.run.smithi079.stderr:2024-03-28T17:57:54.917+0000 7f7f15000700 1 --2- 172.21.15.79:0/3676245392 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7f10105ad0 0x7f7f101a1d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:57:54.919 INFO:teuthology.orchestra.run.smithi079.stderr:2024-03-28T17:57:54.917+0000 7f7f15000700 1 --2- 172.21.15.79:0/3676245392 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f7f10104010 0x7f7f1019a7d0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:57:54.919 INFO:teuthology.orchestra.run.smithi079.stderr:2024-03-28T17:57:54.917+0000 7f7f15000700 1 --2- 172.21.15.79:0/3676245392 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7f101049b0 0x7f7f1019ad10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:57:54.919 INFO:teuthology.orchestra.run.smithi079.stderr:2024-03-28T17:57:54.917+0000 7f7f15000700 1 -- 172.21.15.79:0/3676245392 >> 172.21.15.79:0/3676245392 conn(0x7f7f100ff9b0 msgr2=0x7f7f101088f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T17:57:54.919 INFO:teuthology.orchestra.run.smithi079.stderr:2024-03-28T17:57:54.917+0000 7f7f15000700 1 -- 172.21.15.79:0/3676245392 shutdown_connections 2024-03-28T17:57:54.919 INFO:teuthology.orchestra.run.smithi079.stderr:2024-03-28T17:57:54.917+0000 7f7f15000700 1 -- 172.21.15.79:0/3676245392 wait complete. 2024-03-28T17:57:55.716 DEBUG:teuthology.orchestra.run.smithi079:osd.1> sudo journalctl -f -n 0 -u ceph-fd220748-ed2b-11ee-95d0-87774f69a715@osd.1.service 2024-03-28T17:57:55.721 INFO:tasks.cephadm:Deploying osd.2 on smithi134 with /dev/vg_nvme/lv_4... 2024-03-28T17:57:55.721 DEBUG:teuthology.orchestra.run.smithi134:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fd220748-ed2b-11ee-95d0-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-03-28T17:57:55.925 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:57:55 smithi134 ceph-mon[35037]: pgmap v82: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-28T17:57:55.925 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:57:55 smithi134 ceph-mon[35037]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-03-28T17:57:55.925 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:57:55 smithi134 ceph-mon[35037]: osdmap e11: 2 total, 1 up, 2 in 2024-03-28T17:57:55.925 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:57:55 smithi134 ceph-mon[35037]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-28T17:57:55.925 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:57:55 smithi134 ceph-mon[35037]: from='osd.1 [v2:172.21.15.79:6800/2835549681,v1:172.21.15.79:6801/2835549681]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi079", "root=default"]}]: dispatch 2024-03-28T17:57:55.926 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:57:55 smithi134 ceph-mon[35037]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi079", "root=default"]}]: dispatch 2024-03-28T17:57:55.926 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:57:55 smithi134 ceph-mon[35037]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T17:57:55.926 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:57:55 smithi134 ceph-mon[35037]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T17:57:55.926 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:57:55 smithi134 ceph-mon[35037]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T17:57:55.933 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:57:55 smithi079 ceph-mon[35312]: pgmap v82: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-28T17:57:55.933 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:57:55 smithi079 ceph-mon[35312]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-03-28T17:57:55.933 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:57:55 smithi079 ceph-mon[35312]: osdmap e11: 2 total, 1 up, 2 in 2024-03-28T17:57:55.933 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:57:55 smithi079 ceph-mon[35312]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-28T17:57:55.934 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:57:55 smithi079 ceph-mon[35312]: from='osd.1 [v2:172.21.15.79:6800/2835549681,v1:172.21.15.79:6801/2835549681]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi079", "root=default"]}]: dispatch 2024-03-28T17:57:55.934 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:57:55 smithi079 ceph-mon[35312]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi079", "root=default"]}]: dispatch 2024-03-28T17:57:55.934 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:57:55 smithi079 ceph-mon[35312]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T17:57:55.934 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:57:55 smithi079 ceph-mon[35312]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T17:57:55.934 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:57:55 smithi079 ceph-mon[35312]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T17:57:56.032 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:57:55 smithi022 ceph-mon[31985]: pgmap v82: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-28T17:57:56.032 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:57:55 smithi022 ceph-mon[31985]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-03-28T17:57:56.032 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:57:55 smithi022 ceph-mon[31985]: osdmap e11: 2 total, 1 up, 2 in 2024-03-28T17:57:56.032 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:57:55 smithi022 ceph-mon[31985]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-28T17:57:56.033 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:57:55 smithi022 ceph-mon[31985]: from='osd.1 [v2:172.21.15.79:6800/2835549681,v1:172.21.15.79:6801/2835549681]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi079", "root=default"]}]: dispatch 2024-03-28T17:57:56.033 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:57:55 smithi022 ceph-mon[31985]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi079", "root=default"]}]: dispatch 2024-03-28T17:57:56.033 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:57:55 smithi022 ceph-mon[31985]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T17:57:56.033 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:57:55 smithi022 ceph-mon[31985]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T17:57:56.033 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:57:55 smithi022 ceph-mon[31985]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T17:57:56.153 INFO:teuthology.orchestra.run.smithi134.stderr:Inferring config /var/lib/ceph/fd220748-ed2b-11ee-95d0-87774f69a715/mon.c/config 2024-03-28T17:57:56.925 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:57:56 smithi134 ceph-mon[35037]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi079", "root=default"]}]': finished 2024-03-28T17:57:56.925 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:57:56 smithi134 ceph-mon[35037]: osdmap e12: 2 total, 1 up, 2 in 2024-03-28T17:57:56.925 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:57:56 smithi134 ceph-mon[35037]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-28T17:57:56.925 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:57:56 smithi134 ceph-mon[35037]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-28T17:57:56.933 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:57:56 smithi079 ceph-mon[35312]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi079", "root=default"]}]': finished 2024-03-28T17:57:56.933 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:57:56 smithi079 ceph-mon[35312]: osdmap e12: 2 total, 1 up, 2 in 2024-03-28T17:57:56.933 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:57:56 smithi079 ceph-mon[35312]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-28T17:57:56.933 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:57:56 smithi079 ceph-mon[35312]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-28T17:57:56.933 INFO:journalctl@ceph.osd.1.smithi079.stdout:Mar 28 17:57:56 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-osd-1[39908]: 2024-03-28T17:57:56.627+0000 7ffa2aa00700 -1 osd.1 0 waiting for initial osdmap 2024-03-28T17:57:56.933 INFO:journalctl@ceph.osd.1.smithi079.stdout:Mar 28 17:57:56 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-osd-1[39908]: 2024-03-28T17:57:56.684+0000 7ffa1fe00700 -1 osd.1 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-28T17:57:57.031 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:57:56 smithi022 ceph-mon[31985]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi079", "root=default"]}]': finished 2024-03-28T17:57:57.032 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:57:56 smithi022 ceph-mon[31985]: osdmap e12: 2 total, 1 up, 2 in 2024-03-28T17:57:57.032 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:57:56 smithi022 ceph-mon[31985]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-28T17:57:57.032 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:57:56 smithi022 ceph-mon[31985]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-28T17:57:57.881 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:57:57 smithi134 ceph-mon[35037]: purged_snaps scrub starts 2024-03-28T17:57:57.881 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:57:57 smithi134 ceph-mon[35037]: purged_snaps scrub ok 2024-03-28T17:57:57.881 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:57:57 smithi134 ceph-mon[35037]: pgmap v85: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-28T17:57:57.881 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:57:57 smithi134 ceph-mon[35037]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-28T17:57:57.881 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:57:57 smithi134 ceph-mon[35037]: from='osd.1 ' entity='osd.1' 2024-03-28T17:57:57.881 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:57:57 smithi134 ceph-mon[35037]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-28T17:57:57.906 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:57:57 smithi079 ceph-mon[35312]: purged_snaps scrub starts 2024-03-28T17:57:57.906 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:57:57 smithi079 ceph-mon[35312]: purged_snaps scrub ok 2024-03-28T17:57:57.907 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:57:57 smithi079 ceph-mon[35312]: pgmap v85: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-28T17:57:57.907 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:57:57 smithi079 ceph-mon[35312]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-28T17:57:57.907 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:57:57 smithi079 ceph-mon[35312]: from='osd.1 ' entity='osd.1' 2024-03-28T17:57:57.907 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:57:57 smithi079 ceph-mon[35312]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-28T17:57:58.032 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:57:57 smithi022 ceph-mon[31985]: purged_snaps scrub starts 2024-03-28T17:57:58.032 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:57:57 smithi022 ceph-mon[31985]: purged_snaps scrub ok 2024-03-28T17:57:58.032 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:57:57 smithi022 ceph-mon[31985]: pgmap v85: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-28T17:57:58.032 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:57:57 smithi022 ceph-mon[31985]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-28T17:57:58.032 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:57:57 smithi022 ceph-mon[31985]: from='osd.1 ' entity='osd.1' 2024-03-28T17:57:58.032 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:57:57 smithi022 ceph-mon[31985]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-28T17:57:58.850 INFO:teuthology.orchestra.run.smithi134.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-03-28T17:57:58.850 INFO:teuthology.orchestra.run.smithi134.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-03-28T17:57:58.850 INFO:teuthology.orchestra.run.smithi134.stderr: stderr: 10+0 records in 2024-03-28T17:57:58.850 INFO:teuthology.orchestra.run.smithi134.stderr:10+0 records out 2024-03-28T17:57:58.850 INFO:teuthology.orchestra.run.smithi134.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0338754 s, 310 MB/s 2024-03-28T17:57:58.851 INFO:teuthology.orchestra.run.smithi134.stderr:--> Zapping successful for: 2024-03-28T17:57:58.904 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:57:58 smithi134 ceph-mon[35037]: osd.1 [v2:172.21.15.79:6800/2835549681,v1:172.21.15.79:6801/2835549681] boot 2024-03-28T17:57:58.904 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:57:58 smithi134 ceph-mon[35037]: osdmap e13: 2 total, 2 up, 2 in 2024-03-28T17:57:58.905 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:57:58 smithi134 ceph-mon[35037]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-28T17:57:58.931 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:57:58 smithi079 ceph-mon[35312]: osd.1 [v2:172.21.15.79:6800/2835549681,v1:172.21.15.79:6801/2835549681] boot 2024-03-28T17:57:58.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:57:58 smithi079 ceph-mon[35312]: osdmap e13: 2 total, 2 up, 2 in 2024-03-28T17:57:58.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:57:58 smithi079 ceph-mon[35312]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-28T17:57:59.032 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:57:58 smithi022 ceph-mon[31985]: osd.1 [v2:172.21.15.79:6800/2835549681,v1:172.21.15.79:6801/2835549681] boot 2024-03-28T17:57:59.032 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:57:58 smithi022 ceph-mon[31985]: osdmap e13: 2 total, 2 up, 2 in 2024-03-28T17:57:59.032 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:57:58 smithi022 ceph-mon[31985]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-28T17:57:59.634 DEBUG:teuthology.orchestra.run.smithi134:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fd220748-ed2b-11ee-95d0-87774f69a715 -- ceph orch daemon add osd smithi134:vg_nvme/lv_4 2024-03-28T17:57:59.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:57:59 smithi134 ceph-mon[35037]: pgmap v87: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-28T17:57:59.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:57:59 smithi134 ceph-mon[35037]: osdmap e14: 2 total, 2 up, 2 in 2024-03-28T17:57:59.933 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:57:59 smithi079 ceph-mon[35312]: pgmap v87: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-28T17:57:59.933 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:57:59 smithi079 ceph-mon[35312]: osdmap e14: 2 total, 2 up, 2 in 2024-03-28T17:58:00.032 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:57:59 smithi022 ceph-mon[31985]: pgmap v87: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-28T17:58:00.032 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:57:59 smithi022 ceph-mon[31985]: osdmap e14: 2 total, 2 up, 2 in 2024-03-28T17:58:00.044 INFO:teuthology.orchestra.run.smithi134.stderr:Inferring config /var/lib/ceph/fd220748-ed2b-11ee-95d0-87774f69a715/mon.c/config 2024-03-28T17:58:01.786 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:58:01 smithi134 ceph-mon[35037]: pgmap v89: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-03-28T17:58:01.786 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:58:01 smithi134 ceph-mon[35037]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T17:58:01.786 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:58:01 smithi134 ceph-mon[35037]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T17:58:01.786 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:58:01 smithi134 ceph-mon[35037]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-28T17:58:01.786 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:58:01 smithi134 ceph-mon[35037]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T17:58:01.786 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:58:01 smithi134 ceph-mon[35037]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T17:58:01.786 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:58:01 smithi134 ceph-mon[35037]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T17:58:01.786 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:58:01 smithi134 ceph-mon[35037]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T17:58:01.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:58:01 smithi079 ceph-mon[35312]: pgmap v89: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-03-28T17:58:01.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:58:01 smithi079 ceph-mon[35312]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T17:58:01.933 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:58:01 smithi079 ceph-mon[35312]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T17:58:01.933 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:58:01 smithi079 ceph-mon[35312]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-28T17:58:01.933 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:58:01 smithi079 ceph-mon[35312]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T17:58:01.933 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:58:01 smithi079 ceph-mon[35312]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T17:58:01.933 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:58:01 smithi079 ceph-mon[35312]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T17:58:01.933 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:58:01 smithi079 ceph-mon[35312]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T17:58:01.953 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-28T17:58:01.952+0000 7f78de800700 1 -- 172.21.15.134:0/3618420347 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f78d8105610 msgr2=0x7f78d8105a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:58:01.953 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-28T17:58:01.952+0000 7f78de800700 1 --2- 172.21.15.134:0/3618420347 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f78d8105610 0x7f78d8105a70 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f78c4009990 tx=0x7f78c4009ca0 comp rx=0 tx=0).stop 2024-03-28T17:58:01.953 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-28T17:58:01.952+0000 7f78de800700 1 -- 172.21.15.134:0/3618420347 shutdown_connections 2024-03-28T17:58:01.953 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-28T17:58:01.952+0000 7f78de800700 1 --2- 172.21.15.134:0/3618420347 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f78d8105fb0 0x7f78d810c860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:58:01.953 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-28T17:58:01.952+0000 7f78de800700 1 --2- 172.21.15.134:0/3618420347 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f78d8104410 0x7f78d8104830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:58:01.953 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-28T17:58:01.952+0000 7f78de800700 1 --2- 172.21.15.134:0/3618420347 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f78d8105610 0x7f78d8105a70 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:58:01.953 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-28T17:58:01.952+0000 7f78de800700 1 -- 172.21.15.134:0/3618420347 >> 172.21.15.134:0/3618420347 conn(0x7f78d80ff9b0 msgr2=0x7f78d8101df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T17:58:01.954 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-28T17:58:01.952+0000 7f78de800700 1 -- 172.21.15.134:0/3618420347 shutdown_connections 2024-03-28T17:58:01.954 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-28T17:58:01.952+0000 7f78de800700 1 -- 172.21.15.134:0/3618420347 wait complete. 2024-03-28T17:58:01.956 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-28T17:58:01.953+0000 7f78de800700 1 Processor -- start 2024-03-28T17:58:01.957 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-28T17:58:01.953+0000 7f78de800700 1 -- start start 2024-03-28T17:58:01.957 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-28T17:58:01.954+0000 7f78de800700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f78d8104410 0x7f78d819a640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:58:01.957 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-28T17:58:01.954+0000 7f78de800700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f78d8105610 0x7f78d819ab80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:58:01.957 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-28T17:58:01.954+0000 7f78de800700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f78d8105fb0 0x7f78d81a1c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:58:01.957 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-28T17:58:01.954+0000 7f78de800700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78d8110090 con 0x7f78d8105fb0 2024-03-28T17:58:01.957 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-28T17:58:01.954+0000 7f78de800700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78d810ff10 con 0x7f78d8105610 2024-03-28T17:58:01.957 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-28T17:58:01.954+0000 7f78de800700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78d8110210 con 0x7f78d8104410 2024-03-28T17:58:01.957 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-28T17:58:01.954+0000 7f78d7400700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f78d8104410 0x7f78d819a640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:58:01.958 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-28T17:58:01.954+0000 7f78d7400700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f78d8104410 0x7f78d819a640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.134:39696/0 (socket says 172.21.15.134:39696) 2024-03-28T17:58:01.958 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-28T17:58:01.954+0000 7f78d7400700 1 -- 172.21.15.134:0/3957300580 learned_addr learned my addr 172.21.15.134:0/3957300580 (peer_addr_for_me v2:172.21.15.134:0/0) 2024-03-28T17:58:01.958 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-28T17:58:01.954+0000 7f78d6a00700 1 --2- 172.21.15.134:0/3957300580 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f78d8105610 0x7f78d819ab80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:58:01.958 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-28T17:58:01.954+0000 7f78d7e00700 1 --2- 172.21.15.134:0/3957300580 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f78d8105fb0 0x7f78d81a1c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:58:01.958 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-28T17:58:01.955+0000 7f78d7400700 1 -- 172.21.15.134:0/3957300580 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f78d8105610 msgr2=0x7f78d819ab80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:58:01.958 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-28T17:58:01.955+0000 7f78d7400700 1 --2- 172.21.15.134:0/3957300580 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f78d8105610 0x7f78d819ab80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:58:01.958 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-28T17:58:01.955+0000 7f78d7400700 1 -- 172.21.15.134:0/3957300580 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f78d8105fb0 msgr2=0x7f78d81a1c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:58:01.958 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-28T17:58:01.955+0000 7f78d7400700 1 --2- 172.21.15.134:0/3957300580 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f78d8105fb0 0x7f78d81a1c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:58:01.958 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-28T17:58:01.955+0000 7f78d7400700 1 -- 172.21.15.134:0/3957300580 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f78d81a22a0 con 0x7f78d8104410 2024-03-28T17:58:01.959 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-28T17:58:01.955+0000 7f78d6a00700 1 --2- 172.21.15.134:0/3957300580 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f78d8105610 0x7f78d819ab80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-28T17:58:01.959 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-28T17:58:01.955+0000 7f78d7e00700 1 --2- 172.21.15.134:0/3957300580 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f78d8105fb0 0x7f78d81a1c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-28T17:58:01.959 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-28T17:58:01.955+0000 7f78d7400700 1 --2- 172.21.15.134:0/3957300580 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f78d8104410 0x7f78d819a640 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f78c800d9b0 tx=0x7f78c800dd70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T17:58:01.959 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-28T17:58:01.955+0000 7f78cfe00700 1 -- 172.21.15.134:0/3957300580 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78c8017030 con 0x7f78d8104410 2024-03-28T17:58:01.959 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-28T17:58:01.955+0000 7f78cfe00700 1 -- 172.21.15.134:0/3957300580 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f78c80044f0 con 0x7f78d8104410 2024-03-28T17:58:01.959 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-28T17:58:01.955+0000 7f78cfe00700 1 -- 172.21.15.134:0/3957300580 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78c8003a00 con 0x7f78d8104410 2024-03-28T17:58:01.959 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-28T17:58:01.955+0000 7f78de800700 1 -- 172.21.15.134:0/3957300580 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f78d81a2590 con 0x7f78d8104410 2024-03-28T17:58:01.959 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-28T17:58:01.955+0000 7f78de800700 1 -- 172.21.15.134:0/3957300580 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f78d81a2b00 con 0x7f78d8104410 2024-03-28T17:58:01.959 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-28T17:58:01.957+0000 7f78de800700 1 -- 172.21.15.134:0/3957300580 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f78d8068e50 con 0x7f78d8104410 2024-03-28T17:58:01.960 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-28T17:58:01.960+0000 7f78cfe00700 1 -- 172.21.15.134:0/3957300580 <== mon.1 v2:172.21.15.134:3300/0 4 ==== mgrmap(e 15) v1 ==== 137234+0+0 (secure 0 0 0) 0x7f78c801a070 con 0x7f78d8104410 2024-03-28T17:58:01.962 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-28T17:58:01.961+0000 7f78cfe00700 1 --2- 172.21.15.134:0/3957300580 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7f78bc0a2880 0x7f78bc0a4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:58:01.962 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-28T17:58:01.962+0000 7f78cfe00700 1 -- 172.21.15.134:0/3957300580 <== mon.1 v2:172.21.15.134:3300/0 5 ==== osd_map(14..14 src has 1..14) v4 ==== 2309+0+0 (secure 0 0 0) 0x7f78c80ccfe0 con 0x7f78d8104410 2024-03-28T17:58:01.962 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-28T17:58:01.962+0000 7f78d6a00700 1 --2- 172.21.15.134:0/3957300580 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7f78bc0a2880 0x7f78bc0a4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:58:01.963 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-28T17:58:01.963+0000 7f78d6a00700 1 --2- 172.21.15.134:0/3957300580 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7f78bc0a2880 0x7f78bc0a4d40 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f78c400b5c0 tx=0x7f78c4005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T17:58:01.965 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-28T17:58:01.965+0000 7f78cfe00700 1 -- 172.21.15.134:0/3957300580 <== mon.1 v2:172.21.15.134:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f78c8009940 con 0x7f78d8104410 2024-03-28T17:58:02.032 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:58:01 smithi022 ceph-mon[31985]: pgmap v89: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-03-28T17:58:02.032 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:58:01 smithi022 ceph-mon[31985]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T17:58:02.032 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:58:01 smithi022 ceph-mon[31985]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T17:58:02.032 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:58:01 smithi022 ceph-mon[31985]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-28T17:58:02.032 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:58:01 smithi022 ceph-mon[31985]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T17:58:02.032 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:58:01 smithi022 ceph-mon[31985]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T17:58:02.033 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:58:01 smithi022 ceph-mon[31985]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T17:58:02.033 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:58:01 smithi022 ceph-mon[31985]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T17:58:02.199 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-28T17:58:02.198+0000 7f78de800700 1 -- 172.21.15.134:0/3957300580 --> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi134:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) v1 -- 0x7f78d80625d0 con 0x7f78bc0a2880 2024-03-28T17:58:02.925 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:58:02 smithi134 ceph-mon[35037]: Detected new or changed devices on smithi079 2024-03-28T17:58:02.925 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:58:02 smithi134 ceph-mon[35037]: Adjusting osd_memory_target on smithi079 to 17091M 2024-03-28T17:58:02.925 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:58:02 smithi134 ceph-mon[35037]: from='client.24163 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi134:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-03-28T17:58:02.925 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:58:02 smithi134 ceph-mon[35037]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-28T17:58:02.925 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:58:02 smithi134 ceph-mon[35037]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-28T17:58:02.925 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:58:02 smithi134 ceph-mon[35037]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T17:58:02.933 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:58:02 smithi079 ceph-mon[35312]: Detected new or changed devices on smithi079 2024-03-28T17:58:02.933 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:58:02 smithi079 ceph-mon[35312]: Adjusting osd_memory_target on smithi079 to 17091M 2024-03-28T17:58:02.933 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:58:02 smithi079 ceph-mon[35312]: from='client.24163 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi134:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-03-28T17:58:02.933 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:58:02 smithi079 ceph-mon[35312]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-28T17:58:02.933 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:58:02 smithi079 ceph-mon[35312]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-28T17:58:02.933 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:58:02 smithi079 ceph-mon[35312]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T17:58:03.032 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:58:02 smithi022 ceph-mon[31985]: Detected new or changed devices on smithi079 2024-03-28T17:58:03.032 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:58:02 smithi022 ceph-mon[31985]: Adjusting osd_memory_target on smithi079 to 17091M 2024-03-28T17:58:03.032 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:58:02 smithi022 ceph-mon[31985]: from='client.24163 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi134:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-03-28T17:58:03.032 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:58:02 smithi022 ceph-mon[31985]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-28T17:58:03.032 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:58:02 smithi022 ceph-mon[31985]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-28T17:58:03.032 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:58:02 smithi022 ceph-mon[31985]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T17:58:03.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:58:03 smithi134 ceph-mon[35037]: pgmap v90: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-03-28T17:58:03.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:58:03 smithi079 ceph-mon[35312]: pgmap v90: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-03-28T17:58:04.031 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:58:03 smithi022 ceph-mon[31985]: pgmap v90: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-03-28T17:58:05.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:58:05 smithi134 ceph-mon[35037]: pgmap v91: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-03-28T17:58:05.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:58:05 smithi079 ceph-mon[35312]: pgmap v91: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-03-28T17:58:06.032 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:58:05 smithi022 ceph-mon[31985]: pgmap v91: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-03-28T17:58:06.925 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:58:06 smithi134 ceph-mon[35037]: from='client.? 172.21.15.134:0/475318165' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ed11a5a7-004d-41f3-a5fc-eb2d603bd33f"}]: dispatch 2024-03-28T17:58:06.925 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:58:06 smithi134 ceph-mon[35037]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ed11a5a7-004d-41f3-a5fc-eb2d603bd33f"}]: dispatch 2024-03-28T17:58:06.925 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:58:06 smithi134 ceph-mon[35037]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "ed11a5a7-004d-41f3-a5fc-eb2d603bd33f"}]': finished 2024-03-28T17:58:06.925 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:58:06 smithi134 ceph-mon[35037]: osdmap e15: 3 total, 2 up, 3 in 2024-03-28T17:58:06.925 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:58:06 smithi134 ceph-mon[35037]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-28T17:58:06.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:58:06 smithi079 ceph-mon[35312]: from='client.? 172.21.15.134:0/475318165' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ed11a5a7-004d-41f3-a5fc-eb2d603bd33f"}]: dispatch 2024-03-28T17:58:06.933 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:58:06 smithi079 ceph-mon[35312]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ed11a5a7-004d-41f3-a5fc-eb2d603bd33f"}]: dispatch 2024-03-28T17:58:06.933 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:58:06 smithi079 ceph-mon[35312]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "ed11a5a7-004d-41f3-a5fc-eb2d603bd33f"}]': finished 2024-03-28T17:58:06.933 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:58:06 smithi079 ceph-mon[35312]: osdmap e15: 3 total, 2 up, 3 in 2024-03-28T17:58:06.933 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:58:06 smithi079 ceph-mon[35312]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-28T17:58:07.032 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:58:06 smithi022 ceph-mon[31985]: from='client.? 172.21.15.134:0/475318165' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ed11a5a7-004d-41f3-a5fc-eb2d603bd33f"}]: dispatch 2024-03-28T17:58:07.032 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:58:06 smithi022 ceph-mon[31985]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ed11a5a7-004d-41f3-a5fc-eb2d603bd33f"}]: dispatch 2024-03-28T17:58:07.032 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:58:06 smithi022 ceph-mon[31985]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "ed11a5a7-004d-41f3-a5fc-eb2d603bd33f"}]': finished 2024-03-28T17:58:07.032 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:58:06 smithi022 ceph-mon[31985]: osdmap e15: 3 total, 2 up, 3 in 2024-03-28T17:58:07.032 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:58:06 smithi022 ceph-mon[31985]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-28T17:58:07.933 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:58:07 smithi079 ceph-mon[35312]: pgmap v93: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-03-28T17:58:07.933 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:58:07 smithi079 ceph-mon[35312]: from='client.? 172.21.15.134:0/1506312450' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-28T17:58:08.032 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:58:07 smithi022 ceph-mon[31985]: pgmap v93: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-03-28T17:58:08.032 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:58:07 smithi022 ceph-mon[31985]: from='client.? 172.21.15.134:0/1506312450' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-28T17:58:08.174 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:58:07 smithi134 ceph-mon[35037]: pgmap v93: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-03-28T17:58:08.175 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:58:07 smithi134 ceph-mon[35037]: from='client.? 172.21.15.134:0/1506312450' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-28T17:58:10.032 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:58:09 smithi022 ceph-mon[31985]: pgmap v94: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-03-28T17:58:10.174 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:58:09 smithi134 ceph-mon[35037]: pgmap v94: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-03-28T17:58:10.182 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:58:09 smithi079 ceph-mon[35312]: pgmap v94: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-03-28T17:58:12.032 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:58:11 smithi022 ceph-mon[31985]: pgmap v95: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-03-28T17:58:12.157 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:58:11 smithi134 ceph-mon[35037]: pgmap v95: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-03-28T17:58:12.183 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:58:11 smithi079 ceph-mon[35312]: pgmap v95: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-03-28T17:58:14.032 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:58:13 smithi022 ceph-mon[31985]: pgmap v96: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-03-28T17:58:14.175 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:58:13 smithi134 ceph-mon[35037]: pgmap v96: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-03-28T17:58:14.182 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:58:13 smithi079 ceph-mon[35312]: pgmap v96: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-03-28T17:58:16.032 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:58:15 smithi022 ceph-mon[31985]: pgmap v97: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-03-28T17:58:16.032 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:58:15 smithi022 ceph-mon[31985]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-03-28T17:58:16.032 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:58:15 smithi022 ceph-mon[31985]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T17:58:16.032 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:58:15 smithi022 ceph-mon[31985]: Deploying daemon osd.2 on smithi134 2024-03-28T17:58:16.040 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:58:15 smithi134 ceph-mon[35037]: pgmap v97: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-03-28T17:58:16.040 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:58:15 smithi134 ceph-mon[35037]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-03-28T17:58:16.040 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:58:15 smithi134 ceph-mon[35037]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T17:58:16.040 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:58:15 smithi134 ceph-mon[35037]: Deploying daemon osd.2 on smithi134 2024-03-28T17:58:16.182 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:58:15 smithi079 ceph-mon[35312]: pgmap v97: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-03-28T17:58:16.183 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:58:15 smithi079 ceph-mon[35312]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-03-28T17:58:16.183 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:58:15 smithi079 ceph-mon[35312]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T17:58:16.183 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:58:15 smithi079 ceph-mon[35312]: Deploying daemon osd.2 on smithi134 2024-03-28T17:58:17.925 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:58:17 smithi134 ceph-mon[35037]: pgmap v98: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-03-28T17:58:18.032 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:58:17 smithi022 ceph-mon[31985]: pgmap v98: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-03-28T17:58:18.183 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:58:17 smithi079 ceph-mon[35312]: pgmap v98: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-03-28T17:58:19.800 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:58:19 smithi134 ceph-mon[35037]: pgmap v99: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-03-28T17:58:20.032 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:58:19 smithi022 ceph-mon[31985]: pgmap v99: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-03-28T17:58:20.182 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:58:19 smithi079 ceph-mon[35312]: pgmap v99: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-03-28T17:58:21.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:58:21 smithi134 ceph-mon[35037]: pgmap v100: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-03-28T17:58:22.032 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:58:21 smithi022 ceph-mon[31985]: pgmap v100: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-03-28T17:58:22.182 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:58:21 smithi079 ceph-mon[35312]: pgmap v100: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-03-28T17:58:24.005 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:58:23 smithi134 ceph-mon[35037]: pgmap v101: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-03-28T17:58:24.005 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:58:23 smithi134 ceph-mon[35037]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T17:58:24.006 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:58:23 smithi134 ceph-mon[35037]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T17:58:24.006 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:58:23 smithi134 ceph-mon[35037]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T17:58:24.182 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:58:23 smithi079 ceph-mon[35312]: pgmap v101: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-03-28T17:58:24.183 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:58:23 smithi079 ceph-mon[35312]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T17:58:24.183 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:58:23 smithi079 ceph-mon[35312]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T17:58:24.183 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:58:23 smithi079 ceph-mon[35312]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T17:58:24.282 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:58:23 smithi022 ceph-mon[31985]: pgmap v101: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-03-28T17:58:24.282 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:58:23 smithi022 ceph-mon[31985]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T17:58:24.282 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:58:23 smithi022 ceph-mon[31985]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T17:58:24.282 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:58:23 smithi022 ceph-mon[31985]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T17:58:25.993 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:58:25 smithi134 ceph-mon[35037]: pgmap v102: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-03-28T17:58:25.994 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:58:25 smithi134 ceph-mon[35037]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T17:58:25.994 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:58:25 smithi134 ceph-mon[35037]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T17:58:26.183 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:58:25 smithi079 ceph-mon[35312]: pgmap v102: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-03-28T17:58:26.183 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:58:25 smithi079 ceph-mon[35312]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T17:58:26.183 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:58:25 smithi079 ceph-mon[35312]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T17:58:26.282 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:58:25 smithi022 ceph-mon[31985]: pgmap v102: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-03-28T17:58:26.282 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:58:25 smithi022 ceph-mon[31985]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T17:58:26.282 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:58:25 smithi022 ceph-mon[31985]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T17:58:27.611 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-28T17:58:27.610+0000 7f78cfe00700 1 -- 172.21.15.134:0/3957300580 <== mgr.14152 v2:172.21.15.22:6800/4201048392 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f78d80625d0 con 0x7f78bc0a2880 2024-03-28T17:58:27.611 INFO:teuthology.orchestra.run.smithi134.stdout:Created osd(s) 2 on host 'smithi134' 2024-03-28T17:58:27.615 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-28T17:58:27.614+0000 7f78de800700 1 -- 172.21.15.134:0/3957300580 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7f78bc0a2880 msgr2=0x7f78bc0a4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:58:27.615 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-28T17:58:27.614+0000 7f78de800700 1 --2- 172.21.15.134:0/3957300580 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7f78bc0a2880 0x7f78bc0a4d40 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f78c400b5c0 tx=0x7f78c4005fb0 comp rx=0 tx=0).stop 2024-03-28T17:58:27.615 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-28T17:58:27.614+0000 7f78de800700 1 -- 172.21.15.134:0/3957300580 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f78d8104410 msgr2=0x7f78d819a640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:58:27.615 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-28T17:58:27.614+0000 7f78de800700 1 --2- 172.21.15.134:0/3957300580 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f78d8104410 0x7f78d819a640 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f78c800d9b0 tx=0x7f78c800dd70 comp rx=0 tx=0).stop 2024-03-28T17:58:27.615 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-28T17:58:27.614+0000 7f78de800700 1 -- 172.21.15.134:0/3957300580 shutdown_connections 2024-03-28T17:58:27.615 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-28T17:58:27.614+0000 7f78de800700 1 --2- 172.21.15.134:0/3957300580 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f78d8105fb0 0x7f78d81a1c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:58:27.615 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-28T17:58:27.614+0000 7f78de800700 1 --2- 172.21.15.134:0/3957300580 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7f78bc0a2880 0x7f78bc0a4d40 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:58:27.615 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-28T17:58:27.614+0000 7f78de800700 1 --2- 172.21.15.134:0/3957300580 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f78d8104410 0x7f78d819a640 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:58:27.615 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-28T17:58:27.614+0000 7f78de800700 1 --2- 172.21.15.134:0/3957300580 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f78d8105610 0x7f78d819ab80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:58:27.615 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-28T17:58:27.614+0000 7f78de800700 1 -- 172.21.15.134:0/3957300580 >> 172.21.15.134:0/3957300580 conn(0x7f78d80ff9b0 msgr2=0x7f78d8108910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T17:58:27.615 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-28T17:58:27.614+0000 7f78de800700 1 -- 172.21.15.134:0/3957300580 shutdown_connections 2024-03-28T17:58:27.615 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-28T17:58:27.614+0000 7f78de800700 1 -- 172.21.15.134:0/3957300580 wait complete. 2024-03-28T17:58:28.175 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:58:27 smithi134 ceph-mon[35037]: pgmap v103: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-03-28T17:58:28.175 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:58:27 smithi134 ceph-mon[35037]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T17:58:28.175 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:58:27 smithi134 ceph-mon[35037]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T17:58:28.175 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:58:27 smithi134 ceph-mon[35037]: from='osd.2 [v2:172.21.15.134:6800/470588085,v1:172.21.15.134:6801/470588085]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-03-28T17:58:28.175 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:58:27 smithi134 ceph-mon[35037]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-03-28T17:58:28.175 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:58:27 smithi134 ceph-mon[35037]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T17:58:28.176 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:58:27 smithi134 ceph-mon[35037]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T17:58:28.176 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:58:27 smithi134 ceph-mon[35037]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T17:58:28.176 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:58:27 smithi134 ceph-mon[35037]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T17:58:28.183 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:58:27 smithi079 ceph-mon[35312]: pgmap v103: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-03-28T17:58:28.183 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:58:27 smithi079 ceph-mon[35312]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T17:58:28.183 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:58:27 smithi079 ceph-mon[35312]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T17:58:28.183 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:58:27 smithi079 ceph-mon[35312]: from='osd.2 [v2:172.21.15.134:6800/470588085,v1:172.21.15.134:6801/470588085]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-03-28T17:58:28.183 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:58:27 smithi079 ceph-mon[35312]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-03-28T17:58:28.183 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:58:27 smithi079 ceph-mon[35312]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T17:58:28.183 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:58:27 smithi079 ceph-mon[35312]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T17:58:28.183 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:58:27 smithi079 ceph-mon[35312]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T17:58:28.183 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:58:27 smithi079 ceph-mon[35312]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T17:58:28.282 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:58:27 smithi022 ceph-mon[31985]: pgmap v103: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-03-28T17:58:28.282 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:58:27 smithi022 ceph-mon[31985]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T17:58:28.282 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:58:27 smithi022 ceph-mon[31985]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T17:58:28.282 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:58:27 smithi022 ceph-mon[31985]: from='osd.2 [v2:172.21.15.134:6800/470588085,v1:172.21.15.134:6801/470588085]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-03-28T17:58:28.282 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:58:27 smithi022 ceph-mon[31985]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-03-28T17:58:28.282 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:58:27 smithi022 ceph-mon[31985]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T17:58:28.282 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:58:27 smithi022 ceph-mon[31985]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T17:58:28.282 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:58:27 smithi022 ceph-mon[31985]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T17:58:28.282 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:58:27 smithi022 ceph-mon[31985]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T17:58:28.409 DEBUG:teuthology.orchestra.run.smithi134:osd.2> sudo journalctl -f -n 0 -u ceph-fd220748-ed2b-11ee-95d0-87774f69a715@osd.2.service 2024-03-28T17:58:28.412 INFO:tasks.cephadm:Deploying osd.3 on smithi022 with /dev/vg_nvme/lv_3... 2024-03-28T17:58:28.413 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fd220748-ed2b-11ee-95d0-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-03-28T17:58:28.884 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/fd220748-ed2b-11ee-95d0-87774f69a715/mon.a/config 2024-03-28T17:58:28.925 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:58:28 smithi134 ceph-mon[35037]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-03-28T17:58:28.925 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:58:28 smithi134 ceph-mon[35037]: osdmap e16: 3 total, 2 up, 3 in 2024-03-28T17:58:28.925 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:58:28 smithi134 ceph-mon[35037]: from='osd.2 [v2:172.21.15.134:6800/470588085,v1:172.21.15.134:6801/470588085]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi134", "root=default"]}]: dispatch 2024-03-28T17:58:28.925 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:58:28 smithi134 ceph-mon[35037]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-28T17:58:28.925 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:58:28 smithi134 ceph-mon[35037]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi134", "root=default"]}]: dispatch 2024-03-28T17:58:28.925 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:58:28 smithi134 ceph-mon[35037]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi134", "root=default"]}]': finished 2024-03-28T17:58:28.925 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:58:28 smithi134 ceph-mon[35037]: osdmap e17: 3 total, 2 up, 3 in 2024-03-28T17:58:28.925 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:58:28 smithi134 ceph-mon[35037]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-28T17:58:29.183 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:58:28 smithi079 ceph-mon[35312]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-03-28T17:58:29.183 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:58:28 smithi079 ceph-mon[35312]: osdmap e16: 3 total, 2 up, 3 in 2024-03-28T17:58:29.183 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:58:28 smithi079 ceph-mon[35312]: from='osd.2 [v2:172.21.15.134:6800/470588085,v1:172.21.15.134:6801/470588085]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi134", "root=default"]}]: dispatch 2024-03-28T17:58:29.183 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:58:28 smithi079 ceph-mon[35312]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-28T17:58:29.183 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:58:28 smithi079 ceph-mon[35312]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi134", "root=default"]}]: dispatch 2024-03-28T17:58:29.183 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:58:28 smithi079 ceph-mon[35312]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi134", "root=default"]}]': finished 2024-03-28T17:58:29.183 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:58:28 smithi079 ceph-mon[35312]: osdmap e17: 3 total, 2 up, 3 in 2024-03-28T17:58:29.184 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:58:28 smithi079 ceph-mon[35312]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-28T17:58:29.187 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:58:28 smithi022 ceph-mon[31985]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-03-28T17:58:29.187 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:58:28 smithi022 ceph-mon[31985]: osdmap e16: 3 total, 2 up, 3 in 2024-03-28T17:58:29.187 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:58:28 smithi022 ceph-mon[31985]: from='osd.2 [v2:172.21.15.134:6800/470588085,v1:172.21.15.134:6801/470588085]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi134", "root=default"]}]: dispatch 2024-03-28T17:58:29.187 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:58:28 smithi022 ceph-mon[31985]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-28T17:58:29.187 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:58:28 smithi022 ceph-mon[31985]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi134", "root=default"]}]: dispatch 2024-03-28T17:58:29.187 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:58:28 smithi022 ceph-mon[31985]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi134", "root=default"]}]': finished 2024-03-28T17:58:29.188 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:58:28 smithi022 ceph-mon[31985]: osdmap e17: 3 total, 2 up, 3 in 2024-03-28T17:58:29.188 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:58:28 smithi022 ceph-mon[31985]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-28T17:58:30.036 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:58:29 smithi134 ceph-mon[35037]: pgmap v105: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-03-28T17:58:30.036 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:58:29 smithi134 ceph-mon[35037]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-28T17:58:30.036 INFO:journalctl@ceph.osd.2.smithi134.stdout:Mar 28 17:58:29 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-osd-2[40319]: 2024-03-28T17:58:29.879+0000 7f2e39e00700 -1 osd.2 0 waiting for initial osdmap 2024-03-28T17:58:30.036 INFO:journalctl@ceph.osd.2.smithi134.stdout:Mar 28 17:58:29 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-osd-2[40319]: 2024-03-28T17:58:29.923+0000 7f2e2fc00700 -1 osd.2 17 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-28T17:58:30.183 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:58:29 smithi079 ceph-mon[35312]: pgmap v105: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-03-28T17:58:30.183 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:58:29 smithi079 ceph-mon[35312]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-28T17:58:30.282 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:58:29 smithi022 ceph-mon[31985]: pgmap v105: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-03-28T17:58:30.282 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:58:29 smithi022 ceph-mon[31985]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-28T17:58:30.973 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:58:30 smithi022 ceph-mon[31985]: purged_snaps scrub starts 2024-03-28T17:58:30.974 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:58:30 smithi022 ceph-mon[31985]: purged_snaps scrub ok 2024-03-28T17:58:30.974 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:58:30 smithi022 ceph-mon[31985]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-28T17:58:30.974 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:58:30 smithi022 ceph-mon[31985]: from='osd.2 ' entity='osd.2' 2024-03-28T17:58:31.175 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:58:30 smithi134 ceph-mon[35037]: purged_snaps scrub starts 2024-03-28T17:58:31.175 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:58:30 smithi134 ceph-mon[35037]: purged_snaps scrub ok 2024-03-28T17:58:31.175 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:58:30 smithi134 ceph-mon[35037]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-28T17:58:31.175 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:58:30 smithi134 ceph-mon[35037]: from='osd.2 ' entity='osd.2' 2024-03-28T17:58:31.183 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:58:30 smithi079 ceph-mon[35312]: purged_snaps scrub starts 2024-03-28T17:58:31.183 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:58:30 smithi079 ceph-mon[35312]: purged_snaps scrub ok 2024-03-28T17:58:31.183 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:58:30 smithi079 ceph-mon[35312]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-28T17:58:31.183 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:58:30 smithi079 ceph-mon[35312]: from='osd.2 ' entity='osd.2' 2024-03-28T17:58:31.916 INFO:teuthology.orchestra.run.smithi022.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-03-28T17:58:31.916 INFO:teuthology.orchestra.run.smithi022.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-03-28T17:58:31.916 INFO:teuthology.orchestra.run.smithi022.stderr: stderr: 10+0 records in 2024-03-28T17:58:31.917 INFO:teuthology.orchestra.run.smithi022.stderr:10+0 records out 2024-03-28T17:58:31.917 INFO:teuthology.orchestra.run.smithi022.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.057481 s, 182 MB/s 2024-03-28T17:58:31.917 INFO:teuthology.orchestra.run.smithi022.stderr: stderr: 2024-03-28T17:58:31.917 INFO:teuthology.orchestra.run.smithi022.stderr:--> Zapping successful for: 2024-03-28T17:58:32.175 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:58:31 smithi134 ceph-mon[35037]: pgmap v107: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-03-28T17:58:32.175 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:58:31 smithi134 ceph-mon[35037]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-28T17:58:32.175 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:58:31 smithi134 ceph-mon[35037]: osd.2 [v2:172.21.15.134:6800/470588085,v1:172.21.15.134:6801/470588085] boot 2024-03-28T17:58:32.175 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:58:31 smithi134 ceph-mon[35037]: osdmap e18: 3 total, 3 up, 3 in 2024-03-28T17:58:32.175 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:58:31 smithi134 ceph-mon[35037]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-28T17:58:32.183 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:58:31 smithi079 ceph-mon[35312]: pgmap v107: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-03-28T17:58:32.183 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:58:31 smithi079 ceph-mon[35312]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-28T17:58:32.183 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:58:31 smithi079 ceph-mon[35312]: osd.2 [v2:172.21.15.134:6800/470588085,v1:172.21.15.134:6801/470588085] boot 2024-03-28T17:58:32.183 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:58:31 smithi079 ceph-mon[35312]: osdmap e18: 3 total, 3 up, 3 in 2024-03-28T17:58:32.183 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:58:31 smithi079 ceph-mon[35312]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-28T17:58:32.282 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:58:31 smithi022 ceph-mon[31985]: pgmap v107: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-03-28T17:58:32.282 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:58:31 smithi022 ceph-mon[31985]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-28T17:58:32.282 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:58:31 smithi022 ceph-mon[31985]: osd.2 [v2:172.21.15.134:6800/470588085,v1:172.21.15.134:6801/470588085] boot 2024-03-28T17:58:32.282 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:58:31 smithi022 ceph-mon[31985]: osdmap e18: 3 total, 3 up, 3 in 2024-03-28T17:58:32.282 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:58:31 smithi022 ceph-mon[31985]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-28T17:58:32.743 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fd220748-ed2b-11ee-95d0-87774f69a715 -- ceph orch daemon add osd smithi022:vg_nvme/lv_3 2024-03-28T17:58:33.216 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/fd220748-ed2b-11ee-95d0-87774f69a715/mon.a/config 2024-03-28T17:58:33.525 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:58:33 smithi022 ceph-mon[31985]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-03-28T17:58:33.525 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:58:33 smithi022 ceph-mon[31985]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-03-28T17:58:33.525 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:58:33 smithi022 ceph-mon[31985]: osdmap e19: 3 total, 3 up, 3 in 2024-03-28T17:58:33.525 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:58:33 smithi022 ceph-mon[31985]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-03-28T17:58:33.525 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:58:33 smithi022 ceph-mon[31985]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T17:58:33.525 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:58:33 smithi022 ceph-mon[31985]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T17:58:33.525 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:58:33 smithi022 ceph-mon[31985]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-28T17:58:33.525 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:58:33 smithi022 ceph-mon[31985]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T17:58:33.525 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:58:33 smithi022 ceph-mon[31985]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T17:58:33.525 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:58:33 smithi022 ceph-mon[31985]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T17:58:33.525 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:58:33 smithi022 ceph-mon[31985]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T17:58:33.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:58:33 smithi134 ceph-mon[35037]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-03-28T17:58:33.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:58:33 smithi134 ceph-mon[35037]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-03-28T17:58:33.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:58:33 smithi134 ceph-mon[35037]: osdmap e19: 3 total, 3 up, 3 in 2024-03-28T17:58:33.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:58:33 smithi134 ceph-mon[35037]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-03-28T17:58:33.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:58:33 smithi134 ceph-mon[35037]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T17:58:33.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:58:33 smithi134 ceph-mon[35037]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T17:58:33.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:58:33 smithi134 ceph-mon[35037]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-28T17:58:33.676 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:58:33 smithi134 ceph-mon[35037]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T17:58:33.676 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:58:33 smithi134 ceph-mon[35037]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T17:58:33.676 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:58:33 smithi134 ceph-mon[35037]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T17:58:33.676 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:58:33 smithi134 ceph-mon[35037]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T17:58:33.683 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:58:33 smithi079 ceph-mon[35312]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-03-28T17:58:33.683 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:58:33 smithi079 ceph-mon[35312]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-03-28T17:58:33.683 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:58:33 smithi079 ceph-mon[35312]: osdmap e19: 3 total, 3 up, 3 in 2024-03-28T17:58:33.683 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:58:33 smithi079 ceph-mon[35312]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-03-28T17:58:33.683 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:58:33 smithi079 ceph-mon[35312]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T17:58:33.683 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:58:33 smithi079 ceph-mon[35312]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T17:58:33.683 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:58:33 smithi079 ceph-mon[35312]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-28T17:58:33.683 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:58:33 smithi079 ceph-mon[35312]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T17:58:33.683 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:58:33 smithi079 ceph-mon[35312]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T17:58:33.683 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:58:33 smithi079 ceph-mon[35312]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T17:58:33.683 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:58:33 smithi079 ceph-mon[35312]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T17:58:34.404 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:58:34 smithi022 ceph-mon[31985]: pgmap v109: 0 pgs: ; 0 B data, 88 MiB used, 268 GiB / 268 GiB avail 2024-03-28T17:58:34.404 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:58:34 smithi022 ceph-mon[31985]: Detected new or changed devices on smithi134 2024-03-28T17:58:34.404 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:58:34 smithi022 ceph-mon[31985]: Adjusting osd_memory_target on smithi134 to 17091M 2024-03-28T17:58:34.404 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:58:34 smithi022 ceph-mon[31985]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-03-28T17:58:34.404 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:58:34 smithi022 ceph-mon[31985]: osdmap e20: 3 total, 3 up, 3 in 2024-03-28T17:58:34.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:58:34 smithi134 ceph-mon[35037]: pgmap v109: 0 pgs: ; 0 B data, 88 MiB used, 268 GiB / 268 GiB avail 2024-03-28T17:58:34.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:58:34 smithi134 ceph-mon[35037]: Detected new or changed devices on smithi134 2024-03-28T17:58:34.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:58:34 smithi134 ceph-mon[35037]: Adjusting osd_memory_target on smithi134 to 17091M 2024-03-28T17:58:34.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:58:34 smithi134 ceph-mon[35037]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-03-28T17:58:34.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:58:34 smithi134 ceph-mon[35037]: osdmap e20: 3 total, 3 up, 3 in 2024-03-28T17:58:34.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:58:34 smithi079 ceph-mon[35312]: pgmap v109: 0 pgs: ; 0 B data, 88 MiB used, 268 GiB / 268 GiB avail 2024-03-28T17:58:34.683 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:58:34 smithi079 ceph-mon[35312]: Detected new or changed devices on smithi134 2024-03-28T17:58:34.683 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:58:34 smithi079 ceph-mon[35312]: Adjusting osd_memory_target on smithi134 to 17091M 2024-03-28T17:58:34.683 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:58:34 smithi079 ceph-mon[35312]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-03-28T17:58:34.683 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:58:34 smithi079 ceph-mon[35312]: osdmap e20: 3 total, 3 up, 3 in 2024-03-28T17:58:35.308 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T17:58:35.306+0000 7f025c800700 1 -- 172.21.15.22:0/3651613960 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0254102c40 msgr2=0x7f0254105030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:58:35.308 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T17:58:35.306+0000 7f025c800700 1 --2- 172.21.15.22:0/3651613960 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0254102c40 0x7f0254105030 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f0248009990 tx=0x7f0248009ca0 comp rx=0 tx=0).stop 2024-03-28T17:58:35.308 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T17:58:35.306+0000 7f025c800700 1 -- 172.21.15.22:0/3651613960 shutdown_connections 2024-03-28T17:58:35.308 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T17:58:35.306+0000 7f025c800700 1 --2- 172.21.15.22:0/3651613960 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0254107ea0 0x7f025410a310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:58:35.308 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T17:58:35.306+0000 7f025c800700 1 --2- 172.21.15.22:0/3651613960 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0254102c40 0x7f0254105030 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:58:35.309 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T17:58:35.306+0000 7f025c800700 1 --2- 172.21.15.22:0/3651613960 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f0254105570 0x7f0254107960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:58:35.309 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T17:58:35.306+0000 7f025c800700 1 -- 172.21.15.22:0/3651613960 >> 172.21.15.22:0/3651613960 conn(0x7f02540fc650 msgr2=0x7f02540fea90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T17:58:35.309 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T17:58:35.306+0000 7f025c800700 1 -- 172.21.15.22:0/3651613960 shutdown_connections 2024-03-28T17:58:35.309 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T17:58:35.306+0000 7f025c800700 1 -- 172.21.15.22:0/3651613960 wait complete. 2024-03-28T17:58:35.309 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T17:58:35.307+0000 7f025c800700 1 Processor -- start 2024-03-28T17:58:35.309 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T17:58:35.307+0000 7f025c800700 1 -- start start 2024-03-28T17:58:35.309 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T17:58:35.308+0000 7f025c800700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0254102c40 0x7f025419a5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:58:35.309 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T17:58:35.308+0000 7f025c800700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f0254105570 0x7f025419ab00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:58:35.309 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T17:58:35.308+0000 7f025c800700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0254107ea0 0x7f02541a1b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:58:35.309 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T17:58:35.308+0000 7f025c800700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f025410ddc0 con 0x7f0254102c40 2024-03-28T17:58:35.309 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T17:58:35.308+0000 7f025c800700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f025410dc40 con 0x7f0254105570 2024-03-28T17:58:35.310 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T17:58:35.308+0000 7f025c800700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f025410df40 con 0x7f0254107ea0 2024-03-28T17:58:35.310 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T17:58:35.308+0000 7f0259e00700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0254102c40 0x7f025419a5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:58:35.310 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T17:58:35.308+0000 7f0259e00700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0254102c40 0x7f025419a5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:33116/0 (socket says 172.21.15.22:33116) 2024-03-28T17:58:35.310 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T17:58:35.308+0000 7f0259e00700 1 -- 172.21.15.22:0/3196408980 learned_addr learned my addr 172.21.15.22:0/3196408980 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-28T17:58:35.310 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T17:58:35.308+0000 7f0259400700 1 --2- 172.21.15.22:0/3196408980 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f0254105570 0x7f025419ab00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:58:35.310 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T17:58:35.308+0000 7f025a800700 1 --2- 172.21.15.22:0/3196408980 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0254107ea0 0x7f02541a1b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:58:35.311 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T17:58:35.309+0000 7f0259e00700 1 -- 172.21.15.22:0/3196408980 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0254107ea0 msgr2=0x7f02541a1b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:58:35.311 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T17:58:35.309+0000 7f0259e00700 1 --2- 172.21.15.22:0/3196408980 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0254107ea0 0x7f02541a1b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:58:35.311 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T17:58:35.309+0000 7f0259e00700 1 -- 172.21.15.22:0/3196408980 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f0254105570 msgr2=0x7f025419ab00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:58:35.311 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T17:58:35.309+0000 7f0259e00700 1 --2- 172.21.15.22:0/3196408980 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f0254105570 0x7f025419ab00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:58:35.311 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T17:58:35.309+0000 7f0259e00700 1 -- 172.21.15.22:0/3196408980 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f02541a2220 con 0x7f0254102c40 2024-03-28T17:58:35.311 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T17:58:35.309+0000 7f025a800700 1 --2- 172.21.15.22:0/3196408980 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0254107ea0 0x7f02541a1b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T17:58:35.311 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T17:58:35.309+0000 7f0259400700 1 --2- 172.21.15.22:0/3196408980 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f0254105570 0x7f025419ab00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-28T17:58:35.311 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T17:58:35.309+0000 7f0259e00700 1 --2- 172.21.15.22:0/3196408980 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0254102c40 0x7f025419a5c0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f0248005910 tx=0x7f0248009580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T17:58:35.311 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T17:58:35.309+0000 7f024ea00700 1 -- 172.21.15.22:0/3196408980 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0248027070 con 0x7f0254102c40 2024-03-28T17:58:35.312 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T17:58:35.309+0000 7f024ea00700 1 -- 172.21.15.22:0/3196408980 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f0248004140 con 0x7f0254102c40 2024-03-28T17:58:35.312 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T17:58:35.309+0000 7f025c800700 1 -- 172.21.15.22:0/3196408980 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f02541a24b0 con 0x7f0254102c40 2024-03-28T17:58:35.312 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T17:58:35.309+0000 7f024ea00700 1 -- 172.21.15.22:0/3196408980 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0248022e60 con 0x7f0254102c40 2024-03-28T17:58:35.312 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T17:58:35.309+0000 7f025c800700 1 -- 172.21.15.22:0/3196408980 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f02541a2960 con 0x7f0254102c40 2024-03-28T17:58:35.312 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T17:58:35.310+0000 7f025c800700 1 -- 172.21.15.22:0/3196408980 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0254068e50 con 0x7f0254102c40 2024-03-28T17:58:35.312 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T17:58:35.312+0000 7f024ea00700 1 -- 172.21.15.22:0/3196408980 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 16) v1 ==== 137290+0+0 (secure 0 0 0) 0x7f024802a070 con 0x7f0254102c40 2024-03-28T17:58:35.314 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T17:58:35.314+0000 7f024ea00700 1 --2- 172.21.15.22:0/3196408980 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7f023c0a2910 0x7f023c0a4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:58:35.314 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T17:58:35.314+0000 7f024ea00700 1 -- 172.21.15.22:0/3196408980 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 3237+0+0 (secure 0 0 0) 0x7f02480ce490 con 0x7f0254102c40 2024-03-28T17:58:35.314 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T17:58:35.314+0000 7f0259400700 1 --2- 172.21.15.22:0/3196408980 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7f023c0a2910 0x7f023c0a4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:58:35.314 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T17:58:35.315+0000 7f0259400700 1 --2- 172.21.15.22:0/3196408980 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7f023c0a2910 0x7f023c0a4dd0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f0244005950 tx=0x7f02440058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T17:58:35.318 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T17:58:35.319+0000 7f024ea00700 1 -- 172.21.15.22:0/3196408980 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f0248013070 con 0x7f0254102c40 2024-03-28T17:58:35.532 INFO:journalctl@ceph.osd.0.smithi022.stdout:Mar 28 17:58:35 smithi022 sudo[51338]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-03-28T17:58:35.532 INFO:journalctl@ceph.osd.0.smithi022.stdout:Mar 28 17:58:35 smithi022 sudo[51338]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-03-28T17:58:35.532 INFO:journalctl@ceph.osd.0.smithi022.stdout:Mar 28 17:58:35 smithi022 sudo[51338]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-03-28T17:58:35.532 INFO:journalctl@ceph.osd.0.smithi022.stdout:Mar 28 17:58:35 smithi022 sudo[51338]: pam_unix(sudo:session): session closed for user root 2024-03-28T17:58:35.554 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T17:58:35.555+0000 7f025c800700 1 -- 172.21.15.22:0/3196408980 --> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi022:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) v1 -- 0x7f02540625d0 con 0x7f023c0a2910 2024-03-28T17:58:36.032 INFO:journalctl@ceph.osd.0.smithi022.stdout:Mar 28 17:58:35 smithi022 sudo[51388]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-03-28T17:58:36.032 INFO:journalctl@ceph.osd.0.smithi022.stdout:Mar 28 17:58:35 smithi022 sudo[51388]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-03-28T17:58:36.032 INFO:journalctl@ceph.osd.0.smithi022.stdout:Mar 28 17:58:35 smithi022 sudo[51388]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-03-28T17:58:36.032 INFO:journalctl@ceph.osd.0.smithi022.stdout:Mar 28 17:58:35 smithi022 sudo[51388]: pam_unix(sudo:session): session closed for user root 2024-03-28T17:58:36.465 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:58:36 smithi022 ceph-mon[31985]: pgmap v112: 1 pgs: 1 creating+peering; 0 B data, 88 MiB used, 268 GiB / 268 GiB avail 2024-03-28T17:58:36.465 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:58:36 smithi022 ceph-mon[31985]: osdmap e21: 3 total, 3 up, 3 in 2024-03-28T17:58:36.465 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:58:36 smithi022 ceph-mon[31985]: mgrmap e16: x(active, since 3m), standbys: z, y 2024-03-28T17:58:36.465 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:58:36 smithi022 ceph-mon[31985]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-28T17:58:36.465 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:58:36 smithi022 ceph-mon[31985]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-28T17:58:36.465 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:58:36 smithi022 ceph-mon[31985]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T17:58:36.466 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:58:36 smithi022 ceph-mon[31985]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T17:58:36.485 INFO:journalctl@ceph.osd.1.smithi079.stdout:Mar 28 17:58:36 smithi079 sudo[43981]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-03-28T17:58:36.486 INFO:journalctl@ceph.osd.1.smithi079.stdout:Mar 28 17:58:36 smithi079 sudo[43981]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-03-28T17:58:36.486 INFO:journalctl@ceph.osd.1.smithi079.stdout:Mar 28 17:58:36 smithi079 sudo[43981]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-03-28T17:58:36.486 INFO:journalctl@ceph.osd.1.smithi079.stdout:Mar 28 17:58:36 smithi079 sudo[43981]: pam_unix(sudo:session): session closed for user root 2024-03-28T17:58:36.486 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:58:36 smithi079 ceph-mon[35312]: pgmap v112: 1 pgs: 1 creating+peering; 0 B data, 88 MiB used, 268 GiB / 268 GiB avail 2024-03-28T17:58:36.486 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:58:36 smithi079 ceph-mon[35312]: osdmap e21: 3 total, 3 up, 3 in 2024-03-28T17:58:36.486 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:58:36 smithi079 ceph-mon[35312]: mgrmap e16: x(active, since 3m), standbys: z, y 2024-03-28T17:58:36.486 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:58:36 smithi079 ceph-mon[35312]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-28T17:58:36.487 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:58:36 smithi079 ceph-mon[35312]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-28T17:58:36.487 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:58:36 smithi079 ceph-mon[35312]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T17:58:36.487 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:58:36 smithi079 ceph-mon[35312]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T17:58:36.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:58:36 smithi134 ceph-mon[35037]: pgmap v112: 1 pgs: 1 creating+peering; 0 B data, 88 MiB used, 268 GiB / 268 GiB avail 2024-03-28T17:58:36.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:58:36 smithi134 ceph-mon[35037]: osdmap e21: 3 total, 3 up, 3 in 2024-03-28T17:58:36.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:58:36 smithi134 ceph-mon[35037]: mgrmap e16: x(active, since 3m), standbys: z, y 2024-03-28T17:58:36.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:58:36 smithi134 ceph-mon[35037]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-28T17:58:36.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:58:36 smithi134 ceph-mon[35037]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-28T17:58:36.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:58:36 smithi134 ceph-mon[35037]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T17:58:36.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:58:36 smithi134 ceph-mon[35037]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T17:58:37.183 INFO:journalctl@ceph.osd.1.smithi079.stdout:Mar 28 17:58:36 smithi079 sudo[43984]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-03-28T17:58:37.183 INFO:journalctl@ceph.osd.1.smithi079.stdout:Mar 28 17:58:36 smithi079 sudo[43984]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-03-28T17:58:37.183 INFO:journalctl@ceph.osd.1.smithi079.stdout:Mar 28 17:58:36 smithi079 sudo[43984]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-03-28T17:58:37.183 INFO:journalctl@ceph.osd.1.smithi079.stdout:Mar 28 17:58:36 smithi079 sudo[43984]: pam_unix(sudo:session): session closed for user root 2024-03-28T17:58:37.477 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:58:37 smithi022 ceph-mon[31985]: from='client.14301 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi022:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-03-28T17:58:37.478 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:58:37 smithi022 ceph-mon[31985]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T17:58:37.488 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:58:37 smithi134 ceph-mon[35037]: from='client.14301 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi022:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-03-28T17:58:37.488 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:58:37 smithi134 ceph-mon[35037]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T17:58:37.683 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:58:37 smithi079 ceph-mon[35312]: from='client.14301 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi022:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-03-28T17:58:37.683 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:58:37 smithi079 ceph-mon[35312]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T17:58:37.925 INFO:journalctl@ceph.osd.2.smithi134.stdout:Mar 28 17:58:37 smithi134 sudo[44076]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-03-28T17:58:37.925 INFO:journalctl@ceph.osd.2.smithi134.stdout:Mar 28 17:58:37 smithi134 sudo[44076]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-03-28T17:58:37.925 INFO:journalctl@ceph.osd.2.smithi134.stdout:Mar 28 17:58:37 smithi134 sudo[44076]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-03-28T17:58:37.925 INFO:journalctl@ceph.osd.2.smithi134.stdout:Mar 28 17:58:37 smithi134 sudo[44076]: pam_unix(sudo:session): session closed for user root 2024-03-28T17:58:38.424 INFO:journalctl@ceph.osd.2.smithi134.stdout:Mar 28 17:58:38 smithi134 sudo[44079]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-03-28T17:58:38.425 INFO:journalctl@ceph.osd.2.smithi134.stdout:Mar 28 17:58:38 smithi134 sudo[44079]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-03-28T17:58:38.425 INFO:journalctl@ceph.osd.2.smithi134.stdout:Mar 28 17:58:38 smithi134 sudo[44079]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-03-28T17:58:38.425 INFO:journalctl@ceph.osd.2.smithi134.stdout:Mar 28 17:58:38 smithi134 sudo[44079]: pam_unix(sudo:session): session closed for user root 2024-03-28T17:58:39.032 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:58:38 smithi022 sudo[51542]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-03-28T17:58:39.032 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:58:38 smithi022 sudo[51542]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-03-28T17:58:39.033 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:58:38 smithi022 sudo[51542]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-03-28T17:58:39.033 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:58:38 smithi022 sudo[51542]: pam_unix(sudo:session): session closed for user root 2024-03-28T17:58:39.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:58:39 smithi134 ceph-mon[35037]: pgmap v114: 1 pgs: 1 creating+peering; 0 B data, 88 MiB used, 268 GiB / 268 GiB avail 2024-03-28T17:58:39.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:58:39 smithi134 ceph-mon[35037]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T17:58:39.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:58:39 smithi134 ceph-mon[35037]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-03-28T17:58:39.683 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:58:39 smithi079 ceph-mon[35312]: pgmap v114: 1 pgs: 1 creating+peering; 0 B data, 88 MiB used, 268 GiB / 268 GiB avail 2024-03-28T17:58:39.683 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:58:39 smithi079 ceph-mon[35312]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T17:58:39.683 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:58:39 smithi079 ceph-mon[35312]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-03-28T17:58:39.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:58:39 smithi022 sudo[51583]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-03-28T17:58:39.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:58:39 smithi022 sudo[51583]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-03-28T17:58:39.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:58:39 smithi022 sudo[51583]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-03-28T17:58:39.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:58:39 smithi022 sudo[51583]: pam_unix(sudo:session): session closed for user root 2024-03-28T17:58:39.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:58:39 smithi022 ceph-mon[31985]: pgmap v114: 1 pgs: 1 creating+peering; 0 B data, 88 MiB used, 268 GiB / 268 GiB avail 2024-03-28T17:58:39.783 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:58:39 smithi022 ceph-mon[31985]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T17:58:39.783 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:58:39 smithi022 ceph-mon[31985]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-03-28T17:58:40.175 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:58:39 smithi134 sudo[44082]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-03-28T17:58:40.175 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:58:39 smithi134 sudo[44082]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-03-28T17:58:40.175 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:58:39 smithi134 sudo[44082]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-03-28T17:58:40.175 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:58:39 smithi134 sudo[44082]: pam_unix(sudo:session): session closed for user root 2024-03-28T17:58:40.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:58:40 smithi134 sudo[44085]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-03-28T17:58:40.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:58:40 smithi134 sudo[44085]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-03-28T17:58:40.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:58:40 smithi134 sudo[44085]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-03-28T17:58:40.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:58:40 smithi134 sudo[44085]: pam_unix(sudo:session): session closed for user root 2024-03-28T17:58:41.433 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:58:40 smithi079 sudo[43987]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-03-28T17:58:41.433 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:58:40 smithi079 sudo[43987]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-03-28T17:58:41.433 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:58:40 smithi079 sudo[43987]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-03-28T17:58:41.433 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:58:40 smithi079 sudo[43987]: pam_unix(sudo:session): session closed for user root 2024-03-28T17:58:41.915 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:58:41 smithi022 ceph-mon[31985]: pgmap v115: 1 pgs: 1 creating+peering; 0 B data, 94 MiB used, 268 GiB / 268 GiB avail 2024-03-28T17:58:41.915 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:58:41 smithi022 ceph-mon[31985]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-03-28T17:58:41.915 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:58:41 smithi022 ceph-mon[31985]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-28T17:58:41.916 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:58:41 smithi022 ceph-mon[31985]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-28T17:58:41.916 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:58:41 smithi022 ceph-mon[31985]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-28T17:58:41.916 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:58:41 smithi022 ceph-mon[31985]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-28T17:58:41.916 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:58:41 smithi022 ceph-mon[31985]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-28T17:58:41.916 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:58:41 smithi022 ceph-mon[31985]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-28T17:58:41.916 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:58:41 smithi022 ceph-mon[31985]: from='client.? 172.21.15.22:0/3929096249' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "81a6c547-2a50-44c3-84d7-23ab6831790c"}]: dispatch 2024-03-28T17:58:41.925 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:58:41 smithi134 ceph-mon[35037]: pgmap v115: 1 pgs: 1 creating+peering; 0 B data, 94 MiB used, 268 GiB / 268 GiB avail 2024-03-28T17:58:41.925 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:58:41 smithi134 ceph-mon[35037]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-03-28T17:58:41.925 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:58:41 smithi134 ceph-mon[35037]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-28T17:58:41.925 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:58:41 smithi134 ceph-mon[35037]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-28T17:58:41.925 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:58:41 smithi134 ceph-mon[35037]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-28T17:58:41.925 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:58:41 smithi134 ceph-mon[35037]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-28T17:58:41.925 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:58:41 smithi134 ceph-mon[35037]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-28T17:58:41.925 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:58:41 smithi134 ceph-mon[35037]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-28T17:58:41.925 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:58:41 smithi134 ceph-mon[35037]: from='client.? 172.21.15.22:0/3929096249' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "81a6c547-2a50-44c3-84d7-23ab6831790c"}]: dispatch 2024-03-28T17:58:41.933 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:58:41 smithi079 sudo[43990]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-03-28T17:58:41.933 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:58:41 smithi079 sudo[43990]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-03-28T17:58:41.933 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:58:41 smithi079 sudo[43990]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-03-28T17:58:41.933 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:58:41 smithi079 sudo[43990]: pam_unix(sudo:session): session closed for user root 2024-03-28T17:58:41.933 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:58:41 smithi079 ceph-mon[35312]: pgmap v115: 1 pgs: 1 creating+peering; 0 B data, 94 MiB used, 268 GiB / 268 GiB avail 2024-03-28T17:58:41.933 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:58:41 smithi079 ceph-mon[35312]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-03-28T17:58:41.933 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:58:41 smithi079 ceph-mon[35312]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-28T17:58:41.934 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:58:41 smithi079 ceph-mon[35312]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-28T17:58:41.934 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:58:41 smithi079 ceph-mon[35312]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-28T17:58:41.934 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:58:41 smithi079 ceph-mon[35312]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-28T17:58:41.934 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:58:41 smithi079 ceph-mon[35312]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-28T17:58:41.934 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:58:41 smithi079 ceph-mon[35312]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-28T17:58:41.934 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:58:41 smithi079 ceph-mon[35312]: from='client.? 172.21.15.22:0/3929096249' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "81a6c547-2a50-44c3-84d7-23ab6831790c"}]: dispatch 2024-03-28T17:58:42.925 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:58:42 smithi134 ceph-mon[35037]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-03-28T17:58:42.925 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:58:42 smithi134 ceph-mon[35037]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-03-28T17:58:42.925 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:58:42 smithi134 ceph-mon[35037]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-03-28T17:58:42.925 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:58:42 smithi134 ceph-mon[35037]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-28T17:58:42.925 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:58:42 smithi134 ceph-mon[35037]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-28T17:58:42.925 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:58:42 smithi134 ceph-mon[35037]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-28T17:58:42.925 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:58:42 smithi134 ceph-mon[35037]: pgmap v116: 1 pgs: 1 active+clean; 577 KiB data, 99 MiB used, 268 GiB / 268 GiB avail 2024-03-28T17:58:42.925 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:58:42 smithi134 ceph-mon[35037]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-03-28T17:58:42.925 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:58:42 smithi134 ceph-mon[35037]: from='client.? 172.21.15.22:0/3929096249' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "81a6c547-2a50-44c3-84d7-23ab6831790c"}]': finished 2024-03-28T17:58:42.926 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:58:42 smithi134 ceph-mon[35037]: osdmap e22: 4 total, 3 up, 4 in 2024-03-28T17:58:42.926 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:58:42 smithi134 ceph-mon[35037]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-28T17:58:42.926 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:58:42 smithi134 ceph-mon[35037]: from='client.? 172.21.15.22:0/3282985994' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-28T17:58:42.933 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:58:42 smithi079 ceph-mon[35312]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-03-28T17:58:42.933 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:58:42 smithi079 ceph-mon[35312]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-03-28T17:58:42.933 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:58:42 smithi079 ceph-mon[35312]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-03-28T17:58:42.933 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:58:42 smithi079 ceph-mon[35312]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-28T17:58:42.933 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:58:42 smithi079 ceph-mon[35312]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-28T17:58:42.933 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:58:42 smithi079 ceph-mon[35312]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-28T17:58:42.933 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:58:42 smithi079 ceph-mon[35312]: pgmap v116: 1 pgs: 1 active+clean; 577 KiB data, 99 MiB used, 268 GiB / 268 GiB avail 2024-03-28T17:58:42.933 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:58:42 smithi079 ceph-mon[35312]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-03-28T17:58:42.933 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:58:42 smithi079 ceph-mon[35312]: from='client.? 172.21.15.22:0/3929096249' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "81a6c547-2a50-44c3-84d7-23ab6831790c"}]': finished 2024-03-28T17:58:42.933 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:58:42 smithi079 ceph-mon[35312]: osdmap e22: 4 total, 3 up, 4 in 2024-03-28T17:58:42.933 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:58:42 smithi079 ceph-mon[35312]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-28T17:58:42.933 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:58:42 smithi079 ceph-mon[35312]: from='client.? 172.21.15.22:0/3282985994' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-28T17:58:43.032 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:58:42 smithi022 ceph-mon[31985]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-03-28T17:58:43.032 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:58:42 smithi022 ceph-mon[31985]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-03-28T17:58:43.033 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:58:42 smithi022 ceph-mon[31985]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-03-28T17:58:43.033 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:58:42 smithi022 ceph-mon[31985]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-28T17:58:43.033 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:58:42 smithi022 ceph-mon[31985]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-28T17:58:43.033 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:58:42 smithi022 ceph-mon[31985]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-28T17:58:43.033 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:58:42 smithi022 ceph-mon[31985]: pgmap v116: 1 pgs: 1 active+clean; 577 KiB data, 99 MiB used, 268 GiB / 268 GiB avail 2024-03-28T17:58:43.033 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:58:42 smithi022 ceph-mon[31985]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-03-28T17:58:43.033 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:58:42 smithi022 ceph-mon[31985]: from='client.? 172.21.15.22:0/3929096249' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "81a6c547-2a50-44c3-84d7-23ab6831790c"}]': finished 2024-03-28T17:58:43.033 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:58:42 smithi022 ceph-mon[31985]: osdmap e22: 4 total, 3 up, 4 in 2024-03-28T17:58:43.033 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:58:42 smithi022 ceph-mon[31985]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-28T17:58:43.033 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:58:42 smithi022 ceph-mon[31985]: from='client.? 172.21.15.22:0/3282985994' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-28T17:58:43.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:58:43 smithi134 ceph-mon[35037]: pgmap v118: 1 pgs: 1 active+clean; 577 KiB data, 98 MiB used, 268 GiB / 268 GiB avail 2024-03-28T17:58:43.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:58:43 smithi079 ceph-mon[35312]: pgmap v118: 1 pgs: 1 active+clean; 577 KiB data, 98 MiB used, 268 GiB / 268 GiB avail 2024-03-28T17:58:44.031 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:58:43 smithi022 ceph-mon[31985]: pgmap v118: 1 pgs: 1 active+clean; 577 KiB data, 98 MiB used, 268 GiB / 268 GiB avail 2024-03-28T17:58:45.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:58:45 smithi134 ceph-mon[35037]: pgmap v119: 1 pgs: 1 active+clean; 577 KiB data, 98 MiB used, 268 GiB / 268 GiB avail; 2.5 KiB/s rd, 65 KiB/s wr, 6 op/s 2024-03-28T17:58:45.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:58:45 smithi079 ceph-mon[35312]: pgmap v119: 1 pgs: 1 active+clean; 577 KiB data, 98 MiB used, 268 GiB / 268 GiB avail; 2.5 KiB/s rd, 65 KiB/s wr, 6 op/s 2024-03-28T17:58:46.031 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:58:45 smithi022 ceph-mon[31985]: pgmap v119: 1 pgs: 1 active+clean; 577 KiB data, 98 MiB used, 268 GiB / 268 GiB avail; 2.5 KiB/s rd, 65 KiB/s wr, 6 op/s 2024-03-28T17:58:47.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:58:47 smithi022 ceph-mon[31985]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 98 MiB used, 268 GiB / 268 GiB avail; 1.7 KiB/s rd, 44 KiB/s wr, 4 op/s 2024-03-28T17:58:47.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:58:47 smithi134 ceph-mon[35037]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 98 MiB used, 268 GiB / 268 GiB avail; 1.7 KiB/s rd, 44 KiB/s wr, 4 op/s 2024-03-28T17:58:47.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:58:47 smithi079 ceph-mon[35312]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 98 MiB used, 268 GiB / 268 GiB avail; 1.7 KiB/s rd, 44 KiB/s wr, 4 op/s 2024-03-28T17:58:49.781 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:58:49 smithi022 ceph-mon[31985]: pgmap v121: 1 pgs: 1 active+clean; 577 KiB data, 98 MiB used, 268 GiB / 268 GiB avail; 1.2 KiB/s rd, 33 KiB/s wr, 3 op/s 2024-03-28T17:58:49.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:58:49 smithi134 ceph-mon[35037]: pgmap v121: 1 pgs: 1 active+clean; 577 KiB data, 98 MiB used, 268 GiB / 268 GiB avail; 1.2 KiB/s rd, 33 KiB/s wr, 3 op/s 2024-03-28T17:58:49.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:58:49 smithi079 ceph-mon[35312]: pgmap v121: 1 pgs: 1 active+clean; 577 KiB data, 98 MiB used, 268 GiB / 268 GiB avail; 1.2 KiB/s rd, 33 KiB/s wr, 3 op/s 2024-03-28T17:58:51.838 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:58:51 smithi022 ceph-mon[31985]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 98 MiB used, 268 GiB / 268 GiB avail; 1023 B/s rd, 26 KiB/s wr, 2 op/s 2024-03-28T17:58:51.838 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:58:51 smithi022 ceph-mon[31985]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-03-28T17:58:51.838 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:58:51 smithi022 ceph-mon[31985]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T17:58:51.838 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:58:51 smithi022 ceph-mon[31985]: Deploying daemon osd.3 on smithi022 2024-03-28T17:58:51.925 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:58:51 smithi134 ceph-mon[35037]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 98 MiB used, 268 GiB / 268 GiB avail; 1023 B/s rd, 26 KiB/s wr, 2 op/s 2024-03-28T17:58:51.925 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:58:51 smithi134 ceph-mon[35037]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-03-28T17:58:51.925 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:58:51 smithi134 ceph-mon[35037]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T17:58:51.925 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:58:51 smithi134 ceph-mon[35037]: Deploying daemon osd.3 on smithi022 2024-03-28T17:58:51.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:58:51 smithi079 ceph-mon[35312]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 98 MiB used, 268 GiB / 268 GiB avail; 1023 B/s rd, 26 KiB/s wr, 2 op/s 2024-03-28T17:58:51.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:58:51 smithi079 ceph-mon[35312]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-03-28T17:58:51.933 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:58:51 smithi079 ceph-mon[35312]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T17:58:51.933 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:58:51 smithi079 ceph-mon[35312]: Deploying daemon osd.3 on smithi022 2024-03-28T17:58:53.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:58:53 smithi022 ceph-mon[31985]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 98 MiB used, 268 GiB / 268 GiB avail; 935 B/s rd, 24 KiB/s wr, 2 op/s 2024-03-28T17:58:53.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:58:53 smithi134 ceph-mon[35037]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 98 MiB used, 268 GiB / 268 GiB avail; 935 B/s rd, 24 KiB/s wr, 2 op/s 2024-03-28T17:58:53.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:58:53 smithi079 ceph-mon[35312]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 98 MiB used, 268 GiB / 268 GiB avail; 935 B/s rd, 24 KiB/s wr, 2 op/s 2024-03-28T17:58:55.777 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:58:55 smithi022 ceph-mon[31985]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 98 MiB used, 268 GiB / 268 GiB avail; 853 B/s rd, 22 KiB/s wr, 2 op/s 2024-03-28T17:58:55.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:58:55 smithi134 ceph-mon[35037]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 98 MiB used, 268 GiB / 268 GiB avail; 853 B/s rd, 22 KiB/s wr, 2 op/s 2024-03-28T17:58:55.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:58:55 smithi079 ceph-mon[35312]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 98 MiB used, 268 GiB / 268 GiB avail; 853 B/s rd, 22 KiB/s wr, 2 op/s 2024-03-28T17:58:57.674 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:58:57 smithi134 ceph-mon[35037]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 98 MiB used, 268 GiB / 268 GiB avail 2024-03-28T17:58:57.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:58:57 smithi079 ceph-mon[35312]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 98 MiB used, 268 GiB / 268 GiB avail 2024-03-28T17:58:58.032 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:58:57 smithi022 ceph-mon[31985]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 98 MiB used, 268 GiB / 268 GiB avail 2024-03-28T17:59:00.597 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:59:00 smithi022 ceph-mon[31985]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 98 MiB used, 268 GiB / 268 GiB avail 2024-03-28T17:59:00.598 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:59:00 smithi022 ceph-mon[31985]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T17:59:00.598 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:59:00 smithi022 ceph-mon[31985]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T17:59:00.598 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:59:00 smithi022 ceph-mon[31985]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T17:59:00.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:59:00 smithi134 ceph-mon[35037]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 98 MiB used, 268 GiB / 268 GiB avail 2024-03-28T17:59:00.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:59:00 smithi134 ceph-mon[35037]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T17:59:00.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:59:00 smithi134 ceph-mon[35037]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T17:59:00.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:59:00 smithi134 ceph-mon[35037]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T17:59:00.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:59:00 smithi079 ceph-mon[35312]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 98 MiB used, 268 GiB / 268 GiB avail 2024-03-28T17:59:00.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:59:00 smithi079 ceph-mon[35312]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T17:59:00.683 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:59:00 smithi079 ceph-mon[35312]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T17:59:00.683 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:59:00 smithi079 ceph-mon[35312]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T17:59:02.516 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:59:02 smithi022 ceph-mon[31985]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 98 MiB used, 268 GiB / 268 GiB avail 2024-03-28T17:59:02.674 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:59:02 smithi134 ceph-mon[35037]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 98 MiB used, 268 GiB / 268 GiB avail 2024-03-28T17:59:02.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:59:02 smithi079 ceph-mon[35312]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 98 MiB used, 268 GiB / 268 GiB avail 2024-03-28T17:59:04.021 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:59:03 smithi022 ceph-mon[31985]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 98 MiB used, 268 GiB / 268 GiB avail 2024-03-28T17:59:04.021 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:59:03 smithi022 ceph-mon[31985]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T17:59:04.021 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:59:03 smithi022 ceph-mon[31985]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T17:59:04.021 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:59:03 smithi022 ceph-mon[31985]: from='osd.3 [v2:172.21.15.22:6810/1702441854,v1:172.21.15.22:6811/1702441854]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-03-28T17:59:04.175 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:59:03 smithi134 ceph-mon[35037]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 98 MiB used, 268 GiB / 268 GiB avail 2024-03-28T17:59:04.175 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:59:03 smithi134 ceph-mon[35037]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T17:59:04.175 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:59:03 smithi134 ceph-mon[35037]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T17:59:04.175 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:59:03 smithi134 ceph-mon[35037]: from='osd.3 [v2:172.21.15.22:6810/1702441854,v1:172.21.15.22:6811/1702441854]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-03-28T17:59:04.182 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:59:03 smithi079 ceph-mon[35312]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 98 MiB used, 268 GiB / 268 GiB avail 2024-03-28T17:59:04.183 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:59:03 smithi079 ceph-mon[35312]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T17:59:04.183 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:59:03 smithi079 ceph-mon[35312]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T17:59:04.183 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:59:03 smithi079 ceph-mon[35312]: from='osd.3 [v2:172.21.15.22:6810/1702441854,v1:172.21.15.22:6811/1702441854]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-03-28T17:59:04.614 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T17:59:04.612+0000 7f024ea00700 1 -- 172.21.15.22:0/3196408980 <== mgr.14152 v2:172.21.15.22:6800/4201048392 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f02540625d0 con 0x7f023c0a2910 2024-03-28T17:59:04.614 INFO:teuthology.orchestra.run.smithi022.stdout:Created osd(s) 3 on host 'smithi022' 2024-03-28T17:59:04.616 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T17:59:04.615+0000 7f025c800700 1 -- 172.21.15.22:0/3196408980 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7f023c0a2910 msgr2=0x7f023c0a4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:59:04.616 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T17:59:04.615+0000 7f025c800700 1 --2- 172.21.15.22:0/3196408980 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7f023c0a2910 0x7f023c0a4dd0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f0244005950 tx=0x7f02440058e0 comp rx=0 tx=0).stop 2024-03-28T17:59:04.616 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T17:59:04.615+0000 7f025c800700 1 -- 172.21.15.22:0/3196408980 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0254102c40 msgr2=0x7f025419a5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:59:04.617 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T17:59:04.615+0000 7f025c800700 1 --2- 172.21.15.22:0/3196408980 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0254102c40 0x7f025419a5c0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f0248005910 tx=0x7f0248009580 comp rx=0 tx=0).stop 2024-03-28T17:59:04.617 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T17:59:04.616+0000 7f025c800700 1 -- 172.21.15.22:0/3196408980 shutdown_connections 2024-03-28T17:59:04.617 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T17:59:04.616+0000 7f025c800700 1 --2- 172.21.15.22:0/3196408980 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7f023c0a2910 0x7f023c0a4dd0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:04.617 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T17:59:04.616+0000 7f025c800700 1 --2- 172.21.15.22:0/3196408980 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0254107ea0 0x7f02541a1b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:04.618 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T17:59:04.616+0000 7f025c800700 1 --2- 172.21.15.22:0/3196408980 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0254102c40 0x7f025419a5c0 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:04.618 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T17:59:04.616+0000 7f025c800700 1 --2- 172.21.15.22:0/3196408980 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f0254105570 0x7f025419ab00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:04.618 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T17:59:04.616+0000 7f025c800700 1 -- 172.21.15.22:0/3196408980 >> 172.21.15.22:0/3196408980 conn(0x7f02540fc650 msgr2=0x7f02541022a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T17:59:04.618 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T17:59:04.616+0000 7f025c800700 1 -- 172.21.15.22:0/3196408980 shutdown_connections 2024-03-28T17:59:04.618 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T17:59:04.616+0000 7f025c800700 1 -- 172.21.15.22:0/3196408980 wait complete. 2024-03-28T17:59:05.032 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:59:04 smithi022 ceph-mon[31985]: from='osd.3 [v2:172.21.15.22:6810/1702441854,v1:172.21.15.22:6811/1702441854]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-03-28T17:59:05.032 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:59:04 smithi022 ceph-mon[31985]: osdmap e23: 4 total, 3 up, 4 in 2024-03-28T17:59:05.032 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:59:04 smithi022 ceph-mon[31985]: from='osd.3 [v2:172.21.15.22:6810/1702441854,v1:172.21.15.22:6811/1702441854]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi022", "root=default"]}]: dispatch 2024-03-28T17:59:05.032 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:59:04 smithi022 ceph-mon[31985]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-28T17:59:05.033 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:59:04 smithi022 ceph-mon[31985]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T17:59:05.033 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:59:04 smithi022 ceph-mon[31985]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T17:59:05.033 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:59:04 smithi022 ceph-mon[31985]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T17:59:05.033 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:59:04 smithi022 ceph-mon[31985]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T17:59:05.033 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:59:04 smithi022 ceph-mon[31985]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T17:59:05.033 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:59:04 smithi022 ceph-mon[31985]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T17:59:05.175 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:59:04 smithi134 ceph-mon[35037]: from='osd.3 [v2:172.21.15.22:6810/1702441854,v1:172.21.15.22:6811/1702441854]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-03-28T17:59:05.175 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:59:04 smithi134 ceph-mon[35037]: osdmap e23: 4 total, 3 up, 4 in 2024-03-28T17:59:05.175 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:59:04 smithi134 ceph-mon[35037]: from='osd.3 [v2:172.21.15.22:6810/1702441854,v1:172.21.15.22:6811/1702441854]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi022", "root=default"]}]: dispatch 2024-03-28T17:59:05.175 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:59:04 smithi134 ceph-mon[35037]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-28T17:59:05.175 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:59:04 smithi134 ceph-mon[35037]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T17:59:05.175 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:59:04 smithi134 ceph-mon[35037]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T17:59:05.175 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:59:04 smithi134 ceph-mon[35037]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T17:59:05.175 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:59:04 smithi134 ceph-mon[35037]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T17:59:05.175 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:59:04 smithi134 ceph-mon[35037]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T17:59:05.175 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:59:04 smithi134 ceph-mon[35037]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T17:59:05.183 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:59:04 smithi079 ceph-mon[35312]: from='osd.3 [v2:172.21.15.22:6810/1702441854,v1:172.21.15.22:6811/1702441854]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-03-28T17:59:05.183 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:59:04 smithi079 ceph-mon[35312]: osdmap e23: 4 total, 3 up, 4 in 2024-03-28T17:59:05.183 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:59:04 smithi079 ceph-mon[35312]: from='osd.3 [v2:172.21.15.22:6810/1702441854,v1:172.21.15.22:6811/1702441854]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi022", "root=default"]}]: dispatch 2024-03-28T17:59:05.183 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:59:04 smithi079 ceph-mon[35312]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-28T17:59:05.183 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:59:04 smithi079 ceph-mon[35312]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T17:59:05.183 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:59:04 smithi079 ceph-mon[35312]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T17:59:05.183 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:59:04 smithi079 ceph-mon[35312]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T17:59:05.183 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:59:04 smithi079 ceph-mon[35312]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T17:59:05.183 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:59:04 smithi079 ceph-mon[35312]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T17:59:05.183 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:59:04 smithi079 ceph-mon[35312]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T17:59:05.541 DEBUG:teuthology.orchestra.run.smithi022:osd.3> sudo journalctl -f -n 0 -u ceph-fd220748-ed2b-11ee-95d0-87774f69a715@osd.3.service 2024-03-28T17:59:05.545 INFO:tasks.cephadm:Deploying osd.4 on smithi079 with /dev/vg_nvme/lv_3... 2024-03-28T17:59:05.545 DEBUG:teuthology.orchestra.run.smithi079:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fd220748-ed2b-11ee-95d0-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-03-28T17:59:05.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:59:05 smithi079 ceph-mon[35312]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 98 MiB used, 268 GiB / 268 GiB avail 2024-03-28T17:59:05.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:59:05 smithi079 ceph-mon[35312]: from='osd.3 [v2:172.21.15.22:6810/1702441854,v1:172.21.15.22:6811/1702441854]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi022", "root=default"]}]': finished 2024-03-28T17:59:05.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:59:05 smithi079 ceph-mon[35312]: osdmap e24: 4 total, 3 up, 4 in 2024-03-28T17:59:05.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:59:05 smithi079 ceph-mon[35312]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-28T17:59:05.933 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:59:05 smithi079 ceph-mon[35312]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-28T17:59:05.985 INFO:teuthology.orchestra.run.smithi079.stderr:Inferring config /var/lib/ceph/fd220748-ed2b-11ee-95d0-87774f69a715/mon.b/config 2024-03-28T17:59:06.024 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:59:05 smithi022 ceph-mon[31985]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 98 MiB used, 268 GiB / 268 GiB avail 2024-03-28T17:59:06.024 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:59:05 smithi022 ceph-mon[31985]: from='osd.3 [v2:172.21.15.22:6810/1702441854,v1:172.21.15.22:6811/1702441854]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi022", "root=default"]}]': finished 2024-03-28T17:59:06.024 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:59:05 smithi022 ceph-mon[31985]: osdmap e24: 4 total, 3 up, 4 in 2024-03-28T17:59:06.024 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:59:05 smithi022 ceph-mon[31985]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-28T17:59:06.024 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:59:05 smithi022 ceph-mon[31985]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-28T17:59:06.025 INFO:journalctl@ceph.osd.3.smithi022.stdout:Mar 28 17:59:05 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-osd-3[53646]: 2024-03-28T17:59:05.785+0000 7f3d32200700 -1 osd.3 0 waiting for initial osdmap 2024-03-28T17:59:06.025 INFO:journalctl@ceph.osd.3.smithi022.stdout:Mar 28 17:59:05 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-osd-3[53646]: 2024-03-28T17:59:05.807+0000 7f3d28000700 -1 osd.3 25 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-28T17:59:06.174 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:59:05 smithi134 ceph-mon[35037]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 98 MiB used, 268 GiB / 268 GiB avail 2024-03-28T17:59:06.175 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:59:05 smithi134 ceph-mon[35037]: from='osd.3 [v2:172.21.15.22:6810/1702441854,v1:172.21.15.22:6811/1702441854]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi022", "root=default"]}]': finished 2024-03-28T17:59:06.175 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:59:05 smithi134 ceph-mon[35037]: osdmap e24: 4 total, 3 up, 4 in 2024-03-28T17:59:06.175 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:59:05 smithi134 ceph-mon[35037]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-28T17:59:06.175 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:59:05 smithi134 ceph-mon[35037]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-28T17:59:06.990 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:59:06 smithi022 ceph-mon[31985]: purged_snaps scrub starts 2024-03-28T17:59:06.990 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:59:06 smithi022 ceph-mon[31985]: purged_snaps scrub ok 2024-03-28T17:59:06.990 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:59:06 smithi022 ceph-mon[31985]: osdmap e25: 4 total, 3 up, 4 in 2024-03-28T17:59:06.990 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:59:06 smithi022 ceph-mon[31985]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-28T17:59:06.990 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:59:06 smithi022 ceph-mon[31985]: from='osd.3 [v2:172.21.15.22:6810/1702441854,v1:172.21.15.22:6811/1702441854]' entity='osd.3' 2024-03-28T17:59:06.990 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:59:06 smithi022 ceph-mon[31985]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-28T17:59:07.159 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:59:06 smithi079 ceph-mon[35312]: purged_snaps scrub starts 2024-03-28T17:59:07.159 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:59:06 smithi079 ceph-mon[35312]: purged_snaps scrub ok 2024-03-28T17:59:07.159 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:59:06 smithi079 ceph-mon[35312]: osdmap e25: 4 total, 3 up, 4 in 2024-03-28T17:59:07.159 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:59:06 smithi079 ceph-mon[35312]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-28T17:59:07.159 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:59:06 smithi079 ceph-mon[35312]: from='osd.3 [v2:172.21.15.22:6810/1702441854,v1:172.21.15.22:6811/1702441854]' entity='osd.3' 2024-03-28T17:59:07.159 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:59:06 smithi079 ceph-mon[35312]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-28T17:59:07.174 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:59:06 smithi134 ceph-mon[35037]: purged_snaps scrub starts 2024-03-28T17:59:07.174 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:59:06 smithi134 ceph-mon[35037]: purged_snaps scrub ok 2024-03-28T17:59:07.174 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:59:06 smithi134 ceph-mon[35037]: osdmap e25: 4 total, 3 up, 4 in 2024-03-28T17:59:07.175 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:59:06 smithi134 ceph-mon[35037]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-28T17:59:07.175 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:59:06 smithi134 ceph-mon[35037]: from='osd.3 [v2:172.21.15.22:6810/1702441854,v1:172.21.15.22:6811/1702441854]' entity='osd.3' 2024-03-28T17:59:07.175 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:59:06 smithi134 ceph-mon[35037]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-28T17:59:07.933 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:59:07 smithi079 ceph-mon[35312]: pgmap v133: 1 pgs: 1 active+clean; 577 KiB data, 98 MiB used, 268 GiB / 268 GiB avail 2024-03-28T17:59:07.933 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:59:07 smithi079 ceph-mon[35312]: osd.3 [v2:172.21.15.22:6810/1702441854,v1:172.21.15.22:6811/1702441854] boot 2024-03-28T17:59:07.933 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:59:07 smithi079 ceph-mon[35312]: osdmap e26: 4 total, 4 up, 4 in 2024-03-28T17:59:07.933 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:59:07 smithi079 ceph-mon[35312]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-28T17:59:07.934 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:59:07 smithi079 ceph-mon[35312]: osdmap e27: 4 total, 4 up, 4 in 2024-03-28T17:59:08.033 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:59:07 smithi022 ceph-mon[31985]: pgmap v133: 1 pgs: 1 active+clean; 577 KiB data, 98 MiB used, 268 GiB / 268 GiB avail 2024-03-28T17:59:08.034 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:59:07 smithi022 ceph-mon[31985]: osd.3 [v2:172.21.15.22:6810/1702441854,v1:172.21.15.22:6811/1702441854] boot 2024-03-28T17:59:08.034 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:59:07 smithi022 ceph-mon[31985]: osdmap e26: 4 total, 4 up, 4 in 2024-03-28T17:59:08.034 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:59:07 smithi022 ceph-mon[31985]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-28T17:59:08.034 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:59:07 smithi022 ceph-mon[31985]: osdmap e27: 4 total, 4 up, 4 in 2024-03-28T17:59:08.174 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:59:07 smithi134 ceph-mon[35037]: pgmap v133: 1 pgs: 1 active+clean; 577 KiB data, 98 MiB used, 268 GiB / 268 GiB avail 2024-03-28T17:59:08.175 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:59:07 smithi134 ceph-mon[35037]: osd.3 [v2:172.21.15.22:6810/1702441854,v1:172.21.15.22:6811/1702441854] boot 2024-03-28T17:59:08.175 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:59:07 smithi134 ceph-mon[35037]: osdmap e26: 4 total, 4 up, 4 in 2024-03-28T17:59:08.175 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:59:07 smithi134 ceph-mon[35037]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-28T17:59:08.175 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:59:07 smithi134 ceph-mon[35037]: osdmap e27: 4 total, 4 up, 4 in 2024-03-28T17:59:08.827 INFO:teuthology.orchestra.run.smithi079.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-03-28T17:59:08.827 INFO:teuthology.orchestra.run.smithi079.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-03-28T17:59:08.827 INFO:teuthology.orchestra.run.smithi079.stderr: stderr: 10+0 records in 2024-03-28T17:59:08.827 INFO:teuthology.orchestra.run.smithi079.stderr:10+0 records out 2024-03-28T17:59:08.827 INFO:teuthology.orchestra.run.smithi079.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0385515 s, 272 MB/s 2024-03-28T17:59:08.827 INFO:teuthology.orchestra.run.smithi079.stderr:--> Zapping successful for: 2024-03-28T17:59:09.521 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:59:09 smithi079 ceph-mon[35312]: osdmap e28: 4 total, 4 up, 4 in 2024-03-28T17:59:09.590 DEBUG:teuthology.orchestra.run.smithi079:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fd220748-ed2b-11ee-95d0-87774f69a715 -- ceph orch daemon add osd smithi079:vg_nvme/lv_3 2024-03-28T17:59:09.674 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:59:09 smithi134 ceph-mon[35037]: osdmap e28: 4 total, 4 up, 4 in 2024-03-28T17:59:09.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:59:09 smithi022 ceph-mon[31985]: osdmap e28: 4 total, 4 up, 4 in 2024-03-28T17:59:10.009 INFO:teuthology.orchestra.run.smithi079.stderr:Inferring config /var/lib/ceph/fd220748-ed2b-11ee-95d0-87774f69a715/mon.b/config 2024-03-28T17:59:10.433 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:59:10 smithi079 ceph-mon[35312]: pgmap v137: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 357 GiB / 358 GiB avail 2024-03-28T17:59:10.433 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:59:10 smithi079 ceph-mon[35312]: osdmap e29: 4 total, 4 up, 4 in 2024-03-28T17:59:10.674 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:59:10 smithi134 ceph-mon[35037]: pgmap v137: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 357 GiB / 358 GiB avail 2024-03-28T17:59:10.674 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:59:10 smithi134 ceph-mon[35037]: osdmap e29: 4 total, 4 up, 4 in 2024-03-28T17:59:10.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:59:10 smithi022 ceph-mon[31985]: pgmap v137: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 357 GiB / 358 GiB avail 2024-03-28T17:59:10.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:59:10 smithi022 ceph-mon[31985]: osdmap e29: 4 total, 4 up, 4 in 2024-03-28T17:59:10.782 INFO:journalctl@ceph.mgr.x.smithi022.stdout:Mar 28 17:59:10 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mgr-x[32211]: 2024-03-28T17:59:10.470+0000 7fd828000700 -1 calc_pg_upmaps abort due to max <= 0 2024-03-28T17:59:11.610 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:59:11 smithi079 ceph-mon[35312]: osdmap e30: 4 total, 4 up, 4 in 2024-03-28T17:59:11.610 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:59:11 smithi079 ceph-mon[35312]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T17:59:11.610 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:59:11 smithi079 ceph-mon[35312]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T17:59:11.610 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:59:11 smithi079 ceph-mon[35312]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-28T17:59:11.610 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:59:11 smithi079 ceph-mon[35312]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-28T17:59:11.610 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:59:11 smithi079 ceph-mon[35312]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T17:59:11.610 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:59:11 smithi079 ceph-mon[35312]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T17:59:11.610 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:59:11 smithi079 ceph-mon[35312]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T17:59:11.610 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:59:11 smithi079 ceph-mon[35312]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T17:59:11.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:59:11 smithi134 ceph-mon[35037]: osdmap e30: 4 total, 4 up, 4 in 2024-03-28T17:59:11.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:59:11 smithi134 ceph-mon[35037]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T17:59:11.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:59:11 smithi134 ceph-mon[35037]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T17:59:11.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:59:11 smithi134 ceph-mon[35037]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-28T17:59:11.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:59:11 smithi134 ceph-mon[35037]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-28T17:59:11.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:59:11 smithi134 ceph-mon[35037]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T17:59:11.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:59:11 smithi134 ceph-mon[35037]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T17:59:11.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:59:11 smithi134 ceph-mon[35037]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T17:59:11.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:59:11 smithi134 ceph-mon[35037]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T17:59:11.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:59:11 smithi022 ceph-mon[31985]: osdmap e30: 4 total, 4 up, 4 in 2024-03-28T17:59:11.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:59:11 smithi022 ceph-mon[31985]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T17:59:11.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:59:11 smithi022 ceph-mon[31985]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T17:59:11.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:59:11 smithi022 ceph-mon[31985]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-28T17:59:11.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:59:11 smithi022 ceph-mon[31985]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-28T17:59:11.783 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:59:11 smithi022 ceph-mon[31985]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T17:59:11.783 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:59:11 smithi022 ceph-mon[31985]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T17:59:11.783 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:59:11 smithi022 ceph-mon[31985]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T17:59:11.783 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:59:11 smithi022 ceph-mon[31985]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T17:59:12.089 INFO:teuthology.orchestra.run.smithi079.stderr:2024-03-28T17:59:12.089+0000 7fe48e000700 1 -- 172.21.15.79:0/1899906212 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fe488104410 msgr2=0x7fe488104830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:59:12.089 INFO:teuthology.orchestra.run.smithi079.stderr:2024-03-28T17:59:12.089+0000 7fe48e000700 1 --2- 172.21.15.79:0/1899906212 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fe488104410 0x7fe488104830 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7fe4780099c0 tx=0x7fe478009cd0 comp rx=0 tx=0).stop 2024-03-28T17:59:12.089 INFO:teuthology.orchestra.run.smithi079.stderr:2024-03-28T17:59:12.089+0000 7fe48e000700 1 -- 172.21.15.79:0/1899906212 shutdown_connections 2024-03-28T17:59:12.089 INFO:teuthology.orchestra.run.smithi079.stderr:2024-03-28T17:59:12.089+0000 7fe48e000700 1 --2- 172.21.15.79:0/1899906212 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe488105fb0 0x7fe48810c860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:12.089 INFO:teuthology.orchestra.run.smithi079.stderr:2024-03-28T17:59:12.089+0000 7fe48e000700 1 --2- 172.21.15.79:0/1899906212 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fe488104410 0x7fe488104830 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:12.089 INFO:teuthology.orchestra.run.smithi079.stderr:2024-03-28T17:59:12.089+0000 7fe48e000700 1 --2- 172.21.15.79:0/1899906212 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe488105610 0x7fe488105a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:12.089 INFO:teuthology.orchestra.run.smithi079.stderr:2024-03-28T17:59:12.089+0000 7fe48e000700 1 -- 172.21.15.79:0/1899906212 >> 172.21.15.79:0/1899906212 conn(0x7fe4880ff9b0 msgr2=0x7fe488101df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T17:59:12.089 INFO:teuthology.orchestra.run.smithi079.stderr:2024-03-28T17:59:12.089+0000 7fe48e000700 1 -- 172.21.15.79:0/1899906212 shutdown_connections 2024-03-28T17:59:12.089 INFO:teuthology.orchestra.run.smithi079.stderr:2024-03-28T17:59:12.089+0000 7fe48e000700 1 -- 172.21.15.79:0/1899906212 wait complete. 2024-03-28T17:59:12.090 INFO:teuthology.orchestra.run.smithi079.stderr:2024-03-28T17:59:12.090+0000 7fe48e000700 1 Processor -- start 2024-03-28T17:59:12.090 INFO:teuthology.orchestra.run.smithi079.stderr:2024-03-28T17:59:12.090+0000 7fe48e000700 1 -- start start 2024-03-28T17:59:12.090 INFO:teuthology.orchestra.run.smithi079.stderr:2024-03-28T17:59:12.091+0000 7fe48e000700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe488104410 0x7fe48819a600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:59:12.090 INFO:teuthology.orchestra.run.smithi079.stderr:2024-03-28T17:59:12.091+0000 7fe48e000700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fe488105610 0x7fe48819ab40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:59:12.090 INFO:teuthology.orchestra.run.smithi079.stderr:2024-03-28T17:59:12.091+0000 7fe48e000700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe488105fb0 0x7fe4881a1bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:59:12.091 INFO:teuthology.orchestra.run.smithi079.stderr:2024-03-28T17:59:12.091+0000 7fe48e000700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe48810ffb0 con 0x7fe488104410 2024-03-28T17:59:12.091 INFO:teuthology.orchestra.run.smithi079.stderr:2024-03-28T17:59:12.091+0000 7fe48e000700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe48810fe30 con 0x7fe488105610 2024-03-28T17:59:12.091 INFO:teuthology.orchestra.run.smithi079.stderr:2024-03-28T17:59:12.091+0000 7fe48e000700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe488110130 con 0x7fe488105fb0 2024-03-28T17:59:12.092 INFO:teuthology.orchestra.run.smithi079.stderr:2024-03-28T17:59:12.091+0000 7fe487400700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe488104410 0x7fe48819a600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:59:12.092 INFO:teuthology.orchestra.run.smithi079.stderr:2024-03-28T17:59:12.091+0000 7fe487400700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe488104410 0x7fe48819a600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.79:59844/0 (socket says 172.21.15.79:59844) 2024-03-28T17:59:12.092 INFO:teuthology.orchestra.run.smithi079.stderr:2024-03-28T17:59:12.091+0000 7fe487400700 1 -- 172.21.15.79:0/1249377503 learned_addr learned my addr 172.21.15.79:0/1249377503 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-03-28T17:59:12.092 INFO:teuthology.orchestra.run.smithi079.stderr:2024-03-28T17:59:12.091+0000 7fe486a00700 1 --2- 172.21.15.79:0/1249377503 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fe488105610 0x7fe48819ab40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:59:12.092 INFO:teuthology.orchestra.run.smithi079.stderr:2024-03-28T17:59:12.091+0000 7fe487e00700 1 --2- 172.21.15.79:0/1249377503 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe488105fb0 0x7fe4881a1bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:59:12.092 INFO:teuthology.orchestra.run.smithi079.stderr:2024-03-28T17:59:12.092+0000 7fe486a00700 1 -- 172.21.15.79:0/1249377503 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe488105fb0 msgr2=0x7fe4881a1bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:59:12.092 INFO:teuthology.orchestra.run.smithi079.stderr:2024-03-28T17:59:12.092+0000 7fe486a00700 1 --2- 172.21.15.79:0/1249377503 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe488105fb0 0x7fe4881a1bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:12.092 INFO:teuthology.orchestra.run.smithi079.stderr:2024-03-28T17:59:12.092+0000 7fe486a00700 1 -- 172.21.15.79:0/1249377503 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe488104410 msgr2=0x7fe48819a600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:59:12.093 INFO:teuthology.orchestra.run.smithi079.stderr:2024-03-28T17:59:12.092+0000 7fe486a00700 1 --2- 172.21.15.79:0/1249377503 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe488104410 0x7fe48819a600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:12.093 INFO:teuthology.orchestra.run.smithi079.stderr:2024-03-28T17:59:12.092+0000 7fe486a00700 1 -- 172.21.15.79:0/1249377503 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe4881a2260 con 0x7fe488105610 2024-03-28T17:59:12.093 INFO:teuthology.orchestra.run.smithi079.stderr:2024-03-28T17:59:12.092+0000 7fe486a00700 1 --2- 172.21.15.79:0/1249377503 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fe488105610 0x7fe48819ab40 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fe47400d750 tx=0x7fe47400da60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T17:59:12.093 INFO:teuthology.orchestra.run.smithi079.stderr:2024-03-28T17:59:12.092+0000 7fe47fe00700 1 -- 172.21.15.79:0/1249377503 <== mon.2 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe474016070 con 0x7fe488105610 2024-03-28T17:59:12.093 INFO:teuthology.orchestra.run.smithi079.stderr:2024-03-28T17:59:12.092+0000 7fe47fe00700 1 -- 172.21.15.79:0/1249377503 <== mon.2 v2:172.21.15.79:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7fe474005020 con 0x7fe488105610 2024-03-28T17:59:12.093 INFO:teuthology.orchestra.run.smithi079.stderr:2024-03-28T17:59:12.092+0000 7fe48e000700 1 -- 172.21.15.79:0/1249377503 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe4881a2550 con 0x7fe488105610 2024-03-28T17:59:12.093 INFO:teuthology.orchestra.run.smithi079.stderr:2024-03-28T17:59:12.092+0000 7fe47fe00700 1 -- 172.21.15.79:0/1249377503 <== mon.2 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe47400fb50 con 0x7fe488105610 2024-03-28T17:59:12.093 INFO:teuthology.orchestra.run.smithi079.stderr:2024-03-28T17:59:12.093+0000 7fe48e000700 1 -- 172.21.15.79:0/1249377503 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe4881a2a90 con 0x7fe488105610 2024-03-28T17:59:12.093 INFO:teuthology.orchestra.run.smithi079.stderr:2024-03-28T17:59:12.094+0000 7fe48e000700 1 -- 172.21.15.79:0/1249377503 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe488068e50 con 0x7fe488105610 2024-03-28T17:59:12.095 INFO:teuthology.orchestra.run.smithi079.stderr:2024-03-28T17:59:12.096+0000 7fe47fe00700 1 -- 172.21.15.79:0/1249377503 <== mon.2 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 16) v1 ==== 137290+0+0 (secure 0 0 0) 0x7fe47400fcf0 con 0x7fe488105610 2024-03-28T17:59:12.096 INFO:teuthology.orchestra.run.smithi079.stderr:2024-03-28T17:59:12.097+0000 7fe47fe00700 1 --2- 172.21.15.79:0/1249377503 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7fe46c0a2910 0x7fe46c0a4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:59:12.096 INFO:teuthology.orchestra.run.smithi079.stderr:2024-03-28T17:59:12.097+0000 7fe47fe00700 1 -- 172.21.15.79:0/1249377503 <== mon.2 v2:172.21.15.79:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 3669+0+0 (secure 0 0 0) 0x7fe4740cd690 con 0x7fe488105610 2024-03-28T17:59:12.097 INFO:teuthology.orchestra.run.smithi079.stderr:2024-03-28T17:59:12.098+0000 7fe487400700 1 --2- 172.21.15.79:0/1249377503 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7fe46c0a2910 0x7fe46c0a4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:59:12.098 INFO:teuthology.orchestra.run.smithi079.stderr:2024-03-28T17:59:12.099+0000 7fe487400700 1 --2- 172.21.15.79:0/1249377503 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7fe46c0a2910 0x7fe46c0a4dd0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fe478000c00 tx=0x7fe478005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T17:59:12.100 INFO:teuthology.orchestra.run.smithi079.stderr:2024-03-28T17:59:12.101+0000 7fe47fe00700 1 -- 172.21.15.79:0/1249377503 <== mon.2 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fe47409b8b0 con 0x7fe488105610 2024-03-28T17:59:12.334 INFO:teuthology.orchestra.run.smithi079.stderr:2024-03-28T17:59:12.334+0000 7fe48e000700 1 -- 172.21.15.79:0/1249377503 --> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi079:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) v1 -- 0x7fe488062690 con 0x7fe46c0a2910 2024-03-28T17:59:12.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:59:12 smithi134 ceph-mon[35037]: pgmap v140: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 132 MiB used, 357 GiB / 358 GiB avail; 155 KiB/s, 0 objects/s recovering 2024-03-28T17:59:12.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:59:12 smithi134 ceph-mon[35037]: Detected new or changed devices on smithi022 2024-03-28T17:59:12.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:59:12 smithi134 ceph-mon[35037]: Adjusting osd_memory_target on smithi022 to 8547M 2024-03-28T17:59:12.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:59:12 smithi134 ceph-mon[35037]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-28T17:59:12.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:59:12 smithi134 ceph-mon[35037]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-28T17:59:12.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:59:12 smithi134 ceph-mon[35037]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T17:59:12.679 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:59:12 smithi022 ceph-mon[31985]: pgmap v140: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 132 MiB used, 357 GiB / 358 GiB avail; 155 KiB/s, 0 objects/s recovering 2024-03-28T17:59:12.679 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:59:12 smithi022 ceph-mon[31985]: Detected new or changed devices on smithi022 2024-03-28T17:59:12.679 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:59:12 smithi022 ceph-mon[31985]: Adjusting osd_memory_target on smithi022 to 8547M 2024-03-28T17:59:12.679 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:59:12 smithi022 ceph-mon[31985]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-28T17:59:12.679 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:59:12 smithi022 ceph-mon[31985]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-28T17:59:12.679 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:59:12 smithi022 ceph-mon[31985]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T17:59:12.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:59:12 smithi079 ceph-mon[35312]: pgmap v140: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 132 MiB used, 357 GiB / 358 GiB avail; 155 KiB/s, 0 objects/s recovering 2024-03-28T17:59:12.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:59:12 smithi079 ceph-mon[35312]: Detected new or changed devices on smithi022 2024-03-28T17:59:12.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:59:12 smithi079 ceph-mon[35312]: Adjusting osd_memory_target on smithi022 to 8547M 2024-03-28T17:59:12.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:59:12 smithi079 ceph-mon[35312]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-28T17:59:12.683 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:59:12 smithi079 ceph-mon[35312]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-28T17:59:12.683 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:59:12 smithi079 ceph-mon[35312]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T17:59:13.592 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:59:13 smithi079 ceph-mon[35312]: from='client.34220 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi079:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-03-28T17:59:13.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:59:13 smithi022 ceph-mon[31985]: from='client.34220 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi079:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-03-28T17:59:13.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:59:13 smithi134 ceph-mon[35037]: from='client.34220 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi079:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-03-28T17:59:14.637 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:59:14 smithi079 ceph-mon[35312]: pgmap v141: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 132 MiB used, 357 GiB / 358 GiB avail; 112 KiB/s, 0 objects/s recovering 2024-03-28T17:59:14.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:59:14 smithi022 ceph-mon[31985]: pgmap v141: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 132 MiB used, 357 GiB / 358 GiB avail; 112 KiB/s, 0 objects/s recovering 2024-03-28T17:59:14.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:59:14 smithi134 ceph-mon[35037]: pgmap v141: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 132 MiB used, 357 GiB / 358 GiB avail; 112 KiB/s, 0 objects/s recovering 2024-03-28T17:59:16.683 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:59:16 smithi079 ceph-mon[35312]: pgmap v142: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 357 GiB / 358 GiB avail; 94 KiB/s, 0 objects/s recovering 2024-03-28T17:59:16.683 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:59:16 smithi079 ceph-mon[35312]: from='client.? 172.21.15.79:0/1872350537' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d23a254e-7182-4471-8bca-7eb684ec618e"}]: dispatch 2024-03-28T17:59:16.683 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:59:16 smithi079 ceph-mon[35312]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d23a254e-7182-4471-8bca-7eb684ec618e"}]: dispatch 2024-03-28T17:59:16.683 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:59:16 smithi079 ceph-mon[35312]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "d23a254e-7182-4471-8bca-7eb684ec618e"}]': finished 2024-03-28T17:59:16.683 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:59:16 smithi079 ceph-mon[35312]: osdmap e31: 5 total, 4 up, 5 in 2024-03-28T17:59:16.684 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:59:16 smithi079 ceph-mon[35312]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-28T17:59:16.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:59:16 smithi022 ceph-mon[31985]: pgmap v142: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 357 GiB / 358 GiB avail; 94 KiB/s, 0 objects/s recovering 2024-03-28T17:59:16.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:59:16 smithi022 ceph-mon[31985]: from='client.? 172.21.15.79:0/1872350537' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d23a254e-7182-4471-8bca-7eb684ec618e"}]: dispatch 2024-03-28T17:59:16.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:59:16 smithi022 ceph-mon[31985]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d23a254e-7182-4471-8bca-7eb684ec618e"}]: dispatch 2024-03-28T17:59:16.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:59:16 smithi022 ceph-mon[31985]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "d23a254e-7182-4471-8bca-7eb684ec618e"}]': finished 2024-03-28T17:59:16.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:59:16 smithi022 ceph-mon[31985]: osdmap e31: 5 total, 4 up, 5 in 2024-03-28T17:59:16.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:59:16 smithi022 ceph-mon[31985]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-28T17:59:16.925 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:59:16 smithi134 ceph-mon[35037]: pgmap v142: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 357 GiB / 358 GiB avail; 94 KiB/s, 0 objects/s recovering 2024-03-28T17:59:16.925 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:59:16 smithi134 ceph-mon[35037]: from='client.? 172.21.15.79:0/1872350537' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d23a254e-7182-4471-8bca-7eb684ec618e"}]: dispatch 2024-03-28T17:59:16.925 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:59:16 smithi134 ceph-mon[35037]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d23a254e-7182-4471-8bca-7eb684ec618e"}]: dispatch 2024-03-28T17:59:16.925 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:59:16 smithi134 ceph-mon[35037]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "d23a254e-7182-4471-8bca-7eb684ec618e"}]': finished 2024-03-28T17:59:16.925 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:59:16 smithi134 ceph-mon[35037]: osdmap e31: 5 total, 4 up, 5 in 2024-03-28T17:59:16.925 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:59:16 smithi134 ceph-mon[35037]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-28T17:59:17.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:59:17 smithi022 ceph-mon[31985]: from='client.? 172.21.15.79:0/3686969905' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-28T17:59:17.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:59:17 smithi134 ceph-mon[35037]: from='client.? 172.21.15.79:0/3686969905' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-28T17:59:17.933 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:59:17 smithi079 ceph-mon[35312]: from='client.? 172.21.15.79:0/3686969905' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-28T17:59:18.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:59:18 smithi022 ceph-mon[31985]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 357 GiB / 358 GiB avail 2024-03-28T17:59:18.926 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:59:18 smithi134 ceph-mon[35037]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 357 GiB / 358 GiB avail 2024-03-28T17:59:18.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:59:18 smithi079 ceph-mon[35312]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 357 GiB / 358 GiB avail 2024-03-28T17:59:19.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:59:19 smithi022 ceph-mon[31985]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 357 GiB / 358 GiB avail 2024-03-28T17:59:19.875 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:59:19 smithi079 ceph-mon[35312]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 357 GiB / 358 GiB avail 2024-03-28T17:59:19.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:59:19 smithi134 ceph-mon[35037]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 357 GiB / 358 GiB avail 2024-03-28T17:59:21.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:59:21 smithi079 ceph-mon[35312]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 357 GiB / 358 GiB avail 2024-03-28T17:59:21.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:59:21 smithi022 ceph-mon[31985]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 357 GiB / 358 GiB avail 2024-03-28T17:59:21.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:59:21 smithi134 ceph-mon[35037]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 357 GiB / 358 GiB avail 2024-03-28T17:59:23.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:59:23 smithi134 ceph-mon[35037]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 357 GiB / 358 GiB avail 2024-03-28T17:59:23.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:59:23 smithi079 ceph-mon[35312]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 357 GiB / 358 GiB avail 2024-03-28T17:59:24.032 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:59:23 smithi022 ceph-mon[31985]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 357 GiB / 358 GiB avail 2024-03-28T17:59:25.683 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:59:25 smithi079 ceph-mon[35312]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 357 GiB / 358 GiB avail 2024-03-28T17:59:25.683 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:59:25 smithi079 ceph-mon[35312]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-03-28T17:59:25.683 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:59:25 smithi079 ceph-mon[35312]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T17:59:25.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:59:25 smithi134 ceph-mon[35037]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 357 GiB / 358 GiB avail 2024-03-28T17:59:25.925 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:59:25 smithi134 ceph-mon[35037]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-03-28T17:59:25.925 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:59:25 smithi134 ceph-mon[35037]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T17:59:26.032 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:59:25 smithi022 ceph-mon[31985]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 357 GiB / 358 GiB avail 2024-03-28T17:59:26.032 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:59:25 smithi022 ceph-mon[31985]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-03-28T17:59:26.032 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:59:25 smithi022 ceph-mon[31985]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T17:59:26.769 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:59:26 smithi079 ceph-mon[35312]: Deploying daemon osd.4 on smithi079 2024-03-28T17:59:26.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:59:26 smithi134 ceph-mon[35037]: Deploying daemon osd.4 on smithi079 2024-03-28T17:59:27.032 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:59:26 smithi022 ceph-mon[31985]: Deploying daemon osd.4 on smithi079 2024-03-28T17:59:27.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:59:27 smithi134 ceph-mon[35037]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 357 GiB / 358 GiB avail 2024-03-28T17:59:27.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:59:27 smithi079 ceph-mon[35312]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 357 GiB / 358 GiB avail 2024-03-28T17:59:28.031 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:59:27 smithi022 ceph-mon[31985]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 357 GiB / 358 GiB avail 2024-03-28T17:59:29.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:59:29 smithi134 ceph-mon[35037]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 357 GiB / 358 GiB avail 2024-03-28T17:59:29.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:59:29 smithi079 ceph-mon[35312]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 357 GiB / 358 GiB avail 2024-03-28T17:59:30.032 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:59:29 smithi022 ceph-mon[31985]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 357 GiB / 358 GiB avail 2024-03-28T17:59:31.883 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:59:31 smithi079 ceph-mon[35312]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 357 GiB / 358 GiB avail 2024-03-28T17:59:31.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:59:31 smithi134 ceph-mon[35037]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 357 GiB / 358 GiB avail 2024-03-28T17:59:32.031 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:59:31 smithi022 ceph-mon[31985]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 357 GiB / 358 GiB avail 2024-03-28T17:59:33.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:59:33 smithi134 ceph-mon[35037]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 357 GiB / 358 GiB avail 2024-03-28T17:59:33.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:59:33 smithi134 ceph-mon[35037]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T17:59:33.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:59:33 smithi134 ceph-mon[35037]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T17:59:33.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:59:33 smithi134 ceph-mon[35037]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T17:59:33.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:59:33 smithi079 ceph-mon[35312]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 357 GiB / 358 GiB avail 2024-03-28T17:59:33.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:59:33 smithi079 ceph-mon[35312]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T17:59:33.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:59:33 smithi079 ceph-mon[35312]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T17:59:33.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:59:33 smithi079 ceph-mon[35312]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T17:59:34.032 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:59:33 smithi022 ceph-mon[31985]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 357 GiB / 358 GiB avail 2024-03-28T17:59:34.032 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:59:33 smithi022 ceph-mon[31985]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T17:59:34.032 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:59:33 smithi022 ceph-mon[31985]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T17:59:34.032 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:59:33 smithi022 ceph-mon[31985]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T17:59:35.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:59:35 smithi134 ceph-mon[35037]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 357 GiB / 358 GiB avail 2024-03-28T17:59:35.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:59:35 smithi079 ceph-mon[35312]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 357 GiB / 358 GiB avail 2024-03-28T17:59:36.031 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:59:35 smithi022 ceph-mon[31985]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 357 GiB / 358 GiB avail 2024-03-28T17:59:37.903 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:59:37 smithi079 ceph-mon[35312]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 357 GiB / 358 GiB avail 2024-03-28T17:59:37.904 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:59:37 smithi079 ceph-mon[35312]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T17:59:37.904 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:59:37 smithi079 ceph-mon[35312]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T17:59:37.904 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:59:37 smithi079 ceph-mon[35312]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T17:59:37.904 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:59:37 smithi079 ceph-mon[35312]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T17:59:37.904 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:59:37 smithi079 ceph-mon[35312]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T17:59:37.904 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:59:37 smithi079 ceph-mon[35312]: from='osd.4 [v2:172.21.15.79:6808/630132437,v1:172.21.15.79:6809/630132437]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-03-28T17:59:37.904 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:59:37 smithi079 ceph-mon[35312]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-03-28T17:59:38.000 INFO:teuthology.orchestra.run.smithi079.stdout:Created osd(s) 4 on host 'smithi079' 2024-03-28T17:59:38.000 INFO:teuthology.orchestra.run.smithi079.stderr:2024-03-28T17:59:37.999+0000 7fe47fe00700 1 -- 172.21.15.79:0/1249377503 <== mgr.14152 v2:172.21.15.22:6800/4201048392 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7fe488062690 con 0x7fe46c0a2910 2024-03-28T17:59:38.003 INFO:teuthology.orchestra.run.smithi079.stderr:2024-03-28T17:59:38.002+0000 7fe48e000700 1 -- 172.21.15.79:0/1249377503 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7fe46c0a2910 msgr2=0x7fe46c0a4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:59:38.003 INFO:teuthology.orchestra.run.smithi079.stderr:2024-03-28T17:59:38.002+0000 7fe48e000700 1 --2- 172.21.15.79:0/1249377503 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7fe46c0a2910 0x7fe46c0a4dd0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fe478000c00 tx=0x7fe478005fb0 comp rx=0 tx=0).stop 2024-03-28T17:59:38.003 INFO:teuthology.orchestra.run.smithi079.stderr:2024-03-28T17:59:38.003+0000 7fe48e000700 1 -- 172.21.15.79:0/1249377503 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fe488105610 msgr2=0x7fe48819ab40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:59:38.003 INFO:teuthology.orchestra.run.smithi079.stderr:2024-03-28T17:59:38.003+0000 7fe48e000700 1 --2- 172.21.15.79:0/1249377503 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fe488105610 0x7fe48819ab40 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fe47400d750 tx=0x7fe47400da60 comp rx=0 tx=0).stop 2024-03-28T17:59:38.004 INFO:teuthology.orchestra.run.smithi079.stderr:2024-03-28T17:59:38.003+0000 7fe48e000700 1 -- 172.21.15.79:0/1249377503 shutdown_connections 2024-03-28T17:59:38.004 INFO:teuthology.orchestra.run.smithi079.stderr:2024-03-28T17:59:38.003+0000 7fe48e000700 1 --2- 172.21.15.79:0/1249377503 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7fe46c0a2910 0x7fe46c0a4dd0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:38.004 INFO:teuthology.orchestra.run.smithi079.stderr:2024-03-28T17:59:38.003+0000 7fe48e000700 1 --2- 172.21.15.79:0/1249377503 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe488105fb0 0x7fe4881a1bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:38.004 INFO:teuthology.orchestra.run.smithi079.stderr:2024-03-28T17:59:38.003+0000 7fe48e000700 1 --2- 172.21.15.79:0/1249377503 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe488104410 0x7fe48819a600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:38.004 INFO:teuthology.orchestra.run.smithi079.stderr:2024-03-28T17:59:38.003+0000 7fe48e000700 1 --2- 172.21.15.79:0/1249377503 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fe488105610 0x7fe48819ab40 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:38.004 INFO:teuthology.orchestra.run.smithi079.stderr:2024-03-28T17:59:38.003+0000 7fe48e000700 1 -- 172.21.15.79:0/1249377503 >> 172.21.15.79:0/1249377503 conn(0x7fe4880ff9b0 msgr2=0x7fe488108930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T17:59:38.004 INFO:teuthology.orchestra.run.smithi079.stderr:2024-03-28T17:59:38.003+0000 7fe48e000700 1 -- 172.21.15.79:0/1249377503 shutdown_connections 2024-03-28T17:59:38.004 INFO:teuthology.orchestra.run.smithi079.stderr:2024-03-28T17:59:38.003+0000 7fe48e000700 1 -- 172.21.15.79:0/1249377503 wait complete. 2024-03-28T17:59:38.174 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:59:37 smithi134 ceph-mon[35037]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 357 GiB / 358 GiB avail 2024-03-28T17:59:38.175 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:59:37 smithi134 ceph-mon[35037]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T17:59:38.175 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:59:37 smithi134 ceph-mon[35037]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T17:59:38.175 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:59:37 smithi134 ceph-mon[35037]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T17:59:38.175 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:59:37 smithi134 ceph-mon[35037]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T17:59:38.175 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:59:37 smithi134 ceph-mon[35037]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T17:59:38.175 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:59:37 smithi134 ceph-mon[35037]: from='osd.4 [v2:172.21.15.79:6808/630132437,v1:172.21.15.79:6809/630132437]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-03-28T17:59:38.175 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:59:37 smithi134 ceph-mon[35037]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-03-28T17:59:38.282 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:59:37 smithi022 ceph-mon[31985]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 357 GiB / 358 GiB avail 2024-03-28T17:59:38.282 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:59:37 smithi022 ceph-mon[31985]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T17:59:38.282 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:59:37 smithi022 ceph-mon[31985]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T17:59:38.282 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:59:37 smithi022 ceph-mon[31985]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T17:59:38.282 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:59:37 smithi022 ceph-mon[31985]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T17:59:38.282 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:59:37 smithi022 ceph-mon[31985]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T17:59:38.283 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:59:37 smithi022 ceph-mon[31985]: from='osd.4 [v2:172.21.15.79:6808/630132437,v1:172.21.15.79:6809/630132437]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-03-28T17:59:38.283 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:59:37 smithi022 ceph-mon[31985]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-03-28T17:59:38.898 DEBUG:teuthology.orchestra.run.smithi079:osd.4> sudo journalctl -f -n 0 -u ceph-fd220748-ed2b-11ee-95d0-87774f69a715@osd.4.service 2024-03-28T17:59:38.904 INFO:tasks.cephadm:Deploying osd.5 on smithi134 with /dev/vg_nvme/lv_3... 2024-03-28T17:59:38.904 DEBUG:teuthology.orchestra.run.smithi134:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fd220748-ed2b-11ee-95d0-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-03-28T17:59:39.101 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:59:38 smithi079 ceph-mon[35312]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-03-28T17:59:39.101 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:59:38 smithi079 ceph-mon[35312]: osdmap e32: 5 total, 4 up, 5 in 2024-03-28T17:59:39.101 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:59:38 smithi079 ceph-mon[35312]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-28T17:59:39.101 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:59:38 smithi079 ceph-mon[35312]: from='osd.4 [v2:172.21.15.79:6808/630132437,v1:172.21.15.79:6809/630132437]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi079", "root=default"]}]: dispatch 2024-03-28T17:59:39.101 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:59:38 smithi079 ceph-mon[35312]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi079", "root=default"]}]: dispatch 2024-03-28T17:59:39.101 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:59:38 smithi079 ceph-mon[35312]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T17:59:39.101 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:59:38 smithi079 ceph-mon[35312]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T17:59:39.101 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:59:38 smithi079 ceph-mon[35312]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T17:59:39.175 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:59:38 smithi134 ceph-mon[35037]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-03-28T17:59:39.175 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:59:38 smithi134 ceph-mon[35037]: osdmap e32: 5 total, 4 up, 5 in 2024-03-28T17:59:39.175 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:59:38 smithi134 ceph-mon[35037]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-28T17:59:39.175 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:59:38 smithi134 ceph-mon[35037]: from='osd.4 [v2:172.21.15.79:6808/630132437,v1:172.21.15.79:6809/630132437]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi079", "root=default"]}]: dispatch 2024-03-28T17:59:39.175 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:59:38 smithi134 ceph-mon[35037]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi079", "root=default"]}]: dispatch 2024-03-28T17:59:39.175 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:59:38 smithi134 ceph-mon[35037]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T17:59:39.175 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:59:38 smithi134 ceph-mon[35037]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T17:59:39.175 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:59:38 smithi134 ceph-mon[35037]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T17:59:39.282 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:59:38 smithi022 ceph-mon[31985]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-03-28T17:59:39.282 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:59:38 smithi022 ceph-mon[31985]: osdmap e32: 5 total, 4 up, 5 in 2024-03-28T17:59:39.282 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:59:38 smithi022 ceph-mon[31985]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-28T17:59:39.282 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:59:38 smithi022 ceph-mon[31985]: from='osd.4 [v2:172.21.15.79:6808/630132437,v1:172.21.15.79:6809/630132437]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi079", "root=default"]}]: dispatch 2024-03-28T17:59:39.282 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:59:38 smithi022 ceph-mon[31985]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi079", "root=default"]}]: dispatch 2024-03-28T17:59:39.282 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:59:38 smithi022 ceph-mon[31985]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T17:59:39.282 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:59:38 smithi022 ceph-mon[31985]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T17:59:39.282 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:59:38 smithi022 ceph-mon[31985]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T17:59:39.347 INFO:teuthology.orchestra.run.smithi134.stderr:Inferring config /var/lib/ceph/fd220748-ed2b-11ee-95d0-87774f69a715/mon.c/config 2024-03-28T17:59:40.174 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:59:39 smithi134 ceph-mon[35037]: pgmap v156: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 357 GiB / 358 GiB avail 2024-03-28T17:59:40.175 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:59:39 smithi134 ceph-mon[35037]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi079", "root=default"]}]': finished 2024-03-28T17:59:40.175 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:59:39 smithi134 ceph-mon[35037]: osdmap e33: 5 total, 4 up, 5 in 2024-03-28T17:59:40.175 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:59:39 smithi134 ceph-mon[35037]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-28T17:59:40.175 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:59:39 smithi134 ceph-mon[35037]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-28T17:59:40.182 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:59:39 smithi079 ceph-mon[35312]: pgmap v156: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 357 GiB / 358 GiB avail 2024-03-28T17:59:40.182 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:59:39 smithi079 ceph-mon[35312]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi079", "root=default"]}]': finished 2024-03-28T17:59:40.182 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:59:39 smithi079 ceph-mon[35312]: osdmap e33: 5 total, 4 up, 5 in 2024-03-28T17:59:40.183 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:59:39 smithi079 ceph-mon[35312]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-28T17:59:40.183 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:59:39 smithi079 ceph-mon[35312]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-28T17:59:40.282 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:59:39 smithi022 ceph-mon[31985]: pgmap v156: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 357 GiB / 358 GiB avail 2024-03-28T17:59:40.282 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:59:39 smithi022 ceph-mon[31985]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi079", "root=default"]}]': finished 2024-03-28T17:59:40.282 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:59:39 smithi022 ceph-mon[31985]: osdmap e33: 5 total, 4 up, 5 in 2024-03-28T17:59:40.282 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:59:39 smithi022 ceph-mon[31985]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-28T17:59:40.282 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:59:39 smithi022 ceph-mon[31985]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-28T17:59:41.183 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:59:40 smithi079 ceph-mon[35312]: purged_snaps scrub starts 2024-03-28T17:59:41.183 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:59:40 smithi079 ceph-mon[35312]: purged_snaps scrub ok 2024-03-28T17:59:41.183 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:59:40 smithi079 ceph-mon[35312]: osd.4 [v2:172.21.15.79:6808/630132437,v1:172.21.15.79:6809/630132437] boot 2024-03-28T17:59:41.183 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:59:40 smithi079 ceph-mon[35312]: osdmap e34: 5 total, 5 up, 5 in 2024-03-28T17:59:41.183 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:59:40 smithi079 ceph-mon[35312]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-28T17:59:41.282 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:59:40 smithi022 ceph-mon[31985]: purged_snaps scrub starts 2024-03-28T17:59:41.282 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:59:40 smithi022 ceph-mon[31985]: purged_snaps scrub ok 2024-03-28T17:59:41.282 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:59:40 smithi022 ceph-mon[31985]: osd.4 [v2:172.21.15.79:6808/630132437,v1:172.21.15.79:6809/630132437] boot 2024-03-28T17:59:41.282 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:59:40 smithi022 ceph-mon[31985]: osdmap e34: 5 total, 5 up, 5 in 2024-03-28T17:59:41.282 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:59:40 smithi022 ceph-mon[31985]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-28T17:59:41.319 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:59:40 smithi134 ceph-mon[35037]: purged_snaps scrub starts 2024-03-28T17:59:41.319 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:59:40 smithi134 ceph-mon[35037]: purged_snaps scrub ok 2024-03-28T17:59:41.320 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:59:40 smithi134 ceph-mon[35037]: osd.4 [v2:172.21.15.79:6808/630132437,v1:172.21.15.79:6809/630132437] boot 2024-03-28T17:59:41.320 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:59:40 smithi134 ceph-mon[35037]: osdmap e34: 5 total, 5 up, 5 in 2024-03-28T17:59:41.320 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:59:40 smithi134 ceph-mon[35037]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-28T17:59:42.054 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:59:41 smithi134 ceph-mon[35037]: pgmap v159: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 357 GiB / 358 GiB avail 2024-03-28T17:59:42.054 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:59:41 smithi134 ceph-mon[35037]: osdmap e35: 5 total, 5 up, 5 in 2024-03-28T17:59:42.282 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:59:41 smithi022 ceph-mon[31985]: pgmap v159: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 357 GiB / 358 GiB avail 2024-03-28T17:59:42.282 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:59:41 smithi022 ceph-mon[31985]: osdmap e35: 5 total, 5 up, 5 in 2024-03-28T17:59:42.308 INFO:teuthology.orchestra.run.smithi134.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-03-28T17:59:42.308 INFO:teuthology.orchestra.run.smithi134.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-03-28T17:59:42.308 INFO:teuthology.orchestra.run.smithi134.stderr: stderr: 10+0 records in 2024-03-28T17:59:42.308 INFO:teuthology.orchestra.run.smithi134.stderr:10+0 records out 2024-03-28T17:59:42.309 INFO:teuthology.orchestra.run.smithi134.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0315404 s, 332 MB/s 2024-03-28T17:59:42.309 INFO:teuthology.orchestra.run.smithi134.stderr:--> Zapping successful for: 2024-03-28T17:59:42.432 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:59:41 smithi079 ceph-mon[35312]: pgmap v159: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 357 GiB / 358 GiB avail 2024-03-28T17:59:42.432 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:59:41 smithi079 ceph-mon[35312]: osdmap e35: 5 total, 5 up, 5 in 2024-03-28T17:59:43.075 DEBUG:teuthology.orchestra.run.smithi134:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fd220748-ed2b-11ee-95d0-87774f69a715 -- ceph orch daemon add osd smithi134:vg_nvme/lv_3 2024-03-28T17:59:43.485 INFO:teuthology.orchestra.run.smithi134.stderr:Inferring config /var/lib/ceph/fd220748-ed2b-11ee-95d0-87774f69a715/mon.c/config 2024-03-28T17:59:44.645 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:59:44 smithi134 ceph-mon[35037]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 159 MiB used, 447 GiB / 447 GiB avail 2024-03-28T17:59:44.645 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:59:44 smithi134 ceph-mon[35037]: Detected new or changed devices on smithi079 2024-03-28T17:59:44.645 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:59:44 smithi134 ceph-mon[35037]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T17:59:44.645 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:59:44 smithi134 ceph-mon[35037]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T17:59:44.646 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:59:44 smithi134 ceph-mon[35037]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-28T17:59:44.646 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:59:44 smithi134 ceph-mon[35037]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-28T17:59:44.646 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:59:44 smithi134 ceph-mon[35037]: Adjusting osd_memory_target on smithi079 to 8545M 2024-03-28T17:59:44.646 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:59:44 smithi134 ceph-mon[35037]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T17:59:44.646 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:59:44 smithi134 ceph-mon[35037]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T17:59:44.646 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:59:44 smithi134 ceph-mon[35037]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T17:59:44.646 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:59:44 smithi134 ceph-mon[35037]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T17:59:44.683 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:59:44 smithi079 ceph-mon[35312]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 159 MiB used, 447 GiB / 447 GiB avail 2024-03-28T17:59:44.683 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:59:44 smithi079 ceph-mon[35312]: Detected new or changed devices on smithi079 2024-03-28T17:59:44.683 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:59:44 smithi079 ceph-mon[35312]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T17:59:44.683 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:59:44 smithi079 ceph-mon[35312]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T17:59:44.683 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:59:44 smithi079 ceph-mon[35312]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-28T17:59:44.683 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:59:44 smithi079 ceph-mon[35312]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-28T17:59:44.683 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:59:44 smithi079 ceph-mon[35312]: Adjusting osd_memory_target on smithi079 to 8545M 2024-03-28T17:59:44.683 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:59:44 smithi079 ceph-mon[35312]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T17:59:44.683 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:59:44 smithi079 ceph-mon[35312]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T17:59:44.683 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:59:44 smithi079 ceph-mon[35312]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T17:59:44.683 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:59:44 smithi079 ceph-mon[35312]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T17:59:44.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:59:44 smithi022 ceph-mon[31985]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 159 MiB used, 447 GiB / 447 GiB avail 2024-03-28T17:59:44.783 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:59:44 smithi022 ceph-mon[31985]: Detected new or changed devices on smithi079 2024-03-28T17:59:44.783 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:59:44 smithi022 ceph-mon[31985]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T17:59:44.783 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:59:44 smithi022 ceph-mon[31985]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T17:59:44.783 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:59:44 smithi022 ceph-mon[31985]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-28T17:59:44.783 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:59:44 smithi022 ceph-mon[31985]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-28T17:59:44.783 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:59:44 smithi022 ceph-mon[31985]: Adjusting osd_memory_target on smithi079 to 8545M 2024-03-28T17:59:44.783 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:59:44 smithi022 ceph-mon[31985]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T17:59:44.783 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:59:44 smithi022 ceph-mon[31985]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T17:59:44.783 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:59:44 smithi022 ceph-mon[31985]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T17:59:44.783 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:59:44 smithi022 ceph-mon[31985]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T17:59:45.579 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-28T17:59:45.579+0000 7f966b200700 1 -- 172.21.15.134:0/1261466674 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9664105fb0 msgr2=0x7f966410c860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:59:45.579 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-28T17:59:45.579+0000 7f966b200700 1 --2- 172.21.15.134:0/1261466674 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9664105fb0 0x7f966410c860 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f965c00b0a0 tx=0x7f965c00b3b0 comp rx=0 tx=0).stop 2024-03-28T17:59:45.580 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-28T17:59:45.579+0000 7f966b200700 1 -- 172.21.15.134:0/1261466674 shutdown_connections 2024-03-28T17:59:45.580 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-28T17:59:45.579+0000 7f966b200700 1 --2- 172.21.15.134:0/1261466674 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9664105fb0 0x7f966410c860 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:45.580 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-28T17:59:45.579+0000 7f966b200700 1 --2- 172.21.15.134:0/1261466674 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f9664104410 0x7f9664104830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:45.580 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-28T17:59:45.579+0000 7f966b200700 1 --2- 172.21.15.134:0/1261466674 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9664105610 0x7f9664105a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:45.580 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-28T17:59:45.579+0000 7f966b200700 1 -- 172.21.15.134:0/1261466674 >> 172.21.15.134:0/1261466674 conn(0x7f96640ff9b0 msgr2=0x7f9664101df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T17:59:45.580 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-28T17:59:45.579+0000 7f966b200700 1 -- 172.21.15.134:0/1261466674 shutdown_connections 2024-03-28T17:59:45.580 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-28T17:59:45.579+0000 7f966b200700 1 -- 172.21.15.134:0/1261466674 wait complete. 2024-03-28T17:59:45.582 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-28T17:59:45.580+0000 7f966b200700 1 Processor -- start 2024-03-28T17:59:45.582 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-28T17:59:45.580+0000 7f966b200700 1 -- start start 2024-03-28T17:59:45.582 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-28T17:59:45.581+0000 7f966b200700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9664104410 0x7f966419a590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:59:45.582 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-28T17:59:45.581+0000 7f966b200700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f9664105610 0x7f966419aad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:59:45.582 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-28T17:59:45.581+0000 7f966b200700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9664105fb0 0x7f96641a1b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:59:45.582 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-28T17:59:45.581+0000 7f966b200700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9664110090 con 0x7f9664105fb0 2024-03-28T17:59:45.583 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-28T17:59:45.581+0000 7f966b200700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f966410ff10 con 0x7f9664105610 2024-03-28T17:59:45.583 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-28T17:59:45.581+0000 7f966b200700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9664110210 con 0x7f9664104410 2024-03-28T17:59:45.583 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-28T17:59:45.581+0000 7f9663e00700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9664104410 0x7f966419a590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:59:45.583 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-28T17:59:45.581+0000 7f9663e00700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9664104410 0x7f966419a590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.134:33042/0 (socket says 172.21.15.134:33042) 2024-03-28T17:59:45.583 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-28T17:59:45.581+0000 7f9663e00700 1 -- 172.21.15.134:0/241452362 learned_addr learned my addr 172.21.15.134:0/241452362 (peer_addr_for_me v2:172.21.15.134:0/0) 2024-03-28T17:59:45.583 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-28T17:59:45.581+0000 7f9663400700 1 --2- 172.21.15.134:0/241452362 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f9664105610 0x7f966419aad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:59:45.583 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-28T17:59:45.582+0000 7f9669200700 1 --2- 172.21.15.134:0/241452362 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9664105fb0 0x7f96641a1b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:59:45.583 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-28T17:59:45.582+0000 7f9663e00700 1 -- 172.21.15.134:0/241452362 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f9664105610 msgr2=0x7f966419aad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:59:45.583 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-28T17:59:45.582+0000 7f9663e00700 1 --2- 172.21.15.134:0/241452362 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f9664105610 0x7f966419aad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:45.583 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-28T17:59:45.582+0000 7f9663e00700 1 -- 172.21.15.134:0/241452362 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9664105fb0 msgr2=0x7f96641a1b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T17:59:45.583 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-28T17:59:45.582+0000 7f9663e00700 1 --2- 172.21.15.134:0/241452362 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9664105fb0 0x7f96641a1b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T17:59:45.583 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-28T17:59:45.582+0000 7f9663e00700 1 -- 172.21.15.134:0/241452362 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f96641a21f0 con 0x7f9664104410 2024-03-28T17:59:45.584 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-28T17:59:45.582+0000 7f9669200700 1 --2- 172.21.15.134:0/241452362 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9664105fb0 0x7f96641a1b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T17:59:45.584 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-28T17:59:45.582+0000 7f9663e00700 1 --2- 172.21.15.134:0/241452362 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9664104410 0x7f966419a590 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f965400d9b0 tx=0x7f965400dcc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T17:59:45.584 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-28T17:59:45.583+0000 7f9660c00700 1 -- 172.21.15.134:0/241452362 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9654017070 con 0x7f9664104410 2024-03-28T17:59:45.584 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-28T17:59:45.583+0000 7f9660c00700 1 -- 172.21.15.134:0/241452362 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f9654009e00 con 0x7f9664104410 2024-03-28T17:59:45.585 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-28T17:59:45.583+0000 7f966b200700 1 -- 172.21.15.134:0/241452362 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f96641a24e0 con 0x7f9664104410 2024-03-28T17:59:45.585 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-28T17:59:45.583+0000 7f9660c00700 1 -- 172.21.15.134:0/241452362 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f965400f620 con 0x7f9664104410 2024-03-28T17:59:45.585 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-28T17:59:45.583+0000 7f966b200700 1 -- 172.21.15.134:0/241452362 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f96641a2a20 con 0x7f9664104410 2024-03-28T17:59:45.585 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-28T17:59:45.584+0000 7f966b200700 1 -- 172.21.15.134:0/241452362 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9664068e50 con 0x7f9664104410 2024-03-28T17:59:45.586 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-28T17:59:45.586+0000 7f9660c00700 1 -- 172.21.15.134:0/241452362 <== mon.1 v2:172.21.15.134:3300/0 4 ==== mgrmap(e 16) v1 ==== 137290+0+0 (secure 0 0 0) 0x7f9654010460 con 0x7f9664104410 2024-03-28T17:59:45.587 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-28T17:59:45.587+0000 7f9660c00700 1 --2- 172.21.15.134:0/241452362 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7f96480a2910 0x7f96480a4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T17:59:45.587 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-28T17:59:45.588+0000 7f9660c00700 1 -- 172.21.15.134:0/241452362 <== mon.1 v2:172.21.15.134:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 4101+0+0 (secure 0 0 0) 0x7f96540cd920 con 0x7f9664104410 2024-03-28T17:59:45.588 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-28T17:59:45.588+0000 7f9663400700 1 --2- 172.21.15.134:0/241452362 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7f96480a2910 0x7f96480a4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T17:59:45.589 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-28T17:59:45.589+0000 7f9663400700 1 --2- 172.21.15.134:0/241452362 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7f96480a2910 0x7f96480a4dd0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f9650005950 tx=0x7f965000b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T17:59:45.595 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-28T17:59:45.595+0000 7f9660c00700 1 -- 172.21.15.134:0/241452362 <== mon.1 v2:172.21.15.134:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f9654010e20 con 0x7f9664104410 2024-03-28T17:59:45.827 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-28T17:59:45.826+0000 7f966b200700 1 -- 172.21.15.134:0/241452362 --> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi134:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) v1 -- 0x7f96640625d0 con 0x7f96480a2910 2024-03-28T17:59:46.674 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:59:46 smithi134 ceph-mon[35037]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 159 MiB used, 447 GiB / 447 GiB avail 2024-03-28T17:59:46.674 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:59:46 smithi134 ceph-mon[35037]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-28T17:59:46.674 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:59:46 smithi134 ceph-mon[35037]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-28T17:59:46.674 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:59:46 smithi134 ceph-mon[35037]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T17:59:46.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:59:46 smithi079 ceph-mon[35312]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 159 MiB used, 447 GiB / 447 GiB avail 2024-03-28T17:59:46.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:59:46 smithi079 ceph-mon[35312]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-28T17:59:46.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:59:46 smithi079 ceph-mon[35312]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-28T17:59:46.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:59:46 smithi079 ceph-mon[35312]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T17:59:46.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:59:46 smithi022 ceph-mon[31985]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 159 MiB used, 447 GiB / 447 GiB avail 2024-03-28T17:59:46.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:59:46 smithi022 ceph-mon[31985]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-28T17:59:46.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:59:46 smithi022 ceph-mon[31985]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-28T17:59:46.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:59:46 smithi022 ceph-mon[31985]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T17:59:47.674 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:59:47 smithi134 ceph-mon[35037]: from='client.24232 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi134:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-03-28T17:59:47.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:59:47 smithi079 ceph-mon[35312]: from='client.24232 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi134:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-03-28T17:59:47.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:59:47 smithi022 ceph-mon[31985]: from='client.24232 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi134:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-03-28T17:59:48.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:59:48 smithi134 ceph-mon[35037]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 159 MiB used, 447 GiB / 447 GiB avail 2024-03-28T17:59:48.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:59:48 smithi079 ceph-mon[35312]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 159 MiB used, 447 GiB / 447 GiB avail 2024-03-28T17:59:48.781 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:59:48 smithi022 ceph-mon[31985]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 159 MiB used, 447 GiB / 447 GiB avail 2024-03-28T17:59:50.341 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:59:50 smithi134 ceph-mon[35037]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 159 MiB used, 447 GiB / 447 GiB avail 2024-03-28T17:59:50.341 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:59:50 smithi134 ceph-mon[35037]: from='client.? 172.21.15.134:0/1007382515' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a7b98502-2023-4f0a-bfa0-6840cc495e0c"}]: dispatch 2024-03-28T17:59:50.341 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:59:50 smithi134 ceph-mon[35037]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a7b98502-2023-4f0a-bfa0-6840cc495e0c"}]: dispatch 2024-03-28T17:59:50.341 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:59:50 smithi134 ceph-mon[35037]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "a7b98502-2023-4f0a-bfa0-6840cc495e0c"}]': finished 2024-03-28T17:59:50.341 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:59:50 smithi134 ceph-mon[35037]: osdmap e36: 6 total, 5 up, 6 in 2024-03-28T17:59:50.341 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:59:50 smithi134 ceph-mon[35037]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-28T17:59:50.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:59:50 smithi079 ceph-mon[35312]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 159 MiB used, 447 GiB / 447 GiB avail 2024-03-28T17:59:50.683 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:59:50 smithi079 ceph-mon[35312]: from='client.? 172.21.15.134:0/1007382515' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a7b98502-2023-4f0a-bfa0-6840cc495e0c"}]: dispatch 2024-03-28T17:59:50.683 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:59:50 smithi079 ceph-mon[35312]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a7b98502-2023-4f0a-bfa0-6840cc495e0c"}]: dispatch 2024-03-28T17:59:50.683 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:59:50 smithi079 ceph-mon[35312]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "a7b98502-2023-4f0a-bfa0-6840cc495e0c"}]': finished 2024-03-28T17:59:50.683 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:59:50 smithi079 ceph-mon[35312]: osdmap e36: 6 total, 5 up, 6 in 2024-03-28T17:59:50.683 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:59:50 smithi079 ceph-mon[35312]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-28T17:59:50.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:59:50 smithi022 ceph-mon[31985]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 159 MiB used, 447 GiB / 447 GiB avail 2024-03-28T17:59:50.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:59:50 smithi022 ceph-mon[31985]: from='client.? 172.21.15.134:0/1007382515' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a7b98502-2023-4f0a-bfa0-6840cc495e0c"}]: dispatch 2024-03-28T17:59:50.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:59:50 smithi022 ceph-mon[31985]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a7b98502-2023-4f0a-bfa0-6840cc495e0c"}]: dispatch 2024-03-28T17:59:50.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:59:50 smithi022 ceph-mon[31985]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "a7b98502-2023-4f0a-bfa0-6840cc495e0c"}]': finished 2024-03-28T17:59:50.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:59:50 smithi022 ceph-mon[31985]: osdmap e36: 6 total, 5 up, 6 in 2024-03-28T17:59:50.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:59:50 smithi022 ceph-mon[31985]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-28T17:59:51.674 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:59:51 smithi134 ceph-mon[35037]: from='client.? 172.21.15.134:0/1218136211' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-28T17:59:51.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:59:51 smithi079 ceph-mon[35312]: from='client.? 172.21.15.134:0/1218136211' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-28T17:59:51.683 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:59:51 smithi022 ceph-mon[31985]: from='client.? 172.21.15.134:0/1218136211' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-28T17:59:52.674 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:59:52 smithi134 ceph-mon[35037]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 159 MiB used, 447 GiB / 447 GiB avail 2024-03-28T17:59:52.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:59:52 smithi079 ceph-mon[35312]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 159 MiB used, 447 GiB / 447 GiB avail 2024-03-28T17:59:52.781 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:59:52 smithi022 ceph-mon[31985]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 159 MiB used, 447 GiB / 447 GiB avail 2024-03-28T17:59:54.674 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:59:54 smithi134 ceph-mon[35037]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 159 MiB used, 447 GiB / 447 GiB avail 2024-03-28T17:59:54.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:59:54 smithi079 ceph-mon[35312]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 159 MiB used, 447 GiB / 447 GiB avail 2024-03-28T17:59:54.781 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:59:54 smithi022 ceph-mon[31985]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 159 MiB used, 447 GiB / 447 GiB avail 2024-03-28T17:59:56.514 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:59:56 smithi134 ceph-mon[35037]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 159 MiB used, 447 GiB / 447 GiB avail 2024-03-28T17:59:56.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:59:56 smithi079 ceph-mon[35312]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 159 MiB used, 447 GiB / 447 GiB avail 2024-03-28T17:59:56.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:59:56 smithi022 ceph-mon[31985]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 159 MiB used, 447 GiB / 447 GiB avail 2024-03-28T17:59:58.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:59:58 smithi134 ceph-mon[35037]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 159 MiB used, 447 GiB / 447 GiB avail 2024-03-28T17:59:58.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:59:58 smithi079 ceph-mon[35312]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 159 MiB used, 447 GiB / 447 GiB avail 2024-03-28T17:59:58.781 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:59:58 smithi022 ceph-mon[31985]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 159 MiB used, 447 GiB / 447 GiB avail 2024-03-28T17:59:59.424 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:59:59 smithi134 ceph-mon[35037]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-03-28T17:59:59.424 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 17:59:59 smithi134 ceph-mon[35037]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T17:59:59.683 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:59:59 smithi079 ceph-mon[35312]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-03-28T17:59:59.683 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 17:59:59 smithi079 ceph-mon[35312]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T17:59:59.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:59:59 smithi022 ceph-mon[31985]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-03-28T17:59:59.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 17:59:59 smithi022 ceph-mon[31985]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T18:00:00.482 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:00:00 smithi134 ceph-mon[35037]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 159 MiB used, 447 GiB / 447 GiB avail 2024-03-28T18:00:00.482 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:00:00 smithi134 ceph-mon[35037]: Deploying daemon osd.5 on smithi134 2024-03-28T18:00:00.482 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:00:00 smithi134 ceph-mon[35037]: overall HEALTH_OK 2024-03-28T18:00:00.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:00:00 smithi079 ceph-mon[35312]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 159 MiB used, 447 GiB / 447 GiB avail 2024-03-28T18:00:00.683 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:00:00 smithi079 ceph-mon[35312]: Deploying daemon osd.5 on smithi134 2024-03-28T18:00:00.683 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:00:00 smithi079 ceph-mon[35312]: overall HEALTH_OK 2024-03-28T18:00:00.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:00:00 smithi022 ceph-mon[31985]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 159 MiB used, 447 GiB / 447 GiB avail 2024-03-28T18:00:00.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:00:00 smithi022 ceph-mon[31985]: Deploying daemon osd.5 on smithi134 2024-03-28T18:00:00.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:00:00 smithi022 ceph-mon[31985]: overall HEALTH_OK 2024-03-28T18:00:02.674 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:00:02 smithi134 ceph-mon[35037]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 159 MiB used, 447 GiB / 447 GiB avail 2024-03-28T18:00:02.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:00:02 smithi079 ceph-mon[35312]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 159 MiB used, 447 GiB / 447 GiB avail 2024-03-28T18:00:02.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:00:02 smithi022 ceph-mon[31985]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 159 MiB used, 447 GiB / 447 GiB avail 2024-03-28T18:00:04.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:00:04 smithi134 ceph-mon[35037]: pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 159 MiB used, 447 GiB / 447 GiB avail 2024-03-28T18:00:04.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:00:04 smithi079 ceph-mon[35312]: pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 159 MiB used, 447 GiB / 447 GiB avail 2024-03-28T18:00:04.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:00:04 smithi022 ceph-mon[31985]: pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 159 MiB used, 447 GiB / 447 GiB avail 2024-03-28T18:00:06.638 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:00:06 smithi134 ceph-mon[35037]: pgmap v173: 1 pgs: 1 active+clean; 577 KiB data, 159 MiB used, 447 GiB / 447 GiB avail 2024-03-28T18:00:06.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:00:06 smithi079 ceph-mon[35312]: pgmap v173: 1 pgs: 1 active+clean; 577 KiB data, 159 MiB used, 447 GiB / 447 GiB avail 2024-03-28T18:00:06.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:00:06 smithi022 ceph-mon[31985]: pgmap v173: 1 pgs: 1 active+clean; 577 KiB data, 159 MiB used, 447 GiB / 447 GiB avail 2024-03-28T18:00:08.326 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:00:08 smithi134 ceph-mon[35037]: pgmap v174: 1 pgs: 1 active+clean; 577 KiB data, 159 MiB used, 447 GiB / 447 GiB avail 2024-03-28T18:00:08.326 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:00:08 smithi134 ceph-mon[35037]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T18:00:08.326 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:00:08 smithi134 ceph-mon[35037]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T18:00:08.326 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:00:08 smithi134 ceph-mon[35037]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T18:00:08.432 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:00:08 smithi079 ceph-mon[35312]: pgmap v174: 1 pgs: 1 active+clean; 577 KiB data, 159 MiB used, 447 GiB / 447 GiB avail 2024-03-28T18:00:08.433 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:00:08 smithi079 ceph-mon[35312]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T18:00:08.433 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:00:08 smithi079 ceph-mon[35312]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T18:00:08.433 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:00:08 smithi079 ceph-mon[35312]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T18:00:08.531 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:00:08 smithi022 ceph-mon[31985]: pgmap v174: 1 pgs: 1 active+clean; 577 KiB data, 159 MiB used, 447 GiB / 447 GiB avail 2024-03-28T18:00:08.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:00:08 smithi022 ceph-mon[31985]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T18:00:08.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:00:08 smithi022 ceph-mon[31985]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T18:00:08.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:00:08 smithi022 ceph-mon[31985]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T18:00:10.296 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:00:10 smithi134 ceph-mon[35037]: pgmap v175: 1 pgs: 1 active+clean; 577 KiB data, 159 MiB used, 447 GiB / 447 GiB avail 2024-03-28T18:00:10.432 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:00:10 smithi079 ceph-mon[35312]: pgmap v175: 1 pgs: 1 active+clean; 577 KiB data, 159 MiB used, 447 GiB / 447 GiB avail 2024-03-28T18:00:10.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:00:10 smithi022 ceph-mon[31985]: pgmap v175: 1 pgs: 1 active+clean; 577 KiB data, 159 MiB used, 447 GiB / 447 GiB avail 2024-03-28T18:00:11.674 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:00:11 smithi134 ceph-mon[35037]: pgmap v176: 1 pgs: 1 active+clean; 577 KiB data, 159 MiB used, 447 GiB / 447 GiB avail 2024-03-28T18:00:11.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:00:11 smithi134 ceph-mon[35037]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T18:00:11.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:00:11 smithi134 ceph-mon[35037]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T18:00:11.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:00:11 smithi134 ceph-mon[35037]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T18:00:11.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:00:11 smithi134 ceph-mon[35037]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T18:00:11.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:00:11 smithi134 ceph-mon[35037]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T18:00:11.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:00:11 smithi134 ceph-mon[35037]: from='osd.5 [v2:172.21.15.134:6808/2135406915,v1:172.21.15.134:6809/2135406915]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-03-28T18:00:11.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:00:11 smithi134 ceph-mon[35037]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-03-28T18:00:11.899 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-28T18:00:11.898+0000 7f9660c00700 1 -- 172.21.15.134:0/241452362 <== mgr.14152 v2:172.21.15.22:6800/4201048392 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f96640625d0 con 0x7f96480a2910 2024-03-28T18:00:11.900 INFO:teuthology.orchestra.run.smithi134.stdout:Created osd(s) 5 on host 'smithi134' 2024-03-28T18:00:11.902 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-28T18:00:11.902+0000 7f966b200700 1 -- 172.21.15.134:0/241452362 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7f96480a2910 msgr2=0x7f96480a4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:00:11.902 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-28T18:00:11.902+0000 7f966b200700 1 --2- 172.21.15.134:0/241452362 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7f96480a2910 0x7f96480a4dd0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f9650005950 tx=0x7f965000b410 comp rx=0 tx=0).stop 2024-03-28T18:00:11.902 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-28T18:00:11.902+0000 7f966b200700 1 -- 172.21.15.134:0/241452362 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9664104410 msgr2=0x7f966419a590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:00:11.902 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-28T18:00:11.902+0000 7f966b200700 1 --2- 172.21.15.134:0/241452362 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9664104410 0x7f966419a590 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f965400d9b0 tx=0x7f965400dcc0 comp rx=0 tx=0).stop 2024-03-28T18:00:11.903 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-28T18:00:11.902+0000 7f966b200700 1 -- 172.21.15.134:0/241452362 shutdown_connections 2024-03-28T18:00:11.903 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-28T18:00:11.902+0000 7f966b200700 1 --2- 172.21.15.134:0/241452362 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9664105fb0 0x7f96641a1b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:00:11.903 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-28T18:00:11.902+0000 7f966b200700 1 --2- 172.21.15.134:0/241452362 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7f96480a2910 0x7f96480a4dd0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:00:11.903 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-28T18:00:11.902+0000 7f966b200700 1 --2- 172.21.15.134:0/241452362 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9664104410 0x7f966419a590 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:00:11.903 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-28T18:00:11.902+0000 7f966b200700 1 --2- 172.21.15.134:0/241452362 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f9664105610 0x7f966419aad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:00:11.903 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-28T18:00:11.902+0000 7f966b200700 1 -- 172.21.15.134:0/241452362 >> 172.21.15.134:0/241452362 conn(0x7f96640ff9b0 msgr2=0x7f96641088b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:00:11.903 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-28T18:00:11.902+0000 7f966b200700 1 -- 172.21.15.134:0/241452362 shutdown_connections 2024-03-28T18:00:11.903 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-28T18:00:11.903+0000 7f966b200700 1 -- 172.21.15.134:0/241452362 wait complete. 2024-03-28T18:00:11.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:00:11 smithi079 ceph-mon[35312]: pgmap v176: 1 pgs: 1 active+clean; 577 KiB data, 159 MiB used, 447 GiB / 447 GiB avail 2024-03-28T18:00:11.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:00:11 smithi079 ceph-mon[35312]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T18:00:11.933 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:00:11 smithi079 ceph-mon[35312]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T18:00:11.933 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:00:11 smithi079 ceph-mon[35312]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T18:00:11.933 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:00:11 smithi079 ceph-mon[35312]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T18:00:11.933 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:00:11 smithi079 ceph-mon[35312]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T18:00:11.933 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:00:11 smithi079 ceph-mon[35312]: from='osd.5 [v2:172.21.15.134:6808/2135406915,v1:172.21.15.134:6809/2135406915]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-03-28T18:00:11.933 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:00:11 smithi079 ceph-mon[35312]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-03-28T18:00:12.032 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:00:11 smithi022 ceph-mon[31985]: pgmap v176: 1 pgs: 1 active+clean; 577 KiB data, 159 MiB used, 447 GiB / 447 GiB avail 2024-03-28T18:00:12.032 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:00:11 smithi022 ceph-mon[31985]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T18:00:12.032 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:00:11 smithi022 ceph-mon[31985]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T18:00:12.032 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:00:11 smithi022 ceph-mon[31985]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T18:00:12.032 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:00:11 smithi022 ceph-mon[31985]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T18:00:12.032 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:00:11 smithi022 ceph-mon[31985]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T18:00:12.032 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:00:11 smithi022 ceph-mon[31985]: from='osd.5 [v2:172.21.15.134:6808/2135406915,v1:172.21.15.134:6809/2135406915]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-03-28T18:00:12.032 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:00:11 smithi022 ceph-mon[31985]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-03-28T18:00:12.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:00:12 smithi134 ceph-mon[35037]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-03-28T18:00:12.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:00:12 smithi134 ceph-mon[35037]: osdmap e37: 6 total, 5 up, 6 in 2024-03-28T18:00:12.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:00:12 smithi134 ceph-mon[35037]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-28T18:00:12.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:00:12 smithi134 ceph-mon[35037]: from='osd.5 [v2:172.21.15.134:6808/2135406915,v1:172.21.15.134:6809/2135406915]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi134", "root=default"]}]: dispatch 2024-03-28T18:00:12.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:00:12 smithi134 ceph-mon[35037]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi134", "root=default"]}]: dispatch 2024-03-28T18:00:12.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:00:12 smithi134 ceph-mon[35037]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T18:00:12.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:00:12 smithi134 ceph-mon[35037]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T18:00:12.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:00:12 smithi134 ceph-mon[35037]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T18:00:12.790 DEBUG:teuthology.orchestra.run.smithi134:osd.5> sudo journalctl -f -n 0 -u ceph-fd220748-ed2b-11ee-95d0-87774f69a715@osd.5.service 2024-03-28T18:00:12.795 INFO:tasks.cephadm:Deploying osd.6 on smithi022 with /dev/vg_nvme/lv_2... 2024-03-28T18:00:12.795 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fd220748-ed2b-11ee-95d0-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-03-28T18:00:12.890 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:00:12 smithi022 ceph-mon[31985]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-03-28T18:00:12.890 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:00:12 smithi022 ceph-mon[31985]: osdmap e37: 6 total, 5 up, 6 in 2024-03-28T18:00:12.890 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:00:12 smithi022 ceph-mon[31985]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-28T18:00:12.890 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:00:12 smithi022 ceph-mon[31985]: from='osd.5 [v2:172.21.15.134:6808/2135406915,v1:172.21.15.134:6809/2135406915]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi134", "root=default"]}]: dispatch 2024-03-28T18:00:12.890 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:00:12 smithi022 ceph-mon[31985]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi134", "root=default"]}]: dispatch 2024-03-28T18:00:12.890 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:00:12 smithi022 ceph-mon[31985]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T18:00:12.890 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:00:12 smithi022 ceph-mon[31985]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T18:00:12.890 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:00:12 smithi022 ceph-mon[31985]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T18:00:12.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:00:12 smithi079 ceph-mon[35312]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-03-28T18:00:12.933 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:00:12 smithi079 ceph-mon[35312]: osdmap e37: 6 total, 5 up, 6 in 2024-03-28T18:00:12.933 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:00:12 smithi079 ceph-mon[35312]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-28T18:00:12.933 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:00:12 smithi079 ceph-mon[35312]: from='osd.5 [v2:172.21.15.134:6808/2135406915,v1:172.21.15.134:6809/2135406915]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi134", "root=default"]}]: dispatch 2024-03-28T18:00:12.933 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:00:12 smithi079 ceph-mon[35312]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi134", "root=default"]}]: dispatch 2024-03-28T18:00:12.933 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:00:12 smithi079 ceph-mon[35312]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T18:00:12.933 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:00:12 smithi079 ceph-mon[35312]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T18:00:12.933 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:00:12 smithi079 ceph-mon[35312]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T18:00:13.261 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/fd220748-ed2b-11ee-95d0-87774f69a715/mon.a/config 2024-03-28T18:00:13.640 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:00:13 smithi134 ceph-mon[35037]: pgmap v178: 1 pgs: 1 active+clean; 577 KiB data, 159 MiB used, 447 GiB / 447 GiB avail 2024-03-28T18:00:13.640 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:00:13 smithi134 ceph-mon[35037]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi134", "root=default"]}]': finished 2024-03-28T18:00:13.640 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:00:13 smithi134 ceph-mon[35037]: osdmap e38: 6 total, 5 up, 6 in 2024-03-28T18:00:13.640 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:00:13 smithi134 ceph-mon[35037]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-28T18:00:13.641 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:00:13 smithi134 ceph-mon[35037]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-28T18:00:13.641 INFO:journalctl@ceph.osd.5.smithi134.stdout:Mar 28 18:00:13 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-osd-5[46996]: 2024-03-28T18:00:13.602+0000 7fb396200700 -1 osd.5 0 waiting for initial osdmap 2024-03-28T18:00:13.641 INFO:journalctl@ceph.osd.5.smithi134.stdout:Mar 28 18:00:13 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-osd-5[46996]: 2024-03-28T18:00:13.639+0000 7fb38c000700 -1 osd.5 39 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-28T18:00:13.660 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:00:13 smithi022 ceph-mon[31985]: pgmap v178: 1 pgs: 1 active+clean; 577 KiB data, 159 MiB used, 447 GiB / 447 GiB avail 2024-03-28T18:00:13.660 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:00:13 smithi022 ceph-mon[31985]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi134", "root=default"]}]': finished 2024-03-28T18:00:13.660 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:00:13 smithi022 ceph-mon[31985]: osdmap e38: 6 total, 5 up, 6 in 2024-03-28T18:00:13.660 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:00:13 smithi022 ceph-mon[31985]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-28T18:00:13.660 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:00:13 smithi022 ceph-mon[31985]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-28T18:00:13.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:00:13 smithi079 ceph-mon[35312]: pgmap v178: 1 pgs: 1 active+clean; 577 KiB data, 159 MiB used, 447 GiB / 447 GiB avail 2024-03-28T18:00:13.933 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:00:13 smithi079 ceph-mon[35312]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi134", "root=default"]}]': finished 2024-03-28T18:00:13.933 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:00:13 smithi079 ceph-mon[35312]: osdmap e38: 6 total, 5 up, 6 in 2024-03-28T18:00:13.933 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:00:13 smithi079 ceph-mon[35312]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-28T18:00:13.933 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:00:13 smithi079 ceph-mon[35312]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-28T18:00:14.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:00:14 smithi022 ceph-mon[31985]: purged_snaps scrub starts 2024-03-28T18:00:14.783 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:00:14 smithi022 ceph-mon[31985]: purged_snaps scrub ok 2024-03-28T18:00:14.783 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:00:14 smithi022 ceph-mon[31985]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-28T18:00:14.783 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:00:14 smithi022 ceph-mon[31985]: osdmap e39: 6 total, 5 up, 6 in 2024-03-28T18:00:14.783 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:00:14 smithi022 ceph-mon[31985]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-28T18:00:14.783 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:00:14 smithi022 ceph-mon[31985]: from='osd.5 ' entity='osd.5' 2024-03-28T18:00:14.848 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:00:14 smithi134 ceph-mon[35037]: purged_snaps scrub starts 2024-03-28T18:00:14.848 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:00:14 smithi134 ceph-mon[35037]: purged_snaps scrub ok 2024-03-28T18:00:14.848 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:00:14 smithi134 ceph-mon[35037]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-28T18:00:14.848 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:00:14 smithi134 ceph-mon[35037]: osdmap e39: 6 total, 5 up, 6 in 2024-03-28T18:00:14.848 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:00:14 smithi134 ceph-mon[35037]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-28T18:00:14.848 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:00:14 smithi134 ceph-mon[35037]: from='osd.5 ' entity='osd.5' 2024-03-28T18:00:14.933 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:00:14 smithi079 ceph-mon[35312]: purged_snaps scrub starts 2024-03-28T18:00:14.933 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:00:14 smithi079 ceph-mon[35312]: purged_snaps scrub ok 2024-03-28T18:00:14.933 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:00:14 smithi079 ceph-mon[35312]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-28T18:00:14.933 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:00:14 smithi079 ceph-mon[35312]: osdmap e39: 6 total, 5 up, 6 in 2024-03-28T18:00:14.933 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:00:14 smithi079 ceph-mon[35312]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-28T18:00:14.933 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:00:14 smithi079 ceph-mon[35312]: from='osd.5 ' entity='osd.5' 2024-03-28T18:00:15.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:00:15 smithi134 ceph-mon[35037]: pgmap v181: 1 pgs: 1 remapped+peering; 577 KiB data, 160 MiB used, 447 GiB / 447 GiB avail 2024-03-28T18:00:15.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:00:15 smithi134 ceph-mon[35037]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-28T18:00:15.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:00:15 smithi134 ceph-mon[35037]: osd.5 [v2:172.21.15.134:6808/2135406915,v1:172.21.15.134:6809/2135406915] boot 2024-03-28T18:00:15.925 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:00:15 smithi134 ceph-mon[35037]: osdmap e40: 6 total, 6 up, 6 in 2024-03-28T18:00:15.925 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:00:15 smithi134 ceph-mon[35037]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-28T18:00:15.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:00:15 smithi079 ceph-mon[35312]: pgmap v181: 1 pgs: 1 remapped+peering; 577 KiB data, 160 MiB used, 447 GiB / 447 GiB avail 2024-03-28T18:00:15.933 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:00:15 smithi079 ceph-mon[35312]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-28T18:00:15.933 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:00:15 smithi079 ceph-mon[35312]: osd.5 [v2:172.21.15.134:6808/2135406915,v1:172.21.15.134:6809/2135406915] boot 2024-03-28T18:00:15.933 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:00:15 smithi079 ceph-mon[35312]: osdmap e40: 6 total, 6 up, 6 in 2024-03-28T18:00:15.933 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:00:15 smithi079 ceph-mon[35312]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-28T18:00:16.031 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:00:15 smithi022 ceph-mon[31985]: pgmap v181: 1 pgs: 1 remapped+peering; 577 KiB data, 160 MiB used, 447 GiB / 447 GiB avail 2024-03-28T18:00:16.032 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:00:15 smithi022 ceph-mon[31985]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-28T18:00:16.032 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:00:15 smithi022 ceph-mon[31985]: osd.5 [v2:172.21.15.134:6808/2135406915,v1:172.21.15.134:6809/2135406915] boot 2024-03-28T18:00:16.032 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:00:15 smithi022 ceph-mon[31985]: osdmap e40: 6 total, 6 up, 6 in 2024-03-28T18:00:16.032 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:00:15 smithi022 ceph-mon[31985]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-28T18:00:16.452 INFO:teuthology.orchestra.run.smithi022.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-03-28T18:00:16.452 INFO:teuthology.orchestra.run.smithi022.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-03-28T18:00:16.452 INFO:teuthology.orchestra.run.smithi022.stderr: stderr: 10+0 records in 2024-03-28T18:00:16.452 INFO:teuthology.orchestra.run.smithi022.stderr:10+0 records out 2024-03-28T18:00:16.452 INFO:teuthology.orchestra.run.smithi022.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.038991 s, 269 MB/s 2024-03-28T18:00:16.452 INFO:teuthology.orchestra.run.smithi022.stderr:--> Zapping successful for: 2024-03-28T18:00:16.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:00:16 smithi022 ceph-mon[31985]: osdmap e41: 6 total, 6 up, 6 in 2024-03-28T18:00:16.838 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:00:16 smithi134 ceph-mon[35037]: osdmap e41: 6 total, 6 up, 6 in 2024-03-28T18:00:16.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:00:16 smithi079 ceph-mon[35312]: osdmap e41: 6 total, 6 up, 6 in 2024-03-28T18:00:17.372 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fd220748-ed2b-11ee-95d0-87774f69a715 -- ceph orch daemon add osd smithi022:vg_nvme/lv_2 2024-03-28T18:00:17.816 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/fd220748-ed2b-11ee-95d0-87774f69a715/mon.a/config 2024-03-28T18:00:17.925 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:00:17 smithi134 ceph-mon[35037]: pgmap v184: 1 pgs: 1 remapped+peering; 577 KiB data, 190 MiB used, 536 GiB / 536 GiB avail 2024-03-28T18:00:17.925 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:00:17 smithi134 ceph-mon[35037]: osdmap e42: 6 total, 6 up, 6 in 2024-03-28T18:00:17.925 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:00:17 smithi134 ceph-mon[35037]: Detected new or changed devices on smithi134 2024-03-28T18:00:17.925 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:00:17 smithi134 ceph-mon[35037]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T18:00:17.926 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:00:17 smithi134 ceph-mon[35037]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T18:00:17.926 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:00:17 smithi134 ceph-mon[35037]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-28T18:00:17.926 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:00:17 smithi134 ceph-mon[35037]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-28T18:00:17.926 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:00:17 smithi134 ceph-mon[35037]: Adjusting osd_memory_target on smithi134 to 8545M 2024-03-28T18:00:17.926 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:00:17 smithi134 ceph-mon[35037]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T18:00:17.926 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:00:17 smithi134 ceph-mon[35037]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T18:00:17.926 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:00:17 smithi134 ceph-mon[35037]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T18:00:17.926 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:00:17 smithi134 ceph-mon[35037]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T18:00:17.926 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:00:17 smithi134 ceph-mon[35037]: osdmap e43: 6 total, 6 up, 6 in 2024-03-28T18:00:17.933 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:00:17 smithi079 ceph-mon[35312]: pgmap v184: 1 pgs: 1 remapped+peering; 577 KiB data, 190 MiB used, 536 GiB / 536 GiB avail 2024-03-28T18:00:17.933 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:00:17 smithi079 ceph-mon[35312]: osdmap e42: 6 total, 6 up, 6 in 2024-03-28T18:00:17.933 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:00:17 smithi079 ceph-mon[35312]: Detected new or changed devices on smithi134 2024-03-28T18:00:17.933 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:00:17 smithi079 ceph-mon[35312]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T18:00:17.933 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:00:17 smithi079 ceph-mon[35312]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T18:00:17.933 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:00:17 smithi079 ceph-mon[35312]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-28T18:00:17.933 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:00:17 smithi079 ceph-mon[35312]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-28T18:00:17.933 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:00:17 smithi079 ceph-mon[35312]: Adjusting osd_memory_target on smithi134 to 8545M 2024-03-28T18:00:17.933 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:00:17 smithi079 ceph-mon[35312]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T18:00:17.933 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:00:17 smithi079 ceph-mon[35312]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T18:00:17.933 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:00:17 smithi079 ceph-mon[35312]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T18:00:17.933 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:00:17 smithi079 ceph-mon[35312]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T18:00:17.933 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:00:17 smithi079 ceph-mon[35312]: osdmap e43: 6 total, 6 up, 6 in 2024-03-28T18:00:17.965 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:00:17 smithi022 ceph-mon[31985]: pgmap v184: 1 pgs: 1 remapped+peering; 577 KiB data, 190 MiB used, 536 GiB / 536 GiB avail 2024-03-28T18:00:17.966 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:00:17 smithi022 ceph-mon[31985]: osdmap e42: 6 total, 6 up, 6 in 2024-03-28T18:00:17.966 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:00:17 smithi022 ceph-mon[31985]: Detected new or changed devices on smithi134 2024-03-28T18:00:17.966 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:00:17 smithi022 ceph-mon[31985]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T18:00:17.966 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:00:17 smithi022 ceph-mon[31985]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T18:00:17.966 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:00:17 smithi022 ceph-mon[31985]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-28T18:00:17.966 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:00:17 smithi022 ceph-mon[31985]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-28T18:00:17.966 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:00:17 smithi022 ceph-mon[31985]: Adjusting osd_memory_target on smithi134 to 8545M 2024-03-28T18:00:17.966 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:00:17 smithi022 ceph-mon[31985]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T18:00:17.966 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:00:17 smithi022 ceph-mon[31985]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T18:00:17.966 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:00:17 smithi022 ceph-mon[31985]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T18:00:17.966 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:00:17 smithi022 ceph-mon[31985]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T18:00:17.966 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:00:17 smithi022 ceph-mon[31985]: osdmap e43: 6 total, 6 up, 6 in 2024-03-28T18:00:19.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:00:19 smithi022 ceph-mon[31985]: osdmap e44: 6 total, 6 up, 6 in 2024-03-28T18:00:19.674 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:00:19 smithi134 ceph-mon[35037]: osdmap e44: 6 total, 6 up, 6 in 2024-03-28T18:00:19.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:00:19 smithi079 ceph-mon[35312]: osdmap e44: 6 total, 6 up, 6 in 2024-03-28T18:00:20.046 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:00:20.045+0000 7fc099200700 1 -- 172.21.15.22:0/3191080653 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc0941055e0 msgr2=0x7fc0941079d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:00:20.047 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:00:20.045+0000 7fc099200700 1 --2- 172.21.15.22:0/3191080653 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc0941055e0 0x7fc0941079d0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fc07c009990 tx=0x7fc07c009ca0 comp rx=0 tx=0).stop 2024-03-28T18:00:20.047 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:00:20.045+0000 7fc099200700 1 -- 172.21.15.22:0/3191080653 shutdown_connections 2024-03-28T18:00:20.047 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:00:20.045+0000 7fc099200700 1 --2- 172.21.15.22:0/3191080653 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc094107f10 0x7fc09410a320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:00:20.047 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:00:20.045+0000 7fc099200700 1 --2- 172.21.15.22:0/3191080653 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc094102cb0 0x7fc0941050a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:00:20.047 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:00:20.045+0000 7fc099200700 1 --2- 172.21.15.22:0/3191080653 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc0941055e0 0x7fc0941079d0 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:00:20.047 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:00:20.045+0000 7fc099200700 1 -- 172.21.15.22:0/3191080653 >> 172.21.15.22:0/3191080653 conn(0x7fc0940fc660 msgr2=0x7fc0940feaa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:00:20.047 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:00:20.045+0000 7fc099200700 1 -- 172.21.15.22:0/3191080653 shutdown_connections 2024-03-28T18:00:20.047 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:00:20.045+0000 7fc099200700 1 -- 172.21.15.22:0/3191080653 wait complete. 2024-03-28T18:00:20.048 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:00:20.046+0000 7fc099200700 1 Processor -- start 2024-03-28T18:00:20.049 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:00:20.046+0000 7fc099200700 1 -- start start 2024-03-28T18:00:20.049 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:00:20.047+0000 7fc099200700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc094102cb0 0x7fc09419a650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:00:20.049 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:00:20.047+0000 7fc099200700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc0941055e0 0x7fc09419ab90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:00:20.049 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:00:20.047+0000 7fc099200700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc094107f10 0x7fc0941a1c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:00:20.049 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:00:20.047+0000 7fc099200700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc09410ddf0 con 0x7fc094102cb0 2024-03-28T18:00:20.049 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:00:20.047+0000 7fc099200700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc09410dc70 con 0x7fc094107f10 2024-03-28T18:00:20.049 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:00:20.047+0000 7fc099200700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc09410df70 con 0x7fc0941055e0 2024-03-28T18:00:20.049 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:00:20.047+0000 7fc091e00700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc094102cb0 0x7fc09419a650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:00:20.049 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:00:20.047+0000 7fc091e00700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc094102cb0 0x7fc09419a650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:53466/0 (socket says 172.21.15.22:53466) 2024-03-28T18:00:20.049 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:00:20.047+0000 7fc091e00700 1 -- 172.21.15.22:0/1393166020 learned_addr learned my addr 172.21.15.22:0/1393166020 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-28T18:00:20.049 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:00:20.047+0000 7fc092800700 1 --2- 172.21.15.22:0/1393166020 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc094107f10 0x7fc0941a1c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:00:20.049 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:00:20.047+0000 7fc091e00700 1 -- 172.21.15.22:0/1393166020 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc0941055e0 msgr2=0x7fc09419ab90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:00:20.050 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:00:20.047+0000 7fc089400700 1 --2- 172.21.15.22:0/1393166020 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc0941055e0 0x7fc09419ab90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:00:20.050 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:00:20.048+0000 7fc091e00700 1 --2- 172.21.15.22:0/1393166020 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc0941055e0 0x7fc09419ab90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:00:20.050 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:00:20.048+0000 7fc091e00700 1 -- 172.21.15.22:0/1393166020 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc094107f10 msgr2=0x7fc0941a1c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:00:20.050 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:00:20.048+0000 7fc091e00700 1 --2- 172.21.15.22:0/1393166020 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc094107f10 0x7fc0941a1c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:00:20.050 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:00:20.048+0000 7fc091e00700 1 -- 172.21.15.22:0/1393166020 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc0941a22b0 con 0x7fc094102cb0 2024-03-28T18:00:20.050 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:00:20.048+0000 7fc092800700 1 --2- 172.21.15.22:0/1393166020 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc094107f10 0x7fc0941a1c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T18:00:20.050 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:00:20.048+0000 7fc089400700 1 --2- 172.21.15.22:0/1393166020 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc0941055e0 0x7fc09419ab90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T18:00:20.050 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:00:20.048+0000 7fc091e00700 1 --2- 172.21.15.22:0/1393166020 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc094102cb0 0x7fc09419a650 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fc08400d980 tx=0x7fc08400dd40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:00:20.051 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:00:20.048+0000 7fc08b400700 1 -- 172.21.15.22:0/1393166020 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc084016070 con 0x7fc094102cb0 2024-03-28T18:00:20.051 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:00:20.048+0000 7fc08b400700 1 -- 172.21.15.22:0/1393166020 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7fc084009b40 con 0x7fc094102cb0 2024-03-28T18:00:20.051 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:00:20.048+0000 7fc099200700 1 -- 172.21.15.22:0/1393166020 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc0941a25a0 con 0x7fc094102cb0 2024-03-28T18:00:20.051 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:00:20.048+0000 7fc08b400700 1 -- 172.21.15.22:0/1393166020 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc08400f3c0 con 0x7fc094102cb0 2024-03-28T18:00:20.051 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:00:20.048+0000 7fc099200700 1 -- 172.21.15.22:0/1393166020 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc0941a2b30 con 0x7fc094102cb0 2024-03-28T18:00:20.051 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:00:20.049+0000 7fc099200700 1 -- 172.21.15.22:0/1393166020 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc094068e50 con 0x7fc094102cb0 2024-03-28T18:00:20.052 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:00:20.052+0000 7fc08b400700 1 -- 172.21.15.22:0/1393166020 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 16) v1 ==== 137290+0+0 (secure 0 0 0) 0x7fc084019070 con 0x7fc094102cb0 2024-03-28T18:00:20.055 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:00:20.054+0000 7fc08b400700 1 --2- 172.21.15.22:0/1393166020 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7fc0780a2910 0x7fc0780a4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:00:20.055 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:00:20.054+0000 7fc08b400700 1 -- 172.21.15.22:0/1393166020 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 4533+0+0 (secure 0 0 0) 0x7fc0840cd7e0 con 0x7fc094102cb0 2024-03-28T18:00:20.055 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:00:20.054+0000 7fc089400700 1 --2- 172.21.15.22:0/1393166020 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7fc0780a2910 0x7fc0780a4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:00:20.056 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:00:20.055+0000 7fc089400700 1 --2- 172.21.15.22:0/1393166020 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7fc0780a2910 0x7fc0780a4dd0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fc07c00b5c0 tx=0x7fc07c005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:00:20.058 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:00:20.058+0000 7fc08b400700 1 -- 172.21.15.22:0/1393166020 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fc08409b6a0 con 0x7fc094102cb0 2024-03-28T18:00:20.299 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:00:20.298+0000 7fc099200700 1 -- 172.21.15.22:0/1393166020 --> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi022:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) v1 -- 0x7fc0940625d0 con 0x7fc0780a2910 2024-03-28T18:00:20.674 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:00:20 smithi134 ceph-mon[35037]: pgmap v188: 1 pgs: 1 remapped+peering; 577 KiB data, 191 MiB used, 536 GiB / 536 GiB avail 2024-03-28T18:00:20.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:00:20 smithi134 ceph-mon[35037]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-28T18:00:20.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:00:20 smithi134 ceph-mon[35037]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-28T18:00:20.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:00:20 smithi134 ceph-mon[35037]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T18:00:20.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:00:20 smithi079 ceph-mon[35312]: pgmap v188: 1 pgs: 1 remapped+peering; 577 KiB data, 191 MiB used, 536 GiB / 536 GiB avail 2024-03-28T18:00:20.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:00:20 smithi079 ceph-mon[35312]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-28T18:00:20.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:00:20 smithi079 ceph-mon[35312]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-28T18:00:20.683 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:00:20 smithi079 ceph-mon[35312]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T18:00:20.781 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:00:20 smithi022 ceph-mon[31985]: pgmap v188: 1 pgs: 1 remapped+peering; 577 KiB data, 191 MiB used, 536 GiB / 536 GiB avail 2024-03-28T18:00:20.781 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:00:20 smithi022 ceph-mon[31985]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-28T18:00:20.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:00:20 smithi022 ceph-mon[31985]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-28T18:00:20.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:00:20 smithi022 ceph-mon[31985]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T18:00:21.531 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:00:21 smithi022 ceph-mon[31985]: from='client.14358 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi022:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-03-28T18:00:21.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:00:21 smithi079 ceph-mon[35312]: from='client.14358 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi022:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-03-28T18:00:21.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:00:21 smithi134 ceph-mon[35037]: from='client.14358 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi022:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-03-28T18:00:22.531 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:00:22 smithi022 ceph-mon[31985]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 191 MiB used, 536 GiB / 536 GiB avail; 118 KiB/s, 0 objects/s recovering 2024-03-28T18:00:22.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:00:22 smithi079 ceph-mon[35312]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 191 MiB used, 536 GiB / 536 GiB avail; 118 KiB/s, 0 objects/s recovering 2024-03-28T18:00:22.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:00:22 smithi134 ceph-mon[35037]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 191 MiB used, 536 GiB / 536 GiB avail; 118 KiB/s, 0 objects/s recovering 2024-03-28T18:00:24.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:00:24 smithi134 ceph-mon[35037]: pgmap v190: 1 pgs: 1 active+clean; 577 KiB data, 191 MiB used, 536 GiB / 536 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-03-28T18:00:24.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:00:24 smithi134 ceph-mon[35037]: from='client.? 172.21.15.22:0/618575440' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "634c6562-d683-41c8-8242-fc848c38ce1f"}]: dispatch 2024-03-28T18:00:24.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:00:24 smithi134 ceph-mon[35037]: from='client.? 172.21.15.22:0/618575440' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "634c6562-d683-41c8-8242-fc848c38ce1f"}]': finished 2024-03-28T18:00:24.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:00:24 smithi134 ceph-mon[35037]: osdmap e45: 7 total, 6 up, 7 in 2024-03-28T18:00:24.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:00:24 smithi134 ceph-mon[35037]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-28T18:00:24.762 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:00:24 smithi022 ceph-mon[31985]: pgmap v190: 1 pgs: 1 active+clean; 577 KiB data, 191 MiB used, 536 GiB / 536 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-03-28T18:00:24.762 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:00:24 smithi022 ceph-mon[31985]: from='client.? 172.21.15.22:0/618575440' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "634c6562-d683-41c8-8242-fc848c38ce1f"}]: dispatch 2024-03-28T18:00:24.762 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:00:24 smithi022 ceph-mon[31985]: from='client.? 172.21.15.22:0/618575440' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "634c6562-d683-41c8-8242-fc848c38ce1f"}]': finished 2024-03-28T18:00:24.762 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:00:24 smithi022 ceph-mon[31985]: osdmap e45: 7 total, 6 up, 7 in 2024-03-28T18:00:24.762 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:00:24 smithi022 ceph-mon[31985]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-28T18:00:24.933 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:00:24 smithi079 ceph-mon[35312]: pgmap v190: 1 pgs: 1 active+clean; 577 KiB data, 191 MiB used, 536 GiB / 536 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-03-28T18:00:24.933 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:00:24 smithi079 ceph-mon[35312]: from='client.? 172.21.15.22:0/618575440' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "634c6562-d683-41c8-8242-fc848c38ce1f"}]: dispatch 2024-03-28T18:00:24.933 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:00:24 smithi079 ceph-mon[35312]: from='client.? 172.21.15.22:0/618575440' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "634c6562-d683-41c8-8242-fc848c38ce1f"}]': finished 2024-03-28T18:00:24.933 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:00:24 smithi079 ceph-mon[35312]: osdmap e45: 7 total, 6 up, 7 in 2024-03-28T18:00:24.933 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:00:24 smithi079 ceph-mon[35312]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-28T18:00:25.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:00:25 smithi022 ceph-mon[31985]: from='client.? 172.21.15.22:0/1676402161' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-28T18:00:25.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:00:25 smithi134 ceph-mon[35037]: from='client.? 172.21.15.22:0/1676402161' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-28T18:00:25.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:00:25 smithi079 ceph-mon[35312]: from='client.? 172.21.15.22:0/1676402161' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-28T18:00:26.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:00:26 smithi022 ceph-mon[31985]: pgmap v192: 1 pgs: 1 active+clean; 577 KiB data, 191 MiB used, 536 GiB / 536 GiB avail; 81 KiB/s, 0 objects/s recovering 2024-03-28T18:00:26.802 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:00:26 smithi134 ceph-mon[35037]: pgmap v192: 1 pgs: 1 active+clean; 577 KiB data, 191 MiB used, 536 GiB / 536 GiB avail; 81 KiB/s, 0 objects/s recovering 2024-03-28T18:00:26.933 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:00:26 smithi079 ceph-mon[35312]: pgmap v192: 1 pgs: 1 active+clean; 577 KiB data, 191 MiB used, 536 GiB / 536 GiB avail; 81 KiB/s, 0 objects/s recovering 2024-03-28T18:00:28.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:00:28 smithi022 ceph-mon[31985]: pgmap v193: 1 pgs: 1 active+clean; 577 KiB data, 191 MiB used, 536 GiB / 536 GiB avail; 71 KiB/s, 0 objects/s recovering 2024-03-28T18:00:28.925 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:00:28 smithi134 ceph-mon[35037]: pgmap v193: 1 pgs: 1 active+clean; 577 KiB data, 191 MiB used, 536 GiB / 536 GiB avail; 71 KiB/s, 0 objects/s recovering 2024-03-28T18:00:28.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:00:28 smithi079 ceph-mon[35312]: pgmap v193: 1 pgs: 1 active+clean; 577 KiB data, 191 MiB used, 536 GiB / 536 GiB avail; 71 KiB/s, 0 objects/s recovering 2024-03-28T18:00:30.712 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:00:30 smithi022 ceph-mon[31985]: pgmap v194: 1 pgs: 1 active+clean; 577 KiB data, 191 MiB used, 536 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-03-28T18:00:30.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:00:30 smithi134 ceph-mon[35037]: pgmap v194: 1 pgs: 1 active+clean; 577 KiB data, 191 MiB used, 536 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-03-28T18:00:30.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:00:30 smithi079 ceph-mon[35312]: pgmap v194: 1 pgs: 1 active+clean; 577 KiB data, 191 MiB used, 536 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-03-28T18:00:31.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:00:31 smithi022 ceph-mon[31985]: pgmap v195: 1 pgs: 1 active+clean; 577 KiB data, 191 MiB used, 536 GiB / 536 GiB avail 2024-03-28T18:00:31.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:00:31 smithi134 ceph-mon[35037]: pgmap v195: 1 pgs: 1 active+clean; 577 KiB data, 191 MiB used, 536 GiB / 536 GiB avail 2024-03-28T18:00:31.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:00:31 smithi079 ceph-mon[35312]: pgmap v195: 1 pgs: 1 active+clean; 577 KiB data, 191 MiB used, 536 GiB / 536 GiB avail 2024-03-28T18:00:33.866 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:00:33 smithi022 ceph-mon[31985]: pgmap v196: 1 pgs: 1 active+clean; 577 KiB data, 191 MiB used, 536 GiB / 536 GiB avail 2024-03-28T18:00:33.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:00:33 smithi134 ceph-mon[35037]: pgmap v196: 1 pgs: 1 active+clean; 577 KiB data, 191 MiB used, 536 GiB / 536 GiB avail 2024-03-28T18:00:33.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:00:33 smithi079 ceph-mon[35312]: pgmap v196: 1 pgs: 1 active+clean; 577 KiB data, 191 MiB used, 536 GiB / 536 GiB avail 2024-03-28T18:00:34.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:00:34 smithi134 ceph-mon[35037]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-03-28T18:00:34.925 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:00:34 smithi134 ceph-mon[35037]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T18:00:34.925 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:00:34 smithi134 ceph-mon[35037]: Deploying daemon osd.6 on smithi022 2024-03-28T18:00:34.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:00:34 smithi079 ceph-mon[35312]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-03-28T18:00:34.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:00:34 smithi079 ceph-mon[35312]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T18:00:34.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:00:34 smithi079 ceph-mon[35312]: Deploying daemon osd.6 on smithi022 2024-03-28T18:00:34.961 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:00:34 smithi022 ceph-mon[31985]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-03-28T18:00:34.961 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:00:34 smithi022 ceph-mon[31985]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T18:00:34.961 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:00:34 smithi022 ceph-mon[31985]: Deploying daemon osd.6 on smithi022 2024-03-28T18:00:35.842 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:00:35 smithi022 ceph-mon[31985]: pgmap v197: 1 pgs: 1 active+clean; 577 KiB data, 191 MiB used, 536 GiB / 536 GiB avail 2024-03-28T18:00:35.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:00:35 smithi134 ceph-mon[35037]: pgmap v197: 1 pgs: 1 active+clean; 577 KiB data, 191 MiB used, 536 GiB / 536 GiB avail 2024-03-28T18:00:35.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:00:35 smithi079 ceph-mon[35312]: pgmap v197: 1 pgs: 1 active+clean; 577 KiB data, 191 MiB used, 536 GiB / 536 GiB avail 2024-03-28T18:00:37.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:00:37 smithi134 ceph-mon[35037]: pgmap v198: 1 pgs: 1 active+clean; 577 KiB data, 191 MiB used, 536 GiB / 536 GiB avail 2024-03-28T18:00:37.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:00:37 smithi079 ceph-mon[35312]: pgmap v198: 1 pgs: 1 active+clean; 577 KiB data, 191 MiB used, 536 GiB / 536 GiB avail 2024-03-28T18:00:38.031 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:00:37 smithi022 ceph-mon[31985]: pgmap v198: 1 pgs: 1 active+clean; 577 KiB data, 191 MiB used, 536 GiB / 536 GiB avail 2024-03-28T18:00:39.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:00:39 smithi022 ceph-mon[31985]: pgmap v199: 1 pgs: 1 active+clean; 577 KiB data, 191 MiB used, 536 GiB / 536 GiB avail 2024-03-28T18:00:39.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:00:39 smithi134 ceph-mon[35037]: pgmap v199: 1 pgs: 1 active+clean; 577 KiB data, 191 MiB used, 536 GiB / 536 GiB avail 2024-03-28T18:00:39.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:00:39 smithi079 ceph-mon[35312]: pgmap v199: 1 pgs: 1 active+clean; 577 KiB data, 191 MiB used, 536 GiB / 536 GiB avail 2024-03-28T18:00:41.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:00:41 smithi134 ceph-mon[35037]: pgmap v200: 1 pgs: 1 active+clean; 577 KiB data, 191 MiB used, 536 GiB / 536 GiB avail 2024-03-28T18:00:41.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:00:41 smithi079 ceph-mon[35312]: pgmap v200: 1 pgs: 1 active+clean; 577 KiB data, 191 MiB used, 536 GiB / 536 GiB avail 2024-03-28T18:00:42.032 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:00:41 smithi022 ceph-mon[31985]: pgmap v200: 1 pgs: 1 active+clean; 577 KiB data, 191 MiB used, 536 GiB / 536 GiB avail 2024-03-28T18:00:44.032 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:00:43 smithi022 ceph-mon[31985]: pgmap v201: 1 pgs: 1 active+clean; 577 KiB data, 191 MiB used, 536 GiB / 536 GiB avail 2024-03-28T18:00:44.032 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:00:43 smithi022 ceph-mon[31985]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T18:00:44.032 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:00:43 smithi022 ceph-mon[31985]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T18:00:44.032 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:00:43 smithi022 ceph-mon[31985]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T18:00:44.174 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:00:43 smithi134 ceph-mon[35037]: pgmap v201: 1 pgs: 1 active+clean; 577 KiB data, 191 MiB used, 536 GiB / 536 GiB avail 2024-03-28T18:00:44.174 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:00:43 smithi134 ceph-mon[35037]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T18:00:44.174 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:00:43 smithi134 ceph-mon[35037]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T18:00:44.175 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:00:43 smithi134 ceph-mon[35037]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T18:00:44.182 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:00:43 smithi079 ceph-mon[35312]: pgmap v201: 1 pgs: 1 active+clean; 577 KiB data, 191 MiB used, 536 GiB / 536 GiB avail 2024-03-28T18:00:44.183 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:00:43 smithi079 ceph-mon[35312]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T18:00:44.183 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:00:43 smithi079 ceph-mon[35312]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T18:00:44.183 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:00:43 smithi079 ceph-mon[35312]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T18:00:45.781 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:00:45 smithi022 ceph-mon[31985]: pgmap v202: 1 pgs: 1 active+clean; 577 KiB data, 191 MiB used, 536 GiB / 536 GiB avail 2024-03-28T18:00:46.174 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:00:45 smithi134 ceph-mon[35037]: pgmap v202: 1 pgs: 1 active+clean; 577 KiB data, 191 MiB used, 536 GiB / 536 GiB avail 2024-03-28T18:00:46.182 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:00:45 smithi079 ceph-mon[35312]: pgmap v202: 1 pgs: 1 active+clean; 577 KiB data, 191 MiB used, 536 GiB / 536 GiB avail 2024-03-28T18:00:47.913 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:00:47.912+0000 7fc08b400700 1 -- 172.21.15.22:0/1393166020 <== mgr.14152 v2:172.21.15.22:6800/4201048392 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7fc0940625d0 con 0x7fc0780a2910 2024-03-28T18:00:47.913 INFO:teuthology.orchestra.run.smithi022.stdout:Created osd(s) 6 on host 'smithi022' 2024-03-28T18:00:47.914 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:00:47 smithi022 ceph-mon[31985]: pgmap v203: 1 pgs: 1 active+clean; 577 KiB data, 191 MiB used, 536 GiB / 536 GiB avail 2024-03-28T18:00:47.914 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:00:47 smithi022 ceph-mon[31985]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T18:00:47.914 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:00:47 smithi022 ceph-mon[31985]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T18:00:47.914 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:00:47 smithi022 ceph-mon[31985]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T18:00:47.914 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:00:47 smithi022 ceph-mon[31985]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T18:00:47.914 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:00:47 smithi022 ceph-mon[31985]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T18:00:47.914 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:00:47 smithi022 ceph-mon[31985]: from='osd.6 [v2:172.21.15.22:6818/1400680396,v1:172.21.15.22:6819/1400680396]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-03-28T18:00:47.917 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:00:47.916+0000 7fc099200700 1 -- 172.21.15.22:0/1393166020 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7fc0780a2910 msgr2=0x7fc0780a4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:00:47.917 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:00:47.916+0000 7fc099200700 1 --2- 172.21.15.22:0/1393166020 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7fc0780a2910 0x7fc0780a4dd0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fc07c00b5c0 tx=0x7fc07c005fb0 comp rx=0 tx=0).stop 2024-03-28T18:00:47.917 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:00:47.916+0000 7fc099200700 1 -- 172.21.15.22:0/1393166020 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc094102cb0 msgr2=0x7fc09419a650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:00:47.917 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:00:47.916+0000 7fc099200700 1 --2- 172.21.15.22:0/1393166020 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc094102cb0 0x7fc09419a650 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fc08400d980 tx=0x7fc08400dd40 comp rx=0 tx=0).stop 2024-03-28T18:00:47.917 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:00:47.917+0000 7fc099200700 1 -- 172.21.15.22:0/1393166020 shutdown_connections 2024-03-28T18:00:47.917 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:00:47.917+0000 7fc099200700 1 --2- 172.21.15.22:0/1393166020 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc094107f10 0x7fc0941a1c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:00:47.917 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:00:47.917+0000 7fc099200700 1 --2- 172.21.15.22:0/1393166020 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc094102cb0 0x7fc09419a650 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:00:47.917 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:00:47.917+0000 7fc099200700 1 --2- 172.21.15.22:0/1393166020 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7fc0780a2910 0x7fc0780a4dd0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:00:47.917 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:00:47.917+0000 7fc099200700 1 --2- 172.21.15.22:0/1393166020 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc0941055e0 0x7fc09419ab90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:00:47.917 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:00:47.917+0000 7fc099200700 1 -- 172.21.15.22:0/1393166020 >> 172.21.15.22:0/1393166020 conn(0x7fc0940fc660 msgr2=0x7fc0941024d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:00:47.917 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:00:47.917+0000 7fc099200700 1 -- 172.21.15.22:0/1393166020 shutdown_connections 2024-03-28T18:00:47.917 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:00:47.917+0000 7fc099200700 1 -- 172.21.15.22:0/1393166020 wait complete. 2024-03-28T18:00:47.925 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:00:47 smithi134 ceph-mon[35037]: pgmap v203: 1 pgs: 1 active+clean; 577 KiB data, 191 MiB used, 536 GiB / 536 GiB avail 2024-03-28T18:00:47.925 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:00:47 smithi134 ceph-mon[35037]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T18:00:47.925 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:00:47 smithi134 ceph-mon[35037]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T18:00:47.925 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:00:47 smithi134 ceph-mon[35037]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T18:00:47.925 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:00:47 smithi134 ceph-mon[35037]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T18:00:47.925 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:00:47 smithi134 ceph-mon[35037]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T18:00:47.925 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:00:47 smithi134 ceph-mon[35037]: from='osd.6 [v2:172.21.15.22:6818/1400680396,v1:172.21.15.22:6819/1400680396]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-03-28T18:00:47.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:00:47 smithi079 ceph-mon[35312]: pgmap v203: 1 pgs: 1 active+clean; 577 KiB data, 191 MiB used, 536 GiB / 536 GiB avail 2024-03-28T18:00:47.933 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:00:47 smithi079 ceph-mon[35312]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T18:00:47.933 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:00:47 smithi079 ceph-mon[35312]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T18:00:47.933 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:00:47 smithi079 ceph-mon[35312]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T18:00:47.933 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:00:47 smithi079 ceph-mon[35312]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T18:00:47.933 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:00:47 smithi079 ceph-mon[35312]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T18:00:47.933 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:00:47 smithi079 ceph-mon[35312]: from='osd.6 [v2:172.21.15.22:6818/1400680396,v1:172.21.15.22:6819/1400680396]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-03-28T18:00:48.600 DEBUG:teuthology.orchestra.run.smithi022:osd.6> sudo journalctl -f -n 0 -u ceph-fd220748-ed2b-11ee-95d0-87774f69a715@osd.6.service 2024-03-28T18:00:48.605 INFO:tasks.cephadm:Deploying osd.7 on smithi079 with /dev/vg_nvme/lv_2... 2024-03-28T18:00:48.605 DEBUG:teuthology.orchestra.run.smithi079:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fd220748-ed2b-11ee-95d0-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-03-28T18:00:48.858 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:00:48 smithi022 ceph-mon[31985]: from='osd.6 [v2:172.21.15.22:6818/1400680396,v1:172.21.15.22:6819/1400680396]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-03-28T18:00:48.859 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:00:48 smithi022 ceph-mon[31985]: osdmap e46: 7 total, 6 up, 7 in 2024-03-28T18:00:48.859 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:00:48 smithi022 ceph-mon[31985]: from='osd.6 [v2:172.21.15.22:6818/1400680396,v1:172.21.15.22:6819/1400680396]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi022", "root=default"]}]: dispatch 2024-03-28T18:00:48.859 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:00:48 smithi022 ceph-mon[31985]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-28T18:00:48.859 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:00:48 smithi022 ceph-mon[31985]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T18:00:48.859 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:00:48 smithi022 ceph-mon[31985]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T18:00:48.859 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:00:48 smithi022 ceph-mon[31985]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T18:00:48.925 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:00:48 smithi134 ceph-mon[35037]: from='osd.6 [v2:172.21.15.22:6818/1400680396,v1:172.21.15.22:6819/1400680396]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-03-28T18:00:48.925 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:00:48 smithi134 ceph-mon[35037]: osdmap e46: 7 total, 6 up, 7 in 2024-03-28T18:00:48.925 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:00:48 smithi134 ceph-mon[35037]: from='osd.6 [v2:172.21.15.22:6818/1400680396,v1:172.21.15.22:6819/1400680396]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi022", "root=default"]}]: dispatch 2024-03-28T18:00:48.925 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:00:48 smithi134 ceph-mon[35037]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-28T18:00:48.925 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:00:48 smithi134 ceph-mon[35037]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T18:00:48.925 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:00:48 smithi134 ceph-mon[35037]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T18:00:48.925 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:00:48 smithi134 ceph-mon[35037]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T18:00:48.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:00:48 smithi079 ceph-mon[35312]: from='osd.6 [v2:172.21.15.22:6818/1400680396,v1:172.21.15.22:6819/1400680396]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-03-28T18:00:48.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:00:48 smithi079 ceph-mon[35312]: osdmap e46: 7 total, 6 up, 7 in 2024-03-28T18:00:48.933 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:00:48 smithi079 ceph-mon[35312]: from='osd.6 [v2:172.21.15.22:6818/1400680396,v1:172.21.15.22:6819/1400680396]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi022", "root=default"]}]: dispatch 2024-03-28T18:00:48.933 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:00:48 smithi079 ceph-mon[35312]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-28T18:00:48.933 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:00:48 smithi079 ceph-mon[35312]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T18:00:48.933 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:00:48 smithi079 ceph-mon[35312]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T18:00:48.933 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:00:48 smithi079 ceph-mon[35312]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T18:00:49.044 INFO:teuthology.orchestra.run.smithi079.stderr:Inferring config /var/lib/ceph/fd220748-ed2b-11ee-95d0-87774f69a715/mon.b/config 2024-03-28T18:00:49.877 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:00:49 smithi022 ceph-mon[31985]: pgmap v205: 1 pgs: 1 active+clean; 577 KiB data, 191 MiB used, 536 GiB / 536 GiB avail 2024-03-28T18:00:49.878 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:00:49 smithi022 ceph-mon[31985]: from='osd.6 [v2:172.21.15.22:6818/1400680396,v1:172.21.15.22:6819/1400680396]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi022", "root=default"]}]': finished 2024-03-28T18:00:49.878 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:00:49 smithi022 ceph-mon[31985]: osdmap e47: 7 total, 6 up, 7 in 2024-03-28T18:00:49.878 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:00:49 smithi022 ceph-mon[31985]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-28T18:00:49.878 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:00:49 smithi022 ceph-mon[31985]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-28T18:00:49.878 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:00:49 smithi022 ceph-mon[31985]: from='osd.6 [v2:172.21.15.22:6818/1400680396,v1:172.21.15.22:6819/1400680396]' entity='osd.6' 2024-03-28T18:00:49.878 INFO:journalctl@ceph.osd.6.smithi022.stdout:Mar 28 18:00:49 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-osd-6[60753]: 2024-03-28T18:00:49.560+0000 7fb988200700 -1 osd.6 0 waiting for initial osdmap 2024-03-28T18:00:49.878 INFO:journalctl@ceph.osd.6.smithi022.stdout:Mar 28 18:00:49 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-osd-6[60753]: 2024-03-28T18:00:49.586+0000 7fb97e000700 -1 osd.6 47 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-28T18:00:49.890 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:00:49 smithi079 ceph-mon[35312]: pgmap v205: 1 pgs: 1 active+clean; 577 KiB data, 191 MiB used, 536 GiB / 536 GiB avail 2024-03-28T18:00:49.891 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:00:49 smithi079 ceph-mon[35312]: from='osd.6 [v2:172.21.15.22:6818/1400680396,v1:172.21.15.22:6819/1400680396]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi022", "root=default"]}]': finished 2024-03-28T18:00:49.891 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:00:49 smithi079 ceph-mon[35312]: osdmap e47: 7 total, 6 up, 7 in 2024-03-28T18:00:49.891 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:00:49 smithi079 ceph-mon[35312]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-28T18:00:49.891 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:00:49 smithi079 ceph-mon[35312]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-28T18:00:49.891 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:00:49 smithi079 ceph-mon[35312]: from='osd.6 [v2:172.21.15.22:6818/1400680396,v1:172.21.15.22:6819/1400680396]' entity='osd.6' 2024-03-28T18:00:49.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:00:49 smithi134 ceph-mon[35037]: pgmap v205: 1 pgs: 1 active+clean; 577 KiB data, 191 MiB used, 536 GiB / 536 GiB avail 2024-03-28T18:00:49.925 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:00:49 smithi134 ceph-mon[35037]: from='osd.6 [v2:172.21.15.22:6818/1400680396,v1:172.21.15.22:6819/1400680396]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi022", "root=default"]}]': finished 2024-03-28T18:00:49.925 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:00:49 smithi134 ceph-mon[35037]: osdmap e47: 7 total, 6 up, 7 in 2024-03-28T18:00:49.925 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:00:49 smithi134 ceph-mon[35037]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-28T18:00:49.925 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:00:49 smithi134 ceph-mon[35037]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-28T18:00:49.925 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:00:49 smithi134 ceph-mon[35037]: from='osd.6 [v2:172.21.15.22:6818/1400680396,v1:172.21.15.22:6819/1400680396]' entity='osd.6' 2024-03-28T18:00:50.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:00:50 smithi022 ceph-mon[31985]: purged_snaps scrub starts 2024-03-28T18:00:50.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:00:50 smithi022 ceph-mon[31985]: purged_snaps scrub ok 2024-03-28T18:00:50.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:00:50 smithi022 ceph-mon[31985]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-28T18:00:50.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:00:50 smithi022 ceph-mon[31985]: osd.6 [v2:172.21.15.22:6818/1400680396,v1:172.21.15.22:6819/1400680396] boot 2024-03-28T18:00:50.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:00:50 smithi022 ceph-mon[31985]: osdmap e48: 7 total, 7 up, 7 in 2024-03-28T18:00:50.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:00:50 smithi022 ceph-mon[31985]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-28T18:00:50.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:00:50 smithi134 ceph-mon[35037]: purged_snaps scrub starts 2024-03-28T18:00:50.925 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:00:50 smithi134 ceph-mon[35037]: purged_snaps scrub ok 2024-03-28T18:00:50.925 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:00:50 smithi134 ceph-mon[35037]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-28T18:00:50.925 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:00:50 smithi134 ceph-mon[35037]: osd.6 [v2:172.21.15.22:6818/1400680396,v1:172.21.15.22:6819/1400680396] boot 2024-03-28T18:00:50.925 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:00:50 smithi134 ceph-mon[35037]: osdmap e48: 7 total, 7 up, 7 in 2024-03-28T18:00:50.925 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:00:50 smithi134 ceph-mon[35037]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-28T18:00:50.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:00:50 smithi079 ceph-mon[35312]: purged_snaps scrub starts 2024-03-28T18:00:50.933 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:00:50 smithi079 ceph-mon[35312]: purged_snaps scrub ok 2024-03-28T18:00:50.933 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:00:50 smithi079 ceph-mon[35312]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-28T18:00:50.933 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:00:50 smithi079 ceph-mon[35312]: osd.6 [v2:172.21.15.22:6818/1400680396,v1:172.21.15.22:6819/1400680396] boot 2024-03-28T18:00:50.933 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:00:50 smithi079 ceph-mon[35312]: osdmap e48: 7 total, 7 up, 7 in 2024-03-28T18:00:50.933 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:00:50 smithi079 ceph-mon[35312]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-28T18:00:51.835 INFO:teuthology.orchestra.run.smithi079.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-03-28T18:00:51.835 INFO:teuthology.orchestra.run.smithi079.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-03-28T18:00:51.835 INFO:teuthology.orchestra.run.smithi079.stderr: stderr: 10+0 records in 2024-03-28T18:00:51.835 INFO:teuthology.orchestra.run.smithi079.stderr:10+0 records out 2024-03-28T18:00:51.835 INFO:teuthology.orchestra.run.smithi079.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0346073 s, 303 MB/s 2024-03-28T18:00:51.835 INFO:teuthology.orchestra.run.smithi079.stderr:--> Zapping successful for: 2024-03-28T18:00:51.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:00:51 smithi134 ceph-mon[35037]: pgmap v207: 1 pgs: 1 active+clean; 577 KiB data, 191 MiB used, 536 GiB / 536 GiB avail 2024-03-28T18:00:51.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:00:51 smithi079 ceph-mon[35312]: pgmap v207: 1 pgs: 1 active+clean; 577 KiB data, 191 MiB used, 536 GiB / 536 GiB avail 2024-03-28T18:00:52.032 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:00:51 smithi022 ceph-mon[31985]: pgmap v207: 1 pgs: 1 active+clean; 577 KiB data, 191 MiB used, 536 GiB / 536 GiB avail 2024-03-28T18:00:52.613 DEBUG:teuthology.orchestra.run.smithi079:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fd220748-ed2b-11ee-95d0-87774f69a715 -- ceph orch daemon add osd smithi079:vg_nvme/lv_2 2024-03-28T18:00:52.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:00:52 smithi022 ceph-mon[31985]: osdmap e49: 7 total, 7 up, 7 in 2024-03-28T18:00:52.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:00:52 smithi079 ceph-mon[35312]: osdmap e49: 7 total, 7 up, 7 in 2024-03-28T18:00:53.032 INFO:teuthology.orchestra.run.smithi079.stderr:Inferring config /var/lib/ceph/fd220748-ed2b-11ee-95d0-87774f69a715/mon.b/config 2024-03-28T18:00:53.174 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:00:52 smithi134 ceph-mon[35037]: osdmap e49: 7 total, 7 up, 7 in 2024-03-28T18:00:54.563 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:00:54 smithi079 ceph-mon[35312]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 226 MiB used, 626 GiB / 626 GiB avail 2024-03-28T18:00:54.563 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:00:54 smithi079 ceph-mon[35312]: Detected new or changed devices on smithi022 2024-03-28T18:00:54.563 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:00:54 smithi079 ceph-mon[35312]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T18:00:54.563 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:00:54 smithi079 ceph-mon[35312]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T18:00:54.564 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:00:54 smithi079 ceph-mon[35312]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-28T18:00:54.564 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:00:54 smithi079 ceph-mon[35312]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-28T18:00:54.564 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:00:54 smithi079 ceph-mon[35312]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-03-28T18:00:54.564 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:00:54 smithi079 ceph-mon[35312]: Adjusting osd_memory_target on smithi022 to 5698M 2024-03-28T18:00:54.564 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:00:54 smithi079 ceph-mon[35312]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T18:00:54.564 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:00:54 smithi079 ceph-mon[35312]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T18:00:54.564 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:00:54 smithi079 ceph-mon[35312]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T18:00:54.564 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:00:54 smithi079 ceph-mon[35312]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T18:00:54.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:00:54 smithi134 ceph-mon[35037]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 226 MiB used, 626 GiB / 626 GiB avail 2024-03-28T18:00:54.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:00:54 smithi134 ceph-mon[35037]: Detected new or changed devices on smithi022 2024-03-28T18:00:54.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:00:54 smithi134 ceph-mon[35037]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T18:00:54.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:00:54 smithi134 ceph-mon[35037]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T18:00:54.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:00:54 smithi134 ceph-mon[35037]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-28T18:00:54.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:00:54 smithi134 ceph-mon[35037]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-28T18:00:54.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:00:54 smithi134 ceph-mon[35037]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-03-28T18:00:54.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:00:54 smithi134 ceph-mon[35037]: Adjusting osd_memory_target on smithi022 to 5698M 2024-03-28T18:00:54.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:00:54 smithi134 ceph-mon[35037]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T18:00:54.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:00:54 smithi134 ceph-mon[35037]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T18:00:54.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:00:54 smithi134 ceph-mon[35037]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T18:00:54.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:00:54 smithi134 ceph-mon[35037]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T18:00:54.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:00:54 smithi022 ceph-mon[31985]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 226 MiB used, 626 GiB / 626 GiB avail 2024-03-28T18:00:54.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:00:54 smithi022 ceph-mon[31985]: Detected new or changed devices on smithi022 2024-03-28T18:00:54.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:00:54 smithi022 ceph-mon[31985]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T18:00:54.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:00:54 smithi022 ceph-mon[31985]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T18:00:54.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:00:54 smithi022 ceph-mon[31985]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-28T18:00:54.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:00:54 smithi022 ceph-mon[31985]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-28T18:00:54.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:00:54 smithi022 ceph-mon[31985]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-03-28T18:00:54.783 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:00:54 smithi022 ceph-mon[31985]: Adjusting osd_memory_target on smithi022 to 5698M 2024-03-28T18:00:54.783 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:00:54 smithi022 ceph-mon[31985]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T18:00:54.783 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:00:54 smithi022 ceph-mon[31985]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T18:00:54.783 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:00:54 smithi022 ceph-mon[31985]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T18:00:54.783 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:00:54 smithi022 ceph-mon[31985]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T18:00:55.090 INFO:teuthology.orchestra.run.smithi079.stderr:2024-03-28T18:00:55.089+0000 7fca24200700 1 -- 172.21.15.79:0/1445130336 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fca1c105fb0 msgr2=0x7fca1c10c860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:00:55.090 INFO:teuthology.orchestra.run.smithi079.stderr:2024-03-28T18:00:55.089+0000 7fca24200700 1 --2- 172.21.15.79:0/1445130336 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fca1c105fb0 0x7fca1c10c860 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fca1400b0d0 tx=0x7fca1400b3e0 comp rx=0 tx=0).stop 2024-03-28T18:00:55.090 INFO:teuthology.orchestra.run.smithi079.stderr:2024-03-28T18:00:55.089+0000 7fca24200700 1 -- 172.21.15.79:0/1445130336 shutdown_connections 2024-03-28T18:00:55.090 INFO:teuthology.orchestra.run.smithi079.stderr:2024-03-28T18:00:55.089+0000 7fca24200700 1 --2- 172.21.15.79:0/1445130336 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fca1c105fb0 0x7fca1c10c860 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:00:55.090 INFO:teuthology.orchestra.run.smithi079.stderr:2024-03-28T18:00:55.089+0000 7fca24200700 1 --2- 172.21.15.79:0/1445130336 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fca1c104410 0x7fca1c104830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:00:55.090 INFO:teuthology.orchestra.run.smithi079.stderr:2024-03-28T18:00:55.089+0000 7fca24200700 1 --2- 172.21.15.79:0/1445130336 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fca1c105610 0x7fca1c105a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:00:55.090 INFO:teuthology.orchestra.run.smithi079.stderr:2024-03-28T18:00:55.089+0000 7fca24200700 1 -- 172.21.15.79:0/1445130336 >> 172.21.15.79:0/1445130336 conn(0x7fca1c0ff9b0 msgr2=0x7fca1c101df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:00:55.090 INFO:teuthology.orchestra.run.smithi079.stderr:2024-03-28T18:00:55.089+0000 7fca24200700 1 -- 172.21.15.79:0/1445130336 shutdown_connections 2024-03-28T18:00:55.090 INFO:teuthology.orchestra.run.smithi079.stderr:2024-03-28T18:00:55.089+0000 7fca24200700 1 -- 172.21.15.79:0/1445130336 wait complete. 2024-03-28T18:00:55.092 INFO:teuthology.orchestra.run.smithi079.stderr:2024-03-28T18:00:55.090+0000 7fca24200700 1 Processor -- start 2024-03-28T18:00:55.092 INFO:teuthology.orchestra.run.smithi079.stderr:2024-03-28T18:00:55.091+0000 7fca24200700 1 -- start start 2024-03-28T18:00:55.092 INFO:teuthology.orchestra.run.smithi079.stderr:2024-03-28T18:00:55.091+0000 7fca24200700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fca1c104410 0x7fca1c19a680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:00:55.092 INFO:teuthology.orchestra.run.smithi079.stderr:2024-03-28T18:00:55.091+0000 7fca24200700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fca1c105610 0x7fca1c19abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:00:55.092 INFO:teuthology.orchestra.run.smithi079.stderr:2024-03-28T18:00:55.091+0000 7fca24200700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fca1c105fb0 0x7fca1c1a1c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:00:55.092 INFO:teuthology.orchestra.run.smithi079.stderr:2024-03-28T18:00:55.091+0000 7fca24200700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca1c110090 con 0x7fca1c105610 2024-03-28T18:00:55.093 INFO:teuthology.orchestra.run.smithi079.stderr:2024-03-28T18:00:55.091+0000 7fca24200700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca1c10ff10 con 0x7fca1c104410 2024-03-28T18:00:55.093 INFO:teuthology.orchestra.run.smithi079.stderr:2024-03-28T18:00:55.091+0000 7fca24200700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca1c110210 con 0x7fca1c105fb0 2024-03-28T18:00:55.093 INFO:teuthology.orchestra.run.smithi079.stderr:2024-03-28T18:00:55.091+0000 7fca21800700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fca1c104410 0x7fca1c19a680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:00:55.093 INFO:teuthology.orchestra.run.smithi079.stderr:2024-03-28T18:00:55.091+0000 7fca21800700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fca1c104410 0x7fca1c19a680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:39446/0 (socket says 172.21.15.79:39446) 2024-03-28T18:00:55.093 INFO:teuthology.orchestra.run.smithi079.stderr:2024-03-28T18:00:55.091+0000 7fca21800700 1 -- 172.21.15.79:0/3893423916 learned_addr learned my addr 172.21.15.79:0/3893423916 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-03-28T18:00:55.093 INFO:teuthology.orchestra.run.smithi079.stderr:2024-03-28T18:00:55.091+0000 7fca20e00700 1 --2- 172.21.15.79:0/3893423916 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fca1c105610 0x7fca1c19abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:00:55.093 INFO:teuthology.orchestra.run.smithi079.stderr:2024-03-28T18:00:55.092+0000 7fca22200700 1 --2- 172.21.15.79:0/3893423916 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fca1c105fb0 0x7fca1c1a1c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:00:55.093 INFO:teuthology.orchestra.run.smithi079.stderr:2024-03-28T18:00:55.092+0000 7fca21800700 1 -- 172.21.15.79:0/3893423916 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fca1c105fb0 msgr2=0x7fca1c1a1c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:00:55.093 INFO:teuthology.orchestra.run.smithi079.stderr:2024-03-28T18:00:55.092+0000 7fca21800700 1 --2- 172.21.15.79:0/3893423916 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fca1c105fb0 0x7fca1c1a1c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:00:55.093 INFO:teuthology.orchestra.run.smithi079.stderr:2024-03-28T18:00:55.092+0000 7fca21800700 1 -- 172.21.15.79:0/3893423916 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fca1c105610 msgr2=0x7fca1c19abc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:00:55.093 INFO:teuthology.orchestra.run.smithi079.stderr:2024-03-28T18:00:55.092+0000 7fca21800700 1 --2- 172.21.15.79:0/3893423916 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fca1c105610 0x7fca1c19abc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:00:55.093 INFO:teuthology.orchestra.run.smithi079.stderr:2024-03-28T18:00:55.092+0000 7fca21800700 1 -- 172.21.15.79:0/3893423916 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fca1c1a22e0 con 0x7fca1c104410 2024-03-28T18:00:55.093 INFO:teuthology.orchestra.run.smithi079.stderr:2024-03-28T18:00:55.092+0000 7fca20e00700 1 --2- 172.21.15.79:0/3893423916 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fca1c105610 0x7fca1c19abc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-28T18:00:55.094 INFO:teuthology.orchestra.run.smithi079.stderr:2024-03-28T18:00:55.092+0000 7fca21800700 1 --2- 172.21.15.79:0/3893423916 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fca1c104410 0x7fca1c19a680 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fca1000c970 tx=0x7fca1000cd30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:00:55.094 INFO:teuthology.orchestra.run.smithi079.stderr:2024-03-28T18:00:55.092+0000 7fca1a000700 1 -- 172.21.15.79:0/3893423916 <== mon.2 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca10004220 con 0x7fca1c104410 2024-03-28T18:00:55.095 INFO:teuthology.orchestra.run.smithi079.stderr:2024-03-28T18:00:55.092+0000 7fca1a000700 1 -- 172.21.15.79:0/3893423916 <== mon.2 v2:172.21.15.79:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7fca10007d70 con 0x7fca1c104410 2024-03-28T18:00:55.095 INFO:teuthology.orchestra.run.smithi079.stderr:2024-03-28T18:00:55.092+0000 7fca1a000700 1 -- 172.21.15.79:0/3893423916 <== mon.2 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca10003e10 con 0x7fca1c104410 2024-03-28T18:00:55.095 INFO:teuthology.orchestra.run.smithi079.stderr:2024-03-28T18:00:55.093+0000 7fca24200700 1 -- 172.21.15.79:0/3893423916 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fca1c1a25d0 con 0x7fca1c104410 2024-03-28T18:00:55.095 INFO:teuthology.orchestra.run.smithi079.stderr:2024-03-28T18:00:55.093+0000 7fca24200700 1 -- 172.21.15.79:0/3893423916 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fca1c1a2ac0 con 0x7fca1c104410 2024-03-28T18:00:55.095 INFO:teuthology.orchestra.run.smithi079.stderr:2024-03-28T18:00:55.094+0000 7fca24200700 1 -- 172.21.15.79:0/3893423916 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fca1c068e50 con 0x7fca1c104410 2024-03-28T18:00:55.097 INFO:teuthology.orchestra.run.smithi079.stderr:2024-03-28T18:00:55.096+0000 7fca1a000700 1 -- 172.21.15.79:0/3893423916 <== mon.2 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 16) v1 ==== 137290+0+0 (secure 0 0 0) 0x7fca100078a0 con 0x7fca1c104410 2024-03-28T18:00:55.098 INFO:teuthology.orchestra.run.smithi079.stderr:2024-03-28T18:00:55.098+0000 7fca1a000700 1 --2- 172.21.15.79:0/3893423916 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7fca040a2890 0x7fca040a4d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:00:55.098 INFO:teuthology.orchestra.run.smithi079.stderr:2024-03-28T18:00:55.098+0000 7fca1a000700 1 -- 172.21.15.79:0/3893423916 <== mon.2 v2:172.21.15.79:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 4965+0+0 (secure 0 0 0) 0x7fca100cdd80 con 0x7fca1c104410 2024-03-28T18:00:55.098 INFO:teuthology.orchestra.run.smithi079.stderr:2024-03-28T18:00:55.098+0000 7fca20e00700 1 --2- 172.21.15.79:0/3893423916 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7fca040a2890 0x7fca040a4d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:00:55.099 INFO:teuthology.orchestra.run.smithi079.stderr:2024-03-28T18:00:55.099+0000 7fca20e00700 1 --2- 172.21.15.79:0/3893423916 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7fca040a2890 0x7fca040a4d50 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fca080099d0 tx=0x7fca08008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:00:55.102 INFO:teuthology.orchestra.run.smithi079.stderr:2024-03-28T18:00:55.102+0000 7fca1a000700 1 -- 172.21.15.79:0/3893423916 <== mon.2 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fca10014030 con 0x7fca1c104410 2024-03-28T18:00:55.335 INFO:teuthology.orchestra.run.smithi079.stderr:2024-03-28T18:00:55.334+0000 7fca24200700 1 -- 172.21.15.79:0/3893423916 --> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi079:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) v1 -- 0x7fca1c0625d0 con 0x7fca040a2890 2024-03-28T18:00:56.579 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:00:56 smithi079 ceph-mon[35312]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 226 MiB used, 626 GiB / 626 GiB avail 2024-03-28T18:00:56.579 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:00:56 smithi079 ceph-mon[35312]: from='client.34283 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi079:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-03-28T18:00:56.579 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:00:56 smithi079 ceph-mon[35312]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-28T18:00:56.579 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:00:56 smithi079 ceph-mon[35312]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-28T18:00:56.579 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:00:56 smithi079 ceph-mon[35312]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T18:00:56.674 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:00:56 smithi134 ceph-mon[35037]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 226 MiB used, 626 GiB / 626 GiB avail 2024-03-28T18:00:56.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:00:56 smithi134 ceph-mon[35037]: from='client.34283 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi079:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-03-28T18:00:56.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:00:56 smithi134 ceph-mon[35037]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-28T18:00:56.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:00:56 smithi134 ceph-mon[35037]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-28T18:00:56.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:00:56 smithi134 ceph-mon[35037]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T18:00:56.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:00:56 smithi022 ceph-mon[31985]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 226 MiB used, 626 GiB / 626 GiB avail 2024-03-28T18:00:56.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:00:56 smithi022 ceph-mon[31985]: from='client.34283 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi079:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-03-28T18:00:56.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:00:56 smithi022 ceph-mon[31985]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-28T18:00:56.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:00:56 smithi022 ceph-mon[31985]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-28T18:00:56.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:00:56 smithi022 ceph-mon[31985]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T18:00:58.674 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:00:58 smithi134 ceph-mon[35037]: pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 226 MiB used, 626 GiB / 626 GiB avail 2024-03-28T18:00:58.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:00:58 smithi079 ceph-mon[35312]: pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 226 MiB used, 626 GiB / 626 GiB avail 2024-03-28T18:00:58.781 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:00:58 smithi022 ceph-mon[31985]: pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 226 MiB used, 626 GiB / 626 GiB avail 2024-03-28T18:00:59.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:00:59 smithi134 ceph-mon[35037]: from='client.? 172.21.15.79:0/4106660292' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "504eb4a8-6731-45ad-932c-1b38ef157871"}]: dispatch 2024-03-28T18:00:59.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:00:59 smithi134 ceph-mon[35037]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "504eb4a8-6731-45ad-932c-1b38ef157871"}]: dispatch 2024-03-28T18:00:59.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:00:59 smithi134 ceph-mon[35037]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "504eb4a8-6731-45ad-932c-1b38ef157871"}]': finished 2024-03-28T18:00:59.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:00:59 smithi134 ceph-mon[35037]: osdmap e50: 8 total, 7 up, 8 in 2024-03-28T18:00:59.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:00:59 smithi134 ceph-mon[35037]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-28T18:00:59.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:00:59 smithi079 ceph-mon[35312]: from='client.? 172.21.15.79:0/4106660292' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "504eb4a8-6731-45ad-932c-1b38ef157871"}]: dispatch 2024-03-28T18:00:59.683 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:00:59 smithi079 ceph-mon[35312]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "504eb4a8-6731-45ad-932c-1b38ef157871"}]: dispatch 2024-03-28T18:00:59.683 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:00:59 smithi079 ceph-mon[35312]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "504eb4a8-6731-45ad-932c-1b38ef157871"}]': finished 2024-03-28T18:00:59.683 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:00:59 smithi079 ceph-mon[35312]: osdmap e50: 8 total, 7 up, 8 in 2024-03-28T18:00:59.683 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:00:59 smithi079 ceph-mon[35312]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-28T18:00:59.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:00:59 smithi022 ceph-mon[31985]: from='client.? 172.21.15.79:0/4106660292' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "504eb4a8-6731-45ad-932c-1b38ef157871"}]: dispatch 2024-03-28T18:00:59.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:00:59 smithi022 ceph-mon[31985]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "504eb4a8-6731-45ad-932c-1b38ef157871"}]: dispatch 2024-03-28T18:00:59.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:00:59 smithi022 ceph-mon[31985]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "504eb4a8-6731-45ad-932c-1b38ef157871"}]': finished 2024-03-28T18:00:59.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:00:59 smithi022 ceph-mon[31985]: osdmap e50: 8 total, 7 up, 8 in 2024-03-28T18:00:59.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:00:59 smithi022 ceph-mon[31985]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-28T18:01:00.674 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:01:00 smithi134 ceph-mon[35037]: pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 226 MiB used, 626 GiB / 626 GiB avail 2024-03-28T18:01:00.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:01:00 smithi134 ceph-mon[35037]: from='client.? 172.21.15.79:0/4255520725' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-28T18:01:00.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:01:00 smithi079 ceph-mon[35312]: pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 226 MiB used, 626 GiB / 626 GiB avail 2024-03-28T18:01:00.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:01:00 smithi079 ceph-mon[35312]: from='client.? 172.21.15.79:0/4255520725' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-28T18:01:00.781 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:01:00 smithi022 ceph-mon[31985]: pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 226 MiB used, 626 GiB / 626 GiB avail 2024-03-28T18:01:00.781 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:01:00 smithi022 ceph-mon[31985]: from='client.? 172.21.15.79:0/4255520725' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-28T18:01:02.653 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:01:02 smithi022 ceph-mon[31985]: pgmap v215: 1 pgs: 1 active+clean; 577 KiB data, 226 MiB used, 626 GiB / 626 GiB avail 2024-03-28T18:01:02.674 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:01:02 smithi134 ceph-mon[35037]: pgmap v215: 1 pgs: 1 active+clean; 577 KiB data, 226 MiB used, 626 GiB / 626 GiB avail 2024-03-28T18:01:02.683 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:01:02 smithi079 ceph-mon[35312]: pgmap v215: 1 pgs: 1 active+clean; 577 KiB data, 226 MiB used, 626 GiB / 626 GiB avail 2024-03-28T18:01:04.674 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:01:04 smithi134 ceph-mon[35037]: pgmap v216: 1 pgs: 1 active+clean; 577 KiB data, 226 MiB used, 626 GiB / 626 GiB avail 2024-03-28T18:01:04.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:01:04 smithi079 ceph-mon[35312]: pgmap v216: 1 pgs: 1 active+clean; 577 KiB data, 226 MiB used, 626 GiB / 626 GiB avail 2024-03-28T18:01:04.781 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:01:04 smithi022 ceph-mon[31985]: pgmap v216: 1 pgs: 1 active+clean; 577 KiB data, 226 MiB used, 626 GiB / 626 GiB avail 2024-03-28T18:01:06.674 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:01:06 smithi134 ceph-mon[35037]: pgmap v217: 1 pgs: 1 active+clean; 577 KiB data, 226 MiB used, 626 GiB / 626 GiB avail 2024-03-28T18:01:06.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:01:06 smithi079 ceph-mon[35312]: pgmap v217: 1 pgs: 1 active+clean; 577 KiB data, 226 MiB used, 626 GiB / 626 GiB avail 2024-03-28T18:01:06.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:01:06 smithi022 ceph-mon[31985]: pgmap v217: 1 pgs: 1 active+clean; 577 KiB data, 226 MiB used, 626 GiB / 626 GiB avail 2024-03-28T18:01:08.605 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:01:08 smithi079 ceph-mon[35312]: pgmap v218: 1 pgs: 1 active+clean; 577 KiB data, 226 MiB used, 626 GiB / 626 GiB avail 2024-03-28T18:01:08.605 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:01:08 smithi079 ceph-mon[35312]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-03-28T18:01:08.605 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:01:08 smithi079 ceph-mon[35312]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T18:01:08.674 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:01:08 smithi134 ceph-mon[35037]: pgmap v218: 1 pgs: 1 active+clean; 577 KiB data, 226 MiB used, 626 GiB / 626 GiB avail 2024-03-28T18:01:08.674 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:01:08 smithi134 ceph-mon[35037]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-03-28T18:01:08.674 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:01:08 smithi134 ceph-mon[35037]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T18:01:08.781 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:01:08 smithi022 ceph-mon[31985]: pgmap v218: 1 pgs: 1 active+clean; 577 KiB data, 226 MiB used, 626 GiB / 626 GiB avail 2024-03-28T18:01:08.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:01:08 smithi022 ceph-mon[31985]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-03-28T18:01:08.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:01:08 smithi022 ceph-mon[31985]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T18:01:09.489 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:01:09 smithi079 ceph-mon[35312]: Deploying daemon osd.7 on smithi079 2024-03-28T18:01:09.674 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:01:09 smithi134 ceph-mon[35037]: Deploying daemon osd.7 on smithi079 2024-03-28T18:01:09.781 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:01:09 smithi022 ceph-mon[31985]: Deploying daemon osd.7 on smithi079 2024-03-28T18:01:10.674 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:01:10 smithi134 ceph-mon[35037]: pgmap v219: 1 pgs: 1 active+clean; 577 KiB data, 226 MiB used, 626 GiB / 626 GiB avail 2024-03-28T18:01:10.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:01:10 smithi079 ceph-mon[35312]: pgmap v219: 1 pgs: 1 active+clean; 577 KiB data, 226 MiB used, 626 GiB / 626 GiB avail 2024-03-28T18:01:10.781 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:01:10 smithi022 ceph-mon[31985]: pgmap v219: 1 pgs: 1 active+clean; 577 KiB data, 226 MiB used, 626 GiB / 626 GiB avail 2024-03-28T18:01:12.600 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:01:12 smithi079 ceph-mon[35312]: pgmap v220: 1 pgs: 1 active+clean; 577 KiB data, 226 MiB used, 626 GiB / 626 GiB avail 2024-03-28T18:01:12.674 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:01:12 smithi134 ceph-mon[35037]: pgmap v220: 1 pgs: 1 active+clean; 577 KiB data, 226 MiB used, 626 GiB / 626 GiB avail 2024-03-28T18:01:12.781 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:01:12 smithi022 ceph-mon[31985]: pgmap v220: 1 pgs: 1 active+clean; 577 KiB data, 226 MiB used, 626 GiB / 626 GiB avail 2024-03-28T18:01:14.674 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:01:14 smithi134 ceph-mon[35037]: pgmap v221: 1 pgs: 1 active+clean; 577 KiB data, 226 MiB used, 626 GiB / 626 GiB avail 2024-03-28T18:01:14.683 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:01:14 smithi079 ceph-mon[35312]: pgmap v221: 1 pgs: 1 active+clean; 577 KiB data, 226 MiB used, 626 GiB / 626 GiB avail 2024-03-28T18:01:14.781 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:01:14 smithi022 ceph-mon[31985]: pgmap v221: 1 pgs: 1 active+clean; 577 KiB data, 226 MiB used, 626 GiB / 626 GiB avail 2024-03-28T18:01:16.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:01:16 smithi079 ceph-mon[35312]: pgmap v222: 1 pgs: 1 active+clean; 577 KiB data, 226 MiB used, 626 GiB / 626 GiB avail 2024-03-28T18:01:16.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:01:16 smithi079 ceph-mon[35312]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T18:01:16.683 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:01:16 smithi079 ceph-mon[35312]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T18:01:16.683 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:01:16 smithi079 ceph-mon[35312]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T18:01:16.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:01:16 smithi022 ceph-mon[31985]: pgmap v222: 1 pgs: 1 active+clean; 577 KiB data, 226 MiB used, 626 GiB / 626 GiB avail 2024-03-28T18:01:16.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:01:16 smithi022 ceph-mon[31985]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T18:01:16.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:01:16 smithi022 ceph-mon[31985]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T18:01:16.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:01:16 smithi022 ceph-mon[31985]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T18:01:16.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:01:16 smithi134 ceph-mon[35037]: pgmap v222: 1 pgs: 1 active+clean; 577 KiB data, 226 MiB used, 626 GiB / 626 GiB avail 2024-03-28T18:01:16.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:01:16 smithi134 ceph-mon[35037]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T18:01:16.925 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:01:16 smithi134 ceph-mon[35037]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T18:01:16.925 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:01:16 smithi134 ceph-mon[35037]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T18:01:18.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:01:18 smithi079 ceph-mon[35312]: pgmap v223: 1 pgs: 1 active+clean; 577 KiB data, 226 MiB used, 626 GiB / 626 GiB avail 2024-03-28T18:01:18.781 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:01:18 smithi022 ceph-mon[31985]: pgmap v223: 1 pgs: 1 active+clean; 577 KiB data, 226 MiB used, 626 GiB / 626 GiB avail 2024-03-28T18:01:18.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:01:18 smithi134 ceph-mon[35037]: pgmap v223: 1 pgs: 1 active+clean; 577 KiB data, 226 MiB used, 626 GiB / 626 GiB avail 2024-03-28T18:01:20.593 INFO:teuthology.orchestra.run.smithi079.stdout:Created osd(s) 7 on host 'smithi079' 2024-03-28T18:01:20.593 INFO:teuthology.orchestra.run.smithi079.stderr:2024-03-28T18:01:20.593+0000 7fca1a000700 1 -- 172.21.15.79:0/3893423916 <== mgr.14152 v2:172.21.15.22:6800/4201048392 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7fca1c0625d0 con 0x7fca040a2890 2024-03-28T18:01:20.596 INFO:teuthology.orchestra.run.smithi079.stderr:2024-03-28T18:01:20.596+0000 7fca24200700 1 -- 172.21.15.79:0/3893423916 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7fca040a2890 msgr2=0x7fca040a4d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:01:20.596 INFO:teuthology.orchestra.run.smithi079.stderr:2024-03-28T18:01:20.596+0000 7fca24200700 1 --2- 172.21.15.79:0/3893423916 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7fca040a2890 0x7fca040a4d50 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fca080099d0 tx=0x7fca08008040 comp rx=0 tx=0).stop 2024-03-28T18:01:20.596 INFO:teuthology.orchestra.run.smithi079.stderr:2024-03-28T18:01:20.596+0000 7fca24200700 1 -- 172.21.15.79:0/3893423916 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fca1c104410 msgr2=0x7fca1c19a680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:01:20.596 INFO:teuthology.orchestra.run.smithi079.stderr:2024-03-28T18:01:20.596+0000 7fca24200700 1 --2- 172.21.15.79:0/3893423916 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fca1c104410 0x7fca1c19a680 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fca1000c970 tx=0x7fca1000cd30 comp rx=0 tx=0).stop 2024-03-28T18:01:20.596 INFO:teuthology.orchestra.run.smithi079.stderr:2024-03-28T18:01:20.596+0000 7fca24200700 1 -- 172.21.15.79:0/3893423916 shutdown_connections 2024-03-28T18:01:20.596 INFO:teuthology.orchestra.run.smithi079.stderr:2024-03-28T18:01:20.596+0000 7fca24200700 1 --2- 172.21.15.79:0/3893423916 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7fca040a2890 0x7fca040a4d50 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:01:20.596 INFO:teuthology.orchestra.run.smithi079.stderr:2024-03-28T18:01:20.596+0000 7fca24200700 1 --2- 172.21.15.79:0/3893423916 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fca1c105fb0 0x7fca1c1a1c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:01:20.597 INFO:teuthology.orchestra.run.smithi079.stderr:2024-03-28T18:01:20.596+0000 7fca24200700 1 --2- 172.21.15.79:0/3893423916 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fca1c104410 0x7fca1c19a680 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:01:20.597 INFO:teuthology.orchestra.run.smithi079.stderr:2024-03-28T18:01:20.596+0000 7fca24200700 1 --2- 172.21.15.79:0/3893423916 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fca1c105610 0x7fca1c19abc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:01:20.597 INFO:teuthology.orchestra.run.smithi079.stderr:2024-03-28T18:01:20.596+0000 7fca24200700 1 -- 172.21.15.79:0/3893423916 >> 172.21.15.79:0/3893423916 conn(0x7fca1c0ff9b0 msgr2=0x7fca1c108840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:01:20.597 INFO:teuthology.orchestra.run.smithi079.stderr:2024-03-28T18:01:20.597+0000 7fca24200700 1 -- 172.21.15.79:0/3893423916 shutdown_connections 2024-03-28T18:01:20.597 INFO:teuthology.orchestra.run.smithi079.stderr:2024-03-28T18:01:20.597+0000 7fca24200700 1 -- 172.21.15.79:0/3893423916 wait complete. 2024-03-28T18:01:20.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:01:20 smithi022 ceph-mon[31985]: pgmap v224: 1 pgs: 1 active+clean; 577 KiB data, 226 MiB used, 626 GiB / 626 GiB avail 2024-03-28T18:01:20.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:01:20 smithi022 ceph-mon[31985]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T18:01:20.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:01:20 smithi022 ceph-mon[31985]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T18:01:20.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:01:20 smithi022 ceph-mon[31985]: from='osd.7 [v2:172.21.15.79:6816/1287105314,v1:172.21.15.79:6817/1287105314]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-03-28T18:01:20.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:01:20 smithi022 ceph-mon[31985]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-03-28T18:01:20.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:01:20 smithi134 ceph-mon[35037]: pgmap v224: 1 pgs: 1 active+clean; 577 KiB data, 226 MiB used, 626 GiB / 626 GiB avail 2024-03-28T18:01:20.925 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:01:20 smithi134 ceph-mon[35037]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T18:01:20.925 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:01:20 smithi134 ceph-mon[35037]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T18:01:20.925 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:01:20 smithi134 ceph-mon[35037]: from='osd.7 [v2:172.21.15.79:6816/1287105314,v1:172.21.15.79:6817/1287105314]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-03-28T18:01:20.925 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:01:20 smithi134 ceph-mon[35037]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-03-28T18:01:20.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:01:20 smithi079 ceph-mon[35312]: pgmap v224: 1 pgs: 1 active+clean; 577 KiB data, 226 MiB used, 626 GiB / 626 GiB avail 2024-03-28T18:01:20.933 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:01:20 smithi079 ceph-mon[35312]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T18:01:20.933 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:01:20 smithi079 ceph-mon[35312]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T18:01:20.933 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:01:20 smithi079 ceph-mon[35312]: from='osd.7 [v2:172.21.15.79:6816/1287105314,v1:172.21.15.79:6817/1287105314]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-03-28T18:01:20.933 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:01:20 smithi079 ceph-mon[35312]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-03-28T18:01:21.641 DEBUG:teuthology.orchestra.run.smithi079:osd.7> sudo journalctl -f -n 0 -u ceph-fd220748-ed2b-11ee-95d0-87774f69a715@osd.7.service 2024-03-28T18:01:21.646 INFO:tasks.cephadm:Deploying osd.8 on smithi134 with /dev/vg_nvme/lv_2... 2024-03-28T18:01:21.646 DEBUG:teuthology.orchestra.run.smithi134:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fd220748-ed2b-11ee-95d0-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-03-28T18:01:21.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:01:21 smithi079 ceph-mon[35312]: pgmap v225: 1 pgs: 1 active+clean; 577 KiB data, 226 MiB used, 626 GiB / 626 GiB avail 2024-03-28T18:01:21.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:01:21 smithi079 ceph-mon[35312]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T18:01:21.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:01:21 smithi079 ceph-mon[35312]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T18:01:21.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:01:21 smithi079 ceph-mon[35312]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-03-28T18:01:21.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:01:21 smithi079 ceph-mon[35312]: osdmap e51: 8 total, 7 up, 8 in 2024-03-28T18:01:21.683 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:01:21 smithi079 ceph-mon[35312]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-28T18:01:21.683 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:01:21 smithi079 ceph-mon[35312]: from='osd.7 [v2:172.21.15.79:6816/1287105314,v1:172.21.15.79:6817/1287105314]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi079", "root=default"]}]: dispatch 2024-03-28T18:01:21.683 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:01:21 smithi079 ceph-mon[35312]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi079", "root=default"]}]: dispatch 2024-03-28T18:01:21.925 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:01:21 smithi134 ceph-mon[35037]: pgmap v225: 1 pgs: 1 active+clean; 577 KiB data, 226 MiB used, 626 GiB / 626 GiB avail 2024-03-28T18:01:21.925 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:01:21 smithi134 ceph-mon[35037]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T18:01:21.925 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:01:21 smithi134 ceph-mon[35037]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T18:01:21.925 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:01:21 smithi134 ceph-mon[35037]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-03-28T18:01:21.925 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:01:21 smithi134 ceph-mon[35037]: osdmap e51: 8 total, 7 up, 8 in 2024-03-28T18:01:21.925 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:01:21 smithi134 ceph-mon[35037]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-28T18:01:21.925 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:01:21 smithi134 ceph-mon[35037]: from='osd.7 [v2:172.21.15.79:6816/1287105314,v1:172.21.15.79:6817/1287105314]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi079", "root=default"]}]: dispatch 2024-03-28T18:01:21.925 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:01:21 smithi134 ceph-mon[35037]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi079", "root=default"]}]: dispatch 2024-03-28T18:01:21.987 INFO:journalctl@ceph.osd.7.smithi079.stdout:Mar 28 18:01:21 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-osd-7[53347]: 2024-03-28T18:01:21.810+0000 7f78a9600700 -1 osd.7 0 waiting for initial osdmap 2024-03-28T18:01:21.987 INFO:journalctl@ceph.osd.7.smithi079.stdout:Mar 28 18:01:21 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-osd-7[53347]: 2024-03-28T18:01:21.830+0000 7f78a4400700 -1 osd.7 52 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-28T18:01:22.032 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:01:21 smithi022 ceph-mon[31985]: pgmap v225: 1 pgs: 1 active+clean; 577 KiB data, 226 MiB used, 626 GiB / 626 GiB avail 2024-03-28T18:01:22.032 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:01:21 smithi022 ceph-mon[31985]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T18:01:22.032 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:01:21 smithi022 ceph-mon[31985]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T18:01:22.032 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:01:21 smithi022 ceph-mon[31985]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-03-28T18:01:22.032 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:01:21 smithi022 ceph-mon[31985]: osdmap e51: 8 total, 7 up, 8 in 2024-03-28T18:01:22.032 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:01:21 smithi022 ceph-mon[31985]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-28T18:01:22.032 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:01:21 smithi022 ceph-mon[31985]: from='osd.7 [v2:172.21.15.79:6816/1287105314,v1:172.21.15.79:6817/1287105314]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi079", "root=default"]}]: dispatch 2024-03-28T18:01:22.032 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:01:21 smithi022 ceph-mon[31985]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi079", "root=default"]}]: dispatch 2024-03-28T18:01:22.096 INFO:teuthology.orchestra.run.smithi134.stderr:Inferring config /var/lib/ceph/fd220748-ed2b-11ee-95d0-87774f69a715/mon.c/config 2024-03-28T18:01:22.925 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:01:22 smithi134 ceph-mon[35037]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi079", "root=default"]}]': finished 2024-03-28T18:01:22.925 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:01:22 smithi134 ceph-mon[35037]: osdmap e52: 8 total, 7 up, 8 in 2024-03-28T18:01:22.925 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:01:22 smithi134 ceph-mon[35037]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-28T18:01:22.925 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:01:22 smithi134 ceph-mon[35037]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-28T18:01:22.925 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:01:22 smithi134 ceph-mon[35037]: osd.7 [v2:172.21.15.79:6816/1287105314,v1:172.21.15.79:6817/1287105314] boot 2024-03-28T18:01:22.925 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:01:22 smithi134 ceph-mon[35037]: osdmap e53: 8 total, 8 up, 8 in 2024-03-28T18:01:22.925 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:01:22 smithi134 ceph-mon[35037]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-28T18:01:22.933 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:01:22 smithi079 ceph-mon[35312]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi079", "root=default"]}]': finished 2024-03-28T18:01:22.933 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:01:22 smithi079 ceph-mon[35312]: osdmap e52: 8 total, 7 up, 8 in 2024-03-28T18:01:22.933 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:01:22 smithi079 ceph-mon[35312]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-28T18:01:22.933 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:01:22 smithi079 ceph-mon[35312]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-28T18:01:22.933 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:01:22 smithi079 ceph-mon[35312]: osd.7 [v2:172.21.15.79:6816/1287105314,v1:172.21.15.79:6817/1287105314] boot 2024-03-28T18:01:22.933 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:01:22 smithi079 ceph-mon[35312]: osdmap e53: 8 total, 8 up, 8 in 2024-03-28T18:01:22.933 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:01:22 smithi079 ceph-mon[35312]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-28T18:01:23.282 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:01:22 smithi022 ceph-mon[31985]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi079", "root=default"]}]': finished 2024-03-28T18:01:23.282 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:01:22 smithi022 ceph-mon[31985]: osdmap e52: 8 total, 7 up, 8 in 2024-03-28T18:01:23.282 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:01:22 smithi022 ceph-mon[31985]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-28T18:01:23.282 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:01:22 smithi022 ceph-mon[31985]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-28T18:01:23.282 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:01:22 smithi022 ceph-mon[31985]: osd.7 [v2:172.21.15.79:6816/1287105314,v1:172.21.15.79:6817/1287105314] boot 2024-03-28T18:01:23.282 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:01:22 smithi022 ceph-mon[31985]: osdmap e53: 8 total, 8 up, 8 in 2024-03-28T18:01:23.282 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:01:22 smithi022 ceph-mon[31985]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-28T18:01:24.071 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:01:23 smithi134 ceph-mon[35037]: purged_snaps scrub starts 2024-03-28T18:01:24.071 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:01:23 smithi134 ceph-mon[35037]: purged_snaps scrub ok 2024-03-28T18:01:24.071 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:01:23 smithi134 ceph-mon[35037]: pgmap v229: 1 pgs: 1 active+clean; 577 KiB data, 226 MiB used, 626 GiB / 626 GiB avail 2024-03-28T18:01:24.072 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:01:23 smithi134 ceph-mon[35037]: osdmap e54: 8 total, 8 up, 8 in 2024-03-28T18:01:24.183 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:01:23 smithi079 ceph-mon[35312]: purged_snaps scrub starts 2024-03-28T18:01:24.183 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:01:23 smithi079 ceph-mon[35312]: purged_snaps scrub ok 2024-03-28T18:01:24.183 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:01:23 smithi079 ceph-mon[35312]: pgmap v229: 1 pgs: 1 active+clean; 577 KiB data, 226 MiB used, 626 GiB / 626 GiB avail 2024-03-28T18:01:24.183 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:01:23 smithi079 ceph-mon[35312]: osdmap e54: 8 total, 8 up, 8 in 2024-03-28T18:01:24.282 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:01:23 smithi022 ceph-mon[31985]: purged_snaps scrub starts 2024-03-28T18:01:24.282 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:01:23 smithi022 ceph-mon[31985]: purged_snaps scrub ok 2024-03-28T18:01:24.282 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:01:23 smithi022 ceph-mon[31985]: pgmap v229: 1 pgs: 1 active+clean; 577 KiB data, 226 MiB used, 626 GiB / 626 GiB avail 2024-03-28T18:01:24.282 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:01:23 smithi022 ceph-mon[31985]: osdmap e54: 8 total, 8 up, 8 in 2024-03-28T18:01:25.033 INFO:teuthology.orchestra.run.smithi134.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-03-28T18:01:25.034 INFO:teuthology.orchestra.run.smithi134.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-03-28T18:01:25.034 INFO:teuthology.orchestra.run.smithi134.stderr: stderr: 10+0 records in 2024-03-28T18:01:25.034 INFO:teuthology.orchestra.run.smithi134.stderr:10+0 records out 2024-03-28T18:01:25.034 INFO:teuthology.orchestra.run.smithi134.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0326968 s, 321 MB/s 2024-03-28T18:01:25.034 INFO:teuthology.orchestra.run.smithi134.stderr:--> Zapping successful for: 2024-03-28T18:01:25.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:01:25 smithi079 ceph-mon[35312]: osdmap e55: 8 total, 8 up, 8 in 2024-03-28T18:01:25.781 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:01:25 smithi022 ceph-mon[31985]: osdmap e55: 8 total, 8 up, 8 in 2024-03-28T18:01:25.925 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:01:25 smithi134 ceph-mon[35037]: osdmap e55: 8 total, 8 up, 8 in 2024-03-28T18:01:26.035 DEBUG:teuthology.orchestra.run.smithi134:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fd220748-ed2b-11ee-95d0-87774f69a715 -- ceph orch daemon add osd smithi134:vg_nvme/lv_2 2024-03-28T18:01:26.460 INFO:teuthology.orchestra.run.smithi134.stderr:Inferring config /var/lib/ceph/fd220748-ed2b-11ee-95d0-87774f69a715/mon.c/config 2024-03-28T18:01:26.781 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:01:26 smithi022 ceph-mon[31985]: pgmap v232: 1 pgs: 1 active+clean; 577 KiB data, 253 MiB used, 715 GiB / 715 GiB avail 2024-03-28T18:01:26.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:01:26 smithi022 ceph-mon[31985]: osdmap e56: 8 total, 8 up, 8 in 2024-03-28T18:01:26.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:01:26 smithi022 ceph-mon[31985]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T18:01:26.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:01:26 smithi022 ceph-mon[31985]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T18:01:26.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:01:26 smithi022 ceph-mon[31985]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-28T18:01:26.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:01:26 smithi022 ceph-mon[31985]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-28T18:01:26.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:01:26 smithi022 ceph-mon[31985]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-03-28T18:01:26.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:01:26 smithi022 ceph-mon[31985]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T18:01:26.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:01:26 smithi022 ceph-mon[31985]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T18:01:26.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:01:26 smithi022 ceph-mon[31985]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T18:01:26.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:01:26 smithi022 ceph-mon[31985]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T18:01:26.783 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:01:26 smithi134 ceph-mon[35037]: pgmap v232: 1 pgs: 1 active+clean; 577 KiB data, 253 MiB used, 715 GiB / 715 GiB avail 2024-03-28T18:01:26.783 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:01:26 smithi134 ceph-mon[35037]: osdmap e56: 8 total, 8 up, 8 in 2024-03-28T18:01:26.783 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:01:26 smithi134 ceph-mon[35037]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T18:01:26.783 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:01:26 smithi134 ceph-mon[35037]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T18:01:26.783 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:01:26 smithi134 ceph-mon[35037]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-28T18:01:26.783 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:01:26 smithi134 ceph-mon[35037]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-28T18:01:26.783 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:01:26 smithi134 ceph-mon[35037]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-03-28T18:01:26.783 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:01:26 smithi134 ceph-mon[35037]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T18:01:26.783 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:01:26 smithi134 ceph-mon[35037]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T18:01:26.784 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:01:26 smithi134 ceph-mon[35037]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T18:01:26.784 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:01:26 smithi134 ceph-mon[35037]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T18:01:26.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:01:26 smithi079 ceph-mon[35312]: pgmap v232: 1 pgs: 1 active+clean; 577 KiB data, 253 MiB used, 715 GiB / 715 GiB avail 2024-03-28T18:01:26.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:01:26 smithi079 ceph-mon[35312]: osdmap e56: 8 total, 8 up, 8 in 2024-03-28T18:01:26.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:01:26 smithi079 ceph-mon[35312]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T18:01:26.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:01:26 smithi079 ceph-mon[35312]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T18:01:26.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:01:26 smithi079 ceph-mon[35312]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-28T18:01:26.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:01:26 smithi079 ceph-mon[35312]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-28T18:01:26.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:01:26 smithi079 ceph-mon[35312]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-03-28T18:01:26.933 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:01:26 smithi079 ceph-mon[35312]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T18:01:26.933 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:01:26 smithi079 ceph-mon[35312]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T18:01:26.933 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:01:26 smithi079 ceph-mon[35312]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T18:01:26.933 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:01:26 smithi079 ceph-mon[35312]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T18:01:27.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:01:27 smithi022 ceph-mon[31985]: Detected new or changed devices on smithi079 2024-03-28T18:01:27.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:01:27 smithi022 ceph-mon[31985]: Adjusting osd_memory_target on smithi079 to 5697M 2024-03-28T18:01:27.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:01:27 smithi022 ceph-mon[31985]: osdmap e57: 8 total, 8 up, 8 in 2024-03-28T18:01:27.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:01:27 smithi022 ceph-mon[31985]: pgmap v235: 1 pgs: 1 active+clean; 577 KiB data, 253 MiB used, 715 GiB / 715 GiB avail 2024-03-28T18:01:27.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:01:27 smithi022 ceph-mon[31985]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T18:01:27.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:01:27 smithi022 ceph-mon[31985]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T18:01:27.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:01:27 smithi022 ceph-mon[31985]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T18:01:27.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:01:27 smithi022 ceph-mon[31985]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T18:01:27.805 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:01:27 smithi134 ceph-mon[35037]: Detected new or changed devices on smithi079 2024-03-28T18:01:27.806 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:01:27 smithi134 ceph-mon[35037]: Adjusting osd_memory_target on smithi079 to 5697M 2024-03-28T18:01:27.806 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:01:27 smithi134 ceph-mon[35037]: osdmap e57: 8 total, 8 up, 8 in 2024-03-28T18:01:27.806 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:01:27 smithi134 ceph-mon[35037]: pgmap v235: 1 pgs: 1 active+clean; 577 KiB data, 253 MiB used, 715 GiB / 715 GiB avail 2024-03-28T18:01:27.806 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:01:27 smithi134 ceph-mon[35037]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T18:01:27.806 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:01:27 smithi134 ceph-mon[35037]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T18:01:27.806 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:01:27 smithi134 ceph-mon[35037]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T18:01:27.806 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:01:27 smithi134 ceph-mon[35037]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T18:01:27.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:01:27 smithi079 ceph-mon[35312]: Detected new or changed devices on smithi079 2024-03-28T18:01:27.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:01:27 smithi079 ceph-mon[35312]: Adjusting osd_memory_target on smithi079 to 5697M 2024-03-28T18:01:27.933 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:01:27 smithi079 ceph-mon[35312]: osdmap e57: 8 total, 8 up, 8 in 2024-03-28T18:01:27.933 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:01:27 smithi079 ceph-mon[35312]: pgmap v235: 1 pgs: 1 active+clean; 577 KiB data, 253 MiB used, 715 GiB / 715 GiB avail 2024-03-28T18:01:27.933 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:01:27 smithi079 ceph-mon[35312]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T18:01:27.933 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:01:27 smithi079 ceph-mon[35312]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T18:01:27.933 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:01:27 smithi079 ceph-mon[35312]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T18:01:27.933 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:01:27 smithi079 ceph-mon[35312]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T18:01:28.613 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-28T18:01:28.612+0000 7ffb9fa00700 1 -- 172.21.15.134:0/3245391696 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ffb98105fb0 msgr2=0x7ffb9810c860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:01:28.613 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-28T18:01:28.612+0000 7ffb9fa00700 1 --2- 172.21.15.134:0/3245391696 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ffb98105fb0 0x7ffb9810c860 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7ffb9000b0a0 tx=0x7ffb9000b3b0 comp rx=0 tx=0).stop 2024-03-28T18:01:28.613 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-28T18:01:28.612+0000 7ffb9fa00700 1 -- 172.21.15.134:0/3245391696 shutdown_connections 2024-03-28T18:01:28.613 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-28T18:01:28.612+0000 7ffb9fa00700 1 --2- 172.21.15.134:0/3245391696 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ffb98105fb0 0x7ffb9810c860 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:01:28.613 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-28T18:01:28.612+0000 7ffb9fa00700 1 --2- 172.21.15.134:0/3245391696 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ffb98104410 0x7ffb98104830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:01:28.613 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-28T18:01:28.612+0000 7ffb9fa00700 1 --2- 172.21.15.134:0/3245391696 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ffb98105610 0x7ffb98105a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:01:28.613 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-28T18:01:28.612+0000 7ffb9fa00700 1 -- 172.21.15.134:0/3245391696 >> 172.21.15.134:0/3245391696 conn(0x7ffb980ff9b0 msgr2=0x7ffb98101df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:01:28.613 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-28T18:01:28.612+0000 7ffb9fa00700 1 -- 172.21.15.134:0/3245391696 shutdown_connections 2024-03-28T18:01:28.614 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-28T18:01:28.612+0000 7ffb9fa00700 1 -- 172.21.15.134:0/3245391696 wait complete. 2024-03-28T18:01:28.615 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-28T18:01:28.613+0000 7ffb9fa00700 1 Processor -- start 2024-03-28T18:01:28.615 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-28T18:01:28.614+0000 7ffb9fa00700 1 -- start start 2024-03-28T18:01:28.615 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-28T18:01:28.614+0000 7ffb9fa00700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ffb98104410 0x7ffb9819a450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:01:28.615 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-28T18:01:28.614+0000 7ffb9fa00700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ffb98105610 0x7ffb9819a990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:01:28.615 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-28T18:01:28.614+0000 7ffb9fa00700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ffb98105fb0 0x7ffb981a1a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:01:28.615 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-28T18:01:28.614+0000 7ffb9fa00700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb98110050 con 0x7ffb98104410 2024-03-28T18:01:28.615 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-28T18:01:28.614+0000 7ffb9fa00700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb9810fed0 con 0x7ffb98105fb0 2024-03-28T18:01:28.615 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-28T18:01:28.614+0000 7ffb9fa00700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb981101d0 con 0x7ffb98105610 2024-03-28T18:01:28.615 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-28T18:01:28.614+0000 7ffb97e00700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ffb98105610 0x7ffb9819a990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:01:28.615 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-28T18:01:28.614+0000 7ffb97e00700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ffb98105610 0x7ffb9819a990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.134:38776/0 (socket says 172.21.15.134:38776) 2024-03-28T18:01:28.615 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-28T18:01:28.614+0000 7ffb97e00700 1 -- 172.21.15.134:0/3374842270 learned_addr learned my addr 172.21.15.134:0/3374842270 (peer_addr_for_me v2:172.21.15.134:0/0) 2024-03-28T18:01:28.616 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-28T18:01:28.615+0000 7ffb9d000700 1 --2- 172.21.15.134:0/3374842270 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ffb98104410 0x7ffb9819a450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:01:28.616 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-28T18:01:28.615+0000 7ffb9da00700 1 --2- 172.21.15.134:0/3374842270 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ffb98105fb0 0x7ffb981a1a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:01:28.616 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-28T18:01:28.615+0000 7ffb97e00700 1 -- 172.21.15.134:0/3374842270 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ffb98105fb0 msgr2=0x7ffb981a1a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:01:28.616 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-28T18:01:28.615+0000 7ffb97e00700 1 --2- 172.21.15.134:0/3374842270 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ffb98105fb0 0x7ffb981a1a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:01:28.616 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-28T18:01:28.615+0000 7ffb97e00700 1 -- 172.21.15.134:0/3374842270 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ffb98104410 msgr2=0x7ffb9819a450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:01:28.616 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-28T18:01:28.615+0000 7ffb97e00700 1 --2- 172.21.15.134:0/3374842270 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ffb98104410 0x7ffb9819a450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:01:28.617 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-28T18:01:28.615+0000 7ffb97e00700 1 -- 172.21.15.134:0/3374842270 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb981a20b0 con 0x7ffb98105610 2024-03-28T18:01:28.617 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-28T18:01:28.615+0000 7ffb97e00700 1 --2- 172.21.15.134:0/3374842270 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ffb98105610 0x7ffb9819a990 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7ffb8400c9e0 tx=0x7ffb8400cda0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:01:28.617 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-28T18:01:28.615+0000 7ffb95600700 1 -- 172.21.15.134:0/3374842270 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb84007ac0 con 0x7ffb98105610 2024-03-28T18:01:28.617 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-28T18:01:28.615+0000 7ffb95600700 1 -- 172.21.15.134:0/3374842270 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7ffb84007c60 con 0x7ffb98105610 2024-03-28T18:01:28.617 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-28T18:01:28.615+0000 7ffb95600700 1 -- 172.21.15.134:0/3374842270 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb840186f0 con 0x7ffb98105610 2024-03-28T18:01:28.617 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-28T18:01:28.615+0000 7ffb9fa00700 1 -- 172.21.15.134:0/3374842270 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffb981a23a0 con 0x7ffb98105610 2024-03-28T18:01:28.617 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-28T18:01:28.616+0000 7ffb9fa00700 1 -- 172.21.15.134:0/3374842270 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffb981a2890 con 0x7ffb98105610 2024-03-28T18:01:28.617 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-28T18:01:28.617+0000 7ffb9fa00700 1 -- 172.21.15.134:0/3374842270 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffb98068e50 con 0x7ffb98105610 2024-03-28T18:01:28.618 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-28T18:01:28.619+0000 7ffb95600700 1 -- 172.21.15.134:0/3374842270 <== mon.1 v2:172.21.15.134:3300/0 4 ==== mgrmap(e 16) v1 ==== 137290+0+0 (secure 0 0 0) 0x7ffb8400f450 con 0x7ffb98105610 2024-03-28T18:01:28.620 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-28T18:01:28.620+0000 7ffb95600700 1 --2- 172.21.15.134:0/3374842270 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7ffb800a2890 0x7ffb800a4d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:01:28.620 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-28T18:01:28.620+0000 7ffb95600700 1 -- 172.21.15.134:0/3374842270 <== mon.1 v2:172.21.15.134:3300/0 5 ==== osd_map(57..57 src has 1..57) v4 ==== 5397+0+0 (secure 0 0 0) 0x7ffb840ce1c0 con 0x7ffb98105610 2024-03-28T18:01:28.620 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-28T18:01:28.620+0000 7ffb9d000700 1 --2- 172.21.15.134:0/3374842270 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7ffb800a2890 0x7ffb800a4d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:01:28.621 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-28T18:01:28.621+0000 7ffb9d000700 1 --2- 172.21.15.134:0/3374842270 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7ffb800a2890 0x7ffb800a4d50 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7ffb8c005fd0 tx=0x7ffb8c005e50 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:01:28.623 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-28T18:01:28.623+0000 7ffb95600700 1 -- 172.21.15.134:0/3374842270 <== mon.1 v2:172.21.15.134:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7ffb84014030 con 0x7ffb98105610 2024-03-28T18:01:28.855 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-28T18:01:28.854+0000 7ffb9fa00700 1 -- 172.21.15.134:0/3374842270 --> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi134:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) v1 -- 0x7ffb980625d0 con 0x7ffb800a2890 2024-03-28T18:01:29.851 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:01:29 smithi134 ceph-mon[35037]: pgmap v236: 1 pgs: 1 active+clean; 577 KiB data, 254 MiB used, 715 GiB / 715 GiB avail 2024-03-28T18:01:29.851 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:01:29 smithi134 ceph-mon[35037]: from='client.24286 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi134:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-03-28T18:01:29.851 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:01:29 smithi134 ceph-mon[35037]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-28T18:01:29.851 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:01:29 smithi134 ceph-mon[35037]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-28T18:01:29.852 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:01:29 smithi134 ceph-mon[35037]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T18:01:29.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:01:29 smithi079 ceph-mon[35312]: pgmap v236: 1 pgs: 1 active+clean; 577 KiB data, 254 MiB used, 715 GiB / 715 GiB avail 2024-03-28T18:01:29.933 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:01:29 smithi079 ceph-mon[35312]: from='client.24286 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi134:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-03-28T18:01:29.933 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:01:29 smithi079 ceph-mon[35312]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-28T18:01:29.933 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:01:29 smithi079 ceph-mon[35312]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-28T18:01:29.933 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:01:29 smithi079 ceph-mon[35312]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T18:01:30.032 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:01:29 smithi022 ceph-mon[31985]: pgmap v236: 1 pgs: 1 active+clean; 577 KiB data, 254 MiB used, 715 GiB / 715 GiB avail 2024-03-28T18:01:30.032 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:01:29 smithi022 ceph-mon[31985]: from='client.24286 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi134:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-03-28T18:01:30.032 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:01:29 smithi022 ceph-mon[31985]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-28T18:01:30.032 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:01:29 smithi022 ceph-mon[31985]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-28T18:01:30.032 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:01:29 smithi022 ceph-mon[31985]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T18:01:31.670 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:01:31 smithi134 ceph-mon[35037]: pgmap v237: 1 pgs: 1 active+clean; 577 KiB data, 254 MiB used, 715 GiB / 715 GiB avail 2024-03-28T18:01:31.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:01:31 smithi079 ceph-mon[35312]: pgmap v237: 1 pgs: 1 active+clean; 577 KiB data, 254 MiB used, 715 GiB / 715 GiB avail 2024-03-28T18:01:32.031 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:01:31 smithi022 ceph-mon[31985]: pgmap v237: 1 pgs: 1 active+clean; 577 KiB data, 254 MiB used, 715 GiB / 715 GiB avail 2024-03-28T18:01:33.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:01:33 smithi134 ceph-mon[35037]: pgmap v238: 1 pgs: 1 active+clean; 577 KiB data, 254 MiB used, 715 GiB / 715 GiB avail 2024-03-28T18:01:33.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:01:33 smithi134 ceph-mon[35037]: from='client.? 172.21.15.134:0/949681464' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "20aa66c1-03d6-4cb8-a096-0eae41979c17"}]: dispatch 2024-03-28T18:01:33.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:01:33 smithi134 ceph-mon[35037]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "20aa66c1-03d6-4cb8-a096-0eae41979c17"}]: dispatch 2024-03-28T18:01:33.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:01:33 smithi134 ceph-mon[35037]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "20aa66c1-03d6-4cb8-a096-0eae41979c17"}]': finished 2024-03-28T18:01:33.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:01:33 smithi134 ceph-mon[35037]: osdmap e58: 9 total, 8 up, 9 in 2024-03-28T18:01:33.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:01:33 smithi134 ceph-mon[35037]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-28T18:01:33.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:01:33 smithi022 ceph-mon[31985]: pgmap v238: 1 pgs: 1 active+clean; 577 KiB data, 254 MiB used, 715 GiB / 715 GiB avail 2024-03-28T18:01:33.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:01:33 smithi022 ceph-mon[31985]: from='client.? 172.21.15.134:0/949681464' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "20aa66c1-03d6-4cb8-a096-0eae41979c17"}]: dispatch 2024-03-28T18:01:33.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:01:33 smithi022 ceph-mon[31985]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "20aa66c1-03d6-4cb8-a096-0eae41979c17"}]: dispatch 2024-03-28T18:01:33.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:01:33 smithi022 ceph-mon[31985]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "20aa66c1-03d6-4cb8-a096-0eae41979c17"}]': finished 2024-03-28T18:01:33.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:01:33 smithi022 ceph-mon[31985]: osdmap e58: 9 total, 8 up, 9 in 2024-03-28T18:01:33.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:01:33 smithi022 ceph-mon[31985]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-28T18:01:33.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:01:33 smithi079 ceph-mon[35312]: pgmap v238: 1 pgs: 1 active+clean; 577 KiB data, 254 MiB used, 715 GiB / 715 GiB avail 2024-03-28T18:01:33.933 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:01:33 smithi079 ceph-mon[35312]: from='client.? 172.21.15.134:0/949681464' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "20aa66c1-03d6-4cb8-a096-0eae41979c17"}]: dispatch 2024-03-28T18:01:33.933 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:01:33 smithi079 ceph-mon[35312]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "20aa66c1-03d6-4cb8-a096-0eae41979c17"}]: dispatch 2024-03-28T18:01:33.933 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:01:33 smithi079 ceph-mon[35312]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "20aa66c1-03d6-4cb8-a096-0eae41979c17"}]': finished 2024-03-28T18:01:33.933 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:01:33 smithi079 ceph-mon[35312]: osdmap e58: 9 total, 8 up, 9 in 2024-03-28T18:01:33.933 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:01:33 smithi079 ceph-mon[35312]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-28T18:01:34.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:01:34 smithi134 ceph-mon[35037]: from='client.? 172.21.15.134:0/2403460679' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-28T18:01:34.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:01:34 smithi079 ceph-mon[35312]: from='client.? 172.21.15.134:0/2403460679' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-28T18:01:35.031 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:01:34 smithi022 ceph-mon[31985]: from='client.? 172.21.15.134:0/2403460679' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-28T18:01:35.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:01:35 smithi134 ceph-mon[35037]: pgmap v240: 1 pgs: 1 active+clean; 577 KiB data, 254 MiB used, 715 GiB / 715 GiB avail 2024-03-28T18:01:35.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:01:35 smithi079 ceph-mon[35312]: pgmap v240: 1 pgs: 1 active+clean; 577 KiB data, 254 MiB used, 715 GiB / 715 GiB avail 2024-03-28T18:01:36.031 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:01:35 smithi022 ceph-mon[31985]: pgmap v240: 1 pgs: 1 active+clean; 577 KiB data, 254 MiB used, 715 GiB / 715 GiB avail 2024-03-28T18:01:37.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:01:37 smithi134 ceph-mon[35037]: pgmap v241: 1 pgs: 1 active+clean; 577 KiB data, 254 MiB used, 715 GiB / 715 GiB avail 2024-03-28T18:01:37.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:01:37 smithi079 ceph-mon[35312]: pgmap v241: 1 pgs: 1 active+clean; 577 KiB data, 254 MiB used, 715 GiB / 715 GiB avail 2024-03-28T18:01:38.031 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:01:37 smithi022 ceph-mon[31985]: pgmap v241: 1 pgs: 1 active+clean; 577 KiB data, 254 MiB used, 715 GiB / 715 GiB avail 2024-03-28T18:01:39.683 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:01:39 smithi079 ceph-mon[35312]: pgmap v242: 1 pgs: 1 active+clean; 577 KiB data, 254 MiB used, 715 GiB / 715 GiB avail 2024-03-28T18:01:39.801 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:01:39 smithi134 ceph-mon[35037]: pgmap v242: 1 pgs: 1 active+clean; 577 KiB data, 254 MiB used, 715 GiB / 715 GiB avail 2024-03-28T18:01:40.031 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:01:39 smithi022 ceph-mon[31985]: pgmap v242: 1 pgs: 1 active+clean; 577 KiB data, 254 MiB used, 715 GiB / 715 GiB avail 2024-03-28T18:01:41.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:01:41 smithi134 ceph-mon[35037]: pgmap v243: 1 pgs: 1 active+clean; 577 KiB data, 254 MiB used, 715 GiB / 715 GiB avail 2024-03-28T18:01:41.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:01:41 smithi079 ceph-mon[35312]: pgmap v243: 1 pgs: 1 active+clean; 577 KiB data, 254 MiB used, 715 GiB / 715 GiB avail 2024-03-28T18:01:42.031 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:01:41 smithi022 ceph-mon[31985]: pgmap v243: 1 pgs: 1 active+clean; 577 KiB data, 254 MiB used, 715 GiB / 715 GiB avail 2024-03-28T18:01:42.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:01:42 smithi134 ceph-mon[35037]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-03-28T18:01:42.925 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:01:42 smithi134 ceph-mon[35037]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T18:01:42.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:01:42 smithi079 ceph-mon[35312]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-03-28T18:01:42.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:01:42 smithi079 ceph-mon[35312]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T18:01:43.031 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:01:42 smithi022 ceph-mon[31985]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-03-28T18:01:43.032 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:01:42 smithi022 ceph-mon[31985]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T18:01:43.874 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:01:43 smithi134 ceph-mon[35037]: pgmap v244: 1 pgs: 1 active+clean; 577 KiB data, 254 MiB used, 715 GiB / 715 GiB avail 2024-03-28T18:01:43.874 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:01:43 smithi134 ceph-mon[35037]: Deploying daemon osd.8 on smithi134 2024-03-28T18:01:43.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:01:43 smithi079 ceph-mon[35312]: pgmap v244: 1 pgs: 1 active+clean; 577 KiB data, 254 MiB used, 715 GiB / 715 GiB avail 2024-03-28T18:01:43.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:01:43 smithi079 ceph-mon[35312]: Deploying daemon osd.8 on smithi134 2024-03-28T18:01:44.032 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:01:43 smithi022 ceph-mon[31985]: pgmap v244: 1 pgs: 1 active+clean; 577 KiB data, 254 MiB used, 715 GiB / 715 GiB avail 2024-03-28T18:01:44.032 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:01:43 smithi022 ceph-mon[31985]: Deploying daemon osd.8 on smithi134 2024-03-28T18:01:45.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:01:45 smithi134 ceph-mon[35037]: pgmap v245: 1 pgs: 1 active+clean; 577 KiB data, 254 MiB used, 715 GiB / 715 GiB avail 2024-03-28T18:01:45.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:01:45 smithi079 ceph-mon[35312]: pgmap v245: 1 pgs: 1 active+clean; 577 KiB data, 254 MiB used, 715 GiB / 715 GiB avail 2024-03-28T18:01:46.031 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:01:45 smithi022 ceph-mon[31985]: pgmap v245: 1 pgs: 1 active+clean; 577 KiB data, 254 MiB used, 715 GiB / 715 GiB avail 2024-03-28T18:01:47.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:01:47 smithi079 ceph-mon[35312]: pgmap v246: 1 pgs: 1 active+clean; 577 KiB data, 254 MiB used, 715 GiB / 715 GiB avail 2024-03-28T18:01:48.032 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:01:47 smithi022 ceph-mon[31985]: pgmap v246: 1 pgs: 1 active+clean; 577 KiB data, 254 MiB used, 715 GiB / 715 GiB avail 2024-03-28T18:01:48.071 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:01:47 smithi134 ceph-mon[35037]: pgmap v246: 1 pgs: 1 active+clean; 577 KiB data, 254 MiB used, 715 GiB / 715 GiB avail 2024-03-28T18:01:49.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:01:49 smithi134 ceph-mon[35037]: pgmap v247: 1 pgs: 1 active+clean; 577 KiB data, 254 MiB used, 715 GiB / 715 GiB avail 2024-03-28T18:01:50.031 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:01:49 smithi022 ceph-mon[31985]: pgmap v247: 1 pgs: 1 active+clean; 577 KiB data, 254 MiB used, 715 GiB / 715 GiB avail 2024-03-28T18:01:50.182 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:01:49 smithi079 ceph-mon[35312]: pgmap v247: 1 pgs: 1 active+clean; 577 KiB data, 254 MiB used, 715 GiB / 715 GiB avail 2024-03-28T18:01:52.124 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:01:51 smithi134 ceph-mon[35037]: pgmap v248: 1 pgs: 1 active+clean; 577 KiB data, 254 MiB used, 715 GiB / 715 GiB avail 2024-03-28T18:01:52.124 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:01:51 smithi134 ceph-mon[35037]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T18:01:52.124 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:01:51 smithi134 ceph-mon[35037]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T18:01:52.124 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:01:51 smithi134 ceph-mon[35037]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T18:01:52.182 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:01:51 smithi079 ceph-mon[35312]: pgmap v248: 1 pgs: 1 active+clean; 577 KiB data, 254 MiB used, 715 GiB / 715 GiB avail 2024-03-28T18:01:52.182 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:01:51 smithi079 ceph-mon[35312]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T18:01:52.183 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:01:51 smithi079 ceph-mon[35312]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T18:01:52.183 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:01:51 smithi079 ceph-mon[35312]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T18:01:52.282 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:01:51 smithi022 ceph-mon[31985]: pgmap v248: 1 pgs: 1 active+clean; 577 KiB data, 254 MiB used, 715 GiB / 715 GiB avail 2024-03-28T18:01:52.282 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:01:51 smithi022 ceph-mon[31985]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T18:01:52.282 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:01:51 smithi022 ceph-mon[31985]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T18:01:52.282 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:01:51 smithi022 ceph-mon[31985]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T18:01:54.174 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:01:53 smithi134 ceph-mon[35037]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 254 MiB used, 715 GiB / 715 GiB avail 2024-03-28T18:01:54.182 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:01:53 smithi079 ceph-mon[35312]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 254 MiB used, 715 GiB / 715 GiB avail 2024-03-28T18:01:54.281 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:01:53 smithi022 ceph-mon[31985]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 254 MiB used, 715 GiB / 715 GiB avail 2024-03-28T18:01:56.023 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:01:55 smithi134 ceph-mon[35037]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 254 MiB used, 715 GiB / 715 GiB avail 2024-03-28T18:01:56.024 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:01:55 smithi134 ceph-mon[35037]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T18:01:56.024 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:01:55 smithi134 ceph-mon[35037]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T18:01:56.024 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:01:55 smithi134 ceph-mon[35037]: from='osd.8 [v2:172.21.15.134:6816/2265650388,v1:172.21.15.134:6817/2265650388]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-03-28T18:01:56.024 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:01:55 smithi134 ceph-mon[35037]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-03-28T18:01:56.024 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-28T18:01:56.022+0000 7ffb95600700 1 -- 172.21.15.134:0/3374842270 <== mgr.14152 v2:172.21.15.22:6800/4201048392 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7ffb980625d0 con 0x7ffb800a2890 2024-03-28T18:01:56.024 INFO:teuthology.orchestra.run.smithi134.stdout:Created osd(s) 8 on host 'smithi134' 2024-03-28T18:01:56.026 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-28T18:01:56.026+0000 7ffb9fa00700 1 -- 172.21.15.134:0/3374842270 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7ffb800a2890 msgr2=0x7ffb800a4d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:01:56.027 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-28T18:01:56.026+0000 7ffb9fa00700 1 --2- 172.21.15.134:0/3374842270 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7ffb800a2890 0x7ffb800a4d50 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7ffb8c005fd0 tx=0x7ffb8c005e50 comp rx=0 tx=0).stop 2024-03-28T18:01:56.027 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-28T18:01:56.026+0000 7ffb9fa00700 1 -- 172.21.15.134:0/3374842270 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ffb98105610 msgr2=0x7ffb9819a990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:01:56.027 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-28T18:01:56.026+0000 7ffb9fa00700 1 --2- 172.21.15.134:0/3374842270 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ffb98105610 0x7ffb9819a990 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7ffb8400c9e0 tx=0x7ffb8400cda0 comp rx=0 tx=0).stop 2024-03-28T18:01:56.027 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-28T18:01:56.026+0000 7ffb9fa00700 1 -- 172.21.15.134:0/3374842270 shutdown_connections 2024-03-28T18:01:56.027 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-28T18:01:56.026+0000 7ffb9fa00700 1 --2- 172.21.15.134:0/3374842270 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ffb98105fb0 0x7ffb981a1a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:01:56.027 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-28T18:01:56.026+0000 7ffb9fa00700 1 --2- 172.21.15.134:0/3374842270 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ffb98104410 0x7ffb9819a450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:01:56.027 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-28T18:01:56.026+0000 7ffb9fa00700 1 --2- 172.21.15.134:0/3374842270 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7ffb800a2890 0x7ffb800a4d50 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:01:56.027 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-28T18:01:56.026+0000 7ffb9fa00700 1 --2- 172.21.15.134:0/3374842270 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ffb98105610 0x7ffb9819a990 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:01:56.027 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-28T18:01:56.027+0000 7ffb9fa00700 1 -- 172.21.15.134:0/3374842270 >> 172.21.15.134:0/3374842270 conn(0x7ffb980ff9b0 msgr2=0x7ffb98108840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:01:56.027 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-28T18:01:56.027+0000 7ffb9fa00700 1 -- 172.21.15.134:0/3374842270 shutdown_connections 2024-03-28T18:01:56.027 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-28T18:01:56.027+0000 7ffb9fa00700 1 -- 172.21.15.134:0/3374842270 wait complete. 2024-03-28T18:01:56.032 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:01:55 smithi022 ceph-mon[31985]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 254 MiB used, 715 GiB / 715 GiB avail 2024-03-28T18:01:56.032 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:01:55 smithi022 ceph-mon[31985]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T18:01:56.032 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:01:55 smithi022 ceph-mon[31985]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T18:01:56.032 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:01:55 smithi022 ceph-mon[31985]: from='osd.8 [v2:172.21.15.134:6816/2265650388,v1:172.21.15.134:6817/2265650388]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-03-28T18:01:56.032 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:01:55 smithi022 ceph-mon[31985]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-03-28T18:01:56.182 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:01:55 smithi079 ceph-mon[35312]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 254 MiB used, 715 GiB / 715 GiB avail 2024-03-28T18:01:56.182 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:01:55 smithi079 ceph-mon[35312]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T18:01:56.182 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:01:55 smithi079 ceph-mon[35312]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T18:01:56.183 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:01:55 smithi079 ceph-mon[35312]: from='osd.8 [v2:172.21.15.134:6816/2265650388,v1:172.21.15.134:6817/2265650388]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-03-28T18:01:56.183 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:01:55 smithi079 ceph-mon[35312]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-03-28T18:01:56.914 DEBUG:teuthology.orchestra.run.smithi134:osd.8> sudo journalctl -f -n 0 -u ceph-fd220748-ed2b-11ee-95d0-87774f69a715@osd.8.service 2024-03-28T18:01:56.919 INFO:tasks.cephadm:Deploying osd.9 on smithi022 with /dev/vg_nvme/lv_1... 2024-03-28T18:01:56.919 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fd220748-ed2b-11ee-95d0-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2024-03-28T18:01:56.987 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:01:56 smithi134 ceph-mon[35037]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-03-28T18:01:56.987 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:01:56 smithi134 ceph-mon[35037]: osdmap e59: 9 total, 8 up, 9 in 2024-03-28T18:01:56.987 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:01:56 smithi134 ceph-mon[35037]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-28T18:01:56.987 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:01:56 smithi134 ceph-mon[35037]: from='osd.8 [v2:172.21.15.134:6816/2265650388,v1:172.21.15.134:6817/2265650388]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi134", "root=default"]}]: dispatch 2024-03-28T18:01:56.987 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:01:56 smithi134 ceph-mon[35037]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi134", "root=default"]}]: dispatch 2024-03-28T18:01:56.987 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:01:56 smithi134 ceph-mon[35037]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T18:01:56.987 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:01:56 smithi134 ceph-mon[35037]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T18:01:57.014 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:01:56 smithi022 ceph-mon[31985]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-03-28T18:01:57.014 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:01:56 smithi022 ceph-mon[31985]: osdmap e59: 9 total, 8 up, 9 in 2024-03-28T18:01:57.014 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:01:56 smithi022 ceph-mon[31985]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-28T18:01:57.014 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:01:56 smithi022 ceph-mon[31985]: from='osd.8 [v2:172.21.15.134:6816/2265650388,v1:172.21.15.134:6817/2265650388]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi134", "root=default"]}]: dispatch 2024-03-28T18:01:57.014 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:01:56 smithi022 ceph-mon[31985]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi134", "root=default"]}]: dispatch 2024-03-28T18:01:57.014 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:01:56 smithi022 ceph-mon[31985]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T18:01:57.014 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:01:56 smithi022 ceph-mon[31985]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T18:01:57.183 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:01:56 smithi079 ceph-mon[35312]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-03-28T18:01:57.183 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:01:56 smithi079 ceph-mon[35312]: osdmap e59: 9 total, 8 up, 9 in 2024-03-28T18:01:57.183 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:01:56 smithi079 ceph-mon[35312]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-28T18:01:57.183 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:01:56 smithi079 ceph-mon[35312]: from='osd.8 [v2:172.21.15.134:6816/2265650388,v1:172.21.15.134:6817/2265650388]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi134", "root=default"]}]: dispatch 2024-03-28T18:01:57.183 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:01:56 smithi079 ceph-mon[35312]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi134", "root=default"]}]: dispatch 2024-03-28T18:01:57.183 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:01:56 smithi079 ceph-mon[35312]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T18:01:57.183 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:01:56 smithi079 ceph-mon[35312]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T18:01:57.395 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/fd220748-ed2b-11ee-95d0-87774f69a715/mon.a/config 2024-03-28T18:01:57.845 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:01:57 smithi022 ceph-mon[31985]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 254 MiB used, 715 GiB / 715 GiB avail 2024-03-28T18:01:57.845 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:01:57 smithi022 ceph-mon[31985]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi134", "root=default"]}]': finished 2024-03-28T18:01:57.845 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:01:57 smithi022 ceph-mon[31985]: osdmap e60: 9 total, 8 up, 9 in 2024-03-28T18:01:57.845 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:01:57 smithi022 ceph-mon[31985]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-28T18:01:57.845 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:01:57 smithi022 ceph-mon[31985]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-28T18:01:57.845 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:01:57 smithi022 ceph-mon[31985]: osdmap e61: 9 total, 8 up, 9 in 2024-03-28T18:01:57.845 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:01:57 smithi022 ceph-mon[31985]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-28T18:01:57.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:01:57 smithi134 ceph-mon[35037]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 254 MiB used, 715 GiB / 715 GiB avail 2024-03-28T18:01:57.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:01:57 smithi134 ceph-mon[35037]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi134", "root=default"]}]': finished 2024-03-28T18:01:57.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:01:57 smithi134 ceph-mon[35037]: osdmap e60: 9 total, 8 up, 9 in 2024-03-28T18:01:57.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:01:57 smithi134 ceph-mon[35037]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-28T18:01:57.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:01:57 smithi134 ceph-mon[35037]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-28T18:01:57.925 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:01:57 smithi134 ceph-mon[35037]: osdmap e61: 9 total, 8 up, 9 in 2024-03-28T18:01:57.925 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:01:57 smithi134 ceph-mon[35037]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-28T18:01:57.925 INFO:journalctl@ceph.osd.8.smithi134.stdout:Mar 28 18:01:57 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-osd-8[53687]: 2024-03-28T18:01:57.727+0000 7fdb70c00700 -1 osd.8 0 waiting for initial osdmap 2024-03-28T18:01:57.925 INFO:journalctl@ceph.osd.8.smithi134.stdout:Mar 28 18:01:57 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-osd-8[53687]: 2024-03-28T18:01:57.786+0000 7fdb66000700 -1 osd.8 61 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-28T18:01:58.183 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:01:57 smithi079 ceph-mon[35312]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 254 MiB used, 715 GiB / 715 GiB avail 2024-03-28T18:01:58.183 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:01:57 smithi079 ceph-mon[35312]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi134", "root=default"]}]': finished 2024-03-28T18:01:58.183 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:01:57 smithi079 ceph-mon[35312]: osdmap e60: 9 total, 8 up, 9 in 2024-03-28T18:01:58.183 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:01:57 smithi079 ceph-mon[35312]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-28T18:01:58.183 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:01:57 smithi079 ceph-mon[35312]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-28T18:01:58.183 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:01:57 smithi079 ceph-mon[35312]: osdmap e61: 9 total, 8 up, 9 in 2024-03-28T18:01:58.183 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:01:57 smithi079 ceph-mon[35312]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-28T18:01:58.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:01:58 smithi022 ceph-mon[31985]: purged_snaps scrub starts 2024-03-28T18:01:58.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:01:58 smithi022 ceph-mon[31985]: purged_snaps scrub ok 2024-03-28T18:01:58.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:01:58 smithi022 ceph-mon[31985]: from='osd.8 ' entity='osd.8' 2024-03-28T18:01:58.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:01:58 smithi022 ceph-mon[31985]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-28T18:01:58.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:01:58 smithi022 ceph-mon[31985]: osd.8 [v2:172.21.15.134:6816/2265650388,v1:172.21.15.134:6817/2265650388] boot 2024-03-28T18:01:58.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:01:58 smithi022 ceph-mon[31985]: osdmap e62: 9 total, 9 up, 9 in 2024-03-28T18:01:58.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:01:58 smithi022 ceph-mon[31985]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-28T18:01:58.849 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:01:58 smithi134 ceph-mon[35037]: purged_snaps scrub starts 2024-03-28T18:01:58.850 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:01:58 smithi134 ceph-mon[35037]: purged_snaps scrub ok 2024-03-28T18:01:58.850 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:01:58 smithi134 ceph-mon[35037]: from='osd.8 ' entity='osd.8' 2024-03-28T18:01:58.850 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:01:58 smithi134 ceph-mon[35037]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-28T18:01:58.850 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:01:58 smithi134 ceph-mon[35037]: osd.8 [v2:172.21.15.134:6816/2265650388,v1:172.21.15.134:6817/2265650388] boot 2024-03-28T18:01:58.850 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:01:58 smithi134 ceph-mon[35037]: osdmap e62: 9 total, 9 up, 9 in 2024-03-28T18:01:58.850 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:01:58 smithi134 ceph-mon[35037]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-28T18:01:59.183 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:01:58 smithi079 ceph-mon[35312]: purged_snaps scrub starts 2024-03-28T18:01:59.183 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:01:58 smithi079 ceph-mon[35312]: purged_snaps scrub ok 2024-03-28T18:01:59.183 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:01:58 smithi079 ceph-mon[35312]: from='osd.8 ' entity='osd.8' 2024-03-28T18:01:59.183 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:01:58 smithi079 ceph-mon[35312]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-28T18:01:59.183 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:01:58 smithi079 ceph-mon[35312]: osd.8 [v2:172.21.15.134:6816/2265650388,v1:172.21.15.134:6817/2265650388] boot 2024-03-28T18:01:59.183 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:01:58 smithi079 ceph-mon[35312]: osdmap e62: 9 total, 9 up, 9 in 2024-03-28T18:01:59.183 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:01:58 smithi079 ceph-mon[35312]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-28T18:02:00.587 INFO:teuthology.orchestra.run.smithi022.stderr:--> Zapping: /dev/vg_nvme/lv_1 2024-03-28T18:02:00.587 INFO:teuthology.orchestra.run.smithi022.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2024-03-28T18:02:00.588 INFO:teuthology.orchestra.run.smithi022.stderr: stderr: 10+0 records in 2024-03-28T18:02:00.588 INFO:teuthology.orchestra.run.smithi022.stderr:10+0 records out 2024-03-28T18:02:00.588 INFO:teuthology.orchestra.run.smithi022.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0339629 s, 309 MB/s 2024-03-28T18:02:00.588 INFO:teuthology.orchestra.run.smithi022.stderr:--> Zapping successful for: 2024-03-28T18:02:00.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:02:00 smithi022 ceph-mon[31985]: pgmap v256: 1 pgs: 1 remapped+peering; 577 KiB data, 254 MiB used, 715 GiB / 715 GiB avail 2024-03-28T18:02:00.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:02:00 smithi022 ceph-mon[31985]: osdmap e63: 9 total, 9 up, 9 in 2024-03-28T18:02:00.925 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:02:00 smithi134 ceph-mon[35037]: pgmap v256: 1 pgs: 1 remapped+peering; 577 KiB data, 254 MiB used, 715 GiB / 715 GiB avail 2024-03-28T18:02:00.925 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:02:00 smithi134 ceph-mon[35037]: osdmap e63: 9 total, 9 up, 9 in 2024-03-28T18:02:00.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:02:00 smithi079 ceph-mon[35312]: pgmap v256: 1 pgs: 1 remapped+peering; 577 KiB data, 254 MiB used, 715 GiB / 715 GiB avail 2024-03-28T18:02:00.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:02:00 smithi079 ceph-mon[35312]: osdmap e63: 9 total, 9 up, 9 in 2024-03-28T18:02:01.523 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fd220748-ed2b-11ee-95d0-87774f69a715 -- ceph orch daemon add osd smithi022:vg_nvme/lv_1 2024-03-28T18:02:01.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:02:01 smithi134 ceph-mon[35037]: osdmap e64: 9 total, 9 up, 9 in 2024-03-28T18:02:01.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:02:01 smithi134 ceph-mon[35037]: pgmap v259: 1 pgs: 1 remapped+peering; 577 KiB data, 285 MiB used, 804 GiB / 805 GiB avail 2024-03-28T18:02:01.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:02:01 smithi134 ceph-mon[35037]: Detected new or changed devices on smithi134 2024-03-28T18:02:01.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:02:01 smithi134 ceph-mon[35037]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T18:02:01.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:02:01 smithi134 ceph-mon[35037]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T18:02:01.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:02:01 smithi134 ceph-mon[35037]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-28T18:02:01.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:02:01 smithi134 ceph-mon[35037]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-28T18:02:01.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:02:01 smithi134 ceph-mon[35037]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-03-28T18:02:01.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:02:01 smithi134 ceph-mon[35037]: Adjusting osd_memory_target on smithi134 to 5697M 2024-03-28T18:02:01.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:02:01 smithi134 ceph-mon[35037]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T18:02:01.676 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:02:01 smithi134 ceph-mon[35037]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T18:02:01.676 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:02:01 smithi134 ceph-mon[35037]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T18:02:01.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:02:01 smithi022 ceph-mon[31985]: osdmap e64: 9 total, 9 up, 9 in 2024-03-28T18:02:01.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:02:01 smithi022 ceph-mon[31985]: pgmap v259: 1 pgs: 1 remapped+peering; 577 KiB data, 285 MiB used, 804 GiB / 805 GiB avail 2024-03-28T18:02:01.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:02:01 smithi022 ceph-mon[31985]: Detected new or changed devices on smithi134 2024-03-28T18:02:01.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:02:01 smithi022 ceph-mon[31985]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T18:02:01.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:02:01 smithi022 ceph-mon[31985]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T18:02:01.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:02:01 smithi022 ceph-mon[31985]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-28T18:02:01.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:02:01 smithi022 ceph-mon[31985]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-28T18:02:01.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:02:01 smithi022 ceph-mon[31985]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-03-28T18:02:01.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:02:01 smithi022 ceph-mon[31985]: Adjusting osd_memory_target on smithi134 to 5697M 2024-03-28T18:02:01.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:02:01 smithi022 ceph-mon[31985]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T18:02:01.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:02:01 smithi022 ceph-mon[31985]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T18:02:01.783 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:02:01 smithi022 ceph-mon[31985]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T18:02:01.933 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:02:01 smithi079 ceph-mon[35312]: osdmap e64: 9 total, 9 up, 9 in 2024-03-28T18:02:01.933 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:02:01 smithi079 ceph-mon[35312]: pgmap v259: 1 pgs: 1 remapped+peering; 577 KiB data, 285 MiB used, 804 GiB / 805 GiB avail 2024-03-28T18:02:01.933 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:02:01 smithi079 ceph-mon[35312]: Detected new or changed devices on smithi134 2024-03-28T18:02:01.933 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:02:01 smithi079 ceph-mon[35312]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T18:02:01.933 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:02:01 smithi079 ceph-mon[35312]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T18:02:01.933 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:02:01 smithi079 ceph-mon[35312]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-28T18:02:01.933 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:02:01 smithi079 ceph-mon[35312]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-28T18:02:01.933 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:02:01 smithi079 ceph-mon[35312]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-03-28T18:02:01.933 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:02:01 smithi079 ceph-mon[35312]: Adjusting osd_memory_target on smithi134 to 5697M 2024-03-28T18:02:01.933 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:02:01 smithi079 ceph-mon[35312]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T18:02:01.934 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:02:01 smithi079 ceph-mon[35312]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T18:02:01.934 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:02:01 smithi079 ceph-mon[35312]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T18:02:01.966 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/fd220748-ed2b-11ee-95d0-87774f69a715/mon.a/config 2024-03-28T18:02:02.683 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:02:02 smithi022 ceph-mon[31985]: osdmap e65: 9 total, 9 up, 9 in 2024-03-28T18:02:02.683 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:02:02 smithi022 ceph-mon[31985]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T18:02:02.683 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:02:02 smithi022 ceph-mon[31985]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T18:02:02.683 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:02:02 smithi022 ceph-mon[31985]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T18:02:02.684 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:02:02 smithi022 ceph-mon[31985]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T18:02:02.684 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:02:02 smithi022 ceph-mon[31985]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T18:02:02.925 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:02:02 smithi134 ceph-mon[35037]: osdmap e65: 9 total, 9 up, 9 in 2024-03-28T18:02:02.925 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:02:02 smithi134 ceph-mon[35037]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T18:02:02.925 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:02:02 smithi134 ceph-mon[35037]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T18:02:02.925 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:02:02 smithi134 ceph-mon[35037]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T18:02:02.925 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:02:02 smithi134 ceph-mon[35037]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T18:02:02.925 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:02:02 smithi134 ceph-mon[35037]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T18:02:02.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:02:02 smithi079 ceph-mon[35312]: osdmap e65: 9 total, 9 up, 9 in 2024-03-28T18:02:02.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:02:02 smithi079 ceph-mon[35312]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T18:02:02.933 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:02:02 smithi079 ceph-mon[35312]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T18:02:02.933 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:02:02 smithi079 ceph-mon[35312]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T18:02:02.933 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:02:02 smithi079 ceph-mon[35312]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T18:02:02.933 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:02:02 smithi079 ceph-mon[35312]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T18:02:03.781 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:02:03 smithi022 ceph-mon[31985]: osdmap e66: 9 total, 9 up, 9 in 2024-03-28T18:02:03.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:02:03 smithi022 ceph-mon[31985]: pgmap v262: 1 pgs: 1 remapped+peering; 577 KiB data, 285 MiB used, 804 GiB / 805 GiB avail 2024-03-28T18:02:03.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:02:03 smithi134 ceph-mon[35037]: osdmap e66: 9 total, 9 up, 9 in 2024-03-28T18:02:03.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:02:03 smithi134 ceph-mon[35037]: pgmap v262: 1 pgs: 1 remapped+peering; 577 KiB data, 285 MiB used, 804 GiB / 805 GiB avail 2024-03-28T18:02:03.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:02:03 smithi079 ceph-mon[35312]: osdmap e66: 9 total, 9 up, 9 in 2024-03-28T18:02:03.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:02:03 smithi079 ceph-mon[35312]: pgmap v262: 1 pgs: 1 remapped+peering; 577 KiB data, 285 MiB used, 804 GiB / 805 GiB avail 2024-03-28T18:02:04.352 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:02:04.351+0000 7f0b61c00700 1 -- 172.21.15.22:0/2013420177 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0b5c0761c0 msgr2=0x7f0b5c074620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:02:04.352 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:02:04.351+0000 7f0b61c00700 1 --2- 172.21.15.22:0/2013420177 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0b5c0761c0 0x7f0b5c074620 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f0b4c009990 tx=0x7f0b4c009ca0 comp rx=0 tx=0).stop 2024-03-28T18:02:04.352 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:02:04.352+0000 7f0b61c00700 1 -- 172.21.15.22:0/2013420177 shutdown_connections 2024-03-28T18:02:04.352 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:02:04.352+0000 7f0b61c00700 1 --2- 172.21.15.22:0/2013420177 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f0b5c10e6c0 0x7f0b5c110ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:02:04.352 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:02:04.352+0000 7f0b61c00700 1 --2- 172.21.15.22:0/2013420177 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0b5c0761c0 0x7f0b5c074620 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:02:04.352 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:02:04.352+0000 7f0b61c00700 1 --2- 172.21.15.22:0/2013420177 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0b5c074bf0 0x7f0b5c075070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:02:04.352 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:02:04.352+0000 7f0b61c00700 1 -- 172.21.15.22:0/2013420177 >> 172.21.15.22:0/2013420177 conn(0x7f0b5c0fd6a0 msgr2=0x7f0b5c0ffac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:02:04.352 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:02:04.352+0000 7f0b61c00700 1 -- 172.21.15.22:0/2013420177 shutdown_connections 2024-03-28T18:02:04.353 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:02:04.352+0000 7f0b61c00700 1 -- 172.21.15.22:0/2013420177 wait complete. 2024-03-28T18:02:04.353 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:02:04.353+0000 7f0b61c00700 1 Processor -- start 2024-03-28T18:02:04.353 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:02:04.353+0000 7f0b61c00700 1 -- start start 2024-03-28T18:02:04.355 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:02:04.354+0000 7f0b61c00700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0b5c074bf0 0x7f0b5c19e8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:02:04.355 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:02:04.354+0000 7f0b61c00700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0b5c0761c0 0x7f0b5c19ee30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:02:04.355 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:02:04.354+0000 7f0b61c00700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f0b5c10e6c0 0x7f0b5c1a5eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:02:04.355 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:02:04.354+0000 7f0b61c00700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b5c1142b0 con 0x7f0b5c0761c0 2024-03-28T18:02:04.355 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:02:04.354+0000 7f0b61c00700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b5c114130 con 0x7f0b5c10e6c0 2024-03-28T18:02:04.355 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:02:04.354+0000 7f0b61c00700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b5c114430 con 0x7f0b5c074bf0 2024-03-28T18:02:04.356 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:02:04.354+0000 7f0b5b400700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0b5c0761c0 0x7f0b5c19ee30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:02:04.356 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:02:04.354+0000 7f0b5b400700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0b5c0761c0 0x7f0b5c19ee30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:39694/0 (socket says 172.21.15.22:39694) 2024-03-28T18:02:04.356 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:02:04.354+0000 7f0b5b400700 1 -- 172.21.15.22:0/524860648 learned_addr learned my addr 172.21.15.22:0/524860648 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-28T18:02:04.356 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:02:04.354+0000 7f0b5be00700 1 --2- 172.21.15.22:0/524860648 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0b5c074bf0 0x7f0b5c19e8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:02:04.356 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:02:04.355+0000 7f0b61200700 1 --2- 172.21.15.22:0/524860648 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f0b5c10e6c0 0x7f0b5c1a5eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:02:04.356 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:02:04.355+0000 7f0b5b400700 1 -- 172.21.15.22:0/524860648 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0b5c074bf0 msgr2=0x7f0b5c19e8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:02:04.356 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:02:04.355+0000 7f0b5b400700 1 --2- 172.21.15.22:0/524860648 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0b5c074bf0 0x7f0b5c19e8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:02:04.356 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:02:04.355+0000 7f0b5b400700 1 -- 172.21.15.22:0/524860648 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f0b5c10e6c0 msgr2=0x7f0b5c1a5eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:02:04.356 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:02:04.355+0000 7f0b5b400700 1 --2- 172.21.15.22:0/524860648 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f0b5c10e6c0 0x7f0b5c1a5eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:02:04.356 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:02:04.355+0000 7f0b5b400700 1 -- 172.21.15.22:0/524860648 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0b5c1a6550 con 0x7f0b5c0761c0 2024-03-28T18:02:04.356 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:02:04.355+0000 7f0b61200700 1 --2- 172.21.15.22:0/524860648 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f0b5c10e6c0 0x7f0b5c1a5eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-28T18:02:04.357 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:02:04.355+0000 7f0b5b400700 1 --2- 172.21.15.22:0/524860648 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0b5c0761c0 0x7f0b5c19ee30 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f0b4800d9b0 tx=0x7f0b4800dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:02:04.357 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:02:04.356+0000 7f0b58c00700 1 -- 172.21.15.22:0/524860648 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b48017070 con 0x7f0b5c0761c0 2024-03-28T18:02:04.357 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:02:04.356+0000 7f0b58c00700 1 -- 172.21.15.22:0/524860648 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f0b48009e00 con 0x7f0b5c0761c0 2024-03-28T18:02:04.358 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:02:04.356+0000 7f0b61c00700 1 -- 172.21.15.22:0/524860648 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0b5c1a6840 con 0x7f0b5c0761c0 2024-03-28T18:02:04.358 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:02:04.356+0000 7f0b58c00700 1 -- 172.21.15.22:0/524860648 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b4800f620 con 0x7f0b5c0761c0 2024-03-28T18:02:04.358 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:02:04.356+0000 7f0b61c00700 1 -- 172.21.15.22:0/524860648 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0b5c1a6d30 con 0x7f0b5c0761c0 2024-03-28T18:02:04.358 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:02:04.357+0000 7f0b61c00700 1 -- 172.21.15.22:0/524860648 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0b5c068e50 con 0x7f0b5c0761c0 2024-03-28T18:02:04.359 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:02:04.359+0000 7f0b58c00700 1 -- 172.21.15.22:0/524860648 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 16) v1 ==== 137290+0+0 (secure 0 0 0) 0x7f0b48010460 con 0x7f0b5c0761c0 2024-03-28T18:02:04.360 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:02:04.360+0000 7f0b58c00700 1 --2- 172.21.15.22:0/524860648 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7f0b400a2890 0x7f0b400a4d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:02:04.360 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:02:04.361+0000 7f0b58c00700 1 -- 172.21.15.22:0/524860648 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(66..66 src has 1..66) v4 ==== 5829+0+0 (secure 0 0 0) 0x7f0b480ce040 con 0x7f0b5c0761c0 2024-03-28T18:02:04.361 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:02:04.361+0000 7f0b5be00700 1 --2- 172.21.15.22:0/524860648 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7f0b400a2890 0x7f0b400a4d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:02:04.361 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:02:04.361+0000 7f0b5be00700 1 --2- 172.21.15.22:0/524860648 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7f0b400a2890 0x7f0b400a4d50 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f0b4c003890 tx=0x7f0b4c003960 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:02:04.363 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:02:04.364+0000 7f0b58c00700 1 -- 172.21.15.22:0/524860648 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f0b48010e20 con 0x7f0b5c0761c0 2024-03-28T18:02:04.601 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:02:04.601+0000 7f0b61c00700 1 -- 172.21.15.22:0/524860648 --> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi022:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) v1 -- 0x7f0b5c0625d0 con 0x7f0b400a2890 2024-03-28T18:02:05.904 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:02:05 smithi022 ceph-mon[31985]: pgmap v263: 1 pgs: 1 active+clean; 577 KiB data, 286 MiB used, 804 GiB / 805 GiB avail; 114 KiB/s, 0 objects/s recovering 2024-03-28T18:02:05.904 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:02:05 smithi022 ceph-mon[31985]: from='client.14415 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi022:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-03-28T18:02:05.904 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:02:05 smithi022 ceph-mon[31985]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-28T18:02:05.904 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:02:05 smithi022 ceph-mon[31985]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-28T18:02:05.904 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:02:05 smithi022 ceph-mon[31985]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T18:02:05.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:02:05 smithi134 ceph-mon[35037]: pgmap v263: 1 pgs: 1 active+clean; 577 KiB data, 286 MiB used, 804 GiB / 805 GiB avail; 114 KiB/s, 0 objects/s recovering 2024-03-28T18:02:05.925 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:02:05 smithi134 ceph-mon[35037]: from='client.14415 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi022:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-03-28T18:02:05.925 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:02:05 smithi134 ceph-mon[35037]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-28T18:02:05.925 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:02:05 smithi134 ceph-mon[35037]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-28T18:02:05.925 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:02:05 smithi134 ceph-mon[35037]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T18:02:05.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:02:05 smithi079 ceph-mon[35312]: pgmap v263: 1 pgs: 1 active+clean; 577 KiB data, 286 MiB used, 804 GiB / 805 GiB avail; 114 KiB/s, 0 objects/s recovering 2024-03-28T18:02:05.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:02:05 smithi079 ceph-mon[35312]: from='client.14415 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi022:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-03-28T18:02:05.933 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:02:05 smithi079 ceph-mon[35312]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-28T18:02:05.933 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:02:05 smithi079 ceph-mon[35312]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-28T18:02:05.933 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:02:05 smithi079 ceph-mon[35312]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T18:02:07.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:02:07 smithi134 ceph-mon[35037]: pgmap v264: 1 pgs: 1 active+clean; 577 KiB data, 286 MiB used, 804 GiB / 805 GiB avail; 95 KiB/s, 0 objects/s recovering 2024-03-28T18:02:07.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:02:07 smithi079 ceph-mon[35312]: pgmap v264: 1 pgs: 1 active+clean; 577 KiB data, 286 MiB used, 804 GiB / 805 GiB avail; 95 KiB/s, 0 objects/s recovering 2024-03-28T18:02:08.032 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:02:07 smithi022 ceph-mon[31985]: pgmap v264: 1 pgs: 1 active+clean; 577 KiB data, 286 MiB used, 804 GiB / 805 GiB avail; 95 KiB/s, 0 objects/s recovering 2024-03-28T18:02:09.925 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:02:09 smithi134 ceph-mon[35037]: pgmap v265: 1 pgs: 1 active+clean; 577 KiB data, 286 MiB used, 804 GiB / 805 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-03-28T18:02:09.925 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:02:09 smithi134 ceph-mon[35037]: from='client.? 172.21.15.22:0/3239904368' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e1ae4020-7c2c-4334-a753-ee6b12d98584"}]: dispatch 2024-03-28T18:02:09.925 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:02:09 smithi134 ceph-mon[35037]: from='client.? 172.21.15.22:0/3239904368' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "e1ae4020-7c2c-4334-a753-ee6b12d98584"}]': finished 2024-03-28T18:02:09.925 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:02:09 smithi134 ceph-mon[35037]: osdmap e67: 10 total, 9 up, 10 in 2024-03-28T18:02:09.925 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:02:09 smithi134 ceph-mon[35037]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-28T18:02:09.925 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:02:09 smithi134 ceph-mon[35037]: from='client.? 172.21.15.22:0/787101934' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-28T18:02:09.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:02:09 smithi079 ceph-mon[35312]: pgmap v265: 1 pgs: 1 active+clean; 577 KiB data, 286 MiB used, 804 GiB / 805 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-03-28T18:02:09.933 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:02:09 smithi079 ceph-mon[35312]: from='client.? 172.21.15.22:0/3239904368' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e1ae4020-7c2c-4334-a753-ee6b12d98584"}]: dispatch 2024-03-28T18:02:09.933 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:02:09 smithi079 ceph-mon[35312]: from='client.? 172.21.15.22:0/3239904368' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "e1ae4020-7c2c-4334-a753-ee6b12d98584"}]': finished 2024-03-28T18:02:09.933 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:02:09 smithi079 ceph-mon[35312]: osdmap e67: 10 total, 9 up, 10 in 2024-03-28T18:02:09.933 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:02:09 smithi079 ceph-mon[35312]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-28T18:02:09.933 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:02:09 smithi079 ceph-mon[35312]: from='client.? 172.21.15.22:0/787101934' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-28T18:02:10.032 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:02:09 smithi022 ceph-mon[31985]: pgmap v265: 1 pgs: 1 active+clean; 577 KiB data, 286 MiB used, 804 GiB / 805 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-03-28T18:02:10.032 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:02:09 smithi022 ceph-mon[31985]: from='client.? 172.21.15.22:0/3239904368' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e1ae4020-7c2c-4334-a753-ee6b12d98584"}]: dispatch 2024-03-28T18:02:10.032 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:02:09 smithi022 ceph-mon[31985]: from='client.? 172.21.15.22:0/3239904368' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "e1ae4020-7c2c-4334-a753-ee6b12d98584"}]': finished 2024-03-28T18:02:10.032 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:02:09 smithi022 ceph-mon[31985]: osdmap e67: 10 total, 9 up, 10 in 2024-03-28T18:02:10.033 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:02:09 smithi022 ceph-mon[31985]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-28T18:02:10.033 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:02:09 smithi022 ceph-mon[31985]: from='client.? 172.21.15.22:0/787101934' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-28T18:02:11.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:02:11 smithi134 ceph-mon[35037]: pgmap v267: 1 pgs: 1 active+clean; 577 KiB data, 286 MiB used, 804 GiB / 805 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-03-28T18:02:11.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:02:11 smithi079 ceph-mon[35312]: pgmap v267: 1 pgs: 1 active+clean; 577 KiB data, 286 MiB used, 804 GiB / 805 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-03-28T18:02:12.031 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:02:11 smithi022 ceph-mon[31985]: pgmap v267: 1 pgs: 1 active+clean; 577 KiB data, 286 MiB used, 804 GiB / 805 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-03-28T18:02:13.781 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:02:13 smithi022 ceph-mon[31985]: pgmap v268: 1 pgs: 1 active+clean; 577 KiB data, 290 MiB used, 804 GiB / 805 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-03-28T18:02:13.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:02:13 smithi134 ceph-mon[35037]: pgmap v268: 1 pgs: 1 active+clean; 577 KiB data, 290 MiB used, 804 GiB / 805 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-03-28T18:02:13.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:02:13 smithi079 ceph-mon[35312]: pgmap v268: 1 pgs: 1 active+clean; 577 KiB data, 290 MiB used, 804 GiB / 805 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-03-28T18:02:15.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:02:15 smithi134 ceph-mon[35037]: pgmap v269: 1 pgs: 1 active+clean; 577 KiB data, 290 MiB used, 804 GiB / 805 GiB avail 2024-03-28T18:02:15.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:02:15 smithi079 ceph-mon[35312]: pgmap v269: 1 pgs: 1 active+clean; 577 KiB data, 290 MiB used, 804 GiB / 805 GiB avail 2024-03-28T18:02:15.945 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:02:15 smithi022 ceph-mon[31985]: pgmap v269: 1 pgs: 1 active+clean; 577 KiB data, 290 MiB used, 804 GiB / 805 GiB avail 2024-03-28T18:02:17.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:02:17 smithi134 ceph-mon[35037]: pgmap v270: 1 pgs: 1 active+clean; 577 KiB data, 290 MiB used, 804 GiB / 805 GiB avail 2024-03-28T18:02:17.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:02:17 smithi079 ceph-mon[35312]: pgmap v270: 1 pgs: 1 active+clean; 577 KiB data, 290 MiB used, 804 GiB / 805 GiB avail 2024-03-28T18:02:18.032 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:02:17 smithi022 ceph-mon[31985]: pgmap v270: 1 pgs: 1 active+clean; 577 KiB data, 290 MiB used, 804 GiB / 805 GiB avail 2024-03-28T18:02:18.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:02:18 smithi134 ceph-mon[35037]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-03-28T18:02:18.925 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:02:18 smithi134 ceph-mon[35037]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T18:02:18.925 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:02:18 smithi134 ceph-mon[35037]: Deploying daemon osd.9 on smithi022 2024-03-28T18:02:18.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:02:18 smithi079 ceph-mon[35312]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-03-28T18:02:18.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:02:18 smithi079 ceph-mon[35312]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T18:02:18.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:02:18 smithi079 ceph-mon[35312]: Deploying daemon osd.9 on smithi022 2024-03-28T18:02:19.031 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:02:18 smithi022 ceph-mon[31985]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-03-28T18:02:19.032 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:02:18 smithi022 ceph-mon[31985]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T18:02:19.032 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:02:18 smithi022 ceph-mon[31985]: Deploying daemon osd.9 on smithi022 2024-03-28T18:02:19.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:02:19 smithi134 ceph-mon[35037]: pgmap v271: 1 pgs: 1 active+clean; 577 KiB data, 290 MiB used, 804 GiB / 805 GiB avail 2024-03-28T18:02:19.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:02:19 smithi079 ceph-mon[35312]: pgmap v271: 1 pgs: 1 active+clean; 577 KiB data, 290 MiB used, 804 GiB / 805 GiB avail 2024-03-28T18:02:20.031 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:02:19 smithi022 ceph-mon[31985]: pgmap v271: 1 pgs: 1 active+clean; 577 KiB data, 290 MiB used, 804 GiB / 805 GiB avail 2024-03-28T18:02:21.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:02:21 smithi134 ceph-mon[35037]: pgmap v272: 1 pgs: 1 active+clean; 577 KiB data, 290 MiB used, 804 GiB / 805 GiB avail 2024-03-28T18:02:21.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:02:21 smithi079 ceph-mon[35312]: pgmap v272: 1 pgs: 1 active+clean; 577 KiB data, 290 MiB used, 804 GiB / 805 GiB avail 2024-03-28T18:02:22.032 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:02:21 smithi022 ceph-mon[31985]: pgmap v272: 1 pgs: 1 active+clean; 577 KiB data, 290 MiB used, 804 GiB / 805 GiB avail 2024-03-28T18:02:23.906 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:02:23 smithi022 ceph-mon[31985]: pgmap v273: 1 pgs: 1 active+clean; 577 KiB data, 290 MiB used, 804 GiB / 805 GiB avail 2024-03-28T18:02:23.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:02:23 smithi134 ceph-mon[35037]: pgmap v273: 1 pgs: 1 active+clean; 577 KiB data, 290 MiB used, 804 GiB / 805 GiB avail 2024-03-28T18:02:23.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:02:23 smithi079 ceph-mon[35312]: pgmap v273: 1 pgs: 1 active+clean; 577 KiB data, 290 MiB used, 804 GiB / 805 GiB avail 2024-03-28T18:02:25.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:02:25 smithi022 ceph-mon[31985]: pgmap v274: 1 pgs: 1 active+clean; 577 KiB data, 290 MiB used, 804 GiB / 805 GiB avail 2024-03-28T18:02:25.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:02:25 smithi134 ceph-mon[35037]: pgmap v274: 1 pgs: 1 active+clean; 577 KiB data, 290 MiB used, 804 GiB / 805 GiB avail 2024-03-28T18:02:25.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:02:25 smithi079 ceph-mon[35312]: pgmap v274: 1 pgs: 1 active+clean; 577 KiB data, 290 MiB used, 804 GiB / 805 GiB avail 2024-03-28T18:02:28.014 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:02:27 smithi022 ceph-mon[31985]: pgmap v275: 1 pgs: 1 active+clean; 577 KiB data, 290 MiB used, 804 GiB / 805 GiB avail 2024-03-28T18:02:28.015 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:02:27 smithi022 ceph-mon[31985]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T18:02:28.015 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:02:27 smithi022 ceph-mon[31985]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T18:02:28.015 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:02:27 smithi022 ceph-mon[31985]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T18:02:28.174 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:02:27 smithi134 ceph-mon[35037]: pgmap v275: 1 pgs: 1 active+clean; 577 KiB data, 290 MiB used, 804 GiB / 805 GiB avail 2024-03-28T18:02:28.175 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:02:27 smithi134 ceph-mon[35037]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T18:02:28.175 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:02:27 smithi134 ceph-mon[35037]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T18:02:28.175 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:02:27 smithi134 ceph-mon[35037]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T18:02:28.182 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:02:27 smithi079 ceph-mon[35312]: pgmap v275: 1 pgs: 1 active+clean; 577 KiB data, 290 MiB used, 804 GiB / 805 GiB avail 2024-03-28T18:02:28.182 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:02:27 smithi079 ceph-mon[35312]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T18:02:28.183 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:02:27 smithi079 ceph-mon[35312]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T18:02:28.183 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:02:27 smithi079 ceph-mon[35312]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T18:02:29.847 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:02:29 smithi134 ceph-mon[35037]: pgmap v276: 1 pgs: 1 active+clean; 577 KiB data, 290 MiB used, 804 GiB / 805 GiB avail 2024-03-28T18:02:30.004 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:02:29 smithi022 ceph-mon[31985]: pgmap v276: 1 pgs: 1 active+clean; 577 KiB data, 290 MiB used, 804 GiB / 805 GiB avail 2024-03-28T18:02:30.182 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:02:29 smithi079 ceph-mon[35312]: pgmap v276: 1 pgs: 1 active+clean; 577 KiB data, 290 MiB used, 804 GiB / 805 GiB avail 2024-03-28T18:02:31.712 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:02:31 smithi022 ceph-mon[31985]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T18:02:31.712 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:02:31 smithi022 ceph-mon[31985]: pgmap v277: 1 pgs: 1 active+clean; 577 KiB data, 290 MiB used, 804 GiB / 805 GiB avail 2024-03-28T18:02:31.712 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:02:31 smithi022 ceph-mon[31985]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T18:02:31.712 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:02:31 smithi022 ceph-mon[31985]: from='osd.9 [v2:172.21.15.22:6826/3762222259,v1:172.21.15.22:6827/3762222259]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-03-28T18:02:31.799 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:02:31.799+0000 7f0b58c00700 1 -- 172.21.15.22:0/524860648 <== mgr.14152 v2:172.21.15.22:6800/4201048392 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f0b5c0625d0 con 0x7f0b400a2890 2024-03-28T18:02:31.800 INFO:teuthology.orchestra.run.smithi022.stdout:Created osd(s) 9 on host 'smithi022' 2024-03-28T18:02:31.803 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:02:31.802+0000 7f0b61c00700 1 -- 172.21.15.22:0/524860648 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7f0b400a2890 msgr2=0x7f0b400a4d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:02:31.803 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:02:31.802+0000 7f0b61c00700 1 --2- 172.21.15.22:0/524860648 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7f0b400a2890 0x7f0b400a4d50 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f0b4c003890 tx=0x7f0b4c003960 comp rx=0 tx=0).stop 2024-03-28T18:02:31.803 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:02:31.802+0000 7f0b61c00700 1 -- 172.21.15.22:0/524860648 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0b5c0761c0 msgr2=0x7f0b5c19ee30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:02:31.803 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:02:31.802+0000 7f0b61c00700 1 --2- 172.21.15.22:0/524860648 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0b5c0761c0 0x7f0b5c19ee30 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f0b4800d9b0 tx=0x7f0b4800dcc0 comp rx=0 tx=0).stop 2024-03-28T18:02:31.803 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:02:31.803+0000 7f0b61c00700 1 -- 172.21.15.22:0/524860648 shutdown_connections 2024-03-28T18:02:31.803 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:02:31.803+0000 7f0b61c00700 1 --2- 172.21.15.22:0/524860648 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f0b5c10e6c0 0x7f0b5c1a5eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:02:31.804 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:02:31.803+0000 7f0b61c00700 1 --2- 172.21.15.22:0/524860648 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7f0b400a2890 0x7f0b400a4d50 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:02:31.804 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:02:31.803+0000 7f0b61c00700 1 --2- 172.21.15.22:0/524860648 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0b5c074bf0 0x7f0b5c19e8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:02:31.804 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:02:31.803+0000 7f0b61c00700 1 --2- 172.21.15.22:0/524860648 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0b5c0761c0 0x7f0b5c19ee30 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:02:31.804 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:02:31.803+0000 7f0b61c00700 1 -- 172.21.15.22:0/524860648 >> 172.21.15.22:0/524860648 conn(0x7f0b5c0fd6a0 msgr2=0x7f0b5c101a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:02:31.804 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:02:31.803+0000 7f0b61c00700 1 -- 172.21.15.22:0/524860648 shutdown_connections 2024-03-28T18:02:31.804 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:02:31.803+0000 7f0b61c00700 1 -- 172.21.15.22:0/524860648 wait complete. 2024-03-28T18:02:31.925 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:02:31 smithi134 ceph-mon[35037]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T18:02:31.925 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:02:31 smithi134 ceph-mon[35037]: pgmap v277: 1 pgs: 1 active+clean; 577 KiB data, 290 MiB used, 804 GiB / 805 GiB avail 2024-03-28T18:02:31.926 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:02:31 smithi134 ceph-mon[35037]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T18:02:31.926 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:02:31 smithi134 ceph-mon[35037]: from='osd.9 [v2:172.21.15.22:6826/3762222259,v1:172.21.15.22:6827/3762222259]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-03-28T18:02:31.933 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:02:31 smithi079 ceph-mon[35312]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T18:02:31.933 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:02:31 smithi079 ceph-mon[35312]: pgmap v277: 1 pgs: 1 active+clean; 577 KiB data, 290 MiB used, 804 GiB / 805 GiB avail 2024-03-28T18:02:31.933 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:02:31 smithi079 ceph-mon[35312]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T18:02:31.933 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:02:31 smithi079 ceph-mon[35312]: from='osd.9 [v2:172.21.15.22:6826/3762222259,v1:172.21.15.22:6827/3762222259]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-03-28T18:02:32.697 DEBUG:teuthology.orchestra.run.smithi022:osd.9> sudo journalctl -f -n 0 -u ceph-fd220748-ed2b-11ee-95d0-87774f69a715@osd.9.service 2024-03-28T18:02:32.702 INFO:tasks.cephadm:Deploying osd.10 on smithi079 with /dev/vg_nvme/lv_1... 2024-03-28T18:02:32.703 DEBUG:teuthology.orchestra.run.smithi079:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fd220748-ed2b-11ee-95d0-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2024-03-28T18:02:32.904 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:02:32 smithi022 ceph-mon[31985]: from='osd.9 [v2:172.21.15.22:6826/3762222259,v1:172.21.15.22:6827/3762222259]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-03-28T18:02:32.904 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:02:32 smithi022 ceph-mon[31985]: osdmap e68: 10 total, 9 up, 10 in 2024-03-28T18:02:32.904 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:02:32 smithi022 ceph-mon[31985]: from='osd.9 [v2:172.21.15.22:6826/3762222259,v1:172.21.15.22:6827/3762222259]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi022", "root=default"]}]: dispatch 2024-03-28T18:02:32.904 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:02:32 smithi022 ceph-mon[31985]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-28T18:02:32.904 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:02:32 smithi022 ceph-mon[31985]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T18:02:32.905 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:02:32 smithi022 ceph-mon[31985]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T18:02:32.925 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:02:32 smithi134 ceph-mon[35037]: from='osd.9 [v2:172.21.15.22:6826/3762222259,v1:172.21.15.22:6827/3762222259]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-03-28T18:02:32.925 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:02:32 smithi134 ceph-mon[35037]: osdmap e68: 10 total, 9 up, 10 in 2024-03-28T18:02:32.925 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:02:32 smithi134 ceph-mon[35037]: from='osd.9 [v2:172.21.15.22:6826/3762222259,v1:172.21.15.22:6827/3762222259]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi022", "root=default"]}]: dispatch 2024-03-28T18:02:32.925 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:02:32 smithi134 ceph-mon[35037]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-28T18:02:32.925 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:02:32 smithi134 ceph-mon[35037]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T18:02:32.925 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:02:32 smithi134 ceph-mon[35037]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T18:02:32.933 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:02:32 smithi079 ceph-mon[35312]: from='osd.9 [v2:172.21.15.22:6826/3762222259,v1:172.21.15.22:6827/3762222259]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-03-28T18:02:32.933 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:02:32 smithi079 ceph-mon[35312]: osdmap e68: 10 total, 9 up, 10 in 2024-03-28T18:02:32.933 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:02:32 smithi079 ceph-mon[35312]: from='osd.9 [v2:172.21.15.22:6826/3762222259,v1:172.21.15.22:6827/3762222259]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi022", "root=default"]}]: dispatch 2024-03-28T18:02:32.933 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:02:32 smithi079 ceph-mon[35312]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-28T18:02:32.933 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:02:32 smithi079 ceph-mon[35312]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T18:02:32.933 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:02:32 smithi079 ceph-mon[35312]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T18:02:33.153 INFO:teuthology.orchestra.run.smithi079.stderr:Inferring config /var/lib/ceph/fd220748-ed2b-11ee-95d0-87774f69a715/mon.b/config 2024-03-28T18:02:33.867 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:02:33 smithi022 ceph-mon[31985]: pgmap v279: 1 pgs: 1 active+clean; 577 KiB data, 290 MiB used, 804 GiB / 805 GiB avail 2024-03-28T18:02:33.868 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:02:33 smithi022 ceph-mon[31985]: from='osd.9 [v2:172.21.15.22:6826/3762222259,v1:172.21.15.22:6827/3762222259]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi022", "root=default"]}]': finished 2024-03-28T18:02:33.868 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:02:33 smithi022 ceph-mon[31985]: osdmap e69: 10 total, 9 up, 10 in 2024-03-28T18:02:33.868 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:02:33 smithi022 ceph-mon[31985]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-28T18:02:33.868 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:02:33 smithi022 ceph-mon[31985]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-28T18:02:33.868 INFO:journalctl@ceph.osd.9.smithi022.stdout:Mar 28 18:02:33 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-osd-9[67660]: 2024-03-28T18:02:33.582+0000 7f8dda000700 -1 osd.9 0 waiting for initial osdmap 2024-03-28T18:02:33.869 INFO:journalctl@ceph.osd.9.smithi022.stdout:Mar 28 18:02:33 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-osd-9[67660]: 2024-03-28T18:02:33.633+0000 7f8dcfe00700 -1 osd.9 69 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-28T18:02:33.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:02:33 smithi134 ceph-mon[35037]: pgmap v279: 1 pgs: 1 active+clean; 577 KiB data, 290 MiB used, 804 GiB / 805 GiB avail 2024-03-28T18:02:33.925 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:02:33 smithi134 ceph-mon[35037]: from='osd.9 [v2:172.21.15.22:6826/3762222259,v1:172.21.15.22:6827/3762222259]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi022", "root=default"]}]': finished 2024-03-28T18:02:33.925 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:02:33 smithi134 ceph-mon[35037]: osdmap e69: 10 total, 9 up, 10 in 2024-03-28T18:02:33.925 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:02:33 smithi134 ceph-mon[35037]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-28T18:02:33.925 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:02:33 smithi134 ceph-mon[35037]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-28T18:02:33.933 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:02:33 smithi079 ceph-mon[35312]: pgmap v279: 1 pgs: 1 active+clean; 577 KiB data, 290 MiB used, 804 GiB / 805 GiB avail 2024-03-28T18:02:33.933 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:02:33 smithi079 ceph-mon[35312]: from='osd.9 [v2:172.21.15.22:6826/3762222259,v1:172.21.15.22:6827/3762222259]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi022", "root=default"]}]': finished 2024-03-28T18:02:33.933 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:02:33 smithi079 ceph-mon[35312]: osdmap e69: 10 total, 9 up, 10 in 2024-03-28T18:02:33.933 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:02:33 smithi079 ceph-mon[35312]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-28T18:02:33.933 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:02:33 smithi079 ceph-mon[35312]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-28T18:02:34.711 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:02:34 smithi022 ceph-mon[31985]: purged_snaps scrub starts 2024-03-28T18:02:34.711 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:02:34 smithi022 ceph-mon[31985]: purged_snaps scrub ok 2024-03-28T18:02:34.711 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:02:34 smithi022 ceph-mon[31985]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-28T18:02:34.711 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:02:34 smithi022 ceph-mon[31985]: from='osd.9 [v2:172.21.15.22:6826/3762222259,v1:172.21.15.22:6827/3762222259]' entity='osd.9' 2024-03-28T18:02:34.712 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:02:34 smithi022 ceph-mon[31985]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-28T18:02:34.907 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:02:34 smithi079 ceph-mon[35312]: purged_snaps scrub starts 2024-03-28T18:02:34.907 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:02:34 smithi079 ceph-mon[35312]: purged_snaps scrub ok 2024-03-28T18:02:34.907 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:02:34 smithi079 ceph-mon[35312]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-28T18:02:34.907 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:02:34 smithi079 ceph-mon[35312]: from='osd.9 [v2:172.21.15.22:6826/3762222259,v1:172.21.15.22:6827/3762222259]' entity='osd.9' 2024-03-28T18:02:34.908 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:02:34 smithi079 ceph-mon[35312]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-28T18:02:34.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:02:34 smithi134 ceph-mon[35037]: purged_snaps scrub starts 2024-03-28T18:02:34.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:02:34 smithi134 ceph-mon[35037]: purged_snaps scrub ok 2024-03-28T18:02:34.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:02:34 smithi134 ceph-mon[35037]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-28T18:02:34.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:02:34 smithi134 ceph-mon[35037]: from='osd.9 [v2:172.21.15.22:6826/3762222259,v1:172.21.15.22:6827/3762222259]' entity='osd.9' 2024-03-28T18:02:34.925 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:02:34 smithi134 ceph-mon[35037]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-28T18:02:35.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:02:35 smithi134 ceph-mon[35037]: pgmap v281: 1 pgs: 1 active+clean; 577 KiB data, 290 MiB used, 804 GiB / 805 GiB avail 2024-03-28T18:02:35.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:02:35 smithi134 ceph-mon[35037]: osd.9 [v2:172.21.15.22:6826/3762222259,v1:172.21.15.22:6827/3762222259] boot 2024-03-28T18:02:35.925 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:02:35 smithi134 ceph-mon[35037]: osdmap e70: 10 total, 10 up, 10 in 2024-03-28T18:02:35.925 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:02:35 smithi134 ceph-mon[35037]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-28T18:02:35.933 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:02:35 smithi079 ceph-mon[35312]: pgmap v281: 1 pgs: 1 active+clean; 577 KiB data, 290 MiB used, 804 GiB / 805 GiB avail 2024-03-28T18:02:35.933 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:02:35 smithi079 ceph-mon[35312]: osd.9 [v2:172.21.15.22:6826/3762222259,v1:172.21.15.22:6827/3762222259] boot 2024-03-28T18:02:35.933 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:02:35 smithi079 ceph-mon[35312]: osdmap e70: 10 total, 10 up, 10 in 2024-03-28T18:02:35.933 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:02:35 smithi079 ceph-mon[35312]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-28T18:02:36.031 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:02:35 smithi022 ceph-mon[31985]: pgmap v281: 1 pgs: 1 active+clean; 577 KiB data, 290 MiB used, 804 GiB / 805 GiB avail 2024-03-28T18:02:36.031 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:02:35 smithi022 ceph-mon[31985]: osd.9 [v2:172.21.15.22:6826/3762222259,v1:172.21.15.22:6827/3762222259] boot 2024-03-28T18:02:36.032 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:02:35 smithi022 ceph-mon[31985]: osdmap e70: 10 total, 10 up, 10 in 2024-03-28T18:02:36.032 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:02:35 smithi022 ceph-mon[31985]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-28T18:02:36.106 INFO:teuthology.orchestra.run.smithi079.stderr:--> Zapping: /dev/vg_nvme/lv_1 2024-03-28T18:02:36.106 INFO:teuthology.orchestra.run.smithi079.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2024-03-28T18:02:36.106 INFO:teuthology.orchestra.run.smithi079.stderr: stderr: 10+0 records in 2024-03-28T18:02:36.106 INFO:teuthology.orchestra.run.smithi079.stderr:10+0 records out 2024-03-28T18:02:36.106 INFO:teuthology.orchestra.run.smithi079.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0376515 s, 278 MB/s 2024-03-28T18:02:36.106 INFO:teuthology.orchestra.run.smithi079.stderr:--> Zapping successful for: 2024-03-28T18:02:36.806 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:02:36 smithi079 ceph-mon[35312]: osdmap e71: 10 total, 10 up, 10 in 2024-03-28T18:02:36.884 DEBUG:teuthology.orchestra.run.smithi079:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fd220748-ed2b-11ee-95d0-87774f69a715 -- ceph orch daemon add osd smithi079:vg_nvme/lv_1 2024-03-28T18:02:36.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:02:36 smithi134 ceph-mon[35037]: osdmap e71: 10 total, 10 up, 10 in 2024-03-28T18:02:37.033 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:02:36 smithi022 ceph-mon[31985]: osdmap e71: 10 total, 10 up, 10 in 2024-03-28T18:02:37.326 INFO:teuthology.orchestra.run.smithi079.stderr:Inferring config /var/lib/ceph/fd220748-ed2b-11ee-95d0-87774f69a715/mon.b/config 2024-03-28T18:02:37.783 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:02:37 smithi022 ceph-mon[31985]: pgmap v284: 1 pgs: 1 active+clean; 577 KiB data, 325 MiB used, 894 GiB / 894 GiB avail 2024-03-28T18:02:37.783 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:02:37 smithi022 ceph-mon[31985]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T18:02:37.783 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:02:37 smithi022 ceph-mon[31985]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T18:02:37.783 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:02:37 smithi022 ceph-mon[31985]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-28T18:02:37.783 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:02:37 smithi022 ceph-mon[31985]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-28T18:02:37.783 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:02:37 smithi022 ceph-mon[31985]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-03-28T18:02:37.783 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:02:37 smithi022 ceph-mon[31985]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-03-28T18:02:37.783 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:02:37 smithi022 ceph-mon[31985]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T18:02:37.783 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:02:37 smithi022 ceph-mon[31985]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T18:02:37.783 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:02:37 smithi022 ceph-mon[31985]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T18:02:37.783 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:02:37 smithi022 ceph-mon[31985]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T18:02:37.792 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:02:37 smithi079 ceph-mon[35312]: pgmap v284: 1 pgs: 1 active+clean; 577 KiB data, 325 MiB used, 894 GiB / 894 GiB avail 2024-03-28T18:02:37.792 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:02:37 smithi079 ceph-mon[35312]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T18:02:37.792 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:02:37 smithi079 ceph-mon[35312]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T18:02:37.792 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:02:37 smithi079 ceph-mon[35312]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-28T18:02:37.792 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:02:37 smithi079 ceph-mon[35312]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-28T18:02:37.792 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:02:37 smithi079 ceph-mon[35312]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-03-28T18:02:37.793 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:02:37 smithi079 ceph-mon[35312]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-03-28T18:02:37.793 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:02:37 smithi079 ceph-mon[35312]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T18:02:37.793 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:02:37 smithi079 ceph-mon[35312]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T18:02:37.793 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:02:37 smithi079 ceph-mon[35312]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T18:02:37.793 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:02:37 smithi079 ceph-mon[35312]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T18:02:37.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:02:37 smithi134 ceph-mon[35037]: pgmap v284: 1 pgs: 1 active+clean; 577 KiB data, 325 MiB used, 894 GiB / 894 GiB avail 2024-03-28T18:02:37.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:02:37 smithi134 ceph-mon[35037]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T18:02:37.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:02:37 smithi134 ceph-mon[35037]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T18:02:37.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:02:37 smithi134 ceph-mon[35037]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-28T18:02:37.925 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:02:37 smithi134 ceph-mon[35037]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-28T18:02:37.925 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:02:37 smithi134 ceph-mon[35037]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-03-28T18:02:37.925 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:02:37 smithi134 ceph-mon[35037]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-03-28T18:02:37.925 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:02:37 smithi134 ceph-mon[35037]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T18:02:37.925 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:02:37 smithi134 ceph-mon[35037]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T18:02:37.925 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:02:37 smithi134 ceph-mon[35037]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T18:02:37.925 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:02:37 smithi134 ceph-mon[35037]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T18:02:38.683 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:02:38 smithi079 ceph-mon[35312]: Detected new or changed devices on smithi022 2024-03-28T18:02:38.683 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:02:38 smithi079 ceph-mon[35312]: Adjusting osd_memory_target on smithi022 to 4273M 2024-03-28T18:02:38.683 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:02:38 smithi079 ceph-mon[35312]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T18:02:38.683 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:02:38 smithi079 ceph-mon[35312]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T18:02:38.683 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:02:38 smithi079 ceph-mon[35312]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T18:02:38.683 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:02:38 smithi079 ceph-mon[35312]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T18:02:38.925 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:02:38 smithi134 ceph-mon[35037]: Detected new or changed devices on smithi022 2024-03-28T18:02:38.925 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:02:38 smithi134 ceph-mon[35037]: Adjusting osd_memory_target on smithi022 to 4273M 2024-03-28T18:02:38.925 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:02:38 smithi134 ceph-mon[35037]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T18:02:38.925 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:02:38 smithi134 ceph-mon[35037]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T18:02:38.925 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:02:38 smithi134 ceph-mon[35037]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T18:02:38.925 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:02:38 smithi134 ceph-mon[35037]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T18:02:39.032 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:02:38 smithi022 ceph-mon[31985]: Detected new or changed devices on smithi022 2024-03-28T18:02:39.032 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:02:38 smithi022 ceph-mon[31985]: Adjusting osd_memory_target on smithi022 to 4273M 2024-03-28T18:02:39.032 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:02:38 smithi022 ceph-mon[31985]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T18:02:39.032 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:02:38 smithi022 ceph-mon[31985]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T18:02:39.032 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:02:38 smithi022 ceph-mon[31985]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T18:02:39.032 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:02:38 smithi022 ceph-mon[31985]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T18:02:39.574 INFO:teuthology.orchestra.run.smithi079.stderr:2024-03-28T18:02:39.574+0000 7f0f42600700 1 -- 172.21.15.79:0/1619447300 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f0f3c100a40 msgr2=0x7f0f3c102e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:02:39.574 INFO:teuthology.orchestra.run.smithi079.stderr:2024-03-28T18:02:39.574+0000 7f0f42600700 1 --2- 172.21.15.79:0/1619447300 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f0f3c100a40 0x7f0f3c102e70 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f0f2c009990 tx=0x7f0f2c009ca0 comp rx=0 tx=0).stop 2024-03-28T18:02:39.574 INFO:teuthology.orchestra.run.smithi079.stderr:2024-03-28T18:02:39.574+0000 7f0f42600700 1 -- 172.21.15.79:0/1619447300 shutdown_connections 2024-03-28T18:02:39.575 INFO:teuthology.orchestra.run.smithi079.stderr:2024-03-28T18:02:39.574+0000 7f0f42600700 1 --2- 172.21.15.79:0/1619447300 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0f3c105d80 0x7f0f3c106200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:02:39.575 INFO:teuthology.orchestra.run.smithi079.stderr:2024-03-28T18:02:39.574+0000 7f0f42600700 1 --2- 172.21.15.79:0/1619447300 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f0f3c100a40 0x7f0f3c102e70 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:02:39.575 INFO:teuthology.orchestra.run.smithi079.stderr:2024-03-28T18:02:39.574+0000 7f0f42600700 1 --2- 172.21.15.79:0/1619447300 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0f3c1033b0 0x7f0f3c105840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:02:39.575 INFO:teuthology.orchestra.run.smithi079.stderr:2024-03-28T18:02:39.574+0000 7f0f42600700 1 -- 172.21.15.79:0/1619447300 >> 172.21.15.79:0/1619447300 conn(0x7f0f3c0fc660 msgr2=0x7f0f3c0feaa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:02:39.575 INFO:teuthology.orchestra.run.smithi079.stderr:2024-03-28T18:02:39.574+0000 7f0f42600700 1 -- 172.21.15.79:0/1619447300 shutdown_connections 2024-03-28T18:02:39.575 INFO:teuthology.orchestra.run.smithi079.stderr:2024-03-28T18:02:39.574+0000 7f0f42600700 1 -- 172.21.15.79:0/1619447300 wait complete. 2024-03-28T18:02:39.577 INFO:teuthology.orchestra.run.smithi079.stderr:2024-03-28T18:02:39.575+0000 7f0f42600700 1 Processor -- start 2024-03-28T18:02:39.577 INFO:teuthology.orchestra.run.smithi079.stderr:2024-03-28T18:02:39.575+0000 7f0f42600700 1 -- start start 2024-03-28T18:02:39.577 INFO:teuthology.orchestra.run.smithi079.stderr:2024-03-28T18:02:39.576+0000 7f0f42600700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0f3c100a40 0x7f0f3c19ea10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:02:39.578 INFO:teuthology.orchestra.run.smithi079.stderr:2024-03-28T18:02:39.576+0000 7f0f42600700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0f3c1033b0 0x7f0f3c19ef50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:02:39.578 INFO:teuthology.orchestra.run.smithi079.stderr:2024-03-28T18:02:39.576+0000 7f0f42600700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f0f3c105d80 0x7f0f3c1a5f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:02:39.578 INFO:teuthology.orchestra.run.smithi079.stderr:2024-03-28T18:02:39.576+0000 7f0f42600700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f3c114360 con 0x7f0f3c1033b0 2024-03-28T18:02:39.578 INFO:teuthology.orchestra.run.smithi079.stderr:2024-03-28T18:02:39.576+0000 7f0f42600700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f3c1141e0 con 0x7f0f3c105d80 2024-03-28T18:02:39.578 INFO:teuthology.orchestra.run.smithi079.stderr:2024-03-28T18:02:39.576+0000 7f0f42600700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f3c1144e0 con 0x7f0f3c100a40 2024-03-28T18:02:39.578 INFO:teuthology.orchestra.run.smithi079.stderr:2024-03-28T18:02:39.576+0000 7f0f3be00700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f0f3c105d80 0x7f0f3c1a5f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:02:39.578 INFO:teuthology.orchestra.run.smithi079.stderr:2024-03-28T18:02:39.576+0000 7f0f3be00700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f0f3c105d80 0x7f0f3c1a5f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:40272/0 (socket says 172.21.15.79:40272) 2024-03-28T18:02:39.578 INFO:teuthology.orchestra.run.smithi079.stderr:2024-03-28T18:02:39.576+0000 7f0f3be00700 1 -- 172.21.15.79:0/822842705 learned_addr learned my addr 172.21.15.79:0/822842705 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-03-28T18:02:39.578 INFO:teuthology.orchestra.run.smithi079.stderr:2024-03-28T18:02:39.576+0000 7f0f3b400700 1 --2- 172.21.15.79:0/822842705 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0f3c100a40 0x7f0f3c19ea10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:02:39.579 INFO:teuthology.orchestra.run.smithi079.stderr:2024-03-28T18:02:39.576+0000 7f0f3aa00700 1 --2- 172.21.15.79:0/822842705 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0f3c1033b0 0x7f0f3c19ef50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:02:39.579 INFO:teuthology.orchestra.run.smithi079.stderr:2024-03-28T18:02:39.577+0000 7f0f3be00700 1 -- 172.21.15.79:0/822842705 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0f3c100a40 msgr2=0x7f0f3c19ea10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:02:39.579 INFO:teuthology.orchestra.run.smithi079.stderr:2024-03-28T18:02:39.577+0000 7f0f3be00700 1 --2- 172.21.15.79:0/822842705 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0f3c100a40 0x7f0f3c19ea10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:02:39.579 INFO:teuthology.orchestra.run.smithi079.stderr:2024-03-28T18:02:39.577+0000 7f0f3be00700 1 -- 172.21.15.79:0/822842705 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0f3c1033b0 msgr2=0x7f0f3c19ef50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:02:39.579 INFO:teuthology.orchestra.run.smithi079.stderr:2024-03-28T18:02:39.577+0000 7f0f3be00700 1 --2- 172.21.15.79:0/822842705 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0f3c1033b0 0x7f0f3c19ef50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:02:39.579 INFO:teuthology.orchestra.run.smithi079.stderr:2024-03-28T18:02:39.577+0000 7f0f3be00700 1 -- 172.21.15.79:0/822842705 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0f3c1a6620 con 0x7f0f3c105d80 2024-03-28T18:02:39.579 INFO:teuthology.orchestra.run.smithi079.stderr:2024-03-28T18:02:39.577+0000 7f0f3be00700 1 --2- 172.21.15.79:0/822842705 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f0f3c105d80 0x7f0f3c1a5f80 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f0f34007ee0 tx=0x7f0f3400d3b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:02:39.579 INFO:teuthology.orchestra.run.smithi079.stderr:2024-03-28T18:02:39.577+0000 7f0f33e00700 1 -- 172.21.15.79:0/822842705 <== mon.2 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f34004bb0 con 0x7f0f3c105d80 2024-03-28T18:02:39.579 INFO:teuthology.orchestra.run.smithi079.stderr:2024-03-28T18:02:39.577+0000 7f0f33e00700 1 -- 172.21.15.79:0/822842705 <== mon.2 v2:172.21.15.79:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f0f34004d50 con 0x7f0f3c105d80 2024-03-28T18:02:39.579 INFO:teuthology.orchestra.run.smithi079.stderr:2024-03-28T18:02:39.577+0000 7f0f33e00700 1 -- 172.21.15.79:0/822842705 <== mon.2 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f340059a0 con 0x7f0f3c105d80 2024-03-28T18:02:39.579 INFO:teuthology.orchestra.run.smithi079.stderr:2024-03-28T18:02:39.577+0000 7f0f42600700 1 -- 172.21.15.79:0/822842705 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0f3c1a6910 con 0x7f0f3c105d80 2024-03-28T18:02:39.579 INFO:teuthology.orchestra.run.smithi079.stderr:2024-03-28T18:02:39.578+0000 7f0f42600700 1 -- 172.21.15.79:0/822842705 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0f3c1a6e00 con 0x7f0f3c105d80 2024-03-28T18:02:39.580 INFO:teuthology.orchestra.run.smithi079.stderr:2024-03-28T18:02:39.579+0000 7f0f42600700 1 -- 172.21.15.79:0/822842705 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0f3c068e50 con 0x7f0f3c105d80 2024-03-28T18:02:39.580 INFO:teuthology.orchestra.run.smithi079.stderr:2024-03-28T18:02:39.581+0000 7f0f33e00700 1 -- 172.21.15.79:0/822842705 <== mon.2 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 16) v1 ==== 137290+0+0 (secure 0 0 0) 0x7f0f3401a020 con 0x7f0f3c105d80 2024-03-28T18:02:39.582 INFO:teuthology.orchestra.run.smithi079.stderr:2024-03-28T18:02:39.583+0000 7f0f33e00700 1 --2- 172.21.15.79:0/822842705 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7f0f200a2910 0x7f0f200a4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:02:39.583 INFO:teuthology.orchestra.run.smithi079.stderr:2024-03-28T18:02:39.583+0000 7f0f33e00700 1 -- 172.21.15.79:0/822842705 <== mon.2 v2:172.21.15.79:3300/0 5 ==== osd_map(71..71 src has 1..71) v4 ==== 6261+0+0 (secure 0 0 0) 0x7f0f340cd9c0 con 0x7f0f3c105d80 2024-03-28T18:02:39.583 INFO:teuthology.orchestra.run.smithi079.stderr:2024-03-28T18:02:39.584+0000 7f0f3b400700 1 --2- 172.21.15.79:0/822842705 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7f0f200a2910 0x7f0f200a4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:02:39.584 INFO:teuthology.orchestra.run.smithi079.stderr:2024-03-28T18:02:39.585+0000 7f0f3b400700 1 --2- 172.21.15.79:0/822842705 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7f0f200a2910 0x7f0f200a4dd0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f0f2c00b5c0 tx=0x7f0f2c003960 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:02:39.588 INFO:teuthology.orchestra.run.smithi079.stderr:2024-03-28T18:02:39.589+0000 7f0f33e00700 1 -- 172.21.15.79:0/822842705 <== mon.2 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f0f34010070 con 0x7f0f3c105d80 2024-03-28T18:02:39.822 INFO:teuthology.orchestra.run.smithi079.stderr:2024-03-28T18:02:39.822+0000 7f0f42600700 1 -- 172.21.15.79:0/822842705 --> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi079:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) v1 -- 0x7f0f3c0625d0 con 0x7f0f200a2910 2024-03-28T18:02:40.182 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:02:40 smithi079 ceph-mon[35312]: pgmap v285: 1 pgs: 1 active+clean; 577 KiB data, 325 MiB used, 894 GiB / 894 GiB avail 2024-03-28T18:02:40.183 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:02:40 smithi079 ceph-mon[35312]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-28T18:02:40.183 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:02:40 smithi079 ceph-mon[35312]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-28T18:02:40.183 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:02:40 smithi079 ceph-mon[35312]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T18:02:40.425 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:02:40 smithi134 ceph-mon[35037]: pgmap v285: 1 pgs: 1 active+clean; 577 KiB data, 325 MiB used, 894 GiB / 894 GiB avail 2024-03-28T18:02:40.425 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:02:40 smithi134 ceph-mon[35037]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-28T18:02:40.425 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:02:40 smithi134 ceph-mon[35037]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-28T18:02:40.425 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:02:40 smithi134 ceph-mon[35037]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T18:02:40.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:02:40 smithi022 ceph-mon[31985]: pgmap v285: 1 pgs: 1 active+clean; 577 KiB data, 325 MiB used, 894 GiB / 894 GiB avail 2024-03-28T18:02:40.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:02:40 smithi022 ceph-mon[31985]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-28T18:02:40.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:02:40 smithi022 ceph-mon[31985]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-28T18:02:40.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:02:40 smithi022 ceph-mon[31985]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T18:02:41.174 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:02:41 smithi134 ceph-mon[35037]: from='client.34352 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi079:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-03-28T18:02:41.432 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:02:41 smithi079 ceph-mon[35312]: from='client.34352 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi079:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-03-28T18:02:41.531 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:02:41 smithi022 ceph-mon[31985]: from='client.34352 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi079:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-03-28T18:02:42.406 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:02:42 smithi079 ceph-mon[35312]: pgmap v286: 1 pgs: 1 active+clean; 577 KiB data, 325 MiB used, 894 GiB / 894 GiB avail 2024-03-28T18:02:42.424 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:02:42 smithi134 ceph-mon[35037]: pgmap v286: 1 pgs: 1 active+clean; 577 KiB data, 325 MiB used, 894 GiB / 894 GiB avail 2024-03-28T18:02:42.531 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:02:42 smithi022 ceph-mon[31985]: pgmap v286: 1 pgs: 1 active+clean; 577 KiB data, 325 MiB used, 894 GiB / 894 GiB avail 2024-03-28T18:02:44.425 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:02:44 smithi134 ceph-mon[35037]: pgmap v287: 1 pgs: 1 active+clean; 577 KiB data, 325 MiB used, 894 GiB / 894 GiB avail 2024-03-28T18:02:44.425 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:02:44 smithi134 ceph-mon[35037]: from='client.? 172.21.15.79:0/642854759' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f5198d52-df48-4679-a860-4dcb440b7a9f"}]: dispatch 2024-03-28T18:02:44.425 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:02:44 smithi134 ceph-mon[35037]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f5198d52-df48-4679-a860-4dcb440b7a9f"}]: dispatch 2024-03-28T18:02:44.425 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:02:44 smithi134 ceph-mon[35037]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "f5198d52-df48-4679-a860-4dcb440b7a9f"}]': finished 2024-03-28T18:02:44.425 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:02:44 smithi134 ceph-mon[35037]: osdmap e72: 11 total, 10 up, 11 in 2024-03-28T18:02:44.425 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:02:44 smithi134 ceph-mon[35037]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-03-28T18:02:44.433 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:02:44 smithi079 ceph-mon[35312]: pgmap v287: 1 pgs: 1 active+clean; 577 KiB data, 325 MiB used, 894 GiB / 894 GiB avail 2024-03-28T18:02:44.433 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:02:44 smithi079 ceph-mon[35312]: from='client.? 172.21.15.79:0/642854759' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f5198d52-df48-4679-a860-4dcb440b7a9f"}]: dispatch 2024-03-28T18:02:44.433 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:02:44 smithi079 ceph-mon[35312]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f5198d52-df48-4679-a860-4dcb440b7a9f"}]: dispatch 2024-03-28T18:02:44.433 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:02:44 smithi079 ceph-mon[35312]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "f5198d52-df48-4679-a860-4dcb440b7a9f"}]': finished 2024-03-28T18:02:44.433 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:02:44 smithi079 ceph-mon[35312]: osdmap e72: 11 total, 10 up, 11 in 2024-03-28T18:02:44.433 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:02:44 smithi079 ceph-mon[35312]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-03-28T18:02:44.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:02:44 smithi022 ceph-mon[31985]: pgmap v287: 1 pgs: 1 active+clean; 577 KiB data, 325 MiB used, 894 GiB / 894 GiB avail 2024-03-28T18:02:44.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:02:44 smithi022 ceph-mon[31985]: from='client.? 172.21.15.79:0/642854759' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f5198d52-df48-4679-a860-4dcb440b7a9f"}]: dispatch 2024-03-28T18:02:44.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:02:44 smithi022 ceph-mon[31985]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f5198d52-df48-4679-a860-4dcb440b7a9f"}]: dispatch 2024-03-28T18:02:44.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:02:44 smithi022 ceph-mon[31985]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "f5198d52-df48-4679-a860-4dcb440b7a9f"}]': finished 2024-03-28T18:02:44.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:02:44 smithi022 ceph-mon[31985]: osdmap e72: 11 total, 10 up, 11 in 2024-03-28T18:02:44.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:02:44 smithi022 ceph-mon[31985]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-03-28T18:02:45.424 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:02:45 smithi134 ceph-mon[35037]: from='client.? 172.21.15.79:0/30682015' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-28T18:02:45.432 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:02:45 smithi079 ceph-mon[35312]: from='client.? 172.21.15.79:0/30682015' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-28T18:02:45.531 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:02:45 smithi022 ceph-mon[31985]: from='client.? 172.21.15.79:0/30682015' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-28T18:02:46.424 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:02:46 smithi134 ceph-mon[35037]: pgmap v289: 1 pgs: 1 active+clean; 577 KiB data, 325 MiB used, 894 GiB / 894 GiB avail 2024-03-28T18:02:46.432 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:02:46 smithi079 ceph-mon[35312]: pgmap v289: 1 pgs: 1 active+clean; 577 KiB data, 325 MiB used, 894 GiB / 894 GiB avail 2024-03-28T18:02:46.531 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:02:46 smithi022 ceph-mon[31985]: pgmap v289: 1 pgs: 1 active+clean; 577 KiB data, 325 MiB used, 894 GiB / 894 GiB avail 2024-03-28T18:02:48.424 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:02:48 smithi134 ceph-mon[35037]: pgmap v290: 1 pgs: 1 active+clean; 577 KiB data, 325 MiB used, 894 GiB / 894 GiB avail 2024-03-28T18:02:48.432 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:02:48 smithi079 ceph-mon[35312]: pgmap v290: 1 pgs: 1 active+clean; 577 KiB data, 325 MiB used, 894 GiB / 894 GiB avail 2024-03-28T18:02:48.531 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:02:48 smithi022 ceph-mon[31985]: pgmap v290: 1 pgs: 1 active+clean; 577 KiB data, 325 MiB used, 894 GiB / 894 GiB avail 2024-03-28T18:02:50.395 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:02:50 smithi079 ceph-mon[35312]: pgmap v291: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 894 GiB / 894 GiB avail 2024-03-28T18:02:50.424 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:02:50 smithi134 ceph-mon[35037]: pgmap v291: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 894 GiB / 894 GiB avail 2024-03-28T18:02:50.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:02:50 smithi022 ceph-mon[31985]: pgmap v291: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 894 GiB / 894 GiB avail 2024-03-28T18:02:52.424 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:02:52 smithi134 ceph-mon[35037]: pgmap v292: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 894 GiB / 894 GiB avail 2024-03-28T18:02:52.432 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:02:52 smithi079 ceph-mon[35312]: pgmap v292: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 894 GiB / 894 GiB avail 2024-03-28T18:02:52.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:02:52 smithi022 ceph-mon[31985]: pgmap v292: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 894 GiB / 894 GiB avail 2024-03-28T18:02:54.282 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:02:54 smithi022 ceph-mon[31985]: pgmap v293: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 894 GiB / 894 GiB avail 2024-03-28T18:02:54.282 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:02:54 smithi022 ceph-mon[31985]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-03-28T18:02:54.283 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:02:54 smithi022 ceph-mon[31985]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T18:02:54.283 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:02:54 smithi022 ceph-mon[31985]: Deploying daemon osd.10 on smithi079 2024-03-28T18:02:54.424 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:02:54 smithi134 ceph-mon[35037]: pgmap v293: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 894 GiB / 894 GiB avail 2024-03-28T18:02:54.425 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:02:54 smithi134 ceph-mon[35037]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-03-28T18:02:54.425 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:02:54 smithi134 ceph-mon[35037]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T18:02:54.425 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:02:54 smithi134 ceph-mon[35037]: Deploying daemon osd.10 on smithi079 2024-03-28T18:02:54.432 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:02:54 smithi079 ceph-mon[35312]: pgmap v293: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 894 GiB / 894 GiB avail 2024-03-28T18:02:54.432 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:02:54 smithi079 ceph-mon[35312]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-03-28T18:02:54.432 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:02:54 smithi079 ceph-mon[35312]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T18:02:54.433 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:02:54 smithi079 ceph-mon[35312]: Deploying daemon osd.10 on smithi079 2024-03-28T18:02:56.333 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:02:56 smithi079 ceph-mon[35312]: pgmap v294: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 894 GiB / 894 GiB avail 2024-03-28T18:02:56.424 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:02:56 smithi134 ceph-mon[35037]: pgmap v294: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 894 GiB / 894 GiB avail 2024-03-28T18:02:56.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:02:56 smithi022 ceph-mon[31985]: pgmap v294: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 894 GiB / 894 GiB avail 2024-03-28T18:02:58.424 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:02:58 smithi134 ceph-mon[35037]: pgmap v295: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 894 GiB / 894 GiB avail 2024-03-28T18:02:58.433 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:02:58 smithi079 ceph-mon[35312]: pgmap v295: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 894 GiB / 894 GiB avail 2024-03-28T18:02:58.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:02:58 smithi022 ceph-mon[31985]: pgmap v295: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 894 GiB / 894 GiB avail 2024-03-28T18:03:00.424 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:03:00 smithi134 ceph-mon[35037]: pgmap v296: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 894 GiB / 894 GiB avail 2024-03-28T18:03:00.432 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:03:00 smithi079 ceph-mon[35312]: pgmap v296: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 894 GiB / 894 GiB avail 2024-03-28T18:03:00.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:03:00 smithi022 ceph-mon[31985]: pgmap v296: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 894 GiB / 894 GiB avail 2024-03-28T18:03:02.424 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:03:02 smithi134 ceph-mon[35037]: pgmap v297: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 894 GiB / 894 GiB avail 2024-03-28T18:03:02.424 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:03:02 smithi134 ceph-mon[35037]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T18:03:02.424 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:03:02 smithi134 ceph-mon[35037]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T18:03:02.425 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:03:02 smithi134 ceph-mon[35037]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T18:03:02.432 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:03:02 smithi079 ceph-mon[35312]: pgmap v297: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 894 GiB / 894 GiB avail 2024-03-28T18:03:02.432 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:03:02 smithi079 ceph-mon[35312]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T18:03:02.433 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:03:02 smithi079 ceph-mon[35312]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T18:03:02.433 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:03:02 smithi079 ceph-mon[35312]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T18:03:02.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:03:02 smithi022 ceph-mon[31985]: pgmap v297: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 894 GiB / 894 GiB avail 2024-03-28T18:03:02.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:03:02 smithi022 ceph-mon[31985]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T18:03:02.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:03:02 smithi022 ceph-mon[31985]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T18:03:02.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:03:02 smithi022 ceph-mon[31985]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T18:03:04.424 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:03:04 smithi134 ceph-mon[35037]: pgmap v298: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 894 GiB / 894 GiB avail 2024-03-28T18:03:04.432 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:03:04 smithi079 ceph-mon[35312]: pgmap v298: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 894 GiB / 894 GiB avail 2024-03-28T18:03:04.531 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:03:04 smithi022 ceph-mon[31985]: pgmap v298: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 894 GiB / 894 GiB avail 2024-03-28T18:03:06.640 INFO:teuthology.orchestra.run.smithi079.stdout:Created osd(s) 10 on host 'smithi079' 2024-03-28T18:03:06.640 INFO:teuthology.orchestra.run.smithi079.stderr:2024-03-28T18:03:06.638+0000 7f0f33e00700 1 -- 172.21.15.79:0/822842705 <== mgr.14152 v2:172.21.15.22:6800/4201048392 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+38 (secure 0 0 0) 0x7f0f3c0625d0 con 0x7f0f200a2910 2024-03-28T18:03:06.643 INFO:teuthology.orchestra.run.smithi079.stderr:2024-03-28T18:03:06.642+0000 7f0f42600700 1 -- 172.21.15.79:0/822842705 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7f0f200a2910 msgr2=0x7f0f200a4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:03:06.643 INFO:teuthology.orchestra.run.smithi079.stderr:2024-03-28T18:03:06.642+0000 7f0f42600700 1 --2- 172.21.15.79:0/822842705 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7f0f200a2910 0x7f0f200a4dd0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f0f2c00b5c0 tx=0x7f0f2c003960 comp rx=0 tx=0).stop 2024-03-28T18:03:06.643 INFO:teuthology.orchestra.run.smithi079.stderr:2024-03-28T18:03:06.642+0000 7f0f42600700 1 -- 172.21.15.79:0/822842705 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f0f3c105d80 msgr2=0x7f0f3c1a5f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:03:06.643 INFO:teuthology.orchestra.run.smithi079.stderr:2024-03-28T18:03:06.642+0000 7f0f42600700 1 --2- 172.21.15.79:0/822842705 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f0f3c105d80 0x7f0f3c1a5f80 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f0f34007ee0 tx=0x7f0f3400d3b0 comp rx=0 tx=0).stop 2024-03-28T18:03:06.643 INFO:teuthology.orchestra.run.smithi079.stderr:2024-03-28T18:03:06.642+0000 7f0f42600700 1 -- 172.21.15.79:0/822842705 shutdown_connections 2024-03-28T18:03:06.643 INFO:teuthology.orchestra.run.smithi079.stderr:2024-03-28T18:03:06.642+0000 7f0f42600700 1 --2- 172.21.15.79:0/822842705 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f0f3c105d80 0x7f0f3c1a5f80 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:03:06.643 INFO:teuthology.orchestra.run.smithi079.stderr:2024-03-28T18:03:06.642+0000 7f0f42600700 1 --2- 172.21.15.79:0/822842705 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7f0f200a2910 0x7f0f200a4dd0 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:03:06.644 INFO:teuthology.orchestra.run.smithi079.stderr:2024-03-28T18:03:06.642+0000 7f0f42600700 1 --2- 172.21.15.79:0/822842705 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0f3c100a40 0x7f0f3c19ea10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:03:06.644 INFO:teuthology.orchestra.run.smithi079.stderr:2024-03-28T18:03:06.642+0000 7f0f42600700 1 --2- 172.21.15.79:0/822842705 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0f3c1033b0 0x7f0f3c19ef50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:03:06.644 INFO:teuthology.orchestra.run.smithi079.stderr:2024-03-28T18:03:06.642+0000 7f0f42600700 1 -- 172.21.15.79:0/822842705 >> 172.21.15.79:0/822842705 conn(0x7f0f3c0fc660 msgr2=0x7f0f3c104180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:03:06.644 INFO:teuthology.orchestra.run.smithi079.stderr:2024-03-28T18:03:06.643+0000 7f0f42600700 1 -- 172.21.15.79:0/822842705 shutdown_connections 2024-03-28T18:03:06.644 INFO:teuthology.orchestra.run.smithi079.stderr:2024-03-28T18:03:06.643+0000 7f0f42600700 1 -- 172.21.15.79:0/822842705 wait complete. 2024-03-28T18:03:06.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:03:06 smithi079 ceph-mon[35312]: pgmap v299: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 894 GiB / 894 GiB avail 2024-03-28T18:03:06.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:03:06 smithi079 ceph-mon[35312]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T18:03:06.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:03:06 smithi079 ceph-mon[35312]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T18:03:06.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:03:06 smithi079 ceph-mon[35312]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T18:03:06.683 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:03:06 smithi079 ceph-mon[35312]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T18:03:06.683 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:03:06 smithi079 ceph-mon[35312]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T18:03:06.683 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:03:06 smithi079 ceph-mon[35312]: from='osd.10 [v2:172.21.15.79:6824/1746167904,v1:172.21.15.79:6825/1746167904]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-03-28T18:03:06.683 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:03:06 smithi079 ceph-mon[35312]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-03-28T18:03:06.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:03:06 smithi022 ceph-mon[31985]: pgmap v299: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 894 GiB / 894 GiB avail 2024-03-28T18:03:06.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:03:06 smithi022 ceph-mon[31985]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T18:03:06.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:03:06 smithi022 ceph-mon[31985]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T18:03:06.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:03:06 smithi022 ceph-mon[31985]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T18:03:06.783 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:03:06 smithi022 ceph-mon[31985]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T18:03:06.783 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:03:06 smithi022 ceph-mon[31985]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T18:03:06.783 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:03:06 smithi022 ceph-mon[31985]: from='osd.10 [v2:172.21.15.79:6824/1746167904,v1:172.21.15.79:6825/1746167904]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-03-28T18:03:06.783 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:03:06 smithi022 ceph-mon[31985]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-03-28T18:03:06.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:03:06 smithi134 ceph-mon[35037]: pgmap v299: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 894 GiB / 894 GiB avail 2024-03-28T18:03:06.925 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:03:06 smithi134 ceph-mon[35037]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T18:03:06.925 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:03:06 smithi134 ceph-mon[35037]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T18:03:06.925 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:03:06 smithi134 ceph-mon[35037]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T18:03:06.925 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:03:06 smithi134 ceph-mon[35037]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T18:03:06.925 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:03:06 smithi134 ceph-mon[35037]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T18:03:06.925 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:03:06 smithi134 ceph-mon[35037]: from='osd.10 [v2:172.21.15.79:6824/1746167904,v1:172.21.15.79:6825/1746167904]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-03-28T18:03:06.925 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:03:06 smithi134 ceph-mon[35037]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-03-28T18:03:07.423 DEBUG:teuthology.orchestra.run.smithi079:osd.10> sudo journalctl -f -n 0 -u ceph-fd220748-ed2b-11ee-95d0-87774f69a715@osd.10.service 2024-03-28T18:03:07.428 INFO:tasks.cephadm:Deploying osd.11 on smithi134 with /dev/vg_nvme/lv_1... 2024-03-28T18:03:07.428 DEBUG:teuthology.orchestra.run.smithi134:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fd220748-ed2b-11ee-95d0-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2024-03-28T18:03:07.683 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:03:07 smithi079 ceph-mon[35312]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-03-28T18:03:07.683 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:03:07 smithi079 ceph-mon[35312]: osdmap e73: 11 total, 10 up, 11 in 2024-03-28T18:03:07.683 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:03:07 smithi079 ceph-mon[35312]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-03-28T18:03:07.683 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:03:07 smithi079 ceph-mon[35312]: from='osd.10 [v2:172.21.15.79:6824/1746167904,v1:172.21.15.79:6825/1746167904]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi079", "root=default"]}]: dispatch 2024-03-28T18:03:07.683 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:03:07 smithi079 ceph-mon[35312]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi079", "root=default"]}]: dispatch 2024-03-28T18:03:07.684 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:03:07 smithi079 ceph-mon[35312]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T18:03:07.684 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:03:07 smithi079 ceph-mon[35312]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T18:03:07.684 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:03:07 smithi079 ceph-mon[35312]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T18:03:07.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:03:07 smithi022 ceph-mon[31985]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-03-28T18:03:07.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:03:07 smithi022 ceph-mon[31985]: osdmap e73: 11 total, 10 up, 11 in 2024-03-28T18:03:07.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:03:07 smithi022 ceph-mon[31985]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-03-28T18:03:07.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:03:07 smithi022 ceph-mon[31985]: from='osd.10 [v2:172.21.15.79:6824/1746167904,v1:172.21.15.79:6825/1746167904]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi079", "root=default"]}]: dispatch 2024-03-28T18:03:07.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:03:07 smithi022 ceph-mon[31985]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi079", "root=default"]}]: dispatch 2024-03-28T18:03:07.783 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:03:07 smithi022 ceph-mon[31985]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T18:03:07.783 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:03:07 smithi022 ceph-mon[31985]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T18:03:07.783 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:03:07 smithi022 ceph-mon[31985]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T18:03:07.866 INFO:teuthology.orchestra.run.smithi134.stderr:Inferring config /var/lib/ceph/fd220748-ed2b-11ee-95d0-87774f69a715/mon.c/config 2024-03-28T18:03:07.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:03:07 smithi134 ceph-mon[35037]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-03-28T18:03:07.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:03:07 smithi134 ceph-mon[35037]: osdmap e73: 11 total, 10 up, 11 in 2024-03-28T18:03:07.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:03:07 smithi134 ceph-mon[35037]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-03-28T18:03:07.925 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:03:07 smithi134 ceph-mon[35037]: from='osd.10 [v2:172.21.15.79:6824/1746167904,v1:172.21.15.79:6825/1746167904]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi079", "root=default"]}]: dispatch 2024-03-28T18:03:07.925 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:03:07 smithi134 ceph-mon[35037]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi079", "root=default"]}]: dispatch 2024-03-28T18:03:07.925 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:03:07 smithi134 ceph-mon[35037]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T18:03:07.925 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:03:07 smithi134 ceph-mon[35037]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T18:03:07.925 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:03:07 smithi134 ceph-mon[35037]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T18:03:08.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:03:08 smithi134 ceph-mon[35037]: pgmap v301: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 894 GiB / 894 GiB avail 2024-03-28T18:03:08.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:03:08 smithi134 ceph-mon[35037]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi079", "root=default"]}]': finished 2024-03-28T18:03:08.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:03:08 smithi134 ceph-mon[35037]: osdmap e74: 11 total, 10 up, 11 in 2024-03-28T18:03:08.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:03:08 smithi134 ceph-mon[35037]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-03-28T18:03:08.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:03:08 smithi134 ceph-mon[35037]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-03-28T18:03:08.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:03:08 smithi134 ceph-mon[35037]: osdmap e75: 11 total, 10 up, 11 in 2024-03-28T18:03:08.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:03:08 smithi134 ceph-mon[35037]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-03-28T18:03:08.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:03:08 smithi134 ceph-mon[35037]: from='osd.10 ' entity='osd.10' 2024-03-28T18:03:08.717 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:03:08 smithi079 ceph-mon[35312]: pgmap v301: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 894 GiB / 894 GiB avail 2024-03-28T18:03:08.717 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:03:08 smithi079 ceph-mon[35312]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi079", "root=default"]}]': finished 2024-03-28T18:03:08.718 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:03:08 smithi079 ceph-mon[35312]: osdmap e74: 11 total, 10 up, 11 in 2024-03-28T18:03:08.718 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:03:08 smithi079 ceph-mon[35312]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-03-28T18:03:08.718 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:03:08 smithi079 ceph-mon[35312]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-03-28T18:03:08.718 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:03:08 smithi079 ceph-mon[35312]: osdmap e75: 11 total, 10 up, 11 in 2024-03-28T18:03:08.718 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:03:08 smithi079 ceph-mon[35312]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-03-28T18:03:08.718 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:03:08 smithi079 ceph-mon[35312]: from='osd.10 ' entity='osd.10' 2024-03-28T18:03:08.718 INFO:journalctl@ceph.osd.10.smithi079.stdout:Mar 28 18:03:08 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-osd-10[60034]: 2024-03-28T18:03:08.457+0000 7f743b000700 -1 osd.10 0 waiting for initial osdmap 2024-03-28T18:03:08.719 INFO:journalctl@ceph.osd.10.smithi079.stdout:Mar 28 18:03:08 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-osd-10[60034]: 2024-03-28T18:03:08.522+0000 7f7430400700 -1 osd.10 75 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-28T18:03:08.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:03:08 smithi022 ceph-mon[31985]: pgmap v301: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 894 GiB / 894 GiB avail 2024-03-28T18:03:08.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:03:08 smithi022 ceph-mon[31985]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi079", "root=default"]}]': finished 2024-03-28T18:03:08.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:03:08 smithi022 ceph-mon[31985]: osdmap e74: 11 total, 10 up, 11 in 2024-03-28T18:03:08.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:03:08 smithi022 ceph-mon[31985]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-03-28T18:03:08.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:03:08 smithi022 ceph-mon[31985]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-03-28T18:03:08.783 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:03:08 smithi022 ceph-mon[31985]: osdmap e75: 11 total, 10 up, 11 in 2024-03-28T18:03:08.783 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:03:08 smithi022 ceph-mon[31985]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-03-28T18:03:08.783 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:03:08 smithi022 ceph-mon[31985]: from='osd.10 ' entity='osd.10' 2024-03-28T18:03:09.683 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:03:09 smithi079 ceph-mon[35312]: purged_snaps scrub starts 2024-03-28T18:03:09.683 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:03:09 smithi079 ceph-mon[35312]: purged_snaps scrub ok 2024-03-28T18:03:09.683 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:03:09 smithi079 ceph-mon[35312]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-03-28T18:03:09.683 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:03:09 smithi079 ceph-mon[35312]: osdmap e76: 11 total, 10 up, 11 in 2024-03-28T18:03:09.683 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:03:09 smithi079 ceph-mon[35312]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-03-28T18:03:09.683 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:03:09 smithi079 ceph-mon[35312]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-03-28T18:03:09.752 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:03:09 smithi134 ceph-mon[35037]: purged_snaps scrub starts 2024-03-28T18:03:09.752 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:03:09 smithi134 ceph-mon[35037]: purged_snaps scrub ok 2024-03-28T18:03:09.752 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:03:09 smithi134 ceph-mon[35037]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-03-28T18:03:09.752 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:03:09 smithi134 ceph-mon[35037]: osdmap e76: 11 total, 10 up, 11 in 2024-03-28T18:03:09.752 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:03:09 smithi134 ceph-mon[35037]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-03-28T18:03:09.752 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:03:09 smithi134 ceph-mon[35037]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-03-28T18:03:09.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:03:09 smithi022 ceph-mon[31985]: purged_snaps scrub starts 2024-03-28T18:03:09.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:03:09 smithi022 ceph-mon[31985]: purged_snaps scrub ok 2024-03-28T18:03:09.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:03:09 smithi022 ceph-mon[31985]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-03-28T18:03:09.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:03:09 smithi022 ceph-mon[31985]: osdmap e76: 11 total, 10 up, 11 in 2024-03-28T18:03:09.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:03:09 smithi022 ceph-mon[31985]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-03-28T18:03:09.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:03:09 smithi022 ceph-mon[31985]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-03-28T18:03:10.774 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:03:10 smithi134 ceph-mon[35037]: pgmap v305: 1 pgs: 1 remapped+peering; 577 KiB data, 326 MiB used, 894 GiB / 894 GiB avail 2024-03-28T18:03:10.774 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:03:10 smithi134 ceph-mon[35037]: osd.10 [v2:172.21.15.79:6824/1746167904,v1:172.21.15.79:6825/1746167904] boot 2024-03-28T18:03:10.774 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:03:10 smithi134 ceph-mon[35037]: osdmap e77: 11 total, 11 up, 11 in 2024-03-28T18:03:10.774 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:03:10 smithi134 ceph-mon[35037]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-03-28T18:03:10.774 INFO:teuthology.orchestra.run.smithi134.stderr:--> Zapping: /dev/vg_nvme/lv_1 2024-03-28T18:03:10.774 INFO:teuthology.orchestra.run.smithi134.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2024-03-28T18:03:10.774 INFO:teuthology.orchestra.run.smithi134.stderr: stderr: 10+0 records in 2024-03-28T18:03:10.774 INFO:teuthology.orchestra.run.smithi134.stderr:10+0 records out 2024-03-28T18:03:10.774 INFO:teuthology.orchestra.run.smithi134.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0386478 s, 271 MB/s 2024-03-28T18:03:10.775 INFO:teuthology.orchestra.run.smithi134.stderr:--> Zapping successful for: 2024-03-28T18:03:10.781 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:03:10 smithi022 ceph-mon[31985]: pgmap v305: 1 pgs: 1 remapped+peering; 577 KiB data, 326 MiB used, 894 GiB / 894 GiB avail 2024-03-28T18:03:10.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:03:10 smithi022 ceph-mon[31985]: osd.10 [v2:172.21.15.79:6824/1746167904,v1:172.21.15.79:6825/1746167904] boot 2024-03-28T18:03:10.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:03:10 smithi022 ceph-mon[31985]: osdmap e77: 11 total, 11 up, 11 in 2024-03-28T18:03:10.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:03:10 smithi022 ceph-mon[31985]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-03-28T18:03:10.933 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:03:10 smithi079 ceph-mon[35312]: pgmap v305: 1 pgs: 1 remapped+peering; 577 KiB data, 326 MiB used, 894 GiB / 894 GiB avail 2024-03-28T18:03:10.933 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:03:10 smithi079 ceph-mon[35312]: osd.10 [v2:172.21.15.79:6824/1746167904,v1:172.21.15.79:6825/1746167904] boot 2024-03-28T18:03:10.933 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:03:10 smithi079 ceph-mon[35312]: osdmap e77: 11 total, 11 up, 11 in 2024-03-28T18:03:10.933 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:03:10 smithi079 ceph-mon[35312]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-03-28T18:03:11.674 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:03:11 smithi134 ceph-mon[35037]: osdmap e78: 11 total, 11 up, 11 in 2024-03-28T18:03:11.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:03:11 smithi134 ceph-mon[35037]: pgmap v308: 1 pgs: 1 remapped+peering; 577 KiB data, 357 MiB used, 983 GiB / 983 GiB avail 2024-03-28T18:03:11.740 DEBUG:teuthology.orchestra.run.smithi134:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fd220748-ed2b-11ee-95d0-87774f69a715 -- ceph orch daemon add osd smithi134:vg_nvme/lv_1 2024-03-28T18:03:11.880 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:03:11 smithi079 ceph-mon[35312]: osdmap e78: 11 total, 11 up, 11 in 2024-03-28T18:03:11.880 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:03:11 smithi079 ceph-mon[35312]: pgmap v308: 1 pgs: 1 remapped+peering; 577 KiB data, 357 MiB used, 983 GiB / 983 GiB avail 2024-03-28T18:03:12.032 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:03:11 smithi022 ceph-mon[31985]: osdmap e78: 11 total, 11 up, 11 in 2024-03-28T18:03:12.032 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:03:11 smithi022 ceph-mon[31985]: pgmap v308: 1 pgs: 1 remapped+peering; 577 KiB data, 357 MiB used, 983 GiB / 983 GiB avail 2024-03-28T18:03:12.142 INFO:teuthology.orchestra.run.smithi134.stderr:Inferring config /var/lib/ceph/fd220748-ed2b-11ee-95d0-87774f69a715/mon.c/config 2024-03-28T18:03:12.543 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:03:12 smithi134 ceph-mon[35037]: osdmap e79: 11 total, 11 up, 11 in 2024-03-28T18:03:12.543 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:03:12 smithi134 ceph-mon[35037]: Detected new or changed devices on smithi079 2024-03-28T18:03:12.543 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:03:12 smithi134 ceph-mon[35037]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T18:03:12.543 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:03:12 smithi134 ceph-mon[35037]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T18:03:12.543 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:03:12 smithi134 ceph-mon[35037]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-28T18:03:12.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:03:12 smithi134 ceph-mon[35037]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-28T18:03:12.925 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:03:12 smithi134 ceph-mon[35037]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-03-28T18:03:12.925 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:03:12 smithi134 ceph-mon[35037]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-03-28T18:03:12.925 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:03:12 smithi134 ceph-mon[35037]: Adjusting osd_memory_target on smithi079 to 4272M 2024-03-28T18:03:12.925 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:03:12 smithi134 ceph-mon[35037]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T18:03:12.925 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:03:12 smithi134 ceph-mon[35037]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T18:03:12.925 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:03:12 smithi134 ceph-mon[35037]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T18:03:12.925 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:03:12 smithi134 ceph-mon[35037]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T18:03:12.925 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:03:12 smithi134 ceph-mon[35037]: osdmap e80: 11 total, 11 up, 11 in 2024-03-28T18:03:12.933 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:03:12 smithi079 ceph-mon[35312]: osdmap e79: 11 total, 11 up, 11 in 2024-03-28T18:03:12.933 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:03:12 smithi079 ceph-mon[35312]: Detected new or changed devices on smithi079 2024-03-28T18:03:12.933 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:03:12 smithi079 ceph-mon[35312]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T18:03:12.934 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:03:12 smithi079 ceph-mon[35312]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T18:03:12.934 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:03:12 smithi079 ceph-mon[35312]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-28T18:03:12.934 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:03:12 smithi079 ceph-mon[35312]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-28T18:03:12.934 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:03:12 smithi079 ceph-mon[35312]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-03-28T18:03:12.934 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:03:12 smithi079 ceph-mon[35312]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-03-28T18:03:12.934 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:03:12 smithi079 ceph-mon[35312]: Adjusting osd_memory_target on smithi079 to 4272M 2024-03-28T18:03:12.934 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:03:12 smithi079 ceph-mon[35312]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T18:03:12.934 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:03:12 smithi079 ceph-mon[35312]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T18:03:12.934 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:03:12 smithi079 ceph-mon[35312]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T18:03:12.934 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:03:12 smithi079 ceph-mon[35312]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T18:03:12.934 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:03:12 smithi079 ceph-mon[35312]: osdmap e80: 11 total, 11 up, 11 in 2024-03-28T18:03:13.032 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:03:12 smithi022 ceph-mon[31985]: osdmap e79: 11 total, 11 up, 11 in 2024-03-28T18:03:13.032 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:03:12 smithi022 ceph-mon[31985]: Detected new or changed devices on smithi079 2024-03-28T18:03:13.032 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:03:12 smithi022 ceph-mon[31985]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T18:03:13.032 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:03:12 smithi022 ceph-mon[31985]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T18:03:13.032 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:03:12 smithi022 ceph-mon[31985]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-28T18:03:13.032 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:03:12 smithi022 ceph-mon[31985]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-28T18:03:13.032 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:03:12 smithi022 ceph-mon[31985]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-03-28T18:03:13.033 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:03:12 smithi022 ceph-mon[31985]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-03-28T18:03:13.033 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:03:12 smithi022 ceph-mon[31985]: Adjusting osd_memory_target on smithi079 to 4272M 2024-03-28T18:03:13.033 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:03:12 smithi022 ceph-mon[31985]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T18:03:13.033 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:03:12 smithi022 ceph-mon[31985]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T18:03:13.033 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:03:12 smithi022 ceph-mon[31985]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T18:03:13.033 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:03:12 smithi022 ceph-mon[31985]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T18:03:13.033 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:03:12 smithi022 ceph-mon[31985]: osdmap e80: 11 total, 11 up, 11 in 2024-03-28T18:03:13.674 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:03:13 smithi134 ceph-mon[35037]: pgmap v311: 1 pgs: 1 remapped+peering; 577 KiB data, 357 MiB used, 983 GiB / 983 GiB avail 2024-03-28T18:03:13.674 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:03:13 smithi134 ceph-mon[35037]: osdmap e81: 11 total, 11 up, 11 in 2024-03-28T18:03:13.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:03:13 smithi079 ceph-mon[35312]: pgmap v311: 1 pgs: 1 remapped+peering; 577 KiB data, 357 MiB used, 983 GiB / 983 GiB avail 2024-03-28T18:03:13.933 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:03:13 smithi079 ceph-mon[35312]: osdmap e81: 11 total, 11 up, 11 in 2024-03-28T18:03:14.031 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:03:13 smithi022 ceph-mon[31985]: pgmap v311: 1 pgs: 1 remapped+peering; 577 KiB data, 357 MiB used, 983 GiB / 983 GiB avail 2024-03-28T18:03:14.032 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:03:13 smithi022 ceph-mon[31985]: osdmap e81: 11 total, 11 up, 11 in 2024-03-28T18:03:14.278 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-28T18:03:14.277+0000 7f94dd800700 1 -- 172.21.15.134:0/272688466 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f94d810a090 msgr2=0x7f94d810c520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:03:14.278 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-28T18:03:14.277+0000 7f94dd800700 1 --2- 172.21.15.134:0/272688466 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f94d810a090 0x7f94d810c520 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f94d0009990 tx=0x7f94d0009ca0 comp rx=0 tx=0).stop 2024-03-28T18:03:14.278 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-28T18:03:14.277+0000 7f94dd800700 1 -- 172.21.15.134:0/272688466 shutdown_connections 2024-03-28T18:03:14.278 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-28T18:03:14.277+0000 7f94dd800700 1 --2- 172.21.15.134:0/272688466 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f94d810a090 0x7f94d810c520 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:03:14.279 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-28T18:03:14.277+0000 7f94dd800700 1 --2- 172.21.15.134:0/272688466 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f94d80761c0 0x7f94d8074620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:03:14.279 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-28T18:03:14.277+0000 7f94dd800700 1 --2- 172.21.15.134:0/272688466 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f94d8074bf0 0x7f94d8075070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:03:14.279 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-28T18:03:14.277+0000 7f94dd800700 1 -- 172.21.15.134:0/272688466 >> 172.21.15.134:0/272688466 conn(0x7f94d80fd7a0 msgr2=0x7f94d80ffbc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:03:14.279 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-28T18:03:14.277+0000 7f94dd800700 1 -- 172.21.15.134:0/272688466 shutdown_connections 2024-03-28T18:03:14.279 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-28T18:03:14.277+0000 7f94dd800700 1 -- 172.21.15.134:0/272688466 wait complete. 2024-03-28T18:03:14.279 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-28T18:03:14.278+0000 7f94dd800700 1 Processor -- start 2024-03-28T18:03:14.279 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-28T18:03:14.278+0000 7f94dd800700 1 -- start start 2024-03-28T18:03:14.281 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-28T18:03:14.279+0000 7f94dd800700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f94d8074bf0 0x7f94d819a5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:03:14.281 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-28T18:03:14.279+0000 7f94dd800700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f94d80761c0 0x7f94d819ab10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:03:14.281 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-28T18:03:14.279+0000 7f94dd800700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f94d810a090 0x7f94d81a1b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:03:14.281 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-28T18:03:14.279+0000 7f94dd800700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94d8110020 con 0x7f94d8074bf0 2024-03-28T18:03:14.281 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-28T18:03:14.279+0000 7f94dd800700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94d810fea0 con 0x7f94d80761c0 2024-03-28T18:03:14.281 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-28T18:03:14.279+0000 7f94dd800700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94d81101a0 con 0x7f94d810a090 2024-03-28T18:03:14.281 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-28T18:03:14.279+0000 7f94d7400700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f94d810a090 0x7f94d81a1b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:03:14.281 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-28T18:03:14.279+0000 7f94d7400700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f94d810a090 0x7f94d81a1b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.134:32990/0 (socket says 172.21.15.134:32990) 2024-03-28T18:03:14.281 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-28T18:03:14.279+0000 7f94d7400700 1 -- 172.21.15.134:0/1357907287 learned_addr learned my addr 172.21.15.134:0/1357907287 (peer_addr_for_me v2:172.21.15.134:0/0) 2024-03-28T18:03:14.281 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-28T18:03:14.279+0000 7f94d6a00700 1 --2- 172.21.15.134:0/1357907287 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f94d8074bf0 0x7f94d819a5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:03:14.281 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-28T18:03:14.279+0000 7f94ce000700 1 --2- 172.21.15.134:0/1357907287 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f94d80761c0 0x7f94d819ab10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:03:14.282 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-28T18:03:14.280+0000 7f94d7400700 1 -- 172.21.15.134:0/1357907287 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f94d80761c0 msgr2=0x7f94d819ab10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:03:14.282 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-28T18:03:14.280+0000 7f94d7400700 1 --2- 172.21.15.134:0/1357907287 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f94d80761c0 0x7f94d819ab10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:03:14.282 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-28T18:03:14.280+0000 7f94d7400700 1 -- 172.21.15.134:0/1357907287 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f94d8074bf0 msgr2=0x7f94d819a5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:03:14.282 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-28T18:03:14.280+0000 7f94d7400700 1 --2- 172.21.15.134:0/1357907287 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f94d8074bf0 0x7f94d819a5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:03:14.282 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-28T18:03:14.280+0000 7f94d7400700 1 -- 172.21.15.134:0/1357907287 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f94d81a2230 con 0x7f94d810a090 2024-03-28T18:03:14.282 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-28T18:03:14.280+0000 7f94d7400700 1 --2- 172.21.15.134:0/1357907287 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f94d810a090 0x7f94d81a1b90 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f94d0005910 tx=0x7f94d0003680 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:03:14.282 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-28T18:03:14.280+0000 7f94cfe00700 1 -- 172.21.15.134:0/1357907287 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94d0027070 con 0x7f94d810a090 2024-03-28T18:03:14.282 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-28T18:03:14.280+0000 7f94cfe00700 1 -- 172.21.15.134:0/1357907287 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f94d00041c0 con 0x7f94d810a090 2024-03-28T18:03:14.282 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-28T18:03:14.280+0000 7f94cfe00700 1 -- 172.21.15.134:0/1357907287 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94d000f6c0 con 0x7f94d810a090 2024-03-28T18:03:14.282 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-28T18:03:14.280+0000 7f94dd800700 1 -- 172.21.15.134:0/1357907287 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f94d81a24c0 con 0x7f94d810a090 2024-03-28T18:03:14.282 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-28T18:03:14.280+0000 7f94dd800700 1 -- 172.21.15.134:0/1357907287 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f94d81a2910 con 0x7f94d810a090 2024-03-28T18:03:14.283 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-28T18:03:14.282+0000 7f94dd800700 1 -- 172.21.15.134:0/1357907287 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f94d8068e50 con 0x7f94d810a090 2024-03-28T18:03:14.283 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-28T18:03:14.283+0000 7f94cfe00700 1 -- 172.21.15.134:0/1357907287 <== mon.1 v2:172.21.15.134:3300/0 4 ==== mgrmap(e 16) v1 ==== 137290+0+0 (secure 0 0 0) 0x7f94d002a070 con 0x7f94d810a090 2024-03-28T18:03:14.284 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-28T18:03:14.285+0000 7f94cfe00700 1 --2- 172.21.15.134:0/1357907287 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7f94bc0a2910 0x7f94bc0a4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:03:14.285 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-28T18:03:14.285+0000 7f94cfe00700 1 -- 172.21.15.134:0/1357907287 <== mon.1 v2:172.21.15.134:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 6694+0+0 (secure 0 0 0) 0x7f94d00cf390 con 0x7f94d810a090 2024-03-28T18:03:14.285 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-28T18:03:14.285+0000 7f94d6a00700 1 --2- 172.21.15.134:0/1357907287 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7f94bc0a2910 0x7f94bc0a4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:03:14.286 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-28T18:03:14.286+0000 7f94d6a00700 1 --2- 172.21.15.134:0/1357907287 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7f94bc0a2910 0x7f94bc0a4dd0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f94c80060b0 tx=0x7f94c8007790 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:03:14.289 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-28T18:03:14.289+0000 7f94cfe00700 1 -- 172.21.15.134:0/1357907287 <== mon.1 v2:172.21.15.134:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f94d0013070 con 0x7f94d810a090 2024-03-28T18:03:14.527 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-28T18:03:14.526+0000 7f94dd800700 1 -- 172.21.15.134:0/1357907287 --> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi134:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) v1 -- 0x7f94d80625d0 con 0x7f94bc0a2910 2024-03-28T18:03:14.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:03:14 smithi134 ceph-mon[35037]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-28T18:03:14.925 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:03:14 smithi134 ceph-mon[35037]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-28T18:03:14.925 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:03:14 smithi134 ceph-mon[35037]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T18:03:14.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:03:14 smithi079 ceph-mon[35312]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-28T18:03:14.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:03:14 smithi079 ceph-mon[35312]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-28T18:03:14.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:03:14 smithi079 ceph-mon[35312]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T18:03:15.032 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:03:14 smithi022 ceph-mon[31985]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-28T18:03:15.032 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:03:14 smithi022 ceph-mon[31985]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-28T18:03:15.032 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:03:14 smithi022 ceph-mon[31985]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T18:03:15.766 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:03:15 smithi134 ceph-mon[35037]: from='client.24352 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi134:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-03-28T18:03:15.766 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:03:15 smithi134 ceph-mon[35037]: pgmap v313: 1 pgs: 1 peering; 577 KiB data, 358 MiB used, 983 GiB / 983 GiB avail; 143 KiB/s, 0 objects/s recovering 2024-03-28T18:03:15.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:03:15 smithi079 ceph-mon[35312]: from='client.24352 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi134:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-03-28T18:03:15.933 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:03:15 smithi079 ceph-mon[35312]: pgmap v313: 1 pgs: 1 peering; 577 KiB data, 358 MiB used, 983 GiB / 983 GiB avail; 143 KiB/s, 0 objects/s recovering 2024-03-28T18:03:16.032 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:03:15 smithi022 ceph-mon[31985]: from='client.24352 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi134:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-03-28T18:03:16.032 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:03:15 smithi022 ceph-mon[31985]: pgmap v313: 1 pgs: 1 peering; 577 KiB data, 358 MiB used, 983 GiB / 983 GiB avail; 143 KiB/s, 0 objects/s recovering 2024-03-28T18:03:17.674 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:03:17 smithi134 ceph-mon[35037]: pgmap v314: 1 pgs: 1 peering; 577 KiB data, 358 MiB used, 983 GiB / 983 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-03-28T18:03:17.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:03:17 smithi079 ceph-mon[35312]: pgmap v314: 1 pgs: 1 peering; 577 KiB data, 358 MiB used, 983 GiB / 983 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-03-28T18:03:18.031 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:03:17 smithi022 ceph-mon[31985]: pgmap v314: 1 pgs: 1 peering; 577 KiB data, 358 MiB used, 983 GiB / 983 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-03-28T18:03:18.925 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:03:18 smithi134 ceph-mon[35037]: from='client.? 172.21.15.134:0/2098466337' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "1427bca7-3414-4468-9246-e999156bc2ae"}]: dispatch 2024-03-28T18:03:18.925 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:03:18 smithi134 ceph-mon[35037]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "1427bca7-3414-4468-9246-e999156bc2ae"}]: dispatch 2024-03-28T18:03:18.925 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:03:18 smithi134 ceph-mon[35037]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "1427bca7-3414-4468-9246-e999156bc2ae"}]': finished 2024-03-28T18:03:18.925 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:03:18 smithi134 ceph-mon[35037]: osdmap e82: 12 total, 11 up, 12 in 2024-03-28T18:03:18.925 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:03:18 smithi134 ceph-mon[35037]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-28T18:03:18.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:03:18 smithi079 ceph-mon[35312]: from='client.? 172.21.15.134:0/2098466337' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "1427bca7-3414-4468-9246-e999156bc2ae"}]: dispatch 2024-03-28T18:03:18.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:03:18 smithi079 ceph-mon[35312]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "1427bca7-3414-4468-9246-e999156bc2ae"}]: dispatch 2024-03-28T18:03:18.933 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:03:18 smithi079 ceph-mon[35312]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "1427bca7-3414-4468-9246-e999156bc2ae"}]': finished 2024-03-28T18:03:18.933 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:03:18 smithi079 ceph-mon[35312]: osdmap e82: 12 total, 11 up, 12 in 2024-03-28T18:03:18.933 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:03:18 smithi079 ceph-mon[35312]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-28T18:03:19.032 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:03:18 smithi022 ceph-mon[31985]: from='client.? 172.21.15.134:0/2098466337' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "1427bca7-3414-4468-9246-e999156bc2ae"}]: dispatch 2024-03-28T18:03:19.032 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:03:18 smithi022 ceph-mon[31985]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "1427bca7-3414-4468-9246-e999156bc2ae"}]: dispatch 2024-03-28T18:03:19.032 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:03:18 smithi022 ceph-mon[31985]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "1427bca7-3414-4468-9246-e999156bc2ae"}]': finished 2024-03-28T18:03:19.032 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:03:18 smithi022 ceph-mon[31985]: osdmap e82: 12 total, 11 up, 12 in 2024-03-28T18:03:19.032 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:03:18 smithi022 ceph-mon[31985]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-28T18:03:19.933 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:03:19 smithi079 ceph-mon[35312]: pgmap v316: 1 pgs: 1 active+clean; 577 KiB data, 358 MiB used, 983 GiB / 983 GiB avail; 95 KiB/s, 0 objects/s recovering 2024-03-28T18:03:19.933 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:03:19 smithi079 ceph-mon[35312]: from='client.? 172.21.15.134:0/3239346913' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-28T18:03:20.032 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:03:19 smithi022 ceph-mon[31985]: pgmap v316: 1 pgs: 1 active+clean; 577 KiB data, 358 MiB used, 983 GiB / 983 GiB avail; 95 KiB/s, 0 objects/s recovering 2024-03-28T18:03:20.032 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:03:19 smithi022 ceph-mon[31985]: from='client.? 172.21.15.134:0/3239346913' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-28T18:03:20.174 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:03:19 smithi134 ceph-mon[35037]: pgmap v316: 1 pgs: 1 active+clean; 577 KiB data, 358 MiB used, 983 GiB / 983 GiB avail; 95 KiB/s, 0 objects/s recovering 2024-03-28T18:03:20.175 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:03:19 smithi134 ceph-mon[35037]: from='client.? 172.21.15.134:0/3239346913' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-28T18:03:21.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:03:21 smithi134 ceph-mon[35037]: pgmap v317: 1 pgs: 1 active+clean; 577 KiB data, 358 MiB used, 983 GiB / 983 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-03-28T18:03:21.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:03:21 smithi079 ceph-mon[35312]: pgmap v317: 1 pgs: 1 active+clean; 577 KiB data, 358 MiB used, 983 GiB / 983 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-03-28T18:03:22.032 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:03:21 smithi022 ceph-mon[31985]: pgmap v317: 1 pgs: 1 active+clean; 577 KiB data, 358 MiB used, 983 GiB / 983 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-03-28T18:03:23.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:03:23 smithi134 ceph-mon[35037]: pgmap v318: 1 pgs: 1 active+clean; 577 KiB data, 358 MiB used, 983 GiB / 983 GiB avail 2024-03-28T18:03:23.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:03:23 smithi079 ceph-mon[35312]: pgmap v318: 1 pgs: 1 active+clean; 577 KiB data, 358 MiB used, 983 GiB / 983 GiB avail 2024-03-28T18:03:24.031 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:03:23 smithi022 ceph-mon[31985]: pgmap v318: 1 pgs: 1 active+clean; 577 KiB data, 358 MiB used, 983 GiB / 983 GiB avail 2024-03-28T18:03:25.925 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:03:25 smithi134 ceph-mon[35037]: pgmap v319: 1 pgs: 1 active+clean; 577 KiB data, 358 MiB used, 983 GiB / 983 GiB avail 2024-03-28T18:03:25.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:03:25 smithi079 ceph-mon[35312]: pgmap v319: 1 pgs: 1 active+clean; 577 KiB data, 358 MiB used, 983 GiB / 983 GiB avail 2024-03-28T18:03:26.032 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:03:25 smithi022 ceph-mon[31985]: pgmap v319: 1 pgs: 1 active+clean; 577 KiB data, 358 MiB used, 983 GiB / 983 GiB avail 2024-03-28T18:03:27.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:03:27 smithi134 ceph-mon[35037]: pgmap v320: 1 pgs: 1 active+clean; 577 KiB data, 358 MiB used, 983 GiB / 983 GiB avail 2024-03-28T18:03:28.031 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:03:27 smithi022 ceph-mon[31985]: pgmap v320: 1 pgs: 1 active+clean; 577 KiB data, 358 MiB used, 983 GiB / 983 GiB avail 2024-03-28T18:03:28.182 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:03:27 smithi079 ceph-mon[35312]: pgmap v320: 1 pgs: 1 active+clean; 577 KiB data, 358 MiB used, 983 GiB / 983 GiB avail 2024-03-28T18:03:29.032 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:03:28 smithi022 ceph-mon[31985]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-03-28T18:03:29.032 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:03:28 smithi022 ceph-mon[31985]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T18:03:29.032 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:03:28 smithi022 ceph-mon[31985]: Deploying daemon osd.11 on smithi134 2024-03-28T18:03:29.121 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:03:28 smithi134 ceph-mon[35037]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-03-28T18:03:29.121 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:03:28 smithi134 ceph-mon[35037]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T18:03:29.121 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:03:28 smithi134 ceph-mon[35037]: Deploying daemon osd.11 on smithi134 2024-03-28T18:03:29.182 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:03:28 smithi079 ceph-mon[35312]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-03-28T18:03:29.182 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:03:28 smithi079 ceph-mon[35312]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T18:03:29.182 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:03:28 smithi079 ceph-mon[35312]: Deploying daemon osd.11 on smithi134 2024-03-28T18:03:29.963 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:03:29 smithi134 ceph-mon[35037]: pgmap v321: 1 pgs: 1 active+clean; 577 KiB data, 358 MiB used, 983 GiB / 983 GiB avail 2024-03-28T18:03:30.032 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:03:29 smithi022 ceph-mon[31985]: pgmap v321: 1 pgs: 1 active+clean; 577 KiB data, 358 MiB used, 983 GiB / 983 GiB avail 2024-03-28T18:03:30.182 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:03:29 smithi079 ceph-mon[35312]: pgmap v321: 1 pgs: 1 active+clean; 577 KiB data, 358 MiB used, 983 GiB / 983 GiB avail 2024-03-28T18:03:32.032 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:03:31 smithi022 ceph-mon[31985]: pgmap v322: 1 pgs: 1 active+clean; 577 KiB data, 358 MiB used, 983 GiB / 983 GiB avail 2024-03-28T18:03:32.163 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:03:31 smithi134 ceph-mon[35037]: pgmap v322: 1 pgs: 1 active+clean; 577 KiB data, 358 MiB used, 983 GiB / 983 GiB avail 2024-03-28T18:03:32.182 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:03:31 smithi079 ceph-mon[35312]: pgmap v322: 1 pgs: 1 active+clean; 577 KiB data, 358 MiB used, 983 GiB / 983 GiB avail 2024-03-28T18:03:33.896 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:03:33 smithi134 ceph-mon[35037]: pgmap v323: 1 pgs: 1 active+clean; 577 KiB data, 358 MiB used, 983 GiB / 983 GiB avail 2024-03-28T18:03:34.031 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:03:33 smithi022 ceph-mon[31985]: pgmap v323: 1 pgs: 1 active+clean; 577 KiB data, 358 MiB used, 983 GiB / 983 GiB avail 2024-03-28T18:03:34.182 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:03:33 smithi079 ceph-mon[35312]: pgmap v323: 1 pgs: 1 active+clean; 577 KiB data, 358 MiB used, 983 GiB / 983 GiB avail 2024-03-28T18:03:35.841 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:03:35 smithi134 ceph-mon[35037]: pgmap v324: 1 pgs: 1 active+clean; 577 KiB data, 358 MiB used, 983 GiB / 983 GiB avail 2024-03-28T18:03:36.032 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:03:35 smithi022 ceph-mon[31985]: pgmap v324: 1 pgs: 1 active+clean; 577 KiB data, 358 MiB used, 983 GiB / 983 GiB avail 2024-03-28T18:03:36.182 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:03:35 smithi079 ceph-mon[35312]: pgmap v324: 1 pgs: 1 active+clean; 577 KiB data, 358 MiB used, 983 GiB / 983 GiB avail 2024-03-28T18:03:37.620 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:03:37 smithi134 ceph-mon[35037]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T18:03:37.620 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:03:37 smithi134 ceph-mon[35037]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T18:03:37.620 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:03:37 smithi134 ceph-mon[35037]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T18:03:37.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:03:37 smithi079 ceph-mon[35312]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T18:03:37.683 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:03:37 smithi079 ceph-mon[35312]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T18:03:37.683 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:03:37 smithi079 ceph-mon[35312]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T18:03:37.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:03:37 smithi022 ceph-mon[31985]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T18:03:37.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:03:37 smithi022 ceph-mon[31985]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T18:03:37.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:03:37 smithi022 ceph-mon[31985]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T18:03:38.674 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:03:38 smithi134 ceph-mon[35037]: pgmap v325: 1 pgs: 1 active+clean; 577 KiB data, 358 MiB used, 983 GiB / 983 GiB avail 2024-03-28T18:03:38.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:03:38 smithi079 ceph-mon[35312]: pgmap v325: 1 pgs: 1 active+clean; 577 KiB data, 358 MiB used, 983 GiB / 983 GiB avail 2024-03-28T18:03:38.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:03:38 smithi022 ceph-mon[31985]: pgmap v325: 1 pgs: 1 active+clean; 577 KiB data, 358 MiB used, 983 GiB / 983 GiB avail 2024-03-28T18:03:40.674 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:03:40 smithi134 ceph-mon[35037]: pgmap v326: 1 pgs: 1 active+clean; 577 KiB data, 358 MiB used, 983 GiB / 983 GiB avail 2024-03-28T18:03:40.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:03:40 smithi134 ceph-mon[35037]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T18:03:40.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:03:40 smithi134 ceph-mon[35037]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T18:03:40.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:03:40 smithi134 ceph-mon[35037]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T18:03:40.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:03:40 smithi134 ceph-mon[35037]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T18:03:40.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:03:40 smithi134 ceph-mon[35037]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T18:03:40.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:03:40 smithi134 ceph-mon[35037]: from='osd.11 [v2:172.21.15.134:6824/4271716428,v1:172.21.15.134:6825/4271716428]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-03-28T18:03:40.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:03:40 smithi134 ceph-mon[35037]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-03-28T18:03:40.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:03:40 smithi079 ceph-mon[35312]: pgmap v326: 1 pgs: 1 active+clean; 577 KiB data, 358 MiB used, 983 GiB / 983 GiB avail 2024-03-28T18:03:40.683 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:03:40 smithi079 ceph-mon[35312]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T18:03:40.683 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:03:40 smithi079 ceph-mon[35312]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T18:03:40.683 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:03:40 smithi079 ceph-mon[35312]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T18:03:40.683 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:03:40 smithi079 ceph-mon[35312]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T18:03:40.683 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:03:40 smithi079 ceph-mon[35312]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T18:03:40.683 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:03:40 smithi079 ceph-mon[35312]: from='osd.11 [v2:172.21.15.134:6824/4271716428,v1:172.21.15.134:6825/4271716428]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-03-28T18:03:40.683 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:03:40 smithi079 ceph-mon[35312]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-03-28T18:03:40.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:03:40 smithi022 ceph-mon[31985]: pgmap v326: 1 pgs: 1 active+clean; 577 KiB data, 358 MiB used, 983 GiB / 983 GiB avail 2024-03-28T18:03:40.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:03:40 smithi022 ceph-mon[31985]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T18:03:40.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:03:40 smithi022 ceph-mon[31985]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T18:03:40.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:03:40 smithi022 ceph-mon[31985]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T18:03:40.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:03:40 smithi022 ceph-mon[31985]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T18:03:40.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:03:40 smithi022 ceph-mon[31985]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T18:03:40.783 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:03:40 smithi022 ceph-mon[31985]: from='osd.11 [v2:172.21.15.134:6824/4271716428,v1:172.21.15.134:6825/4271716428]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-03-28T18:03:40.783 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:03:40 smithi022 ceph-mon[31985]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-03-28T18:03:41.357 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-28T18:03:41.355+0000 7f94cfe00700 1 -- 172.21.15.134:0/1357907287 <== mgr.14152 v2:172.21.15.22:6800/4201048392 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+38 (secure 0 0 0) 0x7f94d80625d0 con 0x7f94bc0a2910 2024-03-28T18:03:41.357 INFO:teuthology.orchestra.run.smithi134.stdout:Created osd(s) 11 on host 'smithi134' 2024-03-28T18:03:41.360 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-28T18:03:41.359+0000 7f94dd800700 1 -- 172.21.15.134:0/1357907287 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7f94bc0a2910 msgr2=0x7f94bc0a4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:03:41.360 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-28T18:03:41.359+0000 7f94dd800700 1 --2- 172.21.15.134:0/1357907287 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7f94bc0a2910 0x7f94bc0a4dd0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f94c80060b0 tx=0x7f94c8007790 comp rx=0 tx=0).stop 2024-03-28T18:03:41.360 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-28T18:03:41.359+0000 7f94dd800700 1 -- 172.21.15.134:0/1357907287 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f94d810a090 msgr2=0x7f94d81a1b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:03:41.360 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-28T18:03:41.359+0000 7f94dd800700 1 --2- 172.21.15.134:0/1357907287 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f94d810a090 0x7f94d81a1b90 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f94d0005910 tx=0x7f94d0003680 comp rx=0 tx=0).stop 2024-03-28T18:03:41.360 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-28T18:03:41.359+0000 7f94dd800700 1 -- 172.21.15.134:0/1357907287 shutdown_connections 2024-03-28T18:03:41.360 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-28T18:03:41.359+0000 7f94dd800700 1 --2- 172.21.15.134:0/1357907287 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7f94bc0a2910 0x7f94bc0a4dd0 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:03:41.360 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-28T18:03:41.359+0000 7f94dd800700 1 --2- 172.21.15.134:0/1357907287 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f94d810a090 0x7f94d81a1b90 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:03:41.360 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-28T18:03:41.359+0000 7f94dd800700 1 --2- 172.21.15.134:0/1357907287 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f94d8074bf0 0x7f94d819a5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:03:41.361 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-28T18:03:41.359+0000 7f94dd800700 1 --2- 172.21.15.134:0/1357907287 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f94d80761c0 0x7f94d819ab10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:03:41.361 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-28T18:03:41.360+0000 7f94dd800700 1 -- 172.21.15.134:0/1357907287 >> 172.21.15.134:0/1357907287 conn(0x7f94d80fd7a0 msgr2=0x7f94d8105f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:03:41.361 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-28T18:03:41.360+0000 7f94dd800700 1 -- 172.21.15.134:0/1357907287 shutdown_connections 2024-03-28T18:03:41.361 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-28T18:03:41.360+0000 7f94dd800700 1 -- 172.21.15.134:0/1357907287 wait complete. 2024-03-28T18:03:42.109 DEBUG:teuthology.orchestra.run.smithi134:osd.11> sudo journalctl -f -n 0 -u ceph-fd220748-ed2b-11ee-95d0-87774f69a715@osd.11.service 2024-03-28T18:03:42.114 INFO:tasks.cephadm:Waiting for 12 OSDs to come up... 2024-03-28T18:03:42.114 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fd220748-ed2b-11ee-95d0-87774f69a715 -- ceph osd stat -f json 2024-03-28T18:03:42.353 INFO:journalctl@ceph.osd.11.smithi134.stdout:Mar 28 18:03:42 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-osd-11[60388]: 2024-03-28T18:03:42.225+0000 7f3437000700 -1 osd.11 0 waiting for initial osdmap 2024-03-28T18:03:42.353 INFO:journalctl@ceph.osd.11.smithi134.stdout:Mar 28 18:03:42 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-osd-11[60388]: 2024-03-28T18:03:42.269+0000 7f3431e00700 -1 osd.11 84 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-28T18:03:42.353 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:03:42 smithi134 ceph-mon[35037]: pgmap v327: 1 pgs: 1 active+clean; 577 KiB data, 358 MiB used, 983 GiB / 983 GiB avail 2024-03-28T18:03:42.354 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:03:42 smithi134 ceph-mon[35037]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-03-28T18:03:42.354 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:03:42 smithi134 ceph-mon[35037]: osdmap e83: 12 total, 11 up, 12 in 2024-03-28T18:03:42.354 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:03:42 smithi134 ceph-mon[35037]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-28T18:03:42.354 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:03:42 smithi134 ceph-mon[35037]: from='osd.11 [v2:172.21.15.134:6824/4271716428,v1:172.21.15.134:6825/4271716428]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi134", "root=default"]}]: dispatch 2024-03-28T18:03:42.354 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:03:42 smithi134 ceph-mon[35037]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi134", "root=default"]}]: dispatch 2024-03-28T18:03:42.354 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:03:42 smithi134 ceph-mon[35037]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T18:03:42.354 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:03:42 smithi134 ceph-mon[35037]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T18:03:42.354 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:03:42 smithi134 ceph-mon[35037]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T18:03:42.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:03:42 smithi022 ceph-mon[31985]: pgmap v327: 1 pgs: 1 active+clean; 577 KiB data, 358 MiB used, 983 GiB / 983 GiB avail 2024-03-28T18:03:42.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:03:42 smithi022 ceph-mon[31985]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-03-28T18:03:42.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:03:42 smithi022 ceph-mon[31985]: osdmap e83: 12 total, 11 up, 12 in 2024-03-28T18:03:42.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:03:42 smithi022 ceph-mon[31985]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-28T18:03:42.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:03:42 smithi022 ceph-mon[31985]: from='osd.11 [v2:172.21.15.134:6824/4271716428,v1:172.21.15.134:6825/4271716428]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi134", "root=default"]}]: dispatch 2024-03-28T18:03:42.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:03:42 smithi022 ceph-mon[31985]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi134", "root=default"]}]: dispatch 2024-03-28T18:03:42.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:03:42 smithi022 ceph-mon[31985]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T18:03:42.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:03:42 smithi022 ceph-mon[31985]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T18:03:42.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:03:42 smithi022 ceph-mon[31985]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T18:03:42.586 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/fd220748-ed2b-11ee-95d0-87774f69a715/mon.a/config 2024-03-28T18:03:42.683 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:03:42 smithi079 ceph-mon[35312]: pgmap v327: 1 pgs: 1 active+clean; 577 KiB data, 358 MiB used, 983 GiB / 983 GiB avail 2024-03-28T18:03:42.683 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:03:42 smithi079 ceph-mon[35312]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-03-28T18:03:42.683 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:03:42 smithi079 ceph-mon[35312]: osdmap e83: 12 total, 11 up, 12 in 2024-03-28T18:03:42.683 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:03:42 smithi079 ceph-mon[35312]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-28T18:03:42.683 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:03:42 smithi079 ceph-mon[35312]: from='osd.11 [v2:172.21.15.134:6824/4271716428,v1:172.21.15.134:6825/4271716428]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi134", "root=default"]}]: dispatch 2024-03-28T18:03:42.683 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:03:42 smithi079 ceph-mon[35312]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi134", "root=default"]}]: dispatch 2024-03-28T18:03:42.683 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:03:42 smithi079 ceph-mon[35312]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T18:03:42.683 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:03:42 smithi079 ceph-mon[35312]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T18:03:42.684 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:03:42 smithi079 ceph-mon[35312]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T18:03:43.425 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:03:43 smithi134 ceph-mon[35037]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi134", "root=default"]}]': finished 2024-03-28T18:03:43.425 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:03:43 smithi134 ceph-mon[35037]: osdmap e84: 12 total, 11 up, 12 in 2024-03-28T18:03:43.425 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:03:43 smithi134 ceph-mon[35037]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-28T18:03:43.425 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:03:43 smithi134 ceph-mon[35037]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-28T18:03:43.510 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:03:43 smithi022 ceph-mon[31985]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi134", "root=default"]}]': finished 2024-03-28T18:03:43.510 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:03:43 smithi022 ceph-mon[31985]: osdmap e84: 12 total, 11 up, 12 in 2024-03-28T18:03:43.510 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:03:43 smithi022 ceph-mon[31985]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-28T18:03:43.510 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:03:43 smithi022 ceph-mon[31985]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-28T18:03:43.683 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:03:43 smithi079 ceph-mon[35312]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi134", "root=default"]}]': finished 2024-03-28T18:03:43.683 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:03:43 smithi079 ceph-mon[35312]: osdmap e84: 12 total, 11 up, 12 in 2024-03-28T18:03:43.683 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:03:43 smithi079 ceph-mon[35312]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-28T18:03:43.683 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:03:43 smithi079 ceph-mon[35312]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-28T18:03:44.236 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:03:44 smithi022 ceph-mon[31985]: purged_snaps scrub starts 2024-03-28T18:03:44.236 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:03:44 smithi022 ceph-mon[31985]: purged_snaps scrub ok 2024-03-28T18:03:44.236 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:03:44 smithi022 ceph-mon[31985]: pgmap v330: 1 pgs: 1 active+clean; 577 KiB data, 358 MiB used, 983 GiB / 983 GiB avail 2024-03-28T18:03:44.236 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:03:44 smithi022 ceph-mon[31985]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-28T18:03:44.236 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:03:44 smithi022 ceph-mon[31985]: osd.11 [v2:172.21.15.134:6824/4271716428,v1:172.21.15.134:6825/4271716428] boot 2024-03-28T18:03:44.425 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:03:44 smithi134 ceph-mon[35037]: purged_snaps scrub starts 2024-03-28T18:03:44.425 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:03:44 smithi134 ceph-mon[35037]: purged_snaps scrub ok 2024-03-28T18:03:44.425 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:03:44 smithi134 ceph-mon[35037]: pgmap v330: 1 pgs: 1 active+clean; 577 KiB data, 358 MiB used, 983 GiB / 983 GiB avail 2024-03-28T18:03:44.425 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:03:44 smithi134 ceph-mon[35037]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-28T18:03:44.425 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:03:44 smithi134 ceph-mon[35037]: osd.11 [v2:172.21.15.134:6824/4271716428,v1:172.21.15.134:6825/4271716428] boot 2024-03-28T18:03:44.425 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:03:44 smithi134 ceph-mon[35037]: osdmap e85: 12 total, 12 up, 12 in 2024-03-28T18:03:44.425 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:03:44 smithi134 ceph-mon[35037]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-28T18:03:44.531 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:03:44 smithi022 ceph-mon[31985]: osdmap e85: 12 total, 12 up, 12 in 2024-03-28T18:03:44.531 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:03:44 smithi022 ceph-mon[31985]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-28T18:03:44.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:03:44 smithi079 ceph-mon[35312]: purged_snaps scrub starts 2024-03-28T18:03:44.683 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:03:44 smithi079 ceph-mon[35312]: purged_snaps scrub ok 2024-03-28T18:03:44.683 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:03:44 smithi079 ceph-mon[35312]: pgmap v330: 1 pgs: 1 active+clean; 577 KiB data, 358 MiB used, 983 GiB / 983 GiB avail 2024-03-28T18:03:44.683 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:03:44 smithi079 ceph-mon[35312]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-28T18:03:44.683 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:03:44 smithi079 ceph-mon[35312]: osd.11 [v2:172.21.15.134:6824/4271716428,v1:172.21.15.134:6825/4271716428] boot 2024-03-28T18:03:44.683 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:03:44 smithi079 ceph-mon[35312]: osdmap e85: 12 total, 12 up, 12 in 2024-03-28T18:03:44.683 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:03:44 smithi079 ceph-mon[35312]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-28T18:03:44.866 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:03:44.865+0000 7f7cebe00700 1 -- 172.21.15.22:0/2868080478 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7ce4105fa0 msgr2=0x7f7ce410c850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:03:44.866 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:03:44.865+0000 7f7cebe00700 1 --2- 172.21.15.22:0/2868080478 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7ce4105fa0 0x7f7ce410c850 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f7cdc00b0d0 tx=0x7f7cdc00b3e0 comp rx=0 tx=0).stop 2024-03-28T18:03:44.866 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:03:44.865+0000 7f7cebe00700 1 -- 172.21.15.22:0/2868080478 shutdown_connections 2024-03-28T18:03:44.866 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:03:44.865+0000 7f7cebe00700 1 --2- 172.21.15.22:0/2868080478 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7ce4105fa0 0x7f7ce410c850 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:03:44.866 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:03:44.865+0000 7f7cebe00700 1 --2- 172.21.15.22:0/2868080478 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f7ce4104400 0x7f7ce4104820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:03:44.866 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:03:44.865+0000 7f7cebe00700 1 --2- 172.21.15.22:0/2868080478 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7ce4105600 0x7f7ce4105a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:03:44.866 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:03:44.865+0000 7f7cebe00700 1 -- 172.21.15.22:0/2868080478 >> 172.21.15.22:0/2868080478 conn(0x7f7ce40ff9e0 msgr2=0x7f7ce4101e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:03:44.866 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:03:44.865+0000 7f7cebe00700 1 -- 172.21.15.22:0/2868080478 shutdown_connections 2024-03-28T18:03:44.866 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:03:44.865+0000 7f7cebe00700 1 -- 172.21.15.22:0/2868080478 wait complete. 2024-03-28T18:03:44.868 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:03:44.866+0000 7f7cebe00700 1 Processor -- start 2024-03-28T18:03:44.868 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:03:44.867+0000 7f7cebe00700 1 -- start start 2024-03-28T18:03:44.868 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:03:44.867+0000 7f7cebe00700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f7ce4104400 0x7f7ce419a520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:03:44.868 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:03:44.867+0000 7f7cebe00700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7ce4105600 0x7f7ce419aa60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:03:44.868 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:03:44.867+0000 7f7cebe00700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7ce4105fa0 0x7f7ce41a1ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:03:44.868 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:03:44.867+0000 7f7cebe00700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ce410fff0 con 0x7f7ce4105fa0 2024-03-28T18:03:44.868 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:03:44.867+0000 7f7cebe00700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ce410fe70 con 0x7f7ce4104400 2024-03-28T18:03:44.869 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:03:44.867+0000 7f7cebe00700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ce4110170 con 0x7f7ce4105600 2024-03-28T18:03:44.869 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:03:44.867+0000 7f7ce9e00700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7ce4105fa0 0x7f7ce41a1ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:03:44.869 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:03:44.867+0000 7f7ce9e00700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7ce4105fa0 0x7f7ce41a1ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:38140/0 (socket says 172.21.15.22:38140) 2024-03-28T18:03:44.869 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:03:44.867+0000 7f7ce9e00700 1 -- 172.21.15.22:0/749077439 learned_addr learned my addr 172.21.15.22:0/749077439 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-28T18:03:44.869 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:03:44.868+0000 7f7ce9400700 1 --2- 172.21.15.22:0/749077439 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f7ce4104400 0x7f7ce419a520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:03:44.869 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:03:44.868+0000 7f7ce8a00700 1 --2- 172.21.15.22:0/749077439 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7ce4105600 0x7f7ce419aa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:03:44.869 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:03:44.868+0000 7f7ce9e00700 1 -- 172.21.15.22:0/749077439 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7ce4105600 msgr2=0x7f7ce419aa60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:03:44.869 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:03:44.868+0000 7f7ce9e00700 1 --2- 172.21.15.22:0/749077439 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7ce4105600 0x7f7ce419aa60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:03:44.869 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:03:44.868+0000 7f7ce9e00700 1 -- 172.21.15.22:0/749077439 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f7ce4104400 msgr2=0x7f7ce419a520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:03:44.869 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:03:44.868+0000 7f7ce9e00700 1 --2- 172.21.15.22:0/749077439 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f7ce4104400 0x7f7ce419a520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:03:44.869 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:03:44.868+0000 7f7ce9e00700 1 -- 172.21.15.22:0/749077439 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ce41a2180 con 0x7f7ce4105fa0 2024-03-28T18:03:44.869 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:03:44.868+0000 7f7ce9400700 1 --2- 172.21.15.22:0/749077439 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f7ce4104400 0x7f7ce419a520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-28T18:03:44.869 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:03:44.868+0000 7f7ce9e00700 1 --2- 172.21.15.22:0/749077439 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7ce4105fa0 0x7f7ce41a1ae0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f7cdc007a20 tx=0x7f7cdc007b00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:03:44.870 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:03:44.869+0000 7f7ce2000700 1 -- 172.21.15.22:0/749077439 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7cdc021070 con 0x7f7ce4105fa0 2024-03-28T18:03:44.870 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:03:44.869+0000 7f7ce2000700 1 -- 172.21.15.22:0/749077439 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f7cdc01db20 con 0x7f7ce4105fa0 2024-03-28T18:03:44.870 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:03:44.869+0000 7f7cebe00700 1 -- 172.21.15.22:0/749077439 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7ce41a2410 con 0x7f7ce4105fa0 2024-03-28T18:03:44.870 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:03:44.869+0000 7f7ce2000700 1 -- 172.21.15.22:0/749077439 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7cdc01bbe0 con 0x7f7ce4105fa0 2024-03-28T18:03:44.871 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:03:44.869+0000 7f7cebe00700 1 -- 172.21.15.22:0/749077439 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7ce41a2870 con 0x7f7ce4105fa0 2024-03-28T18:03:44.871 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:03:44.870+0000 7f7cebe00700 1 -- 172.21.15.22:0/749077439 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7ce4068e50 con 0x7f7ce4105fa0 2024-03-28T18:03:44.872 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:03:44.872+0000 7f7ce2000700 1 -- 172.21.15.22:0/749077439 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 16) v1 ==== 137290+0+0 (secure 0 0 0) 0x7f7cdc024070 con 0x7f7ce4105fa0 2024-03-28T18:03:44.873 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:03:44.873+0000 7f7ce2000700 1 --2- 172.21.15.22:0/749077439 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7f7ccc0a2890 0x7f7ccc0a4d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:03:44.873 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:03:44.874+0000 7f7ce2000700 1 -- 172.21.15.22:0/749077439 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f7cdc0cf860 con 0x7f7ce4105fa0 2024-03-28T18:03:44.873 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:03:44.874+0000 7f7ce9400700 1 --2- 172.21.15.22:0/749077439 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7f7ccc0a2890 0x7f7ccc0a4d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:03:44.874 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:03:44.874+0000 7f7ce9400700 1 --2- 172.21.15.22:0/749077439 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7f7ccc0a2890 0x7f7ccc0a4d50 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f7cd8005950 tx=0x7f7cd80058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:03:44.878 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:03:44.878+0000 7f7ce2000700 1 -- 172.21.15.22:0/749077439 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f7cdc09cd90 con 0x7f7ce4105fa0 2024-03-28T18:03:45.107 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:03:45.107+0000 7f7cebe00700 1 -- 172.21.15.22:0/749077439 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd stat", "format": "json"} v 0) v1 -- 0x7f7ce4062da0 con 0x7f7ce4105fa0 2024-03-28T18:03:45.108 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:03:45.108+0000 7f7ce2000700 1 -- 172.21.15.22:0/749077439 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v86) v1 ==== 74+0+133 (secure 0 0 0) 0x7f7cdc09c690 con 0x7f7ce4105fa0 2024-03-28T18:03:45.108 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-03-28T18:03:45.111 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:03:45.111+0000 7f7cebe00700 1 -- 172.21.15.22:0/749077439 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7f7ccc0a2890 msgr2=0x7f7ccc0a4d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:03:45.111 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:03:45.111+0000 7f7cebe00700 1 --2- 172.21.15.22:0/749077439 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7f7ccc0a2890 0x7f7ccc0a4d50 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f7cd8005950 tx=0x7f7cd80058e0 comp rx=0 tx=0).stop 2024-03-28T18:03:45.111 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:03:45.111+0000 7f7cebe00700 1 -- 172.21.15.22:0/749077439 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7ce4105fa0 msgr2=0x7f7ce41a1ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:03:45.111 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:03:45.111+0000 7f7cebe00700 1 --2- 172.21.15.22:0/749077439 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7ce4105fa0 0x7f7ce41a1ae0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f7cdc007a20 tx=0x7f7cdc007b00 comp rx=0 tx=0).stop 2024-03-28T18:03:45.112 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:03:45.111+0000 7f7cebe00700 1 -- 172.21.15.22:0/749077439 shutdown_connections 2024-03-28T18:03:45.112 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:03:45.111+0000 7f7cebe00700 1 --2- 172.21.15.22:0/749077439 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7ce4105fa0 0x7f7ce41a1ae0 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:03:45.112 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:03:45.111+0000 7f7cebe00700 1 --2- 172.21.15.22:0/749077439 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f7ce4104400 0x7f7ce419a520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:03:45.112 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:03:45.111+0000 7f7cebe00700 1 --2- 172.21.15.22:0/749077439 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7f7ccc0a2890 0x7f7ccc0a4d50 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:03:45.112 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:03:45.111+0000 7f7cebe00700 1 --2- 172.21.15.22:0/749077439 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7ce4105600 0x7f7ce419aa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:03:45.112 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:03:45.111+0000 7f7cebe00700 1 -- 172.21.15.22:0/749077439 >> 172.21.15.22:0/749077439 conn(0x7f7ce40ff9e0 msgr2=0x7f7ce41088a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:03:45.112 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:03:45.111+0000 7f7cebe00700 1 -- 172.21.15.22:0/749077439 shutdown_connections 2024-03-28T18:03:45.112 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:03:45.112+0000 7f7cebe00700 1 -- 172.21.15.22:0/749077439 wait complete. 2024-03-28T18:03:45.520 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:03:45 smithi022 ceph-mon[31985]: osdmap e86: 12 total, 12 up, 12 in 2024-03-28T18:03:45.520 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:03:45 smithi022 ceph-mon[31985]: from='client.? 172.21.15.22:0/749077439' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-03-28T18:03:45.674 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:03:45 smithi134 ceph-mon[35037]: osdmap e86: 12 total, 12 up, 12 in 2024-03-28T18:03:45.674 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:03:45 smithi134 ceph-mon[35037]: from='client.? 172.21.15.22:0/749077439' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-03-28T18:03:45.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:03:45 smithi079 ceph-mon[35312]: osdmap e86: 12 total, 12 up, 12 in 2024-03-28T18:03:45.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:03:45 smithi079 ceph-mon[35312]: from='client.? 172.21.15.22:0/749077439' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-03-28T18:03:46.014 INFO:teuthology.orchestra.run.smithi022.stdout:{"epoch":86,"num_osds":12,"num_up_osds":12,"osd_up_since":1711649023,"num_in_osds":12,"osd_in_since":1711648998,"num_remapped_pgs":0} 2024-03-28T18:03:46.015 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033 shell --fsid fd220748-ed2b-11ee-95d0-87774f69a715 -- ceph osd dump --format=json 2024-03-28T18:03:46.424 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:03:46 smithi134 ceph-mon[35037]: pgmap v333: 1 pgs: 1 active+clean; 577 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-28T18:03:46.495 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/fd220748-ed2b-11ee-95d0-87774f69a715/mon.a/config 2024-03-28T18:03:46.531 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:03:46 smithi022 ceph-mon[31985]: pgmap v333: 1 pgs: 1 active+clean; 577 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-28T18:03:46.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:03:46 smithi079 ceph-mon[35312]: pgmap v333: 1 pgs: 1 active+clean; 577 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-28T18:03:47.691 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:03:47 smithi022 ceph-mon[31985]: pgmap v334: 1 pgs: 1 active+clean; 577 KiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-28T18:03:47.691 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:03:47 smithi022 ceph-mon[31985]: Detected new or changed devices on smithi134 2024-03-28T18:03:47.691 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:03:47 smithi022 ceph-mon[31985]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T18:03:47.691 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:03:47 smithi022 ceph-mon[31985]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T18:03:47.691 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:03:47 smithi022 ceph-mon[31985]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-28T18:03:47.692 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:03:47 smithi022 ceph-mon[31985]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-28T18:03:47.692 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:03:47 smithi022 ceph-mon[31985]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-03-28T18:03:47.692 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:03:47 smithi022 ceph-mon[31985]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-03-28T18:03:47.692 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:03:47 smithi022 ceph-mon[31985]: Adjusting osd_memory_target on smithi134 to 4272M 2024-03-28T18:03:47.692 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:03:47 smithi022 ceph-mon[31985]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T18:03:47.692 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:03:47 smithi022 ceph-mon[31985]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T18:03:47.692 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:03:47 smithi022 ceph-mon[31985]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T18:03:47.692 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:03:47 smithi022 ceph-mon[31985]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T18:03:47.925 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:03:47 smithi134 ceph-mon[35037]: pgmap v334: 1 pgs: 1 active+clean; 577 KiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-28T18:03:47.925 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:03:47 smithi134 ceph-mon[35037]: Detected new or changed devices on smithi134 2024-03-28T18:03:47.925 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:03:47 smithi134 ceph-mon[35037]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T18:03:47.925 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:03:47 smithi134 ceph-mon[35037]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T18:03:47.925 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:03:47 smithi134 ceph-mon[35037]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-28T18:03:47.925 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:03:47 smithi134 ceph-mon[35037]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-28T18:03:47.925 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:03:47 smithi134 ceph-mon[35037]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-03-28T18:03:47.925 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:03:47 smithi134 ceph-mon[35037]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-03-28T18:03:47.926 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:03:47 smithi134 ceph-mon[35037]: Adjusting osd_memory_target on smithi134 to 4272M 2024-03-28T18:03:47.926 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:03:47 smithi134 ceph-mon[35037]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T18:03:47.926 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:03:47 smithi134 ceph-mon[35037]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T18:03:47.926 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:03:47 smithi134 ceph-mon[35037]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T18:03:47.926 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:03:47 smithi134 ceph-mon[35037]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T18:03:47.933 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:03:47 smithi079 ceph-mon[35312]: pgmap v334: 1 pgs: 1 active+clean; 577 KiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-28T18:03:47.933 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:03:47 smithi079 ceph-mon[35312]: Detected new or changed devices on smithi134 2024-03-28T18:03:47.933 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:03:47 smithi079 ceph-mon[35312]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T18:03:47.933 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:03:47 smithi079 ceph-mon[35312]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T18:03:47.933 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:03:47 smithi079 ceph-mon[35312]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-28T18:03:47.933 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:03:47 smithi079 ceph-mon[35312]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-28T18:03:47.933 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:03:47 smithi079 ceph-mon[35312]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-03-28T18:03:47.933 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:03:47 smithi079 ceph-mon[35312]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-03-28T18:03:47.933 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:03:47 smithi079 ceph-mon[35312]: Adjusting osd_memory_target on smithi134 to 4272M 2024-03-28T18:03:47.933 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:03:47 smithi079 ceph-mon[35312]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T18:03:47.933 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:03:47 smithi079 ceph-mon[35312]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T18:03:47.933 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:03:47 smithi079 ceph-mon[35312]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T18:03:47.933 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:03:47 smithi079 ceph-mon[35312]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T18:03:48.736 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:03:48.735+0000 7fcee0800700 1 -- 172.21.15.22:0/3868085371 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fced806a930 msgr2=0x7fced806ad10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:03:48.736 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:03:48.735+0000 7fcee0800700 1 --2- 172.21.15.22:0/3868085371 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fced806a930 0x7fced806ad10 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fcecc0099c0 tx=0x7fcecc009cd0 comp rx=0 tx=0).stop 2024-03-28T18:03:48.736 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:03:48.735+0000 7fcee0800700 1 -- 172.21.15.22:0/3868085371 shutdown_connections 2024-03-28T18:03:48.736 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:03:48.735+0000 7fcee0800700 1 --2- 172.21.15.22:0/3868085371 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fced8107510 0x7fced810f090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:03:48.736 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:03:48.735+0000 7fcee0800700 1 --2- 172.21.15.22:0/3868085371 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fced806a930 0x7fced806ad10 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:03:48.737 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:03:48.735+0000 7fcee0800700 1 --2- 172.21.15.22:0/3868085371 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fced806b2e0 0x7fced8106de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:03:48.737 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:03:48.735+0000 7fcee0800700 1 -- 172.21.15.22:0/3868085371 >> 172.21.15.22:0/3868085371 conn(0x7fced80770e0 msgr2=0x7fced80774f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:03:48.737 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:03:48.735+0000 7fcee0800700 1 -- 172.21.15.22:0/3868085371 shutdown_connections 2024-03-28T18:03:48.737 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:03:48.735+0000 7fcee0800700 1 -- 172.21.15.22:0/3868085371 wait complete. 2024-03-28T18:03:48.737 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:03:48.736+0000 7fcee0800700 1 Processor -- start 2024-03-28T18:03:48.737 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:03:48.737+0000 7fcee0800700 1 -- start start 2024-03-28T18:03:48.739 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:03:48.737+0000 7fcee0800700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fced806a930 0x7fced819e650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:03:48.739 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:03:48.737+0000 7fcee0800700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fced806b2e0 0x7fced819eb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:03:48.739 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:03:48.737+0000 7fcee0800700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fced8107510 0x7fced81987d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:03:48.739 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:03:48.737+0000 7fcee0800700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fced81124f0 con 0x7fced8107510 2024-03-28T18:03:48.739 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:03:48.737+0000 7fcee0800700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fced8112370 con 0x7fced806a930 2024-03-28T18:03:48.739 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:03:48.737+0000 7fcee0800700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fced8112670 con 0x7fced806b2e0 2024-03-28T18:03:48.739 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:03:48.737+0000 7fcede800700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fced8107510 0x7fced81987d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:03:48.739 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:03:48.738+0000 7fcede800700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fced8107510 0x7fced81987d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:38084/0 (socket says 172.21.15.22:38084) 2024-03-28T18:03:48.739 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:03:48.738+0000 7fcede800700 1 -- 172.21.15.22:0/1070988111 learned_addr learned my addr 172.21.15.22:0/1070988111 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-28T18:03:48.739 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:03:48.738+0000 7fcedde00700 1 --2- 172.21.15.22:0/1070988111 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fced806a930 0x7fced819e650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:03:48.739 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:03:48.738+0000 7fcedd400700 1 --2- 172.21.15.22:0/1070988111 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fced806b2e0 0x7fced819eb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:03:48.739 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:03:48.738+0000 7fcede800700 1 -- 172.21.15.22:0/1070988111 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fced806b2e0 msgr2=0x7fced819eb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:03:48.739 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:03:48.738+0000 7fcede800700 1 --2- 172.21.15.22:0/1070988111 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fced806b2e0 0x7fced819eb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:03:48.740 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:03:48.738+0000 7fcede800700 1 -- 172.21.15.22:0/1070988111 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fced806a930 msgr2=0x7fced819e650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:03:48.740 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:03:48.738+0000 7fcede800700 1 --2- 172.21.15.22:0/1070988111 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fced806a930 0x7fced819e650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:03:48.740 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:03:48.738+0000 7fcede800700 1 -- 172.21.15.22:0/1070988111 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fced8198fb0 con 0x7fced8107510 2024-03-28T18:03:48.740 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:03:48.738+0000 7fcede800700 1 --2- 172.21.15.22:0/1070988111 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fced8107510 0x7fced81987d0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fced400ada0 tx=0x7fced400c330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:03:48.740 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:03:48.739+0000 7fced2a00700 1 -- 172.21.15.22:0/1070988111 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fced4017020 con 0x7fced8107510 2024-03-28T18:03:48.740 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:03:48.739+0000 7fced2a00700 1 -- 172.21.15.22:0/1070988111 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7fced400ce50 con 0x7fced8107510 2024-03-28T18:03:48.740 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:03:48.739+0000 7fced2a00700 1 -- 172.21.15.22:0/1070988111 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fced4005380 con 0x7fced8107510 2024-03-28T18:03:48.740 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:03:48.739+0000 7fcee0800700 1 -- 172.21.15.22:0/1070988111 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fced81992a0 con 0x7fced8107510 2024-03-28T18:03:48.740 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:03:48.739+0000 7fcee0800700 1 -- 172.21.15.22:0/1070988111 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fced81010c0 con 0x7fced8107510 2024-03-28T18:03:48.740 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:03:48.740+0000 7fcee0800700 1 -- 172.21.15.22:0/1070988111 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fced810a6e0 con 0x7fced8107510 2024-03-28T18:03:48.742 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:03:48.742+0000 7fced2a00700 1 -- 172.21.15.22:0/1070988111 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 16) v1 ==== 137290+0+0 (secure 0 0 0) 0x7fced400c980 con 0x7fced8107510 2024-03-28T18:03:48.743 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:03:48.743+0000 7fced2a00700 1 --2- 172.21.15.22:0/1070988111 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7fcec00a2910 0x7fcec00a4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:03:48.743 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:03:48.743+0000 7fced2a00700 1 -- 172.21.15.22:0/1070988111 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 7127+0+0 (secure 0 0 0) 0x7fced40cdd40 con 0x7fced8107510 2024-03-28T18:03:48.743 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:03:48.744+0000 7fcedde00700 1 --2- 172.21.15.22:0/1070988111 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7fcec00a2910 0x7fcec00a4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:03:48.744 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:03:48.745+0000 7fcedde00700 1 --2- 172.21.15.22:0/1070988111 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7fcec00a2910 0x7fcec00a4dd0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fcecc009990 tx=0x7fcecc000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:03:48.748 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:03:48.749+0000 7fced2a00700 1 -- 172.21.15.22:0/1070988111 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fced4021020 con 0x7fced8107510 2024-03-28T18:03:48.976 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:03:48.976+0000 7fcee0800700 1 -- 172.21.15.22:0/1070988111 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fced8199d00 con 0x7fced8107510 2024-03-28T18:03:48.979 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:03:48.979+0000 7fced2a00700 1 -- 172.21.15.22:0/1070988111 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v86) v1 ==== 74+0+18420 (secure 0 0 0) 0x7fced400cc70 con 0x7fced8107510 2024-03-28T18:03:48.980 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-03-28T18:03:48.980 INFO:teuthology.orchestra.run.smithi022.stdout:{"epoch":86,"fsid":"fd220748-ed2b-11ee-95d0-87774f69a715","created":"2024-03-28T17:54:00.440279+0000","modified":"2024-03-28T18:03:44.232788+0000","last_up_change":"2024-03-28T18:03:43.220095+0000","last_in_change":"2024-03-28T18:03:18.526399+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-28T17:58:33.168872+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"5d708e06-f05b-40ba-bbcf-258ddc4f43d7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6802","nonce":2186290846},{"type":"v1","addr":"172.21.15.22:6803","nonce":2186290846}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6804","nonce":2186290846},{"type":"v1","addr":"172.21.15.22:6805","nonce":2186290846}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6808","nonce":2186290846},{"type":"v1","addr":"172.21.15.22:6809","nonce":2186290846}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6806","nonce":2186290846},{"type":"v1","addr":"172.21.15.22:6807","nonce":2186290846}]},"public_addr":"172.21.15.22:6803/2186290846","cluster_addr":"172.21.15.22:6805/2186290846","heartbeat_back_addr":"172.21.15.22:6809/2186290846","heartbeat_front_addr":"172.21.15.22:6807/2186290846","state":["exists","up"]},{"osd":1,"uuid":"68ad09bb-9256-409a-9867-4e14a2cb94aa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6800","nonce":2835549681},{"type":"v1","addr":"172.21.15.79:6801","nonce":2835549681}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6802","nonce":2835549681},{"type":"v1","addr":"172.21.15.79:6803","nonce":2835549681}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6806","nonce":2835549681},{"type":"v1","addr":"172.21.15.79:6807","nonce":2835549681}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6804","nonce":2835549681},{"type":"v1","addr":"172.21.15.79:6805","nonce":2835549681}]},"public_addr":"172.21.15.79:6801/2835549681","cluster_addr":"172.21.15.79:6803/2835549681","heartbeat_back_addr":"172.21.15.79:6807/2835549681","heartbeat_front_addr":"172.21.15.79:6805/2835549681","state":["exists","up"]},{"osd":2,"uuid":"ed11a5a7-004d-41f3-a5fc-eb2d603bd33f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6800","nonce":470588085},{"type":"v1","addr":"172.21.15.134:6801","nonce":470588085}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6802","nonce":470588085},{"type":"v1","addr":"172.21.15.134:6803","nonce":470588085}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6806","nonce":470588085},{"type":"v1","addr":"172.21.15.134:6807","nonce":470588085}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6804","nonce":470588085},{"type":"v1","addr":"172.21.15.134:6805","nonce":470588085}]},"public_addr":"172.21.15.134:6801/470588085","cluster_addr":"172.21.15.134:6803/470588085","heartbeat_back_addr":"172.21.15.134:6807/470588085","heartbeat_front_addr":"172.21.15.134:6805/470588085","state":["exists","up"]},{"osd":3,"uuid":"81a6c547-2a50-44c3-84d7-23ab6831790c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6810","nonce":1702441854},{"type":"v1","addr":"172.21.15.22:6811","nonce":1702441854}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6812","nonce":1702441854},{"type":"v1","addr":"172.21.15.22:6813","nonce":1702441854}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6816","nonce":1702441854},{"type":"v1","addr":"172.21.15.22:6817","nonce":1702441854}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6814","nonce":1702441854},{"type":"v1","addr":"172.21.15.22:6815","nonce":1702441854}]},"public_addr":"172.21.15.22:6811/1702441854","cluster_addr":"172.21.15.22:6813/1702441854","heartbeat_back_addr":"172.21.15.22:6817/1702441854","heartbeat_front_addr":"172.21.15.22:6815/1702441854","state":["exists","up"]},{"osd":4,"uuid":"d23a254e-7182-4471-8bca-7eb684ec618e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6808","nonce":630132437},{"type":"v1","addr":"172.21.15.79:6809","nonce":630132437}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6810","nonce":630132437},{"type":"v1","addr":"172.21.15.79:6811","nonce":630132437}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6814","nonce":630132437},{"type":"v1","addr":"172.21.15.79:6815","nonce":630132437}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6812","nonce":630132437},{"type":"v1","addr":"172.21.15.79:6813","nonce":630132437}]},"public_addr":"172.21.15.79:6809/630132437","cluster_addr":"172.21.15.79:6811/630132437","heartbeat_back_addr":"172.21.15.79:6815/630132437","heartbeat_front_addr":"172.21.15.79:6813/630132437","state":["exists","up"]},{"osd":5,"uuid":"a7b98502-2023-4f0a-bfa0-6840cc495e0c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":40,"up_thru":55,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6808","nonce":2135406915},{"type":"v1","addr":"172.21.15.134:6809","nonce":2135406915}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6810","nonce":2135406915},{"type":"v1","addr":"172.21.15.134:6811","nonce":2135406915}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6814","nonce":2135406915},{"type":"v1","addr":"172.21.15.134:6815","nonce":2135406915}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6812","nonce":2135406915},{"type":"v1","addr":"172.21.15.134:6813","nonce":2135406915}]},"public_addr":"172.21.15.134:6809/2135406915","cluster_addr":"172.21.15.134:6811/2135406915","heartbeat_back_addr":"172.21.15.134:6815/2135406915","heartbeat_front_addr":"172.21.15.134:6813/2135406915","state":["exists","up"]},{"osd":6,"uuid":"634c6562-d683-41c8-8242-fc848c38ce1f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6818","nonce":1400680396},{"type":"v1","addr":"172.21.15.22:6819","nonce":1400680396}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6820","nonce":1400680396},{"type":"v1","addr":"172.21.15.22:6821","nonce":1400680396}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6824","nonce":1400680396},{"type":"v1","addr":"172.21.15.22:6825","nonce":1400680396}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6822","nonce":1400680396},{"type":"v1","addr":"172.21.15.22:6823","nonce":1400680396}]},"public_addr":"172.21.15.22:6819/1400680396","cluster_addr":"172.21.15.22:6821/1400680396","heartbeat_back_addr":"172.21.15.22:6825/1400680396","heartbeat_front_addr":"172.21.15.22:6823/1400680396","state":["exists","up"]},{"osd":7,"uuid":"504eb4a8-6731-45ad-932c-1b38ef157871","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":53,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6816","nonce":1287105314},{"type":"v1","addr":"172.21.15.79:6817","nonce":1287105314}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6818","nonce":1287105314},{"type":"v1","addr":"172.21.15.79:6819","nonce":1287105314}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6822","nonce":1287105314},{"type":"v1","addr":"172.21.15.79:6823","nonce":1287105314}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6820","nonce":1287105314},{"type":"v1","addr":"172.21.15.79:6821","nonce":1287105314}]},"public_addr":"172.21.15.79:6817/1287105314","cluster_addr":"172.21.15.79:6819/1287105314","heartbeat_back_addr":"172.21.15.79:6823/1287105314","heartbeat_front_addr":"172.21.15.79:6821/1287105314","state":["exists","up"]},{"osd":8,"uuid":"20aa66c1-03d6-4cb8-a096-0eae41979c17","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":62,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6816","nonce":2265650388},{"type":"v1","addr":"172.21.15.134:6817","nonce":2265650388}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6818","nonce":2265650388},{"type":"v1","addr":"172.21.15.134:6819","nonce":2265650388}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6822","nonce":2265650388},{"type":"v1","addr":"172.21.15.134:6823","nonce":2265650388}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6820","nonce":2265650388},{"type":"v1","addr":"172.21.15.134:6821","nonce":2265650388}]},"public_addr":"172.21.15.134:6817/2265650388","cluster_addr":"172.21.15.134:6819/2265650388","heartbeat_back_addr":"172.21.15.134:6823/2265650388","heartbeat_front_addr":"172.21.15.134:6821/2265650388","state":["exists","up"]},{"osd":9,"uuid":"e1ae4020-7c2c-4334-a753-ee6b12d98584","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":70,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6826","nonce":3762222259},{"type":"v1","addr":"172.21.15.22:6827","nonce":3762222259}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6828","nonce":3762222259},{"type":"v1","addr":"172.21.15.22:6829","nonce":3762222259}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6832","nonce":3762222259},{"type":"v1","addr":"172.21.15.22:6833","nonce":3762222259}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6830","nonce":3762222259},{"type":"v1","addr":"172.21.15.22:6831","nonce":3762222259}]},"public_addr":"172.21.15.22:6827/3762222259","cluster_addr":"172.21.15.22:6829/3762222259","heartbeat_back_addr":"172.21.15.22:6833/3762222259","heartbeat_front_addr":"172.21.15.22:6831/3762222259","state":["exists","up"]},{"osd":10,"uuid":"f5198d52-df48-4679-a860-4dcb440b7a9f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":77,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6824","nonce":1746167904},{"type":"v1","addr":"172.21.15.79:6825","nonce":1746167904}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6826","nonce":1746167904},{"type":"v1","addr":"172.21.15.79:6827","nonce":1746167904}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6830","nonce":1746167904},{"type":"v1","addr":"172.21.15.79:6831","nonce":1746167904}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6828","nonce":1746167904},{"type":"v1","addr":"172.21.15.79:6829","nonce":1746167904}]},"public_addr":"172.21.15.79:6825/1746167904","cluster_addr":"172.21.15.79:6827/1746167904","heartbeat_back_addr":"172.21.15.79:6831/1746167904","heartbeat_front_addr":"172.21.15.79:6829/1746167904","state":["exists","up"]},{"osd":11,"uuid":"1427bca7-3414-4468-9246-e999156bc2ae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":85,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6824","nonce":4271716428},{"type":"v1","addr":"172.21.15.134:6825","nonce":4271716428}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6826","nonce":4271716428},{"type":"v1","addr":"172.21.15.134:6827","nonce":4271716428}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6830","nonce":4271716428},{"type":"v1","addr":"172.21.15.134:6831","nonce":4271716428}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6828","nonce":4271716428},{"type":"v1","addr":"172.21.15.134:6829","nonce":4271716428}]},"public_addr":"172.21.15.134:6825/4271716428","cluster_addr":"172.21.15.134:6827/4271716428","heartbeat_back_addr":"172.21.15.134:6831/4271716428","heartbeat_front_addr":"172.21.15.134:6829/4271716428","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T17:57:21.225125+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T17:57:55.197254+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T17:58:27.825136+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T17:59:04.319893+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T17:59:38.407752+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T18:00:12.118805+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T18:00:47.635214+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T18:01:21.231055+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T18:01:55.817861+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T18:02:31.680160+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T18:03:06.783398+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T18:03:41.307347+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.22:0/3521924283":"2024-03-29T17:55:10.425504+0000","172.21.15.22:6801/730635939":"2024-03-29T17:55:10.425504+0000","172.21.15.22:0/2471229459":"2024-03-29T17:54:34.004559+0000","172.21.15.22:6800/3166828081":"2024-03-29T17:54:34.004559+0000","172.21.15.22:6800/730635939":"2024-03-29T17:55:10.425504+0000","172.21.15.22:0/1229047849":"2024-03-29T17:54:34.004559+0000","172.21.15.22:6801/3166828081":"2024-03-29T17:54:34.004559+0000","172.21.15.22:0/2717818856":"2024-03-29T17:55:10.425504+0000","172.21.15.22:0/2157570333":"2024-03-29T17:54:34.004559+0000","172.21.15.22:0/1006589530":"2024-03-29T17:55:10.425504+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-28T18:03:48.983 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:03:48.982+0000 7fcee0800700 1 -- 172.21.15.22:0/1070988111 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7fcec00a2910 msgr2=0x7fcec00a4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:03:48.983 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:03:48.982+0000 7fcee0800700 1 --2- 172.21.15.22:0/1070988111 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7fcec00a2910 0x7fcec00a4dd0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fcecc009990 tx=0x7fcecc000bc0 comp rx=0 tx=0).stop 2024-03-28T18:03:48.983 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:03:48.982+0000 7fcee0800700 1 -- 172.21.15.22:0/1070988111 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fced8107510 msgr2=0x7fced81987d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:03:48.983 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:03:48.982+0000 7fcee0800700 1 --2- 172.21.15.22:0/1070988111 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fced8107510 0x7fced81987d0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fced400ada0 tx=0x7fced400c330 comp rx=0 tx=0).stop 2024-03-28T18:03:48.983 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:03:48.983+0000 7fcee0800700 1 -- 172.21.15.22:0/1070988111 shutdown_connections 2024-03-28T18:03:48.983 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:03:48.983+0000 7fcee0800700 1 --2- 172.21.15.22:0/1070988111 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fced8107510 0x7fced81987d0 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:03:48.983 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:03:48.983+0000 7fcee0800700 1 --2- 172.21.15.22:0/1070988111 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fced806a930 0x7fced819e650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:03:48.983 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:03:48.983+0000 7fcee0800700 1 --2- 172.21.15.22:0/1070988111 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7fcec00a2910 0x7fcec00a4dd0 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:03:48.983 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:03:48.983+0000 7fcee0800700 1 --2- 172.21.15.22:0/1070988111 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fced806b2e0 0x7fced819eb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:03:48.983 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:03:48.983+0000 7fcee0800700 1 -- 172.21.15.22:0/1070988111 >> 172.21.15.22:0/1070988111 conn(0x7fced80770e0 msgr2=0x7fced80ff3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:03:48.983 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:03:48.983+0000 7fcee0800700 1 -- 172.21.15.22:0/1070988111 shutdown_connections 2024-03-28T18:03:48.984 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:03:48.983+0000 7fcee0800700 1 -- 172.21.15.22:0/1070988111 wait complete. 2024-03-28T18:03:49.882 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:03:49 smithi022 ceph-mon[31985]: pgmap v335: 1 pgs: 1 active+clean; 577 KiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-28T18:03:49.882 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:03:49 smithi022 ceph-mon[31985]: from='client.? 172.21.15.22:0/1070988111' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-28T18:03:49.887 INFO:tasks.cephadm.ceph_manager.ceph:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-28T17:58:33.168872+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-28T18:03:49.888 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033 shell --fsid fd220748-ed2b-11ee-95d0-87774f69a715 -- ceph osd pool get .mgr pg_num 2024-03-28T18:03:49.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:03:49 smithi134 ceph-mon[35037]: pgmap v335: 1 pgs: 1 active+clean; 577 KiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-28T18:03:49.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:03:49 smithi134 ceph-mon[35037]: from='client.? 172.21.15.22:0/1070988111' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-28T18:03:49.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:03:49 smithi079 ceph-mon[35312]: pgmap v335: 1 pgs: 1 active+clean; 577 KiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-28T18:03:49.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:03:49 smithi079 ceph-mon[35312]: from='client.? 172.21.15.22:0/1070988111' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-28T18:03:50.350 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/fd220748-ed2b-11ee-95d0-87774f69a715/mon.a/config 2024-03-28T18:03:51.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:03:51 smithi134 ceph-mon[35037]: pgmap v336: 1 pgs: 1 active+clean; 577 KiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-28T18:03:51.933 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:03:51 smithi079 ceph-mon[35312]: pgmap v336: 1 pgs: 1 active+clean; 577 KiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-28T18:03:51.997 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:03:51 smithi022 ceph-mon[31985]: pgmap v336: 1 pgs: 1 active+clean; 577 KiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-28T18:03:52.518 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:03:52.517+0000 7f8a3ba00700 1 -- 172.21.15.22:0/3323486441 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8a34102ba0 msgr2=0x7f8a3410efe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:03:52.518 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:03:52.517+0000 7f8a3ba00700 1 --2- 172.21.15.22:0/3323486441 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8a34102ba0 0x7f8a3410efe0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f8a2c00b0d0 tx=0x7f8a2c00b3e0 comp rx=0 tx=0).stop 2024-03-28T18:03:52.518 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:03:52.517+0000 7f8a3ba00700 1 -- 172.21.15.22:0/3323486441 shutdown_connections 2024-03-28T18:03:52.518 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:03:52.517+0000 7f8a3ba00700 1 --2- 172.21.15.22:0/3323486441 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8a34102ba0 0x7f8a3410efe0 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:03:52.518 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:03:52.518+0000 7f8a3ba00700 1 --2- 172.21.15.22:0/3323486441 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f8a341081c0 0x7f8a341085a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:03:52.518 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:03:52.518+0000 7f8a3ba00700 1 --2- 172.21.15.22:0/3323486441 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8a341021e0 0x7f8a34102660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:03:52.518 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:03:52.518+0000 7f8a3ba00700 1 -- 172.21.15.22:0/3323486441 >> 172.21.15.22:0/3323486441 conn(0x7f8a340792c0 msgr2=0x7f8a340796d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:03:52.518 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:03:52.518+0000 7f8a3ba00700 1 -- 172.21.15.22:0/3323486441 shutdown_connections 2024-03-28T18:03:52.519 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:03:52.518+0000 7f8a3ba00700 1 -- 172.21.15.22:0/3323486441 wait complete. 2024-03-28T18:03:52.520 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:03:52.519+0000 7f8a3ba00700 1 Processor -- start 2024-03-28T18:03:52.520 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:03:52.519+0000 7f8a3ba00700 1 -- start start 2024-03-28T18:03:52.520 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:03:52.519+0000 7f8a3ba00700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8a341021e0 0x7f8a341a0790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:03:52.520 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:03:52.519+0000 7f8a3ba00700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f8a34102ba0 0x7f8a341a0cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:03:52.520 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:03:52.519+0000 7f8a3ba00700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8a341081c0 0x7f8a3419a950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:03:52.520 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:03:52.519+0000 7f8a3ba00700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a34112460 con 0x7f8a341081c0 2024-03-28T18:03:52.520 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:03:52.519+0000 7f8a3ba00700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a341122e0 con 0x7f8a34102ba0 2024-03-28T18:03:52.521 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:03:52.519+0000 7f8a3ba00700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a341125e0 con 0x7f8a341021e0 2024-03-28T18:03:52.521 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:03:52.520+0000 7f8a39a00700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8a341081c0 0x7f8a3419a950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:03:52.521 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:03:52.520+0000 7f8a39a00700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8a341081c0 0x7f8a3419a950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:38116/0 (socket says 172.21.15.22:38116) 2024-03-28T18:03:52.521 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:03:52.520+0000 7f8a39a00700 1 -- 172.21.15.22:0/3712031691 learned_addr learned my addr 172.21.15.22:0/3712031691 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-28T18:03:52.521 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:03:52.520+0000 7f8a39000700 1 --2- 172.21.15.22:0/3712031691 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8a341021e0 0x7f8a341a0790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:03:52.521 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:03:52.520+0000 7f8a2be00700 1 --2- 172.21.15.22:0/3712031691 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f8a34102ba0 0x7f8a341a0cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:03:52.521 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:03:52.520+0000 7f8a39a00700 1 -- 172.21.15.22:0/3712031691 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8a341021e0 msgr2=0x7f8a341a0790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:03:52.521 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:03:52.520+0000 7f8a39a00700 1 --2- 172.21.15.22:0/3712031691 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8a341021e0 0x7f8a341a0790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:03:52.521 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:03:52.520+0000 7f8a39a00700 1 -- 172.21.15.22:0/3712031691 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f8a34102ba0 msgr2=0x7f8a341a0cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:03:52.521 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:03:52.520+0000 7f8a39a00700 1 --2- 172.21.15.22:0/3712031691 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f8a34102ba0 0x7f8a341a0cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:03:52.521 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:03:52.520+0000 7f8a39a00700 1 -- 172.21.15.22:0/3712031691 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8a3419b0b0 con 0x7f8a341081c0 2024-03-28T18:03:52.521 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:03:52.520+0000 7f8a2be00700 1 --2- 172.21.15.22:0/3712031691 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f8a34102ba0 0x7f8a341a0cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-28T18:03:52.522 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:03:52.521+0000 7f8a39a00700 1 --2- 172.21.15.22:0/3712031691 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8a341081c0 0x7f8a3419a950 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f8a2c007a00 tx=0x7f8a2c007a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:03:52.522 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:03:52.521+0000 7f8a32000700 1 -- 172.21.15.22:0/3712031691 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a2c01f070 con 0x7f8a341081c0 2024-03-28T18:03:52.522 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:03:52.521+0000 7f8a32000700 1 -- 172.21.15.22:0/3712031691 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f8a2c003c00 con 0x7f8a341081c0 2024-03-28T18:03:52.522 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:03:52.521+0000 7f8a32000700 1 -- 172.21.15.22:0/3712031691 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a2c01b6a0 con 0x7f8a341081c0 2024-03-28T18:03:52.522 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:03:52.522+0000 7f8a3ba00700 1 -- 172.21.15.22:0/3712031691 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8a3419b340 con 0x7f8a341081c0 2024-03-28T18:03:52.522 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:03:52.522+0000 7f8a3ba00700 1 -- 172.21.15.22:0/3712031691 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8a341a7660 con 0x7f8a341081c0 2024-03-28T18:03:52.523 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:03:52.523+0000 7f8a3ba00700 1 -- 172.21.15.22:0/3712031691 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8a34068e50 con 0x7f8a341081c0 2024-03-28T18:03:52.524 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:03:52.525+0000 7f8a32000700 1 -- 172.21.15.22:0/3712031691 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 16) v1 ==== 137290+0+0 (secure 0 0 0) 0x7f8a2c022070 con 0x7f8a341081c0 2024-03-28T18:03:52.526 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:03:52.526+0000 7f8a32000700 1 --2- 172.21.15.22:0/3712031691 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7f8a140a2910 0x7f8a140a4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:03:52.526 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:03:52.526+0000 7f8a32000700 1 -- 172.21.15.22:0/3712031691 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f8a2c00f020 con 0x7f8a341081c0 2024-03-28T18:03:52.526 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:03:52.527+0000 7f8a39000700 1 --2- 172.21.15.22:0/3712031691 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7f8a140a2910 0x7f8a140a4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:03:52.527 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:03:52.527+0000 7f8a39000700 1 --2- 172.21.15.22:0/3712031691 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7f8a140a2910 0x7f8a140a4dd0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f8a3419ba90 tx=0x7f8a1c00b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:03:52.531 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:03:52.531+0000 7f8a32000700 1 -- 172.21.15.22:0/3712031691 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f8a2c029020 con 0x7f8a341081c0 2024-03-28T18:03:52.756 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:03:52.756+0000 7f8a3ba00700 1 -- 172.21.15.22:0/3712031691 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f8a341a7830 con 0x7f8a341081c0 2024-03-28T18:03:52.757 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:03:52.757+0000 7f8a32000700 1 -- 172.21.15.22:0/3712031691 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v86) v1 ==== 93+0+10 (secure 0 0 0) 0x7f8a2c09ccd0 con 0x7f8a341081c0 2024-03-28T18:03:52.757 INFO:teuthology.orchestra.run.smithi022.stdout:pg_num: 1 2024-03-28T18:03:52.761 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:03:52.761+0000 7f8a3ba00700 1 -- 172.21.15.22:0/3712031691 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7f8a140a2910 msgr2=0x7f8a140a4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:03:52.761 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:03:52.761+0000 7f8a3ba00700 1 --2- 172.21.15.22:0/3712031691 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7f8a140a2910 0x7f8a140a4dd0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f8a3419ba90 tx=0x7f8a1c00b410 comp rx=0 tx=0).stop 2024-03-28T18:03:52.761 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:03:52.761+0000 7f8a3ba00700 1 -- 172.21.15.22:0/3712031691 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8a341081c0 msgr2=0x7f8a3419a950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:03:52.761 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:03:52.761+0000 7f8a3ba00700 1 --2- 172.21.15.22:0/3712031691 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8a341081c0 0x7f8a3419a950 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f8a2c007a00 tx=0x7f8a2c007a30 comp rx=0 tx=0).stop 2024-03-28T18:03:52.761 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:03:52.761+0000 7f8a3ba00700 1 -- 172.21.15.22:0/3712031691 shutdown_connections 2024-03-28T18:03:52.761 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:03:52.761+0000 7f8a3ba00700 1 --2- 172.21.15.22:0/3712031691 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8a341081c0 0x7f8a3419a950 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:03:52.762 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:03:52.761+0000 7f8a3ba00700 1 --2- 172.21.15.22:0/3712031691 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7f8a140a2910 0x7f8a140a4dd0 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:03:52.762 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:03:52.761+0000 7f8a3ba00700 1 --2- 172.21.15.22:0/3712031691 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8a341021e0 0x7f8a341a0790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:03:52.762 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:03:52.761+0000 7f8a3ba00700 1 --2- 172.21.15.22:0/3712031691 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f8a34102ba0 0x7f8a341a0cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:03:52.762 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:03:52.761+0000 7f8a3ba00700 1 -- 172.21.15.22:0/3712031691 >> 172.21.15.22:0/3712031691 conn(0x7f8a340792c0 msgr2=0x7f8a34100bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:03:52.762 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:03:52.761+0000 7f8a3ba00700 1 -- 172.21.15.22:0/3712031691 shutdown_connections 2024-03-28T18:03:52.762 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:03:52.762+0000 7f8a3ba00700 1 -- 172.21.15.22:0/3712031691 wait complete. 2024-03-28T18:03:53.637 INFO:tasks.cephadm:Adding mds.a on smithi022 2024-03-28T18:03:53.637 INFO:tasks.cephadm:Adding mds.d on smithi022 2024-03-28T18:03:53.637 INFO:tasks.cephadm:Adding mds.g on smithi022 2024-03-28T18:03:53.637 INFO:tasks.cephadm:Adding mds.j on smithi022 2024-03-28T18:03:53.637 INFO:tasks.cephadm:Adding mds.b on smithi079 2024-03-28T18:03:53.637 INFO:tasks.cephadm:Adding mds.e on smithi079 2024-03-28T18:03:53.637 INFO:tasks.cephadm:Adding mds.h on smithi079 2024-03-28T18:03:53.637 INFO:tasks.cephadm:Adding mds.k on smithi079 2024-03-28T18:03:53.637 INFO:tasks.cephadm:Adding mds.c on smithi134 2024-03-28T18:03:53.637 INFO:tasks.cephadm:Adding mds.f on smithi134 2024-03-28T18:03:53.638 INFO:tasks.cephadm:Adding mds.i on smithi134 2024-03-28T18:03:53.638 INFO:tasks.cephadm:Adding mds.l on smithi134 2024-03-28T18:03:53.638 DEBUG:teuthology.orchestra.run.smithi134:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fd220748-ed2b-11ee-95d0-87774f69a715 -- ceph orch apply mds all '12;smithi022=a;smithi022=d;smithi022=g;smithi022=j;smithi079=b;smithi079=e;smithi079=h;smithi079=k;smithi134=c;smithi134=f;smithi134=i;smithi134=l' 2024-03-28T18:03:53.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:03:53 smithi134 ceph-mon[35037]: pgmap v337: 1 pgs: 1 active+clean; 577 KiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-28T18:03:53.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:03:53 smithi134 ceph-mon[35037]: from='client.? 172.21.15.22:0/3712031691' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-28T18:03:53.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:03:53 smithi079 ceph-mon[35312]: pgmap v337: 1 pgs: 1 active+clean; 577 KiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-28T18:03:53.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:03:53 smithi079 ceph-mon[35312]: from='client.? 172.21.15.22:0/3712031691' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-28T18:03:54.032 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:03:53 smithi022 ceph-mon[31985]: pgmap v337: 1 pgs: 1 active+clean; 577 KiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-28T18:03:54.032 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:03:53 smithi022 ceph-mon[31985]: from='client.? 172.21.15.22:0/3712031691' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-28T18:03:54.083 INFO:teuthology.orchestra.run.smithi134.stderr:Inferring config /var/lib/ceph/fd220748-ed2b-11ee-95d0-87774f69a715/mon.c/config 2024-03-28T18:03:55.925 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:03:55 smithi134 ceph-mon[35037]: pgmap v338: 1 pgs: 1 active+clean; 577 KiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-28T18:03:55.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:03:55 smithi079 ceph-mon[35312]: pgmap v338: 1 pgs: 1 active+clean; 577 KiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-28T18:03:56.031 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:03:55 smithi022 ceph-mon[31985]: pgmap v338: 1 pgs: 1 active+clean; 577 KiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-28T18:03:56.179 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-28T18:03:56.178+0000 7f15b0600700 1 -- 172.21.15.134:0/3487197909 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f15a8103d60 msgr2=0x7f15a810a610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:03:56.179 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-28T18:03:56.178+0000 7f15b0600700 1 --2- 172.21.15.134:0/3487197909 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f15a8103d60 0x7f15a810a610 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f15a000b0a0 tx=0x7f15a000b3b0 comp rx=0 tx=0).stop 2024-03-28T18:03:56.179 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-28T18:03:56.179+0000 7f15b0600700 1 -- 172.21.15.134:0/3487197909 shutdown_connections 2024-03-28T18:03:56.179 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-28T18:03:56.179+0000 7f15b0600700 1 --2- 172.21.15.134:0/3487197909 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f15a8103d60 0x7f15a810a610 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:03:56.179 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-28T18:03:56.179+0000 7f15b0600700 1 --2- 172.21.15.134:0/3487197909 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f15a81021c0 0x7f15a81025e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:03:56.179 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-28T18:03:56.179+0000 7f15b0600700 1 --2- 172.21.15.134:0/3487197909 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f15a81033c0 0x7f15a8103820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:03:56.179 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-28T18:03:56.179+0000 7f15b0600700 1 -- 172.21.15.134:0/3487197909 >> 172.21.15.134:0/3487197909 conn(0x7f15a80fd7a0 msgr2=0x7f15a80ffbc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:03:56.180 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-28T18:03:56.179+0000 7f15b0600700 1 -- 172.21.15.134:0/3487197909 shutdown_connections 2024-03-28T18:03:56.180 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-28T18:03:56.179+0000 7f15b0600700 1 -- 172.21.15.134:0/3487197909 wait complete. 2024-03-28T18:03:56.182 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-28T18:03:56.180+0000 7f15b0600700 1 Processor -- start 2024-03-28T18:03:56.182 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-28T18:03:56.180+0000 7f15b0600700 1 -- start start 2024-03-28T18:03:56.182 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-28T18:03:56.180+0000 7f15b0600700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f15a81021c0 0x7f15a8075f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:03:56.182 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-28T18:03:56.180+0000 7f15b0600700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f15a81033c0 0x7f15a8074580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:03:56.182 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-28T18:03:56.180+0000 7f15b0600700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f15a8103d60 0x7f15a8074ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:03:56.182 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-28T18:03:56.180+0000 7f15b0600700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15a810de00 con 0x7f15a8103d60 2024-03-28T18:03:56.182 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-28T18:03:56.180+0000 7f15b0600700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15a810dc80 con 0x7f15a81021c0 2024-03-28T18:03:56.182 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-28T18:03:56.180+0000 7f15b0600700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15a810df80 con 0x7f15a81033c0 2024-03-28T18:03:56.182 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-28T18:03:56.181+0000 7f15ad200700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f15a81033c0 0x7f15a8074580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:03:56.182 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-28T18:03:56.181+0000 7f15ad200700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f15a81033c0 0x7f15a8074580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.134:53010/0 (socket says 172.21.15.134:53010) 2024-03-28T18:03:56.183 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-28T18:03:56.181+0000 7f15ad200700 1 -- 172.21.15.134:0/68156717 learned_addr learned my addr 172.21.15.134:0/68156717 (peer_addr_for_me v2:172.21.15.134:0/0) 2024-03-28T18:03:56.183 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-28T18:03:56.181+0000 7f15adc00700 1 --2- 172.21.15.134:0/68156717 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f15a81021c0 0x7f15a8075f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:03:56.183 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-28T18:03:56.181+0000 7f15ae600700 1 --2- 172.21.15.134:0/68156717 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f15a8103d60 0x7f15a8074ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:03:56.183 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-28T18:03:56.181+0000 7f15ad200700 1 -- 172.21.15.134:0/68156717 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f15a81021c0 msgr2=0x7f15a8075f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:03:56.183 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-28T18:03:56.181+0000 7f15ad200700 1 --2- 172.21.15.134:0/68156717 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f15a81021c0 0x7f15a8075f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:03:56.183 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-28T18:03:56.181+0000 7f15ad200700 1 -- 172.21.15.134:0/68156717 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f15a8103d60 msgr2=0x7f15a8074ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:03:56.183 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-28T18:03:56.181+0000 7f15ad200700 1 --2- 172.21.15.134:0/68156717 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f15a8103d60 0x7f15a8074ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:03:56.183 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-28T18:03:56.181+0000 7f15ad200700 1 -- 172.21.15.134:0/68156717 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f15a8075000 con 0x7f15a81033c0 2024-03-28T18:03:56.183 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-28T18:03:56.181+0000 7f15ae600700 1 --2- 172.21.15.134:0/68156717 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f15a8103d60 0x7f15a8074ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T18:03:56.183 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-28T18:03:56.181+0000 7f15ad200700 1 --2- 172.21.15.134:0/68156717 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f15a81033c0 0x7f15a8074580 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f159400d980 tx=0x7f159400dd40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:03:56.183 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-28T18:03:56.182+0000 7f15a6000700 1 -- 172.21.15.134:0/68156717 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1594017070 con 0x7f15a81033c0 2024-03-28T18:03:56.183 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-28T18:03:56.182+0000 7f15a6000700 1 -- 172.21.15.134:0/68156717 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f1594004d00 con 0x7f15a81033c0 2024-03-28T18:03:56.183 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-28T18:03:56.182+0000 7f15a6000700 1 -- 172.21.15.134:0/68156717 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1594005520 con 0x7f15a81033c0 2024-03-28T18:03:56.183 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-28T18:03:56.182+0000 7f15b0600700 1 -- 172.21.15.134:0/68156717 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f15a80752f0 con 0x7f15a81033c0 2024-03-28T18:03:56.183 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-28T18:03:56.182+0000 7f15b0600700 1 -- 172.21.15.134:0/68156717 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f15a81a6e30 con 0x7f15a81033c0 2024-03-28T18:03:56.184 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-28T18:03:56.183+0000 7f15b0600700 1 -- 172.21.15.134:0/68156717 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f15a8068e50 con 0x7f15a81033c0 2024-03-28T18:03:56.185 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-28T18:03:56.185+0000 7f15a6000700 1 -- 172.21.15.134:0/68156717 <== mon.1 v2:172.21.15.134:3300/0 4 ==== mgrmap(e 16) v1 ==== 137290+0+0 (secure 0 0 0) 0x7f1594004750 con 0x7f15a81033c0 2024-03-28T18:03:56.186 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-28T18:03:56.186+0000 7f15a6000700 1 --2- 172.21.15.134:0/68156717 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7f15900a2990 0x7f15900a4e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:03:56.186 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-28T18:03:56.187+0000 7f15a6000700 1 -- 172.21.15.134:0/68156717 <== mon.1 v2:172.21.15.134:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f15940ce520 con 0x7f15a81033c0 2024-03-28T18:03:56.187 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-28T18:03:56.187+0000 7f15adc00700 1 --2- 172.21.15.134:0/68156717 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7f15900a2990 0x7f15900a4e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:03:56.188 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-28T18:03:56.188+0000 7f15adc00700 1 --2- 172.21.15.134:0/68156717 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7f15900a2990 0x7f15900a4e50 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f159c005950 tx=0x7f159c0058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:03:56.191 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-28T18:03:56.191+0000 7f15a6000700 1 -- 172.21.15.134:0/68156717 <== mon.1 v2:172.21.15.134:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f1594020020 con 0x7f15a81033c0 2024-03-28T18:03:56.427 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-28T18:03:56.427+0000 7f15b0600700 1 -- 172.21.15.134:0/68156717 --> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] -- mgr_command(tid 0: {"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi022=a;smithi022=d;smithi022=g;smithi022=j;smithi079=b;smithi079=e;smithi079=h;smithi079=k;smithi134=c;smithi134=f;smithi134=i;smithi134=l", "target": ["mon-mgr", ""]}) v1 -- 0x7f15a8062890 con 0x7f15900a2990 2024-03-28T18:03:56.445 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-28T18:03:56.444+0000 7f15a6000700 1 -- 172.21.15.134:0/68156717 <== mgr.14152 v2:172.21.15.22:6800/4201048392 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+28 (secure 0 0 0) 0x7f15a8062890 con 0x7f15900a2990 2024-03-28T18:03:56.445 INFO:teuthology.orchestra.run.smithi134.stdout:Scheduled mds.all update... 2024-03-28T18:03:56.448 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-28T18:03:56.447+0000 7f15b0600700 1 -- 172.21.15.134:0/68156717 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7f15900a2990 msgr2=0x7f15900a4e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:03:56.448 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-28T18:03:56.447+0000 7f15b0600700 1 --2- 172.21.15.134:0/68156717 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7f15900a2990 0x7f15900a4e50 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f159c005950 tx=0x7f159c0058e0 comp rx=0 tx=0).stop 2024-03-28T18:03:56.448 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-28T18:03:56.447+0000 7f15b0600700 1 -- 172.21.15.134:0/68156717 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f15a81033c0 msgr2=0x7f15a8074580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:03:56.448 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-28T18:03:56.447+0000 7f15b0600700 1 --2- 172.21.15.134:0/68156717 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f15a81033c0 0x7f15a8074580 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f159400d980 tx=0x7f159400dd40 comp rx=0 tx=0).stop 2024-03-28T18:03:56.448 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-28T18:03:56.447+0000 7f15b0600700 1 -- 172.21.15.134:0/68156717 shutdown_connections 2024-03-28T18:03:56.448 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-28T18:03:56.447+0000 7f15b0600700 1 --2- 172.21.15.134:0/68156717 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f15a8103d60 0x7f15a8074ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:03:56.448 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-28T18:03:56.447+0000 7f15b0600700 1 --2- 172.21.15.134:0/68156717 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f15a81021c0 0x7f15a8075f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:03:56.448 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-28T18:03:56.447+0000 7f15b0600700 1 --2- 172.21.15.134:0/68156717 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7f15900a2990 0x7f15900a4e50 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:03:56.448 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-28T18:03:56.447+0000 7f15b0600700 1 --2- 172.21.15.134:0/68156717 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f15a81033c0 0x7f15a8074580 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:03:56.448 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-28T18:03:56.447+0000 7f15b0600700 1 -- 172.21.15.134:0/68156717 >> 172.21.15.134:0/68156717 conn(0x7f15a80fd7a0 msgr2=0x7f15a8106740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:03:56.448 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-28T18:03:56.448+0000 7f15b0600700 1 -- 172.21.15.134:0/68156717 shutdown_connections 2024-03-28T18:03:56.448 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-28T18:03:56.448+0000 7f15b0600700 1 -- 172.21.15.134:0/68156717 wait complete. 2024-03-28T18:03:57.198 DEBUG:teuthology.orchestra.run.smithi022:mds.a> sudo journalctl -f -n 0 -u ceph-fd220748-ed2b-11ee-95d0-87774f69a715@mds.a.service 2024-03-28T18:03:57.206 DEBUG:teuthology.orchestra.run.smithi022:mds.d> sudo journalctl -f -n 0 -u ceph-fd220748-ed2b-11ee-95d0-87774f69a715@mds.d.service 2024-03-28T18:03:57.252 DEBUG:teuthology.orchestra.run.smithi022:mds.g> sudo journalctl -f -n 0 -u ceph-fd220748-ed2b-11ee-95d0-87774f69a715@mds.g.service 2024-03-28T18:03:57.298 DEBUG:teuthology.orchestra.run.smithi022:mds.j> sudo journalctl -f -n 0 -u ceph-fd220748-ed2b-11ee-95d0-87774f69a715@mds.j.service 2024-03-28T18:03:57.344 DEBUG:teuthology.orchestra.run.smithi079:mds.b> sudo journalctl -f -n 0 -u ceph-fd220748-ed2b-11ee-95d0-87774f69a715@mds.b.service 2024-03-28T18:03:57.352 DEBUG:teuthology.orchestra.run.smithi079:mds.e> sudo journalctl -f -n 0 -u ceph-fd220748-ed2b-11ee-95d0-87774f69a715@mds.e.service 2024-03-28T18:03:57.398 DEBUG:teuthology.orchestra.run.smithi079:mds.h> sudo journalctl -f -n 0 -u ceph-fd220748-ed2b-11ee-95d0-87774f69a715@mds.h.service 2024-03-28T18:03:57.445 DEBUG:teuthology.orchestra.run.smithi079:mds.k> sudo journalctl -f -n 0 -u ceph-fd220748-ed2b-11ee-95d0-87774f69a715@mds.k.service 2024-03-28T18:03:57.491 DEBUG:teuthology.orchestra.run.smithi134:mds.c> sudo journalctl -f -n 0 -u ceph-fd220748-ed2b-11ee-95d0-87774f69a715@mds.c.service 2024-03-28T18:03:57.501 DEBUG:teuthology.orchestra.run.smithi134:mds.f> sudo journalctl -f -n 0 -u ceph-fd220748-ed2b-11ee-95d0-87774f69a715@mds.f.service 2024-03-28T18:03:57.547 DEBUG:teuthology.orchestra.run.smithi134:mds.i> sudo journalctl -f -n 0 -u ceph-fd220748-ed2b-11ee-95d0-87774f69a715@mds.i.service 2024-03-28T18:03:57.591 DEBUG:teuthology.orchestra.run.smithi134:mds.l> sudo journalctl -f -n 0 -u ceph-fd220748-ed2b-11ee-95d0-87774f69a715@mds.l.service 2024-03-28T18:03:57.636 INFO:tasks.cephadm:Setting up CephFS filesystem(s)... 2024-03-28T18:03:57.636 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-28T18:03:57.695 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:03:57 smithi134 ceph-mon[35037]: from='client.24394 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi022=a;smithi022=d;smithi022=g;smithi022=j;smithi079=b;smithi079=e;smithi079=h;smithi079=k;smithi134=c;smithi134=f;smithi134=i;smithi134=l", "target": ["mon-mgr", ""]}]: dispatch 2024-03-28T18:03:57.695 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:03:57 smithi134 ceph-mon[35037]: Saving service mds.all spec with placement smithi022=a;smithi022=d;smithi022=g;smithi022=j;smithi079=b;smithi079=e;smithi079=h;smithi079=k;smithi134=c;smithi134=f;smithi134=i;smithi134=l;count:12 2024-03-28T18:03:57.695 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:03:57 smithi134 ceph-mon[35037]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T18:03:57.695 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:03:57 smithi134 ceph-mon[35037]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T18:03:57.695 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:03:57 smithi134 ceph-mon[35037]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T18:03:57.695 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:03:57 smithi134 ceph-mon[35037]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T18:03:57.696 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:03:57 smithi134 ceph-mon[35037]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T18:03:57.696 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:03:57 smithi134 ceph-mon[35037]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-28T18:03:57.696 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:03:57 smithi134 ceph-mon[35037]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-28T18:03:57.696 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:03:57 smithi134 ceph-mon[35037]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T18:03:57.696 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:03:57 smithi134 ceph-mon[35037]: Deploying daemon mds.l on smithi134 2024-03-28T18:03:57.730 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:03:57 smithi022 ceph-mon[31985]: from='client.24394 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi022=a;smithi022=d;smithi022=g;smithi022=j;smithi079=b;smithi079=e;smithi079=h;smithi079=k;smithi134=c;smithi134=f;smithi134=i;smithi134=l", "target": ["mon-mgr", ""]}]: dispatch 2024-03-28T18:03:57.730 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:03:57 smithi022 ceph-mon[31985]: Saving service mds.all spec with placement smithi022=a;smithi022=d;smithi022=g;smithi022=j;smithi079=b;smithi079=e;smithi079=h;smithi079=k;smithi134=c;smithi134=f;smithi134=i;smithi134=l;count:12 2024-03-28T18:03:57.730 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:03:57 smithi022 ceph-mon[31985]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T18:03:57.730 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:03:57 smithi022 ceph-mon[31985]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T18:03:57.730 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:03:57 smithi022 ceph-mon[31985]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T18:03:57.730 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:03:57 smithi022 ceph-mon[31985]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T18:03:57.730 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:03:57 smithi022 ceph-mon[31985]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T18:03:57.730 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:03:57 smithi022 ceph-mon[31985]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-28T18:03:57.730 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:03:57 smithi022 ceph-mon[31985]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-28T18:03:57.730 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:03:57 smithi022 ceph-mon[31985]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T18:03:57.731 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:03:57 smithi022 ceph-mon[31985]: Deploying daemon mds.l on smithi134 2024-03-28T18:03:57.933 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:03:57 smithi079 ceph-mon[35312]: from='client.24394 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi022=a;smithi022=d;smithi022=g;smithi022=j;smithi079=b;smithi079=e;smithi079=h;smithi079=k;smithi134=c;smithi134=f;smithi134=i;smithi134=l", "target": ["mon-mgr", ""]}]: dispatch 2024-03-28T18:03:57.933 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:03:57 smithi079 ceph-mon[35312]: Saving service mds.all spec with placement smithi022=a;smithi022=d;smithi022=g;smithi022=j;smithi079=b;smithi079=e;smithi079=h;smithi079=k;smithi134=c;smithi134=f;smithi134=i;smithi134=l;count:12 2024-03-28T18:03:57.933 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:03:57 smithi079 ceph-mon[35312]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T18:03:57.933 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:03:57 smithi079 ceph-mon[35312]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T18:03:57.933 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:03:57 smithi079 ceph-mon[35312]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T18:03:57.933 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:03:57 smithi079 ceph-mon[35312]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T18:03:57.934 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:03:57 smithi079 ceph-mon[35312]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T18:03:57.934 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:03:57 smithi079 ceph-mon[35312]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-28T18:03:57.934 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:03:57 smithi079 ceph-mon[35312]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-28T18:03:57.934 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:03:57 smithi079 ceph-mon[35312]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T18:03:57.934 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:03:57 smithi079 ceph-mon[35312]: Deploying daemon mds.l on smithi134 2024-03-28T18:03:57.979 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:03:57.978+0000 7fa6e1400640 1 -- 172.21.15.22:0/4005945568 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa6dc1026d0 msgr2=0x7fa6dc102b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:03:57.979 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:03:57.978+0000 7fa6e1400640 1 --2- 172.21.15.22:0/4005945568 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa6dc1026d0 0x7fa6dc102b50 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fa6c8009900 tx=0x7fa6c802bc50 comp rx=0 tx=0).stop 2024-03-28T18:03:57.979 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:03:57.979+0000 7fa6e1400640 1 -- 172.21.15.22:0/4005945568 shutdown_connections 2024-03-28T18:03:57.979 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:03:57.979+0000 7fa6e1400640 1 --2- 172.21.15.22:0/4005945568 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa6dc103090 0x7fa6dc109920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:03:57.979 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:03:57.979+0000 7fa6e1400640 1 --2- 172.21.15.22:0/4005945568 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa6dc1026d0 0x7fa6dc102b50 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:03:57.979 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:03:57.979+0000 7fa6e1400640 1 --2- 172.21.15.22:0/4005945568 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa6dc1014e0 0x7fa6dc101900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:03:57.979 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:03:57.979+0000 7fa6e1400640 1 -- 172.21.15.22:0/4005945568 >> 172.21.15.22:0/4005945568 conn(0x7fa6dc0fca30 msgr2=0x7fa6dc0fee90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:03:57.979 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:03:57.979+0000 7fa6e1400640 1 -- 172.21.15.22:0/4005945568 shutdown_connections 2024-03-28T18:03:57.979 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:03:57.979+0000 7fa6e1400640 1 -- 172.21.15.22:0/4005945568 wait complete. 2024-03-28T18:03:57.980 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:03:57.980+0000 7fa6e1400640 1 Processor -- start 2024-03-28T18:03:57.980 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:03:57.980+0000 7fa6e1400640 1 -- start start 2024-03-28T18:03:57.982 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:03:57.981+0000 7fa6e1400640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa6dc1014e0 0x7fa6dc199a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:03:57.982 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:03:57.981+0000 7fa6e1400640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa6dc1026d0 0x7fa6dc199fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:03:57.982 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:03:57.981+0000 7fa6e1400640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa6dc103090 0x7fa6dc1a0fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:03:57.982 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:03:57.981+0000 7fa6e1400640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6dc10d190 con 0x7fa6dc1014e0 2024-03-28T18:03:57.982 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:03:57.981+0000 7fa6e1400640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6dc10d010 con 0x7fa6dc1026d0 2024-03-28T18:03:57.982 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:03:57.981+0000 7fa6e1400640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6dc10d310 con 0x7fa6dc103090 2024-03-28T18:03:57.982 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:03:57.981+0000 7fa6dbe00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa6dc1014e0 0x7fa6dc199a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:03:57.982 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:03:57.981+0000 7fa6dbe00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa6dc1014e0 0x7fa6dc199a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:58384/0 (socket says 172.21.15.22:58384) 2024-03-28T18:03:57.982 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:03:57.981+0000 7fa6dbe00640 1 -- 172.21.15.22:0/1249801337 learned_addr learned my addr 172.21.15.22:0/1249801337 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-28T18:03:57.982 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:03:57.981+0000 7fa6db400640 1 --2- 172.21.15.22:0/1249801337 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa6dc1026d0 0x7fa6dc199fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:03:57.982 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:03:57.982+0000 7fa6e0a00640 1 --2- 172.21.15.22:0/1249801337 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa6dc103090 0x7fa6dc1a0fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:03:57.982 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:03:57.982+0000 7fa6dbe00640 1 -- 172.21.15.22:0/1249801337 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa6dc103090 msgr2=0x7fa6dc1a0fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:03:57.983 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:03:57.982+0000 7fa6dbe00640 1 --2- 172.21.15.22:0/1249801337 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa6dc103090 0x7fa6dc1a0fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:03:57.983 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:03:57.982+0000 7fa6dbe00640 1 -- 172.21.15.22:0/1249801337 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa6dc1026d0 msgr2=0x7fa6dc199fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:03:57.983 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:03:57.982+0000 7fa6dbe00640 1 --2- 172.21.15.22:0/1249801337 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa6dc1026d0 0x7fa6dc199fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:03:57.983 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:03:57.982+0000 7fa6dbe00640 1 -- 172.21.15.22:0/1249801337 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa6dc1a1640 con 0x7fa6dc1014e0 2024-03-28T18:03:57.983 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:03:57.982+0000 7fa6dbe00640 1 --2- 172.21.15.22:0/1249801337 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa6dc1014e0 0x7fa6dc199a70 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fa6cc00d950 tx=0x7fa6cc00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:03:57.984 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:03:57.982+0000 7fa6d8c00640 1 -- 172.21.15.22:0/1249801337 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6cc017070 con 0x7fa6dc1014e0 2024-03-28T18:03:57.984 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:03:57.982+0000 7fa6d8c00640 1 -- 172.21.15.22:0/1249801337 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7fa6cc009c30 con 0x7fa6dc1014e0 2024-03-28T18:03:57.984 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:03:57.982+0000 7fa6e1400640 1 -- 172.21.15.22:0/1249801337 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa6dc1a1930 con 0x7fa6dc1014e0 2024-03-28T18:03:57.984 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:03:57.982+0000 7fa6d8c00640 1 -- 172.21.15.22:0/1249801337 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6cc00f620 con 0x7fa6dc1014e0 2024-03-28T18:03:57.984 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:03:57.983+0000 7fa6e1400640 1 -- 172.21.15.22:0/1249801337 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa6dc1a1e50 con 0x7fa6dc1014e0 2024-03-28T18:03:57.984 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:03:57.984+0000 7fa6e1400640 1 -- 172.21.15.22:0/1249801337 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa6dc1a2080 con 0x7fa6dc1014e0 2024-03-28T18:03:57.985 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:03:57.986+0000 7fa6d8c00640 1 -- 172.21.15.22:0/1249801337 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 16) v1 ==== 137290+0+0 (secure 0 0 0) 0x7fa6cc01a070 con 0x7fa6dc1014e0 2024-03-28T18:03:57.987 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:03:57.987+0000 7fa6d8c00640 1 --2- 172.21.15.22:0/1249801337 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7fa6c00a2880 0x7fa6c00a4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:03:57.987 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:03:57.987+0000 7fa6d8c00640 1 -- 172.21.15.22:0/1249801337 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 7127+0+0 (secure 0 0 0) 0x7fa6cc0ce0e0 con 0x7fa6dc1014e0 2024-03-28T18:03:57.987 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:03:57.987+0000 7fa6db400640 1 --2- 172.21.15.22:0/1249801337 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7fa6c00a2880 0x7fa6c00a4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:03:57.988 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:03:57.988+0000 7fa6db400640 1 --2- 172.21.15.22:0/1249801337 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7fa6c00a2880 0x7fa6c00a4d40 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fa6c8006010 tx=0x7fa6c8000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:03:57.992 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:03:57.992+0000 7fa6d8c00640 1 -- 172.21.15.22:0/1249801337 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fa6cc009de0 con 0x7fa6dc1014e0 2024-03-28T18:03:58.195 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:03:58.195+0000 7fa6e1400640 1 -- 172.21.15.22:0/1249801337 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fa6dc1a2410 con 0x7fa6dc1014e0 2024-03-28T18:03:58.197 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:03:58.198+0000 7fa6d8c00640 1 -- 172.21.15.22:0/1249801337 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v86) v1 ==== 74+0+18420 (secure 0 0 0) 0x7fa6dc1a2410 con 0x7fa6dc1014e0 2024-03-28T18:03:58.198 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-03-28T18:03:58.198 INFO:teuthology.orchestra.run.smithi022.stdout:{"epoch":86,"fsid":"fd220748-ed2b-11ee-95d0-87774f69a715","created":"2024-03-28T17:54:00.440279+0000","modified":"2024-03-28T18:03:44.232788+0000","last_up_change":"2024-03-28T18:03:43.220095+0000","last_in_change":"2024-03-28T18:03:18.526399+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-28T17:58:33.168872+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"5d708e06-f05b-40ba-bbcf-258ddc4f43d7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6802","nonce":2186290846},{"type":"v1","addr":"172.21.15.22:6803","nonce":2186290846}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6804","nonce":2186290846},{"type":"v1","addr":"172.21.15.22:6805","nonce":2186290846}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6808","nonce":2186290846},{"type":"v1","addr":"172.21.15.22:6809","nonce":2186290846}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6806","nonce":2186290846},{"type":"v1","addr":"172.21.15.22:6807","nonce":2186290846}]},"public_addr":"172.21.15.22:6803/2186290846","cluster_addr":"172.21.15.22:6805/2186290846","heartbeat_back_addr":"172.21.15.22:6809/2186290846","heartbeat_front_addr":"172.21.15.22:6807/2186290846","state":["exists","up"]},{"osd":1,"uuid":"68ad09bb-9256-409a-9867-4e14a2cb94aa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6800","nonce":2835549681},{"type":"v1","addr":"172.21.15.79:6801","nonce":2835549681}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6802","nonce":2835549681},{"type":"v1","addr":"172.21.15.79:6803","nonce":2835549681}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6806","nonce":2835549681},{"type":"v1","addr":"172.21.15.79:6807","nonce":2835549681}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6804","nonce":2835549681},{"type":"v1","addr":"172.21.15.79:6805","nonce":2835549681}]},"public_addr":"172.21.15.79:6801/2835549681","cluster_addr":"172.21.15.79:6803/2835549681","heartbeat_back_addr":"172.21.15.79:6807/2835549681","heartbeat_front_addr":"172.21.15.79:6805/2835549681","state":["exists","up"]},{"osd":2,"uuid":"ed11a5a7-004d-41f3-a5fc-eb2d603bd33f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6800","nonce":470588085},{"type":"v1","addr":"172.21.15.134:6801","nonce":470588085}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6802","nonce":470588085},{"type":"v1","addr":"172.21.15.134:6803","nonce":470588085}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6806","nonce":470588085},{"type":"v1","addr":"172.21.15.134:6807","nonce":470588085}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6804","nonce":470588085},{"type":"v1","addr":"172.21.15.134:6805","nonce":470588085}]},"public_addr":"172.21.15.134:6801/470588085","cluster_addr":"172.21.15.134:6803/470588085","heartbeat_back_addr":"172.21.15.134:6807/470588085","heartbeat_front_addr":"172.21.15.134:6805/470588085","state":["exists","up"]},{"osd":3,"uuid":"81a6c547-2a50-44c3-84d7-23ab6831790c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6810","nonce":1702441854},{"type":"v1","addr":"172.21.15.22:6811","nonce":1702441854}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6812","nonce":1702441854},{"type":"v1","addr":"172.21.15.22:6813","nonce":1702441854}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6816","nonce":1702441854},{"type":"v1","addr":"172.21.15.22:6817","nonce":1702441854}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6814","nonce":1702441854},{"type":"v1","addr":"172.21.15.22:6815","nonce":1702441854}]},"public_addr":"172.21.15.22:6811/1702441854","cluster_addr":"172.21.15.22:6813/1702441854","heartbeat_back_addr":"172.21.15.22:6817/1702441854","heartbeat_front_addr":"172.21.15.22:6815/1702441854","state":["exists","up"]},{"osd":4,"uuid":"d23a254e-7182-4471-8bca-7eb684ec618e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6808","nonce":630132437},{"type":"v1","addr":"172.21.15.79:6809","nonce":630132437}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6810","nonce":630132437},{"type":"v1","addr":"172.21.15.79:6811","nonce":630132437}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6814","nonce":630132437},{"type":"v1","addr":"172.21.15.79:6815","nonce":630132437}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6812","nonce":630132437},{"type":"v1","addr":"172.21.15.79:6813","nonce":630132437}]},"public_addr":"172.21.15.79:6809/630132437","cluster_addr":"172.21.15.79:6811/630132437","heartbeat_back_addr":"172.21.15.79:6815/630132437","heartbeat_front_addr":"172.21.15.79:6813/630132437","state":["exists","up"]},{"osd":5,"uuid":"a7b98502-2023-4f0a-bfa0-6840cc495e0c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":40,"up_thru":55,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6808","nonce":2135406915},{"type":"v1","addr":"172.21.15.134:6809","nonce":2135406915}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6810","nonce":2135406915},{"type":"v1","addr":"172.21.15.134:6811","nonce":2135406915}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6814","nonce":2135406915},{"type":"v1","addr":"172.21.15.134:6815","nonce":2135406915}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6812","nonce":2135406915},{"type":"v1","addr":"172.21.15.134:6813","nonce":2135406915}]},"public_addr":"172.21.15.134:6809/2135406915","cluster_addr":"172.21.15.134:6811/2135406915","heartbeat_back_addr":"172.21.15.134:6815/2135406915","heartbeat_front_addr":"172.21.15.134:6813/2135406915","state":["exists","up"]},{"osd":6,"uuid":"634c6562-d683-41c8-8242-fc848c38ce1f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6818","nonce":1400680396},{"type":"v1","addr":"172.21.15.22:6819","nonce":1400680396}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6820","nonce":1400680396},{"type":"v1","addr":"172.21.15.22:6821","nonce":1400680396}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6824","nonce":1400680396},{"type":"v1","addr":"172.21.15.22:6825","nonce":1400680396}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6822","nonce":1400680396},{"type":"v1","addr":"172.21.15.22:6823","nonce":1400680396}]},"public_addr":"172.21.15.22:6819/1400680396","cluster_addr":"172.21.15.22:6821/1400680396","heartbeat_back_addr":"172.21.15.22:6825/1400680396","heartbeat_front_addr":"172.21.15.22:6823/1400680396","state":["exists","up"]},{"osd":7,"uuid":"504eb4a8-6731-45ad-932c-1b38ef157871","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":53,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6816","nonce":1287105314},{"type":"v1","addr":"172.21.15.79:6817","nonce":1287105314}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6818","nonce":1287105314},{"type":"v1","addr":"172.21.15.79:6819","nonce":1287105314}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6822","nonce":1287105314},{"type":"v1","addr":"172.21.15.79:6823","nonce":1287105314}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6820","nonce":1287105314},{"type":"v1","addr":"172.21.15.79:6821","nonce":1287105314}]},"public_addr":"172.21.15.79:6817/1287105314","cluster_addr":"172.21.15.79:6819/1287105314","heartbeat_back_addr":"172.21.15.79:6823/1287105314","heartbeat_front_addr":"172.21.15.79:6821/1287105314","state":["exists","up"]},{"osd":8,"uuid":"20aa66c1-03d6-4cb8-a096-0eae41979c17","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":62,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6816","nonce":2265650388},{"type":"v1","addr":"172.21.15.134:6817","nonce":2265650388}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6818","nonce":2265650388},{"type":"v1","addr":"172.21.15.134:6819","nonce":2265650388}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6822","nonce":2265650388},{"type":"v1","addr":"172.21.15.134:6823","nonce":2265650388}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6820","nonce":2265650388},{"type":"v1","addr":"172.21.15.134:6821","nonce":2265650388}]},"public_addr":"172.21.15.134:6817/2265650388","cluster_addr":"172.21.15.134:6819/2265650388","heartbeat_back_addr":"172.21.15.134:6823/2265650388","heartbeat_front_addr":"172.21.15.134:6821/2265650388","state":["exists","up"]},{"osd":9,"uuid":"e1ae4020-7c2c-4334-a753-ee6b12d98584","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":70,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6826","nonce":3762222259},{"type":"v1","addr":"172.21.15.22:6827","nonce":3762222259}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6828","nonce":3762222259},{"type":"v1","addr":"172.21.15.22:6829","nonce":3762222259}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6832","nonce":3762222259},{"type":"v1","addr":"172.21.15.22:6833","nonce":3762222259}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6830","nonce":3762222259},{"type":"v1","addr":"172.21.15.22:6831","nonce":3762222259}]},"public_addr":"172.21.15.22:6827/3762222259","cluster_addr":"172.21.15.22:6829/3762222259","heartbeat_back_addr":"172.21.15.22:6833/3762222259","heartbeat_front_addr":"172.21.15.22:6831/3762222259","state":["exists","up"]},{"osd":10,"uuid":"f5198d52-df48-4679-a860-4dcb440b7a9f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":77,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6824","nonce":1746167904},{"type":"v1","addr":"172.21.15.79:6825","nonce":1746167904}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6826","nonce":1746167904},{"type":"v1","addr":"172.21.15.79:6827","nonce":1746167904}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6830","nonce":1746167904},{"type":"v1","addr":"172.21.15.79:6831","nonce":1746167904}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6828","nonce":1746167904},{"type":"v1","addr":"172.21.15.79:6829","nonce":1746167904}]},"public_addr":"172.21.15.79:6825/1746167904","cluster_addr":"172.21.15.79:6827/1746167904","heartbeat_back_addr":"172.21.15.79:6831/1746167904","heartbeat_front_addr":"172.21.15.79:6829/1746167904","state":["exists","up"]},{"osd":11,"uuid":"1427bca7-3414-4468-9246-e999156bc2ae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":85,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6824","nonce":4271716428},{"type":"v1","addr":"172.21.15.134:6825","nonce":4271716428}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6826","nonce":4271716428},{"type":"v1","addr":"172.21.15.134:6827","nonce":4271716428}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6830","nonce":4271716428},{"type":"v1","addr":"172.21.15.134:6831","nonce":4271716428}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6828","nonce":4271716428},{"type":"v1","addr":"172.21.15.134:6829","nonce":4271716428}]},"public_addr":"172.21.15.134:6825/4271716428","cluster_addr":"172.21.15.134:6827/4271716428","heartbeat_back_addr":"172.21.15.134:6831/4271716428","heartbeat_front_addr":"172.21.15.134:6829/4271716428","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T17:57:21.225125+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T17:57:55.197254+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T17:58:27.825136+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T17:59:04.319893+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T17:59:38.407752+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T18:00:12.118805+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T18:00:47.635214+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T18:01:21.231055+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T18:01:55.817861+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T18:02:31.680160+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T18:03:06.783398+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T18:03:41.307347+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.22:0/3521924283":"2024-03-29T17:55:10.425504+0000","172.21.15.22:6801/730635939":"2024-03-29T17:55:10.425504+0000","172.21.15.22:0/2471229459":"2024-03-29T17:54:34.004559+0000","172.21.15.22:6800/3166828081":"2024-03-29T17:54:34.004559+0000","172.21.15.22:6800/730635939":"2024-03-29T17:55:10.425504+0000","172.21.15.22:0/1229047849":"2024-03-29T17:54:34.004559+0000","172.21.15.22:6801/3166828081":"2024-03-29T17:54:34.004559+0000","172.21.15.22:0/2717818856":"2024-03-29T17:55:10.425504+0000","172.21.15.22:0/2157570333":"2024-03-29T17:54:34.004559+0000","172.21.15.22:0/1006589530":"2024-03-29T17:55:10.425504+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-28T18:03:58.200 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:03:58.200+0000 7fa6e1400640 1 -- 172.21.15.22:0/1249801337 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7fa6c00a2880 msgr2=0x7fa6c00a4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:03:58.200 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:03:58.200+0000 7fa6e1400640 1 --2- 172.21.15.22:0/1249801337 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7fa6c00a2880 0x7fa6c00a4d40 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fa6c8006010 tx=0x7fa6c8000bc0 comp rx=0 tx=0).stop 2024-03-28T18:03:58.200 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:03:58.200+0000 7fa6e1400640 1 -- 172.21.15.22:0/1249801337 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa6dc1014e0 msgr2=0x7fa6dc199a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:03:58.200 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:03:58.200+0000 7fa6e1400640 1 --2- 172.21.15.22:0/1249801337 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa6dc1014e0 0x7fa6dc199a70 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fa6cc00d950 tx=0x7fa6cc00de10 comp rx=0 tx=0).stop 2024-03-28T18:03:58.200 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:03:58.200+0000 7fa6e1400640 1 -- 172.21.15.22:0/1249801337 shutdown_connections 2024-03-28T18:03:58.201 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:03:58.200+0000 7fa6e1400640 1 --2- 172.21.15.22:0/1249801337 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7fa6c00a2880 0x7fa6c00a4d40 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:03:58.201 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:03:58.200+0000 7fa6e1400640 1 --2- 172.21.15.22:0/1249801337 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa6dc103090 0x7fa6dc1a0fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:03:58.201 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:03:58.200+0000 7fa6e1400640 1 --2- 172.21.15.22:0/1249801337 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa6dc1026d0 0x7fa6dc199fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:03:58.201 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:03:58.200+0000 7fa6e1400640 1 --2- 172.21.15.22:0/1249801337 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa6dc1014e0 0x7fa6dc199a70 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:03:58.201 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:03:58.200+0000 7fa6e1400640 1 -- 172.21.15.22:0/1249801337 >> 172.21.15.22:0/1249801337 conn(0x7fa6dc0fca30 msgr2=0x7fa6dc0738d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:03:58.201 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:03:58.201+0000 7fa6e1400640 1 -- 172.21.15.22:0/1249801337 shutdown_connections 2024-03-28T18:03:58.201 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:03:58.201+0000 7fa6e1400640 1 -- 172.21.15.22:0/1249801337 wait complete. 2024-03-28T18:03:58.220 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-28T17:58:33.168872+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-28T18:03:58.221 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-28T18:03:58.454 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:03:58.454+0000 7fd411c00640 1 -- 172.21.15.22:0/2925541815 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd40c105350 msgr2=0x7fd40c10bbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:03:58.454 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:03:58.454+0000 7fd411c00640 1 --2- 172.21.15.22:0/2925541815 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd40c105350 0x7fd40c10bbe0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fd404009ca0 tx=0x7fd40402c100 comp rx=0 tx=0).stop 2024-03-28T18:03:58.454 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:03:58.454+0000 7fd411c00640 1 -- 172.21.15.22:0/2925541815 shutdown_connections 2024-03-28T18:03:58.454 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:03:58.454+0000 7fd411c00640 1 --2- 172.21.15.22:0/2925541815 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd40c105350 0x7fd40c10bbe0 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:03:58.454 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:03:58.454+0000 7fd411c00640 1 --2- 172.21.15.22:0/2925541815 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd40c104990 0x7fd40c104e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:03:58.454 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:03:58.454+0000 7fd411c00640 1 --2- 172.21.15.22:0/2925541815 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd40c1037a0 0x7fd40c103bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:03:58.454 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:03:58.454+0000 7fd411c00640 1 -- 172.21.15.22:0/2925541815 >> 172.21.15.22:0/2925541815 conn(0x7fd40c0fed10 msgr2=0x7fd40c101150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:03:58.454 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:03:58.454+0000 7fd411c00640 1 -- 172.21.15.22:0/2925541815 shutdown_connections 2024-03-28T18:03:58.455 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:03:58.454+0000 7fd411c00640 1 -- 172.21.15.22:0/2925541815 wait complete. 2024-03-28T18:03:58.457 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:03:58.455+0000 7fd411c00640 1 Processor -- start 2024-03-28T18:03:58.457 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:03:58.455+0000 7fd411c00640 1 -- start start 2024-03-28T18:03:58.457 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:03:58.455+0000 7fd411c00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd40c1037a0 0x7fd40c199a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:03:58.457 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:03:58.455+0000 7fd411c00640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd40c104990 0x7fd40c199f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:03:58.457 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:03:58.455+0000 7fd411c00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd40c105350 0x7fd40c1a1000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:03:58.457 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:03:58.455+0000 7fd411c00640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd40c10f3f0 con 0x7fd40c105350 2024-03-28T18:03:58.457 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:03:58.455+0000 7fd411c00640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd40c10f270 con 0x7fd40c104990 2024-03-28T18:03:58.457 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:03:58.455+0000 7fd411c00640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd40c10f570 con 0x7fd40c1037a0 2024-03-28T18:03:58.457 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:03:58.456+0000 7fd411200640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd40c105350 0x7fd40c1a1000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:03:58.457 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:03:58.456+0000 7fd411200640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd40c105350 0x7fd40c1a1000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:58408/0 (socket says 172.21.15.22:58408) 2024-03-28T18:03:58.457 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:03:58.456+0000 7fd411200640 1 -- 172.21.15.22:0/3116645869 learned_addr learned my addr 172.21.15.22:0/3116645869 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-28T18:03:58.457 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:03:58.456+0000 7fd403400640 1 --2- 172.21.15.22:0/3116645869 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd40c104990 0x7fd40c199f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:03:58.457 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:03:58.456+0000 7fd40be00640 1 --2- 172.21.15.22:0/3116645869 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd40c1037a0 0x7fd40c199a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:03:58.458 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:03:58.456+0000 7fd411200640 1 -- 172.21.15.22:0/3116645869 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd40c1037a0 msgr2=0x7fd40c199a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:03:58.458 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:03:58.456+0000 7fd411200640 1 --2- 172.21.15.22:0/3116645869 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd40c1037a0 0x7fd40c199a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:03:58.458 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:03:58.456+0000 7fd411200640 1 -- 172.21.15.22:0/3116645869 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd40c104990 msgr2=0x7fd40c199f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:03:58.458 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:03:58.456+0000 7fd411200640 1 --2- 172.21.15.22:0/3116645869 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd40c104990 0x7fd40c199f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:03:58.458 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:03:58.456+0000 7fd411200640 1 -- 172.21.15.22:0/3116645869 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd40c1a1670 con 0x7fd40c105350 2024-03-28T18:03:58.458 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:03:58.456+0000 7fd411200640 1 --2- 172.21.15.22:0/3116645869 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd40c105350 0x7fd40c1a1000 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fd404032540 tx=0x7fd404034cd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:03:58.458 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:03:58.457+0000 7fd409600640 1 -- 172.21.15.22:0/3116645869 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd404039050 con 0x7fd40c105350 2024-03-28T18:03:58.458 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:03:58.457+0000 7fd411c00640 1 -- 172.21.15.22:0/3116645869 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd40c1a1900 con 0x7fd40c105350 2024-03-28T18:03:58.458 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:03:58.457+0000 7fd409600640 1 -- 172.21.15.22:0/3116645869 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7fd40403a040 con 0x7fd40c105350 2024-03-28T18:03:58.458 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:03:58.457+0000 7fd409600640 1 -- 172.21.15.22:0/3116645869 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4040046e0 con 0x7fd40c105350 2024-03-28T18:03:58.458 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:03:58.457+0000 7fd411c00640 1 -- 172.21.15.22:0/3116645869 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd40c1a1df0 con 0x7fd40c105350 2024-03-28T18:03:58.459 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:03:58.458+0000 7fd411c00640 1 -- 172.21.15.22:0/3116645869 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd40c10aa60 con 0x7fd40c105350 2024-03-28T18:03:58.460 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:03:58.460+0000 7fd409600640 1 -- 172.21.15.22:0/3116645869 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 16) v1 ==== 137290+0+0 (secure 0 0 0) 0x7fd404003970 con 0x7fd40c105350 2024-03-28T18:03:58.461 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:03:58.462+0000 7fd409600640 1 --2- 172.21.15.22:0/3116645869 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7fd3f00a2800 0x7fd3f00a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:03:58.461 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:03:58.462+0000 7fd409600640 1 -- 172.21.15.22:0/3116645869 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 7127+0+0 (secure 0 0 0) 0x7fd4040f0f30 con 0x7fd40c105350 2024-03-28T18:03:58.461 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:03:58.462+0000 7fd40be00640 1 --2- 172.21.15.22:0/3116645869 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7fd3f00a2800 0x7fd3f00a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:03:58.462 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:03:58.463+0000 7fd40be00640 1 --2- 172.21.15.22:0/3116645869 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7fd3f00a2800 0x7fd3f00a4cc0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fd3fc005fd0 tx=0x7fd3fc005c70 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:03:58.465 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:03:58.466+0000 7fd409600640 1 -- 172.21.15.22:0/3116645869 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fd4040be470 con 0x7fd40c105350 2024-03-28T18:03:58.670 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:03:58.670+0000 7fd411c00640 1 -- 172.21.15.22:0/3116645869 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fd40c0622a0 con 0x7fd40c105350 2024-03-28T18:03:58.670 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:03:58.670+0000 7fd409600640 1 -- 172.21.15.22:0/3116645869 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v86) v1 ==== 93+0+10 (secure 0 0 0) 0x7fd4040bdd70 con 0x7fd40c105350 2024-03-28T18:03:58.670 INFO:teuthology.orchestra.run.smithi022.stdout:pg_num: 1 2024-03-28T18:03:58.673 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:03:58.673+0000 7fd411c00640 1 -- 172.21.15.22:0/3116645869 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7fd3f00a2800 msgr2=0x7fd3f00a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:03:58.673 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:03:58.673+0000 7fd411c00640 1 --2- 172.21.15.22:0/3116645869 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7fd3f00a2800 0x7fd3f00a4cc0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fd3fc005fd0 tx=0x7fd3fc005c70 comp rx=0 tx=0).stop 2024-03-28T18:03:58.673 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:03:58.673+0000 7fd411c00640 1 -- 172.21.15.22:0/3116645869 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd40c105350 msgr2=0x7fd40c1a1000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:03:58.673 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:03:58.673+0000 7fd411c00640 1 --2- 172.21.15.22:0/3116645869 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd40c105350 0x7fd40c1a1000 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fd404032540 tx=0x7fd404034cd0 comp rx=0 tx=0).stop 2024-03-28T18:03:58.673 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:03:58.673+0000 7fd411c00640 1 -- 172.21.15.22:0/3116645869 shutdown_connections 2024-03-28T18:03:58.673 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:03:58.673+0000 7fd411c00640 1 --2- 172.21.15.22:0/3116645869 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7fd3f00a2800 0x7fd3f00a4cc0 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:03:58.673 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:03:58.673+0000 7fd411c00640 1 --2- 172.21.15.22:0/3116645869 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd40c105350 0x7fd40c1a1000 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:03:58.673 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:03:58.673+0000 7fd411c00640 1 --2- 172.21.15.22:0/3116645869 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd40c104990 0x7fd40c199f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:03:58.673 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:03:58.673+0000 7fd411c00640 1 --2- 172.21.15.22:0/3116645869 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd40c1037a0 0x7fd40c199a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:03:58.673 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:03:58.673+0000 7fd411c00640 1 -- 172.21.15.22:0/3116645869 >> 172.21.15.22:0/3116645869 conn(0x7fd40c0fed10 msgr2=0x7fd40c107c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:03:58.673 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:03:58.673+0000 7fd411c00640 1 -- 172.21.15.22:0/3116645869 shutdown_connections 2024-03-28T18:03:58.674 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:03:58.673+0000 7fd411c00640 1 -- 172.21.15.22:0/3116645869 wait complete. 2024-03-28T18:03:58.692 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-28T18:03:58.777 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:03:58 smithi022 ceph-mon[31985]: pgmap v339: 1 pgs: 1 active+clean; 577 KiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-28T18:03:58.777 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:03:58 smithi022 ceph-mon[31985]: from='client.? 172.21.15.22:0/1249801337' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-28T18:03:58.840 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:03:58 smithi134 ceph-mon[35037]: pgmap v339: 1 pgs: 1 active+clean; 577 KiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-28T18:03:58.840 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:03:58 smithi134 ceph-mon[35037]: from='client.? 172.21.15.22:0/1249801337' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-28T18:03:58.919 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:03:58.919+0000 7fd452a00640 1 -- 172.21.15.22:0/691224127 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd44c1073a0 msgr2=0x7fd44c1097f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:03:58.920 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:03:58.919+0000 7fd452a00640 1 --2- 172.21.15.22:0/691224127 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd44c1073a0 0x7fd44c1097f0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fd444009ca0 tx=0x7fd44402c100 comp rx=0 tx=0).stop 2024-03-28T18:03:58.920 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:03:58.920+0000 7fd452a00640 1 -- 172.21.15.22:0/691224127 shutdown_connections 2024-03-28T18:03:58.920 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:03:58.920+0000 7fd452a00640 1 --2- 172.21.15.22:0/691224127 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd44c1073a0 0x7fd44c1097f0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:03:58.920 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:03:58.920+0000 7fd452a00640 1 --2- 172.21.15.22:0/691224127 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd44c104a70 0x7fd44c106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:03:58.920 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:03:58.920+0000 7fd452a00640 1 --2- 172.21.15.22:0/691224127 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd44c102140 0x7fd44c104530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:03:58.920 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:03:58.920+0000 7fd452a00640 1 -- 172.21.15.22:0/691224127 >> 172.21.15.22:0/691224127 conn(0x7fd44c0fbb50 msgr2=0x7fd44c0fdf70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:03:58.920 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:03:58.920+0000 7fd452a00640 1 -- 172.21.15.22:0/691224127 shutdown_connections 2024-03-28T18:03:58.920 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:03:58.920+0000 7fd452a00640 1 -- 172.21.15.22:0/691224127 wait complete. 2024-03-28T18:03:58.920 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:03:58.921+0000 7fd452a00640 1 Processor -- start 2024-03-28T18:03:58.921 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:03:58.921+0000 7fd452a00640 1 -- start start 2024-03-28T18:03:58.922 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:03:58.921+0000 7fd452a00640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd44c102140 0x7fd44c101ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:03:58.922 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:03:58.921+0000 7fd452a00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd44c104a70 0x7fd44c1000f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:03:58.922 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:03:58.921+0000 7fd452a00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd44c1073a0 0x7fd44c100630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:03:58.922 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:03:58.922+0000 7fd452a00640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd44c10d320 con 0x7fd44c1073a0 2024-03-28T18:03:58.922 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:03:58.922+0000 7fd452a00640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd44c10d1a0 con 0x7fd44c102140 2024-03-28T18:03:58.922 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:03:58.922+0000 7fd452a00640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd44c10d4a0 con 0x7fd44c104a70 2024-03-28T18:03:58.922 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:03:58.922+0000 7fd450a00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd44c1073a0 0x7fd44c100630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:03:58.922 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:03:58.922+0000 7fd450a00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd44c1073a0 0x7fd44c100630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:58428/0 (socket says 172.21.15.22:58428) 2024-03-28T18:03:58.922 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:03:58.922+0000 7fd450a00640 1 -- 172.21.15.22:0/3462476062 learned_addr learned my addr 172.21.15.22:0/3462476062 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-28T18:03:58.923 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:03:58.922+0000 7fd443400640 1 --2- 172.21.15.22:0/3462476062 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd44c104a70 0x7fd44c1000f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:03:58.923 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:03:58.922+0000 7fd44be00640 1 --2- 172.21.15.22:0/3462476062 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd44c102140 0x7fd44c101ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:03:58.923 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:03:58.922+0000 7fd450a00640 1 -- 172.21.15.22:0/3462476062 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd44c104a70 msgr2=0x7fd44c1000f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:03:58.923 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:03:58.922+0000 7fd450a00640 1 --2- 172.21.15.22:0/3462476062 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd44c104a70 0x7fd44c1000f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:03:58.923 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:03:58.922+0000 7fd450a00640 1 -- 172.21.15.22:0/3462476062 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd44c102140 msgr2=0x7fd44c101ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:03:58.923 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:03:58.922+0000 7fd450a00640 1 --2- 172.21.15.22:0/3462476062 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd44c102140 0x7fd44c101ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:03:58.923 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:03:58.922+0000 7fd450a00640 1 -- 172.21.15.22:0/3462476062 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd44c100ec0 con 0x7fd44c1073a0 2024-03-28T18:03:58.923 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:03:58.922+0000 7fd443400640 1 --2- 172.21.15.22:0/3462476062 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd44c104a70 0x7fd44c1000f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-28T18:03:58.923 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:03:58.922+0000 7fd44be00640 1 --2- 172.21.15.22:0/3462476062 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd44c102140 0x7fd44c101ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-28T18:03:58.924 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:03:58.923+0000 7fd450a00640 1 --2- 172.21.15.22:0/3462476062 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd44c1073a0 0x7fd44c100630 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fd444005e30 tx=0x7fd444034cf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:03:58.924 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:03:58.923+0000 7fd449600640 1 -- 172.21.15.22:0/3462476062 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd444039050 con 0x7fd44c1073a0 2024-03-28T18:03:58.924 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:03:58.923+0000 7fd449600640 1 -- 172.21.15.22:0/3462476062 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7fd44403a040 con 0x7fd44c1073a0 2024-03-28T18:03:58.924 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:03:58.923+0000 7fd452a00640 1 -- 172.21.15.22:0/3462476062 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd44c1a19a0 con 0x7fd44c1073a0 2024-03-28T18:03:58.924 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:03:58.923+0000 7fd449600640 1 -- 172.21.15.22:0/3462476062 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4440046e0 con 0x7fd44c1073a0 2024-03-28T18:03:58.924 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:03:58.923+0000 7fd452a00640 1 -- 172.21.15.22:0/3462476062 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd44c1a1f40 con 0x7fd44c1073a0 2024-03-28T18:03:58.924 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:03:58.925+0000 7fd452a00640 1 -- 172.21.15.22:0/3462476062 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd44c0fd640 con 0x7fd44c1073a0 2024-03-28T18:03:58.926 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:03:58.926+0000 7fd449600640 1 -- 172.21.15.22:0/3462476062 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 16) v1 ==== 137290+0+0 (secure 0 0 0) 0x7fd444003970 con 0x7fd44c1073a0 2024-03-28T18:03:58.928 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:03:58.928+0000 7fd449600640 1 --2- 172.21.15.22:0/3462476062 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7fd4300a2880 0x7fd4300a4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:03:58.928 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:03:58.928+0000 7fd449600640 1 -- 172.21.15.22:0/3462476062 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 7127+0+0 (secure 0 0 0) 0x7fd4440f0f30 con 0x7fd44c1073a0 2024-03-28T18:03:58.928 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:03:58.928+0000 7fd44be00640 1 --2- 172.21.15.22:0/3462476062 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7fd4300a2880 0x7fd4300a4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:03:58.928 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:03:58.929+0000 7fd44be00640 1 --2- 172.21.15.22:0/3462476062 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7fd4300a2880 0x7fd4300a4d40 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fd43c00abe0 tx=0x7fd43c00a5c0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:03:58.932 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:03:58.932+0000 7fd449600640 1 -- 172.21.15.22:0/3462476062 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fd4440be3d0 con 0x7fd44c1073a0 2024-03-28T18:03:58.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:03:58 smithi079 ceph-mon[35312]: pgmap v339: 1 pgs: 1 active+clean; 577 KiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-28T18:03:58.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:03:58 smithi079 ceph-mon[35312]: from='client.? 172.21.15.22:0/1249801337' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-28T18:03:59.175 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:03:59.176+0000 7fd452a00640 1 -- 172.21.15.22:0/3462476062 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd44c02d070 con 0x7fd44c1073a0 2024-03-28T18:03:59.176 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:03:59.176+0000 7fd449600640 1 -- 172.21.15.22:0/3462476062 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 1 v1) v1 ==== 93+0+535 (secure 0 0 0) 0x7fd4440bdcd0 con 0x7fd44c1073a0 2024-03-28T18:03:59.176 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-03-28T18:03:59.176 INFO:teuthology.orchestra.run.smithi022.stdout:{"epoch":1,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[]} 2024-03-28T18:03:59.177 INFO:teuthology.orchestra.run.smithi022.stderr:dumped fsmap epoch 1 2024-03-28T18:03:59.178 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:03:59.179+0000 7fd452a00640 1 -- 172.21.15.22:0/3462476062 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7fd4300a2880 msgr2=0x7fd4300a4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:03:59.178 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:03:59.179+0000 7fd452a00640 1 --2- 172.21.15.22:0/3462476062 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7fd4300a2880 0x7fd4300a4d40 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fd43c00abe0 tx=0x7fd43c00a5c0 comp rx=0 tx=0).stop 2024-03-28T18:03:59.179 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:03:59.179+0000 7fd452a00640 1 -- 172.21.15.22:0/3462476062 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd44c1073a0 msgr2=0x7fd44c100630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:03:59.179 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:03:59.179+0000 7fd452a00640 1 --2- 172.21.15.22:0/3462476062 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd44c1073a0 0x7fd44c100630 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fd444005e30 tx=0x7fd444034cf0 comp rx=0 tx=0).stop 2024-03-28T18:03:59.179 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:03:59.179+0000 7fd452a00640 1 -- 172.21.15.22:0/3462476062 shutdown_connections 2024-03-28T18:03:59.179 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:03:59.179+0000 7fd452a00640 1 --2- 172.21.15.22:0/3462476062 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7fd4300a2880 0x7fd4300a4d40 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:03:59.179 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:03:59.179+0000 7fd452a00640 1 --2- 172.21.15.22:0/3462476062 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd44c1073a0 0x7fd44c100630 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:03:59.180 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:03:59.179+0000 7fd452a00640 1 --2- 172.21.15.22:0/3462476062 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd44c104a70 0x7fd44c1000f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:03:59.180 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:03:59.179+0000 7fd452a00640 1 --2- 172.21.15.22:0/3462476062 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd44c102140 0x7fd44c101ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:03:59.180 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:03:59.179+0000 7fd452a00640 1 -- 172.21.15.22:0/3462476062 >> 172.21.15.22:0/3462476062 conn(0x7fd44c0fbb50 msgr2=0x7fd44c102a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:03:59.180 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:03:59.179+0000 7fd452a00640 1 -- 172.21.15.22:0/3462476062 shutdown_connections 2024-03-28T18:03:59.180 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:03:59.179+0000 7fd452a00640 1 -- 172.21.15.22:0/3462476062 wait complete. 2024-03-28T18:03:59.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:03:59 smithi022 ceph-mon[31985]: from='client.? 172.21.15.22:0/3116645869' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-28T18:03:59.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:03:59 smithi022 ceph-mon[31985]: from='client.? 172.21.15.22:0/3462476062' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-28T18:03:59.868 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:03:59 smithi134 ceph-mon[35037]: from='client.? 172.21.15.22:0/3116645869' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-28T18:03:59.868 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:03:59 smithi134 ceph-mon[35037]: from='client.? 172.21.15.22:0/3462476062' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-28T18:03:59.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:03:59 smithi079 ceph-mon[35312]: from='client.? 172.21.15.22:0/3116645869' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-28T18:03:59.933 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:03:59 smithi079 ceph-mon[35312]: from='client.? 172.21.15.22:0/3462476062' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-28T18:04:00.176 INFO:journalctl@ceph.mds.l.smithi134.stdout:Mar 28 18:03:59 smithi134 podman[64808]: 2024-03-28 18:03:59.954421265 +0000 UTC m=+0.160922050 container create 5c76242affe2b71f817aa6104d5c424e07d2a212adcbcf58087e9fa348ec11ad (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-l, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.build-date=20240319, ceph=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, io.buildah.version=1.33.5) 2024-03-28T18:04:00.512 INFO:journalctl@ceph.mds.l.smithi134.stdout:Mar 28 18:04:00 smithi134 podman[64808]: 2024-03-28 18:04:00.198840588 +0000 UTC m=+0.405341370 container init 5c76242affe2b71f817aa6104d5c424e07d2a212adcbcf58087e9fa348ec11ad (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-l, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240319, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2) 2024-03-28T18:04:00.512 INFO:journalctl@ceph.mds.l.smithi134.stdout:Mar 28 18:04:00 smithi134 podman[64808]: 2024-03-28 18:04:00.207919761 +0000 UTC m=+0.414420519 container start 5c76242affe2b71f817aa6104d5c424e07d2a212adcbcf58087e9fa348ec11ad (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-l, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240319, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, ceph=True, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30) 2024-03-28T18:04:00.512 INFO:journalctl@ceph.mds.l.smithi134.stdout:Mar 28 18:04:00 smithi134 ceph-mds[64834]: -- 172.21.15.134:0/2025411058 <== mon.1 v2:172.21.15.134:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x561472a1fa40 con 0x5614729f2800 2024-03-28T18:04:00.512 INFO:journalctl@ceph.mds.l.smithi134.stdout:Mar 28 18:04:00 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-l[64828]: starting mds.l at 2024-03-28T18:04:00.512 INFO:journalctl@ceph.mds.l.smithi134.stdout:Mar 28 18:04:00 smithi134 bash[64808]: 5c76242affe2b71f817aa6104d5c424e07d2a212adcbcf58087e9fa348ec11ad 2024-03-28T18:04:00.512 INFO:journalctl@ceph.mds.l.smithi134.stdout:Mar 28 18:04:00 smithi134 systemd[1]: Started Ceph mds.l for fd220748-ed2b-11ee-95d0-87774f69a715. 2024-03-28T18:04:00.513 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:04:00 smithi134 ceph-mon[35037]: pgmap v340: 1 pgs: 1 active+clean; 577 KiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-28T18:04:00.513 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:04:00 smithi134 ceph-mon[35037]: mds.? [v2:172.21.15.134:6832/3176125362,v1:172.21.15.134:6833/3176125362] up:boot 2024-03-28T18:04:00.513 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:04:00 smithi134 ceph-mon[35037]: fsmap 1 up:standby 2024-03-28T18:04:00.513 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:04:00 smithi134 ceph-mon[35037]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-03-28T18:04:00.513 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:04:00 smithi134 ceph-mon[35037]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T18:04:00.513 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:04:00 smithi134 ceph-mon[35037]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T18:04:00.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:04:00 smithi022 ceph-mon[31985]: pgmap v340: 1 pgs: 1 active+clean; 577 KiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-28T18:04:00.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:04:00 smithi022 ceph-mon[31985]: mds.? [v2:172.21.15.134:6832/3176125362,v1:172.21.15.134:6833/3176125362] up:boot 2024-03-28T18:04:00.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:04:00 smithi022 ceph-mon[31985]: fsmap 1 up:standby 2024-03-28T18:04:00.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:04:00 smithi022 ceph-mon[31985]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-03-28T18:04:00.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:04:00 smithi022 ceph-mon[31985]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T18:04:00.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:04:00 smithi022 ceph-mon[31985]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T18:04:00.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:04:00 smithi079 ceph-mon[35312]: pgmap v340: 1 pgs: 1 active+clean; 577 KiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-28T18:04:00.933 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:04:00 smithi079 ceph-mon[35312]: mds.? [v2:172.21.15.134:6832/3176125362,v1:172.21.15.134:6833/3176125362] up:boot 2024-03-28T18:04:00.933 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:04:00 smithi079 ceph-mon[35312]: fsmap 1 up:standby 2024-03-28T18:04:00.933 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:04:00 smithi079 ceph-mon[35312]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-03-28T18:04:00.933 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:04:00 smithi079 ceph-mon[35312]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T18:04:00.933 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:04:00 smithi079 ceph-mon[35312]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T18:04:01.200 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-28T18:04:01.432 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:01.432+0000 7f7652000640 1 -- 172.21.15.22:0/1911605101 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f764c1096b0 msgr2=0x7f764c10ff40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:04:01.432 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:01.432+0000 7f7652000640 1 --2- 172.21.15.22:0/1911605101 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f764c1096b0 0x7f764c10ff40 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f7644009ca0 tx=0x7f764402c100 comp rx=0 tx=0).stop 2024-03-28T18:04:01.432 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:01.432+0000 7f7652000640 1 -- 172.21.15.22:0/1911605101 shutdown_connections 2024-03-28T18:04:01.432 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:01.432+0000 7f7652000640 1 --2- 172.21.15.22:0/1911605101 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f764c1096b0 0x7f764c10ff40 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:04:01.432 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:01.432+0000 7f7652000640 1 --2- 172.21.15.22:0/1911605101 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f764c108cf0 0x7f764c109170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:04:01.432 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:01.432+0000 7f7652000640 1 --2- 172.21.15.22:0/1911605101 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f764c107b00 0x7f764c107f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:04:01.432 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:01.432+0000 7f7652000640 1 -- 172.21.15.22:0/1911605101 >> 172.21.15.22:0/1911605101 conn(0x7f764c076ab0 msgr2=0x7f764c078ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:04:01.432 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:01.432+0000 7f7652000640 1 -- 172.21.15.22:0/1911605101 shutdown_connections 2024-03-28T18:04:01.432 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:01.432+0000 7f7652000640 1 -- 172.21.15.22:0/1911605101 wait complete. 2024-03-28T18:04:01.434 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:01.433+0000 7f7652000640 1 Processor -- start 2024-03-28T18:04:01.435 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:01.433+0000 7f7652000640 1 -- start start 2024-03-28T18:04:01.435 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:01.433+0000 7f7652000640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f764c107b00 0x7f764c19dd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:04:01.435 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:01.433+0000 7f7652000640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f764c108cf0 0x7f764c19e260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:04:01.435 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:01.433+0000 7f7652000640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f764c1096b0 0x7f764c1a5290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:04:01.435 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:01.433+0000 7f7652000640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f764c113750 con 0x7f764c108cf0 2024-03-28T18:04:01.435 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:01.433+0000 7f7652000640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f764c1135d0 con 0x7f764c107b00 2024-03-28T18:04:01.435 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:01.433+0000 7f7652000640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f764c1138d0 con 0x7f764c1096b0 2024-03-28T18:04:01.435 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:01.434+0000 7f7650c00640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f764c107b00 0x7f764c19dd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:04:01.435 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:01.434+0000 7f7643e00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f764c108cf0 0x7f764c19e260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:04:01.435 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:01.434+0000 7f7650c00640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f764c107b00 0x7f764c19dd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.22:38958/0 (socket says 172.21.15.22:38958) 2024-03-28T18:04:01.435 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:01.434+0000 7f7650c00640 1 -- 172.21.15.22:0/4212368167 learned_addr learned my addr 172.21.15.22:0/4212368167 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-28T18:04:01.435 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:01.434+0000 7f7651600640 1 --2- 172.21.15.22:0/4212368167 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f764c1096b0 0x7f764c1a5290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:04:01.435 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:01.434+0000 7f7643e00640 1 -- 172.21.15.22:0/4212368167 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f764c1096b0 msgr2=0x7f764c1a5290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:04:01.435 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:01.434+0000 7f7643e00640 1 --2- 172.21.15.22:0/4212368167 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f764c1096b0 0x7f764c1a5290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:04:01.436 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:01.434+0000 7f7643e00640 1 -- 172.21.15.22:0/4212368167 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f764c107b00 msgr2=0x7f764c19dd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:04:01.436 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:01.434+0000 7f7643e00640 1 --2- 172.21.15.22:0/4212368167 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f764c107b00 0x7f764c19dd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:04:01.436 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:01.434+0000 7f7643e00640 1 -- 172.21.15.22:0/4212368167 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f764c1a5930 con 0x7f764c108cf0 2024-03-28T18:04:01.436 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:01.434+0000 7f7651600640 1 --2- 172.21.15.22:0/4212368167 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f764c1096b0 0x7f764c1a5290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T18:04:01.436 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:01.434+0000 7f7643e00640 1 --2- 172.21.15.22:0/4212368167 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f764c108cf0 0x7f764c19e260 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f763400e9f0 tx=0x7f763400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:04:01.436 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:01.435+0000 7f764a000640 1 -- 172.21.15.22:0/4212368167 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f763400ccc0 con 0x7f764c108cf0 2024-03-28T18:04:01.436 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:01.435+0000 7f764a000640 1 -- 172.21.15.22:0/4212368167 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f7634010900 con 0x7f764c108cf0 2024-03-28T18:04:01.436 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:01.435+0000 7f764a000640 1 -- 172.21.15.22:0/4212368167 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7634018960 con 0x7f764c108cf0 2024-03-28T18:04:01.436 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:01.435+0000 7f7652000640 1 -- 172.21.15.22:0/4212368167 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f764c1a5c20 con 0x7f764c108cf0 2024-03-28T18:04:01.436 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:01.435+0000 7f7652000640 1 -- 172.21.15.22:0/4212368167 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f764c1a61c0 con 0x7f764c108cf0 2024-03-28T18:04:01.436 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:01.436+0000 7f7652000640 1 -- 172.21.15.22:0/4212368167 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f764c10edc0 con 0x7f764c108cf0 2024-03-28T18:04:01.438 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:01.438+0000 7f764a000640 1 -- 172.21.15.22:0/4212368167 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 16) v1 ==== 137290+0+0 (secure 0 0 0) 0x7f7634010ab0 con 0x7f764c108cf0 2024-03-28T18:04:01.439 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:01.439+0000 7f764a000640 1 --2- 172.21.15.22:0/4212368167 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7f762c0a2880 0x7f762c0a4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:04:01.439 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:01.439+0000 7f764a000640 1 -- 172.21.15.22:0/4212368167 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f76340cea40 con 0x7f764c108cf0 2024-03-28T18:04:01.439 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:01.440+0000 7f7650c00640 1 --2- 172.21.15.22:0/4212368167 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7f762c0a2880 0x7f762c0a4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:04:01.440 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:01.440+0000 7f7650c00640 1 --2- 172.21.15.22:0/4212368167 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7f762c0a2880 0x7f762c0a4d40 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f764c19f1f0 tx=0x7f763c00a3b0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:04:01.443 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:01.443+0000 7f764a000640 1 -- 172.21.15.22:0/4212368167 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f763409bee0 con 0x7f764c108cf0 2024-03-28T18:04:01.692 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:01.692+0000 7f7652000640 1 -- 172.21.15.22:0/4212368167 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f764c02d070 con 0x7f764c108cf0 2024-03-28T18:04:01.692 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:01.693+0000 7f764a000640 1 -- 172.21.15.22:0/4212368167 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 2 v2) v1 ==== 93+0+1328 (secure 0 0 0) 0x7f763409b7e0 con 0x7f764c108cf0 2024-03-28T18:04:01.693 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-03-28T18:04:01.693 INFO:teuthology.orchestra.run.smithi022.stdout:{"epoch":2,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24412,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.134:6833/3176125362","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6832","nonce":3176125362},{"type":"v1","addr":"172.21.15.134:6833","nonce":3176125362}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2}],"filesystems":[]} 2024-03-28T18:04:01.693 INFO:teuthology.orchestra.run.smithi022.stderr:dumped fsmap epoch 2 2024-03-28T18:04:01.696 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:01.696+0000 7f7652000640 1 -- 172.21.15.22:0/4212368167 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7f762c0a2880 msgr2=0x7f762c0a4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:04:01.696 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:01.696+0000 7f7652000640 1 --2- 172.21.15.22:0/4212368167 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7f762c0a2880 0x7f762c0a4d40 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f764c19f1f0 tx=0x7f763c00a3b0 comp rx=0 tx=0).stop 2024-03-28T18:04:01.696 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:01.696+0000 7f7652000640 1 -- 172.21.15.22:0/4212368167 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f764c108cf0 msgr2=0x7f764c19e260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:04:01.696 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:01.696+0000 7f7652000640 1 --2- 172.21.15.22:0/4212368167 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f764c108cf0 0x7f764c19e260 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f763400e9f0 tx=0x7f763400eeb0 comp rx=0 tx=0).stop 2024-03-28T18:04:01.696 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:01.696+0000 7f7652000640 1 -- 172.21.15.22:0/4212368167 shutdown_connections 2024-03-28T18:04:01.696 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:01.696+0000 7f7652000640 1 --2- 172.21.15.22:0/4212368167 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7f762c0a2880 0x7f762c0a4d40 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:04:01.696 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:01.696+0000 7f7652000640 1 --2- 172.21.15.22:0/4212368167 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f764c1096b0 0x7f764c1a5290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:04:01.696 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:01.696+0000 7f7652000640 1 --2- 172.21.15.22:0/4212368167 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f764c108cf0 0x7f764c19e260 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:04:01.696 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:01.696+0000 7f7652000640 1 --2- 172.21.15.22:0/4212368167 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f764c107b00 0x7f764c19dd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:04:01.697 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:01.696+0000 7f7652000640 1 -- 172.21.15.22:0/4212368167 >> 172.21.15.22:0/4212368167 conn(0x7f764c076ab0 msgr2=0x7f764c10bf90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:04:01.697 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:01.696+0000 7f7652000640 1 -- 172.21.15.22:0/4212368167 shutdown_connections 2024-03-28T18:04:01.697 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:01.696+0000 7f7652000640 1 -- 172.21.15.22:0/4212368167 wait complete. 2024-03-28T18:04:01.712 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:04:01 smithi022 ceph-mon[31985]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T18:04:01.713 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:04:01 smithi022 ceph-mon[31985]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-28T18:04:01.713 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:04:01 smithi022 ceph-mon[31985]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-28T18:04:01.713 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:04:01 smithi022 ceph-mon[31985]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T18:04:01.713 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:04:01 smithi022 ceph-mon[31985]: Deploying daemon mds.k on smithi079 2024-03-28T18:04:01.713 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:04:01 smithi022 ceph-mon[31985]: pgmap v341: 1 pgs: 1 active+clean; 577 KiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-28T18:04:01.904 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:04:01 smithi079 ceph-mon[35312]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T18:04:01.904 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:04:01 smithi079 ceph-mon[35312]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-28T18:04:01.904 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:04:01 smithi079 ceph-mon[35312]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-28T18:04:01.904 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:04:01 smithi079 ceph-mon[35312]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T18:04:01.905 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:04:01 smithi079 ceph-mon[35312]: Deploying daemon mds.k on smithi079 2024-03-28T18:04:01.905 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:04:01 smithi079 ceph-mon[35312]: pgmap v341: 1 pgs: 1 active+clean; 577 KiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-28T18:04:01.925 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:04:01 smithi134 ceph-mon[35037]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T18:04:01.925 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:04:01 smithi134 ceph-mon[35037]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-28T18:04:01.925 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:04:01 smithi134 ceph-mon[35037]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-28T18:04:01.925 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:04:01 smithi134 ceph-mon[35037]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T18:04:01.925 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:04:01 smithi134 ceph-mon[35037]: Deploying daemon mds.k on smithi079 2024-03-28T18:04:01.925 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:04:01 smithi134 ceph-mon[35037]: pgmap v341: 1 pgs: 1 active+clean; 577 KiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-28T18:04:02.622 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:04:02 smithi079 ceph-mon[35312]: from='client.? 172.21.15.22:0/4212368167' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-28T18:04:02.925 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:04:02 smithi134 ceph-mon[35037]: from='client.? 172.21.15.22:0/4212368167' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-28T18:04:03.032 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:04:02 smithi022 ceph-mon[31985]: from='client.? 172.21.15.22:0/4212368167' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-28T18:04:03.717 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-28T18:04:03.793 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:04:03 smithi079 ceph-mon[35312]: pgmap v342: 1 pgs: 1 active+clean; 577 KiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-28T18:04:03.810 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:04:03 smithi022 ceph-mon[31985]: pgmap v342: 1 pgs: 1 active+clean; 577 KiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-28T18:04:03.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:04:03 smithi134 ceph-mon[35037]: pgmap v342: 1 pgs: 1 active+clean; 577 KiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-28T18:04:03.949 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:03.948+0000 7f24e0000640 1 -- 172.21.15.22:0/686604735 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f24d81073a0 msgr2=0x7f24d81097f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:04:03.949 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:03.948+0000 7f24e0000640 1 --2- 172.21.15.22:0/686604735 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f24d81073a0 0x7f24d81097f0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f24d0009ca0 tx=0x7f24d002c100 comp rx=0 tx=0).stop 2024-03-28T18:04:03.949 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:03.948+0000 7f24e0000640 1 -- 172.21.15.22:0/686604735 shutdown_connections 2024-03-28T18:04:03.949 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:03.948+0000 7f24e0000640 1 --2- 172.21.15.22:0/686604735 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f24d81073a0 0x7f24d81097f0 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:04:03.950 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:03.948+0000 7f24e0000640 1 --2- 172.21.15.22:0/686604735 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f24d8104a70 0x7f24d8106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:04:03.950 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:03.948+0000 7f24e0000640 1 --2- 172.21.15.22:0/686604735 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f24d8102140 0x7f24d8104530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:04:03.950 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:03.948+0000 7f24e0000640 1 -- 172.21.15.22:0/686604735 >> 172.21.15.22:0/686604735 conn(0x7f24d80fbb50 msgr2=0x7f24d80fdf70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:04:03.950 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:03.948+0000 7f24e0000640 1 -- 172.21.15.22:0/686604735 shutdown_connections 2024-03-28T18:04:03.950 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:03.948+0000 7f24e0000640 1 -- 172.21.15.22:0/686604735 wait complete. 2024-03-28T18:04:03.952 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:03.949+0000 7f24e0000640 1 Processor -- start 2024-03-28T18:04:03.953 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:03.949+0000 7f24e0000640 1 -- start start 2024-03-28T18:04:03.953 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:03.950+0000 7f24e0000640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f24d8102140 0x7f24d8199c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:04:03.953 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:03.950+0000 7f24e0000640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f24d8104a70 0x7f24d819a1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:04:03.953 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:03.950+0000 7f24e0000640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f24d81073a0 0x7f24d81a11e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:04:03.953 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:03.950+0000 7f24e0000640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24d810d3c0 con 0x7f24d8104a70 2024-03-28T18:04:03.953 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:03.950+0000 7f24e0000640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24d810d240 con 0x7f24d8102140 2024-03-28T18:04:03.953 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:03.950+0000 7f24e0000640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24d810d540 con 0x7f24d81073a0 2024-03-28T18:04:03.953 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:03.950+0000 7f24dcc00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f24d8104a70 0x7f24d819a1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:04:03.953 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:03.950+0000 7f24dcc00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f24d8104a70 0x7f24d819a1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:58464/0 (socket says 172.21.15.22:58464) 2024-03-28T18:04:03.953 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:03.950+0000 7f24dcc00640 1 -- 172.21.15.22:0/1071808593 learned_addr learned my addr 172.21.15.22:0/1071808593 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-28T18:04:03.953 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:03.950+0000 7f24dd600640 1 --2- 172.21.15.22:0/1071808593 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f24d8102140 0x7f24d8199c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:04:03.953 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:03.950+0000 7f24de000640 1 --2- 172.21.15.22:0/1071808593 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f24d81073a0 0x7f24d81a11e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:04:03.954 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:03.951+0000 7f24dcc00640 1 -- 172.21.15.22:0/1071808593 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f24d81073a0 msgr2=0x7f24d81a11e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:04:03.954 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:03.951+0000 7f24dcc00640 1 --2- 172.21.15.22:0/1071808593 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f24d81073a0 0x7f24d81a11e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:04:03.954 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:03.951+0000 7f24dcc00640 1 -- 172.21.15.22:0/1071808593 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f24d8102140 msgr2=0x7f24d8199c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:04:03.954 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:03.951+0000 7f24dcc00640 1 --2- 172.21.15.22:0/1071808593 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f24d8102140 0x7f24d8199c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:04:03.954 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:03.951+0000 7f24dcc00640 1 -- 172.21.15.22:0/1071808593 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f24d81a1880 con 0x7f24d8104a70 2024-03-28T18:04:03.954 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:03.951+0000 7f24dcc00640 1 --2- 172.21.15.22:0/1071808593 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f24d8104a70 0x7f24d819a1b0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f24c400d740 tx=0x7f24c400dc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:04:03.954 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:03.951+0000 7f24d6000640 1 -- 172.21.15.22:0/1071808593 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24c4017070 con 0x7f24d8104a70 2024-03-28T18:04:03.954 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:03.951+0000 7f24e0000640 1 -- 172.21.15.22:0/1071808593 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f24d81a1b70 con 0x7f24d8104a70 2024-03-28T18:04:03.954 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:03.951+0000 7f24d6000640 1 -- 172.21.15.22:0/1071808593 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f24c400de50 con 0x7f24d8104a70 2024-03-28T18:04:03.954 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:03.951+0000 7f24d6000640 1 -- 172.21.15.22:0/1071808593 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24c4004e00 con 0x7f24d8104a70 2024-03-28T18:04:03.954 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:03.951+0000 7f24e0000640 1 -- 172.21.15.22:0/1071808593 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f24d81a2070 con 0x7f24d8104a70 2024-03-28T18:04:03.955 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:03.953+0000 7f24e0000640 1 -- 172.21.15.22:0/1071808593 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f24d8108e90 con 0x7f24d8104a70 2024-03-28T18:04:03.956 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:03.955+0000 7f24d6000640 1 -- 172.21.15.22:0/1071808593 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 16) v1 ==== 137290+0+0 (secure 0 0 0) 0x7f24c4004fa0 con 0x7f24d8104a70 2024-03-28T18:04:03.957 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:03.956+0000 7f24d6000640 1 --2- 172.21.15.22:0/1071808593 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7f24c00a2800 0x7f24c00a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:04:03.958 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:03.957+0000 7f24d6000640 1 -- 172.21.15.22:0/1071808593 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f24c40ce150 con 0x7f24d8104a70 2024-03-28T18:04:03.958 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:03.957+0000 7f24dd600640 1 --2- 172.21.15.22:0/1071808593 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7f24c00a2800 0x7f24c00a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:04:03.958 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:03.957+0000 7f24dd600640 1 --2- 172.21.15.22:0/1071808593 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7f24c00a2800 0x7f24c00a4cc0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f24d819b140 tx=0x7f24cc008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:04:03.962 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:03.961+0000 7f24d6000640 1 -- 172.21.15.22:0/1071808593 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f24c409b5f0 con 0x7f24d8104a70 2024-03-28T18:04:04.123 INFO:journalctl@ceph.mds.k.smithi079.stdout:Mar 28 18:04:03 smithi079 podman[64302]: 2024-03-28 18:04:03.891914607 +0000 UTC m=+0.154146636 container create 0f075f827bbd7430ae43de494e20673f89d43d29b773751d23dfc0bf83e75772 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-k, ceph=True, org.label-schema.build-date=20240319, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, org.label-schema.vendor=CentOS, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5) 2024-03-28T18:04:04.207 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:04.206+0000 7f24e0000640 1 -- 172.21.15.22:0/1071808593 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f24d80622d0 con 0x7f24d8104a70 2024-03-28T18:04:04.208 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:04.207+0000 7f24d6000640 1 -- 172.21.15.22:0/1071808593 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 2 v2) v1 ==== 93+0+1328 (secure 0 0 0) 0x7f24c409aef0 con 0x7f24d8104a70 2024-03-28T18:04:04.208 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-03-28T18:04:04.208 INFO:teuthology.orchestra.run.smithi022.stdout:{"epoch":2,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24412,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.134:6833/3176125362","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6832","nonce":3176125362},{"type":"v1","addr":"172.21.15.134:6833","nonce":3176125362}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2}],"filesystems":[]} 2024-03-28T18:04:04.209 INFO:teuthology.orchestra.run.smithi022.stderr:dumped fsmap epoch 2 2024-03-28T18:04:04.212 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:04.210+0000 7f24e0000640 1 -- 172.21.15.22:0/1071808593 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7f24c00a2800 msgr2=0x7f24c00a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:04:04.212 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:04.210+0000 7f24e0000640 1 --2- 172.21.15.22:0/1071808593 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7f24c00a2800 0x7f24c00a4cc0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f24d819b140 tx=0x7f24cc008040 comp rx=0 tx=0).stop 2024-03-28T18:04:04.212 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:04.210+0000 7f24e0000640 1 -- 172.21.15.22:0/1071808593 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f24d8104a70 msgr2=0x7f24d819a1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:04:04.212 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:04.210+0000 7f24e0000640 1 --2- 172.21.15.22:0/1071808593 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f24d8104a70 0x7f24d819a1b0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f24c400d740 tx=0x7f24c400dc00 comp rx=0 tx=0).stop 2024-03-28T18:04:04.212 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:04.211+0000 7f24e0000640 1 -- 172.21.15.22:0/1071808593 shutdown_connections 2024-03-28T18:04:04.212 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:04.211+0000 7f24e0000640 1 --2- 172.21.15.22:0/1071808593 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7f24c00a2800 0x7f24c00a4cc0 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:04:04.212 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:04.211+0000 7f24e0000640 1 --2- 172.21.15.22:0/1071808593 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f24d81073a0 0x7f24d81a11e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:04:04.212 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:04.211+0000 7f24e0000640 1 --2- 172.21.15.22:0/1071808593 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f24d8104a70 0x7f24d819a1b0 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:04:04.212 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:04.211+0000 7f24e0000640 1 --2- 172.21.15.22:0/1071808593 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f24d8102140 0x7f24d8199c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:04:04.212 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:04.211+0000 7f24e0000640 1 -- 172.21.15.22:0/1071808593 >> 172.21.15.22:0/1071808593 conn(0x7f24d80fbb50 msgr2=0x7f24d8101860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:04:04.212 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:04.211+0000 7f24e0000640 1 -- 172.21.15.22:0/1071808593 shutdown_connections 2024-03-28T18:04:04.212 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:04.211+0000 7f24e0000640 1 -- 172.21.15.22:0/1071808593 wait complete. 2024-03-28T18:04:04.563 INFO:journalctl@ceph.mds.k.smithi079.stdout:Mar 28 18:04:04 smithi079 podman[64302]: 2024-03-28 18:04:04.212080451 +0000 UTC m=+0.474312487 container init 0f075f827bbd7430ae43de494e20673f89d43d29b773751d23dfc0bf83e75772 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-k, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.build-date=20240319, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, CEPH_POINT_RELEASE=) 2024-03-28T18:04:04.563 INFO:journalctl@ceph.mds.k.smithi079.stdout:Mar 28 18:04:04 smithi079 podman[64302]: 2024-03-28 18:04:04.222391362 +0000 UTC m=+0.484623361 container start 0f075f827bbd7430ae43de494e20673f89d43d29b773751d23dfc0bf83e75772 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-k, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.schema-version=1.0, ceph=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.build-date=20240319, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-28T18:04:04.563 INFO:journalctl@ceph.mds.k.smithi079.stdout:Mar 28 18:04:04 smithi079 ceph-mds[64328]: -- 172.21.15.79:0/2427854405 <== mon.2 v2:172.21.15.79:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55f4f9ab3a40 con 0x55f4f9a86c00 2024-03-28T18:04:04.563 INFO:journalctl@ceph.mds.k.smithi079.stdout:Mar 28 18:04:04 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-k[64324]: starting mds.k at 2024-03-28T18:04:04.563 INFO:journalctl@ceph.mds.k.smithi079.stdout:Mar 28 18:04:04 smithi079 bash[64302]: 0f075f827bbd7430ae43de494e20673f89d43d29b773751d23dfc0bf83e75772 2024-03-28T18:04:04.563 INFO:journalctl@ceph.mds.k.smithi079.stdout:Mar 28 18:04:04 smithi079 systemd[1]: Started Ceph mds.k for fd220748-ed2b-11ee-95d0-87774f69a715. 2024-03-28T18:04:04.925 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:04:04 smithi134 ceph-mon[35037]: from='client.? 172.21.15.22:0/1071808593' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-28T18:04:04.925 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:04:04 smithi134 ceph-mon[35037]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T18:04:04.925 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:04:04 smithi134 ceph-mon[35037]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T18:04:04.925 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:04:04 smithi134 ceph-mon[35037]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T18:04:04.925 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:04:04 smithi134 ceph-mon[35037]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-28T18:04:04.925 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:04:04 smithi134 ceph-mon[35037]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-28T18:04:04.925 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:04:04 smithi134 ceph-mon[35037]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T18:04:04.925 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:04:04 smithi134 ceph-mon[35037]: Deploying daemon mds.j on smithi022 2024-03-28T18:04:04.933 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:04:04 smithi079 ceph-mon[35312]: from='client.? 172.21.15.22:0/1071808593' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-28T18:04:04.933 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:04:04 smithi079 ceph-mon[35312]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T18:04:04.933 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:04:04 smithi079 ceph-mon[35312]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T18:04:04.933 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:04:04 smithi079 ceph-mon[35312]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T18:04:04.933 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:04:04 smithi079 ceph-mon[35312]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-28T18:04:04.933 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:04:04 smithi079 ceph-mon[35312]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-28T18:04:04.933 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:04:04 smithi079 ceph-mon[35312]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T18:04:04.933 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:04:04 smithi079 ceph-mon[35312]: Deploying daemon mds.j on smithi022 2024-03-28T18:04:05.032 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:04:04 smithi022 ceph-mon[31985]: from='client.? 172.21.15.22:0/1071808593' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-28T18:04:05.032 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:04:04 smithi022 ceph-mon[31985]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T18:04:05.032 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:04:04 smithi022 ceph-mon[31985]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T18:04:05.032 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:04:04 smithi022 ceph-mon[31985]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T18:04:05.032 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:04:04 smithi022 ceph-mon[31985]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-28T18:04:05.032 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:04:04 smithi022 ceph-mon[31985]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-28T18:04:05.032 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:04:04 smithi022 ceph-mon[31985]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T18:04:05.032 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:04:04 smithi022 ceph-mon[31985]: Deploying daemon mds.j on smithi022 2024-03-28T18:04:05.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:04:05 smithi022 ceph-mon[31985]: mds.? [v2:172.21.15.79:6832/856896997,v1:172.21.15.79:6833/856896997] up:boot 2024-03-28T18:04:05.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:04:05 smithi022 ceph-mon[31985]: fsmap 2 up:standby 2024-03-28T18:04:05.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:04:05 smithi022 ceph-mon[31985]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-03-28T18:04:05.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:04:05 smithi022 ceph-mon[31985]: pgmap v343: 1 pgs: 1 active+clean; 577 KiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-28T18:04:05.925 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:04:05 smithi134 ceph-mon[35037]: mds.? [v2:172.21.15.79:6832/856896997,v1:172.21.15.79:6833/856896997] up:boot 2024-03-28T18:04:05.925 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:04:05 smithi134 ceph-mon[35037]: fsmap 2 up:standby 2024-03-28T18:04:05.925 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:04:05 smithi134 ceph-mon[35037]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-03-28T18:04:05.925 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:04:05 smithi134 ceph-mon[35037]: pgmap v343: 1 pgs: 1 active+clean; 577 KiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-28T18:04:05.933 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:04:05 smithi079 ceph-mon[35312]: mds.? [v2:172.21.15.79:6832/856896997,v1:172.21.15.79:6833/856896997] up:boot 2024-03-28T18:04:05.933 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:04:05 smithi079 ceph-mon[35312]: fsmap 2 up:standby 2024-03-28T18:04:05.933 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:04:05 smithi079 ceph-mon[35312]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-03-28T18:04:05.933 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:04:05 smithi079 ceph-mon[35312]: pgmap v343: 1 pgs: 1 active+clean; 577 KiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-28T18:04:06.231 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-28T18:04:06.461 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:06.460+0000 7f45f4a00640 1 -- 172.21.15.22:0/3631342379 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f45f01016c0 msgr2=0x7f45f0101ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:04:06.461 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:06.460+0000 7f45f4a00640 1 --2- 172.21.15.22:0/3631342379 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f45f01016c0 0x7f45f0101ae0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f45e0009900 tx=0x7f45e002bc30 comp rx=0 tx=0).stop 2024-03-28T18:04:06.461 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:06.460+0000 7f45f4a00640 1 -- 172.21.15.22:0/3631342379 shutdown_connections 2024-03-28T18:04:06.462 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:06.460+0000 7f45f4a00640 1 --2- 172.21.15.22:0/3631342379 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f45f0103270 0x7f45f0109b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:04:06.462 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:06.460+0000 7f45f4a00640 1 --2- 172.21.15.22:0/3631342379 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f45f01028b0 0x7f45f0102d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:04:06.462 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:06.460+0000 7f45f4a00640 1 --2- 172.21.15.22:0/3631342379 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f45f01016c0 0x7f45f0101ae0 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:04:06.462 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:06.460+0000 7f45f4a00640 1 -- 172.21.15.22:0/3631342379 >> 172.21.15.22:0/3631342379 conn(0x7f45f00fcc50 msgr2=0x7f45f00ff070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:04:06.462 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:06.460+0000 7f45f4a00640 1 -- 172.21.15.22:0/3631342379 shutdown_connections 2024-03-28T18:04:06.462 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:06.460+0000 7f45f4a00640 1 -- 172.21.15.22:0/3631342379 wait complete. 2024-03-28T18:04:06.464 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:06.461+0000 7f45f4a00640 1 Processor -- start 2024-03-28T18:04:06.464 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:06.461+0000 7f45f4a00640 1 -- start start 2024-03-28T18:04:06.464 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:06.462+0000 7f45f4a00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f45f01016c0 0x7f45f01979c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:04:06.464 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:06.462+0000 7f45f4a00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f45f01028b0 0x7f45f0197f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:04:06.465 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:06.462+0000 7f45f4a00640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f45f0103270 0x7f45f019ef70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:04:06.465 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:06.462+0000 7f45f4a00640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45f010d370 con 0x7f45f01028b0 2024-03-28T18:04:06.465 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:06.462+0000 7f45f4a00640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45f010d1f0 con 0x7f45f0103270 2024-03-28T18:04:06.465 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:06.462+0000 7f45f4a00640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45f010d4f0 con 0x7f45f01016c0 2024-03-28T18:04:06.465 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:06.462+0000 7f45e5400640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f45f01028b0 0x7f45f0197f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:04:06.465 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:06.462+0000 7f45e5400640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f45f01028b0 0x7f45f0197f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:58478/0 (socket says 172.21.15.22:58478) 2024-03-28T18:04:06.465 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:06.462+0000 7f45e5400640 1 -- 172.21.15.22:0/418948957 learned_addr learned my addr 172.21.15.22:0/418948957 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-28T18:04:06.465 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:06.462+0000 7f45ede00640 1 --2- 172.21.15.22:0/418948957 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f45f01016c0 0x7f45f01979c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:04:06.465 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:06.462+0000 7f45ee800640 1 --2- 172.21.15.22:0/418948957 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f45f0103270 0x7f45f019ef70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:04:06.465 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:06.462+0000 7f45e5400640 1 -- 172.21.15.22:0/418948957 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f45f01016c0 msgr2=0x7f45f01979c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:04:06.465 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:06.462+0000 7f45e5400640 1 --2- 172.21.15.22:0/418948957 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f45f01016c0 0x7f45f01979c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:04:06.465 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:06.462+0000 7f45e5400640 1 -- 172.21.15.22:0/418948957 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f45f0103270 msgr2=0x7f45f019ef70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:04:06.465 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:06.462+0000 7f45e5400640 1 --2- 172.21.15.22:0/418948957 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f45f0103270 0x7f45f019ef70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:04:06.465 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:06.462+0000 7f45e5400640 1 -- 172.21.15.22:0/418948957 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f45f019f5e0 con 0x7f45f01028b0 2024-03-28T18:04:06.466 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:06.462+0000 7f45ee800640 1 --2- 172.21.15.22:0/418948957 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f45f0103270 0x7f45f019ef70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T18:04:06.466 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:06.463+0000 7f45e5400640 1 --2- 172.21.15.22:0/418948957 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f45f01028b0 0x7f45f0197f00 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f45d800d950 tx=0x7f45d800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:04:06.466 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:06.463+0000 7f45e7400640 1 -- 172.21.15.22:0/418948957 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45d8017070 con 0x7f45f01028b0 2024-03-28T18:04:06.466 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:06.463+0000 7f45e7400640 1 -- 172.21.15.22:0/418948957 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f45d8009c30 con 0x7f45f01028b0 2024-03-28T18:04:06.466 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:06.463+0000 7f45f4a00640 1 -- 172.21.15.22:0/418948957 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f45f019f8d0 con 0x7f45f01028b0 2024-03-28T18:04:06.466 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:06.463+0000 7f45e7400640 1 -- 172.21.15.22:0/418948957 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45d800f6b0 con 0x7f45f01028b0 2024-03-28T18:04:06.466 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:06.463+0000 7f45f4a00640 1 -- 172.21.15.22:0/418948957 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f45f019fe20 con 0x7f45f01028b0 2024-03-28T18:04:06.466 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:06.464+0000 7f45f4a00640 1 -- 172.21.15.22:0/418948957 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f45f01089e0 con 0x7f45f01028b0 2024-03-28T18:04:06.467 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:06.466+0000 7f45e7400640 1 -- 172.21.15.22:0/418948957 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 16) v1 ==== 137290+0+0 (secure 0 0 0) 0x7f45d8010a50 con 0x7f45f01028b0 2024-03-28T18:04:06.469 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:06.468+0000 7f45e7400640 1 --2- 172.21.15.22:0/418948957 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7f45d40a2800 0x7f45d40a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:04:06.469 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:06.468+0000 7f45e7400640 1 -- 172.21.15.22:0/418948957 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f45d80ce3a0 con 0x7f45f01028b0 2024-03-28T18:04:06.469 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:06.468+0000 7f45ede00640 1 --2- 172.21.15.22:0/418948957 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7f45d40a2800 0x7f45d40a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:04:06.469 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:06.469+0000 7f45ede00640 1 --2- 172.21.15.22:0/418948957 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7f45d40a2800 0x7f45d40a4cc0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f45e002c170 tx=0x7f45e0005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:04:06.473 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:06.472+0000 7f45e7400640 1 -- 172.21.15.22:0/418948957 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f45d8009de0 con 0x7f45f01028b0 2024-03-28T18:04:06.736 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:06.736+0000 7f45f4a00640 1 -- 172.21.15.22:0/418948957 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f45f002d070 con 0x7f45f01028b0 2024-03-28T18:04:06.738 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:06.737+0000 7f45e7400640 1 -- 172.21.15.22:0/418948957 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 3 v3) v1 ==== 93+0+2116 (secure 0 0 0) 0x7f45d809b840 con 0x7f45f01028b0 2024-03-28T18:04:06.739 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-03-28T18:04:06.739 INFO:teuthology.orchestra.run.smithi022.stdout:{"epoch":3,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24412,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.134:6833/3176125362","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6832","nonce":3176125362},{"type":"v1","addr":"172.21.15.134:6833","nonce":3176125362}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":34442,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.79:6833/856896997","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6832","nonce":856896997},{"type":"v1","addr":"172.21.15.79:6833","nonce":856896997}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[]} 2024-03-28T18:04:06.739 INFO:teuthology.orchestra.run.smithi022.stderr:dumped fsmap epoch 3 2024-03-28T18:04:06.742 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:06.740+0000 7f45f4a00640 1 -- 172.21.15.22:0/418948957 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7f45d40a2800 msgr2=0x7f45d40a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:04:06.742 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:06.740+0000 7f45f4a00640 1 --2- 172.21.15.22:0/418948957 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7f45d40a2800 0x7f45d40a4cc0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f45e002c170 tx=0x7f45e0005fd0 comp rx=0 tx=0).stop 2024-03-28T18:04:06.742 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:06.740+0000 7f45f4a00640 1 -- 172.21.15.22:0/418948957 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f45f01028b0 msgr2=0x7f45f0197f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:04:06.742 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:06.740+0000 7f45f4a00640 1 --2- 172.21.15.22:0/418948957 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f45f01028b0 0x7f45f0197f00 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f45d800d950 tx=0x7f45d800de10 comp rx=0 tx=0).stop 2024-03-28T18:04:06.742 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:06.740+0000 7f45f4a00640 1 -- 172.21.15.22:0/418948957 shutdown_connections 2024-03-28T18:04:06.742 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:06.740+0000 7f45f4a00640 1 --2- 172.21.15.22:0/418948957 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7f45d40a2800 0x7f45d40a4cc0 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:04:06.742 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:06.740+0000 7f45f4a00640 1 --2- 172.21.15.22:0/418948957 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f45f0103270 0x7f45f019ef70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:04:06.742 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:06.740+0000 7f45f4a00640 1 --2- 172.21.15.22:0/418948957 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f45f01028b0 0x7f45f0197f00 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:04:06.742 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:06.740+0000 7f45f4a00640 1 --2- 172.21.15.22:0/418948957 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f45f01016c0 0x7f45f01979c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:04:06.742 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:06.740+0000 7f45f4a00640 1 -- 172.21.15.22:0/418948957 >> 172.21.15.22:0/418948957 conn(0x7f45f00fcc50 msgr2=0x7f45f0105bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:04:06.742 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:06.741+0000 7f45f4a00640 1 -- 172.21.15.22:0/418948957 shutdown_connections 2024-03-28T18:04:06.742 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:06.741+0000 7f45f4a00640 1 -- 172.21.15.22:0/418948957 wait complete. 2024-03-28T18:04:07.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:04:07 smithi134 ceph-mon[35037]: pgmap v344: 1 pgs: 1 active+clean; 577 KiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-28T18:04:07.925 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:04:07 smithi134 ceph-mon[35037]: from='client.? 172.21.15.22:0/418948957' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-28T18:04:07.930 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:04:07 smithi022 ceph-mon[31985]: pgmap v344: 1 pgs: 1 active+clean; 577 KiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-28T18:04:07.930 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:04:07 smithi022 ceph-mon[31985]: from='client.? 172.21.15.22:0/418948957' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-28T18:04:07.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:04:07 smithi079 ceph-mon[35312]: pgmap v344: 1 pgs: 1 active+clean; 577 KiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-28T18:04:07.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:04:07 smithi079 ceph-mon[35312]: from='client.? 172.21.15.22:0/418948957' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-28T18:04:08.206 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:04:07 smithi022 podman[72973]: 2024-03-28 18:04:07.950797237 +0000 UTC m=+0.151880885 container create 43ead9efefabdd2ee634528651fdc921807a64f9b114b59fe1378b9d713e6a84 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j, org.label-schema.build-date=20240319, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_CLEAN=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.vendor=CentOS, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD) 2024-03-28T18:04:08.206 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:04:08 smithi022 podman[72973]: 2024-03-28 18:04:08.204997544 +0000 UTC m=+0.406081195 container init 43ead9efefabdd2ee634528651fdc921807a64f9b114b59fe1378b9d713e6a84 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j, GIT_CLEAN=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.license=GPLv2, org.label-schema.build-date=20240319, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0) 2024-03-28T18:04:08.532 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:04:08 smithi022 podman[72973]: 2024-03-28 18:04:08.217311753 +0000 UTC m=+0.418395370 container start 43ead9efefabdd2ee634528651fdc921807a64f9b114b59fe1378b9d713e6a84 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.build-date=20240319, org.label-schema.schema-version=1.0, ceph=True, CEPH_POINT_RELEASE=) 2024-03-28T18:04:08.533 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:04:08 smithi022 ceph-mds[72992]: -- 172.21.15.22:0/924936847 <== mon.0 v2:172.21.15.22:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x555784cada40 con 0x555784c80c00 2024-03-28T18:04:08.533 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:04:08 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[72988]: starting mds.j at 2024-03-28T18:04:08.533 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:04:08 smithi022 bash[72973]: 43ead9efefabdd2ee634528651fdc921807a64f9b114b59fe1378b9d713e6a84 2024-03-28T18:04:08.533 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:04:08 smithi022 systemd[1]: Started Ceph mds.j for fd220748-ed2b-11ee-95d0-87774f69a715. 2024-03-28T18:04:08.763 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-28T18:04:08.977 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:08.976+0000 7fc6cc400640 1 -- 172.21.15.22:0/3713550655 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc6c4076060 msgr2=0x7fc6c4076480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:04:08.978 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:08.976+0000 7fc6cc400640 1 --2- 172.21.15.22:0/3713550655 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc6c4076060 0x7fc6c4076480 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fc6b8009900 tx=0x7fc6b802bc30 comp rx=0 tx=0).stop 2024-03-28T18:04:08.978 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:08.976+0000 7fc6cc400640 1 -- 172.21.15.22:0/3713550655 shutdown_connections 2024-03-28T18:04:08.978 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:08.976+0000 7fc6cc400640 1 --2- 172.21.15.22:0/3713550655 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc6c4077c40 0x7fc6c40784e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:04:08.978 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:08.976+0000 7fc6cc400640 1 --2- 172.21.15.22:0/3713550655 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc6c40772a0 0x7fc6c4077700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:04:08.978 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:08.976+0000 7fc6cc400640 1 --2- 172.21.15.22:0/3713550655 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc6c4076060 0x7fc6c4076480 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:04:08.978 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:08.976+0000 7fc6cc400640 1 -- 172.21.15.22:0/3713550655 >> 172.21.15.22:0/3713550655 conn(0x7fc6c40fee50 msgr2=0x7fc6c4101290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:04:08.978 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:08.976+0000 7fc6cc400640 1 -- 172.21.15.22:0/3713550655 shutdown_connections 2024-03-28T18:04:08.978 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:08.976+0000 7fc6cc400640 1 -- 172.21.15.22:0/3713550655 wait complete. 2024-03-28T18:04:08.980 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:08.977+0000 7fc6cc400640 1 Processor -- start 2024-03-28T18:04:08.980 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:08.977+0000 7fc6cc400640 1 -- start start 2024-03-28T18:04:08.980 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:08.978+0000 7fc6cc400640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc6c4076060 0x7fc6c419df90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:04:08.980 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:08.978+0000 7fc6cc400640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc6c40772a0 0x7fc6c419e4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:04:08.980 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:08.978+0000 7fc6cc400640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc6c4077c40 0x7fc6c41a5540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:04:08.981 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:08.978+0000 7fc6cc400640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6c4113980 con 0x7fc6c40772a0 2024-03-28T18:04:08.981 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:08.978+0000 7fc6cc400640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6c4113800 con 0x7fc6c4077c40 2024-03-28T18:04:08.981 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:08.978+0000 7fc6cc400640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6c4113b00 con 0x7fc6c4076060 2024-03-28T18:04:08.981 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:08.978+0000 7fc6c9000640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc6c40772a0 0x7fc6c419e4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:04:08.981 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:08.978+0000 7fc6c9000640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc6c40772a0 0x7fc6c419e4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:48218/0 (socket says 172.21.15.22:48218) 2024-03-28T18:04:08.981 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:08.978+0000 7fc6c9000640 1 -- 172.21.15.22:0/3538282189 learned_addr learned my addr 172.21.15.22:0/3538282189 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-28T18:04:08.981 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:08.978+0000 7fc6c9a00640 1 --2- 172.21.15.22:0/3538282189 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc6c4076060 0x7fc6c419df90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:04:08.981 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:08.978+0000 7fc6ca400640 1 --2- 172.21.15.22:0/3538282189 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc6c4077c40 0x7fc6c41a5540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:04:08.981 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:08.979+0000 7fc6c9000640 1 -- 172.21.15.22:0/3538282189 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc6c4076060 msgr2=0x7fc6c419df90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:04:08.981 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:08.979+0000 7fc6c9000640 1 --2- 172.21.15.22:0/3538282189 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc6c4076060 0x7fc6c419df90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:04:08.981 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:08.979+0000 7fc6c9000640 1 -- 172.21.15.22:0/3538282189 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc6c4077c40 msgr2=0x7fc6c41a5540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:04:08.981 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:08.979+0000 7fc6c9000640 1 --2- 172.21.15.22:0/3538282189 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc6c4077c40 0x7fc6c41a5540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:04:08.981 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:08.979+0000 7fc6c9000640 1 -- 172.21.15.22:0/3538282189 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc6c41a5bb0 con 0x7fc6c40772a0 2024-03-28T18:04:08.982 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:08.979+0000 7fc6c9000640 1 --2- 172.21.15.22:0/3538282189 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc6c40772a0 0x7fc6c419e4d0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fc6b000d770 tx=0x7fc6b000dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:04:08.982 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:08.979+0000 7fc6c2000640 1 -- 172.21.15.22:0/3538282189 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6b0017070 con 0x7fc6c40772a0 2024-03-28T18:04:08.982 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:08.979+0000 7fc6c2000640 1 -- 172.21.15.22:0/3538282189 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7fc6b00054f0 con 0x7fc6c40772a0 2024-03-28T18:04:08.982 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:08.979+0000 7fc6cc400640 1 -- 172.21.15.22:0/3538282189 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc6c41a5ea0 con 0x7fc6c40772a0 2024-03-28T18:04:08.982 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:08.979+0000 7fc6c2000640 1 -- 172.21.15.22:0/3538282189 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6b0004d60 con 0x7fc6c40772a0 2024-03-28T18:04:08.982 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:08.979+0000 7fc6cc400640 1 -- 172.21.15.22:0/3538282189 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc6c41a6310 con 0x7fc6c40772a0 2024-03-28T18:04:08.982 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:08.981+0000 7fc6cc400640 1 -- 172.21.15.22:0/3538282189 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc6c4110390 con 0x7fc6c40772a0 2024-03-28T18:04:08.983 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:08.983+0000 7fc6c2000640 1 -- 172.21.15.22:0/3538282189 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 16) v1 ==== 137290+0+0 (secure 0 0 0) 0x7fc6b0004f00 con 0x7fc6c40772a0 2024-03-28T18:04:08.985 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:08.984+0000 7fc6c2000640 1 --2- 172.21.15.22:0/3538282189 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7fc6ac0a2800 0x7fc6ac0a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:04:08.985 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:08.984+0000 7fc6c2000640 1 -- 172.21.15.22:0/3538282189 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 7127+0+0 (secure 0 0 0) 0x7fc6b00ce180 con 0x7fc6c40772a0 2024-03-28T18:04:08.985 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:08.984+0000 7fc6c9a00640 1 --2- 172.21.15.22:0/3538282189 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7fc6ac0a2800 0x7fc6ac0a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:04:08.986 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:08.985+0000 7fc6c9a00640 1 --2- 172.21.15.22:0/3538282189 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7fc6ac0a2800 0x7fc6ac0a4cc0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fc6b8006010 tx=0x7fc6b8000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:04:08.990 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:08.989+0000 7fc6c2000640 1 -- 172.21.15.22:0/3538282189 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fc6b009b620 con 0x7fc6c40772a0 2024-03-28T18:04:09.236 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:09.235+0000 7fc6cc400640 1 -- 172.21.15.22:0/3538282189 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc6c40622a0 con 0x7fc6c40772a0 2024-03-28T18:04:09.237 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:09.236+0000 7fc6c2000640 1 -- 172.21.15.22:0/3538282189 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 4 v4) v1 ==== 93+0+2907 (secure 0 0 0) 0x7fc6b009af20 con 0x7fc6c40772a0 2024-03-28T18:04:09.237 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-03-28T18:04:09.237 INFO:teuthology.orchestra.run.smithi022.stdout:{"epoch":4,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14544,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6835/3718238950","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6834","nonce":3718238950},{"type":"v1","addr":"172.21.15.22:6835","nonce":3718238950}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":24412,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.134:6833/3176125362","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6832","nonce":3176125362},{"type":"v1","addr":"172.21.15.134:6833","nonce":3176125362}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":34442,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.79:6833/856896997","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6832","nonce":856896997},{"type":"v1","addr":"172.21.15.79:6833","nonce":856896997}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[]} 2024-03-28T18:04:09.238 INFO:teuthology.orchestra.run.smithi022.stderr:dumped fsmap epoch 4 2024-03-28T18:04:09.240 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:09.239+0000 7fc6cc400640 1 -- 172.21.15.22:0/3538282189 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7fc6ac0a2800 msgr2=0x7fc6ac0a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:04:09.240 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:09.239+0000 7fc6cc400640 1 --2- 172.21.15.22:0/3538282189 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7fc6ac0a2800 0x7fc6ac0a4cc0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fc6b8006010 tx=0x7fc6b8000bc0 comp rx=0 tx=0).stop 2024-03-28T18:04:09.240 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:09.239+0000 7fc6cc400640 1 -- 172.21.15.22:0/3538282189 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc6c40772a0 msgr2=0x7fc6c419e4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:04:09.240 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:09.239+0000 7fc6cc400640 1 --2- 172.21.15.22:0/3538282189 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc6c40772a0 0x7fc6c419e4d0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fc6b000d770 tx=0x7fc6b000dc30 comp rx=0 tx=0).stop 2024-03-28T18:04:09.240 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:09.239+0000 7fc6cc400640 1 -- 172.21.15.22:0/3538282189 shutdown_connections 2024-03-28T18:04:09.240 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:09.239+0000 7fc6cc400640 1 --2- 172.21.15.22:0/3538282189 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7fc6ac0a2800 0x7fc6ac0a4cc0 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:04:09.240 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:09.239+0000 7fc6cc400640 1 --2- 172.21.15.22:0/3538282189 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc6c4077c40 0x7fc6c41a5540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:04:09.240 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:09.239+0000 7fc6cc400640 1 --2- 172.21.15.22:0/3538282189 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc6c40772a0 0x7fc6c419e4d0 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:04:09.240 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:09.239+0000 7fc6cc400640 1 --2- 172.21.15.22:0/3538282189 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc6c4076060 0x7fc6c419df90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:04:09.240 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:09.239+0000 7fc6cc400640 1 -- 172.21.15.22:0/3538282189 >> 172.21.15.22:0/3538282189 conn(0x7fc6c40fee50 msgr2=0x7fc6c4100520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:04:09.241 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:09.239+0000 7fc6cc400640 1 -- 172.21.15.22:0/3538282189 shutdown_connections 2024-03-28T18:04:09.241 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:09.239+0000 7fc6cc400640 1 -- 172.21.15.22:0/3538282189 wait complete. 2024-03-28T18:04:09.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:04:09 smithi022 ceph-mon[31985]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T18:04:09.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:04:09 smithi022 ceph-mon[31985]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T18:04:09.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:04:09 smithi022 ceph-mon[31985]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T18:04:09.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:04:09 smithi022 ceph-mon[31985]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-28T18:04:09.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:04:09 smithi022 ceph-mon[31985]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-28T18:04:09.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:04:09 smithi022 ceph-mon[31985]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T18:04:09.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:04:09 smithi022 ceph-mon[31985]: mds.? [v2:172.21.15.22:6834/3718238950,v1:172.21.15.22:6835/3718238950] up:boot 2024-03-28T18:04:09.533 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:04:09 smithi022 ceph-mon[31985]: fsmap 3 up:standby 2024-03-28T18:04:09.533 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:04:09 smithi022 ceph-mon[31985]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-03-28T18:04:09.533 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:04:09 smithi022 ceph-mon[31985]: from='client.? 172.21.15.22:0/3538282189' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-28T18:04:09.925 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:04:09 smithi134 ceph-mon[35037]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T18:04:09.925 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:04:09 smithi134 ceph-mon[35037]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T18:04:09.925 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:04:09 smithi134 ceph-mon[35037]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T18:04:09.925 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:04:09 smithi134 ceph-mon[35037]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-28T18:04:09.925 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:04:09 smithi134 ceph-mon[35037]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-28T18:04:09.925 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:04:09 smithi134 ceph-mon[35037]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T18:04:09.925 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:04:09 smithi134 ceph-mon[35037]: mds.? [v2:172.21.15.22:6834/3718238950,v1:172.21.15.22:6835/3718238950] up:boot 2024-03-28T18:04:09.926 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:04:09 smithi134 ceph-mon[35037]: fsmap 3 up:standby 2024-03-28T18:04:09.926 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:04:09 smithi134 ceph-mon[35037]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-03-28T18:04:09.926 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:04:09 smithi134 ceph-mon[35037]: from='client.? 172.21.15.22:0/3538282189' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-28T18:04:09.933 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:04:09 smithi079 ceph-mon[35312]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T18:04:09.933 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:04:09 smithi079 ceph-mon[35312]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T18:04:09.933 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:04:09 smithi079 ceph-mon[35312]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T18:04:09.933 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:04:09 smithi079 ceph-mon[35312]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-28T18:04:09.933 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:04:09 smithi079 ceph-mon[35312]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-28T18:04:09.933 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:04:09 smithi079 ceph-mon[35312]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T18:04:09.933 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:04:09 smithi079 ceph-mon[35312]: mds.? [v2:172.21.15.22:6834/3718238950,v1:172.21.15.22:6835/3718238950] up:boot 2024-03-28T18:04:09.933 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:04:09 smithi079 ceph-mon[35312]: fsmap 3 up:standby 2024-03-28T18:04:09.933 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:04:09 smithi079 ceph-mon[35312]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-03-28T18:04:09.933 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:04:09 smithi079 ceph-mon[35312]: from='client.? 172.21.15.22:0/3538282189' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-28T18:04:10.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:04:10 smithi022 ceph-mon[31985]: Deploying daemon mds.d on smithi022 2024-03-28T18:04:10.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:04:10 smithi022 ceph-mon[31985]: pgmap v345: 1 pgs: 1 active+clean; 577 KiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-28T18:04:10.871 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:04:10 smithi134 ceph-mon[35037]: Deploying daemon mds.d on smithi022 2024-03-28T18:04:10.871 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:04:10 smithi134 ceph-mon[35037]: pgmap v345: 1 pgs: 1 active+clean; 577 KiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-28T18:04:10.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:04:10 smithi079 ceph-mon[35312]: Deploying daemon mds.d on smithi022 2024-03-28T18:04:10.933 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:04:10 smithi079 ceph-mon[35312]: pgmap v345: 1 pgs: 1 active+clean; 577 KiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-28T18:04:11.261 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-28T18:04:11.494 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:11.493+0000 7fee1de00640 1 -- 172.21.15.22:0/678575493 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fee181026f0 msgr2=0x7fee18102b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:04:11.494 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:11.493+0000 7fee1de00640 1 --2- 172.21.15.22:0/678575493 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fee181026f0 0x7fee18102b70 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fee04009900 tx=0x7fee0402bc30 comp rx=0 tx=0).stop 2024-03-28T18:04:11.494 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:11.493+0000 7fee1de00640 1 -- 172.21.15.22:0/678575493 shutdown_connections 2024-03-28T18:04:11.494 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:11.493+0000 7fee1de00640 1 --2- 172.21.15.22:0/678575493 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fee181030b0 0x7fee18109940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:04:11.494 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:11.493+0000 7fee1de00640 1 --2- 172.21.15.22:0/678575493 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fee181026f0 0x7fee18102b70 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:04:11.494 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:11.493+0000 7fee1de00640 1 --2- 172.21.15.22:0/678575493 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fee18101500 0x7fee18101920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:04:11.494 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:11.493+0000 7fee1de00640 1 -- 172.21.15.22:0/678575493 >> 172.21.15.22:0/678575493 conn(0x7fee180fca70 msgr2=0x7fee180feeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:04:11.495 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:11.493+0000 7fee1de00640 1 -- 172.21.15.22:0/678575493 shutdown_connections 2024-03-28T18:04:11.495 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:11.493+0000 7fee1de00640 1 -- 172.21.15.22:0/678575493 wait complete. 2024-03-28T18:04:11.497 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:11.494+0000 7fee1de00640 1 Processor -- start 2024-03-28T18:04:11.497 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:11.494+0000 7fee1de00640 1 -- start start 2024-03-28T18:04:11.497 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:11.494+0000 7fee1de00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fee18101500 0x7fee181999c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:04:11.497 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:11.494+0000 7fee1de00640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fee181026f0 0x7fee18199f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:04:11.497 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:11.495+0000 7fee1de00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fee181030b0 0x7fee181a0f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:04:11.497 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:11.495+0000 7fee1de00640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee1810d1b0 con 0x7fee18101500 2024-03-28T18:04:11.497 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:11.495+0000 7fee1de00640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee1810d030 con 0x7fee181026f0 2024-03-28T18:04:11.497 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:11.495+0000 7fee1de00640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee1810d330 con 0x7fee181030b0 2024-03-28T18:04:11.497 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:11.495+0000 7fee1ca00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fee18101500 0x7fee181999c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:04:11.497 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:11.495+0000 7fee1ca00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fee18101500 0x7fee181999c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:48238/0 (socket says 172.21.15.22:48238) 2024-03-28T18:04:11.497 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:11.495+0000 7fee1ca00640 1 -- 172.21.15.22:0/2671237528 learned_addr learned my addr 172.21.15.22:0/2671237528 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-28T18:04:11.497 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:11.495+0000 7fee1d400640 1 --2- 172.21.15.22:0/2671237528 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fee181030b0 0x7fee181a0f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:04:11.497 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:11.495+0000 7fee17e00640 1 --2- 172.21.15.22:0/2671237528 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fee181026f0 0x7fee18199f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:04:11.498 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:11.495+0000 7fee1ca00640 1 -- 172.21.15.22:0/2671237528 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fee181030b0 msgr2=0x7fee181a0f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:04:11.498 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:11.495+0000 7fee1ca00640 1 --2- 172.21.15.22:0/2671237528 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fee181030b0 0x7fee181a0f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:04:11.498 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:11.495+0000 7fee1ca00640 1 -- 172.21.15.22:0/2671237528 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fee181026f0 msgr2=0x7fee18199f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:04:11.498 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:11.495+0000 7fee1ca00640 1 --2- 172.21.15.22:0/2671237528 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fee181026f0 0x7fee18199f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:04:11.498 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:11.495+0000 7fee1ca00640 1 -- 172.21.15.22:0/2671237528 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fee181a15e0 con 0x7fee18101500 2024-03-28T18:04:11.498 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:11.496+0000 7fee1ca00640 1 --2- 172.21.15.22:0/2671237528 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fee18101500 0x7fee181999c0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fee0c00d950 tx=0x7fee0c00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:04:11.498 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:11.496+0000 7fee15600640 1 -- 172.21.15.22:0/2671237528 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee0c017070 con 0x7fee18101500 2024-03-28T18:04:11.498 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:11.496+0000 7fee15600640 1 -- 172.21.15.22:0/2671237528 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7fee0c009c30 con 0x7fee18101500 2024-03-28T18:04:11.498 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:11.496+0000 7fee1de00640 1 -- 172.21.15.22:0/2671237528 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fee181a18d0 con 0x7fee18101500 2024-03-28T18:04:11.498 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:11.496+0000 7fee15600640 1 -- 172.21.15.22:0/2671237528 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee0c00f620 con 0x7fee18101500 2024-03-28T18:04:11.498 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:11.496+0000 7fee1de00640 1 -- 172.21.15.22:0/2671237528 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fee181a1df0 con 0x7fee18101500 2024-03-28T18:04:11.499 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:11.497+0000 7fee1de00640 1 -- 172.21.15.22:0/2671237528 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fee181a1ff0 con 0x7fee18101500 2024-03-28T18:04:11.500 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:11.499+0000 7fee15600640 1 -- 172.21.15.22:0/2671237528 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 16) v1 ==== 137290+0+0 (secure 0 0 0) 0x7fee0c01a070 con 0x7fee18101500 2024-03-28T18:04:11.502 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:11.501+0000 7fee15600640 1 --2- 172.21.15.22:0/2671237528 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7fee000a2950 0x7fee000a4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:04:11.502 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:11.501+0000 7fee15600640 1 -- 172.21.15.22:0/2671237528 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 7127+0+0 (secure 0 0 0) 0x7fee0c0ce070 con 0x7fee18101500 2024-03-28T18:04:11.502 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:11.501+0000 7fee17e00640 1 --2- 172.21.15.22:0/2671237528 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7fee000a2950 0x7fee000a4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:04:11.503 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:11.502+0000 7fee17e00640 1 --2- 172.21.15.22:0/2671237528 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7fee000a2950 0x7fee000a4e10 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fee04006010 tx=0x7fee04000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:04:11.506 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:11.505+0000 7fee15600640 1 -- 172.21.15.22:0/2671237528 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fee0c009de0 con 0x7fee18101500 2024-03-28T18:04:11.777 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:11.776+0000 7fee1de00640 1 -- 172.21.15.22:0/2671237528 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fee181a2260 con 0x7fee18101500 2024-03-28T18:04:11.777 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:11.777+0000 7fee15600640 1 -- 172.21.15.22:0/2671237528 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 4 v4) v1 ==== 93+0+2907 (secure 0 0 0) 0x7fee0c09b510 con 0x7fee18101500 2024-03-28T18:04:11.778 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-03-28T18:04:11.778 INFO:teuthology.orchestra.run.smithi022.stdout:{"epoch":4,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14544,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6835/3718238950","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6834","nonce":3718238950},{"type":"v1","addr":"172.21.15.22:6835","nonce":3718238950}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":24412,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.134:6833/3176125362","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6832","nonce":3176125362},{"type":"v1","addr":"172.21.15.134:6833","nonce":3176125362}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":34442,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.79:6833/856896997","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6832","nonce":856896997},{"type":"v1","addr":"172.21.15.79:6833","nonce":856896997}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[]} 2024-03-28T18:04:11.778 INFO:teuthology.orchestra.run.smithi022.stderr:dumped fsmap epoch 4 2024-03-28T18:04:11.780 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:11.779+0000 7fee1de00640 1 -- 172.21.15.22:0/2671237528 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7fee000a2950 msgr2=0x7fee000a4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:04:11.780 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:11.779+0000 7fee1de00640 1 --2- 172.21.15.22:0/2671237528 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7fee000a2950 0x7fee000a4e10 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fee04006010 tx=0x7fee04000bc0 comp rx=0 tx=0).stop 2024-03-28T18:04:11.780 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:11.779+0000 7fee1de00640 1 -- 172.21.15.22:0/2671237528 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fee18101500 msgr2=0x7fee181999c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:04:11.780 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:11.779+0000 7fee1de00640 1 --2- 172.21.15.22:0/2671237528 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fee18101500 0x7fee181999c0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fee0c00d950 tx=0x7fee0c00de10 comp rx=0 tx=0).stop 2024-03-28T18:04:11.780 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:11.779+0000 7fee1de00640 1 -- 172.21.15.22:0/2671237528 shutdown_connections 2024-03-28T18:04:11.781 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:11.779+0000 7fee1de00640 1 --2- 172.21.15.22:0/2671237528 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7fee000a2950 0x7fee000a4e10 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:04:11.781 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:11.779+0000 7fee1de00640 1 --2- 172.21.15.22:0/2671237528 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fee181030b0 0x7fee181a0f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:04:11.781 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:11.779+0000 7fee1de00640 1 --2- 172.21.15.22:0/2671237528 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fee181026f0 0x7fee18199f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:04:11.781 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:11.779+0000 7fee1de00640 1 --2- 172.21.15.22:0/2671237528 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fee18101500 0x7fee181999c0 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:04:11.781 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:11.779+0000 7fee1de00640 1 -- 172.21.15.22:0/2671237528 >> 172.21.15.22:0/2671237528 conn(0x7fee180fca70 msgr2=0x7fee180738d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:04:11.781 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:11.780+0000 7fee1de00640 1 -- 172.21.15.22:0/2671237528 shutdown_connections 2024-03-28T18:04:11.781 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:11.780+0000 7fee1de00640 1 -- 172.21.15.22:0/2671237528 wait complete. 2024-03-28T18:04:12.322 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:04:12 smithi022 podman[73359]: 2024-03-28 18:04:12.038197249 +0000 UTC m=+0.130907437 container create 9c7b84e0d12419cc7a24a5188bb8d58749794e79dd750596877364c5da92b74c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d, org.label-schema.build-date=20240319, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_CLEAN=True) 2024-03-28T18:04:12.322 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:04:12 smithi022 podman[73359]: 2024-03-28 18:04:12.320508788 +0000 UTC m=+0.413218996 container init 9c7b84e0d12419cc7a24a5188bb8d58749794e79dd750596877364c5da92b74c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, ceph=True, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240319, GIT_CLEAN=True) 2024-03-28T18:04:12.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:04:12 smithi022 ceph-mon[31985]: pgmap v346: 1 pgs: 1 active+clean; 577 KiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-28T18:04:12.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:04:12 smithi022 ceph-mon[31985]: from='client.? 172.21.15.22:0/2671237528' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-28T18:04:12.783 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:04:12 smithi022 podman[73359]: 2024-03-28 18:04:12.331541938 +0000 UTC m=+0.424252095 container start 9c7b84e0d12419cc7a24a5188bb8d58749794e79dd750596877364c5da92b74c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d, io.buildah.version=1.33.5, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.schema-version=1.0, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, GIT_BRANCH=HEAD, org.label-schema.build-date=20240319) 2024-03-28T18:04:12.783 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:04:12 smithi022 ceph-mds[73377]: -- 172.21.15.22:0/3957807514 <== mon.0 v2:172.21.15.22:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55a726c69a40 con 0x55a726c3cc00 2024-03-28T18:04:12.783 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:04:12 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[73373]: starting mds.d at 2024-03-28T18:04:12.783 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:04:12 smithi022 bash[73359]: 9c7b84e0d12419cc7a24a5188bb8d58749794e79dd750596877364c5da92b74c 2024-03-28T18:04:12.783 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:04:12 smithi022 systemd[1]: Started Ceph mds.d for fd220748-ed2b-11ee-95d0-87774f69a715. 2024-03-28T18:04:12.925 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:04:12 smithi134 ceph-mon[35037]: pgmap v346: 1 pgs: 1 active+clean; 577 KiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-28T18:04:12.925 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:04:12 smithi134 ceph-mon[35037]: from='client.? 172.21.15.22:0/2671237528' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-28T18:04:12.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:04:12 smithi079 ceph-mon[35312]: pgmap v346: 1 pgs: 1 active+clean; 577 KiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-28T18:04:12.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:04:12 smithi079 ceph-mon[35312]: from='client.? 172.21.15.22:0/2671237528' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-28T18:04:13.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:04:13 smithi022 ceph-mon[31985]: mds.? [v2:172.21.15.22:6836/3676939386,v1:172.21.15.22:6837/3676939386] up:boot 2024-03-28T18:04:13.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:04:13 smithi022 ceph-mon[31985]: fsmap 4 up:standby 2024-03-28T18:04:13.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:04:13 smithi022 ceph-mon[31985]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-03-28T18:04:13.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:04:13 smithi022 ceph-mon[31985]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T18:04:13.783 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:04:13 smithi022 ceph-mon[31985]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T18:04:13.783 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:04:13 smithi022 ceph-mon[31985]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T18:04:13.783 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:04:13 smithi022 ceph-mon[31985]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-28T18:04:13.783 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:04:13 smithi022 ceph-mon[31985]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-28T18:04:13.783 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:04:13 smithi022 ceph-mon[31985]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T18:04:13.797 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:04:13 smithi079 ceph-mon[35312]: mds.? [v2:172.21.15.22:6836/3676939386,v1:172.21.15.22:6837/3676939386] up:boot 2024-03-28T18:04:13.798 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:04:13 smithi079 ceph-mon[35312]: fsmap 4 up:standby 2024-03-28T18:04:13.798 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:04:13 smithi079 ceph-mon[35312]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-03-28T18:04:13.798 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:04:13 smithi079 ceph-mon[35312]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T18:04:13.798 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:04:13 smithi079 ceph-mon[35312]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T18:04:13.798 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:04:13 smithi079 ceph-mon[35312]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T18:04:13.798 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:04:13 smithi079 ceph-mon[35312]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-28T18:04:13.798 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:04:13 smithi079 ceph-mon[35312]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-28T18:04:13.798 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:04:13 smithi079 ceph-mon[35312]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T18:04:13.801 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-28T18:04:13.925 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:04:13 smithi134 ceph-mon[35037]: mds.? [v2:172.21.15.22:6836/3676939386,v1:172.21.15.22:6837/3676939386] up:boot 2024-03-28T18:04:13.925 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:04:13 smithi134 ceph-mon[35037]: fsmap 4 up:standby 2024-03-28T18:04:13.925 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:04:13 smithi134 ceph-mon[35037]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-03-28T18:04:13.925 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:04:13 smithi134 ceph-mon[35037]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T18:04:13.925 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:04:13 smithi134 ceph-mon[35037]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T18:04:13.925 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:04:13 smithi134 ceph-mon[35037]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T18:04:13.925 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:04:13 smithi134 ceph-mon[35037]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-28T18:04:13.925 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:04:13 smithi134 ceph-mon[35037]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-28T18:04:13.925 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:04:13 smithi134 ceph-mon[35037]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T18:04:14.039 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:14.038+0000 7f4d46c00640 1 -- 172.21.15.22:0/1669160107 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4d401052a0 msgr2=0x7f4d4010bb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:04:14.040 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:14.038+0000 7f4d46c00640 1 --2- 172.21.15.22:0/1669160107 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4d401052a0 0x7f4d4010bb30 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f4d38009ca0 tx=0x7f4d3802c0f0 comp rx=0 tx=0).stop 2024-03-28T18:04:14.040 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:14.038+0000 7f4d46c00640 1 -- 172.21.15.22:0/1669160107 shutdown_connections 2024-03-28T18:04:14.040 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:14.038+0000 7f4d46c00640 1 --2- 172.21.15.22:0/1669160107 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4d401052a0 0x7f4d4010bb30 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:04:14.040 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:14.038+0000 7f4d46c00640 1 --2- 172.21.15.22:0/1669160107 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f4d401048e0 0x7f4d40104d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:04:14.040 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:14.038+0000 7f4d46c00640 1 --2- 172.21.15.22:0/1669160107 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4d401036f0 0x7f4d40103b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:04:14.040 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:14.038+0000 7f4d46c00640 1 -- 172.21.15.22:0/1669160107 >> 172.21.15.22:0/1669160107 conn(0x7f4d400fec80 msgr2=0x7f4d401010a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:04:14.040 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:14.038+0000 7f4d46c00640 1 -- 172.21.15.22:0/1669160107 shutdown_connections 2024-03-28T18:04:14.040 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:14.038+0000 7f4d46c00640 1 -- 172.21.15.22:0/1669160107 wait complete. 2024-03-28T18:04:14.042 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:14.039+0000 7f4d46c00640 1 Processor -- start 2024-03-28T18:04:14.042 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:14.040+0000 7f4d46c00640 1 -- start start 2024-03-28T18:04:14.043 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:14.040+0000 7f4d46c00640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f4d401036f0 0x7f4d40199990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:04:14.043 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:14.040+0000 7f4d46c00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4d401048e0 0x7f4d40199ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:04:14.043 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:14.040+0000 7f4d46c00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4d401052a0 0x7f4d401a0f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:04:14.043 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:14.040+0000 7f4d46c00640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d4010f350 con 0x7f4d401052a0 2024-03-28T18:04:14.043 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:14.040+0000 7f4d46c00640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d4010f1d0 con 0x7f4d401036f0 2024-03-28T18:04:14.043 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:14.040+0000 7f4d46c00640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d4010f4d0 con 0x7f4d401048e0 2024-03-28T18:04:14.043 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:14.040+0000 7f4d45800640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f4d401036f0 0x7f4d40199990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:04:14.043 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:14.040+0000 7f4d46200640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4d401052a0 0x7f4d401a0f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:04:14.043 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:14.041+0000 7f4d45800640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f4d401036f0 0x7f4d40199990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.22:44340/0 (socket says 172.21.15.22:44340) 2024-03-28T18:04:14.043 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:14.041+0000 7f4d45800640 1 -- 172.21.15.22:0/4007970348 learned_addr learned my addr 172.21.15.22:0/4007970348 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-28T18:04:14.043 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:14.041+0000 7f4d44e00640 1 --2- 172.21.15.22:0/4007970348 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4d401048e0 0x7f4d40199ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:04:14.043 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:14.041+0000 7f4d45800640 1 -- 172.21.15.22:0/4007970348 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4d401048e0 msgr2=0x7f4d40199ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:04:14.044 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:14.041+0000 7f4d45800640 1 --2- 172.21.15.22:0/4007970348 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4d401048e0 0x7f4d40199ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:04:14.044 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:14.041+0000 7f4d45800640 1 -- 172.21.15.22:0/4007970348 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4d401052a0 msgr2=0x7f4d401a0f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:04:14.044 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:14.041+0000 7f4d45800640 1 --2- 172.21.15.22:0/4007970348 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4d401052a0 0x7f4d401a0f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:04:14.044 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:14.041+0000 7f4d45800640 1 -- 172.21.15.22:0/4007970348 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4d401a15b0 con 0x7f4d401036f0 2024-03-28T18:04:14.044 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:14.041+0000 7f4d45800640 1 --2- 172.21.15.22:0/4007970348 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f4d401036f0 0x7f4d40199990 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f4d3400d950 tx=0x7f4d3400de10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:04:14.044 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:14.042+0000 7f4d3e000640 1 -- 172.21.15.22:0/4007970348 <== mon.2 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d34017070 con 0x7f4d401036f0 2024-03-28T18:04:14.044 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:14.042+0000 7f4d46c00640 1 -- 172.21.15.22:0/4007970348 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4d401a18a0 con 0x7f4d401036f0 2024-03-28T18:04:14.044 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:14.042+0000 7f4d3e000640 1 -- 172.21.15.22:0/4007970348 <== mon.2 v2:172.21.15.79:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f4d34009c30 con 0x7f4d401036f0 2024-03-28T18:04:14.044 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:14.042+0000 7f4d3e000640 1 -- 172.21.15.22:0/4007970348 <== mon.2 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d3400f620 con 0x7f4d401036f0 2024-03-28T18:04:14.044 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:14.042+0000 7f4d46c00640 1 -- 172.21.15.22:0/4007970348 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4d401a1df0 con 0x7f4d401036f0 2024-03-28T18:04:14.044 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:14.043+0000 7f4d46c00640 1 -- 172.21.15.22:0/4007970348 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4d4010aa10 con 0x7f4d401036f0 2024-03-28T18:04:14.046 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:14.046+0000 7f4d3e000640 1 -- 172.21.15.22:0/4007970348 <== mon.2 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 16) v1 ==== 137290+0+0 (secure 0 0 0) 0x7f4d3401a070 con 0x7f4d401036f0 2024-03-28T18:04:14.048 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:14.048+0000 7f4d3e000640 1 --2- 172.21.15.22:0/4007970348 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7f4d280a2800 0x7f4d280a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:04:14.048 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:14.048+0000 7f4d3e000640 1 -- 172.21.15.22:0/4007970348 <== mon.2 v2:172.21.15.79:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f4d340ce100 con 0x7f4d401036f0 2024-03-28T18:04:14.049 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:14.048+0000 7f4d44e00640 1 --2- 172.21.15.22:0/4007970348 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7f4d280a2800 0x7f4d280a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:04:14.049 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:14.049+0000 7f4d44e00640 1 --2- 172.21.15.22:0/4007970348 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7f4d280a2800 0x7f4d280a4cc0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f4d4019aea0 tx=0x7f4d2c005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:04:14.054 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:14.054+0000 7f4d3e000640 1 -- 172.21.15.22:0/4007970348 <== mon.2 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f4d34009de0 con 0x7f4d401036f0 2024-03-28T18:04:14.298 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:14.296+0000 7f4d46c00640 1 -- 172.21.15.22:0/4007970348 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f4d4002d070 con 0x7f4d401036f0 2024-03-28T18:04:14.298 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:14.297+0000 7f4d3e000640 1 -- 172.21.15.22:0/4007970348 <== mon.2 v2:172.21.15.79:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 5 v5) v1 ==== 93+0+3698 (secure 0 0 0) 0x7f4d3409b5a0 con 0x7f4d401036f0 2024-03-28T18:04:14.298 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-03-28T18:04:14.299 INFO:teuthology.orchestra.run.smithi022.stdout:{"epoch":5,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14544,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6835/3718238950","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6834","nonce":3718238950},{"type":"v1","addr":"172.21.15.22:6835","nonce":3718238950}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14562,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6837/3676939386","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6836","nonce":3676939386},{"type":"v1","addr":"172.21.15.22:6837","nonce":3676939386}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":24412,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.134:6833/3176125362","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6832","nonce":3176125362},{"type":"v1","addr":"172.21.15.134:6833","nonce":3176125362}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":34442,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.79:6833/856896997","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6832","nonce":856896997},{"type":"v1","addr":"172.21.15.79:6833","nonce":856896997}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[]} 2024-03-28T18:04:14.299 INFO:teuthology.orchestra.run.smithi022.stderr:dumped fsmap epoch 5 2024-03-28T18:04:14.302 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:14.300+0000 7f4d46c00640 1 -- 172.21.15.22:0/4007970348 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7f4d280a2800 msgr2=0x7f4d280a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:04:14.302 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:14.300+0000 7f4d46c00640 1 --2- 172.21.15.22:0/4007970348 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7f4d280a2800 0x7f4d280a4cc0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f4d4019aea0 tx=0x7f4d2c005f90 comp rx=0 tx=0).stop 2024-03-28T18:04:14.302 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:14.300+0000 7f4d46c00640 1 -- 172.21.15.22:0/4007970348 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f4d401036f0 msgr2=0x7f4d40199990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:04:14.302 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:14.300+0000 7f4d46c00640 1 --2- 172.21.15.22:0/4007970348 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f4d401036f0 0x7f4d40199990 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f4d3400d950 tx=0x7f4d3400de10 comp rx=0 tx=0).stop 2024-03-28T18:04:14.302 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:14.300+0000 7f4d46c00640 1 -- 172.21.15.22:0/4007970348 shutdown_connections 2024-03-28T18:04:14.302 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:14.300+0000 7f4d46c00640 1 --2- 172.21.15.22:0/4007970348 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7f4d280a2800 0x7f4d280a4cc0 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:04:14.302 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:14.300+0000 7f4d46c00640 1 --2- 172.21.15.22:0/4007970348 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4d401052a0 0x7f4d401a0f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:04:14.302 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:14.300+0000 7f4d46c00640 1 --2- 172.21.15.22:0/4007970348 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4d401048e0 0x7f4d40199ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:04:14.302 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:14.300+0000 7f4d46c00640 1 --2- 172.21.15.22:0/4007970348 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f4d401036f0 0x7f4d40199990 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:04:14.302 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:14.300+0000 7f4d46c00640 1 -- 172.21.15.22:0/4007970348 >> 172.21.15.22:0/4007970348 conn(0x7f4d400fec80 msgr2=0x7f4d40107be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:04:14.302 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:14.301+0000 7f4d46c00640 1 -- 172.21.15.22:0/4007970348 shutdown_connections 2024-03-28T18:04:14.302 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:14.301+0000 7f4d46c00640 1 -- 172.21.15.22:0/4007970348 wait complete. 2024-03-28T18:04:14.658 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:04:14 smithi079 ceph-mon[35312]: pgmap v347: 1 pgs: 1 active+clean; 577 KiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-28T18:04:14.658 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:04:14 smithi079 ceph-mon[35312]: Deploying daemon mds.e on smithi079 2024-03-28T18:04:14.658 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:04:14 smithi079 ceph-mon[35312]: from='client.? 172.21.15.22:0/4007970348' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-28T18:04:14.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:04:14 smithi022 ceph-mon[31985]: pgmap v347: 1 pgs: 1 active+clean; 577 KiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-28T18:04:14.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:04:14 smithi022 ceph-mon[31985]: Deploying daemon mds.e on smithi079 2024-03-28T18:04:14.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:04:14 smithi022 ceph-mon[31985]: from='client.? 172.21.15.22:0/4007970348' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-28T18:04:14.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:04:14 smithi134 ceph-mon[35037]: pgmap v347: 1 pgs: 1 active+clean; 577 KiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-28T18:04:14.925 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:04:14 smithi134 ceph-mon[35037]: Deploying daemon mds.e on smithi079 2024-03-28T18:04:14.925 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:04:14 smithi134 ceph-mon[35037]: from='client.? 172.21.15.22:0/4007970348' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-28T18:04:15.755 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:04:15 smithi079 ceph-mon[35312]: pgmap v348: 1 pgs: 1 active+clean; 577 KiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-28T18:04:15.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:04:15 smithi022 ceph-mon[31985]: pgmap v348: 1 pgs: 1 active+clean; 577 KiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-28T18:04:15.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:04:15 smithi134 ceph-mon[35037]: pgmap v348: 1 pgs: 1 active+clean; 577 KiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-28T18:04:16.107 INFO:journalctl@ceph.mds.e.smithi079.stdout:Mar 28 18:04:15 smithi079 systemd[1]: Starting Ceph mds.e for fd220748-ed2b-11ee-95d0-87774f69a715... 2024-03-28T18:04:16.321 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-28T18:04:16.364 INFO:journalctl@ceph.mds.e.smithi079.stdout:Mar 28 18:04:16 smithi079 podman[64613]: 2024-03-28 18:04:16.011172286 +0000 UTC m=+0.031198612 image pull c664afe56bb01db4f9afa9532c20598e201e664f2e9ba2be50db0d52674a26d2 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc 2024-03-28T18:04:16.365 INFO:journalctl@ceph.mds.e.smithi079.stdout:Mar 28 18:04:16 smithi079 podman[64613]: 2024-03-28 18:04:16.110850035 +0000 UTC m=+0.130876375 container create 4a2d81a3d9f5ca7610f2f9fb45fc4e302623c33905442a766cbab9ad57bc9f6d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-e, org.label-schema.build-date=20240319, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.schema-version=1.0) 2024-03-28T18:04:16.365 INFO:journalctl@ceph.mds.e.smithi079.stdout:Mar 28 18:04:16 smithi079 podman[64613]: 2024-03-28 18:04:16.364368535 +0000 UTC m=+0.384394890 container init 4a2d81a3d9f5ca7610f2f9fb45fc4e302623c33905442a766cbab9ad57bc9f6d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-e, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, org.label-schema.build-date=20240319, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_CLEAN=True, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, io.buildah.version=1.33.5, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30) 2024-03-28T18:04:16.560 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:16.560+0000 7feb36200640 1 -- 172.21.15.22:0/3448787829 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7feb30104ad0 msgr2=0x7feb30104f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:04:16.560 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:16.560+0000 7feb36200640 1 --2- 172.21.15.22:0/3448787829 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7feb30104ad0 0x7feb30104f50 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7feb1c009900 tx=0x7feb1c02bc40 comp rx=0 tx=0).stop 2024-03-28T18:04:16.560 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:16.560+0000 7feb36200640 1 -- 172.21.15.22:0/3448787829 shutdown_connections 2024-03-28T18:04:16.560 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:16.560+0000 7feb36200640 1 --2- 172.21.15.22:0/3448787829 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7feb30105490 0x7feb3010bd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:04:16.560 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:16.560+0000 7feb36200640 1 --2- 172.21.15.22:0/3448787829 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7feb30104ad0 0x7feb30104f50 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:04:16.560 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:16.560+0000 7feb36200640 1 --2- 172.21.15.22:0/3448787829 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7feb301038e0 0x7feb30103d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:04:16.561 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:16.560+0000 7feb36200640 1 -- 172.21.15.22:0/3448787829 >> 172.21.15.22:0/3448787829 conn(0x7feb300fee90 msgr2=0x7feb301012b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:04:16.561 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:16.560+0000 7feb36200640 1 -- 172.21.15.22:0/3448787829 shutdown_connections 2024-03-28T18:04:16.561 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:16.560+0000 7feb36200640 1 -- 172.21.15.22:0/3448787829 wait complete. 2024-03-28T18:04:16.563 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:16.560+0000 7feb36200640 1 Processor -- start 2024-03-28T18:04:16.563 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:16.560+0000 7feb36200640 1 -- start start 2024-03-28T18:04:16.563 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:16.560+0000 7feb36200640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7feb301038e0 0x7feb30199c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:04:16.563 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:16.560+0000 7feb36200640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7feb30104ad0 0x7feb3019a160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:04:16.563 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:16.560+0000 7feb36200640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7feb30105490 0x7feb301a1190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:04:16.563 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:16.560+0000 7feb36200640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb3010f590 con 0x7feb30104ad0 2024-03-28T18:04:16.563 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:16.560+0000 7feb36200640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb3010f410 con 0x7feb301038e0 2024-03-28T18:04:16.564 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:16.560+0000 7feb36200640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb3010f710 con 0x7feb30105490 2024-03-28T18:04:16.564 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:16.561+0000 7feb2ea00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7feb30104ad0 0x7feb3019a160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:04:16.564 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:16.561+0000 7feb2ea00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7feb30104ad0 0x7feb3019a160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:48310/0 (socket says 172.21.15.22:48310) 2024-03-28T18:04:16.564 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:16.561+0000 7feb2ea00640 1 -- 172.21.15.22:0/3011192161 learned_addr learned my addr 172.21.15.22:0/3011192161 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-28T18:04:16.564 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:16.561+0000 7feb2f400640 1 --2- 172.21.15.22:0/3011192161 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7feb301038e0 0x7feb30199c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:04:16.564 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:16.561+0000 7feb2fe00640 1 --2- 172.21.15.22:0/3011192161 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7feb30105490 0x7feb301a1190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:04:16.564 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:16.561+0000 7feb2ea00640 1 -- 172.21.15.22:0/3011192161 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7feb30105490 msgr2=0x7feb301a1190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:04:16.564 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:16.561+0000 7feb2ea00640 1 --2- 172.21.15.22:0/3011192161 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7feb30105490 0x7feb301a1190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:04:16.564 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:16.561+0000 7feb2ea00640 1 -- 172.21.15.22:0/3011192161 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7feb301038e0 msgr2=0x7feb30199c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:04:16.564 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:16.561+0000 7feb2ea00640 1 --2- 172.21.15.22:0/3011192161 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7feb301038e0 0x7feb30199c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:04:16.564 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:16.561+0000 7feb2ea00640 1 -- 172.21.15.22:0/3011192161 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feb301a1830 con 0x7feb30104ad0 2024-03-28T18:04:16.564 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:16.561+0000 7feb2f400640 1 --2- 172.21.15.22:0/3011192161 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7feb301038e0 0x7feb30199c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-28T18:04:16.564 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:16.562+0000 7feb2ea00640 1 --2- 172.21.15.22:0/3011192161 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7feb30104ad0 0x7feb3019a160 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7feb1c000c00 tx=0x7feb1c004f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:04:16.564 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:16.562+0000 7feb27e00640 1 -- 172.21.15.22:0/3011192161 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb1c043020 con 0x7feb30104ad0 2024-03-28T18:04:16.564 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:16.562+0000 7feb27e00640 1 -- 172.21.15.22:0/3011192161 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7feb1c0054d0 con 0x7feb30104ad0 2024-03-28T18:04:16.564 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:16.562+0000 7feb27e00640 1 -- 172.21.15.22:0/3011192161 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb1c003c90 con 0x7feb30104ad0 2024-03-28T18:04:16.565 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:16.562+0000 7feb36200640 1 -- 172.21.15.22:0/3011192161 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feb301a1ac0 con 0x7feb30104ad0 2024-03-28T18:04:16.565 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:16.562+0000 7feb36200640 1 -- 172.21.15.22:0/3011192161 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feb301a1fc0 con 0x7feb30104ad0 2024-03-28T18:04:16.565 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:16.564+0000 7feb36200640 1 -- 172.21.15.22:0/3011192161 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feb3010ac00 con 0x7feb30104ad0 2024-03-28T18:04:16.566 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:16.565+0000 7feb27e00640 1 -- 172.21.15.22:0/3011192161 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 16) v1 ==== 137290+0+0 (secure 0 0 0) 0x7feb1c046020 con 0x7feb30104ad0 2024-03-28T18:04:16.567 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:16.567+0000 7feb27e00640 1 --2- 172.21.15.22:0/3011192161 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7feb140a2880 0x7feb140a4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:04:16.567 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:16.567+0000 7feb27e00640 1 -- 172.21.15.22:0/3011192161 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 7127+0+0 (secure 0 0 0) 0x7feb1c0f09a0 con 0x7feb30104ad0 2024-03-28T18:04:16.568 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:16.567+0000 7feb2f400640 1 --2- 172.21.15.22:0/3011192161 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7feb140a2880 0x7feb140a4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:04:16.568 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:16.568+0000 7feb2f400640 1 --2- 172.21.15.22:0/3011192161 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7feb140a2880 0x7feb140a4d40 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7feb20005fd0 tx=0x7feb20005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:04:16.572 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:16.572+0000 7feb27e00640 1 -- 172.21.15.22:0/3011192161 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7feb1c04d020 con 0x7feb30104ad0 2024-03-28T18:04:16.683 INFO:journalctl@ceph.mds.e.smithi079.stdout:Mar 28 18:04:16 smithi079 podman[64613]: 2024-03-28 18:04:16.372798755 +0000 UTC m=+0.392825074 container start 4a2d81a3d9f5ca7610f2f9fb45fc4e302623c33905442a766cbab9ad57bc9f6d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-e, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, ceph=True, org.label-schema.vendor=CentOS, GIT_CLEAN=True, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.build-date=20240319, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-03-28T18:04:16.683 INFO:journalctl@ceph.mds.e.smithi079.stdout:Mar 28 18:04:16 smithi079 ceph-mds[64632]: -- 172.21.15.79:0/3016308652 <== mon.2 v2:172.21.15.79:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55ecca319a40 con 0x55ecca2ed000 2024-03-28T18:04:16.683 INFO:journalctl@ceph.mds.e.smithi079.stdout:Mar 28 18:04:16 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-e[64628]: starting mds.e at 2024-03-28T18:04:16.683 INFO:journalctl@ceph.mds.e.smithi079.stdout:Mar 28 18:04:16 smithi079 bash[64613]: 4a2d81a3d9f5ca7610f2f9fb45fc4e302623c33905442a766cbab9ad57bc9f6d 2024-03-28T18:04:16.683 INFO:journalctl@ceph.mds.e.smithi079.stdout:Mar 28 18:04:16 smithi079 systemd[1]: Started Ceph mds.e for fd220748-ed2b-11ee-95d0-87774f69a715. 2024-03-28T18:04:16.826 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:16.825+0000 7feb36200640 1 -- 172.21.15.22:0/3011192161 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7feb30062340 con 0x7feb30104ad0 2024-03-28T18:04:16.827 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:16.826+0000 7feb27e00640 1 -- 172.21.15.22:0/3011192161 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 6 v6) v1 ==== 93+0+4489 (secure 0 0 0) 0x7feb1c0bde40 con 0x7feb30104ad0 2024-03-28T18:04:16.827 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-03-28T18:04:16.827 INFO:teuthology.orchestra.run.smithi022.stdout:{"epoch":6,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14544,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6835/3718238950","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6834","nonce":3718238950},{"type":"v1","addr":"172.21.15.22:6835","nonce":3718238950}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14562,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6837/3676939386","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6836","nonce":3676939386},{"type":"v1","addr":"172.21.15.22:6837","nonce":3676939386}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":24412,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.134:6833/3176125362","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6832","nonce":3176125362},{"type":"v1","addr":"172.21.15.134:6833","nonce":3176125362}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":34442,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.79:6833/856896997","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6832","nonce":856896997},{"type":"v1","addr":"172.21.15.79:6833","nonce":856896997}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":34475,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.79:6835/3365773431","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6834","nonce":3365773431},{"type":"v1","addr":"172.21.15.79:6835","nonce":3365773431}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6}],"filesystems":[]} 2024-03-28T18:04:16.828 INFO:teuthology.orchestra.run.smithi022.stderr:dumped fsmap epoch 6 2024-03-28T18:04:16.830 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:16.829+0000 7feb36200640 1 -- 172.21.15.22:0/3011192161 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7feb140a2880 msgr2=0x7feb140a4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:04:16.830 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:16.829+0000 7feb36200640 1 --2- 172.21.15.22:0/3011192161 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7feb140a2880 0x7feb140a4d40 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7feb20005fd0 tx=0x7feb20005850 comp rx=0 tx=0).stop 2024-03-28T18:04:16.830 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:16.829+0000 7feb36200640 1 -- 172.21.15.22:0/3011192161 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7feb30104ad0 msgr2=0x7feb3019a160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:04:16.830 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:16.829+0000 7feb36200640 1 --2- 172.21.15.22:0/3011192161 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7feb30104ad0 0x7feb3019a160 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7feb1c000c00 tx=0x7feb1c004f10 comp rx=0 tx=0).stop 2024-03-28T18:04:16.831 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:16.829+0000 7feb36200640 1 -- 172.21.15.22:0/3011192161 shutdown_connections 2024-03-28T18:04:16.831 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:16.829+0000 7feb36200640 1 --2- 172.21.15.22:0/3011192161 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7feb140a2880 0x7feb140a4d40 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:04:16.831 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:16.829+0000 7feb36200640 1 --2- 172.21.15.22:0/3011192161 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7feb30105490 0x7feb301a1190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:04:16.831 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:16.829+0000 7feb36200640 1 --2- 172.21.15.22:0/3011192161 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7feb30104ad0 0x7feb3019a160 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:04:16.831 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:16.829+0000 7feb36200640 1 --2- 172.21.15.22:0/3011192161 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7feb301038e0 0x7feb30199c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:04:16.831 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:16.829+0000 7feb36200640 1 -- 172.21.15.22:0/3011192161 >> 172.21.15.22:0/3011192161 conn(0x7feb300fee90 msgr2=0x7feb30107dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:04:16.831 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:16.830+0000 7feb36200640 1 -- 172.21.15.22:0/3011192161 shutdown_connections 2024-03-28T18:04:16.831 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:16.830+0000 7feb36200640 1 -- 172.21.15.22:0/3011192161 wait complete. 2024-03-28T18:04:17.877 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:04:17 smithi134 ceph-mon[35037]: mds.? [v2:172.21.15.79:6834/3365773431,v1:172.21.15.79:6835/3365773431] up:boot 2024-03-28T18:04:17.877 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:04:17 smithi134 ceph-mon[35037]: fsmap 5 up:standby 2024-03-28T18:04:17.877 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:04:17 smithi134 ceph-mon[35037]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-03-28T18:04:17.877 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:04:17 smithi134 ceph-mon[35037]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T18:04:17.877 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:04:17 smithi134 ceph-mon[35037]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T18:04:17.877 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:04:17 smithi134 ceph-mon[35037]: pgmap v349: 1 pgs: 1 active+clean; 577 KiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-28T18:04:17.877 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:04:17 smithi134 ceph-mon[35037]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T18:04:17.877 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:04:17 smithi134 ceph-mon[35037]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-28T18:04:17.877 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:04:17 smithi134 ceph-mon[35037]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-28T18:04:17.877 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:04:17 smithi134 ceph-mon[35037]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T18:04:17.878 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:04:17 smithi134 ceph-mon[35037]: Deploying daemon mds.f on smithi134 2024-03-28T18:04:17.878 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:04:17 smithi134 ceph-mon[35037]: from='client.? 172.21.15.22:0/3011192161' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-28T18:04:17.933 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:04:17 smithi079 ceph-mon[35312]: mds.? [v2:172.21.15.79:6834/3365773431,v1:172.21.15.79:6835/3365773431] up:boot 2024-03-28T18:04:17.933 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:04:17 smithi079 ceph-mon[35312]: fsmap 5 up:standby 2024-03-28T18:04:17.933 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:04:17 smithi079 ceph-mon[35312]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-03-28T18:04:17.933 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:04:17 smithi079 ceph-mon[35312]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T18:04:17.933 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:04:17 smithi079 ceph-mon[35312]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T18:04:17.933 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:04:17 smithi079 ceph-mon[35312]: pgmap v349: 1 pgs: 1 active+clean; 577 KiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-28T18:04:17.933 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:04:17 smithi079 ceph-mon[35312]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T18:04:17.933 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:04:17 smithi079 ceph-mon[35312]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-28T18:04:17.933 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:04:17 smithi079 ceph-mon[35312]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-28T18:04:17.933 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:04:17 smithi079 ceph-mon[35312]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T18:04:17.934 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:04:17 smithi079 ceph-mon[35312]: Deploying daemon mds.f on smithi134 2024-03-28T18:04:17.934 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:04:17 smithi079 ceph-mon[35312]: from='client.? 172.21.15.22:0/3011192161' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-28T18:04:18.032 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:04:17 smithi022 ceph-mon[31985]: mds.? [v2:172.21.15.79:6834/3365773431,v1:172.21.15.79:6835/3365773431] up:boot 2024-03-28T18:04:18.032 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:04:17 smithi022 ceph-mon[31985]: fsmap 5 up:standby 2024-03-28T18:04:18.032 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:04:17 smithi022 ceph-mon[31985]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-03-28T18:04:18.032 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:04:17 smithi022 ceph-mon[31985]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T18:04:18.032 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:04:17 smithi022 ceph-mon[31985]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T18:04:18.032 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:04:17 smithi022 ceph-mon[31985]: pgmap v349: 1 pgs: 1 active+clean; 577 KiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-28T18:04:18.032 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:04:17 smithi022 ceph-mon[31985]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T18:04:18.033 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:04:17 smithi022 ceph-mon[31985]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-28T18:04:18.033 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:04:17 smithi022 ceph-mon[31985]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-28T18:04:18.033 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:04:17 smithi022 ceph-mon[31985]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T18:04:18.033 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:04:17 smithi022 ceph-mon[31985]: Deploying daemon mds.f on smithi134 2024-03-28T18:04:18.033 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:04:17 smithi022 ceph-mon[31985]: from='client.? 172.21.15.22:0/3011192161' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-28T18:04:18.852 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-28T18:04:19.088 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:19.088+0000 7f9f1da00640 1 -- 172.21.15.22:0/1841762160 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9f181028b0 msgr2=0x7f9f18102d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:04:19.088 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:19.088+0000 7f9f1da00640 1 --2- 172.21.15.22:0/1841762160 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9f181028b0 0x7f9f18102d30 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f9f04009900 tx=0x7f9f0402bc30 comp rx=0 tx=0).stop 2024-03-28T18:04:19.088 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:19.088+0000 7f9f1da00640 1 -- 172.21.15.22:0/1841762160 shutdown_connections 2024-03-28T18:04:19.088 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:19.088+0000 7f9f1da00640 1 --2- 172.21.15.22:0/1841762160 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f9f18103270 0x7f9f18109b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:04:19.088 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:19.088+0000 7f9f1da00640 1 --2- 172.21.15.22:0/1841762160 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9f181028b0 0x7f9f18102d30 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:04:19.088 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:19.088+0000 7f9f1da00640 1 --2- 172.21.15.22:0/1841762160 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9f181016c0 0x7f9f18101ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:04:19.088 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:19.088+0000 7f9f1da00640 1 -- 172.21.15.22:0/1841762160 >> 172.21.15.22:0/1841762160 conn(0x7f9f180fcc50 msgr2=0x7f9f180ff070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:04:19.088 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:19.088+0000 7f9f1da00640 1 -- 172.21.15.22:0/1841762160 shutdown_connections 2024-03-28T18:04:19.088 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:19.088+0000 7f9f1da00640 1 -- 172.21.15.22:0/1841762160 wait complete. 2024-03-28T18:04:19.090 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:19.089+0000 7f9f1da00640 1 Processor -- start 2024-03-28T18:04:19.091 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:19.089+0000 7f9f1da00640 1 -- start start 2024-03-28T18:04:19.091 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:19.089+0000 7f9f1da00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9f181016c0 0x7f9f18075450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:04:19.091 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:19.089+0000 7f9f1da00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9f181028b0 0x7f9f18073a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:04:19.091 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:19.089+0000 7f9f1da00640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f9f18103270 0x7f9f18073fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:04:19.091 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:19.089+0000 7f9f1da00640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f1810d370 con 0x7f9f181028b0 2024-03-28T18:04:19.091 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:19.089+0000 7f9f1da00640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f1810d1f0 con 0x7f9f18103270 2024-03-28T18:04:19.091 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:19.089+0000 7f9f1da00640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f1810d4f0 con 0x7f9f181016c0 2024-03-28T18:04:19.091 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:19.090+0000 7f9f16000640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9f181028b0 0x7f9f18073a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:04:19.091 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:19.090+0000 7f9f16000640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9f181028b0 0x7f9f18073a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:55578/0 (socket says 172.21.15.22:55578) 2024-03-28T18:04:19.091 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:19.090+0000 7f9f16000640 1 -- 172.21.15.22:0/4213279241 learned_addr learned my addr 172.21.15.22:0/4213279241 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-28T18:04:19.091 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:19.090+0000 7f9f16a00640 1 --2- 172.21.15.22:0/4213279241 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9f181016c0 0x7f9f18075450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:04:19.091 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:19.090+0000 7f9f17400640 1 --2- 172.21.15.22:0/4213279241 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f9f18103270 0x7f9f18073fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:04:19.091 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:19.090+0000 7f9f16000640 1 -- 172.21.15.22:0/4213279241 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9f181016c0 msgr2=0x7f9f18075450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:04:19.092 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:19.090+0000 7f9f16000640 1 --2- 172.21.15.22:0/4213279241 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9f181016c0 0x7f9f18075450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:04:19.092 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:19.090+0000 7f9f16000640 1 -- 172.21.15.22:0/4213279241 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f9f18103270 msgr2=0x7f9f18073fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:04:19.092 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:19.090+0000 7f9f16000640 1 --2- 172.21.15.22:0/4213279241 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f9f18103270 0x7f9f18073fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:04:19.092 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:19.090+0000 7f9f16000640 1 -- 172.21.15.22:0/4213279241 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9f18074860 con 0x7f9f181028b0 2024-03-28T18:04:19.092 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:19.090+0000 7f9f16000640 1 --2- 172.21.15.22:0/4213279241 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9f181028b0 0x7f9f18073a90 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f9f04000c00 tx=0x7f9f04004f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:04:19.092 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:19.091+0000 7f9f0f400640 1 -- 172.21.15.22:0/4213279241 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f04043020 con 0x7f9f181028b0 2024-03-28T18:04:19.092 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:19.091+0000 7f9f0f400640 1 -- 172.21.15.22:0/4213279241 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f9f040054d0 con 0x7f9f181028b0 2024-03-28T18:04:19.092 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:19.091+0000 7f9f0f400640 1 -- 172.21.15.22:0/4213279241 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f04003d40 con 0x7f9f181028b0 2024-03-28T18:04:19.092 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:19.091+0000 7f9f1da00640 1 -- 172.21.15.22:0/4213279241 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9f181a1b30 con 0x7f9f181028b0 2024-03-28T18:04:19.092 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:19.091+0000 7f9f1da00640 1 -- 172.21.15.22:0/4213279241 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9f181a2080 con 0x7f9f181028b0 2024-03-28T18:04:19.092 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:19.092+0000 7f9f1da00640 1 -- 172.21.15.22:0/4213279241 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9f181089e0 con 0x7f9f181028b0 2024-03-28T18:04:19.094 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:19.095+0000 7f9f0f400640 1 -- 172.21.15.22:0/4213279241 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 16) v1 ==== 137290+0+0 (secure 0 0 0) 0x7f9f04046020 con 0x7f9f181028b0 2024-03-28T18:04:19.096 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:19.096+0000 7f9f0f400640 1 --2- 172.21.15.22:0/4213279241 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7f9efc0a2800 0x7f9efc0a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:04:19.096 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:19.096+0000 7f9f0f400640 1 -- 172.21.15.22:0/4213279241 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f9f040f0960 con 0x7f9f181028b0 2024-03-28T18:04:19.096 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:19.096+0000 7f9f16a00640 1 --2- 172.21.15.22:0/4213279241 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7f9efc0a2800 0x7f9efc0a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:04:19.097 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:19.097+0000 7f9f16a00640 1 --2- 172.21.15.22:0/4213279241 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7f9efc0a2800 0x7f9efc0a4cc0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f9f08005fd0 tx=0x7f9f0800b390 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:04:19.100 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:19.101+0000 7f9f0f400640 1 -- 172.21.15.22:0/4213279241 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f9f0404d020 con 0x7f9f181028b0 2024-03-28T18:04:19.349 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:19.349+0000 7f9f1da00640 1 -- 172.21.15.22:0/4213279241 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f9f1802d070 con 0x7f9f181028b0 2024-03-28T18:04:19.350 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:19.350+0000 7f9f0f400640 1 -- 172.21.15.22:0/4213279241 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 6 v6) v1 ==== 93+0+4489 (secure 0 0 0) 0x7f9f040bdf20 con 0x7f9f181028b0 2024-03-28T18:04:19.350 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-03-28T18:04:19.350 INFO:teuthology.orchestra.run.smithi022.stdout:{"epoch":6,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14544,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6835/3718238950","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6834","nonce":3718238950},{"type":"v1","addr":"172.21.15.22:6835","nonce":3718238950}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14562,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6837/3676939386","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6836","nonce":3676939386},{"type":"v1","addr":"172.21.15.22:6837","nonce":3676939386}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":24412,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.134:6833/3176125362","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6832","nonce":3176125362},{"type":"v1","addr":"172.21.15.134:6833","nonce":3176125362}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":34442,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.79:6833/856896997","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6832","nonce":856896997},{"type":"v1","addr":"172.21.15.79:6833","nonce":856896997}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":34475,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.79:6835/3365773431","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6834","nonce":3365773431},{"type":"v1","addr":"172.21.15.79:6835","nonce":3365773431}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6}],"filesystems":[]} 2024-03-28T18:04:19.351 INFO:teuthology.orchestra.run.smithi022.stderr:dumped fsmap epoch 6 2024-03-28T18:04:19.353 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:19.353+0000 7f9f1da00640 1 -- 172.21.15.22:0/4213279241 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7f9efc0a2800 msgr2=0x7f9efc0a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:04:19.353 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:19.353+0000 7f9f1da00640 1 --2- 172.21.15.22:0/4213279241 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7f9efc0a2800 0x7f9efc0a4cc0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f9f08005fd0 tx=0x7f9f0800b390 comp rx=0 tx=0).stop 2024-03-28T18:04:19.353 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:19.353+0000 7f9f1da00640 1 -- 172.21.15.22:0/4213279241 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9f181028b0 msgr2=0x7f9f18073a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:04:19.353 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:19.353+0000 7f9f1da00640 1 --2- 172.21.15.22:0/4213279241 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9f181028b0 0x7f9f18073a90 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f9f04000c00 tx=0x7f9f04004f10 comp rx=0 tx=0).stop 2024-03-28T18:04:19.353 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:19.353+0000 7f9f1da00640 1 -- 172.21.15.22:0/4213279241 shutdown_connections 2024-03-28T18:04:19.353 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:19.353+0000 7f9f1da00640 1 --2- 172.21.15.22:0/4213279241 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7f9efc0a2800 0x7f9efc0a4cc0 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:04:19.353 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:19.353+0000 7f9f1da00640 1 --2- 172.21.15.22:0/4213279241 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f9f18103270 0x7f9f18073fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:04:19.354 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:19.353+0000 7f9f1da00640 1 --2- 172.21.15.22:0/4213279241 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9f181028b0 0x7f9f18073a90 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:04:19.354 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:19.353+0000 7f9f1da00640 1 --2- 172.21.15.22:0/4213279241 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9f181016c0 0x7f9f18075450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:04:19.354 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:19.353+0000 7f9f1da00640 1 -- 172.21.15.22:0/4213279241 >> 172.21.15.22:0/4213279241 conn(0x7f9f180fcc50 msgr2=0x7f9f18105bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:04:19.354 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:19.353+0000 7f9f1da00640 1 -- 172.21.15.22:0/4213279241 shutdown_connections 2024-03-28T18:04:19.354 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:19.353+0000 7f9f1da00640 1 -- 172.21.15.22:0/4213279241 wait complete. 2024-03-28T18:04:19.925 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:04:19 smithi134 ceph-mon[35037]: pgmap v350: 1 pgs: 1 active+clean; 577 KiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-28T18:04:19.925 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:04:19 smithi134 ceph-mon[35037]: from='client.? 172.21.15.22:0/4213279241' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-28T18:04:19.933 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:04:19 smithi079 ceph-mon[35312]: pgmap v350: 1 pgs: 1 active+clean; 577 KiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-28T18:04:19.933 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:04:19 smithi079 ceph-mon[35312]: from='client.? 172.21.15.22:0/4213279241' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-28T18:04:20.032 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:04:19 smithi022 ceph-mon[31985]: pgmap v350: 1 pgs: 1 active+clean; 577 KiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-28T18:04:20.032 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:04:19 smithi022 ceph-mon[31985]: from='client.? 172.21.15.22:0/4213279241' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-28T18:04:20.267 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:04:20 smithi134 podman[65118]: 2024-03-28 18:04:20.115186344 +0000 UTC m=+0.264163583 container create 7fe8640da6db60521ec0366a2d3b24b04b7295e51c6ba6786f37d5cb444425e0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, ceph=True, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.build-date=20240319, org.label-schema.schema-version=1.0) 2024-03-28T18:04:20.619 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:04:20 smithi134 podman[65118]: 2024-03-28 18:04:20.376549194 +0000 UTC m=+0.525526437 container init 7fe8640da6db60521ec0366a2d3b24b04b7295e51c6ba6786f37d5cb444425e0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, org.label-schema.build-date=20240319, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_CLEAN=True) 2024-03-28T18:04:20.620 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:04:20 smithi134 podman[65118]: 2024-03-28 18:04:20.386660614 +0000 UTC m=+0.535637820 container start 7fe8640da6db60521ec0366a2d3b24b04b7295e51c6ba6786f37d5cb444425e0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, ceph=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, org.label-schema.build-date=20240319, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD) 2024-03-28T18:04:20.620 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:04:20 smithi134 ceph-mds[65138]: -- 172.21.15.134:0/4196225394 <== mon.1 v2:172.21.15.134:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x5629a955da40 con 0x5629a9530800 2024-03-28T18:04:20.620 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:04:20 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[65133]: starting mds.f at 2024-03-28T18:04:20.620 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:04:20 smithi134 bash[65118]: 7fe8640da6db60521ec0366a2d3b24b04b7295e51c6ba6786f37d5cb444425e0 2024-03-28T18:04:20.620 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:04:20 smithi134 systemd[1]: Started Ceph mds.f for fd220748-ed2b-11ee-95d0-87774f69a715. 2024-03-28T18:04:21.372 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-28T18:04:21.612 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:21.612+0000 7f1d42600640 1 -- 172.21.15.22:0/3446467121 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1d3c0772a0 msgr2=0x7f1d3c077700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:04:21.612 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:21.612+0000 7f1d42600640 1 --2- 172.21.15.22:0/3446467121 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1d3c0772a0 0x7f1d3c077700 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f1d28009900 tx=0x7f1d2802bc40 comp rx=0 tx=0).stop 2024-03-28T18:04:21.612 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:21.612+0000 7f1d42600640 1 -- 172.21.15.22:0/3446467121 shutdown_connections 2024-03-28T18:04:21.612 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:21.612+0000 7f1d42600640 1 --2- 172.21.15.22:0/3446467121 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f1d3c077c40 0x7f1d3c0784e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:04:21.612 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:21.612+0000 7f1d42600640 1 --2- 172.21.15.22:0/3446467121 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1d3c0772a0 0x7f1d3c077700 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:04:21.612 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:21.612+0000 7f1d42600640 1 --2- 172.21.15.22:0/3446467121 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1d3c076060 0x7f1d3c076480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:04:21.613 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:21.612+0000 7f1d42600640 1 -- 172.21.15.22:0/3446467121 >> 172.21.15.22:0/3446467121 conn(0x7f1d3c0fee50 msgr2=0x7f1d3c101290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:04:21.613 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:21.612+0000 7f1d42600640 1 -- 172.21.15.22:0/3446467121 shutdown_connections 2024-03-28T18:04:21.613 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:21.612+0000 7f1d42600640 1 -- 172.21.15.22:0/3446467121 wait complete. 2024-03-28T18:04:21.614 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:21.613+0000 7f1d42600640 1 Processor -- start 2024-03-28T18:04:21.614 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:21.613+0000 7f1d42600640 1 -- start start 2024-03-28T18:04:21.614 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:21.614+0000 7f1d42600640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f1d3c076060 0x7f1d3c19df90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:04:21.614 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:21.614+0000 7f1d42600640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1d3c0772a0 0x7f1d3c19e4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:04:21.614 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:21.614+0000 7f1d42600640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1d3c077c40 0x7f1d3c1a5540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:04:21.615 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:21.614+0000 7f1d42600640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d3c113980 con 0x7f1d3c0772a0 2024-03-28T18:04:21.615 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:21.614+0000 7f1d42600640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d3c113800 con 0x7f1d3c076060 2024-03-28T18:04:21.615 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:21.614+0000 7f1d42600640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d3c113b00 con 0x7f1d3c077c40 2024-03-28T18:04:21.615 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:21.614+0000 7f1d3aa00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1d3c0772a0 0x7f1d3c19e4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:04:21.615 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:21.614+0000 7f1d3be00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1d3c077c40 0x7f1d3c1a5540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:04:21.615 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:21.614+0000 7f1d3b400640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f1d3c076060 0x7f1d3c19df90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:04:21.615 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:21.614+0000 7f1d3aa00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1d3c0772a0 0x7f1d3c19e4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:55594/0 (socket says 172.21.15.22:55594) 2024-03-28T18:04:21.615 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:21.615+0000 7f1d3aa00640 1 -- 172.21.15.22:0/2284307864 learned_addr learned my addr 172.21.15.22:0/2284307864 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-28T18:04:21.616 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:21.615+0000 7f1d3aa00640 1 -- 172.21.15.22:0/2284307864 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1d3c077c40 msgr2=0x7f1d3c1a5540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:04:21.616 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:21.615+0000 7f1d3aa00640 1 --2- 172.21.15.22:0/2284307864 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1d3c077c40 0x7f1d3c1a5540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:04:21.616 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:21.615+0000 7f1d3aa00640 1 -- 172.21.15.22:0/2284307864 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f1d3c076060 msgr2=0x7f1d3c19df90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:04:21.616 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:21.615+0000 7f1d3aa00640 1 --2- 172.21.15.22:0/2284307864 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f1d3c076060 0x7f1d3c19df90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:04:21.616 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:21.615+0000 7f1d3aa00640 1 -- 172.21.15.22:0/2284307864 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1d3c1a5bb0 con 0x7f1d3c0772a0 2024-03-28T18:04:21.617 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:21.616+0000 7f1d3aa00640 1 --2- 172.21.15.22:0/2284307864 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1d3c0772a0 0x7f1d3c19e4d0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f1d28000c00 tx=0x7f1d28004f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:04:21.617 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:21.616+0000 7f1d33e00640 1 -- 172.21.15.22:0/2284307864 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d28043020 con 0x7f1d3c0772a0 2024-03-28T18:04:21.617 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:21.616+0000 7f1d33e00640 1 -- 172.21.15.22:0/2284307864 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f1d280054d0 con 0x7f1d3c0772a0 2024-03-28T18:04:21.617 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:21.616+0000 7f1d42600640 1 -- 172.21.15.22:0/2284307864 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1d3c1a5e40 con 0x7f1d3c0772a0 2024-03-28T18:04:21.617 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:21.616+0000 7f1d33e00640 1 -- 172.21.15.22:0/2284307864 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d28003c90 con 0x7f1d3c0772a0 2024-03-28T18:04:21.617 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:21.616+0000 7f1d42600640 1 -- 172.21.15.22:0/2284307864 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1d3c1a6300 con 0x7f1d3c0772a0 2024-03-28T18:04:21.617 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:21.617+0000 7f1d42600640 1 -- 172.21.15.22:0/2284307864 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1d3c110390 con 0x7f1d3c0772a0 2024-03-28T18:04:21.620 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:21.620+0000 7f1d33e00640 1 -- 172.21.15.22:0/2284307864 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 16) v1 ==== 137290+0+0 (secure 0 0 0) 0x7f1d28046020 con 0x7f1d3c0772a0 2024-03-28T18:04:21.621 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:21.621+0000 7f1d33e00640 1 --2- 172.21.15.22:0/2284307864 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7f1d200a2800 0x7f1d200a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:04:21.621 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:21.622+0000 7f1d33e00640 1 -- 172.21.15.22:0/2284307864 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f1d280f0820 con 0x7f1d3c0772a0 2024-03-28T18:04:21.621 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:21.622+0000 7f1d3b400640 1 --2- 172.21.15.22:0/2284307864 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7f1d200a2800 0x7f1d200a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:04:21.622 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:21.622+0000 7f1d3b400640 1 --2- 172.21.15.22:0/2284307864 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7f1d200a2800 0x7f1d200a4cc0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f1d2c005fd0 tx=0x7f1d2c00b390 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:04:21.625 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:21.626+0000 7f1d33e00640 1 -- 172.21.15.22:0/2284307864 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f1d2804d020 con 0x7f1d3c0772a0 2024-03-28T18:04:21.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:04:21 smithi022 ceph-mon[31985]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T18:04:21.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:04:21 smithi022 ceph-mon[31985]: pgmap v351: 1 pgs: 1 active+clean; 577 KiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-28T18:04:21.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:04:21 smithi022 ceph-mon[31985]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T18:04:21.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:04:21 smithi022 ceph-mon[31985]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T18:04:21.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:04:21 smithi022 ceph-mon[31985]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-28T18:04:21.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:04:21 smithi022 ceph-mon[31985]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-28T18:04:21.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:04:21 smithi022 ceph-mon[31985]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T18:04:21.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:04:21 smithi022 ceph-mon[31985]: Deploying daemon mds.c on smithi134 2024-03-28T18:04:21.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:04:21 smithi022 ceph-mon[31985]: mds.? [v2:172.21.15.134:6834/3370549207,v1:172.21.15.134:6835/3370549207] up:boot 2024-03-28T18:04:21.783 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:04:21 smithi022 ceph-mon[31985]: fsmap 6 up:standby 2024-03-28T18:04:21.783 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:04:21 smithi022 ceph-mon[31985]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-03-28T18:04:21.861 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:04:21 smithi134 ceph-mon[35037]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T18:04:21.861 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:04:21 smithi134 ceph-mon[35037]: pgmap v351: 1 pgs: 1 active+clean; 577 KiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-28T18:04:21.861 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:04:21 smithi134 ceph-mon[35037]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T18:04:21.861 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:04:21 smithi134 ceph-mon[35037]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T18:04:21.861 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:04:21 smithi134 ceph-mon[35037]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-28T18:04:21.861 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:04:21 smithi134 ceph-mon[35037]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-28T18:04:21.861 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:04:21 smithi134 ceph-mon[35037]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T18:04:21.861 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:04:21 smithi134 ceph-mon[35037]: Deploying daemon mds.c on smithi134 2024-03-28T18:04:21.861 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:04:21 smithi134 ceph-mon[35037]: mds.? [v2:172.21.15.134:6834/3370549207,v1:172.21.15.134:6835/3370549207] up:boot 2024-03-28T18:04:21.862 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:04:21 smithi134 ceph-mon[35037]: fsmap 6 up:standby 2024-03-28T18:04:21.862 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:04:21 smithi134 ceph-mon[35037]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-03-28T18:04:21.866 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:21.866+0000 7f1d42600640 1 -- 172.21.15.22:0/2284307864 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f1d3c02d070 con 0x7f1d3c0772a0 2024-03-28T18:04:21.867 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:21.867+0000 7f1d33e00640 1 -- 172.21.15.22:0/2284307864 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) v1 ==== 93+0+5283 (secure 0 0 0) 0x7f1d280bdcc0 con 0x7f1d3c0772a0 2024-03-28T18:04:21.867 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-03-28T18:04:21.868 INFO:teuthology.orchestra.run.smithi022.stdout:{"epoch":7,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14544,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6835/3718238950","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6834","nonce":3718238950},{"type":"v1","addr":"172.21.15.22:6835","nonce":3718238950}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14562,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6837/3676939386","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6836","nonce":3676939386},{"type":"v1","addr":"172.21.15.22:6837","nonce":3676939386}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":24412,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.134:6833/3176125362","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6832","nonce":3176125362},{"type":"v1","addr":"172.21.15.134:6833","nonce":3176125362}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24463,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.134:6835/3370549207","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6834","nonce":3370549207},{"type":"v1","addr":"172.21.15.134:6835","nonce":3370549207}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":7},{"gid":34442,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.79:6833/856896997","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6832","nonce":856896997},{"type":"v1","addr":"172.21.15.79:6833","nonce":856896997}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":34475,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.79:6835/3365773431","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6834","nonce":3365773431},{"type":"v1","addr":"172.21.15.79:6835","nonce":3365773431}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6}],"filesystems":[]} 2024-03-28T18:04:21.868 INFO:teuthology.orchestra.run.smithi022.stderr:dumped fsmap epoch 7 2024-03-28T18:04:21.870 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:21.870+0000 7f1d42600640 1 -- 172.21.15.22:0/2284307864 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7f1d200a2800 msgr2=0x7f1d200a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:04:21.870 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:21.870+0000 7f1d42600640 1 --2- 172.21.15.22:0/2284307864 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7f1d200a2800 0x7f1d200a4cc0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f1d2c005fd0 tx=0x7f1d2c00b390 comp rx=0 tx=0).stop 2024-03-28T18:04:21.870 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:21.870+0000 7f1d42600640 1 -- 172.21.15.22:0/2284307864 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1d3c0772a0 msgr2=0x7f1d3c19e4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:04:21.870 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:21.870+0000 7f1d42600640 1 --2- 172.21.15.22:0/2284307864 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1d3c0772a0 0x7f1d3c19e4d0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f1d28000c00 tx=0x7f1d28004f10 comp rx=0 tx=0).stop 2024-03-28T18:04:21.870 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:21.870+0000 7f1d42600640 1 -- 172.21.15.22:0/2284307864 shutdown_connections 2024-03-28T18:04:21.870 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:21.870+0000 7f1d42600640 1 --2- 172.21.15.22:0/2284307864 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7f1d200a2800 0x7f1d200a4cc0 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:04:21.870 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:21.870+0000 7f1d42600640 1 --2- 172.21.15.22:0/2284307864 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1d3c077c40 0x7f1d3c1a5540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:04:21.871 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:21.870+0000 7f1d42600640 1 --2- 172.21.15.22:0/2284307864 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1d3c0772a0 0x7f1d3c19e4d0 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:04:21.871 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:21.870+0000 7f1d42600640 1 --2- 172.21.15.22:0/2284307864 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f1d3c076060 0x7f1d3c19df90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:04:21.871 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:21.870+0000 7f1d42600640 1 -- 172.21.15.22:0/2284307864 >> 172.21.15.22:0/2284307864 conn(0x7f1d3c0fee50 msgr2=0x7f1d3c100470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:04:21.871 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:21.870+0000 7f1d42600640 1 -- 172.21.15.22:0/2284307864 shutdown_connections 2024-03-28T18:04:21.871 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:21.870+0000 7f1d42600640 1 -- 172.21.15.22:0/2284307864 wait complete. 2024-03-28T18:04:21.933 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:04:21 smithi079 ceph-mon[35312]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T18:04:21.933 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:04:21 smithi079 ceph-mon[35312]: pgmap v351: 1 pgs: 1 active+clean; 577 KiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-28T18:04:21.933 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:04:21 smithi079 ceph-mon[35312]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T18:04:21.933 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:04:21 smithi079 ceph-mon[35312]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T18:04:21.933 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:04:21 smithi079 ceph-mon[35312]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-28T18:04:21.933 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:04:21 smithi079 ceph-mon[35312]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-28T18:04:21.933 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:04:21 smithi079 ceph-mon[35312]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T18:04:21.933 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:04:21 smithi079 ceph-mon[35312]: Deploying daemon mds.c on smithi134 2024-03-28T18:04:21.933 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:04:21 smithi079 ceph-mon[35312]: mds.? [v2:172.21.15.134:6834/3370549207,v1:172.21.15.134:6835/3370549207] up:boot 2024-03-28T18:04:21.933 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:04:21 smithi079 ceph-mon[35312]: fsmap 6 up:standby 2024-03-28T18:04:21.933 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:04:21 smithi079 ceph-mon[35312]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-03-28T18:04:22.816 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:04:22 smithi134 ceph-mon[35037]: from='client.? 172.21.15.22:0/2284307864' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-28T18:04:22.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:04:22 smithi079 ceph-mon[35312]: from='client.? 172.21.15.22:0/2284307864' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-28T18:04:23.032 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:04:22 smithi022 ceph-mon[31985]: from='client.? 172.21.15.22:0/2284307864' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-28T18:04:23.737 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:04:23 smithi022 ceph-mon[31985]: pgmap v352: 1 pgs: 1 active+clean; 577 KiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-28T18:04:23.892 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-28T18:04:23.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:04:23 smithi134 ceph-mon[35037]: pgmap v352: 1 pgs: 1 active+clean; 577 KiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-28T18:04:23.925 INFO:journalctl@ceph.mds.c.smithi134.stdout:Mar 28 18:04:23 smithi134 systemd[1]: Starting Ceph mds.c for fd220748-ed2b-11ee-95d0-87774f69a715... 2024-03-28T18:04:23.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:04:23 smithi079 ceph-mon[35312]: pgmap v352: 1 pgs: 1 active+clean; 577 KiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-28T18:04:24.111 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:24.111+0000 7ff850400640 1 -- 172.21.15.22:0/1685031260 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff848109890 msgr2=0x7ff848110120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:04:24.112 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:24.111+0000 7ff850400640 1 --2- 172.21.15.22:0/1685031260 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff848109890 0x7ff848110120 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7ff840009ca0 tx=0x7ff84002c0f0 comp rx=0 tx=0).stop 2024-03-28T18:04:24.112 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:24.111+0000 7ff850400640 1 -- 172.21.15.22:0/1685031260 shutdown_connections 2024-03-28T18:04:24.112 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:24.111+0000 7ff850400640 1 --2- 172.21.15.22:0/1685031260 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff848109890 0x7ff848110120 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:04:24.112 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:24.111+0000 7ff850400640 1 --2- 172.21.15.22:0/1685031260 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ff848108ed0 0x7ff848109350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:04:24.112 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:24.111+0000 7ff850400640 1 --2- 172.21.15.22:0/1685031260 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff848107ce0 0x7ff848108100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:04:24.112 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:24.111+0000 7ff850400640 1 -- 172.21.15.22:0/1685031260 >> 172.21.15.22:0/1685031260 conn(0x7ff848076cb0 msgr2=0x7ff8480790d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:04:24.112 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:24.112+0000 7ff850400640 1 -- 172.21.15.22:0/1685031260 shutdown_connections 2024-03-28T18:04:24.112 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:24.112+0000 7ff850400640 1 -- 172.21.15.22:0/1685031260 wait complete. 2024-03-28T18:04:24.114 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:24.112+0000 7ff850400640 1 Processor -- start 2024-03-28T18:04:24.114 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:24.113+0000 7ff850400640 1 -- start start 2024-03-28T18:04:24.114 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:24.113+0000 7ff850400640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ff848107ce0 0x7ff84819df30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:04:24.114 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:24.113+0000 7ff850400640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff848108ed0 0x7ff84819e470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:04:24.115 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:24.113+0000 7ff850400640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff848109890 0x7ff8481a54e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:04:24.115 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:24.113+0000 7ff850400640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff848113940 con 0x7ff848109890 2024-03-28T18:04:24.115 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:24.113+0000 7ff850400640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8481137c0 con 0x7ff848107ce0 2024-03-28T18:04:24.115 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:24.113+0000 7ff850400640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff848113ac0 con 0x7ff848108ed0 2024-03-28T18:04:24.115 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:24.113+0000 7ff84e400640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff848109890 0x7ff8481a54e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:04:24.115 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:24.113+0000 7ff84e400640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff848109890 0x7ff8481a54e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:55620/0 (socket says 172.21.15.22:55620) 2024-03-28T18:04:24.115 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:24.113+0000 7ff84e400640 1 -- 172.21.15.22:0/3408853003 learned_addr learned my addr 172.21.15.22:0/3408853003 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-28T18:04:24.115 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:24.113+0000 7ff84d000640 1 --2- 172.21.15.22:0/3408853003 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff848108ed0 0x7ff84819e470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:04:24.115 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:24.114+0000 7ff84da00640 1 --2- 172.21.15.22:0/3408853003 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ff848107ce0 0x7ff84819df30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:04:24.115 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:24.114+0000 7ff84e400640 1 -- 172.21.15.22:0/3408853003 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff848108ed0 msgr2=0x7ff84819e470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:04:24.115 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:24.114+0000 7ff84e400640 1 --2- 172.21.15.22:0/3408853003 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff848108ed0 0x7ff84819e470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:04:24.115 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:24.114+0000 7ff84e400640 1 -- 172.21.15.22:0/3408853003 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ff848107ce0 msgr2=0x7ff84819df30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:04:24.115 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:24.114+0000 7ff84e400640 1 --2- 172.21.15.22:0/3408853003 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ff848107ce0 0x7ff84819df30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:04:24.116 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:24.114+0000 7ff84e400640 1 -- 172.21.15.22:0/3408853003 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff8481a5b50 con 0x7ff848109890 2024-03-28T18:04:24.116 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:24.114+0000 7ff84e400640 1 --2- 172.21.15.22:0/3408853003 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff848109890 0x7ff8481a54e0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7ff84002c630 tx=0x7ff840032980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:04:24.116 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:24.114+0000 7ff846000640 1 -- 172.21.15.22:0/3408853003 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff840039050 con 0x7ff848109890 2024-03-28T18:04:24.116 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:24.114+0000 7ff846000640 1 -- 172.21.15.22:0/3408853003 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7ff84003a040 con 0x7ff848109890 2024-03-28T18:04:24.116 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:24.114+0000 7ff850400640 1 -- 172.21.15.22:0/3408853003 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff8481a5de0 con 0x7ff848109890 2024-03-28T18:04:24.116 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:24.114+0000 7ff846000640 1 -- 172.21.15.22:0/3408853003 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff840040450 con 0x7ff848109890 2024-03-28T18:04:24.116 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:24.115+0000 7ff850400640 1 -- 172.21.15.22:0/3408853003 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff8481a62a0 con 0x7ff848109890 2024-03-28T18:04:24.116 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:24.116+0000 7ff850400640 1 -- 172.21.15.22:0/3408853003 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff84810efa0 con 0x7ff848109890 2024-03-28T18:04:24.117 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:24.118+0000 7ff846000640 1 -- 172.21.15.22:0/3408853003 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 16) v1 ==== 137290+0+0 (secure 0 0 0) 0x7ff840003c70 con 0x7ff848109890 2024-03-28T18:04:24.119 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:24.119+0000 7ff846000640 1 --2- 172.21.15.22:0/3408853003 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7ff8300a2880 0x7ff8300a4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:04:24.119 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:24.119+0000 7ff846000640 1 -- 172.21.15.22:0/3408853003 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 7127+0+0 (secure 0 0 0) 0x7ff8400f12e0 con 0x7ff848109890 2024-03-28T18:04:24.119 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:24.119+0000 7ff84da00640 1 --2- 172.21.15.22:0/3408853003 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7ff8300a2880 0x7ff8300a4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:04:24.120 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:24.120+0000 7ff84da00640 1 --2- 172.21.15.22:0/3408853003 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7ff8300a2880 0x7ff8300a4d40 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7ff83c0058c0 tx=0x7ff83c00a4c0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:04:24.127 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:24.128+0000 7ff846000640 1 -- 172.21.15.22:0/3408853003 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7ff8400be820 con 0x7ff848109890 2024-03-28T18:04:24.271 INFO:journalctl@ceph.mds.c.smithi134.stdout:Mar 28 18:04:23 smithi134 podman[65440]: 2024-03-28 18:04:23.959562588 +0000 UTC m=+0.126885049 container create a8bde952d364e61b963bfa262fd97437d52268bc669245618881bafe3c922e84 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-c, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.build-date=20240319, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2) 2024-03-28T18:04:24.272 INFO:journalctl@ceph.mds.c.smithi134.stdout:Mar 28 18:04:23 smithi134 podman[65440]: 2024-03-28 18:04:23.865438822 +0000 UTC m=+0.032761246 image pull c664afe56bb01db4f9afa9532c20598e201e664f2e9ba2be50db0d52674a26d2 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc 2024-03-28T18:04:24.272 INFO:journalctl@ceph.mds.c.smithi134.stdout:Mar 28 18:04:24 smithi134 podman[65440]: 2024-03-28 18:04:24.271543211 +0000 UTC m=+0.438865679 container init a8bde952d364e61b963bfa262fd97437d52268bc669245618881bafe3c922e84 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-c, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240319, org.label-schema.license=GPLv2, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_BRANCH=HEAD, ceph=True) 2024-03-28T18:04:24.373 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:24.373+0000 7ff850400640 1 -- 172.21.15.22:0/3408853003 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff84802d070 con 0x7ff848109890 2024-03-28T18:04:24.374 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:24.374+0000 7ff846000640 1 -- 172.21.15.22:0/3408853003 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) v1 ==== 93+0+5283 (secure 0 0 0) 0x7ff8400be120 con 0x7ff848109890 2024-03-28T18:04:24.375 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-03-28T18:04:24.375 INFO:teuthology.orchestra.run.smithi022.stdout:{"epoch":7,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14544,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6835/3718238950","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6834","nonce":3718238950},{"type":"v1","addr":"172.21.15.22:6835","nonce":3718238950}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14562,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6837/3676939386","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6836","nonce":3676939386},{"type":"v1","addr":"172.21.15.22:6837","nonce":3676939386}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":24412,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.134:6833/3176125362","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6832","nonce":3176125362},{"type":"v1","addr":"172.21.15.134:6833","nonce":3176125362}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24463,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.134:6835/3370549207","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6834","nonce":3370549207},{"type":"v1","addr":"172.21.15.134:6835","nonce":3370549207}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":7},{"gid":34442,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.79:6833/856896997","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6832","nonce":856896997},{"type":"v1","addr":"172.21.15.79:6833","nonce":856896997}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":34475,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.79:6835/3365773431","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6834","nonce":3365773431},{"type":"v1","addr":"172.21.15.79:6835","nonce":3365773431}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6}],"filesystems":[]} 2024-03-28T18:04:24.375 INFO:teuthology.orchestra.run.smithi022.stderr:dumped fsmap epoch 7 2024-03-28T18:04:24.378 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:24.377+0000 7ff850400640 1 -- 172.21.15.22:0/3408853003 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7ff8300a2880 msgr2=0x7ff8300a4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:04:24.378 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:24.377+0000 7ff850400640 1 --2- 172.21.15.22:0/3408853003 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7ff8300a2880 0x7ff8300a4d40 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7ff83c0058c0 tx=0x7ff83c00a4c0 comp rx=0 tx=0).stop 2024-03-28T18:04:24.378 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:24.377+0000 7ff850400640 1 -- 172.21.15.22:0/3408853003 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff848109890 msgr2=0x7ff8481a54e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:04:24.378 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:24.377+0000 7ff850400640 1 --2- 172.21.15.22:0/3408853003 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff848109890 0x7ff8481a54e0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7ff84002c630 tx=0x7ff840032980 comp rx=0 tx=0).stop 2024-03-28T18:04:24.378 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:24.377+0000 7ff850400640 1 -- 172.21.15.22:0/3408853003 shutdown_connections 2024-03-28T18:04:24.378 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:24.377+0000 7ff850400640 1 --2- 172.21.15.22:0/3408853003 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7ff8300a2880 0x7ff8300a4d40 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:04:24.378 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:24.377+0000 7ff850400640 1 --2- 172.21.15.22:0/3408853003 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff848109890 0x7ff8481a54e0 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:04:24.378 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:24.377+0000 7ff850400640 1 --2- 172.21.15.22:0/3408853003 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff848108ed0 0x7ff84819e470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:04:24.378 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:24.377+0000 7ff850400640 1 --2- 172.21.15.22:0/3408853003 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ff848107ce0 0x7ff84819df30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:04:24.378 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:24.377+0000 7ff850400640 1 -- 172.21.15.22:0/3408853003 >> 172.21.15.22:0/3408853003 conn(0x7ff848076cb0 msgr2=0x7ff84810c170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:04:24.378 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:24.378+0000 7ff850400640 1 -- 172.21.15.22:0/3408853003 shutdown_connections 2024-03-28T18:04:24.378 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:24.378+0000 7ff850400640 1 -- 172.21.15.22:0/3408853003 wait complete. 2024-03-28T18:04:24.611 INFO:journalctl@ceph.mds.c.smithi134.stdout:Mar 28 18:04:24 smithi134 podman[65440]: 2024-03-28 18:04:24.280719856 +0000 UTC m=+0.448042304 container start a8bde952d364e61b963bfa262fd97437d52268bc669245618881bafe3c922e84 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-c, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240319, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, io.buildah.version=1.33.5) 2024-03-28T18:04:24.611 INFO:journalctl@ceph.mds.c.smithi134.stdout:Mar 28 18:04:24 smithi134 ceph-mds[65458]: -- 172.21.15.134:0/2497816895 <== mon.1 v2:172.21.15.134:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x556204891a40 con 0x556204864c00 2024-03-28T18:04:24.611 INFO:journalctl@ceph.mds.c.smithi134.stdout:Mar 28 18:04:24 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-c[65454]: starting mds.c at 2024-03-28T18:04:24.611 INFO:journalctl@ceph.mds.c.smithi134.stdout:Mar 28 18:04:24 smithi134 bash[65440]: a8bde952d364e61b963bfa262fd97437d52268bc669245618881bafe3c922e84 2024-03-28T18:04:24.611 INFO:journalctl@ceph.mds.c.smithi134.stdout:Mar 28 18:04:24 smithi134 systemd[1]: Started Ceph mds.c for fd220748-ed2b-11ee-95d0-87774f69a715. 2024-03-28T18:04:24.695 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:04:24 smithi022 ceph-mon[31985]: from='client.? 172.21.15.22:0/3408853003' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-28T18:04:24.695 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:04:24 smithi022 ceph-mon[31985]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T18:04:24.695 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:04:24 smithi022 ceph-mon[31985]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T18:04:24.695 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:04:24 smithi022 ceph-mon[31985]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T18:04:24.695 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:04:24 smithi022 ceph-mon[31985]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-28T18:04:24.695 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:04:24 smithi022 ceph-mon[31985]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-28T18:04:24.695 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:04:24 smithi022 ceph-mon[31985]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T18:04:24.925 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:04:24 smithi134 ceph-mon[35037]: from='client.? 172.21.15.22:0/3408853003' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-28T18:04:24.925 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:04:24 smithi134 ceph-mon[35037]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T18:04:24.925 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:04:24 smithi134 ceph-mon[35037]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T18:04:24.925 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:04:24 smithi134 ceph-mon[35037]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T18:04:24.925 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:04:24 smithi134 ceph-mon[35037]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-28T18:04:24.925 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:04:24 smithi134 ceph-mon[35037]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-28T18:04:24.925 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:04:24 smithi134 ceph-mon[35037]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T18:04:24.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:04:24 smithi079 ceph-mon[35312]: from='client.? 172.21.15.22:0/3408853003' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-28T18:04:24.933 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:04:24 smithi079 ceph-mon[35312]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T18:04:24.933 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:04:24 smithi079 ceph-mon[35312]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T18:04:24.933 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:04:24 smithi079 ceph-mon[35312]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T18:04:24.933 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:04:24 smithi079 ceph-mon[35312]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-28T18:04:24.933 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:04:24 smithi079 ceph-mon[35312]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-28T18:04:24.933 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:04:24 smithi079 ceph-mon[35312]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T18:04:25.876 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:04:25 smithi079 ceph-mon[35312]: Deploying daemon mds.h on smithi079 2024-03-28T18:04:25.876 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:04:25 smithi079 ceph-mon[35312]: pgmap v353: 1 pgs: 1 active+clean; 577 KiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-28T18:04:25.876 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:04:25 smithi079 ceph-mon[35312]: mds.? [v2:172.21.15.134:6836/899825581,v1:172.21.15.134:6837/899825581] up:boot 2024-03-28T18:04:25.876 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:04:25 smithi079 ceph-mon[35312]: fsmap 7 up:standby 2024-03-28T18:04:25.876 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:04:25 smithi079 ceph-mon[35312]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-03-28T18:04:25.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:04:25 smithi134 ceph-mon[35037]: Deploying daemon mds.h on smithi079 2024-03-28T18:04:25.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:04:25 smithi134 ceph-mon[35037]: pgmap v353: 1 pgs: 1 active+clean; 577 KiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-28T18:04:25.925 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:04:25 smithi134 ceph-mon[35037]: mds.? [v2:172.21.15.134:6836/899825581,v1:172.21.15.134:6837/899825581] up:boot 2024-03-28T18:04:25.925 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:04:25 smithi134 ceph-mon[35037]: fsmap 7 up:standby 2024-03-28T18:04:25.925 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:04:25 smithi134 ceph-mon[35037]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-03-28T18:04:26.032 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:04:25 smithi022 ceph-mon[31985]: Deploying daemon mds.h on smithi079 2024-03-28T18:04:26.032 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:04:25 smithi022 ceph-mon[31985]: pgmap v353: 1 pgs: 1 active+clean; 577 KiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-28T18:04:26.032 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:04:25 smithi022 ceph-mon[31985]: mds.? [v2:172.21.15.134:6836/899825581,v1:172.21.15.134:6837/899825581] up:boot 2024-03-28T18:04:26.032 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:04:25 smithi022 ceph-mon[31985]: fsmap 7 up:standby 2024-03-28T18:04:26.032 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:04:25 smithi022 ceph-mon[31985]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-03-28T18:04:26.397 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-28T18:04:26.631 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:26.631+0000 7ff59ce00640 1 -- 172.21.15.22:0/659943595 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff5981016c0 msgr2=0x7ff598101ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:04:26.631 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:26.631+0000 7ff59ce00640 1 --2- 172.21.15.22:0/659943595 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff5981016c0 0x7ff598101ae0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7ff588009900 tx=0x7ff58802bc30 comp rx=0 tx=0).stop 2024-03-28T18:04:26.632 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:26.631+0000 7ff59ce00640 1 -- 172.21.15.22:0/659943595 shutdown_connections 2024-03-28T18:04:26.632 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:26.631+0000 7ff59ce00640 1 --2- 172.21.15.22:0/659943595 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff598103270 0x7ff598109b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:04:26.632 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:26.631+0000 7ff59ce00640 1 --2- 172.21.15.22:0/659943595 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ff5981028b0 0x7ff598102d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:04:26.632 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:26.631+0000 7ff59ce00640 1 --2- 172.21.15.22:0/659943595 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff5981016c0 0x7ff598101ae0 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:04:26.632 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:26.631+0000 7ff59ce00640 1 -- 172.21.15.22:0/659943595 >> 172.21.15.22:0/659943595 conn(0x7ff5980fcc50 msgr2=0x7ff5980ff070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:04:26.632 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:26.632+0000 7ff59ce00640 1 -- 172.21.15.22:0/659943595 shutdown_connections 2024-03-28T18:04:26.632 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:26.632+0000 7ff59ce00640 1 -- 172.21.15.22:0/659943595 wait complete. 2024-03-28T18:04:26.633 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:26.633+0000 7ff59ce00640 1 Processor -- start 2024-03-28T18:04:26.633 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:26.633+0000 7ff59ce00640 1 -- start start 2024-03-28T18:04:26.633 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:26.633+0000 7ff59ce00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff5981016c0 0x7ff5981979e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:04:26.634 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:26.633+0000 7ff59ce00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff5981028b0 0x7ff598197f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:04:26.634 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:26.633+0000 7ff59ce00640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ff598103270 0x7ff59819ef90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:04:26.634 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:26.633+0000 7ff59ce00640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff59810d370 con 0x7ff5981028b0 2024-03-28T18:04:26.634 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:26.633+0000 7ff59ce00640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff59810d1f0 con 0x7ff598103270 2024-03-28T18:04:26.634 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:26.633+0000 7ff59ce00640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff59810d4f0 con 0x7ff5981016c0 2024-03-28T18:04:26.634 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:26.633+0000 7ff595400640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff5981028b0 0x7ff598197f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:04:26.634 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:26.633+0000 7ff595400640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff5981028b0 0x7ff598197f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:55632/0 (socket says 172.21.15.22:55632) 2024-03-28T18:04:26.634 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:26.633+0000 7ff595400640 1 -- 172.21.15.22:0/4192955192 learned_addr learned my addr 172.21.15.22:0/4192955192 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-28T18:04:26.634 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:26.634+0000 7ff596800640 1 --2- 172.21.15.22:0/4192955192 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ff598103270 0x7ff59819ef90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:04:26.634 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:26.634+0000 7ff595e00640 1 --2- 172.21.15.22:0/4192955192 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff5981016c0 0x7ff5981979e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:04:26.634 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:26.634+0000 7ff595400640 1 -- 172.21.15.22:0/4192955192 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff5981016c0 msgr2=0x7ff5981979e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:04:26.634 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:26.634+0000 7ff595400640 1 --2- 172.21.15.22:0/4192955192 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff5981016c0 0x7ff5981979e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:04:26.634 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:26.634+0000 7ff595400640 1 -- 172.21.15.22:0/4192955192 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ff598103270 msgr2=0x7ff59819ef90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:04:26.634 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:26.634+0000 7ff595400640 1 --2- 172.21.15.22:0/4192955192 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ff598103270 0x7ff59819ef90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:04:26.635 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:26.634+0000 7ff595400640 1 -- 172.21.15.22:0/4192955192 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff59819f600 con 0x7ff5981028b0 2024-03-28T18:04:26.635 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:26.634+0000 7ff595400640 1 --2- 172.21.15.22:0/4192955192 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff5981028b0 0x7ff598197f20 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7ff58400d950 tx=0x7ff58400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:04:26.635 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:26.634+0000 7ff58ea00640 1 -- 172.21.15.22:0/4192955192 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff584017070 con 0x7ff5981028b0 2024-03-28T18:04:26.635 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:26.634+0000 7ff58ea00640 1 -- 172.21.15.22:0/4192955192 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7ff584009c30 con 0x7ff5981028b0 2024-03-28T18:04:26.636 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:26.634+0000 7ff59ce00640 1 -- 172.21.15.22:0/4192955192 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff59819f8f0 con 0x7ff5981028b0 2024-03-28T18:04:26.636 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:26.634+0000 7ff58ea00640 1 -- 172.21.15.22:0/4192955192 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff58400f6b0 con 0x7ff5981028b0 2024-03-28T18:04:26.636 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:26.635+0000 7ff59ce00640 1 -- 172.21.15.22:0/4192955192 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff59819fe40 con 0x7ff5981028b0 2024-03-28T18:04:26.636 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:26.636+0000 7ff59ce00640 1 -- 172.21.15.22:0/4192955192 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff598108a30 con 0x7ff5981028b0 2024-03-28T18:04:26.638 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:26.638+0000 7ff58ea00640 1 -- 172.21.15.22:0/4192955192 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 16) v1 ==== 137290+0+0 (secure 0 0 0) 0x7ff584010a50 con 0x7ff5981028b0 2024-03-28T18:04:26.640 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:26.640+0000 7ff58ea00640 1 --2- 172.21.15.22:0/4192955192 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7ff57c0a2800 0x7ff57c0a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:04:26.640 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:26.641+0000 7ff58ea00640 1 -- 172.21.15.22:0/4192955192 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 7127+0+0 (secure 0 0 0) 0x7ff5840ce350 con 0x7ff5981028b0 2024-03-28T18:04:26.640 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:26.641+0000 7ff595e00640 1 --2- 172.21.15.22:0/4192955192 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7ff57c0a2800 0x7ff57c0a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:04:26.641 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:26.641+0000 7ff595e00640 1 --2- 172.21.15.22:0/4192955192 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7ff57c0a2800 0x7ff57c0a4cc0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7ff5880096c0 tx=0x7ff588005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:04:26.645 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:26.645+0000 7ff58ea00640 1 -- 172.21.15.22:0/4192955192 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7ff584009de0 con 0x7ff5981028b0 2024-03-28T18:04:26.887 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:26.887+0000 7ff59ce00640 1 -- 172.21.15.22:0/4192955192 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff5980622d0 con 0x7ff5981028b0 2024-03-28T18:04:26.888 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:26.888+0000 7ff58ea00640 1 -- 172.21.15.22:0/4192955192 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+6074 (secure 0 0 0) 0x7ff58409b7f0 con 0x7ff5981028b0 2024-03-28T18:04:26.888 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-03-28T18:04:26.889 INFO:teuthology.orchestra.run.smithi022.stdout:{"epoch":8,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14544,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6835/3718238950","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6834","nonce":3718238950},{"type":"v1","addr":"172.21.15.22:6835","nonce":3718238950}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14562,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6837/3676939386","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6836","nonce":3676939386},{"type":"v1","addr":"172.21.15.22:6837","nonce":3676939386}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":24412,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.134:6833/3176125362","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6832","nonce":3176125362},{"type":"v1","addr":"172.21.15.134:6833","nonce":3176125362}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24463,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.134:6835/3370549207","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6834","nonce":3370549207},{"type":"v1","addr":"172.21.15.134:6835","nonce":3370549207}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":7},{"gid":24478,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.134:6837/899825581","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6836","nonce":899825581},{"type":"v1","addr":"172.21.15.134:6837","nonce":899825581}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":8},{"gid":34442,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.79:6833/856896997","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6832","nonce":856896997},{"type":"v1","addr":"172.21.15.79:6833","nonce":856896997}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":34475,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.79:6835/3365773431","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6834","nonce":3365773431},{"type":"v1","addr":"172.21.15.79:6835","nonce":3365773431}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6}],"filesystems":[]} 2024-03-28T18:04:26.889 INFO:teuthology.orchestra.run.smithi022.stderr:dumped fsmap epoch 8 2024-03-28T18:04:26.891 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:26.891+0000 7ff59ce00640 1 -- 172.21.15.22:0/4192955192 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7ff57c0a2800 msgr2=0x7ff57c0a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:04:26.891 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:26.891+0000 7ff59ce00640 1 --2- 172.21.15.22:0/4192955192 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7ff57c0a2800 0x7ff57c0a4cc0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7ff5880096c0 tx=0x7ff588005fd0 comp rx=0 tx=0).stop 2024-03-28T18:04:26.891 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:26.891+0000 7ff59ce00640 1 -- 172.21.15.22:0/4192955192 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff5981028b0 msgr2=0x7ff598197f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:04:26.891 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:26.891+0000 7ff59ce00640 1 --2- 172.21.15.22:0/4192955192 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff5981028b0 0x7ff598197f20 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7ff58400d950 tx=0x7ff58400de10 comp rx=0 tx=0).stop 2024-03-28T18:04:26.891 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:26.891+0000 7ff59ce00640 1 -- 172.21.15.22:0/4192955192 shutdown_connections 2024-03-28T18:04:26.891 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:26.891+0000 7ff59ce00640 1 --2- 172.21.15.22:0/4192955192 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7ff57c0a2800 0x7ff57c0a4cc0 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:04:26.891 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:26.891+0000 7ff59ce00640 1 --2- 172.21.15.22:0/4192955192 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ff598103270 0x7ff59819ef90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:04:26.891 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:26.891+0000 7ff59ce00640 1 --2- 172.21.15.22:0/4192955192 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff5981028b0 0x7ff598197f20 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:04:26.891 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:26.891+0000 7ff59ce00640 1 --2- 172.21.15.22:0/4192955192 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff5981016c0 0x7ff5981979e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:04:26.891 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:26.891+0000 7ff59ce00640 1 -- 172.21.15.22:0/4192955192 >> 172.21.15.22:0/4192955192 conn(0x7ff5980fcc50 msgr2=0x7ff598105c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:04:26.892 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:26.891+0000 7ff59ce00640 1 -- 172.21.15.22:0/4192955192 shutdown_connections 2024-03-28T18:04:26.892 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:26.891+0000 7ff59ce00640 1 -- 172.21.15.22:0/4192955192 wait complete. 2024-03-28T18:04:27.925 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:04:27 smithi134 ceph-mon[35037]: pgmap v354: 1 pgs: 1 active+clean; 577 KiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-28T18:04:27.925 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:04:27 smithi134 ceph-mon[35037]: from='client.? 172.21.15.22:0/4192955192' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-28T18:04:27.933 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:04:27 smithi079 ceph-mon[35312]: pgmap v354: 1 pgs: 1 active+clean; 577 KiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-28T18:04:27.933 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:04:27 smithi079 ceph-mon[35312]: from='client.? 172.21.15.22:0/4192955192' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-28T18:04:28.031 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:04:27 smithi022 ceph-mon[31985]: pgmap v354: 1 pgs: 1 active+clean; 577 KiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-28T18:04:28.032 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:04:27 smithi022 ceph-mon[31985]: from='client.? 172.21.15.22:0/4192955192' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-28T18:04:28.242 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:04:27 smithi079 podman[64917]: 2024-03-28 18:04:27.996845331 +0000 UTC m=+0.148396272 container create 6faabd85524be22b28f413c870b774965a25d51a2d6fc4b8068370cff2a2f2d8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, org.label-schema.vendor=CentOS, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_CLEAN=True, org.label-schema.build-date=20240319, ceph=True, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5) 2024-03-28T18:04:28.242 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:04:28 smithi079 podman[64917]: 2024-03-28 18:04:28.241937529 +0000 UTC m=+0.393488477 container init 6faabd85524be22b28f413c870b774965a25d51a2d6fc4b8068370cff2a2f2d8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240319, GIT_CLEAN=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=) 2024-03-28T18:04:28.683 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:04:28 smithi079 podman[64917]: 2024-03-28 18:04:28.253376998 +0000 UTC m=+0.404927911 container start 6faabd85524be22b28f413c870b774965a25d51a2d6fc4b8068370cff2a2f2d8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240319, CEPH_POINT_RELEASE=, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True) 2024-03-28T18:04:28.683 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:04:28 smithi079 ceph-mds[64936]: -- 172.21.15.79:0/2370542706 <== mon.2 v2:172.21.15.79:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x5564d6427a40 con 0x5564d63fb000 2024-03-28T18:04:28.683 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:04:28 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[64932]: starting mds.h at 2024-03-28T18:04:28.683 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:04:28 smithi079 bash[64917]: 6faabd85524be22b28f413c870b774965a25d51a2d6fc4b8068370cff2a2f2d8 2024-03-28T18:04:28.683 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:04:28 smithi079 systemd[1]: Started Ceph mds.h for fd220748-ed2b-11ee-95d0-87774f69a715. 2024-03-28T18:04:28.913 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-28T18:04:29.149 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:29.149+0000 7f794e000640 1 -- 172.21.15.22:0/3729906187 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f79481052a0 msgr2=0x7f794810bb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:04:29.150 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:29.149+0000 7f794e000640 1 --2- 172.21.15.22:0/3729906187 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f79481052a0 0x7f794810bb30 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f7940009ca0 tx=0x7f794002c100 comp rx=0 tx=0).stop 2024-03-28T18:04:29.150 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:29.149+0000 7f794e000640 1 -- 172.21.15.22:0/3729906187 shutdown_connections 2024-03-28T18:04:29.150 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:29.149+0000 7f794e000640 1 --2- 172.21.15.22:0/3729906187 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f79481052a0 0x7f794810bb30 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:04:29.150 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:29.149+0000 7f794e000640 1 --2- 172.21.15.22:0/3729906187 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f79481048e0 0x7f7948104d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:04:29.150 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:29.149+0000 7f794e000640 1 --2- 172.21.15.22:0/3729906187 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f79481036f0 0x7f7948103b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:04:29.150 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:29.149+0000 7f794e000640 1 -- 172.21.15.22:0/3729906187 >> 172.21.15.22:0/3729906187 conn(0x7f79480fec80 msgr2=0x7f79481010a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:04:29.150 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:29.150+0000 7f794e000640 1 -- 172.21.15.22:0/3729906187 shutdown_connections 2024-03-28T18:04:29.150 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:29.150+0000 7f794e000640 1 -- 172.21.15.22:0/3729906187 wait complete. 2024-03-28T18:04:29.152 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:29.151+0000 7f794e000640 1 Processor -- start 2024-03-28T18:04:29.152 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:29.151+0000 7f794e000640 1 -- start start 2024-03-28T18:04:29.152 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:29.151+0000 7f794e000640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f79481036f0 0x7f7948199960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:04:29.152 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:29.151+0000 7f794e000640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f79481048e0 0x7f7948199ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:04:29.152 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:29.151+0000 7f794e000640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f79481052a0 0x7f79481a0f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:04:29.153 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:29.151+0000 7f794e000640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f794810f350 con 0x7f79481052a0 2024-03-28T18:04:29.153 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:29.151+0000 7f794e000640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f794810f1d0 con 0x7f79481036f0 2024-03-28T18:04:29.153 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:29.151+0000 7f794e000640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f794810f4d0 con 0x7f79481048e0 2024-03-28T18:04:29.153 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:29.151+0000 7f794d600640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f79481052a0 0x7f79481a0f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:04:29.153 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:29.151+0000 7f794d600640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f79481052a0 0x7f79481a0f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:34944/0 (socket says 172.21.15.22:34944) 2024-03-28T18:04:29.153 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:29.151+0000 7f794d600640 1 -- 172.21.15.22:0/811282982 learned_addr learned my addr 172.21.15.22:0/811282982 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-28T18:04:29.153 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:29.152+0000 7f793fe00640 1 --2- 172.21.15.22:0/811282982 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f79481048e0 0x7f7948199ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:04:29.153 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:29.152+0000 7f794cc00640 1 --2- 172.21.15.22:0/811282982 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f79481036f0 0x7f7948199960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:04:29.153 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:29.152+0000 7f794d600640 1 -- 172.21.15.22:0/811282982 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f79481048e0 msgr2=0x7f7948199ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:04:29.153 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:29.152+0000 7f794d600640 1 --2- 172.21.15.22:0/811282982 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f79481048e0 0x7f7948199ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:04:29.153 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:29.152+0000 7f794d600640 1 -- 172.21.15.22:0/811282982 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f79481036f0 msgr2=0x7f7948199960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:04:29.153 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:29.152+0000 7f794d600640 1 --2- 172.21.15.22:0/811282982 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f79481036f0 0x7f7948199960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:04:29.153 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:29.152+0000 7f794d600640 1 -- 172.21.15.22:0/811282982 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f79481a1580 con 0x7f79481052a0 2024-03-28T18:04:29.153 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:29.152+0000 7f794d600640 1 --2- 172.21.15.22:0/811282982 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f79481052a0 0x7f79481a0f10 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f7940037040 tx=0x7f7940034d70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:04:29.154 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:29.153+0000 7f7946000640 1 -- 172.21.15.22:0/811282982 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7940045070 con 0x7f79481052a0 2024-03-28T18:04:29.154 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:29.153+0000 7f794e000640 1 -- 172.21.15.22:0/811282982 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f79481a1810 con 0x7f79481052a0 2024-03-28T18:04:29.154 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:29.153+0000 7f7946000640 1 -- 172.21.15.22:0/811282982 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f794003a040 con 0x7f79481052a0 2024-03-28T18:04:29.154 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:29.153+0000 7f7946000640 1 -- 172.21.15.22:0/811282982 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7940039050 con 0x7f79481052a0 2024-03-28T18:04:29.154 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:29.153+0000 7f794e000640 1 -- 172.21.15.22:0/811282982 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f79481a1d00 con 0x7f79481052a0 2024-03-28T18:04:29.154 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:29.154+0000 7f794e000640 1 -- 172.21.15.22:0/811282982 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f794810aa10 con 0x7f79481052a0 2024-03-28T18:04:29.155 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:29.156+0000 7f7946000640 1 -- 172.21.15.22:0/811282982 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 16) v1 ==== 137290+0+0 (secure 0 0 0) 0x7f7940048070 con 0x7f79481052a0 2024-03-28T18:04:29.157 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:29.157+0000 7f7946000640 1 --2- 172.21.15.22:0/811282982 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7f792c0a2800 0x7f792c0a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:04:29.157 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:29.157+0000 7f7946000640 1 -- 172.21.15.22:0/811282982 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f79400f04f0 con 0x7f79481052a0 2024-03-28T18:04:29.157 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:29.158+0000 7f794cc00640 1 --2- 172.21.15.22:0/811282982 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7f792c0a2800 0x7f792c0a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:04:29.158 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:29.158+0000 7f794cc00640 1 --2- 172.21.15.22:0/811282982 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7f792c0a2800 0x7f792c0a4cc0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f7938005fd0 tx=0x7f7938005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:04:29.162 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:29.162+0000 7f7946000640 1 -- 172.21.15.22:0/811282982 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f79400bd340 con 0x7f79481052a0 2024-03-28T18:04:29.413 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:29.413+0000 7f794e000640 1 -- 172.21.15.22:0/811282982 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f79480622a0 con 0x7f79481052a0 2024-03-28T18:04:29.414 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:29.414+0000 7f7946000640 1 -- 172.21.15.22:0/811282982 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) v1 ==== 93+0+6865 (secure 0 0 0) 0x7f79400c13e0 con 0x7f79481052a0 2024-03-28T18:04:29.415 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-03-28T18:04:29.415 INFO:teuthology.orchestra.run.smithi022.stdout:{"epoch":9,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14544,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6835/3718238950","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6834","nonce":3718238950},{"type":"v1","addr":"172.21.15.22:6835","nonce":3718238950}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14562,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6837/3676939386","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6836","nonce":3676939386},{"type":"v1","addr":"172.21.15.22:6837","nonce":3676939386}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":24412,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.134:6833/3176125362","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6832","nonce":3176125362},{"type":"v1","addr":"172.21.15.134:6833","nonce":3176125362}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24463,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.134:6835/3370549207","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6834","nonce":3370549207},{"type":"v1","addr":"172.21.15.134:6835","nonce":3370549207}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":7},{"gid":24478,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.134:6837/899825581","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6836","nonce":899825581},{"type":"v1","addr":"172.21.15.134:6837","nonce":899825581}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":8},{"gid":34442,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.79:6833/856896997","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6832","nonce":856896997},{"type":"v1","addr":"172.21.15.79:6833","nonce":856896997}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":34475,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.79:6835/3365773431","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6834","nonce":3365773431},{"type":"v1","addr":"172.21.15.79:6835","nonce":3365773431}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6},{"gid":34511,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.79:6837/1787903366","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6836","nonce":1787903366},{"type":"v1","addr":"172.21.15.79:6837","nonce":1787903366}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":9}],"filesystems":[]} 2024-03-28T18:04:29.415 INFO:teuthology.orchestra.run.smithi022.stderr:dumped fsmap epoch 9 2024-03-28T18:04:29.417 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:29.417+0000 7f794e000640 1 -- 172.21.15.22:0/811282982 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7f792c0a2800 msgr2=0x7f792c0a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:04:29.417 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:29.417+0000 7f794e000640 1 --2- 172.21.15.22:0/811282982 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7f792c0a2800 0x7f792c0a4cc0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f7938005fd0 tx=0x7f7938005850 comp rx=0 tx=0).stop 2024-03-28T18:04:29.417 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:29.417+0000 7f794e000640 1 -- 172.21.15.22:0/811282982 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f79481052a0 msgr2=0x7f79481a0f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:04:29.417 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:29.417+0000 7f794e000640 1 --2- 172.21.15.22:0/811282982 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f79481052a0 0x7f79481a0f10 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f7940037040 tx=0x7f7940034d70 comp rx=0 tx=0).stop 2024-03-28T18:04:29.417 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:29.417+0000 7f794e000640 1 -- 172.21.15.22:0/811282982 shutdown_connections 2024-03-28T18:04:29.417 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:29.417+0000 7f794e000640 1 --2- 172.21.15.22:0/811282982 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7f792c0a2800 0x7f792c0a4cc0 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:04:29.417 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:29.417+0000 7f794e000640 1 --2- 172.21.15.22:0/811282982 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f79481052a0 0x7f79481a0f10 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:04:29.417 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:29.417+0000 7f794e000640 1 --2- 172.21.15.22:0/811282982 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f79481048e0 0x7f7948199ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:04:29.418 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:29.417+0000 7f794e000640 1 --2- 172.21.15.22:0/811282982 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f79481036f0 0x7f7948199960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:04:29.418 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:29.417+0000 7f794e000640 1 -- 172.21.15.22:0/811282982 >> 172.21.15.22:0/811282982 conn(0x7f79480fec80 msgr2=0x7f7948107be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:04:29.418 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:29.417+0000 7f794e000640 1 -- 172.21.15.22:0/811282982 shutdown_connections 2024-03-28T18:04:29.418 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:29.417+0000 7f794e000640 1 -- 172.21.15.22:0/811282982 wait complete. 2024-03-28T18:04:29.756 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:04:29 smithi134 ceph-mon[35037]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T18:04:29.756 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:04:29 smithi134 ceph-mon[35037]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T18:04:29.756 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:04:29 smithi134 ceph-mon[35037]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T18:04:29.756 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:04:29 smithi134 ceph-mon[35037]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-28T18:04:29.756 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:04:29 smithi134 ceph-mon[35037]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-28T18:04:29.756 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:04:29 smithi134 ceph-mon[35037]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T18:04:29.757 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:04:29 smithi134 ceph-mon[35037]: mds.? [v2:172.21.15.79:6836/1787903366,v1:172.21.15.79:6837/1787903366] up:boot 2024-03-28T18:04:29.757 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:04:29 smithi134 ceph-mon[35037]: fsmap 8 up:standby 2024-03-28T18:04:29.757 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:04:29 smithi134 ceph-mon[35037]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-03-28T18:04:29.757 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:04:29 smithi134 ceph-mon[35037]: from='client.? 172.21.15.22:0/811282982' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-28T18:04:29.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:04:29 smithi022 ceph-mon[31985]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T18:04:29.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:04:29 smithi022 ceph-mon[31985]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T18:04:29.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:04:29 smithi022 ceph-mon[31985]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T18:04:29.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:04:29 smithi022 ceph-mon[31985]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-28T18:04:29.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:04:29 smithi022 ceph-mon[31985]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-28T18:04:29.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:04:29 smithi022 ceph-mon[31985]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T18:04:29.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:04:29 smithi022 ceph-mon[31985]: mds.? [v2:172.21.15.79:6836/1787903366,v1:172.21.15.79:6837/1787903366] up:boot 2024-03-28T18:04:29.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:04:29 smithi022 ceph-mon[31985]: fsmap 8 up:standby 2024-03-28T18:04:29.783 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:04:29 smithi022 ceph-mon[31985]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-03-28T18:04:29.783 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:04:29 smithi022 ceph-mon[31985]: from='client.? 172.21.15.22:0/811282982' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-28T18:04:29.933 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:04:29 smithi079 ceph-mon[35312]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T18:04:29.933 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:04:29 smithi079 ceph-mon[35312]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T18:04:29.933 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:04:29 smithi079 ceph-mon[35312]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T18:04:29.933 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:04:29 smithi079 ceph-mon[35312]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-28T18:04:29.933 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:04:29 smithi079 ceph-mon[35312]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-28T18:04:29.933 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:04:29 smithi079 ceph-mon[35312]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T18:04:29.933 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:04:29 smithi079 ceph-mon[35312]: mds.? [v2:172.21.15.79:6836/1787903366,v1:172.21.15.79:6837/1787903366] up:boot 2024-03-28T18:04:29.933 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:04:29 smithi079 ceph-mon[35312]: fsmap 8 up:standby 2024-03-28T18:04:29.933 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:04:29 smithi079 ceph-mon[35312]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-03-28T18:04:29.933 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:04:29 smithi079 ceph-mon[35312]: from='client.? 172.21.15.22:0/811282982' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-28T18:04:30.674 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:04:30 smithi134 ceph-mon[35037]: Deploying daemon mds.i on smithi134 2024-03-28T18:04:30.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:04:30 smithi134 ceph-mon[35037]: pgmap v355: 1 pgs: 1 active+clean; 577 KiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-28T18:04:30.781 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:04:30 smithi022 ceph-mon[31985]: Deploying daemon mds.i on smithi134 2024-03-28T18:04:30.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:04:30 smithi022 ceph-mon[31985]: pgmap v355: 1 pgs: 1 active+clean; 577 KiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-28T18:04:30.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:04:30 smithi079 ceph-mon[35312]: Deploying daemon mds.i on smithi134 2024-03-28T18:04:30.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:04:30 smithi079 ceph-mon[35312]: pgmap v355: 1 pgs: 1 active+clean; 577 KiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-28T18:04:31.439 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-28T18:04:31.676 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:31.676+0000 7f5d62e00640 1 -- 172.21.15.22:0/551802951 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5d5c1026d0 msgr2=0x7f5d5c102b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:04:31.676 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:31.676+0000 7f5d62e00640 1 --2- 172.21.15.22:0/551802951 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5d5c1026d0 0x7f5d5c102b50 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f5d48009900 tx=0x7f5d4802bc30 comp rx=0 tx=0).stop 2024-03-28T18:04:31.676 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:31.676+0000 7f5d62e00640 1 -- 172.21.15.22:0/551802951 shutdown_connections 2024-03-28T18:04:31.677 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:31.676+0000 7f5d62e00640 1 --2- 172.21.15.22:0/551802951 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f5d5c103090 0x7f5d5c109920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:04:31.677 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:31.676+0000 7f5d62e00640 1 --2- 172.21.15.22:0/551802951 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5d5c1026d0 0x7f5d5c102b50 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:04:31.677 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:31.676+0000 7f5d62e00640 1 --2- 172.21.15.22:0/551802951 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5d5c1014e0 0x7f5d5c101900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:04:31.677 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:31.676+0000 7f5d62e00640 1 -- 172.21.15.22:0/551802951 >> 172.21.15.22:0/551802951 conn(0x7f5d5c0fca30 msgr2=0x7f5d5c0fee90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:04:31.677 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:31.676+0000 7f5d62e00640 1 -- 172.21.15.22:0/551802951 shutdown_connections 2024-03-28T18:04:31.677 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:31.676+0000 7f5d62e00640 1 -- 172.21.15.22:0/551802951 wait complete. 2024-03-28T18:04:31.680 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:31.677+0000 7f5d62e00640 1 Processor -- start 2024-03-28T18:04:31.680 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:31.678+0000 7f5d62e00640 1 -- start start 2024-03-28T18:04:31.680 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:31.678+0000 7f5d62e00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5d5c1014e0 0x7f5d5c0753b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:04:31.680 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:31.678+0000 7f5d62e00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5d5c1026d0 0x7f5d5c073a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:04:31.681 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:31.678+0000 7f5d62e00640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f5d5c103090 0x7f5d5c073fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:04:31.681 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:31.678+0000 7f5d62e00640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d5c10d140 con 0x7f5d5c1014e0 2024-03-28T18:04:31.681 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:31.678+0000 7f5d62e00640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d5c10cfc0 con 0x7f5d5c103090 2024-03-28T18:04:31.681 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:31.678+0000 7f5d62e00640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d5c10d2c0 con 0x7f5d5c1026d0 2024-03-28T18:04:31.681 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:31.678+0000 7f5d61a00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5d5c1014e0 0x7f5d5c0753b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:04:31.681 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:31.678+0000 7f5d61a00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5d5c1014e0 0x7f5d5c0753b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:34962/0 (socket says 172.21.15.22:34962) 2024-03-28T18:04:31.681 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:31.678+0000 7f5d61a00640 1 -- 172.21.15.22:0/1719057498 learned_addr learned my addr 172.21.15.22:0/1719057498 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-28T18:04:31.681 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:31.679+0000 7f5d61000640 1 --2- 172.21.15.22:0/1719057498 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5d5c1026d0 0x7f5d5c073a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:04:31.681 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:31.679+0000 7f5d62400640 1 --2- 172.21.15.22:0/1719057498 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f5d5c103090 0x7f5d5c073fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:04:31.681 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:31.679+0000 7f5d61a00640 1 -- 172.21.15.22:0/1719057498 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5d5c1026d0 msgr2=0x7f5d5c073a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:04:31.682 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:31.679+0000 7f5d61a00640 1 --2- 172.21.15.22:0/1719057498 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5d5c1026d0 0x7f5d5c073a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:04:31.682 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:31.679+0000 7f5d61a00640 1 -- 172.21.15.22:0/1719057498 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f5d5c103090 msgr2=0x7f5d5c073fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:04:31.682 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:31.679+0000 7f5d61a00640 1 --2- 172.21.15.22:0/1719057498 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f5d5c103090 0x7f5d5c073fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:04:31.682 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:31.679+0000 7f5d61a00640 1 -- 172.21.15.22:0/1719057498 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d5c0747d0 con 0x7f5d5c1014e0 2024-03-28T18:04:31.682 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:31.679+0000 7f5d61a00640 1 --2- 172.21.15.22:0/1719057498 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5d5c1014e0 0x7f5d5c0753b0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f5d5000e9c0 tx=0x7f5d5000ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:04:31.682 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:31.679+0000 7f5d5a000640 1 -- 172.21.15.22:0/1719057498 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d5000cd80 con 0x7f5d5c1014e0 2024-03-28T18:04:31.682 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:31.680+0000 7f5d62e00640 1 -- 172.21.15.22:0/1719057498 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5d5c1a1930 con 0x7f5d5c1014e0 2024-03-28T18:04:31.682 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:31.680+0000 7f5d5a000640 1 -- 172.21.15.22:0/1719057498 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f5d500035f0 con 0x7f5d5c1014e0 2024-03-28T18:04:31.682 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:31.680+0000 7f5d5a000640 1 -- 172.21.15.22:0/1719057498 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d50010690 con 0x7f5d5c1014e0 2024-03-28T18:04:31.682 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:31.680+0000 7f5d62e00640 1 -- 172.21.15.22:0/1719057498 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5d5c1a1ea0 con 0x7f5d5c1014e0 2024-03-28T18:04:31.683 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:31.681+0000 7f5d62e00640 1 -- 172.21.15.22:0/1719057498 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5d5c108800 con 0x7f5d5c1014e0 2024-03-28T18:04:31.683 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:31.683+0000 7f5d5a000640 1 -- 172.21.15.22:0/1719057498 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 16) v1 ==== 137290+0+0 (secure 0 0 0) 0x7f5d500037a0 con 0x7f5d5c1014e0 2024-03-28T18:04:31.684 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:31.684+0000 7f5d5a000640 1 --2- 172.21.15.22:0/1719057498 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7f5d440a2880 0x7f5d440a4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:04:31.684 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:31.685+0000 7f5d5a000640 1 -- 172.21.15.22:0/1719057498 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f5d500cea20 con 0x7f5d5c1014e0 2024-03-28T18:04:31.684 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:31.685+0000 7f5d61000640 1 --2- 172.21.15.22:0/1719057498 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7f5d440a2880 0x7f5d440a4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:04:31.685 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:31.685+0000 7f5d61000640 1 --2- 172.21.15.22:0/1719057498 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7f5d440a2880 0x7f5d440a4d40 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f5d48005ab0 tx=0x7f5d48005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:04:31.688 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:31.689+0000 7f5d5a000640 1 -- 172.21.15.22:0/1719057498 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f5d5009bec0 con 0x7f5d5c1014e0 2024-03-28T18:04:31.922 INFO:journalctl@ceph.mds.i.smithi134.stdout:Mar 28 18:04:31 smithi134 systemd[1]: Starting Ceph mds.i for fd220748-ed2b-11ee-95d0-87774f69a715... 2024-03-28T18:04:31.932 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:31.933+0000 7f5d62e00640 1 -- 172.21.15.22:0/1719057498 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f5d5c0622d0 con 0x7f5d5c1014e0 2024-03-28T18:04:31.933 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:31.933+0000 7f5d5a000640 1 -- 172.21.15.22:0/1719057498 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) v1 ==== 93+0+6865 (secure 0 0 0) 0x7f5d5009b7c0 con 0x7f5d5c1014e0 2024-03-28T18:04:31.934 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-03-28T18:04:31.934 INFO:teuthology.orchestra.run.smithi022.stdout:{"epoch":9,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14544,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6835/3718238950","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6834","nonce":3718238950},{"type":"v1","addr":"172.21.15.22:6835","nonce":3718238950}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14562,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6837/3676939386","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6836","nonce":3676939386},{"type":"v1","addr":"172.21.15.22:6837","nonce":3676939386}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":24412,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.134:6833/3176125362","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6832","nonce":3176125362},{"type":"v1","addr":"172.21.15.134:6833","nonce":3176125362}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24463,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.134:6835/3370549207","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6834","nonce":3370549207},{"type":"v1","addr":"172.21.15.134:6835","nonce":3370549207}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":7},{"gid":24478,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.134:6837/899825581","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6836","nonce":899825581},{"type":"v1","addr":"172.21.15.134:6837","nonce":899825581}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":8},{"gid":34442,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.79:6833/856896997","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6832","nonce":856896997},{"type":"v1","addr":"172.21.15.79:6833","nonce":856896997}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":34475,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.79:6835/3365773431","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6834","nonce":3365773431},{"type":"v1","addr":"172.21.15.79:6835","nonce":3365773431}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6},{"gid":34511,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.79:6837/1787903366","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6836","nonce":1787903366},{"type":"v1","addr":"172.21.15.79:6837","nonce":1787903366}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":9}],"filesystems":[]} 2024-03-28T18:04:31.934 INFO:teuthology.orchestra.run.smithi022.stderr:dumped fsmap epoch 9 2024-03-28T18:04:31.936 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:31.936+0000 7f5d62e00640 1 -- 172.21.15.22:0/1719057498 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7f5d440a2880 msgr2=0x7f5d440a4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:04:31.936 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:31.936+0000 7f5d62e00640 1 --2- 172.21.15.22:0/1719057498 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7f5d440a2880 0x7f5d440a4d40 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f5d48005ab0 tx=0x7f5d48005fd0 comp rx=0 tx=0).stop 2024-03-28T18:04:31.937 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:31.936+0000 7f5d62e00640 1 -- 172.21.15.22:0/1719057498 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5d5c1014e0 msgr2=0x7f5d5c0753b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:04:31.937 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:31.936+0000 7f5d62e00640 1 --2- 172.21.15.22:0/1719057498 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5d5c1014e0 0x7f5d5c0753b0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f5d5000e9c0 tx=0x7f5d5000ee80 comp rx=0 tx=0).stop 2024-03-28T18:04:31.937 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:31.936+0000 7f5d62e00640 1 -- 172.21.15.22:0/1719057498 shutdown_connections 2024-03-28T18:04:31.937 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:31.936+0000 7f5d62e00640 1 --2- 172.21.15.22:0/1719057498 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7f5d440a2880 0x7f5d440a4d40 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:04:31.937 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:31.936+0000 7f5d62e00640 1 --2- 172.21.15.22:0/1719057498 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f5d5c103090 0x7f5d5c073fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:04:31.937 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:31.936+0000 7f5d62e00640 1 --2- 172.21.15.22:0/1719057498 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5d5c1026d0 0x7f5d5c073a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:04:31.937 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:31.936+0000 7f5d62e00640 1 --2- 172.21.15.22:0/1719057498 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5d5c1014e0 0x7f5d5c0753b0 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:04:31.937 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:31.936+0000 7f5d62e00640 1 -- 172.21.15.22:0/1719057498 >> 172.21.15.22:0/1719057498 conn(0x7f5d5c0fca30 msgr2=0x7f5d5c1059d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:04:31.937 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:31.936+0000 7f5d62e00640 1 -- 172.21.15.22:0/1719057498 shutdown_connections 2024-03-28T18:04:31.937 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:31.936+0000 7f5d62e00640 1 -- 172.21.15.22:0/1719057498 wait complete. 2024-03-28T18:04:32.175 INFO:journalctl@ceph.mds.i.smithi134.stdout:Mar 28 18:04:31 smithi134 podman[65743]: 2024-03-28 18:04:31.827166567 +0000 UTC m=+0.033601950 image pull c664afe56bb01db4f9afa9532c20598e201e664f2e9ba2be50db0d52674a26d2 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc 2024-03-28T18:04:32.175 INFO:journalctl@ceph.mds.i.smithi134.stdout:Mar 28 18:04:31 smithi134 podman[65743]: 2024-03-28 18:04:31.946195215 +0000 UTC m=+0.152630635 container create 43deaed0d42ab92ffaa250efb5a005b1ec8d70a77ceb34d662e1dc1873c95f0b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-i, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , ceph=True, GIT_CLEAN=True, io.buildah.version=1.33.5, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.build-date=20240319) 2024-03-28T18:04:32.504 INFO:journalctl@ceph.mds.i.smithi134.stdout:Mar 28 18:04:32 smithi134 podman[65743]: 2024-03-28 18:04:32.208606425 +0000 UTC m=+0.415041885 container init 43deaed0d42ab92ffaa250efb5a005b1ec8d70a77ceb34d662e1dc1873c95f0b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-i, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240319, ceph=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-28T18:04:32.504 INFO:journalctl@ceph.mds.i.smithi134.stdout:Mar 28 18:04:32 smithi134 podman[65743]: 2024-03-28 18:04:32.218858087 +0000 UTC m=+0.425293514 container start 43deaed0d42ab92ffaa250efb5a005b1ec8d70a77ceb34d662e1dc1873c95f0b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-i, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.build-date=20240319, GIT_CLEAN=True, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, org.label-schema.vendor=CentOS) 2024-03-28T18:04:32.504 INFO:journalctl@ceph.mds.i.smithi134.stdout:Mar 28 18:04:32 smithi134 ceph-mds[65762]: -- 172.21.15.134:0/3427959576 <== mon.1 v2:172.21.15.134:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x5576d18fda40 con 0x5576d18d0800 2024-03-28T18:04:32.504 INFO:journalctl@ceph.mds.i.smithi134.stdout:Mar 28 18:04:32 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-i[65758]: starting mds.i at 2024-03-28T18:04:32.504 INFO:journalctl@ceph.mds.i.smithi134.stdout:Mar 28 18:04:32 smithi134 bash[65743]: 43deaed0d42ab92ffaa250efb5a005b1ec8d70a77ceb34d662e1dc1873c95f0b 2024-03-28T18:04:32.504 INFO:journalctl@ceph.mds.i.smithi134.stdout:Mar 28 18:04:32 smithi134 systemd[1]: Started Ceph mds.i for fd220748-ed2b-11ee-95d0-87774f69a715. 2024-03-28T18:04:32.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:04:32 smithi022 ceph-mon[31985]: pgmap v356: 1 pgs: 1 active+clean; 577 KiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-28T18:04:32.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:04:32 smithi022 ceph-mon[31985]: from='client.? 172.21.15.22:0/1719057498' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-28T18:04:32.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:04:32 smithi022 ceph-mon[31985]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T18:04:32.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:04:32 smithi134 ceph-mon[35037]: pgmap v356: 1 pgs: 1 active+clean; 577 KiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-28T18:04:32.925 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:04:32 smithi134 ceph-mon[35037]: from='client.? 172.21.15.22:0/1719057498' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-28T18:04:32.925 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:04:32 smithi134 ceph-mon[35037]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T18:04:32.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:04:32 smithi079 ceph-mon[35312]: pgmap v356: 1 pgs: 1 active+clean; 577 KiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-28T18:04:32.933 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:04:32 smithi079 ceph-mon[35312]: from='client.? 172.21.15.22:0/1719057498' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-28T18:04:32.933 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:04:32 smithi079 ceph-mon[35312]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T18:04:33.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:04:33 smithi022 ceph-mon[31985]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T18:04:33.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:04:33 smithi022 ceph-mon[31985]: mds.? [v2:172.21.15.134:6838/3400263217,v1:172.21.15.134:6839/3400263217] up:boot 2024-03-28T18:04:33.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:04:33 smithi022 ceph-mon[31985]: fsmap 9 up:standby 2024-03-28T18:04:33.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:04:33 smithi022 ceph-mon[31985]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-03-28T18:04:33.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:04:33 smithi022 ceph-mon[31985]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T18:04:33.783 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:04:33 smithi022 ceph-mon[31985]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-28T18:04:33.783 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:04:33 smithi022 ceph-mon[31985]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-28T18:04:33.783 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:04:33 smithi022 ceph-mon[31985]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T18:04:33.925 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:04:33 smithi134 ceph-mon[35037]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T18:04:33.925 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:04:33 smithi134 ceph-mon[35037]: mds.? [v2:172.21.15.134:6838/3400263217,v1:172.21.15.134:6839/3400263217] up:boot 2024-03-28T18:04:33.925 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:04:33 smithi134 ceph-mon[35037]: fsmap 9 up:standby 2024-03-28T18:04:33.925 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:04:33 smithi134 ceph-mon[35037]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-03-28T18:04:33.925 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:04:33 smithi134 ceph-mon[35037]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T18:04:33.925 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:04:33 smithi134 ceph-mon[35037]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-28T18:04:33.925 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:04:33 smithi134 ceph-mon[35037]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-28T18:04:33.925 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:04:33 smithi134 ceph-mon[35037]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T18:04:33.933 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:04:33 smithi079 ceph-mon[35312]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T18:04:33.933 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:04:33 smithi079 ceph-mon[35312]: mds.? [v2:172.21.15.134:6838/3400263217,v1:172.21.15.134:6839/3400263217] up:boot 2024-03-28T18:04:33.933 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:04:33 smithi079 ceph-mon[35312]: fsmap 9 up:standby 2024-03-28T18:04:33.933 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:04:33 smithi079 ceph-mon[35312]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-03-28T18:04:33.933 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:04:33 smithi079 ceph-mon[35312]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T18:04:33.933 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:04:33 smithi079 ceph-mon[35312]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-28T18:04:33.933 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:04:33 smithi079 ceph-mon[35312]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-28T18:04:33.933 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:04:33 smithi079 ceph-mon[35312]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T18:04:33.955 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-28T18:04:34.183 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:34.182+0000 7fb996000640 1 -- 172.21.15.22:0/1395997704 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb9901048a0 msgr2=0x7fb990106c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:04:34.183 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:34.182+0000 7fb996000640 1 --2- 172.21.15.22:0/1395997704 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb9901048a0 0x7fb990106c90 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fb978009900 tx=0x7fb97802bc30 comp rx=0 tx=0).stop 2024-03-28T18:04:34.183 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:34.182+0000 7fb996000640 1 -- 172.21.15.22:0/1395997704 shutdown_connections 2024-03-28T18:04:34.183 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:34.182+0000 7fb996000640 1 --2- 172.21.15.22:0/1395997704 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fb9901071d0 0x7fb990109620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:04:34.183 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:34.182+0000 7fb996000640 1 --2- 172.21.15.22:0/1395997704 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb9901048a0 0x7fb990106c90 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:04:34.183 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:34.182+0000 7fb996000640 1 --2- 172.21.15.22:0/1395997704 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb990101f70 0x7fb990104360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:04:34.183 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:34.182+0000 7fb996000640 1 -- 172.21.15.22:0/1395997704 >> 172.21.15.22:0/1395997704 conn(0x7fb9900fb960 msgr2=0x7fb9900fdda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:04:34.183 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:34.182+0000 7fb996000640 1 -- 172.21.15.22:0/1395997704 shutdown_connections 2024-03-28T18:04:34.183 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:34.183+0000 7fb996000640 1 -- 172.21.15.22:0/1395997704 wait complete. 2024-03-28T18:04:34.186 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:34.184+0000 7fb996000640 1 Processor -- start 2024-03-28T18:04:34.186 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:34.184+0000 7fb996000640 1 -- start start 2024-03-28T18:04:34.186 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:34.184+0000 7fb996000640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fb990101f70 0x7fb9901999d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:04:34.187 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:34.184+0000 7fb996000640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb9901048a0 0x7fb990199f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:04:34.187 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:34.184+0000 7fb996000640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb9901071d0 0x7fb9901a0f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:04:34.187 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:34.184+0000 7fb996000640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb99010d1a0 con 0x7fb9901048a0 2024-03-28T18:04:34.187 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:34.184+0000 7fb996000640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb99010d020 con 0x7fb990101f70 2024-03-28T18:04:34.187 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:34.184+0000 7fb996000640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb99010d320 con 0x7fb9901071d0 2024-03-28T18:04:34.187 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:34.184+0000 7fb987e00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb9901048a0 0x7fb990199f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:04:34.187 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:34.184+0000 7fb987e00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb9901048a0 0x7fb990199f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:34992/0 (socket says 172.21.15.22:34992) 2024-03-28T18:04:34.187 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:34.184+0000 7fb987e00640 1 -- 172.21.15.22:0/2344144698 learned_addr learned my addr 172.21.15.22:0/2344144698 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-28T18:04:34.187 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:34.185+0000 7fb994c00640 1 --2- 172.21.15.22:0/2344144698 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fb990101f70 0x7fb9901999d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:04:34.187 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:34.185+0000 7fb995600640 1 --2- 172.21.15.22:0/2344144698 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb9901071d0 0x7fb9901a0f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:04:34.187 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:34.185+0000 7fb987e00640 1 -- 172.21.15.22:0/2344144698 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb9901071d0 msgr2=0x7fb9901a0f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:04:34.187 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:34.185+0000 7fb987e00640 1 --2- 172.21.15.22:0/2344144698 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb9901071d0 0x7fb9901a0f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:04:34.187 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:34.185+0000 7fb987e00640 1 -- 172.21.15.22:0/2344144698 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fb990101f70 msgr2=0x7fb9901999d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:04:34.188 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:34.185+0000 7fb987e00640 1 --2- 172.21.15.22:0/2344144698 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fb990101f70 0x7fb9901999d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:04:34.188 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:34.185+0000 7fb987e00640 1 -- 172.21.15.22:0/2344144698 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb9901a15f0 con 0x7fb9901048a0 2024-03-28T18:04:34.188 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:34.185+0000 7fb995600640 1 --2- 172.21.15.22:0/2344144698 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb9901071d0 0x7fb9901a0f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T18:04:34.188 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:34.185+0000 7fb994c00640 1 --2- 172.21.15.22:0/2344144698 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fb990101f70 0x7fb9901999d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-28T18:04:34.188 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:34.185+0000 7fb987e00640 1 --2- 172.21.15.22:0/2344144698 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb9901048a0 0x7fb990199f10 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fb97802c170 tx=0x7fb978004f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:04:34.188 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:34.186+0000 7fb98e000640 1 -- 172.21.15.22:0/2344144698 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb978043020 con 0x7fb9901048a0 2024-03-28T18:04:34.188 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:34.186+0000 7fb996000640 1 -- 172.21.15.22:0/2344144698 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb9901a1880 con 0x7fb9901048a0 2024-03-28T18:04:34.188 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:34.186+0000 7fb98e000640 1 -- 172.21.15.22:0/2344144698 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7fb978004410 con 0x7fb9901048a0 2024-03-28T18:04:34.188 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:34.186+0000 7fb98e000640 1 -- 172.21.15.22:0/2344144698 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb978003df0 con 0x7fb9901048a0 2024-03-28T18:04:34.188 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:34.186+0000 7fb996000640 1 -- 172.21.15.22:0/2344144698 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb9901a1dd0 con 0x7fb9901048a0 2024-03-28T18:04:34.189 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:34.187+0000 7fb996000640 1 -- 172.21.15.22:0/2344144698 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb990108d20 con 0x7fb9901048a0 2024-03-28T18:04:34.189 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:34.189+0000 7fb98e000640 1 -- 172.21.15.22:0/2344144698 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 16) v1 ==== 137290+0+0 (secure 0 0 0) 0x7fb978046020 con 0x7fb9901048a0 2024-03-28T18:04:34.191 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:34.192+0000 7fb98e000640 1 --2- 172.21.15.22:0/2344144698 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7fb9740a2800 0x7fb9740a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:04:34.192 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:34.192+0000 7fb98e000640 1 -- 172.21.15.22:0/2344144698 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 7127+0+0 (secure 0 0 0) 0x7fb9780f0f50 con 0x7fb9901048a0 2024-03-28T18:04:34.192 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:34.192+0000 7fb994c00640 1 --2- 172.21.15.22:0/2344144698 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7fb9740a2800 0x7fb9740a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:04:34.192 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:34.193+0000 7fb994c00640 1 --2- 172.21.15.22:0/2344144698 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7fb9740a2800 0x7fb9740a4cc0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fb980005fd0 tx=0x7fb980005ea0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:04:34.194 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:34.194+0000 7fb98e000640 1 -- 172.21.15.22:0/2344144698 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fb97804d020 con 0x7fb9901048a0 2024-03-28T18:04:34.446 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:34.447+0000 7fb996000640 1 -- 172.21.15.22:0/2344144698 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fb99002d070 con 0x7fb9901048a0 2024-03-28T18:04:34.448 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:34.448+0000 7fb98e000640 1 -- 172.21.15.22:0/2344144698 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 10 v10) v1 ==== 94+0+7661 (secure 0 0 0) 0x7fb9780be3f0 con 0x7fb9901048a0 2024-03-28T18:04:34.448 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-03-28T18:04:34.449 INFO:teuthology.orchestra.run.smithi022.stdout:{"epoch":10,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14544,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6835/3718238950","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6834","nonce":3718238950},{"type":"v1","addr":"172.21.15.22:6835","nonce":3718238950}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14562,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6837/3676939386","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6836","nonce":3676939386},{"type":"v1","addr":"172.21.15.22:6837","nonce":3676939386}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":24412,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.134:6833/3176125362","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6832","nonce":3176125362},{"type":"v1","addr":"172.21.15.134:6833","nonce":3176125362}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24463,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.134:6835/3370549207","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6834","nonce":3370549207},{"type":"v1","addr":"172.21.15.134:6835","nonce":3370549207}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":7},{"gid":24478,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.134:6837/899825581","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6836","nonce":899825581},{"type":"v1","addr":"172.21.15.134:6837","nonce":899825581}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":8},{"gid":24502,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.134:6839/3400263217","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6838","nonce":3400263217},{"type":"v1","addr":"172.21.15.134:6839","nonce":3400263217}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":10},{"gid":34442,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.79:6833/856896997","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6832","nonce":856896997},{"type":"v1","addr":"172.21.15.79:6833","nonce":856896997}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":34475,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.79:6835/3365773431","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6834","nonce":3365773431},{"type":"v1","addr":"172.21.15.79:6835","nonce":3365773431}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6},{"gid":34511,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.79:6837/1787903366","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6836","nonce":1787903366},{"type":"v1","addr":"172.21.15.79:6837","nonce":1787903366}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":9}],"filesystems":[]} 2024-03-28T18:04:34.449 INFO:teuthology.orchestra.run.smithi022.stderr:dumped fsmap epoch 10 2024-03-28T18:04:34.451 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:34.451+0000 7fb996000640 1 -- 172.21.15.22:0/2344144698 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7fb9740a2800 msgr2=0x7fb9740a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:04:34.451 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:34.451+0000 7fb996000640 1 --2- 172.21.15.22:0/2344144698 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7fb9740a2800 0x7fb9740a4cc0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fb980005fd0 tx=0x7fb980005ea0 comp rx=0 tx=0).stop 2024-03-28T18:04:34.451 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:34.451+0000 7fb996000640 1 -- 172.21.15.22:0/2344144698 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb9901048a0 msgr2=0x7fb990199f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:04:34.451 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:34.451+0000 7fb996000640 1 --2- 172.21.15.22:0/2344144698 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb9901048a0 0x7fb990199f10 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fb97802c170 tx=0x7fb978004f10 comp rx=0 tx=0).stop 2024-03-28T18:04:34.451 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:34.451+0000 7fb996000640 1 -- 172.21.15.22:0/2344144698 shutdown_connections 2024-03-28T18:04:34.451 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:34.451+0000 7fb996000640 1 --2- 172.21.15.22:0/2344144698 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7fb9740a2800 0x7fb9740a4cc0 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:04:34.451 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:34.451+0000 7fb996000640 1 --2- 172.21.15.22:0/2344144698 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb9901071d0 0x7fb9901a0f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:04:34.451 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:34.451+0000 7fb996000640 1 --2- 172.21.15.22:0/2344144698 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb9901048a0 0x7fb990199f10 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:04:34.452 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:34.451+0000 7fb996000640 1 --2- 172.21.15.22:0/2344144698 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fb990101f70 0x7fb9901999d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:04:34.452 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:34.451+0000 7fb996000640 1 -- 172.21.15.22:0/2344144698 >> 172.21.15.22:0/2344144698 conn(0x7fb9900fb960 msgr2=0x7fb9901015a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:04:34.452 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:34.451+0000 7fb996000640 1 -- 172.21.15.22:0/2344144698 shutdown_connections 2024-03-28T18:04:34.452 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:34.451+0000 7fb996000640 1 -- 172.21.15.22:0/2344144698 wait complete. 2024-03-28T18:04:34.664 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:04:34 smithi022 ceph-mon[31985]: Deploying daemon mds.g on smithi022 2024-03-28T18:04:34.664 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:04:34 smithi022 ceph-mon[31985]: pgmap v357: 1 pgs: 1 active+clean; 577 KiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-28T18:04:34.665 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:04:34 smithi022 ceph-mon[31985]: from='client.? 172.21.15.22:0/2344144698' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-28T18:04:34.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:04:34 smithi134 ceph-mon[35037]: Deploying daemon mds.g on smithi022 2024-03-28T18:04:34.925 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:04:34 smithi134 ceph-mon[35037]: pgmap v357: 1 pgs: 1 active+clean; 577 KiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-28T18:04:34.925 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:04:34 smithi134 ceph-mon[35037]: from='client.? 172.21.15.22:0/2344144698' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-28T18:04:34.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:04:34 smithi079 ceph-mon[35312]: Deploying daemon mds.g on smithi022 2024-03-28T18:04:34.933 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:04:34 smithi079 ceph-mon[35312]: pgmap v357: 1 pgs: 1 active+clean; 577 KiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-28T18:04:34.933 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:04:34 smithi079 ceph-mon[35312]: from='client.? 172.21.15.22:0/2344144698' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-28T18:04:35.871 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:04:35 smithi022 ceph-mon[31985]: pgmap v358: 1 pgs: 1 active+clean; 577 KiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-28T18:04:35.872 INFO:journalctl@ceph.mds.g.smithi022.stdout:Mar 28 18:04:35 smithi022 systemd[1]: Starting Ceph mds.g for fd220748-ed2b-11ee-95d0-87774f69a715... 2024-03-28T18:04:35.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:04:35 smithi134 ceph-mon[35037]: pgmap v358: 1 pgs: 1 active+clean; 577 KiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-28T18:04:35.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:04:35 smithi079 ceph-mon[35312]: pgmap v358: 1 pgs: 1 active+clean; 577 KiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-28T18:04:36.177 INFO:journalctl@ceph.mds.g.smithi022.stdout:Mar 28 18:04:35 smithi022 podman[74046]: 2024-03-28 18:04:35.887166436 +0000 UTC m=+0.040536163 image pull c664afe56bb01db4f9afa9532c20598e201e664f2e9ba2be50db0d52674a26d2 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc 2024-03-28T18:04:36.177 INFO:journalctl@ceph.mds.g.smithi022.stdout:Mar 28 18:04:35 smithi022 podman[74046]: 2024-03-28 18:04:35.99194263 +0000 UTC m=+0.145312408 container create 7fac2ccc2c03ac7ba3580a8ffa81e325d3bc121afb563be8a343ef42804dc799 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-g, io.buildah.version=1.33.5, org.label-schema.build-date=20240319, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, ceph=True, GIT_CLEAN=True) 2024-03-28T18:04:36.472 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-28T18:04:36.532 INFO:journalctl@ceph.mds.g.smithi022.stdout:Mar 28 18:04:36 smithi022 podman[74046]: 2024-03-28 18:04:36.277959352 +0000 UTC m=+0.431329124 container init 7fac2ccc2c03ac7ba3580a8ffa81e325d3bc121afb563be8a343ef42804dc799 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-g, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.build-date=20240319, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c) 2024-03-28T18:04:36.532 INFO:journalctl@ceph.mds.g.smithi022.stdout:Mar 28 18:04:36 smithi022 podman[74046]: 2024-03-28 18:04:36.291334647 +0000 UTC m=+0.444704419 container start 7fac2ccc2c03ac7ba3580a8ffa81e325d3bc121afb563be8a343ef42804dc799 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-g, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.build-date=20240319, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True) 2024-03-28T18:04:36.532 INFO:journalctl@ceph.mds.g.smithi022.stdout:Mar 28 18:04:36 smithi022 ceph-mds[74070]: -- 172.21.15.22:0/4064725779 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x555a61f5a4e0 con 0x555a61f30800 2024-03-28T18:04:36.532 INFO:journalctl@ceph.mds.g.smithi022.stdout:Mar 28 18:04:36 smithi022 ceph-mds[74070]: -- 172.21.15.22:0/4064725779 <== mon.0 v2:172.21.15.22:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x555a61f5da40 con 0x555a61f30800 2024-03-28T18:04:36.532 INFO:journalctl@ceph.mds.g.smithi022.stdout:Mar 28 18:04:36 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-g[74064]: starting mds.g at 2024-03-28T18:04:36.532 INFO:journalctl@ceph.mds.g.smithi022.stdout:Mar 28 18:04:36 smithi022 bash[74046]: 7fac2ccc2c03ac7ba3580a8ffa81e325d3bc121afb563be8a343ef42804dc799 2024-03-28T18:04:36.532 INFO:journalctl@ceph.mds.g.smithi022.stdout:Mar 28 18:04:36 smithi022 systemd[1]: Started Ceph mds.g for fd220748-ed2b-11ee-95d0-87774f69a715. 2024-03-28T18:04:36.677 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:36.676+0000 7f4613200640 1 -- 172.21.15.22:0/3988149599 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f460c1030c0 msgr2=0x7f460c109950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:04:36.677 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:36.676+0000 7f4613200640 1 --2- 172.21.15.22:0/3988149599 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f460c1030c0 0x7f460c109950 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f4608009ca0 tx=0x7f460802c0f0 comp rx=0 tx=0).stop 2024-03-28T18:04:36.677 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:36.676+0000 7f4613200640 1 -- 172.21.15.22:0/3988149599 shutdown_connections 2024-03-28T18:04:36.677 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:36.676+0000 7f4613200640 1 --2- 172.21.15.22:0/3988149599 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f460c1030c0 0x7f460c109950 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:04:36.677 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:36.676+0000 7f4613200640 1 --2- 172.21.15.22:0/3988149599 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f460c102700 0x7f460c102b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:04:36.677 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:36.676+0000 7f4613200640 1 --2- 172.21.15.22:0/3988149599 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f460c101510 0x7f460c101930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:04:36.677 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:36.676+0000 7f4613200640 1 -- 172.21.15.22:0/3988149599 >> 172.21.15.22:0/3988149599 conn(0x7f460c0fca80 msgr2=0x7f460c0feec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:04:36.678 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:36.676+0000 7f4613200640 1 -- 172.21.15.22:0/3988149599 shutdown_connections 2024-03-28T18:04:36.678 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:36.676+0000 7f4613200640 1 -- 172.21.15.22:0/3988149599 wait complete. 2024-03-28T18:04:36.678 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:36.677+0000 7f4613200640 1 Processor -- start 2024-03-28T18:04:36.678 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:36.677+0000 7f4613200640 1 -- start start 2024-03-28T18:04:36.681 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:36.677+0000 7f4613200640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f460c101510 0x7f460c199900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:04:36.681 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:36.677+0000 7f4613200640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f460c102700 0x7f460c199e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:04:36.681 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:36.677+0000 7f4613200640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f460c1030c0 0x7f460c1a0eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:04:36.681 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:36.677+0000 7f4613200640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f460c10d170 con 0x7f460c102700 2024-03-28T18:04:36.681 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:36.677+0000 7f4613200640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f460c10cff0 con 0x7f460c101510 2024-03-28T18:04:36.681 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:36.677+0000 7f4613200640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f460c10d2f0 con 0x7f460c1030c0 2024-03-28T18:04:36.681 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:36.678+0000 7f4611400640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f460c102700 0x7f460c199e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:04:36.681 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:36.678+0000 7f4611400640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f460c102700 0x7f460c199e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:35034/0 (socket says 172.21.15.22:35034) 2024-03-28T18:04:36.681 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:36.678+0000 7f4611400640 1 -- 172.21.15.22:0/1485986265 learned_addr learned my addr 172.21.15.22:0/1485986265 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-28T18:04:36.682 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:36.678+0000 7f4611e00640 1 --2- 172.21.15.22:0/1485986265 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f460c101510 0x7f460c199900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:04:36.682 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:36.678+0000 7f4612800640 1 --2- 172.21.15.22:0/1485986265 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f460c1030c0 0x7f460c1a0eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:04:36.682 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:36.678+0000 7f4611400640 1 -- 172.21.15.22:0/1485986265 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f460c1030c0 msgr2=0x7f460c1a0eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:04:36.682 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:36.678+0000 7f4611400640 1 --2- 172.21.15.22:0/1485986265 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f460c1030c0 0x7f460c1a0eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:04:36.682 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:36.678+0000 7f4611400640 1 -- 172.21.15.22:0/1485986265 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f460c101510 msgr2=0x7f460c199900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:04:36.682 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:36.678+0000 7f4611400640 1 --2- 172.21.15.22:0/1485986265 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f460c101510 0x7f460c199900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:04:36.682 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:36.678+0000 7f4611400640 1 -- 172.21.15.22:0/1485986265 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f460c1a1520 con 0x7f460c102700 2024-03-28T18:04:36.682 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:36.678+0000 7f4611e00640 1 --2- 172.21.15.22:0/1485986265 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f460c101510 0x7f460c199900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-28T18:04:36.682 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:36.679+0000 7f4611400640 1 --2- 172.21.15.22:0/1485986265 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f460c102700 0x7f460c199e40 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f45fc00c920 tx=0x7f45fc00cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:04:36.682 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:36.679+0000 7f4606a00640 1 -- 172.21.15.22:0/1485986265 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45fc007c70 con 0x7f460c102700 2024-03-28T18:04:36.683 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:36.679+0000 7f4606a00640 1 -- 172.21.15.22:0/1485986265 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f45fc007e10 con 0x7f460c102700 2024-03-28T18:04:36.683 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:36.679+0000 7f4613200640 1 -- 172.21.15.22:0/1485986265 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f460c1a1810 con 0x7f460c102700 2024-03-28T18:04:36.683 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:36.679+0000 7f4606a00640 1 -- 172.21.15.22:0/1485986265 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45fc018850 con 0x7f460c102700 2024-03-28T18:04:36.683 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:36.679+0000 7f4613200640 1 -- 172.21.15.22:0/1485986265 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f460c1a1d60 con 0x7f460c102700 2024-03-28T18:04:36.683 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:36.680+0000 7f4613200640 1 -- 172.21.15.22:0/1485986265 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f460c1a1fc0 con 0x7f460c102700 2024-03-28T18:04:36.683 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:36.684+0000 7f4606a00640 1 -- 172.21.15.22:0/1485986265 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 16) v1 ==== 137290+0+0 (secure 0 0 0) 0x7f45fc018a30 con 0x7f460c102700 2024-03-28T18:04:36.686 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:36.686+0000 7f4606a00640 1 --2- 172.21.15.22:0/1485986265 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7f45f40a2800 0x7f45f40a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:04:36.686 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:36.686+0000 7f4606a00640 1 -- 172.21.15.22:0/1485986265 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f45fc0ce870 con 0x7f460c102700 2024-03-28T18:04:36.686 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:36.686+0000 7f4611e00640 1 --2- 172.21.15.22:0/1485986265 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7f45f40a2800 0x7f45f40a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:04:36.687 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:36.687+0000 7f4611e00640 1 --2- 172.21.15.22:0/1485986265 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7f45f40a2800 0x7f45f40a4cc0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f460c102560 tx=0x7f460000a560 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:04:36.690 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:36.690+0000 7f4606a00640 1 -- 172.21.15.22:0/1485986265 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f45fc014030 con 0x7f460c102700 2024-03-28T18:04:36.932 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:36.932+0000 7f4613200640 1 -- 172.21.15.22:0/1485986265 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f460c1a2230 con 0x7f460c102700 2024-03-28T18:04:36.933 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:36.933+0000 7f4606a00640 1 -- 172.21.15.22:0/1485986265 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 11 v11) v1 ==== 94+0+8453 (secure 0 0 0) 0x7f45fc01f300 con 0x7f460c102700 2024-03-28T18:04:36.934 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-03-28T18:04:36.934 INFO:teuthology.orchestra.run.smithi022.stdout:{"epoch":11,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14544,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6835/3718238950","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6834","nonce":3718238950},{"type":"v1","addr":"172.21.15.22:6835","nonce":3718238950}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14562,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6837/3676939386","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6836","nonce":3676939386},{"type":"v1","addr":"172.21.15.22:6837","nonce":3676939386}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":14640,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6839/3979998310","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6838","nonce":3979998310},{"type":"v1","addr":"172.21.15.22:6839","nonce":3979998310}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":24412,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.134:6833/3176125362","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6832","nonce":3176125362},{"type":"v1","addr":"172.21.15.134:6833","nonce":3176125362}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24463,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.134:6835/3370549207","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6834","nonce":3370549207},{"type":"v1","addr":"172.21.15.134:6835","nonce":3370549207}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":7},{"gid":24478,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.134:6837/899825581","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6836","nonce":899825581},{"type":"v1","addr":"172.21.15.134:6837","nonce":899825581}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":8},{"gid":24502,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.134:6839/3400263217","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6838","nonce":3400263217},{"type":"v1","addr":"172.21.15.134:6839","nonce":3400263217}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":10},{"gid":34442,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.79:6833/856896997","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6832","nonce":856896997},{"type":"v1","addr":"172.21.15.79:6833","nonce":856896997}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":34475,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.79:6835/3365773431","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6834","nonce":3365773431},{"type":"v1","addr":"172.21.15.79:6835","nonce":3365773431}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6},{"gid":34511,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.79:6837/1787903366","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6836","nonce":1787903366},{"type":"v1","addr":"172.21.15.79:6837","nonce":1787903366}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":9}],"filesystems":[]} 2024-03-28T18:04:36.935 INFO:teuthology.orchestra.run.smithi022.stderr:dumped fsmap epoch 11 2024-03-28T18:04:36.937 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:36.936+0000 7f4613200640 1 -- 172.21.15.22:0/1485986265 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7f45f40a2800 msgr2=0x7f45f40a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:04:36.937 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:36.936+0000 7f4613200640 1 --2- 172.21.15.22:0/1485986265 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7f45f40a2800 0x7f45f40a4cc0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f460c102560 tx=0x7f460000a560 comp rx=0 tx=0).stop 2024-03-28T18:04:36.937 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:36.936+0000 7f4613200640 1 -- 172.21.15.22:0/1485986265 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f460c102700 msgr2=0x7f460c199e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:04:36.937 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:36.936+0000 7f4613200640 1 --2- 172.21.15.22:0/1485986265 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f460c102700 0x7f460c199e40 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f45fc00c920 tx=0x7f45fc00cde0 comp rx=0 tx=0).stop 2024-03-28T18:04:36.937 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:36.936+0000 7f4613200640 1 -- 172.21.15.22:0/1485986265 shutdown_connections 2024-03-28T18:04:36.937 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:36.936+0000 7f4613200640 1 --2- 172.21.15.22:0/1485986265 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7f45f40a2800 0x7f45f40a4cc0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:04:36.937 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:36.936+0000 7f4613200640 1 --2- 172.21.15.22:0/1485986265 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f460c1030c0 0x7f460c1a0eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:04:36.937 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:36.936+0000 7f4613200640 1 --2- 172.21.15.22:0/1485986265 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f460c102700 0x7f460c199e40 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:04:36.938 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:36.936+0000 7f4613200640 1 --2- 172.21.15.22:0/1485986265 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f460c101510 0x7f460c199900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:04:36.938 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:36.936+0000 7f4613200640 1 -- 172.21.15.22:0/1485986265 >> 172.21.15.22:0/1485986265 conn(0x7f460c0fca80 msgr2=0x7f460c0738d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:04:36.938 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:36.937+0000 7f4613200640 1 -- 172.21.15.22:0/1485986265 shutdown_connections 2024-03-28T18:04:36.938 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:36.937+0000 7f4613200640 1 -- 172.21.15.22:0/1485986265 wait complete. 2024-03-28T18:04:37.746 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:04:37 smithi022 ceph-mon[31985]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T18:04:37.746 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:04:37 smithi022 ceph-mon[31985]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T18:04:37.746 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:04:37 smithi022 ceph-mon[31985]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T18:04:37.746 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:04:37 smithi022 ceph-mon[31985]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-28T18:04:37.746 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:04:37 smithi022 ceph-mon[31985]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-28T18:04:37.746 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:04:37 smithi022 ceph-mon[31985]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T18:04:37.746 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:04:37 smithi022 ceph-mon[31985]: mds.? [v2:172.21.15.22:6838/3979998310,v1:172.21.15.22:6839/3979998310] up:boot 2024-03-28T18:04:37.747 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:04:37 smithi022 ceph-mon[31985]: fsmap 10 up:standby 2024-03-28T18:04:37.747 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:04:37 smithi022 ceph-mon[31985]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-03-28T18:04:37.747 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:04:37 smithi022 ceph-mon[31985]: from='client.? 172.21.15.22:0/1485986265' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-28T18:04:37.925 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:04:37 smithi134 ceph-mon[35037]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T18:04:37.925 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:04:37 smithi134 ceph-mon[35037]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T18:04:37.925 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:04:37 smithi134 ceph-mon[35037]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T18:04:37.925 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:04:37 smithi134 ceph-mon[35037]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-28T18:04:37.925 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:04:37 smithi134 ceph-mon[35037]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-28T18:04:37.926 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:04:37 smithi134 ceph-mon[35037]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T18:04:37.926 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:04:37 smithi134 ceph-mon[35037]: mds.? [v2:172.21.15.22:6838/3979998310,v1:172.21.15.22:6839/3979998310] up:boot 2024-03-28T18:04:37.926 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:04:37 smithi134 ceph-mon[35037]: fsmap 10 up:standby 2024-03-28T18:04:37.926 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:04:37 smithi134 ceph-mon[35037]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-03-28T18:04:37.926 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:04:37 smithi134 ceph-mon[35037]: from='client.? 172.21.15.22:0/1485986265' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-28T18:04:37.933 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:04:37 smithi079 ceph-mon[35312]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T18:04:37.933 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:04:37 smithi079 ceph-mon[35312]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T18:04:37.933 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:04:37 smithi079 ceph-mon[35312]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T18:04:37.933 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:04:37 smithi079 ceph-mon[35312]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-28T18:04:37.933 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:04:37 smithi079 ceph-mon[35312]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-28T18:04:37.933 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:04:37 smithi079 ceph-mon[35312]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T18:04:37.933 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:04:37 smithi079 ceph-mon[35312]: mds.? [v2:172.21.15.22:6838/3979998310,v1:172.21.15.22:6839/3979998310] up:boot 2024-03-28T18:04:37.933 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:04:37 smithi079 ceph-mon[35312]: fsmap 10 up:standby 2024-03-28T18:04:37.934 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:04:37 smithi079 ceph-mon[35312]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-03-28T18:04:37.934 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:04:37 smithi079 ceph-mon[35312]: from='client.? 172.21.15.22:0/1485986265' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-28T18:04:38.684 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:04:38 smithi022 ceph-mon[31985]: Deploying daemon mds.a on smithi022 2024-03-28T18:04:38.684 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:04:38 smithi022 ceph-mon[31985]: pgmap v359: 1 pgs: 1 active+clean; 577 KiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-28T18:04:38.760 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:04:38 smithi079 ceph-mon[35312]: Deploying daemon mds.a on smithi022 2024-03-28T18:04:38.760 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:04:38 smithi079 ceph-mon[35312]: pgmap v359: 1 pgs: 1 active+clean; 577 KiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-28T18:04:38.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:04:38 smithi134 ceph-mon[35037]: Deploying daemon mds.a on smithi022 2024-03-28T18:04:38.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:04:38 smithi134 ceph-mon[35037]: pgmap v359: 1 pgs: 1 active+clean; 577 KiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-28T18:04:38.957 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-28T18:04:39.279 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:39.278+0000 7f7b18200640 1 -- 172.21.15.22:0/1030357173 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f7b101086a0 msgr2=0x7f7b10108ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:04:39.279 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:39.278+0000 7f7b18200640 1 --2- 172.21.15.22:0/1030357173 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f7b101086a0 0x7f7b10108ac0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f7b04009900 tx=0x7f7b0402bc30 comp rx=0 tx=0).stop 2024-03-28T18:04:39.279 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:39.278+0000 7f7b18200640 1 -- 172.21.15.22:0/1030357173 shutdown_connections 2024-03-28T18:04:39.280 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:39.278+0000 7f7b18200640 1 --2- 172.21.15.22:0/1030357173 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7b1010ff20 0x7f7b10112310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:04:39.280 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:39.278+0000 7f7b18200640 1 --2- 172.21.15.22:0/1030357173 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7b10109000 0x7f7b1010f9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:04:39.280 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:39.278+0000 7f7b18200640 1 --2- 172.21.15.22:0/1030357173 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f7b101086a0 0x7f7b10108ac0 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:04:39.280 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:39.278+0000 7f7b18200640 1 -- 172.21.15.22:0/1030357173 >> 172.21.15.22:0/1030357173 conn(0x7f7b1006e510 msgr2=0x7f7b10070950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:04:39.280 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:39.279+0000 7f7b18200640 1 -- 172.21.15.22:0/1030357173 shutdown_connections 2024-03-28T18:04:39.280 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:39.279+0000 7f7b18200640 1 -- 172.21.15.22:0/1030357173 wait complete. 2024-03-28T18:04:39.283 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:39.280+0000 7f7b18200640 1 Processor -- start 2024-03-28T18:04:39.283 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:39.280+0000 7f7b18200640 1 -- start start 2024-03-28T18:04:39.283 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:39.280+0000 7f7b18200640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7b101086a0 0x7f7b101a68b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:04:39.283 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:39.280+0000 7f7b18200640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f7b10109000 0x7f7b101a6df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:04:39.284 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:39.280+0000 7f7b18200640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7b1010ff20 0x7f7b101ade60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:04:39.284 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:39.280+0000 7f7b18200640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b1010b2a0 con 0x7f7b1010ff20 2024-03-28T18:04:39.284 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:39.280+0000 7f7b18200640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b1010b120 con 0x7f7b10109000 2024-03-28T18:04:39.284 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:39.280+0000 7f7b18200640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b1010b420 con 0x7f7b101086a0 2024-03-28T18:04:39.284 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:39.281+0000 7f7b16200640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7b1010ff20 0x7f7b101ade60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:04:39.284 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:39.281+0000 7f7b16200640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7b1010ff20 0x7f7b101ade60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:34318/0 (socket says 172.21.15.22:34318) 2024-03-28T18:04:39.284 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:39.281+0000 7f7b16200640 1 -- 172.21.15.22:0/2415007221 learned_addr learned my addr 172.21.15.22:0/2415007221 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-28T18:04:39.284 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:39.281+0000 7f7b15800640 1 --2- 172.21.15.22:0/2415007221 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7b101086a0 0x7f7b101a68b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:04:39.284 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:39.281+0000 7f7b14e00640 1 --2- 172.21.15.22:0/2415007221 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f7b10109000 0x7f7b101a6df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:04:39.285 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:39.282+0000 7f7b15800640 1 -- 172.21.15.22:0/2415007221 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f7b10109000 msgr2=0x7f7b101a6df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:04:39.285 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:39.282+0000 7f7b15800640 1 --2- 172.21.15.22:0/2415007221 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f7b10109000 0x7f7b101a6df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:04:39.285 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:39.282+0000 7f7b15800640 1 -- 172.21.15.22:0/2415007221 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7b1010ff20 msgr2=0x7f7b101ade60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:04:39.285 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:39.282+0000 7f7b15800640 1 --2- 172.21.15.22:0/2415007221 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7b1010ff20 0x7f7b101ade60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:04:39.285 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:39.282+0000 7f7b15800640 1 -- 172.21.15.22:0/2415007221 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7b101ae4d0 con 0x7f7b101086a0 2024-03-28T18:04:39.285 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:39.282+0000 7f7b16200640 1 --2- 172.21.15.22:0/2415007221 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7b1010ff20 0x7f7b101ade60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-28T18:04:39.285 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:39.283+0000 7f7b15800640 1 --2- 172.21.15.22:0/2415007221 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7b101086a0 0x7f7b101a68b0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f7b04009230 tx=0x7f7b04005e60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:04:39.285 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:39.283+0000 7f7b0e000640 1 -- 172.21.15.22:0/2415007221 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b04043020 con 0x7f7b101086a0 2024-03-28T18:04:39.285 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:39.283+0000 7f7b18200640 1 -- 172.21.15.22:0/2415007221 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7b101ae760 con 0x7f7b101086a0 2024-03-28T18:04:39.285 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:39.283+0000 7f7b18200640 1 -- 172.21.15.22:0/2415007221 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7b101aed50 con 0x7f7b101086a0 2024-03-28T18:04:39.286 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:39.283+0000 7f7b0e000640 1 -- 172.21.15.22:0/2415007221 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f7b04005470 con 0x7f7b101086a0 2024-03-28T18:04:39.286 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:39.283+0000 7f7b0e000640 1 -- 172.21.15.22:0/2415007221 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b04003b50 con 0x7f7b101086a0 2024-03-28T18:04:39.286 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:39.284+0000 7f7b18200640 1 -- 172.21.15.22:0/2415007221 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7b10068490 con 0x7f7b101086a0 2024-03-28T18:04:39.288 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:39.288+0000 7f7b0e000640 1 -- 172.21.15.22:0/2415007221 <== mon.1 v2:172.21.15.134:3300/0 4 ==== mgrmap(e 16) v1 ==== 137290+0+0 (secure 0 0 0) 0x7f7b04046020 con 0x7f7b101086a0 2024-03-28T18:04:39.289 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:39.289+0000 7f7b0e000640 1 --2- 172.21.15.22:0/2415007221 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7f7af80a2800 0x7f7af80a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:04:39.290 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:39.290+0000 7f7b0e000640 1 -- 172.21.15.22:0/2415007221 <== mon.1 v2:172.21.15.134:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f7b040f0890 con 0x7f7b101086a0 2024-03-28T18:04:39.290 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:39.290+0000 7f7b14e00640 1 --2- 172.21.15.22:0/2415007221 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7f7af80a2800 0x7f7af80a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:04:39.290 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:39.290+0000 7f7b14e00640 1 --2- 172.21.15.22:0/2415007221 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7f7af80a2800 0x7f7af80a4cc0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f7b101a7dc0 tx=0x7f7afc009110 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:04:39.302 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:39.302+0000 7f7b0e000640 1 -- 172.21.15.22:0/2415007221 <== mon.1 v2:172.21.15.134:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f7b0404d030 con 0x7f7b101086a0 2024-03-28T18:04:39.552 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:39.553+0000 7f7b18200640 1 -- 172.21.15.22:0/2415007221 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f7b10117b40 con 0x7f7b101086a0 2024-03-28T18:04:39.554 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:39.554+0000 7f7b0e000640 1 -- 172.21.15.22:0/2415007221 <== mon.1 v2:172.21.15.134:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 11 v11) v1 ==== 94+0+8453 (secure 0 0 0) 0x7f7b04003cf0 con 0x7f7b101086a0 2024-03-28T18:04:39.555 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-03-28T18:04:39.555 INFO:teuthology.orchestra.run.smithi022.stdout:{"epoch":11,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14544,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6835/3718238950","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6834","nonce":3718238950},{"type":"v1","addr":"172.21.15.22:6835","nonce":3718238950}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14562,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6837/3676939386","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6836","nonce":3676939386},{"type":"v1","addr":"172.21.15.22:6837","nonce":3676939386}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":14640,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6839/3979998310","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6838","nonce":3979998310},{"type":"v1","addr":"172.21.15.22:6839","nonce":3979998310}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":24412,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.134:6833/3176125362","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6832","nonce":3176125362},{"type":"v1","addr":"172.21.15.134:6833","nonce":3176125362}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24463,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.134:6835/3370549207","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6834","nonce":3370549207},{"type":"v1","addr":"172.21.15.134:6835","nonce":3370549207}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":7},{"gid":24478,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.134:6837/899825581","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6836","nonce":899825581},{"type":"v1","addr":"172.21.15.134:6837","nonce":899825581}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":8},{"gid":24502,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.134:6839/3400263217","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6838","nonce":3400263217},{"type":"v1","addr":"172.21.15.134:6839","nonce":3400263217}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":10},{"gid":34442,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.79:6833/856896997","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6832","nonce":856896997},{"type":"v1","addr":"172.21.15.79:6833","nonce":856896997}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":34475,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.79:6835/3365773431","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6834","nonce":3365773431},{"type":"v1","addr":"172.21.15.79:6835","nonce":3365773431}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6},{"gid":34511,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.79:6837/1787903366","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6836","nonce":1787903366},{"type":"v1","addr":"172.21.15.79:6837","nonce":1787903366}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":9}],"filesystems":[]} 2024-03-28T18:04:39.555 INFO:teuthology.orchestra.run.smithi022.stderr:dumped fsmap epoch 11 2024-03-28T18:04:39.557 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:39.557+0000 7f7b18200640 1 -- 172.21.15.22:0/2415007221 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7f7af80a2800 msgr2=0x7f7af80a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:04:39.557 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:39.557+0000 7f7b18200640 1 --2- 172.21.15.22:0/2415007221 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7f7af80a2800 0x7f7af80a4cc0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f7b101a7dc0 tx=0x7f7afc009110 comp rx=0 tx=0).stop 2024-03-28T18:04:39.558 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:39.557+0000 7f7b18200640 1 -- 172.21.15.22:0/2415007221 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7b101086a0 msgr2=0x7f7b101a68b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:04:39.558 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:39.557+0000 7f7b18200640 1 --2- 172.21.15.22:0/2415007221 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7b101086a0 0x7f7b101a68b0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f7b04009230 tx=0x7f7b04005e60 comp rx=0 tx=0).stop 2024-03-28T18:04:39.558 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:39.557+0000 7f7b18200640 1 -- 172.21.15.22:0/2415007221 shutdown_connections 2024-03-28T18:04:39.558 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:39.557+0000 7f7b18200640 1 --2- 172.21.15.22:0/2415007221 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7f7af80a2800 0x7f7af80a4cc0 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:04:39.558 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:39.557+0000 7f7b18200640 1 --2- 172.21.15.22:0/2415007221 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7b1010ff20 0x7f7b101ade60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:04:39.558 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:39.557+0000 7f7b18200640 1 --2- 172.21.15.22:0/2415007221 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f7b10109000 0x7f7b101a6df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:04:39.558 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:39.557+0000 7f7b18200640 1 --2- 172.21.15.22:0/2415007221 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7b101086a0 0x7f7b101a68b0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:04:39.558 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:39.557+0000 7f7b18200640 1 -- 172.21.15.22:0/2415007221 >> 172.21.15.22:0/2415007221 conn(0x7f7b1006e510 msgr2=0x7f7b10070220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:04:39.558 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:39.557+0000 7f7b18200640 1 -- 172.21.15.22:0/2415007221 shutdown_connections 2024-03-28T18:04:39.558 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:39.557+0000 7f7b18200640 1 -- 172.21.15.22:0/2415007221 wait complete. 2024-03-28T18:04:40.192 INFO:journalctl@ceph.mds.a.smithi022.stdout:Mar 28 18:04:39 smithi022 podman[74435]: 2024-03-28 18:04:39.95346156 +0000 UTC m=+0.215488833 container create 48f0c7328169d07bba83f7393a656ca2212b65abfe40ce9d98da1fa42b6a653b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-a, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, ceph=True, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_BRANCH=HEAD, org.label-schema.build-date=20240319) 2024-03-28T18:04:40.192 INFO:journalctl@ceph.mds.a.smithi022.stdout:Mar 28 18:04:40 smithi022 podman[74435]: 2024-03-28 18:04:40.190704261 +0000 UTC m=+0.452731541 container init 48f0c7328169d07bba83f7393a656ca2212b65abfe40ce9d98da1fa42b6a653b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-a, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, ceph=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240319, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_CLEAN=True, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5) 2024-03-28T18:04:40.519 INFO:journalctl@ceph.mds.a.smithi022.stdout:Mar 28 18:04:40 smithi022 podman[74435]: 2024-03-28 18:04:40.201350622 +0000 UTC m=+0.463377902 container start 48f0c7328169d07bba83f7393a656ca2212b65abfe40ce9d98da1fa42b6a653b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-a, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, org.label-schema.build-date=20240319, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, ceph=True, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30) 2024-03-28T18:04:40.520 INFO:journalctl@ceph.mds.a.smithi022.stdout:Mar 28 18:04:40 smithi022 ceph-mds[74453]: -- 172.21.15.22:0/3460725860 <== mon.0 v2:172.21.15.22:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x5584c1e21a40 con 0x5584c1df4800 2024-03-28T18:04:40.520 INFO:journalctl@ceph.mds.a.smithi022.stdout:Mar 28 18:04:40 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-a[74449]: starting mds.a at 2024-03-28T18:04:40.520 INFO:journalctl@ceph.mds.a.smithi022.stdout:Mar 28 18:04:40 smithi022 bash[74435]: 48f0c7328169d07bba83f7393a656ca2212b65abfe40ce9d98da1fa42b6a653b 2024-03-28T18:04:40.520 INFO:journalctl@ceph.mds.a.smithi022.stdout:Mar 28 18:04:40 smithi022 systemd[1]: Started Ceph mds.a for fd220748-ed2b-11ee-95d0-87774f69a715. 2024-03-28T18:04:40.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:04:40 smithi079 ceph-mon[35312]: pgmap v360: 1 pgs: 1 active+clean; 577 KiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-28T18:04:40.683 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:04:40 smithi079 ceph-mon[35312]: from='client.? 172.21.15.22:0/2415007221' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-28T18:04:40.683 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:04:40 smithi079 ceph-mon[35312]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T18:04:40.683 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:04:40 smithi079 ceph-mon[35312]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T18:04:40.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:04:40 smithi022 ceph-mon[31985]: pgmap v360: 1 pgs: 1 active+clean; 577 KiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-28T18:04:40.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:04:40 smithi022 ceph-mon[31985]: from='client.? 172.21.15.22:0/2415007221' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-28T18:04:40.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:04:40 smithi022 ceph-mon[31985]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T18:04:40.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:04:40 smithi022 ceph-mon[31985]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T18:04:40.925 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:04:40 smithi134 ceph-mon[35037]: pgmap v360: 1 pgs: 1 active+clean; 577 KiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-28T18:04:40.925 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:04:40 smithi134 ceph-mon[35037]: from='client.? 172.21.15.22:0/2415007221' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-28T18:04:40.925 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:04:40 smithi134 ceph-mon[35037]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T18:04:40.925 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:04:40 smithi134 ceph-mon[35037]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T18:04:41.580 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-28T18:04:41.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:04:41 smithi022 ceph-mon[31985]: mds.? [v2:172.21.15.22:6840/3069056661,v1:172.21.15.22:6841/3069056661] up:boot 2024-03-28T18:04:41.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:04:41 smithi022 ceph-mon[31985]: fsmap 11 up:standby 2024-03-28T18:04:41.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:04:41 smithi022 ceph-mon[31985]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-03-28T18:04:41.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:04:41 smithi022 ceph-mon[31985]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T18:04:41.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:04:41 smithi022 ceph-mon[31985]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-28T18:04:41.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:04:41 smithi022 ceph-mon[31985]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-28T18:04:41.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:04:41 smithi022 ceph-mon[31985]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T18:04:41.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:04:41 smithi022 ceph-mon[31985]: Deploying daemon mds.b on smithi079 2024-03-28T18:04:41.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:04:41 smithi022 ceph-mon[31985]: pgmap v361: 1 pgs: 1 active+clean; 577 KiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-28T18:04:41.820 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:41.820+0000 7fddf9600640 1 -- 172.21.15.22:0/611514163 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fddf41052a0 msgr2=0x7fddf410bb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:04:41.820 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:41.820+0000 7fddf9600640 1 --2- 172.21.15.22:0/611514163 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fddf41052a0 0x7fddf410bb30 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fddec009ca0 tx=0x7fddec02c100 comp rx=0 tx=0).stop 2024-03-28T18:04:41.820 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:41.820+0000 7fddf9600640 1 -- 172.21.15.22:0/611514163 shutdown_connections 2024-03-28T18:04:41.820 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:41.820+0000 7fddf9600640 1 --2- 172.21.15.22:0/611514163 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fddf41052a0 0x7fddf410bb30 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:04:41.821 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:41.820+0000 7fddf9600640 1 --2- 172.21.15.22:0/611514163 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fddf41048e0 0x7fddf4104d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:04:41.821 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:41.820+0000 7fddf9600640 1 --2- 172.21.15.22:0/611514163 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fddf41036f0 0x7fddf4103b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:04:41.821 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:41.820+0000 7fddf9600640 1 -- 172.21.15.22:0/611514163 >> 172.21.15.22:0/611514163 conn(0x7fddf40fec80 msgr2=0x7fddf41010a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:04:41.821 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:41.820+0000 7fddf9600640 1 -- 172.21.15.22:0/611514163 shutdown_connections 2024-03-28T18:04:41.821 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:41.820+0000 7fddf9600640 1 -- 172.21.15.22:0/611514163 wait complete. 2024-03-28T18:04:41.823 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:41.821+0000 7fddf9600640 1 Processor -- start 2024-03-28T18:04:41.823 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:41.821+0000 7fddf9600640 1 -- start start 2024-03-28T18:04:41.823 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:41.822+0000 7fddf9600640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fddf41036f0 0x7fddf4199900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:04:41.823 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:41.822+0000 7fddf9600640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fddf41048e0 0x7fddf4199e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:04:41.823 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:41.822+0000 7fddf9600640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fddf41052a0 0x7fddf41a0eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:04:41.824 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:41.822+0000 7fddf9600640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fddf410f350 con 0x7fddf41036f0 2024-03-28T18:04:41.824 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:41.822+0000 7fddf9600640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fddf410f1d0 con 0x7fddf41048e0 2024-03-28T18:04:41.824 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:41.822+0000 7fddf9600640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fddf410f4d0 con 0x7fddf41052a0 2024-03-28T18:04:41.824 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:41.822+0000 7fddf3e00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fddf41036f0 0x7fddf4199900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:04:41.824 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:41.822+0000 7fddf3e00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fddf41036f0 0x7fddf4199900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:34342/0 (socket says 172.21.15.22:34342) 2024-03-28T18:04:41.824 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:41.822+0000 7fddf3e00640 1 -- 172.21.15.22:0/135981939 learned_addr learned my addr 172.21.15.22:0/135981939 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-28T18:04:41.824 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:41.822+0000 7fddf3400640 1 --2- 172.21.15.22:0/135981939 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fddf41048e0 0x7fddf4199e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:04:41.824 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:41.822+0000 7fddf8c00640 1 --2- 172.21.15.22:0/135981939 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fddf41052a0 0x7fddf41a0eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:04:41.824 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:41.823+0000 7fddf3e00640 1 -- 172.21.15.22:0/135981939 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fddf41052a0 msgr2=0x7fddf41a0eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:04:41.824 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:41.823+0000 7fddf3e00640 1 --2- 172.21.15.22:0/135981939 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fddf41052a0 0x7fddf41a0eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:04:41.824 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:41.823+0000 7fddf3e00640 1 -- 172.21.15.22:0/135981939 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fddf41048e0 msgr2=0x7fddf4199e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:04:41.824 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:41.823+0000 7fddf3e00640 1 --2- 172.21.15.22:0/135981939 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fddf41048e0 0x7fddf4199e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:04:41.824 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:41.823+0000 7fddf3e00640 1 -- 172.21.15.22:0/135981939 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fddf41a1520 con 0x7fddf41036f0 2024-03-28T18:04:41.824 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:41.823+0000 7fddf3400640 1 --2- 172.21.15.22:0/135981939 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fddf41048e0 0x7fddf4199e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-28T18:04:41.824 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:41.823+0000 7fddf3e00640 1 --2- 172.21.15.22:0/135981939 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fddf41036f0 0x7fddf4199900 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fdde400c970 tx=0x7fdde400ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:04:41.825 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:41.823+0000 7fddf0c00640 1 -- 172.21.15.22:0/135981939 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdde4007c10 con 0x7fddf41036f0 2024-03-28T18:04:41.825 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:41.823+0000 7fddf0c00640 1 -- 172.21.15.22:0/135981939 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7fdde4007db0 con 0x7fddf41036f0 2024-03-28T18:04:41.825 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:41.823+0000 7fddf9600640 1 -- 172.21.15.22:0/135981939 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fddf41a1810 con 0x7fddf41036f0 2024-03-28T18:04:41.825 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:41.823+0000 7fddf0c00640 1 -- 172.21.15.22:0/135981939 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdde400f8e0 con 0x7fddf41036f0 2024-03-28T18:04:41.825 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:41.823+0000 7fddf9600640 1 -- 172.21.15.22:0/135981939 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fddf41a1d60 con 0x7fddf41036f0 2024-03-28T18:04:41.825 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:41.825+0000 7fddf9600640 1 -- 172.21.15.22:0/135981939 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fddf410a9b0 con 0x7fddf41036f0 2024-03-28T18:04:41.826 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:41.827+0000 7fddf0c00640 1 -- 172.21.15.22:0/135981939 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 16) v1 ==== 137290+0+0 (secure 0 0 0) 0x7fdde400fa80 con 0x7fddf41036f0 2024-03-28T18:04:41.828 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:41.828+0000 7fddf0c00640 1 --2- 172.21.15.22:0/135981939 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7fddd80a2800 0x7fddd80a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:04:41.828 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:41.828+0000 7fddf0c00640 1 -- 172.21.15.22:0/135981939 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 7127+0+0 (secure 0 0 0) 0x7fdde40cea40 con 0x7fddf41036f0 2024-03-28T18:04:41.828 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:41.828+0000 7fddf3400640 1 --2- 172.21.15.22:0/135981939 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7fddd80a2800 0x7fddd80a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:04:41.829 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:41.829+0000 7fddf3400640 1 --2- 172.21.15.22:0/135981939 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7fddd80a2800 0x7fddd80a4cc0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fdde00058c0 tx=0x7fdde0005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:04:41.834 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:41.834+0000 7fddf0c00640 1 -- 172.21.15.22:0/135981939 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fdde409bf70 con 0x7fddf41036f0 2024-03-28T18:04:41.869 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:04:41 smithi079 ceph-mon[35312]: mds.? [v2:172.21.15.22:6840/3069056661,v1:172.21.15.22:6841/3069056661] up:boot 2024-03-28T18:04:41.869 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:04:41 smithi079 ceph-mon[35312]: fsmap 11 up:standby 2024-03-28T18:04:41.869 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:04:41 smithi079 ceph-mon[35312]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-03-28T18:04:41.869 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:04:41 smithi079 ceph-mon[35312]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T18:04:41.869 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:04:41 smithi079 ceph-mon[35312]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-28T18:04:41.869 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:04:41 smithi079 ceph-mon[35312]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-28T18:04:41.869 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:04:41 smithi079 ceph-mon[35312]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T18:04:41.870 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:04:41 smithi079 ceph-mon[35312]: Deploying daemon mds.b on smithi079 2024-03-28T18:04:41.870 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:04:41 smithi079 ceph-mon[35312]: pgmap v361: 1 pgs: 1 active+clean; 577 KiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-28T18:04:41.925 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:04:41 smithi134 ceph-mon[35037]: mds.? [v2:172.21.15.22:6840/3069056661,v1:172.21.15.22:6841/3069056661] up:boot 2024-03-28T18:04:41.925 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:04:41 smithi134 ceph-mon[35037]: fsmap 11 up:standby 2024-03-28T18:04:41.925 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:04:41 smithi134 ceph-mon[35037]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-03-28T18:04:41.925 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:04:41 smithi134 ceph-mon[35037]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T18:04:41.925 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:04:41 smithi134 ceph-mon[35037]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-28T18:04:41.925 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:04:41 smithi134 ceph-mon[35037]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-28T18:04:41.925 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:04:41 smithi134 ceph-mon[35037]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T18:04:41.925 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:04:41 smithi134 ceph-mon[35037]: Deploying daemon mds.b on smithi079 2024-03-28T18:04:41.925 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:04:41 smithi134 ceph-mon[35037]: pgmap v361: 1 pgs: 1 active+clean; 577 KiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-28T18:04:42.081 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:42.081+0000 7fddf9600640 1 -- 172.21.15.22:0/135981939 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fddf402d070 con 0x7fddf41036f0 2024-03-28T18:04:42.083 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:42.083+0000 7fddf0c00640 1 -- 172.21.15.22:0/135981939 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 12 v12) v1 ==== 94+0+9245 (secure 0 0 0) 0x7fddf402d070 con 0x7fddf41036f0 2024-03-28T18:04:42.083 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-03-28T18:04:42.083 INFO:teuthology.orchestra.run.smithi022.stdout:{"epoch":12,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14544,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6835/3718238950","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6834","nonce":3718238950},{"type":"v1","addr":"172.21.15.22:6835","nonce":3718238950}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14562,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6837/3676939386","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6836","nonce":3676939386},{"type":"v1","addr":"172.21.15.22:6837","nonce":3676939386}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":14640,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6839/3979998310","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6838","nonce":3979998310},{"type":"v1","addr":"172.21.15.22:6839","nonce":3979998310}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":14655,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6841/3069056661","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6840","nonce":3069056661},{"type":"v1","addr":"172.21.15.22:6841","nonce":3069056661}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":24412,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.134:6833/3176125362","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6832","nonce":3176125362},{"type":"v1","addr":"172.21.15.134:6833","nonce":3176125362}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24463,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.134:6835/3370549207","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6834","nonce":3370549207},{"type":"v1","addr":"172.21.15.134:6835","nonce":3370549207}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":7},{"gid":24478,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.134:6837/899825581","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6836","nonce":899825581},{"type":"v1","addr":"172.21.15.134:6837","nonce":899825581}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":8},{"gid":24502,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.134:6839/3400263217","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6838","nonce":3400263217},{"type":"v1","addr":"172.21.15.134:6839","nonce":3400263217}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":10},{"gid":34442,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.79:6833/856896997","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6832","nonce":856896997},{"type":"v1","addr":"172.21.15.79:6833","nonce":856896997}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":34475,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.79:6835/3365773431","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6834","nonce":3365773431},{"type":"v1","addr":"172.21.15.79:6835","nonce":3365773431}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6},{"gid":34511,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.79:6837/1787903366","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6836","nonce":1787903366},{"type":"v1","addr":"172.21.15.79:6837","nonce":1787903366}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":9}],"filesystems":[]} 2024-03-28T18:04:42.084 INFO:teuthology.orchestra.run.smithi022.stderr:dumped fsmap epoch 12 2024-03-28T18:04:42.086 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:42.086+0000 7fddf9600640 1 -- 172.21.15.22:0/135981939 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7fddd80a2800 msgr2=0x7fddd80a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:04:42.086 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:42.086+0000 7fddf9600640 1 --2- 172.21.15.22:0/135981939 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7fddd80a2800 0x7fddd80a4cc0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fdde00058c0 tx=0x7fdde0005850 comp rx=0 tx=0).stop 2024-03-28T18:04:42.087 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:42.086+0000 7fddf9600640 1 -- 172.21.15.22:0/135981939 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fddf41036f0 msgr2=0x7fddf4199900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:04:42.087 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:42.086+0000 7fddf9600640 1 --2- 172.21.15.22:0/135981939 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fddf41036f0 0x7fddf4199900 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fdde400c970 tx=0x7fdde400ce30 comp rx=0 tx=0).stop 2024-03-28T18:04:42.087 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:42.086+0000 7fddf9600640 1 -- 172.21.15.22:0/135981939 shutdown_connections 2024-03-28T18:04:42.087 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:42.086+0000 7fddf9600640 1 --2- 172.21.15.22:0/135981939 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7fddd80a2800 0x7fddd80a4cc0 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:04:42.087 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:42.086+0000 7fddf9600640 1 --2- 172.21.15.22:0/135981939 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fddf41052a0 0x7fddf41a0eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:04:42.087 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:42.086+0000 7fddf9600640 1 --2- 172.21.15.22:0/135981939 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fddf41048e0 0x7fddf4199e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:04:42.087 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:42.086+0000 7fddf9600640 1 --2- 172.21.15.22:0/135981939 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fddf41036f0 0x7fddf4199900 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:04:42.087 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:42.086+0000 7fddf9600640 1 -- 172.21.15.22:0/135981939 >> 172.21.15.22:0/135981939 conn(0x7fddf40fec80 msgr2=0x7fddf4107b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:04:42.087 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:42.087+0000 7fddf9600640 1 -- 172.21.15.22:0/135981939 shutdown_connections 2024-03-28T18:04:42.087 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:42.087+0000 7fddf9600640 1 -- 172.21.15.22:0/135981939 wait complete. 2024-03-28T18:04:42.569 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:04:42 smithi079 ceph-mon[35312]: from='client.? 172.21.15.22:0/135981939' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-28T18:04:42.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:04:42 smithi134 ceph-mon[35037]: from='client.? 172.21.15.22:0/135981939' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-28T18:04:43.032 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:04:42 smithi022 ceph-mon[31985]: from='client.? 172.21.15.22:0/135981939' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-28T18:04:43.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:04:43 smithi134 ceph-mon[35037]: pgmap v362: 1 pgs: 1 active+clean; 577 KiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-28T18:04:43.933 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:04:43 smithi079 ceph-mon[35312]: pgmap v362: 1 pgs: 1 active+clean; 577 KiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-28T18:04:43.933 INFO:journalctl@ceph.mds.b.smithi079.stdout:Mar 28 18:04:43 smithi079 systemd[1]: Starting Ceph mds.b for fd220748-ed2b-11ee-95d0-87774f69a715... 2024-03-28T18:04:44.031 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:04:43 smithi022 ceph-mon[31985]: pgmap v362: 1 pgs: 1 active+clean; 577 KiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-28T18:04:44.106 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-28T18:04:44.220 INFO:journalctl@ceph.mds.b.smithi079.stdout:Mar 28 18:04:43 smithi079 podman[65230]: 2024-03-28 18:04:43.844555771 +0000 UTC m=+0.032203421 image pull c664afe56bb01db4f9afa9532c20598e201e664f2e9ba2be50db0d52674a26d2 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc 2024-03-28T18:04:44.220 INFO:journalctl@ceph.mds.b.smithi079.stdout:Mar 28 18:04:43 smithi079 podman[65230]: 2024-03-28 18:04:43.948962469 +0000 UTC m=+0.136610150 container create 23e0c2fd50ec3c2e9788e47bb12ac627a7c872011839acf093deb6b8d13a9ede (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-b, GIT_CLEAN=True, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.build-date=20240319, org.label-schema.license=GPLv2, ceph=True, org.label-schema.schema-version=1.0, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux ) 2024-03-28T18:04:44.220 INFO:journalctl@ceph.mds.b.smithi079.stdout:Mar 28 18:04:44 smithi079 podman[65230]: 2024-03-28 18:04:44.219329557 +0000 UTC m=+0.406977244 container init 23e0c2fd50ec3c2e9788e47bb12ac627a7c872011839acf093deb6b8d13a9ede (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-b, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, GIT_CLEAN=True, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.build-date=20240319, CEPH_POINT_RELEASE=, ceph=True) 2024-03-28T18:04:44.339 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:44.339+0000 7f329a000640 1 -- 172.21.15.22:0/3252918724 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f32941030b0 msgr2=0x7f3294109940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:04:44.340 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:44.339+0000 7f329a000640 1 --2- 172.21.15.22:0/3252918724 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f32941030b0 0x7f3294109940 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f328c009900 tx=0x7f328c02bc30 comp rx=0 tx=0).stop 2024-03-28T18:04:44.340 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:44.339+0000 7f329a000640 1 -- 172.21.15.22:0/3252918724 shutdown_connections 2024-03-28T18:04:44.340 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:44.339+0000 7f329a000640 1 --2- 172.21.15.22:0/3252918724 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f32941030b0 0x7f3294109940 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:04:44.340 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:44.339+0000 7f329a000640 1 --2- 172.21.15.22:0/3252918724 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f32941026f0 0x7f3294102b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:04:44.340 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:44.339+0000 7f329a000640 1 --2- 172.21.15.22:0/3252918724 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3294101500 0x7f3294101920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:04:44.340 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:44.339+0000 7f329a000640 1 -- 172.21.15.22:0/3252918724 >> 172.21.15.22:0/3252918724 conn(0x7f32940fca70 msgr2=0x7f32940feeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:04:44.340 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:44.339+0000 7f329a000640 1 -- 172.21.15.22:0/3252918724 shutdown_connections 2024-03-28T18:04:44.340 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:44.339+0000 7f329a000640 1 -- 172.21.15.22:0/3252918724 wait complete. 2024-03-28T18:04:44.342 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:44.340+0000 7f329a000640 1 Processor -- start 2024-03-28T18:04:44.342 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:44.341+0000 7f329a000640 1 -- start start 2024-03-28T18:04:44.343 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:44.341+0000 7f329a000640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f3294101500 0x7f32941999c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:04:44.343 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:44.341+0000 7f329a000640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f32941026f0 0x7f3294199f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:04:44.343 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:44.341+0000 7f329a000640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f32941030b0 0x7f32941a0f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:04:44.343 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:44.341+0000 7f329a000640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f329410d1b0 con 0x7f32941026f0 2024-03-28T18:04:44.343 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:44.341+0000 7f329a000640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f329410d030 con 0x7f3294101500 2024-03-28T18:04:44.343 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:44.341+0000 7f329a000640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f329410d330 con 0x7f32941030b0 2024-03-28T18:04:44.343 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:44.341+0000 7f3293e00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f32941026f0 0x7f3294199f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:04:44.343 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:44.341+0000 7f3293e00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f32941026f0 0x7f3294199f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:34358/0 (socket says 172.21.15.22:34358) 2024-03-28T18:04:44.343 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:44.341+0000 7f3293e00640 1 -- 172.21.15.22:0/2696080688 learned_addr learned my addr 172.21.15.22:0/2696080688 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-28T18:04:44.343 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:44.341+0000 7f3298c00640 1 --2- 172.21.15.22:0/2696080688 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f3294101500 0x7f32941999c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:04:44.343 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:44.342+0000 7f3299600640 1 --2- 172.21.15.22:0/2696080688 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f32941030b0 0x7f32941a0f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:04:44.343 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:44.342+0000 7f3293e00640 1 -- 172.21.15.22:0/2696080688 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f32941030b0 msgr2=0x7f32941a0f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:04:44.343 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:44.342+0000 7f3293e00640 1 --2- 172.21.15.22:0/2696080688 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f32941030b0 0x7f32941a0f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:04:44.344 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:44.342+0000 7f3293e00640 1 -- 172.21.15.22:0/2696080688 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f3294101500 msgr2=0x7f32941999c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:04:44.344 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:44.342+0000 7f3293e00640 1 --2- 172.21.15.22:0/2696080688 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f3294101500 0x7f32941999c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:04:44.344 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:44.342+0000 7f3293e00640 1 -- 172.21.15.22:0/2696080688 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f32941a15e0 con 0x7f32941026f0 2024-03-28T18:04:44.344 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:44.342+0000 7f3298c00640 1 --2- 172.21.15.22:0/2696080688 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f3294101500 0x7f32941999c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-28T18:04:44.344 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:44.342+0000 7f3299600640 1 --2- 172.21.15.22:0/2696080688 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f32941030b0 0x7f32941a0f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T18:04:44.344 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:44.342+0000 7f3293e00640 1 --2- 172.21.15.22:0/2696080688 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f32941026f0 0x7f3294199f00 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f327c00d950 tx=0x7f327c00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:04:44.344 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:44.342+0000 7f3291600640 1 -- 172.21.15.22:0/2696080688 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f327c017030 con 0x7f32941026f0 2024-03-28T18:04:44.344 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:44.342+0000 7f3291600640 1 -- 172.21.15.22:0/2696080688 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f327c004020 con 0x7f32941026f0 2024-03-28T18:04:44.344 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:44.342+0000 7f329a000640 1 -- 172.21.15.22:0/2696080688 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f32941a18d0 con 0x7f32941026f0 2024-03-28T18:04:44.344 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:44.342+0000 7f3291600640 1 -- 172.21.15.22:0/2696080688 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f327c0047a0 con 0x7f32941026f0 2024-03-28T18:04:44.344 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:44.342+0000 7f329a000640 1 -- 172.21.15.22:0/2696080688 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f32941a1e20 con 0x7f32941026f0 2024-03-28T18:04:44.344 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:44.343+0000 7f329a000640 1 -- 172.21.15.22:0/2696080688 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f32941a20b0 con 0x7f32941026f0 2024-03-28T18:04:44.345 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:44.346+0000 7f3291600640 1 -- 172.21.15.22:0/2696080688 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 16) v1 ==== 137290+0+0 (secure 0 0 0) 0x7f327c0041d0 con 0x7f32941026f0 2024-03-28T18:04:44.347 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:44.348+0000 7f3291600640 1 --2- 172.21.15.22:0/2696080688 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7f32780a2800 0x7f32780a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:04:44.348 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:44.348+0000 7f3291600640 1 -- 172.21.15.22:0/2696080688 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f327c0ce140 con 0x7f32941026f0 2024-03-28T18:04:44.348 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:44.348+0000 7f3298c00640 1 --2- 172.21.15.22:0/2696080688 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7f32780a2800 0x7f32780a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:04:44.349 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:44.349+0000 7f3298c00640 1 --2- 172.21.15.22:0/2696080688 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7f32780a2800 0x7f32780a4cc0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f3294102550 tx=0x7f3284009520 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:04:44.353 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:44.353+0000 7f3291600640 1 -- 172.21.15.22:0/2696080688 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f327c09b5e0 con 0x7f32941026f0 2024-03-28T18:04:44.609 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:44.609+0000 7f329a000640 1 -- 172.21.15.22:0/2696080688 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f32941a2280 con 0x7f32941026f0 2024-03-28T18:04:44.611 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:44.611+0000 7f3291600640 1 -- 172.21.15.22:0/2696080688 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 13 v13) v1 ==== 94+0+10037 (secure 0 0 0) 0x7f327c09aee0 con 0x7f32941026f0 2024-03-28T18:04:44.611 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-03-28T18:04:44.612 INFO:teuthology.orchestra.run.smithi022.stdout:{"epoch":13,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14544,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6835/3718238950","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6834","nonce":3718238950},{"type":"v1","addr":"172.21.15.22:6835","nonce":3718238950}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14562,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6837/3676939386","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6836","nonce":3676939386},{"type":"v1","addr":"172.21.15.22:6837","nonce":3676939386}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":14640,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6839/3979998310","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6838","nonce":3979998310},{"type":"v1","addr":"172.21.15.22:6839","nonce":3979998310}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":14655,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6841/3069056661","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6840","nonce":3069056661},{"type":"v1","addr":"172.21.15.22:6841","nonce":3069056661}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":24412,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.134:6833/3176125362","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6832","nonce":3176125362},{"type":"v1","addr":"172.21.15.134:6833","nonce":3176125362}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24463,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.134:6835/3370549207","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6834","nonce":3370549207},{"type":"v1","addr":"172.21.15.134:6835","nonce":3370549207}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":7},{"gid":24478,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.134:6837/899825581","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6836","nonce":899825581},{"type":"v1","addr":"172.21.15.134:6837","nonce":899825581}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":8},{"gid":24502,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.134:6839/3400263217","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6838","nonce":3400263217},{"type":"v1","addr":"172.21.15.134:6839","nonce":3400263217}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":10},{"gid":34442,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.79:6833/856896997","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6832","nonce":856896997},{"type":"v1","addr":"172.21.15.79:6833","nonce":856896997}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":34475,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.79:6835/3365773431","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6834","nonce":3365773431},{"type":"v1","addr":"172.21.15.79:6835","nonce":3365773431}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6},{"gid":34511,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.79:6837/1787903366","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6836","nonce":1787903366},{"type":"v1","addr":"172.21.15.79:6837","nonce":1787903366}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":9},{"gid":34553,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.79:6839/2441542443","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6838","nonce":2441542443},{"type":"v1","addr":"172.21.15.79:6839","nonce":2441542443}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":13}],"filesystems":[]} 2024-03-28T18:04:44.612 INFO:teuthology.orchestra.run.smithi022.stderr:dumped fsmap epoch 13 2024-03-28T18:04:44.614 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:44.614+0000 7f329a000640 1 -- 172.21.15.22:0/2696080688 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7f32780a2800 msgr2=0x7f32780a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:04:44.614 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:44.614+0000 7f329a000640 1 --2- 172.21.15.22:0/2696080688 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7f32780a2800 0x7f32780a4cc0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f3294102550 tx=0x7f3284009520 comp rx=0 tx=0).stop 2024-03-28T18:04:44.614 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:44.614+0000 7f329a000640 1 -- 172.21.15.22:0/2696080688 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f32941026f0 msgr2=0x7f3294199f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:04:44.614 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:44.614+0000 7f329a000640 1 --2- 172.21.15.22:0/2696080688 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f32941026f0 0x7f3294199f00 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f327c00d950 tx=0x7f327c00de10 comp rx=0 tx=0).stop 2024-03-28T18:04:44.614 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:44.614+0000 7f329a000640 1 -- 172.21.15.22:0/2696080688 shutdown_connections 2024-03-28T18:04:44.614 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:44.614+0000 7f329a000640 1 --2- 172.21.15.22:0/2696080688 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7f32780a2800 0x7f32780a4cc0 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:04:44.614 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:44.614+0000 7f329a000640 1 --2- 172.21.15.22:0/2696080688 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f32941030b0 0x7f32941a0f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:04:44.614 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:44.614+0000 7f329a000640 1 --2- 172.21.15.22:0/2696080688 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f32941026f0 0x7f3294199f00 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:04:44.614 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:44.614+0000 7f329a000640 1 --2- 172.21.15.22:0/2696080688 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f3294101500 0x7f32941999c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:04:44.614 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:44.614+0000 7f329a000640 1 -- 172.21.15.22:0/2696080688 >> 172.21.15.22:0/2696080688 conn(0x7f32940fca70 msgr2=0x7f32940738d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:04:44.614 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:44.614+0000 7f329a000640 1 -- 172.21.15.22:0/2696080688 shutdown_connections 2024-03-28T18:04:44.614 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:44.614+0000 7f329a000640 1 -- 172.21.15.22:0/2696080688 wait complete. 2024-03-28T18:04:44.638 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-28T18:04:44.682 INFO:journalctl@ceph.mds.b.smithi079.stdout:Mar 28 18:04:44 smithi079 podman[65230]: 2024-03-28 18:04:44.22933863 +0000 UTC m=+0.416986319 container start 23e0c2fd50ec3c2e9788e47bb12ac627a7c872011839acf093deb6b8d13a9ede (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-b, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240319, GIT_CLEAN=True, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=) 2024-03-28T18:04:44.682 INFO:journalctl@ceph.mds.b.smithi079.stdout:Mar 28 18:04:44 smithi079 ceph-mds[65249]: -- 172.21.15.79:0/1466138415 <== mon.2 v2:172.21.15.79:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x558709ab3a40 con 0x558709a86c00 2024-03-28T18:04:44.682 INFO:journalctl@ceph.mds.b.smithi079.stdout:Mar 28 18:04:44 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-b[65245]: starting mds.b at 2024-03-28T18:04:44.682 INFO:journalctl@ceph.mds.b.smithi079.stdout:Mar 28 18:04:44 smithi079 bash[65230]: 23e0c2fd50ec3c2e9788e47bb12ac627a7c872011839acf093deb6b8d13a9ede 2024-03-28T18:04:44.683 INFO:journalctl@ceph.mds.b.smithi079.stdout:Mar 28 18:04:44 smithi079 systemd[1]: Started Ceph mds.b for fd220748-ed2b-11ee-95d0-87774f69a715. 2024-03-28T18:04:44.831 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:44.831+0000 7fa27cc00640 1 -- 172.21.15.22:0/3035033954 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa2781036f0 msgr2=0x7fa278103b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:04:44.832 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:44.831+0000 7fa27cc00640 1 --2- 172.21.15.22:0/3035033954 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa2781036f0 0x7fa278103b10 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fa268009900 tx=0x7fa26802bc30 comp rx=0 tx=0).stop 2024-03-28T18:04:44.832 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:44.831+0000 7fa27cc00640 1 -- 172.21.15.22:0/3035033954 shutdown_connections 2024-03-28T18:04:44.832 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:44.831+0000 7fa27cc00640 1 --2- 172.21.15.22:0/3035033954 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa2781052a0 0x7fa27810bb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:04:44.832 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:44.831+0000 7fa27cc00640 1 --2- 172.21.15.22:0/3035033954 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa2781048e0 0x7fa278104d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:04:44.832 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:44.831+0000 7fa27cc00640 1 --2- 172.21.15.22:0/3035033954 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa2781036f0 0x7fa278103b10 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:04:44.832 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:44.831+0000 7fa27cc00640 1 -- 172.21.15.22:0/3035033954 >> 172.21.15.22:0/3035033954 conn(0x7fa2780fec80 msgr2=0x7fa2781010a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:04:44.832 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:44.831+0000 7fa27cc00640 1 -- 172.21.15.22:0/3035033954 shutdown_connections 2024-03-28T18:04:44.832 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:44.831+0000 7fa27cc00640 1 -- 172.21.15.22:0/3035033954 wait complete. 2024-03-28T18:04:44.832 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:44.832+0000 7fa27cc00640 1 Processor -- start 2024-03-28T18:04:44.832 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:44.832+0000 7fa27cc00640 1 -- start start 2024-03-28T18:04:44.834 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:44.832+0000 7fa27cc00640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa2781036f0 0x7fa278199a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:04:44.835 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:44.832+0000 7fa27cc00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa2781048e0 0x7fa278199f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:04:44.835 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:44.832+0000 7fa27cc00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa2781052a0 0x7fa2781a1000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:04:44.835 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:44.832+0000 7fa27cc00640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa27810f3a0 con 0x7fa2781052a0 2024-03-28T18:04:44.835 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:44.832+0000 7fa27cc00640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa27810f220 con 0x7fa2781036f0 2024-03-28T18:04:44.835 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:44.832+0000 7fa27cc00640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa27810f520 con 0x7fa2781048e0 2024-03-28T18:04:44.835 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:44.833+0000 7fa277e00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa2781052a0 0x7fa2781a1000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:04:44.835 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:44.833+0000 7fa277e00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa2781052a0 0x7fa2781a1000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:34372/0 (socket says 172.21.15.22:34372) 2024-03-28T18:04:44.835 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:44.833+0000 7fa277e00640 1 -- 172.21.15.22:0/3522287826 learned_addr learned my addr 172.21.15.22:0/3522287826 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-28T18:04:44.835 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:44.833+0000 7fa26fe00640 1 --2- 172.21.15.22:0/3522287826 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa2781048e0 0x7fa278199f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:04:44.835 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:44.833+0000 7fa277400640 1 --2- 172.21.15.22:0/3522287826 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa2781036f0 0x7fa278199a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:04:44.835 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:44.833+0000 7fa277e00640 1 -- 172.21.15.22:0/3522287826 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa2781048e0 msgr2=0x7fa278199f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:04:44.835 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:44.833+0000 7fa277e00640 1 --2- 172.21.15.22:0/3522287826 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa2781048e0 0x7fa278199f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:04:44.836 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:44.833+0000 7fa277e00640 1 -- 172.21.15.22:0/3522287826 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa2781036f0 msgr2=0x7fa278199a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:04:44.836 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:44.833+0000 7fa277e00640 1 --2- 172.21.15.22:0/3522287826 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa2781036f0 0x7fa278199a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:04:44.836 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:44.833+0000 7fa277e00640 1 -- 172.21.15.22:0/3522287826 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa2781a1670 con 0x7fa2781052a0 2024-03-28T18:04:44.836 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:44.833+0000 7fa277e00640 1 --2- 172.21.15.22:0/3522287826 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa2781052a0 0x7fa2781a1000 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fa27000eaa0 tx=0x7fa27000ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:04:44.836 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:44.834+0000 7fa274c00640 1 -- 172.21.15.22:0/3522287826 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa27000cdb0 con 0x7fa2781052a0 2024-03-28T18:04:44.836 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:44.834+0000 7fa274c00640 1 -- 172.21.15.22:0/3522287826 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7fa270005020 con 0x7fa2781052a0 2024-03-28T18:04:44.836 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:44.834+0000 7fa274c00640 1 -- 172.21.15.22:0/3522287826 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2700116b0 con 0x7fa2781052a0 2024-03-28T18:04:44.836 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:44.834+0000 7fa27cc00640 1 -- 172.21.15.22:0/3522287826 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa2781a1960 con 0x7fa2781052a0 2024-03-28T18:04:44.836 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:44.834+0000 7fa27cc00640 1 -- 172.21.15.22:0/3522287826 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa2781a1eb0 con 0x7fa2781052a0 2024-03-28T18:04:44.836 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:44.835+0000 7fa27cc00640 1 -- 172.21.15.22:0/3522287826 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa27810a9b0 con 0x7fa2781052a0 2024-03-28T18:04:44.836 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:44.837+0000 7fa274c00640 1 -- 172.21.15.22:0/3522287826 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 16) v1 ==== 137290+0+0 (secure 0 0 0) 0x7fa2700051d0 con 0x7fa2781052a0 2024-03-28T18:04:44.838 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:44.838+0000 7fa274c00640 1 --2- 172.21.15.22:0/3522287826 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7fa25c0a2800 0x7fa25c0a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:04:44.838 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:44.838+0000 7fa274c00640 1 -- 172.21.15.22:0/3522287826 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 7127+0+0 (secure 0 0 0) 0x7fa2700ce920 con 0x7fa2781052a0 2024-03-28T18:04:44.838 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:44.838+0000 7fa277400640 1 --2- 172.21.15.22:0/3522287826 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7fa25c0a2800 0x7fa25c0a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:04:44.839 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:44.839+0000 7fa277400640 1 --2- 172.21.15.22:0/3522287826 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7fa25c0a2800 0x7fa25c0a4cc0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fa26802c170 tx=0x7fa268005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:04:44.842 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:44.842+0000 7fa274c00640 1 -- 172.21.15.22:0/3522287826 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fa270016030 con 0x7fa2781052a0 2024-03-28T18:04:45.042 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:45.043+0000 7fa27cc00640 1 -- 172.21.15.22:0/3522287826 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fa2780622a0 con 0x7fa2781052a0 2024-03-28T18:04:45.044 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:45.044+0000 7fa274c00640 1 -- 172.21.15.22:0/3522287826 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v86) v1 ==== 74+0+18420 (secure 0 0 0) 0x7fa27009bdc0 con 0x7fa2781052a0 2024-03-28T18:04:45.044 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-03-28T18:04:45.045 INFO:teuthology.orchestra.run.smithi022.stdout:{"epoch":86,"fsid":"fd220748-ed2b-11ee-95d0-87774f69a715","created":"2024-03-28T17:54:00.440279+0000","modified":"2024-03-28T18:03:44.232788+0000","last_up_change":"2024-03-28T18:03:43.220095+0000","last_in_change":"2024-03-28T18:03:18.526399+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-28T17:58:33.168872+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"5d708e06-f05b-40ba-bbcf-258ddc4f43d7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6802","nonce":2186290846},{"type":"v1","addr":"172.21.15.22:6803","nonce":2186290846}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6804","nonce":2186290846},{"type":"v1","addr":"172.21.15.22:6805","nonce":2186290846}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6808","nonce":2186290846},{"type":"v1","addr":"172.21.15.22:6809","nonce":2186290846}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6806","nonce":2186290846},{"type":"v1","addr":"172.21.15.22:6807","nonce":2186290846}]},"public_addr":"172.21.15.22:6803/2186290846","cluster_addr":"172.21.15.22:6805/2186290846","heartbeat_back_addr":"172.21.15.22:6809/2186290846","heartbeat_front_addr":"172.21.15.22:6807/2186290846","state":["exists","up"]},{"osd":1,"uuid":"68ad09bb-9256-409a-9867-4e14a2cb94aa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6800","nonce":2835549681},{"type":"v1","addr":"172.21.15.79:6801","nonce":2835549681}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6802","nonce":2835549681},{"type":"v1","addr":"172.21.15.79:6803","nonce":2835549681}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6806","nonce":2835549681},{"type":"v1","addr":"172.21.15.79:6807","nonce":2835549681}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6804","nonce":2835549681},{"type":"v1","addr":"172.21.15.79:6805","nonce":2835549681}]},"public_addr":"172.21.15.79:6801/2835549681","cluster_addr":"172.21.15.79:6803/2835549681","heartbeat_back_addr":"172.21.15.79:6807/2835549681","heartbeat_front_addr":"172.21.15.79:6805/2835549681","state":["exists","up"]},{"osd":2,"uuid":"ed11a5a7-004d-41f3-a5fc-eb2d603bd33f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6800","nonce":470588085},{"type":"v1","addr":"172.21.15.134:6801","nonce":470588085}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6802","nonce":470588085},{"type":"v1","addr":"172.21.15.134:6803","nonce":470588085}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6806","nonce":470588085},{"type":"v1","addr":"172.21.15.134:6807","nonce":470588085}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6804","nonce":470588085},{"type":"v1","addr":"172.21.15.134:6805","nonce":470588085}]},"public_addr":"172.21.15.134:6801/470588085","cluster_addr":"172.21.15.134:6803/470588085","heartbeat_back_addr":"172.21.15.134:6807/470588085","heartbeat_front_addr":"172.21.15.134:6805/470588085","state":["exists","up"]},{"osd":3,"uuid":"81a6c547-2a50-44c3-84d7-23ab6831790c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6810","nonce":1702441854},{"type":"v1","addr":"172.21.15.22:6811","nonce":1702441854}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6812","nonce":1702441854},{"type":"v1","addr":"172.21.15.22:6813","nonce":1702441854}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6816","nonce":1702441854},{"type":"v1","addr":"172.21.15.22:6817","nonce":1702441854}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6814","nonce":1702441854},{"type":"v1","addr":"172.21.15.22:6815","nonce":1702441854}]},"public_addr":"172.21.15.22:6811/1702441854","cluster_addr":"172.21.15.22:6813/1702441854","heartbeat_back_addr":"172.21.15.22:6817/1702441854","heartbeat_front_addr":"172.21.15.22:6815/1702441854","state":["exists","up"]},{"osd":4,"uuid":"d23a254e-7182-4471-8bca-7eb684ec618e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6808","nonce":630132437},{"type":"v1","addr":"172.21.15.79:6809","nonce":630132437}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6810","nonce":630132437},{"type":"v1","addr":"172.21.15.79:6811","nonce":630132437}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6814","nonce":630132437},{"type":"v1","addr":"172.21.15.79:6815","nonce":630132437}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6812","nonce":630132437},{"type":"v1","addr":"172.21.15.79:6813","nonce":630132437}]},"public_addr":"172.21.15.79:6809/630132437","cluster_addr":"172.21.15.79:6811/630132437","heartbeat_back_addr":"172.21.15.79:6815/630132437","heartbeat_front_addr":"172.21.15.79:6813/630132437","state":["exists","up"]},{"osd":5,"uuid":"a7b98502-2023-4f0a-bfa0-6840cc495e0c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":40,"up_thru":55,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6808","nonce":2135406915},{"type":"v1","addr":"172.21.15.134:6809","nonce":2135406915}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6810","nonce":2135406915},{"type":"v1","addr":"172.21.15.134:6811","nonce":2135406915}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6814","nonce":2135406915},{"type":"v1","addr":"172.21.15.134:6815","nonce":2135406915}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6812","nonce":2135406915},{"type":"v1","addr":"172.21.15.134:6813","nonce":2135406915}]},"public_addr":"172.21.15.134:6809/2135406915","cluster_addr":"172.21.15.134:6811/2135406915","heartbeat_back_addr":"172.21.15.134:6815/2135406915","heartbeat_front_addr":"172.21.15.134:6813/2135406915","state":["exists","up"]},{"osd":6,"uuid":"634c6562-d683-41c8-8242-fc848c38ce1f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6818","nonce":1400680396},{"type":"v1","addr":"172.21.15.22:6819","nonce":1400680396}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6820","nonce":1400680396},{"type":"v1","addr":"172.21.15.22:6821","nonce":1400680396}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6824","nonce":1400680396},{"type":"v1","addr":"172.21.15.22:6825","nonce":1400680396}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6822","nonce":1400680396},{"type":"v1","addr":"172.21.15.22:6823","nonce":1400680396}]},"public_addr":"172.21.15.22:6819/1400680396","cluster_addr":"172.21.15.22:6821/1400680396","heartbeat_back_addr":"172.21.15.22:6825/1400680396","heartbeat_front_addr":"172.21.15.22:6823/1400680396","state":["exists","up"]},{"osd":7,"uuid":"504eb4a8-6731-45ad-932c-1b38ef157871","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":53,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6816","nonce":1287105314},{"type":"v1","addr":"172.21.15.79:6817","nonce":1287105314}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6818","nonce":1287105314},{"type":"v1","addr":"172.21.15.79:6819","nonce":1287105314}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6822","nonce":1287105314},{"type":"v1","addr":"172.21.15.79:6823","nonce":1287105314}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6820","nonce":1287105314},{"type":"v1","addr":"172.21.15.79:6821","nonce":1287105314}]},"public_addr":"172.21.15.79:6817/1287105314","cluster_addr":"172.21.15.79:6819/1287105314","heartbeat_back_addr":"172.21.15.79:6823/1287105314","heartbeat_front_addr":"172.21.15.79:6821/1287105314","state":["exists","up"]},{"osd":8,"uuid":"20aa66c1-03d6-4cb8-a096-0eae41979c17","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":62,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6816","nonce":2265650388},{"type":"v1","addr":"172.21.15.134:6817","nonce":2265650388}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6818","nonce":2265650388},{"type":"v1","addr":"172.21.15.134:6819","nonce":2265650388}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6822","nonce":2265650388},{"type":"v1","addr":"172.21.15.134:6823","nonce":2265650388}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6820","nonce":2265650388},{"type":"v1","addr":"172.21.15.134:6821","nonce":2265650388}]},"public_addr":"172.21.15.134:6817/2265650388","cluster_addr":"172.21.15.134:6819/2265650388","heartbeat_back_addr":"172.21.15.134:6823/2265650388","heartbeat_front_addr":"172.21.15.134:6821/2265650388","state":["exists","up"]},{"osd":9,"uuid":"e1ae4020-7c2c-4334-a753-ee6b12d98584","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":70,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6826","nonce":3762222259},{"type":"v1","addr":"172.21.15.22:6827","nonce":3762222259}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6828","nonce":3762222259},{"type":"v1","addr":"172.21.15.22:6829","nonce":3762222259}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6832","nonce":3762222259},{"type":"v1","addr":"172.21.15.22:6833","nonce":3762222259}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6830","nonce":3762222259},{"type":"v1","addr":"172.21.15.22:6831","nonce":3762222259}]},"public_addr":"172.21.15.22:6827/3762222259","cluster_addr":"172.21.15.22:6829/3762222259","heartbeat_back_addr":"172.21.15.22:6833/3762222259","heartbeat_front_addr":"172.21.15.22:6831/3762222259","state":["exists","up"]},{"osd":10,"uuid":"f5198d52-df48-4679-a860-4dcb440b7a9f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":77,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6824","nonce":1746167904},{"type":"v1","addr":"172.21.15.79:6825","nonce":1746167904}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6826","nonce":1746167904},{"type":"v1","addr":"172.21.15.79:6827","nonce":1746167904}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6830","nonce":1746167904},{"type":"v1","addr":"172.21.15.79:6831","nonce":1746167904}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6828","nonce":1746167904},{"type":"v1","addr":"172.21.15.79:6829","nonce":1746167904}]},"public_addr":"172.21.15.79:6825/1746167904","cluster_addr":"172.21.15.79:6827/1746167904","heartbeat_back_addr":"172.21.15.79:6831/1746167904","heartbeat_front_addr":"172.21.15.79:6829/1746167904","state":["exists","up"]},{"osd":11,"uuid":"1427bca7-3414-4468-9246-e999156bc2ae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":85,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6824","nonce":4271716428},{"type":"v1","addr":"172.21.15.134:6825","nonce":4271716428}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6826","nonce":4271716428},{"type":"v1","addr":"172.21.15.134:6827","nonce":4271716428}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6830","nonce":4271716428},{"type":"v1","addr":"172.21.15.134:6831","nonce":4271716428}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6828","nonce":4271716428},{"type":"v1","addr":"172.21.15.134:6829","nonce":4271716428}]},"public_addr":"172.21.15.134:6825/4271716428","cluster_addr":"172.21.15.134:6827/4271716428","heartbeat_back_addr":"172.21.15.134:6831/4271716428","heartbeat_front_addr":"172.21.15.134:6829/4271716428","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T17:57:21.225125+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T17:57:55.197254+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T17:58:27.825136+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T17:59:04.319893+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T17:59:38.407752+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T18:00:12.118805+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T18:00:47.635214+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T18:01:21.231055+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T18:01:55.817861+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T18:02:31.680160+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T18:03:06.783398+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T18:03:41.307347+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.22:0/3521924283":"2024-03-29T17:55:10.425504+0000","172.21.15.22:6801/730635939":"2024-03-29T17:55:10.425504+0000","172.21.15.22:0/2471229459":"2024-03-29T17:54:34.004559+0000","172.21.15.22:6800/3166828081":"2024-03-29T17:54:34.004559+0000","172.21.15.22:6800/730635939":"2024-03-29T17:55:10.425504+0000","172.21.15.22:0/1229047849":"2024-03-29T17:54:34.004559+0000","172.21.15.22:6801/3166828081":"2024-03-29T17:54:34.004559+0000","172.21.15.22:0/2717818856":"2024-03-29T17:55:10.425504+0000","172.21.15.22:0/2157570333":"2024-03-29T17:54:34.004559+0000","172.21.15.22:0/1006589530":"2024-03-29T17:55:10.425504+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-28T18:04:45.047 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:45.047+0000 7fa27cc00640 1 -- 172.21.15.22:0/3522287826 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7fa25c0a2800 msgr2=0x7fa25c0a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:04:45.047 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:45.047+0000 7fa27cc00640 1 --2- 172.21.15.22:0/3522287826 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7fa25c0a2800 0x7fa25c0a4cc0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fa26802c170 tx=0x7fa268005fd0 comp rx=0 tx=0).stop 2024-03-28T18:04:45.047 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:45.047+0000 7fa27cc00640 1 -- 172.21.15.22:0/3522287826 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa2781052a0 msgr2=0x7fa2781a1000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:04:45.047 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:45.047+0000 7fa27cc00640 1 --2- 172.21.15.22:0/3522287826 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa2781052a0 0x7fa2781a1000 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fa27000eaa0 tx=0x7fa27000ef60 comp rx=0 tx=0).stop 2024-03-28T18:04:45.047 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:45.047+0000 7fa27cc00640 1 -- 172.21.15.22:0/3522287826 shutdown_connections 2024-03-28T18:04:45.047 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:45.047+0000 7fa27cc00640 1 --2- 172.21.15.22:0/3522287826 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7fa25c0a2800 0x7fa25c0a4cc0 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:04:45.048 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:45.047+0000 7fa27cc00640 1 --2- 172.21.15.22:0/3522287826 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa2781052a0 0x7fa2781a1000 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:04:45.048 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:45.047+0000 7fa27cc00640 1 --2- 172.21.15.22:0/3522287826 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa2781048e0 0x7fa278199f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:04:45.048 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:45.047+0000 7fa27cc00640 1 --2- 172.21.15.22:0/3522287826 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa2781036f0 0x7fa278199a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:04:45.048 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:45.047+0000 7fa27cc00640 1 -- 172.21.15.22:0/3522287826 >> 172.21.15.22:0/3522287826 conn(0x7fa2780fec80 msgr2=0x7fa278107b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:04:45.048 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:45.047+0000 7fa27cc00640 1 -- 172.21.15.22:0/3522287826 shutdown_connections 2024-03-28T18:04:45.048 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:45.047+0000 7fa27cc00640 1 -- 172.21.15.22:0/3522287826 wait complete. 2024-03-28T18:04:45.064 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-28T17:58:33.168872+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-28T18:04:45.065 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-28T18:04:45.275 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:45.275+0000 7fa39d800640 1 -- 172.21.15.22:0/1645123447 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa398103280 msgr2=0x7fa398109b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:04:45.275 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:45.275+0000 7fa39d800640 1 --2- 172.21.15.22:0/1645123447 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa398103280 0x7fa398109b10 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fa390009ca0 tx=0x7fa39002c0f0 comp rx=0 tx=0).stop 2024-03-28T18:04:45.275 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:45.275+0000 7fa39d800640 1 -- 172.21.15.22:0/1645123447 shutdown_connections 2024-03-28T18:04:45.275 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:45.275+0000 7fa39d800640 1 --2- 172.21.15.22:0/1645123447 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa398103280 0x7fa398109b10 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:04:45.275 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:45.275+0000 7fa39d800640 1 --2- 172.21.15.22:0/1645123447 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa3981028c0 0x7fa398102d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:04:45.275 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:45.275+0000 7fa39d800640 1 --2- 172.21.15.22:0/1645123447 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa3981016d0 0x7fa398101af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:04:45.275 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:45.275+0000 7fa39d800640 1 -- 172.21.15.22:0/1645123447 >> 172.21.15.22:0/1645123447 conn(0x7fa3980fcc40 msgr2=0x7fa3980ff080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:04:45.276 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:45.275+0000 7fa39d800640 1 -- 172.21.15.22:0/1645123447 shutdown_connections 2024-03-28T18:04:45.276 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:45.275+0000 7fa39d800640 1 -- 172.21.15.22:0/1645123447 wait complete. 2024-03-28T18:04:45.277 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:45.276+0000 7fa39d800640 1 Processor -- start 2024-03-28T18:04:45.278 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:45.276+0000 7fa39d800640 1 -- start start 2024-03-28T18:04:45.278 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:45.277+0000 7fa39d800640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa3981016d0 0x7fa398197990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:04:45.278 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:45.277+0000 7fa39d800640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa3981028c0 0x7fa398197ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:04:45.278 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:45.277+0000 7fa39d800640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa398103280 0x7fa39819ef40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:04:45.278 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:45.277+0000 7fa39d800640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa39810d320 con 0x7fa3981016d0 2024-03-28T18:04:45.278 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:45.277+0000 7fa39d800640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa39810d1a0 con 0x7fa398103280 2024-03-28T18:04:45.278 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:45.277+0000 7fa39d800640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa39810d4a0 con 0x7fa3981028c0 2024-03-28T18:04:45.278 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:45.277+0000 7fa396a00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa3981016d0 0x7fa398197990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:04:45.278 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:45.277+0000 7fa396a00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa3981016d0 0x7fa398197990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:34396/0 (socket says 172.21.15.22:34396) 2024-03-28T18:04:45.278 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:45.277+0000 7fa396a00640 1 -- 172.21.15.22:0/2135974677 learned_addr learned my addr 172.21.15.22:0/2135974677 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-28T18:04:45.278 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:45.277+0000 7fa396000640 1 --2- 172.21.15.22:0/2135974677 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa3981028c0 0x7fa398197ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:04:45.278 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:45.277+0000 7fa397400640 1 --2- 172.21.15.22:0/2135974677 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa398103280 0x7fa39819ef40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:04:45.278 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:45.278+0000 7fa396a00640 1 -- 172.21.15.22:0/2135974677 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa3981028c0 msgr2=0x7fa398197ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:04:45.278 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:45.278+0000 7fa396a00640 1 --2- 172.21.15.22:0/2135974677 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa3981028c0 0x7fa398197ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:04:45.279 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:45.278+0000 7fa396a00640 1 -- 172.21.15.22:0/2135974677 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa398103280 msgr2=0x7fa39819ef40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:04:45.279 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:45.278+0000 7fa396a00640 1 --2- 172.21.15.22:0/2135974677 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa398103280 0x7fa39819ef40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:04:45.279 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:45.278+0000 7fa396a00640 1 -- 172.21.15.22:0/2135974677 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa39819f5b0 con 0x7fa3981016d0 2024-03-28T18:04:45.279 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:45.278+0000 7fa397400640 1 --2- 172.21.15.22:0/2135974677 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa398103280 0x7fa39819ef40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-28T18:04:45.279 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:45.278+0000 7fa396a00640 1 --2- 172.21.15.22:0/2135974677 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa3981016d0 0x7fa398197990 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fa38800d770 tx=0x7fa38800dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:04:45.279 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:45.278+0000 7fa38f400640 1 -- 172.21.15.22:0/2135974677 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa388017070 con 0x7fa3981016d0 2024-03-28T18:04:45.280 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:45.278+0000 7fa38f400640 1 -- 172.21.15.22:0/2135974677 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7fa388009c30 con 0x7fa3981016d0 2024-03-28T18:04:45.280 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:45.278+0000 7fa38f400640 1 -- 172.21.15.22:0/2135974677 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa388003e10 con 0x7fa3981016d0 2024-03-28T18:04:45.280 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:45.278+0000 7fa39d800640 1 -- 172.21.15.22:0/2135974677 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa39819f8a0 con 0x7fa3981016d0 2024-03-28T18:04:45.280 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:45.279+0000 7fa39d800640 1 -- 172.21.15.22:0/2135974677 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa39819fe40 con 0x7fa3981016d0 2024-03-28T18:04:45.280 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:45.280+0000 7fa39d800640 1 -- 172.21.15.22:0/2135974677 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa398108ac0 con 0x7fa3981016d0 2024-03-28T18:04:45.281 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:45.282+0000 7fa38f400640 1 -- 172.21.15.22:0/2135974677 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 16) v1 ==== 137290+0+0 (secure 0 0 0) 0x7fa388004020 con 0x7fa3981016d0 2024-03-28T18:04:45.283 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:45.283+0000 7fa38f400640 1 --2- 172.21.15.22:0/2135974677 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7fa37c0a2880 0x7fa37c0a4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:04:45.283 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:45.283+0000 7fa38f400640 1 -- 172.21.15.22:0/2135974677 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 7127+0+0 (secure 0 0 0) 0x7fa3880ce380 con 0x7fa3981016d0 2024-03-28T18:04:45.283 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:45.284+0000 7fa396000640 1 --2- 172.21.15.22:0/2135974677 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7fa37c0a2880 0x7fa37c0a4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:04:45.284 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:45.284+0000 7fa396000640 1 --2- 172.21.15.22:0/2135974677 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7fa37c0a2880 0x7fa37c0a4d40 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fa398198ea0 tx=0x7fa38400b390 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:04:45.288 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:45.288+0000 7fa38f400640 1 -- 172.21.15.22:0/2135974677 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fa38809b820 con 0x7fa3981016d0 2024-03-28T18:04:45.492 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:45.492+0000 7fa39d800640 1 -- 172.21.15.22:0/2135974677 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fa39802d070 con 0x7fa3981016d0 2024-03-28T18:04:45.493 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:45.493+0000 7fa38f400640 1 -- 172.21.15.22:0/2135974677 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v86) v1 ==== 93+0+10 (secure 0 0 0) 0x7fa39802d070 con 0x7fa3981016d0 2024-03-28T18:04:45.493 INFO:teuthology.orchestra.run.smithi022.stdout:pg_num: 1 2024-03-28T18:04:45.496 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:45.496+0000 7fa39d800640 1 -- 172.21.15.22:0/2135974677 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7fa37c0a2880 msgr2=0x7fa37c0a4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:04:45.496 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:45.496+0000 7fa39d800640 1 --2- 172.21.15.22:0/2135974677 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7fa37c0a2880 0x7fa37c0a4d40 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fa398198ea0 tx=0x7fa38400b390 comp rx=0 tx=0).stop 2024-03-28T18:04:45.496 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:45.496+0000 7fa39d800640 1 -- 172.21.15.22:0/2135974677 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa3981016d0 msgr2=0x7fa398197990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:04:45.496 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:45.496+0000 7fa39d800640 1 --2- 172.21.15.22:0/2135974677 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa3981016d0 0x7fa398197990 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fa38800d770 tx=0x7fa38800dc30 comp rx=0 tx=0).stop 2024-03-28T18:04:45.496 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:45.496+0000 7fa39d800640 1 -- 172.21.15.22:0/2135974677 shutdown_connections 2024-03-28T18:04:45.496 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:45.496+0000 7fa39d800640 1 --2- 172.21.15.22:0/2135974677 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7fa37c0a2880 0x7fa37c0a4d40 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:04:45.496 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:45.496+0000 7fa39d800640 1 --2- 172.21.15.22:0/2135974677 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa398103280 0x7fa39819ef40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:04:45.496 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:45.496+0000 7fa39d800640 1 --2- 172.21.15.22:0/2135974677 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa3981028c0 0x7fa398197ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:04:45.496 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:45.496+0000 7fa39d800640 1 --2- 172.21.15.22:0/2135974677 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa3981016d0 0x7fa398197990 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:04:45.496 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:45.496+0000 7fa39d800640 1 -- 172.21.15.22:0/2135974677 >> 172.21.15.22:0/2135974677 conn(0x7fa3980fcc40 msgr2=0x7fa398105c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:04:45.496 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:45.496+0000 7fa39d800640 1 -- 172.21.15.22:0/2135974677 shutdown_connections 2024-03-28T18:04:45.496 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:45.496+0000 7fa39d800640 1 -- 172.21.15.22:0/2135974677 wait complete. 2024-03-28T18:04:45.516 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-03-28T18:04:45.674 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:04:45 smithi134 ceph-mon[35037]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T18:04:45.674 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:04:45 smithi134 ceph-mon[35037]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T18:04:45.674 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:04:45 smithi134 ceph-mon[35037]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T18:04:45.674 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:04:45 smithi134 ceph-mon[35037]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T18:04:45.674 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:04:45 smithi134 ceph-mon[35037]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T18:04:45.674 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:04:45 smithi134 ceph-mon[35037]: mds.? [v2:172.21.15.79:6838/2441542443,v1:172.21.15.79:6839/2441542443] up:boot 2024-03-28T18:04:45.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:04:45 smithi134 ceph-mon[35037]: fsmap 12 up:standby 2024-03-28T18:04:45.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:04:45 smithi134 ceph-mon[35037]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-03-28T18:04:45.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:04:45 smithi134 ceph-mon[35037]: from='client.? 172.21.15.22:0/2696080688' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-28T18:04:45.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:04:45 smithi134 ceph-mon[35037]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T18:04:45.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:04:45 smithi134 ceph-mon[35037]: from='client.? 172.21.15.22:0/3522287826' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-28T18:04:45.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:04:45 smithi079 ceph-mon[35312]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T18:04:45.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:04:45 smithi079 ceph-mon[35312]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T18:04:45.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:04:45 smithi079 ceph-mon[35312]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T18:04:45.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:04:45 smithi079 ceph-mon[35312]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T18:04:45.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:04:45 smithi079 ceph-mon[35312]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T18:04:45.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:04:45 smithi079 ceph-mon[35312]: mds.? [v2:172.21.15.79:6838/2441542443,v1:172.21.15.79:6839/2441542443] up:boot 2024-03-28T18:04:45.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:04:45 smithi079 ceph-mon[35312]: fsmap 12 up:standby 2024-03-28T18:04:45.683 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:04:45 smithi079 ceph-mon[35312]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-03-28T18:04:45.683 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:04:45 smithi079 ceph-mon[35312]: from='client.? 172.21.15.22:0/2696080688' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-28T18:04:45.683 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:04:45 smithi079 ceph-mon[35312]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T18:04:45.683 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:04:45 smithi079 ceph-mon[35312]: from='client.? 172.21.15.22:0/3522287826' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-28T18:04:45.722 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:45.722+0000 7f0e3aa00640 1 -- 172.21.15.22:0/356696427 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f0e34104a90 msgr2=0x7f0e34106e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:04:45.722 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:45.722+0000 7f0e3aa00640 1 --2- 172.21.15.22:0/356696427 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f0e34104a90 0x7f0e34106e80 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f0e20009900 tx=0x7f0e2002bc30 comp rx=0 tx=0).stop 2024-03-28T18:04:45.722 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:45.722+0000 7f0e3aa00640 1 -- 172.21.15.22:0/356696427 shutdown_connections 2024-03-28T18:04:45.722 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:45.722+0000 7f0e3aa00640 1 --2- 172.21.15.22:0/356696427 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0e341073c0 0x7f0e34109810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:04:45.722 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:45.722+0000 7f0e3aa00640 1 --2- 172.21.15.22:0/356696427 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f0e34104a90 0x7f0e34106e80 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:04:45.722 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:45.722+0000 7f0e3aa00640 1 --2- 172.21.15.22:0/356696427 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0e34102160 0x7f0e34104550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:04:45.722 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:45.722+0000 7f0e3aa00640 1 -- 172.21.15.22:0/356696427 >> 172.21.15.22:0/356696427 conn(0x7f0e340fbb50 msgr2=0x7f0e340fdf90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:04:45.722 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:45.722+0000 7f0e3aa00640 1 -- 172.21.15.22:0/356696427 shutdown_connections 2024-03-28T18:04:45.722 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:45.722+0000 7f0e3aa00640 1 -- 172.21.15.22:0/356696427 wait complete. 2024-03-28T18:04:45.723 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:45.724+0000 7f0e3aa00640 1 Processor -- start 2024-03-28T18:04:45.724 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:45.724+0000 7f0e3aa00640 1 -- start start 2024-03-28T18:04:45.724 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:45.724+0000 7f0e3aa00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0e34102160 0x7f0e34199c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:04:45.724 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:45.724+0000 7f0e3aa00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0e34104a90 0x7f0e3419a180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:04:45.724 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:45.724+0000 7f0e3aa00640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f0e341073c0 0x7f0e341a11b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:04:45.724 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:45.724+0000 7f0e3aa00640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e3410d370 con 0x7f0e34102160 2024-03-28T18:04:45.724 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:45.724+0000 7f0e3aa00640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e3410d1f0 con 0x7f0e341073c0 2024-03-28T18:04:45.725 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:45.724+0000 7f0e3aa00640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e3410d4f0 con 0x7f0e34104a90 2024-03-28T18:04:45.725 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:45.725+0000 7f0e33e00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0e34102160 0x7f0e34199c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:04:45.725 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:45.725+0000 7f0e33e00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0e34102160 0x7f0e34199c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:34410/0 (socket says 172.21.15.22:34410) 2024-03-28T18:04:45.725 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:45.725+0000 7f0e33e00640 1 -- 172.21.15.22:0/2347412085 learned_addr learned my addr 172.21.15.22:0/2347412085 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-28T18:04:45.725 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:45.725+0000 7f0e38a00640 1 --2- 172.21.15.22:0/2347412085 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f0e341073c0 0x7f0e341a11b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:04:45.725 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:45.725+0000 7f0e33400640 1 --2- 172.21.15.22:0/2347412085 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0e34104a90 0x7f0e3419a180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:04:45.726 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:45.725+0000 7f0e33e00640 1 -- 172.21.15.22:0/2347412085 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0e34104a90 msgr2=0x7f0e3419a180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:04:45.726 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:45.725+0000 7f0e33e00640 1 --2- 172.21.15.22:0/2347412085 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0e34104a90 0x7f0e3419a180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:04:45.726 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:45.725+0000 7f0e33e00640 1 -- 172.21.15.22:0/2347412085 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f0e341073c0 msgr2=0x7f0e341a11b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:04:45.726 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:45.725+0000 7f0e33e00640 1 --2- 172.21.15.22:0/2347412085 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f0e341073c0 0x7f0e341a11b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:04:45.726 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:45.725+0000 7f0e33e00640 1 -- 172.21.15.22:0/2347412085 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e341a1850 con 0x7f0e34102160 2024-03-28T18:04:45.726 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:45.726+0000 7f0e33e00640 1 --2- 172.21.15.22:0/2347412085 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0e34102160 0x7f0e34199c40 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f0e2400ea60 tx=0x7f0e2400ee60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:04:45.726 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:45.726+0000 7f0e30c00640 1 -- 172.21.15.22:0/2347412085 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e2400cd20 con 0x7f0e34102160 2024-03-28T18:04:45.726 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:45.726+0000 7f0e30c00640 1 -- 172.21.15.22:0/2347412085 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f0e240035f0 con 0x7f0e34102160 2024-03-28T18:04:45.726 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:45.726+0000 7f0e3aa00640 1 -- 172.21.15.22:0/2347412085 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0e341a1b40 con 0x7f0e34102160 2024-03-28T18:04:45.726 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:45.726+0000 7f0e30c00640 1 -- 172.21.15.22:0/2347412085 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e24010690 con 0x7f0e34102160 2024-03-28T18:04:45.727 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:45.726+0000 7f0e3aa00640 1 -- 172.21.15.22:0/2347412085 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0e341a2090 con 0x7f0e34102160 2024-03-28T18:04:45.727 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:45.727+0000 7f0e3aa00640 1 -- 172.21.15.22:0/2347412085 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0e34108eb0 con 0x7f0e34102160 2024-03-28T18:04:45.729 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:45.730+0000 7f0e30c00640 1 -- 172.21.15.22:0/2347412085 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 16) v1 ==== 137290+0+0 (secure 0 0 0) 0x7f0e240037a0 con 0x7f0e34102160 2024-03-28T18:04:45.731 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:45.731+0000 7f0e30c00640 1 --2- 172.21.15.22:0/2347412085 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7f0e180a2800 0x7f0e180a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:04:45.731 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:45.731+0000 7f0e30c00640 1 -- 172.21.15.22:0/2347412085 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f0e240ce9b0 con 0x7f0e34102160 2024-03-28T18:04:45.731 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:45.731+0000 7f0e33400640 1 --2- 172.21.15.22:0/2347412085 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7f0e180a2800 0x7f0e180a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:04:45.732 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:45.732+0000 7f0e33400640 1 --2- 172.21.15.22:0/2347412085 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7f0e180a2800 0x7f0e180a4cc0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f0e2002c170 tx=0x7f0e2002c820 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:04:45.735 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:45.735+0000 7f0e30c00640 1 -- 172.21.15.22:0/2347412085 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f0e2409be50 con 0x7f0e34102160 2024-03-28T18:04:45.781 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:04:45 smithi022 ceph-mon[31985]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T18:04:45.781 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:04:45 smithi022 ceph-mon[31985]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T18:04:45.781 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:04:45 smithi022 ceph-mon[31985]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T18:04:45.781 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:04:45 smithi022 ceph-mon[31985]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T18:04:45.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:04:45 smithi022 ceph-mon[31985]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T18:04:45.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:04:45 smithi022 ceph-mon[31985]: mds.? [v2:172.21.15.79:6838/2441542443,v1:172.21.15.79:6839/2441542443] up:boot 2024-03-28T18:04:45.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:04:45 smithi022 ceph-mon[31985]: fsmap 12 up:standby 2024-03-28T18:04:45.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:04:45 smithi022 ceph-mon[31985]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-03-28T18:04:45.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:04:45 smithi022 ceph-mon[31985]: from='client.? 172.21.15.22:0/2696080688' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-28T18:04:45.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:04:45 smithi022 ceph-mon[31985]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T18:04:45.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:04:45 smithi022 ceph-mon[31985]: from='client.? 172.21.15.22:0/3522287826' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-28T18:04:45.966 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:45.966+0000 7f0e3aa00640 1 -- 172.21.15.22:0/2347412085 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f0e34062340 con 0x7f0e34102160 2024-03-28T18:04:45.966 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:45.967+0000 7f0e30c00640 1 -- 172.21.15.22:0/2347412085 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v86) v1 ==== 84+0+65 (secure 0 0 0) 0x7f0e2409b750 con 0x7f0e34102160 2024-03-28T18:04:45.966 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-03-28T18:04:45.967 INFO:teuthology.orchestra.run.smithi022.stdout:[ 2024-03-28T18:04:45.967 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-03-28T18:04:45.967 INFO:teuthology.orchestra.run.smithi022.stdout: "poolnum": 1, 2024-03-28T18:04:45.967 INFO:teuthology.orchestra.run.smithi022.stdout: "poolname": ".mgr" 2024-03-28T18:04:45.967 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-03-28T18:04:45.967 INFO:teuthology.orchestra.run.smithi022.stdout:] 2024-03-28T18:04:45.969 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:45.969+0000 7f0e3aa00640 1 -- 172.21.15.22:0/2347412085 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7f0e180a2800 msgr2=0x7f0e180a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:04:45.969 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:45.969+0000 7f0e3aa00640 1 --2- 172.21.15.22:0/2347412085 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7f0e180a2800 0x7f0e180a4cc0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f0e2002c170 tx=0x7f0e2002c820 comp rx=0 tx=0).stop 2024-03-28T18:04:45.969 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:45.969+0000 7f0e3aa00640 1 -- 172.21.15.22:0/2347412085 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0e34102160 msgr2=0x7f0e34199c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:04:45.969 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:45.969+0000 7f0e3aa00640 1 --2- 172.21.15.22:0/2347412085 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0e34102160 0x7f0e34199c40 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f0e2400ea60 tx=0x7f0e2400ee60 comp rx=0 tx=0).stop 2024-03-28T18:04:45.970 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:45.969+0000 7f0e3aa00640 1 -- 172.21.15.22:0/2347412085 shutdown_connections 2024-03-28T18:04:45.970 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:45.969+0000 7f0e3aa00640 1 --2- 172.21.15.22:0/2347412085 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7f0e180a2800 0x7f0e180a4cc0 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:04:45.970 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:45.969+0000 7f0e3aa00640 1 --2- 172.21.15.22:0/2347412085 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f0e341073c0 0x7f0e341a11b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:04:45.970 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:45.969+0000 7f0e3aa00640 1 --2- 172.21.15.22:0/2347412085 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0e34104a90 0x7f0e3419a180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:04:45.970 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:45.969+0000 7f0e3aa00640 1 --2- 172.21.15.22:0/2347412085 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0e34102160 0x7f0e34199c40 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:04:45.970 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:45.969+0000 7f0e3aa00640 1 -- 172.21.15.22:0/2347412085 >> 172.21.15.22:0/2347412085 conn(0x7f0e340fbb50 msgr2=0x7f0e341018c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:04:45.970 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:45.970+0000 7f0e3aa00640 1 -- 172.21.15.22:0/2347412085 shutdown_connections 2024-03-28T18:04:45.970 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:45.970+0000 7f0e3aa00640 1 -- 172.21.15.22:0/2347412085 wait complete. 2024-03-28T18:04:45.987 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-03-28T18:04:45.987 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-03-28T18:04:46.209 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:46.209+0000 7f1ace600640 1 -- 172.21.15.22:0/1017389003 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1ac8103260 msgr2=0x7f1ac8109af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:04:46.209 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:46.209+0000 7f1ace600640 1 --2- 172.21.15.22:0/1017389003 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1ac8103260 0x7f1ac8109af0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f1ac0009ca0 tx=0x7f1ac002c0f0 comp rx=0 tx=0).stop 2024-03-28T18:04:46.209 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:46.209+0000 7f1ace600640 1 -- 172.21.15.22:0/1017389003 shutdown_connections 2024-03-28T18:04:46.209 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:46.209+0000 7f1ace600640 1 --2- 172.21.15.22:0/1017389003 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1ac8103260 0x7f1ac8109af0 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:04:46.209 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:46.209+0000 7f1ace600640 1 --2- 172.21.15.22:0/1017389003 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f1ac81028a0 0x7f1ac8102d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:04:46.209 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:46.209+0000 7f1ace600640 1 --2- 172.21.15.22:0/1017389003 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1ac81016b0 0x7f1ac8101ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:04:46.210 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:46.209+0000 7f1ace600640 1 -- 172.21.15.22:0/1017389003 >> 172.21.15.22:0/1017389003 conn(0x7f1ac80fcc20 msgr2=0x7f1ac80ff060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:04:46.210 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:46.209+0000 7f1ace600640 1 -- 172.21.15.22:0/1017389003 shutdown_connections 2024-03-28T18:04:46.210 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:46.209+0000 7f1ace600640 1 -- 172.21.15.22:0/1017389003 wait complete. 2024-03-28T18:04:46.211 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:46.210+0000 7f1ace600640 1 Processor -- start 2024-03-28T18:04:46.211 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:46.210+0000 7f1ace600640 1 -- start start 2024-03-28T18:04:46.211 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:46.211+0000 7f1ace600640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1ac81016b0 0x7f1ac8075450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:04:46.211 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:46.211+0000 7f1ace600640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1ac81028a0 0x7f1ac8073a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:04:46.211 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:46.211+0000 7f1ace600640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f1ac8103260 0x7f1ac8073fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:04:46.211 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:46.211+0000 7f1ace600640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ac810d310 con 0x7f1ac81028a0 2024-03-28T18:04:46.211 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:46.211+0000 7f1ace600640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ac810d190 con 0x7f1ac8103260 2024-03-28T18:04:46.211 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:46.211+0000 7f1ace600640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ac810d490 con 0x7f1ac81016b0 2024-03-28T18:04:46.211 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:46.211+0000 7f1ac6a00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1ac81028a0 0x7f1ac8073a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:04:46.211 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:46.211+0000 7f1ac6a00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1ac81028a0 0x7f1ac8073a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:34420/0 (socket says 172.21.15.22:34420) 2024-03-28T18:04:46.212 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:46.211+0000 7f1ac6a00640 1 -- 172.21.15.22:0/4076542414 learned_addr learned my addr 172.21.15.22:0/4076542414 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-28T18:04:46.212 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:46.211+0000 7f1ac7e00640 1 --2- 172.21.15.22:0/4076542414 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f1ac8103260 0x7f1ac8073fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:04:46.212 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:46.211+0000 7f1ac7400640 1 --2- 172.21.15.22:0/4076542414 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1ac81016b0 0x7f1ac8075450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:04:46.213 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:46.212+0000 7f1ac6a00640 1 -- 172.21.15.22:0/4076542414 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1ac81016b0 msgr2=0x7f1ac8075450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:04:46.213 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:46.212+0000 7f1ac6a00640 1 --2- 172.21.15.22:0/4076542414 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1ac81016b0 0x7f1ac8075450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:04:46.213 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:46.212+0000 7f1ac6a00640 1 -- 172.21.15.22:0/4076542414 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f1ac8103260 msgr2=0x7f1ac8073fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:04:46.213 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:46.212+0000 7f1ac6a00640 1 --2- 172.21.15.22:0/4076542414 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f1ac8103260 0x7f1ac8073fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:04:46.213 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:46.212+0000 7f1ac6a00640 1 -- 172.21.15.22:0/4076542414 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ac8074510 con 0x7f1ac81028a0 2024-03-28T18:04:46.213 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:46.212+0000 7f1ac6a00640 1 --2- 172.21.15.22:0/4076542414 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1ac81028a0 0x7f1ac8073a90 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f1ab400ea60 tx=0x7f1ab400ef20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:04:46.213 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:46.212+0000 7f1abfe00640 1 -- 172.21.15.22:0/4076542414 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ab400cde0 con 0x7f1ac81028a0 2024-03-28T18:04:46.213 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:46.213+0000 7f1abfe00640 1 -- 172.21.15.22:0/4076542414 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f1ab40035f0 con 0x7f1ac81028a0 2024-03-28T18:04:46.213 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:46.213+0000 7f1ace600640 1 -- 172.21.15.22:0/4076542414 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1ac8074800 con 0x7f1ac81028a0 2024-03-28T18:04:46.213 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:46.213+0000 7f1abfe00640 1 -- 172.21.15.22:0/4076542414 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ab4010690 con 0x7f1ac81028a0 2024-03-28T18:04:46.213 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:46.213+0000 7f1ace600640 1 -- 172.21.15.22:0/4076542414 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1ac81a6460 con 0x7f1ac81028a0 2024-03-28T18:04:46.214 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:46.214+0000 7f1ace600640 1 -- 172.21.15.22:0/4076542414 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1ac8108a20 con 0x7f1ac81028a0 2024-03-28T18:04:46.216 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:46.217+0000 7f1abfe00640 1 -- 172.21.15.22:0/4076542414 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 16) v1 ==== 137290+0+0 (secure 0 0 0) 0x7f1ab40037a0 con 0x7f1ac81028a0 2024-03-28T18:04:46.218 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:46.218+0000 7f1abfe00640 1 --2- 172.21.15.22:0/4076542414 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7f1aac0a2800 0x7f1aac0a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:04:46.218 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:46.219+0000 7f1abfe00640 1 -- 172.21.15.22:0/4076542414 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f1ab40ce960 con 0x7f1ac81028a0 2024-03-28T18:04:46.218 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:46.219+0000 7f1ac7400640 1 --2- 172.21.15.22:0/4076542414 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7f1aac0a2800 0x7f1aac0a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:04:46.219 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:46.219+0000 7f1ac7400640 1 --2- 172.21.15.22:0/4076542414 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7f1aac0a2800 0x7f1aac0a4cc0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f1ac8102700 tx=0x7f1ab8005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:04:46.222 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:46.223+0000 7f1abfe00640 1 -- 172.21.15.22:0/4076542414 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f1ab409be00 con 0x7f1ac81028a0 2024-03-28T18:04:46.425 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:04:46 smithi134 ceph-mon[35037]: pgmap v363: 1 pgs: 1 active+clean; 577 KiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-28T18:04:46.425 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:04:46 smithi134 ceph-mon[35037]: from='client.? 172.21.15.22:0/2135974677' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-28T18:04:46.425 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:04:46 smithi134 ceph-mon[35037]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T18:04:46.425 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:04:46 smithi134 ceph-mon[35037]: from='client.? 172.21.15.22:0/2347412085' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-03-28T18:04:46.429 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:46.430+0000 7f1ace600640 1 -- 172.21.15.22:0/4076542414 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7f1ac8062340 con 0x7f1ac81028a0 2024-03-28T18:04:46.433 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:04:46 smithi079 ceph-mon[35312]: pgmap v363: 1 pgs: 1 active+clean; 577 KiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-28T18:04:46.433 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:04:46 smithi079 ceph-mon[35312]: from='client.? 172.21.15.22:0/2135974677' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-28T18:04:46.433 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:04:46 smithi079 ceph-mon[35312]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T18:04:46.433 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:04:46 smithi079 ceph-mon[35312]: from='client.? 172.21.15.22:0/2347412085' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-03-28T18:04:46.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:04:46 smithi022 ceph-mon[31985]: pgmap v363: 1 pgs: 1 active+clean; 577 KiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-28T18:04:46.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:04:46 smithi022 ceph-mon[31985]: from='client.? 172.21.15.22:0/2135974677' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-28T18:04:46.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:04:46 smithi022 ceph-mon[31985]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T18:04:46.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:04:46 smithi022 ceph-mon[31985]: from='client.? 172.21.15.22:0/2347412085' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-03-28T18:04:47.423 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:47.423+0000 7f1abfe00640 1 -- 172.21.15.22:0/4076542414 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v87) v1 ==== 138+0+0 (secure 0 0 0) 0x7f1ab409b700 con 0x7f1ac81028a0 2024-03-28T18:04:47.423 INFO:teuthology.orchestra.run.smithi022.stderr:pool 'cephfs_metadata' created 2024-03-28T18:04:47.426 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:47.426+0000 7f1ace600640 1 -- 172.21.15.22:0/4076542414 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7f1aac0a2800 msgr2=0x7f1aac0a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:04:47.426 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:47.426+0000 7f1ace600640 1 --2- 172.21.15.22:0/4076542414 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7f1aac0a2800 0x7f1aac0a4cc0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f1ac8102700 tx=0x7f1ab8005f90 comp rx=0 tx=0).stop 2024-03-28T18:04:47.426 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:47.426+0000 7f1ace600640 1 -- 172.21.15.22:0/4076542414 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1ac81028a0 msgr2=0x7f1ac8073a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:04:47.426 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:47.426+0000 7f1ace600640 1 --2- 172.21.15.22:0/4076542414 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1ac81028a0 0x7f1ac8073a90 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f1ab400ea60 tx=0x7f1ab400ef20 comp rx=0 tx=0).stop 2024-03-28T18:04:47.426 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:47.426+0000 7f1ace600640 1 -- 172.21.15.22:0/4076542414 shutdown_connections 2024-03-28T18:04:47.426 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:47.426+0000 7f1ace600640 1 --2- 172.21.15.22:0/4076542414 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7f1aac0a2800 0x7f1aac0a4cc0 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:04:47.427 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:47.426+0000 7f1ace600640 1 --2- 172.21.15.22:0/4076542414 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f1ac8103260 0x7f1ac8073fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:04:47.427 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:47.426+0000 7f1ace600640 1 --2- 172.21.15.22:0/4076542414 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1ac81028a0 0x7f1ac8073a90 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:04:47.427 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:47.426+0000 7f1ace600640 1 --2- 172.21.15.22:0/4076542414 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1ac81016b0 0x7f1ac8075450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:04:47.427 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:47.426+0000 7f1ace600640 1 -- 172.21.15.22:0/4076542414 >> 172.21.15.22:0/4076542414 conn(0x7f1ac80fcc20 msgr2=0x7f1ac8105bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:04:47.427 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:47.426+0000 7f1ace600640 1 -- 172.21.15.22:0/4076542414 shutdown_connections 2024-03-28T18:04:47.427 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:47.426+0000 7f1ace600640 1 -- 172.21.15.22:0/4076542414 wait complete. 2024-03-28T18:04:47.457 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-03-28T18:04:47.664 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:47.664+0000 7ff078000640 1 -- 172.21.15.22:0/1275349134 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff070104ad0 msgr2=0x7ff070106ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:04:47.664 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:47.664+0000 7ff078000640 1 --2- 172.21.15.22:0/1275349134 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff070104ad0 0x7ff070106ec0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7ff05c009900 tx=0x7ff05c02bc30 comp rx=0 tx=0).stop 2024-03-28T18:04:47.664 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:47.664+0000 7ff078000640 1 -- 172.21.15.22:0/1275349134 shutdown_connections 2024-03-28T18:04:47.665 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:47.664+0000 7ff078000640 1 --2- 172.21.15.22:0/1275349134 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff070107400 0x7ff0701097f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:04:47.665 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:47.664+0000 7ff078000640 1 --2- 172.21.15.22:0/1275349134 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff070104ad0 0x7ff070106ec0 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:04:47.665 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:47.664+0000 7ff078000640 1 --2- 172.21.15.22:0/1275349134 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ff0701021a0 0x7ff070104590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:04:47.665 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:47.664+0000 7ff078000640 1 -- 172.21.15.22:0/1275349134 >> 172.21.15.22:0/1275349134 conn(0x7ff0700fbb30 msgr2=0x7ff0700fdf50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:04:47.665 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:47.665+0000 7ff078000640 1 -- 172.21.15.22:0/1275349134 shutdown_connections 2024-03-28T18:04:47.665 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:47.665+0000 7ff078000640 1 -- 172.21.15.22:0/1275349134 wait complete. 2024-03-28T18:04:47.668 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:47.666+0000 7ff078000640 1 Processor -- start 2024-03-28T18:04:47.668 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:47.666+0000 7ff078000640 1 -- start start 2024-03-28T18:04:47.668 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:47.666+0000 7ff078000640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff0701021a0 0x7ff070199c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:04:47.668 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:47.666+0000 7ff078000640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ff070104ad0 0x7ff07019a160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:04:47.668 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:47.666+0000 7ff078000640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff070107400 0x7ff0701a1190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:04:47.668 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:47.666+0000 7ff078000640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff07010d350 con 0x7ff0701021a0 2024-03-28T18:04:47.668 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:47.666+0000 7ff078000640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff07010d1d0 con 0x7ff070104ad0 2024-03-28T18:04:47.668 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:47.666+0000 7ff078000640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff07010d4d0 con 0x7ff070107400 2024-03-28T18:04:47.668 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:47.666+0000 7ff075600640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff0701021a0 0x7ff070199c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:04:47.668 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:47.666+0000 7ff075600640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff0701021a0 0x7ff070199c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:49600/0 (socket says 172.21.15.22:49600) 2024-03-28T18:04:47.669 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:47.666+0000 7ff075600640 1 -- 172.21.15.22:0/1613132568 learned_addr learned my addr 172.21.15.22:0/1613132568 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-28T18:04:47.669 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:47.667+0000 7ff076000640 1 --2- 172.21.15.22:0/1613132568 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff070107400 0x7ff0701a1190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:04:47.669 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:47.667+0000 7ff074c00640 1 --2- 172.21.15.22:0/1613132568 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ff070104ad0 0x7ff07019a160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:04:47.669 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:47.667+0000 7ff075600640 1 -- 172.21.15.22:0/1613132568 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff070107400 msgr2=0x7ff0701a1190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:04:47.669 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:47.667+0000 7ff075600640 1 --2- 172.21.15.22:0/1613132568 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff070107400 0x7ff0701a1190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:04:47.669 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:47.667+0000 7ff075600640 1 -- 172.21.15.22:0/1613132568 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ff070104ad0 msgr2=0x7ff07019a160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:04:47.669 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:47.667+0000 7ff075600640 1 --2- 172.21.15.22:0/1613132568 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ff070104ad0 0x7ff07019a160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:04:47.669 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:47.667+0000 7ff075600640 1 -- 172.21.15.22:0/1613132568 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff0701a1830 con 0x7ff0701021a0 2024-03-28T18:04:47.669 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:47.667+0000 7ff075600640 1 --2- 172.21.15.22:0/1613132568 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff0701021a0 0x7ff070199c20 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7ff06400e9f0 tx=0x7ff06400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:04:47.669 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:47.668+0000 7ff06e000640 1 -- 172.21.15.22:0/1613132568 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff06400cd80 con 0x7ff0701021a0 2024-03-28T18:04:47.669 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:47.668+0000 7ff06e000640 1 -- 172.21.15.22:0/1613132568 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7ff064010900 con 0x7ff0701021a0 2024-03-28T18:04:47.669 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:47.668+0000 7ff078000640 1 -- 172.21.15.22:0/1613132568 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff0701a1b20 con 0x7ff0701021a0 2024-03-28T18:04:47.669 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:47.668+0000 7ff06e000640 1 -- 172.21.15.22:0/1613132568 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0640189e0 con 0x7ff0701021a0 2024-03-28T18:04:47.669 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:47.668+0000 7ff078000640 1 -- 172.21.15.22:0/1613132568 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff0701a20c0 con 0x7ff0701021a0 2024-03-28T18:04:47.670 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:47.669+0000 7ff078000640 1 -- 172.21.15.22:0/1613132568 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff070108ef0 con 0x7ff0701021a0 2024-03-28T18:04:47.671 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:47.671+0000 7ff06e000640 1 -- 172.21.15.22:0/1613132568 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 16) v1 ==== 137290+0+0 (secure 0 0 0) 0x7ff064010ab0 con 0x7ff0701021a0 2024-03-28T18:04:47.672 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:47.673+0000 7ff06e000640 1 --2- 172.21.15.22:0/1613132568 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7ff0580a2880 0x7ff0580a4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:04:47.673 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:47.673+0000 7ff06e000640 1 -- 172.21.15.22:0/1613132568 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 7497+0+0 (secure 0 0 0) 0x7ff0640cefe0 con 0x7ff0701021a0 2024-03-28T18:04:47.673 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:47.673+0000 7ff074c00640 1 --2- 172.21.15.22:0/1613132568 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7ff0580a2880 0x7ff0580a4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:04:47.673 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:47.674+0000 7ff074c00640 1 --2- 172.21.15.22:0/1613132568 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7ff0580a2880 0x7ff0580a4d40 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7ff05c02c170 tx=0x7ff05c005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:04:47.674 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:04:47 smithi134 ceph-mon[35037]: from='client.? 172.21.15.22:0/4076542414' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-03-28T18:04:47.676 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:47.676+0000 7ff06e000640 1 -- 172.21.15.22:0/1613132568 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7ff06409c3b0 con 0x7ff0701021a0 2024-03-28T18:04:47.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:04:47 smithi079 ceph-mon[35312]: from='client.? 172.21.15.22:0/4076542414' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-03-28T18:04:47.781 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:04:47 smithi022 ceph-mon[31985]: from='client.? 172.21.15.22:0/4076542414' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-03-28T18:04:47.886 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:47.887+0000 7ff078000640 1 -- 172.21.15.22:0/1613132568 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7ff07002d070 con 0x7ff0701021a0 2024-03-28T18:04:48.471 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:48.471+0000 7ff06e000640 1 -- 172.21.15.22:0/1613132568 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v88) v1 ==== 170+0+0 (secure 0 0 0) 0x7ff06409bcb0 con 0x7ff0701021a0 2024-03-28T18:04:48.471 INFO:teuthology.orchestra.run.smithi022.stderr:pool 'cephfs_data' created 2024-03-28T18:04:48.475 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:48.475+0000 7ff078000640 1 -- 172.21.15.22:0/1613132568 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7ff0580a2880 msgr2=0x7ff0580a4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:04:48.475 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:48.475+0000 7ff078000640 1 --2- 172.21.15.22:0/1613132568 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7ff0580a2880 0x7ff0580a4d40 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7ff05c02c170 tx=0x7ff05c005fd0 comp rx=0 tx=0).stop 2024-03-28T18:04:48.475 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:48.475+0000 7ff078000640 1 -- 172.21.15.22:0/1613132568 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff0701021a0 msgr2=0x7ff070199c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:04:48.475 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:48.475+0000 7ff078000640 1 --2- 172.21.15.22:0/1613132568 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff0701021a0 0x7ff070199c20 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7ff06400e9f0 tx=0x7ff06400eeb0 comp rx=0 tx=0).stop 2024-03-28T18:04:48.475 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:48.475+0000 7ff078000640 1 -- 172.21.15.22:0/1613132568 shutdown_connections 2024-03-28T18:04:48.476 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:48.475+0000 7ff078000640 1 --2- 172.21.15.22:0/1613132568 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7ff0580a2880 0x7ff0580a4d40 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:04:48.476 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:48.475+0000 7ff078000640 1 --2- 172.21.15.22:0/1613132568 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff070107400 0x7ff0701a1190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:04:48.476 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:48.475+0000 7ff078000640 1 --2- 172.21.15.22:0/1613132568 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ff070104ad0 0x7ff07019a160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:04:48.476 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:48.475+0000 7ff078000640 1 --2- 172.21.15.22:0/1613132568 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff0701021a0 0x7ff070199c20 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:04:48.476 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:48.475+0000 7ff078000640 1 -- 172.21.15.22:0/1613132568 >> 172.21.15.22:0/1613132568 conn(0x7ff0700fbb30 msgr2=0x7ff0701018c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:04:48.476 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:48.475+0000 7ff078000640 1 -- 172.21.15.22:0/1613132568 shutdown_connections 2024-03-28T18:04:48.476 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:48.475+0000 7ff078000640 1 -- 172.21.15.22:0/1613132568 wait complete. 2024-03-28T18:04:48.521 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-03-28T18:04:48.715 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:48.715+0000 7ff320800640 1 -- 172.21.15.22:0/176724475 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff318101260 msgr2=0x7ff318108b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:04:48.715 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:48.715+0000 7ff320800640 1 --2- 172.21.15.22:0/176724475 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff318101260 0x7ff318108b00 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7ff314009ca0 tx=0x7ff31402c100 comp rx=0 tx=0).stop 2024-03-28T18:04:48.715 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:48.715+0000 7ff320800640 1 -- 172.21.15.22:0/176724475 shutdown_connections 2024-03-28T18:04:48.715 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:48.715+0000 7ff320800640 1 --2- 172.21.15.22:0/176724475 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff318101260 0x7ff318108b00 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:04:48.716 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:48.715+0000 7ff320800640 1 --2- 172.21.15.22:0/176724475 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff3181008a0 0x7ff318100d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:04:48.716 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:48.715+0000 7ff320800640 1 --2- 172.21.15.22:0/176724475 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ff318101b70 0x7ff318101f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:04:48.716 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:48.715+0000 7ff320800640 1 -- 172.21.15.22:0/176724475 >> 172.21.15.22:0/176724475 conn(0x7ff3180fb2d0 msgr2=0x7ff3180fd710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:04:48.716 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:48.715+0000 7ff320800640 1 -- 172.21.15.22:0/176724475 shutdown_connections 2024-03-28T18:04:48.716 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:48.715+0000 7ff320800640 1 -- 172.21.15.22:0/176724475 wait complete. 2024-03-28T18:04:48.719 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:48.716+0000 7ff320800640 1 Processor -- start 2024-03-28T18:04:48.719 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:48.716+0000 7ff320800640 1 -- start start 2024-03-28T18:04:48.719 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:48.717+0000 7ff320800640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ff3181008a0 0x7ff318100360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:04:48.719 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:48.717+0000 7ff320800640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff318101260 0x7ff31819a420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:04:48.719 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:48.717+0000 7ff320800640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff318101b70 0x7ff31819aa80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:04:48.719 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:48.717+0000 7ff320800640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff31810c250 con 0x7ff318101b70 2024-03-28T18:04:48.720 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:48.717+0000 7ff320800640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff31810c0d0 con 0x7ff3181008a0 2024-03-28T18:04:48.720 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:48.717+0000 7ff320800640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff31810c3d0 con 0x7ff318101260 2024-03-28T18:04:48.720 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:48.717+0000 7ff31e800640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff318101b70 0x7ff31819aa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:04:48.720 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:48.717+0000 7ff31e800640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff318101b70 0x7ff31819aa80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:49616/0 (socket says 172.21.15.22:49616) 2024-03-28T18:04:48.720 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:48.717+0000 7ff31e800640 1 -- 172.21.15.22:0/2826238541 learned_addr learned my addr 172.21.15.22:0/2826238541 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-28T18:04:48.720 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:48.717+0000 7ff31de00640 1 --2- 172.21.15.22:0/2826238541 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ff3181008a0 0x7ff318100360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:04:48.720 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:48.717+0000 7ff31d400640 1 --2- 172.21.15.22:0/2826238541 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff318101260 0x7ff31819a420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:04:48.720 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:48.718+0000 7ff31e800640 1 -- 172.21.15.22:0/2826238541 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff318101260 msgr2=0x7ff31819a420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:04:48.720 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:48.718+0000 7ff31e800640 1 --2- 172.21.15.22:0/2826238541 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff318101260 0x7ff31819a420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:04:48.720 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:48.718+0000 7ff31e800640 1 -- 172.21.15.22:0/2826238541 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ff3181008a0 msgr2=0x7ff318100360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:04:48.721 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:48.718+0000 7ff31e800640 1 --2- 172.21.15.22:0/2826238541 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ff3181008a0 0x7ff318100360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:04:48.721 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:48.718+0000 7ff31e800640 1 -- 172.21.15.22:0/2826238541 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff31819b340 con 0x7ff318101b70 2024-03-28T18:04:48.721 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:48.718+0000 7ff31d400640 1 --2- 172.21.15.22:0/2826238541 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff318101260 0x7ff31819a420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-28T18:04:48.721 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:48.718+0000 7ff31de00640 1 --2- 172.21.15.22:0/2826238541 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ff3181008a0 0x7ff318100360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-28T18:04:48.721 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:48.718+0000 7ff31e800640 1 --2- 172.21.15.22:0/2826238541 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff318101b70 0x7ff31819aa80 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7ff314009bf0 tx=0x7ff314034cd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:04:48.721 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:48.718+0000 7ff312a00640 1 -- 172.21.15.22:0/2826238541 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff314039050 con 0x7ff318101b70 2024-03-28T18:04:48.721 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:48.719+0000 7ff312a00640 1 -- 172.21.15.22:0/2826238541 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7ff31403a040 con 0x7ff318101b70 2024-03-28T18:04:48.721 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:48.719+0000 7ff312a00640 1 -- 172.21.15.22:0/2826238541 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3140046a0 con 0x7ff318101b70 2024-03-28T18:04:48.721 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:48.719+0000 7ff320800640 1 -- 172.21.15.22:0/2826238541 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff31819b5d0 con 0x7ff318101b70 2024-03-28T18:04:48.721 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:48.719+0000 7ff320800640 1 -- 172.21.15.22:0/2826238541 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff31819bac0 con 0x7ff318101b70 2024-03-28T18:04:48.722 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:48.720+0000 7ff320800640 1 -- 172.21.15.22:0/2826238541 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff31819c5a0 con 0x7ff318101b70 2024-03-28T18:04:48.722 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:48.722+0000 7ff312a00640 1 -- 172.21.15.22:0/2826238541 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 16) v1 ==== 137290+0+0 (secure 0 0 0) 0x7ff314003970 con 0x7ff318101b70 2024-03-28T18:04:48.724 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:48.724+0000 7ff312a00640 1 --2- 172.21.15.22:0/2826238541 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7ff3000a2800 0x7ff3000a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:04:48.724 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:48.724+0000 7ff312a00640 1 -- 172.21.15.22:0/2826238541 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 7879+0+0 (secure 0 0 0) 0x7ff3140f1d90 con 0x7ff318101b70 2024-03-28T18:04:48.724 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:48.724+0000 7ff31de00640 1 --2- 172.21.15.22:0/2826238541 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7ff3000a2800 0x7ff3000a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:04:48.725 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:48.725+0000 7ff31de00640 1 --2- 172.21.15.22:0/2826238541 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7ff3000a2800 0x7ff3000a4cc0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7ff318069970 tx=0x7ff30c005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:04:48.732 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:48.732+0000 7ff312a00640 1 -- 172.21.15.22:0/2826238541 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7ff3140bef40 con 0x7ff318101b70 2024-03-28T18:04:48.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:04:48 smithi022 ceph-mon[31985]: pgmap v364: 1 pgs: 1 active+clean; 577 KiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-28T18:04:48.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:04:48 smithi022 ceph-mon[31985]: from='client.? 172.21.15.22:0/4076542414' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-03-28T18:04:48.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:04:48 smithi022 ceph-mon[31985]: osdmap e87: 12 total, 12 up, 12 in 2024-03-28T18:04:48.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:04:48 smithi022 ceph-mon[31985]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T18:04:48.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:04:48 smithi022 ceph-mon[31985]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T18:04:48.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:04:48 smithi022 ceph-mon[31985]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T18:04:48.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:04:48 smithi022 ceph-mon[31985]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T18:04:48.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:04:48 smithi022 ceph-mon[31985]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T18:04:48.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:04:48 smithi022 ceph-mon[31985]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T18:04:48.925 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:04:48 smithi134 ceph-mon[35037]: pgmap v364: 1 pgs: 1 active+clean; 577 KiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-28T18:04:48.925 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:04:48 smithi134 ceph-mon[35037]: from='client.? 172.21.15.22:0/4076542414' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-03-28T18:04:48.925 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:04:48 smithi134 ceph-mon[35037]: osdmap e87: 12 total, 12 up, 12 in 2024-03-28T18:04:48.925 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:04:48 smithi134 ceph-mon[35037]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T18:04:48.925 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:04:48 smithi134 ceph-mon[35037]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T18:04:48.925 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:04:48 smithi134 ceph-mon[35037]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T18:04:48.926 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:04:48 smithi134 ceph-mon[35037]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T18:04:48.926 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:04:48 smithi134 ceph-mon[35037]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T18:04:48.926 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:04:48 smithi134 ceph-mon[35037]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T18:04:48.933 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:04:48 smithi079 ceph-mon[35312]: pgmap v364: 1 pgs: 1 active+clean; 577 KiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-28T18:04:48.933 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:04:48 smithi079 ceph-mon[35312]: from='client.? 172.21.15.22:0/4076542414' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-03-28T18:04:48.933 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:04:48 smithi079 ceph-mon[35312]: osdmap e87: 12 total, 12 up, 12 in 2024-03-28T18:04:48.933 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:04:48 smithi079 ceph-mon[35312]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T18:04:48.933 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:04:48 smithi079 ceph-mon[35312]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T18:04:48.933 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:04:48 smithi079 ceph-mon[35312]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T18:04:48.933 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:04:48 smithi079 ceph-mon[35312]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T18:04:48.933 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:04:48 smithi079 ceph-mon[35312]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T18:04:48.933 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:04:48 smithi079 ceph-mon[35312]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T18:04:48.989 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:48.989+0000 7ff320800640 1 -- 172.21.15.22:0/2826238541 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7ff31819c770 con 0x7ff318101b70 2024-03-28T18:04:49.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:04:49 smithi022 ceph-mon[31985]: from='client.? 172.21.15.22:0/1613132568' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-03-28T18:04:49.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:04:49 smithi022 ceph-mon[31985]: from='client.? 172.21.15.22:0/1613132568' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-03-28T18:04:49.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:04:49 smithi022 ceph-mon[31985]: osdmap e88: 12 total, 12 up, 12 in 2024-03-28T18:04:49.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:04:49 smithi022 ceph-mon[31985]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T18:04:49.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:04:49 smithi022 ceph-mon[31985]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T18:04:49.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:04:49 smithi022 ceph-mon[31985]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T18:04:49.783 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:04:49 smithi022 ceph-mon[31985]: from='client.? 172.21.15.22:0/2826238541' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-03-28T18:04:49.783 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:04:49 smithi022 ceph-mon[31985]: daemon mds.b assigned to filesystem cephfs as rank 0 2024-03-28T18:04:49.783 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:04:49 smithi022 ceph-mon[31985]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T18:04:49.851 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:49.851+0000 7ff312a00640 1 -- 172.21.15.22:0/2826238541 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 2 and data pool 3 v14) v1 ==== 172+0+0 (secure 0 0 0) 0x7ff3140098b0 con 0x7ff318101b70 2024-03-28T18:04:49.851 INFO:teuthology.orchestra.run.smithi022.stderr:new fs with metadata pool 2 and data pool 3 2024-03-28T18:04:49.855 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:49.855+0000 7ff320800640 1 -- 172.21.15.22:0/2826238541 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7ff3000a2800 msgr2=0x7ff3000a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:04:49.855 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:49.855+0000 7ff320800640 1 --2- 172.21.15.22:0/2826238541 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7ff3000a2800 0x7ff3000a4cc0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7ff318069970 tx=0x7ff30c005f90 comp rx=0 tx=0).stop 2024-03-28T18:04:49.855 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:49.855+0000 7ff320800640 1 -- 172.21.15.22:0/2826238541 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff318101b70 msgr2=0x7ff31819aa80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:04:49.855 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:49.855+0000 7ff320800640 1 --2- 172.21.15.22:0/2826238541 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff318101b70 0x7ff31819aa80 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7ff314009bf0 tx=0x7ff314034cd0 comp rx=0 tx=0).stop 2024-03-28T18:04:49.855 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:49.855+0000 7ff320800640 1 -- 172.21.15.22:0/2826238541 shutdown_connections 2024-03-28T18:04:49.855 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:49.855+0000 7ff320800640 1 --2- 172.21.15.22:0/2826238541 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7ff3000a2800 0x7ff3000a4cc0 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:04:49.856 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:49.855+0000 7ff320800640 1 --2- 172.21.15.22:0/2826238541 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff318101b70 0x7ff31819aa80 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:04:49.856 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:49.855+0000 7ff320800640 1 --2- 172.21.15.22:0/2826238541 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff318101260 0x7ff31819a420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:04:49.856 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:49.855+0000 7ff320800640 1 --2- 172.21.15.22:0/2826238541 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ff3181008a0 0x7ff318100360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:04:49.856 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:49.855+0000 7ff320800640 1 -- 172.21.15.22:0/2826238541 >> 172.21.15.22:0/2826238541 conn(0x7ff3180fb2d0 msgr2=0x7ff318104bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:04:49.856 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:49.855+0000 7ff320800640 1 -- 172.21.15.22:0/2826238541 shutdown_connections 2024-03-28T18:04:49.856 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:49.855+0000 7ff320800640 1 -- 172.21.15.22:0/2826238541 wait complete. 2024-03-28T18:04:49.884 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-28T18:04:49.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:04:49 smithi134 ceph-mon[35037]: from='client.? 172.21.15.22:0/1613132568' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-03-28T18:04:49.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:04:49 smithi134 ceph-mon[35037]: from='client.? 172.21.15.22:0/1613132568' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-03-28T18:04:49.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:04:49 smithi134 ceph-mon[35037]: osdmap e88: 12 total, 12 up, 12 in 2024-03-28T18:04:49.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:04:49 smithi134 ceph-mon[35037]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T18:04:49.925 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:04:49 smithi134 ceph-mon[35037]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T18:04:49.925 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:04:49 smithi134 ceph-mon[35037]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T18:04:49.925 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:04:49 smithi134 ceph-mon[35037]: from='client.? 172.21.15.22:0/2826238541' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-03-28T18:04:49.925 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:04:49 smithi134 ceph-mon[35037]: daemon mds.b assigned to filesystem cephfs as rank 0 2024-03-28T18:04:49.925 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:04:49 smithi134 ceph-mon[35037]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T18:04:49.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:04:49 smithi079 ceph-mon[35312]: from='client.? 172.21.15.22:0/1613132568' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-03-28T18:04:49.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:04:49 smithi079 ceph-mon[35312]: from='client.? 172.21.15.22:0/1613132568' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-03-28T18:04:49.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:04:49 smithi079 ceph-mon[35312]: osdmap e88: 12 total, 12 up, 12 in 2024-03-28T18:04:49.933 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:04:49 smithi079 ceph-mon[35312]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T18:04:49.933 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:04:49 smithi079 ceph-mon[35312]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T18:04:49.933 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:04:49 smithi079 ceph-mon[35312]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T18:04:49.933 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:04:49 smithi079 ceph-mon[35312]: from='client.? 172.21.15.22:0/2826238541' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-03-28T18:04:49.933 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:04:49 smithi079 ceph-mon[35312]: daemon mds.b assigned to filesystem cephfs as rank 0 2024-03-28T18:04:49.933 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:04:49 smithi079 ceph-mon[35312]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T18:04:50.082 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:50.082+0000 7f8a1a000640 1 -- 172.21.15.22:0/2151749631 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8a141073c0 msgr2=0x7f8a14109810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:04:50.083 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:50.082+0000 7f8a1a000640 1 --2- 172.21.15.22:0/2151749631 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8a141073c0 0x7f8a14109810 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f8a0c009be0 tx=0x7f8a0c02bef0 comp rx=0 tx=0).stop 2024-03-28T18:04:50.083 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:50.082+0000 7f8a1a000640 1 -- 172.21.15.22:0/2151749631 shutdown_connections 2024-03-28T18:04:50.083 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:50.082+0000 7f8a1a000640 1 --2- 172.21.15.22:0/2151749631 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8a141073c0 0x7f8a14109810 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:04:50.083 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:50.082+0000 7f8a1a000640 1 --2- 172.21.15.22:0/2151749631 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f8a14104a90 0x7f8a14106e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:04:50.083 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:50.082+0000 7f8a1a000640 1 --2- 172.21.15.22:0/2151749631 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8a14102160 0x7f8a14104550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:04:50.083 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:50.082+0000 7f8a1a000640 1 -- 172.21.15.22:0/2151749631 >> 172.21.15.22:0/2151749631 conn(0x7f8a140fbb50 msgr2=0x7f8a140fdf90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:04:50.083 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:50.083+0000 7f8a1a000640 1 -- 172.21.15.22:0/2151749631 shutdown_connections 2024-03-28T18:04:50.083 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:50.083+0000 7f8a1a000640 1 -- 172.21.15.22:0/2151749631 wait complete. 2024-03-28T18:04:50.085 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:50.083+0000 7f8a1a000640 1 Processor -- start 2024-03-28T18:04:50.085 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:50.084+0000 7f8a1a000640 1 -- start start 2024-03-28T18:04:50.085 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:50.084+0000 7f8a1a000640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8a14102160 0x7f8a14101a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:04:50.085 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:50.084+0000 7f8a1a000640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f8a14104a90 0x7f8a14100110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:04:50.086 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:50.084+0000 7f8a1a000640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8a141073c0 0x7f8a14100650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:04:50.086 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:50.084+0000 7f8a1a000640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a1410d360 con 0x7f8a141073c0 2024-03-28T18:04:50.086 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:50.084+0000 7f8a1a000640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a1410d1e0 con 0x7f8a14104a90 2024-03-28T18:04:50.086 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:50.084+0000 7f8a1a000640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a1410d4e0 con 0x7f8a14102160 2024-03-28T18:04:50.086 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:50.084+0000 7f8a13400640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8a14102160 0x7f8a14101a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:04:50.086 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:50.084+0000 7f8a13e00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8a141073c0 0x7f8a14100650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:04:50.086 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:50.084+0000 7f8a13400640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8a14102160 0x7f8a14101a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.22:56338/0 (socket says 172.21.15.22:56338) 2024-03-28T18:04:50.086 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:50.084+0000 7f8a13400640 1 -- 172.21.15.22:0/4135112061 learned_addr learned my addr 172.21.15.22:0/4135112061 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-28T18:04:50.086 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:50.084+0000 7f8a12a00640 1 --2- 172.21.15.22:0/4135112061 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f8a14104a90 0x7f8a14100110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:04:50.086 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:50.085+0000 7f8a12a00640 1 -- 172.21.15.22:0/4135112061 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8a14102160 msgr2=0x7f8a14101a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:04:50.086 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:50.085+0000 7f8a12a00640 1 --2- 172.21.15.22:0/4135112061 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8a14102160 0x7f8a14101a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:04:50.086 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:50.085+0000 7f8a12a00640 1 -- 172.21.15.22:0/4135112061 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8a141073c0 msgr2=0x7f8a14100650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:04:50.086 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:50.085+0000 7f8a12a00640 1 --2- 172.21.15.22:0/4135112061 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8a141073c0 0x7f8a14100650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:04:50.086 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:50.085+0000 7f8a12a00640 1 -- 172.21.15.22:0/4135112061 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8a14100f10 con 0x7f8a14104a90 2024-03-28T18:04:50.086 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:50.085+0000 7f8a13400640 1 --2- 172.21.15.22:0/4135112061 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8a14102160 0x7f8a14101a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-28T18:04:50.087 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:50.085+0000 7f8a12a00640 1 --2- 172.21.15.22:0/4135112061 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f8a14104a90 0x7f8a14100110 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f8a0000e9c0 tx=0x7f8a0000ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:04:50.087 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:50.085+0000 7f8a0be00640 1 -- 172.21.15.22:0/4135112061 <== mon.2 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a0000ccc0 con 0x7f8a14104a90 2024-03-28T18:04:50.087 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:50.085+0000 7f8a1a000640 1 -- 172.21.15.22:0/4135112061 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8a141a1b00 con 0x7f8a14104a90 2024-03-28T18:04:50.087 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:50.086+0000 7f8a0be00640 1 -- 172.21.15.22:0/4135112061 <== mon.2 v2:172.21.15.79:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f8a00010900 con 0x7f8a14104a90 2024-03-28T18:04:50.087 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:50.086+0000 7f8a0be00640 1 -- 172.21.15.22:0/4135112061 <== mon.2 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a00018880 con 0x7f8a14104a90 2024-03-28T18:04:50.087 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:50.086+0000 7f8a1a000640 1 -- 172.21.15.22:0/4135112061 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8a141a2070 con 0x7f8a14104a90 2024-03-28T18:04:50.087 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:50.087+0000 7f8a1a000640 1 -- 172.21.15.22:0/4135112061 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8a140fd640 con 0x7f8a14104a90 2024-03-28T18:04:50.088 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:50.089+0000 7f8a0be00640 1 -- 172.21.15.22:0/4135112061 <== mon.2 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 16) v1 ==== 137290+0+0 (secure 0 0 0) 0x7f8a00010ab0 con 0x7f8a14104a90 2024-03-28T18:04:50.090 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:50.090+0000 7f8a0be00640 1 --2- 172.21.15.22:0/4135112061 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7f89f80a2800 0x7f89f80a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:04:50.090 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:50.090+0000 7f8a0be00640 1 -- 172.21.15.22:0/4135112061 <== mon.2 v2:172.21.15.79:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f8a000cec90 con 0x7f8a14104a90 2024-03-28T18:04:50.090 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:50.090+0000 7f8a13400640 1 --2- 172.21.15.22:0/4135112061 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7f89f80a2800 0x7f89f80a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:04:50.090 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:50.091+0000 7f8a13400640 1 --2- 172.21.15.22:0/4135112061 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7f89f80a2800 0x7f89f80a4cc0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f8a04011550 tx=0x7f8a04009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:04:50.093 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:50.093+0000 7f8a0be00640 1 -- 172.21.15.22:0/4135112061 <== mon.2 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f8a0009b790 con 0x7f8a14104a90 2024-03-28T18:04:50.298 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:50.298+0000 7f8a1a000640 1 -- 172.21.15.22:0/4135112061 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f8a1402d270 con 0x7f8a14104a90 2024-03-28T18:04:50.300 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:50.300+0000 7f8a0be00640 1 -- 172.21.15.22:0/4135112061 <== mon.2 v2:172.21.15.79:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v89) v1 ==== 74+0+22196 (secure 0 0 0) 0x7f8a0009f830 con 0x7f8a14104a90 2024-03-28T18:04:50.301 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-03-28T18:04:50.301 INFO:teuthology.orchestra.run.smithi022.stdout:{"epoch":89,"fsid":"fd220748-ed2b-11ee-95d0-87774f69a715","created":"2024-03-28T17:54:00.440279+0000","modified":"2024-03-28T18:04:48.990937+0000","last_up_change":"2024-03-28T18:03:43.220095+0000","last_in_change":"2024-03-28T18:03:18.526399+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-28T17:58:33.168872+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-28T18:04:47.004797+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-28T18:04:48.465502+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"5d708e06-f05b-40ba-bbcf-258ddc4f43d7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6802","nonce":2186290846},{"type":"v1","addr":"172.21.15.22:6803","nonce":2186290846}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6804","nonce":2186290846},{"type":"v1","addr":"172.21.15.22:6805","nonce":2186290846}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6808","nonce":2186290846},{"type":"v1","addr":"172.21.15.22:6809","nonce":2186290846}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6806","nonce":2186290846},{"type":"v1","addr":"172.21.15.22:6807","nonce":2186290846}]},"public_addr":"172.21.15.22:6803/2186290846","cluster_addr":"172.21.15.22:6805/2186290846","heartbeat_back_addr":"172.21.15.22:6809/2186290846","heartbeat_front_addr":"172.21.15.22:6807/2186290846","state":["exists","up"]},{"osd":1,"uuid":"68ad09bb-9256-409a-9867-4e14a2cb94aa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6800","nonce":2835549681},{"type":"v1","addr":"172.21.15.79:6801","nonce":2835549681}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6802","nonce":2835549681},{"type":"v1","addr":"172.21.15.79:6803","nonce":2835549681}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6806","nonce":2835549681},{"type":"v1","addr":"172.21.15.79:6807","nonce":2835549681}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6804","nonce":2835549681},{"type":"v1","addr":"172.21.15.79:6805","nonce":2835549681}]},"public_addr":"172.21.15.79:6801/2835549681","cluster_addr":"172.21.15.79:6803/2835549681","heartbeat_back_addr":"172.21.15.79:6807/2835549681","heartbeat_front_addr":"172.21.15.79:6805/2835549681","state":["exists","up"]},{"osd":2,"uuid":"ed11a5a7-004d-41f3-a5fc-eb2d603bd33f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6800","nonce":470588085},{"type":"v1","addr":"172.21.15.134:6801","nonce":470588085}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6802","nonce":470588085},{"type":"v1","addr":"172.21.15.134:6803","nonce":470588085}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6806","nonce":470588085},{"type":"v1","addr":"172.21.15.134:6807","nonce":470588085}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6804","nonce":470588085},{"type":"v1","addr":"172.21.15.134:6805","nonce":470588085}]},"public_addr":"172.21.15.134:6801/470588085","cluster_addr":"172.21.15.134:6803/470588085","heartbeat_back_addr":"172.21.15.134:6807/470588085","heartbeat_front_addr":"172.21.15.134:6805/470588085","state":["exists","up"]},{"osd":3,"uuid":"81a6c547-2a50-44c3-84d7-23ab6831790c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6810","nonce":1702441854},{"type":"v1","addr":"172.21.15.22:6811","nonce":1702441854}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6812","nonce":1702441854},{"type":"v1","addr":"172.21.15.22:6813","nonce":1702441854}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6816","nonce":1702441854},{"type":"v1","addr":"172.21.15.22:6817","nonce":1702441854}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6814","nonce":1702441854},{"type":"v1","addr":"172.21.15.22:6815","nonce":1702441854}]},"public_addr":"172.21.15.22:6811/1702441854","cluster_addr":"172.21.15.22:6813/1702441854","heartbeat_back_addr":"172.21.15.22:6817/1702441854","heartbeat_front_addr":"172.21.15.22:6815/1702441854","state":["exists","up"]},{"osd":4,"uuid":"d23a254e-7182-4471-8bca-7eb684ec618e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6808","nonce":630132437},{"type":"v1","addr":"172.21.15.79:6809","nonce":630132437}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6810","nonce":630132437},{"type":"v1","addr":"172.21.15.79:6811","nonce":630132437}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6814","nonce":630132437},{"type":"v1","addr":"172.21.15.79:6815","nonce":630132437}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6812","nonce":630132437},{"type":"v1","addr":"172.21.15.79:6813","nonce":630132437}]},"public_addr":"172.21.15.79:6809/630132437","cluster_addr":"172.21.15.79:6811/630132437","heartbeat_back_addr":"172.21.15.79:6815/630132437","heartbeat_front_addr":"172.21.15.79:6813/630132437","state":["exists","up"]},{"osd":5,"uuid":"a7b98502-2023-4f0a-bfa0-6840cc495e0c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":40,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6808","nonce":2135406915},{"type":"v1","addr":"172.21.15.134:6809","nonce":2135406915}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6810","nonce":2135406915},{"type":"v1","addr":"172.21.15.134:6811","nonce":2135406915}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6814","nonce":2135406915},{"type":"v1","addr":"172.21.15.134:6815","nonce":2135406915}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6812","nonce":2135406915},{"type":"v1","addr":"172.21.15.134:6813","nonce":2135406915}]},"public_addr":"172.21.15.134:6809/2135406915","cluster_addr":"172.21.15.134:6811/2135406915","heartbeat_back_addr":"172.21.15.134:6815/2135406915","heartbeat_front_addr":"172.21.15.134:6813/2135406915","state":["exists","up"]},{"osd":6,"uuid":"634c6562-d683-41c8-8242-fc848c38ce1f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6818","nonce":1400680396},{"type":"v1","addr":"172.21.15.22:6819","nonce":1400680396}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6820","nonce":1400680396},{"type":"v1","addr":"172.21.15.22:6821","nonce":1400680396}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6824","nonce":1400680396},{"type":"v1","addr":"172.21.15.22:6825","nonce":1400680396}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6822","nonce":1400680396},{"type":"v1","addr":"172.21.15.22:6823","nonce":1400680396}]},"public_addr":"172.21.15.22:6819/1400680396","cluster_addr":"172.21.15.22:6821/1400680396","heartbeat_back_addr":"172.21.15.22:6825/1400680396","heartbeat_front_addr":"172.21.15.22:6823/1400680396","state":["exists","up"]},{"osd":7,"uuid":"504eb4a8-6731-45ad-932c-1b38ef157871","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":53,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6816","nonce":1287105314},{"type":"v1","addr":"172.21.15.79:6817","nonce":1287105314}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6818","nonce":1287105314},{"type":"v1","addr":"172.21.15.79:6819","nonce":1287105314}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6822","nonce":1287105314},{"type":"v1","addr":"172.21.15.79:6823","nonce":1287105314}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6820","nonce":1287105314},{"type":"v1","addr":"172.21.15.79:6821","nonce":1287105314}]},"public_addr":"172.21.15.79:6817/1287105314","cluster_addr":"172.21.15.79:6819/1287105314","heartbeat_back_addr":"172.21.15.79:6823/1287105314","heartbeat_front_addr":"172.21.15.79:6821/1287105314","state":["exists","up"]},{"osd":8,"uuid":"20aa66c1-03d6-4cb8-a096-0eae41979c17","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":62,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6816","nonce":2265650388},{"type":"v1","addr":"172.21.15.134:6817","nonce":2265650388}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6818","nonce":2265650388},{"type":"v1","addr":"172.21.15.134:6819","nonce":2265650388}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6822","nonce":2265650388},{"type":"v1","addr":"172.21.15.134:6823","nonce":2265650388}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6820","nonce":2265650388},{"type":"v1","addr":"172.21.15.134:6821","nonce":2265650388}]},"public_addr":"172.21.15.134:6817/2265650388","cluster_addr":"172.21.15.134:6819/2265650388","heartbeat_back_addr":"172.21.15.134:6823/2265650388","heartbeat_front_addr":"172.21.15.134:6821/2265650388","state":["exists","up"]},{"osd":9,"uuid":"e1ae4020-7c2c-4334-a753-ee6b12d98584","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":70,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6826","nonce":3762222259},{"type":"v1","addr":"172.21.15.22:6827","nonce":3762222259}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6828","nonce":3762222259},{"type":"v1","addr":"172.21.15.22:6829","nonce":3762222259}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6832","nonce":3762222259},{"type":"v1","addr":"172.21.15.22:6833","nonce":3762222259}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6830","nonce":3762222259},{"type":"v1","addr":"172.21.15.22:6831","nonce":3762222259}]},"public_addr":"172.21.15.22:6827/3762222259","cluster_addr":"172.21.15.22:6829/3762222259","heartbeat_back_addr":"172.21.15.22:6833/3762222259","heartbeat_front_addr":"172.21.15.22:6831/3762222259","state":["exists","up"]},{"osd":10,"uuid":"f5198d52-df48-4679-a860-4dcb440b7a9f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":77,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6824","nonce":1746167904},{"type":"v1","addr":"172.21.15.79:6825","nonce":1746167904}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6826","nonce":1746167904},{"type":"v1","addr":"172.21.15.79:6827","nonce":1746167904}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6830","nonce":1746167904},{"type":"v1","addr":"172.21.15.79:6831","nonce":1746167904}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6828","nonce":1746167904},{"type":"v1","addr":"172.21.15.79:6829","nonce":1746167904}]},"public_addr":"172.21.15.79:6825/1746167904","cluster_addr":"172.21.15.79:6827/1746167904","heartbeat_back_addr":"172.21.15.79:6831/1746167904","heartbeat_front_addr":"172.21.15.79:6829/1746167904","state":["exists","up"]},{"osd":11,"uuid":"1427bca7-3414-4468-9246-e999156bc2ae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":85,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6824","nonce":4271716428},{"type":"v1","addr":"172.21.15.134:6825","nonce":4271716428}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6826","nonce":4271716428},{"type":"v1","addr":"172.21.15.134:6827","nonce":4271716428}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6830","nonce":4271716428},{"type":"v1","addr":"172.21.15.134:6831","nonce":4271716428}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6828","nonce":4271716428},{"type":"v1","addr":"172.21.15.134:6829","nonce":4271716428}]},"public_addr":"172.21.15.134:6825/4271716428","cluster_addr":"172.21.15.134:6827/4271716428","heartbeat_back_addr":"172.21.15.134:6831/4271716428","heartbeat_front_addr":"172.21.15.134:6829/4271716428","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T17:57:21.225125+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T17:57:55.197254+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T17:58:27.825136+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T17:59:04.319893+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T17:59:38.407752+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T18:00:12.118805+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T18:00:47.635214+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T18:01:21.231055+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T18:01:55.817861+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T18:02:31.680160+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T18:03:06.783398+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T18:03:41.307347+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.22:0/3521924283":"2024-03-29T17:55:10.425504+0000","172.21.15.22:6801/730635939":"2024-03-29T17:55:10.425504+0000","172.21.15.22:0/2471229459":"2024-03-29T17:54:34.004559+0000","172.21.15.22:6800/3166828081":"2024-03-29T17:54:34.004559+0000","172.21.15.22:6800/730635939":"2024-03-29T17:55:10.425504+0000","172.21.15.22:0/1229047849":"2024-03-29T17:54:34.004559+0000","172.21.15.22:6801/3166828081":"2024-03-29T17:54:34.004559+0000","172.21.15.22:0/2717818856":"2024-03-29T17:55:10.425504+0000","172.21.15.22:0/2157570333":"2024-03-29T17:54:34.004559+0000","172.21.15.22:0/1006589530":"2024-03-29T17:55:10.425504+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-28T18:04:50.303 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:50.303+0000 7f8a1a000640 1 -- 172.21.15.22:0/4135112061 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7f89f80a2800 msgr2=0x7f89f80a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:04:50.303 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:50.303+0000 7f8a1a000640 1 --2- 172.21.15.22:0/4135112061 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7f89f80a2800 0x7f89f80a4cc0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f8a04011550 tx=0x7f8a04009260 comp rx=0 tx=0).stop 2024-03-28T18:04:50.303 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:50.303+0000 7f8a1a000640 1 -- 172.21.15.22:0/4135112061 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f8a14104a90 msgr2=0x7f8a14100110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:04:50.304 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:50.303+0000 7f8a1a000640 1 --2- 172.21.15.22:0/4135112061 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f8a14104a90 0x7f8a14100110 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f8a0000e9c0 tx=0x7f8a0000ee80 comp rx=0 tx=0).stop 2024-03-28T18:04:50.304 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:50.303+0000 7f8a1a000640 1 -- 172.21.15.22:0/4135112061 shutdown_connections 2024-03-28T18:04:50.304 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:50.303+0000 7f8a1a000640 1 --2- 172.21.15.22:0/4135112061 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7f89f80a2800 0x7f89f80a4cc0 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:04:50.304 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:50.303+0000 7f8a1a000640 1 --2- 172.21.15.22:0/4135112061 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8a141073c0 0x7f8a14100650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:04:50.304 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:50.303+0000 7f8a1a000640 1 --2- 172.21.15.22:0/4135112061 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f8a14104a90 0x7f8a14100110 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:04:50.304 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:50.303+0000 7f8a1a000640 1 --2- 172.21.15.22:0/4135112061 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8a14102160 0x7f8a14101a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:04:50.304 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:50.303+0000 7f8a1a000640 1 -- 172.21.15.22:0/4135112061 >> 172.21.15.22:0/4135112061 conn(0x7f8a140fbb50 msgr2=0x7f8a14102a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:04:50.304 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:50.304+0000 7f8a1a000640 1 -- 172.21.15.22:0/4135112061 shutdown_connections 2024-03-28T18:04:50.304 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:50.304+0000 7f8a1a000640 1 -- 172.21.15.22:0/4135112061 wait complete. 2024-03-28T18:04:50.325 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-28T18:04:50.515 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:50.515+0000 7f17c3200640 1 -- 172.21.15.22:0/936055285 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f17c41048e0 msgr2=0x7f17c4104d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:04:50.515 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:50.515+0000 7f17c3200640 1 --2- 172.21.15.22:0/936055285 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f17c41048e0 0x7f17c4104d60 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f17b0009900 tx=0x7f17b002bc30 comp rx=0 tx=0).stop 2024-03-28T18:04:50.515 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:50.515+0000 7f17c3200640 1 -- 172.21.15.22:0/936055285 shutdown_connections 2024-03-28T18:04:50.515 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:50.515+0000 7f17c3200640 1 --2- 172.21.15.22:0/936055285 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f17c41052a0 0x7f17c410bb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:04:50.515 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:50.515+0000 7f17c3200640 1 --2- 172.21.15.22:0/936055285 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f17c41048e0 0x7f17c4104d60 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:04:50.515 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:50.515+0000 7f17c3200640 1 --2- 172.21.15.22:0/936055285 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f17c41036f0 0x7f17c4103b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:04:50.515 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:50.515+0000 7f17c3200640 1 -- 172.21.15.22:0/936055285 >> 172.21.15.22:0/936055285 conn(0x7f17c40fec80 msgr2=0x7f17c41010a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:04:50.515 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:50.515+0000 7f17c3200640 1 -- 172.21.15.22:0/936055285 shutdown_connections 2024-03-28T18:04:50.515 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:50.515+0000 7f17c3200640 1 -- 172.21.15.22:0/936055285 wait complete. 2024-03-28T18:04:50.517 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:50.516+0000 7f17c3200640 1 Processor -- start 2024-03-28T18:04:50.517 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:50.516+0000 7f17c3200640 1 -- start start 2024-03-28T18:04:50.517 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:50.516+0000 7f17c3200640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f17c41036f0 0x7f17c41999c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:04:50.517 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:50.516+0000 7f17c3200640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f17c41048e0 0x7f17c4199f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:04:50.517 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:50.516+0000 7f17c3200640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f17c41052a0 0x7f17c41a0f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:04:50.518 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:50.516+0000 7f17c3200640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17c410f3a0 con 0x7f17c41048e0 2024-03-28T18:04:50.518 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:50.516+0000 7f17c3200640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17c410f220 con 0x7f17c41036f0 2024-03-28T18:04:50.518 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:50.516+0000 7f17c3200640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17c410f520 con 0x7f17c41052a0 2024-03-28T18:04:50.518 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:50.517+0000 7f17c1400640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f17c41048e0 0x7f17c4199f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:04:50.518 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:50.517+0000 7f17c1400640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f17c41048e0 0x7f17c4199f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:49666/0 (socket says 172.21.15.22:49666) 2024-03-28T18:04:50.518 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:50.517+0000 7f17c1400640 1 -- 172.21.15.22:0/3610256576 learned_addr learned my addr 172.21.15.22:0/3610256576 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-28T18:04:50.518 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:50.517+0000 7f17c1e00640 1 --2- 172.21.15.22:0/3610256576 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f17c41036f0 0x7f17c41999c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:04:50.518 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:50.517+0000 7f17c1400640 1 -- 172.21.15.22:0/3610256576 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f17c41052a0 msgr2=0x7f17c41a0f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:04:50.518 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:50.517+0000 7f17c1400640 1 --2- 172.21.15.22:0/3610256576 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f17c41052a0 0x7f17c41a0f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:04:50.518 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:50.517+0000 7f17c1400640 1 -- 172.21.15.22:0/3610256576 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f17c41036f0 msgr2=0x7f17c41999c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:04:50.518 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:50.517+0000 7f17c1400640 1 --2- 172.21.15.22:0/3610256576 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f17c41036f0 0x7f17c41999c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:04:50.518 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:50.517+0000 7f17c1400640 1 -- 172.21.15.22:0/3610256576 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f17c41a15e0 con 0x7f17c41048e0 2024-03-28T18:04:50.518 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:50.517+0000 7f17c1e00640 1 --2- 172.21.15.22:0/3610256576 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f17c41036f0 0x7f17c41999c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T18:04:50.519 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:50.517+0000 7f17c1400640 1 --2- 172.21.15.22:0/3610256576 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f17c41048e0 0x7f17c4199f00 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f17b0000c00 tx=0x7f17b0004f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:04:50.519 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:50.518+0000 7f17baa00640 1 -- 172.21.15.22:0/3610256576 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17b0043020 con 0x7f17c41048e0 2024-03-28T18:04:50.519 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:50.518+0000 7f17c3200640 1 -- 172.21.15.22:0/3610256576 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f17c41a1870 con 0x7f17c41048e0 2024-03-28T18:04:50.519 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:50.518+0000 7f17baa00640 1 -- 172.21.15.22:0/3610256576 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f17b00054d0 con 0x7f17c41048e0 2024-03-28T18:04:50.519 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:50.518+0000 7f17baa00640 1 -- 172.21.15.22:0/3610256576 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17b0003d40 con 0x7f17c41048e0 2024-03-28T18:04:50.519 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:50.518+0000 7f17c3200640 1 -- 172.21.15.22:0/3610256576 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f17c41a1dc0 con 0x7f17c41048e0 2024-03-28T18:04:50.519 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:50.519+0000 7f17c3200640 1 -- 172.21.15.22:0/3610256576 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f17c410aa10 con 0x7f17c41048e0 2024-03-28T18:04:50.520 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:50.521+0000 7f17baa00640 1 -- 172.21.15.22:0/3610256576 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 16) v1 ==== 137290+0+0 (secure 0 0 0) 0x7f17b0046020 con 0x7f17c41048e0 2024-03-28T18:04:50.522 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:50.522+0000 7f17baa00640 1 --2- 172.21.15.22:0/3610256576 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7f17a80a2800 0x7f17a80a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:04:50.522 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:50.522+0000 7f17baa00640 1 -- 172.21.15.22:0/3610256576 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f17b00f0db0 con 0x7f17c41048e0 2024-03-28T18:04:50.522 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:50.522+0000 7f17c1e00640 1 --2- 172.21.15.22:0/3610256576 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7f17a80a2800 0x7f17a80a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:04:50.522 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:50.523+0000 7f17c1e00640 1 --2- 172.21.15.22:0/3610256576 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7f17a80a2800 0x7f17a80a4cc0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f17c4104740 tx=0x7f17b4005e70 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:04:50.526 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:50.527+0000 7f17baa00640 1 -- 172.21.15.22:0/3610256576 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f17b00f5070 con 0x7f17c41048e0 2024-03-28T18:04:50.610 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:04:50 smithi022 ceph-mon[31985]: pgmap v367: 97 pgs: 5 creating+peering, 91 unknown, 1 active+clean; 577 KiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-28T18:04:50.610 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:04:50 smithi022 ceph-mon[31985]: Health check failed: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-28T18:04:50.610 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:04:50 smithi022 ceph-mon[31985]: osdmap e89: 12 total, 12 up, 12 in 2024-03-28T18:04:50.610 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:04:50 smithi022 ceph-mon[31985]: from='client.? 172.21.15.22:0/2826238541' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-03-28T18:04:50.610 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:04:50 smithi022 ceph-mon[31985]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2024-03-28T18:04:50.610 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:04:50 smithi022 ceph-mon[31985]: daemon mds.b is now active in filesystem cephfs as rank 0 2024-03-28T18:04:50.610 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:04:50 smithi022 ceph-mon[31985]: from='client.? 172.21.15.22:0/4135112061' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-28T18:04:50.737 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:50.737+0000 7f17c3200640 1 -- 172.21.15.22:0/3610256576 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f17c4062340 con 0x7f17c41048e0 2024-03-28T18:04:50.741 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:50.741+0000 7f17baa00640 1 -- 172.21.15.22:0/3610256576 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v90) v1 ==== 74+0+22183 (secure 0 0 0) 0x7f17b004d020 con 0x7f17c41048e0 2024-03-28T18:04:50.742 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-03-28T18:04:50.742 INFO:teuthology.orchestra.run.smithi022.stdout:{"epoch":90,"fsid":"fd220748-ed2b-11ee-95d0-87774f69a715","created":"2024-03-28T17:54:00.440279+0000","modified":"2024-03-28T18:04:50.509330+0000","last_up_change":"2024-03-28T18:03:43.220095+0000","last_in_change":"2024-03-28T18:03:18.526399+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-28T17:58:33.168872+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-28T18:04:47.004797+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-28T18:04:48.465502+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"90","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"5d708e06-f05b-40ba-bbcf-258ddc4f43d7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6802","nonce":2186290846},{"type":"v1","addr":"172.21.15.22:6803","nonce":2186290846}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6804","nonce":2186290846},{"type":"v1","addr":"172.21.15.22:6805","nonce":2186290846}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6808","nonce":2186290846},{"type":"v1","addr":"172.21.15.22:6809","nonce":2186290846}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6806","nonce":2186290846},{"type":"v1","addr":"172.21.15.22:6807","nonce":2186290846}]},"public_addr":"172.21.15.22:6803/2186290846","cluster_addr":"172.21.15.22:6805/2186290846","heartbeat_back_addr":"172.21.15.22:6809/2186290846","heartbeat_front_addr":"172.21.15.22:6807/2186290846","state":["exists","up"]},{"osd":1,"uuid":"68ad09bb-9256-409a-9867-4e14a2cb94aa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6800","nonce":2835549681},{"type":"v1","addr":"172.21.15.79:6801","nonce":2835549681}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6802","nonce":2835549681},{"type":"v1","addr":"172.21.15.79:6803","nonce":2835549681}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6806","nonce":2835549681},{"type":"v1","addr":"172.21.15.79:6807","nonce":2835549681}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6804","nonce":2835549681},{"type":"v1","addr":"172.21.15.79:6805","nonce":2835549681}]},"public_addr":"172.21.15.79:6801/2835549681","cluster_addr":"172.21.15.79:6803/2835549681","heartbeat_back_addr":"172.21.15.79:6807/2835549681","heartbeat_front_addr":"172.21.15.79:6805/2835549681","state":["exists","up"]},{"osd":2,"uuid":"ed11a5a7-004d-41f3-a5fc-eb2d603bd33f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6800","nonce":470588085},{"type":"v1","addr":"172.21.15.134:6801","nonce":470588085}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6802","nonce":470588085},{"type":"v1","addr":"172.21.15.134:6803","nonce":470588085}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6806","nonce":470588085},{"type":"v1","addr":"172.21.15.134:6807","nonce":470588085}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6804","nonce":470588085},{"type":"v1","addr":"172.21.15.134:6805","nonce":470588085}]},"public_addr":"172.21.15.134:6801/470588085","cluster_addr":"172.21.15.134:6803/470588085","heartbeat_back_addr":"172.21.15.134:6807/470588085","heartbeat_front_addr":"172.21.15.134:6805/470588085","state":["exists","up"]},{"osd":3,"uuid":"81a6c547-2a50-44c3-84d7-23ab6831790c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6810","nonce":1702441854},{"type":"v1","addr":"172.21.15.22:6811","nonce":1702441854}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6812","nonce":1702441854},{"type":"v1","addr":"172.21.15.22:6813","nonce":1702441854}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6816","nonce":1702441854},{"type":"v1","addr":"172.21.15.22:6817","nonce":1702441854}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6814","nonce":1702441854},{"type":"v1","addr":"172.21.15.22:6815","nonce":1702441854}]},"public_addr":"172.21.15.22:6811/1702441854","cluster_addr":"172.21.15.22:6813/1702441854","heartbeat_back_addr":"172.21.15.22:6817/1702441854","heartbeat_front_addr":"172.21.15.22:6815/1702441854","state":["exists","up"]},{"osd":4,"uuid":"d23a254e-7182-4471-8bca-7eb684ec618e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6808","nonce":630132437},{"type":"v1","addr":"172.21.15.79:6809","nonce":630132437}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6810","nonce":630132437},{"type":"v1","addr":"172.21.15.79:6811","nonce":630132437}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6814","nonce":630132437},{"type":"v1","addr":"172.21.15.79:6815","nonce":630132437}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6812","nonce":630132437},{"type":"v1","addr":"172.21.15.79:6813","nonce":630132437}]},"public_addr":"172.21.15.79:6809/630132437","cluster_addr":"172.21.15.79:6811/630132437","heartbeat_back_addr":"172.21.15.79:6815/630132437","heartbeat_front_addr":"172.21.15.79:6813/630132437","state":["exists","up"]},{"osd":5,"uuid":"a7b98502-2023-4f0a-bfa0-6840cc495e0c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":40,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6808","nonce":2135406915},{"type":"v1","addr":"172.21.15.134:6809","nonce":2135406915}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6810","nonce":2135406915},{"type":"v1","addr":"172.21.15.134:6811","nonce":2135406915}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6814","nonce":2135406915},{"type":"v1","addr":"172.21.15.134:6815","nonce":2135406915}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6812","nonce":2135406915},{"type":"v1","addr":"172.21.15.134:6813","nonce":2135406915}]},"public_addr":"172.21.15.134:6809/2135406915","cluster_addr":"172.21.15.134:6811/2135406915","heartbeat_back_addr":"172.21.15.134:6815/2135406915","heartbeat_front_addr":"172.21.15.134:6813/2135406915","state":["exists","up"]},{"osd":6,"uuid":"634c6562-d683-41c8-8242-fc848c38ce1f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6818","nonce":1400680396},{"type":"v1","addr":"172.21.15.22:6819","nonce":1400680396}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6820","nonce":1400680396},{"type":"v1","addr":"172.21.15.22:6821","nonce":1400680396}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6824","nonce":1400680396},{"type":"v1","addr":"172.21.15.22:6825","nonce":1400680396}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6822","nonce":1400680396},{"type":"v1","addr":"172.21.15.22:6823","nonce":1400680396}]},"public_addr":"172.21.15.22:6819/1400680396","cluster_addr":"172.21.15.22:6821/1400680396","heartbeat_back_addr":"172.21.15.22:6825/1400680396","heartbeat_front_addr":"172.21.15.22:6823/1400680396","state":["exists","up"]},{"osd":7,"uuid":"504eb4a8-6731-45ad-932c-1b38ef157871","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":53,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6816","nonce":1287105314},{"type":"v1","addr":"172.21.15.79:6817","nonce":1287105314}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6818","nonce":1287105314},{"type":"v1","addr":"172.21.15.79:6819","nonce":1287105314}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6822","nonce":1287105314},{"type":"v1","addr":"172.21.15.79:6823","nonce":1287105314}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6820","nonce":1287105314},{"type":"v1","addr":"172.21.15.79:6821","nonce":1287105314}]},"public_addr":"172.21.15.79:6817/1287105314","cluster_addr":"172.21.15.79:6819/1287105314","heartbeat_back_addr":"172.21.15.79:6823/1287105314","heartbeat_front_addr":"172.21.15.79:6821/1287105314","state":["exists","up"]},{"osd":8,"uuid":"20aa66c1-03d6-4cb8-a096-0eae41979c17","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":62,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6816","nonce":2265650388},{"type":"v1","addr":"172.21.15.134:6817","nonce":2265650388}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6818","nonce":2265650388},{"type":"v1","addr":"172.21.15.134:6819","nonce":2265650388}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6822","nonce":2265650388},{"type":"v1","addr":"172.21.15.134:6823","nonce":2265650388}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6820","nonce":2265650388},{"type":"v1","addr":"172.21.15.134:6821","nonce":2265650388}]},"public_addr":"172.21.15.134:6817/2265650388","cluster_addr":"172.21.15.134:6819/2265650388","heartbeat_back_addr":"172.21.15.134:6823/2265650388","heartbeat_front_addr":"172.21.15.134:6821/2265650388","state":["exists","up"]},{"osd":9,"uuid":"e1ae4020-7c2c-4334-a753-ee6b12d98584","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":70,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6826","nonce":3762222259},{"type":"v1","addr":"172.21.15.22:6827","nonce":3762222259}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6828","nonce":3762222259},{"type":"v1","addr":"172.21.15.22:6829","nonce":3762222259}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6832","nonce":3762222259},{"type":"v1","addr":"172.21.15.22:6833","nonce":3762222259}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6830","nonce":3762222259},{"type":"v1","addr":"172.21.15.22:6831","nonce":3762222259}]},"public_addr":"172.21.15.22:6827/3762222259","cluster_addr":"172.21.15.22:6829/3762222259","heartbeat_back_addr":"172.21.15.22:6833/3762222259","heartbeat_front_addr":"172.21.15.22:6831/3762222259","state":["exists","up"]},{"osd":10,"uuid":"f5198d52-df48-4679-a860-4dcb440b7a9f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":77,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6824","nonce":1746167904},{"type":"v1","addr":"172.21.15.79:6825","nonce":1746167904}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6826","nonce":1746167904},{"type":"v1","addr":"172.21.15.79:6827","nonce":1746167904}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6830","nonce":1746167904},{"type":"v1","addr":"172.21.15.79:6831","nonce":1746167904}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6828","nonce":1746167904},{"type":"v1","addr":"172.21.15.79:6829","nonce":1746167904}]},"public_addr":"172.21.15.79:6825/1746167904","cluster_addr":"172.21.15.79:6827/1746167904","heartbeat_back_addr":"172.21.15.79:6831/1746167904","heartbeat_front_addr":"172.21.15.79:6829/1746167904","state":["exists","up"]},{"osd":11,"uuid":"1427bca7-3414-4468-9246-e999156bc2ae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":85,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6824","nonce":4271716428},{"type":"v1","addr":"172.21.15.134:6825","nonce":4271716428}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6826","nonce":4271716428},{"type":"v1","addr":"172.21.15.134:6827","nonce":4271716428}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6830","nonce":4271716428},{"type":"v1","addr":"172.21.15.134:6831","nonce":4271716428}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6828","nonce":4271716428},{"type":"v1","addr":"172.21.15.134:6829","nonce":4271716428}]},"public_addr":"172.21.15.134:6825/4271716428","cluster_addr":"172.21.15.134:6827/4271716428","heartbeat_back_addr":"172.21.15.134:6831/4271716428","heartbeat_front_addr":"172.21.15.134:6829/4271716428","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T17:57:21.225125+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T17:57:55.197254+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T17:58:27.825136+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T17:59:04.319893+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T17:59:38.407752+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T18:00:12.118805+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T18:00:47.635214+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T18:01:21.231055+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T18:01:55.817861+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T18:02:31.680160+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T18:03:06.783398+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T18:03:41.307347+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.22:0/3521924283":"2024-03-29T17:55:10.425504+0000","172.21.15.22:6801/730635939":"2024-03-29T17:55:10.425504+0000","172.21.15.22:0/2471229459":"2024-03-29T17:54:34.004559+0000","172.21.15.22:6800/3166828081":"2024-03-29T17:54:34.004559+0000","172.21.15.22:6800/730635939":"2024-03-29T17:55:10.425504+0000","172.21.15.22:0/1229047849":"2024-03-29T17:54:34.004559+0000","172.21.15.22:6801/3166828081":"2024-03-29T17:54:34.004559+0000","172.21.15.22:0/2717818856":"2024-03-29T17:55:10.425504+0000","172.21.15.22:0/2157570333":"2024-03-29T17:54:34.004559+0000","172.21.15.22:0/1006589530":"2024-03-29T17:55:10.425504+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-28T18:04:50.744 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:50.744+0000 7f17c3200640 1 -- 172.21.15.22:0/3610256576 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7f17a80a2800 msgr2=0x7f17a80a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:04:50.744 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:50.744+0000 7f17c3200640 1 --2- 172.21.15.22:0/3610256576 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7f17a80a2800 0x7f17a80a4cc0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f17c4104740 tx=0x7f17b4005e70 comp rx=0 tx=0).stop 2024-03-28T18:04:50.744 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:50.744+0000 7f17c3200640 1 -- 172.21.15.22:0/3610256576 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f17c41048e0 msgr2=0x7f17c4199f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:04:50.745 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:50.744+0000 7f17c3200640 1 --2- 172.21.15.22:0/3610256576 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f17c41048e0 0x7f17c4199f00 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f17b0000c00 tx=0x7f17b0004f10 comp rx=0 tx=0).stop 2024-03-28T18:04:50.745 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:50.744+0000 7f17c3200640 1 -- 172.21.15.22:0/3610256576 shutdown_connections 2024-03-28T18:04:50.745 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:50.744+0000 7f17c3200640 1 --2- 172.21.15.22:0/3610256576 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7f17a80a2800 0x7f17a80a4cc0 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:04:50.745 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:50.744+0000 7f17c3200640 1 --2- 172.21.15.22:0/3610256576 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f17c41052a0 0x7f17c41a0f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:04:50.745 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:50.744+0000 7f17c3200640 1 --2- 172.21.15.22:0/3610256576 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f17c41048e0 0x7f17c4199f00 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:04:50.745 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:50.744+0000 7f17c3200640 1 --2- 172.21.15.22:0/3610256576 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f17c41036f0 0x7f17c41999c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:04:50.745 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:50.744+0000 7f17c3200640 1 -- 172.21.15.22:0/3610256576 >> 172.21.15.22:0/3610256576 conn(0x7f17c40fec80 msgr2=0x7f17c4107be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:04:50.745 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:50.745+0000 7f17c3200640 1 -- 172.21.15.22:0/3610256576 shutdown_connections 2024-03-28T18:04:50.745 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:50.745+0000 7f17c3200640 1 -- 172.21.15.22:0/3610256576 wait complete. 2024-03-28T18:04:50.753 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:04:50 smithi079 ceph-mon[35312]: pgmap v367: 97 pgs: 5 creating+peering, 91 unknown, 1 active+clean; 577 KiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-28T18:04:50.754 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:04:50 smithi079 ceph-mon[35312]: Health check failed: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-28T18:04:50.754 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:04:50 smithi079 ceph-mon[35312]: osdmap e89: 12 total, 12 up, 12 in 2024-03-28T18:04:50.754 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:04:50 smithi079 ceph-mon[35312]: from='client.? 172.21.15.22:0/2826238541' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-03-28T18:04:50.754 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:04:50 smithi079 ceph-mon[35312]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2024-03-28T18:04:50.754 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:04:50 smithi079 ceph-mon[35312]: daemon mds.b is now active in filesystem cephfs as rank 0 2024-03-28T18:04:50.754 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:04:50 smithi079 ceph-mon[35312]: from='client.? 172.21.15.22:0/4135112061' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-28T18:04:50.763 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-03-28T18:04:50.925 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:04:50 smithi134 ceph-mon[35037]: pgmap v367: 97 pgs: 5 creating+peering, 91 unknown, 1 active+clean; 577 KiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-28T18:04:50.925 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:04:50 smithi134 ceph-mon[35037]: Health check failed: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-28T18:04:50.925 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:04:50 smithi134 ceph-mon[35037]: osdmap e89: 12 total, 12 up, 12 in 2024-03-28T18:04:50.925 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:04:50 smithi134 ceph-mon[35037]: from='client.? 172.21.15.22:0/2826238541' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-03-28T18:04:50.925 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:04:50 smithi134 ceph-mon[35037]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2024-03-28T18:04:50.925 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:04:50 smithi134 ceph-mon[35037]: daemon mds.b is now active in filesystem cephfs as rank 0 2024-03-28T18:04:50.925 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:04:50 smithi134 ceph-mon[35037]: from='client.? 172.21.15.22:0/4135112061' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-28T18:04:50.971 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:50.971+0000 7f9402200640 1 -- 172.21.15.22:0/904296627 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f93fc076060 msgr2=0x7f93fc076480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:04:50.971 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:50.971+0000 7f9402200640 1 --2- 172.21.15.22:0/904296627 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f93fc076060 0x7f93fc076480 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f93ec009ca0 tx=0x7f93ec02c100 comp rx=0 tx=0).stop 2024-03-28T18:04:50.971 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:50.971+0000 7f9402200640 1 -- 172.21.15.22:0/904296627 shutdown_connections 2024-03-28T18:04:50.972 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:50.971+0000 7f9402200640 1 --2- 172.21.15.22:0/904296627 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f93fc077c40 0x7f93fc0784e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:04:50.972 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:50.971+0000 7f9402200640 1 --2- 172.21.15.22:0/904296627 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f93fc0772a0 0x7f93fc077700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:04:50.972 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:50.971+0000 7f9402200640 1 --2- 172.21.15.22:0/904296627 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f93fc076060 0x7f93fc076480 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:04:50.972 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:50.971+0000 7f9402200640 1 -- 172.21.15.22:0/904296627 >> 172.21.15.22:0/904296627 conn(0x7f93fc0fee50 msgr2=0x7f93fc101290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:04:50.972 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:50.972+0000 7f9402200640 1 -- 172.21.15.22:0/904296627 shutdown_connections 2024-03-28T18:04:50.972 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:50.972+0000 7f9402200640 1 -- 172.21.15.22:0/904296627 wait complete. 2024-03-28T18:04:50.974 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:50.972+0000 7f9402200640 1 Processor -- start 2024-03-28T18:04:50.974 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:50.973+0000 7f9402200640 1 -- start start 2024-03-28T18:04:50.974 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:50.973+0000 7f9402200640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f93fc076060 0x7f93fc19df90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:04:50.974 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:50.973+0000 7f9402200640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f93fc0772a0 0x7f93fc19e4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:04:50.974 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:50.973+0000 7f9402200640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f93fc077c40 0x7f93fc1a5540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:04:50.974 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:50.973+0000 7f9402200640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93fc113920 con 0x7f93fc077c40 2024-03-28T18:04:50.974 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:50.973+0000 7f9402200640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93fc1137a0 con 0x7f93fc076060 2024-03-28T18:04:50.974 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:50.973+0000 7f9402200640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93fc113aa0 con 0x7f93fc0772a0 2024-03-28T18:04:50.974 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:50.973+0000 7f93fbe00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f93fc077c40 0x7f93fc1a5540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:04:50.974 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:50.973+0000 7f93fbe00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f93fc077c40 0x7f93fc1a5540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:49682/0 (socket says 172.21.15.22:49682) 2024-03-28T18:04:50.974 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:50.973+0000 7f93fbe00640 1 -- 172.21.15.22:0/3539404311 learned_addr learned my addr 172.21.15.22:0/3539404311 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-28T18:04:50.974 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:50.974+0000 7f93fb400640 1 --2- 172.21.15.22:0/3539404311 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f93fc076060 0x7f93fc19df90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:04:50.974 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:50.974+0000 7f93faa00640 1 --2- 172.21.15.22:0/3539404311 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f93fc0772a0 0x7f93fc19e4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:04:50.975 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:50.974+0000 7f93fbe00640 1 -- 172.21.15.22:0/3539404311 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f93fc0772a0 msgr2=0x7f93fc19e4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:04:50.975 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:50.974+0000 7f93fbe00640 1 --2- 172.21.15.22:0/3539404311 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f93fc0772a0 0x7f93fc19e4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:04:50.975 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:50.974+0000 7f93fbe00640 1 -- 172.21.15.22:0/3539404311 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f93fc076060 msgr2=0x7f93fc19df90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:04:50.975 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:50.974+0000 7f93fbe00640 1 --2- 172.21.15.22:0/3539404311 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f93fc076060 0x7f93fc19df90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:04:50.975 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:50.974+0000 7f93fbe00640 1 -- 172.21.15.22:0/3539404311 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f93fc1a5bb0 con 0x7f93fc077c40 2024-03-28T18:04:50.975 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:50.974+0000 7f93fbe00640 1 --2- 172.21.15.22:0/3539404311 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f93fc077c40 0x7f93fc1a5540 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f93f400c970 tx=0x7f93f400ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:04:50.976 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:50.974+0000 7f93f3e00640 1 -- 172.21.15.22:0/3539404311 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93f4007bb0 con 0x7f93fc077c40 2024-03-28T18:04:50.976 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:50.974+0000 7f93f3e00640 1 -- 172.21.15.22:0/3539404311 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f93f4007d50 con 0x7f93fc077c40 2024-03-28T18:04:50.976 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:50.974+0000 7f9402200640 1 -- 172.21.15.22:0/3539404311 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f93fc1a5ea0 con 0x7f93fc077c40 2024-03-28T18:04:50.976 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:50.975+0000 7f93f3e00640 1 -- 172.21.15.22:0/3539404311 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93f400f450 con 0x7f93fc077c40 2024-03-28T18:04:50.976 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:50.975+0000 7f9402200640 1 -- 172.21.15.22:0/3539404311 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f93fc1a6310 con 0x7f93fc077c40 2024-03-28T18:04:50.976 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:50.976+0000 7f9402200640 1 -- 172.21.15.22:0/3539404311 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f93fc110410 con 0x7f93fc077c40 2024-03-28T18:04:50.977 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:50.978+0000 7f93f3e00640 1 -- 172.21.15.22:0/3539404311 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 16) v1 ==== 137290+0+0 (secure 0 0 0) 0x7f93f4004f30 con 0x7f93fc077c40 2024-03-28T18:04:50.979 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:50.979+0000 7f93f3e00640 1 --2- 172.21.15.22:0/3539404311 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7f93e00a2800 0x7f93e00a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:04:50.979 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:50.979+0000 7f93f3e00640 1 -- 172.21.15.22:0/3539404311 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f93f40ce6d0 con 0x7f93fc077c40 2024-03-28T18:04:50.979 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:50.979+0000 7f93fb400640 1 --2- 172.21.15.22:0/3539404311 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7f93e00a2800 0x7f93e00a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:04:50.979 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:50.980+0000 7f93fb400640 1 --2- 172.21.15.22:0/3539404311 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7f93e00a2800 0x7f93e00a4cc0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f93ec006010 tx=0x7f93ec0323f0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:04:50.983 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:50.983+0000 7f93f3e00640 1 -- 172.21.15.22:0/3539404311 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f93f4014030 con 0x7f93fc077c40 2024-03-28T18:04:51.245 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:51.245+0000 7f9402200640 1 -- 172.21.15.22:0/3539404311 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f93fc02d2c0 con 0x7f93fc077c40 2024-03-28T18:04:51.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:04:51 smithi022 ceph-mon[31985]: osdmap e90: 12 total, 12 up, 12 in 2024-03-28T18:04:51.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:04:51 smithi022 ceph-mon[31985]: from='client.? 172.21.15.22:0/3610256576' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-28T18:04:51.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:04:51 smithi022 ceph-mon[31985]: mds.? [v2:172.21.15.79:6838/2441542443,v1:172.21.15.79:6839/2441542443] up:active 2024-03-28T18:04:51.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:04:51 smithi022 ceph-mon[31985]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-03-28T18:04:51.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:04:51 smithi022 ceph-mon[31985]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-03-28T18:04:51.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:04:51 smithi022 ceph-mon[31985]: from='client.? 172.21.15.22:0/3539404311' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-03-28T18:04:51.864 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:51.864+0000 7f93f3e00640 1 -- 172.21.15.22:0/3539404311 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v17) v1 ==== 118+0+0 (secure 0 0 0) 0x7f93f409b8c0 con 0x7f93fc077c40 2024-03-28T18:04:51.868 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:51.867+0000 7f9402200640 1 -- 172.21.15.22:0/3539404311 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7f93e00a2800 msgr2=0x7f93e00a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:04:51.868 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:51.867+0000 7f9402200640 1 --2- 172.21.15.22:0/3539404311 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7f93e00a2800 0x7f93e00a4cc0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f93ec006010 tx=0x7f93ec0323f0 comp rx=0 tx=0).stop 2024-03-28T18:04:51.868 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:51.867+0000 7f9402200640 1 -- 172.21.15.22:0/3539404311 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f93fc077c40 msgr2=0x7f93fc1a5540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:04:51.868 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:51.868+0000 7f9402200640 1 --2- 172.21.15.22:0/3539404311 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f93fc077c40 0x7f93fc1a5540 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f93f400c970 tx=0x7f93f400ce30 comp rx=0 tx=0).stop 2024-03-28T18:04:51.868 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:51.868+0000 7f9402200640 1 -- 172.21.15.22:0/3539404311 shutdown_connections 2024-03-28T18:04:51.868 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:51.868+0000 7f9402200640 1 --2- 172.21.15.22:0/3539404311 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7f93e00a2800 0x7f93e00a4cc0 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:04:51.868 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:51.868+0000 7f9402200640 1 --2- 172.21.15.22:0/3539404311 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f93fc077c40 0x7f93fc1a5540 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:04:51.868 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:51.868+0000 7f9402200640 1 --2- 172.21.15.22:0/3539404311 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f93fc0772a0 0x7f93fc19e4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:04:51.868 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:51.868+0000 7f9402200640 1 --2- 172.21.15.22:0/3539404311 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f93fc076060 0x7f93fc19df90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:04:51.868 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:51.868+0000 7f9402200640 1 -- 172.21.15.22:0/3539404311 >> 172.21.15.22:0/3539404311 conn(0x7f93fc0fee50 msgr2=0x7f93fc100560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:04:51.868 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:51.868+0000 7f9402200640 1 -- 172.21.15.22:0/3539404311 shutdown_connections 2024-03-28T18:04:51.868 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:51.868+0000 7f9402200640 1 -- 172.21.15.22:0/3539404311 wait complete. 2024-03-28T18:04:51.887 DEBUG:tasks.cephfs.filesystem:fs_config: {'max_mds': 5, 'session_timeout': 300, 'standby_replay': True, 'subvols': {'create': 2, 'subvol_options': '--size 25000000000'}} 2024-03-28T18:04:51.887 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs max_mds 5 2024-03-28T18:04:51.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:04:51 smithi134 ceph-mon[35037]: osdmap e90: 12 total, 12 up, 12 in 2024-03-28T18:04:51.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:04:51 smithi134 ceph-mon[35037]: from='client.? 172.21.15.22:0/3610256576' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-28T18:04:51.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:04:51 smithi134 ceph-mon[35037]: mds.? [v2:172.21.15.79:6838/2441542443,v1:172.21.15.79:6839/2441542443] up:active 2024-03-28T18:04:51.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:04:51 smithi134 ceph-mon[35037]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-03-28T18:04:51.925 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:04:51 smithi134 ceph-mon[35037]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-03-28T18:04:51.925 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:04:51 smithi134 ceph-mon[35037]: from='client.? 172.21.15.22:0/3539404311' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-03-28T18:04:51.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:04:51 smithi079 ceph-mon[35312]: osdmap e90: 12 total, 12 up, 12 in 2024-03-28T18:04:51.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:04:51 smithi079 ceph-mon[35312]: from='client.? 172.21.15.22:0/3610256576' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-28T18:04:51.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:04:51 smithi079 ceph-mon[35312]: mds.? [v2:172.21.15.79:6838/2441542443,v1:172.21.15.79:6839/2441542443] up:active 2024-03-28T18:04:51.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:04:51 smithi079 ceph-mon[35312]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-03-28T18:04:51.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:04:51 smithi079 ceph-mon[35312]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-03-28T18:04:51.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:04:51 smithi079 ceph-mon[35312]: from='client.? 172.21.15.22:0/3539404311' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-03-28T18:04:52.096 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:52.096+0000 7f1984800640 1 -- 172.21.15.22:0/794362035 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f197c105240 msgr2=0x7f197c1056c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:04:52.096 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:52.096+0000 7f1984800640 1 --2- 172.21.15.22:0/794362035 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f197c105240 0x7f197c1056c0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f1978009ca0 tx=0x7f197802c100 comp rx=0 tx=0).stop 2024-03-28T18:04:52.096 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:52.096+0000 7f1984800640 1 -- 172.21.15.22:0/794362035 shutdown_connections 2024-03-28T18:04:52.096 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:52.096+0000 7f1984800640 1 --2- 172.21.15.22:0/794362035 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f197c105240 0x7f197c1056c0 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:04:52.097 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:52.096+0000 7f1984800640 1 --2- 172.21.15.22:0/794362035 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f197c102890 0x7f197c104d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:04:52.097 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:52.096+0000 7f1984800640 1 --2- 172.21.15.22:0/794362035 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f197c0fff20 0x7f197c102350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:04:52.097 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:52.096+0000 7f1984800640 1 -- 172.21.15.22:0/794362035 >> 172.21.15.22:0/794362035 conn(0x7f197c0fbb40 msgr2=0x7f197c0fdf60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:04:52.097 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:52.096+0000 7f1984800640 1 -- 172.21.15.22:0/794362035 shutdown_connections 2024-03-28T18:04:52.097 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:52.096+0000 7f1984800640 1 -- 172.21.15.22:0/794362035 wait complete. 2024-03-28T18:04:52.099 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:52.097+0000 7f1984800640 1 Processor -- start 2024-03-28T18:04:52.099 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:52.097+0000 7f1984800640 1 -- start start 2024-03-28T18:04:52.100 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:52.098+0000 7f1984800640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f197c0fff20 0x7f197c19dfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:04:52.100 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:52.098+0000 7f1984800640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f197c102890 0x7f197c19e4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:04:52.100 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:52.098+0000 7f1984800640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f197c105240 0x7f197c1a5550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:04:52.100 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:52.098+0000 7f1984800640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f197c113920 con 0x7f197c102890 2024-03-28T18:04:52.100 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:52.098+0000 7f1984800640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f197c1137a0 con 0x7f197c0fff20 2024-03-28T18:04:52.100 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:52.098+0000 7f1984800640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f197c113aa0 con 0x7f197c105240 2024-03-28T18:04:52.100 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:52.098+0000 7f1981400640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f197c102890 0x7f197c19e4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:04:52.100 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:52.098+0000 7f1981400640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f197c102890 0x7f197c19e4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:49710/0 (socket says 172.21.15.22:49710) 2024-03-28T18:04:52.100 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:52.098+0000 7f1981400640 1 -- 172.21.15.22:0/4187516479 learned_addr learned my addr 172.21.15.22:0/4187516479 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-28T18:04:52.100 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:52.098+0000 7f1981e00640 1 --2- 172.21.15.22:0/4187516479 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f197c0fff20 0x7f197c19dfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:04:52.100 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:52.098+0000 7f1982800640 1 --2- 172.21.15.22:0/4187516479 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f197c105240 0x7f197c1a5550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:04:52.100 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:52.099+0000 7f1981400640 1 -- 172.21.15.22:0/4187516479 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f197c105240 msgr2=0x7f197c1a5550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:04:52.100 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:52.099+0000 7f1981400640 1 --2- 172.21.15.22:0/4187516479 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f197c105240 0x7f197c1a5550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:04:52.100 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:52.099+0000 7f1981400640 1 -- 172.21.15.22:0/4187516479 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f197c0fff20 msgr2=0x7f197c19dfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:04:52.100 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:52.099+0000 7f1981400640 1 --2- 172.21.15.22:0/4187516479 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f197c0fff20 0x7f197c19dfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:04:52.101 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:52.099+0000 7f1981400640 1 -- 172.21.15.22:0/4187516479 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f197c1a5bc0 con 0x7f197c102890 2024-03-28T18:04:52.101 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:52.099+0000 7f1981400640 1 --2- 172.21.15.22:0/4187516479 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f197c102890 0x7f197c19e4e0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f196c00d770 tx=0x7f196c00dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:04:52.101 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:52.099+0000 7f1976a00640 1 -- 172.21.15.22:0/4187516479 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f196c017070 con 0x7f197c102890 2024-03-28T18:04:52.101 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:52.099+0000 7f1976a00640 1 -- 172.21.15.22:0/4187516479 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f196c0054f0 con 0x7f197c102890 2024-03-28T18:04:52.101 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:52.099+0000 7f1984800640 1 -- 172.21.15.22:0/4187516479 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f197c1a5eb0 con 0x7f197c102890 2024-03-28T18:04:52.101 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:52.100+0000 7f1976a00640 1 -- 172.21.15.22:0/4187516479 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f196c004d60 con 0x7f197c102890 2024-03-28T18:04:52.101 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:52.100+0000 7f1984800640 1 -- 172.21.15.22:0/4187516479 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f197c1a6370 con 0x7f197c102890 2024-03-28T18:04:52.101 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:52.101+0000 7f1984800640 1 -- 172.21.15.22:0/4187516479 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f197c0fd630 con 0x7f197c102890 2024-03-28T18:04:52.103 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:52.104+0000 7f1976a00640 1 -- 172.21.15.22:0/4187516479 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 16) v1 ==== 137290+0+0 (secure 0 0 0) 0x7f196c004f00 con 0x7f197c102890 2024-03-28T18:04:52.105 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:52.105+0000 7f1976a00640 1 --2- 172.21.15.22:0/4187516479 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7f19640a2880 0x7f19640a4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:04:52.105 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:52.106+0000 7f1976a00640 1 -- 172.21.15.22:0/4187516479 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f196c0ce2b0 con 0x7f197c102890 2024-03-28T18:04:52.105 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:52.106+0000 7f1981e00640 1 --2- 172.21.15.22:0/4187516479 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7f19640a2880 0x7f19640a4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:04:52.106 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:52.106+0000 7f1981e00640 1 --2- 172.21.15.22:0/4187516479 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7f19640a2880 0x7f19640a4d40 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f1970009bd0 tx=0x7f1970005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:04:52.109 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:52.110+0000 7f1976a00640 1 -- 172.21.15.22:0/4187516479 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f196c09b4a0 con 0x7f197c102890 2024-03-28T18:04:52.364 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:52.364+0000 7f1984800640 1 -- 172.21.15.22:0/4187516479 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"} v 0) v1 -- 0x7f197c02d2b0 con 0x7f197c102890 2024-03-28T18:04:52.570 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:52.570+0000 7f1976a00640 1 -- 172.21.15.22:0/4187516479 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]=0 v18) v1 ==== 105+0+0 (secure 0 0 0) 0x7f196c09ada0 con 0x7f197c102890 2024-03-28T18:04:52.573 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:52.574+0000 7f1984800640 1 -- 172.21.15.22:0/4187516479 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7f19640a2880 msgr2=0x7f19640a4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:04:52.573 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:52.574+0000 7f1984800640 1 --2- 172.21.15.22:0/4187516479 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7f19640a2880 0x7f19640a4d40 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f1970009bd0 tx=0x7f1970005f90 comp rx=0 tx=0).stop 2024-03-28T18:04:52.573 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:52.574+0000 7f1984800640 1 -- 172.21.15.22:0/4187516479 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f197c102890 msgr2=0x7f197c19e4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:04:52.574 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:52.574+0000 7f1984800640 1 --2- 172.21.15.22:0/4187516479 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f197c102890 0x7f197c19e4e0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f196c00d770 tx=0x7f196c00dc30 comp rx=0 tx=0).stop 2024-03-28T18:04:52.574 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:52.574+0000 7f1984800640 1 -- 172.21.15.22:0/4187516479 shutdown_connections 2024-03-28T18:04:52.574 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:52.574+0000 7f1984800640 1 --2- 172.21.15.22:0/4187516479 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7f19640a2880 0x7f19640a4d40 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:04:52.574 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:52.574+0000 7f1984800640 1 --2- 172.21.15.22:0/4187516479 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f197c105240 0x7f197c1a5550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:04:52.575 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:52.574+0000 7f1984800640 1 --2- 172.21.15.22:0/4187516479 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f197c102890 0x7f197c19e4e0 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:04:52.575 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:52.574+0000 7f1984800640 1 --2- 172.21.15.22:0/4187516479 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f197c0fff20 0x7f197c19dfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:04:52.575 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:52.574+0000 7f1984800640 1 -- 172.21.15.22:0/4187516479 >> 172.21.15.22:0/4187516479 conn(0x7f197c0fbb40 msgr2=0x7f197c100800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:04:52.575 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:52.574+0000 7f1984800640 1 -- 172.21.15.22:0/4187516479 shutdown_connections 2024-03-28T18:04:52.575 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:52.575+0000 7f1984800640 1 -- 172.21.15.22:0/4187516479 wait complete. 2024-03-28T18:04:52.604 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay true 2024-03-28T18:04:52.781 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:04:52 smithi022 ceph-mon[31985]: pgmap v370: 97 pgs: 22 creating+peering, 60 unknown, 15 active+clean; 578 KiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 767 B/s wr, 1 op/s 2024-03-28T18:04:52.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:04:52 smithi022 ceph-mon[31985]: Health check cleared: POOL_APP_NOT_ENABLED (was: 2 pool(s) do not have an application enabled) 2024-03-28T18:04:52.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:04:52 smithi022 ceph-mon[31985]: Cluster is now healthy 2024-03-28T18:04:52.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:04:52 smithi022 ceph-mon[31985]: from='client.? 172.21.15.22:0/3539404311' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-03-28T18:04:52.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:04:52 smithi022 ceph-mon[31985]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-03-28T18:04:52.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:04:52 smithi022 ceph-mon[31985]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T18:04:52.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:04:52 smithi022 ceph-mon[31985]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T18:04:52.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:04:52 smithi022 ceph-mon[31985]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T18:04:52.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:04:52 smithi022 ceph-mon[31985]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T18:04:52.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:04:52 smithi022 ceph-mon[31985]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T18:04:52.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:04:52 smithi022 ceph-mon[31985]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T18:04:52.783 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:04:52 smithi022 ceph-mon[31985]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T18:04:52.783 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:04:52 smithi022 ceph-mon[31985]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T18:04:52.783 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:04:52 smithi022 ceph-mon[31985]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T18:04:52.783 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:04:52 smithi022 ceph-mon[31985]: from='client.? 172.21.15.22:0/4187516479' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]: dispatch 2024-03-28T18:04:52.812 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:52.812+0000 7f9f66600640 1 -- 172.21.15.22:0/2973474845 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9f60109870 msgr2=0x7f9f60110100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:04:52.812 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:52.812+0000 7f9f66600640 1 --2- 172.21.15.22:0/2973474845 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9f60109870 0x7f9f60110100 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f9f58009ca0 tx=0x7f9f5802c0f0 comp rx=0 tx=0).stop 2024-03-28T18:04:52.812 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:52.812+0000 7f9f66600640 1 -- 172.21.15.22:0/2973474845 shutdown_connections 2024-03-28T18:04:52.812 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:52.812+0000 7f9f66600640 1 --2- 172.21.15.22:0/2973474845 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9f60109870 0x7f9f60110100 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:04:52.812 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:52.812+0000 7f9f66600640 1 --2- 172.21.15.22:0/2973474845 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f9f60108eb0 0x7f9f60109330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:04:52.812 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:52.812+0000 7f9f66600640 1 --2- 172.21.15.22:0/2973474845 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9f60107cc0 0x7f9f601080e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:04:52.812 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:52.812+0000 7f9f66600640 1 -- 172.21.15.22:0/2973474845 >> 172.21.15.22:0/2973474845 conn(0x7f9f60076c60 msgr2=0x7f9f600790a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:04:52.813 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:52.812+0000 7f9f66600640 1 -- 172.21.15.22:0/2973474845 shutdown_connections 2024-03-28T18:04:52.813 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:52.812+0000 7f9f66600640 1 -- 172.21.15.22:0/2973474845 wait complete. 2024-03-28T18:04:52.813 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:52.813+0000 7f9f66600640 1 Processor -- start 2024-03-28T18:04:52.813 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:52.813+0000 7f9f66600640 1 -- start start 2024-03-28T18:04:52.814 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:52.814+0000 7f9f66600640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f9f60107cc0 0x7f9f6019df90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:04:52.814 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:52.814+0000 7f9f66600640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9f60108eb0 0x7f9f6019e4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:04:52.814 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:52.814+0000 7f9f66600640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9f60109870 0x7f9f601a5540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:04:52.814 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:52.814+0000 7f9f66600640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f60113920 con 0x7f9f60109870 2024-03-28T18:04:52.814 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:52.814+0000 7f9f66600640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f601137a0 con 0x7f9f60107cc0 2024-03-28T18:04:52.814 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:52.814+0000 7f9f66600640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f60113aa0 con 0x7f9f60108eb0 2024-03-28T18:04:52.815 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:52.814+0000 7f9f5fe00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9f60109870 0x7f9f601a5540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:04:52.816 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:52.814+0000 7f9f5fe00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9f60109870 0x7f9f601a5540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:49726/0 (socket says 172.21.15.22:49726) 2024-03-28T18:04:52.816 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:52.814+0000 7f9f5fe00640 1 -- 172.21.15.22:0/3782619694 learned_addr learned my addr 172.21.15.22:0/3782619694 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-28T18:04:52.816 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:52.814+0000 7f9f5f400640 1 --2- 172.21.15.22:0/3782619694 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f9f60107cc0 0x7f9f6019df90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:04:52.816 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:52.814+0000 7f9f5ea00640 1 --2- 172.21.15.22:0/3782619694 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9f60108eb0 0x7f9f6019e4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:04:52.816 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:52.815+0000 7f9f5fe00640 1 -- 172.21.15.22:0/3782619694 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9f60108eb0 msgr2=0x7f9f6019e4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:04:52.816 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:52.815+0000 7f9f5fe00640 1 --2- 172.21.15.22:0/3782619694 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9f60108eb0 0x7f9f6019e4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:04:52.816 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:52.815+0000 7f9f5fe00640 1 -- 172.21.15.22:0/3782619694 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f9f60107cc0 msgr2=0x7f9f6019df90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:04:52.816 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:52.815+0000 7f9f5fe00640 1 --2- 172.21.15.22:0/3782619694 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f9f60107cc0 0x7f9f6019df90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:04:52.816 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:52.815+0000 7f9f5fe00640 1 -- 172.21.15.22:0/3782619694 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9f601a5bb0 con 0x7f9f60109870 2024-03-28T18:04:52.816 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:52.815+0000 7f9f5fe00640 1 --2- 172.21.15.22:0/3782619694 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9f60109870 0x7f9f601a5540 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f9f5802c630 tx=0x7f9f58032980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:04:52.816 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:52.815+0000 7f9f57e00640 1 -- 172.21.15.22:0/3782619694 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f58039050 con 0x7f9f60109870 2024-03-28T18:04:52.816 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:52.815+0000 7f9f57e00640 1 -- 172.21.15.22:0/3782619694 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f9f5803a040 con 0x7f9f60109870 2024-03-28T18:04:52.816 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:52.815+0000 7f9f57e00640 1 -- 172.21.15.22:0/3782619694 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f58040450 con 0x7f9f60109870 2024-03-28T18:04:52.816 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:52.815+0000 7f9f66600640 1 -- 172.21.15.22:0/3782619694 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9f601a5e40 con 0x7f9f60109870 2024-03-28T18:04:52.817 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:52.816+0000 7f9f66600640 1 -- 172.21.15.22:0/3782619694 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9f601a6300 con 0x7f9f60109870 2024-03-28T18:04:52.817 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:52.817+0000 7f9f66600640 1 -- 172.21.15.22:0/3782619694 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9f6010efe0 con 0x7f9f60109870 2024-03-28T18:04:52.818 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:52.819+0000 7f9f57e00640 1 -- 172.21.15.22:0/3782619694 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 16) v1 ==== 137290+0+0 (secure 0 0 0) 0x7f9f58004040 con 0x7f9f60109870 2024-03-28T18:04:52.820 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:52.820+0000 7f9f57e00640 1 --2- 172.21.15.22:0/3782619694 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7f9f440a2880 0x7f9f440a4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:04:52.820 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:52.820+0000 7f9f57e00640 1 -- 172.21.15.22:0/3782619694 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f9f580f20f0 con 0x7f9f60109870 2024-03-28T18:04:52.820 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:52.820+0000 7f9f5f400640 1 --2- 172.21.15.22:0/3782619694 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7f9f440a2880 0x7f9f440a4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:04:52.820 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:52.821+0000 7f9f5f400640 1 --2- 172.21.15.22:0/3782619694 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7f9f440a2880 0x7f9f440a4d40 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f9f50005fd0 tx=0x7f9f5000b390 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:04:52.822 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:52.822+0000 7f9f57e00640 1 -- 172.21.15.22:0/3782619694 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f9f58004330 con 0x7f9f60109870 2024-03-28T18:04:52.925 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:04:52 smithi134 ceph-mon[35037]: pgmap v370: 97 pgs: 22 creating+peering, 60 unknown, 15 active+clean; 578 KiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 767 B/s wr, 1 op/s 2024-03-28T18:04:52.925 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:04:52 smithi134 ceph-mon[35037]: Health check cleared: POOL_APP_NOT_ENABLED (was: 2 pool(s) do not have an application enabled) 2024-03-28T18:04:52.925 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:04:52 smithi134 ceph-mon[35037]: Cluster is now healthy 2024-03-28T18:04:52.925 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:04:52 smithi134 ceph-mon[35037]: from='client.? 172.21.15.22:0/3539404311' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-03-28T18:04:52.925 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:04:52 smithi134 ceph-mon[35037]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-03-28T18:04:52.925 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:04:52 smithi134 ceph-mon[35037]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T18:04:52.925 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:04:52 smithi134 ceph-mon[35037]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T18:04:52.925 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:04:52 smithi134 ceph-mon[35037]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T18:04:52.925 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:04:52 smithi134 ceph-mon[35037]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T18:04:52.926 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:04:52 smithi134 ceph-mon[35037]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T18:04:52.926 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:04:52 smithi134 ceph-mon[35037]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T18:04:52.926 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:04:52 smithi134 ceph-mon[35037]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T18:04:52.926 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:04:52 smithi134 ceph-mon[35037]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T18:04:52.926 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:04:52 smithi134 ceph-mon[35037]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T18:04:52.926 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:04:52 smithi134 ceph-mon[35037]: from='client.? 172.21.15.22:0/4187516479' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]: dispatch 2024-03-28T18:04:52.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:04:52 smithi079 ceph-mon[35312]: pgmap v370: 97 pgs: 22 creating+peering, 60 unknown, 15 active+clean; 578 KiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 767 B/s wr, 1 op/s 2024-03-28T18:04:52.933 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:04:52 smithi079 ceph-mon[35312]: Health check cleared: POOL_APP_NOT_ENABLED (was: 2 pool(s) do not have an application enabled) 2024-03-28T18:04:52.933 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:04:52 smithi079 ceph-mon[35312]: Cluster is now healthy 2024-03-28T18:04:52.933 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:04:52 smithi079 ceph-mon[35312]: from='client.? 172.21.15.22:0/3539404311' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-03-28T18:04:52.933 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:04:52 smithi079 ceph-mon[35312]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-03-28T18:04:52.933 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:04:52 smithi079 ceph-mon[35312]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T18:04:52.933 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:04:52 smithi079 ceph-mon[35312]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T18:04:52.933 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:04:52 smithi079 ceph-mon[35312]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T18:04:52.933 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:04:52 smithi079 ceph-mon[35312]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T18:04:52.933 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:04:52 smithi079 ceph-mon[35312]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T18:04:52.933 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:04:52 smithi079 ceph-mon[35312]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T18:04:52.933 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:04:52 smithi079 ceph-mon[35312]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T18:04:52.933 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:04:52 smithi079 ceph-mon[35312]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T18:04:52.933 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:04:52 smithi079 ceph-mon[35312]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T18:04:52.933 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:04:52 smithi079 ceph-mon[35312]: from='client.? 172.21.15.22:0/4187516479' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]: dispatch 2024-03-28T18:04:53.065 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:53.065+0000 7f9f66600640 1 -- 172.21.15.22:0/3782619694 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"} v 0) v1 -- 0x7f9f6002d070 con 0x7f9f60109870 2024-03-28T18:04:53.583 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:53.582+0000 7f9f57e00640 1 -- 172.21.15.22:0/3782619694 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]=0 v19) v1 ==== 121+0+0 (secure 0 0 0) 0x7f9f580bf360 con 0x7f9f60109870 2024-03-28T18:04:53.586 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:53.586+0000 7f9f66600640 1 -- 172.21.15.22:0/3782619694 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7f9f440a2880 msgr2=0x7f9f440a4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:04:53.586 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:53.586+0000 7f9f66600640 1 --2- 172.21.15.22:0/3782619694 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7f9f440a2880 0x7f9f440a4d40 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f9f50005fd0 tx=0x7f9f5000b390 comp rx=0 tx=0).stop 2024-03-28T18:04:53.587 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:53.586+0000 7f9f66600640 1 -- 172.21.15.22:0/3782619694 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9f60109870 msgr2=0x7f9f601a5540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:04:53.587 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:53.586+0000 7f9f66600640 1 --2- 172.21.15.22:0/3782619694 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9f60109870 0x7f9f601a5540 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f9f5802c630 tx=0x7f9f58032980 comp rx=0 tx=0).stop 2024-03-28T18:04:53.587 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:53.586+0000 7f9f66600640 1 -- 172.21.15.22:0/3782619694 shutdown_connections 2024-03-28T18:04:53.587 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:53.586+0000 7f9f66600640 1 --2- 172.21.15.22:0/3782619694 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7f9f440a2880 0x7f9f440a4d40 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:04:53.587 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:53.586+0000 7f9f66600640 1 --2- 172.21.15.22:0/3782619694 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9f60109870 0x7f9f601a5540 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:04:53.587 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:53.586+0000 7f9f66600640 1 --2- 172.21.15.22:0/3782619694 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9f60108eb0 0x7f9f6019e4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:04:53.587 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:53.586+0000 7f9f66600640 1 --2- 172.21.15.22:0/3782619694 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f9f60107cc0 0x7f9f6019df90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:04:53.587 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:53.586+0000 7f9f66600640 1 -- 172.21.15.22:0/3782619694 >> 172.21.15.22:0/3782619694 conn(0x7f9f60076c60 msgr2=0x7f9f6010c1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:04:53.587 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:53.587+0000 7f9f66600640 1 -- 172.21.15.22:0/3782619694 shutdown_connections 2024-03-28T18:04:53.587 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:53.587+0000 7f9f66600640 1 -- 172.21.15.22:0/3782619694 wait complete. 2024-03-28T18:04:53.614 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs session_timeout 300 2024-03-28T18:04:53.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:04:53 smithi022 ceph-mon[31985]: daemon mds.h assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-03-28T18:04:53.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:04:53 smithi022 ceph-mon[31985]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-03-28T18:04:53.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:04:53 smithi022 ceph-mon[31985]: from='client.? 172.21.15.22:0/4187516479' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]': finished 2024-03-28T18:04:53.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:04:53 smithi022 ceph-mon[31985]: fsmap cephfs:2 {0=b=up:active,1=h=up:creating} 10 up:standby 2024-03-28T18:04:53.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:04:53 smithi022 ceph-mon[31985]: pgmap v371: 97 pgs: 22 creating+peering, 16 unknown, 59 active+clean; 579 KiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s wr, 4 op/s 2024-03-28T18:04:53.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:04:53 smithi022 ceph-mon[31985]: daemon mds.h is now active in filesystem cephfs as rank 1 2024-03-28T18:04:53.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:04:53 smithi022 ceph-mon[31985]: from='client.? 172.21.15.22:0/3782619694' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-03-28T18:04:53.824 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:53.824+0000 7f6e91400640 1 -- 172.21.15.22:0/3190503037 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6e8c1037a0 msgr2=0x7f6e8c103bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:04:53.824 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:53.824+0000 7f6e91400640 1 --2- 172.21.15.22:0/3190503037 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6e8c1037a0 0x7f6e8c103bc0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f6e7c009900 tx=0x7f6e7c02bc30 comp rx=0 tx=0).stop 2024-03-28T18:04:53.824 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:53.824+0000 7f6e91400640 1 -- 172.21.15.22:0/3190503037 shutdown_connections 2024-03-28T18:04:53.824 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:53.824+0000 7f6e91400640 1 --2- 172.21.15.22:0/3190503037 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f6e8c105350 0x7f6e8c10bbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:04:53.824 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:53.824+0000 7f6e91400640 1 --2- 172.21.15.22:0/3190503037 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6e8c104990 0x7f6e8c104e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:04:53.824 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:53.824+0000 7f6e91400640 1 --2- 172.21.15.22:0/3190503037 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6e8c1037a0 0x7f6e8c103bc0 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:04:53.824 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:53.824+0000 7f6e91400640 1 -- 172.21.15.22:0/3190503037 >> 172.21.15.22:0/3190503037 conn(0x7f6e8c0fed10 msgr2=0x7f6e8c101150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:04:53.824 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:53.824+0000 7f6e91400640 1 -- 172.21.15.22:0/3190503037 shutdown_connections 2024-03-28T18:04:53.824 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:53.824+0000 7f6e91400640 1 -- 172.21.15.22:0/3190503037 wait complete. 2024-03-28T18:04:53.827 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:53.825+0000 7f6e91400640 1 Processor -- start 2024-03-28T18:04:53.827 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:53.825+0000 7f6e91400640 1 -- start start 2024-03-28T18:04:53.827 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:53.825+0000 7f6e91400640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6e8c1037a0 0x7f6e8c199ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:04:53.827 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:53.825+0000 7f6e91400640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6e8c104990 0x7f6e8c19a000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:04:53.827 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:53.825+0000 7f6e91400640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f6e8c105350 0x7f6e8c1a1070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:04:53.827 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:53.826+0000 7f6e91400640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e8c10f450 con 0x7f6e8c1037a0 2024-03-28T18:04:53.827 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:53.826+0000 7f6e91400640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e8c10f2d0 con 0x7f6e8c105350 2024-03-28T18:04:53.827 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:53.826+0000 7f6e91400640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e8c10f5d0 con 0x7f6e8c104990 2024-03-28T18:04:53.827 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:53.826+0000 7f6e8be00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6e8c1037a0 0x7f6e8c199ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:04:53.828 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:53.826+0000 7f6e8be00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6e8c1037a0 0x7f6e8c199ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:49746/0 (socket says 172.21.15.22:49746) 2024-03-28T18:04:53.828 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:53.826+0000 7f6e8be00640 1 -- 172.21.15.22:0/1395764611 learned_addr learned my addr 172.21.15.22:0/1395764611 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-28T18:04:53.828 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:53.826+0000 7f6e90a00640 1 --2- 172.21.15.22:0/1395764611 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f6e8c105350 0x7f6e8c1a1070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:04:53.828 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:53.826+0000 7f6e83400640 1 --2- 172.21.15.22:0/1395764611 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6e8c104990 0x7f6e8c19a000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:04:53.828 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:53.826+0000 7f6e8be00640 1 -- 172.21.15.22:0/1395764611 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6e8c104990 msgr2=0x7f6e8c19a000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:04:53.828 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:53.826+0000 7f6e8be00640 1 --2- 172.21.15.22:0/1395764611 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6e8c104990 0x7f6e8c19a000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:04:53.828 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:53.826+0000 7f6e8be00640 1 -- 172.21.15.22:0/1395764611 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f6e8c105350 msgr2=0x7f6e8c1a1070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:04:53.828 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:53.826+0000 7f6e8be00640 1 --2- 172.21.15.22:0/1395764611 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f6e8c105350 0x7f6e8c1a1070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:04:53.828 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:53.826+0000 7f6e8be00640 1 -- 172.21.15.22:0/1395764611 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e8c1a16e0 con 0x7f6e8c1037a0 2024-03-28T18:04:53.828 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:53.826+0000 7f6e90a00640 1 --2- 172.21.15.22:0/1395764611 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f6e8c105350 0x7f6e8c1a1070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T18:04:53.828 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:53.827+0000 7f6e8be00640 1 --2- 172.21.15.22:0/1395764611 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6e8c1037a0 0x7f6e8c199ac0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f6e7c000c00 tx=0x7f6e7c004f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:04:53.828 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:53.827+0000 7f6e89600640 1 -- 172.21.15.22:0/1395764611 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e7c043020 con 0x7f6e8c1037a0 2024-03-28T18:04:53.829 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:53.827+0000 7f6e91400640 1 -- 172.21.15.22:0/1395764611 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6e8c1a1970 con 0x7f6e8c1037a0 2024-03-28T18:04:53.829 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:53.827+0000 7f6e89600640 1 -- 172.21.15.22:0/1395764611 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f6e7c0054d0 con 0x7f6e8c1037a0 2024-03-28T18:04:53.829 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:53.827+0000 7f6e89600640 1 -- 172.21.15.22:0/1395764611 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e7c003d40 con 0x7f6e8c1037a0 2024-03-28T18:04:53.829 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:53.827+0000 7f6e91400640 1 -- 172.21.15.22:0/1395764611 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6e8c1a1f10 con 0x7f6e8c1037a0 2024-03-28T18:04:53.829 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:53.828+0000 7f6e91400640 1 -- 172.21.15.22:0/1395764611 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6e8c10aac0 con 0x7f6e8c1037a0 2024-03-28T18:04:53.832 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:53.832+0000 7f6e89600640 1 -- 172.21.15.22:0/1395764611 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 16) v1 ==== 137290+0+0 (secure 0 0 0) 0x7f6e7c046020 con 0x7f6e8c1037a0 2024-03-28T18:04:53.833 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:53.833+0000 7f6e89600640 1 --2- 172.21.15.22:0/1395764611 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7f6e700a2800 0x7f6e700a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:04:53.834 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:53.834+0000 7f6e89600640 1 -- 172.21.15.22:0/1395764611 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f6e7c0f1090 con 0x7f6e8c1037a0 2024-03-28T18:04:53.834 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:53.834+0000 7f6e83400640 1 --2- 172.21.15.22:0/1395764611 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7f6e700a2800 0x7f6e700a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:04:53.834 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:53.834+0000 7f6e89600640 1 -- 172.21.15.22:0/1395764611 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f6e7c04d020 con 0x7f6e8c1037a0 2024-03-28T18:04:53.834 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:53.835+0000 7f6e83400640 1 --2- 172.21.15.22:0/1395764611 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7f6e700a2800 0x7f6e700a4cc0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f6e74005fd0 tx=0x7f6e74005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:04:53.925 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:04:53 smithi134 ceph-mon[35037]: daemon mds.h assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-03-28T18:04:53.925 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:04:53 smithi134 ceph-mon[35037]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-03-28T18:04:53.925 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:04:53 smithi134 ceph-mon[35037]: from='client.? 172.21.15.22:0/4187516479' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]': finished 2024-03-28T18:04:53.925 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:04:53 smithi134 ceph-mon[35037]: fsmap cephfs:2 {0=b=up:active,1=h=up:creating} 10 up:standby 2024-03-28T18:04:53.925 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:04:53 smithi134 ceph-mon[35037]: pgmap v371: 97 pgs: 22 creating+peering, 16 unknown, 59 active+clean; 579 KiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s wr, 4 op/s 2024-03-28T18:04:53.925 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:04:53 smithi134 ceph-mon[35037]: daemon mds.h is now active in filesystem cephfs as rank 1 2024-03-28T18:04:53.925 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:04:53 smithi134 ceph-mon[35037]: from='client.? 172.21.15.22:0/3782619694' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-03-28T18:04:53.933 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:04:53 smithi079 ceph-mon[35312]: daemon mds.h assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-03-28T18:04:53.933 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:04:53 smithi079 ceph-mon[35312]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-03-28T18:04:53.933 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:04:53 smithi079 ceph-mon[35312]: from='client.? 172.21.15.22:0/4187516479' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]': finished 2024-03-28T18:04:53.933 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:04:53 smithi079 ceph-mon[35312]: fsmap cephfs:2 {0=b=up:active,1=h=up:creating} 10 up:standby 2024-03-28T18:04:53.933 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:04:53 smithi079 ceph-mon[35312]: pgmap v371: 97 pgs: 22 creating+peering, 16 unknown, 59 active+clean; 579 KiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s wr, 4 op/s 2024-03-28T18:04:53.933 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:04:53 smithi079 ceph-mon[35312]: daemon mds.h is now active in filesystem cephfs as rank 1 2024-03-28T18:04:53.933 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:04:53 smithi079 ceph-mon[35312]: from='client.? 172.21.15.22:0/3782619694' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-03-28T18:04:54.082 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:54.083+0000 7f6e91400640 1 -- 172.21.15.22:0/1395764611 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"} v 0) v1 -- 0x7f6e8c02d2e0 con 0x7f6e8c1037a0 2024-03-28T18:04:54.597 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:54.596+0000 7f6e89600640 1 -- 172.21.15.22:0/1395764611 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]=0 v21) v1 ==== 115+0+0 (secure 0 0 0) 0x7f6e7c0be280 con 0x7f6e8c1037a0 2024-03-28T18:04:54.600 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:54.600+0000 7f6e91400640 1 -- 172.21.15.22:0/1395764611 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7f6e700a2800 msgr2=0x7f6e700a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:04:54.600 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:54.600+0000 7f6e91400640 1 --2- 172.21.15.22:0/1395764611 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7f6e700a2800 0x7f6e700a4cc0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f6e74005fd0 tx=0x7f6e74005850 comp rx=0 tx=0).stop 2024-03-28T18:04:54.600 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:54.600+0000 7f6e91400640 1 -- 172.21.15.22:0/1395764611 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6e8c1037a0 msgr2=0x7f6e8c199ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:04:54.600 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:54.600+0000 7f6e91400640 1 --2- 172.21.15.22:0/1395764611 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6e8c1037a0 0x7f6e8c199ac0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f6e7c000c00 tx=0x7f6e7c004f10 comp rx=0 tx=0).stop 2024-03-28T18:04:54.600 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:54.600+0000 7f6e91400640 1 -- 172.21.15.22:0/1395764611 shutdown_connections 2024-03-28T18:04:54.600 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:54.600+0000 7f6e91400640 1 --2- 172.21.15.22:0/1395764611 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7f6e700a2800 0x7f6e700a4cc0 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:04:54.600 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:54.600+0000 7f6e91400640 1 --2- 172.21.15.22:0/1395764611 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f6e8c105350 0x7f6e8c1a1070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:04:54.600 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:54.600+0000 7f6e91400640 1 --2- 172.21.15.22:0/1395764611 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6e8c104990 0x7f6e8c19a000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:04:54.600 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:54.600+0000 7f6e91400640 1 --2- 172.21.15.22:0/1395764611 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6e8c1037a0 0x7f6e8c199ac0 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:04:54.600 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:54.600+0000 7f6e91400640 1 -- 172.21.15.22:0/1395764611 >> 172.21.15.22:0/1395764611 conn(0x7f6e8c0fed10 msgr2=0x7f6e8c107c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:04:54.601 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:54.600+0000 7f6e91400640 1 -- 172.21.15.22:0/1395764611 shutdown_connections 2024-03-28T18:04:54.601 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:54.600+0000 7f6e91400640 1 -- 172.21.15.22:0/1395764611 wait complete. 2024-03-28T18:04:54.631 DEBUG:tasks.cephfs.filesystem:Creating {'create': 2, 'subvol_options': '--size 25000000000'} subvols for filesystem 'cephfs' 2024-03-28T18:04:54.631 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolumegroup create cephfs qa 2024-03-28T18:04:54.695 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:04:54 smithi022 ceph-mon[31985]: from='client.? 172.21.15.22:0/3782619694' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-03-28T18:04:54.695 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:04:54 smithi022 ceph-mon[31985]: mds.? [v2:172.21.15.79:6836/1787903366,v1:172.21.15.79:6837/1787903366] up:active 2024-03-28T18:04:54.695 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:04:54 smithi022 ceph-mon[31985]: daemon mds.e assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-03-28T18:04:54.695 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:04:54 smithi022 ceph-mon[31985]: fsmap cephfs:2 {0=b=up:active,1=h=up:active} 10 up:standby 2024-03-28T18:04:54.695 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:04:54 smithi022 ceph-mon[31985]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=e=up:creating} 9 up:standby 2024-03-28T18:04:54.695 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:04:54 smithi022 ceph-mon[31985]: daemon mds.e is now active in filesystem cephfs as rank 2 2024-03-28T18:04:54.695 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:04:54 smithi022 ceph-mon[31985]: from='client.? 172.21.15.22:0/1395764611' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-03-28T18:04:54.832 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:54.832+0000 7f79a8c00640 1 -- 172.21.15.22:0/3876291540 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f79a4101580 msgr2=0x7f79a4101a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:04:54.832 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:54.832+0000 7f79a8c00640 1 --2- 172.21.15.22:0/3876291540 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f79a4101580 0x7f79a4101a00 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f798c009900 tx=0x7f798c02bc30 comp rx=0 tx=0).stop 2024-03-28T18:04:54.832 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:54.832+0000 7f79a8c00640 1 -- 172.21.15.22:0/3876291540 shutdown_connections 2024-03-28T18:04:54.832 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:54.832+0000 7f79a8c00640 1 --2- 172.21.15.22:0/3876291540 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f79a4101f40 0x7f79a41087d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:04:54.832 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:54.832+0000 7f79a8c00640 1 --2- 172.21.15.22:0/3876291540 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f79a4101580 0x7f79a4101a00 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:04:54.832 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:54.832+0000 7f79a8c00640 1 --2- 172.21.15.22:0/3876291540 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f79a4100390 0x7f79a41007b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:04:54.832 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:54.832+0000 7f79a8c00640 1 -- 172.21.15.22:0/3876291540 >> 172.21.15.22:0/3876291540 conn(0x7f79a40fb940 msgr2=0x7f79a40fdd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:04:54.832 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:54.832+0000 7f79a8c00640 1 -- 172.21.15.22:0/3876291540 shutdown_connections 2024-03-28T18:04:54.833 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:54.832+0000 7f79a8c00640 1 -- 172.21.15.22:0/3876291540 wait complete. 2024-03-28T18:04:54.835 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:54.833+0000 7f79a8c00640 1 Processor -- start 2024-03-28T18:04:54.835 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:54.833+0000 7f79a8c00640 1 -- start start 2024-03-28T18:04:54.835 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:54.834+0000 7f79a8c00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f79a4100390 0x7f79a4199a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:04:54.835 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:54.834+0000 7f79a8c00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f79a4101580 0x7f79a4199f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:04:54.835 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:54.834+0000 7f79a8c00640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f79a4101f40 0x7f79a41a0fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:04:54.835 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:54.834+0000 7f79a8c00640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79a410c040 con 0x7f79a4101580 2024-03-28T18:04:54.835 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:54.834+0000 7f79a8c00640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79a410bec0 con 0x7f79a4101f40 2024-03-28T18:04:54.835 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:54.834+0000 7f79a8c00640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79a410c1c0 con 0x7f79a4100390 2024-03-28T18:04:54.835 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:54.834+0000 7f79a2a00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f79a4101580 0x7f79a4199f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:04:54.836 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:54.834+0000 7f79a2a00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f79a4101580 0x7f79a4199f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:49754/0 (socket says 172.21.15.22:49754) 2024-03-28T18:04:54.836 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:54.834+0000 7f79a2a00640 1 -- 172.21.15.22:0/3729675655 learned_addr learned my addr 172.21.15.22:0/3729675655 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-28T18:04:54.836 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:54.834+0000 7f79a3400640 1 --2- 172.21.15.22:0/3729675655 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f79a4100390 0x7f79a4199a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:04:54.836 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:54.834+0000 7f79a3e00640 1 --2- 172.21.15.22:0/3729675655 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f79a4101f40 0x7f79a41a0fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:04:54.836 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:54.834+0000 7f79a2a00640 1 -- 172.21.15.22:0/3729675655 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f79a4100390 msgr2=0x7f79a4199a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:04:54.836 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:54.834+0000 7f79a2a00640 1 --2- 172.21.15.22:0/3729675655 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f79a4100390 0x7f79a4199a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:04:54.836 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:54.834+0000 7f79a2a00640 1 -- 172.21.15.22:0/3729675655 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f79a4101f40 msgr2=0x7f79a41a0fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:04:54.836 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:54.834+0000 7f79a2a00640 1 --2- 172.21.15.22:0/3729675655 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f79a4101f40 0x7f79a41a0fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:04:54.836 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:54.834+0000 7f79a2a00640 1 -- 172.21.15.22:0/3729675655 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f79a41a1650 con 0x7f79a4101580 2024-03-28T18:04:54.836 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:54.835+0000 7f79a2a00640 1 --2- 172.21.15.22:0/3729675655 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f79a4101580 0x7f79a4199f70 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f798c005ab0 tx=0x7f798c005e40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:04:54.836 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:54.835+0000 7f799be00640 1 -- 172.21.15.22:0/3729675655 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f798c043020 con 0x7f79a4101580 2024-03-28T18:04:54.836 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:54.835+0000 7f799be00640 1 -- 172.21.15.22:0/3729675655 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f798c005470 con 0x7f79a4101580 2024-03-28T18:04:54.836 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:54.835+0000 7f799be00640 1 -- 172.21.15.22:0/3729675655 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f798c003d00 con 0x7f79a4101580 2024-03-28T18:04:54.836 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:54.835+0000 7f79a8c00640 1 -- 172.21.15.22:0/3729675655 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f79a41a18e0 con 0x7f79a4101580 2024-03-28T18:04:54.836 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:54.835+0000 7f79a8c00640 1 -- 172.21.15.22:0/3729675655 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f79a41a1e30 con 0x7f79a4101580 2024-03-28T18:04:54.837 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:54.837+0000 7f79a8c00640 1 -- 172.21.15.22:0/3729675655 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f79a40683b0 con 0x7f79a4101580 2024-03-28T18:04:54.838 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:54.839+0000 7f799be00640 1 -- 172.21.15.22:0/3729675655 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 16) v1 ==== 137290+0+0 (secure 0 0 0) 0x7f798c046020 con 0x7f79a4101580 2024-03-28T18:04:54.840 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:54.840+0000 7f799be00640 1 --2- 172.21.15.22:0/3729675655 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7f79880a2800 0x7f79880a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:04:54.840 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:54.840+0000 7f799be00640 1 -- 172.21.15.22:0/3729675655 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f798c0f0ca0 con 0x7f79a4101580 2024-03-28T18:04:54.840 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:54.840+0000 7f79a3400640 1 --2- 172.21.15.22:0/3729675655 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7f79880a2800 0x7f79880a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:04:54.841 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:54.841+0000 7f79a3400640 1 --2- 172.21.15.22:0/3729675655 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7f79880a2800 0x7f79880a4cc0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f79a41013e0 tx=0x7f7994009500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:04:54.844 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:54.844+0000 7f799be00640 1 -- 172.21.15.22:0/3729675655 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f798c04d020 con 0x7f79a4101580 2024-03-28T18:04:54.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:04:54 smithi134 ceph-mon[35037]: from='client.? 172.21.15.22:0/3782619694' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-03-28T18:04:54.925 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:04:54 smithi134 ceph-mon[35037]: mds.? [v2:172.21.15.79:6836/1787903366,v1:172.21.15.79:6837/1787903366] up:active 2024-03-28T18:04:54.925 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:04:54 smithi134 ceph-mon[35037]: daemon mds.e assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-03-28T18:04:54.925 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:04:54 smithi134 ceph-mon[35037]: fsmap cephfs:2 {0=b=up:active,1=h=up:active} 10 up:standby 2024-03-28T18:04:54.925 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:04:54 smithi134 ceph-mon[35037]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=e=up:creating} 9 up:standby 2024-03-28T18:04:54.925 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:04:54 smithi134 ceph-mon[35037]: daemon mds.e is now active in filesystem cephfs as rank 2 2024-03-28T18:04:54.925 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:04:54 smithi134 ceph-mon[35037]: from='client.? 172.21.15.22:0/1395764611' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-03-28T18:04:54.933 INFO:journalctl@ceph.mds.b.smithi079.stdout:Mar 28 18:04:54 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-b[65245]: 2024-03-28T18:04:54.581+0000 7f845f400700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-03-28T18:04:54.933 INFO:journalctl@ceph.mds.b.smithi079.stdout:Mar 28 18:04:54 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-b[65245]: 2024-03-28T18:04:54.581+0000 7f845f400700 -1 mds.pinger is_rank_lagging: rank=1 was never sent ping request. 2024-03-28T18:04:54.933 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:04:54 smithi079 ceph-mon[35312]: from='client.? 172.21.15.22:0/3782619694' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-03-28T18:04:54.933 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:04:54 smithi079 ceph-mon[35312]: mds.? [v2:172.21.15.79:6836/1787903366,v1:172.21.15.79:6837/1787903366] up:active 2024-03-28T18:04:54.933 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:04:54 smithi079 ceph-mon[35312]: daemon mds.e assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-03-28T18:04:54.933 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:04:54 smithi079 ceph-mon[35312]: fsmap cephfs:2 {0=b=up:active,1=h=up:active} 10 up:standby 2024-03-28T18:04:54.933 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:04:54 smithi079 ceph-mon[35312]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=e=up:creating} 9 up:standby 2024-03-28T18:04:54.933 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:04:54 smithi079 ceph-mon[35312]: daemon mds.e is now active in filesystem cephfs as rank 2 2024-03-28T18:04:54.933 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:04:54 smithi079 ceph-mon[35312]: from='client.? 172.21.15.22:0/1395764611' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-03-28T18:04:55.071 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:55.071+0000 7f79a8c00640 1 -- 172.21.15.22:0/3729675655 --> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] -- mgr_command(tid 0: {"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}) v1 -- 0x7f79a40fdad0 con 0x7f79880a2800 2024-03-28T18:04:55.343 INFO:journalctl@ceph.mgr.x.smithi022.stdout:Mar 28 18:04:55 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mgr-x[32211]: 2024-03-28T18:04:55.080+0000 7fd7ef800700 -1 client.0 error registering admin socket command: (17) File exists 2024-03-28T18:04:55.343 INFO:journalctl@ceph.mgr.x.smithi022.stdout:Mar 28 18:04:55 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mgr-x[32211]: 2024-03-28T18:04:55.080+0000 7fd7ef800700 -1 client.0 error registering admin socket command: (17) File exists 2024-03-28T18:04:55.343 INFO:journalctl@ceph.mgr.x.smithi022.stdout:Mar 28 18:04:55 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mgr-x[32211]: 2024-03-28T18:04:55.080+0000 7fd7ef800700 -1 client.0 error registering admin socket command: (17) File exists 2024-03-28T18:04:55.343 INFO:journalctl@ceph.mgr.x.smithi022.stdout:Mar 28 18:04:55 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mgr-x[32211]: 2024-03-28T18:04:55.080+0000 7fd7ef800700 -1 client.0 error registering admin socket command: (17) File exists 2024-03-28T18:04:55.344 INFO:journalctl@ceph.mgr.x.smithi022.stdout:Mar 28 18:04:55 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mgr-x[32211]: 2024-03-28T18:04:55.080+0000 7fd7ef800700 -1 client.0 error registering admin socket command: (17) File exists 2024-03-28T18:04:55.662 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:55.662+0000 7f799be00640 1 -- 172.21.15.22:0/3729675655 <== mgr.14152 v2:172.21.15.22:6800/4201048392 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f79a40fdad0 con 0x7f79880a2800 2024-03-28T18:04:55.665 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:55.665+0000 7f79a8c00640 1 -- 172.21.15.22:0/3729675655 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7f79880a2800 msgr2=0x7f79880a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:04:55.666 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:55.665+0000 7f79a8c00640 1 --2- 172.21.15.22:0/3729675655 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7f79880a2800 0x7f79880a4cc0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f79a41013e0 tx=0x7f7994009500 comp rx=0 tx=0).stop 2024-03-28T18:04:55.666 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:55.665+0000 7f79a8c00640 1 -- 172.21.15.22:0/3729675655 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f79a4101580 msgr2=0x7f79a4199f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:04:55.666 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:55.665+0000 7f79a8c00640 1 --2- 172.21.15.22:0/3729675655 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f79a4101580 0x7f79a4199f70 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f798c005ab0 tx=0x7f798c005e40 comp rx=0 tx=0).stop 2024-03-28T18:04:55.666 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:55.665+0000 7f79a8c00640 1 -- 172.21.15.22:0/3729675655 shutdown_connections 2024-03-28T18:04:55.666 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:55.665+0000 7f79a8c00640 1 --2- 172.21.15.22:0/3729675655 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7f79880a2800 0x7f79880a4cc0 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:04:55.666 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:55.666+0000 7f79a8c00640 1 --2- 172.21.15.22:0/3729675655 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f79a4101f40 0x7f79a41a0fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:04:55.666 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:55.666+0000 7f79a8c00640 1 --2- 172.21.15.22:0/3729675655 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f79a4101580 0x7f79a4199f70 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:04:55.666 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:55.666+0000 7f79a8c00640 1 --2- 172.21.15.22:0/3729675655 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f79a4100390 0x7f79a4199a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:04:55.666 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:55.666+0000 7f79a8c00640 1 -- 172.21.15.22:0/3729675655 >> 172.21.15.22:0/3729675655 conn(0x7f79a40fb940 msgr2=0x7f79a40fd3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:04:55.666 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:55.666+0000 7f79a8c00640 1 -- 172.21.15.22:0/3729675655 shutdown_connections 2024-03-28T18:04:55.666 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:55.666+0000 7f79a8c00640 1 -- 172.21.15.22:0/3729675655 wait complete. 2024-03-28T18:04:55.687 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume create cephfs sv_0 --group_name qa 2024-03-28T18:04:55.914 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:55.914+0000 7fbbece00640 1 -- 172.21.15.22:0/2973296584 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fbbe81038e0 msgr2=0x7fbbe8103d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:04:55.914 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:55.914+0000 7fbbece00640 1 --2- 172.21.15.22:0/2973296584 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fbbe81038e0 0x7fbbe8103d00 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fbbd0009900 tx=0x7fbbd002bc40 comp rx=0 tx=0).stop 2024-03-28T18:04:55.914 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:55.914+0000 7fbbece00640 1 -- 172.21.15.22:0/2973296584 shutdown_connections 2024-03-28T18:04:55.914 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:55.914+0000 7fbbece00640 1 --2- 172.21.15.22:0/2973296584 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fbbe8105490 0x7fbbe810bd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:04:55.914 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:55.914+0000 7fbbece00640 1 --2- 172.21.15.22:0/2973296584 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbbe8104ad0 0x7fbbe8104f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:04:55.914 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:55.914+0000 7fbbece00640 1 --2- 172.21.15.22:0/2973296584 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fbbe81038e0 0x7fbbe8103d00 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:04:55.914 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:55.914+0000 7fbbece00640 1 -- 172.21.15.22:0/2973296584 >> 172.21.15.22:0/2973296584 conn(0x7fbbe80fee50 msgr2=0x7fbbe8101290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:04:55.914 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:55.914+0000 7fbbece00640 1 -- 172.21.15.22:0/2973296584 shutdown_connections 2024-03-28T18:04:55.914 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:55.914+0000 7fbbece00640 1 -- 172.21.15.22:0/2973296584 wait complete. 2024-03-28T18:04:55.916 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:55.915+0000 7fbbece00640 1 Processor -- start 2024-03-28T18:04:55.916 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:55.915+0000 7fbbece00640 1 -- start start 2024-03-28T18:04:55.916 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:55.916+0000 7fbbece00640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fbbe81038e0 0x7fbbe8199bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:04:55.916 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:55.916+0000 7fbbece00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fbbe8104ad0 0x7fbbe819a0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:04:55.916 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:55.916+0000 7fbbece00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbbe8105490 0x7fbbe81a1160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:04:55.916 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:55.916+0000 7fbbece00640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbbe810f590 con 0x7fbbe8104ad0 2024-03-28T18:04:55.916 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:55.916+0000 7fbbece00640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbbe810f410 con 0x7fbbe81038e0 2024-03-28T18:04:55.916 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:55.916+0000 7fbbece00640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbbe810f710 con 0x7fbbe8105490 2024-03-28T18:04:55.916 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:55.916+0000 7fbbe5400640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fbbe8104ad0 0x7fbbe819a0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:04:55.916 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:55.916+0000 7fbbe5e00640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fbbe81038e0 0x7fbbe8199bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:04:55.916 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:55.916+0000 7fbbe5400640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fbbe8104ad0 0x7fbbe819a0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:49782/0 (socket says 172.21.15.22:49782) 2024-03-28T18:04:55.917 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:55.916+0000 7fbbe5400640 1 -- 172.21.15.22:0/1768455002 learned_addr learned my addr 172.21.15.22:0/1768455002 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-28T18:04:55.918 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:55.916+0000 7fbbe6800640 1 --2- 172.21.15.22:0/1768455002 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbbe8105490 0x7fbbe81a1160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:04:55.918 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:55.917+0000 7fbbe5e00640 1 -- 172.21.15.22:0/1768455002 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbbe8105490 msgr2=0x7fbbe81a1160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:04:55.918 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:55.917+0000 7fbbe5e00640 1 --2- 172.21.15.22:0/1768455002 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbbe8105490 0x7fbbe81a1160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:04:55.918 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:55.917+0000 7fbbe5e00640 1 -- 172.21.15.22:0/1768455002 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fbbe8104ad0 msgr2=0x7fbbe819a0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:04:55.918 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:55.917+0000 7fbbe5e00640 1 --2- 172.21.15.22:0/1768455002 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fbbe8104ad0 0x7fbbe819a0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:04:55.918 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:55.917+0000 7fbbe5e00640 1 -- 172.21.15.22:0/1768455002 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbbe81a17d0 con 0x7fbbe81038e0 2024-03-28T18:04:55.918 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:55.917+0000 7fbbe5400640 1 --2- 172.21.15.22:0/1768455002 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fbbe8104ad0 0x7fbbe819a0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-28T18:04:55.918 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:55.917+0000 7fbbe5e00640 1 --2- 172.21.15.22:0/1768455002 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fbbe81038e0 0x7fbbe8199bb0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fbbd0009230 tx=0x7fbbd0005e60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:04:55.918 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:55.917+0000 7fbbdea00640 1 -- 172.21.15.22:0/1768455002 <== mon.2 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbbd0043020 con 0x7fbbe81038e0 2024-03-28T18:04:55.918 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:55.917+0000 7fbbece00640 1 -- 172.21.15.22:0/1768455002 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbbe81a1a60 con 0x7fbbe81038e0 2024-03-28T18:04:55.919 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:55.917+0000 7fbbdea00640 1 -- 172.21.15.22:0/1768455002 <== mon.2 v2:172.21.15.79:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7fbbd0005470 con 0x7fbbe81038e0 2024-03-28T18:04:55.919 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:55.917+0000 7fbbdea00640 1 -- 172.21.15.22:0/1768455002 <== mon.2 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbbd0003b10 con 0x7fbbe81038e0 2024-03-28T18:04:55.919 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:55.918+0000 7fbbece00640 1 -- 172.21.15.22:0/1768455002 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbbe81a1fd0 con 0x7fbbe81038e0 2024-03-28T18:04:55.919 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:55.919+0000 7fbbece00640 1 -- 172.21.15.22:0/1768455002 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbbe810ac00 con 0x7fbbe81038e0 2024-03-28T18:04:55.920 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:55.921+0000 7fbbdea00640 1 -- 172.21.15.22:0/1768455002 <== mon.2 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 16) v1 ==== 137290+0+0 (secure 0 0 0) 0x7fbbd0046020 con 0x7fbbe81038e0 2024-03-28T18:04:55.922 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:55.922+0000 7fbbdea00640 1 --2- 172.21.15.22:0/1768455002 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7fbbcc0a2880 0x7fbbcc0a4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:04:55.922 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:55.922+0000 7fbbdea00640 1 -- 172.21.15.22:0/1768455002 <== mon.2 v2:172.21.15.79:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 7979+0+0 (secure 0 0 0) 0x7fbbd00f0f70 con 0x7fbbe81038e0 2024-03-28T18:04:55.922 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:55.922+0000 7fbbe5400640 1 --2- 172.21.15.22:0/1768455002 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7fbbcc0a2880 0x7fbbcc0a4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:04:55.922 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:55.923+0000 7fbbe5400640 1 --2- 172.21.15.22:0/1768455002 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7fbbcc0a2880 0x7fbbcc0a4d40 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fbbe819b0c0 tx=0x7fbbd8007670 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:04:55.925 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:04:55 smithi134 ceph-mon[35037]: from='client.? 172.21.15.22:0/1395764611' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-03-28T18:04:55.925 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:04:55 smithi134 ceph-mon[35037]: pgmap v372: 97 pgs: 97 active+clean; 579 KiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s wr, 11 op/s 2024-03-28T18:04:55.925 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:04:55 smithi134 ceph-mon[35037]: mds.? [v2:172.21.15.79:6834/3365773431,v1:172.21.15.79:6835/3365773431] up:active 2024-03-28T18:04:55.925 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:04:55 smithi134 ceph-mon[35037]: daemon mds.k assigned to filesystem cephfs as rank 3 (now has 4 ranks) 2024-03-28T18:04:55.925 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:04:55 smithi134 ceph-mon[35037]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=e=up:active} 9 up:standby 2024-03-28T18:04:55.925 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:04:55 smithi134 ceph-mon[35037]: fsmap cephfs:4 {0=b=up:active,1=h=up:active,2=e=up:active,3=k=up:creating} 8 up:standby 2024-03-28T18:04:55.925 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:04:55 smithi134 ceph-mon[35037]: daemon mds.k is now active in filesystem cephfs as rank 3 2024-03-28T18:04:55.925 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:04:55 smithi134 ceph-mon[35037]: from='client.14748 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-03-28T18:04:55.926 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:55.927+0000 7fbbdea00640 1 -- 172.21.15.22:0/1768455002 <== mon.2 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fbbd004d020 con 0x7fbbe81038e0 2024-03-28T18:04:55.933 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:04:55 smithi079 ceph-mon[35312]: from='client.? 172.21.15.22:0/1395764611' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-03-28T18:04:55.933 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:04:55 smithi079 ceph-mon[35312]: pgmap v372: 97 pgs: 97 active+clean; 579 KiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s wr, 11 op/s 2024-03-28T18:04:55.933 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:04:55 smithi079 ceph-mon[35312]: mds.? [v2:172.21.15.79:6834/3365773431,v1:172.21.15.79:6835/3365773431] up:active 2024-03-28T18:04:55.933 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:04:55 smithi079 ceph-mon[35312]: daemon mds.k assigned to filesystem cephfs as rank 3 (now has 4 ranks) 2024-03-28T18:04:55.933 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:04:55 smithi079 ceph-mon[35312]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=e=up:active} 9 up:standby 2024-03-28T18:04:55.933 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:04:55 smithi079 ceph-mon[35312]: fsmap cephfs:4 {0=b=up:active,1=h=up:active,2=e=up:active,3=k=up:creating} 8 up:standby 2024-03-28T18:04:55.933 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:04:55 smithi079 ceph-mon[35312]: daemon mds.k is now active in filesystem cephfs as rank 3 2024-03-28T18:04:55.933 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:04:55 smithi079 ceph-mon[35312]: from='client.14748 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-03-28T18:04:56.032 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:04:55 smithi022 ceph-mon[31985]: from='client.? 172.21.15.22:0/1395764611' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-03-28T18:04:56.032 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:04:55 smithi022 ceph-mon[31985]: pgmap v372: 97 pgs: 97 active+clean; 579 KiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s wr, 11 op/s 2024-03-28T18:04:56.032 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:04:55 smithi022 ceph-mon[31985]: mds.? [v2:172.21.15.79:6834/3365773431,v1:172.21.15.79:6835/3365773431] up:active 2024-03-28T18:04:56.032 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:04:55 smithi022 ceph-mon[31985]: daemon mds.k assigned to filesystem cephfs as rank 3 (now has 4 ranks) 2024-03-28T18:04:56.032 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:04:55 smithi022 ceph-mon[31985]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=e=up:active} 9 up:standby 2024-03-28T18:04:56.033 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:04:55 smithi022 ceph-mon[31985]: fsmap cephfs:4 {0=b=up:active,1=h=up:active,2=e=up:active,3=k=up:creating} 8 up:standby 2024-03-28T18:04:56.033 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:04:55 smithi022 ceph-mon[31985]: daemon mds.k is now active in filesystem cephfs as rank 3 2024-03-28T18:04:56.033 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:04:55 smithi022 ceph-mon[31985]: from='client.14748 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-03-28T18:04:56.152 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:56.153+0000 7fbbece00640 1 -- 172.21.15.22:0/1768455002 --> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}) v1 -- 0x7fbbe8061b70 con 0x7fbbcc0a2880 2024-03-28T18:04:56.196 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:56.196+0000 7fbbdea00640 1 -- 172.21.15.22:0/1768455002 <== mgr.14152 v2:172.21.15.22:6800/4201048392 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7fbbe8061b70 con 0x7fbbcc0a2880 2024-03-28T18:04:56.199 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:56.199+0000 7fbbece00640 1 -- 172.21.15.22:0/1768455002 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7fbbcc0a2880 msgr2=0x7fbbcc0a4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:04:56.199 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:56.199+0000 7fbbece00640 1 --2- 172.21.15.22:0/1768455002 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7fbbcc0a2880 0x7fbbcc0a4d40 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fbbe819b0c0 tx=0x7fbbd8007670 comp rx=0 tx=0).stop 2024-03-28T18:04:56.199 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:56.199+0000 7fbbece00640 1 -- 172.21.15.22:0/1768455002 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fbbe81038e0 msgr2=0x7fbbe8199bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:04:56.199 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:56.199+0000 7fbbece00640 1 --2- 172.21.15.22:0/1768455002 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fbbe81038e0 0x7fbbe8199bb0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fbbd0009230 tx=0x7fbbd0005e60 comp rx=0 tx=0).stop 2024-03-28T18:04:56.200 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:56.199+0000 7fbbece00640 1 -- 172.21.15.22:0/1768455002 shutdown_connections 2024-03-28T18:04:56.200 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:56.199+0000 7fbbece00640 1 --2- 172.21.15.22:0/1768455002 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7fbbcc0a2880 0x7fbbcc0a4d40 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:04:56.200 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:56.199+0000 7fbbece00640 1 --2- 172.21.15.22:0/1768455002 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbbe8105490 0x7fbbe81a1160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:04:56.200 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:56.199+0000 7fbbece00640 1 --2- 172.21.15.22:0/1768455002 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fbbe8104ad0 0x7fbbe819a0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:04:56.200 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:56.199+0000 7fbbece00640 1 --2- 172.21.15.22:0/1768455002 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fbbe81038e0 0x7fbbe8199bb0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:04:56.200 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:56.199+0000 7fbbece00640 1 -- 172.21.15.22:0/1768455002 >> 172.21.15.22:0/1768455002 conn(0x7fbbe80fee50 msgr2=0x7fbbe8107dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:04:56.200 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:56.199+0000 7fbbece00640 1 -- 172.21.15.22:0/1768455002 shutdown_connections 2024-03-28T18:04:56.200 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:56.200+0000 7fbbece00640 1 -- 172.21.15.22:0/1768455002 wait complete. 2024-03-28T18:04:56.219 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs --group_name qa sv_0 2024-03-28T18:04:56.451 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:56.451+0000 7efe04400640 1 -- 172.21.15.22:0/2203035879 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7efdfc1038e0 msgr2=0x7efdfc103d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:04:56.452 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:56.451+0000 7efe04400640 1 --2- 172.21.15.22:0/2203035879 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7efdfc1038e0 0x7efdfc103d00 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7efdec009900 tx=0x7efdec02bc50 comp rx=0 tx=0).stop 2024-03-28T18:04:56.452 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:56.451+0000 7efe04400640 1 -- 172.21.15.22:0/2203035879 shutdown_connections 2024-03-28T18:04:56.452 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:56.451+0000 7efe04400640 1 --2- 172.21.15.22:0/2203035879 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7efdfc105490 0x7efdfc10bd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:04:56.452 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:56.451+0000 7efe04400640 1 --2- 172.21.15.22:0/2203035879 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7efdfc104ad0 0x7efdfc104f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:04:56.452 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:56.451+0000 7efe04400640 1 --2- 172.21.15.22:0/2203035879 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7efdfc1038e0 0x7efdfc103d00 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:04:56.452 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:56.451+0000 7efe04400640 1 -- 172.21.15.22:0/2203035879 >> 172.21.15.22:0/2203035879 conn(0x7efdfc0fee50 msgr2=0x7efdfc101290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:04:56.452 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:56.452+0000 7efe04400640 1 -- 172.21.15.22:0/2203035879 shutdown_connections 2024-03-28T18:04:56.452 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:56.452+0000 7efe04400640 1 -- 172.21.15.22:0/2203035879 wait complete. 2024-03-28T18:04:56.454 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:56.453+0000 7efe04400640 1 Processor -- start 2024-03-28T18:04:56.454 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:56.453+0000 7efe04400640 1 -- start start 2024-03-28T18:04:56.454 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:56.453+0000 7efe04400640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7efdfc1038e0 0x7efdfc199c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:04:56.455 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:56.453+0000 7efe04400640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7efdfc104ad0 0x7efdfc19a1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:04:56.455 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:56.453+0000 7efe04400640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7efdfc105490 0x7efdfc1a11e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:04:56.455 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:56.453+0000 7efe04400640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efdfc10f580 con 0x7efdfc104ad0 2024-03-28T18:04:56.455 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:56.453+0000 7efe04400640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efdfc10f400 con 0x7efdfc1038e0 2024-03-28T18:04:56.455 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:56.453+0000 7efe04400640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efdfc10f700 con 0x7efdfc105490 2024-03-28T18:04:56.455 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:56.453+0000 7efe01000640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7efdfc104ad0 0x7efdfc19a1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:04:56.455 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:56.453+0000 7efe01000640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7efdfc104ad0 0x7efdfc19a1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:49792/0 (socket says 172.21.15.22:49792) 2024-03-28T18:04:56.455 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:56.453+0000 7efe01000640 1 -- 172.21.15.22:0/2540927016 learned_addr learned my addr 172.21.15.22:0/2540927016 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-28T18:04:56.455 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:56.453+0000 7efe01a00640 1 --2- 172.21.15.22:0/2540927016 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7efdfc1038e0 0x7efdfc199c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:04:56.455 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:56.454+0000 7efe01000640 1 -- 172.21.15.22:0/2540927016 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7efdfc105490 msgr2=0x7efdfc1a11e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:04:56.455 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:56.454+0000 7efe02400640 1 --2- 172.21.15.22:0/2540927016 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7efdfc105490 0x7efdfc1a11e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:04:56.455 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:56.454+0000 7efe01000640 1 --2- 172.21.15.22:0/2540927016 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7efdfc105490 0x7efdfc1a11e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:04:56.455 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:56.454+0000 7efe01000640 1 -- 172.21.15.22:0/2540927016 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7efdfc1038e0 msgr2=0x7efdfc199c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:04:56.455 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:56.454+0000 7efe01000640 1 --2- 172.21.15.22:0/2540927016 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7efdfc1038e0 0x7efdfc199c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:04:56.455 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:56.454+0000 7efe01000640 1 -- 172.21.15.22:0/2540927016 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efdfc1a1880 con 0x7efdfc104ad0 2024-03-28T18:04:56.456 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:56.454+0000 7efe01000640 1 --2- 172.21.15.22:0/2540927016 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7efdfc104ad0 0x7efdfc19a1b0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7efde800d950 tx=0x7efde800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:04:56.456 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:56.454+0000 7efdfa000640 1 -- 172.21.15.22:0/2540927016 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efde8017030 con 0x7efdfc104ad0 2024-03-28T18:04:56.456 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:56.454+0000 7efdfa000640 1 -- 172.21.15.22:0/2540927016 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7efde8004020 con 0x7efdfc104ad0 2024-03-28T18:04:56.456 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:56.454+0000 7efdfa000640 1 -- 172.21.15.22:0/2540927016 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efde80047a0 con 0x7efdfc104ad0 2024-03-28T18:04:56.456 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:56.454+0000 7efe04400640 1 -- 172.21.15.22:0/2540927016 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efdfc1a1b70 con 0x7efdfc104ad0 2024-03-28T18:04:56.456 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:56.454+0000 7efe04400640 1 -- 172.21.15.22:0/2540927016 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efdfc1a2040 con 0x7efdfc104ad0 2024-03-28T18:04:56.456 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:56.456+0000 7efe04400640 1 -- 172.21.15.22:0/2540927016 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efdfc10ac70 con 0x7efdfc104ad0 2024-03-28T18:04:56.458 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:56.459+0000 7efdfa000640 1 -- 172.21.15.22:0/2540927016 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 16) v1 ==== 137290+0+0 (secure 0 0 0) 0x7efde80041d0 con 0x7efdfc104ad0 2024-03-28T18:04:56.461 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:56.461+0000 7efdfa000640 1 --2- 172.21.15.22:0/2540927016 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7efde00a2880 0x7efde00a4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:04:56.461 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:56.461+0000 7efdfa000640 1 -- 172.21.15.22:0/2540927016 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 7979+0+0 (secure 0 0 0) 0x7efde80ce480 con 0x7efdfc104ad0 2024-03-28T18:04:56.461 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:56.461+0000 7efe01a00640 1 --2- 172.21.15.22:0/2540927016 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7efde00a2880 0x7efde00a4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:04:56.461 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:56.462+0000 7efe01a00640 1 --2- 172.21.15.22:0/2540927016 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7efde00a2880 0x7efde00a4d40 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7efdec02c190 tx=0x7efdec02c840 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:04:56.465 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:56.465+0000 7efdfa000640 1 -- 172.21.15.22:0/2540927016 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7efde809b5d0 con 0x7efdfc104ad0 2024-03-28T18:04:56.709 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:56.709+0000 7efe04400640 1 -- 172.21.15.22:0/2540927016 --> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}) v1 -- 0x7efdfc061ae0 con 0x7efde00a2880 2024-03-28T18:04:56.722 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:56.722+0000 7efdfa000640 1 -- 172.21.15.22:0/2540927016 <== mgr.14152 v2:172.21.15.22:6800/4201048392 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+54 (secure 0 0 0) 0x7efdfc061ae0 con 0x7efde00a2880 2024-03-28T18:04:56.722 INFO:teuthology.orchestra.run.smithi022.stdout:/volumes/qa/sv_0/abc125a7-3abc-4b82-bcad-de8d02f4df3d 2024-03-28T18:04:56.725 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:56.725+0000 7efe04400640 1 -- 172.21.15.22:0/2540927016 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7efde00a2880 msgr2=0x7efde00a4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:04:56.725 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:56.725+0000 7efe04400640 1 --2- 172.21.15.22:0/2540927016 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7efde00a2880 0x7efde00a4d40 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7efdec02c190 tx=0x7efdec02c840 comp rx=0 tx=0).stop 2024-03-28T18:04:56.725 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:56.725+0000 7efe04400640 1 -- 172.21.15.22:0/2540927016 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7efdfc104ad0 msgr2=0x7efdfc19a1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:04:56.725 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:56.725+0000 7efe04400640 1 --2- 172.21.15.22:0/2540927016 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7efdfc104ad0 0x7efdfc19a1b0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7efde800d950 tx=0x7efde800de10 comp rx=0 tx=0).stop 2024-03-28T18:04:56.725 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:56.725+0000 7efe04400640 1 -- 172.21.15.22:0/2540927016 shutdown_connections 2024-03-28T18:04:56.725 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:56.725+0000 7efe04400640 1 --2- 172.21.15.22:0/2540927016 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7efde00a2880 0x7efde00a4d40 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:04:56.725 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:56.725+0000 7efe04400640 1 --2- 172.21.15.22:0/2540927016 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7efdfc105490 0x7efdfc1a11e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:04:56.725 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:56.725+0000 7efe04400640 1 --2- 172.21.15.22:0/2540927016 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7efdfc104ad0 0x7efdfc19a1b0 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:04:56.725 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:56.725+0000 7efe04400640 1 --2- 172.21.15.22:0/2540927016 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7efdfc1038e0 0x7efdfc199c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:04:56.725 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:56.725+0000 7efe04400640 1 -- 172.21.15.22:0/2540927016 >> 172.21.15.22:0/2540927016 conn(0x7efdfc0fee50 msgr2=0x7efdfc107e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:04:56.725 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:56.725+0000 7efe04400640 1 -- 172.21.15.22:0/2540927016 shutdown_connections 2024-03-28T18:04:56.726 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:56.725+0000 7efe04400640 1 -- 172.21.15.22:0/2540927016 wait complete. 2024-03-28T18:04:56.743 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume create cephfs sv_1 --group_name qa 2024-03-28T18:04:56.925 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:04:56 smithi134 ceph-mon[35037]: mds.? [v2:172.21.15.79:6832/856896997,v1:172.21.15.79:6833/856896997] up:active 2024-03-28T18:04:56.925 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:04:56 smithi134 ceph-mon[35037]: daemon mds.i assigned to filesystem cephfs as rank 4 (now has 5 ranks) 2024-03-28T18:04:56.925 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:04:56 smithi134 ceph-mon[35037]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-03-28T18:04:56.925 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:04:56 smithi134 ceph-mon[35037]: Cluster is now healthy 2024-03-28T18:04:56.925 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:04:56 smithi134 ceph-mon[35037]: fsmap cephfs:4 {0=b=up:active,1=h=up:active,2=e=up:active,3=k=up:active} 8 up:standby 2024-03-28T18:04:56.925 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:04:56 smithi134 ceph-mon[35037]: fsmap cephfs:5 1 up:creating 7 up:standby 4 up:active 2024-03-28T18:04:56.925 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:04:56 smithi134 ceph-mon[35037]: daemon mds.i is now active in filesystem cephfs as rank 4 2024-03-28T18:04:56.925 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:04:56 smithi134 ceph-mon[35037]: from='client.34637 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-03-28T18:04:56.933 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:04:56 smithi079 ceph-mon[35312]: mds.? [v2:172.21.15.79:6832/856896997,v1:172.21.15.79:6833/856896997] up:active 2024-03-28T18:04:56.933 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:04:56 smithi079 ceph-mon[35312]: daemon mds.i assigned to filesystem cephfs as rank 4 (now has 5 ranks) 2024-03-28T18:04:56.933 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:04:56 smithi079 ceph-mon[35312]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-03-28T18:04:56.933 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:04:56 smithi079 ceph-mon[35312]: Cluster is now healthy 2024-03-28T18:04:56.933 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:04:56 smithi079 ceph-mon[35312]: fsmap cephfs:4 {0=b=up:active,1=h=up:active,2=e=up:active,3=k=up:active} 8 up:standby 2024-03-28T18:04:56.933 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:04:56 smithi079 ceph-mon[35312]: fsmap cephfs:5 1 up:creating 7 up:standby 4 up:active 2024-03-28T18:04:56.933 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:04:56 smithi079 ceph-mon[35312]: daemon mds.i is now active in filesystem cephfs as rank 4 2024-03-28T18:04:56.933 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:04:56 smithi079 ceph-mon[35312]: from='client.34637 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-03-28T18:04:56.961 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:56.961+0000 7f0746400640 1 -- 172.21.15.22:0/4004106175 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f07400756e0 msgr2=0x7f0740073b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:04:56.961 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:56.961+0000 7f0746400640 1 --2- 172.21.15.22:0/4004106175 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f07400756e0 0x7f0740073b30 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f0730009900 tx=0x7f073002bc30 comp rx=0 tx=0).stop 2024-03-28T18:04:56.961 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:56.961+0000 7f0746400640 1 -- 172.21.15.22:0/4004106175 shutdown_connections 2024-03-28T18:04:56.961 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:56.961+0000 7f0746400640 1 --2- 172.21.15.22:0/4004106175 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f0740109460 0x7f074010b8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:04:56.961 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:56.961+0000 7f0746400640 1 --2- 172.21.15.22:0/4004106175 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0740074100 0x7f0740074580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:04:56.961 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:56.961+0000 7f0746400640 1 --2- 172.21.15.22:0/4004106175 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f07400756e0 0x7f0740073b30 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:04:56.961 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:56.961+0000 7f0746400640 1 -- 172.21.15.22:0/4004106175 >> 172.21.15.22:0/4004106175 conn(0x7f07400fcb10 msgr2=0x7f07400fef50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:04:56.961 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:56.961+0000 7f0746400640 1 -- 172.21.15.22:0/4004106175 shutdown_connections 2024-03-28T18:04:56.961 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:56.961+0000 7f0746400640 1 -- 172.21.15.22:0/4004106175 wait complete. 2024-03-28T18:04:56.962 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:56.962+0000 7f0746400640 1 Processor -- start 2024-03-28T18:04:56.964 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:56.963+0000 7f0746400640 1 -- start start 2024-03-28T18:04:56.964 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:56.963+0000 7f0746400640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0740074100 0x7f0740199ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:04:56.964 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:56.963+0000 7f0746400640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f07400756e0 0x7f0740199ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:04:56.964 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:56.963+0000 7f0746400640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f0740109460 0x7f07401a1060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:04:56.964 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:56.963+0000 7f0746400640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f074010f3f0 con 0x7f07400756e0 2024-03-28T18:04:56.964 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:56.963+0000 7f0746400640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f074010f270 con 0x7f0740109460 2024-03-28T18:04:56.964 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:56.963+0000 7f0746400640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f074010f570 con 0x7f0740074100 2024-03-28T18:04:56.964 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:56.963+0000 7f073fe00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f07400756e0 0x7f0740199ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:04:56.964 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:56.963+0000 7f073fe00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f07400756e0 0x7f0740199ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:49808/0 (socket says 172.21.15.22:49808) 2024-03-28T18:04:56.964 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:56.963+0000 7f073fe00640 1 -- 172.21.15.22:0/2426478855 learned_addr learned my addr 172.21.15.22:0/2426478855 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-28T18:04:56.964 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:56.964+0000 7f0745a00640 1 --2- 172.21.15.22:0/2426478855 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f0740109460 0x7f07401a1060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:04:56.965 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:56.964+0000 7f0745000640 1 --2- 172.21.15.22:0/2426478855 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0740074100 0x7f0740199ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:04:56.965 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:56.964+0000 7f073fe00640 1 -- 172.21.15.22:0/2426478855 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0740074100 msgr2=0x7f0740199ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:04:56.965 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:56.964+0000 7f073fe00640 1 --2- 172.21.15.22:0/2426478855 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0740074100 0x7f0740199ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:04:56.965 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:56.964+0000 7f073fe00640 1 -- 172.21.15.22:0/2426478855 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f0740109460 msgr2=0x7f07401a1060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:04:56.965 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:56.964+0000 7f073fe00640 1 --2- 172.21.15.22:0/2426478855 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f0740109460 0x7f07401a1060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:04:56.965 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:56.964+0000 7f073fe00640 1 -- 172.21.15.22:0/2426478855 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f07401a16d0 con 0x7f07400756e0 2024-03-28T18:04:56.965 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:56.964+0000 7f0745000640 1 --2- 172.21.15.22:0/2426478855 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0740074100 0x7f0740199ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-28T18:04:56.965 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:56.964+0000 7f073fe00640 1 --2- 172.21.15.22:0/2426478855 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f07400756e0 0x7f0740199ff0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f072800d770 tx=0x7f072800dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:04:56.966 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:56.964+0000 7f073d600640 1 -- 172.21.15.22:0/2426478855 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0728017070 con 0x7f07400756e0 2024-03-28T18:04:56.966 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:56.965+0000 7f073d600640 1 -- 172.21.15.22:0/2426478855 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f07280054f0 con 0x7f07400756e0 2024-03-28T18:04:56.966 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:56.965+0000 7f073d600640 1 -- 172.21.15.22:0/2426478855 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0728004d60 con 0x7f07400756e0 2024-03-28T18:04:56.966 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:56.965+0000 7f0746400640 1 -- 172.21.15.22:0/2426478855 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f07401a19c0 con 0x7f07400756e0 2024-03-28T18:04:56.966 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:56.965+0000 7f0746400640 1 -- 172.21.15.22:0/2426478855 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f07401a1ec0 con 0x7f07400756e0 2024-03-28T18:04:56.966 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:56.966+0000 7f0746400640 1 -- 172.21.15.22:0/2426478855 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0740108190 con 0x7f07400756e0 2024-03-28T18:04:56.967 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:56.968+0000 7f073d600640 1 -- 172.21.15.22:0/2426478855 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 16) v1 ==== 137290+0+0 (secure 0 0 0) 0x7f0728004f00 con 0x7f07400756e0 2024-03-28T18:04:56.969 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:56.969+0000 7f073d600640 1 --2- 172.21.15.22:0/2426478855 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7f07240a2880 0x7f07240a4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:04:56.969 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:56.969+0000 7f073d600640 1 -- 172.21.15.22:0/2426478855 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f07280ce3c0 con 0x7f07400756e0 2024-03-28T18:04:56.969 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:56.970+0000 7f0745000640 1 --2- 172.21.15.22:0/2426478855 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7f07240a2880 0x7f07240a4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:04:56.970 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:56.970+0000 7f0745000640 1 --2- 172.21.15.22:0/2426478855 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7f07240a2880 0x7f07240a4d40 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f073002c880 tx=0x7f0730000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:04:56.974 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:56.974+0000 7f073d600640 1 -- 172.21.15.22:0/2426478855 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f072809b510 con 0x7f07400756e0 2024-03-28T18:04:57.031 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:04:56 smithi022 ceph-mon[31985]: mds.? [v2:172.21.15.79:6832/856896997,v1:172.21.15.79:6833/856896997] up:active 2024-03-28T18:04:57.032 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:04:56 smithi022 ceph-mon[31985]: daemon mds.i assigned to filesystem cephfs as rank 4 (now has 5 ranks) 2024-03-28T18:04:57.032 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:04:56 smithi022 ceph-mon[31985]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-03-28T18:04:57.032 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:04:56 smithi022 ceph-mon[31985]: Cluster is now healthy 2024-03-28T18:04:57.032 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:04:56 smithi022 ceph-mon[31985]: fsmap cephfs:4 {0=b=up:active,1=h=up:active,2=e=up:active,3=k=up:active} 8 up:standby 2024-03-28T18:04:57.032 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:04:56 smithi022 ceph-mon[31985]: fsmap cephfs:5 1 up:creating 7 up:standby 4 up:active 2024-03-28T18:04:57.032 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:04:56 smithi022 ceph-mon[31985]: daemon mds.i is now active in filesystem cephfs as rank 4 2024-03-28T18:04:57.032 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:04:56 smithi022 ceph-mon[31985]: from='client.34637 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-03-28T18:04:57.209 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:57.209+0000 7f0746400640 1 -- 172.21.15.22:0/2426478855 --> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}) v1 -- 0x7f0740061ae0 con 0x7f07240a2880 2024-03-28T18:04:57.261 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:57.261+0000 7f073d600640 1 -- 172.21.15.22:0/2426478855 <== mgr.14152 v2:172.21.15.22:6800/4201048392 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f0740061ae0 con 0x7f07240a2880 2024-03-28T18:04:57.264 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:57.264+0000 7f0746400640 1 -- 172.21.15.22:0/2426478855 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7f07240a2880 msgr2=0x7f07240a4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:04:57.265 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:57.264+0000 7f0746400640 1 --2- 172.21.15.22:0/2426478855 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7f07240a2880 0x7f07240a4d40 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f073002c880 tx=0x7f0730000bc0 comp rx=0 tx=0).stop 2024-03-28T18:04:57.265 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:57.264+0000 7f0746400640 1 -- 172.21.15.22:0/2426478855 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f07400756e0 msgr2=0x7f0740199ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:04:57.265 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:57.264+0000 7f0746400640 1 --2- 172.21.15.22:0/2426478855 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f07400756e0 0x7f0740199ff0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f072800d770 tx=0x7f072800dc30 comp rx=0 tx=0).stop 2024-03-28T18:04:57.265 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:57.264+0000 7f0746400640 1 -- 172.21.15.22:0/2426478855 shutdown_connections 2024-03-28T18:04:57.265 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:57.264+0000 7f0746400640 1 --2- 172.21.15.22:0/2426478855 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7f07240a2880 0x7f07240a4d40 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:04:57.265 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:57.264+0000 7f0746400640 1 --2- 172.21.15.22:0/2426478855 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f0740109460 0x7f07401a1060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:04:57.265 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:57.264+0000 7f0746400640 1 --2- 172.21.15.22:0/2426478855 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f07400756e0 0x7f0740199ff0 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:04:57.265 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:57.264+0000 7f0746400640 1 --2- 172.21.15.22:0/2426478855 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0740074100 0x7f0740199ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:04:57.265 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:57.264+0000 7f0746400640 1 -- 172.21.15.22:0/2426478855 >> 172.21.15.22:0/2426478855 conn(0x7f07400fcb10 msgr2=0x7f0740105360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:04:57.265 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:57.265+0000 7f0746400640 1 -- 172.21.15.22:0/2426478855 shutdown_connections 2024-03-28T18:04:57.265 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:57.265+0000 7f0746400640 1 -- 172.21.15.22:0/2426478855 wait complete. 2024-03-28T18:04:57.285 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs --group_name qa sv_1 2024-03-28T18:04:57.519 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:57.518+0000 7f7a7d200640 1 -- 172.21.15.22:0/786347157 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f7a781037a0 msgr2=0x7f7a78103bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:04:57.519 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:57.518+0000 7f7a7d200640 1 --2- 172.21.15.22:0/786347157 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f7a781037a0 0x7f7a78103bc0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f7a68009900 tx=0x7f7a6802bc50 comp rx=0 tx=0).stop 2024-03-28T18:04:57.519 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:57.519+0000 7f7a7d200640 1 -- 172.21.15.22:0/786347157 shutdown_connections 2024-03-28T18:04:57.519 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:57.519+0000 7f7a7d200640 1 --2- 172.21.15.22:0/786347157 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7a78105350 0x7f7a7810bbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:04:57.519 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:57.519+0000 7f7a7d200640 1 --2- 172.21.15.22:0/786347157 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7a78104990 0x7f7a78104e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:04:57.519 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:57.519+0000 7f7a7d200640 1 --2- 172.21.15.22:0/786347157 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f7a781037a0 0x7f7a78103bc0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:04:57.519 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:57.519+0000 7f7a7d200640 1 -- 172.21.15.22:0/786347157 >> 172.21.15.22:0/786347157 conn(0x7f7a780fed10 msgr2=0x7f7a78101150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:04:57.519 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:57.519+0000 7f7a7d200640 1 -- 172.21.15.22:0/786347157 shutdown_connections 2024-03-28T18:04:57.519 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:57.519+0000 7f7a7d200640 1 -- 172.21.15.22:0/786347157 wait complete. 2024-03-28T18:04:57.520 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:57.520+0000 7f7a7d200640 1 Processor -- start 2024-03-28T18:04:57.520 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:57.520+0000 7f7a7d200640 1 -- start start 2024-03-28T18:04:57.521 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:57.520+0000 7f7a7d200640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f7a781037a0 0x7f7a78199af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:04:57.521 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:57.520+0000 7f7a7d200640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7a78104990 0x7f7a7819a030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:04:57.521 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:57.520+0000 7f7a7d200640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7a78105350 0x7f7a781a10a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:04:57.521 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:57.520+0000 7f7a7d200640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a7810f3f0 con 0x7f7a78104990 2024-03-28T18:04:57.521 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:57.520+0000 7f7a7d200640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a7810f270 con 0x7f7a781037a0 2024-03-28T18:04:57.521 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:57.520+0000 7f7a7d200640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a7810f570 con 0x7f7a78105350 2024-03-28T18:04:57.521 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:57.521+0000 7f7a6fe00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7a78104990 0x7f7a7819a030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:04:57.521 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:57.521+0000 7f7a77400640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f7a781037a0 0x7f7a78199af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:04:57.521 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:57.521+0000 7f7a77400640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f7a781037a0 0x7f7a78199af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.22:37292/0 (socket says 172.21.15.22:37292) 2024-03-28T18:04:57.521 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:57.521+0000 7f7a77400640 1 -- 172.21.15.22:0/882449852 learned_addr learned my addr 172.21.15.22:0/882449852 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-28T18:04:57.521 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:57.521+0000 7f7a6fe00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7a78104990 0x7f7a7819a030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:47450/0 (socket says 172.21.15.22:47450) 2024-03-28T18:04:57.522 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:57.521+0000 7f7a77e00640 1 --2- 172.21.15.22:0/882449852 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7a78105350 0x7f7a781a10a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:04:57.522 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:57.521+0000 7f7a6fe00640 1 -- 172.21.15.22:0/882449852 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7a78105350 msgr2=0x7f7a781a10a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:04:57.523 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:57.521+0000 7f7a6fe00640 1 --2- 172.21.15.22:0/882449852 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7a78105350 0x7f7a781a10a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:04:57.523 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:57.521+0000 7f7a6fe00640 1 -- 172.21.15.22:0/882449852 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f7a781037a0 msgr2=0x7f7a78199af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:04:57.523 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:57.521+0000 7f7a6fe00640 1 --2- 172.21.15.22:0/882449852 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f7a781037a0 0x7f7a78199af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:04:57.523 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:57.521+0000 7f7a6fe00640 1 -- 172.21.15.22:0/882449852 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a781a1710 con 0x7f7a78104990 2024-03-28T18:04:57.523 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:57.522+0000 7f7a6fe00640 1 --2- 172.21.15.22:0/882449852 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7a78104990 0x7f7a7819a030 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f7a6000d6d0 tx=0x7f7a6000db90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:04:57.523 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:57.522+0000 7f7a74c00640 1 -- 172.21.15.22:0/882449852 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a60017070 con 0x7f7a78104990 2024-03-28T18:04:57.523 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:57.522+0000 7f7a7d200640 1 -- 172.21.15.22:0/882449852 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7a781a1a00 con 0x7f7a78104990 2024-03-28T18:04:57.523 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:57.522+0000 7f7a74c00640 1 -- 172.21.15.22:0/882449852 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f7a60009820 con 0x7f7a78104990 2024-03-28T18:04:57.523 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:57.522+0000 7f7a74c00640 1 -- 172.21.15.22:0/882449852 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a60004cd0 con 0x7f7a78104990 2024-03-28T18:04:57.523 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:57.522+0000 7f7a7d200640 1 -- 172.21.15.22:0/882449852 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7a781a1fa0 con 0x7f7a78104990 2024-03-28T18:04:57.524 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:57.524+0000 7f7a7d200640 1 -- 172.21.15.22:0/882449852 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7a7810ab30 con 0x7f7a78104990 2024-03-28T18:04:57.525 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:57.526+0000 7f7a74c00640 1 -- 172.21.15.22:0/882449852 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 16) v1 ==== 137290+0+0 (secure 0 0 0) 0x7f7a600099d0 con 0x7f7a78104990 2024-03-28T18:04:57.527 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:57.527+0000 7f7a74c00640 1 --2- 172.21.15.22:0/882449852 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7f7a5c0a2900 0x7f7a5c0a4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:04:57.527 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:57.527+0000 7f7a74c00640 1 -- 172.21.15.22:0/882449852 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f7a600ce420 con 0x7f7a78104990 2024-03-28T18:04:57.527 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:57.528+0000 7f7a77400640 1 --2- 172.21.15.22:0/882449852 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7f7a5c0a2900 0x7f7a5c0a4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:04:57.528 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:57.528+0000 7f7a77400640 1 --2- 172.21.15.22:0/882449852 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7f7a5c0a2900 0x7f7a5c0a4dc0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f7a680096c0 tx=0x7f7a6802c630 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:04:57.532 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:57.532+0000 7f7a74c00640 1 -- 172.21.15.22:0/882449852 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f7a6009b570 con 0x7f7a78104990 2024-03-28T18:04:57.681 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:57.681+0000 7f7a74c00640 1 -- 172.21.15.22:0/882449852 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mgrmap(e 17) v1 ==== 137341+0+0 (secure 0 0 0) 0x7f7a600974e0 con 0x7f7a78104990 2024-03-28T18:04:57.779 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:57.779+0000 7f7a7d200640 1 -- 172.21.15.22:0/882449852 --> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}) v1 -- 0x7f7a78061ae0 con 0x7f7a5c0a2900 2024-03-28T18:04:57.787 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:57.787+0000 7f7a74c00640 1 -- 172.21.15.22:0/882449852 <== mgr.14152 v2:172.21.15.22:6800/4201048392 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+54 (secure 0 0 0) 0x7f7a78061ae0 con 0x7f7a5c0a2900 2024-03-28T18:04:57.787 INFO:teuthology.orchestra.run.smithi022.stdout:/volumes/qa/sv_1/341c2f40-685c-41af-a8d0-b6c91be412ba 2024-03-28T18:04:57.790 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:57.789+0000 7f7a7d200640 1 -- 172.21.15.22:0/882449852 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7f7a5c0a2900 msgr2=0x7f7a5c0a4dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:04:57.790 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:57.789+0000 7f7a7d200640 1 --2- 172.21.15.22:0/882449852 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7f7a5c0a2900 0x7f7a5c0a4dc0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f7a680096c0 tx=0x7f7a6802c630 comp rx=0 tx=0).stop 2024-03-28T18:04:57.790 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:57.789+0000 7f7a7d200640 1 -- 172.21.15.22:0/882449852 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7a78104990 msgr2=0x7f7a7819a030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:04:57.790 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:57.789+0000 7f7a7d200640 1 --2- 172.21.15.22:0/882449852 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7a78104990 0x7f7a7819a030 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f7a6000d6d0 tx=0x7f7a6000db90 comp rx=0 tx=0).stop 2024-03-28T18:04:57.790 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:57.790+0000 7f7a7d200640 1 -- 172.21.15.22:0/882449852 shutdown_connections 2024-03-28T18:04:57.790 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:57.790+0000 7f7a7d200640 1 --2- 172.21.15.22:0/882449852 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7f7a5c0a2900 0x7f7a5c0a4dc0 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:04:57.790 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:57.790+0000 7f7a7d200640 1 --2- 172.21.15.22:0/882449852 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7a78105350 0x7f7a781a10a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:04:57.790 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:57.790+0000 7f7a7d200640 1 --2- 172.21.15.22:0/882449852 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7a78104990 0x7f7a7819a030 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:04:57.790 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:57.790+0000 7f7a7d200640 1 --2- 172.21.15.22:0/882449852 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f7a781037a0 0x7f7a78199af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:04:57.790 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:57.790+0000 7f7a7d200640 1 -- 172.21.15.22:0/882449852 >> 172.21.15.22:0/882449852 conn(0x7f7a780fed10 msgr2=0x7f7a78107d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:04:57.790 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:57.790+0000 7f7a7d200640 1 -- 172.21.15.22:0/882449852 shutdown_connections 2024-03-28T18:04:57.790 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:57.790+0000 7f7a7d200640 1 -- 172.21.15.22:0/882449852 wait complete. 2024-03-28T18:04:57.807 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-28T18:04:57.894 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:04:57 smithi022 ceph-mon[31985]: pgmap v373: 97 pgs: 97 active+clean; 583 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s wr, 18 op/s 2024-03-28T18:04:57.894 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:04:57 smithi022 ceph-mon[31985]: mds.? [v2:172.21.15.134:6838/3400263217,v1:172.21.15.134:6839/3400263217] up:active 2024-03-28T18:04:57.894 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:04:57 smithi022 ceph-mon[31985]: fsmap cephfs:5 7 up:standby 5 up:active 2024-03-28T18:04:57.894 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:04:57 smithi022 ceph-mon[31985]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-28T18:04:57.894 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:04:57 smithi022 ceph-mon[31985]: from='client.14766 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-03-28T18:04:57.894 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:04:57 smithi022 ceph-mon[31985]: from='client.14772 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-03-28T18:04:57.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:04:57 smithi134 ceph-mon[35037]: pgmap v373: 97 pgs: 97 active+clean; 583 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s wr, 18 op/s 2024-03-28T18:04:57.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:04:57 smithi134 ceph-mon[35037]: mds.? [v2:172.21.15.134:6838/3400263217,v1:172.21.15.134:6839/3400263217] up:active 2024-03-28T18:04:57.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:04:57 smithi134 ceph-mon[35037]: fsmap cephfs:5 7 up:standby 5 up:active 2024-03-28T18:04:57.925 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:04:57 smithi134 ceph-mon[35037]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-28T18:04:57.925 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:04:57 smithi134 ceph-mon[35037]: from='client.14766 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-03-28T18:04:57.925 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:04:57 smithi134 ceph-mon[35037]: from='client.14772 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-03-28T18:04:57.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:04:57 smithi079 ceph-mon[35312]: pgmap v373: 97 pgs: 97 active+clean; 583 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s wr, 18 op/s 2024-03-28T18:04:57.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:04:57 smithi079 ceph-mon[35312]: mds.? [v2:172.21.15.134:6838/3400263217,v1:172.21.15.134:6839/3400263217] up:active 2024-03-28T18:04:57.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:04:57 smithi079 ceph-mon[35312]: fsmap cephfs:5 7 up:standby 5 up:active 2024-03-28T18:04:57.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:04:57 smithi079 ceph-mon[35312]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-28T18:04:57.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:04:57 smithi079 ceph-mon[35312]: from='client.14766 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-03-28T18:04:57.933 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:04:57 smithi079 ceph-mon[35312]: from='client.14772 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-03-28T18:04:58.029 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:58.029+0000 7f1f8d200640 1 -- 172.21.15.22:0/2241786740 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1f881026f0 msgr2=0x7f1f88102b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:04:58.029 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:58.029+0000 7f1f8d200640 1 --2- 172.21.15.22:0/2241786740 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1f881026f0 0x7f1f88102b70 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f1f70009900 tx=0x7f1f7002bc40 comp rx=0 tx=0).stop 2024-03-28T18:04:58.029 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:58.029+0000 7f1f8d200640 1 -- 172.21.15.22:0/2241786740 shutdown_connections 2024-03-28T18:04:58.029 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:58.029+0000 7f1f8d200640 1 --2- 172.21.15.22:0/2241786740 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1f881030b0 0x7f1f88109940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:04:58.029 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:58.029+0000 7f1f8d200640 1 --2- 172.21.15.22:0/2241786740 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1f881026f0 0x7f1f88102b70 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:04:58.029 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:58.029+0000 7f1f8d200640 1 --2- 172.21.15.22:0/2241786740 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f1f88101500 0x7f1f88101920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:04:58.029 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:58.029+0000 7f1f8d200640 1 -- 172.21.15.22:0/2241786740 >> 172.21.15.22:0/2241786740 conn(0x7f1f880fca70 msgr2=0x7f1f880feeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:04:58.030 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:58.029+0000 7f1f8d200640 1 -- 172.21.15.22:0/2241786740 shutdown_connections 2024-03-28T18:04:58.030 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:58.029+0000 7f1f8d200640 1 -- 172.21.15.22:0/2241786740 wait complete. 2024-03-28T18:04:58.032 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:58.030+0000 7f1f8d200640 1 Processor -- start 2024-03-28T18:04:58.032 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:58.030+0000 7f1f8d200640 1 -- start start 2024-03-28T18:04:58.032 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:58.031+0000 7f1f8d200640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1f88101500 0x7f1f88075450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:04:58.032 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:58.031+0000 7f1f8d200640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f1f881030b0 0x7f1f88073a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:04:58.032 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:58.031+0000 7f1f8d200640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1f88073fd0 0x7f1f88074480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:04:58.032 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:58.031+0000 7f1f8d200640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f8810d1b0 con 0x7f1f88073fd0 2024-03-28T18:04:58.033 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:58.031+0000 7f1f8d200640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f8810d030 con 0x7f1f881030b0 2024-03-28T18:04:58.033 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:58.031+0000 7f1f8d200640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f8810d330 con 0x7f1f88101500 2024-03-28T18:04:58.033 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:58.031+0000 7f1f87e00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1f88073fd0 0x7f1f88074480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:04:58.033 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:58.031+0000 7f1f87e00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1f88073fd0 0x7f1f88074480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:47472/0 (socket says 172.21.15.22:47472) 2024-03-28T18:04:58.033 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:58.031+0000 7f1f87e00640 1 -- 172.21.15.22:0/1366171867 learned_addr learned my addr 172.21.15.22:0/1366171867 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-28T18:04:58.033 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:58.031+0000 7f1f7fe00640 1 --2- 172.21.15.22:0/1366171867 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f1f881030b0 0x7f1f88073a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:04:58.033 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:58.031+0000 7f1f87400640 1 --2- 172.21.15.22:0/1366171867 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1f88101500 0x7f1f88075450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:04:58.033 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:58.031+0000 7f1f87e00640 1 -- 172.21.15.22:0/1366171867 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1f88101500 msgr2=0x7f1f88075450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:04:58.033 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:58.031+0000 7f1f87e00640 1 --2- 172.21.15.22:0/1366171867 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1f88101500 0x7f1f88075450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:04:58.033 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:58.031+0000 7f1f87e00640 1 -- 172.21.15.22:0/1366171867 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f1f881030b0 msgr2=0x7f1f88073a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:04:58.033 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:58.031+0000 7f1f87e00640 1 --2- 172.21.15.22:0/1366171867 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f1f881030b0 0x7f1f88073a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:04:58.033 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:58.031+0000 7f1f87e00640 1 -- 172.21.15.22:0/1366171867 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f881a1be0 con 0x7f1f88073fd0 2024-03-28T18:04:58.033 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:58.031+0000 7f1f87400640 1 --2- 172.21.15.22:0/1366171867 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1f88101500 0x7f1f88075450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T18:04:58.034 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:58.032+0000 7f1f87e00640 1 --2- 172.21.15.22:0/1366171867 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1f88073fd0 0x7f1f88074480 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f1f8000eaa0 tx=0x7f1f8000ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:04:58.034 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:58.032+0000 7f1f84c00640 1 -- 172.21.15.22:0/1366171867 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f8000cdb0 con 0x7f1f88073fd0 2024-03-28T18:04:58.034 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:58.032+0000 7f1f84c00640 1 -- 172.21.15.22:0/1366171867 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f1f80005020 con 0x7f1f88073fd0 2024-03-28T18:04:58.034 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:58.032+0000 7f1f8d200640 1 -- 172.21.15.22:0/1366171867 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1f881a1ed0 con 0x7f1f88073fd0 2024-03-28T18:04:58.034 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:58.032+0000 7f1f84c00640 1 -- 172.21.15.22:0/1366171867 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f800116b0 con 0x7f1f88073fd0 2024-03-28T18:04:58.034 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:58.032+0000 7f1f8d200640 1 -- 172.21.15.22:0/1366171867 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1f881a2390 con 0x7f1f88073fd0 2024-03-28T18:04:58.034 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:58.033+0000 7f1f8d200640 1 -- 172.21.15.22:0/1366171867 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1f88108820 con 0x7f1f88073fd0 2024-03-28T18:04:58.035 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:58.035+0000 7f1f84c00640 1 -- 172.21.15.22:0/1366171867 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 17) v1 ==== 137341+0+0 (secure 0 0 0) 0x7f1f800051d0 con 0x7f1f88073fd0 2024-03-28T18:04:58.036 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:58.037+0000 7f1f84c00640 1 --2- 172.21.15.22:0/1366171867 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7f1f6c0a2890 0x7f1f6c0a4d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:04:58.037 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:58.037+0000 7f1f84c00640 1 -- 172.21.15.22:0/1366171867 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f1f800cecb0 con 0x7f1f88073fd0 2024-03-28T18:04:58.037 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:58.037+0000 7f1f87400640 1 --2- 172.21.15.22:0/1366171867 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7f1f6c0a2890 0x7f1f6c0a4d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:04:58.037 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:58.038+0000 7f1f87400640 1 --2- 172.21.15.22:0/1366171867 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7f1f6c0a2890 0x7f1f6c0a4d50 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f1f78005d00 tx=0x7f1f78005ef0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:04:58.041 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:58.042+0000 7f1f84c00640 1 -- 172.21.15.22:0/1366171867 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f1f80016030 con 0x7f1f88073fd0 2024-03-28T18:04:58.287 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:58.287+0000 7f1f8d200640 1 -- 172.21.15.22:0/1366171867 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f1f8802d270 con 0x7f1f88073fd0 2024-03-28T18:04:58.288 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:58.289+0000 7f1f84c00640 1 -- 172.21.15.22:0/1366171867 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) v1 ==== 94+0+11452 (secure 0 0 0) 0x7f1f8009bda0 con 0x7f1f88073fd0 2024-03-28T18:04:58.289 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-03-28T18:04:58.289 INFO:teuthology.orchestra.run.smithi022.stdout:{"epoch":26,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14544,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6835/3718238950","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6834","nonce":3718238950},{"type":"v1","addr":"172.21.15.22:6835","nonce":3718238950}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14562,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6837/3676939386","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6836","nonce":3676939386},{"type":"v1","addr":"172.21.15.22:6837","nonce":3676939386}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":26,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-28T18:04:48.991622+0000","modified":"2024-03-28T18:04:56.643864+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":34553,"mds_1":34511,"mds_2":34475,"mds_3":34442,"mds_4":24502},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14640":{"gid":14640,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.22:6839/3979998310","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6838","nonce":3979998310},{"type":"v1","addr":"172.21.15.22:6839","nonce":3979998310}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_14655":{"gid":14655,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.22:6841/3069056661","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6840","nonce":3069056661},{"type":"v1","addr":"172.21.15.22:6841","nonce":3069056661}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24412":{"gid":24412,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.134:6833/3176125362","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6832","nonce":3176125362},{"type":"v1","addr":"172.21.15.134:6833","nonce":3176125362}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24463":{"gid":24463,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.134:6835/3370549207","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6834","nonce":3370549207},{"type":"v1","addr":"172.21.15.134:6835","nonce":3370549207}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24478":{"gid":24478,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.134:6837/899825581","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6836","nonce":899825581},{"type":"v1","addr":"172.21.15.134:6837","nonce":899825581}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24502":{"gid":24502,"name":"i","rank":4,"incarnation":24,"state":"up:active","state_seq":7,"addr":"172.21.15.134:6839/3400263217","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6838","nonce":3400263217},{"type":"v1","addr":"172.21.15.134:6839","nonce":3400263217}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_34442":{"gid":34442,"name":"k","rank":3,"incarnation":22,"state":"up:active","state_seq":14,"addr":"172.21.15.79:6833/856896997","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6832","nonce":856896997},{"type":"v1","addr":"172.21.15.79:6833","nonce":856896997}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_34475":{"gid":34475,"name":"e","rank":2,"incarnation":20,"state":"up:active","state_seq":11,"addr":"172.21.15.79:6835/3365773431","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6834","nonce":3365773431},{"type":"v1","addr":"172.21.15.79:6835","nonce":3365773431}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_34511":{"gid":34511,"name":"h","rank":1,"incarnation":18,"state":"up:active","state_seq":8,"addr":"172.21.15.79:6837/1787903366","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6836","nonce":1787903366},{"type":"v1","addr":"172.21.15.79:6837","nonce":1787903366}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_34553":{"gid":34553,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.79:6839/2441542443","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6838","nonce":2441542443},{"type":"v1","addr":"172.21.15.79:6839","nonce":2441542443}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":34553,"qdb_cluster":[34511,34553,34475,34442,24502]},"id":1}]} 2024-03-28T18:04:58.289 INFO:teuthology.orchestra.run.smithi022.stderr:dumped fsmap epoch 26 2024-03-28T18:04:58.291 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:58.291+0000 7f1f8d200640 1 -- 172.21.15.22:0/1366171867 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7f1f6c0a2890 msgr2=0x7f1f6c0a4d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:04:58.291 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:58.291+0000 7f1f8d200640 1 --2- 172.21.15.22:0/1366171867 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7f1f6c0a2890 0x7f1f6c0a4d50 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f1f78005d00 tx=0x7f1f78005ef0 comp rx=0 tx=0).stop 2024-03-28T18:04:58.291 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:58.291+0000 7f1f8d200640 1 -- 172.21.15.22:0/1366171867 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1f88073fd0 msgr2=0x7f1f88074480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:04:58.291 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:58.291+0000 7f1f8d200640 1 --2- 172.21.15.22:0/1366171867 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1f88073fd0 0x7f1f88074480 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f1f8000eaa0 tx=0x7f1f8000ef60 comp rx=0 tx=0).stop 2024-03-28T18:04:58.292 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:58.292+0000 7f1f8d200640 1 -- 172.21.15.22:0/1366171867 shutdown_connections 2024-03-28T18:04:58.292 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:58.292+0000 7f1f8d200640 1 --2- 172.21.15.22:0/1366171867 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7f1f6c0a2890 0x7f1f6c0a4d50 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:04:58.292 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:58.292+0000 7f1f8d200640 1 --2- 172.21.15.22:0/1366171867 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1f88073fd0 0x7f1f88074480 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:04:58.292 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:58.292+0000 7f1f8d200640 1 --2- 172.21.15.22:0/1366171867 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f1f881030b0 0x7f1f88073a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:04:58.292 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:58.292+0000 7f1f8d200640 1 --2- 172.21.15.22:0/1366171867 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1f88101500 0x7f1f88075450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:04:58.292 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:58.292+0000 7f1f8d200640 1 -- 172.21.15.22:0/1366171867 >> 172.21.15.22:0/1366171867 conn(0x7f1f880fca70 msgr2=0x7f1f881059f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:04:58.292 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:58.292+0000 7f1f8d200640 1 -- 172.21.15.22:0/1366171867 shutdown_connections 2024-03-28T18:04:58.292 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:58.292+0000 7f1f8d200640 1 -- 172.21.15.22:0/1366171867 wait complete. 2024-03-28T18:04:58.311 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-28T18:04:58.531 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:58.531+0000 7fb405c00640 1 -- 172.21.15.22:0/525613562 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb400077c40 msgr2=0x7fb4000784e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:04:58.531 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:58.531+0000 7fb405c00640 1 --2- 172.21.15.22:0/525613562 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb400077c40 0x7fb4000784e0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fb3f8009ca0 tx=0x7fb3f802c0f0 comp rx=0 tx=0).stop 2024-03-28T18:04:58.532 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:58.531+0000 7fb405c00640 1 -- 172.21.15.22:0/525613562 shutdown_connections 2024-03-28T18:04:58.532 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:58.531+0000 7fb405c00640 1 --2- 172.21.15.22:0/525613562 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb400077c40 0x7fb4000784e0 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:04:58.532 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:58.531+0000 7fb405c00640 1 --2- 172.21.15.22:0/525613562 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fb4000772a0 0x7fb400077700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:04:58.532 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:58.531+0000 7fb405c00640 1 --2- 172.21.15.22:0/525613562 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb400076060 0x7fb400076480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:04:58.532 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:58.531+0000 7fb405c00640 1 -- 172.21.15.22:0/525613562 >> 172.21.15.22:0/525613562 conn(0x7fb4000fee50 msgr2=0x7fb400101290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:04:58.532 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:58.531+0000 7fb405c00640 1 -- 172.21.15.22:0/525613562 shutdown_connections 2024-03-28T18:04:58.532 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:58.531+0000 7fb405c00640 1 -- 172.21.15.22:0/525613562 wait complete. 2024-03-28T18:04:58.534 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:58.532+0000 7fb405c00640 1 Processor -- start 2024-03-28T18:04:58.534 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:58.533+0000 7fb405c00640 1 -- start start 2024-03-28T18:04:58.534 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:58.533+0000 7fb405c00640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fb400076060 0x7fb40019dfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:04:58.535 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:58.533+0000 7fb405c00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb4000772a0 0x7fb40019e4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:04:58.535 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:58.533+0000 7fb405c00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb400077c40 0x7fb4001a5550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:04:58.535 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:58.533+0000 7fb405c00640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb400113950 con 0x7fb4000772a0 2024-03-28T18:04:58.535 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:58.533+0000 7fb405c00640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4001137d0 con 0x7fb400076060 2024-03-28T18:04:58.535 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:58.533+0000 7fb405c00640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb400113ad0 con 0x7fb400077c40 2024-03-28T18:04:58.535 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:58.533+0000 7fb3fe000640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb4000772a0 0x7fb40019e4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:04:58.535 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:58.533+0000 7fb3fe000640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb4000772a0 0x7fb40019e4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:47492/0 (socket says 172.21.15.22:47492) 2024-03-28T18:04:58.535 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:58.533+0000 7fb3fe000640 1 -- 172.21.15.22:0/660513257 learned_addr learned my addr 172.21.15.22:0/660513257 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-28T18:04:58.535 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:58.533+0000 7fb3fea00640 1 --2- 172.21.15.22:0/660513257 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fb400076060 0x7fb40019dfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:04:58.535 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:58.534+0000 7fb3ff400640 1 --2- 172.21.15.22:0/660513257 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb400077c40 0x7fb4001a5550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:04:58.535 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:58.534+0000 7fb3fe000640 1 -- 172.21.15.22:0/660513257 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb400077c40 msgr2=0x7fb4001a5550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:04:58.535 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:58.534+0000 7fb3fe000640 1 --2- 172.21.15.22:0/660513257 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb400077c40 0x7fb4001a5550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:04:58.535 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:58.534+0000 7fb3fe000640 1 -- 172.21.15.22:0/660513257 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fb400076060 msgr2=0x7fb40019dfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:04:58.536 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:58.534+0000 7fb3fe000640 1 --2- 172.21.15.22:0/660513257 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fb400076060 0x7fb40019dfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:04:58.536 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:58.534+0000 7fb3fe000640 1 -- 172.21.15.22:0/660513257 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb4001a5bc0 con 0x7fb4000772a0 2024-03-28T18:04:58.536 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:58.534+0000 7fb3fe000640 1 --2- 172.21.15.22:0/660513257 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb4000772a0 0x7fb40019e4e0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fb3ec00e9f0 tx=0x7fb3ec00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:04:58.536 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:58.534+0000 7fb3f7400640 1 -- 172.21.15.22:0/660513257 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3ec00cd80 con 0x7fb4000772a0 2024-03-28T18:04:58.536 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:58.534+0000 7fb3f7400640 1 -- 172.21.15.22:0/660513257 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7fb3ec010900 con 0x7fb4000772a0 2024-03-28T18:04:58.536 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:58.534+0000 7fb3f7400640 1 -- 172.21.15.22:0/660513257 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3ec0189e0 con 0x7fb4000772a0 2024-03-28T18:04:58.536 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:58.534+0000 7fb405c00640 1 -- 172.21.15.22:0/660513257 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb4001a5eb0 con 0x7fb4000772a0 2024-03-28T18:04:58.536 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:58.535+0000 7fb405c00640 1 -- 172.21.15.22:0/660513257 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb4001a6320 con 0x7fb4000772a0 2024-03-28T18:04:58.536 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:58.536+0000 7fb405c00640 1 -- 172.21.15.22:0/660513257 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb400110390 con 0x7fb4000772a0 2024-03-28T18:04:58.538 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:58.538+0000 7fb3f7400640 1 -- 172.21.15.22:0/660513257 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 17) v1 ==== 137341+0+0 (secure 0 0 0) 0x7fb3ec010ab0 con 0x7fb4000772a0 2024-03-28T18:04:58.539 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:58.540+0000 7fb3f7400640 1 --2- 172.21.15.22:0/660513257 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7fb3e40a2910 0x7fb3e40a4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:04:58.540 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:58.540+0000 7fb3f7400640 1 -- 172.21.15.22:0/660513257 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 7979+0+0 (secure 0 0 0) 0x7fb3ec0cee70 con 0x7fb4000772a0 2024-03-28T18:04:58.540 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:58.540+0000 7fb3fea00640 1 --2- 172.21.15.22:0/660513257 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7fb3e40a2910 0x7fb3e40a4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:04:58.540 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:58.541+0000 7fb3fea00640 1 --2- 172.21.15.22:0/660513257 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7fb3e40a2910 0x7fb3e40a4dd0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fb3f0009bd0 tx=0x7fb3f0005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:04:58.544 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:58.545+0000 7fb3f7400640 1 -- 172.21.15.22:0/660513257 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fb3ec09be60 con 0x7fb4000772a0 2024-03-28T18:04:58.752 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:58.752+0000 7fb405c00640 1 -- 172.21.15.22:0/660513257 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fb40002d2e0 con 0x7fb4000772a0 2024-03-28T18:04:58.756 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:58.756+0000 7fb3f7400640 1 -- 172.21.15.22:0/660513257 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v90) v1 ==== 74+0+22183 (secure 0 0 0) 0x7fb3ec09b760 con 0x7fb4000772a0 2024-03-28T18:04:58.757 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-03-28T18:04:58.757 INFO:teuthology.orchestra.run.smithi022.stdout:{"epoch":90,"fsid":"fd220748-ed2b-11ee-95d0-87774f69a715","created":"2024-03-28T17:54:00.440279+0000","modified":"2024-03-28T18:04:50.509330+0000","last_up_change":"2024-03-28T18:03:43.220095+0000","last_in_change":"2024-03-28T18:03:18.526399+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-28T17:58:33.168872+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-28T18:04:47.004797+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-28T18:04:48.465502+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"90","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"5d708e06-f05b-40ba-bbcf-258ddc4f43d7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6802","nonce":2186290846},{"type":"v1","addr":"172.21.15.22:6803","nonce":2186290846}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6804","nonce":2186290846},{"type":"v1","addr":"172.21.15.22:6805","nonce":2186290846}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6808","nonce":2186290846},{"type":"v1","addr":"172.21.15.22:6809","nonce":2186290846}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6806","nonce":2186290846},{"type":"v1","addr":"172.21.15.22:6807","nonce":2186290846}]},"public_addr":"172.21.15.22:6803/2186290846","cluster_addr":"172.21.15.22:6805/2186290846","heartbeat_back_addr":"172.21.15.22:6809/2186290846","heartbeat_front_addr":"172.21.15.22:6807/2186290846","state":["exists","up"]},{"osd":1,"uuid":"68ad09bb-9256-409a-9867-4e14a2cb94aa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6800","nonce":2835549681},{"type":"v1","addr":"172.21.15.79:6801","nonce":2835549681}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6802","nonce":2835549681},{"type":"v1","addr":"172.21.15.79:6803","nonce":2835549681}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6806","nonce":2835549681},{"type":"v1","addr":"172.21.15.79:6807","nonce":2835549681}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6804","nonce":2835549681},{"type":"v1","addr":"172.21.15.79:6805","nonce":2835549681}]},"public_addr":"172.21.15.79:6801/2835549681","cluster_addr":"172.21.15.79:6803/2835549681","heartbeat_back_addr":"172.21.15.79:6807/2835549681","heartbeat_front_addr":"172.21.15.79:6805/2835549681","state":["exists","up"]},{"osd":2,"uuid":"ed11a5a7-004d-41f3-a5fc-eb2d603bd33f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6800","nonce":470588085},{"type":"v1","addr":"172.21.15.134:6801","nonce":470588085}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6802","nonce":470588085},{"type":"v1","addr":"172.21.15.134:6803","nonce":470588085}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6806","nonce":470588085},{"type":"v1","addr":"172.21.15.134:6807","nonce":470588085}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6804","nonce":470588085},{"type":"v1","addr":"172.21.15.134:6805","nonce":470588085}]},"public_addr":"172.21.15.134:6801/470588085","cluster_addr":"172.21.15.134:6803/470588085","heartbeat_back_addr":"172.21.15.134:6807/470588085","heartbeat_front_addr":"172.21.15.134:6805/470588085","state":["exists","up"]},{"osd":3,"uuid":"81a6c547-2a50-44c3-84d7-23ab6831790c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6810","nonce":1702441854},{"type":"v1","addr":"172.21.15.22:6811","nonce":1702441854}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6812","nonce":1702441854},{"type":"v1","addr":"172.21.15.22:6813","nonce":1702441854}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6816","nonce":1702441854},{"type":"v1","addr":"172.21.15.22:6817","nonce":1702441854}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6814","nonce":1702441854},{"type":"v1","addr":"172.21.15.22:6815","nonce":1702441854}]},"public_addr":"172.21.15.22:6811/1702441854","cluster_addr":"172.21.15.22:6813/1702441854","heartbeat_back_addr":"172.21.15.22:6817/1702441854","heartbeat_front_addr":"172.21.15.22:6815/1702441854","state":["exists","up"]},{"osd":4,"uuid":"d23a254e-7182-4471-8bca-7eb684ec618e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6808","nonce":630132437},{"type":"v1","addr":"172.21.15.79:6809","nonce":630132437}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6810","nonce":630132437},{"type":"v1","addr":"172.21.15.79:6811","nonce":630132437}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6814","nonce":630132437},{"type":"v1","addr":"172.21.15.79:6815","nonce":630132437}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6812","nonce":630132437},{"type":"v1","addr":"172.21.15.79:6813","nonce":630132437}]},"public_addr":"172.21.15.79:6809/630132437","cluster_addr":"172.21.15.79:6811/630132437","heartbeat_back_addr":"172.21.15.79:6815/630132437","heartbeat_front_addr":"172.21.15.79:6813/630132437","state":["exists","up"]},{"osd":5,"uuid":"a7b98502-2023-4f0a-bfa0-6840cc495e0c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":40,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6808","nonce":2135406915},{"type":"v1","addr":"172.21.15.134:6809","nonce":2135406915}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6810","nonce":2135406915},{"type":"v1","addr":"172.21.15.134:6811","nonce":2135406915}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6814","nonce":2135406915},{"type":"v1","addr":"172.21.15.134:6815","nonce":2135406915}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6812","nonce":2135406915},{"type":"v1","addr":"172.21.15.134:6813","nonce":2135406915}]},"public_addr":"172.21.15.134:6809/2135406915","cluster_addr":"172.21.15.134:6811/2135406915","heartbeat_back_addr":"172.21.15.134:6815/2135406915","heartbeat_front_addr":"172.21.15.134:6813/2135406915","state":["exists","up"]},{"osd":6,"uuid":"634c6562-d683-41c8-8242-fc848c38ce1f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6818","nonce":1400680396},{"type":"v1","addr":"172.21.15.22:6819","nonce":1400680396}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6820","nonce":1400680396},{"type":"v1","addr":"172.21.15.22:6821","nonce":1400680396}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6824","nonce":1400680396},{"type":"v1","addr":"172.21.15.22:6825","nonce":1400680396}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6822","nonce":1400680396},{"type":"v1","addr":"172.21.15.22:6823","nonce":1400680396}]},"public_addr":"172.21.15.22:6819/1400680396","cluster_addr":"172.21.15.22:6821/1400680396","heartbeat_back_addr":"172.21.15.22:6825/1400680396","heartbeat_front_addr":"172.21.15.22:6823/1400680396","state":["exists","up"]},{"osd":7,"uuid":"504eb4a8-6731-45ad-932c-1b38ef157871","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":53,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6816","nonce":1287105314},{"type":"v1","addr":"172.21.15.79:6817","nonce":1287105314}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6818","nonce":1287105314},{"type":"v1","addr":"172.21.15.79:6819","nonce":1287105314}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6822","nonce":1287105314},{"type":"v1","addr":"172.21.15.79:6823","nonce":1287105314}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6820","nonce":1287105314},{"type":"v1","addr":"172.21.15.79:6821","nonce":1287105314}]},"public_addr":"172.21.15.79:6817/1287105314","cluster_addr":"172.21.15.79:6819/1287105314","heartbeat_back_addr":"172.21.15.79:6823/1287105314","heartbeat_front_addr":"172.21.15.79:6821/1287105314","state":["exists","up"]},{"osd":8,"uuid":"20aa66c1-03d6-4cb8-a096-0eae41979c17","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":62,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6816","nonce":2265650388},{"type":"v1","addr":"172.21.15.134:6817","nonce":2265650388}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6818","nonce":2265650388},{"type":"v1","addr":"172.21.15.134:6819","nonce":2265650388}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6822","nonce":2265650388},{"type":"v1","addr":"172.21.15.134:6823","nonce":2265650388}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6820","nonce":2265650388},{"type":"v1","addr":"172.21.15.134:6821","nonce":2265650388}]},"public_addr":"172.21.15.134:6817/2265650388","cluster_addr":"172.21.15.134:6819/2265650388","heartbeat_back_addr":"172.21.15.134:6823/2265650388","heartbeat_front_addr":"172.21.15.134:6821/2265650388","state":["exists","up"]},{"osd":9,"uuid":"e1ae4020-7c2c-4334-a753-ee6b12d98584","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":70,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6826","nonce":3762222259},{"type":"v1","addr":"172.21.15.22:6827","nonce":3762222259}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6828","nonce":3762222259},{"type":"v1","addr":"172.21.15.22:6829","nonce":3762222259}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6832","nonce":3762222259},{"type":"v1","addr":"172.21.15.22:6833","nonce":3762222259}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6830","nonce":3762222259},{"type":"v1","addr":"172.21.15.22:6831","nonce":3762222259}]},"public_addr":"172.21.15.22:6827/3762222259","cluster_addr":"172.21.15.22:6829/3762222259","heartbeat_back_addr":"172.21.15.22:6833/3762222259","heartbeat_front_addr":"172.21.15.22:6831/3762222259","state":["exists","up"]},{"osd":10,"uuid":"f5198d52-df48-4679-a860-4dcb440b7a9f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":77,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6824","nonce":1746167904},{"type":"v1","addr":"172.21.15.79:6825","nonce":1746167904}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6826","nonce":1746167904},{"type":"v1","addr":"172.21.15.79:6827","nonce":1746167904}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6830","nonce":1746167904},{"type":"v1","addr":"172.21.15.79:6831","nonce":1746167904}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6828","nonce":1746167904},{"type":"v1","addr":"172.21.15.79:6829","nonce":1746167904}]},"public_addr":"172.21.15.79:6825/1746167904","cluster_addr":"172.21.15.79:6827/1746167904","heartbeat_back_addr":"172.21.15.79:6831/1746167904","heartbeat_front_addr":"172.21.15.79:6829/1746167904","state":["exists","up"]},{"osd":11,"uuid":"1427bca7-3414-4468-9246-e999156bc2ae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":85,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6824","nonce":4271716428},{"type":"v1","addr":"172.21.15.134:6825","nonce":4271716428}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6826","nonce":4271716428},{"type":"v1","addr":"172.21.15.134:6827","nonce":4271716428}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6830","nonce":4271716428},{"type":"v1","addr":"172.21.15.134:6831","nonce":4271716428}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6828","nonce":4271716428},{"type":"v1","addr":"172.21.15.134:6829","nonce":4271716428}]},"public_addr":"172.21.15.134:6825/4271716428","cluster_addr":"172.21.15.134:6827/4271716428","heartbeat_back_addr":"172.21.15.134:6831/4271716428","heartbeat_front_addr":"172.21.15.134:6829/4271716428","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T17:57:21.225125+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T17:57:55.197254+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T17:58:27.825136+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T17:59:04.319893+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T17:59:38.407752+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T18:00:12.118805+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T18:00:47.635214+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T18:01:21.231055+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T18:01:55.817861+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T18:02:31.680160+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T18:03:06.783398+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T18:03:41.307347+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.22:0/3521924283":"2024-03-29T17:55:10.425504+0000","172.21.15.22:6801/730635939":"2024-03-29T17:55:10.425504+0000","172.21.15.22:0/2471229459":"2024-03-29T17:54:34.004559+0000","172.21.15.22:6800/3166828081":"2024-03-29T17:54:34.004559+0000","172.21.15.22:6800/730635939":"2024-03-29T17:55:10.425504+0000","172.21.15.22:0/1229047849":"2024-03-29T17:54:34.004559+0000","172.21.15.22:6801/3166828081":"2024-03-29T17:54:34.004559+0000","172.21.15.22:0/2717818856":"2024-03-29T17:55:10.425504+0000","172.21.15.22:0/2157570333":"2024-03-29T17:54:34.004559+0000","172.21.15.22:0/1006589530":"2024-03-29T17:55:10.425504+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-28T18:04:58.760 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:58.760+0000 7fb405c00640 1 -- 172.21.15.22:0/660513257 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7fb3e40a2910 msgr2=0x7fb3e40a4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:04:58.760 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:58.760+0000 7fb405c00640 1 --2- 172.21.15.22:0/660513257 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7fb3e40a2910 0x7fb3e40a4dd0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fb3f0009bd0 tx=0x7fb3f0005f90 comp rx=0 tx=0).stop 2024-03-28T18:04:58.760 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:58.760+0000 7fb405c00640 1 -- 172.21.15.22:0/660513257 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb4000772a0 msgr2=0x7fb40019e4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:04:58.760 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:58.760+0000 7fb405c00640 1 --2- 172.21.15.22:0/660513257 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb4000772a0 0x7fb40019e4e0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fb3ec00e9f0 tx=0x7fb3ec00eeb0 comp rx=0 tx=0).stop 2024-03-28T18:04:58.761 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:58.760+0000 7fb405c00640 1 -- 172.21.15.22:0/660513257 shutdown_connections 2024-03-28T18:04:58.761 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:58.760+0000 7fb405c00640 1 --2- 172.21.15.22:0/660513257 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7fb3e40a2910 0x7fb3e40a4dd0 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:04:58.761 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:58.760+0000 7fb405c00640 1 --2- 172.21.15.22:0/660513257 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb400077c40 0x7fb4001a5550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:04:58.761 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:58.760+0000 7fb405c00640 1 --2- 172.21.15.22:0/660513257 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb4000772a0 0x7fb40019e4e0 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:04:58.761 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:58.760+0000 7fb405c00640 1 --2- 172.21.15.22:0/660513257 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fb400076060 0x7fb40019dfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:04:58.761 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:58.760+0000 7fb405c00640 1 -- 172.21.15.22:0/660513257 >> 172.21.15.22:0/660513257 conn(0x7fb4000fee50 msgr2=0x7fb4001005b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:04:58.761 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:58.760+0000 7fb405c00640 1 -- 172.21.15.22:0/660513257 shutdown_connections 2024-03-28T18:04:58.761 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:58.760+0000 7fb405c00640 1 -- 172.21.15.22:0/660513257 wait complete. 2024-03-28T18:04:58.778 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-03-28T18:04:58.778 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-28T18:04:58.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:04:58 smithi134 ceph-mon[35037]: mgrmap e17: x(active, since 9m), standbys: z, y 2024-03-28T18:04:58.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:04:58 smithi134 ceph-mon[35037]: from='client.14778 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-03-28T18:04:58.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:04:58 smithi134 ceph-mon[35037]: from='client.? 172.21.15.22:0/1366171867' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-28T18:04:58.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:04:58 smithi079 ceph-mon[35312]: mgrmap e17: x(active, since 9m), standbys: z, y 2024-03-28T18:04:58.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:04:58 smithi079 ceph-mon[35312]: from='client.14778 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-03-28T18:04:58.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:04:58 smithi079 ceph-mon[35312]: from='client.? 172.21.15.22:0/1366171867' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-28T18:04:59.001 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:59.001+0000 7f2627e00640 1 -- 172.21.15.22:0/130402307 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f26281052a0 msgr2=0x7f262810bb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:04:59.001 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:59.001+0000 7f2627e00640 1 --2- 172.21.15.22:0/130402307 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f26281052a0 0x7f262810bb30 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f2620009ca0 tx=0x7f262002c100 comp rx=0 tx=0).stop 2024-03-28T18:04:59.001 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:59.001+0000 7f2627e00640 1 -- 172.21.15.22:0/130402307 shutdown_connections 2024-03-28T18:04:59.001 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:59.001+0000 7f2627e00640 1 --2- 172.21.15.22:0/130402307 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f26281052a0 0x7f262810bb30 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:04:59.001 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:59.001+0000 7f2627e00640 1 --2- 172.21.15.22:0/130402307 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f26281048e0 0x7f2628104d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:04:59.001 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:59.001+0000 7f2627e00640 1 --2- 172.21.15.22:0/130402307 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f26281036f0 0x7f2628103b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:04:59.001 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:59.001+0000 7f2627e00640 1 -- 172.21.15.22:0/130402307 >> 172.21.15.22:0/130402307 conn(0x7f26280fec80 msgr2=0x7f26281010a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:04:59.001 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:59.001+0000 7f2627e00640 1 -- 172.21.15.22:0/130402307 shutdown_connections 2024-03-28T18:04:59.001 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:59.001+0000 7f2627e00640 1 -- 172.21.15.22:0/130402307 wait complete. 2024-03-28T18:04:59.003 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:59.002+0000 7f2627e00640 1 Processor -- start 2024-03-28T18:04:59.003 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:59.002+0000 7f2627e00640 1 -- start start 2024-03-28T18:04:59.003 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:59.002+0000 7f2627e00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f26281036f0 0x7f2628199950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:04:59.003 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:59.002+0000 7f2627e00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f26281048e0 0x7f2628199e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:04:59.003 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:59.003+0000 7f2627e00640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f26281052a0 0x7f26281a0f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:04:59.003 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:59.003+0000 7f2627e00640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f262810f340 con 0x7f26281048e0 2024-03-28T18:04:59.003 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:59.003+0000 7f2627e00640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f262810f1c0 con 0x7f26281052a0 2024-03-28T18:04:59.003 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:59.003+0000 7f2627e00640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f262810f4c0 con 0x7f26281036f0 2024-03-28T18:04:59.003 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:59.003+0000 7f261fe00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f26281048e0 0x7f2628199e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:04:59.003 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:59.003+0000 7f261fe00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f26281048e0 0x7f2628199e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:47522/0 (socket says 172.21.15.22:47522) 2024-03-28T18:04:59.003 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:59.003+0000 7f261fe00640 1 -- 172.21.15.22:0/2834241066 learned_addr learned my addr 172.21.15.22:0/2834241066 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-28T18:04:59.003 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:59.003+0000 7f2627400640 1 --2- 172.21.15.22:0/2834241066 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f26281052a0 0x7f26281a0f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:04:59.003 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:59.003+0000 7f2626a00640 1 --2- 172.21.15.22:0/2834241066 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f26281036f0 0x7f2628199950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:04:59.005 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:59.003+0000 7f261fe00640 1 -- 172.21.15.22:0/2834241066 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f26281036f0 msgr2=0x7f2628199950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:04:59.005 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:59.003+0000 7f261fe00640 1 --2- 172.21.15.22:0/2834241066 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f26281036f0 0x7f2628199950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:04:59.005 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:59.003+0000 7f261fe00640 1 -- 172.21.15.22:0/2834241066 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f26281052a0 msgr2=0x7f26281a0f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:04:59.005 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:59.003+0000 7f261fe00640 1 --2- 172.21.15.22:0/2834241066 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f26281052a0 0x7f26281a0f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:04:59.005 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:59.003+0000 7f261fe00640 1 -- 172.21.15.22:0/2834241066 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f26281a1570 con 0x7f26281048e0 2024-03-28T18:04:59.005 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:59.004+0000 7f261fe00640 1 --2- 172.21.15.22:0/2834241066 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f26281048e0 0x7f2628199e90 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f261000d950 tx=0x7f261000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:04:59.005 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:59.004+0000 7f262ca00640 1 -- 172.21.15.22:0/2834241066 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2610017070 con 0x7f26281048e0 2024-03-28T18:04:59.005 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:59.004+0000 7f262ca00640 1 -- 172.21.15.22:0/2834241066 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f2610009c30 con 0x7f26281048e0 2024-03-28T18:04:59.005 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:59.004+0000 7f2627e00640 1 -- 172.21.15.22:0/2834241066 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f26281a1860 con 0x7f26281048e0 2024-03-28T18:04:59.005 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:59.004+0000 7f262ca00640 1 -- 172.21.15.22:0/2834241066 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f261000f6b0 con 0x7f26281048e0 2024-03-28T18:04:59.005 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:59.004+0000 7f2627e00640 1 -- 172.21.15.22:0/2834241066 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f26281a1db0 con 0x7f26281048e0 2024-03-28T18:04:59.006 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:59.006+0000 7f2627e00640 1 -- 172.21.15.22:0/2834241066 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f262810aa10 con 0x7f26281048e0 2024-03-28T18:04:59.007 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:59.008+0000 7f262ca00640 1 -- 172.21.15.22:0/2834241066 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 17) v1 ==== 137341+0+0 (secure 0 0 0) 0x7f2610010a50 con 0x7f26281048e0 2024-03-28T18:04:59.008 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:59.009+0000 7f262ca00640 1 --2- 172.21.15.22:0/2834241066 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7f260c0a2890 0x7f260c0a4d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:04:59.009 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:59.009+0000 7f262ca00640 1 -- 172.21.15.22:0/2834241066 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f26100ce5e0 con 0x7f26281048e0 2024-03-28T18:04:59.009 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:59.009+0000 7f2626a00640 1 --2- 172.21.15.22:0/2834241066 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7f260c0a2890 0x7f260c0a4d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:04:59.009 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:59.010+0000 7f2626a00640 1 --2- 172.21.15.22:0/2834241066 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7f260c0a2890 0x7f260c0a4d50 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f2628104740 tx=0x7f2618005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:04:59.013 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:59.014+0000 7f262ca00640 1 -- 172.21.15.22:0/2834241066 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f2610009de0 con 0x7f26281048e0 2024-03-28T18:04:59.031 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:04:58 smithi022 ceph-mon[31985]: mgrmap e17: x(active, since 9m), standbys: z, y 2024-03-28T18:04:59.031 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:04:58 smithi022 ceph-mon[31985]: from='client.14778 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-03-28T18:04:59.031 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:04:58 smithi022 ceph-mon[31985]: from='client.? 172.21.15.22:0/1366171867' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-28T18:04:59.205 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:59.206+0000 7f2627e00640 1 -- 172.21.15.22:0/2834241066 --> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f2628061ae0 con 0x7f260c0a2890 2024-03-28T18:04:59.220 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:59.220+0000 7f262ca00640 1 -- 172.21.15.22:0/2834241066 <== mgr.14152 v2:172.21.15.22:6800/4201048392 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+258868 (secure 0 0 0) 0x7f2628061ae0 con 0x7f260c0a2890 2024-03-28T18:04:59.223 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-03-28T18:04:59.225 INFO:teuthology.orchestra.run.smithi022.stderr:dumped all 2024-03-28T18:04:59.227 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:59.224+0000 7f2627e00640 1 -- 172.21.15.22:0/2834241066 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7f260c0a2890 msgr2=0x7f260c0a4d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:04:59.227 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:59.224+0000 7f2627e00640 1 --2- 172.21.15.22:0/2834241066 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7f260c0a2890 0x7f260c0a4d50 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f2628104740 tx=0x7f2618005f90 comp rx=0 tx=0).stop 2024-03-28T18:04:59.227 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:59.224+0000 7f2627e00640 1 -- 172.21.15.22:0/2834241066 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f26281048e0 msgr2=0x7f2628199e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:04:59.227 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:59.224+0000 7f2627e00640 1 --2- 172.21.15.22:0/2834241066 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f26281048e0 0x7f2628199e90 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f261000d950 tx=0x7f261000de10 comp rx=0 tx=0).stop 2024-03-28T18:04:59.227 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:59.225+0000 7f2627e00640 1 -- 172.21.15.22:0/2834241066 shutdown_connections 2024-03-28T18:04:59.227 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:59.225+0000 7f2627e00640 1 --2- 172.21.15.22:0/2834241066 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7f260c0a2890 0x7f260c0a4d50 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:04:59.227 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:59.225+0000 7f2627e00640 1 --2- 172.21.15.22:0/2834241066 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f26281052a0 0x7f26281a0f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:04:59.227 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:59.225+0000 7f2627e00640 1 --2- 172.21.15.22:0/2834241066 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f26281048e0 0x7f2628199e90 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:04:59.227 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:59.225+0000 7f2627e00640 1 --2- 172.21.15.22:0/2834241066 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f26281036f0 0x7f2628199950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:04:59.227 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:59.225+0000 7f2627e00640 1 -- 172.21.15.22:0/2834241066 >> 172.21.15.22:0/2834241066 conn(0x7f26280fec80 msgr2=0x7f2628107be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:04:59.227 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:59.225+0000 7f2627e00640 1 -- 172.21.15.22:0/2834241066 shutdown_connections 2024-03-28T18:04:59.227 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:59.225+0000 7f2627e00640 1 -- 172.21.15.22:0/2834241066 wait complete. 2024-03-28T18:04:59.247 INFO:teuthology.orchestra.run.smithi022.stdout:{"pg_ready":true,"pg_map":{"version":374,"stamp":"2024-03-28T18:04:58.601732+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":598539,"num_objects":88,"num_object_clones":0,"num_object_copies":264,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":88,"num_whiteouts":0,"num_read":144,"num_read_kb":124,"num_write":364,"num_write_kb":3000,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":56,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":295,"ondisk_log_size":295,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":395880,"kb_used_data":13448,"kb_used_omap":18,"kb_used_meta":382317,"kb_avail":1124545944,"statfs":{"total":1151940427776,"available":1151535046656,"internally_reserved":0,"allocated":13770752,"data_stored":7399839,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19141,"internal_metadata":391492923},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[3,2,2,12],"upper_bound":16},"perf_stat":{"commit_latency_ms":25,"apply_latency_ms":25,"commit_latency_ns":25000000,"apply_latency_ns":25000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":8171,"num_objects":86,"num_object_clones":0,"num_object_copies":258,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":86,"num_whiteouts":0,"num_read":8,"num_read_kb":5,"num_write":183,"num_write_kb":62,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":56,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"9.108825"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":13,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-28T18:04:51.313643+0000","last_change":"2024-03-28T18:04:49.545022+0000","last_active":"2024-03-28T18:04:51.313643+0000","last_peered":"2024-03-28T18:04:51.313643+0000","last_clean":"2024-03-28T18:04:51.313643+0000","last_became_active":"2024-03-28T18:04:49.544584+0000","last_became_peered":"2024-03-28T18:04:49.544584+0000","last_unstale":"2024-03-28T18:04:51.313643+0000","last_undegraded":"2024-03-28T18:04:51.313643+0000","last_fullsized":"2024-03-28T18:04:51.313643+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_clean_scrub_stamp":"2024-03-28T18:04:48.465760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T01:47:05.644293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":13,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-28T18:04:51.313105+0000","last_change":"2024-03-28T18:04:49.526720+0000","last_active":"2024-03-28T18:04:51.313105+0000","last_peered":"2024-03-28T18:04:51.313105+0000","last_clean":"2024-03-28T18:04:51.313105+0000","last_became_active":"2024-03-28T18:04:49.524545+0000","last_became_peered":"2024-03-28T18:04:49.524545+0000","last_unstale":"2024-03-28T18:04:51.313105+0000","last_undegraded":"2024-03-28T18:04:51.313105+0000","last_fullsized":"2024-03-28T18:04:51.313105+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_clean_scrub_stamp":"2024-03-28T18:04:48.465760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T01:00:54.926308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":13,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-28T18:04:51.313306+0000","last_change":"2024-03-28T18:04:49.532425+0000","last_active":"2024-03-28T18:04:51.313306+0000","last_peered":"2024-03-28T18:04:51.313306+0000","last_clean":"2024-03-28T18:04:51.313306+0000","last_became_active":"2024-03-28T18:04:49.531146+0000","last_became_peered":"2024-03-28T18:04:49.531146+0000","last_unstale":"2024-03-28T18:04:51.313306+0000","last_undegraded":"2024-03-28T18:04:51.313306+0000","last_fullsized":"2024-03-28T18:04:51.313306+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_clean_scrub_stamp":"2024-03-28T18:04:48.465760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T04:58:45.131773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":13,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-28T18:04:50.575241+0000","last_change":"2024-03-28T18:04:49.533371+0000","last_active":"2024-03-28T18:04:50.575241+0000","last_peered":"2024-03-28T18:04:50.575241+0000","last_clean":"2024-03-28T18:04:50.575241+0000","last_became_active":"2024-03-28T18:04:49.532544+0000","last_became_peered":"2024-03-28T18:04:49.532544+0000","last_unstale":"2024-03-28T18:04:50.575241+0000","last_undegraded":"2024-03-28T18:04:50.575241+0000","last_fullsized":"2024-03-28T18:04:50.575241+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_clean_scrub_stamp":"2024-03-28T18:04:48.465760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T20:36:25.664137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":13,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-28T18:04:50.871493+0000","last_change":"2024-03-28T18:04:49.530875+0000","last_active":"2024-03-28T18:04:50.871493+0000","last_peered":"2024-03-28T18:04:50.871493+0000","last_clean":"2024-03-28T18:04:50.871493+0000","last_became_active":"2024-03-28T18:04:49.530354+0000","last_became_peered":"2024-03-28T18:04:49.530354+0000","last_unstale":"2024-03-28T18:04:50.871493+0000","last_undegraded":"2024-03-28T18:04:50.871493+0000","last_fullsized":"2024-03-28T18:04:50.871493+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_clean_scrub_stamp":"2024-03-28T18:04:48.465760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T21:02:17.187989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":13,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-28T18:04:51.313314+0000","last_change":"2024-03-28T18:04:49.538260+0000","last_active":"2024-03-28T18:04:51.313314+0000","last_peered":"2024-03-28T18:04:51.313314+0000","last_clean":"2024-03-28T18:04:51.313314+0000","last_became_active":"2024-03-28T18:04:49.537863+0000","last_became_peered":"2024-03-28T18:04:49.537863+0000","last_unstale":"2024-03-28T18:04:51.313314+0000","last_undegraded":"2024-03-28T18:04:51.313314+0000","last_fullsized":"2024-03-28T18:04:51.313314+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_clean_scrub_stamp":"2024-03-28T18:04:48.465760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T03:01:25.363839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":13,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-28T18:04:51.314134+0000","last_change":"2024-03-28T18:04:49.534636+0000","last_active":"2024-03-28T18:04:51.314134+0000","last_peered":"2024-03-28T18:04:51.314134+0000","last_clean":"2024-03-28T18:04:51.314134+0000","last_became_active":"2024-03-28T18:04:49.534236+0000","last_became_peered":"2024-03-28T18:04:49.534236+0000","last_unstale":"2024-03-28T18:04:51.314134+0000","last_undegraded":"2024-03-28T18:04:51.314134+0000","last_fullsized":"2024-03-28T18:04:51.314134+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_clean_scrub_stamp":"2024-03-28T18:04:48.465760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T05:55:23.616641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":13,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-28T18:04:50.870443+0000","last_change":"2024-03-28T18:04:49.534848+0000","last_active":"2024-03-28T18:04:50.870443+0000","last_peered":"2024-03-28T18:04:50.870443+0000","last_clean":"2024-03-28T18:04:50.870443+0000","last_became_active":"2024-03-28T18:04:49.534302+0000","last_became_peered":"2024-03-28T18:04:49.534302+0000","last_unstale":"2024-03-28T18:04:50.870443+0000","last_undegraded":"2024-03-28T18:04:50.870443+0000","last_fullsized":"2024-03-28T18:04:50.870443+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_clean_scrub_stamp":"2024-03-28T18:04:48.465760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T05:48:24.707305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":13,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-28T18:04:50.517515+0000","last_change":"2024-03-28T18:04:49.536568+0000","last_active":"2024-03-28T18:04:50.517515+0000","last_peered":"2024-03-28T18:04:50.517515+0000","last_clean":"2024-03-28T18:04:50.517515+0000","last_became_active":"2024-03-28T18:04:49.535675+0000","last_became_peered":"2024-03-28T18:04:49.535675+0000","last_unstale":"2024-03-28T18:04:50.517515+0000","last_undegraded":"2024-03-28T18:04:50.517515+0000","last_fullsized":"2024-03-28T18:04:50.517515+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_clean_scrub_stamp":"2024-03-28T18:04:48.465760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T02:32:15.179869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":13,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-28T18:04:51.313844+0000","last_change":"2024-03-28T18:04:49.539363+0000","last_active":"2024-03-28T18:04:51.313844+0000","last_peered":"2024-03-28T18:04:51.313844+0000","last_clean":"2024-03-28T18:04:51.313844+0000","last_became_active":"2024-03-28T18:04:49.539002+0000","last_became_peered":"2024-03-28T18:04:49.539002+0000","last_unstale":"2024-03-28T18:04:51.313844+0000","last_undegraded":"2024-03-28T18:04:51.313844+0000","last_fullsized":"2024-03-28T18:04:51.313844+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_clean_scrub_stamp":"2024-03-28T18:04:48.465760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T18:24:28.846081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":13,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-28T18:04:51.314984+0000","last_change":"2024-03-28T18:04:49.532385+0000","last_active":"2024-03-28T18:04:51.314984+0000","last_peered":"2024-03-28T18:04:51.314984+0000","last_clean":"2024-03-28T18:04:51.314984+0000","last_became_active":"2024-03-28T18:04:49.532004+0000","last_became_peered":"2024-03-28T18:04:49.532004+0000","last_unstale":"2024-03-28T18:04:51.314984+0000","last_undegraded":"2024-03-28T18:04:51.314984+0000","last_fullsized":"2024-03-28T18:04:51.314984+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_clean_scrub_stamp":"2024-03-28T18:04:48.465760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T19:05:13.959313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":13,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-28T18:04:50.574994+0000","last_change":"2024-03-28T18:04:49.534465+0000","last_active":"2024-03-28T18:04:50.574994+0000","last_peered":"2024-03-28T18:04:50.574994+0000","last_clean":"2024-03-28T18:04:50.574994+0000","last_became_active":"2024-03-28T18:04:49.532989+0000","last_became_peered":"2024-03-28T18:04:49.532989+0000","last_unstale":"2024-03-28T18:04:50.574994+0000","last_undegraded":"2024-03-28T18:04:50.574994+0000","last_fullsized":"2024-03-28T18:04:50.574994+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_clean_scrub_stamp":"2024-03-28T18:04:48.465760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T23:56:50.607770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":13,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-28T18:04:50.514240+0000","last_change":"2024-03-28T18:04:49.535205+0000","last_active":"2024-03-28T18:04:50.514240+0000","last_peered":"2024-03-28T18:04:50.514240+0000","last_clean":"2024-03-28T18:04:50.514240+0000","last_became_active":"2024-03-28T18:04:49.533025+0000","last_became_peered":"2024-03-28T18:04:49.533025+0000","last_unstale":"2024-03-28T18:04:50.514240+0000","last_undegraded":"2024-03-28T18:04:50.514240+0000","last_fullsized":"2024-03-28T18:04:50.514240+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_clean_scrub_stamp":"2024-03-28T18:04:48.465760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T21:41:51.815907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":13,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-28T18:04:51.312708+0000","last_change":"2024-03-28T18:04:49.520974+0000","last_active":"2024-03-28T18:04:51.312708+0000","last_peered":"2024-03-28T18:04:51.312708+0000","last_clean":"2024-03-28T18:04:51.312708+0000","last_became_active":"2024-03-28T18:04:49.520509+0000","last_became_peered":"2024-03-28T18:04:49.520509+0000","last_unstale":"2024-03-28T18:04:51.312708+0000","last_undegraded":"2024-03-28T18:04:51.312708+0000","last_fullsized":"2024-03-28T18:04:51.312708+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_clean_scrub_stamp":"2024-03-28T18:04:48.465760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T03:15:16.346859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":13,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-28T18:04:51.312748+0000","last_change":"2024-03-28T18:04:49.539053+0000","last_active":"2024-03-28T18:04:51.312748+0000","last_peered":"2024-03-28T18:04:51.312748+0000","last_clean":"2024-03-28T18:04:51.312748+0000","last_became_active":"2024-03-28T18:04:49.537327+0000","last_became_peered":"2024-03-28T18:04:49.537327+0000","last_unstale":"2024-03-28T18:04:51.312748+0000","last_undegraded":"2024-03-28T18:04:51.312748+0000","last_fullsized":"2024-03-28T18:04:51.312748+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_clean_scrub_stamp":"2024-03-28T18:04:48.465760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T00:24:42.319504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":13,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-28T18:04:50.516563+0000","last_change":"2024-03-28T18:04:49.525855+0000","last_active":"2024-03-28T18:04:50.516563+0000","last_peered":"2024-03-28T18:04:50.516563+0000","last_clean":"2024-03-28T18:04:50.516563+0000","last_became_active":"2024-03-28T18:04:49.525448+0000","last_became_peered":"2024-03-28T18:04:49.525448+0000","last_unstale":"2024-03-28T18:04:50.516563+0000","last_undegraded":"2024-03-28T18:04:50.516563+0000","last_fullsized":"2024-03-28T18:04:50.516563+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_clean_scrub_stamp":"2024-03-28T18:04:48.465760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T23:53:00.452692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":13,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-28T18:04:51.313695+0000","last_change":"2024-03-28T18:04:49.543605+0000","last_active":"2024-03-28T18:04:51.313695+0000","last_peered":"2024-03-28T18:04:51.313695+0000","last_clean":"2024-03-28T18:04:51.313695+0000","last_became_active":"2024-03-28T18:04:49.543280+0000","last_became_peered":"2024-03-28T18:04:49.543280+0000","last_unstale":"2024-03-28T18:04:51.313695+0000","last_undegraded":"2024-03-28T18:04:51.313695+0000","last_fullsized":"2024-03-28T18:04:51.313695+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_clean_scrub_stamp":"2024-03-28T18:04:48.465760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T21:27:26.216639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":13,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-28T18:04:50.866144+0000","last_change":"2024-03-28T18:04:49.526745+0000","last_active":"2024-03-28T18:04:50.866144+0000","last_peered":"2024-03-28T18:04:50.866144+0000","last_clean":"2024-03-28T18:04:50.866144+0000","last_became_active":"2024-03-28T18:04:49.526351+0000","last_became_peered":"2024-03-28T18:04:49.526351+0000","last_unstale":"2024-03-28T18:04:50.866144+0000","last_undegraded":"2024-03-28T18:04:50.866144+0000","last_fullsized":"2024-03-28T18:04:50.866144+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_clean_scrub_stamp":"2024-03-28T18:04:48.465760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T05:31:47.687345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":13,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-28T18:04:51.312740+0000","last_change":"2024-03-28T18:04:49.523346+0000","last_active":"2024-03-28T18:04:51.312740+0000","last_peered":"2024-03-28T18:04:51.312740+0000","last_clean":"2024-03-28T18:04:51.312740+0000","last_became_active":"2024-03-28T18:04:49.522041+0000","last_became_peered":"2024-03-28T18:04:49.522041+0000","last_unstale":"2024-03-28T18:04:51.312740+0000","last_undegraded":"2024-03-28T18:04:51.312740+0000","last_fullsized":"2024-03-28T18:04:51.312740+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_clean_scrub_stamp":"2024-03-28T18:04:48.465760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T01:35:02.419328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"90'11","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-28T18:04:54.627438+0000","last_change":"2024-03-28T18:04:48.510917+0000","last_active":"2024-03-28T18:04:54.627438+0000","last_peered":"2024-03-28T18:04:54.627438+0000","last_clean":"2024-03-28T18:04:54.627438+0000","last_became_active":"2024-03-28T18:04:48.509978+0000","last_became_peered":"2024-03-28T18:04:48.509978+0000","last_unstale":"2024-03-28T18:04:54.627438+0000","last_undegraded":"2024-03-28T18:04:54.627438+0000","last_fullsized":"2024-03-28T18:04:54.627438+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:04:47.412563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:04:47.412563+0000","last_clean_scrub_stamp":"2024-03-28T18:04:47.412563+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T03:28:13.185099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":13,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-28T18:04:50.871301+0000","last_change":"2024-03-28T18:04:49.535543+0000","last_active":"2024-03-28T18:04:50.871301+0000","last_peered":"2024-03-28T18:04:50.871301+0000","last_clean":"2024-03-28T18:04:50.871301+0000","last_became_active":"2024-03-28T18:04:49.535171+0000","last_became_peered":"2024-03-28T18:04:49.535171+0000","last_unstale":"2024-03-28T18:04:50.871301+0000","last_undegraded":"2024-03-28T18:04:50.871301+0000","last_fullsized":"2024-03-28T18:04:50.871301+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_clean_scrub_stamp":"2024-03-28T18:04:48.465760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T22:12:22.258424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"90'2","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-28T18:04:50.871501+0000","last_change":"2024-03-28T18:04:48.503190+0000","last_active":"2024-03-28T18:04:50.871501+0000","last_peered":"2024-03-28T18:04:50.871501+0000","last_clean":"2024-03-28T18:04:50.871501+0000","last_became_active":"2024-03-28T18:04:48.501858+0000","last_became_peered":"2024-03-28T18:04:48.501858+0000","last_unstale":"2024-03-28T18:04:50.871501+0000","last_undegraded":"2024-03-28T18:04:50.871501+0000","last_fullsized":"2024-03-28T18:04:50.871501+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:04:47.412563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:04:47.412563+0000","last_clean_scrub_stamp":"2024-03-28T18:04:47.412563+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T19:56:43.878630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":13,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-28T18:04:51.313439+0000","last_change":"2024-03-28T18:04:49.537739+0000","last_active":"2024-03-28T18:04:51.313439+0000","last_peered":"2024-03-28T18:04:51.313439+0000","last_clean":"2024-03-28T18:04:51.313439+0000","last_became_active":"2024-03-28T18:04:49.535990+0000","last_became_peered":"2024-03-28T18:04:49.535990+0000","last_unstale":"2024-03-28T18:04:51.313439+0000","last_undegraded":"2024-03-28T18:04:51.313439+0000","last_fullsized":"2024-03-28T18:04:51.313439+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_clean_scrub_stamp":"2024-03-28T18:04:48.465760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T21:54:15.729494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"90'4","reported_seq":19,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-28T18:04:53.617983+0000","last_change":"2024-03-28T18:04:48.514268+0000","last_active":"2024-03-28T18:04:53.617983+0000","last_peered":"2024-03-28T18:04:53.617983+0000","last_clean":"2024-03-28T18:04:53.617983+0000","last_became_active":"2024-03-28T18:04:48.513602+0000","last_became_peered":"2024-03-28T18:04:48.513602+0000","last_unstale":"2024-03-28T18:04:53.617983+0000","last_undegraded":"2024-03-28T18:04:53.617983+0000","last_fullsized":"2024-03-28T18:04:53.617983+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:04:47.412563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:04:47.412563+0000","last_clean_scrub_stamp":"2024-03-28T18:04:47.412563+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T01:12:17.758721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":13,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-28T18:04:50.516993+0000","last_change":"2024-03-28T18:04:49.529316+0000","last_active":"2024-03-28T18:04:50.516993+0000","last_peered":"2024-03-28T18:04:50.516993+0000","last_clean":"2024-03-28T18:04:50.516993+0000","last_became_active":"2024-03-28T18:04:49.528975+0000","last_became_peered":"2024-03-28T18:04:49.528975+0000","last_unstale":"2024-03-28T18:04:50.516993+0000","last_undegraded":"2024-03-28T18:04:50.516993+0000","last_fullsized":"2024-03-28T18:04:50.516993+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_clean_scrub_stamp":"2024-03-28T18:04:48.465760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T00:06:45.268757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"89'2","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-28T18:04:50.871893+0000","last_change":"2024-03-28T18:04:48.515841+0000","last_active":"2024-03-28T18:04:50.871893+0000","last_peered":"2024-03-28T18:04:50.871893+0000","last_clean":"2024-03-28T18:04:50.871893+0000","last_became_active":"2024-03-28T18:04:48.513852+0000","last_became_peered":"2024-03-28T18:04:48.513852+0000","last_unstale":"2024-03-28T18:04:50.871893+0000","last_undegraded":"2024-03-28T18:04:50.871893+0000","last_fullsized":"2024-03-28T18:04:50.871893+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:04:47.412563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:04:47.412563+0000","last_clean_scrub_stamp":"2024-03-28T18:04:47.412563+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T00:07:14.751545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":13,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-28T18:04:50.521476+0000","last_change":"2024-03-28T18:04:49.542238+0000","last_active":"2024-03-28T18:04:50.521476+0000","last_peered":"2024-03-28T18:04:50.521476+0000","last_clean":"2024-03-28T18:04:50.521476+0000","last_became_active":"2024-03-28T18:04:49.539723+0000","last_became_peered":"2024-03-28T18:04:49.539723+0000","last_unstale":"2024-03-28T18:04:50.521476+0000","last_undegraded":"2024-03-28T18:04:50.521476+0000","last_fullsized":"2024-03-28T18:04:50.521476+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_clean_scrub_stamp":"2024-03-28T18:04:48.465760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T22:45:07.310148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"90'6","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-28T18:04:52.604487+0000","last_change":"2024-03-28T18:04:48.511349+0000","last_active":"2024-03-28T18:04:52.604487+0000","last_peered":"2024-03-28T18:04:52.604487+0000","last_clean":"2024-03-28T18:04:52.604487+0000","last_became_active":"2024-03-28T18:04:48.510990+0000","last_became_peered":"2024-03-28T18:04:48.510990+0000","last_unstale":"2024-03-28T18:04:52.604487+0000","last_undegraded":"2024-03-28T18:04:52.604487+0000","last_fullsized":"2024-03-28T18:04:52.604487+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:04:47.412563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:04:47.412563+0000","last_clean_scrub_stamp":"2024-03-28T18:04:47.412563+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T21:56:15.411149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1013,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":13,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-28T18:04:50.872146+0000","last_change":"2024-03-28T18:04:49.538066+0000","last_active":"2024-03-28T18:04:50.872146+0000","last_peered":"2024-03-28T18:04:50.872146+0000","last_clean":"2024-03-28T18:04:50.872146+0000","last_became_active":"2024-03-28T18:04:49.537202+0000","last_became_peered":"2024-03-28T18:04:49.537202+0000","last_unstale":"2024-03-28T18:04:50.872146+0000","last_undegraded":"2024-03-28T18:04:50.872146+0000","last_fullsized":"2024-03-28T18:04:50.872146+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_clean_scrub_stamp":"2024-03-28T18:04:48.465760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T22:35:19.060311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"90'7","reported_seq":22,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-28T18:04:54.623927+0000","last_change":"2024-03-28T18:04:48.513722+0000","last_active":"2024-03-28T18:04:54.623927+0000","last_peered":"2024-03-28T18:04:54.623927+0000","last_clean":"2024-03-28T18:04:54.623927+0000","last_became_active":"2024-03-28T18:04:48.513048+0000","last_became_peered":"2024-03-28T18:04:48.513048+0000","last_unstale":"2024-03-28T18:04:54.623927+0000","last_undegraded":"2024-03-28T18:04:54.623927+0000","last_fullsized":"2024-03-28T18:04:54.623927+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:04:47.412563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:04:47.412563+0000","last_clean_scrub_stamp":"2024-03-28T18:04:47.412563+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T04:48:10.831297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":13,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-28T18:04:50.520925+0000","last_change":"2024-03-28T18:04:49.539172+0000","last_active":"2024-03-28T18:04:50.520925+0000","last_peered":"2024-03-28T18:04:50.520925+0000","last_clean":"2024-03-28T18:04:50.520925+0000","last_became_active":"2024-03-28T18:04:49.538779+0000","last_became_peered":"2024-03-28T18:04:49.538779+0000","last_unstale":"2024-03-28T18:04:50.520925+0000","last_undegraded":"2024-03-28T18:04:50.520925+0000","last_fullsized":"2024-03-28T18:04:50.520925+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_clean_scrub_stamp":"2024-03-28T18:04:48.465760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T04:59:08.361762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"90'3","reported_seq":18,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-28T18:04:53.613193+0000","last_change":"2024-03-28T18:04:48.500471+0000","last_active":"2024-03-28T18:04:53.613193+0000","last_peered":"2024-03-28T18:04:53.613193+0000","last_clean":"2024-03-28T18:04:53.613193+0000","last_became_active":"2024-03-28T18:04:48.500094+0000","last_became_peered":"2024-03-28T18:04:48.500094+0000","last_unstale":"2024-03-28T18:04:53.613193+0000","last_undegraded":"2024-03-28T18:04:53.613193+0000","last_fullsized":"2024-03-28T18:04:53.613193+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:04:47.412563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:04:47.412563+0000","last_clean_scrub_stamp":"2024-03-28T18:04:47.412563+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T21:52:51.964254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":13,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-28T18:04:50.870317+0000","last_change":"2024-03-28T18:04:49.522120+0000","last_active":"2024-03-28T18:04:50.870317+0000","last_peered":"2024-03-28T18:04:50.870317+0000","last_clean":"2024-03-28T18:04:50.870317+0000","last_became_active":"2024-03-28T18:04:49.521656+0000","last_became_peered":"2024-03-28T18:04:49.521656+0000","last_unstale":"2024-03-28T18:04:50.870317+0000","last_undegraded":"2024-03-28T18:04:50.870317+0000","last_fullsized":"2024-03-28T18:04:50.870317+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_clean_scrub_stamp":"2024-03-28T18:04:48.465760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T18:29:55.605765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"90'2","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-28T18:04:52.597054+0000","last_change":"2024-03-28T18:04:48.508723+0000","last_active":"2024-03-28T18:04:52.597054+0000","last_peered":"2024-03-28T18:04:52.597054+0000","last_clean":"2024-03-28T18:04:52.597054+0000","last_became_active":"2024-03-28T18:04:48.508314+0000","last_became_peered":"2024-03-28T18:04:48.508314+0000","last_unstale":"2024-03-28T18:04:52.597054+0000","last_undegraded":"2024-03-28T18:04:52.597054+0000","last_fullsized":"2024-03-28T18:04:52.597054+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:04:47.412563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:04:47.412563+0000","last_clean_scrub_stamp":"2024-03-28T18:04:47.412563+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T03:03:59.353454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":13,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-28T18:04:50.865374+0000","last_change":"2024-03-28T18:04:49.531469+0000","last_active":"2024-03-28T18:04:50.865374+0000","last_peered":"2024-03-28T18:04:50.865374+0000","last_clean":"2024-03-28T18:04:50.865374+0000","last_became_active":"2024-03-28T18:04:49.529263+0000","last_became_peered":"2024-03-28T18:04:49.529263+0000","last_unstale":"2024-03-28T18:04:50.865374+0000","last_undegraded":"2024-03-28T18:04:50.865374+0000","last_fullsized":"2024-03-28T18:04:50.865374+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_clean_scrub_stamp":"2024-03-28T18:04:48.465760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T05:46:31.593877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"90'4","reported_seq":19,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-28T18:04:52.599084+0000","last_change":"2024-03-28T18:04:48.500644+0000","last_active":"2024-03-28T18:04:52.599084+0000","last_peered":"2024-03-28T18:04:52.599084+0000","last_clean":"2024-03-28T18:04:52.599084+0000","last_became_active":"2024-03-28T18:04:48.499330+0000","last_became_peered":"2024-03-28T18:04:48.499330+0000","last_unstale":"2024-03-28T18:04:52.599084+0000","last_undegraded":"2024-03-28T18:04:52.599084+0000","last_fullsized":"2024-03-28T18:04:52.599084+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:04:47.412563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:04:47.412563+0000","last_clean_scrub_stamp":"2024-03-28T18:04:47.412563+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T18:50:54.361266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":13,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-28T18:04:50.517486+0000","last_change":"2024-03-28T18:04:49.528728+0000","last_active":"2024-03-28T18:04:50.517486+0000","last_peered":"2024-03-28T18:04:50.517486+0000","last_clean":"2024-03-28T18:04:50.517486+0000","last_became_active":"2024-03-28T18:04:49.528312+0000","last_became_peered":"2024-03-28T18:04:49.528312+0000","last_unstale":"2024-03-28T18:04:50.517486+0000","last_undegraded":"2024-03-28T18:04:50.517486+0000","last_fullsized":"2024-03-28T18:04:50.517486+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_clean_scrub_stamp":"2024-03-28T18:04:48.465760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T05:59:37.308002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"90'4","reported_seq":19,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-28T18:04:55.646686+0000","last_change":"2024-03-28T18:04:48.508766+0000","last_active":"2024-03-28T18:04:55.646686+0000","last_peered":"2024-03-28T18:04:55.646686+0000","last_clean":"2024-03-28T18:04:55.646686+0000","last_became_active":"2024-03-28T18:04:48.506780+0000","last_became_peered":"2024-03-28T18:04:48.506780+0000","last_unstale":"2024-03-28T18:04:55.646686+0000","last_undegraded":"2024-03-28T18:04:55.646686+0000","last_fullsized":"2024-03-28T18:04:55.646686+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:04:47.412563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:04:47.412563+0000","last_clean_scrub_stamp":"2024-03-28T18:04:47.412563+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T04:39:07.394655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":13,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-28T18:04:51.313379+0000","last_change":"2024-03-28T18:04:49.537165+0000","last_active":"2024-03-28T18:04:51.313379+0000","last_peered":"2024-03-28T18:04:51.313379+0000","last_clean":"2024-03-28T18:04:51.313379+0000","last_became_active":"2024-03-28T18:04:49.534543+0000","last_became_peered":"2024-03-28T18:04:49.534543+0000","last_unstale":"2024-03-28T18:04:51.313379+0000","last_undegraded":"2024-03-28T18:04:51.313379+0000","last_fullsized":"2024-03-28T18:04:51.313379+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_clean_scrub_stamp":"2024-03-28T18:04:48.465760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T03:15:58.056148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"90'5","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-28T18:04:52.598827+0000","last_change":"2024-03-28T18:04:48.503866+0000","last_active":"2024-03-28T18:04:52.598827+0000","last_peered":"2024-03-28T18:04:52.598827+0000","last_clean":"2024-03-28T18:04:52.598827+0000","last_became_active":"2024-03-28T18:04:48.503223+0000","last_became_peered":"2024-03-28T18:04:48.503223+0000","last_unstale":"2024-03-28T18:04:52.598827+0000","last_undegraded":"2024-03-28T18:04:52.598827+0000","last_fullsized":"2024-03-28T18:04:52.598827+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:04:47.412563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:04:47.412563+0000","last_clean_scrub_stamp":"2024-03-28T18:04:47.412563+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T20:14:01.307483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":13,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-28T18:04:50.517779+0000","last_change":"2024-03-28T18:04:49.539494+0000","last_active":"2024-03-28T18:04:50.517779+0000","last_peered":"2024-03-28T18:04:50.517779+0000","last_clean":"2024-03-28T18:04:50.517779+0000","last_became_active":"2024-03-28T18:04:49.539044+0000","last_became_peered":"2024-03-28T18:04:49.539044+0000","last_unstale":"2024-03-28T18:04:50.517779+0000","last_undegraded":"2024-03-28T18:04:50.517779+0000","last_fullsized":"2024-03-28T18:04:50.517779+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_clean_scrub_stamp":"2024-03-28T18:04:48.465760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T04:08:30.807788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"90'8","reported_seq":23,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-28T18:04:52.589290+0000","last_change":"2024-03-28T18:04:48.507703+0000","last_active":"2024-03-28T18:04:52.589290+0000","last_peered":"2024-03-28T18:04:52.589290+0000","last_clean":"2024-03-28T18:04:52.589290+0000","last_became_active":"2024-03-28T18:04:48.507310+0000","last_became_peered":"2024-03-28T18:04:48.507310+0000","last_unstale":"2024-03-28T18:04:52.589290+0000","last_undegraded":"2024-03-28T18:04:52.589290+0000","last_fullsized":"2024-03-28T18:04:52.589290+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:04:47.412563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:04:47.412563+0000","last_clean_scrub_stamp":"2024-03-28T18:04:47.412563+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T19:14:13.910042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":13,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-28T18:04:50.871891+0000","last_change":"2024-03-28T18:04:49.529928+0000","last_active":"2024-03-28T18:04:50.871891+0000","last_peered":"2024-03-28T18:04:50.871891+0000","last_clean":"2024-03-28T18:04:50.871891+0000","last_became_active":"2024-03-28T18:04:49.529527+0000","last_became_peered":"2024-03-28T18:04:49.529527+0000","last_unstale":"2024-03-28T18:04:50.871891+0000","last_undegraded":"2024-03-28T18:04:50.871891+0000","last_fullsized":"2024-03-28T18:04:50.871891+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_clean_scrub_stamp":"2024-03-28T18:04:48.465760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T05:50:07.129140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"90'9","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-28T18:04:56.658257+0000","last_change":"2024-03-28T18:04:48.494746+0000","last_active":"2024-03-28T18:04:56.658257+0000","last_peered":"2024-03-28T18:04:56.658257+0000","last_clean":"2024-03-28T18:04:56.658257+0000","last_became_active":"2024-03-28T18:04:48.494296+0000","last_became_peered":"2024-03-28T18:04:48.494296+0000","last_unstale":"2024-03-28T18:04:56.658257+0000","last_undegraded":"2024-03-28T18:04:56.658257+0000","last_fullsized":"2024-03-28T18:04:56.658257+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:04:47.412563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:04:47.412563+0000","last_clean_scrub_stamp":"2024-03-28T18:04:47.412563+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T22:32:56.923067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":13,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-28T18:04:50.513932+0000","last_change":"2024-03-28T18:04:49.528070+0000","last_active":"2024-03-28T18:04:50.513932+0000","last_peered":"2024-03-28T18:04:50.513932+0000","last_clean":"2024-03-28T18:04:50.513932+0000","last_became_active":"2024-03-28T18:04:49.526747+0000","last_became_peered":"2024-03-28T18:04:49.526747+0000","last_unstale":"2024-03-28T18:04:50.513932+0000","last_undegraded":"2024-03-28T18:04:50.513932+0000","last_fullsized":"2024-03-28T18:04:50.513932+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_clean_scrub_stamp":"2024-03-28T18:04:48.465760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T22:32:05.750094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"90'4","reported_seq":19,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-28T18:04:52.600128+0000","last_change":"2024-03-28T18:04:48.505723+0000","last_active":"2024-03-28T18:04:52.600128+0000","last_peered":"2024-03-28T18:04:52.600128+0000","last_clean":"2024-03-28T18:04:52.600128+0000","last_became_active":"2024-03-28T18:04:48.504088+0000","last_became_peered":"2024-03-28T18:04:48.504088+0000","last_unstale":"2024-03-28T18:04:52.600128+0000","last_undegraded":"2024-03-28T18:04:52.600128+0000","last_fullsized":"2024-03-28T18:04:52.600128+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:04:47.412563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:04:47.412563+0000","last_clean_scrub_stamp":"2024-03-28T18:04:47.412563+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T23:25:37.138054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":13,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-28T18:04:51.314719+0000","last_change":"2024-03-28T18:04:49.518872+0000","last_active":"2024-03-28T18:04:51.314719+0000","last_peered":"2024-03-28T18:04:51.314719+0000","last_clean":"2024-03-28T18:04:51.314719+0000","last_became_active":"2024-03-28T18:04:49.518507+0000","last_became_peered":"2024-03-28T18:04:49.518507+0000","last_unstale":"2024-03-28T18:04:51.314719+0000","last_undegraded":"2024-03-28T18:04:51.314719+0000","last_fullsized":"2024-03-28T18:04:51.314719+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_clean_scrub_stamp":"2024-03-28T18:04:48.465760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T20:11:31.044321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"90'6","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-28T18:04:54.618891+0000","last_change":"2024-03-28T18:04:48.503058+0000","last_active":"2024-03-28T18:04:54.618891+0000","last_peered":"2024-03-28T18:04:54.618891+0000","last_clean":"2024-03-28T18:04:54.618891+0000","last_became_active":"2024-03-28T18:04:48.501596+0000","last_became_peered":"2024-03-28T18:04:48.501596+0000","last_unstale":"2024-03-28T18:04:54.618891+0000","last_undegraded":"2024-03-28T18:04:54.618891+0000","last_fullsized":"2024-03-28T18:04:54.618891+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:04:47.412563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:04:47.412563+0000","last_clean_scrub_stamp":"2024-03-28T18:04:47.412563+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T23:46:10.442771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":979,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":13,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-28T18:04:51.314045+0000","last_change":"2024-03-28T18:04:49.535102+0000","last_active":"2024-03-28T18:04:51.314045+0000","last_peered":"2024-03-28T18:04:51.314045+0000","last_clean":"2024-03-28T18:04:51.314045+0000","last_became_active":"2024-03-28T18:04:49.534171+0000","last_became_peered":"2024-03-28T18:04:49.534171+0000","last_unstale":"2024-03-28T18:04:51.314045+0000","last_undegraded":"2024-03-28T18:04:51.314045+0000","last_fullsized":"2024-03-28T18:04:51.314045+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_clean_scrub_stamp":"2024-03-28T18:04:48.465760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T18:16:27.703327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"90'15","reported_seq":30,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-28T18:04:53.619126+0000","last_change":"2024-03-28T18:04:48.517034+0000","last_active":"2024-03-28T18:04:53.619126+0000","last_peered":"2024-03-28T18:04:53.619126+0000","last_clean":"2024-03-28T18:04:53.619126+0000","last_became_active":"2024-03-28T18:04:48.516562+0000","last_became_peered":"2024-03-28T18:04:48.516562+0000","last_unstale":"2024-03-28T18:04:53.619126+0000","last_undegraded":"2024-03-28T18:04:53.619126+0000","last_fullsized":"2024-03-28T18:04:53.619126+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:04:47.412563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:04:47.412563+0000","last_clean_scrub_stamp":"2024-03-28T18:04:47.412563+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T02:28:55.753779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":13,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-28T18:04:50.520183+0000","last_change":"2024-03-28T18:04:49.541501+0000","last_active":"2024-03-28T18:04:50.520183+0000","last_peered":"2024-03-28T18:04:50.520183+0000","last_clean":"2024-03-28T18:04:50.520183+0000","last_became_active":"2024-03-28T18:04:49.541117+0000","last_became_peered":"2024-03-28T18:04:49.541117+0000","last_unstale":"2024-03-28T18:04:50.520183+0000","last_undegraded":"2024-03-28T18:04:50.520183+0000","last_fullsized":"2024-03-28T18:04:50.520183+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_clean_scrub_stamp":"2024-03-28T18:04:48.465760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T04:18:29.081489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"89'1","reported_seq":16,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-28T18:04:50.871856+0000","last_change":"2024-03-28T18:04:48.510859+0000","last_active":"2024-03-28T18:04:50.871856+0000","last_peered":"2024-03-28T18:04:50.871856+0000","last_clean":"2024-03-28T18:04:50.871856+0000","last_became_active":"2024-03-28T18:04:48.510522+0000","last_became_peered":"2024-03-28T18:04:48.510522+0000","last_unstale":"2024-03-28T18:04:50.871856+0000","last_undegraded":"2024-03-28T18:04:50.871856+0000","last_fullsized":"2024-03-28T18:04:50.871856+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:04:47.412563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:04:47.412563+0000","last_clean_scrub_stamp":"2024-03-28T18:04:47.412563+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T22:31:37.817361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":13,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-28T18:04:51.313494+0000","last_change":"2024-03-28T18:04:49.541981+0000","last_active":"2024-03-28T18:04:51.313494+0000","last_peered":"2024-03-28T18:04:51.313494+0000","last_clean":"2024-03-28T18:04:51.313494+0000","last_became_active":"2024-03-28T18:04:49.541606+0000","last_became_peered":"2024-03-28T18:04:49.541606+0000","last_unstale":"2024-03-28T18:04:51.313494+0000","last_undegraded":"2024-03-28T18:04:51.313494+0000","last_fullsized":"2024-03-28T18:04:51.313494+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_clean_scrub_stamp":"2024-03-28T18:04:48.465760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T21:56:03.632490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"90'1","reported_seq":16,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-28T18:04:50.865724+0000","last_change":"2024-03-28T18:04:48.505847+0000","last_active":"2024-03-28T18:04:50.865724+0000","last_peered":"2024-03-28T18:04:50.865724+0000","last_clean":"2024-03-28T18:04:50.865724+0000","last_became_active":"2024-03-28T18:04:48.501005+0000","last_became_peered":"2024-03-28T18:04:48.501005+0000","last_unstale":"2024-03-28T18:04:50.865724+0000","last_undegraded":"2024-03-28T18:04:50.865724+0000","last_fullsized":"2024-03-28T18:04:50.865724+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:04:47.412563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:04:47.412563+0000","last_clean_scrub_stamp":"2024-03-28T18:04:47.412563+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T01:33:45.539324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":13,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-28T18:04:51.313734+0000","last_change":"2024-03-28T18:04:49.526938+0000","last_active":"2024-03-28T18:04:51.313734+0000","last_peered":"2024-03-28T18:04:51.313734+0000","last_clean":"2024-03-28T18:04:51.313734+0000","last_became_active":"2024-03-28T18:04:49.526290+0000","last_became_peered":"2024-03-28T18:04:49.526290+0000","last_unstale":"2024-03-28T18:04:51.313734+0000","last_undegraded":"2024-03-28T18:04:51.313734+0000","last_fullsized":"2024-03-28T18:04:51.313734+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_clean_scrub_stamp":"2024-03-28T18:04:48.465760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T03:06:54.561470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"90'5","reported_seq":23,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-28T18:04:56.665115+0000","last_change":"2024-03-28T18:04:48.512553+0000","last_active":"2024-03-28T18:04:56.665115+0000","last_peered":"2024-03-28T18:04:56.665115+0000","last_clean":"2024-03-28T18:04:56.665115+0000","last_became_active":"2024-03-28T18:04:48.511520+0000","last_became_peered":"2024-03-28T18:04:48.511520+0000","last_unstale":"2024-03-28T18:04:56.665115+0000","last_undegraded":"2024-03-28T18:04:56.665115+0000","last_fullsized":"2024-03-28T18:04:56.665115+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:04:47.412563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:04:47.412563+0000","last_clean_scrub_stamp":"2024-03-28T18:04:47.412563+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T21:13:32.970027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":3,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":13,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-28T18:04:51.313575+0000","last_change":"2024-03-28T18:04:49.539484+0000","last_active":"2024-03-28T18:04:51.313575+0000","last_peered":"2024-03-28T18:04:51.313575+0000","last_clean":"2024-03-28T18:04:51.313575+0000","last_became_active":"2024-03-28T18:04:49.539066+0000","last_became_peered":"2024-03-28T18:04:49.539066+0000","last_unstale":"2024-03-28T18:04:51.313575+0000","last_undegraded":"2024-03-28T18:04:51.313575+0000","last_fullsized":"2024-03-28T18:04:51.313575+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_clean_scrub_stamp":"2024-03-28T18:04:48.465760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T05:22:32.989771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":13,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-28T18:04:50.520810+0000","last_change":"2024-03-28T18:04:49.540438+0000","last_active":"2024-03-28T18:04:50.520810+0000","last_peered":"2024-03-28T18:04:50.520810+0000","last_clean":"2024-03-28T18:04:50.520810+0000","last_became_active":"2024-03-28T18:04:49.540069+0000","last_became_peered":"2024-03-28T18:04:49.540069+0000","last_unstale":"2024-03-28T18:04:50.520810+0000","last_undegraded":"2024-03-28T18:04:50.520810+0000","last_fullsized":"2024-03-28T18:04:50.520810+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_clean_scrub_stamp":"2024-03-28T18:04:48.465760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T19:31:49.254101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"90'5","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-28T18:04:53.614616+0000","last_change":"2024-03-28T18:04:48.508839+0000","last_active":"2024-03-28T18:04:53.614616+0000","last_peered":"2024-03-28T18:04:53.614616+0000","last_clean":"2024-03-28T18:04:53.614616+0000","last_became_active":"2024-03-28T18:04:48.508475+0000","last_became_peered":"2024-03-28T18:04:48.508475+0000","last_unstale":"2024-03-28T18:04:53.614616+0000","last_undegraded":"2024-03-28T18:04:53.614616+0000","last_fullsized":"2024-03-28T18:04:53.614616+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:04:47.412563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:04:47.412563+0000","last_clean_scrub_stamp":"2024-03-28T18:04:47.412563+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T04:31:10.463284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":13,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-28T18:04:50.871099+0000","last_change":"2024-03-28T18:04:49.534449+0000","last_active":"2024-03-28T18:04:50.871099+0000","last_peered":"2024-03-28T18:04:50.871099+0000","last_clean":"2024-03-28T18:04:50.871099+0000","last_became_active":"2024-03-28T18:04:49.534098+0000","last_became_peered":"2024-03-28T18:04:49.534098+0000","last_unstale":"2024-03-28T18:04:50.871099+0000","last_undegraded":"2024-03-28T18:04:50.871099+0000","last_fullsized":"2024-03-28T18:04:50.871099+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_clean_scrub_stamp":"2024-03-28T18:04:48.465760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T23:47:07.886931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"90'6","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-28T18:04:54.618740+0000","last_change":"2024-03-28T18:04:48.501790+0000","last_active":"2024-03-28T18:04:54.618740+0000","last_peered":"2024-03-28T18:04:54.618740+0000","last_clean":"2024-03-28T18:04:54.618740+0000","last_became_active":"2024-03-28T18:04:48.501415+0000","last_became_peered":"2024-03-28T18:04:48.501415+0000","last_unstale":"2024-03-28T18:04:54.618740+0000","last_undegraded":"2024-03-28T18:04:54.618740+0000","last_fullsized":"2024-03-28T18:04:54.618740+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:04:47.412563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:04:47.412563+0000","last_clean_scrub_stamp":"2024-03-28T18:04:47.412563+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T19:17:22.319883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"22'137","reported_seq":361,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-28T18:04:51.314050+0000","last_change":"2024-03-28T18:03:13.522588+0000","last_active":"2024-03-28T18:04:51.314050+0000","last_peered":"2024-03-28T18:04:51.314050+0000","last_clean":"2024-03-28T18:04:51.314050+0000","last_became_active":"2024-03-28T18:03:13.521185+0000","last_became_peered":"2024-03-28T18:03:13.521185+0000","last_unstale":"2024-03-28T18:04:51.314050+0000","last_undegraded":"2024-03-28T18:04:51.314050+0000","last_fullsized":"2024-03-28T18:04:51.314050+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T17:58:33.169981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T17:58:33.169981+0000","last_clean_scrub_stamp":"2024-03-28T17:58:33.169981+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T19:10:06.087374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":13,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-28T18:04:50.513606+0000","last_change":"2024-03-28T18:04:49.538619+0000","last_active":"2024-03-28T18:04:50.513606+0000","last_peered":"2024-03-28T18:04:50.513606+0000","last_clean":"2024-03-28T18:04:50.513606+0000","last_became_active":"2024-03-28T18:04:49.538276+0000","last_became_peered":"2024-03-28T18:04:49.538276+0000","last_unstale":"2024-03-28T18:04:50.513606+0000","last_undegraded":"2024-03-28T18:04:50.513606+0000","last_fullsized":"2024-03-28T18:04:50.513606+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_clean_scrub_stamp":"2024-03-28T18:04:48.465760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T01:39:37.139544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"90'4","reported_seq":19,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-28T18:04:53.618532+0000","last_change":"2024-03-28T18:04:48.513621+0000","last_active":"2024-03-28T18:04:53.618532+0000","last_peered":"2024-03-28T18:04:53.618532+0000","last_clean":"2024-03-28T18:04:53.618532+0000","last_became_active":"2024-03-28T18:04:48.513019+0000","last_became_peered":"2024-03-28T18:04:48.513019+0000","last_unstale":"2024-03-28T18:04:53.618532+0000","last_undegraded":"2024-03-28T18:04:53.618532+0000","last_fullsized":"2024-03-28T18:04:53.618532+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:04:47.412563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:04:47.412563+0000","last_clean_scrub_stamp":"2024-03-28T18:04:47.412563+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T23:38:18.082830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"90'1","reported_seq":16,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-28T18:04:50.871920+0000","last_change":"2024-03-28T18:04:48.513999+0000","last_active":"2024-03-28T18:04:50.871920+0000","last_peered":"2024-03-28T18:04:50.871920+0000","last_clean":"2024-03-28T18:04:50.871920+0000","last_became_active":"2024-03-28T18:04:48.512836+0000","last_became_peered":"2024-03-28T18:04:48.512836+0000","last_unstale":"2024-03-28T18:04:50.871920+0000","last_undegraded":"2024-03-28T18:04:50.871920+0000","last_fullsized":"2024-03-28T18:04:50.871920+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:04:47.412563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:04:47.412563+0000","last_clean_scrub_stamp":"2024-03-28T18:04:47.412563+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T18:29:42.015843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":13,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-28T18:04:50.513833+0000","last_change":"2024-03-28T18:04:49.519323+0000","last_active":"2024-03-28T18:04:50.513833+0000","last_peered":"2024-03-28T18:04:50.513833+0000","last_clean":"2024-03-28T18:04:50.513833+0000","last_became_active":"2024-03-28T18:04:49.518898+0000","last_became_peered":"2024-03-28T18:04:49.518898+0000","last_unstale":"2024-03-28T18:04:50.513833+0000","last_undegraded":"2024-03-28T18:04:50.513833+0000","last_fullsized":"2024-03-28T18:04:50.513833+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_clean_scrub_stamp":"2024-03-28T18:04:48.465760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T18:41:12.683606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"90'7","reported_seq":22,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-28T18:04:54.627777+0000","last_change":"2024-03-28T18:04:48.512673+0000","last_active":"2024-03-28T18:04:54.627777+0000","last_peered":"2024-03-28T18:04:54.627777+0000","last_clean":"2024-03-28T18:04:54.627777+0000","last_became_active":"2024-03-28T18:04:48.512137+0000","last_became_peered":"2024-03-28T18:04:48.512137+0000","last_unstale":"2024-03-28T18:04:54.627777+0000","last_undegraded":"2024-03-28T18:04:54.627777+0000","last_fullsized":"2024-03-28T18:04:54.627777+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:04:47.412563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:04:47.412563+0000","last_clean_scrub_stamp":"2024-03-28T18:04:47.412563+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T21:29:41.731701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":13,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-28T18:04:50.871544+0000","last_change":"2024-03-28T18:04:49.525055+0000","last_active":"2024-03-28T18:04:50.871544+0000","last_peered":"2024-03-28T18:04:50.871544+0000","last_clean":"2024-03-28T18:04:50.871544+0000","last_became_active":"2024-03-28T18:04:49.522038+0000","last_became_peered":"2024-03-28T18:04:49.522038+0000","last_unstale":"2024-03-28T18:04:50.871544+0000","last_undegraded":"2024-03-28T18:04:50.871544+0000","last_fullsized":"2024-03-28T18:04:50.871544+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_clean_scrub_stamp":"2024-03-28T18:04:48.465760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T05:21:44.970397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"90'7","reported_seq":22,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-28T18:04:55.649798+0000","last_change":"2024-03-28T18:04:48.509951+0000","last_active":"2024-03-28T18:04:55.649798+0000","last_peered":"2024-03-28T18:04:55.649798+0000","last_clean":"2024-03-28T18:04:55.649798+0000","last_became_active":"2024-03-28T18:04:48.509141+0000","last_became_peered":"2024-03-28T18:04:48.509141+0000","last_unstale":"2024-03-28T18:04:55.649798+0000","last_undegraded":"2024-03-28T18:04:55.649798+0000","last_fullsized":"2024-03-28T18:04:55.649798+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:04:47.412563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:04:47.412563+0000","last_clean_scrub_stamp":"2024-03-28T18:04:47.412563+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T03:54:05.569940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":13,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-28T18:04:51.313317+0000","last_change":"2024-03-28T18:04:49.537174+0000","last_active":"2024-03-28T18:04:51.313317+0000","last_peered":"2024-03-28T18:04:51.313317+0000","last_clean":"2024-03-28T18:04:51.313317+0000","last_became_active":"2024-03-28T18:04:49.536801+0000","last_became_peered":"2024-03-28T18:04:49.536801+0000","last_unstale":"2024-03-28T18:04:51.313317+0000","last_undegraded":"2024-03-28T18:04:51.313317+0000","last_fullsized":"2024-03-28T18:04:51.313317+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_clean_scrub_stamp":"2024-03-28T18:04:48.465760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T05:27:55.908196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"90'2","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-28T18:04:52.596526+0000","last_change":"2024-03-28T18:04:48.505498+0000","last_active":"2024-03-28T18:04:52.596526+0000","last_peered":"2024-03-28T18:04:52.596526+0000","last_clean":"2024-03-28T18:04:52.596526+0000","last_became_active":"2024-03-28T18:04:48.505116+0000","last_became_peered":"2024-03-28T18:04:48.505116+0000","last_unstale":"2024-03-28T18:04:52.596526+0000","last_undegraded":"2024-03-28T18:04:52.596526+0000","last_fullsized":"2024-03-28T18:04:52.596526+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:04:47.412563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:04:47.412563+0000","last_clean_scrub_stamp":"2024-03-28T18:04:47.412563+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T23:34:53.694339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":13,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-28T18:04:50.871345+0000","last_change":"2024-03-28T18:04:49.523970+0000","last_active":"2024-03-28T18:04:50.871345+0000","last_peered":"2024-03-28T18:04:50.871345+0000","last_clean":"2024-03-28T18:04:50.871345+0000","last_became_active":"2024-03-28T18:04:49.523559+0000","last_became_peered":"2024-03-28T18:04:49.523559+0000","last_unstale":"2024-03-28T18:04:50.871345+0000","last_undegraded":"2024-03-28T18:04:50.871345+0000","last_fullsized":"2024-03-28T18:04:50.871345+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_clean_scrub_stamp":"2024-03-28T18:04:48.465760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T05:50:25.593232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"90'7","reported_seq":22,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-28T18:04:55.651538+0000","last_change":"2024-03-28T18:04:48.492067+0000","last_active":"2024-03-28T18:04:55.651538+0000","last_peered":"2024-03-28T18:04:55.651538+0000","last_clean":"2024-03-28T18:04:55.651538+0000","last_became_active":"2024-03-28T18:04:48.490633+0000","last_became_peered":"2024-03-28T18:04:48.490633+0000","last_unstale":"2024-03-28T18:04:55.651538+0000","last_undegraded":"2024-03-28T18:04:55.651538+0000","last_fullsized":"2024-03-28T18:04:55.651538+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:04:47.412563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:04:47.412563+0000","last_clean_scrub_stamp":"2024-03-28T18:04:47.412563+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T23:00:52.703618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":13,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-28T18:04:51.313030+0000","last_change":"2024-03-28T18:04:49.531554+0000","last_active":"2024-03-28T18:04:51.313030+0000","last_peered":"2024-03-28T18:04:51.313030+0000","last_clean":"2024-03-28T18:04:51.313030+0000","last_became_active":"2024-03-28T18:04:49.530273+0000","last_became_peered":"2024-03-28T18:04:49.530273+0000","last_unstale":"2024-03-28T18:04:51.313030+0000","last_undegraded":"2024-03-28T18:04:51.313030+0000","last_fullsized":"2024-03-28T18:04:51.313030+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_clean_scrub_stamp":"2024-03-28T18:04:48.465760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T22:17:04.032614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"90'3","reported_seq":18,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-28T18:04:54.626801+0000","last_change":"2024-03-28T18:04:48.514974+0000","last_active":"2024-03-28T18:04:54.626801+0000","last_peered":"2024-03-28T18:04:54.626801+0000","last_clean":"2024-03-28T18:04:54.626801+0000","last_became_active":"2024-03-28T18:04:48.514356+0000","last_became_peered":"2024-03-28T18:04:48.514356+0000","last_unstale":"2024-03-28T18:04:54.626801+0000","last_undegraded":"2024-03-28T18:04:54.626801+0000","last_fullsized":"2024-03-28T18:04:54.626801+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:04:47.412563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:04:47.412563+0000","last_clean_scrub_stamp":"2024-03-28T18:04:47.412563+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T01:26:37.202440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":13,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-28T18:04:50.871187+0000","last_change":"2024-03-28T18:04:49.518064+0000","last_active":"2024-03-28T18:04:50.871187+0000","last_peered":"2024-03-28T18:04:50.871187+0000","last_clean":"2024-03-28T18:04:50.871187+0000","last_became_active":"2024-03-28T18:04:49.517674+0000","last_became_peered":"2024-03-28T18:04:49.517674+0000","last_unstale":"2024-03-28T18:04:50.871187+0000","last_undegraded":"2024-03-28T18:04:50.871187+0000","last_fullsized":"2024-03-28T18:04:50.871187+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_clean_scrub_stamp":"2024-03-28T18:04:48.465760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T00:10:44.537164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"90'4","reported_seq":19,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-28T18:04:52.602055+0000","last_change":"2024-03-28T18:04:48.517028+0000","last_active":"2024-03-28T18:04:52.602055+0000","last_peered":"2024-03-28T18:04:52.602055+0000","last_clean":"2024-03-28T18:04:52.602055+0000","last_became_active":"2024-03-28T18:04:48.516387+0000","last_became_peered":"2024-03-28T18:04:48.516387+0000","last_unstale":"2024-03-28T18:04:52.602055+0000","last_undegraded":"2024-03-28T18:04:52.602055+0000","last_fullsized":"2024-03-28T18:04:52.602055+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:04:47.412563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:04:47.412563+0000","last_clean_scrub_stamp":"2024-03-28T18:04:47.412563+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T01:09:24.142422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":13,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-28T18:04:51.314727+0000","last_change":"2024-03-28T18:04:49.527032+0000","last_active":"2024-03-28T18:04:51.314727+0000","last_peered":"2024-03-28T18:04:51.314727+0000","last_clean":"2024-03-28T18:04:51.314727+0000","last_became_active":"2024-03-28T18:04:49.523888+0000","last_became_peered":"2024-03-28T18:04:49.523888+0000","last_unstale":"2024-03-28T18:04:51.314727+0000","last_undegraded":"2024-03-28T18:04:51.314727+0000","last_fullsized":"2024-03-28T18:04:51.314727+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_clean_scrub_stamp":"2024-03-28T18:04:48.465760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T01:19:01.265719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"90'5","reported_seq":23,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-28T18:04:56.663825+0000","last_change":"2024-03-28T18:04:48.502638+0000","last_active":"2024-03-28T18:04:56.663825+0000","last_peered":"2024-03-28T18:04:56.663825+0000","last_clean":"2024-03-28T18:04:56.663825+0000","last_became_active":"2024-03-28T18:04:48.502249+0000","last_became_peered":"2024-03-28T18:04:48.502249+0000","last_unstale":"2024-03-28T18:04:56.663825+0000","last_undegraded":"2024-03-28T18:04:56.663825+0000","last_fullsized":"2024-03-28T18:04:56.663825+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:04:47.412563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:04:47.412563+0000","last_clean_scrub_stamp":"2024-03-28T18:04:47.412563+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T03:23:51.740832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1170,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":2,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":13,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-28T18:04:51.315004+0000","last_change":"2024-03-28T18:04:49.532802+0000","last_active":"2024-03-28T18:04:51.315004+0000","last_peered":"2024-03-28T18:04:51.315004+0000","last_clean":"2024-03-28T18:04:51.315004+0000","last_became_active":"2024-03-28T18:04:49.530666+0000","last_became_peered":"2024-03-28T18:04:49.530666+0000","last_unstale":"2024-03-28T18:04:51.315004+0000","last_undegraded":"2024-03-28T18:04:51.315004+0000","last_fullsized":"2024-03-28T18:04:51.315004+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_clean_scrub_stamp":"2024-03-28T18:04:48.465760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T22:21:28.469402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"90'2","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-28T18:04:50.638662+0000","last_change":"2024-03-28T18:04:48.503582+0000","last_active":"2024-03-28T18:04:50.638662+0000","last_peered":"2024-03-28T18:04:50.638662+0000","last_clean":"2024-03-28T18:04:50.638662+0000","last_became_active":"2024-03-28T18:04:48.502716+0000","last_became_peered":"2024-03-28T18:04:48.502716+0000","last_unstale":"2024-03-28T18:04:50.638662+0000","last_undegraded":"2024-03-28T18:04:50.638662+0000","last_fullsized":"2024-03-28T18:04:50.638662+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:04:47.412563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:04:47.412563+0000","last_clean_scrub_stamp":"2024-03-28T18:04:47.412563+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T21:41:06.333420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"90'1","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-28T18:04:52.829660+0000","last_change":"2024-03-28T18:04:49.534083+0000","last_active":"2024-03-28T18:04:52.829660+0000","last_peered":"2024-03-28T18:04:52.829660+0000","last_clean":"2024-03-28T18:04:52.829660+0000","last_became_active":"2024-03-28T18:04:49.533646+0000","last_became_peered":"2024-03-28T18:04:49.533646+0000","last_unstale":"2024-03-28T18:04:52.829660+0000","last_undegraded":"2024-03-28T18:04:52.829660+0000","last_fullsized":"2024-03-28T18:04:52.829660+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_clean_scrub_stamp":"2024-03-28T18:04:48.465760+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T21:15:13.018986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"90'5","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-28T18:04:55.652444+0000","last_change":"2024-03-28T18:04:48.507658+0000","last_active":"2024-03-28T18:04:55.652444+0000","last_peered":"2024-03-28T18:04:55.652444+0000","last_clean":"2024-03-28T18:04:55.652444+0000","last_became_active":"2024-03-28T18:04:48.506855+0000","last_became_peered":"2024-03-28T18:04:48.506855+0000","last_unstale":"2024-03-28T18:04:55.652444+0000","last_undegraded":"2024-03-28T18:04:55.652444+0000","last_fullsized":"2024-03-28T18:04:55.652444+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:04:47.412563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:04:47.412563+0000","last_clean_scrub_stamp":"2024-03-28T18:04:47.412563+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T23:08:23.550379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":13,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-28T18:04:51.315134+0000","last_change":"2024-03-28T18:04:49.521122+0000","last_active":"2024-03-28T18:04:51.315134+0000","last_peered":"2024-03-28T18:04:51.315134+0000","last_clean":"2024-03-28T18:04:51.315134+0000","last_became_active":"2024-03-28T18:04:49.520711+0000","last_became_peered":"2024-03-28T18:04:49.520711+0000","last_unstale":"2024-03-28T18:04:51.315134+0000","last_undegraded":"2024-03-28T18:04:51.315134+0000","last_fullsized":"2024-03-28T18:04:51.315134+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_clean_scrub_stamp":"2024-03-28T18:04:48.465760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T19:32:33.556979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":13,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-28T18:04:51.313210+0000","last_change":"2024-03-28T18:04:49.534929+0000","last_active":"2024-03-28T18:04:51.313210+0000","last_peered":"2024-03-28T18:04:51.313210+0000","last_clean":"2024-03-28T18:04:51.313210+0000","last_became_active":"2024-03-28T18:04:49.534485+0000","last_became_peered":"2024-03-28T18:04:49.534485+0000","last_unstale":"2024-03-28T18:04:51.313210+0000","last_undegraded":"2024-03-28T18:04:51.313210+0000","last_fullsized":"2024-03-28T18:04:51.313210+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_clean_scrub_stamp":"2024-03-28T18:04:48.465760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T01:47:34.937536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":13,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-28T18:04:51.312446+0000","last_change":"2024-03-28T18:04:49.534822+0000","last_active":"2024-03-28T18:04:51.312446+0000","last_peered":"2024-03-28T18:04:51.312446+0000","last_clean":"2024-03-28T18:04:51.312446+0000","last_became_active":"2024-03-28T18:04:49.534495+0000","last_became_peered":"2024-03-28T18:04:49.534495+0000","last_unstale":"2024-03-28T18:04:51.312446+0000","last_undegraded":"2024-03-28T18:04:51.312446+0000","last_fullsized":"2024-03-28T18:04:51.312446+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_clean_scrub_stamp":"2024-03-28T18:04:48.465760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T01:38:06.312054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":13,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-28T18:04:51.312331+0000","last_change":"2024-03-28T18:04:49.531999+0000","last_active":"2024-03-28T18:04:51.312331+0000","last_peered":"2024-03-28T18:04:51.312331+0000","last_clean":"2024-03-28T18:04:51.312331+0000","last_became_active":"2024-03-28T18:04:49.528641+0000","last_became_peered":"2024-03-28T18:04:49.528641+0000","last_unstale":"2024-03-28T18:04:51.312331+0000","last_undegraded":"2024-03-28T18:04:51.312331+0000","last_fullsized":"2024-03-28T18:04:51.312331+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_clean_scrub_stamp":"2024-03-28T18:04:48.465760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T21:24:43.861635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":13,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-28T18:04:51.313076+0000","last_change":"2024-03-28T18:04:49.540417+0000","last_active":"2024-03-28T18:04:51.313076+0000","last_peered":"2024-03-28T18:04:51.313076+0000","last_clean":"2024-03-28T18:04:51.313076+0000","last_became_active":"2024-03-28T18:04:49.539987+0000","last_became_peered":"2024-03-28T18:04:49.539987+0000","last_unstale":"2024-03-28T18:04:51.313076+0000","last_undegraded":"2024-03-28T18:04:51.313076+0000","last_fullsized":"2024-03-28T18:04:51.313076+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_clean_scrub_stamp":"2024-03-28T18:04:48.465760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T02:53:41.444850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":13,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-28T18:04:50.513862+0000","last_change":"2024-03-28T18:04:49.540014+0000","last_active":"2024-03-28T18:04:50.513862+0000","last_peered":"2024-03-28T18:04:50.513862+0000","last_clean":"2024-03-28T18:04:50.513862+0000","last_became_active":"2024-03-28T18:04:49.538239+0000","last_became_peered":"2024-03-28T18:04:49.538239+0000","last_unstale":"2024-03-28T18:04:50.513862+0000","last_undegraded":"2024-03-28T18:04:50.513862+0000","last_fullsized":"2024-03-28T18:04:50.513862+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_clean_scrub_stamp":"2024-03-28T18:04:48.465760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T23:03:58.449336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":13,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-28T18:04:51.313332+0000","last_change":"2024-03-28T18:04:49.538628+0000","last_active":"2024-03-28T18:04:51.313332+0000","last_peered":"2024-03-28T18:04:51.313332+0000","last_clean":"2024-03-28T18:04:51.313332+0000","last_became_active":"2024-03-28T18:04:49.538299+0000","last_became_peered":"2024-03-28T18:04:49.538299+0000","last_unstale":"2024-03-28T18:04:51.313332+0000","last_undegraded":"2024-03-28T18:04:51.313332+0000","last_fullsized":"2024-03-28T18:04:51.313332+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_clean_scrub_stamp":"2024-03-28T18:04:48.465760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T18:45:21.117985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":13,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-28T18:04:51.313178+0000","last_change":"2024-03-28T18:04:49.531135+0000","last_active":"2024-03-28T18:04:51.313178+0000","last_peered":"2024-03-28T18:04:51.313178+0000","last_clean":"2024-03-28T18:04:51.313178+0000","last_became_active":"2024-03-28T18:04:49.530673+0000","last_became_peered":"2024-03-28T18:04:49.530673+0000","last_unstale":"2024-03-28T18:04:51.313178+0000","last_undegraded":"2024-03-28T18:04:51.313178+0000","last_fullsized":"2024-03-28T18:04:51.313178+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_clean_scrub_stamp":"2024-03-28T18:04:48.465760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T05:51:13.950946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":13,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-28T18:04:50.870682+0000","last_change":"2024-03-28T18:04:49.526803+0000","last_active":"2024-03-28T18:04:50.870682+0000","last_peered":"2024-03-28T18:04:50.870682+0000","last_clean":"2024-03-28T18:04:50.870682+0000","last_became_active":"2024-03-28T18:04:49.526474+0000","last_became_peered":"2024-03-28T18:04:49.526474+0000","last_unstale":"2024-03-28T18:04:50.870682+0000","last_undegraded":"2024-03-28T18:04:50.870682+0000","last_fullsized":"2024-03-28T18:04:50.870682+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_clean_scrub_stamp":"2024-03-28T18:04:48.465760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T01:28:27.530864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":13,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-28T18:04:50.520426+0000","last_change":"2024-03-28T18:04:49.534015+0000","last_active":"2024-03-28T18:04:50.520426+0000","last_peered":"2024-03-28T18:04:50.520426+0000","last_clean":"2024-03-28T18:04:50.520426+0000","last_became_active":"2024-03-28T18:04:49.533560+0000","last_became_peered":"2024-03-28T18:04:49.533560+0000","last_unstale":"2024-03-28T18:04:50.520426+0000","last_undegraded":"2024-03-28T18:04:50.520426+0000","last_fullsized":"2024-03-28T18:04:50.520426+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_clean_scrub_stamp":"2024-03-28T18:04:48.465760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T22:41:30.965737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":13,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-28T18:04:50.870214+0000","last_change":"2024-03-28T18:04:49.523240+0000","last_active":"2024-03-28T18:04:50.870214+0000","last_peered":"2024-03-28T18:04:50.870214+0000","last_clean":"2024-03-28T18:04:50.870214+0000","last_became_active":"2024-03-28T18:04:49.522795+0000","last_became_peered":"2024-03-28T18:04:49.522795+0000","last_unstale":"2024-03-28T18:04:50.870214+0000","last_undegraded":"2024-03-28T18:04:50.870214+0000","last_fullsized":"2024-03-28T18:04:50.870214+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_clean_scrub_stamp":"2024-03-28T18:04:48.465760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T23:08:37.087977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":13,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-28T18:04:50.517509+0000","last_change":"2024-03-28T18:04:49.530965+0000","last_active":"2024-03-28T18:04:50.517509+0000","last_peered":"2024-03-28T18:04:50.517509+0000","last_clean":"2024-03-28T18:04:50.517509+0000","last_became_active":"2024-03-28T18:04:49.530619+0000","last_became_peered":"2024-03-28T18:04:49.530619+0000","last_unstale":"2024-03-28T18:04:50.517509+0000","last_undegraded":"2024-03-28T18:04:50.517509+0000","last_fullsized":"2024-03-28T18:04:50.517509+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_clean_scrub_stamp":"2024-03-28T18:04:48.465760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T05:29:26.754012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":13,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-28T18:04:50.514038+0000","last_change":"2024-03-28T18:04:49.523017+0000","last_active":"2024-03-28T18:04:50.514038+0000","last_peered":"2024-03-28T18:04:50.514038+0000","last_clean":"2024-03-28T18:04:50.514038+0000","last_became_active":"2024-03-28T18:04:49.520411+0000","last_became_peered":"2024-03-28T18:04:49.520411+0000","last_unstale":"2024-03-28T18:04:50.514038+0000","last_undegraded":"2024-03-28T18:04:50.514038+0000","last_fullsized":"2024-03-28T18:04:50.514038+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_clean_scrub_stamp":"2024-03-28T18:04:48.465760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T22:52:45.893036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":13,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-28T18:04:51.313045+0000","last_change":"2024-03-28T18:04:49.544272+0000","last_active":"2024-03-28T18:04:51.313045+0000","last_peered":"2024-03-28T18:04:51.313045+0000","last_clean":"2024-03-28T18:04:51.313045+0000","last_became_active":"2024-03-28T18:04:49.541754+0000","last_became_peered":"2024-03-28T18:04:49.541754+0000","last_unstale":"2024-03-28T18:04:51.313045+0000","last_undegraded":"2024-03-28T18:04:51.313045+0000","last_fullsized":"2024-03-28T18:04:51.313045+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_clean_scrub_stamp":"2024-03-28T18:04:48.465760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T19:06:00.799916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":351,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1,"ondisk_log_size":1,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4751360,"data_stored":4722944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":8054,"num_objects":85,"num_object_clones":0,"num_object_copies":255,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":85,"num_whiteouts":0,"num_read":8,"num_read_kb":5,"num_write":182,"num_write_kb":61,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":56,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":356352,"data_stored":41777,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":157,"ondisk_log_size":157,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":85,"seq":365072220177,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27964,"kb_used_data":948,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717188,"statfs":{"total":95995035648,"available":95966400512,"internally_reserved":0,"allocated":970752,"data_stored":465737,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":10,"up_from":77,"seq":330712481817,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32756,"kb_used_data":1580,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712396,"statfs":{"total":95995035648,"available":95961493504,"internally_reserved":0,"allocated":1617920,"data_stored":1059419,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":9,"up_from":70,"seq":300647710751,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36308,"kb_used_data":972,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708844,"statfs":{"total":95995035648,"available":95957856256,"internally_reserved":0,"allocated":995328,"data_stored":466645,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":36174272},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":8,"up_from":62,"seq":266287972389,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32136,"kb_used_data":960,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713016,"statfs":{"total":95995035648,"available":95962128384,"internally_reserved":0,"allocated":983040,"data_stored":466363,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[3,2,2,12],"upper_bound":16},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":7,"up_from":53,"seq":227633266732,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28580,"kb_used_data":1564,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716572,"statfs":{"total":95995035648,"available":95965769728,"internally_reserved":0,"allocated":1601536,"data_stored":1060146,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":48,"seq":206158430258,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36292,"kb_used_data":956,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708860,"statfs":{"total":95995035648,"available":95957872640,"internally_reserved":0,"allocated":978944,"data_stored":465849,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574934,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36296,"kb_used_data":960,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708856,"statfs":{"total":95995035648,"available":95957868544,"internally_reserved":0,"allocated":983040,"data_stored":467349,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738461,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28004,"kb_used_data":988,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717148,"statfs":{"total":95995035648,"available":95966359552,"internally_reserved":0,"allocated":1011712,"data_stored":467915,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411409,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36344,"kb_used_data":1008,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708808,"statfs":{"total":95995035648,"available":95957819392,"internally_reserved":0,"allocated":1032192,"data_stored":488995,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":3,"up_from":26,"seq":111669149769,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36316,"kb_used_data":980,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708836,"statfs":{"total":95995035648,"available":95957848064,"internally_reserved":0,"allocated":1003520,"data_stored":466284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":36174280},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":4,"up_from":34,"seq":146028888129,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27980,"kb_used_data":964,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717172,"statfs":{"total":95995035648,"available":95966384128,"internally_reserved":0,"allocated":987136,"data_stored":466794,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":40,"seq":171798691900,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36904,"kb_used_data":1568,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708248,"statfs":{"total":95995035648,"available":95957245952,"internally_reserved":0,"allocated":1605632,"data_stored":1058343,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2178,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":23258,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1091,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2629,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":722,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":3924,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":908,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":3705,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-28T18:04:59.253 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-28T18:04:59.512 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:59.512+0000 7f6bc2400640 1 -- 172.21.15.22:0/2279693446 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6bbc0ffb70 msgr2=0x7f6bbc0fff90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:04:59.513 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:59.512+0000 7f6bc2400640 1 --2- 172.21.15.22:0/2279693446 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6bbc0ffb70 0x7f6bbc0fff90 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f6bb0009900 tx=0x7f6bb002bc40 comp rx=0 tx=0).stop 2024-03-28T18:04:59.513 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:59.512+0000 7f6bc2400640 1 -- 172.21.15.22:0/2279693446 shutdown_connections 2024-03-28T18:04:59.513 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:59.512+0000 7f6bc2400640 1 --2- 172.21.15.22:0/2279693446 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f6bbc101720 0x7f6bbc107fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:04:59.513 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:59.512+0000 7f6bc2400640 1 --2- 172.21.15.22:0/2279693446 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6bbc100d60 0x7f6bbc1011e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:04:59.513 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:59.512+0000 7f6bc2400640 1 --2- 172.21.15.22:0/2279693446 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6bbc0ffb70 0x7f6bbc0fff90 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:04:59.513 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:59.512+0000 7f6bc2400640 1 -- 172.21.15.22:0/2279693446 >> 172.21.15.22:0/2279693446 conn(0x7f6bbc0fb0e0 msgr2=0x7f6bbc0fd520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:04:59.513 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:59.512+0000 7f6bc2400640 1 -- 172.21.15.22:0/2279693446 shutdown_connections 2024-03-28T18:04:59.513 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:59.513+0000 7f6bc2400640 1 -- 172.21.15.22:0/2279693446 wait complete. 2024-03-28T18:04:59.515 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:59.513+0000 7f6bc2400640 1 Processor -- start 2024-03-28T18:04:59.515 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:59.514+0000 7f6bc2400640 1 -- start start 2024-03-28T18:04:59.515 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:59.514+0000 7f6bc2400640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6bbc0ffb70 0x7f6bbc199a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:04:59.515 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:59.514+0000 7f6bc2400640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6bbc100d60 0x7f6bbc199fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:04:59.515 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:59.514+0000 7f6bc2400640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f6bbc101720 0x7f6bbc1a1020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:04:59.516 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:59.514+0000 7f6bc2400640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6bbc10af50 con 0x7f6bbc100d60 2024-03-28T18:04:59.516 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:59.514+0000 7f6bc2400640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6bbc10add0 con 0x7f6bbc101720 2024-03-28T18:04:59.516 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:59.514+0000 7f6bc2400640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6bbc10b0d0 con 0x7f6bbc0ffb70 2024-03-28T18:04:59.516 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:59.514+0000 7f6bbbe00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6bbc100d60 0x7f6bbc199fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:04:59.516 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:59.514+0000 7f6bbbe00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6bbc100d60 0x7f6bbc199fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:47540/0 (socket says 172.21.15.22:47540) 2024-03-28T18:04:59.516 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:59.514+0000 7f6bbbe00640 1 -- 172.21.15.22:0/1082159139 learned_addr learned my addr 172.21.15.22:0/1082159139 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-28T18:04:59.516 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:59.514+0000 7f6bc1a00640 1 --2- 172.21.15.22:0/1082159139 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f6bbc101720 0x7f6bbc1a1020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:04:59.516 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:59.514+0000 7f6bc1000640 1 --2- 172.21.15.22:0/1082159139 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6bbc0ffb70 0x7f6bbc199a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:04:59.516 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:59.515+0000 7f6bbbe00640 1 -- 172.21.15.22:0/1082159139 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6bbc0ffb70 msgr2=0x7f6bbc199a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:04:59.516 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:59.515+0000 7f6bbbe00640 1 --2- 172.21.15.22:0/1082159139 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6bbc0ffb70 0x7f6bbc199a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:04:59.516 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:59.515+0000 7f6bbbe00640 1 -- 172.21.15.22:0/1082159139 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f6bbc101720 msgr2=0x7f6bbc1a1020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:04:59.516 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:59.515+0000 7f6bbbe00640 1 --2- 172.21.15.22:0/1082159139 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f6bbc101720 0x7f6bbc1a1020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:04:59.516 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:59.515+0000 7f6bbbe00640 1 -- 172.21.15.22:0/1082159139 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6bbc1a1690 con 0x7f6bbc100d60 2024-03-28T18:04:59.516 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:59.515+0000 7f6bbbe00640 1 --2- 172.21.15.22:0/1082159139 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6bbc100d60 0x7f6bbc199fb0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f6ba800c890 tx=0x7f6ba800cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:04:59.516 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:59.515+0000 7f6bb9600640 1 -- 172.21.15.22:0/1082159139 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ba8004220 con 0x7f6bbc100d60 2024-03-28T18:04:59.517 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:59.515+0000 7f6bc2400640 1 -- 172.21.15.22:0/1082159139 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6bbc1a1980 con 0x7f6bbc100d60 2024-03-28T18:04:59.517 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:59.515+0000 7f6bb9600640 1 -- 172.21.15.22:0/1082159139 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f6ba80043c0 con 0x7f6bbc100d60 2024-03-28T18:04:59.517 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:59.515+0000 7f6bb9600640 1 -- 172.21.15.22:0/1082159139 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ba8004a00 con 0x7f6bbc100d60 2024-03-28T18:04:59.517 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:59.515+0000 7f6bc2400640 1 -- 172.21.15.22:0/1082159139 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6bbc1a1f20 con 0x7f6bbc100d60 2024-03-28T18:04:59.517 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:59.516+0000 7f6bc2400640 1 -- 172.21.15.22:0/1082159139 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6bbc106ec0 con 0x7f6bbc100d60 2024-03-28T18:04:59.519 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:59.519+0000 7f6bb9600640 1 -- 172.21.15.22:0/1082159139 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 17) v1 ==== 137341+0+0 (secure 0 0 0) 0x7f6ba8004ba0 con 0x7f6bbc100d60 2024-03-28T18:04:59.520 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:59.520+0000 7f6bb9600640 1 --2- 172.21.15.22:0/1082159139 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7f6ba40a2890 0x7f6ba40a4d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:04:59.520 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:59.521+0000 7f6bb9600640 1 -- 172.21.15.22:0/1082159139 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f6ba80ce8a0 con 0x7f6bbc100d60 2024-03-28T18:04:59.521 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:59.521+0000 7f6bc1000640 1 --2- 172.21.15.22:0/1082159139 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7f6ba40a2890 0x7f6ba40a4d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:04:59.521 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:59.521+0000 7f6bc1000640 1 --2- 172.21.15.22:0/1082159139 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7f6ba40a2890 0x7f6ba40a4d50 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f6bb0006010 tx=0x7f6bb002df90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:04:59.524 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:59.525+0000 7f6bb9600640 1 -- 172.21.15.22:0/1082159139 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f6ba8014030 con 0x7f6bbc100d60 2024-03-28T18:04:59.718 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:59.719+0000 7f6bc2400640 1 -- 172.21.15.22:0/1082159139 --> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f6bbc061ae0 con 0x7f6ba40a2890 2024-03-28T18:04:59.741 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:59.741+0000 7f6bb9600640 1 -- 172.21.15.22:0/1082159139 <== mgr.14152 v2:172.21.15.22:6800/4201048392 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+258868 (secure 0 0 0) 0x7f6bbc061ae0 con 0x7f6ba40a2890 2024-03-28T18:04:59.745 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-03-28T18:04:59.747 INFO:teuthology.orchestra.run.smithi022.stderr:dumped all 2024-03-28T18:04:59.748 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:59.747+0000 7f6bc2400640 1 -- 172.21.15.22:0/1082159139 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7f6ba40a2890 msgr2=0x7f6ba40a4d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:04:59.748 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:59.747+0000 7f6bc2400640 1 --2- 172.21.15.22:0/1082159139 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7f6ba40a2890 0x7f6ba40a4d50 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f6bb0006010 tx=0x7f6bb002df90 comp rx=0 tx=0).stop 2024-03-28T18:04:59.748 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:59.747+0000 7f6bc2400640 1 -- 172.21.15.22:0/1082159139 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6bbc100d60 msgr2=0x7f6bbc199fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:04:59.748 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:59.747+0000 7f6bc2400640 1 --2- 172.21.15.22:0/1082159139 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6bbc100d60 0x7f6bbc199fb0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f6ba800c890 tx=0x7f6ba800cd50 comp rx=0 tx=0).stop 2024-03-28T18:04:59.748 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:59.747+0000 7f6bc2400640 1 -- 172.21.15.22:0/1082159139 shutdown_connections 2024-03-28T18:04:59.748 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:59.747+0000 7f6bc2400640 1 --2- 172.21.15.22:0/1082159139 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7f6ba40a2890 0x7f6ba40a4d50 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:04:59.749 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:59.747+0000 7f6bc2400640 1 --2- 172.21.15.22:0/1082159139 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f6bbc101720 0x7f6bbc1a1020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:04:59.749 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:59.747+0000 7f6bc2400640 1 --2- 172.21.15.22:0/1082159139 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6bbc100d60 0x7f6bbc199fb0 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:04:59.749 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:59.747+0000 7f6bc2400640 1 --2- 172.21.15.22:0/1082159139 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6bbc0ffb70 0x7f6bbc199a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:04:59.749 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:59.747+0000 7f6bc2400640 1 -- 172.21.15.22:0/1082159139 >> 172.21.15.22:0/1082159139 conn(0x7f6bbc0fb0e0 msgr2=0x7f6bbc104090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:04:59.749 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:59.747+0000 7f6bc2400640 1 -- 172.21.15.22:0/1082159139 shutdown_connections 2024-03-28T18:04:59.749 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:04:59.747+0000 7f6bc2400640 1 -- 172.21.15.22:0/1082159139 wait complete. 2024-03-28T18:04:59.766 INFO:teuthology.orchestra.run.smithi022.stdout:{"pg_ready":true,"pg_map":{"version":374,"stamp":"2024-03-28T18:04:58.601732+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":598539,"num_objects":88,"num_object_clones":0,"num_object_copies":264,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":88,"num_whiteouts":0,"num_read":144,"num_read_kb":124,"num_write":364,"num_write_kb":3000,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":56,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":295,"ondisk_log_size":295,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":395880,"kb_used_data":13448,"kb_used_omap":18,"kb_used_meta":382317,"kb_avail":1124545944,"statfs":{"total":1151940427776,"available":1151535046656,"internally_reserved":0,"allocated":13770752,"data_stored":7399839,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19141,"internal_metadata":391492923},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[3,2,2,12],"upper_bound":16},"perf_stat":{"commit_latency_ms":25,"apply_latency_ms":25,"commit_latency_ns":25000000,"apply_latency_ns":25000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":8171,"num_objects":86,"num_object_clones":0,"num_object_copies":258,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":86,"num_whiteouts":0,"num_read":8,"num_read_kb":5,"num_write":183,"num_write_kb":62,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":56,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"9.108825"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":13,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-28T18:04:51.313643+0000","last_change":"2024-03-28T18:04:49.545022+0000","last_active":"2024-03-28T18:04:51.313643+0000","last_peered":"2024-03-28T18:04:51.313643+0000","last_clean":"2024-03-28T18:04:51.313643+0000","last_became_active":"2024-03-28T18:04:49.544584+0000","last_became_peered":"2024-03-28T18:04:49.544584+0000","last_unstale":"2024-03-28T18:04:51.313643+0000","last_undegraded":"2024-03-28T18:04:51.313643+0000","last_fullsized":"2024-03-28T18:04:51.313643+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_clean_scrub_stamp":"2024-03-28T18:04:48.465760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T01:47:05.644293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":13,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-28T18:04:51.313105+0000","last_change":"2024-03-28T18:04:49.526720+0000","last_active":"2024-03-28T18:04:51.313105+0000","last_peered":"2024-03-28T18:04:51.313105+0000","last_clean":"2024-03-28T18:04:51.313105+0000","last_became_active":"2024-03-28T18:04:49.524545+0000","last_became_peered":"2024-03-28T18:04:49.524545+0000","last_unstale":"2024-03-28T18:04:51.313105+0000","last_undegraded":"2024-03-28T18:04:51.313105+0000","last_fullsized":"2024-03-28T18:04:51.313105+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_clean_scrub_stamp":"2024-03-28T18:04:48.465760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T01:00:54.926308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":13,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-28T18:04:51.313306+0000","last_change":"2024-03-28T18:04:49.532425+0000","last_active":"2024-03-28T18:04:51.313306+0000","last_peered":"2024-03-28T18:04:51.313306+0000","last_clean":"2024-03-28T18:04:51.313306+0000","last_became_active":"2024-03-28T18:04:49.531146+0000","last_became_peered":"2024-03-28T18:04:49.531146+0000","last_unstale":"2024-03-28T18:04:51.313306+0000","last_undegraded":"2024-03-28T18:04:51.313306+0000","last_fullsized":"2024-03-28T18:04:51.313306+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_clean_scrub_stamp":"2024-03-28T18:04:48.465760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T04:58:45.131773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":13,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-28T18:04:50.575241+0000","last_change":"2024-03-28T18:04:49.533371+0000","last_active":"2024-03-28T18:04:50.575241+0000","last_peered":"2024-03-28T18:04:50.575241+0000","last_clean":"2024-03-28T18:04:50.575241+0000","last_became_active":"2024-03-28T18:04:49.532544+0000","last_became_peered":"2024-03-28T18:04:49.532544+0000","last_unstale":"2024-03-28T18:04:50.575241+0000","last_undegraded":"2024-03-28T18:04:50.575241+0000","last_fullsized":"2024-03-28T18:04:50.575241+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_clean_scrub_stamp":"2024-03-28T18:04:48.465760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T20:36:25.664137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":13,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-28T18:04:50.871493+0000","last_change":"2024-03-28T18:04:49.530875+0000","last_active":"2024-03-28T18:04:50.871493+0000","last_peered":"2024-03-28T18:04:50.871493+0000","last_clean":"2024-03-28T18:04:50.871493+0000","last_became_active":"2024-03-28T18:04:49.530354+0000","last_became_peered":"2024-03-28T18:04:49.530354+0000","last_unstale":"2024-03-28T18:04:50.871493+0000","last_undegraded":"2024-03-28T18:04:50.871493+0000","last_fullsized":"2024-03-28T18:04:50.871493+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_clean_scrub_stamp":"2024-03-28T18:04:48.465760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T21:02:17.187989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":13,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-28T18:04:51.313314+0000","last_change":"2024-03-28T18:04:49.538260+0000","last_active":"2024-03-28T18:04:51.313314+0000","last_peered":"2024-03-28T18:04:51.313314+0000","last_clean":"2024-03-28T18:04:51.313314+0000","last_became_active":"2024-03-28T18:04:49.537863+0000","last_became_peered":"2024-03-28T18:04:49.537863+0000","last_unstale":"2024-03-28T18:04:51.313314+0000","last_undegraded":"2024-03-28T18:04:51.313314+0000","last_fullsized":"2024-03-28T18:04:51.313314+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_clean_scrub_stamp":"2024-03-28T18:04:48.465760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T03:01:25.363839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":13,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-28T18:04:51.314134+0000","last_change":"2024-03-28T18:04:49.534636+0000","last_active":"2024-03-28T18:04:51.314134+0000","last_peered":"2024-03-28T18:04:51.314134+0000","last_clean":"2024-03-28T18:04:51.314134+0000","last_became_active":"2024-03-28T18:04:49.534236+0000","last_became_peered":"2024-03-28T18:04:49.534236+0000","last_unstale":"2024-03-28T18:04:51.314134+0000","last_undegraded":"2024-03-28T18:04:51.314134+0000","last_fullsized":"2024-03-28T18:04:51.314134+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_clean_scrub_stamp":"2024-03-28T18:04:48.465760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T05:55:23.616641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":13,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-28T18:04:50.870443+0000","last_change":"2024-03-28T18:04:49.534848+0000","last_active":"2024-03-28T18:04:50.870443+0000","last_peered":"2024-03-28T18:04:50.870443+0000","last_clean":"2024-03-28T18:04:50.870443+0000","last_became_active":"2024-03-28T18:04:49.534302+0000","last_became_peered":"2024-03-28T18:04:49.534302+0000","last_unstale":"2024-03-28T18:04:50.870443+0000","last_undegraded":"2024-03-28T18:04:50.870443+0000","last_fullsized":"2024-03-28T18:04:50.870443+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_clean_scrub_stamp":"2024-03-28T18:04:48.465760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T05:48:24.707305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":13,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-28T18:04:50.517515+0000","last_change":"2024-03-28T18:04:49.536568+0000","last_active":"2024-03-28T18:04:50.517515+0000","last_peered":"2024-03-28T18:04:50.517515+0000","last_clean":"2024-03-28T18:04:50.517515+0000","last_became_active":"2024-03-28T18:04:49.535675+0000","last_became_peered":"2024-03-28T18:04:49.535675+0000","last_unstale":"2024-03-28T18:04:50.517515+0000","last_undegraded":"2024-03-28T18:04:50.517515+0000","last_fullsized":"2024-03-28T18:04:50.517515+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_clean_scrub_stamp":"2024-03-28T18:04:48.465760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T02:32:15.179869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":13,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-28T18:04:51.313844+0000","last_change":"2024-03-28T18:04:49.539363+0000","last_active":"2024-03-28T18:04:51.313844+0000","last_peered":"2024-03-28T18:04:51.313844+0000","last_clean":"2024-03-28T18:04:51.313844+0000","last_became_active":"2024-03-28T18:04:49.539002+0000","last_became_peered":"2024-03-28T18:04:49.539002+0000","last_unstale":"2024-03-28T18:04:51.313844+0000","last_undegraded":"2024-03-28T18:04:51.313844+0000","last_fullsized":"2024-03-28T18:04:51.313844+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_clean_scrub_stamp":"2024-03-28T18:04:48.465760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T18:24:28.846081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":13,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-28T18:04:51.314984+0000","last_change":"2024-03-28T18:04:49.532385+0000","last_active":"2024-03-28T18:04:51.314984+0000","last_peered":"2024-03-28T18:04:51.314984+0000","last_clean":"2024-03-28T18:04:51.314984+0000","last_became_active":"2024-03-28T18:04:49.532004+0000","last_became_peered":"2024-03-28T18:04:49.532004+0000","last_unstale":"2024-03-28T18:04:51.314984+0000","last_undegraded":"2024-03-28T18:04:51.314984+0000","last_fullsized":"2024-03-28T18:04:51.314984+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_clean_scrub_stamp":"2024-03-28T18:04:48.465760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T19:05:13.959313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":13,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-28T18:04:50.574994+0000","last_change":"2024-03-28T18:04:49.534465+0000","last_active":"2024-03-28T18:04:50.574994+0000","last_peered":"2024-03-28T18:04:50.574994+0000","last_clean":"2024-03-28T18:04:50.574994+0000","last_became_active":"2024-03-28T18:04:49.532989+0000","last_became_peered":"2024-03-28T18:04:49.532989+0000","last_unstale":"2024-03-28T18:04:50.574994+0000","last_undegraded":"2024-03-28T18:04:50.574994+0000","last_fullsized":"2024-03-28T18:04:50.574994+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_clean_scrub_stamp":"2024-03-28T18:04:48.465760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T23:56:50.607770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":13,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-28T18:04:50.514240+0000","last_change":"2024-03-28T18:04:49.535205+0000","last_active":"2024-03-28T18:04:50.514240+0000","last_peered":"2024-03-28T18:04:50.514240+0000","last_clean":"2024-03-28T18:04:50.514240+0000","last_became_active":"2024-03-28T18:04:49.533025+0000","last_became_peered":"2024-03-28T18:04:49.533025+0000","last_unstale":"2024-03-28T18:04:50.514240+0000","last_undegraded":"2024-03-28T18:04:50.514240+0000","last_fullsized":"2024-03-28T18:04:50.514240+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_clean_scrub_stamp":"2024-03-28T18:04:48.465760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T21:41:51.815907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":13,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-28T18:04:51.312708+0000","last_change":"2024-03-28T18:04:49.520974+0000","last_active":"2024-03-28T18:04:51.312708+0000","last_peered":"2024-03-28T18:04:51.312708+0000","last_clean":"2024-03-28T18:04:51.312708+0000","last_became_active":"2024-03-28T18:04:49.520509+0000","last_became_peered":"2024-03-28T18:04:49.520509+0000","last_unstale":"2024-03-28T18:04:51.312708+0000","last_undegraded":"2024-03-28T18:04:51.312708+0000","last_fullsized":"2024-03-28T18:04:51.312708+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_clean_scrub_stamp":"2024-03-28T18:04:48.465760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T03:15:16.346859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":13,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-28T18:04:51.312748+0000","last_change":"2024-03-28T18:04:49.539053+0000","last_active":"2024-03-28T18:04:51.312748+0000","last_peered":"2024-03-28T18:04:51.312748+0000","last_clean":"2024-03-28T18:04:51.312748+0000","last_became_active":"2024-03-28T18:04:49.537327+0000","last_became_peered":"2024-03-28T18:04:49.537327+0000","last_unstale":"2024-03-28T18:04:51.312748+0000","last_undegraded":"2024-03-28T18:04:51.312748+0000","last_fullsized":"2024-03-28T18:04:51.312748+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_clean_scrub_stamp":"2024-03-28T18:04:48.465760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T00:24:42.319504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":13,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-28T18:04:50.516563+0000","last_change":"2024-03-28T18:04:49.525855+0000","last_active":"2024-03-28T18:04:50.516563+0000","last_peered":"2024-03-28T18:04:50.516563+0000","last_clean":"2024-03-28T18:04:50.516563+0000","last_became_active":"2024-03-28T18:04:49.525448+0000","last_became_peered":"2024-03-28T18:04:49.525448+0000","last_unstale":"2024-03-28T18:04:50.516563+0000","last_undegraded":"2024-03-28T18:04:50.516563+0000","last_fullsized":"2024-03-28T18:04:50.516563+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_clean_scrub_stamp":"2024-03-28T18:04:48.465760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T23:53:00.452692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":13,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-28T18:04:51.313695+0000","last_change":"2024-03-28T18:04:49.543605+0000","last_active":"2024-03-28T18:04:51.313695+0000","last_peered":"2024-03-28T18:04:51.313695+0000","last_clean":"2024-03-28T18:04:51.313695+0000","last_became_active":"2024-03-28T18:04:49.543280+0000","last_became_peered":"2024-03-28T18:04:49.543280+0000","last_unstale":"2024-03-28T18:04:51.313695+0000","last_undegraded":"2024-03-28T18:04:51.313695+0000","last_fullsized":"2024-03-28T18:04:51.313695+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_clean_scrub_stamp":"2024-03-28T18:04:48.465760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T21:27:26.216639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":13,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-28T18:04:50.866144+0000","last_change":"2024-03-28T18:04:49.526745+0000","last_active":"2024-03-28T18:04:50.866144+0000","last_peered":"2024-03-28T18:04:50.866144+0000","last_clean":"2024-03-28T18:04:50.866144+0000","last_became_active":"2024-03-28T18:04:49.526351+0000","last_became_peered":"2024-03-28T18:04:49.526351+0000","last_unstale":"2024-03-28T18:04:50.866144+0000","last_undegraded":"2024-03-28T18:04:50.866144+0000","last_fullsized":"2024-03-28T18:04:50.866144+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_clean_scrub_stamp":"2024-03-28T18:04:48.465760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T05:31:47.687345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":13,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-28T18:04:51.312740+0000","last_change":"2024-03-28T18:04:49.523346+0000","last_active":"2024-03-28T18:04:51.312740+0000","last_peered":"2024-03-28T18:04:51.312740+0000","last_clean":"2024-03-28T18:04:51.312740+0000","last_became_active":"2024-03-28T18:04:49.522041+0000","last_became_peered":"2024-03-28T18:04:49.522041+0000","last_unstale":"2024-03-28T18:04:51.312740+0000","last_undegraded":"2024-03-28T18:04:51.312740+0000","last_fullsized":"2024-03-28T18:04:51.312740+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_clean_scrub_stamp":"2024-03-28T18:04:48.465760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T01:35:02.419328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"90'11","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-28T18:04:54.627438+0000","last_change":"2024-03-28T18:04:48.510917+0000","last_active":"2024-03-28T18:04:54.627438+0000","last_peered":"2024-03-28T18:04:54.627438+0000","last_clean":"2024-03-28T18:04:54.627438+0000","last_became_active":"2024-03-28T18:04:48.509978+0000","last_became_peered":"2024-03-28T18:04:48.509978+0000","last_unstale":"2024-03-28T18:04:54.627438+0000","last_undegraded":"2024-03-28T18:04:54.627438+0000","last_fullsized":"2024-03-28T18:04:54.627438+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:04:47.412563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:04:47.412563+0000","last_clean_scrub_stamp":"2024-03-28T18:04:47.412563+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T03:28:13.185099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":13,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-28T18:04:50.871301+0000","last_change":"2024-03-28T18:04:49.535543+0000","last_active":"2024-03-28T18:04:50.871301+0000","last_peered":"2024-03-28T18:04:50.871301+0000","last_clean":"2024-03-28T18:04:50.871301+0000","last_became_active":"2024-03-28T18:04:49.535171+0000","last_became_peered":"2024-03-28T18:04:49.535171+0000","last_unstale":"2024-03-28T18:04:50.871301+0000","last_undegraded":"2024-03-28T18:04:50.871301+0000","last_fullsized":"2024-03-28T18:04:50.871301+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_clean_scrub_stamp":"2024-03-28T18:04:48.465760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T22:12:22.258424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"90'2","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-28T18:04:50.871501+0000","last_change":"2024-03-28T18:04:48.503190+0000","last_active":"2024-03-28T18:04:50.871501+0000","last_peered":"2024-03-28T18:04:50.871501+0000","last_clean":"2024-03-28T18:04:50.871501+0000","last_became_active":"2024-03-28T18:04:48.501858+0000","last_became_peered":"2024-03-28T18:04:48.501858+0000","last_unstale":"2024-03-28T18:04:50.871501+0000","last_undegraded":"2024-03-28T18:04:50.871501+0000","last_fullsized":"2024-03-28T18:04:50.871501+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:04:47.412563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:04:47.412563+0000","last_clean_scrub_stamp":"2024-03-28T18:04:47.412563+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T19:56:43.878630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":13,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-28T18:04:51.313439+0000","last_change":"2024-03-28T18:04:49.537739+0000","last_active":"2024-03-28T18:04:51.313439+0000","last_peered":"2024-03-28T18:04:51.313439+0000","last_clean":"2024-03-28T18:04:51.313439+0000","last_became_active":"2024-03-28T18:04:49.535990+0000","last_became_peered":"2024-03-28T18:04:49.535990+0000","last_unstale":"2024-03-28T18:04:51.313439+0000","last_undegraded":"2024-03-28T18:04:51.313439+0000","last_fullsized":"2024-03-28T18:04:51.313439+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_clean_scrub_stamp":"2024-03-28T18:04:48.465760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T21:54:15.729494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"90'4","reported_seq":19,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-28T18:04:53.617983+0000","last_change":"2024-03-28T18:04:48.514268+0000","last_active":"2024-03-28T18:04:53.617983+0000","last_peered":"2024-03-28T18:04:53.617983+0000","last_clean":"2024-03-28T18:04:53.617983+0000","last_became_active":"2024-03-28T18:04:48.513602+0000","last_became_peered":"2024-03-28T18:04:48.513602+0000","last_unstale":"2024-03-28T18:04:53.617983+0000","last_undegraded":"2024-03-28T18:04:53.617983+0000","last_fullsized":"2024-03-28T18:04:53.617983+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:04:47.412563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:04:47.412563+0000","last_clean_scrub_stamp":"2024-03-28T18:04:47.412563+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T01:12:17.758721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":13,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-28T18:04:50.516993+0000","last_change":"2024-03-28T18:04:49.529316+0000","last_active":"2024-03-28T18:04:50.516993+0000","last_peered":"2024-03-28T18:04:50.516993+0000","last_clean":"2024-03-28T18:04:50.516993+0000","last_became_active":"2024-03-28T18:04:49.528975+0000","last_became_peered":"2024-03-28T18:04:49.528975+0000","last_unstale":"2024-03-28T18:04:50.516993+0000","last_undegraded":"2024-03-28T18:04:50.516993+0000","last_fullsized":"2024-03-28T18:04:50.516993+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_clean_scrub_stamp":"2024-03-28T18:04:48.465760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T00:06:45.268757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"89'2","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-28T18:04:50.871893+0000","last_change":"2024-03-28T18:04:48.515841+0000","last_active":"2024-03-28T18:04:50.871893+0000","last_peered":"2024-03-28T18:04:50.871893+0000","last_clean":"2024-03-28T18:04:50.871893+0000","last_became_active":"2024-03-28T18:04:48.513852+0000","last_became_peered":"2024-03-28T18:04:48.513852+0000","last_unstale":"2024-03-28T18:04:50.871893+0000","last_undegraded":"2024-03-28T18:04:50.871893+0000","last_fullsized":"2024-03-28T18:04:50.871893+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:04:47.412563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:04:47.412563+0000","last_clean_scrub_stamp":"2024-03-28T18:04:47.412563+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T00:07:14.751545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":13,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-28T18:04:50.521476+0000","last_change":"2024-03-28T18:04:49.542238+0000","last_active":"2024-03-28T18:04:50.521476+0000","last_peered":"2024-03-28T18:04:50.521476+0000","last_clean":"2024-03-28T18:04:50.521476+0000","last_became_active":"2024-03-28T18:04:49.539723+0000","last_became_peered":"2024-03-28T18:04:49.539723+0000","last_unstale":"2024-03-28T18:04:50.521476+0000","last_undegraded":"2024-03-28T18:04:50.521476+0000","last_fullsized":"2024-03-28T18:04:50.521476+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_clean_scrub_stamp":"2024-03-28T18:04:48.465760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T22:45:07.310148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"90'6","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-28T18:04:52.604487+0000","last_change":"2024-03-28T18:04:48.511349+0000","last_active":"2024-03-28T18:04:52.604487+0000","last_peered":"2024-03-28T18:04:52.604487+0000","last_clean":"2024-03-28T18:04:52.604487+0000","last_became_active":"2024-03-28T18:04:48.510990+0000","last_became_peered":"2024-03-28T18:04:48.510990+0000","last_unstale":"2024-03-28T18:04:52.604487+0000","last_undegraded":"2024-03-28T18:04:52.604487+0000","last_fullsized":"2024-03-28T18:04:52.604487+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:04:47.412563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:04:47.412563+0000","last_clean_scrub_stamp":"2024-03-28T18:04:47.412563+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T21:56:15.411149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1013,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":13,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-28T18:04:50.872146+0000","last_change":"2024-03-28T18:04:49.538066+0000","last_active":"2024-03-28T18:04:50.872146+0000","last_peered":"2024-03-28T18:04:50.872146+0000","last_clean":"2024-03-28T18:04:50.872146+0000","last_became_active":"2024-03-28T18:04:49.537202+0000","last_became_peered":"2024-03-28T18:04:49.537202+0000","last_unstale":"2024-03-28T18:04:50.872146+0000","last_undegraded":"2024-03-28T18:04:50.872146+0000","last_fullsized":"2024-03-28T18:04:50.872146+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_clean_scrub_stamp":"2024-03-28T18:04:48.465760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T22:35:19.060311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"90'7","reported_seq":22,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-28T18:04:54.623927+0000","last_change":"2024-03-28T18:04:48.513722+0000","last_active":"2024-03-28T18:04:54.623927+0000","last_peered":"2024-03-28T18:04:54.623927+0000","last_clean":"2024-03-28T18:04:54.623927+0000","last_became_active":"2024-03-28T18:04:48.513048+0000","last_became_peered":"2024-03-28T18:04:48.513048+0000","last_unstale":"2024-03-28T18:04:54.623927+0000","last_undegraded":"2024-03-28T18:04:54.623927+0000","last_fullsized":"2024-03-28T18:04:54.623927+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:04:47.412563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:04:47.412563+0000","last_clean_scrub_stamp":"2024-03-28T18:04:47.412563+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T04:48:10.831297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":13,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-28T18:04:50.520925+0000","last_change":"2024-03-28T18:04:49.539172+0000","last_active":"2024-03-28T18:04:50.520925+0000","last_peered":"2024-03-28T18:04:50.520925+0000","last_clean":"2024-03-28T18:04:50.520925+0000","last_became_active":"2024-03-28T18:04:49.538779+0000","last_became_peered":"2024-03-28T18:04:49.538779+0000","last_unstale":"2024-03-28T18:04:50.520925+0000","last_undegraded":"2024-03-28T18:04:50.520925+0000","last_fullsized":"2024-03-28T18:04:50.520925+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_clean_scrub_stamp":"2024-03-28T18:04:48.465760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T04:59:08.361762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"90'3","reported_seq":18,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-28T18:04:53.613193+0000","last_change":"2024-03-28T18:04:48.500471+0000","last_active":"2024-03-28T18:04:53.613193+0000","last_peered":"2024-03-28T18:04:53.613193+0000","last_clean":"2024-03-28T18:04:53.613193+0000","last_became_active":"2024-03-28T18:04:48.500094+0000","last_became_peered":"2024-03-28T18:04:48.500094+0000","last_unstale":"2024-03-28T18:04:53.613193+0000","last_undegraded":"2024-03-28T18:04:53.613193+0000","last_fullsized":"2024-03-28T18:04:53.613193+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:04:47.412563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:04:47.412563+0000","last_clean_scrub_stamp":"2024-03-28T18:04:47.412563+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T21:52:51.964254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":13,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-28T18:04:50.870317+0000","last_change":"2024-03-28T18:04:49.522120+0000","last_active":"2024-03-28T18:04:50.870317+0000","last_peered":"2024-03-28T18:04:50.870317+0000","last_clean":"2024-03-28T18:04:50.870317+0000","last_became_active":"2024-03-28T18:04:49.521656+0000","last_became_peered":"2024-03-28T18:04:49.521656+0000","last_unstale":"2024-03-28T18:04:50.870317+0000","last_undegraded":"2024-03-28T18:04:50.870317+0000","last_fullsized":"2024-03-28T18:04:50.870317+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_clean_scrub_stamp":"2024-03-28T18:04:48.465760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T18:29:55.605765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"90'2","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-28T18:04:52.597054+0000","last_change":"2024-03-28T18:04:48.508723+0000","last_active":"2024-03-28T18:04:52.597054+0000","last_peered":"2024-03-28T18:04:52.597054+0000","last_clean":"2024-03-28T18:04:52.597054+0000","last_became_active":"2024-03-28T18:04:48.508314+0000","last_became_peered":"2024-03-28T18:04:48.508314+0000","last_unstale":"2024-03-28T18:04:52.597054+0000","last_undegraded":"2024-03-28T18:04:52.597054+0000","last_fullsized":"2024-03-28T18:04:52.597054+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:04:47.412563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:04:47.412563+0000","last_clean_scrub_stamp":"2024-03-28T18:04:47.412563+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T03:03:59.353454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":13,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-28T18:04:50.865374+0000","last_change":"2024-03-28T18:04:49.531469+0000","last_active":"2024-03-28T18:04:50.865374+0000","last_peered":"2024-03-28T18:04:50.865374+0000","last_clean":"2024-03-28T18:04:50.865374+0000","last_became_active":"2024-03-28T18:04:49.529263+0000","last_became_peered":"2024-03-28T18:04:49.529263+0000","last_unstale":"2024-03-28T18:04:50.865374+0000","last_undegraded":"2024-03-28T18:04:50.865374+0000","last_fullsized":"2024-03-28T18:04:50.865374+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_clean_scrub_stamp":"2024-03-28T18:04:48.465760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T05:46:31.593877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"90'4","reported_seq":19,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-28T18:04:52.599084+0000","last_change":"2024-03-28T18:04:48.500644+0000","last_active":"2024-03-28T18:04:52.599084+0000","last_peered":"2024-03-28T18:04:52.599084+0000","last_clean":"2024-03-28T18:04:52.599084+0000","last_became_active":"2024-03-28T18:04:48.499330+0000","last_became_peered":"2024-03-28T18:04:48.499330+0000","last_unstale":"2024-03-28T18:04:52.599084+0000","last_undegraded":"2024-03-28T18:04:52.599084+0000","last_fullsized":"2024-03-28T18:04:52.599084+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:04:47.412563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:04:47.412563+0000","last_clean_scrub_stamp":"2024-03-28T18:04:47.412563+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T18:50:54.361266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":13,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-28T18:04:50.517486+0000","last_change":"2024-03-28T18:04:49.528728+0000","last_active":"2024-03-28T18:04:50.517486+0000","last_peered":"2024-03-28T18:04:50.517486+0000","last_clean":"2024-03-28T18:04:50.517486+0000","last_became_active":"2024-03-28T18:04:49.528312+0000","last_became_peered":"2024-03-28T18:04:49.528312+0000","last_unstale":"2024-03-28T18:04:50.517486+0000","last_undegraded":"2024-03-28T18:04:50.517486+0000","last_fullsized":"2024-03-28T18:04:50.517486+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_clean_scrub_stamp":"2024-03-28T18:04:48.465760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T05:59:37.308002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"90'4","reported_seq":19,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-28T18:04:55.646686+0000","last_change":"2024-03-28T18:04:48.508766+0000","last_active":"2024-03-28T18:04:55.646686+0000","last_peered":"2024-03-28T18:04:55.646686+0000","last_clean":"2024-03-28T18:04:55.646686+0000","last_became_active":"2024-03-28T18:04:48.506780+0000","last_became_peered":"2024-03-28T18:04:48.506780+0000","last_unstale":"2024-03-28T18:04:55.646686+0000","last_undegraded":"2024-03-28T18:04:55.646686+0000","last_fullsized":"2024-03-28T18:04:55.646686+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:04:47.412563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:04:47.412563+0000","last_clean_scrub_stamp":"2024-03-28T18:04:47.412563+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T04:39:07.394655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":13,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-28T18:04:51.313379+0000","last_change":"2024-03-28T18:04:49.537165+0000","last_active":"2024-03-28T18:04:51.313379+0000","last_peered":"2024-03-28T18:04:51.313379+0000","last_clean":"2024-03-28T18:04:51.313379+0000","last_became_active":"2024-03-28T18:04:49.534543+0000","last_became_peered":"2024-03-28T18:04:49.534543+0000","last_unstale":"2024-03-28T18:04:51.313379+0000","last_undegraded":"2024-03-28T18:04:51.313379+0000","last_fullsized":"2024-03-28T18:04:51.313379+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_clean_scrub_stamp":"2024-03-28T18:04:48.465760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T03:15:58.056148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"90'5","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-28T18:04:52.598827+0000","last_change":"2024-03-28T18:04:48.503866+0000","last_active":"2024-03-28T18:04:52.598827+0000","last_peered":"2024-03-28T18:04:52.598827+0000","last_clean":"2024-03-28T18:04:52.598827+0000","last_became_active":"2024-03-28T18:04:48.503223+0000","last_became_peered":"2024-03-28T18:04:48.503223+0000","last_unstale":"2024-03-28T18:04:52.598827+0000","last_undegraded":"2024-03-28T18:04:52.598827+0000","last_fullsized":"2024-03-28T18:04:52.598827+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:04:47.412563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:04:47.412563+0000","last_clean_scrub_stamp":"2024-03-28T18:04:47.412563+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T20:14:01.307483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":13,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-28T18:04:50.517779+0000","last_change":"2024-03-28T18:04:49.539494+0000","last_active":"2024-03-28T18:04:50.517779+0000","last_peered":"2024-03-28T18:04:50.517779+0000","last_clean":"2024-03-28T18:04:50.517779+0000","last_became_active":"2024-03-28T18:04:49.539044+0000","last_became_peered":"2024-03-28T18:04:49.539044+0000","last_unstale":"2024-03-28T18:04:50.517779+0000","last_undegraded":"2024-03-28T18:04:50.517779+0000","last_fullsized":"2024-03-28T18:04:50.517779+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_clean_scrub_stamp":"2024-03-28T18:04:48.465760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T04:08:30.807788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"90'8","reported_seq":23,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-28T18:04:52.589290+0000","last_change":"2024-03-28T18:04:48.507703+0000","last_active":"2024-03-28T18:04:52.589290+0000","last_peered":"2024-03-28T18:04:52.589290+0000","last_clean":"2024-03-28T18:04:52.589290+0000","last_became_active":"2024-03-28T18:04:48.507310+0000","last_became_peered":"2024-03-28T18:04:48.507310+0000","last_unstale":"2024-03-28T18:04:52.589290+0000","last_undegraded":"2024-03-28T18:04:52.589290+0000","last_fullsized":"2024-03-28T18:04:52.589290+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:04:47.412563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:04:47.412563+0000","last_clean_scrub_stamp":"2024-03-28T18:04:47.412563+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T19:14:13.910042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":13,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-28T18:04:50.871891+0000","last_change":"2024-03-28T18:04:49.529928+0000","last_active":"2024-03-28T18:04:50.871891+0000","last_peered":"2024-03-28T18:04:50.871891+0000","last_clean":"2024-03-28T18:04:50.871891+0000","last_became_active":"2024-03-28T18:04:49.529527+0000","last_became_peered":"2024-03-28T18:04:49.529527+0000","last_unstale":"2024-03-28T18:04:50.871891+0000","last_undegraded":"2024-03-28T18:04:50.871891+0000","last_fullsized":"2024-03-28T18:04:50.871891+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_clean_scrub_stamp":"2024-03-28T18:04:48.465760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T05:50:07.129140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"90'9","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-28T18:04:56.658257+0000","last_change":"2024-03-28T18:04:48.494746+0000","last_active":"2024-03-28T18:04:56.658257+0000","last_peered":"2024-03-28T18:04:56.658257+0000","last_clean":"2024-03-28T18:04:56.658257+0000","last_became_active":"2024-03-28T18:04:48.494296+0000","last_became_peered":"2024-03-28T18:04:48.494296+0000","last_unstale":"2024-03-28T18:04:56.658257+0000","last_undegraded":"2024-03-28T18:04:56.658257+0000","last_fullsized":"2024-03-28T18:04:56.658257+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:04:47.412563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:04:47.412563+0000","last_clean_scrub_stamp":"2024-03-28T18:04:47.412563+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T22:32:56.923067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":13,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-28T18:04:50.513932+0000","last_change":"2024-03-28T18:04:49.528070+0000","last_active":"2024-03-28T18:04:50.513932+0000","last_peered":"2024-03-28T18:04:50.513932+0000","last_clean":"2024-03-28T18:04:50.513932+0000","last_became_active":"2024-03-28T18:04:49.526747+0000","last_became_peered":"2024-03-28T18:04:49.526747+0000","last_unstale":"2024-03-28T18:04:50.513932+0000","last_undegraded":"2024-03-28T18:04:50.513932+0000","last_fullsized":"2024-03-28T18:04:50.513932+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_clean_scrub_stamp":"2024-03-28T18:04:48.465760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T22:32:05.750094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"90'4","reported_seq":19,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-28T18:04:52.600128+0000","last_change":"2024-03-28T18:04:48.505723+0000","last_active":"2024-03-28T18:04:52.600128+0000","last_peered":"2024-03-28T18:04:52.600128+0000","last_clean":"2024-03-28T18:04:52.600128+0000","last_became_active":"2024-03-28T18:04:48.504088+0000","last_became_peered":"2024-03-28T18:04:48.504088+0000","last_unstale":"2024-03-28T18:04:52.600128+0000","last_undegraded":"2024-03-28T18:04:52.600128+0000","last_fullsized":"2024-03-28T18:04:52.600128+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:04:47.412563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:04:47.412563+0000","last_clean_scrub_stamp":"2024-03-28T18:04:47.412563+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T23:25:37.138054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":13,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-28T18:04:51.314719+0000","last_change":"2024-03-28T18:04:49.518872+0000","last_active":"2024-03-28T18:04:51.314719+0000","last_peered":"2024-03-28T18:04:51.314719+0000","last_clean":"2024-03-28T18:04:51.314719+0000","last_became_active":"2024-03-28T18:04:49.518507+0000","last_became_peered":"2024-03-28T18:04:49.518507+0000","last_unstale":"2024-03-28T18:04:51.314719+0000","last_undegraded":"2024-03-28T18:04:51.314719+0000","last_fullsized":"2024-03-28T18:04:51.314719+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_clean_scrub_stamp":"2024-03-28T18:04:48.465760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T20:11:31.044321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"90'6","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-28T18:04:54.618891+0000","last_change":"2024-03-28T18:04:48.503058+0000","last_active":"2024-03-28T18:04:54.618891+0000","last_peered":"2024-03-28T18:04:54.618891+0000","last_clean":"2024-03-28T18:04:54.618891+0000","last_became_active":"2024-03-28T18:04:48.501596+0000","last_became_peered":"2024-03-28T18:04:48.501596+0000","last_unstale":"2024-03-28T18:04:54.618891+0000","last_undegraded":"2024-03-28T18:04:54.618891+0000","last_fullsized":"2024-03-28T18:04:54.618891+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:04:47.412563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:04:47.412563+0000","last_clean_scrub_stamp":"2024-03-28T18:04:47.412563+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T23:46:10.442771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":979,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":13,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-28T18:04:51.314045+0000","last_change":"2024-03-28T18:04:49.535102+0000","last_active":"2024-03-28T18:04:51.314045+0000","last_peered":"2024-03-28T18:04:51.314045+0000","last_clean":"2024-03-28T18:04:51.314045+0000","last_became_active":"2024-03-28T18:04:49.534171+0000","last_became_peered":"2024-03-28T18:04:49.534171+0000","last_unstale":"2024-03-28T18:04:51.314045+0000","last_undegraded":"2024-03-28T18:04:51.314045+0000","last_fullsized":"2024-03-28T18:04:51.314045+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_clean_scrub_stamp":"2024-03-28T18:04:48.465760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T18:16:27.703327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"90'15","reported_seq":30,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-28T18:04:53.619126+0000","last_change":"2024-03-28T18:04:48.517034+0000","last_active":"2024-03-28T18:04:53.619126+0000","last_peered":"2024-03-28T18:04:53.619126+0000","last_clean":"2024-03-28T18:04:53.619126+0000","last_became_active":"2024-03-28T18:04:48.516562+0000","last_became_peered":"2024-03-28T18:04:48.516562+0000","last_unstale":"2024-03-28T18:04:53.619126+0000","last_undegraded":"2024-03-28T18:04:53.619126+0000","last_fullsized":"2024-03-28T18:04:53.619126+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:04:47.412563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:04:47.412563+0000","last_clean_scrub_stamp":"2024-03-28T18:04:47.412563+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T02:28:55.753779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":13,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-28T18:04:50.520183+0000","last_change":"2024-03-28T18:04:49.541501+0000","last_active":"2024-03-28T18:04:50.520183+0000","last_peered":"2024-03-28T18:04:50.520183+0000","last_clean":"2024-03-28T18:04:50.520183+0000","last_became_active":"2024-03-28T18:04:49.541117+0000","last_became_peered":"2024-03-28T18:04:49.541117+0000","last_unstale":"2024-03-28T18:04:50.520183+0000","last_undegraded":"2024-03-28T18:04:50.520183+0000","last_fullsized":"2024-03-28T18:04:50.520183+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_clean_scrub_stamp":"2024-03-28T18:04:48.465760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T04:18:29.081489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"89'1","reported_seq":16,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-28T18:04:50.871856+0000","last_change":"2024-03-28T18:04:48.510859+0000","last_active":"2024-03-28T18:04:50.871856+0000","last_peered":"2024-03-28T18:04:50.871856+0000","last_clean":"2024-03-28T18:04:50.871856+0000","last_became_active":"2024-03-28T18:04:48.510522+0000","last_became_peered":"2024-03-28T18:04:48.510522+0000","last_unstale":"2024-03-28T18:04:50.871856+0000","last_undegraded":"2024-03-28T18:04:50.871856+0000","last_fullsized":"2024-03-28T18:04:50.871856+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:04:47.412563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:04:47.412563+0000","last_clean_scrub_stamp":"2024-03-28T18:04:47.412563+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T22:31:37.817361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":13,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-28T18:04:51.313494+0000","last_change":"2024-03-28T18:04:49.541981+0000","last_active":"2024-03-28T18:04:51.313494+0000","last_peered":"2024-03-28T18:04:51.313494+0000","last_clean":"2024-03-28T18:04:51.313494+0000","last_became_active":"2024-03-28T18:04:49.541606+0000","last_became_peered":"2024-03-28T18:04:49.541606+0000","last_unstale":"2024-03-28T18:04:51.313494+0000","last_undegraded":"2024-03-28T18:04:51.313494+0000","last_fullsized":"2024-03-28T18:04:51.313494+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_clean_scrub_stamp":"2024-03-28T18:04:48.465760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T21:56:03.632490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"90'1","reported_seq":16,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-28T18:04:50.865724+0000","last_change":"2024-03-28T18:04:48.505847+0000","last_active":"2024-03-28T18:04:50.865724+0000","last_peered":"2024-03-28T18:04:50.865724+0000","last_clean":"2024-03-28T18:04:50.865724+0000","last_became_active":"2024-03-28T18:04:48.501005+0000","last_became_peered":"2024-03-28T18:04:48.501005+0000","last_unstale":"2024-03-28T18:04:50.865724+0000","last_undegraded":"2024-03-28T18:04:50.865724+0000","last_fullsized":"2024-03-28T18:04:50.865724+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:04:47.412563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:04:47.412563+0000","last_clean_scrub_stamp":"2024-03-28T18:04:47.412563+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T01:33:45.539324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":13,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-28T18:04:51.313734+0000","last_change":"2024-03-28T18:04:49.526938+0000","last_active":"2024-03-28T18:04:51.313734+0000","last_peered":"2024-03-28T18:04:51.313734+0000","last_clean":"2024-03-28T18:04:51.313734+0000","last_became_active":"2024-03-28T18:04:49.526290+0000","last_became_peered":"2024-03-28T18:04:49.526290+0000","last_unstale":"2024-03-28T18:04:51.313734+0000","last_undegraded":"2024-03-28T18:04:51.313734+0000","last_fullsized":"2024-03-28T18:04:51.313734+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_clean_scrub_stamp":"2024-03-28T18:04:48.465760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T03:06:54.561470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"90'5","reported_seq":23,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-28T18:04:56.665115+0000","last_change":"2024-03-28T18:04:48.512553+0000","last_active":"2024-03-28T18:04:56.665115+0000","last_peered":"2024-03-28T18:04:56.665115+0000","last_clean":"2024-03-28T18:04:56.665115+0000","last_became_active":"2024-03-28T18:04:48.511520+0000","last_became_peered":"2024-03-28T18:04:48.511520+0000","last_unstale":"2024-03-28T18:04:56.665115+0000","last_undegraded":"2024-03-28T18:04:56.665115+0000","last_fullsized":"2024-03-28T18:04:56.665115+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:04:47.412563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:04:47.412563+0000","last_clean_scrub_stamp":"2024-03-28T18:04:47.412563+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T21:13:32.970027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":3,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":13,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-28T18:04:51.313575+0000","last_change":"2024-03-28T18:04:49.539484+0000","last_active":"2024-03-28T18:04:51.313575+0000","last_peered":"2024-03-28T18:04:51.313575+0000","last_clean":"2024-03-28T18:04:51.313575+0000","last_became_active":"2024-03-28T18:04:49.539066+0000","last_became_peered":"2024-03-28T18:04:49.539066+0000","last_unstale":"2024-03-28T18:04:51.313575+0000","last_undegraded":"2024-03-28T18:04:51.313575+0000","last_fullsized":"2024-03-28T18:04:51.313575+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_clean_scrub_stamp":"2024-03-28T18:04:48.465760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T05:22:32.989771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":13,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-28T18:04:50.520810+0000","last_change":"2024-03-28T18:04:49.540438+0000","last_active":"2024-03-28T18:04:50.520810+0000","last_peered":"2024-03-28T18:04:50.520810+0000","last_clean":"2024-03-28T18:04:50.520810+0000","last_became_active":"2024-03-28T18:04:49.540069+0000","last_became_peered":"2024-03-28T18:04:49.540069+0000","last_unstale":"2024-03-28T18:04:50.520810+0000","last_undegraded":"2024-03-28T18:04:50.520810+0000","last_fullsized":"2024-03-28T18:04:50.520810+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_clean_scrub_stamp":"2024-03-28T18:04:48.465760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T19:31:49.254101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"90'5","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-28T18:04:53.614616+0000","last_change":"2024-03-28T18:04:48.508839+0000","last_active":"2024-03-28T18:04:53.614616+0000","last_peered":"2024-03-28T18:04:53.614616+0000","last_clean":"2024-03-28T18:04:53.614616+0000","last_became_active":"2024-03-28T18:04:48.508475+0000","last_became_peered":"2024-03-28T18:04:48.508475+0000","last_unstale":"2024-03-28T18:04:53.614616+0000","last_undegraded":"2024-03-28T18:04:53.614616+0000","last_fullsized":"2024-03-28T18:04:53.614616+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:04:47.412563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:04:47.412563+0000","last_clean_scrub_stamp":"2024-03-28T18:04:47.412563+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T04:31:10.463284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":13,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-28T18:04:50.871099+0000","last_change":"2024-03-28T18:04:49.534449+0000","last_active":"2024-03-28T18:04:50.871099+0000","last_peered":"2024-03-28T18:04:50.871099+0000","last_clean":"2024-03-28T18:04:50.871099+0000","last_became_active":"2024-03-28T18:04:49.534098+0000","last_became_peered":"2024-03-28T18:04:49.534098+0000","last_unstale":"2024-03-28T18:04:50.871099+0000","last_undegraded":"2024-03-28T18:04:50.871099+0000","last_fullsized":"2024-03-28T18:04:50.871099+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_clean_scrub_stamp":"2024-03-28T18:04:48.465760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T23:47:07.886931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"90'6","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-28T18:04:54.618740+0000","last_change":"2024-03-28T18:04:48.501790+0000","last_active":"2024-03-28T18:04:54.618740+0000","last_peered":"2024-03-28T18:04:54.618740+0000","last_clean":"2024-03-28T18:04:54.618740+0000","last_became_active":"2024-03-28T18:04:48.501415+0000","last_became_peered":"2024-03-28T18:04:48.501415+0000","last_unstale":"2024-03-28T18:04:54.618740+0000","last_undegraded":"2024-03-28T18:04:54.618740+0000","last_fullsized":"2024-03-28T18:04:54.618740+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:04:47.412563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:04:47.412563+0000","last_clean_scrub_stamp":"2024-03-28T18:04:47.412563+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T19:17:22.319883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"22'137","reported_seq":361,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-28T18:04:51.314050+0000","last_change":"2024-03-28T18:03:13.522588+0000","last_active":"2024-03-28T18:04:51.314050+0000","last_peered":"2024-03-28T18:04:51.314050+0000","last_clean":"2024-03-28T18:04:51.314050+0000","last_became_active":"2024-03-28T18:03:13.521185+0000","last_became_peered":"2024-03-28T18:03:13.521185+0000","last_unstale":"2024-03-28T18:04:51.314050+0000","last_undegraded":"2024-03-28T18:04:51.314050+0000","last_fullsized":"2024-03-28T18:04:51.314050+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T17:58:33.169981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T17:58:33.169981+0000","last_clean_scrub_stamp":"2024-03-28T17:58:33.169981+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T19:10:06.087374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":13,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-28T18:04:50.513606+0000","last_change":"2024-03-28T18:04:49.538619+0000","last_active":"2024-03-28T18:04:50.513606+0000","last_peered":"2024-03-28T18:04:50.513606+0000","last_clean":"2024-03-28T18:04:50.513606+0000","last_became_active":"2024-03-28T18:04:49.538276+0000","last_became_peered":"2024-03-28T18:04:49.538276+0000","last_unstale":"2024-03-28T18:04:50.513606+0000","last_undegraded":"2024-03-28T18:04:50.513606+0000","last_fullsized":"2024-03-28T18:04:50.513606+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_clean_scrub_stamp":"2024-03-28T18:04:48.465760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T01:39:37.139544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"90'4","reported_seq":19,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-28T18:04:53.618532+0000","last_change":"2024-03-28T18:04:48.513621+0000","last_active":"2024-03-28T18:04:53.618532+0000","last_peered":"2024-03-28T18:04:53.618532+0000","last_clean":"2024-03-28T18:04:53.618532+0000","last_became_active":"2024-03-28T18:04:48.513019+0000","last_became_peered":"2024-03-28T18:04:48.513019+0000","last_unstale":"2024-03-28T18:04:53.618532+0000","last_undegraded":"2024-03-28T18:04:53.618532+0000","last_fullsized":"2024-03-28T18:04:53.618532+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:04:47.412563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:04:47.412563+0000","last_clean_scrub_stamp":"2024-03-28T18:04:47.412563+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T23:38:18.082830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"90'1","reported_seq":16,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-28T18:04:50.871920+0000","last_change":"2024-03-28T18:04:48.513999+0000","last_active":"2024-03-28T18:04:50.871920+0000","last_peered":"2024-03-28T18:04:50.871920+0000","last_clean":"2024-03-28T18:04:50.871920+0000","last_became_active":"2024-03-28T18:04:48.512836+0000","last_became_peered":"2024-03-28T18:04:48.512836+0000","last_unstale":"2024-03-28T18:04:50.871920+0000","last_undegraded":"2024-03-28T18:04:50.871920+0000","last_fullsized":"2024-03-28T18:04:50.871920+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:04:47.412563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:04:47.412563+0000","last_clean_scrub_stamp":"2024-03-28T18:04:47.412563+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T18:29:42.015843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":13,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-28T18:04:50.513833+0000","last_change":"2024-03-28T18:04:49.519323+0000","last_active":"2024-03-28T18:04:50.513833+0000","last_peered":"2024-03-28T18:04:50.513833+0000","last_clean":"2024-03-28T18:04:50.513833+0000","last_became_active":"2024-03-28T18:04:49.518898+0000","last_became_peered":"2024-03-28T18:04:49.518898+0000","last_unstale":"2024-03-28T18:04:50.513833+0000","last_undegraded":"2024-03-28T18:04:50.513833+0000","last_fullsized":"2024-03-28T18:04:50.513833+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_clean_scrub_stamp":"2024-03-28T18:04:48.465760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T18:41:12.683606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"90'7","reported_seq":22,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-28T18:04:54.627777+0000","last_change":"2024-03-28T18:04:48.512673+0000","last_active":"2024-03-28T18:04:54.627777+0000","last_peered":"2024-03-28T18:04:54.627777+0000","last_clean":"2024-03-28T18:04:54.627777+0000","last_became_active":"2024-03-28T18:04:48.512137+0000","last_became_peered":"2024-03-28T18:04:48.512137+0000","last_unstale":"2024-03-28T18:04:54.627777+0000","last_undegraded":"2024-03-28T18:04:54.627777+0000","last_fullsized":"2024-03-28T18:04:54.627777+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:04:47.412563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:04:47.412563+0000","last_clean_scrub_stamp":"2024-03-28T18:04:47.412563+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T21:29:41.731701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":13,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-28T18:04:50.871544+0000","last_change":"2024-03-28T18:04:49.525055+0000","last_active":"2024-03-28T18:04:50.871544+0000","last_peered":"2024-03-28T18:04:50.871544+0000","last_clean":"2024-03-28T18:04:50.871544+0000","last_became_active":"2024-03-28T18:04:49.522038+0000","last_became_peered":"2024-03-28T18:04:49.522038+0000","last_unstale":"2024-03-28T18:04:50.871544+0000","last_undegraded":"2024-03-28T18:04:50.871544+0000","last_fullsized":"2024-03-28T18:04:50.871544+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_clean_scrub_stamp":"2024-03-28T18:04:48.465760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T05:21:44.970397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"90'7","reported_seq":22,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-28T18:04:55.649798+0000","last_change":"2024-03-28T18:04:48.509951+0000","last_active":"2024-03-28T18:04:55.649798+0000","last_peered":"2024-03-28T18:04:55.649798+0000","last_clean":"2024-03-28T18:04:55.649798+0000","last_became_active":"2024-03-28T18:04:48.509141+0000","last_became_peered":"2024-03-28T18:04:48.509141+0000","last_unstale":"2024-03-28T18:04:55.649798+0000","last_undegraded":"2024-03-28T18:04:55.649798+0000","last_fullsized":"2024-03-28T18:04:55.649798+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:04:47.412563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:04:47.412563+0000","last_clean_scrub_stamp":"2024-03-28T18:04:47.412563+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T03:54:05.569940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":13,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-28T18:04:51.313317+0000","last_change":"2024-03-28T18:04:49.537174+0000","last_active":"2024-03-28T18:04:51.313317+0000","last_peered":"2024-03-28T18:04:51.313317+0000","last_clean":"2024-03-28T18:04:51.313317+0000","last_became_active":"2024-03-28T18:04:49.536801+0000","last_became_peered":"2024-03-28T18:04:49.536801+0000","last_unstale":"2024-03-28T18:04:51.313317+0000","last_undegraded":"2024-03-28T18:04:51.313317+0000","last_fullsized":"2024-03-28T18:04:51.313317+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_clean_scrub_stamp":"2024-03-28T18:04:48.465760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T05:27:55.908196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"90'2","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-28T18:04:52.596526+0000","last_change":"2024-03-28T18:04:48.505498+0000","last_active":"2024-03-28T18:04:52.596526+0000","last_peered":"2024-03-28T18:04:52.596526+0000","last_clean":"2024-03-28T18:04:52.596526+0000","last_became_active":"2024-03-28T18:04:48.505116+0000","last_became_peered":"2024-03-28T18:04:48.505116+0000","last_unstale":"2024-03-28T18:04:52.596526+0000","last_undegraded":"2024-03-28T18:04:52.596526+0000","last_fullsized":"2024-03-28T18:04:52.596526+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:04:47.412563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:04:47.412563+0000","last_clean_scrub_stamp":"2024-03-28T18:04:47.412563+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T23:34:53.694339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":13,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-28T18:04:50.871345+0000","last_change":"2024-03-28T18:04:49.523970+0000","last_active":"2024-03-28T18:04:50.871345+0000","last_peered":"2024-03-28T18:04:50.871345+0000","last_clean":"2024-03-28T18:04:50.871345+0000","last_became_active":"2024-03-28T18:04:49.523559+0000","last_became_peered":"2024-03-28T18:04:49.523559+0000","last_unstale":"2024-03-28T18:04:50.871345+0000","last_undegraded":"2024-03-28T18:04:50.871345+0000","last_fullsized":"2024-03-28T18:04:50.871345+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_clean_scrub_stamp":"2024-03-28T18:04:48.465760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T05:50:25.593232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"90'7","reported_seq":22,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-28T18:04:55.651538+0000","last_change":"2024-03-28T18:04:48.492067+0000","last_active":"2024-03-28T18:04:55.651538+0000","last_peered":"2024-03-28T18:04:55.651538+0000","last_clean":"2024-03-28T18:04:55.651538+0000","last_became_active":"2024-03-28T18:04:48.490633+0000","last_became_peered":"2024-03-28T18:04:48.490633+0000","last_unstale":"2024-03-28T18:04:55.651538+0000","last_undegraded":"2024-03-28T18:04:55.651538+0000","last_fullsized":"2024-03-28T18:04:55.651538+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:04:47.412563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:04:47.412563+0000","last_clean_scrub_stamp":"2024-03-28T18:04:47.412563+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T23:00:52.703618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":13,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-28T18:04:51.313030+0000","last_change":"2024-03-28T18:04:49.531554+0000","last_active":"2024-03-28T18:04:51.313030+0000","last_peered":"2024-03-28T18:04:51.313030+0000","last_clean":"2024-03-28T18:04:51.313030+0000","last_became_active":"2024-03-28T18:04:49.530273+0000","last_became_peered":"2024-03-28T18:04:49.530273+0000","last_unstale":"2024-03-28T18:04:51.313030+0000","last_undegraded":"2024-03-28T18:04:51.313030+0000","last_fullsized":"2024-03-28T18:04:51.313030+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_clean_scrub_stamp":"2024-03-28T18:04:48.465760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T22:17:04.032614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"90'3","reported_seq":18,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-28T18:04:54.626801+0000","last_change":"2024-03-28T18:04:48.514974+0000","last_active":"2024-03-28T18:04:54.626801+0000","last_peered":"2024-03-28T18:04:54.626801+0000","last_clean":"2024-03-28T18:04:54.626801+0000","last_became_active":"2024-03-28T18:04:48.514356+0000","last_became_peered":"2024-03-28T18:04:48.514356+0000","last_unstale":"2024-03-28T18:04:54.626801+0000","last_undegraded":"2024-03-28T18:04:54.626801+0000","last_fullsized":"2024-03-28T18:04:54.626801+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:04:47.412563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:04:47.412563+0000","last_clean_scrub_stamp":"2024-03-28T18:04:47.412563+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T01:26:37.202440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":13,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-28T18:04:50.871187+0000","last_change":"2024-03-28T18:04:49.518064+0000","last_active":"2024-03-28T18:04:50.871187+0000","last_peered":"2024-03-28T18:04:50.871187+0000","last_clean":"2024-03-28T18:04:50.871187+0000","last_became_active":"2024-03-28T18:04:49.517674+0000","last_became_peered":"2024-03-28T18:04:49.517674+0000","last_unstale":"2024-03-28T18:04:50.871187+0000","last_undegraded":"2024-03-28T18:04:50.871187+0000","last_fullsized":"2024-03-28T18:04:50.871187+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_clean_scrub_stamp":"2024-03-28T18:04:48.465760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T00:10:44.537164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"90'4","reported_seq":19,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-28T18:04:52.602055+0000","last_change":"2024-03-28T18:04:48.517028+0000","last_active":"2024-03-28T18:04:52.602055+0000","last_peered":"2024-03-28T18:04:52.602055+0000","last_clean":"2024-03-28T18:04:52.602055+0000","last_became_active":"2024-03-28T18:04:48.516387+0000","last_became_peered":"2024-03-28T18:04:48.516387+0000","last_unstale":"2024-03-28T18:04:52.602055+0000","last_undegraded":"2024-03-28T18:04:52.602055+0000","last_fullsized":"2024-03-28T18:04:52.602055+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:04:47.412563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:04:47.412563+0000","last_clean_scrub_stamp":"2024-03-28T18:04:47.412563+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T01:09:24.142422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":13,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-28T18:04:51.314727+0000","last_change":"2024-03-28T18:04:49.527032+0000","last_active":"2024-03-28T18:04:51.314727+0000","last_peered":"2024-03-28T18:04:51.314727+0000","last_clean":"2024-03-28T18:04:51.314727+0000","last_became_active":"2024-03-28T18:04:49.523888+0000","last_became_peered":"2024-03-28T18:04:49.523888+0000","last_unstale":"2024-03-28T18:04:51.314727+0000","last_undegraded":"2024-03-28T18:04:51.314727+0000","last_fullsized":"2024-03-28T18:04:51.314727+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_clean_scrub_stamp":"2024-03-28T18:04:48.465760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T01:19:01.265719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"90'5","reported_seq":23,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-28T18:04:56.663825+0000","last_change":"2024-03-28T18:04:48.502638+0000","last_active":"2024-03-28T18:04:56.663825+0000","last_peered":"2024-03-28T18:04:56.663825+0000","last_clean":"2024-03-28T18:04:56.663825+0000","last_became_active":"2024-03-28T18:04:48.502249+0000","last_became_peered":"2024-03-28T18:04:48.502249+0000","last_unstale":"2024-03-28T18:04:56.663825+0000","last_undegraded":"2024-03-28T18:04:56.663825+0000","last_fullsized":"2024-03-28T18:04:56.663825+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:04:47.412563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:04:47.412563+0000","last_clean_scrub_stamp":"2024-03-28T18:04:47.412563+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T03:23:51.740832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1170,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":2,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":13,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-28T18:04:51.315004+0000","last_change":"2024-03-28T18:04:49.532802+0000","last_active":"2024-03-28T18:04:51.315004+0000","last_peered":"2024-03-28T18:04:51.315004+0000","last_clean":"2024-03-28T18:04:51.315004+0000","last_became_active":"2024-03-28T18:04:49.530666+0000","last_became_peered":"2024-03-28T18:04:49.530666+0000","last_unstale":"2024-03-28T18:04:51.315004+0000","last_undegraded":"2024-03-28T18:04:51.315004+0000","last_fullsized":"2024-03-28T18:04:51.315004+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_clean_scrub_stamp":"2024-03-28T18:04:48.465760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T22:21:28.469402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"90'2","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-28T18:04:50.638662+0000","last_change":"2024-03-28T18:04:48.503582+0000","last_active":"2024-03-28T18:04:50.638662+0000","last_peered":"2024-03-28T18:04:50.638662+0000","last_clean":"2024-03-28T18:04:50.638662+0000","last_became_active":"2024-03-28T18:04:48.502716+0000","last_became_peered":"2024-03-28T18:04:48.502716+0000","last_unstale":"2024-03-28T18:04:50.638662+0000","last_undegraded":"2024-03-28T18:04:50.638662+0000","last_fullsized":"2024-03-28T18:04:50.638662+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:04:47.412563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:04:47.412563+0000","last_clean_scrub_stamp":"2024-03-28T18:04:47.412563+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T21:41:06.333420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"90'1","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-28T18:04:52.829660+0000","last_change":"2024-03-28T18:04:49.534083+0000","last_active":"2024-03-28T18:04:52.829660+0000","last_peered":"2024-03-28T18:04:52.829660+0000","last_clean":"2024-03-28T18:04:52.829660+0000","last_became_active":"2024-03-28T18:04:49.533646+0000","last_became_peered":"2024-03-28T18:04:49.533646+0000","last_unstale":"2024-03-28T18:04:52.829660+0000","last_undegraded":"2024-03-28T18:04:52.829660+0000","last_fullsized":"2024-03-28T18:04:52.829660+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_clean_scrub_stamp":"2024-03-28T18:04:48.465760+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T21:15:13.018986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"90'5","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-28T18:04:55.652444+0000","last_change":"2024-03-28T18:04:48.507658+0000","last_active":"2024-03-28T18:04:55.652444+0000","last_peered":"2024-03-28T18:04:55.652444+0000","last_clean":"2024-03-28T18:04:55.652444+0000","last_became_active":"2024-03-28T18:04:48.506855+0000","last_became_peered":"2024-03-28T18:04:48.506855+0000","last_unstale":"2024-03-28T18:04:55.652444+0000","last_undegraded":"2024-03-28T18:04:55.652444+0000","last_fullsized":"2024-03-28T18:04:55.652444+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:04:47.412563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:04:47.412563+0000","last_clean_scrub_stamp":"2024-03-28T18:04:47.412563+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T23:08:23.550379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":13,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-28T18:04:51.315134+0000","last_change":"2024-03-28T18:04:49.521122+0000","last_active":"2024-03-28T18:04:51.315134+0000","last_peered":"2024-03-28T18:04:51.315134+0000","last_clean":"2024-03-28T18:04:51.315134+0000","last_became_active":"2024-03-28T18:04:49.520711+0000","last_became_peered":"2024-03-28T18:04:49.520711+0000","last_unstale":"2024-03-28T18:04:51.315134+0000","last_undegraded":"2024-03-28T18:04:51.315134+0000","last_fullsized":"2024-03-28T18:04:51.315134+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_clean_scrub_stamp":"2024-03-28T18:04:48.465760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T19:32:33.556979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":13,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-28T18:04:51.313210+0000","last_change":"2024-03-28T18:04:49.534929+0000","last_active":"2024-03-28T18:04:51.313210+0000","last_peered":"2024-03-28T18:04:51.313210+0000","last_clean":"2024-03-28T18:04:51.313210+0000","last_became_active":"2024-03-28T18:04:49.534485+0000","last_became_peered":"2024-03-28T18:04:49.534485+0000","last_unstale":"2024-03-28T18:04:51.313210+0000","last_undegraded":"2024-03-28T18:04:51.313210+0000","last_fullsized":"2024-03-28T18:04:51.313210+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_clean_scrub_stamp":"2024-03-28T18:04:48.465760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T01:47:34.937536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":13,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-28T18:04:51.312446+0000","last_change":"2024-03-28T18:04:49.534822+0000","last_active":"2024-03-28T18:04:51.312446+0000","last_peered":"2024-03-28T18:04:51.312446+0000","last_clean":"2024-03-28T18:04:51.312446+0000","last_became_active":"2024-03-28T18:04:49.534495+0000","last_became_peered":"2024-03-28T18:04:49.534495+0000","last_unstale":"2024-03-28T18:04:51.312446+0000","last_undegraded":"2024-03-28T18:04:51.312446+0000","last_fullsized":"2024-03-28T18:04:51.312446+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_clean_scrub_stamp":"2024-03-28T18:04:48.465760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T01:38:06.312054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":13,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-28T18:04:51.312331+0000","last_change":"2024-03-28T18:04:49.531999+0000","last_active":"2024-03-28T18:04:51.312331+0000","last_peered":"2024-03-28T18:04:51.312331+0000","last_clean":"2024-03-28T18:04:51.312331+0000","last_became_active":"2024-03-28T18:04:49.528641+0000","last_became_peered":"2024-03-28T18:04:49.528641+0000","last_unstale":"2024-03-28T18:04:51.312331+0000","last_undegraded":"2024-03-28T18:04:51.312331+0000","last_fullsized":"2024-03-28T18:04:51.312331+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_clean_scrub_stamp":"2024-03-28T18:04:48.465760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T21:24:43.861635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":13,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-28T18:04:51.313076+0000","last_change":"2024-03-28T18:04:49.540417+0000","last_active":"2024-03-28T18:04:51.313076+0000","last_peered":"2024-03-28T18:04:51.313076+0000","last_clean":"2024-03-28T18:04:51.313076+0000","last_became_active":"2024-03-28T18:04:49.539987+0000","last_became_peered":"2024-03-28T18:04:49.539987+0000","last_unstale":"2024-03-28T18:04:51.313076+0000","last_undegraded":"2024-03-28T18:04:51.313076+0000","last_fullsized":"2024-03-28T18:04:51.313076+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_clean_scrub_stamp":"2024-03-28T18:04:48.465760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T02:53:41.444850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":13,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-28T18:04:50.513862+0000","last_change":"2024-03-28T18:04:49.540014+0000","last_active":"2024-03-28T18:04:50.513862+0000","last_peered":"2024-03-28T18:04:50.513862+0000","last_clean":"2024-03-28T18:04:50.513862+0000","last_became_active":"2024-03-28T18:04:49.538239+0000","last_became_peered":"2024-03-28T18:04:49.538239+0000","last_unstale":"2024-03-28T18:04:50.513862+0000","last_undegraded":"2024-03-28T18:04:50.513862+0000","last_fullsized":"2024-03-28T18:04:50.513862+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_clean_scrub_stamp":"2024-03-28T18:04:48.465760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T23:03:58.449336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":13,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-28T18:04:51.313332+0000","last_change":"2024-03-28T18:04:49.538628+0000","last_active":"2024-03-28T18:04:51.313332+0000","last_peered":"2024-03-28T18:04:51.313332+0000","last_clean":"2024-03-28T18:04:51.313332+0000","last_became_active":"2024-03-28T18:04:49.538299+0000","last_became_peered":"2024-03-28T18:04:49.538299+0000","last_unstale":"2024-03-28T18:04:51.313332+0000","last_undegraded":"2024-03-28T18:04:51.313332+0000","last_fullsized":"2024-03-28T18:04:51.313332+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_clean_scrub_stamp":"2024-03-28T18:04:48.465760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T18:45:21.117985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":13,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-28T18:04:51.313178+0000","last_change":"2024-03-28T18:04:49.531135+0000","last_active":"2024-03-28T18:04:51.313178+0000","last_peered":"2024-03-28T18:04:51.313178+0000","last_clean":"2024-03-28T18:04:51.313178+0000","last_became_active":"2024-03-28T18:04:49.530673+0000","last_became_peered":"2024-03-28T18:04:49.530673+0000","last_unstale":"2024-03-28T18:04:51.313178+0000","last_undegraded":"2024-03-28T18:04:51.313178+0000","last_fullsized":"2024-03-28T18:04:51.313178+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_clean_scrub_stamp":"2024-03-28T18:04:48.465760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T05:51:13.950946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":13,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-28T18:04:50.870682+0000","last_change":"2024-03-28T18:04:49.526803+0000","last_active":"2024-03-28T18:04:50.870682+0000","last_peered":"2024-03-28T18:04:50.870682+0000","last_clean":"2024-03-28T18:04:50.870682+0000","last_became_active":"2024-03-28T18:04:49.526474+0000","last_became_peered":"2024-03-28T18:04:49.526474+0000","last_unstale":"2024-03-28T18:04:50.870682+0000","last_undegraded":"2024-03-28T18:04:50.870682+0000","last_fullsized":"2024-03-28T18:04:50.870682+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_clean_scrub_stamp":"2024-03-28T18:04:48.465760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T01:28:27.530864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":13,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-28T18:04:50.520426+0000","last_change":"2024-03-28T18:04:49.534015+0000","last_active":"2024-03-28T18:04:50.520426+0000","last_peered":"2024-03-28T18:04:50.520426+0000","last_clean":"2024-03-28T18:04:50.520426+0000","last_became_active":"2024-03-28T18:04:49.533560+0000","last_became_peered":"2024-03-28T18:04:49.533560+0000","last_unstale":"2024-03-28T18:04:50.520426+0000","last_undegraded":"2024-03-28T18:04:50.520426+0000","last_fullsized":"2024-03-28T18:04:50.520426+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_clean_scrub_stamp":"2024-03-28T18:04:48.465760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T22:41:30.965737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":13,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-28T18:04:50.870214+0000","last_change":"2024-03-28T18:04:49.523240+0000","last_active":"2024-03-28T18:04:50.870214+0000","last_peered":"2024-03-28T18:04:50.870214+0000","last_clean":"2024-03-28T18:04:50.870214+0000","last_became_active":"2024-03-28T18:04:49.522795+0000","last_became_peered":"2024-03-28T18:04:49.522795+0000","last_unstale":"2024-03-28T18:04:50.870214+0000","last_undegraded":"2024-03-28T18:04:50.870214+0000","last_fullsized":"2024-03-28T18:04:50.870214+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_clean_scrub_stamp":"2024-03-28T18:04:48.465760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T23:08:37.087977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":13,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-28T18:04:50.517509+0000","last_change":"2024-03-28T18:04:49.530965+0000","last_active":"2024-03-28T18:04:50.517509+0000","last_peered":"2024-03-28T18:04:50.517509+0000","last_clean":"2024-03-28T18:04:50.517509+0000","last_became_active":"2024-03-28T18:04:49.530619+0000","last_became_peered":"2024-03-28T18:04:49.530619+0000","last_unstale":"2024-03-28T18:04:50.517509+0000","last_undegraded":"2024-03-28T18:04:50.517509+0000","last_fullsized":"2024-03-28T18:04:50.517509+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_clean_scrub_stamp":"2024-03-28T18:04:48.465760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T05:29:26.754012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":13,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-28T18:04:50.514038+0000","last_change":"2024-03-28T18:04:49.523017+0000","last_active":"2024-03-28T18:04:50.514038+0000","last_peered":"2024-03-28T18:04:50.514038+0000","last_clean":"2024-03-28T18:04:50.514038+0000","last_became_active":"2024-03-28T18:04:49.520411+0000","last_became_peered":"2024-03-28T18:04:49.520411+0000","last_unstale":"2024-03-28T18:04:50.514038+0000","last_undegraded":"2024-03-28T18:04:50.514038+0000","last_fullsized":"2024-03-28T18:04:50.514038+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_clean_scrub_stamp":"2024-03-28T18:04:48.465760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T22:52:45.893036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":13,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-28T18:04:51.313045+0000","last_change":"2024-03-28T18:04:49.544272+0000","last_active":"2024-03-28T18:04:51.313045+0000","last_peered":"2024-03-28T18:04:51.313045+0000","last_clean":"2024-03-28T18:04:51.313045+0000","last_became_active":"2024-03-28T18:04:49.541754+0000","last_became_peered":"2024-03-28T18:04:49.541754+0000","last_unstale":"2024-03-28T18:04:51.313045+0000","last_undegraded":"2024-03-28T18:04:51.313045+0000","last_fullsized":"2024-03-28T18:04:51.313045+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_clean_scrub_stamp":"2024-03-28T18:04:48.465760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T19:06:00.799916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":351,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1,"ondisk_log_size":1,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4751360,"data_stored":4722944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":8054,"num_objects":85,"num_object_clones":0,"num_object_copies":255,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":85,"num_whiteouts":0,"num_read":8,"num_read_kb":5,"num_write":182,"num_write_kb":61,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":56,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":356352,"data_stored":41777,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":157,"ondisk_log_size":157,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":85,"seq":365072220177,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27964,"kb_used_data":948,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717188,"statfs":{"total":95995035648,"available":95966400512,"internally_reserved":0,"allocated":970752,"data_stored":465737,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":10,"up_from":77,"seq":330712481817,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32756,"kb_used_data":1580,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712396,"statfs":{"total":95995035648,"available":95961493504,"internally_reserved":0,"allocated":1617920,"data_stored":1059419,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":9,"up_from":70,"seq":300647710751,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36308,"kb_used_data":972,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708844,"statfs":{"total":95995035648,"available":95957856256,"internally_reserved":0,"allocated":995328,"data_stored":466645,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":36174272},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":8,"up_from":62,"seq":266287972389,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32136,"kb_used_data":960,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713016,"statfs":{"total":95995035648,"available":95962128384,"internally_reserved":0,"allocated":983040,"data_stored":466363,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[3,2,2,12],"upper_bound":16},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":7,"up_from":53,"seq":227633266732,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28580,"kb_used_data":1564,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716572,"statfs":{"total":95995035648,"available":95965769728,"internally_reserved":0,"allocated":1601536,"data_stored":1060146,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":48,"seq":206158430258,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36292,"kb_used_data":956,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708860,"statfs":{"total":95995035648,"available":95957872640,"internally_reserved":0,"allocated":978944,"data_stored":465849,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574934,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36296,"kb_used_data":960,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708856,"statfs":{"total":95995035648,"available":95957868544,"internally_reserved":0,"allocated":983040,"data_stored":467349,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738461,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28004,"kb_used_data":988,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717148,"statfs":{"total":95995035648,"available":95966359552,"internally_reserved":0,"allocated":1011712,"data_stored":467915,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411409,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36344,"kb_used_data":1008,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708808,"statfs":{"total":95995035648,"available":95957819392,"internally_reserved":0,"allocated":1032192,"data_stored":488995,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":3,"up_from":26,"seq":111669149769,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36316,"kb_used_data":980,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708836,"statfs":{"total":95995035648,"available":95957848064,"internally_reserved":0,"allocated":1003520,"data_stored":466284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":36174280},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":4,"up_from":34,"seq":146028888129,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27980,"kb_used_data":964,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717172,"statfs":{"total":95995035648,"available":95966384128,"internally_reserved":0,"allocated":987136,"data_stored":466794,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":40,"seq":171798691900,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36904,"kb_used_data":1568,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708248,"statfs":{"total":95995035648,"available":95957245952,"internally_reserved":0,"allocated":1605632,"data_stored":1058343,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2178,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":23258,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1091,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2629,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":722,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":3924,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":908,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":3705,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-28T18:04:59.770 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-03-28T18:04:59.770 INFO:tasks.cephadm:Setting up client nodes... 2024-03-28T18:04:59.770 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fd220748-ed2b-11ee-95d0-87774f69a715 -- ceph auth get-or-create client.0 mon 'allow *' osd 'allow *' mds 'allow *' mgr 'allow *' 2024-03-28T18:04:59.932 INFO:journalctl@ceph.mds.b.smithi079.stdout:Mar 28 18:04:59 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-b[65245]: 2024-03-28T18:04:59.581+0000 7f845f400700 -1 mds.pinger is_rank_lagging: rank=4 was never sent ping request. 2024-03-28T18:04:59.933 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:04:59 smithi079 ceph-mon[35312]: pgmap v374: 97 pgs: 97 active+clean; 585 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 562 B/s rd, 6.8 KiB/s wr, 20 op/s 2024-03-28T18:04:59.933 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:04:59 smithi079 ceph-mon[35312]: from='client.? 172.21.15.22:0/660513257' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-28T18:04:59.933 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:04:59 smithi079 ceph-mon[35312]: from='client.14796 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-28T18:05:00.032 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:04:59 smithi022 ceph-mon[31985]: pgmap v374: 97 pgs: 97 active+clean; 585 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 562 B/s rd, 6.8 KiB/s wr, 20 op/s 2024-03-28T18:05:00.032 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:04:59 smithi022 ceph-mon[31985]: from='client.? 172.21.15.22:0/660513257' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-28T18:05:00.032 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:04:59 smithi022 ceph-mon[31985]: from='client.14796 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-28T18:05:00.175 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:04:59 smithi134 ceph-mon[35037]: pgmap v374: 97 pgs: 97 active+clean; 585 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 562 B/s rd, 6.8 KiB/s wr, 20 op/s 2024-03-28T18:05:00.175 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:04:59 smithi134 ceph-mon[35037]: from='client.? 172.21.15.22:0/660513257' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-28T18:05:00.175 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:04:59 smithi134 ceph-mon[35037]: from='client.14796 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-28T18:05:00.287 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/fd220748-ed2b-11ee-95d0-87774f69a715/mon.a/config 2024-03-28T18:05:00.777 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:05:00 smithi022 ceph-mon[31985]: from='client.14802 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-28T18:05:00.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:05:00 smithi134 ceph-mon[35037]: from='client.14802 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-28T18:05:00.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:05:00 smithi079 ceph-mon[35312]: from='client.14802 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-28T18:05:01.911 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:05:01 smithi022 ceph-mon[31985]: pgmap v375: 97 pgs: 97 active+clean; 627 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 7.4 KiB/s rd, 11 KiB/s wr, 26 op/s 2024-03-28T18:05:01.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:05:01 smithi134 ceph-mon[35037]: pgmap v375: 97 pgs: 97 active+clean; 627 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 7.4 KiB/s rd, 11 KiB/s wr, 26 op/s 2024-03-28T18:05:02.183 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:05:01 smithi079 ceph-mon[35312]: pgmap v375: 97 pgs: 97 active+clean; 627 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 7.4 KiB/s rd, 11 KiB/s wr, 26 op/s 2024-03-28T18:05:02.510 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:02.508+0000 7fa936000700 1 -- 172.21.15.22:0/2398434353 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa9301021e0 msgr2=0x7fa930102600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:05:02.510 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:02.508+0000 7fa936000700 1 --2- 172.21.15.22:0/2398434353 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa9301021e0 0x7fa930102600 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fa9200099c0 tx=0x7fa920009cd0 comp rx=0 tx=0).stop 2024-03-28T18:05:02.510 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:02.508+0000 7fa936000700 1 -- 172.21.15.22:0/2398434353 shutdown_connections 2024-03-28T18:05:02.510 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:02.508+0000 7fa936000700 1 --2- 172.21.15.22:0/2398434353 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa930103d80 0x7fa93010a630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:05:02.511 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:02.508+0000 7fa936000700 1 --2- 172.21.15.22:0/2398434353 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa9301021e0 0x7fa930102600 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:05:02.511 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:02.508+0000 7fa936000700 1 --2- 172.21.15.22:0/2398434353 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa9301033e0 0x7fa930103840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:05:02.511 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:02.508+0000 7fa936000700 1 -- 172.21.15.22:0/2398434353 >> 172.21.15.22:0/2398434353 conn(0x7fa9300fd7a0 msgr2=0x7fa9300ffbc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:05:02.511 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:02.509+0000 7fa936000700 1 -- 172.21.15.22:0/2398434353 shutdown_connections 2024-03-28T18:05:02.511 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:02.509+0000 7fa936000700 1 -- 172.21.15.22:0/2398434353 wait complete. 2024-03-28T18:05:02.511 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:02.510+0000 7fa936000700 1 Processor -- start 2024-03-28T18:05:02.511 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:02.510+0000 7fa936000700 1 -- start start 2024-03-28T18:05:02.511 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:02.510+0000 7fa936000700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa9301021e0 0x7fa930075f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:05:02.511 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:02.510+0000 7fa936000700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa9301033e0 0x7fa930074580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:05:02.512 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:02.510+0000 7fa936000700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa930103d80 0x7fa930074ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:05:02.512 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:02.510+0000 7fa936000700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa93010de20 con 0x7fa930103d80 2024-03-28T18:05:02.512 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:02.510+0000 7fa936000700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa93010dca0 con 0x7fa9301033e0 2024-03-28T18:05:02.512 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:02.510+0000 7fa936000700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa93010dfa0 con 0x7fa9301021e0 2024-03-28T18:05:02.513 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:02.510+0000 7fa92fe00700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa930103d80 0x7fa930074ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:05:02.513 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:02.511+0000 7fa927e00700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa9301033e0 0x7fa930074580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:05:02.514 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:02.511+0000 7fa92fe00700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa930103d80 0x7fa930074ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:47566/0 (socket says 172.21.15.22:47566) 2024-03-28T18:05:02.514 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:02.511+0000 7fa92fe00700 1 -- 172.21.15.22:0/1659927140 learned_addr learned my addr 172.21.15.22:0/1659927140 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-28T18:05:02.514 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:02.511+0000 7fa92f400700 1 --2- 172.21.15.22:0/1659927140 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa9301021e0 0x7fa930075f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:05:02.514 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:02.511+0000 7fa92fe00700 1 -- 172.21.15.22:0/1659927140 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa9301021e0 msgr2=0x7fa930075f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:05:02.514 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:02.511+0000 7fa92fe00700 1 --2- 172.21.15.22:0/1659927140 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa9301021e0 0x7fa930075f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:05:02.514 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:02.511+0000 7fa92fe00700 1 -- 172.21.15.22:0/1659927140 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa9301033e0 msgr2=0x7fa930074580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:05:02.514 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:02.511+0000 7fa92fe00700 1 --2- 172.21.15.22:0/1659927140 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa9301033e0 0x7fa930074580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:05:02.514 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:02.511+0000 7fa92fe00700 1 -- 172.21.15.22:0/1659927140 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa930075370 con 0x7fa930103d80 2024-03-28T18:05:02.514 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:02.511+0000 7fa927e00700 1 --2- 172.21.15.22:0/1659927140 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa9301033e0 0x7fa930074580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-28T18:05:02.515 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:02.511+0000 7fa92fe00700 1 --2- 172.21.15.22:0/1659927140 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa930103d80 0x7fa930074ac0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fa928007d70 tx=0x7fa92800a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:05:02.515 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:02.512+0000 7fa92cc00700 1 -- 172.21.15.22:0/1659927140 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa928016020 con 0x7fa930103d80 2024-03-28T18:05:02.515 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:02.512+0000 7fa92cc00700 1 -- 172.21.15.22:0/1659927140 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7fa92800ad40 con 0x7fa930103d80 2024-03-28T18:05:02.515 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:02.512+0000 7fa936000700 1 -- 172.21.15.22:0/1659927140 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa9301a2460 con 0x7fa930103d80 2024-03-28T18:05:02.515 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:02.512+0000 7fa92cc00700 1 -- 172.21.15.22:0/1659927140 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9280115e0 con 0x7fa930103d80 2024-03-28T18:05:02.515 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:02.512+0000 7fa936000700 1 -- 172.21.15.22:0/1659927140 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa9301a2970 con 0x7fa930103d80 2024-03-28T18:05:02.515 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:02.513+0000 7fa936000700 1 -- 172.21.15.22:0/1659927140 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa9300506a0 con 0x7fa930103d80 2024-03-28T18:05:02.516 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:02.515+0000 7fa92cc00700 1 -- 172.21.15.22:0/1659927140 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 17) v1 ==== 137341+0+0 (secure 0 0 0) 0x7fa928019020 con 0x7fa930103d80 2024-03-28T18:05:02.517 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:02.516+0000 7fa92cc00700 1 --2- 172.21.15.22:0/1659927140 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7fa9140a2a50 0x7fa9140a4f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:05:02.517 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:02.517+0000 7fa92cc00700 1 -- 172.21.15.22:0/1659927140 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 7979+0+0 (secure 0 0 0) 0x7fa9280cea00 con 0x7fa930103d80 2024-03-28T18:05:02.517 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:02.517+0000 7fa92f400700 1 --2- 172.21.15.22:0/1659927140 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7fa9140a2a50 0x7fa9140a4f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:05:02.517 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:02.518+0000 7fa92f400700 1 --2- 172.21.15.22:0/1659927140 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7fa9140a2a50 0x7fa9140a4f10 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fa92000b580 tx=0x7fa920005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:05:02.521 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:02.521+0000 7fa92cc00700 1 -- 172.21.15.22:0/1659927140 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fa928020020 con 0x7fa930103d80 2024-03-28T18:05:02.854 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:02.853+0000 7fa936000700 1 -- 172.21.15.22:0/1659927140 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]} v 0) v1 -- 0x7fa93002da30 con 0x7fa930103d80 2024-03-28T18:05:02.857 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:02.857+0000 7fa92cc00700 1 -- 172.21.15.22:0/1659927140 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]=0 v33) v1 ==== 170+0+59 (secure 0 0 0) 0x7fa92809bb00 con 0x7fa930103d80 2024-03-28T18:05:02.858 INFO:teuthology.orchestra.run.smithi022.stdout:[client.0] 2024-03-28T18:05:02.858 INFO:teuthology.orchestra.run.smithi022.stdout: key = AQBOsQVmwBgAMxAAvdTHBUWgxrUW/BZKSUj+9g== 2024-03-28T18:05:02.861 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:02.860+0000 7fa936000700 1 -- 172.21.15.22:0/1659927140 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7fa9140a2a50 msgr2=0x7fa9140a4f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:05:02.861 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:02.860+0000 7fa936000700 1 --2- 172.21.15.22:0/1659927140 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7fa9140a2a50 0x7fa9140a4f10 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fa92000b580 tx=0x7fa920005fb0 comp rx=0 tx=0).stop 2024-03-28T18:05:02.861 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:02.861+0000 7fa936000700 1 -- 172.21.15.22:0/1659927140 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa930103d80 msgr2=0x7fa930074ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:05:02.861 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:02.861+0000 7fa936000700 1 --2- 172.21.15.22:0/1659927140 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa930103d80 0x7fa930074ac0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fa928007d70 tx=0x7fa92800a340 comp rx=0 tx=0).stop 2024-03-28T18:05:02.861 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:02.861+0000 7fa936000700 1 -- 172.21.15.22:0/1659927140 shutdown_connections 2024-03-28T18:05:02.862 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:02.861+0000 7fa936000700 1 --2- 172.21.15.22:0/1659927140 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa930103d80 0x7fa930074ac0 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:05:02.862 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:02.861+0000 7fa936000700 1 --2- 172.21.15.22:0/1659927140 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7fa9140a2a50 0x7fa9140a4f10 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:05:02.862 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:02.861+0000 7fa936000700 1 --2- 172.21.15.22:0/1659927140 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa9301021e0 0x7fa930075f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:05:02.862 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:02.861+0000 7fa936000700 1 --2- 172.21.15.22:0/1659927140 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa9301033e0 0x7fa930074580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:05:02.862 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:02.861+0000 7fa936000700 1 -- 172.21.15.22:0/1659927140 >> 172.21.15.22:0/1659927140 conn(0x7fa9300fd7a0 msgr2=0x7fa930106680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:05:02.862 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:02.861+0000 7fa936000700 1 -- 172.21.15.22:0/1659927140 shutdown_connections 2024-03-28T18:05:02.862 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:02.861+0000 7fa936000700 1 -- 172.21.15.22:0/1659927140 wait complete. 2024-03-28T18:05:03.667 DEBUG:teuthology.orchestra.run.smithi022:> set -ex 2024-03-28T18:05:03.692 DEBUG:teuthology.orchestra.run.smithi022:> sudo dd of=/etc/ceph/ceph.client.0.keyring 2024-03-28T18:05:03.692 DEBUG:teuthology.orchestra.run.smithi022:> sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2024-03-28T18:05:03.838 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:05:03 smithi022 ceph-mon[31985]: pgmap v376: 97 pgs: 97 active+clean; 627 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 8.0 KiB/s rd, 9.3 KiB/s wr, 24 op/s 2024-03-28T18:05:03.838 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:05:03 smithi022 ceph-mon[31985]: from='client.? 172.21.15.22:0/1659927140' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-03-28T18:05:03.838 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:05:03 smithi022 ceph-mon[31985]: from='client.? 172.21.15.22:0/1659927140' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-03-28T18:05:03.855 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-03-28T18:05:03.856 INFO:tasks.cephadm.ceph_manager.ceph:waiting for mgr available 2024-03-28T18:05:03.856 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033 shell --fsid fd220748-ed2b-11ee-95d0-87774f69a715 -- ceph mgr dump --format=json 2024-03-28T18:05:04.174 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:05:03 smithi134 ceph-mon[35037]: pgmap v376: 97 pgs: 97 active+clean; 627 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 8.0 KiB/s rd, 9.3 KiB/s wr, 24 op/s 2024-03-28T18:05:04.175 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:05:03 smithi134 ceph-mon[35037]: from='client.? 172.21.15.22:0/1659927140' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-03-28T18:05:04.175 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:05:03 smithi134 ceph-mon[35037]: from='client.? 172.21.15.22:0/1659927140' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-03-28T18:05:04.182 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:05:03 smithi079 ceph-mon[35312]: pgmap v376: 97 pgs: 97 active+clean; 627 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 8.0 KiB/s rd, 9.3 KiB/s wr, 24 op/s 2024-03-28T18:05:04.182 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:05:03 smithi079 ceph-mon[35312]: from='client.? 172.21.15.22:0/1659927140' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-03-28T18:05:04.182 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:05:03 smithi079 ceph-mon[35312]: from='client.? 172.21.15.22:0/1659927140' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-03-28T18:05:04.376 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/fd220748-ed2b-11ee-95d0-87774f69a715/mon.a/config 2024-03-28T18:05:05.953 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:05:05 smithi022 ceph-mon[31985]: pgmap v377: 97 pgs: 97 active+clean; 627 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 8.0 KiB/s rd, 8.5 KiB/s wr, 23 op/s 2024-03-28T18:05:06.174 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:05:05 smithi134 ceph-mon[35037]: pgmap v377: 97 pgs: 97 active+clean; 627 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 8.0 KiB/s rd, 8.5 KiB/s wr, 23 op/s 2024-03-28T18:05:06.182 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:05:05 smithi079 ceph-mon[35312]: pgmap v377: 97 pgs: 97 active+clean; 627 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 8.0 KiB/s rd, 8.5 KiB/s wr, 23 op/s 2024-03-28T18:05:06.486 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:06.485+0000 7f5133e00700 1 -- 172.21.15.22:0/794251330 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f512c069e40 msgr2=0x7f512c06a2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:05:06.486 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:06.485+0000 7f5133e00700 1 --2- 172.21.15.22:0/794251330 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f512c069e40 0x7f512c06a2a0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f51180099c0 tx=0x7f5118009cd0 comp rx=0 tx=0).stop 2024-03-28T18:05:06.486 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:06.486+0000 7f5133e00700 1 -- 172.21.15.22:0/794251330 shutdown_connections 2024-03-28T18:05:06.486 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:06.486+0000 7f5133e00700 1 --2- 172.21.15.22:0/794251330 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f512c06a7e0 0x7f512c10f020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:05:06.486 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:06.486+0000 7f5133e00700 1 --2- 172.21.15.22:0/794251330 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f512c1081e0 0x7f512c1085c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:05:06.486 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:06.486+0000 7f5133e00700 1 --2- 172.21.15.22:0/794251330 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f512c069e40 0x7f512c06a2a0 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:05:06.486 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:06.486+0000 7f5133e00700 1 -- 172.21.15.22:0/794251330 >> 172.21.15.22:0/794251330 conn(0x7f512c077fc0 msgr2=0x7f512c0783d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:05:06.486 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:06.486+0000 7f5133e00700 1 -- 172.21.15.22:0/794251330 shutdown_connections 2024-03-28T18:05:06.486 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:06.486+0000 7f5133e00700 1 -- 172.21.15.22:0/794251330 wait complete. 2024-03-28T18:05:06.487 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:06.487+0000 7f5133e00700 1 Processor -- start 2024-03-28T18:05:06.487 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:06.487+0000 7f5133e00700 1 -- start start 2024-03-28T18:05:06.488 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:06.488+0000 7f5133e00700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f512c069e40 0x7f512c19e5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:05:06.488 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:06.488+0000 7f5133e00700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f512c06a7e0 0x7f512c19eb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:05:06.488 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:06.488+0000 7f5133e00700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f512c1081e0 0x7f512c1986b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:05:06.488 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:06.488+0000 7f5133e00700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f512c112410 con 0x7f512c1081e0 2024-03-28T18:05:06.488 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:06.488+0000 7f5133e00700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f512c112290 con 0x7f512c069e40 2024-03-28T18:05:06.488 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:06.488+0000 7f5133e00700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f512c112590 con 0x7f512c06a7e0 2024-03-28T18:05:06.488 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:06.488+0000 7f5131e00700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f512c1081e0 0x7f512c1986b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:05:06.488 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:06.488+0000 7f5131e00700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f512c1081e0 0x7f512c1986b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:47580/0 (socket says 172.21.15.22:47580) 2024-03-28T18:05:06.488 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:06.488+0000 7f5131e00700 1 -- 172.21.15.22:0/1711168920 learned_addr learned my addr 172.21.15.22:0/1711168920 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-28T18:05:06.489 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:06.488+0000 7f5131400700 1 --2- 172.21.15.22:0/1711168920 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f512c069e40 0x7f512c19e5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:05:06.489 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:06.488+0000 7f5130a00700 1 --2- 172.21.15.22:0/1711168920 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f512c06a7e0 0x7f512c19eb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:05:06.489 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:06.489+0000 7f5131e00700 1 -- 172.21.15.22:0/1711168920 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f512c06a7e0 msgr2=0x7f512c19eb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:05:06.490 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:06.489+0000 7f5131e00700 1 --2- 172.21.15.22:0/1711168920 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f512c06a7e0 0x7f512c19eb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:05:06.490 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:06.489+0000 7f5131e00700 1 -- 172.21.15.22:0/1711168920 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f512c069e40 msgr2=0x7f512c19e5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:05:06.490 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:06.489+0000 7f5131e00700 1 --2- 172.21.15.22:0/1711168920 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f512c069e40 0x7f512c19e5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:05:06.490 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:06.489+0000 7f5131e00700 1 -- 172.21.15.22:0/1711168920 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f512c198f90 con 0x7f512c1081e0 2024-03-28T18:05:06.490 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:06.489+0000 7f5130a00700 1 --2- 172.21.15.22:0/1711168920 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f512c06a7e0 0x7f512c19eb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-28T18:05:06.490 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:06.489+0000 7f5131e00700 1 --2- 172.21.15.22:0/1711168920 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f512c1081e0 0x7f512c1986b0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f5124007ee0 tx=0x7f512400d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:05:06.490 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:06.489+0000 7f512a000700 1 -- 172.21.15.22:0/1711168920 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f512400dbd0 con 0x7f512c1081e0 2024-03-28T18:05:06.490 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:06.489+0000 7f512a000700 1 -- 172.21.15.22:0/1711168920 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f512400dd70 con 0x7f512c1081e0 2024-03-28T18:05:06.490 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:06.489+0000 7f5133e00700 1 -- 172.21.15.22:0/1711168920 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f512c199280 con 0x7f512c1081e0 2024-03-28T18:05:06.490 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:06.489+0000 7f512a000700 1 -- 172.21.15.22:0/1711168920 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51240059b0 con 0x7f512c1081e0 2024-03-28T18:05:06.490 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:06.490+0000 7f5133e00700 1 -- 172.21.15.22:0/1711168920 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f512c1a5560 con 0x7f512c1081e0 2024-03-28T18:05:06.491 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:06.491+0000 7f511f400700 1 -- 172.21.15.22:0/1711168920 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f512c103890 con 0x7f512c1081e0 2024-03-28T18:05:06.493 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:06.493+0000 7f512a000700 1 -- 172.21.15.22:0/1711168920 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 17) v1 ==== 137341+0+0 (secure 0 0 0) 0x7f512401a020 con 0x7f512c1081e0 2024-03-28T18:05:06.495 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:06.495+0000 7f512a000700 1 --2- 172.21.15.22:0/1711168920 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7f51140a29a0 0x7f51140a4e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:05:06.495 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:06.495+0000 7f512a000700 1 -- 172.21.15.22:0/1711168920 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f51240cdff0 con 0x7f512c1081e0 2024-03-28T18:05:06.495 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:06.495+0000 7f5131400700 1 --2- 172.21.15.22:0/1711168920 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7f51140a29a0 0x7f51140a4e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:05:06.495 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:06.496+0000 7f5131400700 1 --2- 172.21.15.22:0/1711168920 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7f51140a29a0 0x7f51140a4e60 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f512000abd0 tx=0x7f512000a380 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:05:06.502 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:06.502+0000 7f512a000700 1 -- 172.21.15.22:0/1711168920 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f5124010070 con 0x7f512c1081e0 2024-03-28T18:05:06.799 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:06.799+0000 7f511f400700 1 -- 172.21.15.22:0/1711168920 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) v1 -- 0x7f512c199be0 con 0x7f512c1081e0 2024-03-28T18:05:06.811 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:06.812+0000 7f512a000700 1 -- 172.21.15.22:0/1711168920 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v17) v1 ==== 74+0+262661 (secure 0 0 0) 0x7f512409b040 con 0x7f512c1081e0 2024-03-28T18:05:06.817 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-03-28T18:05:06.819 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:06.817+0000 7f511f400700 1 -- 172.21.15.22:0/1711168920 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7f51140a29a0 msgr2=0x7f51140a4e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:05:06.819 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:06.817+0000 7f511f400700 1 --2- 172.21.15.22:0/1711168920 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7f51140a29a0 0x7f51140a4e60 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f512000abd0 tx=0x7f512000a380 comp rx=0 tx=0).stop 2024-03-28T18:05:06.820 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:06.817+0000 7f511f400700 1 -- 172.21.15.22:0/1711168920 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f512c1081e0 msgr2=0x7f512c1986b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:05:06.820 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:06.817+0000 7f511f400700 1 --2- 172.21.15.22:0/1711168920 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f512c1081e0 0x7f512c1986b0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f5124007ee0 tx=0x7f512400d3b0 comp rx=0 tx=0).stop 2024-03-28T18:05:06.820 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:06.818+0000 7f511f400700 1 -- 172.21.15.22:0/1711168920 shutdown_connections 2024-03-28T18:05:06.820 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:06.818+0000 7f511f400700 1 --2- 172.21.15.22:0/1711168920 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f512c1081e0 0x7f512c1986b0 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:05:06.820 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:06.818+0000 7f511f400700 1 --2- 172.21.15.22:0/1711168920 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f512c069e40 0x7f512c19e5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:05:06.820 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:06.818+0000 7f511f400700 1 --2- 172.21.15.22:0/1711168920 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7f51140a29a0 0x7f51140a4e60 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:05:06.820 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:06.818+0000 7f511f400700 1 --2- 172.21.15.22:0/1711168920 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f512c06a7e0 0x7f512c19eb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:05:06.820 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:06.818+0000 7f511f400700 1 -- 172.21.15.22:0/1711168920 >> 172.21.15.22:0/1711168920 conn(0x7f512c077fc0 msgr2=0x7f512c0ffc80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:05:06.820 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:06.818+0000 7f511f400700 1 -- 172.21.15.22:0/1711168920 shutdown_connections 2024-03-28T18:05:06.820 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:06.818+0000 7f511f400700 1 -- 172.21.15.22:0/1711168920 wait complete. 2024-03-28T18:05:07.642 INFO:teuthology.orchestra.run.smithi022.stdout:{"epoch":17,"flags":0,"active_gid":14152,"active_name":"x","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6800","nonce":4201048392},{"type":"v1","addr":"172.21.15.22:6801","nonce":4201048392}]},"active_addr":"172.21.15.22:6801/4201048392","active_change":"2024-03-28T17:55:10.425773+0000","active_mgr_features":4540701547738038271,"available":true,"standbys":[{"gid":24115,"name":"z","mgr_features":4540701547738038271,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.0.0","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]},{"gid":34112,"name":"y","mgr_features":4540701547738038271,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.0.0","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["cephadm","dashboard","iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.0.0","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{"dashboard":"https://172.21.15.22:8443/"},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"squid":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":3,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.22:0","nonce":2956708401}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.22:0","nonce":1901766106}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.22:0","nonce":1072073804}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.22:0","nonce":1593587499}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.22:0","nonce":3933020312}]}]} 2024-03-28T18:05:07.647 INFO:tasks.cephadm.ceph_manager.ceph:mgr available! 2024-03-28T18:05:07.647 INFO:tasks.cephadm.ceph_manager.ceph:waiting for all up 2024-03-28T18:05:07.648 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033 shell --fsid fd220748-ed2b-11ee-95d0-87774f69a715 -- ceph osd dump --format=json 2024-03-28T18:05:07.985 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:05:07 smithi079 ceph-mon[35312]: pgmap v378: 97 pgs: 97 active+clean; 640 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 8.8 KiB/s wr, 24 op/s 2024-03-28T18:05:07.985 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:05:07 smithi079 ceph-mon[35312]: from='client.? 172.21.15.22:0/1711168920' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-03-28T18:05:08.031 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:05:07 smithi022 ceph-mon[31985]: pgmap v378: 97 pgs: 97 active+clean; 640 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 8.8 KiB/s wr, 24 op/s 2024-03-28T18:05:08.032 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:05:07 smithi022 ceph-mon[31985]: from='client.? 172.21.15.22:0/1711168920' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-03-28T18:05:08.104 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/fd220748-ed2b-11ee-95d0-87774f69a715/mon.a/config 2024-03-28T18:05:08.174 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:05:07 smithi134 ceph-mon[35037]: pgmap v378: 97 pgs: 97 active+clean; 640 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 8.8 KiB/s wr, 24 op/s 2024-03-28T18:05:08.174 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:05:07 smithi134 ceph-mon[35037]: from='client.? 172.21.15.22:0/1711168920' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-03-28T18:05:09.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:05:09 smithi079 ceph-mon[35312]: pgmap v379: 97 pgs: 97 active+clean; 640 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 6.1 KiB/s wr, 18 op/s 2024-03-28T18:05:09.933 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:05:09 smithi022 ceph-mon[31985]: pgmap v379: 97 pgs: 97 active+clean; 640 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 6.1 KiB/s wr, 18 op/s 2024-03-28T18:05:10.174 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:05:09 smithi134 ceph-mon[35037]: pgmap v379: 97 pgs: 97 active+clean; 640 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 6.1 KiB/s wr, 18 op/s 2024-03-28T18:05:10.223 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:10.223+0000 7fb895400700 1 -- 172.21.15.22:0/788471815 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb890102ba0 msgr2=0x7fb89010efe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:05:10.223 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:10.223+0000 7fb895400700 1 --2- 172.21.15.22:0/788471815 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb890102ba0 0x7fb89010efe0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7fb8880099c0 tx=0x7fb888009cd0 comp rx=0 tx=0).stop 2024-03-28T18:05:10.224 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:10.223+0000 7fb895400700 1 -- 172.21.15.22:0/788471815 shutdown_connections 2024-03-28T18:05:10.224 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:10.223+0000 7fb895400700 1 --2- 172.21.15.22:0/788471815 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb890102ba0 0x7fb89010efe0 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:05:10.224 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:10.223+0000 7fb895400700 1 --2- 172.21.15.22:0/788471815 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fb8901081c0 0x7fb8901085a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:05:10.224 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:10.223+0000 7fb895400700 1 --2- 172.21.15.22:0/788471815 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb8901021e0 0x7fb890102660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:05:10.224 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:10.223+0000 7fb895400700 1 -- 172.21.15.22:0/788471815 >> 172.21.15.22:0/788471815 conn(0x7fb8900792c0 msgr2=0x7fb8900796d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:05:10.224 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:10.223+0000 7fb895400700 1 -- 172.21.15.22:0/788471815 shutdown_connections 2024-03-28T18:05:10.225 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:10.223+0000 7fb895400700 1 -- 172.21.15.22:0/788471815 wait complete. 2024-03-28T18:05:10.226 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:10.225+0000 7fb895400700 1 Processor -- start 2024-03-28T18:05:10.226 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:10.225+0000 7fb895400700 1 -- start start 2024-03-28T18:05:10.226 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:10.226+0000 7fb895400700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb8901021e0 0x7fb8901a06e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:05:10.226 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:10.226+0000 7fb895400700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb890102ba0 0x7fb8901a0c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:05:10.226 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:10.226+0000 7fb895400700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fb8901081c0 0x7fb89019a8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:05:10.226 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:10.226+0000 7fb895400700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb890112480 con 0x7fb890102ba0 2024-03-28T18:05:10.226 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:10.226+0000 7fb895400700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb890112300 con 0x7fb8901081c0 2024-03-28T18:05:10.226 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:10.226+0000 7fb895400700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb890112600 con 0x7fb8901021e0 2024-03-28T18:05:10.227 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:10.226+0000 7fb885e00700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb890102ba0 0x7fb8901a0c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:05:10.227 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:10.226+0000 7fb88f200700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fb8901081c0 0x7fb89019a8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:05:10.227 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:10.226+0000 7fb885e00700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb890102ba0 0x7fb8901a0c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:34622/0 (socket says 172.21.15.22:34622) 2024-03-28T18:05:10.227 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:10.226+0000 7fb885e00700 1 -- 172.21.15.22:0/3179774315 learned_addr learned my addr 172.21.15.22:0/3179774315 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-28T18:05:10.227 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:10.226+0000 7fb88e800700 1 --2- 172.21.15.22:0/3179774315 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb8901021e0 0x7fb8901a06e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:05:10.228 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:10.226+0000 7fb885e00700 1 -- 172.21.15.22:0/3179774315 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb8901021e0 msgr2=0x7fb8901a06e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:05:10.228 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:10.226+0000 7fb885e00700 1 --2- 172.21.15.22:0/3179774315 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb8901021e0 0x7fb8901a06e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:05:10.228 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:10.226+0000 7fb885e00700 1 -- 172.21.15.22:0/3179774315 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fb8901081c0 msgr2=0x7fb89019a8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:05:10.228 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:10.227+0000 7fb885e00700 1 --2- 172.21.15.22:0/3179774315 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fb8901081c0 0x7fb89019a8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:05:10.228 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:10.227+0000 7fb885e00700 1 -- 172.21.15.22:0/3179774315 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb89019b000 con 0x7fb890102ba0 2024-03-28T18:05:10.228 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:10.227+0000 7fb88f200700 1 --2- 172.21.15.22:0/3179774315 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fb8901081c0 0x7fb89019a8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-28T18:05:10.228 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:10.227+0000 7fb885e00700 1 --2- 172.21.15.22:0/3179774315 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb890102ba0 0x7fb8901a0c20 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fb87800d950 tx=0x7fb87800dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:05:10.228 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:10.227+0000 7fb887e00700 1 -- 172.21.15.22:0/3179774315 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb878016070 con 0x7fb890102ba0 2024-03-28T18:05:10.228 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:10.227+0000 7fb887e00700 1 -- 172.21.15.22:0/3179774315 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7fb878009e10 con 0x7fb890102ba0 2024-03-28T18:05:10.228 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:10.227+0000 7fb895400700 1 -- 172.21.15.22:0/3179774315 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb89019b2f0 con 0x7fb890102ba0 2024-03-28T18:05:10.228 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:10.227+0000 7fb887e00700 1 -- 172.21.15.22:0/3179774315 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb87800f620 con 0x7fb890102ba0 2024-03-28T18:05:10.229 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:10.227+0000 7fb895400700 1 -- 172.21.15.22:0/3179774315 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb8901a75b0 con 0x7fb890102ba0 2024-03-28T18:05:10.229 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:10.228+0000 7fb895400700 1 -- 172.21.15.22:0/3179774315 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb890068e50 con 0x7fb890102ba0 2024-03-28T18:05:10.230 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:10.230+0000 7fb887e00700 1 -- 172.21.15.22:0/3179774315 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 17) v1 ==== 137341+0+0 (secure 0 0 0) 0x7fb878019070 con 0x7fb890102ba0 2024-03-28T18:05:10.231 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:10.232+0000 7fb887e00700 1 --2- 172.21.15.22:0/3179774315 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7fb8740a29a0 0x7fb8740a4e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:05:10.232 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:10.232+0000 7fb887e00700 1 -- 172.21.15.22:0/3179774315 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 7979+0+0 (secure 0 0 0) 0x7fb8780ce680 con 0x7fb890102ba0 2024-03-28T18:05:10.232 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:10.232+0000 7fb88e800700 1 --2- 172.21.15.22:0/3179774315 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7fb8740a29a0 0x7fb8740a4e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:05:10.232 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:10.233+0000 7fb88e800700 1 --2- 172.21.15.22:0/3179774315 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7fb8740a29a0 0x7fb8740a4e60 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fb89019b9e0 tx=0x7fb88000b040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:05:10.236 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:10.236+0000 7fb887e00700 1 -- 172.21.15.22:0/3179774315 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fb87809b670 con 0x7fb890102ba0 2024-03-28T18:05:10.466 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:10.466+0000 7fb895400700 1 -- 172.21.15.22:0/3179774315 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fb8901a7810 con 0x7fb890102ba0 2024-03-28T18:05:10.467 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:10.468+0000 7fb887e00700 1 -- 172.21.15.22:0/3179774315 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v90) v1 ==== 74+0+22183 (secure 0 0 0) 0x7fb87809af70 con 0x7fb890102ba0 2024-03-28T18:05:10.468 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-03-28T18:05:10.468 INFO:teuthology.orchestra.run.smithi022.stdout:{"epoch":90,"fsid":"fd220748-ed2b-11ee-95d0-87774f69a715","created":"2024-03-28T17:54:00.440279+0000","modified":"2024-03-28T18:04:50.509330+0000","last_up_change":"2024-03-28T18:03:43.220095+0000","last_in_change":"2024-03-28T18:03:18.526399+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-28T17:58:33.168872+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-28T18:04:47.004797+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-28T18:04:48.465502+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"90","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"5d708e06-f05b-40ba-bbcf-258ddc4f43d7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6802","nonce":2186290846},{"type":"v1","addr":"172.21.15.22:6803","nonce":2186290846}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6804","nonce":2186290846},{"type":"v1","addr":"172.21.15.22:6805","nonce":2186290846}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6808","nonce":2186290846},{"type":"v1","addr":"172.21.15.22:6809","nonce":2186290846}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6806","nonce":2186290846},{"type":"v1","addr":"172.21.15.22:6807","nonce":2186290846}]},"public_addr":"172.21.15.22:6803/2186290846","cluster_addr":"172.21.15.22:6805/2186290846","heartbeat_back_addr":"172.21.15.22:6809/2186290846","heartbeat_front_addr":"172.21.15.22:6807/2186290846","state":["exists","up"]},{"osd":1,"uuid":"68ad09bb-9256-409a-9867-4e14a2cb94aa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6800","nonce":2835549681},{"type":"v1","addr":"172.21.15.79:6801","nonce":2835549681}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6802","nonce":2835549681},{"type":"v1","addr":"172.21.15.79:6803","nonce":2835549681}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6806","nonce":2835549681},{"type":"v1","addr":"172.21.15.79:6807","nonce":2835549681}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6804","nonce":2835549681},{"type":"v1","addr":"172.21.15.79:6805","nonce":2835549681}]},"public_addr":"172.21.15.79:6801/2835549681","cluster_addr":"172.21.15.79:6803/2835549681","heartbeat_back_addr":"172.21.15.79:6807/2835549681","heartbeat_front_addr":"172.21.15.79:6805/2835549681","state":["exists","up"]},{"osd":2,"uuid":"ed11a5a7-004d-41f3-a5fc-eb2d603bd33f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6800","nonce":470588085},{"type":"v1","addr":"172.21.15.134:6801","nonce":470588085}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6802","nonce":470588085},{"type":"v1","addr":"172.21.15.134:6803","nonce":470588085}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6806","nonce":470588085},{"type":"v1","addr":"172.21.15.134:6807","nonce":470588085}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6804","nonce":470588085},{"type":"v1","addr":"172.21.15.134:6805","nonce":470588085}]},"public_addr":"172.21.15.134:6801/470588085","cluster_addr":"172.21.15.134:6803/470588085","heartbeat_back_addr":"172.21.15.134:6807/470588085","heartbeat_front_addr":"172.21.15.134:6805/470588085","state":["exists","up"]},{"osd":3,"uuid":"81a6c547-2a50-44c3-84d7-23ab6831790c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6810","nonce":1702441854},{"type":"v1","addr":"172.21.15.22:6811","nonce":1702441854}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6812","nonce":1702441854},{"type":"v1","addr":"172.21.15.22:6813","nonce":1702441854}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6816","nonce":1702441854},{"type":"v1","addr":"172.21.15.22:6817","nonce":1702441854}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6814","nonce":1702441854},{"type":"v1","addr":"172.21.15.22:6815","nonce":1702441854}]},"public_addr":"172.21.15.22:6811/1702441854","cluster_addr":"172.21.15.22:6813/1702441854","heartbeat_back_addr":"172.21.15.22:6817/1702441854","heartbeat_front_addr":"172.21.15.22:6815/1702441854","state":["exists","up"]},{"osd":4,"uuid":"d23a254e-7182-4471-8bca-7eb684ec618e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6808","nonce":630132437},{"type":"v1","addr":"172.21.15.79:6809","nonce":630132437}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6810","nonce":630132437},{"type":"v1","addr":"172.21.15.79:6811","nonce":630132437}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6814","nonce":630132437},{"type":"v1","addr":"172.21.15.79:6815","nonce":630132437}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6812","nonce":630132437},{"type":"v1","addr":"172.21.15.79:6813","nonce":630132437}]},"public_addr":"172.21.15.79:6809/630132437","cluster_addr":"172.21.15.79:6811/630132437","heartbeat_back_addr":"172.21.15.79:6815/630132437","heartbeat_front_addr":"172.21.15.79:6813/630132437","state":["exists","up"]},{"osd":5,"uuid":"a7b98502-2023-4f0a-bfa0-6840cc495e0c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":40,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6808","nonce":2135406915},{"type":"v1","addr":"172.21.15.134:6809","nonce":2135406915}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6810","nonce":2135406915},{"type":"v1","addr":"172.21.15.134:6811","nonce":2135406915}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6814","nonce":2135406915},{"type":"v1","addr":"172.21.15.134:6815","nonce":2135406915}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6812","nonce":2135406915},{"type":"v1","addr":"172.21.15.134:6813","nonce":2135406915}]},"public_addr":"172.21.15.134:6809/2135406915","cluster_addr":"172.21.15.134:6811/2135406915","heartbeat_back_addr":"172.21.15.134:6815/2135406915","heartbeat_front_addr":"172.21.15.134:6813/2135406915","state":["exists","up"]},{"osd":6,"uuid":"634c6562-d683-41c8-8242-fc848c38ce1f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6818","nonce":1400680396},{"type":"v1","addr":"172.21.15.22:6819","nonce":1400680396}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6820","nonce":1400680396},{"type":"v1","addr":"172.21.15.22:6821","nonce":1400680396}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6824","nonce":1400680396},{"type":"v1","addr":"172.21.15.22:6825","nonce":1400680396}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6822","nonce":1400680396},{"type":"v1","addr":"172.21.15.22:6823","nonce":1400680396}]},"public_addr":"172.21.15.22:6819/1400680396","cluster_addr":"172.21.15.22:6821/1400680396","heartbeat_back_addr":"172.21.15.22:6825/1400680396","heartbeat_front_addr":"172.21.15.22:6823/1400680396","state":["exists","up"]},{"osd":7,"uuid":"504eb4a8-6731-45ad-932c-1b38ef157871","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":53,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6816","nonce":1287105314},{"type":"v1","addr":"172.21.15.79:6817","nonce":1287105314}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6818","nonce":1287105314},{"type":"v1","addr":"172.21.15.79:6819","nonce":1287105314}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6822","nonce":1287105314},{"type":"v1","addr":"172.21.15.79:6823","nonce":1287105314}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6820","nonce":1287105314},{"type":"v1","addr":"172.21.15.79:6821","nonce":1287105314}]},"public_addr":"172.21.15.79:6817/1287105314","cluster_addr":"172.21.15.79:6819/1287105314","heartbeat_back_addr":"172.21.15.79:6823/1287105314","heartbeat_front_addr":"172.21.15.79:6821/1287105314","state":["exists","up"]},{"osd":8,"uuid":"20aa66c1-03d6-4cb8-a096-0eae41979c17","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":62,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6816","nonce":2265650388},{"type":"v1","addr":"172.21.15.134:6817","nonce":2265650388}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6818","nonce":2265650388},{"type":"v1","addr":"172.21.15.134:6819","nonce":2265650388}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6822","nonce":2265650388},{"type":"v1","addr":"172.21.15.134:6823","nonce":2265650388}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6820","nonce":2265650388},{"type":"v1","addr":"172.21.15.134:6821","nonce":2265650388}]},"public_addr":"172.21.15.134:6817/2265650388","cluster_addr":"172.21.15.134:6819/2265650388","heartbeat_back_addr":"172.21.15.134:6823/2265650388","heartbeat_front_addr":"172.21.15.134:6821/2265650388","state":["exists","up"]},{"osd":9,"uuid":"e1ae4020-7c2c-4334-a753-ee6b12d98584","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":70,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6826","nonce":3762222259},{"type":"v1","addr":"172.21.15.22:6827","nonce":3762222259}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6828","nonce":3762222259},{"type":"v1","addr":"172.21.15.22:6829","nonce":3762222259}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6832","nonce":3762222259},{"type":"v1","addr":"172.21.15.22:6833","nonce":3762222259}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6830","nonce":3762222259},{"type":"v1","addr":"172.21.15.22:6831","nonce":3762222259}]},"public_addr":"172.21.15.22:6827/3762222259","cluster_addr":"172.21.15.22:6829/3762222259","heartbeat_back_addr":"172.21.15.22:6833/3762222259","heartbeat_front_addr":"172.21.15.22:6831/3762222259","state":["exists","up"]},{"osd":10,"uuid":"f5198d52-df48-4679-a860-4dcb440b7a9f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":77,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6824","nonce":1746167904},{"type":"v1","addr":"172.21.15.79:6825","nonce":1746167904}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6826","nonce":1746167904},{"type":"v1","addr":"172.21.15.79:6827","nonce":1746167904}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6830","nonce":1746167904},{"type":"v1","addr":"172.21.15.79:6831","nonce":1746167904}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6828","nonce":1746167904},{"type":"v1","addr":"172.21.15.79:6829","nonce":1746167904}]},"public_addr":"172.21.15.79:6825/1746167904","cluster_addr":"172.21.15.79:6827/1746167904","heartbeat_back_addr":"172.21.15.79:6831/1746167904","heartbeat_front_addr":"172.21.15.79:6829/1746167904","state":["exists","up"]},{"osd":11,"uuid":"1427bca7-3414-4468-9246-e999156bc2ae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":85,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6824","nonce":4271716428},{"type":"v1","addr":"172.21.15.134:6825","nonce":4271716428}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6826","nonce":4271716428},{"type":"v1","addr":"172.21.15.134:6827","nonce":4271716428}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6830","nonce":4271716428},{"type":"v1","addr":"172.21.15.134:6831","nonce":4271716428}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6828","nonce":4271716428},{"type":"v1","addr":"172.21.15.134:6829","nonce":4271716428}]},"public_addr":"172.21.15.134:6825/4271716428","cluster_addr":"172.21.15.134:6827/4271716428","heartbeat_back_addr":"172.21.15.134:6831/4271716428","heartbeat_front_addr":"172.21.15.134:6829/4271716428","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T17:57:21.225125+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T17:57:55.197254+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T17:58:27.825136+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T17:59:04.319893+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T17:59:38.407752+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T18:00:12.118805+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T18:00:47.635214+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T18:01:21.231055+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T18:01:55.817861+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T18:02:31.680160+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T18:03:06.783398+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T18:03:41.307347+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.22:0/3521924283":"2024-03-29T17:55:10.425504+0000","172.21.15.22:6801/730635939":"2024-03-29T17:55:10.425504+0000","172.21.15.22:0/2471229459":"2024-03-29T17:54:34.004559+0000","172.21.15.22:6800/3166828081":"2024-03-29T17:54:34.004559+0000","172.21.15.22:6800/730635939":"2024-03-29T17:55:10.425504+0000","172.21.15.22:0/1229047849":"2024-03-29T17:54:34.004559+0000","172.21.15.22:6801/3166828081":"2024-03-29T17:54:34.004559+0000","172.21.15.22:0/2717818856":"2024-03-29T17:55:10.425504+0000","172.21.15.22:0/2157570333":"2024-03-29T17:54:34.004559+0000","172.21.15.22:0/1006589530":"2024-03-29T17:55:10.425504+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-28T18:05:10.470 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:10.471+0000 7fb895400700 1 -- 172.21.15.22:0/3179774315 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7fb8740a29a0 msgr2=0x7fb8740a4e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:05:10.471 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:10.471+0000 7fb895400700 1 --2- 172.21.15.22:0/3179774315 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7fb8740a29a0 0x7fb8740a4e60 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fb89019b9e0 tx=0x7fb88000b040 comp rx=0 tx=0).stop 2024-03-28T18:05:10.471 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:10.471+0000 7fb895400700 1 -- 172.21.15.22:0/3179774315 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb890102ba0 msgr2=0x7fb8901a0c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:05:10.471 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:10.471+0000 7fb895400700 1 --2- 172.21.15.22:0/3179774315 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb890102ba0 0x7fb8901a0c20 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fb87800d950 tx=0x7fb87800dc60 comp rx=0 tx=0).stop 2024-03-28T18:05:10.471 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:10.471+0000 7fb895400700 1 -- 172.21.15.22:0/3179774315 shutdown_connections 2024-03-28T18:05:10.471 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:10.471+0000 7fb895400700 1 --2- 172.21.15.22:0/3179774315 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fb8901081c0 0x7fb89019a8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:05:10.471 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:10.471+0000 7fb895400700 1 --2- 172.21.15.22:0/3179774315 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7fb8740a29a0 0x7fb8740a4e60 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:05:10.471 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:10.471+0000 7fb895400700 1 --2- 172.21.15.22:0/3179774315 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb8901021e0 0x7fb8901a06e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:05:10.471 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:10.471+0000 7fb895400700 1 --2- 172.21.15.22:0/3179774315 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb890102ba0 0x7fb8901a0c20 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:05:10.472 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:10.471+0000 7fb895400700 1 -- 172.21.15.22:0/3179774315 >> 172.21.15.22:0/3179774315 conn(0x7fb8900792c0 msgr2=0x7fb8900745f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:05:10.472 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:10.471+0000 7fb895400700 1 -- 172.21.15.22:0/3179774315 shutdown_connections 2024-03-28T18:05:10.472 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:10.471+0000 7fb895400700 1 -- 172.21.15.22:0/3179774315 wait complete. 2024-03-28T18:05:10.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:05:10 smithi022 ceph-mon[31985]: from='client.? 172.21.15.22:0/3179774315' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-28T18:05:10.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:05:10 smithi022 ceph-mon[31985]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.8", "id": [2, 11]}]: dispatch 2024-03-28T18:05:10.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:05:10 smithi022 ceph-mon[31985]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1d", "id": [8, 11]}]: dispatch 2024-03-28T18:05:10.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:05:10 smithi022 ceph-mon[31985]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3", "id": [4, 9]}]: dispatch 2024-03-28T18:05:10.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:05:10 smithi022 ceph-mon[31985]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.c", "id": [3, 9]}]: dispatch 2024-03-28T18:05:10.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:05:10 smithi022 ceph-mon[31985]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.21", "id": [4, 8]}]: dispatch 2024-03-28T18:05:10.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:05:10 smithi022 ceph-mon[31985]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.25", "id": [5, 9]}]: dispatch 2024-03-28T18:05:11.175 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:05:10 smithi134 ceph-mon[35037]: from='client.? 172.21.15.22:0/3179774315' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-28T18:05:11.175 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:05:10 smithi134 ceph-mon[35037]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.8", "id": [2, 11]}]: dispatch 2024-03-28T18:05:11.175 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:05:10 smithi134 ceph-mon[35037]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1d", "id": [8, 11]}]: dispatch 2024-03-28T18:05:11.175 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:05:10 smithi134 ceph-mon[35037]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3", "id": [4, 9]}]: dispatch 2024-03-28T18:05:11.175 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:05:10 smithi134 ceph-mon[35037]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.c", "id": [3, 9]}]: dispatch 2024-03-28T18:05:11.175 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:05:10 smithi134 ceph-mon[35037]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.21", "id": [4, 8]}]: dispatch 2024-03-28T18:05:11.175 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:05:10 smithi134 ceph-mon[35037]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.25", "id": [5, 9]}]: dispatch 2024-03-28T18:05:11.183 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:05:10 smithi079 ceph-mon[35312]: from='client.? 172.21.15.22:0/3179774315' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-28T18:05:11.183 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:05:10 smithi079 ceph-mon[35312]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.8", "id": [2, 11]}]: dispatch 2024-03-28T18:05:11.183 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:05:10 smithi079 ceph-mon[35312]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1d", "id": [8, 11]}]: dispatch 2024-03-28T18:05:11.183 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:05:10 smithi079 ceph-mon[35312]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3", "id": [4, 9]}]: dispatch 2024-03-28T18:05:11.183 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:05:10 smithi079 ceph-mon[35312]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.c", "id": [3, 9]}]: dispatch 2024-03-28T18:05:11.183 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:05:10 smithi079 ceph-mon[35312]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.21", "id": [4, 8]}]: dispatch 2024-03-28T18:05:11.183 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:05:10 smithi079 ceph-mon[35312]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.25", "id": [5, 9]}]: dispatch 2024-03-28T18:05:11.332 INFO:tasks.cephadm.ceph_manager.ceph:all up! 2024-03-28T18:05:11.332 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033 shell --fsid fd220748-ed2b-11ee-95d0-87774f69a715 -- ceph osd dump --format=json 2024-03-28T18:05:11.809 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/fd220748-ed2b-11ee-95d0-87774f69a715/mon.a/config 2024-03-28T18:05:11.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:05:11 smithi134 ceph-mon[35037]: pgmap v380: 97 pgs: 97 active+clean; 640 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 5.7 KiB/s wr, 16 op/s 2024-03-28T18:05:11.925 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:05:11 smithi134 ceph-mon[35037]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.8", "id": [2, 11]}]': finished 2024-03-28T18:05:11.925 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:05:11 smithi134 ceph-mon[35037]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1d", "id": [8, 11]}]': finished 2024-03-28T18:05:11.925 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:05:11 smithi134 ceph-mon[35037]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3", "id": [4, 9]}]': finished 2024-03-28T18:05:11.925 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:05:11 smithi134 ceph-mon[35037]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.c", "id": [3, 9]}]': finished 2024-03-28T18:05:11.925 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:05:11 smithi134 ceph-mon[35037]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.21", "id": [4, 8]}]': finished 2024-03-28T18:05:11.925 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:05:11 smithi134 ceph-mon[35037]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.25", "id": [5, 9]}]': finished 2024-03-28T18:05:11.925 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:05:11 smithi134 ceph-mon[35037]: osdmap e91: 12 total, 12 up, 12 in 2024-03-28T18:05:12.020 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:05:11 smithi022 ceph-mon[31985]: pgmap v380: 97 pgs: 97 active+clean; 640 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 5.7 KiB/s wr, 16 op/s 2024-03-28T18:05:12.020 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:05:11 smithi022 ceph-mon[31985]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.8", "id": [2, 11]}]': finished 2024-03-28T18:05:12.020 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:05:11 smithi022 ceph-mon[31985]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1d", "id": [8, 11]}]': finished 2024-03-28T18:05:12.021 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:05:11 smithi022 ceph-mon[31985]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3", "id": [4, 9]}]': finished 2024-03-28T18:05:12.021 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:05:11 smithi022 ceph-mon[31985]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.c", "id": [3, 9]}]': finished 2024-03-28T18:05:12.021 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:05:11 smithi022 ceph-mon[31985]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.21", "id": [4, 8]}]': finished 2024-03-28T18:05:12.021 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:05:11 smithi022 ceph-mon[31985]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.25", "id": [5, 9]}]': finished 2024-03-28T18:05:12.021 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:05:11 smithi022 ceph-mon[31985]: osdmap e91: 12 total, 12 up, 12 in 2024-03-28T18:05:12.183 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:05:11 smithi079 ceph-mon[35312]: pgmap v380: 97 pgs: 97 active+clean; 640 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 5.7 KiB/s wr, 16 op/s 2024-03-28T18:05:12.183 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:05:11 smithi079 ceph-mon[35312]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.8", "id": [2, 11]}]': finished 2024-03-28T18:05:12.183 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:05:11 smithi079 ceph-mon[35312]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1d", "id": [8, 11]}]': finished 2024-03-28T18:05:12.183 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:05:11 smithi079 ceph-mon[35312]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3", "id": [4, 9]}]': finished 2024-03-28T18:05:12.183 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:05:11 smithi079 ceph-mon[35312]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.c", "id": [3, 9]}]': finished 2024-03-28T18:05:12.183 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:05:11 smithi079 ceph-mon[35312]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.21", "id": [4, 8]}]': finished 2024-03-28T18:05:12.183 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:05:11 smithi079 ceph-mon[35312]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.25", "id": [5, 9]}]': finished 2024-03-28T18:05:12.183 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:05:11 smithi079 ceph-mon[35312]: osdmap e91: 12 total, 12 up, 12 in 2024-03-28T18:05:13.032 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:05:12 smithi022 ceph-mon[31985]: osdmap e92: 12 total, 12 up, 12 in 2024-03-28T18:05:13.174 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:05:12 smithi134 ceph-mon[35037]: osdmap e92: 12 total, 12 up, 12 in 2024-03-28T18:05:13.182 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:05:12 smithi079 ceph-mon[35312]: osdmap e92: 12 total, 12 up, 12 in 2024-03-28T18:05:14.007 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:14.007+0000 7ff9a1800700 1 -- 172.21.15.22:0/221717312 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff99c1073b0 msgr2=0x7ff99c10ef80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:05:14.007 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:14.007+0000 7ff9a1800700 1 --2- 172.21.15.22:0/221717312 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff99c1073b0 0x7ff99c10ef80 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7ff99400b0d0 tx=0x7ff99400b3e0 comp rx=0 tx=0).stop 2024-03-28T18:05:14.008 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:05:13 smithi022 ceph-mon[31985]: pgmap v383: 97 pgs: 97 active+clean; 640 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 1.9 KiB/s wr, 12 op/s 2024-03-28T18:05:14.009 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:14.007+0000 7ff9a1800700 1 -- 172.21.15.22:0/221717312 shutdown_connections 2024-03-28T18:05:14.009 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:14.007+0000 7ff9a1800700 1 --2- 172.21.15.22:0/221717312 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff99c1073b0 0x7ff99c10ef80 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:05:14.009 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:14.007+0000 7ff9a1800700 1 --2- 172.21.15.22:0/221717312 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff99c1022d0 0x7ff99c1026b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:05:14.009 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:14.007+0000 7ff9a1800700 1 --2- 172.21.15.22:0/221717312 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ff99c102c80 0x7ff99c106cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:05:14.009 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:14.007+0000 7ff9a1800700 1 -- 172.21.15.22:0/221717312 >> 172.21.15.22:0/221717312 conn(0x7ff99c0fdb60 msgr2=0x7ff99c0fff80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:05:14.009 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:14.007+0000 7ff9a1800700 1 -- 172.21.15.22:0/221717312 shutdown_connections 2024-03-28T18:05:14.009 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:14.007+0000 7ff9a1800700 1 -- 172.21.15.22:0/221717312 wait complete. 2024-03-28T18:05:14.009 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:14.008+0000 7ff9a1800700 1 Processor -- start 2024-03-28T18:05:14.009 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:14.008+0000 7ff9a1800700 1 -- start start 2024-03-28T18:05:14.009 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:14.009+0000 7ff9a1800700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff99c1022d0 0x7ff99c074490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:05:14.009 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:14.009+0000 7ff9a1800700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ff99c102c80 0x7ff99c0749d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:05:14.010 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:14.009+0000 7ff9a1800700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff99c1073b0 0x7ff99c074f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:05:14.010 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:14.009+0000 7ff9a1800700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff99c112400 con 0x7ff99c1022d0 2024-03-28T18:05:14.010 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:14.009+0000 7ff9a1800700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff99c112280 con 0x7ff99c102c80 2024-03-28T18:05:14.010 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:14.009+0000 7ff9a1800700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff99c112580 con 0x7ff99c1073b0 2024-03-28T18:05:14.011 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:14.009+0000 7ff99be00700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff99c1022d0 0x7ff99c074490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:05:14.011 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:14.009+0000 7ff99b400700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ff99c102c80 0x7ff99c0749d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:05:14.011 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:14.009+0000 7ff99b400700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ff99c102c80 0x7ff99c0749d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.22:55510/0 (socket says 172.21.15.22:55510) 2024-03-28T18:05:14.011 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:14.009+0000 7ff99b400700 1 -- 172.21.15.22:0/412209693 learned_addr learned my addr 172.21.15.22:0/412209693 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-28T18:05:14.012 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:14.010+0000 7ff9a0e00700 1 --2- 172.21.15.22:0/412209693 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff99c1073b0 0x7ff99c074f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:05:14.012 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:14.010+0000 7ff99be00700 1 -- 172.21.15.22:0/412209693 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff99c1073b0 msgr2=0x7ff99c074f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:05:14.012 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:14.010+0000 7ff99be00700 1 --2- 172.21.15.22:0/412209693 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff99c1073b0 0x7ff99c074f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:05:14.012 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:14.010+0000 7ff99be00700 1 -- 172.21.15.22:0/412209693 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ff99c102c80 msgr2=0x7ff99c0749d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:05:14.012 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:14.010+0000 7ff99be00700 1 --2- 172.21.15.22:0/412209693 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ff99c102c80 0x7ff99c0749d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:05:14.012 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:14.010+0000 7ff99be00700 1 -- 172.21.15.22:0/412209693 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff99c1a6d40 con 0x7ff99c1022d0 2024-03-28T18:05:14.012 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:14.010+0000 7ff99be00700 1 --2- 172.21.15.22:0/412209693 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff99c1022d0 0x7ff99c074490 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7ff98c00d750 tx=0x7ff98c00da60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:05:14.012 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:14.011+0000 7ff998c00700 1 -- 172.21.15.22:0/412209693 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff98c017070 con 0x7ff99c1022d0 2024-03-28T18:05:14.012 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:14.011+0000 7ff9a1800700 1 -- 172.21.15.22:0/412209693 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff99c1a6fd0 con 0x7ff99c1022d0 2024-03-28T18:05:14.012 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:14.011+0000 7ff998c00700 1 -- 172.21.15.22:0/412209693 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7ff98c005020 con 0x7ff99c1022d0 2024-03-28T18:05:14.012 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:14.011+0000 7ff998c00700 1 -- 172.21.15.22:0/412209693 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff98c00fa80 con 0x7ff99c1022d0 2024-03-28T18:05:14.012 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:14.011+0000 7ff9a1800700 1 -- 172.21.15.22:0/412209693 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff99c1a74e0 con 0x7ff99c1022d0 2024-03-28T18:05:14.013 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:14.012+0000 7ff9a1800700 1 -- 172.21.15.22:0/412209693 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff99c068e50 con 0x7ff99c1022d0 2024-03-28T18:05:14.013 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:14.014+0000 7ff998c00700 1 -- 172.21.15.22:0/412209693 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 17) v1 ==== 137341+0+0 (secure 0 0 0) 0x7ff98c00fca0 con 0x7ff99c1022d0 2024-03-28T18:05:14.015 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:14.015+0000 7ff998c00700 1 --2- 172.21.15.22:0/412209693 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7ff9800a29a0 0x7ff9800a4e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:05:14.015 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:14.015+0000 7ff998c00700 1 -- 172.21.15.22:0/412209693 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 8153+0+0 (secure 0 0 0) 0x7ff98c0ce310 con 0x7ff99c1022d0 2024-03-28T18:05:14.015 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:14.016+0000 7ff99b400700 1 --2- 172.21.15.22:0/412209693 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7ff9800a29a0 0x7ff9800a4e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:05:14.016 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:14.016+0000 7ff99b400700 1 --2- 172.21.15.22:0/412209693 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7ff9800a29a0 0x7ff9800a4e60 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7ff99c108900 tx=0x7ff988007480 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:05:14.020 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:14.020+0000 7ff998c00700 1 -- 172.21.15.22:0/412209693 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7ff98c09b250 con 0x7ff99c1022d0 2024-03-28T18:05:14.174 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:05:13 smithi134 ceph-mon[35037]: pgmap v383: 97 pgs: 97 active+clean; 640 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 1.9 KiB/s wr, 12 op/s 2024-03-28T18:05:14.182 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:05:13 smithi079 ceph-mon[35312]: pgmap v383: 97 pgs: 97 active+clean; 640 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 1.9 KiB/s wr, 12 op/s 2024-03-28T18:05:14.248 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:14.248+0000 7ff9a1800700 1 -- 172.21.15.22:0/412209693 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ff99c1a7770 con 0x7ff99c1022d0 2024-03-28T18:05:14.253 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:14.253+0000 7ff998c00700 1 -- 172.21.15.22:0/412209693 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v92) v1 ==== 74+0+22522 (secure 0 0 0) 0x7ff98c09ab50 con 0x7ff99c1022d0 2024-03-28T18:05:14.253 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-03-28T18:05:14.253 INFO:teuthology.orchestra.run.smithi022.stdout:{"epoch":92,"fsid":"fd220748-ed2b-11ee-95d0-87774f69a715","created":"2024-03-28T17:54:00.440279+0000","modified":"2024-03-28T18:05:11.725341+0000","last_up_change":"2024-03-28T18:03:43.220095+0000","last_in_change":"2024-03-28T18:03:18.526399+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-28T17:58:33.168872+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-28T18:04:47.004797+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-28T18:04:48.465502+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"90","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"5d708e06-f05b-40ba-bbcf-258ddc4f43d7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6802","nonce":2186290846},{"type":"v1","addr":"172.21.15.22:6803","nonce":2186290846}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6804","nonce":2186290846},{"type":"v1","addr":"172.21.15.22:6805","nonce":2186290846}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6808","nonce":2186290846},{"type":"v1","addr":"172.21.15.22:6809","nonce":2186290846}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6806","nonce":2186290846},{"type":"v1","addr":"172.21.15.22:6807","nonce":2186290846}]},"public_addr":"172.21.15.22:6803/2186290846","cluster_addr":"172.21.15.22:6805/2186290846","heartbeat_back_addr":"172.21.15.22:6809/2186290846","heartbeat_front_addr":"172.21.15.22:6807/2186290846","state":["exists","up"]},{"osd":1,"uuid":"68ad09bb-9256-409a-9867-4e14a2cb94aa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6800","nonce":2835549681},{"type":"v1","addr":"172.21.15.79:6801","nonce":2835549681}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6802","nonce":2835549681},{"type":"v1","addr":"172.21.15.79:6803","nonce":2835549681}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6806","nonce":2835549681},{"type":"v1","addr":"172.21.15.79:6807","nonce":2835549681}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6804","nonce":2835549681},{"type":"v1","addr":"172.21.15.79:6805","nonce":2835549681}]},"public_addr":"172.21.15.79:6801/2835549681","cluster_addr":"172.21.15.79:6803/2835549681","heartbeat_back_addr":"172.21.15.79:6807/2835549681","heartbeat_front_addr":"172.21.15.79:6805/2835549681","state":["exists","up"]},{"osd":2,"uuid":"ed11a5a7-004d-41f3-a5fc-eb2d603bd33f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6800","nonce":470588085},{"type":"v1","addr":"172.21.15.134:6801","nonce":470588085}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6802","nonce":470588085},{"type":"v1","addr":"172.21.15.134:6803","nonce":470588085}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6806","nonce":470588085},{"type":"v1","addr":"172.21.15.134:6807","nonce":470588085}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6804","nonce":470588085},{"type":"v1","addr":"172.21.15.134:6805","nonce":470588085}]},"public_addr":"172.21.15.134:6801/470588085","cluster_addr":"172.21.15.134:6803/470588085","heartbeat_back_addr":"172.21.15.134:6807/470588085","heartbeat_front_addr":"172.21.15.134:6805/470588085","state":["exists","up"]},{"osd":3,"uuid":"81a6c547-2a50-44c3-84d7-23ab6831790c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6810","nonce":1702441854},{"type":"v1","addr":"172.21.15.22:6811","nonce":1702441854}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6812","nonce":1702441854},{"type":"v1","addr":"172.21.15.22:6813","nonce":1702441854}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6816","nonce":1702441854},{"type":"v1","addr":"172.21.15.22:6817","nonce":1702441854}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6814","nonce":1702441854},{"type":"v1","addr":"172.21.15.22:6815","nonce":1702441854}]},"public_addr":"172.21.15.22:6811/1702441854","cluster_addr":"172.21.15.22:6813/1702441854","heartbeat_back_addr":"172.21.15.22:6817/1702441854","heartbeat_front_addr":"172.21.15.22:6815/1702441854","state":["exists","up"]},{"osd":4,"uuid":"d23a254e-7182-4471-8bca-7eb684ec618e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6808","nonce":630132437},{"type":"v1","addr":"172.21.15.79:6809","nonce":630132437}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6810","nonce":630132437},{"type":"v1","addr":"172.21.15.79:6811","nonce":630132437}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6814","nonce":630132437},{"type":"v1","addr":"172.21.15.79:6815","nonce":630132437}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6812","nonce":630132437},{"type":"v1","addr":"172.21.15.79:6813","nonce":630132437}]},"public_addr":"172.21.15.79:6809/630132437","cluster_addr":"172.21.15.79:6811/630132437","heartbeat_back_addr":"172.21.15.79:6815/630132437","heartbeat_front_addr":"172.21.15.79:6813/630132437","state":["exists","up"]},{"osd":5,"uuid":"a7b98502-2023-4f0a-bfa0-6840cc495e0c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":40,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6808","nonce":2135406915},{"type":"v1","addr":"172.21.15.134:6809","nonce":2135406915}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6810","nonce":2135406915},{"type":"v1","addr":"172.21.15.134:6811","nonce":2135406915}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6814","nonce":2135406915},{"type":"v1","addr":"172.21.15.134:6815","nonce":2135406915}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6812","nonce":2135406915},{"type":"v1","addr":"172.21.15.134:6813","nonce":2135406915}]},"public_addr":"172.21.15.134:6809/2135406915","cluster_addr":"172.21.15.134:6811/2135406915","heartbeat_back_addr":"172.21.15.134:6815/2135406915","heartbeat_front_addr":"172.21.15.134:6813/2135406915","state":["exists","up"]},{"osd":6,"uuid":"634c6562-d683-41c8-8242-fc848c38ce1f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6818","nonce":1400680396},{"type":"v1","addr":"172.21.15.22:6819","nonce":1400680396}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6820","nonce":1400680396},{"type":"v1","addr":"172.21.15.22:6821","nonce":1400680396}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6824","nonce":1400680396},{"type":"v1","addr":"172.21.15.22:6825","nonce":1400680396}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6822","nonce":1400680396},{"type":"v1","addr":"172.21.15.22:6823","nonce":1400680396}]},"public_addr":"172.21.15.22:6819/1400680396","cluster_addr":"172.21.15.22:6821/1400680396","heartbeat_back_addr":"172.21.15.22:6825/1400680396","heartbeat_front_addr":"172.21.15.22:6823/1400680396","state":["exists","up"]},{"osd":7,"uuid":"504eb4a8-6731-45ad-932c-1b38ef157871","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":53,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6816","nonce":1287105314},{"type":"v1","addr":"172.21.15.79:6817","nonce":1287105314}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6818","nonce":1287105314},{"type":"v1","addr":"172.21.15.79:6819","nonce":1287105314}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6822","nonce":1287105314},{"type":"v1","addr":"172.21.15.79:6823","nonce":1287105314}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6820","nonce":1287105314},{"type":"v1","addr":"172.21.15.79:6821","nonce":1287105314}]},"public_addr":"172.21.15.79:6817/1287105314","cluster_addr":"172.21.15.79:6819/1287105314","heartbeat_back_addr":"172.21.15.79:6823/1287105314","heartbeat_front_addr":"172.21.15.79:6821/1287105314","state":["exists","up"]},{"osd":8,"uuid":"20aa66c1-03d6-4cb8-a096-0eae41979c17","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":62,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6816","nonce":2265650388},{"type":"v1","addr":"172.21.15.134:6817","nonce":2265650388}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6818","nonce":2265650388},{"type":"v1","addr":"172.21.15.134:6819","nonce":2265650388}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6822","nonce":2265650388},{"type":"v1","addr":"172.21.15.134:6823","nonce":2265650388}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6820","nonce":2265650388},{"type":"v1","addr":"172.21.15.134:6821","nonce":2265650388}]},"public_addr":"172.21.15.134:6817/2265650388","cluster_addr":"172.21.15.134:6819/2265650388","heartbeat_back_addr":"172.21.15.134:6823/2265650388","heartbeat_front_addr":"172.21.15.134:6821/2265650388","state":["exists","up"]},{"osd":9,"uuid":"e1ae4020-7c2c-4334-a753-ee6b12d98584","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":70,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6826","nonce":3762222259},{"type":"v1","addr":"172.21.15.22:6827","nonce":3762222259}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6828","nonce":3762222259},{"type":"v1","addr":"172.21.15.22:6829","nonce":3762222259}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6832","nonce":3762222259},{"type":"v1","addr":"172.21.15.22:6833","nonce":3762222259}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6830","nonce":3762222259},{"type":"v1","addr":"172.21.15.22:6831","nonce":3762222259}]},"public_addr":"172.21.15.22:6827/3762222259","cluster_addr":"172.21.15.22:6829/3762222259","heartbeat_back_addr":"172.21.15.22:6833/3762222259","heartbeat_front_addr":"172.21.15.22:6831/3762222259","state":["exists","up"]},{"osd":10,"uuid":"f5198d52-df48-4679-a860-4dcb440b7a9f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":77,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6824","nonce":1746167904},{"type":"v1","addr":"172.21.15.79:6825","nonce":1746167904}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6826","nonce":1746167904},{"type":"v1","addr":"172.21.15.79:6827","nonce":1746167904}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6830","nonce":1746167904},{"type":"v1","addr":"172.21.15.79:6831","nonce":1746167904}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6828","nonce":1746167904},{"type":"v1","addr":"172.21.15.79:6829","nonce":1746167904}]},"public_addr":"172.21.15.79:6825/1746167904","cluster_addr":"172.21.15.79:6827/1746167904","heartbeat_back_addr":"172.21.15.79:6831/1746167904","heartbeat_front_addr":"172.21.15.79:6829/1746167904","state":["exists","up"]},{"osd":11,"uuid":"1427bca7-3414-4468-9246-e999156bc2ae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":85,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6824","nonce":4271716428},{"type":"v1","addr":"172.21.15.134:6825","nonce":4271716428}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6826","nonce":4271716428},{"type":"v1","addr":"172.21.15.134:6827","nonce":4271716428}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6830","nonce":4271716428},{"type":"v1","addr":"172.21.15.134:6831","nonce":4271716428}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6828","nonce":4271716428},{"type":"v1","addr":"172.21.15.134:6829","nonce":4271716428}]},"public_addr":"172.21.15.134:6825/4271716428","cluster_addr":"172.21.15.134:6827/4271716428","heartbeat_back_addr":"172.21.15.134:6831/4271716428","heartbeat_front_addr":"172.21.15.134:6829/4271716428","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T17:57:21.225125+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T17:57:55.197254+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T17:58:27.825136+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T17:59:04.319893+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T17:59:38.407752+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T18:00:12.118805+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T18:00:47.635214+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T18:01:21.231055+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T18:01:55.817861+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T18:02:31.680160+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T18:03:06.783398+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T18:03:41.307347+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.8","mappings":[{"from":2,"to":11}]},{"pgid":"2.1d","mappings":[{"from":8,"to":11}]},{"pgid":"3.3","mappings":[{"from":4,"to":9}]},{"pgid":"3.c","mappings":[{"from":3,"to":9}]},{"pgid":"3.21","mappings":[{"from":4,"to":8}]},{"pgid":"3.25","mappings":[{"from":5,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.22:0/3521924283":"2024-03-29T17:55:10.425504+0000","172.21.15.22:6801/730635939":"2024-03-29T17:55:10.425504+0000","172.21.15.22:0/2471229459":"2024-03-29T17:54:34.004559+0000","172.21.15.22:6800/3166828081":"2024-03-29T17:54:34.004559+0000","172.21.15.22:6800/730635939":"2024-03-29T17:55:10.425504+0000","172.21.15.22:0/1229047849":"2024-03-29T17:54:34.004559+0000","172.21.15.22:6801/3166828081":"2024-03-29T17:54:34.004559+0000","172.21.15.22:0/2717818856":"2024-03-29T17:55:10.425504+0000","172.21.15.22:0/2157570333":"2024-03-29T17:54:34.004559+0000","172.21.15.22:0/1006589530":"2024-03-29T17:55:10.425504+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-28T18:05:14.256 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:14.256+0000 7ff9a1800700 1 -- 172.21.15.22:0/412209693 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7ff9800a29a0 msgr2=0x7ff9800a4e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:05:14.256 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:14.256+0000 7ff9a1800700 1 --2- 172.21.15.22:0/412209693 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7ff9800a29a0 0x7ff9800a4e60 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7ff99c108900 tx=0x7ff988007480 comp rx=0 tx=0).stop 2024-03-28T18:05:14.257 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:14.256+0000 7ff9a1800700 1 -- 172.21.15.22:0/412209693 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff99c1022d0 msgr2=0x7ff99c074490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:05:14.257 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:14.256+0000 7ff9a1800700 1 --2- 172.21.15.22:0/412209693 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff99c1022d0 0x7ff99c074490 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7ff98c00d750 tx=0x7ff98c00da60 comp rx=0 tx=0).stop 2024-03-28T18:05:14.257 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:14.256+0000 7ff9a1800700 1 -- 172.21.15.22:0/412209693 shutdown_connections 2024-03-28T18:05:14.257 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:14.256+0000 7ff9a1800700 1 --2- 172.21.15.22:0/412209693 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7ff9800a29a0 0x7ff9800a4e60 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:05:14.257 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:14.256+0000 7ff9a1800700 1 --2- 172.21.15.22:0/412209693 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff99c1073b0 0x7ff99c074f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:05:14.257 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:14.256+0000 7ff9a1800700 1 --2- 172.21.15.22:0/412209693 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff99c1022d0 0x7ff99c074490 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:05:14.257 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:14.256+0000 7ff9a1800700 1 --2- 172.21.15.22:0/412209693 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ff99c102c80 0x7ff99c0749d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:05:14.257 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:14.256+0000 7ff9a1800700 1 -- 172.21.15.22:0/412209693 >> 172.21.15.22:0/412209693 conn(0x7ff99c0fdb60 msgr2=0x7ff99c0fe640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:05:14.257 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:14.256+0000 7ff9a1800700 1 -- 172.21.15.22:0/412209693 shutdown_connections 2024-03-28T18:05:14.257 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:14.257+0000 7ff9a1800700 1 -- 172.21.15.22:0/412209693 wait complete. 2024-03-28T18:05:15.032 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:05:14 smithi022 ceph-mon[31985]: from='client.? 172.21.15.22:0/412209693' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-28T18:05:15.139 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033 shell --fsid fd220748-ed2b-11ee-95d0-87774f69a715 -- ceph tell osd.0 flush_pg_stats 2024-03-28T18:05:15.140 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033 shell --fsid fd220748-ed2b-11ee-95d0-87774f69a715 -- ceph tell osd.1 flush_pg_stats 2024-03-28T18:05:15.140 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033 shell --fsid fd220748-ed2b-11ee-95d0-87774f69a715 -- ceph tell osd.2 flush_pg_stats 2024-03-28T18:05:15.140 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033 shell --fsid fd220748-ed2b-11ee-95d0-87774f69a715 -- ceph tell osd.3 flush_pg_stats 2024-03-28T18:05:15.141 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033 shell --fsid fd220748-ed2b-11ee-95d0-87774f69a715 -- ceph tell osd.4 flush_pg_stats 2024-03-28T18:05:15.141 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033 shell --fsid fd220748-ed2b-11ee-95d0-87774f69a715 -- ceph tell osd.5 flush_pg_stats 2024-03-28T18:05:15.142 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033 shell --fsid fd220748-ed2b-11ee-95d0-87774f69a715 -- ceph tell osd.6 flush_pg_stats 2024-03-28T18:05:15.142 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033 shell --fsid fd220748-ed2b-11ee-95d0-87774f69a715 -- ceph tell osd.7 flush_pg_stats 2024-03-28T18:05:15.143 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033 shell --fsid fd220748-ed2b-11ee-95d0-87774f69a715 -- ceph tell osd.8 flush_pg_stats 2024-03-28T18:05:15.143 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033 shell --fsid fd220748-ed2b-11ee-95d0-87774f69a715 -- ceph tell osd.9 flush_pg_stats 2024-03-28T18:05:15.143 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033 shell --fsid fd220748-ed2b-11ee-95d0-87774f69a715 -- ceph tell osd.10 flush_pg_stats 2024-03-28T18:05:15.144 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033 shell --fsid fd220748-ed2b-11ee-95d0-87774f69a715 -- ceph tell osd.11 flush_pg_stats 2024-03-28T18:05:15.174 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:05:14 smithi134 ceph-mon[35037]: from='client.? 172.21.15.22:0/412209693' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-28T18:05:15.182 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:05:14 smithi079 ceph-mon[35312]: from='client.? 172.21.15.22:0/412209693' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-28T18:05:15.775 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:05:15 smithi022 ceph-mon[31985]: pgmap v384: 97 pgs: 97 active+clean; 640 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 7 op/s; 14 B/s, 1 keys/s, 1 objects/s recovering 2024-03-28T18:05:16.174 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:05:15 smithi134 ceph-mon[35037]: pgmap v384: 97 pgs: 97 active+clean; 640 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 7 op/s; 14 B/s, 1 keys/s, 1 objects/s recovering 2024-03-28T18:05:16.182 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:05:15 smithi079 ceph-mon[35312]: pgmap v384: 97 pgs: 97 active+clean; 640 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 7 op/s; 14 B/s, 1 keys/s, 1 objects/s recovering 2024-03-28T18:05:16.253 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/fd220748-ed2b-11ee-95d0-87774f69a715/mon.a/config 2024-03-28T18:05:16.253 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/fd220748-ed2b-11ee-95d0-87774f69a715/mon.a/config 2024-03-28T18:05:16.254 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/fd220748-ed2b-11ee-95d0-87774f69a715/mon.a/config 2024-03-28T18:05:16.268 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/fd220748-ed2b-11ee-95d0-87774f69a715/mon.a/config 2024-03-28T18:05:16.275 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/fd220748-ed2b-11ee-95d0-87774f69a715/mon.a/config 2024-03-28T18:05:16.289 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/fd220748-ed2b-11ee-95d0-87774f69a715/mon.a/config 2024-03-28T18:05:16.294 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/fd220748-ed2b-11ee-95d0-87774f69a715/mon.a/config 2024-03-28T18:05:16.294 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/fd220748-ed2b-11ee-95d0-87774f69a715/mon.a/config 2024-03-28T18:05:18.032 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:05:17 smithi022 ceph-mon[31985]: pgmap v385: 97 pgs: 97 active+clean; 642 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 511 B/s wr, 11 op/s; 14 B/s, 1 keys/s, 1 objects/s recovering 2024-03-28T18:05:18.174 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:05:17 smithi134 ceph-mon[35037]: pgmap v385: 97 pgs: 97 active+clean; 642 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 511 B/s wr, 11 op/s; 14 B/s, 1 keys/s, 1 objects/s recovering 2024-03-28T18:05:18.182 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:05:17 smithi079 ceph-mon[35312]: pgmap v385: 97 pgs: 97 active+clean; 642 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 511 B/s wr, 11 op/s; 14 B/s, 1 keys/s, 1 objects/s recovering 2024-03-28T18:05:20.032 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:05:19 smithi022 ceph-mon[31985]: pgmap v386: 97 pgs: 97 active+clean; 642 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 511 B/s wr, 9 op/s; 14 B/s, 1 keys/s, 1 objects/s recovering 2024-03-28T18:05:20.174 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:05:19 smithi134 ceph-mon[35037]: pgmap v386: 97 pgs: 97 active+clean; 642 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 511 B/s wr, 9 op/s; 14 B/s, 1 keys/s, 1 objects/s recovering 2024-03-28T18:05:20.182 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:05:19 smithi079 ceph-mon[35312]: pgmap v386: 97 pgs: 97 active+clean; 642 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 511 B/s wr, 9 op/s; 14 B/s, 1 keys/s, 1 objects/s recovering 2024-03-28T18:05:21.205 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/fd220748-ed2b-11ee-95d0-87774f69a715/mon.a/config 2024-03-28T18:05:21.208 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/fd220748-ed2b-11ee-95d0-87774f69a715/mon.a/config 2024-03-28T18:05:21.213 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/fd220748-ed2b-11ee-95d0-87774f69a715/mon.a/config 2024-03-28T18:05:21.859 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:05:21 smithi022 ceph-mon[31985]: pgmap v387: 97 pgs: 97 active+clean; 642 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 414 B/s wr, 9 op/s; 11 B/s, 1 keys/s, 0 objects/s recovering 2024-03-28T18:05:22.174 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:05:21 smithi134 ceph-mon[35037]: pgmap v387: 97 pgs: 97 active+clean; 642 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 414 B/s wr, 9 op/s; 11 B/s, 1 keys/s, 0 objects/s recovering 2024-03-28T18:05:22.182 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:05:21 smithi079 ceph-mon[35312]: pgmap v387: 97 pgs: 97 active+clean; 642 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 414 B/s wr, 9 op/s; 11 B/s, 1 keys/s, 0 objects/s recovering 2024-03-28T18:05:23.315 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/fd220748-ed2b-11ee-95d0-87774f69a715/mon.a/config 2024-03-28T18:05:23.809 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:05:23 smithi022 ceph-mon[31985]: pgmap v388: 97 pgs: 97 active+clean; 642 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 376 B/s wr, 9 op/s; 10 B/s, 0 keys/s, 0 objects/s recovering 2024-03-28T18:05:24.174 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:05:23 smithi134 ceph-mon[35037]: pgmap v388: 97 pgs: 97 active+clean; 642 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 376 B/s wr, 9 op/s; 10 B/s, 0 keys/s, 0 objects/s recovering 2024-03-28T18:05:24.182 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:05:23 smithi079 ceph-mon[35312]: pgmap v388: 97 pgs: 97 active+clean; 642 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 376 B/s wr, 9 op/s; 10 B/s, 0 keys/s, 0 objects/s recovering 2024-03-28T18:05:24.920 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:23.763+0000 7f5cbb600700 1 -- 172.21.15.22:0/2826615285 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5cb406a760 msgr2=0x7f5cb410e700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:05:24.920 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:23.763+0000 7f5cbb600700 1 --2- 172.21.15.22:0/2826615285 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5cb406a760 0x7f5cb410e700 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f5c9c009990 tx=0x7f5c9c009ca0 comp rx=0 tx=0).stop 2024-03-28T18:05:24.920 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:23.764+0000 7f5cbb600700 1 -- 172.21.15.22:0/2826615285 shutdown_connections 2024-03-28T18:05:24.920 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:23.764+0000 7f5cbb600700 1 --2- 172.21.15.22:0/2826615285 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f5cb410ec40 0x7f5cb4111030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:05:24.920 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:23.764+0000 7f5cbb600700 1 --2- 172.21.15.22:0/2826615285 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5cb4069e40 0x7f5cb406a220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:05:24.920 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:23.764+0000 7f5cbb600700 1 --2- 172.21.15.22:0/2826615285 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5cb406a760 0x7f5cb410e700 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:05:24.920 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:23.764+0000 7f5cbb600700 1 -- 172.21.15.22:0/2826615285 >> 172.21.15.22:0/2826615285 conn(0x7f5cb4077fc0 msgr2=0x7f5cb40783d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:05:24.920 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:23.764+0000 7f5cbb600700 1 -- 172.21.15.22:0/2826615285 shutdown_connections 2024-03-28T18:05:24.920 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:23.764+0000 7f5cbb600700 1 -- 172.21.15.22:0/2826615285 wait complete. 2024-03-28T18:05:24.920 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:23.765+0000 7f5cbb600700 1 Processor -- start 2024-03-28T18:05:24.920 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:23.765+0000 7f5cbb600700 1 -- start start 2024-03-28T18:05:24.921 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:23.766+0000 7f5cbb600700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5cb4069e40 0x7f5cb41a0800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:05:24.921 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:23.766+0000 7f5cbb600700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5cb406a760 0x7f5cb41a0d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:05:24.921 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:23.766+0000 7f5cbb600700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f5cb410ec40 0x7f5cb419aa10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:05:24.921 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:23.766+0000 7f5cbb600700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5cb41146b0 con 0x7f5cb4069e40 2024-03-28T18:05:24.921 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:23.766+0000 7f5cbb600700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5cb4114530 con 0x7f5cb410ec40 2024-03-28T18:05:24.921 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:23.766+0000 7f5cbb600700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5cb4114830 con 0x7f5cb406a760 2024-03-28T18:05:24.921 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:23.766+0000 7f5cb8c00700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5cb4069e40 0x7f5cb41a0800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:05:24.921 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:23.766+0000 7f5cb8c00700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5cb4069e40 0x7f5cb41a0800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:35600/0 (socket says 172.21.15.22:35600) 2024-03-28T18:05:24.921 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:23.766+0000 7f5cb8c00700 1 -- 172.21.15.22:0/3982709044 learned_addr learned my addr 172.21.15.22:0/3982709044 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-28T18:05:24.921 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:23.766+0000 7f5cb9600700 1 --2- 172.21.15.22:0/3982709044 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f5cb410ec40 0x7f5cb419aa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:05:24.921 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:23.767+0000 7f5cabe00700 1 --2- 172.21.15.22:0/3982709044 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5cb406a760 0x7f5cb41a0d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:05:24.921 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:23.767+0000 7f5cb8c00700 1 -- 172.21.15.22:0/3982709044 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5cb406a760 msgr2=0x7f5cb41a0d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:05:24.921 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:23.767+0000 7f5cb8c00700 1 --2- 172.21.15.22:0/3982709044 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5cb406a760 0x7f5cb41a0d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:05:24.921 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:23.767+0000 7f5cb8c00700 1 -- 172.21.15.22:0/3982709044 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f5cb410ec40 msgr2=0x7f5cb419aa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:05:24.922 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:23.767+0000 7f5cb8c00700 1 --2- 172.21.15.22:0/3982709044 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f5cb410ec40 0x7f5cb419aa10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:05:24.922 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:23.767+0000 7f5cb8c00700 1 -- 172.21.15.22:0/3982709044 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5cb419b170 con 0x7f5cb4069e40 2024-03-28T18:05:24.922 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:23.767+0000 7f5cabe00700 1 --2- 172.21.15.22:0/3982709044 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5cb406a760 0x7f5cb41a0d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T18:05:24.922 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:23.767+0000 7f5cb9600700 1 --2- 172.21.15.22:0/3982709044 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f5cb410ec40 0x7f5cb419aa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-28T18:05:24.922 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:23.767+0000 7f5cb8c00700 1 --2- 172.21.15.22:0/3982709044 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5cb4069e40 0x7f5cb41a0800 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f5ca400d980 tx=0x7f5ca400dd40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:05:24.922 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:23.767+0000 7f5cb2000700 1 -- 172.21.15.22:0/3982709044 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ca4017070 con 0x7f5cb4069e40 2024-03-28T18:05:24.922 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:23.767+0000 7f5cbb600700 1 -- 172.21.15.22:0/3982709044 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5cb419b460 con 0x7f5cb4069e40 2024-03-28T18:05:24.922 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:23.767+0000 7f5cb2000700 1 -- 172.21.15.22:0/3982709044 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f5ca4004d00 con 0x7f5cb4069e40 2024-03-28T18:05:24.922 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:23.767+0000 7f5cbb600700 1 -- 172.21.15.22:0/3982709044 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5cb41a76d0 con 0x7f5cb4069e40 2024-03-28T18:05:24.922 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:23.767+0000 7f5cb2000700 1 -- 172.21.15.22:0/3982709044 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ca40054c0 con 0x7f5cb4069e40 2024-03-28T18:05:24.922 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:23.768+0000 7f5cbb600700 1 -- 172.21.15.22:0/3982709044 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f5cb4062920 con 0x7f5cb4069e40 2024-03-28T18:05:24.922 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:23.771+0000 7f5cb2000700 1 -- 172.21.15.22:0/3982709044 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 17) v1 ==== 137341+0+0 (secure 0 0 0) 0x7f5ca4004750 con 0x7f5cb4069e40 2024-03-28T18:05:24.922 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:23.772+0000 7f5cb2000700 1 --2- 172.21.15.22:0/3982709044 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7f5c980a2920 0x7f5c980a4de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:05:24.922 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:23.772+0000 7f5cb2000700 1 -- 172.21.15.22:0/3982709044 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 8153+0+0 (secure 0 0 0) 0x7f5ca40cf190 con 0x7f5cb4069e40 2024-03-28T18:05:24.922 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:23.772+0000 7f5cabe00700 1 --2- 172.21.15.22:0/3982709044 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7f5c980a2920 0x7f5c980a4de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:05:24.923 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:23.773+0000 7f5cb2000700 1 --2- 172.21.15.22:0/3982709044 >> [v2:172.21.15.134:6816/2265650388,v1:172.21.15.134:6817/2265650388] conn(0x7f5c980ac360 0x7f5c980ae800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:05:24.923 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:23.773+0000 7f5cb2000700 1 -- 172.21.15.22:0/3982709044 --> [v2:172.21.15.134:6816/2265650388,v1:172.21.15.134:6817/2265650388] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f5ca401ce50 con 0x7f5c980ac360 2024-03-28T18:05:24.923 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:23.773+0000 7f5cb2000700 1 -- 172.21.15.22:0/3982709044 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_get_version_reply(handle=1 version=92) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5ca40cf540 con 0x7f5cb4069e40 2024-03-28T18:05:24.923 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:23.773+0000 7f5cabe00700 1 --2- 172.21.15.22:0/3982709044 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7f5c980a2920 0x7f5c980a4de0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f5c9c00b2e0 tx=0x7f5c9c005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:05:24.923 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:23.785+0000 7f5cb9600700 1 --2- 172.21.15.22:0/3982709044 >> [v2:172.21.15.134:6816/2265650388,v1:172.21.15.134:6817/2265650388] conn(0x7f5c980ac360 0x7f5c980ae800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:05:24.923 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:23.786+0000 7f5cb9600700 1 --2- 172.21.15.22:0/3982709044 >> [v2:172.21.15.134:6816/2265650388,v1:172.21.15.134:6817/2265650388] conn(0x7f5c980ac360 0x7f5c980ae800 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:05:24.923 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:23.788+0000 7f5cb2000700 1 -- 172.21.15.22:0/3982709044 <== osd.8 v2:172.21.15.134:6816/2265650388 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7f5ca401ce50 con 0x7f5c980ac360 2024-03-28T18:05:24.924 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:23.808+0000 7f5cbb600700 1 -- 172.21.15.22:0/3982709044 --> [v2:172.21.15.134:6816/2265650388,v1:172.21.15.134:6817/2265650388] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f5cb4062d20 con 0x7f5c980ac360 2024-03-28T18:05:24.924 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:23.809+0000 7f5cb2000700 1 -- 172.21.15.22:0/3982709044 <== osd.8 v2:172.21.15.134:6816/2265650388 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7f5cb4062d20 con 0x7f5c980ac360 2024-03-28T18:05:24.924 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:23.810+0000 7f5cbb600700 1 -- 172.21.15.22:0/3982709044 >> [v2:172.21.15.134:6816/2265650388,v1:172.21.15.134:6817/2265650388] conn(0x7f5c980ac360 msgr2=0x7f5c980ae800 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:05:24.924 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:23.810+0000 7f5cbb600700 1 --2- 172.21.15.22:0/3982709044 >> [v2:172.21.15.134:6816/2265650388,v1:172.21.15.134:6817/2265650388] conn(0x7f5c980ac360 0x7f5c980ae800 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:05:24.924 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:23.810+0000 7f5cbb600700 1 -- 172.21.15.22:0/3982709044 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7f5c980a2920 msgr2=0x7f5c980a4de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:05:24.924 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:23.810+0000 7f5cbb600700 1 --2- 172.21.15.22:0/3982709044 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7f5c980a2920 0x7f5c980a4de0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f5c9c00b2e0 tx=0x7f5c9c005fb0 comp rx=0 tx=0).stop 2024-03-28T18:05:24.924 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:23.810+0000 7f5cbb600700 1 -- 172.21.15.22:0/3982709044 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5cb4069e40 msgr2=0x7f5cb41a0800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:05:24.924 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:23.810+0000 7f5cbb600700 1 --2- 172.21.15.22:0/3982709044 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5cb4069e40 0x7f5cb41a0800 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f5ca400d980 tx=0x7f5ca400dd40 comp rx=0 tx=0).stop 2024-03-28T18:05:24.924 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:23.810+0000 7f5cb9600700 1 -- 172.21.15.22:0/3982709044 reap_dead start 2024-03-28T18:05:24.924 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:23.810+0000 7f5cbb600700 1 -- 172.21.15.22:0/3982709044 shutdown_connections 2024-03-28T18:05:24.924 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:23.810+0000 7f5cbb600700 1 -- 172.21.15.22:0/3982709044 >> 172.21.15.22:0/3982709044 conn(0x7f5cb4077fc0 msgr2=0x7f5cb41100d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:05:24.924 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:23.810+0000 7f5cbb600700 1 -- 172.21.15.22:0/3982709044 shutdown_connections 2024-03-28T18:05:24.924 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:23.810+0000 7f5cbb600700 1 -- 172.21.15.22:0/3982709044 wait complete. 2024-03-28T18:05:26.032 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:05:25 smithi022 ceph-mon[31985]: pgmap v389: 97 pgs: 97 active+clean; 642 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 341 B/s wr, 8 op/s; 9 B/s, 0 keys/s, 0 objects/s recovering 2024-03-28T18:05:26.174 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:05:25 smithi134 ceph-mon[35037]: pgmap v389: 97 pgs: 97 active+clean; 642 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 341 B/s wr, 8 op/s; 9 B/s, 0 keys/s, 0 objects/s recovering 2024-03-28T18:05:26.182 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:05:25 smithi079 ceph-mon[35312]: pgmap v389: 97 pgs: 97 active+clean; 642 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 341 B/s wr, 8 op/s; 9 B/s, 0 keys/s, 0 objects/s recovering 2024-03-28T18:05:26.686 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:26.275+0000 7fe0e8e00700 1 -- 172.21.15.22:0/4219162628 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe0e406a7e0 msgr2=0x7fe0e410f020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:05:26.687 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:26.275+0000 7fe0e8e00700 1 --2- 172.21.15.22:0/4219162628 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe0e406a7e0 0x7fe0e410f020 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fe0dc00b0d0 tx=0x7fe0dc00b3e0 comp rx=0 tx=0).stop 2024-03-28T18:05:26.687 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:26.275+0000 7fe0e8e00700 1 -- 172.21.15.22:0/4219162628 shutdown_connections 2024-03-28T18:05:26.687 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:26.275+0000 7fe0e8e00700 1 --2- 172.21.15.22:0/4219162628 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe0e406a7e0 0x7fe0e410f020 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:05:26.687 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:26.275+0000 7fe0e8e00700 1 --2- 172.21.15.22:0/4219162628 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe0e41081e0 0x7fe0e41085c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:05:26.687 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:26.275+0000 7fe0e8e00700 1 --2- 172.21.15.22:0/4219162628 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fe0e4069e40 0x7fe0e406a2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:05:26.687 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:26.275+0000 7fe0e8e00700 1 -- 172.21.15.22:0/4219162628 >> 172.21.15.22:0/4219162628 conn(0x7fe0e4077fc0 msgr2=0x7fe0e40783d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:05:26.687 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:26.276+0000 7fe0e8e00700 1 -- 172.21.15.22:0/4219162628 shutdown_connections 2024-03-28T18:05:26.687 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:26.276+0000 7fe0e8e00700 1 -- 172.21.15.22:0/4219162628 wait complete. 2024-03-28T18:05:26.687 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:26.277+0000 7fe0e8e00700 1 Processor -- start 2024-03-28T18:05:26.687 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:26.277+0000 7fe0e8e00700 1 -- start start 2024-03-28T18:05:26.687 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:26.278+0000 7fe0e8e00700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe0e4069e40 0x7fe0e419e620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:05:26.687 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:26.278+0000 7fe0e8e00700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fe0e406a7e0 0x7fe0e419eb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:05:26.688 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:26.278+0000 7fe0e8e00700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe0e41081e0 0x7fe0e41986f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:05:26.688 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:26.278+0000 7fe0e8e00700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0e4112470 con 0x7fe0e4069e40 2024-03-28T18:05:26.688 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:26.278+0000 7fe0e8e00700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0e41122f0 con 0x7fe0e406a7e0 2024-03-28T18:05:26.688 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:26.278+0000 7fe0e8e00700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0e41125f0 con 0x7fe0e41081e0 2024-03-28T18:05:26.688 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:26.278+0000 7fe0e1e00700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe0e4069e40 0x7fe0e419e620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:05:26.688 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:26.278+0000 7fe0e1e00700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe0e4069e40 0x7fe0e419e620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:35616/0 (socket says 172.21.15.22:35616) 2024-03-28T18:05:26.688 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:26.278+0000 7fe0e1e00700 1 -- 172.21.15.22:0/1265089688 learned_addr learned my addr 172.21.15.22:0/1265089688 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-28T18:05:26.688 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:26.278+0000 7fe0dbe00700 1 --2- 172.21.15.22:0/1265089688 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fe0e406a7e0 0x7fe0e419eb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:05:26.688 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:26.279+0000 7fe0e2800700 1 --2- 172.21.15.22:0/1265089688 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe0e41081e0 0x7fe0e41986f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:05:26.688 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:26.279+0000 7fe0e1e00700 1 -- 172.21.15.22:0/1265089688 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe0e41081e0 msgr2=0x7fe0e41986f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:05:26.688 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:26.279+0000 7fe0e1e00700 1 --2- 172.21.15.22:0/1265089688 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe0e41081e0 0x7fe0e41986f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:05:26.688 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:26.279+0000 7fe0e1e00700 1 -- 172.21.15.22:0/1265089688 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fe0e406a7e0 msgr2=0x7fe0e419eb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:05:26.688 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:26.279+0000 7fe0e1e00700 1 --2- 172.21.15.22:0/1265089688 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fe0e406a7e0 0x7fe0e419eb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:05:26.688 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:26.279+0000 7fe0e1e00700 1 -- 172.21.15.22:0/1265089688 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe0e4198fd0 con 0x7fe0e4069e40 2024-03-28T18:05:26.689 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:26.279+0000 7fe0e1e00700 1 --2- 172.21.15.22:0/1265089688 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe0e4069e40 0x7fe0e419e620 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fe0d400d950 tx=0x7fe0d400dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:05:26.689 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:26.280+0000 7fe0daa00700 1 -- 172.21.15.22:0/1265089688 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0d4016070 con 0x7fe0e4069e40 2024-03-28T18:05:26.689 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:26.280+0000 7fe0e8e00700 1 -- 172.21.15.22:0/1265089688 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe0e41992c0 con 0x7fe0e4069e40 2024-03-28T18:05:26.689 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:26.280+0000 7fe0daa00700 1 -- 172.21.15.22:0/1265089688 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7fe0d4009e10 con 0x7fe0e4069e40 2024-03-28T18:05:26.689 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:26.280+0000 7fe0daa00700 1 -- 172.21.15.22:0/1265089688 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0d400f620 con 0x7fe0e4069e40 2024-03-28T18:05:26.689 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:26.280+0000 7fe0e8e00700 1 -- 172.21.15.22:0/1265089688 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe0e41a55a0 con 0x7fe0e4069e40 2024-03-28T18:05:26.689 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:26.281+0000 7fe0e8e00700 1 -- 172.21.15.22:0/1265089688 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fe0e40628a0 con 0x7fe0e4069e40 2024-03-28T18:05:26.689 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:26.284+0000 7fe0daa00700 1 -- 172.21.15.22:0/1265089688 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 17) v1 ==== 137341+0+0 (secure 0 0 0) 0x7fe0d4019070 con 0x7fe0e4069e40 2024-03-28T18:05:26.690 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:26.285+0000 7fe0daa00700 1 --2- 172.21.15.22:0/1265089688 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7fe0c80a29a0 0x7fe0c80a4e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:05:26.690 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:26.285+0000 7fe0daa00700 1 -- 172.21.15.22:0/1265089688 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 8153+0+0 (secure 0 0 0) 0x7fe0d40cec10 con 0x7fe0e4069e40 2024-03-28T18:05:26.690 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:26.286+0000 7fe0dbe00700 1 --2- 172.21.15.22:0/1265089688 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7fe0c80a29a0 0x7fe0c80a4e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:05:26.690 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:26.286+0000 7fe0daa00700 1 --2- 172.21.15.22:0/1265089688 >> [v2:172.21.15.134:6800/470588085,v1:172.21.15.134:6801/470588085] conn(0x7fe0c80ac3e0 0x7fe0c80ae880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:05:26.690 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:26.286+0000 7fe0daa00700 1 -- 172.21.15.22:0/1265089688 --> [v2:172.21.15.134:6800/470588085,v1:172.21.15.134:6801/470588085] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fe0d4005690 con 0x7fe0c80ac3e0 2024-03-28T18:05:26.690 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:26.286+0000 7fe0daa00700 1 -- 172.21.15.22:0/1265089688 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_get_version_reply(handle=1 version=92) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe0d40cefc0 con 0x7fe0e4069e40 2024-03-28T18:05:26.690 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:26.286+0000 7fe0e2800700 1 --2- 172.21.15.22:0/1265089688 >> [v2:172.21.15.134:6800/470588085,v1:172.21.15.134:6801/470588085] conn(0x7fe0c80ac3e0 0x7fe0c80ae880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:05:26.690 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:26.286+0000 7fe0dbe00700 1 --2- 172.21.15.22:0/1265089688 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7fe0c80a29a0 0x7fe0c80a4e60 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fe0e4199eb0 tx=0x7fe0cc009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:05:26.690 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:26.287+0000 7fe0e2800700 1 --2- 172.21.15.22:0/1265089688 >> [v2:172.21.15.134:6800/470588085,v1:172.21.15.134:6801/470588085] conn(0x7fe0c80ac3e0 0x7fe0c80ae880 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:05:26.690 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:26.289+0000 7fe0daa00700 1 -- 172.21.15.22:0/1265089688 <== osd.2 v2:172.21.15.134:6800/470588085 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7fe0d4005690 con 0x7fe0c80ac3e0 2024-03-28T18:05:26.690 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:26.311+0000 7fe0e8e00700 1 -- 172.21.15.22:0/1265089688 --> [v2:172.21.15.134:6800/470588085,v1:172.21.15.134:6801/470588085] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fe0e4062ce0 con 0x7fe0c80ac3e0 2024-03-28T18:05:26.690 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:26.312+0000 7fe0daa00700 1 -- 172.21.15.22:0/1265089688 <== osd.2 v2:172.21.15.134:6800/470588085 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7fe0e4062ce0 con 0x7fe0c80ac3e0 2024-03-28T18:05:26.690 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:26.312+0000 7fe0e8e00700 1 -- 172.21.15.22:0/1265089688 >> [v2:172.21.15.134:6800/470588085,v1:172.21.15.134:6801/470588085] conn(0x7fe0c80ac3e0 msgr2=0x7fe0c80ae880 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:05:26.690 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:26.312+0000 7fe0e8e00700 1 --2- 172.21.15.22:0/1265089688 >> [v2:172.21.15.134:6800/470588085,v1:172.21.15.134:6801/470588085] conn(0x7fe0c80ac3e0 0x7fe0c80ae880 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:05:26.691 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:26.312+0000 7fe0e8e00700 1 -- 172.21.15.22:0/1265089688 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7fe0c80a29a0 msgr2=0x7fe0c80a4e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:05:26.691 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:26.312+0000 7fe0e8e00700 1 --2- 172.21.15.22:0/1265089688 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7fe0c80a29a0 0x7fe0c80a4e60 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fe0e4199eb0 tx=0x7fe0cc009260 comp rx=0 tx=0).stop 2024-03-28T18:05:26.691 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:26.312+0000 7fe0e8e00700 1 -- 172.21.15.22:0/1265089688 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe0e4069e40 msgr2=0x7fe0e419e620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:05:26.691 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:26.312+0000 7fe0e8e00700 1 --2- 172.21.15.22:0/1265089688 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe0e4069e40 0x7fe0e419e620 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fe0d400d950 tx=0x7fe0d400dc60 comp rx=0 tx=0).stop 2024-03-28T18:05:26.691 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:26.313+0000 7fe0e2800700 1 -- 172.21.15.22:0/1265089688 reap_dead start 2024-03-28T18:05:26.691 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:26.313+0000 7fe0e8e00700 1 -- 172.21.15.22:0/1265089688 shutdown_connections 2024-03-28T18:05:26.691 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:26.313+0000 7fe0e8e00700 1 -- 172.21.15.22:0/1265089688 >> 172.21.15.22:0/1265089688 conn(0x7fe0e4077fc0 msgr2=0x7fe0e40ffe70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:05:26.691 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:26.313+0000 7fe0e8e00700 1 -- 172.21.15.22:0/1265089688 shutdown_connections 2024-03-28T18:05:26.691 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:26.313+0000 7fe0e8e00700 1 -- 172.21.15.22:0/1265089688 wait complete. 2024-03-28T18:05:27.957 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:05:27 smithi022 ceph-mon[31985]: pgmap v390: 97 pgs: 97 active+clean; 642 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 341 B/s wr, 12 op/s 2024-03-28T18:05:28.048 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:27.879+0000 7f0a85400700 1 -- 172.21.15.22:0/700223934 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0a800743c0 msgr2=0x7f0a800747a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:05:28.048 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:27.879+0000 7f0a85400700 1 --2- 172.21.15.22:0/700223934 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0a800743c0 0x7f0a800747a0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f0a70009990 tx=0x7f0a70009ca0 comp rx=0 tx=0).stop 2024-03-28T18:05:28.048 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:27.879+0000 7f0a85400700 1 -- 172.21.15.22:0/700223934 shutdown_connections 2024-03-28T18:05:28.048 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:27.879+0000 7f0a85400700 1 --2- 172.21.15.22:0/700223934 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f0a8010eb90 0x7f0a80110f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:05:28.048 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:27.879+0000 7f0a85400700 1 --2- 172.21.15.22:0/700223934 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0a800743c0 0x7f0a800747a0 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:05:28.048 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:27.879+0000 7f0a85400700 1 --2- 172.21.15.22:0/700223934 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0a80074ce0 0x7f0a8010e650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:05:28.048 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:27.879+0000 7f0a85400700 1 -- 172.21.15.22:0/700223934 >> 172.21.15.22:0/700223934 conn(0x7f0a800792c0 msgr2=0x7f0a800796d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:05:28.049 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:27.879+0000 7f0a85400700 1 -- 172.21.15.22:0/700223934 shutdown_connections 2024-03-28T18:05:28.049 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:27.880+0000 7f0a85400700 1 -- 172.21.15.22:0/700223934 wait complete. 2024-03-28T18:05:28.049 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:27.881+0000 7f0a85400700 1 Processor -- start 2024-03-28T18:05:28.049 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:27.881+0000 7f0a85400700 1 -- start start 2024-03-28T18:05:28.049 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:27.881+0000 7f0a85400700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0a800743c0 0x7f0a801a07e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:05:28.049 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:27.882+0000 7f0a85400700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f0a80074ce0 0x7f0a801a0d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:05:28.049 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:27.882+0000 7f0a85400700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0a8010eb90 0x7f0a8019a9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:05:28.049 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:27.882+0000 7f0a85400700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a801146b0 con 0x7f0a8010eb90 2024-03-28T18:05:28.049 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:27.882+0000 7f0a85400700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a80114530 con 0x7f0a80074ce0 2024-03-28T18:05:28.049 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:27.882+0000 7f0a85400700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a80114830 con 0x7f0a800743c0 2024-03-28T18:05:28.049 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:27.882+0000 7f0a7de00700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f0a80074ce0 0x7f0a801a0d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:05:28.049 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:27.882+0000 7f0a7de00700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f0a80074ce0 0x7f0a801a0d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.22:56892/0 (socket says 172.21.15.22:56892) 2024-03-28T18:05:28.049 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:27.882+0000 7f0a7de00700 1 -- 172.21.15.22:0/2685409215 learned_addr learned my addr 172.21.15.22:0/2685409215 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-28T18:05:28.049 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:27.882+0000 7f0a7f200700 1 --2- 172.21.15.22:0/2685409215 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0a8010eb90 0x7f0a8019a9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:05:28.050 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:27.882+0000 7f0a7e800700 1 --2- 172.21.15.22:0/2685409215 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0a800743c0 0x7f0a801a07e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:05:28.050 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:27.883+0000 7f0a7de00700 1 -- 172.21.15.22:0/2685409215 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0a800743c0 msgr2=0x7f0a801a07e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:05:28.050 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:27.883+0000 7f0a7de00700 1 --2- 172.21.15.22:0/2685409215 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0a800743c0 0x7f0a801a07e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:05:28.050 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:27.883+0000 7f0a7de00700 1 -- 172.21.15.22:0/2685409215 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0a8010eb90 msgr2=0x7f0a8019a9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:05:28.050 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:27.883+0000 7f0a7de00700 1 --2- 172.21.15.22:0/2685409215 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0a8010eb90 0x7f0a8019a9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:05:28.050 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:27.883+0000 7f0a7de00700 1 -- 172.21.15.22:0/2685409215 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a8019b150 con 0x7f0a80074ce0 2024-03-28T18:05:28.050 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:27.883+0000 7f0a7de00700 1 --2- 172.21.15.22:0/2685409215 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f0a80074ce0 0x7f0a801a0d20 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f0a6c00d980 tx=0x7f0a6c00dc90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:05:28.050 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:27.883+0000 7f0a77400700 1 -- 172.21.15.22:0/2685409215 <== mon.2 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a6c017070 con 0x7f0a80074ce0 2024-03-28T18:05:28.050 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:27.883+0000 7f0a77400700 1 -- 172.21.15.22:0/2685409215 <== mon.2 v2:172.21.15.79:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f0a6c005020 con 0x7f0a80074ce0 2024-03-28T18:05:28.050 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:27.883+0000 7f0a85400700 1 -- 172.21.15.22:0/2685409215 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0a8019b440 con 0x7f0a80074ce0 2024-03-28T18:05:28.050 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:27.883+0000 7f0a77400700 1 -- 172.21.15.22:0/2685409215 <== mon.2 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a6c009a90 con 0x7f0a80074ce0 2024-03-28T18:05:28.050 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:27.883+0000 7f0a85400700 1 -- 172.21.15.22:0/2685409215 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0a801a76b0 con 0x7f0a80074ce0 2024-03-28T18:05:28.050 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:27.884+0000 7f0a85400700 1 -- 172.21.15.22:0/2685409215 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f0a800628a0 con 0x7f0a80074ce0 2024-03-28T18:05:28.050 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:27.888+0000 7f0a77400700 1 -- 172.21.15.22:0/2685409215 <== mon.2 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 17) v1 ==== 137341+0+0 (secure 0 0 0) 0x7f0a6c009c30 con 0x7f0a80074ce0 2024-03-28T18:05:28.051 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:27.889+0000 7f0a77400700 1 --2- 172.21.15.22:0/2685409215 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7f0a640a2920 0x7f0a640a4de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:05:28.051 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:27.889+0000 7f0a77400700 1 -- 172.21.15.22:0/2685409215 <== mon.2 v2:172.21.15.79:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 8153+0+0 (secure 0 0 0) 0x7f0a6c0cef90 con 0x7f0a80074ce0 2024-03-28T18:05:28.051 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:27.889+0000 7f0a7e800700 1 --2- 172.21.15.22:0/2685409215 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7f0a640a2920 0x7f0a640a4de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:05:28.051 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:27.890+0000 7f0a77400700 1 --2- 172.21.15.22:0/2685409215 >> [v2:172.21.15.79:6816/1287105314,v1:172.21.15.79:6817/1287105314] conn(0x7f0a640ac360 0x7f0a640ae800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:05:28.051 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:27.890+0000 7f0a77400700 1 -- 172.21.15.22:0/2685409215 --> [v2:172.21.15.79:6816/1287105314,v1:172.21.15.79:6817/1287105314] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f0a640aeed0 con 0x7f0a640ac360 2024-03-28T18:05:28.051 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:27.890+0000 7f0a77400700 1 -- 172.21.15.22:0/2685409215 <== mon.2 v2:172.21.15.79:3300/0 6 ==== mon_get_version_reply(handle=1 version=92) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0a6c0cf340 con 0x7f0a80074ce0 2024-03-28T18:05:28.051 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:27.890+0000 7f0a7f200700 1 --2- 172.21.15.22:0/2685409215 >> [v2:172.21.15.79:6816/1287105314,v1:172.21.15.79:6817/1287105314] conn(0x7f0a640ac360 0x7f0a640ae800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:05:28.051 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:27.890+0000 7f0a7e800700 1 --2- 172.21.15.22:0/2685409215 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7f0a640a2920 0x7f0a640a4de0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f0a80079a70 tx=0x7f0a7000b260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:05:28.051 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:27.891+0000 7f0a7f200700 1 --2- 172.21.15.22:0/2685409215 >> [v2:172.21.15.79:6816/1287105314,v1:172.21.15.79:6817/1287105314] conn(0x7f0a640ac360 0x7f0a640ae800 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:05:28.051 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:27.893+0000 7f0a77400700 1 -- 172.21.15.22:0/2685409215 <== osd.7 v2:172.21.15.79:6816/1287105314 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7f0a640aeed0 con 0x7f0a640ac360 2024-03-28T18:05:28.052 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:27.915+0000 7f0a85400700 1 -- 172.21.15.22:0/2685409215 --> [v2:172.21.15.79:6816/1287105314,v1:172.21.15.79:6817/1287105314] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f0a80062d20 con 0x7f0a640ac360 2024-03-28T18:05:28.052 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:27.916+0000 7f0a77400700 1 -- 172.21.15.22:0/2685409215 <== osd.7 v2:172.21.15.79:6816/1287105314 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7f0a80062d20 con 0x7f0a640ac360 2024-03-28T18:05:28.052 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:27.916+0000 7f0a85400700 1 -- 172.21.15.22:0/2685409215 >> [v2:172.21.15.79:6816/1287105314,v1:172.21.15.79:6817/1287105314] conn(0x7f0a640ac360 msgr2=0x7f0a640ae800 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:05:28.052 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:27.916+0000 7f0a85400700 1 --2- 172.21.15.22:0/2685409215 >> [v2:172.21.15.79:6816/1287105314,v1:172.21.15.79:6817/1287105314] conn(0x7f0a640ac360 0x7f0a640ae800 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:05:28.052 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:27.916+0000 7f0a85400700 1 -- 172.21.15.22:0/2685409215 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7f0a640a2920 msgr2=0x7f0a640a4de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:05:28.052 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:27.916+0000 7f0a85400700 1 --2- 172.21.15.22:0/2685409215 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7f0a640a2920 0x7f0a640a4de0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f0a80079a70 tx=0x7f0a7000b260 comp rx=0 tx=0).stop 2024-03-28T18:05:28.052 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:27.916+0000 7f0a85400700 1 -- 172.21.15.22:0/2685409215 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f0a80074ce0 msgr2=0x7f0a801a0d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:05:28.052 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:27.916+0000 7f0a85400700 1 --2- 172.21.15.22:0/2685409215 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f0a80074ce0 0x7f0a801a0d20 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f0a6c00d980 tx=0x7f0a6c00dc90 comp rx=0 tx=0).stop 2024-03-28T18:05:28.052 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:27.916+0000 7f0a7f200700 1 -- 172.21.15.22:0/2685409215 reap_dead start 2024-03-28T18:05:28.052 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:27.917+0000 7f0a85400700 1 -- 172.21.15.22:0/2685409215 shutdown_connections 2024-03-28T18:05:28.053 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:27.917+0000 7f0a85400700 1 -- 172.21.15.22:0/2685409215 >> 172.21.15.22:0/2685409215 conn(0x7f0a800792c0 msgr2=0x7f0a800ff3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:05:28.053 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:27.917+0000 7f0a85400700 1 -- 172.21.15.22:0/2685409215 shutdown_connections 2024-03-28T18:05:28.053 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:27.917+0000 7f0a85400700 1 -- 172.21.15.22:0/2685409215 wait complete. 2024-03-28T18:05:28.174 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:05:27 smithi134 ceph-mon[35037]: pgmap v390: 97 pgs: 97 active+clean; 642 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 341 B/s wr, 12 op/s 2024-03-28T18:05:28.182 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:05:27 smithi079 ceph-mon[35312]: pgmap v390: 97 pgs: 97 active+clean; 642 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 341 B/s wr, 12 op/s 2024-03-28T18:05:30.032 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:05:29 smithi022 ceph-mon[31985]: pgmap v391: 97 pgs: 97 active+clean; 642 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-03-28T18:05:30.174 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:05:29 smithi134 ceph-mon[35037]: pgmap v391: 97 pgs: 97 active+clean; 642 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-03-28T18:05:30.182 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:05:29 smithi079 ceph-mon[35312]: pgmap v391: 97 pgs: 97 active+clean; 642 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-03-28T18:05:30.344 INFO:teuthology.orchestra.run.smithi022.stdout:77309411415 2024-03-28T18:05:30.344 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033 shell --fsid fd220748-ed2b-11ee-95d0-87774f69a715 -- ceph osd last-stat-seq osd.2 2024-03-28T18:05:30.896 INFO:teuthology.orchestra.run.smithi022.stdout:266287972396 2024-03-28T18:05:30.897 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033 shell --fsid fd220748-ed2b-11ee-95d0-87774f69a715 -- ceph osd last-stat-seq osd.8 2024-03-28T18:05:31.325 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:29.858+0000 7ff195c00700 1 -- 172.21.15.22:0/3896317036 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff190109160 msgr2=0x7ff190115670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:05:31.325 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:29.858+0000 7ff195c00700 1 --2- 172.21.15.22:0/3896317036 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff190109160 0x7ff190115670 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7ff18800b0a0 tx=0x7ff18800b3b0 comp rx=0 tx=0).stop 2024-03-28T18:05:31.325 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:29.858+0000 7ff195c00700 1 -- 172.21.15.22:0/3896317036 shutdown_connections 2024-03-28T18:05:31.325 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:29.858+0000 7ff195c00700 1 --2- 172.21.15.22:0/3896317036 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff190109160 0x7ff190115670 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:05:31.325 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:29.858+0000 7ff195c00700 1 --2- 172.21.15.22:0/3896317036 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ff19010e7a0 0x7ff19010eb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:05:31.325 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:29.858+0000 7ff195c00700 1 --2- 172.21.15.22:0/3896317036 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff1901087a0 0x7ff190108c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:05:31.325 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:29.858+0000 7ff195c00700 1 -- 172.21.15.22:0/3896317036 >> 172.21.15.22:0/3896317036 conn(0x7ff190075960 msgr2=0x7ff190077d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:05:31.325 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:29.858+0000 7ff195c00700 1 -- 172.21.15.22:0/3896317036 shutdown_connections 2024-03-28T18:05:31.325 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:29.858+0000 7ff195c00700 1 -- 172.21.15.22:0/3896317036 wait complete. 2024-03-28T18:05:31.325 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:29.859+0000 7ff195c00700 1 Processor -- start 2024-03-28T18:05:31.326 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:29.859+0000 7ff195c00700 1 -- start start 2024-03-28T18:05:31.326 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:29.860+0000 7ff195c00700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff1901087a0 0x7ff19019eea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:05:31.326 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:29.860+0000 7ff195c00700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff190109160 0x7ff19019f3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:05:31.326 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:29.860+0000 7ff195c00700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ff19010e7a0 0x7ff1901a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:05:31.326 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:29.860+0000 7ff195c00700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff190118b50 con 0x7ff1901087a0 2024-03-28T18:05:31.326 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:29.860+0000 7ff195c00700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1901189d0 con 0x7ff19010e7a0 2024-03-28T18:05:31.326 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:29.860+0000 7ff195c00700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff190118cd0 con 0x7ff190109160 2024-03-28T18:05:31.326 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:29.860+0000 7ff18ea00700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff1901087a0 0x7ff19019eea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:05:31.326 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:29.860+0000 7ff18ea00700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff1901087a0 0x7ff19019eea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:50512/0 (socket says 172.21.15.22:50512) 2024-03-28T18:05:31.326 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:29.860+0000 7ff18ea00700 1 -- 172.21.15.22:0/3971913254 learned_addr learned my addr 172.21.15.22:0/3971913254 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-28T18:05:31.326 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:29.861+0000 7ff18e000700 1 --2- 172.21.15.22:0/3971913254 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff190109160 0x7ff19019f3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:05:31.326 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:29.861+0000 7ff18f400700 1 --2- 172.21.15.22:0/3971913254 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ff19010e7a0 0x7ff1901a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:05:31.326 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:29.861+0000 7ff18ea00700 1 -- 172.21.15.22:0/3971913254 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff190109160 msgr2=0x7ff19019f3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:05:31.326 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:29.861+0000 7ff18ea00700 1 --2- 172.21.15.22:0/3971913254 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff190109160 0x7ff19019f3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:05:31.326 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:29.861+0000 7ff18ea00700 1 -- 172.21.15.22:0/3971913254 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ff19010e7a0 msgr2=0x7ff1901a3770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:05:31.327 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:29.861+0000 7ff18ea00700 1 --2- 172.21.15.22:0/3971913254 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ff19010e7a0 0x7ff1901a3770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:05:31.327 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:29.861+0000 7ff18ea00700 1 -- 172.21.15.22:0/3971913254 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff1901a3eb0 con 0x7ff1901087a0 2024-03-28T18:05:31.327 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:29.861+0000 7ff18f400700 1 --2- 172.21.15.22:0/3971913254 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ff19010e7a0 0x7ff1901a3770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-28T18:05:31.327 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:29.861+0000 7ff18ea00700 1 --2- 172.21.15.22:0/3971913254 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff1901087a0 0x7ff19019eea0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7ff18000d950 tx=0x7ff18000dd10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:05:31.327 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:29.862+0000 7ff187400700 1 -- 172.21.15.22:0/3971913254 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff180016070 con 0x7ff1901087a0 2024-03-28T18:05:31.327 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:29.862+0000 7ff187400700 1 -- 172.21.15.22:0/3971913254 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7ff180009b40 con 0x7ff1901087a0 2024-03-28T18:05:31.327 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:29.862+0000 7ff195c00700 1 -- 172.21.15.22:0/3971913254 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff1901a41a0 con 0x7ff1901087a0 2024-03-28T18:05:31.327 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:29.862+0000 7ff187400700 1 -- 172.21.15.22:0/3971913254 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff18000f620 con 0x7ff1901087a0 2024-03-28T18:05:31.327 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:29.862+0000 7ff195c00700 1 -- 172.21.15.22:0/3971913254 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff1901aba20 con 0x7ff1901087a0 2024-03-28T18:05:31.327 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:29.863+0000 7ff195c00700 1 -- 172.21.15.22:0/3971913254 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7ff190076840 con 0x7ff1901087a0 2024-03-28T18:05:31.327 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:29.865+0000 7ff187400700 1 -- 172.21.15.22:0/3971913254 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 17) v1 ==== 137341+0+0 (secure 0 0 0) 0x7ff180019070 con 0x7ff1901087a0 2024-03-28T18:05:31.327 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:29.866+0000 7ff187400700 1 --2- 172.21.15.22:0/3971913254 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7ff1740a2920 0x7ff1740a4de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:05:31.327 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:29.867+0000 7ff187400700 1 -- 172.21.15.22:0/3971913254 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 8153+0+0 (secure 0 0 0) 0x7ff1800cebe0 con 0x7ff1901087a0 2024-03-28T18:05:31.327 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:29.867+0000 7ff18e000700 1 --2- 172.21.15.22:0/3971913254 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7ff1740a2920 0x7ff1740a4de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:05:31.327 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:29.867+0000 7ff187400700 1 --2- 172.21.15.22:0/3971913254 >> [v2:172.21.15.22:6826/3762222259,v1:172.21.15.22:6827/3762222259] conn(0x7ff1740ac360 0x7ff1740ae800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:05:31.328 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:29.867+0000 7ff187400700 1 -- 172.21.15.22:0/3971913254 --> [v2:172.21.15.22:6826/3762222259,v1:172.21.15.22:6827/3762222259] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7ff18000ae50 con 0x7ff1740ac360 2024-03-28T18:05:31.328 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:29.867+0000 7ff187400700 1 -- 172.21.15.22:0/3971913254 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_get_version_reply(handle=1 version=92) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff1800cf0c0 con 0x7ff1901087a0 2024-03-28T18:05:31.328 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:29.867+0000 7ff18f400700 1 --2- 172.21.15.22:0/3971913254 >> [v2:172.21.15.22:6826/3762222259,v1:172.21.15.22:6827/3762222259] conn(0x7ff1740ac360 0x7ff1740ae800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:05:31.328 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:29.868+0000 7ff18e000700 1 --2- 172.21.15.22:0/3971913254 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7ff1740a2920 0x7ff1740a4de0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7ff17c005950 tx=0x7ff17c0058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:05:31.328 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:29.868+0000 7ff18f400700 1 --2- 172.21.15.22:0/3971913254 >> [v2:172.21.15.22:6826/3762222259,v1:172.21.15.22:6827/3762222259] conn(0x7ff1740ac360 0x7ff1740ae800 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.9 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:05:31.328 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:29.869+0000 7ff187400700 1 -- 172.21.15.22:0/3971913254 <== osd.9 v2:172.21.15.22:6826/3762222259 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7ff18000ae50 con 0x7ff1740ac360 2024-03-28T18:05:31.328 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:29.889+0000 7ff195c00700 1 -- 172.21.15.22:0/3971913254 --> [v2:172.21.15.22:6826/3762222259,v1:172.21.15.22:6827/3762222259] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7ff190109ee0 con 0x7ff1740ac360 2024-03-28T18:05:31.328 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:29.890+0000 7ff187400700 1 -- 172.21.15.22:0/3971913254 <== osd.9 v2:172.21.15.22:6826/3762222259 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7ff190109ee0 con 0x7ff1740ac360 2024-03-28T18:05:31.329 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:29.890+0000 7ff195c00700 1 -- 172.21.15.22:0/3971913254 >> [v2:172.21.15.22:6826/3762222259,v1:172.21.15.22:6827/3762222259] conn(0x7ff1740ac360 msgr2=0x7ff1740ae800 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:05:31.329 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:29.890+0000 7ff195c00700 1 --2- 172.21.15.22:0/3971913254 >> [v2:172.21.15.22:6826/3762222259,v1:172.21.15.22:6827/3762222259] conn(0x7ff1740ac360 0x7ff1740ae800 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:05:31.329 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:29.891+0000 7ff195c00700 1 -- 172.21.15.22:0/3971913254 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7ff1740a2920 msgr2=0x7ff1740a4de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:05:31.329 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:29.891+0000 7ff195c00700 1 --2- 172.21.15.22:0/3971913254 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7ff1740a2920 0x7ff1740a4de0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7ff17c005950 tx=0x7ff17c0058e0 comp rx=0 tx=0).stop 2024-03-28T18:05:31.329 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:29.891+0000 7ff195c00700 1 -- 172.21.15.22:0/3971913254 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff1901087a0 msgr2=0x7ff19019eea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:05:31.329 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:29.891+0000 7ff195c00700 1 --2- 172.21.15.22:0/3971913254 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff1901087a0 0x7ff19019eea0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7ff18000d950 tx=0x7ff18000dd10 comp rx=0 tx=0).stop 2024-03-28T18:05:31.329 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:29.891+0000 7ff18f400700 1 -- 172.21.15.22:0/3971913254 reap_dead start 2024-03-28T18:05:31.329 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:29.891+0000 7ff195c00700 1 -- 172.21.15.22:0/3971913254 shutdown_connections 2024-03-28T18:05:31.329 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:29.891+0000 7ff195c00700 1 -- 172.21.15.22:0/3971913254 >> 172.21.15.22:0/3971913254 conn(0x7ff190075960 msgr2=0x7ff190075de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:05:31.329 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:29.891+0000 7ff195c00700 1 -- 172.21.15.22:0/3971913254 shutdown_connections 2024-03-28T18:05:31.329 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:29.891+0000 7ff195c00700 1 -- 172.21.15.22:0/3971913254 wait complete. 2024-03-28T18:05:31.970 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:05:31 smithi022 ceph-mon[31985]: pgmap v392: 97 pgs: 97 active+clean; 642 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-03-28T18:05:32.174 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:05:31 smithi134 ceph-mon[35037]: pgmap v392: 97 pgs: 97 active+clean; 642 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-03-28T18:05:32.182 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:05:31 smithi079 ceph-mon[35312]: pgmap v392: 97 pgs: 97 active+clean; 642 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-03-28T18:05:32.213 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:32.143+0000 7f9220800700 1 -- 172.21.15.22:0/753088682 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f921810ea50 msgr2=0x7f9218110e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:05:32.213 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:32.143+0000 7f9220800700 1 --2- 172.21.15.22:0/753088682 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f921810ea50 0x7f9218110e60 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f921400b0a0 tx=0x7f921400b3b0 comp rx=0 tx=0).stop 2024-03-28T18:05:32.213 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:32.143+0000 7f9220800700 1 -- 172.21.15.22:0/753088682 shutdown_connections 2024-03-28T18:05:32.213 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:32.143+0000 7f9220800700 1 --2- 172.21.15.22:0/753088682 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f921810ea50 0x7f9218110e60 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:05:32.213 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:32.143+0000 7f9220800700 1 --2- 172.21.15.22:0/753088682 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9218074560 0x7f9218074940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:05:32.213 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:32.143+0000 7f9220800700 1 --2- 172.21.15.22:0/753088682 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f9218074e80 0x7f921810e510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:05:32.213 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:32.143+0000 7f9220800700 1 -- 172.21.15.22:0/753088682 >> 172.21.15.22:0/753088682 conn(0x7f92180fdc60 msgr2=0x7f9218100080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:05:32.214 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:32.143+0000 7f9220800700 1 -- 172.21.15.22:0/753088682 shutdown_connections 2024-03-28T18:05:32.214 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:32.143+0000 7f9220800700 1 -- 172.21.15.22:0/753088682 wait complete. 2024-03-28T18:05:32.214 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:32.144+0000 7f9220800700 1 Processor -- start 2024-03-28T18:05:32.214 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:32.145+0000 7f9220800700 1 -- start start 2024-03-28T18:05:32.214 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:32.145+0000 7f9220800700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9218074560 0x7f92181a0710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:05:32.214 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:32.145+0000 7f9220800700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9218074e80 0x7f92181a0c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:05:32.214 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:32.145+0000 7f9220800700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f921810ea50 0x7f921819a8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:05:32.214 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:32.145+0000 7f9220800700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92181146b0 con 0x7f9218074e80 2024-03-28T18:05:32.214 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:32.145+0000 7f9220800700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9218114530 con 0x7f921810ea50 2024-03-28T18:05:32.214 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:32.145+0000 7f9220800700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9218114830 con 0x7f9218074560 2024-03-28T18:05:32.214 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:32.145+0000 7f921d400700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9218074e80 0x7f92181a0c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:05:32.214 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:32.145+0000 7f921d400700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9218074e80 0x7f92181a0c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:50540/0 (socket says 172.21.15.22:50540) 2024-03-28T18:05:32.214 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:32.145+0000 7f921d400700 1 -- 172.21.15.22:0/818564366 learned_addr learned my addr 172.21.15.22:0/818564366 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-28T18:05:32.214 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:32.145+0000 7f921e800700 1 --2- 172.21.15.22:0/818564366 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f921810ea50 0x7f921819a8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:05:32.215 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:32.145+0000 7f921de00700 1 --2- 172.21.15.22:0/818564366 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9218074560 0x7f92181a0710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:05:32.215 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:32.146+0000 7f921d400700 1 -- 172.21.15.22:0/818564366 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9218074560 msgr2=0x7f92181a0710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:05:32.215 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:32.146+0000 7f921d400700 1 --2- 172.21.15.22:0/818564366 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9218074560 0x7f92181a0710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:05:32.215 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:32.146+0000 7f921d400700 1 -- 172.21.15.22:0/818564366 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f921810ea50 msgr2=0x7f921819a8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:05:32.215 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:32.146+0000 7f921d400700 1 --2- 172.21.15.22:0/818564366 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f921810ea50 0x7f921819a8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:05:32.215 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:32.146+0000 7f921d400700 1 -- 172.21.15.22:0/818564366 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f921819b030 con 0x7f9218074e80 2024-03-28T18:05:32.215 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:32.146+0000 7f921de00700 1 --2- 172.21.15.22:0/818564366 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9218074560 0x7f92181a0710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-28T18:05:32.215 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:32.146+0000 7f921d400700 1 --2- 172.21.15.22:0/818564366 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9218074e80 0x7f92181a0c50 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f920800d980 tx=0x7f920800dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:05:32.215 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:32.146+0000 7f9212a00700 1 -- 172.21.15.22:0/818564366 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9208016070 con 0x7f9218074e80 2024-03-28T18:05:32.215 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:32.147+0000 7f9212a00700 1 -- 172.21.15.22:0/818564366 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f9208004d00 con 0x7f9218074e80 2024-03-28T18:05:32.215 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:32.147+0000 7f9220800700 1 -- 172.21.15.22:0/818564366 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f921819b320 con 0x7f9218074e80 2024-03-28T18:05:32.215 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:32.147+0000 7f9212a00700 1 -- 172.21.15.22:0/818564366 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9208005550 con 0x7f9218074e80 2024-03-28T18:05:32.215 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:32.147+0000 7f9220800700 1 -- 172.21.15.22:0/818564366 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f92181a75e0 con 0x7f9218074e80 2024-03-28T18:05:32.215 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:32.148+0000 7f9220800700 1 -- 172.21.15.22:0/818564366 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f9218062920 con 0x7f9218074e80 2024-03-28T18:05:32.215 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:32.150+0000 7f9212a00700 1 -- 172.21.15.22:0/818564366 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 17) v1 ==== 137341+0+0 (secure 0 0 0) 0x7f9208004750 con 0x7f9218074e80 2024-03-28T18:05:32.215 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:32.151+0000 7f9212a00700 1 --2- 172.21.15.22:0/818564366 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7f92000a29a0 0x7f92000a4e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:05:32.216 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:32.151+0000 7f9212a00700 1 -- 172.21.15.22:0/818564366 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 8153+0+0 (secure 0 0 0) 0x7f92080cefd0 con 0x7f9218074e80 2024-03-28T18:05:32.216 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:32.151+0000 7f921de00700 1 --2- 172.21.15.22:0/818564366 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7f92000a29a0 0x7f92000a4e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:05:32.216 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:32.152+0000 7f9212a00700 1 --2- 172.21.15.22:0/818564366 >> [v2:172.21.15.22:6818/1400680396,v1:172.21.15.22:6819/1400680396] conn(0x7f92000ac3e0 0x7f92000ae880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:05:32.216 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:32.152+0000 7f9212a00700 1 -- 172.21.15.22:0/818564366 --> [v2:172.21.15.22:6818/1400680396,v1:172.21.15.22:6819/1400680396] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f9208015ec0 con 0x7f92000ac3e0 2024-03-28T18:05:32.216 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:32.152+0000 7f9212a00700 1 -- 172.21.15.22:0/818564366 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_get_version_reply(handle=1 version=92) v2 ==== 24+0+0 (secure 0 0 0) 0x7f92080cf380 con 0x7f9218074e80 2024-03-28T18:05:32.216 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:32.152+0000 7f921e800700 1 --2- 172.21.15.22:0/818564366 >> [v2:172.21.15.22:6818/1400680396,v1:172.21.15.22:6819/1400680396] conn(0x7f92000ac3e0 0x7f92000ae880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:05:32.216 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:32.152+0000 7f921de00700 1 --2- 172.21.15.22:0/818564366 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7f92000a29a0 0x7f92000a4e60 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f920c005fd0 tx=0x7f920c005f60 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:05:32.216 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:32.152+0000 7f921e800700 1 --2- 172.21.15.22:0/818564366 >> [v2:172.21.15.22:6818/1400680396,v1:172.21.15.22:6819/1400680396] conn(0x7f92000ac3e0 0x7f92000ae880 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:05:32.216 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:32.153+0000 7f9212a00700 1 -- 172.21.15.22:0/818564366 <== osd.6 v2:172.21.15.22:6818/1400680396 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7f9208015ec0 con 0x7f92000ac3e0 2024-03-28T18:05:32.216 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:32.172+0000 7f9220800700 1 -- 172.21.15.22:0/818564366 --> [v2:172.21.15.22:6818/1400680396,v1:172.21.15.22:6819/1400680396] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f9218062d20 con 0x7f92000ac3e0 2024-03-28T18:05:32.217 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:32.172+0000 7f9212a00700 1 -- 172.21.15.22:0/818564366 <== osd.6 v2:172.21.15.22:6818/1400680396 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7f9218062d20 con 0x7f92000ac3e0 2024-03-28T18:05:32.217 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:32.173+0000 7f9220800700 1 -- 172.21.15.22:0/818564366 >> [v2:172.21.15.22:6818/1400680396,v1:172.21.15.22:6819/1400680396] conn(0x7f92000ac3e0 msgr2=0x7f92000ae880 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:05:32.217 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:32.173+0000 7f9220800700 1 --2- 172.21.15.22:0/818564366 >> [v2:172.21.15.22:6818/1400680396,v1:172.21.15.22:6819/1400680396] conn(0x7f92000ac3e0 0x7f92000ae880 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:05:32.217 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:32.173+0000 7f9220800700 1 -- 172.21.15.22:0/818564366 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7f92000a29a0 msgr2=0x7f92000a4e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:05:32.217 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:32.173+0000 7f9220800700 1 --2- 172.21.15.22:0/818564366 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7f92000a29a0 0x7f92000a4e60 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f920c005fd0 tx=0x7f920c005f60 comp rx=0 tx=0).stop 2024-03-28T18:05:32.217 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:32.173+0000 7f9220800700 1 -- 172.21.15.22:0/818564366 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9218074e80 msgr2=0x7f92181a0c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:05:32.217 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:32.173+0000 7f9220800700 1 --2- 172.21.15.22:0/818564366 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9218074e80 0x7f92181a0c50 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f920800d980 tx=0x7f920800dc90 comp rx=0 tx=0).stop 2024-03-28T18:05:32.217 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:32.173+0000 7f921e800700 1 -- 172.21.15.22:0/818564366 reap_dead start 2024-03-28T18:05:32.217 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:32.174+0000 7f9220800700 1 -- 172.21.15.22:0/818564366 shutdown_connections 2024-03-28T18:05:32.217 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:32.174+0000 7f9220800700 1 -- 172.21.15.22:0/818564366 >> 172.21.15.22:0/818564366 conn(0x7f92180fdc60 msgr2=0x7f92180fe930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:05:32.217 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:32.174+0000 7f9220800700 1 -- 172.21.15.22:0/818564366 shutdown_connections 2024-03-28T18:05:32.217 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:32.174+0000 7f9220800700 1 -- 172.21.15.22:0/818564366 wait complete. 2024-03-28T18:05:33.572 INFO:teuthology.orchestra.run.smithi022.stdout:300647710758 2024-03-28T18:05:33.572 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033 shell --fsid fd220748-ed2b-11ee-95d0-87774f69a715 -- ceph osd last-stat-seq osd.9 2024-03-28T18:05:34.118 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:05:33 smithi022 ceph-mon[31985]: pgmap v393: 97 pgs: 97 active+clean; 642 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 10 op/s 2024-03-28T18:05:34.174 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:05:33 smithi134 ceph-mon[35037]: pgmap v393: 97 pgs: 97 active+clean; 642 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 10 op/s 2024-03-28T18:05:34.182 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:05:33 smithi079 ceph-mon[35312]: pgmap v393: 97 pgs: 97 active+clean; 642 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 10 op/s 2024-03-28T18:05:34.813 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:34.745+0000 7f8e35a00700 1 -- 172.21.15.22:0/3324445793 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8e30102dc0 msgr2=0x7f8e30106e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:05:34.814 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:34.745+0000 7f8e35a00700 1 --2- 172.21.15.22:0/3324445793 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8e30102dc0 0x7f8e30106e10 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f8e1c0099c0 tx=0x7f8e1c009cd0 comp rx=0 tx=0).stop 2024-03-28T18:05:34.814 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:34.745+0000 7f8e35a00700 1 -- 172.21.15.22:0/3324445793 shutdown_connections 2024-03-28T18:05:34.814 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:34.745+0000 7f8e35a00700 1 --2- 172.21.15.22:0/3324445793 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8e301074f0 0x7f8e3010f0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:05:34.814 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:34.745+0000 7f8e35a00700 1 --2- 172.21.15.22:0/3324445793 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f8e30102410 0x7f8e301027f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:05:34.814 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:34.745+0000 7f8e35a00700 1 --2- 172.21.15.22:0/3324445793 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8e30102dc0 0x7f8e30106e10 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:05:34.814 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:34.745+0000 7f8e35a00700 1 -- 172.21.15.22:0/3324445793 >> 172.21.15.22:0/3324445793 conn(0x7f8e300fdc60 msgr2=0x7f8e30100080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:05:34.814 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:34.745+0000 7f8e35a00700 1 -- 172.21.15.22:0/3324445793 shutdown_connections 2024-03-28T18:05:34.814 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:34.745+0000 7f8e35a00700 1 -- 172.21.15.22:0/3324445793 wait complete. 2024-03-28T18:05:34.814 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:34.746+0000 7f8e35a00700 1 Processor -- start 2024-03-28T18:05:34.814 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:34.747+0000 7f8e35a00700 1 -- start start 2024-03-28T18:05:34.814 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:34.747+0000 7f8e35a00700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8e30102410 0x7f8e301a07e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:05:34.815 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:34.747+0000 7f8e35a00700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8e30102dc0 0x7f8e301a0d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:05:34.815 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:34.747+0000 7f8e35a00700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f8e301074f0 0x7f8e3019a9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:05:34.815 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:34.747+0000 7f8e35a00700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e30112470 con 0x7f8e30102dc0 2024-03-28T18:05:34.815 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:34.747+0000 7f8e35a00700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e301122f0 con 0x7f8e301074f0 2024-03-28T18:05:34.815 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:34.747+0000 7f8e35a00700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e301125f0 con 0x7f8e30102410 2024-03-28T18:05:34.815 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:34.748+0000 7f8e2e000700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8e30102dc0 0x7f8e301a0d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:05:34.815 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:34.748+0000 7f8e2e000700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8e30102dc0 0x7f8e301a0d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:50548/0 (socket says 172.21.15.22:50548) 2024-03-28T18:05:34.815 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:34.748+0000 7f8e2e000700 1 -- 172.21.15.22:0/2589023268 learned_addr learned my addr 172.21.15.22:0/2589023268 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-28T18:05:34.815 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:34.748+0000 7f8e2ea00700 1 --2- 172.21.15.22:0/2589023268 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8e30102410 0x7f8e301a07e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:05:34.815 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:34.748+0000 7f8e2f400700 1 --2- 172.21.15.22:0/2589023268 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f8e301074f0 0x7f8e3019a9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:05:34.815 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:34.748+0000 7f8e2e000700 1 -- 172.21.15.22:0/2589023268 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8e30102410 msgr2=0x7f8e301a07e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:05:34.815 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:34.748+0000 7f8e2e000700 1 --2- 172.21.15.22:0/2589023268 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8e30102410 0x7f8e301a07e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:05:34.815 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:34.748+0000 7f8e2e000700 1 -- 172.21.15.22:0/2589023268 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f8e301074f0 msgr2=0x7f8e3019a9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:05:34.815 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:34.748+0000 7f8e2e000700 1 --2- 172.21.15.22:0/2589023268 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f8e301074f0 0x7f8e3019a9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:05:34.815 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:34.748+0000 7f8e2e000700 1 -- 172.21.15.22:0/2589023268 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8e3019b150 con 0x7f8e30102dc0 2024-03-28T18:05:34.816 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:34.748+0000 7f8e2f400700 1 --2- 172.21.15.22:0/2589023268 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f8e301074f0 0x7f8e3019a9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T18:05:34.816 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:34.748+0000 7f8e2e000700 1 --2- 172.21.15.22:0/2589023268 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8e30102dc0 0x7f8e301a0d20 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f8e1c0059d0 tx=0x7f8e1c005e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:05:34.816 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:34.749+0000 7f8e27400700 1 -- 172.21.15.22:0/2589023268 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e1c027070 con 0x7f8e30102dc0 2024-03-28T18:05:34.816 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:34.749+0000 7f8e27400700 1 -- 172.21.15.22:0/2589023268 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f8e1c0040b0 con 0x7f8e30102dc0 2024-03-28T18:05:34.816 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:34.749+0000 7f8e35a00700 1 -- 172.21.15.22:0/2589023268 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8e3019b3e0 con 0x7f8e30102dc0 2024-03-28T18:05:34.816 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:34.749+0000 7f8e27400700 1 -- 172.21.15.22:0/2589023268 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e1c022d40 con 0x7f8e30102dc0 2024-03-28T18:05:34.816 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:34.749+0000 7f8e35a00700 1 -- 172.21.15.22:0/2589023268 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8e301a76b0 con 0x7f8e30102dc0 2024-03-28T18:05:34.816 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:34.750+0000 7f8e35a00700 1 -- 172.21.15.22:0/2589023268 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f8e30075720 con 0x7f8e30102dc0 2024-03-28T18:05:34.816 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:34.752+0000 7f8e27400700 1 -- 172.21.15.22:0/2589023268 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 17) v1 ==== 137341+0+0 (secure 0 0 0) 0x7f8e1c02a070 con 0x7f8e30102dc0 2024-03-28T18:05:34.816 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:34.755+0000 7f8e27400700 1 --2- 172.21.15.22:0/2589023268 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7f8e140a29a0 0x7f8e140a4e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:05:34.816 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:34.755+0000 7f8e27400700 1 -- 172.21.15.22:0/2589023268 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 8153+0+0 (secure 0 0 0) 0x7f8e1c0cfb80 con 0x7f8e30102dc0 2024-03-28T18:05:34.816 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:34.755+0000 7f8e2ea00700 1 --2- 172.21.15.22:0/2589023268 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7f8e140a29a0 0x7f8e140a4e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:05:34.816 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:34.755+0000 7f8e27400700 1 --2- 172.21.15.22:0/2589023268 >> [v2:172.21.15.79:6800/2835549681,v1:172.21.15.79:6801/2835549681] conn(0x7f8e140ac3e0 0x7f8e140ae880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:05:34.816 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:34.756+0000 7f8e27400700 1 -- 172.21.15.22:0/2589023268 --> [v2:172.21.15.79:6800/2835549681,v1:172.21.15.79:6801/2835549681] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f8e1c02ee50 con 0x7f8e140ac3e0 2024-03-28T18:05:34.816 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:34.756+0000 7f8e27400700 1 -- 172.21.15.22:0/2589023268 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_get_version_reply(handle=1 version=92) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8e1c0d0060 con 0x7f8e30102dc0 2024-03-28T18:05:34.816 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:34.756+0000 7f8e2ea00700 1 --2- 172.21.15.22:0/2589023268 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7f8e140a29a0 0x7f8e140a4e60 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f8e20005950 tx=0x7f8e200058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:05:34.817 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:34.756+0000 7f8e2f400700 1 --2- 172.21.15.22:0/2589023268 >> [v2:172.21.15.79:6800/2835549681,v1:172.21.15.79:6801/2835549681] conn(0x7f8e140ac3e0 0x7f8e140ae880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:05:34.817 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:34.757+0000 7f8e2f400700 1 --2- 172.21.15.22:0/2589023268 >> [v2:172.21.15.79:6800/2835549681,v1:172.21.15.79:6801/2835549681] conn(0x7f8e140ac3e0 0x7f8e140ae880 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:05:34.817 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:34.758+0000 7f8e27400700 1 -- 172.21.15.22:0/2589023268 <== osd.1 v2:172.21.15.79:6800/2835549681 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7f8e1c02ee50 con 0x7f8e140ac3e0 2024-03-28T18:05:34.817 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:34.777+0000 7f8e35a00700 1 -- 172.21.15.22:0/2589023268 --> [v2:172.21.15.79:6800/2835549681,v1:172.21.15.79:6801/2835549681] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f8e301038f0 con 0x7f8e140ac3e0 2024-03-28T18:05:34.817 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:34.777+0000 7f8e27400700 1 -- 172.21.15.22:0/2589023268 <== osd.1 v2:172.21.15.79:6800/2835549681 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f8e301038f0 con 0x7f8e140ac3e0 2024-03-28T18:05:34.817 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:34.778+0000 7f8e35a00700 1 -- 172.21.15.22:0/2589023268 >> [v2:172.21.15.79:6800/2835549681,v1:172.21.15.79:6801/2835549681] conn(0x7f8e140ac3e0 msgr2=0x7f8e140ae880 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:05:34.817 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:34.778+0000 7f8e35a00700 1 --2- 172.21.15.22:0/2589023268 >> [v2:172.21.15.79:6800/2835549681,v1:172.21.15.79:6801/2835549681] conn(0x7f8e140ac3e0 0x7f8e140ae880 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:05:34.817 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:34.778+0000 7f8e35a00700 1 -- 172.21.15.22:0/2589023268 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7f8e140a29a0 msgr2=0x7f8e140a4e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:05:34.818 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:34.778+0000 7f8e35a00700 1 --2- 172.21.15.22:0/2589023268 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7f8e140a29a0 0x7f8e140a4e60 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f8e20005950 tx=0x7f8e200058e0 comp rx=0 tx=0).stop 2024-03-28T18:05:34.818 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:34.778+0000 7f8e35a00700 1 -- 172.21.15.22:0/2589023268 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8e30102dc0 msgr2=0x7f8e301a0d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:05:34.818 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:34.778+0000 7f8e35a00700 1 --2- 172.21.15.22:0/2589023268 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8e30102dc0 0x7f8e301a0d20 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f8e1c0059d0 tx=0x7f8e1c005e80 comp rx=0 tx=0).stop 2024-03-28T18:05:34.818 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:34.778+0000 7f8e2f400700 1 -- 172.21.15.22:0/2589023268 reap_dead start 2024-03-28T18:05:34.818 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:34.778+0000 7f8e35a00700 1 -- 172.21.15.22:0/2589023268 shutdown_connections 2024-03-28T18:05:34.818 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:34.778+0000 7f8e35a00700 1 -- 172.21.15.22:0/2589023268 >> 172.21.15.22:0/2589023268 conn(0x7f8e300fdc60 msgr2=0x7f8e300fff50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:05:34.818 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:34.778+0000 7f8e35a00700 1 -- 172.21.15.22:0/2589023268 shutdown_connections 2024-03-28T18:05:34.818 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:34.778+0000 7f8e35a00700 1 -- 172.21.15.22:0/2589023268 wait complete. 2024-03-28T18:05:36.083 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:05:35 smithi022 ceph-mon[31985]: pgmap v394: 97 pgs: 97 active+clean; 642 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 8 op/s 2024-03-28T18:05:36.174 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:05:35 smithi134 ceph-mon[35037]: pgmap v394: 97 pgs: 97 active+clean; 642 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 8 op/s 2024-03-28T18:05:36.182 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:05:35 smithi079 ceph-mon[35312]: pgmap v394: 97 pgs: 97 active+clean; 642 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 8 op/s 2024-03-28T18:05:36.669 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:36.668+0000 7f923ca00700 1 -- 172.21.15.22:0/1385935581 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f9238102df0 msgr2=0x7f9238106e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:05:36.669 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:36.668+0000 7f923ca00700 1 --2- 172.21.15.22:0/1385935581 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f9238102df0 0x7f9238106e30 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f92200098f0 tx=0x7f9220009c00 comp rx=0 tx=0).stop 2024-03-28T18:05:36.670 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:36.669+0000 7f923ca00700 1 -- 172.21.15.22:0/1385935581 shutdown_connections 2024-03-28T18:05:36.670 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:36.669+0000 7f923ca00700 1 --2- 172.21.15.22:0/1385935581 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9238107510 0x7f923810f0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:05:36.670 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:36.669+0000 7f923ca00700 1 --2- 172.21.15.22:0/1385935581 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9238102440 0x7f9238102820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:05:36.670 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:36.669+0000 7f923ca00700 1 --2- 172.21.15.22:0/1385935581 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f9238102df0 0x7f9238106e30 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:05:36.670 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:36.669+0000 7f923ca00700 1 -- 172.21.15.22:0/1385935581 >> 172.21.15.22:0/1385935581 conn(0x7f92380792f0 msgr2=0x7f9238079700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:05:36.670 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:36.669+0000 7f923ca00700 1 -- 172.21.15.22:0/1385935581 shutdown_connections 2024-03-28T18:05:36.670 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:36.669+0000 7f923ca00700 1 -- 172.21.15.22:0/1385935581 wait complete. 2024-03-28T18:05:36.670 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:36.671+0000 7f923ca00700 1 Processor -- start 2024-03-28T18:05:36.671 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:36.671+0000 7f923ca00700 1 -- start start 2024-03-28T18:05:36.671 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:36.671+0000 7f923ca00700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f9238102440 0x7f9238074490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:05:36.671 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:36.672+0000 7f923ca00700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9238102df0 0x7f92380749d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:05:36.671 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:36.672+0000 7f923ca00700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9238107510 0x7f9238074f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:05:36.671 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:36.672+0000 7f923ca00700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92381124e0 con 0x7f9238102df0 2024-03-28T18:05:36.672 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:36.672+0000 7f923ca00700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9238112360 con 0x7f9238102440 2024-03-28T18:05:36.672 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:36.672+0000 7f923ca00700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9238112660 con 0x7f9238107510 2024-03-28T18:05:36.672 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:36.672+0000 7f9235e00700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f9238102440 0x7f9238074490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:05:36.672 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:36.672+0000 7f922d400700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9238102df0 0x7f92380749d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:05:36.672 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:36.672+0000 7f9235e00700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f9238102440 0x7f9238074490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.22:56980/0 (socket says 172.21.15.22:56980) 2024-03-28T18:05:36.672 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:36.672+0000 7f9235e00700 1 -- 172.21.15.22:0/1528696817 learned_addr learned my addr 172.21.15.22:0/1528696817 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-28T18:05:36.673 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:36.672+0000 7f9235e00700 1 -- 172.21.15.22:0/1528696817 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9238107510 msgr2=0x7f9238074f10 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-03-28T18:05:36.673 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:36.672+0000 7f9235e00700 1 --2- 172.21.15.22:0/1528696817 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9238107510 0x7f9238074f10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:05:36.673 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:36.673+0000 7f9235e00700 1 -- 172.21.15.22:0/1528696817 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9238102df0 msgr2=0x7f92380749d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:05:36.673 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:36.673+0000 7f9235e00700 1 --2- 172.21.15.22:0/1528696817 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9238102df0 0x7f92380749d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:05:36.673 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:36.673+0000 7f9235e00700 1 -- 172.21.15.22:0/1528696817 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f92381a7000 con 0x7f9238102440 2024-03-28T18:05:36.673 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:36.673+0000 7f9235e00700 1 --2- 172.21.15.22:0/1528696817 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f9238102440 0x7f9238074490 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f9228007f10 tx=0x7f922800d3b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:05:36.673 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:36.673+0000 7f922f400700 1 -- 172.21.15.22:0/1528696817 <== mon.2 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f922800dbd0 con 0x7f9238102440 2024-03-28T18:05:36.674 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:36.674+0000 7f922f400700 1 -- 172.21.15.22:0/1528696817 <== mon.2 v2:172.21.15.79:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f922800dd70 con 0x7f9238102440 2024-03-28T18:05:36.674 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:36.674+0000 7f922f400700 1 -- 172.21.15.22:0/1528696817 <== mon.2 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92280133f0 con 0x7f9238102440 2024-03-28T18:05:36.674 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:36.674+0000 7f923ca00700 1 -- 172.21.15.22:0/1528696817 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f92381a72f0 con 0x7f9238102440 2024-03-28T18:05:36.675 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:36.675+0000 7f923ca00700 1 -- 172.21.15.22:0/1528696817 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f92381a77b0 con 0x7f9238102440 2024-03-28T18:05:36.676 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:36.676+0000 7f923ca00700 1 -- 172.21.15.22:0/1528696817 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f92380628a0 con 0x7f9238102440 2024-03-28T18:05:36.678 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:36.678+0000 7f922f400700 1 -- 172.21.15.22:0/1528696817 <== mon.2 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 17) v1 ==== 137341+0+0 (secure 0 0 0) 0x7f922801a020 con 0x7f9238102440 2024-03-28T18:05:36.679 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:36.680+0000 7f922f400700 1 --2- 172.21.15.22:0/1528696817 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7f921c0a29a0 0x7f921c0a4e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:05:36.680 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:36.680+0000 7f922f400700 1 -- 172.21.15.22:0/1528696817 <== mon.2 v2:172.21.15.79:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 8153+0+0 (secure 0 0 0) 0x7f92280ce8d0 con 0x7f9238102440 2024-03-28T18:05:36.680 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:36.680+0000 7f922d400700 1 --2- 172.21.15.22:0/1528696817 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7f921c0a29a0 0x7f921c0a4e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:05:36.680 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:36.680+0000 7f922f400700 1 --2- 172.21.15.22:0/1528696817 >> [v2:172.21.15.79:6808/630132437,v1:172.21.15.79:6809/630132437] conn(0x7f921c0ac3e0 0x7f921c0ae880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:05:36.680 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:36.680+0000 7f922f400700 1 -- 172.21.15.22:0/1528696817 --> [v2:172.21.15.79:6808/630132437,v1:172.21.15.79:6809/630132437] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f9228014e50 con 0x7f921c0ac3e0 2024-03-28T18:05:36.680 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:36.680+0000 7f922f400700 1 -- 172.21.15.22:0/1528696817 <== mon.2 v2:172.21.15.79:3300/0 6 ==== mon_get_version_reply(handle=1 version=92) v2 ==== 24+0+0 (secure 0 0 0) 0x7f92280cec80 con 0x7f9238102440 2024-03-28T18:05:36.681 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:36.681+0000 7f9236800700 1 --2- 172.21.15.22:0/1528696817 >> [v2:172.21.15.79:6808/630132437,v1:172.21.15.79:6809/630132437] conn(0x7f921c0ac3e0 0x7f921c0ae880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:05:36.681 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:36.681+0000 7f922d400700 1 --2- 172.21.15.22:0/1528696817 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7f921c0a29a0 0x7f921c0a4e60 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f92200098c0 tx=0x7f9220019040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:05:36.681 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:36.682+0000 7f9236800700 1 --2- 172.21.15.22:0/1528696817 >> [v2:172.21.15.79:6808/630132437,v1:172.21.15.79:6809/630132437] conn(0x7f921c0ac3e0 0x7f921c0ae880 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:05:36.683 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:36.683+0000 7f922f400700 1 -- 172.21.15.22:0/1528696817 <== osd.4 v2:172.21.15.79:6808/630132437 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7f9228014e50 con 0x7f921c0ac3e0 2024-03-28T18:05:36.702 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:36.702+0000 7f923ca00700 1 -- 172.21.15.22:0/1528696817 --> [v2:172.21.15.79:6808/630132437,v1:172.21.15.79:6809/630132437] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f9238062ca0 con 0x7f921c0ac3e0 2024-03-28T18:05:36.702 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:36.703+0000 7f922f400700 1 -- 172.21.15.22:0/1528696817 <== osd.4 v2:172.21.15.79:6808/630132437 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7f9238062ca0 con 0x7f921c0ac3e0 2024-03-28T18:05:36.703 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:36.703+0000 7f923ca00700 1 -- 172.21.15.22:0/1528696817 >> [v2:172.21.15.79:6808/630132437,v1:172.21.15.79:6809/630132437] conn(0x7f921c0ac3e0 msgr2=0x7f921c0ae880 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:05:36.703 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:36.703+0000 7f923ca00700 1 --2- 172.21.15.22:0/1528696817 >> [v2:172.21.15.79:6808/630132437,v1:172.21.15.79:6809/630132437] conn(0x7f921c0ac3e0 0x7f921c0ae880 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:05:36.703 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:36.703+0000 7f923ca00700 1 -- 172.21.15.22:0/1528696817 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7f921c0a29a0 msgr2=0x7f921c0a4e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:05:36.703 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:36.703+0000 7f923ca00700 1 --2- 172.21.15.22:0/1528696817 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7f921c0a29a0 0x7f921c0a4e60 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f92200098c0 tx=0x7f9220019040 comp rx=0 tx=0).stop 2024-03-28T18:05:36.704 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:36.703+0000 7f923ca00700 1 -- 172.21.15.22:0/1528696817 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f9238102440 msgr2=0x7f9238074490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:05:36.704 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:36.703+0000 7f923ca00700 1 --2- 172.21.15.22:0/1528696817 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f9238102440 0x7f9238074490 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f9228007f10 tx=0x7f922800d3b0 comp rx=0 tx=0).stop 2024-03-28T18:05:36.704 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:36.703+0000 7f9236800700 1 -- 172.21.15.22:0/1528696817 reap_dead start 2024-03-28T18:05:36.704 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:36.703+0000 7f923ca00700 1 -- 172.21.15.22:0/1528696817 shutdown_connections 2024-03-28T18:05:36.704 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:36.703+0000 7f923ca00700 1 -- 172.21.15.22:0/1528696817 >> 172.21.15.22:0/1528696817 conn(0x7f92380792f0 msgr2=0x7f9238100530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:05:36.704 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:36.704+0000 7f923ca00700 1 -- 172.21.15.22:0/1528696817 shutdown_connections 2024-03-28T18:05:36.704 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:36.704+0000 7f923ca00700 1 -- 172.21.15.22:0/1528696817 wait complete. 2024-03-28T18:05:36.795 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:36.768+0000 7f64cda00700 1 -- 172.21.15.22:0/4273601643 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f64c8104600 msgr2=0x7f64c81049e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:05:36.795 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:36.768+0000 7f64cda00700 1 --2- 172.21.15.22:0/4273601643 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f64c8104600 0x7f64c81049e0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f64b8009990 tx=0x7f64b8009ca0 comp rx=0 tx=0).stop 2024-03-28T18:05:36.796 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:36.768+0000 7f64cda00700 1 -- 172.21.15.22:0/4273601643 shutdown_connections 2024-03-28T18:05:36.796 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:36.768+0000 7f64cda00700 1 --2- 172.21.15.22:0/4273601643 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f64c81096e0 0x7f64c81112b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:05:36.796 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:36.768+0000 7f64cda00700 1 --2- 172.21.15.22:0/4273601643 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f64c8104600 0x7f64c81049e0 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:05:36.796 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:36.768+0000 7f64cda00700 1 --2- 172.21.15.22:0/4273601643 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f64c8104fb0 0x7f64c8109000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:05:36.796 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:36.768+0000 7f64cda00700 1 -- 172.21.15.22:0/4273601643 >> 172.21.15.22:0/4273601643 conn(0x7f64c80ffe90 msgr2=0x7f64c81022b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:05:36.796 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:36.769+0000 7f64cda00700 1 -- 172.21.15.22:0/4273601643 shutdown_connections 2024-03-28T18:05:36.796 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:36.769+0000 7f64cda00700 1 -- 172.21.15.22:0/4273601643 wait complete. 2024-03-28T18:05:36.796 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:36.770+0000 7f64cda00700 1 Processor -- start 2024-03-28T18:05:36.796 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:36.770+0000 7f64cda00700 1 -- start start 2024-03-28T18:05:36.796 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:36.771+0000 7f64cda00700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f64c8104600 0x7f64c80738d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:05:36.796 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:36.771+0000 7f64cda00700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f64c8104fb0 0x7f64c8073e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:05:36.796 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:36.771+0000 7f64cda00700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f64c81096e0 0x7f64c806cbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:05:36.796 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:36.771+0000 7f64cda00700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64c81146b0 con 0x7f64c8104600 2024-03-28T18:05:36.796 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:36.771+0000 7f64cda00700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64c8114530 con 0x7f64c8104fb0 2024-03-28T18:05:36.797 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:36.771+0000 7f64cda00700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64c8114830 con 0x7f64c81096e0 2024-03-28T18:05:36.797 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:36.771+0000 7f64c6a00700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f64c8104600 0x7f64c80738d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:05:36.797 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:36.771+0000 7f64c6a00700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f64c8104600 0x7f64c80738d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:50578/0 (socket says 172.21.15.22:50578) 2024-03-28T18:05:36.797 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:36.771+0000 7f64c6a00700 1 -- 172.21.15.22:0/1165902669 learned_addr learned my addr 172.21.15.22:0/1165902669 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-28T18:05:36.797 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:36.771+0000 7f64c6000700 1 --2- 172.21.15.22:0/1165902669 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f64c8104fb0 0x7f64c8073e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:05:36.797 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:36.771+0000 7f64c7400700 1 --2- 172.21.15.22:0/1165902669 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f64c81096e0 0x7f64c806cbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:05:36.797 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:36.772+0000 7f64c6a00700 1 -- 172.21.15.22:0/1165902669 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f64c81096e0 msgr2=0x7f64c806cbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:05:36.797 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:36.772+0000 7f64c6a00700 1 --2- 172.21.15.22:0/1165902669 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f64c81096e0 0x7f64c806cbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:05:36.797 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:36.772+0000 7f64c6a00700 1 -- 172.21.15.22:0/1165902669 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f64c8104fb0 msgr2=0x7f64c8073e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:05:36.797 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:36.772+0000 7f64c6a00700 1 --2- 172.21.15.22:0/1165902669 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f64c8104fb0 0x7f64c8073e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:05:36.797 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:36.772+0000 7f64c6a00700 1 -- 172.21.15.22:0/1165902669 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f64c806d4b0 con 0x7f64c8104600 2024-03-28T18:05:36.797 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:36.772+0000 7f64c6a00700 1 --2- 172.21.15.22:0/1165902669 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f64c8104600 0x7f64c80738d0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f64b8009590 tx=0x7f64b8005590 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:05:36.797 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:36.773+0000 7f64bf400700 1 -- 172.21.15.22:0/1165902669 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64b8026070 con 0x7f64c8104600 2024-03-28T18:05:36.797 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:36.773+0000 7f64bf400700 1 -- 172.21.15.22:0/1165902669 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f64b80056b0 con 0x7f64c8104600 2024-03-28T18:05:36.798 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:36.773+0000 7f64bf400700 1 -- 172.21.15.22:0/1165902669 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64b80173f0 con 0x7f64c8104600 2024-03-28T18:05:36.798 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:36.773+0000 7f64cda00700 1 -- 172.21.15.22:0/1165902669 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f64c806d740 con 0x7f64c8104600 2024-03-28T18:05:36.798 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:36.773+0000 7f64cda00700 1 -- 172.21.15.22:0/1165902669 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f64c80715d0 con 0x7f64c8104600 2024-03-28T18:05:36.798 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:36.774+0000 7f64cda00700 1 -- 172.21.15.22:0/1165902669 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f64c8071bc0 con 0x7f64c8104600 2024-03-28T18:05:36.798 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:36.777+0000 7f64bf400700 1 -- 172.21.15.22:0/1165902669 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 17) v1 ==== 137341+0+0 (secure 0 0 0) 0x7f64b8029070 con 0x7f64c8104600 2024-03-28T18:05:36.798 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:36.779+0000 7f64bf400700 1 --2- 172.21.15.22:0/1165902669 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7f64ac0a29a0 0x7f64ac0a4e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:05:36.798 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:36.779+0000 7f64bf400700 1 -- 172.21.15.22:0/1165902669 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 8153+0+0 (secure 0 0 0) 0x7f64b80cf6d0 con 0x7f64c8104600 2024-03-28T18:05:36.798 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:36.779+0000 7f64c6000700 1 --2- 172.21.15.22:0/1165902669 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7f64ac0a29a0 0x7f64ac0a4e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:05:36.798 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:36.779+0000 7f64bf400700 1 --2- 172.21.15.22:0/1165902669 >> [v2:172.21.15.134:6808/2135406915,v1:172.21.15.134:6809/2135406915] conn(0x7f64ac0ac3e0 0x7f64ac0ae880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:05:36.798 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:36.779+0000 7f64bf400700 1 -- 172.21.15.22:0/1165902669 --> [v2:172.21.15.134:6808/2135406915,v1:172.21.15.134:6809/2135406915] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f64b800fe50 con 0x7f64ac0ac3e0 2024-03-28T18:05:36.798 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:36.779+0000 7f64bf400700 1 -- 172.21.15.22:0/1165902669 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_get_version_reply(handle=1 version=92) v2 ==== 24+0+0 (secure 0 0 0) 0x7f64b80cfa80 con 0x7f64c8104600 2024-03-28T18:05:36.798 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:36.780+0000 7f64c7400700 1 --2- 172.21.15.22:0/1165902669 >> [v2:172.21.15.134:6808/2135406915,v1:172.21.15.134:6809/2135406915] conn(0x7f64ac0ac3e0 0x7f64ac0ae880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:05:36.798 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:36.780+0000 7f64c6000700 1 --2- 172.21.15.22:0/1165902669 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7f64ac0a29a0 0x7f64ac0a4e60 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f64c806e120 tx=0x7f64b4009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:05:36.798 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:36.781+0000 7f64c7400700 1 --2- 172.21.15.22:0/1165902669 >> [v2:172.21.15.134:6808/2135406915,v1:172.21.15.134:6809/2135406915] conn(0x7f64ac0ac3e0 0x7f64ac0ae880 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:05:36.799 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:36.783+0000 7f64bf400700 1 -- 172.21.15.22:0/1165902669 <== osd.5 v2:172.21.15.134:6808/2135406915 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7f64b800fe50 con 0x7f64ac0ac3e0 2024-03-28T18:05:36.801 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:36.802+0000 7f64cda00700 1 -- 172.21.15.22:0/1165902669 --> [v2:172.21.15.134:6808/2135406915,v1:172.21.15.134:6809/2135406915] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f64c8072090 con 0x7f64ac0ac3e0 2024-03-28T18:05:36.802 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:36.803+0000 7f64bf400700 1 -- 172.21.15.22:0/1165902669 <== osd.5 v2:172.21.15.134:6808/2135406915 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7f64c8072090 con 0x7f64ac0ac3e0 2024-03-28T18:05:36.803 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:36.803+0000 7f64cda00700 1 -- 172.21.15.22:0/1165902669 >> [v2:172.21.15.134:6808/2135406915,v1:172.21.15.134:6809/2135406915] conn(0x7f64ac0ac3e0 msgr2=0x7f64ac0ae880 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:05:36.803 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:36.803+0000 7f64cda00700 1 --2- 172.21.15.22:0/1165902669 >> [v2:172.21.15.134:6808/2135406915,v1:172.21.15.134:6809/2135406915] conn(0x7f64ac0ac3e0 0x7f64ac0ae880 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:05:36.804 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:36.803+0000 7f64cda00700 1 -- 172.21.15.22:0/1165902669 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7f64ac0a29a0 msgr2=0x7f64ac0a4e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:05:36.804 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:36.803+0000 7f64cda00700 1 --2- 172.21.15.22:0/1165902669 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7f64ac0a29a0 0x7f64ac0a4e60 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f64c806e120 tx=0x7f64b4009260 comp rx=0 tx=0).stop 2024-03-28T18:05:36.804 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:36.803+0000 7f64cda00700 1 -- 172.21.15.22:0/1165902669 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f64c8104600 msgr2=0x7f64c80738d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:05:36.804 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:36.803+0000 7f64cda00700 1 --2- 172.21.15.22:0/1165902669 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f64c8104600 0x7f64c80738d0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f64b8009590 tx=0x7f64b8005590 comp rx=0 tx=0).stop 2024-03-28T18:05:36.804 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:36.803+0000 7f64c7400700 1 -- 172.21.15.22:0/1165902669 reap_dead start 2024-03-28T18:05:36.804 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:36.804+0000 7f64cda00700 1 -- 172.21.15.22:0/1165902669 shutdown_connections 2024-03-28T18:05:36.804 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:36.804+0000 7f64cda00700 1 -- 172.21.15.22:0/1165902669 >> 172.21.15.22:0/1165902669 conn(0x7f64c80ffe90 msgr2=0x7f64c8108870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:05:36.804 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:36.804+0000 7f64cda00700 1 -- 172.21.15.22:0/1165902669 shutdown_connections 2024-03-28T18:05:36.804 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:36.804+0000 7f64cda00700 1 -- 172.21.15.22:0/1165902669 wait complete. 2024-03-28T18:05:37.363 INFO:teuthology.orchestra.run.smithi022.stdout:206158430266 2024-03-28T18:05:37.364 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033 shell --fsid fd220748-ed2b-11ee-95d0-87774f69a715 -- ceph osd last-stat-seq osd.6 2024-03-28T18:05:37.923 INFO:teuthology.orchestra.run.smithi022.stdout:55834574943 2024-03-28T18:05:37.923 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033 shell --fsid fd220748-ed2b-11ee-95d0-87774f69a715 -- ceph osd last-stat-seq osd.1 2024-03-28T18:05:38.031 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:05:37 smithi022 ceph-mon[31985]: pgmap v395: 97 pgs: 97 active+clean; 642 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2024-03-28T18:05:38.174 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:05:37 smithi134 ceph-mon[35037]: pgmap v395: 97 pgs: 97 active+clean; 642 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2024-03-28T18:05:38.182 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:05:37 smithi079 ceph-mon[35312]: pgmap v395: 97 pgs: 97 active+clean; 642 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2024-03-28T18:05:39.299 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:38.939+0000 7fd1c0000700 1 -- 172.21.15.22:0/28775263 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd1b8104f90 msgr2=0x7fd1b8108fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:05:39.300 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:38.939+0000 7fd1c0000700 1 --2- 172.21.15.22:0/28775263 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd1b8104f90 0x7fd1b8108fe0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fd1a40099c0 tx=0x7fd1a4009cd0 comp rx=0 tx=0).stop 2024-03-28T18:05:39.300 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:38.939+0000 7fd1c0000700 1 -- 172.21.15.22:0/28775263 shutdown_connections 2024-03-28T18:05:39.300 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:38.939+0000 7fd1c0000700 1 --2- 172.21.15.22:0/28775263 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd1b81096c0 0x7fd1b8111290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:05:39.300 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:38.939+0000 7fd1c0000700 1 --2- 172.21.15.22:0/28775263 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd1b81045e0 0x7fd1b81049c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:05:39.300 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:38.939+0000 7fd1c0000700 1 --2- 172.21.15.22:0/28775263 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd1b8104f90 0x7fd1b8108fe0 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:05:39.300 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:38.939+0000 7fd1c0000700 1 -- 172.21.15.22:0/28775263 >> 172.21.15.22:0/28775263 conn(0x7fd1b80ffe70 msgr2=0x7fd1b8102290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:05:39.300 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:38.939+0000 7fd1c0000700 1 -- 172.21.15.22:0/28775263 shutdown_connections 2024-03-28T18:05:39.300 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:38.939+0000 7fd1c0000700 1 -- 172.21.15.22:0/28775263 wait complete. 2024-03-28T18:05:39.300 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:38.940+0000 7fd1c0000700 1 Processor -- start 2024-03-28T18:05:39.300 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:38.940+0000 7fd1c0000700 1 -- start start 2024-03-28T18:05:39.300 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:38.941+0000 7fd1c0000700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd1b81045e0 0x7fd1b81a0770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:05:39.300 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:38.941+0000 7fd1c0000700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd1b8104f90 0x7fd1b81a0cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:05:39.301 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:38.941+0000 7fd1c0000700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd1b81096c0 0x7fd1b819a930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:05:39.301 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:38.941+0000 7fd1c0000700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1b8114690 con 0x7fd1b81045e0 2024-03-28T18:05:39.301 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:38.941+0000 7fd1c0000700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1b8114510 con 0x7fd1b81096c0 2024-03-28T18:05:39.301 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:38.941+0000 7fd1c0000700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1b8114810 con 0x7fd1b8104f90 2024-03-28T18:05:39.301 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:38.941+0000 7fd1bd600700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd1b81045e0 0x7fd1b81a0770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:05:39.301 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:38.941+0000 7fd1bd600700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd1b81045e0 0x7fd1b81a0770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:51200/0 (socket says 172.21.15.22:51200) 2024-03-28T18:05:39.301 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:38.941+0000 7fd1bd600700 1 -- 172.21.15.22:0/674972573 learned_addr learned my addr 172.21.15.22:0/674972573 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-28T18:05:39.301 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:38.941+0000 7fd1be000700 1 --2- 172.21.15.22:0/674972573 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd1b81096c0 0x7fd1b819a930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:05:39.301 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:38.941+0000 7fd1bcc00700 1 --2- 172.21.15.22:0/674972573 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd1b8104f90 0x7fd1b81a0cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:05:39.301 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:38.942+0000 7fd1bd600700 1 -- 172.21.15.22:0/674972573 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd1b8104f90 msgr2=0x7fd1b81a0cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:05:39.301 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:38.942+0000 7fd1bd600700 1 --2- 172.21.15.22:0/674972573 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd1b8104f90 0x7fd1b81a0cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:05:39.301 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:38.942+0000 7fd1bd600700 1 -- 172.21.15.22:0/674972573 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd1b81096c0 msgr2=0x7fd1b819a930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:05:39.301 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:38.942+0000 7fd1bd600700 1 --2- 172.21.15.22:0/674972573 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd1b81096c0 0x7fd1b819a930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:05:39.301 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:38.942+0000 7fd1bd600700 1 -- 172.21.15.22:0/674972573 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd1b819b090 con 0x7fd1b81045e0 2024-03-28T18:05:39.302 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:38.942+0000 7fd1bd600700 1 --2- 172.21.15.22:0/674972573 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd1b81045e0 0x7fd1b81a0770 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fd1ac00d980 tx=0x7fd1ac00dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:05:39.302 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:38.943+0000 7fd1b6000700 1 -- 172.21.15.22:0/674972573 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1ac017070 con 0x7fd1b81045e0 2024-03-28T18:05:39.302 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:38.943+0000 7fd1c0000700 1 -- 172.21.15.22:0/674972573 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd1b819b380 con 0x7fd1b81045e0 2024-03-28T18:05:39.302 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:38.943+0000 7fd1b6000700 1 -- 172.21.15.22:0/674972573 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7fd1ac005020 con 0x7fd1b81045e0 2024-03-28T18:05:39.302 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:38.943+0000 7fd1b6000700 1 -- 172.21.15.22:0/674972573 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1ac00fa60 con 0x7fd1b81045e0 2024-03-28T18:05:39.302 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:38.943+0000 7fd1c0000700 1 -- 172.21.15.22:0/674972573 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd1b81a7640 con 0x7fd1b81045e0 2024-03-28T18:05:39.302 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:38.944+0000 7fd1c0000700 1 -- 172.21.15.22:0/674972573 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fd1b8062920 con 0x7fd1b81045e0 2024-03-28T18:05:39.302 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:38.946+0000 7fd1b6000700 1 -- 172.21.15.22:0/674972573 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 17) v1 ==== 137341+0+0 (secure 0 0 0) 0x7fd1ac0051d0 con 0x7fd1b81045e0 2024-03-28T18:05:39.302 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:38.947+0000 7fd1b6000700 1 --2- 172.21.15.22:0/674972573 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7fd1a00a2920 0x7fd1a00a4de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:05:39.302 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:38.948+0000 7fd1b6000700 1 -- 172.21.15.22:0/674972573 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 8153+0+0 (secure 0 0 0) 0x7fd1ac0cef90 con 0x7fd1b81045e0 2024-03-28T18:05:39.302 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:38.948+0000 7fd1bcc00700 1 --2- 172.21.15.22:0/674972573 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7fd1a00a2920 0x7fd1a00a4de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:05:39.302 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:38.948+0000 7fd1b6000700 1 --2- 172.21.15.22:0/674972573 >> [v2:172.21.15.22:6802/2186290846,v1:172.21.15.22:6803/2186290846] conn(0x7fd1a00ac360 0x7fd1a00ae800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:05:39.303 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:38.948+0000 7fd1b6000700 1 -- 172.21.15.22:0/674972573 --> [v2:172.21.15.22:6802/2186290846,v1:172.21.15.22:6803/2186290846] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fd1a00aeed0 con 0x7fd1a00ac360 2024-03-28T18:05:39.303 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:38.948+0000 7fd1b6000700 1 -- 172.21.15.22:0/674972573 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_get_version_reply(handle=1 version=92) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd1ac0cf340 con 0x7fd1b81045e0 2024-03-28T18:05:39.303 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:38.948+0000 7fd1be000700 1 --2- 172.21.15.22:0/674972573 >> [v2:172.21.15.22:6802/2186290846,v1:172.21.15.22:6803/2186290846] conn(0x7fd1a00ac360 0x7fd1a00ae800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:05:39.303 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:38.948+0000 7fd1bcc00700 1 --2- 172.21.15.22:0/674972573 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7fd1a00a2920 0x7fd1a00a4de0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fd1a4000c00 tx=0x7fd1a4005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:05:39.303 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:38.949+0000 7fd1be000700 1 --2- 172.21.15.22:0/674972573 >> [v2:172.21.15.22:6802/2186290846,v1:172.21.15.22:6803/2186290846] conn(0x7fd1a00ac360 0x7fd1a00ae800 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:05:39.303 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:38.950+0000 7fd1b6000700 1 -- 172.21.15.22:0/674972573 <== osd.0 v2:172.21.15.22:6802/2186290846 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7fd1a00aeed0 con 0x7fd1a00ac360 2024-03-28T18:05:39.303 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:38.969+0000 7fd1c0000700 1 -- 172.21.15.22:0/674972573 --> [v2:172.21.15.22:6802/2186290846,v1:172.21.15.22:6803/2186290846] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fd1b8062da0 con 0x7fd1a00ac360 2024-03-28T18:05:39.303 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:38.970+0000 7fd1b6000700 1 -- 172.21.15.22:0/674972573 <== osd.0 v2:172.21.15.22:6802/2186290846 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7fd1b8062da0 con 0x7fd1a00ac360 2024-03-28T18:05:39.304 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:38.971+0000 7fd1c0000700 1 -- 172.21.15.22:0/674972573 >> [v2:172.21.15.22:6802/2186290846,v1:172.21.15.22:6803/2186290846] conn(0x7fd1a00ac360 msgr2=0x7fd1a00ae800 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:05:39.304 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:38.971+0000 7fd1c0000700 1 --2- 172.21.15.22:0/674972573 >> [v2:172.21.15.22:6802/2186290846,v1:172.21.15.22:6803/2186290846] conn(0x7fd1a00ac360 0x7fd1a00ae800 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:05:39.304 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:38.971+0000 7fd1c0000700 1 -- 172.21.15.22:0/674972573 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7fd1a00a2920 msgr2=0x7fd1a00a4de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:05:39.304 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:38.971+0000 7fd1c0000700 1 --2- 172.21.15.22:0/674972573 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7fd1a00a2920 0x7fd1a00a4de0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fd1a4000c00 tx=0x7fd1a4005fb0 comp rx=0 tx=0).stop 2024-03-28T18:05:39.304 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:38.971+0000 7fd1c0000700 1 -- 172.21.15.22:0/674972573 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd1b81045e0 msgr2=0x7fd1b81a0770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:05:39.304 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:38.971+0000 7fd1c0000700 1 --2- 172.21.15.22:0/674972573 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd1b81045e0 0x7fd1b81a0770 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fd1ac00d980 tx=0x7fd1ac00dc90 comp rx=0 tx=0).stop 2024-03-28T18:05:39.304 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:38.971+0000 7fd1be000700 1 -- 172.21.15.22:0/674972573 reap_dead start 2024-03-28T18:05:39.304 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:38.971+0000 7fd1c0000700 1 -- 172.21.15.22:0/674972573 shutdown_connections 2024-03-28T18:05:39.304 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:38.971+0000 7fd1c0000700 1 -- 172.21.15.22:0/674972573 >> 172.21.15.22:0/674972573 conn(0x7fd1b80ffe70 msgr2=0x7fd1b8100970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:05:39.304 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:38.971+0000 7fd1c0000700 1 -- 172.21.15.22:0/674972573 shutdown_connections 2024-03-28T18:05:39.304 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:38.971+0000 7fd1c0000700 1 -- 172.21.15.22:0/674972573 wait complete. 2024-03-28T18:05:39.879 INFO:teuthology.orchestra.run.smithi022.stdout:227633266739 2024-03-28T18:05:39.879 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033 shell --fsid fd220748-ed2b-11ee-95d0-87774f69a715 -- ceph osd last-stat-seq osd.7 2024-03-28T18:05:40.031 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:05:39 smithi022 ceph-mon[31985]: pgmap v396: 97 pgs: 97 active+clean; 642 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-28T18:05:40.103 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:40.092+0000 7f36f2200700 1 -- 172.21.15.22:0/2617653591 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f36ec10ea50 msgr2=0x7f36ec110e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:05:40.104 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:40.092+0000 7f36f2200700 1 --2- 172.21.15.22:0/2617653591 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f36ec10ea50 0x7f36ec110e60 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f36e400b0a0 tx=0x7f36e400b3b0 comp rx=0 tx=0).stop 2024-03-28T18:05:40.104 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:40.092+0000 7f36f2200700 1 -- 172.21.15.22:0/2617653591 shutdown_connections 2024-03-28T18:05:40.104 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:40.092+0000 7f36f2200700 1 --2- 172.21.15.22:0/2617653591 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f36ec10ea50 0x7f36ec110e60 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:05:40.104 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:40.092+0000 7f36f2200700 1 --2- 172.21.15.22:0/2617653591 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f36ec074560 0x7f36ec074940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:05:40.104 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:40.092+0000 7f36f2200700 1 --2- 172.21.15.22:0/2617653591 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f36ec074e80 0x7f36ec10e510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:05:40.104 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:40.092+0000 7f36f2200700 1 -- 172.21.15.22:0/2617653591 >> 172.21.15.22:0/2617653591 conn(0x7f36ec0fdc60 msgr2=0x7f36ec100080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:05:40.104 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:40.092+0000 7f36f2200700 1 -- 172.21.15.22:0/2617653591 shutdown_connections 2024-03-28T18:05:40.104 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:40.092+0000 7f36f2200700 1 -- 172.21.15.22:0/2617653591 wait complete. 2024-03-28T18:05:40.104 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:40.093+0000 7f36f2200700 1 Processor -- start 2024-03-28T18:05:40.104 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:40.093+0000 7f36f2200700 1 -- start start 2024-03-28T18:05:40.104 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:40.094+0000 7f36f2200700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f36ec074560 0x7f36ec1a07e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:05:40.104 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:40.094+0000 7f36f2200700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f36ec074e80 0x7f36ec1a0d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:05:40.105 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:40.094+0000 7f36f2200700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f36ec10ea50 0x7f36ec19a9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:05:40.105 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:40.094+0000 7f36f2200700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36ec1146d0 con 0x7f36ec10ea50 2024-03-28T18:05:40.105 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:40.094+0000 7f36f2200700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36ec114550 con 0x7f36ec074560 2024-03-28T18:05:40.105 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:40.094+0000 7f36f2200700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36ec114850 con 0x7f36ec074e80 2024-03-28T18:05:40.105 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:40.094+0000 7f36ebe00700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f36ec10ea50 0x7f36ec19a9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:05:40.105 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:40.094+0000 7f36ebe00700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f36ec10ea50 0x7f36ec19a9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:51224/0 (socket says 172.21.15.22:51224) 2024-03-28T18:05:40.105 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:40.094+0000 7f36ebe00700 1 -- 172.21.15.22:0/3101108597 learned_addr learned my addr 172.21.15.22:0/3101108597 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-28T18:05:40.105 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:40.094+0000 7f36eb400700 1 --2- 172.21.15.22:0/3101108597 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f36ec074560 0x7f36ec1a07e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:05:40.105 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:40.094+0000 7f36eaa00700 1 --2- 172.21.15.22:0/3101108597 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f36ec074e80 0x7f36ec1a0d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:05:40.105 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:40.095+0000 7f36ebe00700 1 -- 172.21.15.22:0/3101108597 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f36ec074e80 msgr2=0x7f36ec1a0d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:05:40.105 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:40.095+0000 7f36ebe00700 1 --2- 172.21.15.22:0/3101108597 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f36ec074e80 0x7f36ec1a0d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:05:40.105 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:40.095+0000 7f36ebe00700 1 -- 172.21.15.22:0/3101108597 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f36ec074560 msgr2=0x7f36ec1a07e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:05:40.105 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:40.095+0000 7f36ebe00700 1 --2- 172.21.15.22:0/3101108597 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f36ec074560 0x7f36ec1a07e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:05:40.105 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:40.095+0000 7f36ebe00700 1 -- 172.21.15.22:0/3101108597 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f36ec19b150 con 0x7f36ec10ea50 2024-03-28T18:05:40.105 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:40.095+0000 7f36eaa00700 1 --2- 172.21.15.22:0/3101108597 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f36ec074e80 0x7f36ec1a0d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-28T18:05:40.106 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:40.095+0000 7f36ebe00700 1 --2- 172.21.15.22:0/3101108597 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f36ec10ea50 0x7f36ec19a9f0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f36e400d010 tx=0x7f36e4004d00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:05:40.106 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:40.095+0000 7f36e3e00700 1 -- 172.21.15.22:0/3101108597 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36e4021070 con 0x7f36ec10ea50 2024-03-28T18:05:40.106 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:40.095+0000 7f36e3e00700 1 -- 172.21.15.22:0/3101108597 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f36e4007e50 con 0x7f36ec10ea50 2024-03-28T18:05:40.106 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:40.095+0000 7f36f2200700 1 -- 172.21.15.22:0/3101108597 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f36ec19b3e0 con 0x7f36ec10ea50 2024-03-28T18:05:40.106 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:40.095+0000 7f36e3e00700 1 -- 172.21.15.22:0/3101108597 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36e4004780 con 0x7f36ec10ea50 2024-03-28T18:05:40.106 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:40.096+0000 7f36f2200700 1 -- 172.21.15.22:0/3101108597 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f36ec1a76b0 con 0x7f36ec10ea50 2024-03-28T18:05:40.106 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:40.096+0000 7f36f2200700 1 -- 172.21.15.22:0/3101108597 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f36ec0628a0 con 0x7f36ec10ea50 2024-03-28T18:05:40.106 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:40.099+0000 7f36e3e00700 1 -- 172.21.15.22:0/3101108597 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 17) v1 ==== 137341+0+0 (secure 0 0 0) 0x7f36e4007980 con 0x7f36ec10ea50 2024-03-28T18:05:40.106 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:40.100+0000 7f36e3e00700 1 --2- 172.21.15.22:0/3101108597 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7f36d00a2920 0x7f36d00a4de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:05:40.106 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:40.100+0000 7f36e3e00700 1 -- 172.21.15.22:0/3101108597 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 8153+0+0 (secure 0 0 0) 0x7f36e400f020 con 0x7f36ec10ea50 2024-03-28T18:05:40.106 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:40.101+0000 7f36eb400700 1 --2- 172.21.15.22:0/3101108597 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7f36d00a2920 0x7f36d00a4de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:05:40.106 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:40.101+0000 7f36e3e00700 1 --2- 172.21.15.22:0/3101108597 >> [v2:172.21.15.134:6824/4271716428,v1:172.21.15.134:6825/4271716428] conn(0x7f36d00ac360 0x7f36d00ae800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:05:40.106 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:40.101+0000 7f36e3e00700 1 -- 172.21.15.22:0/3101108597 --> [v2:172.21.15.134:6824/4271716428,v1:172.21.15.134:6825/4271716428] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f36e401eec0 con 0x7f36d00ac360 2024-03-28T18:05:40.106 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:40.101+0000 7f36e3e00700 1 -- 172.21.15.22:0/3101108597 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_get_version_reply(handle=1 version=92) v2 ==== 24+0+0 (secure 0 0 0) 0x7f36e40d05a0 con 0x7f36ec10ea50 2024-03-28T18:05:40.107 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:40.101+0000 7f36eb400700 1 --2- 172.21.15.22:0/3101108597 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7f36d00a2920 0x7f36d00a4de0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f36ec111200 tx=0x7f36dc00b3c0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:05:40.107 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:40.102+0000 7f36eaa00700 1 --2- 172.21.15.22:0/3101108597 >> [v2:172.21.15.134:6824/4271716428,v1:172.21.15.134:6825/4271716428] conn(0x7f36d00ac360 0x7f36d00ae800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:05:40.107 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:40.103+0000 7f36eaa00700 1 --2- 172.21.15.22:0/3101108597 >> [v2:172.21.15.134:6824/4271716428,v1:172.21.15.134:6825/4271716428] conn(0x7f36d00ac360 0x7f36d00ae800 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.11 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:05:40.107 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:40.105+0000 7f36e3e00700 1 -- 172.21.15.22:0/3101108597 <== osd.11 v2:172.21.15.134:6824/4271716428 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7f36e401eec0 con 0x7f36d00ac360 2024-03-28T18:05:40.123 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:40.123+0000 7f36f2200700 1 -- 172.21.15.22:0/3101108597 --> [v2:172.21.15.134:6824/4271716428,v1:172.21.15.134:6825/4271716428] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f36ec062d20 con 0x7f36d00ac360 2024-03-28T18:05:40.124 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:40.124+0000 7f36e3e00700 1 -- 172.21.15.22:0/3101108597 <== osd.11 v2:172.21.15.134:6824/4271716428 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7f36ec062d20 con 0x7f36d00ac360 2024-03-28T18:05:40.125 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:40.125+0000 7f36f2200700 1 -- 172.21.15.22:0/3101108597 >> [v2:172.21.15.134:6824/4271716428,v1:172.21.15.134:6825/4271716428] conn(0x7f36d00ac360 msgr2=0x7f36d00ae800 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:05:40.125 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:40.125+0000 7f36f2200700 1 --2- 172.21.15.22:0/3101108597 >> [v2:172.21.15.134:6824/4271716428,v1:172.21.15.134:6825/4271716428] conn(0x7f36d00ac360 0x7f36d00ae800 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:05:40.125 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:40.125+0000 7f36f2200700 1 -- 172.21.15.22:0/3101108597 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7f36d00a2920 msgr2=0x7f36d00a4de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:05:40.125 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:40.125+0000 7f36f2200700 1 --2- 172.21.15.22:0/3101108597 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7f36d00a2920 0x7f36d00a4de0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f36ec111200 tx=0x7f36dc00b3c0 comp rx=0 tx=0).stop 2024-03-28T18:05:40.125 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:40.125+0000 7f36f2200700 1 -- 172.21.15.22:0/3101108597 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f36ec10ea50 msgr2=0x7f36ec19a9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:05:40.125 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:40.125+0000 7f36f2200700 1 --2- 172.21.15.22:0/3101108597 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f36ec10ea50 0x7f36ec19a9f0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f36e400d010 tx=0x7f36e4004d00 comp rx=0 tx=0).stop 2024-03-28T18:05:40.125 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:40.125+0000 7f36ebe00700 1 -- 172.21.15.22:0/3101108597 reap_dead start 2024-03-28T18:05:40.125 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:40.125+0000 7f36f2200700 1 -- 172.21.15.22:0/3101108597 shutdown_connections 2024-03-28T18:05:40.125 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:40.125+0000 7f36f2200700 1 -- 172.21.15.22:0/3101108597 >> 172.21.15.22:0/3101108597 conn(0x7f36ec0fdc60 msgr2=0x7f36ec0fe970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:05:40.126 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:40.126+0000 7f36f2200700 1 -- 172.21.15.22:0/3101108597 shutdown_connections 2024-03-28T18:05:40.126 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:40.126+0000 7f36f2200700 1 -- 172.21.15.22:0/3101108597 wait complete. 2024-03-28T18:05:40.174 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:05:39 smithi134 ceph-mon[35037]: pgmap v396: 97 pgs: 97 active+clean; 642 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-28T18:05:40.182 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:05:39 smithi079 ceph-mon[35312]: pgmap v396: 97 pgs: 97 active+clean; 642 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-28T18:05:40.734 INFO:teuthology.orchestra.run.smithi022.stdout:146028888138 2024-03-28T18:05:40.734 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033 shell --fsid fd220748-ed2b-11ee-95d0-87774f69a715 -- ceph osd last-stat-seq osd.4 2024-03-28T18:05:41.465 INFO:teuthology.orchestra.run.smithi022.stdout:365072220187 2024-03-28T18:05:41.465 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033 shell --fsid fd220748-ed2b-11ee-95d0-87774f69a715 -- ceph osd last-stat-seq osd.11 2024-03-28T18:05:42.031 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:05:41 smithi022 ceph-mon[31985]: pgmap v397: 97 pgs: 97 active+clean; 642 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-03-28T18:05:42.174 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:05:41 smithi134 ceph-mon[35037]: pgmap v397: 97 pgs: 97 active+clean; 642 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-03-28T18:05:42.182 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:05:41 smithi079 ceph-mon[35312]: pgmap v397: 97 pgs: 97 active+clean; 642 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-03-28T18:05:42.213 INFO:teuthology.orchestra.run.smithi022.stdout:34359738470 2024-03-28T18:05:42.213 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033 shell --fsid fd220748-ed2b-11ee-95d0-87774f69a715 -- ceph osd last-stat-seq osd.0 2024-03-28T18:05:42.536 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/fd220748-ed2b-11ee-95d0-87774f69a715/mon.a/config 2024-03-28T18:05:42.539 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/fd220748-ed2b-11ee-95d0-87774f69a715/mon.a/config 2024-03-28T18:05:42.565 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/fd220748-ed2b-11ee-95d0-87774f69a715/mon.a/config 2024-03-28T18:05:42.571 INFO:teuthology.orchestra.run.smithi022.stdout:171798691909 2024-03-28T18:05:42.572 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033 shell --fsid fd220748-ed2b-11ee-95d0-87774f69a715 -- ceph osd last-stat-seq osd.5 2024-03-28T18:05:43.753 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:43.171+0000 7f15fa200700 1 -- 172.21.15.22:0/2880787397 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f15f4109160 msgr2=0x7f15f4115670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:05:43.754 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:43.171+0000 7f15fa200700 1 --2- 172.21.15.22:0/2880787397 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f15f4109160 0x7f15f4115670 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f15ec00b0a0 tx=0x7f15ec00b3b0 comp rx=0 tx=0).stop 2024-03-28T18:05:43.754 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:43.171+0000 7f15fa200700 1 -- 172.21.15.22:0/2880787397 shutdown_connections 2024-03-28T18:05:43.754 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:43.171+0000 7f15fa200700 1 --2- 172.21.15.22:0/2880787397 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f15f4109160 0x7f15f4115670 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:05:43.754 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:43.171+0000 7f15fa200700 1 --2- 172.21.15.22:0/2880787397 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f15f410e7a0 0x7f15f410eb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:05:43.754 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:43.171+0000 7f15fa200700 1 --2- 172.21.15.22:0/2880787397 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f15f41087a0 0x7f15f4108c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:05:43.754 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:43.171+0000 7f15fa200700 1 -- 172.21.15.22:0/2880787397 >> 172.21.15.22:0/2880787397 conn(0x7f15f4075960 msgr2=0x7f15f4077d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:05:43.755 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:43.171+0000 7f15fa200700 1 -- 172.21.15.22:0/2880787397 shutdown_connections 2024-03-28T18:05:43.755 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:43.171+0000 7f15fa200700 1 -- 172.21.15.22:0/2880787397 wait complete. 2024-03-28T18:05:43.755 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:43.172+0000 7f15fa200700 1 Processor -- start 2024-03-28T18:05:43.755 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:43.173+0000 7f15fa200700 1 -- start start 2024-03-28T18:05:43.755 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:43.173+0000 7f15fa200700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f15f41087a0 0x7f15f419ef30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:05:43.755 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:43.173+0000 7f15fa200700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f15f4109160 0x7f15f419f470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:05:43.755 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:43.173+0000 7f15fa200700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f15f410e7a0 0x7f15f41a37b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:05:43.755 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:43.173+0000 7f15fa200700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15f4118b00 con 0x7f15f410e7a0 2024-03-28T18:05:43.755 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:43.173+0000 7f15fa200700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15f4118980 con 0x7f15f41087a0 2024-03-28T18:05:43.756 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:43.173+0000 7f15fa200700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15f4118c80 con 0x7f15f4109160 2024-03-28T18:05:43.756 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:43.173+0000 7f15f3e00700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f15f410e7a0 0x7f15f41a37b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:05:43.756 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:43.174+0000 7f15f3e00700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f15f410e7a0 0x7f15f41a37b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:51244/0 (socket says 172.21.15.22:51244) 2024-03-28T18:05:43.756 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:43.174+0000 7f15f3e00700 1 -- 172.21.15.22:0/171745920 learned_addr learned my addr 172.21.15.22:0/171745920 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-28T18:05:43.756 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:43.174+0000 7f15f3400700 1 --2- 172.21.15.22:0/171745920 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f15f41087a0 0x7f15f419ef30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:05:43.757 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:43.174+0000 7f15eaa00700 1 --2- 172.21.15.22:0/171745920 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f15f4109160 0x7f15f419f470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:05:43.757 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:43.174+0000 7f15f3e00700 1 -- 172.21.15.22:0/171745920 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f15f4109160 msgr2=0x7f15f419f470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:05:43.757 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:43.174+0000 7f15f3e00700 1 --2- 172.21.15.22:0/171745920 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f15f4109160 0x7f15f419f470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:05:43.757 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:43.174+0000 7f15f3e00700 1 -- 172.21.15.22:0/171745920 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f15f41087a0 msgr2=0x7f15f419ef30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:05:43.758 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:43.174+0000 7f15f3e00700 1 --2- 172.21.15.22:0/171745920 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f15f41087a0 0x7f15f419ef30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:05:43.758 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:43.174+0000 7f15f3e00700 1 -- 172.21.15.22:0/171745920 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f15f41a3ef0 con 0x7f15f410e7a0 2024-03-28T18:05:43.758 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:43.174+0000 7f15f3e00700 1 --2- 172.21.15.22:0/171745920 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f15f410e7a0 0x7f15f41a37b0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f15ec00b830 tx=0x7f15ec007a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:05:43.758 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:43.175+0000 7f15f0c00700 1 -- 172.21.15.22:0/171745920 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15ec021070 con 0x7f15f410e7a0 2024-03-28T18:05:43.758 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:43.175+0000 7f15fa200700 1 -- 172.21.15.22:0/171745920 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f15f41a4180 con 0x7f15f410e7a0 2024-03-28T18:05:43.758 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:43.175+0000 7f15f0c00700 1 -- 172.21.15.22:0/171745920 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f15ec01db20 con 0x7f15f410e7a0 2024-03-28T18:05:43.758 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:43.175+0000 7f15f0c00700 1 -- 172.21.15.22:0/171745920 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15ec01bb50 con 0x7f15f410e7a0 2024-03-28T18:05:43.758 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:43.175+0000 7f15fa200700 1 -- 172.21.15.22:0/171745920 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f15f41aba60 con 0x7f15f410e7a0 2024-03-28T18:05:43.759 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:43.176+0000 7f15fa200700 1 -- 172.21.15.22:0/171745920 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f15f4062920 con 0x7f15f410e7a0 2024-03-28T18:05:43.759 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:43.178+0000 7f15f0c00700 1 -- 172.21.15.22:0/171745920 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 17) v1 ==== 137341+0+0 (secure 0 0 0) 0x7f15ec024070 con 0x7f15f410e7a0 2024-03-28T18:05:43.759 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:43.180+0000 7f15f0c00700 1 --2- 172.21.15.22:0/171745920 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7f15d80a2920 0x7f15d80a4de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:05:43.759 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:43.180+0000 7f15f0c00700 1 -- 172.21.15.22:0/171745920 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 8153+0+0 (secure 0 0 0) 0x7f15ec00f020 con 0x7f15f410e7a0 2024-03-28T18:05:43.759 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:43.180+0000 7f15f3400700 1 --2- 172.21.15.22:0/171745920 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7f15d80a2920 0x7f15d80a4de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:05:43.759 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:43.180+0000 7f15f0c00700 1 --2- 172.21.15.22:0/171745920 >> [v2:172.21.15.22:6810/1702441854,v1:172.21.15.22:6811/1702441854] conn(0x7f15d80ac360 0x7f15d80ae800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:05:43.759 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:43.180+0000 7f15f0c00700 1 -- 172.21.15.22:0/171745920 --> [v2:172.21.15.22:6810/1702441854,v1:172.21.15.22:6811/1702441854] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f15ec026e50 con 0x7f15d80ac360 2024-03-28T18:05:43.760 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:43.180+0000 7f15f0c00700 1 -- 172.21.15.22:0/171745920 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_get_version_reply(handle=1 version=92) v2 ==== 24+0+0 (secure 0 0 0) 0x7f15ec0d0fe0 con 0x7f15f410e7a0 2024-03-28T18:05:43.760 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:43.180+0000 7f15eaa00700 1 --2- 172.21.15.22:0/171745920 >> [v2:172.21.15.22:6810/1702441854,v1:172.21.15.22:6811/1702441854] conn(0x7f15d80ac360 0x7f15d80ae800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:05:43.760 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:43.181+0000 7f15f3400700 1 --2- 172.21.15.22:0/171745920 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7f15d80a2920 0x7f15d80a4de0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f15e4009a10 tx=0x7f15e4009190 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:05:43.760 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:43.181+0000 7f15eaa00700 1 --2- 172.21.15.22:0/171745920 >> [v2:172.21.15.22:6810/1702441854,v1:172.21.15.22:6811/1702441854] conn(0x7f15d80ac360 0x7f15d80ae800 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:05:43.760 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:43.182+0000 7f15f0c00700 1 -- 172.21.15.22:0/171745920 <== osd.3 v2:172.21.15.22:6810/1702441854 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7f15ec026e50 con 0x7f15d80ac360 2024-03-28T18:05:43.760 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:43.202+0000 7f15fa200700 1 -- 172.21.15.22:0/171745920 --> [v2:172.21.15.22:6810/1702441854,v1:172.21.15.22:6811/1702441854] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f15f4062d50 con 0x7f15d80ac360 2024-03-28T18:05:43.760 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:43.203+0000 7f15f0c00700 1 -- 172.21.15.22:0/171745920 <== osd.3 v2:172.21.15.22:6810/1702441854 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7f15f4062d50 con 0x7f15d80ac360 2024-03-28T18:05:43.760 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:43.203+0000 7f15fa200700 1 -- 172.21.15.22:0/171745920 >> [v2:172.21.15.22:6810/1702441854,v1:172.21.15.22:6811/1702441854] conn(0x7f15d80ac360 msgr2=0x7f15d80ae800 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:05:43.761 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:43.203+0000 7f15fa200700 1 --2- 172.21.15.22:0/171745920 >> [v2:172.21.15.22:6810/1702441854,v1:172.21.15.22:6811/1702441854] conn(0x7f15d80ac360 0x7f15d80ae800 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:05:43.761 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:43.204+0000 7f15fa200700 1 -- 172.21.15.22:0/171745920 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7f15d80a2920 msgr2=0x7f15d80a4de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:05:43.761 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:43.204+0000 7f15fa200700 1 --2- 172.21.15.22:0/171745920 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7f15d80a2920 0x7f15d80a4de0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f15e4009a10 tx=0x7f15e4009190 comp rx=0 tx=0).stop 2024-03-28T18:05:43.761 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:43.204+0000 7f15fa200700 1 -- 172.21.15.22:0/171745920 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f15f410e7a0 msgr2=0x7f15f41a37b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:05:43.761 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:43.204+0000 7f15fa200700 1 --2- 172.21.15.22:0/171745920 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f15f410e7a0 0x7f15f41a37b0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f15ec00b830 tx=0x7f15ec007a20 comp rx=0 tx=0).stop 2024-03-28T18:05:43.761 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:43.204+0000 7f15f3e00700 1 -- 172.21.15.22:0/171745920 reap_dead start 2024-03-28T18:05:43.761 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:43.204+0000 7f15fa200700 1 -- 172.21.15.22:0/171745920 shutdown_connections 2024-03-28T18:05:43.761 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:43.204+0000 7f15fa200700 1 -- 172.21.15.22:0/171745920 >> 172.21.15.22:0/171745920 conn(0x7f15f4075960 msgr2=0x7f15f4075e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:05:43.762 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:43.204+0000 7f15fa200700 1 -- 172.21.15.22:0/171745920 shutdown_connections 2024-03-28T18:05:43.762 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:43.204+0000 7f15fa200700 1 -- 172.21.15.22:0/171745920 wait complete. 2024-03-28T18:05:44.014 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/fd220748-ed2b-11ee-95d0-87774f69a715/mon.a/config 2024-03-28T18:05:44.033 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:05:43 smithi022 ceph-mon[31985]: pgmap v398: 97 pgs: 97 active+clean; 642 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-03-28T18:05:44.174 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:05:43 smithi134 ceph-mon[35037]: pgmap v398: 97 pgs: 97 active+clean; 642 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-03-28T18:05:44.182 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:05:43 smithi079 ceph-mon[35312]: pgmap v398: 97 pgs: 97 active+clean; 642 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-03-28T18:05:44.372 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:44.372+0000 7ff57d200700 1 -- 172.21.15.22:0/75605886 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff5780766c0 msgr2=0x7ff578076aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:05:44.373 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:44.372+0000 7ff57d200700 1 --2- 172.21.15.22:0/75605886 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff5780766c0 0x7ff578076aa0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7ff568009990 tx=0x7ff568009ca0 comp rx=0 tx=0).stop 2024-03-28T18:05:44.373 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:44.373+0000 7ff57d200700 1 -- 172.21.15.22:0/75605886 shutdown_connections 2024-03-28T18:05:44.373 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:44.373+0000 7ff57d200700 1 --2- 172.21.15.22:0/75605886 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff578112f00 0x7ff5781152f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:05:44.373 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:44.373+0000 7ff57d200700 1 --2- 172.21.15.22:0/75605886 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff5780766c0 0x7ff578076aa0 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:05:44.373 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:44.373+0000 7ff57d200700 1 --2- 172.21.15.22:0/75605886 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ff578076fe0 0x7ff5781129c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:05:44.373 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:44.373+0000 7ff57d200700 1 -- 172.21.15.22:0/75605886 >> 172.21.15.22:0/75605886 conn(0x7ff5780ffd70 msgr2=0x7ff578102190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:05:44.373 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:44.373+0000 7ff57d200700 1 -- 172.21.15.22:0/75605886 shutdown_connections 2024-03-28T18:05:44.373 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:44.373+0000 7ff57d200700 1 -- 172.21.15.22:0/75605886 wait complete. 2024-03-28T18:05:44.374 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:44.374+0000 7ff57d200700 1 Processor -- start 2024-03-28T18:05:44.374 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:44.375+0000 7ff57d200700 1 -- start start 2024-03-28T18:05:44.375 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:44.375+0000 7ff57d200700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ff5780766c0 0x7ff57819eee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:05:44.375 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:44.375+0000 7ff57d200700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff578076fe0 0x7ff57819f420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:05:44.375 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:44.375+0000 7ff57d200700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff578112f00 0x7ff5781a3760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:05:44.375 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:44.375+0000 7ff57d200700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5781189d0 con 0x7ff578076fe0 2024-03-28T18:05:44.375 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:44.375+0000 7ff57d200700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff578118850 con 0x7ff5780766c0 2024-03-28T18:05:44.375 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:44.375+0000 7ff57d200700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff578118b50 con 0x7ff578112f00 2024-03-28T18:05:44.376 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:44.375+0000 7ff576a00700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff578076fe0 0x7ff57819f420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:05:44.377 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:44.375+0000 7ff577400700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ff5780766c0 0x7ff57819eee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:05:44.377 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:44.375+0000 7ff577400700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ff5780766c0 0x7ff57819eee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.22:45946/0 (socket says 172.21.15.22:45946) 2024-03-28T18:05:44.377 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:44.375+0000 7ff577400700 1 -- 172.21.15.22:0/1478858240 learned_addr learned my addr 172.21.15.22:0/1478858240 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-28T18:05:44.377 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:44.376+0000 7ff576a00700 1 -- 172.21.15.22:0/1478858240 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff578112f00 msgr2=0x7ff5781a3760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:05:44.377 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:44.376+0000 7ff577e00700 1 --2- 172.21.15.22:0/1478858240 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff578112f00 0x7ff5781a3760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:05:44.377 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:44.376+0000 7ff576a00700 1 --2- 172.21.15.22:0/1478858240 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff578112f00 0x7ff5781a3760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:05:44.377 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:44.376+0000 7ff576a00700 1 -- 172.21.15.22:0/1478858240 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ff5780766c0 msgr2=0x7ff57819eee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:05:44.377 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:44.376+0000 7ff576a00700 1 --2- 172.21.15.22:0/1478858240 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ff5780766c0 0x7ff57819eee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:05:44.377 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:44.376+0000 7ff576a00700 1 -- 172.21.15.22:0/1478858240 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff5781a3ea0 con 0x7ff578076fe0 2024-03-28T18:05:44.377 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:44.376+0000 7ff577400700 1 --2- 172.21.15.22:0/1478858240 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ff5780766c0 0x7ff57819eee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-28T18:05:44.377 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:44.376+0000 7ff577e00700 1 --2- 172.21.15.22:0/1478858240 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff578112f00 0x7ff5781a3760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T18:05:44.377 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:44.376+0000 7ff576a00700 1 --2- 172.21.15.22:0/1478858240 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff578076fe0 0x7ff57819f420 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7ff56400e9f0 tx=0x7ff56400ed00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:05:44.377 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:44.377+0000 7ff56fe00700 1 -- 172.21.15.22:0/1478858240 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff56400cbd0 con 0x7ff578076fe0 2024-03-28T18:05:44.378 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:44.377+0000 7ff57d200700 1 -- 172.21.15.22:0/1478858240 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff5781a4190 con 0x7ff578076fe0 2024-03-28T18:05:44.378 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:44.377+0000 7ff56fe00700 1 -- 172.21.15.22:0/1478858240 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7ff56400cd70 con 0x7ff578076fe0 2024-03-28T18:05:44.378 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:44.377+0000 7ff56fe00700 1 -- 172.21.15.22:0/1478858240 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff564018710 con 0x7ff578076fe0 2024-03-28T18:05:44.378 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:44.377+0000 7ff57d200700 1 -- 172.21.15.22:0/1478858240 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff5781aba10 con 0x7ff578076fe0 2024-03-28T18:05:44.378 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:44.378+0000 7ff57d200700 1 -- 172.21.15.22:0/1478858240 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7ff5780628a0 con 0x7ff578076fe0 2024-03-28T18:05:44.380 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:44.380+0000 7ff56fe00700 1 -- 172.21.15.22:0/1478858240 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 17) v1 ==== 137341+0+0 (secure 0 0 0) 0x7ff564010430 con 0x7ff578076fe0 2024-03-28T18:05:44.382 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:44.382+0000 7ff56fe00700 1 --2- 172.21.15.22:0/1478858240 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7ff55c0a29a0 0x7ff55c0a4e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:05:44.382 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:44.382+0000 7ff56fe00700 1 -- 172.21.15.22:0/1478858240 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 8153+0+0 (secure 0 0 0) 0x7ff5640cf3f0 con 0x7ff578076fe0 2024-03-28T18:05:44.382 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:44.382+0000 7ff577400700 1 --2- 172.21.15.22:0/1478858240 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7ff55c0a29a0 0x7ff55c0a4e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:05:44.382 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:44.382+0000 7ff56fe00700 1 --2- 172.21.15.22:0/1478858240 >> [v2:172.21.15.79:6824/1746167904,v1:172.21.15.79:6825/1746167904] conn(0x7ff55c0ac3e0 0x7ff55c0ae880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:05:44.382 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:44.382+0000 7ff56fe00700 1 -- 172.21.15.22:0/1478858240 --> [v2:172.21.15.79:6824/1746167904,v1:172.21.15.79:6825/1746167904] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7ff56401bec0 con 0x7ff55c0ac3e0 2024-03-28T18:05:44.382 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:44.382+0000 7ff56fe00700 1 -- 172.21.15.22:0/1478858240 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_get_version_reply(handle=1 version=92) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff5640cf8d0 con 0x7ff578076fe0 2024-03-28T18:05:44.383 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:44.383+0000 7ff577e00700 1 --2- 172.21.15.22:0/1478858240 >> [v2:172.21.15.79:6824/1746167904,v1:172.21.15.79:6825/1746167904] conn(0x7ff55c0ac3e0 0x7ff55c0ae880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:05:44.383 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:44.383+0000 7ff577400700 1 --2- 172.21.15.22:0/1478858240 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7ff55c0a29a0 0x7ff55c0a4e60 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7ff568006010 tx=0x7ff568009570 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:05:44.383 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:44.383+0000 7ff577e00700 1 --2- 172.21.15.22:0/1478858240 >> [v2:172.21.15.79:6824/1746167904,v1:172.21.15.79:6825/1746167904] conn(0x7ff55c0ac3e0 0x7ff55c0ae880 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.10 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:05:44.385 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:44.386+0000 7ff56fe00700 1 -- 172.21.15.22:0/1478858240 <== osd.10 v2:172.21.15.79:6824/1746167904 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7ff56401bec0 con 0x7ff55c0ac3e0 2024-03-28T18:05:44.404 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:44.404+0000 7ff57d200700 1 -- 172.21.15.22:0/1478858240 --> [v2:172.21.15.79:6824/1746167904,v1:172.21.15.79:6825/1746167904] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7ff578062ca0 con 0x7ff55c0ac3e0 2024-03-28T18:05:44.405 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:44.405+0000 7ff56fe00700 1 -- 172.21.15.22:0/1478858240 <== osd.10 v2:172.21.15.79:6824/1746167904 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7ff578062ca0 con 0x7ff55c0ac3e0 2024-03-28T18:05:44.405 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:44.406+0000 7ff57d200700 1 -- 172.21.15.22:0/1478858240 >> [v2:172.21.15.79:6824/1746167904,v1:172.21.15.79:6825/1746167904] conn(0x7ff55c0ac3e0 msgr2=0x7ff55c0ae880 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:05:44.405 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:44.406+0000 7ff57d200700 1 --2- 172.21.15.22:0/1478858240 >> [v2:172.21.15.79:6824/1746167904,v1:172.21.15.79:6825/1746167904] conn(0x7ff55c0ac3e0 0x7ff55c0ae880 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:05:44.406 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:44.406+0000 7ff57d200700 1 -- 172.21.15.22:0/1478858240 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7ff55c0a29a0 msgr2=0x7ff55c0a4e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:05:44.406 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:44.406+0000 7ff57d200700 1 --2- 172.21.15.22:0/1478858240 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7ff55c0a29a0 0x7ff55c0a4e60 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7ff568006010 tx=0x7ff568009570 comp rx=0 tx=0).stop 2024-03-28T18:05:44.406 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:44.406+0000 7ff57d200700 1 -- 172.21.15.22:0/1478858240 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff578076fe0 msgr2=0x7ff57819f420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:05:44.406 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:44.406+0000 7ff57d200700 1 --2- 172.21.15.22:0/1478858240 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff578076fe0 0x7ff57819f420 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7ff56400e9f0 tx=0x7ff56400ed00 comp rx=0 tx=0).stop 2024-03-28T18:05:44.406 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:44.406+0000 7ff577e00700 1 -- 172.21.15.22:0/1478858240 reap_dead start 2024-03-28T18:05:44.406 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:44.406+0000 7ff57d200700 1 -- 172.21.15.22:0/1478858240 shutdown_connections 2024-03-28T18:05:44.406 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:44.406+0000 7ff57d200700 1 -- 172.21.15.22:0/1478858240 >> 172.21.15.22:0/1478858240 conn(0x7ff5780ffd70 msgr2=0x7ff578102160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:05:44.406 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:44.406+0000 7ff57d200700 1 -- 172.21.15.22:0/1478858240 shutdown_connections 2024-03-28T18:05:44.406 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:44.406+0000 7ff57d200700 1 -- 172.21.15.22:0/1478858240 wait complete. 2024-03-28T18:05:46.119 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:05:45 smithi022 ceph-mon[31985]: pgmap v399: 97 pgs: 97 active+clean; 642 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:05:46.174 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:05:45 smithi134 ceph-mon[35037]: pgmap v399: 97 pgs: 97 active+clean; 642 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:05:46.182 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:05:45 smithi079 ceph-mon[35312]: pgmap v399: 97 pgs: 97 active+clean; 642 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:05:46.183 INFO:teuthology.orchestra.run.smithi022.stdout:330712481827 2024-03-28T18:05:46.183 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033 shell --fsid fd220748-ed2b-11ee-95d0-87774f69a715 -- ceph osd last-stat-seq osd.10 2024-03-28T18:05:46.610 INFO:teuthology.orchestra.run.smithi022.stdout:111669149779 2024-03-28T18:05:46.611 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033 shell --fsid fd220748-ed2b-11ee-95d0-87774f69a715 -- ceph osd last-stat-seq osd.3 2024-03-28T18:05:47.988 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/fd220748-ed2b-11ee-95d0-87774f69a715/mon.a/config 2024-03-28T18:05:48.031 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:05:47 smithi022 ceph-mon[31985]: pgmap v400: 97 pgs: 97 active+clean; 642 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 11 op/s 2024-03-28T18:05:48.174 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:05:47 smithi134 ceph-mon[35037]: pgmap v400: 97 pgs: 97 active+clean; 642 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 11 op/s 2024-03-28T18:05:48.182 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:05:47 smithi079 ceph-mon[35312]: pgmap v400: 97 pgs: 97 active+clean; 642 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 11 op/s 2024-03-28T18:05:48.858 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/fd220748-ed2b-11ee-95d0-87774f69a715/mon.a/config 2024-03-28T18:05:49.240 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:49.239+0000 7fb213800700 1 -- 172.21.15.22:0/380220650 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb20c1045e0 msgr2=0x7fb20c1049c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:05:49.240 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:49.239+0000 7fb213800700 1 --2- 172.21.15.22:0/380220650 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb20c1045e0 0x7fb20c1049c0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fb2000099c0 tx=0x7fb200009cd0 comp rx=0 tx=0).stop 2024-03-28T18:05:49.240 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:49.239+0000 7fb213800700 1 -- 172.21.15.22:0/380220650 shutdown_connections 2024-03-28T18:05:49.240 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:49.240+0000 7fb213800700 1 --2- 172.21.15.22:0/380220650 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fb20c1096c0 0x7fb20c111290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:05:49.240 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:49.240+0000 7fb213800700 1 --2- 172.21.15.22:0/380220650 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb20c1045e0 0x7fb20c1049c0 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:05:49.240 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:49.240+0000 7fb213800700 1 --2- 172.21.15.22:0/380220650 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb20c104f90 0x7fb20c108fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:05:49.240 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:49.240+0000 7fb213800700 1 -- 172.21.15.22:0/380220650 >> 172.21.15.22:0/380220650 conn(0x7fb20c0ffe70 msgr2=0x7fb20c102290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:05:49.240 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:49.240+0000 7fb213800700 1 -- 172.21.15.22:0/380220650 shutdown_connections 2024-03-28T18:05:49.240 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:49.240+0000 7fb213800700 1 -- 172.21.15.22:0/380220650 wait complete. 2024-03-28T18:05:49.241 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:49.241+0000 7fb213800700 1 Processor -- start 2024-03-28T18:05:49.241 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:49.242+0000 7fb213800700 1 -- start start 2024-03-28T18:05:49.242 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:49.242+0000 7fb213800700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb20c1045e0 0x7fb20c1a07d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:05:49.242 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:49.242+0000 7fb213800700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fb20c104f90 0x7fb20c1a0d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:05:49.242 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:49.242+0000 7fb213800700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb20c1096c0 0x7fb20c19a9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:05:49.242 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:49.242+0000 7fb213800700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb20c1146e0 con 0x7fb20c1096c0 2024-03-28T18:05:49.242 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:49.242+0000 7fb213800700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb20c114560 con 0x7fb20c104f90 2024-03-28T18:05:49.242 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:49.242+0000 7fb213800700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb20c114860 con 0x7fb20c1045e0 2024-03-28T18:05:49.243 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:49.242+0000 7fb211800700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb20c1096c0 0x7fb20c19a9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:05:49.244 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:49.242+0000 7fb211800700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb20c1096c0 0x7fb20c19a9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:49172/0 (socket says 172.21.15.22:49172) 2024-03-28T18:05:49.244 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:49.242+0000 7fb211800700 1 -- 172.21.15.22:0/413179600 learned_addr learned my addr 172.21.15.22:0/413179600 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-28T18:05:49.244 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:49.242+0000 7fb20be00700 1 --2- 172.21.15.22:0/413179600 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fb20c104f90 0x7fb20c1a0d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:05:49.244 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:49.243+0000 7fb210e00700 1 --2- 172.21.15.22:0/413179600 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb20c1045e0 0x7fb20c1a07d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:05:49.244 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:49.243+0000 7fb20be00700 1 -- 172.21.15.22:0/413179600 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb20c1045e0 msgr2=0x7fb20c1a07d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:05:49.244 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:49.243+0000 7fb20be00700 1 --2- 172.21.15.22:0/413179600 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb20c1045e0 0x7fb20c1a07d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:05:49.244 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:49.243+0000 7fb20be00700 1 -- 172.21.15.22:0/413179600 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb20c1096c0 msgr2=0x7fb20c19a9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:05:49.244 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:49.243+0000 7fb20be00700 1 --2- 172.21.15.22:0/413179600 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb20c1096c0 0x7fb20c19a9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:05:49.244 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:49.243+0000 7fb20be00700 1 -- 172.21.15.22:0/413179600 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb20c19b140 con 0x7fb20c104f90 2024-03-28T18:05:49.244 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:49.243+0000 7fb211800700 1 --2- 172.21.15.22:0/413179600 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb20c1096c0 0x7fb20c19a9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-28T18:05:49.244 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:49.243+0000 7fb20be00700 1 --2- 172.21.15.22:0/413179600 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fb20c104f90 0x7fb20c1a0d10 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fb20c06b980 tx=0x7fb1f800dc30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:05:49.244 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:49.244+0000 7fb209600700 1 -- 172.21.15.22:0/413179600 <== mon.2 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1f8016070 con 0x7fb20c104f90 2024-03-28T18:05:49.245 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:49.244+0000 7fb213800700 1 -- 172.21.15.22:0/413179600 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb20c19b430 con 0x7fb20c104f90 2024-03-28T18:05:49.245 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:49.244+0000 7fb209600700 1 -- 172.21.15.22:0/413179600 <== mon.2 v2:172.21.15.79:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7fb1f800fd60 con 0x7fb20c104f90 2024-03-28T18:05:49.245 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:49.244+0000 7fb209600700 1 -- 172.21.15.22:0/413179600 <== mon.2 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1f800e3b0 con 0x7fb20c104f90 2024-03-28T18:05:49.245 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:49.244+0000 7fb213800700 1 -- 172.21.15.22:0/413179600 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb20c1a76a0 con 0x7fb20c104f90 2024-03-28T18:05:49.245 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:49.245+0000 7fb213800700 1 -- 172.21.15.22:0/413179600 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb20c068e50 con 0x7fb20c104f90 2024-03-28T18:05:49.247 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:49.248+0000 7fb209600700 1 -- 172.21.15.22:0/413179600 <== mon.2 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 17) v1 ==== 137341+0+0 (secure 0 0 0) 0x7fb1f8019070 con 0x7fb20c104f90 2024-03-28T18:05:49.249 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:49.249+0000 7fb209600700 1 --2- 172.21.15.22:0/413179600 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7fb1f40a2920 0x7fb1f40a4de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:05:49.249 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:49.249+0000 7fb209600700 1 -- 172.21.15.22:0/413179600 <== mon.2 v2:172.21.15.79:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 8153+0+0 (secure 0 0 0) 0x7fb1f80ceb60 con 0x7fb20c104f90 2024-03-28T18:05:49.249 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:49.249+0000 7fb210e00700 1 --2- 172.21.15.22:0/413179600 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7fb1f40a2920 0x7fb1f40a4de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:05:49.249 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:49.250+0000 7fb210e00700 1 --2- 172.21.15.22:0/413179600 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7fb1f40a2920 0x7fb1f40a4de0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fb200000c00 tx=0x7fb200005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:05:49.263 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:49.263+0000 7fb209600700 1 -- 172.21.15.22:0/413179600 <== mon.2 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fb1f809baa0 con 0x7fb20c104f90 2024-03-28T18:05:49.500 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:49.500+0000 7fb213800700 1 -- 172.21.15.22:0/413179600 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7fb20c02da30 con 0x7fb20c104f90 2024-03-28T18:05:49.500 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:49.501+0000 7fb209600700 1 -- 172.21.15.22:0/413179600 <== mon.2 v2:172.21.15.79:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fb1f809b3a0 con 0x7fb20c104f90 2024-03-28T18:05:49.500 INFO:teuthology.orchestra.run.smithi022.stdout:77309411420 2024-03-28T18:05:49.504 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:49.504+0000 7fb213800700 1 -- 172.21.15.22:0/413179600 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7fb1f40a2920 msgr2=0x7fb1f40a4de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:05:49.504 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:49.504+0000 7fb213800700 1 --2- 172.21.15.22:0/413179600 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7fb1f40a2920 0x7fb1f40a4de0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fb200000c00 tx=0x7fb200005fb0 comp rx=0 tx=0).stop 2024-03-28T18:05:49.504 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:49.504+0000 7fb213800700 1 -- 172.21.15.22:0/413179600 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fb20c104f90 msgr2=0x7fb20c1a0d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:05:49.504 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:49.504+0000 7fb213800700 1 --2- 172.21.15.22:0/413179600 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fb20c104f90 0x7fb20c1a0d10 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fb20c06b980 tx=0x7fb1f800dc30 comp rx=0 tx=0).stop 2024-03-28T18:05:49.504 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:49.504+0000 7fb213800700 1 -- 172.21.15.22:0/413179600 shutdown_connections 2024-03-28T18:05:49.504 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:49.504+0000 7fb213800700 1 --2- 172.21.15.22:0/413179600 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb20c1096c0 0x7fb20c19a9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:05:49.504 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:49.504+0000 7fb213800700 1 --2- 172.21.15.22:0/413179600 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7fb1f40a2920 0x7fb1f40a4de0 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:05:49.505 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:49.504+0000 7fb213800700 1 --2- 172.21.15.22:0/413179600 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb20c1045e0 0x7fb20c1a07d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:05:49.505 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:49.504+0000 7fb213800700 1 --2- 172.21.15.22:0/413179600 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fb20c104f90 0x7fb20c1a0d10 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:05:49.505 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:49.504+0000 7fb213800700 1 -- 172.21.15.22:0/413179600 >> 172.21.15.22:0/413179600 conn(0x7fb20c0ffe70 msgr2=0x7fb20c102160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:05:49.505 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:49.504+0000 7fb213800700 1 -- 172.21.15.22:0/413179600 shutdown_connections 2024-03-28T18:05:49.505 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:49.504+0000 7fb213800700 1 -- 172.21.15.22:0/413179600 wait complete. 2024-03-28T18:05:49.995 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/fd220748-ed2b-11ee-95d0-87774f69a715/mon.a/config 2024-03-28T18:05:50.103 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:05:49 smithi022 ceph-mon[31985]: pgmap v401: 97 pgs: 97 active+clean; 642 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-03-28T18:05:50.103 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:05:49 smithi022 ceph-mon[31985]: from='client.? 172.21.15.22:0/413179600' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-03-28T18:05:50.174 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:05:49 smithi134 ceph-mon[35037]: pgmap v401: 97 pgs: 97 active+clean; 642 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-03-28T18:05:50.174 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:05:49 smithi134 ceph-mon[35037]: from='client.? 172.21.15.22:0/413179600' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-03-28T18:05:50.182 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:05:49 smithi079 ceph-mon[35312]: pgmap v401: 97 pgs: 97 active+clean; 642 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-03-28T18:05:50.182 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:05:49 smithi079 ceph-mon[35312]: from='client.? 172.21.15.22:0/413179600' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-03-28T18:05:51.759 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:51.758+0000 7fedd0a00700 1 -- 172.21.15.22:0/3713324815 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fedcc101140 msgr2=0x7fedcc10f0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:05:51.759 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:51.758+0000 7fedd0a00700 1 --2- 172.21.15.22:0/3713324815 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fedcc101140 0x7fedcc10f0a0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7fedc4009990 tx=0x7fedc4009ca0 comp rx=0 tx=0).stop 2024-03-28T18:05:51.759 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:51.758+0000 7fedd0a00700 1 -- 172.21.15.22:0/3713324815 shutdown_connections 2024-03-28T18:05:51.759 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:51.759+0000 7fedd0a00700 1 --2- 172.21.15.22:0/3713324815 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fedcc101140 0x7fedcc10f0a0 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:05:51.759 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:51.759+0000 7fedd0a00700 1 --2- 172.21.15.22:0/3713324815 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fedcc107200 0x7fedcc1075e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:05:51.759 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:51.759+0000 7fedd0a00700 1 --2- 172.21.15.22:0/3713324815 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fedcc100780 0x7fedcc100c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:05:51.759 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:51.759+0000 7fedd0a00700 1 -- 172.21.15.22:0/3713324815 >> 172.21.15.22:0/3713324815 conn(0x7fedcc077690 msgr2=0x7fedcc0feab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:05:51.759 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:51.759+0000 7fedd0a00700 1 -- 172.21.15.22:0/3713324815 shutdown_connections 2024-03-28T18:05:51.759 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:51.759+0000 7fedd0a00700 1 -- 172.21.15.22:0/3713324815 wait complete. 2024-03-28T18:05:51.761 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:51.760+0000 7fedd0a00700 1 Processor -- start 2024-03-28T18:05:51.761 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:51.760+0000 7fedd0a00700 1 -- start start 2024-03-28T18:05:51.761 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:51.760+0000 7fedd0a00700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fedcc100780 0x7fedcc101960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:05:51.761 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:51.760+0000 7fedd0a00700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fedcc101140 0x7fedcc101ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:05:51.761 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:51.760+0000 7fedd0a00700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fedcc107200 0x7fedcc1023e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:05:51.761 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:51.760+0000 7fedd0a00700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fedcc112620 con 0x7fedcc101140 2024-03-28T18:05:51.761 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:51.760+0000 7fedd0a00700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fedcc1124a0 con 0x7fedcc100780 2024-03-28T18:05:51.762 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:51.760+0000 7fedd0a00700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fedcc1127a0 con 0x7fedcc107200 2024-03-28T18:05:51.762 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:51.761+0000 7fedc1400700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fedcc101140 0x7fedcc101ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:05:51.762 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:51.761+0000 7fedc1400700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fedcc101140 0x7fedcc101ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:49192/0 (socket says 172.21.15.22:49192) 2024-03-28T18:05:51.762 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:51.761+0000 7fedc1400700 1 -- 172.21.15.22:0/2069422398 learned_addr learned my addr 172.21.15.22:0/2069422398 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-28T18:05:51.762 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:51.761+0000 7fedc9e00700 1 --2- 172.21.15.22:0/2069422398 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fedcc100780 0x7fedcc101960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:05:51.762 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:51.761+0000 7fedca800700 1 --2- 172.21.15.22:0/2069422398 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fedcc107200 0x7fedcc1023e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:05:51.762 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:51.761+0000 7fedc1400700 1 -- 172.21.15.22:0/2069422398 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fedcc107200 msgr2=0x7fedcc1023e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:05:51.762 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:51.761+0000 7fedc1400700 1 --2- 172.21.15.22:0/2069422398 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fedcc107200 0x7fedcc1023e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:05:51.762 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:51.761+0000 7fedc1400700 1 -- 172.21.15.22:0/2069422398 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fedcc100780 msgr2=0x7fedcc101960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:05:51.762 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:51.761+0000 7fedc1400700 1 --2- 172.21.15.22:0/2069422398 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fedcc100780 0x7fedcc101960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:05:51.762 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:51.761+0000 7fedc1400700 1 -- 172.21.15.22:0/2069422398 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fedcc1a6e70 con 0x7fedcc101140 2024-03-28T18:05:51.762 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:51.761+0000 7fedc1400700 1 --2- 172.21.15.22:0/2069422398 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fedcc101140 0x7fedcc101ea0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7fedb4007f10 tx=0x7fedb400d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:05:51.763 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:51.762+0000 7fedc3400700 1 -- 172.21.15.22:0/2069422398 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fedb400ddd0 con 0x7fedcc101140 2024-03-28T18:05:51.763 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:51.762+0000 7fedd0a00700 1 -- 172.21.15.22:0/2069422398 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fedcc1a7160 con 0x7fedcc101140 2024-03-28T18:05:51.763 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:51.762+0000 7fedc3400700 1 -- 172.21.15.22:0/2069422398 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7fedb4004bb0 con 0x7fedcc101140 2024-03-28T18:05:51.763 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:51.762+0000 7fedc3400700 1 -- 172.21.15.22:0/2069422398 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fedb4004810 con 0x7fedcc101140 2024-03-28T18:05:51.764 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:51.762+0000 7fedd0a00700 1 -- 172.21.15.22:0/2069422398 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fedcc1a76f0 con 0x7fedcc101140 2024-03-28T18:05:51.764 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:51.763+0000 7fedd0a00700 1 -- 172.21.15.22:0/2069422398 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fedcc068e50 con 0x7fedcc101140 2024-03-28T18:05:51.765 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:51.765+0000 7fedc3400700 1 -- 172.21.15.22:0/2069422398 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 17) v1 ==== 137341+0+0 (secure 0 0 0) 0x7fedb401c020 con 0x7fedcc101140 2024-03-28T18:05:51.766 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:51.767+0000 7fedc3400700 1 --2- 172.21.15.22:0/2069422398 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7fedb00a29a0 0x7fedb00a4e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:05:51.767 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:51.767+0000 7fedc3400700 1 -- 172.21.15.22:0/2069422398 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 8153+0+0 (secure 0 0 0) 0x7fedb40ce470 con 0x7fedcc101140 2024-03-28T18:05:51.767 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:51.767+0000 7fedc9e00700 1 --2- 172.21.15.22:0/2069422398 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7fedb00a29a0 0x7fedb00a4e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:05:51.767 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:51.768+0000 7fedc9e00700 1 --2- 172.21.15.22:0/2069422398 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7fedb00a29a0 0x7fedb00a4e60 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fedbc00abd0 tx=0x7fedbc00a380 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:05:51.771 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:51.771+0000 7fedc3400700 1 -- 172.21.15.22:0/2069422398 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fedb4010070 con 0x7fedcc101140 2024-03-28T18:05:51.976 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/fd220748-ed2b-11ee-95d0-87774f69a715/mon.a/config 2024-03-28T18:05:51.983 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:05:51 smithi022 ceph-mon[31985]: pgmap v402: 97 pgs: 97 active+clean; 642 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-03-28T18:05:52.024 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:52.024+0000 7fedd0a00700 1 -- 172.21.15.22:0/2069422398 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 9} v 0) v1 -- 0x7fedcc02da30 con 0x7fedcc101140 2024-03-28T18:05:52.025 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:52.025+0000 7fedc3400700 1 -- 172.21.15.22:0/2069422398 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 9}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fedb409b3b0 con 0x7fedcc101140 2024-03-28T18:05:52.025 INFO:teuthology.orchestra.run.smithi022.stdout:300647710763 2024-03-28T18:05:52.028 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:52.028+0000 7fedd0a00700 1 -- 172.21.15.22:0/2069422398 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7fedb00a29a0 msgr2=0x7fedb00a4e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:05:52.028 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:52.028+0000 7fedd0a00700 1 --2- 172.21.15.22:0/2069422398 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7fedb00a29a0 0x7fedb00a4e60 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fedbc00abd0 tx=0x7fedbc00a380 comp rx=0 tx=0).stop 2024-03-28T18:05:52.028 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:52.028+0000 7fedd0a00700 1 -- 172.21.15.22:0/2069422398 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fedcc101140 msgr2=0x7fedcc101ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:05:52.028 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:52.028+0000 7fedd0a00700 1 --2- 172.21.15.22:0/2069422398 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fedcc101140 0x7fedcc101ea0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7fedb4007f10 tx=0x7fedb400d3b0 comp rx=0 tx=0).stop 2024-03-28T18:05:52.028 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:52.028+0000 7fedd0a00700 1 -- 172.21.15.22:0/2069422398 shutdown_connections 2024-03-28T18:05:52.028 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:52.028+0000 7fedd0a00700 1 --2- 172.21.15.22:0/2069422398 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7fedb00a29a0 0x7fedb00a4e60 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:05:52.028 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:52.028+0000 7fedd0a00700 1 --2- 172.21.15.22:0/2069422398 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fedcc107200 0x7fedcc1023e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:05:52.028 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:52.028+0000 7fedd0a00700 1 --2- 172.21.15.22:0/2069422398 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fedcc100780 0x7fedcc101960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:05:52.028 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:52.028+0000 7fedd0a00700 1 --2- 172.21.15.22:0/2069422398 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fedcc101140 0x7fedcc101ea0 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:05:52.029 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:52.028+0000 7fedd0a00700 1 -- 172.21.15.22:0/2069422398 >> 172.21.15.22:0/2069422398 conn(0x7fedcc077690 msgr2=0x7fedcc105130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:05:52.029 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:52.028+0000 7fedd0a00700 1 -- 172.21.15.22:0/2069422398 shutdown_connections 2024-03-28T18:05:52.029 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:52.028+0000 7fedd0a00700 1 -- 172.21.15.22:0/2069422398 wait complete. 2024-03-28T18:05:52.174 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:05:51 smithi134 ceph-mon[35037]: pgmap v402: 97 pgs: 97 active+clean; 642 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-03-28T18:05:52.182 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:05:51 smithi079 ceph-mon[35312]: pgmap v402: 97 pgs: 97 active+clean; 642 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-03-28T18:05:53.013 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:05:52 smithi022 ceph-mon[31985]: from='client.? 172.21.15.22:0/2069422398' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-03-28T18:05:53.013 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:05:52 smithi022 ceph-mon[31985]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T18:05:53.174 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:05:52 smithi134 ceph-mon[35037]: from='client.? 172.21.15.22:0/2069422398' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-03-28T18:05:53.174 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:05:52 smithi134 ceph-mon[35037]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T18:05:53.182 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:05:52 smithi079 ceph-mon[35312]: from='client.? 172.21.15.22:0/2069422398' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-03-28T18:05:53.182 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:05:52 smithi079 ceph-mon[35312]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T18:05:53.659 INFO:tasks.cephadm.ceph_manager.ceph:need seq 77309411415 got 77309411420 for osd.2 2024-03-28T18:05:53.659 DEBUG:teuthology.parallel:result is None 2024-03-28T18:05:54.175 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:05:53 smithi134 ceph-mon[35037]: pgmap v403: 97 pgs: 97 active+clean; 642 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-03-28T18:05:54.175 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:05:53 smithi134 ceph-mon[35037]: from='client.? 172.21.15.22:0/3189066156' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-03-28T18:05:54.175 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:05:53 smithi134 ceph-mon[35037]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T18:05:54.175 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:05:53 smithi134 ceph-mon[35037]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T18:05:54.182 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:05:53 smithi079 ceph-mon[35312]: pgmap v403: 97 pgs: 97 active+clean; 642 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-03-28T18:05:54.182 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:05:53 smithi079 ceph-mon[35312]: from='client.? 172.21.15.22:0/3189066156' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-03-28T18:05:54.182 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:05:53 smithi079 ceph-mon[35312]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T18:05:54.182 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:05:53 smithi079 ceph-mon[35312]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T18:05:54.281 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:05:53 smithi022 ceph-mon[31985]: pgmap v403: 97 pgs: 97 active+clean; 642 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-03-28T18:05:54.282 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:05:53 smithi022 ceph-mon[31985]: from='client.? 172.21.15.22:0/3189066156' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-03-28T18:05:54.282 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:05:53 smithi022 ceph-mon[31985]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T18:05:54.282 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:05:53 smithi022 ceph-mon[31985]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T18:05:54.491 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:52.842+0000 7f006f600700 1 -- 172.21.15.22:0/1991089653 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f0068102210 msgr2=0x7f0068102690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:05:54.491 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:52.842+0000 7f006f600700 1 --2- 172.21.15.22:0/1991089653 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f0068102210 0x7f0068102690 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f00540099c0 tx=0x7f0054009cd0 comp rx=0 tx=0).stop 2024-03-28T18:05:54.491 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:52.842+0000 7f006f600700 1 -- 172.21.15.22:0/1991089653 shutdown_connections 2024-03-28T18:05:54.492 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:52.842+0000 7f006f600700 1 --2- 172.21.15.22:0/1991089653 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0068102bd0 0x7f006810f060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:05:54.492 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:52.842+0000 7f006f600700 1 --2- 172.21.15.22:0/1991089653 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0068108260 0x7f0068108640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:05:54.492 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:52.842+0000 7f006f600700 1 --2- 172.21.15.22:0/1991089653 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f0068102210 0x7f0068102690 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:05:54.492 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:52.842+0000 7f006f600700 1 -- 172.21.15.22:0/1991089653 >> 172.21.15.22:0/1991089653 conn(0x7f00680792e0 msgr2=0x7f00680796f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:05:54.492 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:52.842+0000 7f006f600700 1 -- 172.21.15.22:0/1991089653 shutdown_connections 2024-03-28T18:05:54.492 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:52.842+0000 7f006f600700 1 -- 172.21.15.22:0/1991089653 wait complete. 2024-03-28T18:05:54.492 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:52.843+0000 7f006f600700 1 Processor -- start 2024-03-28T18:05:54.492 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:52.843+0000 7f006f600700 1 -- start start 2024-03-28T18:05:54.492 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:52.844+0000 7f006f600700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0068102210 0x7f006819e660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:05:54.492 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:52.844+0000 7f006f600700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f0068102bd0 0x7f006819eba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:05:54.492 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:52.844+0000 7f006f600700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0068108260 0x7f0068198730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:05:54.492 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:52.844+0000 7f006f600700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00681124b0 con 0x7f0068108260 2024-03-28T18:05:54.492 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:52.844+0000 7f006f600700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0068112330 con 0x7f0068102bd0 2024-03-28T18:05:54.492 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:52.844+0000 7f006f600700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0068112630 con 0x7f0068102210 2024-03-28T18:05:54.493 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:52.844+0000 7f006d600700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0068108260 0x7f0068198730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:05:54.493 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:52.844+0000 7f006cc00700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0068102210 0x7f006819e660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:05:54.493 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:52.844+0000 7f006d600700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0068108260 0x7f0068198730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:49220/0 (socket says 172.21.15.22:49220) 2024-03-28T18:05:54.493 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:52.844+0000 7f006d600700 1 -- 172.21.15.22:0/3189066156 learned_addr learned my addr 172.21.15.22:0/3189066156 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-28T18:05:54.493 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:52.844+0000 7f0067e00700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f0068102bd0 0x7f006819eba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:05:54.493 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:52.845+0000 7f006d600700 1 -- 172.21.15.22:0/3189066156 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0068102210 msgr2=0x7f006819e660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:05:54.493 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:52.845+0000 7f006d600700 1 --2- 172.21.15.22:0/3189066156 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0068102210 0x7f006819e660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:05:54.493 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:52.845+0000 7f006d600700 1 -- 172.21.15.22:0/3189066156 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f0068102bd0 msgr2=0x7f006819eba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:05:54.493 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:52.845+0000 7f006d600700 1 --2- 172.21.15.22:0/3189066156 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f0068102bd0 0x7f006819eba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:05:54.493 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:52.845+0000 7f006d600700 1 -- 172.21.15.22:0/3189066156 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0068199010 con 0x7f0068108260 2024-03-28T18:05:54.493 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:52.845+0000 7f006d600700 1 --2- 172.21.15.22:0/3189066156 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0068108260 0x7f0068198730 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f006000efc0 tx=0x7f006000c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:05:54.493 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:52.845+0000 7f0065600700 1 -- 172.21.15.22:0/3189066156 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00600092e0 con 0x7f0068108260 2024-03-28T18:05:54.493 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:52.845+0000 7f0065600700 1 -- 172.21.15.22:0/3189066156 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f0060004bb0 con 0x7f0068108260 2024-03-28T18:05:54.493 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:52.845+0000 7f006f600700 1 -- 172.21.15.22:0/3189066156 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f00681992a0 con 0x7f0068108260 2024-03-28T18:05:54.494 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:52.845+0000 7f0065600700 1 -- 172.21.15.22:0/3189066156 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0060004020 con 0x7f0068108260 2024-03-28T18:05:54.494 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:52.845+0000 7f006f600700 1 -- 172.21.15.22:0/3189066156 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f00681a55e0 con 0x7f0068108260 2024-03-28T18:05:54.494 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:52.847+0000 7f006f600700 1 -- 172.21.15.22:0/3189066156 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f00681038d0 con 0x7f0068108260 2024-03-28T18:05:54.494 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:52.849+0000 7f0065600700 1 -- 172.21.15.22:0/3189066156 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 17) v1 ==== 137341+0+0 (secure 0 0 0) 0x7f006001b030 con 0x7f0068108260 2024-03-28T18:05:54.494 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:52.851+0000 7f0065600700 1 --2- 172.21.15.22:0/3189066156 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7f00500a2920 0x7f00500a4de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:05:54.494 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:52.852+0000 7f0065600700 1 -- 172.21.15.22:0/3189066156 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 8153+0+0 (secure 0 0 0) 0x7f00600ce520 con 0x7f0068108260 2024-03-28T18:05:54.494 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:52.852+0000 7f006cc00700 1 --2- 172.21.15.22:0/3189066156 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7f00500a2920 0x7f00500a4de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:05:54.494 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:52.852+0000 7f006cc00700 1 --2- 172.21.15.22:0/3189066156 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7f00500a2920 0x7f00500a4de0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f005c009be0 tx=0x7f005c009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:05:54.494 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:52.855+0000 7f0065600700 1 -- 172.21.15.22:0/3189066156 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f0060022020 con 0x7f0068108260 2024-03-28T18:05:54.494 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:53.094+0000 7f006f600700 1 -- 172.21.15.22:0/3189066156 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) v1 -- 0x7f006802da30 con 0x7f0068108260 2024-03-28T18:05:54.494 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:53.095+0000 7f0065600700 1 -- 172.21.15.22:0/3189066156 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f0060010040 con 0x7f0068108260 2024-03-28T18:05:54.495 INFO:teuthology.orchestra.run.smithi022.stdout:227633266743 2024-03-28T18:05:54.495 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:53.098+0000 7f006f600700 1 -- 172.21.15.22:0/3189066156 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7f00500a2920 msgr2=0x7f00500a4de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:05:54.495 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:53.098+0000 7f006f600700 1 --2- 172.21.15.22:0/3189066156 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7f00500a2920 0x7f00500a4de0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f005c009be0 tx=0x7f005c009260 comp rx=0 tx=0).stop 2024-03-28T18:05:54.495 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:53.098+0000 7f006f600700 1 -- 172.21.15.22:0/3189066156 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0068108260 msgr2=0x7f0068198730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:05:54.495 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:53.098+0000 7f006f600700 1 --2- 172.21.15.22:0/3189066156 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0068108260 0x7f0068198730 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f006000efc0 tx=0x7f006000c3a0 comp rx=0 tx=0).stop 2024-03-28T18:05:54.495 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:53.098+0000 7f006f600700 1 -- 172.21.15.22:0/3189066156 shutdown_connections 2024-03-28T18:05:54.495 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:53.098+0000 7f006f600700 1 --2- 172.21.15.22:0/3189066156 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0068108260 0x7f0068198730 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:05:54.495 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:53.098+0000 7f006f600700 1 --2- 172.21.15.22:0/3189066156 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7f00500a2920 0x7f00500a4de0 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:05:54.495 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:53.098+0000 7f006f600700 1 --2- 172.21.15.22:0/3189066156 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0068102210 0x7f006819e660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:05:54.495 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:53.098+0000 7f006f600700 1 --2- 172.21.15.22:0/3189066156 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f0068102bd0 0x7f006819eba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:05:54.496 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:53.098+0000 7f006f600700 1 -- 172.21.15.22:0/3189066156 >> 172.21.15.22:0/3189066156 conn(0x7f00680792e0 msgr2=0x7f0068100f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:05:54.496 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:53.098+0000 7f006f600700 1 -- 172.21.15.22:0/3189066156 shutdown_connections 2024-03-28T18:05:54.496 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:53.098+0000 7f006f600700 1 -- 172.21.15.22:0/3189066156 wait complete. 2024-03-28T18:05:54.846 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/fd220748-ed2b-11ee-95d0-87774f69a715/mon.a/config 2024-03-28T18:05:54.863 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/fd220748-ed2b-11ee-95d0-87774f69a715/mon.a/config 2024-03-28T18:05:55.180 INFO:tasks.cephadm.ceph_manager.ceph:need seq 227633266739 got 227633266743 for osd.7 2024-03-28T18:05:55.180 DEBUG:teuthology.parallel:result is None 2024-03-28T18:05:56.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:05:56 smithi022 ceph-mon[31985]: pgmap v404: 97 pgs: 97 active+clean; 642 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-28T18:05:56.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:05:56 smithi022 ceph-mon[31985]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T18:05:56.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:05:56 smithi022 ceph-mon[31985]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T18:05:56.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:05:56 smithi022 ceph-mon[31985]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T18:05:56.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:05:56 smithi022 ceph-mon[31985]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T18:05:56.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:05:56 smithi134 ceph-mon[35037]: pgmap v404: 97 pgs: 97 active+clean; 642 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-28T18:05:56.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:05:56 smithi134 ceph-mon[35037]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T18:05:56.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:05:56 smithi134 ceph-mon[35037]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T18:05:56.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:05:56 smithi134 ceph-mon[35037]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T18:05:56.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:05:56 smithi134 ceph-mon[35037]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T18:05:56.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:05:56 smithi079 ceph-mon[35312]: pgmap v404: 97 pgs: 97 active+clean; 642 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-28T18:05:56.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:05:56 smithi079 ceph-mon[35312]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T18:05:56.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:05:56 smithi079 ceph-mon[35312]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T18:05:56.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:05:56 smithi079 ceph-mon[35312]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T18:05:56.683 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:05:56 smithi079 ceph-mon[35312]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T18:05:57.873 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:05:57 smithi022 ceph-mon[31985]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T18:05:57.873 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:05:57 smithi022 ceph-mon[31985]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T18:05:57.873 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:05:57 smithi022 ceph-mon[31985]: pgmap v405: 97 pgs: 97 active+clean; 642 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 10 op/s 2024-03-28T18:05:57.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:05:57 smithi134 ceph-mon[35037]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T18:05:57.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:05:57 smithi134 ceph-mon[35037]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T18:05:57.925 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:05:57 smithi134 ceph-mon[35037]: pgmap v405: 97 pgs: 97 active+clean; 642 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 10 op/s 2024-03-28T18:05:57.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:05:57 smithi079 ceph-mon[35312]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T18:05:57.933 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:05:57 smithi079 ceph-mon[35312]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T18:05:57.933 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:05:57 smithi079 ceph-mon[35312]: pgmap v405: 97 pgs: 97 active+clean; 642 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 10 op/s 2024-03-28T18:05:58.212 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:58.212+0000 7f821de00700 1 -- 172.21.15.22:0/1901345972 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f821806a7e0 msgr2=0x7f821810f020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:05:58.212 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:58.212+0000 7f821de00700 1 --2- 172.21.15.22:0/1901345972 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f821806a7e0 0x7f821810f020 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f821000b0a0 tx=0x7f821000b3b0 comp rx=0 tx=0).stop 2024-03-28T18:05:58.213 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:58.212+0000 7f821de00700 1 -- 172.21.15.22:0/1901345972 shutdown_connections 2024-03-28T18:05:58.213 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:58.212+0000 7f821de00700 1 --2- 172.21.15.22:0/1901345972 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f821806a7e0 0x7f821810f020 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:05:58.213 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:58.212+0000 7f821de00700 1 --2- 172.21.15.22:0/1901345972 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f82181081e0 0x7f82181085c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:05:58.213 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:58.212+0000 7f821de00700 1 --2- 172.21.15.22:0/1901345972 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8218069e40 0x7f821806a2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:05:58.213 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:58.212+0000 7f821de00700 1 -- 172.21.15.22:0/1901345972 >> 172.21.15.22:0/1901345972 conn(0x7f8218077fc0 msgr2=0x7f82180783d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:05:58.213 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:58.212+0000 7f821de00700 1 -- 172.21.15.22:0/1901345972 shutdown_connections 2024-03-28T18:05:58.213 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:58.212+0000 7f821de00700 1 -- 172.21.15.22:0/1901345972 wait complete. 2024-03-28T18:05:58.213 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:58.214+0000 7f821de00700 1 Processor -- start 2024-03-28T18:05:58.213 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:58.214+0000 7f821de00700 1 -- start start 2024-03-28T18:05:58.214 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:58.214+0000 7f821de00700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f8218069e40 0x7f82181a0720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:05:58.215 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:58.214+0000 7f821de00700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f821806a7e0 0x7f82181a0c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:05:58.215 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:58.214+0000 7f821de00700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f82181081e0 0x7f821819a8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:05:58.215 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:58.214+0000 7f821de00700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8218112420 con 0x7f82181081e0 2024-03-28T18:05:58.215 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:58.214+0000 7f821de00700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82181122a0 con 0x7f8218069e40 2024-03-28T18:05:58.215 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:58.214+0000 7f821de00700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82181125a0 con 0x7f821806a7e0 2024-03-28T18:05:58.215 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:58.215+0000 7f8216a00700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f8218069e40 0x7f82181a0720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:05:58.215 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:58.215+0000 7f8216a00700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f8218069e40 0x7f82181a0720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.22:41756/0 (socket says 172.21.15.22:41756) 2024-03-28T18:05:58.215 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:58.215+0000 7f8216a00700 1 -- 172.21.15.22:0/460392090 learned_addr learned my addr 172.21.15.22:0/460392090 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-28T18:05:58.215 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:58.215+0000 7f8217400700 1 --2- 172.21.15.22:0/460392090 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f82181081e0 0x7f821819a8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:05:58.215 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:58.215+0000 7f820e000700 1 --2- 172.21.15.22:0/460392090 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f821806a7e0 0x7f82181a0c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:05:58.216 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:58.215+0000 7f8216a00700 1 -- 172.21.15.22:0/460392090 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f821806a7e0 msgr2=0x7f82181a0c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:05:58.216 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:58.215+0000 7f8216a00700 1 --2- 172.21.15.22:0/460392090 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f821806a7e0 0x7f82181a0c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:05:58.216 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:58.215+0000 7f8216a00700 1 -- 172.21.15.22:0/460392090 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f82181081e0 msgr2=0x7f821819a8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:05:58.216 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:58.215+0000 7f8216a00700 1 --2- 172.21.15.22:0/460392090 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f82181081e0 0x7f821819a8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:05:58.216 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:58.215+0000 7f8216a00700 1 -- 172.21.15.22:0/460392090 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f821819b040 con 0x7f8218069e40 2024-03-28T18:05:58.216 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:58.215+0000 7f820e000700 1 --2- 172.21.15.22:0/460392090 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f821806a7e0 0x7f82181a0c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T18:05:58.216 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:58.216+0000 7f8216a00700 1 --2- 172.21.15.22:0/460392090 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f8218069e40 0x7f82181a0720 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f820800e9a0 tx=0x7f820800ecb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:05:58.217 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:58.216+0000 7f821ca00700 1 -- 172.21.15.22:0/460392090 <== mon.2 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f820800cbd0 con 0x7f8218069e40 2024-03-28T18:05:58.217 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:58.216+0000 7f821ca00700 1 -- 172.21.15.22:0/460392090 <== mon.2 v2:172.21.15.79:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f820800cd70 con 0x7f8218069e40 2024-03-28T18:05:58.217 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:58.216+0000 7f821de00700 1 -- 172.21.15.22:0/460392090 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f821819b330 con 0x7f8218069e40 2024-03-28T18:05:58.217 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:58.216+0000 7f821ca00700 1 -- 172.21.15.22:0/460392090 <== mon.2 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8208018680 con 0x7f8218069e40 2024-03-28T18:05:58.217 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:58.216+0000 7f821de00700 1 -- 172.21.15.22:0/460392090 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f82181a75f0 con 0x7f8218069e40 2024-03-28T18:05:58.217 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:58.217+0000 7f821de00700 1 -- 172.21.15.22:0/460392090 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8218068e50 con 0x7f8218069e40 2024-03-28T18:05:58.219 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:58.219+0000 7f821ca00700 1 -- 172.21.15.22:0/460392090 <== mon.2 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 17) v1 ==== 137341+0+0 (secure 0 0 0) 0x7f8208010430 con 0x7f8218069e40 2024-03-28T18:05:58.221 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:58.221+0000 7f821ca00700 1 --2- 172.21.15.22:0/460392090 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7f81fc0a2920 0x7f81fc0a4de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:05:58.221 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:58.221+0000 7f821ca00700 1 -- 172.21.15.22:0/460392090 <== mon.2 v2:172.21.15.79:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 8153+0+0 (secure 0 0 0) 0x7f82080cf2d0 con 0x7f8218069e40 2024-03-28T18:05:58.221 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:58.221+0000 7f820e000700 1 --2- 172.21.15.22:0/460392090 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7f81fc0a2920 0x7f81fc0a4de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:05:58.221 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:58.222+0000 7f820e000700 1 --2- 172.21.15.22:0/460392090 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7f81fc0a2920 0x7f81fc0a4de0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f8200005fd0 tx=0x7f8200005dc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:05:58.226 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:58.226+0000 7f821ca00700 1 -- 172.21.15.22:0/460392090 <== mon.2 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f820809c210 con 0x7f8218069e40 2024-03-28T18:05:58.460 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:58.460+0000 7f821de00700 1 -- 172.21.15.22:0/460392090 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 8} v 0) v1 -- 0x7f82181a7820 con 0x7f8218069e40 2024-03-28T18:05:58.460 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:58.461+0000 7f821ca00700 1 -- 172.21.15.22:0/460392090 <== mon.2 v2:172.21.15.79:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 8}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f820809bb10 con 0x7f8218069e40 2024-03-28T18:05:58.460 INFO:teuthology.orchestra.run.smithi022.stdout:266287972402 2024-03-28T18:05:58.464 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:58.464+0000 7f821de00700 1 -- 172.21.15.22:0/460392090 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7f81fc0a2920 msgr2=0x7f81fc0a4de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:05:58.464 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:58.464+0000 7f821de00700 1 --2- 172.21.15.22:0/460392090 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7f81fc0a2920 0x7f81fc0a4de0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f8200005fd0 tx=0x7f8200005dc0 comp rx=0 tx=0).stop 2024-03-28T18:05:58.464 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:58.464+0000 7f821de00700 1 -- 172.21.15.22:0/460392090 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f8218069e40 msgr2=0x7f82181a0720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:05:58.464 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:58.464+0000 7f821de00700 1 --2- 172.21.15.22:0/460392090 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f8218069e40 0x7f82181a0720 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f820800e9a0 tx=0x7f820800ecb0 comp rx=0 tx=0).stop 2024-03-28T18:05:58.464 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:58.464+0000 7f821de00700 1 -- 172.21.15.22:0/460392090 shutdown_connections 2024-03-28T18:05:58.464 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:58.464+0000 7f821de00700 1 --2- 172.21.15.22:0/460392090 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f82181081e0 0x7f821819a8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:05:58.464 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:58.464+0000 7f821de00700 1 --2- 172.21.15.22:0/460392090 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f8218069e40 0x7f82181a0720 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:05:58.464 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:58.464+0000 7f821de00700 1 --2- 172.21.15.22:0/460392090 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7f81fc0a2920 0x7f81fc0a4de0 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:05:58.464 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:58.464+0000 7f821de00700 1 --2- 172.21.15.22:0/460392090 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f821806a7e0 0x7f82181a0c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:05:58.465 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:58.464+0000 7f821de00700 1 -- 172.21.15.22:0/460392090 >> 172.21.15.22:0/460392090 conn(0x7f8218077fc0 msgr2=0x7f821810d370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:05:58.465 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:58.464+0000 7f821de00700 1 -- 172.21.15.22:0/460392090 shutdown_connections 2024-03-28T18:05:58.465 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:58.464+0000 7f821de00700 1 -- 172.21.15.22:0/460392090 wait complete. 2024-03-28T18:05:58.882 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:05:58 smithi022 ceph-mon[31985]: from='client.? 172.21.15.22:0/712830237' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-03-28T18:05:58.882 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:05:58 smithi022 ceph-mon[31985]: from='client.? 172.21.15.22:0/460392090' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-03-28T18:05:58.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:05:58 smithi134 ceph-mon[35037]: from='client.? 172.21.15.22:0/712830237' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-03-28T18:05:58.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:05:58 smithi134 ceph-mon[35037]: from='client.? 172.21.15.22:0/460392090' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-03-28T18:05:58.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:05:58 smithi079 ceph-mon[35312]: from='client.? 172.21.15.22:0/712830237' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-03-28T18:05:58.933 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:05:58 smithi079 ceph-mon[35312]: from='client.? 172.21.15.22:0/460392090' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-03-28T18:05:59.135 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:59.135+0000 7f2e05c00700 1 -- 172.21.15.22:0/3144864966 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2e0010e7a0 msgr2=0x7f2e0010eb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:05:59.135 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:59.135+0000 7f2e05c00700 1 --2- 172.21.15.22:0/3144864966 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2e0010e7a0 0x7f2e0010eb80 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f2df0009990 tx=0x7f2df0009ca0 comp rx=0 tx=0).stop 2024-03-28T18:05:59.135 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:59.135+0000 7f2e05c00700 1 -- 172.21.15.22:0/3144864966 shutdown_connections 2024-03-28T18:05:59.135 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:59.135+0000 7f2e05c00700 1 --2- 172.21.15.22:0/3144864966 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2e00109160 0x7f2e00115670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:05:59.135 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:59.135+0000 7f2e05c00700 1 --2- 172.21.15.22:0/3144864966 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2e0010e7a0 0x7f2e0010eb80 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:05:59.135 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:59.135+0000 7f2e05c00700 1 --2- 172.21.15.22:0/3144864966 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f2e001087a0 0x7f2e00108c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:05:59.135 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:59.135+0000 7f2e05c00700 1 -- 172.21.15.22:0/3144864966 >> 172.21.15.22:0/3144864966 conn(0x7f2e00075960 msgr2=0x7f2e00077d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:05:59.135 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:59.135+0000 7f2e05c00700 1 -- 172.21.15.22:0/3144864966 shutdown_connections 2024-03-28T18:05:59.135 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:59.135+0000 7f2e05c00700 1 -- 172.21.15.22:0/3144864966 wait complete. 2024-03-28T18:05:59.137 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:59.136+0000 7f2e05c00700 1 Processor -- start 2024-03-28T18:05:59.137 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:59.136+0000 7f2e05c00700 1 -- start start 2024-03-28T18:05:59.137 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:59.136+0000 7f2e05c00700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f2e001087a0 0x7f2e0019eea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:05:59.137 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:59.136+0000 7f2e05c00700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2e00109160 0x7f2e0019f3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:05:59.137 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:59.136+0000 7f2e05c00700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2e0010e7a0 0x7f2e001a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:05:59.137 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:59.136+0000 7f2e05c00700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e00118af0 con 0x7f2e00109160 2024-03-28T18:05:59.137 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:59.136+0000 7f2e05c00700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e00118970 con 0x7f2e001087a0 2024-03-28T18:05:59.137 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:59.136+0000 7f2e05c00700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e00118c70 con 0x7f2e0010e7a0 2024-03-28T18:05:59.137 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:59.137+0000 7f2dfe000700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2e00109160 0x7f2e0019f3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:05:59.137 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:59.137+0000 7f2dfe000700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2e00109160 0x7f2e0019f3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:54272/0 (socket says 172.21.15.22:54272) 2024-03-28T18:05:59.137 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:59.137+0000 7f2dfe000700 1 -- 172.21.15.22:0/4019833682 learned_addr learned my addr 172.21.15.22:0/4019833682 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-28T18:05:59.137 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:59.137+0000 7f2dfea00700 1 --2- 172.21.15.22:0/4019833682 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f2e001087a0 0x7f2e0019eea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:05:59.137 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:59.137+0000 7f2dff400700 1 --2- 172.21.15.22:0/4019833682 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2e0010e7a0 0x7f2e001a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:05:59.138 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:59.137+0000 7f2dfe000700 1 -- 172.21.15.22:0/4019833682 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2e0010e7a0 msgr2=0x7f2e001a3770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:05:59.139 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:59.137+0000 7f2dfe000700 1 --2- 172.21.15.22:0/4019833682 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2e0010e7a0 0x7f2e001a3770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:05:59.139 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:59.137+0000 7f2dfe000700 1 -- 172.21.15.22:0/4019833682 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f2e001087a0 msgr2=0x7f2e0019eea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:05:59.139 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:59.137+0000 7f2dfe000700 1 --2- 172.21.15.22:0/4019833682 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f2e001087a0 0x7f2e0019eea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:05:59.139 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:59.137+0000 7f2dfe000700 1 -- 172.21.15.22:0/4019833682 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2e001a3eb0 con 0x7f2e00109160 2024-03-28T18:05:59.139 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:59.137+0000 7f2dfea00700 1 --2- 172.21.15.22:0/4019833682 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f2e001087a0 0x7f2e0019eea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-28T18:05:59.139 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:59.138+0000 7f2dfe000700 1 --2- 172.21.15.22:0/4019833682 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2e00109160 0x7f2e0019f3e0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f2dec00d950 tx=0x7f2dec00dd10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:05:59.139 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:59.138+0000 7f2df7400700 1 -- 172.21.15.22:0/4019833682 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2dec016070 con 0x7f2e00109160 2024-03-28T18:05:59.139 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:59.138+0000 7f2e05c00700 1 -- 172.21.15.22:0/4019833682 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2e001a41a0 con 0x7f2e00109160 2024-03-28T18:05:59.139 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:59.138+0000 7f2df7400700 1 -- 172.21.15.22:0/4019833682 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f2dec009b40 con 0x7f2e00109160 2024-03-28T18:05:59.140 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:59.138+0000 7f2df7400700 1 -- 172.21.15.22:0/4019833682 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2dec00f620 con 0x7f2e00109160 2024-03-28T18:05:59.140 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:59.138+0000 7f2e05c00700 1 -- 172.21.15.22:0/4019833682 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2e001aba20 con 0x7f2e00109160 2024-03-28T18:05:59.140 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:59.139+0000 7f2e05c00700 1 -- 172.21.15.22:0/4019833682 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2e00068e50 con 0x7f2e00109160 2024-03-28T18:05:59.141 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:59.142+0000 7f2df7400700 1 -- 172.21.15.22:0/4019833682 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 17) v1 ==== 137341+0+0 (secure 0 0 0) 0x7f2dec019070 con 0x7f2e00109160 2024-03-28T18:05:59.143 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:59.143+0000 7f2df7400700 1 --2- 172.21.15.22:0/4019833682 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7f2de40a29a0 0x7f2de40a4e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:05:59.143 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:59.143+0000 7f2df7400700 1 -- 172.21.15.22:0/4019833682 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 8153+0+0 (secure 0 0 0) 0x7f2dec0ce6e0 con 0x7f2e00109160 2024-03-28T18:05:59.143 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:59.143+0000 7f2dfea00700 1 --2- 172.21.15.22:0/4019833682 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7f2de40a29a0 0x7f2de40a4e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:05:59.143 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:59.144+0000 7f2dfea00700 1 --2- 172.21.15.22:0/4019833682 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7f2de40a29a0 0x7f2de40a4e60 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f2df0006010 tx=0x7f2df000b540 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:05:59.146 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:59.146+0000 7f2df7400700 1 -- 172.21.15.22:0/4019833682 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f2dec09b720 con 0x7f2e00109160 2024-03-28T18:05:59.383 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:59.384+0000 7f2e05c00700 1 -- 172.21.15.22:0/4019833682 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f2e001abc50 con 0x7f2e00109160 2024-03-28T18:05:59.385 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:59.384+0000 7f2df7400700 1 -- 172.21.15.22:0/4019833682 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f2dec09b020 con 0x7f2e00109160 2024-03-28T18:05:59.385 INFO:teuthology.orchestra.run.smithi022.stdout:55834574947 2024-03-28T18:05:59.387 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:59.387+0000 7f2e05c00700 1 -- 172.21.15.22:0/4019833682 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7f2de40a29a0 msgr2=0x7f2de40a4e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:05:59.388 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:59.387+0000 7f2e05c00700 1 --2- 172.21.15.22:0/4019833682 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7f2de40a29a0 0x7f2de40a4e60 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f2df0006010 tx=0x7f2df000b540 comp rx=0 tx=0).stop 2024-03-28T18:05:59.388 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:59.387+0000 7f2e05c00700 1 -- 172.21.15.22:0/4019833682 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2e00109160 msgr2=0x7f2e0019f3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:05:59.388 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:59.387+0000 7f2e05c00700 1 --2- 172.21.15.22:0/4019833682 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2e00109160 0x7f2e0019f3e0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f2dec00d950 tx=0x7f2dec00dd10 comp rx=0 tx=0).stop 2024-03-28T18:05:59.388 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:59.388+0000 7f2e05c00700 1 -- 172.21.15.22:0/4019833682 shutdown_connections 2024-03-28T18:05:59.388 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:59.388+0000 7f2e05c00700 1 --2- 172.21.15.22:0/4019833682 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7f2de40a29a0 0x7f2de40a4e60 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:05:59.388 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:59.388+0000 7f2e05c00700 1 --2- 172.21.15.22:0/4019833682 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2e0010e7a0 0x7f2e001a3770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:05:59.388 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:59.388+0000 7f2e05c00700 1 --2- 172.21.15.22:0/4019833682 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f2e001087a0 0x7f2e0019eea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:05:59.388 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:59.388+0000 7f2e05c00700 1 --2- 172.21.15.22:0/4019833682 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2e00109160 0x7f2e0019f3e0 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:05:59.388 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:59.388+0000 7f2e05c00700 1 -- 172.21.15.22:0/4019833682 >> 172.21.15.22:0/4019833682 conn(0x7f2e00075960 msgr2=0x7f2e00075e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:05:59.388 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:59.388+0000 7f2e05c00700 1 -- 172.21.15.22:0/4019833682 shutdown_connections 2024-03-28T18:05:59.388 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:59.388+0000 7f2e05c00700 1 -- 172.21.15.22:0/4019833682 wait complete. 2024-03-28T18:05:59.765 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:05:59 smithi022 ceph-mon[31985]: pgmap v406: 97 pgs: 97 active+clean; 642 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-03-28T18:05:59.765 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:05:59 smithi022 ceph-mon[31985]: from='client.? 172.21.15.22:0/4019833682' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-03-28T18:05:59.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:05:59 smithi134 ceph-mon[35037]: pgmap v406: 97 pgs: 97 active+clean; 642 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-03-28T18:05:59.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:05:59 smithi134 ceph-mon[35037]: from='client.? 172.21.15.22:0/4019833682' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-03-28T18:05:59.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:05:59 smithi079 ceph-mon[35312]: pgmap v406: 97 pgs: 97 active+clean; 642 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-03-28T18:05:59.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:05:59 smithi079 ceph-mon[35312]: from='client.? 172.21.15.22:0/4019833682' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-03-28T18:06:00.458 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:58.119+0000 7f043d600700 1 -- 172.21.15.22:0/497651127 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f04381023d0 msgr2=0x7f04381027b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:06:00.458 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:58.119+0000 7f043d600700 1 --2- 172.21.15.22:0/497651127 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f04381023d0 0x7f04381027b0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f04280099c0 tx=0x7f0428009cd0 comp rx=0 tx=0).stop 2024-03-28T18:06:00.459 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:58.119+0000 7f043d600700 1 -- 172.21.15.22:0/497651127 shutdown_connections 2024-03-28T18:06:00.459 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:58.119+0000 7f043d600700 1 --2- 172.21.15.22:0/497651127 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f04381074b0 0x7f043810f080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:06:00.459 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:58.119+0000 7f043d600700 1 --2- 172.21.15.22:0/497651127 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f04381023d0 0x7f04381027b0 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:06:00.459 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:58.119+0000 7f043d600700 1 --2- 172.21.15.22:0/497651127 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0438102d80 0x7f0438106dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:06:00.459 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:58.119+0000 7f043d600700 1 -- 172.21.15.22:0/497651127 >> 172.21.15.22:0/497651127 conn(0x7f04380fdc40 msgr2=0x7f0438100060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:06:00.459 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:58.119+0000 7f043d600700 1 -- 172.21.15.22:0/497651127 shutdown_connections 2024-03-28T18:06:00.459 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:58.119+0000 7f043d600700 1 -- 172.21.15.22:0/497651127 wait complete. 2024-03-28T18:06:00.459 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:58.121+0000 7f043d600700 1 Processor -- start 2024-03-28T18:06:00.459 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:58.121+0000 7f043d600700 1 -- start start 2024-03-28T18:06:00.459 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:58.122+0000 7f043d600700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f04381023d0 0x7f04381a07a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:06:00.459 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:58.122+0000 7f043d600700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0438102d80 0x7f04381a0ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:06:00.459 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:58.122+0000 7f043d600700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f04381074b0 0x7f043819aa00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:06:00.459 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:58.122+0000 7f043d600700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04381124d0 con 0x7f04381074b0 2024-03-28T18:06:00.460 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:58.122+0000 7f043d600700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0438112350 con 0x7f04381023d0 2024-03-28T18:06:00.460 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:58.122+0000 7f043d600700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0438112650 con 0x7f0438102d80 2024-03-28T18:06:00.460 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:58.122+0000 7f0436a00700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f04381023d0 0x7f04381a07a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:06:00.460 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:58.122+0000 7f0436a00700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f04381023d0 0x7f04381a07a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.22:41748/0 (socket says 172.21.15.22:41748) 2024-03-28T18:06:00.460 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:58.122+0000 7f0436a00700 1 -- 172.21.15.22:0/712830237 learned_addr learned my addr 172.21.15.22:0/712830237 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-28T18:06:00.460 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:58.122+0000 7f0437400700 1 --2- 172.21.15.22:0/712830237 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f04381074b0 0x7f043819aa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:06:00.460 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:58.123+0000 7f0436000700 1 --2- 172.21.15.22:0/712830237 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0438102d80 0x7f04381a0ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:06:00.460 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:58.123+0000 7f0437400700 1 -- 172.21.15.22:0/712830237 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0438102d80 msgr2=0x7f04381a0ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:06:00.460 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:58.123+0000 7f0437400700 1 --2- 172.21.15.22:0/712830237 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0438102d80 0x7f04381a0ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:06:00.460 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:58.123+0000 7f0437400700 1 -- 172.21.15.22:0/712830237 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f04381023d0 msgr2=0x7f04381a07a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:06:00.460 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:58.123+0000 7f0437400700 1 --2- 172.21.15.22:0/712830237 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f04381023d0 0x7f04381a07a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:06:00.460 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:58.123+0000 7f0437400700 1 -- 172.21.15.22:0/712830237 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f043819b160 con 0x7f04381074b0 2024-03-28T18:06:00.460 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:58.125+0000 7f0437400700 1 --2- 172.21.15.22:0/712830237 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f04381074b0 0x7f043819aa00 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f043000efc0 tx=0x7f043000c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:06:00.461 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:58.125+0000 7f042f400700 1 -- 172.21.15.22:0/712830237 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04300092e0 con 0x7f04381074b0 2024-03-28T18:06:00.461 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:58.125+0000 7f042f400700 1 -- 172.21.15.22:0/712830237 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f0430004bb0 con 0x7f04381074b0 2024-03-28T18:06:00.461 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:58.125+0000 7f042f400700 1 -- 172.21.15.22:0/712830237 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0430004280 con 0x7f04381074b0 2024-03-28T18:06:00.461 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:58.125+0000 7f043d600700 1 -- 172.21.15.22:0/712830237 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f043819b3f0 con 0x7f04381074b0 2024-03-28T18:06:00.461 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:58.126+0000 7f043d600700 1 -- 172.21.15.22:0/712830237 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f04381a7670 con 0x7f04381074b0 2024-03-28T18:06:00.461 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:58.127+0000 7f043d600700 1 -- 172.21.15.22:0/712830237 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0438068e50 con 0x7f04381074b0 2024-03-28T18:06:00.461 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:58.131+0000 7f042f400700 1 -- 172.21.15.22:0/712830237 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 17) v1 ==== 137341+0+0 (secure 0 0 0) 0x7f043001b030 con 0x7f04381074b0 2024-03-28T18:06:00.461 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:58.133+0000 7f042f400700 1 --2- 172.21.15.22:0/712830237 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7f041c0a29a0 0x7f041c0a4e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:06:00.461 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:58.134+0000 7f042f400700 1 -- 172.21.15.22:0/712830237 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 8153+0+0 (secure 0 0 0) 0x7f04300cef80 con 0x7f04381074b0 2024-03-28T18:06:00.461 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:58.134+0000 7f0436a00700 1 --2- 172.21.15.22:0/712830237 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7f041c0a29a0 0x7f041c0a4e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:06:00.461 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:58.135+0000 7f0436a00700 1 --2- 172.21.15.22:0/712830237 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7f041c0a29a0 0x7f041c0a4e60 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f0428009990 tx=0x7f0428005a90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:06:00.461 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:58.136+0000 7f042f400700 1 -- 172.21.15.22:0/712830237 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f0430022020 con 0x7f04381074b0 2024-03-28T18:06:00.461 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:58.375+0000 7f043d600700 1 -- 172.21.15.22:0/712830237 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) v1 -- 0x7f04381a78a0 con 0x7f04381074b0 2024-03-28T18:06:00.461 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:58.376+0000 7f042f400700 1 -- 172.21.15.22:0/712830237 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f0430010040 con 0x7f04381074b0 2024-03-28T18:06:00.462 INFO:teuthology.orchestra.run.smithi022.stdout:146028888142 2024-03-28T18:06:00.462 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:58.379+0000 7f043d600700 1 -- 172.21.15.22:0/712830237 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7f041c0a29a0 msgr2=0x7f041c0a4e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:06:00.462 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:58.379+0000 7f043d600700 1 --2- 172.21.15.22:0/712830237 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7f041c0a29a0 0x7f041c0a4e60 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f0428009990 tx=0x7f0428005a90 comp rx=0 tx=0).stop 2024-03-28T18:06:00.462 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:58.379+0000 7f043d600700 1 -- 172.21.15.22:0/712830237 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f04381074b0 msgr2=0x7f043819aa00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:06:00.462 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:58.379+0000 7f043d600700 1 --2- 172.21.15.22:0/712830237 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f04381074b0 0x7f043819aa00 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f043000efc0 tx=0x7f043000c3a0 comp rx=0 tx=0).stop 2024-03-28T18:06:00.462 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:58.379+0000 7f043d600700 1 -- 172.21.15.22:0/712830237 shutdown_connections 2024-03-28T18:06:00.462 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:58.379+0000 7f043d600700 1 --2- 172.21.15.22:0/712830237 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f04381074b0 0x7f043819aa00 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:06:00.462 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:58.379+0000 7f043d600700 1 --2- 172.21.15.22:0/712830237 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f04381023d0 0x7f04381a07a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:06:00.463 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:58.379+0000 7f043d600700 1 --2- 172.21.15.22:0/712830237 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7f041c0a29a0 0x7f041c0a4e60 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:06:00.463 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:58.379+0000 7f043d600700 1 --2- 172.21.15.22:0/712830237 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0438102d80 0x7f04381a0ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:06:00.463 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:58.379+0000 7f043d600700 1 -- 172.21.15.22:0/712830237 >> 172.21.15.22:0/712830237 conn(0x7f04380fdc40 msgr2=0x7f04380fefa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:06:00.463 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:58.379+0000 7f043d600700 1 -- 172.21.15.22:0/712830237 shutdown_connections 2024-03-28T18:06:00.463 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:05:58.379+0000 7f043d600700 1 -- 172.21.15.22:0/712830237 wait complete. 2024-03-28T18:06:01.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:06:01 smithi022 ceph-mon[31985]: pgmap v407: 97 pgs: 97 active+clean; 642 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-03-28T18:06:01.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:06:01 smithi022 ceph-mon[31985]: from='client.? 172.21.15.22:0/999778315' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-03-28T18:06:01.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:06:01 smithi134 ceph-mon[35037]: pgmap v407: 97 pgs: 97 active+clean; 642 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-03-28T18:06:01.925 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:06:01 smithi134 ceph-mon[35037]: from='client.? 172.21.15.22:0/999778315' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-03-28T18:06:01.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:06:01 smithi079 ceph-mon[35312]: pgmap v407: 97 pgs: 97 active+clean; 642 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-03-28T18:06:01.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:06:01 smithi079 ceph-mon[35312]: from='client.? 172.21.15.22:0/999778315' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-03-28T18:06:02.269 INFO:tasks.cephadm.ceph_manager.ceph:need seq 146028888138 got 146028888142 for osd.4 2024-03-28T18:06:02.269 DEBUG:teuthology.parallel:result is None 2024-03-28T18:06:02.572 INFO:tasks.cephadm.ceph_manager.ceph:need seq 266287972396 got 266287972402 for osd.8 2024-03-28T18:06:02.572 DEBUG:teuthology.parallel:result is None 2024-03-28T18:06:02.614 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:00.453+0000 7f25c6000700 1 -- 172.21.15.22:0/338213612 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f25c0069e40 msgr2=0x7f25c006a2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:06:02.615 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:00.453+0000 7f25c6000700 1 --2- 172.21.15.22:0/338213612 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f25c0069e40 0x7f25c006a2a0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f25ac0099c0 tx=0x7f25ac009cd0 comp rx=0 tx=0).stop 2024-03-28T18:06:02.615 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:00.453+0000 7f25c6000700 1 -- 172.21.15.22:0/338213612 shutdown_connections 2024-03-28T18:06:02.615 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:00.453+0000 7f25c6000700 1 --2- 172.21.15.22:0/338213612 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f25c006a7e0 0x7f25c010f020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:06:02.615 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:00.453+0000 7f25c6000700 1 --2- 172.21.15.22:0/338213612 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f25c01081e0 0x7f25c01085c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:06:02.615 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:00.453+0000 7f25c6000700 1 --2- 172.21.15.22:0/338213612 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f25c0069e40 0x7f25c006a2a0 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:06:02.615 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:00.453+0000 7f25c6000700 1 -- 172.21.15.22:0/338213612 >> 172.21.15.22:0/338213612 conn(0x7f25c0077fc0 msgr2=0x7f25c00783d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:06:02.615 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:00.454+0000 7f25c6000700 1 -- 172.21.15.22:0/338213612 shutdown_connections 2024-03-28T18:06:02.615 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:00.454+0000 7f25c6000700 1 -- 172.21.15.22:0/338213612 wait complete. 2024-03-28T18:06:02.615 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:00.455+0000 7f25c6000700 1 Processor -- start 2024-03-28T18:06:02.615 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:00.455+0000 7f25c6000700 1 -- start start 2024-03-28T18:06:02.615 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:00.455+0000 7f25c6000700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f25c0069e40 0x7f25c01a0840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:06:02.616 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:00.455+0000 7f25c6000700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f25c006a7e0 0x7f25c01a0d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:06:02.616 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:00.455+0000 7f25c6000700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f25c01081e0 0x7f25c019aa50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:06:02.616 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:00.455+0000 7f25c6000700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25c0112410 con 0x7f25c006a7e0 2024-03-28T18:06:02.616 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:00.455+0000 7f25c6000700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25c0112290 con 0x7f25c01081e0 2024-03-28T18:06:02.616 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:00.455+0000 7f25c6000700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25c0112590 con 0x7f25c0069e40 2024-03-28T18:06:02.616 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:00.455+0000 7f25bea00700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f25c006a7e0 0x7f25c01a0d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:06:02.616 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:00.455+0000 7f25bea00700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f25c006a7e0 0x7f25c01a0d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:54282/0 (socket says 172.21.15.22:54282) 2024-03-28T18:06:02.616 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:00.456+0000 7f25bea00700 1 -- 172.21.15.22:0/999778315 learned_addr learned my addr 172.21.15.22:0/999778315 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-28T18:06:02.616 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:00.456+0000 7f25bf400700 1 --2- 172.21.15.22:0/999778315 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f25c0069e40 0x7f25c01a0840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:06:02.616 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:00.456+0000 7f25bfe00700 1 --2- 172.21.15.22:0/999778315 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f25c01081e0 0x7f25c019aa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:06:02.616 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:00.456+0000 7f25bea00700 1 -- 172.21.15.22:0/999778315 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f25c0069e40 msgr2=0x7f25c01a0840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:06:02.616 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:00.456+0000 7f25bea00700 1 --2- 172.21.15.22:0/999778315 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f25c0069e40 0x7f25c01a0840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:06:02.616 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:00.456+0000 7f25bea00700 1 -- 172.21.15.22:0/999778315 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f25c01081e0 msgr2=0x7f25c019aa50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:06:02.617 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:00.456+0000 7f25bea00700 1 --2- 172.21.15.22:0/999778315 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f25c01081e0 0x7f25c019aa50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:06:02.617 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:00.456+0000 7f25bea00700 1 -- 172.21.15.22:0/999778315 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f25c019b1b0 con 0x7f25c006a7e0 2024-03-28T18:06:02.617 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:00.456+0000 7f25bf400700 1 --2- 172.21.15.22:0/999778315 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f25c0069e40 0x7f25c01a0840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-28T18:06:02.617 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:00.456+0000 7f25bea00700 1 --2- 172.21.15.22:0/999778315 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f25c006a7e0 0x7f25c01a0d80 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f25ac006010 tx=0x7f25ac005e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:06:02.617 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:00.457+0000 7f25b7e00700 1 -- 172.21.15.22:0/999778315 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25ac027070 con 0x7f25c006a7e0 2024-03-28T18:06:02.617 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:00.457+0000 7f25b7e00700 1 -- 172.21.15.22:0/999778315 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f25ac0040b0 con 0x7f25c006a7e0 2024-03-28T18:06:02.617 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:00.457+0000 7f25c6000700 1 -- 172.21.15.22:0/999778315 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f25c019b440 con 0x7f25c006a7e0 2024-03-28T18:06:02.617 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:00.457+0000 7f25b7e00700 1 -- 172.21.15.22:0/999778315 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25ac022df0 con 0x7f25c006a7e0 2024-03-28T18:06:02.617 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:00.457+0000 7f25c6000700 1 -- 172.21.15.22:0/999778315 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f25c01a7710 con 0x7f25c006a7e0 2024-03-28T18:06:02.617 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:00.458+0000 7f25c6000700 1 -- 172.21.15.22:0/999778315 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f25c0103890 con 0x7f25c006a7e0 2024-03-28T18:06:02.617 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:00.461+0000 7f25b7e00700 1 -- 172.21.15.22:0/999778315 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 17) v1 ==== 137341+0+0 (secure 0 0 0) 0x7f25ac02a070 con 0x7f25c006a7e0 2024-03-28T18:06:02.617 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:00.462+0000 7f25b7e00700 1 --2- 172.21.15.22:0/999778315 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7f25a40a2920 0x7f25a40a4de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:06:02.617 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:00.462+0000 7f25b7e00700 1 -- 172.21.15.22:0/999778315 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 8153+0+0 (secure 0 0 0) 0x7f25ac0cfaf0 con 0x7f25c006a7e0 2024-03-28T18:06:02.617 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:00.462+0000 7f25bf400700 1 --2- 172.21.15.22:0/999778315 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7f25a40a2920 0x7f25a40a4de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:06:02.617 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:00.463+0000 7f25bf400700 1 --2- 172.21.15.22:0/999778315 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7f25a40a2920 0x7f25a40a4de0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f25c00ff710 tx=0x7f25b0005d80 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:06:02.618 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:00.465+0000 7f25b7e00700 1 -- 172.21.15.22:0/999778315 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f25ac013070 con 0x7f25c006a7e0 2024-03-28T18:06:02.618 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:00.724+0000 7f25c6000700 1 -- 172.21.15.22:0/999778315 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 11} v 0) v1 -- 0x7f25c0068e50 con 0x7f25c006a7e0 2024-03-28T18:06:02.618 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:00.725+0000 7f25b7e00700 1 -- 172.21.15.22:0/999778315 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 11}]=0 v0) v1 ==== 75+0+13 (secure 0 0 0) 0x7f25c0068e50 con 0x7f25c006a7e0 2024-03-28T18:06:02.618 INFO:teuthology.orchestra.run.smithi022.stdout:365072220191 2024-03-28T18:06:02.618 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:00.728+0000 7f25c6000700 1 -- 172.21.15.22:0/999778315 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7f25a40a2920 msgr2=0x7f25a40a4de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:06:02.618 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:00.728+0000 7f25c6000700 1 --2- 172.21.15.22:0/999778315 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7f25a40a2920 0x7f25a40a4de0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f25c00ff710 tx=0x7f25b0005d80 comp rx=0 tx=0).stop 2024-03-28T18:06:02.618 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:00.728+0000 7f25c6000700 1 -- 172.21.15.22:0/999778315 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f25c006a7e0 msgr2=0x7f25c01a0d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:06:02.619 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:00.728+0000 7f25c6000700 1 --2- 172.21.15.22:0/999778315 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f25c006a7e0 0x7f25c01a0d80 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f25ac006010 tx=0x7f25ac005e80 comp rx=0 tx=0).stop 2024-03-28T18:06:02.619 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:00.728+0000 7f25c6000700 1 -- 172.21.15.22:0/999778315 shutdown_connections 2024-03-28T18:06:02.619 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:00.728+0000 7f25c6000700 1 --2- 172.21.15.22:0/999778315 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f25c01081e0 0x7f25c019aa50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:06:02.619 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:00.728+0000 7f25c6000700 1 --2- 172.21.15.22:0/999778315 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7f25a40a2920 0x7f25a40a4de0 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:06:02.619 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:00.728+0000 7f25c6000700 1 --2- 172.21.15.22:0/999778315 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f25c0069e40 0x7f25c01a0840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:06:02.619 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:00.728+0000 7f25c6000700 1 --2- 172.21.15.22:0/999778315 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f25c006a7e0 0x7f25c01a0d80 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:06:02.619 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:00.728+0000 7f25c6000700 1 -- 172.21.15.22:0/999778315 >> 172.21.15.22:0/999778315 conn(0x7f25c0077fc0 msgr2=0x7f25c00ffb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:06:02.619 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:00.728+0000 7f25c6000700 1 -- 172.21.15.22:0/999778315 shutdown_connections 2024-03-28T18:06:02.619 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:00.729+0000 7f25c6000700 1 -- 172.21.15.22:0/999778315 wait complete. 2024-03-28T18:06:03.343 INFO:tasks.cephadm.ceph_manager.ceph:need seq 365072220187 got 365072220191 for osd.11 2024-03-28T18:06:03.343 DEBUG:teuthology.parallel:result is None 2024-03-28T18:06:03.833 INFO:tasks.cephadm.ceph_manager.ceph:need seq 300647710758 got 300647710763 for osd.9 2024-03-28T18:06:03.833 DEBUG:teuthology.parallel:result is None 2024-03-28T18:06:03.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:06:03 smithi134 ceph-mon[35037]: pgmap v408: 97 pgs: 97 active+clean; 642 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-03-28T18:06:03.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:06:03 smithi079 ceph-mon[35312]: pgmap v408: 97 pgs: 97 active+clean; 642 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-03-28T18:06:03.978 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:06:03 smithi022 ceph-mon[31985]: pgmap v408: 97 pgs: 97 active+clean; 642 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-03-28T18:06:04.384 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/fd220748-ed2b-11ee-95d0-87774f69a715/mon.a/config 2024-03-28T18:06:04.385 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/fd220748-ed2b-11ee-95d0-87774f69a715/mon.a/config 2024-03-28T18:06:04.418 INFO:tasks.cephadm.ceph_manager.ceph:need seq 55834574943 got 55834574947 for osd.1 2024-03-28T18:06:04.418 DEBUG:teuthology.parallel:result is None 2024-03-28T18:06:05.063 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:04.995+0000 7f953e400700 1 -- 172.21.15.22:0/1344689284 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9538102c80 msgr2=0x7f9538106cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:06:05.063 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:04.995+0000 7f953e400700 1 --2- 172.21.15.22:0/1344689284 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9538102c80 0x7f9538106cd0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f95240099c0 tx=0x7f9524009cd0 comp rx=0 tx=0).stop 2024-03-28T18:06:05.063 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:04.996+0000 7f953e400700 1 -- 172.21.15.22:0/1344689284 shutdown_connections 2024-03-28T18:06:05.063 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:04.996+0000 7f953e400700 1 --2- 172.21.15.22:0/1344689284 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f95381073b0 0x7f953810ef80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:06:05.063 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:04.996+0000 7f953e400700 1 --2- 172.21.15.22:0/1344689284 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f95381022d0 0x7f95381026b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:06:05.063 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:04.996+0000 7f953e400700 1 --2- 172.21.15.22:0/1344689284 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9538102c80 0x7f9538106cd0 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:06:05.063 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:04.996+0000 7f953e400700 1 -- 172.21.15.22:0/1344689284 >> 172.21.15.22:0/1344689284 conn(0x7f95380fdb60 msgr2=0x7f95380fff80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:06:05.063 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:04.996+0000 7f953e400700 1 -- 172.21.15.22:0/1344689284 shutdown_connections 2024-03-28T18:06:05.063 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:04.996+0000 7f953e400700 1 -- 172.21.15.22:0/1344689284 wait complete. 2024-03-28T18:06:05.064 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:04.997+0000 7f953e400700 1 Processor -- start 2024-03-28T18:06:05.064 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:04.997+0000 7f953e400700 1 -- start start 2024-03-28T18:06:05.064 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:04.997+0000 7f953e400700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f95381022d0 0x7f95381a0660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:06:05.064 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:04.997+0000 7f953e400700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9538102c80 0x7f95381a0ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:06:05.064 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:04.997+0000 7f953e400700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f95381073b0 0x7f953819a7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:06:05.064 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:04.997+0000 7f953e400700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95381123b0 con 0x7f95381073b0 2024-03-28T18:06:05.064 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:04.997+0000 7f953e400700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9538112230 con 0x7f95381022d0 2024-03-28T18:06:05.064 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:04.997+0000 7f953e400700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9538112530 con 0x7f9538102c80 2024-03-28T18:06:05.064 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:04.998+0000 7f953da00700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f95381073b0 0x7f953819a7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:06:05.064 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:04.998+0000 7f953d000700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f95381022d0 0x7f95381a0660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:06:05.064 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:04.998+0000 7f953da00700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f95381073b0 0x7f953819a7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:54296/0 (socket says 172.21.15.22:54296) 2024-03-28T18:06:05.064 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:04.998+0000 7f953da00700 1 -- 172.21.15.22:0/2433045976 learned_addr learned my addr 172.21.15.22:0/2433045976 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-28T18:06:05.064 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:04.998+0000 7f952fe00700 1 --2- 172.21.15.22:0/2433045976 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9538102c80 0x7f95381a0ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:06:05.064 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:04.998+0000 7f953da00700 1 -- 172.21.15.22:0/2433045976 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9538102c80 msgr2=0x7f95381a0ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:06:05.065 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:04.998+0000 7f953da00700 1 --2- 172.21.15.22:0/2433045976 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9538102c80 0x7f95381a0ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:06:05.065 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:04.998+0000 7f953da00700 1 -- 172.21.15.22:0/2433045976 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f95381022d0 msgr2=0x7f95381a0660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:06:05.065 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:04.998+0000 7f953da00700 1 --2- 172.21.15.22:0/2433045976 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f95381022d0 0x7f95381a0660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:06:05.065 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:04.998+0000 7f953da00700 1 -- 172.21.15.22:0/2433045976 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f953819afc0 con 0x7f95381073b0 2024-03-28T18:06:05.065 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:04.998+0000 7f953da00700 1 --2- 172.21.15.22:0/2433045976 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f95381073b0 0x7f953819a7e0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f9530007d70 tx=0x7f953000a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:06:05.065 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:04.999+0000 7f9536000700 1 -- 172.21.15.22:0/2433045976 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9530016020 con 0x7f95381073b0 2024-03-28T18:06:05.065 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:04.999+0000 7f9536000700 1 -- 172.21.15.22:0/2433045976 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f953000ad40 con 0x7f95381073b0 2024-03-28T18:06:05.065 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:04.999+0000 7f953e400700 1 -- 172.21.15.22:0/2433045976 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f953819b2b0 con 0x7f95381073b0 2024-03-28T18:06:05.065 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:04.999+0000 7f9536000700 1 -- 172.21.15.22:0/2433045976 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95300115e0 con 0x7f95381073b0 2024-03-28T18:06:05.065 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:04.999+0000 7f953e400700 1 -- 172.21.15.22:0/2433045976 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f95381a74c0 con 0x7f95381073b0 2024-03-28T18:06:05.065 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:05.000+0000 7f953e400700 1 -- 172.21.15.22:0/2433045976 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f95381037b0 con 0x7f95381073b0 2024-03-28T18:06:05.065 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:05.003+0000 7f9536000700 1 -- 172.21.15.22:0/2433045976 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 17) v1 ==== 137341+0+0 (secure 0 0 0) 0x7f9530019020 con 0x7f95381073b0 2024-03-28T18:06:05.065 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:05.004+0000 7f9536000700 1 --2- 172.21.15.22:0/2433045976 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7f951c0a2920 0x7f951c0a4de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:06:05.065 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:05.004+0000 7f9536000700 1 -- 172.21.15.22:0/2433045976 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 8153+0+0 (secure 0 0 0) 0x7f95300ceaa0 con 0x7f95381073b0 2024-03-28T18:06:05.065 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:05.004+0000 7f953d000700 1 --2- 172.21.15.22:0/2433045976 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7f951c0a2920 0x7f951c0a4de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:06:05.066 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:05.005+0000 7f953d000700 1 --2- 172.21.15.22:0/2433045976 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7f951c0a2920 0x7f951c0a4de0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f9528009c60 tx=0x7f9528009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:06:05.066 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:05.009+0000 7f9536000700 1 -- 172.21.15.22:0/2433045976 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f9530020020 con 0x7f95381073b0 2024-03-28T18:06:05.256 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:05.256+0000 7f953e400700 1 -- 172.21.15.22:0/2433045976 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) v1 -- 0x7f95381a76f0 con 0x7f95381073b0 2024-03-28T18:06:05.257 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:05.257+0000 7f9536000700 1 -- 172.21.15.22:0/2433045976 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f953009baf0 con 0x7f95381073b0 2024-03-28T18:06:05.257 INFO:teuthology.orchestra.run.smithi022.stdout:171798691914 2024-03-28T18:06:05.260 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:05.260+0000 7f953e400700 1 -- 172.21.15.22:0/2433045976 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7f951c0a2920 msgr2=0x7f951c0a4de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:06:05.260 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:05.260+0000 7f953e400700 1 --2- 172.21.15.22:0/2433045976 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7f951c0a2920 0x7f951c0a4de0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f9528009c60 tx=0x7f9528009260 comp rx=0 tx=0).stop 2024-03-28T18:06:05.260 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:05.260+0000 7f953e400700 1 -- 172.21.15.22:0/2433045976 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f95381073b0 msgr2=0x7f953819a7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:06:05.260 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:05.260+0000 7f953e400700 1 --2- 172.21.15.22:0/2433045976 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f95381073b0 0x7f953819a7e0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f9530007d70 tx=0x7f953000a340 comp rx=0 tx=0).stop 2024-03-28T18:06:05.260 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:05.260+0000 7f953e400700 1 -- 172.21.15.22:0/2433045976 shutdown_connections 2024-03-28T18:06:05.260 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:05.260+0000 7f953e400700 1 --2- 172.21.15.22:0/2433045976 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f95381073b0 0x7f953819a7e0 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:06:05.260 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:05.260+0000 7f953e400700 1 --2- 172.21.15.22:0/2433045976 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f95381022d0 0x7f95381a0660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:06:05.260 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:05.260+0000 7f953e400700 1 --2- 172.21.15.22:0/2433045976 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7f951c0a2920 0x7f951c0a4de0 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:06:05.260 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:05.260+0000 7f953e400700 1 --2- 172.21.15.22:0/2433045976 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9538102c80 0x7f95381a0ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:06:05.261 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:05.260+0000 7f953e400700 1 -- 172.21.15.22:0/2433045976 >> 172.21.15.22:0/2433045976 conn(0x7f95380fdb60 msgr2=0x7f953810d5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:06:05.261 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:05.260+0000 7f953e400700 1 -- 172.21.15.22:0/2433045976 shutdown_connections 2024-03-28T18:06:05.261 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:05.260+0000 7f953e400700 1 -- 172.21.15.22:0/2433045976 wait complete. 2024-03-28T18:06:05.821 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:06:05 smithi022 ceph-mon[31985]: pgmap v409: 97 pgs: 97 active+clean; 642 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-28T18:06:05.821 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:06:05 smithi022 ceph-mon[31985]: from='client.? 172.21.15.22:0/2433045976' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-03-28T18:06:06.080 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:06.066+0000 7f6388a00700 1 -- 172.21.15.22:0/59299603 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f63840743c0 msgr2=0x7f63840747a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:06:06.080 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:06.066+0000 7f6388a00700 1 --2- 172.21.15.22:0/59299603 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f63840743c0 0x7f63840747a0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f6374009990 tx=0x7f6374009ca0 comp rx=0 tx=0).stop 2024-03-28T18:06:06.080 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:06.066+0000 7f6388a00700 1 -- 172.21.15.22:0/59299603 shutdown_connections 2024-03-28T18:06:06.080 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:06.066+0000 7f6388a00700 1 --2- 172.21.15.22:0/59299603 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f638410eb90 0x7f6384110f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:06:06.080 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:06.066+0000 7f6388a00700 1 --2- 172.21.15.22:0/59299603 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f63840743c0 0x7f63840747a0 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:06:06.080 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:06.066+0000 7f6388a00700 1 --2- 172.21.15.22:0/59299603 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6384074ce0 0x7f638410e650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:06:06.080 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:06.066+0000 7f6388a00700 1 -- 172.21.15.22:0/59299603 >> 172.21.15.22:0/59299603 conn(0x7f63840792f0 msgr2=0x7f6384079700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:06:06.080 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:06.066+0000 7f6388a00700 1 -- 172.21.15.22:0/59299603 shutdown_connections 2024-03-28T18:06:06.080 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:06.066+0000 7f6388a00700 1 -- 172.21.15.22:0/59299603 wait complete. 2024-03-28T18:06:06.080 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:06.067+0000 7f6388a00700 1 Processor -- start 2024-03-28T18:06:06.081 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:06.067+0000 7f6388a00700 1 -- start start 2024-03-28T18:06:06.081 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:06.068+0000 7f6388a00700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f63840743c0 0x7f63841a0860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:06:06.081 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:06.068+0000 7f6388a00700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6384074ce0 0x7f63841a0da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:06:06.081 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:06.068+0000 7f6388a00700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f638410eb90 0x7f638419aa20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:06:06.081 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:06.068+0000 7f6388a00700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63841147a0 con 0x7f6384074ce0 2024-03-28T18:06:06.081 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:06.068+0000 7f6388a00700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6384114620 con 0x7f63840743c0 2024-03-28T18:06:06.081 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:06.068+0000 7f6388a00700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6384114920 con 0x7f638410eb90 2024-03-28T18:06:06.081 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:06.068+0000 7f6381400700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6384074ce0 0x7f63841a0da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:06:06.081 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:06.068+0000 7f6381e00700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f63840743c0 0x7f63841a0860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:06:06.081 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:06.068+0000 7f6381e00700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f63840743c0 0x7f63841a0860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.22:41804/0 (socket says 172.21.15.22:41804) 2024-03-28T18:06:06.081 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:06.068+0000 7f6381400700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6384074ce0 0x7f63841a0da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:54324/0 (socket says 172.21.15.22:54324) 2024-03-28T18:06:06.081 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:06.068+0000 7f6381e00700 1 -- 172.21.15.22:0/1735131089 learned_addr learned my addr 172.21.15.22:0/1735131089 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-28T18:06:06.081 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:06.068+0000 7f6382800700 1 --2- 172.21.15.22:0/1735131089 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f638410eb90 0x7f638419aa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:06:06.081 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:06.069+0000 7f6381e00700 1 -- 172.21.15.22:0/1735131089 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f638410eb90 msgr2=0x7f638419aa20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:06:06.082 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:06.069+0000 7f6381e00700 1 --2- 172.21.15.22:0/1735131089 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f638410eb90 0x7f638419aa20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:06:06.082 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:06.069+0000 7f6381e00700 1 -- 172.21.15.22:0/1735131089 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6384074ce0 msgr2=0x7f63841a0da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:06:06.082 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:06.069+0000 7f6381e00700 1 --2- 172.21.15.22:0/1735131089 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6384074ce0 0x7f63841a0da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:06:06.082 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:06.069+0000 7f6381e00700 1 -- 172.21.15.22:0/1735131089 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f638419b180 con 0x7f63840743c0 2024-03-28T18:06:06.082 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:06.069+0000 7f6381400700 1 --2- 172.21.15.22:0/1735131089 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6384074ce0 0x7f63841a0da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-28T18:06:06.082 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:06.069+0000 7f6381e00700 1 --2- 172.21.15.22:0/1735131089 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f63840743c0 0x7f63841a0860 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f6374000c00 tx=0x7f63740057c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:06:06.082 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:06.069+0000 7f637aa00700 1 -- 172.21.15.22:0/1735131089 <== mon.2 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6374027070 con 0x7f63840743c0 2024-03-28T18:06:06.082 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:06.069+0000 7f6388a00700 1 -- 172.21.15.22:0/1735131089 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f638419b410 con 0x7f63840743c0 2024-03-28T18:06:06.082 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:06.069+0000 7f637aa00700 1 -- 172.21.15.22:0/1735131089 <== mon.2 v2:172.21.15.79:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f6374022470 con 0x7f63840743c0 2024-03-28T18:06:06.082 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:06.069+0000 7f637aa00700 1 -- 172.21.15.22:0/1735131089 <== mon.2 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6374017650 con 0x7f63840743c0 2024-03-28T18:06:06.082 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:06.070+0000 7f6388a00700 1 -- 172.21.15.22:0/1735131089 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f63841a7730 con 0x7f63840743c0 2024-03-28T18:06:06.082 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:06.071+0000 7f6388a00700 1 -- 172.21.15.22:0/1735131089 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6384075af0 con 0x7f63840743c0 2024-03-28T18:06:06.082 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:06.073+0000 7f637aa00700 1 -- 172.21.15.22:0/1735131089 <== mon.2 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 17) v1 ==== 137341+0+0 (secure 0 0 0) 0x7f637402a070 con 0x7f63840743c0 2024-03-28T18:06:06.082 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:06.074+0000 7f637aa00700 1 --2- 172.21.15.22:0/1735131089 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7f63680a2920 0x7f63680a4de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:06:06.083 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:06.074+0000 7f637aa00700 1 -- 172.21.15.22:0/1735131089 <== mon.2 v2:172.21.15.79:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 8153+0+0 (secure 0 0 0) 0x7f63740cf9f0 con 0x7f63840743c0 2024-03-28T18:06:06.083 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:06.075+0000 7f6381400700 1 --2- 172.21.15.22:0/1735131089 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7f63680a2920 0x7f63680a4de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:06:06.083 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:06.075+0000 7f6381400700 1 --2- 172.21.15.22:0/1735131089 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7f63680a2920 0x7f63680a4de0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f63841107e0 tx=0x7f637000b3c0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:06:06.083 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:06.082+0000 7f637aa00700 1 -- 172.21.15.22:0/1735131089 <== mon.2 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f6374013070 con 0x7f63840743c0 2024-03-28T18:06:06.174 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:06:05 smithi134 ceph-mon[35037]: pgmap v409: 97 pgs: 97 active+clean; 642 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-28T18:06:06.175 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:06:05 smithi134 ceph-mon[35037]: from='client.? 172.21.15.22:0/2433045976' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-03-28T18:06:06.182 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:06:05 smithi079 ceph-mon[35312]: pgmap v409: 97 pgs: 97 active+clean; 642 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-28T18:06:06.182 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:06:05 smithi079 ceph-mon[35312]: from='client.? 172.21.15.22:0/2433045976' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-03-28T18:06:06.313 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:06.313+0000 7f6388a00700 1 -- 172.21.15.22:0/1735131089 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) v1 -- 0x7f6384068e50 con 0x7f63840743c0 2024-03-28T18:06:06.313 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:06.313+0000 7f637aa00700 1 -- 172.21.15.22:0/1735131089 <== mon.2 v2:172.21.15.79:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f63740095a0 con 0x7f63840743c0 2024-03-28T18:06:06.313 INFO:teuthology.orchestra.run.smithi022.stdout:206158430272 2024-03-28T18:06:06.316 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:06.316+0000 7f6388a00700 1 -- 172.21.15.22:0/1735131089 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7f63680a2920 msgr2=0x7f63680a4de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:06:06.316 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:06.316+0000 7f6388a00700 1 --2- 172.21.15.22:0/1735131089 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7f63680a2920 0x7f63680a4de0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f63841107e0 tx=0x7f637000b3c0 comp rx=0 tx=0).stop 2024-03-28T18:06:06.316 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:06.316+0000 7f6388a00700 1 -- 172.21.15.22:0/1735131089 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f63840743c0 msgr2=0x7f63841a0860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:06:06.316 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:06.316+0000 7f6388a00700 1 --2- 172.21.15.22:0/1735131089 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f63840743c0 0x7f63841a0860 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f6374000c00 tx=0x7f63740057c0 comp rx=0 tx=0).stop 2024-03-28T18:06:06.317 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:06.316+0000 7f6388a00700 1 -- 172.21.15.22:0/1735131089 shutdown_connections 2024-03-28T18:06:06.317 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:06.316+0000 7f6388a00700 1 --2- 172.21.15.22:0/1735131089 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7f63680a2920 0x7f63680a4de0 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:06:06.317 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:06.316+0000 7f6388a00700 1 --2- 172.21.15.22:0/1735131089 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f638410eb90 0x7f638419aa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:06:06.317 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:06.316+0000 7f6388a00700 1 --2- 172.21.15.22:0/1735131089 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f63840743c0 0x7f63841a0860 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:06:06.317 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:06.316+0000 7f6388a00700 1 --2- 172.21.15.22:0/1735131089 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6384074ce0 0x7f63841a0da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:06:06.317 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:06.316+0000 7f6388a00700 1 -- 172.21.15.22:0/1735131089 >> 172.21.15.22:0/1735131089 conn(0x7f63840792f0 msgr2=0x7f63840ff360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:06:06.317 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:06.317+0000 7f6388a00700 1 -- 172.21.15.22:0/1735131089 shutdown_connections 2024-03-28T18:06:06.317 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:06.317+0000 7f6388a00700 1 -- 172.21.15.22:0/1735131089 wait complete. 2024-03-28T18:06:07.032 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:06:06 smithi022 ceph-mon[31985]: from='client.? 172.21.15.22:0/1735131089' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-03-28T18:06:07.083 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:06:06 smithi079 ceph-mon[35312]: from='client.? 172.21.15.22:0/1735131089' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-03-28T18:06:07.174 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:06:06 smithi134 ceph-mon[35037]: from='client.? 172.21.15.22:0/1735131089' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-03-28T18:06:07.896 INFO:tasks.cephadm.ceph_manager.ceph:need seq 206158430266 got 206158430272 for osd.6 2024-03-28T18:06:07.897 DEBUG:teuthology.parallel:result is None 2024-03-28T18:06:07.956 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:06:07 smithi022 ceph-mon[31985]: pgmap v410: 97 pgs: 97 active+clean; 642 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2024-03-28T18:06:08.129 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:08.129+0000 7f04ee600700 1 -- 172.21.15.22:0/794653670 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f04e81070d0 msgr2=0x7f04e81074b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:06:08.129 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:08.129+0000 7f04ee600700 1 --2- 172.21.15.22:0/794653670 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f04e81070d0 0x7f04e81074b0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f04d8009990 tx=0x7f04d8009ca0 comp rx=0 tx=0).stop 2024-03-28T18:06:08.129 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:08.129+0000 7f04ee600700 1 -- 172.21.15.22:0/794653670 shutdown_connections 2024-03-28T18:06:08.129 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:08.129+0000 7f04ee600700 1 --2- 172.21.15.22:0/794653670 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f04e8101a30 0x7f04e810df10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:06:08.129 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:08.129+0000 7f04ee600700 1 --2- 172.21.15.22:0/794653670 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f04e81070d0 0x7f04e81074b0 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:06:08.130 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:08.129+0000 7f04ee600700 1 --2- 172.21.15.22:0/794653670 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f04e8101090 0x7f04e81014f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:06:08.130 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:08.129+0000 7f04ee600700 1 -- 172.21.15.22:0/794653670 >> 172.21.15.22:0/794653670 conn(0x7f04e8077fc0 msgr2=0x7f04e80783d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:06:08.130 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:08.129+0000 7f04ee600700 1 -- 172.21.15.22:0/794653670 shutdown_connections 2024-03-28T18:06:08.130 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:08.129+0000 7f04ee600700 1 -- 172.21.15.22:0/794653670 wait complete. 2024-03-28T18:06:08.131 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:08.130+0000 7f04ee600700 1 Processor -- start 2024-03-28T18:06:08.132 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:08.130+0000 7f04ee600700 1 -- start start 2024-03-28T18:06:08.132 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:08.131+0000 7f04ee600700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f04e8101090 0x7f04e81a07d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:06:08.132 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:08.131+0000 7f04ee600700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f04e8101a30 0x7f04e81a0d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:06:08.132 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:08.131+0000 7f04ee600700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f04e81070d0 0x7f04e819a990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:06:08.132 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:08.131+0000 7f04ee600700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04e8111350 con 0x7f04e8101090 2024-03-28T18:06:08.132 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:08.131+0000 7f04ee600700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04e81111d0 con 0x7f04e81070d0 2024-03-28T18:06:08.132 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:08.131+0000 7f04ee600700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04e81114d0 con 0x7f04e8101a30 2024-03-28T18:06:08.132 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:08.131+0000 7f04e7400700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f04e8101090 0x7f04e81a07d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:06:08.132 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:08.131+0000 7f04e7400700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f04e8101090 0x7f04e81a07d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:43328/0 (socket says 172.21.15.22:43328) 2024-03-28T18:06:08.132 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:08.131+0000 7f04e7400700 1 -- 172.21.15.22:0/3287682410 learned_addr learned my addr 172.21.15.22:0/3287682410 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-28T18:06:08.132 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:08.131+0000 7f04e7e00700 1 --2- 172.21.15.22:0/3287682410 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f04e81070d0 0x7f04e819a990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:06:08.132 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:08.131+0000 7f04e6a00700 1 --2- 172.21.15.22:0/3287682410 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f04e8101a30 0x7f04e81a0d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:06:08.132 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:08.132+0000 7f04e7400700 1 -- 172.21.15.22:0/3287682410 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f04e8101a30 msgr2=0x7f04e81a0d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:06:08.133 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:08.132+0000 7f04e7400700 1 --2- 172.21.15.22:0/3287682410 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f04e8101a30 0x7f04e81a0d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:06:08.133 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:08.132+0000 7f04e7400700 1 -- 172.21.15.22:0/3287682410 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f04e81070d0 msgr2=0x7f04e819a990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:06:08.133 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:08.132+0000 7f04e7400700 1 --2- 172.21.15.22:0/3287682410 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f04e81070d0 0x7f04e819a990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:06:08.133 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:08.132+0000 7f04e7400700 1 -- 172.21.15.22:0/3287682410 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f04e819b0f0 con 0x7f04e8101090 2024-03-28T18:06:08.133 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:08.132+0000 7f04e6a00700 1 --2- 172.21.15.22:0/3287682410 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f04e8101a30 0x7f04e81a0d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-28T18:06:08.133 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:08.132+0000 7f04e7400700 1 --2- 172.21.15.22:0/3287682410 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f04e8101090 0x7f04e81a07d0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f04d8000c00 tx=0x7f04d80057c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:06:08.134 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:08.132+0000 7f04dfe00700 1 -- 172.21.15.22:0/3287682410 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04d8027070 con 0x7f04e8101090 2024-03-28T18:06:08.134 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:08.132+0000 7f04dfe00700 1 -- 172.21.15.22:0/3287682410 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f04d8022470 con 0x7f04e8101090 2024-03-28T18:06:08.134 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:08.132+0000 7f04ee600700 1 -- 172.21.15.22:0/3287682410 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f04e819b380 con 0x7f04e8101090 2024-03-28T18:06:08.134 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:08.132+0000 7f04dfe00700 1 -- 172.21.15.22:0/3287682410 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04d8017650 con 0x7f04e8101090 2024-03-28T18:06:08.134 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:08.133+0000 7f04ee600700 1 -- 172.21.15.22:0/3287682410 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f04e81a76a0 con 0x7f04e8101090 2024-03-28T18:06:08.134 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:08.134+0000 7f04ee600700 1 -- 172.21.15.22:0/3287682410 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f04e8068e50 con 0x7f04e8101090 2024-03-28T18:06:08.135 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:08.136+0000 7f04dfe00700 1 -- 172.21.15.22:0/3287682410 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 17) v1 ==== 137341+0+0 (secure 0 0 0) 0x7f04d802a070 con 0x7f04e8101090 2024-03-28T18:06:08.136 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:08.137+0000 7f04dfe00700 1 --2- 172.21.15.22:0/3287682410 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7f04cc0a29a0 0x7f04cc0a4e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:06:08.137 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:08.137+0000 7f04dfe00700 1 -- 172.21.15.22:0/3287682410 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 8153+0+0 (secure 0 0 0) 0x7f04d80cf990 con 0x7f04e8101090 2024-03-28T18:06:08.137 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:08.137+0000 7f04e6a00700 1 --2- 172.21.15.22:0/3287682410 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7f04cc0a29a0 0x7f04cc0a4e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:06:08.137 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:08.138+0000 7f04e6a00700 1 --2- 172.21.15.22:0/3287682410 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7f04cc0a29a0 0x7f04cc0a4e60 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f04e80783e0 tx=0x7f04d4009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:06:08.141 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:08.142+0000 7f04dfe00700 1 -- 172.21.15.22:0/3287682410 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f04d8013070 con 0x7f04e8101090 2024-03-28T18:06:08.174 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:06:07 smithi134 ceph-mon[35037]: pgmap v410: 97 pgs: 97 active+clean; 642 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2024-03-28T18:06:08.182 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:06:07 smithi079 ceph-mon[35312]: pgmap v410: 97 pgs: 97 active+clean; 642 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2024-03-28T18:06:08.410 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:08.410+0000 7f04ee600700 1 -- 172.21.15.22:0/3287682410 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f04e81a7870 con 0x7f04e8101090 2024-03-28T18:06:08.410 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:08.410+0000 7f04dfe00700 1 -- 172.21.15.22:0/3287682410 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f04d80095a0 con 0x7f04e8101090 2024-03-28T18:06:08.411 INFO:teuthology.orchestra.run.smithi022.stdout:34359738475 2024-03-28T18:06:08.415 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:08.414+0000 7f04ee600700 1 -- 172.21.15.22:0/3287682410 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7f04cc0a29a0 msgr2=0x7f04cc0a4e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:06:08.415 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:08.414+0000 7f04ee600700 1 --2- 172.21.15.22:0/3287682410 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7f04cc0a29a0 0x7f04cc0a4e60 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f04e80783e0 tx=0x7f04d4009260 comp rx=0 tx=0).stop 2024-03-28T18:06:08.415 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:08.414+0000 7f04ee600700 1 -- 172.21.15.22:0/3287682410 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f04e8101090 msgr2=0x7f04e81a07d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:06:08.415 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:08.414+0000 7f04ee600700 1 --2- 172.21.15.22:0/3287682410 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f04e8101090 0x7f04e81a07d0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f04d8000c00 tx=0x7f04d80057c0 comp rx=0 tx=0).stop 2024-03-28T18:06:08.415 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:08.415+0000 7f04ee600700 1 -- 172.21.15.22:0/3287682410 shutdown_connections 2024-03-28T18:06:08.415 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:08.415+0000 7f04ee600700 1 --2- 172.21.15.22:0/3287682410 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f04e81070d0 0x7f04e819a990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:06:08.415 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:08.415+0000 7f04ee600700 1 --2- 172.21.15.22:0/3287682410 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f04e8101090 0x7f04e81a07d0 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:06:08.415 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:08.415+0000 7f04ee600700 1 --2- 172.21.15.22:0/3287682410 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7f04cc0a29a0 0x7f04cc0a4e60 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:06:08.415 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:08.415+0000 7f04ee600700 1 --2- 172.21.15.22:0/3287682410 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f04e8101a30 0x7f04e81a0d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:06:08.415 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:08.415+0000 7f04ee600700 1 -- 172.21.15.22:0/3287682410 >> 172.21.15.22:0/3287682410 conn(0x7f04e8077fc0 msgr2=0x7f04e8069ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:06:08.415 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:08.415+0000 7f04ee600700 1 -- 172.21.15.22:0/3287682410 shutdown_connections 2024-03-28T18:06:08.415 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:08.415+0000 7f04ee600700 1 -- 172.21.15.22:0/3287682410 wait complete. 2024-03-28T18:06:08.853 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:06:08 smithi022 ceph-mon[31985]: from='client.? 172.21.15.22:0/3287682410' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-03-28T18:06:09.174 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:06:08 smithi134 ceph-mon[35037]: from='client.? 172.21.15.22:0/3287682410' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-03-28T18:06:09.182 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:06:08 smithi079 ceph-mon[35312]: from='client.? 172.21.15.22:0/3287682410' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-03-28T18:06:09.748 INFO:tasks.cephadm.ceph_manager.ceph:need seq 171798691909 got 171798691914 for osd.5 2024-03-28T18:06:09.748 DEBUG:teuthology.parallel:result is None 2024-03-28T18:06:10.032 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:06:09 smithi022 ceph-mon[31985]: pgmap v411: 97 pgs: 97 active+clean; 642 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-03-28T18:06:10.174 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:06:09 smithi134 ceph-mon[35037]: pgmap v411: 97 pgs: 97 active+clean; 642 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-03-28T18:06:10.182 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:06:09 smithi079 ceph-mon[35312]: pgmap v411: 97 pgs: 97 active+clean; 642 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-03-28T18:06:11.022 INFO:tasks.cephadm.ceph_manager.ceph:need seq 34359738470 got 34359738475 for osd.0 2024-03-28T18:06:11.022 DEBUG:teuthology.parallel:result is None 2024-03-28T18:06:11.597 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:11.597+0000 7fbe3ba00700 1 -- 172.21.15.22:0/2599702338 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fbe3406a7e0 msgr2=0x7fbe3410f020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:06:11.598 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:11.597+0000 7fbe3ba00700 1 --2- 172.21.15.22:0/2599702338 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fbe3406a7e0 0x7fbe3410f020 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7fbe2c00b0d0 tx=0x7fbe2c00b3e0 comp rx=0 tx=0).stop 2024-03-28T18:06:11.598 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:11.597+0000 7fbe3ba00700 1 -- 172.21.15.22:0/2599702338 shutdown_connections 2024-03-28T18:06:11.598 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:11.597+0000 7fbe3ba00700 1 --2- 172.21.15.22:0/2599702338 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fbe3406a7e0 0x7fbe3410f020 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:06:11.598 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:11.597+0000 7fbe3ba00700 1 --2- 172.21.15.22:0/2599702338 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fbe341081e0 0x7fbe341085c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:06:11.598 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:11.597+0000 7fbe3ba00700 1 --2- 172.21.15.22:0/2599702338 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbe34069e40 0x7fbe3406a2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:06:11.598 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:11.597+0000 7fbe3ba00700 1 -- 172.21.15.22:0/2599702338 >> 172.21.15.22:0/2599702338 conn(0x7fbe34077fc0 msgr2=0x7fbe340783d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:06:11.598 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:11.598+0000 7fbe3ba00700 1 -- 172.21.15.22:0/2599702338 shutdown_connections 2024-03-28T18:06:11.598 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:11.598+0000 7fbe3ba00700 1 -- 172.21.15.22:0/2599702338 wait complete. 2024-03-28T18:06:11.599 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:11.599+0000 7fbe3ba00700 1 Processor -- start 2024-03-28T18:06:11.599 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:11.600+0000 7fbe3ba00700 1 -- start start 2024-03-28T18:06:11.600 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:11.600+0000 7fbe3ba00700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fbe34069e40 0x7fbe34101160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:06:11.600 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:11.600+0000 7fbe3ba00700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbe3406a7e0 0x7fbe341016a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:06:11.600 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:11.600+0000 7fbe3ba00700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fbe341081e0 0x7fbe34101be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:06:11.600 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:11.600+0000 7fbe3ba00700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe34112470 con 0x7fbe341081e0 2024-03-28T18:06:11.600 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:11.600+0000 7fbe3ba00700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe341122f0 con 0x7fbe34069e40 2024-03-28T18:06:11.600 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:11.600+0000 7fbe3ba00700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe341125f0 con 0x7fbe3406a7e0 2024-03-28T18:06:11.601 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:11.600+0000 7fbe39000700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fbe34069e40 0x7fbe34101160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:06:11.601 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:11.600+0000 7fbe39a00700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fbe341081e0 0x7fbe34101be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:06:11.601 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:11.601+0000 7fbe39000700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fbe34069e40 0x7fbe34101160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.22:60374/0 (socket says 172.21.15.22:60374) 2024-03-28T18:06:11.601 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:11.601+0000 7fbe39000700 1 -- 172.21.15.22:0/439075642 learned_addr learned my addr 172.21.15.22:0/439075642 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-28T18:06:11.601 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:11.601+0000 7fbe33e00700 1 --2- 172.21.15.22:0/439075642 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbe3406a7e0 0x7fbe341016a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:06:11.601 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:11.601+0000 7fbe39000700 1 -- 172.21.15.22:0/439075642 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbe3406a7e0 msgr2=0x7fbe341016a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:06:11.601 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:11.601+0000 7fbe39000700 1 --2- 172.21.15.22:0/439075642 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbe3406a7e0 0x7fbe341016a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:06:11.601 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:11.601+0000 7fbe39000700 1 -- 172.21.15.22:0/439075642 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fbe341081e0 msgr2=0x7fbe34101be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:06:11.601 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:11.601+0000 7fbe39000700 1 --2- 172.21.15.22:0/439075642 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fbe341081e0 0x7fbe34101be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:06:11.601 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:11.601+0000 7fbe39000700 1 -- 172.21.15.22:0/439075642 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbe341a6e10 con 0x7fbe34069e40 2024-03-28T18:06:11.601 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:11.601+0000 7fbe33e00700 1 --2- 172.21.15.22:0/439075642 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbe3406a7e0 0x7fbe341016a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-28T18:06:11.602 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:11.601+0000 7fbe39000700 1 --2- 172.21.15.22:0/439075642 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fbe34069e40 0x7fbe34101160 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fbe2800eb30 tx=0x7fbe2800ee40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:06:11.602 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:11.602+0000 7fbe31600700 1 -- 172.21.15.22:0/439075642 <== mon.2 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe2800cc90 con 0x7fbe34069e40 2024-03-28T18:06:11.602 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:11.602+0000 7fbe3ba00700 1 -- 172.21.15.22:0/439075642 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbe341a7100 con 0x7fbe34069e40 2024-03-28T18:06:11.602 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:11.602+0000 7fbe3ba00700 1 -- 172.21.15.22:0/439075642 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbe341a7690 con 0x7fbe34069e40 2024-03-28T18:06:11.603 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:11.602+0000 7fbe31600700 1 -- 172.21.15.22:0/439075642 <== mon.2 v2:172.21.15.79:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7fbe280044f0 con 0x7fbe34069e40 2024-03-28T18:06:11.603 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:11.602+0000 7fbe31600700 1 -- 172.21.15.22:0/439075642 <== mon.2 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe28010430 con 0x7fbe34069e40 2024-03-28T18:06:11.603 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:11.603+0000 7fbe3ba00700 1 -- 172.21.15.22:0/439075642 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbe14005330 con 0x7fbe34069e40 2024-03-28T18:06:11.605 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:11.606+0000 7fbe31600700 1 -- 172.21.15.22:0/439075642 <== mon.2 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 17) v1 ==== 137341+0+0 (secure 0 0 0) 0x7fbe28003680 con 0x7fbe34069e40 2024-03-28T18:06:11.609 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:11.607+0000 7fbe31600700 1 --2- 172.21.15.22:0/439075642 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7fbe1c0a2920 0x7fbe1c0a4de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:06:11.609 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:11.607+0000 7fbe31600700 1 -- 172.21.15.22:0/439075642 <== mon.2 v2:172.21.15.79:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 8153+0+0 (secure 0 0 0) 0x7fbe280cee10 con 0x7fbe34069e40 2024-03-28T18:06:11.610 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:11.608+0000 7fbe33e00700 1 --2- 172.21.15.22:0/439075642 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7fbe1c0a2920 0x7fbe1c0a4de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:06:11.610 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:11.608+0000 7fbe33e00700 1 --2- 172.21.15.22:0/439075642 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7fbe1c0a2920 0x7fbe1c0a4de0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fbe34078530 tx=0x7fbe2000b5f0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:06:11.619 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:11.619+0000 7fbe31600700 1 -- 172.21.15.22:0/439075642 <== mon.2 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fbe2809bde0 con 0x7fbe34069e40 2024-03-28T18:06:11.752 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:11.751+0000 7fbe31600700 1 -- 172.21.15.22:0/439075642 <== mon.2 v2:172.21.15.79:3300/0 7 ==== mgrmap(e 18) v1 ==== 137290+0+0 (secure 0 0 0) 0x7fbe28097d50 con 0x7fbe34069e40 2024-03-28T18:06:11.866 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:06:11 smithi022 ceph-mon[31985]: pgmap v412: 97 pgs: 97 active+clean; 642 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-03-28T18:06:11.875 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:11.875+0000 7fbe3ba00700 1 -- 172.21.15.22:0/439075642 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7fbe140058f0 con 0x7fbe34069e40 2024-03-28T18:06:11.875 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:11.876+0000 7fbe31600700 1 -- 172.21.15.22:0/439075642 <== mon.2 v2:172.21.15.79:3300/0 8 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fbe280c5b60 con 0x7fbe34069e40 2024-03-28T18:06:11.876 INFO:teuthology.orchestra.run.smithi022.stdout:111669149784 2024-03-28T18:06:11.879 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:11.879+0000 7fbe3ba00700 1 -- 172.21.15.22:0/439075642 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7fbe1c0a2920 msgr2=0x7fbe1c0a4de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:06:11.879 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:11.879+0000 7fbe3ba00700 1 --2- 172.21.15.22:0/439075642 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7fbe1c0a2920 0x7fbe1c0a4de0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fbe34078530 tx=0x7fbe2000b5f0 comp rx=0 tx=0).stop 2024-03-28T18:06:11.879 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:11.879+0000 7fbe3ba00700 1 -- 172.21.15.22:0/439075642 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fbe34069e40 msgr2=0x7fbe34101160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:06:11.879 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:11.879+0000 7fbe3ba00700 1 --2- 172.21.15.22:0/439075642 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fbe34069e40 0x7fbe34101160 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fbe2800eb30 tx=0x7fbe2800ee40 comp rx=0 tx=0).stop 2024-03-28T18:06:11.879 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:11.879+0000 7fbe3ba00700 1 -- 172.21.15.22:0/439075642 shutdown_connections 2024-03-28T18:06:11.879 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:11.879+0000 7fbe3ba00700 1 --2- 172.21.15.22:0/439075642 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fbe341081e0 0x7fbe34101be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:06:11.880 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:11.879+0000 7fbe3ba00700 1 --2- 172.21.15.22:0/439075642 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fbe34069e40 0x7fbe34101160 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:06:11.880 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:11.879+0000 7fbe3ba00700 1 --2- 172.21.15.22:0/439075642 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7fbe1c0a2920 0x7fbe1c0a4de0 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:06:11.880 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:11.879+0000 7fbe3ba00700 1 --2- 172.21.15.22:0/439075642 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbe3406a7e0 0x7fbe341016a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:06:11.880 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:11.879+0000 7fbe3ba00700 1 -- 172.21.15.22:0/439075642 >> 172.21.15.22:0/439075642 conn(0x7fbe34077fc0 msgr2=0x7fbe340ffbe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:06:11.880 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:11.880+0000 7fbe3ba00700 1 -- 172.21.15.22:0/439075642 shutdown_connections 2024-03-28T18:06:11.880 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:11.880+0000 7fbe3ba00700 1 -- 172.21.15.22:0/439075642 wait complete. 2024-03-28T18:06:12.153 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:12.153+0000 7ff693200700 1 -- 172.21.15.22:0/484766985 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff68c100e20 msgr2=0x7ff68c101200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:06:12.154 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:12.153+0000 7ff693200700 1 --2- 172.21.15.22:0/484766985 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff68c100e20 0x7ff68c101200 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7ff680009990 tx=0x7ff680009ca0 comp rx=0 tx=0).stop 2024-03-28T18:06:12.154 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:12.153+0000 7ff693200700 1 -- 172.21.15.22:0/484766985 shutdown_connections 2024-03-28T18:06:12.154 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:12.153+0000 7ff693200700 1 --2- 172.21.15.22:0/484766985 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ff68c10e9d0 0x7ff68c110dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:06:12.154 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:12.153+0000 7ff693200700 1 --2- 172.21.15.22:0/484766985 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff68c100e20 0x7ff68c101200 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:06:12.154 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:12.153+0000 7ff693200700 1 --2- 172.21.15.22:0/484766985 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff68c1017d0 0x7ff68c10e490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:06:12.154 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:12.153+0000 7ff693200700 1 -- 172.21.15.22:0/484766985 >> 172.21.15.22:0/484766985 conn(0x7ff68c0fca60 msgr2=0x7ff68c0fee80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:06:12.154 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:12.154+0000 7ff693200700 1 -- 172.21.15.22:0/484766985 shutdown_connections 2024-03-28T18:06:12.154 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:12.154+0000 7ff693200700 1 -- 172.21.15.22:0/484766985 wait complete. 2024-03-28T18:06:12.155 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:12.155+0000 7ff693200700 1 Processor -- start 2024-03-28T18:06:12.155 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:12.155+0000 7ff693200700 1 -- start start 2024-03-28T18:06:12.156 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:12.155+0000 7ff693200700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff68c100e20 0x7ff68c1a0710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:06:12.156 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:12.155+0000 7ff693200700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff68c1017d0 0x7ff68c1a0c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:06:12.156 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:12.156+0000 7ff693200700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ff68c10e9d0 0x7ff68c19a8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:06:12.156 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:12.156+0000 7ff693200700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff68c114650 con 0x7ff68c100e20 2024-03-28T18:06:12.156 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:12.156+0000 7ff693200700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff68c1144d0 con 0x7ff68c10e9d0 2024-03-28T18:06:12.156 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:12.156+0000 7ff693200700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff68c1147d0 con 0x7ff68c1017d0 2024-03-28T18:06:12.156 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:12.156+0000 7ff691e00700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff68c100e20 0x7ff68c1a0710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:06:12.157 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:12.156+0000 7ff691e00700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff68c100e20 0x7ff68c1a0710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:43366/0 (socket says 172.21.15.22:43366) 2024-03-28T18:06:12.157 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:12.156+0000 7ff691e00700 1 -- 172.21.15.22:0/324696406 learned_addr learned my addr 172.21.15.22:0/324696406 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-28T18:06:12.157 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:12.156+0000 7ff692800700 1 --2- 172.21.15.22:0/324696406 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ff68c10e9d0 0x7ff68c19a8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:06:12.157 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:12.156+0000 7ff691400700 1 --2- 172.21.15.22:0/324696406 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff68c1017d0 0x7ff68c1a0c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:06:12.157 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:12.156+0000 7ff691e00700 1 -- 172.21.15.22:0/324696406 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff68c1017d0 msgr2=0x7ff68c1a0c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:06:12.157 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:12.156+0000 7ff691e00700 1 --2- 172.21.15.22:0/324696406 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff68c1017d0 0x7ff68c1a0c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:06:12.157 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:12.156+0000 7ff691e00700 1 -- 172.21.15.22:0/324696406 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ff68c10e9d0 msgr2=0x7ff68c19a8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:06:12.157 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:12.156+0000 7ff691e00700 1 --2- 172.21.15.22:0/324696406 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ff68c10e9d0 0x7ff68c19a8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:06:12.157 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:12.156+0000 7ff691e00700 1 -- 172.21.15.22:0/324696406 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff68c19b030 con 0x7ff68c100e20 2024-03-28T18:06:12.157 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:12.157+0000 7ff691e00700 1 --2- 172.21.15.22:0/324696406 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff68c100e20 0x7ff68c1a0710 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7ff680000c00 tx=0x7ff6800038b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:06:12.158 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:12.157+0000 7ff686a00700 1 -- 172.21.15.22:0/324696406 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff680027070 con 0x7ff68c100e20 2024-03-28T18:06:12.158 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:12.157+0000 7ff686a00700 1 -- 172.21.15.22:0/324696406 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7ff680005690 con 0x7ff68c100e20 2024-03-28T18:06:12.158 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:12.157+0000 7ff693200700 1 -- 172.21.15.22:0/324696406 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff68c19b2c0 con 0x7ff68c100e20 2024-03-28T18:06:12.158 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:12.157+0000 7ff686a00700 1 -- 172.21.15.22:0/324696406 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff680022c90 con 0x7ff68c100e20 2024-03-28T18:06:12.158 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:12.157+0000 7ff693200700 1 -- 172.21.15.22:0/324696406 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff68c1a75e0 con 0x7ff68c100e20 2024-03-28T18:06:12.158 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:12.158+0000 7ff693200700 1 -- 172.21.15.22:0/324696406 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff68c068e50 con 0x7ff68c100e20 2024-03-28T18:06:12.160 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:12.160+0000 7ff686a00700 1 -- 172.21.15.22:0/324696406 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 18) v1 ==== 137290+0+0 (secure 0 0 0) 0x7ff68002a070 con 0x7ff68c100e20 2024-03-28T18:06:12.162 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:12.162+0000 7ff686a00700 1 --2- 172.21.15.22:0/324696406 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7ff6740a2910 0x7ff6740a4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:06:12.162 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:12.163+0000 7ff686a00700 1 -- 172.21.15.22:0/324696406 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 8153+0+0 (secure 0 0 0) 0x7ff6800cfb40 con 0x7ff68c100e20 2024-03-28T18:06:12.162 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:12.163+0000 7ff691400700 1 --2- 172.21.15.22:0/324696406 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7ff6740a2910 0x7ff6740a4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:06:12.163 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:12.163+0000 7ff691400700 1 --2- 172.21.15.22:0/324696406 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7ff6740a2910 0x7ff6740a4dd0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7ff67c006fd0 tx=0x7ff67c008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:06:12.167 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:12.167+0000 7ff686a00700 1 -- 172.21.15.22:0/324696406 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7ff68005e740 con 0x7ff68c100e20 2024-03-28T18:06:12.174 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:06:11 smithi134 ceph-mon[35037]: pgmap v412: 97 pgs: 97 active+clean; 642 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-03-28T18:06:12.182 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:06:11 smithi079 ceph-mon[35312]: pgmap v412: 97 pgs: 97 active+clean; 642 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-03-28T18:06:12.405 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:12.405+0000 7ff693200700 1 -- 172.21.15.22:0/324696406 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 10} v 0) v1 -- 0x7ff68c02da30 con 0x7ff68c100e20 2024-03-28T18:06:12.406 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:12.406+0000 7ff686a00700 1 -- 172.21.15.22:0/324696406 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 10}]=0 v0) v1 ==== 75+0+13 (secure 0 0 0) 0x7ff680013070 con 0x7ff68c100e20 2024-03-28T18:06:12.406 INFO:teuthology.orchestra.run.smithi022.stdout:330712481833 2024-03-28T18:06:12.409 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:12.409+0000 7ff693200700 1 -- 172.21.15.22:0/324696406 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7ff6740a2910 msgr2=0x7ff6740a4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:06:12.409 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:12.409+0000 7ff693200700 1 --2- 172.21.15.22:0/324696406 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7ff6740a2910 0x7ff6740a4dd0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7ff67c006fd0 tx=0x7ff67c008040 comp rx=0 tx=0).stop 2024-03-28T18:06:12.410 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:12.409+0000 7ff693200700 1 -- 172.21.15.22:0/324696406 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff68c100e20 msgr2=0x7ff68c1a0710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:06:12.410 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:12.410+0000 7ff693200700 1 --2- 172.21.15.22:0/324696406 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff68c100e20 0x7ff68c1a0710 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7ff680000c00 tx=0x7ff6800038b0 comp rx=0 tx=0).stop 2024-03-28T18:06:12.410 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:12.410+0000 7ff693200700 1 -- 172.21.15.22:0/324696406 shutdown_connections 2024-03-28T18:06:12.410 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:12.410+0000 7ff693200700 1 --2- 172.21.15.22:0/324696406 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ff68c10e9d0 0x7ff68c19a8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:06:12.410 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:12.410+0000 7ff693200700 1 --2- 172.21.15.22:0/324696406 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff68c100e20 0x7ff68c1a0710 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:06:12.410 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:12.410+0000 7ff693200700 1 --2- 172.21.15.22:0/324696406 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7ff6740a2910 0x7ff6740a4dd0 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:06:12.411 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:12.410+0000 7ff693200700 1 --2- 172.21.15.22:0/324696406 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff68c1017d0 0x7ff68c1a0c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:06:12.411 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:12.410+0000 7ff693200700 1 -- 172.21.15.22:0/324696406 >> 172.21.15.22:0/324696406 conn(0x7ff68c0fca60 msgr2=0x7ff68c0fee50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:06:12.411 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:12.410+0000 7ff693200700 1 -- 172.21.15.22:0/324696406 shutdown_connections 2024-03-28T18:06:12.411 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:12.410+0000 7ff693200700 1 -- 172.21.15.22:0/324696406 wait complete. 2024-03-28T18:06:13.032 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:06:12 smithi022 ceph-mon[31985]: mgrmap e18: x(active, since 11m), standbys: z, y 2024-03-28T18:06:13.032 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:06:12 smithi022 ceph-mon[31985]: from='client.? 172.21.15.22:0/439075642' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-03-28T18:06:13.032 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:06:12 smithi022 ceph-mon[31985]: from='client.? 172.21.15.22:0/324696406' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-03-28T18:06:13.168 INFO:tasks.cephadm.ceph_manager.ceph:need seq 330712481827 got 330712481833 for osd.10 2024-03-28T18:06:13.168 DEBUG:teuthology.parallel:result is None 2024-03-28T18:06:13.174 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:06:12 smithi134 ceph-mon[35037]: mgrmap e18: x(active, since 11m), standbys: z, y 2024-03-28T18:06:13.174 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:06:12 smithi134 ceph-mon[35037]: from='client.? 172.21.15.22:0/439075642' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-03-28T18:06:13.175 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:06:12 smithi134 ceph-mon[35037]: from='client.? 172.21.15.22:0/324696406' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-03-28T18:06:13.182 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:06:12 smithi079 ceph-mon[35312]: mgrmap e18: x(active, since 11m), standbys: z, y 2024-03-28T18:06:13.182 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:06:12 smithi079 ceph-mon[35312]: from='client.? 172.21.15.22:0/439075642' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-03-28T18:06:13.182 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:06:12 smithi079 ceph-mon[35312]: from='client.? 172.21.15.22:0/324696406' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-03-28T18:06:13.519 INFO:tasks.cephadm.ceph_manager.ceph:need seq 111669149779 got 111669149784 for osd.3 2024-03-28T18:06:13.519 DEBUG:teuthology.parallel:result is None 2024-03-28T18:06:13.519 INFO:tasks.cephadm.ceph_manager.ceph:waiting for clean 2024-03-28T18:06:13.519 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033 shell --fsid fd220748-ed2b-11ee-95d0-87774f69a715 -- ceph pg dump --format=json 2024-03-28T18:06:13.986 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/fd220748-ed2b-11ee-95d0-87774f69a715/mon.a/config 2024-03-28T18:06:14.031 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:06:13 smithi022 ceph-mon[31985]: pgmap v413: 97 pgs: 97 active+clean; 642 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 255 B/s wr, 10 op/s 2024-03-28T18:06:14.174 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:06:13 smithi134 ceph-mon[35037]: pgmap v413: 97 pgs: 97 active+clean; 642 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 255 B/s wr, 10 op/s 2024-03-28T18:06:14.182 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:06:13 smithi079 ceph-mon[35312]: pgmap v413: 97 pgs: 97 active+clean; 642 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 255 B/s wr, 10 op/s 2024-03-28T18:06:15.282 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:06:14 smithi022 ceph-mon[31985]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T18:06:15.282 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:06:14 smithi022 ceph-mon[31985]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T18:06:15.424 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:06:14 smithi134 ceph-mon[35037]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T18:06:15.424 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:06:14 smithi134 ceph-mon[35037]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T18:06:15.432 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:06:14 smithi079 ceph-mon[35312]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T18:06:15.432 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:06:14 smithi079 ceph-mon[35312]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T18:06:16.099 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:16.099+0000 7fcdb3200700 1 -- 172.21.15.22:0/4292499153 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fcdac1081c0 msgr2=0x7fcdac1085a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:06:16.100 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:16.099+0000 7fcdb3200700 1 --2- 172.21.15.22:0/4292499153 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fcdac1081c0 0x7fcdac1085a0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7fcd9c0099c0 tx=0x7fcd9c009cd0 comp rx=0 tx=0).stop 2024-03-28T18:06:16.100 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:16.099+0000 7fcdb3200700 1 -- 172.21.15.22:0/4292499153 shutdown_connections 2024-03-28T18:06:16.100 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:16.099+0000 7fcdb3200700 1 --2- 172.21.15.22:0/4292499153 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcdac102ba0 0x7fcdac10efe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:06:16.100 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:16.099+0000 7fcdb3200700 1 --2- 172.21.15.22:0/4292499153 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fcdac1081c0 0x7fcdac1085a0 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:06:16.100 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:16.099+0000 7fcdb3200700 1 --2- 172.21.15.22:0/4292499153 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fcdac1021e0 0x7fcdac102660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:06:16.100 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:16.099+0000 7fcdb3200700 1 -- 172.21.15.22:0/4292499153 >> 172.21.15.22:0/4292499153 conn(0x7fcdac0792c0 msgr2=0x7fcdac0796d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:06:16.100 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:16.099+0000 7fcdb3200700 1 -- 172.21.15.22:0/4292499153 shutdown_connections 2024-03-28T18:06:16.100 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:16.099+0000 7fcdb3200700 1 -- 172.21.15.22:0/4292499153 wait complete. 2024-03-28T18:06:16.101 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:16.101+0000 7fcdb3200700 1 Processor -- start 2024-03-28T18:06:16.101 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:16.101+0000 7fcdb3200700 1 -- start start 2024-03-28T18:06:16.102 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:16.102+0000 7fcdb3200700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcdac1021e0 0x7fcdac1a06a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:06:16.102 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:16.102+0000 7fcdb3200700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fcdac102ba0 0x7fcdac1a0be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:06:16.102 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:16.102+0000 7fcdb3200700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fcdac1081c0 0x7fcdac19a860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:06:16.102 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:16.102+0000 7fcdb3200700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcdac112430 con 0x7fcdac1081c0 2024-03-28T18:06:16.102 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:16.102+0000 7fcdb3200700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcdac1122b0 con 0x7fcdac102ba0 2024-03-28T18:06:16.102 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:16.102+0000 7fcdb3200700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcdac1125b0 con 0x7fcdac1021e0 2024-03-28T18:06:16.102 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:16.102+0000 7fcdb1200700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fcdac1081c0 0x7fcdac19a860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:06:16.102 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:16.102+0000 7fcdb1200700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fcdac1081c0 0x7fcdac19a860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:43394/0 (socket says 172.21.15.22:43394) 2024-03-28T18:06:16.102 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:16.102+0000 7fcdb1200700 1 -- 172.21.15.22:0/2556348205 learned_addr learned my addr 172.21.15.22:0/2556348205 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-28T18:06:16.102 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:16.102+0000 7fcdabe00700 1 --2- 172.21.15.22:0/2556348205 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcdac1021e0 0x7fcdac1a06a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:06:16.103 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:16.102+0000 7fcdab400700 1 --2- 172.21.15.22:0/2556348205 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fcdac102ba0 0x7fcdac1a0be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:06:16.103 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:16.103+0000 7fcdb1200700 1 -- 172.21.15.22:0/2556348205 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcdac1021e0 msgr2=0x7fcdac1a06a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:06:16.103 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:16.103+0000 7fcdb1200700 1 --2- 172.21.15.22:0/2556348205 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcdac1021e0 0x7fcdac1a06a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:06:16.104 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:16.103+0000 7fcdb1200700 1 -- 172.21.15.22:0/2556348205 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fcdac102ba0 msgr2=0x7fcdac1a0be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:06:16.104 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:16.103+0000 7fcdb1200700 1 --2- 172.21.15.22:0/2556348205 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fcdac102ba0 0x7fcdac1a0be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:06:16.104 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:16.103+0000 7fcdb1200700 1 -- 172.21.15.22:0/2556348205 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcdac19afc0 con 0x7fcdac1081c0 2024-03-28T18:06:16.104 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:16.103+0000 7fcdab400700 1 --2- 172.21.15.22:0/2556348205 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fcdac102ba0 0x7fcdac1a0be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-28T18:06:16.104 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:16.103+0000 7fcdb1200700 1 --2- 172.21.15.22:0/2556348205 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fcdac1081c0 0x7fcdac19a860 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7fcda4007ee0 tx=0x7fcda400d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:06:16.104 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:16.103+0000 7fcda8c00700 1 -- 172.21.15.22:0/2556348205 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcda400dbd0 con 0x7fcdac1081c0 2024-03-28T18:06:16.104 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:16.103+0000 7fcda8c00700 1 -- 172.21.15.22:0/2556348205 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7fcda400dd70 con 0x7fcdac1081c0 2024-03-28T18:06:16.104 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:16.103+0000 7fcdb3200700 1 -- 172.21.15.22:0/2556348205 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcdac19b2b0 con 0x7fcdac1081c0 2024-03-28T18:06:16.105 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:16.103+0000 7fcda8c00700 1 -- 172.21.15.22:0/2556348205 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcda40059b0 con 0x7fcdac1081c0 2024-03-28T18:06:16.105 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:16.104+0000 7fcdb3200700 1 -- 172.21.15.22:0/2556348205 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcdac1a7570 con 0x7fcdac1081c0 2024-03-28T18:06:16.105 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:16.105+0000 7fcdb3200700 1 -- 172.21.15.22:0/2556348205 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcdac068e50 con 0x7fcdac1081c0 2024-03-28T18:06:16.106 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:16.106+0000 7fcda8c00700 1 -- 172.21.15.22:0/2556348205 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 18) v1 ==== 137290+0+0 (secure 0 0 0) 0x7fcda401a020 con 0x7fcdac1081c0 2024-03-28T18:06:16.107 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:16.108+0000 7fcda8c00700 1 --2- 172.21.15.22:0/2556348205 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7fcd900a2910 0x7fcd900a4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:06:16.108 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:16.108+0000 7fcda8c00700 1 -- 172.21.15.22:0/2556348205 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 8153+0+0 (secure 0 0 0) 0x7fcda40ce1c0 con 0x7fcdac1081c0 2024-03-28T18:06:16.108 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:16.108+0000 7fcdabe00700 1 --2- 172.21.15.22:0/2556348205 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7fcd900a2910 0x7fcd900a4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:06:16.109 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:16.109+0000 7fcdabe00700 1 --2- 172.21.15.22:0/2556348205 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7fcd900a2910 0x7fcd900a4dd0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fcd9c000c00 tx=0x7fcd9c005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:06:16.113 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:16.112+0000 7fcda8c00700 1 -- 172.21.15.22:0/2556348205 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fcda4010070 con 0x7fcdac1081c0 2024-03-28T18:06:16.282 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:06:15 smithi022 ceph-mon[31985]: pgmap v414: 97 pgs: 97 active+clean; 642 KiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-03-28T18:06:16.282 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:06:15 smithi022 ceph-mon[31985]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T18:06:16.282 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:06:15 smithi022 ceph-mon[31985]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T18:06:16.282 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:06:15 smithi022 ceph-mon[31985]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T18:06:16.335 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:16.336+0000 7fcdb3200700 1 -- 172.21.15.22:0/2556348205 --> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fcdac075de0 con 0x7fcd900a2910 2024-03-28T18:06:16.361 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:16.361+0000 7fcda8c00700 1 -- 172.21.15.22:0/2556348205 <== mgr.14152 v2:172.21.15.22:6800/4201048392 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+259087 (secure 0 0 0) 0x7fcdac075de0 con 0x7fcd900a2910 2024-03-28T18:06:16.362 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-03-28T18:06:16.368 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:16.367+0000 7fcdb3200700 1 -- 172.21.15.22:0/2556348205 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7fcd900a2910 msgr2=0x7fcd900a4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:06:16.368 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:16.367+0000 7fcdb3200700 1 --2- 172.21.15.22:0/2556348205 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7fcd900a2910 0x7fcd900a4dd0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fcd9c000c00 tx=0x7fcd9c005fb0 comp rx=0 tx=0).stop 2024-03-28T18:06:16.368 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:16.367+0000 7fcdb3200700 1 -- 172.21.15.22:0/2556348205 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fcdac1081c0 msgr2=0x7fcdac19a860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:06:16.368 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:16.367+0000 7fcdb3200700 1 --2- 172.21.15.22:0/2556348205 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fcdac1081c0 0x7fcdac19a860 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7fcda4007ee0 tx=0x7fcda400d3b0 comp rx=0 tx=0).stop 2024-03-28T18:06:16.368 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:16.367+0000 7fcdb3200700 1 -- 172.21.15.22:0/2556348205 shutdown_connections 2024-03-28T18:06:16.368 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:16.367+0000 7fcdb3200700 1 --2- 172.21.15.22:0/2556348205 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fcdac1081c0 0x7fcdac19a860 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:06:16.368 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:16.367+0000 7fcdb3200700 1 --2- 172.21.15.22:0/2556348205 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7fcd900a2910 0x7fcd900a4dd0 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:06:16.369 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:16.367+0000 7fcdb3200700 1 --2- 172.21.15.22:0/2556348205 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcdac1021e0 0x7fcdac1a06a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:06:16.369 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:16.367+0000 7fcdb3200700 1 --2- 172.21.15.22:0/2556348205 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fcdac102ba0 0x7fcdac1a0be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:06:16.369 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:16.367+0000 7fcdb3200700 1 -- 172.21.15.22:0/2556348205 >> 172.21.15.22:0/2556348205 conn(0x7fcdac0792c0 msgr2=0x7fcdac0746c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:06:16.369 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:16.368+0000 7fcdb3200700 1 -- 172.21.15.22:0/2556348205 shutdown_connections 2024-03-28T18:06:16.369 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:16.368+0000 7fcdb3200700 1 -- 172.21.15.22:0/2556348205 wait complete. 2024-03-28T18:06:16.369 INFO:teuthology.orchestra.run.smithi022.stderr:dumped all 2024-03-28T18:06:16.424 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:06:15 smithi134 ceph-mon[35037]: pgmap v414: 97 pgs: 97 active+clean; 642 KiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-03-28T18:06:16.424 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:06:15 smithi134 ceph-mon[35037]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T18:06:16.425 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:06:15 smithi134 ceph-mon[35037]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T18:06:16.425 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:06:15 smithi134 ceph-mon[35037]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T18:06:16.432 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:06:15 smithi079 ceph-mon[35312]: pgmap v414: 97 pgs: 97 active+clean; 642 KiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-03-28T18:06:16.432 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:06:15 smithi079 ceph-mon[35312]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T18:06:16.432 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:06:15 smithi079 ceph-mon[35312]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T18:06:16.432 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:06:15 smithi079 ceph-mon[35312]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T18:06:17.032 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:06:16 smithi022 ceph-mon[31985]: from='client.14976 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-28T18:06:17.175 INFO:teuthology.orchestra.run.smithi022.stdout:{"pg_ready":true,"pg_map":{"version":414,"stamp":"2024-03-28T18:06:14.637286+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":657779,"num_objects":102,"num_object_clones":0,"num_object_copies":306,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":916,"num_read_kb":579,"num_write":423,"num_write_kb":3075,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13,"num_bytes_recovered":1180853,"num_keys_recovered":10,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":814,"ondisk_log_size":814,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":400588,"kb_used_data":13980,"kb_used_omap":18,"kb_used_meta":386477,"kb_avail":1124541236,"statfs":{"total":1151940427776,"available":1151530225664,"internally_reserved":0,"allocated":14315520,"data_stored":7765437,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19141,"internal_metadata":395752763},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[3,15,4,14,7,12,9],"upper_bound":128},"perf_stat":{"commit_latency_ms":68,"apply_latency_ms":68,"commit_latency_ns":68000000,"apply_latency_ns":68000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":282,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":105,"num_read_kb":51,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004592"},"pg_stats":[{"pgid":"3.3f","version":"92'8","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-28T18:06:10.621904+0000","last_change":"2024-03-28T18:04:49.545022+0000","last_active":"2024-03-28T18:06:10.621904+0000","last_peered":"2024-03-28T18:06:10.621904+0000","last_clean":"2024-03-28T18:06:10.621904+0000","last_became_active":"2024-03-28T18:04:49.544584+0000","last_became_peered":"2024-03-28T18:04:49.544584+0000","last_unstale":"2024-03-28T18:06:10.621904+0000","last_undegraded":"2024-03-28T18:06:10.621904+0000","last_fullsized":"2024-03-28T18:06:10.621904+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_clean_scrub_stamp":"2024-03-28T18:04:48.465760+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T01:47:05.644293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"92'4","reported_seq":21,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-28T18:06:10.603212+0000","last_change":"2024-03-28T18:04:49.526720+0000","last_active":"2024-03-28T18:06:10.603212+0000","last_peered":"2024-03-28T18:06:10.603212+0000","last_clean":"2024-03-28T18:06:10.603212+0000","last_became_active":"2024-03-28T18:04:49.524545+0000","last_became_peered":"2024-03-28T18:04:49.524545+0000","last_unstale":"2024-03-28T18:06:10.603212+0000","last_undegraded":"2024-03-28T18:06:10.603212+0000","last_fullsized":"2024-03-28T18:06:10.603212+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_clean_scrub_stamp":"2024-03-28T18:04:48.465760+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T01:00:54.926308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"92'7","reported_seq":24,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-28T18:06:10.625694+0000","last_change":"2024-03-28T18:04:49.532425+0000","last_active":"2024-03-28T18:06:10.625694+0000","last_peered":"2024-03-28T18:06:10.625694+0000","last_clean":"2024-03-28T18:06:10.625694+0000","last_became_active":"2024-03-28T18:04:49.531146+0000","last_became_peered":"2024-03-28T18:04:49.531146+0000","last_unstale":"2024-03-28T18:06:10.625694+0000","last_undegraded":"2024-03-28T18:06:10.625694+0000","last_fullsized":"2024-03-28T18:06:10.625694+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_clean_scrub_stamp":"2024-03-28T18:04:48.465760+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T04:58:45.131773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"92'8","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-28T18:06:10.609205+0000","last_change":"2024-03-28T18:04:49.533371+0000","last_active":"2024-03-28T18:06:10.609205+0000","last_peered":"2024-03-28T18:06:10.609205+0000","last_clean":"2024-03-28T18:06:10.609205+0000","last_became_active":"2024-03-28T18:04:49.532544+0000","last_became_peered":"2024-03-28T18:04:49.532544+0000","last_unstale":"2024-03-28T18:06:10.609205+0000","last_undegraded":"2024-03-28T18:06:10.609205+0000","last_fullsized":"2024-03-28T18:06:10.609205+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_clean_scrub_stamp":"2024-03-28T18:04:48.465760+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T20:36:25.664137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"92'4","reported_seq":21,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-28T18:06:10.592721+0000","last_change":"2024-03-28T18:04:49.530875+0000","last_active":"2024-03-28T18:06:10.592721+0000","last_peered":"2024-03-28T18:06:10.592721+0000","last_clean":"2024-03-28T18:06:10.592721+0000","last_became_active":"2024-03-28T18:04:49.530354+0000","last_became_peered":"2024-03-28T18:04:49.530354+0000","last_unstale":"2024-03-28T18:06:10.592721+0000","last_undegraded":"2024-03-28T18:06:10.592721+0000","last_fullsized":"2024-03-28T18:06:10.592721+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_clean_scrub_stamp":"2024-03-28T18:04:48.465760+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T21:02:17.187989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":17,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-28T18:05:11.732188+0000","last_change":"2024-03-28T18:04:49.538260+0000","last_active":"2024-03-28T18:05:11.732188+0000","last_peered":"2024-03-28T18:05:11.732188+0000","last_clean":"2024-03-28T18:05:11.732188+0000","last_became_active":"2024-03-28T18:04:49.537863+0000","last_became_peered":"2024-03-28T18:04:49.537863+0000","last_unstale":"2024-03-28T18:05:11.732188+0000","last_undegraded":"2024-03-28T18:05:11.732188+0000","last_fullsized":"2024-03-28T18:05:11.732188+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_clean_scrub_stamp":"2024-03-28T18:04:48.465760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T03:01:25.363839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"92'5","reported_seq":22,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-28T18:06:10.612305+0000","last_change":"2024-03-28T18:04:49.534636+0000","last_active":"2024-03-28T18:06:10.612305+0000","last_peered":"2024-03-28T18:06:10.612305+0000","last_clean":"2024-03-28T18:06:10.612305+0000","last_became_active":"2024-03-28T18:04:49.534236+0000","last_became_peered":"2024-03-28T18:04:49.534236+0000","last_unstale":"2024-03-28T18:06:10.612305+0000","last_undegraded":"2024-03-28T18:06:10.612305+0000","last_fullsized":"2024-03-28T18:06:10.612305+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_clean_scrub_stamp":"2024-03-28T18:04:48.465760+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T05:55:23.616641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"92'9","reported_seq":26,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-28T18:06:10.605001+0000","last_change":"2024-03-28T18:04:49.534848+0000","last_active":"2024-03-28T18:06:10.605001+0000","last_peered":"2024-03-28T18:06:10.605001+0000","last_clean":"2024-03-28T18:06:10.605001+0000","last_became_active":"2024-03-28T18:04:49.534302+0000","last_became_peered":"2024-03-28T18:04:49.534302+0000","last_unstale":"2024-03-28T18:06:10.605001+0000","last_undegraded":"2024-03-28T18:06:10.605001+0000","last_fullsized":"2024-03-28T18:06:10.605001+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_clean_scrub_stamp":"2024-03-28T18:04:48.465760+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T05:48:24.707305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"92'11","reported_seq":28,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-28T18:06:10.622058+0000","last_change":"2024-03-28T18:04:49.536568+0000","last_active":"2024-03-28T18:06:10.622058+0000","last_peered":"2024-03-28T18:06:10.622058+0000","last_clean":"2024-03-28T18:06:10.622058+0000","last_became_active":"2024-03-28T18:04:49.535675+0000","last_became_peered":"2024-03-28T18:04:49.535675+0000","last_unstale":"2024-03-28T18:06:10.622058+0000","last_undegraded":"2024-03-28T18:06:10.622058+0000","last_fullsized":"2024-03-28T18:06:10.622058+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_clean_scrub_stamp":"2024-03-28T18:04:48.465760+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T02:32:15.179869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":17,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-28T18:05:11.731516+0000","last_change":"2024-03-28T18:04:49.539363+0000","last_active":"2024-03-28T18:05:11.731516+0000","last_peered":"2024-03-28T18:05:11.731516+0000","last_clean":"2024-03-28T18:05:11.731516+0000","last_became_active":"2024-03-28T18:04:49.539002+0000","last_became_peered":"2024-03-28T18:04:49.539002+0000","last_unstale":"2024-03-28T18:05:11.731516+0000","last_undegraded":"2024-03-28T18:05:11.731516+0000","last_fullsized":"2024-03-28T18:05:11.731516+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_clean_scrub_stamp":"2024-03-28T18:04:48.465760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T18:24:28.846081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"92'6","reported_seq":23,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-28T18:06:10.626220+0000","last_change":"2024-03-28T18:04:49.532385+0000","last_active":"2024-03-28T18:06:10.626220+0000","last_peered":"2024-03-28T18:06:10.626220+0000","last_clean":"2024-03-28T18:06:10.626220+0000","last_became_active":"2024-03-28T18:04:49.532004+0000","last_became_peered":"2024-03-28T18:04:49.532004+0000","last_unstale":"2024-03-28T18:06:10.626220+0000","last_undegraded":"2024-03-28T18:06:10.626220+0000","last_fullsized":"2024-03-28T18:06:10.626220+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_clean_scrub_stamp":"2024-03-28T18:04:48.465760+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T19:05:13.959313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"92'7","reported_seq":24,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-28T18:06:10.608755+0000","last_change":"2024-03-28T18:04:49.534465+0000","last_active":"2024-03-28T18:06:10.608755+0000","last_peered":"2024-03-28T18:06:10.608755+0000","last_clean":"2024-03-28T18:06:10.608755+0000","last_became_active":"2024-03-28T18:04:49.532989+0000","last_became_peered":"2024-03-28T18:04:49.532989+0000","last_unstale":"2024-03-28T18:06:10.608755+0000","last_undegraded":"2024-03-28T18:06:10.608755+0000","last_fullsized":"2024-03-28T18:06:10.608755+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_clean_scrub_stamp":"2024-03-28T18:04:48.465760+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T23:56:50.607770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"92'10","reported_seq":27,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-28T18:06:10.611908+0000","last_change":"2024-03-28T18:04:49.535205+0000","last_active":"2024-03-28T18:06:10.611908+0000","last_peered":"2024-03-28T18:06:10.611908+0000","last_clean":"2024-03-28T18:06:10.611908+0000","last_became_active":"2024-03-28T18:04:49.533025+0000","last_became_peered":"2024-03-28T18:04:49.533025+0000","last_unstale":"2024-03-28T18:06:10.611908+0000","last_undegraded":"2024-03-28T18:06:10.611908+0000","last_fullsized":"2024-03-28T18:06:10.611908+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_clean_scrub_stamp":"2024-03-28T18:04:48.465760+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T21:41:51.815907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"92'7","reported_seq":24,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-28T18:06:10.615333+0000","last_change":"2024-03-28T18:04:49.520974+0000","last_active":"2024-03-28T18:06:10.615333+0000","last_peered":"2024-03-28T18:06:10.615333+0000","last_clean":"2024-03-28T18:06:10.615333+0000","last_became_active":"2024-03-28T18:04:49.520509+0000","last_became_peered":"2024-03-28T18:04:49.520509+0000","last_unstale":"2024-03-28T18:06:10.615333+0000","last_undegraded":"2024-03-28T18:06:10.615333+0000","last_fullsized":"2024-03-28T18:06:10.615333+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_clean_scrub_stamp":"2024-03-28T18:04:48.465760+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T03:15:16.346859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"92'7","reported_seq":24,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-28T18:06:10.608689+0000","last_change":"2024-03-28T18:04:49.539053+0000","last_active":"2024-03-28T18:06:10.608689+0000","last_peered":"2024-03-28T18:06:10.608689+0000","last_clean":"2024-03-28T18:06:10.608689+0000","last_became_active":"2024-03-28T18:04:49.537327+0000","last_became_peered":"2024-03-28T18:04:49.537327+0000","last_unstale":"2024-03-28T18:06:10.608689+0000","last_undegraded":"2024-03-28T18:06:10.608689+0000","last_fullsized":"2024-03-28T18:06:10.608689+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_clean_scrub_stamp":"2024-03-28T18:04:48.465760+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T00:24:42.319504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"92'9","reported_seq":26,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-28T18:06:10.625215+0000","last_change":"2024-03-28T18:04:49.525855+0000","last_active":"2024-03-28T18:06:10.625215+0000","last_peered":"2024-03-28T18:06:10.625215+0000","last_clean":"2024-03-28T18:06:10.625215+0000","last_became_active":"2024-03-28T18:04:49.525448+0000","last_became_peered":"2024-03-28T18:04:49.525448+0000","last_unstale":"2024-03-28T18:06:10.625215+0000","last_undegraded":"2024-03-28T18:06:10.625215+0000","last_fullsized":"2024-03-28T18:06:10.625215+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_clean_scrub_stamp":"2024-03-28T18:04:48.465760+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T23:53:00.452692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"92'8","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-28T18:06:10.622056+0000","last_change":"2024-03-28T18:04:49.543605+0000","last_active":"2024-03-28T18:06:10.622056+0000","last_peered":"2024-03-28T18:06:10.622056+0000","last_clean":"2024-03-28T18:06:10.622056+0000","last_became_active":"2024-03-28T18:04:49.543280+0000","last_became_peered":"2024-03-28T18:04:49.543280+0000","last_unstale":"2024-03-28T18:06:10.622056+0000","last_undegraded":"2024-03-28T18:06:10.622056+0000","last_fullsized":"2024-03-28T18:06:10.622056+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_clean_scrub_stamp":"2024-03-28T18:04:48.465760+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T21:27:26.216639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"92'10","reported_seq":22,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-28T18:06:10.600162+0000","last_change":"2024-03-28T18:04:49.526745+0000","last_active":"2024-03-28T18:06:10.600162+0000","last_peered":"2024-03-28T18:06:10.600162+0000","last_clean":"2024-03-28T18:06:10.600162+0000","last_became_active":"2024-03-28T18:04:49.526351+0000","last_became_peered":"2024-03-28T18:04:49.526351+0000","last_unstale":"2024-03-28T18:06:10.600162+0000","last_undegraded":"2024-03-28T18:06:10.600162+0000","last_fullsized":"2024-03-28T18:06:10.600162+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_clean_scrub_stamp":"2024-03-28T18:04:48.465760+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T05:31:47.687345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"92'7","reported_seq":24,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-28T18:06:10.595666+0000","last_change":"2024-03-28T18:04:49.523346+0000","last_active":"2024-03-28T18:06:10.595666+0000","last_peered":"2024-03-28T18:06:10.595666+0000","last_clean":"2024-03-28T18:06:10.595666+0000","last_became_active":"2024-03-28T18:04:49.522041+0000","last_became_peered":"2024-03-28T18:04:49.522041+0000","last_unstale":"2024-03-28T18:06:10.595666+0000","last_undegraded":"2024-03-28T18:06:10.595666+0000","last_fullsized":"2024-03-28T18:06:10.595666+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_clean_scrub_stamp":"2024-03-28T18:04:48.465760+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T01:35:02.419328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"90'13","reported_seq":107,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-28T18:06:08.833028+0000","last_change":"2024-03-28T18:04:48.510917+0000","last_active":"2024-03-28T18:06:08.833028+0000","last_peered":"2024-03-28T18:06:08.833028+0000","last_clean":"2024-03-28T18:06:08.833028+0000","last_became_active":"2024-03-28T18:04:48.509978+0000","last_became_peered":"2024-03-28T18:04:48.509978+0000","last_unstale":"2024-03-28T18:06:08.833028+0000","last_undegraded":"2024-03-28T18:06:08.833028+0000","last_fullsized":"2024-03-28T18:06:08.833028+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:04:47.412563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:04:47.412563+0000","last_clean_scrub_stamp":"2024-03-28T18:04:47.412563+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T03:28:13.185099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":75,"num_read_kb":75,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"92'5","reported_seq":22,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-28T18:06:10.603693+0000","last_change":"2024-03-28T18:04:49.535543+0000","last_active":"2024-03-28T18:06:10.603693+0000","last_peered":"2024-03-28T18:06:10.603693+0000","last_clean":"2024-03-28T18:06:10.603693+0000","last_became_active":"2024-03-28T18:04:49.535171+0000","last_became_peered":"2024-03-28T18:04:49.535171+0000","last_unstale":"2024-03-28T18:06:10.603693+0000","last_undegraded":"2024-03-28T18:06:10.603693+0000","last_fullsized":"2024-03-28T18:06:10.603693+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_clean_scrub_stamp":"2024-03-28T18:04:48.465760+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T22:12:22.258424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"90'2","reported_seq":21,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-28T18:05:11.741041+0000","last_change":"2024-03-28T18:04:48.503190+0000","last_active":"2024-03-28T18:05:11.741041+0000","last_peered":"2024-03-28T18:05:11.741041+0000","last_clean":"2024-03-28T18:05:11.741041+0000","last_became_active":"2024-03-28T18:04:48.501858+0000","last_became_peered":"2024-03-28T18:04:48.501858+0000","last_unstale":"2024-03-28T18:05:11.741041+0000","last_undegraded":"2024-03-28T18:05:11.741041+0000","last_fullsized":"2024-03-28T18:05:11.741041+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:04:47.412563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:04:47.412563+0000","last_clean_scrub_stamp":"2024-03-28T18:04:47.412563+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T19:56:43.878630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"92'6","reported_seq":23,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-28T18:06:10.625778+0000","last_change":"2024-03-28T18:04:49.537739+0000","last_active":"2024-03-28T18:06:10.625778+0000","last_peered":"2024-03-28T18:06:10.625778+0000","last_clean":"2024-03-28T18:06:10.625778+0000","last_became_active":"2024-03-28T18:04:49.535990+0000","last_became_peered":"2024-03-28T18:04:49.535990+0000","last_unstale":"2024-03-28T18:06:10.625778+0000","last_undegraded":"2024-03-28T18:06:10.625778+0000","last_fullsized":"2024-03-28T18:06:10.625778+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_clean_scrub_stamp":"2024-03-28T18:04:48.465760+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T21:54:15.729494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"90'12","reported_seq":27,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-28T18:05:11.768035+0000","last_change":"2024-03-28T18:05:11.767512+0000","last_active":"2024-03-28T18:05:11.768035+0000","last_peered":"2024-03-28T18:05:11.768035+0000","last_clean":"2024-03-28T18:05:11.768035+0000","last_became_active":"2024-03-28T18:05:11.752452+0000","last_became_peered":"2024-03-28T18:05:11.752452+0000","last_unstale":"2024-03-28T18:05:11.768035+0000","last_undegraded":"2024-03-28T18:05:11.768035+0000","last_fullsized":"2024-03-28T18:05:11.768035+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:04:47.412563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:04:47.412563+0000","last_clean_scrub_stamp":"2024-03-28T18:04:47.412563+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T00:09:13.980288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":0,"num_keys_recovered":10,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,7],"acting":[11,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1c","version":"92'9","reported_seq":26,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-28T18:06:10.623876+0000","last_change":"2024-03-28T18:04:49.529316+0000","last_active":"2024-03-28T18:06:10.623876+0000","last_peered":"2024-03-28T18:06:10.623876+0000","last_clean":"2024-03-28T18:06:10.623876+0000","last_became_active":"2024-03-28T18:04:49.528975+0000","last_became_peered":"2024-03-28T18:04:49.528975+0000","last_unstale":"2024-03-28T18:06:10.623876+0000","last_undegraded":"2024-03-28T18:06:10.623876+0000","last_fullsized":"2024-03-28T18:06:10.623876+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_clean_scrub_stamp":"2024-03-28T18:04:48.465760+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T00:06:45.268757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"90'5","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-28T18:05:11.734761+0000","last_change":"2024-03-28T18:04:48.515841+0000","last_active":"2024-03-28T18:05:11.734761+0000","last_peered":"2024-03-28T18:05:11.734761+0000","last_clean":"2024-03-28T18:05:11.734761+0000","last_became_active":"2024-03-28T18:04:48.513852+0000","last_became_peered":"2024-03-28T18:04:48.513852+0000","last_unstale":"2024-03-28T18:05:11.734761+0000","last_undegraded":"2024-03-28T18:05:11.734761+0000","last_fullsized":"2024-03-28T18:05:11.734761+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:04:47.412563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:04:47.412563+0000","last_clean_scrub_stamp":"2024-03-28T18:04:47.412563+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T00:07:14.751545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"92'15","reported_seq":32,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-28T18:06:10.609956+0000","last_change":"2024-03-28T18:04:49.542238+0000","last_active":"2024-03-28T18:06:10.609956+0000","last_peered":"2024-03-28T18:06:10.609956+0000","last_clean":"2024-03-28T18:06:10.609956+0000","last_became_active":"2024-03-28T18:04:49.539723+0000","last_became_peered":"2024-03-28T18:04:49.539723+0000","last_unstale":"2024-03-28T18:06:10.609956+0000","last_undegraded":"2024-03-28T18:06:10.609956+0000","last_fullsized":"2024-03-28T18:06:10.609956+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_clean_scrub_stamp":"2024-03-28T18:04:48.465760+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T22:45:07.310148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"92'7","reported_seq":102,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-28T18:06:09.847936+0000","last_change":"2024-03-28T18:04:48.511349+0000","last_active":"2024-03-28T18:06:09.847936+0000","last_peered":"2024-03-28T18:06:09.847936+0000","last_clean":"2024-03-28T18:06:09.847936+0000","last_became_active":"2024-03-28T18:04:48.510990+0000","last_became_peered":"2024-03-28T18:04:48.510990+0000","last_unstale":"2024-03-28T18:06:09.847936+0000","last_undegraded":"2024-03-28T18:06:09.847936+0000","last_fullsized":"2024-03-28T18:06:09.847936+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:04:47.412563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:04:47.412563+0000","last_clean_scrub_stamp":"2024-03-28T18:04:47.412563+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T21:56:15.411149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1154,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":76,"num_read_kb":2,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"92'7","reported_seq":24,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-28T18:06:10.604922+0000","last_change":"2024-03-28T18:04:49.538066+0000","last_active":"2024-03-28T18:06:10.604922+0000","last_peered":"2024-03-28T18:06:10.604922+0000","last_clean":"2024-03-28T18:06:10.604922+0000","last_became_active":"2024-03-28T18:04:49.537202+0000","last_became_peered":"2024-03-28T18:04:49.537202+0000","last_unstale":"2024-03-28T18:06:10.604922+0000","last_undegraded":"2024-03-28T18:06:10.604922+0000","last_fullsized":"2024-03-28T18:06:10.604922+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_clean_scrub_stamp":"2024-03-28T18:04:48.465760+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T22:35:19.060311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"90'7","reported_seq":26,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-28T18:05:11.732185+0000","last_change":"2024-03-28T18:04:48.513722+0000","last_active":"2024-03-28T18:05:11.732185+0000","last_peered":"2024-03-28T18:05:11.732185+0000","last_clean":"2024-03-28T18:05:11.732185+0000","last_became_active":"2024-03-28T18:04:48.513048+0000","last_became_peered":"2024-03-28T18:04:48.513048+0000","last_unstale":"2024-03-28T18:05:11.732185+0000","last_undegraded":"2024-03-28T18:05:11.732185+0000","last_fullsized":"2024-03-28T18:05:11.732185+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:04:47.412563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:04:47.412563+0000","last_clean_scrub_stamp":"2024-03-28T18:04:47.412563+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T04:48:10.831297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"92'8","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-28T18:06:10.612752+0000","last_change":"2024-03-28T18:04:49.539172+0000","last_active":"2024-03-28T18:06:10.612752+0000","last_peered":"2024-03-28T18:06:10.612752+0000","last_clean":"2024-03-28T18:06:10.612752+0000","last_became_active":"2024-03-28T18:04:49.538779+0000","last_became_peered":"2024-03-28T18:04:49.538779+0000","last_unstale":"2024-03-28T18:06:10.612752+0000","last_undegraded":"2024-03-28T18:06:10.612752+0000","last_fullsized":"2024-03-28T18:06:10.612752+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_clean_scrub_stamp":"2024-03-28T18:04:48.465760+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T04:59:08.361762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"90'3","reported_seq":22,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-28T18:05:11.949905+0000","last_change":"2024-03-28T18:04:48.500471+0000","last_active":"2024-03-28T18:05:11.949905+0000","last_peered":"2024-03-28T18:05:11.949905+0000","last_clean":"2024-03-28T18:05:11.949905+0000","last_became_active":"2024-03-28T18:04:48.500094+0000","last_became_peered":"2024-03-28T18:04:48.500094+0000","last_unstale":"2024-03-28T18:05:11.949905+0000","last_undegraded":"2024-03-28T18:05:11.949905+0000","last_fullsized":"2024-03-28T18:05:11.949905+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:04:47.412563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:04:47.412563+0000","last_clean_scrub_stamp":"2024-03-28T18:04:47.412563+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T21:52:51.964254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"92'8","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-28T18:06:10.600231+0000","last_change":"2024-03-28T18:04:49.522120+0000","last_active":"2024-03-28T18:06:10.600231+0000","last_peered":"2024-03-28T18:06:10.600231+0000","last_clean":"2024-03-28T18:06:10.600231+0000","last_became_active":"2024-03-28T18:04:49.521656+0000","last_became_peered":"2024-03-28T18:04:49.521656+0000","last_unstale":"2024-03-28T18:06:10.600231+0000","last_undegraded":"2024-03-28T18:06:10.600231+0000","last_fullsized":"2024-03-28T18:06:10.600231+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_clean_scrub_stamp":"2024-03-28T18:04:48.465760+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T18:29:55.605765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"90'3","reported_seq":23,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-28T18:05:11.740918+0000","last_change":"2024-03-28T18:04:48.508723+0000","last_active":"2024-03-28T18:05:11.740918+0000","last_peered":"2024-03-28T18:05:11.740918+0000","last_clean":"2024-03-28T18:05:11.740918+0000","last_became_active":"2024-03-28T18:04:48.508314+0000","last_became_peered":"2024-03-28T18:04:48.508314+0000","last_unstale":"2024-03-28T18:05:11.740918+0000","last_undegraded":"2024-03-28T18:05:11.740918+0000","last_fullsized":"2024-03-28T18:05:11.740918+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:04:47.412563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:04:47.412563+0000","last_clean_scrub_stamp":"2024-03-28T18:04:47.412563+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T03:03:59.353454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"92'11","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-28T18:06:10.602791+0000","last_change":"2024-03-28T18:04:49.531469+0000","last_active":"2024-03-28T18:06:10.602791+0000","last_peered":"2024-03-28T18:06:10.602791+0000","last_clean":"2024-03-28T18:06:10.602791+0000","last_became_active":"2024-03-28T18:04:49.529263+0000","last_became_peered":"2024-03-28T18:04:49.529263+0000","last_unstale":"2024-03-28T18:06:10.602791+0000","last_undegraded":"2024-03-28T18:06:10.602791+0000","last_fullsized":"2024-03-28T18:06:10.602791+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_clean_scrub_stamp":"2024-03-28T18:04:48.465760+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T05:46:31.593877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"90'4","reported_seq":171,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-28T18:06:09.846555+0000","last_change":"2024-03-28T18:04:48.500644+0000","last_active":"2024-03-28T18:06:09.846555+0000","last_peered":"2024-03-28T18:06:09.846555+0000","last_clean":"2024-03-28T18:06:09.846555+0000","last_became_active":"2024-03-28T18:04:48.499330+0000","last_became_peered":"2024-03-28T18:04:48.499330+0000","last_unstale":"2024-03-28T18:06:09.846555+0000","last_undegraded":"2024-03-28T18:06:09.846555+0000","last_fullsized":"2024-03-28T18:06:09.846555+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:04:47.412563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:04:47.412563+0000","last_clean_scrub_stamp":"2024-03-28T18:04:47.412563+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T18:50:54.361266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":148,"num_read_kb":148,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"92'10","reported_seq":27,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-28T18:06:10.623296+0000","last_change":"2024-03-28T18:04:49.528728+0000","last_active":"2024-03-28T18:06:10.623296+0000","last_peered":"2024-03-28T18:06:10.623296+0000","last_clean":"2024-03-28T18:06:10.623296+0000","last_became_active":"2024-03-28T18:04:49.528312+0000","last_became_peered":"2024-03-28T18:04:49.528312+0000","last_unstale":"2024-03-28T18:06:10.623296+0000","last_undegraded":"2024-03-28T18:06:10.623296+0000","last_fullsized":"2024-03-28T18:06:10.623296+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_clean_scrub_stamp":"2024-03-28T18:04:48.465760+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T05:59:37.308002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"90'4","reported_seq":23,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-28T18:05:11.736858+0000","last_change":"2024-03-28T18:04:48.508766+0000","last_active":"2024-03-28T18:05:11.736858+0000","last_peered":"2024-03-28T18:05:11.736858+0000","last_clean":"2024-03-28T18:05:11.736858+0000","last_became_active":"2024-03-28T18:04:48.506780+0000","last_became_peered":"2024-03-28T18:04:48.506780+0000","last_unstale":"2024-03-28T18:05:11.736858+0000","last_undegraded":"2024-03-28T18:05:11.736858+0000","last_fullsized":"2024-03-28T18:05:11.736858+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:04:47.412563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:04:47.412563+0000","last_clean_scrub_stamp":"2024-03-28T18:04:47.412563+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T04:39:07.394655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"92'9","reported_seq":26,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-28T18:06:10.605283+0000","last_change":"2024-03-28T18:04:49.537165+0000","last_active":"2024-03-28T18:06:10.605283+0000","last_peered":"2024-03-28T18:06:10.605283+0000","last_clean":"2024-03-28T18:06:10.605283+0000","last_became_active":"2024-03-28T18:04:49.534543+0000","last_became_peered":"2024-03-28T18:04:49.534543+0000","last_unstale":"2024-03-28T18:06:10.605283+0000","last_undegraded":"2024-03-28T18:06:10.605283+0000","last_fullsized":"2024-03-28T18:06:10.605283+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_clean_scrub_stamp":"2024-03-28T18:04:48.465760+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T03:15:58.056148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"90'5","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-28T18:05:11.746325+0000","last_change":"2024-03-28T18:04:48.503866+0000","last_active":"2024-03-28T18:05:11.746325+0000","last_peered":"2024-03-28T18:05:11.746325+0000","last_clean":"2024-03-28T18:05:11.746325+0000","last_became_active":"2024-03-28T18:04:48.503223+0000","last_became_peered":"2024-03-28T18:04:48.503223+0000","last_unstale":"2024-03-28T18:05:11.746325+0000","last_undegraded":"2024-03-28T18:05:11.746325+0000","last_fullsized":"2024-03-28T18:05:11.746325+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:04:47.412563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:04:47.412563+0000","last_clean_scrub_stamp":"2024-03-28T18:04:47.412563+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T20:14:01.307483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"92'11","reported_seq":28,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-28T18:06:10.624541+0000","last_change":"2024-03-28T18:04:49.539494+0000","last_active":"2024-03-28T18:06:10.624541+0000","last_peered":"2024-03-28T18:06:10.624541+0000","last_clean":"2024-03-28T18:06:10.624541+0000","last_became_active":"2024-03-28T18:04:49.539044+0000","last_became_peered":"2024-03-28T18:04:49.539044+0000","last_unstale":"2024-03-28T18:06:10.624541+0000","last_undegraded":"2024-03-28T18:06:10.624541+0000","last_fullsized":"2024-03-28T18:06:10.624541+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_clean_scrub_stamp":"2024-03-28T18:04:48.465760+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T04:08:30.807788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"92'28","reported_seq":201,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-28T18:06:09.846766+0000","last_change":"2024-03-28T18:04:48.507703+0000","last_active":"2024-03-28T18:06:09.846766+0000","last_peered":"2024-03-28T18:06:09.846766+0000","last_clean":"2024-03-28T18:06:09.846766+0000","last_became_active":"2024-03-28T18:04:48.507310+0000","last_became_peered":"2024-03-28T18:04:48.507310+0000","last_unstale":"2024-03-28T18:06:09.846766+0000","last_undegraded":"2024-03-28T18:06:09.846766+0000","last_fullsized":"2024-03-28T18:06:09.846766+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:04:47.412563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:04:47.412563+0000","last_clean_scrub_stamp":"2024-03-28T18:04:47.412563+0000","objects_scrubbed":0,"log_size":28,"log_dups_size":0,"ondisk_log_size":28,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T19:14:13.910042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":59691,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":154,"num_read_kb":135,"num_write":30,"num_write_kb":67,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"92'6","reported_seq":23,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-28T18:06:10.618239+0000","last_change":"2024-03-28T18:04:49.529928+0000","last_active":"2024-03-28T18:06:10.618239+0000","last_peered":"2024-03-28T18:06:10.618239+0000","last_clean":"2024-03-28T18:06:10.618239+0000","last_became_active":"2024-03-28T18:04:49.529527+0000","last_became_peered":"2024-03-28T18:04:49.529527+0000","last_unstale":"2024-03-28T18:06:10.618239+0000","last_undegraded":"2024-03-28T18:06:10.618239+0000","last_fullsized":"2024-03-28T18:06:10.618239+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_clean_scrub_stamp":"2024-03-28T18:04:48.465760+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T05:50:07.129140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"90'9","reported_seq":30,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-28T18:05:11.746751+0000","last_change":"2024-03-28T18:04:48.494746+0000","last_active":"2024-03-28T18:05:11.746751+0000","last_peered":"2024-03-28T18:05:11.746751+0000","last_clean":"2024-03-28T18:05:11.746751+0000","last_became_active":"2024-03-28T18:04:48.494296+0000","last_became_peered":"2024-03-28T18:04:48.494296+0000","last_unstale":"2024-03-28T18:05:11.746751+0000","last_undegraded":"2024-03-28T18:05:11.746751+0000","last_fullsized":"2024-03-28T18:05:11.746751+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:04:47.412563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:04:47.412563+0000","last_clean_scrub_stamp":"2024-03-28T18:04:47.412563+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T22:32:56.923067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"92'9","reported_seq":26,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-28T18:06:10.624132+0000","last_change":"2024-03-28T18:04:49.528070+0000","last_active":"2024-03-28T18:06:10.624132+0000","last_peered":"2024-03-28T18:06:10.624132+0000","last_clean":"2024-03-28T18:06:10.624132+0000","last_became_active":"2024-03-28T18:04:49.526747+0000","last_became_peered":"2024-03-28T18:04:49.526747+0000","last_unstale":"2024-03-28T18:06:10.624132+0000","last_undegraded":"2024-03-28T18:06:10.624132+0000","last_fullsized":"2024-03-28T18:06:10.624132+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_clean_scrub_stamp":"2024-03-28T18:04:48.465760+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T22:32:05.750094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"90'5","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-28T18:05:11.741286+0000","last_change":"2024-03-28T18:04:48.505723+0000","last_active":"2024-03-28T18:05:11.741286+0000","last_peered":"2024-03-28T18:05:11.741286+0000","last_clean":"2024-03-28T18:05:11.741286+0000","last_became_active":"2024-03-28T18:04:48.504088+0000","last_became_peered":"2024-03-28T18:04:48.504088+0000","last_unstale":"2024-03-28T18:05:11.741286+0000","last_undegraded":"2024-03-28T18:05:11.741286+0000","last_fullsized":"2024-03-28T18:05:11.741286+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:04:47.412563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:04:47.412563+0000","last_clean_scrub_stamp":"2024-03-28T18:04:47.412563+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T23:25:37.138054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"92'15","reported_seq":32,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-28T18:06:10.625128+0000","last_change":"2024-03-28T18:04:49.518872+0000","last_active":"2024-03-28T18:06:10.625128+0000","last_peered":"2024-03-28T18:06:10.625128+0000","last_clean":"2024-03-28T18:06:10.625128+0000","last_became_active":"2024-03-28T18:04:49.518507+0000","last_became_peered":"2024-03-28T18:04:49.518507+0000","last_unstale":"2024-03-28T18:06:10.625128+0000","last_undegraded":"2024-03-28T18:06:10.625128+0000","last_fullsized":"2024-03-28T18:06:10.625128+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_clean_scrub_stamp":"2024-03-28T18:04:48.465760+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T20:11:31.044321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"92'7","reported_seq":104,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-28T18:06:10.850137+0000","last_change":"2024-03-28T18:04:48.503058+0000","last_active":"2024-03-28T18:06:10.850137+0000","last_peered":"2024-03-28T18:06:10.850137+0000","last_clean":"2024-03-28T18:06:10.850137+0000","last_became_active":"2024-03-28T18:04:48.501596+0000","last_became_peered":"2024-03-28T18:04:48.501596+0000","last_unstale":"2024-03-28T18:06:10.850137+0000","last_undegraded":"2024-03-28T18:06:10.850137+0000","last_fullsized":"2024-03-28T18:06:10.850137+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:04:47.412563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:04:47.412563+0000","last_clean_scrub_stamp":"2024-03-28T18:04:47.412563+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T23:46:10.442771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1120,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":79,"num_read_kb":4,"num_write":7,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"92'6","reported_seq":23,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-28T18:06:10.614297+0000","last_change":"2024-03-28T18:04:49.535102+0000","last_active":"2024-03-28T18:06:10.614297+0000","last_peered":"2024-03-28T18:06:10.614297+0000","last_clean":"2024-03-28T18:06:10.614297+0000","last_became_active":"2024-03-28T18:04:49.534171+0000","last_became_peered":"2024-03-28T18:04:49.534171+0000","last_unstale":"2024-03-28T18:06:10.614297+0000","last_undegraded":"2024-03-28T18:06:10.614297+0000","last_fullsized":"2024-03-28T18:06:10.614297+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_clean_scrub_stamp":"2024-03-28T18:04:48.465760+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T18:16:27.703327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"90'17","reported_seq":37,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-28T18:05:11.734179+0000","last_change":"2024-03-28T18:04:48.517034+0000","last_active":"2024-03-28T18:05:11.734179+0000","last_peered":"2024-03-28T18:05:11.734179+0000","last_clean":"2024-03-28T18:05:11.734179+0000","last_became_active":"2024-03-28T18:04:48.516562+0000","last_became_peered":"2024-03-28T18:04:48.516562+0000","last_unstale":"2024-03-28T18:05:11.734179+0000","last_undegraded":"2024-03-28T18:05:11.734179+0000","last_fullsized":"2024-03-28T18:05:11.734179+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:04:47.412563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:04:47.412563+0000","last_clean_scrub_stamp":"2024-03-28T18:04:47.412563+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T02:28:55.753779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"92'6","reported_seq":23,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-28T18:06:10.609902+0000","last_change":"2024-03-28T18:04:49.541501+0000","last_active":"2024-03-28T18:06:10.609902+0000","last_peered":"2024-03-28T18:06:10.609902+0000","last_clean":"2024-03-28T18:06:10.609902+0000","last_became_active":"2024-03-28T18:04:49.541117+0000","last_became_peered":"2024-03-28T18:04:49.541117+0000","last_unstale":"2024-03-28T18:06:10.609902+0000","last_undegraded":"2024-03-28T18:06:10.609902+0000","last_fullsized":"2024-03-28T18:06:10.609902+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_clean_scrub_stamp":"2024-03-28T18:04:48.465760+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T04:18:29.081489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"90'2","reported_seq":22,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-28T18:05:11.735202+0000","last_change":"2024-03-28T18:04:48.510859+0000","last_active":"2024-03-28T18:05:11.735202+0000","last_peered":"2024-03-28T18:05:11.735202+0000","last_clean":"2024-03-28T18:05:11.735202+0000","last_became_active":"2024-03-28T18:04:48.510522+0000","last_became_peered":"2024-03-28T18:04:48.510522+0000","last_unstale":"2024-03-28T18:05:11.735202+0000","last_undegraded":"2024-03-28T18:05:11.735202+0000","last_fullsized":"2024-03-28T18:05:11.735202+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:04:47.412563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:04:47.412563+0000","last_clean_scrub_stamp":"2024-03-28T18:04:47.412563+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T22:31:37.817361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"92'5","reported_seq":22,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-28T18:06:10.605268+0000","last_change":"2024-03-28T18:04:49.541981+0000","last_active":"2024-03-28T18:06:10.605268+0000","last_peered":"2024-03-28T18:06:10.605268+0000","last_clean":"2024-03-28T18:06:10.605268+0000","last_became_active":"2024-03-28T18:04:49.541606+0000","last_became_peered":"2024-03-28T18:04:49.541606+0000","last_unstale":"2024-03-28T18:06:10.605268+0000","last_undegraded":"2024-03-28T18:06:10.605268+0000","last_fullsized":"2024-03-28T18:06:10.605268+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_clean_scrub_stamp":"2024-03-28T18:04:48.465760+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T21:56:03.632490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"90'1","reported_seq":20,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-28T18:05:11.730832+0000","last_change":"2024-03-28T18:04:48.505847+0000","last_active":"2024-03-28T18:05:11.730832+0000","last_peered":"2024-03-28T18:05:11.730832+0000","last_clean":"2024-03-28T18:05:11.730832+0000","last_became_active":"2024-03-28T18:04:48.501005+0000","last_became_peered":"2024-03-28T18:04:48.501005+0000","last_unstale":"2024-03-28T18:05:11.730832+0000","last_undegraded":"2024-03-28T18:05:11.730832+0000","last_fullsized":"2024-03-28T18:05:11.730832+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:04:47.412563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:04:47.412563+0000","last_clean_scrub_stamp":"2024-03-28T18:04:47.412563+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T01:33:45.539324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"90'1","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-28T18:05:11.731254+0000","last_change":"2024-03-28T18:04:49.526938+0000","last_active":"2024-03-28T18:05:11.731254+0000","last_peered":"2024-03-28T18:05:11.731254+0000","last_clean":"2024-03-28T18:05:11.731254+0000","last_became_active":"2024-03-28T18:04:49.526290+0000","last_became_peered":"2024-03-28T18:04:49.526290+0000","last_unstale":"2024-03-28T18:05:11.731254+0000","last_undegraded":"2024-03-28T18:05:11.731254+0000","last_fullsized":"2024-03-28T18:05:11.731254+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_clean_scrub_stamp":"2024-03-28T18:04:48.465760+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T03:06:54.561470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"90'5","reported_seq":101,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-28T18:06:10.839144+0000","last_change":"2024-03-28T18:04:48.512553+0000","last_active":"2024-03-28T18:06:10.839144+0000","last_peered":"2024-03-28T18:06:10.839144+0000","last_clean":"2024-03-28T18:06:10.839144+0000","last_became_active":"2024-03-28T18:04:48.511520+0000","last_became_peered":"2024-03-28T18:04:48.511520+0000","last_unstale":"2024-03-28T18:06:10.839144+0000","last_undegraded":"2024-03-28T18:06:10.839144+0000","last_fullsized":"2024-03-28T18:06:10.839144+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:04:47.412563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:04:47.412563+0000","last_clean_scrub_stamp":"2024-03-28T18:04:47.412563+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T21:13:32.970027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":77,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":23,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-28T18:05:11.739596+0000","last_change":"2024-03-28T18:05:11.739596+0000","last_active":"2024-03-28T18:05:11.739596+0000","last_peered":"2024-03-28T18:05:11.739596+0000","last_clean":"2024-03-28T18:05:11.739596+0000","last_became_active":"2024-03-28T18:05:11.739156+0000","last_became_peered":"2024-03-28T18:05:11.739156+0000","last_unstale":"2024-03-28T18:05:11.739596+0000","last_undegraded":"2024-03-28T18:05:11.739596+0000","last_fullsized":"2024-03-28T18:05:11.739596+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_clean_scrub_stamp":"2024-03-28T18:04:48.465760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T05:23:00.446713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,0],"acting":[10,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"92'8","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-28T18:06:10.609312+0000","last_change":"2024-03-28T18:04:49.540438+0000","last_active":"2024-03-28T18:06:10.609312+0000","last_peered":"2024-03-28T18:06:10.609312+0000","last_clean":"2024-03-28T18:06:10.609312+0000","last_became_active":"2024-03-28T18:04:49.540069+0000","last_became_peered":"2024-03-28T18:04:49.540069+0000","last_unstale":"2024-03-28T18:06:10.609312+0000","last_undegraded":"2024-03-28T18:06:10.609312+0000","last_fullsized":"2024-03-28T18:06:10.609312+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_clean_scrub_stamp":"2024-03-28T18:04:48.465760+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T19:31:49.254101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"92'6","reported_seq":100,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-28T18:06:09.846771+0000","last_change":"2024-03-28T18:04:48.508839+0000","last_active":"2024-03-28T18:06:09.846771+0000","last_peered":"2024-03-28T18:06:09.846771+0000","last_clean":"2024-03-28T18:06:09.846771+0000","last_became_active":"2024-03-28T18:04:48.508475+0000","last_became_peered":"2024-03-28T18:04:48.508475+0000","last_unstale":"2024-03-28T18:06:09.846771+0000","last_undegraded":"2024-03-28T18:06:09.846771+0000","last_fullsized":"2024-03-28T18:06:09.846771+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:04:47.412563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:04:47.412563+0000","last_clean_scrub_stamp":"2024-03-28T18:04:47.412563+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T04:31:10.463284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":75,"num_read_kb":75,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"92'8","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-28T18:06:10.621649+0000","last_change":"2024-03-28T18:04:49.534449+0000","last_active":"2024-03-28T18:06:10.621649+0000","last_peered":"2024-03-28T18:06:10.621649+0000","last_clean":"2024-03-28T18:06:10.621649+0000","last_became_active":"2024-03-28T18:04:49.534098+0000","last_became_peered":"2024-03-28T18:04:49.534098+0000","last_unstale":"2024-03-28T18:06:10.621649+0000","last_undegraded":"2024-03-28T18:06:10.621649+0000","last_fullsized":"2024-03-28T18:06:10.621649+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_clean_scrub_stamp":"2024-03-28T18:04:48.465760+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T23:47:07.886931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"90'6","reported_seq":28,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-28T18:05:11.948329+0000","last_change":"2024-03-28T18:04:48.501790+0000","last_active":"2024-03-28T18:05:11.948329+0000","last_peered":"2024-03-28T18:05:11.948329+0000","last_clean":"2024-03-28T18:05:11.948329+0000","last_became_active":"2024-03-28T18:04:48.501415+0000","last_became_peered":"2024-03-28T18:04:48.501415+0000","last_unstale":"2024-03-28T18:05:11.948329+0000","last_undegraded":"2024-03-28T18:05:11.948329+0000","last_fullsized":"2024-03-28T18:05:11.948329+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:04:47.412563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:04:47.412563+0000","last_clean_scrub_stamp":"2024-03-28T18:04:47.412563+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T19:17:22.319883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"22'137","reported_seq":366,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-28T18:05:11.735054+0000","last_change":"2024-03-28T18:03:13.522588+0000","last_active":"2024-03-28T18:05:11.735054+0000","last_peered":"2024-03-28T18:05:11.735054+0000","last_clean":"2024-03-28T18:05:11.735054+0000","last_became_active":"2024-03-28T18:03:13.521185+0000","last_became_peered":"2024-03-28T18:03:13.521185+0000","last_unstale":"2024-03-28T18:05:11.735054+0000","last_undegraded":"2024-03-28T18:05:11.735054+0000","last_fullsized":"2024-03-28T18:05:11.735054+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T17:58:33.169981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T17:58:33.169981+0000","last_clean_scrub_stamp":"2024-03-28T17:58:33.169981+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T19:10:06.087374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"92'5","reported_seq":22,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-28T18:06:10.606883+0000","last_change":"2024-03-28T18:04:49.538619+0000","last_active":"2024-03-28T18:06:10.606883+0000","last_peered":"2024-03-28T18:06:10.606883+0000","last_clean":"2024-03-28T18:06:10.606883+0000","last_became_active":"2024-03-28T18:04:49.538276+0000","last_became_peered":"2024-03-28T18:04:49.538276+0000","last_unstale":"2024-03-28T18:06:10.606883+0000","last_undegraded":"2024-03-28T18:06:10.606883+0000","last_fullsized":"2024-03-28T18:06:10.606883+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_clean_scrub_stamp":"2024-03-28T18:04:48.465760+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T01:39:37.139544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"90'4","reported_seq":23,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-28T18:05:11.735171+0000","last_change":"2024-03-28T18:04:48.513621+0000","last_active":"2024-03-28T18:05:11.735171+0000","last_peered":"2024-03-28T18:05:11.735171+0000","last_clean":"2024-03-28T18:05:11.735171+0000","last_became_active":"2024-03-28T18:04:48.513019+0000","last_became_peered":"2024-03-28T18:04:48.513019+0000","last_unstale":"2024-03-28T18:05:11.735171+0000","last_undegraded":"2024-03-28T18:05:11.735171+0000","last_fullsized":"2024-03-28T18:05:11.735171+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:04:47.412563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:04:47.412563+0000","last_clean_scrub_stamp":"2024-03-28T18:04:47.412563+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T23:38:18.082830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"90'4","reported_seq":24,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-28T18:05:11.734872+0000","last_change":"2024-03-28T18:04:48.513999+0000","last_active":"2024-03-28T18:05:11.734872+0000","last_peered":"2024-03-28T18:05:11.734872+0000","last_clean":"2024-03-28T18:05:11.734872+0000","last_became_active":"2024-03-28T18:04:48.512836+0000","last_became_peered":"2024-03-28T18:04:48.512836+0000","last_unstale":"2024-03-28T18:05:11.734872+0000","last_undegraded":"2024-03-28T18:05:11.734872+0000","last_fullsized":"2024-03-28T18:05:11.734872+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:04:47.412563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:04:47.412563+0000","last_clean_scrub_stamp":"2024-03-28T18:04:47.412563+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T18:29:42.015843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"92'9","reported_seq":26,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-28T18:06:10.601878+0000","last_change":"2024-03-28T18:04:49.519323+0000","last_active":"2024-03-28T18:06:10.601878+0000","last_peered":"2024-03-28T18:06:10.601878+0000","last_clean":"2024-03-28T18:06:10.601878+0000","last_became_active":"2024-03-28T18:04:49.518898+0000","last_became_peered":"2024-03-28T18:04:49.518898+0000","last_unstale":"2024-03-28T18:06:10.601878+0000","last_undegraded":"2024-03-28T18:06:10.601878+0000","last_fullsized":"2024-03-28T18:06:10.601878+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_clean_scrub_stamp":"2024-03-28T18:04:48.465760+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T18:41:12.683606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"90'13","reported_seq":32,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-28T18:05:11.730909+0000","last_change":"2024-03-28T18:04:48.512673+0000","last_active":"2024-03-28T18:05:11.730909+0000","last_peered":"2024-03-28T18:05:11.730909+0000","last_clean":"2024-03-28T18:05:11.730909+0000","last_became_active":"2024-03-28T18:04:48.512137+0000","last_became_peered":"2024-03-28T18:04:48.512137+0000","last_unstale":"2024-03-28T18:05:11.730909+0000","last_undegraded":"2024-03-28T18:05:11.730909+0000","last_fullsized":"2024-03-28T18:05:11.730909+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:04:47.412563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:04:47.412563+0000","last_clean_scrub_stamp":"2024-03-28T18:04:47.412563+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T21:29:41.731701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":22,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"92'5","reported_seq":22,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-28T18:06:10.624287+0000","last_change":"2024-03-28T18:04:49.525055+0000","last_active":"2024-03-28T18:06:10.624287+0000","last_peered":"2024-03-28T18:06:10.624287+0000","last_clean":"2024-03-28T18:06:10.624287+0000","last_became_active":"2024-03-28T18:04:49.522038+0000","last_became_peered":"2024-03-28T18:04:49.522038+0000","last_unstale":"2024-03-28T18:06:10.624287+0000","last_undegraded":"2024-03-28T18:06:10.624287+0000","last_fullsized":"2024-03-28T18:06:10.624287+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_clean_scrub_stamp":"2024-03-28T18:04:48.465760+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T05:21:44.970397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"90'7","reported_seq":27,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-28T18:05:11.949039+0000","last_change":"2024-03-28T18:04:48.509951+0000","last_active":"2024-03-28T18:05:11.949039+0000","last_peered":"2024-03-28T18:05:11.949039+0000","last_clean":"2024-03-28T18:05:11.949039+0000","last_became_active":"2024-03-28T18:04:48.509141+0000","last_became_peered":"2024-03-28T18:04:48.509141+0000","last_unstale":"2024-03-28T18:05:11.949039+0000","last_undegraded":"2024-03-28T18:05:11.949039+0000","last_fullsized":"2024-03-28T18:05:11.949039+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:04:47.412563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:04:47.412563+0000","last_clean_scrub_stamp":"2024-03-28T18:04:47.412563+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T03:54:05.569940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"92'7","reported_seq":24,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-28T18:06:10.612641+0000","last_change":"2024-03-28T18:04:49.537174+0000","last_active":"2024-03-28T18:06:10.612641+0000","last_peered":"2024-03-28T18:06:10.612641+0000","last_clean":"2024-03-28T18:06:10.612641+0000","last_became_active":"2024-03-28T18:04:49.536801+0000","last_became_peered":"2024-03-28T18:04:49.536801+0000","last_unstale":"2024-03-28T18:06:10.612641+0000","last_undegraded":"2024-03-28T18:06:10.612641+0000","last_fullsized":"2024-03-28T18:06:10.612641+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_clean_scrub_stamp":"2024-03-28T18:04:48.465760+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T05:27:55.908196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"90'2","reported_seq":31,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-28T18:05:11.756894+0000","last_change":"2024-03-28T18:05:11.756894+0000","last_active":"2024-03-28T18:05:11.756894+0000","last_peered":"2024-03-28T18:05:11.756894+0000","last_clean":"2024-03-28T18:05:11.756894+0000","last_became_active":"2024-03-28T18:05:11.750825+0000","last_became_peered":"2024-03-28T18:05:11.750825+0000","last_unstale":"2024-03-28T18:05:11.756894+0000","last_undegraded":"2024-03-28T18:05:11.756894+0000","last_fullsized":"2024-03-28T18:05:11.756894+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:04:47.412563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:04:47.412563+0000","last_clean_scrub_stamp":"2024-03-28T18:04:47.412563+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T23:11:42.105850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,11,0],"acting":[1,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"92'2","reported_seq":19,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-28T18:06:10.566718+0000","last_change":"2024-03-28T18:04:49.523970+0000","last_active":"2024-03-28T18:06:10.566718+0000","last_peered":"2024-03-28T18:06:10.566718+0000","last_clean":"2024-03-28T18:06:10.566718+0000","last_became_active":"2024-03-28T18:04:49.523559+0000","last_became_peered":"2024-03-28T18:04:49.523559+0000","last_unstale":"2024-03-28T18:06:10.566718+0000","last_undegraded":"2024-03-28T18:06:10.566718+0000","last_fullsized":"2024-03-28T18:06:10.566718+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_clean_scrub_stamp":"2024-03-28T18:04:48.465760+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T05:50:25.593232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"90'7","reported_seq":26,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-28T18:05:11.739173+0000","last_change":"2024-03-28T18:04:48.492067+0000","last_active":"2024-03-28T18:05:11.739173+0000","last_peered":"2024-03-28T18:05:11.739173+0000","last_clean":"2024-03-28T18:05:11.739173+0000","last_became_active":"2024-03-28T18:04:48.490633+0000","last_became_peered":"2024-03-28T18:04:48.490633+0000","last_unstale":"2024-03-28T18:05:11.739173+0000","last_undegraded":"2024-03-28T18:05:11.739173+0000","last_fullsized":"2024-03-28T18:05:11.739173+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:04:47.412563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:04:47.412563+0000","last_clean_scrub_stamp":"2024-03-28T18:04:47.412563+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T23:00:52.703618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"92'9","reported_seq":26,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-28T18:06:10.608491+0000","last_change":"2024-03-28T18:04:49.531554+0000","last_active":"2024-03-28T18:06:10.608491+0000","last_peered":"2024-03-28T18:06:10.608491+0000","last_clean":"2024-03-28T18:06:10.608491+0000","last_became_active":"2024-03-28T18:04:49.530273+0000","last_became_peered":"2024-03-28T18:04:49.530273+0000","last_unstale":"2024-03-28T18:06:10.608491+0000","last_undegraded":"2024-03-28T18:06:10.608491+0000","last_fullsized":"2024-03-28T18:06:10.608491+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_clean_scrub_stamp":"2024-03-28T18:04:48.465760+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T22:17:04.032614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"90'3","reported_seq":22,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-28T18:05:11.743439+0000","last_change":"2024-03-28T18:04:48.514974+0000","last_active":"2024-03-28T18:05:11.743439+0000","last_peered":"2024-03-28T18:05:11.743439+0000","last_clean":"2024-03-28T18:05:11.743439+0000","last_became_active":"2024-03-28T18:04:48.514356+0000","last_became_peered":"2024-03-28T18:04:48.514356+0000","last_unstale":"2024-03-28T18:05:11.743439+0000","last_undegraded":"2024-03-28T18:05:11.743439+0000","last_fullsized":"2024-03-28T18:05:11.743439+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:04:47.412563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:04:47.412563+0000","last_clean_scrub_stamp":"2024-03-28T18:04:47.412563+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T01:26:37.202440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"92'8","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-28T18:06:10.612963+0000","last_change":"2024-03-28T18:04:49.518064+0000","last_active":"2024-03-28T18:06:10.612963+0000","last_peered":"2024-03-28T18:06:10.612963+0000","last_clean":"2024-03-28T18:06:10.612963+0000","last_became_active":"2024-03-28T18:04:49.517674+0000","last_became_peered":"2024-03-28T18:04:49.517674+0000","last_unstale":"2024-03-28T18:06:10.612963+0000","last_undegraded":"2024-03-28T18:06:10.612963+0000","last_fullsized":"2024-03-28T18:06:10.612963+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_clean_scrub_stamp":"2024-03-28T18:04:48.465760+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T00:10:44.537164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"90'4","reported_seq":24,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-28T18:05:11.734723+0000","last_change":"2024-03-28T18:04:48.517028+0000","last_active":"2024-03-28T18:05:11.734723+0000","last_peered":"2024-03-28T18:05:11.734723+0000","last_clean":"2024-03-28T18:05:11.734723+0000","last_became_active":"2024-03-28T18:04:48.516387+0000","last_became_peered":"2024-03-28T18:04:48.516387+0000","last_unstale":"2024-03-28T18:05:11.734723+0000","last_undegraded":"2024-03-28T18:05:11.734723+0000","last_fullsized":"2024-03-28T18:05:11.734723+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:04:47.412563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:04:47.412563+0000","last_clean_scrub_stamp":"2024-03-28T18:04:47.412563+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T01:09:24.142422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"92'5","reported_seq":22,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-28T18:06:10.625708+0000","last_change":"2024-03-28T18:04:49.527032+0000","last_active":"2024-03-28T18:06:10.625708+0000","last_peered":"2024-03-28T18:06:10.625708+0000","last_clean":"2024-03-28T18:06:10.625708+0000","last_became_active":"2024-03-28T18:04:49.523888+0000","last_became_peered":"2024-03-28T18:04:49.523888+0000","last_unstale":"2024-03-28T18:06:10.625708+0000","last_undegraded":"2024-03-28T18:06:10.625708+0000","last_fullsized":"2024-03-28T18:06:10.625708+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_clean_scrub_stamp":"2024-03-28T18:04:48.465760+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T01:19:01.265719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"90'5","reported_seq":102,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-28T18:06:10.850505+0000","last_change":"2024-03-28T18:04:48.502638+0000","last_active":"2024-03-28T18:06:10.850505+0000","last_peered":"2024-03-28T18:06:10.850505+0000","last_clean":"2024-03-28T18:06:10.850505+0000","last_became_active":"2024-03-28T18:04:48.502249+0000","last_became_peered":"2024-03-28T18:04:48.502249+0000","last_unstale":"2024-03-28T18:06:10.850505+0000","last_undegraded":"2024-03-28T18:06:10.850505+0000","last_fullsized":"2024-03-28T18:06:10.850505+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:04:47.412563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:04:47.412563+0000","last_clean_scrub_stamp":"2024-03-28T18:04:47.412563+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T03:23:51.740832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1170,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":78,"num_read_kb":2,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"92'7","reported_seq":24,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-28T18:06:10.626328+0000","last_change":"2024-03-28T18:04:49.532802+0000","last_active":"2024-03-28T18:06:10.626328+0000","last_peered":"2024-03-28T18:06:10.626328+0000","last_clean":"2024-03-28T18:06:10.626328+0000","last_became_active":"2024-03-28T18:04:49.530666+0000","last_became_peered":"2024-03-28T18:04:49.530666+0000","last_unstale":"2024-03-28T18:06:10.626328+0000","last_undegraded":"2024-03-28T18:06:10.626328+0000","last_fullsized":"2024-03-28T18:06:10.626328+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_clean_scrub_stamp":"2024-03-28T18:04:48.465760+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T22:21:28.469402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"90'3","reported_seq":23,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-28T18:05:11.744965+0000","last_change":"2024-03-28T18:04:48.503582+0000","last_active":"2024-03-28T18:05:11.744965+0000","last_peered":"2024-03-28T18:05:11.744965+0000","last_clean":"2024-03-28T18:05:11.744965+0000","last_became_active":"2024-03-28T18:04:48.502716+0000","last_became_peered":"2024-03-28T18:04:48.502716+0000","last_unstale":"2024-03-28T18:05:11.744965+0000","last_undegraded":"2024-03-28T18:05:11.744965+0000","last_fullsized":"2024-03-28T18:05:11.744965+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:04:47.412563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:04:47.412563+0000","last_clean_scrub_stamp":"2024-03-28T18:04:47.412563+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T21:41:06.333420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"92'7","reported_seq":34,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-28T18:06:10.611619+0000","last_change":"2024-03-28T18:05:11.753703+0000","last_active":"2024-03-28T18:06:10.611619+0000","last_peered":"2024-03-28T18:06:10.611619+0000","last_clean":"2024-03-28T18:06:10.611619+0000","last_became_active":"2024-03-28T18:05:11.748793+0000","last_became_peered":"2024-03-28T18:05:11.748793+0000","last_unstale":"2024-03-28T18:06:10.611619+0000","last_undegraded":"2024-03-28T18:06:10.611619+0000","last_fullsized":"2024-03-28T18:06:10.611619+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_clean_scrub_stamp":"2024-03-28T18:04:48.465760+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T19:38:38.432027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":117,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"90'5","reported_seq":24,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-28T18:05:11.950367+0000","last_change":"2024-03-28T18:04:48.507658+0000","last_active":"2024-03-28T18:05:11.950367+0000","last_peered":"2024-03-28T18:05:11.950367+0000","last_clean":"2024-03-28T18:05:11.950367+0000","last_became_active":"2024-03-28T18:04:48.506855+0000","last_became_peered":"2024-03-28T18:04:48.506855+0000","last_unstale":"2024-03-28T18:05:11.950367+0000","last_undegraded":"2024-03-28T18:05:11.950367+0000","last_fullsized":"2024-03-28T18:05:11.950367+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:04:47.412563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:04:47.412563+0000","last_clean_scrub_stamp":"2024-03-28T18:04:47.412563+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T23:08:23.550379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"92'9","reported_seq":26,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-28T18:06:10.615429+0000","last_change":"2024-03-28T18:04:49.521122+0000","last_active":"2024-03-28T18:06:10.615429+0000","last_peered":"2024-03-28T18:06:10.615429+0000","last_clean":"2024-03-28T18:06:10.615429+0000","last_became_active":"2024-03-28T18:04:49.520711+0000","last_became_peered":"2024-03-28T18:04:49.520711+0000","last_unstale":"2024-03-28T18:06:10.615429+0000","last_undegraded":"2024-03-28T18:06:10.615429+0000","last_fullsized":"2024-03-28T18:06:10.615429+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_clean_scrub_stamp":"2024-03-28T18:04:48.465760+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T19:32:33.556979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"92'9","reported_seq":26,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-28T18:06:10.604008+0000","last_change":"2024-03-28T18:04:49.534929+0000","last_active":"2024-03-28T18:06:10.604008+0000","last_peered":"2024-03-28T18:06:10.604008+0000","last_clean":"2024-03-28T18:06:10.604008+0000","last_became_active":"2024-03-28T18:04:49.534485+0000","last_became_peered":"2024-03-28T18:04:49.534485+0000","last_unstale":"2024-03-28T18:06:10.604008+0000","last_undegraded":"2024-03-28T18:06:10.604008+0000","last_fullsized":"2024-03-28T18:06:10.604008+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_clean_scrub_stamp":"2024-03-28T18:04:48.465760+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T01:47:34.937536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"92'12","reported_seq":35,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-28T18:06:10.602737+0000","last_change":"2024-03-28T18:05:11.752731+0000","last_active":"2024-03-28T18:06:10.602737+0000","last_peered":"2024-03-28T18:06:10.602737+0000","last_clean":"2024-03-28T18:06:10.602737+0000","last_became_active":"2024-03-28T18:05:11.752121+0000","last_became_peered":"2024-03-28T18:05:11.752121+0000","last_unstale":"2024-03-28T18:06:10.602737+0000","last_undegraded":"2024-03-28T18:06:10.602737+0000","last_fullsized":"2024-03-28T18:06:10.602737+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_clean_scrub_stamp":"2024-03-28T18:04:48.465760+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T20:10:02.491231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,8],"acting":[2,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"92'8","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-28T18:06:10.609018+0000","last_change":"2024-03-28T18:04:49.531999+0000","last_active":"2024-03-28T18:06:10.609018+0000","last_peered":"2024-03-28T18:06:10.609018+0000","last_clean":"2024-03-28T18:06:10.609018+0000","last_became_active":"2024-03-28T18:04:49.528641+0000","last_became_peered":"2024-03-28T18:04:49.528641+0000","last_unstale":"2024-03-28T18:06:10.609018+0000","last_undegraded":"2024-03-28T18:06:10.609018+0000","last_fullsized":"2024-03-28T18:06:10.609018+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_clean_scrub_stamp":"2024-03-28T18:04:48.465760+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T21:24:43.861635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"92'7","reported_seq":24,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-28T18:06:10.610135+0000","last_change":"2024-03-28T18:04:49.540417+0000","last_active":"2024-03-28T18:06:10.610135+0000","last_peered":"2024-03-28T18:06:10.610135+0000","last_clean":"2024-03-28T18:06:10.610135+0000","last_became_active":"2024-03-28T18:04:49.539987+0000","last_became_peered":"2024-03-28T18:04:49.539987+0000","last_unstale":"2024-03-28T18:06:10.610135+0000","last_undegraded":"2024-03-28T18:06:10.610135+0000","last_fullsized":"2024-03-28T18:06:10.610135+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_clean_scrub_stamp":"2024-03-28T18:04:48.465760+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T02:53:41.444850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"92'4","reported_seq":21,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-28T18:06:10.609348+0000","last_change":"2024-03-28T18:04:49.540014+0000","last_active":"2024-03-28T18:06:10.609348+0000","last_peered":"2024-03-28T18:06:10.609348+0000","last_clean":"2024-03-28T18:06:10.609348+0000","last_became_active":"2024-03-28T18:04:49.538239+0000","last_became_peered":"2024-03-28T18:04:49.538239+0000","last_unstale":"2024-03-28T18:06:10.609348+0000","last_undegraded":"2024-03-28T18:06:10.609348+0000","last_fullsized":"2024-03-28T18:06:10.609348+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_clean_scrub_stamp":"2024-03-28T18:04:48.465760+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T23:03:58.449336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"92'6","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-28T18:06:10.603625+0000","last_change":"2024-03-28T18:05:11.748781+0000","last_active":"2024-03-28T18:06:10.603625+0000","last_peered":"2024-03-28T18:06:10.603625+0000","last_clean":"2024-03-28T18:06:10.603625+0000","last_became_active":"2024-03-28T18:05:11.747710+0000","last_became_peered":"2024-03-28T18:05:11.747710+0000","last_unstale":"2024-03-28T18:06:10.603625+0000","last_undegraded":"2024-03-28T18:06:10.603625+0000","last_fullsized":"2024-03-28T18:06:10.603625+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_clean_scrub_stamp":"2024-03-28T18:04:48.465760+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T02:01:44.708646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,9],"acting":[11,8,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"92'15","reported_seq":32,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-28T18:06:10.618750+0000","last_change":"2024-03-28T18:04:49.531135+0000","last_active":"2024-03-28T18:06:10.618750+0000","last_peered":"2024-03-28T18:06:10.618750+0000","last_clean":"2024-03-28T18:06:10.618750+0000","last_became_active":"2024-03-28T18:04:49.530673+0000","last_became_peered":"2024-03-28T18:04:49.530673+0000","last_unstale":"2024-03-28T18:06:10.618750+0000","last_undegraded":"2024-03-28T18:06:10.618750+0000","last_fullsized":"2024-03-28T18:06:10.618750+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_clean_scrub_stamp":"2024-03-28T18:04:48.465760+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T05:51:13.950946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"92'2","reported_seq":19,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-28T18:06:10.602517+0000","last_change":"2024-03-28T18:04:49.526803+0000","last_active":"2024-03-28T18:06:10.602517+0000","last_peered":"2024-03-28T18:06:10.602517+0000","last_clean":"2024-03-28T18:06:10.602517+0000","last_became_active":"2024-03-28T18:04:49.526474+0000","last_became_peered":"2024-03-28T18:04:49.526474+0000","last_unstale":"2024-03-28T18:06:10.602517+0000","last_undegraded":"2024-03-28T18:06:10.602517+0000","last_fullsized":"2024-03-28T18:06:10.602517+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_clean_scrub_stamp":"2024-03-28T18:04:48.465760+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T01:28:27.530864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"92'6","reported_seq":23,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-28T18:06:10.612610+0000","last_change":"2024-03-28T18:04:49.534015+0000","last_active":"2024-03-28T18:06:10.612610+0000","last_peered":"2024-03-28T18:06:10.612610+0000","last_clean":"2024-03-28T18:06:10.612610+0000","last_became_active":"2024-03-28T18:04:49.533560+0000","last_became_peered":"2024-03-28T18:04:49.533560+0000","last_unstale":"2024-03-28T18:06:10.612610+0000","last_undegraded":"2024-03-28T18:06:10.612610+0000","last_fullsized":"2024-03-28T18:06:10.612610+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_clean_scrub_stamp":"2024-03-28T18:04:48.465760+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T22:41:30.965737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"92'8","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-28T18:06:10.607634+0000","last_change":"2024-03-28T18:04:49.523240+0000","last_active":"2024-03-28T18:06:10.607634+0000","last_peered":"2024-03-28T18:06:10.607634+0000","last_clean":"2024-03-28T18:06:10.607634+0000","last_became_active":"2024-03-28T18:04:49.522795+0000","last_became_peered":"2024-03-28T18:04:49.522795+0000","last_unstale":"2024-03-28T18:06:10.607634+0000","last_undegraded":"2024-03-28T18:06:10.607634+0000","last_fullsized":"2024-03-28T18:06:10.607634+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_clean_scrub_stamp":"2024-03-28T18:04:48.465760+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T23:08:37.087977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"92'10","reported_seq":27,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-28T18:06:10.626065+0000","last_change":"2024-03-28T18:04:49.530965+0000","last_active":"2024-03-28T18:06:10.626065+0000","last_peered":"2024-03-28T18:06:10.626065+0000","last_clean":"2024-03-28T18:06:10.626065+0000","last_became_active":"2024-03-28T18:04:49.530619+0000","last_became_peered":"2024-03-28T18:04:49.530619+0000","last_unstale":"2024-03-28T18:06:10.626065+0000","last_undegraded":"2024-03-28T18:06:10.626065+0000","last_fullsized":"2024-03-28T18:06:10.626065+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_clean_scrub_stamp":"2024-03-28T18:04:48.465760+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T05:29:26.754012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"92'11","reported_seq":28,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-28T18:06:10.624663+0000","last_change":"2024-03-28T18:04:49.523017+0000","last_active":"2024-03-28T18:06:10.624663+0000","last_peered":"2024-03-28T18:06:10.624663+0000","last_clean":"2024-03-28T18:06:10.624663+0000","last_became_active":"2024-03-28T18:04:49.520411+0000","last_became_peered":"2024-03-28T18:04:49.520411+0000","last_unstale":"2024-03-28T18:06:10.624663+0000","last_undegraded":"2024-03-28T18:06:10.624663+0000","last_fullsized":"2024-03-28T18:06:10.624663+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_clean_scrub_stamp":"2024-03-28T18:04:48.465760+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T22:52:45.893036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"92'8","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-28T18:06:10.608198+0000","last_change":"2024-03-28T18:04:49.544272+0000","last_active":"2024-03-28T18:06:10.608198+0000","last_peered":"2024-03-28T18:06:10.608198+0000","last_clean":"2024-03-28T18:06:10.608198+0000","last_became_active":"2024-03-28T18:04:49.541754+0000","last_became_peered":"2024-03-28T18:04:49.541754+0000","last_unstale":"2024-03-28T18:06:10.608198+0000","last_undegraded":"2024-03-28T18:06:10.608198+0000","last_fullsized":"2024-03-28T18:06:10.608198+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_clean_scrub_stamp":"2024-03-28T18:04:48.465760+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T19:06:00.799916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":117,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":469,"ondisk_log_size":469,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4751360,"data_stored":4722944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":67177,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":780,"num_read_kb":460,"num_write":240,"num_write_kb":135,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":0,"num_keys_recovered":10,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":565248,"data_stored":201888,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":208,"ondisk_log_size":208,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":85,"seq":365072220194,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28000,"kb_used_data":984,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717152,"statfs":{"total":95995035648,"available":95966363648,"internally_reserved":0,"allocated":1007616,"data_stored":483849,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":10,"up_from":77,"seq":330712481833,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32844,"kb_used_data":1668,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712308,"statfs":{"total":95995035648,"available":95961403392,"internally_reserved":0,"allocated":1708032,"data_stored":1134960,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":70,"seq":300647710767,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36356,"kb_used_data":1012,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708796,"statfs":{"total":95995035648,"available":95957807104,"internally_reserved":0,"allocated":1036288,"data_stored":483896,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":36174272},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":8,"up_from":62,"seq":266287972406,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36340,"kb_used_data":1004,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708812,"statfs":{"total":95995035648,"available":95957823488,"internally_reserved":0,"allocated":1028096,"data_stored":484157,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":7,"up_from":53,"seq":227633266748,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28604,"kb_used_data":1588,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716548,"statfs":{"total":95995035648,"available":95965745152,"internally_reserved":0,"allocated":1626112,"data_stored":1077348,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":16,"apply_latency_ms":16,"commit_latency_ns":16000000,"apply_latency_ns":16000000},"alerts":[]},{"osd":6,"up_from":48,"seq":206158430274,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36332,"kb_used_data":992,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708820,"statfs":{"total":95995035648,"available":95957831680,"internally_reserved":0,"allocated":1015808,"data_stored":483496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[3,15,4,14,7,12,9],"upper_bound":128},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574951,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36388,"kb_used_data":1052,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708764,"statfs":{"total":95995035648,"available":95957774336,"internally_reserved":0,"allocated":1077248,"data_stored":544349,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":12,"apply_latency_ms":12,"commit_latency_ns":12000000,"apply_latency_ns":12000000},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738477,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28028,"kb_used_data":1012,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717124,"statfs":{"total":95995035648,"available":95966334976,"internally_reserved":0,"allocated":1036288,"data_stored":484835,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411425,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36408,"kb_used_data":1072,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708744,"statfs":{"total":95995035648,"available":95957753856,"internally_reserved":0,"allocated":1097728,"data_stored":545101,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":3,"up_from":26,"seq":111669149785,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36340,"kb_used_data":1004,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708812,"statfs":{"total":95995035648,"available":95957823488,"internally_reserved":0,"allocated":1028096,"data_stored":483204,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":36174280},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":4,"up_from":34,"seq":146028888145,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28008,"kb_used_data":992,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717144,"statfs":{"total":95995035648,"available":95966355456,"internally_reserved":0,"allocated":1015808,"data_stored":483889,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":5,"up_from":40,"seq":171798691916,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36940,"kb_used_data":1600,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708212,"statfs":{"total":95995035648,"available":95957209088,"internally_reserved":0,"allocated":1638400,"data_stored":1076353,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":10,"apply_latency_ms":10,"commit_latency_ns":10000000,"apply_latency_ns":10000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2178,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":81920,"data_stored":61692,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":102400,"data_stored":62444,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1232,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":3352,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":722,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":4206,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":1500,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":61818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-28T18:06:17.180 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033 shell --fsid fd220748-ed2b-11ee-95d0-87774f69a715 -- ceph pg dump --format=json 2024-03-28T18:06:17.425 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:06:16 smithi134 ceph-mon[35037]: from='client.14976 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-28T18:06:17.432 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:06:16 smithi079 ceph-mon[35312]: from='client.14976 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-28T18:06:17.699 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/fd220748-ed2b-11ee-95d0-87774f69a715/mon.a/config 2024-03-28T18:06:18.192 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:06:17 smithi022 ceph-mon[31985]: pgmap v415: 97 pgs: 97 active+clean; 643 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 597 B/s wr, 12 op/s 2024-03-28T18:06:18.424 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:06:17 smithi134 ceph-mon[35037]: pgmap v415: 97 pgs: 97 active+clean; 643 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 597 B/s wr, 12 op/s 2024-03-28T18:06:18.432 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:06:17 smithi079 ceph-mon[35312]: pgmap v415: 97 pgs: 97 active+clean; 643 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 597 B/s wr, 12 op/s 2024-03-28T18:06:19.824 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:19.823+0000 7f36caa00700 1 -- 172.21.15.22:0/3235468525 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f36c406a380 msgr2=0x7f36c406a7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:06:19.824 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:19.823+0000 7f36caa00700 1 --2- 172.21.15.22:0/3235468525 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f36c406a380 0x7f36c406a7e0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f36ac009990 tx=0x7f36ac009ca0 comp rx=0 tx=0).stop 2024-03-28T18:06:19.824 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:19.823+0000 7f36caa00700 1 -- 172.21.15.22:0/3235468525 shutdown_connections 2024-03-28T18:06:19.824 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:19.823+0000 7f36caa00700 1 --2- 172.21.15.22:0/3235468525 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f36c4104b90 0x7f36c4105020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:06:19.824 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:19.823+0000 7f36caa00700 1 --2- 172.21.15.22:0/3235468525 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f36c4107080 0x7f36c4069e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:06:19.824 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:19.823+0000 7f36caa00700 1 --2- 172.21.15.22:0/3235468525 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f36c406a380 0x7f36c406a7e0 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:06:19.824 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:19.823+0000 7f36caa00700 1 -- 172.21.15.22:0/3235468525 >> 172.21.15.22:0/3235468525 conn(0x7f36c4077fc0 msgr2=0x7f36c40783d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:06:19.824 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:19.824+0000 7f36caa00700 1 -- 172.21.15.22:0/3235468525 shutdown_connections 2024-03-28T18:06:19.824 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:19.824+0000 7f36caa00700 1 -- 172.21.15.22:0/3235468525 wait complete. 2024-03-28T18:06:19.825 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:19.825+0000 7f36caa00700 1 Processor -- start 2024-03-28T18:06:19.826 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:19.825+0000 7f36caa00700 1 -- start start 2024-03-28T18:06:19.826 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:19.825+0000 7f36caa00700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f36c406a380 0x7f36c419ee80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:06:19.826 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:19.825+0000 7f36caa00700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f36c4104b90 0x7f36c419f3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:06:19.826 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:19.825+0000 7f36caa00700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f36c4107080 0x7f36c41a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:06:19.826 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:19.825+0000 7f36caa00700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36c4118ab0 con 0x7f36c4107080 2024-03-28T18:06:19.826 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:19.825+0000 7f36caa00700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36c4118930 con 0x7f36c406a380 2024-03-28T18:06:19.826 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:19.825+0000 7f36caa00700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36c4118c30 con 0x7f36c4104b90 2024-03-28T18:06:19.826 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:19.826+0000 7f36c3e00700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f36c406a380 0x7f36c419ee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:06:19.826 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:19.826+0000 7f36c3e00700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f36c406a380 0x7f36c419ee80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.22:38650/0 (socket says 172.21.15.22:38650) 2024-03-28T18:06:19.826 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:19.826+0000 7f36c3e00700 1 -- 172.21.15.22:0/2089191015 learned_addr learned my addr 172.21.15.22:0/2089191015 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-28T18:06:19.826 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:19.826+0000 7f36c8a00700 1 --2- 172.21.15.22:0/2089191015 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f36c4107080 0x7f36c41a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:06:19.827 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:19.826+0000 7f36bb400700 1 --2- 172.21.15.22:0/2089191015 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f36c4104b90 0x7f36c419f3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:06:19.827 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:19.826+0000 7f36c8a00700 1 -- 172.21.15.22:0/2089191015 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f36c4104b90 msgr2=0x7f36c419f3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:06:19.828 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:19.826+0000 7f36c8a00700 1 --2- 172.21.15.22:0/2089191015 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f36c4104b90 0x7f36c419f3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:06:19.828 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:19.826+0000 7f36c8a00700 1 -- 172.21.15.22:0/2089191015 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f36c406a380 msgr2=0x7f36c419ee80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:06:19.828 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:19.826+0000 7f36c8a00700 1 --2- 172.21.15.22:0/2089191015 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f36c406a380 0x7f36c419ee80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:06:19.828 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:19.826+0000 7f36c8a00700 1 -- 172.21.15.22:0/2089191015 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f36c41a3e90 con 0x7f36c4107080 2024-03-28T18:06:19.828 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:19.826+0000 7f36c3e00700 1 --2- 172.21.15.22:0/2089191015 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f36c406a380 0x7f36c419ee80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-28T18:06:19.828 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:19.827+0000 7f36c8a00700 1 --2- 172.21.15.22:0/2089191015 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f36c4107080 0x7f36c41a3750 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f36bc00c970 tx=0x7f36bc00cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:06:19.828 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:19.827+0000 7f36c1600700 1 -- 172.21.15.22:0/2089191015 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36bc00e880 con 0x7f36c4107080 2024-03-28T18:06:19.828 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:19.827+0000 7f36c1600700 1 -- 172.21.15.22:0/2089191015 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f36bc00f450 con 0x7f36c4107080 2024-03-28T18:06:19.828 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:19.827+0000 7f36c1600700 1 -- 172.21.15.22:0/2089191015 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36bc018660 con 0x7f36c4107080 2024-03-28T18:06:19.828 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:19.827+0000 7f36caa00700 1 -- 172.21.15.22:0/2089191015 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f36c41a4180 con 0x7f36c4107080 2024-03-28T18:06:19.828 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:19.827+0000 7f36caa00700 1 -- 172.21.15.22:0/2089191015 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f36c41aba00 con 0x7f36c4107080 2024-03-28T18:06:19.829 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:19.828+0000 7f36caa00700 1 -- 172.21.15.22:0/2089191015 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f36c4068e50 con 0x7f36c4107080 2024-03-28T18:06:19.830 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:19.830+0000 7f36c1600700 1 -- 172.21.15.22:0/2089191015 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 18) v1 ==== 137290+0+0 (secure 0 0 0) 0x7f36bc00fa20 con 0x7f36c4107080 2024-03-28T18:06:19.831 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:19.832+0000 7f36c1600700 1 --2- 172.21.15.22:0/2089191015 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7f36a80a2890 0x7f36a80a4d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:06:19.831 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:19.832+0000 7f36c1600700 1 -- 172.21.15.22:0/2089191015 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 8153+0+0 (secure 0 0 0) 0x7f36bc0ce7d0 con 0x7f36c4107080 2024-03-28T18:06:19.832 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:19.832+0000 7f36c3e00700 1 --2- 172.21.15.22:0/2089191015 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7f36a80a2890 0x7f36a80a4d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:06:19.832 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:19.833+0000 7f36c3e00700 1 --2- 172.21.15.22:0/2089191015 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7f36a80a2890 0x7f36a80a4d50 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f36b4009c10 tx=0x7f36b400b040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:06:19.836 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:19.836+0000 7f36c1600700 1 -- 172.21.15.22:0/2089191015 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f36bc014030 con 0x7f36c4107080 2024-03-28T18:06:20.064 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:20.064+0000 7f36caa00700 1 -- 172.21.15.22:0/2089191015 --> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f36c4100790 con 0x7f36a80a2890 2024-03-28T18:06:20.086 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:20.086+0000 7f36c1600700 1 -- 172.21.15.22:0/2089191015 <== mgr.14152 v2:172.21.15.22:6800/4201048392 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+258965 (secure 0 0 0) 0x7f36c4100790 con 0x7f36a80a2890 2024-03-28T18:06:20.088 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-03-28T18:06:20.093 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:20.093+0000 7f36caa00700 1 -- 172.21.15.22:0/2089191015 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7f36a80a2890 msgr2=0x7f36a80a4d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:06:20.094 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:20.093+0000 7f36caa00700 1 --2- 172.21.15.22:0/2089191015 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7f36a80a2890 0x7f36a80a4d50 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f36b4009c10 tx=0x7f36b400b040 comp rx=0 tx=0).stop 2024-03-28T18:06:20.094 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:20.093+0000 7f36caa00700 1 -- 172.21.15.22:0/2089191015 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f36c4107080 msgr2=0x7f36c41a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:06:20.094 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:20.093+0000 7f36caa00700 1 --2- 172.21.15.22:0/2089191015 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f36c4107080 0x7f36c41a3750 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f36bc00c970 tx=0x7f36bc00cc80 comp rx=0 tx=0).stop 2024-03-28T18:06:20.094 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:20.093+0000 7f36caa00700 1 -- 172.21.15.22:0/2089191015 shutdown_connections 2024-03-28T18:06:20.094 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:20.093+0000 7f36caa00700 1 --2- 172.21.15.22:0/2089191015 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f36c4107080 0x7f36c41a3750 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:06:20.094 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:20.093+0000 7f36caa00700 1 --2- 172.21.15.22:0/2089191015 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f36c406a380 0x7f36c419ee80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:06:20.094 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:20.093+0000 7f36caa00700 1 --2- 172.21.15.22:0/2089191015 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7f36a80a2890 0x7f36a80a4d50 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:06:20.094 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:20.093+0000 7f36caa00700 1 --2- 172.21.15.22:0/2089191015 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f36c4104b90 0x7f36c419f3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:06:20.094 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:20.093+0000 7f36caa00700 1 -- 172.21.15.22:0/2089191015 >> 172.21.15.22:0/2089191015 conn(0x7f36c4077fc0 msgr2=0x7f36c4100080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:06:20.094 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:20.093+0000 7f36caa00700 1 -- 172.21.15.22:0/2089191015 shutdown_connections 2024-03-28T18:06:20.094 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:20.093+0000 7f36caa00700 1 -- 172.21.15.22:0/2089191015 wait complete. 2024-03-28T18:06:20.095 INFO:teuthology.orchestra.run.smithi022.stderr:dumped all 2024-03-28T18:06:20.282 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:06:19 smithi022 ceph-mon[31985]: pgmap v416: 97 pgs: 97 active+clean; 643 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 597 B/s wr, 10 op/s 2024-03-28T18:06:20.424 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:06:19 smithi134 ceph-mon[35037]: pgmap v416: 97 pgs: 97 active+clean; 643 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 597 B/s wr, 10 op/s 2024-03-28T18:06:20.432 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:06:19 smithi079 ceph-mon[35312]: pgmap v416: 97 pgs: 97 active+clean; 643 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 597 B/s wr, 10 op/s 2024-03-28T18:06:20.861 INFO:teuthology.orchestra.run.smithi022.stdout:{"pg_ready":true,"pg_map":{"version":416,"stamp":"2024-03-28T18:06:18.638754+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":658028,"num_objects":102,"num_object_clones":0,"num_object_copies":306,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":968,"num_read_kb":606,"num_write":429,"num_write_kb":3079,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13,"num_bytes_recovered":1180853,"num_keys_recovered":10,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":852,"ondisk_log_size":852,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":404732,"kb_used_data":13980,"kb_used_omap":18,"kb_used_meta":390637,"kb_avail":1124537092,"statfs":{"total":1151940427776,"available":1151525982208,"internally_reserved":0,"allocated":14315520,"data_stored":7765968,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19141,"internal_metadata":400012603},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":21,"apply_latency_ms":21,"commit_latency_ns":21000000,"apply_latency_ns":21000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":531,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":112,"num_read_kb":53,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004527"},"pg_stats":[{"pgid":"3.3f","version":"92'8","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-28T18:06:10.621904+0000","last_change":"2024-03-28T18:04:49.545022+0000","last_active":"2024-03-28T18:06:10.621904+0000","last_peered":"2024-03-28T18:06:10.621904+0000","last_clean":"2024-03-28T18:06:10.621904+0000","last_became_active":"2024-03-28T18:04:49.544584+0000","last_became_peered":"2024-03-28T18:04:49.544584+0000","last_unstale":"2024-03-28T18:06:10.621904+0000","last_undegraded":"2024-03-28T18:06:10.621904+0000","last_fullsized":"2024-03-28T18:06:10.621904+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_clean_scrub_stamp":"2024-03-28T18:04:48.465760+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T01:47:05.644293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"92'4","reported_seq":21,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-28T18:06:10.603212+0000","last_change":"2024-03-28T18:04:49.526720+0000","last_active":"2024-03-28T18:06:10.603212+0000","last_peered":"2024-03-28T18:06:10.603212+0000","last_clean":"2024-03-28T18:06:10.603212+0000","last_became_active":"2024-03-28T18:04:49.524545+0000","last_became_peered":"2024-03-28T18:04:49.524545+0000","last_unstale":"2024-03-28T18:06:10.603212+0000","last_undegraded":"2024-03-28T18:06:10.603212+0000","last_fullsized":"2024-03-28T18:06:10.603212+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_clean_scrub_stamp":"2024-03-28T18:04:48.465760+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T01:00:54.926308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"92'7","reported_seq":24,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-28T18:06:10.625694+0000","last_change":"2024-03-28T18:04:49.532425+0000","last_active":"2024-03-28T18:06:10.625694+0000","last_peered":"2024-03-28T18:06:10.625694+0000","last_clean":"2024-03-28T18:06:10.625694+0000","last_became_active":"2024-03-28T18:04:49.531146+0000","last_became_peered":"2024-03-28T18:04:49.531146+0000","last_unstale":"2024-03-28T18:06:10.625694+0000","last_undegraded":"2024-03-28T18:06:10.625694+0000","last_fullsized":"2024-03-28T18:06:10.625694+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_clean_scrub_stamp":"2024-03-28T18:04:48.465760+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T04:58:45.131773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"92'8","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-28T18:06:10.609205+0000","last_change":"2024-03-28T18:04:49.533371+0000","last_active":"2024-03-28T18:06:10.609205+0000","last_peered":"2024-03-28T18:06:10.609205+0000","last_clean":"2024-03-28T18:06:10.609205+0000","last_became_active":"2024-03-28T18:04:49.532544+0000","last_became_peered":"2024-03-28T18:04:49.532544+0000","last_unstale":"2024-03-28T18:06:10.609205+0000","last_undegraded":"2024-03-28T18:06:10.609205+0000","last_fullsized":"2024-03-28T18:06:10.609205+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_clean_scrub_stamp":"2024-03-28T18:04:48.465760+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T20:36:25.664137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"92'4","reported_seq":21,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-28T18:06:10.592721+0000","last_change":"2024-03-28T18:04:49.530875+0000","last_active":"2024-03-28T18:06:10.592721+0000","last_peered":"2024-03-28T18:06:10.592721+0000","last_clean":"2024-03-28T18:06:10.592721+0000","last_became_active":"2024-03-28T18:04:49.530354+0000","last_became_peered":"2024-03-28T18:04:49.530354+0000","last_unstale":"2024-03-28T18:06:10.592721+0000","last_undegraded":"2024-03-28T18:06:10.592721+0000","last_fullsized":"2024-03-28T18:06:10.592721+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_clean_scrub_stamp":"2024-03-28T18:04:48.465760+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T21:02:17.187989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"92'6","reported_seq":23,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-28T18:06:10.587630+0000","last_change":"2024-03-28T18:04:49.538260+0000","last_active":"2024-03-28T18:06:10.587630+0000","last_peered":"2024-03-28T18:06:10.587630+0000","last_clean":"2024-03-28T18:06:10.587630+0000","last_became_active":"2024-03-28T18:04:49.537863+0000","last_became_peered":"2024-03-28T18:04:49.537863+0000","last_unstale":"2024-03-28T18:06:10.587630+0000","last_undegraded":"2024-03-28T18:06:10.587630+0000","last_fullsized":"2024-03-28T18:06:10.587630+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_clean_scrub_stamp":"2024-03-28T18:04:48.465760+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T03:01:25.363839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"92'5","reported_seq":22,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-28T18:06:10.612305+0000","last_change":"2024-03-28T18:04:49.534636+0000","last_active":"2024-03-28T18:06:10.612305+0000","last_peered":"2024-03-28T18:06:10.612305+0000","last_clean":"2024-03-28T18:06:10.612305+0000","last_became_active":"2024-03-28T18:04:49.534236+0000","last_became_peered":"2024-03-28T18:04:49.534236+0000","last_unstale":"2024-03-28T18:06:10.612305+0000","last_undegraded":"2024-03-28T18:06:10.612305+0000","last_fullsized":"2024-03-28T18:06:10.612305+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_clean_scrub_stamp":"2024-03-28T18:04:48.465760+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T05:55:23.616641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"92'9","reported_seq":26,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-28T18:06:10.605001+0000","last_change":"2024-03-28T18:04:49.534848+0000","last_active":"2024-03-28T18:06:10.605001+0000","last_peered":"2024-03-28T18:06:10.605001+0000","last_clean":"2024-03-28T18:06:10.605001+0000","last_became_active":"2024-03-28T18:04:49.534302+0000","last_became_peered":"2024-03-28T18:04:49.534302+0000","last_unstale":"2024-03-28T18:06:10.605001+0000","last_undegraded":"2024-03-28T18:06:10.605001+0000","last_fullsized":"2024-03-28T18:06:10.605001+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_clean_scrub_stamp":"2024-03-28T18:04:48.465760+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T05:48:24.707305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"92'11","reported_seq":28,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-28T18:06:10.622058+0000","last_change":"2024-03-28T18:04:49.536568+0000","last_active":"2024-03-28T18:06:10.622058+0000","last_peered":"2024-03-28T18:06:10.622058+0000","last_clean":"2024-03-28T18:06:10.622058+0000","last_became_active":"2024-03-28T18:04:49.535675+0000","last_became_peered":"2024-03-28T18:04:49.535675+0000","last_unstale":"2024-03-28T18:06:10.622058+0000","last_undegraded":"2024-03-28T18:06:10.622058+0000","last_fullsized":"2024-03-28T18:06:10.622058+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_clean_scrub_stamp":"2024-03-28T18:04:48.465760+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T02:32:15.179869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"92'16","reported_seq":33,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-28T18:06:10.587830+0000","last_change":"2024-03-28T18:04:49.539363+0000","last_active":"2024-03-28T18:06:10.587830+0000","last_peered":"2024-03-28T18:06:10.587830+0000","last_clean":"2024-03-28T18:06:10.587830+0000","last_became_active":"2024-03-28T18:04:49.539002+0000","last_became_peered":"2024-03-28T18:04:49.539002+0000","last_unstale":"2024-03-28T18:06:10.587830+0000","last_undegraded":"2024-03-28T18:06:10.587830+0000","last_fullsized":"2024-03-28T18:06:10.587830+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_clean_scrub_stamp":"2024-03-28T18:04:48.465760+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T18:24:28.846081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"92'6","reported_seq":23,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-28T18:06:10.626220+0000","last_change":"2024-03-28T18:04:49.532385+0000","last_active":"2024-03-28T18:06:10.626220+0000","last_peered":"2024-03-28T18:06:10.626220+0000","last_clean":"2024-03-28T18:06:10.626220+0000","last_became_active":"2024-03-28T18:04:49.532004+0000","last_became_peered":"2024-03-28T18:04:49.532004+0000","last_unstale":"2024-03-28T18:06:10.626220+0000","last_undegraded":"2024-03-28T18:06:10.626220+0000","last_fullsized":"2024-03-28T18:06:10.626220+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_clean_scrub_stamp":"2024-03-28T18:04:48.465760+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T19:05:13.959313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"92'7","reported_seq":24,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-28T18:06:10.608755+0000","last_change":"2024-03-28T18:04:49.534465+0000","last_active":"2024-03-28T18:06:10.608755+0000","last_peered":"2024-03-28T18:06:10.608755+0000","last_clean":"2024-03-28T18:06:10.608755+0000","last_became_active":"2024-03-28T18:04:49.532989+0000","last_became_peered":"2024-03-28T18:04:49.532989+0000","last_unstale":"2024-03-28T18:06:10.608755+0000","last_undegraded":"2024-03-28T18:06:10.608755+0000","last_fullsized":"2024-03-28T18:06:10.608755+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_clean_scrub_stamp":"2024-03-28T18:04:48.465760+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T23:56:50.607770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"92'10","reported_seq":27,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-28T18:06:10.611908+0000","last_change":"2024-03-28T18:04:49.535205+0000","last_active":"2024-03-28T18:06:10.611908+0000","last_peered":"2024-03-28T18:06:10.611908+0000","last_clean":"2024-03-28T18:06:10.611908+0000","last_became_active":"2024-03-28T18:04:49.533025+0000","last_became_peered":"2024-03-28T18:04:49.533025+0000","last_unstale":"2024-03-28T18:06:10.611908+0000","last_undegraded":"2024-03-28T18:06:10.611908+0000","last_fullsized":"2024-03-28T18:06:10.611908+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_clean_scrub_stamp":"2024-03-28T18:04:48.465760+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T21:41:51.815907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"92'7","reported_seq":24,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-28T18:06:10.615333+0000","last_change":"2024-03-28T18:04:49.520974+0000","last_active":"2024-03-28T18:06:10.615333+0000","last_peered":"2024-03-28T18:06:10.615333+0000","last_clean":"2024-03-28T18:06:10.615333+0000","last_became_active":"2024-03-28T18:04:49.520509+0000","last_became_peered":"2024-03-28T18:04:49.520509+0000","last_unstale":"2024-03-28T18:06:10.615333+0000","last_undegraded":"2024-03-28T18:06:10.615333+0000","last_fullsized":"2024-03-28T18:06:10.615333+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_clean_scrub_stamp":"2024-03-28T18:04:48.465760+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T03:15:16.346859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"92'7","reported_seq":24,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-28T18:06:10.608689+0000","last_change":"2024-03-28T18:04:49.539053+0000","last_active":"2024-03-28T18:06:10.608689+0000","last_peered":"2024-03-28T18:06:10.608689+0000","last_clean":"2024-03-28T18:06:10.608689+0000","last_became_active":"2024-03-28T18:04:49.537327+0000","last_became_peered":"2024-03-28T18:04:49.537327+0000","last_unstale":"2024-03-28T18:06:10.608689+0000","last_undegraded":"2024-03-28T18:06:10.608689+0000","last_fullsized":"2024-03-28T18:06:10.608689+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_clean_scrub_stamp":"2024-03-28T18:04:48.465760+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T00:24:42.319504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"92'9","reported_seq":26,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-28T18:06:10.625215+0000","last_change":"2024-03-28T18:04:49.525855+0000","last_active":"2024-03-28T18:06:10.625215+0000","last_peered":"2024-03-28T18:06:10.625215+0000","last_clean":"2024-03-28T18:06:10.625215+0000","last_became_active":"2024-03-28T18:04:49.525448+0000","last_became_peered":"2024-03-28T18:04:49.525448+0000","last_unstale":"2024-03-28T18:06:10.625215+0000","last_undegraded":"2024-03-28T18:06:10.625215+0000","last_fullsized":"2024-03-28T18:06:10.625215+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_clean_scrub_stamp":"2024-03-28T18:04:48.465760+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T23:53:00.452692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"92'8","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-28T18:06:10.622056+0000","last_change":"2024-03-28T18:04:49.543605+0000","last_active":"2024-03-28T18:06:10.622056+0000","last_peered":"2024-03-28T18:06:10.622056+0000","last_clean":"2024-03-28T18:06:10.622056+0000","last_became_active":"2024-03-28T18:04:49.543280+0000","last_became_peered":"2024-03-28T18:04:49.543280+0000","last_unstale":"2024-03-28T18:06:10.622056+0000","last_undegraded":"2024-03-28T18:06:10.622056+0000","last_fullsized":"2024-03-28T18:06:10.622056+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_clean_scrub_stamp":"2024-03-28T18:04:48.465760+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T21:27:26.216639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"92'10","reported_seq":27,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-28T18:06:10.619055+0000","last_change":"2024-03-28T18:04:49.526745+0000","last_active":"2024-03-28T18:06:10.619055+0000","last_peered":"2024-03-28T18:06:10.619055+0000","last_clean":"2024-03-28T18:06:10.619055+0000","last_became_active":"2024-03-28T18:04:49.526351+0000","last_became_peered":"2024-03-28T18:04:49.526351+0000","last_unstale":"2024-03-28T18:06:10.619055+0000","last_undegraded":"2024-03-28T18:06:10.619055+0000","last_fullsized":"2024-03-28T18:06:10.619055+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_clean_scrub_stamp":"2024-03-28T18:04:48.465760+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T05:31:47.687345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"92'7","reported_seq":24,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-28T18:06:10.595666+0000","last_change":"2024-03-28T18:04:49.523346+0000","last_active":"2024-03-28T18:06:10.595666+0000","last_peered":"2024-03-28T18:06:10.595666+0000","last_clean":"2024-03-28T18:06:10.595666+0000","last_became_active":"2024-03-28T18:04:49.522041+0000","last_became_peered":"2024-03-28T18:04:49.522041+0000","last_unstale":"2024-03-28T18:06:10.595666+0000","last_undegraded":"2024-03-28T18:06:10.595666+0000","last_fullsized":"2024-03-28T18:06:10.595666+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_clean_scrub_stamp":"2024-03-28T18:04:48.465760+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T01:35:02.419328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"90'13","reported_seq":112,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-28T18:06:13.844129+0000","last_change":"2024-03-28T18:04:48.510917+0000","last_active":"2024-03-28T18:06:13.844129+0000","last_peered":"2024-03-28T18:06:13.844129+0000","last_clean":"2024-03-28T18:06:13.844129+0000","last_became_active":"2024-03-28T18:04:48.509978+0000","last_became_peered":"2024-03-28T18:04:48.509978+0000","last_unstale":"2024-03-28T18:06:13.844129+0000","last_undegraded":"2024-03-28T18:06:13.844129+0000","last_fullsized":"2024-03-28T18:06:13.844129+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:04:47.412563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:04:47.412563+0000","last_clean_scrub_stamp":"2024-03-28T18:04:47.412563+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T03:28:13.185099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":80,"num_read_kb":80,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"92'5","reported_seq":22,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-28T18:06:10.603693+0000","last_change":"2024-03-28T18:04:49.535543+0000","last_active":"2024-03-28T18:06:10.603693+0000","last_peered":"2024-03-28T18:06:10.603693+0000","last_clean":"2024-03-28T18:06:10.603693+0000","last_became_active":"2024-03-28T18:04:49.535171+0000","last_became_peered":"2024-03-28T18:04:49.535171+0000","last_unstale":"2024-03-28T18:06:10.603693+0000","last_undegraded":"2024-03-28T18:06:10.603693+0000","last_fullsized":"2024-03-28T18:06:10.603693+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_clean_scrub_stamp":"2024-03-28T18:04:48.465760+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T22:12:22.258424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"90'2","reported_seq":21,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-28T18:05:11.741041+0000","last_change":"2024-03-28T18:04:48.503190+0000","last_active":"2024-03-28T18:05:11.741041+0000","last_peered":"2024-03-28T18:05:11.741041+0000","last_clean":"2024-03-28T18:05:11.741041+0000","last_became_active":"2024-03-28T18:04:48.501858+0000","last_became_peered":"2024-03-28T18:04:48.501858+0000","last_unstale":"2024-03-28T18:05:11.741041+0000","last_undegraded":"2024-03-28T18:05:11.741041+0000","last_fullsized":"2024-03-28T18:05:11.741041+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:04:47.412563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:04:47.412563+0000","last_clean_scrub_stamp":"2024-03-28T18:04:47.412563+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T19:56:43.878630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"92'6","reported_seq":23,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-28T18:06:10.625778+0000","last_change":"2024-03-28T18:04:49.537739+0000","last_active":"2024-03-28T18:06:10.625778+0000","last_peered":"2024-03-28T18:06:10.625778+0000","last_clean":"2024-03-28T18:06:10.625778+0000","last_became_active":"2024-03-28T18:04:49.535990+0000","last_became_peered":"2024-03-28T18:04:49.535990+0000","last_unstale":"2024-03-28T18:06:10.625778+0000","last_undegraded":"2024-03-28T18:06:10.625778+0000","last_fullsized":"2024-03-28T18:06:10.625778+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_clean_scrub_stamp":"2024-03-28T18:04:48.465760+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T21:54:15.729494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"90'12","reported_seq":27,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-28T18:05:11.768035+0000","last_change":"2024-03-28T18:05:11.767512+0000","last_active":"2024-03-28T18:05:11.768035+0000","last_peered":"2024-03-28T18:05:11.768035+0000","last_clean":"2024-03-28T18:05:11.768035+0000","last_became_active":"2024-03-28T18:05:11.752452+0000","last_became_peered":"2024-03-28T18:05:11.752452+0000","last_unstale":"2024-03-28T18:05:11.768035+0000","last_undegraded":"2024-03-28T18:05:11.768035+0000","last_fullsized":"2024-03-28T18:05:11.768035+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:04:47.412563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:04:47.412563+0000","last_clean_scrub_stamp":"2024-03-28T18:04:47.412563+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T00:09:13.980288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":0,"num_keys_recovered":10,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,7],"acting":[11,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1c","version":"92'9","reported_seq":26,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-28T18:06:10.623876+0000","last_change":"2024-03-28T18:04:49.529316+0000","last_active":"2024-03-28T18:06:10.623876+0000","last_peered":"2024-03-28T18:06:10.623876+0000","last_clean":"2024-03-28T18:06:10.623876+0000","last_became_active":"2024-03-28T18:04:49.528975+0000","last_became_peered":"2024-03-28T18:04:49.528975+0000","last_unstale":"2024-03-28T18:06:10.623876+0000","last_undegraded":"2024-03-28T18:06:10.623876+0000","last_fullsized":"2024-03-28T18:06:10.623876+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_clean_scrub_stamp":"2024-03-28T18:04:48.465760+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T00:06:45.268757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"90'5","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-28T18:05:11.734761+0000","last_change":"2024-03-28T18:04:48.515841+0000","last_active":"2024-03-28T18:05:11.734761+0000","last_peered":"2024-03-28T18:05:11.734761+0000","last_clean":"2024-03-28T18:05:11.734761+0000","last_became_active":"2024-03-28T18:04:48.513852+0000","last_became_peered":"2024-03-28T18:04:48.513852+0000","last_unstale":"2024-03-28T18:05:11.734761+0000","last_undegraded":"2024-03-28T18:05:11.734761+0000","last_fullsized":"2024-03-28T18:05:11.734761+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:04:47.412563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:04:47.412563+0000","last_clean_scrub_stamp":"2024-03-28T18:04:47.412563+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T00:07:14.751545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"92'15","reported_seq":32,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-28T18:06:10.609956+0000","last_change":"2024-03-28T18:04:49.542238+0000","last_active":"2024-03-28T18:06:10.609956+0000","last_peered":"2024-03-28T18:06:10.609956+0000","last_clean":"2024-03-28T18:06:10.609956+0000","last_became_active":"2024-03-28T18:04:49.539723+0000","last_became_peered":"2024-03-28T18:04:49.539723+0000","last_unstale":"2024-03-28T18:06:10.609956+0000","last_undegraded":"2024-03-28T18:06:10.609956+0000","last_fullsized":"2024-03-28T18:06:10.609956+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_clean_scrub_stamp":"2024-03-28T18:04:48.465760+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T22:45:07.310148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"92'7","reported_seq":108,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-28T18:06:13.859627+0000","last_change":"2024-03-28T18:04:48.511349+0000","last_active":"2024-03-28T18:06:13.859627+0000","last_peered":"2024-03-28T18:06:13.859627+0000","last_clean":"2024-03-28T18:06:13.859627+0000","last_became_active":"2024-03-28T18:04:48.510990+0000","last_became_peered":"2024-03-28T18:04:48.510990+0000","last_unstale":"2024-03-28T18:06:13.859627+0000","last_undegraded":"2024-03-28T18:06:13.859627+0000","last_fullsized":"2024-03-28T18:06:13.859627+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:04:47.412563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:04:47.412563+0000","last_clean_scrub_stamp":"2024-03-28T18:04:47.412563+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T21:56:15.411149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1154,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":82,"num_read_kb":3,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"92'7","reported_seq":24,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-28T18:06:10.604922+0000","last_change":"2024-03-28T18:04:49.538066+0000","last_active":"2024-03-28T18:06:10.604922+0000","last_peered":"2024-03-28T18:06:10.604922+0000","last_clean":"2024-03-28T18:06:10.604922+0000","last_became_active":"2024-03-28T18:04:49.537202+0000","last_became_peered":"2024-03-28T18:04:49.537202+0000","last_unstale":"2024-03-28T18:06:10.604922+0000","last_undegraded":"2024-03-28T18:06:10.604922+0000","last_fullsized":"2024-03-28T18:06:10.604922+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_clean_scrub_stamp":"2024-03-28T18:04:48.465760+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T22:35:19.060311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"90'7","reported_seq":26,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-28T18:05:11.732185+0000","last_change":"2024-03-28T18:04:48.513722+0000","last_active":"2024-03-28T18:05:11.732185+0000","last_peered":"2024-03-28T18:05:11.732185+0000","last_clean":"2024-03-28T18:05:11.732185+0000","last_became_active":"2024-03-28T18:04:48.513048+0000","last_became_peered":"2024-03-28T18:04:48.513048+0000","last_unstale":"2024-03-28T18:05:11.732185+0000","last_undegraded":"2024-03-28T18:05:11.732185+0000","last_fullsized":"2024-03-28T18:05:11.732185+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:04:47.412563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:04:47.412563+0000","last_clean_scrub_stamp":"2024-03-28T18:04:47.412563+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T04:48:10.831297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"92'8","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-28T18:06:10.612752+0000","last_change":"2024-03-28T18:04:49.539172+0000","last_active":"2024-03-28T18:06:10.612752+0000","last_peered":"2024-03-28T18:06:10.612752+0000","last_clean":"2024-03-28T18:06:10.612752+0000","last_became_active":"2024-03-28T18:04:49.538779+0000","last_became_peered":"2024-03-28T18:04:49.538779+0000","last_unstale":"2024-03-28T18:06:10.612752+0000","last_undegraded":"2024-03-28T18:06:10.612752+0000","last_fullsized":"2024-03-28T18:06:10.612752+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_clean_scrub_stamp":"2024-03-28T18:04:48.465760+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T04:59:08.361762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"90'3","reported_seq":22,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-28T18:05:11.949905+0000","last_change":"2024-03-28T18:04:48.500471+0000","last_active":"2024-03-28T18:05:11.949905+0000","last_peered":"2024-03-28T18:05:11.949905+0000","last_clean":"2024-03-28T18:05:11.949905+0000","last_became_active":"2024-03-28T18:04:48.500094+0000","last_became_peered":"2024-03-28T18:04:48.500094+0000","last_unstale":"2024-03-28T18:05:11.949905+0000","last_undegraded":"2024-03-28T18:05:11.949905+0000","last_fullsized":"2024-03-28T18:05:11.949905+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:04:47.412563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:04:47.412563+0000","last_clean_scrub_stamp":"2024-03-28T18:04:47.412563+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T21:52:51.964254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"92'8","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-28T18:06:10.600231+0000","last_change":"2024-03-28T18:04:49.522120+0000","last_active":"2024-03-28T18:06:10.600231+0000","last_peered":"2024-03-28T18:06:10.600231+0000","last_clean":"2024-03-28T18:06:10.600231+0000","last_became_active":"2024-03-28T18:04:49.521656+0000","last_became_peered":"2024-03-28T18:04:49.521656+0000","last_unstale":"2024-03-28T18:06:10.600231+0000","last_undegraded":"2024-03-28T18:06:10.600231+0000","last_fullsized":"2024-03-28T18:06:10.600231+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_clean_scrub_stamp":"2024-03-28T18:04:48.465760+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T18:29:55.605765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"90'3","reported_seq":23,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-28T18:05:11.740918+0000","last_change":"2024-03-28T18:04:48.508723+0000","last_active":"2024-03-28T18:05:11.740918+0000","last_peered":"2024-03-28T18:05:11.740918+0000","last_clean":"2024-03-28T18:05:11.740918+0000","last_became_active":"2024-03-28T18:04:48.508314+0000","last_became_peered":"2024-03-28T18:04:48.508314+0000","last_unstale":"2024-03-28T18:05:11.740918+0000","last_undegraded":"2024-03-28T18:05:11.740918+0000","last_fullsized":"2024-03-28T18:05:11.740918+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:04:47.412563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:04:47.412563+0000","last_clean_scrub_stamp":"2024-03-28T18:04:47.412563+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T03:03:59.353454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"92'11","reported_seq":28,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-28T18:06:10.611719+0000","last_change":"2024-03-28T18:04:49.531469+0000","last_active":"2024-03-28T18:06:10.611719+0000","last_peered":"2024-03-28T18:06:10.611719+0000","last_clean":"2024-03-28T18:06:10.611719+0000","last_became_active":"2024-03-28T18:04:49.529263+0000","last_became_peered":"2024-03-28T18:04:49.529263+0000","last_unstale":"2024-03-28T18:06:10.611719+0000","last_undegraded":"2024-03-28T18:06:10.611719+0000","last_fullsized":"2024-03-28T18:06:10.611719+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_clean_scrub_stamp":"2024-03-28T18:04:48.465760+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T05:46:31.593877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"92'5","reported_seq":182,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-28T18:06:14.858199+0000","last_change":"2024-03-28T18:04:48.500644+0000","last_active":"2024-03-28T18:06:14.858199+0000","last_peered":"2024-03-28T18:06:14.858199+0000","last_clean":"2024-03-28T18:06:14.858199+0000","last_became_active":"2024-03-28T18:04:48.499330+0000","last_became_peered":"2024-03-28T18:04:48.499330+0000","last_unstale":"2024-03-28T18:06:14.858199+0000","last_undegraded":"2024-03-28T18:06:14.858199+0000","last_fullsized":"2024-03-28T18:06:14.858199+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:04:47.412563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:04:47.412563+0000","last_clean_scrub_stamp":"2024-03-28T18:04:47.412563+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T18:50:54.361266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":158,"num_read_kb":158,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"92'10","reported_seq":27,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-28T18:06:10.623296+0000","last_change":"2024-03-28T18:04:49.528728+0000","last_active":"2024-03-28T18:06:10.623296+0000","last_peered":"2024-03-28T18:06:10.623296+0000","last_clean":"2024-03-28T18:06:10.623296+0000","last_became_active":"2024-03-28T18:04:49.528312+0000","last_became_peered":"2024-03-28T18:04:49.528312+0000","last_unstale":"2024-03-28T18:06:10.623296+0000","last_undegraded":"2024-03-28T18:06:10.623296+0000","last_fullsized":"2024-03-28T18:06:10.623296+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_clean_scrub_stamp":"2024-03-28T18:04:48.465760+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T05:59:37.308002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"90'4","reported_seq":23,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-28T18:05:11.736858+0000","last_change":"2024-03-28T18:04:48.508766+0000","last_active":"2024-03-28T18:05:11.736858+0000","last_peered":"2024-03-28T18:05:11.736858+0000","last_clean":"2024-03-28T18:05:11.736858+0000","last_became_active":"2024-03-28T18:04:48.506780+0000","last_became_peered":"2024-03-28T18:04:48.506780+0000","last_unstale":"2024-03-28T18:05:11.736858+0000","last_undegraded":"2024-03-28T18:05:11.736858+0000","last_fullsized":"2024-03-28T18:05:11.736858+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:04:47.412563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:04:47.412563+0000","last_clean_scrub_stamp":"2024-03-28T18:04:47.412563+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T04:39:07.394655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"92'9","reported_seq":26,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-28T18:06:10.605283+0000","last_change":"2024-03-28T18:04:49.537165+0000","last_active":"2024-03-28T18:06:10.605283+0000","last_peered":"2024-03-28T18:06:10.605283+0000","last_clean":"2024-03-28T18:06:10.605283+0000","last_became_active":"2024-03-28T18:04:49.534543+0000","last_became_peered":"2024-03-28T18:04:49.534543+0000","last_unstale":"2024-03-28T18:06:10.605283+0000","last_undegraded":"2024-03-28T18:06:10.605283+0000","last_fullsized":"2024-03-28T18:06:10.605283+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_clean_scrub_stamp":"2024-03-28T18:04:48.465760+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T03:15:58.056148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"90'5","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-28T18:05:11.746325+0000","last_change":"2024-03-28T18:04:48.503866+0000","last_active":"2024-03-28T18:05:11.746325+0000","last_peered":"2024-03-28T18:05:11.746325+0000","last_clean":"2024-03-28T18:05:11.746325+0000","last_became_active":"2024-03-28T18:04:48.503223+0000","last_became_peered":"2024-03-28T18:04:48.503223+0000","last_unstale":"2024-03-28T18:05:11.746325+0000","last_undegraded":"2024-03-28T18:05:11.746325+0000","last_fullsized":"2024-03-28T18:05:11.746325+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:04:47.412563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:04:47.412563+0000","last_clean_scrub_stamp":"2024-03-28T18:04:47.412563+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T20:14:01.307483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"92'11","reported_seq":28,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-28T18:06:10.624541+0000","last_change":"2024-03-28T18:04:49.539494+0000","last_active":"2024-03-28T18:06:10.624541+0000","last_peered":"2024-03-28T18:06:10.624541+0000","last_clean":"2024-03-28T18:06:10.624541+0000","last_became_active":"2024-03-28T18:04:49.539044+0000","last_became_peered":"2024-03-28T18:04:49.539044+0000","last_unstale":"2024-03-28T18:06:10.624541+0000","last_undegraded":"2024-03-28T18:06:10.624541+0000","last_fullsized":"2024-03-28T18:06:10.624541+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_clean_scrub_stamp":"2024-03-28T18:04:48.465760+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T04:08:30.807788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"92'31","reported_seq":215,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-28T18:06:14.861580+0000","last_change":"2024-03-28T18:04:48.507703+0000","last_active":"2024-03-28T18:06:14.861580+0000","last_peered":"2024-03-28T18:06:14.861580+0000","last_clean":"2024-03-28T18:06:14.861580+0000","last_became_active":"2024-03-28T18:04:48.507310+0000","last_became_peered":"2024-03-28T18:04:48.507310+0000","last_unstale":"2024-03-28T18:06:14.861580+0000","last_undegraded":"2024-03-28T18:06:14.861580+0000","last_fullsized":"2024-03-28T18:06:14.861580+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:04:47.412563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:04:47.412563+0000","last_clean_scrub_stamp":"2024-03-28T18:04:47.412563+0000","objects_scrubbed":0,"log_size":31,"log_dups_size":0,"ondisk_log_size":31,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T19:14:13.910042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":59940,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":165,"num_read_kb":141,"num_write":33,"num_write_kb":70,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"92'6","reported_seq":23,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-28T18:06:10.618239+0000","last_change":"2024-03-28T18:04:49.529928+0000","last_active":"2024-03-28T18:06:10.618239+0000","last_peered":"2024-03-28T18:06:10.618239+0000","last_clean":"2024-03-28T18:06:10.618239+0000","last_became_active":"2024-03-28T18:04:49.529527+0000","last_became_peered":"2024-03-28T18:04:49.529527+0000","last_unstale":"2024-03-28T18:06:10.618239+0000","last_undegraded":"2024-03-28T18:06:10.618239+0000","last_fullsized":"2024-03-28T18:06:10.618239+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_clean_scrub_stamp":"2024-03-28T18:04:48.465760+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T05:50:07.129140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"90'9","reported_seq":30,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-28T18:05:11.746751+0000","last_change":"2024-03-28T18:04:48.494746+0000","last_active":"2024-03-28T18:05:11.746751+0000","last_peered":"2024-03-28T18:05:11.746751+0000","last_clean":"2024-03-28T18:05:11.746751+0000","last_became_active":"2024-03-28T18:04:48.494296+0000","last_became_peered":"2024-03-28T18:04:48.494296+0000","last_unstale":"2024-03-28T18:05:11.746751+0000","last_undegraded":"2024-03-28T18:05:11.746751+0000","last_fullsized":"2024-03-28T18:05:11.746751+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:04:47.412563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:04:47.412563+0000","last_clean_scrub_stamp":"2024-03-28T18:04:47.412563+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T22:32:56.923067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"92'9","reported_seq":26,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-28T18:06:10.624132+0000","last_change":"2024-03-28T18:04:49.528070+0000","last_active":"2024-03-28T18:06:10.624132+0000","last_peered":"2024-03-28T18:06:10.624132+0000","last_clean":"2024-03-28T18:06:10.624132+0000","last_became_active":"2024-03-28T18:04:49.526747+0000","last_became_peered":"2024-03-28T18:04:49.526747+0000","last_unstale":"2024-03-28T18:06:10.624132+0000","last_undegraded":"2024-03-28T18:06:10.624132+0000","last_fullsized":"2024-03-28T18:06:10.624132+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_clean_scrub_stamp":"2024-03-28T18:04:48.465760+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T22:32:05.750094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"90'5","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-28T18:05:11.741286+0000","last_change":"2024-03-28T18:04:48.505723+0000","last_active":"2024-03-28T18:05:11.741286+0000","last_peered":"2024-03-28T18:05:11.741286+0000","last_clean":"2024-03-28T18:05:11.741286+0000","last_became_active":"2024-03-28T18:04:48.504088+0000","last_became_peered":"2024-03-28T18:04:48.504088+0000","last_unstale":"2024-03-28T18:05:11.741286+0000","last_undegraded":"2024-03-28T18:05:11.741286+0000","last_fullsized":"2024-03-28T18:05:11.741286+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:04:47.412563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:04:47.412563+0000","last_clean_scrub_stamp":"2024-03-28T18:04:47.412563+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T23:25:37.138054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"92'15","reported_seq":32,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-28T18:06:10.625128+0000","last_change":"2024-03-28T18:04:49.518872+0000","last_active":"2024-03-28T18:06:10.625128+0000","last_peered":"2024-03-28T18:06:10.625128+0000","last_clean":"2024-03-28T18:06:10.625128+0000","last_became_active":"2024-03-28T18:04:49.518507+0000","last_became_peered":"2024-03-28T18:04:49.518507+0000","last_unstale":"2024-03-28T18:06:10.625128+0000","last_undegraded":"2024-03-28T18:06:10.625128+0000","last_fullsized":"2024-03-28T18:06:10.625128+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_clean_scrub_stamp":"2024-03-28T18:04:48.465760+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T20:11:31.044321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"92'7","reported_seq":109,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-28T18:06:14.862350+0000","last_change":"2024-03-28T18:04:48.503058+0000","last_active":"2024-03-28T18:06:14.862350+0000","last_peered":"2024-03-28T18:06:14.862350+0000","last_clean":"2024-03-28T18:06:14.862350+0000","last_became_active":"2024-03-28T18:04:48.501596+0000","last_became_peered":"2024-03-28T18:04:48.501596+0000","last_unstale":"2024-03-28T18:06:14.862350+0000","last_undegraded":"2024-03-28T18:06:14.862350+0000","last_fullsized":"2024-03-28T18:06:14.862350+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:04:47.412563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:04:47.412563+0000","last_clean_scrub_stamp":"2024-03-28T18:04:47.412563+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T23:46:10.442771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1120,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":84,"num_read_kb":4,"num_write":7,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"92'6","reported_seq":23,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-28T18:06:10.614297+0000","last_change":"2024-03-28T18:04:49.535102+0000","last_active":"2024-03-28T18:06:10.614297+0000","last_peered":"2024-03-28T18:06:10.614297+0000","last_clean":"2024-03-28T18:06:10.614297+0000","last_became_active":"2024-03-28T18:04:49.534171+0000","last_became_peered":"2024-03-28T18:04:49.534171+0000","last_unstale":"2024-03-28T18:06:10.614297+0000","last_undegraded":"2024-03-28T18:06:10.614297+0000","last_fullsized":"2024-03-28T18:06:10.614297+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_clean_scrub_stamp":"2024-03-28T18:04:48.465760+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T18:16:27.703327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"90'17","reported_seq":37,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-28T18:05:11.734179+0000","last_change":"2024-03-28T18:04:48.517034+0000","last_active":"2024-03-28T18:05:11.734179+0000","last_peered":"2024-03-28T18:05:11.734179+0000","last_clean":"2024-03-28T18:05:11.734179+0000","last_became_active":"2024-03-28T18:04:48.516562+0000","last_became_peered":"2024-03-28T18:04:48.516562+0000","last_unstale":"2024-03-28T18:05:11.734179+0000","last_undegraded":"2024-03-28T18:05:11.734179+0000","last_fullsized":"2024-03-28T18:05:11.734179+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:04:47.412563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:04:47.412563+0000","last_clean_scrub_stamp":"2024-03-28T18:04:47.412563+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T02:28:55.753779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"92'6","reported_seq":23,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-28T18:06:10.609902+0000","last_change":"2024-03-28T18:04:49.541501+0000","last_active":"2024-03-28T18:06:10.609902+0000","last_peered":"2024-03-28T18:06:10.609902+0000","last_clean":"2024-03-28T18:06:10.609902+0000","last_became_active":"2024-03-28T18:04:49.541117+0000","last_became_peered":"2024-03-28T18:04:49.541117+0000","last_unstale":"2024-03-28T18:06:10.609902+0000","last_undegraded":"2024-03-28T18:06:10.609902+0000","last_fullsized":"2024-03-28T18:06:10.609902+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_clean_scrub_stamp":"2024-03-28T18:04:48.465760+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T04:18:29.081489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"90'2","reported_seq":22,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-28T18:05:11.735202+0000","last_change":"2024-03-28T18:04:48.510859+0000","last_active":"2024-03-28T18:05:11.735202+0000","last_peered":"2024-03-28T18:05:11.735202+0000","last_clean":"2024-03-28T18:05:11.735202+0000","last_became_active":"2024-03-28T18:04:48.510522+0000","last_became_peered":"2024-03-28T18:04:48.510522+0000","last_unstale":"2024-03-28T18:05:11.735202+0000","last_undegraded":"2024-03-28T18:05:11.735202+0000","last_fullsized":"2024-03-28T18:05:11.735202+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:04:47.412563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:04:47.412563+0000","last_clean_scrub_stamp":"2024-03-28T18:04:47.412563+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T22:31:37.817361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"92'5","reported_seq":22,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-28T18:06:10.605268+0000","last_change":"2024-03-28T18:04:49.541981+0000","last_active":"2024-03-28T18:06:10.605268+0000","last_peered":"2024-03-28T18:06:10.605268+0000","last_clean":"2024-03-28T18:06:10.605268+0000","last_became_active":"2024-03-28T18:04:49.541606+0000","last_became_peered":"2024-03-28T18:04:49.541606+0000","last_unstale":"2024-03-28T18:06:10.605268+0000","last_undegraded":"2024-03-28T18:06:10.605268+0000","last_fullsized":"2024-03-28T18:06:10.605268+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_clean_scrub_stamp":"2024-03-28T18:04:48.465760+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T21:56:03.632490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"90'1","reported_seq":20,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-28T18:05:11.730832+0000","last_change":"2024-03-28T18:04:48.505847+0000","last_active":"2024-03-28T18:05:11.730832+0000","last_peered":"2024-03-28T18:05:11.730832+0000","last_clean":"2024-03-28T18:05:11.730832+0000","last_became_active":"2024-03-28T18:04:48.501005+0000","last_became_peered":"2024-03-28T18:04:48.501005+0000","last_unstale":"2024-03-28T18:05:11.730832+0000","last_undegraded":"2024-03-28T18:05:11.730832+0000","last_fullsized":"2024-03-28T18:05:11.730832+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:04:47.412563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:04:47.412563+0000","last_clean_scrub_stamp":"2024-03-28T18:04:47.412563+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T01:33:45.539324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"92'5","reported_seq":22,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-28T18:06:10.604271+0000","last_change":"2024-03-28T18:04:49.526938+0000","last_active":"2024-03-28T18:06:10.604271+0000","last_peered":"2024-03-28T18:06:10.604271+0000","last_clean":"2024-03-28T18:06:10.604271+0000","last_became_active":"2024-03-28T18:04:49.526290+0000","last_became_peered":"2024-03-28T18:04:49.526290+0000","last_unstale":"2024-03-28T18:06:10.604271+0000","last_undegraded":"2024-03-28T18:06:10.604271+0000","last_fullsized":"2024-03-28T18:06:10.604271+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_clean_scrub_stamp":"2024-03-28T18:04:48.465760+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T03:06:54.561470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"90'5","reported_seq":106,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-28T18:06:15.849593+0000","last_change":"2024-03-28T18:04:48.512553+0000","last_active":"2024-03-28T18:06:15.849593+0000","last_peered":"2024-03-28T18:06:15.849593+0000","last_clean":"2024-03-28T18:06:15.849593+0000","last_became_active":"2024-03-28T18:04:48.511520+0000","last_became_peered":"2024-03-28T18:04:48.511520+0000","last_unstale":"2024-03-28T18:06:15.849593+0000","last_undegraded":"2024-03-28T18:06:15.849593+0000","last_fullsized":"2024-03-28T18:06:15.849593+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:04:47.412563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:04:47.412563+0000","last_clean_scrub_stamp":"2024-03-28T18:04:47.412563+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T21:13:32.970027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":82,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"92'7","reported_seq":30,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-28T18:06:10.606977+0000","last_change":"2024-03-28T18:05:11.739596+0000","last_active":"2024-03-28T18:06:10.606977+0000","last_peered":"2024-03-28T18:06:10.606977+0000","last_clean":"2024-03-28T18:06:10.606977+0000","last_became_active":"2024-03-28T18:05:11.739156+0000","last_became_peered":"2024-03-28T18:05:11.739156+0000","last_unstale":"2024-03-28T18:06:10.606977+0000","last_undegraded":"2024-03-28T18:06:10.606977+0000","last_fullsized":"2024-03-28T18:06:10.606977+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_clean_scrub_stamp":"2024-03-28T18:04:48.465760+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T05:23:00.446713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,0],"acting":[10,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"92'8","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-28T18:06:10.609312+0000","last_change":"2024-03-28T18:04:49.540438+0000","last_active":"2024-03-28T18:06:10.609312+0000","last_peered":"2024-03-28T18:06:10.609312+0000","last_clean":"2024-03-28T18:06:10.609312+0000","last_became_active":"2024-03-28T18:04:49.540069+0000","last_became_peered":"2024-03-28T18:04:49.540069+0000","last_unstale":"2024-03-28T18:06:10.609312+0000","last_undegraded":"2024-03-28T18:06:10.609312+0000","last_fullsized":"2024-03-28T18:06:10.609312+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_clean_scrub_stamp":"2024-03-28T18:04:48.465760+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T19:31:49.254101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"92'6","reported_seq":105,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-28T18:06:13.859294+0000","last_change":"2024-03-28T18:04:48.508839+0000","last_active":"2024-03-28T18:06:13.859294+0000","last_peered":"2024-03-28T18:06:13.859294+0000","last_clean":"2024-03-28T18:06:13.859294+0000","last_became_active":"2024-03-28T18:04:48.508475+0000","last_became_peered":"2024-03-28T18:04:48.508475+0000","last_unstale":"2024-03-28T18:06:13.859294+0000","last_undegraded":"2024-03-28T18:06:13.859294+0000","last_fullsized":"2024-03-28T18:06:13.859294+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:04:47.412563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:04:47.412563+0000","last_clean_scrub_stamp":"2024-03-28T18:04:47.412563+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T04:31:10.463284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":80,"num_read_kb":80,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"92'8","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-28T18:06:10.621649+0000","last_change":"2024-03-28T18:04:49.534449+0000","last_active":"2024-03-28T18:06:10.621649+0000","last_peered":"2024-03-28T18:06:10.621649+0000","last_clean":"2024-03-28T18:06:10.621649+0000","last_became_active":"2024-03-28T18:04:49.534098+0000","last_became_peered":"2024-03-28T18:04:49.534098+0000","last_unstale":"2024-03-28T18:06:10.621649+0000","last_undegraded":"2024-03-28T18:06:10.621649+0000","last_fullsized":"2024-03-28T18:06:10.621649+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_clean_scrub_stamp":"2024-03-28T18:04:48.465760+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T23:47:07.886931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"90'6","reported_seq":28,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-28T18:05:11.948329+0000","last_change":"2024-03-28T18:04:48.501790+0000","last_active":"2024-03-28T18:05:11.948329+0000","last_peered":"2024-03-28T18:05:11.948329+0000","last_clean":"2024-03-28T18:05:11.948329+0000","last_became_active":"2024-03-28T18:04:48.501415+0000","last_became_peered":"2024-03-28T18:04:48.501415+0000","last_unstale":"2024-03-28T18:05:11.948329+0000","last_undegraded":"2024-03-28T18:05:11.948329+0000","last_fullsized":"2024-03-28T18:05:11.948329+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:04:47.412563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:04:47.412563+0000","last_clean_scrub_stamp":"2024-03-28T18:04:47.412563+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T19:17:22.319883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"22'137","reported_seq":366,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-28T18:05:11.735054+0000","last_change":"2024-03-28T18:03:13.522588+0000","last_active":"2024-03-28T18:05:11.735054+0000","last_peered":"2024-03-28T18:05:11.735054+0000","last_clean":"2024-03-28T18:05:11.735054+0000","last_became_active":"2024-03-28T18:03:13.521185+0000","last_became_peered":"2024-03-28T18:03:13.521185+0000","last_unstale":"2024-03-28T18:05:11.735054+0000","last_undegraded":"2024-03-28T18:05:11.735054+0000","last_fullsized":"2024-03-28T18:05:11.735054+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T17:58:33.169981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T17:58:33.169981+0000","last_clean_scrub_stamp":"2024-03-28T17:58:33.169981+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T19:10:06.087374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"92'5","reported_seq":22,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-28T18:06:10.606883+0000","last_change":"2024-03-28T18:04:49.538619+0000","last_active":"2024-03-28T18:06:10.606883+0000","last_peered":"2024-03-28T18:06:10.606883+0000","last_clean":"2024-03-28T18:06:10.606883+0000","last_became_active":"2024-03-28T18:04:49.538276+0000","last_became_peered":"2024-03-28T18:04:49.538276+0000","last_unstale":"2024-03-28T18:06:10.606883+0000","last_undegraded":"2024-03-28T18:06:10.606883+0000","last_fullsized":"2024-03-28T18:06:10.606883+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_clean_scrub_stamp":"2024-03-28T18:04:48.465760+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T01:39:37.139544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"90'4","reported_seq":23,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-28T18:05:11.735171+0000","last_change":"2024-03-28T18:04:48.513621+0000","last_active":"2024-03-28T18:05:11.735171+0000","last_peered":"2024-03-28T18:05:11.735171+0000","last_clean":"2024-03-28T18:05:11.735171+0000","last_became_active":"2024-03-28T18:04:48.513019+0000","last_became_peered":"2024-03-28T18:04:48.513019+0000","last_unstale":"2024-03-28T18:05:11.735171+0000","last_undegraded":"2024-03-28T18:05:11.735171+0000","last_fullsized":"2024-03-28T18:05:11.735171+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:04:47.412563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:04:47.412563+0000","last_clean_scrub_stamp":"2024-03-28T18:04:47.412563+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T23:38:18.082830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"90'4","reported_seq":24,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-28T18:05:11.734872+0000","last_change":"2024-03-28T18:04:48.513999+0000","last_active":"2024-03-28T18:05:11.734872+0000","last_peered":"2024-03-28T18:05:11.734872+0000","last_clean":"2024-03-28T18:05:11.734872+0000","last_became_active":"2024-03-28T18:04:48.512836+0000","last_became_peered":"2024-03-28T18:04:48.512836+0000","last_unstale":"2024-03-28T18:05:11.734872+0000","last_undegraded":"2024-03-28T18:05:11.734872+0000","last_fullsized":"2024-03-28T18:05:11.734872+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:04:47.412563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:04:47.412563+0000","last_clean_scrub_stamp":"2024-03-28T18:04:47.412563+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T18:29:42.015843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"92'9","reported_seq":26,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-28T18:06:10.601878+0000","last_change":"2024-03-28T18:04:49.519323+0000","last_active":"2024-03-28T18:06:10.601878+0000","last_peered":"2024-03-28T18:06:10.601878+0000","last_clean":"2024-03-28T18:06:10.601878+0000","last_became_active":"2024-03-28T18:04:49.518898+0000","last_became_peered":"2024-03-28T18:04:49.518898+0000","last_unstale":"2024-03-28T18:06:10.601878+0000","last_undegraded":"2024-03-28T18:06:10.601878+0000","last_fullsized":"2024-03-28T18:06:10.601878+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_clean_scrub_stamp":"2024-03-28T18:04:48.465760+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T18:41:12.683606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"92'14","reported_seq":33,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-28T18:05:11.730962+0000","last_change":"2024-03-28T18:04:48.512673+0000","last_active":"2024-03-28T18:05:11.730962+0000","last_peered":"2024-03-28T18:05:11.730962+0000","last_clean":"2024-03-28T18:05:11.730962+0000","last_became_active":"2024-03-28T18:04:48.512137+0000","last_became_peered":"2024-03-28T18:04:48.512137+0000","last_unstale":"2024-03-28T18:05:11.730962+0000","last_undegraded":"2024-03-28T18:05:11.730962+0000","last_fullsized":"2024-03-28T18:05:11.730962+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:04:47.412563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:04:47.412563+0000","last_clean_scrub_stamp":"2024-03-28T18:04:47.412563+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T21:29:41.731701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":24,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"92'5","reported_seq":22,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-28T18:06:10.624287+0000","last_change":"2024-03-28T18:04:49.525055+0000","last_active":"2024-03-28T18:06:10.624287+0000","last_peered":"2024-03-28T18:06:10.624287+0000","last_clean":"2024-03-28T18:06:10.624287+0000","last_became_active":"2024-03-28T18:04:49.522038+0000","last_became_peered":"2024-03-28T18:04:49.522038+0000","last_unstale":"2024-03-28T18:06:10.624287+0000","last_undegraded":"2024-03-28T18:06:10.624287+0000","last_fullsized":"2024-03-28T18:06:10.624287+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_clean_scrub_stamp":"2024-03-28T18:04:48.465760+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T05:21:44.970397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"90'7","reported_seq":27,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-28T18:05:11.949039+0000","last_change":"2024-03-28T18:04:48.509951+0000","last_active":"2024-03-28T18:05:11.949039+0000","last_peered":"2024-03-28T18:05:11.949039+0000","last_clean":"2024-03-28T18:05:11.949039+0000","last_became_active":"2024-03-28T18:04:48.509141+0000","last_became_peered":"2024-03-28T18:04:48.509141+0000","last_unstale":"2024-03-28T18:05:11.949039+0000","last_undegraded":"2024-03-28T18:05:11.949039+0000","last_fullsized":"2024-03-28T18:05:11.949039+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:04:47.412563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:04:47.412563+0000","last_clean_scrub_stamp":"2024-03-28T18:04:47.412563+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T03:54:05.569940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"92'7","reported_seq":24,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-28T18:06:10.612641+0000","last_change":"2024-03-28T18:04:49.537174+0000","last_active":"2024-03-28T18:06:10.612641+0000","last_peered":"2024-03-28T18:06:10.612641+0000","last_clean":"2024-03-28T18:06:10.612641+0000","last_became_active":"2024-03-28T18:04:49.536801+0000","last_became_peered":"2024-03-28T18:04:49.536801+0000","last_unstale":"2024-03-28T18:06:10.612641+0000","last_undegraded":"2024-03-28T18:06:10.612641+0000","last_fullsized":"2024-03-28T18:06:10.612641+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_clean_scrub_stamp":"2024-03-28T18:04:48.465760+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T05:27:55.908196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"90'2","reported_seq":31,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-28T18:05:11.756894+0000","last_change":"2024-03-28T18:05:11.756894+0000","last_active":"2024-03-28T18:05:11.756894+0000","last_peered":"2024-03-28T18:05:11.756894+0000","last_clean":"2024-03-28T18:05:11.756894+0000","last_became_active":"2024-03-28T18:05:11.750825+0000","last_became_peered":"2024-03-28T18:05:11.750825+0000","last_unstale":"2024-03-28T18:05:11.756894+0000","last_undegraded":"2024-03-28T18:05:11.756894+0000","last_fullsized":"2024-03-28T18:05:11.756894+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:04:47.412563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:04:47.412563+0000","last_clean_scrub_stamp":"2024-03-28T18:04:47.412563+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T23:11:42.105850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,11,0],"acting":[1,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"92'2","reported_seq":19,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-28T18:06:10.566718+0000","last_change":"2024-03-28T18:04:49.523970+0000","last_active":"2024-03-28T18:06:10.566718+0000","last_peered":"2024-03-28T18:06:10.566718+0000","last_clean":"2024-03-28T18:06:10.566718+0000","last_became_active":"2024-03-28T18:04:49.523559+0000","last_became_peered":"2024-03-28T18:04:49.523559+0000","last_unstale":"2024-03-28T18:06:10.566718+0000","last_undegraded":"2024-03-28T18:06:10.566718+0000","last_fullsized":"2024-03-28T18:06:10.566718+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_clean_scrub_stamp":"2024-03-28T18:04:48.465760+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T05:50:25.593232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"90'7","reported_seq":26,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-28T18:05:11.739173+0000","last_change":"2024-03-28T18:04:48.492067+0000","last_active":"2024-03-28T18:05:11.739173+0000","last_peered":"2024-03-28T18:05:11.739173+0000","last_clean":"2024-03-28T18:05:11.739173+0000","last_became_active":"2024-03-28T18:04:48.490633+0000","last_became_peered":"2024-03-28T18:04:48.490633+0000","last_unstale":"2024-03-28T18:05:11.739173+0000","last_undegraded":"2024-03-28T18:05:11.739173+0000","last_fullsized":"2024-03-28T18:05:11.739173+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:04:47.412563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:04:47.412563+0000","last_clean_scrub_stamp":"2024-03-28T18:04:47.412563+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T23:00:52.703618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"92'9","reported_seq":26,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-28T18:06:10.608491+0000","last_change":"2024-03-28T18:04:49.531554+0000","last_active":"2024-03-28T18:06:10.608491+0000","last_peered":"2024-03-28T18:06:10.608491+0000","last_clean":"2024-03-28T18:06:10.608491+0000","last_became_active":"2024-03-28T18:04:49.530273+0000","last_became_peered":"2024-03-28T18:04:49.530273+0000","last_unstale":"2024-03-28T18:06:10.608491+0000","last_undegraded":"2024-03-28T18:06:10.608491+0000","last_fullsized":"2024-03-28T18:06:10.608491+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_clean_scrub_stamp":"2024-03-28T18:04:48.465760+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T22:17:04.032614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"90'3","reported_seq":22,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-28T18:05:11.743439+0000","last_change":"2024-03-28T18:04:48.514974+0000","last_active":"2024-03-28T18:05:11.743439+0000","last_peered":"2024-03-28T18:05:11.743439+0000","last_clean":"2024-03-28T18:05:11.743439+0000","last_became_active":"2024-03-28T18:04:48.514356+0000","last_became_peered":"2024-03-28T18:04:48.514356+0000","last_unstale":"2024-03-28T18:05:11.743439+0000","last_undegraded":"2024-03-28T18:05:11.743439+0000","last_fullsized":"2024-03-28T18:05:11.743439+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:04:47.412563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:04:47.412563+0000","last_clean_scrub_stamp":"2024-03-28T18:04:47.412563+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T01:26:37.202440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"92'8","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-28T18:06:10.612963+0000","last_change":"2024-03-28T18:04:49.518064+0000","last_active":"2024-03-28T18:06:10.612963+0000","last_peered":"2024-03-28T18:06:10.612963+0000","last_clean":"2024-03-28T18:06:10.612963+0000","last_became_active":"2024-03-28T18:04:49.517674+0000","last_became_peered":"2024-03-28T18:04:49.517674+0000","last_unstale":"2024-03-28T18:06:10.612963+0000","last_undegraded":"2024-03-28T18:06:10.612963+0000","last_fullsized":"2024-03-28T18:06:10.612963+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_clean_scrub_stamp":"2024-03-28T18:04:48.465760+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T00:10:44.537164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"90'4","reported_seq":24,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-28T18:05:11.734723+0000","last_change":"2024-03-28T18:04:48.517028+0000","last_active":"2024-03-28T18:05:11.734723+0000","last_peered":"2024-03-28T18:05:11.734723+0000","last_clean":"2024-03-28T18:05:11.734723+0000","last_became_active":"2024-03-28T18:04:48.516387+0000","last_became_peered":"2024-03-28T18:04:48.516387+0000","last_unstale":"2024-03-28T18:05:11.734723+0000","last_undegraded":"2024-03-28T18:05:11.734723+0000","last_fullsized":"2024-03-28T18:05:11.734723+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:04:47.412563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:04:47.412563+0000","last_clean_scrub_stamp":"2024-03-28T18:04:47.412563+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T01:09:24.142422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"92'5","reported_seq":22,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-28T18:06:10.625708+0000","last_change":"2024-03-28T18:04:49.527032+0000","last_active":"2024-03-28T18:06:10.625708+0000","last_peered":"2024-03-28T18:06:10.625708+0000","last_clean":"2024-03-28T18:06:10.625708+0000","last_became_active":"2024-03-28T18:04:49.523888+0000","last_became_peered":"2024-03-28T18:04:49.523888+0000","last_unstale":"2024-03-28T18:06:10.625708+0000","last_undegraded":"2024-03-28T18:06:10.625708+0000","last_fullsized":"2024-03-28T18:06:10.625708+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_clean_scrub_stamp":"2024-03-28T18:04:48.465760+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T01:19:01.265719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"90'5","reported_seq":107,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-28T18:06:15.861667+0000","last_change":"2024-03-28T18:04:48.502638+0000","last_active":"2024-03-28T18:06:15.861667+0000","last_peered":"2024-03-28T18:06:15.861667+0000","last_clean":"2024-03-28T18:06:15.861667+0000","last_became_active":"2024-03-28T18:04:48.502249+0000","last_became_peered":"2024-03-28T18:04:48.502249+0000","last_unstale":"2024-03-28T18:06:15.861667+0000","last_undegraded":"2024-03-28T18:06:15.861667+0000","last_fullsized":"2024-03-28T18:06:15.861667+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:04:47.412563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:04:47.412563+0000","last_clean_scrub_stamp":"2024-03-28T18:04:47.412563+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T03:23:51.740832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1170,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":83,"num_read_kb":2,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"92'7","reported_seq":24,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-28T18:06:10.626328+0000","last_change":"2024-03-28T18:04:49.532802+0000","last_active":"2024-03-28T18:06:10.626328+0000","last_peered":"2024-03-28T18:06:10.626328+0000","last_clean":"2024-03-28T18:06:10.626328+0000","last_became_active":"2024-03-28T18:04:49.530666+0000","last_became_peered":"2024-03-28T18:04:49.530666+0000","last_unstale":"2024-03-28T18:06:10.626328+0000","last_undegraded":"2024-03-28T18:06:10.626328+0000","last_fullsized":"2024-03-28T18:06:10.626328+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_clean_scrub_stamp":"2024-03-28T18:04:48.465760+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T22:21:28.469402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"90'3","reported_seq":23,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-28T18:05:11.744965+0000","last_change":"2024-03-28T18:04:48.503582+0000","last_active":"2024-03-28T18:05:11.744965+0000","last_peered":"2024-03-28T18:05:11.744965+0000","last_clean":"2024-03-28T18:05:11.744965+0000","last_became_active":"2024-03-28T18:04:48.502716+0000","last_became_peered":"2024-03-28T18:04:48.502716+0000","last_unstale":"2024-03-28T18:05:11.744965+0000","last_undegraded":"2024-03-28T18:05:11.744965+0000","last_fullsized":"2024-03-28T18:05:11.744965+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:04:47.412563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:04:47.412563+0000","last_clean_scrub_stamp":"2024-03-28T18:04:47.412563+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T21:41:06.333420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"92'7","reported_seq":34,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-28T18:06:10.611619+0000","last_change":"2024-03-28T18:05:11.753703+0000","last_active":"2024-03-28T18:06:10.611619+0000","last_peered":"2024-03-28T18:06:10.611619+0000","last_clean":"2024-03-28T18:06:10.611619+0000","last_became_active":"2024-03-28T18:05:11.748793+0000","last_became_peered":"2024-03-28T18:05:11.748793+0000","last_unstale":"2024-03-28T18:06:10.611619+0000","last_undegraded":"2024-03-28T18:06:10.611619+0000","last_fullsized":"2024-03-28T18:06:10.611619+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_clean_scrub_stamp":"2024-03-28T18:04:48.465760+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T19:38:38.432027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":117,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"90'5","reported_seq":24,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-28T18:05:11.950367+0000","last_change":"2024-03-28T18:04:48.507658+0000","last_active":"2024-03-28T18:05:11.950367+0000","last_peered":"2024-03-28T18:05:11.950367+0000","last_clean":"2024-03-28T18:05:11.950367+0000","last_became_active":"2024-03-28T18:04:48.506855+0000","last_became_peered":"2024-03-28T18:04:48.506855+0000","last_unstale":"2024-03-28T18:05:11.950367+0000","last_undegraded":"2024-03-28T18:05:11.950367+0000","last_fullsized":"2024-03-28T18:05:11.950367+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:04:47.412563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:04:47.412563+0000","last_clean_scrub_stamp":"2024-03-28T18:04:47.412563+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T23:08:23.550379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"92'9","reported_seq":26,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-28T18:06:10.615429+0000","last_change":"2024-03-28T18:04:49.521122+0000","last_active":"2024-03-28T18:06:10.615429+0000","last_peered":"2024-03-28T18:06:10.615429+0000","last_clean":"2024-03-28T18:06:10.615429+0000","last_became_active":"2024-03-28T18:04:49.520711+0000","last_became_peered":"2024-03-28T18:04:49.520711+0000","last_unstale":"2024-03-28T18:06:10.615429+0000","last_undegraded":"2024-03-28T18:06:10.615429+0000","last_fullsized":"2024-03-28T18:06:10.615429+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_clean_scrub_stamp":"2024-03-28T18:04:48.465760+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T19:32:33.556979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"92'9","reported_seq":26,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-28T18:06:10.604008+0000","last_change":"2024-03-28T18:04:49.534929+0000","last_active":"2024-03-28T18:06:10.604008+0000","last_peered":"2024-03-28T18:06:10.604008+0000","last_clean":"2024-03-28T18:06:10.604008+0000","last_became_active":"2024-03-28T18:04:49.534485+0000","last_became_peered":"2024-03-28T18:04:49.534485+0000","last_unstale":"2024-03-28T18:06:10.604008+0000","last_undegraded":"2024-03-28T18:06:10.604008+0000","last_fullsized":"2024-03-28T18:06:10.604008+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_clean_scrub_stamp":"2024-03-28T18:04:48.465760+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T01:47:34.937536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"92'12","reported_seq":35,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-28T18:06:10.602737+0000","last_change":"2024-03-28T18:05:11.752731+0000","last_active":"2024-03-28T18:06:10.602737+0000","last_peered":"2024-03-28T18:06:10.602737+0000","last_clean":"2024-03-28T18:06:10.602737+0000","last_became_active":"2024-03-28T18:05:11.752121+0000","last_became_peered":"2024-03-28T18:05:11.752121+0000","last_unstale":"2024-03-28T18:06:10.602737+0000","last_undegraded":"2024-03-28T18:06:10.602737+0000","last_fullsized":"2024-03-28T18:06:10.602737+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_clean_scrub_stamp":"2024-03-28T18:04:48.465760+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T20:10:02.491231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,8],"acting":[2,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"92'8","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-28T18:06:10.609018+0000","last_change":"2024-03-28T18:04:49.531999+0000","last_active":"2024-03-28T18:06:10.609018+0000","last_peered":"2024-03-28T18:06:10.609018+0000","last_clean":"2024-03-28T18:06:10.609018+0000","last_became_active":"2024-03-28T18:04:49.528641+0000","last_became_peered":"2024-03-28T18:04:49.528641+0000","last_unstale":"2024-03-28T18:06:10.609018+0000","last_undegraded":"2024-03-28T18:06:10.609018+0000","last_fullsized":"2024-03-28T18:06:10.609018+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_clean_scrub_stamp":"2024-03-28T18:04:48.465760+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T21:24:43.861635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"92'7","reported_seq":24,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-28T18:06:10.610135+0000","last_change":"2024-03-28T18:04:49.540417+0000","last_active":"2024-03-28T18:06:10.610135+0000","last_peered":"2024-03-28T18:06:10.610135+0000","last_clean":"2024-03-28T18:06:10.610135+0000","last_became_active":"2024-03-28T18:04:49.539987+0000","last_became_peered":"2024-03-28T18:04:49.539987+0000","last_unstale":"2024-03-28T18:06:10.610135+0000","last_undegraded":"2024-03-28T18:06:10.610135+0000","last_fullsized":"2024-03-28T18:06:10.610135+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_clean_scrub_stamp":"2024-03-28T18:04:48.465760+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T02:53:41.444850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"92'4","reported_seq":21,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-28T18:06:10.609348+0000","last_change":"2024-03-28T18:04:49.540014+0000","last_active":"2024-03-28T18:06:10.609348+0000","last_peered":"2024-03-28T18:06:10.609348+0000","last_clean":"2024-03-28T18:06:10.609348+0000","last_became_active":"2024-03-28T18:04:49.538239+0000","last_became_peered":"2024-03-28T18:04:49.538239+0000","last_unstale":"2024-03-28T18:06:10.609348+0000","last_undegraded":"2024-03-28T18:06:10.609348+0000","last_fullsized":"2024-03-28T18:06:10.609348+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_clean_scrub_stamp":"2024-03-28T18:04:48.465760+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T23:03:58.449336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"92'6","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-28T18:06:10.603625+0000","last_change":"2024-03-28T18:05:11.748781+0000","last_active":"2024-03-28T18:06:10.603625+0000","last_peered":"2024-03-28T18:06:10.603625+0000","last_clean":"2024-03-28T18:06:10.603625+0000","last_became_active":"2024-03-28T18:05:11.747710+0000","last_became_peered":"2024-03-28T18:05:11.747710+0000","last_unstale":"2024-03-28T18:06:10.603625+0000","last_undegraded":"2024-03-28T18:06:10.603625+0000","last_fullsized":"2024-03-28T18:06:10.603625+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_clean_scrub_stamp":"2024-03-28T18:04:48.465760+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T02:01:44.708646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,9],"acting":[11,8,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"92'15","reported_seq":32,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-28T18:06:10.618750+0000","last_change":"2024-03-28T18:04:49.531135+0000","last_active":"2024-03-28T18:06:10.618750+0000","last_peered":"2024-03-28T18:06:10.618750+0000","last_clean":"2024-03-28T18:06:10.618750+0000","last_became_active":"2024-03-28T18:04:49.530673+0000","last_became_peered":"2024-03-28T18:04:49.530673+0000","last_unstale":"2024-03-28T18:06:10.618750+0000","last_undegraded":"2024-03-28T18:06:10.618750+0000","last_fullsized":"2024-03-28T18:06:10.618750+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_clean_scrub_stamp":"2024-03-28T18:04:48.465760+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T05:51:13.950946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"92'2","reported_seq":19,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-28T18:06:10.602517+0000","last_change":"2024-03-28T18:04:49.526803+0000","last_active":"2024-03-28T18:06:10.602517+0000","last_peered":"2024-03-28T18:06:10.602517+0000","last_clean":"2024-03-28T18:06:10.602517+0000","last_became_active":"2024-03-28T18:04:49.526474+0000","last_became_peered":"2024-03-28T18:04:49.526474+0000","last_unstale":"2024-03-28T18:06:10.602517+0000","last_undegraded":"2024-03-28T18:06:10.602517+0000","last_fullsized":"2024-03-28T18:06:10.602517+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_clean_scrub_stamp":"2024-03-28T18:04:48.465760+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T01:28:27.530864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"92'6","reported_seq":23,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-28T18:06:10.612610+0000","last_change":"2024-03-28T18:04:49.534015+0000","last_active":"2024-03-28T18:06:10.612610+0000","last_peered":"2024-03-28T18:06:10.612610+0000","last_clean":"2024-03-28T18:06:10.612610+0000","last_became_active":"2024-03-28T18:04:49.533560+0000","last_became_peered":"2024-03-28T18:04:49.533560+0000","last_unstale":"2024-03-28T18:06:10.612610+0000","last_undegraded":"2024-03-28T18:06:10.612610+0000","last_fullsized":"2024-03-28T18:06:10.612610+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_clean_scrub_stamp":"2024-03-28T18:04:48.465760+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T22:41:30.965737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"92'8","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-28T18:06:10.607634+0000","last_change":"2024-03-28T18:04:49.523240+0000","last_active":"2024-03-28T18:06:10.607634+0000","last_peered":"2024-03-28T18:06:10.607634+0000","last_clean":"2024-03-28T18:06:10.607634+0000","last_became_active":"2024-03-28T18:04:49.522795+0000","last_became_peered":"2024-03-28T18:04:49.522795+0000","last_unstale":"2024-03-28T18:06:10.607634+0000","last_undegraded":"2024-03-28T18:06:10.607634+0000","last_fullsized":"2024-03-28T18:06:10.607634+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_clean_scrub_stamp":"2024-03-28T18:04:48.465760+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T23:08:37.087977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"92'10","reported_seq":27,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-28T18:06:10.626065+0000","last_change":"2024-03-28T18:04:49.530965+0000","last_active":"2024-03-28T18:06:10.626065+0000","last_peered":"2024-03-28T18:06:10.626065+0000","last_clean":"2024-03-28T18:06:10.626065+0000","last_became_active":"2024-03-28T18:04:49.530619+0000","last_became_peered":"2024-03-28T18:04:49.530619+0000","last_unstale":"2024-03-28T18:06:10.626065+0000","last_undegraded":"2024-03-28T18:06:10.626065+0000","last_fullsized":"2024-03-28T18:06:10.626065+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_clean_scrub_stamp":"2024-03-28T18:04:48.465760+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T05:29:26.754012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"92'11","reported_seq":28,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-28T18:06:10.624663+0000","last_change":"2024-03-28T18:04:49.523017+0000","last_active":"2024-03-28T18:06:10.624663+0000","last_peered":"2024-03-28T18:06:10.624663+0000","last_clean":"2024-03-28T18:06:10.624663+0000","last_became_active":"2024-03-28T18:04:49.520411+0000","last_became_peered":"2024-03-28T18:04:49.520411+0000","last_unstale":"2024-03-28T18:06:10.624663+0000","last_undegraded":"2024-03-28T18:06:10.624663+0000","last_fullsized":"2024-03-28T18:06:10.624663+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_clean_scrub_stamp":"2024-03-28T18:04:48.465760+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T22:52:45.893036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"92'8","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-28T18:06:10.608198+0000","last_change":"2024-03-28T18:04:49.544272+0000","last_active":"2024-03-28T18:06:10.608198+0000","last_peered":"2024-03-28T18:06:10.608198+0000","last_clean":"2024-03-28T18:06:10.608198+0000","last_became_active":"2024-03-28T18:04:49.541754+0000","last_became_peered":"2024-03-28T18:04:49.541754+0000","last_unstale":"2024-03-28T18:06:10.608198+0000","last_undegraded":"2024-03-28T18:06:10.608198+0000","last_fullsized":"2024-03-28T18:06:10.608198+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_clean_scrub_stamp":"2024-03-28T18:04:48.465760+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T19:06:00.799916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":117,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":502,"ondisk_log_size":502,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4751360,"data_stored":4722944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":67426,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":832,"num_read_kb":487,"num_write":246,"num_write_kb":139,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":0,"num_keys_recovered":10,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":565248,"data_stored":202278,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":213,"ondisk_log_size":213,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":85,"seq":365072220194,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28000,"kb_used_data":984,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717152,"statfs":{"total":95995035648,"available":95966363648,"internally_reserved":0,"allocated":1007616,"data_stored":483849,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":10,"up_from":77,"seq":330712481834,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":37004,"kb_used_data":1668,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708148,"statfs":{"total":95995035648,"available":95957143552,"internally_reserved":0,"allocated":1708032,"data_stored":1135350,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":36174272},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":9,"up_from":70,"seq":300647710768,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36348,"kb_used_data":1012,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708804,"statfs":{"total":95995035648,"available":95957815296,"internally_reserved":0,"allocated":1036288,"data_stored":483896,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":36174272},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":62,"seq":266287972406,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36340,"kb_used_data":1004,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708812,"statfs":{"total":95995035648,"available":95957823488,"internally_reserved":0,"allocated":1028096,"data_stored":484157,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":7,"up_from":53,"seq":227633266749,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28604,"kb_used_data":1588,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716548,"statfs":{"total":95995035648,"available":95965745152,"internally_reserved":0,"allocated":1626112,"data_stored":1077348,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":48,"seq":206158430275,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36328,"kb_used_data":992,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708824,"statfs":{"total":95995035648,"available":95957835776,"internally_reserved":0,"allocated":1015808,"data_stored":483496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574951,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36388,"kb_used_data":1052,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708764,"statfs":{"total":95995035648,"available":95957774336,"internally_reserved":0,"allocated":1077248,"data_stored":544349,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":12,"apply_latency_ms":12,"commit_latency_ns":12000000,"apply_latency_ns":12000000},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738478,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28028,"kb_used_data":1012,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717124,"statfs":{"total":95995035648,"available":95966334976,"internally_reserved":0,"allocated":1036288,"data_stored":484835,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411426,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36408,"kb_used_data":1072,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708744,"statfs":{"total":95995035648,"available":95957753856,"internally_reserved":0,"allocated":1097728,"data_stored":545101,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":26,"seq":111669149786,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36340,"kb_used_data":1004,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708812,"statfs":{"total":95995035648,"available":95957823488,"internally_reserved":0,"allocated":1028096,"data_stored":483204,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":36174280},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":34,"seq":146028888146,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28008,"kb_used_data":992,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717144,"statfs":{"total":95995035648,"available":95966355456,"internally_reserved":0,"allocated":1015808,"data_stored":483889,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":40,"seq":171798691917,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36936,"kb_used_data":1600,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708216,"statfs":{"total":95995035648,"available":95957213184,"internally_reserved":0,"allocated":1638400,"data_stored":1076494,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2178,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":81920,"data_stored":61692,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":102400,"data_stored":62444,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1232,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":3352,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":722,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":4206,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":1500,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":62208,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-28T18:06:20.865 INFO:tasks.cephadm.ceph_manager.ceph:clean! 2024-03-28T18:06:20.865 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-03-28T18:06:20.865 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy 2024-03-28T18:06:20.866 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033 shell --fsid fd220748-ed2b-11ee-95d0-87774f69a715 -- ceph health --format=json 2024-03-28T18:06:21.032 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:06:20 smithi022 ceph-mon[31985]: from='client.14982 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-28T18:06:21.396 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/fd220748-ed2b-11ee-95d0-87774f69a715/mon.a/config 2024-03-28T18:06:21.424 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:06:20 smithi134 ceph-mon[35037]: from='client.14982 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-28T18:06:21.432 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:06:20 smithi079 ceph-mon[35312]: from='client.14982 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-28T18:06:22.282 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:06:21 smithi022 ceph-mon[31985]: pgmap v417: 97 pgs: 97 active+clean; 643 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 597 B/s wr, 11 op/s 2024-03-28T18:06:22.424 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:06:21 smithi134 ceph-mon[35037]: pgmap v417: 97 pgs: 97 active+clean; 643 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 597 B/s wr, 11 op/s 2024-03-28T18:06:22.432 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:06:21 smithi079 ceph-mon[35312]: pgmap v417: 97 pgs: 97 active+clean; 643 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 597 B/s wr, 11 op/s 2024-03-28T18:06:23.498 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:23.497+0000 7feb55000700 1 -- 172.21.15.22:0/1433116756 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7feb5010a470 msgr2=0x7feb5010a850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:06:23.498 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:23.497+0000 7feb55000700 1 --2- 172.21.15.22:0/1433116756 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7feb5010a470 0x7feb5010a850 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7feb40009990 tx=0x7feb40009ca0 comp rx=0 tx=0).stop 2024-03-28T18:06:23.498 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:23.497+0000 7feb55000700 1 -- 172.21.15.22:0/1433116756 shutdown_connections 2024-03-28T18:06:23.498 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:23.497+0000 7feb55000700 1 --2- 172.21.15.22:0/1433116756 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7feb50104da0 0x7feb501112b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:06:23.498 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:23.497+0000 7feb55000700 1 --2- 172.21.15.22:0/1433116756 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7feb5010a470 0x7feb5010a850 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:06:23.498 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:23.497+0000 7feb55000700 1 --2- 172.21.15.22:0/1433116756 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7feb501043e0 0x7feb50104860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:06:23.498 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:23.497+0000 7feb55000700 1 -- 172.21.15.22:0/1433116756 >> 172.21.15.22:0/1433116756 conn(0x7feb500fff30 msgr2=0x7feb50102350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:06:23.498 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:23.498+0000 7feb55000700 1 -- 172.21.15.22:0/1433116756 shutdown_connections 2024-03-28T18:06:23.498 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:23.498+0000 7feb55000700 1 -- 172.21.15.22:0/1433116756 wait complete. 2024-03-28T18:06:23.500 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:23.499+0000 7feb55000700 1 Processor -- start 2024-03-28T18:06:23.500 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:23.499+0000 7feb55000700 1 -- start start 2024-03-28T18:06:23.500 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:23.499+0000 7feb55000700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7feb501043e0 0x7feb5019ade0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:06:23.500 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:23.499+0000 7feb55000700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7feb50104da0 0x7feb5019b320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:06:23.500 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:23.499+0000 7feb55000700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7feb5019f6b0 0x7feb5019fb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:06:23.500 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:23.499+0000 7feb55000700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb50114700 con 0x7feb501043e0 2024-03-28T18:06:23.500 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:23.499+0000 7feb55000700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb50114580 con 0x7feb50104da0 2024-03-28T18:06:23.500 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:23.499+0000 7feb55000700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb50114880 con 0x7feb5019f6b0 2024-03-28T18:06:23.500 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:23.500+0000 7feb4de00700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7feb501043e0 0x7feb5019ade0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:06:23.500 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:23.500+0000 7feb4d400700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7feb50104da0 0x7feb5019b320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:06:23.500 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:23.500+0000 7feb4d400700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7feb50104da0 0x7feb5019b320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.22:38660/0 (socket says 172.21.15.22:38660) 2024-03-28T18:06:23.501 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:23.500+0000 7feb4d400700 1 -- 172.21.15.22:0/3555239538 learned_addr learned my addr 172.21.15.22:0/3555239538 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-28T18:06:23.501 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:23.500+0000 7feb4e800700 1 --2- 172.21.15.22:0/3555239538 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7feb5019f6b0 0x7feb5019fb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:06:23.501 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:23.500+0000 7feb4d400700 1 -- 172.21.15.22:0/3555239538 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7feb5019f6b0 msgr2=0x7feb5019fb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:06:23.501 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:23.500+0000 7feb4d400700 1 --2- 172.21.15.22:0/3555239538 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7feb5019f6b0 0x7feb5019fb30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:06:23.501 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:23.500+0000 7feb4d400700 1 -- 172.21.15.22:0/3555239538 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7feb501043e0 msgr2=0x7feb5019ade0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:06:23.501 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:23.500+0000 7feb4d400700 1 --2- 172.21.15.22:0/3555239538 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7feb501043e0 0x7feb5019ade0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:06:23.501 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:23.500+0000 7feb4d400700 1 -- 172.21.15.22:0/3555239538 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feb501a01d0 con 0x7feb50104da0 2024-03-28T18:06:23.502 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:23.500+0000 7feb4d400700 1 --2- 172.21.15.22:0/3555239538 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7feb50104da0 0x7feb5019b320 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7feb3c00eb30 tx=0x7feb3c00ee40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:06:23.502 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:23.501+0000 7feb46a00700 1 -- 172.21.15.22:0/3555239538 <== mon.2 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb3c00cc90 con 0x7feb50104da0 2024-03-28T18:06:23.502 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:23.501+0000 7feb55000700 1 -- 172.21.15.22:0/3555239538 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feb501a04c0 con 0x7feb50104da0 2024-03-28T18:06:23.502 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:23.501+0000 7feb46a00700 1 -- 172.21.15.22:0/3555239538 <== mon.2 v2:172.21.15.79:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7feb3c0044f0 con 0x7feb50104da0 2024-03-28T18:06:23.502 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:23.501+0000 7feb46a00700 1 -- 172.21.15.22:0/3555239538 <== mon.2 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb3c010430 con 0x7feb50104da0 2024-03-28T18:06:23.502 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:23.501+0000 7feb55000700 1 -- 172.21.15.22:0/3555239538 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feb501a7cf0 con 0x7feb50104da0 2024-03-28T18:06:23.502 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:23.502+0000 7feb55000700 1 -- 172.21.15.22:0/3555239538 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feb50068e50 con 0x7feb50104da0 2024-03-28T18:06:23.505 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:23.505+0000 7feb46a00700 1 -- 172.21.15.22:0/3555239538 <== mon.2 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 18) v1 ==== 137290+0+0 (secure 0 0 0) 0x7feb3c003f40 con 0x7feb50104da0 2024-03-28T18:06:23.507 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:23.507+0000 7feb46a00700 1 --2- 172.21.15.22:0/3555239538 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7feb340a2890 0x7feb340a4d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:06:23.507 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:23.507+0000 7feb46a00700 1 -- 172.21.15.22:0/3555239538 <== mon.2 v2:172.21.15.79:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 8153+0+0 (secure 0 0 0) 0x7feb3c0cee90 con 0x7feb50104da0 2024-03-28T18:06:23.507 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:23.507+0000 7feb4de00700 1 --2- 172.21.15.22:0/3555239538 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7feb340a2890 0x7feb340a4d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:06:23.507 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:23.508+0000 7feb4de00700 1 --2- 172.21.15.22:0/3555239538 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7feb340a2890 0x7feb340a4d50 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7feb400059d0 tx=0x7feb40005960 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:06:23.517 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:23.517+0000 7feb46a00700 1 -- 172.21.15.22:0/3555239538 <== mon.2 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7feb3c09bfc0 con 0x7feb50104da0 2024-03-28T18:06:23.799 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:23.799+0000 7feb55000700 1 -- 172.21.15.22:0/3555239538 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7feb5019c170 con 0x7feb50104da0 2024-03-28T18:06:23.800 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:23.800+0000 7feb46a00700 1 -- 172.21.15.22:0/3555239538 <== mon.2 v2:172.21.15.79:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+46 (secure 0 0 0) 0x7feb5019c170 con 0x7feb50104da0 2024-03-28T18:06:23.800 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-03-28T18:06:23.800 INFO:teuthology.orchestra.run.smithi022.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-03-28T18:06:23.803 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:23.803+0000 7feb55000700 1 -- 172.21.15.22:0/3555239538 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7feb340a2890 msgr2=0x7feb340a4d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:06:23.803 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:23.803+0000 7feb55000700 1 --2- 172.21.15.22:0/3555239538 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7feb340a2890 0x7feb340a4d50 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7feb400059d0 tx=0x7feb40005960 comp rx=0 tx=0).stop 2024-03-28T18:06:23.803 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:23.803+0000 7feb55000700 1 -- 172.21.15.22:0/3555239538 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7feb50104da0 msgr2=0x7feb5019b320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:06:23.803 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:23.803+0000 7feb55000700 1 --2- 172.21.15.22:0/3555239538 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7feb50104da0 0x7feb5019b320 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7feb3c00eb30 tx=0x7feb3c00ee40 comp rx=0 tx=0).stop 2024-03-28T18:06:23.803 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:23.803+0000 7feb55000700 1 -- 172.21.15.22:0/3555239538 shutdown_connections 2024-03-28T18:06:23.804 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:23.803+0000 7feb55000700 1 --2- 172.21.15.22:0/3555239538 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7feb340a2890 0x7feb340a4d50 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:06:23.804 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:23.803+0000 7feb55000700 1 --2- 172.21.15.22:0/3555239538 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7feb5019f6b0 0x7feb5019fb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:06:23.804 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:23.803+0000 7feb55000700 1 --2- 172.21.15.22:0/3555239538 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7feb501043e0 0x7feb5019ade0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:06:23.804 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:23.803+0000 7feb55000700 1 --2- 172.21.15.22:0/3555239538 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7feb50104da0 0x7feb5019b320 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:06:23.804 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:23.803+0000 7feb55000700 1 -- 172.21.15.22:0/3555239538 >> 172.21.15.22:0/3555239538 conn(0x7feb500fff30 msgr2=0x7feb50100880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:06:23.804 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:23.803+0000 7feb55000700 1 -- 172.21.15.22:0/3555239538 shutdown_connections 2024-03-28T18:06:23.804 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:23.803+0000 7feb55000700 1 -- 172.21.15.22:0/3555239538 wait complete. 2024-03-28T18:06:24.153 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:06:24 smithi022 ceph-mon[31985]: pgmap v418: 97 pgs: 97 active+clean; 643 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 597 B/s wr, 10 op/s 2024-03-28T18:06:24.153 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:06:24 smithi022 ceph-mon[31985]: from='client.? 172.21.15.22:0/3555239538' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-03-28T18:06:24.424 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:06:24 smithi134 ceph-mon[35037]: pgmap v418: 97 pgs: 97 active+clean; 643 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 597 B/s wr, 10 op/s 2024-03-28T18:06:24.425 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:06:24 smithi134 ceph-mon[35037]: from='client.? 172.21.15.22:0/3555239538' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-03-28T18:06:24.432 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:06:24 smithi079 ceph-mon[35312]: pgmap v418: 97 pgs: 97 active+clean; 643 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 597 B/s wr, 10 op/s 2024-03-28T18:06:24.432 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:06:24 smithi079 ceph-mon[35312]: from='client.? 172.21.15.22:0/3555239538' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-03-28T18:06:24.561 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy done 2024-03-28T18:06:24.562 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-28T18:06:24.777 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:24.777+0000 7f53a5000640 1 -- 172.21.15.22:0/142949480 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f53a0104ad0 msgr2=0x7f53a0104f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:06:24.777 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:24.777+0000 7f53a5000640 1 --2- 172.21.15.22:0/142949480 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f53a0104ad0 0x7f53a0104f50 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f538c009900 tx=0x7f538c02bc30 comp rx=0 tx=0).stop 2024-03-28T18:06:24.777 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:24.777+0000 7f53a5000640 1 -- 172.21.15.22:0/142949480 shutdown_connections 2024-03-28T18:06:24.777 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:24.777+0000 7f53a5000640 1 --2- 172.21.15.22:0/142949480 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f53a0105490 0x7f53a010bd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:06:24.777 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:24.777+0000 7f53a5000640 1 --2- 172.21.15.22:0/142949480 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f53a0104ad0 0x7f53a0104f50 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:06:24.777 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:24.777+0000 7f53a5000640 1 --2- 172.21.15.22:0/142949480 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f53a01038e0 0x7f53a0103d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:06:24.778 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:24.777+0000 7f53a5000640 1 -- 172.21.15.22:0/142949480 >> 172.21.15.22:0/142949480 conn(0x7f53a00fee90 msgr2=0x7f53a01012b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:06:24.778 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:24.777+0000 7f53a5000640 1 -- 172.21.15.22:0/142949480 shutdown_connections 2024-03-28T18:06:24.778 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:24.777+0000 7f53a5000640 1 -- 172.21.15.22:0/142949480 wait complete. 2024-03-28T18:06:24.780 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:24.778+0000 7f53a5000640 1 Processor -- start 2024-03-28T18:06:24.780 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:24.778+0000 7f53a5000640 1 -- start start 2024-03-28T18:06:24.780 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:24.778+0000 7f53a5000640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f53a01038e0 0x7f53a0199be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:06:24.781 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:24.778+0000 7f53a5000640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f53a0104ad0 0x7f53a019a120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:06:24.781 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:24.779+0000 7f53a5000640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f53a0105490 0x7f53a01a1190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:06:24.781 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:24.779+0000 7f53a5000640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53a010f590 con 0x7f53a01038e0 2024-03-28T18:06:24.781 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:24.779+0000 7f53a5000640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53a010f410 con 0x7f53a0104ad0 2024-03-28T18:06:24.781 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:24.779+0000 7f53a5000640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53a010f710 con 0x7f53a0105490 2024-03-28T18:06:24.781 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:24.779+0000 7f539d400640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f53a0104ad0 0x7f53a019a120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:06:24.781 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:24.779+0000 7f539de00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f53a01038e0 0x7f53a0199be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:06:24.781 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:24.779+0000 7f539d400640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f53a0104ad0 0x7f53a019a120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.22:38682/0 (socket says 172.21.15.22:38682) 2024-03-28T18:06:24.781 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:24.779+0000 7f539d400640 1 -- 172.21.15.22:0/2442563231 learned_addr learned my addr 172.21.15.22:0/2442563231 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-28T18:06:24.781 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:24.779+0000 7f539e800640 1 --2- 172.21.15.22:0/2442563231 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f53a0105490 0x7f53a01a1190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:06:24.781 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:24.779+0000 7f539de00640 1 -- 172.21.15.22:0/2442563231 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f53a0105490 msgr2=0x7f53a01a1190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:06:24.782 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:24.779+0000 7f539de00640 1 --2- 172.21.15.22:0/2442563231 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f53a0105490 0x7f53a01a1190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:06:24.782 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:24.779+0000 7f539de00640 1 -- 172.21.15.22:0/2442563231 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f53a0104ad0 msgr2=0x7f53a019a120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:06:24.782 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:24.779+0000 7f539de00640 1 --2- 172.21.15.22:0/2442563231 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f53a0104ad0 0x7f53a019a120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:06:24.782 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:24.779+0000 7f539de00640 1 -- 172.21.15.22:0/2442563231 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f53a01a1800 con 0x7f53a01038e0 2024-03-28T18:06:24.782 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:24.779+0000 7f539d400640 1 --2- 172.21.15.22:0/2442563231 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f53a0104ad0 0x7f53a019a120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-28T18:06:24.782 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:24.779+0000 7f539e800640 1 --2- 172.21.15.22:0/2442563231 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f53a0105490 0x7f53a01a1190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-28T18:06:24.782 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:24.780+0000 7f539de00640 1 --2- 172.21.15.22:0/2442563231 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f53a01038e0 0x7f53a0199be0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f539000d6a0 tx=0x7f539000db60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:06:24.782 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:24.780+0000 7f5396a00640 1 -- 172.21.15.22:0/2442563231 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5390017070 con 0x7f53a01038e0 2024-03-28T18:06:24.782 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:24.780+0000 7f53a5000640 1 -- 172.21.15.22:0/2442563231 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f53a01a1af0 con 0x7f53a01038e0 2024-03-28T18:06:24.782 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:24.780+0000 7f5396a00640 1 -- 172.21.15.22:0/2442563231 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f5390009cf0 con 0x7f53a01038e0 2024-03-28T18:06:24.782 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:24.780+0000 7f53a5000640 1 -- 172.21.15.22:0/2442563231 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f53a01a2090 con 0x7f53a01038e0 2024-03-28T18:06:24.783 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:24.780+0000 7f5396a00640 1 -- 172.21.15.22:0/2442563231 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5390020060 con 0x7f53a01038e0 2024-03-28T18:06:24.783 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:24.781+0000 7f53a5000640 1 -- 172.21.15.22:0/2442563231 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f53a010ac00 con 0x7f53a01038e0 2024-03-28T18:06:24.783 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:24.784+0000 7f5396a00640 1 -- 172.21.15.22:0/2442563231 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 18) v1 ==== 137290+0+0 (secure 0 0 0) 0x7f5390003c30 con 0x7f53a01038e0 2024-03-28T18:06:24.785 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:24.785+0000 7f5396a00640 1 --2- 172.21.15.22:0/2442563231 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7f53840a2800 0x7f53840a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:06:24.785 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:24.785+0000 7f5396a00640 1 -- 172.21.15.22:0/2442563231 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 8153+0+0 (secure 0 0 0) 0x7f53900ce4e0 con 0x7f53a01038e0 2024-03-28T18:06:24.785 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:24.785+0000 7f539d400640 1 --2- 172.21.15.22:0/2442563231 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7f53840a2800 0x7f53840a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:06:24.786 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:24.786+0000 7f539d400640 1 --2- 172.21.15.22:0/2442563231 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7f53840a2800 0x7f53840a4cc0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f538c02c710 tx=0x7f538c02c820 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:06:24.789 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:24.790+0000 7f5396a00640 1 -- 172.21.15.22:0/2442563231 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f539009b580 con 0x7f53a01038e0 2024-03-28T18:06:24.996 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:24.996+0000 7f53a5000640 1 -- 172.21.15.22:0/2442563231 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f53a002d070 con 0x7f53a01038e0 2024-03-28T18:06:25.000 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:25.001+0000 7f5396a00640 1 -- 172.21.15.22:0/2442563231 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v92) v1 ==== 74+0+22522 (secure 0 0 0) 0x7f539009ae80 con 0x7f53a01038e0 2024-03-28T18:06:25.001 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-03-28T18:06:25.001 INFO:teuthology.orchestra.run.smithi022.stdout:{"epoch":92,"fsid":"fd220748-ed2b-11ee-95d0-87774f69a715","created":"2024-03-28T17:54:00.440279+0000","modified":"2024-03-28T18:05:11.725341+0000","last_up_change":"2024-03-28T18:03:43.220095+0000","last_in_change":"2024-03-28T18:03:18.526399+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-28T17:58:33.168872+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-28T18:04:47.004797+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-28T18:04:48.465502+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"90","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"5d708e06-f05b-40ba-bbcf-258ddc4f43d7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6802","nonce":2186290846},{"type":"v1","addr":"172.21.15.22:6803","nonce":2186290846}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6804","nonce":2186290846},{"type":"v1","addr":"172.21.15.22:6805","nonce":2186290846}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6808","nonce":2186290846},{"type":"v1","addr":"172.21.15.22:6809","nonce":2186290846}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6806","nonce":2186290846},{"type":"v1","addr":"172.21.15.22:6807","nonce":2186290846}]},"public_addr":"172.21.15.22:6803/2186290846","cluster_addr":"172.21.15.22:6805/2186290846","heartbeat_back_addr":"172.21.15.22:6809/2186290846","heartbeat_front_addr":"172.21.15.22:6807/2186290846","state":["exists","up"]},{"osd":1,"uuid":"68ad09bb-9256-409a-9867-4e14a2cb94aa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6800","nonce":2835549681},{"type":"v1","addr":"172.21.15.79:6801","nonce":2835549681}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6802","nonce":2835549681},{"type":"v1","addr":"172.21.15.79:6803","nonce":2835549681}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6806","nonce":2835549681},{"type":"v1","addr":"172.21.15.79:6807","nonce":2835549681}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6804","nonce":2835549681},{"type":"v1","addr":"172.21.15.79:6805","nonce":2835549681}]},"public_addr":"172.21.15.79:6801/2835549681","cluster_addr":"172.21.15.79:6803/2835549681","heartbeat_back_addr":"172.21.15.79:6807/2835549681","heartbeat_front_addr":"172.21.15.79:6805/2835549681","state":["exists","up"]},{"osd":2,"uuid":"ed11a5a7-004d-41f3-a5fc-eb2d603bd33f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6800","nonce":470588085},{"type":"v1","addr":"172.21.15.134:6801","nonce":470588085}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6802","nonce":470588085},{"type":"v1","addr":"172.21.15.134:6803","nonce":470588085}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6806","nonce":470588085},{"type":"v1","addr":"172.21.15.134:6807","nonce":470588085}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6804","nonce":470588085},{"type":"v1","addr":"172.21.15.134:6805","nonce":470588085}]},"public_addr":"172.21.15.134:6801/470588085","cluster_addr":"172.21.15.134:6803/470588085","heartbeat_back_addr":"172.21.15.134:6807/470588085","heartbeat_front_addr":"172.21.15.134:6805/470588085","state":["exists","up"]},{"osd":3,"uuid":"81a6c547-2a50-44c3-84d7-23ab6831790c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6810","nonce":1702441854},{"type":"v1","addr":"172.21.15.22:6811","nonce":1702441854}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6812","nonce":1702441854},{"type":"v1","addr":"172.21.15.22:6813","nonce":1702441854}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6816","nonce":1702441854},{"type":"v1","addr":"172.21.15.22:6817","nonce":1702441854}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6814","nonce":1702441854},{"type":"v1","addr":"172.21.15.22:6815","nonce":1702441854}]},"public_addr":"172.21.15.22:6811/1702441854","cluster_addr":"172.21.15.22:6813/1702441854","heartbeat_back_addr":"172.21.15.22:6817/1702441854","heartbeat_front_addr":"172.21.15.22:6815/1702441854","state":["exists","up"]},{"osd":4,"uuid":"d23a254e-7182-4471-8bca-7eb684ec618e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6808","nonce":630132437},{"type":"v1","addr":"172.21.15.79:6809","nonce":630132437}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6810","nonce":630132437},{"type":"v1","addr":"172.21.15.79:6811","nonce":630132437}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6814","nonce":630132437},{"type":"v1","addr":"172.21.15.79:6815","nonce":630132437}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6812","nonce":630132437},{"type":"v1","addr":"172.21.15.79:6813","nonce":630132437}]},"public_addr":"172.21.15.79:6809/630132437","cluster_addr":"172.21.15.79:6811/630132437","heartbeat_back_addr":"172.21.15.79:6815/630132437","heartbeat_front_addr":"172.21.15.79:6813/630132437","state":["exists","up"]},{"osd":5,"uuid":"a7b98502-2023-4f0a-bfa0-6840cc495e0c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":40,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6808","nonce":2135406915},{"type":"v1","addr":"172.21.15.134:6809","nonce":2135406915}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6810","nonce":2135406915},{"type":"v1","addr":"172.21.15.134:6811","nonce":2135406915}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6814","nonce":2135406915},{"type":"v1","addr":"172.21.15.134:6815","nonce":2135406915}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6812","nonce":2135406915},{"type":"v1","addr":"172.21.15.134:6813","nonce":2135406915}]},"public_addr":"172.21.15.134:6809/2135406915","cluster_addr":"172.21.15.134:6811/2135406915","heartbeat_back_addr":"172.21.15.134:6815/2135406915","heartbeat_front_addr":"172.21.15.134:6813/2135406915","state":["exists","up"]},{"osd":6,"uuid":"634c6562-d683-41c8-8242-fc848c38ce1f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6818","nonce":1400680396},{"type":"v1","addr":"172.21.15.22:6819","nonce":1400680396}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6820","nonce":1400680396},{"type":"v1","addr":"172.21.15.22:6821","nonce":1400680396}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6824","nonce":1400680396},{"type":"v1","addr":"172.21.15.22:6825","nonce":1400680396}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6822","nonce":1400680396},{"type":"v1","addr":"172.21.15.22:6823","nonce":1400680396}]},"public_addr":"172.21.15.22:6819/1400680396","cluster_addr":"172.21.15.22:6821/1400680396","heartbeat_back_addr":"172.21.15.22:6825/1400680396","heartbeat_front_addr":"172.21.15.22:6823/1400680396","state":["exists","up"]},{"osd":7,"uuid":"504eb4a8-6731-45ad-932c-1b38ef157871","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":53,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6816","nonce":1287105314},{"type":"v1","addr":"172.21.15.79:6817","nonce":1287105314}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6818","nonce":1287105314},{"type":"v1","addr":"172.21.15.79:6819","nonce":1287105314}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6822","nonce":1287105314},{"type":"v1","addr":"172.21.15.79:6823","nonce":1287105314}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6820","nonce":1287105314},{"type":"v1","addr":"172.21.15.79:6821","nonce":1287105314}]},"public_addr":"172.21.15.79:6817/1287105314","cluster_addr":"172.21.15.79:6819/1287105314","heartbeat_back_addr":"172.21.15.79:6823/1287105314","heartbeat_front_addr":"172.21.15.79:6821/1287105314","state":["exists","up"]},{"osd":8,"uuid":"20aa66c1-03d6-4cb8-a096-0eae41979c17","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":62,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6816","nonce":2265650388},{"type":"v1","addr":"172.21.15.134:6817","nonce":2265650388}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6818","nonce":2265650388},{"type":"v1","addr":"172.21.15.134:6819","nonce":2265650388}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6822","nonce":2265650388},{"type":"v1","addr":"172.21.15.134:6823","nonce":2265650388}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6820","nonce":2265650388},{"type":"v1","addr":"172.21.15.134:6821","nonce":2265650388}]},"public_addr":"172.21.15.134:6817/2265650388","cluster_addr":"172.21.15.134:6819/2265650388","heartbeat_back_addr":"172.21.15.134:6823/2265650388","heartbeat_front_addr":"172.21.15.134:6821/2265650388","state":["exists","up"]},{"osd":9,"uuid":"e1ae4020-7c2c-4334-a753-ee6b12d98584","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":70,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6826","nonce":3762222259},{"type":"v1","addr":"172.21.15.22:6827","nonce":3762222259}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6828","nonce":3762222259},{"type":"v1","addr":"172.21.15.22:6829","nonce":3762222259}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6832","nonce":3762222259},{"type":"v1","addr":"172.21.15.22:6833","nonce":3762222259}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6830","nonce":3762222259},{"type":"v1","addr":"172.21.15.22:6831","nonce":3762222259}]},"public_addr":"172.21.15.22:6827/3762222259","cluster_addr":"172.21.15.22:6829/3762222259","heartbeat_back_addr":"172.21.15.22:6833/3762222259","heartbeat_front_addr":"172.21.15.22:6831/3762222259","state":["exists","up"]},{"osd":10,"uuid":"f5198d52-df48-4679-a860-4dcb440b7a9f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":77,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6824","nonce":1746167904},{"type":"v1","addr":"172.21.15.79:6825","nonce":1746167904}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6826","nonce":1746167904},{"type":"v1","addr":"172.21.15.79:6827","nonce":1746167904}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6830","nonce":1746167904},{"type":"v1","addr":"172.21.15.79:6831","nonce":1746167904}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6828","nonce":1746167904},{"type":"v1","addr":"172.21.15.79:6829","nonce":1746167904}]},"public_addr":"172.21.15.79:6825/1746167904","cluster_addr":"172.21.15.79:6827/1746167904","heartbeat_back_addr":"172.21.15.79:6831/1746167904","heartbeat_front_addr":"172.21.15.79:6829/1746167904","state":["exists","up"]},{"osd":11,"uuid":"1427bca7-3414-4468-9246-e999156bc2ae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":85,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6824","nonce":4271716428},{"type":"v1","addr":"172.21.15.134:6825","nonce":4271716428}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6826","nonce":4271716428},{"type":"v1","addr":"172.21.15.134:6827","nonce":4271716428}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6830","nonce":4271716428},{"type":"v1","addr":"172.21.15.134:6831","nonce":4271716428}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6828","nonce":4271716428},{"type":"v1","addr":"172.21.15.134:6829","nonce":4271716428}]},"public_addr":"172.21.15.134:6825/4271716428","cluster_addr":"172.21.15.134:6827/4271716428","heartbeat_back_addr":"172.21.15.134:6831/4271716428","heartbeat_front_addr":"172.21.15.134:6829/4271716428","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T17:57:21.225125+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T17:57:55.197254+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T17:58:27.825136+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T17:59:04.319893+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T17:59:38.407752+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T18:00:12.118805+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T18:00:47.635214+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T18:01:21.231055+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T18:01:55.817861+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T18:02:31.680160+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T18:03:06.783398+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T18:03:41.307347+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.8","mappings":[{"from":2,"to":11}]},{"pgid":"2.1d","mappings":[{"from":8,"to":11}]},{"pgid":"3.3","mappings":[{"from":4,"to":9}]},{"pgid":"3.c","mappings":[{"from":3,"to":9}]},{"pgid":"3.21","mappings":[{"from":4,"to":8}]},{"pgid":"3.25","mappings":[{"from":5,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.22:0/3521924283":"2024-03-29T17:55:10.425504+0000","172.21.15.22:6801/730635939":"2024-03-29T17:55:10.425504+0000","172.21.15.22:0/2471229459":"2024-03-29T17:54:34.004559+0000","172.21.15.22:6800/3166828081":"2024-03-29T17:54:34.004559+0000","172.21.15.22:6800/730635939":"2024-03-29T17:55:10.425504+0000","172.21.15.22:0/1229047849":"2024-03-29T17:54:34.004559+0000","172.21.15.22:6801/3166828081":"2024-03-29T17:54:34.004559+0000","172.21.15.22:0/2717818856":"2024-03-29T17:55:10.425504+0000","172.21.15.22:0/2157570333":"2024-03-29T17:54:34.004559+0000","172.21.15.22:0/1006589530":"2024-03-29T17:55:10.425504+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-28T18:06:25.004 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:25.004+0000 7f53a5000640 1 -- 172.21.15.22:0/2442563231 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7f53840a2800 msgr2=0x7f53840a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:06:25.004 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:25.004+0000 7f53a5000640 1 --2- 172.21.15.22:0/2442563231 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7f53840a2800 0x7f53840a4cc0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f538c02c710 tx=0x7f538c02c820 comp rx=0 tx=0).stop 2024-03-28T18:06:25.004 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:25.004+0000 7f53a5000640 1 -- 172.21.15.22:0/2442563231 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f53a01038e0 msgr2=0x7f53a0199be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:06:25.004 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:25.004+0000 7f53a5000640 1 --2- 172.21.15.22:0/2442563231 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f53a01038e0 0x7f53a0199be0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f539000d6a0 tx=0x7f539000db60 comp rx=0 tx=0).stop 2024-03-28T18:06:25.004 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:25.004+0000 7f53a5000640 1 -- 172.21.15.22:0/2442563231 shutdown_connections 2024-03-28T18:06:25.004 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:25.004+0000 7f53a5000640 1 --2- 172.21.15.22:0/2442563231 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7f53840a2800 0x7f53840a4cc0 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:06:25.004 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:25.004+0000 7f53a5000640 1 --2- 172.21.15.22:0/2442563231 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f53a0105490 0x7f53a01a1190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:06:25.004 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:25.004+0000 7f53a5000640 1 --2- 172.21.15.22:0/2442563231 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f53a0104ad0 0x7f53a019a120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:06:25.004 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:25.004+0000 7f53a5000640 1 --2- 172.21.15.22:0/2442563231 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f53a01038e0 0x7f53a0199be0 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:06:25.005 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:25.004+0000 7f53a5000640 1 -- 172.21.15.22:0/2442563231 >> 172.21.15.22:0/2442563231 conn(0x7f53a00fee90 msgr2=0x7f53a0107dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:06:25.005 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:25.004+0000 7f53a5000640 1 -- 172.21.15.22:0/2442563231 shutdown_connections 2024-03-28T18:06:25.005 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:25.004+0000 7f53a5000640 1 -- 172.21.15.22:0/2442563231 wait complete. 2024-03-28T18:06:25.024 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-28T17:58:33.168872+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-28T18:04:47.004797+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '89', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-03-28T18:04:48.465502+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '90', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-28T18:06:25.025 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-28T18:06:25.243 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:25.243+0000 7fc30fe00640 1 -- 172.21.15.22:0/2220661864 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc310101670 msgr2=0x7fc310101af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:06:25.243 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:25.243+0000 7fc30fe00640 1 --2- 172.21.15.22:0/2220661864 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc310101670 0x7fc310101af0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fc2fc009900 tx=0x7fc2fc02bc30 comp rx=0 tx=0).stop 2024-03-28T18:06:25.243 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:25.243+0000 7fc30fe00640 1 -- 172.21.15.22:0/2220661864 shutdown_connections 2024-03-28T18:06:25.243 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:25.243+0000 7fc30fe00640 1 --2- 172.21.15.22:0/2220661864 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc310102030 0x7fc3101088c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:06:25.243 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:25.243+0000 7fc30fe00640 1 --2- 172.21.15.22:0/2220661864 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc310101670 0x7fc310101af0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:06:25.243 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:25.243+0000 7fc30fe00640 1 --2- 172.21.15.22:0/2220661864 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc310100480 0x7fc3101008a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:06:25.243 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:25.243+0000 7fc30fe00640 1 -- 172.21.15.22:0/2220661864 >> 172.21.15.22:0/2220661864 conn(0x7fc3100fba10 msgr2=0x7fc3100fde30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:06:25.243 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:25.243+0000 7fc30fe00640 1 -- 172.21.15.22:0/2220661864 shutdown_connections 2024-03-28T18:06:25.244 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:25.243+0000 7fc30fe00640 1 -- 172.21.15.22:0/2220661864 wait complete. 2024-03-28T18:06:25.245 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:25.244+0000 7fc30fe00640 1 Processor -- start 2024-03-28T18:06:25.245 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:25.244+0000 7fc30fe00640 1 -- start start 2024-03-28T18:06:25.246 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:25.244+0000 7fc30fe00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc310100480 0x7fc310199af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:06:25.246 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:25.244+0000 7fc30fe00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc310101670 0x7fc31019a030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:06:25.246 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:25.244+0000 7fc30fe00640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc310102030 0x7fc3101a10a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:06:25.246 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:25.244+0000 7fc30fe00640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc31010c130 con 0x7fc310101670 2024-03-28T18:06:25.246 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:25.244+0000 7fc30fe00640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc31010bfb0 con 0x7fc310102030 2024-03-28T18:06:25.246 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:25.244+0000 7fc30fe00640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc31010c2b0 con 0x7fc310100480 2024-03-28T18:06:25.246 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:25.245+0000 7fc30e000640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc310101670 0x7fc31019a030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:06:25.246 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:25.245+0000 7fc30e000640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc310101670 0x7fc31019a030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:49956/0 (socket says 172.21.15.22:49956) 2024-03-28T18:06:25.246 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:25.245+0000 7fc30e000640 1 -- 172.21.15.22:0/3699539692 learned_addr learned my addr 172.21.15.22:0/3699539692 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-28T18:06:25.246 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:25.245+0000 7fc30f400640 1 --2- 172.21.15.22:0/3699539692 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc310102030 0x7fc3101a10a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:06:25.246 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:25.245+0000 7fc30ea00640 1 --2- 172.21.15.22:0/3699539692 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc310100480 0x7fc310199af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:06:25.246 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:25.245+0000 7fc30e000640 1 -- 172.21.15.22:0/3699539692 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc310100480 msgr2=0x7fc310199af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:06:25.246 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:25.245+0000 7fc30e000640 1 --2- 172.21.15.22:0/3699539692 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc310100480 0x7fc310199af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:06:25.247 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:25.245+0000 7fc30e000640 1 -- 172.21.15.22:0/3699539692 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc310102030 msgr2=0x7fc3101a10a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:06:25.247 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:25.245+0000 7fc30e000640 1 --2- 172.21.15.22:0/3699539692 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc310102030 0x7fc3101a10a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:06:25.247 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:25.245+0000 7fc30e000640 1 -- 172.21.15.22:0/3699539692 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc3101a1710 con 0x7fc310101670 2024-03-28T18:06:25.247 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:25.245+0000 7fc30f400640 1 --2- 172.21.15.22:0/3699539692 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc310102030 0x7fc3101a10a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-28T18:06:25.247 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:25.245+0000 7fc30e000640 1 --2- 172.21.15.22:0/3699539692 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc310101670 0x7fc31019a030 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7fc2fc000c00 tx=0x7fc2fc004e20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:06:25.247 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:25.246+0000 7fc307400640 1 -- 172.21.15.22:0/3699539692 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2fc043020 con 0x7fc310101670 2024-03-28T18:06:25.247 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:25.246+0000 7fc307400640 1 -- 172.21.15.22:0/3699539692 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7fc2fc005460 con 0x7fc310101670 2024-03-28T18:06:25.247 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:25.246+0000 7fc307400640 1 -- 172.21.15.22:0/3699539692 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2fc003d00 con 0x7fc310101670 2024-03-28T18:06:25.247 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:25.246+0000 7fc30fe00640 1 -- 172.21.15.22:0/3699539692 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc3101a19a0 con 0x7fc310101670 2024-03-28T18:06:25.247 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:25.246+0000 7fc30fe00640 1 -- 172.21.15.22:0/3699539692 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc3101a1ef0 con 0x7fc310101670 2024-03-28T18:06:25.247 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:25.247+0000 7fc30fe00640 1 -- 172.21.15.22:0/3699539692 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc3101a20c0 con 0x7fc310101670 2024-03-28T18:06:25.249 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:25.249+0000 7fc307400640 1 -- 172.21.15.22:0/3699539692 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 18) v1 ==== 137290+0+0 (secure 0 0 0) 0x7fc2fc046020 con 0x7fc310101670 2024-03-28T18:06:25.250 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:25.250+0000 7fc307400640 1 --2- 172.21.15.22:0/3699539692 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7fc2f40a2800 0x7fc2f40a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:06:25.250 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:25.251+0000 7fc307400640 1 -- 172.21.15.22:0/3699539692 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 8153+0+0 (secure 0 0 0) 0x7fc2fc0f10f0 con 0x7fc310101670 2024-03-28T18:06:25.250 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:25.251+0000 7fc30ea00640 1 --2- 172.21.15.22:0/3699539692 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7fc2f40a2800 0x7fc2f40a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:06:25.251 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:25.251+0000 7fc30ea00640 1 --2- 172.21.15.22:0/3699539692 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7fc2f40a2800 0x7fc2f40a4cc0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fc300005fd0 tx=0x7fc300005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:06:25.253 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:25.253+0000 7fc307400640 1 -- 172.21.15.22:0/3699539692 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fc2fc04d020 con 0x7fc310101670 2024-03-28T18:06:25.281 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:06:25 smithi022 ceph-mon[31985]: from='client.? 172.21.15.22:0/2442563231' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-28T18:06:25.424 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:06:25 smithi134 ceph-mon[35037]: from='client.? 172.21.15.22:0/2442563231' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-28T18:06:25.432 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:06:25 smithi079 ceph-mon[35312]: from='client.? 172.21.15.22:0/2442563231' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-28T18:06:25.456 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:25.457+0000 7fc30fe00640 1 -- 172.21.15.22:0/3699539692 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fc3101a2290 con 0x7fc310101670 2024-03-28T18:06:25.457 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:25.457+0000 7fc307400640 1 -- 172.21.15.22:0/3699539692 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v92) v1 ==== 93+0+10 (secure 0 0 0) 0x7fc2fc0be190 con 0x7fc310101670 2024-03-28T18:06:25.457 INFO:teuthology.orchestra.run.smithi022.stdout:pg_num: 1 2024-03-28T18:06:25.460 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:25.460+0000 7fc30fe00640 1 -- 172.21.15.22:0/3699539692 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7fc2f40a2800 msgr2=0x7fc2f40a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:06:25.460 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:25.460+0000 7fc30fe00640 1 --2- 172.21.15.22:0/3699539692 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7fc2f40a2800 0x7fc2f40a4cc0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fc300005fd0 tx=0x7fc300005850 comp rx=0 tx=0).stop 2024-03-28T18:06:25.460 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:25.460+0000 7fc30fe00640 1 -- 172.21.15.22:0/3699539692 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc310101670 msgr2=0x7fc31019a030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:06:25.460 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:25.460+0000 7fc30fe00640 1 --2- 172.21.15.22:0/3699539692 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc310101670 0x7fc31019a030 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7fc2fc000c00 tx=0x7fc2fc004e20 comp rx=0 tx=0).stop 2024-03-28T18:06:25.460 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:25.460+0000 7fc30fe00640 1 -- 172.21.15.22:0/3699539692 shutdown_connections 2024-03-28T18:06:25.460 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:25.460+0000 7fc30fe00640 1 --2- 172.21.15.22:0/3699539692 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7fc2f40a2800 0x7fc2f40a4cc0 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:06:25.460 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:25.460+0000 7fc30fe00640 1 --2- 172.21.15.22:0/3699539692 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc310102030 0x7fc3101a10a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:06:25.461 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:25.460+0000 7fc30fe00640 1 --2- 172.21.15.22:0/3699539692 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc310101670 0x7fc31019a030 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:06:25.461 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:25.460+0000 7fc30fe00640 1 --2- 172.21.15.22:0/3699539692 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc310100480 0x7fc310199af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:06:25.461 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:25.460+0000 7fc30fe00640 1 -- 172.21.15.22:0/3699539692 >> 172.21.15.22:0/3699539692 conn(0x7fc3100fba10 msgr2=0x7fc3100fd4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:06:25.461 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:25.461+0000 7fc30fe00640 1 -- 172.21.15.22:0/3699539692 shutdown_connections 2024-03-28T18:06:25.461 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:25.461+0000 7fc30fe00640 1 -- 172.21.15.22:0/3699539692 wait complete. 2024-03-28T18:06:25.479 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-28T18:06:25.689 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:25.688+0000 7ff709400640 1 -- 172.21.15.22:0/9173370 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff704100480 msgr2=0x7ff7041008a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:06:25.689 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:25.688+0000 7ff709400640 1 --2- 172.21.15.22:0/9173370 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff704100480 0x7ff7041008a0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7ff6f4009900 tx=0x7ff6f402bc30 comp rx=0 tx=0).stop 2024-03-28T18:06:25.689 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:25.688+0000 7ff709400640 1 -- 172.21.15.22:0/9173370 shutdown_connections 2024-03-28T18:06:25.689 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:25.688+0000 7ff709400640 1 --2- 172.21.15.22:0/9173370 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff704102030 0x7ff7041088c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:06:25.689 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:25.688+0000 7ff709400640 1 --2- 172.21.15.22:0/9173370 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ff704101670 0x7ff704101af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:06:25.689 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:25.688+0000 7ff709400640 1 --2- 172.21.15.22:0/9173370 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff704100480 0x7ff7041008a0 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:06:25.689 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:25.688+0000 7ff709400640 1 -- 172.21.15.22:0/9173370 >> 172.21.15.22:0/9173370 conn(0x7ff7040fba10 msgr2=0x7ff7040fde30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:06:25.689 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:25.688+0000 7ff709400640 1 -- 172.21.15.22:0/9173370 shutdown_connections 2024-03-28T18:06:25.689 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:25.689+0000 7ff709400640 1 -- 172.21.15.22:0/9173370 wait complete. 2024-03-28T18:06:25.691 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:25.689+0000 7ff709400640 1 Processor -- start 2024-03-28T18:06:25.691 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:25.690+0000 7ff709400640 1 -- start start 2024-03-28T18:06:25.691 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:25.690+0000 7ff709400640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ff704101670 0x7ff704104410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:06:25.692 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:25.690+0000 7ff709400640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff704102030 0x7ff704102a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:06:25.692 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:25.690+0000 7ff709400640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff704102f90 0x7ff704103440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:06:25.692 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:25.690+0000 7ff709400640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff70410c130 con 0x7ff704102f90 2024-03-28T18:06:25.692 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:25.690+0000 7ff709400640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff70410bfb0 con 0x7ff704101670 2024-03-28T18:06:25.692 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:25.690+0000 7ff709400640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff70410c2b0 con 0x7ff704102030 2024-03-28T18:06:25.692 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:25.690+0000 7ff708a00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff704102f90 0x7ff704103440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:06:25.692 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:25.690+0000 7ff703e00640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ff704101670 0x7ff704104410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:06:25.692 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:25.690+0000 7ff708a00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff704102f90 0x7ff704103440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:49976/0 (socket says 172.21.15.22:49976) 2024-03-28T18:06:25.692 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:25.690+0000 7ff708a00640 1 -- 172.21.15.22:0/3518460910 learned_addr learned my addr 172.21.15.22:0/3518460910 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-28T18:06:25.692 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:25.691+0000 7ff703400640 1 --2- 172.21.15.22:0/3518460910 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff704102030 0x7ff704102a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:06:25.692 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:25.691+0000 7ff708a00640 1 -- 172.21.15.22:0/3518460910 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff704102030 msgr2=0x7ff704102a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:06:25.692 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:25.691+0000 7ff708a00640 1 --2- 172.21.15.22:0/3518460910 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff704102030 0x7ff704102a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:06:25.693 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:25.691+0000 7ff708a00640 1 -- 172.21.15.22:0/3518460910 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ff704101670 msgr2=0x7ff704104410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:06:25.693 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:25.691+0000 7ff708a00640 1 --2- 172.21.15.22:0/3518460910 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ff704101670 0x7ff704104410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:06:25.693 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:25.691+0000 7ff708a00640 1 -- 172.21.15.22:0/3518460910 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff7041a1d00 con 0x7ff704102f90 2024-03-28T18:06:25.693 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:25.691+0000 7ff708a00640 1 --2- 172.21.15.22:0/3518460910 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff704102f90 0x7ff704103440 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7ff6fc00eaa0 tx=0x7ff6fc00ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:06:25.693 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:25.691+0000 7ff700c00640 1 -- 172.21.15.22:0/3518460910 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6fc00cdb0 con 0x7ff704102f90 2024-03-28T18:06:25.693 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:25.691+0000 7ff700c00640 1 -- 172.21.15.22:0/3518460910 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7ff6fc005020 con 0x7ff704102f90 2024-03-28T18:06:25.693 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:25.691+0000 7ff709400640 1 -- 172.21.15.22:0/3518460910 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff7041a1ff0 con 0x7ff704102f90 2024-03-28T18:06:25.693 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:25.691+0000 7ff700c00640 1 -- 172.21.15.22:0/3518460910 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6fc0116d0 con 0x7ff704102f90 2024-03-28T18:06:25.693 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:25.691+0000 7ff709400640 1 -- 172.21.15.22:0/3518460910 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff7041a24b0 con 0x7ff704102f90 2024-03-28T18:06:25.694 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:25.692+0000 7ff709400640 1 -- 172.21.15.22:0/3518460910 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff7041a2770 con 0x7ff704102f90 2024-03-28T18:06:25.694 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:25.695+0000 7ff700c00640 1 -- 172.21.15.22:0/3518460910 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 18) v1 ==== 137290+0+0 (secure 0 0 0) 0x7ff6fc0051d0 con 0x7ff704102f90 2024-03-28T18:06:25.696 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:25.696+0000 7ff700c00640 1 --2- 172.21.15.22:0/3518460910 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7ff6e40a2880 0x7ff6e40a4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:06:25.696 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:25.696+0000 7ff700c00640 1 -- 172.21.15.22:0/3518460910 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 8153+0+0 (secure 0 0 0) 0x7ff6fc0cf050 con 0x7ff704102f90 2024-03-28T18:06:25.696 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:25.696+0000 7ff703e00640 1 --2- 172.21.15.22:0/3518460910 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7ff6e40a2880 0x7ff6e40a4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:06:25.696 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:25.697+0000 7ff703e00640 1 --2- 172.21.15.22:0/3518460910 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7ff6e40a2880 0x7ff6e40a4d40 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7ff6f402c240 tx=0x7ff6f402c820 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:06:25.698 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:25.698+0000 7ff700c00640 1 -- 172.21.15.22:0/3518460910 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7ff6fc016030 con 0x7ff704102f90 2024-03-28T18:06:25.904 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:25.904+0000 7ff709400640 1 -- 172.21.15.22:0/3518460910 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7ff7041a2940 con 0x7ff704102f90 2024-03-28T18:06:25.905 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:25.905+0000 7ff700c00640 1 -- 172.21.15.22:0/3518460910 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v92) v1 ==== 104+0+11 (secure 0 0 0) 0x7ff6fc09c0f0 con 0x7ff704102f90 2024-03-28T18:06:25.905 INFO:teuthology.orchestra.run.smithi022.stdout:pg_num: 32 2024-03-28T18:06:25.908 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:25.908+0000 7ff709400640 1 -- 172.21.15.22:0/3518460910 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7ff6e40a2880 msgr2=0x7ff6e40a4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:06:25.908 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:25.908+0000 7ff709400640 1 --2- 172.21.15.22:0/3518460910 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7ff6e40a2880 0x7ff6e40a4d40 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7ff6f402c240 tx=0x7ff6f402c820 comp rx=0 tx=0).stop 2024-03-28T18:06:25.908 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:25.908+0000 7ff709400640 1 -- 172.21.15.22:0/3518460910 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff704102f90 msgr2=0x7ff704103440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:06:25.908 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:25.908+0000 7ff709400640 1 --2- 172.21.15.22:0/3518460910 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff704102f90 0x7ff704103440 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7ff6fc00eaa0 tx=0x7ff6fc00ef60 comp rx=0 tx=0).stop 2024-03-28T18:06:25.908 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:25.908+0000 7ff709400640 1 -- 172.21.15.22:0/3518460910 shutdown_connections 2024-03-28T18:06:25.908 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:25.908+0000 7ff709400640 1 --2- 172.21.15.22:0/3518460910 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7ff6e40a2880 0x7ff6e40a4d40 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:06:25.908 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:25.908+0000 7ff709400640 1 --2- 172.21.15.22:0/3518460910 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff704102f90 0x7ff704103440 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:06:25.908 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:25.908+0000 7ff709400640 1 --2- 172.21.15.22:0/3518460910 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff704102030 0x7ff704102a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:06:25.908 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:25.908+0000 7ff709400640 1 --2- 172.21.15.22:0/3518460910 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ff704101670 0x7ff704104410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:06:25.908 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:25.908+0000 7ff709400640 1 -- 172.21.15.22:0/3518460910 >> 172.21.15.22:0/3518460910 conn(0x7ff7040fba10 msgr2=0x7ff7040fd530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:06:25.909 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:25.908+0000 7ff709400640 1 -- 172.21.15.22:0/3518460910 shutdown_connections 2024-03-28T18:06:25.909 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:25.909+0000 7ff709400640 1 -- 172.21.15.22:0/3518460910 wait complete. 2024-03-28T18:06:25.928 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-28T18:06:26.160 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:26.160+0000 7f3dc7200640 1 -- 172.21.15.22:0/1093163267 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3dc8109430 msgr2=0x7f3dc810b8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:06:26.160 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:26.160+0000 7f3dc7200640 1 --2- 172.21.15.22:0/1093163267 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3dc8109430 0x7f3dc810b8c0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f3dc0009ca0 tx=0x7f3dc002c180 comp rx=0 tx=0).stop 2024-03-28T18:06:26.160 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:26.160+0000 7f3dc7200640 1 -- 172.21.15.22:0/1093163267 shutdown_connections 2024-03-28T18:06:26.160 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:26.160+0000 7f3dc7200640 1 --2- 172.21.15.22:0/1093163267 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3dc8109430 0x7f3dc810b8c0 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:06:26.160 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:26.160+0000 7f3dc7200640 1 --2- 172.21.15.22:0/1093163267 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f3dc8074080 0x7f3dc80744e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:06:26.161 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:26.160+0000 7f3dc7200640 1 --2- 172.21.15.22:0/1093163267 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3dc8075660 0x7f3dc8073ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:06:26.161 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:26.160+0000 7f3dc7200640 1 -- 172.21.15.22:0/1093163267 >> 172.21.15.22:0/1093163267 conn(0x7f3dc80fcb10 msgr2=0x7f3dc80fef30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:06:26.161 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:26.160+0000 7f3dc7200640 1 -- 172.21.15.22:0/1093163267 shutdown_connections 2024-03-28T18:06:26.161 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:26.160+0000 7f3dc7200640 1 -- 172.21.15.22:0/1093163267 wait complete. 2024-03-28T18:06:26.163 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:26.161+0000 7f3dc7200640 1 Processor -- start 2024-03-28T18:06:26.163 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:26.161+0000 7f3dc7200640 1 -- start start 2024-03-28T18:06:26.163 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:26.162+0000 7f3dc7200640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3dc8074080 0x7f3dc8199a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:06:26.163 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:26.162+0000 7f3dc7200640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f3dc8075660 0x7f3dc8199f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:06:26.163 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:26.162+0000 7f3dc7200640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3dc8109430 0x7f3dc81a0fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:06:26.163 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:26.162+0000 7f3dc7200640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3dc810f3f0 con 0x7f3dc8074080 2024-03-28T18:06:26.163 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:26.162+0000 7f3dc7200640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3dc810f270 con 0x7f3dc8075660 2024-03-28T18:06:26.163 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:26.162+0000 7f3dc7200640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3dc810f570 con 0x7f3dc8109430 2024-03-28T18:06:26.164 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:26.162+0000 7f3dc5e00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3dc8074080 0x7f3dc8199a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:06:26.164 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:26.162+0000 7f3dc5e00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3dc8074080 0x7f3dc8199a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:49994/0 (socket says 172.21.15.22:49994) 2024-03-28T18:06:26.164 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:26.162+0000 7f3dc5e00640 1 -- 172.21.15.22:0/3903491373 learned_addr learned my addr 172.21.15.22:0/3903491373 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-28T18:06:26.164 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:26.162+0000 7f3dbfe00640 1 --2- 172.21.15.22:0/3903491373 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f3dc8075660 0x7f3dc8199f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:06:26.164 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:26.162+0000 7f3dc6800640 1 --2- 172.21.15.22:0/3903491373 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3dc8109430 0x7f3dc81a0fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:06:26.164 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:26.162+0000 7f3dc5e00640 1 -- 172.21.15.22:0/3903491373 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3dc8109430 msgr2=0x7f3dc81a0fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:06:26.164 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:26.162+0000 7f3dc5e00640 1 --2- 172.21.15.22:0/3903491373 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3dc8109430 0x7f3dc81a0fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:06:26.164 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:26.162+0000 7f3dc5e00640 1 -- 172.21.15.22:0/3903491373 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f3dc8075660 msgr2=0x7f3dc8199f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:06:26.164 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:26.162+0000 7f3dc5e00640 1 --2- 172.21.15.22:0/3903491373 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f3dc8075660 0x7f3dc8199f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:06:26.164 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:26.162+0000 7f3dc5e00640 1 -- 172.21.15.22:0/3903491373 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3dc81a1650 con 0x7f3dc8074080 2024-03-28T18:06:26.164 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:26.163+0000 7f3dc5e00640 1 --2- 172.21.15.22:0/3903491373 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3dc8074080 0x7f3dc8199a30 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f3db800d770 tx=0x7f3db800dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:06:26.164 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:26.163+0000 7f3dbea00640 1 -- 172.21.15.22:0/3903491373 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3db8017070 con 0x7f3dc8074080 2024-03-28T18:06:26.164 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:26.163+0000 7f3dbea00640 1 -- 172.21.15.22:0/3903491373 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f3db80054f0 con 0x7f3dc8074080 2024-03-28T18:06:26.164 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:26.163+0000 7f3dc7200640 1 -- 172.21.15.22:0/3903491373 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3dc81a1940 con 0x7f3dc8074080 2024-03-28T18:06:26.165 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:26.163+0000 7f3dbea00640 1 -- 172.21.15.22:0/3903491373 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3db8004d60 con 0x7f3dc8074080 2024-03-28T18:06:26.165 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:26.163+0000 7f3dc7200640 1 -- 172.21.15.22:0/3903491373 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3dc81a1e40 con 0x7f3dc8074080 2024-03-28T18:06:26.165 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:26.164+0000 7f3dc7200640 1 -- 172.21.15.22:0/3903491373 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3dc81081f0 con 0x7f3dc8074080 2024-03-28T18:06:26.166 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:26.166+0000 7f3dbea00640 1 -- 172.21.15.22:0/3903491373 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 18) v1 ==== 137290+0+0 (secure 0 0 0) 0x7f3db8004f00 con 0x7f3dc8074080 2024-03-28T18:06:26.167 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:26.167+0000 7f3dbea00640 1 --2- 172.21.15.22:0/3903491373 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7f3dac0a2800 0x7f3dac0a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:06:26.167 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:26.168+0000 7f3dbea00640 1 -- 172.21.15.22:0/3903491373 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 8153+0+0 (secure 0 0 0) 0x7f3db80ce960 con 0x7f3dc8074080 2024-03-28T18:06:26.167 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:26.168+0000 7f3dbfe00640 1 --2- 172.21.15.22:0/3903491373 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7f3dac0a2800 0x7f3dac0a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:06:26.168 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:26.169+0000 7f3dbfe00640 1 --2- 172.21.15.22:0/3903491373 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7f3dac0a2800 0x7f3dac0a4cc0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f3dc819af40 tx=0x7f3db0005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:06:26.169 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:26.170+0000 7f3dbea00640 1 -- 172.21.15.22:0/3903491373 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f3db809ba00 con 0x7f3dc8074080 2024-03-28T18:06:26.281 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:06:26 smithi022 ceph-mon[31985]: pgmap v419: 97 pgs: 97 active+clean; 643 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 341 B/s wr, 9 op/s 2024-03-28T18:06:26.281 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:06:26 smithi022 ceph-mon[31985]: from='client.? 172.21.15.22:0/3699539692' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-28T18:06:26.282 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:06:26 smithi022 ceph-mon[31985]: from='client.? 172.21.15.22:0/3518460910' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-28T18:06:26.373 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:26.373+0000 7f3dc7200640 1 -- 172.21.15.22:0/3903491373 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f3dc802d070 con 0x7f3dc8074080 2024-03-28T18:06:26.374 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:26.374+0000 7f3dbea00640 1 -- 172.21.15.22:0/3903491373 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v92) v1 ==== 100+0+11 (secure 0 0 0) 0x7f3db809b300 con 0x7f3dc8074080 2024-03-28T18:06:26.374 INFO:teuthology.orchestra.run.smithi022.stdout:pg_num: 64 2024-03-28T18:06:26.377 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:26.377+0000 7f3dc7200640 1 -- 172.21.15.22:0/3903491373 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7f3dac0a2800 msgr2=0x7f3dac0a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:06:26.377 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:26.377+0000 7f3dc7200640 1 --2- 172.21.15.22:0/3903491373 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7f3dac0a2800 0x7f3dac0a4cc0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f3dc819af40 tx=0x7f3db0005f90 comp rx=0 tx=0).stop 2024-03-28T18:06:26.377 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:26.377+0000 7f3dc7200640 1 -- 172.21.15.22:0/3903491373 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3dc8074080 msgr2=0x7f3dc8199a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:06:26.377 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:26.377+0000 7f3dc7200640 1 --2- 172.21.15.22:0/3903491373 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3dc8074080 0x7f3dc8199a30 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f3db800d770 tx=0x7f3db800dc30 comp rx=0 tx=0).stop 2024-03-28T18:06:26.377 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:26.377+0000 7f3dc7200640 1 -- 172.21.15.22:0/3903491373 shutdown_connections 2024-03-28T18:06:26.377 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:26.377+0000 7f3dc7200640 1 --2- 172.21.15.22:0/3903491373 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7f3dac0a2800 0x7f3dac0a4cc0 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:06:26.377 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:26.377+0000 7f3dc7200640 1 --2- 172.21.15.22:0/3903491373 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3dc8109430 0x7f3dc81a0fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:06:26.377 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:26.377+0000 7f3dc7200640 1 --2- 172.21.15.22:0/3903491373 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f3dc8075660 0x7f3dc8199f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:06:26.377 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:26.377+0000 7f3dc7200640 1 --2- 172.21.15.22:0/3903491373 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3dc8074080 0x7f3dc8199a30 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:06:26.377 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:26.377+0000 7f3dc7200640 1 -- 172.21.15.22:0/3903491373 >> 172.21.15.22:0/3903491373 conn(0x7f3dc80fcb10 msgr2=0x7f3dc81053c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:06:26.378 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:26.377+0000 7f3dc7200640 1 -- 172.21.15.22:0/3903491373 shutdown_connections 2024-03-28T18:06:26.378 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:26.377+0000 7f3dc7200640 1 -- 172.21.15.22:0/3903491373 wait complete. 2024-03-28T18:06:26.396 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-28T18:06:26.424 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:06:26 smithi134 ceph-mon[35037]: pgmap v419: 97 pgs: 97 active+clean; 643 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 341 B/s wr, 9 op/s 2024-03-28T18:06:26.424 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:06:26 smithi134 ceph-mon[35037]: from='client.? 172.21.15.22:0/3699539692' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-28T18:06:26.424 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:06:26 smithi134 ceph-mon[35037]: from='client.? 172.21.15.22:0/3518460910' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-28T18:06:26.432 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:06:26 smithi079 ceph-mon[35312]: pgmap v419: 97 pgs: 97 active+clean; 643 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 341 B/s wr, 9 op/s 2024-03-28T18:06:26.432 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:06:26 smithi079 ceph-mon[35312]: from='client.? 172.21.15.22:0/3699539692' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-28T18:06:26.432 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:06:26 smithi079 ceph-mon[35312]: from='client.? 172.21.15.22:0/3518460910' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-28T18:06:26.622 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:26.622+0000 7fab5e600640 1 -- 172.21.15.22:0/3374120667 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fab580740a0 msgr2=0x7fab58074500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:06:26.622 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:26.622+0000 7fab5e600640 1 --2- 172.21.15.22:0/3374120667 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fab580740a0 0x7fab58074500 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7fab40009900 tx=0x7fab4002bcd0 comp rx=0 tx=0).stop 2024-03-28T18:06:26.622 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:26.622+0000 7fab5e600640 1 -- 172.21.15.22:0/3374120667 shutdown_connections 2024-03-28T18:06:26.622 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:26.622+0000 7fab5e600640 1 --2- 172.21.15.22:0/3374120667 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fab58109540 0x7fab5810b9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:06:26.623 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:26.622+0000 7fab5e600640 1 --2- 172.21.15.22:0/3374120667 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fab580740a0 0x7fab58074500 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:06:26.623 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:26.622+0000 7fab5e600640 1 --2- 172.21.15.22:0/3374120667 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fab58075680 0x7fab58073ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:06:26.623 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:26.622+0000 7fab5e600640 1 -- 172.21.15.22:0/3374120667 >> 172.21.15.22:0/3374120667 conn(0x7fab580fcc80 msgr2=0x7fab580ff0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:06:26.623 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:26.622+0000 7fab5e600640 1 -- 172.21.15.22:0/3374120667 shutdown_connections 2024-03-28T18:06:26.623 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:26.623+0000 7fab5e600640 1 -- 172.21.15.22:0/3374120667 wait complete. 2024-03-28T18:06:26.624 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:26.623+0000 7fab5e600640 1 Processor -- start 2024-03-28T18:06:26.624 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:26.624+0000 7fab5e600640 1 -- start start 2024-03-28T18:06:26.625 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:26.624+0000 7fab5e600640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fab580740a0 0x7fab58199ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:06:26.625 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:26.624+0000 7fab5e600640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fab58075680 0x7fab5819a0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:06:26.625 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:26.624+0000 7fab5e600640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fab58109540 0x7fab581a1150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:06:26.625 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:26.624+0000 7fab5e600640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab5810f560 con 0x7fab58109540 2024-03-28T18:06:26.625 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:26.624+0000 7fab5e600640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab5810f3e0 con 0x7fab58075680 2024-03-28T18:06:26.625 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:26.624+0000 7fab5e600640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab5810f6e0 con 0x7fab580740a0 2024-03-28T18:06:26.625 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:26.624+0000 7fab57e00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fab58109540 0x7fab581a1150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:06:26.625 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:26.624+0000 7fab57e00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fab58109540 0x7fab581a1150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:50026/0 (socket says 172.21.15.22:50026) 2024-03-28T18:06:26.625 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:26.624+0000 7fab57e00640 1 -- 172.21.15.22:0/2436774832 learned_addr learned my addr 172.21.15.22:0/2436774832 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-28T18:06:26.625 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:26.624+0000 7fab4ea00640 1 --2- 172.21.15.22:0/2436774832 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fab58075680 0x7fab5819a0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:06:26.625 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:26.625+0000 7fab57400640 1 --2- 172.21.15.22:0/2436774832 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fab580740a0 0x7fab58199ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:06:26.625 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:26.625+0000 7fab57e00640 1 -- 172.21.15.22:0/2436774832 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fab580740a0 msgr2=0x7fab58199ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:06:26.625 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:26.625+0000 7fab57e00640 1 --2- 172.21.15.22:0/2436774832 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fab580740a0 0x7fab58199ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:06:26.625 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:26.625+0000 7fab57e00640 1 -- 172.21.15.22:0/2436774832 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fab58075680 msgr2=0x7fab5819a0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:06:26.625 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:26.625+0000 7fab57e00640 1 --2- 172.21.15.22:0/2436774832 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fab58075680 0x7fab5819a0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:06:26.626 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:26.625+0000 7fab57e00640 1 -- 172.21.15.22:0/2436774832 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab581a17c0 con 0x7fab58109540 2024-03-28T18:06:26.627 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:26.625+0000 7fab57e00640 1 --2- 172.21.15.22:0/2436774832 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fab58109540 0x7fab581a1150 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7fab5000d750 tx=0x7fab5000dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:06:26.627 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:26.625+0000 7fab54c00640 1 -- 172.21.15.22:0/2436774832 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab50017070 con 0x7fab58109540 2024-03-28T18:06:26.627 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:26.626+0000 7fab54c00640 1 -- 172.21.15.22:0/2436774832 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7fab500054f0 con 0x7fab58109540 2024-03-28T18:06:26.627 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:26.626+0000 7fab5e600640 1 -- 172.21.15.22:0/2436774832 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fab581a1ab0 con 0x7fab58109540 2024-03-28T18:06:26.627 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:26.626+0000 7fab54c00640 1 -- 172.21.15.22:0/2436774832 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab50004cf0 con 0x7fab58109540 2024-03-28T18:06:26.627 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:26.626+0000 7fab5e600640 1 -- 172.21.15.22:0/2436774832 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fab581a1fd0 con 0x7fab58109540 2024-03-28T18:06:26.627 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:26.627+0000 7fab5e600640 1 -- 172.21.15.22:0/2436774832 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fab58108300 con 0x7fab58109540 2024-03-28T18:06:26.628 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:26.629+0000 7fab54c00640 1 -- 172.21.15.22:0/2436774832 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 18) v1 ==== 137290+0+0 (secure 0 0 0) 0x7fab50004e90 con 0x7fab58109540 2024-03-28T18:06:26.630 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:26.630+0000 7fab54c00640 1 --2- 172.21.15.22:0/2436774832 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7fab3c0a28d0 0x7fab3c0a4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:06:26.630 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:26.630+0000 7fab54c00640 1 -- 172.21.15.22:0/2436774832 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 8153+0+0 (secure 0 0 0) 0x7fab500ce310 con 0x7fab58109540 2024-03-28T18:06:26.630 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:26.630+0000 7fab57400640 1 --2- 172.21.15.22:0/2436774832 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7fab3c0a28d0 0x7fab3c0a4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:06:26.630 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:26.631+0000 7fab57400640 1 --2- 172.21.15.22:0/2436774832 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7fab3c0a28d0 0x7fab3c0a4d90 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fab48005fd0 tx=0x7fab48009870 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:06:26.634 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:26.635+0000 7fab54c00640 1 -- 172.21.15.22:0/2436774832 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fab5009b450 con 0x7fab58109540 2024-03-28T18:06:26.880 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:26.880+0000 7fab5e600640 1 -- 172.21.15.22:0/2436774832 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fab580622a0 con 0x7fab58109540 2024-03-28T18:06:26.881 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:26.882+0000 7fab54c00640 1 -- 172.21.15.22:0/2436774832 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) v1 ==== 94+0+11452 (secure 0 0 0) 0x7fab5009ad50 con 0x7fab58109540 2024-03-28T18:06:26.882 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-03-28T18:06:26.882 INFO:teuthology.orchestra.run.smithi022.stdout:{"epoch":26,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14544,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6835/3718238950","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6834","nonce":3718238950},{"type":"v1","addr":"172.21.15.22:6835","nonce":3718238950}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14562,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6837/3676939386","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6836","nonce":3676939386},{"type":"v1","addr":"172.21.15.22:6837","nonce":3676939386}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":26,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-28T18:04:48.991622+0000","modified":"2024-03-28T18:04:56.643864+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":34553,"mds_1":34511,"mds_2":34475,"mds_3":34442,"mds_4":24502},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14640":{"gid":14640,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.22:6839/3979998310","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6838","nonce":3979998310},{"type":"v1","addr":"172.21.15.22:6839","nonce":3979998310}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_14655":{"gid":14655,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.22:6841/3069056661","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6840","nonce":3069056661},{"type":"v1","addr":"172.21.15.22:6841","nonce":3069056661}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24412":{"gid":24412,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.134:6833/3176125362","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6832","nonce":3176125362},{"type":"v1","addr":"172.21.15.134:6833","nonce":3176125362}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24463":{"gid":24463,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.134:6835/3370549207","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6834","nonce":3370549207},{"type":"v1","addr":"172.21.15.134:6835","nonce":3370549207}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24478":{"gid":24478,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.134:6837/899825581","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6836","nonce":899825581},{"type":"v1","addr":"172.21.15.134:6837","nonce":899825581}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24502":{"gid":24502,"name":"i","rank":4,"incarnation":24,"state":"up:active","state_seq":7,"addr":"172.21.15.134:6839/3400263217","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6838","nonce":3400263217},{"type":"v1","addr":"172.21.15.134:6839","nonce":3400263217}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_34442":{"gid":34442,"name":"k","rank":3,"incarnation":22,"state":"up:active","state_seq":14,"addr":"172.21.15.79:6833/856896997","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6832","nonce":856896997},{"type":"v1","addr":"172.21.15.79:6833","nonce":856896997}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_34475":{"gid":34475,"name":"e","rank":2,"incarnation":20,"state":"up:active","state_seq":11,"addr":"172.21.15.79:6835/3365773431","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6834","nonce":3365773431},{"type":"v1","addr":"172.21.15.79:6835","nonce":3365773431}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_34511":{"gid":34511,"name":"h","rank":1,"incarnation":18,"state":"up:active","state_seq":8,"addr":"172.21.15.79:6837/1787903366","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6836","nonce":1787903366},{"type":"v1","addr":"172.21.15.79:6837","nonce":1787903366}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_34553":{"gid":34553,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.79:6839/2441542443","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6838","nonce":2441542443},{"type":"v1","addr":"172.21.15.79:6839","nonce":2441542443}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":34553,"qdb_cluster":[34511,34553,34475,34442,24502]},"id":1}]} 2024-03-28T18:06:26.882 INFO:teuthology.orchestra.run.smithi022.stderr:dumped fsmap epoch 26 2024-03-28T18:06:26.884 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:26.885+0000 7fab5e600640 1 -- 172.21.15.22:0/2436774832 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7fab3c0a28d0 msgr2=0x7fab3c0a4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:06:26.885 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:26.885+0000 7fab5e600640 1 --2- 172.21.15.22:0/2436774832 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7fab3c0a28d0 0x7fab3c0a4d90 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fab48005fd0 tx=0x7fab48009870 comp rx=0 tx=0).stop 2024-03-28T18:06:26.885 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:26.885+0000 7fab5e600640 1 -- 172.21.15.22:0/2436774832 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fab58109540 msgr2=0x7fab581a1150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:06:26.885 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:26.885+0000 7fab5e600640 1 --2- 172.21.15.22:0/2436774832 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fab58109540 0x7fab581a1150 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7fab5000d750 tx=0x7fab5000dc10 comp rx=0 tx=0).stop 2024-03-28T18:06:26.885 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:26.885+0000 7fab5e600640 1 -- 172.21.15.22:0/2436774832 shutdown_connections 2024-03-28T18:06:26.885 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:26.885+0000 7fab5e600640 1 --2- 172.21.15.22:0/2436774832 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7fab3c0a28d0 0x7fab3c0a4d90 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:06:26.885 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:26.885+0000 7fab5e600640 1 --2- 172.21.15.22:0/2436774832 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fab58109540 0x7fab581a1150 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:06:26.885 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:26.885+0000 7fab5e600640 1 --2- 172.21.15.22:0/2436774832 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fab58075680 0x7fab5819a0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:06:26.885 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:26.885+0000 7fab5e600640 1 --2- 172.21.15.22:0/2436774832 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fab580740a0 0x7fab58199ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:06:26.885 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:26.885+0000 7fab5e600640 1 -- 172.21.15.22:0/2436774832 >> 172.21.15.22:0/2436774832 conn(0x7fab580fcc80 msgr2=0x7fab581054d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:06:26.885 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:26.885+0000 7fab5e600640 1 -- 172.21.15.22:0/2436774832 shutdown_connections 2024-03-28T18:06:26.885 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:26.885+0000 7fab5e600640 1 -- 172.21.15.22:0/2436774832 wait complete. 2024-03-28T18:06:26.904 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-28T18:06:27.131 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:27.131+0000 7ff36d800640 1 -- 172.21.15.22:0/570670950 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff3681052a0 msgr2=0x7ff36810bb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:06:27.131 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:27.131+0000 7ff36d800640 1 --2- 172.21.15.22:0/570670950 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff3681052a0 0x7ff36810bb30 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7ff360009900 tx=0x7ff36002bc30 comp rx=0 tx=0).stop 2024-03-28T18:06:27.131 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:27.131+0000 7ff36d800640 1 -- 172.21.15.22:0/570670950 shutdown_connections 2024-03-28T18:06:27.131 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:27.131+0000 7ff36d800640 1 --2- 172.21.15.22:0/570670950 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff3681052a0 0x7ff36810bb30 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:06:27.132 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:27.131+0000 7ff36d800640 1 --2- 172.21.15.22:0/570670950 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ff3681048e0 0x7ff368104d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:06:27.132 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:27.131+0000 7ff36d800640 1 --2- 172.21.15.22:0/570670950 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff3681036f0 0x7ff368103b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:06:27.132 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:27.131+0000 7ff36d800640 1 -- 172.21.15.22:0/570670950 >> 172.21.15.22:0/570670950 conn(0x7ff3680fec80 msgr2=0x7ff3681010a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:06:27.132 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:27.131+0000 7ff36d800640 1 -- 172.21.15.22:0/570670950 shutdown_connections 2024-03-28T18:06:27.132 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:27.131+0000 7ff36d800640 1 -- 172.21.15.22:0/570670950 wait complete. 2024-03-28T18:06:27.133 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:27.132+0000 7ff36d800640 1 Processor -- start 2024-03-28T18:06:27.133 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:27.133+0000 7ff36d800640 1 -- start start 2024-03-28T18:06:27.133 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:27.133+0000 7ff36d800640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff3681036f0 0x7ff368199960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:06:27.133 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:27.133+0000 7ff36d800640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ff3681048e0 0x7ff368199ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:06:27.134 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:27.133+0000 7ff36d800640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff3681052a0 0x7ff3681a0f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:06:27.134 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:27.133+0000 7ff36d800640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff36810f3a0 con 0x7ff3681052a0 2024-03-28T18:06:27.134 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:27.133+0000 7ff36d800640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff36810f220 con 0x7ff3681048e0 2024-03-28T18:06:27.134 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:27.133+0000 7ff36d800640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff36810f520 con 0x7ff3681036f0 2024-03-28T18:06:27.134 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:27.133+0000 7ff36ce00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff3681052a0 0x7ff3681a0f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:06:27.134 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:27.133+0000 7ff36ce00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff3681052a0 0x7ff3681a0f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:48974/0 (socket says 172.21.15.22:48974) 2024-03-28T18:06:27.134 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:27.133+0000 7ff36ce00640 1 -- 172.21.15.22:0/3716085499 learned_addr learned my addr 172.21.15.22:0/3716085499 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-28T18:06:27.134 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:27.133+0000 7ff367400640 1 --2- 172.21.15.22:0/3716085499 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ff3681048e0 0x7ff368199ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:06:27.134 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:27.133+0000 7ff367e00640 1 --2- 172.21.15.22:0/3716085499 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff3681036f0 0x7ff368199960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:06:27.134 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:27.134+0000 7ff36ce00640 1 -- 172.21.15.22:0/3716085499 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff3681036f0 msgr2=0x7ff368199960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:06:27.134 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:27.134+0000 7ff36ce00640 1 --2- 172.21.15.22:0/3716085499 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff3681036f0 0x7ff368199960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:06:27.134 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:27.134+0000 7ff36ce00640 1 -- 172.21.15.22:0/3716085499 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ff3681048e0 msgr2=0x7ff368199ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:06:27.134 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:27.134+0000 7ff36ce00640 1 --2- 172.21.15.22:0/3716085499 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ff3681048e0 0x7ff368199ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:06:27.137 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:27.134+0000 7ff36ce00640 1 -- 172.21.15.22:0/3716085499 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff3681a1580 con 0x7ff3681052a0 2024-03-28T18:06:27.137 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:27.134+0000 7ff367400640 1 --2- 172.21.15.22:0/3716085499 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ff3681048e0 0x7ff368199ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-28T18:06:27.138 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:27.134+0000 7ff36ce00640 1 --2- 172.21.15.22:0/3716085499 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff3681052a0 0x7ff3681a0f10 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7ff360036040 tx=0x7ff360005de0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:06:27.138 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:27.134+0000 7ff364c00640 1 -- 172.21.15.22:0/3716085499 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff360043020 con 0x7ff3681052a0 2024-03-28T18:06:27.138 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:27.134+0000 7ff364c00640 1 -- 172.21.15.22:0/3716085499 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7ff3600053f0 con 0x7ff3681052a0 2024-03-28T18:06:27.138 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:27.134+0000 7ff36d800640 1 -- 172.21.15.22:0/3716085499 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff3681a1810 con 0x7ff3681052a0 2024-03-28T18:06:27.139 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:27.134+0000 7ff364c00640 1 -- 172.21.15.22:0/3716085499 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff360003c50 con 0x7ff3681052a0 2024-03-28T18:06:27.139 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:27.135+0000 7ff36d800640 1 -- 172.21.15.22:0/3716085499 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff3681a1db0 con 0x7ff3681052a0 2024-03-28T18:06:27.139 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:27.136+0000 7ff36d800640 1 -- 172.21.15.22:0/3716085499 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff36810a9b0 con 0x7ff3681052a0 2024-03-28T18:06:27.139 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:27.139+0000 7ff364c00640 1 -- 172.21.15.22:0/3716085499 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 18) v1 ==== 137290+0+0 (secure 0 0 0) 0x7ff360046020 con 0x7ff3681052a0 2024-03-28T18:06:27.140 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:27.141+0000 7ff364c00640 1 --2- 172.21.15.22:0/3716085499 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7ff34c0a2880 0x7ff34c0a4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:06:27.140 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:27.141+0000 7ff364c00640 1 -- 172.21.15.22:0/3716085499 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 8153+0+0 (secure 0 0 0) 0x7ff3600f10b0 con 0x7ff3681052a0 2024-03-28T18:06:27.140 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:27.141+0000 7ff367e00640 1 --2- 172.21.15.22:0/3716085499 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7ff34c0a2880 0x7ff34c0a4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:06:27.141 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:27.142+0000 7ff367e00640 1 --2- 172.21.15.22:0/3716085499 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7ff34c0a2880 0x7ff34c0a4d40 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7ff350005fd0 tx=0x7ff350005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:06:27.143 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:27.143+0000 7ff364c00640 1 -- 172.21.15.22:0/3716085499 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7ff36004d020 con 0x7ff3681052a0 2024-03-28T18:06:27.282 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:06:27 smithi022 ceph-mon[31985]: from='client.? 172.21.15.22:0/3903491373' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-28T18:06:27.282 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:06:27 smithi022 ceph-mon[31985]: from='client.? 172.21.15.22:0/2436774832' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-28T18:06:27.347 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:27.347+0000 7ff36d800640 1 -- 172.21.15.22:0/3716085499 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ff3680622a0 con 0x7ff3681052a0 2024-03-28T18:06:27.351 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:27.352+0000 7ff364c00640 1 -- 172.21.15.22:0/3716085499 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v92) v1 ==== 74+0+22522 (secure 0 0 0) 0x7ff3600055a0 con 0x7ff3681052a0 2024-03-28T18:06:27.352 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-03-28T18:06:27.352 INFO:teuthology.orchestra.run.smithi022.stdout:{"epoch":92,"fsid":"fd220748-ed2b-11ee-95d0-87774f69a715","created":"2024-03-28T17:54:00.440279+0000","modified":"2024-03-28T18:05:11.725341+0000","last_up_change":"2024-03-28T18:03:43.220095+0000","last_in_change":"2024-03-28T18:03:18.526399+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-28T17:58:33.168872+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-28T18:04:47.004797+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-28T18:04:48.465502+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"90","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"5d708e06-f05b-40ba-bbcf-258ddc4f43d7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6802","nonce":2186290846},{"type":"v1","addr":"172.21.15.22:6803","nonce":2186290846}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6804","nonce":2186290846},{"type":"v1","addr":"172.21.15.22:6805","nonce":2186290846}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6808","nonce":2186290846},{"type":"v1","addr":"172.21.15.22:6809","nonce":2186290846}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6806","nonce":2186290846},{"type":"v1","addr":"172.21.15.22:6807","nonce":2186290846}]},"public_addr":"172.21.15.22:6803/2186290846","cluster_addr":"172.21.15.22:6805/2186290846","heartbeat_back_addr":"172.21.15.22:6809/2186290846","heartbeat_front_addr":"172.21.15.22:6807/2186290846","state":["exists","up"]},{"osd":1,"uuid":"68ad09bb-9256-409a-9867-4e14a2cb94aa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6800","nonce":2835549681},{"type":"v1","addr":"172.21.15.79:6801","nonce":2835549681}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6802","nonce":2835549681},{"type":"v1","addr":"172.21.15.79:6803","nonce":2835549681}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6806","nonce":2835549681},{"type":"v1","addr":"172.21.15.79:6807","nonce":2835549681}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6804","nonce":2835549681},{"type":"v1","addr":"172.21.15.79:6805","nonce":2835549681}]},"public_addr":"172.21.15.79:6801/2835549681","cluster_addr":"172.21.15.79:6803/2835549681","heartbeat_back_addr":"172.21.15.79:6807/2835549681","heartbeat_front_addr":"172.21.15.79:6805/2835549681","state":["exists","up"]},{"osd":2,"uuid":"ed11a5a7-004d-41f3-a5fc-eb2d603bd33f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6800","nonce":470588085},{"type":"v1","addr":"172.21.15.134:6801","nonce":470588085}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6802","nonce":470588085},{"type":"v1","addr":"172.21.15.134:6803","nonce":470588085}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6806","nonce":470588085},{"type":"v1","addr":"172.21.15.134:6807","nonce":470588085}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6804","nonce":470588085},{"type":"v1","addr":"172.21.15.134:6805","nonce":470588085}]},"public_addr":"172.21.15.134:6801/470588085","cluster_addr":"172.21.15.134:6803/470588085","heartbeat_back_addr":"172.21.15.134:6807/470588085","heartbeat_front_addr":"172.21.15.134:6805/470588085","state":["exists","up"]},{"osd":3,"uuid":"81a6c547-2a50-44c3-84d7-23ab6831790c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6810","nonce":1702441854},{"type":"v1","addr":"172.21.15.22:6811","nonce":1702441854}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6812","nonce":1702441854},{"type":"v1","addr":"172.21.15.22:6813","nonce":1702441854}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6816","nonce":1702441854},{"type":"v1","addr":"172.21.15.22:6817","nonce":1702441854}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6814","nonce":1702441854},{"type":"v1","addr":"172.21.15.22:6815","nonce":1702441854}]},"public_addr":"172.21.15.22:6811/1702441854","cluster_addr":"172.21.15.22:6813/1702441854","heartbeat_back_addr":"172.21.15.22:6817/1702441854","heartbeat_front_addr":"172.21.15.22:6815/1702441854","state":["exists","up"]},{"osd":4,"uuid":"d23a254e-7182-4471-8bca-7eb684ec618e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6808","nonce":630132437},{"type":"v1","addr":"172.21.15.79:6809","nonce":630132437}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6810","nonce":630132437},{"type":"v1","addr":"172.21.15.79:6811","nonce":630132437}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6814","nonce":630132437},{"type":"v1","addr":"172.21.15.79:6815","nonce":630132437}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6812","nonce":630132437},{"type":"v1","addr":"172.21.15.79:6813","nonce":630132437}]},"public_addr":"172.21.15.79:6809/630132437","cluster_addr":"172.21.15.79:6811/630132437","heartbeat_back_addr":"172.21.15.79:6815/630132437","heartbeat_front_addr":"172.21.15.79:6813/630132437","state":["exists","up"]},{"osd":5,"uuid":"a7b98502-2023-4f0a-bfa0-6840cc495e0c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":40,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6808","nonce":2135406915},{"type":"v1","addr":"172.21.15.134:6809","nonce":2135406915}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6810","nonce":2135406915},{"type":"v1","addr":"172.21.15.134:6811","nonce":2135406915}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6814","nonce":2135406915},{"type":"v1","addr":"172.21.15.134:6815","nonce":2135406915}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6812","nonce":2135406915},{"type":"v1","addr":"172.21.15.134:6813","nonce":2135406915}]},"public_addr":"172.21.15.134:6809/2135406915","cluster_addr":"172.21.15.134:6811/2135406915","heartbeat_back_addr":"172.21.15.134:6815/2135406915","heartbeat_front_addr":"172.21.15.134:6813/2135406915","state":["exists","up"]},{"osd":6,"uuid":"634c6562-d683-41c8-8242-fc848c38ce1f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6818","nonce":1400680396},{"type":"v1","addr":"172.21.15.22:6819","nonce":1400680396}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6820","nonce":1400680396},{"type":"v1","addr":"172.21.15.22:6821","nonce":1400680396}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6824","nonce":1400680396},{"type":"v1","addr":"172.21.15.22:6825","nonce":1400680396}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6822","nonce":1400680396},{"type":"v1","addr":"172.21.15.22:6823","nonce":1400680396}]},"public_addr":"172.21.15.22:6819/1400680396","cluster_addr":"172.21.15.22:6821/1400680396","heartbeat_back_addr":"172.21.15.22:6825/1400680396","heartbeat_front_addr":"172.21.15.22:6823/1400680396","state":["exists","up"]},{"osd":7,"uuid":"504eb4a8-6731-45ad-932c-1b38ef157871","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":53,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6816","nonce":1287105314},{"type":"v1","addr":"172.21.15.79:6817","nonce":1287105314}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6818","nonce":1287105314},{"type":"v1","addr":"172.21.15.79:6819","nonce":1287105314}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6822","nonce":1287105314},{"type":"v1","addr":"172.21.15.79:6823","nonce":1287105314}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6820","nonce":1287105314},{"type":"v1","addr":"172.21.15.79:6821","nonce":1287105314}]},"public_addr":"172.21.15.79:6817/1287105314","cluster_addr":"172.21.15.79:6819/1287105314","heartbeat_back_addr":"172.21.15.79:6823/1287105314","heartbeat_front_addr":"172.21.15.79:6821/1287105314","state":["exists","up"]},{"osd":8,"uuid":"20aa66c1-03d6-4cb8-a096-0eae41979c17","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":62,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6816","nonce":2265650388},{"type":"v1","addr":"172.21.15.134:6817","nonce":2265650388}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6818","nonce":2265650388},{"type":"v1","addr":"172.21.15.134:6819","nonce":2265650388}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6822","nonce":2265650388},{"type":"v1","addr":"172.21.15.134:6823","nonce":2265650388}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6820","nonce":2265650388},{"type":"v1","addr":"172.21.15.134:6821","nonce":2265650388}]},"public_addr":"172.21.15.134:6817/2265650388","cluster_addr":"172.21.15.134:6819/2265650388","heartbeat_back_addr":"172.21.15.134:6823/2265650388","heartbeat_front_addr":"172.21.15.134:6821/2265650388","state":["exists","up"]},{"osd":9,"uuid":"e1ae4020-7c2c-4334-a753-ee6b12d98584","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":70,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6826","nonce":3762222259},{"type":"v1","addr":"172.21.15.22:6827","nonce":3762222259}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6828","nonce":3762222259},{"type":"v1","addr":"172.21.15.22:6829","nonce":3762222259}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6832","nonce":3762222259},{"type":"v1","addr":"172.21.15.22:6833","nonce":3762222259}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6830","nonce":3762222259},{"type":"v1","addr":"172.21.15.22:6831","nonce":3762222259}]},"public_addr":"172.21.15.22:6827/3762222259","cluster_addr":"172.21.15.22:6829/3762222259","heartbeat_back_addr":"172.21.15.22:6833/3762222259","heartbeat_front_addr":"172.21.15.22:6831/3762222259","state":["exists","up"]},{"osd":10,"uuid":"f5198d52-df48-4679-a860-4dcb440b7a9f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":77,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6824","nonce":1746167904},{"type":"v1","addr":"172.21.15.79:6825","nonce":1746167904}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6826","nonce":1746167904},{"type":"v1","addr":"172.21.15.79:6827","nonce":1746167904}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6830","nonce":1746167904},{"type":"v1","addr":"172.21.15.79:6831","nonce":1746167904}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6828","nonce":1746167904},{"type":"v1","addr":"172.21.15.79:6829","nonce":1746167904}]},"public_addr":"172.21.15.79:6825/1746167904","cluster_addr":"172.21.15.79:6827/1746167904","heartbeat_back_addr":"172.21.15.79:6831/1746167904","heartbeat_front_addr":"172.21.15.79:6829/1746167904","state":["exists","up"]},{"osd":11,"uuid":"1427bca7-3414-4468-9246-e999156bc2ae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":85,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6824","nonce":4271716428},{"type":"v1","addr":"172.21.15.134:6825","nonce":4271716428}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6826","nonce":4271716428},{"type":"v1","addr":"172.21.15.134:6827","nonce":4271716428}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6830","nonce":4271716428},{"type":"v1","addr":"172.21.15.134:6831","nonce":4271716428}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6828","nonce":4271716428},{"type":"v1","addr":"172.21.15.134:6829","nonce":4271716428}]},"public_addr":"172.21.15.134:6825/4271716428","cluster_addr":"172.21.15.134:6827/4271716428","heartbeat_back_addr":"172.21.15.134:6831/4271716428","heartbeat_front_addr":"172.21.15.134:6829/4271716428","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T17:57:21.225125+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T17:57:55.197254+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T17:58:27.825136+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T17:59:04.319893+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T17:59:38.407752+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T18:00:12.118805+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T18:00:47.635214+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T18:01:21.231055+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T18:01:55.817861+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T18:02:31.680160+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T18:03:06.783398+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T18:03:41.307347+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.8","mappings":[{"from":2,"to":11}]},{"pgid":"2.1d","mappings":[{"from":8,"to":11}]},{"pgid":"3.3","mappings":[{"from":4,"to":9}]},{"pgid":"3.c","mappings":[{"from":3,"to":9}]},{"pgid":"3.21","mappings":[{"from":4,"to":8}]},{"pgid":"3.25","mappings":[{"from":5,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.22:0/3521924283":"2024-03-29T17:55:10.425504+0000","172.21.15.22:6801/730635939":"2024-03-29T17:55:10.425504+0000","172.21.15.22:0/2471229459":"2024-03-29T17:54:34.004559+0000","172.21.15.22:6800/3166828081":"2024-03-29T17:54:34.004559+0000","172.21.15.22:6800/730635939":"2024-03-29T17:55:10.425504+0000","172.21.15.22:0/1229047849":"2024-03-29T17:54:34.004559+0000","172.21.15.22:6801/3166828081":"2024-03-29T17:54:34.004559+0000","172.21.15.22:0/2717818856":"2024-03-29T17:55:10.425504+0000","172.21.15.22:0/2157570333":"2024-03-29T17:54:34.004559+0000","172.21.15.22:0/1006589530":"2024-03-29T17:55:10.425504+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-28T18:06:27.354 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:27.354+0000 7ff36d800640 1 -- 172.21.15.22:0/3716085499 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7ff34c0a2880 msgr2=0x7ff34c0a4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:06:27.354 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:27.354+0000 7ff36d800640 1 --2- 172.21.15.22:0/3716085499 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7ff34c0a2880 0x7ff34c0a4d40 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7ff350005fd0 tx=0x7ff350005850 comp rx=0 tx=0).stop 2024-03-28T18:06:27.354 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:27.354+0000 7ff36d800640 1 -- 172.21.15.22:0/3716085499 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff3681052a0 msgr2=0x7ff3681a0f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:06:27.354 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:27.354+0000 7ff36d800640 1 --2- 172.21.15.22:0/3716085499 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff3681052a0 0x7ff3681a0f10 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7ff360036040 tx=0x7ff360005de0 comp rx=0 tx=0).stop 2024-03-28T18:06:27.355 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:27.355+0000 7ff36d800640 1 -- 172.21.15.22:0/3716085499 shutdown_connections 2024-03-28T18:06:27.355 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:27.355+0000 7ff36d800640 1 --2- 172.21.15.22:0/3716085499 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7ff34c0a2880 0x7ff34c0a4d40 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:06:27.355 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:27.355+0000 7ff36d800640 1 --2- 172.21.15.22:0/3716085499 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff3681052a0 0x7ff3681a0f10 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:06:27.355 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:27.355+0000 7ff36d800640 1 --2- 172.21.15.22:0/3716085499 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ff3681048e0 0x7ff368199ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:06:27.355 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:27.355+0000 7ff36d800640 1 --2- 172.21.15.22:0/3716085499 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff3681036f0 0x7ff368199960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:06:27.355 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:27.355+0000 7ff36d800640 1 -- 172.21.15.22:0/3716085499 >> 172.21.15.22:0/3716085499 conn(0x7ff3680fec80 msgr2=0x7ff368107b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:06:27.355 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:27.355+0000 7ff36d800640 1 -- 172.21.15.22:0/3716085499 shutdown_connections 2024-03-28T18:06:27.355 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:27.355+0000 7ff36d800640 1 -- 172.21.15.22:0/3716085499 wait complete. 2024-03-28T18:06:27.374 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-28T17:58:33.168872+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-28T18:04:47.004797+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '89', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-03-28T18:04:48.465502+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '90', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-28T18:06:27.374 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-28T18:06:27.424 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:06:27 smithi134 ceph-mon[35037]: from='client.? 172.21.15.22:0/3903491373' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-28T18:06:27.424 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:06:27 smithi134 ceph-mon[35037]: from='client.? 172.21.15.22:0/2436774832' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-28T18:06:27.432 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:06:27 smithi079 ceph-mon[35312]: from='client.? 172.21.15.22:0/3903491373' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-28T18:06:27.432 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:06:27 smithi079 ceph-mon[35312]: from='client.? 172.21.15.22:0/2436774832' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-28T18:06:27.602 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:27.602+0000 7f9788600640 1 -- 172.21.15.22:0/677520113 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9780105490 msgr2=0x7f978010bd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:06:27.603 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:27.602+0000 7f9788600640 1 --2- 172.21.15.22:0/677520113 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9780105490 0x7f978010bd20 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f9778009900 tx=0x7f977802bc30 comp rx=0 tx=0).stop 2024-03-28T18:06:27.603 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:27.602+0000 7f9788600640 1 -- 172.21.15.22:0/677520113 shutdown_connections 2024-03-28T18:06:27.603 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:27.602+0000 7f9788600640 1 --2- 172.21.15.22:0/677520113 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9780105490 0x7f978010bd20 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:06:27.603 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:27.602+0000 7f9788600640 1 --2- 172.21.15.22:0/677520113 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9780104ad0 0x7f9780104f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:06:27.603 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:27.602+0000 7f9788600640 1 --2- 172.21.15.22:0/677520113 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f97801038e0 0x7f9780103d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:06:27.603 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:27.602+0000 7f9788600640 1 -- 172.21.15.22:0/677520113 >> 172.21.15.22:0/677520113 conn(0x7f97800fee50 msgr2=0x7f9780101290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:06:27.603 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:27.602+0000 7f9788600640 1 -- 172.21.15.22:0/677520113 shutdown_connections 2024-03-28T18:06:27.603 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:27.602+0000 7f9788600640 1 -- 172.21.15.22:0/677520113 wait complete. 2024-03-28T18:06:27.605 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:27.603+0000 7f9788600640 1 Processor -- start 2024-03-28T18:06:27.605 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:27.603+0000 7f9788600640 1 -- start start 2024-03-28T18:06:27.605 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:27.604+0000 7f9788600640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f97801038e0 0x7f9780079450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:06:27.605 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:27.604+0000 7f9788600640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9780104ad0 0x7f9780079990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:06:27.605 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:27.604+0000 7f9788600640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9780105490 0x7f9780075fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:06:27.605 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:27.604+0000 7f9788600640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f978010f590 con 0x7f9780104ad0 2024-03-28T18:06:27.605 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:27.604+0000 7f9788600640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f978010f410 con 0x7f97801038e0 2024-03-28T18:06:27.605 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:27.604+0000 7f9788600640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f978010f710 con 0x7f9780105490 2024-03-28T18:06:27.605 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:27.604+0000 7f9785200640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9780104ad0 0x7f9780079990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:06:27.606 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:27.604+0000 7f9785200640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9780104ad0 0x7f9780079990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:48996/0 (socket says 172.21.15.22:48996) 2024-03-28T18:06:27.606 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:27.604+0000 7f9785200640 1 -- 172.21.15.22:0/3743812155 learned_addr learned my addr 172.21.15.22:0/3743812155 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-28T18:06:27.606 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:27.604+0000 7f9785c00640 1 --2- 172.21.15.22:0/3743812155 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f97801038e0 0x7f9780079450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:06:27.606 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:27.604+0000 7f9785200640 1 -- 172.21.15.22:0/3743812155 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9780105490 msgr2=0x7f9780075fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:06:27.606 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:27.604+0000 7f9785200640 1 --2- 172.21.15.22:0/3743812155 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9780105490 0x7f9780075fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:06:27.606 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:27.604+0000 7f9785200640 1 -- 172.21.15.22:0/3743812155 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f97801038e0 msgr2=0x7f9780079450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:06:27.606 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:27.604+0000 7f9785200640 1 --2- 172.21.15.22:0/3743812155 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f97801038e0 0x7f9780079450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:06:27.606 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:27.604+0000 7f9785200640 1 -- 172.21.15.22:0/3743812155 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9780076880 con 0x7f9780104ad0 2024-03-28T18:06:27.606 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:27.605+0000 7f9785200640 1 --2- 172.21.15.22:0/3743812155 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9780104ad0 0x7f9780079990 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f976c00d770 tx=0x7f976c00dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:06:27.606 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:27.605+0000 7f977e000640 1 -- 172.21.15.22:0/3743812155 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f976c017070 con 0x7f9780104ad0 2024-03-28T18:06:27.606 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:27.605+0000 7f977e000640 1 -- 172.21.15.22:0/3743812155 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f976c0054f0 con 0x7f9780104ad0 2024-03-28T18:06:27.606 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:27.605+0000 7f977e000640 1 -- 172.21.15.22:0/3743812155 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f976c004d60 con 0x7f9780104ad0 2024-03-28T18:06:27.606 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:27.605+0000 7f9788600640 1 -- 172.21.15.22:0/3743812155 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f97801a5fa0 con 0x7f9780104ad0 2024-03-28T18:06:27.606 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:27.605+0000 7f9788600640 1 -- 172.21.15.22:0/3743812155 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f97801a6350 con 0x7f9780104ad0 2024-03-28T18:06:27.607 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:27.607+0000 7f9788600640 1 -- 172.21.15.22:0/3743812155 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9780068910 con 0x7f9780104ad0 2024-03-28T18:06:27.609 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:27.609+0000 7f977e000640 1 -- 172.21.15.22:0/3743812155 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 18) v1 ==== 137290+0+0 (secure 0 0 0) 0x7f976c004f00 con 0x7f9780104ad0 2024-03-28T18:06:27.611 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:27.611+0000 7f977e000640 1 --2- 172.21.15.22:0/3743812155 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7f97640a2880 0x7f97640a4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:06:27.611 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:27.612+0000 7f977e000640 1 -- 172.21.15.22:0/3743812155 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 8153+0+0 (secure 0 0 0) 0x7f976c0ce420 con 0x7f9780104ad0 2024-03-28T18:06:27.611 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:27.612+0000 7f9785c00640 1 --2- 172.21.15.22:0/3743812155 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7f97640a2880 0x7f97640a4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:06:27.612 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:27.612+0000 7f9785c00640 1 --2- 172.21.15.22:0/3743812155 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7f97640a2880 0x7f97640a4d40 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f9780104930 tx=0x7f9770009500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:06:27.615 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:27.616+0000 7f977e000640 1 -- 172.21.15.22:0/3743812155 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f9780068910 con 0x7f9780104ad0 2024-03-28T18:06:27.818 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:27.819+0000 7f9788600640 1 -- 172.21.15.22:0/3743812155 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f978002d0f0 con 0x7f9780104ad0 2024-03-28T18:06:27.819 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:27.820+0000 7f977e000640 1 -- 172.21.15.22:0/3743812155 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v92) v1 ==== 93+0+10 (secure 0 0 0) 0x7f976c09b560 con 0x7f9780104ad0 2024-03-28T18:06:27.819 INFO:teuthology.orchestra.run.smithi022.stdout:pg_num: 1 2024-03-28T18:06:27.822 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:27.822+0000 7f9788600640 1 -- 172.21.15.22:0/3743812155 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7f97640a2880 msgr2=0x7f97640a4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:06:27.823 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:27.822+0000 7f9788600640 1 --2- 172.21.15.22:0/3743812155 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7f97640a2880 0x7f97640a4d40 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f9780104930 tx=0x7f9770009500 comp rx=0 tx=0).stop 2024-03-28T18:06:27.823 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:27.822+0000 7f9788600640 1 -- 172.21.15.22:0/3743812155 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9780104ad0 msgr2=0x7f9780079990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:06:27.823 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:27.822+0000 7f9788600640 1 --2- 172.21.15.22:0/3743812155 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9780104ad0 0x7f9780079990 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f976c00d770 tx=0x7f976c00dc30 comp rx=0 tx=0).stop 2024-03-28T18:06:27.823 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:27.823+0000 7f9788600640 1 -- 172.21.15.22:0/3743812155 shutdown_connections 2024-03-28T18:06:27.823 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:27.823+0000 7f9788600640 1 --2- 172.21.15.22:0/3743812155 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7f97640a2880 0x7f97640a4d40 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:06:27.823 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:27.823+0000 7f9788600640 1 --2- 172.21.15.22:0/3743812155 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9780105490 0x7f9780075fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:06:27.823 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:27.823+0000 7f9788600640 1 --2- 172.21.15.22:0/3743812155 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9780104ad0 0x7f9780079990 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:06:27.823 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:27.823+0000 7f9788600640 1 --2- 172.21.15.22:0/3743812155 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f97801038e0 0x7f9780079450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:06:27.823 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:27.823+0000 7f9788600640 1 -- 172.21.15.22:0/3743812155 >> 172.21.15.22:0/3743812155 conn(0x7f97800fee50 msgr2=0x7f9780107d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:06:27.823 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:27.823+0000 7f9788600640 1 -- 172.21.15.22:0/3743812155 shutdown_connections 2024-03-28T18:06:27.823 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:27.823+0000 7f9788600640 1 -- 172.21.15.22:0/3743812155 wait complete. 2024-03-28T18:06:27.844 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-28T18:06:28.072 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:28.072+0000 7f6efa400640 1 -- 172.21.15.22:0/3141685397 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6ef410db70 msgr2=0x7f6ef410ff60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:06:28.072 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:28.072+0000 7f6efa400640 1 --2- 172.21.15.22:0/3141685397 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6ef410db70 0x7f6ef410ff60 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f6eec009cd0 tx=0x7f6eec02c0f0 comp rx=0 tx=0).stop 2024-03-28T18:06:28.072 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:28.072+0000 7f6efa400640 1 -- 172.21.15.22:0/3141685397 shutdown_connections 2024-03-28T18:06:28.072 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:28.072+0000 7f6efa400640 1 --2- 172.21.15.22:0/3141685397 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6ef410db70 0x7f6ef410ff60 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:06:28.072 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:28.072+0000 7f6efa400640 1 --2- 172.21.15.22:0/3141685397 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f6ef4074100 0x7f6ef4074580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:06:28.073 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:28.072+0000 7f6efa400640 1 --2- 172.21.15.22:0/3141685397 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6ef40756e0 0x7f6ef4073b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:06:28.073 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:28.072+0000 7f6efa400640 1 -- 172.21.15.22:0/3141685397 >> 172.21.15.22:0/3141685397 conn(0x7f6ef40fcb10 msgr2=0x7f6ef40fef50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:06:28.073 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:28.072+0000 7f6efa400640 1 -- 172.21.15.22:0/3141685397 shutdown_connections 2024-03-28T18:06:28.073 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:28.072+0000 7f6efa400640 1 -- 172.21.15.22:0/3141685397 wait complete. 2024-03-28T18:06:28.075 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:28.073+0000 7f6efa400640 1 Processor -- start 2024-03-28T18:06:28.075 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:28.073+0000 7f6efa400640 1 -- start start 2024-03-28T18:06:28.075 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:28.074+0000 7f6efa400640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f6ef4074100 0x7f6ef419de70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:06:28.075 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:28.074+0000 7f6efa400640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6ef40756e0 0x7f6ef419e3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:06:28.075 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:28.074+0000 7f6efa400640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6ef410db70 0x7f6ef41a5420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:06:28.075 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:28.074+0000 7f6efa400640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ef4113800 con 0x7f6ef410db70 2024-03-28T18:06:28.075 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:28.074+0000 7f6efa400640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ef4113680 con 0x7f6ef4074100 2024-03-28T18:06:28.075 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:28.074+0000 7f6efa400640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ef4113980 con 0x7f6ef40756e0 2024-03-28T18:06:28.076 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:28.074+0000 7f6ef9a00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6ef410db70 0x7f6ef41a5420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:06:28.076 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:28.074+0000 7f6ef9a00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6ef410db70 0x7f6ef41a5420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:49012/0 (socket says 172.21.15.22:49012) 2024-03-28T18:06:28.076 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:28.074+0000 7f6ef9a00640 1 -- 172.21.15.22:0/791991780 learned_addr learned my addr 172.21.15.22:0/791991780 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-28T18:06:28.076 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:28.074+0000 7f6ef9000640 1 --2- 172.21.15.22:0/791991780 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f6ef4074100 0x7f6ef419de70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:06:28.076 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:28.074+0000 7f6eebe00640 1 --2- 172.21.15.22:0/791991780 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6ef40756e0 0x7f6ef419e3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:06:28.076 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:28.074+0000 7f6ef9a00640 1 -- 172.21.15.22:0/791991780 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6ef40756e0 msgr2=0x7f6ef419e3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:06:28.076 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:28.074+0000 7f6ef9a00640 1 --2- 172.21.15.22:0/791991780 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6ef40756e0 0x7f6ef419e3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:06:28.076 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:28.074+0000 7f6ef9a00640 1 -- 172.21.15.22:0/791991780 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f6ef4074100 msgr2=0x7f6ef419de70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:06:28.076 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:28.074+0000 7f6ef9a00640 1 --2- 172.21.15.22:0/791991780 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f6ef4074100 0x7f6ef419de70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:06:28.076 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:28.074+0000 7f6ef9a00640 1 -- 172.21.15.22:0/791991780 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ef41a5a90 con 0x7f6ef410db70 2024-03-28T18:06:28.076 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:28.074+0000 7f6ef9000640 1 --2- 172.21.15.22:0/791991780 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f6ef4074100 0x7f6ef419de70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-28T18:06:28.076 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:28.075+0000 7f6ef9a00640 1 --2- 172.21.15.22:0/791991780 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6ef410db70 0x7f6ef41a5420 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f6eec02c630 tx=0x7f6eec032980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:06:28.076 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:28.075+0000 7f6ef2000640 1 -- 172.21.15.22:0/791991780 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6eec039050 con 0x7f6ef410db70 2024-03-28T18:06:28.076 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:28.075+0000 7f6ef2000640 1 -- 172.21.15.22:0/791991780 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f6eec03a040 con 0x7f6ef410db70 2024-03-28T18:06:28.076 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:28.075+0000 7f6efa400640 1 -- 172.21.15.22:0/791991780 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6ef41a5d20 con 0x7f6ef410db70 2024-03-28T18:06:28.077 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:28.075+0000 7f6ef2000640 1 -- 172.21.15.22:0/791991780 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6eec040450 con 0x7f6ef410db70 2024-03-28T18:06:28.077 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:28.075+0000 7f6efa400640 1 -- 172.21.15.22:0/791991780 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6ef41a61c0 con 0x7f6ef410db70 2024-03-28T18:06:28.077 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:28.077+0000 7f6efa400640 1 -- 172.21.15.22:0/791991780 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6ef4103d40 con 0x7f6ef410db70 2024-03-28T18:06:28.078 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:28.078+0000 7f6ef2000640 1 -- 172.21.15.22:0/791991780 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 18) v1 ==== 137290+0+0 (secure 0 0 0) 0x7f6eec003c00 con 0x7f6ef410db70 2024-03-28T18:06:28.079 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:28.080+0000 7f6ef2000640 1 --2- 172.21.15.22:0/791991780 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7f6ed40a2800 0x7f6ed40a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:06:28.080 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:28.080+0000 7f6ef2000640 1 -- 172.21.15.22:0/791991780 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 8153+0+0 (secure 0 0 0) 0x7f6eec0f16b0 con 0x7f6ef410db70 2024-03-28T18:06:28.080 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:28.080+0000 7f6ef9000640 1 --2- 172.21.15.22:0/791991780 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7f6ed40a2800 0x7f6ed40a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:06:28.080 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:28.081+0000 7f6ef9000640 1 --2- 172.21.15.22:0/791991780 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7f6ed40a2800 0x7f6ed40a4cc0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f6edc0058c0 tx=0x7f6edc005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:06:28.084 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:28.084+0000 7f6ef2000640 1 -- 172.21.15.22:0/791991780 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f6eec0be750 con 0x7f6ef410db70 2024-03-28T18:06:28.282 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:06:28 smithi022 ceph-mon[31985]: pgmap v420: 97 pgs: 97 active+clean; 643 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 341 B/s wr, 12 op/s 2024-03-28T18:06:28.282 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:06:28 smithi022 ceph-mon[31985]: from='client.? 172.21.15.22:0/3716085499' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-28T18:06:28.282 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:06:28 smithi022 ceph-mon[31985]: from='client.? 172.21.15.22:0/3743812155' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-28T18:06:28.283 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:28.284+0000 7f6efa400640 1 -- 172.21.15.22:0/791991780 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f6ef40622a0 con 0x7f6ef410db70 2024-03-28T18:06:28.284 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:28.285+0000 7f6ef2000640 1 -- 172.21.15.22:0/791991780 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v92) v1 ==== 104+0+11 (secure 0 0 0) 0x7f6eec0be050 con 0x7f6ef410db70 2024-03-28T18:06:28.285 INFO:teuthology.orchestra.run.smithi022.stdout:pg_num: 32 2024-03-28T18:06:28.288 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:28.288+0000 7f6efa400640 1 -- 172.21.15.22:0/791991780 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7f6ed40a2800 msgr2=0x7f6ed40a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:06:28.288 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:28.288+0000 7f6efa400640 1 --2- 172.21.15.22:0/791991780 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7f6ed40a2800 0x7f6ed40a4cc0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f6edc0058c0 tx=0x7f6edc005850 comp rx=0 tx=0).stop 2024-03-28T18:06:28.288 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:28.288+0000 7f6efa400640 1 -- 172.21.15.22:0/791991780 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6ef410db70 msgr2=0x7f6ef41a5420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:06:28.288 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:28.288+0000 7f6efa400640 1 --2- 172.21.15.22:0/791991780 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6ef410db70 0x7f6ef41a5420 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f6eec02c630 tx=0x7f6eec032980 comp rx=0 tx=0).stop 2024-03-28T18:06:28.288 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:28.288+0000 7f6efa400640 1 -- 172.21.15.22:0/791991780 shutdown_connections 2024-03-28T18:06:28.288 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:28.288+0000 7f6efa400640 1 --2- 172.21.15.22:0/791991780 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7f6ed40a2800 0x7f6ed40a4cc0 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:06:28.288 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:28.288+0000 7f6efa400640 1 --2- 172.21.15.22:0/791991780 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6ef410db70 0x7f6ef41a5420 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:06:28.288 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:28.288+0000 7f6efa400640 1 --2- 172.21.15.22:0/791991780 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6ef40756e0 0x7f6ef419e3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:06:28.288 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:28.288+0000 7f6efa400640 1 --2- 172.21.15.22:0/791991780 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f6ef4074100 0x7f6ef419de70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:06:28.288 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:28.288+0000 7f6efa400640 1 -- 172.21.15.22:0/791991780 >> 172.21.15.22:0/791991780 conn(0x7f6ef40fcb10 msgr2=0x7f6ef4100f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:06:28.289 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:28.288+0000 7f6efa400640 1 -- 172.21.15.22:0/791991780 shutdown_connections 2024-03-28T18:06:28.289 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:28.288+0000 7f6efa400640 1 -- 172.21.15.22:0/791991780 wait complete. 2024-03-28T18:06:28.309 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-28T18:06:28.424 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:06:28 smithi134 ceph-mon[35037]: pgmap v420: 97 pgs: 97 active+clean; 643 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 341 B/s wr, 12 op/s 2024-03-28T18:06:28.425 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:06:28 smithi134 ceph-mon[35037]: from='client.? 172.21.15.22:0/3716085499' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-28T18:06:28.425 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:06:28 smithi134 ceph-mon[35037]: from='client.? 172.21.15.22:0/3743812155' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-28T18:06:28.432 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:06:28 smithi079 ceph-mon[35312]: pgmap v420: 97 pgs: 97 active+clean; 643 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 341 B/s wr, 12 op/s 2024-03-28T18:06:28.432 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:06:28 smithi079 ceph-mon[35312]: from='client.? 172.21.15.22:0/3716085499' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-28T18:06:28.432 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:06:28 smithi079 ceph-mon[35312]: from='client.? 172.21.15.22:0/3743812155' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-28T18:06:28.537 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:28.537+0000 7f58d0600640 1 -- 172.21.15.22:0/1828147468 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f58c81028a0 msgr2=0x7f58c8102d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:06:28.537 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:28.537+0000 7f58d0600640 1 --2- 172.21.15.22:0/1828147468 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f58c81028a0 0x7f58c8102d20 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f58b8009900 tx=0x7f58b802bc40 comp rx=0 tx=0).stop 2024-03-28T18:06:28.537 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:28.537+0000 7f58d0600640 1 -- 172.21.15.22:0/1828147468 shutdown_connections 2024-03-28T18:06:28.537 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:28.537+0000 7f58d0600640 1 --2- 172.21.15.22:0/1828147468 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f58c8103260 0x7f58c8109af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:06:28.537 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:28.537+0000 7f58d0600640 1 --2- 172.21.15.22:0/1828147468 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f58c81028a0 0x7f58c8102d20 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:06:28.537 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:28.537+0000 7f58d0600640 1 --2- 172.21.15.22:0/1828147468 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f58c81016b0 0x7f58c8101ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:06:28.537 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:28.537+0000 7f58d0600640 1 -- 172.21.15.22:0/1828147468 >> 172.21.15.22:0/1828147468 conn(0x7f58c80fcc40 msgr2=0x7f58c80ff060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:06:28.537 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:28.537+0000 7f58d0600640 1 -- 172.21.15.22:0/1828147468 shutdown_connections 2024-03-28T18:06:28.537 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:28.537+0000 7f58d0600640 1 -- 172.21.15.22:0/1828147468 wait complete. 2024-03-28T18:06:28.538 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:28.538+0000 7f58d0600640 1 Processor -- start 2024-03-28T18:06:28.538 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:28.538+0000 7f58d0600640 1 -- start start 2024-03-28T18:06:28.538 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:28.538+0000 7f58d0600640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f58c81016b0 0x7f58c8075450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:06:28.539 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:28.539+0000 7f58d0600640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f58c81028a0 0x7f58c8073a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:06:28.539 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:28.539+0000 7f58d0600640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f58c8103260 0x7f58c8073fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:06:28.539 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:28.539+0000 7f58d0600640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58c810d360 con 0x7f58c8103260 2024-03-28T18:06:28.539 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:28.539+0000 7f58d0600640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58c810d1e0 con 0x7f58c81016b0 2024-03-28T18:06:28.539 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:28.539+0000 7f58d0600640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58c810d4e0 con 0x7f58c81028a0 2024-03-28T18:06:28.539 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:28.539+0000 7f58ce600640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f58c8103260 0x7f58c8073fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:06:28.539 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:28.539+0000 7f58ce600640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f58c8103260 0x7f58c8073fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:49022/0 (socket says 172.21.15.22:49022) 2024-03-28T18:06:28.539 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:28.539+0000 7f58ce600640 1 -- 172.21.15.22:0/2671672304 learned_addr learned my addr 172.21.15.22:0/2671672304 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-28T18:06:28.539 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:28.539+0000 7f58cdc00640 1 --2- 172.21.15.22:0/2671672304 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f58c81016b0 0x7f58c8075450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:06:28.540 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:28.539+0000 7f58cd200640 1 --2- 172.21.15.22:0/2671672304 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f58c81028a0 0x7f58c8073a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:06:28.540 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:28.539+0000 7f58ce600640 1 -- 172.21.15.22:0/2671672304 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f58c81028a0 msgr2=0x7f58c8073a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:06:28.540 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:28.539+0000 7f58ce600640 1 --2- 172.21.15.22:0/2671672304 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f58c81028a0 0x7f58c8073a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:06:28.540 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:28.539+0000 7f58ce600640 1 -- 172.21.15.22:0/2671672304 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f58c81016b0 msgr2=0x7f58c8075450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:06:28.540 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:28.539+0000 7f58ce600640 1 --2- 172.21.15.22:0/2671672304 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f58c81016b0 0x7f58c8075450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:06:28.540 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:28.539+0000 7f58ce600640 1 -- 172.21.15.22:0/2671672304 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f58c8074860 con 0x7f58c8103260 2024-03-28T18:06:28.541 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:28.539+0000 7f58cd200640 1 --2- 172.21.15.22:0/2671672304 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f58c81028a0 0x7f58c8073a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T18:06:28.541 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:28.540+0000 7f58ce600640 1 --2- 172.21.15.22:0/2671672304 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f58c8103260 0x7f58c8073fd0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f58c000eaa0 tx=0x7f58c000ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:06:28.541 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:28.540+0000 7f58c6000640 1 -- 172.21.15.22:0/2671672304 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58c000cdb0 con 0x7f58c8103260 2024-03-28T18:06:28.541 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:28.540+0000 7f58c6000640 1 -- 172.21.15.22:0/2671672304 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f58c0005020 con 0x7f58c8103260 2024-03-28T18:06:28.541 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:28.540+0000 7f58c6000640 1 -- 172.21.15.22:0/2671672304 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58c00116b0 con 0x7f58c8103260 2024-03-28T18:06:28.541 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:28.540+0000 7f58d0600640 1 -- 172.21.15.22:0/2671672304 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f58c81a1b20 con 0x7f58c8103260 2024-03-28T18:06:28.541 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:28.540+0000 7f58d0600640 1 -- 172.21.15.22:0/2671672304 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f58c81a2070 con 0x7f58c8103260 2024-03-28T18:06:28.541 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:28.541+0000 7f58d0600640 1 -- 172.21.15.22:0/2671672304 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f58c81089d0 con 0x7f58c8103260 2024-03-28T18:06:28.543 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:28.543+0000 7f58c6000640 1 -- 172.21.15.22:0/2671672304 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 18) v1 ==== 137290+0+0 (secure 0 0 0) 0x7f58c00051d0 con 0x7f58c8103260 2024-03-28T18:06:28.545 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:28.545+0000 7f58c6000640 1 --2- 172.21.15.22:0/2671672304 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7f58b00a2880 0x7f58b00a4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:06:28.545 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:28.545+0000 7f58c6000640 1 -- 172.21.15.22:0/2671672304 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 8153+0+0 (secure 0 0 0) 0x7f58c00ced40 con 0x7f58c8103260 2024-03-28T18:06:28.545 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:28.545+0000 7f58cdc00640 1 --2- 172.21.15.22:0/2671672304 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7f58b00a2880 0x7f58b00a4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:06:28.545 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:28.546+0000 7f58cdc00640 1 --2- 172.21.15.22:0/2671672304 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7f58b00a2880 0x7f58b00a4d40 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f58bc005fd0 tx=0x7f58bc005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:06:28.549 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:28.549+0000 7f58c6000640 1 -- 172.21.15.22:0/2671672304 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f58c0016030 con 0x7f58c8103260 2024-03-28T18:06:28.753 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:28.753+0000 7f58d0600640 1 -- 172.21.15.22:0/2671672304 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f58c8062340 con 0x7f58c8103260 2024-03-28T18:06:28.753 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:28.754+0000 7f58c6000640 1 -- 172.21.15.22:0/2671672304 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v92) v1 ==== 100+0+11 (secure 0 0 0) 0x7f58c009bde0 con 0x7f58c8103260 2024-03-28T18:06:28.753 INFO:teuthology.orchestra.run.smithi022.stdout:pg_num: 64 2024-03-28T18:06:28.756 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:28.756+0000 7f58d0600640 1 -- 172.21.15.22:0/2671672304 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7f58b00a2880 msgr2=0x7f58b00a4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:06:28.756 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:28.756+0000 7f58d0600640 1 --2- 172.21.15.22:0/2671672304 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7f58b00a2880 0x7f58b00a4d40 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f58bc005fd0 tx=0x7f58bc005850 comp rx=0 tx=0).stop 2024-03-28T18:06:28.756 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:28.756+0000 7f58d0600640 1 -- 172.21.15.22:0/2671672304 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f58c8103260 msgr2=0x7f58c8073fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:06:28.756 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:28.756+0000 7f58d0600640 1 --2- 172.21.15.22:0/2671672304 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f58c8103260 0x7f58c8073fd0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f58c000eaa0 tx=0x7f58c000ef60 comp rx=0 tx=0).stop 2024-03-28T18:06:28.757 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:28.757+0000 7f58d0600640 1 -- 172.21.15.22:0/2671672304 shutdown_connections 2024-03-28T18:06:28.757 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:28.757+0000 7f58d0600640 1 --2- 172.21.15.22:0/2671672304 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7f58b00a2880 0x7f58b00a4d40 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:06:28.757 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:28.757+0000 7f58d0600640 1 --2- 172.21.15.22:0/2671672304 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f58c8103260 0x7f58c8073fd0 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:06:28.757 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:28.757+0000 7f58d0600640 1 --2- 172.21.15.22:0/2671672304 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f58c81028a0 0x7f58c8073a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:06:28.757 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:28.757+0000 7f58d0600640 1 --2- 172.21.15.22:0/2671672304 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f58c81016b0 0x7f58c8075450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:06:28.757 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:28.757+0000 7f58d0600640 1 -- 172.21.15.22:0/2671672304 >> 172.21.15.22:0/2671672304 conn(0x7f58c80fcc40 msgr2=0x7f58c8105ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:06:28.757 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:28.757+0000 7f58d0600640 1 -- 172.21.15.22:0/2671672304 shutdown_connections 2024-03-28T18:06:28.757 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:28.757+0000 7f58d0600640 1 -- 172.21.15.22:0/2671672304 wait complete. 2024-03-28T18:06:28.776 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-28T18:06:28.998 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:28.998+0000 7f2995800640 1 -- 172.21.15.22:0/770880487 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f29901036f0 msgr2=0x7f2990103b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:06:28.999 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:28.998+0000 7f2995800640 1 --2- 172.21.15.22:0/770880487 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f29901036f0 0x7f2990103b10 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f2980009900 tx=0x7f298002bc30 comp rx=0 tx=0).stop 2024-03-28T18:06:28.999 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:28.998+0000 7f2995800640 1 -- 172.21.15.22:0/770880487 shutdown_connections 2024-03-28T18:06:28.999 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:28.998+0000 7f2995800640 1 --2- 172.21.15.22:0/770880487 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f29901052a0 0x7f299010bb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:06:28.999 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:28.998+0000 7f2995800640 1 --2- 172.21.15.22:0/770880487 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f29901048e0 0x7f2990104d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:06:28.999 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:28.998+0000 7f2995800640 1 --2- 172.21.15.22:0/770880487 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f29901036f0 0x7f2990103b10 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:06:28.999 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:28.998+0000 7f2995800640 1 -- 172.21.15.22:0/770880487 >> 172.21.15.22:0/770880487 conn(0x7f29900fec80 msgr2=0x7f29901010a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:06:28.999 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:28.999+0000 7f2995800640 1 -- 172.21.15.22:0/770880487 shutdown_connections 2024-03-28T18:06:28.999 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:28.999+0000 7f2995800640 1 -- 172.21.15.22:0/770880487 wait complete. 2024-03-28T18:06:29.001 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:29.000+0000 7f2995800640 1 Processor -- start 2024-03-28T18:06:29.001 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:29.000+0000 7f2995800640 1 -- start start 2024-03-28T18:06:29.001 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:29.000+0000 7f2995800640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f29901036f0 0x7f29901999c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:06:29.001 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:29.000+0000 7f2995800640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f29901048e0 0x7f2990199f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:06:29.001 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:29.000+0000 7f2995800640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f29901052a0 0x7f29901a0f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:06:29.001 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:29.000+0000 7f2995800640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f299010f3a0 con 0x7f29901048e0 2024-03-28T18:06:29.001 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:29.000+0000 7f2995800640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f299010f220 con 0x7f29901036f0 2024-03-28T18:06:29.001 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:29.000+0000 7f2995800640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f299010f520 con 0x7f29901052a0 2024-03-28T18:06:29.001 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:29.000+0000 7f298f400640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f29901048e0 0x7f2990199f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:06:29.001 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:29.000+0000 7f298f400640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f29901048e0 0x7f2990199f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:49030/0 (socket says 172.21.15.22:49030) 2024-03-28T18:06:29.001 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:29.000+0000 7f298f400640 1 -- 172.21.15.22:0/4241701771 learned_addr learned my addr 172.21.15.22:0/4241701771 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-28T18:06:29.001 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:29.000+0000 7f298fe00640 1 --2- 172.21.15.22:0/4241701771 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f29901036f0 0x7f29901999c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:06:29.001 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:29.001+0000 7f2994e00640 1 --2- 172.21.15.22:0/4241701771 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f29901052a0 0x7f29901a0f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:06:29.002 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:29.001+0000 7f298f400640 1 -- 172.21.15.22:0/4241701771 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f29901052a0 msgr2=0x7f29901a0f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:06:29.002 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:29.001+0000 7f298f400640 1 --2- 172.21.15.22:0/4241701771 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f29901052a0 0x7f29901a0f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:06:29.002 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:29.001+0000 7f298f400640 1 -- 172.21.15.22:0/4241701771 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f29901036f0 msgr2=0x7f29901999c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:06:29.002 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:29.001+0000 7f298f400640 1 --2- 172.21.15.22:0/4241701771 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f29901036f0 0x7f29901999c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:06:29.002 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:29.001+0000 7f298f400640 1 -- 172.21.15.22:0/4241701771 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f29901a15e0 con 0x7f29901048e0 2024-03-28T18:06:29.002 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:29.001+0000 7f298f400640 1 --2- 172.21.15.22:0/4241701771 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f29901048e0 0x7f2990199f00 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f297c00e9f0 tx=0x7f297c00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:06:29.003 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:29.002+0000 7f298cc00640 1 -- 172.21.15.22:0/4241701771 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f297c00cd80 con 0x7f29901048e0 2024-03-28T18:06:29.003 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:29.002+0000 7f298cc00640 1 -- 172.21.15.22:0/4241701771 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f297c010900 con 0x7f29901048e0 2024-03-28T18:06:29.003 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:29.002+0000 7f298cc00640 1 -- 172.21.15.22:0/4241701771 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f297c0189e0 con 0x7f29901048e0 2024-03-28T18:06:29.003 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:29.002+0000 7f2995800640 1 -- 172.21.15.22:0/4241701771 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f29901a18d0 con 0x7f29901048e0 2024-03-28T18:06:29.003 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:29.002+0000 7f2995800640 1 -- 172.21.15.22:0/4241701771 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f29901a1dd0 con 0x7f29901048e0 2024-03-28T18:06:29.003 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:29.003+0000 7f2995800640 1 -- 172.21.15.22:0/4241701771 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f299010a9b0 con 0x7f29901048e0 2024-03-28T18:06:29.005 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:29.006+0000 7f298cc00640 1 -- 172.21.15.22:0/4241701771 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 18) v1 ==== 137290+0+0 (secure 0 0 0) 0x7f297c010ab0 con 0x7f29901048e0 2024-03-28T18:06:29.007 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:29.008+0000 7f298cc00640 1 --2- 172.21.15.22:0/4241701771 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7f29740a2880 0x7f29740a4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:06:29.007 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:29.008+0000 7f298cc00640 1 -- 172.21.15.22:0/4241701771 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 8153+0+0 (secure 0 0 0) 0x7f297c0cedb0 con 0x7f29901048e0 2024-03-28T18:06:29.007 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:29.008+0000 7f298fe00640 1 --2- 172.21.15.22:0/4241701771 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7f29740a2880 0x7f29740a4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:06:29.008 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:29.009+0000 7f298fe00640 1 --2- 172.21.15.22:0/4241701771 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7f29740a2880 0x7f29740a4d40 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f298002c170 tx=0x7f298002c820 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:06:29.012 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:29.012+0000 7f298cc00640 1 -- 172.21.15.22:0/4241701771 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f297c0d3070 con 0x7f29901048e0 2024-03-28T18:06:29.257 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:29.257+0000 7f2995800640 1 -- 172.21.15.22:0/4241701771 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f299002d070 con 0x7f29901048e0 2024-03-28T18:06:29.258 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:29.258+0000 7f298cc00640 1 -- 172.21.15.22:0/4241701771 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) v1 ==== 94+0+11452 (secure 0 0 0) 0x7f297c09be50 con 0x7f29901048e0 2024-03-28T18:06:29.258 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-03-28T18:06:29.259 INFO:teuthology.orchestra.run.smithi022.stdout:{"epoch":26,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14544,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6835/3718238950","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6834","nonce":3718238950},{"type":"v1","addr":"172.21.15.22:6835","nonce":3718238950}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14562,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6837/3676939386","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6836","nonce":3676939386},{"type":"v1","addr":"172.21.15.22:6837","nonce":3676939386}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":26,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-28T18:04:48.991622+0000","modified":"2024-03-28T18:04:56.643864+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":34553,"mds_1":34511,"mds_2":34475,"mds_3":34442,"mds_4":24502},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14640":{"gid":14640,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.22:6839/3979998310","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6838","nonce":3979998310},{"type":"v1","addr":"172.21.15.22:6839","nonce":3979998310}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_14655":{"gid":14655,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.22:6841/3069056661","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6840","nonce":3069056661},{"type":"v1","addr":"172.21.15.22:6841","nonce":3069056661}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24412":{"gid":24412,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.134:6833/3176125362","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6832","nonce":3176125362},{"type":"v1","addr":"172.21.15.134:6833","nonce":3176125362}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24463":{"gid":24463,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.134:6835/3370549207","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6834","nonce":3370549207},{"type":"v1","addr":"172.21.15.134:6835","nonce":3370549207}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24478":{"gid":24478,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.134:6837/899825581","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6836","nonce":899825581},{"type":"v1","addr":"172.21.15.134:6837","nonce":899825581}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24502":{"gid":24502,"name":"i","rank":4,"incarnation":24,"state":"up:active","state_seq":7,"addr":"172.21.15.134:6839/3400263217","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6838","nonce":3400263217},{"type":"v1","addr":"172.21.15.134:6839","nonce":3400263217}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_34442":{"gid":34442,"name":"k","rank":3,"incarnation":22,"state":"up:active","state_seq":14,"addr":"172.21.15.79:6833/856896997","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6832","nonce":856896997},{"type":"v1","addr":"172.21.15.79:6833","nonce":856896997}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_34475":{"gid":34475,"name":"e","rank":2,"incarnation":20,"state":"up:active","state_seq":11,"addr":"172.21.15.79:6835/3365773431","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6834","nonce":3365773431},{"type":"v1","addr":"172.21.15.79:6835","nonce":3365773431}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_34511":{"gid":34511,"name":"h","rank":1,"incarnation":18,"state":"up:active","state_seq":8,"addr":"172.21.15.79:6837/1787903366","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6836","nonce":1787903366},{"type":"v1","addr":"172.21.15.79:6837","nonce":1787903366}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_34553":{"gid":34553,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.79:6839/2441542443","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6838","nonce":2441542443},{"type":"v1","addr":"172.21.15.79:6839","nonce":2441542443}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":34553,"qdb_cluster":[34511,34553,34475,34442,24502]},"id":1}]} 2024-03-28T18:06:29.259 INFO:teuthology.orchestra.run.smithi022.stderr:dumped fsmap epoch 26 2024-03-28T18:06:29.261 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:29.261+0000 7f2995800640 1 -- 172.21.15.22:0/4241701771 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7f29740a2880 msgr2=0x7f29740a4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:06:29.262 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:29.261+0000 7f2995800640 1 --2- 172.21.15.22:0/4241701771 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7f29740a2880 0x7f29740a4d40 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f298002c170 tx=0x7f298002c820 comp rx=0 tx=0).stop 2024-03-28T18:06:29.262 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:29.261+0000 7f2995800640 1 -- 172.21.15.22:0/4241701771 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f29901048e0 msgr2=0x7f2990199f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:06:29.262 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:29.261+0000 7f2995800640 1 --2- 172.21.15.22:0/4241701771 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f29901048e0 0x7f2990199f00 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f297c00e9f0 tx=0x7f297c00eeb0 comp rx=0 tx=0).stop 2024-03-28T18:06:29.262 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:29.262+0000 7f2995800640 1 -- 172.21.15.22:0/4241701771 shutdown_connections 2024-03-28T18:06:29.262 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:29.262+0000 7f2995800640 1 --2- 172.21.15.22:0/4241701771 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7f29740a2880 0x7f29740a4d40 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:06:29.262 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:29.262+0000 7f2995800640 1 --2- 172.21.15.22:0/4241701771 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f29901052a0 0x7f29901a0f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:06:29.262 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:29.262+0000 7f2995800640 1 --2- 172.21.15.22:0/4241701771 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f29901048e0 0x7f2990199f00 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:06:29.262 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:29.262+0000 7f2995800640 1 --2- 172.21.15.22:0/4241701771 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f29901036f0 0x7f29901999c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:06:29.262 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:29.262+0000 7f2995800640 1 -- 172.21.15.22:0/4241701771 >> 172.21.15.22:0/4241701771 conn(0x7f29900fec80 msgr2=0x7f2990107b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:06:29.262 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:29.262+0000 7f2995800640 1 -- 172.21.15.22:0/4241701771 shutdown_connections 2024-03-28T18:06:29.262 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:29.262+0000 7f2995800640 1 -- 172.21.15.22:0/4241701771 wait complete. 2024-03-28T18:06:29.281 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:06:29 smithi022 ceph-mon[31985]: from='client.? 172.21.15.22:0/791991780' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-28T18:06:29.281 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:06:29 smithi022 ceph-mon[31985]: from='client.? 172.21.15.22:0/2671672304' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-28T18:06:29.282 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-28T18:06:29.326 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:06:29 smithi134 ceph-mon[35037]: from='client.? 172.21.15.22:0/791991780' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-28T18:06:29.326 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:06:29 smithi134 ceph-mon[35037]: from='client.? 172.21.15.22:0/2671672304' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-28T18:06:29.432 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:06:29 smithi079 ceph-mon[35312]: from='client.? 172.21.15.22:0/791991780' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-28T18:06:29.432 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:06:29 smithi079 ceph-mon[35312]: from='client.? 172.21.15.22:0/2671672304' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-28T18:06:29.508 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:29.508+0000 7f9912400640 1 -- 172.21.15.22:0/4281758876 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f990c1030b0 msgr2=0x7f990c109940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:06:29.508 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:29.508+0000 7f9912400640 1 --2- 172.21.15.22:0/4281758876 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f990c1030b0 0x7f990c109940 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f9904009ca0 tx=0x7f990402c100 comp rx=0 tx=0).stop 2024-03-28T18:06:29.508 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:29.508+0000 7f9912400640 1 -- 172.21.15.22:0/4281758876 shutdown_connections 2024-03-28T18:06:29.508 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:29.508+0000 7f9912400640 1 --2- 172.21.15.22:0/4281758876 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f990c1030b0 0x7f990c109940 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:06:29.508 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:29.508+0000 7f9912400640 1 --2- 172.21.15.22:0/4281758876 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f990c1026f0 0x7f990c102b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:06:29.508 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:29.508+0000 7f9912400640 1 --2- 172.21.15.22:0/4281758876 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f990c101500 0x7f990c101920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:06:29.508 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:29.508+0000 7f9912400640 1 -- 172.21.15.22:0/4281758876 >> 172.21.15.22:0/4281758876 conn(0x7f990c0fca70 msgr2=0x7f990c0feeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:06:29.508 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:29.508+0000 7f9912400640 1 -- 172.21.15.22:0/4281758876 shutdown_connections 2024-03-28T18:06:29.509 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:29.508+0000 7f9912400640 1 -- 172.21.15.22:0/4281758876 wait complete. 2024-03-28T18:06:29.510 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:29.509+0000 7f9912400640 1 Processor -- start 2024-03-28T18:06:29.510 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:29.509+0000 7f9912400640 1 -- start start 2024-03-28T18:06:29.510 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:29.510+0000 7f9912400640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f990c101500 0x7f990c1999e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:06:29.510 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:29.510+0000 7f9912400640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f990c1026f0 0x7f990c199f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:06:29.510 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:29.510+0000 7f9912400640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f990c1030b0 0x7f990c1a0f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:06:29.510 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:29.510+0000 7f9912400640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f990c10d160 con 0x7f990c1030b0 2024-03-28T18:06:29.511 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:29.510+0000 7f9912400640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f990c10cfe0 con 0x7f990c1026f0 2024-03-28T18:06:29.511 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:29.510+0000 7f9912400640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f990c10d2e0 con 0x7f990c101500 2024-03-28T18:06:29.511 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:29.510+0000 7f9911a00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f990c1030b0 0x7f990c1a0f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:06:29.511 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:29.510+0000 7f9911a00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f990c1030b0 0x7f990c1a0f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:49056/0 (socket says 172.21.15.22:49056) 2024-03-28T18:06:29.511 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:29.510+0000 7f9911a00640 1 -- 172.21.15.22:0/71410541 learned_addr learned my addr 172.21.15.22:0/71410541 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-28T18:06:29.511 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:29.510+0000 7f9903e00640 1 --2- 172.21.15.22:0/71410541 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f990c1026f0 0x7f990c199f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:06:29.511 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:29.510+0000 7f9911000640 1 --2- 172.21.15.22:0/71410541 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f990c101500 0x7f990c1999e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:06:29.511 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:29.510+0000 7f9911a00640 1 -- 172.21.15.22:0/71410541 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f990c101500 msgr2=0x7f990c1999e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:06:29.511 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:29.510+0000 7f9911a00640 1 --2- 172.21.15.22:0/71410541 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f990c101500 0x7f990c1999e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:06:29.511 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:29.510+0000 7f9911a00640 1 -- 172.21.15.22:0/71410541 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f990c1026f0 msgr2=0x7f990c199f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:06:29.511 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:29.510+0000 7f9911a00640 1 --2- 172.21.15.22:0/71410541 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f990c1026f0 0x7f990c199f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:06:29.511 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:29.510+0000 7f9911a00640 1 -- 172.21.15.22:0/71410541 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f990c1a1600 con 0x7f990c1030b0 2024-03-28T18:06:29.511 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:29.511+0000 7f9911a00640 1 --2- 172.21.15.22:0/71410541 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f990c1030b0 0x7f990c1a0f90 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f9904009bf0 tx=0x7f9904034cd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:06:29.512 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:29.511+0000 7f990a000640 1 -- 172.21.15.22:0/71410541 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9904039050 con 0x7f990c1030b0 2024-03-28T18:06:29.512 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:29.511+0000 7f990a000640 1 -- 172.21.15.22:0/71410541 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f990403a040 con 0x7f990c1030b0 2024-03-28T18:06:29.512 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:29.511+0000 7f990a000640 1 -- 172.21.15.22:0/71410541 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99040046a0 con 0x7f990c1030b0 2024-03-28T18:06:29.512 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:29.511+0000 7f9912400640 1 -- 172.21.15.22:0/71410541 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f990c1a1890 con 0x7f990c1030b0 2024-03-28T18:06:29.512 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:29.511+0000 7f9912400640 1 -- 172.21.15.22:0/71410541 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f990c1a1d30 con 0x7f990c1030b0 2024-03-28T18:06:29.513 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:29.512+0000 7f9912400640 1 -- 172.21.15.22:0/71410541 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f990c1a1f90 con 0x7f990c1030b0 2024-03-28T18:06:29.514 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:29.514+0000 7f990a000640 1 -- 172.21.15.22:0/71410541 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 18) v1 ==== 137290+0+0 (secure 0 0 0) 0x7f9904003970 con 0x7f990c1030b0 2024-03-28T18:06:29.516 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:29.517+0000 7f990a000640 1 --2- 172.21.15.22:0/71410541 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7f98f00a2880 0x7f98f00a4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:06:29.516 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:29.517+0000 7f990a000640 1 -- 172.21.15.22:0/71410541 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 8153+0+0 (secure 0 0 0) 0x7f99040f1d50 con 0x7f990c1030b0 2024-03-28T18:06:29.516 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:29.517+0000 7f9911000640 1 --2- 172.21.15.22:0/71410541 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7f98f00a2880 0x7f98f00a4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:06:29.517 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:29.518+0000 7f9911000640 1 --2- 172.21.15.22:0/71410541 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7f98f00a2880 0x7f98f00a4d40 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f98fc005fd0 tx=0x7f98fc005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:06:29.519 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:29.520+0000 7f990a000640 1 -- 172.21.15.22:0/71410541 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f99040bedf0 con 0x7f990c1030b0 2024-03-28T18:06:29.725 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:29.725+0000 7f9912400640 1 -- 172.21.15.22:0/71410541 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f990c1a2320 con 0x7f990c1030b0 2024-03-28T18:06:29.729 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:29.729+0000 7f990a000640 1 -- 172.21.15.22:0/71410541 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v92) v1 ==== 74+0+22522 (secure 0 0 0) 0x7f99040be6f0 con 0x7f990c1030b0 2024-03-28T18:06:29.729 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-03-28T18:06:29.729 INFO:teuthology.orchestra.run.smithi022.stdout:{"epoch":92,"fsid":"fd220748-ed2b-11ee-95d0-87774f69a715","created":"2024-03-28T17:54:00.440279+0000","modified":"2024-03-28T18:05:11.725341+0000","last_up_change":"2024-03-28T18:03:43.220095+0000","last_in_change":"2024-03-28T18:03:18.526399+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-28T17:58:33.168872+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-28T18:04:47.004797+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-28T18:04:48.465502+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"90","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"5d708e06-f05b-40ba-bbcf-258ddc4f43d7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6802","nonce":2186290846},{"type":"v1","addr":"172.21.15.22:6803","nonce":2186290846}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6804","nonce":2186290846},{"type":"v1","addr":"172.21.15.22:6805","nonce":2186290846}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6808","nonce":2186290846},{"type":"v1","addr":"172.21.15.22:6809","nonce":2186290846}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6806","nonce":2186290846},{"type":"v1","addr":"172.21.15.22:6807","nonce":2186290846}]},"public_addr":"172.21.15.22:6803/2186290846","cluster_addr":"172.21.15.22:6805/2186290846","heartbeat_back_addr":"172.21.15.22:6809/2186290846","heartbeat_front_addr":"172.21.15.22:6807/2186290846","state":["exists","up"]},{"osd":1,"uuid":"68ad09bb-9256-409a-9867-4e14a2cb94aa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6800","nonce":2835549681},{"type":"v1","addr":"172.21.15.79:6801","nonce":2835549681}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6802","nonce":2835549681},{"type":"v1","addr":"172.21.15.79:6803","nonce":2835549681}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6806","nonce":2835549681},{"type":"v1","addr":"172.21.15.79:6807","nonce":2835549681}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6804","nonce":2835549681},{"type":"v1","addr":"172.21.15.79:6805","nonce":2835549681}]},"public_addr":"172.21.15.79:6801/2835549681","cluster_addr":"172.21.15.79:6803/2835549681","heartbeat_back_addr":"172.21.15.79:6807/2835549681","heartbeat_front_addr":"172.21.15.79:6805/2835549681","state":["exists","up"]},{"osd":2,"uuid":"ed11a5a7-004d-41f3-a5fc-eb2d603bd33f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6800","nonce":470588085},{"type":"v1","addr":"172.21.15.134:6801","nonce":470588085}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6802","nonce":470588085},{"type":"v1","addr":"172.21.15.134:6803","nonce":470588085}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6806","nonce":470588085},{"type":"v1","addr":"172.21.15.134:6807","nonce":470588085}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6804","nonce":470588085},{"type":"v1","addr":"172.21.15.134:6805","nonce":470588085}]},"public_addr":"172.21.15.134:6801/470588085","cluster_addr":"172.21.15.134:6803/470588085","heartbeat_back_addr":"172.21.15.134:6807/470588085","heartbeat_front_addr":"172.21.15.134:6805/470588085","state":["exists","up"]},{"osd":3,"uuid":"81a6c547-2a50-44c3-84d7-23ab6831790c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6810","nonce":1702441854},{"type":"v1","addr":"172.21.15.22:6811","nonce":1702441854}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6812","nonce":1702441854},{"type":"v1","addr":"172.21.15.22:6813","nonce":1702441854}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6816","nonce":1702441854},{"type":"v1","addr":"172.21.15.22:6817","nonce":1702441854}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6814","nonce":1702441854},{"type":"v1","addr":"172.21.15.22:6815","nonce":1702441854}]},"public_addr":"172.21.15.22:6811/1702441854","cluster_addr":"172.21.15.22:6813/1702441854","heartbeat_back_addr":"172.21.15.22:6817/1702441854","heartbeat_front_addr":"172.21.15.22:6815/1702441854","state":["exists","up"]},{"osd":4,"uuid":"d23a254e-7182-4471-8bca-7eb684ec618e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6808","nonce":630132437},{"type":"v1","addr":"172.21.15.79:6809","nonce":630132437}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6810","nonce":630132437},{"type":"v1","addr":"172.21.15.79:6811","nonce":630132437}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6814","nonce":630132437},{"type":"v1","addr":"172.21.15.79:6815","nonce":630132437}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6812","nonce":630132437},{"type":"v1","addr":"172.21.15.79:6813","nonce":630132437}]},"public_addr":"172.21.15.79:6809/630132437","cluster_addr":"172.21.15.79:6811/630132437","heartbeat_back_addr":"172.21.15.79:6815/630132437","heartbeat_front_addr":"172.21.15.79:6813/630132437","state":["exists","up"]},{"osd":5,"uuid":"a7b98502-2023-4f0a-bfa0-6840cc495e0c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":40,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6808","nonce":2135406915},{"type":"v1","addr":"172.21.15.134:6809","nonce":2135406915}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6810","nonce":2135406915},{"type":"v1","addr":"172.21.15.134:6811","nonce":2135406915}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6814","nonce":2135406915},{"type":"v1","addr":"172.21.15.134:6815","nonce":2135406915}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6812","nonce":2135406915},{"type":"v1","addr":"172.21.15.134:6813","nonce":2135406915}]},"public_addr":"172.21.15.134:6809/2135406915","cluster_addr":"172.21.15.134:6811/2135406915","heartbeat_back_addr":"172.21.15.134:6815/2135406915","heartbeat_front_addr":"172.21.15.134:6813/2135406915","state":["exists","up"]},{"osd":6,"uuid":"634c6562-d683-41c8-8242-fc848c38ce1f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6818","nonce":1400680396},{"type":"v1","addr":"172.21.15.22:6819","nonce":1400680396}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6820","nonce":1400680396},{"type":"v1","addr":"172.21.15.22:6821","nonce":1400680396}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6824","nonce":1400680396},{"type":"v1","addr":"172.21.15.22:6825","nonce":1400680396}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6822","nonce":1400680396},{"type":"v1","addr":"172.21.15.22:6823","nonce":1400680396}]},"public_addr":"172.21.15.22:6819/1400680396","cluster_addr":"172.21.15.22:6821/1400680396","heartbeat_back_addr":"172.21.15.22:6825/1400680396","heartbeat_front_addr":"172.21.15.22:6823/1400680396","state":["exists","up"]},{"osd":7,"uuid":"504eb4a8-6731-45ad-932c-1b38ef157871","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":53,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6816","nonce":1287105314},{"type":"v1","addr":"172.21.15.79:6817","nonce":1287105314}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6818","nonce":1287105314},{"type":"v1","addr":"172.21.15.79:6819","nonce":1287105314}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6822","nonce":1287105314},{"type":"v1","addr":"172.21.15.79:6823","nonce":1287105314}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6820","nonce":1287105314},{"type":"v1","addr":"172.21.15.79:6821","nonce":1287105314}]},"public_addr":"172.21.15.79:6817/1287105314","cluster_addr":"172.21.15.79:6819/1287105314","heartbeat_back_addr":"172.21.15.79:6823/1287105314","heartbeat_front_addr":"172.21.15.79:6821/1287105314","state":["exists","up"]},{"osd":8,"uuid":"20aa66c1-03d6-4cb8-a096-0eae41979c17","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":62,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6816","nonce":2265650388},{"type":"v1","addr":"172.21.15.134:6817","nonce":2265650388}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6818","nonce":2265650388},{"type":"v1","addr":"172.21.15.134:6819","nonce":2265650388}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6822","nonce":2265650388},{"type":"v1","addr":"172.21.15.134:6823","nonce":2265650388}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6820","nonce":2265650388},{"type":"v1","addr":"172.21.15.134:6821","nonce":2265650388}]},"public_addr":"172.21.15.134:6817/2265650388","cluster_addr":"172.21.15.134:6819/2265650388","heartbeat_back_addr":"172.21.15.134:6823/2265650388","heartbeat_front_addr":"172.21.15.134:6821/2265650388","state":["exists","up"]},{"osd":9,"uuid":"e1ae4020-7c2c-4334-a753-ee6b12d98584","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":70,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6826","nonce":3762222259},{"type":"v1","addr":"172.21.15.22:6827","nonce":3762222259}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6828","nonce":3762222259},{"type":"v1","addr":"172.21.15.22:6829","nonce":3762222259}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6832","nonce":3762222259},{"type":"v1","addr":"172.21.15.22:6833","nonce":3762222259}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6830","nonce":3762222259},{"type":"v1","addr":"172.21.15.22:6831","nonce":3762222259}]},"public_addr":"172.21.15.22:6827/3762222259","cluster_addr":"172.21.15.22:6829/3762222259","heartbeat_back_addr":"172.21.15.22:6833/3762222259","heartbeat_front_addr":"172.21.15.22:6831/3762222259","state":["exists","up"]},{"osd":10,"uuid":"f5198d52-df48-4679-a860-4dcb440b7a9f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":77,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6824","nonce":1746167904},{"type":"v1","addr":"172.21.15.79:6825","nonce":1746167904}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6826","nonce":1746167904},{"type":"v1","addr":"172.21.15.79:6827","nonce":1746167904}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6830","nonce":1746167904},{"type":"v1","addr":"172.21.15.79:6831","nonce":1746167904}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6828","nonce":1746167904},{"type":"v1","addr":"172.21.15.79:6829","nonce":1746167904}]},"public_addr":"172.21.15.79:6825/1746167904","cluster_addr":"172.21.15.79:6827/1746167904","heartbeat_back_addr":"172.21.15.79:6831/1746167904","heartbeat_front_addr":"172.21.15.79:6829/1746167904","state":["exists","up"]},{"osd":11,"uuid":"1427bca7-3414-4468-9246-e999156bc2ae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":85,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6824","nonce":4271716428},{"type":"v1","addr":"172.21.15.134:6825","nonce":4271716428}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6826","nonce":4271716428},{"type":"v1","addr":"172.21.15.134:6827","nonce":4271716428}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6830","nonce":4271716428},{"type":"v1","addr":"172.21.15.134:6831","nonce":4271716428}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6828","nonce":4271716428},{"type":"v1","addr":"172.21.15.134:6829","nonce":4271716428}]},"public_addr":"172.21.15.134:6825/4271716428","cluster_addr":"172.21.15.134:6827/4271716428","heartbeat_back_addr":"172.21.15.134:6831/4271716428","heartbeat_front_addr":"172.21.15.134:6829/4271716428","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T17:57:21.225125+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T17:57:55.197254+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T17:58:27.825136+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T17:59:04.319893+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T17:59:38.407752+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T18:00:12.118805+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T18:00:47.635214+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T18:01:21.231055+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T18:01:55.817861+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T18:02:31.680160+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T18:03:06.783398+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T18:03:41.307347+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.8","mappings":[{"from":2,"to":11}]},{"pgid":"2.1d","mappings":[{"from":8,"to":11}]},{"pgid":"3.3","mappings":[{"from":4,"to":9}]},{"pgid":"3.c","mappings":[{"from":3,"to":9}]},{"pgid":"3.21","mappings":[{"from":4,"to":8}]},{"pgid":"3.25","mappings":[{"from":5,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.22:0/3521924283":"2024-03-29T17:55:10.425504+0000","172.21.15.22:6801/730635939":"2024-03-29T17:55:10.425504+0000","172.21.15.22:0/2471229459":"2024-03-29T17:54:34.004559+0000","172.21.15.22:6800/3166828081":"2024-03-29T17:54:34.004559+0000","172.21.15.22:6800/730635939":"2024-03-29T17:55:10.425504+0000","172.21.15.22:0/1229047849":"2024-03-29T17:54:34.004559+0000","172.21.15.22:6801/3166828081":"2024-03-29T17:54:34.004559+0000","172.21.15.22:0/2717818856":"2024-03-29T17:55:10.425504+0000","172.21.15.22:0/2157570333":"2024-03-29T17:54:34.004559+0000","172.21.15.22:0/1006589530":"2024-03-29T17:55:10.425504+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-28T18:06:29.732 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:29.732+0000 7f9912400640 1 -- 172.21.15.22:0/71410541 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7f98f00a2880 msgr2=0x7f98f00a4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:06:29.732 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:29.732+0000 7f9912400640 1 --2- 172.21.15.22:0/71410541 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7f98f00a2880 0x7f98f00a4d40 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f98fc005fd0 tx=0x7f98fc005850 comp rx=0 tx=0).stop 2024-03-28T18:06:29.732 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:29.732+0000 7f9912400640 1 -- 172.21.15.22:0/71410541 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f990c1030b0 msgr2=0x7f990c1a0f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:06:29.732 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:29.732+0000 7f9912400640 1 --2- 172.21.15.22:0/71410541 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f990c1030b0 0x7f990c1a0f90 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f9904009bf0 tx=0x7f9904034cd0 comp rx=0 tx=0).stop 2024-03-28T18:06:29.732 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:29.732+0000 7f9912400640 1 -- 172.21.15.22:0/71410541 shutdown_connections 2024-03-28T18:06:29.732 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:29.732+0000 7f9912400640 1 --2- 172.21.15.22:0/71410541 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7f98f00a2880 0x7f98f00a4d40 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:06:29.732 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:29.732+0000 7f9912400640 1 --2- 172.21.15.22:0/71410541 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f990c1030b0 0x7f990c1a0f90 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:06:29.732 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:29.732+0000 7f9912400640 1 --2- 172.21.15.22:0/71410541 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f990c1026f0 0x7f990c199f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:06:29.732 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:29.732+0000 7f9912400640 1 --2- 172.21.15.22:0/71410541 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f990c101500 0x7f990c1999e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:06:29.733 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:29.732+0000 7f9912400640 1 -- 172.21.15.22:0/71410541 >> 172.21.15.22:0/71410541 conn(0x7f990c0fca70 msgr2=0x7f990c0738d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:06:29.733 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:29.733+0000 7f9912400640 1 -- 172.21.15.22:0/71410541 shutdown_connections 2024-03-28T18:06:29.733 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:29.733+0000 7f9912400640 1 -- 172.21.15.22:0/71410541 wait complete. 2024-03-28T18:06:29.753 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-28T18:06:29.970 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:29.970+0000 7fcc58a00640 1 -- 172.21.15.22:0/3756795962 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fcc54107b00 msgr2=0x7fcc54107f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:06:29.970 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:29.970+0000 7fcc58a00640 1 --2- 172.21.15.22:0/3756795962 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fcc54107b00 0x7fcc54107f20 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7fcc44009900 tx=0x7fcc4402bc30 comp rx=0 tx=0).stop 2024-03-28T18:06:29.970 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:29.970+0000 7fcc58a00640 1 -- 172.21.15.22:0/3756795962 shutdown_connections 2024-03-28T18:06:29.970 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:29.970+0000 7fcc58a00640 1 --2- 172.21.15.22:0/3756795962 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fcc541096b0 0x7fcc5410ff40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:06:29.970 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:29.970+0000 7fcc58a00640 1 --2- 172.21.15.22:0/3756795962 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcc54108cf0 0x7fcc54109170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:06:29.970 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:29.970+0000 7fcc58a00640 1 --2- 172.21.15.22:0/3756795962 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fcc54107b00 0x7fcc54107f20 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:06:29.970 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:29.970+0000 7fcc58a00640 1 -- 172.21.15.22:0/3756795962 >> 172.21.15.22:0/3756795962 conn(0x7fcc54076ab0 msgr2=0x7fcc54078ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:06:29.970 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:29.970+0000 7fcc58a00640 1 -- 172.21.15.22:0/3756795962 shutdown_connections 2024-03-28T18:06:29.971 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:29.970+0000 7fcc58a00640 1 -- 172.21.15.22:0/3756795962 wait complete. 2024-03-28T18:06:29.973 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:29.971+0000 7fcc58a00640 1 Processor -- start 2024-03-28T18:06:29.973 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:29.971+0000 7fcc58a00640 1 -- start start 2024-03-28T18:06:29.973 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:29.971+0000 7fcc58a00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcc54107b00 0x7fcc5419de30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:06:29.973 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:29.971+0000 7fcc58a00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fcc54108cf0 0x7fcc5419e370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:06:29.973 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:29.971+0000 7fcc58a00640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fcc541096b0 0x7fcc541a5390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:06:29.973 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:29.971+0000 7fcc58a00640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc54113760 con 0x7fcc54108cf0 2024-03-28T18:06:29.973 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:29.971+0000 7fcc58a00640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc541135e0 con 0x7fcc541096b0 2024-03-28T18:06:29.973 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:29.971+0000 7fcc58a00640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc541138e0 con 0x7fcc54107b00 2024-03-28T18:06:29.973 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:29.972+0000 7fcc53200640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fcc541096b0 0x7fcc541a5390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:06:29.973 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:29.972+0000 7fcc53200640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fcc541096b0 0x7fcc541a5390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.22:41338/0 (socket says 172.21.15.22:41338) 2024-03-28T18:06:29.973 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:29.972+0000 7fcc53200640 1 -- 172.21.15.22:0/4244182015 learned_addr learned my addr 172.21.15.22:0/4244182015 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-28T18:06:29.973 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:29.972+0000 7fcc4be00640 1 --2- 172.21.15.22:0/4244182015 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fcc54108cf0 0x7fcc5419e370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:06:29.974 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:29.972+0000 7fcc52800640 1 --2- 172.21.15.22:0/4244182015 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcc54107b00 0x7fcc5419de30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:06:29.974 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:29.972+0000 7fcc53200640 1 -- 172.21.15.22:0/4244182015 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcc54107b00 msgr2=0x7fcc5419de30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:06:29.974 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:29.972+0000 7fcc53200640 1 --2- 172.21.15.22:0/4244182015 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcc54107b00 0x7fcc5419de30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:06:29.974 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:29.972+0000 7fcc53200640 1 -- 172.21.15.22:0/4244182015 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fcc54108cf0 msgr2=0x7fcc5419e370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:06:29.974 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:29.972+0000 7fcc53200640 1 --2- 172.21.15.22:0/4244182015 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fcc54108cf0 0x7fcc5419e370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:06:29.974 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:29.972+0000 7fcc53200640 1 -- 172.21.15.22:0/4244182015 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcc541a5a00 con 0x7fcc541096b0 2024-03-28T18:06:29.974 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:29.972+0000 7fcc4be00640 1 --2- 172.21.15.22:0/4244182015 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fcc54108cf0 0x7fcc5419e370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-28T18:06:29.974 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:29.973+0000 7fcc53200640 1 --2- 172.21.15.22:0/4244182015 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fcc541096b0 0x7fcc541a5390 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fcc4c00eb10 tx=0x7fcc4c00c2d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:06:29.974 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:29.973+0000 7fcc4b400640 1 -- 172.21.15.22:0/4244182015 <== mon.2 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc4c004b20 con 0x7fcc541096b0 2024-03-28T18:06:29.974 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:29.973+0000 7fcc58a00640 1 -- 172.21.15.22:0/4244182015 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcc541a5cf0 con 0x7fcc541096b0 2024-03-28T18:06:29.974 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:29.973+0000 7fcc4b400640 1 -- 172.21.15.22:0/4244182015 <== mon.2 v2:172.21.15.79:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7fcc4c004cc0 con 0x7fcc541096b0 2024-03-28T18:06:29.974 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:29.973+0000 7fcc4b400640 1 -- 172.21.15.22:0/4244182015 <== mon.2 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc4c010700 con 0x7fcc541096b0 2024-03-28T18:06:29.974 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:29.973+0000 7fcc58a00640 1 -- 172.21.15.22:0/4244182015 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcc541a61e0 con 0x7fcc541096b0 2024-03-28T18:06:29.975 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:29.974+0000 7fcc58a00640 1 -- 172.21.15.22:0/4244182015 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcc5410ee20 con 0x7fcc541096b0 2024-03-28T18:06:29.976 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:29.977+0000 7fcc4b400640 1 -- 172.21.15.22:0/4244182015 <== mon.2 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 18) v1 ==== 137290+0+0 (secure 0 0 0) 0x7fcc4c004e70 con 0x7fcc541096b0 2024-03-28T18:06:29.977 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:29.978+0000 7fcc4b400640 1 --2- 172.21.15.22:0/4244182015 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7fcc380a2800 0x7fcc380a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:06:29.977 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:29.978+0000 7fcc4b400640 1 -- 172.21.15.22:0/4244182015 <== mon.2 v2:172.21.15.79:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 8153+0+0 (secure 0 0 0) 0x7fcc4c0cf2c0 con 0x7fcc541096b0 2024-03-28T18:06:29.978 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:29.978+0000 7fcc52800640 1 --2- 172.21.15.22:0/4244182015 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7fcc380a2800 0x7fcc380a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:06:29.978 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:29.979+0000 7fcc52800640 1 --2- 172.21.15.22:0/4244182015 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7fcc380a2800 0x7fcc380a4cc0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fcc44006010 tx=0x7fcc44000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:06:29.989 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:29.989+0000 7fcc4b400640 1 -- 172.21.15.22:0/4244182015 <== mon.2 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fcc4c053740 con 0x7fcc541096b0 2024-03-28T18:06:30.237 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:30.237+0000 7fcc58a00640 1 -- 172.21.15.22:0/4244182015 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fcc5402d070 con 0x7fcc541096b0 2024-03-28T18:06:30.238 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:30.239+0000 7fcc4b400640 1 -- 172.21.15.22:0/4244182015 <== mon.2 v2:172.21.15.79:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) v1 ==== 94+0+11452 (secure 0 0 0) 0x7fcc4c09c360 con 0x7fcc541096b0 2024-03-28T18:06:30.239 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-03-28T18:06:30.239 INFO:teuthology.orchestra.run.smithi022.stdout:{"epoch":26,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14544,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6835/3718238950","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6834","nonce":3718238950},{"type":"v1","addr":"172.21.15.22:6835","nonce":3718238950}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14562,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6837/3676939386","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6836","nonce":3676939386},{"type":"v1","addr":"172.21.15.22:6837","nonce":3676939386}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":26,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-28T18:04:48.991622+0000","modified":"2024-03-28T18:04:56.643864+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":34553,"mds_1":34511,"mds_2":34475,"mds_3":34442,"mds_4":24502},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14640":{"gid":14640,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.22:6839/3979998310","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6838","nonce":3979998310},{"type":"v1","addr":"172.21.15.22:6839","nonce":3979998310}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_14655":{"gid":14655,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.22:6841/3069056661","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6840","nonce":3069056661},{"type":"v1","addr":"172.21.15.22:6841","nonce":3069056661}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24412":{"gid":24412,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.134:6833/3176125362","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6832","nonce":3176125362},{"type":"v1","addr":"172.21.15.134:6833","nonce":3176125362}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24463":{"gid":24463,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.134:6835/3370549207","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6834","nonce":3370549207},{"type":"v1","addr":"172.21.15.134:6835","nonce":3370549207}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24478":{"gid":24478,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.134:6837/899825581","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6836","nonce":899825581},{"type":"v1","addr":"172.21.15.134:6837","nonce":899825581}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24502":{"gid":24502,"name":"i","rank":4,"incarnation":24,"state":"up:active","state_seq":7,"addr":"172.21.15.134:6839/3400263217","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6838","nonce":3400263217},{"type":"v1","addr":"172.21.15.134:6839","nonce":3400263217}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_34442":{"gid":34442,"name":"k","rank":3,"incarnation":22,"state":"up:active","state_seq":14,"addr":"172.21.15.79:6833/856896997","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6832","nonce":856896997},{"type":"v1","addr":"172.21.15.79:6833","nonce":856896997}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_34475":{"gid":34475,"name":"e","rank":2,"incarnation":20,"state":"up:active","state_seq":11,"addr":"172.21.15.79:6835/3365773431","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6834","nonce":3365773431},{"type":"v1","addr":"172.21.15.79:6835","nonce":3365773431}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_34511":{"gid":34511,"name":"h","rank":1,"incarnation":18,"state":"up:active","state_seq":8,"addr":"172.21.15.79:6837/1787903366","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6836","nonce":1787903366},{"type":"v1","addr":"172.21.15.79:6837","nonce":1787903366}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_34553":{"gid":34553,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.79:6839/2441542443","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6838","nonce":2441542443},{"type":"v1","addr":"172.21.15.79:6839","nonce":2441542443}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":34553,"qdb_cluster":[34511,34553,34475,34442,24502]},"id":1}]} 2024-03-28T18:06:30.239 INFO:teuthology.orchestra.run.smithi022.stderr:dumped fsmap epoch 26 2024-03-28T18:06:30.241 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:30.241+0000 7fcc58a00640 1 -- 172.21.15.22:0/4244182015 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7fcc380a2800 msgr2=0x7fcc380a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:06:30.242 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:30.241+0000 7fcc58a00640 1 --2- 172.21.15.22:0/4244182015 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7fcc380a2800 0x7fcc380a4cc0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fcc44006010 tx=0x7fcc44000bc0 comp rx=0 tx=0).stop 2024-03-28T18:06:30.242 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:30.241+0000 7fcc58a00640 1 -- 172.21.15.22:0/4244182015 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fcc541096b0 msgr2=0x7fcc541a5390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:06:30.242 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:30.241+0000 7fcc58a00640 1 --2- 172.21.15.22:0/4244182015 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fcc541096b0 0x7fcc541a5390 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fcc4c00eb10 tx=0x7fcc4c00c2d0 comp rx=0 tx=0).stop 2024-03-28T18:06:30.242 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:30.242+0000 7fcc58a00640 1 -- 172.21.15.22:0/4244182015 shutdown_connections 2024-03-28T18:06:30.242 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:30.242+0000 7fcc58a00640 1 --2- 172.21.15.22:0/4244182015 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7fcc380a2800 0x7fcc380a4cc0 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:06:30.242 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:30.242+0000 7fcc58a00640 1 --2- 172.21.15.22:0/4244182015 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fcc541096b0 0x7fcc541a5390 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:06:30.242 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:30.242+0000 7fcc58a00640 1 --2- 172.21.15.22:0/4244182015 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fcc54108cf0 0x7fcc5419e370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:06:30.242 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:30.242+0000 7fcc58a00640 1 --2- 172.21.15.22:0/4244182015 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcc54107b00 0x7fcc5419de30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:06:30.242 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:30.242+0000 7fcc58a00640 1 -- 172.21.15.22:0/4244182015 >> 172.21.15.22:0/4244182015 conn(0x7fcc54076ab0 msgr2=0x7fcc5410bff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:06:30.242 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:30.242+0000 7fcc58a00640 1 -- 172.21.15.22:0/4244182015 shutdown_connections 2024-03-28T18:06:30.242 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:30.242+0000 7fcc58a00640 1 -- 172.21.15.22:0/4244182015 wait complete. 2024-03-28T18:06:30.263 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 26, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-03-28T18:04:48.991622+0000', 'modified': '2024-03-28T18:04:56.643864+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 5, 'in': [0, 1, 2, 3, 4], 'up': {'mds_0': 34553, 'mds_1': 34511, 'mds_2': 34475, 'mds_3': 34442, 'mds_4': 24502}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_14640': {'gid': 14640, 'name': 'g', 'rank': 4, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.22:6839/3979998310', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.22:6838', 'nonce': 3979998310}, {'type': 'v1', 'addr': '172.21.15.22:6839', 'nonce': 3979998310}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_14655': {'gid': 14655, 'name': 'a', 'rank': 3, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.22:6841/3069056661', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.22:6840', 'nonce': 3069056661}, {'type': 'v1', 'addr': '172.21.15.22:6841', 'nonce': 3069056661}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_24412': {'gid': 24412, 'name': 'l', 'rank': 2, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.134:6833/3176125362', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.134:6832', 'nonce': 3176125362}, {'type': 'v1', 'addr': '172.21.15.134:6833', 'nonce': 3176125362}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_24463': {'gid': 24463, 'name': 'f', 'rank': 1, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.134:6835/3370549207', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.134:6834', 'nonce': 3370549207}, {'type': 'v1', 'addr': '172.21.15.134:6835', 'nonce': 3370549207}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_24478': {'gid': 24478, 'name': 'c', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.134:6837/899825581', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.134:6836', 'nonce': 899825581}, {'type': 'v1', 'addr': '172.21.15.134:6837', 'nonce': 899825581}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_24502': {'gid': 24502, 'name': 'i', 'rank': 4, 'incarnation': 24, 'state': 'up:active', 'state_seq': 7, 'addr': '172.21.15.134:6839/3400263217', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.134:6838', 'nonce': 3400263217}, {'type': 'v1', 'addr': '172.21.15.134:6839', 'nonce': 3400263217}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_34442': {'gid': 34442, 'name': 'k', 'rank': 3, 'incarnation': 22, 'state': 'up:active', 'state_seq': 14, 'addr': '172.21.15.79:6833/856896997', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.79:6832', 'nonce': 856896997}, {'type': 'v1', 'addr': '172.21.15.79:6833', 'nonce': 856896997}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_34475': {'gid': 34475, 'name': 'e', 'rank': 2, 'incarnation': 20, 'state': 'up:active', 'state_seq': 11, 'addr': '172.21.15.79:6835/3365773431', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.79:6834', 'nonce': 3365773431}, {'type': 'v1', 'addr': '172.21.15.79:6835', 'nonce': 3365773431}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_34511': {'gid': 34511, 'name': 'h', 'rank': 1, 'incarnation': 18, 'state': 'up:active', 'state_seq': 8, 'addr': '172.21.15.79:6837/1787903366', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.79:6836', 'nonce': 1787903366}, {'type': 'v1', 'addr': '172.21.15.79:6837', 'nonce': 1787903366}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_34553': {'gid': 34553, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.79:6839/2441542443', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.79:6838', 'nonce': 2441542443}, {'type': 'v1', 'addr': '172.21.15.79:6839', 'nonce': 2441542443}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [3], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 34553, 'qdb_cluster': [34511, 34553, 34475, 34442, 24502]} 2024-03-28T18:06:30.263 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 5/5 2024-03-28T18:06:30.263 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.i status 2024-03-28T18:06:30.351 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:06:30 smithi022 ceph-mon[31985]: pgmap v421: 97 pgs: 97 active+clean; 643 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-03-28T18:06:30.351 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:06:30 smithi022 ceph-mon[31985]: from='client.? 172.21.15.22:0/4241701771' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-28T18:06:30.351 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:06:30 smithi022 ceph-mon[31985]: from='client.? 172.21.15.22:0/71410541' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-28T18:06:30.425 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:06:30 smithi134 ceph-mon[35037]: pgmap v421: 97 pgs: 97 active+clean; 643 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-03-28T18:06:30.425 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:06:30 smithi134 ceph-mon[35037]: from='client.? 172.21.15.22:0/4241701771' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-28T18:06:30.425 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:06:30 smithi134 ceph-mon[35037]: from='client.? 172.21.15.22:0/71410541' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-28T18:06:30.432 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:06:30 smithi079 ceph-mon[35312]: pgmap v421: 97 pgs: 97 active+clean; 643 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-03-28T18:06:30.432 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:06:30 smithi079 ceph-mon[35312]: from='client.? 172.21.15.22:0/4241701771' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-28T18:06:30.433 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:06:30 smithi079 ceph-mon[35312]: from='client.? 172.21.15.22:0/71410541' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-28T18:06:30.493 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:30.493+0000 7f25b7200640 1 -- 172.21.15.22:0/1274370871 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f25b8102760 msgr2=0x7f25b8104bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:06:30.494 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:30.493+0000 7f25b7200640 1 --2- 172.21.15.22:0/1274370871 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f25b8102760 0x7f25b8104bd0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f25a0009900 tx=0x7f25a002bc30 comp rx=0 tx=0).stop 2024-03-28T18:06:30.494 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:30.494+0000 7f25b7200640 1 -- 172.21.15.22:0/1274370871 shutdown_connections 2024-03-28T18:06:30.494 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:30.494+0000 7f25b7200640 1 --2- 172.21.15.22:0/1274370871 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f25b8105110 0x7f25b8105590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:06:30.494 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:30.494+0000 7f25b7200640 1 --2- 172.21.15.22:0/1274370871 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f25b8102760 0x7f25b8104bd0 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:06:30.494 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:30.494+0000 7f25b7200640 1 --2- 172.21.15.22:0/1274370871 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f25b80ffdf0 0x7f25b8102220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:06:30.494 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:30.494+0000 7f25b7200640 1 -- 172.21.15.22:0/1274370871 >> 172.21.15.22:0/1274370871 conn(0x7f25b80fba10 msgr2=0x7f25b80fde30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:06:30.494 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:30.494+0000 7f25b7200640 1 -- 172.21.15.22:0/1274370871 shutdown_connections 2024-03-28T18:06:30.494 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:30.494+0000 7f25b7200640 1 -- 172.21.15.22:0/1274370871 wait complete. 2024-03-28T18:06:30.496 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:30.495+0000 7f25b7200640 1 Processor -- start 2024-03-28T18:06:30.496 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:30.495+0000 7f25b7200640 1 -- start start 2024-03-28T18:06:30.496 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:30.495+0000 7f25b7200640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f25b80ffdf0 0x7f25b819de90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:06:30.496 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:30.495+0000 7f25b7200640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f25b8102760 0x7f25b819e3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:06:30.496 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:30.495+0000 7f25b7200640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f25b8105110 0x7f25b81a5440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:06:30.496 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:30.495+0000 7f25b7200640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25b8113840 con 0x7f25b80ffdf0 2024-03-28T18:06:30.496 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:30.495+0000 7f25b7200640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25b81136c0 con 0x7f25b8102760 2024-03-28T18:06:30.496 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:30.495+0000 7f25b7200640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25b81139c0 con 0x7f25b8105110 2024-03-28T18:06:30.496 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:30.496+0000 7f25b5e00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f25b80ffdf0 0x7f25b819de90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:06:30.496 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:30.496+0000 7f25b5e00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f25b80ffdf0 0x7f25b819de90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:49080/0 (socket says 172.21.15.22:49080) 2024-03-28T18:06:30.496 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:30.496+0000 7f25b5e00640 1 -- 172.21.15.22:0/143257529 learned_addr learned my addr 172.21.15.22:0/143257529 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-28T18:06:30.496 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:30.496+0000 7f25ad400640 1 --2- 172.21.15.22:0/143257529 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f25b8102760 0x7f25b819e3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:06:30.497 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:30.496+0000 7f25b6800640 1 --2- 172.21.15.22:0/143257529 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f25b8105110 0x7f25b81a5440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:06:30.497 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:30.496+0000 7f25b5e00640 1 -- 172.21.15.22:0/143257529 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f25b8105110 msgr2=0x7f25b81a5440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:06:30.497 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:30.496+0000 7f25b5e00640 1 --2- 172.21.15.22:0/143257529 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f25b8105110 0x7f25b81a5440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:06:30.497 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:30.496+0000 7f25b5e00640 1 -- 172.21.15.22:0/143257529 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f25b8102760 msgr2=0x7f25b819e3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:06:30.497 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:30.496+0000 7f25b5e00640 1 --2- 172.21.15.22:0/143257529 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f25b8102760 0x7f25b819e3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:06:30.497 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:30.496+0000 7f25b5e00640 1 -- 172.21.15.22:0/143257529 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f25b81a5ab0 con 0x7f25b80ffdf0 2024-03-28T18:06:30.497 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:30.496+0000 7f25b5e00640 1 --2- 172.21.15.22:0/143257529 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f25b80ffdf0 0x7f25b819de90 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f25a800d770 tx=0x7f25a800dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:06:30.498 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:30.497+0000 7f25af400640 1 -- 172.21.15.22:0/143257529 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25a8017070 con 0x7f25b80ffdf0 2024-03-28T18:06:30.498 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:30.497+0000 7f25af400640 1 -- 172.21.15.22:0/143257529 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f25a80054f0 con 0x7f25b80ffdf0 2024-03-28T18:06:30.498 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:30.497+0000 7f25af400640 1 -- 172.21.15.22:0/143257529 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25a8004d60 con 0x7f25b80ffdf0 2024-03-28T18:06:30.498 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:30.497+0000 7f25b7200640 1 -- 172.21.15.22:0/143257529 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f25b81a5da0 con 0x7f25b80ffdf0 2024-03-28T18:06:30.498 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:30.497+0000 7f25b7200640 1 -- 172.21.15.22:0/143257529 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f25b81a61b0 con 0x7f25b80ffdf0 2024-03-28T18:06:30.500 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:30.501+0000 7f25bf9ef740 1 Processor -- start 2024-03-28T18:06:30.501 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:30.501+0000 7f25bf9ef740 1 -- start start 2024-03-28T18:06:30.501 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:30.501+0000 7f25af400640 1 -- 172.21.15.22:0/143257529 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 18) v1 ==== 137290+0+0 (secure 0 0 0) 0x7f25a8004f00 con 0x7f25b80ffdf0 2024-03-28T18:06:30.502 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:30.502+0000 7f25bf9ef740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d4614ead80 0x55d4614eb1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:06:30.502 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:30.502+0000 7f25bf9ef740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d4614eb770 0x55d4614f0270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:06:30.502 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:30.502+0000 7f25bf9ef740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55d4614f08e0 0x55d4614f2d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:06:30.502 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:30.502+0000 7f25bf9ef740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d46144fc90 con 0x55d4614eb770 2024-03-28T18:06:30.502 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:30.502+0000 7f25bf9ef740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d4613f8980 con 0x55d4614f08e0 2024-03-28T18:06:30.502 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:30.502+0000 7f25bf9ef740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d46144bd30 con 0x55d4614ead80 2024-03-28T18:06:30.502 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:30.502+0000 7f25b5e00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d4614eb770 0x55d4614f0270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:06:30.502 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:30.502+0000 7f25ad400640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55d4614f08e0 0x55d4614f2d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:06:30.502 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:30.502+0000 7f25b5e00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d4614eb770 0x55d4614f0270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:49084/0 (socket says 172.21.15.22:49084) 2024-03-28T18:06:30.502 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:30.502+0000 7f25b5e00640 1 -- 172.21.15.22:0/1479313382 learned_addr learned my addr 172.21.15.22:0/1479313382 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-28T18:06:30.502 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:30.502+0000 7f25b6800640 1 --2- 172.21.15.22:0/1479313382 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d4614ead80 0x55d4614eb1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:06:30.504 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:30.502+0000 7f25b5e00640 1 -- 172.21.15.22:0/1479313382 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d4614ead80 msgr2=0x55d4614eb1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T18:06:30.504 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:30.502+0000 7f25b5e00640 1 --2- 172.21.15.22:0/1479313382 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d4614ead80 0x55d4614eb1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:06:30.504 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:30.502+0000 7f25b5e00640 1 -- 172.21.15.22:0/1479313382 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55d4614f08e0 msgr2=0x55d4614f2d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T18:06:30.504 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:30.502+0000 7f25b5e00640 1 --2- 172.21.15.22:0/1479313382 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55d4614f08e0 0x55d4614f2d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:06:30.505 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:30.502+0000 7f25b5e00640 1 -- 172.21.15.22:0/1479313382 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d461410510 con 0x55d4614eb770 2024-03-28T18:06:30.505 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:30.502+0000 7f25ad400640 1 --2- 172.21.15.22:0/1479313382 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55d4614f08e0 0x55d4614f2d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-28T18:06:30.505 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:30.503+0000 7f25b5e00640 1 --2- 172.21.15.22:0/1479313382 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d4614eb770 0x55d4614f0270 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f25a80d65d0 tx=0x7f25a80d6a90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=398a1251c94a0006 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:06:30.505 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:30.503+0000 7f25a7000640 1 -- 172.21.15.22:0/1479313382 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25a80e2070 con 0x55d4614eb770 2024-03-28T18:06:30.505 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:30.503+0000 7f25a7000640 1 -- 172.21.15.22:0/1479313382 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f25a80cecd0 con 0x55d4614eb770 2024-03-28T18:06:30.505 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:30.503+0000 7f25a7000640 1 -- 172.21.15.22:0/1479313382 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25a80d4bd0 con 0x55d4614eb770 2024-03-28T18:06:30.505 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:30.503+0000 7f25af400640 1 --2- 172.21.15.22:0/143257529 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7f259c0a2800 0x7f259c0a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:06:30.505 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:30.503+0000 7f25af400640 1 -- 172.21.15.22:0/143257529 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 8153+0+0 (secure 0 0 0) 0x7f25a80ce4d0 con 0x7f25b80ffdf0 2024-03-28T18:06:30.505 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:30.503+0000 7f25b6800640 1 --2- 172.21.15.22:0/143257529 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7f259c0a2800 0x7f259c0a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:06:30.505 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:30.504+0000 7f25bf9ef740 1 -- 172.21.15.22:0/1479313382 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d4614eb770 msgr2=0x55d4614f0270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:06:30.505 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:30.504+0000 7f25bf9ef740 1 --2- 172.21.15.22:0/1479313382 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d4614eb770 0x55d4614f0270 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f25a80d65d0 tx=0x7f25a80d6a90 comp rx=0 tx=0).stop 2024-03-28T18:06:30.505 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:30.504+0000 7f25bf9ef740 1 -- 172.21.15.22:0/1479313382 shutdown_connections 2024-03-28T18:06:30.505 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:30.504+0000 7f25bf9ef740 1 --2- 172.21.15.22:0/1479313382 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55d4614f08e0 0x55d4614f2d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:06:30.505 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:30.504+0000 7f25bf9ef740 1 --2- 172.21.15.22:0/1479313382 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d4614eb770 0x55d4614f0270 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:06:30.506 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:30.504+0000 7f25bf9ef740 1 --2- 172.21.15.22:0/1479313382 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d4614ead80 0x55d4614eb1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:06:30.506 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:30.504+0000 7f25bf9ef740 1 -- 172.21.15.22:0/1479313382 >> 172.21.15.22:0/1479313382 conn(0x55d4614e62f0 msgr2=0x55d4614e66f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:06:30.506 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:30.504+0000 7f25bf9ef740 1 -- 172.21.15.22:0/1479313382 shutdown_connections 2024-03-28T18:06:30.506 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:30.504+0000 7f25b6800640 1 --2- 172.21.15.22:0/143257529 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7f259c0a2800 0x7f259c0a4cc0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x55d4614ebfd0 tx=0x7f25b0009500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:06:30.506 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:30.504+0000 7f25bf9ef740 1 -- 172.21.15.22:0/1479313382 wait complete. 2024-03-28T18:06:30.506 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:30.505+0000 7f25bf9ef740 1 -- start start 2024-03-28T18:06:30.506 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:30.505+0000 7f25bf9ef740 1 Processor -- start 2024-03-28T18:06:30.509 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:30.505+0000 7f25bf9ef740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d4614ead80 0x55d4614f52a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:06:30.509 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:30.505+0000 7f25bf9ef740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55d4614eb770 0x55d4614f57e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:06:30.509 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:30.505+0000 7f25bf9ef740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d4614f08e0 0x55d4614fc820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:06:30.509 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:30.505+0000 7f25bf9ef740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d4614f3890 con 0x55d4614ead80 2024-03-28T18:06:30.509 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:30.505+0000 7f25bf9ef740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d46140bdd0 con 0x55d4614eb770 2024-03-28T18:06:30.509 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:30.505+0000 7f25bf9ef740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d4614f3a10 con 0x55d4614f08e0 2024-03-28T18:06:30.509 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:30.506+0000 7f25bf9ef740 10 client.0 _ll_register_callbacks cb 0x55d461163ac0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-28T18:06:30.510 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:30.506+0000 7f25b5e00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d4614ead80 0x55d4614f52a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:06:30.510 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:30.506+0000 7f25b5e00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d4614ead80 0x55d4614f52a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:49086/0 (socket says 172.21.15.22:49086) 2024-03-28T18:06:30.510 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:30.506+0000 7f25b5e00640 1 -- 172.21.15.22:0/2313872938 learned_addr learned my addr 172.21.15.22:0/2313872938 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-28T18:06:30.510 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:30.506+0000 7f25ad400640 1 --2- 172.21.15.22:0/2313872938 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55d4614eb770 0x55d4614f57e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:06:30.510 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:30.506+0000 7f25b6800640 1 --2- 172.21.15.22:0/2313872938 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d4614f08e0 0x55d4614fc820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:06:30.510 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:30.506+0000 7f25b5e00640 1 -- 172.21.15.22:0/2313872938 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d4614f08e0 msgr2=0x55d4614fc820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T18:06:30.510 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:30.506+0000 7f25b5e00640 1 --2- 172.21.15.22:0/2313872938 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d4614f08e0 0x55d4614fc820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:06:30.510 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:30.506+0000 7f25b5e00640 1 -- 172.21.15.22:0/2313872938 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55d4614eb770 msgr2=0x55d4614f57e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T18:06:30.510 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:30.506+0000 7f25b5e00640 1 --2- 172.21.15.22:0/2313872938 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55d4614eb770 0x55d4614f57e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:06:30.510 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:30.506+0000 7f25b5e00640 1 -- 172.21.15.22:0/2313872938 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55d4611642d0 con 0x55d4614ead80 2024-03-28T18:06:30.510 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:30.506+0000 7f25b6800640 1 --2- 172.21.15.22:0/2313872938 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d4614f08e0 0x55d4614fc820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T18:06:30.510 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:30.506+0000 7f25b5e00640 1 --2- 172.21.15.22:0/2313872938 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d4614ead80 0x55d4614f52a0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f25a8097840 tx=0x7f25a8009eb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=def1d325b0f6f029 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:06:30.510 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:30.506+0000 7f259b400640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-03-28T18:06:30.511 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:30.507+0000 7f259b400640 1 -- 172.21.15.22:0/2313872938 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25a8020030 con 0x55d4614ead80 2024-03-28T18:06:30.511 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:30.507+0000 7f259b400640 1 -- 172.21.15.22:0/2313872938 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 8153+0+0 (secure 0 0 0) 0x7f25a80cc0e0 con 0x55d4614ead80 2024-03-28T18:06:30.511 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:30.507+0000 7f25bf9ef740 1 -- 172.21.15.22:0/2313872938 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f25a80cecd0 con 0x55d4614ead80 2024-03-28T18:06:30.511 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:30.507+0000 7f259b400640 1 -- 172.21.15.22:0/2313872938 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7f25a8097110 con 0x55d4614ead80 2024-03-28T18:06:30.511 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:30.507+0000 7f25bf9ef740 10 client.15072 fetch_fsmap learned FSMap version 26 2024-03-28T18:06:30.511 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:30.507+0000 7f25bf9ef740 1 -- 172.21.15.22:0/2313872938 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x55d46122e6b0 con 0x55d4614ead80 2024-03-28T18:06:30.511 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:30.507+0000 7f259b400640 1 -- 172.21.15.22:0/2313872938 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f25a80cecd0 con 0x55d4614ead80 2024-03-28T18:06:30.511 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:30.507+0000 7f259b400640 1 -- 172.21.15.22:0/2313872938 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 26) v1 ==== 7934+0+0 (secure 0 0 0) 0x7f25a80974e0 con 0x55d4614ead80 2024-03-28T18:06:30.511 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:30.507+0000 7f25bf9ef740 10 client.15072 fetch_fsmap finished waiting for FSMap version 26 2024-03-28T18:06:30.511 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:30.507+0000 7f25bf9ef740 10 client.15072 resolve_mds: resolved name 'i' to daemon mds.i 2024-03-28T18:06:30.511 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:30.507+0000 7f25bf9ef740 20 client.15072 populate_metadata read hostname 'smithi022' 2024-03-28T18:06:30.511 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:30.507+0000 7f25bf9ef740 1 --2- 172.21.15.22:0/2313872938 >> [v2:172.21.15.134:6838/3400263217,v1:172.21.15.134:6839/3400263217] conn(0x55d4615013e0 0x55d461505860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:06:30.511 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:30.507+0000 7f25bf9ef740 4 client.15072 mds_command: new command op to 24502 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-28T18:06:30.511 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:30.507+0000 7f25bf9ef740 1 -- 172.21.15.22:0/2313872938 --> [v2:172.21.15.134:6838/3400263217,v1:172.21.15.134:6839/3400263217] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55d4613f1520 con 0x55d4615013e0 2024-03-28T18:06:30.511 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:30.508+0000 7f25ad400640 1 --2- 172.21.15.22:0/2313872938 >> [v2:172.21.15.134:6838/3400263217,v1:172.21.15.134:6839/3400263217] conn(0x55d4615013e0 0x55d461505860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:06:30.512 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:30.509+0000 7f25ad400640 1 --2- 172.21.15.22:0/2313872938 >> [v2:172.21.15.134:6838/3400263217,v1:172.21.15.134:6839/3400263217] conn(0x55d4615013e0 0x55d461505860 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=85cf9f126c6cc7a6 server_cookie=86ad4f9b1dd7a97e in_seq=0 out_seq=0 2024-03-28T18:06:30.512 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:30.509+0000 7f259b400640 10 client.15072 ms_handle_connect on v2:172.21.15.134:6838/3400263217 2024-03-28T18:06:30.513 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:30.511+0000 7f259b400640 1 -- 172.21.15.22:0/2313872938 <== mds.4 v2:172.21.15.134:6838/3400263217 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x55d4613f1520 con 0x55d4615013e0 2024-03-28T18:06:30.513 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:30.511+0000 7f259b400640 10 client.15072 handle_command_reply: tid=0 2024-03-28T18:06:30.513 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:30.511+0000 7f25bf9ef740 1 client.15072 shutdown 2024-03-28T18:06:30.513 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:30.511+0000 7f25bf9ef740 1 -- 172.21.15.22:0/2313872938 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d4614ead80 msgr2=0x55d4614f52a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:06:30.513 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:30.511+0000 7f25bf9ef740 1 --2- 172.21.15.22:0/2313872938 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d4614ead80 0x55d4614f52a0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f25a8097840 tx=0x7f25a8009eb0 comp rx=0 tx=0).stop 2024-03-28T18:06:30.513 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:30.511+0000 7f25bf9ef740 1 -- 172.21.15.22:0/2313872938 shutdown_connections 2024-03-28T18:06:30.513 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:30.511+0000 7f25bf9ef740 1 --2- 172.21.15.22:0/2313872938 >> [v2:172.21.15.134:6838/3400263217,v1:172.21.15.134:6839/3400263217] conn(0x55d4615013e0 0x55d461505860 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:06:30.513 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:30.511+0000 7f25bf9ef740 1 --2- 172.21.15.22:0/2313872938 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d4614f08e0 0x55d4614fc820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:06:30.513 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:30.511+0000 7f25bf9ef740 1 --2- 172.21.15.22:0/2313872938 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55d4614eb770 0x55d4614f57e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:06:30.513 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:30.511+0000 7f25bf9ef740 1 --2- 172.21.15.22:0/2313872938 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d4614ead80 0x55d4614f52a0 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:06:30.513 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:30.511+0000 7f259b400640 0 client.15072 ms_handle_reset on v2:172.21.15.134:6838/3400263217 2024-03-28T18:06:30.514 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:30.511+0000 7f25bf9ef740 1 -- 172.21.15.22:0/2313872938 >> 172.21.15.22:0/2313872938 conn(0x55d4614e62f0 msgr2=0x55d4614e54f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:06:30.514 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:30.511+0000 7f25bf9ef740 1 -- 172.21.15.22:0/2313872938 shutdown_connections 2024-03-28T18:06:30.514 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:30.511+0000 7f25bf9ef740 1 -- 172.21.15.22:0/2313872938 wait complete. 2024-03-28T18:06:30.514 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:30.511+0000 7f25bf9ef740 20 client.15072 trim_cache size 0 max 16384 2024-03-28T18:06:30.530 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:30.530+0000 7f25bf9ef740 1 Processor -- start 2024-03-28T18:06:30.530 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:30.530+0000 7f25bf9ef740 1 -- start start 2024-03-28T18:06:30.530 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:30.530+0000 7f25bf9ef740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55d4614ead80 0x55d4614f4eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:06:30.530 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:30.530+0000 7f25bf9ef740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d4614eb770 0x55d4614fc7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:06:30.530 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:30.530+0000 7f25bf9ef740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d4614f08e0 0x55d4614f53f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:06:30.530 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:30.530+0000 7f25bf9ef740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d461500c00 con 0x55d4614f08e0 2024-03-28T18:06:30.530 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:30.530+0000 7f25bf9ef740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d461501260 con 0x55d4614ead80 2024-03-28T18:06:30.530 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:30.530+0000 7f25bf9ef740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d4615018c0 con 0x55d4614eb770 2024-03-28T18:06:30.531 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:30.531+0000 7f25b5e00640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55d4614ead80 0x55d4614f4eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:06:30.531 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:30.531+0000 7f25b5e00640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55d4614ead80 0x55d4614f4eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.22:41390/0 (socket says 172.21.15.22:41390) 2024-03-28T18:06:30.532 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:30.531+0000 7f25b5e00640 1 -- 172.21.15.22:0/552727142 learned_addr learned my addr 172.21.15.22:0/552727142 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-28T18:06:30.532 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:30.531+0000 7f25b6800640 1 --2- 172.21.15.22:0/552727142 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d4614f08e0 0x55d4614f53f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:06:30.532 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:30.531+0000 7f25ad400640 1 --2- 172.21.15.22:0/552727142 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d4614eb770 0x55d4614fc7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:06:30.532 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:30.531+0000 7f25b5e00640 1 -- 172.21.15.22:0/552727142 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d4614eb770 msgr2=0x55d4614fc7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T18:06:30.532 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:30.531+0000 7f25b5e00640 1 --2- 172.21.15.22:0/552727142 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d4614eb770 0x55d4614fc7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:06:30.532 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:30.531+0000 7f25b5e00640 1 -- 172.21.15.22:0/552727142 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d4614f08e0 msgr2=0x55d4614f53f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T18:06:30.532 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:30.531+0000 7f25b5e00640 1 --2- 172.21.15.22:0/552727142 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d4614f08e0 0x55d4614f53f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:06:30.532 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:30.531+0000 7f25b5e00640 1 -- 172.21.15.22:0/552727142 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d4614d8170 con 0x55d4614ead80 2024-03-28T18:06:30.532 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:30.531+0000 7f25b5e00640 1 --2- 172.21.15.22:0/552727142 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55d4614ead80 0x55d4614f4eb0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f25a809ab70 tx=0x7f25a8099380 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e32c6f9067f2aac8 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:06:30.532 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:30.532+0000 7f25a6600640 1 -- 172.21.15.22:0/552727142 <== mon.2 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25a80b7070 con 0x55d4614ead80 2024-03-28T18:06:30.532 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:30.532+0000 7f25a6600640 1 -- 172.21.15.22:0/552727142 <== mon.2 v2:172.21.15.79:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f25a80d61c0 con 0x55d4614ead80 2024-03-28T18:06:30.532 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:30.532+0000 7f25a6600640 1 -- 172.21.15.22:0/552727142 <== mon.2 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25a8020070 con 0x55d4614ead80 2024-03-28T18:06:30.534 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:30.532+0000 7f25bf9ef740 1 -- 172.21.15.22:0/552727142 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55d4614ead80 msgr2=0x55d4614f4eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:06:30.534 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:30.532+0000 7f25bf9ef740 1 --2- 172.21.15.22:0/552727142 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55d4614ead80 0x55d4614f4eb0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f25a809ab70 tx=0x7f25a8099380 comp rx=0 tx=0).stop 2024-03-28T18:06:30.534 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:30.532+0000 7f25bf9ef740 1 -- 172.21.15.22:0/552727142 shutdown_connections 2024-03-28T18:06:30.534 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:30.532+0000 7f25bf9ef740 1 --2- 172.21.15.22:0/552727142 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d4614f08e0 0x55d4614f53f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:06:30.534 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:30.532+0000 7f25bf9ef740 1 --2- 172.21.15.22:0/552727142 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d4614eb770 0x55d4614fc7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:06:30.534 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:30.532+0000 7f25bf9ef740 1 --2- 172.21.15.22:0/552727142 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55d4614ead80 0x55d4614f4eb0 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:06:30.534 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:30.532+0000 7f25bf9ef740 1 -- 172.21.15.22:0/552727142 >> 172.21.15.22:0/552727142 conn(0x55d4614e62f0 msgr2=0x55d4614e66d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:06:30.534 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:30.533+0000 7f25bf9ef740 1 -- 172.21.15.22:0/552727142 shutdown_connections 2024-03-28T18:06:30.534 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:30.533+0000 7f25bf9ef740 1 -- 172.21.15.22:0/552727142 wait complete. 2024-03-28T18:06:30.534 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:30.533+0000 7f25bf9ef740 1 -- start start 2024-03-28T18:06:30.534 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:30.533+0000 7f25bf9ef740 1 Processor -- start 2024-03-28T18:06:30.534 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:30.534+0000 7f25bf9ef740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d4614ead80 0x55d4614fc080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:06:30.535 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:30.534+0000 7f25bf9ef740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55d4614eb770 0x55d4614fc5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:06:30.535 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:30.534+0000 7f25bf9ef740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d4614f08e0 0x55d4614f8b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:06:30.535 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:30.534+0000 7f25bf9ef740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d461502580 con 0x55d4614ead80 2024-03-28T18:06:30.535 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:30.534+0000 7f25bf9ef740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d461501f20 con 0x55d4614eb770 2024-03-28T18:06:30.535 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:30.534+0000 7f25bf9ef740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d461502be0 con 0x55d4614f08e0 2024-03-28T18:06:30.535 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:30.534+0000 7f25bf9ef740 10 client.0 _ll_register_callbacks cb 0x7f25a80e0070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-28T18:06:30.537 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:30.534+0000 7f25b5e00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d4614ead80 0x55d4614fc080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:06:30.537 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:30.534+0000 7f25b5e00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d4614ead80 0x55d4614fc080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:49102/0 (socket says 172.21.15.22:49102) 2024-03-28T18:06:30.537 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:30.534+0000 7f25b5e00640 1 -- 172.21.15.22:0/1515348727 learned_addr learned my addr 172.21.15.22:0/1515348727 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-28T18:06:30.537 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:30.535+0000 7f25ad400640 1 --2- 172.21.15.22:0/1515348727 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55d4614eb770 0x55d4614fc5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:06:30.537 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:30.535+0000 7f25b5e00640 1 -- 172.21.15.22:0/1515348727 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d4614f08e0 msgr2=0x55d4614f8b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T18:06:30.537 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:30.535+0000 7f25b6800640 1 --2- 172.21.15.22:0/1515348727 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d4614f08e0 0x55d4614f8b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:06:30.537 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:30.535+0000 7f25b5e00640 1 --2- 172.21.15.22:0/1515348727 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d4614f08e0 0x55d4614f8b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:06:30.538 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:30.535+0000 7f25b5e00640 1 -- 172.21.15.22:0/1515348727 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55d4614eb770 msgr2=0x55d4614fc5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T18:06:30.538 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:30.535+0000 7f25b5e00640 1 --2- 172.21.15.22:0/1515348727 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55d4614eb770 0x55d4614fc5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:06:30.538 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:30.535+0000 7f25b5e00640 1 -- 172.21.15.22:0/1515348727 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55d4614f9260 con 0x55d4614ead80 2024-03-28T18:06:30.538 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:30.535+0000 7f25b6800640 1 --2- 172.21.15.22:0/1515348727 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d4614f08e0 0x55d4614f8b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T18:06:30.538 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:30.535+0000 7f25ad400640 1 --2- 172.21.15.22:0/1515348727 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55d4614eb770 0x55d4614fc5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-28T18:06:30.538 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:30.535+0000 7f25b5e00640 1 --2- 172.21.15.22:0/1515348727 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d4614ead80 0x55d4614fc080 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f25a8009610 tx=0x7f25a809b6e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fafb5e8d2e0f90fc server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:06:30.538 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:30.535+0000 7f259b400640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-03-28T18:06:30.538 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:30.536+0000 7f259b400640 1 -- 172.21.15.22:0/1515348727 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25a80b7020 con 0x55d4614ead80 2024-03-28T18:06:30.538 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:30.536+0000 7f259b400640 1 -- 172.21.15.22:0/1515348727 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 8153+0+0 (secure 0 0 0) 0x7f25a80a3180 con 0x55d4614ead80 2024-03-28T18:06:30.538 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:30.536+0000 7f25bf9ef740 1 -- 172.21.15.22:0/1515348727 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f25a80d61c0 con 0x55d4614ead80 2024-03-28T18:06:30.538 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:30.536+0000 7f259b400640 1 -- 172.21.15.22:0/1515348727 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7f25a8020070 con 0x55d4614ead80 2024-03-28T18:06:30.538 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:30.536+0000 7f25bf9ef740 10 client.15078 fetch_fsmap learned FSMap version 26 2024-03-28T18:06:30.538 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:30.536+0000 7f25bf9ef740 1 -- 172.21.15.22:0/1515348727 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x55d4614f9260 con 0x55d4614ead80 2024-03-28T18:06:30.539 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:30.536+0000 7f259b400640 1 -- 172.21.15.22:0/1515348727 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f25a80d61c0 con 0x55d4614ead80 2024-03-28T18:06:30.539 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:30.536+0000 7f259b400640 1 -- 172.21.15.22:0/1515348727 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 26) v1 ==== 7934+0+0 (secure 0 0 0) 0x7f25a8079ce0 con 0x55d4614ead80 2024-03-28T18:06:30.539 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:30.536+0000 7f25bf9ef740 10 client.15078 fetch_fsmap finished waiting for FSMap version 26 2024-03-28T18:06:30.539 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:30.536+0000 7f25bf9ef740 10 client.15078 resolve_mds: resolved name 'i' to daemon mds.i 2024-03-28T18:06:30.539 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:30.536+0000 7f25bf9ef740 20 client.15078 populate_metadata read hostname 'smithi022' 2024-03-28T18:06:30.539 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:30.536+0000 7f25bf9ef740 1 --2- 172.21.15.22:0/1515348727 >> [v2:172.21.15.134:6838/3400263217,v1:172.21.15.134:6839/3400263217] conn(0x55d461561500 0x55d46155d580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:06:30.539 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:30.536+0000 7f25bf9ef740 4 client.15078 mds_command: new command op to 24502 tid=0 multi_id=0 [{"prefix": "status"}] 2024-03-28T18:06:30.539 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:30.536+0000 7f25bf9ef740 1 -- 172.21.15.22:0/1515348727 --> [v2:172.21.15.134:6838/3400263217,v1:172.21.15.134:6839/3400263217] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55d46145d1d0 con 0x55d461561500 2024-03-28T18:06:30.539 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:30.537+0000 7f25ad400640 1 --2- 172.21.15.22:0/1515348727 >> [v2:172.21.15.134:6838/3400263217,v1:172.21.15.134:6839/3400263217] conn(0x55d461561500 0x55d46155d580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:06:30.540 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:30.538+0000 7f25ad400640 1 --2- 172.21.15.22:0/1515348727 >> [v2:172.21.15.134:6838/3400263217,v1:172.21.15.134:6839/3400263217] conn(0x55d461561500 0x55d46155d580 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=d77d9a689055893c server_cookie=5a77b8b46a212512 in_seq=0 out_seq=0 2024-03-28T18:06:30.540 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:30.538+0000 7f259b400640 10 client.15078 ms_handle_connect on v2:172.21.15.134:6838/3400263217 2024-03-28T18:06:30.540 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:30.538+0000 7f259b400640 1 -- 172.21.15.22:0/1515348727 <== mds.4 v2:172.21.15.134:6838/3400263217 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+327 (crc 0 0 0) 0x55d46145d1d0 con 0x55d461561500 2024-03-28T18:06:30.540 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:30.538+0000 7f259b400640 10 client.15078 handle_command_reply: tid=0 2024-03-28T18:06:30.541 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:30.538+0000 7f25bf9ef740 1 client.15078 shutdown 2024-03-28T18:06:30.541 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:30.539+0000 7f25bf9ef740 1 -- 172.21.15.22:0/1515348727 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d4614ead80 msgr2=0x55d4614fc080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:06:30.541 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:30.539+0000 7f25bf9ef740 1 --2- 172.21.15.22:0/1515348727 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d4614ead80 0x55d4614fc080 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f25a8009610 tx=0x7f25a809b6e0 comp rx=0 tx=0).stop 2024-03-28T18:06:30.541 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:30.539+0000 7f25bf9ef740 1 -- 172.21.15.22:0/1515348727 shutdown_connections 2024-03-28T18:06:30.541 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:30.539+0000 7f25bf9ef740 1 --2- 172.21.15.22:0/1515348727 >> [v2:172.21.15.134:6838/3400263217,v1:172.21.15.134:6839/3400263217] conn(0x55d461561500 0x55d46155d580 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:06:30.541 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:30.539+0000 7f25bf9ef740 1 --2- 172.21.15.22:0/1515348727 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d4614f08e0 0x55d4614f8b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:06:30.541 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:30.539+0000 7f25bf9ef740 1 --2- 172.21.15.22:0/1515348727 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55d4614eb770 0x55d4614fc5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:06:30.541 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:30.539+0000 7f25bf9ef740 1 --2- 172.21.15.22:0/1515348727 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d4614ead80 0x55d4614fc080 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:06:30.541 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:30.539+0000 7f259b400640 0 client.15078 ms_handle_reset on v2:172.21.15.134:6838/3400263217 2024-03-28T18:06:30.541 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:30.539+0000 7f25bf9ef740 1 -- 172.21.15.22:0/1515348727 >> 172.21.15.22:0/1515348727 conn(0x55d4614e62f0 msgr2=0x55d4614f4e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:06:30.541 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:30.539+0000 7f25bf9ef740 1 -- 172.21.15.22:0/1515348727 shutdown_connections 2024-03-28T18:06:30.541 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:30.539+0000 7f25bf9ef740 1 -- 172.21.15.22:0/1515348727 wait complete. 2024-03-28T18:06:30.541 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:30.539+0000 7f25bf9ef740 20 client.15078 trim_cache size 0 max 16384 2024-03-28T18:06:30.542 INFO:teuthology.orchestra.run.smithi022.stdout:{ 2024-03-28T18:06:30.542 INFO:teuthology.orchestra.run.smithi022.stdout: "cluster_fsid": "fd220748-ed2b-11ee-95d0-87774f69a715", 2024-03-28T18:06:30.542 INFO:teuthology.orchestra.run.smithi022.stdout: "whoami": 4, 2024-03-28T18:06:30.542 INFO:teuthology.orchestra.run.smithi022.stdout: "id": 24502, 2024-03-28T18:06:30.542 INFO:teuthology.orchestra.run.smithi022.stdout: "want_state": "up:active", 2024-03-28T18:06:30.542 INFO:teuthology.orchestra.run.smithi022.stdout: "state": "up:active", 2024-03-28T18:06:30.542 INFO:teuthology.orchestra.run.smithi022.stdout: "fs_name": "cephfs", 2024-03-28T18:06:30.542 INFO:teuthology.orchestra.run.smithi022.stdout: "rank_uptime": 94.909369115000004, 2024-03-28T18:06:30.542 INFO:teuthology.orchestra.run.smithi022.stdout: "mdsmap_epoch": 26, 2024-03-28T18:06:30.542 INFO:teuthology.orchestra.run.smithi022.stdout: "osdmap_epoch": 92, 2024-03-28T18:06:30.542 INFO:teuthology.orchestra.run.smithi022.stdout: "osdmap_epoch_barrier": 90, 2024-03-28T18:06:30.542 INFO:teuthology.orchestra.run.smithi022.stdout: "uptime": 118.268944695 2024-03-28T18:06:30.542 INFO:teuthology.orchestra.run.smithi022.stdout:} 2024-03-28T18:06:30.543 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:30.540+0000 7f25b7200640 1 -- 172.21.15.22:0/143257529 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7f259c0a2800 msgr2=0x7f259c0a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:06:30.543 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:30.540+0000 7f25b7200640 1 --2- 172.21.15.22:0/143257529 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7f259c0a2800 0x7f259c0a4cc0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x55d4614ebfd0 tx=0x7f25b0009500 comp rx=0 tx=0).stop 2024-03-28T18:06:30.543 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:30.540+0000 7f25b7200640 1 -- 172.21.15.22:0/143257529 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f25b80ffdf0 msgr2=0x7f25b819de90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:06:30.543 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:30.540+0000 7f25b7200640 1 --2- 172.21.15.22:0/143257529 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f25b80ffdf0 0x7f25b819de90 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f25a800d770 tx=0x7f25a800dc30 comp rx=0 tx=0).stop 2024-03-28T18:06:30.543 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:30.540+0000 7f25b7200640 1 -- 172.21.15.22:0/143257529 shutdown_connections 2024-03-28T18:06:30.543 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:30.540+0000 7f25b7200640 1 --2- 172.21.15.22:0/143257529 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7f259c0a2800 0x7f259c0a4cc0 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:06:30.543 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:30.540+0000 7f25b7200640 1 --2- 172.21.15.22:0/143257529 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f25b8105110 0x7f25b81a5440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:06:30.544 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:30.540+0000 7f25b7200640 1 --2- 172.21.15.22:0/143257529 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f25b8102760 0x7f25b819e3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:06:30.544 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:30.540+0000 7f25b7200640 1 --2- 172.21.15.22:0/143257529 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f25b80ffdf0 0x7f25b819de90 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:06:30.544 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:30.540+0000 7f25b7200640 1 -- 172.21.15.22:0/143257529 >> 172.21.15.22:0/143257529 conn(0x7f25b80fba10 msgr2=0x7f25b8100730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:06:30.544 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:30.540+0000 7f25b7200640 1 -- 172.21.15.22:0/143257529 shutdown_connections 2024-03-28T18:06:30.544 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:30.540+0000 7f25b7200640 1 -- 172.21.15.22:0/143257529 wait complete. 2024-03-28T18:06:30.559 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.k status 2024-03-28T18:06:30.784 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:30.784+0000 7f0356400640 1 -- 172.21.15.22:0/3390052872 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0350104930 msgr2=0x7f0350106d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:06:30.784 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:30.784+0000 7f0356400640 1 --2- 172.21.15.22:0/3390052872 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0350104930 0x7f0350106d20 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f0338009900 tx=0x7f033802bc50 comp rx=0 tx=0).stop 2024-03-28T18:06:30.784 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:30.784+0000 7f0356400640 1 -- 172.21.15.22:0/3390052872 shutdown_connections 2024-03-28T18:06:30.784 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:30.784+0000 7f0356400640 1 --2- 172.21.15.22:0/3390052872 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0350107260 0x7f03501096b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:06:30.784 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:30.784+0000 7f0356400640 1 --2- 172.21.15.22:0/3390052872 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0350104930 0x7f0350106d20 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:06:30.784 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:30.784+0000 7f0356400640 1 --2- 172.21.15.22:0/3390052872 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f0350102000 0x7f03501043f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:06:30.785 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:30.784+0000 7f0356400640 1 -- 172.21.15.22:0/3390052872 >> 172.21.15.22:0/3390052872 conn(0x7f03500fba10 msgr2=0x7f03500fde30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:06:30.785 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:30.784+0000 7f0356400640 1 -- 172.21.15.22:0/3390052872 shutdown_connections 2024-03-28T18:06:30.785 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:30.784+0000 7f0356400640 1 -- 172.21.15.22:0/3390052872 wait complete. 2024-03-28T18:06:30.786 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:30.785+0000 7f0356400640 1 Processor -- start 2024-03-28T18:06:30.786 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:30.785+0000 7f0356400640 1 -- start start 2024-03-28T18:06:30.786 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:30.786+0000 7f0356400640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0350102000 0x7f03501978a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:06:30.786 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:30.786+0000 7f0356400640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0350104930 0x7f0350197de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:06:30.786 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:30.786+0000 7f0356400640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f0350107260 0x7f035019ee50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:06:30.787 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:30.786+0000 7f0356400640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f035010d230 con 0x7f0350104930 2024-03-28T18:06:30.787 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:30.786+0000 7f0356400640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f035010d0b0 con 0x7f0350107260 2024-03-28T18:06:30.787 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:30.786+0000 7f0356400640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f035010d3b0 con 0x7f0350102000 2024-03-28T18:06:30.787 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:30.786+0000 7f0347e00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0350104930 0x7f0350197de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:06:30.787 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:30.786+0000 7f0347e00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0350104930 0x7f0350197de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:49124/0 (socket says 172.21.15.22:49124) 2024-03-28T18:06:30.787 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:30.786+0000 7f0347e00640 1 -- 172.21.15.22:0/203038342 learned_addr learned my addr 172.21.15.22:0/203038342 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-28T18:06:30.787 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:30.786+0000 7f0355000640 1 --2- 172.21.15.22:0/203038342 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0350102000 0x7f03501978a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:06:30.787 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:30.786+0000 7f0355a00640 1 --2- 172.21.15.22:0/203038342 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f0350107260 0x7f035019ee50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:06:30.787 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:30.786+0000 7f0347e00640 1 -- 172.21.15.22:0/203038342 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0350102000 msgr2=0x7f03501978a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:06:30.787 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:30.786+0000 7f0347e00640 1 --2- 172.21.15.22:0/203038342 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0350102000 0x7f03501978a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:06:30.787 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:30.786+0000 7f0347e00640 1 -- 172.21.15.22:0/203038342 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f0350107260 msgr2=0x7f035019ee50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:06:30.787 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:30.786+0000 7f0347e00640 1 --2- 172.21.15.22:0/203038342 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f0350107260 0x7f035019ee50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:06:30.787 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:30.786+0000 7f0347e00640 1 -- 172.21.15.22:0/203038342 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f035019f4c0 con 0x7f0350104930 2024-03-28T18:06:30.787 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:30.787+0000 7f0347e00640 1 --2- 172.21.15.22:0/203038342 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0350104930 0x7f0350197de0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f0338000c00 tx=0x7f0338004e20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:06:30.788 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:30.787+0000 7f034e000640 1 -- 172.21.15.22:0/203038342 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0338043020 con 0x7f0350104930 2024-03-28T18:06:30.788 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:30.787+0000 7f034e000640 1 -- 172.21.15.22:0/203038342 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f0338005460 con 0x7f0350104930 2024-03-28T18:06:30.788 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:30.787+0000 7f0356400640 1 -- 172.21.15.22:0/203038342 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f035019f750 con 0x7f0350104930 2024-03-28T18:06:30.788 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:30.787+0000 7f034e000640 1 -- 172.21.15.22:0/203038342 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0338003d00 con 0x7f0350104930 2024-03-28T18:06:30.788 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:30.787+0000 7f0356400640 1 -- 172.21.15.22:0/203038342 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f035019fca0 con 0x7f0350104930 2024-03-28T18:06:30.791 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:30.791+0000 7f034e000640 1 -- 172.21.15.22:0/203038342 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 18) v1 ==== 137290+0+0 (secure 0 0 0) 0x7f0338046020 con 0x7f0350104930 2024-03-28T18:06:30.791 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:30.791+0000 7f035a621740 1 Processor -- start 2024-03-28T18:06:30.791 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:30.791+0000 7f035a621740 1 -- start start 2024-03-28T18:06:30.791 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:30.791+0000 7f035a621740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5598d291ed80 0x5598d291f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:06:30.791 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:30.791+0000 7f035a621740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5598d291f770 0x5598d2924270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:06:30.791 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:30.791+0000 7f035a621740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5598d29248e0 0x5598d2926d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:06:30.791 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:30.791+0000 7f035a621740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x5598d2883c90 con 0x5598d29248e0 2024-03-28T18:06:30.791 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:30.791+0000 7f035a621740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x5598d282c980 con 0x5598d291ed80 2024-03-28T18:06:30.791 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:30.791+0000 7f035a621740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x5598d287fd30 con 0x5598d291f770 2024-03-28T18:06:30.792 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:30.791+0000 7f0347e00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5598d29248e0 0x5598d2926d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:06:30.792 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:30.791+0000 7f0347e00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5598d29248e0 0x5598d2926d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:49136/0 (socket says 172.21.15.22:49136) 2024-03-28T18:06:30.792 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:30.791+0000 7f0355a00640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5598d291ed80 0x5598d291f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:06:30.792 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:30.792+0000 7f0347e00640 1 -- 172.21.15.22:0/1371063872 learned_addr learned my addr 172.21.15.22:0/1371063872 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-28T18:06:30.793 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:30.792+0000 7f0355000640 1 --2- 172.21.15.22:0/1371063872 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5598d291f770 0x5598d2924270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:06:30.793 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:30.792+0000 7f0347e00640 1 -- 172.21.15.22:0/1371063872 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5598d291f770 msgr2=0x5598d2924270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T18:06:30.793 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:30.792+0000 7f0347e00640 1 --2- 172.21.15.22:0/1371063872 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5598d291f770 0x5598d2924270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:06:30.793 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:30.792+0000 7f0347e00640 1 -- 172.21.15.22:0/1371063872 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5598d291ed80 msgr2=0x5598d291f1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T18:06:30.793 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:30.792+0000 7f0347e00640 1 --2- 172.21.15.22:0/1371063872 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5598d291ed80 0x5598d291f1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:06:30.793 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:30.792+0000 7f0347e00640 1 -- 172.21.15.22:0/1371063872 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5598d2844510 con 0x5598d29248e0 2024-03-28T18:06:30.793 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:30.792+0000 7f034e000640 1 --2- 172.21.15.22:0/203038342 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7f03340a2800 0x7f03340a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:06:30.793 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:30.792+0000 7f0347e00640 1 --2- 172.21.15.22:0/1371063872 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5598d29248e0 0x5598d2926d70 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f03380f4ce0 tx=0x7f03380f4dd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5b27a6fa23e9d803 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:06:30.793 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:30.793+0000 7f0355a00640 1 --2- 172.21.15.22:0/203038342 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7f03340a2800 0x7f03340a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:06:30.793 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:30.793+0000 7f034e000640 1 -- 172.21.15.22:0/203038342 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 8153+0+0 (secure 0 0 0) 0x7f03380f0d40 con 0x7f0350104930 2024-03-28T18:06:30.793 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:30.793+0000 7f0346000640 1 -- 172.21.15.22:0/1371063872 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03380b9a00 con 0x5598d29248e0 2024-03-28T18:06:30.793 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:30.793+0000 7f0346000640 1 -- 172.21.15.22:0/1371063872 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f0338004c70 con 0x5598d29248e0 2024-03-28T18:06:30.793 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:30.793+0000 7f0346000640 1 -- 172.21.15.22:0/1371063872 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03380bd0c0 con 0x5598d29248e0 2024-03-28T18:06:30.797 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:30.793+0000 7f035a621740 1 -- 172.21.15.22:0/1371063872 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5598d29248e0 msgr2=0x5598d2926d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:06:30.797 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:30.793+0000 7f0355a00640 1 --2- 172.21.15.22:0/203038342 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7f03340a2800 0x7f03340a4cc0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x5598d291ffd0 tx=0x7f03480096e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:06:30.797 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:30.793+0000 7f035a621740 1 --2- 172.21.15.22:0/1371063872 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5598d29248e0 0x5598d2926d70 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f03380f4ce0 tx=0x7f03380f4dd0 comp rx=0 tx=0).stop 2024-03-28T18:06:30.797 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:30.794+0000 7f035a621740 1 -- 172.21.15.22:0/1371063872 shutdown_connections 2024-03-28T18:06:30.797 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:30.794+0000 7f035a621740 1 --2- 172.21.15.22:0/1371063872 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5598d29248e0 0x5598d2926d70 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:06:30.797 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:30.794+0000 7f035a621740 1 --2- 172.21.15.22:0/1371063872 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5598d291f770 0x5598d2924270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:06:30.797 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:30.794+0000 7f035a621740 1 --2- 172.21.15.22:0/1371063872 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5598d291ed80 0x5598d291f1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:06:30.797 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:30.794+0000 7f035a621740 1 -- 172.21.15.22:0/1371063872 >> 172.21.15.22:0/1371063872 conn(0x5598d291a2f0 msgr2=0x5598d291a6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:06:30.798 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:30.794+0000 7f035a621740 1 -- 172.21.15.22:0/1371063872 shutdown_connections 2024-03-28T18:06:30.798 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:30.794+0000 7f035a621740 1 -- 172.21.15.22:0/1371063872 wait complete. 2024-03-28T18:06:30.798 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:30.794+0000 7f035a621740 1 -- start start 2024-03-28T18:06:30.798 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:30.794+0000 7f035a621740 1 Processor -- start 2024-03-28T18:06:30.798 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:30.795+0000 7f035a621740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5598d291ed80 0x5598d2928fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:06:30.798 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:30.795+0000 7f035a621740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5598d291f770 0x5598d2929500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:06:30.798 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:30.795+0000 7f035a621740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5598d29248e0 0x5598d2930540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:06:30.798 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:30.795+0000 7f035a621740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x5598d2927650 con 0x5598d29248e0 2024-03-28T18:06:30.798 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:30.795+0000 7f035a621740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x5598d283fdd0 con 0x5598d291f770 2024-03-28T18:06:30.798 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:30.795+0000 7f035a621740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x5598d29277d0 con 0x5598d291ed80 2024-03-28T18:06:30.798 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:30.795+0000 7f035a621740 10 client.0 _ll_register_callbacks cb 0x5598d2597ac0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-28T18:06:30.798 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:30.795+0000 7f0355a00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5598d29248e0 0x5598d2930540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:06:30.798 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:30.795+0000 7f0355a00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5598d29248e0 0x5598d2930540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:49140/0 (socket says 172.21.15.22:49140) 2024-03-28T18:06:30.798 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:30.795+0000 7f0355a00640 1 -- 172.21.15.22:0/421930235 learned_addr learned my addr 172.21.15.22:0/421930235 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-28T18:06:30.799 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:30.795+0000 7f0347e00640 1 --2- 172.21.15.22:0/421930235 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5598d291f770 0x5598d2929500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:06:30.799 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:30.795+0000 7f0355000640 1 --2- 172.21.15.22:0/421930235 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5598d291ed80 0x5598d2928fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:06:30.799 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:30.795+0000 7f0355a00640 1 -- 172.21.15.22:0/421930235 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5598d291ed80 msgr2=0x5598d2928fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T18:06:30.799 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:30.796+0000 7f0355a00640 1 --2- 172.21.15.22:0/421930235 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5598d291ed80 0x5598d2928fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:06:30.799 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:30.796+0000 7f0355a00640 1 -- 172.21.15.22:0/421930235 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5598d291f770 msgr2=0x5598d2929500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T18:06:30.799 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:30.796+0000 7f0355a00640 1 --2- 172.21.15.22:0/421930235 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5598d291f770 0x5598d2929500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:06:30.799 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:30.796+0000 7f0355a00640 1 -- 172.21.15.22:0/421930235 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5598d25982d0 con 0x5598d29248e0 2024-03-28T18:06:30.799 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:30.796+0000 7f0355a00640 1 --2- 172.21.15.22:0/421930235 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5598d29248e0 0x5598d2930540 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f0348010880 tx=0x7f0348004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d1205a5b019a164e server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:06:30.799 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:30.796+0000 7f033ea00640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-03-28T18:06:30.799 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:30.796+0000 7f033ea00640 1 -- 172.21.15.22:0/421930235 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f034800eae0 con 0x5598d29248e0 2024-03-28T18:06:30.799 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:30.796+0000 7f035a621740 1 -- 172.21.15.22:0/421930235 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f0338004c70 con 0x5598d29248e0 2024-03-28T18:06:30.799 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:30.796+0000 7f033ea00640 1 -- 172.21.15.22:0/421930235 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 8153+0+0 (secure 0 0 0) 0x7f034800ec80 con 0x5598d29248e0 2024-03-28T18:06:30.799 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:30.796+0000 7f033ea00640 1 -- 172.21.15.22:0/421930235 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0348027400 con 0x5598d29248e0 2024-03-28T18:06:30.799 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:30.796+0000 7f035a621740 10 client.15090 fetch_fsmap learned FSMap version 26 2024-03-28T18:06:30.800 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:30.796+0000 7f035a621740 1 -- 172.21.15.22:0/421930235 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x5598d26626b0 con 0x5598d29248e0 2024-03-28T18:06:30.800 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:30.797+0000 7f033ea00640 1 -- 172.21.15.22:0/421930235 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f0338004c70 con 0x5598d29248e0 2024-03-28T18:06:30.800 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:30.797+0000 7f033ea00640 1 -- 172.21.15.22:0/421930235 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 26) v1 ==== 7934+0+0 (secure 0 0 0) 0x7f034800f040 con 0x5598d29248e0 2024-03-28T18:06:30.800 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:30.797+0000 7f035a621740 10 client.15090 fetch_fsmap finished waiting for FSMap version 26 2024-03-28T18:06:30.800 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:30.797+0000 7f035a621740 10 client.15090 resolve_mds: resolved name 'k' to daemon mds.k 2024-03-28T18:06:30.800 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:30.797+0000 7f035a621740 20 client.15090 populate_metadata read hostname 'smithi022' 2024-03-28T18:06:30.800 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:30.797+0000 7f035a621740 1 --2- 172.21.15.22:0/421930235 >> [v2:172.21.15.79:6832/856896997,v1:172.21.15.79:6833/856896997] conn(0x5598d2935840 0x5598d2934930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:06:30.800 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:30.797+0000 7f035a621740 4 client.15090 mds_command: new command op to 34442 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-28T18:06:30.800 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:30.797+0000 7f035a621740 1 -- 172.21.15.22:0/421930235 --> [v2:172.21.15.79:6832/856896997,v1:172.21.15.79:6833/856896997] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5598d2825520 con 0x5598d2935840 2024-03-28T18:06:30.800 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:30.797+0000 7f0355000640 1 --2- 172.21.15.22:0/421930235 >> [v2:172.21.15.79:6832/856896997,v1:172.21.15.79:6833/856896997] conn(0x5598d2935840 0x5598d2934930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:06:30.801 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:30.798+0000 7f0355000640 1 --2- 172.21.15.22:0/421930235 >> [v2:172.21.15.79:6832/856896997,v1:172.21.15.79:6833/856896997] conn(0x5598d2935840 0x5598d2934930 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=aae50e979cb07ac7 server_cookie=83cd621f32968cb3 in_seq=0 out_seq=0 2024-03-28T18:06:30.801 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:30.798+0000 7f033ea00640 10 client.15090 ms_handle_connect on v2:172.21.15.79:6832/856896997 2024-03-28T18:06:30.802 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:30.800+0000 7f033ea00640 1 -- 172.21.15.22:0/421930235 <== mds.3 v2:172.21.15.79:6832/856896997 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x5598d2825520 con 0x5598d2935840 2024-03-28T18:06:30.802 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:30.800+0000 7f033ea00640 10 client.15090 handle_command_reply: tid=0 2024-03-28T18:06:30.802 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:30.800+0000 7f035a621740 1 client.15090 shutdown 2024-03-28T18:06:30.802 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:30.800+0000 7f035a621740 1 -- 172.21.15.22:0/421930235 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5598d29248e0 msgr2=0x5598d2930540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:06:30.802 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:30.800+0000 7f035a621740 1 --2- 172.21.15.22:0/421930235 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5598d29248e0 0x5598d2930540 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f0348010880 tx=0x7f0348004a40 comp rx=0 tx=0).stop 2024-03-28T18:06:30.802 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:30.800+0000 7f035a621740 1 -- 172.21.15.22:0/421930235 shutdown_connections 2024-03-28T18:06:30.802 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:30.800+0000 7f035a621740 1 --2- 172.21.15.22:0/421930235 >> [v2:172.21.15.79:6832/856896997,v1:172.21.15.79:6833/856896997] conn(0x5598d2935840 0x5598d2934930 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:06:30.802 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:30.800+0000 7f035a621740 1 --2- 172.21.15.22:0/421930235 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5598d29248e0 0x5598d2930540 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:06:30.802 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:30.800+0000 7f035a621740 1 --2- 172.21.15.22:0/421930235 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5598d291f770 0x5598d2929500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:06:30.802 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:30.800+0000 7f035a621740 1 --2- 172.21.15.22:0/421930235 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5598d291ed80 0x5598d2928fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:06:30.802 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:30.800+0000 7f033ea00640 0 client.15090 ms_handle_reset on v2:172.21.15.79:6832/856896997 2024-03-28T18:06:30.802 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:30.800+0000 7f035a621740 1 -- 172.21.15.22:0/421930235 >> 172.21.15.22:0/421930235 conn(0x5598d291a2f0 msgr2=0x5598d29194f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:06:30.802 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:30.801+0000 7f035a621740 1 -- 172.21.15.22:0/421930235 shutdown_connections 2024-03-28T18:06:30.802 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:30.801+0000 7f035a621740 1 -- 172.21.15.22:0/421930235 wait complete. 2024-03-28T18:06:30.803 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:30.801+0000 7f035a621740 20 client.15090 trim_cache size 0 max 16384 2024-03-28T18:06:30.818 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:30.819+0000 7f035a621740 1 Processor -- start 2024-03-28T18:06:30.819 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:30.819+0000 7f035a621740 1 -- start start 2024-03-28T18:06:30.820 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:30.819+0000 7f035a621740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5598d291ed80 0x5598d2928fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:06:30.820 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:30.819+0000 7f035a621740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5598d291f770 0x5598d2929500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:06:30.820 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:30.820+0000 7f035a621740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5598d29248e0 0x5598d2920c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:06:30.820 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:30.820+0000 7f035a621740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x5598d2935060 con 0x5598d291ed80 2024-03-28T18:06:30.820 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:30.820+0000 7f035a621740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x5598d29356c0 con 0x5598d291f770 2024-03-28T18:06:30.820 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:30.820+0000 7f035a621740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x5598d2935d20 con 0x5598d29248e0 2024-03-28T18:06:30.820 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:30.820+0000 7f0355000640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5598d291ed80 0x5598d2928fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:06:30.820 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:30.820+0000 7f0355000640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5598d291ed80 0x5598d2928fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:49148/0 (socket says 172.21.15.22:49148) 2024-03-28T18:06:30.820 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:30.820+0000 7f0355000640 1 -- 172.21.15.22:0/1347265004 learned_addr learned my addr 172.21.15.22:0/1347265004 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-28T18:06:30.820 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:30.820+0000 7f0347e00640 1 --2- 172.21.15.22:0/1347265004 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5598d291f770 0x5598d2929500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:06:30.820 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:30.820+0000 7f0355a00640 1 --2- 172.21.15.22:0/1347265004 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5598d29248e0 0x5598d2920c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:06:30.820 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:30.820+0000 7f0355000640 1 -- 172.21.15.22:0/1347265004 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5598d29248e0 msgr2=0x5598d2920c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T18:06:30.821 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:30.820+0000 7f0355000640 1 --2- 172.21.15.22:0/1347265004 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5598d29248e0 0x5598d2920c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:06:30.821 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:30.820+0000 7f0355000640 1 -- 172.21.15.22:0/1347265004 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5598d291f770 msgr2=0x5598d2929500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T18:06:30.821 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:30.820+0000 7f0355000640 1 --2- 172.21.15.22:0/1347265004 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5598d291f770 0x5598d2929500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:06:30.821 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:30.820+0000 7f0355000640 1 -- 172.21.15.22:0/1347265004 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5598d290c170 con 0x5598d291ed80 2024-03-28T18:06:30.824 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:30.820+0000 7f0355000640 1 --2- 172.21.15.22:0/1347265004 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5598d291ed80 0x5598d2928fc0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x5598d291fb70 tx=0x7f034001dbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7ef9c1908240afc2 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:06:30.824 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:30.821+0000 7f0345600640 1 -- 172.21.15.22:0/1347265004 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0340004030 con 0x5598d291ed80 2024-03-28T18:06:30.824 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:30.821+0000 7f0345600640 1 -- 172.21.15.22:0/1347265004 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f0340013de0 con 0x5598d291ed80 2024-03-28T18:06:30.824 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:30.821+0000 7f0345600640 1 -- 172.21.15.22:0/1347265004 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0340003ab0 con 0x5598d291ed80 2024-03-28T18:06:30.824 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:30.821+0000 7f035a621740 1 -- 172.21.15.22:0/1347265004 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5598d291ed80 msgr2=0x5598d2928fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:06:30.824 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:30.821+0000 7f035a621740 1 --2- 172.21.15.22:0/1347265004 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5598d291ed80 0x5598d2928fc0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x5598d291fb70 tx=0x7f034001dbc0 comp rx=0 tx=0).stop 2024-03-28T18:06:30.824 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:30.822+0000 7f035a621740 1 -- 172.21.15.22:0/1347265004 shutdown_connections 2024-03-28T18:06:30.824 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:30.822+0000 7f035a621740 1 --2- 172.21.15.22:0/1347265004 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5598d29248e0 0x5598d2920c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:06:30.824 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:30.822+0000 7f035a621740 1 --2- 172.21.15.22:0/1347265004 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5598d291f770 0x5598d2929500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:06:30.825 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:30.822+0000 7f035a621740 1 --2- 172.21.15.22:0/1347265004 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5598d291ed80 0x5598d2928fc0 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:06:30.825 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:30.822+0000 7f035a621740 1 -- 172.21.15.22:0/1347265004 >> 172.21.15.22:0/1347265004 conn(0x5598d291a2f0 msgr2=0x5598d291a6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:06:30.825 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:30.822+0000 7f035a621740 1 -- 172.21.15.22:0/1347265004 shutdown_connections 2024-03-28T18:06:30.825 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:30.822+0000 7f035a621740 1 -- 172.21.15.22:0/1347265004 wait complete. 2024-03-28T18:06:30.825 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:30.822+0000 7f035a621740 1 -- start start 2024-03-28T18:06:30.825 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:30.822+0000 7f035a621740 1 Processor -- start 2024-03-28T18:06:30.825 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:30.823+0000 7f035a621740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5598d291ed80 0x5598d298cb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:06:30.825 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:30.823+0000 7f035a621740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5598d291f770 0x5598d298d0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:06:30.825 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:30.823+0000 7f035a621740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5598d29248e0 0x5598d2994110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:06:30.825 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:30.823+0000 7f035a621740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x5598d29369e0 con 0x5598d291ed80 2024-03-28T18:06:30.825 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:30.823+0000 7f035a621740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x5598d2936380 con 0x5598d29248e0 2024-03-28T18:06:30.825 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:30.823+0000 7f035a621740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x5598d2937040 con 0x5598d291f770 2024-03-28T18:06:30.825 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:30.823+0000 7f035a621740 10 client.0 _ll_register_callbacks cb 0x7f03480093c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-28T18:06:30.826 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:30.823+0000 7f0355000640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5598d291ed80 0x5598d298cb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:06:30.826 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:30.823+0000 7f0355000640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5598d291ed80 0x5598d298cb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:49162/0 (socket says 172.21.15.22:49162) 2024-03-28T18:06:30.826 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:30.823+0000 7f0355000640 1 -- 172.21.15.22:0/1350906450 learned_addr learned my addr 172.21.15.22:0/1350906450 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-28T18:06:30.826 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:30.823+0000 7f0355a00640 1 --2- 172.21.15.22:0/1350906450 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5598d29248e0 0x5598d2994110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:06:30.826 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:30.823+0000 7f0347e00640 1 --2- 172.21.15.22:0/1350906450 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5598d291f770 0x5598d298d0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:06:30.826 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:30.823+0000 7f0355000640 1 -- 172.21.15.22:0/1350906450 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5598d291f770 msgr2=0x5598d298d0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T18:06:30.826 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:30.823+0000 7f0355000640 1 --2- 172.21.15.22:0/1350906450 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5598d291f770 0x5598d298d0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:06:30.826 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:30.823+0000 7f0355000640 1 -- 172.21.15.22:0/1350906450 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5598d29248e0 msgr2=0x5598d2994110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T18:06:30.826 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:30.823+0000 7f0355000640 1 --2- 172.21.15.22:0/1350906450 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5598d29248e0 0x5598d2994110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:06:30.826 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:30.823+0000 7f0355000640 1 -- 172.21.15.22:0/1350906450 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5598d2994700 con 0x5598d291ed80 2024-03-28T18:06:30.826 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:30.823+0000 7f0355a00640 1 --2- 172.21.15.22:0/1350906450 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5598d29248e0 0x5598d2994110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-28T18:06:30.826 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:30.824+0000 7f0355000640 1 --2- 172.21.15.22:0/1350906450 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5598d291ed80 0x5598d298cb90 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f034001ff50 tx=0x7f034001ff80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9b35d7e86b0ecd75 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:06:30.826 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:30.824+0000 7f033ea00640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-03-28T18:06:30.826 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:30.824+0000 7f033ea00640 1 -- 172.21.15.22:0/1350906450 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0340013b10 con 0x5598d291ed80 2024-03-28T18:06:30.827 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:30.824+0000 7f033ea00640 1 -- 172.21.15.22:0/1350906450 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 8153+0+0 (secure 0 0 0) 0x7f0340011440 con 0x5598d291ed80 2024-03-28T18:06:30.827 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:30.824+0000 7f035a621740 1 -- 172.21.15.22:0/1350906450 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f0340013de0 con 0x5598d291ed80 2024-03-28T18:06:30.827 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:30.824+0000 7f033ea00640 1 -- 172.21.15.22:0/1350906450 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0340011640 con 0x5598d291ed80 2024-03-28T18:06:30.827 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:30.824+0000 7f035a621740 10 client.15096 fetch_fsmap learned FSMap version 26 2024-03-28T18:06:30.827 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:30.824+0000 7f035a621740 1 -- 172.21.15.22:0/1350906450 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x5598d2995a00 con 0x5598d291ed80 2024-03-28T18:06:30.827 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:30.825+0000 7f033ea00640 1 -- 172.21.15.22:0/1350906450 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f0340013de0 con 0x5598d291ed80 2024-03-28T18:06:30.829 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:30.825+0000 7f033ea00640 1 -- 172.21.15.22:0/1350906450 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 26) v1 ==== 7934+0+0 (secure 0 0 0) 0x7f0340026810 con 0x5598d291ed80 2024-03-28T18:06:30.829 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:30.825+0000 7f035a621740 10 client.15096 fetch_fsmap finished waiting for FSMap version 26 2024-03-28T18:06:30.829 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:30.825+0000 7f035a621740 10 client.15096 resolve_mds: resolved name 'k' to daemon mds.k 2024-03-28T18:06:30.829 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:30.825+0000 7f035a621740 20 client.15096 populate_metadata read hostname 'smithi022' 2024-03-28T18:06:30.829 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:30.825+0000 7f035a621740 1 --2- 172.21.15.22:0/1350906450 >> [v2:172.21.15.79:6832/856896997,v1:172.21.15.79:6833/856896997] conn(0x5598d2999ea0 0x5598d2995c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:06:30.829 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:30.825+0000 7f035a621740 4 client.15096 mds_command: new command op to 34442 tid=0 multi_id=0 [{"prefix": "status"}] 2024-03-28T18:06:30.829 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:30.825+0000 7f035a621740 1 -- 172.21.15.22:0/1350906450 --> [v2:172.21.15.79:6832/856896997,v1:172.21.15.79:6833/856896997] -- command(tid 0: {"prefix": "status"}) v1 -- 0x5598d28911d0 con 0x5598d2999ea0 2024-03-28T18:06:30.829 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:30.825+0000 7f0347e00640 1 --2- 172.21.15.22:0/1350906450 >> [v2:172.21.15.79:6832/856896997,v1:172.21.15.79:6833/856896997] conn(0x5598d2999ea0 0x5598d2995c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:06:30.829 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:30.826+0000 7f0347e00640 1 --2- 172.21.15.22:0/1350906450 >> [v2:172.21.15.79:6832/856896997,v1:172.21.15.79:6833/856896997] conn(0x5598d2999ea0 0x5598d2995c90 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=5a4d452324b889fd server_cookie=b14808719b274858 in_seq=0 out_seq=0 2024-03-28T18:06:30.829 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:30.826+0000 7f033ea00640 10 client.15096 ms_handle_connect on v2:172.21.15.79:6832/856896997 2024-03-28T18:06:30.829 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:30.826+0000 7f033ea00640 1 -- 172.21.15.22:0/1350906450 <== mds.3 v2:172.21.15.79:6832/856896997 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+321 (crc 0 0 0) 0x5598d28911d0 con 0x5598d2999ea0 2024-03-28T18:06:30.829 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:30.826+0000 7f033ea00640 10 client.15096 handle_command_reply: tid=0 2024-03-28T18:06:30.829 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:30.826+0000 7f035a621740 1 client.15096 shutdown 2024-03-28T18:06:30.829 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:30.827+0000 7f035a621740 1 -- 172.21.15.22:0/1350906450 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5598d291ed80 msgr2=0x5598d298cb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:06:30.830 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:30.827+0000 7f035a621740 1 --2- 172.21.15.22:0/1350906450 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5598d291ed80 0x5598d298cb90 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f034001ff50 tx=0x7f034001ff80 comp rx=0 tx=0).stop 2024-03-28T18:06:30.830 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:30.827+0000 7f035a621740 1 -- 172.21.15.22:0/1350906450 shutdown_connections 2024-03-28T18:06:30.830 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:30.827+0000 7f035a621740 1 --2- 172.21.15.22:0/1350906450 >> [v2:172.21.15.79:6832/856896997,v1:172.21.15.79:6833/856896997] conn(0x5598d2999ea0 0x5598d2995c90 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:06:30.830 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:30.827+0000 7f035a621740 1 --2- 172.21.15.22:0/1350906450 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5598d29248e0 0x5598d2994110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:06:30.830 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:30.827+0000 7f033ea00640 0 client.15096 ms_handle_reset on v2:172.21.15.79:6832/856896997 2024-03-28T18:06:30.830 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:30.827+0000 7f035a621740 1 --2- 172.21.15.22:0/1350906450 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5598d291f770 0x5598d298d0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:06:30.830 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:30.827+0000 7f035a621740 1 --2- 172.21.15.22:0/1350906450 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5598d291ed80 0x5598d298cb90 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:06:30.830 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:30.827+0000 7f035a621740 1 -- 172.21.15.22:0/1350906450 >> 172.21.15.22:0/1350906450 conn(0x5598d291a2f0 msgr2=0x5598d2927b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:06:30.830 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:30.827+0000 7f035a621740 1 -- 172.21.15.22:0/1350906450 shutdown_connections 2024-03-28T18:06:30.830 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:30.827+0000 7f035a621740 1 -- 172.21.15.22:0/1350906450 wait complete. 2024-03-28T18:06:30.830 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:30.827+0000 7f035a621740 20 client.15096 trim_cache size 0 max 16384 2024-03-28T18:06:30.831 INFO:teuthology.orchestra.run.smithi022.stdout:{ 2024-03-28T18:06:30.831 INFO:teuthology.orchestra.run.smithi022.stdout: "cluster_fsid": "fd220748-ed2b-11ee-95d0-87774f69a715", 2024-03-28T18:06:30.831 INFO:teuthology.orchestra.run.smithi022.stdout: "whoami": 3, 2024-03-28T18:06:30.831 INFO:teuthology.orchestra.run.smithi022.stdout: "id": 34442, 2024-03-28T18:06:30.831 INFO:teuthology.orchestra.run.smithi022.stdout: "want_state": "up:active", 2024-03-28T18:06:30.831 INFO:teuthology.orchestra.run.smithi022.stdout: "state": "up:active", 2024-03-28T18:06:30.831 INFO:teuthology.orchestra.run.smithi022.stdout: "fs_name": "cephfs", 2024-03-28T18:06:30.831 INFO:teuthology.orchestra.run.smithi022.stdout: "rank_uptime": 96.220165502, 2024-03-28T18:06:30.831 INFO:teuthology.orchestra.run.smithi022.stdout: "mdsmap_epoch": 26, 2024-03-28T18:06:30.831 INFO:teuthology.orchestra.run.smithi022.stdout: "osdmap_epoch": 92, 2024-03-28T18:06:30.831 INFO:teuthology.orchestra.run.smithi022.stdout: "osdmap_epoch_barrier": 90, 2024-03-28T18:06:30.831 INFO:teuthology.orchestra.run.smithi022.stdout: "uptime": 146.558094627 2024-03-28T18:06:30.831 INFO:teuthology.orchestra.run.smithi022.stdout:} 2024-03-28T18:06:30.832 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:30.828+0000 7f0356400640 1 -- 172.21.15.22:0/203038342 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7f03340a2800 msgr2=0x7f03340a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:06:30.832 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:30.828+0000 7f0356400640 1 --2- 172.21.15.22:0/203038342 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7f03340a2800 0x7f03340a4cc0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x5598d291ffd0 tx=0x7f03480096e0 comp rx=0 tx=0).stop 2024-03-28T18:06:30.832 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:30.828+0000 7f0356400640 1 -- 172.21.15.22:0/203038342 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0350104930 msgr2=0x7f0350197de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:06:30.832 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:30.828+0000 7f0356400640 1 --2- 172.21.15.22:0/203038342 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0350104930 0x7f0350197de0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f0338000c00 tx=0x7f0338004e20 comp rx=0 tx=0).stop 2024-03-28T18:06:30.832 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:30.828+0000 7f0356400640 1 -- 172.21.15.22:0/203038342 shutdown_connections 2024-03-28T18:06:30.833 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:30.828+0000 7f0356400640 1 --2- 172.21.15.22:0/203038342 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7f03340a2800 0x7f03340a4cc0 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:06:30.833 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:30.828+0000 7f0356400640 1 --2- 172.21.15.22:0/203038342 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f0350107260 0x7f035019ee50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:06:30.833 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:30.828+0000 7f0356400640 1 --2- 172.21.15.22:0/203038342 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0350104930 0x7f0350197de0 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:06:30.833 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:30.828+0000 7f0356400640 1 --2- 172.21.15.22:0/203038342 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0350102000 0x7f03501978a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:06:30.833 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:30.828+0000 7f0356400640 1 -- 172.21.15.22:0/203038342 >> 172.21.15.22:0/203038342 conn(0x7f03500fba10 msgr2=0x7f0350102940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:06:30.833 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:30.828+0000 7f0356400640 1 -- 172.21.15.22:0/203038342 shutdown_connections 2024-03-28T18:06:30.833 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:30.828+0000 7f0356400640 1 -- 172.21.15.22:0/203038342 wait complete. 2024-03-28T18:06:30.846 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.e status 2024-03-28T18:06:31.069 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:31.069+0000 7f4e7b000640 1 -- 172.21.15.22:0/3605777254 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4e74075680 msgr2=0x7f4e74073ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:06:31.069 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:31.069+0000 7f4e7b000640 1 --2- 172.21.15.22:0/3605777254 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4e74075680 0x7f4e74073ad0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f4e64009ca0 tx=0x7f4e6402c180 comp rx=0 tx=0).stop 2024-03-28T18:06:31.069 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:31.069+0000 7f4e7b000640 1 -- 172.21.15.22:0/3605777254 shutdown_connections 2024-03-28T18:06:31.070 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:31.069+0000 7f4e7b000640 1 --2- 172.21.15.22:0/3605777254 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4e74109540 0x7f4e7410b9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:06:31.070 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:31.069+0000 7f4e7b000640 1 --2- 172.21.15.22:0/3605777254 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f4e740740a0 0x7f4e74074500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:06:31.070 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:31.069+0000 7f4e7b000640 1 --2- 172.21.15.22:0/3605777254 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4e74075680 0x7f4e74073ad0 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:06:31.070 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:31.069+0000 7f4e7b000640 1 -- 172.21.15.22:0/3605777254 >> 172.21.15.22:0/3605777254 conn(0x7f4e740fcc40 msgr2=0x7f4e740ff060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:06:31.070 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:31.069+0000 7f4e7b000640 1 -- 172.21.15.22:0/3605777254 shutdown_connections 2024-03-28T18:06:31.070 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:31.069+0000 7f4e7b000640 1 -- 172.21.15.22:0/3605777254 wait complete. 2024-03-28T18:06:31.072 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:31.070+0000 7f4e7b000640 1 Processor -- start 2024-03-28T18:06:31.072 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:31.071+0000 7f4e7b000640 1 -- start start 2024-03-28T18:06:31.072 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:31.071+0000 7f4e7b000640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4e740740a0 0x7f4e74199be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:06:31.072 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:31.071+0000 7f4e7b000640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f4e74075680 0x7f4e7419a120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:06:31.072 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:31.071+0000 7f4e7b000640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4e74109540 0x7f4e741a1190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:06:31.072 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:31.071+0000 7f4e7b000640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e7410f510 con 0x7f4e740740a0 2024-03-28T18:06:31.072 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:31.071+0000 7f4e7b000640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e7410f390 con 0x7f4e74075680 2024-03-28T18:06:31.072 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:31.071+0000 7f4e7b000640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e7410f690 con 0x7f4e74109540 2024-03-28T18:06:31.072 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:31.071+0000 7f4e73e00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4e740740a0 0x7f4e74199be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:06:31.073 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:31.071+0000 7f4e73e00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4e740740a0 0x7f4e74199be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:49178/0 (socket says 172.21.15.22:49178) 2024-03-28T18:06:31.073 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:31.071+0000 7f4e73e00640 1 -- 172.21.15.22:0/1037137780 learned_addr learned my addr 172.21.15.22:0/1037137780 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-28T18:06:31.073 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:31.071+0000 7f4e6b400640 1 --2- 172.21.15.22:0/1037137780 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f4e74075680 0x7f4e7419a120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:06:31.073 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:31.072+0000 7f4e79000640 1 --2- 172.21.15.22:0/1037137780 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4e74109540 0x7f4e741a1190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:06:31.073 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:31.072+0000 7f4e73e00640 1 -- 172.21.15.22:0/1037137780 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4e74109540 msgr2=0x7f4e741a1190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:06:31.073 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:31.072+0000 7f4e73e00640 1 --2- 172.21.15.22:0/1037137780 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4e74109540 0x7f4e741a1190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:06:31.073 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:31.072+0000 7f4e73e00640 1 -- 172.21.15.22:0/1037137780 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f4e74075680 msgr2=0x7f4e7419a120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:06:31.073 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:31.072+0000 7f4e73e00640 1 --2- 172.21.15.22:0/1037137780 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f4e74075680 0x7f4e7419a120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:06:31.073 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:31.072+0000 7f4e73e00640 1 -- 172.21.15.22:0/1037137780 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4e741a1800 con 0x7f4e740740a0 2024-03-28T18:06:31.073 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:31.072+0000 7f4e73e00640 1 --2- 172.21.15.22:0/1037137780 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4e740740a0 0x7f4e74199be0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f4e64000f80 tx=0x7f4e64003c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:06:31.073 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:31.072+0000 7f4e71600640 1 -- 172.21.15.22:0/1037137780 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e64039050 con 0x7f4e740740a0 2024-03-28T18:06:31.073 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:31.072+0000 7f4e71600640 1 -- 172.21.15.22:0/1037137780 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f4e6403a040 con 0x7f4e740740a0 2024-03-28T18:06:31.073 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:31.072+0000 7f4e7b000640 1 -- 172.21.15.22:0/1037137780 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4e741a1a90 con 0x7f4e740740a0 2024-03-28T18:06:31.073 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:31.072+0000 7f4e71600640 1 -- 172.21.15.22:0/1037137780 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e64004790 con 0x7f4e740740a0 2024-03-28T18:06:31.074 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:31.072+0000 7f4e7b000640 1 -- 172.21.15.22:0/1037137780 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4e741a1f80 con 0x7f4e740740a0 2024-03-28T18:06:31.075 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:31.075+0000 7f4e7dde6740 1 Processor -- start 2024-03-28T18:06:31.075 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:31.076+0000 7f4e71600640 1 -- 172.21.15.22:0/1037137780 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 18) v1 ==== 137290+0+0 (secure 0 0 0) 0x7f4e64032b00 con 0x7f4e740740a0 2024-03-28T18:06:31.075 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:31.076+0000 7f4e7dde6740 1 -- start start 2024-03-28T18:06:31.077 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:31.076+0000 7f4e7dde6740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557498a1cba0 0x557498a1cfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:06:31.077 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:31.076+0000 7f4e7dde6740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557498a1d590 0x557498a22090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:06:31.077 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:31.076+0000 7f4e7dde6740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x557498a22700 0x557498a24b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:06:31.077 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:31.076+0000 7f4e7dde6740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x557498981c90 con 0x557498a1cba0 2024-03-28T18:06:31.077 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:31.076+0000 7f4e7dde6740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x55749892a980 con 0x557498a22700 2024-03-28T18:06:31.077 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:31.076+0000 7f4e7dde6740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55749897dd30 con 0x557498a1d590 2024-03-28T18:06:31.078 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:31.076+0000 7f4e79000640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557498a1cba0 0x557498a1cfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:06:31.078 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:31.076+0000 7f4e79000640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557498a1cba0 0x557498a1cfc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:49188/0 (socket says 172.21.15.22:49188) 2024-03-28T18:06:31.078 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:31.076+0000 7f4e79000640 1 -- 172.21.15.22:0/2102218406 learned_addr learned my addr 172.21.15.22:0/2102218406 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-28T18:06:31.078 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:31.077+0000 7f4e6b400640 1 --2- 172.21.15.22:0/2102218406 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x557498a22700 0x557498a24b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:06:31.078 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:31.077+0000 7f4e73e00640 1 --2- 172.21.15.22:0/2102218406 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557498a1d590 0x557498a22090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:06:31.078 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:31.077+0000 7f4e79000640 1 -- 172.21.15.22:0/2102218406 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557498a1d590 msgr2=0x557498a22090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T18:06:31.078 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:31.077+0000 7f4e79000640 1 --2- 172.21.15.22:0/2102218406 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557498a1d590 0x557498a22090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:06:31.078 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:31.077+0000 7f4e79000640 1 -- 172.21.15.22:0/2102218406 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x557498a22700 msgr2=0x557498a24b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T18:06:31.078 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:31.077+0000 7f4e79000640 1 --2- 172.21.15.22:0/2102218406 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x557498a22700 0x557498a24b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:06:31.078 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:31.077+0000 7f4e79000640 1 -- 172.21.15.22:0/2102218406 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557498942510 con 0x557498a1cba0 2024-03-28T18:06:31.078 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:31.077+0000 7f4e73e00640 1 --2- 172.21.15.22:0/2102218406 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557498a1d590 0x557498a22090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T18:06:31.078 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:31.077+0000 7f4e71600640 1 --2- 172.21.15.22:0/1037137780 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7f4e580a2800 0x7f4e580a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:06:31.078 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:31.077+0000 7f4e79000640 1 --2- 172.21.15.22:0/2102218406 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557498a1cba0 0x557498a1cfc0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f4e6c009b40 tx=0x7f4e6c00e2f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5c47dce6f2d17507 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:06:31.079 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:31.077+0000 7f4e71600640 1 -- 172.21.15.22:0/1037137780 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 8153+0+0 (secure 0 0 0) 0x7f4e640f1410 con 0x7f4e740740a0 2024-03-28T18:06:31.079 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:31.077+0000 7f4e73e00640 1 --2- 172.21.15.22:0/1037137780 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7f4e580a2800 0x7f4e580a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:06:31.079 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:31.078+0000 7f4e69600640 1 -- 172.21.15.22:0/2102218406 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e6c016070 con 0x557498a1cba0 2024-03-28T18:06:31.079 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:31.078+0000 7f4e69600640 1 -- 172.21.15.22:0/2102218406 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f4e6c00ed20 con 0x557498a1cba0 2024-03-28T18:06:31.079 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:31.078+0000 7f4e69600640 1 -- 172.21.15.22:0/2102218406 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e6c0107c0 con 0x557498a1cba0 2024-03-28T18:06:31.082 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:31.078+0000 7f4e7dde6740 1 -- 172.21.15.22:0/2102218406 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557498a1cba0 msgr2=0x557498a1cfc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:06:31.082 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:31.078+0000 7f4e73e00640 1 --2- 172.21.15.22:0/1037137780 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7f4e580a2800 0x7f4e580a4cc0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f4e640eef10 tx=0x7f4e6402c180 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:06:31.082 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:31.078+0000 7f4e7dde6740 1 --2- 172.21.15.22:0/2102218406 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557498a1cba0 0x557498a1cfc0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f4e6c009b40 tx=0x7f4e6c00e2f0 comp rx=0 tx=0).stop 2024-03-28T18:06:31.082 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:31.079+0000 7f4e7dde6740 1 -- 172.21.15.22:0/2102218406 shutdown_connections 2024-03-28T18:06:31.082 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:31.079+0000 7f4e7dde6740 1 --2- 172.21.15.22:0/2102218406 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x557498a22700 0x557498a24b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:06:31.082 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:31.079+0000 7f4e7dde6740 1 --2- 172.21.15.22:0/2102218406 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557498a1d590 0x557498a22090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:06:31.082 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:31.079+0000 7f4e7dde6740 1 --2- 172.21.15.22:0/2102218406 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557498a1cba0 0x557498a1cfc0 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:06:31.082 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:31.079+0000 7f4e7dde6740 1 -- 172.21.15.22:0/2102218406 >> 172.21.15.22:0/2102218406 conn(0x557498a18110 msgr2=0x557498a18510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:06:31.082 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:31.079+0000 7f4e7dde6740 1 -- 172.21.15.22:0/2102218406 shutdown_connections 2024-03-28T18:06:31.083 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:31.079+0000 7f4e7dde6740 1 -- 172.21.15.22:0/2102218406 wait complete. 2024-03-28T18:06:31.083 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:31.079+0000 7f4e7dde6740 1 -- start start 2024-03-28T18:06:31.083 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:31.079+0000 7f4e7dde6740 1 Processor -- start 2024-03-28T18:06:31.083 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:31.080+0000 7f4e7dde6740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557498a1cba0 0x557498a270e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:06:31.083 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:31.080+0000 7f4e7dde6740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x557498a1d590 0x557498a27620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:06:31.083 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:31.080+0000 7f4e7dde6740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557498a22700 0x557498a2e660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:06:31.083 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:31.080+0000 7f4e7dde6740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x557498a256b0 con 0x557498a1cba0 2024-03-28T18:06:31.083 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:31.080+0000 7f4e7dde6740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x55749893ddd0 con 0x557498a1d590 2024-03-28T18:06:31.083 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:31.080+0000 7f4e7dde6740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x557498a25830 con 0x557498a22700 2024-03-28T18:06:31.083 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:31.080+0000 7f4e7dde6740 10 client.0 _ll_register_callbacks cb 0x557498695ac0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-28T18:06:31.083 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:31.080+0000 7f4e79000640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557498a1cba0 0x557498a270e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:06:31.083 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:31.080+0000 7f4e79000640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557498a1cba0 0x557498a270e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:49190/0 (socket says 172.21.15.22:49190) 2024-03-28T18:06:31.083 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:31.080+0000 7f4e79000640 1 -- 172.21.15.22:0/1583572421 learned_addr learned my addr 172.21.15.22:0/1583572421 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-28T18:06:31.083 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:31.080+0000 7f4e6b400640 1 --2- 172.21.15.22:0/1583572421 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x557498a1d590 0x557498a27620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:06:31.084 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:31.080+0000 7f4e79000640 1 -- 172.21.15.22:0/1583572421 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557498a22700 msgr2=0x557498a2e660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T18:06:31.084 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:31.080+0000 7f4e79000640 1 --2- 172.21.15.22:0/1583572421 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557498a22700 0x557498a2e660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:06:31.084 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:31.081+0000 7f4e79000640 1 -- 172.21.15.22:0/1583572421 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x557498a1d590 msgr2=0x557498a27620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T18:06:31.084 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:31.081+0000 7f4e79000640 1 --2- 172.21.15.22:0/1583572421 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x557498a1d590 0x557498a27620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:06:31.084 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:31.081+0000 7f4e79000640 1 -- 172.21.15.22:0/1583572421 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5574986962d0 con 0x557498a1cba0 2024-03-28T18:06:31.084 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:31.081+0000 7f4e79000640 1 --2- 172.21.15.22:0/1583572421 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557498a1cba0 0x557498a270e0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f4e6c005750 tx=0x7f4e6c005780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=47eda97168695fb0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:06:31.084 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:31.081+0000 7f4e62000640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-03-28T18:06:31.084 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:31.081+0000 7f4e62000640 1 -- 172.21.15.22:0/1583572421 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e6c016070 con 0x557498a1cba0 2024-03-28T18:06:31.084 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:31.081+0000 7f4e62000640 1 -- 172.21.15.22:0/1583572421 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 8153+0+0 (secure 0 0 0) 0x7f4e6c004260 con 0x557498a1cba0 2024-03-28T18:06:31.084 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:31.081+0000 7f4e7dde6740 1 -- 172.21.15.22:0/1583572421 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f4e6c00ed20 con 0x557498a1cba0 2024-03-28T18:06:31.084 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:31.082+0000 7f4e62000640 1 -- 172.21.15.22:0/1583572421 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4e6c004460 con 0x557498a1cba0 2024-03-28T18:06:31.084 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:31.082+0000 7f4e7dde6740 10 client.15108 fetch_fsmap learned FSMap version 26 2024-03-28T18:06:31.084 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:31.082+0000 7f4e7dde6740 1 -- 172.21.15.22:0/1583572421 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x5574987606b0 con 0x557498a1cba0 2024-03-28T18:06:31.084 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:31.082+0000 7f4e62000640 1 -- 172.21.15.22:0/1583572421 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f4e6c00ed20 con 0x557498a1cba0 2024-03-28T18:06:31.084 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:31.082+0000 7f4e62000640 1 -- 172.21.15.22:0/1583572421 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 26) v1 ==== 7934+0+0 (secure 0 0 0) 0x7f4e6c00f040 con 0x557498a1cba0 2024-03-28T18:06:31.085 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:31.082+0000 7f4e7dde6740 10 client.15108 fetch_fsmap finished waiting for FSMap version 26 2024-03-28T18:06:31.085 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:31.082+0000 7f4e7dde6740 10 client.15108 resolve_mds: resolved name 'e' to daemon mds.e 2024-03-28T18:06:31.085 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:31.082+0000 7f4e7dde6740 20 client.15108 populate_metadata read hostname 'smithi022' 2024-03-28T18:06:31.085 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:31.082+0000 7f4e7dde6740 1 --2- 172.21.15.22:0/1583572421 >> [v2:172.21.15.79:6834/3365773431,v1:172.21.15.79:6835/3365773431] conn(0x557498a33570 0x557498a327e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:06:31.085 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:31.082+0000 7f4e7dde6740 4 client.15108 mds_command: new command op to 34475 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-28T18:06:31.085 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:31.082+0000 7f4e7dde6740 1 -- 172.21.15.22:0/1583572421 --> [v2:172.21.15.79:6834/3365773431,v1:172.21.15.79:6835/3365773431] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x557498923520 con 0x557498a33570 2024-03-28T18:06:31.086 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:31.083+0000 7f4e6b400640 1 --2- 172.21.15.22:0/1583572421 >> [v2:172.21.15.79:6834/3365773431,v1:172.21.15.79:6835/3365773431] conn(0x557498a33570 0x557498a327e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:06:31.086 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:31.083+0000 7f4e6b400640 1 --2- 172.21.15.22:0/1583572421 >> [v2:172.21.15.79:6834/3365773431,v1:172.21.15.79:6835/3365773431] conn(0x557498a33570 0x557498a327e0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=9bc81d69eee829fa server_cookie=e85ba0fc8de17ef7 in_seq=0 out_seq=0 2024-03-28T18:06:31.086 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:31.083+0000 7f4e62000640 10 client.15108 ms_handle_connect on v2:172.21.15.79:6834/3365773431 2024-03-28T18:06:31.086 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:31.085+0000 7f4e62000640 1 -- 172.21.15.22:0/1583572421 <== mds.2 v2:172.21.15.79:6834/3365773431 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x557498923520 con 0x557498a33570 2024-03-28T18:06:31.086 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:31.085+0000 7f4e62000640 10 client.15108 handle_command_reply: tid=0 2024-03-28T18:06:31.087 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:31.086+0000 7f4e7dde6740 1 client.15108 shutdown 2024-03-28T18:06:31.087 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:31.086+0000 7f4e7dde6740 1 -- 172.21.15.22:0/1583572421 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557498a1cba0 msgr2=0x557498a270e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:06:31.087 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:31.086+0000 7f4e7dde6740 1 --2- 172.21.15.22:0/1583572421 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557498a1cba0 0x557498a270e0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f4e6c005750 tx=0x7f4e6c005780 comp rx=0 tx=0).stop 2024-03-28T18:06:31.087 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:31.086+0000 7f4e7dde6740 1 -- 172.21.15.22:0/1583572421 shutdown_connections 2024-03-28T18:06:31.087 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:31.086+0000 7f4e7dde6740 1 --2- 172.21.15.22:0/1583572421 >> [v2:172.21.15.79:6834/3365773431,v1:172.21.15.79:6835/3365773431] conn(0x557498a33570 0x557498a327e0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:06:31.087 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:31.086+0000 7f4e7dde6740 1 --2- 172.21.15.22:0/1583572421 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557498a22700 0x557498a2e660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:06:31.087 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:31.086+0000 7f4e7dde6740 1 --2- 172.21.15.22:0/1583572421 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x557498a1d590 0x557498a27620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:06:31.087 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:31.086+0000 7f4e7dde6740 1 --2- 172.21.15.22:0/1583572421 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557498a1cba0 0x557498a270e0 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:06:31.087 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:31.086+0000 7f4e62000640 0 client.15108 ms_handle_reset on v2:172.21.15.79:6834/3365773431 2024-03-28T18:06:31.087 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:31.086+0000 7f4e7dde6740 1 -- 172.21.15.22:0/1583572421 >> 172.21.15.22:0/1583572421 conn(0x557498a18110 msgr2=0x557498a17310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:06:31.087 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:31.086+0000 7f4e7dde6740 1 -- 172.21.15.22:0/1583572421 shutdown_connections 2024-03-28T18:06:31.087 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:31.086+0000 7f4e7dde6740 1 -- 172.21.15.22:0/1583572421 wait complete. 2024-03-28T18:06:31.087 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:31.086+0000 7f4e7dde6740 20 client.15108 trim_cache size 0 max 16384 2024-03-28T18:06:31.105 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:31.105+0000 7f4e7dde6740 1 Processor -- start 2024-03-28T18:06:31.105 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:31.105+0000 7f4e7dde6740 1 -- start start 2024-03-28T18:06:31.105 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:31.105+0000 7f4e7dde6740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557498a1cba0 0x557498a1ba80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:06:31.106 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:31.105+0000 7f4e7dde6740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557498a1d590 0x557498a2e5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:06:31.106 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:31.105+0000 7f4e7dde6740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x557498a22700 0x557498a3ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:06:31.106 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:31.105+0000 7f4e7dde6740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x557498a32d90 con 0x557498a1d590 2024-03-28T18:06:31.106 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:31.105+0000 7f4e7dde6740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x557498a333f0 con 0x557498a22700 2024-03-28T18:06:31.106 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:31.105+0000 7f4e7dde6740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x557498a33a50 con 0x557498a1cba0 2024-03-28T18:06:31.106 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:31.105+0000 7f4e6b400640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557498a1d590 0x557498a2e5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:06:31.106 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:31.105+0000 7f4e6b400640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557498a1d590 0x557498a2e5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:49202/0 (socket says 172.21.15.22:49202) 2024-03-28T18:06:31.106 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:31.105+0000 7f4e6b400640 1 -- 172.21.15.22:0/4032250932 learned_addr learned my addr 172.21.15.22:0/4032250932 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-28T18:06:31.106 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:31.105+0000 7f4e79000640 1 --2- 172.21.15.22:0/4032250932 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x557498a22700 0x557498a3ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:06:31.106 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:31.106+0000 7f4e6b400640 1 -- 172.21.15.22:0/4032250932 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557498a1cba0 msgr2=0x557498a1ba80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T18:06:31.106 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:31.106+0000 7f4e6b400640 1 --2- 172.21.15.22:0/4032250932 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557498a1cba0 0x557498a1ba80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:06:31.106 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:31.106+0000 7f4e6b400640 1 -- 172.21.15.22:0/4032250932 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x557498a22700 msgr2=0x557498a3ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T18:06:31.106 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:31.106+0000 7f4e6b400640 1 --2- 172.21.15.22:0/4032250932 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x557498a22700 0x557498a3ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:06:31.106 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:31.106+0000 7f4e6b400640 1 -- 172.21.15.22:0/4032250932 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557498a18e80 con 0x557498a1d590 2024-03-28T18:06:31.107 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:31.106+0000 7f4e79000640 1 --2- 172.21.15.22:0/4032250932 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x557498a22700 0x557498a3ba10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-28T18:06:31.107 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:31.106+0000 7f4e6b400640 1 --2- 172.21.15.22:0/4032250932 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557498a1d590 0x557498a2e5f0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x557498981d20 tx=0x7f4e5c005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9f4ac0144be8725d server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:06:31.109 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:31.106+0000 7f4e68c00640 1 -- 172.21.15.22:0/4032250932 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e5c026070 con 0x557498a1d590 2024-03-28T18:06:31.109 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:31.106+0000 7f4e68c00640 1 -- 172.21.15.22:0/4032250932 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f4e5c012670 con 0x557498a1d590 2024-03-28T18:06:31.109 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:31.106+0000 7f4e68c00640 1 -- 172.21.15.22:0/4032250932 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e5c003850 con 0x557498a1d590 2024-03-28T18:06:31.109 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:31.107+0000 7f4e7dde6740 1 -- 172.21.15.22:0/4032250932 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557498a1d590 msgr2=0x557498a2e5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:06:31.109 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:31.107+0000 7f4e7dde6740 1 --2- 172.21.15.22:0/4032250932 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557498a1d590 0x557498a2e5f0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x557498981d20 tx=0x7f4e5c005f90 comp rx=0 tx=0).stop 2024-03-28T18:06:31.109 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:31.107+0000 7f4e7dde6740 1 -- 172.21.15.22:0/4032250932 shutdown_connections 2024-03-28T18:06:31.109 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:31.107+0000 7f4e7dde6740 1 --2- 172.21.15.22:0/4032250932 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x557498a22700 0x557498a3ba10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:06:31.110 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:31.107+0000 7f4e7dde6740 1 --2- 172.21.15.22:0/4032250932 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557498a1d590 0x557498a2e5f0 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:06:31.110 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:31.107+0000 7f4e7dde6740 1 --2- 172.21.15.22:0/4032250932 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557498a1cba0 0x557498a1ba80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:06:31.110 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:31.107+0000 7f4e7dde6740 1 -- 172.21.15.22:0/4032250932 >> 172.21.15.22:0/4032250932 conn(0x557498a18110 msgr2=0x557498a1b370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:06:31.110 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:31.107+0000 7f4e7dde6740 1 -- 172.21.15.22:0/4032250932 shutdown_connections 2024-03-28T18:06:31.110 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:31.107+0000 7f4e7dde6740 1 -- 172.21.15.22:0/4032250932 wait complete. 2024-03-28T18:06:31.110 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:31.107+0000 7f4e7dde6740 1 -- start start 2024-03-28T18:06:31.110 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:31.108+0000 7f4e7dde6740 1 Processor -- start 2024-03-28T18:06:31.110 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:31.108+0000 7f4e7dde6740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557498a1cba0 0x557498a2b6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:06:31.110 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:31.108+0000 7f4e7dde6740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557498a1d590 0x557498a2bbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:06:31.110 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:31.108+0000 7f4e7dde6740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x557498a22700 0x557498a96de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:06:31.110 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:31.108+0000 7f4e7dde6740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x557498a34710 con 0x557498a1d590 2024-03-28T18:06:31.110 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:31.108+0000 7f4e7dde6740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x557498a340b0 con 0x557498a22700 2024-03-28T18:06:31.110 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:31.108+0000 7f4e7dde6740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x557498a34d70 con 0x557498a1cba0 2024-03-28T18:06:31.111 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:31.108+0000 7f4e7dde6740 10 client.0 _ll_register_callbacks cb 0x7f4e5c009590 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-28T18:06:31.111 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:31.108+0000 7f4e6b400640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557498a1d590 0x557498a2bbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:06:31.111 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:31.108+0000 7f4e6b400640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557498a1d590 0x557498a2bbe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:49218/0 (socket says 172.21.15.22:49218) 2024-03-28T18:06:31.111 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:31.108+0000 7f4e6b400640 1 -- 172.21.15.22:0/2911656603 learned_addr learned my addr 172.21.15.22:0/2911656603 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-28T18:06:31.111 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:31.109+0000 7f4e79000640 1 --2- 172.21.15.22:0/2911656603 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x557498a22700 0x557498a96de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:06:31.111 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:31.109+0000 7f4e6b400640 1 -- 172.21.15.22:0/2911656603 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557498a1cba0 msgr2=0x557498a2b6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T18:06:31.111 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:31.109+0000 7f4e6b400640 1 --2- 172.21.15.22:0/2911656603 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557498a1cba0 0x557498a2b6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:06:31.111 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:31.109+0000 7f4e6b400640 1 -- 172.21.15.22:0/2911656603 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x557498a22700 msgr2=0x557498a96de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T18:06:31.111 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:31.109+0000 7f4e6b400640 1 --2- 172.21.15.22:0/2911656603 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x557498a22700 0x557498a96de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:06:31.111 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:31.109+0000 7f4e6b400640 1 -- 172.21.15.22:0/2911656603 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x557498a97350 con 0x557498a1d590 2024-03-28T18:06:31.111 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:31.109+0000 7f4e79000640 1 --2- 172.21.15.22:0/2911656603 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x557498a22700 0x557498a96de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T18:06:31.111 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:31.109+0000 7f4e6b400640 1 --2- 172.21.15.22:0/2911656603 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557498a1d590 0x557498a2bbe0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f4e5c005d80 tx=0x7f4e5c009b10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1d6b6489f57e0db8 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:06:31.111 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:31.109+0000 7f4e62000640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-03-28T18:06:31.111 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:31.109+0000 7f4e62000640 1 -- 172.21.15.22:0/2911656603 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e5c026040 con 0x557498a1d590 2024-03-28T18:06:31.111 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:31.109+0000 7f4e62000640 1 -- 172.21.15.22:0/2911656603 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 8153+0+0 (secure 0 0 0) 0x7f4e5c004250 con 0x557498a1d590 2024-03-28T18:06:31.112 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:31.109+0000 7f4e7dde6740 1 -- 172.21.15.22:0/2911656603 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f4e5c012670 con 0x557498a1d590 2024-03-28T18:06:31.112 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:31.110+0000 7f4e62000640 1 -- 172.21.15.22:0/2911656603 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4e5c012820 con 0x557498a1d590 2024-03-28T18:06:31.112 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:31.110+0000 7f4e7dde6740 10 client.15114 fetch_fsmap learned FSMap version 26 2024-03-28T18:06:31.112 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:31.110+0000 7f4e7dde6740 1 -- 172.21.15.22:0/2911656603 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x557498a98630 con 0x557498a1d590 2024-03-28T18:06:31.113 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:31.110+0000 7f4e62000640 1 -- 172.21.15.22:0/2911656603 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f4e5c012670 con 0x557498a1d590 2024-03-28T18:06:31.114 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:31.110+0000 7f4e62000640 1 -- 172.21.15.22:0/2911656603 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 26) v1 ==== 7934+0+0 (secure 0 0 0) 0x7f4e5c033030 con 0x557498a1d590 2024-03-28T18:06:31.114 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:31.110+0000 7f4e7dde6740 10 client.15114 fetch_fsmap finished waiting for FSMap version 26 2024-03-28T18:06:31.114 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:31.110+0000 7f4e7dde6740 10 client.15114 resolve_mds: resolved name 'e' to daemon mds.e 2024-03-28T18:06:31.114 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:31.110+0000 7f4e7dde6740 20 client.15114 populate_metadata read hostname 'smithi022' 2024-03-28T18:06:31.114 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:31.110+0000 7f4e7dde6740 1 --2- 172.21.15.22:0/2911656603 >> [v2:172.21.15.79:6834/3365773431,v1:172.21.15.79:6835/3365773431] conn(0x557498a9c9f0 0x557498a98910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:06:31.114 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:31.110+0000 7f4e7dde6740 4 client.15114 mds_command: new command op to 34475 tid=0 multi_id=0 [{"prefix": "status"}] 2024-03-28T18:06:31.114 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:31.110+0000 7f4e7dde6740 1 -- 172.21.15.22:0/2911656603 --> [v2:172.21.15.79:6834/3365773431,v1:172.21.15.79:6835/3365773431] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55749898f1d0 con 0x557498a9c9f0 2024-03-28T18:06:31.114 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:31.111+0000 7f4e79000640 1 --2- 172.21.15.22:0/2911656603 >> [v2:172.21.15.79:6834/3365773431,v1:172.21.15.79:6835/3365773431] conn(0x557498a9c9f0 0x557498a98910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:06:31.114 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:31.111+0000 7f4e79000640 1 --2- 172.21.15.22:0/2911656603 >> [v2:172.21.15.79:6834/3365773431,v1:172.21.15.79:6835/3365773431] conn(0x557498a9c9f0 0x557498a98910 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=2e3f4f4543ddebae server_cookie=4dd8f83787bd7cfe in_seq=0 out_seq=0 2024-03-28T18:06:31.114 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:31.111+0000 7f4e62000640 10 client.15114 ms_handle_connect on v2:172.21.15.79:6834/3365773431 2024-03-28T18:06:31.114 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:31.112+0000 7f4e62000640 1 -- 172.21.15.22:0/2911656603 <== mds.2 v2:172.21.15.79:6834/3365773431 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+332 (crc 0 0 0) 0x55749898f1d0 con 0x557498a9c9f0 2024-03-28T18:06:31.114 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:31.112+0000 7f4e62000640 10 client.15114 handle_command_reply: tid=0 2024-03-28T18:06:31.114 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:31.112+0000 7f4e7dde6740 1 client.15114 shutdown 2024-03-28T18:06:31.114 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:31.112+0000 7f4e7dde6740 1 -- 172.21.15.22:0/2911656603 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557498a1d590 msgr2=0x557498a2bbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:06:31.115 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:31.112+0000 7f4e7dde6740 1 --2- 172.21.15.22:0/2911656603 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557498a1d590 0x557498a2bbe0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f4e5c005d80 tx=0x7f4e5c009b10 comp rx=0 tx=0).stop 2024-03-28T18:06:31.115 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:31.112+0000 7f4e7dde6740 1 -- 172.21.15.22:0/2911656603 shutdown_connections 2024-03-28T18:06:31.115 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:31.112+0000 7f4e7dde6740 1 --2- 172.21.15.22:0/2911656603 >> [v2:172.21.15.79:6834/3365773431,v1:172.21.15.79:6835/3365773431] conn(0x557498a9c9f0 0x557498a98910 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:06:31.115 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:31.112+0000 7f4e7dde6740 1 --2- 172.21.15.22:0/2911656603 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x557498a22700 0x557498a96de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:06:31.115 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:31.112+0000 7f4e7dde6740 1 --2- 172.21.15.22:0/2911656603 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557498a1d590 0x557498a2bbe0 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:06:31.115 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:31.112+0000 7f4e7dde6740 1 --2- 172.21.15.22:0/2911656603 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557498a1cba0 0x557498a2b6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:06:31.115 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:31.112+0000 7f4e62000640 0 client.15114 ms_handle_reset on v2:172.21.15.79:6834/3365773431 2024-03-28T18:06:31.115 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:31.112+0000 7f4e7dde6740 1 -- 172.21.15.22:0/2911656603 >> 172.21.15.22:0/2911656603 conn(0x557498a18110 msgr2=0x557498a1bc00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:06:31.115 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:31.112+0000 7f4e7dde6740 1 -- 172.21.15.22:0/2911656603 shutdown_connections 2024-03-28T18:06:31.115 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:31.112+0000 7f4e7dde6740 1 -- 172.21.15.22:0/2911656603 wait complete. 2024-03-28T18:06:31.115 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:31.112+0000 7f4e7dde6740 20 client.15114 trim_cache size 0 max 16384 2024-03-28T18:06:31.115 INFO:teuthology.orchestra.run.smithi022.stdout:{ 2024-03-28T18:06:31.116 INFO:teuthology.orchestra.run.smithi022.stdout: "cluster_fsid": "fd220748-ed2b-11ee-95d0-87774f69a715", 2024-03-28T18:06:31.116 INFO:teuthology.orchestra.run.smithi022.stdout: "whoami": 2, 2024-03-28T18:06:31.116 INFO:teuthology.orchestra.run.smithi022.stdout: "id": 34475, 2024-03-28T18:06:31.116 INFO:teuthology.orchestra.run.smithi022.stdout: "want_state": "up:active", 2024-03-28T18:06:31.116 INFO:teuthology.orchestra.run.smithi022.stdout: "state": "up:active", 2024-03-28T18:06:31.116 INFO:teuthology.orchestra.run.smithi022.stdout: "fs_name": "cephfs", 2024-03-28T18:06:31.116 INFO:teuthology.orchestra.run.smithi022.stdout: "rank_uptime": 97.516876046999997, 2024-03-28T18:06:31.116 INFO:teuthology.orchestra.run.smithi022.stdout: "mdsmap_epoch": 26, 2024-03-28T18:06:31.116 INFO:teuthology.orchestra.run.smithi022.stdout: "osdmap_epoch": 92, 2024-03-28T18:06:31.116 INFO:teuthology.orchestra.run.smithi022.stdout: "osdmap_epoch_barrier": 90, 2024-03-28T18:06:31.116 INFO:teuthology.orchestra.run.smithi022.stdout: "uptime": 134.68881687800001 2024-03-28T18:06:31.116 INFO:teuthology.orchestra.run.smithi022.stdout:} 2024-03-28T18:06:31.117 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:31.113+0000 7f4e7b000640 1 -- 172.21.15.22:0/1037137780 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7f4e580a2800 msgr2=0x7f4e580a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:06:31.117 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:31.113+0000 7f4e7b000640 1 --2- 172.21.15.22:0/1037137780 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7f4e580a2800 0x7f4e580a4cc0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f4e640eef10 tx=0x7f4e6402c180 comp rx=0 tx=0).stop 2024-03-28T18:06:31.117 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:31.113+0000 7f4e7b000640 1 -- 172.21.15.22:0/1037137780 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4e740740a0 msgr2=0x7f4e74199be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:06:31.117 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:31.113+0000 7f4e7b000640 1 --2- 172.21.15.22:0/1037137780 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4e740740a0 0x7f4e74199be0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f4e64000f80 tx=0x7f4e64003c20 comp rx=0 tx=0).stop 2024-03-28T18:06:31.117 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:31.113+0000 7f4e7b000640 1 -- 172.21.15.22:0/1037137780 shutdown_connections 2024-03-28T18:06:31.117 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:31.113+0000 7f4e7b000640 1 --2- 172.21.15.22:0/1037137780 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7f4e580a2800 0x7f4e580a4cc0 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:06:31.117 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:31.113+0000 7f4e7b000640 1 --2- 172.21.15.22:0/1037137780 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4e74109540 0x7f4e741a1190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:06:31.117 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:31.113+0000 7f4e7b000640 1 --2- 172.21.15.22:0/1037137780 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f4e74075680 0x7f4e7419a120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:06:31.118 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:31.113+0000 7f4e7b000640 1 --2- 172.21.15.22:0/1037137780 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4e740740a0 0x7f4e74199be0 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:06:31.118 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:31.113+0000 7f4e7b000640 1 -- 172.21.15.22:0/1037137780 >> 172.21.15.22:0/1037137780 conn(0x7f4e740fcc40 msgr2=0x7f4e741054d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:06:31.118 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:31.113+0000 7f4e7b000640 1 -- 172.21.15.22:0/1037137780 shutdown_connections 2024-03-28T18:06:31.118 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:31.113+0000 7f4e7b000640 1 -- 172.21.15.22:0/1037137780 wait complete. 2024-03-28T18:06:31.131 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.h status 2024-03-28T18:06:31.193 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:06:31 smithi022 ceph-mon[31985]: from='client.? 172.21.15.22:0/4244182015' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-28T18:06:31.337 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:31.337+0000 7f4d37a00640 1 -- 172.21.15.22:0/1779004849 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4d30107390 msgr2=0x7f4d301097e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:06:31.338 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:31.337+0000 7f4d37a00640 1 --2- 172.21.15.22:0/1779004849 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4d30107390 0x7f4d301097e0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f4d28009900 tx=0x7f4d2802bc30 comp rx=0 tx=0).stop 2024-03-28T18:06:31.338 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:31.338+0000 7f4d37a00640 1 -- 172.21.15.22:0/1779004849 shutdown_connections 2024-03-28T18:06:31.338 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:31.338+0000 7f4d37a00640 1 --2- 172.21.15.22:0/1779004849 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4d30107390 0x7f4d301097e0 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:06:31.338 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:31.338+0000 7f4d37a00640 1 --2- 172.21.15.22:0/1779004849 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f4d300ffd00 0x7f4d30106e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:06:31.338 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:31.338+0000 7f4d37a00640 1 --2- 172.21.15.22:0/1779004849 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4d300ff310 0x7f4d300ff730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:06:31.338 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:31.338+0000 7f4d37a00640 1 -- 172.21.15.22:0/1779004849 >> 172.21.15.22:0/1779004849 conn(0x7f4d300fae60 msgr2=0x7f4d300fd2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:06:31.338 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:31.338+0000 7f4d37a00640 1 -- 172.21.15.22:0/1779004849 shutdown_connections 2024-03-28T18:06:31.338 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:31.338+0000 7f4d37a00640 1 -- 172.21.15.22:0/1779004849 wait complete. 2024-03-28T18:06:31.339 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:31.339+0000 7f4d37a00640 1 Processor -- start 2024-03-28T18:06:31.339 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:31.339+0000 7f4d37a00640 1 -- start start 2024-03-28T18:06:31.340 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:31.340+0000 7f4d37a00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4d300ff310 0x7f4d30199bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:06:31.340 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:31.340+0000 7f4d37a00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4d300ffd00 0x7f4d3019a130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:06:31.340 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:31.340+0000 7f4d37a00640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f4d30107390 0x7f4d301a1150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:06:31.340 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:31.340+0000 7f4d37a00640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d3010d310 con 0x7f4d300ffd00 2024-03-28T18:06:31.340 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:31.340+0000 7f4d37a00640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d3010d190 con 0x7f4d30107390 2024-03-28T18:06:31.341 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:31.340+0000 7f4d37a00640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d3010d490 con 0x7f4d300ff310 2024-03-28T18:06:31.341 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:31.340+0000 7f4d2fe00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4d300ffd00 0x7f4d3019a130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:06:31.341 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:31.340+0000 7f4d2fe00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4d300ffd00 0x7f4d3019a130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:49232/0 (socket says 172.21.15.22:49232) 2024-03-28T18:06:31.341 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:31.340+0000 7f4d2fe00640 1 -- 172.21.15.22:0/3300031523 learned_addr learned my addr 172.21.15.22:0/3300031523 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-28T18:06:31.341 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:31.340+0000 7f4d35a00640 1 --2- 172.21.15.22:0/3300031523 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f4d30107390 0x7f4d301a1150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:06:31.341 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:31.340+0000 7f4d35000640 1 --2- 172.21.15.22:0/3300031523 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4d300ff310 0x7f4d30199bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:06:31.341 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:31.341+0000 7f4d2fe00640 1 -- 172.21.15.22:0/3300031523 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4d300ff310 msgr2=0x7f4d30199bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:06:31.341 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:31.341+0000 7f4d2fe00640 1 --2- 172.21.15.22:0/3300031523 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4d300ff310 0x7f4d30199bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:06:31.341 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:31.341+0000 7f4d2fe00640 1 -- 172.21.15.22:0/3300031523 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f4d30107390 msgr2=0x7f4d301a1150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:06:31.341 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:31.341+0000 7f4d2fe00640 1 --2- 172.21.15.22:0/3300031523 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f4d30107390 0x7f4d301a1150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:06:31.341 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:31.341+0000 7f4d2fe00640 1 -- 172.21.15.22:0/3300031523 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4d301a17c0 con 0x7f4d300ffd00 2024-03-28T18:06:31.341 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:31.341+0000 7f4d2fe00640 1 --2- 172.21.15.22:0/3300031523 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4d300ffd00 0x7f4d3019a130 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f4d2000ebc0 tx=0x7f4d2000c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:06:31.342 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:31.341+0000 7f4d2d600640 1 -- 172.21.15.22:0/3300031523 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d20010630 con 0x7f4d300ffd00 2024-03-28T18:06:31.342 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:31.341+0000 7f4d2d600640 1 -- 172.21.15.22:0/3300031523 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f4d20010ca0 con 0x7f4d300ffd00 2024-03-28T18:06:31.342 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:31.341+0000 7f4d37a00640 1 -- 172.21.15.22:0/3300031523 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4d301a1ab0 con 0x7f4d300ffd00 2024-03-28T18:06:31.342 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:31.341+0000 7f4d2d600640 1 -- 172.21.15.22:0/3300031523 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d20018c00 con 0x7f4d300ffd00 2024-03-28T18:06:31.342 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:31.341+0000 7f4d37a00640 1 -- 172.21.15.22:0/3300031523 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4d301a2050 con 0x7f4d300ffd00 2024-03-28T18:06:31.345 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:31.345+0000 7f4d2d600640 1 -- 172.21.15.22:0/3300031523 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 18) v1 ==== 137290+0+0 (secure 0 0 0) 0x7f4d200107d0 con 0x7f4d300ffd00 2024-03-28T18:06:31.345 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:31.345+0000 7f4d3a7ac740 1 Processor -- start 2024-03-28T18:06:31.345 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:31.345+0000 7f4d3a7ac740 1 -- start start 2024-03-28T18:06:31.345 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:31.345+0000 7f4d3a7ac740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5627289e0ba0 0x5627289e0fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:06:31.345 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:31.345+0000 7f4d3a7ac740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5627289e1590 0x5627289e6090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:06:31.345 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:31.345+0000 7f4d3a7ac740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5627289e6700 0x5627289e8b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:06:31.345 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:31.345+0000 7f4d3a7ac740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x562728945c90 con 0x5627289e0ba0 2024-03-28T18:06:31.346 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:31.345+0000 7f4d3a7ac740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x5627288ee980 con 0x5627289e6700 2024-03-28T18:06:31.346 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:31.345+0000 7f4d3a7ac740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x562728941d30 con 0x5627289e1590 2024-03-28T18:06:31.346 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:31.345+0000 7f4d35a00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5627289e0ba0 0x5627289e0fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:06:31.346 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:31.345+0000 7f4d35a00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5627289e0ba0 0x5627289e0fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:49238/0 (socket says 172.21.15.22:49238) 2024-03-28T18:06:31.346 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:31.345+0000 7f4d35a00640 1 -- 172.21.15.22:0/735294238 learned_addr learned my addr 172.21.15.22:0/735294238 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-28T18:06:31.346 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:31.346+0000 7f4d2fe00640 1 --2- 172.21.15.22:0/735294238 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5627289e6700 0x5627289e8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:06:31.346 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:31.346+0000 7f4d35000640 1 --2- 172.21.15.22:0/735294238 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5627289e1590 0x5627289e6090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:06:31.348 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:31.346+0000 7f4d35a00640 1 -- 172.21.15.22:0/735294238 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5627289e1590 msgr2=0x5627289e6090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T18:06:31.348 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:31.346+0000 7f4d35a00640 1 --2- 172.21.15.22:0/735294238 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5627289e1590 0x5627289e6090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:06:31.348 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:31.346+0000 7f4d35a00640 1 -- 172.21.15.22:0/735294238 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5627289e6700 msgr2=0x5627289e8b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T18:06:31.348 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:31.346+0000 7f4d35a00640 1 --2- 172.21.15.22:0/735294238 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5627289e6700 0x5627289e8b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:06:31.348 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:31.346+0000 7f4d35a00640 1 -- 172.21.15.22:0/735294238 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562728906510 con 0x5627289e0ba0 2024-03-28T18:06:31.348 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:31.346+0000 7f4d2fe00640 1 --2- 172.21.15.22:0/735294238 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5627289e6700 0x5627289e8b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-28T18:06:31.348 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:31.346+0000 7f4d2d600640 1 --2- 172.21.15.22:0/3300031523 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7f4d140a2880 0x7f4d140a4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:06:31.348 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:31.346+0000 7f4d35a00640 1 --2- 172.21.15.22:0/735294238 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5627289e0ba0 0x5627289e0fc0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f4d30100260 tx=0x7f4d28009610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2dff4288ead60915 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:06:31.349 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:31.346+0000 7f4d2d600640 1 -- 172.21.15.22:0/3300031523 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 8153+0+0 (secure 0 0 0) 0x7f4d200cf0b0 con 0x7f4d300ffd00 2024-03-28T18:06:31.349 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:31.346+0000 7f4d35000640 1 --2- 172.21.15.22:0/3300031523 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7f4d140a2880 0x7f4d140a4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:06:31.349 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:31.347+0000 7f4d26000640 1 -- 172.21.15.22:0/735294238 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d2802c9e0 con 0x5627289e0ba0 2024-03-28T18:06:31.349 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:31.347+0000 7f4d26000640 1 -- 172.21.15.22:0/735294238 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f4d2802cb80 con 0x5627289e0ba0 2024-03-28T18:06:31.349 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:31.347+0000 7f4d26000640 1 -- 172.21.15.22:0/735294238 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d28005620 con 0x5627289e0ba0 2024-03-28T18:06:31.349 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:31.347+0000 7f4d3a7ac740 1 -- 172.21.15.22:0/735294238 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5627289e0ba0 msgr2=0x5627289e0fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:06:31.349 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:31.347+0000 7f4d35000640 1 --2- 172.21.15.22:0/3300031523 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7f4d140a2880 0x7f4d140a4d40 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f4d18005fd0 tx=0x7f4d1800b650 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:06:31.349 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:31.347+0000 7f4d3a7ac740 1 --2- 172.21.15.22:0/735294238 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5627289e0ba0 0x5627289e0fc0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f4d30100260 tx=0x7f4d28009610 comp rx=0 tx=0).stop 2024-03-28T18:06:31.349 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:31.348+0000 7f4d3a7ac740 1 -- 172.21.15.22:0/735294238 shutdown_connections 2024-03-28T18:06:31.349 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:31.348+0000 7f4d3a7ac740 1 --2- 172.21.15.22:0/735294238 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5627289e6700 0x5627289e8b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:06:31.349 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:31.348+0000 7f4d3a7ac740 1 --2- 172.21.15.22:0/735294238 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5627289e1590 0x5627289e6090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:06:31.349 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:31.348+0000 7f4d3a7ac740 1 --2- 172.21.15.22:0/735294238 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5627289e0ba0 0x5627289e0fc0 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:06:31.349 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:31.348+0000 7f4d3a7ac740 1 -- 172.21.15.22:0/735294238 >> 172.21.15.22:0/735294238 conn(0x5627289dc110 msgr2=0x5627289dc510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:06:31.349 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:31.348+0000 7f4d3a7ac740 1 -- 172.21.15.22:0/735294238 shutdown_connections 2024-03-28T18:06:31.349 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:31.348+0000 7f4d3a7ac740 1 -- 172.21.15.22:0/735294238 wait complete. 2024-03-28T18:06:31.350 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:31.348+0000 7f4d3a7ac740 1 -- start start 2024-03-28T18:06:31.350 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:31.348+0000 7f4d3a7ac740 1 Processor -- start 2024-03-28T18:06:31.352 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:31.349+0000 7f4d3a7ac740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5627289e0ba0 0x5627289eb110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:06:31.352 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:31.349+0000 7f4d3a7ac740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5627289e1590 0x5627289eb650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:06:31.353 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:31.349+0000 7f4d3a7ac740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5627289e6700 0x5627289f2690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:06:31.353 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:31.349+0000 7f4d3a7ac740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x5627289e9650 con 0x5627289e0ba0 2024-03-28T18:06:31.353 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:31.349+0000 7f4d3a7ac740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x562728901dd0 con 0x5627289e6700 2024-03-28T18:06:31.353 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:31.349+0000 7f4d3a7ac740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x5627289e97d0 con 0x5627289e1590 2024-03-28T18:06:31.353 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:31.349+0000 7f4d3a7ac740 10 client.0 _ll_register_callbacks cb 0x562728659ac0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-28T18:06:31.353 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:31.349+0000 7f4d35000640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5627289e0ba0 0x5627289eb110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:06:31.353 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:31.349+0000 7f4d35000640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5627289e0ba0 0x5627289eb110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:49250/0 (socket says 172.21.15.22:49250) 2024-03-28T18:06:31.353 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:31.349+0000 7f4d35000640 1 -- 172.21.15.22:0/120448542 learned_addr learned my addr 172.21.15.22:0/120448542 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-28T18:06:31.353 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:31.349+0000 7f4d35a00640 1 --2- 172.21.15.22:0/120448542 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5627289e6700 0x5627289f2690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:06:31.353 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:31.350+0000 7f4d2fe00640 1 --2- 172.21.15.22:0/120448542 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5627289e1590 0x5627289eb650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:06:31.353 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:31.350+0000 7f4d35000640 1 -- 172.21.15.22:0/120448542 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5627289e1590 msgr2=0x5627289eb650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T18:06:31.353 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:31.350+0000 7f4d35000640 1 --2- 172.21.15.22:0/120448542 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5627289e1590 0x5627289eb650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:06:31.353 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:31.350+0000 7f4d35000640 1 -- 172.21.15.22:0/120448542 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5627289e6700 msgr2=0x5627289f2690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T18:06:31.353 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:31.350+0000 7f4d35000640 1 --2- 172.21.15.22:0/120448542 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5627289e6700 0x5627289f2690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:06:31.354 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:31.350+0000 7f4d35000640 1 -- 172.21.15.22:0/120448542 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56272865a2d0 con 0x5627289e0ba0 2024-03-28T18:06:31.354 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:31.350+0000 7f4d35000640 1 --2- 172.21.15.22:0/120448542 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5627289e0ba0 0x5627289eb110 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f4d18012010 tx=0x7f4d180046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2dc9d3ae83aa2e35 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:06:31.354 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:31.350+0000 7f4d1ea00640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-03-28T18:06:31.354 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:31.350+0000 7f4d1ea00640 1 -- 172.21.15.22:0/120448542 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d18004230 con 0x5627289e0ba0 2024-03-28T18:06:31.354 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:31.350+0000 7f4d1ea00640 1 -- 172.21.15.22:0/120448542 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 8153+0+0 (secure 0 0 0) 0x7f4d180043d0 con 0x5627289e0ba0 2024-03-28T18:06:31.354 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:31.350+0000 7f4d3a7ac740 1 -- 172.21.15.22:0/120448542 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f4d2802cb80 con 0x5627289e0ba0 2024-03-28T18:06:31.354 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:31.351+0000 7f4d1ea00640 1 -- 172.21.15.22:0/120448542 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4d1801f630 con 0x5627289e0ba0 2024-03-28T18:06:31.354 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:31.351+0000 7f4d3a7ac740 10 client.15126 fetch_fsmap learned FSMap version 26 2024-03-28T18:06:31.354 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:31.351+0000 7f4d3a7ac740 1 -- 172.21.15.22:0/120448542 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x5627287246b0 con 0x5627289e0ba0 2024-03-28T18:06:31.354 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:31.351+0000 7f4d1ea00640 1 -- 172.21.15.22:0/120448542 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f4d2802cb80 con 0x5627289e0ba0 2024-03-28T18:06:31.354 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:31.351+0000 7f4d1ea00640 1 -- 172.21.15.22:0/120448542 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 26) v1 ==== 7934+0+0 (secure 0 0 0) 0x7f4d18003b90 con 0x5627289e0ba0 2024-03-28T18:06:31.354 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:31.351+0000 7f4d3a7ac740 10 client.15126 fetch_fsmap finished waiting for FSMap version 26 2024-03-28T18:06:31.354 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:31.351+0000 7f4d3a7ac740 10 client.15126 resolve_mds: resolved name 'h' to daemon mds.h 2024-03-28T18:06:31.354 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:31.351+0000 7f4d3a7ac740 20 client.15126 populate_metadata read hostname 'smithi022' 2024-03-28T18:06:31.355 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:31.351+0000 7f4d3a7ac740 1 --2- 172.21.15.22:0/120448542 >> [v2:172.21.15.79:6836/1787903366,v1:172.21.15.79:6837/1787903366] conn(0x5627289f7a10 0x5627289f6a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:06:31.355 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:31.351+0000 7f4d3a7ac740 4 client.15126 mds_command: new command op to 34511 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-28T18:06:31.355 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:31.351+0000 7f4d3a7ac740 1 -- 172.21.15.22:0/120448542 --> [v2:172.21.15.79:6836/1787903366,v1:172.21.15.79:6837/1787903366] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5627288e7520 con 0x5627289f7a10 2024-03-28T18:06:31.355 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:31.352+0000 7f4d2fe00640 1 --2- 172.21.15.22:0/120448542 >> [v2:172.21.15.79:6836/1787903366,v1:172.21.15.79:6837/1787903366] conn(0x5627289f7a10 0x5627289f6a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:06:31.355 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:31.352+0000 7f4d2fe00640 1 --2- 172.21.15.22:0/120448542 >> [v2:172.21.15.79:6836/1787903366,v1:172.21.15.79:6837/1787903366] conn(0x5627289f7a10 0x5627289f6a30 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=803671186b9e47fe server_cookie=abdd890d0b2e023d in_seq=0 out_seq=0 2024-03-28T18:06:31.355 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:31.352+0000 7f4d1ea00640 10 client.15126 ms_handle_connect on v2:172.21.15.79:6836/1787903366 2024-03-28T18:06:31.356 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:31.354+0000 7f4d1ea00640 1 -- 172.21.15.22:0/120448542 <== mds.1 v2:172.21.15.79:6836/1787903366 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x5627288e7520 con 0x5627289f7a10 2024-03-28T18:06:31.356 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:31.354+0000 7f4d1ea00640 10 client.15126 handle_command_reply: tid=0 2024-03-28T18:06:31.356 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:31.355+0000 7f4d3a7ac740 1 client.15126 shutdown 2024-03-28T18:06:31.356 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:31.355+0000 7f4d3a7ac740 1 -- 172.21.15.22:0/120448542 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5627289e0ba0 msgr2=0x5627289eb110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:06:31.356 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:31.355+0000 7f4d3a7ac740 1 --2- 172.21.15.22:0/120448542 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5627289e0ba0 0x5627289eb110 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f4d18012010 tx=0x7f4d180046c0 comp rx=0 tx=0).stop 2024-03-28T18:06:31.356 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:31.355+0000 7f4d3a7ac740 1 -- 172.21.15.22:0/120448542 shutdown_connections 2024-03-28T18:06:31.356 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:31.355+0000 7f4d3a7ac740 1 --2- 172.21.15.22:0/120448542 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5627289e6700 0x5627289f2690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:06:31.356 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:31.355+0000 7f4d3a7ac740 1 --2- 172.21.15.22:0/120448542 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5627289e1590 0x5627289eb650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:06:31.357 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:31.355+0000 7f4d3a7ac740 1 --2- 172.21.15.22:0/120448542 >> [v2:172.21.15.79:6836/1787903366,v1:172.21.15.79:6837/1787903366] conn(0x5627289f7a10 0x5627289f6a30 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:06:31.357 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:31.355+0000 7f4d3a7ac740 1 --2- 172.21.15.22:0/120448542 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5627289e0ba0 0x5627289eb110 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:06:31.357 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:31.355+0000 7f4d3a7ac740 1 -- 172.21.15.22:0/120448542 >> 172.21.15.22:0/120448542 conn(0x5627289dc110 msgr2=0x5627289db310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:06:31.357 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:31.355+0000 7f4d1ea00640 0 client.15126 ms_handle_reset on v2:172.21.15.79:6836/1787903366 2024-03-28T18:06:31.357 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:31.355+0000 7f4d3a7ac740 1 -- 172.21.15.22:0/120448542 shutdown_connections 2024-03-28T18:06:31.357 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:31.355+0000 7f4d3a7ac740 1 -- 172.21.15.22:0/120448542 wait complete. 2024-03-28T18:06:31.357 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:31.355+0000 7f4d3a7ac740 20 client.15126 trim_cache size 0 max 16384 2024-03-28T18:06:31.374 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:31.374+0000 7f4d3a7ac740 1 Processor -- start 2024-03-28T18:06:31.374 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:31.374+0000 7f4d3a7ac740 1 -- start start 2024-03-28T18:06:31.374 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:31.374+0000 7f4d3a7ac740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5627289e0ba0 0x5627289eb5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:06:31.374 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:31.374+0000 7f4d3a7ac740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5627289e1590 0x5627289f2650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:06:31.374 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:31.374+0000 7f4d3a7ac740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5627289e6700 0x5627289d6e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:06:31.374 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:31.374+0000 7f4d3a7ac740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x5627289f7230 con 0x5627289e0ba0 2024-03-28T18:06:31.374 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:31.374+0000 7f4d3a7ac740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x5627289f7890 con 0x5627289e6700 2024-03-28T18:06:31.374 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:31.374+0000 7f4d3a7ac740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x5627289f7ef0 con 0x5627289e1590 2024-03-28T18:06:31.375 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:31.375+0000 7f4d35000640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5627289e0ba0 0x5627289eb5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:06:31.376 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:31.375+0000 7f4d35000640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5627289e0ba0 0x5627289eb5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:49254/0 (socket says 172.21.15.22:49254) 2024-03-28T18:06:31.376 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:31.375+0000 7f4d35000640 1 -- 172.21.15.22:0/4025718695 learned_addr learned my addr 172.21.15.22:0/4025718695 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-28T18:06:31.376 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:31.375+0000 7f4d35a00640 1 --2- 172.21.15.22:0/4025718695 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5627289e6700 0x5627289d6e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:06:31.376 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:31.375+0000 7f4d2fe00640 1 --2- 172.21.15.22:0/4025718695 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5627289e1590 0x5627289f2650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:06:31.376 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:31.375+0000 7f4d35000640 1 -- 172.21.15.22:0/4025718695 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5627289e1590 msgr2=0x5627289f2650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T18:06:31.376 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:31.375+0000 7f4d35000640 1 --2- 172.21.15.22:0/4025718695 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5627289e1590 0x5627289f2650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:06:31.376 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:31.375+0000 7f4d35000640 1 -- 172.21.15.22:0/4025718695 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5627289e6700 msgr2=0x5627289d6e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T18:06:31.376 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:31.375+0000 7f4d35000640 1 --2- 172.21.15.22:0/4025718695 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5627289e6700 0x5627289d6e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:06:31.376 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:31.375+0000 7f4d35000640 1 -- 172.21.15.22:0/4025718695 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5627289ec240 con 0x5627289e0ba0 2024-03-28T18:06:31.376 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:31.375+0000 7f4d35a00640 1 --2- 172.21.15.22:0/4025718695 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5627289e6700 0x5627289d6e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-28T18:06:31.376 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:31.376+0000 7f4d35000640 1 --2- 172.21.15.22:0/4025718695 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5627289e0ba0 0x5627289eb5b0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f4d18000c00 tx=0x7f4d18027870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=16146e22d644176b server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:06:31.376 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:31.376+0000 7f4d25600640 1 -- 172.21.15.22:0/4025718695 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d18003fd0 con 0x5627289e0ba0 2024-03-28T18:06:31.376 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:31.376+0000 7f4d25600640 1 -- 172.21.15.22:0/4025718695 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f4d180278e0 con 0x5627289e0ba0 2024-03-28T18:06:31.377 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:31.376+0000 7f4d25600640 1 -- 172.21.15.22:0/4025718695 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d18005550 con 0x5627289e0ba0 2024-03-28T18:06:31.379 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:31.377+0000 7f4d3a7ac740 1 -- 172.21.15.22:0/4025718695 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5627289e0ba0 msgr2=0x5627289eb5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:06:31.379 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:31.377+0000 7f4d3a7ac740 1 --2- 172.21.15.22:0/4025718695 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5627289e0ba0 0x5627289eb5b0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f4d18000c00 tx=0x7f4d18027870 comp rx=0 tx=0).stop 2024-03-28T18:06:31.379 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:31.377+0000 7f4d3a7ac740 1 -- 172.21.15.22:0/4025718695 shutdown_connections 2024-03-28T18:06:31.379 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:31.377+0000 7f4d3a7ac740 1 --2- 172.21.15.22:0/4025718695 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5627289e6700 0x5627289d6e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:06:31.379 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:31.377+0000 7f4d3a7ac740 1 --2- 172.21.15.22:0/4025718695 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5627289e1590 0x5627289f2650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:06:31.379 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:31.377+0000 7f4d3a7ac740 1 --2- 172.21.15.22:0/4025718695 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5627289e0ba0 0x5627289eb5b0 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:06:31.379 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:31.377+0000 7f4d3a7ac740 1 -- 172.21.15.22:0/4025718695 >> 172.21.15.22:0/4025718695 conn(0x5627289dc110 msgr2=0x5627289f6a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:06:31.379 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:31.377+0000 7f4d3a7ac740 1 -- 172.21.15.22:0/4025718695 shutdown_connections 2024-03-28T18:06:31.379 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:31.377+0000 7f4d3a7ac740 1 -- 172.21.15.22:0/4025718695 wait complete. 2024-03-28T18:06:31.379 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:31.377+0000 7f4d3a7ac740 1 -- start start 2024-03-28T18:06:31.379 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:31.377+0000 7f4d3a7ac740 1 Processor -- start 2024-03-28T18:06:31.380 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:31.378+0000 7f4d3a7ac740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5627289e0ba0 0x5627289ee680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:06:31.380 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:31.378+0000 7f4d3a7ac740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5627289e1590 0x5627289eebc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:06:31.380 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:31.378+0000 7f4d3a7ac740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5627289e6700 0x5627289f1b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:06:31.380 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:31.378+0000 7f4d3a7ac740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x5627289f8bb0 con 0x5627289e1590 2024-03-28T18:06:31.380 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:31.378+0000 7f4d3a7ac740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x5627289f8550 con 0x5627289e6700 2024-03-28T18:06:31.380 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:31.378+0000 7f4d3a7ac740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x5627289f9210 con 0x5627289e0ba0 2024-03-28T18:06:31.380 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:31.378+0000 7f4d3a7ac740 10 client.0 _ll_register_callbacks cb 0x7f4d1800bdd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-28T18:06:31.380 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:31.378+0000 7f4d2fe00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5627289e1590 0x5627289eebc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:06:31.380 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:31.378+0000 7f4d35a00640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5627289e6700 0x5627289f1b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:06:31.380 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:31.378+0000 7f4d2fe00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5627289e1590 0x5627289eebc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:49270/0 (socket says 172.21.15.22:49270) 2024-03-28T18:06:31.380 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:31.378+0000 7f4d2fe00640 1 -- 172.21.15.22:0/2819230384 learned_addr learned my addr 172.21.15.22:0/2819230384 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-28T18:06:31.380 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:31.378+0000 7f4d35000640 1 --2- 172.21.15.22:0/2819230384 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5627289e0ba0 0x5627289ee680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:06:31.380 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:31.378+0000 7f4d2fe00640 1 -- 172.21.15.22:0/2819230384 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5627289e0ba0 msgr2=0x5627289ee680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T18:06:31.380 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:31.378+0000 7f4d2fe00640 1 --2- 172.21.15.22:0/2819230384 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5627289e0ba0 0x5627289ee680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:06:31.381 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:31.378+0000 7f4d2fe00640 1 -- 172.21.15.22:0/2819230384 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5627289e6700 msgr2=0x5627289f1b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T18:06:31.381 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:31.378+0000 7f4d2fe00640 1 --2- 172.21.15.22:0/2819230384 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5627289e6700 0x5627289f1b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:06:31.381 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:31.378+0000 7f4d2fe00640 1 -- 172.21.15.22:0/2819230384 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5627289f2160 con 0x5627289e1590 2024-03-28T18:06:31.381 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:31.379+0000 7f4d2fe00640 1 --2- 172.21.15.22:0/2819230384 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5627289e1590 0x5627289eebc0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f4d20098420 tx=0x7f4d2000ef50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=21a1b02ff5e5e206 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:06:31.381 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:31.379+0000 7f4d1ea00640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-03-28T18:06:31.381 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:31.379+0000 7f4d1ea00640 1 -- 172.21.15.22:0/2819230384 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d2009b3b0 con 0x5627289e1590 2024-03-28T18:06:31.381 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:31.379+0000 7f4d1ea00640 1 -- 172.21.15.22:0/2819230384 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 8153+0+0 (secure 0 0 0) 0x7f4d200980c0 con 0x5627289e1590 2024-03-28T18:06:31.381 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:31.379+0000 7f4d3a7ac740 1 -- 172.21.15.22:0/2819230384 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f4d180278e0 con 0x5627289e1590 2024-03-28T18:06:31.381 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:31.379+0000 7f4d1ea00640 1 -- 172.21.15.22:0/2819230384 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4d20010ac0 con 0x5627289e1590 2024-03-28T18:06:31.381 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:31.379+0000 7f4d3a7ac740 10 client.15132 fetch_fsmap learned FSMap version 26 2024-03-28T18:06:31.381 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:31.379+0000 7f4d3a7ac740 1 -- 172.21.15.22:0/2819230384 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x562728a53320 con 0x5627289e1590 2024-03-28T18:06:31.383 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:31.380+0000 7f4d1ea00640 1 -- 172.21.15.22:0/2819230384 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f4d180278e0 con 0x5627289e1590 2024-03-28T18:06:31.383 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:31.380+0000 7f4d1ea00640 1 -- 172.21.15.22:0/2819230384 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 26) v1 ==== 7934+0+0 (secure 0 0 0) 0x7f4d200a3d60 con 0x5627289e1590 2024-03-28T18:06:31.383 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:31.380+0000 7f4d3a7ac740 10 client.15132 fetch_fsmap finished waiting for FSMap version 26 2024-03-28T18:06:31.383 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:31.380+0000 7f4d3a7ac740 10 client.15132 resolve_mds: resolved name 'h' to daemon mds.h 2024-03-28T18:06:31.383 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:31.380+0000 7f4d3a7ac740 20 client.15132 populate_metadata read hostname 'smithi022' 2024-03-28T18:06:31.383 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:31.380+0000 7f4d3a7ac740 1 --2- 172.21.15.22:0/2819230384 >> [v2:172.21.15.79:6836/1787903366,v1:172.21.15.79:6837/1787903366] conn(0x562728a579a0 0x562728a53610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:06:31.383 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:31.380+0000 7f4d3a7ac740 4 client.15132 mds_command: new command op to 34511 tid=0 multi_id=0 [{"prefix": "status"}] 2024-03-28T18:06:31.384 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:31.380+0000 7f4d3a7ac740 1 -- 172.21.15.22:0/2819230384 --> [v2:172.21.15.79:6836/1787903366,v1:172.21.15.79:6837/1787903366] -- command(tid 0: {"prefix": "status"}) v1 -- 0x5627289531d0 con 0x562728a579a0 2024-03-28T18:06:31.384 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:31.380+0000 7f4d35000640 1 --2- 172.21.15.22:0/2819230384 >> [v2:172.21.15.79:6836/1787903366,v1:172.21.15.79:6837/1787903366] conn(0x562728a579a0 0x562728a53610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:06:31.384 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:31.381+0000 7f4d35000640 1 --2- 172.21.15.22:0/2819230384 >> [v2:172.21.15.79:6836/1787903366,v1:172.21.15.79:6837/1787903366] conn(0x562728a579a0 0x562728a53610 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=1faa3c82c42ce900 server_cookie=ff4c339fca75a442 in_seq=0 out_seq=0 2024-03-28T18:06:31.384 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:31.381+0000 7f4d1ea00640 10 client.15132 ms_handle_connect on v2:172.21.15.79:6836/1787903366 2024-03-28T18:06:31.384 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:31.381+0000 7f4d1ea00640 1 -- 172.21.15.22:0/2819230384 <== mds.1 v2:172.21.15.79:6836/1787903366 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+327 (crc 0 0 0) 0x5627289531d0 con 0x562728a579a0 2024-03-28T18:06:31.384 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:31.381+0000 7f4d1ea00640 10 client.15132 handle_command_reply: tid=0 2024-03-28T18:06:31.384 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:31.381+0000 7f4d3a7ac740 1 client.15132 shutdown 2024-03-28T18:06:31.384 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:31.382+0000 7f4d3a7ac740 1 -- 172.21.15.22:0/2819230384 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5627289e1590 msgr2=0x5627289eebc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:06:31.384 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:31.382+0000 7f4d3a7ac740 1 --2- 172.21.15.22:0/2819230384 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5627289e1590 0x5627289eebc0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f4d20098420 tx=0x7f4d2000ef50 comp rx=0 tx=0).stop 2024-03-28T18:06:31.384 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:31.382+0000 7f4d3a7ac740 1 -- 172.21.15.22:0/2819230384 shutdown_connections 2024-03-28T18:06:31.384 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:31.382+0000 7f4d3a7ac740 1 --2- 172.21.15.22:0/2819230384 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5627289e6700 0x5627289f1b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:06:31.384 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:31.382+0000 7f4d3a7ac740 1 --2- 172.21.15.22:0/2819230384 >> [v2:172.21.15.79:6836/1787903366,v1:172.21.15.79:6837/1787903366] conn(0x562728a579a0 0x562728a53610 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:06:31.384 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:31.382+0000 7f4d3a7ac740 1 --2- 172.21.15.22:0/2819230384 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5627289e1590 0x5627289eebc0 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:06:31.385 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:31.382+0000 7f4d3a7ac740 1 --2- 172.21.15.22:0/2819230384 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5627289e0ba0 0x5627289ee680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:06:31.385 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:31.382+0000 7f4d3a7ac740 1 -- 172.21.15.22:0/2819230384 >> 172.21.15.22:0/2819230384 conn(0x5627289dc110 msgr2=0x5627289d6e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:06:31.385 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:31.382+0000 7f4d1ea00640 0 client.15132 ms_handle_reset on v2:172.21.15.79:6836/1787903366 2024-03-28T18:06:31.385 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:31.382+0000 7f4d3a7ac740 1 -- 172.21.15.22:0/2819230384 shutdown_connections 2024-03-28T18:06:31.385 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:31.382+0000 7f4d3a7ac740 1 -- 172.21.15.22:0/2819230384 wait complete. 2024-03-28T18:06:31.385 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:31.382+0000 7f4d3a7ac740 20 client.15132 trim_cache size 0 max 16384 2024-03-28T18:06:31.385 INFO:teuthology.orchestra.run.smithi022.stdout:{ 2024-03-28T18:06:31.385 INFO:teuthology.orchestra.run.smithi022.stdout: "cluster_fsid": "fd220748-ed2b-11ee-95d0-87774f69a715", 2024-03-28T18:06:31.385 INFO:teuthology.orchestra.run.smithi022.stdout: "whoami": 1, 2024-03-28T18:06:31.385 INFO:teuthology.orchestra.run.smithi022.stdout: "id": 34511, 2024-03-28T18:06:31.385 INFO:teuthology.orchestra.run.smithi022.stdout: "want_state": "up:active", 2024-03-28T18:06:31.385 INFO:teuthology.orchestra.run.smithi022.stdout: "state": "up:active", 2024-03-28T18:06:31.386 INFO:teuthology.orchestra.run.smithi022.stdout: "fs_name": "cephfs", 2024-03-28T18:06:31.386 INFO:teuthology.orchestra.run.smithi022.stdout: "rank_uptime": 98.808843746999997, 2024-03-28T18:06:31.386 INFO:teuthology.orchestra.run.smithi022.stdout: "mdsmap_epoch": 26, 2024-03-28T18:06:31.386 INFO:teuthology.orchestra.run.smithi022.stdout: "osdmap_epoch": 92, 2024-03-28T18:06:31.386 INFO:teuthology.orchestra.run.smithi022.stdout: "osdmap_epoch_barrier": 90, 2024-03-28T18:06:31.386 INFO:teuthology.orchestra.run.smithi022.stdout: "uptime": 123.074437942 2024-03-28T18:06:31.386 INFO:teuthology.orchestra.run.smithi022.stdout:} 2024-03-28T18:06:31.387 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:31.383+0000 7f4d37a00640 1 -- 172.21.15.22:0/3300031523 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7f4d140a2880 msgr2=0x7f4d140a4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:06:31.387 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:31.383+0000 7f4d37a00640 1 --2- 172.21.15.22:0/3300031523 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7f4d140a2880 0x7f4d140a4d40 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f4d18005fd0 tx=0x7f4d1800b650 comp rx=0 tx=0).stop 2024-03-28T18:06:31.387 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:31.383+0000 7f4d37a00640 1 -- 172.21.15.22:0/3300031523 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4d300ffd00 msgr2=0x7f4d3019a130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:06:31.387 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:31.383+0000 7f4d37a00640 1 --2- 172.21.15.22:0/3300031523 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4d300ffd00 0x7f4d3019a130 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f4d2000ebc0 tx=0x7f4d2000c6a0 comp rx=0 tx=0).stop 2024-03-28T18:06:31.387 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:31.383+0000 7f4d37a00640 1 -- 172.21.15.22:0/3300031523 shutdown_connections 2024-03-28T18:06:31.387 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:31.383+0000 7f4d37a00640 1 --2- 172.21.15.22:0/3300031523 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7f4d140a2880 0x7f4d140a4d40 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:06:31.387 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:31.383+0000 7f4d37a00640 1 --2- 172.21.15.22:0/3300031523 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f4d30107390 0x7f4d301a1150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:06:31.387 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:31.383+0000 7f4d37a00640 1 --2- 172.21.15.22:0/3300031523 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4d300ffd00 0x7f4d3019a130 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:06:31.387 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:31.383+0000 7f4d37a00640 1 --2- 172.21.15.22:0/3300031523 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4d300ff310 0x7f4d30199bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:06:31.387 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:31.383+0000 7f4d37a00640 1 -- 172.21.15.22:0/3300031523 >> 172.21.15.22:0/3300031523 conn(0x7f4d300fae60 msgr2=0x7f4d300fd250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:06:31.387 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:31.383+0000 7f4d37a00640 1 -- 172.21.15.22:0/3300031523 shutdown_connections 2024-03-28T18:06:31.387 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:31.383+0000 7f4d37a00640 1 -- 172.21.15.22:0/3300031523 wait complete. 2024-03-28T18:06:31.400 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2024-03-28T18:06:31.424 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:06:31 smithi134 ceph-mon[35037]: from='client.? 172.21.15.22:0/4244182015' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-28T18:06:31.432 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:06:31 smithi079 ceph-mon[35312]: from='client.? 172.21.15.22:0/4244182015' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-28T18:06:31.623 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:31.623+0000 7f8a28a00640 1 -- 172.21.15.22:0/2961251020 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8a241015a0 msgr2=0x7f8a241019c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:06:31.623 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:31.623+0000 7f8a28a00640 1 --2- 172.21.15.22:0/2961251020 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8a241015a0 0x7f8a241019c0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f8a14009900 tx=0x7f8a1402bc30 comp rx=0 tx=0).stop 2024-03-28T18:06:31.623 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:31.623+0000 7f8a28a00640 1 -- 172.21.15.22:0/2961251020 shutdown_connections 2024-03-28T18:06:31.623 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:31.623+0000 7f8a28a00640 1 --2- 172.21.15.22:0/2961251020 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8a24103150 0x7f8a241099e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:06:31.623 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:31.623+0000 7f8a28a00640 1 --2- 172.21.15.22:0/2961251020 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f8a24102790 0x7f8a24102c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:06:31.624 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:31.623+0000 7f8a28a00640 1 --2- 172.21.15.22:0/2961251020 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8a241015a0 0x7f8a241019c0 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:06:31.624 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:31.623+0000 7f8a28a00640 1 -- 172.21.15.22:0/2961251020 >> 172.21.15.22:0/2961251020 conn(0x7f8a240fcb10 msgr2=0x7f8a240fef50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:06:31.624 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:31.623+0000 7f8a28a00640 1 -- 172.21.15.22:0/2961251020 shutdown_connections 2024-03-28T18:06:31.624 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:31.623+0000 7f8a28a00640 1 -- 172.21.15.22:0/2961251020 wait complete. 2024-03-28T18:06:31.626 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:31.624+0000 7f8a28a00640 1 Processor -- start 2024-03-28T18:06:31.626 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:31.624+0000 7f8a28a00640 1 -- start start 2024-03-28T18:06:31.626 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:31.625+0000 7f8a28a00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8a241015a0 0x7f8a24199ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:06:31.626 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:31.625+0000 7f8a28a00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8a24102790 0x7f8a2419a000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:06:31.626 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:31.625+0000 7f8a28a00640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f8a24103150 0x7f8a241a1070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:06:31.626 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:31.625+0000 7f8a28a00640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a2410d200 con 0x7f8a241015a0 2024-03-28T18:06:31.626 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:31.625+0000 7f8a28a00640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a2410d080 con 0x7f8a24103150 2024-03-28T18:06:31.626 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:31.625+0000 7f8a28a00640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a2410d380 con 0x7f8a24102790 2024-03-28T18:06:31.626 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:31.625+0000 7f8a23400640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8a241015a0 0x7f8a24199ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:06:31.626 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:31.625+0000 7f8a23400640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8a241015a0 0x7f8a24199ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:49296/0 (socket says 172.21.15.22:49296) 2024-03-28T18:06:31.627 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:31.625+0000 7f8a23400640 1 -- 172.21.15.22:0/3293893861 learned_addr learned my addr 172.21.15.22:0/3293893861 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-28T18:06:31.627 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:31.625+0000 7f8a1aa00640 1 --2- 172.21.15.22:0/3293893861 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8a24102790 0x7f8a2419a000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:06:31.627 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:31.625+0000 7f8a23400640 1 -- 172.21.15.22:0/3293893861 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8a24102790 msgr2=0x7f8a2419a000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:06:31.627 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:31.625+0000 7f8a23e00640 1 --2- 172.21.15.22:0/3293893861 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f8a24103150 0x7f8a241a1070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:06:31.627 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:31.625+0000 7f8a23400640 1 --2- 172.21.15.22:0/3293893861 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8a24102790 0x7f8a2419a000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:06:31.627 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:31.625+0000 7f8a23400640 1 -- 172.21.15.22:0/3293893861 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f8a24103150 msgr2=0x7f8a241a1070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:06:31.627 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:31.625+0000 7f8a23400640 1 --2- 172.21.15.22:0/3293893861 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f8a24103150 0x7f8a241a1070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:06:31.627 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:31.625+0000 7f8a23400640 1 -- 172.21.15.22:0/3293893861 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8a241a16e0 con 0x7f8a241015a0 2024-03-28T18:06:31.627 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:31.625+0000 7f8a1aa00640 1 --2- 172.21.15.22:0/3293893861 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8a24102790 0x7f8a2419a000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T18:06:31.627 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:31.626+0000 7f8a23400640 1 --2- 172.21.15.22:0/3293893861 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8a241015a0 0x7f8a24199ac0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f8a14005ab0 tx=0x7f8a14004e90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:06:31.627 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:31.626+0000 7f8a20c00640 1 -- 172.21.15.22:0/3293893861 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a14043020 con 0x7f8a241015a0 2024-03-28T18:06:31.627 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:31.626+0000 7f8a20c00640 1 -- 172.21.15.22:0/3293893861 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f8a140054f0 con 0x7f8a241015a0 2024-03-28T18:06:31.627 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:31.626+0000 7f8a28a00640 1 -- 172.21.15.22:0/3293893861 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8a241a1970 con 0x7f8a241015a0 2024-03-28T18:06:31.627 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:31.626+0000 7f8a20c00640 1 -- 172.21.15.22:0/3293893861 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a14003d10 con 0x7f8a241015a0 2024-03-28T18:06:31.627 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:31.626+0000 7f8a28a00640 1 -- 172.21.15.22:0/3293893861 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8a241a1ec0 con 0x7f8a241015a0 2024-03-28T18:06:31.629 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:31.629+0000 7f8a20c00640 1 -- 172.21.15.22:0/3293893861 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 18) v1 ==== 137290+0+0 (secure 0 0 0) 0x7f8a14046020 con 0x7f8a241015a0 2024-03-28T18:06:31.629 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:31.629+0000 7f8a2ce1f740 1 Processor -- start 2024-03-28T18:06:31.629 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:31.629+0000 7f8a2ce1f740 1 -- start start 2024-03-28T18:06:31.631 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:31.630+0000 7f8a2ce1f740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55bc2fb90d80 0x55bc2fb911a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:06:31.631 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:31.630+0000 7f8a2ce1f740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55bc2fb91770 0x55bc2fb96270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:06:31.631 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:31.630+0000 7f8a2ce1f740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55bc2fb968e0 0x55bc2fb98d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:06:31.631 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:31.630+0000 7f8a2ce1f740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bc2faf5c90 con 0x55bc2fb90d80 2024-03-28T18:06:31.631 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:31.630+0000 7f8a2ce1f740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bc2fa9e980 con 0x55bc2fb968e0 2024-03-28T18:06:31.631 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:31.630+0000 7f8a2ce1f740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bc2faf1d30 con 0x55bc2fb91770 2024-03-28T18:06:31.631 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:31.630+0000 7f8a23e00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55bc2fb90d80 0x55bc2fb911a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:06:31.632 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:31.630+0000 7f8a23e00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55bc2fb90d80 0x55bc2fb911a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:49308/0 (socket says 172.21.15.22:49308) 2024-03-28T18:06:31.632 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:31.630+0000 7f8a23e00640 1 -- 172.21.15.22:0/2084341895 learned_addr learned my addr 172.21.15.22:0/2084341895 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-28T18:06:31.632 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:31.630+0000 7f8a1aa00640 1 --2- 172.21.15.22:0/2084341895 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55bc2fb968e0 0x55bc2fb98d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:06:31.632 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:31.631+0000 7f8a23400640 1 --2- 172.21.15.22:0/2084341895 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55bc2fb91770 0x55bc2fb96270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:06:31.632 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:31.631+0000 7f8a23e00640 1 -- 172.21.15.22:0/2084341895 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55bc2fb91770 msgr2=0x55bc2fb96270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T18:06:31.632 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:31.631+0000 7f8a23e00640 1 --2- 172.21.15.22:0/2084341895 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55bc2fb91770 0x55bc2fb96270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:06:31.632 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:31.631+0000 7f8a23e00640 1 -- 172.21.15.22:0/2084341895 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55bc2fb968e0 msgr2=0x55bc2fb98d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T18:06:31.632 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:31.631+0000 7f8a23e00640 1 --2- 172.21.15.22:0/2084341895 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55bc2fb968e0 0x55bc2fb98d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:06:31.632 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:31.631+0000 7f8a23e00640 1 -- 172.21.15.22:0/2084341895 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55bc2fab6510 con 0x55bc2fb90d80 2024-03-28T18:06:31.632 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:31.631+0000 7f8a23400640 1 --2- 172.21.15.22:0/2084341895 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55bc2fb91770 0x55bc2fb96270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T18:06:31.632 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:31.631+0000 7f8a20c00640 1 --2- 172.21.15.22:0/3293893861 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7f8a080a2800 0x7f8a080a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:06:31.632 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:31.631+0000 7f8a23e00640 1 --2- 172.21.15.22:0/2084341895 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55bc2fb90d80 0x55bc2fb911a0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f8a1c008c80 tx=0x7f8a1c00d710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bbbe77febc78fa7b server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:06:31.632 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:31.631+0000 7f8a23400640 1 --2- 172.21.15.22:0/3293893861 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7f8a080a2800 0x7f8a080a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:06:31.633 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:31.631+0000 7f8a20c00640 1 -- 172.21.15.22:0/3293893861 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 8153+0+0 (secure 0 0 0) 0x7f8a140f0870 con 0x7f8a241015a0 2024-03-28T18:06:31.633 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:31.631+0000 7f8a13e00640 1 -- 172.21.15.22:0/2084341895 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a1c018680 con 0x55bc2fb90d80 2024-03-28T18:06:31.633 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:31.631+0000 7f8a13e00640 1 -- 172.21.15.22:0/2084341895 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f8a1c018cf0 con 0x55bc2fb90d80 2024-03-28T18:06:31.633 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:31.631+0000 7f8a13e00640 1 -- 172.21.15.22:0/2084341895 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a1c00fe60 con 0x55bc2fb90d80 2024-03-28T18:06:31.636 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:31.632+0000 7f8a2ce1f740 1 -- 172.21.15.22:0/2084341895 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55bc2fb90d80 msgr2=0x55bc2fb911a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:06:31.636 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:31.632+0000 7f8a2ce1f740 1 --2- 172.21.15.22:0/2084341895 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55bc2fb90d80 0x55bc2fb911a0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f8a1c008c80 tx=0x7f8a1c00d710 comp rx=0 tx=0).stop 2024-03-28T18:06:31.636 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:31.632+0000 7f8a23400640 1 --2- 172.21.15.22:0/3293893861 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7f8a080a2800 0x7f8a080a4cc0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f8a140ee370 tx=0x7f8a14009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:06:31.636 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:31.632+0000 7f8a2ce1f740 1 -- 172.21.15.22:0/2084341895 shutdown_connections 2024-03-28T18:06:31.636 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:31.632+0000 7f8a2ce1f740 1 --2- 172.21.15.22:0/2084341895 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55bc2fb968e0 0x55bc2fb98d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:06:31.636 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:31.632+0000 7f8a2ce1f740 1 --2- 172.21.15.22:0/2084341895 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55bc2fb91770 0x55bc2fb96270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:06:31.637 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:31.632+0000 7f8a2ce1f740 1 --2- 172.21.15.22:0/2084341895 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55bc2fb90d80 0x55bc2fb911a0 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:06:31.637 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:31.632+0000 7f8a2ce1f740 1 -- 172.21.15.22:0/2084341895 >> 172.21.15.22:0/2084341895 conn(0x55bc2fb8c2f0 msgr2=0x55bc2fb8c6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:06:31.637 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:31.632+0000 7f8a2ce1f740 1 -- 172.21.15.22:0/2084341895 shutdown_connections 2024-03-28T18:06:31.637 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:31.632+0000 7f8a2ce1f740 1 -- 172.21.15.22:0/2084341895 wait complete. 2024-03-28T18:06:31.637 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:31.633+0000 7f8a2ce1f740 1 -- start start 2024-03-28T18:06:31.637 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:31.633+0000 7f8a2ce1f740 1 Processor -- start 2024-03-28T18:06:31.637 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:31.634+0000 7f8a2ce1f740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55bc2fb90d80 0x55bc2fb9b290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:06:31.637 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:31.634+0000 7f8a2ce1f740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55bc2fb91770 0x55bc2fb9b7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:06:31.637 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:31.634+0000 7f8a2ce1f740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55bc2fb968e0 0x55bc2fba2810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:06:31.637 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:31.634+0000 7f8a2ce1f740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bc2fb99890 con 0x55bc2fb90d80 2024-03-28T18:06:31.637 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:31.634+0000 7f8a2ce1f740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bc2fab1dd0 con 0x55bc2fb91770 2024-03-28T18:06:31.637 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:31.634+0000 7f8a2ce1f740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bc2fb99a10 con 0x55bc2fb968e0 2024-03-28T18:06:31.637 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:31.634+0000 7f8a2ce1f740 10 client.0 _ll_register_callbacks cb 0x55bc2f809ac0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-28T18:06:31.637 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:31.634+0000 7f8a23e00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55bc2fb90d80 0x55bc2fb9b290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:06:31.638 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:31.634+0000 7f8a23e00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55bc2fb90d80 0x55bc2fb9b290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:49316/0 (socket says 172.21.15.22:49316) 2024-03-28T18:06:31.638 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:31.634+0000 7f8a23e00640 1 -- 172.21.15.22:0/3344934726 learned_addr learned my addr 172.21.15.22:0/3344934726 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-28T18:06:31.638 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:31.634+0000 7f8a1aa00640 1 --2- 172.21.15.22:0/3344934726 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55bc2fb91770 0x55bc2fb9b7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:06:31.638 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:31.634+0000 7f8a23e00640 1 -- 172.21.15.22:0/3344934726 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55bc2fb968e0 msgr2=0x55bc2fba2810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T18:06:31.638 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:31.634+0000 7f8a23e00640 1 --2- 172.21.15.22:0/3344934726 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55bc2fb968e0 0x55bc2fba2810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:06:31.638 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:31.634+0000 7f8a23e00640 1 -- 172.21.15.22:0/3344934726 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55bc2fb91770 msgr2=0x55bc2fb9b7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T18:06:31.638 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:31.634+0000 7f8a23e00640 1 --2- 172.21.15.22:0/3344934726 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55bc2fb91770 0x55bc2fb9b7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:06:31.638 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:31.634+0000 7f8a23e00640 1 -- 172.21.15.22:0/3344934726 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55bc2f80a2d0 con 0x55bc2fb90d80 2024-03-28T18:06:31.638 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:31.635+0000 7f8a23e00640 1 --2- 172.21.15.22:0/3344934726 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55bc2fb90d80 0x55bc2fb9b290 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f8a1c00f8f0 tx=0x7f8a1c0089c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ec2e0864d39de098 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:06:31.638 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:31.635+0000 7f8a10c00640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-03-28T18:06:31.638 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:31.635+0000 7f8a10c00640 1 -- 172.21.15.22:0/3344934726 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a1c004b00 con 0x55bc2fb90d80 2024-03-28T18:06:31.638 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:31.635+0000 7f8a10c00640 1 -- 172.21.15.22:0/3344934726 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 8153+0+0 (secure 0 0 0) 0x7f8a1c020070 con 0x55bc2fb90d80 2024-03-28T18:06:31.638 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:31.635+0000 7f8a2ce1f740 1 -- 172.21.15.22:0/3344934726 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f8a1c018cf0 con 0x55bc2fb90d80 2024-03-28T18:06:31.638 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:31.635+0000 7f8a10c00640 1 -- 172.21.15.22:0/3344934726 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8a1c004ca0 con 0x55bc2fb90d80 2024-03-28T18:06:31.639 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:31.635+0000 7f8a2ce1f740 10 client.15144 fetch_fsmap learned FSMap version 26 2024-03-28T18:06:31.639 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:31.635+0000 7f8a2ce1f740 1 -- 172.21.15.22:0/3344934726 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x55bc2f8d46b0 con 0x55bc2fb90d80 2024-03-28T18:06:31.639 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:31.636+0000 7f8a10c00640 1 -- 172.21.15.22:0/3344934726 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f8a1c018cf0 con 0x55bc2fb90d80 2024-03-28T18:06:31.639 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:31.636+0000 7f8a10c00640 1 -- 172.21.15.22:0/3344934726 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 26) v1 ==== 7934+0+0 (secure 0 0 0) 0x7f8a1c02c020 con 0x55bc2fb90d80 2024-03-28T18:06:31.639 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:31.636+0000 7f8a2ce1f740 10 client.15144 fetch_fsmap finished waiting for FSMap version 26 2024-03-28T18:06:31.639 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:31.636+0000 7f8a2ce1f740 10 client.15144 resolve_mds: resolved name 'b' to daemon mds.b 2024-03-28T18:06:31.639 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:31.636+0000 7f8a2ce1f740 20 client.15144 populate_metadata read hostname 'smithi022' 2024-03-28T18:06:31.639 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:31.636+0000 7f8a2ce1f740 1 --2- 172.21.15.22:0/3344934726 >> [v2:172.21.15.79:6838/2441542443,v1:172.21.15.79:6839/2441542443] conn(0x55bc2fba74f0 0x55bc2fbab970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:06:31.639 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:31.636+0000 7f8a2ce1f740 4 client.15144 mds_command: new command op to 34553 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-28T18:06:31.639 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:31.636+0000 7f8a2ce1f740 1 -- 172.21.15.22:0/3344934726 --> [v2:172.21.15.79:6838/2441542443,v1:172.21.15.79:6839/2441542443] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55bc2fa97520 con 0x55bc2fba74f0 2024-03-28T18:06:31.639 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:31.636+0000 7f8a1aa00640 1 --2- 172.21.15.22:0/3344934726 >> [v2:172.21.15.79:6838/2441542443,v1:172.21.15.79:6839/2441542443] conn(0x55bc2fba74f0 0x55bc2fbab970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:06:31.640 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:31.637+0000 7f8a1aa00640 1 --2- 172.21.15.22:0/3344934726 >> [v2:172.21.15.79:6838/2441542443,v1:172.21.15.79:6839/2441542443] conn(0x55bc2fba74f0 0x55bc2fbab970 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=254dd1351a8d41e2 server_cookie=7b367e79807a8240 in_seq=0 out_seq=0 2024-03-28T18:06:31.641 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:31.637+0000 7f8a10c00640 10 client.15144 ms_handle_connect on v2:172.21.15.79:6838/2441542443 2024-03-28T18:06:31.641 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:31.639+0000 7f8a10c00640 1 -- 172.21.15.22:0/3344934726 <== mds.0 v2:172.21.15.79:6838/2441542443 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x55bc2fa97520 con 0x55bc2fba74f0 2024-03-28T18:06:31.641 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:31.639+0000 7f8a10c00640 10 client.15144 handle_command_reply: tid=0 2024-03-28T18:06:31.641 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:31.639+0000 7f8a2ce1f740 1 client.15144 shutdown 2024-03-28T18:06:31.641 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:31.639+0000 7f8a2ce1f740 1 -- 172.21.15.22:0/3344934726 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55bc2fb90d80 msgr2=0x55bc2fb9b290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:06:31.641 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:31.639+0000 7f8a2ce1f740 1 --2- 172.21.15.22:0/3344934726 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55bc2fb90d80 0x55bc2fb9b290 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f8a1c00f8f0 tx=0x7f8a1c0089c0 comp rx=0 tx=0).stop 2024-03-28T18:06:31.641 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:31.639+0000 7f8a2ce1f740 1 -- 172.21.15.22:0/3344934726 shutdown_connections 2024-03-28T18:06:31.641 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:31.639+0000 7f8a2ce1f740 1 --2- 172.21.15.22:0/3344934726 >> [v2:172.21.15.79:6838/2441542443,v1:172.21.15.79:6839/2441542443] conn(0x55bc2fba74f0 0x55bc2fbab970 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:06:31.641 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:31.639+0000 7f8a2ce1f740 1 --2- 172.21.15.22:0/3344934726 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55bc2fb968e0 0x55bc2fba2810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:06:31.641 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:31.639+0000 7f8a2ce1f740 1 --2- 172.21.15.22:0/3344934726 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55bc2fb91770 0x55bc2fb9b7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:06:31.641 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:31.639+0000 7f8a2ce1f740 1 --2- 172.21.15.22:0/3344934726 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55bc2fb90d80 0x55bc2fb9b290 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:06:31.641 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:31.639+0000 7f8a10c00640 0 client.15144 ms_handle_reset on v2:172.21.15.79:6838/2441542443 2024-03-28T18:06:31.641 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:31.639+0000 7f8a2ce1f740 1 -- 172.21.15.22:0/3344934726 >> 172.21.15.22:0/3344934726 conn(0x55bc2fb8c2f0 msgr2=0x55bc2fb8b4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:06:31.641 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:31.639+0000 7f8a2ce1f740 1 -- 172.21.15.22:0/3344934726 shutdown_connections 2024-03-28T18:06:31.642 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:31.639+0000 7f8a2ce1f740 1 -- 172.21.15.22:0/3344934726 wait complete. 2024-03-28T18:06:31.642 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:31.639+0000 7f8a2ce1f740 20 client.15144 trim_cache size 0 max 16384 2024-03-28T18:06:31.657 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:31.658+0000 7f8a2ce1f740 1 Processor -- start 2024-03-28T18:06:31.658 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:31.658+0000 7f8a2ce1f740 1 -- start start 2024-03-28T18:06:31.658 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:31.658+0000 7f8a2ce1f740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55bc2fb90d80 0x55bc2fb9b5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:06:31.658 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:31.658+0000 7f8a2ce1f740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55bc2fb91770 0x55bc2fba27a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:06:31.658 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:31.658+0000 7f8a2ce1f740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55bc2fb968e0 0x55bc2fb8ac60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:06:31.658 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:31.658+0000 7f8a2ce1f740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bc2fba6d10 con 0x55bc2fb91770 2024-03-28T18:06:31.658 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:31.658+0000 7f8a2ce1f740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bc2fba7370 con 0x55bc2fb90d80 2024-03-28T18:06:31.658 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:31.658+0000 7f8a2ce1f740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bc2fba79d0 con 0x55bc2fb968e0 2024-03-28T18:06:31.660 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:31.658+0000 7f8a23e00640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55bc2fb90d80 0x55bc2fb9b5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:06:31.660 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:31.658+0000 7f8a1aa00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55bc2fb91770 0x55bc2fba27a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:06:31.660 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:31.658+0000 7f8a23e00640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55bc2fb90d80 0x55bc2fb9b5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.22:41600/0 (socket says 172.21.15.22:41600) 2024-03-28T18:06:31.660 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:31.658+0000 7f8a23e00640 1 -- 172.21.15.22:0/629902748 learned_addr learned my addr 172.21.15.22:0/629902748 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-28T18:06:31.660 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:31.659+0000 7f8a23e00640 1 --2- 172.21.15.22:0/629902748 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55bc2fb968e0 0x55bc2fb8ac60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:06:31.660 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:31.659+0000 7f8a1aa00640 1 -- 172.21.15.22:0/629902748 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55bc2fb968e0 msgr2=0x55bc2fb8ac60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T18:06:31.660 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:31.659+0000 7f8a1aa00640 1 --2- 172.21.15.22:0/629902748 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55bc2fb968e0 0x55bc2fb8ac60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:06:31.660 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:31.659+0000 7f8a1aa00640 1 -- 172.21.15.22:0/629902748 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55bc2fb90d80 msgr2=0x55bc2fb9b5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T18:06:31.660 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:31.659+0000 7f8a1aa00640 1 --2- 172.21.15.22:0/629902748 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55bc2fb90d80 0x55bc2fb9b5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:06:31.660 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:31.659+0000 7f8a1aa00640 1 -- 172.21.15.22:0/629902748 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55bc2fb7e170 con 0x55bc2fb91770 2024-03-28T18:06:31.660 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:31.659+0000 7f8a1aa00640 1 --2- 172.21.15.22:0/629902748 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55bc2fb91770 0x55bc2fba27a0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x55bc2fba78d0 tx=0x7f8a0c009560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=62735c7d7daea3 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:06:31.660 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:31.659+0000 7f8a13400640 1 -- 172.21.15.22:0/629902748 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a0c005050 con 0x55bc2fb91770 2024-03-28T18:06:31.661 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:31.659+0000 7f8a13400640 1 -- 172.21.15.22:0/629902748 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f8a0c004730 con 0x55bc2fb91770 2024-03-28T18:06:31.661 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:31.659+0000 7f8a13400640 1 -- 172.21.15.22:0/629902748 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a0c014850 con 0x55bc2fb91770 2024-03-28T18:06:31.661 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:31.660+0000 7f8a2ce1f740 1 -- 172.21.15.22:0/629902748 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55bc2fb91770 msgr2=0x55bc2fba27a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:06:31.661 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:31.660+0000 7f8a2ce1f740 1 --2- 172.21.15.22:0/629902748 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55bc2fb91770 0x55bc2fba27a0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x55bc2fba78d0 tx=0x7f8a0c009560 comp rx=0 tx=0).stop 2024-03-28T18:06:31.661 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:31.660+0000 7f8a2ce1f740 1 -- 172.21.15.22:0/629902748 shutdown_connections 2024-03-28T18:06:31.661 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:31.660+0000 7f8a2ce1f740 1 --2- 172.21.15.22:0/629902748 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55bc2fb968e0 0x55bc2fb8ac60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:06:31.661 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:31.660+0000 7f8a2ce1f740 1 --2- 172.21.15.22:0/629902748 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55bc2fb91770 0x55bc2fba27a0 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:06:31.661 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:31.660+0000 7f8a2ce1f740 1 --2- 172.21.15.22:0/629902748 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55bc2fb90d80 0x55bc2fb9b5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:06:31.661 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:31.660+0000 7f8a2ce1f740 1 -- 172.21.15.22:0/629902748 >> 172.21.15.22:0/629902748 conn(0x55bc2fb8c2f0 msgr2=0x55bc2fb9aea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:06:31.661 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:31.660+0000 7f8a2ce1f740 1 -- 172.21.15.22:0/629902748 shutdown_connections 2024-03-28T18:06:31.661 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:31.660+0000 7f8a2ce1f740 1 -- 172.21.15.22:0/629902748 wait complete. 2024-03-28T18:06:31.664 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:31.661+0000 7f8a2ce1f740 1 -- start start 2024-03-28T18:06:31.664 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:31.661+0000 7f8a2ce1f740 1 Processor -- start 2024-03-28T18:06:31.664 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:31.661+0000 7f8a2ce1f740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55bc2fb90d80 0x55bc2fb9e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:06:31.664 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:31.661+0000 7f8a2ce1f740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55bc2fb91770 0x55bc2fb9eee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:06:31.664 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:31.661+0000 7f8a2ce1f740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55bc2fb968e0 0x55bc2fba1f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:06:31.664 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:31.661+0000 7f8a2ce1f740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bc2fba8690 con 0x55bc2fb968e0 2024-03-28T18:06:31.664 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:31.661+0000 7f8a2ce1f740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bc2fba8030 con 0x55bc2fb91770 2024-03-28T18:06:31.664 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:31.661+0000 7f8a2ce1f740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bc2fba8cf0 con 0x55bc2fb90d80 2024-03-28T18:06:31.665 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:31.661+0000 7f8a2ce1f740 10 client.0 _ll_register_callbacks cb 0x7f8a1c008ad0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-28T18:06:31.665 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:31.661+0000 7f8a23e00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55bc2fb968e0 0x55bc2fba1f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:06:31.665 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:31.661+0000 7f8a23e00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55bc2fb968e0 0x55bc2fba1f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:49338/0 (socket says 172.21.15.22:49338) 2024-03-28T18:06:31.665 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:31.661+0000 7f8a23e00640 1 -- 172.21.15.22:0/640000818 learned_addr learned my addr 172.21.15.22:0/640000818 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-28T18:06:31.665 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:31.661+0000 7f8a1aa00640 1 --2- 172.21.15.22:0/640000818 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55bc2fb91770 0x55bc2fb9eee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:06:31.665 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:31.662+0000 7f8a23e00640 1 --2- 172.21.15.22:0/640000818 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55bc2fb90d80 0x55bc2fb9e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:06:31.665 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:31.662+0000 7f8a23e00640 1 -- 172.21.15.22:0/640000818 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55bc2fb90d80 msgr2=0x55bc2fb9e9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T18:06:31.665 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:31.662+0000 7f8a23e00640 1 --2- 172.21.15.22:0/640000818 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55bc2fb90d80 0x55bc2fb9e9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:06:31.665 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:31.662+0000 7f8a23e00640 1 -- 172.21.15.22:0/640000818 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55bc2fb91770 msgr2=0x55bc2fb9eee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T18:06:31.665 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:31.662+0000 7f8a23e00640 1 --2- 172.21.15.22:0/640000818 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55bc2fb91770 0x55bc2fb9eee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:06:31.665 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:31.662+0000 7f8a23e00640 1 -- 172.21.15.22:0/640000818 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55bc2fba24f0 con 0x55bc2fb968e0 2024-03-28T18:06:31.665 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:31.662+0000 7f8a23e00640 1 --2- 172.21.15.22:0/640000818 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55bc2fb968e0 0x55bc2fba1f00 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f8a1c005350 tx=0x7f8a1c000f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=68e8bedd6cf4387 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:06:31.665 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:31.662+0000 7f8a10c00640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-03-28T18:06:31.666 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:31.662+0000 7f8a10c00640 1 -- 172.21.15.22:0/640000818 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a1c03e060 con 0x55bc2fb968e0 2024-03-28T18:06:31.666 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:31.662+0000 7f8a10c00640 1 -- 172.21.15.22:0/640000818 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 8153+0+0 (secure 0 0 0) 0x7f8a1c020040 con 0x55bc2fb968e0 2024-03-28T18:06:31.666 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:31.663+0000 7f8a2ce1f740 1 -- 172.21.15.22:0/640000818 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f8a0c004730 con 0x55bc2fb968e0 2024-03-28T18:06:31.666 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:31.663+0000 7f8a10c00640 1 -- 172.21.15.22:0/640000818 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8a1c004e40 con 0x55bc2fb968e0 2024-03-28T18:06:31.666 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:31.663+0000 7f8a2ce1f740 10 client.15150 fetch_fsmap learned FSMap version 26 2024-03-28T18:06:31.666 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:31.663+0000 7f8a2ce1f740 1 -- 172.21.15.22:0/640000818 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x55bc2fc0bdc0 con 0x55bc2fb968e0 2024-03-28T18:06:31.666 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:31.663+0000 7f8a10c00640 1 -- 172.21.15.22:0/640000818 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f8a0c004730 con 0x55bc2fb968e0 2024-03-28T18:06:31.666 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:31.663+0000 7f8a10c00640 1 -- 172.21.15.22:0/640000818 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 26) v1 ==== 7934+0+0 (secure 0 0 0) 0x7f8a1c022070 con 0x55bc2fb968e0 2024-03-28T18:06:31.666 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:31.663+0000 7f8a2ce1f740 10 client.15150 fetch_fsmap finished waiting for FSMap version 26 2024-03-28T18:06:31.666 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:31.663+0000 7f8a2ce1f740 10 client.15150 resolve_mds: resolved name 'b' to daemon mds.b 2024-03-28T18:06:31.666 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:31.663+0000 7f8a2ce1f740 20 client.15150 populate_metadata read hostname 'smithi022' 2024-03-28T18:06:31.666 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:31.663+0000 7f8a2ce1f740 1 --2- 172.21.15.22:0/640000818 >> [v2:172.21.15.79:6838/2441542443,v1:172.21.15.79:6839/2441542443] conn(0x55bc2fc104a0 0x55bc2fc0c050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:06:31.666 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:31.663+0000 7f8a2ce1f740 4 client.15150 mds_command: new command op to 34553 tid=0 multi_id=0 [{"prefix": "status"}] 2024-03-28T18:06:31.666 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:31.663+0000 7f8a2ce1f740 1 -- 172.21.15.22:0/640000818 --> [v2:172.21.15.79:6838/2441542443,v1:172.21.15.79:6839/2441542443] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55bc2fb031d0 con 0x55bc2fc104a0 2024-03-28T18:06:31.667 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:31.664+0000 7f8a1aa00640 1 --2- 172.21.15.22:0/640000818 >> [v2:172.21.15.79:6838/2441542443,v1:172.21.15.79:6839/2441542443] conn(0x55bc2fc104a0 0x55bc2fc0c050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:06:31.668 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:31.665+0000 7f8a1aa00640 1 --2- 172.21.15.22:0/640000818 >> [v2:172.21.15.79:6838/2441542443,v1:172.21.15.79:6839/2441542443] conn(0x55bc2fc104a0 0x55bc2fc0c050 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9cbe0bee0f0ceac4 server_cookie=95ea343fae723fd2 in_seq=0 out_seq=0 2024-03-28T18:06:31.668 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:31.665+0000 7f8a10c00640 10 client.15150 ms_handle_connect on v2:172.21.15.79:6838/2441542443 2024-03-28T18:06:31.668 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:31.665+0000 7f8a10c00640 1 -- 172.21.15.22:0/640000818 <== mds.0 v2:172.21.15.79:6838/2441542443 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+326 (crc 0 0 0) 0x55bc2fb031d0 con 0x55bc2fc104a0 2024-03-28T18:06:31.668 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:31.665+0000 7f8a10c00640 10 client.15150 handle_command_reply: tid=0 2024-03-28T18:06:31.668 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:31.665+0000 7f8a2ce1f740 1 client.15150 shutdown 2024-03-28T18:06:31.668 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:31.666+0000 7f8a2ce1f740 1 -- 172.21.15.22:0/640000818 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55bc2fb968e0 msgr2=0x55bc2fba1f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:06:31.668 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:31.666+0000 7f8a2ce1f740 1 --2- 172.21.15.22:0/640000818 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55bc2fb968e0 0x55bc2fba1f00 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f8a1c005350 tx=0x7f8a1c000f40 comp rx=0 tx=0).stop 2024-03-28T18:06:31.668 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:31.666+0000 7f8a2ce1f740 1 -- 172.21.15.22:0/640000818 shutdown_connections 2024-03-28T18:06:31.668 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:31.666+0000 7f8a2ce1f740 1 --2- 172.21.15.22:0/640000818 >> [v2:172.21.15.79:6838/2441542443,v1:172.21.15.79:6839/2441542443] conn(0x55bc2fc104a0 0x55bc2fc0c050 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:06:31.668 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:31.666+0000 7f8a2ce1f740 1 --2- 172.21.15.22:0/640000818 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55bc2fb968e0 0x55bc2fba1f00 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:06:31.668 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:31.666+0000 7f8a2ce1f740 1 --2- 172.21.15.22:0/640000818 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55bc2fb91770 0x55bc2fb9eee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:06:31.668 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:31.666+0000 7f8a2ce1f740 1 --2- 172.21.15.22:0/640000818 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55bc2fb90d80 0x55bc2fb9e9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:06:31.669 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:31.666+0000 7f8a2ce1f740 1 -- 172.21.15.22:0/640000818 >> 172.21.15.22:0/640000818 conn(0x55bc2fb8c2f0 msgr2=0x55bc2fb8f550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:06:31.669 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:31.666+0000 7f8a10c00640 0 client.15150 ms_handle_reset on v2:172.21.15.79:6838/2441542443 2024-03-28T18:06:31.669 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:31.666+0000 7f8a2ce1f740 1 -- 172.21.15.22:0/640000818 shutdown_connections 2024-03-28T18:06:31.669 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:31.666+0000 7f8a2ce1f740 1 -- 172.21.15.22:0/640000818 wait complete. 2024-03-28T18:06:31.669 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:31.666+0000 7f8a2ce1f740 20 client.15150 trim_cache size 0 max 16384 2024-03-28T18:06:31.669 INFO:teuthology.orchestra.run.smithi022.stdout:{ 2024-03-28T18:06:31.669 INFO:teuthology.orchestra.run.smithi022.stdout: "cluster_fsid": "fd220748-ed2b-11ee-95d0-87774f69a715", 2024-03-28T18:06:31.669 INFO:teuthology.orchestra.run.smithi022.stdout: "whoami": 0, 2024-03-28T18:06:31.669 INFO:teuthology.orchestra.run.smithi022.stdout: "id": 34553, 2024-03-28T18:06:31.669 INFO:teuthology.orchestra.run.smithi022.stdout: "want_state": "up:active", 2024-03-28T18:06:31.669 INFO:teuthology.orchestra.run.smithi022.stdout: "state": "up:active", 2024-03-28T18:06:31.669 INFO:teuthology.orchestra.run.smithi022.stdout: "fs_name": "cephfs", 2024-03-28T18:06:31.670 INFO:teuthology.orchestra.run.smithi022.stdout: "rank_uptime": 101.81138424, 2024-03-28T18:06:31.670 INFO:teuthology.orchestra.run.smithi022.stdout: "mdsmap_epoch": 26, 2024-03-28T18:06:31.670 INFO:teuthology.orchestra.run.smithi022.stdout: "osdmap_epoch": 92, 2024-03-28T18:06:31.670 INFO:teuthology.orchestra.run.smithi022.stdout: "osdmap_epoch_barrier": 90, 2024-03-28T18:06:31.670 INFO:teuthology.orchestra.run.smithi022.stdout: "uptime": 107.38579144800001 2024-03-28T18:06:31.670 INFO:teuthology.orchestra.run.smithi022.stdout:} 2024-03-28T18:06:31.670 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:31.667+0000 7f8a28a00640 1 -- 172.21.15.22:0/3293893861 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7f8a080a2800 msgr2=0x7f8a080a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:06:31.671 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:31.667+0000 7f8a28a00640 1 --2- 172.21.15.22:0/3293893861 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7f8a080a2800 0x7f8a080a4cc0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f8a140ee370 tx=0x7f8a14009260 comp rx=0 tx=0).stop 2024-03-28T18:06:31.671 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:31.667+0000 7f8a28a00640 1 -- 172.21.15.22:0/3293893861 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8a241015a0 msgr2=0x7f8a24199ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:06:31.671 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:31.667+0000 7f8a28a00640 1 --2- 172.21.15.22:0/3293893861 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8a241015a0 0x7f8a24199ac0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f8a14005ab0 tx=0x7f8a14004e90 comp rx=0 tx=0).stop 2024-03-28T18:06:31.671 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:31.667+0000 7f8a28a00640 1 -- 172.21.15.22:0/3293893861 shutdown_connections 2024-03-28T18:06:31.671 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:31.667+0000 7f8a28a00640 1 --2- 172.21.15.22:0/3293893861 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7f8a080a2800 0x7f8a080a4cc0 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:06:31.671 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:31.667+0000 7f8a28a00640 1 --2- 172.21.15.22:0/3293893861 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f8a24103150 0x7f8a241a1070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:06:31.671 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:31.667+0000 7f8a28a00640 1 --2- 172.21.15.22:0/3293893861 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8a24102790 0x7f8a2419a000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:06:31.671 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:31.667+0000 7f8a28a00640 1 --2- 172.21.15.22:0/3293893861 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8a241015a0 0x7f8a24199ac0 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:06:31.671 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:31.667+0000 7f8a28a00640 1 -- 172.21.15.22:0/3293893861 >> 172.21.15.22:0/3293893861 conn(0x7f8a240fcb10 msgr2=0x7f8a240738d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:06:31.671 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:31.667+0000 7f8a28a00640 1 -- 172.21.15.22:0/3293893861 shutdown_connections 2024-03-28T18:06:31.671 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:31.667+0000 7f8a28a00640 1 -- 172.21.15.22:0/3293893861 wait complete. 2024-03-28T18:06:31.683 INFO:tasks.cephadm:Setup complete, yielding 2024-03-28T18:06:31.684 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-03-28T18:06:31.694 INFO:tasks.cephadm:Running commands on role mon.a host ubuntu@smithi022.front.sepia.ceph.com 2024-03-28T18:06:31.694 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fd220748-ed2b-11ee-95d0-87774f69a715 -- bash -c 'ceph orch status' 2024-03-28T18:06:32.159 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/fd220748-ed2b-11ee-95d0-87774f69a715/mon.a/config 2024-03-28T18:06:32.306 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:06:32 smithi022 ceph-mon[31985]: pgmap v422: 97 pgs: 97 active+clean; 643 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-03-28T18:06:32.424 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:06:32 smithi134 ceph-mon[35037]: pgmap v422: 97 pgs: 97 active+clean; 643 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-03-28T18:06:32.432 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:06:32 smithi079 ceph-mon[35312]: pgmap v422: 97 pgs: 97 active+clean; 643 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-03-28T18:06:34.189 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:06:34 smithi022 ceph-mon[31985]: pgmap v423: 97 pgs: 97 active+clean; 643 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-03-28T18:06:34.365 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:34.365+0000 7f58cc400700 1 -- 172.21.15.22:0/2755073875 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f58c4109a10 msgr2=0x7f58c4109e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:06:34.365 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:34.365+0000 7f58cc400700 1 --2- 172.21.15.22:0/2755073875 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f58c4109a10 0x7f58c4109e70 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f58b00099c0 tx=0x7f58b0009cd0 comp rx=0 tx=0).stop 2024-03-28T18:06:34.365 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:34.365+0000 7f58cc400700 1 -- 172.21.15.22:0/2755073875 shutdown_connections 2024-03-28T18:06:34.365 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:34.365+0000 7f58cc400700 1 --2- 172.21.15.22:0/2755073875 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f58c410a3b0 0x7f58c4110c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:06:34.365 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:34.365+0000 7f58cc400700 1 --2- 172.21.15.22:0/2755073875 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f58c4108810 0x7f58c4108c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:06:34.365 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:34.365+0000 7f58cc400700 1 --2- 172.21.15.22:0/2755073875 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f58c4109a10 0x7f58c4109e70 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:06:34.365 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:34.365+0000 7f58cc400700 1 -- 172.21.15.22:0/2755073875 >> 172.21.15.22:0/2755073875 conn(0x7f58c40777e0 msgr2=0x7f58c4079c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:06:34.365 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:34.365+0000 7f58cc400700 1 -- 172.21.15.22:0/2755073875 shutdown_connections 2024-03-28T18:06:34.366 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:34.365+0000 7f58cc400700 1 -- 172.21.15.22:0/2755073875 wait complete. 2024-03-28T18:06:34.366 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:34.366+0000 7f58cc400700 1 Processor -- start 2024-03-28T18:06:34.366 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:34.366+0000 7f58cc400700 1 -- start start 2024-03-28T18:06:34.367 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:34.367+0000 7f58cc400700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f58c4108810 0x7f58c419e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:06:34.367 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:34.367+0000 7f58cc400700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f58c4109a10 0x7f58c419eee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:06:34.368 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:34.367+0000 7f58cc400700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f58c410a3b0 0x7f58c41a5f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:06:34.368 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:34.367+0000 7f58cc400700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58c41143f0 con 0x7f58c4109a10 2024-03-28T18:06:34.368 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:34.367+0000 7f58cc400700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58c4114270 con 0x7f58c4108810 2024-03-28T18:06:34.368 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:34.367+0000 7f58cc400700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58c4114570 con 0x7f58c410a3b0 2024-03-28T18:06:34.368 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:34.367+0000 7f58c9000700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f58c4109a10 0x7f58c419eee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:06:34.368 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:34.367+0000 7f58c9000700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f58c4109a10 0x7f58c419eee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:49360/0 (socket says 172.21.15.22:49360) 2024-03-28T18:06:34.368 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:34.367+0000 7f58c9000700 1 -- 172.21.15.22:0/1934471632 learned_addr learned my addr 172.21.15.22:0/1934471632 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-28T18:06:34.368 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:34.367+0000 7f58c9a00700 1 --2- 172.21.15.22:0/1934471632 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f58c4108810 0x7f58c419e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:06:34.368 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:34.368+0000 7f58ca400700 1 --2- 172.21.15.22:0/1934471632 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f58c410a3b0 0x7f58c41a5f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:06:34.368 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:34.368+0000 7f58c9000700 1 -- 172.21.15.22:0/1934471632 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f58c410a3b0 msgr2=0x7f58c41a5f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:06:34.368 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:34.368+0000 7f58c9000700 1 --2- 172.21.15.22:0/1934471632 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f58c410a3b0 0x7f58c41a5f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:06:34.368 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:34.368+0000 7f58c9000700 1 -- 172.21.15.22:0/1934471632 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f58c4108810 msgr2=0x7f58c419e9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:06:34.368 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:34.368+0000 7f58c9000700 1 --2- 172.21.15.22:0/1934471632 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f58c4108810 0x7f58c419e9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:06:34.369 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:34.368+0000 7f58c9000700 1 -- 172.21.15.22:0/1934471632 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f58c41a6600 con 0x7f58c4109a10 2024-03-28T18:06:34.369 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:34.368+0000 7f58ca400700 1 --2- 172.21.15.22:0/1934471632 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f58c410a3b0 0x7f58c41a5f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T18:06:34.369 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:34.368+0000 7f58c9000700 1 --2- 172.21.15.22:0/1934471632 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f58c4109a10 0x7f58c419eee0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f58b0000c00 tx=0x7f58b000bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:06:34.369 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:34.368+0000 7f58c2000700 1 -- 172.21.15.22:0/1934471632 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58b0009e60 con 0x7f58c4109a10 2024-03-28T18:06:34.370 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:34.368+0000 7f58c2000700 1 -- 172.21.15.22:0/1934471632 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f58b000f460 con 0x7f58c4109a10 2024-03-28T18:06:34.370 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:34.368+0000 7f58c2000700 1 -- 172.21.15.22:0/1934471632 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58b0017650 con 0x7f58c4109a10 2024-03-28T18:06:34.370 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:34.368+0000 7f58cc400700 1 -- 172.21.15.22:0/1934471632 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f58c41a6890 con 0x7f58c4109a10 2024-03-28T18:06:34.370 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:34.369+0000 7f58cc400700 1 -- 172.21.15.22:0/1934471632 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f58c41a6c90 con 0x7f58c4109a10 2024-03-28T18:06:34.370 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:34.370+0000 7f58cc400700 1 -- 172.21.15.22:0/1934471632 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f58c4068e50 con 0x7f58c4109a10 2024-03-28T18:06:34.371 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:34.372+0000 7f58c2000700 1 -- 172.21.15.22:0/1934471632 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 18) v1 ==== 137290+0+0 (secure 0 0 0) 0x7f58b000fa30 con 0x7f58c4109a10 2024-03-28T18:06:34.373 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:34.373+0000 7f58c2000700 1 --2- 172.21.15.22:0/1934471632 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7f58ac0a2890 0x7f58ac0a4d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:06:34.373 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:34.373+0000 7f58c2000700 1 -- 172.21.15.22:0/1934471632 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 8153+0+0 (secure 0 0 0) 0x7f58b00cfa10 con 0x7f58c4109a10 2024-03-28T18:06:34.373 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:34.373+0000 7f58c9a00700 1 --2- 172.21.15.22:0/1934471632 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7f58ac0a2890 0x7f58ac0a4d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:06:34.373 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:34.374+0000 7f58c9a00700 1 --2- 172.21.15.22:0/1934471632 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7f58ac0a2890 0x7f58ac0a4d50 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f58c4109870 tx=0x7f58b8009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:06:34.378 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:34.378+0000 7f58c2000700 1 -- 172.21.15.22:0/1934471632 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f58b0013070 con 0x7f58c4109a10 2024-03-28T18:06:34.424 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:06:34 smithi134 ceph-mon[35037]: pgmap v423: 97 pgs: 97 active+clean; 643 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-03-28T18:06:34.432 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:06:34 smithi079 ceph-mon[35312]: pgmap v423: 97 pgs: 97 active+clean; 643 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-03-28T18:06:34.628 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:34.628+0000 7f58cc400700 1 -- 172.21.15.22:0/1934471632 --> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] -- mgr_command(tid 0: {"prefix": "orch status", "target": ["mon-mgr", ""]}) v1 -- 0x7f58c40625d0 con 0x7f58ac0a2890 2024-03-28T18:06:34.630 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:34.631+0000 7f58c2000700 1 -- 172.21.15.22:0/1934471632 <== mgr.14152 v2:172.21.15.22:6800/4201048392 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+43 (secure 0 0 0) 0x7f58c40625d0 con 0x7f58ac0a2890 2024-03-28T18:06:34.631 INFO:teuthology.orchestra.run.smithi022.stdout:Backend: cephadm 2024-03-28T18:06:34.631 INFO:teuthology.orchestra.run.smithi022.stdout:Available: Yes 2024-03-28T18:06:34.631 INFO:teuthology.orchestra.run.smithi022.stdout:Paused: No 2024-03-28T18:06:34.634 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:34.634+0000 7f58cc400700 1 -- 172.21.15.22:0/1934471632 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7f58ac0a2890 msgr2=0x7f58ac0a4d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:06:34.634 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:34.634+0000 7f58cc400700 1 --2- 172.21.15.22:0/1934471632 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7f58ac0a2890 0x7f58ac0a4d50 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f58c4109870 tx=0x7f58b8009260 comp rx=0 tx=0).stop 2024-03-28T18:06:34.634 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:34.634+0000 7f58cc400700 1 -- 172.21.15.22:0/1934471632 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f58c4109a10 msgr2=0x7f58c419eee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:06:34.634 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:34.634+0000 7f58cc400700 1 --2- 172.21.15.22:0/1934471632 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f58c4109a10 0x7f58c419eee0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f58b0000c00 tx=0x7f58b000bee0 comp rx=0 tx=0).stop 2024-03-28T18:06:34.634 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:34.634+0000 7f58cc400700 1 -- 172.21.15.22:0/1934471632 shutdown_connections 2024-03-28T18:06:34.634 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:34.634+0000 7f58cc400700 1 --2- 172.21.15.22:0/1934471632 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7f58ac0a2890 0x7f58ac0a4d50 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:06:34.634 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:34.634+0000 7f58cc400700 1 --2- 172.21.15.22:0/1934471632 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f58c410a3b0 0x7f58c41a5f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:06:34.634 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:34.634+0000 7f58cc400700 1 --2- 172.21.15.22:0/1934471632 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f58c4108810 0x7f58c419e9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:06:34.634 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:34.634+0000 7f58cc400700 1 --2- 172.21.15.22:0/1934471632 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f58c4109a10 0x7f58c419eee0 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:06:34.635 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:34.634+0000 7f58cc400700 1 -- 172.21.15.22:0/1934471632 >> 172.21.15.22:0/1934471632 conn(0x7f58c40777e0 msgr2=0x7f58c410cd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:06:34.635 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:34.634+0000 7f58cc400700 1 -- 172.21.15.22:0/1934471632 shutdown_connections 2024-03-28T18:06:34.635 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:34.635+0000 7f58cc400700 1 -- 172.21.15.22:0/1934471632 wait complete. 2024-03-28T18:06:35.414 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fd220748-ed2b-11ee-95d0-87774f69a715 -- bash -c 'ceph orch ps' 2024-03-28T18:06:35.892 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/fd220748-ed2b-11ee-95d0-87774f69a715/mon.a/config 2024-03-28T18:06:36.409 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:06:36 smithi022 ceph-mon[31985]: from='client.15156 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-28T18:06:36.409 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:06:36 smithi022 ceph-mon[31985]: pgmap v424: 97 pgs: 97 active+clean; 643 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-28T18:06:36.424 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:06:36 smithi134 ceph-mon[35037]: from='client.15156 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-28T18:06:36.424 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:06:36 smithi134 ceph-mon[35037]: pgmap v424: 97 pgs: 97 active+clean; 643 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-28T18:06:36.432 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:06:36 smithi079 ceph-mon[35312]: from='client.15156 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-28T18:06:36.432 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:06:36 smithi079 ceph-mon[35312]: pgmap v424: 97 pgs: 97 active+clean; 643 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-28T18:06:37.952 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:37.950+0000 7f0556000700 1 -- 172.21.15.22:0/781420795 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0550107eb0 msgr2=0x7f055010a320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:06:37.952 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:37.950+0000 7f0556000700 1 --2- 172.21.15.22:0/781420795 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0550107eb0 0x7f055010a320 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f054800b0a0 tx=0x7f054800b3b0 comp rx=0 tx=0).stop 2024-03-28T18:06:37.953 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:37.950+0000 7f0556000700 1 -- 172.21.15.22:0/781420795 shutdown_connections 2024-03-28T18:06:37.953 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:37.950+0000 7f0556000700 1 --2- 172.21.15.22:0/781420795 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0550107eb0 0x7f055010a320 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:06:37.953 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:37.950+0000 7f0556000700 1 --2- 172.21.15.22:0/781420795 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0550102c50 0x7f0550105040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:06:37.953 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:37.950+0000 7f0556000700 1 --2- 172.21.15.22:0/781420795 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f0550105580 0x7f0550107970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:06:37.953 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:37.950+0000 7f0556000700 1 -- 172.21.15.22:0/781420795 >> 172.21.15.22:0/781420795 conn(0x7f05500fc660 msgr2=0x7f05500feaa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:06:37.953 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:37.950+0000 7f0556000700 1 -- 172.21.15.22:0/781420795 shutdown_connections 2024-03-28T18:06:37.953 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:37.950+0000 7f0556000700 1 -- 172.21.15.22:0/781420795 wait complete. 2024-03-28T18:06:37.953 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:37.951+0000 7f0556000700 1 Processor -- start 2024-03-28T18:06:37.953 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:37.951+0000 7f0556000700 1 -- start start 2024-03-28T18:06:37.953 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:37.951+0000 7f0556000700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0550102c50 0x7f055019a630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:06:37.953 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:37.951+0000 7f0556000700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0550105580 0x7f055019ab70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:06:37.953 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:37.951+0000 7f0556000700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f0550107eb0 0x7f05501a1bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:06:37.953 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:37.951+0000 7f0556000700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f055010ddb0 con 0x7f0550102c50 2024-03-28T18:06:37.954 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:37.951+0000 7f0556000700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f055010dc30 con 0x7f0550107eb0 2024-03-28T18:06:37.954 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:37.951+0000 7f0556000700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f055010df30 con 0x7f0550105580 2024-03-28T18:06:37.954 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:37.952+0000 7f054fe00700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f0550107eb0 0x7f05501a1bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:06:37.954 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:37.952+0000 7f054fe00700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f0550107eb0 0x7f05501a1bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.22:45852/0 (socket says 172.21.15.22:45852) 2024-03-28T18:06:37.954 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:37.952+0000 7f054fe00700 1 -- 172.21.15.22:0/199729086 learned_addr learned my addr 172.21.15.22:0/199729086 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-28T18:06:37.954 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:37.952+0000 7f054f400700 1 --2- 172.21.15.22:0/199729086 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0550102c50 0x7f055019a630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:06:37.954 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:37.952+0000 7f0546a00700 1 --2- 172.21.15.22:0/199729086 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0550105580 0x7f055019ab70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:06:37.955 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:37.952+0000 7f054fe00700 1 -- 172.21.15.22:0/199729086 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0550105580 msgr2=0x7f055019ab70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:06:37.955 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:37.952+0000 7f054fe00700 1 --2- 172.21.15.22:0/199729086 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0550105580 0x7f055019ab70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:06:37.955 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:37.952+0000 7f054fe00700 1 -- 172.21.15.22:0/199729086 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0550102c50 msgr2=0x7f055019a630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:06:37.955 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:37.952+0000 7f054fe00700 1 --2- 172.21.15.22:0/199729086 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0550102c50 0x7f055019a630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:06:37.955 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:37.952+0000 7f054fe00700 1 -- 172.21.15.22:0/199729086 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f05501a2290 con 0x7f0550107eb0 2024-03-28T18:06:37.955 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:37.952+0000 7f054f400700 1 --2- 172.21.15.22:0/199729086 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0550102c50 0x7f055019a630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-28T18:06:37.955 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:37.952+0000 7f054fe00700 1 --2- 172.21.15.22:0/199729086 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f0550107eb0 0x7f05501a1bf0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f0548006070 tx=0x7f05480095c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:06:37.955 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:37.953+0000 7f054cc00700 1 -- 172.21.15.22:0/199729086 <== mon.2 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0548021070 con 0x7f0550107eb0 2024-03-28T18:06:37.956 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:37.953+0000 7f0556000700 1 -- 172.21.15.22:0/199729086 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f05501a2520 con 0x7f0550107eb0 2024-03-28T18:06:37.956 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:37.953+0000 7f054cc00700 1 -- 172.21.15.22:0/199729086 <== mon.2 v2:172.21.15.79:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f0548007c80 con 0x7f0550107eb0 2024-03-28T18:06:37.956 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:37.953+0000 7f054cc00700 1 -- 172.21.15.22:0/199729086 <== mon.2 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f054801d450 con 0x7f0550107eb0 2024-03-28T18:06:37.956 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:37.953+0000 7f0556000700 1 -- 172.21.15.22:0/199729086 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f05501a2a20 con 0x7f0550107eb0 2024-03-28T18:06:37.956 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:37.954+0000 7f0556000700 1 -- 172.21.15.22:0/199729086 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f052c005330 con 0x7f0550107eb0 2024-03-28T18:06:37.957 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:37.957+0000 7f054cc00700 1 -- 172.21.15.22:0/199729086 <== mon.2 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 18) v1 ==== 137290+0+0 (secure 0 0 0) 0x7f0548004000 con 0x7f0550107eb0 2024-03-28T18:06:37.959 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:37.958+0000 7f054cc00700 1 --2- 172.21.15.22:0/199729086 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7f05340a2910 0x7f05340a4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:06:37.959 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:37.958+0000 7f054cc00700 1 -- 172.21.15.22:0/199729086 <== mon.2 v2:172.21.15.79:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 8153+0+0 (secure 0 0 0) 0x7f054800f020 con 0x7f0550107eb0 2024-03-28T18:06:37.960 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:37.958+0000 7f054f400700 1 --2- 172.21.15.22:0/199729086 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7f05340a2910 0x7f05340a4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:06:37.960 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:37.959+0000 7f054f400700 1 --2- 172.21.15.22:0/199729086 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7f05340a2910 0x7f05340a4dd0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f0540005950 tx=0x7f054000a300 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:06:37.967 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:37.966+0000 7f054cc00700 1 -- 172.21.15.22:0/199729086 <== mon.2 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f054802a030 con 0x7f0550107eb0 2024-03-28T18:06:38.207 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:38.206+0000 7f0556000700 1 -- 172.21.15.22:0/199729086 --> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f052c000bc0 con 0x7f05340a2910 2024-03-28T18:06:38.222 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:38.222+0000 7f054cc00700 1 -- 172.21.15.22:0/199729086 <== mgr.14152 v2:172.21.15.22:6800/4201048392 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+4185 (secure 0 0 0) 0x7f052c000bc0 con 0x7f05340a2910 2024-03-28T18:06:38.223 INFO:teuthology.orchestra.run.smithi022.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-03-28T18:06:38.224 INFO:teuthology.orchestra.run.smithi022.stdout:mds.a smithi022 running (118s) 24s ago 117s 15.8M - 19.0.0-2145-g3be9a5c9 c664afe56bb0 48f0c7328169 2024-03-28T18:06:38.224 INFO:teuthology.orchestra.run.smithi022.stdout:mds.b smithi079 running (114s) 41s ago 113s 24.8M - 19.0.0-2145-g3be9a5c9 c664afe56bb0 23e0c2fd50ec 2024-03-28T18:06:38.224 INFO:teuthology.orchestra.run.smithi022.stdout:mds.c smithi134 running (2m) 42s ago 2m 16.7M - 19.0.0-2145-g3be9a5c9 c664afe56bb0 a8bde952d364 2024-03-28T18:06:38.224 INFO:teuthology.orchestra.run.smithi022.stdout:mds.d smithi022 running (2m) 24s ago 2m 12.4M - 19.0.0-2145-g3be9a5c9 c664afe56bb0 9c7b84e0d124 2024-03-28T18:06:38.224 INFO:teuthology.orchestra.run.smithi022.stdout:mds.e smithi079 running (2m) 41s ago 2m 16.6M - 19.0.0-2145-g3be9a5c9 c664afe56bb0 4a2d81a3d9f5 2024-03-28T18:06:38.224 INFO:teuthology.orchestra.run.smithi022.stdout:mds.f smithi134 running (2m) 42s ago 2m 15.4M - 19.0.0-2145-g3be9a5c9 c664afe56bb0 7fe8640da6db 2024-03-28T18:06:38.224 INFO:teuthology.orchestra.run.smithi022.stdout:mds.g smithi022 running (2m) 24s ago 2m 15.9M - 19.0.0-2145-g3be9a5c9 c664afe56bb0 7fac2ccc2c03 2024-03-28T18:06:38.224 INFO:teuthology.orchestra.run.smithi022.stdout:mds.h smithi079 running (2m) 41s ago 2m 16.4M - 19.0.0-2145-g3be9a5c9 c664afe56bb0 6faabd85524b 2024-03-28T18:06:38.224 INFO:teuthology.orchestra.run.smithi022.stdout:mds.i smithi134 running (2m) 42s ago 2m 16.0M - 19.0.0-2145-g3be9a5c9 c664afe56bb0 43deaed0d42a 2024-03-28T18:06:38.224 INFO:teuthology.orchestra.run.smithi022.stdout:mds.j smithi022 running (2m) 24s ago 2m 12.5M - 19.0.0-2145-g3be9a5c9 c664afe56bb0 43ead9efefab 2024-03-28T18:06:38.224 INFO:teuthology.orchestra.run.smithi022.stdout:mds.k smithi079 running (2m) 41s ago 2m 16.2M - 19.0.0-2145-g3be9a5c9 c664afe56bb0 0f075f827bbd 2024-03-28T18:06:38.224 INFO:teuthology.orchestra.run.smithi022.stdout:mds.l smithi134 running (2m) 42s ago 2m 15.6M - 19.0.0-2145-g3be9a5c9 c664afe56bb0 5c76242affe2 2024-03-28T18:06:38.224 INFO:teuthology.orchestra.run.smithi022.stdout:mgr.x smithi022 *:9283,8765 running (12m) 24s ago 12m 516M - 19.0.0-2145-g3be9a5c9 c664afe56bb0 6c8e21b01d7b 2024-03-28T18:06:38.224 INFO:teuthology.orchestra.run.smithi022.stdout:mgr.y smithi079 *:8443,8765 running (9m) 41s ago 9m 451M - 19.0.0-2145-g3be9a5c9 c664afe56bb0 35ad6cc68d5d 2024-03-28T18:06:38.225 INFO:teuthology.orchestra.run.smithi022.stdout:mgr.z smithi134 *:8443,8765 running (9m) 42s ago 9m 451M - 19.0.0-2145-g3be9a5c9 c664afe56bb0 f12167a387b9 2024-03-28T18:06:38.225 INFO:teuthology.orchestra.run.smithi022.stdout:mon.a smithi022 running (12m) 24s ago 12m 50.1M 2048M 19.0.0-2145-g3be9a5c9 c664afe56bb0 c8a13c8d55b2 2024-03-28T18:06:38.225 INFO:teuthology.orchestra.run.smithi022.stdout:mon.b smithi079 running (10m) 41s ago 10m 40.3M 2048M 19.0.0-2145-g3be9a5c9 c664afe56bb0 7fcef29b8b90 2024-03-28T18:06:38.225 INFO:teuthology.orchestra.run.smithi022.stdout:mon.c smithi134 running (10m) 42s ago 10m 39.3M 2048M 19.0.0-2145-g3be9a5c9 c664afe56bb0 a88e0c158577 2024-03-28T18:06:38.225 INFO:teuthology.orchestra.run.smithi022.stdout:osd.0 smithi022 running (9m) 24s ago 9m 47.9M 4273M 19.0.0-2145-g3be9a5c9 c664afe56bb0 2a4299361b6f 2024-03-28T18:06:38.225 INFO:teuthology.orchestra.run.smithi022.stdout:osd.1 smithi079 running (8m) 41s ago 8m 57.1M 4272M 19.0.0-2145-g3be9a5c9 c664afe56bb0 2ca3871abede 2024-03-28T18:06:38.225 INFO:teuthology.orchestra.run.smithi022.stdout:osd.2 smithi134 running (8m) 42s ago 8m 57.4M 4272M 19.0.0-2145-g3be9a5c9 c664afe56bb0 f91abfbc1d47 2024-03-28T18:06:38.225 INFO:teuthology.orchestra.run.smithi022.stdout:osd.3 smithi022 running (7m) 24s ago 7m 53.5M 4273M 19.0.0-2145-g3be9a5c9 c664afe56bb0 21e43297b9a2 2024-03-28T18:06:38.225 INFO:teuthology.orchestra.run.smithi022.stdout:osd.4 smithi079 running (7m) 41s ago 7m 44.9M 4272M 19.0.0-2145-g3be9a5c9 c664afe56bb0 85c1b1febe2a 2024-03-28T18:06:38.225 INFO:teuthology.orchestra.run.smithi022.stdout:osd.5 smithi134 running (6m) 42s ago 6m 53.0M 4272M 19.0.0-2145-g3be9a5c9 c664afe56bb0 f1ac1c93984c 2024-03-28T18:06:38.225 INFO:teuthology.orchestra.run.smithi022.stdout:osd.6 smithi022 running (5m) 24s ago 5m 50.3M 4273M 19.0.0-2145-g3be9a5c9 c664afe56bb0 0b2fff686be7 2024-03-28T18:06:38.225 INFO:teuthology.orchestra.run.smithi022.stdout:osd.7 smithi079 running (5m) 41s ago 5m 46.7M 4272M 19.0.0-2145-g3be9a5c9 c664afe56bb0 db1b4aadd13b 2024-03-28T18:06:38.225 INFO:teuthology.orchestra.run.smithi022.stdout:osd.8 smithi134 running (4m) 42s ago 4m 53.2M 4272M 19.0.0-2145-g3be9a5c9 c664afe56bb0 f359171f2f67 2024-03-28T18:06:38.225 INFO:teuthology.orchestra.run.smithi022.stdout:osd.9 smithi022 running (4m) 24s ago 4m 50.6M 4273M 19.0.0-2145-g3be9a5c9 c664afe56bb0 6a9fbdfb464d 2024-03-28T18:06:38.225 INFO:teuthology.orchestra.run.smithi022.stdout:osd.10 smithi079 running (3m) 41s ago 3m 52.8M 4272M 19.0.0-2145-g3be9a5c9 c664afe56bb0 ef47a1a30fad 2024-03-28T18:06:38.226 INFO:teuthology.orchestra.run.smithi022.stdout:osd.11 smithi134 running (3m) 42s ago 3m 43.0M 4272M 19.0.0-2145-g3be9a5c9 c664afe56bb0 81090a228b3e 2024-03-28T18:06:38.226 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:38.225+0000 7f0556000700 1 -- 172.21.15.22:0/199729086 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7f05340a2910 msgr2=0x7f05340a4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:06:38.226 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:38.225+0000 7f0556000700 1 --2- 172.21.15.22:0/199729086 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7f05340a2910 0x7f05340a4dd0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f0540005950 tx=0x7f054000a300 comp rx=0 tx=0).stop 2024-03-28T18:06:38.227 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:38.225+0000 7f0556000700 1 -- 172.21.15.22:0/199729086 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f0550107eb0 msgr2=0x7f05501a1bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:06:38.227 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:38.225+0000 7f0556000700 1 --2- 172.21.15.22:0/199729086 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f0550107eb0 0x7f05501a1bf0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f0548006070 tx=0x7f05480095c0 comp rx=0 tx=0).stop 2024-03-28T18:06:38.227 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:38.225+0000 7f0556000700 1 -- 172.21.15.22:0/199729086 shutdown_connections 2024-03-28T18:06:38.227 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:38.225+0000 7f0556000700 1 --2- 172.21.15.22:0/199729086 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f0550107eb0 0x7f05501a1bf0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:06:38.227 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:38.225+0000 7f0556000700 1 --2- 172.21.15.22:0/199729086 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0550102c50 0x7f055019a630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:06:38.227 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:38.225+0000 7f0556000700 1 --2- 172.21.15.22:0/199729086 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7f05340a2910 0x7f05340a4dd0 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:06:38.227 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:38.225+0000 7f0556000700 1 --2- 172.21.15.22:0/199729086 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0550105580 0x7f055019ab70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:06:38.227 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:38.225+0000 7f0556000700 1 -- 172.21.15.22:0/199729086 >> 172.21.15.22:0/199729086 conn(0x7f05500fc660 msgr2=0x7f05501023c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:06:38.227 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:38.225+0000 7f0556000700 1 -- 172.21.15.22:0/199729086 shutdown_connections 2024-03-28T18:06:38.227 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:38.225+0000 7f0556000700 1 -- 172.21.15.22:0/199729086 wait complete. 2024-03-28T18:06:38.424 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:06:38 smithi134 ceph-mon[35037]: pgmap v425: 97 pgs: 97 active+clean; 643 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2024-03-28T18:06:38.432 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:06:38 smithi079 ceph-mon[35312]: pgmap v425: 97 pgs: 97 active+clean; 643 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2024-03-28T18:06:38.531 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:06:38 smithi022 ceph-mon[31985]: pgmap v425: 97 pgs: 97 active+clean; 643 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2024-03-28T18:06:38.997 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fd220748-ed2b-11ee-95d0-87774f69a715 -- bash -c 'ceph orch ls' 2024-03-28T18:06:39.281 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:06:39 smithi022 ceph-mon[31985]: from='client.35030 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-28T18:06:39.424 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:06:39 smithi134 ceph-mon[35037]: from='client.35030 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-28T18:06:39.432 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:06:39 smithi079 ceph-mon[35312]: from='client.35030 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-28T18:06:39.465 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/fd220748-ed2b-11ee-95d0-87774f69a715/mon.a/config 2024-03-28T18:06:40.282 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:06:40 smithi022 ceph-mon[31985]: pgmap v426: 97 pgs: 97 active+clean; 643 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-03-28T18:06:40.424 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:06:40 smithi134 ceph-mon[35037]: pgmap v426: 97 pgs: 97 active+clean; 643 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-03-28T18:06:40.432 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:06:40 smithi079 ceph-mon[35312]: pgmap v426: 97 pgs: 97 active+clean; 643 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-03-28T18:06:41.525 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:41.523+0000 7fd1b5a00700 1 -- 172.21.15.22:0/3214096397 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd1b0107eb0 msgr2=0x7fd1b010a320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:06:41.525 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:41.523+0000 7fd1b5a00700 1 --2- 172.21.15.22:0/3214096397 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd1b0107eb0 0x7fd1b010a320 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7fd1a800b0a0 tx=0x7fd1a800b3b0 comp rx=0 tx=0).stop 2024-03-28T18:06:41.525 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:41.524+0000 7fd1b5a00700 1 -- 172.21.15.22:0/3214096397 shutdown_connections 2024-03-28T18:06:41.526 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:41.524+0000 7fd1b5a00700 1 --2- 172.21.15.22:0/3214096397 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd1b0107eb0 0x7fd1b010a320 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:06:41.526 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:41.524+0000 7fd1b5a00700 1 --2- 172.21.15.22:0/3214096397 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd1b0102c50 0x7fd1b0105040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:06:41.526 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:41.524+0000 7fd1b5a00700 1 --2- 172.21.15.22:0/3214096397 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd1b0105580 0x7fd1b0107970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:06:41.526 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:41.524+0000 7fd1b5a00700 1 -- 172.21.15.22:0/3214096397 >> 172.21.15.22:0/3214096397 conn(0x7fd1b00fc660 msgr2=0x7fd1b00feaa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:06:41.526 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:41.524+0000 7fd1b5a00700 1 -- 172.21.15.22:0/3214096397 shutdown_connections 2024-03-28T18:06:41.526 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:41.524+0000 7fd1b5a00700 1 -- 172.21.15.22:0/3214096397 wait complete. 2024-03-28T18:06:41.527 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:41.525+0000 7fd1b5a00700 1 Processor -- start 2024-03-28T18:06:41.527 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:41.526+0000 7fd1b5a00700 1 -- start start 2024-03-28T18:06:41.527 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:41.526+0000 7fd1b5a00700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd1b0102c50 0x7fd1b019a5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:06:41.527 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:41.526+0000 7fd1b5a00700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd1b0105580 0x7fd1b019ab00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:06:41.527 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:41.526+0000 7fd1b5a00700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd1b0107eb0 0x7fd1b01a1b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:06:41.527 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:41.526+0000 7fd1b5a00700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1b010dd80 con 0x7fd1b0102c50 2024-03-28T18:06:41.527 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:41.526+0000 7fd1b5a00700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1b010dc00 con 0x7fd1b0107eb0 2024-03-28T18:06:41.527 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:41.526+0000 7fd1b5a00700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1b010df00 con 0x7fd1b0105580 2024-03-28T18:06:41.528 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:41.526+0000 7fd1aea00700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd1b0102c50 0x7fd1b019a5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:06:41.528 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:41.527+0000 7fd1aea00700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd1b0102c50 0x7fd1b019a5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:54672/0 (socket says 172.21.15.22:54672) 2024-03-28T18:06:41.528 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:41.527+0000 7fd1aea00700 1 -- 172.21.15.22:0/760777150 learned_addr learned my addr 172.21.15.22:0/760777150 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-28T18:06:41.528 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:41.527+0000 7fd1ae000700 1 --2- 172.21.15.22:0/760777150 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd1b0105580 0x7fd1b019ab00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:06:41.528 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:41.527+0000 7fd1af400700 1 --2- 172.21.15.22:0/760777150 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd1b0107eb0 0x7fd1b01a1b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:06:41.528 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:41.527+0000 7fd1aea00700 1 -- 172.21.15.22:0/760777150 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd1b0105580 msgr2=0x7fd1b019ab00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:06:41.528 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:41.527+0000 7fd1aea00700 1 --2- 172.21.15.22:0/760777150 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd1b0105580 0x7fd1b019ab00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:06:41.529 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:41.527+0000 7fd1aea00700 1 -- 172.21.15.22:0/760777150 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd1b0107eb0 msgr2=0x7fd1b01a1b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:06:41.529 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:41.527+0000 7fd1aea00700 1 --2- 172.21.15.22:0/760777150 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd1b0107eb0 0x7fd1b01a1b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:06:41.529 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:41.527+0000 7fd1aea00700 1 -- 172.21.15.22:0/760777150 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd1b01a2220 con 0x7fd1b0102c50 2024-03-28T18:06:41.529 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:41.527+0000 7fd1af400700 1 --2- 172.21.15.22:0/760777150 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd1b0107eb0 0x7fd1b01a1b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-28T18:06:41.529 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:41.528+0000 7fd1aea00700 1 --2- 172.21.15.22:0/760777150 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd1b0102c50 0x7fd1b019a5c0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7fd1a000d980 tx=0x7fd1a000dd40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:06:41.529 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:41.528+0000 7fd1a7400700 1 -- 172.21.15.22:0/760777150 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1a0016070 con 0x7fd1b0102c50 2024-03-28T18:06:41.529 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:41.528+0000 7fd1a7400700 1 -- 172.21.15.22:0/760777150 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7fd1a0009b40 con 0x7fd1b0102c50 2024-03-28T18:06:41.530 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:41.528+0000 7fd1a7400700 1 -- 172.21.15.22:0/760777150 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1a000f3c0 con 0x7fd1b0102c50 2024-03-28T18:06:41.530 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:41.528+0000 7fd1b5a00700 1 -- 172.21.15.22:0/760777150 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd1b01a2510 con 0x7fd1b0102c50 2024-03-28T18:06:41.530 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:41.528+0000 7fd1b5a00700 1 -- 172.21.15.22:0/760777150 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd1b01a2a00 con 0x7fd1b0102c50 2024-03-28T18:06:41.530 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:41.530+0000 7fd1b5a00700 1 -- 172.21.15.22:0/760777150 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd1b0068e50 con 0x7fd1b0102c50 2024-03-28T18:06:41.534 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:41.533+0000 7fd1a7400700 1 -- 172.21.15.22:0/760777150 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 18) v1 ==== 137290+0+0 (secure 0 0 0) 0x7fd1a0019070 con 0x7fd1b0102c50 2024-03-28T18:06:41.535 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:41.535+0000 7fd1a7400700 1 --2- 172.21.15.22:0/760777150 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7fd1940a2910 0x7fd1940a4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:06:41.536 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:41.535+0000 7fd1a7400700 1 -- 172.21.15.22:0/760777150 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 8153+0+0 (secure 0 0 0) 0x7fd1a00ce600 con 0x7fd1b0102c50 2024-03-28T18:06:41.536 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:41.535+0000 7fd1ae000700 1 --2- 172.21.15.22:0/760777150 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7fd1940a2910 0x7fd1940a4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:06:41.537 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:41.536+0000 7fd1ae000700 1 --2- 172.21.15.22:0/760777150 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7fd1940a2910 0x7fd1940a4dd0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fd19c005950 tx=0x7fd19c0058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:06:41.539 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:41.539+0000 7fd1a7400700 1 -- 172.21.15.22:0/760777150 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fd1a009b6a0 con 0x7fd1b0102c50 2024-03-28T18:06:41.780 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:41.779+0000 7fd1b5a00700 1 -- 172.21.15.22:0/760777150 --> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] -- mgr_command(tid 0: {"prefix": "orch ls", "target": ["mon-mgr", ""]}) v1 -- 0x7fd1b00625d0 con 0x7fd1940a2910 2024-03-28T18:06:41.797 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:41.796+0000 7fd1a7400700 1 -- 172.21.15.22:0/760777150 <== mgr.14152 v2:172.21.15.22:6800/4201048392 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+980 (secure 0 0 0) 0x7fd1b00625d0 con 0x7fd1940a2910 2024-03-28T18:06:41.797 INFO:teuthology.orchestra.run.smithi022.stdout:NAME PORTS RUNNING REFRESHED AGE PLACEMENT 2024-03-28T18:06:41.797 INFO:teuthology.orchestra.run.smithi022.stdout:mds.all 12/12 46s ago 2m smithi022=a;smithi022=d;smithi022=g;smithi022=j;smithi079=b;smithi079=e;smithi079=h;smithi079=k;smithi134=c;smithi134=f;smithi134=i;smithi134=l;count:12 2024-03-28T18:06:41.797 INFO:teuthology.orchestra.run.smithi022.stdout:mgr 3/3 46s ago 9m smithi022=x;smithi079=y;smithi134=z;count:3 2024-03-28T18:06:41.797 INFO:teuthology.orchestra.run.smithi022.stdout:mon 3/3 46s ago 10m smithi022:172.21.15.22=a;smithi079:172.21.15.79=b;smithi134:172.21.15.134=c;count:3 2024-03-28T18:06:41.797 INFO:teuthology.orchestra.run.smithi022.stdout:osd 12 46s ago - 2024-03-28T18:06:41.800 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:41.799+0000 7fd1b5a00700 1 -- 172.21.15.22:0/760777150 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7fd1940a2910 msgr2=0x7fd1940a4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:06:41.800 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:41.799+0000 7fd1b5a00700 1 --2- 172.21.15.22:0/760777150 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7fd1940a2910 0x7fd1940a4dd0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fd19c005950 tx=0x7fd19c0058e0 comp rx=0 tx=0).stop 2024-03-28T18:06:41.800 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:41.799+0000 7fd1b5a00700 1 -- 172.21.15.22:0/760777150 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd1b0102c50 msgr2=0x7fd1b019a5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:06:41.800 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:41.799+0000 7fd1b5a00700 1 --2- 172.21.15.22:0/760777150 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd1b0102c50 0x7fd1b019a5c0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7fd1a000d980 tx=0x7fd1a000dd40 comp rx=0 tx=0).stop 2024-03-28T18:06:41.801 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:41.799+0000 7fd1b5a00700 1 -- 172.21.15.22:0/760777150 shutdown_connections 2024-03-28T18:06:41.801 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:41.799+0000 7fd1b5a00700 1 --2- 172.21.15.22:0/760777150 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd1b0107eb0 0x7fd1b01a1b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:06:41.801 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:41.799+0000 7fd1b5a00700 1 --2- 172.21.15.22:0/760777150 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd1b0102c50 0x7fd1b019a5c0 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:06:41.801 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:41.799+0000 7fd1b5a00700 1 --2- 172.21.15.22:0/760777150 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7fd1940a2910 0x7fd1940a4dd0 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:06:41.801 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:41.799+0000 7fd1b5a00700 1 --2- 172.21.15.22:0/760777150 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd1b0105580 0x7fd1b019ab00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:06:41.801 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:41.799+0000 7fd1b5a00700 1 -- 172.21.15.22:0/760777150 >> 172.21.15.22:0/760777150 conn(0x7fd1b00fc660 msgr2=0x7fd1b0102380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:06:41.801 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:41.800+0000 7fd1b5a00700 1 -- 172.21.15.22:0/760777150 shutdown_connections 2024-03-28T18:06:41.801 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:41.800+0000 7fd1b5a00700 1 -- 172.21.15.22:0/760777150 wait complete. 2024-03-28T18:06:42.424 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:06:42 smithi134 ceph-mon[35037]: pgmap v427: 97 pgs: 97 active+clean; 643 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-03-28T18:06:42.432 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:06:42 smithi079 ceph-mon[35312]: pgmap v427: 97 pgs: 97 active+clean; 643 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-03-28T18:06:42.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:06:42 smithi022 ceph-mon[31985]: pgmap v427: 97 pgs: 97 active+clean; 643 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-03-28T18:06:42.614 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fd220748-ed2b-11ee-95d0-87774f69a715 -- bash -c 'ceph orch host ls' 2024-03-28T18:06:43.080 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/fd220748-ed2b-11ee-95d0-87774f69a715/mon.a/config 2024-03-28T18:06:43.374 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:06:43 smithi022 ceph-mon[31985]: from='client.15168 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-03-28T18:06:43.424 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:06:43 smithi134 ceph-mon[35037]: from='client.15168 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-03-28T18:06:43.432 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:06:43 smithi079 ceph-mon[35312]: from='client.15168 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-03-28T18:06:44.424 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:06:44 smithi022 ceph-mon[31985]: pgmap v428: 97 pgs: 97 active+clean; 643 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-03-28T18:06:44.424 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:06:44 smithi134 ceph-mon[35037]: pgmap v428: 97 pgs: 97 active+clean; 643 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-03-28T18:06:44.432 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:06:44 smithi079 ceph-mon[35312]: pgmap v428: 97 pgs: 97 active+clean; 643 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-03-28T18:06:45.241 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:45.240+0000 7f7405a00700 1 -- 172.21.15.22:0/1302233726 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f74001021c0 msgr2=0x7f74001025e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:06:45.242 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:45.240+0000 7f7405a00700 1 --2- 172.21.15.22:0/1302233726 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f74001021c0 0x7f74001025e0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f73f00099c0 tx=0x7f73f0009cd0 comp rx=0 tx=0).stop 2024-03-28T18:06:45.242 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:45.240+0000 7f7405a00700 1 -- 172.21.15.22:0/1302233726 shutdown_connections 2024-03-28T18:06:45.242 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:45.240+0000 7f7405a00700 1 --2- 172.21.15.22:0/1302233726 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f7400103d60 0x7f740010a610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:06:45.242 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:45.240+0000 7f7405a00700 1 --2- 172.21.15.22:0/1302233726 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f74001021c0 0x7f74001025e0 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:06:45.242 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:45.240+0000 7f7405a00700 1 --2- 172.21.15.22:0/1302233726 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f74001033c0 0x7f7400103820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:06:45.242 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:45.240+0000 7f7405a00700 1 -- 172.21.15.22:0/1302233726 >> 172.21.15.22:0/1302233726 conn(0x7f74000fd7a0 msgr2=0x7f74000ffbc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:06:45.242 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:45.240+0000 7f7405a00700 1 -- 172.21.15.22:0/1302233726 shutdown_connections 2024-03-28T18:06:45.242 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:45.240+0000 7f7405a00700 1 -- 172.21.15.22:0/1302233726 wait complete. 2024-03-28T18:06:45.243 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:45.242+0000 7f7405a00700 1 Processor -- start 2024-03-28T18:06:45.243 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:45.242+0000 7f7405a00700 1 -- start start 2024-03-28T18:06:45.243 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:45.242+0000 7f7405a00700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f74001021c0 0x7f7400198440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:06:45.243 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:45.242+0000 7f7405a00700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f74001033c0 0x7f7400198980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:06:45.244 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:45.242+0000 7f7405a00700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7400103d60 0x7f740019fa00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:06:45.244 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:45.242+0000 7f7405a00700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f740010ddb0 con 0x7f74001033c0 2024-03-28T18:06:45.244 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:45.242+0000 7f7405a00700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f740010dc30 con 0x7f74001021c0 2024-03-28T18:06:45.244 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:45.243+0000 7f7405a00700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f740010df30 con 0x7f7400103d60 2024-03-28T18:06:45.244 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:45.243+0000 7f73fe000700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f74001033c0 0x7f7400198980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:06:45.244 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:45.243+0000 7f73fe000700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f74001033c0 0x7f7400198980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:54688/0 (socket says 172.21.15.22:54688) 2024-03-28T18:06:45.244 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:45.243+0000 7f73fe000700 1 -- 172.21.15.22:0/2592516130 learned_addr learned my addr 172.21.15.22:0/2592516130 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-28T18:06:45.244 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:45.243+0000 7f73fea00700 1 --2- 172.21.15.22:0/2592516130 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f74001021c0 0x7f7400198440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:06:45.245 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:45.243+0000 7f73ff400700 1 --2- 172.21.15.22:0/2592516130 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7400103d60 0x7f740019fa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:06:45.245 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:45.243+0000 7f73fe000700 1 -- 172.21.15.22:0/2592516130 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7400103d60 msgr2=0x7f740019fa00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:06:45.245 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:45.243+0000 7f73fe000700 1 --2- 172.21.15.22:0/2592516130 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7400103d60 0x7f740019fa00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:06:45.245 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:45.243+0000 7f73fe000700 1 -- 172.21.15.22:0/2592516130 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f74001021c0 msgr2=0x7f7400198440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:06:45.245 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:45.243+0000 7f73fe000700 1 --2- 172.21.15.22:0/2592516130 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f74001021c0 0x7f7400198440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:06:45.246 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:45.243+0000 7f73fe000700 1 -- 172.21.15.22:0/2592516130 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f74001a00a0 con 0x7f74001033c0 2024-03-28T18:06:45.246 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:45.244+0000 7f73fe000700 1 --2- 172.21.15.22:0/2592516130 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f74001033c0 0x7f7400198980 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f73ec00c950 tx=0x7f73ec00cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:06:45.246 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:45.244+0000 7f73f7400700 1 -- 172.21.15.22:0/2592516130 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73ec007ac0 con 0x7f74001033c0 2024-03-28T18:06:45.246 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:45.244+0000 7f73f7400700 1 -- 172.21.15.22:0/2592516130 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f73ec004d00 con 0x7f74001033c0 2024-03-28T18:06:45.246 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:45.244+0000 7f73f7400700 1 -- 172.21.15.22:0/2592516130 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73ec005680 con 0x7f74001033c0 2024-03-28T18:06:45.246 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:45.244+0000 7f7405a00700 1 -- 172.21.15.22:0/2592516130 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f74001a0390 con 0x7f74001033c0 2024-03-28T18:06:45.246 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:45.244+0000 7f7405a00700 1 -- 172.21.15.22:0/2592516130 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f74001a0920 con 0x7f74001033c0 2024-03-28T18:06:45.246 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:45.245+0000 7f7405a00700 1 -- 172.21.15.22:0/2592516130 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7400068e50 con 0x7f74001033c0 2024-03-28T18:06:45.248 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:45.247+0000 7f73f7400700 1 -- 172.21.15.22:0/2592516130 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 18) v1 ==== 137290+0+0 (secure 0 0 0) 0x7f73ec004750 con 0x7f74001033c0 2024-03-28T18:06:45.249 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:45.249+0000 7f73f7400700 1 --2- 172.21.15.22:0/2592516130 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7f73e40a2910 0x7f73e40a4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:06:45.250 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:45.249+0000 7f73f7400700 1 -- 172.21.15.22:0/2592516130 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 8153+0+0 (secure 0 0 0) 0x7f73ec0ce8b0 con 0x7f74001033c0 2024-03-28T18:06:45.250 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:45.249+0000 7f73fea00700 1 --2- 172.21.15.22:0/2592516130 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7f73e40a2910 0x7f73e40a4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:06:45.250 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:45.250+0000 7f73fea00700 1 --2- 172.21.15.22:0/2592516130 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7f73e40a2910 0x7f73e40a4dd0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f73f000b580 tx=0x7f73f0005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:06:45.254 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:45.254+0000 7f73f7400700 1 -- 172.21.15.22:0/2592516130 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f73ec014030 con 0x7f74001033c0 2024-03-28T18:06:45.489 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:45.488+0000 7f7405a00700 1 -- 172.21.15.22:0/2592516130 --> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""]}) v1 -- 0x7f74000625d0 con 0x7f73e40a2910 2024-03-28T18:06:45.492 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:45.492+0000 7f73f7400700 1 -- 172.21.15.22:0/2592516130 <== mgr.14152 v2:172.21.15.22:6800/4201048392 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+191 (secure 0 0 0) 0x7f74000625d0 con 0x7f73e40a2910 2024-03-28T18:06:45.493 INFO:teuthology.orchestra.run.smithi022.stdout:HOST ADDR LABELS STATUS 2024-03-28T18:06:45.493 INFO:teuthology.orchestra.run.smithi022.stdout:smithi022 172.21.15.22 2024-03-28T18:06:45.493 INFO:teuthology.orchestra.run.smithi022.stdout:smithi079 172.21.15.79 2024-03-28T18:06:45.493 INFO:teuthology.orchestra.run.smithi022.stdout:smithi134 172.21.15.134 2024-03-28T18:06:45.493 INFO:teuthology.orchestra.run.smithi022.stdout:3 hosts in cluster 2024-03-28T18:06:45.495 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:45.495+0000 7f7405a00700 1 -- 172.21.15.22:0/2592516130 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7f73e40a2910 msgr2=0x7f73e40a4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:06:45.495 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:45.495+0000 7f7405a00700 1 --2- 172.21.15.22:0/2592516130 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7f73e40a2910 0x7f73e40a4dd0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f73f000b580 tx=0x7f73f0005fb0 comp rx=0 tx=0).stop 2024-03-28T18:06:45.496 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:45.495+0000 7f7405a00700 1 -- 172.21.15.22:0/2592516130 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f74001033c0 msgr2=0x7f7400198980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:06:45.496 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:45.495+0000 7f7405a00700 1 --2- 172.21.15.22:0/2592516130 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f74001033c0 0x7f7400198980 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f73ec00c950 tx=0x7f73ec00cc60 comp rx=0 tx=0).stop 2024-03-28T18:06:45.496 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:45.495+0000 7f7405a00700 1 -- 172.21.15.22:0/2592516130 shutdown_connections 2024-03-28T18:06:45.496 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:45.495+0000 7f7405a00700 1 --2- 172.21.15.22:0/2592516130 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7f73e40a2910 0x7f73e40a4dd0 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:06:45.496 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:45.495+0000 7f7405a00700 1 --2- 172.21.15.22:0/2592516130 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7400103d60 0x7f740019fa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:06:45.496 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:45.495+0000 7f7405a00700 1 --2- 172.21.15.22:0/2592516130 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f74001021c0 0x7f7400198440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:06:45.496 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:45.495+0000 7f7405a00700 1 --2- 172.21.15.22:0/2592516130 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f74001033c0 0x7f7400198980 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:06:45.497 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:45.495+0000 7f7405a00700 1 -- 172.21.15.22:0/2592516130 >> 172.21.15.22:0/2592516130 conn(0x7f74000fd7a0 msgr2=0x7f7400106730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:06:45.497 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:45.495+0000 7f7405a00700 1 -- 172.21.15.22:0/2592516130 shutdown_connections 2024-03-28T18:06:45.497 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:45.496+0000 7f7405a00700 1 -- 172.21.15.22:0/2592516130 wait complete. 2024-03-28T18:06:46.232 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:06:46 smithi022 ceph-mon[31985]: pgmap v429: 97 pgs: 97 active+clean; 643 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-28T18:06:46.232 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:06:46 smithi022 ceph-mon[31985]: from='client.15174 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-03-28T18:06:46.287 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fd220748-ed2b-11ee-95d0-87774f69a715 -- bash -c 'ceph orch device ls' 2024-03-28T18:06:46.425 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:06:46 smithi134 ceph-mon[35037]: pgmap v429: 97 pgs: 97 active+clean; 643 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-28T18:06:46.425 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:06:46 smithi134 ceph-mon[35037]: from='client.15174 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-03-28T18:06:46.432 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:06:46 smithi079 ceph-mon[35312]: pgmap v429: 97 pgs: 97 active+clean; 643 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-28T18:06:46.432 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:06:46 smithi079 ceph-mon[35312]: from='client.15174 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-03-28T18:06:46.756 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/fd220748-ed2b-11ee-95d0-87774f69a715/mon.a/config 2024-03-28T18:06:48.424 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:06:48 smithi134 ceph-mon[35037]: pgmap v430: 97 pgs: 97 active+clean; 643 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2024-03-28T18:06:48.432 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:06:48 smithi079 ceph-mon[35312]: pgmap v430: 97 pgs: 97 active+clean; 643 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2024-03-28T18:06:48.509 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:06:48 smithi022 ceph-mon[31985]: pgmap v430: 97 pgs: 97 active+clean; 643 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2024-03-28T18:06:49.063 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:49.061+0000 7fc128c00700 1 -- 172.21.15.22:0/3277693823 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc124078750 msgr2=0x7fc124078fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:06:49.063 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:49.061+0000 7fc128c00700 1 --2- 172.21.15.22:0/3277693823 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc124078750 0x7fc124078fd0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7fc11c00b0a0 tx=0x7fc11c00b3b0 comp rx=0 tx=0).stop 2024-03-28T18:06:49.063 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:49.061+0000 7fc128c00700 1 -- 172.21.15.22:0/3277693823 shutdown_connections 2024-03-28T18:06:49.063 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:49.061+0000 7fc128c00700 1 --2- 172.21.15.22:0/3277693823 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc124078750 0x7fc124078fd0 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:06:49.063 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:49.062+0000 7fc128c00700 1 --2- 172.21.15.22:0/3277693823 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc124076b40 0x7fc124076f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:06:49.063 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:49.062+0000 7fc128c00700 1 --2- 172.21.15.22:0/3277693823 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc124077d90 0x7fc124078210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:06:49.063 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:49.062+0000 7fc128c00700 1 -- 172.21.15.22:0/3277693823 >> 172.21.15.22:0/3277693823 conn(0x7fc1240ff9b0 msgr2=0x7fc124101df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:06:49.064 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:49.062+0000 7fc128c00700 1 -- 172.21.15.22:0/3277693823 shutdown_connections 2024-03-28T18:06:49.064 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:49.062+0000 7fc128c00700 1 -- 172.21.15.22:0/3277693823 wait complete. 2024-03-28T18:06:49.066 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:49.063+0000 7fc128c00700 1 Processor -- start 2024-03-28T18:06:49.066 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:49.063+0000 7fc128c00700 1 -- start start 2024-03-28T18:06:49.066 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:49.063+0000 7fc128c00700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc124076b40 0x7fc12419e980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:06:49.066 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:49.063+0000 7fc128c00700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc124077d90 0x7fc12419eec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:06:49.066 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:49.063+0000 7fc128c00700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc124078750 0x7fc1241a5ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:06:49.066 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:49.063+0000 7fc128c00700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc124114410 con 0x7fc124076b40 2024-03-28T18:06:49.067 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:49.063+0000 7fc128c00700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc124114290 con 0x7fc124077d90 2024-03-28T18:06:49.067 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:49.063+0000 7fc128c00700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc124114590 con 0x7fc124078750 2024-03-28T18:06:49.067 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:49.064+0000 7fc121e00700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc124076b40 0x7fc12419e980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:06:49.067 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:49.064+0000 7fc121e00700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc124076b40 0x7fc12419e980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:40056/0 (socket says 172.21.15.22:40056) 2024-03-28T18:06:49.067 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:49.064+0000 7fc121e00700 1 -- 172.21.15.22:0/1874595742 learned_addr learned my addr 172.21.15.22:0/1874595742 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-28T18:06:49.067 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:49.064+0000 7fc121400700 1 --2- 172.21.15.22:0/1874595742 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc124077d90 0x7fc12419eec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:06:49.067 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:49.064+0000 7fc122800700 1 --2- 172.21.15.22:0/1874595742 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc124078750 0x7fc1241a5ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:06:49.067 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:49.064+0000 7fc121e00700 1 -- 172.21.15.22:0/1874595742 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc124078750 msgr2=0x7fc1241a5ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:06:49.067 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:49.064+0000 7fc121e00700 1 --2- 172.21.15.22:0/1874595742 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc124078750 0x7fc1241a5ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:06:49.068 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:49.064+0000 7fc121e00700 1 -- 172.21.15.22:0/1874595742 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc124077d90 msgr2=0x7fc12419eec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:06:49.068 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:49.064+0000 7fc121e00700 1 --2- 172.21.15.22:0/1874595742 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc124077d90 0x7fc12419eec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:06:49.068 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:49.064+0000 7fc121e00700 1 -- 172.21.15.22:0/1874595742 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc1241a6590 con 0x7fc124076b40 2024-03-28T18:06:49.068 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:49.064+0000 7fc121400700 1 --2- 172.21.15.22:0/1874595742 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc124077d90 0x7fc12419eec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-28T18:06:49.068 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:49.064+0000 7fc121e00700 1 --2- 172.21.15.22:0/1874595742 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc124076b40 0x7fc12419e980 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7fc11400e9d0 tx=0x7fc11400ece0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:06:49.068 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:49.065+0000 7fc11aa00700 1 -- 172.21.15.22:0/1874595742 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc11400cbd0 con 0x7fc124076b40 2024-03-28T18:06:49.068 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:49.065+0000 7fc11aa00700 1 -- 172.21.15.22:0/1874595742 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7fc11400ee70 con 0x7fc124076b40 2024-03-28T18:06:49.068 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:49.065+0000 7fc128c00700 1 -- 172.21.15.22:0/1874595742 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc1241a6880 con 0x7fc124076b40 2024-03-28T18:06:49.068 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:49.065+0000 7fc11aa00700 1 -- 172.21.15.22:0/1874595742 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc114018680 con 0x7fc124076b40 2024-03-28T18:06:49.068 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:49.065+0000 7fc128c00700 1 -- 172.21.15.22:0/1874595742 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc1241a6dc0 con 0x7fc124076b40 2024-03-28T18:06:49.069 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:49.066+0000 7fc128c00700 1 -- 172.21.15.22:0/1874595742 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc124068e50 con 0x7fc124076b40 2024-03-28T18:06:49.069 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:49.068+0000 7fc11aa00700 1 -- 172.21.15.22:0/1874595742 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 18) v1 ==== 137290+0+0 (secure 0 0 0) 0x7fc114010430 con 0x7fc124076b40 2024-03-28T18:06:49.070 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:49.070+0000 7fc11aa00700 1 --2- 172.21.15.22:0/1874595742 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7fc1080a2890 0x7fc1080a4d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:06:49.071 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:49.070+0000 7fc11aa00700 1 -- 172.21.15.22:0/1874595742 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 8153+0+0 (secure 0 0 0) 0x7fc1140cec80 con 0x7fc124076b40 2024-03-28T18:06:49.071 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:49.070+0000 7fc121400700 1 --2- 172.21.15.22:0/1874595742 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7fc1080a2890 0x7fc1080a4d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:06:49.071 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:49.071+0000 7fc121400700 1 --2- 172.21.15.22:0/1874595742 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7fc1080a2890 0x7fc1080a4d50 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fc110005950 tx=0x7fc11000b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:06:49.075 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:49.074+0000 7fc11aa00700 1 -- 172.21.15.22:0/1874595742 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fc11409bd20 con 0x7fc124076b40 2024-03-28T18:06:49.316 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:49.315+0000 7fc128c00700 1 -- 172.21.15.22:0/1874595742 --> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] -- mgr_command(tid 0: {"prefix": "orch device ls", "target": ["mon-mgr", ""]}) v1 -- 0x7fc1240625d0 con 0x7fc1080a2890 2024-03-28T18:06:49.320 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:49.319+0000 7fc11aa00700 1 -- 172.21.15.22:0/1874595742 <== mgr.14152 v2:172.21.15.22:6800/4201048392 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+532 (secure 0 0 0) 0x7fc1240625d0 con 0x7fc1080a2890 2024-03-28T18:06:49.320 INFO:teuthology.orchestra.run.smithi022.stdout:HOST PATH TYPE DEVICE ID SIZE AVAILABLE REFRESHED REJECT REASONS 2024-03-28T18:06:49.321 INFO:teuthology.orchestra.run.smithi022.stdout:smithi022 /dev/nvme0n1 ssd INTEL_SSDPED1D480GA_PHMB747402EC480DGN 447G No 4m ago Has a FileSystem, LVM detected 2024-03-28T18:06:49.321 INFO:teuthology.orchestra.run.smithi022.stdout:smithi079 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_PHFT6204013G400BGN 372G No 3m ago Has a FileSystem, LVM detected 2024-03-28T18:06:49.321 INFO:teuthology.orchestra.run.smithi022.stdout:smithi134 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_CVFT62420002400BGN 372G No 3m ago Has a FileSystem, LVM detected 2024-03-28T18:06:49.324 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:49.322+0000 7fc128c00700 1 -- 172.21.15.22:0/1874595742 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7fc1080a2890 msgr2=0x7fc1080a4d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:06:49.324 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:49.323+0000 7fc128c00700 1 --2- 172.21.15.22:0/1874595742 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7fc1080a2890 0x7fc1080a4d50 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fc110005950 tx=0x7fc11000b410 comp rx=0 tx=0).stop 2024-03-28T18:06:49.324 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:49.323+0000 7fc128c00700 1 -- 172.21.15.22:0/1874595742 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc124076b40 msgr2=0x7fc12419e980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:06:49.324 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:49.323+0000 7fc128c00700 1 --2- 172.21.15.22:0/1874595742 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc124076b40 0x7fc12419e980 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7fc11400e9d0 tx=0x7fc11400ece0 comp rx=0 tx=0).stop 2024-03-28T18:06:49.324 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:49.323+0000 7fc128c00700 1 -- 172.21.15.22:0/1874595742 shutdown_connections 2024-03-28T18:06:49.324 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:49.323+0000 7fc128c00700 1 --2- 172.21.15.22:0/1874595742 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7fc1080a2890 0x7fc1080a4d50 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:06:49.324 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:49.323+0000 7fc128c00700 1 --2- 172.21.15.22:0/1874595742 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc124078750 0x7fc1241a5ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:06:49.324 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:49.323+0000 7fc128c00700 1 --2- 172.21.15.22:0/1874595742 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc124076b40 0x7fc12419e980 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:06:49.324 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:49.323+0000 7fc128c00700 1 --2- 172.21.15.22:0/1874595742 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc124077d90 0x7fc12419eec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:06:49.324 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:49.323+0000 7fc128c00700 1 -- 172.21.15.22:0/1874595742 >> 172.21.15.22:0/1874595742 conn(0x7fc1240ff9b0 msgr2=0x7fc1241014b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:06:49.324 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:49.323+0000 7fc128c00700 1 -- 172.21.15.22:0/1874595742 shutdown_connections 2024-03-28T18:06:49.325 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:49.323+0000 7fc128c00700 1 -- 172.21.15.22:0/1874595742 wait complete. 2024-03-28T18:06:50.187 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-03-28T18:06:50.195 INFO:tasks.cephadm:Running commands on role mon.a host ubuntu@smithi022.front.sepia.ceph.com 2024-03-28T18:06:50.196 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fd220748-ed2b-11ee-95d0-87774f69a715 -- bash -c 'ceph fs dump' 2024-03-28T18:06:50.424 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:06:50 smithi134 ceph-mon[35037]: pgmap v431: 97 pgs: 97 active+clean; 643 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-03-28T18:06:50.425 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:06:50 smithi134 ceph-mon[35037]: from='client.15180 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-03-28T18:06:50.432 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:06:50 smithi079 ceph-mon[35312]: pgmap v431: 97 pgs: 97 active+clean; 643 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-03-28T18:06:50.432 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:06:50 smithi079 ceph-mon[35312]: from='client.15180 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-03-28T18:06:50.531 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:06:50 smithi022 ceph-mon[31985]: pgmap v431: 97 pgs: 97 active+clean; 643 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-03-28T18:06:50.531 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:06:50 smithi022 ceph-mon[31985]: from='client.15180 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-03-28T18:06:50.667 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/fd220748-ed2b-11ee-95d0-87774f69a715/mon.a/config 2024-03-28T18:06:52.424 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:06:52 smithi134 ceph-mon[35037]: pgmap v432: 97 pgs: 97 active+clean; 643 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-03-28T18:06:52.432 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:06:52 smithi079 ceph-mon[35312]: pgmap v432: 97 pgs: 97 active+clean; 643 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-03-28T18:06:52.465 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:06:52 smithi022 ceph-mon[31985]: pgmap v432: 97 pgs: 97 active+clean; 643 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-03-28T18:06:52.643 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:52.643+0000 7fc15f200700 1 -- 172.21.15.22:0/2272941569 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc158105eb0 msgr2=0x7fc15810c760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:06:52.643 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:52.643+0000 7fc15f200700 1 --2- 172.21.15.22:0/2272941569 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc158105eb0 0x7fc15810c760 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7fc15400b0d0 tx=0x7fc15400b3e0 comp rx=0 tx=0).stop 2024-03-28T18:06:52.644 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:52.643+0000 7fc15f200700 1 -- 172.21.15.22:0/2272941569 shutdown_connections 2024-03-28T18:06:52.644 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:52.643+0000 7fc15f200700 1 --2- 172.21.15.22:0/2272941569 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc158105eb0 0x7fc15810c760 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:06:52.644 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:52.643+0000 7fc15f200700 1 --2- 172.21.15.22:0/2272941569 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc158104310 0x7fc158104730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:06:52.644 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:52.643+0000 7fc15f200700 1 --2- 172.21.15.22:0/2272941569 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc158105510 0x7fc158105970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:06:52.644 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:52.643+0000 7fc15f200700 1 -- 172.21.15.22:0/2272941569 >> 172.21.15.22:0/2272941569 conn(0x7fc1580ff8b0 msgr2=0x7fc158101cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:06:52.644 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:52.643+0000 7fc15f200700 1 -- 172.21.15.22:0/2272941569 shutdown_connections 2024-03-28T18:06:52.644 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:52.643+0000 7fc15f200700 1 -- 172.21.15.22:0/2272941569 wait complete. 2024-03-28T18:06:52.645 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:52.644+0000 7fc15f200700 1 Processor -- start 2024-03-28T18:06:52.645 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:52.645+0000 7fc15f200700 1 -- start start 2024-03-28T18:06:52.645 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:52.645+0000 7fc15f200700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc158104310 0x7fc15819a480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:06:52.645 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:52.645+0000 7fc15f200700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc158105510 0x7fc15819a9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:06:52.645 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:52.645+0000 7fc15f200700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc158105eb0 0x7fc1581a1a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:06:52.645 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:52.645+0000 7fc15f200700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc15810ff90 con 0x7fc158105eb0 2024-03-28T18:06:52.645 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:52.645+0000 7fc15f200700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc15810fe10 con 0x7fc158105510 2024-03-28T18:06:52.645 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:52.645+0000 7fc15f200700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc158110110 con 0x7fc158104310 2024-03-28T18:06:52.646 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:52.646+0000 7fc15d400700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc158105510 0x7fc15819a9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:06:52.646 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:52.646+0000 7fc15d400700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc158105510 0x7fc15819a9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.22:53266/0 (socket says 172.21.15.22:53266) 2024-03-28T18:06:52.646 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:52.646+0000 7fc15e800700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc158105eb0 0x7fc1581a1a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:06:52.646 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:52.646+0000 7fc15d400700 1 -- 172.21.15.22:0/684634941 learned_addr learned my addr 172.21.15.22:0/684634941 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-28T18:06:52.647 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:52.646+0000 7fc15de00700 1 --2- 172.21.15.22:0/684634941 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc158104310 0x7fc15819a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:06:52.647 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:52.646+0000 7fc15e800700 1 -- 172.21.15.22:0/684634941 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc158104310 msgr2=0x7fc15819a480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:06:52.647 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:52.646+0000 7fc15e800700 1 --2- 172.21.15.22:0/684634941 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc158104310 0x7fc15819a480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:06:52.647 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:52.646+0000 7fc15e800700 1 -- 172.21.15.22:0/684634941 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc158105510 msgr2=0x7fc15819a9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:06:52.647 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:52.646+0000 7fc15e800700 1 --2- 172.21.15.22:0/684634941 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc158105510 0x7fc15819a9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:06:52.647 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:52.646+0000 7fc15e800700 1 -- 172.21.15.22:0/684634941 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc1581a20e0 con 0x7fc158105eb0 2024-03-28T18:06:52.647 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:52.647+0000 7fc15e800700 1 --2- 172.21.15.22:0/684634941 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc158105eb0 0x7fc1581a1a40 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7fc154009fd0 tx=0x7fc154008f60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:06:52.647 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:52.647+0000 7fc152a00700 1 -- 172.21.15.22:0/684634941 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc154021020 con 0x7fc158105eb0 2024-03-28T18:06:52.648 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:52.647+0000 7fc152a00700 1 -- 172.21.15.22:0/684634941 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7fc154007b80 con 0x7fc158105eb0 2024-03-28T18:06:52.648 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:52.647+0000 7fc15f200700 1 -- 172.21.15.22:0/684634941 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc1581a2370 con 0x7fc158105eb0 2024-03-28T18:06:52.648 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:52.647+0000 7fc152a00700 1 -- 172.21.15.22:0/684634941 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc15401b720 con 0x7fc158105eb0 2024-03-28T18:06:52.648 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:52.647+0000 7fc15f200700 1 -- 172.21.15.22:0/684634941 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc1581a27c0 con 0x7fc158105eb0 2024-03-28T18:06:52.648 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:52.649+0000 7fc15f200700 1 -- 172.21.15.22:0/684634941 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc158068e50 con 0x7fc158105eb0 2024-03-28T18:06:52.650 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:52.651+0000 7fc152a00700 1 -- 172.21.15.22:0/684634941 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 18) v1 ==== 137290+0+0 (secure 0 0 0) 0x7fc154024020 con 0x7fc158105eb0 2024-03-28T18:06:52.652 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:52.652+0000 7fc152a00700 1 --2- 172.21.15.22:0/684634941 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7fc1400a2890 0x7fc1400a4d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:06:52.652 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:52.652+0000 7fc152a00700 1 -- 172.21.15.22:0/684634941 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 8153+0+0 (secure 0 0 0) 0x7fc15400f020 con 0x7fc158105eb0 2024-03-28T18:06:52.652 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:52.652+0000 7fc15de00700 1 --2- 172.21.15.22:0/684634941 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7fc1400a2890 0x7fc1400a4d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:06:52.652 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:52.653+0000 7fc15de00700 1 --2- 172.21.15.22:0/684634941 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7fc1400a2890 0x7fc1400a4d50 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fc158105370 tx=0x7fc14c00b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:06:52.656 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:52.656+0000 7fc152a00700 1 -- 172.21.15.22:0/684634941 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fc15409c990 con 0x7fc158105eb0 2024-03-28T18:06:52.935 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:52.935+0000 7fc15f200700 1 -- 172.21.15.22:0/684634941 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7fc158062da0 con 0x7fc158105eb0 2024-03-28T18:06:52.935 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:52.935+0000 7fc152a00700 1 -- 172.21.15.22:0/684634941 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 26 v26) v1 ==== 76+0+3032 (secure 0 0 0) 0x7fc15409c290 con 0x7fc158105eb0 2024-03-28T18:06:52.937 INFO:teuthology.orchestra.run.smithi022.stdout:e26 2024-03-28T18:06:52.937 INFO:teuthology.orchestra.run.smithi022.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-03-28T18:06:52.937 INFO:teuthology.orchestra.run.smithi022.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments} 2024-03-28T18:06:52.937 INFO:teuthology.orchestra.run.smithi022.stdout:legacy client fscid: 1 2024-03-28T18:06:52.937 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-03-28T18:06:52.937 INFO:teuthology.orchestra.run.smithi022.stdout:Filesystem 'cephfs' (1) 2024-03-28T18:06:52.937 INFO:teuthology.orchestra.run.smithi022.stdout:fs_name cephfs 2024-03-28T18:06:52.937 INFO:teuthology.orchestra.run.smithi022.stdout:epoch 26 2024-03-28T18:06:52.937 INFO:teuthology.orchestra.run.smithi022.stdout:flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2024-03-28T18:06:52.937 INFO:teuthology.orchestra.run.smithi022.stdout:created 2024-03-28T18:04:48.991622+0000 2024-03-28T18:06:52.937 INFO:teuthology.orchestra.run.smithi022.stdout:modified 2024-03-28T18:04:56.643864+0000 2024-03-28T18:06:52.938 INFO:teuthology.orchestra.run.smithi022.stdout:tableserver 0 2024-03-28T18:06:52.938 INFO:teuthology.orchestra.run.smithi022.stdout:root 0 2024-03-28T18:06:52.938 INFO:teuthology.orchestra.run.smithi022.stdout:session_timeout 300 2024-03-28T18:06:52.938 INFO:teuthology.orchestra.run.smithi022.stdout:session_autoclose 300 2024-03-28T18:06:52.938 INFO:teuthology.orchestra.run.smithi022.stdout:max_file_size 1099511627776 2024-03-28T18:06:52.938 INFO:teuthology.orchestra.run.smithi022.stdout:max_xattr_size 65536 2024-03-28T18:06:52.938 INFO:teuthology.orchestra.run.smithi022.stdout:required_client_features {} 2024-03-28T18:06:52.938 INFO:teuthology.orchestra.run.smithi022.stdout:last_failure 0 2024-03-28T18:06:52.938 INFO:teuthology.orchestra.run.smithi022.stdout:last_failure_osd_epoch 0 2024-03-28T18:06:52.938 INFO:teuthology.orchestra.run.smithi022.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments} 2024-03-28T18:06:52.938 INFO:teuthology.orchestra.run.smithi022.stdout:max_mds 5 2024-03-28T18:06:52.938 INFO:teuthology.orchestra.run.smithi022.stdout:in 0,1,2,3,4 2024-03-28T18:06:52.938 INFO:teuthology.orchestra.run.smithi022.stdout:up {0=34553,1=34511,2=34475,3=34442,4=24502} 2024-03-28T18:06:52.938 INFO:teuthology.orchestra.run.smithi022.stdout:failed 2024-03-28T18:06:52.939 INFO:teuthology.orchestra.run.smithi022.stdout:damaged 2024-03-28T18:06:52.939 INFO:teuthology.orchestra.run.smithi022.stdout:stopped 2024-03-28T18:06:52.939 INFO:teuthology.orchestra.run.smithi022.stdout:data_pools [3] 2024-03-28T18:06:52.939 INFO:teuthology.orchestra.run.smithi022.stdout:metadata_pool 2 2024-03-28T18:06:52.939 INFO:teuthology.orchestra.run.smithi022.stdout:inline_data disabled 2024-03-28T18:06:52.939 INFO:teuthology.orchestra.run.smithi022.stdout:balancer 2024-03-28T18:06:52.939 INFO:teuthology.orchestra.run.smithi022.stdout:bal_rank_mask -1 2024-03-28T18:06:52.939 INFO:teuthology.orchestra.run.smithi022.stdout:standby_count_wanted 0 2024-03-28T18:06:52.939 INFO:teuthology.orchestra.run.smithi022.stdout:qdb_cluster leader: 34553 members: 34511,34553,34475,34442,24502 2024-03-28T18:06:52.939 INFO:teuthology.orchestra.run.smithi022.stdout:[mds.b{0:34553} state up:active seq 3 addr [v2:172.21.15.79:6838/2441542443,v1:172.21.15.79:6839/2441542443] compat {c=[1],r=[1],i=[fff]}] 2024-03-28T18:06:52.939 INFO:teuthology.orchestra.run.smithi022.stdout:[mds.c{0:24478} state up:standby-replay seq 1 addr [v2:172.21.15.134:6836/899825581,v1:172.21.15.134:6837/899825581] compat {c=[1],r=[1],i=[fff]}] 2024-03-28T18:06:52.939 INFO:teuthology.orchestra.run.smithi022.stdout:[mds.h{1:34511} state up:active seq 8 addr [v2:172.21.15.79:6836/1787903366,v1:172.21.15.79:6837/1787903366] compat {c=[1],r=[1],i=[fff]}] 2024-03-28T18:06:52.939 INFO:teuthology.orchestra.run.smithi022.stdout:[mds.f{1:24463} state up:standby-replay seq 1 addr [v2:172.21.15.134:6834/3370549207,v1:172.21.15.134:6835/3370549207] compat {c=[1],r=[1],i=[fff]}] 2024-03-28T18:06:52.939 INFO:teuthology.orchestra.run.smithi022.stdout:[mds.e{2:34475} state up:active seq 11 addr [v2:172.21.15.79:6834/3365773431,v1:172.21.15.79:6835/3365773431] compat {c=[1],r=[1],i=[fff]}] 2024-03-28T18:06:52.939 INFO:teuthology.orchestra.run.smithi022.stdout:[mds.l{2:24412} state up:standby-replay seq 1 addr [v2:172.21.15.134:6832/3176125362,v1:172.21.15.134:6833/3176125362] compat {c=[1],r=[1],i=[fff]}] 2024-03-28T18:06:52.940 INFO:teuthology.orchestra.run.smithi022.stdout:[mds.k{3:34442} state up:active seq 14 addr [v2:172.21.15.79:6832/856896997,v1:172.21.15.79:6833/856896997] compat {c=[1],r=[1],i=[fff]}] 2024-03-28T18:06:52.940 INFO:teuthology.orchestra.run.smithi022.stdout:[mds.a{3:14655} state up:standby-replay seq 1 addr [v2:172.21.15.22:6840/3069056661,v1:172.21.15.22:6841/3069056661] compat {c=[1],r=[1],i=[fff]}] 2024-03-28T18:06:52.940 INFO:teuthology.orchestra.run.smithi022.stdout:[mds.i{4:24502} state up:active seq 7 addr [v2:172.21.15.134:6838/3400263217,v1:172.21.15.134:6839/3400263217] compat {c=[1],r=[1],i=[fff]}] 2024-03-28T18:06:52.940 INFO:teuthology.orchestra.run.smithi022.stdout:[mds.g{4:14640} state up:standby-replay seq 1 addr [v2:172.21.15.22:6838/3979998310,v1:172.21.15.22:6839/3979998310] compat {c=[1],r=[1],i=[fff]}] 2024-03-28T18:06:52.940 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-03-28T18:06:52.940 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-03-28T18:06:52.940 INFO:teuthology.orchestra.run.smithi022.stdout:Standby daemons: 2024-03-28T18:06:52.940 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-03-28T18:06:52.940 INFO:teuthology.orchestra.run.smithi022.stdout:[mds.j{-1:14544} state up:standby seq 1 addr [v2:172.21.15.22:6834/3718238950,v1:172.21.15.22:6835/3718238950] compat {c=[1],r=[1],i=[fff]}] 2024-03-28T18:06:52.940 INFO:teuthology.orchestra.run.smithi022.stdout:[mds.d{-1:14562} state up:standby seq 1 addr [v2:172.21.15.22:6836/3676939386,v1:172.21.15.22:6837/3676939386] compat {c=[1],r=[1],i=[fff]}] 2024-03-28T18:06:52.941 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:52.938+0000 7fc15f200700 1 -- 172.21.15.22:0/684634941 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7fc1400a2890 msgr2=0x7fc1400a4d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:06:52.941 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:52.938+0000 7fc15f200700 1 --2- 172.21.15.22:0/684634941 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7fc1400a2890 0x7fc1400a4d50 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fc158105370 tx=0x7fc14c00b410 comp rx=0 tx=0).stop 2024-03-28T18:06:52.941 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:52.938+0000 7fc15f200700 1 -- 172.21.15.22:0/684634941 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc158105eb0 msgr2=0x7fc1581a1a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:06:52.941 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:52.938+0000 7fc15f200700 1 --2- 172.21.15.22:0/684634941 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc158105eb0 0x7fc1581a1a40 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7fc154009fd0 tx=0x7fc154008f60 comp rx=0 tx=0).stop 2024-03-28T18:06:52.941 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:52.939+0000 7fc15f200700 1 -- 172.21.15.22:0/684634941 shutdown_connections 2024-03-28T18:06:52.941 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:52.939+0000 7fc15f200700 1 --2- 172.21.15.22:0/684634941 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc158105eb0 0x7fc1581a1a40 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:06:52.941 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:52.939+0000 7fc15f200700 1 --2- 172.21.15.22:0/684634941 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7fc1400a2890 0x7fc1400a4d50 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:06:52.941 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:52.939+0000 7fc15f200700 1 --2- 172.21.15.22:0/684634941 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc158104310 0x7fc15819a480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:06:52.942 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:52.939+0000 7fc15f200700 1 --2- 172.21.15.22:0/684634941 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc158105510 0x7fc15819a9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:06:52.942 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:52.939+0000 7fc15f200700 1 -- 172.21.15.22:0/684634941 >> 172.21.15.22:0/684634941 conn(0x7fc1580ff8b0 msgr2=0x7fc158108740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:06:52.942 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:52.939+0000 7fc15f200700 1 -- 172.21.15.22:0/684634941 shutdown_connections 2024-03-28T18:06:52.942 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:52.939+0000 7fc15f200700 1 -- 172.21.15.22:0/684634941 wait complete. 2024-03-28T18:06:52.942 INFO:teuthology.orchestra.run.smithi022.stderr:dumped fsmap epoch 26 2024-03-28T18:06:53.281 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:06:53 smithi022 ceph-mon[31985]: from='client.? 172.21.15.22:0/684634941' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-28T18:06:53.424 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:06:53 smithi134 ceph-mon[35037]: from='client.? 172.21.15.22:0/684634941' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-28T18:06:53.432 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:06:53 smithi079 ceph-mon[35312]: from='client.? 172.21.15.22:0/684634941' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-28T18:06:53.797 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fd220748-ed2b-11ee-95d0-87774f69a715 -- bash -c 'ceph osd dump' 2024-03-28T18:06:54.284 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/fd220748-ed2b-11ee-95d0-87774f69a715/mon.a/config 2024-03-28T18:06:54.424 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:06:54 smithi134 ceph-mon[35037]: pgmap v433: 97 pgs: 97 active+clean; 643 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-03-28T18:06:54.432 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:06:54 smithi079 ceph-mon[35312]: pgmap v433: 97 pgs: 97 active+clean; 643 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-03-28T18:06:54.436 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:06:54 smithi022 ceph-mon[31985]: pgmap v433: 97 pgs: 97 active+clean; 643 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-03-28T18:06:56.282 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:06:56 smithi022 ceph-mon[31985]: pgmap v434: 97 pgs: 97 active+clean; 643 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-28T18:06:56.424 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:06:56 smithi134 ceph-mon[35037]: pgmap v434: 97 pgs: 97 active+clean; 643 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-28T18:06:56.432 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:06:56 smithi079 ceph-mon[35312]: pgmap v434: 97 pgs: 97 active+clean; 643 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-28T18:06:56.541 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:56.540+0000 7f7097800700 1 -- 172.21.15.22:0/2589054430 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7090109a10 msgr2=0x7f7090109e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:06:56.541 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:56.540+0000 7f7097800700 1 --2- 172.21.15.22:0/2589054430 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7090109a10 0x7f7090109e70 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f707c0099c0 tx=0x7f707c009cd0 comp rx=0 tx=0).stop 2024-03-28T18:06:56.541 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:56.540+0000 7f7097800700 1 -- 172.21.15.22:0/2589054430 shutdown_connections 2024-03-28T18:06:56.541 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:56.540+0000 7f7097800700 1 --2- 172.21.15.22:0/2589054430 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f709010a3b0 0x7f7090110c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:06:56.542 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:56.540+0000 7f7097800700 1 --2- 172.21.15.22:0/2589054430 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7090108810 0x7f7090108c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:06:56.542 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:56.540+0000 7f7097800700 1 --2- 172.21.15.22:0/2589054430 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7090109a10 0x7f7090109e70 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:06:56.542 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:56.540+0000 7f7097800700 1 -- 172.21.15.22:0/2589054430 >> 172.21.15.22:0/2589054430 conn(0x7f70900777e0 msgr2=0x7f7090079c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:06:56.542 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:56.540+0000 7f7097800700 1 -- 172.21.15.22:0/2589054430 shutdown_connections 2024-03-28T18:06:56.542 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:56.540+0000 7f7097800700 1 -- 172.21.15.22:0/2589054430 wait complete. 2024-03-28T18:06:56.542 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:56.542+0000 7f7097800700 1 Processor -- start 2024-03-28T18:06:56.542 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:56.542+0000 7f7097800700 1 -- start start 2024-03-28T18:06:56.544 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:56.542+0000 7f7097800700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7090108810 0x7f709019e9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:06:56.544 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:56.542+0000 7f7097800700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f7090109a10 0x7f709019ef20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:06:56.544 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:56.542+0000 7f7097800700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f709010a3b0 0x7f70901a5fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:06:56.544 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:56.542+0000 7f7097800700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7090114430 con 0x7f7090108810 2024-03-28T18:06:56.544 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:56.542+0000 7f7097800700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70901142b0 con 0x7f7090109a10 2024-03-28T18:06:56.544 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:56.542+0000 7f7097800700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70901145b0 con 0x7f709010a3b0 2024-03-28T18:06:56.544 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:56.542+0000 7f7094e00700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7090108810 0x7f709019e9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:06:56.544 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:56.543+0000 7f7094e00700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7090108810 0x7f709019e9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:40086/0 (socket says 172.21.15.22:40086) 2024-03-28T18:06:56.544 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:56.543+0000 7f7094e00700 1 -- 172.21.15.22:0/3063489089 learned_addr learned my addr 172.21.15.22:0/3063489089 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-28T18:06:56.544 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:56.543+0000 7f708fe00700 1 --2- 172.21.15.22:0/3063489089 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f7090109a10 0x7f709019ef20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:06:56.544 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:56.543+0000 7f7095800700 1 --2- 172.21.15.22:0/3063489089 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f709010a3b0 0x7f70901a5fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:06:56.544 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:56.543+0000 7f7094e00700 1 -- 172.21.15.22:0/3063489089 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f709010a3b0 msgr2=0x7f70901a5fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:06:56.544 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:56.543+0000 7f7094e00700 1 --2- 172.21.15.22:0/3063489089 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f709010a3b0 0x7f70901a5fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:06:56.545 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:56.543+0000 7f7094e00700 1 -- 172.21.15.22:0/3063489089 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f7090109a10 msgr2=0x7f709019ef20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:06:56.545 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:56.543+0000 7f7094e00700 1 --2- 172.21.15.22:0/3063489089 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f7090109a10 0x7f709019ef20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:06:56.545 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:56.543+0000 7f7094e00700 1 -- 172.21.15.22:0/3063489089 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f70901a6640 con 0x7f7090108810 2024-03-28T18:06:56.545 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:56.543+0000 7f7094e00700 1 --2- 172.21.15.22:0/3063489089 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7090108810 0x7f709019e9e0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f708400d9b0 tx=0x7f708400dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:06:56.545 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:56.544+0000 7f708d600700 1 -- 172.21.15.22:0/3063489089 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7084017070 con 0x7f7090108810 2024-03-28T18:06:56.545 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:56.544+0000 7f708d600700 1 -- 172.21.15.22:0/3063489089 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f7084009e00 con 0x7f7090108810 2024-03-28T18:06:56.545 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:56.544+0000 7f708d600700 1 -- 172.21.15.22:0/3063489089 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f708400f620 con 0x7f7090108810 2024-03-28T18:06:56.545 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:56.544+0000 7f7097800700 1 -- 172.21.15.22:0/3063489089 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f70901a6930 con 0x7f7090108810 2024-03-28T18:06:56.545 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:56.544+0000 7f7097800700 1 -- 172.21.15.22:0/3063489089 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f70901a6e20 con 0x7f7090108810 2024-03-28T18:06:56.545 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:56.545+0000 7f7097800700 1 -- 172.21.15.22:0/3063489089 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7090068e50 con 0x7f7090108810 2024-03-28T18:06:56.546 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:56.547+0000 7f708d600700 1 -- 172.21.15.22:0/3063489089 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 18) v1 ==== 137290+0+0 (secure 0 0 0) 0x7f7084010460 con 0x7f7090108810 2024-03-28T18:06:56.548 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:56.548+0000 7f708d600700 1 --2- 172.21.15.22:0/3063489089 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7f70780a2890 0x7f70780a4d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:06:56.548 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:56.549+0000 7f708d600700 1 -- 172.21.15.22:0/3063489089 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 8153+0+0 (secure 0 0 0) 0x7f70840ce980 con 0x7f7090108810 2024-03-28T18:06:56.548 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:56.549+0000 7f708fe00700 1 --2- 172.21.15.22:0/3063489089 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7f70780a2890 0x7f70780a4d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:06:56.549 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:56.549+0000 7f708fe00700 1 --2- 172.21.15.22:0/3063489089 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7f70780a2890 0x7f70780a4d50 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f707c000c00 tx=0x7f707c005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:06:56.552 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:56.553+0000 7f708d600700 1 -- 172.21.15.22:0/3063489089 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f7084010e20 con 0x7f7090108810 2024-03-28T18:06:56.778 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:56.778+0000 7f7097800700 1 -- 172.21.15.22:0/3063489089 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd dump"} v 0) v1 -- 0x7f7090062da0 con 0x7f7090108810 2024-03-28T18:06:56.779 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:56.780+0000 7f708d600700 1 -- 172.21.15.22:0/3063489089 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump"}]=0 v92) v1 ==== 56+0+5219 (secure 0 0 0) 0x7f70840b9e20 con 0x7f7090108810 2024-03-28T18:06:56.780 INFO:teuthology.orchestra.run.smithi022.stdout:epoch 92 2024-03-28T18:06:56.780 INFO:teuthology.orchestra.run.smithi022.stdout:fsid fd220748-ed2b-11ee-95d0-87774f69a715 2024-03-28T18:06:56.780 INFO:teuthology.orchestra.run.smithi022.stdout:created 2024-03-28T17:54:00.440279+0000 2024-03-28T18:06:56.780 INFO:teuthology.orchestra.run.smithi022.stdout:modified 2024-03-28T18:05:11.725341+0000 2024-03-28T18:06:56.780 INFO:teuthology.orchestra.run.smithi022.stdout:flags sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit 2024-03-28T18:06:56.781 INFO:teuthology.orchestra.run.smithi022.stdout:crush_version 26 2024-03-28T18:06:56.781 INFO:teuthology.orchestra.run.smithi022.stdout:full_ratio 0.95 2024-03-28T18:06:56.781 INFO:teuthology.orchestra.run.smithi022.stdout:backfillfull_ratio 0.9 2024-03-28T18:06:56.781 INFO:teuthology.orchestra.run.smithi022.stdout:nearfull_ratio 0.85 2024-03-28T18:06:56.781 INFO:teuthology.orchestra.run.smithi022.stdout:require_min_compat_client luminous 2024-03-28T18:06:56.781 INFO:teuthology.orchestra.run.smithi022.stdout:min_compat_client luminous 2024-03-28T18:06:56.781 INFO:teuthology.orchestra.run.smithi022.stdout:require_osd_release squid 2024-03-28T18:06:56.781 INFO:teuthology.orchestra.run.smithi022.stdout:stretch_mode_enabled false 2024-03-28T18:06:56.781 INFO:teuthology.orchestra.run.smithi022.stdout:pool 1 '.mgr' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 1 pgp_num 1 autoscale_mode off last_change 21 flags hashpspool stripe_width 0 pg_num_max 32 pg_num_min 1 application mgr read_balance_score 12.00 2024-03-28T18:06:56.781 INFO:teuthology.orchestra.run.smithi022.stdout:pool 2 'cephfs_metadata' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 32 pgp_num 32 autoscale_mode off last_change 89 flags hashpspool stripe_width 0 pg_autoscale_bias 4 pg_num_min 16 recovery_priority 5 application cephfs read_balance_score 1.88 2024-03-28T18:06:56.781 INFO:teuthology.orchestra.run.smithi022.stdout:pool 3 'cephfs_data' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 64 pgp_num 64 autoscale_mode off last_change 90 flags hashpspool stripe_width 0 pg_num_min 64 target_size_ratio 0.9 application cephfs read_balance_score 1.50 2024-03-28T18:06:56.781 INFO:teuthology.orchestra.run.smithi022.stdout:max_osd 12 2024-03-28T18:06:56.781 INFO:teuthology.orchestra.run.smithi022.stdout:osd.0 up in weight 1 up_from 8 up_thru 88 down_at 0 last_clean_interval [0,0) [v2:172.21.15.22:6802/2186290846,v1:172.21.15.22:6803/2186290846] [v2:172.21.15.22:6804/2186290846,v1:172.21.15.22:6805/2186290846] exists,up 5d708e06-f05b-40ba-bbcf-258ddc4f43d7 2024-03-28T18:06:56.781 INFO:teuthology.orchestra.run.smithi022.stdout:osd.1 up in weight 1 up_from 13 up_thru 91 down_at 0 last_clean_interval [0,0) [v2:172.21.15.79:6800/2835549681,v1:172.21.15.79:6801/2835549681] [v2:172.21.15.79:6802/2835549681,v1:172.21.15.79:6803/2835549681] exists,up 68ad09bb-9256-409a-9867-4e14a2cb94aa 2024-03-28T18:06:56.782 INFO:teuthology.orchestra.run.smithi022.stdout:osd.2 up in weight 1 up_from 18 up_thru 91 down_at 0 last_clean_interval [0,0) [v2:172.21.15.134:6800/470588085,v1:172.21.15.134:6801/470588085] [v2:172.21.15.134:6802/470588085,v1:172.21.15.134:6803/470588085] exists,up ed11a5a7-004d-41f3-a5fc-eb2d603bd33f 2024-03-28T18:06:56.782 INFO:teuthology.orchestra.run.smithi022.stdout:osd.3 up in weight 1 up_from 26 up_thru 88 down_at 0 last_clean_interval [0,0) [v2:172.21.15.22:6810/1702441854,v1:172.21.15.22:6811/1702441854] [v2:172.21.15.22:6812/1702441854,v1:172.21.15.22:6813/1702441854] exists,up 81a6c547-2a50-44c3-84d7-23ab6831790c 2024-03-28T18:06:56.782 INFO:teuthology.orchestra.run.smithi022.stdout:osd.4 up in weight 1 up_from 34 up_thru 88 down_at 0 last_clean_interval [0,0) [v2:172.21.15.79:6808/630132437,v1:172.21.15.79:6809/630132437] [v2:172.21.15.79:6810/630132437,v1:172.21.15.79:6811/630132437] exists,up d23a254e-7182-4471-8bca-7eb684ec618e 2024-03-28T18:06:56.782 INFO:teuthology.orchestra.run.smithi022.stdout:osd.5 up in weight 1 up_from 40 up_thru 88 down_at 0 last_clean_interval [0,0) [v2:172.21.15.134:6808/2135406915,v1:172.21.15.134:6809/2135406915] [v2:172.21.15.134:6810/2135406915,v1:172.21.15.134:6811/2135406915] exists,up a7b98502-2023-4f0a-bfa0-6840cc495e0c 2024-03-28T18:06:56.782 INFO:teuthology.orchestra.run.smithi022.stdout:osd.6 up in weight 1 up_from 48 up_thru 88 down_at 0 last_clean_interval [0,0) [v2:172.21.15.22:6818/1400680396,v1:172.21.15.22:6819/1400680396] [v2:172.21.15.22:6820/1400680396,v1:172.21.15.22:6821/1400680396] exists,up 634c6562-d683-41c8-8242-fc848c38ce1f 2024-03-28T18:06:56.782 INFO:teuthology.orchestra.run.smithi022.stdout:osd.7 up in weight 1 up_from 53 up_thru 91 down_at 0 last_clean_interval [0,0) [v2:172.21.15.79:6816/1287105314,v1:172.21.15.79:6817/1287105314] [v2:172.21.15.79:6818/1287105314,v1:172.21.15.79:6819/1287105314] exists,up 504eb4a8-6731-45ad-932c-1b38ef157871 2024-03-28T18:06:56.782 INFO:teuthology.orchestra.run.smithi022.stdout:osd.8 up in weight 1 up_from 62 up_thru 88 down_at 0 last_clean_interval [0,0) [v2:172.21.15.134:6816/2265650388,v1:172.21.15.134:6817/2265650388] [v2:172.21.15.134:6818/2265650388,v1:172.21.15.134:6819/2265650388] exists,up 20aa66c1-03d6-4cb8-a096-0eae41979c17 2024-03-28T18:06:56.782 INFO:teuthology.orchestra.run.smithi022.stdout:osd.9 up in weight 1 up_from 70 up_thru 88 down_at 0 last_clean_interval [0,0) [v2:172.21.15.22:6826/3762222259,v1:172.21.15.22:6827/3762222259] [v2:172.21.15.22:6828/3762222259,v1:172.21.15.22:6829/3762222259] exists,up e1ae4020-7c2c-4334-a753-ee6b12d98584 2024-03-28T18:06:56.782 INFO:teuthology.orchestra.run.smithi022.stdout:osd.10 up in weight 1 up_from 77 up_thru 91 down_at 0 last_clean_interval [0,0) [v2:172.21.15.79:6824/1746167904,v1:172.21.15.79:6825/1746167904] [v2:172.21.15.79:6826/1746167904,v1:172.21.15.79:6827/1746167904] exists,up f5198d52-df48-4679-a860-4dcb440b7a9f 2024-03-28T18:06:56.782 INFO:teuthology.orchestra.run.smithi022.stdout:osd.11 up in weight 1 up_from 85 up_thru 91 down_at 0 last_clean_interval [0,0) [v2:172.21.15.134:6824/4271716428,v1:172.21.15.134:6825/4271716428] [v2:172.21.15.134:6826/4271716428,v1:172.21.15.134:6827/4271716428] exists,up 1427bca7-3414-4468-9246-e999156bc2ae 2024-03-28T18:06:56.782 INFO:teuthology.orchestra.run.smithi022.stdout:pg_upmap_items 2.8 [2,11] 2024-03-28T18:06:56.782 INFO:teuthology.orchestra.run.smithi022.stdout:pg_upmap_items 2.1d [8,11] 2024-03-28T18:06:56.782 INFO:teuthology.orchestra.run.smithi022.stdout:pg_upmap_items 3.3 [4,9] 2024-03-28T18:06:56.782 INFO:teuthology.orchestra.run.smithi022.stdout:pg_upmap_items 3.c [3,9] 2024-03-28T18:06:56.783 INFO:teuthology.orchestra.run.smithi022.stdout:pg_upmap_items 3.21 [4,8] 2024-03-28T18:06:56.783 INFO:teuthology.orchestra.run.smithi022.stdout:pg_upmap_items 3.25 [5,9] 2024-03-28T18:06:56.783 INFO:teuthology.orchestra.run.smithi022.stdout:blocklist 172.21.15.22:0/3521924283 expires 2024-03-29T17:55:10.425504+0000 2024-03-28T18:06:56.783 INFO:teuthology.orchestra.run.smithi022.stdout:blocklist 172.21.15.22:6801/730635939 expires 2024-03-29T17:55:10.425504+0000 2024-03-28T18:06:56.783 INFO:teuthology.orchestra.run.smithi022.stdout:blocklist 172.21.15.22:0/2471229459 expires 2024-03-29T17:54:34.004559+0000 2024-03-28T18:06:56.783 INFO:teuthology.orchestra.run.smithi022.stdout:blocklist 172.21.15.22:6800/3166828081 expires 2024-03-29T17:54:34.004559+0000 2024-03-28T18:06:56.783 INFO:teuthology.orchestra.run.smithi022.stdout:blocklist 172.21.15.22:6800/730635939 expires 2024-03-29T17:55:10.425504+0000 2024-03-28T18:06:56.783 INFO:teuthology.orchestra.run.smithi022.stdout:blocklist 172.21.15.22:0/1229047849 expires 2024-03-29T17:54:34.004559+0000 2024-03-28T18:06:56.783 INFO:teuthology.orchestra.run.smithi022.stdout:blocklist 172.21.15.22:6801/3166828081 expires 2024-03-29T17:54:34.004559+0000 2024-03-28T18:06:56.783 INFO:teuthology.orchestra.run.smithi022.stdout:blocklist 172.21.15.22:0/2717818856 expires 2024-03-29T17:55:10.425504+0000 2024-03-28T18:06:56.783 INFO:teuthology.orchestra.run.smithi022.stdout:blocklist 172.21.15.22:0/2157570333 expires 2024-03-29T17:54:34.004559+0000 2024-03-28T18:06:56.783 INFO:teuthology.orchestra.run.smithi022.stdout:blocklist 172.21.15.22:0/1006589530 expires 2024-03-29T17:55:10.425504+0000 2024-03-28T18:06:56.784 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:56.782+0000 7f7097800700 1 -- 172.21.15.22:0/3063489089 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7f70780a2890 msgr2=0x7f70780a4d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:06:56.784 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:56.782+0000 7f7097800700 1 --2- 172.21.15.22:0/3063489089 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7f70780a2890 0x7f70780a4d50 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f707c000c00 tx=0x7f707c005fb0 comp rx=0 tx=0).stop 2024-03-28T18:06:56.784 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:56.783+0000 7f7097800700 1 -- 172.21.15.22:0/3063489089 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7090108810 msgr2=0x7f709019e9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:06:56.784 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:56.783+0000 7f7097800700 1 --2- 172.21.15.22:0/3063489089 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7090108810 0x7f709019e9e0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f708400d9b0 tx=0x7f708400dcc0 comp rx=0 tx=0).stop 2024-03-28T18:06:56.784 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:56.783+0000 7f7097800700 1 -- 172.21.15.22:0/3063489089 shutdown_connections 2024-03-28T18:06:56.784 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:56.783+0000 7f7097800700 1 --2- 172.21.15.22:0/3063489089 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7f70780a2890 0x7f70780a4d50 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:06:56.784 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:56.783+0000 7f7097800700 1 --2- 172.21.15.22:0/3063489089 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f709010a3b0 0x7f70901a5fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:06:56.785 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:56.783+0000 7f7097800700 1 --2- 172.21.15.22:0/3063489089 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7090108810 0x7f709019e9e0 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:06:56.785 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:56.783+0000 7f7097800700 1 --2- 172.21.15.22:0/3063489089 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f7090109a10 0x7f709019ef20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:06:56.785 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:56.783+0000 7f7097800700 1 -- 172.21.15.22:0/3063489089 >> 172.21.15.22:0/3063489089 conn(0x7f70900777e0 msgr2=0x7f709010ccb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:06:56.785 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:56.783+0000 7f7097800700 1 -- 172.21.15.22:0/3063489089 shutdown_connections 2024-03-28T18:06:56.785 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:56.783+0000 7f7097800700 1 -- 172.21.15.22:0/3063489089 wait complete. 2024-03-28T18:06:57.424 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:06:57 smithi134 ceph-mon[35037]: from='client.? 172.21.15.22:0/3063489089' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-03-28T18:06:57.432 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:06:57 smithi079 ceph-mon[35312]: from='client.? 172.21.15.22:0/3063489089' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-03-28T18:06:57.457 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:06:57 smithi022 ceph-mon[31985]: from='client.? 172.21.15.22:0/3063489089' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-03-28T18:06:57.550 INFO:teuthology.run_tasks:Running task fs.ready... 2024-03-28T18:06:57.561 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-28T18:06:57.795 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:57.795+0000 7ff5b6200640 1 -- 172.21.15.22:0/3217172007 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff5b0104ad0 msgr2=0x7ff5b0104f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:06:57.795 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:57.795+0000 7ff5b6200640 1 --2- 172.21.15.22:0/3217172007 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff5b0104ad0 0x7ff5b0104f50 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7ff59c009900 tx=0x7ff59c02bc30 comp rx=0 tx=0).stop 2024-03-28T18:06:57.795 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:57.795+0000 7ff5b6200640 1 -- 172.21.15.22:0/3217172007 shutdown_connections 2024-03-28T18:06:57.795 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:57.795+0000 7ff5b6200640 1 --2- 172.21.15.22:0/3217172007 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ff5b0105490 0x7ff5b010bd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:06:57.795 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:57.795+0000 7ff5b6200640 1 --2- 172.21.15.22:0/3217172007 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff5b0104ad0 0x7ff5b0104f50 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:06:57.795 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:57.795+0000 7ff5b6200640 1 --2- 172.21.15.22:0/3217172007 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff5b01038e0 0x7ff5b0103d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:06:57.795 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:57.795+0000 7ff5b6200640 1 -- 172.21.15.22:0/3217172007 >> 172.21.15.22:0/3217172007 conn(0x7ff5b00fee90 msgr2=0x7ff5b01012b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:06:57.795 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:57.795+0000 7ff5b6200640 1 -- 172.21.15.22:0/3217172007 shutdown_connections 2024-03-28T18:06:57.795 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:57.795+0000 7ff5b6200640 1 -- 172.21.15.22:0/3217172007 wait complete. 2024-03-28T18:06:57.797 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:57.796+0000 7ff5b6200640 1 Processor -- start 2024-03-28T18:06:57.797 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:57.796+0000 7ff5b6200640 1 -- start start 2024-03-28T18:06:57.797 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:57.796+0000 7ff5b6200640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ff5b01038e0 0x7ff5b0199be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:06:57.798 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:57.796+0000 7ff5b6200640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff5b0104ad0 0x7ff5b019a120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:06:57.798 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:57.796+0000 7ff5b6200640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff5b0105490 0x7ff5b01a1190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:06:57.798 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:57.796+0000 7ff5b6200640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5b010f540 con 0x7ff5b0105490 2024-03-28T18:06:57.798 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:57.796+0000 7ff5b6200640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5b010f3c0 con 0x7ff5b01038e0 2024-03-28T18:06:57.798 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:57.796+0000 7ff5b6200640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5b010f6c0 con 0x7ff5b0104ad0 2024-03-28T18:06:57.798 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:57.797+0000 7ff5afe00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff5b0105490 0x7ff5b01a1190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:06:57.798 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:57.797+0000 7ff5afe00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff5b0105490 0x7ff5b01a1190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:39306/0 (socket says 172.21.15.22:39306) 2024-03-28T18:06:57.798 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:57.797+0000 7ff5afe00640 1 -- 172.21.15.22:0/2126894899 learned_addr learned my addr 172.21.15.22:0/2126894899 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-28T18:06:57.798 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:57.797+0000 7ff5af400640 1 --2- 172.21.15.22:0/2126894899 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ff5b01038e0 0x7ff5b0199be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:06:57.798 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:57.797+0000 7ff5aea00640 1 --2- 172.21.15.22:0/2126894899 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff5b0104ad0 0x7ff5b019a120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:06:57.798 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:57.797+0000 7ff5afe00640 1 -- 172.21.15.22:0/2126894899 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff5b0104ad0 msgr2=0x7ff5b019a120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:06:57.798 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:57.797+0000 7ff5afe00640 1 --2- 172.21.15.22:0/2126894899 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff5b0104ad0 0x7ff5b019a120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:06:57.798 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:57.797+0000 7ff5afe00640 1 -- 172.21.15.22:0/2126894899 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ff5b01038e0 msgr2=0x7ff5b0199be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:06:57.799 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:57.797+0000 7ff5afe00640 1 --2- 172.21.15.22:0/2126894899 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ff5b01038e0 0x7ff5b0199be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:06:57.799 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:57.797+0000 7ff5afe00640 1 -- 172.21.15.22:0/2126894899 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff5b01a1800 con 0x7ff5b0105490 2024-03-28T18:06:57.799 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:57.797+0000 7ff5afe00640 1 --2- 172.21.15.22:0/2126894899 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff5b0105490 0x7ff5b01a1190 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7ff5a800eb10 tx=0x7ff5a800c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:06:57.799 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:57.798+0000 7ff5a7e00640 1 -- 172.21.15.22:0/2126894899 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5a8004b20 con 0x7ff5b0105490 2024-03-28T18:06:57.799 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:57.798+0000 7ff5a7e00640 1 -- 172.21.15.22:0/2126894899 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7ff5a8004cc0 con 0x7ff5b0105490 2024-03-28T18:06:57.799 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:57.798+0000 7ff5b6200640 1 -- 172.21.15.22:0/2126894899 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff5b01a1af0 con 0x7ff5b0105490 2024-03-28T18:06:57.799 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:57.798+0000 7ff5a7e00640 1 -- 172.21.15.22:0/2126894899 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5a8010700 con 0x7ff5b0105490 2024-03-28T18:06:57.799 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:57.798+0000 7ff5b6200640 1 -- 172.21.15.22:0/2126894899 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff5b01a2040 con 0x7ff5b0105490 2024-03-28T18:06:57.799 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:57.799+0000 7ff5b6200640 1 -- 172.21.15.22:0/2126894899 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff5b010ac00 con 0x7ff5b0105490 2024-03-28T18:06:57.801 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:57.801+0000 7ff5a7e00640 1 -- 172.21.15.22:0/2126894899 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 18) v1 ==== 137290+0+0 (secure 0 0 0) 0x7ff5a8004e70 con 0x7ff5b0105490 2024-03-28T18:06:57.802 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:57.802+0000 7ff5a7e00640 1 --2- 172.21.15.22:0/2126894899 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7ff5940a2880 0x7ff5940a4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:06:57.802 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:57.803+0000 7ff5a7e00640 1 -- 172.21.15.22:0/2126894899 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 8153+0+0 (secure 0 0 0) 0x7ff5a80cefe0 con 0x7ff5b0105490 2024-03-28T18:06:57.802 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:57.803+0000 7ff5af400640 1 --2- 172.21.15.22:0/2126894899 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7ff5940a2880 0x7ff5940a4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:06:57.803 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:57.804+0000 7ff5af400640 1 --2- 172.21.15.22:0/2126894899 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7ff5940a2880 0x7ff5940a4d40 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7ff5a0005fd0 tx=0x7ff5a000b390 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:06:57.807 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:57.807+0000 7ff5a7e00640 1 -- 172.21.15.22:0/2126894899 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7ff5a809c0c0 con 0x7ff5b0105490 2024-03-28T18:06:58.020 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:58.020+0000 7ff5b6200640 1 -- 172.21.15.22:0/2126894899 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ff5b0062340 con 0x7ff5b0105490 2024-03-28T18:06:58.023 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:58.024+0000 7ff5a7e00640 1 -- 172.21.15.22:0/2126894899 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v92) v1 ==== 74+0+22522 (secure 0 0 0) 0x7ff5a809b9c0 con 0x7ff5b0105490 2024-03-28T18:06:58.024 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-03-28T18:06:58.024 INFO:teuthology.orchestra.run.smithi022.stdout:{"epoch":92,"fsid":"fd220748-ed2b-11ee-95d0-87774f69a715","created":"2024-03-28T17:54:00.440279+0000","modified":"2024-03-28T18:05:11.725341+0000","last_up_change":"2024-03-28T18:03:43.220095+0000","last_in_change":"2024-03-28T18:03:18.526399+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-28T17:58:33.168872+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-28T18:04:47.004797+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-28T18:04:48.465502+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"90","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"5d708e06-f05b-40ba-bbcf-258ddc4f43d7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6802","nonce":2186290846},{"type":"v1","addr":"172.21.15.22:6803","nonce":2186290846}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6804","nonce":2186290846},{"type":"v1","addr":"172.21.15.22:6805","nonce":2186290846}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6808","nonce":2186290846},{"type":"v1","addr":"172.21.15.22:6809","nonce":2186290846}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6806","nonce":2186290846},{"type":"v1","addr":"172.21.15.22:6807","nonce":2186290846}]},"public_addr":"172.21.15.22:6803/2186290846","cluster_addr":"172.21.15.22:6805/2186290846","heartbeat_back_addr":"172.21.15.22:6809/2186290846","heartbeat_front_addr":"172.21.15.22:6807/2186290846","state":["exists","up"]},{"osd":1,"uuid":"68ad09bb-9256-409a-9867-4e14a2cb94aa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6800","nonce":2835549681},{"type":"v1","addr":"172.21.15.79:6801","nonce":2835549681}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6802","nonce":2835549681},{"type":"v1","addr":"172.21.15.79:6803","nonce":2835549681}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6806","nonce":2835549681},{"type":"v1","addr":"172.21.15.79:6807","nonce":2835549681}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6804","nonce":2835549681},{"type":"v1","addr":"172.21.15.79:6805","nonce":2835549681}]},"public_addr":"172.21.15.79:6801/2835549681","cluster_addr":"172.21.15.79:6803/2835549681","heartbeat_back_addr":"172.21.15.79:6807/2835549681","heartbeat_front_addr":"172.21.15.79:6805/2835549681","state":["exists","up"]},{"osd":2,"uuid":"ed11a5a7-004d-41f3-a5fc-eb2d603bd33f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6800","nonce":470588085},{"type":"v1","addr":"172.21.15.134:6801","nonce":470588085}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6802","nonce":470588085},{"type":"v1","addr":"172.21.15.134:6803","nonce":470588085}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6806","nonce":470588085},{"type":"v1","addr":"172.21.15.134:6807","nonce":470588085}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6804","nonce":470588085},{"type":"v1","addr":"172.21.15.134:6805","nonce":470588085}]},"public_addr":"172.21.15.134:6801/470588085","cluster_addr":"172.21.15.134:6803/470588085","heartbeat_back_addr":"172.21.15.134:6807/470588085","heartbeat_front_addr":"172.21.15.134:6805/470588085","state":["exists","up"]},{"osd":3,"uuid":"81a6c547-2a50-44c3-84d7-23ab6831790c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6810","nonce":1702441854},{"type":"v1","addr":"172.21.15.22:6811","nonce":1702441854}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6812","nonce":1702441854},{"type":"v1","addr":"172.21.15.22:6813","nonce":1702441854}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6816","nonce":1702441854},{"type":"v1","addr":"172.21.15.22:6817","nonce":1702441854}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6814","nonce":1702441854},{"type":"v1","addr":"172.21.15.22:6815","nonce":1702441854}]},"public_addr":"172.21.15.22:6811/1702441854","cluster_addr":"172.21.15.22:6813/1702441854","heartbeat_back_addr":"172.21.15.22:6817/1702441854","heartbeat_front_addr":"172.21.15.22:6815/1702441854","state":["exists","up"]},{"osd":4,"uuid":"d23a254e-7182-4471-8bca-7eb684ec618e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6808","nonce":630132437},{"type":"v1","addr":"172.21.15.79:6809","nonce":630132437}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6810","nonce":630132437},{"type":"v1","addr":"172.21.15.79:6811","nonce":630132437}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6814","nonce":630132437},{"type":"v1","addr":"172.21.15.79:6815","nonce":630132437}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6812","nonce":630132437},{"type":"v1","addr":"172.21.15.79:6813","nonce":630132437}]},"public_addr":"172.21.15.79:6809/630132437","cluster_addr":"172.21.15.79:6811/630132437","heartbeat_back_addr":"172.21.15.79:6815/630132437","heartbeat_front_addr":"172.21.15.79:6813/630132437","state":["exists","up"]},{"osd":5,"uuid":"a7b98502-2023-4f0a-bfa0-6840cc495e0c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":40,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6808","nonce":2135406915},{"type":"v1","addr":"172.21.15.134:6809","nonce":2135406915}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6810","nonce":2135406915},{"type":"v1","addr":"172.21.15.134:6811","nonce":2135406915}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6814","nonce":2135406915},{"type":"v1","addr":"172.21.15.134:6815","nonce":2135406915}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6812","nonce":2135406915},{"type":"v1","addr":"172.21.15.134:6813","nonce":2135406915}]},"public_addr":"172.21.15.134:6809/2135406915","cluster_addr":"172.21.15.134:6811/2135406915","heartbeat_back_addr":"172.21.15.134:6815/2135406915","heartbeat_front_addr":"172.21.15.134:6813/2135406915","state":["exists","up"]},{"osd":6,"uuid":"634c6562-d683-41c8-8242-fc848c38ce1f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6818","nonce":1400680396},{"type":"v1","addr":"172.21.15.22:6819","nonce":1400680396}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6820","nonce":1400680396},{"type":"v1","addr":"172.21.15.22:6821","nonce":1400680396}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6824","nonce":1400680396},{"type":"v1","addr":"172.21.15.22:6825","nonce":1400680396}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6822","nonce":1400680396},{"type":"v1","addr":"172.21.15.22:6823","nonce":1400680396}]},"public_addr":"172.21.15.22:6819/1400680396","cluster_addr":"172.21.15.22:6821/1400680396","heartbeat_back_addr":"172.21.15.22:6825/1400680396","heartbeat_front_addr":"172.21.15.22:6823/1400680396","state":["exists","up"]},{"osd":7,"uuid":"504eb4a8-6731-45ad-932c-1b38ef157871","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":53,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6816","nonce":1287105314},{"type":"v1","addr":"172.21.15.79:6817","nonce":1287105314}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6818","nonce":1287105314},{"type":"v1","addr":"172.21.15.79:6819","nonce":1287105314}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6822","nonce":1287105314},{"type":"v1","addr":"172.21.15.79:6823","nonce":1287105314}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6820","nonce":1287105314},{"type":"v1","addr":"172.21.15.79:6821","nonce":1287105314}]},"public_addr":"172.21.15.79:6817/1287105314","cluster_addr":"172.21.15.79:6819/1287105314","heartbeat_back_addr":"172.21.15.79:6823/1287105314","heartbeat_front_addr":"172.21.15.79:6821/1287105314","state":["exists","up"]},{"osd":8,"uuid":"20aa66c1-03d6-4cb8-a096-0eae41979c17","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":62,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6816","nonce":2265650388},{"type":"v1","addr":"172.21.15.134:6817","nonce":2265650388}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6818","nonce":2265650388},{"type":"v1","addr":"172.21.15.134:6819","nonce":2265650388}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6822","nonce":2265650388},{"type":"v1","addr":"172.21.15.134:6823","nonce":2265650388}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6820","nonce":2265650388},{"type":"v1","addr":"172.21.15.134:6821","nonce":2265650388}]},"public_addr":"172.21.15.134:6817/2265650388","cluster_addr":"172.21.15.134:6819/2265650388","heartbeat_back_addr":"172.21.15.134:6823/2265650388","heartbeat_front_addr":"172.21.15.134:6821/2265650388","state":["exists","up"]},{"osd":9,"uuid":"e1ae4020-7c2c-4334-a753-ee6b12d98584","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":70,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6826","nonce":3762222259},{"type":"v1","addr":"172.21.15.22:6827","nonce":3762222259}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6828","nonce":3762222259},{"type":"v1","addr":"172.21.15.22:6829","nonce":3762222259}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6832","nonce":3762222259},{"type":"v1","addr":"172.21.15.22:6833","nonce":3762222259}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6830","nonce":3762222259},{"type":"v1","addr":"172.21.15.22:6831","nonce":3762222259}]},"public_addr":"172.21.15.22:6827/3762222259","cluster_addr":"172.21.15.22:6829/3762222259","heartbeat_back_addr":"172.21.15.22:6833/3762222259","heartbeat_front_addr":"172.21.15.22:6831/3762222259","state":["exists","up"]},{"osd":10,"uuid":"f5198d52-df48-4679-a860-4dcb440b7a9f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":77,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6824","nonce":1746167904},{"type":"v1","addr":"172.21.15.79:6825","nonce":1746167904}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6826","nonce":1746167904},{"type":"v1","addr":"172.21.15.79:6827","nonce":1746167904}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6830","nonce":1746167904},{"type":"v1","addr":"172.21.15.79:6831","nonce":1746167904}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6828","nonce":1746167904},{"type":"v1","addr":"172.21.15.79:6829","nonce":1746167904}]},"public_addr":"172.21.15.79:6825/1746167904","cluster_addr":"172.21.15.79:6827/1746167904","heartbeat_back_addr":"172.21.15.79:6831/1746167904","heartbeat_front_addr":"172.21.15.79:6829/1746167904","state":["exists","up"]},{"osd":11,"uuid":"1427bca7-3414-4468-9246-e999156bc2ae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":85,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6824","nonce":4271716428},{"type":"v1","addr":"172.21.15.134:6825","nonce":4271716428}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6826","nonce":4271716428},{"type":"v1","addr":"172.21.15.134:6827","nonce":4271716428}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6830","nonce":4271716428},{"type":"v1","addr":"172.21.15.134:6831","nonce":4271716428}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6828","nonce":4271716428},{"type":"v1","addr":"172.21.15.134:6829","nonce":4271716428}]},"public_addr":"172.21.15.134:6825/4271716428","cluster_addr":"172.21.15.134:6827/4271716428","heartbeat_back_addr":"172.21.15.134:6831/4271716428","heartbeat_front_addr":"172.21.15.134:6829/4271716428","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T17:57:21.225125+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T17:57:55.197254+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T17:58:27.825136+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T17:59:04.319893+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T17:59:38.407752+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T18:00:12.118805+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T18:00:47.635214+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T18:01:21.231055+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T18:01:55.817861+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T18:02:31.680160+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T18:03:06.783398+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T18:03:41.307347+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.8","mappings":[{"from":2,"to":11}]},{"pgid":"2.1d","mappings":[{"from":8,"to":11}]},{"pgid":"3.3","mappings":[{"from":4,"to":9}]},{"pgid":"3.c","mappings":[{"from":3,"to":9}]},{"pgid":"3.21","mappings":[{"from":4,"to":8}]},{"pgid":"3.25","mappings":[{"from":5,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.22:0/3521924283":"2024-03-29T17:55:10.425504+0000","172.21.15.22:6801/730635939":"2024-03-29T17:55:10.425504+0000","172.21.15.22:0/2471229459":"2024-03-29T17:54:34.004559+0000","172.21.15.22:6800/3166828081":"2024-03-29T17:54:34.004559+0000","172.21.15.22:6800/730635939":"2024-03-29T17:55:10.425504+0000","172.21.15.22:0/1229047849":"2024-03-29T17:54:34.004559+0000","172.21.15.22:6801/3166828081":"2024-03-29T17:54:34.004559+0000","172.21.15.22:0/2717818856":"2024-03-29T17:55:10.425504+0000","172.21.15.22:0/2157570333":"2024-03-29T17:54:34.004559+0000","172.21.15.22:0/1006589530":"2024-03-29T17:55:10.425504+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-28T18:06:58.026 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:58.026+0000 7ff5b6200640 1 -- 172.21.15.22:0/2126894899 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7ff5940a2880 msgr2=0x7ff5940a4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:06:58.026 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:58.026+0000 7ff5b6200640 1 --2- 172.21.15.22:0/2126894899 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7ff5940a2880 0x7ff5940a4d40 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7ff5a0005fd0 tx=0x7ff5a000b390 comp rx=0 tx=0).stop 2024-03-28T18:06:58.026 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:58.026+0000 7ff5b6200640 1 -- 172.21.15.22:0/2126894899 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff5b0105490 msgr2=0x7ff5b01a1190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:06:58.026 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:58.026+0000 7ff5b6200640 1 --2- 172.21.15.22:0/2126894899 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff5b0105490 0x7ff5b01a1190 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7ff5a800eb10 tx=0x7ff5a800c2d0 comp rx=0 tx=0).stop 2024-03-28T18:06:58.026 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:58.027+0000 7ff5b6200640 1 -- 172.21.15.22:0/2126894899 shutdown_connections 2024-03-28T18:06:58.027 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:58.027+0000 7ff5b6200640 1 --2- 172.21.15.22:0/2126894899 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7ff5940a2880 0x7ff5940a4d40 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:06:58.027 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:58.027+0000 7ff5b6200640 1 --2- 172.21.15.22:0/2126894899 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff5b0105490 0x7ff5b01a1190 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:06:58.027 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:58.027+0000 7ff5b6200640 1 --2- 172.21.15.22:0/2126894899 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff5b0104ad0 0x7ff5b019a120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:06:58.027 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:58.027+0000 7ff5b6200640 1 --2- 172.21.15.22:0/2126894899 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ff5b01038e0 0x7ff5b0199be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:06:58.027 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:58.027+0000 7ff5b6200640 1 -- 172.21.15.22:0/2126894899 >> 172.21.15.22:0/2126894899 conn(0x7ff5b00fee90 msgr2=0x7ff5b0107dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:06:58.027 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:58.027+0000 7ff5b6200640 1 -- 172.21.15.22:0/2126894899 shutdown_connections 2024-03-28T18:06:58.027 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:58.027+0000 7ff5b6200640 1 -- 172.21.15.22:0/2126894899 wait complete. 2024-03-28T18:06:58.047 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-28T17:58:33.168872+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-28T18:04:47.004797+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '89', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-03-28T18:04:48.465502+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '90', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-28T18:06:58.048 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-28T18:06:58.274 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:58.274+0000 7f2f01000640 1 -- 172.21.15.22:0/2189862156 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2efc1037a0 msgr2=0x7f2efc103bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:06:58.274 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:58.274+0000 7f2f01000640 1 --2- 172.21.15.22:0/2189862156 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2efc1037a0 0x7f2efc103bc0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f2eec009ca0 tx=0x7f2eec02c0f0 comp rx=0 tx=0).stop 2024-03-28T18:06:58.275 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:58.274+0000 7f2f01000640 1 -- 172.21.15.22:0/2189862156 shutdown_connections 2024-03-28T18:06:58.275 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:58.274+0000 7f2f01000640 1 --2- 172.21.15.22:0/2189862156 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2efc105350 0x7f2efc10bbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:06:58.275 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:58.274+0000 7f2f01000640 1 --2- 172.21.15.22:0/2189862156 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f2efc104990 0x7f2efc104e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:06:58.275 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:58.274+0000 7f2f01000640 1 --2- 172.21.15.22:0/2189862156 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2efc1037a0 0x7f2efc103bc0 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:06:58.275 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:58.274+0000 7f2f01000640 1 -- 172.21.15.22:0/2189862156 >> 172.21.15.22:0/2189862156 conn(0x7f2efc0fed10 msgr2=0x7f2efc101150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:06:58.275 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:58.275+0000 7f2f01000640 1 -- 172.21.15.22:0/2189862156 shutdown_connections 2024-03-28T18:06:58.275 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:58.275+0000 7f2f01000640 1 -- 172.21.15.22:0/2189862156 wait complete. 2024-03-28T18:06:58.276 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:58.276+0000 7f2f01000640 1 Processor -- start 2024-03-28T18:06:58.276 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:58.276+0000 7f2f01000640 1 -- start start 2024-03-28T18:06:58.276 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:58.276+0000 7f2f01000640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2efc1037a0 0x7f2efc199ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:06:58.276 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:58.276+0000 7f2f01000640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2efc104990 0x7f2efc199ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:06:58.276 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:58.276+0000 7f2f01000640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f2efc105350 0x7f2efc1a1060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:06:58.277 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:58.276+0000 7f2f01000640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2efc10f400 con 0x7f2efc1037a0 2024-03-28T18:06:58.277 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:58.276+0000 7f2f01000640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2efc10f280 con 0x7f2efc105350 2024-03-28T18:06:58.277 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:58.276+0000 7f2f01000640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2efc10f580 con 0x7f2efc104990 2024-03-28T18:06:58.277 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:58.276+0000 7f2efb400640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2efc1037a0 0x7f2efc199ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:06:58.277 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:58.277+0000 7f2efb400640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2efc1037a0 0x7f2efc199ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:39314/0 (socket says 172.21.15.22:39314) 2024-03-28T18:06:58.277 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:58.277+0000 7f2efb400640 1 -- 172.21.15.22:0/839745348 learned_addr learned my addr 172.21.15.22:0/839745348 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-28T18:06:58.277 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:58.277+0000 7f2efbe00640 1 --2- 172.21.15.22:0/839745348 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f2efc105350 0x7f2efc1a1060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:06:58.277 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:58.277+0000 7f2efaa00640 1 --2- 172.21.15.22:0/839745348 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2efc104990 0x7f2efc199ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:06:58.278 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:58.277+0000 7f2efb400640 1 -- 172.21.15.22:0/839745348 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2efc104990 msgr2=0x7f2efc199ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:06:58.278 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:58.277+0000 7f2efb400640 1 --2- 172.21.15.22:0/839745348 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2efc104990 0x7f2efc199ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:06:58.278 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:58.277+0000 7f2efb400640 1 -- 172.21.15.22:0/839745348 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f2efc105350 msgr2=0x7f2efc1a1060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:06:58.278 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:58.277+0000 7f2efb400640 1 --2- 172.21.15.22:0/839745348 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f2efc105350 0x7f2efc1a1060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:06:58.278 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:58.277+0000 7f2efb400640 1 -- 172.21.15.22:0/839745348 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2efc1a16d0 con 0x7f2efc1037a0 2024-03-28T18:06:58.278 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:58.277+0000 7f2efb400640 1 --2- 172.21.15.22:0/839745348 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2efc1037a0 0x7f2efc199ab0 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f2eec000f80 tx=0x7f2eec032900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:06:58.278 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:58.278+0000 7f2ef3e00640 1 -- 172.21.15.22:0/839745348 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2eec039050 con 0x7f2efc1037a0 2024-03-28T18:06:58.279 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:58.278+0000 7f2ef3e00640 1 -- 172.21.15.22:0/839745348 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f2eec03a040 con 0x7f2efc1037a0 2024-03-28T18:06:58.279 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:58.278+0000 7f2ef3e00640 1 -- 172.21.15.22:0/839745348 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2eec040450 con 0x7f2efc1037a0 2024-03-28T18:06:58.279 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:58.278+0000 7f2f01000640 1 -- 172.21.15.22:0/839745348 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2efc1a1960 con 0x7f2efc1037a0 2024-03-28T18:06:58.279 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:58.278+0000 7f2f01000640 1 -- 172.21.15.22:0/839745348 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2efc1a1e00 con 0x7f2efc1037a0 2024-03-28T18:06:58.279 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:58.279+0000 7f2f01000640 1 -- 172.21.15.22:0/839745348 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2efc10aac0 con 0x7f2efc1037a0 2024-03-28T18:06:58.281 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:58.281+0000 7f2ef3e00640 1 -- 172.21.15.22:0/839745348 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 18) v1 ==== 137290+0+0 (secure 0 0 0) 0x7f2eec003b70 con 0x7f2efc1037a0 2024-03-28T18:06:58.282 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:58.283+0000 7f2ef3e00640 1 --2- 172.21.15.22:0/839745348 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7f2ee00a2880 0x7f2ee00a4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:06:58.282 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:58.283+0000 7f2ef3e00640 1 -- 172.21.15.22:0/839745348 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 8153+0+0 (secure 0 0 0) 0x7f2eec0f1ef0 con 0x7f2efc1037a0 2024-03-28T18:06:58.283 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:58.283+0000 7f2efaa00640 1 --2- 172.21.15.22:0/839745348 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7f2ee00a2880 0x7f2ee00a4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:06:58.283 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:58.284+0000 7f2efaa00640 1 --2- 172.21.15.22:0/839745348 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7f2ee00a2880 0x7f2ee00a4d40 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f2efc19afc0 tx=0x7f2ee4005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:06:58.285 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:58.286+0000 7f2ef3e00640 1 -- 172.21.15.22:0/839745348 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f2eec0bef90 con 0x7f2efc1037a0 2024-03-28T18:06:58.424 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:06:58 smithi134 ceph-mon[35037]: pgmap v435: 97 pgs: 97 active+clean; 643 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2024-03-28T18:06:58.425 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:06:58 smithi134 ceph-mon[35037]: from='client.? 172.21.15.22:0/2126894899' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-28T18:06:58.432 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:06:58 smithi079 ceph-mon[35312]: pgmap v435: 97 pgs: 97 active+clean; 643 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2024-03-28T18:06:58.432 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:06:58 smithi079 ceph-mon[35312]: from='client.? 172.21.15.22:0/2126894899' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-28T18:06:58.491 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:58.491+0000 7f2f01000640 1 -- 172.21.15.22:0/839745348 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f2efc02d200 con 0x7f2efc1037a0 2024-03-28T18:06:58.492 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:58.492+0000 7f2ef3e00640 1 -- 172.21.15.22:0/839745348 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v92) v1 ==== 93+0+10 (secure 0 0 0) 0x7f2eec0098b0 con 0x7f2efc1037a0 2024-03-28T18:06:58.492 INFO:teuthology.orchestra.run.smithi022.stdout:pg_num: 1 2024-03-28T18:06:58.495 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:58.495+0000 7f2f01000640 1 -- 172.21.15.22:0/839745348 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7f2ee00a2880 msgr2=0x7f2ee00a4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:06:58.495 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:58.495+0000 7f2f01000640 1 --2- 172.21.15.22:0/839745348 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7f2ee00a2880 0x7f2ee00a4d40 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f2efc19afc0 tx=0x7f2ee4005f90 comp rx=0 tx=0).stop 2024-03-28T18:06:58.495 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:58.495+0000 7f2f01000640 1 -- 172.21.15.22:0/839745348 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2efc1037a0 msgr2=0x7f2efc199ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:06:58.495 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:58.495+0000 7f2f01000640 1 --2- 172.21.15.22:0/839745348 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2efc1037a0 0x7f2efc199ab0 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f2eec000f80 tx=0x7f2eec032900 comp rx=0 tx=0).stop 2024-03-28T18:06:58.495 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:58.495+0000 7f2f01000640 1 -- 172.21.15.22:0/839745348 shutdown_connections 2024-03-28T18:06:58.495 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:58.495+0000 7f2f01000640 1 --2- 172.21.15.22:0/839745348 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7f2ee00a2880 0x7f2ee00a4d40 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:06:58.496 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:58.495+0000 7f2f01000640 1 --2- 172.21.15.22:0/839745348 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f2efc105350 0x7f2efc1a1060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:06:58.496 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:58.495+0000 7f2f01000640 1 --2- 172.21.15.22:0/839745348 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2efc104990 0x7f2efc199ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:06:58.496 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:58.495+0000 7f2f01000640 1 --2- 172.21.15.22:0/839745348 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2efc1037a0 0x7f2efc199ab0 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:06:58.496 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:58.495+0000 7f2f01000640 1 -- 172.21.15.22:0/839745348 >> 172.21.15.22:0/839745348 conn(0x7f2efc0fed10 msgr2=0x7f2efc107c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:06:58.496 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:58.495+0000 7f2f01000640 1 -- 172.21.15.22:0/839745348 shutdown_connections 2024-03-28T18:06:58.496 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:58.496+0000 7f2f01000640 1 -- 172.21.15.22:0/839745348 wait complete. 2024-03-28T18:06:58.512 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:06:58 smithi022 ceph-mon[31985]: pgmap v435: 97 pgs: 97 active+clean; 643 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2024-03-28T18:06:58.512 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:06:58 smithi022 ceph-mon[31985]: from='client.? 172.21.15.22:0/2126894899' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-28T18:06:58.516 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-28T18:06:58.746 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:58.746+0000 7fdd41e00640 1 -- 172.21.15.22:0/3749809934 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fdd3c104a60 msgr2=0x7fdd3c106e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:06:58.747 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:58.746+0000 7fdd41e00640 1 --2- 172.21.15.22:0/3749809934 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fdd3c104a60 0x7fdd3c106e50 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7fdd24009900 tx=0x7fdd2402bc30 comp rx=0 tx=0).stop 2024-03-28T18:06:58.747 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:58.746+0000 7fdd41e00640 1 -- 172.21.15.22:0/3749809934 shutdown_connections 2024-03-28T18:06:58.747 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:58.746+0000 7fdd41e00640 1 --2- 172.21.15.22:0/3749809934 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fdd3c107390 0x7fdd3c1097e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:06:58.747 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:58.746+0000 7fdd41e00640 1 --2- 172.21.15.22:0/3749809934 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fdd3c104a60 0x7fdd3c106e50 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:06:58.747 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:58.746+0000 7fdd41e00640 1 --2- 172.21.15.22:0/3749809934 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdd3c102130 0x7fdd3c104520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:06:58.747 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:58.746+0000 7fdd41e00640 1 -- 172.21.15.22:0/3749809934 >> 172.21.15.22:0/3749809934 conn(0x7fdd3c0fbb40 msgr2=0x7fdd3c0fdf60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:06:58.747 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:58.747+0000 7fdd41e00640 1 -- 172.21.15.22:0/3749809934 shutdown_connections 2024-03-28T18:06:58.747 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:58.747+0000 7fdd41e00640 1 -- 172.21.15.22:0/3749809934 wait complete. 2024-03-28T18:06:58.749 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:58.748+0000 7fdd41e00640 1 Processor -- start 2024-03-28T18:06:58.750 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:58.748+0000 7fdd41e00640 1 -- start start 2024-03-28T18:06:58.750 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:58.748+0000 7fdd41e00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdd3c102130 0x7fdd3c197ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:06:58.750 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:58.748+0000 7fdd41e00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fdd3c104a60 0x7fdd3c198000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:06:58.750 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:58.748+0000 7fdd41e00640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fdd3c107390 0x7fdd3c19f070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:06:58.750 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:58.748+0000 7fdd41e00640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd3c10d3c0 con 0x7fdd3c104a60 2024-03-28T18:06:58.750 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:58.748+0000 7fdd41e00640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd3c10d240 con 0x7fdd3c107390 2024-03-28T18:06:58.750 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:58.748+0000 7fdd41e00640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd3c10d540 con 0x7fdd3c102130 2024-03-28T18:06:58.750 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:58.748+0000 7fdd3a000640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fdd3c104a60 0x7fdd3c198000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:06:58.750 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:58.749+0000 7fdd3b400640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fdd3c107390 0x7fdd3c19f070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:06:58.750 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:58.749+0000 7fdd3b400640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fdd3c107390 0x7fdd3c19f070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.22:46730/0 (socket says 172.21.15.22:46730) 2024-03-28T18:06:58.750 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:58.749+0000 7fdd3b400640 1 -- 172.21.15.22:0/82780647 learned_addr learned my addr 172.21.15.22:0/82780647 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-28T18:06:58.750 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:58.749+0000 7fdd3a000640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fdd3c104a60 0x7fdd3c198000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:39338/0 (socket says 172.21.15.22:39338) 2024-03-28T18:06:58.750 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:58.749+0000 7fdd3aa00640 1 --2- 172.21.15.22:0/82780647 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdd3c102130 0x7fdd3c197ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:06:58.751 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:58.749+0000 7fdd3b400640 1 -- 172.21.15.22:0/82780647 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdd3c102130 msgr2=0x7fdd3c197ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:06:58.751 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:58.749+0000 7fdd3b400640 1 --2- 172.21.15.22:0/82780647 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdd3c102130 0x7fdd3c197ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:06:58.751 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:58.749+0000 7fdd3b400640 1 -- 172.21.15.22:0/82780647 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fdd3c104a60 msgr2=0x7fdd3c198000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:06:58.751 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:58.749+0000 7fdd3b400640 1 --2- 172.21.15.22:0/82780647 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fdd3c104a60 0x7fdd3c198000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:06:58.751 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:58.749+0000 7fdd3b400640 1 -- 172.21.15.22:0/82780647 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd3c19f6e0 con 0x7fdd3c107390 2024-03-28T18:06:58.751 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:58.749+0000 7fdd3a000640 1 --2- 172.21.15.22:0/82780647 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fdd3c104a60 0x7fdd3c198000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-28T18:06:58.751 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:58.749+0000 7fdd3b400640 1 --2- 172.21.15.22:0/82780647 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fdd3c107390 0x7fdd3c19f070 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fdd3400eb10 tx=0x7fdd3400c2d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:06:58.751 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:58.750+0000 7fdd40a00640 1 -- 172.21.15.22:0/82780647 <== mon.2 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd34004b20 con 0x7fdd3c107390 2024-03-28T18:06:58.751 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:58.750+0000 7fdd41e00640 1 -- 172.21.15.22:0/82780647 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdd3c19f9d0 con 0x7fdd3c107390 2024-03-28T18:06:58.751 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:58.750+0000 7fdd40a00640 1 -- 172.21.15.22:0/82780647 <== mon.2 v2:172.21.15.79:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7fdd34004cc0 con 0x7fdd3c107390 2024-03-28T18:06:58.751 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:58.750+0000 7fdd40a00640 1 -- 172.21.15.22:0/82780647 <== mon.2 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd34010700 con 0x7fdd3c107390 2024-03-28T18:06:58.751 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:58.750+0000 7fdd41e00640 1 -- 172.21.15.22:0/82780647 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdd3c19ff20 con 0x7fdd3c107390 2024-03-28T18:06:58.752 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:58.751+0000 7fdd41e00640 1 -- 172.21.15.22:0/82780647 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdd3c0fd630 con 0x7fdd3c107390 2024-03-28T18:06:58.753 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:58.754+0000 7fdd40a00640 1 -- 172.21.15.22:0/82780647 <== mon.2 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 18) v1 ==== 137290+0+0 (secure 0 0 0) 0x7fdd34004e70 con 0x7fdd3c107390 2024-03-28T18:06:58.755 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:58.755+0000 7fdd40a00640 1 --2- 172.21.15.22:0/82780647 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7fdd200a2800 0x7fdd200a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:06:58.755 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:58.755+0000 7fdd40a00640 1 -- 172.21.15.22:0/82780647 <== mon.2 v2:172.21.15.79:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 8153+0+0 (secure 0 0 0) 0x7fdd340cf270 con 0x7fdd3c107390 2024-03-28T18:06:58.755 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:58.756+0000 7fdd3aa00640 1 --2- 172.21.15.22:0/82780647 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7fdd200a2800 0x7fdd200a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:06:58.755 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:58.756+0000 7fdd3aa00640 1 --2- 172.21.15.22:0/82780647 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7fdd200a2800 0x7fdd200a4cc0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7fdd2c009890 tx=0x7fdd2c008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:06:58.764 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:58.764+0000 7fdd40a00640 1 -- 172.21.15.22:0/82780647 <== mon.2 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fdd34053740 con 0x7fdd3c107390 2024-03-28T18:06:58.968 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:58.968+0000 7fdd41e00640 1 -- 172.21.15.22:0/82780647 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fdd3c02d070 con 0x7fdd3c107390 2024-03-28T18:06:58.968 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:58.969+0000 7fdd40a00640 1 -- 172.21.15.22:0/82780647 <== mon.2 v2:172.21.15.79:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v92) v1 ==== 104+0+11 (secure 0 0 0) 0x7fdd3409c310 con 0x7fdd3c107390 2024-03-28T18:06:58.969 INFO:teuthology.orchestra.run.smithi022.stdout:pg_num: 32 2024-03-28T18:06:58.972 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:58.972+0000 7fdd41e00640 1 -- 172.21.15.22:0/82780647 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7fdd200a2800 msgr2=0x7fdd200a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:06:58.972 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:58.972+0000 7fdd41e00640 1 --2- 172.21.15.22:0/82780647 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7fdd200a2800 0x7fdd200a4cc0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7fdd2c009890 tx=0x7fdd2c008040 comp rx=0 tx=0).stop 2024-03-28T18:06:58.972 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:58.972+0000 7fdd41e00640 1 -- 172.21.15.22:0/82780647 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fdd3c107390 msgr2=0x7fdd3c19f070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:06:58.972 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:58.972+0000 7fdd41e00640 1 --2- 172.21.15.22:0/82780647 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fdd3c107390 0x7fdd3c19f070 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fdd3400eb10 tx=0x7fdd3400c2d0 comp rx=0 tx=0).stop 2024-03-28T18:06:58.972 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:58.972+0000 7fdd41e00640 1 -- 172.21.15.22:0/82780647 shutdown_connections 2024-03-28T18:06:58.972 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:58.972+0000 7fdd41e00640 1 --2- 172.21.15.22:0/82780647 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7fdd200a2800 0x7fdd200a4cc0 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:06:58.972 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:58.972+0000 7fdd41e00640 1 --2- 172.21.15.22:0/82780647 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fdd3c107390 0x7fdd3c19f070 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:06:58.972 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:58.972+0000 7fdd41e00640 1 --2- 172.21.15.22:0/82780647 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fdd3c104a60 0x7fdd3c198000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:06:58.972 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:58.972+0000 7fdd41e00640 1 --2- 172.21.15.22:0/82780647 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdd3c102130 0x7fdd3c197ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:06:58.973 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:58.972+0000 7fdd41e00640 1 -- 172.21.15.22:0/82780647 >> 172.21.15.22:0/82780647 conn(0x7fdd3c0fbb40 msgr2=0x7fdd3c102bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:06:58.973 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:58.972+0000 7fdd41e00640 1 -- 172.21.15.22:0/82780647 shutdown_connections 2024-03-28T18:06:58.973 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:58.972+0000 7fdd41e00640 1 -- 172.21.15.22:0/82780647 wait complete. 2024-03-28T18:06:58.991 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-28T18:06:59.214 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:59.214+0000 7fc9d1a00640 1 -- 172.21.15.22:0/2239402651 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc9cc105490 msgr2=0x7fc9cc10bd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:06:59.215 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:59.214+0000 7fc9d1a00640 1 --2- 172.21.15.22:0/2239402651 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc9cc105490 0x7fc9cc10bd20 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7fc9c4009900 tx=0x7fc9c402bc30 comp rx=0 tx=0).stop 2024-03-28T18:06:59.215 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:59.214+0000 7fc9d1a00640 1 -- 172.21.15.22:0/2239402651 shutdown_connections 2024-03-28T18:06:59.215 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:59.214+0000 7fc9d1a00640 1 --2- 172.21.15.22:0/2239402651 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc9cc105490 0x7fc9cc10bd20 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:06:59.215 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:59.214+0000 7fc9d1a00640 1 --2- 172.21.15.22:0/2239402651 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc9cc104ad0 0x7fc9cc104f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:06:59.215 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:59.214+0000 7fc9d1a00640 1 --2- 172.21.15.22:0/2239402651 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc9cc1038e0 0x7fc9cc103d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:06:59.215 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:59.214+0000 7fc9d1a00640 1 -- 172.21.15.22:0/2239402651 >> 172.21.15.22:0/2239402651 conn(0x7fc9cc0fee50 msgr2=0x7fc9cc101290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:06:59.215 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:59.214+0000 7fc9d1a00640 1 -- 172.21.15.22:0/2239402651 shutdown_connections 2024-03-28T18:06:59.215 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:59.215+0000 7fc9d1a00640 1 -- 172.21.15.22:0/2239402651 wait complete. 2024-03-28T18:06:59.217 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:59.215+0000 7fc9d1a00640 1 Processor -- start 2024-03-28T18:06:59.217 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:59.216+0000 7fc9d1a00640 1 -- start start 2024-03-28T18:06:59.217 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:59.216+0000 7fc9d1a00640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc9cc1038e0 0x7fc9cc199b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:06:59.217 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:59.216+0000 7fc9d1a00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc9cc104ad0 0x7fc9cc19a0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:06:59.218 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:59.216+0000 7fc9d1a00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc9cc105490 0x7fc9cc1a1120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:06:59.218 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:59.216+0000 7fc9d1a00640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9cc10f590 con 0x7fc9cc105490 2024-03-28T18:06:59.218 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:59.216+0000 7fc9d1a00640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9cc10f410 con 0x7fc9cc1038e0 2024-03-28T18:06:59.218 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:59.216+0000 7fc9d1a00640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9cc10f710 con 0x7fc9cc104ad0 2024-03-28T18:06:59.218 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:59.216+0000 7fc9cb400640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc9cc105490 0x7fc9cc1a1120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:06:59.218 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:59.216+0000 7fc9cb400640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc9cc105490 0x7fc9cc1a1120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:39362/0 (socket says 172.21.15.22:39362) 2024-03-28T18:06:59.218 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:59.216+0000 7fc9cb400640 1 -- 172.21.15.22:0/2789982228 learned_addr learned my addr 172.21.15.22:0/2789982228 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-28T18:06:59.218 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:59.216+0000 7fc9caa00640 1 --2- 172.21.15.22:0/2789982228 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc9cc1038e0 0x7fc9cc199b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:06:59.218 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:59.217+0000 7fc9ca000640 1 --2- 172.21.15.22:0/2789982228 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc9cc104ad0 0x7fc9cc19a0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:06:59.218 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:59.217+0000 7fc9cb400640 1 -- 172.21.15.22:0/2789982228 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc9cc104ad0 msgr2=0x7fc9cc19a0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:06:59.218 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:59.217+0000 7fc9cb400640 1 --2- 172.21.15.22:0/2789982228 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc9cc104ad0 0x7fc9cc19a0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:06:59.218 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:59.217+0000 7fc9cb400640 1 -- 172.21.15.22:0/2789982228 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc9cc1038e0 msgr2=0x7fc9cc199b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:06:59.218 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:59.217+0000 7fc9cb400640 1 --2- 172.21.15.22:0/2789982228 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc9cc1038e0 0x7fc9cc199b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:06:59.218 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:59.217+0000 7fc9cb400640 1 -- 172.21.15.22:0/2789982228 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc9cc1a1790 con 0x7fc9cc105490 2024-03-28T18:06:59.219 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:59.217+0000 7fc9cb400640 1 --2- 172.21.15.22:0/2789982228 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc9cc105490 0x7fc9cc1a1120 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7fc9c4036040 tx=0x7fc9c4005de0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:06:59.219 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:59.217+0000 7fc9c3400640 1 -- 172.21.15.22:0/2789982228 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9c4043020 con 0x7fc9cc105490 2024-03-28T18:06:59.219 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:59.217+0000 7fc9c3400640 1 -- 172.21.15.22:0/2789982228 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7fc9c40053f0 con 0x7fc9cc105490 2024-03-28T18:06:59.219 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:59.217+0000 7fc9d1a00640 1 -- 172.21.15.22:0/2789982228 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc9cc1a1a20 con 0x7fc9cc105490 2024-03-28T18:06:59.219 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:59.217+0000 7fc9c3400640 1 -- 172.21.15.22:0/2789982228 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9c4003c50 con 0x7fc9cc105490 2024-03-28T18:06:59.219 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:59.218+0000 7fc9d1a00640 1 -- 172.21.15.22:0/2789982228 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc9cc1a1fc0 con 0x7fc9cc105490 2024-03-28T18:06:59.219 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:59.219+0000 7fc9d1a00640 1 -- 172.21.15.22:0/2789982228 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc9cc10ac00 con 0x7fc9cc105490 2024-03-28T18:06:59.220 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:59.221+0000 7fc9c3400640 1 -- 172.21.15.22:0/2789982228 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 18) v1 ==== 137290+0+0 (secure 0 0 0) 0x7fc9c4046020 con 0x7fc9cc105490 2024-03-28T18:06:59.221 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:59.222+0000 7fc9c3400640 1 --2- 172.21.15.22:0/2789982228 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7fc9b00a2880 0x7fc9b00a4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:06:59.222 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:59.222+0000 7fc9c3400640 1 -- 172.21.15.22:0/2789982228 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 8153+0+0 (secure 0 0 0) 0x7fc9c40f0f50 con 0x7fc9cc105490 2024-03-28T18:06:59.222 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:59.222+0000 7fc9caa00640 1 --2- 172.21.15.22:0/2789982228 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7fc9b00a2880 0x7fc9b00a4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:06:59.223 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:59.223+0000 7fc9caa00640 1 --2- 172.21.15.22:0/2789982228 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7fc9b00a2880 0x7fc9b00a4d40 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fc9cc104930 tx=0x7fc9bc009500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:06:59.224 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:59.225+0000 7fc9c3400640 1 -- 172.21.15.22:0/2789982228 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fc9c404d020 con 0x7fc9cc105490 2024-03-28T18:06:59.281 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:06:59 smithi022 ceph-mon[31985]: from='client.? 172.21.15.22:0/839745348' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-28T18:06:59.281 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:06:59 smithi022 ceph-mon[31985]: from='client.? 172.21.15.22:0/82780647' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-28T18:06:59.424 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:06:59 smithi134 ceph-mon[35037]: from='client.? 172.21.15.22:0/839745348' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-28T18:06:59.425 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:06:59 smithi134 ceph-mon[35037]: from='client.? 172.21.15.22:0/82780647' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-28T18:06:59.429 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:59.429+0000 7fc9d1a00640 1 -- 172.21.15.22:0/2789982228 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fc9cc02d220 con 0x7fc9cc105490 2024-03-28T18:06:59.430 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:59.430+0000 7fc9c3400640 1 -- 172.21.15.22:0/2789982228 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v92) v1 ==== 100+0+11 (secure 0 0 0) 0x7fc9c40be090 con 0x7fc9cc105490 2024-03-28T18:06:59.430 INFO:teuthology.orchestra.run.smithi022.stdout:pg_num: 64 2024-03-28T18:06:59.432 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:06:59 smithi079 ceph-mon[35312]: from='client.? 172.21.15.22:0/839745348' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-28T18:06:59.432 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:06:59 smithi079 ceph-mon[35312]: from='client.? 172.21.15.22:0/82780647' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-28T18:06:59.433 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:59.433+0000 7fc9d1a00640 1 -- 172.21.15.22:0/2789982228 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7fc9b00a2880 msgr2=0x7fc9b00a4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:06:59.433 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:59.433+0000 7fc9d1a00640 1 --2- 172.21.15.22:0/2789982228 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7fc9b00a2880 0x7fc9b00a4d40 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fc9cc104930 tx=0x7fc9bc009500 comp rx=0 tx=0).stop 2024-03-28T18:06:59.433 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:59.433+0000 7fc9d1a00640 1 -- 172.21.15.22:0/2789982228 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc9cc105490 msgr2=0x7fc9cc1a1120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:06:59.433 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:59.433+0000 7fc9d1a00640 1 --2- 172.21.15.22:0/2789982228 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc9cc105490 0x7fc9cc1a1120 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7fc9c4036040 tx=0x7fc9c4005de0 comp rx=0 tx=0).stop 2024-03-28T18:06:59.433 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:59.433+0000 7fc9d1a00640 1 -- 172.21.15.22:0/2789982228 shutdown_connections 2024-03-28T18:06:59.433 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:59.433+0000 7fc9d1a00640 1 --2- 172.21.15.22:0/2789982228 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7fc9b00a2880 0x7fc9b00a4d40 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:06:59.433 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:59.433+0000 7fc9d1a00640 1 --2- 172.21.15.22:0/2789982228 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc9cc105490 0x7fc9cc1a1120 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:06:59.434 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:59.433+0000 7fc9d1a00640 1 --2- 172.21.15.22:0/2789982228 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc9cc104ad0 0x7fc9cc19a0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:06:59.434 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:59.433+0000 7fc9d1a00640 1 --2- 172.21.15.22:0/2789982228 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc9cc1038e0 0x7fc9cc199b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:06:59.434 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:59.433+0000 7fc9d1a00640 1 -- 172.21.15.22:0/2789982228 >> 172.21.15.22:0/2789982228 conn(0x7fc9cc0fee50 msgr2=0x7fc9cc107dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:06:59.434 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:59.433+0000 7fc9d1a00640 1 -- 172.21.15.22:0/2789982228 shutdown_connections 2024-03-28T18:06:59.434 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:59.434+0000 7fc9d1a00640 1 -- 172.21.15.22:0/2789982228 wait complete. 2024-03-28T18:06:59.451 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-28T18:06:59.673 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:59.673+0000 7f0414c00640 1 -- 172.21.15.22:0/421354163 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0410109890 msgr2=0x7f0410110120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:06:59.673 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:59.673+0000 7f0414c00640 1 --2- 172.21.15.22:0/421354163 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0410109890 0x7f0410110120 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f0408009ca0 tx=0x7f040802c100 comp rx=0 tx=0).stop 2024-03-28T18:06:59.673 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:59.673+0000 7f0414c00640 1 -- 172.21.15.22:0/421354163 shutdown_connections 2024-03-28T18:06:59.673 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:59.673+0000 7f0414c00640 1 --2- 172.21.15.22:0/421354163 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0410109890 0x7f0410110120 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:06:59.673 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:59.673+0000 7f0414c00640 1 --2- 172.21.15.22:0/421354163 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f0410108ed0 0x7f0410109350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:06:59.673 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:59.673+0000 7f0414c00640 1 --2- 172.21.15.22:0/421354163 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0410107ce0 0x7f0410108100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:06:59.673 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:59.673+0000 7f0414c00640 1 -- 172.21.15.22:0/421354163 >> 172.21.15.22:0/421354163 conn(0x7f0410076cb0 msgr2=0x7f04100790d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:06:59.673 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:59.673+0000 7f0414c00640 1 -- 172.21.15.22:0/421354163 shutdown_connections 2024-03-28T18:06:59.674 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:59.673+0000 7f0414c00640 1 -- 172.21.15.22:0/421354163 wait complete. 2024-03-28T18:06:59.676 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:59.674+0000 7f0414c00640 1 Processor -- start 2024-03-28T18:06:59.676 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:59.674+0000 7f0414c00640 1 -- start start 2024-03-28T18:06:59.676 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:59.674+0000 7f0414c00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0410107ce0 0x7f041019df80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:06:59.676 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:59.674+0000 7f0414c00640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f0410108ed0 0x7f041019e4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:06:59.676 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:59.674+0000 7f0414c00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0410109890 0x7f04101a5530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:06:59.676 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:59.674+0000 7f0414c00640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0410113940 con 0x7f0410109890 2024-03-28T18:06:59.676 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:59.674+0000 7f0414c00640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04101137c0 con 0x7f0410108ed0 2024-03-28T18:06:59.676 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:59.674+0000 7f0414c00640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0410113ac0 con 0x7f0410107ce0 2024-03-28T18:06:59.677 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:59.675+0000 7f040e800640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0410109890 0x7f04101a5530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:06:59.677 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:59.675+0000 7f040e800640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0410109890 0x7f04101a5530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:39380/0 (socket says 172.21.15.22:39380) 2024-03-28T18:06:59.677 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:59.675+0000 7f040e800640 1 -- 172.21.15.22:0/3346212494 learned_addr learned my addr 172.21.15.22:0/3346212494 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-28T18:06:59.677 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:59.675+0000 7f040d400640 1 --2- 172.21.15.22:0/3346212494 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f0410108ed0 0x7f041019e4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:06:59.677 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:59.675+0000 7f040de00640 1 --2- 172.21.15.22:0/3346212494 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0410107ce0 0x7f041019df80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:06:59.677 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:59.675+0000 7f040e800640 1 -- 172.21.15.22:0/3346212494 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0410107ce0 msgr2=0x7f041019df80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:06:59.677 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:59.675+0000 7f040e800640 1 --2- 172.21.15.22:0/3346212494 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0410107ce0 0x7f041019df80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:06:59.677 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:59.675+0000 7f040e800640 1 -- 172.21.15.22:0/3346212494 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f0410108ed0 msgr2=0x7f041019e4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:06:59.677 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:59.675+0000 7f040e800640 1 --2- 172.21.15.22:0/3346212494 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f0410108ed0 0x7f041019e4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:06:59.677 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:59.675+0000 7f040e800640 1 -- 172.21.15.22:0/3346212494 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f04101a5ba0 con 0x7f0410109890 2024-03-28T18:06:59.677 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:59.675+0000 7f040d400640 1 --2- 172.21.15.22:0/3346212494 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f0410108ed0 0x7f041019e4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-28T18:06:59.677 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:59.676+0000 7f040e800640 1 --2- 172.21.15.22:0/3346212494 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0410109890 0x7f04101a5530 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f0408037040 tx=0x7f0408034d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:06:59.677 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:59.676+0000 7f0406a00640 1 -- 172.21.15.22:0/3346212494 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0408039050 con 0x7f0410109890 2024-03-28T18:06:59.677 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:59.676+0000 7f0406a00640 1 -- 172.21.15.22:0/3346212494 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f0408045070 con 0x7f0410109890 2024-03-28T18:06:59.678 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:59.676+0000 7f0414c00640 1 -- 172.21.15.22:0/3346212494 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f04101a5e30 con 0x7f0410109890 2024-03-28T18:06:59.678 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:59.676+0000 7f0406a00640 1 -- 172.21.15.22:0/3346212494 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0408004620 con 0x7f0410109890 2024-03-28T18:06:59.678 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:59.676+0000 7f0414c00640 1 -- 172.21.15.22:0/3346212494 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f04101a6290 con 0x7f0410109890 2024-03-28T18:06:59.678 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:59.677+0000 7f0414c00640 1 -- 172.21.15.22:0/3346212494 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f041010f000 con 0x7f0410109890 2024-03-28T18:06:59.679 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:59.679+0000 7f0406a00640 1 -- 172.21.15.22:0/3346212494 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 18) v1 ==== 137290+0+0 (secure 0 0 0) 0x7f0408003970 con 0x7f0410109890 2024-03-28T18:06:59.680 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:59.681+0000 7f0406a00640 1 --2- 172.21.15.22:0/3346212494 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7f03f40a2800 0x7f03f40a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:06:59.680 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:59.681+0000 7f0406a00640 1 -- 172.21.15.22:0/3346212494 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 8153+0+0 (secure 0 0 0) 0x7f04080f21a0 con 0x7f0410109890 2024-03-28T18:06:59.681 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:59.681+0000 7f040de00640 1 --2- 172.21.15.22:0/3346212494 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7f03f40a2800 0x7f03f40a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:06:59.681 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:59.682+0000 7f040de00640 1 --2- 172.21.15.22:0/3346212494 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7f03f40a2800 0x7f03f40a4cc0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f04000058c0 tx=0x7f040000a4c0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:06:59.683 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:59.683+0000 7f0406a00640 1 -- 172.21.15.22:0/3346212494 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f04080bf240 con 0x7f0410109890 2024-03-28T18:06:59.928 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:59.929+0000 7f0414c00640 1 -- 172.21.15.22:0/3346212494 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f041002d070 con 0x7f0410109890 2024-03-28T18:06:59.929 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:59.930+0000 7f0406a00640 1 -- 172.21.15.22:0/3346212494 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) v1 ==== 94+0+11452 (secure 0 0 0) 0x7f04080beb40 con 0x7f0410109890 2024-03-28T18:06:59.930 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-03-28T18:06:59.930 INFO:teuthology.orchestra.run.smithi022.stdout:{"epoch":26,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14544,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6835/3718238950","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6834","nonce":3718238950},{"type":"v1","addr":"172.21.15.22:6835","nonce":3718238950}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14562,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6837/3676939386","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6836","nonce":3676939386},{"type":"v1","addr":"172.21.15.22:6837","nonce":3676939386}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":26,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-28T18:04:48.991622+0000","modified":"2024-03-28T18:04:56.643864+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":34553,"mds_1":34511,"mds_2":34475,"mds_3":34442,"mds_4":24502},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14640":{"gid":14640,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.22:6839/3979998310","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6838","nonce":3979998310},{"type":"v1","addr":"172.21.15.22:6839","nonce":3979998310}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_14655":{"gid":14655,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.22:6841/3069056661","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6840","nonce":3069056661},{"type":"v1","addr":"172.21.15.22:6841","nonce":3069056661}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24412":{"gid":24412,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.134:6833/3176125362","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6832","nonce":3176125362},{"type":"v1","addr":"172.21.15.134:6833","nonce":3176125362}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24463":{"gid":24463,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.134:6835/3370549207","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6834","nonce":3370549207},{"type":"v1","addr":"172.21.15.134:6835","nonce":3370549207}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24478":{"gid":24478,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.134:6837/899825581","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6836","nonce":899825581},{"type":"v1","addr":"172.21.15.134:6837","nonce":899825581}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24502":{"gid":24502,"name":"i","rank":4,"incarnation":24,"state":"up:active","state_seq":7,"addr":"172.21.15.134:6839/3400263217","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6838","nonce":3400263217},{"type":"v1","addr":"172.21.15.134:6839","nonce":3400263217}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_34442":{"gid":34442,"name":"k","rank":3,"incarnation":22,"state":"up:active","state_seq":14,"addr":"172.21.15.79:6833/856896997","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6832","nonce":856896997},{"type":"v1","addr":"172.21.15.79:6833","nonce":856896997}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_34475":{"gid":34475,"name":"e","rank":2,"incarnation":20,"state":"up:active","state_seq":11,"addr":"172.21.15.79:6835/3365773431","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6834","nonce":3365773431},{"type":"v1","addr":"172.21.15.79:6835","nonce":3365773431}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_34511":{"gid":34511,"name":"h","rank":1,"incarnation":18,"state":"up:active","state_seq":8,"addr":"172.21.15.79:6837/1787903366","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6836","nonce":1787903366},{"type":"v1","addr":"172.21.15.79:6837","nonce":1787903366}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_34553":{"gid":34553,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.79:6839/2441542443","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6838","nonce":2441542443},{"type":"v1","addr":"172.21.15.79:6839","nonce":2441542443}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":34553,"qdb_cluster":[34511,34553,34475,34442,24502]},"id":1}]} 2024-03-28T18:06:59.930 INFO:teuthology.orchestra.run.smithi022.stderr:dumped fsmap epoch 26 2024-03-28T18:06:59.932 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:59.933+0000 7f0414c00640 1 -- 172.21.15.22:0/3346212494 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7f03f40a2800 msgr2=0x7f03f40a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:06:59.933 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:59.933+0000 7f0414c00640 1 --2- 172.21.15.22:0/3346212494 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7f03f40a2800 0x7f03f40a4cc0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f04000058c0 tx=0x7f040000a4c0 comp rx=0 tx=0).stop 2024-03-28T18:06:59.933 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:59.933+0000 7f0414c00640 1 -- 172.21.15.22:0/3346212494 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0410109890 msgr2=0x7f04101a5530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:06:59.933 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:59.933+0000 7f0414c00640 1 --2- 172.21.15.22:0/3346212494 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0410109890 0x7f04101a5530 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f0408037040 tx=0x7f0408034d50 comp rx=0 tx=0).stop 2024-03-28T18:06:59.933 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:59.933+0000 7f0414c00640 1 -- 172.21.15.22:0/3346212494 shutdown_connections 2024-03-28T18:06:59.933 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:59.933+0000 7f0414c00640 1 --2- 172.21.15.22:0/3346212494 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7f03f40a2800 0x7f03f40a4cc0 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:06:59.933 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:59.933+0000 7f0414c00640 1 --2- 172.21.15.22:0/3346212494 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0410109890 0x7f04101a5530 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:06:59.933 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:59.933+0000 7f0414c00640 1 --2- 172.21.15.22:0/3346212494 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f0410108ed0 0x7f041019e4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:06:59.933 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:59.933+0000 7f0414c00640 1 --2- 172.21.15.22:0/3346212494 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0410107ce0 0x7f041019df80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:06:59.933 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:59.933+0000 7f0414c00640 1 -- 172.21.15.22:0/3346212494 >> 172.21.15.22:0/3346212494 conn(0x7f0410076cb0 msgr2=0x7f041010c1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:06:59.933 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:59.933+0000 7f0414c00640 1 -- 172.21.15.22:0/3346212494 shutdown_connections 2024-03-28T18:06:59.933 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:06:59.933+0000 7f0414c00640 1 -- 172.21.15.22:0/3346212494 wait complete. 2024-03-28T18:06:59.953 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-28T18:07:00.179 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:00.179+0000 7f8617000640 1 -- 172.21.15.22:0/588433006 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8610105490 msgr2=0x7f861010bd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:07:00.179 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:00.179+0000 7f8617000640 1 --2- 172.21.15.22:0/588433006 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8610105490 0x7f861010bd20 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f8608009ca0 tx=0x7f860802c100 comp rx=0 tx=0).stop 2024-03-28T18:07:00.179 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:00.179+0000 7f8617000640 1 -- 172.21.15.22:0/588433006 shutdown_connections 2024-03-28T18:07:00.180 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:00.179+0000 7f8617000640 1 --2- 172.21.15.22:0/588433006 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8610105490 0x7f861010bd20 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:07:00.180 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:00.179+0000 7f8617000640 1 --2- 172.21.15.22:0/588433006 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f8610104ad0 0x7f8610104f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:07:00.180 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:00.179+0000 7f8617000640 1 --2- 172.21.15.22:0/588433006 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f86101038e0 0x7f8610103d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:07:00.180 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:00.179+0000 7f8617000640 1 -- 172.21.15.22:0/588433006 >> 172.21.15.22:0/588433006 conn(0x7f86100fee50 msgr2=0x7f8610101290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:07:00.180 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:00.180+0000 7f8617000640 1 -- 172.21.15.22:0/588433006 shutdown_connections 2024-03-28T18:07:00.180 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:00.180+0000 7f8617000640 1 -- 172.21.15.22:0/588433006 wait complete. 2024-03-28T18:07:00.182 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:00.180+0000 7f8617000640 1 Processor -- start 2024-03-28T18:07:00.182 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:00.181+0000 7f8617000640 1 -- start start 2024-03-28T18:07:00.182 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:00.181+0000 7f8617000640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f86101038e0 0x7f8610199ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:07:00.182 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:00.181+0000 7f8617000640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f8610104ad0 0x7f861019a0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:07:00.182 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:00.181+0000 7f8617000640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8610105490 0x7f86101a1150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:07:00.183 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:00.181+0000 7f8617000640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f861010f5c0 con 0x7f86101038e0 2024-03-28T18:07:00.183 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:00.181+0000 7f8617000640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f861010f440 con 0x7f8610104ad0 2024-03-28T18:07:00.183 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:00.181+0000 7f8617000640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f861010f740 con 0x7f8610105490 2024-03-28T18:07:00.183 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:00.181+0000 7f860fe00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f86101038e0 0x7f8610199ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:07:00.183 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:00.181+0000 7f860fe00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f86101038e0 0x7f8610199ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:39402/0 (socket says 172.21.15.22:39402) 2024-03-28T18:07:00.183 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:00.181+0000 7f860fe00640 1 -- 172.21.15.22:0/1099092586 learned_addr learned my addr 172.21.15.22:0/1099092586 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-28T18:07:00.183 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:00.181+0000 7f8615000640 1 --2- 172.21.15.22:0/1099092586 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8610105490 0x7f86101a1150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:07:00.183 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:00.181+0000 7f860f400640 1 --2- 172.21.15.22:0/1099092586 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f8610104ad0 0x7f861019a0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:07:00.183 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:00.182+0000 7f860fe00640 1 -- 172.21.15.22:0/1099092586 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8610105490 msgr2=0x7f86101a1150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:07:00.183 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:00.182+0000 7f860fe00640 1 --2- 172.21.15.22:0/1099092586 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8610105490 0x7f86101a1150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:07:00.183 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:00.182+0000 7f860fe00640 1 -- 172.21.15.22:0/1099092586 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f8610104ad0 msgr2=0x7f861019a0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:07:00.183 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:00.182+0000 7f860fe00640 1 --2- 172.21.15.22:0/1099092586 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f8610104ad0 0x7f861019a0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:07:00.183 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:00.182+0000 7f860fe00640 1 -- 172.21.15.22:0/1099092586 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f86101a17c0 con 0x7f86101038e0 2024-03-28T18:07:00.183 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:00.182+0000 7f860f400640 1 --2- 172.21.15.22:0/1099092586 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f8610104ad0 0x7f861019a0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T18:07:00.183 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:00.182+0000 7f860fe00640 1 --2- 172.21.15.22:0/1099092586 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f86101038e0 0x7f8610199ba0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f860000b450 tx=0x7f860000b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:07:00.184 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:00.182+0000 7f860cc00640 1 -- 172.21.15.22:0/1099092586 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8600015020 con 0x7f86101038e0 2024-03-28T18:07:00.184 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:00.182+0000 7f860cc00640 1 -- 172.21.15.22:0/1099092586 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f8600011d60 con 0x7f86101038e0 2024-03-28T18:07:00.184 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:00.182+0000 7f8617000640 1 -- 172.21.15.22:0/1099092586 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f86101a1ab0 con 0x7f86101038e0 2024-03-28T18:07:00.184 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:00.182+0000 7f860cc00640 1 -- 172.21.15.22:0/1099092586 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f860000f340 con 0x7f86101038e0 2024-03-28T18:07:00.184 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:00.182+0000 7f8617000640 1 -- 172.21.15.22:0/1099092586 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f86101a2050 con 0x7f86101038e0 2024-03-28T18:07:00.184 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:00.183+0000 7f8617000640 1 -- 172.21.15.22:0/1099092586 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f861010aba0 con 0x7f86101038e0 2024-03-28T18:07:00.185 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:00.185+0000 7f860cc00640 1 -- 172.21.15.22:0/1099092586 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 18) v1 ==== 137290+0+0 (secure 0 0 0) 0x7f8600018020 con 0x7f86101038e0 2024-03-28T18:07:00.186 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:00.187+0000 7f860cc00640 1 --2- 172.21.15.22:0/1099092586 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7f85f40a2880 0x7f85f40a4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:07:00.186 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:00.187+0000 7f860cc00640 1 -- 172.21.15.22:0/1099092586 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 8153+0+0 (secure 0 0 0) 0x7f86000ce590 con 0x7f86101038e0 2024-03-28T18:07:00.187 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:00.187+0000 7f860f400640 1 --2- 172.21.15.22:0/1099092586 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7f85f40a2880 0x7f85f40a4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:07:00.187 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:00.188+0000 7f860f400640 1 --2- 172.21.15.22:0/1099092586 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7f85f40a2880 0x7f85f40a4d40 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f861019b0b0 tx=0x7f85fc005d90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:07:00.190 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:00.191+0000 7f860cc00640 1 -- 172.21.15.22:0/1099092586 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f860001f020 con 0x7f86101038e0 2024-03-28T18:07:00.281 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:07:00 smithi022 ceph-mon[31985]: pgmap v436: 97 pgs: 97 active+clean; 643 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-03-28T18:07:00.282 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:07:00 smithi022 ceph-mon[31985]: from='client.? 172.21.15.22:0/2789982228' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-28T18:07:00.282 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:07:00 smithi022 ceph-mon[31985]: from='client.? 172.21.15.22:0/3346212494' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-28T18:07:00.396 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:00.397+0000 7f8617000640 1 -- 172.21.15.22:0/1099092586 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f86100623a0 con 0x7f86101038e0 2024-03-28T18:07:00.398 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:00.399+0000 7f860cc00640 1 -- 172.21.15.22:0/1099092586 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v92) v1 ==== 74+0+22522 (secure 0 0 0) 0x7f860000d040 con 0x7f86101038e0 2024-03-28T18:07:00.399 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-03-28T18:07:00.399 INFO:teuthology.orchestra.run.smithi022.stdout:{"epoch":92,"fsid":"fd220748-ed2b-11ee-95d0-87774f69a715","created":"2024-03-28T17:54:00.440279+0000","modified":"2024-03-28T18:05:11.725341+0000","last_up_change":"2024-03-28T18:03:43.220095+0000","last_in_change":"2024-03-28T18:03:18.526399+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-28T17:58:33.168872+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-28T18:04:47.004797+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-28T18:04:48.465502+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"90","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"5d708e06-f05b-40ba-bbcf-258ddc4f43d7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6802","nonce":2186290846},{"type":"v1","addr":"172.21.15.22:6803","nonce":2186290846}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6804","nonce":2186290846},{"type":"v1","addr":"172.21.15.22:6805","nonce":2186290846}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6808","nonce":2186290846},{"type":"v1","addr":"172.21.15.22:6809","nonce":2186290846}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6806","nonce":2186290846},{"type":"v1","addr":"172.21.15.22:6807","nonce":2186290846}]},"public_addr":"172.21.15.22:6803/2186290846","cluster_addr":"172.21.15.22:6805/2186290846","heartbeat_back_addr":"172.21.15.22:6809/2186290846","heartbeat_front_addr":"172.21.15.22:6807/2186290846","state":["exists","up"]},{"osd":1,"uuid":"68ad09bb-9256-409a-9867-4e14a2cb94aa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6800","nonce":2835549681},{"type":"v1","addr":"172.21.15.79:6801","nonce":2835549681}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6802","nonce":2835549681},{"type":"v1","addr":"172.21.15.79:6803","nonce":2835549681}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6806","nonce":2835549681},{"type":"v1","addr":"172.21.15.79:6807","nonce":2835549681}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6804","nonce":2835549681},{"type":"v1","addr":"172.21.15.79:6805","nonce":2835549681}]},"public_addr":"172.21.15.79:6801/2835549681","cluster_addr":"172.21.15.79:6803/2835549681","heartbeat_back_addr":"172.21.15.79:6807/2835549681","heartbeat_front_addr":"172.21.15.79:6805/2835549681","state":["exists","up"]},{"osd":2,"uuid":"ed11a5a7-004d-41f3-a5fc-eb2d603bd33f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6800","nonce":470588085},{"type":"v1","addr":"172.21.15.134:6801","nonce":470588085}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6802","nonce":470588085},{"type":"v1","addr":"172.21.15.134:6803","nonce":470588085}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6806","nonce":470588085},{"type":"v1","addr":"172.21.15.134:6807","nonce":470588085}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6804","nonce":470588085},{"type":"v1","addr":"172.21.15.134:6805","nonce":470588085}]},"public_addr":"172.21.15.134:6801/470588085","cluster_addr":"172.21.15.134:6803/470588085","heartbeat_back_addr":"172.21.15.134:6807/470588085","heartbeat_front_addr":"172.21.15.134:6805/470588085","state":["exists","up"]},{"osd":3,"uuid":"81a6c547-2a50-44c3-84d7-23ab6831790c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6810","nonce":1702441854},{"type":"v1","addr":"172.21.15.22:6811","nonce":1702441854}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6812","nonce":1702441854},{"type":"v1","addr":"172.21.15.22:6813","nonce":1702441854}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6816","nonce":1702441854},{"type":"v1","addr":"172.21.15.22:6817","nonce":1702441854}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6814","nonce":1702441854},{"type":"v1","addr":"172.21.15.22:6815","nonce":1702441854}]},"public_addr":"172.21.15.22:6811/1702441854","cluster_addr":"172.21.15.22:6813/1702441854","heartbeat_back_addr":"172.21.15.22:6817/1702441854","heartbeat_front_addr":"172.21.15.22:6815/1702441854","state":["exists","up"]},{"osd":4,"uuid":"d23a254e-7182-4471-8bca-7eb684ec618e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6808","nonce":630132437},{"type":"v1","addr":"172.21.15.79:6809","nonce":630132437}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6810","nonce":630132437},{"type":"v1","addr":"172.21.15.79:6811","nonce":630132437}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6814","nonce":630132437},{"type":"v1","addr":"172.21.15.79:6815","nonce":630132437}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6812","nonce":630132437},{"type":"v1","addr":"172.21.15.79:6813","nonce":630132437}]},"public_addr":"172.21.15.79:6809/630132437","cluster_addr":"172.21.15.79:6811/630132437","heartbeat_back_addr":"172.21.15.79:6815/630132437","heartbeat_front_addr":"172.21.15.79:6813/630132437","state":["exists","up"]},{"osd":5,"uuid":"a7b98502-2023-4f0a-bfa0-6840cc495e0c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":40,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6808","nonce":2135406915},{"type":"v1","addr":"172.21.15.134:6809","nonce":2135406915}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6810","nonce":2135406915},{"type":"v1","addr":"172.21.15.134:6811","nonce":2135406915}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6814","nonce":2135406915},{"type":"v1","addr":"172.21.15.134:6815","nonce":2135406915}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6812","nonce":2135406915},{"type":"v1","addr":"172.21.15.134:6813","nonce":2135406915}]},"public_addr":"172.21.15.134:6809/2135406915","cluster_addr":"172.21.15.134:6811/2135406915","heartbeat_back_addr":"172.21.15.134:6815/2135406915","heartbeat_front_addr":"172.21.15.134:6813/2135406915","state":["exists","up"]},{"osd":6,"uuid":"634c6562-d683-41c8-8242-fc848c38ce1f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6818","nonce":1400680396},{"type":"v1","addr":"172.21.15.22:6819","nonce":1400680396}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6820","nonce":1400680396},{"type":"v1","addr":"172.21.15.22:6821","nonce":1400680396}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6824","nonce":1400680396},{"type":"v1","addr":"172.21.15.22:6825","nonce":1400680396}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6822","nonce":1400680396},{"type":"v1","addr":"172.21.15.22:6823","nonce":1400680396}]},"public_addr":"172.21.15.22:6819/1400680396","cluster_addr":"172.21.15.22:6821/1400680396","heartbeat_back_addr":"172.21.15.22:6825/1400680396","heartbeat_front_addr":"172.21.15.22:6823/1400680396","state":["exists","up"]},{"osd":7,"uuid":"504eb4a8-6731-45ad-932c-1b38ef157871","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":53,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6816","nonce":1287105314},{"type":"v1","addr":"172.21.15.79:6817","nonce":1287105314}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6818","nonce":1287105314},{"type":"v1","addr":"172.21.15.79:6819","nonce":1287105314}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6822","nonce":1287105314},{"type":"v1","addr":"172.21.15.79:6823","nonce":1287105314}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6820","nonce":1287105314},{"type":"v1","addr":"172.21.15.79:6821","nonce":1287105314}]},"public_addr":"172.21.15.79:6817/1287105314","cluster_addr":"172.21.15.79:6819/1287105314","heartbeat_back_addr":"172.21.15.79:6823/1287105314","heartbeat_front_addr":"172.21.15.79:6821/1287105314","state":["exists","up"]},{"osd":8,"uuid":"20aa66c1-03d6-4cb8-a096-0eae41979c17","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":62,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6816","nonce":2265650388},{"type":"v1","addr":"172.21.15.134:6817","nonce":2265650388}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6818","nonce":2265650388},{"type":"v1","addr":"172.21.15.134:6819","nonce":2265650388}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6822","nonce":2265650388},{"type":"v1","addr":"172.21.15.134:6823","nonce":2265650388}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6820","nonce":2265650388},{"type":"v1","addr":"172.21.15.134:6821","nonce":2265650388}]},"public_addr":"172.21.15.134:6817/2265650388","cluster_addr":"172.21.15.134:6819/2265650388","heartbeat_back_addr":"172.21.15.134:6823/2265650388","heartbeat_front_addr":"172.21.15.134:6821/2265650388","state":["exists","up"]},{"osd":9,"uuid":"e1ae4020-7c2c-4334-a753-ee6b12d98584","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":70,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6826","nonce":3762222259},{"type":"v1","addr":"172.21.15.22:6827","nonce":3762222259}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6828","nonce":3762222259},{"type":"v1","addr":"172.21.15.22:6829","nonce":3762222259}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6832","nonce":3762222259},{"type":"v1","addr":"172.21.15.22:6833","nonce":3762222259}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6830","nonce":3762222259},{"type":"v1","addr":"172.21.15.22:6831","nonce":3762222259}]},"public_addr":"172.21.15.22:6827/3762222259","cluster_addr":"172.21.15.22:6829/3762222259","heartbeat_back_addr":"172.21.15.22:6833/3762222259","heartbeat_front_addr":"172.21.15.22:6831/3762222259","state":["exists","up"]},{"osd":10,"uuid":"f5198d52-df48-4679-a860-4dcb440b7a9f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":77,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6824","nonce":1746167904},{"type":"v1","addr":"172.21.15.79:6825","nonce":1746167904}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6826","nonce":1746167904},{"type":"v1","addr":"172.21.15.79:6827","nonce":1746167904}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6830","nonce":1746167904},{"type":"v1","addr":"172.21.15.79:6831","nonce":1746167904}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6828","nonce":1746167904},{"type":"v1","addr":"172.21.15.79:6829","nonce":1746167904}]},"public_addr":"172.21.15.79:6825/1746167904","cluster_addr":"172.21.15.79:6827/1746167904","heartbeat_back_addr":"172.21.15.79:6831/1746167904","heartbeat_front_addr":"172.21.15.79:6829/1746167904","state":["exists","up"]},{"osd":11,"uuid":"1427bca7-3414-4468-9246-e999156bc2ae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":85,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6824","nonce":4271716428},{"type":"v1","addr":"172.21.15.134:6825","nonce":4271716428}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6826","nonce":4271716428},{"type":"v1","addr":"172.21.15.134:6827","nonce":4271716428}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6830","nonce":4271716428},{"type":"v1","addr":"172.21.15.134:6831","nonce":4271716428}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6828","nonce":4271716428},{"type":"v1","addr":"172.21.15.134:6829","nonce":4271716428}]},"public_addr":"172.21.15.134:6825/4271716428","cluster_addr":"172.21.15.134:6827/4271716428","heartbeat_back_addr":"172.21.15.134:6831/4271716428","heartbeat_front_addr":"172.21.15.134:6829/4271716428","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T17:57:21.225125+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T17:57:55.197254+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T17:58:27.825136+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T17:59:04.319893+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T17:59:38.407752+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T18:00:12.118805+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T18:00:47.635214+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T18:01:21.231055+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T18:01:55.817861+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T18:02:31.680160+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T18:03:06.783398+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T18:03:41.307347+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.8","mappings":[{"from":2,"to":11}]},{"pgid":"2.1d","mappings":[{"from":8,"to":11}]},{"pgid":"3.3","mappings":[{"from":4,"to":9}]},{"pgid":"3.c","mappings":[{"from":3,"to":9}]},{"pgid":"3.21","mappings":[{"from":4,"to":8}]},{"pgid":"3.25","mappings":[{"from":5,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.22:0/3521924283":"2024-03-29T17:55:10.425504+0000","172.21.15.22:6801/730635939":"2024-03-29T17:55:10.425504+0000","172.21.15.22:0/2471229459":"2024-03-29T17:54:34.004559+0000","172.21.15.22:6800/3166828081":"2024-03-29T17:54:34.004559+0000","172.21.15.22:6800/730635939":"2024-03-29T17:55:10.425504+0000","172.21.15.22:0/1229047849":"2024-03-29T17:54:34.004559+0000","172.21.15.22:6801/3166828081":"2024-03-29T17:54:34.004559+0000","172.21.15.22:0/2717818856":"2024-03-29T17:55:10.425504+0000","172.21.15.22:0/2157570333":"2024-03-29T17:54:34.004559+0000","172.21.15.22:0/1006589530":"2024-03-29T17:55:10.425504+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-28T18:07:00.402 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:00.402+0000 7f8617000640 1 -- 172.21.15.22:0/1099092586 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7f85f40a2880 msgr2=0x7f85f40a4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:07:00.402 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:00.402+0000 7f8617000640 1 --2- 172.21.15.22:0/1099092586 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7f85f40a2880 0x7f85f40a4d40 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f861019b0b0 tx=0x7f85fc005d90 comp rx=0 tx=0).stop 2024-03-28T18:07:00.402 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:00.402+0000 7f8617000640 1 -- 172.21.15.22:0/1099092586 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f86101038e0 msgr2=0x7f8610199ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:07:00.402 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:00.402+0000 7f8617000640 1 --2- 172.21.15.22:0/1099092586 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f86101038e0 0x7f8610199ba0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f860000b450 tx=0x7f860000b910 comp rx=0 tx=0).stop 2024-03-28T18:07:00.402 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:00.402+0000 7f8617000640 1 -- 172.21.15.22:0/1099092586 shutdown_connections 2024-03-28T18:07:00.402 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:00.402+0000 7f8617000640 1 --2- 172.21.15.22:0/1099092586 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7f85f40a2880 0x7f85f40a4d40 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:07:00.402 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:00.402+0000 7f8617000640 1 --2- 172.21.15.22:0/1099092586 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8610105490 0x7f86101a1150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:07:00.402 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:00.402+0000 7f8617000640 1 --2- 172.21.15.22:0/1099092586 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f8610104ad0 0x7f861019a0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:07:00.402 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:00.402+0000 7f8617000640 1 --2- 172.21.15.22:0/1099092586 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f86101038e0 0x7f8610199ba0 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:07:00.402 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:00.402+0000 7f8617000640 1 -- 172.21.15.22:0/1099092586 >> 172.21.15.22:0/1099092586 conn(0x7f86100fee50 msgr2=0x7f8610107d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:07:00.402 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:00.402+0000 7f8617000640 1 -- 172.21.15.22:0/1099092586 shutdown_connections 2024-03-28T18:07:00.403 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:00.402+0000 7f8617000640 1 -- 172.21.15.22:0/1099092586 wait complete. 2024-03-28T18:07:00.421 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-28T17:58:33.168872+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-28T18:04:47.004797+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '89', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-03-28T18:04:48.465502+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '90', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-28T18:07:00.422 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-28T18:07:00.424 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:07:00 smithi134 ceph-mon[35037]: pgmap v436: 97 pgs: 97 active+clean; 643 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-03-28T18:07:00.424 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:07:00 smithi134 ceph-mon[35037]: from='client.? 172.21.15.22:0/2789982228' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-28T18:07:00.424 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:07:00 smithi134 ceph-mon[35037]: from='client.? 172.21.15.22:0/3346212494' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-28T18:07:00.432 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:07:00 smithi079 ceph-mon[35312]: pgmap v436: 97 pgs: 97 active+clean; 643 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-03-28T18:07:00.432 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:07:00 smithi079 ceph-mon[35312]: from='client.? 172.21.15.22:0/2789982228' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-28T18:07:00.432 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:07:00 smithi079 ceph-mon[35312]: from='client.? 172.21.15.22:0/3346212494' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-28T18:07:00.649 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:00.648+0000 7f35f7600640 1 -- 172.21.15.22:0/1287492548 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f35f0076060 msgr2=0x7f35f0076480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:07:00.649 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:00.648+0000 7f35f7600640 1 --2- 172.21.15.22:0/1287492548 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f35f0076060 0x7f35f0076480 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f35e0009ca0 tx=0x7f35e002c0f0 comp rx=0 tx=0).stop 2024-03-28T18:07:00.649 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:00.649+0000 7f35f7600640 1 -- 172.21.15.22:0/1287492548 shutdown_connections 2024-03-28T18:07:00.649 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:00.649+0000 7f35f7600640 1 --2- 172.21.15.22:0/1287492548 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f35f0077c40 0x7f35f00784e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:07:00.649 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:00.649+0000 7f35f7600640 1 --2- 172.21.15.22:0/1287492548 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f35f00772a0 0x7f35f0077700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:07:00.649 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:00.649+0000 7f35f7600640 1 --2- 172.21.15.22:0/1287492548 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f35f0076060 0x7f35f0076480 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:07:00.649 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:00.649+0000 7f35f7600640 1 -- 172.21.15.22:0/1287492548 >> 172.21.15.22:0/1287492548 conn(0x7f35f00fee50 msgr2=0x7f35f0101290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:07:00.649 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:00.649+0000 7f35f7600640 1 -- 172.21.15.22:0/1287492548 shutdown_connections 2024-03-28T18:07:00.649 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:00.649+0000 7f35f7600640 1 -- 172.21.15.22:0/1287492548 wait complete. 2024-03-28T18:07:00.652 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:00.650+0000 7f35f7600640 1 Processor -- start 2024-03-28T18:07:00.652 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:00.650+0000 7f35f7600640 1 -- start start 2024-03-28T18:07:00.652 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:00.650+0000 7f35f7600640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f35f0076060 0x7f35f019e010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:07:00.652 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:00.650+0000 7f35f7600640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f35f00772a0 0x7f35f019e550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:07:00.652 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:00.650+0000 7f35f7600640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f35f0077c40 0x7f35f01a55c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:07:00.652 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:00.650+0000 7f35f7600640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35f0113940 con 0x7f35f0076060 2024-03-28T18:07:00.652 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:00.650+0000 7f35f7600640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35f01137c0 con 0x7f35f0077c40 2024-03-28T18:07:00.652 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:00.650+0000 7f35f7600640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35f0113ac0 con 0x7f35f00772a0 2024-03-28T18:07:00.652 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:00.650+0000 7f35f4c00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f35f0076060 0x7f35f019e010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:07:00.653 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:00.650+0000 7f35f4c00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f35f0076060 0x7f35f019e010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:39412/0 (socket says 172.21.15.22:39412) 2024-03-28T18:07:00.653 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:00.650+0000 7f35f4c00640 1 -- 172.21.15.22:0/1133814155 learned_addr learned my addr 172.21.15.22:0/1133814155 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-28T18:07:00.653 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:00.651+0000 7f35f5600640 1 --2- 172.21.15.22:0/1133814155 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f35f0077c40 0x7f35f01a55c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:07:00.653 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:00.651+0000 7f35e7e00640 1 --2- 172.21.15.22:0/1133814155 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f35f00772a0 0x7f35f019e550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:07:00.653 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:00.651+0000 7f35f4c00640 1 -- 172.21.15.22:0/1133814155 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f35f00772a0 msgr2=0x7f35f019e550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:07:00.653 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:00.651+0000 7f35f4c00640 1 --2- 172.21.15.22:0/1133814155 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f35f00772a0 0x7f35f019e550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:07:00.653 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:00.651+0000 7f35f4c00640 1 -- 172.21.15.22:0/1133814155 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f35f0077c40 msgr2=0x7f35f01a55c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:07:00.653 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:00.651+0000 7f35f4c00640 1 --2- 172.21.15.22:0/1133814155 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f35f0077c40 0x7f35f01a55c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:07:00.653 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:00.651+0000 7f35f4c00640 1 -- 172.21.15.22:0/1133814155 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f35f01a5c30 con 0x7f35f0076060 2024-03-28T18:07:00.653 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:00.651+0000 7f35f4c00640 1 --2- 172.21.15.22:0/1133814155 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f35f0076060 0x7f35f019e010 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f35e0000f80 tx=0x7f35e0032900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:07:00.653 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:00.651+0000 7f35ee000640 1 -- 172.21.15.22:0/1133814155 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35e0039050 con 0x7f35f0076060 2024-03-28T18:07:00.653 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:00.651+0000 7f35ee000640 1 -- 172.21.15.22:0/1133814155 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f35e003a040 con 0x7f35f0076060 2024-03-28T18:07:00.653 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:00.652+0000 7f35f7600640 1 -- 172.21.15.22:0/1133814155 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f35f01a5ec0 con 0x7f35f0076060 2024-03-28T18:07:00.653 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:00.652+0000 7f35ee000640 1 -- 172.21.15.22:0/1133814155 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35e0040450 con 0x7f35f0076060 2024-03-28T18:07:00.653 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:00.652+0000 7f35f7600640 1 -- 172.21.15.22:0/1133814155 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f35f01a6320 con 0x7f35f0076060 2024-03-28T18:07:00.654 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:00.653+0000 7f35f7600640 1 -- 172.21.15.22:0/1133814155 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f35f01104e0 con 0x7f35f0076060 2024-03-28T18:07:00.654 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:00.655+0000 7f35ee000640 1 -- 172.21.15.22:0/1133814155 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 18) v1 ==== 137290+0+0 (secure 0 0 0) 0x7f35e0003b70 con 0x7f35f0076060 2024-03-28T18:07:00.656 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:00.656+0000 7f35ee000640 1 --2- 172.21.15.22:0/1133814155 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7f35d40a2880 0x7f35d40a4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:07:00.656 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:00.656+0000 7f35ee000640 1 -- 172.21.15.22:0/1133814155 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 8153+0+0 (secure 0 0 0) 0x7f35e00f1430 con 0x7f35f0076060 2024-03-28T18:07:00.656 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:00.657+0000 7f35e7e00640 1 --2- 172.21.15.22:0/1133814155 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7f35d40a2880 0x7f35d40a4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:07:00.657 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:00.657+0000 7f35e7e00640 1 --2- 172.21.15.22:0/1133814155 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7f35d40a2880 0x7f35d40a4d40 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f35f019f520 tx=0x7f35d8005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:07:00.660 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:00.660+0000 7f35ee000640 1 -- 172.21.15.22:0/1133814155 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f35e00be4d0 con 0x7f35f0076060 2024-03-28T18:07:00.864 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:00.864+0000 7f35f7600640 1 -- 172.21.15.22:0/1133814155 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f35f00623e0 con 0x7f35f0076060 2024-03-28T18:07:00.865 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:00.865+0000 7f35ee000640 1 -- 172.21.15.22:0/1133814155 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v92) v1 ==== 93+0+10 (secure 0 0 0) 0x7f35e00098b0 con 0x7f35f0076060 2024-03-28T18:07:00.865 INFO:teuthology.orchestra.run.smithi022.stdout:pg_num: 1 2024-03-28T18:07:00.868 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:00.868+0000 7f35f7600640 1 -- 172.21.15.22:0/1133814155 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7f35d40a2880 msgr2=0x7f35d40a4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:07:00.868 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:00.868+0000 7f35f7600640 1 --2- 172.21.15.22:0/1133814155 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7f35d40a2880 0x7f35d40a4d40 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f35f019f520 tx=0x7f35d8005f90 comp rx=0 tx=0).stop 2024-03-28T18:07:00.868 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:00.868+0000 7f35f7600640 1 -- 172.21.15.22:0/1133814155 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f35f0076060 msgr2=0x7f35f019e010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:07:00.868 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:00.868+0000 7f35f7600640 1 --2- 172.21.15.22:0/1133814155 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f35f0076060 0x7f35f019e010 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f35e0000f80 tx=0x7f35e0032900 comp rx=0 tx=0).stop 2024-03-28T18:07:00.869 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:00.868+0000 7f35f7600640 1 -- 172.21.15.22:0/1133814155 shutdown_connections 2024-03-28T18:07:00.869 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:00.868+0000 7f35f7600640 1 --2- 172.21.15.22:0/1133814155 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7f35d40a2880 0x7f35d40a4d40 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:07:00.869 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:00.868+0000 7f35f7600640 1 --2- 172.21.15.22:0/1133814155 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f35f0077c40 0x7f35f01a55c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:07:00.869 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:00.868+0000 7f35f7600640 1 --2- 172.21.15.22:0/1133814155 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f35f00772a0 0x7f35f019e550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:07:00.869 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:00.868+0000 7f35f7600640 1 --2- 172.21.15.22:0/1133814155 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f35f0076060 0x7f35f019e010 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:07:00.869 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:00.868+0000 7f35f7600640 1 -- 172.21.15.22:0/1133814155 >> 172.21.15.22:0/1133814155 conn(0x7f35f00fee50 msgr2=0x7f35f01005c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:07:00.869 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:00.869+0000 7f35f7600640 1 -- 172.21.15.22:0/1133814155 shutdown_connections 2024-03-28T18:07:00.869 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:00.869+0000 7f35f7600640 1 -- 172.21.15.22:0/1133814155 wait complete. 2024-03-28T18:07:00.889 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-28T18:07:01.116 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:01.116+0000 7ff637e00640 1 -- 172.21.15.22:0/3254089292 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff638107260 msgr2=0x7ff6381096b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:07:01.116 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:01.116+0000 7ff637e00640 1 --2- 172.21.15.22:0/3254089292 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff638107260 0x7ff6381096b0 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7ff630009ca0 tx=0x7ff63002c100 comp rx=0 tx=0).stop 2024-03-28T18:07:01.117 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:01.116+0000 7ff637e00640 1 -- 172.21.15.22:0/3254089292 shutdown_connections 2024-03-28T18:07:01.117 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:01.116+0000 7ff637e00640 1 --2- 172.21.15.22:0/3254089292 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff638107260 0x7ff6381096b0 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:07:01.117 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:01.116+0000 7ff637e00640 1 --2- 172.21.15.22:0/3254089292 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff638104930 0x7ff638106d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:07:01.117 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:01.116+0000 7ff637e00640 1 --2- 172.21.15.22:0/3254089292 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ff638102000 0x7ff6381043f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:07:01.117 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:01.116+0000 7ff637e00640 1 -- 172.21.15.22:0/3254089292 >> 172.21.15.22:0/3254089292 conn(0x7ff6380fba10 msgr2=0x7ff6380fde30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:07:01.117 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:01.116+0000 7ff637e00640 1 -- 172.21.15.22:0/3254089292 shutdown_connections 2024-03-28T18:07:01.117 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:01.117+0000 7ff637e00640 1 -- 172.21.15.22:0/3254089292 wait complete. 2024-03-28T18:07:01.119 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:01.117+0000 7ff637e00640 1 Processor -- start 2024-03-28T18:07:01.119 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:01.118+0000 7ff637e00640 1 -- start start 2024-03-28T18:07:01.119 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:01.118+0000 7ff637e00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff638102000 0x7ff638199a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:07:01.119 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:01.118+0000 7ff637e00640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ff638104930 0x7ff638199fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:07:01.119 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:01.118+0000 7ff637e00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff638107260 0x7ff6381a1040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:07:01.119 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:01.118+0000 7ff637e00640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff63810d200 con 0x7ff638107260 2024-03-28T18:07:01.119 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:01.118+0000 7ff637e00640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff63810d080 con 0x7ff638104930 2024-03-28T18:07:01.120 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:01.118+0000 7ff637e00640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff63810d380 con 0x7ff638102000 2024-03-28T18:07:01.120 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:01.118+0000 7ff637400640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff638107260 0x7ff6381a1040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:07:01.120 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:01.118+0000 7ff637400640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff638107260 0x7ff6381a1040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:39436/0 (socket says 172.21.15.22:39436) 2024-03-28T18:07:01.120 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:01.118+0000 7ff637400640 1 -- 172.21.15.22:0/2957115777 learned_addr learned my addr 172.21.15.22:0/2957115777 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-28T18:07:01.120 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:01.118+0000 7ff636000640 1 --2- 172.21.15.22:0/2957115777 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ff638104930 0x7ff638199fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:07:01.120 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:01.118+0000 7ff636a00640 1 --2- 172.21.15.22:0/2957115777 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff638102000 0x7ff638199a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:07:01.120 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:01.119+0000 7ff637400640 1 -- 172.21.15.22:0/2957115777 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff638102000 msgr2=0x7ff638199a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:07:01.120 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:01.119+0000 7ff637400640 1 --2- 172.21.15.22:0/2957115777 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff638102000 0x7ff638199a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:07:01.120 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:01.119+0000 7ff637400640 1 -- 172.21.15.22:0/2957115777 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ff638104930 msgr2=0x7ff638199fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:07:01.120 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:01.119+0000 7ff637400640 1 --2- 172.21.15.22:0/2957115777 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ff638104930 0x7ff638199fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:07:01.120 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:01.119+0000 7ff637400640 1 -- 172.21.15.22:0/2957115777 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff6381a16b0 con 0x7ff638107260 2024-03-28T18:07:01.120 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:01.119+0000 7ff637400640 1 --2- 172.21.15.22:0/2957115777 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff638107260 0x7ff6381a1040 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7ff630009bf0 tx=0x7ff630034cd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:07:01.120 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:01.119+0000 7ff62f400640 1 -- 172.21.15.22:0/2957115777 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff630039050 con 0x7ff638107260 2024-03-28T18:07:01.120 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:01.119+0000 7ff62f400640 1 -- 172.21.15.22:0/2957115777 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7ff63003a040 con 0x7ff638107260 2024-03-28T18:07:01.121 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:01.119+0000 7ff62f400640 1 -- 172.21.15.22:0/2957115777 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6300046a0 con 0x7ff638107260 2024-03-28T18:07:01.121 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:01.119+0000 7ff637e00640 1 -- 172.21.15.22:0/2957115777 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff6381a1940 con 0x7ff638107260 2024-03-28T18:07:01.121 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:01.119+0000 7ff637e00640 1 -- 172.21.15.22:0/2957115777 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff6381a1de0 con 0x7ff638107260 2024-03-28T18:07:01.121 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:01.121+0000 7ff637e00640 1 -- 172.21.15.22:0/2957115777 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff638108d50 con 0x7ff638107260 2024-03-28T18:07:01.122 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:01.123+0000 7ff62f400640 1 -- 172.21.15.22:0/2957115777 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 18) v1 ==== 137290+0+0 (secure 0 0 0) 0x7ff630003970 con 0x7ff638107260 2024-03-28T18:07:01.124 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:01.124+0000 7ff62f400640 1 --2- 172.21.15.22:0/2957115777 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7ff61c0a2880 0x7ff61c0a4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:07:01.124 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:01.124+0000 7ff62f400640 1 -- 172.21.15.22:0/2957115777 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 8153+0+0 (secure 0 0 0) 0x7ff6300f1290 con 0x7ff638107260 2024-03-28T18:07:01.124 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:01.124+0000 7ff636a00640 1 --2- 172.21.15.22:0/2957115777 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7ff61c0a2880 0x7ff61c0a4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:07:01.125 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:01.125+0000 7ff636a00640 1 --2- 172.21.15.22:0/2957115777 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7ff61c0a2880 0x7ff61c0a4d40 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7ff6280095f0 tx=0x7ff628005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:07:01.128 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:01.128+0000 7ff62f400640 1 -- 172.21.15.22:0/2957115777 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7ff6300be3d0 con 0x7ff638107260 2024-03-28T18:07:01.333 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:01.333+0000 7ff637e00640 1 -- 172.21.15.22:0/2957115777 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7ff63802d2c0 con 0x7ff638107260 2024-03-28T18:07:01.334 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:01.334+0000 7ff62f400640 1 -- 172.21.15.22:0/2957115777 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v92) v1 ==== 104+0+11 (secure 0 0 0) 0x7ff6300098b0 con 0x7ff638107260 2024-03-28T18:07:01.334 INFO:teuthology.orchestra.run.smithi022.stdout:pg_num: 32 2024-03-28T18:07:01.337 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:01.337+0000 7ff637e00640 1 -- 172.21.15.22:0/2957115777 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7ff61c0a2880 msgr2=0x7ff61c0a4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:07:01.337 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:01.337+0000 7ff637e00640 1 --2- 172.21.15.22:0/2957115777 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7ff61c0a2880 0x7ff61c0a4d40 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7ff6280095f0 tx=0x7ff628005f90 comp rx=0 tx=0).stop 2024-03-28T18:07:01.337 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:01.337+0000 7ff637e00640 1 -- 172.21.15.22:0/2957115777 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff638107260 msgr2=0x7ff6381a1040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:07:01.337 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:01.337+0000 7ff637e00640 1 --2- 172.21.15.22:0/2957115777 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff638107260 0x7ff6381a1040 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7ff630009bf0 tx=0x7ff630034cd0 comp rx=0 tx=0).stop 2024-03-28T18:07:01.337 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:01.337+0000 7ff637e00640 1 -- 172.21.15.22:0/2957115777 shutdown_connections 2024-03-28T18:07:01.337 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:01.337+0000 7ff637e00640 1 --2- 172.21.15.22:0/2957115777 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7ff61c0a2880 0x7ff61c0a4d40 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:07:01.337 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:01.337+0000 7ff637e00640 1 --2- 172.21.15.22:0/2957115777 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff638107260 0x7ff6381a1040 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:07:01.337 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:01.337+0000 7ff637e00640 1 --2- 172.21.15.22:0/2957115777 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ff638104930 0x7ff638199fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:07:01.337 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:01.337+0000 7ff637e00640 1 --2- 172.21.15.22:0/2957115777 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff638102000 0x7ff638199a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:07:01.337 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:01.337+0000 7ff637e00640 1 -- 172.21.15.22:0/2957115777 >> 172.21.15.22:0/2957115777 conn(0x7ff6380fba10 msgr2=0x7ff638101730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:07:01.337 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:01.337+0000 7ff637e00640 1 -- 172.21.15.22:0/2957115777 shutdown_connections 2024-03-28T18:07:01.337 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:01.337+0000 7ff637e00640 1 -- 172.21.15.22:0/2957115777 wait complete. 2024-03-28T18:07:01.355 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-28T18:07:01.436 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:07:01 smithi022 ceph-mon[31985]: from='client.? 172.21.15.22:0/1099092586' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-28T18:07:01.436 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:07:01 smithi022 ceph-mon[31985]: from='client.? 172.21.15.22:0/1133814155' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-28T18:07:01.583 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:01.583+0000 7f5048800640 1 -- 172.21.15.22:0/4259487776 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5040107cc0 msgr2=0x7f50401080e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:07:01.583 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:01.583+0000 7f5048800640 1 --2- 172.21.15.22:0/4259487776 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5040107cc0 0x7f50401080e0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f5034009900 tx=0x7f503402bc30 comp rx=0 tx=0).stop 2024-03-28T18:07:01.583 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:01.583+0000 7f5048800640 1 -- 172.21.15.22:0/4259487776 shutdown_connections 2024-03-28T18:07:01.583 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:01.583+0000 7f5048800640 1 --2- 172.21.15.22:0/4259487776 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5040109870 0x7f5040110100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:07:01.583 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:01.583+0000 7f5048800640 1 --2- 172.21.15.22:0/4259487776 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f5040108eb0 0x7f5040109330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:07:01.583 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:01.583+0000 7f5048800640 1 --2- 172.21.15.22:0/4259487776 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5040107cc0 0x7f50401080e0 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:07:01.583 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:01.583+0000 7f5048800640 1 -- 172.21.15.22:0/4259487776 >> 172.21.15.22:0/4259487776 conn(0x7f5040076c60 msgr2=0x7f50400790a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:07:01.584 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:01.583+0000 7f5048800640 1 -- 172.21.15.22:0/4259487776 shutdown_connections 2024-03-28T18:07:01.584 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:01.583+0000 7f5048800640 1 -- 172.21.15.22:0/4259487776 wait complete. 2024-03-28T18:07:01.586 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:01.584+0000 7f5048800640 1 Processor -- start 2024-03-28T18:07:01.586 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:01.584+0000 7f5048800640 1 -- start start 2024-03-28T18:07:01.586 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:01.585+0000 7f5048800640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5040107cc0 0x7f504019dff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:07:01.586 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:01.585+0000 7f5048800640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f5040108eb0 0x7f504019e530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:07:01.586 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:01.585+0000 7f5048800640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5040109870 0x7f50401a55a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:07:01.586 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:01.585+0000 7f5048800640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5040113920 con 0x7f5040107cc0 2024-03-28T18:07:01.586 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:01.585+0000 7f5048800640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50401137a0 con 0x7f5040108eb0 2024-03-28T18:07:01.586 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:01.585+0000 7f5048800640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5040113aa0 con 0x7f5040109870 2024-03-28T18:07:01.586 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:01.585+0000 7f5045e00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5040107cc0 0x7f504019dff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:07:01.586 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:01.585+0000 7f5045e00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5040107cc0 0x7f504019dff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:39448/0 (socket says 172.21.15.22:39448) 2024-03-28T18:07:01.587 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:01.585+0000 7f5045e00640 1 -- 172.21.15.22:0/602584698 learned_addr learned my addr 172.21.15.22:0/602584698 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-28T18:07:01.587 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:01.585+0000 7f5045400640 1 --2- 172.21.15.22:0/602584698 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f5040108eb0 0x7f504019e530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:07:01.587 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:01.585+0000 7f5046800640 1 --2- 172.21.15.22:0/602584698 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5040109870 0x7f50401a55a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:07:01.587 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:01.585+0000 7f5045e00640 1 -- 172.21.15.22:0/602584698 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5040109870 msgr2=0x7f50401a55a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:07:01.587 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:01.585+0000 7f5045e00640 1 --2- 172.21.15.22:0/602584698 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5040109870 0x7f50401a55a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:07:01.587 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:01.585+0000 7f5045e00640 1 -- 172.21.15.22:0/602584698 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f5040108eb0 msgr2=0x7f504019e530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:07:01.587 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:01.585+0000 7f5045e00640 1 --2- 172.21.15.22:0/602584698 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f5040108eb0 0x7f504019e530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:07:01.587 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:01.586+0000 7f5045e00640 1 -- 172.21.15.22:0/602584698 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f50401a5c10 con 0x7f5040107cc0 2024-03-28T18:07:01.587 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:01.586+0000 7f5046800640 1 --2- 172.21.15.22:0/602584698 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5040109870 0x7f50401a55a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T18:07:01.587 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:01.586+0000 7f5045e00640 1 --2- 172.21.15.22:0/602584698 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5040107cc0 0x7f504019dff0 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f5034005ab0 tx=0x7f5034004e90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:07:01.587 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:01.586+0000 7f503aa00640 1 -- 172.21.15.22:0/602584698 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5034043020 con 0x7f5040107cc0 2024-03-28T18:07:01.587 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:01.586+0000 7f503aa00640 1 -- 172.21.15.22:0/602584698 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f50340054f0 con 0x7f5040107cc0 2024-03-28T18:07:01.587 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:01.586+0000 7f503aa00640 1 -- 172.21.15.22:0/602584698 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5034003d10 con 0x7f5040107cc0 2024-03-28T18:07:01.587 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:01.586+0000 7f5048800640 1 -- 172.21.15.22:0/602584698 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f50401a5ea0 con 0x7f5040107cc0 2024-03-28T18:07:01.587 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:01.586+0000 7f5048800640 1 -- 172.21.15.22:0/602584698 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f50401a62b0 con 0x7f5040107cc0 2024-03-28T18:07:01.588 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:01.588+0000 7f5048800640 1 -- 172.21.15.22:0/602584698 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f504010f060 con 0x7f5040107cc0 2024-03-28T18:07:01.589 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:01.590+0000 7f503aa00640 1 -- 172.21.15.22:0/602584698 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 18) v1 ==== 137290+0+0 (secure 0 0 0) 0x7f5034046020 con 0x7f5040107cc0 2024-03-28T18:07:01.591 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:01.591+0000 7f503aa00640 1 --2- 172.21.15.22:0/602584698 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7f50280a2880 0x7f50280a4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:07:01.591 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:01.591+0000 7f503aa00640 1 -- 172.21.15.22:0/602584698 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 8153+0+0 (secure 0 0 0) 0x7f50340f0750 con 0x7f5040107cc0 2024-03-28T18:07:01.591 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:01.591+0000 7f5045400640 1 --2- 172.21.15.22:0/602584698 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7f50280a2880 0x7f50280a4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:07:01.591 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:01.592+0000 7f5045400640 1 --2- 172.21.15.22:0/602584698 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7f50280a2880 0x7f50280a4d40 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f504019f500 tx=0x7f5030009870 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:07:01.594 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:01.595+0000 7f503aa00640 1 -- 172.21.15.22:0/602584698 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f503404d070 con 0x7f5040107cc0 2024-03-28T18:07:01.674 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:07:01 smithi134 ceph-mon[35037]: from='client.? 172.21.15.22:0/1099092586' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-28T18:07:01.674 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:07:01 smithi134 ceph-mon[35037]: from='client.? 172.21.15.22:0/1133814155' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-28T18:07:01.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:07:01 smithi079 ceph-mon[35312]: from='client.? 172.21.15.22:0/1099092586' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-28T18:07:01.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:07:01 smithi079 ceph-mon[35312]: from='client.? 172.21.15.22:0/1133814155' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-28T18:07:01.799 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:01.799+0000 7f5048800640 1 -- 172.21.15.22:0/602584698 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f504002d300 con 0x7f5040107cc0 2024-03-28T18:07:01.799 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:01.800+0000 7f503aa00640 1 -- 172.21.15.22:0/602584698 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v92) v1 ==== 100+0+11 (secure 0 0 0) 0x7f50340bd890 con 0x7f5040107cc0 2024-03-28T18:07:01.800 INFO:teuthology.orchestra.run.smithi022.stdout:pg_num: 64 2024-03-28T18:07:01.802 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:01.803+0000 7f5048800640 1 -- 172.21.15.22:0/602584698 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7f50280a2880 msgr2=0x7f50280a4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:07:01.802 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:01.803+0000 7f5048800640 1 --2- 172.21.15.22:0/602584698 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7f50280a2880 0x7f50280a4d40 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f504019f500 tx=0x7f5030009870 comp rx=0 tx=0).stop 2024-03-28T18:07:01.803 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:01.803+0000 7f5048800640 1 -- 172.21.15.22:0/602584698 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5040107cc0 msgr2=0x7f504019dff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:07:01.803 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:01.803+0000 7f5048800640 1 --2- 172.21.15.22:0/602584698 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5040107cc0 0x7f504019dff0 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f5034005ab0 tx=0x7f5034004e90 comp rx=0 tx=0).stop 2024-03-28T18:07:01.803 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:01.803+0000 7f5048800640 1 -- 172.21.15.22:0/602584698 shutdown_connections 2024-03-28T18:07:01.803 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:01.803+0000 7f5048800640 1 --2- 172.21.15.22:0/602584698 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7f50280a2880 0x7f50280a4d40 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:07:01.803 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:01.803+0000 7f5048800640 1 --2- 172.21.15.22:0/602584698 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5040109870 0x7f50401a55a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:07:01.803 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:01.803+0000 7f5048800640 1 --2- 172.21.15.22:0/602584698 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f5040108eb0 0x7f504019e530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:07:01.803 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:01.803+0000 7f5048800640 1 --2- 172.21.15.22:0/602584698 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5040107cc0 0x7f504019dff0 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:07:01.803 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:01.803+0000 7f5048800640 1 -- 172.21.15.22:0/602584698 >> 172.21.15.22:0/602584698 conn(0x7f5040076c60 msgr2=0x7f504010c230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:07:01.804 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:01.803+0000 7f5048800640 1 -- 172.21.15.22:0/602584698 shutdown_connections 2024-03-28T18:07:01.804 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:01.803+0000 7f5048800640 1 -- 172.21.15.22:0/602584698 wait complete. 2024-03-28T18:07:01.821 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-28T18:07:02.042 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:02.042+0000 7f60e8c00640 1 -- 172.21.15.22:0/4122986268 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f60e41026f0 msgr2=0x7f60e4102b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:07:02.042 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:02.042+0000 7f60e8c00640 1 --2- 172.21.15.22:0/4122986268 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f60e41026f0 0x7f60e4102b70 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f60d4009900 tx=0x7f60d402bc40 comp rx=0 tx=0).stop 2024-03-28T18:07:02.042 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:02.042+0000 7f60e8c00640 1 -- 172.21.15.22:0/4122986268 shutdown_connections 2024-03-28T18:07:02.042 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:02.042+0000 7f60e8c00640 1 --2- 172.21.15.22:0/4122986268 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f60e41030b0 0x7f60e4109940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:07:02.042 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:02.042+0000 7f60e8c00640 1 --2- 172.21.15.22:0/4122986268 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f60e41026f0 0x7f60e4102b70 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:07:02.042 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:02.042+0000 7f60e8c00640 1 --2- 172.21.15.22:0/4122986268 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f60e4101500 0x7f60e4101920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:07:02.042 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:02.042+0000 7f60e8c00640 1 -- 172.21.15.22:0/4122986268 >> 172.21.15.22:0/4122986268 conn(0x7f60e40fca70 msgr2=0x7f60e40feeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:07:02.042 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:02.042+0000 7f60e8c00640 1 -- 172.21.15.22:0/4122986268 shutdown_connections 2024-03-28T18:07:02.043 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:02.042+0000 7f60e8c00640 1 -- 172.21.15.22:0/4122986268 wait complete. 2024-03-28T18:07:02.044 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:02.043+0000 7f60e8c00640 1 Processor -- start 2024-03-28T18:07:02.044 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:02.043+0000 7f60e8c00640 1 -- start start 2024-03-28T18:07:02.044 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:02.044+0000 7f60e8c00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f60e4101500 0x7f60e4075450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:07:02.044 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:02.044+0000 7f60e8c00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f60e41026f0 0x7f60e4073a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:07:02.045 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:02.044+0000 7f60e8c00640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f60e41030b0 0x7f60e4073fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:07:02.045 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:02.044+0000 7f60e8c00640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60e410d160 con 0x7f60e4101500 2024-03-28T18:07:02.045 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:02.044+0000 7f60e8c00640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60e410cfe0 con 0x7f60e41030b0 2024-03-28T18:07:02.045 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:02.044+0000 7f60e8c00640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60e410d2e0 con 0x7f60e41026f0 2024-03-28T18:07:02.045 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:02.044+0000 7f60e3400640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f60e4101500 0x7f60e4075450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:07:02.045 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:02.044+0000 7f60e3400640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f60e4101500 0x7f60e4075450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:39468/0 (socket says 172.21.15.22:39468) 2024-03-28T18:07:02.045 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:02.044+0000 7f60e3400640 1 -- 172.21.15.22:0/1930933532 learned_addr learned my addr 172.21.15.22:0/1930933532 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-28T18:07:02.045 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:02.044+0000 7f60e3e00640 1 --2- 172.21.15.22:0/1930933532 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f60e41030b0 0x7f60e4073fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:07:02.045 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:02.044+0000 7f60e2a00640 1 --2- 172.21.15.22:0/1930933532 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f60e41026f0 0x7f60e4073a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:07:02.045 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:02.044+0000 7f60e3400640 1 -- 172.21.15.22:0/1930933532 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f60e41026f0 msgr2=0x7f60e4073a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:07:02.045 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:02.044+0000 7f60e3400640 1 --2- 172.21.15.22:0/1930933532 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f60e41026f0 0x7f60e4073a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:07:02.045 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:02.044+0000 7f60e3400640 1 -- 172.21.15.22:0/1930933532 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f60e41030b0 msgr2=0x7f60e4073fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:07:02.045 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:02.044+0000 7f60e3400640 1 --2- 172.21.15.22:0/1930933532 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f60e41030b0 0x7f60e4073fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:07:02.045 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:02.045+0000 7f60e3400640 1 -- 172.21.15.22:0/1930933532 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f60e4074860 con 0x7f60e4101500 2024-03-28T18:07:02.045 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:02.045+0000 7f60e3400640 1 --2- 172.21.15.22:0/1930933532 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f60e4101500 0x7f60e4075450 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f60cc00d770 tx=0x7f60cc00dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:07:02.046 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:02.045+0000 7f60dbe00640 1 -- 172.21.15.22:0/1930933532 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60cc017070 con 0x7f60e4101500 2024-03-28T18:07:02.046 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:02.045+0000 7f60dbe00640 1 -- 172.21.15.22:0/1930933532 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f60cc009c30 con 0x7f60e4101500 2024-03-28T18:07:02.046 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:02.045+0000 7f60dbe00640 1 -- 172.21.15.22:0/1930933532 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60cc003e10 con 0x7f60e4101500 2024-03-28T18:07:02.046 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:02.045+0000 7f60e8c00640 1 -- 172.21.15.22:0/1930933532 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f60e41a1930 con 0x7f60e4101500 2024-03-28T18:07:02.047 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:02.045+0000 7f60e8c00640 1 -- 172.21.15.22:0/1930933532 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f60e41a1e30 con 0x7f60e4101500 2024-03-28T18:07:02.047 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:02.046+0000 7f60e8c00640 1 -- 172.21.15.22:0/1930933532 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f60e4108890 con 0x7f60e4101500 2024-03-28T18:07:02.048 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:02.048+0000 7f60dbe00640 1 -- 172.21.15.22:0/1930933532 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 18) v1 ==== 137290+0+0 (secure 0 0 0) 0x7f60cc004020 con 0x7f60e4101500 2024-03-28T18:07:02.049 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:02.050+0000 7f60dbe00640 1 --2- 172.21.15.22:0/1930933532 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7f60c80a2880 0x7f60c80a4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:07:02.049 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:02.050+0000 7f60dbe00640 1 -- 172.21.15.22:0/1930933532 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 8153+0+0 (secure 0 0 0) 0x7f60cc0ce830 con 0x7f60e4101500 2024-03-28T18:07:02.049 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:02.050+0000 7f60e2a00640 1 --2- 172.21.15.22:0/1930933532 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7f60c80a2880 0x7f60c80a4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:07:02.050 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:02.051+0000 7f60e2a00640 1 --2- 172.21.15.22:0/1930933532 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7f60c80a2880 0x7f60c80a4d40 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f60d402c720 tx=0x7f60d4005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:07:02.053 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:02.054+0000 7f60dbe00640 1 -- 172.21.15.22:0/1930933532 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f60cc09b8d0 con 0x7f60e4101500 2024-03-28T18:07:02.303 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:02.303+0000 7f60e8c00640 1 -- 172.21.15.22:0/1930933532 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f60e402d070 con 0x7f60e4101500 2024-03-28T18:07:02.304 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:02.305+0000 7f60dbe00640 1 -- 172.21.15.22:0/1930933532 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) v1 ==== 94+0+11452 (secure 0 0 0) 0x7f60cc09b1d0 con 0x7f60e4101500 2024-03-28T18:07:02.305 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-03-28T18:07:02.305 INFO:teuthology.orchestra.run.smithi022.stdout:{"epoch":26,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14544,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6835/3718238950","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6834","nonce":3718238950},{"type":"v1","addr":"172.21.15.22:6835","nonce":3718238950}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14562,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6837/3676939386","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6836","nonce":3676939386},{"type":"v1","addr":"172.21.15.22:6837","nonce":3676939386}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":26,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-28T18:04:48.991622+0000","modified":"2024-03-28T18:04:56.643864+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":34553,"mds_1":34511,"mds_2":34475,"mds_3":34442,"mds_4":24502},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14640":{"gid":14640,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.22:6839/3979998310","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6838","nonce":3979998310},{"type":"v1","addr":"172.21.15.22:6839","nonce":3979998310}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_14655":{"gid":14655,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.22:6841/3069056661","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6840","nonce":3069056661},{"type":"v1","addr":"172.21.15.22:6841","nonce":3069056661}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24412":{"gid":24412,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.134:6833/3176125362","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6832","nonce":3176125362},{"type":"v1","addr":"172.21.15.134:6833","nonce":3176125362}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24463":{"gid":24463,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.134:6835/3370549207","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6834","nonce":3370549207},{"type":"v1","addr":"172.21.15.134:6835","nonce":3370549207}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24478":{"gid":24478,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.134:6837/899825581","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6836","nonce":899825581},{"type":"v1","addr":"172.21.15.134:6837","nonce":899825581}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24502":{"gid":24502,"name":"i","rank":4,"incarnation":24,"state":"up:active","state_seq":7,"addr":"172.21.15.134:6839/3400263217","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6838","nonce":3400263217},{"type":"v1","addr":"172.21.15.134:6839","nonce":3400263217}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_34442":{"gid":34442,"name":"k","rank":3,"incarnation":22,"state":"up:active","state_seq":14,"addr":"172.21.15.79:6833/856896997","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6832","nonce":856896997},{"type":"v1","addr":"172.21.15.79:6833","nonce":856896997}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_34475":{"gid":34475,"name":"e","rank":2,"incarnation":20,"state":"up:active","state_seq":11,"addr":"172.21.15.79:6835/3365773431","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6834","nonce":3365773431},{"type":"v1","addr":"172.21.15.79:6835","nonce":3365773431}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_34511":{"gid":34511,"name":"h","rank":1,"incarnation":18,"state":"up:active","state_seq":8,"addr":"172.21.15.79:6837/1787903366","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6836","nonce":1787903366},{"type":"v1","addr":"172.21.15.79:6837","nonce":1787903366}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_34553":{"gid":34553,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.79:6839/2441542443","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6838","nonce":2441542443},{"type":"v1","addr":"172.21.15.79:6839","nonce":2441542443}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":34553,"qdb_cluster":[34511,34553,34475,34442,24502]},"id":1}]} 2024-03-28T18:07:02.305 INFO:teuthology.orchestra.run.smithi022.stderr:dumped fsmap epoch 26 2024-03-28T18:07:02.308 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:02.308+0000 7f60e8c00640 1 -- 172.21.15.22:0/1930933532 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7f60c80a2880 msgr2=0x7f60c80a4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:07:02.308 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:02.308+0000 7f60e8c00640 1 --2- 172.21.15.22:0/1930933532 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7f60c80a2880 0x7f60c80a4d40 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f60d402c720 tx=0x7f60d4005fd0 comp rx=0 tx=0).stop 2024-03-28T18:07:02.308 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:02.308+0000 7f60e8c00640 1 -- 172.21.15.22:0/1930933532 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f60e4101500 msgr2=0x7f60e4075450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:07:02.308 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:02.308+0000 7f60e8c00640 1 --2- 172.21.15.22:0/1930933532 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f60e4101500 0x7f60e4075450 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f60cc00d770 tx=0x7f60cc00dc30 comp rx=0 tx=0).stop 2024-03-28T18:07:02.308 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:02.308+0000 7f60e8c00640 1 -- 172.21.15.22:0/1930933532 shutdown_connections 2024-03-28T18:07:02.308 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:02.308+0000 7f60e8c00640 1 --2- 172.21.15.22:0/1930933532 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7f60c80a2880 0x7f60c80a4d40 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:07:02.308 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:02.308+0000 7f60e8c00640 1 --2- 172.21.15.22:0/1930933532 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f60e41030b0 0x7f60e4073fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:07:02.308 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:02.308+0000 7f60e8c00640 1 --2- 172.21.15.22:0/1930933532 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f60e41026f0 0x7f60e4073a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:07:02.308 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:02.308+0000 7f60e8c00640 1 --2- 172.21.15.22:0/1930933532 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f60e4101500 0x7f60e4075450 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:07:02.308 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:02.308+0000 7f60e8c00640 1 -- 172.21.15.22:0/1930933532 >> 172.21.15.22:0/1930933532 conn(0x7f60e40fca70 msgr2=0x7f60e4105a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:07:02.308 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:02.308+0000 7f60e8c00640 1 -- 172.21.15.22:0/1930933532 shutdown_connections 2024-03-28T18:07:02.308 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:02.308+0000 7f60e8c00640 1 -- 172.21.15.22:0/1930933532 wait complete. 2024-03-28T18:07:02.329 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-28T18:07:02.531 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:07:02 smithi022 ceph-mon[31985]: pgmap v437: 97 pgs: 97 active+clean; 643 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-03-28T18:07:02.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:07:02 smithi022 ceph-mon[31985]: from='client.? 172.21.15.22:0/2957115777' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-28T18:07:02.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:07:02 smithi022 ceph-mon[31985]: from='client.? 172.21.15.22:0/602584698' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-28T18:07:02.552 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:02.552+0000 7fe663000640 1 -- 172.21.15.22:0/2616353248 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe65c0756e0 msgr2=0x7fe65c073b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:07:02.552 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:02.552+0000 7fe663000640 1 --2- 172.21.15.22:0/2616353248 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe65c0756e0 0x7fe65c073b30 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7fe650009930 tx=0x7fe65002bc30 comp rx=0 tx=0).stop 2024-03-28T18:07:02.552 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:02.552+0000 7fe663000640 1 -- 172.21.15.22:0/2616353248 shutdown_connections 2024-03-28T18:07:02.552 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:02.552+0000 7fe663000640 1 --2- 172.21.15.22:0/2616353248 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe65c10dab0 0x7fe65c10fea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:07:02.552 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:02.552+0000 7fe663000640 1 --2- 172.21.15.22:0/2616353248 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fe65c074100 0x7fe65c074580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:07:02.552 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:02.552+0000 7fe663000640 1 --2- 172.21.15.22:0/2616353248 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe65c0756e0 0x7fe65c073b30 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:07:02.552 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:02.552+0000 7fe663000640 1 -- 172.21.15.22:0/2616353248 >> 172.21.15.22:0/2616353248 conn(0x7fe65c0fca30 msgr2=0x7fe65c0fee90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:07:02.552 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:02.552+0000 7fe663000640 1 -- 172.21.15.22:0/2616353248 shutdown_connections 2024-03-28T18:07:02.552 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:02.552+0000 7fe663000640 1 -- 172.21.15.22:0/2616353248 wait complete. 2024-03-28T18:07:02.553 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:02.553+0000 7fe663000640 1 Processor -- start 2024-03-28T18:07:02.553 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:02.554+0000 7fe663000640 1 -- start start 2024-03-28T18:07:02.555 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:02.554+0000 7fe663000640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe65c074100 0x7fe65c19de60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:07:02.555 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:02.554+0000 7fe663000640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fe65c0756e0 0x7fe65c19e3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:07:02.555 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:02.554+0000 7fe663000640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe65c10dab0 0x7fe65c1a5410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:07:02.555 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:02.554+0000 7fe663000640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe65c113790 con 0x7fe65c074100 2024-03-28T18:07:02.555 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:02.554+0000 7fe663000640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe65c113610 con 0x7fe65c0756e0 2024-03-28T18:07:02.555 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:02.554+0000 7fe663000640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe65c113910 con 0x7fe65c10dab0 2024-03-28T18:07:02.555 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:02.554+0000 7fe661c00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe65c074100 0x7fe65c19de60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:07:02.555 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:02.554+0000 7fe661c00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe65c074100 0x7fe65c19de60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:39486/0 (socket says 172.21.15.22:39486) 2024-03-28T18:07:02.555 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:02.554+0000 7fe661c00640 1 -- 172.21.15.22:0/3429492398 learned_addr learned my addr 172.21.15.22:0/3429492398 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-28T18:07:02.555 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:02.555+0000 7fe661200640 1 --2- 172.21.15.22:0/3429492398 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fe65c0756e0 0x7fe65c19e3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:07:02.555 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:02.555+0000 7fe662600640 1 --2- 172.21.15.22:0/3429492398 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe65c10dab0 0x7fe65c1a5410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:07:02.555 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:02.555+0000 7fe661c00640 1 -- 172.21.15.22:0/3429492398 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe65c10dab0 msgr2=0x7fe65c1a5410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:07:02.556 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:02.555+0000 7fe661c00640 1 --2- 172.21.15.22:0/3429492398 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe65c10dab0 0x7fe65c1a5410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:07:02.556 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:02.555+0000 7fe661c00640 1 -- 172.21.15.22:0/3429492398 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fe65c0756e0 msgr2=0x7fe65c19e3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:07:02.556 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:02.555+0000 7fe661c00640 1 --2- 172.21.15.22:0/3429492398 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fe65c0756e0 0x7fe65c19e3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:07:02.556 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:02.555+0000 7fe661c00640 1 -- 172.21.15.22:0/3429492398 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe65c1a5a80 con 0x7fe65c074100 2024-03-28T18:07:02.556 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:02.555+0000 7fe661c00640 1 --2- 172.21.15.22:0/3429492398 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe65c074100 0x7fe65c19de60 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7fe650000c00 tx=0x7fe650004e20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:07:02.557 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:02.556+0000 7fe65a000640 1 -- 172.21.15.22:0/3429492398 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe650043020 con 0x7fe65c074100 2024-03-28T18:07:02.557 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:02.556+0000 7fe65a000640 1 -- 172.21.15.22:0/3429492398 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7fe650005460 con 0x7fe65c074100 2024-03-28T18:07:02.557 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:02.556+0000 7fe663000640 1 -- 172.21.15.22:0/3429492398 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe65c1a5d10 con 0x7fe65c074100 2024-03-28T18:07:02.557 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:02.556+0000 7fe65a000640 1 -- 172.21.15.22:0/3429492398 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe650003d00 con 0x7fe65c074100 2024-03-28T18:07:02.557 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:02.556+0000 7fe663000640 1 -- 172.21.15.22:0/3429492398 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe65c1a61f0 con 0x7fe65c074100 2024-03-28T18:07:02.557 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:02.557+0000 7fe663000640 1 -- 172.21.15.22:0/3429492398 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe65c103c80 con 0x7fe65c074100 2024-03-28T18:07:02.558 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:02.559+0000 7fe65a000640 1 -- 172.21.15.22:0/3429492398 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 18) v1 ==== 137290+0+0 (secure 0 0 0) 0x7fe650046020 con 0x7fe65c074100 2024-03-28T18:07:02.560 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:02.560+0000 7fe65a000640 1 --2- 172.21.15.22:0/3429492398 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7fe6440a2880 0x7fe6440a4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:07:02.561 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:02.561+0000 7fe65a000640 1 -- 172.21.15.22:0/3429492398 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 8153+0+0 (secure 0 0 0) 0x7fe6500f0c70 con 0x7fe65c074100 2024-03-28T18:07:02.561 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:02.561+0000 7fe661200640 1 --2- 172.21.15.22:0/3429492398 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7fe6440a2880 0x7fe6440a4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:07:02.561 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:02.561+0000 7fe661200640 1 --2- 172.21.15.22:0/3429492398 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7fe6440a2880 0x7fe6440a4d40 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fe65c19f370 tx=0x7fe648005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:07:02.564 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:02.565+0000 7fe65a000640 1 -- 172.21.15.22:0/3429492398 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fe65004d020 con 0x7fe65c074100 2024-03-28T18:07:02.674 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:07:02 smithi134 ceph-mon[35037]: pgmap v437: 97 pgs: 97 active+clean; 643 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-03-28T18:07:02.674 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:07:02 smithi134 ceph-mon[35037]: from='client.? 172.21.15.22:0/2957115777' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-28T18:07:02.674 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:07:02 smithi134 ceph-mon[35037]: from='client.? 172.21.15.22:0/602584698' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-28T18:07:02.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:07:02 smithi079 ceph-mon[35312]: pgmap v437: 97 pgs: 97 active+clean; 643 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-03-28T18:07:02.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:07:02 smithi079 ceph-mon[35312]: from='client.? 172.21.15.22:0/2957115777' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-28T18:07:02.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:07:02 smithi079 ceph-mon[35312]: from='client.? 172.21.15.22:0/602584698' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-28T18:07:02.770 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:02.771+0000 7fe663000640 1 -- 172.21.15.22:0/3429492398 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fe65c0623a0 con 0x7fe65c074100 2024-03-28T18:07:02.773 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:02.774+0000 7fe65a000640 1 -- 172.21.15.22:0/3429492398 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v92) v1 ==== 74+0+22522 (secure 0 0 0) 0x7fe6500bdd10 con 0x7fe65c074100 2024-03-28T18:07:02.774 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-03-28T18:07:02.774 INFO:teuthology.orchestra.run.smithi022.stdout:{"epoch":92,"fsid":"fd220748-ed2b-11ee-95d0-87774f69a715","created":"2024-03-28T17:54:00.440279+0000","modified":"2024-03-28T18:05:11.725341+0000","last_up_change":"2024-03-28T18:03:43.220095+0000","last_in_change":"2024-03-28T18:03:18.526399+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-28T17:58:33.168872+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-28T18:04:47.004797+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-28T18:04:48.465502+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"90","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"5d708e06-f05b-40ba-bbcf-258ddc4f43d7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6802","nonce":2186290846},{"type":"v1","addr":"172.21.15.22:6803","nonce":2186290846}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6804","nonce":2186290846},{"type":"v1","addr":"172.21.15.22:6805","nonce":2186290846}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6808","nonce":2186290846},{"type":"v1","addr":"172.21.15.22:6809","nonce":2186290846}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6806","nonce":2186290846},{"type":"v1","addr":"172.21.15.22:6807","nonce":2186290846}]},"public_addr":"172.21.15.22:6803/2186290846","cluster_addr":"172.21.15.22:6805/2186290846","heartbeat_back_addr":"172.21.15.22:6809/2186290846","heartbeat_front_addr":"172.21.15.22:6807/2186290846","state":["exists","up"]},{"osd":1,"uuid":"68ad09bb-9256-409a-9867-4e14a2cb94aa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6800","nonce":2835549681},{"type":"v1","addr":"172.21.15.79:6801","nonce":2835549681}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6802","nonce":2835549681},{"type":"v1","addr":"172.21.15.79:6803","nonce":2835549681}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6806","nonce":2835549681},{"type":"v1","addr":"172.21.15.79:6807","nonce":2835549681}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6804","nonce":2835549681},{"type":"v1","addr":"172.21.15.79:6805","nonce":2835549681}]},"public_addr":"172.21.15.79:6801/2835549681","cluster_addr":"172.21.15.79:6803/2835549681","heartbeat_back_addr":"172.21.15.79:6807/2835549681","heartbeat_front_addr":"172.21.15.79:6805/2835549681","state":["exists","up"]},{"osd":2,"uuid":"ed11a5a7-004d-41f3-a5fc-eb2d603bd33f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6800","nonce":470588085},{"type":"v1","addr":"172.21.15.134:6801","nonce":470588085}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6802","nonce":470588085},{"type":"v1","addr":"172.21.15.134:6803","nonce":470588085}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6806","nonce":470588085},{"type":"v1","addr":"172.21.15.134:6807","nonce":470588085}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6804","nonce":470588085},{"type":"v1","addr":"172.21.15.134:6805","nonce":470588085}]},"public_addr":"172.21.15.134:6801/470588085","cluster_addr":"172.21.15.134:6803/470588085","heartbeat_back_addr":"172.21.15.134:6807/470588085","heartbeat_front_addr":"172.21.15.134:6805/470588085","state":["exists","up"]},{"osd":3,"uuid":"81a6c547-2a50-44c3-84d7-23ab6831790c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6810","nonce":1702441854},{"type":"v1","addr":"172.21.15.22:6811","nonce":1702441854}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6812","nonce":1702441854},{"type":"v1","addr":"172.21.15.22:6813","nonce":1702441854}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6816","nonce":1702441854},{"type":"v1","addr":"172.21.15.22:6817","nonce":1702441854}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6814","nonce":1702441854},{"type":"v1","addr":"172.21.15.22:6815","nonce":1702441854}]},"public_addr":"172.21.15.22:6811/1702441854","cluster_addr":"172.21.15.22:6813/1702441854","heartbeat_back_addr":"172.21.15.22:6817/1702441854","heartbeat_front_addr":"172.21.15.22:6815/1702441854","state":["exists","up"]},{"osd":4,"uuid":"d23a254e-7182-4471-8bca-7eb684ec618e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6808","nonce":630132437},{"type":"v1","addr":"172.21.15.79:6809","nonce":630132437}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6810","nonce":630132437},{"type":"v1","addr":"172.21.15.79:6811","nonce":630132437}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6814","nonce":630132437},{"type":"v1","addr":"172.21.15.79:6815","nonce":630132437}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6812","nonce":630132437},{"type":"v1","addr":"172.21.15.79:6813","nonce":630132437}]},"public_addr":"172.21.15.79:6809/630132437","cluster_addr":"172.21.15.79:6811/630132437","heartbeat_back_addr":"172.21.15.79:6815/630132437","heartbeat_front_addr":"172.21.15.79:6813/630132437","state":["exists","up"]},{"osd":5,"uuid":"a7b98502-2023-4f0a-bfa0-6840cc495e0c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":40,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6808","nonce":2135406915},{"type":"v1","addr":"172.21.15.134:6809","nonce":2135406915}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6810","nonce":2135406915},{"type":"v1","addr":"172.21.15.134:6811","nonce":2135406915}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6814","nonce":2135406915},{"type":"v1","addr":"172.21.15.134:6815","nonce":2135406915}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6812","nonce":2135406915},{"type":"v1","addr":"172.21.15.134:6813","nonce":2135406915}]},"public_addr":"172.21.15.134:6809/2135406915","cluster_addr":"172.21.15.134:6811/2135406915","heartbeat_back_addr":"172.21.15.134:6815/2135406915","heartbeat_front_addr":"172.21.15.134:6813/2135406915","state":["exists","up"]},{"osd":6,"uuid":"634c6562-d683-41c8-8242-fc848c38ce1f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6818","nonce":1400680396},{"type":"v1","addr":"172.21.15.22:6819","nonce":1400680396}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6820","nonce":1400680396},{"type":"v1","addr":"172.21.15.22:6821","nonce":1400680396}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6824","nonce":1400680396},{"type":"v1","addr":"172.21.15.22:6825","nonce":1400680396}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6822","nonce":1400680396},{"type":"v1","addr":"172.21.15.22:6823","nonce":1400680396}]},"public_addr":"172.21.15.22:6819/1400680396","cluster_addr":"172.21.15.22:6821/1400680396","heartbeat_back_addr":"172.21.15.22:6825/1400680396","heartbeat_front_addr":"172.21.15.22:6823/1400680396","state":["exists","up"]},{"osd":7,"uuid":"504eb4a8-6731-45ad-932c-1b38ef157871","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":53,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6816","nonce":1287105314},{"type":"v1","addr":"172.21.15.79:6817","nonce":1287105314}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6818","nonce":1287105314},{"type":"v1","addr":"172.21.15.79:6819","nonce":1287105314}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6822","nonce":1287105314},{"type":"v1","addr":"172.21.15.79:6823","nonce":1287105314}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6820","nonce":1287105314},{"type":"v1","addr":"172.21.15.79:6821","nonce":1287105314}]},"public_addr":"172.21.15.79:6817/1287105314","cluster_addr":"172.21.15.79:6819/1287105314","heartbeat_back_addr":"172.21.15.79:6823/1287105314","heartbeat_front_addr":"172.21.15.79:6821/1287105314","state":["exists","up"]},{"osd":8,"uuid":"20aa66c1-03d6-4cb8-a096-0eae41979c17","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":62,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6816","nonce":2265650388},{"type":"v1","addr":"172.21.15.134:6817","nonce":2265650388}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6818","nonce":2265650388},{"type":"v1","addr":"172.21.15.134:6819","nonce":2265650388}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6822","nonce":2265650388},{"type":"v1","addr":"172.21.15.134:6823","nonce":2265650388}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6820","nonce":2265650388},{"type":"v1","addr":"172.21.15.134:6821","nonce":2265650388}]},"public_addr":"172.21.15.134:6817/2265650388","cluster_addr":"172.21.15.134:6819/2265650388","heartbeat_back_addr":"172.21.15.134:6823/2265650388","heartbeat_front_addr":"172.21.15.134:6821/2265650388","state":["exists","up"]},{"osd":9,"uuid":"e1ae4020-7c2c-4334-a753-ee6b12d98584","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":70,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6826","nonce":3762222259},{"type":"v1","addr":"172.21.15.22:6827","nonce":3762222259}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6828","nonce":3762222259},{"type":"v1","addr":"172.21.15.22:6829","nonce":3762222259}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6832","nonce":3762222259},{"type":"v1","addr":"172.21.15.22:6833","nonce":3762222259}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6830","nonce":3762222259},{"type":"v1","addr":"172.21.15.22:6831","nonce":3762222259}]},"public_addr":"172.21.15.22:6827/3762222259","cluster_addr":"172.21.15.22:6829/3762222259","heartbeat_back_addr":"172.21.15.22:6833/3762222259","heartbeat_front_addr":"172.21.15.22:6831/3762222259","state":["exists","up"]},{"osd":10,"uuid":"f5198d52-df48-4679-a860-4dcb440b7a9f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":77,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6824","nonce":1746167904},{"type":"v1","addr":"172.21.15.79:6825","nonce":1746167904}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6826","nonce":1746167904},{"type":"v1","addr":"172.21.15.79:6827","nonce":1746167904}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6830","nonce":1746167904},{"type":"v1","addr":"172.21.15.79:6831","nonce":1746167904}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6828","nonce":1746167904},{"type":"v1","addr":"172.21.15.79:6829","nonce":1746167904}]},"public_addr":"172.21.15.79:6825/1746167904","cluster_addr":"172.21.15.79:6827/1746167904","heartbeat_back_addr":"172.21.15.79:6831/1746167904","heartbeat_front_addr":"172.21.15.79:6829/1746167904","state":["exists","up"]},{"osd":11,"uuid":"1427bca7-3414-4468-9246-e999156bc2ae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":85,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6824","nonce":4271716428},{"type":"v1","addr":"172.21.15.134:6825","nonce":4271716428}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6826","nonce":4271716428},{"type":"v1","addr":"172.21.15.134:6827","nonce":4271716428}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6830","nonce":4271716428},{"type":"v1","addr":"172.21.15.134:6831","nonce":4271716428}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6828","nonce":4271716428},{"type":"v1","addr":"172.21.15.134:6829","nonce":4271716428}]},"public_addr":"172.21.15.134:6825/4271716428","cluster_addr":"172.21.15.134:6827/4271716428","heartbeat_back_addr":"172.21.15.134:6831/4271716428","heartbeat_front_addr":"172.21.15.134:6829/4271716428","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T17:57:21.225125+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T17:57:55.197254+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T17:58:27.825136+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T17:59:04.319893+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T17:59:38.407752+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T18:00:12.118805+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T18:00:47.635214+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T18:01:21.231055+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T18:01:55.817861+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T18:02:31.680160+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T18:03:06.783398+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T18:03:41.307347+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.8","mappings":[{"from":2,"to":11}]},{"pgid":"2.1d","mappings":[{"from":8,"to":11}]},{"pgid":"3.3","mappings":[{"from":4,"to":9}]},{"pgid":"3.c","mappings":[{"from":3,"to":9}]},{"pgid":"3.21","mappings":[{"from":4,"to":8}]},{"pgid":"3.25","mappings":[{"from":5,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.22:0/3521924283":"2024-03-29T17:55:10.425504+0000","172.21.15.22:6801/730635939":"2024-03-29T17:55:10.425504+0000","172.21.15.22:0/2471229459":"2024-03-29T17:54:34.004559+0000","172.21.15.22:6800/3166828081":"2024-03-29T17:54:34.004559+0000","172.21.15.22:6800/730635939":"2024-03-29T17:55:10.425504+0000","172.21.15.22:0/1229047849":"2024-03-29T17:54:34.004559+0000","172.21.15.22:6801/3166828081":"2024-03-29T17:54:34.004559+0000","172.21.15.22:0/2717818856":"2024-03-29T17:55:10.425504+0000","172.21.15.22:0/2157570333":"2024-03-29T17:54:34.004559+0000","172.21.15.22:0/1006589530":"2024-03-29T17:55:10.425504+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-28T18:07:02.777 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:02.776+0000 7fe663000640 1 -- 172.21.15.22:0/3429492398 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7fe6440a2880 msgr2=0x7fe6440a4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:07:02.777 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:02.776+0000 7fe663000640 1 --2- 172.21.15.22:0/3429492398 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7fe6440a2880 0x7fe6440a4d40 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fe65c19f370 tx=0x7fe648005f90 comp rx=0 tx=0).stop 2024-03-28T18:07:02.777 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:02.776+0000 7fe663000640 1 -- 172.21.15.22:0/3429492398 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe65c074100 msgr2=0x7fe65c19de60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:07:02.777 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:02.776+0000 7fe663000640 1 --2- 172.21.15.22:0/3429492398 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe65c074100 0x7fe65c19de60 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7fe650000c00 tx=0x7fe650004e20 comp rx=0 tx=0).stop 2024-03-28T18:07:02.777 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:02.776+0000 7fe663000640 1 -- 172.21.15.22:0/3429492398 shutdown_connections 2024-03-28T18:07:02.777 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:02.776+0000 7fe663000640 1 --2- 172.21.15.22:0/3429492398 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7fe6440a2880 0x7fe6440a4d40 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:07:02.777 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:02.777+0000 7fe663000640 1 --2- 172.21.15.22:0/3429492398 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe65c10dab0 0x7fe65c1a5410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:07:02.777 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:02.777+0000 7fe663000640 1 --2- 172.21.15.22:0/3429492398 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fe65c0756e0 0x7fe65c19e3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:07:02.777 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:02.777+0000 7fe663000640 1 --2- 172.21.15.22:0/3429492398 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe65c074100 0x7fe65c19de60 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:07:02.777 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:02.777+0000 7fe663000640 1 -- 172.21.15.22:0/3429492398 >> 172.21.15.22:0/3429492398 conn(0x7fe65c0fca30 msgr2=0x7fe65c100e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:07:02.777 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:02.777+0000 7fe663000640 1 -- 172.21.15.22:0/3429492398 shutdown_connections 2024-03-28T18:07:02.777 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:02.777+0000 7fe663000640 1 -- 172.21.15.22:0/3429492398 wait complete. 2024-03-28T18:07:02.797 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 26, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-03-28T18:04:48.991622+0000', 'modified': '2024-03-28T18:04:56.643864+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 5, 'in': [0, 1, 2, 3, 4], 'up': {'mds_0': 34553, 'mds_1': 34511, 'mds_2': 34475, 'mds_3': 34442, 'mds_4': 24502}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_14640': {'gid': 14640, 'name': 'g', 'rank': 4, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.22:6839/3979998310', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.22:6838', 'nonce': 3979998310}, {'type': 'v1', 'addr': '172.21.15.22:6839', 'nonce': 3979998310}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_14655': {'gid': 14655, 'name': 'a', 'rank': 3, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.22:6841/3069056661', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.22:6840', 'nonce': 3069056661}, {'type': 'v1', 'addr': '172.21.15.22:6841', 'nonce': 3069056661}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_24412': {'gid': 24412, 'name': 'l', 'rank': 2, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.134:6833/3176125362', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.134:6832', 'nonce': 3176125362}, {'type': 'v1', 'addr': '172.21.15.134:6833', 'nonce': 3176125362}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_24463': {'gid': 24463, 'name': 'f', 'rank': 1, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.134:6835/3370549207', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.134:6834', 'nonce': 3370549207}, {'type': 'v1', 'addr': '172.21.15.134:6835', 'nonce': 3370549207}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_24478': {'gid': 24478, 'name': 'c', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.134:6837/899825581', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.134:6836', 'nonce': 899825581}, {'type': 'v1', 'addr': '172.21.15.134:6837', 'nonce': 899825581}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_24502': {'gid': 24502, 'name': 'i', 'rank': 4, 'incarnation': 24, 'state': 'up:active', 'state_seq': 7, 'addr': '172.21.15.134:6839/3400263217', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.134:6838', 'nonce': 3400263217}, {'type': 'v1', 'addr': '172.21.15.134:6839', 'nonce': 3400263217}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_34442': {'gid': 34442, 'name': 'k', 'rank': 3, 'incarnation': 22, 'state': 'up:active', 'state_seq': 14, 'addr': '172.21.15.79:6833/856896997', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.79:6832', 'nonce': 856896997}, {'type': 'v1', 'addr': '172.21.15.79:6833', 'nonce': 856896997}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_34475': {'gid': 34475, 'name': 'e', 'rank': 2, 'incarnation': 20, 'state': 'up:active', 'state_seq': 11, 'addr': '172.21.15.79:6835/3365773431', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.79:6834', 'nonce': 3365773431}, {'type': 'v1', 'addr': '172.21.15.79:6835', 'nonce': 3365773431}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_34511': {'gid': 34511, 'name': 'h', 'rank': 1, 'incarnation': 18, 'state': 'up:active', 'state_seq': 8, 'addr': '172.21.15.79:6837/1787903366', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.79:6836', 'nonce': 1787903366}, {'type': 'v1', 'addr': '172.21.15.79:6837', 'nonce': 1787903366}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_34553': {'gid': 34553, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.79:6839/2441542443', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.79:6838', 'nonce': 2441542443}, {'type': 'v1', 'addr': '172.21.15.79:6839', 'nonce': 2441542443}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [3], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 34553, 'qdb_cluster': [34511, 34553, 34475, 34442, 24502]} 2024-03-28T18:07:02.797 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 5/5 2024-03-28T18:07:02.798 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.i status 2024-03-28T18:07:03.029 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.029+0000 7f225ca00640 1 -- 172.21.15.22:0/504357667 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f22581037c0 msgr2=0x7f2258103be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:07:03.029 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.029+0000 7f225ca00640 1 --2- 172.21.15.22:0/504357667 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f22581037c0 0x7f2258103be0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f2248009900 tx=0x7f224802bc30 comp rx=0 tx=0).stop 2024-03-28T18:07:03.029 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.029+0000 7f225ca00640 1 -- 172.21.15.22:0/504357667 shutdown_connections 2024-03-28T18:07:03.029 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.029+0000 7f225ca00640 1 --2- 172.21.15.22:0/504357667 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f2258105370 0x7f225810bc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:07:03.029 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.029+0000 7f225ca00640 1 --2- 172.21.15.22:0/504357667 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f22581049b0 0x7f2258104e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:07:03.029 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.029+0000 7f225ca00640 1 --2- 172.21.15.22:0/504357667 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f22581037c0 0x7f2258103be0 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:07:03.029 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.029+0000 7f225ca00640 1 -- 172.21.15.22:0/504357667 >> 172.21.15.22:0/504357667 conn(0x7f22580fed50 msgr2=0x7f2258101170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:07:03.029 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.029+0000 7f225ca00640 1 -- 172.21.15.22:0/504357667 shutdown_connections 2024-03-28T18:07:03.029 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.029+0000 7f225ca00640 1 -- 172.21.15.22:0/504357667 wait complete. 2024-03-28T18:07:03.030 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.030+0000 7f225ca00640 1 Processor -- start 2024-03-28T18:07:03.030 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.030+0000 7f225ca00640 1 -- start start 2024-03-28T18:07:03.032 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.031+0000 7f225ca00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f22581037c0 0x7f2258199ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:07:03.032 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.031+0000 7f225ca00640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f22581049b0 0x7f2258199ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:07:03.032 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.031+0000 7f225ca00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2258105370 0x7f22581a1060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:07:03.032 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.031+0000 7f225ca00640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f225810f470 con 0x7f2258105370 2024-03-28T18:07:03.032 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.031+0000 7f225ca00640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f225810f2f0 con 0x7f22581049b0 2024-03-28T18:07:03.032 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.031+0000 7f225ca00640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f225810f5f0 con 0x7f22581037c0 2024-03-28T18:07:03.032 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.031+0000 7f2257200640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2258105370 0x7f22581a1060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:07:03.032 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.031+0000 7f2257200640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2258105370 0x7f22581a1060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:39512/0 (socket says 172.21.15.22:39512) 2024-03-28T18:07:03.032 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.031+0000 7f2257200640 1 -- 172.21.15.22:0/4217356451 learned_addr learned my addr 172.21.15.22:0/4217356451 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-28T18:07:03.032 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.031+0000 7f2256800640 1 --2- 172.21.15.22:0/4217356451 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f22581037c0 0x7f2258199ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:07:03.033 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.031+0000 7f224fe00640 1 --2- 172.21.15.22:0/4217356451 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f22581049b0 0x7f2258199ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:07:03.033 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.032+0000 7f2257200640 1 -- 172.21.15.22:0/4217356451 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f22581037c0 msgr2=0x7f2258199ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:07:03.033 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.032+0000 7f2257200640 1 --2- 172.21.15.22:0/4217356451 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f22581037c0 0x7f2258199ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:07:03.033 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.032+0000 7f2257200640 1 -- 172.21.15.22:0/4217356451 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f22581049b0 msgr2=0x7f2258199ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:07:03.033 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.032+0000 7f2257200640 1 --2- 172.21.15.22:0/4217356451 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f22581049b0 0x7f2258199ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:07:03.033 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.032+0000 7f2257200640 1 -- 172.21.15.22:0/4217356451 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f22581a16d0 con 0x7f2258105370 2024-03-28T18:07:03.033 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.032+0000 7f2257200640 1 --2- 172.21.15.22:0/4217356451 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2258105370 0x7f22581a1060 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f225000eb10 tx=0x7f225000c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:07:03.033 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.032+0000 7f224f400640 1 -- 172.21.15.22:0/4217356451 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2250004b20 con 0x7f2258105370 2024-03-28T18:07:03.033 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.032+0000 7f224f400640 1 -- 172.21.15.22:0/4217356451 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f2250004cc0 con 0x7f2258105370 2024-03-28T18:07:03.033 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.032+0000 7f225ca00640 1 -- 172.21.15.22:0/4217356451 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f22581a19c0 con 0x7f2258105370 2024-03-28T18:07:03.033 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.032+0000 7f224f400640 1 -- 172.21.15.22:0/4217356451 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2250010700 con 0x7f2258105370 2024-03-28T18:07:03.033 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.032+0000 7f225ca00640 1 -- 172.21.15.22:0/4217356451 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f22581a1ec0 con 0x7f2258105370 2024-03-28T18:07:03.035 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.036+0000 7f22601af740 1 Processor -- start 2024-03-28T18:07:03.036 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.036+0000 7f224f400640 1 -- 172.21.15.22:0/4217356451 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 18) v1 ==== 137290+0+0 (secure 0 0 0) 0x7f2250004e70 con 0x7f2258105370 2024-03-28T18:07:03.036 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.036+0000 7f22601af740 1 -- start start 2024-03-28T18:07:03.038 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.036+0000 7f22601af740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a02b68ed80 0x55a02b68f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:07:03.038 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.036+0000 7f22601af740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55a02b68f770 0x55a02b694270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:07:03.038 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.036+0000 7f22601af740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a02b6948e0 0x55a02b696d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:07:03.038 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.036+0000 7f22601af740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a02b5f3c90 con 0x55a02b68ed80 2024-03-28T18:07:03.038 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.036+0000 7f22601af740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a02b59c980 con 0x55a02b68f770 2024-03-28T18:07:03.038 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.036+0000 7f22601af740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a02b5efd30 con 0x55a02b6948e0 2024-03-28T18:07:03.038 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.037+0000 7f224fe00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a02b68ed80 0x55a02b68f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:07:03.038 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.037+0000 7f224fe00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a02b68ed80 0x55a02b68f1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:39516/0 (socket says 172.21.15.22:39516) 2024-03-28T18:07:03.038 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.037+0000 7f224fe00640 1 -- 172.21.15.22:0/1755101918 learned_addr learned my addr 172.21.15.22:0/1755101918 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-28T18:07:03.038 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.037+0000 7f2256800640 1 --2- 172.21.15.22:0/1755101918 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55a02b68f770 0x55a02b694270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:07:03.038 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.037+0000 7f224fe00640 1 --2- 172.21.15.22:0/1755101918 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a02b6948e0 0x55a02b696d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:07:03.038 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.037+0000 7f224fe00640 1 -- 172.21.15.22:0/1755101918 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a02b6948e0 msgr2=0x55a02b696d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T18:07:03.039 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.037+0000 7f224fe00640 1 --2- 172.21.15.22:0/1755101918 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a02b6948e0 0x55a02b696d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:07:03.039 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.037+0000 7f224fe00640 1 -- 172.21.15.22:0/1755101918 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55a02b68f770 msgr2=0x55a02b694270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T18:07:03.039 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.037+0000 7f224fe00640 1 --2- 172.21.15.22:0/1755101918 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55a02b68f770 0x55a02b694270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:07:03.039 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.037+0000 7f224fe00640 1 -- 172.21.15.22:0/1755101918 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a02b5b4510 con 0x55a02b68ed80 2024-03-28T18:07:03.039 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.037+0000 7f224f400640 1 --2- 172.21.15.22:0/4217356451 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7f223c0a2800 0x7f223c0a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:07:03.039 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.038+0000 7f224f400640 1 -- 172.21.15.22:0/4217356451 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 8153+0+0 (secure 0 0 0) 0x7f22500cf080 con 0x7f2258105370 2024-03-28T18:07:03.039 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.038+0000 7f224fe00640 1 --2- 172.21.15.22:0/1755101918 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a02b68ed80 0x55a02b68f1a0 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f224000cee0 tx=0x7f224000c620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3200b955530d09ce server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:07:03.039 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.038+0000 7f2256800640 1 --2- 172.21.15.22:0/4217356451 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7f223c0a2800 0x7f223c0a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:07:03.039 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.038+0000 7f2247400640 1 -- 172.21.15.22:0/1755101918 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2240005220 con 0x55a02b68ed80 2024-03-28T18:07:03.039 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.038+0000 7f2247400640 1 -- 172.21.15.22:0/1755101918 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f2240016030 con 0x55a02b68ed80 2024-03-28T18:07:03.039 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.038+0000 7f2247400640 1 -- 172.21.15.22:0/1755101918 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2240012950 con 0x55a02b68ed80 2024-03-28T18:07:03.043 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.039+0000 7f22601af740 1 -- 172.21.15.22:0/1755101918 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a02b68ed80 msgr2=0x55a02b68f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:07:03.043 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.039+0000 7f22601af740 1 --2- 172.21.15.22:0/1755101918 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a02b68ed80 0x55a02b68f1a0 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f224000cee0 tx=0x7f224000c620 comp rx=0 tx=0).stop 2024-03-28T18:07:03.043 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.039+0000 7f2256800640 1 --2- 172.21.15.22:0/4217356451 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7f223c0a2800 0x7f223c0a4cc0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f224802c170 tx=0x7f2248005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:07:03.043 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.039+0000 7f22601af740 1 -- 172.21.15.22:0/1755101918 shutdown_connections 2024-03-28T18:07:03.043 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.039+0000 7f22601af740 1 --2- 172.21.15.22:0/1755101918 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a02b6948e0 0x55a02b696d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:07:03.043 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.039+0000 7f22601af740 1 --2- 172.21.15.22:0/1755101918 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55a02b68f770 0x55a02b694270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:07:03.043 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.039+0000 7f22601af740 1 --2- 172.21.15.22:0/1755101918 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a02b68ed80 0x55a02b68f1a0 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:07:03.043 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.039+0000 7f22601af740 1 -- 172.21.15.22:0/1755101918 >> 172.21.15.22:0/1755101918 conn(0x55a02b68a2f0 msgr2=0x55a02b68a6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:07:03.043 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.039+0000 7f22601af740 1 -- 172.21.15.22:0/1755101918 shutdown_connections 2024-03-28T18:07:03.043 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.039+0000 7f22601af740 1 -- 172.21.15.22:0/1755101918 wait complete. 2024-03-28T18:07:03.043 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.039+0000 7f22601af740 1 -- start start 2024-03-28T18:07:03.044 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.040+0000 7f22601af740 1 Processor -- start 2024-03-28T18:07:03.044 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.040+0000 7f22601af740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a02b68ed80 0x55a02b6992c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:07:03.044 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.040+0000 7f22601af740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a02b68f770 0x55a02b699800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:07:03.044 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.040+0000 7f22601af740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55a02b6948e0 0x55a02b6a0840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:07:03.044 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.040+0000 7f22601af740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a02b697890 con 0x55a02b68f770 2024-03-28T18:07:03.044 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.040+0000 7f22601af740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a02b5afdd0 con 0x55a02b6948e0 2024-03-28T18:07:03.044 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.040+0000 7f22601af740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a02b697a10 con 0x55a02b68ed80 2024-03-28T18:07:03.044 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.040+0000 7f22601af740 10 client.0 _ll_register_callbacks cb 0x55a02b307ac0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-28T18:07:03.044 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.040+0000 7f224fe00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a02b68f770 0x55a02b699800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:07:03.044 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.040+0000 7f224fe00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a02b68f770 0x55a02b699800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:39524/0 (socket says 172.21.15.22:39524) 2024-03-28T18:07:03.044 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.040+0000 7f224fe00640 1 -- 172.21.15.22:0/3558458314 learned_addr learned my addr 172.21.15.22:0/3558458314 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-28T18:07:03.044 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.041+0000 7f2257200640 1 --2- 172.21.15.22:0/3558458314 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55a02b6948e0 0x55a02b6a0840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:07:03.044 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.041+0000 7f2256800640 1 --2- 172.21.15.22:0/3558458314 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a02b68ed80 0x55a02b6992c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:07:03.044 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.041+0000 7f224fe00640 1 -- 172.21.15.22:0/3558458314 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a02b68ed80 msgr2=0x55a02b6992c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T18:07:03.044 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.041+0000 7f224fe00640 1 --2- 172.21.15.22:0/3558458314 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a02b68ed80 0x55a02b6992c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:07:03.045 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.041+0000 7f224fe00640 1 -- 172.21.15.22:0/3558458314 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55a02b6948e0 msgr2=0x55a02b6a0840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T18:07:03.045 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.041+0000 7f224fe00640 1 --2- 172.21.15.22:0/3558458314 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55a02b6948e0 0x55a02b6a0840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:07:03.045 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.041+0000 7f224fe00640 1 -- 172.21.15.22:0/3558458314 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55a02b3082d0 con 0x55a02b68f770 2024-03-28T18:07:03.045 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.041+0000 7f224fe00640 1 --2- 172.21.15.22:0/3558458314 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a02b68f770 0x55a02b699800 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f22400057c0 tx=0x7f224000c3f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6cc3d7eba81ba462 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:07:03.045 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.041+0000 7f223be00640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-03-28T18:07:03.045 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.041+0000 7f223be00640 1 -- 172.21.15.22:0/3558458314 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2240010e70 con 0x55a02b68f770 2024-03-28T18:07:03.045 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.042+0000 7f22601af740 1 -- 172.21.15.22:0/3558458314 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f2240016030 con 0x55a02b68f770 2024-03-28T18:07:03.045 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.042+0000 7f223be00640 1 -- 172.21.15.22:0/3558458314 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 8153+0+0 (secure 0 0 0) 0x7f2240005210 con 0x55a02b68f770 2024-03-28T18:07:03.045 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.042+0000 7f223be00640 1 -- 172.21.15.22:0/3558458314 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2240004050 con 0x55a02b68f770 2024-03-28T18:07:03.045 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.042+0000 7f22601af740 10 client.15270 fetch_fsmap learned FSMap version 26 2024-03-28T18:07:03.045 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.042+0000 7f22601af740 1 -- 172.21.15.22:0/3558458314 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x55a02b3d26b0 con 0x55a02b68f770 2024-03-28T18:07:03.045 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.042+0000 7f223be00640 1 -- 172.21.15.22:0/3558458314 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f2240016030 con 0x55a02b68f770 2024-03-28T18:07:03.045 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.042+0000 7f223be00640 1 -- 172.21.15.22:0/3558458314 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 26) v1 ==== 7934+0+0 (secure 0 0 0) 0x7f22400126d0 con 0x55a02b68f770 2024-03-28T18:07:03.045 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.042+0000 7f22601af740 10 client.15270 fetch_fsmap finished waiting for FSMap version 26 2024-03-28T18:07:03.045 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.043+0000 7f22601af740 10 client.15270 resolve_mds: resolved name 'i' to daemon mds.i 2024-03-28T18:07:03.045 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.043+0000 7f22601af740 20 client.15270 populate_metadata read hostname 'smithi022' 2024-03-28T18:07:03.045 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.043+0000 7f22601af740 1 --2- 172.21.15.22:0/3558458314 >> [v2:172.21.15.134:6838/3400263217,v1:172.21.15.134:6839/3400263217] conn(0x55a02b6a57a0 0x55a02b6a4a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:07:03.046 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.043+0000 7f22601af740 4 client.15270 mds_command: new command op to 24502 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-28T18:07:03.046 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.043+0000 7f22601af740 1 -- 172.21.15.22:0/3558458314 --> [v2:172.21.15.134:6838/3400263217,v1:172.21.15.134:6839/3400263217] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55a02b595520 con 0x55a02b6a57a0 2024-03-28T18:07:03.046 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.043+0000 7f2256800640 1 --2- 172.21.15.22:0/3558458314 >> [v2:172.21.15.134:6838/3400263217,v1:172.21.15.134:6839/3400263217] conn(0x55a02b6a57a0 0x55a02b6a4a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:07:03.047 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.044+0000 7f2256800640 1 --2- 172.21.15.22:0/3558458314 >> [v2:172.21.15.134:6838/3400263217,v1:172.21.15.134:6839/3400263217] conn(0x55a02b6a57a0 0x55a02b6a4a50 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=5a27d872f348cd51 server_cookie=de674c892609f0de in_seq=0 out_seq=0 2024-03-28T18:07:03.047 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.045+0000 7f223be00640 10 client.15270 ms_handle_connect on v2:172.21.15.134:6838/3400263217 2024-03-28T18:07:03.047 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.047+0000 7f223be00640 1 -- 172.21.15.22:0/3558458314 <== mds.4 v2:172.21.15.134:6838/3400263217 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x55a02b595520 con 0x55a02b6a57a0 2024-03-28T18:07:03.047 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.047+0000 7f223be00640 10 client.15270 handle_command_reply: tid=0 2024-03-28T18:07:03.047 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.047+0000 7f22601af740 1 client.15270 shutdown 2024-03-28T18:07:03.047 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.047+0000 7f22601af740 1 -- 172.21.15.22:0/3558458314 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a02b68f770 msgr2=0x55a02b699800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:07:03.047 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.047+0000 7f22601af740 1 --2- 172.21.15.22:0/3558458314 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a02b68f770 0x55a02b699800 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f22400057c0 tx=0x7f224000c3f0 comp rx=0 tx=0).stop 2024-03-28T18:07:03.047 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.047+0000 7f22601af740 1 -- 172.21.15.22:0/3558458314 shutdown_connections 2024-03-28T18:07:03.047 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.047+0000 7f22601af740 1 --2- 172.21.15.22:0/3558458314 >> [v2:172.21.15.134:6838/3400263217,v1:172.21.15.134:6839/3400263217] conn(0x55a02b6a57a0 0x55a02b6a4a50 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:07:03.048 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.047+0000 7f22601af740 1 --2- 172.21.15.22:0/3558458314 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55a02b6948e0 0x55a02b6a0840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:07:03.048 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.047+0000 7f22601af740 1 --2- 172.21.15.22:0/3558458314 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a02b68f770 0x55a02b699800 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:07:03.048 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.047+0000 7f22601af740 1 --2- 172.21.15.22:0/3558458314 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a02b68ed80 0x55a02b6992c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:07:03.048 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.047+0000 7f22601af740 1 -- 172.21.15.22:0/3558458314 >> 172.21.15.22:0/3558458314 conn(0x55a02b68a2f0 msgr2=0x55a02b6894f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:07:03.048 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.047+0000 7f223be00640 0 client.15270 ms_handle_reset on v2:172.21.15.134:6838/3400263217 2024-03-28T18:07:03.048 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.047+0000 7f22601af740 1 -- 172.21.15.22:0/3558458314 shutdown_connections 2024-03-28T18:07:03.048 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.047+0000 7f22601af740 1 -- 172.21.15.22:0/3558458314 wait complete. 2024-03-28T18:07:03.048 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.047+0000 7f22601af740 20 client.15270 trim_cache size 0 max 16384 2024-03-28T18:07:03.066 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.066+0000 7f22601af740 1 Processor -- start 2024-03-28T18:07:03.066 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.066+0000 7f22601af740 1 -- start start 2024-03-28T18:07:03.067 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.066+0000 7f22601af740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55a02b68a2f0 0x55a02b697db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:07:03.067 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.066+0000 7f22601af740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a02b68ed80 0x55a02b6a0800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:07:03.067 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.066+0000 7f22601af740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a02b68f770 0x55a02b6982f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:07:03.067 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.066+0000 7f22601af740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a02b6a4fc0 con 0x55a02b68ed80 2024-03-28T18:07:03.067 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.066+0000 7f22601af740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a02b6a5620 con 0x55a02b68a2f0 2024-03-28T18:07:03.067 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.066+0000 7f22601af740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a02b6a5c80 con 0x55a02b68f770 2024-03-28T18:07:03.067 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.067+0000 7f224fe00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a02b68ed80 0x55a02b6a0800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:07:03.067 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.067+0000 7f224fe00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a02b68ed80 0x55a02b6a0800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:39540/0 (socket says 172.21.15.22:39540) 2024-03-28T18:07:03.068 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.067+0000 7f224fe00640 1 -- 172.21.15.22:0/287778946 learned_addr learned my addr 172.21.15.22:0/287778946 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-28T18:07:03.068 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.067+0000 7f2257200640 1 --2- 172.21.15.22:0/287778946 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a02b68f770 0x55a02b6982f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:07:03.068 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.067+0000 7f2256800640 1 --2- 172.21.15.22:0/287778946 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55a02b68a2f0 0x55a02b697db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:07:03.068 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.067+0000 7f224fe00640 1 -- 172.21.15.22:0/287778946 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a02b68f770 msgr2=0x55a02b6982f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T18:07:03.068 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.067+0000 7f224fe00640 1 --2- 172.21.15.22:0/287778946 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a02b68f770 0x55a02b6982f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:07:03.068 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.067+0000 7f224fe00640 1 -- 172.21.15.22:0/287778946 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55a02b68a2f0 msgr2=0x55a02b697db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T18:07:03.068 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.067+0000 7f224fe00640 1 --2- 172.21.15.22:0/287778946 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55a02b68a2f0 0x55a02b697db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:07:03.068 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.067+0000 7f224fe00640 1 -- 172.21.15.22:0/287778946 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a02b69a3f0 con 0x55a02b68ed80 2024-03-28T18:07:03.068 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.067+0000 7f2256800640 1 --2- 172.21.15.22:0/287778946 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55a02b68a2f0 0x55a02b697db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-28T18:07:03.068 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.068+0000 7f224fe00640 1 --2- 172.21.15.22:0/287778946 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a02b68ed80 0x55a02b6a0800 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f2240003610 tx=0x7f224000c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2e460840b67816e8 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:07:03.070 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.068+0000 7f2246a00640 1 -- 172.21.15.22:0/287778946 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2240035030 con 0x55a02b68ed80 2024-03-28T18:07:03.070 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.068+0000 7f2246a00640 1 -- 172.21.15.22:0/287778946 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f2240016040 con 0x55a02b68ed80 2024-03-28T18:07:03.070 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.068+0000 7f2246a00640 1 -- 172.21.15.22:0/287778946 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2240003b50 con 0x55a02b68ed80 2024-03-28T18:07:03.070 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.069+0000 7f22601af740 1 -- 172.21.15.22:0/287778946 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a02b68ed80 msgr2=0x55a02b6a0800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:07:03.070 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.069+0000 7f22601af740 1 --2- 172.21.15.22:0/287778946 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a02b68ed80 0x55a02b6a0800 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f2240003610 tx=0x7f224000c910 comp rx=0 tx=0).stop 2024-03-28T18:07:03.070 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.069+0000 7f22601af740 1 -- 172.21.15.22:0/287778946 shutdown_connections 2024-03-28T18:07:03.071 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.069+0000 7f22601af740 1 --2- 172.21.15.22:0/287778946 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a02b68f770 0x55a02b6982f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:07:03.071 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.069+0000 7f22601af740 1 --2- 172.21.15.22:0/287778946 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a02b68ed80 0x55a02b6a0800 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:07:03.071 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.069+0000 7f22601af740 1 --2- 172.21.15.22:0/287778946 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55a02b68a2f0 0x55a02b697db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:07:03.071 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.069+0000 7f22601af740 1 -- 172.21.15.22:0/287778946 >> 172.21.15.22:0/287778946 conn(0x55a02b690550 msgr2=0x55a02b68a6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:07:03.071 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.069+0000 7f22601af740 1 -- 172.21.15.22:0/287778946 shutdown_connections 2024-03-28T18:07:03.071 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.069+0000 7f22601af740 1 -- 172.21.15.22:0/287778946 wait complete. 2024-03-28T18:07:03.071 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.069+0000 7f22601af740 1 -- start start 2024-03-28T18:07:03.071 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.069+0000 7f22601af740 1 Processor -- start 2024-03-28T18:07:03.071 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.070+0000 7f22601af740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a02b68a2f0 0x55a02b69ffc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:07:03.071 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.070+0000 7f22601af740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55a02b68ed80 0x55a02b6a0500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:07:03.071 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.070+0000 7f22601af740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a02b68f770 0x55a02b69ca60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:07:03.071 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.070+0000 7f22601af740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a02b6a6940 con 0x55a02b68f770 2024-03-28T18:07:03.071 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.070+0000 7f22601af740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a02b6a62e0 con 0x55a02b68ed80 2024-03-28T18:07:03.071 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.070+0000 7f22601af740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a02b6a6fa0 con 0x55a02b68a2f0 2024-03-28T18:07:03.072 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.070+0000 7f22601af740 10 client.0 _ll_register_callbacks cb 0x7f22500093a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-28T18:07:03.072 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.070+0000 7f2257200640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a02b68f770 0x55a02b69ca60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:07:03.072 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.070+0000 7f2257200640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a02b68f770 0x55a02b69ca60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:39556/0 (socket says 172.21.15.22:39556) 2024-03-28T18:07:03.072 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.070+0000 7f2257200640 1 -- 172.21.15.22:0/838394552 learned_addr learned my addr 172.21.15.22:0/838394552 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-28T18:07:03.072 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.070+0000 7f224fe00640 1 --2- 172.21.15.22:0/838394552 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55a02b68ed80 0x55a02b6a0500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:07:03.072 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.070+0000 7f2256800640 1 --2- 172.21.15.22:0/838394552 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a02b68a2f0 0x55a02b69ffc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:07:03.072 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.071+0000 7f2257200640 1 -- 172.21.15.22:0/838394552 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a02b68a2f0 msgr2=0x55a02b69ffc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T18:07:03.072 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.071+0000 7f2257200640 1 --2- 172.21.15.22:0/838394552 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a02b68a2f0 0x55a02b69ffc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:07:03.072 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.071+0000 7f2257200640 1 -- 172.21.15.22:0/838394552 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55a02b68ed80 msgr2=0x55a02b6a0500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T18:07:03.072 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.071+0000 7f2257200640 1 --2- 172.21.15.22:0/838394552 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55a02b68ed80 0x55a02b6a0500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:07:03.072 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.071+0000 7f2257200640 1 -- 172.21.15.22:0/838394552 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55a02b69d050 con 0x55a02b68f770 2024-03-28T18:07:03.074 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.071+0000 7f2257200640 1 --2- 172.21.15.22:0/838394552 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a02b68f770 0x55a02b69ca60 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f2250097f20 tx=0x7f2250005580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7ac7fd8ad48f315e server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:07:03.074 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.071+0000 7f223be00640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-03-28T18:07:03.074 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.071+0000 7f223be00640 1 -- 172.21.15.22:0/838394552 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22500b8020 con 0x55a02b68f770 2024-03-28T18:07:03.074 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.071+0000 7f223be00640 1 -- 172.21.15.22:0/838394552 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 8153+0+0 (secure 0 0 0) 0x7f2250016df0 con 0x55a02b68f770 2024-03-28T18:07:03.074 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.071+0000 7f22601af740 1 -- 172.21.15.22:0/838394552 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f2240016040 con 0x55a02b68f770 2024-03-28T18:07:03.074 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.071+0000 7f223be00640 1 -- 172.21.15.22:0/838394552 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7f225009fc10 con 0x55a02b68f770 2024-03-28T18:07:03.074 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.071+0000 7f22601af740 10 client.15276 fetch_fsmap learned FSMap version 26 2024-03-28T18:07:03.074 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.071+0000 7f22601af740 1 -- 172.21.15.22:0/838394552 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x55a02b701410 con 0x55a02b68f770 2024-03-28T18:07:03.074 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.072+0000 7f223be00640 1 -- 172.21.15.22:0/838394552 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f2240016040 con 0x55a02b68f770 2024-03-28T18:07:03.074 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.072+0000 7f223be00640 1 -- 172.21.15.22:0/838394552 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 26) v1 ==== 7934+0+0 (secure 0 0 0) 0x7f22500a3dd0 con 0x55a02b68f770 2024-03-28T18:07:03.074 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.072+0000 7f22601af740 10 client.15276 fetch_fsmap finished waiting for FSMap version 26 2024-03-28T18:07:03.074 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.072+0000 7f22601af740 10 client.15276 resolve_mds: resolved name 'i' to daemon mds.i 2024-03-28T18:07:03.075 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.072+0000 7f22601af740 20 client.15276 populate_metadata read hostname 'smithi022' 2024-03-28T18:07:03.075 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.072+0000 7f22601af740 1 --2- 172.21.15.22:0/838394552 >> [v2:172.21.15.134:6838/3400263217,v1:172.21.15.134:6839/3400263217] conn(0x55a02b6948e0 0x55a02b701750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:07:03.075 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.072+0000 7f22601af740 4 client.15276 mds_command: new command op to 24502 tid=0 multi_id=0 [{"prefix": "status"}] 2024-03-28T18:07:03.075 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.072+0000 7f22601af740 1 -- 172.21.15.22:0/838394552 --> [v2:172.21.15.134:6838/3400263217,v1:172.21.15.134:6839/3400263217] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55a02b6011d0 con 0x55a02b6948e0 2024-03-28T18:07:03.075 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.073+0000 7f2256800640 1 --2- 172.21.15.22:0/838394552 >> [v2:172.21.15.134:6838/3400263217,v1:172.21.15.134:6839/3400263217] conn(0x55a02b6948e0 0x55a02b701750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:07:03.075 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.074+0000 7f2256800640 1 --2- 172.21.15.22:0/838394552 >> [v2:172.21.15.134:6838/3400263217,v1:172.21.15.134:6839/3400263217] conn(0x55a02b6948e0 0x55a02b701750 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=b664bde4464d99d5 server_cookie=3ee9c0384cef3daa in_seq=0 out_seq=0 2024-03-28T18:07:03.075 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.074+0000 7f223be00640 10 client.15276 ms_handle_connect on v2:172.21.15.134:6838/3400263217 2024-03-28T18:07:03.076 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.075+0000 7f223be00640 1 -- 172.21.15.22:0/838394552 <== mds.4 v2:172.21.15.134:6838/3400263217 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+327 (crc 0 0 0) 0x55a02b6011d0 con 0x55a02b6948e0 2024-03-28T18:07:03.076 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.075+0000 7f223be00640 10 client.15276 handle_command_reply: tid=0 2024-03-28T18:07:03.076 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.075+0000 7f22601af740 1 client.15276 shutdown 2024-03-28T18:07:03.076 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.075+0000 7f22601af740 1 -- 172.21.15.22:0/838394552 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a02b68f770 msgr2=0x55a02b69ca60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:07:03.076 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.075+0000 7f22601af740 1 --2- 172.21.15.22:0/838394552 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a02b68f770 0x55a02b69ca60 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f2250097f20 tx=0x7f2250005580 comp rx=0 tx=0).stop 2024-03-28T18:07:03.076 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.075+0000 7f22601af740 1 -- 172.21.15.22:0/838394552 shutdown_connections 2024-03-28T18:07:03.076 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.075+0000 7f22601af740 1 --2- 172.21.15.22:0/838394552 >> [v2:172.21.15.134:6838/3400263217,v1:172.21.15.134:6839/3400263217] conn(0x55a02b6948e0 0x55a02b701750 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:07:03.076 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.075+0000 7f22601af740 1 --2- 172.21.15.22:0/838394552 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a02b68f770 0x55a02b69ca60 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:07:03.076 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.075+0000 7f22601af740 1 --2- 172.21.15.22:0/838394552 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55a02b68ed80 0x55a02b6a0500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:07:03.076 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.075+0000 7f22601af740 1 --2- 172.21.15.22:0/838394552 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a02b68a2f0 0x55a02b69ffc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:07:03.076 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.075+0000 7f223be00640 0 client.15276 ms_handle_reset on v2:172.21.15.134:6838/3400263217 2024-03-28T18:07:03.077 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.075+0000 7f22601af740 1 -- 172.21.15.22:0/838394552 >> 172.21.15.22:0/838394552 conn(0x55a02b690550 msgr2=0x55a02b68d550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:07:03.077 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.075+0000 7f22601af740 1 -- 172.21.15.22:0/838394552 shutdown_connections 2024-03-28T18:07:03.077 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.075+0000 7f22601af740 1 -- 172.21.15.22:0/838394552 wait complete. 2024-03-28T18:07:03.077 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.075+0000 7f22601af740 20 client.15276 trim_cache size 0 max 16384 2024-03-28T18:07:03.077 INFO:teuthology.orchestra.run.smithi022.stdout:{ 2024-03-28T18:07:03.077 INFO:teuthology.orchestra.run.smithi022.stdout: "cluster_fsid": "fd220748-ed2b-11ee-95d0-87774f69a715", 2024-03-28T18:07:03.077 INFO:teuthology.orchestra.run.smithi022.stdout: "whoami": 4, 2024-03-28T18:07:03.077 INFO:teuthology.orchestra.run.smithi022.stdout: "id": 24502, 2024-03-28T18:07:03.077 INFO:teuthology.orchestra.run.smithi022.stdout: "want_state": "up:active", 2024-03-28T18:07:03.077 INFO:teuthology.orchestra.run.smithi022.stdout: "state": "up:active", 2024-03-28T18:07:03.077 INFO:teuthology.orchestra.run.smithi022.stdout: "fs_name": "cephfs", 2024-03-28T18:07:03.077 INFO:teuthology.orchestra.run.smithi022.stdout: "rank_uptime": 127.44535468399999, 2024-03-28T18:07:03.078 INFO:teuthology.orchestra.run.smithi022.stdout: "mdsmap_epoch": 26, 2024-03-28T18:07:03.078 INFO:teuthology.orchestra.run.smithi022.stdout: "osdmap_epoch": 92, 2024-03-28T18:07:03.078 INFO:teuthology.orchestra.run.smithi022.stdout: "osdmap_epoch_barrier": 90, 2024-03-28T18:07:03.078 INFO:teuthology.orchestra.run.smithi022.stdout: "uptime": 150.804933401 2024-03-28T18:07:03.078 INFO:teuthology.orchestra.run.smithi022.stdout:} 2024-03-28T18:07:03.078 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.076+0000 7f225ca00640 1 -- 172.21.15.22:0/4217356451 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7f223c0a2800 msgr2=0x7f223c0a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:07:03.079 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.076+0000 7f225ca00640 1 --2- 172.21.15.22:0/4217356451 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7f223c0a2800 0x7f223c0a4cc0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f224802c170 tx=0x7f2248005fd0 comp rx=0 tx=0).stop 2024-03-28T18:07:03.079 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.076+0000 7f225ca00640 1 -- 172.21.15.22:0/4217356451 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2258105370 msgr2=0x7f22581a1060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:07:03.079 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.076+0000 7f225ca00640 1 --2- 172.21.15.22:0/4217356451 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2258105370 0x7f22581a1060 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f225000eb10 tx=0x7f225000c2d0 comp rx=0 tx=0).stop 2024-03-28T18:07:03.079 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.076+0000 7f225ca00640 1 -- 172.21.15.22:0/4217356451 shutdown_connections 2024-03-28T18:07:03.079 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.076+0000 7f225ca00640 1 --2- 172.21.15.22:0/4217356451 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7f223c0a2800 0x7f223c0a4cc0 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:07:03.079 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.076+0000 7f225ca00640 1 --2- 172.21.15.22:0/4217356451 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2258105370 0x7f22581a1060 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:07:03.079 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.076+0000 7f225ca00640 1 --2- 172.21.15.22:0/4217356451 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f22581049b0 0x7f2258199ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:07:03.079 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.076+0000 7f225ca00640 1 --2- 172.21.15.22:0/4217356451 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f22581037c0 0x7f2258199ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:07:03.079 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.076+0000 7f225ca00640 1 -- 172.21.15.22:0/4217356451 >> 172.21.15.22:0/4217356451 conn(0x7f22580fed50 msgr2=0x7f2258107c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:07:03.079 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.076+0000 7f225ca00640 1 -- 172.21.15.22:0/4217356451 shutdown_connections 2024-03-28T18:07:03.079 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.076+0000 7f225ca00640 1 -- 172.21.15.22:0/4217356451 wait complete. 2024-03-28T18:07:03.095 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.k status 2024-03-28T18:07:03.322 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.322+0000 7fb3e3e00640 1 -- 172.21.15.22:0/1931537850 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb3e4102790 msgr2=0x7fb3e4102c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:07:03.322 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.322+0000 7fb3e3e00640 1 --2- 172.21.15.22:0/1931537850 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb3e4102790 0x7fb3e4102c10 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7fb3cc009900 tx=0x7fb3cc02bc30 comp rx=0 tx=0).stop 2024-03-28T18:07:03.322 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.322+0000 7fb3e3e00640 1 -- 172.21.15.22:0/1931537850 shutdown_connections 2024-03-28T18:07:03.322 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.322+0000 7fb3e3e00640 1 --2- 172.21.15.22:0/1931537850 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fb3e4103150 0x7fb3e41099e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:07:03.322 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.322+0000 7fb3e3e00640 1 --2- 172.21.15.22:0/1931537850 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb3e4102790 0x7fb3e4102c10 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:07:03.322 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.322+0000 7fb3e3e00640 1 --2- 172.21.15.22:0/1931537850 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb3e41015a0 0x7fb3e41019c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:07:03.323 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.322+0000 7fb3e3e00640 1 -- 172.21.15.22:0/1931537850 >> 172.21.15.22:0/1931537850 conn(0x7fb3e40fcb10 msgr2=0x7fb3e40fef50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:07:03.323 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.322+0000 7fb3e3e00640 1 -- 172.21.15.22:0/1931537850 shutdown_connections 2024-03-28T18:07:03.323 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.322+0000 7fb3e3e00640 1 -- 172.21.15.22:0/1931537850 wait complete. 2024-03-28T18:07:03.324 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.323+0000 7fb3e3e00640 1 Processor -- start 2024-03-28T18:07:03.324 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.324+0000 7fb3e3e00640 1 -- start start 2024-03-28T18:07:03.324 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.324+0000 7fb3e3e00640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fb3e41015a0 0x7fb3e4075450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:07:03.324 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.324+0000 7fb3e3e00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb3e4102790 0x7fb3e4073a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:07:03.325 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.324+0000 7fb3e3e00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb3e4103150 0x7fb3e4073fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:07:03.325 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.324+0000 7fb3e3e00640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3e410d250 con 0x7fb3e4102790 2024-03-28T18:07:03.325 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.324+0000 7fb3e3e00640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3e410d0d0 con 0x7fb3e41015a0 2024-03-28T18:07:03.325 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.324+0000 7fb3e3e00640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3e410d3d0 con 0x7fb3e4103150 2024-03-28T18:07:03.325 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.324+0000 7fb3da000640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb3e4102790 0x7fb3e4073a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:07:03.325 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.324+0000 7fb3e2a00640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fb3e41015a0 0x7fb3e4075450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:07:03.325 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.324+0000 7fb3da000640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb3e4102790 0x7fb3e4073a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:39576/0 (socket says 172.21.15.22:39576) 2024-03-28T18:07:03.325 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.324+0000 7fb3da000640 1 -- 172.21.15.22:0/4085968283 learned_addr learned my addr 172.21.15.22:0/4085968283 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-28T18:07:03.325 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.325+0000 7fb3e3400640 1 --2- 172.21.15.22:0/4085968283 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb3e4103150 0x7fb3e4073fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:07:03.325 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.325+0000 7fb3da000640 1 -- 172.21.15.22:0/4085968283 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb3e4103150 msgr2=0x7fb3e4073fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:07:03.325 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.325+0000 7fb3da000640 1 --2- 172.21.15.22:0/4085968283 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb3e4103150 0x7fb3e4073fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:07:03.325 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.325+0000 7fb3da000640 1 -- 172.21.15.22:0/4085968283 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fb3e41015a0 msgr2=0x7fb3e4075450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:07:03.325 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.325+0000 7fb3da000640 1 --2- 172.21.15.22:0/4085968283 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fb3e41015a0 0x7fb3e4075450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:07:03.326 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.325+0000 7fb3da000640 1 -- 172.21.15.22:0/4085968283 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb3e4074860 con 0x7fb3e4102790 2024-03-28T18:07:03.326 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.325+0000 7fb3e3400640 1 --2- 172.21.15.22:0/4085968283 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb3e4103150 0x7fb3e4073fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T18:07:03.326 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.325+0000 7fb3da000640 1 --2- 172.21.15.22:0/4085968283 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb3e4102790 0x7fb3e4073a90 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7fb3cc000c00 tx=0x7fb3cc004e20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:07:03.326 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.325+0000 7fb3dbe00640 1 -- 172.21.15.22:0/4085968283 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3cc043020 con 0x7fb3e4102790 2024-03-28T18:07:03.326 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.325+0000 7fb3dbe00640 1 -- 172.21.15.22:0/4085968283 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7fb3cc005460 con 0x7fb3e4102790 2024-03-28T18:07:03.327 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.325+0000 7fb3dbe00640 1 -- 172.21.15.22:0/4085968283 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3cc003d00 con 0x7fb3e4102790 2024-03-28T18:07:03.327 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.325+0000 7fb3e3e00640 1 -- 172.21.15.22:0/4085968283 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb3e41a1a10 con 0x7fb3e4102790 2024-03-28T18:07:03.327 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.325+0000 7fb3e3e00640 1 -- 172.21.15.22:0/4085968283 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb3e41a1fb0 con 0x7fb3e4102790 2024-03-28T18:07:03.328 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.328+0000 7fb3dbe00640 1 -- 172.21.15.22:0/4085968283 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 18) v1 ==== 137290+0+0 (secure 0 0 0) 0x7fb3cc046020 con 0x7fb3e4102790 2024-03-28T18:07:03.328 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.329+0000 7fb3ec403740 1 Processor -- start 2024-03-28T18:07:03.328 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.329+0000 7fb3ec403740 1 -- start start 2024-03-28T18:07:03.330 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.329+0000 7fb3ec403740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56265d23ad80 0x56265d23b1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:07:03.330 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.329+0000 7fb3ec403740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56265d23b770 0x56265d240270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:07:03.330 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.329+0000 7fb3ec403740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56265d2408e0 0x56265d242d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:07:03.330 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.329+0000 7fb3ec403740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x56265d19fc90 con 0x56265d23b770 2024-03-28T18:07:03.330 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.329+0000 7fb3ec403740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x56265d148980 con 0x56265d2408e0 2024-03-28T18:07:03.330 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.329+0000 7fb3ec403740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x56265d19bd30 con 0x56265d23ad80 2024-03-28T18:07:03.330 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.329+0000 7fb3e2a00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56265d23b770 0x56265d240270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:07:03.330 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.329+0000 7fb3e2a00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56265d23b770 0x56265d240270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:39582/0 (socket says 172.21.15.22:39582) 2024-03-28T18:07:03.330 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.329+0000 7fb3e2a00640 1 -- 172.21.15.22:0/67332274 learned_addr learned my addr 172.21.15.22:0/67332274 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-28T18:07:03.330 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.329+0000 7fb3da000640 1 --2- 172.21.15.22:0/67332274 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56265d2408e0 0x56265d242d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:07:03.330 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.330+0000 7fb3e3400640 1 --2- 172.21.15.22:0/67332274 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56265d23ad80 0x56265d23b1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:07:03.331 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.330+0000 7fb3e2a00640 1 -- 172.21.15.22:0/67332274 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56265d23ad80 msgr2=0x56265d23b1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T18:07:03.331 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.330+0000 7fb3e2a00640 1 --2- 172.21.15.22:0/67332274 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56265d23ad80 0x56265d23b1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:07:03.331 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.330+0000 7fb3e2a00640 1 -- 172.21.15.22:0/67332274 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56265d2408e0 msgr2=0x56265d242d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T18:07:03.331 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.330+0000 7fb3e2a00640 1 --2- 172.21.15.22:0/67332274 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56265d2408e0 0x56265d242d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:07:03.331 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.330+0000 7fb3e2a00640 1 -- 172.21.15.22:0/67332274 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56265d160510 con 0x56265d23b770 2024-03-28T18:07:03.331 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.330+0000 7fb3da000640 1 --2- 172.21.15.22:0/67332274 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56265d2408e0 0x56265d242d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-28T18:07:03.331 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.330+0000 7fb3e2a00640 1 --2- 172.21.15.22:0/67332274 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56265d23b770 0x56265d240270 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7fb3d4009fd0 tx=0x7fb3d4010880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9f413c71eacc3dda server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:07:03.331 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.330+0000 7fb3dbe00640 1 --2- 172.21.15.22:0/4085968283 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7fb3c80a2880 0x7fb3c80a4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:07:03.334 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.331+0000 7fb3d3400640 1 -- 172.21.15.22:0/67332274 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3d4016070 con 0x56265d23b770 2024-03-28T18:07:03.334 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.331+0000 7fb3dbe00640 1 -- 172.21.15.22:0/4085968283 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 8153+0+0 (secure 0 0 0) 0x7fb3cc0f0bb0 con 0x7fb3e4102790 2024-03-28T18:07:03.334 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.331+0000 7fb3d3400640 1 -- 172.21.15.22:0/67332274 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7fb3d4003b70 con 0x56265d23b770 2024-03-28T18:07:03.334 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.331+0000 7fb3e3400640 1 --2- 172.21.15.22:0/4085968283 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7fb3c80a2880 0x7fb3c80a4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:07:03.334 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.331+0000 7fb3d3400640 1 -- 172.21.15.22:0/67332274 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3d4012940 con 0x56265d23b770 2024-03-28T18:07:03.334 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.331+0000 7fb3ec403740 1 -- 172.21.15.22:0/67332274 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56265d23b770 msgr2=0x56265d240270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:07:03.334 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.331+0000 7fb3e3400640 1 --2- 172.21.15.22:0/4085968283 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7fb3c80a2880 0x7fb3c80a4d40 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fb3dc0095f0 tx=0x7fb3dc005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:07:03.334 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.331+0000 7fb3ec403740 1 --2- 172.21.15.22:0/67332274 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56265d23b770 0x56265d240270 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7fb3d4009fd0 tx=0x7fb3d4010880 comp rx=0 tx=0).stop 2024-03-28T18:07:03.335 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.332+0000 7fb3ec403740 1 -- 172.21.15.22:0/67332274 shutdown_connections 2024-03-28T18:07:03.335 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.332+0000 7fb3ec403740 1 --2- 172.21.15.22:0/67332274 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56265d2408e0 0x56265d242d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:07:03.335 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.332+0000 7fb3ec403740 1 --2- 172.21.15.22:0/67332274 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56265d23b770 0x56265d240270 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:07:03.335 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.332+0000 7fb3ec403740 1 --2- 172.21.15.22:0/67332274 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56265d23ad80 0x56265d23b1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:07:03.335 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.332+0000 7fb3ec403740 1 -- 172.21.15.22:0/67332274 >> 172.21.15.22:0/67332274 conn(0x56265d2362f0 msgr2=0x56265d2366f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:07:03.335 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.332+0000 7fb3ec403740 1 -- 172.21.15.22:0/67332274 shutdown_connections 2024-03-28T18:07:03.335 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.332+0000 7fb3ec403740 1 -- 172.21.15.22:0/67332274 wait complete. 2024-03-28T18:07:03.335 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.332+0000 7fb3ec403740 1 -- start start 2024-03-28T18:07:03.335 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.332+0000 7fb3ec403740 1 Processor -- start 2024-03-28T18:07:03.335 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.333+0000 7fb3ec403740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56265d23ad80 0x56265d245290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:07:03.335 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.333+0000 7fb3ec403740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56265d23b770 0x56265d2457d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:07:03.335 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.333+0000 7fb3ec403740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56265d2408e0 0x56265d24c810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:07:03.335 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.333+0000 7fb3ec403740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x56265d243890 con 0x56265d23b770 2024-03-28T18:07:03.335 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.333+0000 7fb3ec403740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x56265d15bdd0 con 0x56265d23ad80 2024-03-28T18:07:03.336 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.333+0000 7fb3ec403740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x56265d243a10 con 0x56265d2408e0 2024-03-28T18:07:03.336 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.333+0000 7fb3ec403740 10 client.0 _ll_register_callbacks cb 0x56265ceb3ac0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-28T18:07:03.336 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.333+0000 7fb3da000640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56265d23b770 0x56265d2457d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:07:03.336 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.333+0000 7fb3da000640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56265d23b770 0x56265d2457d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:39598/0 (socket says 172.21.15.22:39598) 2024-03-28T18:07:03.336 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.333+0000 7fb3e2a00640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56265d23ad80 0x56265d245290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:07:03.336 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.333+0000 7fb3da000640 1 -- 172.21.15.22:0/2167578768 learned_addr learned my addr 172.21.15.22:0/2167578768 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-28T18:07:03.336 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.333+0000 7fb3e3400640 1 --2- 172.21.15.22:0/2167578768 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56265d2408e0 0x56265d24c810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:07:03.336 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.333+0000 7fb3da000640 1 -- 172.21.15.22:0/2167578768 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56265d2408e0 msgr2=0x56265d24c810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T18:07:03.336 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.333+0000 7fb3da000640 1 --2- 172.21.15.22:0/2167578768 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56265d2408e0 0x56265d24c810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:07:03.336 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.333+0000 7fb3da000640 1 -- 172.21.15.22:0/2167578768 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56265d23ad80 msgr2=0x56265d245290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T18:07:03.336 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.333+0000 7fb3da000640 1 --2- 172.21.15.22:0/2167578768 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56265d23ad80 0x56265d245290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:07:03.336 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.334+0000 7fb3da000640 1 -- 172.21.15.22:0/2167578768 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56265ceb42d0 con 0x56265d23b770 2024-03-28T18:07:03.336 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.334+0000 7fb3e3400640 1 --2- 172.21.15.22:0/2167578768 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56265d2408e0 0x56265d24c810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T18:07:03.336 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.334+0000 7fb3da000640 1 --2- 172.21.15.22:0/2167578768 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56265d23b770 0x56265d2457d0 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7fb3cc0b9a50 tx=0x7fb3cc003ea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c106fa785e33252c server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:07:03.336 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.334+0000 7fb3c7e00640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-03-28T18:07:03.337 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.334+0000 7fb3c7e00640 1 -- 172.21.15.22:0/2167578768 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3cc0bd130 con 0x56265d23b770 2024-03-28T18:07:03.337 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.334+0000 7fb3c7e00640 1 -- 172.21.15.22:0/2167578768 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 8153+0+0 (secure 0 0 0) 0x7fb3cc0bb8f0 con 0x56265d23b770 2024-03-28T18:07:03.337 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.334+0000 7fb3ec403740 1 -- 172.21.15.22:0/2167578768 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fb3d4003b70 con 0x56265d23b770 2024-03-28T18:07:03.337 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.334+0000 7fb3c7e00640 1 -- 172.21.15.22:0/2167578768 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb3cc0bcf50 con 0x56265d23b770 2024-03-28T18:07:03.337 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.334+0000 7fb3ec403740 10 client.15288 fetch_fsmap learned FSMap version 26 2024-03-28T18:07:03.337 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.334+0000 7fb3ec403740 1 -- 172.21.15.22:0/2167578768 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x56265cf7e6b0 con 0x56265d23b770 2024-03-28T18:07:03.337 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.335+0000 7fb3c7e00640 1 -- 172.21.15.22:0/2167578768 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7fb3d4003b70 con 0x56265d23b770 2024-03-28T18:07:03.339 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.335+0000 7fb3c7e00640 1 -- 172.21.15.22:0/2167578768 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 26) v1 ==== 7934+0+0 (secure 0 0 0) 0x7fb3cc0c5900 con 0x56265d23b770 2024-03-28T18:07:03.339 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.335+0000 7fb3ec403740 10 client.15288 fetch_fsmap finished waiting for FSMap version 26 2024-03-28T18:07:03.339 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.335+0000 7fb3ec403740 10 client.15288 resolve_mds: resolved name 'k' to daemon mds.k 2024-03-28T18:07:03.339 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.335+0000 7fb3ec403740 20 client.15288 populate_metadata read hostname 'smithi022' 2024-03-28T18:07:03.339 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.335+0000 7fb3ec403740 1 --2- 172.21.15.22:0/2167578768 >> [v2:172.21.15.79:6832/856896997,v1:172.21.15.79:6833/856896997] conn(0x56265d251ae0 0x56265d250970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:07:03.339 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.335+0000 7fb3ec403740 4 client.15288 mds_command: new command op to 34442 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-28T18:07:03.339 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.335+0000 7fb3ec403740 1 -- 172.21.15.22:0/2167578768 --> [v2:172.21.15.79:6832/856896997,v1:172.21.15.79:6833/856896997] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56265d141520 con 0x56265d251ae0 2024-03-28T18:07:03.339 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.335+0000 7fb3e2a00640 1 --2- 172.21.15.22:0/2167578768 >> [v2:172.21.15.79:6832/856896997,v1:172.21.15.79:6833/856896997] conn(0x56265d251ae0 0x56265d250970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:07:03.339 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.336+0000 7fb3e2a00640 1 --2- 172.21.15.22:0/2167578768 >> [v2:172.21.15.79:6832/856896997,v1:172.21.15.79:6833/856896997] conn(0x56265d251ae0 0x56265d250970 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=a01317438a6ef566 server_cookie=b790ea60b7d62c86 in_seq=0 out_seq=0 2024-03-28T18:07:03.340 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.336+0000 7fb3c7e00640 10 client.15288 ms_handle_connect on v2:172.21.15.79:6832/856896997 2024-03-28T18:07:03.340 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.337+0000 7fb3c7e00640 1 -- 172.21.15.22:0/2167578768 <== mds.3 v2:172.21.15.79:6832/856896997 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x56265d141520 con 0x56265d251ae0 2024-03-28T18:07:03.340 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.338+0000 7fb3c7e00640 10 client.15288 handle_command_reply: tid=0 2024-03-28T18:07:03.340 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.338+0000 7fb3ec403740 1 client.15288 shutdown 2024-03-28T18:07:03.340 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.338+0000 7fb3ec403740 1 -- 172.21.15.22:0/2167578768 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56265d23b770 msgr2=0x56265d2457d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:07:03.340 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.338+0000 7fb3ec403740 1 --2- 172.21.15.22:0/2167578768 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56265d23b770 0x56265d2457d0 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7fb3cc0b9a50 tx=0x7fb3cc003ea0 comp rx=0 tx=0).stop 2024-03-28T18:07:03.340 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.338+0000 7fb3ec403740 1 -- 172.21.15.22:0/2167578768 shutdown_connections 2024-03-28T18:07:03.340 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.338+0000 7fb3ec403740 1 --2- 172.21.15.22:0/2167578768 >> [v2:172.21.15.79:6832/856896997,v1:172.21.15.79:6833/856896997] conn(0x56265d251ae0 0x56265d250970 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:07:03.340 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.338+0000 7fb3ec403740 1 --2- 172.21.15.22:0/2167578768 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56265d2408e0 0x56265d24c810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:07:03.340 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.338+0000 7fb3ec403740 1 --2- 172.21.15.22:0/2167578768 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56265d23b770 0x56265d2457d0 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:07:03.340 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.338+0000 7fb3ec403740 1 --2- 172.21.15.22:0/2167578768 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56265d23ad80 0x56265d245290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:07:03.340 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.338+0000 7fb3ec403740 1 -- 172.21.15.22:0/2167578768 >> 172.21.15.22:0/2167578768 conn(0x56265d2362f0 msgr2=0x56265d2354f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:07:03.340 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.338+0000 7fb3c7e00640 0 client.15288 ms_handle_reset on v2:172.21.15.79:6832/856896997 2024-03-28T18:07:03.340 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.338+0000 7fb3ec403740 1 -- 172.21.15.22:0/2167578768 shutdown_connections 2024-03-28T18:07:03.340 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.338+0000 7fb3ec403740 1 -- 172.21.15.22:0/2167578768 wait complete. 2024-03-28T18:07:03.340 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.338+0000 7fb3ec403740 20 client.15288 trim_cache size 0 max 16384 2024-03-28T18:07:03.358 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.357+0000 7fb3ec403740 1 Processor -- start 2024-03-28T18:07:03.358 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.357+0000 7fb3ec403740 1 -- start start 2024-03-28T18:07:03.358 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.357+0000 7fb3ec403740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56265d23ad80 0x56265d2366d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:07:03.358 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.357+0000 7fb3ec403740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56265d23b770 0x56265d24c7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:07:03.358 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.357+0000 7fb3ec403740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56265d2408e0 0x56265d23cc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:07:03.359 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.357+0000 7fb3ec403740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x56265d251300 con 0x56265d23ad80 2024-03-28T18:07:03.359 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.357+0000 7fb3ec403740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x56265d251960 con 0x56265d23b770 2024-03-28T18:07:03.359 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.357+0000 7fb3ec403740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x56265d251fc0 con 0x56265d2408e0 2024-03-28T18:07:03.359 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.357+0000 7fb3e2a00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56265d23ad80 0x56265d2366d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:07:03.359 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.358+0000 7fb3e2a00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56265d23ad80 0x56265d2366d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:39600/0 (socket says 172.21.15.22:39600) 2024-03-28T18:07:03.359 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.358+0000 7fb3e2a00640 1 -- 172.21.15.22:0/2337570151 learned_addr learned my addr 172.21.15.22:0/2337570151 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-28T18:07:03.359 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.358+0000 7fb3da000640 1 --2- 172.21.15.22:0/2337570151 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56265d23b770 0x56265d24c7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:07:03.359 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.358+0000 7fb3e3400640 1 --2- 172.21.15.22:0/2337570151 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56265d2408e0 0x56265d23cc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:07:03.359 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.358+0000 7fb3e2a00640 1 -- 172.21.15.22:0/2337570151 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56265d2408e0 msgr2=0x56265d23cc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T18:07:03.359 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.358+0000 7fb3e2a00640 1 --2- 172.21.15.22:0/2337570151 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56265d2408e0 0x56265d23cc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:07:03.359 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.358+0000 7fb3e2a00640 1 -- 172.21.15.22:0/2337570151 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56265d23b770 msgr2=0x56265d24c7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T18:07:03.359 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.358+0000 7fb3e2a00640 1 --2- 172.21.15.22:0/2337570151 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56265d23b770 0x56265d24c7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:07:03.360 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.358+0000 7fb3e2a00640 1 -- 172.21.15.22:0/2337570151 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56265d23c000 con 0x56265d23ad80 2024-03-28T18:07:03.360 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.358+0000 7fb3e3400640 1 --2- 172.21.15.22:0/2337570151 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56265d2408e0 0x56265d23cc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T18:07:03.360 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.358+0000 7fb3e2a00640 1 --2- 172.21.15.22:0/2337570151 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56265d23ad80 0x56265d2366d0 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7fb3d4009fd0 tx=0x7fb3d4009680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=43635eba5433db7c server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:07:03.360 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.358+0000 7fb3d2a00640 1 -- 172.21.15.22:0/2337570151 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3d4016040 con 0x56265d23ad80 2024-03-28T18:07:03.360 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.359+0000 7fb3d2a00640 1 -- 172.21.15.22:0/2337570151 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7fb3d40282d0 con 0x56265d23ad80 2024-03-28T18:07:03.360 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.359+0000 7fb3d2a00640 1 -- 172.21.15.22:0/2337570151 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3d40050d0 con 0x56265d23ad80 2024-03-28T18:07:03.364 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.359+0000 7fb3ec403740 1 -- 172.21.15.22:0/2337570151 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56265d23ad80 msgr2=0x56265d2366d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:07:03.364 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.359+0000 7fb3ec403740 1 --2- 172.21.15.22:0/2337570151 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56265d23ad80 0x56265d2366d0 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7fb3d4009fd0 tx=0x7fb3d4009680 comp rx=0 tx=0).stop 2024-03-28T18:07:03.365 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.359+0000 7fb3ec403740 1 -- 172.21.15.22:0/2337570151 shutdown_connections 2024-03-28T18:07:03.365 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.359+0000 7fb3ec403740 1 --2- 172.21.15.22:0/2337570151 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56265d2408e0 0x56265d23cc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:07:03.365 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.359+0000 7fb3ec403740 1 --2- 172.21.15.22:0/2337570151 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56265d23b770 0x56265d24c7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:07:03.365 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.359+0000 7fb3ec403740 1 --2- 172.21.15.22:0/2337570151 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56265d23ad80 0x56265d2366d0 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:07:03.365 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.359+0000 7fb3ec403740 1 -- 172.21.15.22:0/2337570151 >> 172.21.15.22:0/2337570151 conn(0x56265d2362f0 msgr2=0x56265d243e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:07:03.365 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.359+0000 7fb3ec403740 1 -- 172.21.15.22:0/2337570151 shutdown_connections 2024-03-28T18:07:03.365 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.359+0000 7fb3ec403740 1 -- 172.21.15.22:0/2337570151 wait complete. 2024-03-28T18:07:03.365 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.360+0000 7fb3ec403740 1 -- start start 2024-03-28T18:07:03.365 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.360+0000 7fb3ec403740 1 Processor -- start 2024-03-28T18:07:03.365 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.360+0000 7fb3ec403740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56265d23ad80 0x56265d248460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:07:03.365 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.360+0000 7fb3ec403740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56265d23b770 0x56265d2489a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:07:03.365 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.360+0000 7fb3ec403740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56265d2408e0 0x56265d24b970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:07:03.365 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.360+0000 7fb3ec403740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x56265d252c80 con 0x56265d2408e0 2024-03-28T18:07:03.366 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.360+0000 7fb3ec403740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x56265d252620 con 0x56265d23b770 2024-03-28T18:07:03.366 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.360+0000 7fb3ec403740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x56265d2532e0 con 0x56265d23ad80 2024-03-28T18:07:03.366 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.360+0000 7fb3ec403740 10 client.0 _ll_register_callbacks cb 0x7fb3cc009260 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-28T18:07:03.366 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.361+0000 7fb3e3400640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56265d2408e0 0x56265d24b970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:07:03.366 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.361+0000 7fb3e3400640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56265d2408e0 0x56265d24b970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:39614/0 (socket says 172.21.15.22:39614) 2024-03-28T18:07:03.366 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.361+0000 7fb3e3400640 1 -- 172.21.15.22:0/1740772200 learned_addr learned my addr 172.21.15.22:0/1740772200 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-28T18:07:03.366 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.361+0000 7fb3da000640 1 --2- 172.21.15.22:0/1740772200 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56265d23b770 0x56265d2489a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:07:03.366 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.361+0000 7fb3e2a00640 1 --2- 172.21.15.22:0/1740772200 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56265d23ad80 0x56265d248460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:07:03.366 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.361+0000 7fb3e3400640 1 -- 172.21.15.22:0/1740772200 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56265d23ad80 msgr2=0x56265d248460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T18:07:03.366 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.361+0000 7fb3e3400640 1 --2- 172.21.15.22:0/1740772200 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56265d23ad80 0x56265d248460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:07:03.366 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.361+0000 7fb3e3400640 1 -- 172.21.15.22:0/1740772200 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56265d23b770 msgr2=0x56265d2489a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T18:07:03.366 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.361+0000 7fb3e3400640 1 --2- 172.21.15.22:0/1740772200 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56265d23b770 0x56265d2489a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:07:03.366 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.361+0000 7fb3e3400640 1 -- 172.21.15.22:0/1740772200 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56265d24bf60 con 0x56265d2408e0 2024-03-28T18:07:03.366 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.361+0000 7fb3e3400640 1 --2- 172.21.15.22:0/1740772200 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56265d2408e0 0x56265d24b970 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7fb3dc0095c0 tx=0x7fb3dc0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2ef2df215a6357a4 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:07:03.366 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.361+0000 7fb3c7e00640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-03-28T18:07:03.367 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.362+0000 7fb3c7e00640 1 -- 172.21.15.22:0/1740772200 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3dc01a8d0 con 0x56265d2408e0 2024-03-28T18:07:03.367 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.362+0000 7fb3c7e00640 1 -- 172.21.15.22:0/1740772200 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 8153+0+0 (secure 0 0 0) 0x7fb3dc01aa70 con 0x56265d2408e0 2024-03-28T18:07:03.367 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.362+0000 7fb3ec403740 1 -- 172.21.15.22:0/1740772200 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fb3d40282d0 con 0x56265d2408e0 2024-03-28T18:07:03.367 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.362+0000 7fb3c7e00640 1 -- 172.21.15.22:0/1740772200 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb3dc01ac70 con 0x56265d2408e0 2024-03-28T18:07:03.367 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.362+0000 7fb3ec403740 10 client.15294 fetch_fsmap learned FSMap version 26 2024-03-28T18:07:03.367 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.362+0000 7fb3ec403740 1 -- 172.21.15.22:0/1740772200 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x56265d2ad070 con 0x56265d2408e0 2024-03-28T18:07:03.367 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.362+0000 7fb3c7e00640 1 -- 172.21.15.22:0/1740772200 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7fb3d40282d0 con 0x56265d2408e0 2024-03-28T18:07:03.367 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.362+0000 7fb3c7e00640 1 -- 172.21.15.22:0/1740772200 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 26) v1 ==== 7934+0+0 (secure 0 0 0) 0x7fb3dc019040 con 0x56265d2408e0 2024-03-28T18:07:03.367 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.362+0000 7fb3ec403740 10 client.15294 fetch_fsmap finished waiting for FSMap version 26 2024-03-28T18:07:03.367 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.363+0000 7fb3ec403740 10 client.15294 resolve_mds: resolved name 'k' to daemon mds.k 2024-03-28T18:07:03.367 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.363+0000 7fb3ec403740 20 client.15294 populate_metadata read hostname 'smithi022' 2024-03-28T18:07:03.367 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.363+0000 7fb3ec403740 1 --2- 172.21.15.22:0/1740772200 >> [v2:172.21.15.79:6832/856896997,v1:172.21.15.79:6833/856896997] conn(0x56265d2b1850 0x56265d2ad3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:07:03.367 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.363+0000 7fb3ec403740 4 client.15294 mds_command: new command op to 34442 tid=0 multi_id=0 [{"prefix": "status"}] 2024-03-28T18:07:03.367 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.363+0000 7fb3ec403740 1 -- 172.21.15.22:0/1740772200 --> [v2:172.21.15.79:6832/856896997,v1:172.21.15.79:6833/856896997] -- command(tid 0: {"prefix": "status"}) v1 -- 0x56265d1ad1d0 con 0x56265d2b1850 2024-03-28T18:07:03.367 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.363+0000 7fb3e2a00640 1 --2- 172.21.15.22:0/1740772200 >> [v2:172.21.15.79:6832/856896997,v1:172.21.15.79:6833/856896997] conn(0x56265d2b1850 0x56265d2ad3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:07:03.367 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.364+0000 7fb3e2a00640 1 --2- 172.21.15.22:0/1740772200 >> [v2:172.21.15.79:6832/856896997,v1:172.21.15.79:6833/856896997] conn(0x56265d2b1850 0x56265d2ad3b0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=ce30d77418620785 server_cookie=624683727da743a6 in_seq=0 out_seq=0 2024-03-28T18:07:03.368 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.364+0000 7fb3c7e00640 10 client.15294 ms_handle_connect on v2:172.21.15.79:6832/856896997 2024-03-28T18:07:03.368 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.364+0000 7fb3c7e00640 1 -- 172.21.15.22:0/1740772200 <== mds.3 v2:172.21.15.79:6832/856896997 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+322 (crc 0 0 0) 0x56265d1ad1d0 con 0x56265d2b1850 2024-03-28T18:07:03.368 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.364+0000 7fb3c7e00640 10 client.15294 handle_command_reply: tid=0 2024-03-28T18:07:03.368 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.364+0000 7fb3ec403740 1 client.15294 shutdown 2024-03-28T18:07:03.368 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.364+0000 7fb3ec403740 1 -- 172.21.15.22:0/1740772200 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56265d2408e0 msgr2=0x56265d24b970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:07:03.368 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.364+0000 7fb3ec403740 1 --2- 172.21.15.22:0/1740772200 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56265d2408e0 0x56265d24b970 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7fb3dc0095c0 tx=0x7fb3dc0046c0 comp rx=0 tx=0).stop 2024-03-28T18:07:03.368 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.364+0000 7fb3ec403740 1 -- 172.21.15.22:0/1740772200 shutdown_connections 2024-03-28T18:07:03.368 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.364+0000 7fb3ec403740 1 --2- 172.21.15.22:0/1740772200 >> [v2:172.21.15.79:6832/856896997,v1:172.21.15.79:6833/856896997] conn(0x56265d2b1850 0x56265d2ad3b0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:07:03.368 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.364+0000 7fb3ec403740 1 --2- 172.21.15.22:0/1740772200 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56265d2408e0 0x56265d24b970 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:07:03.368 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.364+0000 7fb3ec403740 1 --2- 172.21.15.22:0/1740772200 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56265d23b770 0x56265d2489a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:07:03.368 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.364+0000 7fb3ec403740 1 --2- 172.21.15.22:0/1740772200 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56265d23ad80 0x56265d248460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:07:03.368 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.364+0000 7fb3c7e00640 0 client.15294 ms_handle_reset on v2:172.21.15.79:6832/856896997 2024-03-28T18:07:03.368 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.364+0000 7fb3ec403740 1 -- 172.21.15.22:0/1740772200 >> 172.21.15.22:0/1740772200 conn(0x56265d2362f0 msgr2=0x56265d2366d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:07:03.368 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.365+0000 7fb3ec403740 1 -- 172.21.15.22:0/1740772200 shutdown_connections 2024-03-28T18:07:03.368 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.365+0000 7fb3ec403740 1 -- 172.21.15.22:0/1740772200 wait complete. 2024-03-28T18:07:03.368 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.365+0000 7fb3ec403740 20 client.15294 trim_cache size 0 max 16384 2024-03-28T18:07:03.369 INFO:teuthology.orchestra.run.smithi022.stdout:{ 2024-03-28T18:07:03.369 INFO:teuthology.orchestra.run.smithi022.stdout: "cluster_fsid": "fd220748-ed2b-11ee-95d0-87774f69a715", 2024-03-28T18:07:03.369 INFO:teuthology.orchestra.run.smithi022.stdout: "whoami": 3, 2024-03-28T18:07:03.369 INFO:teuthology.orchestra.run.smithi022.stdout: "id": 34442, 2024-03-28T18:07:03.369 INFO:teuthology.orchestra.run.smithi022.stdout: "want_state": "up:active", 2024-03-28T18:07:03.369 INFO:teuthology.orchestra.run.smithi022.stdout: "state": "up:active", 2024-03-28T18:07:03.369 INFO:teuthology.orchestra.run.smithi022.stdout: "fs_name": "cephfs", 2024-03-28T18:07:03.369 INFO:teuthology.orchestra.run.smithi022.stdout: "rank_uptime": 128.757880599, 2024-03-28T18:07:03.369 INFO:teuthology.orchestra.run.smithi022.stdout: "mdsmap_epoch": 26, 2024-03-28T18:07:03.369 INFO:teuthology.orchestra.run.smithi022.stdout: "osdmap_epoch": 92, 2024-03-28T18:07:03.369 INFO:teuthology.orchestra.run.smithi022.stdout: "osdmap_epoch_barrier": 90, 2024-03-28T18:07:03.369 INFO:teuthology.orchestra.run.smithi022.stdout: "uptime": 179.095804706 2024-03-28T18:07:03.370 INFO:teuthology.orchestra.run.smithi022.stdout:} 2024-03-28T18:07:03.370 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.365+0000 7fb3e3e00640 1 -- 172.21.15.22:0/4085968283 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7fb3c80a2880 msgr2=0x7fb3c80a4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:07:03.370 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.365+0000 7fb3e3e00640 1 --2- 172.21.15.22:0/4085968283 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7fb3c80a2880 0x7fb3c80a4d40 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fb3dc0095f0 tx=0x7fb3dc005f90 comp rx=0 tx=0).stop 2024-03-28T18:07:03.370 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.365+0000 7fb3e3e00640 1 -- 172.21.15.22:0/4085968283 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb3e4102790 msgr2=0x7fb3e4073a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:07:03.370 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.365+0000 7fb3e3e00640 1 --2- 172.21.15.22:0/4085968283 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb3e4102790 0x7fb3e4073a90 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7fb3cc000c00 tx=0x7fb3cc004e20 comp rx=0 tx=0).stop 2024-03-28T18:07:03.370 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.365+0000 7fb3e3e00640 1 -- 172.21.15.22:0/4085968283 shutdown_connections 2024-03-28T18:07:03.370 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.365+0000 7fb3e3e00640 1 --2- 172.21.15.22:0/4085968283 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7fb3c80a2880 0x7fb3c80a4d40 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:07:03.371 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.365+0000 7fb3e3e00640 1 --2- 172.21.15.22:0/4085968283 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb3e4103150 0x7fb3e4073fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:07:03.371 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.365+0000 7fb3e3e00640 1 --2- 172.21.15.22:0/4085968283 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb3e4102790 0x7fb3e4073a90 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:07:03.371 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.365+0000 7fb3e3e00640 1 --2- 172.21.15.22:0/4085968283 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fb3e41015a0 0x7fb3e4075450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:07:03.371 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.365+0000 7fb3e3e00640 1 -- 172.21.15.22:0/4085968283 >> 172.21.15.22:0/4085968283 conn(0x7fb3e40fcb10 msgr2=0x7fb3e4105a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:07:03.371 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.366+0000 7fb3e3e00640 1 -- 172.21.15.22:0/4085968283 shutdown_connections 2024-03-28T18:07:03.371 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.366+0000 7fb3e3e00640 1 -- 172.21.15.22:0/4085968283 wait complete. 2024-03-28T18:07:03.380 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:07:03 smithi022 ceph-mon[31985]: from='client.? 172.21.15.22:0/1930933532' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-28T18:07:03.380 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:07:03 smithi022 ceph-mon[31985]: from='client.? 172.21.15.22:0/3429492398' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-28T18:07:03.384 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.e status 2024-03-28T18:07:03.613 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.613+0000 7fa668800640 1 -- 172.21.15.22:0/382000615 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa660104ad0 msgr2=0x7fa660104f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:07:03.613 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.613+0000 7fa668800640 1 --2- 172.21.15.22:0/382000615 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa660104ad0 0x7fa660104f50 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7fa650009900 tx=0x7fa65002bc30 comp rx=0 tx=0).stop 2024-03-28T18:07:03.613 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.613+0000 7fa668800640 1 -- 172.21.15.22:0/382000615 shutdown_connections 2024-03-28T18:07:03.613 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.613+0000 7fa668800640 1 --2- 172.21.15.22:0/382000615 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa660105490 0x7fa66010bd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:07:03.613 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.613+0000 7fa668800640 1 --2- 172.21.15.22:0/382000615 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa660104ad0 0x7fa660104f50 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:07:03.613 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.613+0000 7fa668800640 1 --2- 172.21.15.22:0/382000615 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa6601038e0 0x7fa660103d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:07:03.613 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.613+0000 7fa668800640 1 -- 172.21.15.22:0/382000615 >> 172.21.15.22:0/382000615 conn(0x7fa6600fee50 msgr2=0x7fa660101290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:07:03.613 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.613+0000 7fa668800640 1 -- 172.21.15.22:0/382000615 shutdown_connections 2024-03-28T18:07:03.613 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.613+0000 7fa668800640 1 -- 172.21.15.22:0/382000615 wait complete. 2024-03-28T18:07:03.615 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.614+0000 7fa668800640 1 Processor -- start 2024-03-28T18:07:03.616 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.614+0000 7fa668800640 1 -- start start 2024-03-28T18:07:03.616 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.614+0000 7fa668800640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa6601038e0 0x7fa660199c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:07:03.616 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.614+0000 7fa668800640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa660104ad0 0x7fa66019a1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:07:03.616 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.614+0000 7fa668800640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa660105490 0x7fa6601a11d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:07:03.616 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.614+0000 7fa668800640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa66010f540 con 0x7fa660105490 2024-03-28T18:07:03.616 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.614+0000 7fa668800640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa66010f3c0 con 0x7fa6601038e0 2024-03-28T18:07:03.616 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.614+0000 7fa668800640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa66010f6c0 con 0x7fa660104ad0 2024-03-28T18:07:03.616 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.615+0000 7fa666800640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa660105490 0x7fa6601a11d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:07:03.616 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.615+0000 7fa666800640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa660105490 0x7fa6601a11d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:39630/0 (socket says 172.21.15.22:39630) 2024-03-28T18:07:03.616 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.615+0000 7fa666800640 1 -- 172.21.15.22:0/423701398 learned_addr learned my addr 172.21.15.22:0/423701398 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-28T18:07:03.616 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.615+0000 7fa665e00640 1 --2- 172.21.15.22:0/423701398 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa6601038e0 0x7fa660199c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:07:03.616 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.615+0000 7fa665400640 1 --2- 172.21.15.22:0/423701398 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa660104ad0 0x7fa66019a1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:07:03.616 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.615+0000 7fa666800640 1 -- 172.21.15.22:0/423701398 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa660104ad0 msgr2=0x7fa66019a1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:07:03.617 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.615+0000 7fa666800640 1 --2- 172.21.15.22:0/423701398 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa660104ad0 0x7fa66019a1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:07:03.617 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.615+0000 7fa666800640 1 -- 172.21.15.22:0/423701398 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa6601038e0 msgr2=0x7fa660199c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:07:03.617 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.615+0000 7fa666800640 1 --2- 172.21.15.22:0/423701398 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa6601038e0 0x7fa660199c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:07:03.617 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.615+0000 7fa666800640 1 -- 172.21.15.22:0/423701398 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa6601a1870 con 0x7fa660105490 2024-03-28T18:07:03.617 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.615+0000 7fa665e00640 1 --2- 172.21.15.22:0/423701398 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa6601038e0 0x7fa660199c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-28T18:07:03.617 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.615+0000 7fa666800640 1 --2- 172.21.15.22:0/423701398 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa660105490 0x7fa6601a11d0 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7fa65c00eb10 tx=0x7fa65c00c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:07:03.617 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.616+0000 7fa65aa00640 1 -- 172.21.15.22:0/423701398 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa65c004b20 con 0x7fa660105490 2024-03-28T18:07:03.617 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.616+0000 7fa65aa00640 1 -- 172.21.15.22:0/423701398 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7fa65c004cc0 con 0x7fa660105490 2024-03-28T18:07:03.617 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.616+0000 7fa65aa00640 1 -- 172.21.15.22:0/423701398 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa65c010700 con 0x7fa660105490 2024-03-28T18:07:03.617 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.616+0000 7fa668800640 1 -- 172.21.15.22:0/423701398 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa6601a1b60 con 0x7fa660105490 2024-03-28T18:07:03.617 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.616+0000 7fa668800640 1 -- 172.21.15.22:0/423701398 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa6601a2060 con 0x7fa660105490 2024-03-28T18:07:03.619 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.619+0000 7fa65aa00640 1 -- 172.21.15.22:0/423701398 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 18) v1 ==== 137290+0+0 (secure 0 0 0) 0x7fa65c004e70 con 0x7fa660105490 2024-03-28T18:07:03.619 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.619+0000 7fa66b58d740 1 Processor -- start 2024-03-28T18:07:03.619 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.620+0000 7fa66b58d740 1 -- start start 2024-03-28T18:07:03.621 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.620+0000 7fa66b58d740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55707524cba0 0x55707524cfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:07:03.621 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.620+0000 7fa66b58d740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55707524d590 0x557075252090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:07:03.621 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.620+0000 7fa66b58d740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x557075252700 0x557075254b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:07:03.621 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.620+0000 7fa66b58d740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x5570751b1c90 con 0x55707524cba0 2024-03-28T18:07:03.621 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.620+0000 7fa66b58d740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x55707515a980 con 0x557075252700 2024-03-28T18:07:03.621 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.620+0000 7fa66b58d740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x5570751add30 con 0x55707524d590 2024-03-28T18:07:03.622 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.620+0000 7fa665400640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55707524cba0 0x55707524cfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:07:03.622 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.620+0000 7fa665400640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55707524cba0 0x55707524cfc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:39634/0 (socket says 172.21.15.22:39634) 2024-03-28T18:07:03.622 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.620+0000 7fa665400640 1 -- 172.21.15.22:0/2735302947 learned_addr learned my addr 172.21.15.22:0/2735302947 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-28T18:07:03.622 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.621+0000 7fa665400640 1 --2- 172.21.15.22:0/2735302947 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x557075252700 0x557075254b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:07:03.622 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.621+0000 7fa665e00640 1 --2- 172.21.15.22:0/2735302947 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55707524d590 0x557075252090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:07:03.622 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.621+0000 7fa665400640 1 -- 172.21.15.22:0/2735302947 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55707524d590 msgr2=0x557075252090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T18:07:03.622 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.621+0000 7fa665400640 1 --2- 172.21.15.22:0/2735302947 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55707524d590 0x557075252090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:07:03.622 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.621+0000 7fa665400640 1 -- 172.21.15.22:0/2735302947 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x557075252700 msgr2=0x557075254b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T18:07:03.622 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.621+0000 7fa665400640 1 --2- 172.21.15.22:0/2735302947 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x557075252700 0x557075254b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:07:03.622 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.621+0000 7fa665400640 1 -- 172.21.15.22:0/2735302947 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557075172510 con 0x55707524cba0 2024-03-28T18:07:03.622 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.621+0000 7fa665e00640 1 --2- 172.21.15.22:0/2735302947 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55707524d590 0x557075252090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T18:07:03.622 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.621+0000 7fa65aa00640 1 --2- 172.21.15.22:0/423701398 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7fa6480a2800 0x7fa6480a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:07:03.622 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.621+0000 7fa665400640 1 --2- 172.21.15.22:0/2735302947 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55707524cba0 0x55707524cfc0 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7fa650005ab0 tx=0x7fa650005a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d1272d26817127a8 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:07:03.622 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.621+0000 7fa65aa00640 1 -- 172.21.15.22:0/423701398 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 8153+0+0 (secure 0 0 0) 0x7fa65c0cf110 con 0x7fa660105490 2024-03-28T18:07:03.623 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.621+0000 7fa665e00640 1 --2- 172.21.15.22:0/423701398 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7fa6480a2800 0x7fa6480a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:07:03.623 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.622+0000 7fa64f000640 1 -- 172.21.15.22:0/2735302947 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa650048070 con 0x55707524cba0 2024-03-28T18:07:03.623 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.622+0000 7fa64f000640 1 -- 172.21.15.22:0/2735302947 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7fa650005040 con 0x55707524cba0 2024-03-28T18:07:03.623 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.622+0000 7fa64f000640 1 -- 172.21.15.22:0/2735302947 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6500048c0 con 0x55707524cba0 2024-03-28T18:07:03.626 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.622+0000 7fa66b58d740 1 -- 172.21.15.22:0/2735302947 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55707524cba0 msgr2=0x55707524cfc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:07:03.626 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.622+0000 7fa665e00640 1 --2- 172.21.15.22:0/423701398 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7fa6480a2800 0x7fa6480a4cc0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x55707524d990 tx=0x7fa654005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:07:03.626 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.622+0000 7fa66b58d740 1 --2- 172.21.15.22:0/2735302947 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55707524cba0 0x55707524cfc0 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7fa650005ab0 tx=0x7fa650005a40 comp rx=0 tx=0).stop 2024-03-28T18:07:03.626 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.623+0000 7fa66b58d740 1 -- 172.21.15.22:0/2735302947 shutdown_connections 2024-03-28T18:07:03.626 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.623+0000 7fa66b58d740 1 --2- 172.21.15.22:0/2735302947 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x557075252700 0x557075254b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:07:03.627 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.623+0000 7fa66b58d740 1 --2- 172.21.15.22:0/2735302947 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55707524d590 0x557075252090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:07:03.627 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.623+0000 7fa66b58d740 1 --2- 172.21.15.22:0/2735302947 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55707524cba0 0x55707524cfc0 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:07:03.627 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.623+0000 7fa66b58d740 1 -- 172.21.15.22:0/2735302947 >> 172.21.15.22:0/2735302947 conn(0x557075248110 msgr2=0x557075248510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:07:03.627 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.623+0000 7fa66b58d740 1 -- 172.21.15.22:0/2735302947 shutdown_connections 2024-03-28T18:07:03.627 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.623+0000 7fa66b58d740 1 -- 172.21.15.22:0/2735302947 wait complete. 2024-03-28T18:07:03.627 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.623+0000 7fa66b58d740 1 -- start start 2024-03-28T18:07:03.627 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.623+0000 7fa66b58d740 1 Processor -- start 2024-03-28T18:07:03.627 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.624+0000 7fa66b58d740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55707524cba0 0x5570752570e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:07:03.627 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.624+0000 7fa66b58d740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55707524d590 0x557075257620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:07:03.627 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.624+0000 7fa66b58d740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557075252700 0x55707525e660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:07:03.627 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.624+0000 7fa66b58d740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x557075255650 con 0x557075252700 2024-03-28T18:07:03.627 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.624+0000 7fa66b58d740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x55707516ddd0 con 0x55707524cba0 2024-03-28T18:07:03.627 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.624+0000 7fa66b58d740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x5570752557d0 con 0x55707524d590 2024-03-28T18:07:03.628 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.624+0000 7fa66b58d740 10 client.0 _ll_register_callbacks cb 0x557074ec5ac0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-28T18:07:03.628 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.624+0000 7fa666800640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557075252700 0x55707525e660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:07:03.628 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.624+0000 7fa666800640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557075252700 0x55707525e660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:39638/0 (socket says 172.21.15.22:39638) 2024-03-28T18:07:03.628 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.624+0000 7fa666800640 1 -- 172.21.15.22:0/648847012 learned_addr learned my addr 172.21.15.22:0/648847012 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-28T18:07:03.628 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.624+0000 7fa665e00640 1 --2- 172.21.15.22:0/648847012 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55707524cba0 0x5570752570e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:07:03.628 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.624+0000 7fa665400640 1 --2- 172.21.15.22:0/648847012 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55707524d590 0x557075257620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:07:03.628 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.624+0000 7fa666800640 1 -- 172.21.15.22:0/648847012 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55707524d590 msgr2=0x557075257620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T18:07:03.628 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.624+0000 7fa666800640 1 --2- 172.21.15.22:0/648847012 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55707524d590 0x557075257620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:07:03.628 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.625+0000 7fa666800640 1 -- 172.21.15.22:0/648847012 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55707524cba0 msgr2=0x5570752570e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T18:07:03.628 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.625+0000 7fa666800640 1 --2- 172.21.15.22:0/648847012 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55707524cba0 0x5570752570e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:07:03.628 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.625+0000 7fa666800640 1 -- 172.21.15.22:0/648847012 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x557074ec62d0 con 0x557075252700 2024-03-28T18:07:03.628 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.625+0000 7fa666800640 1 --2- 172.21.15.22:0/648847012 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557075252700 0x55707525e660 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7fa65c098490 tx=0x7fa65c09b570 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b03a9133693e1558 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:07:03.628 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.625+0000 7fa643400640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-03-28T18:07:03.628 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.625+0000 7fa643400640 1 -- 172.21.15.22:0/648847012 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa65c0b3020 con 0x557075252700 2024-03-28T18:07:03.629 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.625+0000 7fa643400640 1 -- 172.21.15.22:0/648847012 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 8153+0+0 (secure 0 0 0) 0x7fa65c09bfb0 con 0x557075252700 2024-03-28T18:07:03.629 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.625+0000 7fa66b58d740 1 -- 172.21.15.22:0/648847012 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fa650005040 con 0x557075252700 2024-03-28T18:07:03.629 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.625+0000 7fa643400640 1 -- 172.21.15.22:0/648847012 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa65c09bb50 con 0x557075252700 2024-03-28T18:07:03.629 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.625+0000 7fa66b58d740 10 client.15306 fetch_fsmap learned FSMap version 26 2024-03-28T18:07:03.629 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.625+0000 7fa66b58d740 1 -- 172.21.15.22:0/648847012 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x557074f906b0 con 0x557075252700 2024-03-28T18:07:03.629 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.626+0000 7fa643400640 1 -- 172.21.15.22:0/648847012 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7fa650005040 con 0x557075252700 2024-03-28T18:07:03.629 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.626+0000 7fa643400640 1 -- 172.21.15.22:0/648847012 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 26) v1 ==== 7934+0+0 (secure 0 0 0) 0x7fa65c0ba140 con 0x557075252700 2024-03-28T18:07:03.629 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.626+0000 7fa66b58d740 10 client.15306 fetch_fsmap finished waiting for FSMap version 26 2024-03-28T18:07:03.629 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.626+0000 7fa66b58d740 10 client.15306 resolve_mds: resolved name 'e' to daemon mds.e 2024-03-28T18:07:03.629 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.626+0000 7fa66b58d740 20 client.15306 populate_metadata read hostname 'smithi022' 2024-03-28T18:07:03.629 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.626+0000 7fa66b58d740 1 --2- 172.21.15.22:0/648847012 >> [v2:172.21.15.79:6834/3365773431,v1:172.21.15.79:6835/3365773431] conn(0x557075263890 0x557075262970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:07:03.629 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.626+0000 7fa66b58d740 4 client.15306 mds_command: new command op to 34475 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-28T18:07:03.629 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.626+0000 7fa66b58d740 1 -- 172.21.15.22:0/648847012 --> [v2:172.21.15.79:6834/3365773431,v1:172.21.15.79:6835/3365773431] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x557075153520 con 0x557075263890 2024-03-28T18:07:03.629 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.627+0000 7fa665e00640 1 --2- 172.21.15.22:0/648847012 >> [v2:172.21.15.79:6834/3365773431,v1:172.21.15.79:6835/3365773431] conn(0x557075263890 0x557075262970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:07:03.631 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.627+0000 7fa665e00640 1 --2- 172.21.15.22:0/648847012 >> [v2:172.21.15.79:6834/3365773431,v1:172.21.15.79:6835/3365773431] conn(0x557075263890 0x557075262970 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=56d0adeae152e694 server_cookie=7f9dfa37bab6c756 in_seq=0 out_seq=0 2024-03-28T18:07:03.631 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.627+0000 7fa643400640 10 client.15306 ms_handle_connect on v2:172.21.15.79:6834/3365773431 2024-03-28T18:07:03.631 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.629+0000 7fa643400640 1 -- 172.21.15.22:0/648847012 <== mds.2 v2:172.21.15.79:6834/3365773431 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x557075153520 con 0x557075263890 2024-03-28T18:07:03.631 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.629+0000 7fa643400640 10 client.15306 handle_command_reply: tid=0 2024-03-28T18:07:03.631 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.629+0000 7fa66b58d740 1 client.15306 shutdown 2024-03-28T18:07:03.631 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.629+0000 7fa66b58d740 1 -- 172.21.15.22:0/648847012 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557075252700 msgr2=0x55707525e660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:07:03.631 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.629+0000 7fa66b58d740 1 --2- 172.21.15.22:0/648847012 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557075252700 0x55707525e660 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7fa65c098490 tx=0x7fa65c09b570 comp rx=0 tx=0).stop 2024-03-28T18:07:03.631 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.630+0000 7fa66b58d740 1 -- 172.21.15.22:0/648847012 shutdown_connections 2024-03-28T18:07:03.631 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.630+0000 7fa66b58d740 1 --2- 172.21.15.22:0/648847012 >> [v2:172.21.15.79:6834/3365773431,v1:172.21.15.79:6835/3365773431] conn(0x557075263890 0x557075262970 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:07:03.631 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.630+0000 7fa66b58d740 1 --2- 172.21.15.22:0/648847012 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557075252700 0x55707525e660 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:07:03.631 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.630+0000 7fa66b58d740 1 --2- 172.21.15.22:0/648847012 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55707524d590 0x557075257620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:07:03.631 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.630+0000 7fa66b58d740 1 --2- 172.21.15.22:0/648847012 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55707524cba0 0x5570752570e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:07:03.631 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.630+0000 7fa643400640 0 client.15306 ms_handle_reset on v2:172.21.15.79:6834/3365773431 2024-03-28T18:07:03.632 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.630+0000 7fa66b58d740 1 -- 172.21.15.22:0/648847012 >> 172.21.15.22:0/648847012 conn(0x557075248110 msgr2=0x557075247310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:07:03.632 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.630+0000 7fa66b58d740 1 -- 172.21.15.22:0/648847012 shutdown_connections 2024-03-28T18:07:03.632 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.630+0000 7fa66b58d740 1 -- 172.21.15.22:0/648847012 wait complete. 2024-03-28T18:07:03.632 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.630+0000 7fa66b58d740 20 client.15306 trim_cache size 0 max 16384 2024-03-28T18:07:03.648 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.648+0000 7fa66b58d740 1 Processor -- start 2024-03-28T18:07:03.648 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.648+0000 7fa66b58d740 1 -- start start 2024-03-28T18:07:03.648 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.649+0000 7fa66b58d740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55707524cba0 0x557075254af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:07:03.648 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.649+0000 7fa66b58d740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55707524d590 0x55707525e620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:07:03.648 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.649+0000 7fa66b58d740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557075252700 0x55707524e030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:07:03.649 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.649+0000 7fa66b58d740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x5570752630b0 con 0x557075252700 2024-03-28T18:07:03.649 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.649+0000 7fa66b58d740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x557075263710 con 0x55707524cba0 2024-03-28T18:07:03.649 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.649+0000 7fa66b58d740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x557075263d70 con 0x55707524d590 2024-03-28T18:07:03.651 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.649+0000 7fa666800640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557075252700 0x55707524e030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:07:03.651 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.649+0000 7fa666800640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557075252700 0x55707524e030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:39648/0 (socket says 172.21.15.22:39648) 2024-03-28T18:07:03.651 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.649+0000 7fa666800640 1 -- 172.21.15.22:0/4250020636 learned_addr learned my addr 172.21.15.22:0/4250020636 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-28T18:07:03.651 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.649+0000 7fa665e00640 1 --2- 172.21.15.22:0/4250020636 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55707524cba0 0x557075254af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:07:03.651 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.649+0000 7fa665400640 1 --2- 172.21.15.22:0/4250020636 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55707524d590 0x55707525e620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:07:03.651 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.650+0000 7fa666800640 1 -- 172.21.15.22:0/4250020636 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55707524d590 msgr2=0x55707525e620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T18:07:03.651 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.650+0000 7fa666800640 1 --2- 172.21.15.22:0/4250020636 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55707524d590 0x55707525e620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:07:03.651 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.650+0000 7fa666800640 1 -- 172.21.15.22:0/4250020636 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55707524cba0 msgr2=0x557075254af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T18:07:03.651 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.650+0000 7fa666800640 1 --2- 172.21.15.22:0/4250020636 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55707524cba0 0x557075254af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:07:03.651 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.650+0000 7fa666800640 1 -- 172.21.15.22:0/4250020636 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55707524e570 con 0x557075252700 2024-03-28T18:07:03.651 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.650+0000 7fa666800640 1 --2- 172.21.15.22:0/4250020636 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557075252700 0x55707524e030 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7fa65c097f20 tx=0x7fa65c017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2d4df614fd7692d7 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:07:03.651 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.650+0000 7fa64e600640 1 -- 172.21.15.22:0/4250020636 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa65c0b3070 con 0x557075252700 2024-03-28T18:07:03.651 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.650+0000 7fa64e600640 1 -- 172.21.15.22:0/4250020636 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7fa65c09fca0 con 0x557075252700 2024-03-28T18:07:03.652 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.650+0000 7fa64e600640 1 -- 172.21.15.22:0/4250020636 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa65c09bdb0 con 0x557075252700 2024-03-28T18:07:03.652 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.651+0000 7fa66b58d740 1 -- 172.21.15.22:0/4250020636 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557075252700 msgr2=0x55707524e030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:07:03.652 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.651+0000 7fa66b58d740 1 --2- 172.21.15.22:0/4250020636 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557075252700 0x55707524e030 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7fa65c097f20 tx=0x7fa65c017000 comp rx=0 tx=0).stop 2024-03-28T18:07:03.652 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.651+0000 7fa66b58d740 1 -- 172.21.15.22:0/4250020636 shutdown_connections 2024-03-28T18:07:03.652 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.651+0000 7fa66b58d740 1 --2- 172.21.15.22:0/4250020636 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557075252700 0x55707524e030 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:07:03.652 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.651+0000 7fa66b58d740 1 --2- 172.21.15.22:0/4250020636 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55707524d590 0x55707525e620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:07:03.652 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.651+0000 7fa66b58d740 1 --2- 172.21.15.22:0/4250020636 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55707524cba0 0x557075254af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:07:03.652 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.651+0000 7fa66b58d740 1 -- 172.21.15.22:0/4250020636 >> 172.21.15.22:0/4250020636 conn(0x557075248110 msgr2=0x557075262970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:07:03.652 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.651+0000 7fa66b58d740 1 -- 172.21.15.22:0/4250020636 shutdown_connections 2024-03-28T18:07:03.652 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.651+0000 7fa66b58d740 1 -- 172.21.15.22:0/4250020636 wait complete. 2024-03-28T18:07:03.656 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.651+0000 7fa66b58d740 1 -- start start 2024-03-28T18:07:03.656 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.652+0000 7fa66b58d740 1 Processor -- start 2024-03-28T18:07:03.656 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.652+0000 7fa66b58d740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55707524cba0 0x55707525a630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:07:03.656 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.652+0000 7fa66b58d740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55707524d590 0x55707525ab70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:07:03.656 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.652+0000 7fa66b58d740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557075252700 0x55707525daf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:07:03.656 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.652+0000 7fa66b58d740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x557075264a30 con 0x55707524cba0 2024-03-28T18:07:03.656 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.652+0000 7fa66b58d740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x5570752643d0 con 0x55707524d590 2024-03-28T18:07:03.656 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.652+0000 7fa66b58d740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x557075265090 con 0x557075252700 2024-03-28T18:07:03.656 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.652+0000 7fa66b58d740 10 client.0 _ll_register_callbacks cb 0x7fa65c09b9b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-28T18:07:03.656 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.652+0000 7fa665e00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55707524cba0 0x55707525a630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:07:03.657 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.652+0000 7fa665400640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55707524d590 0x55707525ab70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:07:03.657 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.652+0000 7fa665e00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55707524cba0 0x55707525a630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:39654/0 (socket says 172.21.15.22:39654) 2024-03-28T18:07:03.657 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.652+0000 7fa665400640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55707524d590 0x55707525ab70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.22:47000/0 (socket says 172.21.15.22:47000) 2024-03-28T18:07:03.657 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.652+0000 7fa665e00640 1 -- 172.21.15.22:0/3722671644 learned_addr learned my addr 172.21.15.22:0/3722671644 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-28T18:07:03.657 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.653+0000 7fa666800640 1 --2- 172.21.15.22:0/3722671644 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557075252700 0x55707525daf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:07:03.657 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.653+0000 7fa665e00640 1 -- 172.21.15.22:0/3722671644 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557075252700 msgr2=0x55707525daf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T18:07:03.657 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.653+0000 7fa665e00640 1 --2- 172.21.15.22:0/3722671644 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557075252700 0x55707525daf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:07:03.657 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.653+0000 7fa665e00640 1 -- 172.21.15.22:0/3722671644 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55707524d590 msgr2=0x55707525ab70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T18:07:03.657 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.653+0000 7fa665e00640 1 --2- 172.21.15.22:0/3722671644 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55707524d590 0x55707525ab70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:07:03.657 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.653+0000 7fa665e00640 1 -- 172.21.15.22:0/3722671644 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55707525e0e0 con 0x55707524cba0 2024-03-28T18:07:03.657 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.653+0000 7fa666800640 1 --2- 172.21.15.22:0/3722671644 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557075252700 0x55707525daf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T18:07:03.657 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.653+0000 7fa665400640 1 --2- 172.21.15.22:0/3722671644 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55707524d590 0x55707525ab70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-28T18:07:03.657 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.653+0000 7fa665e00640 1 --2- 172.21.15.22:0/3722671644 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55707524cba0 0x55707525a630 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7fa654012fd0 tx=0x7fa6540046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f97c2d1328f9d42f server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:07:03.657 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.653+0000 7fa643400640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-03-28T18:07:03.658 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.653+0000 7fa643400640 1 -- 172.21.15.22:0/3722671644 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa654005220 con 0x55707524cba0 2024-03-28T18:07:03.658 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.653+0000 7fa643400640 1 -- 172.21.15.22:0/3722671644 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 8153+0+0 (secure 0 0 0) 0x7fa6540053c0 con 0x55707524cba0 2024-03-28T18:07:03.658 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.653+0000 7fa66b58d740 1 -- 172.21.15.22:0/3722671644 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fa65c09fca0 con 0x55707524cba0 2024-03-28T18:07:03.658 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.654+0000 7fa643400640 1 -- 172.21.15.22:0/3722671644 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa6540055c0 con 0x55707524cba0 2024-03-28T18:07:03.658 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.654+0000 7fa66b58d740 10 client.15312 fetch_fsmap learned FSMap version 26 2024-03-28T18:07:03.658 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.654+0000 7fa66b58d740 1 -- 172.21.15.22:0/3722671644 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x5570752bf1e0 con 0x55707524cba0 2024-03-28T18:07:03.658 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.654+0000 7fa643400640 1 -- 172.21.15.22:0/3722671644 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7fa65c09fca0 con 0x55707524cba0 2024-03-28T18:07:03.658 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.654+0000 7fa643400640 1 -- 172.21.15.22:0/3722671644 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 26) v1 ==== 7934+0+0 (secure 0 0 0) 0x7fa654030070 con 0x55707524cba0 2024-03-28T18:07:03.658 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.654+0000 7fa66b58d740 10 client.15312 fetch_fsmap finished waiting for FSMap version 26 2024-03-28T18:07:03.658 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.654+0000 7fa66b58d740 10 client.15312 resolve_mds: resolved name 'e' to daemon mds.e 2024-03-28T18:07:03.658 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.654+0000 7fa66b58d740 20 client.15312 populate_metadata read hostname 'smithi022' 2024-03-28T18:07:03.658 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.654+0000 7fa66b58d740 1 --2- 172.21.15.22:0/3722671644 >> [v2:172.21.15.79:6834/3365773431,v1:172.21.15.79:6835/3365773431] conn(0x5570752c38f0 0x5570752bf4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:07:03.658 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.654+0000 7fa66b58d740 4 client.15312 mds_command: new command op to 34475 tid=0 multi_id=0 [{"prefix": "status"}] 2024-03-28T18:07:03.658 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.654+0000 7fa66b58d740 1 -- 172.21.15.22:0/3722671644 --> [v2:172.21.15.79:6834/3365773431,v1:172.21.15.79:6835/3365773431] -- command(tid 0: {"prefix": "status"}) v1 -- 0x5570751bf1d0 con 0x5570752c38f0 2024-03-28T18:07:03.659 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.655+0000 7fa665400640 1 --2- 172.21.15.22:0/3722671644 >> [v2:172.21.15.79:6834/3365773431,v1:172.21.15.79:6835/3365773431] conn(0x5570752c38f0 0x5570752bf4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:07:03.659 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.655+0000 7fa665400640 1 --2- 172.21.15.22:0/3722671644 >> [v2:172.21.15.79:6834/3365773431,v1:172.21.15.79:6835/3365773431] conn(0x5570752c38f0 0x5570752bf4d0 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=81e31a2d420ce8c8 server_cookie=4147857177f6af38 in_seq=0 out_seq=0 2024-03-28T18:07:03.659 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.655+0000 7fa643400640 10 client.15312 ms_handle_connect on v2:172.21.15.79:6834/3365773431 2024-03-28T18:07:03.659 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.656+0000 7fa643400640 1 -- 172.21.15.22:0/3722671644 <== mds.2 v2:172.21.15.79:6834/3365773431 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+332 (crc 0 0 0) 0x5570751bf1d0 con 0x5570752c38f0 2024-03-28T18:07:03.659 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.656+0000 7fa643400640 10 client.15312 handle_command_reply: tid=0 2024-03-28T18:07:03.659 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.656+0000 7fa66b58d740 1 client.15312 shutdown 2024-03-28T18:07:03.659 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.656+0000 7fa66b58d740 1 -- 172.21.15.22:0/3722671644 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55707524cba0 msgr2=0x55707525a630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:07:03.659 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.656+0000 7fa66b58d740 1 --2- 172.21.15.22:0/3722671644 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55707524cba0 0x55707525a630 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7fa654012fd0 tx=0x7fa6540046c0 comp rx=0 tx=0).stop 2024-03-28T18:07:03.659 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.656+0000 7fa66b58d740 1 -- 172.21.15.22:0/3722671644 shutdown_connections 2024-03-28T18:07:03.659 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.656+0000 7fa66b58d740 1 --2- 172.21.15.22:0/3722671644 >> [v2:172.21.15.79:6834/3365773431,v1:172.21.15.79:6835/3365773431] conn(0x5570752c38f0 0x5570752bf4d0 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:07:03.659 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.656+0000 7fa66b58d740 1 --2- 172.21.15.22:0/3722671644 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557075252700 0x55707525daf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:07:03.659 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.656+0000 7fa66b58d740 1 --2- 172.21.15.22:0/3722671644 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55707524d590 0x55707525ab70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:07:03.660 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.656+0000 7fa643400640 0 client.15312 ms_handle_reset on v2:172.21.15.79:6834/3365773431 2024-03-28T18:07:03.660 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.656+0000 7fa66b58d740 1 --2- 172.21.15.22:0/3722671644 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55707524cba0 0x55707525a630 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:07:03.660 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.656+0000 7fa66b58d740 1 -- 172.21.15.22:0/3722671644 >> 172.21.15.22:0/3722671644 conn(0x557075248110 msgr2=0x557075255ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:07:03.660 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.656+0000 7fa66b58d740 1 -- 172.21.15.22:0/3722671644 shutdown_connections 2024-03-28T18:07:03.660 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.656+0000 7fa66b58d740 1 -- 172.21.15.22:0/3722671644 wait complete. 2024-03-28T18:07:03.660 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.656+0000 7fa66b58d740 20 client.15312 trim_cache size 0 max 16384 2024-03-28T18:07:03.661 INFO:teuthology.orchestra.run.smithi022.stdout:{ 2024-03-28T18:07:03.661 INFO:teuthology.orchestra.run.smithi022.stdout: "cluster_fsid": "fd220748-ed2b-11ee-95d0-87774f69a715", 2024-03-28T18:07:03.661 INFO:teuthology.orchestra.run.smithi022.stdout: "whoami": 2, 2024-03-28T18:07:03.661 INFO:teuthology.orchestra.run.smithi022.stdout: "id": 34475, 2024-03-28T18:07:03.661 INFO:teuthology.orchestra.run.smithi022.stdout: "want_state": "up:active", 2024-03-28T18:07:03.661 INFO:teuthology.orchestra.run.smithi022.stdout: "state": "up:active", 2024-03-28T18:07:03.661 INFO:teuthology.orchestra.run.smithi022.stdout: "fs_name": "cephfs", 2024-03-28T18:07:03.661 INFO:teuthology.orchestra.run.smithi022.stdout: "rank_uptime": 130.06077200999999, 2024-03-28T18:07:03.661 INFO:teuthology.orchestra.run.smithi022.stdout: "mdsmap_epoch": 26, 2024-03-28T18:07:03.661 INFO:teuthology.orchestra.run.smithi022.stdout: "osdmap_epoch": 92, 2024-03-28T18:07:03.661 INFO:teuthology.orchestra.run.smithi022.stdout: "osdmap_epoch_barrier": 90, 2024-03-28T18:07:03.661 INFO:teuthology.orchestra.run.smithi022.stdout: "uptime": 167.23271182600001 2024-03-28T18:07:03.661 INFO:teuthology.orchestra.run.smithi022.stdout:} 2024-03-28T18:07:03.662 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.657+0000 7fa668800640 1 -- 172.21.15.22:0/423701398 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7fa6480a2800 msgr2=0x7fa6480a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:07:03.662 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.657+0000 7fa668800640 1 --2- 172.21.15.22:0/423701398 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7fa6480a2800 0x7fa6480a4cc0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x55707524d990 tx=0x7fa654005f90 comp rx=0 tx=0).stop 2024-03-28T18:07:03.662 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.657+0000 7fa668800640 1 -- 172.21.15.22:0/423701398 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa660105490 msgr2=0x7fa6601a11d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:07:03.662 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.657+0000 7fa668800640 1 --2- 172.21.15.22:0/423701398 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa660105490 0x7fa6601a11d0 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7fa65c00eb10 tx=0x7fa65c00c2d0 comp rx=0 tx=0).stop 2024-03-28T18:07:03.662 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.657+0000 7fa668800640 1 -- 172.21.15.22:0/423701398 shutdown_connections 2024-03-28T18:07:03.662 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.657+0000 7fa668800640 1 --2- 172.21.15.22:0/423701398 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7fa6480a2800 0x7fa6480a4cc0 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:07:03.662 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.657+0000 7fa668800640 1 --2- 172.21.15.22:0/423701398 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa660105490 0x7fa6601a11d0 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:07:03.662 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.657+0000 7fa668800640 1 --2- 172.21.15.22:0/423701398 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa660104ad0 0x7fa66019a1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:07:03.663 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.657+0000 7fa668800640 1 --2- 172.21.15.22:0/423701398 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa6601038e0 0x7fa660199c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:07:03.663 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.657+0000 7fa668800640 1 -- 172.21.15.22:0/423701398 >> 172.21.15.22:0/423701398 conn(0x7fa6600fee50 msgr2=0x7fa660107ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:07:03.663 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.657+0000 7fa668800640 1 -- 172.21.15.22:0/423701398 shutdown_connections 2024-03-28T18:07:03.663 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.657+0000 7fa668800640 1 -- 172.21.15.22:0/423701398 wait complete. 2024-03-28T18:07:03.674 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:07:03 smithi134 ceph-mon[35037]: from='client.? 172.21.15.22:0/1930933532' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-28T18:07:03.674 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:07:03 smithi134 ceph-mon[35037]: from='client.? 172.21.15.22:0/3429492398' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-28T18:07:03.677 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.h status 2024-03-28T18:07:03.683 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:07:03 smithi079 ceph-mon[35312]: from='client.? 172.21.15.22:0/1930933532' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-28T18:07:03.683 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:07:03 smithi079 ceph-mon[35312]: from='client.? 172.21.15.22:0/3429492398' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-28T18:07:03.889 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.889+0000 7f5d6fa00640 1 -- 172.21.15.22:0/2709134902 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5d68108eb0 msgr2=0x7f5d68109330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:07:03.889 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.889+0000 7f5d6fa00640 1 --2- 172.21.15.22:0/2709134902 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5d68108eb0 0x7f5d68109330 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f5d54009900 tx=0x7f5d5402bc30 comp rx=0 tx=0).stop 2024-03-28T18:07:03.889 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.889+0000 7f5d6fa00640 1 -- 172.21.15.22:0/2709134902 shutdown_connections 2024-03-28T18:07:03.889 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.889+0000 7f5d6fa00640 1 --2- 172.21.15.22:0/2709134902 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f5d68109870 0x7f5d68110100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:07:03.889 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.889+0000 7f5d6fa00640 1 --2- 172.21.15.22:0/2709134902 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5d68108eb0 0x7f5d68109330 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:07:03.889 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.889+0000 7f5d6fa00640 1 --2- 172.21.15.22:0/2709134902 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5d68107cc0 0x7f5d681080e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:07:03.889 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.889+0000 7f5d6fa00640 1 -- 172.21.15.22:0/2709134902 >> 172.21.15.22:0/2709134902 conn(0x7f5d68076c60 msgr2=0x7f5d680790a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:07:03.889 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.889+0000 7f5d6fa00640 1 -- 172.21.15.22:0/2709134902 shutdown_connections 2024-03-28T18:07:03.889 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.889+0000 7f5d6fa00640 1 -- 172.21.15.22:0/2709134902 wait complete. 2024-03-28T18:07:03.890 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.890+0000 7f5d6fa00640 1 Processor -- start 2024-03-28T18:07:03.890 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.890+0000 7f5d6fa00640 1 -- start start 2024-03-28T18:07:03.892 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.891+0000 7f5d6fa00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5d68107cc0 0x7f5d6819dff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:07:03.892 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.891+0000 7f5d6fa00640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f5d68108eb0 0x7f5d6819e530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:07:03.892 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.891+0000 7f5d6fa00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5d68109870 0x7f5d681a55a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:07:03.892 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.891+0000 7f5d6fa00640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d68113970 con 0x7f5d68109870 2024-03-28T18:07:03.892 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.891+0000 7f5d6fa00640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d681137f0 con 0x7f5d68108eb0 2024-03-28T18:07:03.892 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.891+0000 7f5d6fa00640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d68113af0 con 0x7f5d68107cc0 2024-03-28T18:07:03.892 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.891+0000 7f5d6da00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5d68109870 0x7f5d681a55a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:07:03.892 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.891+0000 7f5d6da00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5d68109870 0x7f5d681a55a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:39666/0 (socket says 172.21.15.22:39666) 2024-03-28T18:07:03.892 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.891+0000 7f5d6da00640 1 -- 172.21.15.22:0/346689624 learned_addr learned my addr 172.21.15.22:0/346689624 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-28T18:07:03.893 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.891+0000 7f5d67e00640 1 --2- 172.21.15.22:0/346689624 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f5d68108eb0 0x7f5d6819e530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:07:03.893 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.891+0000 7f5d6d000640 1 --2- 172.21.15.22:0/346689624 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5d68107cc0 0x7f5d6819dff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:07:03.893 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.891+0000 7f5d6da00640 1 -- 172.21.15.22:0/346689624 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5d68107cc0 msgr2=0x7f5d6819dff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:07:03.893 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.892+0000 7f5d6da00640 1 --2- 172.21.15.22:0/346689624 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5d68107cc0 0x7f5d6819dff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:07:03.893 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.892+0000 7f5d6da00640 1 -- 172.21.15.22:0/346689624 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f5d68108eb0 msgr2=0x7f5d6819e530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:07:03.893 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.892+0000 7f5d6da00640 1 --2- 172.21.15.22:0/346689624 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f5d68108eb0 0x7f5d6819e530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:07:03.893 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.892+0000 7f5d6da00640 1 -- 172.21.15.22:0/346689624 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d681a5c10 con 0x7f5d68109870 2024-03-28T18:07:03.893 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.892+0000 7f5d67e00640 1 --2- 172.21.15.22:0/346689624 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f5d68108eb0 0x7f5d6819e530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-28T18:07:03.893 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.892+0000 7f5d6da00640 1 --2- 172.21.15.22:0/346689624 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5d68109870 0x7f5d681a55a0 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f5d6000eb10 tx=0x7f5d6000c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:07:03.893 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.892+0000 7f5d65600640 1 -- 172.21.15.22:0/346689624 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d60004b20 con 0x7f5d68109870 2024-03-28T18:07:03.893 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.892+0000 7f5d65600640 1 -- 172.21.15.22:0/346689624 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f5d60004cc0 con 0x7f5d68109870 2024-03-28T18:07:03.893 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.892+0000 7f5d6fa00640 1 -- 172.21.15.22:0/346689624 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5d681a5f00 con 0x7f5d68109870 2024-03-28T18:07:03.893 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.892+0000 7f5d65600640 1 -- 172.21.15.22:0/346689624 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d60010700 con 0x7f5d68109870 2024-03-28T18:07:03.893 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.892+0000 7f5d6fa00640 1 -- 172.21.15.22:0/346689624 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5d681a63c0 con 0x7f5d68109870 2024-03-28T18:07:03.895 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.895+0000 7f5d7282e740 1 Processor -- start 2024-03-28T18:07:03.895 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.895+0000 7f5d65600640 1 -- 172.21.15.22:0/346689624 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 18) v1 ==== 137290+0+0 (secure 0 0 0) 0x7f5d60004e70 con 0x7f5d68109870 2024-03-28T18:07:03.895 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.896+0000 7f5d7282e740 1 -- start start 2024-03-28T18:07:03.897 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.896+0000 7f5d7282e740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f69b1ecba0 0x55f69b1ecfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:07:03.897 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.896+0000 7f5d7282e740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f69b1ed590 0x55f69b1f2090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:07:03.898 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.896+0000 7f5d7282e740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55f69b1f2700 0x55f69b1f4b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:07:03.898 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.896+0000 7f5d7282e740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f69b151c90 con 0x55f69b1ed590 2024-03-28T18:07:03.898 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.896+0000 7f5d7282e740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f69b0fa980 con 0x55f69b1f2700 2024-03-28T18:07:03.898 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.896+0000 7f5d7282e740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f69b14dd30 con 0x55f69b1ecba0 2024-03-28T18:07:03.898 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.897+0000 7f5d6d000640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f69b1ed590 0x55f69b1f2090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:07:03.898 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.897+0000 7f5d6d000640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f69b1ed590 0x55f69b1f2090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:39682/0 (socket says 172.21.15.22:39682) 2024-03-28T18:07:03.898 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.897+0000 7f5d6d000640 1 -- 172.21.15.22:0/1172075723 learned_addr learned my addr 172.21.15.22:0/1172075723 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-28T18:07:03.898 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.897+0000 7f5d67e00640 1 --2- 172.21.15.22:0/1172075723 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55f69b1f2700 0x55f69b1f4b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:07:03.898 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.897+0000 7f5d67e00640 1 --2- 172.21.15.22:0/1172075723 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f69b1ecba0 0x55f69b1ecfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:07:03.898 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.897+0000 7f5d6d000640 1 -- 172.21.15.22:0/1172075723 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f69b1ecba0 msgr2=0x55f69b1ecfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T18:07:03.898 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.897+0000 7f5d6d000640 1 --2- 172.21.15.22:0/1172075723 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f69b1ecba0 0x55f69b1ecfc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:07:03.898 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.897+0000 7f5d6d000640 1 -- 172.21.15.22:0/1172075723 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55f69b1f2700 msgr2=0x55f69b1f4b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T18:07:03.898 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.897+0000 7f5d6d000640 1 --2- 172.21.15.22:0/1172075723 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55f69b1f2700 0x55f69b1f4b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:07:03.898 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.897+0000 7f5d65600640 1 --2- 172.21.15.22:0/346689624 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7f5d500a2880 0x7f5d500a4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:07:03.899 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.897+0000 7f5d6d000640 1 -- 172.21.15.22:0/1172075723 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f69b112510 con 0x55f69b1ed590 2024-03-28T18:07:03.899 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.897+0000 7f5d65600640 1 -- 172.21.15.22:0/346689624 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 8153+0+0 (secure 0 0 0) 0x7f5d600cf0c0 con 0x7f5d68109870 2024-03-28T18:07:03.899 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.897+0000 7f5d6da00640 1 --2- 172.21.15.22:0/346689624 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7f5d500a2880 0x7f5d500a4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:07:03.899 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.897+0000 7f5d6d000640 1 --2- 172.21.15.22:0/1172075723 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f69b1ed590 0x55f69b1f2090 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f5d5c009c20 tx=0x7f5d5c0117b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f674597c4770626d server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:07:03.899 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.898+0000 7f5d5a000640 1 -- 172.21.15.22:0/1172075723 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d5c012450 con 0x55f69b1ed590 2024-03-28T18:07:03.899 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.898+0000 7f5d5a000640 1 -- 172.21.15.22:0/1172075723 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f5d5c012ac0 con 0x55f69b1ed590 2024-03-28T18:07:03.899 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.898+0000 7f5d5a000640 1 -- 172.21.15.22:0/1172075723 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d5c00fbb0 con 0x55f69b1ed590 2024-03-28T18:07:03.902 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.899+0000 7f5d7282e740 1 -- 172.21.15.22:0/1172075723 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f69b1ed590 msgr2=0x55f69b1f2090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:07:03.902 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.899+0000 7f5d7282e740 1 --2- 172.21.15.22:0/1172075723 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f69b1ed590 0x55f69b1f2090 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f5d5c009c20 tx=0x7f5d5c0117b0 comp rx=0 tx=0).stop 2024-03-28T18:07:03.902 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.899+0000 7f5d6da00640 1 --2- 172.21.15.22:0/346689624 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7f5d500a2880 0x7f5d500a4d40 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f5d60097ec0 tx=0x7f5d60009a70 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:07:03.902 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.899+0000 7f5d7282e740 1 -- 172.21.15.22:0/1172075723 shutdown_connections 2024-03-28T18:07:03.903 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.899+0000 7f5d7282e740 1 --2- 172.21.15.22:0/1172075723 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55f69b1f2700 0x55f69b1f4b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:07:03.903 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.899+0000 7f5d7282e740 1 --2- 172.21.15.22:0/1172075723 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f69b1ed590 0x55f69b1f2090 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:07:03.903 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.899+0000 7f5d7282e740 1 --2- 172.21.15.22:0/1172075723 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f69b1ecba0 0x55f69b1ecfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:07:03.903 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.899+0000 7f5d7282e740 1 -- 172.21.15.22:0/1172075723 >> 172.21.15.22:0/1172075723 conn(0x55f69b1e8110 msgr2=0x55f69b1e8510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:07:03.903 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.899+0000 7f5d7282e740 1 -- 172.21.15.22:0/1172075723 shutdown_connections 2024-03-28T18:07:03.903 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.899+0000 7f5d7282e740 1 -- 172.21.15.22:0/1172075723 wait complete. 2024-03-28T18:07:03.903 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.899+0000 7f5d7282e740 1 -- start start 2024-03-28T18:07:03.903 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.899+0000 7f5d7282e740 1 Processor -- start 2024-03-28T18:07:03.903 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.900+0000 7f5d7282e740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f69b1ecba0 0x55f69b1f7090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:07:03.903 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.900+0000 7f5d7282e740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55f69b1ed590 0x55f69b1f75d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:07:03.903 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.900+0000 7f5d7282e740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f69b1f2700 0x55f69b1fe610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:07:03.903 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.900+0000 7f5d7282e740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f69b1f56b0 con 0x55f69b1ecba0 2024-03-28T18:07:03.903 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.900+0000 7f5d7282e740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f69b10ddd0 con 0x55f69b1ed590 2024-03-28T18:07:03.903 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.900+0000 7f5d7282e740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f69b1f5830 con 0x55f69b1f2700 2024-03-28T18:07:03.904 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.900+0000 7f5d7282e740 10 client.0 _ll_register_callbacks cb 0x55f69ae65ac0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-28T18:07:03.904 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.900+0000 7f5d67e00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f69b1ecba0 0x55f69b1f7090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:07:03.904 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.900+0000 7f5d67e00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f69b1ecba0 0x55f69b1f7090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:39694/0 (socket says 172.21.15.22:39694) 2024-03-28T18:07:03.904 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.900+0000 7f5d67e00640 1 -- 172.21.15.22:0/3727853399 learned_addr learned my addr 172.21.15.22:0/3727853399 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-28T18:07:03.904 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.900+0000 7f5d6d000640 1 --2- 172.21.15.22:0/3727853399 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55f69b1ed590 0x55f69b1f75d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:07:03.904 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.901+0000 7f5d67e00640 1 -- 172.21.15.22:0/3727853399 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f69b1f2700 msgr2=0x55f69b1fe610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T18:07:03.904 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.901+0000 7f5d67e00640 1 --2- 172.21.15.22:0/3727853399 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f69b1f2700 0x55f69b1fe610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:07:03.904 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.901+0000 7f5d67e00640 1 -- 172.21.15.22:0/3727853399 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55f69b1ed590 msgr2=0x55f69b1f75d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T18:07:03.904 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.901+0000 7f5d67e00640 1 --2- 172.21.15.22:0/3727853399 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55f69b1ed590 0x55f69b1f75d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:07:03.904 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.901+0000 7f5d67e00640 1 -- 172.21.15.22:0/3727853399 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55f69ae662d0 con 0x55f69b1ecba0 2024-03-28T18:07:03.904 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.901+0000 7f5d67e00640 1 --2- 172.21.15.22:0/3727853399 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f69b1ecba0 0x55f69b1f7090 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f5d5402c240 tx=0x7f5d54004250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4868c6a68bf1f080 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:07:03.904 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.901+0000 7f5d4ea00640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-03-28T18:07:03.904 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.901+0000 7f5d4ea00640 1 -- 172.21.15.22:0/3727853399 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d54047070 con 0x55f69b1ecba0 2024-03-28T18:07:03.904 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.901+0000 7f5d4ea00640 1 -- 172.21.15.22:0/3727853399 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 8153+0+0 (secure 0 0 0) 0x7f5d54005300 con 0x55f69b1ecba0 2024-03-28T18:07:03.905 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.901+0000 7f5d7282e740 1 -- 172.21.15.22:0/3727853399 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f5d5c012ac0 con 0x55f69b1ecba0 2024-03-28T18:07:03.905 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.901+0000 7f5d4ea00640 1 -- 172.21.15.22:0/3727853399 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5d54053050 con 0x55f69b1ecba0 2024-03-28T18:07:03.905 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.902+0000 7f5d7282e740 10 client.15324 fetch_fsmap learned FSMap version 26 2024-03-28T18:07:03.905 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.902+0000 7f5d7282e740 1 -- 172.21.15.22:0/3727853399 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x55f69af306b0 con 0x55f69b1ecba0 2024-03-28T18:07:03.905 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.902+0000 7f5d4ea00640 1 -- 172.21.15.22:0/3727853399 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f5d5c012ac0 con 0x55f69b1ecba0 2024-03-28T18:07:03.905 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.902+0000 7f5d4ea00640 1 -- 172.21.15.22:0/3727853399 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 26) v1 ==== 7934+0+0 (secure 0 0 0) 0x7f5d540037b0 con 0x55f69b1ecba0 2024-03-28T18:07:03.905 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.902+0000 7f5d7282e740 10 client.15324 fetch_fsmap finished waiting for FSMap version 26 2024-03-28T18:07:03.905 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.902+0000 7f5d7282e740 10 client.15324 resolve_mds: resolved name 'h' to daemon mds.h 2024-03-28T18:07:03.905 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.902+0000 7f5d7282e740 20 client.15324 populate_metadata read hostname 'smithi022' 2024-03-28T18:07:03.905 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.902+0000 7f5d7282e740 1 --2- 172.21.15.22:0/3727853399 >> [v2:172.21.15.79:6836/1787903366,v1:172.21.15.79:6837/1787903366] conn(0x55f69b203860 0x55f69b202900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:07:03.905 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.902+0000 7f5d7282e740 4 client.15324 mds_command: new command op to 34511 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-28T18:07:03.905 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.902+0000 7f5d7282e740 1 -- 172.21.15.22:0/3727853399 --> [v2:172.21.15.79:6836/1787903366,v1:172.21.15.79:6837/1787903366] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55f69b0f3520 con 0x55f69b203860 2024-03-28T18:07:03.905 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.903+0000 7f5d6d000640 1 --2- 172.21.15.22:0/3727853399 >> [v2:172.21.15.79:6836/1787903366,v1:172.21.15.79:6837/1787903366] conn(0x55f69b203860 0x55f69b202900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:07:03.906 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.903+0000 7f5d6d000640 1 --2- 172.21.15.22:0/3727853399 >> [v2:172.21.15.79:6836/1787903366,v1:172.21.15.79:6837/1787903366] conn(0x55f69b203860 0x55f69b202900 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=262147d70d62738f server_cookie=1680f7b7800ada47 in_seq=0 out_seq=0 2024-03-28T18:07:03.907 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.903+0000 7f5d4ea00640 10 client.15324 ms_handle_connect on v2:172.21.15.79:6836/1787903366 2024-03-28T18:07:03.907 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.905+0000 7f5d4ea00640 1 -- 172.21.15.22:0/3727853399 <== mds.1 v2:172.21.15.79:6836/1787903366 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x55f69b0f3520 con 0x55f69b203860 2024-03-28T18:07:03.907 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.905+0000 7f5d4ea00640 10 client.15324 handle_command_reply: tid=0 2024-03-28T18:07:03.907 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.905+0000 7f5d7282e740 1 client.15324 shutdown 2024-03-28T18:07:03.907 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.905+0000 7f5d7282e740 1 -- 172.21.15.22:0/3727853399 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f69b1ecba0 msgr2=0x55f69b1f7090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:07:03.907 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.905+0000 7f5d7282e740 1 --2- 172.21.15.22:0/3727853399 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f69b1ecba0 0x55f69b1f7090 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f5d5402c240 tx=0x7f5d54004250 comp rx=0 tx=0).stop 2024-03-28T18:07:03.907 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.905+0000 7f5d7282e740 1 -- 172.21.15.22:0/3727853399 shutdown_connections 2024-03-28T18:07:03.907 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.905+0000 7f5d7282e740 1 --2- 172.21.15.22:0/3727853399 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f69b1f2700 0x55f69b1fe610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:07:03.907 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.905+0000 7f5d7282e740 1 --2- 172.21.15.22:0/3727853399 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55f69b1ed590 0x55f69b1f75d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:07:03.907 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.905+0000 7f5d7282e740 1 --2- 172.21.15.22:0/3727853399 >> [v2:172.21.15.79:6836/1787903366,v1:172.21.15.79:6837/1787903366] conn(0x55f69b203860 0x55f69b202900 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:07:03.907 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.905+0000 7f5d7282e740 1 --2- 172.21.15.22:0/3727853399 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f69b1ecba0 0x55f69b1f7090 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:07:03.907 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.905+0000 7f5d4ea00640 0 client.15324 ms_handle_reset on v2:172.21.15.79:6836/1787903366 2024-03-28T18:07:03.907 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.905+0000 7f5d7282e740 1 -- 172.21.15.22:0/3727853399 >> 172.21.15.22:0/3727853399 conn(0x55f69b1e8110 msgr2=0x55f69b1e7310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:07:03.908 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.905+0000 7f5d7282e740 1 -- 172.21.15.22:0/3727853399 shutdown_connections 2024-03-28T18:07:03.908 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.905+0000 7f5d7282e740 1 -- 172.21.15.22:0/3727853399 wait complete. 2024-03-28T18:07:03.908 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.905+0000 7f5d7282e740 20 client.15324 trim_cache size 0 max 16384 2024-03-28T18:07:03.923 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.924+0000 7f5d7282e740 1 Processor -- start 2024-03-28T18:07:03.924 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.924+0000 7f5d7282e740 1 -- start start 2024-03-28T18:07:03.924 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.924+0000 7f5d7282e740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55f69b1ecba0 0x55f69b1f6ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:07:03.924 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.924+0000 7f5d7282e740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f69b1ed590 0x55f69b1fe5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:07:03.924 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.924+0000 7f5d7282e740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f69b1f2700 0x55f69b1f71e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:07:03.924 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.924+0000 7f5d7282e740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f69b203080 con 0x55f69b1f2700 2024-03-28T18:07:03.924 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.924+0000 7f5d7282e740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f69b2036e0 con 0x55f69b1ecba0 2024-03-28T18:07:03.924 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.924+0000 7f5d7282e740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f69b203d40 con 0x55f69b1ed590 2024-03-28T18:07:03.925 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.925+0000 7f5d6d000640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f69b1ed590 0x55f69b1fe5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:07:03.925 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.925+0000 7f5d6d000640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f69b1ed590 0x55f69b1fe5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.22:43944/0 (socket says 172.21.15.22:43944) 2024-03-28T18:07:03.925 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.925+0000 7f5d6d000640 1 -- 172.21.15.22:0/1518504498 learned_addr learned my addr 172.21.15.22:0/1518504498 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-28T18:07:03.925 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.925+0000 7f5d67e00640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55f69b1ecba0 0x55f69b1f6ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:07:03.925 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.925+0000 7f5d67e00640 1 --2- 172.21.15.22:0/1518504498 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f69b1f2700 0x55f69b1f71e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:07:03.925 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.925+0000 7f5d6d000640 1 -- 172.21.15.22:0/1518504498 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55f69b1ecba0 msgr2=0x55f69b1f6ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T18:07:03.925 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.925+0000 7f5d6d000640 1 --2- 172.21.15.22:0/1518504498 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55f69b1ecba0 0x55f69b1f6ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:07:03.925 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.925+0000 7f5d6d000640 1 -- 172.21.15.22:0/1518504498 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f69b1f2700 msgr2=0x55f69b1f71e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T18:07:03.925 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.925+0000 7f5d6d000640 1 --2- 172.21.15.22:0/1518504498 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f69b1f2700 0x55f69b1f71e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:07:03.926 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.925+0000 7f5d6d000640 1 -- 172.21.15.22:0/1518504498 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f69b1f1ce0 con 0x55f69b1ed590 2024-03-28T18:07:03.926 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.925+0000 7f5d67e00640 1 --2- 172.21.15.22:0/1518504498 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55f69b1ecba0 0x55f69b1f6ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-28T18:07:03.926 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.926+0000 7f5d6d000640 1 --2- 172.21.15.22:0/1518504498 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f69b1ed590 0x55f69b1fe5d0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x55f69b1ee000 tx=0x7f5d5c00f4e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b897632d5385be30 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:07:03.926 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.926+0000 7f5d59600640 1 -- 172.21.15.22:0/1518504498 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d5c009730 con 0x55f69b1ed590 2024-03-28T18:07:03.926 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.927+0000 7f5d59600640 1 -- 172.21.15.22:0/1518504498 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f5d5c005580 con 0x55f69b1ed590 2024-03-28T18:07:03.926 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.927+0000 7f5d59600640 1 -- 172.21.15.22:0/1518504498 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d5c003e50 con 0x55f69b1ed590 2024-03-28T18:07:03.927 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.927+0000 7f5d7282e740 1 -- 172.21.15.22:0/1518504498 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f69b1ed590 msgr2=0x55f69b1fe5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:07:03.927 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.927+0000 7f5d7282e740 1 --2- 172.21.15.22:0/1518504498 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f69b1ed590 0x55f69b1fe5d0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x55f69b1ee000 tx=0x7f5d5c00f4e0 comp rx=0 tx=0).stop 2024-03-28T18:07:03.928 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.927+0000 7f5d7282e740 1 -- 172.21.15.22:0/1518504498 shutdown_connections 2024-03-28T18:07:03.928 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.927+0000 7f5d7282e740 1 --2- 172.21.15.22:0/1518504498 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f69b1f2700 0x55f69b1f71e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:07:03.928 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.927+0000 7f5d7282e740 1 --2- 172.21.15.22:0/1518504498 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f69b1ed590 0x55f69b1fe5d0 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:07:03.928 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.927+0000 7f5d7282e740 1 --2- 172.21.15.22:0/1518504498 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55f69b1ecba0 0x55f69b1f6ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:07:03.928 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.927+0000 7f5d7282e740 1 -- 172.21.15.22:0/1518504498 >> 172.21.15.22:0/1518504498 conn(0x55f69b1e8110 msgr2=0x55f69b1e84f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:07:03.928 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.927+0000 7f5d7282e740 1 -- 172.21.15.22:0/1518504498 shutdown_connections 2024-03-28T18:07:03.928 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.927+0000 7f5d7282e740 1 -- 172.21.15.22:0/1518504498 wait complete. 2024-03-28T18:07:03.928 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.928+0000 7f5d7282e740 1 -- start start 2024-03-28T18:07:03.928 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.928+0000 7f5d7282e740 1 Processor -- start 2024-03-28T18:07:03.930 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.928+0000 7f5d7282e740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f69b1ecba0 0x55f69b1fe310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:07:03.931 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.928+0000 7f5d7282e740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55f69b1ed590 0x55f69b1fadb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:07:03.931 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.928+0000 7f5d7282e740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f69b1f2700 0x55f69b1fb2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:07:03.931 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.928+0000 7f5d7282e740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f69b204a00 con 0x55f69b1f2700 2024-03-28T18:07:03.931 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.928+0000 7f5d7282e740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f69b2043a0 con 0x55f69b1ed590 2024-03-28T18:07:03.931 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.928+0000 7f5d7282e740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f69b205060 con 0x55f69b1ecba0 2024-03-28T18:07:03.931 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.928+0000 7f5d7282e740 10 client.0 _ll_register_callbacks cb 0x7f5d5c005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-28T18:07:03.931 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.929+0000 7f5d6d000640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55f69b1ed590 0x55f69b1fadb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:07:03.931 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.929+0000 7f5d67e00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f69b1f2700 0x55f69b1fb2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:07:03.931 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.929+0000 7f5d6d000640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55f69b1ed590 0x55f69b1fadb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.22:47064/0 (socket says 172.21.15.22:47064) 2024-03-28T18:07:03.931 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.929+0000 7f5d6d000640 1 -- 172.21.15.22:0/2575007332 learned_addr learned my addr 172.21.15.22:0/2575007332 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-28T18:07:03.931 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.929+0000 7f5d67e00640 1 --2- 172.21.15.22:0/2575007332 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f69b1ecba0 0x55f69b1fe310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:07:03.931 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.929+0000 7f5d6d000640 1 -- 172.21.15.22:0/2575007332 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f69b1ecba0 msgr2=0x55f69b1fe310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T18:07:03.931 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.929+0000 7f5d6d000640 1 --2- 172.21.15.22:0/2575007332 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f69b1ecba0 0x55f69b1fe310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:07:03.931 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.929+0000 7f5d6d000640 1 -- 172.21.15.22:0/2575007332 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f69b1f2700 msgr2=0x55f69b1fb2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T18:07:03.932 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.929+0000 7f5d6d000640 1 --2- 172.21.15.22:0/2575007332 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f69b1f2700 0x55f69b1fb2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:07:03.932 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.929+0000 7f5d6d000640 1 -- 172.21.15.22:0/2575007332 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55f69b1fb8b0 con 0x55f69b1ed590 2024-03-28T18:07:03.932 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.929+0000 7f5d67e00640 1 --2- 172.21.15.22:0/2575007332 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f69b1f2700 0x55f69b1fb2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-28T18:07:03.932 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.930+0000 7f5d6d000640 1 --2- 172.21.15.22:0/2575007332 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55f69b1ed590 0x55f69b1fadb0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f5d5c0057a0 tx=0x7f5d5c0037b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9248ac7ef9493b4e server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:07:03.932 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.930+0000 7f5d4ea00640 10 client.0 ms_handle_connect on v2:172.21.15.79:3300/0 2024-03-28T18:07:03.932 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.930+0000 7f5d4ea00640 1 -- 172.21.15.22:0/2575007332 <== mon.2 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d5c0050b0 con 0x55f69b1ed590 2024-03-28T18:07:03.932 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.930+0000 7f5d4ea00640 1 -- 172.21.15.22:0/2575007332 <== mon.2 v2:172.21.15.79:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 8153+0+0 (secure 0 0 0) 0x7f5d5c005250 con 0x55f69b1ed590 2024-03-28T18:07:03.932 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.930+0000 7f5d7282e740 1 -- 172.21.15.22:0/2575007332 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f5d5c005580 con 0x55f69b1ed590 2024-03-28T18:07:03.932 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.930+0000 7f5d4ea00640 1 -- 172.21.15.22:0/2575007332 <== mon.2 v2:172.21.15.79:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5d5c009530 con 0x55f69b1ed590 2024-03-28T18:07:03.932 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.930+0000 7f5d7282e740 10 client.35192 fetch_fsmap learned FSMap version 26 2024-03-28T18:07:03.932 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.930+0000 7f5d7282e740 1 -- 172.21.15.22:0/2575007332 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x55f69b25f900 con 0x55f69b1ed590 2024-03-28T18:07:03.932 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.931+0000 7f5d4ea00640 1 -- 172.21.15.22:0/2575007332 <== mon.2 v2:172.21.15.79:3300/0 4 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f5d5c005580 con 0x55f69b1ed590 2024-03-28T18:07:03.932 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.931+0000 7f5d4ea00640 1 -- 172.21.15.22:0/2575007332 <== mon.2 v2:172.21.15.79:3300/0 5 ==== fsmap(e 26) v1 ==== 7934+0+0 (secure 0 0 0) 0x7f5d5c030470 con 0x55f69b1ed590 2024-03-28T18:07:03.932 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.931+0000 7f5d7282e740 10 client.35192 fetch_fsmap finished waiting for FSMap version 26 2024-03-28T18:07:03.933 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.931+0000 7f5d7282e740 10 client.35192 resolve_mds: resolved name 'h' to daemon mds.h 2024-03-28T18:07:03.933 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.931+0000 7f5d7282e740 20 client.35192 populate_metadata read hostname 'smithi022' 2024-03-28T18:07:03.933 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.931+0000 7f5d7282e740 1 --2- 172.21.15.22:0/2575007332 >> [v2:172.21.15.79:6836/1787903366,v1:172.21.15.79:6837/1787903366] conn(0x55f69b2638d0 0x55f69b25fad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:07:03.933 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.931+0000 7f5d7282e740 4 client.35192 mds_command: new command op to 34511 tid=0 multi_id=0 [{"prefix": "status"}] 2024-03-28T18:07:03.933 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.931+0000 7f5d7282e740 1 -- 172.21.15.22:0/2575007332 --> [v2:172.21.15.79:6836/1787903366,v1:172.21.15.79:6837/1787903366] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55f69b15f1d0 con 0x55f69b2638d0 2024-03-28T18:07:03.934 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.931+0000 7f5d67e00640 1 --2- 172.21.15.22:0/2575007332 >> [v2:172.21.15.79:6836/1787903366,v1:172.21.15.79:6837/1787903366] conn(0x55f69b2638d0 0x55f69b25fad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:07:03.934 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.932+0000 7f5d67e00640 1 --2- 172.21.15.22:0/2575007332 >> [v2:172.21.15.79:6836/1787903366,v1:172.21.15.79:6837/1787903366] conn(0x55f69b2638d0 0x55f69b25fad0 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=a993c1fe20fdc88e server_cookie=b4c794a662c4f241 in_seq=0 out_seq=0 2024-03-28T18:07:03.934 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.932+0000 7f5d4ea00640 10 client.35192 ms_handle_connect on v2:172.21.15.79:6836/1787903366 2024-03-28T18:07:03.934 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.933+0000 7f5d4ea00640 1 -- 172.21.15.22:0/2575007332 <== mds.1 v2:172.21.15.79:6836/1787903366 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+332 (crc 0 0 0) 0x55f69b15f1d0 con 0x55f69b2638d0 2024-03-28T18:07:03.934 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.933+0000 7f5d4ea00640 10 client.35192 handle_command_reply: tid=0 2024-03-28T18:07:03.934 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.933+0000 7f5d7282e740 1 client.35192 shutdown 2024-03-28T18:07:03.934 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.933+0000 7f5d7282e740 1 -- 172.21.15.22:0/2575007332 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55f69b1ed590 msgr2=0x55f69b1fadb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:07:03.935 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.933+0000 7f5d7282e740 1 --2- 172.21.15.22:0/2575007332 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55f69b1ed590 0x55f69b1fadb0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f5d5c0057a0 tx=0x7f5d5c0037b0 comp rx=0 tx=0).stop 2024-03-28T18:07:03.935 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.933+0000 7f5d7282e740 1 -- 172.21.15.22:0/2575007332 shutdown_connections 2024-03-28T18:07:03.935 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.933+0000 7f5d7282e740 1 --2- 172.21.15.22:0/2575007332 >> [v2:172.21.15.79:6836/1787903366,v1:172.21.15.79:6837/1787903366] conn(0x55f69b2638d0 0x55f69b25fad0 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:07:03.935 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.933+0000 7f5d7282e740 1 --2- 172.21.15.22:0/2575007332 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f69b1f2700 0x55f69b1fb2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:07:03.935 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.933+0000 7f5d7282e740 1 --2- 172.21.15.22:0/2575007332 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55f69b1ed590 0x55f69b1fadb0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:07:03.935 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.933+0000 7f5d4ea00640 0 client.35192 ms_handle_reset on v2:172.21.15.79:6836/1787903366 2024-03-28T18:07:03.935 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.933+0000 7f5d7282e740 1 --2- 172.21.15.22:0/2575007332 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f69b1ecba0 0x55f69b1fe310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:07:03.935 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.933+0000 7f5d7282e740 1 -- 172.21.15.22:0/2575007332 >> 172.21.15.22:0/2575007332 conn(0x55f69b1e8110 msgr2=0x55f69b1f6ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:07:03.935 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.933+0000 7f5d7282e740 1 -- 172.21.15.22:0/2575007332 shutdown_connections 2024-03-28T18:07:03.935 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.933+0000 7f5d7282e740 1 -- 172.21.15.22:0/2575007332 wait complete. 2024-03-28T18:07:03.935 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.933+0000 7f5d7282e740 20 client.35192 trim_cache size 0 max 16384 2024-03-28T18:07:03.935 INFO:teuthology.orchestra.run.smithi022.stdout:{ 2024-03-28T18:07:03.935 INFO:teuthology.orchestra.run.smithi022.stdout: "cluster_fsid": "fd220748-ed2b-11ee-95d0-87774f69a715", 2024-03-28T18:07:03.936 INFO:teuthology.orchestra.run.smithi022.stdout: "whoami": 1, 2024-03-28T18:07:03.936 INFO:teuthology.orchestra.run.smithi022.stdout: "id": 34511, 2024-03-28T18:07:03.936 INFO:teuthology.orchestra.run.smithi022.stdout: "want_state": "up:active", 2024-03-28T18:07:03.936 INFO:teuthology.orchestra.run.smithi022.stdout: "state": "up:active", 2024-03-28T18:07:03.936 INFO:teuthology.orchestra.run.smithi022.stdout: "fs_name": "cephfs", 2024-03-28T18:07:03.936 INFO:teuthology.orchestra.run.smithi022.stdout: "rank_uptime": 131.36003685700001, 2024-03-28T18:07:03.936 INFO:teuthology.orchestra.run.smithi022.stdout: "mdsmap_epoch": 26, 2024-03-28T18:07:03.936 INFO:teuthology.orchestra.run.smithi022.stdout: "osdmap_epoch": 92, 2024-03-28T18:07:03.936 INFO:teuthology.orchestra.run.smithi022.stdout: "osdmap_epoch_barrier": 90, 2024-03-28T18:07:03.936 INFO:teuthology.orchestra.run.smithi022.stdout: "uptime": 155.62562479799999 2024-03-28T18:07:03.936 INFO:teuthology.orchestra.run.smithi022.stdout:} 2024-03-28T18:07:03.937 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.934+0000 7f5d6fa00640 1 -- 172.21.15.22:0/346689624 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7f5d500a2880 msgr2=0x7f5d500a4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:07:03.937 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.934+0000 7f5d6fa00640 1 --2- 172.21.15.22:0/346689624 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7f5d500a2880 0x7f5d500a4d40 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f5d60097ec0 tx=0x7f5d60009a70 comp rx=0 tx=0).stop 2024-03-28T18:07:03.937 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.934+0000 7f5d6fa00640 1 -- 172.21.15.22:0/346689624 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5d68109870 msgr2=0x7f5d681a55a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:07:03.937 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.934+0000 7f5d6fa00640 1 --2- 172.21.15.22:0/346689624 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5d68109870 0x7f5d681a55a0 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f5d6000eb10 tx=0x7f5d6000c2d0 comp rx=0 tx=0).stop 2024-03-28T18:07:03.937 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.934+0000 7f5d6fa00640 1 -- 172.21.15.22:0/346689624 shutdown_connections 2024-03-28T18:07:03.937 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.934+0000 7f5d6fa00640 1 --2- 172.21.15.22:0/346689624 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7f5d500a2880 0x7f5d500a4d40 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:07:03.937 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.934+0000 7f5d6fa00640 1 --2- 172.21.15.22:0/346689624 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5d68109870 0x7f5d681a55a0 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:07:03.937 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.934+0000 7f5d6fa00640 1 --2- 172.21.15.22:0/346689624 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f5d68108eb0 0x7f5d6819e530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:07:03.937 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.934+0000 7f5d6fa00640 1 --2- 172.21.15.22:0/346689624 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5d68107cc0 0x7f5d6819dff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:07:03.937 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.934+0000 7f5d6fa00640 1 -- 172.21.15.22:0/346689624 >> 172.21.15.22:0/346689624 conn(0x7f5d68076c60 msgr2=0x7f5d6810c230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:07:03.937 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.934+0000 7f5d6fa00640 1 -- 172.21.15.22:0/346689624 shutdown_connections 2024-03-28T18:07:03.937 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:03.934+0000 7f5d6fa00640 1 -- 172.21.15.22:0/346689624 wait complete. 2024-03-28T18:07:03.951 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2024-03-28T18:07:04.165 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:04.165+0000 7f885f200640 1 -- 172.21.15.22:0/3363710994 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8860109430 msgr2=0x7f886010b8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:07:04.165 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:04.165+0000 7f885f200640 1 --2- 172.21.15.22:0/3363710994 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8860109430 0x7f886010b8d0 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f8858009ca0 tx=0x7f885802c190 comp rx=0 tx=0).stop 2024-03-28T18:07:04.165 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:04.165+0000 7f885f200640 1 -- 172.21.15.22:0/3363710994 shutdown_connections 2024-03-28T18:07:04.166 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:04.165+0000 7f885f200640 1 --2- 172.21.15.22:0/3363710994 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8860109430 0x7f886010b8d0 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:07:04.166 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:04.165+0000 7f885f200640 1 --2- 172.21.15.22:0/3363710994 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f88600740a0 0x7f8860074500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:07:04.166 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:04.165+0000 7f885f200640 1 --2- 172.21.15.22:0/3363710994 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8860075680 0x7f8860073ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:07:04.166 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:04.165+0000 7f885f200640 1 -- 172.21.15.22:0/3363710994 >> 172.21.15.22:0/3363710994 conn(0x7f88600fcb10 msgr2=0x7f88600fef50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:07:04.166 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:04.166+0000 7f885f200640 1 -- 172.21.15.22:0/3363710994 shutdown_connections 2024-03-28T18:07:04.166 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:04.166+0000 7f885f200640 1 -- 172.21.15.22:0/3363710994 wait complete. 2024-03-28T18:07:04.168 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:04.167+0000 7f885f200640 1 Processor -- start 2024-03-28T18:07:04.168 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:04.167+0000 7f885f200640 1 -- start start 2024-03-28T18:07:04.168 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:04.167+0000 7f885f200640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f88600740a0 0x7f8860199a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:07:04.168 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:04.167+0000 7f885f200640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8860075680 0x7f8860199f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:07:04.168 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:04.167+0000 7f885f200640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f8860109430 0x7f88601a0fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:07:04.168 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:04.167+0000 7f885f200640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f886010f400 con 0x7f8860075680 2024-03-28T18:07:04.168 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:04.167+0000 7f885f200640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f886010f280 con 0x7f8860109430 2024-03-28T18:07:04.168 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:04.167+0000 7f885f200640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f886010f580 con 0x7f88600740a0 2024-03-28T18:07:04.168 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:04.167+0000 7f885e800640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f8860109430 0x7f88601a0fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:07:04.168 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:04.167+0000 7f885e800640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f8860109430 0x7f88601a0fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.22:47070/0 (socket says 172.21.15.22:47070) 2024-03-28T18:07:04.169 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:04.168+0000 7f885e800640 1 -- 172.21.15.22:0/2272608175 learned_addr learned my addr 172.21.15.22:0/2272608175 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-28T18:07:04.169 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:04.168+0000 7f885de00640 1 --2- 172.21.15.22:0/2272608175 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f88600740a0 0x7f8860199a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:07:04.169 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:04.168+0000 7f885d400640 1 --2- 172.21.15.22:0/2272608175 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8860075680 0x7f8860199f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:07:04.169 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:04.168+0000 7f885e800640 1 -- 172.21.15.22:0/2272608175 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f88600740a0 msgr2=0x7f8860199a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:07:04.169 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:04.168+0000 7f885e800640 1 --2- 172.21.15.22:0/2272608175 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f88600740a0 0x7f8860199a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:07:04.169 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:04.168+0000 7f885e800640 1 -- 172.21.15.22:0/2272608175 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8860075680 msgr2=0x7f8860199f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:07:04.169 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:04.168+0000 7f885e800640 1 --2- 172.21.15.22:0/2272608175 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8860075680 0x7f8860199f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:07:04.169 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:04.168+0000 7f885e800640 1 -- 172.21.15.22:0/2272608175 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f88601a1640 con 0x7f8860109430 2024-03-28T18:07:04.170 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:04.169+0000 7f885e800640 1 --2- 172.21.15.22:0/2272608175 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f8860109430 0x7f88601a0fd0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f8858032650 tx=0x7f8858034cf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:07:04.170 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:04.169+0000 7f8856a00640 1 -- 172.21.15.22:0/2272608175 <== mon.2 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8858034e60 con 0x7f8860109430 2024-03-28T18:07:04.170 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:04.169+0000 7f885f200640 1 -- 172.21.15.22:0/2272608175 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f88601a18d0 con 0x7f8860109430 2024-03-28T18:07:04.170 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:04.169+0000 7f885f200640 1 -- 172.21.15.22:0/2272608175 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f88601a1e10 con 0x7f8860109430 2024-03-28T18:07:04.170 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:04.169+0000 7f8856a00640 1 -- 172.21.15.22:0/2272608175 <== mon.2 v2:172.21.15.79:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f8858032bc0 con 0x7f8860109430 2024-03-28T18:07:04.170 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:04.169+0000 7f8856a00640 1 -- 172.21.15.22:0/2272608175 <== mon.2 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8858039040 con 0x7f8860109430 2024-03-28T18:07:04.171 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:04.172+0000 7f8867603740 1 Processor -- start 2024-03-28T18:07:04.174 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:04.173+0000 7f8856a00640 1 -- 172.21.15.22:0/2272608175 <== mon.2 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 18) v1 ==== 137290+0+0 (secure 0 0 0) 0x7f8858003b70 con 0x7f8860109430 2024-03-28T18:07:04.175 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:04.173+0000 7f8867603740 1 -- start start 2024-03-28T18:07:04.175 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:04.174+0000 7f8867603740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560114296d80 0x5601142971a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:07:04.175 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:04.174+0000 7f8867603740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560114297770 0x56011429c270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:07:04.175 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:04.174+0000 7f8867603740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56011429c8e0 0x56011429ed70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:07:04.175 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:04.174+0000 7f8867603740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x5601141fbc90 con 0x560114297770 2024-03-28T18:07:04.175 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:04.174+0000 7f8867603740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x5601141a4980 con 0x56011429c8e0 2024-03-28T18:07:04.175 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:04.174+0000 7f8867603740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x5601141f7d30 con 0x560114296d80 2024-03-28T18:07:04.175 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:04.174+0000 7f885de00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560114297770 0x56011429c270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:07:04.175 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:04.174+0000 7f885de00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560114297770 0x56011429c270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:39734/0 (socket says 172.21.15.22:39734) 2024-03-28T18:07:04.175 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:04.174+0000 7f885de00640 1 -- 172.21.15.22:0/2669445359 learned_addr learned my addr 172.21.15.22:0/2669445359 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-28T18:07:04.175 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:04.174+0000 7f885d400640 1 --2- 172.21.15.22:0/2669445359 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56011429c8e0 0x56011429ed70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:07:04.176 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:04.174+0000 7f885d400640 1 --2- 172.21.15.22:0/2669445359 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560114296d80 0x5601142971a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:07:04.176 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:04.174+0000 7f885de00640 1 -- 172.21.15.22:0/2669445359 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560114296d80 msgr2=0x5601142971a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T18:07:04.176 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:04.174+0000 7f885de00640 1 --2- 172.21.15.22:0/2669445359 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560114296d80 0x5601142971a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:07:04.176 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:04.174+0000 7f885de00640 1 -- 172.21.15.22:0/2669445359 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56011429c8e0 msgr2=0x56011429ed70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T18:07:04.176 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:04.174+0000 7f885de00640 1 --2- 172.21.15.22:0/2669445359 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56011429c8e0 0x56011429ed70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:07:04.176 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:04.174+0000 7f885de00640 1 -- 172.21.15.22:0/2669445359 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5601141bc510 con 0x560114297770 2024-03-28T18:07:04.176 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:04.175+0000 7f885de00640 1 --2- 172.21.15.22:0/2669445359 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560114297770 0x56011429c270 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f885000b760 tx=0x7f885000bc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7513833053c77ddb server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:07:04.176 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:04.175+0000 7f884f000640 1 -- 172.21.15.22:0/2669445359 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8850003dd0 con 0x560114297770 2024-03-28T18:07:04.176 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:04.175+0000 7f884f000640 1 -- 172.21.15.22:0/2669445359 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f8850015070 con 0x560114297770 2024-03-28T18:07:04.176 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:04.175+0000 7f884f000640 1 -- 172.21.15.22:0/2669445359 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8850004e30 con 0x560114297770 2024-03-28T18:07:04.176 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:04.175+0000 7f8856a00640 1 --2- 172.21.15.22:0/2272608175 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7f88440a2800 0x7f88440a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:07:04.180 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:04.175+0000 7f8856a00640 1 -- 172.21.15.22:0/2272608175 <== mon.2 v2:172.21.15.79:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 8153+0+0 (secure 0 0 0) 0x7f88580f0af0 con 0x7f8860109430 2024-03-28T18:07:04.181 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:04.176+0000 7f885d400640 1 --2- 172.21.15.22:0/2272608175 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7f88440a2800 0x7f88440a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:07:04.181 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:04.176+0000 7f8867603740 1 -- 172.21.15.22:0/2669445359 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560114297770 msgr2=0x56011429c270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:07:04.181 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:04.176+0000 7f8867603740 1 --2- 172.21.15.22:0/2669445359 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560114297770 0x56011429c270 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f885000b760 tx=0x7f885000bc20 comp rx=0 tx=0).stop 2024-03-28T18:07:04.181 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:04.176+0000 7f8867603740 1 -- 172.21.15.22:0/2669445359 shutdown_connections 2024-03-28T18:07:04.181 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:04.176+0000 7f8867603740 1 --2- 172.21.15.22:0/2669445359 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56011429c8e0 0x56011429ed70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:07:04.181 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:04.176+0000 7f8867603740 1 --2- 172.21.15.22:0/2669445359 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560114297770 0x56011429c270 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:07:04.181 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:04.176+0000 7f8867603740 1 --2- 172.21.15.22:0/2669445359 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560114296d80 0x5601142971a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:07:04.181 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:04.176+0000 7f8867603740 1 -- 172.21.15.22:0/2669445359 >> 172.21.15.22:0/2669445359 conn(0x5601142922f0 msgr2=0x5601142926f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:07:04.181 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:04.176+0000 7f8867603740 1 -- 172.21.15.22:0/2669445359 shutdown_connections 2024-03-28T18:07:04.181 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:04.176+0000 7f8867603740 1 -- 172.21.15.22:0/2669445359 wait complete. 2024-03-28T18:07:04.181 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:04.176+0000 7f885d400640 1 --2- 172.21.15.22:0/2272608175 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7f88440a2800 0x7f88440a4cc0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x560114297fd0 tx=0x7f8848012040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:07:04.181 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:04.177+0000 7f8867603740 1 -- start start 2024-03-28T18:07:04.181 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:04.177+0000 7f8867603740 1 Processor -- start 2024-03-28T18:07:04.181 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:04.177+0000 7f8867603740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560114296d80 0x5601142a1290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:07:04.182 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:04.177+0000 7f8867603740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x560114297770 0x5601142a17d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:07:04.182 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:04.177+0000 7f8867603740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56011429c8e0 0x5601142a8810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:07:04.182 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:04.177+0000 7f8867603740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x56011429f890 con 0x560114296d80 2024-03-28T18:07:04.182 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:04.177+0000 7f8867603740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x5601141b7dd0 con 0x560114297770 2024-03-28T18:07:04.182 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:04.177+0000 7f8867603740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x56011429fa10 con 0x56011429c8e0 2024-03-28T18:07:04.182 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:04.177+0000 7f8867603740 10 client.0 _ll_register_callbacks cb 0x560113f0fac0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-28T18:07:04.182 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:04.177+0000 7f885de00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560114296d80 0x5601142a1290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:07:04.182 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:04.178+0000 7f885de00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560114296d80 0x5601142a1290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:39740/0 (socket says 172.21.15.22:39740) 2024-03-28T18:07:04.182 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:04.178+0000 7f885de00640 1 -- 172.21.15.22:0/2796915301 learned_addr learned my addr 172.21.15.22:0/2796915301 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-28T18:07:04.182 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:04.178+0000 7f885d400640 1 --2- 172.21.15.22:0/2796915301 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x560114297770 0x5601142a17d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:07:04.182 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:04.178+0000 7f885de00640 1 -- 172.21.15.22:0/2796915301 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56011429c8e0 msgr2=0x5601142a8810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T18:07:04.182 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:04.178+0000 7f885de00640 1 --2- 172.21.15.22:0/2796915301 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56011429c8e0 0x5601142a8810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:07:04.182 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:04.178+0000 7f885de00640 1 -- 172.21.15.22:0/2796915301 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x560114297770 msgr2=0x5601142a17d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T18:07:04.182 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:04.178+0000 7f885de00640 1 --2- 172.21.15.22:0/2796915301 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x560114297770 0x5601142a17d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:07:04.183 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:04.178+0000 7f885de00640 1 -- 172.21.15.22:0/2796915301 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x560113f102d0 con 0x560114296d80 2024-03-28T18:07:04.183 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:04.178+0000 7f885de00640 1 --2- 172.21.15.22:0/2796915301 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560114296d80 0x5601142a1290 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f8850004960 tx=0x7f8850004de0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=55cc778eee4a1af1 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:07:04.183 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:04.178+0000 7f8843400640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-03-28T18:07:04.183 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:04.178+0000 7f8843400640 1 -- 172.21.15.22:0/2796915301 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8850028060 con 0x560114296d80 2024-03-28T18:07:04.183 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:04.178+0000 7f8843400640 1 -- 172.21.15.22:0/2796915301 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 8153+0+0 (secure 0 0 0) 0x7f8850004fd0 con 0x560114296d80 2024-03-28T18:07:04.183 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:04.179+0000 7f8867603740 1 -- 172.21.15.22:0/2796915301 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f8850015070 con 0x560114296d80 2024-03-28T18:07:04.183 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:04.179+0000 7f8843400640 1 -- 172.21.15.22:0/2796915301 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7f885000be70 con 0x560114296d80 2024-03-28T18:07:04.183 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:04.179+0000 7f8867603740 10 client.15342 fetch_fsmap learned FSMap version 26 2024-03-28T18:07:04.183 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:04.179+0000 7f8867603740 1 -- 172.21.15.22:0/2796915301 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x560113fda6b0 con 0x560114296d80 2024-03-28T18:07:04.183 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:04.179+0000 7f8843400640 1 -- 172.21.15.22:0/2796915301 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f8850015070 con 0x560114296d80 2024-03-28T18:07:04.183 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:04.179+0000 7f8843400640 1 -- 172.21.15.22:0/2796915301 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 26) v1 ==== 7934+0+0 (secure 0 0 0) 0x7f8850022070 con 0x560114296d80 2024-03-28T18:07:04.183 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:04.179+0000 7f8867603740 10 client.15342 fetch_fsmap finished waiting for FSMap version 26 2024-03-28T18:07:04.183 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:04.179+0000 7f8867603740 10 client.15342 resolve_mds: resolved name 'b' to daemon mds.b 2024-03-28T18:07:04.183 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:04.179+0000 7f8867603740 20 client.15342 populate_metadata read hostname 'smithi022' 2024-03-28T18:07:04.183 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:04.180+0000 7f8867603740 1 --2- 172.21.15.22:0/2796915301 >> [v2:172.21.15.79:6838/2441542443,v1:172.21.15.79:6839/2441542443] conn(0x5601142aca20 0x5601142b1ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:07:04.184 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:04.180+0000 7f8867603740 4 client.15342 mds_command: new command op to 34553 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-28T18:07:04.184 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:04.180+0000 7f8867603740 1 -- 172.21.15.22:0/2796915301 --> [v2:172.21.15.79:6838/2441542443,v1:172.21.15.79:6839/2441542443] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56011419d520 con 0x5601142aca20 2024-03-28T18:07:04.184 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:04.180+0000 7f885d400640 1 --2- 172.21.15.22:0/2796915301 >> [v2:172.21.15.79:6838/2441542443,v1:172.21.15.79:6839/2441542443] conn(0x5601142aca20 0x5601142b1ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:07:04.184 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:04.181+0000 7f885d400640 1 --2- 172.21.15.22:0/2796915301 >> [v2:172.21.15.79:6838/2441542443,v1:172.21.15.79:6839/2441542443] conn(0x5601142aca20 0x5601142b1ae0 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=22199d64f6f292d3 server_cookie=cc2aedabfc0b19a2 in_seq=0 out_seq=0 2024-03-28T18:07:04.184 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:04.181+0000 7f8843400640 10 client.15342 ms_handle_connect on v2:172.21.15.79:6838/2441542443 2024-03-28T18:07:04.185 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:04.182+0000 7f8843400640 1 -- 172.21.15.22:0/2796915301 <== mds.0 v2:172.21.15.79:6838/2441542443 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x56011419d520 con 0x5601142aca20 2024-03-28T18:07:04.185 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:04.182+0000 7f8843400640 10 client.15342 handle_command_reply: tid=0 2024-03-28T18:07:04.185 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:04.183+0000 7f8867603740 1 client.15342 shutdown 2024-03-28T18:07:04.185 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:04.183+0000 7f8867603740 1 -- 172.21.15.22:0/2796915301 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560114296d80 msgr2=0x5601142a1290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:07:04.185 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:04.183+0000 7f8867603740 1 --2- 172.21.15.22:0/2796915301 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560114296d80 0x5601142a1290 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f8850004960 tx=0x7f8850004de0 comp rx=0 tx=0).stop 2024-03-28T18:07:04.185 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:04.183+0000 7f8867603740 1 -- 172.21.15.22:0/2796915301 shutdown_connections 2024-03-28T18:07:04.185 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:04.183+0000 7f8867603740 1 --2- 172.21.15.22:0/2796915301 >> [v2:172.21.15.79:6838/2441542443,v1:172.21.15.79:6839/2441542443] conn(0x5601142aca20 0x5601142b1ae0 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:07:04.185 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:04.183+0000 7f8867603740 1 --2- 172.21.15.22:0/2796915301 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56011429c8e0 0x5601142a8810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:07:04.186 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:04.183+0000 7f8867603740 1 --2- 172.21.15.22:0/2796915301 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x560114297770 0x5601142a17d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:07:04.186 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:04.183+0000 7f8867603740 1 --2- 172.21.15.22:0/2796915301 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560114296d80 0x5601142a1290 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:07:04.186 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:04.183+0000 7f8867603740 1 -- 172.21.15.22:0/2796915301 >> 172.21.15.22:0/2796915301 conn(0x5601142922f0 msgr2=0x5601142914f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:07:04.186 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:04.183+0000 7f8843400640 0 client.15342 ms_handle_reset on v2:172.21.15.79:6838/2441542443 2024-03-28T18:07:04.186 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:04.183+0000 7f8867603740 1 -- 172.21.15.22:0/2796915301 shutdown_connections 2024-03-28T18:07:04.186 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:04.183+0000 7f8867603740 1 -- 172.21.15.22:0/2796915301 wait complete. 2024-03-28T18:07:04.186 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:04.183+0000 7f8867603740 20 client.15342 trim_cache size 0 max 16384 2024-03-28T18:07:04.202 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:04.202+0000 7f8867603740 1 Processor -- start 2024-03-28T18:07:04.202 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:04.202+0000 7f8867603740 1 -- start start 2024-03-28T18:07:04.202 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:04.202+0000 7f8867603740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560114296d80 0x560114290c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:07:04.202 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:04.202+0000 7f8867603740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560114297770 0x5601142a87d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:07:04.202 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:04.202+0000 7f8867603740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56011429c8e0 0x5601142911a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:07:04.202 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:04.202+0000 7f8867603740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x5601142ace80 con 0x560114297770 2024-03-28T18:07:04.202 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:04.202+0000 7f8867603740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x5601142ad4e0 con 0x56011429c8e0 2024-03-28T18:07:04.202 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:04.202+0000 7f8867603740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x5601142adb40 con 0x560114296d80 2024-03-28T18:07:04.204 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:04.203+0000 7f885e800640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56011429c8e0 0x5601142911a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:07:04.204 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:04.203+0000 7f885d400640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560114297770 0x5601142a87d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:07:04.204 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:04.203+0000 7f885d400640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560114297770 0x5601142a87d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:39750/0 (socket says 172.21.15.22:39750) 2024-03-28T18:07:04.204 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:04.203+0000 7f885e800640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56011429c8e0 0x5601142911a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.22:47098/0 (socket says 172.21.15.22:47098) 2024-03-28T18:07:04.204 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:04.203+0000 7f885d400640 1 -- 172.21.15.22:0/208433644 learned_addr learned my addr 172.21.15.22:0/208433644 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-28T18:07:04.204 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:04.203+0000 7f885de00640 1 --2- 172.21.15.22:0/208433644 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560114296d80 0x560114290c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:07:04.204 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:04.203+0000 7f885d400640 1 -- 172.21.15.22:0/208433644 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560114296d80 msgr2=0x560114290c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T18:07:04.204 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:04.203+0000 7f885d400640 1 --2- 172.21.15.22:0/208433644 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560114296d80 0x560114290c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:07:04.204 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:04.203+0000 7f885d400640 1 -- 172.21.15.22:0/208433644 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56011429c8e0 msgr2=0x5601142911a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T18:07:04.204 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:04.203+0000 7f885d400640 1 --2- 172.21.15.22:0/208433644 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56011429c8e0 0x5601142911a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:07:04.204 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:04.204+0000 7f885d400640 1 -- 172.21.15.22:0/208433644 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560114284170 con 0x560114297770 2024-03-28T18:07:04.204 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:04.204+0000 7f885d400640 1 --2- 172.21.15.22:0/208433644 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560114297770 0x5601142a87d0 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f8848009d40 tx=0x7f8848005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5e3d4364973414d server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:07:04.205 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:04.206+0000 7f884e600640 1 -- 172.21.15.22:0/208433644 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8848005020 con 0x560114297770 2024-03-28T18:07:04.205 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:04.206+0000 7f884e600640 1 -- 172.21.15.22:0/208433644 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f88480047a0 con 0x560114297770 2024-03-28T18:07:04.205 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:04.206+0000 7f884e600640 1 -- 172.21.15.22:0/208433644 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f884802a970 con 0x560114297770 2024-03-28T18:07:04.206 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:04.207+0000 7f8867603740 1 -- 172.21.15.22:0/208433644 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560114297770 msgr2=0x5601142a87d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:07:04.207 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:04.207+0000 7f8867603740 1 --2- 172.21.15.22:0/208433644 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560114297770 0x5601142a87d0 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f8848009d40 tx=0x7f8848005f90 comp rx=0 tx=0).stop 2024-03-28T18:07:04.207 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:04.207+0000 7f8867603740 1 -- 172.21.15.22:0/208433644 shutdown_connections 2024-03-28T18:07:04.207 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:04.207+0000 7f8867603740 1 --2- 172.21.15.22:0/208433644 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56011429c8e0 0x5601142911a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:07:04.207 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:04.207+0000 7f8867603740 1 --2- 172.21.15.22:0/208433644 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560114297770 0x5601142a87d0 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:07:04.207 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:04.207+0000 7f8867603740 1 --2- 172.21.15.22:0/208433644 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560114296d80 0x560114290c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:07:04.207 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:04.207+0000 7f8867603740 1 -- 172.21.15.22:0/208433644 >> 172.21.15.22:0/208433644 conn(0x5601142922f0 msgr2=0x5601142926d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:07:04.207 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:04.207+0000 7f8867603740 1 -- 172.21.15.22:0/208433644 shutdown_connections 2024-03-28T18:07:04.207 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:04.207+0000 7f8867603740 1 -- 172.21.15.22:0/208433644 wait complete. 2024-03-28T18:07:04.209 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:04.207+0000 7f8867603740 1 -- start start 2024-03-28T18:07:04.209 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:04.208+0000 7f8867603740 1 Processor -- start 2024-03-28T18:07:04.209 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:04.208+0000 7f8867603740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560114296d80 0x5601142a4310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:07:04.209 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:04.208+0000 7f8867603740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560114297770 0x5601142a4850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:07:04.209 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:04.208+0000 7f8867603740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56011429c8e0 0x5601142a7870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:07:04.209 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:04.208+0000 7f8867603740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x5601142ae800 con 0x560114296d80 2024-03-28T18:07:04.209 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:04.208+0000 7f8867603740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x5601142ae1a0 con 0x56011429c8e0 2024-03-28T18:07:04.209 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:04.208+0000 7f8867603740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x5601142aee60 con 0x560114297770 2024-03-28T18:07:04.209 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:04.208+0000 7f8867603740 10 client.0 _ll_register_callbacks cb 0x7f88480097f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-28T18:07:04.209 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:04.208+0000 7f885de00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560114296d80 0x5601142a4310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:07:04.209 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:04.208+0000 7f885de00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560114296d80 0x5601142a4310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:39756/0 (socket says 172.21.15.22:39756) 2024-03-28T18:07:04.209 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:04.208+0000 7f885de00640 1 -- 172.21.15.22:0/1949991746 learned_addr learned my addr 172.21.15.22:0/1949991746 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-28T18:07:04.209 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:04.208+0000 7f885e800640 1 --2- 172.21.15.22:0/1949991746 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56011429c8e0 0x5601142a7870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:07:04.210 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:04.209+0000 7f885d400640 1 --2- 172.21.15.22:0/1949991746 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560114297770 0x5601142a4850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:07:04.210 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:04.209+0000 7f885de00640 1 -- 172.21.15.22:0/1949991746 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560114297770 msgr2=0x5601142a4850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T18:07:04.210 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:04.209+0000 7f885de00640 1 --2- 172.21.15.22:0/1949991746 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560114297770 0x5601142a4850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:07:04.210 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:04.209+0000 7f885de00640 1 -- 172.21.15.22:0/1949991746 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56011429c8e0 msgr2=0x5601142a7870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T18:07:04.210 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:04.209+0000 7f885de00640 1 --2- 172.21.15.22:0/1949991746 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56011429c8e0 0x5601142a7870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:07:04.210 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:04.209+0000 7f885de00640 1 -- 172.21.15.22:0/1949991746 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5601142a7e60 con 0x560114296d80 2024-03-28T18:07:04.210 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:04.209+0000 7f885e800640 1 --2- 172.21.15.22:0/1949991746 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56011429c8e0 0x5601142a7870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-28T18:07:04.210 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:04.209+0000 7f885de00640 1 --2- 172.21.15.22:0/1949991746 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560114296d80 0x5601142a4310 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f885000bc60 tx=0x7f8850011440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=61be35a8967268b6 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:07:04.210 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:04.209+0000 7f8843400640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-03-28T18:07:04.212 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:04.210+0000 7f8843400640 1 -- 172.21.15.22:0/1949991746 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8850032040 con 0x560114296d80 2024-03-28T18:07:04.213 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:04.210+0000 7f8843400640 1 -- 172.21.15.22:0/1949991746 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 8153+0+0 (secure 0 0 0) 0x7f8850003de0 con 0x560114296d80 2024-03-28T18:07:04.213 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:04.210+0000 7f8867603740 1 -- 172.21.15.22:0/1949991746 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f88480047a0 con 0x560114296d80 2024-03-28T18:07:04.213 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:04.210+0000 7f8843400640 1 -- 172.21.15.22:0/1949991746 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8850004020 con 0x560114296d80 2024-03-28T18:07:04.213 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:04.210+0000 7f8867603740 10 client.15348 fetch_fsmap learned FSMap version 26 2024-03-28T18:07:04.213 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:04.210+0000 7f8867603740 1 -- 172.21.15.22:0/1949991746 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x560114309110 con 0x560114296d80 2024-03-28T18:07:04.213 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:04.210+0000 7f8843400640 1 -- 172.21.15.22:0/1949991746 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f88480047a0 con 0x560114296d80 2024-03-28T18:07:04.213 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:04.210+0000 7f8843400640 1 -- 172.21.15.22:0/1949991746 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 26) v1 ==== 7934+0+0 (secure 0 0 0) 0x7f8850035070 con 0x560114296d80 2024-03-28T18:07:04.213 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:04.210+0000 7f8867603740 10 client.15348 fetch_fsmap finished waiting for FSMap version 26 2024-03-28T18:07:04.213 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:04.210+0000 7f8867603740 10 client.15348 resolve_mds: resolved name 'b' to daemon mds.b 2024-03-28T18:07:04.213 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:04.210+0000 7f8867603740 20 client.15348 populate_metadata read hostname 'smithi022' 2024-03-28T18:07:04.213 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:04.210+0000 7f8867603740 1 --2- 172.21.15.22:0/1949991746 >> [v2:172.21.15.79:6838/2441542443,v1:172.21.15.79:6839/2441542443] conn(0x56011430d750 0x5601143093a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:07:04.213 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:04.211+0000 7f8867603740 4 client.15348 mds_command: new command op to 34553 tid=0 multi_id=0 [{"prefix": "status"}] 2024-03-28T18:07:04.213 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:04.211+0000 7f8867603740 1 -- 172.21.15.22:0/1949991746 --> [v2:172.21.15.79:6838/2441542443,v1:172.21.15.79:6839/2441542443] -- command(tid 0: {"prefix": "status"}) v1 -- 0x5601142091d0 con 0x56011430d750 2024-03-28T18:07:04.214 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:04.211+0000 7f885d400640 1 --2- 172.21.15.22:0/1949991746 >> [v2:172.21.15.79:6838/2441542443,v1:172.21.15.79:6839/2441542443] conn(0x56011430d750 0x5601143093a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:07:04.214 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:04.211+0000 7f885d400640 1 --2- 172.21.15.22:0/1949991746 >> [v2:172.21.15.79:6838/2441542443,v1:172.21.15.79:6839/2441542443] conn(0x56011430d750 0x5601143093a0 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=35f68b9142b08736 server_cookie=be29261e066c42c in_seq=0 out_seq=0 2024-03-28T18:07:04.214 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:04.211+0000 7f8843400640 10 client.15348 ms_handle_connect on v2:172.21.15.79:6838/2441542443 2024-03-28T18:07:04.214 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:04.212+0000 7f8843400640 1 -- 172.21.15.22:0/1949991746 <== mds.0 v2:172.21.15.79:6838/2441542443 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+327 (crc 0 0 0) 0x5601142091d0 con 0x56011430d750 2024-03-28T18:07:04.214 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:04.212+0000 7f8843400640 10 client.15348 handle_command_reply: tid=0 2024-03-28T18:07:04.214 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:04.212+0000 7f8867603740 1 client.15348 shutdown 2024-03-28T18:07:04.214 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:04.212+0000 7f8867603740 1 -- 172.21.15.22:0/1949991746 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560114296d80 msgr2=0x5601142a4310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:07:04.214 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:04.212+0000 7f8867603740 1 --2- 172.21.15.22:0/1949991746 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560114296d80 0x5601142a4310 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f885000bc60 tx=0x7f8850011440 comp rx=0 tx=0).stop 2024-03-28T18:07:04.214 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:04.212+0000 7f8867603740 1 -- 172.21.15.22:0/1949991746 shutdown_connections 2024-03-28T18:07:04.214 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:04.212+0000 7f8867603740 1 --2- 172.21.15.22:0/1949991746 >> [v2:172.21.15.79:6838/2441542443,v1:172.21.15.79:6839/2441542443] conn(0x56011430d750 0x5601143093a0 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:07:04.214 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:04.212+0000 7f8867603740 1 --2- 172.21.15.22:0/1949991746 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56011429c8e0 0x5601142a7870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:07:04.214 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:04.212+0000 7f8867603740 1 --2- 172.21.15.22:0/1949991746 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560114297770 0x5601142a4850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:07:04.214 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:04.212+0000 7f8867603740 1 --2- 172.21.15.22:0/1949991746 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560114296d80 0x5601142a4310 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:07:04.214 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:04.212+0000 7f8867603740 1 -- 172.21.15.22:0/1949991746 >> 172.21.15.22:0/1949991746 conn(0x5601142922f0 msgr2=0x560114295550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:07:04.215 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:04.212+0000 7f8843400640 0 client.15348 ms_handle_reset on v2:172.21.15.79:6838/2441542443 2024-03-28T18:07:04.215 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:04.212+0000 7f8867603740 1 -- 172.21.15.22:0/1949991746 shutdown_connections 2024-03-28T18:07:04.215 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:04.212+0000 7f8867603740 1 -- 172.21.15.22:0/1949991746 wait complete. 2024-03-28T18:07:04.215 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:04.212+0000 7f8867603740 20 client.15348 trim_cache size 0 max 16384 2024-03-28T18:07:04.215 INFO:teuthology.orchestra.run.smithi022.stdout:{ 2024-03-28T18:07:04.215 INFO:teuthology.orchestra.run.smithi022.stdout: "cluster_fsid": "fd220748-ed2b-11ee-95d0-87774f69a715", 2024-03-28T18:07:04.215 INFO:teuthology.orchestra.run.smithi022.stdout: "whoami": 0, 2024-03-28T18:07:04.215 INFO:teuthology.orchestra.run.smithi022.stdout: "id": 34553, 2024-03-28T18:07:04.215 INFO:teuthology.orchestra.run.smithi022.stdout: "want_state": "up:active", 2024-03-28T18:07:04.215 INFO:teuthology.orchestra.run.smithi022.stdout: "state": "up:active", 2024-03-28T18:07:04.215 INFO:teuthology.orchestra.run.smithi022.stdout: "fs_name": "cephfs", 2024-03-28T18:07:04.215 INFO:teuthology.orchestra.run.smithi022.stdout: "rank_uptime": 134.35796642400001, 2024-03-28T18:07:04.216 INFO:teuthology.orchestra.run.smithi022.stdout: "mdsmap_epoch": 26, 2024-03-28T18:07:04.216 INFO:teuthology.orchestra.run.smithi022.stdout: "osdmap_epoch": 92, 2024-03-28T18:07:04.216 INFO:teuthology.orchestra.run.smithi022.stdout: "osdmap_epoch_barrier": 90, 2024-03-28T18:07:04.216 INFO:teuthology.orchestra.run.smithi022.stdout: "uptime": 139.932379693 2024-03-28T18:07:04.216 INFO:teuthology.orchestra.run.smithi022.stdout:} 2024-03-28T18:07:04.217 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:04.213+0000 7f885f200640 1 -- 172.21.15.22:0/2272608175 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7f88440a2800 msgr2=0x7f88440a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:07:04.217 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:04.213+0000 7f885f200640 1 --2- 172.21.15.22:0/2272608175 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7f88440a2800 0x7f88440a4cc0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x560114297fd0 tx=0x7f8848012040 comp rx=0 tx=0).stop 2024-03-28T18:07:04.217 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:04.213+0000 7f885f200640 1 -- 172.21.15.22:0/2272608175 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f8860109430 msgr2=0x7f88601a0fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:07:04.217 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:04.213+0000 7f885f200640 1 --2- 172.21.15.22:0/2272608175 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f8860109430 0x7f88601a0fd0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f8858032650 tx=0x7f8858034cf0 comp rx=0 tx=0).stop 2024-03-28T18:07:04.217 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:04.213+0000 7f885f200640 1 -- 172.21.15.22:0/2272608175 shutdown_connections 2024-03-28T18:07:04.217 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:04.213+0000 7f885f200640 1 --2- 172.21.15.22:0/2272608175 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7f88440a2800 0x7f88440a4cc0 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:07:04.217 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:04.213+0000 7f885f200640 1 --2- 172.21.15.22:0/2272608175 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f8860109430 0x7f88601a0fd0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:07:04.217 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:04.213+0000 7f885f200640 1 --2- 172.21.15.22:0/2272608175 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8860075680 0x7f8860199f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:07:04.217 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:04.213+0000 7f885f200640 1 --2- 172.21.15.22:0/2272608175 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f88600740a0 0x7f8860199a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:07:04.217 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:04.213+0000 7f885f200640 1 -- 172.21.15.22:0/2272608175 >> 172.21.15.22:0/2272608175 conn(0x7f88600fcb10 msgr2=0x7f88601053c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:07:04.217 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:04.213+0000 7f885f200640 1 -- 172.21.15.22:0/2272608175 shutdown_connections 2024-03-28T18:07:04.217 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:04.213+0000 7f885f200640 1 -- 172.21.15.22:0/2272608175 wait complete. 2024-03-28T18:07:04.231 INFO:teuthology.run_tasks:Running task ceph-fuse... 2024-03-28T18:07:04.255 INFO:tasks.ceph_fuse:Running ceph_fuse task... 2024-03-28T18:07:04.255 INFO:tasks.ceph_fuse:config is {'client.0': {}} 2024-03-28T18:07:04.255 INFO:tasks.ceph_fuse:client.0 config is {'mount_subvol_num': 0} 2024-03-28T18:07:04.256 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-03-28T18:07:04.256 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-03-28T18:07:04.256 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-28T18:07:04.256 DEBUG:teuthology.orchestra.run.smithi022:> ip netns list 2024-03-28T18:07:04.316 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-28T18:07:04.316 DEBUG:teuthology.orchestra.run.smithi022:> sudo ip link delete ceph-brx 2024-03-28T18:07:04.424 INFO:teuthology.orchestra.run.smithi022.stderr:Cannot find device "ceph-brx" 2024-03-28T18:07:04.430 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-28T18:07:04.431 INFO:tasks.ceph_fuse:Mounting ceph-fuse clients... 2024-03-28T18:07:04.431 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033 shell --fsid fd220748-ed2b-11ee-95d0-87774f69a715 -- ceph fs ls 2024-03-28T18:07:04.520 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:07:04 smithi022 ceph-mon[31985]: pgmap v438: 97 pgs: 97 active+clean; 643 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-03-28T18:07:04.674 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:07:04 smithi134 ceph-mon[35037]: pgmap v438: 97 pgs: 97 active+clean; 643 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-03-28T18:07:04.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:07:04 smithi079 ceph-mon[35312]: pgmap v438: 97 pgs: 97 active+clean; 643 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-03-28T18:07:04.900 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/fd220748-ed2b-11ee-95d0-87774f69a715/mon.a/config 2024-03-28T18:07:06.479 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:07:06 smithi022 ceph-mon[31985]: pgmap v439: 97 pgs: 97 active+clean; 643 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-28T18:07:06.674 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:07:06 smithi134 ceph-mon[35037]: pgmap v439: 97 pgs: 97 active+clean; 643 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-28T18:07:06.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:07:06 smithi079 ceph-mon[35312]: pgmap v439: 97 pgs: 97 active+clean; 643 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-28T18:07:07.220 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:07.220+0000 7f8b2a200700 1 -- 172.21.15.22:0/2460005966 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8b241062c0 msgr2=0x7f8b2410de90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:07:07.221 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:07.220+0000 7f8b2a200700 1 --2- 172.21.15.22:0/2460005966 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8b241062c0 0x7f8b2410de90 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f8b1c0099c0 tx=0x7f8b1c009cd0 comp rx=0 tx=0).stop 2024-03-28T18:07:07.221 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:07.220+0000 7f8b2a200700 1 -- 172.21.15.22:0/2460005966 shutdown_connections 2024-03-28T18:07:07.221 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:07.220+0000 7f8b2a200700 1 --2- 172.21.15.22:0/2460005966 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8b241062c0 0x7f8b2410de90 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:07:07.221 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:07.220+0000 7f8b2a200700 1 --2- 172.21.15.22:0/2460005966 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f8b241011e0 0x7f8b241015c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:07:07.221 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:07.220+0000 7f8b2a200700 1 --2- 172.21.15.22:0/2460005966 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8b24101b90 0x7f8b24105be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:07:07.221 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:07.220+0000 7f8b2a200700 1 -- 172.21.15.22:0/2460005966 >> 172.21.15.22:0/2460005966 conn(0x7f8b240fca50 msgr2=0x7f8b240fee70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:07:07.221 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:07.220+0000 7f8b2a200700 1 -- 172.21.15.22:0/2460005966 shutdown_connections 2024-03-28T18:07:07.221 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:07.220+0000 7f8b2a200700 1 -- 172.21.15.22:0/2460005966 wait complete. 2024-03-28T18:07:07.222 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:07.222+0000 7f8b2a200700 1 Processor -- start 2024-03-28T18:07:07.222 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:07.223+0000 7f8b2a200700 1 -- start start 2024-03-28T18:07:07.223 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:07.223+0000 7f8b2a200700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8b241011e0 0x7f8b241a06b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:07:07.223 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:07.223+0000 7f8b2a200700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f8b24101b90 0x7f8b241a0bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:07:07.223 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:07.223+0000 7f8b2a200700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8b241062c0 0x7f8b2419a8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:07:07.223 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:07.223+0000 7f8b2a200700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b24111310 con 0x7f8b241011e0 2024-03-28T18:07:07.223 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:07.223+0000 7f8b2a200700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b24111190 con 0x7f8b24101b90 2024-03-28T18:07:07.223 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:07.223+0000 7f8b2a200700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b24111490 con 0x7f8b241062c0 2024-03-28T18:07:07.224 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:07.224+0000 7f8b23e00700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f8b24101b90 0x7f8b241a0bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:07:07.224 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:07.224+0000 7f8b23e00700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f8b24101b90 0x7f8b241a0bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.22:44176/0 (socket says 172.21.15.22:44176) 2024-03-28T18:07:07.224 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:07.224+0000 7f8b23e00700 1 -- 172.21.15.22:0/2688427705 learned_addr learned my addr 172.21.15.22:0/2688427705 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-28T18:07:07.224 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:07.224+0000 7f8b29800700 1 --2- 172.21.15.22:0/2688427705 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8b241062c0 0x7f8b2419a8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:07:07.224 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:07.224+0000 7f8b28e00700 1 --2- 172.21.15.22:0/2688427705 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8b241011e0 0x7f8b241a06b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:07:07.224 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:07.224+0000 7f8b28e00700 1 -- 172.21.15.22:0/2688427705 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8b241062c0 msgr2=0x7f8b2419a8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:07:07.224 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:07.224+0000 7f8b28e00700 1 --2- 172.21.15.22:0/2688427705 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8b241062c0 0x7f8b2419a8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:07:07.224 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:07.224+0000 7f8b28e00700 1 -- 172.21.15.22:0/2688427705 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f8b24101b90 msgr2=0x7f8b241a0bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:07:07.225 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:07.224+0000 7f8b28e00700 1 --2- 172.21.15.22:0/2688427705 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f8b24101b90 0x7f8b241a0bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:07:07.225 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:07.224+0000 7f8b28e00700 1 -- 172.21.15.22:0/2688427705 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b2419b020 con 0x7f8b241011e0 2024-03-28T18:07:07.225 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:07.224+0000 7f8b23e00700 1 --2- 172.21.15.22:0/2688427705 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f8b24101b90 0x7f8b241a0bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-28T18:07:07.225 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:07.225+0000 7f8b28e00700 1 --2- 172.21.15.22:0/2688427705 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8b241011e0 0x7f8b241a06b0 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f8b1800efc0 tx=0x7f8b1800c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:07:07.225 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:07.225+0000 7f8b21600700 1 -- 172.21.15.22:0/2688427705 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b180092e0 con 0x7f8b241011e0 2024-03-28T18:07:07.225 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:07.225+0000 7f8b21600700 1 -- 172.21.15.22:0/2688427705 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f8b18003a00 con 0x7f8b241011e0 2024-03-28T18:07:07.226 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:07.225+0000 7f8b2a200700 1 -- 172.21.15.22:0/2688427705 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8b2419b2b0 con 0x7f8b241011e0 2024-03-28T18:07:07.226 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:07.225+0000 7f8b21600700 1 -- 172.21.15.22:0/2688427705 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b18004600 con 0x7f8b241011e0 2024-03-28T18:07:07.226 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:07.225+0000 7f8b2a200700 1 -- 172.21.15.22:0/2688427705 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8b241a7580 con 0x7f8b241011e0 2024-03-28T18:07:07.226 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:07.227+0000 7f8b2a200700 1 -- 172.21.15.22:0/2688427705 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8b24068e50 con 0x7f8b241011e0 2024-03-28T18:07:07.228 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:07.229+0000 7f8b21600700 1 -- 172.21.15.22:0/2688427705 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 18) v1 ==== 137290+0+0 (secure 0 0 0) 0x7f8b1801b030 con 0x7f8b241011e0 2024-03-28T18:07:07.230 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:07.230+0000 7f8b21600700 1 --2- 172.21.15.22:0/2688427705 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7f8b0c0a2890 0x7f8b0c0a4d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:07:07.230 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:07.230+0000 7f8b21600700 1 -- 172.21.15.22:0/2688427705 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 8153+0+0 (secure 0 0 0) 0x7f8b180ce3c0 con 0x7f8b241011e0 2024-03-28T18:07:07.230 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:07.230+0000 7f8b23e00700 1 --2- 172.21.15.22:0/2688427705 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7f8b0c0a2890 0x7f8b0c0a4d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:07:07.230 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:07.231+0000 7f8b23e00700 1 --2- 172.21.15.22:0/2688427705 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7f8b0c0a2890 0x7f8b0c0a4d50 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f8b2406b410 tx=0x7f8b1000b460 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:07:07.234 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:07.235+0000 7f8b21600700 1 -- 172.21.15.22:0/2688427705 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f8b18022020 con 0x7f8b241011e0 2024-03-28T18:07:07.511 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:07.512+0000 7f8b2a200700 1 -- 172.21.15.22:0/2688427705 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f8b241a7750 con 0x7f8b241011e0 2024-03-28T18:07:07.512 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:07.512+0000 7f8b21600700 1 -- 172.21.15.22:0/2688427705 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v26) v1 ==== 53+0+73 (secure 0 0 0) 0x7f8b18010040 con 0x7f8b241011e0 2024-03-28T18:07:07.512 INFO:teuthology.orchestra.run.smithi022.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-03-28T18:07:07.516 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:07.516+0000 7f8b2a200700 1 -- 172.21.15.22:0/2688427705 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7f8b0c0a2890 msgr2=0x7f8b0c0a4d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:07:07.516 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:07.516+0000 7f8b2a200700 1 --2- 172.21.15.22:0/2688427705 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7f8b0c0a2890 0x7f8b0c0a4d50 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f8b2406b410 tx=0x7f8b1000b460 comp rx=0 tx=0).stop 2024-03-28T18:07:07.516 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:07.516+0000 7f8b2a200700 1 -- 172.21.15.22:0/2688427705 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8b241011e0 msgr2=0x7f8b241a06b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:07:07.516 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:07.516+0000 7f8b2a200700 1 --2- 172.21.15.22:0/2688427705 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8b241011e0 0x7f8b241a06b0 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f8b1800efc0 tx=0x7f8b1800c3a0 comp rx=0 tx=0).stop 2024-03-28T18:07:07.516 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:07.516+0000 7f8b2a200700 1 -- 172.21.15.22:0/2688427705 shutdown_connections 2024-03-28T18:07:07.516 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:07.516+0000 7f8b2a200700 1 --2- 172.21.15.22:0/2688427705 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7f8b0c0a2890 0x7f8b0c0a4d50 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:07:07.516 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:07.516+0000 7f8b2a200700 1 --2- 172.21.15.22:0/2688427705 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8b241062c0 0x7f8b2419a8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:07:07.516 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:07.516+0000 7f8b2a200700 1 --2- 172.21.15.22:0/2688427705 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8b241011e0 0x7f8b241a06b0 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:07:07.516 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:07.516+0000 7f8b2a200700 1 --2- 172.21.15.22:0/2688427705 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f8b24101b90 0x7f8b241a0bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:07:07.516 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:07.516+0000 7f8b2a200700 1 -- 172.21.15.22:0/2688427705 >> 172.21.15.22:0/2688427705 conn(0x7f8b240fca50 msgr2=0x7f8b2410cdd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:07:07.517 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:07.516+0000 7f8b2a200700 1 -- 172.21.15.22:0/2688427705 shutdown_connections 2024-03-28T18:07:07.517 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:07.516+0000 7f8b2a200700 1 -- 172.21.15.22:0/2688427705 wait complete. 2024-03-28T18:07:08.421 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-03-28T18:07:08.421 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-03-28T18:07:08.422 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi022.front.sepia.ceph.com 2024-03-28T18:07:08.422 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-03-28T18:07:08.422 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-03-28T18:07:08.422 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-03-28T18:07:08.422 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-03-28T18:07:08.422 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-03-28T18:07:08.422 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.0' 2024-03-28T18:07:08.422 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-28T18:07:08.422 DEBUG:teuthology.orchestra.run.smithi022:> ip addr 2024-03-28T18:07:08.475 INFO:teuthology.orchestra.run.smithi022.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-03-28T18:07:08.476 INFO:teuthology.orchestra.run.smithi022.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-03-28T18:07:08.476 INFO:teuthology.orchestra.run.smithi022.stdout: inet 127.0.0.1/8 scope host lo 2024-03-28T18:07:08.476 INFO:teuthology.orchestra.run.smithi022.stdout: valid_lft forever preferred_lft forever 2024-03-28T18:07:08.476 INFO:teuthology.orchestra.run.smithi022.stdout: inet6 ::1/128 scope host 2024-03-28T18:07:08.476 INFO:teuthology.orchestra.run.smithi022.stdout: valid_lft forever preferred_lft forever 2024-03-28T18:07:08.476 INFO:teuthology.orchestra.run.smithi022.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-03-28T18:07:08.476 INFO:teuthology.orchestra.run.smithi022.stdout: link/ether 0c:c4:7a:6b:fd:9a brd ff:ff:ff:ff:ff:ff 2024-03-28T18:07:08.476 INFO:teuthology.orchestra.run.smithi022.stdout: altname enp7s0f0 2024-03-28T18:07:08.477 INFO:teuthology.orchestra.run.smithi022.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-03-28T18:07:08.477 INFO:teuthology.orchestra.run.smithi022.stdout: link/ether 0c:c4:7a:6b:fd:9b brd ff:ff:ff:ff:ff:ff 2024-03-28T18:07:08.477 INFO:teuthology.orchestra.run.smithi022.stdout: altname enp7s0f1 2024-03-28T18:07:08.477 INFO:teuthology.orchestra.run.smithi022.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-03-28T18:07:08.477 INFO:teuthology.orchestra.run.smithi022.stdout: link/ether 0c:c4:7a:59:5d:e4 brd ff:ff:ff:ff:ff:ff 2024-03-28T18:07:08.477 INFO:teuthology.orchestra.run.smithi022.stdout: altname enp1s0f0 2024-03-28T18:07:08.477 INFO:teuthology.orchestra.run.smithi022.stdout: inet 172.21.15.22/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2024-03-28T18:07:08.477 INFO:teuthology.orchestra.run.smithi022.stdout: valid_lft 41767sec preferred_lft 41767sec 2024-03-28T18:07:08.477 INFO:teuthology.orchestra.run.smithi022.stdout: inet6 fe80::ec4:7aff:fe59:5de4/64 scope link noprefixroute 2024-03-28T18:07:08.477 INFO:teuthology.orchestra.run.smithi022.stdout: valid_lft forever preferred_lft forever 2024-03-28T18:07:08.477 INFO:teuthology.orchestra.run.smithi022.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-03-28T18:07:08.477 INFO:teuthology.orchestra.run.smithi022.stdout: link/ether 0c:c4:7a:59:5d:e5 brd ff:ff:ff:ff:ff:ff 2024-03-28T18:07:08.477 INFO:teuthology.orchestra.run.smithi022.stdout: altname enp1s0f1 2024-03-28T18:07:08.478 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-03-28T18:07:08.479 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-28T18:07:08.479 DEBUG:teuthology.orchestra.run.smithi022:> (cd / && exec bash -c ' 2024-03-28T18:07:08.479 DEBUG:teuthology.orchestra.run.smithi022:> set -e 2024-03-28T18:07:08.479 DEBUG:teuthology.orchestra.run.smithi022:> sudo ip link add name ceph-brx type bridge 2024-03-28T18:07:08.479 DEBUG:teuthology.orchestra.run.smithi022:> sudo ip addr flush dev ceph-brx 2024-03-28T18:07:08.479 DEBUG:teuthology.orchestra.run.smithi022:> sudo ip link set ceph-brx up 2024-03-28T18:07:08.480 DEBUG:teuthology.orchestra.run.smithi022:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-03-28T18:07:08.480 DEBUG:teuthology.orchestra.run.smithi022:> ') 2024-03-28T18:07:08.483 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:07:08 smithi079 ceph-mon[35312]: pgmap v440: 97 pgs: 97 active+clean; 643 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2024-03-28T18:07:08.484 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:07:08 smithi079 ceph-mon[35312]: from='client.? 172.21.15.22:0/2688427705' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-03-28T18:07:08.531 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:07:08 smithi022 ceph-mon[31985]: pgmap v440: 97 pgs: 97 active+clean; 643 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2024-03-28T18:07:08.531 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:07:08 smithi022 ceph-mon[31985]: from='client.? 172.21.15.22:0/2688427705' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-03-28T18:07:08.674 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:07:08 smithi134 ceph-mon[35037]: pgmap v440: 97 pgs: 97 active+clean; 643 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2024-03-28T18:07:08.674 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:07:08 smithi134 ceph-mon[35037]: from='client.? 172.21.15.22:0/2688427705' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-03-28T18:07:08.905 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-28T18:07:08.905 DEBUG:teuthology.orchestra.run.smithi022:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-03-28T18:07:08.988 INFO:teuthology.orchestra.run.smithi022.stdout:1 2024-03-28T18:07:08.992 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-28T18:07:08.992 DEBUG:teuthology.orchestra.run.smithi022:> route 2024-03-28T18:07:09.043 INFO:teuthology.orchestra.run.smithi022.stdout:Kernel IP routing table 2024-03-28T18:07:09.043 INFO:teuthology.orchestra.run.smithi022.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-03-28T18:07:09.044 INFO:teuthology.orchestra.run.smithi022.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-03-28T18:07:09.044 INFO:teuthology.orchestra.run.smithi022.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-03-28T18:07:09.044 INFO:teuthology.orchestra.run.smithi022.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-03-28T18:07:09.044 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-28T18:07:09.044 DEBUG:teuthology.orchestra.run.smithi022:> (cd / && exec bash -c ' 2024-03-28T18:07:09.044 DEBUG:teuthology.orchestra.run.smithi022:> set -e 2024-03-28T18:07:09.044 DEBUG:teuthology.orchestra.run.smithi022:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-03-28T18:07:09.044 DEBUG:teuthology.orchestra.run.smithi022:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-03-28T18:07:09.044 DEBUG:teuthology.orchestra.run.smithi022:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-03-28T18:07:09.044 DEBUG:teuthology.orchestra.run.smithi022:> ') 2024-03-28T18:07:09.751 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-28T18:07:09.752 DEBUG:teuthology.orchestra.run.smithi022:> ip netns list 2024-03-28T18:07:09.811 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-28T18:07:09.811 DEBUG:teuthology.orchestra.run.smithi022:> ip netns list-id 2024-03-28T18:07:09.867 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-28T18:07:09.867 DEBUG:teuthology.orchestra.run.smithi022:> (cd / && exec bash -c ' 2024-03-28T18:07:09.867 DEBUG:teuthology.orchestra.run.smithi022:> set -e 2024-03-28T18:07:09.867 DEBUG:teuthology.orchestra.run.smithi022:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.0 2024-03-28T18:07:09.867 DEBUG:teuthology.orchestra.run.smithi022:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.0 0 2024-03-28T18:07:09.867 DEBUG:teuthology.orchestra.run.smithi022:> ') 2024-03-28T18:07:10.019 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.0' with 192.168.0.1/16 2024-03-28T18:07:10.020 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-28T18:07:10.020 DEBUG:teuthology.orchestra.run.smithi022:> (cd / && exec bash -c ' 2024-03-28T18:07:10.020 DEBUG:teuthology.orchestra.run.smithi022:> set -e 2024-03-28T18:07:10.020 DEBUG:teuthology.orchestra.run.smithi022:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.0 type veth peer name brx.0 2024-03-28T18:07:10.020 DEBUG:teuthology.orchestra.run.smithi022:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-03-28T18:07:10.020 DEBUG:teuthology.orchestra.run.smithi022:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set veth0 up 2024-03-28T18:07:10.020 DEBUG:teuthology.orchestra.run.smithi022:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set lo up 2024-03-28T18:07:10.020 DEBUG:teuthology.orchestra.run.smithi022:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip route add default via 192.168.255.254 2024-03-28T18:07:10.020 DEBUG:teuthology.orchestra.run.smithi022:> ') 2024-03-28T18:07:10.260 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:07:10 smithi022 ceph-mon[31985]: pgmap v441: 97 pgs: 97 active+clean; 643 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-03-28T18:07:10.322 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-28T18:07:10.322 DEBUG:teuthology.orchestra.run.smithi022:> (cd / && exec bash -c ' 2024-03-28T18:07:10.322 DEBUG:teuthology.orchestra.run.smithi022:> set -e 2024-03-28T18:07:10.322 DEBUG:teuthology.orchestra.run.smithi022:> sudo ip link set brx.0 up 2024-03-28T18:07:10.322 DEBUG:teuthology.orchestra.run.smithi022:> sudo ip link set dev brx.0 master ceph-brx 2024-03-28T18:07:10.322 DEBUG:teuthology.orchestra.run.smithi022:> ') 2024-03-28T18:07:10.501 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {'mount_subvol_num': 0} 2024-03-28T18:07:10.501 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-03-28T18:07:10.502 DEBUG:teuthology.orchestra.run.smithi022:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-03-28T18:07:10.551 INFO:teuthology.orchestra.run.smithi022.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-03-28T18:07:10.551 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-03-28T18:07:10.551 DEBUG:teuthology.orchestra.run.smithi022:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-03-28T18:07:10.596 DEBUG:teuthology.orchestra.run.smithi022:> sudo modprobe fuse 2024-03-28T18:07:10.674 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:07:10 smithi134 ceph-mon[35037]: pgmap v441: 97 pgs: 97 active+clean; 643 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-03-28T18:07:10.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:07:10 smithi079 ceph-mon[35312]: pgmap v441: 97 pgs: 97 active+clean; 643 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-03-28T18:07:10.741 DEBUG:teuthology.orchestra.run.smithi022:> cat /proc/self/mounts | awk '{print $2}' 2024-03-28T18:07:10.792 INFO:teuthology.orchestra.run.smithi022.stdout:/proc 2024-03-28T18:07:10.793 INFO:teuthology.orchestra.run.smithi022.stdout:/sys 2024-03-28T18:07:10.793 INFO:teuthology.orchestra.run.smithi022.stdout:/dev 2024-03-28T18:07:10.793 INFO:teuthology.orchestra.run.smithi022.stdout:/sys/kernel/security 2024-03-28T18:07:10.793 INFO:teuthology.orchestra.run.smithi022.stdout:/dev/shm 2024-03-28T18:07:10.793 INFO:teuthology.orchestra.run.smithi022.stdout:/dev/pts 2024-03-28T18:07:10.793 INFO:teuthology.orchestra.run.smithi022.stdout:/run 2024-03-28T18:07:10.793 INFO:teuthology.orchestra.run.smithi022.stdout:/sys/fs/cgroup 2024-03-28T18:07:10.793 INFO:teuthology.orchestra.run.smithi022.stdout:/sys/fs/pstore 2024-03-28T18:07:10.793 INFO:teuthology.orchestra.run.smithi022.stdout:/sys/fs/bpf 2024-03-28T18:07:10.793 INFO:teuthology.orchestra.run.smithi022.stdout:/ 2024-03-28T18:07:10.793 INFO:teuthology.orchestra.run.smithi022.stdout:/sys/fs/selinux 2024-03-28T18:07:10.793 INFO:teuthology.orchestra.run.smithi022.stdout:/proc/sys/fs/binfmt_misc 2024-03-28T18:07:10.793 INFO:teuthology.orchestra.run.smithi022.stdout:/dev/mqueue 2024-03-28T18:07:10.793 INFO:teuthology.orchestra.run.smithi022.stdout:/dev/hugepages 2024-03-28T18:07:10.794 INFO:teuthology.orchestra.run.smithi022.stdout:/sys/kernel/debug 2024-03-28T18:07:10.794 INFO:teuthology.orchestra.run.smithi022.stdout:/sys/kernel/tracing 2024-03-28T18:07:10.794 INFO:teuthology.orchestra.run.smithi022.stdout:/sys/fs/fuse/connections 2024-03-28T18:07:10.794 INFO:teuthology.orchestra.run.smithi022.stdout:/sys/kernel/config 2024-03-28T18:07:10.794 INFO:teuthology.orchestra.run.smithi022.stdout:/run/credentials/systemd-sysctl.service 2024-03-28T18:07:10.794 INFO:teuthology.orchestra.run.smithi022.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-03-28T18:07:10.794 INFO:teuthology.orchestra.run.smithi022.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-03-28T18:07:10.794 INFO:teuthology.orchestra.run.smithi022.stdout:/var/lib/nfs/rpc_pipefs 2024-03-28T18:07:10.794 INFO:teuthology.orchestra.run.smithi022.stdout:/run/user/1000 2024-03-28T18:07:10.794 INFO:teuthology.orchestra.run.smithi022.stdout:/var/lib/ceph 2024-03-28T18:07:10.794 INFO:teuthology.orchestra.run.smithi022.stdout:/var/lib/containers/storage/overlay 2024-03-28T18:07:10.794 INFO:teuthology.orchestra.run.smithi022.stdout:/var/lib/containers/storage/overlay/0003813f3e70bfdfafc3c0cbf156b052d465bc8dcad7f4363f448ba06e91413f/merged 2024-03-28T18:07:10.794 INFO:teuthology.orchestra.run.smithi022.stdout:/var/lib/containers/storage/overlay/0cb78753be0f6201f840a9bf93c4983cc9d5a317f94a5e5b7e64ccb76f15cf54/merged 2024-03-28T18:07:10.794 INFO:teuthology.orchestra.run.smithi022.stdout:/run/user/0 2024-03-28T18:07:10.795 INFO:teuthology.orchestra.run.smithi022.stdout:/proc/sys/fs/binfmt_misc 2024-03-28T18:07:10.795 INFO:teuthology.orchestra.run.smithi022.stdout:/var/lib/containers/storage/overlay/1d863290e69c851d1ca6d51aa2c70f306b45f930b5fc35cf549f09325b5140c2/merged 2024-03-28T18:07:10.795 INFO:teuthology.orchestra.run.smithi022.stdout:/var/lib/containers/storage/overlay/265bc374ecad4273bc50d57a5fb40e2972576732378716b3f3248f51617c3b7a/merged 2024-03-28T18:07:10.795 INFO:teuthology.orchestra.run.smithi022.stdout:/var/lib/containers/storage/overlay/b5595bf22c867e6a9731654c404aaee67c2857c5ef0edb7127e7bf846942c5d9/merged 2024-03-28T18:07:10.795 INFO:teuthology.orchestra.run.smithi022.stdout:/var/lib/containers/storage/overlay/43b5e66d6307c10cafcd8e723cc0a4582d9f3f4502ca0c67fcd575a1dbb558f6/merged 2024-03-28T18:07:10.795 INFO:teuthology.orchestra.run.smithi022.stdout:/var/lib/containers/storage/overlay/991b9aad6d7a5bf2da739ec1ba234d7e296fdd404d12f6d554869fe3d9627162/merged 2024-03-28T18:07:10.795 INFO:teuthology.orchestra.run.smithi022.stdout:/var/lib/containers/storage/overlay/3b3cb2e3b8f31cec0401cf2979e589b3c556473c10e024a53b3d84babcf9e1fd/merged 2024-03-28T18:07:10.795 INFO:teuthology.orchestra.run.smithi022.stdout:/var/lib/containers/storage/overlay/c7bbd21cb7608d09f6124cad4abd42fbb82486881112d646c9c06966df4423b4/merged 2024-03-28T18:07:10.795 INFO:teuthology.orchestra.run.smithi022.stdout:/var/lib/containers/storage/overlay/2235f3e743ce7906ae4c5671def2c41155829abfeeb7394efdfc5e34f780cf0a/merged 2024-03-28T18:07:10.795 INFO:teuthology.orchestra.run.smithi022.stdout:/run/netns 2024-03-28T18:07:10.795 INFO:teuthology.orchestra.run.smithi022.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-03-28T18:07:10.795 INFO:teuthology.orchestra.run.smithi022.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-03-28T18:07:10.796 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-28T18:07:10.796 DEBUG:teuthology.orchestra.run.smithi022:> ls /sys/fs/fuse/connections 2024-03-28T18:07:10.846 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-03-28T18:07:10.846 DEBUG:teuthology.orchestra.run.smithi022:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0 --client_mountpoint=/volumes/qa/sv_0/abc125a7-3abc-4b82-bcad-de8d02f4df3d --client_fs=cephfs) 2024-03-28T18:07:10.851 DEBUG:teuthology.orchestra.run.smithi022:> sudo modprobe fuse 2024-03-28T18:07:10.926 DEBUG:teuthology.orchestra.run.smithi022:> cat /proc/self/mounts | awk '{print $2}' 2024-03-28T18:07:10.971 INFO:teuthology.orchestra.run.smithi022.stdout:/proc 2024-03-28T18:07:10.971 INFO:teuthology.orchestra.run.smithi022.stdout:/sys 2024-03-28T18:07:10.971 INFO:teuthology.orchestra.run.smithi022.stdout:/dev 2024-03-28T18:07:10.971 INFO:teuthology.orchestra.run.smithi022.stdout:/sys/kernel/security 2024-03-28T18:07:10.971 INFO:teuthology.orchestra.run.smithi022.stdout:/dev/shm 2024-03-28T18:07:10.971 INFO:teuthology.orchestra.run.smithi022.stdout:/dev/pts 2024-03-28T18:07:10.971 INFO:teuthology.orchestra.run.smithi022.stdout:/run 2024-03-28T18:07:10.971 INFO:teuthology.orchestra.run.smithi022.stdout:/sys/fs/cgroup 2024-03-28T18:07:10.971 INFO:teuthology.orchestra.run.smithi022.stdout:/sys/fs/pstore 2024-03-28T18:07:10.972 INFO:teuthology.orchestra.run.smithi022.stdout:/sys/fs/bpf 2024-03-28T18:07:10.972 INFO:teuthology.orchestra.run.smithi022.stdout:/ 2024-03-28T18:07:10.972 INFO:teuthology.orchestra.run.smithi022.stdout:/sys/fs/selinux 2024-03-28T18:07:10.972 INFO:teuthology.orchestra.run.smithi022.stdout:/proc/sys/fs/binfmt_misc 2024-03-28T18:07:10.972 INFO:teuthology.orchestra.run.smithi022.stdout:/dev/mqueue 2024-03-28T18:07:10.972 INFO:teuthology.orchestra.run.smithi022.stdout:/dev/hugepages 2024-03-28T18:07:10.972 INFO:teuthology.orchestra.run.smithi022.stdout:/sys/kernel/debug 2024-03-28T18:07:10.972 INFO:teuthology.orchestra.run.smithi022.stdout:/sys/kernel/tracing 2024-03-28T18:07:10.972 INFO:teuthology.orchestra.run.smithi022.stdout:/sys/fs/fuse/connections 2024-03-28T18:07:10.972 INFO:teuthology.orchestra.run.smithi022.stdout:/sys/kernel/config 2024-03-28T18:07:10.972 INFO:teuthology.orchestra.run.smithi022.stdout:/run/credentials/systemd-sysctl.service 2024-03-28T18:07:10.972 INFO:teuthology.orchestra.run.smithi022.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-03-28T18:07:10.972 INFO:teuthology.orchestra.run.smithi022.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-03-28T18:07:10.972 INFO:teuthology.orchestra.run.smithi022.stdout:/var/lib/nfs/rpc_pipefs 2024-03-28T18:07:10.972 INFO:teuthology.orchestra.run.smithi022.stdout:/run/user/1000 2024-03-28T18:07:10.973 INFO:teuthology.orchestra.run.smithi022.stdout:/var/lib/ceph 2024-03-28T18:07:10.973 INFO:teuthology.orchestra.run.smithi022.stdout:/var/lib/containers/storage/overlay 2024-03-28T18:07:10.973 INFO:teuthology.orchestra.run.smithi022.stdout:/var/lib/containers/storage/overlay/0003813f3e70bfdfafc3c0cbf156b052d465bc8dcad7f4363f448ba06e91413f/merged 2024-03-28T18:07:10.973 INFO:teuthology.orchestra.run.smithi022.stdout:/var/lib/containers/storage/overlay/0cb78753be0f6201f840a9bf93c4983cc9d5a317f94a5e5b7e64ccb76f15cf54/merged 2024-03-28T18:07:10.973 INFO:teuthology.orchestra.run.smithi022.stdout:/run/user/0 2024-03-28T18:07:10.973 INFO:teuthology.orchestra.run.smithi022.stdout:/proc/sys/fs/binfmt_misc 2024-03-28T18:07:10.973 INFO:teuthology.orchestra.run.smithi022.stdout:/var/lib/containers/storage/overlay/1d863290e69c851d1ca6d51aa2c70f306b45f930b5fc35cf549f09325b5140c2/merged 2024-03-28T18:07:10.973 INFO:teuthology.orchestra.run.smithi022.stdout:/var/lib/containers/storage/overlay/265bc374ecad4273bc50d57a5fb40e2972576732378716b3f3248f51617c3b7a/merged 2024-03-28T18:07:10.973 INFO:teuthology.orchestra.run.smithi022.stdout:/var/lib/containers/storage/overlay/b5595bf22c867e6a9731654c404aaee67c2857c5ef0edb7127e7bf846942c5d9/merged 2024-03-28T18:07:10.973 INFO:teuthology.orchestra.run.smithi022.stdout:/var/lib/containers/storage/overlay/43b5e66d6307c10cafcd8e723cc0a4582d9f3f4502ca0c67fcd575a1dbb558f6/merged 2024-03-28T18:07:10.973 INFO:teuthology.orchestra.run.smithi022.stdout:/var/lib/containers/storage/overlay/991b9aad6d7a5bf2da739ec1ba234d7e296fdd404d12f6d554869fe3d9627162/merged 2024-03-28T18:07:10.973 INFO:teuthology.orchestra.run.smithi022.stdout:/var/lib/containers/storage/overlay/3b3cb2e3b8f31cec0401cf2979e589b3c556473c10e024a53b3d84babcf9e1fd/merged 2024-03-28T18:07:10.973 INFO:teuthology.orchestra.run.smithi022.stdout:/var/lib/containers/storage/overlay/c7bbd21cb7608d09f6124cad4abd42fbb82486881112d646c9c06966df4423b4/merged 2024-03-28T18:07:10.973 INFO:teuthology.orchestra.run.smithi022.stdout:/var/lib/containers/storage/overlay/2235f3e743ce7906ae4c5671def2c41155829abfeeb7394efdfc5e34f780cf0a/merged 2024-03-28T18:07:10.973 INFO:teuthology.orchestra.run.smithi022.stdout:/run/netns 2024-03-28T18:07:10.974 INFO:teuthology.orchestra.run.smithi022.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-03-28T18:07:10.974 INFO:teuthology.orchestra.run.smithi022.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-03-28T18:07:10.974 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-28T18:07:10.975 DEBUG:teuthology.orchestra.run.smithi022:> ls /sys/fs/fuse/connections 2024-03-28T18:07:11.070 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi022.stderr:2024-03-28T18:07:11.070+0000 7f79abc41480 -1 init, newargv = 0x55a7b6ae45e0 newargc=15 2024-03-28T18:07:11.070 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi022.stderr:ceph-fuse[84970]: starting ceph client 2024-03-28T18:07:11.114 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi022.stderr:ceph-fuse[84970]: starting fuse 2024-03-28T18:07:12.165 DEBUG:teuthology.orchestra.run.smithi022:> sudo modprobe fuse 2024-03-28T18:07:12.276 DEBUG:teuthology.orchestra.run.smithi022:> cat /proc/self/mounts | awk '{print $2}' 2024-03-28T18:07:12.337 INFO:teuthology.orchestra.run.smithi022.stdout:/proc 2024-03-28T18:07:12.337 INFO:teuthology.orchestra.run.smithi022.stdout:/sys 2024-03-28T18:07:12.338 INFO:teuthology.orchestra.run.smithi022.stdout:/dev 2024-03-28T18:07:12.338 INFO:teuthology.orchestra.run.smithi022.stdout:/sys/kernel/security 2024-03-28T18:07:12.338 INFO:teuthology.orchestra.run.smithi022.stdout:/dev/shm 2024-03-28T18:07:12.338 INFO:teuthology.orchestra.run.smithi022.stdout:/dev/pts 2024-03-28T18:07:12.338 INFO:teuthology.orchestra.run.smithi022.stdout:/run 2024-03-28T18:07:12.338 INFO:teuthology.orchestra.run.smithi022.stdout:/sys/fs/cgroup 2024-03-28T18:07:12.338 INFO:teuthology.orchestra.run.smithi022.stdout:/sys/fs/pstore 2024-03-28T18:07:12.338 INFO:teuthology.orchestra.run.smithi022.stdout:/sys/fs/bpf 2024-03-28T18:07:12.338 INFO:teuthology.orchestra.run.smithi022.stdout:/ 2024-03-28T18:07:12.338 INFO:teuthology.orchestra.run.smithi022.stdout:/sys/fs/selinux 2024-03-28T18:07:12.338 INFO:teuthology.orchestra.run.smithi022.stdout:/proc/sys/fs/binfmt_misc 2024-03-28T18:07:12.338 INFO:teuthology.orchestra.run.smithi022.stdout:/dev/mqueue 2024-03-28T18:07:12.338 INFO:teuthology.orchestra.run.smithi022.stdout:/dev/hugepages 2024-03-28T18:07:12.339 INFO:teuthology.orchestra.run.smithi022.stdout:/sys/kernel/debug 2024-03-28T18:07:12.339 INFO:teuthology.orchestra.run.smithi022.stdout:/sys/kernel/tracing 2024-03-28T18:07:12.339 INFO:teuthology.orchestra.run.smithi022.stdout:/sys/fs/fuse/connections 2024-03-28T18:07:12.339 INFO:teuthology.orchestra.run.smithi022.stdout:/sys/kernel/config 2024-03-28T18:07:12.339 INFO:teuthology.orchestra.run.smithi022.stdout:/run/credentials/systemd-sysctl.service 2024-03-28T18:07:12.339 INFO:teuthology.orchestra.run.smithi022.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-03-28T18:07:12.339 INFO:teuthology.orchestra.run.smithi022.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-03-28T18:07:12.339 INFO:teuthology.orchestra.run.smithi022.stdout:/var/lib/nfs/rpc_pipefs 2024-03-28T18:07:12.339 INFO:teuthology.orchestra.run.smithi022.stdout:/run/user/1000 2024-03-28T18:07:12.339 INFO:teuthology.orchestra.run.smithi022.stdout:/var/lib/ceph 2024-03-28T18:07:12.339 INFO:teuthology.orchestra.run.smithi022.stdout:/var/lib/containers/storage/overlay 2024-03-28T18:07:12.339 INFO:teuthology.orchestra.run.smithi022.stdout:/var/lib/containers/storage/overlay/0003813f3e70bfdfafc3c0cbf156b052d465bc8dcad7f4363f448ba06e91413f/merged 2024-03-28T18:07:12.339 INFO:teuthology.orchestra.run.smithi022.stdout:/var/lib/containers/storage/overlay/0cb78753be0f6201f840a9bf93c4983cc9d5a317f94a5e5b7e64ccb76f15cf54/merged 2024-03-28T18:07:12.339 INFO:teuthology.orchestra.run.smithi022.stdout:/run/user/0 2024-03-28T18:07:12.339 INFO:teuthology.orchestra.run.smithi022.stdout:/proc/sys/fs/binfmt_misc 2024-03-28T18:07:12.340 INFO:teuthology.orchestra.run.smithi022.stdout:/var/lib/containers/storage/overlay/1d863290e69c851d1ca6d51aa2c70f306b45f930b5fc35cf549f09325b5140c2/merged 2024-03-28T18:07:12.340 INFO:teuthology.orchestra.run.smithi022.stdout:/var/lib/containers/storage/overlay/265bc374ecad4273bc50d57a5fb40e2972576732378716b3f3248f51617c3b7a/merged 2024-03-28T18:07:12.340 INFO:teuthology.orchestra.run.smithi022.stdout:/var/lib/containers/storage/overlay/b5595bf22c867e6a9731654c404aaee67c2857c5ef0edb7127e7bf846942c5d9/merged 2024-03-28T18:07:12.340 INFO:teuthology.orchestra.run.smithi022.stdout:/var/lib/containers/storage/overlay/43b5e66d6307c10cafcd8e723cc0a4582d9f3f4502ca0c67fcd575a1dbb558f6/merged 2024-03-28T18:07:12.340 INFO:teuthology.orchestra.run.smithi022.stdout:/var/lib/containers/storage/overlay/991b9aad6d7a5bf2da739ec1ba234d7e296fdd404d12f6d554869fe3d9627162/merged 2024-03-28T18:07:12.340 INFO:teuthology.orchestra.run.smithi022.stdout:/var/lib/containers/storage/overlay/3b3cb2e3b8f31cec0401cf2979e589b3c556473c10e024a53b3d84babcf9e1fd/merged 2024-03-28T18:07:12.340 INFO:teuthology.orchestra.run.smithi022.stdout:/var/lib/containers/storage/overlay/c7bbd21cb7608d09f6124cad4abd42fbb82486881112d646c9c06966df4423b4/merged 2024-03-28T18:07:12.340 INFO:teuthology.orchestra.run.smithi022.stdout:/var/lib/containers/storage/overlay/2235f3e743ce7906ae4c5671def2c41155829abfeeb7394efdfc5e34f780cf0a/merged 2024-03-28T18:07:12.340 INFO:teuthology.orchestra.run.smithi022.stdout:/run/netns 2024-03-28T18:07:12.340 INFO:teuthology.orchestra.run.smithi022.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-03-28T18:07:12.340 INFO:teuthology.orchestra.run.smithi022.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-03-28T18:07:12.340 INFO:teuthology.orchestra.run.smithi022.stdout:/home/ubuntu/cephtest/mnt.0 2024-03-28T18:07:12.341 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-28T18:07:12.341 DEBUG:teuthology.orchestra.run.smithi022:> ls /sys/fs/fuse/connections 2024-03-28T18:07:12.397 INFO:teuthology.orchestra.run.smithi022.stdout:118 2024-03-28T18:07:12.397 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [118] 2024-03-28T18:07:12.397 DEBUG:teuthology.orchestra.run.smithi022:> sudo stdin-killer -- python3 -c ' 2024-03-28T18:07:12.398 DEBUG:teuthology.orchestra.run.smithi022:> import glob 2024-03-28T18:07:12.398 DEBUG:teuthology.orchestra.run.smithi022:> import re 2024-03-28T18:07:12.398 DEBUG:teuthology.orchestra.run.smithi022:> import os 2024-03-28T18:07:12.398 DEBUG:teuthology.orchestra.run.smithi022:> import subprocess 2024-03-28T18:07:12.398 DEBUG:teuthology.orchestra.run.smithi022:> 2024-03-28T18:07:12.398 DEBUG:teuthology.orchestra.run.smithi022:> def _find_admin_socket(client_name): 2024-03-28T18:07:12.398 DEBUG:teuthology.orchestra.run.smithi022:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2024-03-28T18:07:12.398 DEBUG:teuthology.orchestra.run.smithi022:> files = glob.glob(asok_path) 2024-03-28T18:07:12.398 DEBUG:teuthology.orchestra.run.smithi022:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2024-03-28T18:07:12.398 DEBUG:teuthology.orchestra.run.smithi022:> 2024-03-28T18:07:12.398 DEBUG:teuthology.orchestra.run.smithi022:> # Given a non-glob path, it better be there 2024-03-28T18:07:12.399 DEBUG:teuthology.orchestra.run.smithi022:> if "*" not in asok_path: 2024-03-28T18:07:12.399 DEBUG:teuthology.orchestra.run.smithi022:> assert(len(files) == 1) 2024-03-28T18:07:12.399 DEBUG:teuthology.orchestra.run.smithi022:> return files[0] 2024-03-28T18:07:12.399 DEBUG:teuthology.orchestra.run.smithi022:> 2024-03-28T18:07:12.399 DEBUG:teuthology.orchestra.run.smithi022:> for f in files: 2024-03-28T18:07:12.399 DEBUG:teuthology.orchestra.run.smithi022:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-03-28T18:07:12.399 DEBUG:teuthology.orchestra.run.smithi022:> if os.path.exists("/proc/{0}".format(pid)): 2024-03-28T18:07:12.399 DEBUG:teuthology.orchestra.run.smithi022:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-03-28T18:07:12.399 DEBUG:teuthology.orchestra.run.smithi022:> contents = proc_f.read() 2024-03-28T18:07:12.399 DEBUG:teuthology.orchestra.run.smithi022:> if mountpoint in contents: 2024-03-28T18:07:12.399 DEBUG:teuthology.orchestra.run.smithi022:> return f 2024-03-28T18:07:12.399 DEBUG:teuthology.orchestra.run.smithi022:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-03-28T18:07:12.399 DEBUG:teuthology.orchestra.run.smithi022:> 2024-03-28T18:07:12.399 DEBUG:teuthology.orchestra.run.smithi022:> print(_find_admin_socket("client.0")) 2024-03-28T18:07:12.399 DEBUG:teuthology.orchestra.run.smithi022:> ' 2024-03-28T18:07:12.485 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:07:12 smithi022 ceph-mon[31985]: pgmap v442: 97 pgs: 97 active+clean; 643 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-03-28T18:07:12.589 INFO:teuthology.orchestra.run.smithi022.stdout:/var/run/ceph/ceph-client.0.84970.asok 2024-03-28T18:07:12.594 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:12 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-28T18:07:12.608 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.84970.asok 2024-03-28T18:07:12.608 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-28T18:07:12.608 DEBUG:teuthology.orchestra.run.smithi022:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.84970.asok status 2024-03-28T18:07:12.674 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:07:12 smithi134 ceph-mon[35037]: pgmap v442: 97 pgs: 97 active+clean; 643 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-03-28T18:07:12.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:07:12 smithi079 ceph-mon[35312]: pgmap v442: 97 pgs: 97 active+clean; 643 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-03-28T18:07:12.860 INFO:teuthology.orchestra.run.smithi022.stdout:{ 2024-03-28T18:07:12.860 INFO:teuthology.orchestra.run.smithi022.stdout: "metadata": { 2024-03-28T18:07:12.860 INFO:teuthology.orchestra.run.smithi022.stdout: "ceph_sha1": "3be9a5c9bc793e11e2800a8c0c696e8b46742033", 2024-03-28T18:07:12.860 INFO:teuthology.orchestra.run.smithi022.stdout: "ceph_version": "ceph version 19.0.0-2145-g3be9a5c9 (3be9a5c9bc793e11e2800a8c0c696e8b46742033) squid (dev)", 2024-03-28T18:07:12.860 INFO:teuthology.orchestra.run.smithi022.stdout: "entity_id": "0", 2024-03-28T18:07:12.860 INFO:teuthology.orchestra.run.smithi022.stdout: "hostname": "smithi022", 2024-03-28T18:07:12.860 INFO:teuthology.orchestra.run.smithi022.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2024-03-28T18:07:12.860 INFO:teuthology.orchestra.run.smithi022.stdout: "pid": "84970", 2024-03-28T18:07:12.860 INFO:teuthology.orchestra.run.smithi022.stdout: "root": "/volumes/qa/sv_0/abc125a7-3abc-4b82-bcad-de8d02f4df3d" 2024-03-28T18:07:12.861 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-28T18:07:12.861 INFO:teuthology.orchestra.run.smithi022.stdout: "dentry_count": 0, 2024-03-28T18:07:12.861 INFO:teuthology.orchestra.run.smithi022.stdout: "dentry_pinned_count": 0, 2024-03-28T18:07:12.861 INFO:teuthology.orchestra.run.smithi022.stdout: "id": 35222, 2024-03-28T18:07:12.861 INFO:teuthology.orchestra.run.smithi022.stdout: "inst": { 2024-03-28T18:07:12.861 INFO:teuthology.orchestra.run.smithi022.stdout: "name": { 2024-03-28T18:07:12.861 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "client", 2024-03-28T18:07:12.861 INFO:teuthology.orchestra.run.smithi022.stdout: "num": 35222 2024-03-28T18:07:12.861 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-28T18:07:12.861 INFO:teuthology.orchestra.run.smithi022.stdout: "addr": { 2024-03-28T18:07:12.861 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "v1", 2024-03-28T18:07:12.861 INFO:teuthology.orchestra.run.smithi022.stdout: "addr": "172.21.15.22:0", 2024-03-28T18:07:12.861 INFO:teuthology.orchestra.run.smithi022.stdout: "nonce": 3729116781 2024-03-28T18:07:12.861 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-03-28T18:07:12.861 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-28T18:07:12.862 INFO:teuthology.orchestra.run.smithi022.stdout: "addr": { 2024-03-28T18:07:12.862 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "v1", 2024-03-28T18:07:12.862 INFO:teuthology.orchestra.run.smithi022.stdout: "addr": "172.21.15.22:0", 2024-03-28T18:07:12.862 INFO:teuthology.orchestra.run.smithi022.stdout: "nonce": 3729116781 2024-03-28T18:07:12.862 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-28T18:07:12.862 INFO:teuthology.orchestra.run.smithi022.stdout: "inst_str": "client.35222 172.21.15.22:0/3729116781", 2024-03-28T18:07:12.862 INFO:teuthology.orchestra.run.smithi022.stdout: "addr_str": "172.21.15.22:0/3729116781", 2024-03-28T18:07:12.862 INFO:teuthology.orchestra.run.smithi022.stdout: "inode_count": 5, 2024-03-28T18:07:12.862 INFO:teuthology.orchestra.run.smithi022.stdout: "mds_epoch": 26, 2024-03-28T18:07:12.862 INFO:teuthology.orchestra.run.smithi022.stdout: "osd_epoch": 92, 2024-03-28T18:07:12.862 INFO:teuthology.orchestra.run.smithi022.stdout: "osd_epoch_barrier": 0, 2024-03-28T18:07:12.862 INFO:teuthology.orchestra.run.smithi022.stdout: "blocklisted": false, 2024-03-28T18:07:12.862 INFO:teuthology.orchestra.run.smithi022.stdout: "fs_name": "cephfs" 2024-03-28T18:07:12.862 INFO:teuthology.orchestra.run.smithi022.stdout:} 2024-03-28T18:07:12.876 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-28T18:07:12.876 DEBUG:teuthology.orchestra.run.smithi022:> stat --file-system '--printf=%T 2024-03-28T18:07:12.876 DEBUG:teuthology.orchestra.run.smithi022:> ' -- /home/ubuntu/cephtest/mnt.0 2024-03-28T18:07:12.928 INFO:teuthology.orchestra.run.smithi022.stdout:fuseblk 2024-03-28T18:07:12.928 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2024-03-28T18:07:12.928 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-28T18:07:12.928 DEBUG:teuthology.orchestra.run.smithi022:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-03-28T18:07:13.028 INFO:teuthology.run_tasks:Running task exec... 2024-03-28T18:07:13.039 INFO:teuthology.task.exec:Executing custom commands... 2024-03-28T18:07:13.039 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi022.front.sepia.ceph.com 2024-03-28T18:07:13.039 DEBUG:teuthology.orchestra.run.smithi022:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs set cephfs balance_automate true' 2024-03-28T18:07:13.259 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:13.259+0000 7f5e03000640 1 -- 172.21.15.22:0/1173525637 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5dfc100720 msgr2=0x7f5dfc100ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:07:13.260 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:13.259+0000 7f5e03000640 1 --2- 172.21.15.22:0/1173525637 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5dfc100720 0x7f5dfc100ba0 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f5de8009900 tx=0x7f5de802bc30 comp rx=0 tx=0).stop 2024-03-28T18:07:13.260 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:13.260+0000 7f5e03000640 1 -- 172.21.15.22:0/1173525637 shutdown_connections 2024-03-28T18:07:13.260 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:13.260+0000 7f5e03000640 1 --2- 172.21.15.22:0/1173525637 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5dfc1010e0 0x7f5dfc101960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:07:13.260 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:13.260+0000 7f5e03000640 1 --2- 172.21.15.22:0/1173525637 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5dfc100720 0x7f5dfc100ba0 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:07:13.260 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:13.260+0000 7f5e03000640 1 --2- 172.21.15.22:0/1173525637 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f5dfc0ff9c0 0x7f5dfc0ffde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:07:13.260 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:13.260+0000 7f5e03000640 1 -- 172.21.15.22:0/1173525637 >> 172.21.15.22:0/1173525637 conn(0x7f5dfc0fb2d0 msgr2=0x7f5dfc0fd730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:07:13.260 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:13.260+0000 7f5e03000640 1 -- 172.21.15.22:0/1173525637 shutdown_connections 2024-03-28T18:07:13.260 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:13.260+0000 7f5e03000640 1 -- 172.21.15.22:0/1173525637 wait complete. 2024-03-28T18:07:13.262 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:13.261+0000 7f5e03000640 1 Processor -- start 2024-03-28T18:07:13.262 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:13.261+0000 7f5e03000640 1 -- start start 2024-03-28T18:07:13.262 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:13.261+0000 7f5e03000640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5dfc0ff9c0 0x7f5dfc199b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:07:13.262 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:13.261+0000 7f5e03000640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5dfc100720 0x7f5dfc19a080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:07:13.262 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:13.261+0000 7f5e03000640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f5dfc1010e0 0x7f5dfc1a10f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:07:13.262 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:13.261+0000 7f5e03000640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5dfc10f600 con 0x7f5dfc100720 2024-03-28T18:07:13.262 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:13.261+0000 7f5e03000640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5dfc10f480 con 0x7f5dfc1010e0 2024-03-28T18:07:13.262 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:13.261+0000 7f5e03000640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5dfc10f780 con 0x7f5dfc0ff9c0 2024-03-28T18:07:13.262 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:13.261+0000 7f5dfb400640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5dfc100720 0x7f5dfc19a080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:07:13.262 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:13.262+0000 7f5dfb400640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5dfc100720 0x7f5dfc19a080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:36730/0 (socket says 172.21.15.22:36730) 2024-03-28T18:07:13.263 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:13.262+0000 7f5dfb400640 1 -- 172.21.15.22:0/3732021772 learned_addr learned my addr 172.21.15.22:0/3732021772 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-28T18:07:13.263 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:13.262+0000 7f5e01000640 1 --2- 172.21.15.22:0/3732021772 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f5dfc1010e0 0x7f5dfc1a10f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:07:13.263 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:13.262+0000 7f5dfbe00640 1 --2- 172.21.15.22:0/3732021772 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5dfc0ff9c0 0x7f5dfc199b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:07:13.263 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:13.262+0000 7f5dfb400640 1 -- 172.21.15.22:0/3732021772 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5dfc0ff9c0 msgr2=0x7f5dfc199b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:07:13.263 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:13.262+0000 7f5dfb400640 1 --2- 172.21.15.22:0/3732021772 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5dfc0ff9c0 0x7f5dfc199b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:07:13.263 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:13.262+0000 7f5dfb400640 1 -- 172.21.15.22:0/3732021772 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f5dfc1010e0 msgr2=0x7f5dfc1a10f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:07:13.263 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:13.262+0000 7f5dfb400640 1 --2- 172.21.15.22:0/3732021772 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f5dfc1010e0 0x7f5dfc1a10f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:07:13.263 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:13.262+0000 7f5dfb400640 1 -- 172.21.15.22:0/3732021772 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5dfc1a1760 con 0x7f5dfc100720 2024-03-28T18:07:13.263 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:13.262+0000 7f5dfbe00640 1 --2- 172.21.15.22:0/3732021772 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5dfc0ff9c0 0x7f5dfc199b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T18:07:13.263 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:13.262+0000 7f5dfb400640 1 --2- 172.21.15.22:0/3732021772 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5dfc100720 0x7f5dfc19a080 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f5de8000c00 tx=0x7f5de8005e40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:07:13.264 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:13.263+0000 7f5df8c00640 1 -- 172.21.15.22:0/3732021772 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5de8043020 con 0x7f5dfc100720 2024-03-28T18:07:13.264 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:13.263+0000 7f5df8c00640 1 -- 172.21.15.22:0/3732021772 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f5de8005470 con 0x7f5dfc100720 2024-03-28T18:07:13.264 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:13.263+0000 7f5df8c00640 1 -- 172.21.15.22:0/3732021772 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5de8003c90 con 0x7f5dfc100720 2024-03-28T18:07:13.264 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:13.263+0000 7f5e03000640 1 -- 172.21.15.22:0/3732021772 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5dfc1a19f0 con 0x7f5dfc100720 2024-03-28T18:07:13.264 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:13.263+0000 7f5e03000640 1 -- 172.21.15.22:0/3732021772 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5dfc1a1f90 con 0x7f5dfc100720 2024-03-28T18:07:13.264 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:13.264+0000 7f5e03000640 1 -- 172.21.15.22:0/3732021772 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5dfc10b4d0 con 0x7f5dfc100720 2024-03-28T18:07:13.265 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:13.266+0000 7f5df8c00640 1 -- 172.21.15.22:0/3732021772 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 18) v1 ==== 137290+0+0 (secure 0 0 0) 0x7f5de8046020 con 0x7f5dfc100720 2024-03-28T18:07:13.266 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:13.267+0000 7f5df8c00640 1 --2- 172.21.15.22:0/3732021772 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7f5de00a2920 0x7f5de00a4de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:07:13.267 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:13.267+0000 7f5df8c00640 1 -- 172.21.15.22:0/3732021772 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 8153+0+0 (secure 0 0 0) 0x7f5de80f0bb0 con 0x7f5dfc100720 2024-03-28T18:07:13.267 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:13.268+0000 7f5dfbe00640 1 --2- 172.21.15.22:0/3732021772 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7f5de00a2920 0x7f5de00a4de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:07:13.268 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:13.268+0000 7f5dfbe00640 1 --2- 172.21.15.22:0/3732021772 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7f5de00a2920 0x7f5de00a4de0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f5dec009b50 tx=0x7f5dec005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:07:13.271 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:13.272+0000 7f5df8c00640 1 -- 172.21.15.22:0/3732021772 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f5de804d020 con 0x7f5dfc100720 2024-03-28T18:07:13.519 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:13.519+0000 7f5e03000640 1 -- 172.21.15.22:0/3732021772 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"} v 0) v1 -- 0x7f5dfc0622a0 con 0x7f5dfc100720 2024-03-28T18:07:14.240 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:14.240+0000 7f5df8c00640 1 -- 172.21.15.22:0/3732021772 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]=0 v27) v1 ==== 117+0+0 (secure 0 0 0) 0x7f5de80bdc50 con 0x7f5dfc100720 2024-03-28T18:07:14.245 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:14.245+0000 7f5e03000640 1 -- 172.21.15.22:0/3732021772 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7f5de00a2920 msgr2=0x7f5de00a4de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:07:14.245 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:14.245+0000 7f5e03000640 1 --2- 172.21.15.22:0/3732021772 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7f5de00a2920 0x7f5de00a4de0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f5dec009b50 tx=0x7f5dec005f90 comp rx=0 tx=0).stop 2024-03-28T18:07:14.245 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:14.245+0000 7f5e03000640 1 -- 172.21.15.22:0/3732021772 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5dfc100720 msgr2=0x7f5dfc19a080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:07:14.245 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:14.245+0000 7f5e03000640 1 --2- 172.21.15.22:0/3732021772 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5dfc100720 0x7f5dfc19a080 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f5de8000c00 tx=0x7f5de8005e40 comp rx=0 tx=0).stop 2024-03-28T18:07:14.246 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:14.245+0000 7f5e03000640 1 -- 172.21.15.22:0/3732021772 shutdown_connections 2024-03-28T18:07:14.246 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:14.245+0000 7f5e03000640 1 --2- 172.21.15.22:0/3732021772 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7f5de00a2920 0x7f5de00a4de0 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:07:14.246 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:14.245+0000 7f5e03000640 1 --2- 172.21.15.22:0/3732021772 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f5dfc1010e0 0x7f5dfc1a10f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:07:14.246 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:14.245+0000 7f5e03000640 1 --2- 172.21.15.22:0/3732021772 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5dfc100720 0x7f5dfc19a080 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:07:14.246 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:14.245+0000 7f5e03000640 1 --2- 172.21.15.22:0/3732021772 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5dfc0ff9c0 0x7f5dfc199b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:07:14.246 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:14.245+0000 7f5e03000640 1 -- 172.21.15.22:0/3732021772 >> 172.21.15.22:0/3732021772 conn(0x7f5dfc0fb2d0 msgr2=0x7f5dfc0fcdd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:07:14.246 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:14.246+0000 7f5e03000640 1 -- 172.21.15.22:0/3732021772 shutdown_connections 2024-03-28T18:07:14.246 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:14.246+0000 7f5e03000640 1 -- 172.21.15.22:0/3732021772 wait complete. 2024-03-28T18:07:14.270 INFO:teuthology.run_tasks:Running task check-counter... 2024-03-28T18:07:14.281 INFO:teuthology.run_tasks:Running task fwd_scrub... 2024-03-28T18:07:14.293 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-28T18:07:14.516 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:14.516+0000 7f04f1400640 1 -- 172.21.15.22:0/4260639036 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04ec0329f0 con 0x7f04f41093b0 2024-03-28T18:07:14.516 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:14.517+0000 7f04f3200640 1 -- 172.21.15.22:0/4260639036 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f04f41093b0 msgr2=0x7f04f410b840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:07:14.517 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:14.517+0000 7f04f3200640 1 --2- 172.21.15.22:0/4260639036 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f04f41093b0 0x7f04f410b840 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f04ec009ca0 tx=0x7f04ec02c190 comp rx=0 tx=0).stop 2024-03-28T18:07:14.517 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:14.517+0000 7f04f3200640 1 -- 172.21.15.22:0/4260639036 shutdown_connections 2024-03-28T18:07:14.517 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:14.517+0000 7f04f3200640 1 --2- 172.21.15.22:0/4260639036 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f04f41093b0 0x7f04f410b840 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:07:14.518 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:14.517+0000 7f04f3200640 1 --2- 172.21.15.22:0/4260639036 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f04f4074080 0x7f04f40744e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:07:14.518 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:14.517+0000 7f04f3200640 1 --2- 172.21.15.22:0/4260639036 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f04f4075660 0x7f04f4073ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:07:14.518 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:14.517+0000 7f04f3200640 1 -- 172.21.15.22:0/4260639036 >> 172.21.15.22:0/4260639036 conn(0x7f04f40fca70 msgr2=0x7f04f40feeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:07:14.518 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:14.517+0000 7f04f3200640 1 -- 172.21.15.22:0/4260639036 shutdown_connections 2024-03-28T18:07:14.518 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:14.518+0000 7f04f3200640 1 -- 172.21.15.22:0/4260639036 wait complete. 2024-03-28T18:07:14.520 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:14.518+0000 7f04f3200640 1 Processor -- start 2024-03-28T18:07:14.520 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:14.519+0000 7f04f3200640 1 -- start start 2024-03-28T18:07:14.520 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:14.519+0000 7f04f3200640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f04f4074080 0x7f04f41048a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:07:14.520 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:14.519+0000 7f04f3200640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f04f4075660 0x7f04f4104de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:07:14.520 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:14.519+0000 7f04f3200640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f04f41093b0 0x7f04f4101410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:07:14.521 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:14.519+0000 7f04f3200640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04f410f370 con 0x7f04f4074080 2024-03-28T18:07:14.521 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:14.519+0000 7f04f3200640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04f410f1f0 con 0x7f04f4075660 2024-03-28T18:07:14.521 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:14.519+0000 7f04f3200640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04f410f4f0 con 0x7f04f41093b0 2024-03-28T18:07:14.521 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:14.519+0000 7f04f1e00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f04f4074080 0x7f04f41048a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:07:14.521 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:14.519+0000 7f04f1e00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f04f4074080 0x7f04f41048a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:36762/0 (socket says 172.21.15.22:36762) 2024-03-28T18:07:14.521 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:14.519+0000 7f04f1e00640 1 -- 172.21.15.22:0/3345843750 learned_addr learned my addr 172.21.15.22:0/3345843750 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-28T18:07:14.521 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:14.519+0000 7f04e9400640 1 --2- 172.21.15.22:0/3345843750 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f04f4075660 0x7f04f4104de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:07:14.521 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:14.520+0000 7f04f2800640 1 --2- 172.21.15.22:0/3345843750 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f04f41093b0 0x7f04f4101410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:07:14.521 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:14.520+0000 7f04f1e00640 1 -- 172.21.15.22:0/3345843750 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f04f41093b0 msgr2=0x7f04f4101410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:07:14.521 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:14.520+0000 7f04f1e00640 1 --2- 172.21.15.22:0/3345843750 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f04f41093b0 0x7f04f4101410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:07:14.521 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:14.520+0000 7f04f1e00640 1 -- 172.21.15.22:0/3345843750 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f04f4075660 msgr2=0x7f04f4104de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:07:14.521 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:14.520+0000 7f04f1e00640 1 --2- 172.21.15.22:0/3345843750 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f04f4075660 0x7f04f4104de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:07:14.521 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:14.520+0000 7f04f1e00640 1 -- 172.21.15.22:0/3345843750 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f04f4101cd0 con 0x7f04f4074080 2024-03-28T18:07:14.522 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:14.520+0000 7f04f1e00640 1 --2- 172.21.15.22:0/3345843750 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f04f4074080 0x7f04f41048a0 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f04e400d950 tx=0x7f04e400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:07:14.522 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:14.520+0000 7f04eb400640 1 -- 172.21.15.22:0/3345843750 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04e4017070 con 0x7f04f4074080 2024-03-28T18:07:14.522 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:14.520+0000 7f04eb400640 1 -- 172.21.15.22:0/3345843750 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f04e4009c30 con 0x7f04f4074080 2024-03-28T18:07:14.522 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:14.520+0000 7f04eb400640 1 -- 172.21.15.22:0/3345843750 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04e400f6b0 con 0x7f04f4074080 2024-03-28T18:07:14.522 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:14.520+0000 7f04f3200640 1 -- 172.21.15.22:0/3345843750 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f04f41a5e20 con 0x7f04f4074080 2024-03-28T18:07:14.522 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:14.521+0000 7f04f3200640 1 -- 172.21.15.22:0/3345843750 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f04f41a61d0 con 0x7f04f4074080 2024-03-28T18:07:14.522 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:14.522+0000 7f04f3200640 1 -- 172.21.15.22:0/3345843750 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f04f4108170 con 0x7f04f4074080 2024-03-28T18:07:14.523 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:14.524+0000 7f04eb400640 1 -- 172.21.15.22:0/3345843750 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 18) v1 ==== 137290+0+0 (secure 0 0 0) 0x7f04e4010a50 con 0x7f04f4074080 2024-03-28T18:07:14.526 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:14.526+0000 7f04eb400640 1 --2- 172.21.15.22:0/3345843750 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7f04d80a2880 0x7f04d80a4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:07:14.526 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:14.526+0000 7f04eb400640 1 -- 172.21.15.22:0/3345843750 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 8153+0+0 (secure 0 0 0) 0x7f04e40ce670 con 0x7f04f4074080 2024-03-28T18:07:14.526 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:14.526+0000 7f04e9400640 1 --2- 172.21.15.22:0/3345843750 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7f04d80a2880 0x7f04d80a4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:07:14.526 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:14.527+0000 7f04e9400640 1 --2- 172.21.15.22:0/3345843750 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7f04d80a2880 0x7f04d80a4d40 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f04f4102820 tx=0x7f04dc00b390 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:07:14.530 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:14.531+0000 7f04eb400640 1 -- 172.21.15.22:0/3345843750 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f04e4009de0 con 0x7f04f4074080 2024-03-28T18:07:14.531 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:07:14 smithi022 ceph-mon[31985]: pgmap v443: 97 pgs: 97 active+clean; 643 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 255 B/s wr, 10 op/s 2024-03-28T18:07:14.531 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:07:14 smithi022 ceph-mon[31985]: from='client.? 172.21.15.22:0/3732021772' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]: dispatch 2024-03-28T18:07:14.674 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:07:14 smithi134 ceph-mon[35037]: pgmap v443: 97 pgs: 97 active+clean; 643 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 255 B/s wr, 10 op/s 2024-03-28T18:07:14.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:07:14 smithi134 ceph-mon[35037]: from='client.? 172.21.15.22:0/3732021772' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]: dispatch 2024-03-28T18:07:14.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:07:14 smithi079 ceph-mon[35312]: pgmap v443: 97 pgs: 97 active+clean; 643 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 255 B/s wr, 10 op/s 2024-03-28T18:07:14.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:07:14 smithi079 ceph-mon[35312]: from='client.? 172.21.15.22:0/3732021772' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]: dispatch 2024-03-28T18:07:14.738 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:14.738+0000 7f04f3200640 1 -- 172.21.15.22:0/3345843750 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f04f40622d0 con 0x7f04f4074080 2024-03-28T18:07:14.741 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:14.741+0000 7f04eb400640 1 -- 172.21.15.22:0/3345843750 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v92) v1 ==== 74+0+22522 (secure 0 0 0) 0x7f04e409b710 con 0x7f04f4074080 2024-03-28T18:07:14.741 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-03-28T18:07:14.742 INFO:teuthology.orchestra.run.smithi022.stdout:{"epoch":92,"fsid":"fd220748-ed2b-11ee-95d0-87774f69a715","created":"2024-03-28T17:54:00.440279+0000","modified":"2024-03-28T18:05:11.725341+0000","last_up_change":"2024-03-28T18:03:43.220095+0000","last_in_change":"2024-03-28T18:03:18.526399+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-28T17:58:33.168872+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-28T18:04:47.004797+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-28T18:04:48.465502+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"90","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"5d708e06-f05b-40ba-bbcf-258ddc4f43d7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6802","nonce":2186290846},{"type":"v1","addr":"172.21.15.22:6803","nonce":2186290846}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6804","nonce":2186290846},{"type":"v1","addr":"172.21.15.22:6805","nonce":2186290846}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6808","nonce":2186290846},{"type":"v1","addr":"172.21.15.22:6809","nonce":2186290846}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6806","nonce":2186290846},{"type":"v1","addr":"172.21.15.22:6807","nonce":2186290846}]},"public_addr":"172.21.15.22:6803/2186290846","cluster_addr":"172.21.15.22:6805/2186290846","heartbeat_back_addr":"172.21.15.22:6809/2186290846","heartbeat_front_addr":"172.21.15.22:6807/2186290846","state":["exists","up"]},{"osd":1,"uuid":"68ad09bb-9256-409a-9867-4e14a2cb94aa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6800","nonce":2835549681},{"type":"v1","addr":"172.21.15.79:6801","nonce":2835549681}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6802","nonce":2835549681},{"type":"v1","addr":"172.21.15.79:6803","nonce":2835549681}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6806","nonce":2835549681},{"type":"v1","addr":"172.21.15.79:6807","nonce":2835549681}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6804","nonce":2835549681},{"type":"v1","addr":"172.21.15.79:6805","nonce":2835549681}]},"public_addr":"172.21.15.79:6801/2835549681","cluster_addr":"172.21.15.79:6803/2835549681","heartbeat_back_addr":"172.21.15.79:6807/2835549681","heartbeat_front_addr":"172.21.15.79:6805/2835549681","state":["exists","up"]},{"osd":2,"uuid":"ed11a5a7-004d-41f3-a5fc-eb2d603bd33f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6800","nonce":470588085},{"type":"v1","addr":"172.21.15.134:6801","nonce":470588085}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6802","nonce":470588085},{"type":"v1","addr":"172.21.15.134:6803","nonce":470588085}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6806","nonce":470588085},{"type":"v1","addr":"172.21.15.134:6807","nonce":470588085}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6804","nonce":470588085},{"type":"v1","addr":"172.21.15.134:6805","nonce":470588085}]},"public_addr":"172.21.15.134:6801/470588085","cluster_addr":"172.21.15.134:6803/470588085","heartbeat_back_addr":"172.21.15.134:6807/470588085","heartbeat_front_addr":"172.21.15.134:6805/470588085","state":["exists","up"]},{"osd":3,"uuid":"81a6c547-2a50-44c3-84d7-23ab6831790c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6810","nonce":1702441854},{"type":"v1","addr":"172.21.15.22:6811","nonce":1702441854}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6812","nonce":1702441854},{"type":"v1","addr":"172.21.15.22:6813","nonce":1702441854}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6816","nonce":1702441854},{"type":"v1","addr":"172.21.15.22:6817","nonce":1702441854}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6814","nonce":1702441854},{"type":"v1","addr":"172.21.15.22:6815","nonce":1702441854}]},"public_addr":"172.21.15.22:6811/1702441854","cluster_addr":"172.21.15.22:6813/1702441854","heartbeat_back_addr":"172.21.15.22:6817/1702441854","heartbeat_front_addr":"172.21.15.22:6815/1702441854","state":["exists","up"]},{"osd":4,"uuid":"d23a254e-7182-4471-8bca-7eb684ec618e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6808","nonce":630132437},{"type":"v1","addr":"172.21.15.79:6809","nonce":630132437}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6810","nonce":630132437},{"type":"v1","addr":"172.21.15.79:6811","nonce":630132437}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6814","nonce":630132437},{"type":"v1","addr":"172.21.15.79:6815","nonce":630132437}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6812","nonce":630132437},{"type":"v1","addr":"172.21.15.79:6813","nonce":630132437}]},"public_addr":"172.21.15.79:6809/630132437","cluster_addr":"172.21.15.79:6811/630132437","heartbeat_back_addr":"172.21.15.79:6815/630132437","heartbeat_front_addr":"172.21.15.79:6813/630132437","state":["exists","up"]},{"osd":5,"uuid":"a7b98502-2023-4f0a-bfa0-6840cc495e0c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":40,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6808","nonce":2135406915},{"type":"v1","addr":"172.21.15.134:6809","nonce":2135406915}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6810","nonce":2135406915},{"type":"v1","addr":"172.21.15.134:6811","nonce":2135406915}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6814","nonce":2135406915},{"type":"v1","addr":"172.21.15.134:6815","nonce":2135406915}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6812","nonce":2135406915},{"type":"v1","addr":"172.21.15.134:6813","nonce":2135406915}]},"public_addr":"172.21.15.134:6809/2135406915","cluster_addr":"172.21.15.134:6811/2135406915","heartbeat_back_addr":"172.21.15.134:6815/2135406915","heartbeat_front_addr":"172.21.15.134:6813/2135406915","state":["exists","up"]},{"osd":6,"uuid":"634c6562-d683-41c8-8242-fc848c38ce1f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6818","nonce":1400680396},{"type":"v1","addr":"172.21.15.22:6819","nonce":1400680396}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6820","nonce":1400680396},{"type":"v1","addr":"172.21.15.22:6821","nonce":1400680396}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6824","nonce":1400680396},{"type":"v1","addr":"172.21.15.22:6825","nonce":1400680396}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6822","nonce":1400680396},{"type":"v1","addr":"172.21.15.22:6823","nonce":1400680396}]},"public_addr":"172.21.15.22:6819/1400680396","cluster_addr":"172.21.15.22:6821/1400680396","heartbeat_back_addr":"172.21.15.22:6825/1400680396","heartbeat_front_addr":"172.21.15.22:6823/1400680396","state":["exists","up"]},{"osd":7,"uuid":"504eb4a8-6731-45ad-932c-1b38ef157871","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":53,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6816","nonce":1287105314},{"type":"v1","addr":"172.21.15.79:6817","nonce":1287105314}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6818","nonce":1287105314},{"type":"v1","addr":"172.21.15.79:6819","nonce":1287105314}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6822","nonce":1287105314},{"type":"v1","addr":"172.21.15.79:6823","nonce":1287105314}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6820","nonce":1287105314},{"type":"v1","addr":"172.21.15.79:6821","nonce":1287105314}]},"public_addr":"172.21.15.79:6817/1287105314","cluster_addr":"172.21.15.79:6819/1287105314","heartbeat_back_addr":"172.21.15.79:6823/1287105314","heartbeat_front_addr":"172.21.15.79:6821/1287105314","state":["exists","up"]},{"osd":8,"uuid":"20aa66c1-03d6-4cb8-a096-0eae41979c17","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":62,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6816","nonce":2265650388},{"type":"v1","addr":"172.21.15.134:6817","nonce":2265650388}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6818","nonce":2265650388},{"type":"v1","addr":"172.21.15.134:6819","nonce":2265650388}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6822","nonce":2265650388},{"type":"v1","addr":"172.21.15.134:6823","nonce":2265650388}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6820","nonce":2265650388},{"type":"v1","addr":"172.21.15.134:6821","nonce":2265650388}]},"public_addr":"172.21.15.134:6817/2265650388","cluster_addr":"172.21.15.134:6819/2265650388","heartbeat_back_addr":"172.21.15.134:6823/2265650388","heartbeat_front_addr":"172.21.15.134:6821/2265650388","state":["exists","up"]},{"osd":9,"uuid":"e1ae4020-7c2c-4334-a753-ee6b12d98584","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":70,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6826","nonce":3762222259},{"type":"v1","addr":"172.21.15.22:6827","nonce":3762222259}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6828","nonce":3762222259},{"type":"v1","addr":"172.21.15.22:6829","nonce":3762222259}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6832","nonce":3762222259},{"type":"v1","addr":"172.21.15.22:6833","nonce":3762222259}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6830","nonce":3762222259},{"type":"v1","addr":"172.21.15.22:6831","nonce":3762222259}]},"public_addr":"172.21.15.22:6827/3762222259","cluster_addr":"172.21.15.22:6829/3762222259","heartbeat_back_addr":"172.21.15.22:6833/3762222259","heartbeat_front_addr":"172.21.15.22:6831/3762222259","state":["exists","up"]},{"osd":10,"uuid":"f5198d52-df48-4679-a860-4dcb440b7a9f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":77,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6824","nonce":1746167904},{"type":"v1","addr":"172.21.15.79:6825","nonce":1746167904}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6826","nonce":1746167904},{"type":"v1","addr":"172.21.15.79:6827","nonce":1746167904}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6830","nonce":1746167904},{"type":"v1","addr":"172.21.15.79:6831","nonce":1746167904}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6828","nonce":1746167904},{"type":"v1","addr":"172.21.15.79:6829","nonce":1746167904}]},"public_addr":"172.21.15.79:6825/1746167904","cluster_addr":"172.21.15.79:6827/1746167904","heartbeat_back_addr":"172.21.15.79:6831/1746167904","heartbeat_front_addr":"172.21.15.79:6829/1746167904","state":["exists","up"]},{"osd":11,"uuid":"1427bca7-3414-4468-9246-e999156bc2ae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":85,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6824","nonce":4271716428},{"type":"v1","addr":"172.21.15.134:6825","nonce":4271716428}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6826","nonce":4271716428},{"type":"v1","addr":"172.21.15.134:6827","nonce":4271716428}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6830","nonce":4271716428},{"type":"v1","addr":"172.21.15.134:6831","nonce":4271716428}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6828","nonce":4271716428},{"type":"v1","addr":"172.21.15.134:6829","nonce":4271716428}]},"public_addr":"172.21.15.134:6825/4271716428","cluster_addr":"172.21.15.134:6827/4271716428","heartbeat_back_addr":"172.21.15.134:6831/4271716428","heartbeat_front_addr":"172.21.15.134:6829/4271716428","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T17:57:21.225125+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T17:57:55.197254+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T17:58:27.825136+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T17:59:04.319893+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T17:59:38.407752+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T18:00:12.118805+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T18:00:47.635214+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T18:01:21.231055+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T18:01:55.817861+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T18:02:31.680160+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T18:03:06.783398+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T18:03:41.307347+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.8","mappings":[{"from":2,"to":11}]},{"pgid":"2.1d","mappings":[{"from":8,"to":11}]},{"pgid":"3.3","mappings":[{"from":4,"to":9}]},{"pgid":"3.c","mappings":[{"from":3,"to":9}]},{"pgid":"3.21","mappings":[{"from":4,"to":8}]},{"pgid":"3.25","mappings":[{"from":5,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.22:0/3521924283":"2024-03-29T17:55:10.425504+0000","172.21.15.22:6801/730635939":"2024-03-29T17:55:10.425504+0000","172.21.15.22:0/2471229459":"2024-03-29T17:54:34.004559+0000","172.21.15.22:6800/3166828081":"2024-03-29T17:54:34.004559+0000","172.21.15.22:6800/730635939":"2024-03-29T17:55:10.425504+0000","172.21.15.22:0/1229047849":"2024-03-29T17:54:34.004559+0000","172.21.15.22:6801/3166828081":"2024-03-29T17:54:34.004559+0000","172.21.15.22:0/2717818856":"2024-03-29T17:55:10.425504+0000","172.21.15.22:0/2157570333":"2024-03-29T17:54:34.004559+0000","172.21.15.22:0/1006589530":"2024-03-29T17:55:10.425504+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-28T18:07:14.744 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:14.744+0000 7f04f3200640 1 -- 172.21.15.22:0/3345843750 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7f04d80a2880 msgr2=0x7f04d80a4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:07:14.744 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:14.744+0000 7f04f3200640 1 --2- 172.21.15.22:0/3345843750 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7f04d80a2880 0x7f04d80a4d40 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f04f4102820 tx=0x7f04dc00b390 comp rx=0 tx=0).stop 2024-03-28T18:07:14.744 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:14.744+0000 7f04f3200640 1 -- 172.21.15.22:0/3345843750 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f04f4074080 msgr2=0x7f04f41048a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:07:14.744 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:14.744+0000 7f04f3200640 1 --2- 172.21.15.22:0/3345843750 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f04f4074080 0x7f04f41048a0 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f04e400d950 tx=0x7f04e400de10 comp rx=0 tx=0).stop 2024-03-28T18:07:14.744 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:14.744+0000 7f04f3200640 1 -- 172.21.15.22:0/3345843750 shutdown_connections 2024-03-28T18:07:14.744 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:14.744+0000 7f04f3200640 1 --2- 172.21.15.22:0/3345843750 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7f04d80a2880 0x7f04d80a4d40 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:07:14.744 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:14.744+0000 7f04f3200640 1 --2- 172.21.15.22:0/3345843750 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f04f41093b0 0x7f04f4101410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:07:14.744 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:14.744+0000 7f04f3200640 1 --2- 172.21.15.22:0/3345843750 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f04f4075660 0x7f04f4104de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:07:14.745 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:14.744+0000 7f04f3200640 1 --2- 172.21.15.22:0/3345843750 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f04f4074080 0x7f04f41048a0 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:07:14.745 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:14.744+0000 7f04f3200640 1 -- 172.21.15.22:0/3345843750 >> 172.21.15.22:0/3345843750 conn(0x7f04f40fca70 msgr2=0x7f04f4105340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:07:14.745 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:14.744+0000 7f04f3200640 1 -- 172.21.15.22:0/3345843750 shutdown_connections 2024-03-28T18:07:14.745 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:14.744+0000 7f04f3200640 1 -- 172.21.15.22:0/3345843750 wait complete. 2024-03-28T18:07:14.764 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-28T17:58:33.168872+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-28T18:04:47.004797+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '89', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-03-28T18:04:48.465502+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '90', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-28T18:07:14.765 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-28T18:07:14.991 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:14.990+0000 7f92f6600640 1 -- 172.21.15.22:0/349510728 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f92f0075680 msgr2=0x7f92f0073ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:07:14.991 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:14.990+0000 7f92f6600640 1 --2- 172.21.15.22:0/349510728 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f92f0075680 0x7f92f0073ad0 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f92e4009900 tx=0x7f92e402bcd0 comp rx=0 tx=0).stop 2024-03-28T18:07:14.991 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:14.991+0000 7f92f6600640 1 -- 172.21.15.22:0/349510728 shutdown_connections 2024-03-28T18:07:14.991 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:14.991+0000 7f92f6600640 1 --2- 172.21.15.22:0/349510728 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f92f0109410 0x7f92f010b8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:07:14.991 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:14.991+0000 7f92f6600640 1 --2- 172.21.15.22:0/349510728 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f92f00740a0 0x7f92f0074500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:07:14.991 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:14.991+0000 7f92f6600640 1 --2- 172.21.15.22:0/349510728 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f92f0075680 0x7f92f0073ad0 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:07:14.991 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:14.991+0000 7f92f6600640 1 -- 172.21.15.22:0/349510728 >> 172.21.15.22:0/349510728 conn(0x7f92f00fcb10 msgr2=0x7f92f00fef30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:07:14.991 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:14.991+0000 7f92f6600640 1 -- 172.21.15.22:0/349510728 shutdown_connections 2024-03-28T18:07:14.991 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:14.991+0000 7f92f6600640 1 -- 172.21.15.22:0/349510728 wait complete. 2024-03-28T18:07:14.994 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:14.992+0000 7f92f6600640 1 Processor -- start 2024-03-28T18:07:14.994 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:14.992+0000 7f92f6600640 1 -- start start 2024-03-28T18:07:14.994 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:14.992+0000 7f92f6600640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f92f00740a0 0x7f92f0199a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:07:14.994 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:14.992+0000 7f92f6600640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f92f0075680 0x7f92f0199fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:07:14.994 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:14.992+0000 7f92f6600640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f92f0109410 0x7f92f01a1010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:07:14.994 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:14.992+0000 7f92f6600640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92f010f430 con 0x7f92f0075680 2024-03-28T18:07:14.994 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:14.992+0000 7f92f6600640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92f010f2b0 con 0x7f92f00740a0 2024-03-28T18:07:14.994 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:14.992+0000 7f92f6600640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92f010f5b0 con 0x7f92f0109410 2024-03-28T18:07:14.994 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:14.992+0000 7f92efe00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f92f0075680 0x7f92f0199fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:07:14.994 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:14.992+0000 7f92efe00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f92f0075680 0x7f92f0199fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:36780/0 (socket says 172.21.15.22:36780) 2024-03-28T18:07:14.994 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:14.992+0000 7f92efe00640 1 -- 172.21.15.22:0/1008692103 learned_addr learned my addr 172.21.15.22:0/1008692103 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-28T18:07:14.994 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:14.993+0000 7f92f5200640 1 --2- 172.21.15.22:0/1008692103 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f92f00740a0 0x7f92f0199a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:07:14.994 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:14.993+0000 7f92f5c00640 1 --2- 172.21.15.22:0/1008692103 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f92f0109410 0x7f92f01a1010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:07:14.995 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:14.993+0000 7f92efe00640 1 -- 172.21.15.22:0/1008692103 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f92f0109410 msgr2=0x7f92f01a1010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:07:14.995 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:14.993+0000 7f92efe00640 1 --2- 172.21.15.22:0/1008692103 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f92f0109410 0x7f92f01a1010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:07:14.995 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:14.993+0000 7f92efe00640 1 -- 172.21.15.22:0/1008692103 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f92f00740a0 msgr2=0x7f92f0199a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:07:14.995 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:14.993+0000 7f92efe00640 1 --2- 172.21.15.22:0/1008692103 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f92f00740a0 0x7f92f0199a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:07:14.995 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:14.993+0000 7f92efe00640 1 -- 172.21.15.22:0/1008692103 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f92f01a1680 con 0x7f92f0075680 2024-03-28T18:07:14.995 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:14.993+0000 7f92f5c00640 1 --2- 172.21.15.22:0/1008692103 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f92f0109410 0x7f92f01a1010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T18:07:14.995 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:14.993+0000 7f92efe00640 1 --2- 172.21.15.22:0/1008692103 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f92f0075680 0x7f92f0199fa0 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f92dc00e9f0 tx=0x7f92dc00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:07:14.995 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:14.993+0000 7f92ed600640 1 -- 172.21.15.22:0/1008692103 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92dc00cd80 con 0x7f92f0075680 2024-03-28T18:07:14.995 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:14.994+0000 7f92f6600640 1 -- 172.21.15.22:0/1008692103 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f92f01a1970 con 0x7f92f0075680 2024-03-28T18:07:14.995 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:14.994+0000 7f92ed600640 1 -- 172.21.15.22:0/1008692103 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f92dc010900 con 0x7f92f0075680 2024-03-28T18:07:14.995 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:14.994+0000 7f92ed600640 1 -- 172.21.15.22:0/1008692103 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92dc0189e0 con 0x7f92f0075680 2024-03-28T18:07:14.995 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:14.994+0000 7f92f6600640 1 -- 172.21.15.22:0/1008692103 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f92f01a1f10 con 0x7f92f0075680 2024-03-28T18:07:14.997 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:14.995+0000 7f92f6600640 1 -- 172.21.15.22:0/1008692103 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f92f01081d0 con 0x7f92f0075680 2024-03-28T18:07:14.997 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:14.998+0000 7f92ed600640 1 -- 172.21.15.22:0/1008692103 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 18) v1 ==== 137290+0+0 (secure 0 0 0) 0x7f92dc010ab0 con 0x7f92f0075680 2024-03-28T18:07:14.999 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:14.999+0000 7f92ed600640 1 --2- 172.21.15.22:0/1008692103 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7f92d80a2880 0x7f92d80a4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:07:14.999 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:14.999+0000 7f92ed600640 1 -- 172.21.15.22:0/1008692103 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 8153+0+0 (secure 0 0 0) 0x7f92dc0ced00 con 0x7f92f0075680 2024-03-28T18:07:14.999 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:14.999+0000 7f92f5200640 1 --2- 172.21.15.22:0/1008692103 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7f92d80a2880 0x7f92d80a4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:07:15.000 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:15.000+0000 7f92f5200640 1 --2- 172.21.15.22:0/1008692103 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7f92d80a2880 0x7f92d80a4d40 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f92e4000c00 tx=0x7f92e402c8c0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:07:15.003 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:15.003+0000 7f92ed600640 1 -- 172.21.15.22:0/1008692103 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f92dc09be40 con 0x7f92f0075680 2024-03-28T18:07:15.208 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:15.208+0000 7f92f6600640 1 -- 172.21.15.22:0/1008692103 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f92f002d2c0 con 0x7f92f0075680 2024-03-28T18:07:15.209 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:15.209+0000 7f92ed600640 1 -- 172.21.15.22:0/1008692103 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v92) v1 ==== 93+0+10 (secure 0 0 0) 0x7f92dc09b740 con 0x7f92f0075680 2024-03-28T18:07:15.209 INFO:teuthology.orchestra.run.smithi022.stdout:pg_num: 1 2024-03-28T18:07:15.211 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:15.211+0000 7f92f6600640 1 -- 172.21.15.22:0/1008692103 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7f92d80a2880 msgr2=0x7f92d80a4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:07:15.211 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:15.211+0000 7f92f6600640 1 --2- 172.21.15.22:0/1008692103 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7f92d80a2880 0x7f92d80a4d40 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f92e4000c00 tx=0x7f92e402c8c0 comp rx=0 tx=0).stop 2024-03-28T18:07:15.212 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:15.211+0000 7f92f6600640 1 -- 172.21.15.22:0/1008692103 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f92f0075680 msgr2=0x7f92f0199fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:07:15.212 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:15.212+0000 7f92f6600640 1 --2- 172.21.15.22:0/1008692103 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f92f0075680 0x7f92f0199fa0 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f92dc00e9f0 tx=0x7f92dc00eeb0 comp rx=0 tx=0).stop 2024-03-28T18:07:15.212 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:15.212+0000 7f92f6600640 1 -- 172.21.15.22:0/1008692103 shutdown_connections 2024-03-28T18:07:15.212 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:15.212+0000 7f92f6600640 1 --2- 172.21.15.22:0/1008692103 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7f92d80a2880 0x7f92d80a4d40 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:07:15.212 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:15.212+0000 7f92f6600640 1 --2- 172.21.15.22:0/1008692103 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f92f0109410 0x7f92f01a1010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:07:15.212 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:15.212+0000 7f92f6600640 1 --2- 172.21.15.22:0/1008692103 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f92f0075680 0x7f92f0199fa0 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:07:15.212 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:15.212+0000 7f92f6600640 1 --2- 172.21.15.22:0/1008692103 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f92f00740a0 0x7f92f0199a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:07:15.212 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:15.212+0000 7f92f6600640 1 -- 172.21.15.22:0/1008692103 >> 172.21.15.22:0/1008692103 conn(0x7f92f00fcb10 msgr2=0x7f92f01053a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:07:15.212 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:15.212+0000 7f92f6600640 1 -- 172.21.15.22:0/1008692103 shutdown_connections 2024-03-28T18:07:15.212 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:15.212+0000 7f92f6600640 1 -- 172.21.15.22:0/1008692103 wait complete. 2024-03-28T18:07:15.231 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-28T18:07:15.457 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:15.457+0000 7fc211400640 1 -- 172.21.15.22:0/1347918974 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc20c105370 msgr2=0x7fc20c10bc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:07:15.457 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:15.457+0000 7fc211400640 1 --2- 172.21.15.22:0/1347918974 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc20c105370 0x7fc20c10bc00 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7fc204009900 tx=0x7fc20402bc40 comp rx=0 tx=0).stop 2024-03-28T18:07:15.457 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:15.457+0000 7fc211400640 1 -- 172.21.15.22:0/1347918974 shutdown_connections 2024-03-28T18:07:15.457 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:15.457+0000 7fc211400640 1 --2- 172.21.15.22:0/1347918974 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc20c105370 0x7fc20c10bc00 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:07:15.457 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:15.457+0000 7fc211400640 1 --2- 172.21.15.22:0/1347918974 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc20c1049b0 0x7fc20c104e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:07:15.457 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:15.457+0000 7fc211400640 1 --2- 172.21.15.22:0/1347918974 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc20c1037c0 0x7fc20c103be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:07:15.457 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:15.457+0000 7fc211400640 1 -- 172.21.15.22:0/1347918974 >> 172.21.15.22:0/1347918974 conn(0x7fc20c0fed50 msgr2=0x7fc20c101170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:07:15.457 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:15.457+0000 7fc211400640 1 -- 172.21.15.22:0/1347918974 shutdown_connections 2024-03-28T18:07:15.458 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:15.457+0000 7fc211400640 1 -- 172.21.15.22:0/1347918974 wait complete. 2024-03-28T18:07:15.460 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:15.458+0000 7fc211400640 1 Processor -- start 2024-03-28T18:07:15.460 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:15.458+0000 7fc211400640 1 -- start start 2024-03-28T18:07:15.460 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:15.458+0000 7fc211400640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc20c1037c0 0x7fc20c1999d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:07:15.460 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:15.458+0000 7fc211400640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc20c1049b0 0x7fc20c199f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:07:15.460 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:15.458+0000 7fc211400640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc20c105370 0x7fc20c1a0f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:07:15.460 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:15.458+0000 7fc211400640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc20c10f470 con 0x7fc20c105370 2024-03-28T18:07:15.460 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:15.458+0000 7fc211400640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc20c10f2f0 con 0x7fc20c1037c0 2024-03-28T18:07:15.460 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:15.458+0000 7fc211400640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc20c10f5f0 con 0x7fc20c1049b0 2024-03-28T18:07:15.460 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:15.459+0000 7fc210a00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc20c105370 0x7fc20c1a0f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:07:15.460 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:15.459+0000 7fc20be00640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc20c1037c0 0x7fc20c1999d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:07:15.460 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:15.459+0000 7fc210a00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc20c105370 0x7fc20c1a0f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:36790/0 (socket says 172.21.15.22:36790) 2024-03-28T18:07:15.460 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:15.459+0000 7fc210a00640 1 -- 172.21.15.22:0/2380349652 learned_addr learned my addr 172.21.15.22:0/2380349652 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-28T18:07:15.461 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:15.459+0000 7fc20b400640 1 --2- 172.21.15.22:0/2380349652 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc20c1049b0 0x7fc20c199f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:07:15.461 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:15.459+0000 7fc210a00640 1 -- 172.21.15.22:0/2380349652 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc20c1049b0 msgr2=0x7fc20c199f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:07:15.461 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:15.459+0000 7fc210a00640 1 --2- 172.21.15.22:0/2380349652 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc20c1049b0 0x7fc20c199f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:07:15.461 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:15.459+0000 7fc210a00640 1 -- 172.21.15.22:0/2380349652 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc20c1037c0 msgr2=0x7fc20c1999d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:07:15.461 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:15.459+0000 7fc210a00640 1 --2- 172.21.15.22:0/2380349652 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc20c1037c0 0x7fc20c1999d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:07:15.461 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:15.459+0000 7fc210a00640 1 -- 172.21.15.22:0/2380349652 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc20c1a15f0 con 0x7fc20c105370 2024-03-28T18:07:15.461 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:15.459+0000 7fc20b400640 1 --2- 172.21.15.22:0/2380349652 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc20c1049b0 0x7fc20c199f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T18:07:15.461 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:15.459+0000 7fc20be00640 1 --2- 172.21.15.22:0/2380349652 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc20c1037c0 0x7fc20c1999d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-28T18:07:15.461 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:15.459+0000 7fc210a00640 1 --2- 172.21.15.22:0/2380349652 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc20c105370 0x7fc20c1a0f80 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7fc204036040 tx=0x7fc204005de0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:07:15.461 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:15.460+0000 7fc208c00640 1 -- 172.21.15.22:0/2380349652 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc204043020 con 0x7fc20c105370 2024-03-28T18:07:15.461 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:15.460+0000 7fc208c00640 1 -- 172.21.15.22:0/2380349652 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7fc2040053f0 con 0x7fc20c105370 2024-03-28T18:07:15.461 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:15.460+0000 7fc208c00640 1 -- 172.21.15.22:0/2380349652 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc204003bb0 con 0x7fc20c105370 2024-03-28T18:07:15.461 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:15.460+0000 7fc211400640 1 -- 172.21.15.22:0/2380349652 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc20c1a1880 con 0x7fc20c105370 2024-03-28T18:07:15.461 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:15.460+0000 7fc211400640 1 -- 172.21.15.22:0/2380349652 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc20c1a1d80 con 0x7fc20c105370 2024-03-28T18:07:15.462 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:15.461+0000 7fc211400640 1 -- 172.21.15.22:0/2380349652 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc20c10aa10 con 0x7fc20c105370 2024-03-28T18:07:15.463 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:15.463+0000 7fc208c00640 1 -- 172.21.15.22:0/2380349652 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 18) v1 ==== 137290+0+0 (secure 0 0 0) 0x7fc204046020 con 0x7fc20c105370 2024-03-28T18:07:15.464 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:15.464+0000 7fc208c00640 1 --2- 172.21.15.22:0/2380349652 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7fc1f00a2880 0x7fc1f00a4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:07:15.464 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:15.465+0000 7fc208c00640 1 -- 172.21.15.22:0/2380349652 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 8153+0+0 (secure 0 0 0) 0x7fc2040f0b10 con 0x7fc20c105370 2024-03-28T18:07:15.464 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:15.465+0000 7fc20be00640 1 --2- 172.21.15.22:0/2380349652 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7fc1f00a2880 0x7fc1f00a4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:07:15.465 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:15.466+0000 7fc20be00640 1 --2- 172.21.15.22:0/2380349652 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7fc1f00a2880 0x7fc1f00a4d40 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fc20c104810 tx=0x7fc1fc007600 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:07:15.468 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:15.469+0000 7fc208c00640 1 -- 172.21.15.22:0/2380349652 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fc20404d020 con 0x7fc20c105370 2024-03-28T18:07:15.531 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:07:15 smithi022 ceph-mon[31985]: from='client.? 172.21.15.22:0/3732021772' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]': finished 2024-03-28T18:07:15.531 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:07:15 smithi022 ceph-mon[31985]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-28T18:07:15.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:07:15 smithi022 ceph-mon[31985]: from='client.? 172.21.15.22:0/3345843750' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-28T18:07:15.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:07:15 smithi022 ceph-mon[31985]: from='client.? 172.21.15.22:0/1008692103' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-28T18:07:15.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:07:15 smithi134 ceph-mon[35037]: from='client.? 172.21.15.22:0/3732021772' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]': finished 2024-03-28T18:07:15.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:07:15 smithi134 ceph-mon[35037]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-28T18:07:15.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:07:15 smithi134 ceph-mon[35037]: from='client.? 172.21.15.22:0/3345843750' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-28T18:07:15.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:07:15 smithi134 ceph-mon[35037]: from='client.? 172.21.15.22:0/1008692103' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-28T18:07:15.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:07:15 smithi079 ceph-mon[35312]: from='client.? 172.21.15.22:0/3732021772' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]': finished 2024-03-28T18:07:15.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:07:15 smithi079 ceph-mon[35312]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-28T18:07:15.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:07:15 smithi079 ceph-mon[35312]: from='client.? 172.21.15.22:0/3345843750' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-28T18:07:15.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:07:15 smithi079 ceph-mon[35312]: from='client.? 172.21.15.22:0/1008692103' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-28T18:07:15.683 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:15.684+0000 7fc211400640 1 -- 172.21.15.22:0/2380349652 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fc20c0622a0 con 0x7fc20c105370 2024-03-28T18:07:15.684 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:15.685+0000 7fc208c00640 1 -- 172.21.15.22:0/2380349652 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v92) v1 ==== 104+0+11 (secure 0 0 0) 0x7fc2040055a0 con 0x7fc20c105370 2024-03-28T18:07:15.685 INFO:teuthology.orchestra.run.smithi022.stdout:pg_num: 32 2024-03-28T18:07:15.688 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:15.687+0000 7fc211400640 1 -- 172.21.15.22:0/2380349652 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7fc1f00a2880 msgr2=0x7fc1f00a4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:07:15.688 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:15.687+0000 7fc211400640 1 --2- 172.21.15.22:0/2380349652 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7fc1f00a2880 0x7fc1f00a4d40 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fc20c104810 tx=0x7fc1fc007600 comp rx=0 tx=0).stop 2024-03-28T18:07:15.688 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:15.688+0000 7fc211400640 1 -- 172.21.15.22:0/2380349652 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc20c105370 msgr2=0x7fc20c1a0f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:07:15.688 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:15.688+0000 7fc211400640 1 --2- 172.21.15.22:0/2380349652 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc20c105370 0x7fc20c1a0f80 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7fc204036040 tx=0x7fc204005de0 comp rx=0 tx=0).stop 2024-03-28T18:07:15.688 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:15.688+0000 7fc211400640 1 -- 172.21.15.22:0/2380349652 shutdown_connections 2024-03-28T18:07:15.688 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:15.688+0000 7fc211400640 1 --2- 172.21.15.22:0/2380349652 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7fc1f00a2880 0x7fc1f00a4d40 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:07:15.688 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:15.688+0000 7fc211400640 1 --2- 172.21.15.22:0/2380349652 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc20c105370 0x7fc20c1a0f80 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:07:15.688 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:15.688+0000 7fc211400640 1 --2- 172.21.15.22:0/2380349652 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc20c1049b0 0x7fc20c199f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:07:15.688 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:15.688+0000 7fc211400640 1 --2- 172.21.15.22:0/2380349652 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc20c1037c0 0x7fc20c1999d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:07:15.688 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:15.688+0000 7fc211400640 1 -- 172.21.15.22:0/2380349652 >> 172.21.15.22:0/2380349652 conn(0x7fc20c0fed50 msgr2=0x7fc20c107be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:07:15.688 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:15.688+0000 7fc211400640 1 -- 172.21.15.22:0/2380349652 shutdown_connections 2024-03-28T18:07:15.688 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:15.688+0000 7fc211400640 1 -- 172.21.15.22:0/2380349652 wait complete. 2024-03-28T18:07:15.707 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-28T18:07:15.926 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:15.926+0000 7f1352400640 1 -- 172.21.15.22:0/2681300160 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f134c1028a0 msgr2=0x7f134c102d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:07:15.926 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:15.926+0000 7f1352400640 1 --2- 172.21.15.22:0/2681300160 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f134c1028a0 0x7f134c102d20 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f1338009900 tx=0x7f133802bc30 comp rx=0 tx=0).stop 2024-03-28T18:07:15.926 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:15.926+0000 7f1352400640 1 -- 172.21.15.22:0/2681300160 shutdown_connections 2024-03-28T18:07:15.926 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:15.926+0000 7f1352400640 1 --2- 172.21.15.22:0/2681300160 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f134c103260 0x7f134c109af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:07:15.926 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:15.926+0000 7f1352400640 1 --2- 172.21.15.22:0/2681300160 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f134c1028a0 0x7f134c102d20 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:07:15.926 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:15.926+0000 7f1352400640 1 --2- 172.21.15.22:0/2681300160 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f134c1016b0 0x7f134c101ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:07:15.926 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:15.926+0000 7f1352400640 1 -- 172.21.15.22:0/2681300160 >> 172.21.15.22:0/2681300160 conn(0x7f134c0fcc40 msgr2=0x7f134c0ff060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:07:15.927 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:15.926+0000 7f1352400640 1 -- 172.21.15.22:0/2681300160 shutdown_connections 2024-03-28T18:07:15.927 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:15.926+0000 7f1352400640 1 -- 172.21.15.22:0/2681300160 wait complete. 2024-03-28T18:07:15.928 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:15.927+0000 7f1352400640 1 Processor -- start 2024-03-28T18:07:15.928 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:15.927+0000 7f1352400640 1 -- start start 2024-03-28T18:07:15.928 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:15.927+0000 7f1352400640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f134c1016b0 0x7f134c1979e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:07:15.928 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:15.928+0000 7f1352400640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f134c1028a0 0x7f134c197f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:07:15.928 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:15.928+0000 7f1352400640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f134c103260 0x7f134c19ef90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:07:15.928 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:15.928+0000 7f1352400640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f134c10d310 con 0x7f134c103260 2024-03-28T18:07:15.928 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:15.928+0000 7f1352400640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f134c10d190 con 0x7f134c1016b0 2024-03-28T18:07:15.928 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:15.928+0000 7f1352400640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f134c10d490 con 0x7f134c1028a0 2024-03-28T18:07:15.928 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:15.928+0000 7f134b400640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f134c1016b0 0x7f134c1979e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:07:15.928 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:15.928+0000 7f134aa00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f134c1028a0 0x7f134c197f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:07:15.929 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:15.928+0000 7f134b400640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f134c1016b0 0x7f134c1979e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.22:44266/0 (socket says 172.21.15.22:44266) 2024-03-28T18:07:15.929 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:15.928+0000 7f134b400640 1 -- 172.21.15.22:0/3410923332 learned_addr learned my addr 172.21.15.22:0/3410923332 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-28T18:07:15.929 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:15.928+0000 7f134be00640 1 --2- 172.21.15.22:0/3410923332 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f134c103260 0x7f134c19ef90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:07:15.929 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:15.928+0000 7f134b400640 1 -- 172.21.15.22:0/3410923332 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f134c1028a0 msgr2=0x7f134c197f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:07:15.929 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:15.928+0000 7f134b400640 1 --2- 172.21.15.22:0/3410923332 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f134c1028a0 0x7f134c197f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:07:15.929 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:15.928+0000 7f134b400640 1 -- 172.21.15.22:0/3410923332 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f134c103260 msgr2=0x7f134c19ef90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:07:15.929 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:15.928+0000 7f134b400640 1 --2- 172.21.15.22:0/3410923332 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f134c103260 0x7f134c19ef90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:07:15.929 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:15.928+0000 7f134b400640 1 -- 172.21.15.22:0/3410923332 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f134c19f600 con 0x7f134c1016b0 2024-03-28T18:07:15.930 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:15.929+0000 7f134b400640 1 --2- 172.21.15.22:0/3410923332 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f134c1016b0 0x7f134c1979e0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f133c00e9c0 tx=0x7f133c00ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:07:15.930 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:15.929+0000 7f1343e00640 1 -- 172.21.15.22:0/3410923332 <== mon.2 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f133c00cd80 con 0x7f134c1016b0 2024-03-28T18:07:15.930 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:15.929+0000 7f1343e00640 1 -- 172.21.15.22:0/3410923332 <== mon.2 v2:172.21.15.79:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f133c0035f0 con 0x7f134c1016b0 2024-03-28T18:07:15.930 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:15.929+0000 7f1343e00640 1 -- 172.21.15.22:0/3410923332 <== mon.2 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f133c010690 con 0x7f134c1016b0 2024-03-28T18:07:15.930 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:15.929+0000 7f1352400640 1 -- 172.21.15.22:0/3410923332 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f134c19f8f0 con 0x7f134c1016b0 2024-03-28T18:07:15.930 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:15.929+0000 7f1352400640 1 -- 172.21.15.22:0/3410923332 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f134c19fe10 con 0x7f134c1016b0 2024-03-28T18:07:15.930 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:15.931+0000 7f1352400640 1 -- 172.21.15.22:0/3410923332 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f134c108a50 con 0x7f134c1016b0 2024-03-28T18:07:15.932 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:15.932+0000 7f1343e00640 1 -- 172.21.15.22:0/3410923332 <== mon.2 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 18) v1 ==== 137290+0+0 (secure 0 0 0) 0x7f133c0037a0 con 0x7f134c1016b0 2024-03-28T18:07:15.934 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:15.934+0000 7f1343e00640 1 --2- 172.21.15.22:0/3410923332 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7f13300a2880 0x7f13300a4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:07:15.934 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:15.934+0000 7f1343e00640 1 -- 172.21.15.22:0/3410923332 <== mon.2 v2:172.21.15.79:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 8153+0+0 (secure 0 0 0) 0x7f133c0ceda0 con 0x7f134c1016b0 2024-03-28T18:07:15.934 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:15.934+0000 7f134aa00640 1 --2- 172.21.15.22:0/3410923332 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7f13300a2880 0x7f13300a4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:07:15.935 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:15.935+0000 7f134aa00640 1 --2- 172.21.15.22:0/3410923332 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7f13300a2880 0x7f13300a4d40 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f1338005ab0 tx=0x7f1338005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:07:15.943 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:15.943+0000 7f1343e00640 1 -- 172.21.15.22:0/3410923332 <== mon.2 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f133c09be40 con 0x7f134c1016b0 2024-03-28T18:07:16.148 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:16.148+0000 7f1352400640 1 -- 172.21.15.22:0/3410923332 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f134c0622d0 con 0x7f134c1016b0 2024-03-28T18:07:16.149 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:16.149+0000 7f1343e00640 1 -- 172.21.15.22:0/3410923332 <== mon.2 v2:172.21.15.79:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v92) v1 ==== 100+0+11 (secure 0 0 0) 0x7f133c09b740 con 0x7f134c1016b0 2024-03-28T18:07:16.149 INFO:teuthology.orchestra.run.smithi022.stdout:pg_num: 64 2024-03-28T18:07:16.152 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:16.152+0000 7f1352400640 1 -- 172.21.15.22:0/3410923332 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7f13300a2880 msgr2=0x7f13300a4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:07:16.152 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:16.152+0000 7f1352400640 1 --2- 172.21.15.22:0/3410923332 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7f13300a2880 0x7f13300a4d40 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f1338005ab0 tx=0x7f1338005fd0 comp rx=0 tx=0).stop 2024-03-28T18:07:16.152 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:16.152+0000 7f1352400640 1 -- 172.21.15.22:0/3410923332 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f134c1016b0 msgr2=0x7f134c1979e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:07:16.152 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:16.152+0000 7f1352400640 1 --2- 172.21.15.22:0/3410923332 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f134c1016b0 0x7f134c1979e0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f133c00e9c0 tx=0x7f133c00ee80 comp rx=0 tx=0).stop 2024-03-28T18:07:16.152 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:16.152+0000 7f1352400640 1 -- 172.21.15.22:0/3410923332 shutdown_connections 2024-03-28T18:07:16.152 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:16.152+0000 7f1352400640 1 --2- 172.21.15.22:0/3410923332 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7f13300a2880 0x7f13300a4d40 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:07:16.152 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:16.152+0000 7f1352400640 1 --2- 172.21.15.22:0/3410923332 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f134c103260 0x7f134c19ef90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:07:16.152 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:16.152+0000 7f1352400640 1 --2- 172.21.15.22:0/3410923332 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f134c1028a0 0x7f134c197f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:07:16.152 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:16.152+0000 7f1352400640 1 --2- 172.21.15.22:0/3410923332 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f134c1016b0 0x7f134c1979e0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:07:16.153 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:16.152+0000 7f1352400640 1 -- 172.21.15.22:0/3410923332 >> 172.21.15.22:0/3410923332 conn(0x7f134c0fcc40 msgr2=0x7f134c105c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:07:16.153 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:16.152+0000 7f1352400640 1 -- 172.21.15.22:0/3410923332 shutdown_connections 2024-03-28T18:07:16.153 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:16.152+0000 7f1352400640 1 -- 172.21.15.22:0/3410923332 wait complete. 2024-03-28T18:07:16.171 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-28T18:07:16.379 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:16.379+0000 7ffa9f200640 1 -- 172.21.15.22:0/1739422522 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ffaa01030b0 msgr2=0x7ffaa0109940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:07:16.380 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:16.379+0000 7ffa9f200640 1 --2- 172.21.15.22:0/1739422522 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ffaa01030b0 0x7ffaa0109940 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7ffa98009ca0 tx=0x7ffa9802c0f0 comp rx=0 tx=0).stop 2024-03-28T18:07:16.380 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:16.379+0000 7ffa9f200640 1 -- 172.21.15.22:0/1739422522 shutdown_connections 2024-03-28T18:07:16.380 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:16.379+0000 7ffa9f200640 1 --2- 172.21.15.22:0/1739422522 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ffaa01030b0 0x7ffaa0109940 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:07:16.380 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:16.379+0000 7ffa9f200640 1 --2- 172.21.15.22:0/1739422522 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ffaa01026f0 0x7ffaa0102b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:07:16.380 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:16.379+0000 7ffa9f200640 1 --2- 172.21.15.22:0/1739422522 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ffaa0101500 0x7ffaa0101920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:07:16.380 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:16.379+0000 7ffa9f200640 1 -- 172.21.15.22:0/1739422522 >> 172.21.15.22:0/1739422522 conn(0x7ffaa00fca70 msgr2=0x7ffaa00feeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:07:16.380 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:16.379+0000 7ffa9f200640 1 -- 172.21.15.22:0/1739422522 shutdown_connections 2024-03-28T18:07:16.380 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:16.380+0000 7ffa9f200640 1 -- 172.21.15.22:0/1739422522 wait complete. 2024-03-28T18:07:16.382 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:16.380+0000 7ffa9f200640 1 Processor -- start 2024-03-28T18:07:16.382 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:16.381+0000 7ffa9f200640 1 -- start start 2024-03-28T18:07:16.382 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:16.381+0000 7ffa9f200640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ffaa0101500 0x7ffaa01999c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:07:16.382 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:16.381+0000 7ffa9f200640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ffaa01026f0 0x7ffaa0199f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:07:16.383 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:16.381+0000 7ffa9f200640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ffaa01030b0 0x7ffaa01a0f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:07:16.383 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:16.381+0000 7ffa9f200640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffaa010d160 con 0x7ffaa01026f0 2024-03-28T18:07:16.383 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:16.381+0000 7ffa9f200640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffaa010cfe0 con 0x7ffaa01030b0 2024-03-28T18:07:16.383 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:16.381+0000 7ffa9f200640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffaa010d2e0 con 0x7ffaa0101500 2024-03-28T18:07:16.383 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:16.381+0000 7ffa9e800640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ffaa01030b0 0x7ffaa01a0f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:07:16.383 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:16.381+0000 7ffa9de00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ffaa0101500 0x7ffaa01999c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:07:16.383 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:16.381+0000 7ffa9d400640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ffaa01026f0 0x7ffaa0199f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:07:16.383 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:16.381+0000 7ffa9e800640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ffaa01030b0 0x7ffaa01a0f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.22:44284/0 (socket says 172.21.15.22:44284) 2024-03-28T18:07:16.383 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:16.381+0000 7ffa9e800640 1 -- 172.21.15.22:0/2589180031 learned_addr learned my addr 172.21.15.22:0/2589180031 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-28T18:07:16.383 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:16.381+0000 7ffa9de00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ffaa0101500 0x7ffaa01999c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.22:33808/0 (socket says 172.21.15.22:33808) 2024-03-28T18:07:16.383 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:16.382+0000 7ffa9d400640 1 -- 172.21.15.22:0/2589180031 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ffaa0101500 msgr2=0x7ffaa01999c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:07:16.383 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:16.382+0000 7ffa9d400640 1 --2- 172.21.15.22:0/2589180031 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ffaa0101500 0x7ffaa01999c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:07:16.383 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:16.382+0000 7ffa9d400640 1 -- 172.21.15.22:0/2589180031 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ffaa01030b0 msgr2=0x7ffaa01a0f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:07:16.383 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:16.382+0000 7ffa9d400640 1 --2- 172.21.15.22:0/2589180031 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ffaa01030b0 0x7ffaa01a0f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:07:16.384 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:16.382+0000 7ffa9d400640 1 -- 172.21.15.22:0/2589180031 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffaa01a15e0 con 0x7ffaa01026f0 2024-03-28T18:07:16.384 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:16.382+0000 7ffa9e800640 1 --2- 172.21.15.22:0/2589180031 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ffaa01030b0 0x7ffaa01a0f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-28T18:07:16.384 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:16.382+0000 7ffa9d400640 1 --2- 172.21.15.22:0/2589180031 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ffaa01026f0 0x7ffaa0199f00 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7ffa8c00e9f0 tx=0x7ffa8c00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:07:16.384 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:16.382+0000 7ffa96a00640 1 -- 172.21.15.22:0/2589180031 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa8c00ccc0 con 0x7ffaa01026f0 2024-03-28T18:07:16.384 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:16.382+0000 7ffa96a00640 1 -- 172.21.15.22:0/2589180031 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7ffa8c010900 con 0x7ffaa01026f0 2024-03-28T18:07:16.384 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:16.382+0000 7ffa9f200640 1 -- 172.21.15.22:0/2589180031 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffaa01a18d0 con 0x7ffaa01026f0 2024-03-28T18:07:16.384 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:16.383+0000 7ffa96a00640 1 -- 172.21.15.22:0/2589180031 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa8c018960 con 0x7ffaa01026f0 2024-03-28T18:07:16.384 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:16.383+0000 7ffa9f200640 1 -- 172.21.15.22:0/2589180031 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffaa01a1e20 con 0x7ffaa01026f0 2024-03-28T18:07:16.384 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:16.384+0000 7ffa9f200640 1 -- 172.21.15.22:0/2589180031 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffaa01a2050 con 0x7ffaa01026f0 2024-03-28T18:07:16.386 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:16.387+0000 7ffa96a00640 1 -- 172.21.15.22:0/2589180031 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 18) v1 ==== 137290+0+0 (secure 0 0 0) 0x7ffa8c010ab0 con 0x7ffaa01026f0 2024-03-28T18:07:16.388 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:16.389+0000 7ffa96a00640 1 --2- 172.21.15.22:0/2589180031 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7ffa840a2800 0x7ffa840a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:07:16.388 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:16.389+0000 7ffa96a00640 1 -- 172.21.15.22:0/2589180031 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 8153+0+0 (secure 0 0 0) 0x7ffa8c0cece0 con 0x7ffaa01026f0 2024-03-28T18:07:16.389 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:16.389+0000 7ffa9de00640 1 --2- 172.21.15.22:0/2589180031 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7ffa840a2800 0x7ffa840a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:07:16.389 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:16.390+0000 7ffa9de00640 1 --2- 172.21.15.22:0/2589180031 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7ffa840a2800 0x7ffa840a4cc0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7ffa90005fd0 tx=0x7ffa9000b390 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:07:16.394 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:16.394+0000 7ffa96a00640 1 -- 172.21.15.22:0/2589180031 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7ffa8c09b7d0 con 0x7ffaa01026f0 2024-03-28T18:07:16.531 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:07:16 smithi022 ceph-mon[31985]: pgmap v444: 97 pgs: 97 active+clean; 644 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 426 B/s wr, 9 op/s 2024-03-28T18:07:16.531 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:07:16 smithi022 ceph-mon[31985]: from='client.? 172.21.15.22:0/2380349652' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-28T18:07:16.531 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:07:16 smithi022 ceph-mon[31985]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T18:07:16.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:07:16 smithi022 ceph-mon[31985]: from='client.? 172.21.15.22:0/3410923332' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-28T18:07:16.608 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:16.609+0000 7ffa9f200640 1 -- 172.21.15.22:0/2589180031 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ffaa01a2340 con 0x7ffaa01026f0 2024-03-28T18:07:16.610 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:16.611+0000 7ffa96a00640 1 -- 172.21.15.22:0/2589180031 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v92) v1 ==== 74+0+22522 (secure 0 0 0) 0x7ffa8c09f870 con 0x7ffaa01026f0 2024-03-28T18:07:16.611 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-03-28T18:07:16.611 INFO:teuthology.orchestra.run.smithi022.stdout:{"epoch":92,"fsid":"fd220748-ed2b-11ee-95d0-87774f69a715","created":"2024-03-28T17:54:00.440279+0000","modified":"2024-03-28T18:05:11.725341+0000","last_up_change":"2024-03-28T18:03:43.220095+0000","last_in_change":"2024-03-28T18:03:18.526399+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-28T17:58:33.168872+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-28T18:04:47.004797+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-28T18:04:48.465502+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"90","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"5d708e06-f05b-40ba-bbcf-258ddc4f43d7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6802","nonce":2186290846},{"type":"v1","addr":"172.21.15.22:6803","nonce":2186290846}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6804","nonce":2186290846},{"type":"v1","addr":"172.21.15.22:6805","nonce":2186290846}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6808","nonce":2186290846},{"type":"v1","addr":"172.21.15.22:6809","nonce":2186290846}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6806","nonce":2186290846},{"type":"v1","addr":"172.21.15.22:6807","nonce":2186290846}]},"public_addr":"172.21.15.22:6803/2186290846","cluster_addr":"172.21.15.22:6805/2186290846","heartbeat_back_addr":"172.21.15.22:6809/2186290846","heartbeat_front_addr":"172.21.15.22:6807/2186290846","state":["exists","up"]},{"osd":1,"uuid":"68ad09bb-9256-409a-9867-4e14a2cb94aa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6800","nonce":2835549681},{"type":"v1","addr":"172.21.15.79:6801","nonce":2835549681}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6802","nonce":2835549681},{"type":"v1","addr":"172.21.15.79:6803","nonce":2835549681}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6806","nonce":2835549681},{"type":"v1","addr":"172.21.15.79:6807","nonce":2835549681}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6804","nonce":2835549681},{"type":"v1","addr":"172.21.15.79:6805","nonce":2835549681}]},"public_addr":"172.21.15.79:6801/2835549681","cluster_addr":"172.21.15.79:6803/2835549681","heartbeat_back_addr":"172.21.15.79:6807/2835549681","heartbeat_front_addr":"172.21.15.79:6805/2835549681","state":["exists","up"]},{"osd":2,"uuid":"ed11a5a7-004d-41f3-a5fc-eb2d603bd33f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6800","nonce":470588085},{"type":"v1","addr":"172.21.15.134:6801","nonce":470588085}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6802","nonce":470588085},{"type":"v1","addr":"172.21.15.134:6803","nonce":470588085}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6806","nonce":470588085},{"type":"v1","addr":"172.21.15.134:6807","nonce":470588085}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6804","nonce":470588085},{"type":"v1","addr":"172.21.15.134:6805","nonce":470588085}]},"public_addr":"172.21.15.134:6801/470588085","cluster_addr":"172.21.15.134:6803/470588085","heartbeat_back_addr":"172.21.15.134:6807/470588085","heartbeat_front_addr":"172.21.15.134:6805/470588085","state":["exists","up"]},{"osd":3,"uuid":"81a6c547-2a50-44c3-84d7-23ab6831790c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6810","nonce":1702441854},{"type":"v1","addr":"172.21.15.22:6811","nonce":1702441854}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6812","nonce":1702441854},{"type":"v1","addr":"172.21.15.22:6813","nonce":1702441854}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6816","nonce":1702441854},{"type":"v1","addr":"172.21.15.22:6817","nonce":1702441854}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6814","nonce":1702441854},{"type":"v1","addr":"172.21.15.22:6815","nonce":1702441854}]},"public_addr":"172.21.15.22:6811/1702441854","cluster_addr":"172.21.15.22:6813/1702441854","heartbeat_back_addr":"172.21.15.22:6817/1702441854","heartbeat_front_addr":"172.21.15.22:6815/1702441854","state":["exists","up"]},{"osd":4,"uuid":"d23a254e-7182-4471-8bca-7eb684ec618e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6808","nonce":630132437},{"type":"v1","addr":"172.21.15.79:6809","nonce":630132437}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6810","nonce":630132437},{"type":"v1","addr":"172.21.15.79:6811","nonce":630132437}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6814","nonce":630132437},{"type":"v1","addr":"172.21.15.79:6815","nonce":630132437}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6812","nonce":630132437},{"type":"v1","addr":"172.21.15.79:6813","nonce":630132437}]},"public_addr":"172.21.15.79:6809/630132437","cluster_addr":"172.21.15.79:6811/630132437","heartbeat_back_addr":"172.21.15.79:6815/630132437","heartbeat_front_addr":"172.21.15.79:6813/630132437","state":["exists","up"]},{"osd":5,"uuid":"a7b98502-2023-4f0a-bfa0-6840cc495e0c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":40,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6808","nonce":2135406915},{"type":"v1","addr":"172.21.15.134:6809","nonce":2135406915}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6810","nonce":2135406915},{"type":"v1","addr":"172.21.15.134:6811","nonce":2135406915}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6814","nonce":2135406915},{"type":"v1","addr":"172.21.15.134:6815","nonce":2135406915}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6812","nonce":2135406915},{"type":"v1","addr":"172.21.15.134:6813","nonce":2135406915}]},"public_addr":"172.21.15.134:6809/2135406915","cluster_addr":"172.21.15.134:6811/2135406915","heartbeat_back_addr":"172.21.15.134:6815/2135406915","heartbeat_front_addr":"172.21.15.134:6813/2135406915","state":["exists","up"]},{"osd":6,"uuid":"634c6562-d683-41c8-8242-fc848c38ce1f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6818","nonce":1400680396},{"type":"v1","addr":"172.21.15.22:6819","nonce":1400680396}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6820","nonce":1400680396},{"type":"v1","addr":"172.21.15.22:6821","nonce":1400680396}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6824","nonce":1400680396},{"type":"v1","addr":"172.21.15.22:6825","nonce":1400680396}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6822","nonce":1400680396},{"type":"v1","addr":"172.21.15.22:6823","nonce":1400680396}]},"public_addr":"172.21.15.22:6819/1400680396","cluster_addr":"172.21.15.22:6821/1400680396","heartbeat_back_addr":"172.21.15.22:6825/1400680396","heartbeat_front_addr":"172.21.15.22:6823/1400680396","state":["exists","up"]},{"osd":7,"uuid":"504eb4a8-6731-45ad-932c-1b38ef157871","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":53,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6816","nonce":1287105314},{"type":"v1","addr":"172.21.15.79:6817","nonce":1287105314}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6818","nonce":1287105314},{"type":"v1","addr":"172.21.15.79:6819","nonce":1287105314}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6822","nonce":1287105314},{"type":"v1","addr":"172.21.15.79:6823","nonce":1287105314}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6820","nonce":1287105314},{"type":"v1","addr":"172.21.15.79:6821","nonce":1287105314}]},"public_addr":"172.21.15.79:6817/1287105314","cluster_addr":"172.21.15.79:6819/1287105314","heartbeat_back_addr":"172.21.15.79:6823/1287105314","heartbeat_front_addr":"172.21.15.79:6821/1287105314","state":["exists","up"]},{"osd":8,"uuid":"20aa66c1-03d6-4cb8-a096-0eae41979c17","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":62,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6816","nonce":2265650388},{"type":"v1","addr":"172.21.15.134:6817","nonce":2265650388}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6818","nonce":2265650388},{"type":"v1","addr":"172.21.15.134:6819","nonce":2265650388}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6822","nonce":2265650388},{"type":"v1","addr":"172.21.15.134:6823","nonce":2265650388}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6820","nonce":2265650388},{"type":"v1","addr":"172.21.15.134:6821","nonce":2265650388}]},"public_addr":"172.21.15.134:6817/2265650388","cluster_addr":"172.21.15.134:6819/2265650388","heartbeat_back_addr":"172.21.15.134:6823/2265650388","heartbeat_front_addr":"172.21.15.134:6821/2265650388","state":["exists","up"]},{"osd":9,"uuid":"e1ae4020-7c2c-4334-a753-ee6b12d98584","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":70,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6826","nonce":3762222259},{"type":"v1","addr":"172.21.15.22:6827","nonce":3762222259}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6828","nonce":3762222259},{"type":"v1","addr":"172.21.15.22:6829","nonce":3762222259}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6832","nonce":3762222259},{"type":"v1","addr":"172.21.15.22:6833","nonce":3762222259}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6830","nonce":3762222259},{"type":"v1","addr":"172.21.15.22:6831","nonce":3762222259}]},"public_addr":"172.21.15.22:6827/3762222259","cluster_addr":"172.21.15.22:6829/3762222259","heartbeat_back_addr":"172.21.15.22:6833/3762222259","heartbeat_front_addr":"172.21.15.22:6831/3762222259","state":["exists","up"]},{"osd":10,"uuid":"f5198d52-df48-4679-a860-4dcb440b7a9f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":77,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6824","nonce":1746167904},{"type":"v1","addr":"172.21.15.79:6825","nonce":1746167904}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6826","nonce":1746167904},{"type":"v1","addr":"172.21.15.79:6827","nonce":1746167904}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6830","nonce":1746167904},{"type":"v1","addr":"172.21.15.79:6831","nonce":1746167904}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6828","nonce":1746167904},{"type":"v1","addr":"172.21.15.79:6829","nonce":1746167904}]},"public_addr":"172.21.15.79:6825/1746167904","cluster_addr":"172.21.15.79:6827/1746167904","heartbeat_back_addr":"172.21.15.79:6831/1746167904","heartbeat_front_addr":"172.21.15.79:6829/1746167904","state":["exists","up"]},{"osd":11,"uuid":"1427bca7-3414-4468-9246-e999156bc2ae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":85,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6824","nonce":4271716428},{"type":"v1","addr":"172.21.15.134:6825","nonce":4271716428}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6826","nonce":4271716428},{"type":"v1","addr":"172.21.15.134:6827","nonce":4271716428}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6830","nonce":4271716428},{"type":"v1","addr":"172.21.15.134:6831","nonce":4271716428}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6828","nonce":4271716428},{"type":"v1","addr":"172.21.15.134:6829","nonce":4271716428}]},"public_addr":"172.21.15.134:6825/4271716428","cluster_addr":"172.21.15.134:6827/4271716428","heartbeat_back_addr":"172.21.15.134:6831/4271716428","heartbeat_front_addr":"172.21.15.134:6829/4271716428","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T17:57:21.225125+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T17:57:55.197254+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T17:58:27.825136+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T17:59:04.319893+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T17:59:38.407752+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T18:00:12.118805+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T18:00:47.635214+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T18:01:21.231055+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T18:01:55.817861+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T18:02:31.680160+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T18:03:06.783398+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T18:03:41.307347+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.8","mappings":[{"from":2,"to":11}]},{"pgid":"2.1d","mappings":[{"from":8,"to":11}]},{"pgid":"3.3","mappings":[{"from":4,"to":9}]},{"pgid":"3.c","mappings":[{"from":3,"to":9}]},{"pgid":"3.21","mappings":[{"from":4,"to":8}]},{"pgid":"3.25","mappings":[{"from":5,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.22:0/3521924283":"2024-03-29T17:55:10.425504+0000","172.21.15.22:6801/730635939":"2024-03-29T17:55:10.425504+0000","172.21.15.22:0/2471229459":"2024-03-29T17:54:34.004559+0000","172.21.15.22:6800/3166828081":"2024-03-29T17:54:34.004559+0000","172.21.15.22:6800/730635939":"2024-03-29T17:55:10.425504+0000","172.21.15.22:0/1229047849":"2024-03-29T17:54:34.004559+0000","172.21.15.22:6801/3166828081":"2024-03-29T17:54:34.004559+0000","172.21.15.22:0/2717818856":"2024-03-29T17:55:10.425504+0000","172.21.15.22:0/2157570333":"2024-03-29T17:54:34.004559+0000","172.21.15.22:0/1006589530":"2024-03-29T17:55:10.425504+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-28T18:07:16.613 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:16.613+0000 7ffa9f200640 1 -- 172.21.15.22:0/2589180031 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7ffa840a2800 msgr2=0x7ffa840a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:07:16.613 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:16.613+0000 7ffa9f200640 1 --2- 172.21.15.22:0/2589180031 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7ffa840a2800 0x7ffa840a4cc0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7ffa90005fd0 tx=0x7ffa9000b390 comp rx=0 tx=0).stop 2024-03-28T18:07:16.614 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:16.613+0000 7ffa9f200640 1 -- 172.21.15.22:0/2589180031 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ffaa01026f0 msgr2=0x7ffaa0199f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:07:16.614 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:16.613+0000 7ffa9f200640 1 --2- 172.21.15.22:0/2589180031 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ffaa01026f0 0x7ffaa0199f00 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7ffa8c00e9f0 tx=0x7ffa8c00eeb0 comp rx=0 tx=0).stop 2024-03-28T18:07:16.614 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:16.613+0000 7ffa9f200640 1 -- 172.21.15.22:0/2589180031 shutdown_connections 2024-03-28T18:07:16.614 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:16.613+0000 7ffa9f200640 1 --2- 172.21.15.22:0/2589180031 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7ffa840a2800 0x7ffa840a4cc0 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:07:16.614 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:16.613+0000 7ffa9f200640 1 --2- 172.21.15.22:0/2589180031 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ffaa01030b0 0x7ffaa01a0f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:07:16.614 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:16.613+0000 7ffa9f200640 1 --2- 172.21.15.22:0/2589180031 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ffaa01026f0 0x7ffaa0199f00 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:07:16.614 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:16.613+0000 7ffa9f200640 1 --2- 172.21.15.22:0/2589180031 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ffaa0101500 0x7ffaa01999c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:07:16.614 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:16.613+0000 7ffa9f200640 1 -- 172.21.15.22:0/2589180031 >> 172.21.15.22:0/2589180031 conn(0x7ffaa00fca70 msgr2=0x7ffaa00738d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:07:16.614 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:16.614+0000 7ffa9f200640 1 -- 172.21.15.22:0/2589180031 shutdown_connections 2024-03-28T18:07:16.614 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:16.614+0000 7ffa9f200640 1 -- 172.21.15.22:0/2589180031 wait complete. 2024-03-28T18:07:16.630 INFO:tasks.fwd_scrub.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-28T17:58:33.168872+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-28T18:04:47.004797+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '89', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-03-28T18:04:48.465502+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '90', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-28T18:07:16.631 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-28T18:07:16.674 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:07:16 smithi134 ceph-mon[35037]: pgmap v444: 97 pgs: 97 active+clean; 644 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 426 B/s wr, 9 op/s 2024-03-28T18:07:16.674 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:07:16 smithi134 ceph-mon[35037]: from='client.? 172.21.15.22:0/2380349652' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-28T18:07:16.674 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:07:16 smithi134 ceph-mon[35037]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T18:07:16.674 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:07:16 smithi134 ceph-mon[35037]: from='client.? 172.21.15.22:0/3410923332' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-28T18:07:16.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:07:16 smithi079 ceph-mon[35312]: pgmap v444: 97 pgs: 97 active+clean; 644 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 426 B/s wr, 9 op/s 2024-03-28T18:07:16.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:07:16 smithi079 ceph-mon[35312]: from='client.? 172.21.15.22:0/2380349652' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-28T18:07:16.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:07:16 smithi079 ceph-mon[35312]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T18:07:16.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:07:16 smithi079 ceph-mon[35312]: from='client.? 172.21.15.22:0/3410923332' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-28T18:07:16.828 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:16.828+0000 7f4d9ba00640 1 -- 172.21.15.22:0/1519686938 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4d94105470 msgr2=0x7f4d9410bd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:07:16.828 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:16.828+0000 7f4d9ba00640 1 --2- 172.21.15.22:0/1519686938 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4d94105470 0x7f4d9410bd00 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f4d8c009ca0 tx=0x7f4d8c02c100 comp rx=0 tx=0).stop 2024-03-28T18:07:16.829 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:16.828+0000 7f4d9ba00640 1 -- 172.21.15.22:0/1519686938 shutdown_connections 2024-03-28T18:07:16.829 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:16.828+0000 7f4d9ba00640 1 --2- 172.21.15.22:0/1519686938 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4d94105470 0x7f4d9410bd00 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:07:16.829 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:16.828+0000 7f4d9ba00640 1 --2- 172.21.15.22:0/1519686938 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f4d94104ab0 0x7f4d94104f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:07:16.829 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:16.828+0000 7f4d9ba00640 1 --2- 172.21.15.22:0/1519686938 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4d941038c0 0x7f4d94103ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:07:16.829 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:16.828+0000 7f4d9ba00640 1 -- 172.21.15.22:0/1519686938 >> 172.21.15.22:0/1519686938 conn(0x7f4d940fee50 msgr2=0x7f4d94101270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:07:16.829 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:16.828+0000 7f4d9ba00640 1 -- 172.21.15.22:0/1519686938 shutdown_connections 2024-03-28T18:07:16.829 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:16.828+0000 7f4d9ba00640 1 -- 172.21.15.22:0/1519686938 wait complete. 2024-03-28T18:07:16.832 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:16.829+0000 7f4d9ba00640 1 Processor -- start 2024-03-28T18:07:16.832 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:16.829+0000 7f4d9ba00640 1 -- start start 2024-03-28T18:07:16.832 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:16.830+0000 7f4d9ba00640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f4d941038c0 0x7f4d94199b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:07:16.832 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:16.830+0000 7f4d9ba00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4d94104ab0 0x7f4d9419a090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:07:16.832 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:16.830+0000 7f4d9ba00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4d94105470 0x7f4d941a1100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:07:16.832 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:16.830+0000 7f4d9ba00640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d9410f510 con 0x7f4d94105470 2024-03-28T18:07:16.832 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:16.830+0000 7f4d9ba00640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d9410f390 con 0x7f4d941038c0 2024-03-28T18:07:16.832 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:16.830+0000 7f4d9ba00640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d9410f690 con 0x7f4d94104ab0 2024-03-28T18:07:16.832 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:16.830+0000 7f4d99000640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f4d941038c0 0x7f4d94199b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:07:16.832 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:16.830+0000 7f4d99000640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f4d941038c0 0x7f4d94199b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.22:44298/0 (socket says 172.21.15.22:44298) 2024-03-28T18:07:16.832 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:16.830+0000 7f4d99000640 1 -- 172.21.15.22:0/4105694191 learned_addr learned my addr 172.21.15.22:0/4105694191 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-28T18:07:16.832 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:16.830+0000 7f4d99a00640 1 --2- 172.21.15.22:0/4105694191 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4d94105470 0x7f4d941a1100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:07:16.833 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:16.830+0000 7f4d93e00640 1 --2- 172.21.15.22:0/4105694191 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4d94104ab0 0x7f4d9419a090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:07:16.833 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:16.831+0000 7f4d99a00640 1 -- 172.21.15.22:0/4105694191 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4d94104ab0 msgr2=0x7f4d9419a090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:07:16.833 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:16.831+0000 7f4d99a00640 1 --2- 172.21.15.22:0/4105694191 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4d94104ab0 0x7f4d9419a090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:07:16.833 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:16.831+0000 7f4d99a00640 1 -- 172.21.15.22:0/4105694191 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f4d941038c0 msgr2=0x7f4d94199b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:07:16.833 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:16.831+0000 7f4d99a00640 1 --2- 172.21.15.22:0/4105694191 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f4d941038c0 0x7f4d94199b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:07:16.833 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:16.831+0000 7f4d99a00640 1 -- 172.21.15.22:0/4105694191 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4d941a1770 con 0x7f4d94105470 2024-03-28T18:07:16.833 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:16.831+0000 7f4d99a00640 1 --2- 172.21.15.22:0/4105694191 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4d94105470 0x7f4d941a1100 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f4d8c02c640 tx=0x7f4d8c034d70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:07:16.833 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:16.831+0000 7f4d91600640 1 -- 172.21.15.22:0/4105694191 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d8c039050 con 0x7f4d94105470 2024-03-28T18:07:16.833 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:16.831+0000 7f4d91600640 1 -- 172.21.15.22:0/4105694191 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f4d8c03a040 con 0x7f4d94105470 2024-03-28T18:07:16.833 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:16.831+0000 7f4d91600640 1 -- 172.21.15.22:0/4105694191 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d8c0046b0 con 0x7f4d94105470 2024-03-28T18:07:16.833 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:16.831+0000 7f4d9ba00640 1 -- 172.21.15.22:0/4105694191 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4d941a1a00 con 0x7f4d94105470 2024-03-28T18:07:16.833 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:16.832+0000 7f4d9ba00640 1 -- 172.21.15.22:0/4105694191 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4d941a1ea0 con 0x7f4d94105470 2024-03-28T18:07:16.834 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:16.833+0000 7f4d9ba00640 1 -- 172.21.15.22:0/4105694191 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4d9410abe0 con 0x7f4d94105470 2024-03-28T18:07:16.834 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:16.835+0000 7f4d91600640 1 -- 172.21.15.22:0/4105694191 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 18) v1 ==== 137290+0+0 (secure 0 0 0) 0x7f4d8c003970 con 0x7f4d94105470 2024-03-28T18:07:16.836 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:16.836+0000 7f4d91600640 1 --2- 172.21.15.22:0/4105694191 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7f4d7c0a2880 0x7f4d7c0a4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:07:16.836 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:16.836+0000 7f4d91600640 1 -- 172.21.15.22:0/4105694191 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 8153+0+0 (secure 0 0 0) 0x7f4d8c0f1160 con 0x7f4d94105470 2024-03-28T18:07:16.836 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:16.836+0000 7f4d99000640 1 --2- 172.21.15.22:0/4105694191 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7f4d7c0a2880 0x7f4d7c0a4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:07:16.837 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:16.837+0000 7f4d99000640 1 --2- 172.21.15.22:0/4105694191 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7f4d7c0a2880 0x7f4d7c0a4d40 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f4d8800abd0 tx=0x7f4d8800a380 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:07:16.840 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:16.841+0000 7f4d91600640 1 -- 172.21.15.22:0/4105694191 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f4d8c0be200 con 0x7f4d94105470 2024-03-28T18:07:17.048 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:17.048+0000 7f4d9ba00640 1 -- 172.21.15.22:0/4105694191 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f4d9402d290 con 0x7f4d94105470 2024-03-28T18:07:17.049 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:17.049+0000 7f4d91600640 1 -- 172.21.15.22:0/4105694191 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v92) v1 ==== 93+0+10 (secure 0 0 0) 0x7f4d8c0bdb00 con 0x7f4d94105470 2024-03-28T18:07:17.049 INFO:teuthology.orchestra.run.smithi022.stdout:pg_num: 1 2024-03-28T18:07:17.052 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:17.052+0000 7f4d9ba00640 1 -- 172.21.15.22:0/4105694191 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7f4d7c0a2880 msgr2=0x7f4d7c0a4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:07:17.052 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:17.052+0000 7f4d9ba00640 1 --2- 172.21.15.22:0/4105694191 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7f4d7c0a2880 0x7f4d7c0a4d40 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f4d8800abd0 tx=0x7f4d8800a380 comp rx=0 tx=0).stop 2024-03-28T18:07:17.052 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:17.052+0000 7f4d9ba00640 1 -- 172.21.15.22:0/4105694191 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4d94105470 msgr2=0x7f4d941a1100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:07:17.052 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:17.052+0000 7f4d9ba00640 1 --2- 172.21.15.22:0/4105694191 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4d94105470 0x7f4d941a1100 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f4d8c02c640 tx=0x7f4d8c034d70 comp rx=0 tx=0).stop 2024-03-28T18:07:17.052 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:17.052+0000 7f4d9ba00640 1 -- 172.21.15.22:0/4105694191 shutdown_connections 2024-03-28T18:07:17.052 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:17.052+0000 7f4d9ba00640 1 --2- 172.21.15.22:0/4105694191 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7f4d7c0a2880 0x7f4d7c0a4d40 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:07:17.052 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:17.052+0000 7f4d9ba00640 1 --2- 172.21.15.22:0/4105694191 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4d94105470 0x7f4d941a1100 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:07:17.053 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:17.052+0000 7f4d9ba00640 1 --2- 172.21.15.22:0/4105694191 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4d94104ab0 0x7f4d9419a090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:07:17.053 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:17.052+0000 7f4d9ba00640 1 --2- 172.21.15.22:0/4105694191 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f4d941038c0 0x7f4d94199b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:07:17.053 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:17.052+0000 7f4d9ba00640 1 -- 172.21.15.22:0/4105694191 >> 172.21.15.22:0/4105694191 conn(0x7f4d940fee50 msgr2=0x7f4d94107db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:07:17.053 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:17.053+0000 7f4d9ba00640 1 -- 172.21.15.22:0/4105694191 shutdown_connections 2024-03-28T18:07:17.053 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:17.053+0000 7f4d9ba00640 1 -- 172.21.15.22:0/4105694191 wait complete. 2024-03-28T18:07:17.070 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-28T18:07:17.277 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:17.276+0000 7f69d1e00640 1 -- 172.21.15.22:0/936297412 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f69cc1028a0 msgr2=0x7f69cc102d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:07:17.277 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:17.276+0000 7f69d1e00640 1 --2- 172.21.15.22:0/936297412 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f69cc1028a0 0x7f69cc102d20 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f69b4009900 tx=0x7f69b402bc50 comp rx=0 tx=0).stop 2024-03-28T18:07:17.277 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:17.277+0000 7f69d1e00640 1 -- 172.21.15.22:0/936297412 shutdown_connections 2024-03-28T18:07:17.277 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:17.277+0000 7f69d1e00640 1 --2- 172.21.15.22:0/936297412 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f69cc103260 0x7f69cc109af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:07:17.277 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:17.277+0000 7f69d1e00640 1 --2- 172.21.15.22:0/936297412 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f69cc1028a0 0x7f69cc102d20 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:07:17.277 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:17.277+0000 7f69d1e00640 1 --2- 172.21.15.22:0/936297412 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f69cc1016b0 0x7f69cc101ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:07:17.277 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:17.277+0000 7f69d1e00640 1 -- 172.21.15.22:0/936297412 >> 172.21.15.22:0/936297412 conn(0x7f69cc0fcc40 msgr2=0x7f69cc0ff060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:07:17.277 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:17.277+0000 7f69d1e00640 1 -- 172.21.15.22:0/936297412 shutdown_connections 2024-03-28T18:07:17.277 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:17.277+0000 7f69d1e00640 1 -- 172.21.15.22:0/936297412 wait complete. 2024-03-28T18:07:17.280 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:17.278+0000 7f69d1e00640 1 Processor -- start 2024-03-28T18:07:17.280 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:17.278+0000 7f69d1e00640 1 -- start start 2024-03-28T18:07:17.280 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:17.278+0000 7f69d1e00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f69cc1016b0 0x7f69cc075450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:07:17.280 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:17.278+0000 7f69d1e00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f69cc1028a0 0x7f69cc073a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:07:17.280 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:17.278+0000 7f69d1e00640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f69cc103260 0x7f69cc073fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:07:17.280 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:17.278+0000 7f69d1e00640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69cc10d310 con 0x7f69cc1016b0 2024-03-28T18:07:17.280 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:17.278+0000 7f69d1e00640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69cc10d190 con 0x7f69cc103260 2024-03-28T18:07:17.280 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:17.278+0000 7f69d1e00640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69cc10d490 con 0x7f69cc1028a0 2024-03-28T18:07:17.280 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:17.279+0000 7f69caa00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f69cc1016b0 0x7f69cc075450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:07:17.280 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:17.279+0000 7f69caa00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f69cc1016b0 0x7f69cc075450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:55372/0 (socket says 172.21.15.22:55372) 2024-03-28T18:07:17.280 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:17.279+0000 7f69caa00640 1 -- 172.21.15.22:0/3955163630 learned_addr learned my addr 172.21.15.22:0/3955163630 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-28T18:07:17.281 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:17.279+0000 7f69cb400640 1 --2- 172.21.15.22:0/3955163630 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f69cc103260 0x7f69cc073fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:07:17.281 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:17.279+0000 7f69c3e00640 1 --2- 172.21.15.22:0/3955163630 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f69cc1028a0 0x7f69cc073a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:07:17.281 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:17.279+0000 7f69caa00640 1 -- 172.21.15.22:0/3955163630 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f69cc1028a0 msgr2=0x7f69cc073a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:07:17.281 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:17.279+0000 7f69caa00640 1 --2- 172.21.15.22:0/3955163630 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f69cc1028a0 0x7f69cc073a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:07:17.281 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:17.279+0000 7f69caa00640 1 -- 172.21.15.22:0/3955163630 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f69cc103260 msgr2=0x7f69cc073fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:07:17.281 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:17.279+0000 7f69caa00640 1 --2- 172.21.15.22:0/3955163630 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f69cc103260 0x7f69cc073fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:07:17.281 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:17.279+0000 7f69caa00640 1 -- 172.21.15.22:0/3955163630 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f69cc074860 con 0x7f69cc1016b0 2024-03-28T18:07:17.281 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:17.280+0000 7f69caa00640 1 --2- 172.21.15.22:0/3955163630 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f69cc1016b0 0x7f69cc075450 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f69bc00e9c0 tx=0x7f69bc00ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:07:17.281 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:17.280+0000 7f69d0a00640 1 -- 172.21.15.22:0/3955163630 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69bc00cd80 con 0x7f69cc1016b0 2024-03-28T18:07:17.281 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:17.280+0000 7f69d0a00640 1 -- 172.21.15.22:0/3955163630 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f69bc0035f0 con 0x7f69cc1016b0 2024-03-28T18:07:17.281 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:17.280+0000 7f69d1e00640 1 -- 172.21.15.22:0/3955163630 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f69cc1a1b00 con 0x7f69cc1016b0 2024-03-28T18:07:17.281 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:17.280+0000 7f69d0a00640 1 -- 172.21.15.22:0/3955163630 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69bc010690 con 0x7f69cc1016b0 2024-03-28T18:07:17.281 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:17.280+0000 7f69d1e00640 1 -- 172.21.15.22:0/3955163630 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f69cc1a1fd0 con 0x7f69cc1016b0 2024-03-28T18:07:17.282 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:17.281+0000 7f69d1e00640 1 -- 172.21.15.22:0/3955163630 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f69cc0688e0 con 0x7f69cc1016b0 2024-03-28T18:07:17.283 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:17.284+0000 7f69d0a00640 1 -- 172.21.15.22:0/3955163630 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 18) v1 ==== 137290+0+0 (secure 0 0 0) 0x7f69bc0037a0 con 0x7f69cc1016b0 2024-03-28T18:07:17.285 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:17.285+0000 7f69d0a00640 1 --2- 172.21.15.22:0/3955163630 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7f69b00a2800 0x7f69b00a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:07:17.285 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:17.285+0000 7f69d0a00640 1 -- 172.21.15.22:0/3955163630 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 8153+0+0 (secure 0 0 0) 0x7f69bc0cee60 con 0x7f69cc1016b0 2024-03-28T18:07:17.285 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:17.285+0000 7f69c3e00640 1 --2- 172.21.15.22:0/3955163630 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7f69b00a2800 0x7f69b00a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:07:17.286 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:17.286+0000 7f69c3e00640 1 --2- 172.21.15.22:0/3955163630 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7f69b00a2800 0x7f69b00a4cc0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f69b4006010 tx=0x7f69b4000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:07:17.289 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:17.289+0000 7f69d0a00640 1 -- 172.21.15.22:0/3955163630 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f69bc09bf00 con 0x7f69cc1016b0 2024-03-28T18:07:17.500 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:17.501+0000 7f69d1e00640 1 -- 172.21.15.22:0/3955163630 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f69cc0622d0 con 0x7f69cc1016b0 2024-03-28T18:07:17.501 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:17.501+0000 7f69d0a00640 1 -- 172.21.15.22:0/3955163630 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v92) v1 ==== 104+0+11 (secure 0 0 0) 0x7f69bc09b800 con 0x7f69cc1016b0 2024-03-28T18:07:17.501 INFO:teuthology.orchestra.run.smithi022.stdout:pg_num: 32 2024-03-28T18:07:17.505 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:17.503+0000 7f69d1e00640 1 -- 172.21.15.22:0/3955163630 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7f69b00a2800 msgr2=0x7f69b00a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:07:17.505 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:17.503+0000 7f69d1e00640 1 --2- 172.21.15.22:0/3955163630 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7f69b00a2800 0x7f69b00a4cc0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f69b4006010 tx=0x7f69b4000bc0 comp rx=0 tx=0).stop 2024-03-28T18:07:17.505 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:17.503+0000 7f69d1e00640 1 -- 172.21.15.22:0/3955163630 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f69cc1016b0 msgr2=0x7f69cc075450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:07:17.505 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:17.503+0000 7f69d1e00640 1 --2- 172.21.15.22:0/3955163630 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f69cc1016b0 0x7f69cc075450 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f69bc00e9c0 tx=0x7f69bc00ee80 comp rx=0 tx=0).stop 2024-03-28T18:07:17.505 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:17.504+0000 7f69d1e00640 1 -- 172.21.15.22:0/3955163630 shutdown_connections 2024-03-28T18:07:17.505 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:17.504+0000 7f69d1e00640 1 --2- 172.21.15.22:0/3955163630 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7f69b00a2800 0x7f69b00a4cc0 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:07:17.505 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:17.504+0000 7f69d1e00640 1 --2- 172.21.15.22:0/3955163630 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f69cc103260 0x7f69cc073fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:07:17.505 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:17.504+0000 7f69d1e00640 1 --2- 172.21.15.22:0/3955163630 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f69cc1028a0 0x7f69cc073a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:07:17.505 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:17.504+0000 7f69d1e00640 1 --2- 172.21.15.22:0/3955163630 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f69cc1016b0 0x7f69cc075450 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:07:17.505 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:17.504+0000 7f69d1e00640 1 -- 172.21.15.22:0/3955163630 >> 172.21.15.22:0/3955163630 conn(0x7f69cc0fcc40 msgr2=0x7f69cc105c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:07:17.506 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:17.504+0000 7f69d1e00640 1 -- 172.21.15.22:0/3955163630 shutdown_connections 2024-03-28T18:07:17.506 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:17.504+0000 7f69d1e00640 1 -- 172.21.15.22:0/3955163630 wait complete. 2024-03-28T18:07:17.522 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-28T18:07:17.725 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:17.725+0000 7f5197600640 1 -- 172.21.15.22:0/1028640018 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f519006db30 msgr2=0x7f519006f9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:07:17.725 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:17.725+0000 7f5197600640 1 --2- 172.21.15.22:0/1028640018 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f519006db30 0x7f519006f9b0 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f5188009ca0 tx=0x7f518802c0f0 comp rx=0 tx=0).stop 2024-03-28T18:07:17.725 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:17.725+0000 7f5197600640 1 -- 172.21.15.22:0/1028640018 shutdown_connections 2024-03-28T18:07:17.725 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:17.725+0000 7f5197600640 1 --2- 172.21.15.22:0/1028640018 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f519006db30 0x7f519006f9b0 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:07:17.725 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:17.725+0000 7f5197600640 1 --2- 172.21.15.22:0/1028640018 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f519006d190 0x7f519006d5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:07:17.725 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:17.725+0000 7f5197600640 1 --2- 172.21.15.22:0/1028640018 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f519006ec10 0x7f519006f030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:07:17.726 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:17.725+0000 7f5197600640 1 -- 172.21.15.22:0/1028640018 >> 172.21.15.22:0/1028640018 conn(0x7f519006c0a0 msgr2=0x7f519006c4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:07:17.726 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:17.725+0000 7f5197600640 1 -- 172.21.15.22:0/1028640018 shutdown_connections 2024-03-28T18:07:17.726 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:17.725+0000 7f5197600640 1 -- 172.21.15.22:0/1028640018 wait complete. 2024-03-28T18:07:17.727 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:17.726+0000 7f5197600640 1 Processor -- start 2024-03-28T18:07:17.727 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:17.726+0000 7f5197600640 1 -- start start 2024-03-28T18:07:17.727 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:17.727+0000 7f5197600640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f519006d190 0x7f51901a68d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:07:17.727 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:17.727+0000 7f5197600640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f519006db30 0x7f51901a6e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:07:17.728 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:17.727+0000 7f5197600640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f519006ec10 0x7f51901ade30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:07:17.728 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:17.727+0000 7f5197600640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f519011c220 con 0x7f519006d190 2024-03-28T18:07:17.728 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:17.727+0000 7f5197600640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f519011c0a0 con 0x7f519006db30 2024-03-28T18:07:17.728 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:17.727+0000 7f5197600640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f519011c3a0 con 0x7f519006ec10 2024-03-28T18:07:17.728 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:17.727+0000 7f5194c00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f519006d190 0x7f51901a68d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:07:17.728 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:17.727+0000 7f5194c00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f519006d190 0x7f51901a68d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:55392/0 (socket says 172.21.15.22:55392) 2024-03-28T18:07:17.728 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:17.727+0000 7f5194c00640 1 -- 172.21.15.22:0/3629167582 learned_addr learned my addr 172.21.15.22:0/3629167582 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-28T18:07:17.728 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:17.727+0000 7f518fe00640 1 --2- 172.21.15.22:0/3629167582 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f519006db30 0x7f51901a6e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:07:17.728 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:17.727+0000 7f5195600640 1 --2- 172.21.15.22:0/3629167582 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f519006ec10 0x7f51901ade30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:07:17.728 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:17.728+0000 7f5194c00640 1 -- 172.21.15.22:0/3629167582 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f519006ec10 msgr2=0x7f51901ade30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:07:17.728 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:17.728+0000 7f5194c00640 1 --2- 172.21.15.22:0/3629167582 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f519006ec10 0x7f51901ade30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:07:17.728 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:17.728+0000 7f5194c00640 1 -- 172.21.15.22:0/3629167582 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f519006db30 msgr2=0x7f51901a6e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:07:17.728 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:17.728+0000 7f5194c00640 1 --2- 172.21.15.22:0/3629167582 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f519006db30 0x7f51901a6e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:07:17.728 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:17.728+0000 7f5194c00640 1 -- 172.21.15.22:0/3629167582 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51901ae4a0 con 0x7f519006d190 2024-03-28T18:07:17.729 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:17.728+0000 7f5195600640 1 --2- 172.21.15.22:0/3629167582 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f519006ec10 0x7f51901ade30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-28T18:07:17.729 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:17.728+0000 7f5194c00640 1 --2- 172.21.15.22:0/3629167582 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f519006d190 0x7f51901a68d0 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f518400e9f0 tx=0x7f518400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:07:17.729 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:17.728+0000 7f518d600640 1 -- 172.21.15.22:0/3629167582 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f518400cd80 con 0x7f519006d190 2024-03-28T18:07:17.729 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:17.728+0000 7f5197600640 1 -- 172.21.15.22:0/3629167582 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f51901ae790 con 0x7f519006d190 2024-03-28T18:07:17.729 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:17.728+0000 7f518d600640 1 -- 172.21.15.22:0/3629167582 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f5184010900 con 0x7f519006d190 2024-03-28T18:07:17.730 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:17.728+0000 7f518d600640 1 -- 172.21.15.22:0/3629167582 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51840189e0 con 0x7f519006d190 2024-03-28T18:07:17.730 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:17.728+0000 7f5197600640 1 -- 172.21.15.22:0/3629167582 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f51901aed30 con 0x7f519006d190 2024-03-28T18:07:17.730 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:17.729+0000 7f5197600640 1 -- 172.21.15.22:0/3629167582 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5190068490 con 0x7f519006d190 2024-03-28T18:07:17.731 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:17.732+0000 7f518d600640 1 -- 172.21.15.22:0/3629167582 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 18) v1 ==== 137290+0+0 (secure 0 0 0) 0x7f5184010ab0 con 0x7f519006d190 2024-03-28T18:07:17.733 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:17.734+0000 7f518d600640 1 --2- 172.21.15.22:0/3629167582 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7f51780a2800 0x7f51780a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:07:17.734 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:17.734+0000 7f518d600640 1 -- 172.21.15.22:0/3629167582 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 8153+0+0 (secure 0 0 0) 0x7f51840cf230 con 0x7f519006d190 2024-03-28T18:07:17.734 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:17.734+0000 7f518fe00640 1 --2- 172.21.15.22:0/3629167582 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7f51780a2800 0x7f51780a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:07:17.735 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:17.735+0000 7f518fe00640 1 --2- 172.21.15.22:0/3629167582 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7f51780a2800 0x7f51780a4cc0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f51901a7d90 tx=0x7f517c00b390 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:07:17.737 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:17.737+0000 7f518d600640 1 -- 172.21.15.22:0/3629167582 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f518409c370 con 0x7f519006d190 2024-03-28T18:07:17.781 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:07:17 smithi022 ceph-mon[31985]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T18:07:17.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:07:17 smithi022 ceph-mon[31985]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T18:07:17.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:07:17 smithi022 ceph-mon[31985]: from='client.? 172.21.15.22:0/2589180031' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-28T18:07:17.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:07:17 smithi022 ceph-mon[31985]: pgmap v445: 97 pgs: 97 active+clean; 649 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 1.7 KiB/s wr, 13 op/s 2024-03-28T18:07:17.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:07:17 smithi022 ceph-mon[31985]: from='client.? 172.21.15.22:0/4105694191' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-28T18:07:17.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:07:17 smithi022 ceph-mon[31985]: from='client.? 172.21.15.22:0/3955163630' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-28T18:07:17.863 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:07:17 smithi134 ceph-mon[35037]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T18:07:17.863 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:07:17 smithi134 ceph-mon[35037]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T18:07:17.863 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:07:17 smithi134 ceph-mon[35037]: from='client.? 172.21.15.22:0/2589180031' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-28T18:07:17.863 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:07:17 smithi134 ceph-mon[35037]: pgmap v445: 97 pgs: 97 active+clean; 649 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 1.7 KiB/s wr, 13 op/s 2024-03-28T18:07:17.863 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:07:17 smithi134 ceph-mon[35037]: from='client.? 172.21.15.22:0/4105694191' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-28T18:07:17.863 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:07:17 smithi134 ceph-mon[35037]: from='client.? 172.21.15.22:0/3955163630' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-28T18:07:17.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:07:17 smithi079 ceph-mon[35312]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T18:07:17.933 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:07:17 smithi079 ceph-mon[35312]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T18:07:17.933 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:07:17 smithi079 ceph-mon[35312]: from='client.? 172.21.15.22:0/2589180031' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-28T18:07:17.933 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:07:17 smithi079 ceph-mon[35312]: pgmap v445: 97 pgs: 97 active+clean; 649 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 1.7 KiB/s wr, 13 op/s 2024-03-28T18:07:17.933 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:07:17 smithi079 ceph-mon[35312]: from='client.? 172.21.15.22:0/4105694191' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-28T18:07:17.933 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:07:17 smithi079 ceph-mon[35312]: from='client.? 172.21.15.22:0/3955163630' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-28T18:07:17.946 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:17.946+0000 7f5197600640 1 -- 172.21.15.22:0/3629167582 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f519002d070 con 0x7f519006d190 2024-03-28T18:07:17.947 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:17.947+0000 7f518d600640 1 -- 172.21.15.22:0/3629167582 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v92) v1 ==== 100+0+11 (secure 0 0 0) 0x7f518409bc70 con 0x7f519006d190 2024-03-28T18:07:17.947 INFO:teuthology.orchestra.run.smithi022.stdout:pg_num: 64 2024-03-28T18:07:17.950 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:17.950+0000 7f5197600640 1 -- 172.21.15.22:0/3629167582 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7f51780a2800 msgr2=0x7f51780a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:07:17.950 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:17.950+0000 7f5197600640 1 --2- 172.21.15.22:0/3629167582 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7f51780a2800 0x7f51780a4cc0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f51901a7d90 tx=0x7f517c00b390 comp rx=0 tx=0).stop 2024-03-28T18:07:17.950 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:17.950+0000 7f5197600640 1 -- 172.21.15.22:0/3629167582 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f519006d190 msgr2=0x7f51901a68d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:07:17.950 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:17.950+0000 7f5197600640 1 --2- 172.21.15.22:0/3629167582 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f519006d190 0x7f51901a68d0 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f518400e9f0 tx=0x7f518400eeb0 comp rx=0 tx=0).stop 2024-03-28T18:07:17.950 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:17.950+0000 7f5197600640 1 -- 172.21.15.22:0/3629167582 shutdown_connections 2024-03-28T18:07:17.950 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:17.950+0000 7f5197600640 1 --2- 172.21.15.22:0/3629167582 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7f51780a2800 0x7f51780a4cc0 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:07:17.950 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:17.950+0000 7f5197600640 1 --2- 172.21.15.22:0/3629167582 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f519006ec10 0x7f51901ade30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:07:17.950 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:17.950+0000 7f5197600640 1 --2- 172.21.15.22:0/3629167582 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f519006db30 0x7f51901a6e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:07:17.951 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:17.950+0000 7f5197600640 1 --2- 172.21.15.22:0/3629167582 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f519006d190 0x7f51901a68d0 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:07:17.951 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:17.950+0000 7f5197600640 1 -- 172.21.15.22:0/3629167582 >> 172.21.15.22:0/3629167582 conn(0x7f519006c0a0 msgr2=0x7f519010a600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:07:17.951 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:17.950+0000 7f5197600640 1 -- 172.21.15.22:0/3629167582 shutdown_connections 2024-03-28T18:07:17.951 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:17.950+0000 7f5197600640 1 -- 172.21.15.22:0/3629167582 wait complete. 2024-03-28T18:07:17.968 INFO:tasks.fwd_scrub:Wait for all MDSs to reach steady state... 2024-03-28T18:07:17.968 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-28T18:07:18.163 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:18.163+0000 7fd9c2c00640 1 -- 172.21.15.22:0/4021298678 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd9bc0fff50 msgr2=0x7fd9bc100370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:07:18.163 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:18.163+0000 7fd9c2c00640 1 --2- 172.21.15.22:0/4021298678 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd9bc0fff50 0x7fd9bc100370 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7fd9ac009900 tx=0x7fd9ac02bc30 comp rx=0 tx=0).stop 2024-03-28T18:07:18.164 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:18.163+0000 7fd9c2c00640 1 -- 172.21.15.22:0/4021298678 shutdown_connections 2024-03-28T18:07:18.164 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:18.163+0000 7fd9c2c00640 1 --2- 172.21.15.22:0/4021298678 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd9bc101250 0x7fd9bc1016d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:07:18.164 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:18.163+0000 7fd9c2c00640 1 --2- 172.21.15.22:0/4021298678 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd9bc1008b0 0x7fd9bc100d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:07:18.164 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:18.163+0000 7fd9c2c00640 1 --2- 172.21.15.22:0/4021298678 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd9bc0fff50 0x7fd9bc100370 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:07:18.164 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:18.163+0000 7fd9c2c00640 1 -- 172.21.15.22:0/4021298678 >> 172.21.15.22:0/4021298678 conn(0x7fd9bc0fbb50 msgr2=0x7fd9bc0fdf90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:07:18.164 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:18.163+0000 7fd9c2c00640 1 -- 172.21.15.22:0/4021298678 shutdown_connections 2024-03-28T18:07:18.164 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:18.163+0000 7fd9c2c00640 1 -- 172.21.15.22:0/4021298678 wait complete. 2024-03-28T18:07:18.166 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:18.164+0000 7fd9c2c00640 1 Processor -- start 2024-03-28T18:07:18.166 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:18.165+0000 7fd9c2c00640 1 -- start start 2024-03-28T18:07:18.166 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:18.165+0000 7fd9c2c00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd9bc0fff50 0x7fd9bc199bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:07:18.166 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:18.165+0000 7fd9c2c00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd9bc1008b0 0x7fd9bc19a110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:07:18.166 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:18.165+0000 7fd9c2c00640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd9bc101250 0x7fd9bc1a1180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:07:18.166 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:18.165+0000 7fd9c2c00640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9bc10f5a0 con 0x7fd9bc0fff50 2024-03-28T18:07:18.166 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:18.165+0000 7fd9c2c00640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9bc10f420 con 0x7fd9bc101250 2024-03-28T18:07:18.166 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:18.165+0000 7fd9c2c00640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9bc10f720 con 0x7fd9bc1008b0 2024-03-28T18:07:18.166 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:18.165+0000 7fd9bbe00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd9bc0fff50 0x7fd9bc199bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:07:18.166 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:18.165+0000 7fd9bbe00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd9bc0fff50 0x7fd9bc199bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:55412/0 (socket says 172.21.15.22:55412) 2024-03-28T18:07:18.166 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:18.165+0000 7fd9bbe00640 1 -- 172.21.15.22:0/2598948896 learned_addr learned my addr 172.21.15.22:0/2598948896 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-28T18:07:18.166 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:18.165+0000 7fd9c0c00640 1 --2- 172.21.15.22:0/2598948896 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd9bc101250 0x7fd9bc1a1180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:07:18.167 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:18.165+0000 7fd9bb400640 1 --2- 172.21.15.22:0/2598948896 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd9bc1008b0 0x7fd9bc19a110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:07:18.167 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:18.166+0000 7fd9bbe00640 1 -- 172.21.15.22:0/2598948896 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd9bc1008b0 msgr2=0x7fd9bc19a110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:07:18.167 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:18.166+0000 7fd9bbe00640 1 --2- 172.21.15.22:0/2598948896 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd9bc1008b0 0x7fd9bc19a110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:07:18.167 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:18.166+0000 7fd9bbe00640 1 -- 172.21.15.22:0/2598948896 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd9bc101250 msgr2=0x7fd9bc1a1180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:07:18.167 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:18.166+0000 7fd9bbe00640 1 --2- 172.21.15.22:0/2598948896 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd9bc101250 0x7fd9bc1a1180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:07:18.167 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:18.166+0000 7fd9bbe00640 1 -- 172.21.15.22:0/2598948896 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd9bc1a17f0 con 0x7fd9bc0fff50 2024-03-28T18:07:18.167 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:18.166+0000 7fd9bbe00640 1 --2- 172.21.15.22:0/2598948896 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd9bc0fff50 0x7fd9bc199bd0 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7fd9ac000c00 tx=0x7fd9ac004e20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:07:18.168 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:18.166+0000 7fd9b8c00640 1 -- 172.21.15.22:0/2598948896 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9ac043020 con 0x7fd9bc0fff50 2024-03-28T18:07:18.168 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:18.166+0000 7fd9b8c00640 1 -- 172.21.15.22:0/2598948896 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7fd9ac005460 con 0x7fd9bc0fff50 2024-03-28T18:07:18.168 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:18.167+0000 7fd9b8c00640 1 -- 172.21.15.22:0/2598948896 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9ac003d00 con 0x7fd9bc0fff50 2024-03-28T18:07:18.168 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:18.167+0000 7fd9c2c00640 1 -- 172.21.15.22:0/2598948896 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd9bc1a1a80 con 0x7fd9bc0fff50 2024-03-28T18:07:18.168 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:18.167+0000 7fd9c2c00640 1 -- 172.21.15.22:0/2598948896 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd9bc1a2020 con 0x7fd9bc0fff50 2024-03-28T18:07:18.168 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:18.168+0000 7fd9c2c00640 1 -- 172.21.15.22:0/2598948896 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd9bc10ae40 con 0x7fd9bc0fff50 2024-03-28T18:07:18.169 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:18.170+0000 7fd9b8c00640 1 -- 172.21.15.22:0/2598948896 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 18) v1 ==== 137290+0+0 (secure 0 0 0) 0x7fd9ac046020 con 0x7fd9bc0fff50 2024-03-28T18:07:18.171 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:18.171+0000 7fd9b8c00640 1 --2- 172.21.15.22:0/2598948896 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7fd9a00a2880 0x7fd9a00a4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:07:18.171 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:18.172+0000 7fd9b8c00640 1 -- 172.21.15.22:0/2598948896 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 8153+0+0 (secure 0 0 0) 0x7fd9ac0f0f40 con 0x7fd9bc0fff50 2024-03-28T18:07:18.171 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:18.172+0000 7fd9bb400640 1 --2- 172.21.15.22:0/2598948896 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7fd9a00a2880 0x7fd9a00a4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:07:18.172 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:18.173+0000 7fd9bb400640 1 --2- 172.21.15.22:0/2598948896 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7fd9a00a2880 0x7fd9a00a4d40 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fd9bc19b0e0 tx=0x7fd9a8005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:07:18.174 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:18.175+0000 7fd9b8c00640 1 -- 172.21.15.22:0/2598948896 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fd9ac04d020 con 0x7fd9bc0fff50 2024-03-28T18:07:18.458 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:18.458+0000 7fd9c2c00640 1 -- 172.21.15.22:0/2598948896 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd9bc02d2c0 con 0x7fd9bc0fff50 2024-03-28T18:07:18.460 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:18.460+0000 7fd9b8c00640 1 -- 172.21.15.22:0/2598948896 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) v1 ==== 94+0+11452 (secure 0 0 0) 0x7fd9ac0be080 con 0x7fd9bc0fff50 2024-03-28T18:07:18.460 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-03-28T18:07:18.460 INFO:teuthology.orchestra.run.smithi022.stdout:{"epoch":27,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14544,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6835/3718238950","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6834","nonce":3718238950},{"type":"v1","addr":"172.21.15.22:6835","nonce":3718238950}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14562,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6837/3676939386","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6836","nonce":3676939386},{"type":"v1","addr":"172.21.15.22:6837","nonce":3676939386}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":27,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-28T18:04:48.991622+0000","modified":"2024-03-28T18:07:13.521383+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":34553,"mds_1":34511,"mds_2":34475,"mds_3":34442,"mds_4":24502},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14640":{"gid":14640,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.22:6839/3979998310","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6838","nonce":3979998310},{"type":"v1","addr":"172.21.15.22:6839","nonce":3979998310}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_14655":{"gid":14655,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.22:6841/3069056661","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6840","nonce":3069056661},{"type":"v1","addr":"172.21.15.22:6841","nonce":3069056661}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24412":{"gid":24412,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.134:6833/3176125362","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6832","nonce":3176125362},{"type":"v1","addr":"172.21.15.134:6833","nonce":3176125362}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24463":{"gid":24463,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.134:6835/3370549207","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6834","nonce":3370549207},{"type":"v1","addr":"172.21.15.134:6835","nonce":3370549207}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24478":{"gid":24478,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.134:6837/899825581","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6836","nonce":899825581},{"type":"v1","addr":"172.21.15.134:6837","nonce":899825581}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24502":{"gid":24502,"name":"i","rank":4,"incarnation":24,"state":"up:active","state_seq":7,"addr":"172.21.15.134:6839/3400263217","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6838","nonce":3400263217},{"type":"v1","addr":"172.21.15.134:6839","nonce":3400263217}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_34442":{"gid":34442,"name":"k","rank":3,"incarnation":22,"state":"up:active","state_seq":14,"addr":"172.21.15.79:6833/856896997","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6832","nonce":856896997},{"type":"v1","addr":"172.21.15.79:6833","nonce":856896997}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_34475":{"gid":34475,"name":"e","rank":2,"incarnation":20,"state":"up:active","state_seq":11,"addr":"172.21.15.79:6835/3365773431","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6834","nonce":3365773431},{"type":"v1","addr":"172.21.15.79:6835","nonce":3365773431}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_34511":{"gid":34511,"name":"h","rank":1,"incarnation":18,"state":"up:active","state_seq":8,"addr":"172.21.15.79:6837/1787903366","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6836","nonce":1787903366},{"type":"v1","addr":"172.21.15.79:6837","nonce":1787903366}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_34553":{"gid":34553,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.79:6839/2441542443","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6838","nonce":2441542443},{"type":"v1","addr":"172.21.15.79:6839","nonce":2441542443}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":34553,"qdb_cluster":[24502,34442,34475,34511,34553]},"id":1}]} 2024-03-28T18:07:18.460 INFO:teuthology.orchestra.run.smithi022.stderr:dumped fsmap epoch 27 2024-03-28T18:07:18.463 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:18.463+0000 7fd9c2c00640 1 -- 172.21.15.22:0/2598948896 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7fd9a00a2880 msgr2=0x7fd9a00a4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:07:18.463 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:18.463+0000 7fd9c2c00640 1 --2- 172.21.15.22:0/2598948896 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7fd9a00a2880 0x7fd9a00a4d40 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fd9bc19b0e0 tx=0x7fd9a8005f90 comp rx=0 tx=0).stop 2024-03-28T18:07:18.463 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:18.463+0000 7fd9c2c00640 1 -- 172.21.15.22:0/2598948896 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd9bc0fff50 msgr2=0x7fd9bc199bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:07:18.463 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:18.463+0000 7fd9c2c00640 1 --2- 172.21.15.22:0/2598948896 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd9bc0fff50 0x7fd9bc199bd0 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7fd9ac000c00 tx=0x7fd9ac004e20 comp rx=0 tx=0).stop 2024-03-28T18:07:18.463 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:18.463+0000 7fd9c2c00640 1 -- 172.21.15.22:0/2598948896 shutdown_connections 2024-03-28T18:07:18.463 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:18.463+0000 7fd9c2c00640 1 --2- 172.21.15.22:0/2598948896 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7fd9a00a2880 0x7fd9a00a4d40 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:07:18.463 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:18.463+0000 7fd9c2c00640 1 --2- 172.21.15.22:0/2598948896 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd9bc101250 0x7fd9bc1a1180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:07:18.463 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:18.463+0000 7fd9c2c00640 1 --2- 172.21.15.22:0/2598948896 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd9bc1008b0 0x7fd9bc19a110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:07:18.463 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:18.463+0000 7fd9c2c00640 1 --2- 172.21.15.22:0/2598948896 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd9bc0fff50 0x7fd9bc199bd0 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:07:18.463 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:18.463+0000 7fd9c2c00640 1 -- 172.21.15.22:0/2598948896 >> 172.21.15.22:0/2598948896 conn(0x7fd9bc0fbb50 msgr2=0x7fd9bc0fd240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:07:18.463 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:18.463+0000 7fd9c2c00640 1 -- 172.21.15.22:0/2598948896 shutdown_connections 2024-03-28T18:07:18.464 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:18.463+0000 7fd9c2c00640 1 -- 172.21.15.22:0/2598948896 wait complete. 2024-03-28T18:07:18.480 INFO:tasks.fwd_scrub:Ready to start scrub thrashing 2024-03-28T18:07:18.480 INFO:tasks.fwd_scrub.ceph_manager:waiting for clean 2024-03-28T18:07:18.480 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-28T18:07:18.694 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:18.693+0000 7f6fbf400640 1 -- 172.21.15.22:0/2408344516 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6fb8104ad0 msgr2=0x7f6fb8104f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:07:18.694 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:18.693+0000 7f6fbf400640 1 --2- 172.21.15.22:0/2408344516 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6fb8104ad0 0x7f6fb8104f50 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f6fa4009900 tx=0x7f6fa402bc30 comp rx=0 tx=0).stop 2024-03-28T18:07:18.694 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:18.694+0000 7f6fbf400640 1 -- 172.21.15.22:0/2408344516 shutdown_connections 2024-03-28T18:07:18.694 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:18.694+0000 7f6fbf400640 1 --2- 172.21.15.22:0/2408344516 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f6fb8105490 0x7f6fb810bd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:07:18.694 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:18.694+0000 7f6fbf400640 1 --2- 172.21.15.22:0/2408344516 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6fb8104ad0 0x7f6fb8104f50 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:07:18.694 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:18.694+0000 7f6fbf400640 1 --2- 172.21.15.22:0/2408344516 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6fb81038e0 0x7f6fb8103d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:07:18.694 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:18.694+0000 7f6fbf400640 1 -- 172.21.15.22:0/2408344516 >> 172.21.15.22:0/2408344516 conn(0x7f6fb80fee50 msgr2=0x7f6fb8101290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:07:18.694 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:18.694+0000 7f6fbf400640 1 -- 172.21.15.22:0/2408344516 shutdown_connections 2024-03-28T18:07:18.694 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:18.694+0000 7f6fbf400640 1 -- 172.21.15.22:0/2408344516 wait complete. 2024-03-28T18:07:18.696 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:18.695+0000 7f6fbf400640 1 Processor -- start 2024-03-28T18:07:18.696 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:18.695+0000 7f6fbf400640 1 -- start start 2024-03-28T18:07:18.696 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:18.695+0000 7f6fbf400640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6fb81038e0 0x7f6fb8199bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:07:18.696 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:18.695+0000 7f6fbf400640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6fb8104ad0 0x7f6fb819a0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:07:18.696 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:18.695+0000 7f6fbf400640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f6fb8105490 0x7f6fb81a1160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:07:18.696 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:18.695+0000 7f6fbf400640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6fb810f590 con 0x7f6fb81038e0 2024-03-28T18:07:18.696 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:18.695+0000 7f6fbf400640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6fb810f410 con 0x7f6fb8105490 2024-03-28T18:07:18.696 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:18.695+0000 7f6fbf400640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6fb810f710 con 0x7f6fb8104ad0 2024-03-28T18:07:18.696 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:18.696+0000 7f6fbca00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6fb81038e0 0x7f6fb8199bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:07:18.697 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:18.696+0000 7f6fb7e00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6fb8104ad0 0x7f6fb819a0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:07:18.697 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:18.696+0000 7f6fb7e00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6fb8104ad0 0x7f6fb819a0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.22:46954/0 (socket says 172.21.15.22:46954) 2024-03-28T18:07:18.697 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:18.696+0000 7f6fb7e00640 1 -- 172.21.15.22:0/2985747119 learned_addr learned my addr 172.21.15.22:0/2985747119 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-28T18:07:18.697 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:18.696+0000 7f6fbca00640 1 -- 172.21.15.22:0/2985747119 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6fb8104ad0 msgr2=0x7f6fb819a0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:07:18.697 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:18.696+0000 7f6fbca00640 1 --2- 172.21.15.22:0/2985747119 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6fb8104ad0 0x7f6fb819a0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:07:18.697 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:18.696+0000 7f6fbca00640 1 -- 172.21.15.22:0/2985747119 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f6fb8105490 msgr2=0x7f6fb81a1160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:07:18.697 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:18.696+0000 7f6fbca00640 1 --2- 172.21.15.22:0/2985747119 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f6fb8105490 0x7f6fb81a1160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:07:18.697 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:18.696+0000 7f6fbca00640 1 -- 172.21.15.22:0/2985747119 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6fb81a17d0 con 0x7f6fb81038e0 2024-03-28T18:07:18.697 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:18.696+0000 7f6fb7e00640 1 --2- 172.21.15.22:0/2985747119 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6fb8104ad0 0x7f6fb819a0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-28T18:07:18.697 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:18.697+0000 7f6fbca00640 1 --2- 172.21.15.22:0/2985747119 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6fb81038e0 0x7f6fb8199bb0 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f6fa8005fd0 tx=0x7f6fa800ef90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:07:18.699 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:18.699+0000 7f6fb5600640 1 -- 172.21.15.22:0/2985747119 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6fa8003970 con 0x7f6fb81038e0 2024-03-28T18:07:18.699 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:18.699+0000 7f6fb5600640 1 -- 172.21.15.22:0/2985747119 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f6fa8004030 con 0x7f6fb81038e0 2024-03-28T18:07:18.699 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:18.699+0000 7f6fb5600640 1 -- 172.21.15.22:0/2985747119 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6fa8010760 con 0x7f6fb81038e0 2024-03-28T18:07:18.699 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:18.699+0000 7f6fbf400640 1 -- 172.21.15.22:0/2985747119 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6fb81a1ac0 con 0x7f6fb81038e0 2024-03-28T18:07:18.699 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:18.700+0000 7f6fbf400640 1 -- 172.21.15.22:0/2985747119 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6fb81a1fc0 con 0x7f6fb81038e0 2024-03-28T18:07:18.701 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:18.701+0000 7f6fbf400640 1 -- 172.21.15.22:0/2985747119 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6fb810ac00 con 0x7f6fb81038e0 2024-03-28T18:07:18.702 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:18.703+0000 7f6fb5600640 1 -- 172.21.15.22:0/2985747119 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 18) v1 ==== 137290+0+0 (secure 0 0 0) 0x7f6fa80041e0 con 0x7f6fb81038e0 2024-03-28T18:07:18.704 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:18.704+0000 7f6fb5600640 1 --2- 172.21.15.22:0/2985747119 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7f6f9c0a2880 0x7f6f9c0a4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:07:18.704 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:18.705+0000 7f6fb5600640 1 -- 172.21.15.22:0/2985747119 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 8153+0+0 (secure 0 0 0) 0x7f6fa80cef80 con 0x7f6fb81038e0 2024-03-28T18:07:18.704 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:18.705+0000 7f6fb7e00640 1 --2- 172.21.15.22:0/2985747119 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7f6f9c0a2880 0x7f6f9c0a4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:07:18.705 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:18.706+0000 7f6fb7e00640 1 --2- 172.21.15.22:0/2985747119 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7f6f9c0a2880 0x7f6f9c0a4d40 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f6fa402c710 tx=0x7f6fa402c820 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:07:18.709 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:18.710+0000 7f6fb5600640 1 -- 172.21.15.22:0/2985747119 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f6fa809c020 con 0x7f6fb81038e0 2024-03-28T18:07:18.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:07:18 smithi022 ceph-mon[31985]: from='client.? 172.21.15.22:0/3629167582' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-28T18:07:18.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:07:18 smithi022 ceph-mon[31985]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T18:07:18.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:07:18 smithi022 ceph-mon[31985]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T18:07:18.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:07:18 smithi022 ceph-mon[31985]: from='client.? 172.21.15.22:0/2598948896' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-28T18:07:18.913 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:18.913+0000 7f6fbf400640 1 -- 172.21.15.22:0/2985747119 --> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f6fb8061ae0 con 0x7f6f9c0a2880 2024-03-28T18:07:18.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:07:18 smithi134 ceph-mon[35037]: from='client.? 172.21.15.22:0/3629167582' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-28T18:07:18.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:07:18 smithi134 ceph-mon[35037]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T18:07:18.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:07:18 smithi134 ceph-mon[35037]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T18:07:18.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:07:18 smithi134 ceph-mon[35037]: from='client.? 172.21.15.22:0/2598948896' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-28T18:07:18.926 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:18.926+0000 7f6fb5600640 1 -- 172.21.15.22:0/2985747119 <== mgr.14152 v2:172.21.15.22:6800/4201048392 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+258984 (secure 0 0 0) 0x7f6fb8061ae0 con 0x7f6f9c0a2880 2024-03-28T18:07:18.930 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-03-28T18:07:18.931 INFO:teuthology.orchestra.run.smithi022.stderr:dumped all 2024-03-28T18:07:18.933 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:18.932+0000 7f6fbf400640 1 -- 172.21.15.22:0/2985747119 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7f6f9c0a2880 msgr2=0x7f6f9c0a4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:07:18.933 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:18.932+0000 7f6fbf400640 1 --2- 172.21.15.22:0/2985747119 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7f6f9c0a2880 0x7f6f9c0a4d40 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f6fa402c710 tx=0x7f6fa402c820 comp rx=0 tx=0).stop 2024-03-28T18:07:18.933 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:18.932+0000 7f6fbf400640 1 -- 172.21.15.22:0/2985747119 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6fb81038e0 msgr2=0x7f6fb8199bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:07:18.933 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:18.932+0000 7f6fbf400640 1 --2- 172.21.15.22:0/2985747119 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6fb81038e0 0x7f6fb8199bb0 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f6fa8005fd0 tx=0x7f6fa800ef90 comp rx=0 tx=0).stop 2024-03-28T18:07:18.933 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:18.932+0000 7f6fbf400640 1 -- 172.21.15.22:0/2985747119 shutdown_connections 2024-03-28T18:07:18.933 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:18.932+0000 7f6fbf400640 1 --2- 172.21.15.22:0/2985747119 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7f6f9c0a2880 0x7f6f9c0a4d40 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:07:18.933 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:18.932+0000 7f6fbf400640 1 --2- 172.21.15.22:0/2985747119 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f6fb8105490 0x7f6fb81a1160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:07:18.933 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:18.932+0000 7f6fbf400640 1 --2- 172.21.15.22:0/2985747119 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6fb8104ad0 0x7f6fb819a0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:07:18.933 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:18.932+0000 7f6fbf400640 1 --2- 172.21.15.22:0/2985747119 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6fb81038e0 0x7f6fb8199bb0 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:07:18.933 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:18.932+0000 7f6fbf400640 1 -- 172.21.15.22:0/2985747119 >> 172.21.15.22:0/2985747119 conn(0x7f6fb80fee50 msgr2=0x7f6fb8107dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:07:18.933 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:18.932+0000 7f6fbf400640 1 -- 172.21.15.22:0/2985747119 shutdown_connections 2024-03-28T18:07:18.933 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:18.932+0000 7f6fbf400640 1 -- 172.21.15.22:0/2985747119 wait complete. 2024-03-28T18:07:18.934 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:07:18 smithi079 ceph-mon[35312]: from='client.? 172.21.15.22:0/3629167582' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-28T18:07:18.934 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:07:18 smithi079 ceph-mon[35312]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T18:07:18.934 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:07:18 smithi079 ceph-mon[35312]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T18:07:18.934 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:07:18 smithi079 ceph-mon[35312]: from='client.? 172.21.15.22:0/2598948896' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-28T18:07:18.952 INFO:teuthology.orchestra.run.smithi022.stdout:{"pg_ready":true,"pg_map":{"version":446,"stamp":"2024-03-28T18:07:18.662318+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":664807,"num_objects":102,"num_object_clones":0,"num_object_copies":306,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":1572,"num_read_kb":916,"num_write":450,"num_write_kb":3099,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13,"num_bytes_recovered":1180853,"num_keys_recovered":10,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":868,"ondisk_log_size":868,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":404740,"kb_used_data":13988,"kb_used_omap":18,"kb_used_meta":390637,"kb_avail":1124537084,"statfs":{"total":1151940427776,"available":1151525974016,"internally_reserved":0,"allocated":14323712,"data_stored":7781991,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19141,"internal_metadata":400012603},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":6779,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":116,"num_read_kb":60,"num_write":21,"num_write_kb":20,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004700"},"pg_stats":[{"pgid":"3.3f","version":"92'8","reported_seq":26,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-28T18:06:10.622027+0000","last_change":"2024-03-28T18:04:49.545022+0000","last_active":"2024-03-28T18:06:10.622027+0000","last_peered":"2024-03-28T18:06:10.622027+0000","last_clean":"2024-03-28T18:06:10.622027+0000","last_became_active":"2024-03-28T18:04:49.544584+0000","last_became_peered":"2024-03-28T18:04:49.544584+0000","last_unstale":"2024-03-28T18:06:10.622027+0000","last_undegraded":"2024-03-28T18:06:10.622027+0000","last_fullsized":"2024-03-28T18:06:10.622027+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_clean_scrub_stamp":"2024-03-28T18:04:48.465760+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T01:47:05.644293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"92'4","reported_seq":22,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-28T18:06:10.615631+0000","last_change":"2024-03-28T18:04:49.526720+0000","last_active":"2024-03-28T18:06:10.615631+0000","last_peered":"2024-03-28T18:06:10.615631+0000","last_clean":"2024-03-28T18:06:10.615631+0000","last_became_active":"2024-03-28T18:04:49.524545+0000","last_became_peered":"2024-03-28T18:04:49.524545+0000","last_unstale":"2024-03-28T18:06:10.615631+0000","last_undegraded":"2024-03-28T18:06:10.615631+0000","last_fullsized":"2024-03-28T18:06:10.615631+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_clean_scrub_stamp":"2024-03-28T18:04:48.465760+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T01:00:54.926308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"92'7","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-28T18:06:10.625795+0000","last_change":"2024-03-28T18:04:49.532425+0000","last_active":"2024-03-28T18:06:10.625795+0000","last_peered":"2024-03-28T18:06:10.625795+0000","last_clean":"2024-03-28T18:06:10.625795+0000","last_became_active":"2024-03-28T18:04:49.531146+0000","last_became_peered":"2024-03-28T18:04:49.531146+0000","last_unstale":"2024-03-28T18:06:10.625795+0000","last_undegraded":"2024-03-28T18:06:10.625795+0000","last_fullsized":"2024-03-28T18:06:10.625795+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_clean_scrub_stamp":"2024-03-28T18:04:48.465760+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T04:58:45.131773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"92'8","reported_seq":26,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-28T18:06:10.609313+0000","last_change":"2024-03-28T18:04:49.533371+0000","last_active":"2024-03-28T18:06:10.609313+0000","last_peered":"2024-03-28T18:06:10.609313+0000","last_clean":"2024-03-28T18:06:10.609313+0000","last_became_active":"2024-03-28T18:04:49.532544+0000","last_became_peered":"2024-03-28T18:04:49.532544+0000","last_unstale":"2024-03-28T18:06:10.609313+0000","last_undegraded":"2024-03-28T18:06:10.609313+0000","last_fullsized":"2024-03-28T18:06:10.609313+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_clean_scrub_stamp":"2024-03-28T18:04:48.465760+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T20:36:25.664137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"92'4","reported_seq":21,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-28T18:06:10.592721+0000","last_change":"2024-03-28T18:04:49.530875+0000","last_active":"2024-03-28T18:06:10.592721+0000","last_peered":"2024-03-28T18:06:10.592721+0000","last_clean":"2024-03-28T18:06:10.592721+0000","last_became_active":"2024-03-28T18:04:49.530354+0000","last_became_peered":"2024-03-28T18:04:49.530354+0000","last_unstale":"2024-03-28T18:06:10.592721+0000","last_undegraded":"2024-03-28T18:06:10.592721+0000","last_fullsized":"2024-03-28T18:06:10.592721+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_clean_scrub_stamp":"2024-03-28T18:04:48.465760+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T21:02:17.187989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"92'6","reported_seq":23,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-28T18:06:10.587630+0000","last_change":"2024-03-28T18:04:49.538260+0000","last_active":"2024-03-28T18:06:10.587630+0000","last_peered":"2024-03-28T18:06:10.587630+0000","last_clean":"2024-03-28T18:06:10.587630+0000","last_became_active":"2024-03-28T18:04:49.537863+0000","last_became_peered":"2024-03-28T18:04:49.537863+0000","last_unstale":"2024-03-28T18:06:10.587630+0000","last_undegraded":"2024-03-28T18:06:10.587630+0000","last_fullsized":"2024-03-28T18:06:10.587630+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_clean_scrub_stamp":"2024-03-28T18:04:48.465760+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T03:01:25.363839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"92'5","reported_seq":22,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-28T18:06:10.612305+0000","last_change":"2024-03-28T18:04:49.534636+0000","last_active":"2024-03-28T18:06:10.612305+0000","last_peered":"2024-03-28T18:06:10.612305+0000","last_clean":"2024-03-28T18:06:10.612305+0000","last_became_active":"2024-03-28T18:04:49.534236+0000","last_became_peered":"2024-03-28T18:04:49.534236+0000","last_unstale":"2024-03-28T18:06:10.612305+0000","last_undegraded":"2024-03-28T18:06:10.612305+0000","last_fullsized":"2024-03-28T18:06:10.612305+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_clean_scrub_stamp":"2024-03-28T18:04:48.465760+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T05:55:23.616641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"92'9","reported_seq":26,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-28T18:06:10.605001+0000","last_change":"2024-03-28T18:04:49.534848+0000","last_active":"2024-03-28T18:06:10.605001+0000","last_peered":"2024-03-28T18:06:10.605001+0000","last_clean":"2024-03-28T18:06:10.605001+0000","last_became_active":"2024-03-28T18:04:49.534302+0000","last_became_peered":"2024-03-28T18:04:49.534302+0000","last_unstale":"2024-03-28T18:06:10.605001+0000","last_undegraded":"2024-03-28T18:06:10.605001+0000","last_fullsized":"2024-03-28T18:06:10.605001+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_clean_scrub_stamp":"2024-03-28T18:04:48.465760+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T05:48:24.707305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"92'11","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-28T18:06:10.622540+0000","last_change":"2024-03-28T18:04:49.536568+0000","last_active":"2024-03-28T18:06:10.622540+0000","last_peered":"2024-03-28T18:06:10.622540+0000","last_clean":"2024-03-28T18:06:10.622540+0000","last_became_active":"2024-03-28T18:04:49.535675+0000","last_became_peered":"2024-03-28T18:04:49.535675+0000","last_unstale":"2024-03-28T18:06:10.622540+0000","last_undegraded":"2024-03-28T18:06:10.622540+0000","last_fullsized":"2024-03-28T18:06:10.622540+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_clean_scrub_stamp":"2024-03-28T18:04:48.465760+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T02:32:15.179869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"92'16","reported_seq":34,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-28T18:06:10.614038+0000","last_change":"2024-03-28T18:04:49.539363+0000","last_active":"2024-03-28T18:06:10.614038+0000","last_peered":"2024-03-28T18:06:10.614038+0000","last_clean":"2024-03-28T18:06:10.614038+0000","last_became_active":"2024-03-28T18:04:49.539002+0000","last_became_peered":"2024-03-28T18:04:49.539002+0000","last_unstale":"2024-03-28T18:06:10.614038+0000","last_undegraded":"2024-03-28T18:06:10.614038+0000","last_fullsized":"2024-03-28T18:06:10.614038+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_clean_scrub_stamp":"2024-03-28T18:04:48.465760+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T18:24:28.846081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"92'6","reported_seq":24,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-28T18:06:10.626273+0000","last_change":"2024-03-28T18:04:49.532385+0000","last_active":"2024-03-28T18:06:10.626273+0000","last_peered":"2024-03-28T18:06:10.626273+0000","last_clean":"2024-03-28T18:06:10.626273+0000","last_became_active":"2024-03-28T18:04:49.532004+0000","last_became_peered":"2024-03-28T18:04:49.532004+0000","last_unstale":"2024-03-28T18:06:10.626273+0000","last_undegraded":"2024-03-28T18:06:10.626273+0000","last_fullsized":"2024-03-28T18:06:10.626273+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_clean_scrub_stamp":"2024-03-28T18:04:48.465760+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T19:05:13.959313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"92'7","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-28T18:06:10.608967+0000","last_change":"2024-03-28T18:04:49.534465+0000","last_active":"2024-03-28T18:06:10.608967+0000","last_peered":"2024-03-28T18:06:10.608967+0000","last_clean":"2024-03-28T18:06:10.608967+0000","last_became_active":"2024-03-28T18:04:49.532989+0000","last_became_peered":"2024-03-28T18:04:49.532989+0000","last_unstale":"2024-03-28T18:06:10.608967+0000","last_undegraded":"2024-03-28T18:06:10.608967+0000","last_fullsized":"2024-03-28T18:06:10.608967+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_clean_scrub_stamp":"2024-03-28T18:04:48.465760+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T23:56:50.607770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"92'10","reported_seq":28,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-28T18:06:10.612024+0000","last_change":"2024-03-28T18:04:49.535205+0000","last_active":"2024-03-28T18:06:10.612024+0000","last_peered":"2024-03-28T18:06:10.612024+0000","last_clean":"2024-03-28T18:06:10.612024+0000","last_became_active":"2024-03-28T18:04:49.533025+0000","last_became_peered":"2024-03-28T18:04:49.533025+0000","last_unstale":"2024-03-28T18:06:10.612024+0000","last_undegraded":"2024-03-28T18:06:10.612024+0000","last_fullsized":"2024-03-28T18:06:10.612024+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_clean_scrub_stamp":"2024-03-28T18:04:48.465760+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T21:41:51.815907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"92'7","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-28T18:06:10.615451+0000","last_change":"2024-03-28T18:04:49.520974+0000","last_active":"2024-03-28T18:06:10.615451+0000","last_peered":"2024-03-28T18:06:10.615451+0000","last_clean":"2024-03-28T18:06:10.615451+0000","last_became_active":"2024-03-28T18:04:49.520509+0000","last_became_peered":"2024-03-28T18:04:49.520509+0000","last_unstale":"2024-03-28T18:06:10.615451+0000","last_undegraded":"2024-03-28T18:06:10.615451+0000","last_fullsized":"2024-03-28T18:06:10.615451+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_clean_scrub_stamp":"2024-03-28T18:04:48.465760+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T03:15:16.346859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"92'7","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-28T18:06:10.608830+0000","last_change":"2024-03-28T18:04:49.539053+0000","last_active":"2024-03-28T18:06:10.608830+0000","last_peered":"2024-03-28T18:06:10.608830+0000","last_clean":"2024-03-28T18:06:10.608830+0000","last_became_active":"2024-03-28T18:04:49.537327+0000","last_became_peered":"2024-03-28T18:04:49.537327+0000","last_unstale":"2024-03-28T18:06:10.608830+0000","last_undegraded":"2024-03-28T18:06:10.608830+0000","last_fullsized":"2024-03-28T18:06:10.608830+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_clean_scrub_stamp":"2024-03-28T18:04:48.465760+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T00:24:42.319504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"92'9","reported_seq":27,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-28T18:06:10.625483+0000","last_change":"2024-03-28T18:04:49.525855+0000","last_active":"2024-03-28T18:06:10.625483+0000","last_peered":"2024-03-28T18:06:10.625483+0000","last_clean":"2024-03-28T18:06:10.625483+0000","last_became_active":"2024-03-28T18:04:49.525448+0000","last_became_peered":"2024-03-28T18:04:49.525448+0000","last_unstale":"2024-03-28T18:06:10.625483+0000","last_undegraded":"2024-03-28T18:06:10.625483+0000","last_fullsized":"2024-03-28T18:06:10.625483+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_clean_scrub_stamp":"2024-03-28T18:04:48.465760+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T23:53:00.452692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"92'8","reported_seq":26,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-28T18:06:10.622131+0000","last_change":"2024-03-28T18:04:49.543605+0000","last_active":"2024-03-28T18:06:10.622131+0000","last_peered":"2024-03-28T18:06:10.622131+0000","last_clean":"2024-03-28T18:06:10.622131+0000","last_became_active":"2024-03-28T18:04:49.543280+0000","last_became_peered":"2024-03-28T18:04:49.543280+0000","last_unstale":"2024-03-28T18:06:10.622131+0000","last_undegraded":"2024-03-28T18:06:10.622131+0000","last_fullsized":"2024-03-28T18:06:10.622131+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_clean_scrub_stamp":"2024-03-28T18:04:48.465760+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T21:27:26.216639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"92'10","reported_seq":28,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-28T18:06:10.619171+0000","last_change":"2024-03-28T18:04:49.526745+0000","last_active":"2024-03-28T18:06:10.619171+0000","last_peered":"2024-03-28T18:06:10.619171+0000","last_clean":"2024-03-28T18:06:10.619171+0000","last_became_active":"2024-03-28T18:04:49.526351+0000","last_became_peered":"2024-03-28T18:04:49.526351+0000","last_unstale":"2024-03-28T18:06:10.619171+0000","last_undegraded":"2024-03-28T18:06:10.619171+0000","last_fullsized":"2024-03-28T18:06:10.619171+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_clean_scrub_stamp":"2024-03-28T18:04:48.465760+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T05:31:47.687345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"92'7","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-28T18:06:10.596535+0000","last_change":"2024-03-28T18:04:49.523346+0000","last_active":"2024-03-28T18:06:10.596535+0000","last_peered":"2024-03-28T18:06:10.596535+0000","last_clean":"2024-03-28T18:06:10.596535+0000","last_became_active":"2024-03-28T18:04:49.522041+0000","last_became_peered":"2024-03-28T18:04:49.522041+0000","last_unstale":"2024-03-28T18:06:10.596535+0000","last_undegraded":"2024-03-28T18:06:10.596535+0000","last_fullsized":"2024-03-28T18:06:10.596535+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_clean_scrub_stamp":"2024-03-28T18:04:48.465760+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T01:35:02.419328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"92'14","reported_seq":174,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-28T18:07:13.984992+0000","last_change":"2024-03-28T18:04:48.510917+0000","last_active":"2024-03-28T18:07:13.984992+0000","last_peered":"2024-03-28T18:07:13.984992+0000","last_clean":"2024-03-28T18:07:13.984992+0000","last_became_active":"2024-03-28T18:04:48.509978+0000","last_became_peered":"2024-03-28T18:04:48.509978+0000","last_unstale":"2024-03-28T18:07:13.984992+0000","last_undegraded":"2024-03-28T18:07:13.984992+0000","last_fullsized":"2024-03-28T18:07:13.984992+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:04:47.412563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:04:47.412563+0000","last_clean_scrub_stamp":"2024-03-28T18:04:47.412563+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T03:28:13.185099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":140,"num_read_kb":140,"num_write":15,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"92'5","reported_seq":23,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-28T18:06:10.603875+0000","last_change":"2024-03-28T18:04:49.535543+0000","last_active":"2024-03-28T18:06:10.603875+0000","last_peered":"2024-03-28T18:06:10.603875+0000","last_clean":"2024-03-28T18:06:10.603875+0000","last_became_active":"2024-03-28T18:04:49.535171+0000","last_became_peered":"2024-03-28T18:04:49.535171+0000","last_unstale":"2024-03-28T18:06:10.603875+0000","last_undegraded":"2024-03-28T18:06:10.603875+0000","last_fullsized":"2024-03-28T18:06:10.603875+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_clean_scrub_stamp":"2024-03-28T18:04:48.465760+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T22:12:22.258424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"90'2","reported_seq":22,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-28T18:05:11.741109+0000","last_change":"2024-03-28T18:04:48.503190+0000","last_active":"2024-03-28T18:05:11.741109+0000","last_peered":"2024-03-28T18:05:11.741109+0000","last_clean":"2024-03-28T18:05:11.741109+0000","last_became_active":"2024-03-28T18:04:48.501858+0000","last_became_peered":"2024-03-28T18:04:48.501858+0000","last_unstale":"2024-03-28T18:05:11.741109+0000","last_undegraded":"2024-03-28T18:05:11.741109+0000","last_fullsized":"2024-03-28T18:05:11.741109+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:04:47.412563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:04:47.412563+0000","last_clean_scrub_stamp":"2024-03-28T18:04:47.412563+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T19:56:43.878630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"92'6","reported_seq":24,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-28T18:06:10.625888+0000","last_change":"2024-03-28T18:04:49.537739+0000","last_active":"2024-03-28T18:06:10.625888+0000","last_peered":"2024-03-28T18:06:10.625888+0000","last_clean":"2024-03-28T18:06:10.625888+0000","last_became_active":"2024-03-28T18:04:49.535990+0000","last_became_peered":"2024-03-28T18:04:49.535990+0000","last_unstale":"2024-03-28T18:06:10.625888+0000","last_undegraded":"2024-03-28T18:06:10.625888+0000","last_fullsized":"2024-03-28T18:06:10.625888+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_clean_scrub_stamp":"2024-03-28T18:04:48.465760+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T21:54:15.729494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"90'12","reported_seq":27,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-28T18:05:11.768035+0000","last_change":"2024-03-28T18:05:11.767512+0000","last_active":"2024-03-28T18:05:11.768035+0000","last_peered":"2024-03-28T18:05:11.768035+0000","last_clean":"2024-03-28T18:05:11.768035+0000","last_became_active":"2024-03-28T18:05:11.752452+0000","last_became_peered":"2024-03-28T18:05:11.752452+0000","last_unstale":"2024-03-28T18:05:11.768035+0000","last_undegraded":"2024-03-28T18:05:11.768035+0000","last_fullsized":"2024-03-28T18:05:11.768035+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:04:47.412563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:04:47.412563+0000","last_clean_scrub_stamp":"2024-03-28T18:04:47.412563+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T00:09:13.980288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":0,"num_keys_recovered":10,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,7],"acting":[11,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1c","version":"92'9","reported_seq":26,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-28T18:06:10.623876+0000","last_change":"2024-03-28T18:04:49.529316+0000","last_active":"2024-03-28T18:06:10.623876+0000","last_peered":"2024-03-28T18:06:10.623876+0000","last_clean":"2024-03-28T18:06:10.623876+0000","last_became_active":"2024-03-28T18:04:49.528975+0000","last_became_peered":"2024-03-28T18:04:49.528975+0000","last_unstale":"2024-03-28T18:06:10.623876+0000","last_undegraded":"2024-03-28T18:06:10.623876+0000","last_fullsized":"2024-03-28T18:06:10.623876+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_clean_scrub_stamp":"2024-03-28T18:04:48.465760+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T00:06:45.268757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"90'5","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-28T18:05:11.734761+0000","last_change":"2024-03-28T18:04:48.515841+0000","last_active":"2024-03-28T18:05:11.734761+0000","last_peered":"2024-03-28T18:05:11.734761+0000","last_clean":"2024-03-28T18:05:11.734761+0000","last_became_active":"2024-03-28T18:04:48.513852+0000","last_became_peered":"2024-03-28T18:04:48.513852+0000","last_unstale":"2024-03-28T18:05:11.734761+0000","last_undegraded":"2024-03-28T18:05:11.734761+0000","last_fullsized":"2024-03-28T18:05:11.734761+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:04:47.412563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:04:47.412563+0000","last_clean_scrub_stamp":"2024-03-28T18:04:47.412563+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T00:07:14.751545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"92'15","reported_seq":32,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-28T18:06:10.609956+0000","last_change":"2024-03-28T18:04:49.542238+0000","last_active":"2024-03-28T18:06:10.609956+0000","last_peered":"2024-03-28T18:06:10.609956+0000","last_clean":"2024-03-28T18:06:10.609956+0000","last_became_active":"2024-03-28T18:04:49.539723+0000","last_became_peered":"2024-03-28T18:04:49.539723+0000","last_unstale":"2024-03-28T18:06:10.609956+0000","last_undegraded":"2024-03-28T18:06:10.609956+0000","last_fullsized":"2024-03-28T18:06:10.609956+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_clean_scrub_stamp":"2024-03-28T18:04:48.465760+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T22:45:07.310148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"92'8","reported_seq":170,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-28T18:07:14.022275+0000","last_change":"2024-03-28T18:04:48.511349+0000","last_active":"2024-03-28T18:07:14.022275+0000","last_peered":"2024-03-28T18:07:14.022275+0000","last_clean":"2024-03-28T18:07:14.022275+0000","last_became_active":"2024-03-28T18:04:48.510990+0000","last_became_peered":"2024-03-28T18:04:48.510990+0000","last_unstale":"2024-03-28T18:07:14.022275+0000","last_undegraded":"2024-03-28T18:07:14.022275+0000","last_fullsized":"2024-03-28T18:07:14.022275+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:04:47.412563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:04:47.412563+0000","last_clean_scrub_stamp":"2024-03-28T18:04:47.412563+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T21:56:15.411149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1647,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":143,"num_read_kb":4,"num_write":5,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"92'7","reported_seq":24,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-28T18:06:10.604922+0000","last_change":"2024-03-28T18:04:49.538066+0000","last_active":"2024-03-28T18:06:10.604922+0000","last_peered":"2024-03-28T18:06:10.604922+0000","last_clean":"2024-03-28T18:06:10.604922+0000","last_became_active":"2024-03-28T18:04:49.537202+0000","last_became_peered":"2024-03-28T18:04:49.537202+0000","last_unstale":"2024-03-28T18:06:10.604922+0000","last_undegraded":"2024-03-28T18:06:10.604922+0000","last_fullsized":"2024-03-28T18:06:10.604922+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_clean_scrub_stamp":"2024-03-28T18:04:48.465760+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T22:35:19.060311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"90'7","reported_seq":26,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-28T18:05:11.732185+0000","last_change":"2024-03-28T18:04:48.513722+0000","last_active":"2024-03-28T18:05:11.732185+0000","last_peered":"2024-03-28T18:05:11.732185+0000","last_clean":"2024-03-28T18:05:11.732185+0000","last_became_active":"2024-03-28T18:04:48.513048+0000","last_became_peered":"2024-03-28T18:04:48.513048+0000","last_unstale":"2024-03-28T18:05:11.732185+0000","last_undegraded":"2024-03-28T18:05:11.732185+0000","last_fullsized":"2024-03-28T18:05:11.732185+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:04:47.412563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:04:47.412563+0000","last_clean_scrub_stamp":"2024-03-28T18:04:47.412563+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T04:48:10.831297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"92'8","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-28T18:06:10.612752+0000","last_change":"2024-03-28T18:04:49.539172+0000","last_active":"2024-03-28T18:06:10.612752+0000","last_peered":"2024-03-28T18:06:10.612752+0000","last_clean":"2024-03-28T18:06:10.612752+0000","last_became_active":"2024-03-28T18:04:49.538779+0000","last_became_peered":"2024-03-28T18:04:49.538779+0000","last_unstale":"2024-03-28T18:06:10.612752+0000","last_undegraded":"2024-03-28T18:06:10.612752+0000","last_fullsized":"2024-03-28T18:06:10.612752+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_clean_scrub_stamp":"2024-03-28T18:04:48.465760+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T04:59:08.361762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"90'3","reported_seq":23,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-28T18:05:11.950057+0000","last_change":"2024-03-28T18:04:48.500471+0000","last_active":"2024-03-28T18:05:11.950057+0000","last_peered":"2024-03-28T18:05:11.950057+0000","last_clean":"2024-03-28T18:05:11.950057+0000","last_became_active":"2024-03-28T18:04:48.500094+0000","last_became_peered":"2024-03-28T18:04:48.500094+0000","last_unstale":"2024-03-28T18:05:11.950057+0000","last_undegraded":"2024-03-28T18:05:11.950057+0000","last_fullsized":"2024-03-28T18:05:11.950057+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:04:47.412563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:04:47.412563+0000","last_clean_scrub_stamp":"2024-03-28T18:04:47.412563+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T21:52:51.964254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"92'8","reported_seq":26,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-28T18:06:10.600363+0000","last_change":"2024-03-28T18:04:49.522120+0000","last_active":"2024-03-28T18:06:10.600363+0000","last_peered":"2024-03-28T18:06:10.600363+0000","last_clean":"2024-03-28T18:06:10.600363+0000","last_became_active":"2024-03-28T18:04:49.521656+0000","last_became_peered":"2024-03-28T18:04:49.521656+0000","last_unstale":"2024-03-28T18:06:10.600363+0000","last_undegraded":"2024-03-28T18:06:10.600363+0000","last_fullsized":"2024-03-28T18:06:10.600363+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_clean_scrub_stamp":"2024-03-28T18:04:48.465760+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T18:29:55.605765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"90'3","reported_seq":24,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-28T18:05:11.741130+0000","last_change":"2024-03-28T18:04:48.508723+0000","last_active":"2024-03-28T18:05:11.741130+0000","last_peered":"2024-03-28T18:05:11.741130+0000","last_clean":"2024-03-28T18:05:11.741130+0000","last_became_active":"2024-03-28T18:04:48.508314+0000","last_became_peered":"2024-03-28T18:04:48.508314+0000","last_unstale":"2024-03-28T18:05:11.741130+0000","last_undegraded":"2024-03-28T18:05:11.741130+0000","last_fullsized":"2024-03-28T18:05:11.741130+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:04:47.412563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:04:47.412563+0000","last_clean_scrub_stamp":"2024-03-28T18:04:47.412563+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T03:03:59.353454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"92'11","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-28T18:06:10.611886+0000","last_change":"2024-03-28T18:04:49.531469+0000","last_active":"2024-03-28T18:06:10.611886+0000","last_peered":"2024-03-28T18:06:10.611886+0000","last_clean":"2024-03-28T18:06:10.611886+0000","last_became_active":"2024-03-28T18:04:49.529263+0000","last_became_peered":"2024-03-28T18:04:49.529263+0000","last_unstale":"2024-03-28T18:06:10.611886+0000","last_undegraded":"2024-03-28T18:06:10.611886+0000","last_fullsized":"2024-03-28T18:06:10.611886+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_clean_scrub_stamp":"2024-03-28T18:04:48.465760+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T05:46:31.593877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"92'7","reported_seq":305,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-28T18:07:15.023693+0000","last_change":"2024-03-28T18:04:48.500644+0000","last_active":"2024-03-28T18:07:15.023693+0000","last_peered":"2024-03-28T18:07:15.023693+0000","last_clean":"2024-03-28T18:07:15.023693+0000","last_became_active":"2024-03-28T18:04:48.499330+0000","last_became_peered":"2024-03-28T18:04:48.499330+0000","last_unstale":"2024-03-28T18:07:15.023693+0000","last_undegraded":"2024-03-28T18:07:15.023693+0000","last_fullsized":"2024-03-28T18:07:15.023693+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:04:47.412563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:04:47.412563+0000","last_clean_scrub_stamp":"2024-03-28T18:04:47.412563+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T18:50:54.361266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":278,"num_read_kb":278,"num_write":8,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"92'10","reported_seq":28,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-28T18:06:10.623432+0000","last_change":"2024-03-28T18:04:49.528728+0000","last_active":"2024-03-28T18:06:10.623432+0000","last_peered":"2024-03-28T18:06:10.623432+0000","last_clean":"2024-03-28T18:06:10.623432+0000","last_became_active":"2024-03-28T18:04:49.528312+0000","last_became_peered":"2024-03-28T18:04:49.528312+0000","last_unstale":"2024-03-28T18:06:10.623432+0000","last_undegraded":"2024-03-28T18:06:10.623432+0000","last_fullsized":"2024-03-28T18:06:10.623432+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_clean_scrub_stamp":"2024-03-28T18:04:48.465760+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T05:59:37.308002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"90'4","reported_seq":24,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-28T18:05:11.736913+0000","last_change":"2024-03-28T18:04:48.508766+0000","last_active":"2024-03-28T18:05:11.736913+0000","last_peered":"2024-03-28T18:05:11.736913+0000","last_clean":"2024-03-28T18:05:11.736913+0000","last_became_active":"2024-03-28T18:04:48.506780+0000","last_became_peered":"2024-03-28T18:04:48.506780+0000","last_unstale":"2024-03-28T18:05:11.736913+0000","last_undegraded":"2024-03-28T18:05:11.736913+0000","last_fullsized":"2024-03-28T18:05:11.736913+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:04:47.412563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:04:47.412563+0000","last_clean_scrub_stamp":"2024-03-28T18:04:47.412563+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T04:39:07.394655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"92'9","reported_seq":27,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-28T18:06:10.610898+0000","last_change":"2024-03-28T18:04:49.537165+0000","last_active":"2024-03-28T18:06:10.610898+0000","last_peered":"2024-03-28T18:06:10.610898+0000","last_clean":"2024-03-28T18:06:10.610898+0000","last_became_active":"2024-03-28T18:04:49.534543+0000","last_became_peered":"2024-03-28T18:04:49.534543+0000","last_unstale":"2024-03-28T18:06:10.610898+0000","last_undegraded":"2024-03-28T18:06:10.610898+0000","last_fullsized":"2024-03-28T18:06:10.610898+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_clean_scrub_stamp":"2024-03-28T18:04:48.465760+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T03:15:58.056148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"90'5","reported_seq":26,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-28T18:05:11.746431+0000","last_change":"2024-03-28T18:04:48.503866+0000","last_active":"2024-03-28T18:05:11.746431+0000","last_peered":"2024-03-28T18:05:11.746431+0000","last_clean":"2024-03-28T18:05:11.746431+0000","last_became_active":"2024-03-28T18:04:48.503223+0000","last_became_peered":"2024-03-28T18:04:48.503223+0000","last_unstale":"2024-03-28T18:05:11.746431+0000","last_undegraded":"2024-03-28T18:05:11.746431+0000","last_fullsized":"2024-03-28T18:05:11.746431+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:04:47.412563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:04:47.412563+0000","last_clean_scrub_stamp":"2024-03-28T18:04:47.412563+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T20:14:01.307483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"92'11","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-28T18:06:10.624750+0000","last_change":"2024-03-28T18:04:49.539494+0000","last_active":"2024-03-28T18:06:10.624750+0000","last_peered":"2024-03-28T18:06:10.624750+0000","last_clean":"2024-03-28T18:06:10.624750+0000","last_became_active":"2024-03-28T18:04:49.539044+0000","last_became_peered":"2024-03-28T18:04:49.539044+0000","last_unstale":"2024-03-28T18:06:10.624750+0000","last_undegraded":"2024-03-28T18:06:10.624750+0000","last_fullsized":"2024-03-28T18:06:10.624750+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_clean_scrub_stamp":"2024-03-28T18:04:48.465760+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T04:08:30.807788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"92'34","reported_seq":341,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-28T18:07:15.010701+0000","last_change":"2024-03-28T18:04:48.507703+0000","last_active":"2024-03-28T18:07:15.010701+0000","last_peered":"2024-03-28T18:07:15.010701+0000","last_clean":"2024-03-28T18:07:15.010701+0000","last_became_active":"2024-03-28T18:04:48.507310+0000","last_became_peered":"2024-03-28T18:04:48.507310+0000","last_unstale":"2024-03-28T18:07:15.010701+0000","last_undegraded":"2024-03-28T18:07:15.010701+0000","last_fullsized":"2024-03-28T18:07:15.010701+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:04:47.412563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:04:47.412563+0000","last_clean_scrub_stamp":"2024-03-28T18:04:47.412563+0000","objects_scrubbed":0,"log_size":34,"log_dups_size":0,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T19:14:13.910042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":64747,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":287,"num_read_kb":207,"num_write":36,"num_write_kb":77,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"92'6","reported_seq":24,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-28T18:06:10.618389+0000","last_change":"2024-03-28T18:04:49.529928+0000","last_active":"2024-03-28T18:06:10.618389+0000","last_peered":"2024-03-28T18:06:10.618389+0000","last_clean":"2024-03-28T18:06:10.618389+0000","last_became_active":"2024-03-28T18:04:49.529527+0000","last_became_peered":"2024-03-28T18:04:49.529527+0000","last_unstale":"2024-03-28T18:06:10.618389+0000","last_undegraded":"2024-03-28T18:06:10.618389+0000","last_fullsized":"2024-03-28T18:06:10.618389+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_clean_scrub_stamp":"2024-03-28T18:04:48.465760+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T05:50:07.129140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"90'9","reported_seq":31,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-28T18:05:11.749738+0000","last_change":"2024-03-28T18:04:48.494746+0000","last_active":"2024-03-28T18:05:11.749738+0000","last_peered":"2024-03-28T18:05:11.749738+0000","last_clean":"2024-03-28T18:05:11.749738+0000","last_became_active":"2024-03-28T18:04:48.494296+0000","last_became_peered":"2024-03-28T18:04:48.494296+0000","last_unstale":"2024-03-28T18:05:11.749738+0000","last_undegraded":"2024-03-28T18:05:11.749738+0000","last_fullsized":"2024-03-28T18:05:11.749738+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:04:47.412563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:04:47.412563+0000","last_clean_scrub_stamp":"2024-03-28T18:04:47.412563+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T22:32:56.923067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"92'9","reported_seq":27,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-28T18:06:10.624303+0000","last_change":"2024-03-28T18:04:49.528070+0000","last_active":"2024-03-28T18:06:10.624303+0000","last_peered":"2024-03-28T18:06:10.624303+0000","last_clean":"2024-03-28T18:06:10.624303+0000","last_became_active":"2024-03-28T18:04:49.526747+0000","last_became_peered":"2024-03-28T18:04:49.526747+0000","last_unstale":"2024-03-28T18:06:10.624303+0000","last_undegraded":"2024-03-28T18:06:10.624303+0000","last_fullsized":"2024-03-28T18:06:10.624303+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_clean_scrub_stamp":"2024-03-28T18:04:48.465760+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T22:32:05.750094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"90'5","reported_seq":26,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-28T18:05:11.741413+0000","last_change":"2024-03-28T18:04:48.505723+0000","last_active":"2024-03-28T18:05:11.741413+0000","last_peered":"2024-03-28T18:05:11.741413+0000","last_clean":"2024-03-28T18:05:11.741413+0000","last_became_active":"2024-03-28T18:04:48.504088+0000","last_became_peered":"2024-03-28T18:04:48.504088+0000","last_unstale":"2024-03-28T18:05:11.741413+0000","last_undegraded":"2024-03-28T18:05:11.741413+0000","last_fullsized":"2024-03-28T18:05:11.741413+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:04:47.412563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:04:47.412563+0000","last_clean_scrub_stamp":"2024-03-28T18:04:47.412563+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T23:25:37.138054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"92'15","reported_seq":33,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-28T18:06:10.625240+0000","last_change":"2024-03-28T18:04:49.518872+0000","last_active":"2024-03-28T18:06:10.625240+0000","last_peered":"2024-03-28T18:06:10.625240+0000","last_clean":"2024-03-28T18:06:10.625240+0000","last_became_active":"2024-03-28T18:04:49.518507+0000","last_became_peered":"2024-03-28T18:04:49.518507+0000","last_unstale":"2024-03-28T18:06:10.625240+0000","last_undegraded":"2024-03-28T18:06:10.625240+0000","last_fullsized":"2024-03-28T18:06:10.625240+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_clean_scrub_stamp":"2024-03-28T18:04:48.465760+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T20:11:31.044321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"92'8","reported_seq":171,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-28T18:07:14.021044+0000","last_change":"2024-03-28T18:04:48.503058+0000","last_active":"2024-03-28T18:07:14.021044+0000","last_peered":"2024-03-28T18:07:14.021044+0000","last_clean":"2024-03-28T18:07:14.021044+0000","last_became_active":"2024-03-28T18:04:48.501596+0000","last_became_peered":"2024-03-28T18:04:48.501596+0000","last_unstale":"2024-03-28T18:07:14.021044+0000","last_undegraded":"2024-03-28T18:07:14.021044+0000","last_fullsized":"2024-03-28T18:07:14.021044+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:04:47.412563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:04:47.412563+0000","last_clean_scrub_stamp":"2024-03-28T18:04:47.412563+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T23:46:10.442771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1613,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":144,"num_read_kb":5,"num_write":8,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"92'6","reported_seq":24,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-28T18:06:10.614449+0000","last_change":"2024-03-28T18:04:49.535102+0000","last_active":"2024-03-28T18:06:10.614449+0000","last_peered":"2024-03-28T18:06:10.614449+0000","last_clean":"2024-03-28T18:06:10.614449+0000","last_became_active":"2024-03-28T18:04:49.534171+0000","last_became_peered":"2024-03-28T18:04:49.534171+0000","last_unstale":"2024-03-28T18:06:10.614449+0000","last_undegraded":"2024-03-28T18:06:10.614449+0000","last_fullsized":"2024-03-28T18:06:10.614449+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_clean_scrub_stamp":"2024-03-28T18:04:48.465760+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T18:16:27.703327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"90'17","reported_seq":38,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-28T18:05:11.734330+0000","last_change":"2024-03-28T18:04:48.517034+0000","last_active":"2024-03-28T18:05:11.734330+0000","last_peered":"2024-03-28T18:05:11.734330+0000","last_clean":"2024-03-28T18:05:11.734330+0000","last_became_active":"2024-03-28T18:04:48.516562+0000","last_became_peered":"2024-03-28T18:04:48.516562+0000","last_unstale":"2024-03-28T18:05:11.734330+0000","last_undegraded":"2024-03-28T18:05:11.734330+0000","last_fullsized":"2024-03-28T18:05:11.734330+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:04:47.412563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:04:47.412563+0000","last_clean_scrub_stamp":"2024-03-28T18:04:47.412563+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T02:28:55.753779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"92'6","reported_seq":24,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-28T18:06:10.617799+0000","last_change":"2024-03-28T18:04:49.541501+0000","last_active":"2024-03-28T18:06:10.617799+0000","last_peered":"2024-03-28T18:06:10.617799+0000","last_clean":"2024-03-28T18:06:10.617799+0000","last_became_active":"2024-03-28T18:04:49.541117+0000","last_became_peered":"2024-03-28T18:04:49.541117+0000","last_unstale":"2024-03-28T18:06:10.617799+0000","last_undegraded":"2024-03-28T18:06:10.617799+0000","last_fullsized":"2024-03-28T18:06:10.617799+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_clean_scrub_stamp":"2024-03-28T18:04:48.465760+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T04:18:29.081489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"92'3","reported_seq":24,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-28T18:07:03.790727+0000","last_change":"2024-03-28T18:04:48.510859+0000","last_active":"2024-03-28T18:07:03.790727+0000","last_peered":"2024-03-28T18:07:03.790727+0000","last_clean":"2024-03-28T18:07:03.790727+0000","last_became_active":"2024-03-28T18:04:48.510522+0000","last_became_peered":"2024-03-28T18:04:48.510522+0000","last_unstale":"2024-03-28T18:07:03.790727+0000","last_undegraded":"2024-03-28T18:07:03.790727+0000","last_fullsized":"2024-03-28T18:07:03.790727+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:04:47.412563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:04:47.412563+0000","last_clean_scrub_stamp":"2024-03-28T18:04:47.412563+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T22:31:37.817361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"92'5","reported_seq":23,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-28T18:06:10.607186+0000","last_change":"2024-03-28T18:04:49.541981+0000","last_active":"2024-03-28T18:06:10.607186+0000","last_peered":"2024-03-28T18:06:10.607186+0000","last_clean":"2024-03-28T18:06:10.607186+0000","last_became_active":"2024-03-28T18:04:49.541606+0000","last_became_peered":"2024-03-28T18:04:49.541606+0000","last_unstale":"2024-03-28T18:06:10.607186+0000","last_undegraded":"2024-03-28T18:06:10.607186+0000","last_fullsized":"2024-03-28T18:06:10.607186+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_clean_scrub_stamp":"2024-03-28T18:04:48.465760+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T21:56:03.632490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"90'1","reported_seq":21,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-28T18:05:11.730922+0000","last_change":"2024-03-28T18:04:48.505847+0000","last_active":"2024-03-28T18:05:11.730922+0000","last_peered":"2024-03-28T18:05:11.730922+0000","last_clean":"2024-03-28T18:05:11.730922+0000","last_became_active":"2024-03-28T18:04:48.501005+0000","last_became_peered":"2024-03-28T18:04:48.501005+0000","last_unstale":"2024-03-28T18:05:11.730922+0000","last_undegraded":"2024-03-28T18:05:11.730922+0000","last_fullsized":"2024-03-28T18:05:11.730922+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:04:47.412563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:04:47.412563+0000","last_clean_scrub_stamp":"2024-03-28T18:04:47.412563+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T01:33:45.539324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"92'5","reported_seq":23,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-28T18:06:10.614333+0000","last_change":"2024-03-28T18:04:49.526938+0000","last_active":"2024-03-28T18:06:10.614333+0000","last_peered":"2024-03-28T18:06:10.614333+0000","last_clean":"2024-03-28T18:06:10.614333+0000","last_became_active":"2024-03-28T18:04:49.526290+0000","last_became_peered":"2024-03-28T18:04:49.526290+0000","last_unstale":"2024-03-28T18:06:10.614333+0000","last_undegraded":"2024-03-28T18:06:10.614333+0000","last_fullsized":"2024-03-28T18:06:10.614333+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_clean_scrub_stamp":"2024-03-28T18:04:48.465760+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T03:06:54.561470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"92'6","reported_seq":169,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-28T18:07:15.990542+0000","last_change":"2024-03-28T18:04:48.512553+0000","last_active":"2024-03-28T18:07:15.990542+0000","last_peered":"2024-03-28T18:07:15.990542+0000","last_clean":"2024-03-28T18:07:15.990542+0000","last_became_active":"2024-03-28T18:04:48.511520+0000","last_became_peered":"2024-03-28T18:04:48.511520+0000","last_unstale":"2024-03-28T18:07:15.990542+0000","last_undegraded":"2024-03-28T18:07:15.990542+0000","last_fullsized":"2024-03-28T18:07:15.990542+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:04:47.412563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:04:47.412563+0000","last_clean_scrub_stamp":"2024-03-28T18:04:47.412563+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T21:13:32.970027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1081,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":143,"num_read_kb":2,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"92'7","reported_seq":31,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-28T18:06:10.613152+0000","last_change":"2024-03-28T18:05:11.739596+0000","last_active":"2024-03-28T18:06:10.613152+0000","last_peered":"2024-03-28T18:06:10.613152+0000","last_clean":"2024-03-28T18:06:10.613152+0000","last_became_active":"2024-03-28T18:05:11.739156+0000","last_became_peered":"2024-03-28T18:05:11.739156+0000","last_unstale":"2024-03-28T18:06:10.613152+0000","last_undegraded":"2024-03-28T18:06:10.613152+0000","last_fullsized":"2024-03-28T18:06:10.613152+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_clean_scrub_stamp":"2024-03-28T18:04:48.465760+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T05:23:00.446713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,0],"acting":[10,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"92'8","reported_seq":26,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-28T18:06:10.609498+0000","last_change":"2024-03-28T18:04:49.540438+0000","last_active":"2024-03-28T18:06:10.609498+0000","last_peered":"2024-03-28T18:06:10.609498+0000","last_clean":"2024-03-28T18:06:10.609498+0000","last_became_active":"2024-03-28T18:04:49.540069+0000","last_became_peered":"2024-03-28T18:04:49.540069+0000","last_unstale":"2024-03-28T18:06:10.609498+0000","last_undegraded":"2024-03-28T18:06:10.609498+0000","last_fullsized":"2024-03-28T18:06:10.609498+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_clean_scrub_stamp":"2024-03-28T18:04:48.465760+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T19:31:49.254101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"92'7","reported_seq":167,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-28T18:07:14.020116+0000","last_change":"2024-03-28T18:04:48.508839+0000","last_active":"2024-03-28T18:07:14.020116+0000","last_peered":"2024-03-28T18:07:14.020116+0000","last_clean":"2024-03-28T18:07:14.020116+0000","last_became_active":"2024-03-28T18:04:48.508475+0000","last_became_peered":"2024-03-28T18:04:48.508475+0000","last_unstale":"2024-03-28T18:07:14.020116+0000","last_undegraded":"2024-03-28T18:07:14.020116+0000","last_fullsized":"2024-03-28T18:07:14.020116+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:04:47.412563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:04:47.412563+0000","last_clean_scrub_stamp":"2024-03-28T18:04:47.412563+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T04:31:10.463284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":140,"num_read_kb":140,"num_write":7,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"92'8","reported_seq":26,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-28T18:06:10.621737+0000","last_change":"2024-03-28T18:04:49.534449+0000","last_active":"2024-03-28T18:06:10.621737+0000","last_peered":"2024-03-28T18:06:10.621737+0000","last_clean":"2024-03-28T18:06:10.621737+0000","last_became_active":"2024-03-28T18:04:49.534098+0000","last_became_peered":"2024-03-28T18:04:49.534098+0000","last_unstale":"2024-03-28T18:06:10.621737+0000","last_undegraded":"2024-03-28T18:06:10.621737+0000","last_fullsized":"2024-03-28T18:06:10.621737+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_clean_scrub_stamp":"2024-03-28T18:04:48.465760+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T23:47:07.886931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"90'6","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-28T18:05:11.948432+0000","last_change":"2024-03-28T18:04:48.501790+0000","last_active":"2024-03-28T18:05:11.948432+0000","last_peered":"2024-03-28T18:05:11.948432+0000","last_clean":"2024-03-28T18:05:11.948432+0000","last_became_active":"2024-03-28T18:04:48.501415+0000","last_became_peered":"2024-03-28T18:04:48.501415+0000","last_unstale":"2024-03-28T18:05:11.948432+0000","last_undegraded":"2024-03-28T18:05:11.948432+0000","last_fullsized":"2024-03-28T18:05:11.948432+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:04:47.412563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:04:47.412563+0000","last_clean_scrub_stamp":"2024-03-28T18:04:47.412563+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T19:17:22.319883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"22'137","reported_seq":366,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-28T18:05:11.735054+0000","last_change":"2024-03-28T18:03:13.522588+0000","last_active":"2024-03-28T18:05:11.735054+0000","last_peered":"2024-03-28T18:05:11.735054+0000","last_clean":"2024-03-28T18:05:11.735054+0000","last_became_active":"2024-03-28T18:03:13.521185+0000","last_became_peered":"2024-03-28T18:03:13.521185+0000","last_unstale":"2024-03-28T18:05:11.735054+0000","last_undegraded":"2024-03-28T18:05:11.735054+0000","last_fullsized":"2024-03-28T18:05:11.735054+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T17:58:33.169981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T17:58:33.169981+0000","last_clean_scrub_stamp":"2024-03-28T17:58:33.169981+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T19:10:06.087374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"92'5","reported_seq":23,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-28T18:06:10.607609+0000","last_change":"2024-03-28T18:04:49.538619+0000","last_active":"2024-03-28T18:06:10.607609+0000","last_peered":"2024-03-28T18:06:10.607609+0000","last_clean":"2024-03-28T18:06:10.607609+0000","last_became_active":"2024-03-28T18:04:49.538276+0000","last_became_peered":"2024-03-28T18:04:49.538276+0000","last_unstale":"2024-03-28T18:06:10.607609+0000","last_undegraded":"2024-03-28T18:06:10.607609+0000","last_fullsized":"2024-03-28T18:06:10.607609+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_clean_scrub_stamp":"2024-03-28T18:04:48.465760+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T01:39:37.139544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"90'4","reported_seq":24,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-28T18:05:11.735222+0000","last_change":"2024-03-28T18:04:48.513621+0000","last_active":"2024-03-28T18:05:11.735222+0000","last_peered":"2024-03-28T18:05:11.735222+0000","last_clean":"2024-03-28T18:05:11.735222+0000","last_became_active":"2024-03-28T18:04:48.513019+0000","last_became_peered":"2024-03-28T18:04:48.513019+0000","last_unstale":"2024-03-28T18:05:11.735222+0000","last_undegraded":"2024-03-28T18:05:11.735222+0000","last_fullsized":"2024-03-28T18:05:11.735222+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:04:47.412563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:04:47.412563+0000","last_clean_scrub_stamp":"2024-03-28T18:04:47.412563+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T23:38:18.082830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"90'4","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-28T18:05:11.735056+0000","last_change":"2024-03-28T18:04:48.513999+0000","last_active":"2024-03-28T18:05:11.735056+0000","last_peered":"2024-03-28T18:05:11.735056+0000","last_clean":"2024-03-28T18:05:11.735056+0000","last_became_active":"2024-03-28T18:04:48.512836+0000","last_became_peered":"2024-03-28T18:04:48.512836+0000","last_unstale":"2024-03-28T18:05:11.735056+0000","last_undegraded":"2024-03-28T18:05:11.735056+0000","last_fullsized":"2024-03-28T18:05:11.735056+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:04:47.412563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:04:47.412563+0000","last_clean_scrub_stamp":"2024-03-28T18:04:47.412563+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T18:29:42.015843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"92'9","reported_seq":27,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-28T18:06:10.605559+0000","last_change":"2024-03-28T18:04:49.519323+0000","last_active":"2024-03-28T18:06:10.605559+0000","last_peered":"2024-03-28T18:06:10.605559+0000","last_clean":"2024-03-28T18:06:10.605559+0000","last_became_active":"2024-03-28T18:04:49.518898+0000","last_became_peered":"2024-03-28T18:04:49.518898+0000","last_unstale":"2024-03-28T18:06:10.605559+0000","last_undegraded":"2024-03-28T18:06:10.605559+0000","last_fullsized":"2024-03-28T18:06:10.605559+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_clean_scrub_stamp":"2024-03-28T18:04:48.465760+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T18:41:12.683606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"92'16","reported_seq":36,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-28T18:07:11.878264+0000","last_change":"2024-03-28T18:04:48.512673+0000","last_active":"2024-03-28T18:07:11.878264+0000","last_peered":"2024-03-28T18:07:11.878264+0000","last_clean":"2024-03-28T18:07:11.878264+0000","last_became_active":"2024-03-28T18:04:48.512137+0000","last_became_peered":"2024-03-28T18:04:48.512137+0000","last_unstale":"2024-03-28T18:07:11.878264+0000","last_undegraded":"2024-03-28T18:07:11.878264+0000","last_fullsized":"2024-03-28T18:07:11.878264+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:04:47.412563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:04:47.412563+0000","last_clean_scrub_stamp":"2024-03-28T18:04:47.412563+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T21:29:41.731701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":28,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"92'5","reported_seq":23,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-28T18:06:10.624351+0000","last_change":"2024-03-28T18:04:49.525055+0000","last_active":"2024-03-28T18:06:10.624351+0000","last_peered":"2024-03-28T18:06:10.624351+0000","last_clean":"2024-03-28T18:06:10.624351+0000","last_became_active":"2024-03-28T18:04:49.522038+0000","last_became_peered":"2024-03-28T18:04:49.522038+0000","last_unstale":"2024-03-28T18:06:10.624351+0000","last_undegraded":"2024-03-28T18:06:10.624351+0000","last_fullsized":"2024-03-28T18:06:10.624351+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_clean_scrub_stamp":"2024-03-28T18:04:48.465760+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T05:21:44.970397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"90'7","reported_seq":28,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-28T18:05:11.949110+0000","last_change":"2024-03-28T18:04:48.509951+0000","last_active":"2024-03-28T18:05:11.949110+0000","last_peered":"2024-03-28T18:05:11.949110+0000","last_clean":"2024-03-28T18:05:11.949110+0000","last_became_active":"2024-03-28T18:04:48.509141+0000","last_became_peered":"2024-03-28T18:04:48.509141+0000","last_unstale":"2024-03-28T18:05:11.949110+0000","last_undegraded":"2024-03-28T18:05:11.949110+0000","last_fullsized":"2024-03-28T18:05:11.949110+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:04:47.412563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:04:47.412563+0000","last_clean_scrub_stamp":"2024-03-28T18:04:47.412563+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T03:54:05.569940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"92'7","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-28T18:06:10.612768+0000","last_change":"2024-03-28T18:04:49.537174+0000","last_active":"2024-03-28T18:06:10.612768+0000","last_peered":"2024-03-28T18:06:10.612768+0000","last_clean":"2024-03-28T18:06:10.612768+0000","last_became_active":"2024-03-28T18:04:49.536801+0000","last_became_peered":"2024-03-28T18:04:49.536801+0000","last_unstale":"2024-03-28T18:06:10.612768+0000","last_undegraded":"2024-03-28T18:06:10.612768+0000","last_fullsized":"2024-03-28T18:06:10.612768+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_clean_scrub_stamp":"2024-03-28T18:04:48.465760+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T05:27:55.908196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"90'2","reported_seq":32,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-28T18:05:11.757387+0000","last_change":"2024-03-28T18:05:11.756894+0000","last_active":"2024-03-28T18:05:11.757387+0000","last_peered":"2024-03-28T18:05:11.757387+0000","last_clean":"2024-03-28T18:05:11.757387+0000","last_became_active":"2024-03-28T18:05:11.750825+0000","last_became_peered":"2024-03-28T18:05:11.750825+0000","last_unstale":"2024-03-28T18:05:11.757387+0000","last_undegraded":"2024-03-28T18:05:11.757387+0000","last_fullsized":"2024-03-28T18:05:11.757387+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:04:47.412563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:04:47.412563+0000","last_clean_scrub_stamp":"2024-03-28T18:04:47.412563+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T23:11:42.105850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,11,0],"acting":[1,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"92'2","reported_seq":20,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-28T18:06:10.611454+0000","last_change":"2024-03-28T18:04:49.523970+0000","last_active":"2024-03-28T18:06:10.611454+0000","last_peered":"2024-03-28T18:06:10.611454+0000","last_clean":"2024-03-28T18:06:10.611454+0000","last_became_active":"2024-03-28T18:04:49.523559+0000","last_became_peered":"2024-03-28T18:04:49.523559+0000","last_unstale":"2024-03-28T18:06:10.611454+0000","last_undegraded":"2024-03-28T18:06:10.611454+0000","last_fullsized":"2024-03-28T18:06:10.611454+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_clean_scrub_stamp":"2024-03-28T18:04:48.465760+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T05:50:25.593232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"90'7","reported_seq":27,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-28T18:05:11.739263+0000","last_change":"2024-03-28T18:04:48.492067+0000","last_active":"2024-03-28T18:05:11.739263+0000","last_peered":"2024-03-28T18:05:11.739263+0000","last_clean":"2024-03-28T18:05:11.739263+0000","last_became_active":"2024-03-28T18:04:48.490633+0000","last_became_peered":"2024-03-28T18:04:48.490633+0000","last_unstale":"2024-03-28T18:05:11.739263+0000","last_undegraded":"2024-03-28T18:05:11.739263+0000","last_fullsized":"2024-03-28T18:05:11.739263+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:04:47.412563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:04:47.412563+0000","last_clean_scrub_stamp":"2024-03-28T18:04:47.412563+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T23:00:52.703618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"92'9","reported_seq":27,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-28T18:06:10.608650+0000","last_change":"2024-03-28T18:04:49.531554+0000","last_active":"2024-03-28T18:06:10.608650+0000","last_peered":"2024-03-28T18:06:10.608650+0000","last_clean":"2024-03-28T18:06:10.608650+0000","last_became_active":"2024-03-28T18:04:49.530273+0000","last_became_peered":"2024-03-28T18:04:49.530273+0000","last_unstale":"2024-03-28T18:06:10.608650+0000","last_undegraded":"2024-03-28T18:06:10.608650+0000","last_fullsized":"2024-03-28T18:06:10.608650+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_clean_scrub_stamp":"2024-03-28T18:04:48.465760+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T22:17:04.032614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"92'5","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-28T18:07:11.597743+0000","last_change":"2024-03-28T18:04:48.514974+0000","last_active":"2024-03-28T18:07:11.597743+0000","last_peered":"2024-03-28T18:07:11.597743+0000","last_clean":"2024-03-28T18:07:11.597743+0000","last_became_active":"2024-03-28T18:04:48.514356+0000","last_became_peered":"2024-03-28T18:04:48.514356+0000","last_unstale":"2024-03-28T18:07:11.597743+0000","last_undegraded":"2024-03-28T18:07:11.597743+0000","last_fullsized":"2024-03-28T18:07:11.597743+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:04:47.412563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:04:47.412563+0000","last_clean_scrub_stamp":"2024-03-28T18:04:47.412563+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T01:26:37.202440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"92'8","reported_seq":26,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-28T18:06:10.613171+0000","last_change":"2024-03-28T18:04:49.518064+0000","last_active":"2024-03-28T18:06:10.613171+0000","last_peered":"2024-03-28T18:06:10.613171+0000","last_clean":"2024-03-28T18:06:10.613171+0000","last_became_active":"2024-03-28T18:04:49.517674+0000","last_became_peered":"2024-03-28T18:04:49.517674+0000","last_unstale":"2024-03-28T18:06:10.613171+0000","last_undegraded":"2024-03-28T18:06:10.613171+0000","last_fullsized":"2024-03-28T18:06:10.613171+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_clean_scrub_stamp":"2024-03-28T18:04:48.465760+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T00:10:44.537164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"90'4","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-28T18:05:11.734862+0000","last_change":"2024-03-28T18:04:48.517028+0000","last_active":"2024-03-28T18:05:11.734862+0000","last_peered":"2024-03-28T18:05:11.734862+0000","last_clean":"2024-03-28T18:05:11.734862+0000","last_became_active":"2024-03-28T18:04:48.516387+0000","last_became_peered":"2024-03-28T18:04:48.516387+0000","last_unstale":"2024-03-28T18:05:11.734862+0000","last_undegraded":"2024-03-28T18:05:11.734862+0000","last_fullsized":"2024-03-28T18:05:11.734862+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:04:47.412563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:04:47.412563+0000","last_clean_scrub_stamp":"2024-03-28T18:04:47.412563+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T01:09:24.142422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"92'5","reported_seq":23,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-28T18:06:10.625821+0000","last_change":"2024-03-28T18:04:49.527032+0000","last_active":"2024-03-28T18:06:10.625821+0000","last_peered":"2024-03-28T18:06:10.625821+0000","last_clean":"2024-03-28T18:06:10.625821+0000","last_became_active":"2024-03-28T18:04:49.523888+0000","last_became_peered":"2024-03-28T18:04:49.523888+0000","last_unstale":"2024-03-28T18:06:10.625821+0000","last_undegraded":"2024-03-28T18:06:10.625821+0000","last_fullsized":"2024-03-28T18:06:10.625821+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_clean_scrub_stamp":"2024-03-28T18:04:48.465760+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T01:19:01.265719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"92'6","reported_seq":169,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-28T18:07:15.011653+0000","last_change":"2024-03-28T18:04:48.502638+0000","last_active":"2024-03-28T18:07:15.011653+0000","last_peered":"2024-03-28T18:07:15.011653+0000","last_clean":"2024-03-28T18:07:15.011653+0000","last_became_active":"2024-03-28T18:04:48.502249+0000","last_became_peered":"2024-03-28T18:04:48.502249+0000","last_unstale":"2024-03-28T18:07:15.011653+0000","last_undegraded":"2024-03-28T18:07:15.011653+0000","last_fullsized":"2024-03-28T18:07:15.011653+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:04:47.412563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:04:47.412563+0000","last_clean_scrub_stamp":"2024-03-28T18:04:47.412563+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T03:23:51.740832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1663,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":143,"num_read_kb":3,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"92'7","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-28T18:06:10.626385+0000","last_change":"2024-03-28T18:04:49.532802+0000","last_active":"2024-03-28T18:06:10.626385+0000","last_peered":"2024-03-28T18:06:10.626385+0000","last_clean":"2024-03-28T18:06:10.626385+0000","last_became_active":"2024-03-28T18:04:49.530666+0000","last_became_peered":"2024-03-28T18:04:49.530666+0000","last_unstale":"2024-03-28T18:06:10.626385+0000","last_undegraded":"2024-03-28T18:06:10.626385+0000","last_fullsized":"2024-03-28T18:06:10.626385+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_clean_scrub_stamp":"2024-03-28T18:04:48.465760+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T22:21:28.469402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"90'3","reported_seq":24,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-28T18:05:11.747786+0000","last_change":"2024-03-28T18:04:48.503582+0000","last_active":"2024-03-28T18:05:11.747786+0000","last_peered":"2024-03-28T18:05:11.747786+0000","last_clean":"2024-03-28T18:05:11.747786+0000","last_became_active":"2024-03-28T18:04:48.502716+0000","last_became_peered":"2024-03-28T18:04:48.502716+0000","last_unstale":"2024-03-28T18:05:11.747786+0000","last_undegraded":"2024-03-28T18:05:11.747786+0000","last_fullsized":"2024-03-28T18:05:11.747786+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:04:47.412563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:04:47.412563+0000","last_clean_scrub_stamp":"2024-03-28T18:04:47.412563+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T21:41:06.333420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"92'7","reported_seq":35,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-28T18:06:10.615938+0000","last_change":"2024-03-28T18:05:11.753703+0000","last_active":"2024-03-28T18:06:10.615938+0000","last_peered":"2024-03-28T18:06:10.615938+0000","last_clean":"2024-03-28T18:06:10.615938+0000","last_became_active":"2024-03-28T18:05:11.748793+0000","last_became_peered":"2024-03-28T18:05:11.748793+0000","last_unstale":"2024-03-28T18:06:10.615938+0000","last_undegraded":"2024-03-28T18:06:10.615938+0000","last_fullsized":"2024-03-28T18:06:10.615938+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_clean_scrub_stamp":"2024-03-28T18:04:48.465760+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T19:38:38.432027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":117,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"90'5","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-28T18:05:11.950534+0000","last_change":"2024-03-28T18:04:48.507658+0000","last_active":"2024-03-28T18:05:11.950534+0000","last_peered":"2024-03-28T18:05:11.950534+0000","last_clean":"2024-03-28T18:05:11.950534+0000","last_became_active":"2024-03-28T18:04:48.506855+0000","last_became_peered":"2024-03-28T18:04:48.506855+0000","last_unstale":"2024-03-28T18:05:11.950534+0000","last_undegraded":"2024-03-28T18:05:11.950534+0000","last_fullsized":"2024-03-28T18:05:11.950534+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:04:47.412563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:04:47.412563+0000","last_clean_scrub_stamp":"2024-03-28T18:04:47.412563+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T23:08:23.550379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"92'9","reported_seq":27,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-28T18:06:10.615589+0000","last_change":"2024-03-28T18:04:49.521122+0000","last_active":"2024-03-28T18:06:10.615589+0000","last_peered":"2024-03-28T18:06:10.615589+0000","last_clean":"2024-03-28T18:06:10.615589+0000","last_became_active":"2024-03-28T18:04:49.520711+0000","last_became_peered":"2024-03-28T18:04:49.520711+0000","last_unstale":"2024-03-28T18:06:10.615589+0000","last_undegraded":"2024-03-28T18:06:10.615589+0000","last_fullsized":"2024-03-28T18:06:10.615589+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_clean_scrub_stamp":"2024-03-28T18:04:48.465760+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T19:32:33.556979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"92'9","reported_seq":27,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-28T18:06:10.605702+0000","last_change":"2024-03-28T18:04:49.534929+0000","last_active":"2024-03-28T18:06:10.605702+0000","last_peered":"2024-03-28T18:06:10.605702+0000","last_clean":"2024-03-28T18:06:10.605702+0000","last_became_active":"2024-03-28T18:04:49.534485+0000","last_became_peered":"2024-03-28T18:04:49.534485+0000","last_unstale":"2024-03-28T18:06:10.605702+0000","last_undegraded":"2024-03-28T18:06:10.605702+0000","last_fullsized":"2024-03-28T18:06:10.605702+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_clean_scrub_stamp":"2024-03-28T18:04:48.465760+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T01:47:34.937536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"92'12","reported_seq":36,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-28T18:06:10.606584+0000","last_change":"2024-03-28T18:05:11.752731+0000","last_active":"2024-03-28T18:06:10.606584+0000","last_peered":"2024-03-28T18:06:10.606584+0000","last_clean":"2024-03-28T18:06:10.606584+0000","last_became_active":"2024-03-28T18:05:11.752121+0000","last_became_peered":"2024-03-28T18:05:11.752121+0000","last_unstale":"2024-03-28T18:06:10.606584+0000","last_undegraded":"2024-03-28T18:06:10.606584+0000","last_fullsized":"2024-03-28T18:06:10.606584+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_clean_scrub_stamp":"2024-03-28T18:04:48.465760+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T20:10:02.491231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,8],"acting":[2,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"92'8","reported_seq":26,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-28T18:06:10.609164+0000","last_change":"2024-03-28T18:04:49.531999+0000","last_active":"2024-03-28T18:06:10.609164+0000","last_peered":"2024-03-28T18:06:10.609164+0000","last_clean":"2024-03-28T18:06:10.609164+0000","last_became_active":"2024-03-28T18:04:49.528641+0000","last_became_peered":"2024-03-28T18:04:49.528641+0000","last_unstale":"2024-03-28T18:06:10.609164+0000","last_undegraded":"2024-03-28T18:06:10.609164+0000","last_fullsized":"2024-03-28T18:06:10.609164+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_clean_scrub_stamp":"2024-03-28T18:04:48.465760+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T21:24:43.861635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"92'7","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-28T18:06:10.610718+0000","last_change":"2024-03-28T18:04:49.540417+0000","last_active":"2024-03-28T18:06:10.610718+0000","last_peered":"2024-03-28T18:06:10.610718+0000","last_clean":"2024-03-28T18:06:10.610718+0000","last_became_active":"2024-03-28T18:04:49.539987+0000","last_became_peered":"2024-03-28T18:04:49.539987+0000","last_unstale":"2024-03-28T18:06:10.610718+0000","last_undegraded":"2024-03-28T18:06:10.610718+0000","last_fullsized":"2024-03-28T18:06:10.610718+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_clean_scrub_stamp":"2024-03-28T18:04:48.465760+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T02:53:41.444850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"92'4","reported_seq":22,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-28T18:06:10.609488+0000","last_change":"2024-03-28T18:04:49.540014+0000","last_active":"2024-03-28T18:06:10.609488+0000","last_peered":"2024-03-28T18:06:10.609488+0000","last_clean":"2024-03-28T18:06:10.609488+0000","last_became_active":"2024-03-28T18:04:49.538239+0000","last_became_peered":"2024-03-28T18:04:49.538239+0000","last_unstale":"2024-03-28T18:06:10.609488+0000","last_undegraded":"2024-03-28T18:06:10.609488+0000","last_fullsized":"2024-03-28T18:06:10.609488+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_clean_scrub_stamp":"2024-03-28T18:04:48.465760+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T23:03:58.449336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"92'6","reported_seq":30,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-28T18:06:10.603750+0000","last_change":"2024-03-28T18:05:11.748781+0000","last_active":"2024-03-28T18:06:10.603750+0000","last_peered":"2024-03-28T18:06:10.603750+0000","last_clean":"2024-03-28T18:06:10.603750+0000","last_became_active":"2024-03-28T18:05:11.747710+0000","last_became_peered":"2024-03-28T18:05:11.747710+0000","last_unstale":"2024-03-28T18:06:10.603750+0000","last_undegraded":"2024-03-28T18:06:10.603750+0000","last_fullsized":"2024-03-28T18:06:10.603750+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_clean_scrub_stamp":"2024-03-28T18:04:48.465760+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T02:01:44.708646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,9],"acting":[11,8,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"92'15","reported_seq":33,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-28T18:06:10.618896+0000","last_change":"2024-03-28T18:04:49.531135+0000","last_active":"2024-03-28T18:06:10.618896+0000","last_peered":"2024-03-28T18:06:10.618896+0000","last_clean":"2024-03-28T18:06:10.618896+0000","last_became_active":"2024-03-28T18:04:49.530673+0000","last_became_peered":"2024-03-28T18:04:49.530673+0000","last_unstale":"2024-03-28T18:06:10.618896+0000","last_undegraded":"2024-03-28T18:06:10.618896+0000","last_fullsized":"2024-03-28T18:06:10.618896+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_clean_scrub_stamp":"2024-03-28T18:04:48.465760+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T05:51:13.950946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"92'2","reported_seq":20,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-28T18:06:10.602677+0000","last_change":"2024-03-28T18:04:49.526803+0000","last_active":"2024-03-28T18:06:10.602677+0000","last_peered":"2024-03-28T18:06:10.602677+0000","last_clean":"2024-03-28T18:06:10.602677+0000","last_became_active":"2024-03-28T18:04:49.526474+0000","last_became_peered":"2024-03-28T18:04:49.526474+0000","last_unstale":"2024-03-28T18:06:10.602677+0000","last_undegraded":"2024-03-28T18:06:10.602677+0000","last_fullsized":"2024-03-28T18:06:10.602677+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_clean_scrub_stamp":"2024-03-28T18:04:48.465760+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T01:28:27.530864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"92'6","reported_seq":24,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-28T18:06:10.612723+0000","last_change":"2024-03-28T18:04:49.534015+0000","last_active":"2024-03-28T18:06:10.612723+0000","last_peered":"2024-03-28T18:06:10.612723+0000","last_clean":"2024-03-28T18:06:10.612723+0000","last_became_active":"2024-03-28T18:04:49.533560+0000","last_became_peered":"2024-03-28T18:04:49.533560+0000","last_unstale":"2024-03-28T18:06:10.612723+0000","last_undegraded":"2024-03-28T18:06:10.612723+0000","last_fullsized":"2024-03-28T18:06:10.612723+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_clean_scrub_stamp":"2024-03-28T18:04:48.465760+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T22:41:30.965737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"92'8","reported_seq":26,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-28T18:06:10.608221+0000","last_change":"2024-03-28T18:04:49.523240+0000","last_active":"2024-03-28T18:06:10.608221+0000","last_peered":"2024-03-28T18:06:10.608221+0000","last_clean":"2024-03-28T18:06:10.608221+0000","last_became_active":"2024-03-28T18:04:49.522795+0000","last_became_peered":"2024-03-28T18:04:49.522795+0000","last_unstale":"2024-03-28T18:06:10.608221+0000","last_undegraded":"2024-03-28T18:06:10.608221+0000","last_fullsized":"2024-03-28T18:06:10.608221+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_clean_scrub_stamp":"2024-03-28T18:04:48.465760+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T23:08:37.087977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"92'10","reported_seq":28,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-28T18:06:10.626330+0000","last_change":"2024-03-28T18:04:49.530965+0000","last_active":"2024-03-28T18:06:10.626330+0000","last_peered":"2024-03-28T18:06:10.626330+0000","last_clean":"2024-03-28T18:06:10.626330+0000","last_became_active":"2024-03-28T18:04:49.530619+0000","last_became_peered":"2024-03-28T18:04:49.530619+0000","last_unstale":"2024-03-28T18:06:10.626330+0000","last_undegraded":"2024-03-28T18:06:10.626330+0000","last_fullsized":"2024-03-28T18:06:10.626330+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_clean_scrub_stamp":"2024-03-28T18:04:48.465760+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T05:29:26.754012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"92'11","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-28T18:06:10.624772+0000","last_change":"2024-03-28T18:04:49.523017+0000","last_active":"2024-03-28T18:06:10.624772+0000","last_peered":"2024-03-28T18:06:10.624772+0000","last_clean":"2024-03-28T18:06:10.624772+0000","last_became_active":"2024-03-28T18:04:49.520411+0000","last_became_peered":"2024-03-28T18:04:49.520411+0000","last_unstale":"2024-03-28T18:06:10.624772+0000","last_undegraded":"2024-03-28T18:06:10.624772+0000","last_fullsized":"2024-03-28T18:06:10.624772+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_clean_scrub_stamp":"2024-03-28T18:04:48.465760+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T22:52:45.893036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"92'8","reported_seq":26,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-28T18:06:10.608384+0000","last_change":"2024-03-28T18:04:49.544272+0000","last_active":"2024-03-28T18:06:10.608384+0000","last_peered":"2024-03-28T18:06:10.608384+0000","last_clean":"2024-03-28T18:06:10.608384+0000","last_became_active":"2024-03-28T18:04:49.541754+0000","last_became_peered":"2024-03-28T18:04:49.541754+0000","last_unstale":"2024-03-28T18:06:10.608384+0000","last_undegraded":"2024-03-28T18:06:10.608384+0000","last_fullsized":"2024-03-28T18:06:10.608384+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_clean_scrub_stamp":"2024-03-28T18:04:48.465760+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T19:06:00.799916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":117,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":502,"ondisk_log_size":502,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4751360,"data_stored":4722944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":74205,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":1436,"num_read_kb":797,"num_write":267,"num_write_kb":159,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":0,"num_keys_recovered":10,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":573440,"data_stored":218301,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":229,"ondisk_log_size":229,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":85,"seq":365072220206,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28000,"kb_used_data":984,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717152,"statfs":{"total":95995035648,"available":95966363648,"internally_reserved":0,"allocated":1007616,"data_stored":484342,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":10,"up_from":77,"seq":330712481846,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":37008,"kb_used_data":1672,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708144,"statfs":{"total":95995035648,"available":95957139456,"internally_reserved":0,"allocated":1712128,"data_stored":1140650,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":36174272},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":70,"seq":300647710780,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36348,"kb_used_data":1012,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708804,"statfs":{"total":95995035648,"available":95957815296,"internally_reserved":0,"allocated":1036288,"data_stored":483896,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":36174272},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":62,"seq":266287972418,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36340,"kb_used_data":1004,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708812,"statfs":{"total":95995035648,"available":95957823488,"internally_reserved":0,"allocated":1028096,"data_stored":484157,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":7,"up_from":53,"seq":227633266761,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28604,"kb_used_data":1588,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716548,"statfs":{"total":95995035648,"available":95965745152,"internally_reserved":0,"allocated":1626112,"data_stored":1078827,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":48,"seq":206158430287,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36328,"kb_used_data":992,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708824,"statfs":{"total":95995035648,"available":95957835776,"internally_reserved":0,"allocated":1015808,"data_stored":483989,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574963,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36388,"kb_used_data":1052,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708764,"statfs":{"total":95995035648,"available":95957774336,"internally_reserved":0,"allocated":1077248,"data_stored":545335,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738490,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28028,"kb_used_data":1012,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717124,"statfs":{"total":95995035648,"available":95966334976,"internally_reserved":0,"allocated":1036288,"data_stored":485328,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411438,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36412,"kb_used_data":1076,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708740,"statfs":{"total":95995035648,"available":95957749760,"internally_reserved":0,"allocated":1101824,"data_stored":550894,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":3,"up_from":26,"seq":111669149798,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36340,"kb_used_data":1004,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708812,"statfs":{"total":95995035648,"available":95957823488,"internally_reserved":0,"allocated":1028096,"data_stored":483204,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":36174280},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":34,"seq":146028888158,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28008,"kb_used_data":992,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717144,"statfs":{"total":95995035648,"available":95966355456,"internally_reserved":0,"allocated":1015808,"data_stored":484382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":40,"seq":171798691929,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36936,"kb_used_data":1600,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708216,"statfs":{"total":95995035648,"available":95957213184,"internally_reserved":0,"allocated":1638400,"data_stored":1076987,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2671,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":81920,"data_stored":62678,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":106496,"data_stored":68237,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1725,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":3845,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1215,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":5685,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":1500,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":118784,"data_stored":67508,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1685,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-28T18:07:18.956 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-28T18:07:19.206 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:19.206+0000 7fb972000640 1 -- 172.21.15.22:0/3099099727 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb96c1021c0 msgr2=0x7fb96c1045b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:07:19.206 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:19.206+0000 7fb972000640 1 --2- 172.21.15.22:0/3099099727 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb96c1021c0 0x7fb96c1045b0 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7fb95c009900 tx=0x7fb95c02bc30 comp rx=0 tx=0).stop 2024-03-28T18:07:19.206 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:19.206+0000 7fb972000640 1 -- 172.21.15.22:0/3099099727 shutdown_connections 2024-03-28T18:07:19.206 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:19.206+0000 7fb972000640 1 --2- 172.21.15.22:0/3099099727 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fb96c107420 0x7fb96c109870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:07:19.206 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:19.206+0000 7fb972000640 1 --2- 172.21.15.22:0/3099099727 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb96c104af0 0x7fb96c106ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:07:19.206 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:19.206+0000 7fb972000640 1 --2- 172.21.15.22:0/3099099727 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb96c1021c0 0x7fb96c1045b0 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:07:19.206 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:19.206+0000 7fb972000640 1 -- 172.21.15.22:0/3099099727 >> 172.21.15.22:0/3099099727 conn(0x7fb96c0fbbd0 msgr2=0x7fb96c0fdff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:07:19.206 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:19.206+0000 7fb972000640 1 -- 172.21.15.22:0/3099099727 shutdown_connections 2024-03-28T18:07:19.206 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:19.206+0000 7fb972000640 1 -- 172.21.15.22:0/3099099727 wait complete. 2024-03-28T18:07:19.207 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:19.207+0000 7fb972000640 1 Processor -- start 2024-03-28T18:07:19.207 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:19.207+0000 7fb972000640 1 -- start start 2024-03-28T18:07:19.207 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:19.208+0000 7fb972000640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fb96c1021c0 0x7fb96c197a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:07:19.207 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:19.208+0000 7fb972000640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb96c104af0 0x7fb96c197f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:07:19.207 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:19.208+0000 7fb972000640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb96c107420 0x7fb96c19eff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:07:19.207 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:19.208+0000 7fb972000640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb96c10d3a0 con 0x7fb96c107420 2024-03-28T18:07:19.208 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:19.208+0000 7fb972000640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb96c10d220 con 0x7fb96c1021c0 2024-03-28T18:07:19.208 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:19.208+0000 7fb972000640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb96c10d520 con 0x7fb96c104af0 2024-03-28T18:07:19.208 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:19.208+0000 7fb96b400640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fb96c1021c0 0x7fb96c197a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:07:19.208 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:19.208+0000 7fb96b400640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fb96c1021c0 0x7fb96c197a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.22:59176/0 (socket says 172.21.15.22:59176) 2024-03-28T18:07:19.209 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:19.208+0000 7fb96b400640 1 -- 172.21.15.22:0/3656302087 learned_addr learned my addr 172.21.15.22:0/3656302087 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-28T18:07:19.209 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:19.208+0000 7fb96be00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb96c107420 0x7fb96c19eff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:07:19.209 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:19.208+0000 7fb963e00640 1 --2- 172.21.15.22:0/3656302087 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb96c104af0 0x7fb96c197f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:07:19.209 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:19.209+0000 7fb96b400640 1 -- 172.21.15.22:0/3656302087 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb96c104af0 msgr2=0x7fb96c197f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:07:19.209 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:19.209+0000 7fb96b400640 1 --2- 172.21.15.22:0/3656302087 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb96c104af0 0x7fb96c197f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:07:19.209 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:19.209+0000 7fb96b400640 1 -- 172.21.15.22:0/3656302087 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb96c107420 msgr2=0x7fb96c19eff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:07:19.209 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:19.209+0000 7fb96b400640 1 --2- 172.21.15.22:0/3656302087 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb96c107420 0x7fb96c19eff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:07:19.209 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:19.209+0000 7fb96b400640 1 -- 172.21.15.22:0/3656302087 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb96c19f660 con 0x7fb96c1021c0 2024-03-28T18:07:19.209 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:19.209+0000 7fb96be00640 1 --2- 172.21.15.22:0/3656302087 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb96c107420 0x7fb96c19eff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-28T18:07:19.209 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:19.209+0000 7fb96b400640 1 --2- 172.21.15.22:0/3656302087 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fb96c1021c0 0x7fb96c197a40 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fb95c000c00 tx=0x7fb95c004e20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:07:19.210 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:19.209+0000 7fb968c00640 1 -- 172.21.15.22:0/3656302087 <== mon.2 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb95c043020 con 0x7fb96c1021c0 2024-03-28T18:07:19.210 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:19.210+0000 7fb972000640 1 -- 172.21.15.22:0/3656302087 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb96c19f8f0 con 0x7fb96c1021c0 2024-03-28T18:07:19.210 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:19.210+0000 7fb968c00640 1 -- 172.21.15.22:0/3656302087 <== mon.2 v2:172.21.15.79:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7fb95c005460 con 0x7fb96c1021c0 2024-03-28T18:07:19.210 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:19.210+0000 7fb968c00640 1 -- 172.21.15.22:0/3656302087 <== mon.2 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb95c003d00 con 0x7fb96c1021c0 2024-03-28T18:07:19.210 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:19.210+0000 7fb972000640 1 -- 172.21.15.22:0/3656302087 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb96c19fe90 con 0x7fb96c1021c0 2024-03-28T18:07:19.210 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:19.211+0000 7fb972000640 1 -- 172.21.15.22:0/3656302087 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb96c0fd6c0 con 0x7fb96c1021c0 2024-03-28T18:07:19.214 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:19.215+0000 7fb968c00640 1 -- 172.21.15.22:0/3656302087 <== mon.2 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 18) v1 ==== 137290+0+0 (secure 0 0 0) 0x7fb95c046020 con 0x7fb96c1021c0 2024-03-28T18:07:19.216 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:19.216+0000 7fb968c00640 1 --2- 172.21.15.22:0/3656302087 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7fb9500a2880 0x7fb9500a4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:07:19.216 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:19.216+0000 7fb968c00640 1 -- 172.21.15.22:0/3656302087 <== mon.2 v2:172.21.15.79:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 8153+0+0 (secure 0 0 0) 0x7fb95c0f0f70 con 0x7fb96c1021c0 2024-03-28T18:07:19.216 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:19.217+0000 7fb963e00640 1 --2- 172.21.15.22:0/3656302087 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7fb9500a2880 0x7fb9500a4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:07:19.217 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:19.217+0000 7fb963e00640 1 --2- 172.21.15.22:0/3656302087 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7fb9500a2880 0x7fb9500a4d40 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7fb96c198f50 tx=0x7fb95400b390 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:07:19.223 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:19.223+0000 7fb968c00640 1 -- 172.21.15.22:0/3656302087 <== mon.2 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fb95c04d020 con 0x7fb96c1021c0 2024-03-28T18:07:19.453 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:19.454+0000 7fb972000640 1 -- 172.21.15.22:0/3656302087 --> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fb96c061ae0 con 0x7fb9500a2880 2024-03-28T18:07:19.474 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:19.474+0000 7fb968c00640 1 -- 172.21.15.22:0/3656302087 <== mgr.14152 v2:172.21.15.22:6800/4201048392 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+258984 (secure 0 0 0) 0x7fb96c061ae0 con 0x7fb9500a2880 2024-03-28T18:07:19.478 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-03-28T18:07:19.479 INFO:teuthology.orchestra.run.smithi022.stderr:dumped all 2024-03-28T18:07:19.480 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:19.479+0000 7fb972000640 1 -- 172.21.15.22:0/3656302087 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7fb9500a2880 msgr2=0x7fb9500a4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:07:19.480 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:19.479+0000 7fb972000640 1 --2- 172.21.15.22:0/3656302087 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7fb9500a2880 0x7fb9500a4d40 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7fb96c198f50 tx=0x7fb95400b390 comp rx=0 tx=0).stop 2024-03-28T18:07:19.480 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:19.479+0000 7fb972000640 1 -- 172.21.15.22:0/3656302087 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fb96c1021c0 msgr2=0x7fb96c197a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:07:19.480 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:19.479+0000 7fb972000640 1 --2- 172.21.15.22:0/3656302087 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fb96c1021c0 0x7fb96c197a40 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fb95c000c00 tx=0x7fb95c004e20 comp rx=0 tx=0).stop 2024-03-28T18:07:19.480 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:19.479+0000 7fb972000640 1 -- 172.21.15.22:0/3656302087 shutdown_connections 2024-03-28T18:07:19.480 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:19.479+0000 7fb972000640 1 --2- 172.21.15.22:0/3656302087 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7fb9500a2880 0x7fb9500a4d40 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:07:19.481 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:19.479+0000 7fb972000640 1 --2- 172.21.15.22:0/3656302087 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb96c107420 0x7fb96c19eff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:07:19.481 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:19.479+0000 7fb972000640 1 --2- 172.21.15.22:0/3656302087 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb96c104af0 0x7fb96c197f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:07:19.481 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:19.479+0000 7fb972000640 1 --2- 172.21.15.22:0/3656302087 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fb96c1021c0 0x7fb96c197a40 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:07:19.481 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:19.479+0000 7fb972000640 1 -- 172.21.15.22:0/3656302087 >> 172.21.15.22:0/3656302087 conn(0x7fb96c0fbbd0 msgr2=0x7fb96c102be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:07:19.481 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:19.479+0000 7fb972000640 1 -- 172.21.15.22:0/3656302087 shutdown_connections 2024-03-28T18:07:19.481 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:19.479+0000 7fb972000640 1 -- 172.21.15.22:0/3656302087 wait complete. 2024-03-28T18:07:19.497 INFO:teuthology.orchestra.run.smithi022.stdout:{"pg_ready":true,"pg_map":{"version":446,"stamp":"2024-03-28T18:07:18.662318+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":664807,"num_objects":102,"num_object_clones":0,"num_object_copies":306,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":1572,"num_read_kb":916,"num_write":450,"num_write_kb":3099,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13,"num_bytes_recovered":1180853,"num_keys_recovered":10,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":868,"ondisk_log_size":868,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":404740,"kb_used_data":13988,"kb_used_omap":18,"kb_used_meta":390637,"kb_avail":1124537084,"statfs":{"total":1151940427776,"available":1151525974016,"internally_reserved":0,"allocated":14323712,"data_stored":7781991,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19141,"internal_metadata":400012603},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":6779,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":116,"num_read_kb":60,"num_write":21,"num_write_kb":20,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004700"},"pg_stats":[{"pgid":"3.3f","version":"92'8","reported_seq":26,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-28T18:06:10.622027+0000","last_change":"2024-03-28T18:04:49.545022+0000","last_active":"2024-03-28T18:06:10.622027+0000","last_peered":"2024-03-28T18:06:10.622027+0000","last_clean":"2024-03-28T18:06:10.622027+0000","last_became_active":"2024-03-28T18:04:49.544584+0000","last_became_peered":"2024-03-28T18:04:49.544584+0000","last_unstale":"2024-03-28T18:06:10.622027+0000","last_undegraded":"2024-03-28T18:06:10.622027+0000","last_fullsized":"2024-03-28T18:06:10.622027+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_clean_scrub_stamp":"2024-03-28T18:04:48.465760+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T01:47:05.644293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"92'4","reported_seq":22,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-28T18:06:10.615631+0000","last_change":"2024-03-28T18:04:49.526720+0000","last_active":"2024-03-28T18:06:10.615631+0000","last_peered":"2024-03-28T18:06:10.615631+0000","last_clean":"2024-03-28T18:06:10.615631+0000","last_became_active":"2024-03-28T18:04:49.524545+0000","last_became_peered":"2024-03-28T18:04:49.524545+0000","last_unstale":"2024-03-28T18:06:10.615631+0000","last_undegraded":"2024-03-28T18:06:10.615631+0000","last_fullsized":"2024-03-28T18:06:10.615631+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_clean_scrub_stamp":"2024-03-28T18:04:48.465760+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T01:00:54.926308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"92'7","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-28T18:06:10.625795+0000","last_change":"2024-03-28T18:04:49.532425+0000","last_active":"2024-03-28T18:06:10.625795+0000","last_peered":"2024-03-28T18:06:10.625795+0000","last_clean":"2024-03-28T18:06:10.625795+0000","last_became_active":"2024-03-28T18:04:49.531146+0000","last_became_peered":"2024-03-28T18:04:49.531146+0000","last_unstale":"2024-03-28T18:06:10.625795+0000","last_undegraded":"2024-03-28T18:06:10.625795+0000","last_fullsized":"2024-03-28T18:06:10.625795+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_clean_scrub_stamp":"2024-03-28T18:04:48.465760+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T04:58:45.131773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"92'8","reported_seq":26,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-28T18:06:10.609313+0000","last_change":"2024-03-28T18:04:49.533371+0000","last_active":"2024-03-28T18:06:10.609313+0000","last_peered":"2024-03-28T18:06:10.609313+0000","last_clean":"2024-03-28T18:06:10.609313+0000","last_became_active":"2024-03-28T18:04:49.532544+0000","last_became_peered":"2024-03-28T18:04:49.532544+0000","last_unstale":"2024-03-28T18:06:10.609313+0000","last_undegraded":"2024-03-28T18:06:10.609313+0000","last_fullsized":"2024-03-28T18:06:10.609313+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_clean_scrub_stamp":"2024-03-28T18:04:48.465760+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T20:36:25.664137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"92'4","reported_seq":21,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-28T18:06:10.592721+0000","last_change":"2024-03-28T18:04:49.530875+0000","last_active":"2024-03-28T18:06:10.592721+0000","last_peered":"2024-03-28T18:06:10.592721+0000","last_clean":"2024-03-28T18:06:10.592721+0000","last_became_active":"2024-03-28T18:04:49.530354+0000","last_became_peered":"2024-03-28T18:04:49.530354+0000","last_unstale":"2024-03-28T18:06:10.592721+0000","last_undegraded":"2024-03-28T18:06:10.592721+0000","last_fullsized":"2024-03-28T18:06:10.592721+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_clean_scrub_stamp":"2024-03-28T18:04:48.465760+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T21:02:17.187989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"92'6","reported_seq":23,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-28T18:06:10.587630+0000","last_change":"2024-03-28T18:04:49.538260+0000","last_active":"2024-03-28T18:06:10.587630+0000","last_peered":"2024-03-28T18:06:10.587630+0000","last_clean":"2024-03-28T18:06:10.587630+0000","last_became_active":"2024-03-28T18:04:49.537863+0000","last_became_peered":"2024-03-28T18:04:49.537863+0000","last_unstale":"2024-03-28T18:06:10.587630+0000","last_undegraded":"2024-03-28T18:06:10.587630+0000","last_fullsized":"2024-03-28T18:06:10.587630+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_clean_scrub_stamp":"2024-03-28T18:04:48.465760+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T03:01:25.363839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"92'5","reported_seq":22,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-28T18:06:10.612305+0000","last_change":"2024-03-28T18:04:49.534636+0000","last_active":"2024-03-28T18:06:10.612305+0000","last_peered":"2024-03-28T18:06:10.612305+0000","last_clean":"2024-03-28T18:06:10.612305+0000","last_became_active":"2024-03-28T18:04:49.534236+0000","last_became_peered":"2024-03-28T18:04:49.534236+0000","last_unstale":"2024-03-28T18:06:10.612305+0000","last_undegraded":"2024-03-28T18:06:10.612305+0000","last_fullsized":"2024-03-28T18:06:10.612305+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_clean_scrub_stamp":"2024-03-28T18:04:48.465760+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T05:55:23.616641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"92'9","reported_seq":26,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-28T18:06:10.605001+0000","last_change":"2024-03-28T18:04:49.534848+0000","last_active":"2024-03-28T18:06:10.605001+0000","last_peered":"2024-03-28T18:06:10.605001+0000","last_clean":"2024-03-28T18:06:10.605001+0000","last_became_active":"2024-03-28T18:04:49.534302+0000","last_became_peered":"2024-03-28T18:04:49.534302+0000","last_unstale":"2024-03-28T18:06:10.605001+0000","last_undegraded":"2024-03-28T18:06:10.605001+0000","last_fullsized":"2024-03-28T18:06:10.605001+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_clean_scrub_stamp":"2024-03-28T18:04:48.465760+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T05:48:24.707305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"92'11","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-28T18:06:10.622540+0000","last_change":"2024-03-28T18:04:49.536568+0000","last_active":"2024-03-28T18:06:10.622540+0000","last_peered":"2024-03-28T18:06:10.622540+0000","last_clean":"2024-03-28T18:06:10.622540+0000","last_became_active":"2024-03-28T18:04:49.535675+0000","last_became_peered":"2024-03-28T18:04:49.535675+0000","last_unstale":"2024-03-28T18:06:10.622540+0000","last_undegraded":"2024-03-28T18:06:10.622540+0000","last_fullsized":"2024-03-28T18:06:10.622540+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_clean_scrub_stamp":"2024-03-28T18:04:48.465760+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T02:32:15.179869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"92'16","reported_seq":34,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-28T18:06:10.614038+0000","last_change":"2024-03-28T18:04:49.539363+0000","last_active":"2024-03-28T18:06:10.614038+0000","last_peered":"2024-03-28T18:06:10.614038+0000","last_clean":"2024-03-28T18:06:10.614038+0000","last_became_active":"2024-03-28T18:04:49.539002+0000","last_became_peered":"2024-03-28T18:04:49.539002+0000","last_unstale":"2024-03-28T18:06:10.614038+0000","last_undegraded":"2024-03-28T18:06:10.614038+0000","last_fullsized":"2024-03-28T18:06:10.614038+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_clean_scrub_stamp":"2024-03-28T18:04:48.465760+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T18:24:28.846081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"92'6","reported_seq":24,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-28T18:06:10.626273+0000","last_change":"2024-03-28T18:04:49.532385+0000","last_active":"2024-03-28T18:06:10.626273+0000","last_peered":"2024-03-28T18:06:10.626273+0000","last_clean":"2024-03-28T18:06:10.626273+0000","last_became_active":"2024-03-28T18:04:49.532004+0000","last_became_peered":"2024-03-28T18:04:49.532004+0000","last_unstale":"2024-03-28T18:06:10.626273+0000","last_undegraded":"2024-03-28T18:06:10.626273+0000","last_fullsized":"2024-03-28T18:06:10.626273+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_clean_scrub_stamp":"2024-03-28T18:04:48.465760+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T19:05:13.959313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"92'7","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-28T18:06:10.608967+0000","last_change":"2024-03-28T18:04:49.534465+0000","last_active":"2024-03-28T18:06:10.608967+0000","last_peered":"2024-03-28T18:06:10.608967+0000","last_clean":"2024-03-28T18:06:10.608967+0000","last_became_active":"2024-03-28T18:04:49.532989+0000","last_became_peered":"2024-03-28T18:04:49.532989+0000","last_unstale":"2024-03-28T18:06:10.608967+0000","last_undegraded":"2024-03-28T18:06:10.608967+0000","last_fullsized":"2024-03-28T18:06:10.608967+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_clean_scrub_stamp":"2024-03-28T18:04:48.465760+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T23:56:50.607770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"92'10","reported_seq":28,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-28T18:06:10.612024+0000","last_change":"2024-03-28T18:04:49.535205+0000","last_active":"2024-03-28T18:06:10.612024+0000","last_peered":"2024-03-28T18:06:10.612024+0000","last_clean":"2024-03-28T18:06:10.612024+0000","last_became_active":"2024-03-28T18:04:49.533025+0000","last_became_peered":"2024-03-28T18:04:49.533025+0000","last_unstale":"2024-03-28T18:06:10.612024+0000","last_undegraded":"2024-03-28T18:06:10.612024+0000","last_fullsized":"2024-03-28T18:06:10.612024+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_clean_scrub_stamp":"2024-03-28T18:04:48.465760+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T21:41:51.815907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"92'7","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-28T18:06:10.615451+0000","last_change":"2024-03-28T18:04:49.520974+0000","last_active":"2024-03-28T18:06:10.615451+0000","last_peered":"2024-03-28T18:06:10.615451+0000","last_clean":"2024-03-28T18:06:10.615451+0000","last_became_active":"2024-03-28T18:04:49.520509+0000","last_became_peered":"2024-03-28T18:04:49.520509+0000","last_unstale":"2024-03-28T18:06:10.615451+0000","last_undegraded":"2024-03-28T18:06:10.615451+0000","last_fullsized":"2024-03-28T18:06:10.615451+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_clean_scrub_stamp":"2024-03-28T18:04:48.465760+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T03:15:16.346859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"92'7","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-28T18:06:10.608830+0000","last_change":"2024-03-28T18:04:49.539053+0000","last_active":"2024-03-28T18:06:10.608830+0000","last_peered":"2024-03-28T18:06:10.608830+0000","last_clean":"2024-03-28T18:06:10.608830+0000","last_became_active":"2024-03-28T18:04:49.537327+0000","last_became_peered":"2024-03-28T18:04:49.537327+0000","last_unstale":"2024-03-28T18:06:10.608830+0000","last_undegraded":"2024-03-28T18:06:10.608830+0000","last_fullsized":"2024-03-28T18:06:10.608830+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_clean_scrub_stamp":"2024-03-28T18:04:48.465760+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T00:24:42.319504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"92'9","reported_seq":27,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-28T18:06:10.625483+0000","last_change":"2024-03-28T18:04:49.525855+0000","last_active":"2024-03-28T18:06:10.625483+0000","last_peered":"2024-03-28T18:06:10.625483+0000","last_clean":"2024-03-28T18:06:10.625483+0000","last_became_active":"2024-03-28T18:04:49.525448+0000","last_became_peered":"2024-03-28T18:04:49.525448+0000","last_unstale":"2024-03-28T18:06:10.625483+0000","last_undegraded":"2024-03-28T18:06:10.625483+0000","last_fullsized":"2024-03-28T18:06:10.625483+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_clean_scrub_stamp":"2024-03-28T18:04:48.465760+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T23:53:00.452692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"92'8","reported_seq":26,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-28T18:06:10.622131+0000","last_change":"2024-03-28T18:04:49.543605+0000","last_active":"2024-03-28T18:06:10.622131+0000","last_peered":"2024-03-28T18:06:10.622131+0000","last_clean":"2024-03-28T18:06:10.622131+0000","last_became_active":"2024-03-28T18:04:49.543280+0000","last_became_peered":"2024-03-28T18:04:49.543280+0000","last_unstale":"2024-03-28T18:06:10.622131+0000","last_undegraded":"2024-03-28T18:06:10.622131+0000","last_fullsized":"2024-03-28T18:06:10.622131+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_clean_scrub_stamp":"2024-03-28T18:04:48.465760+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T21:27:26.216639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"92'10","reported_seq":28,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-28T18:06:10.619171+0000","last_change":"2024-03-28T18:04:49.526745+0000","last_active":"2024-03-28T18:06:10.619171+0000","last_peered":"2024-03-28T18:06:10.619171+0000","last_clean":"2024-03-28T18:06:10.619171+0000","last_became_active":"2024-03-28T18:04:49.526351+0000","last_became_peered":"2024-03-28T18:04:49.526351+0000","last_unstale":"2024-03-28T18:06:10.619171+0000","last_undegraded":"2024-03-28T18:06:10.619171+0000","last_fullsized":"2024-03-28T18:06:10.619171+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_clean_scrub_stamp":"2024-03-28T18:04:48.465760+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T05:31:47.687345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"92'7","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-28T18:06:10.596535+0000","last_change":"2024-03-28T18:04:49.523346+0000","last_active":"2024-03-28T18:06:10.596535+0000","last_peered":"2024-03-28T18:06:10.596535+0000","last_clean":"2024-03-28T18:06:10.596535+0000","last_became_active":"2024-03-28T18:04:49.522041+0000","last_became_peered":"2024-03-28T18:04:49.522041+0000","last_unstale":"2024-03-28T18:06:10.596535+0000","last_undegraded":"2024-03-28T18:06:10.596535+0000","last_fullsized":"2024-03-28T18:06:10.596535+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_clean_scrub_stamp":"2024-03-28T18:04:48.465760+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T01:35:02.419328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"92'14","reported_seq":174,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-28T18:07:13.984992+0000","last_change":"2024-03-28T18:04:48.510917+0000","last_active":"2024-03-28T18:07:13.984992+0000","last_peered":"2024-03-28T18:07:13.984992+0000","last_clean":"2024-03-28T18:07:13.984992+0000","last_became_active":"2024-03-28T18:04:48.509978+0000","last_became_peered":"2024-03-28T18:04:48.509978+0000","last_unstale":"2024-03-28T18:07:13.984992+0000","last_undegraded":"2024-03-28T18:07:13.984992+0000","last_fullsized":"2024-03-28T18:07:13.984992+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:04:47.412563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:04:47.412563+0000","last_clean_scrub_stamp":"2024-03-28T18:04:47.412563+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T03:28:13.185099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":140,"num_read_kb":140,"num_write":15,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"92'5","reported_seq":23,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-28T18:06:10.603875+0000","last_change":"2024-03-28T18:04:49.535543+0000","last_active":"2024-03-28T18:06:10.603875+0000","last_peered":"2024-03-28T18:06:10.603875+0000","last_clean":"2024-03-28T18:06:10.603875+0000","last_became_active":"2024-03-28T18:04:49.535171+0000","last_became_peered":"2024-03-28T18:04:49.535171+0000","last_unstale":"2024-03-28T18:06:10.603875+0000","last_undegraded":"2024-03-28T18:06:10.603875+0000","last_fullsized":"2024-03-28T18:06:10.603875+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_clean_scrub_stamp":"2024-03-28T18:04:48.465760+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T22:12:22.258424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"90'2","reported_seq":22,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-28T18:05:11.741109+0000","last_change":"2024-03-28T18:04:48.503190+0000","last_active":"2024-03-28T18:05:11.741109+0000","last_peered":"2024-03-28T18:05:11.741109+0000","last_clean":"2024-03-28T18:05:11.741109+0000","last_became_active":"2024-03-28T18:04:48.501858+0000","last_became_peered":"2024-03-28T18:04:48.501858+0000","last_unstale":"2024-03-28T18:05:11.741109+0000","last_undegraded":"2024-03-28T18:05:11.741109+0000","last_fullsized":"2024-03-28T18:05:11.741109+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:04:47.412563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:04:47.412563+0000","last_clean_scrub_stamp":"2024-03-28T18:04:47.412563+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T19:56:43.878630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"92'6","reported_seq":24,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-28T18:06:10.625888+0000","last_change":"2024-03-28T18:04:49.537739+0000","last_active":"2024-03-28T18:06:10.625888+0000","last_peered":"2024-03-28T18:06:10.625888+0000","last_clean":"2024-03-28T18:06:10.625888+0000","last_became_active":"2024-03-28T18:04:49.535990+0000","last_became_peered":"2024-03-28T18:04:49.535990+0000","last_unstale":"2024-03-28T18:06:10.625888+0000","last_undegraded":"2024-03-28T18:06:10.625888+0000","last_fullsized":"2024-03-28T18:06:10.625888+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_clean_scrub_stamp":"2024-03-28T18:04:48.465760+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T21:54:15.729494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"90'12","reported_seq":27,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-28T18:05:11.768035+0000","last_change":"2024-03-28T18:05:11.767512+0000","last_active":"2024-03-28T18:05:11.768035+0000","last_peered":"2024-03-28T18:05:11.768035+0000","last_clean":"2024-03-28T18:05:11.768035+0000","last_became_active":"2024-03-28T18:05:11.752452+0000","last_became_peered":"2024-03-28T18:05:11.752452+0000","last_unstale":"2024-03-28T18:05:11.768035+0000","last_undegraded":"2024-03-28T18:05:11.768035+0000","last_fullsized":"2024-03-28T18:05:11.768035+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:04:47.412563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:04:47.412563+0000","last_clean_scrub_stamp":"2024-03-28T18:04:47.412563+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T00:09:13.980288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":0,"num_keys_recovered":10,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,7],"acting":[11,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1c","version":"92'9","reported_seq":26,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-28T18:06:10.623876+0000","last_change":"2024-03-28T18:04:49.529316+0000","last_active":"2024-03-28T18:06:10.623876+0000","last_peered":"2024-03-28T18:06:10.623876+0000","last_clean":"2024-03-28T18:06:10.623876+0000","last_became_active":"2024-03-28T18:04:49.528975+0000","last_became_peered":"2024-03-28T18:04:49.528975+0000","last_unstale":"2024-03-28T18:06:10.623876+0000","last_undegraded":"2024-03-28T18:06:10.623876+0000","last_fullsized":"2024-03-28T18:06:10.623876+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_clean_scrub_stamp":"2024-03-28T18:04:48.465760+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T00:06:45.268757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"90'5","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-28T18:05:11.734761+0000","last_change":"2024-03-28T18:04:48.515841+0000","last_active":"2024-03-28T18:05:11.734761+0000","last_peered":"2024-03-28T18:05:11.734761+0000","last_clean":"2024-03-28T18:05:11.734761+0000","last_became_active":"2024-03-28T18:04:48.513852+0000","last_became_peered":"2024-03-28T18:04:48.513852+0000","last_unstale":"2024-03-28T18:05:11.734761+0000","last_undegraded":"2024-03-28T18:05:11.734761+0000","last_fullsized":"2024-03-28T18:05:11.734761+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:04:47.412563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:04:47.412563+0000","last_clean_scrub_stamp":"2024-03-28T18:04:47.412563+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T00:07:14.751545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"92'15","reported_seq":32,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-28T18:06:10.609956+0000","last_change":"2024-03-28T18:04:49.542238+0000","last_active":"2024-03-28T18:06:10.609956+0000","last_peered":"2024-03-28T18:06:10.609956+0000","last_clean":"2024-03-28T18:06:10.609956+0000","last_became_active":"2024-03-28T18:04:49.539723+0000","last_became_peered":"2024-03-28T18:04:49.539723+0000","last_unstale":"2024-03-28T18:06:10.609956+0000","last_undegraded":"2024-03-28T18:06:10.609956+0000","last_fullsized":"2024-03-28T18:06:10.609956+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_clean_scrub_stamp":"2024-03-28T18:04:48.465760+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T22:45:07.310148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"92'8","reported_seq":170,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-28T18:07:14.022275+0000","last_change":"2024-03-28T18:04:48.511349+0000","last_active":"2024-03-28T18:07:14.022275+0000","last_peered":"2024-03-28T18:07:14.022275+0000","last_clean":"2024-03-28T18:07:14.022275+0000","last_became_active":"2024-03-28T18:04:48.510990+0000","last_became_peered":"2024-03-28T18:04:48.510990+0000","last_unstale":"2024-03-28T18:07:14.022275+0000","last_undegraded":"2024-03-28T18:07:14.022275+0000","last_fullsized":"2024-03-28T18:07:14.022275+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:04:47.412563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:04:47.412563+0000","last_clean_scrub_stamp":"2024-03-28T18:04:47.412563+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T21:56:15.411149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1647,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":143,"num_read_kb":4,"num_write":5,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"92'7","reported_seq":24,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-28T18:06:10.604922+0000","last_change":"2024-03-28T18:04:49.538066+0000","last_active":"2024-03-28T18:06:10.604922+0000","last_peered":"2024-03-28T18:06:10.604922+0000","last_clean":"2024-03-28T18:06:10.604922+0000","last_became_active":"2024-03-28T18:04:49.537202+0000","last_became_peered":"2024-03-28T18:04:49.537202+0000","last_unstale":"2024-03-28T18:06:10.604922+0000","last_undegraded":"2024-03-28T18:06:10.604922+0000","last_fullsized":"2024-03-28T18:06:10.604922+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_clean_scrub_stamp":"2024-03-28T18:04:48.465760+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T22:35:19.060311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"90'7","reported_seq":26,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-28T18:05:11.732185+0000","last_change":"2024-03-28T18:04:48.513722+0000","last_active":"2024-03-28T18:05:11.732185+0000","last_peered":"2024-03-28T18:05:11.732185+0000","last_clean":"2024-03-28T18:05:11.732185+0000","last_became_active":"2024-03-28T18:04:48.513048+0000","last_became_peered":"2024-03-28T18:04:48.513048+0000","last_unstale":"2024-03-28T18:05:11.732185+0000","last_undegraded":"2024-03-28T18:05:11.732185+0000","last_fullsized":"2024-03-28T18:05:11.732185+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:04:47.412563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:04:47.412563+0000","last_clean_scrub_stamp":"2024-03-28T18:04:47.412563+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T04:48:10.831297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"92'8","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-28T18:06:10.612752+0000","last_change":"2024-03-28T18:04:49.539172+0000","last_active":"2024-03-28T18:06:10.612752+0000","last_peered":"2024-03-28T18:06:10.612752+0000","last_clean":"2024-03-28T18:06:10.612752+0000","last_became_active":"2024-03-28T18:04:49.538779+0000","last_became_peered":"2024-03-28T18:04:49.538779+0000","last_unstale":"2024-03-28T18:06:10.612752+0000","last_undegraded":"2024-03-28T18:06:10.612752+0000","last_fullsized":"2024-03-28T18:06:10.612752+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_clean_scrub_stamp":"2024-03-28T18:04:48.465760+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T04:59:08.361762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"90'3","reported_seq":23,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-28T18:05:11.950057+0000","last_change":"2024-03-28T18:04:48.500471+0000","last_active":"2024-03-28T18:05:11.950057+0000","last_peered":"2024-03-28T18:05:11.950057+0000","last_clean":"2024-03-28T18:05:11.950057+0000","last_became_active":"2024-03-28T18:04:48.500094+0000","last_became_peered":"2024-03-28T18:04:48.500094+0000","last_unstale":"2024-03-28T18:05:11.950057+0000","last_undegraded":"2024-03-28T18:05:11.950057+0000","last_fullsized":"2024-03-28T18:05:11.950057+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:04:47.412563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:04:47.412563+0000","last_clean_scrub_stamp":"2024-03-28T18:04:47.412563+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T21:52:51.964254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"92'8","reported_seq":26,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-28T18:06:10.600363+0000","last_change":"2024-03-28T18:04:49.522120+0000","last_active":"2024-03-28T18:06:10.600363+0000","last_peered":"2024-03-28T18:06:10.600363+0000","last_clean":"2024-03-28T18:06:10.600363+0000","last_became_active":"2024-03-28T18:04:49.521656+0000","last_became_peered":"2024-03-28T18:04:49.521656+0000","last_unstale":"2024-03-28T18:06:10.600363+0000","last_undegraded":"2024-03-28T18:06:10.600363+0000","last_fullsized":"2024-03-28T18:06:10.600363+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_clean_scrub_stamp":"2024-03-28T18:04:48.465760+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T18:29:55.605765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"90'3","reported_seq":24,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-28T18:05:11.741130+0000","last_change":"2024-03-28T18:04:48.508723+0000","last_active":"2024-03-28T18:05:11.741130+0000","last_peered":"2024-03-28T18:05:11.741130+0000","last_clean":"2024-03-28T18:05:11.741130+0000","last_became_active":"2024-03-28T18:04:48.508314+0000","last_became_peered":"2024-03-28T18:04:48.508314+0000","last_unstale":"2024-03-28T18:05:11.741130+0000","last_undegraded":"2024-03-28T18:05:11.741130+0000","last_fullsized":"2024-03-28T18:05:11.741130+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:04:47.412563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:04:47.412563+0000","last_clean_scrub_stamp":"2024-03-28T18:04:47.412563+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T03:03:59.353454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"92'11","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-28T18:06:10.611886+0000","last_change":"2024-03-28T18:04:49.531469+0000","last_active":"2024-03-28T18:06:10.611886+0000","last_peered":"2024-03-28T18:06:10.611886+0000","last_clean":"2024-03-28T18:06:10.611886+0000","last_became_active":"2024-03-28T18:04:49.529263+0000","last_became_peered":"2024-03-28T18:04:49.529263+0000","last_unstale":"2024-03-28T18:06:10.611886+0000","last_undegraded":"2024-03-28T18:06:10.611886+0000","last_fullsized":"2024-03-28T18:06:10.611886+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_clean_scrub_stamp":"2024-03-28T18:04:48.465760+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T05:46:31.593877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"92'7","reported_seq":305,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-28T18:07:15.023693+0000","last_change":"2024-03-28T18:04:48.500644+0000","last_active":"2024-03-28T18:07:15.023693+0000","last_peered":"2024-03-28T18:07:15.023693+0000","last_clean":"2024-03-28T18:07:15.023693+0000","last_became_active":"2024-03-28T18:04:48.499330+0000","last_became_peered":"2024-03-28T18:04:48.499330+0000","last_unstale":"2024-03-28T18:07:15.023693+0000","last_undegraded":"2024-03-28T18:07:15.023693+0000","last_fullsized":"2024-03-28T18:07:15.023693+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:04:47.412563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:04:47.412563+0000","last_clean_scrub_stamp":"2024-03-28T18:04:47.412563+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T18:50:54.361266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":278,"num_read_kb":278,"num_write":8,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"92'10","reported_seq":28,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-28T18:06:10.623432+0000","last_change":"2024-03-28T18:04:49.528728+0000","last_active":"2024-03-28T18:06:10.623432+0000","last_peered":"2024-03-28T18:06:10.623432+0000","last_clean":"2024-03-28T18:06:10.623432+0000","last_became_active":"2024-03-28T18:04:49.528312+0000","last_became_peered":"2024-03-28T18:04:49.528312+0000","last_unstale":"2024-03-28T18:06:10.623432+0000","last_undegraded":"2024-03-28T18:06:10.623432+0000","last_fullsized":"2024-03-28T18:06:10.623432+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_clean_scrub_stamp":"2024-03-28T18:04:48.465760+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T05:59:37.308002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"90'4","reported_seq":24,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-28T18:05:11.736913+0000","last_change":"2024-03-28T18:04:48.508766+0000","last_active":"2024-03-28T18:05:11.736913+0000","last_peered":"2024-03-28T18:05:11.736913+0000","last_clean":"2024-03-28T18:05:11.736913+0000","last_became_active":"2024-03-28T18:04:48.506780+0000","last_became_peered":"2024-03-28T18:04:48.506780+0000","last_unstale":"2024-03-28T18:05:11.736913+0000","last_undegraded":"2024-03-28T18:05:11.736913+0000","last_fullsized":"2024-03-28T18:05:11.736913+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:04:47.412563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:04:47.412563+0000","last_clean_scrub_stamp":"2024-03-28T18:04:47.412563+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T04:39:07.394655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"92'9","reported_seq":27,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-28T18:06:10.610898+0000","last_change":"2024-03-28T18:04:49.537165+0000","last_active":"2024-03-28T18:06:10.610898+0000","last_peered":"2024-03-28T18:06:10.610898+0000","last_clean":"2024-03-28T18:06:10.610898+0000","last_became_active":"2024-03-28T18:04:49.534543+0000","last_became_peered":"2024-03-28T18:04:49.534543+0000","last_unstale":"2024-03-28T18:06:10.610898+0000","last_undegraded":"2024-03-28T18:06:10.610898+0000","last_fullsized":"2024-03-28T18:06:10.610898+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_clean_scrub_stamp":"2024-03-28T18:04:48.465760+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T03:15:58.056148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"90'5","reported_seq":26,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-28T18:05:11.746431+0000","last_change":"2024-03-28T18:04:48.503866+0000","last_active":"2024-03-28T18:05:11.746431+0000","last_peered":"2024-03-28T18:05:11.746431+0000","last_clean":"2024-03-28T18:05:11.746431+0000","last_became_active":"2024-03-28T18:04:48.503223+0000","last_became_peered":"2024-03-28T18:04:48.503223+0000","last_unstale":"2024-03-28T18:05:11.746431+0000","last_undegraded":"2024-03-28T18:05:11.746431+0000","last_fullsized":"2024-03-28T18:05:11.746431+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:04:47.412563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:04:47.412563+0000","last_clean_scrub_stamp":"2024-03-28T18:04:47.412563+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T20:14:01.307483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"92'11","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-28T18:06:10.624750+0000","last_change":"2024-03-28T18:04:49.539494+0000","last_active":"2024-03-28T18:06:10.624750+0000","last_peered":"2024-03-28T18:06:10.624750+0000","last_clean":"2024-03-28T18:06:10.624750+0000","last_became_active":"2024-03-28T18:04:49.539044+0000","last_became_peered":"2024-03-28T18:04:49.539044+0000","last_unstale":"2024-03-28T18:06:10.624750+0000","last_undegraded":"2024-03-28T18:06:10.624750+0000","last_fullsized":"2024-03-28T18:06:10.624750+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_clean_scrub_stamp":"2024-03-28T18:04:48.465760+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T04:08:30.807788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"92'34","reported_seq":341,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-28T18:07:15.010701+0000","last_change":"2024-03-28T18:04:48.507703+0000","last_active":"2024-03-28T18:07:15.010701+0000","last_peered":"2024-03-28T18:07:15.010701+0000","last_clean":"2024-03-28T18:07:15.010701+0000","last_became_active":"2024-03-28T18:04:48.507310+0000","last_became_peered":"2024-03-28T18:04:48.507310+0000","last_unstale":"2024-03-28T18:07:15.010701+0000","last_undegraded":"2024-03-28T18:07:15.010701+0000","last_fullsized":"2024-03-28T18:07:15.010701+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:04:47.412563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:04:47.412563+0000","last_clean_scrub_stamp":"2024-03-28T18:04:47.412563+0000","objects_scrubbed":0,"log_size":34,"log_dups_size":0,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T19:14:13.910042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":64747,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":287,"num_read_kb":207,"num_write":36,"num_write_kb":77,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"92'6","reported_seq":24,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-28T18:06:10.618389+0000","last_change":"2024-03-28T18:04:49.529928+0000","last_active":"2024-03-28T18:06:10.618389+0000","last_peered":"2024-03-28T18:06:10.618389+0000","last_clean":"2024-03-28T18:06:10.618389+0000","last_became_active":"2024-03-28T18:04:49.529527+0000","last_became_peered":"2024-03-28T18:04:49.529527+0000","last_unstale":"2024-03-28T18:06:10.618389+0000","last_undegraded":"2024-03-28T18:06:10.618389+0000","last_fullsized":"2024-03-28T18:06:10.618389+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_clean_scrub_stamp":"2024-03-28T18:04:48.465760+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T05:50:07.129140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"90'9","reported_seq":31,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-28T18:05:11.749738+0000","last_change":"2024-03-28T18:04:48.494746+0000","last_active":"2024-03-28T18:05:11.749738+0000","last_peered":"2024-03-28T18:05:11.749738+0000","last_clean":"2024-03-28T18:05:11.749738+0000","last_became_active":"2024-03-28T18:04:48.494296+0000","last_became_peered":"2024-03-28T18:04:48.494296+0000","last_unstale":"2024-03-28T18:05:11.749738+0000","last_undegraded":"2024-03-28T18:05:11.749738+0000","last_fullsized":"2024-03-28T18:05:11.749738+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:04:47.412563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:04:47.412563+0000","last_clean_scrub_stamp":"2024-03-28T18:04:47.412563+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T22:32:56.923067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"92'9","reported_seq":27,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-28T18:06:10.624303+0000","last_change":"2024-03-28T18:04:49.528070+0000","last_active":"2024-03-28T18:06:10.624303+0000","last_peered":"2024-03-28T18:06:10.624303+0000","last_clean":"2024-03-28T18:06:10.624303+0000","last_became_active":"2024-03-28T18:04:49.526747+0000","last_became_peered":"2024-03-28T18:04:49.526747+0000","last_unstale":"2024-03-28T18:06:10.624303+0000","last_undegraded":"2024-03-28T18:06:10.624303+0000","last_fullsized":"2024-03-28T18:06:10.624303+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_clean_scrub_stamp":"2024-03-28T18:04:48.465760+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T22:32:05.750094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"90'5","reported_seq":26,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-28T18:05:11.741413+0000","last_change":"2024-03-28T18:04:48.505723+0000","last_active":"2024-03-28T18:05:11.741413+0000","last_peered":"2024-03-28T18:05:11.741413+0000","last_clean":"2024-03-28T18:05:11.741413+0000","last_became_active":"2024-03-28T18:04:48.504088+0000","last_became_peered":"2024-03-28T18:04:48.504088+0000","last_unstale":"2024-03-28T18:05:11.741413+0000","last_undegraded":"2024-03-28T18:05:11.741413+0000","last_fullsized":"2024-03-28T18:05:11.741413+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:04:47.412563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:04:47.412563+0000","last_clean_scrub_stamp":"2024-03-28T18:04:47.412563+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T23:25:37.138054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"92'15","reported_seq":33,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-28T18:06:10.625240+0000","last_change":"2024-03-28T18:04:49.518872+0000","last_active":"2024-03-28T18:06:10.625240+0000","last_peered":"2024-03-28T18:06:10.625240+0000","last_clean":"2024-03-28T18:06:10.625240+0000","last_became_active":"2024-03-28T18:04:49.518507+0000","last_became_peered":"2024-03-28T18:04:49.518507+0000","last_unstale":"2024-03-28T18:06:10.625240+0000","last_undegraded":"2024-03-28T18:06:10.625240+0000","last_fullsized":"2024-03-28T18:06:10.625240+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_clean_scrub_stamp":"2024-03-28T18:04:48.465760+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T20:11:31.044321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"92'8","reported_seq":171,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-28T18:07:14.021044+0000","last_change":"2024-03-28T18:04:48.503058+0000","last_active":"2024-03-28T18:07:14.021044+0000","last_peered":"2024-03-28T18:07:14.021044+0000","last_clean":"2024-03-28T18:07:14.021044+0000","last_became_active":"2024-03-28T18:04:48.501596+0000","last_became_peered":"2024-03-28T18:04:48.501596+0000","last_unstale":"2024-03-28T18:07:14.021044+0000","last_undegraded":"2024-03-28T18:07:14.021044+0000","last_fullsized":"2024-03-28T18:07:14.021044+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:04:47.412563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:04:47.412563+0000","last_clean_scrub_stamp":"2024-03-28T18:04:47.412563+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T23:46:10.442771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1613,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":144,"num_read_kb":5,"num_write":8,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"92'6","reported_seq":24,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-28T18:06:10.614449+0000","last_change":"2024-03-28T18:04:49.535102+0000","last_active":"2024-03-28T18:06:10.614449+0000","last_peered":"2024-03-28T18:06:10.614449+0000","last_clean":"2024-03-28T18:06:10.614449+0000","last_became_active":"2024-03-28T18:04:49.534171+0000","last_became_peered":"2024-03-28T18:04:49.534171+0000","last_unstale":"2024-03-28T18:06:10.614449+0000","last_undegraded":"2024-03-28T18:06:10.614449+0000","last_fullsized":"2024-03-28T18:06:10.614449+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_clean_scrub_stamp":"2024-03-28T18:04:48.465760+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T18:16:27.703327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"90'17","reported_seq":38,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-28T18:05:11.734330+0000","last_change":"2024-03-28T18:04:48.517034+0000","last_active":"2024-03-28T18:05:11.734330+0000","last_peered":"2024-03-28T18:05:11.734330+0000","last_clean":"2024-03-28T18:05:11.734330+0000","last_became_active":"2024-03-28T18:04:48.516562+0000","last_became_peered":"2024-03-28T18:04:48.516562+0000","last_unstale":"2024-03-28T18:05:11.734330+0000","last_undegraded":"2024-03-28T18:05:11.734330+0000","last_fullsized":"2024-03-28T18:05:11.734330+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:04:47.412563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:04:47.412563+0000","last_clean_scrub_stamp":"2024-03-28T18:04:47.412563+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T02:28:55.753779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"92'6","reported_seq":24,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-28T18:06:10.617799+0000","last_change":"2024-03-28T18:04:49.541501+0000","last_active":"2024-03-28T18:06:10.617799+0000","last_peered":"2024-03-28T18:06:10.617799+0000","last_clean":"2024-03-28T18:06:10.617799+0000","last_became_active":"2024-03-28T18:04:49.541117+0000","last_became_peered":"2024-03-28T18:04:49.541117+0000","last_unstale":"2024-03-28T18:06:10.617799+0000","last_undegraded":"2024-03-28T18:06:10.617799+0000","last_fullsized":"2024-03-28T18:06:10.617799+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_clean_scrub_stamp":"2024-03-28T18:04:48.465760+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T04:18:29.081489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"92'3","reported_seq":24,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-28T18:07:03.790727+0000","last_change":"2024-03-28T18:04:48.510859+0000","last_active":"2024-03-28T18:07:03.790727+0000","last_peered":"2024-03-28T18:07:03.790727+0000","last_clean":"2024-03-28T18:07:03.790727+0000","last_became_active":"2024-03-28T18:04:48.510522+0000","last_became_peered":"2024-03-28T18:04:48.510522+0000","last_unstale":"2024-03-28T18:07:03.790727+0000","last_undegraded":"2024-03-28T18:07:03.790727+0000","last_fullsized":"2024-03-28T18:07:03.790727+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:04:47.412563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:04:47.412563+0000","last_clean_scrub_stamp":"2024-03-28T18:04:47.412563+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T22:31:37.817361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"92'5","reported_seq":23,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-28T18:06:10.607186+0000","last_change":"2024-03-28T18:04:49.541981+0000","last_active":"2024-03-28T18:06:10.607186+0000","last_peered":"2024-03-28T18:06:10.607186+0000","last_clean":"2024-03-28T18:06:10.607186+0000","last_became_active":"2024-03-28T18:04:49.541606+0000","last_became_peered":"2024-03-28T18:04:49.541606+0000","last_unstale":"2024-03-28T18:06:10.607186+0000","last_undegraded":"2024-03-28T18:06:10.607186+0000","last_fullsized":"2024-03-28T18:06:10.607186+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_clean_scrub_stamp":"2024-03-28T18:04:48.465760+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T21:56:03.632490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"90'1","reported_seq":21,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-28T18:05:11.730922+0000","last_change":"2024-03-28T18:04:48.505847+0000","last_active":"2024-03-28T18:05:11.730922+0000","last_peered":"2024-03-28T18:05:11.730922+0000","last_clean":"2024-03-28T18:05:11.730922+0000","last_became_active":"2024-03-28T18:04:48.501005+0000","last_became_peered":"2024-03-28T18:04:48.501005+0000","last_unstale":"2024-03-28T18:05:11.730922+0000","last_undegraded":"2024-03-28T18:05:11.730922+0000","last_fullsized":"2024-03-28T18:05:11.730922+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:04:47.412563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:04:47.412563+0000","last_clean_scrub_stamp":"2024-03-28T18:04:47.412563+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T01:33:45.539324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"92'5","reported_seq":23,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-28T18:06:10.614333+0000","last_change":"2024-03-28T18:04:49.526938+0000","last_active":"2024-03-28T18:06:10.614333+0000","last_peered":"2024-03-28T18:06:10.614333+0000","last_clean":"2024-03-28T18:06:10.614333+0000","last_became_active":"2024-03-28T18:04:49.526290+0000","last_became_peered":"2024-03-28T18:04:49.526290+0000","last_unstale":"2024-03-28T18:06:10.614333+0000","last_undegraded":"2024-03-28T18:06:10.614333+0000","last_fullsized":"2024-03-28T18:06:10.614333+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_clean_scrub_stamp":"2024-03-28T18:04:48.465760+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T03:06:54.561470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"92'6","reported_seq":169,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-28T18:07:15.990542+0000","last_change":"2024-03-28T18:04:48.512553+0000","last_active":"2024-03-28T18:07:15.990542+0000","last_peered":"2024-03-28T18:07:15.990542+0000","last_clean":"2024-03-28T18:07:15.990542+0000","last_became_active":"2024-03-28T18:04:48.511520+0000","last_became_peered":"2024-03-28T18:04:48.511520+0000","last_unstale":"2024-03-28T18:07:15.990542+0000","last_undegraded":"2024-03-28T18:07:15.990542+0000","last_fullsized":"2024-03-28T18:07:15.990542+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:04:47.412563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:04:47.412563+0000","last_clean_scrub_stamp":"2024-03-28T18:04:47.412563+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T21:13:32.970027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1081,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":143,"num_read_kb":2,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"92'7","reported_seq":31,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-28T18:06:10.613152+0000","last_change":"2024-03-28T18:05:11.739596+0000","last_active":"2024-03-28T18:06:10.613152+0000","last_peered":"2024-03-28T18:06:10.613152+0000","last_clean":"2024-03-28T18:06:10.613152+0000","last_became_active":"2024-03-28T18:05:11.739156+0000","last_became_peered":"2024-03-28T18:05:11.739156+0000","last_unstale":"2024-03-28T18:06:10.613152+0000","last_undegraded":"2024-03-28T18:06:10.613152+0000","last_fullsized":"2024-03-28T18:06:10.613152+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_clean_scrub_stamp":"2024-03-28T18:04:48.465760+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T05:23:00.446713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,0],"acting":[10,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"92'8","reported_seq":26,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-28T18:06:10.609498+0000","last_change":"2024-03-28T18:04:49.540438+0000","last_active":"2024-03-28T18:06:10.609498+0000","last_peered":"2024-03-28T18:06:10.609498+0000","last_clean":"2024-03-28T18:06:10.609498+0000","last_became_active":"2024-03-28T18:04:49.540069+0000","last_became_peered":"2024-03-28T18:04:49.540069+0000","last_unstale":"2024-03-28T18:06:10.609498+0000","last_undegraded":"2024-03-28T18:06:10.609498+0000","last_fullsized":"2024-03-28T18:06:10.609498+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_clean_scrub_stamp":"2024-03-28T18:04:48.465760+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T19:31:49.254101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"92'7","reported_seq":167,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-28T18:07:14.020116+0000","last_change":"2024-03-28T18:04:48.508839+0000","last_active":"2024-03-28T18:07:14.020116+0000","last_peered":"2024-03-28T18:07:14.020116+0000","last_clean":"2024-03-28T18:07:14.020116+0000","last_became_active":"2024-03-28T18:04:48.508475+0000","last_became_peered":"2024-03-28T18:04:48.508475+0000","last_unstale":"2024-03-28T18:07:14.020116+0000","last_undegraded":"2024-03-28T18:07:14.020116+0000","last_fullsized":"2024-03-28T18:07:14.020116+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:04:47.412563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:04:47.412563+0000","last_clean_scrub_stamp":"2024-03-28T18:04:47.412563+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T04:31:10.463284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":140,"num_read_kb":140,"num_write":7,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"92'8","reported_seq":26,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-28T18:06:10.621737+0000","last_change":"2024-03-28T18:04:49.534449+0000","last_active":"2024-03-28T18:06:10.621737+0000","last_peered":"2024-03-28T18:06:10.621737+0000","last_clean":"2024-03-28T18:06:10.621737+0000","last_became_active":"2024-03-28T18:04:49.534098+0000","last_became_peered":"2024-03-28T18:04:49.534098+0000","last_unstale":"2024-03-28T18:06:10.621737+0000","last_undegraded":"2024-03-28T18:06:10.621737+0000","last_fullsized":"2024-03-28T18:06:10.621737+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_clean_scrub_stamp":"2024-03-28T18:04:48.465760+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T23:47:07.886931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"90'6","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-28T18:05:11.948432+0000","last_change":"2024-03-28T18:04:48.501790+0000","last_active":"2024-03-28T18:05:11.948432+0000","last_peered":"2024-03-28T18:05:11.948432+0000","last_clean":"2024-03-28T18:05:11.948432+0000","last_became_active":"2024-03-28T18:04:48.501415+0000","last_became_peered":"2024-03-28T18:04:48.501415+0000","last_unstale":"2024-03-28T18:05:11.948432+0000","last_undegraded":"2024-03-28T18:05:11.948432+0000","last_fullsized":"2024-03-28T18:05:11.948432+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:04:47.412563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:04:47.412563+0000","last_clean_scrub_stamp":"2024-03-28T18:04:47.412563+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T19:17:22.319883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"22'137","reported_seq":366,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-28T18:05:11.735054+0000","last_change":"2024-03-28T18:03:13.522588+0000","last_active":"2024-03-28T18:05:11.735054+0000","last_peered":"2024-03-28T18:05:11.735054+0000","last_clean":"2024-03-28T18:05:11.735054+0000","last_became_active":"2024-03-28T18:03:13.521185+0000","last_became_peered":"2024-03-28T18:03:13.521185+0000","last_unstale":"2024-03-28T18:05:11.735054+0000","last_undegraded":"2024-03-28T18:05:11.735054+0000","last_fullsized":"2024-03-28T18:05:11.735054+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T17:58:33.169981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T17:58:33.169981+0000","last_clean_scrub_stamp":"2024-03-28T17:58:33.169981+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T19:10:06.087374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"92'5","reported_seq":23,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-28T18:06:10.607609+0000","last_change":"2024-03-28T18:04:49.538619+0000","last_active":"2024-03-28T18:06:10.607609+0000","last_peered":"2024-03-28T18:06:10.607609+0000","last_clean":"2024-03-28T18:06:10.607609+0000","last_became_active":"2024-03-28T18:04:49.538276+0000","last_became_peered":"2024-03-28T18:04:49.538276+0000","last_unstale":"2024-03-28T18:06:10.607609+0000","last_undegraded":"2024-03-28T18:06:10.607609+0000","last_fullsized":"2024-03-28T18:06:10.607609+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_clean_scrub_stamp":"2024-03-28T18:04:48.465760+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T01:39:37.139544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"90'4","reported_seq":24,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-28T18:05:11.735222+0000","last_change":"2024-03-28T18:04:48.513621+0000","last_active":"2024-03-28T18:05:11.735222+0000","last_peered":"2024-03-28T18:05:11.735222+0000","last_clean":"2024-03-28T18:05:11.735222+0000","last_became_active":"2024-03-28T18:04:48.513019+0000","last_became_peered":"2024-03-28T18:04:48.513019+0000","last_unstale":"2024-03-28T18:05:11.735222+0000","last_undegraded":"2024-03-28T18:05:11.735222+0000","last_fullsized":"2024-03-28T18:05:11.735222+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:04:47.412563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:04:47.412563+0000","last_clean_scrub_stamp":"2024-03-28T18:04:47.412563+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T23:38:18.082830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"90'4","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-28T18:05:11.735056+0000","last_change":"2024-03-28T18:04:48.513999+0000","last_active":"2024-03-28T18:05:11.735056+0000","last_peered":"2024-03-28T18:05:11.735056+0000","last_clean":"2024-03-28T18:05:11.735056+0000","last_became_active":"2024-03-28T18:04:48.512836+0000","last_became_peered":"2024-03-28T18:04:48.512836+0000","last_unstale":"2024-03-28T18:05:11.735056+0000","last_undegraded":"2024-03-28T18:05:11.735056+0000","last_fullsized":"2024-03-28T18:05:11.735056+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:04:47.412563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:04:47.412563+0000","last_clean_scrub_stamp":"2024-03-28T18:04:47.412563+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T18:29:42.015843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"92'9","reported_seq":27,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-28T18:06:10.605559+0000","last_change":"2024-03-28T18:04:49.519323+0000","last_active":"2024-03-28T18:06:10.605559+0000","last_peered":"2024-03-28T18:06:10.605559+0000","last_clean":"2024-03-28T18:06:10.605559+0000","last_became_active":"2024-03-28T18:04:49.518898+0000","last_became_peered":"2024-03-28T18:04:49.518898+0000","last_unstale":"2024-03-28T18:06:10.605559+0000","last_undegraded":"2024-03-28T18:06:10.605559+0000","last_fullsized":"2024-03-28T18:06:10.605559+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_clean_scrub_stamp":"2024-03-28T18:04:48.465760+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T18:41:12.683606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"92'16","reported_seq":36,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-28T18:07:11.878264+0000","last_change":"2024-03-28T18:04:48.512673+0000","last_active":"2024-03-28T18:07:11.878264+0000","last_peered":"2024-03-28T18:07:11.878264+0000","last_clean":"2024-03-28T18:07:11.878264+0000","last_became_active":"2024-03-28T18:04:48.512137+0000","last_became_peered":"2024-03-28T18:04:48.512137+0000","last_unstale":"2024-03-28T18:07:11.878264+0000","last_undegraded":"2024-03-28T18:07:11.878264+0000","last_fullsized":"2024-03-28T18:07:11.878264+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:04:47.412563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:04:47.412563+0000","last_clean_scrub_stamp":"2024-03-28T18:04:47.412563+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T21:29:41.731701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":28,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"92'5","reported_seq":23,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-28T18:06:10.624351+0000","last_change":"2024-03-28T18:04:49.525055+0000","last_active":"2024-03-28T18:06:10.624351+0000","last_peered":"2024-03-28T18:06:10.624351+0000","last_clean":"2024-03-28T18:06:10.624351+0000","last_became_active":"2024-03-28T18:04:49.522038+0000","last_became_peered":"2024-03-28T18:04:49.522038+0000","last_unstale":"2024-03-28T18:06:10.624351+0000","last_undegraded":"2024-03-28T18:06:10.624351+0000","last_fullsized":"2024-03-28T18:06:10.624351+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_clean_scrub_stamp":"2024-03-28T18:04:48.465760+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T05:21:44.970397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"90'7","reported_seq":28,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-28T18:05:11.949110+0000","last_change":"2024-03-28T18:04:48.509951+0000","last_active":"2024-03-28T18:05:11.949110+0000","last_peered":"2024-03-28T18:05:11.949110+0000","last_clean":"2024-03-28T18:05:11.949110+0000","last_became_active":"2024-03-28T18:04:48.509141+0000","last_became_peered":"2024-03-28T18:04:48.509141+0000","last_unstale":"2024-03-28T18:05:11.949110+0000","last_undegraded":"2024-03-28T18:05:11.949110+0000","last_fullsized":"2024-03-28T18:05:11.949110+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:04:47.412563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:04:47.412563+0000","last_clean_scrub_stamp":"2024-03-28T18:04:47.412563+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T03:54:05.569940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"92'7","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-28T18:06:10.612768+0000","last_change":"2024-03-28T18:04:49.537174+0000","last_active":"2024-03-28T18:06:10.612768+0000","last_peered":"2024-03-28T18:06:10.612768+0000","last_clean":"2024-03-28T18:06:10.612768+0000","last_became_active":"2024-03-28T18:04:49.536801+0000","last_became_peered":"2024-03-28T18:04:49.536801+0000","last_unstale":"2024-03-28T18:06:10.612768+0000","last_undegraded":"2024-03-28T18:06:10.612768+0000","last_fullsized":"2024-03-28T18:06:10.612768+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_clean_scrub_stamp":"2024-03-28T18:04:48.465760+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T05:27:55.908196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"90'2","reported_seq":32,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-28T18:05:11.757387+0000","last_change":"2024-03-28T18:05:11.756894+0000","last_active":"2024-03-28T18:05:11.757387+0000","last_peered":"2024-03-28T18:05:11.757387+0000","last_clean":"2024-03-28T18:05:11.757387+0000","last_became_active":"2024-03-28T18:05:11.750825+0000","last_became_peered":"2024-03-28T18:05:11.750825+0000","last_unstale":"2024-03-28T18:05:11.757387+0000","last_undegraded":"2024-03-28T18:05:11.757387+0000","last_fullsized":"2024-03-28T18:05:11.757387+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:04:47.412563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:04:47.412563+0000","last_clean_scrub_stamp":"2024-03-28T18:04:47.412563+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T23:11:42.105850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,11,0],"acting":[1,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"92'2","reported_seq":20,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-28T18:06:10.611454+0000","last_change":"2024-03-28T18:04:49.523970+0000","last_active":"2024-03-28T18:06:10.611454+0000","last_peered":"2024-03-28T18:06:10.611454+0000","last_clean":"2024-03-28T18:06:10.611454+0000","last_became_active":"2024-03-28T18:04:49.523559+0000","last_became_peered":"2024-03-28T18:04:49.523559+0000","last_unstale":"2024-03-28T18:06:10.611454+0000","last_undegraded":"2024-03-28T18:06:10.611454+0000","last_fullsized":"2024-03-28T18:06:10.611454+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_clean_scrub_stamp":"2024-03-28T18:04:48.465760+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T05:50:25.593232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"90'7","reported_seq":27,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-28T18:05:11.739263+0000","last_change":"2024-03-28T18:04:48.492067+0000","last_active":"2024-03-28T18:05:11.739263+0000","last_peered":"2024-03-28T18:05:11.739263+0000","last_clean":"2024-03-28T18:05:11.739263+0000","last_became_active":"2024-03-28T18:04:48.490633+0000","last_became_peered":"2024-03-28T18:04:48.490633+0000","last_unstale":"2024-03-28T18:05:11.739263+0000","last_undegraded":"2024-03-28T18:05:11.739263+0000","last_fullsized":"2024-03-28T18:05:11.739263+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:04:47.412563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:04:47.412563+0000","last_clean_scrub_stamp":"2024-03-28T18:04:47.412563+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T23:00:52.703618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"92'9","reported_seq":27,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-28T18:06:10.608650+0000","last_change":"2024-03-28T18:04:49.531554+0000","last_active":"2024-03-28T18:06:10.608650+0000","last_peered":"2024-03-28T18:06:10.608650+0000","last_clean":"2024-03-28T18:06:10.608650+0000","last_became_active":"2024-03-28T18:04:49.530273+0000","last_became_peered":"2024-03-28T18:04:49.530273+0000","last_unstale":"2024-03-28T18:06:10.608650+0000","last_undegraded":"2024-03-28T18:06:10.608650+0000","last_fullsized":"2024-03-28T18:06:10.608650+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_clean_scrub_stamp":"2024-03-28T18:04:48.465760+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T22:17:04.032614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"92'5","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-28T18:07:11.597743+0000","last_change":"2024-03-28T18:04:48.514974+0000","last_active":"2024-03-28T18:07:11.597743+0000","last_peered":"2024-03-28T18:07:11.597743+0000","last_clean":"2024-03-28T18:07:11.597743+0000","last_became_active":"2024-03-28T18:04:48.514356+0000","last_became_peered":"2024-03-28T18:04:48.514356+0000","last_unstale":"2024-03-28T18:07:11.597743+0000","last_undegraded":"2024-03-28T18:07:11.597743+0000","last_fullsized":"2024-03-28T18:07:11.597743+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:04:47.412563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:04:47.412563+0000","last_clean_scrub_stamp":"2024-03-28T18:04:47.412563+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T01:26:37.202440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"92'8","reported_seq":26,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-28T18:06:10.613171+0000","last_change":"2024-03-28T18:04:49.518064+0000","last_active":"2024-03-28T18:06:10.613171+0000","last_peered":"2024-03-28T18:06:10.613171+0000","last_clean":"2024-03-28T18:06:10.613171+0000","last_became_active":"2024-03-28T18:04:49.517674+0000","last_became_peered":"2024-03-28T18:04:49.517674+0000","last_unstale":"2024-03-28T18:06:10.613171+0000","last_undegraded":"2024-03-28T18:06:10.613171+0000","last_fullsized":"2024-03-28T18:06:10.613171+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_clean_scrub_stamp":"2024-03-28T18:04:48.465760+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T00:10:44.537164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"90'4","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-28T18:05:11.734862+0000","last_change":"2024-03-28T18:04:48.517028+0000","last_active":"2024-03-28T18:05:11.734862+0000","last_peered":"2024-03-28T18:05:11.734862+0000","last_clean":"2024-03-28T18:05:11.734862+0000","last_became_active":"2024-03-28T18:04:48.516387+0000","last_became_peered":"2024-03-28T18:04:48.516387+0000","last_unstale":"2024-03-28T18:05:11.734862+0000","last_undegraded":"2024-03-28T18:05:11.734862+0000","last_fullsized":"2024-03-28T18:05:11.734862+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:04:47.412563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:04:47.412563+0000","last_clean_scrub_stamp":"2024-03-28T18:04:47.412563+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T01:09:24.142422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"92'5","reported_seq":23,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-28T18:06:10.625821+0000","last_change":"2024-03-28T18:04:49.527032+0000","last_active":"2024-03-28T18:06:10.625821+0000","last_peered":"2024-03-28T18:06:10.625821+0000","last_clean":"2024-03-28T18:06:10.625821+0000","last_became_active":"2024-03-28T18:04:49.523888+0000","last_became_peered":"2024-03-28T18:04:49.523888+0000","last_unstale":"2024-03-28T18:06:10.625821+0000","last_undegraded":"2024-03-28T18:06:10.625821+0000","last_fullsized":"2024-03-28T18:06:10.625821+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_clean_scrub_stamp":"2024-03-28T18:04:48.465760+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T01:19:01.265719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"92'6","reported_seq":169,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-28T18:07:15.011653+0000","last_change":"2024-03-28T18:04:48.502638+0000","last_active":"2024-03-28T18:07:15.011653+0000","last_peered":"2024-03-28T18:07:15.011653+0000","last_clean":"2024-03-28T18:07:15.011653+0000","last_became_active":"2024-03-28T18:04:48.502249+0000","last_became_peered":"2024-03-28T18:04:48.502249+0000","last_unstale":"2024-03-28T18:07:15.011653+0000","last_undegraded":"2024-03-28T18:07:15.011653+0000","last_fullsized":"2024-03-28T18:07:15.011653+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:04:47.412563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:04:47.412563+0000","last_clean_scrub_stamp":"2024-03-28T18:04:47.412563+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T03:23:51.740832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1663,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":143,"num_read_kb":3,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"92'7","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-28T18:06:10.626385+0000","last_change":"2024-03-28T18:04:49.532802+0000","last_active":"2024-03-28T18:06:10.626385+0000","last_peered":"2024-03-28T18:06:10.626385+0000","last_clean":"2024-03-28T18:06:10.626385+0000","last_became_active":"2024-03-28T18:04:49.530666+0000","last_became_peered":"2024-03-28T18:04:49.530666+0000","last_unstale":"2024-03-28T18:06:10.626385+0000","last_undegraded":"2024-03-28T18:06:10.626385+0000","last_fullsized":"2024-03-28T18:06:10.626385+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_clean_scrub_stamp":"2024-03-28T18:04:48.465760+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T22:21:28.469402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"90'3","reported_seq":24,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-28T18:05:11.747786+0000","last_change":"2024-03-28T18:04:48.503582+0000","last_active":"2024-03-28T18:05:11.747786+0000","last_peered":"2024-03-28T18:05:11.747786+0000","last_clean":"2024-03-28T18:05:11.747786+0000","last_became_active":"2024-03-28T18:04:48.502716+0000","last_became_peered":"2024-03-28T18:04:48.502716+0000","last_unstale":"2024-03-28T18:05:11.747786+0000","last_undegraded":"2024-03-28T18:05:11.747786+0000","last_fullsized":"2024-03-28T18:05:11.747786+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:04:47.412563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:04:47.412563+0000","last_clean_scrub_stamp":"2024-03-28T18:04:47.412563+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T21:41:06.333420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"92'7","reported_seq":35,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-28T18:06:10.615938+0000","last_change":"2024-03-28T18:05:11.753703+0000","last_active":"2024-03-28T18:06:10.615938+0000","last_peered":"2024-03-28T18:06:10.615938+0000","last_clean":"2024-03-28T18:06:10.615938+0000","last_became_active":"2024-03-28T18:05:11.748793+0000","last_became_peered":"2024-03-28T18:05:11.748793+0000","last_unstale":"2024-03-28T18:06:10.615938+0000","last_undegraded":"2024-03-28T18:06:10.615938+0000","last_fullsized":"2024-03-28T18:06:10.615938+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_clean_scrub_stamp":"2024-03-28T18:04:48.465760+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T19:38:38.432027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":117,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"90'5","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-28T18:05:11.950534+0000","last_change":"2024-03-28T18:04:48.507658+0000","last_active":"2024-03-28T18:05:11.950534+0000","last_peered":"2024-03-28T18:05:11.950534+0000","last_clean":"2024-03-28T18:05:11.950534+0000","last_became_active":"2024-03-28T18:04:48.506855+0000","last_became_peered":"2024-03-28T18:04:48.506855+0000","last_unstale":"2024-03-28T18:05:11.950534+0000","last_undegraded":"2024-03-28T18:05:11.950534+0000","last_fullsized":"2024-03-28T18:05:11.950534+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:04:47.412563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:04:47.412563+0000","last_clean_scrub_stamp":"2024-03-28T18:04:47.412563+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T23:08:23.550379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"92'9","reported_seq":27,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-28T18:06:10.615589+0000","last_change":"2024-03-28T18:04:49.521122+0000","last_active":"2024-03-28T18:06:10.615589+0000","last_peered":"2024-03-28T18:06:10.615589+0000","last_clean":"2024-03-28T18:06:10.615589+0000","last_became_active":"2024-03-28T18:04:49.520711+0000","last_became_peered":"2024-03-28T18:04:49.520711+0000","last_unstale":"2024-03-28T18:06:10.615589+0000","last_undegraded":"2024-03-28T18:06:10.615589+0000","last_fullsized":"2024-03-28T18:06:10.615589+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_clean_scrub_stamp":"2024-03-28T18:04:48.465760+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T19:32:33.556979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"92'9","reported_seq":27,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-28T18:06:10.605702+0000","last_change":"2024-03-28T18:04:49.534929+0000","last_active":"2024-03-28T18:06:10.605702+0000","last_peered":"2024-03-28T18:06:10.605702+0000","last_clean":"2024-03-28T18:06:10.605702+0000","last_became_active":"2024-03-28T18:04:49.534485+0000","last_became_peered":"2024-03-28T18:04:49.534485+0000","last_unstale":"2024-03-28T18:06:10.605702+0000","last_undegraded":"2024-03-28T18:06:10.605702+0000","last_fullsized":"2024-03-28T18:06:10.605702+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_clean_scrub_stamp":"2024-03-28T18:04:48.465760+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T01:47:34.937536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"92'12","reported_seq":36,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-28T18:06:10.606584+0000","last_change":"2024-03-28T18:05:11.752731+0000","last_active":"2024-03-28T18:06:10.606584+0000","last_peered":"2024-03-28T18:06:10.606584+0000","last_clean":"2024-03-28T18:06:10.606584+0000","last_became_active":"2024-03-28T18:05:11.752121+0000","last_became_peered":"2024-03-28T18:05:11.752121+0000","last_unstale":"2024-03-28T18:06:10.606584+0000","last_undegraded":"2024-03-28T18:06:10.606584+0000","last_fullsized":"2024-03-28T18:06:10.606584+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_clean_scrub_stamp":"2024-03-28T18:04:48.465760+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T20:10:02.491231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,8],"acting":[2,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"92'8","reported_seq":26,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-28T18:06:10.609164+0000","last_change":"2024-03-28T18:04:49.531999+0000","last_active":"2024-03-28T18:06:10.609164+0000","last_peered":"2024-03-28T18:06:10.609164+0000","last_clean":"2024-03-28T18:06:10.609164+0000","last_became_active":"2024-03-28T18:04:49.528641+0000","last_became_peered":"2024-03-28T18:04:49.528641+0000","last_unstale":"2024-03-28T18:06:10.609164+0000","last_undegraded":"2024-03-28T18:06:10.609164+0000","last_fullsized":"2024-03-28T18:06:10.609164+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_clean_scrub_stamp":"2024-03-28T18:04:48.465760+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T21:24:43.861635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"92'7","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-28T18:06:10.610718+0000","last_change":"2024-03-28T18:04:49.540417+0000","last_active":"2024-03-28T18:06:10.610718+0000","last_peered":"2024-03-28T18:06:10.610718+0000","last_clean":"2024-03-28T18:06:10.610718+0000","last_became_active":"2024-03-28T18:04:49.539987+0000","last_became_peered":"2024-03-28T18:04:49.539987+0000","last_unstale":"2024-03-28T18:06:10.610718+0000","last_undegraded":"2024-03-28T18:06:10.610718+0000","last_fullsized":"2024-03-28T18:06:10.610718+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_clean_scrub_stamp":"2024-03-28T18:04:48.465760+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T02:53:41.444850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"92'4","reported_seq":22,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-28T18:06:10.609488+0000","last_change":"2024-03-28T18:04:49.540014+0000","last_active":"2024-03-28T18:06:10.609488+0000","last_peered":"2024-03-28T18:06:10.609488+0000","last_clean":"2024-03-28T18:06:10.609488+0000","last_became_active":"2024-03-28T18:04:49.538239+0000","last_became_peered":"2024-03-28T18:04:49.538239+0000","last_unstale":"2024-03-28T18:06:10.609488+0000","last_undegraded":"2024-03-28T18:06:10.609488+0000","last_fullsized":"2024-03-28T18:06:10.609488+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_clean_scrub_stamp":"2024-03-28T18:04:48.465760+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T23:03:58.449336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"92'6","reported_seq":30,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-28T18:06:10.603750+0000","last_change":"2024-03-28T18:05:11.748781+0000","last_active":"2024-03-28T18:06:10.603750+0000","last_peered":"2024-03-28T18:06:10.603750+0000","last_clean":"2024-03-28T18:06:10.603750+0000","last_became_active":"2024-03-28T18:05:11.747710+0000","last_became_peered":"2024-03-28T18:05:11.747710+0000","last_unstale":"2024-03-28T18:06:10.603750+0000","last_undegraded":"2024-03-28T18:06:10.603750+0000","last_fullsized":"2024-03-28T18:06:10.603750+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_clean_scrub_stamp":"2024-03-28T18:04:48.465760+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T02:01:44.708646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,9],"acting":[11,8,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"92'15","reported_seq":33,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-28T18:06:10.618896+0000","last_change":"2024-03-28T18:04:49.531135+0000","last_active":"2024-03-28T18:06:10.618896+0000","last_peered":"2024-03-28T18:06:10.618896+0000","last_clean":"2024-03-28T18:06:10.618896+0000","last_became_active":"2024-03-28T18:04:49.530673+0000","last_became_peered":"2024-03-28T18:04:49.530673+0000","last_unstale":"2024-03-28T18:06:10.618896+0000","last_undegraded":"2024-03-28T18:06:10.618896+0000","last_fullsized":"2024-03-28T18:06:10.618896+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_clean_scrub_stamp":"2024-03-28T18:04:48.465760+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T05:51:13.950946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"92'2","reported_seq":20,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-28T18:06:10.602677+0000","last_change":"2024-03-28T18:04:49.526803+0000","last_active":"2024-03-28T18:06:10.602677+0000","last_peered":"2024-03-28T18:06:10.602677+0000","last_clean":"2024-03-28T18:06:10.602677+0000","last_became_active":"2024-03-28T18:04:49.526474+0000","last_became_peered":"2024-03-28T18:04:49.526474+0000","last_unstale":"2024-03-28T18:06:10.602677+0000","last_undegraded":"2024-03-28T18:06:10.602677+0000","last_fullsized":"2024-03-28T18:06:10.602677+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_clean_scrub_stamp":"2024-03-28T18:04:48.465760+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T01:28:27.530864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"92'6","reported_seq":24,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-28T18:06:10.612723+0000","last_change":"2024-03-28T18:04:49.534015+0000","last_active":"2024-03-28T18:06:10.612723+0000","last_peered":"2024-03-28T18:06:10.612723+0000","last_clean":"2024-03-28T18:06:10.612723+0000","last_became_active":"2024-03-28T18:04:49.533560+0000","last_became_peered":"2024-03-28T18:04:49.533560+0000","last_unstale":"2024-03-28T18:06:10.612723+0000","last_undegraded":"2024-03-28T18:06:10.612723+0000","last_fullsized":"2024-03-28T18:06:10.612723+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_clean_scrub_stamp":"2024-03-28T18:04:48.465760+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T22:41:30.965737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"92'8","reported_seq":26,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-28T18:06:10.608221+0000","last_change":"2024-03-28T18:04:49.523240+0000","last_active":"2024-03-28T18:06:10.608221+0000","last_peered":"2024-03-28T18:06:10.608221+0000","last_clean":"2024-03-28T18:06:10.608221+0000","last_became_active":"2024-03-28T18:04:49.522795+0000","last_became_peered":"2024-03-28T18:04:49.522795+0000","last_unstale":"2024-03-28T18:06:10.608221+0000","last_undegraded":"2024-03-28T18:06:10.608221+0000","last_fullsized":"2024-03-28T18:06:10.608221+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_clean_scrub_stamp":"2024-03-28T18:04:48.465760+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T23:08:37.087977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"92'10","reported_seq":28,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-28T18:06:10.626330+0000","last_change":"2024-03-28T18:04:49.530965+0000","last_active":"2024-03-28T18:06:10.626330+0000","last_peered":"2024-03-28T18:06:10.626330+0000","last_clean":"2024-03-28T18:06:10.626330+0000","last_became_active":"2024-03-28T18:04:49.530619+0000","last_became_peered":"2024-03-28T18:04:49.530619+0000","last_unstale":"2024-03-28T18:06:10.626330+0000","last_undegraded":"2024-03-28T18:06:10.626330+0000","last_fullsized":"2024-03-28T18:06:10.626330+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_clean_scrub_stamp":"2024-03-28T18:04:48.465760+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T05:29:26.754012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"92'11","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-28T18:06:10.624772+0000","last_change":"2024-03-28T18:04:49.523017+0000","last_active":"2024-03-28T18:06:10.624772+0000","last_peered":"2024-03-28T18:06:10.624772+0000","last_clean":"2024-03-28T18:06:10.624772+0000","last_became_active":"2024-03-28T18:04:49.520411+0000","last_became_peered":"2024-03-28T18:04:49.520411+0000","last_unstale":"2024-03-28T18:06:10.624772+0000","last_undegraded":"2024-03-28T18:06:10.624772+0000","last_fullsized":"2024-03-28T18:06:10.624772+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_clean_scrub_stamp":"2024-03-28T18:04:48.465760+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T22:52:45.893036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"92'8","reported_seq":26,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-28T18:06:10.608384+0000","last_change":"2024-03-28T18:04:49.544272+0000","last_active":"2024-03-28T18:06:10.608384+0000","last_peered":"2024-03-28T18:06:10.608384+0000","last_clean":"2024-03-28T18:06:10.608384+0000","last_became_active":"2024-03-28T18:04:49.541754+0000","last_became_peered":"2024-03-28T18:04:49.541754+0000","last_unstale":"2024-03-28T18:06:10.608384+0000","last_undegraded":"2024-03-28T18:06:10.608384+0000","last_fullsized":"2024-03-28T18:06:10.608384+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_clean_scrub_stamp":"2024-03-28T18:04:48.465760+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T19:06:00.799916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":117,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":502,"ondisk_log_size":502,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4751360,"data_stored":4722944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":74205,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":1436,"num_read_kb":797,"num_write":267,"num_write_kb":159,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":0,"num_keys_recovered":10,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":573440,"data_stored":218301,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":229,"ondisk_log_size":229,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":85,"seq":365072220206,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28000,"kb_used_data":984,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717152,"statfs":{"total":95995035648,"available":95966363648,"internally_reserved":0,"allocated":1007616,"data_stored":484342,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":10,"up_from":77,"seq":330712481846,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":37008,"kb_used_data":1672,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708144,"statfs":{"total":95995035648,"available":95957139456,"internally_reserved":0,"allocated":1712128,"data_stored":1140650,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":36174272},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":70,"seq":300647710780,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36348,"kb_used_data":1012,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708804,"statfs":{"total":95995035648,"available":95957815296,"internally_reserved":0,"allocated":1036288,"data_stored":483896,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":36174272},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":62,"seq":266287972418,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36340,"kb_used_data":1004,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708812,"statfs":{"total":95995035648,"available":95957823488,"internally_reserved":0,"allocated":1028096,"data_stored":484157,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":7,"up_from":53,"seq":227633266761,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28604,"kb_used_data":1588,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716548,"statfs":{"total":95995035648,"available":95965745152,"internally_reserved":0,"allocated":1626112,"data_stored":1078827,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":48,"seq":206158430287,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36328,"kb_used_data":992,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708824,"statfs":{"total":95995035648,"available":95957835776,"internally_reserved":0,"allocated":1015808,"data_stored":483989,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574963,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36388,"kb_used_data":1052,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708764,"statfs":{"total":95995035648,"available":95957774336,"internally_reserved":0,"allocated":1077248,"data_stored":545335,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738490,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28028,"kb_used_data":1012,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717124,"statfs":{"total":95995035648,"available":95966334976,"internally_reserved":0,"allocated":1036288,"data_stored":485328,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411438,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36412,"kb_used_data":1076,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708740,"statfs":{"total":95995035648,"available":95957749760,"internally_reserved":0,"allocated":1101824,"data_stored":550894,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":3,"up_from":26,"seq":111669149798,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36340,"kb_used_data":1004,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708812,"statfs":{"total":95995035648,"available":95957823488,"internally_reserved":0,"allocated":1028096,"data_stored":483204,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":36174280},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":34,"seq":146028888158,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28008,"kb_used_data":992,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717144,"statfs":{"total":95995035648,"available":95966355456,"internally_reserved":0,"allocated":1015808,"data_stored":484382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":40,"seq":171798691929,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36936,"kb_used_data":1600,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708216,"statfs":{"total":95995035648,"available":95957213184,"internally_reserved":0,"allocated":1638400,"data_stored":1076987,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2671,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":81920,"data_stored":62678,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":106496,"data_stored":68237,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1725,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":3845,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1215,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":5685,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":1500,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":118784,"data_stored":67508,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1685,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-28T18:07:19.501 INFO:tasks.fwd_scrub.ceph_manager:clean! 2024-03-28T18:07:19.501 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-28T18:07:19.762 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:19.762+0000 7f055c600640 1 -- 172.21.15.22:0/4161709123 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0554076060 msgr2=0x7f0554076480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:07:19.762 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:19.762+0000 7f055c600640 1 --2- 172.21.15.22:0/4161709123 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0554076060 0x7f0554076480 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f0548009900 tx=0x7f054802bc30 comp rx=0 tx=0).stop 2024-03-28T18:07:19.762 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:19.762+0000 7f055c600640 1 -- 172.21.15.22:0/4161709123 shutdown_connections 2024-03-28T18:07:19.762 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:19.762+0000 7f055c600640 1 --2- 172.21.15.22:0/4161709123 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f0554077c40 0x7f05540784e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:07:19.763 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:19.762+0000 7f055c600640 1 --2- 172.21.15.22:0/4161709123 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f05540772a0 0x7f0554077700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:07:19.763 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:19.762+0000 7f055c600640 1 --2- 172.21.15.22:0/4161709123 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0554076060 0x7f0554076480 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:07:19.763 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:19.762+0000 7f055c600640 1 -- 172.21.15.22:0/4161709123 >> 172.21.15.22:0/4161709123 conn(0x7f05540fee50 msgr2=0x7f0554101270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:07:19.763 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:19.762+0000 7f055c600640 1 -- 172.21.15.22:0/4161709123 shutdown_connections 2024-03-28T18:07:19.763 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:19.762+0000 7f055c600640 1 -- 172.21.15.22:0/4161709123 wait complete. 2024-03-28T18:07:19.763 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:19.764+0000 7f055c600640 1 Processor -- start 2024-03-28T18:07:19.763 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:19.764+0000 7f055c600640 1 -- start start 2024-03-28T18:07:19.764 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:19.764+0000 7f055c600640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0554076060 0x7f055419e060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:07:19.764 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:19.764+0000 7f055c600640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f05540772a0 0x7f055419e5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:07:19.764 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:19.764+0000 7f055c600640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0554077c40 0x7f05541a55a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:07:19.764 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:19.764+0000 7f055c600640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0554113960 con 0x7f0554076060 2024-03-28T18:07:19.765 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:19.764+0000 7f055c600640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05541137e0 con 0x7f05540772a0 2024-03-28T18:07:19.765 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:19.764+0000 7f055c600640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0554113ae0 con 0x7f0554077c40 2024-03-28T18:07:19.765 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:19.765+0000 7f0559200640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f05540772a0 0x7f055419e5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:07:19.765 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:19.765+0000 7f0559200640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f05540772a0 0x7f055419e5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.22:59194/0 (socket says 172.21.15.22:59194) 2024-03-28T18:07:19.765 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:19.765+0000 7f0559c00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0554076060 0x7f055419e060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:07:19.765 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:19.765+0000 7f0559200640 1 -- 172.21.15.22:0/3124085391 learned_addr learned my addr 172.21.15.22:0/3124085391 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-28T18:07:19.765 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:19.765+0000 7f055a600640 1 --2- 172.21.15.22:0/3124085391 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0554077c40 0x7f05541a55a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:07:19.766 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:19.765+0000 7f0559c00640 1 -- 172.21.15.22:0/3124085391 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0554077c40 msgr2=0x7f05541a55a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:07:19.766 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:19.765+0000 7f0559c00640 1 --2- 172.21.15.22:0/3124085391 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0554077c40 0x7f05541a55a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:07:19.766 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:19.765+0000 7f0559c00640 1 -- 172.21.15.22:0/3124085391 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f05540772a0 msgr2=0x7f055419e5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:07:19.766 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:19.765+0000 7f0559c00640 1 --2- 172.21.15.22:0/3124085391 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f05540772a0 0x7f055419e5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:07:19.766 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:19.765+0000 7f0559c00640 1 -- 172.21.15.22:0/3124085391 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f05541a5c10 con 0x7f0554076060 2024-03-28T18:07:19.766 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:19.765+0000 7f0559c00640 1 --2- 172.21.15.22:0/3124085391 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0554076060 0x7f055419e060 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f0548036040 tx=0x7f054802c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:07:19.766 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:19.766+0000 7f0552000640 1 -- 172.21.15.22:0/3124085391 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0548043020 con 0x7f0554076060 2024-03-28T18:07:19.767 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:19.766+0000 7f0552000640 1 -- 172.21.15.22:0/3124085391 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f05480054b0 con 0x7f0554076060 2024-03-28T18:07:19.767 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:19.766+0000 7f055c600640 1 -- 172.21.15.22:0/3124085391 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f05541a5ea0 con 0x7f0554076060 2024-03-28T18:07:19.767 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:19.766+0000 7f0552000640 1 -- 172.21.15.22:0/3124085391 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0548003d10 con 0x7f0554076060 2024-03-28T18:07:19.767 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:19.766+0000 7f055c600640 1 -- 172.21.15.22:0/3124085391 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f05541a6300 con 0x7f0554076060 2024-03-28T18:07:19.767 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:19.768+0000 7f055c600640 1 -- 172.21.15.22:0/3124085391 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0554110230 con 0x7f0554076060 2024-03-28T18:07:19.769 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:19.769+0000 7f0552000640 1 -- 172.21.15.22:0/3124085391 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 18) v1 ==== 137290+0+0 (secure 0 0 0) 0x7f0548046020 con 0x7f0554076060 2024-03-28T18:07:19.770 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:19.770+0000 7f0552000640 1 --2- 172.21.15.22:0/3124085391 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7f053c0a2850 0x7f053c0a4d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:07:19.770 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:19.771+0000 7f0552000640 1 -- 172.21.15.22:0/3124085391 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 8153+0+0 (secure 0 0 0) 0x7f05480f0e00 con 0x7f0554076060 2024-03-28T18:07:19.770 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:19.771+0000 7f0559200640 1 --2- 172.21.15.22:0/3124085391 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7f053c0a2850 0x7f053c0a4d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:07:19.771 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:19.771+0000 7f0559200640 1 --2- 172.21.15.22:0/3124085391 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7f053c0a2850 0x7f053c0a4d10 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f055419f500 tx=0x7f054000b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:07:19.774 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:19.774+0000 7f0552000640 1 -- 172.21.15.22:0/3124085391 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f054804d020 con 0x7f0554076060 2024-03-28T18:07:19.975 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:19.975+0000 7f055c600640 1 -- 172.21.15.22:0/3124085391 --> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f0554061ae0 con 0x7f053c0a2850 2024-03-28T18:07:19.988 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:19.988+0000 7f0552000640 1 -- 172.21.15.22:0/3124085391 <== mgr.14152 v2:172.21.15.22:6800/4201048392 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+258984 (secure 0 0 0) 0x7f0554061ae0 con 0x7f053c0a2850 2024-03-28T18:07:19.992 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-03-28T18:07:19.993 INFO:teuthology.orchestra.run.smithi022.stderr:dumped all 2024-03-28T18:07:19.994 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:19.994+0000 7f055c600640 1 -- 172.21.15.22:0/3124085391 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7f053c0a2850 msgr2=0x7f053c0a4d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:07:19.994 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:19.994+0000 7f055c600640 1 --2- 172.21.15.22:0/3124085391 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7f053c0a2850 0x7f053c0a4d10 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f055419f500 tx=0x7f054000b410 comp rx=0 tx=0).stop 2024-03-28T18:07:19.995 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:19.994+0000 7f055c600640 1 -- 172.21.15.22:0/3124085391 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0554076060 msgr2=0x7f055419e060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:07:19.995 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:19.994+0000 7f055c600640 1 --2- 172.21.15.22:0/3124085391 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0554076060 0x7f055419e060 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f0548036040 tx=0x7f054802c6a0 comp rx=0 tx=0).stop 2024-03-28T18:07:19.995 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:19.994+0000 7f055c600640 1 -- 172.21.15.22:0/3124085391 shutdown_connections 2024-03-28T18:07:19.995 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:19.994+0000 7f055c600640 1 --2- 172.21.15.22:0/3124085391 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7f053c0a2850 0x7f053c0a4d10 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:07:19.995 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:19.994+0000 7f055c600640 1 --2- 172.21.15.22:0/3124085391 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0554077c40 0x7f05541a55a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:07:19.995 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:19.994+0000 7f055c600640 1 --2- 172.21.15.22:0/3124085391 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f05540772a0 0x7f055419e5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:07:19.995 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:19.994+0000 7f055c600640 1 --2- 172.21.15.22:0/3124085391 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0554076060 0x7f055419e060 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:07:19.995 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:19.994+0000 7f055c600640 1 -- 172.21.15.22:0/3124085391 >> 172.21.15.22:0/3124085391 conn(0x7f05540fee50 msgr2=0x7f0554100950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:07:19.995 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:19.995+0000 7f055c600640 1 -- 172.21.15.22:0/3124085391 shutdown_connections 2024-03-28T18:07:19.995 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:19.995+0000 7f055c600640 1 -- 172.21.15.22:0/3124085391 wait complete. 2024-03-28T18:07:20.012 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:07:19 smithi022 ceph-mon[31985]: pgmap v446: 97 pgs: 97 active+clean; 649 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 1.7 KiB/s wr, 11 op/s 2024-03-28T18:07:20.012 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:07:19 smithi022 ceph-mon[31985]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T18:07:20.013 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:07:19 smithi022 ceph-mon[31985]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T18:07:20.013 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:07:19 smithi022 ceph-mon[31985]: from='client.15426 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-28T18:07:20.013 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:07:19 smithi022 ceph-mon[31985]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T18:07:20.013 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:07:19 smithi022 ceph-mon[31985]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T18:07:20.013 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:07:19 smithi022 ceph-mon[31985]: from='client.35291 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-28T18:07:20.016 INFO:teuthology.orchestra.run.smithi022.stdout:{"pg_ready":true,"pg_map":{"version":446,"stamp":"2024-03-28T18:07:18.662318+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":664807,"num_objects":102,"num_object_clones":0,"num_object_copies":306,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":1572,"num_read_kb":916,"num_write":450,"num_write_kb":3099,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13,"num_bytes_recovered":1180853,"num_keys_recovered":10,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":868,"ondisk_log_size":868,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":404740,"kb_used_data":13988,"kb_used_omap":18,"kb_used_meta":390637,"kb_avail":1124537084,"statfs":{"total":1151940427776,"available":1151525974016,"internally_reserved":0,"allocated":14323712,"data_stored":7781991,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19141,"internal_metadata":400012603},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":6779,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":116,"num_read_kb":60,"num_write":21,"num_write_kb":20,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004700"},"pg_stats":[{"pgid":"3.3f","version":"92'8","reported_seq":26,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-28T18:06:10.622027+0000","last_change":"2024-03-28T18:04:49.545022+0000","last_active":"2024-03-28T18:06:10.622027+0000","last_peered":"2024-03-28T18:06:10.622027+0000","last_clean":"2024-03-28T18:06:10.622027+0000","last_became_active":"2024-03-28T18:04:49.544584+0000","last_became_peered":"2024-03-28T18:04:49.544584+0000","last_unstale":"2024-03-28T18:06:10.622027+0000","last_undegraded":"2024-03-28T18:06:10.622027+0000","last_fullsized":"2024-03-28T18:06:10.622027+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_clean_scrub_stamp":"2024-03-28T18:04:48.465760+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T01:47:05.644293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"92'4","reported_seq":22,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-28T18:06:10.615631+0000","last_change":"2024-03-28T18:04:49.526720+0000","last_active":"2024-03-28T18:06:10.615631+0000","last_peered":"2024-03-28T18:06:10.615631+0000","last_clean":"2024-03-28T18:06:10.615631+0000","last_became_active":"2024-03-28T18:04:49.524545+0000","last_became_peered":"2024-03-28T18:04:49.524545+0000","last_unstale":"2024-03-28T18:06:10.615631+0000","last_undegraded":"2024-03-28T18:06:10.615631+0000","last_fullsized":"2024-03-28T18:06:10.615631+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_clean_scrub_stamp":"2024-03-28T18:04:48.465760+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T01:00:54.926308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"92'7","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-28T18:06:10.625795+0000","last_change":"2024-03-28T18:04:49.532425+0000","last_active":"2024-03-28T18:06:10.625795+0000","last_peered":"2024-03-28T18:06:10.625795+0000","last_clean":"2024-03-28T18:06:10.625795+0000","last_became_active":"2024-03-28T18:04:49.531146+0000","last_became_peered":"2024-03-28T18:04:49.531146+0000","last_unstale":"2024-03-28T18:06:10.625795+0000","last_undegraded":"2024-03-28T18:06:10.625795+0000","last_fullsized":"2024-03-28T18:06:10.625795+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_clean_scrub_stamp":"2024-03-28T18:04:48.465760+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T04:58:45.131773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"92'8","reported_seq":26,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-28T18:06:10.609313+0000","last_change":"2024-03-28T18:04:49.533371+0000","last_active":"2024-03-28T18:06:10.609313+0000","last_peered":"2024-03-28T18:06:10.609313+0000","last_clean":"2024-03-28T18:06:10.609313+0000","last_became_active":"2024-03-28T18:04:49.532544+0000","last_became_peered":"2024-03-28T18:04:49.532544+0000","last_unstale":"2024-03-28T18:06:10.609313+0000","last_undegraded":"2024-03-28T18:06:10.609313+0000","last_fullsized":"2024-03-28T18:06:10.609313+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_clean_scrub_stamp":"2024-03-28T18:04:48.465760+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T20:36:25.664137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"92'4","reported_seq":21,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-28T18:06:10.592721+0000","last_change":"2024-03-28T18:04:49.530875+0000","last_active":"2024-03-28T18:06:10.592721+0000","last_peered":"2024-03-28T18:06:10.592721+0000","last_clean":"2024-03-28T18:06:10.592721+0000","last_became_active":"2024-03-28T18:04:49.530354+0000","last_became_peered":"2024-03-28T18:04:49.530354+0000","last_unstale":"2024-03-28T18:06:10.592721+0000","last_undegraded":"2024-03-28T18:06:10.592721+0000","last_fullsized":"2024-03-28T18:06:10.592721+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_clean_scrub_stamp":"2024-03-28T18:04:48.465760+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T21:02:17.187989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"92'6","reported_seq":23,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-28T18:06:10.587630+0000","last_change":"2024-03-28T18:04:49.538260+0000","last_active":"2024-03-28T18:06:10.587630+0000","last_peered":"2024-03-28T18:06:10.587630+0000","last_clean":"2024-03-28T18:06:10.587630+0000","last_became_active":"2024-03-28T18:04:49.537863+0000","last_became_peered":"2024-03-28T18:04:49.537863+0000","last_unstale":"2024-03-28T18:06:10.587630+0000","last_undegraded":"2024-03-28T18:06:10.587630+0000","last_fullsized":"2024-03-28T18:06:10.587630+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_clean_scrub_stamp":"2024-03-28T18:04:48.465760+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T03:01:25.363839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"92'5","reported_seq":22,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-28T18:06:10.612305+0000","last_change":"2024-03-28T18:04:49.534636+0000","last_active":"2024-03-28T18:06:10.612305+0000","last_peered":"2024-03-28T18:06:10.612305+0000","last_clean":"2024-03-28T18:06:10.612305+0000","last_became_active":"2024-03-28T18:04:49.534236+0000","last_became_peered":"2024-03-28T18:04:49.534236+0000","last_unstale":"2024-03-28T18:06:10.612305+0000","last_undegraded":"2024-03-28T18:06:10.612305+0000","last_fullsized":"2024-03-28T18:06:10.612305+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_clean_scrub_stamp":"2024-03-28T18:04:48.465760+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T05:55:23.616641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"92'9","reported_seq":26,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-28T18:06:10.605001+0000","last_change":"2024-03-28T18:04:49.534848+0000","last_active":"2024-03-28T18:06:10.605001+0000","last_peered":"2024-03-28T18:06:10.605001+0000","last_clean":"2024-03-28T18:06:10.605001+0000","last_became_active":"2024-03-28T18:04:49.534302+0000","last_became_peered":"2024-03-28T18:04:49.534302+0000","last_unstale":"2024-03-28T18:06:10.605001+0000","last_undegraded":"2024-03-28T18:06:10.605001+0000","last_fullsized":"2024-03-28T18:06:10.605001+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_clean_scrub_stamp":"2024-03-28T18:04:48.465760+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T05:48:24.707305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"92'11","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-28T18:06:10.622540+0000","last_change":"2024-03-28T18:04:49.536568+0000","last_active":"2024-03-28T18:06:10.622540+0000","last_peered":"2024-03-28T18:06:10.622540+0000","last_clean":"2024-03-28T18:06:10.622540+0000","last_became_active":"2024-03-28T18:04:49.535675+0000","last_became_peered":"2024-03-28T18:04:49.535675+0000","last_unstale":"2024-03-28T18:06:10.622540+0000","last_undegraded":"2024-03-28T18:06:10.622540+0000","last_fullsized":"2024-03-28T18:06:10.622540+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_clean_scrub_stamp":"2024-03-28T18:04:48.465760+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T02:32:15.179869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"92'16","reported_seq":34,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-28T18:06:10.614038+0000","last_change":"2024-03-28T18:04:49.539363+0000","last_active":"2024-03-28T18:06:10.614038+0000","last_peered":"2024-03-28T18:06:10.614038+0000","last_clean":"2024-03-28T18:06:10.614038+0000","last_became_active":"2024-03-28T18:04:49.539002+0000","last_became_peered":"2024-03-28T18:04:49.539002+0000","last_unstale":"2024-03-28T18:06:10.614038+0000","last_undegraded":"2024-03-28T18:06:10.614038+0000","last_fullsized":"2024-03-28T18:06:10.614038+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_clean_scrub_stamp":"2024-03-28T18:04:48.465760+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T18:24:28.846081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"92'6","reported_seq":24,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-28T18:06:10.626273+0000","last_change":"2024-03-28T18:04:49.532385+0000","last_active":"2024-03-28T18:06:10.626273+0000","last_peered":"2024-03-28T18:06:10.626273+0000","last_clean":"2024-03-28T18:06:10.626273+0000","last_became_active":"2024-03-28T18:04:49.532004+0000","last_became_peered":"2024-03-28T18:04:49.532004+0000","last_unstale":"2024-03-28T18:06:10.626273+0000","last_undegraded":"2024-03-28T18:06:10.626273+0000","last_fullsized":"2024-03-28T18:06:10.626273+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_clean_scrub_stamp":"2024-03-28T18:04:48.465760+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T19:05:13.959313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"92'7","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-28T18:06:10.608967+0000","last_change":"2024-03-28T18:04:49.534465+0000","last_active":"2024-03-28T18:06:10.608967+0000","last_peered":"2024-03-28T18:06:10.608967+0000","last_clean":"2024-03-28T18:06:10.608967+0000","last_became_active":"2024-03-28T18:04:49.532989+0000","last_became_peered":"2024-03-28T18:04:49.532989+0000","last_unstale":"2024-03-28T18:06:10.608967+0000","last_undegraded":"2024-03-28T18:06:10.608967+0000","last_fullsized":"2024-03-28T18:06:10.608967+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_clean_scrub_stamp":"2024-03-28T18:04:48.465760+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T23:56:50.607770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"92'10","reported_seq":28,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-28T18:06:10.612024+0000","last_change":"2024-03-28T18:04:49.535205+0000","last_active":"2024-03-28T18:06:10.612024+0000","last_peered":"2024-03-28T18:06:10.612024+0000","last_clean":"2024-03-28T18:06:10.612024+0000","last_became_active":"2024-03-28T18:04:49.533025+0000","last_became_peered":"2024-03-28T18:04:49.533025+0000","last_unstale":"2024-03-28T18:06:10.612024+0000","last_undegraded":"2024-03-28T18:06:10.612024+0000","last_fullsized":"2024-03-28T18:06:10.612024+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_clean_scrub_stamp":"2024-03-28T18:04:48.465760+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T21:41:51.815907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"92'7","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-28T18:06:10.615451+0000","last_change":"2024-03-28T18:04:49.520974+0000","last_active":"2024-03-28T18:06:10.615451+0000","last_peered":"2024-03-28T18:06:10.615451+0000","last_clean":"2024-03-28T18:06:10.615451+0000","last_became_active":"2024-03-28T18:04:49.520509+0000","last_became_peered":"2024-03-28T18:04:49.520509+0000","last_unstale":"2024-03-28T18:06:10.615451+0000","last_undegraded":"2024-03-28T18:06:10.615451+0000","last_fullsized":"2024-03-28T18:06:10.615451+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_clean_scrub_stamp":"2024-03-28T18:04:48.465760+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T03:15:16.346859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"92'7","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-28T18:06:10.608830+0000","last_change":"2024-03-28T18:04:49.539053+0000","last_active":"2024-03-28T18:06:10.608830+0000","last_peered":"2024-03-28T18:06:10.608830+0000","last_clean":"2024-03-28T18:06:10.608830+0000","last_became_active":"2024-03-28T18:04:49.537327+0000","last_became_peered":"2024-03-28T18:04:49.537327+0000","last_unstale":"2024-03-28T18:06:10.608830+0000","last_undegraded":"2024-03-28T18:06:10.608830+0000","last_fullsized":"2024-03-28T18:06:10.608830+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_clean_scrub_stamp":"2024-03-28T18:04:48.465760+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T00:24:42.319504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"92'9","reported_seq":27,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-28T18:06:10.625483+0000","last_change":"2024-03-28T18:04:49.525855+0000","last_active":"2024-03-28T18:06:10.625483+0000","last_peered":"2024-03-28T18:06:10.625483+0000","last_clean":"2024-03-28T18:06:10.625483+0000","last_became_active":"2024-03-28T18:04:49.525448+0000","last_became_peered":"2024-03-28T18:04:49.525448+0000","last_unstale":"2024-03-28T18:06:10.625483+0000","last_undegraded":"2024-03-28T18:06:10.625483+0000","last_fullsized":"2024-03-28T18:06:10.625483+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_clean_scrub_stamp":"2024-03-28T18:04:48.465760+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T23:53:00.452692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"92'8","reported_seq":26,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-28T18:06:10.622131+0000","last_change":"2024-03-28T18:04:49.543605+0000","last_active":"2024-03-28T18:06:10.622131+0000","last_peered":"2024-03-28T18:06:10.622131+0000","last_clean":"2024-03-28T18:06:10.622131+0000","last_became_active":"2024-03-28T18:04:49.543280+0000","last_became_peered":"2024-03-28T18:04:49.543280+0000","last_unstale":"2024-03-28T18:06:10.622131+0000","last_undegraded":"2024-03-28T18:06:10.622131+0000","last_fullsized":"2024-03-28T18:06:10.622131+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_clean_scrub_stamp":"2024-03-28T18:04:48.465760+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T21:27:26.216639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"92'10","reported_seq":28,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-28T18:06:10.619171+0000","last_change":"2024-03-28T18:04:49.526745+0000","last_active":"2024-03-28T18:06:10.619171+0000","last_peered":"2024-03-28T18:06:10.619171+0000","last_clean":"2024-03-28T18:06:10.619171+0000","last_became_active":"2024-03-28T18:04:49.526351+0000","last_became_peered":"2024-03-28T18:04:49.526351+0000","last_unstale":"2024-03-28T18:06:10.619171+0000","last_undegraded":"2024-03-28T18:06:10.619171+0000","last_fullsized":"2024-03-28T18:06:10.619171+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_clean_scrub_stamp":"2024-03-28T18:04:48.465760+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T05:31:47.687345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"92'7","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-28T18:06:10.596535+0000","last_change":"2024-03-28T18:04:49.523346+0000","last_active":"2024-03-28T18:06:10.596535+0000","last_peered":"2024-03-28T18:06:10.596535+0000","last_clean":"2024-03-28T18:06:10.596535+0000","last_became_active":"2024-03-28T18:04:49.522041+0000","last_became_peered":"2024-03-28T18:04:49.522041+0000","last_unstale":"2024-03-28T18:06:10.596535+0000","last_undegraded":"2024-03-28T18:06:10.596535+0000","last_fullsized":"2024-03-28T18:06:10.596535+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_clean_scrub_stamp":"2024-03-28T18:04:48.465760+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T01:35:02.419328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"92'14","reported_seq":174,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-28T18:07:13.984992+0000","last_change":"2024-03-28T18:04:48.510917+0000","last_active":"2024-03-28T18:07:13.984992+0000","last_peered":"2024-03-28T18:07:13.984992+0000","last_clean":"2024-03-28T18:07:13.984992+0000","last_became_active":"2024-03-28T18:04:48.509978+0000","last_became_peered":"2024-03-28T18:04:48.509978+0000","last_unstale":"2024-03-28T18:07:13.984992+0000","last_undegraded":"2024-03-28T18:07:13.984992+0000","last_fullsized":"2024-03-28T18:07:13.984992+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:04:47.412563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:04:47.412563+0000","last_clean_scrub_stamp":"2024-03-28T18:04:47.412563+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T03:28:13.185099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":140,"num_read_kb":140,"num_write":15,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"92'5","reported_seq":23,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-28T18:06:10.603875+0000","last_change":"2024-03-28T18:04:49.535543+0000","last_active":"2024-03-28T18:06:10.603875+0000","last_peered":"2024-03-28T18:06:10.603875+0000","last_clean":"2024-03-28T18:06:10.603875+0000","last_became_active":"2024-03-28T18:04:49.535171+0000","last_became_peered":"2024-03-28T18:04:49.535171+0000","last_unstale":"2024-03-28T18:06:10.603875+0000","last_undegraded":"2024-03-28T18:06:10.603875+0000","last_fullsized":"2024-03-28T18:06:10.603875+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_clean_scrub_stamp":"2024-03-28T18:04:48.465760+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T22:12:22.258424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"90'2","reported_seq":22,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-28T18:05:11.741109+0000","last_change":"2024-03-28T18:04:48.503190+0000","last_active":"2024-03-28T18:05:11.741109+0000","last_peered":"2024-03-28T18:05:11.741109+0000","last_clean":"2024-03-28T18:05:11.741109+0000","last_became_active":"2024-03-28T18:04:48.501858+0000","last_became_peered":"2024-03-28T18:04:48.501858+0000","last_unstale":"2024-03-28T18:05:11.741109+0000","last_undegraded":"2024-03-28T18:05:11.741109+0000","last_fullsized":"2024-03-28T18:05:11.741109+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:04:47.412563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:04:47.412563+0000","last_clean_scrub_stamp":"2024-03-28T18:04:47.412563+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T19:56:43.878630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"92'6","reported_seq":24,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-28T18:06:10.625888+0000","last_change":"2024-03-28T18:04:49.537739+0000","last_active":"2024-03-28T18:06:10.625888+0000","last_peered":"2024-03-28T18:06:10.625888+0000","last_clean":"2024-03-28T18:06:10.625888+0000","last_became_active":"2024-03-28T18:04:49.535990+0000","last_became_peered":"2024-03-28T18:04:49.535990+0000","last_unstale":"2024-03-28T18:06:10.625888+0000","last_undegraded":"2024-03-28T18:06:10.625888+0000","last_fullsized":"2024-03-28T18:06:10.625888+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_clean_scrub_stamp":"2024-03-28T18:04:48.465760+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T21:54:15.729494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"90'12","reported_seq":27,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-28T18:05:11.768035+0000","last_change":"2024-03-28T18:05:11.767512+0000","last_active":"2024-03-28T18:05:11.768035+0000","last_peered":"2024-03-28T18:05:11.768035+0000","last_clean":"2024-03-28T18:05:11.768035+0000","last_became_active":"2024-03-28T18:05:11.752452+0000","last_became_peered":"2024-03-28T18:05:11.752452+0000","last_unstale":"2024-03-28T18:05:11.768035+0000","last_undegraded":"2024-03-28T18:05:11.768035+0000","last_fullsized":"2024-03-28T18:05:11.768035+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:04:47.412563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:04:47.412563+0000","last_clean_scrub_stamp":"2024-03-28T18:04:47.412563+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T00:09:13.980288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":0,"num_keys_recovered":10,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,7],"acting":[11,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1c","version":"92'9","reported_seq":26,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-28T18:06:10.623876+0000","last_change":"2024-03-28T18:04:49.529316+0000","last_active":"2024-03-28T18:06:10.623876+0000","last_peered":"2024-03-28T18:06:10.623876+0000","last_clean":"2024-03-28T18:06:10.623876+0000","last_became_active":"2024-03-28T18:04:49.528975+0000","last_became_peered":"2024-03-28T18:04:49.528975+0000","last_unstale":"2024-03-28T18:06:10.623876+0000","last_undegraded":"2024-03-28T18:06:10.623876+0000","last_fullsized":"2024-03-28T18:06:10.623876+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_clean_scrub_stamp":"2024-03-28T18:04:48.465760+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T00:06:45.268757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"90'5","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-28T18:05:11.734761+0000","last_change":"2024-03-28T18:04:48.515841+0000","last_active":"2024-03-28T18:05:11.734761+0000","last_peered":"2024-03-28T18:05:11.734761+0000","last_clean":"2024-03-28T18:05:11.734761+0000","last_became_active":"2024-03-28T18:04:48.513852+0000","last_became_peered":"2024-03-28T18:04:48.513852+0000","last_unstale":"2024-03-28T18:05:11.734761+0000","last_undegraded":"2024-03-28T18:05:11.734761+0000","last_fullsized":"2024-03-28T18:05:11.734761+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:04:47.412563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:04:47.412563+0000","last_clean_scrub_stamp":"2024-03-28T18:04:47.412563+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T00:07:14.751545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"92'15","reported_seq":32,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-28T18:06:10.609956+0000","last_change":"2024-03-28T18:04:49.542238+0000","last_active":"2024-03-28T18:06:10.609956+0000","last_peered":"2024-03-28T18:06:10.609956+0000","last_clean":"2024-03-28T18:06:10.609956+0000","last_became_active":"2024-03-28T18:04:49.539723+0000","last_became_peered":"2024-03-28T18:04:49.539723+0000","last_unstale":"2024-03-28T18:06:10.609956+0000","last_undegraded":"2024-03-28T18:06:10.609956+0000","last_fullsized":"2024-03-28T18:06:10.609956+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_clean_scrub_stamp":"2024-03-28T18:04:48.465760+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T22:45:07.310148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"92'8","reported_seq":170,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-28T18:07:14.022275+0000","last_change":"2024-03-28T18:04:48.511349+0000","last_active":"2024-03-28T18:07:14.022275+0000","last_peered":"2024-03-28T18:07:14.022275+0000","last_clean":"2024-03-28T18:07:14.022275+0000","last_became_active":"2024-03-28T18:04:48.510990+0000","last_became_peered":"2024-03-28T18:04:48.510990+0000","last_unstale":"2024-03-28T18:07:14.022275+0000","last_undegraded":"2024-03-28T18:07:14.022275+0000","last_fullsized":"2024-03-28T18:07:14.022275+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:04:47.412563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:04:47.412563+0000","last_clean_scrub_stamp":"2024-03-28T18:04:47.412563+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T21:56:15.411149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1647,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":143,"num_read_kb":4,"num_write":5,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"92'7","reported_seq":24,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-28T18:06:10.604922+0000","last_change":"2024-03-28T18:04:49.538066+0000","last_active":"2024-03-28T18:06:10.604922+0000","last_peered":"2024-03-28T18:06:10.604922+0000","last_clean":"2024-03-28T18:06:10.604922+0000","last_became_active":"2024-03-28T18:04:49.537202+0000","last_became_peered":"2024-03-28T18:04:49.537202+0000","last_unstale":"2024-03-28T18:06:10.604922+0000","last_undegraded":"2024-03-28T18:06:10.604922+0000","last_fullsized":"2024-03-28T18:06:10.604922+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_clean_scrub_stamp":"2024-03-28T18:04:48.465760+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T22:35:19.060311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"90'7","reported_seq":26,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-28T18:05:11.732185+0000","last_change":"2024-03-28T18:04:48.513722+0000","last_active":"2024-03-28T18:05:11.732185+0000","last_peered":"2024-03-28T18:05:11.732185+0000","last_clean":"2024-03-28T18:05:11.732185+0000","last_became_active":"2024-03-28T18:04:48.513048+0000","last_became_peered":"2024-03-28T18:04:48.513048+0000","last_unstale":"2024-03-28T18:05:11.732185+0000","last_undegraded":"2024-03-28T18:05:11.732185+0000","last_fullsized":"2024-03-28T18:05:11.732185+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:04:47.412563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:04:47.412563+0000","last_clean_scrub_stamp":"2024-03-28T18:04:47.412563+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T04:48:10.831297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"92'8","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-28T18:06:10.612752+0000","last_change":"2024-03-28T18:04:49.539172+0000","last_active":"2024-03-28T18:06:10.612752+0000","last_peered":"2024-03-28T18:06:10.612752+0000","last_clean":"2024-03-28T18:06:10.612752+0000","last_became_active":"2024-03-28T18:04:49.538779+0000","last_became_peered":"2024-03-28T18:04:49.538779+0000","last_unstale":"2024-03-28T18:06:10.612752+0000","last_undegraded":"2024-03-28T18:06:10.612752+0000","last_fullsized":"2024-03-28T18:06:10.612752+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_clean_scrub_stamp":"2024-03-28T18:04:48.465760+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T04:59:08.361762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"90'3","reported_seq":23,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-28T18:05:11.950057+0000","last_change":"2024-03-28T18:04:48.500471+0000","last_active":"2024-03-28T18:05:11.950057+0000","last_peered":"2024-03-28T18:05:11.950057+0000","last_clean":"2024-03-28T18:05:11.950057+0000","last_became_active":"2024-03-28T18:04:48.500094+0000","last_became_peered":"2024-03-28T18:04:48.500094+0000","last_unstale":"2024-03-28T18:05:11.950057+0000","last_undegraded":"2024-03-28T18:05:11.950057+0000","last_fullsized":"2024-03-28T18:05:11.950057+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:04:47.412563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:04:47.412563+0000","last_clean_scrub_stamp":"2024-03-28T18:04:47.412563+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T21:52:51.964254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"92'8","reported_seq":26,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-28T18:06:10.600363+0000","last_change":"2024-03-28T18:04:49.522120+0000","last_active":"2024-03-28T18:06:10.600363+0000","last_peered":"2024-03-28T18:06:10.600363+0000","last_clean":"2024-03-28T18:06:10.600363+0000","last_became_active":"2024-03-28T18:04:49.521656+0000","last_became_peered":"2024-03-28T18:04:49.521656+0000","last_unstale":"2024-03-28T18:06:10.600363+0000","last_undegraded":"2024-03-28T18:06:10.600363+0000","last_fullsized":"2024-03-28T18:06:10.600363+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_clean_scrub_stamp":"2024-03-28T18:04:48.465760+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T18:29:55.605765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"90'3","reported_seq":24,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-28T18:05:11.741130+0000","last_change":"2024-03-28T18:04:48.508723+0000","last_active":"2024-03-28T18:05:11.741130+0000","last_peered":"2024-03-28T18:05:11.741130+0000","last_clean":"2024-03-28T18:05:11.741130+0000","last_became_active":"2024-03-28T18:04:48.508314+0000","last_became_peered":"2024-03-28T18:04:48.508314+0000","last_unstale":"2024-03-28T18:05:11.741130+0000","last_undegraded":"2024-03-28T18:05:11.741130+0000","last_fullsized":"2024-03-28T18:05:11.741130+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:04:47.412563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:04:47.412563+0000","last_clean_scrub_stamp":"2024-03-28T18:04:47.412563+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T03:03:59.353454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"92'11","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-28T18:06:10.611886+0000","last_change":"2024-03-28T18:04:49.531469+0000","last_active":"2024-03-28T18:06:10.611886+0000","last_peered":"2024-03-28T18:06:10.611886+0000","last_clean":"2024-03-28T18:06:10.611886+0000","last_became_active":"2024-03-28T18:04:49.529263+0000","last_became_peered":"2024-03-28T18:04:49.529263+0000","last_unstale":"2024-03-28T18:06:10.611886+0000","last_undegraded":"2024-03-28T18:06:10.611886+0000","last_fullsized":"2024-03-28T18:06:10.611886+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_clean_scrub_stamp":"2024-03-28T18:04:48.465760+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T05:46:31.593877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"92'7","reported_seq":305,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-28T18:07:15.023693+0000","last_change":"2024-03-28T18:04:48.500644+0000","last_active":"2024-03-28T18:07:15.023693+0000","last_peered":"2024-03-28T18:07:15.023693+0000","last_clean":"2024-03-28T18:07:15.023693+0000","last_became_active":"2024-03-28T18:04:48.499330+0000","last_became_peered":"2024-03-28T18:04:48.499330+0000","last_unstale":"2024-03-28T18:07:15.023693+0000","last_undegraded":"2024-03-28T18:07:15.023693+0000","last_fullsized":"2024-03-28T18:07:15.023693+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:04:47.412563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:04:47.412563+0000","last_clean_scrub_stamp":"2024-03-28T18:04:47.412563+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T18:50:54.361266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":278,"num_read_kb":278,"num_write":8,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"92'10","reported_seq":28,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-28T18:06:10.623432+0000","last_change":"2024-03-28T18:04:49.528728+0000","last_active":"2024-03-28T18:06:10.623432+0000","last_peered":"2024-03-28T18:06:10.623432+0000","last_clean":"2024-03-28T18:06:10.623432+0000","last_became_active":"2024-03-28T18:04:49.528312+0000","last_became_peered":"2024-03-28T18:04:49.528312+0000","last_unstale":"2024-03-28T18:06:10.623432+0000","last_undegraded":"2024-03-28T18:06:10.623432+0000","last_fullsized":"2024-03-28T18:06:10.623432+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_clean_scrub_stamp":"2024-03-28T18:04:48.465760+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T05:59:37.308002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"90'4","reported_seq":24,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-28T18:05:11.736913+0000","last_change":"2024-03-28T18:04:48.508766+0000","last_active":"2024-03-28T18:05:11.736913+0000","last_peered":"2024-03-28T18:05:11.736913+0000","last_clean":"2024-03-28T18:05:11.736913+0000","last_became_active":"2024-03-28T18:04:48.506780+0000","last_became_peered":"2024-03-28T18:04:48.506780+0000","last_unstale":"2024-03-28T18:05:11.736913+0000","last_undegraded":"2024-03-28T18:05:11.736913+0000","last_fullsized":"2024-03-28T18:05:11.736913+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:04:47.412563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:04:47.412563+0000","last_clean_scrub_stamp":"2024-03-28T18:04:47.412563+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T04:39:07.394655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"92'9","reported_seq":27,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-28T18:06:10.610898+0000","last_change":"2024-03-28T18:04:49.537165+0000","last_active":"2024-03-28T18:06:10.610898+0000","last_peered":"2024-03-28T18:06:10.610898+0000","last_clean":"2024-03-28T18:06:10.610898+0000","last_became_active":"2024-03-28T18:04:49.534543+0000","last_became_peered":"2024-03-28T18:04:49.534543+0000","last_unstale":"2024-03-28T18:06:10.610898+0000","last_undegraded":"2024-03-28T18:06:10.610898+0000","last_fullsized":"2024-03-28T18:06:10.610898+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_clean_scrub_stamp":"2024-03-28T18:04:48.465760+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T03:15:58.056148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"90'5","reported_seq":26,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-28T18:05:11.746431+0000","last_change":"2024-03-28T18:04:48.503866+0000","last_active":"2024-03-28T18:05:11.746431+0000","last_peered":"2024-03-28T18:05:11.746431+0000","last_clean":"2024-03-28T18:05:11.746431+0000","last_became_active":"2024-03-28T18:04:48.503223+0000","last_became_peered":"2024-03-28T18:04:48.503223+0000","last_unstale":"2024-03-28T18:05:11.746431+0000","last_undegraded":"2024-03-28T18:05:11.746431+0000","last_fullsized":"2024-03-28T18:05:11.746431+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:04:47.412563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:04:47.412563+0000","last_clean_scrub_stamp":"2024-03-28T18:04:47.412563+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T20:14:01.307483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"92'11","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-28T18:06:10.624750+0000","last_change":"2024-03-28T18:04:49.539494+0000","last_active":"2024-03-28T18:06:10.624750+0000","last_peered":"2024-03-28T18:06:10.624750+0000","last_clean":"2024-03-28T18:06:10.624750+0000","last_became_active":"2024-03-28T18:04:49.539044+0000","last_became_peered":"2024-03-28T18:04:49.539044+0000","last_unstale":"2024-03-28T18:06:10.624750+0000","last_undegraded":"2024-03-28T18:06:10.624750+0000","last_fullsized":"2024-03-28T18:06:10.624750+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_clean_scrub_stamp":"2024-03-28T18:04:48.465760+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T04:08:30.807788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"92'34","reported_seq":341,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-28T18:07:15.010701+0000","last_change":"2024-03-28T18:04:48.507703+0000","last_active":"2024-03-28T18:07:15.010701+0000","last_peered":"2024-03-28T18:07:15.010701+0000","last_clean":"2024-03-28T18:07:15.010701+0000","last_became_active":"2024-03-28T18:04:48.507310+0000","last_became_peered":"2024-03-28T18:04:48.507310+0000","last_unstale":"2024-03-28T18:07:15.010701+0000","last_undegraded":"2024-03-28T18:07:15.010701+0000","last_fullsized":"2024-03-28T18:07:15.010701+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:04:47.412563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:04:47.412563+0000","last_clean_scrub_stamp":"2024-03-28T18:04:47.412563+0000","objects_scrubbed":0,"log_size":34,"log_dups_size":0,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T19:14:13.910042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":64747,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":287,"num_read_kb":207,"num_write":36,"num_write_kb":77,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"92'6","reported_seq":24,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-28T18:06:10.618389+0000","last_change":"2024-03-28T18:04:49.529928+0000","last_active":"2024-03-28T18:06:10.618389+0000","last_peered":"2024-03-28T18:06:10.618389+0000","last_clean":"2024-03-28T18:06:10.618389+0000","last_became_active":"2024-03-28T18:04:49.529527+0000","last_became_peered":"2024-03-28T18:04:49.529527+0000","last_unstale":"2024-03-28T18:06:10.618389+0000","last_undegraded":"2024-03-28T18:06:10.618389+0000","last_fullsized":"2024-03-28T18:06:10.618389+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_clean_scrub_stamp":"2024-03-28T18:04:48.465760+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T05:50:07.129140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"90'9","reported_seq":31,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-28T18:05:11.749738+0000","last_change":"2024-03-28T18:04:48.494746+0000","last_active":"2024-03-28T18:05:11.749738+0000","last_peered":"2024-03-28T18:05:11.749738+0000","last_clean":"2024-03-28T18:05:11.749738+0000","last_became_active":"2024-03-28T18:04:48.494296+0000","last_became_peered":"2024-03-28T18:04:48.494296+0000","last_unstale":"2024-03-28T18:05:11.749738+0000","last_undegraded":"2024-03-28T18:05:11.749738+0000","last_fullsized":"2024-03-28T18:05:11.749738+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:04:47.412563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:04:47.412563+0000","last_clean_scrub_stamp":"2024-03-28T18:04:47.412563+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T22:32:56.923067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"92'9","reported_seq":27,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-28T18:06:10.624303+0000","last_change":"2024-03-28T18:04:49.528070+0000","last_active":"2024-03-28T18:06:10.624303+0000","last_peered":"2024-03-28T18:06:10.624303+0000","last_clean":"2024-03-28T18:06:10.624303+0000","last_became_active":"2024-03-28T18:04:49.526747+0000","last_became_peered":"2024-03-28T18:04:49.526747+0000","last_unstale":"2024-03-28T18:06:10.624303+0000","last_undegraded":"2024-03-28T18:06:10.624303+0000","last_fullsized":"2024-03-28T18:06:10.624303+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_clean_scrub_stamp":"2024-03-28T18:04:48.465760+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T22:32:05.750094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"90'5","reported_seq":26,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-28T18:05:11.741413+0000","last_change":"2024-03-28T18:04:48.505723+0000","last_active":"2024-03-28T18:05:11.741413+0000","last_peered":"2024-03-28T18:05:11.741413+0000","last_clean":"2024-03-28T18:05:11.741413+0000","last_became_active":"2024-03-28T18:04:48.504088+0000","last_became_peered":"2024-03-28T18:04:48.504088+0000","last_unstale":"2024-03-28T18:05:11.741413+0000","last_undegraded":"2024-03-28T18:05:11.741413+0000","last_fullsized":"2024-03-28T18:05:11.741413+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:04:47.412563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:04:47.412563+0000","last_clean_scrub_stamp":"2024-03-28T18:04:47.412563+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T23:25:37.138054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"92'15","reported_seq":33,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-28T18:06:10.625240+0000","last_change":"2024-03-28T18:04:49.518872+0000","last_active":"2024-03-28T18:06:10.625240+0000","last_peered":"2024-03-28T18:06:10.625240+0000","last_clean":"2024-03-28T18:06:10.625240+0000","last_became_active":"2024-03-28T18:04:49.518507+0000","last_became_peered":"2024-03-28T18:04:49.518507+0000","last_unstale":"2024-03-28T18:06:10.625240+0000","last_undegraded":"2024-03-28T18:06:10.625240+0000","last_fullsized":"2024-03-28T18:06:10.625240+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_clean_scrub_stamp":"2024-03-28T18:04:48.465760+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T20:11:31.044321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"92'8","reported_seq":171,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-28T18:07:14.021044+0000","last_change":"2024-03-28T18:04:48.503058+0000","last_active":"2024-03-28T18:07:14.021044+0000","last_peered":"2024-03-28T18:07:14.021044+0000","last_clean":"2024-03-28T18:07:14.021044+0000","last_became_active":"2024-03-28T18:04:48.501596+0000","last_became_peered":"2024-03-28T18:04:48.501596+0000","last_unstale":"2024-03-28T18:07:14.021044+0000","last_undegraded":"2024-03-28T18:07:14.021044+0000","last_fullsized":"2024-03-28T18:07:14.021044+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:04:47.412563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:04:47.412563+0000","last_clean_scrub_stamp":"2024-03-28T18:04:47.412563+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T23:46:10.442771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1613,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":144,"num_read_kb":5,"num_write":8,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"92'6","reported_seq":24,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-28T18:06:10.614449+0000","last_change":"2024-03-28T18:04:49.535102+0000","last_active":"2024-03-28T18:06:10.614449+0000","last_peered":"2024-03-28T18:06:10.614449+0000","last_clean":"2024-03-28T18:06:10.614449+0000","last_became_active":"2024-03-28T18:04:49.534171+0000","last_became_peered":"2024-03-28T18:04:49.534171+0000","last_unstale":"2024-03-28T18:06:10.614449+0000","last_undegraded":"2024-03-28T18:06:10.614449+0000","last_fullsized":"2024-03-28T18:06:10.614449+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_clean_scrub_stamp":"2024-03-28T18:04:48.465760+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T18:16:27.703327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"90'17","reported_seq":38,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-28T18:05:11.734330+0000","last_change":"2024-03-28T18:04:48.517034+0000","last_active":"2024-03-28T18:05:11.734330+0000","last_peered":"2024-03-28T18:05:11.734330+0000","last_clean":"2024-03-28T18:05:11.734330+0000","last_became_active":"2024-03-28T18:04:48.516562+0000","last_became_peered":"2024-03-28T18:04:48.516562+0000","last_unstale":"2024-03-28T18:05:11.734330+0000","last_undegraded":"2024-03-28T18:05:11.734330+0000","last_fullsized":"2024-03-28T18:05:11.734330+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:04:47.412563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:04:47.412563+0000","last_clean_scrub_stamp":"2024-03-28T18:04:47.412563+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T02:28:55.753779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"92'6","reported_seq":24,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-28T18:06:10.617799+0000","last_change":"2024-03-28T18:04:49.541501+0000","last_active":"2024-03-28T18:06:10.617799+0000","last_peered":"2024-03-28T18:06:10.617799+0000","last_clean":"2024-03-28T18:06:10.617799+0000","last_became_active":"2024-03-28T18:04:49.541117+0000","last_became_peered":"2024-03-28T18:04:49.541117+0000","last_unstale":"2024-03-28T18:06:10.617799+0000","last_undegraded":"2024-03-28T18:06:10.617799+0000","last_fullsized":"2024-03-28T18:06:10.617799+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_clean_scrub_stamp":"2024-03-28T18:04:48.465760+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T04:18:29.081489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"92'3","reported_seq":24,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-28T18:07:03.790727+0000","last_change":"2024-03-28T18:04:48.510859+0000","last_active":"2024-03-28T18:07:03.790727+0000","last_peered":"2024-03-28T18:07:03.790727+0000","last_clean":"2024-03-28T18:07:03.790727+0000","last_became_active":"2024-03-28T18:04:48.510522+0000","last_became_peered":"2024-03-28T18:04:48.510522+0000","last_unstale":"2024-03-28T18:07:03.790727+0000","last_undegraded":"2024-03-28T18:07:03.790727+0000","last_fullsized":"2024-03-28T18:07:03.790727+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:04:47.412563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:04:47.412563+0000","last_clean_scrub_stamp":"2024-03-28T18:04:47.412563+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T22:31:37.817361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"92'5","reported_seq":23,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-28T18:06:10.607186+0000","last_change":"2024-03-28T18:04:49.541981+0000","last_active":"2024-03-28T18:06:10.607186+0000","last_peered":"2024-03-28T18:06:10.607186+0000","last_clean":"2024-03-28T18:06:10.607186+0000","last_became_active":"2024-03-28T18:04:49.541606+0000","last_became_peered":"2024-03-28T18:04:49.541606+0000","last_unstale":"2024-03-28T18:06:10.607186+0000","last_undegraded":"2024-03-28T18:06:10.607186+0000","last_fullsized":"2024-03-28T18:06:10.607186+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_clean_scrub_stamp":"2024-03-28T18:04:48.465760+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T21:56:03.632490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"90'1","reported_seq":21,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-28T18:05:11.730922+0000","last_change":"2024-03-28T18:04:48.505847+0000","last_active":"2024-03-28T18:05:11.730922+0000","last_peered":"2024-03-28T18:05:11.730922+0000","last_clean":"2024-03-28T18:05:11.730922+0000","last_became_active":"2024-03-28T18:04:48.501005+0000","last_became_peered":"2024-03-28T18:04:48.501005+0000","last_unstale":"2024-03-28T18:05:11.730922+0000","last_undegraded":"2024-03-28T18:05:11.730922+0000","last_fullsized":"2024-03-28T18:05:11.730922+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:04:47.412563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:04:47.412563+0000","last_clean_scrub_stamp":"2024-03-28T18:04:47.412563+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T01:33:45.539324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"92'5","reported_seq":23,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-28T18:06:10.614333+0000","last_change":"2024-03-28T18:04:49.526938+0000","last_active":"2024-03-28T18:06:10.614333+0000","last_peered":"2024-03-28T18:06:10.614333+0000","last_clean":"2024-03-28T18:06:10.614333+0000","last_became_active":"2024-03-28T18:04:49.526290+0000","last_became_peered":"2024-03-28T18:04:49.526290+0000","last_unstale":"2024-03-28T18:06:10.614333+0000","last_undegraded":"2024-03-28T18:06:10.614333+0000","last_fullsized":"2024-03-28T18:06:10.614333+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_clean_scrub_stamp":"2024-03-28T18:04:48.465760+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T03:06:54.561470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"92'6","reported_seq":169,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-28T18:07:15.990542+0000","last_change":"2024-03-28T18:04:48.512553+0000","last_active":"2024-03-28T18:07:15.990542+0000","last_peered":"2024-03-28T18:07:15.990542+0000","last_clean":"2024-03-28T18:07:15.990542+0000","last_became_active":"2024-03-28T18:04:48.511520+0000","last_became_peered":"2024-03-28T18:04:48.511520+0000","last_unstale":"2024-03-28T18:07:15.990542+0000","last_undegraded":"2024-03-28T18:07:15.990542+0000","last_fullsized":"2024-03-28T18:07:15.990542+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:04:47.412563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:04:47.412563+0000","last_clean_scrub_stamp":"2024-03-28T18:04:47.412563+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T21:13:32.970027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1081,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":143,"num_read_kb":2,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"92'7","reported_seq":31,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-28T18:06:10.613152+0000","last_change":"2024-03-28T18:05:11.739596+0000","last_active":"2024-03-28T18:06:10.613152+0000","last_peered":"2024-03-28T18:06:10.613152+0000","last_clean":"2024-03-28T18:06:10.613152+0000","last_became_active":"2024-03-28T18:05:11.739156+0000","last_became_peered":"2024-03-28T18:05:11.739156+0000","last_unstale":"2024-03-28T18:06:10.613152+0000","last_undegraded":"2024-03-28T18:06:10.613152+0000","last_fullsized":"2024-03-28T18:06:10.613152+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_clean_scrub_stamp":"2024-03-28T18:04:48.465760+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T05:23:00.446713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,0],"acting":[10,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"92'8","reported_seq":26,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-28T18:06:10.609498+0000","last_change":"2024-03-28T18:04:49.540438+0000","last_active":"2024-03-28T18:06:10.609498+0000","last_peered":"2024-03-28T18:06:10.609498+0000","last_clean":"2024-03-28T18:06:10.609498+0000","last_became_active":"2024-03-28T18:04:49.540069+0000","last_became_peered":"2024-03-28T18:04:49.540069+0000","last_unstale":"2024-03-28T18:06:10.609498+0000","last_undegraded":"2024-03-28T18:06:10.609498+0000","last_fullsized":"2024-03-28T18:06:10.609498+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_clean_scrub_stamp":"2024-03-28T18:04:48.465760+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T19:31:49.254101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"92'7","reported_seq":167,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-28T18:07:14.020116+0000","last_change":"2024-03-28T18:04:48.508839+0000","last_active":"2024-03-28T18:07:14.020116+0000","last_peered":"2024-03-28T18:07:14.020116+0000","last_clean":"2024-03-28T18:07:14.020116+0000","last_became_active":"2024-03-28T18:04:48.508475+0000","last_became_peered":"2024-03-28T18:04:48.508475+0000","last_unstale":"2024-03-28T18:07:14.020116+0000","last_undegraded":"2024-03-28T18:07:14.020116+0000","last_fullsized":"2024-03-28T18:07:14.020116+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:04:47.412563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:04:47.412563+0000","last_clean_scrub_stamp":"2024-03-28T18:04:47.412563+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T04:31:10.463284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":140,"num_read_kb":140,"num_write":7,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"92'8","reported_seq":26,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-28T18:06:10.621737+0000","last_change":"2024-03-28T18:04:49.534449+0000","last_active":"2024-03-28T18:06:10.621737+0000","last_peered":"2024-03-28T18:06:10.621737+0000","last_clean":"2024-03-28T18:06:10.621737+0000","last_became_active":"2024-03-28T18:04:49.534098+0000","last_became_peered":"2024-03-28T18:04:49.534098+0000","last_unstale":"2024-03-28T18:06:10.621737+0000","last_undegraded":"2024-03-28T18:06:10.621737+0000","last_fullsized":"2024-03-28T18:06:10.621737+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_clean_scrub_stamp":"2024-03-28T18:04:48.465760+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T23:47:07.886931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"90'6","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-28T18:05:11.948432+0000","last_change":"2024-03-28T18:04:48.501790+0000","last_active":"2024-03-28T18:05:11.948432+0000","last_peered":"2024-03-28T18:05:11.948432+0000","last_clean":"2024-03-28T18:05:11.948432+0000","last_became_active":"2024-03-28T18:04:48.501415+0000","last_became_peered":"2024-03-28T18:04:48.501415+0000","last_unstale":"2024-03-28T18:05:11.948432+0000","last_undegraded":"2024-03-28T18:05:11.948432+0000","last_fullsized":"2024-03-28T18:05:11.948432+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:04:47.412563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:04:47.412563+0000","last_clean_scrub_stamp":"2024-03-28T18:04:47.412563+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T19:17:22.319883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"22'137","reported_seq":366,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-28T18:05:11.735054+0000","last_change":"2024-03-28T18:03:13.522588+0000","last_active":"2024-03-28T18:05:11.735054+0000","last_peered":"2024-03-28T18:05:11.735054+0000","last_clean":"2024-03-28T18:05:11.735054+0000","last_became_active":"2024-03-28T18:03:13.521185+0000","last_became_peered":"2024-03-28T18:03:13.521185+0000","last_unstale":"2024-03-28T18:05:11.735054+0000","last_undegraded":"2024-03-28T18:05:11.735054+0000","last_fullsized":"2024-03-28T18:05:11.735054+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T17:58:33.169981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T17:58:33.169981+0000","last_clean_scrub_stamp":"2024-03-28T17:58:33.169981+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T19:10:06.087374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"92'5","reported_seq":23,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-28T18:06:10.607609+0000","last_change":"2024-03-28T18:04:49.538619+0000","last_active":"2024-03-28T18:06:10.607609+0000","last_peered":"2024-03-28T18:06:10.607609+0000","last_clean":"2024-03-28T18:06:10.607609+0000","last_became_active":"2024-03-28T18:04:49.538276+0000","last_became_peered":"2024-03-28T18:04:49.538276+0000","last_unstale":"2024-03-28T18:06:10.607609+0000","last_undegraded":"2024-03-28T18:06:10.607609+0000","last_fullsized":"2024-03-28T18:06:10.607609+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_clean_scrub_stamp":"2024-03-28T18:04:48.465760+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T01:39:37.139544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"90'4","reported_seq":24,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-28T18:05:11.735222+0000","last_change":"2024-03-28T18:04:48.513621+0000","last_active":"2024-03-28T18:05:11.735222+0000","last_peered":"2024-03-28T18:05:11.735222+0000","last_clean":"2024-03-28T18:05:11.735222+0000","last_became_active":"2024-03-28T18:04:48.513019+0000","last_became_peered":"2024-03-28T18:04:48.513019+0000","last_unstale":"2024-03-28T18:05:11.735222+0000","last_undegraded":"2024-03-28T18:05:11.735222+0000","last_fullsized":"2024-03-28T18:05:11.735222+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:04:47.412563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:04:47.412563+0000","last_clean_scrub_stamp":"2024-03-28T18:04:47.412563+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T23:38:18.082830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"90'4","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-28T18:05:11.735056+0000","last_change":"2024-03-28T18:04:48.513999+0000","last_active":"2024-03-28T18:05:11.735056+0000","last_peered":"2024-03-28T18:05:11.735056+0000","last_clean":"2024-03-28T18:05:11.735056+0000","last_became_active":"2024-03-28T18:04:48.512836+0000","last_became_peered":"2024-03-28T18:04:48.512836+0000","last_unstale":"2024-03-28T18:05:11.735056+0000","last_undegraded":"2024-03-28T18:05:11.735056+0000","last_fullsized":"2024-03-28T18:05:11.735056+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:04:47.412563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:04:47.412563+0000","last_clean_scrub_stamp":"2024-03-28T18:04:47.412563+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T18:29:42.015843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"92'9","reported_seq":27,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-28T18:06:10.605559+0000","last_change":"2024-03-28T18:04:49.519323+0000","last_active":"2024-03-28T18:06:10.605559+0000","last_peered":"2024-03-28T18:06:10.605559+0000","last_clean":"2024-03-28T18:06:10.605559+0000","last_became_active":"2024-03-28T18:04:49.518898+0000","last_became_peered":"2024-03-28T18:04:49.518898+0000","last_unstale":"2024-03-28T18:06:10.605559+0000","last_undegraded":"2024-03-28T18:06:10.605559+0000","last_fullsized":"2024-03-28T18:06:10.605559+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_clean_scrub_stamp":"2024-03-28T18:04:48.465760+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T18:41:12.683606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"92'16","reported_seq":36,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-28T18:07:11.878264+0000","last_change":"2024-03-28T18:04:48.512673+0000","last_active":"2024-03-28T18:07:11.878264+0000","last_peered":"2024-03-28T18:07:11.878264+0000","last_clean":"2024-03-28T18:07:11.878264+0000","last_became_active":"2024-03-28T18:04:48.512137+0000","last_became_peered":"2024-03-28T18:04:48.512137+0000","last_unstale":"2024-03-28T18:07:11.878264+0000","last_undegraded":"2024-03-28T18:07:11.878264+0000","last_fullsized":"2024-03-28T18:07:11.878264+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:04:47.412563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:04:47.412563+0000","last_clean_scrub_stamp":"2024-03-28T18:04:47.412563+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T21:29:41.731701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":28,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"92'5","reported_seq":23,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-28T18:06:10.624351+0000","last_change":"2024-03-28T18:04:49.525055+0000","last_active":"2024-03-28T18:06:10.624351+0000","last_peered":"2024-03-28T18:06:10.624351+0000","last_clean":"2024-03-28T18:06:10.624351+0000","last_became_active":"2024-03-28T18:04:49.522038+0000","last_became_peered":"2024-03-28T18:04:49.522038+0000","last_unstale":"2024-03-28T18:06:10.624351+0000","last_undegraded":"2024-03-28T18:06:10.624351+0000","last_fullsized":"2024-03-28T18:06:10.624351+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_clean_scrub_stamp":"2024-03-28T18:04:48.465760+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T05:21:44.970397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"90'7","reported_seq":28,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-28T18:05:11.949110+0000","last_change":"2024-03-28T18:04:48.509951+0000","last_active":"2024-03-28T18:05:11.949110+0000","last_peered":"2024-03-28T18:05:11.949110+0000","last_clean":"2024-03-28T18:05:11.949110+0000","last_became_active":"2024-03-28T18:04:48.509141+0000","last_became_peered":"2024-03-28T18:04:48.509141+0000","last_unstale":"2024-03-28T18:05:11.949110+0000","last_undegraded":"2024-03-28T18:05:11.949110+0000","last_fullsized":"2024-03-28T18:05:11.949110+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:04:47.412563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:04:47.412563+0000","last_clean_scrub_stamp":"2024-03-28T18:04:47.412563+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T03:54:05.569940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"92'7","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-28T18:06:10.612768+0000","last_change":"2024-03-28T18:04:49.537174+0000","last_active":"2024-03-28T18:06:10.612768+0000","last_peered":"2024-03-28T18:06:10.612768+0000","last_clean":"2024-03-28T18:06:10.612768+0000","last_became_active":"2024-03-28T18:04:49.536801+0000","last_became_peered":"2024-03-28T18:04:49.536801+0000","last_unstale":"2024-03-28T18:06:10.612768+0000","last_undegraded":"2024-03-28T18:06:10.612768+0000","last_fullsized":"2024-03-28T18:06:10.612768+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_clean_scrub_stamp":"2024-03-28T18:04:48.465760+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T05:27:55.908196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"90'2","reported_seq":32,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-28T18:05:11.757387+0000","last_change":"2024-03-28T18:05:11.756894+0000","last_active":"2024-03-28T18:05:11.757387+0000","last_peered":"2024-03-28T18:05:11.757387+0000","last_clean":"2024-03-28T18:05:11.757387+0000","last_became_active":"2024-03-28T18:05:11.750825+0000","last_became_peered":"2024-03-28T18:05:11.750825+0000","last_unstale":"2024-03-28T18:05:11.757387+0000","last_undegraded":"2024-03-28T18:05:11.757387+0000","last_fullsized":"2024-03-28T18:05:11.757387+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:04:47.412563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:04:47.412563+0000","last_clean_scrub_stamp":"2024-03-28T18:04:47.412563+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T23:11:42.105850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,11,0],"acting":[1,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"92'2","reported_seq":20,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-28T18:06:10.611454+0000","last_change":"2024-03-28T18:04:49.523970+0000","last_active":"2024-03-28T18:06:10.611454+0000","last_peered":"2024-03-28T18:06:10.611454+0000","last_clean":"2024-03-28T18:06:10.611454+0000","last_became_active":"2024-03-28T18:04:49.523559+0000","last_became_peered":"2024-03-28T18:04:49.523559+0000","last_unstale":"2024-03-28T18:06:10.611454+0000","last_undegraded":"2024-03-28T18:06:10.611454+0000","last_fullsized":"2024-03-28T18:06:10.611454+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_clean_scrub_stamp":"2024-03-28T18:04:48.465760+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T05:50:25.593232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"90'7","reported_seq":27,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-28T18:05:11.739263+0000","last_change":"2024-03-28T18:04:48.492067+0000","last_active":"2024-03-28T18:05:11.739263+0000","last_peered":"2024-03-28T18:05:11.739263+0000","last_clean":"2024-03-28T18:05:11.739263+0000","last_became_active":"2024-03-28T18:04:48.490633+0000","last_became_peered":"2024-03-28T18:04:48.490633+0000","last_unstale":"2024-03-28T18:05:11.739263+0000","last_undegraded":"2024-03-28T18:05:11.739263+0000","last_fullsized":"2024-03-28T18:05:11.739263+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:04:47.412563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:04:47.412563+0000","last_clean_scrub_stamp":"2024-03-28T18:04:47.412563+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T23:00:52.703618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"92'9","reported_seq":27,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-28T18:06:10.608650+0000","last_change":"2024-03-28T18:04:49.531554+0000","last_active":"2024-03-28T18:06:10.608650+0000","last_peered":"2024-03-28T18:06:10.608650+0000","last_clean":"2024-03-28T18:06:10.608650+0000","last_became_active":"2024-03-28T18:04:49.530273+0000","last_became_peered":"2024-03-28T18:04:49.530273+0000","last_unstale":"2024-03-28T18:06:10.608650+0000","last_undegraded":"2024-03-28T18:06:10.608650+0000","last_fullsized":"2024-03-28T18:06:10.608650+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_clean_scrub_stamp":"2024-03-28T18:04:48.465760+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T22:17:04.032614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"92'5","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-28T18:07:11.597743+0000","last_change":"2024-03-28T18:04:48.514974+0000","last_active":"2024-03-28T18:07:11.597743+0000","last_peered":"2024-03-28T18:07:11.597743+0000","last_clean":"2024-03-28T18:07:11.597743+0000","last_became_active":"2024-03-28T18:04:48.514356+0000","last_became_peered":"2024-03-28T18:04:48.514356+0000","last_unstale":"2024-03-28T18:07:11.597743+0000","last_undegraded":"2024-03-28T18:07:11.597743+0000","last_fullsized":"2024-03-28T18:07:11.597743+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:04:47.412563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:04:47.412563+0000","last_clean_scrub_stamp":"2024-03-28T18:04:47.412563+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T01:26:37.202440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"92'8","reported_seq":26,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-28T18:06:10.613171+0000","last_change":"2024-03-28T18:04:49.518064+0000","last_active":"2024-03-28T18:06:10.613171+0000","last_peered":"2024-03-28T18:06:10.613171+0000","last_clean":"2024-03-28T18:06:10.613171+0000","last_became_active":"2024-03-28T18:04:49.517674+0000","last_became_peered":"2024-03-28T18:04:49.517674+0000","last_unstale":"2024-03-28T18:06:10.613171+0000","last_undegraded":"2024-03-28T18:06:10.613171+0000","last_fullsized":"2024-03-28T18:06:10.613171+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_clean_scrub_stamp":"2024-03-28T18:04:48.465760+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T00:10:44.537164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"90'4","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-28T18:05:11.734862+0000","last_change":"2024-03-28T18:04:48.517028+0000","last_active":"2024-03-28T18:05:11.734862+0000","last_peered":"2024-03-28T18:05:11.734862+0000","last_clean":"2024-03-28T18:05:11.734862+0000","last_became_active":"2024-03-28T18:04:48.516387+0000","last_became_peered":"2024-03-28T18:04:48.516387+0000","last_unstale":"2024-03-28T18:05:11.734862+0000","last_undegraded":"2024-03-28T18:05:11.734862+0000","last_fullsized":"2024-03-28T18:05:11.734862+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:04:47.412563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:04:47.412563+0000","last_clean_scrub_stamp":"2024-03-28T18:04:47.412563+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T01:09:24.142422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"92'5","reported_seq":23,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-28T18:06:10.625821+0000","last_change":"2024-03-28T18:04:49.527032+0000","last_active":"2024-03-28T18:06:10.625821+0000","last_peered":"2024-03-28T18:06:10.625821+0000","last_clean":"2024-03-28T18:06:10.625821+0000","last_became_active":"2024-03-28T18:04:49.523888+0000","last_became_peered":"2024-03-28T18:04:49.523888+0000","last_unstale":"2024-03-28T18:06:10.625821+0000","last_undegraded":"2024-03-28T18:06:10.625821+0000","last_fullsized":"2024-03-28T18:06:10.625821+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_clean_scrub_stamp":"2024-03-28T18:04:48.465760+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T01:19:01.265719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"92'6","reported_seq":169,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-28T18:07:15.011653+0000","last_change":"2024-03-28T18:04:48.502638+0000","last_active":"2024-03-28T18:07:15.011653+0000","last_peered":"2024-03-28T18:07:15.011653+0000","last_clean":"2024-03-28T18:07:15.011653+0000","last_became_active":"2024-03-28T18:04:48.502249+0000","last_became_peered":"2024-03-28T18:04:48.502249+0000","last_unstale":"2024-03-28T18:07:15.011653+0000","last_undegraded":"2024-03-28T18:07:15.011653+0000","last_fullsized":"2024-03-28T18:07:15.011653+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:04:47.412563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:04:47.412563+0000","last_clean_scrub_stamp":"2024-03-28T18:04:47.412563+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T03:23:51.740832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1663,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":143,"num_read_kb":3,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"92'7","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-28T18:06:10.626385+0000","last_change":"2024-03-28T18:04:49.532802+0000","last_active":"2024-03-28T18:06:10.626385+0000","last_peered":"2024-03-28T18:06:10.626385+0000","last_clean":"2024-03-28T18:06:10.626385+0000","last_became_active":"2024-03-28T18:04:49.530666+0000","last_became_peered":"2024-03-28T18:04:49.530666+0000","last_unstale":"2024-03-28T18:06:10.626385+0000","last_undegraded":"2024-03-28T18:06:10.626385+0000","last_fullsized":"2024-03-28T18:06:10.626385+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_clean_scrub_stamp":"2024-03-28T18:04:48.465760+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T22:21:28.469402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"90'3","reported_seq":24,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-28T18:05:11.747786+0000","last_change":"2024-03-28T18:04:48.503582+0000","last_active":"2024-03-28T18:05:11.747786+0000","last_peered":"2024-03-28T18:05:11.747786+0000","last_clean":"2024-03-28T18:05:11.747786+0000","last_became_active":"2024-03-28T18:04:48.502716+0000","last_became_peered":"2024-03-28T18:04:48.502716+0000","last_unstale":"2024-03-28T18:05:11.747786+0000","last_undegraded":"2024-03-28T18:05:11.747786+0000","last_fullsized":"2024-03-28T18:05:11.747786+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:04:47.412563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:04:47.412563+0000","last_clean_scrub_stamp":"2024-03-28T18:04:47.412563+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T21:41:06.333420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"92'7","reported_seq":35,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-28T18:06:10.615938+0000","last_change":"2024-03-28T18:05:11.753703+0000","last_active":"2024-03-28T18:06:10.615938+0000","last_peered":"2024-03-28T18:06:10.615938+0000","last_clean":"2024-03-28T18:06:10.615938+0000","last_became_active":"2024-03-28T18:05:11.748793+0000","last_became_peered":"2024-03-28T18:05:11.748793+0000","last_unstale":"2024-03-28T18:06:10.615938+0000","last_undegraded":"2024-03-28T18:06:10.615938+0000","last_fullsized":"2024-03-28T18:06:10.615938+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_clean_scrub_stamp":"2024-03-28T18:04:48.465760+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T19:38:38.432027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":117,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"90'5","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-28T18:05:11.950534+0000","last_change":"2024-03-28T18:04:48.507658+0000","last_active":"2024-03-28T18:05:11.950534+0000","last_peered":"2024-03-28T18:05:11.950534+0000","last_clean":"2024-03-28T18:05:11.950534+0000","last_became_active":"2024-03-28T18:04:48.506855+0000","last_became_peered":"2024-03-28T18:04:48.506855+0000","last_unstale":"2024-03-28T18:05:11.950534+0000","last_undegraded":"2024-03-28T18:05:11.950534+0000","last_fullsized":"2024-03-28T18:05:11.950534+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:04:47.412563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:04:47.412563+0000","last_clean_scrub_stamp":"2024-03-28T18:04:47.412563+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T23:08:23.550379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"92'9","reported_seq":27,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-28T18:06:10.615589+0000","last_change":"2024-03-28T18:04:49.521122+0000","last_active":"2024-03-28T18:06:10.615589+0000","last_peered":"2024-03-28T18:06:10.615589+0000","last_clean":"2024-03-28T18:06:10.615589+0000","last_became_active":"2024-03-28T18:04:49.520711+0000","last_became_peered":"2024-03-28T18:04:49.520711+0000","last_unstale":"2024-03-28T18:06:10.615589+0000","last_undegraded":"2024-03-28T18:06:10.615589+0000","last_fullsized":"2024-03-28T18:06:10.615589+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_clean_scrub_stamp":"2024-03-28T18:04:48.465760+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T19:32:33.556979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"92'9","reported_seq":27,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-28T18:06:10.605702+0000","last_change":"2024-03-28T18:04:49.534929+0000","last_active":"2024-03-28T18:06:10.605702+0000","last_peered":"2024-03-28T18:06:10.605702+0000","last_clean":"2024-03-28T18:06:10.605702+0000","last_became_active":"2024-03-28T18:04:49.534485+0000","last_became_peered":"2024-03-28T18:04:49.534485+0000","last_unstale":"2024-03-28T18:06:10.605702+0000","last_undegraded":"2024-03-28T18:06:10.605702+0000","last_fullsized":"2024-03-28T18:06:10.605702+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_clean_scrub_stamp":"2024-03-28T18:04:48.465760+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T01:47:34.937536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"92'12","reported_seq":36,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-28T18:06:10.606584+0000","last_change":"2024-03-28T18:05:11.752731+0000","last_active":"2024-03-28T18:06:10.606584+0000","last_peered":"2024-03-28T18:06:10.606584+0000","last_clean":"2024-03-28T18:06:10.606584+0000","last_became_active":"2024-03-28T18:05:11.752121+0000","last_became_peered":"2024-03-28T18:05:11.752121+0000","last_unstale":"2024-03-28T18:06:10.606584+0000","last_undegraded":"2024-03-28T18:06:10.606584+0000","last_fullsized":"2024-03-28T18:06:10.606584+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_clean_scrub_stamp":"2024-03-28T18:04:48.465760+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T20:10:02.491231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,8],"acting":[2,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"92'8","reported_seq":26,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-28T18:06:10.609164+0000","last_change":"2024-03-28T18:04:49.531999+0000","last_active":"2024-03-28T18:06:10.609164+0000","last_peered":"2024-03-28T18:06:10.609164+0000","last_clean":"2024-03-28T18:06:10.609164+0000","last_became_active":"2024-03-28T18:04:49.528641+0000","last_became_peered":"2024-03-28T18:04:49.528641+0000","last_unstale":"2024-03-28T18:06:10.609164+0000","last_undegraded":"2024-03-28T18:06:10.609164+0000","last_fullsized":"2024-03-28T18:06:10.609164+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_clean_scrub_stamp":"2024-03-28T18:04:48.465760+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T21:24:43.861635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"92'7","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-28T18:06:10.610718+0000","last_change":"2024-03-28T18:04:49.540417+0000","last_active":"2024-03-28T18:06:10.610718+0000","last_peered":"2024-03-28T18:06:10.610718+0000","last_clean":"2024-03-28T18:06:10.610718+0000","last_became_active":"2024-03-28T18:04:49.539987+0000","last_became_peered":"2024-03-28T18:04:49.539987+0000","last_unstale":"2024-03-28T18:06:10.610718+0000","last_undegraded":"2024-03-28T18:06:10.610718+0000","last_fullsized":"2024-03-28T18:06:10.610718+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_clean_scrub_stamp":"2024-03-28T18:04:48.465760+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T02:53:41.444850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"92'4","reported_seq":22,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-28T18:06:10.609488+0000","last_change":"2024-03-28T18:04:49.540014+0000","last_active":"2024-03-28T18:06:10.609488+0000","last_peered":"2024-03-28T18:06:10.609488+0000","last_clean":"2024-03-28T18:06:10.609488+0000","last_became_active":"2024-03-28T18:04:49.538239+0000","last_became_peered":"2024-03-28T18:04:49.538239+0000","last_unstale":"2024-03-28T18:06:10.609488+0000","last_undegraded":"2024-03-28T18:06:10.609488+0000","last_fullsized":"2024-03-28T18:06:10.609488+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_clean_scrub_stamp":"2024-03-28T18:04:48.465760+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T23:03:58.449336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"92'6","reported_seq":30,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-28T18:06:10.603750+0000","last_change":"2024-03-28T18:05:11.748781+0000","last_active":"2024-03-28T18:06:10.603750+0000","last_peered":"2024-03-28T18:06:10.603750+0000","last_clean":"2024-03-28T18:06:10.603750+0000","last_became_active":"2024-03-28T18:05:11.747710+0000","last_became_peered":"2024-03-28T18:05:11.747710+0000","last_unstale":"2024-03-28T18:06:10.603750+0000","last_undegraded":"2024-03-28T18:06:10.603750+0000","last_fullsized":"2024-03-28T18:06:10.603750+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_clean_scrub_stamp":"2024-03-28T18:04:48.465760+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T02:01:44.708646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,9],"acting":[11,8,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"92'15","reported_seq":33,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-28T18:06:10.618896+0000","last_change":"2024-03-28T18:04:49.531135+0000","last_active":"2024-03-28T18:06:10.618896+0000","last_peered":"2024-03-28T18:06:10.618896+0000","last_clean":"2024-03-28T18:06:10.618896+0000","last_became_active":"2024-03-28T18:04:49.530673+0000","last_became_peered":"2024-03-28T18:04:49.530673+0000","last_unstale":"2024-03-28T18:06:10.618896+0000","last_undegraded":"2024-03-28T18:06:10.618896+0000","last_fullsized":"2024-03-28T18:06:10.618896+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_clean_scrub_stamp":"2024-03-28T18:04:48.465760+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T05:51:13.950946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"92'2","reported_seq":20,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-28T18:06:10.602677+0000","last_change":"2024-03-28T18:04:49.526803+0000","last_active":"2024-03-28T18:06:10.602677+0000","last_peered":"2024-03-28T18:06:10.602677+0000","last_clean":"2024-03-28T18:06:10.602677+0000","last_became_active":"2024-03-28T18:04:49.526474+0000","last_became_peered":"2024-03-28T18:04:49.526474+0000","last_unstale":"2024-03-28T18:06:10.602677+0000","last_undegraded":"2024-03-28T18:06:10.602677+0000","last_fullsized":"2024-03-28T18:06:10.602677+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_clean_scrub_stamp":"2024-03-28T18:04:48.465760+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T01:28:27.530864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"92'6","reported_seq":24,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-28T18:06:10.612723+0000","last_change":"2024-03-28T18:04:49.534015+0000","last_active":"2024-03-28T18:06:10.612723+0000","last_peered":"2024-03-28T18:06:10.612723+0000","last_clean":"2024-03-28T18:06:10.612723+0000","last_became_active":"2024-03-28T18:04:49.533560+0000","last_became_peered":"2024-03-28T18:04:49.533560+0000","last_unstale":"2024-03-28T18:06:10.612723+0000","last_undegraded":"2024-03-28T18:06:10.612723+0000","last_fullsized":"2024-03-28T18:06:10.612723+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_clean_scrub_stamp":"2024-03-28T18:04:48.465760+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T22:41:30.965737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"92'8","reported_seq":26,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-28T18:06:10.608221+0000","last_change":"2024-03-28T18:04:49.523240+0000","last_active":"2024-03-28T18:06:10.608221+0000","last_peered":"2024-03-28T18:06:10.608221+0000","last_clean":"2024-03-28T18:06:10.608221+0000","last_became_active":"2024-03-28T18:04:49.522795+0000","last_became_peered":"2024-03-28T18:04:49.522795+0000","last_unstale":"2024-03-28T18:06:10.608221+0000","last_undegraded":"2024-03-28T18:06:10.608221+0000","last_fullsized":"2024-03-28T18:06:10.608221+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_clean_scrub_stamp":"2024-03-28T18:04:48.465760+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T23:08:37.087977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"92'10","reported_seq":28,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-28T18:06:10.626330+0000","last_change":"2024-03-28T18:04:49.530965+0000","last_active":"2024-03-28T18:06:10.626330+0000","last_peered":"2024-03-28T18:06:10.626330+0000","last_clean":"2024-03-28T18:06:10.626330+0000","last_became_active":"2024-03-28T18:04:49.530619+0000","last_became_peered":"2024-03-28T18:04:49.530619+0000","last_unstale":"2024-03-28T18:06:10.626330+0000","last_undegraded":"2024-03-28T18:06:10.626330+0000","last_fullsized":"2024-03-28T18:06:10.626330+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_clean_scrub_stamp":"2024-03-28T18:04:48.465760+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T05:29:26.754012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"92'11","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-28T18:06:10.624772+0000","last_change":"2024-03-28T18:04:49.523017+0000","last_active":"2024-03-28T18:06:10.624772+0000","last_peered":"2024-03-28T18:06:10.624772+0000","last_clean":"2024-03-28T18:06:10.624772+0000","last_became_active":"2024-03-28T18:04:49.520411+0000","last_became_peered":"2024-03-28T18:04:49.520411+0000","last_unstale":"2024-03-28T18:06:10.624772+0000","last_undegraded":"2024-03-28T18:06:10.624772+0000","last_fullsized":"2024-03-28T18:06:10.624772+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_clean_scrub_stamp":"2024-03-28T18:04:48.465760+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T22:52:45.893036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"92'8","reported_seq":26,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-28T18:06:10.608384+0000","last_change":"2024-03-28T18:04:49.544272+0000","last_active":"2024-03-28T18:06:10.608384+0000","last_peered":"2024-03-28T18:06:10.608384+0000","last_clean":"2024-03-28T18:06:10.608384+0000","last_became_active":"2024-03-28T18:04:49.541754+0000","last_became_peered":"2024-03-28T18:04:49.541754+0000","last_unstale":"2024-03-28T18:06:10.608384+0000","last_undegraded":"2024-03-28T18:06:10.608384+0000","last_fullsized":"2024-03-28T18:06:10.608384+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T18:04:48.465760+0000","last_clean_scrub_stamp":"2024-03-28T18:04:48.465760+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T19:06:00.799916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":117,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":502,"ondisk_log_size":502,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4751360,"data_stored":4722944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":74205,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":1436,"num_read_kb":797,"num_write":267,"num_write_kb":159,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":0,"num_keys_recovered":10,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":573440,"data_stored":218301,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":229,"ondisk_log_size":229,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":85,"seq":365072220206,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28000,"kb_used_data":984,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717152,"statfs":{"total":95995035648,"available":95966363648,"internally_reserved":0,"allocated":1007616,"data_stored":484342,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":10,"up_from":77,"seq":330712481846,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":37008,"kb_used_data":1672,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708144,"statfs":{"total":95995035648,"available":95957139456,"internally_reserved":0,"allocated":1712128,"data_stored":1140650,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":36174272},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":70,"seq":300647710780,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36348,"kb_used_data":1012,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708804,"statfs":{"total":95995035648,"available":95957815296,"internally_reserved":0,"allocated":1036288,"data_stored":483896,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":36174272},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":62,"seq":266287972418,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36340,"kb_used_data":1004,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708812,"statfs":{"total":95995035648,"available":95957823488,"internally_reserved":0,"allocated":1028096,"data_stored":484157,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":7,"up_from":53,"seq":227633266761,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28604,"kb_used_data":1588,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716548,"statfs":{"total":95995035648,"available":95965745152,"internally_reserved":0,"allocated":1626112,"data_stored":1078827,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":48,"seq":206158430287,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36328,"kb_used_data":992,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708824,"statfs":{"total":95995035648,"available":95957835776,"internally_reserved":0,"allocated":1015808,"data_stored":483989,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574963,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36388,"kb_used_data":1052,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708764,"statfs":{"total":95995035648,"available":95957774336,"internally_reserved":0,"allocated":1077248,"data_stored":545335,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738490,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28028,"kb_used_data":1012,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717124,"statfs":{"total":95995035648,"available":95966334976,"internally_reserved":0,"allocated":1036288,"data_stored":485328,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411438,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36412,"kb_used_data":1076,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708740,"statfs":{"total":95995035648,"available":95957749760,"internally_reserved":0,"allocated":1101824,"data_stored":550894,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":3,"up_from":26,"seq":111669149798,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36340,"kb_used_data":1004,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708812,"statfs":{"total":95995035648,"available":95957823488,"internally_reserved":0,"allocated":1028096,"data_stored":483204,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":36174280},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":34,"seq":146028888158,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28008,"kb_used_data":992,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717144,"statfs":{"total":95995035648,"available":95966355456,"internally_reserved":0,"allocated":1015808,"data_stored":484382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":40,"seq":171798691929,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36936,"kb_used_data":1600,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708216,"statfs":{"total":95995035648,"available":95957213184,"internally_reserved":0,"allocated":1638400,"data_stored":1076987,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2671,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":81920,"data_stored":62678,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":106496,"data_stored":68237,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1725,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":3845,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1215,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":5685,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":1500,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":118784,"data_stored":67508,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1685,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-28T18:07:20.020 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-28T18:07:20.174 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:07:19 smithi134 ceph-mon[35037]: pgmap v446: 97 pgs: 97 active+clean; 649 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 1.7 KiB/s wr, 11 op/s 2024-03-28T18:07:20.174 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:07:19 smithi134 ceph-mon[35037]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T18:07:20.174 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:07:19 smithi134 ceph-mon[35037]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T18:07:20.175 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:07:19 smithi134 ceph-mon[35037]: from='client.15426 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-28T18:07:20.175 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:07:19 smithi134 ceph-mon[35037]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T18:07:20.175 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:07:19 smithi134 ceph-mon[35037]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T18:07:20.175 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:07:19 smithi134 ceph-mon[35037]: from='client.35291 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-28T18:07:20.182 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:07:19 smithi079 ceph-mon[35312]: pgmap v446: 97 pgs: 97 active+clean; 649 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 1.7 KiB/s wr, 11 op/s 2024-03-28T18:07:20.182 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:07:19 smithi079 ceph-mon[35312]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T18:07:20.182 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:07:19 smithi079 ceph-mon[35312]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T18:07:20.182 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:07:19 smithi079 ceph-mon[35312]: from='client.15426 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-28T18:07:20.182 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:07:19 smithi079 ceph-mon[35312]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T18:07:20.183 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:07:19 smithi079 ceph-mon[35312]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T18:07:20.183 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:07:19 smithi079 ceph-mon[35312]: from='client.35291 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-28T18:07:20.281 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:20.281+0000 7ff09aa00640 1 -- 172.21.15.22:0/4249503908 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff0941048e0 msgr2=0x7ff094104d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:07:20.281 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:20.281+0000 7ff09aa00640 1 --2- 172.21.15.22:0/4249503908 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff0941048e0 0x7ff094104d60 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7ff080009900 tx=0x7ff08002bc30 comp rx=0 tx=0).stop 2024-03-28T18:07:20.281 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:20.281+0000 7ff09aa00640 1 -- 172.21.15.22:0/4249503908 shutdown_connections 2024-03-28T18:07:20.281 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:20.281+0000 7ff09aa00640 1 --2- 172.21.15.22:0/4249503908 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff0941052a0 0x7ff09410bb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:07:20.281 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:20.281+0000 7ff09aa00640 1 --2- 172.21.15.22:0/4249503908 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff0941048e0 0x7ff094104d60 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:07:20.281 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:20.281+0000 7ff09aa00640 1 --2- 172.21.15.22:0/4249503908 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ff0941036f0 0x7ff094103b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:07:20.281 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:20.281+0000 7ff09aa00640 1 -- 172.21.15.22:0/4249503908 >> 172.21.15.22:0/4249503908 conn(0x7ff0940fec80 msgr2=0x7ff0941010a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:07:20.282 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:20.282+0000 7ff09aa00640 1 -- 172.21.15.22:0/4249503908 shutdown_connections 2024-03-28T18:07:20.282 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:20.282+0000 7ff09aa00640 1 -- 172.21.15.22:0/4249503908 wait complete. 2024-03-28T18:07:20.282 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:20.283+0000 7ff09aa00640 1 Processor -- start 2024-03-28T18:07:20.282 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:20.283+0000 7ff09aa00640 1 -- start start 2024-03-28T18:07:20.283 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:20.283+0000 7ff09aa00640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ff0941036f0 0x7ff094199990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:07:20.283 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:20.283+0000 7ff09aa00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff0941048e0 0x7ff094199ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:07:20.283 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:20.283+0000 7ff09aa00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff0941052a0 0x7ff0941a0f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:07:20.283 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:20.283+0000 7ff09aa00640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff09410f350 con 0x7ff0941052a0 2024-03-28T18:07:20.283 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:20.283+0000 7ff09aa00640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff09410f1d0 con 0x7ff0941036f0 2024-03-28T18:07:20.284 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:20.283+0000 7ff09aa00640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff09410f4d0 con 0x7ff0941048e0 2024-03-28T18:07:20.284 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:20.284+0000 7ff09a000640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff0941052a0 0x7ff0941a0f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:07:20.284 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:20.284+0000 7ff09a000640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff0941052a0 0x7ff0941a0f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:55508/0 (socket says 172.21.15.22:55508) 2024-03-28T18:07:20.284 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:20.284+0000 7ff09a000640 1 -- 172.21.15.22:0/2805852466 learned_addr learned my addr 172.21.15.22:0/2805852466 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-28T18:07:20.284 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:20.284+0000 7ff099600640 1 --2- 172.21.15.22:0/2805852466 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ff0941036f0 0x7ff094199990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:07:20.284 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:20.284+0000 7ff098c00640 1 --2- 172.21.15.22:0/2805852466 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff0941048e0 0x7ff094199ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:07:20.285 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:20.284+0000 7ff09a000640 1 -- 172.21.15.22:0/2805852466 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff0941048e0 msgr2=0x7ff094199ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:07:20.285 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:20.284+0000 7ff09a000640 1 --2- 172.21.15.22:0/2805852466 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff0941048e0 0x7ff094199ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:07:20.285 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:20.284+0000 7ff09a000640 1 -- 172.21.15.22:0/2805852466 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ff0941036f0 msgr2=0x7ff094199990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:07:20.285 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:20.284+0000 7ff09a000640 1 --2- 172.21.15.22:0/2805852466 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ff0941036f0 0x7ff094199990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:07:20.285 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:20.284+0000 7ff09a000640 1 -- 172.21.15.22:0/2805852466 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff0941a15b0 con 0x7ff0941052a0 2024-03-28T18:07:20.285 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:20.285+0000 7ff09a000640 1 --2- 172.21.15.22:0/2805852466 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff0941052a0 0x7ff0941a0f40 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7ff08c00eb10 tx=0x7ff08c00c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:07:20.285 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:20.285+0000 7ff092000640 1 -- 172.21.15.22:0/2805852466 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff08c004b20 con 0x7ff0941052a0 2024-03-28T18:07:20.285 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:20.285+0000 7ff092000640 1 -- 172.21.15.22:0/2805852466 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7ff08c004cc0 con 0x7ff0941052a0 2024-03-28T18:07:20.285 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:20.285+0000 7ff092000640 1 -- 172.21.15.22:0/2805852466 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff08c010700 con 0x7ff0941052a0 2024-03-28T18:07:20.285 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:20.285+0000 7ff09aa00640 1 -- 172.21.15.22:0/2805852466 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff0941a18a0 con 0x7ff0941052a0 2024-03-28T18:07:20.285 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:20.285+0000 7ff09aa00640 1 -- 172.21.15.22:0/2805852466 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff0941a1dc0 con 0x7ff0941052a0 2024-03-28T18:07:20.286 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:20.286+0000 7ff09aa00640 1 -- 172.21.15.22:0/2805852466 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff09410aa10 con 0x7ff0941052a0 2024-03-28T18:07:20.289 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:20.289+0000 7ff092000640 1 -- 172.21.15.22:0/2805852466 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 18) v1 ==== 137290+0+0 (secure 0 0 0) 0x7ff08c004e70 con 0x7ff0941052a0 2024-03-28T18:07:20.290 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:20.290+0000 7ff092000640 1 --2- 172.21.15.22:0/2805852466 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7ff07c0a2880 0x7ff07c0a4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:07:20.290 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:20.290+0000 7ff092000640 1 -- 172.21.15.22:0/2805852466 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 8153+0+0 (secure 0 0 0) 0x7ff08c0cef80 con 0x7ff0941052a0 2024-03-28T18:07:20.290 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:20.291+0000 7ff099600640 1 --2- 172.21.15.22:0/2805852466 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7ff07c0a2880 0x7ff07c0a4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:07:20.291 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:20.291+0000 7ff099600640 1 --2- 172.21.15.22:0/2805852466 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7ff07c0a2880 0x7ff07c0a4d40 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7ff0880095f0 tx=0x7ff088005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:07:20.294 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:20.294+0000 7ff092000640 1 -- 172.21.15.22:0/2805852466 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7ff08c09c020 con 0x7ff0941052a0 2024-03-28T18:07:20.504 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:20.504+0000 7ff09aa00640 1 -- 172.21.15.22:0/2805852466 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ff0940622d0 con 0x7ff0941052a0 2024-03-28T18:07:20.506 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:20.506+0000 7ff092000640 1 -- 172.21.15.22:0/2805852466 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v92) v1 ==== 74+0+22522 (secure 0 0 0) 0x7ff08c09b920 con 0x7ff0941052a0 2024-03-28T18:07:20.506 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-03-28T18:07:20.507 INFO:teuthology.orchestra.run.smithi022.stdout:{"epoch":92,"fsid":"fd220748-ed2b-11ee-95d0-87774f69a715","created":"2024-03-28T17:54:00.440279+0000","modified":"2024-03-28T18:05:11.725341+0000","last_up_change":"2024-03-28T18:03:43.220095+0000","last_in_change":"2024-03-28T18:03:18.526399+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-28T17:58:33.168872+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-28T18:04:47.004797+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-28T18:04:48.465502+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"90","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"5d708e06-f05b-40ba-bbcf-258ddc4f43d7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6802","nonce":2186290846},{"type":"v1","addr":"172.21.15.22:6803","nonce":2186290846}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6804","nonce":2186290846},{"type":"v1","addr":"172.21.15.22:6805","nonce":2186290846}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6808","nonce":2186290846},{"type":"v1","addr":"172.21.15.22:6809","nonce":2186290846}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6806","nonce":2186290846},{"type":"v1","addr":"172.21.15.22:6807","nonce":2186290846}]},"public_addr":"172.21.15.22:6803/2186290846","cluster_addr":"172.21.15.22:6805/2186290846","heartbeat_back_addr":"172.21.15.22:6809/2186290846","heartbeat_front_addr":"172.21.15.22:6807/2186290846","state":["exists","up"]},{"osd":1,"uuid":"68ad09bb-9256-409a-9867-4e14a2cb94aa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6800","nonce":2835549681},{"type":"v1","addr":"172.21.15.79:6801","nonce":2835549681}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6802","nonce":2835549681},{"type":"v1","addr":"172.21.15.79:6803","nonce":2835549681}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6806","nonce":2835549681},{"type":"v1","addr":"172.21.15.79:6807","nonce":2835549681}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6804","nonce":2835549681},{"type":"v1","addr":"172.21.15.79:6805","nonce":2835549681}]},"public_addr":"172.21.15.79:6801/2835549681","cluster_addr":"172.21.15.79:6803/2835549681","heartbeat_back_addr":"172.21.15.79:6807/2835549681","heartbeat_front_addr":"172.21.15.79:6805/2835549681","state":["exists","up"]},{"osd":2,"uuid":"ed11a5a7-004d-41f3-a5fc-eb2d603bd33f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6800","nonce":470588085},{"type":"v1","addr":"172.21.15.134:6801","nonce":470588085}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6802","nonce":470588085},{"type":"v1","addr":"172.21.15.134:6803","nonce":470588085}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6806","nonce":470588085},{"type":"v1","addr":"172.21.15.134:6807","nonce":470588085}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6804","nonce":470588085},{"type":"v1","addr":"172.21.15.134:6805","nonce":470588085}]},"public_addr":"172.21.15.134:6801/470588085","cluster_addr":"172.21.15.134:6803/470588085","heartbeat_back_addr":"172.21.15.134:6807/470588085","heartbeat_front_addr":"172.21.15.134:6805/470588085","state":["exists","up"]},{"osd":3,"uuid":"81a6c547-2a50-44c3-84d7-23ab6831790c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6810","nonce":1702441854},{"type":"v1","addr":"172.21.15.22:6811","nonce":1702441854}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6812","nonce":1702441854},{"type":"v1","addr":"172.21.15.22:6813","nonce":1702441854}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6816","nonce":1702441854},{"type":"v1","addr":"172.21.15.22:6817","nonce":1702441854}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6814","nonce":1702441854},{"type":"v1","addr":"172.21.15.22:6815","nonce":1702441854}]},"public_addr":"172.21.15.22:6811/1702441854","cluster_addr":"172.21.15.22:6813/1702441854","heartbeat_back_addr":"172.21.15.22:6817/1702441854","heartbeat_front_addr":"172.21.15.22:6815/1702441854","state":["exists","up"]},{"osd":4,"uuid":"d23a254e-7182-4471-8bca-7eb684ec618e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6808","nonce":630132437},{"type":"v1","addr":"172.21.15.79:6809","nonce":630132437}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6810","nonce":630132437},{"type":"v1","addr":"172.21.15.79:6811","nonce":630132437}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6814","nonce":630132437},{"type":"v1","addr":"172.21.15.79:6815","nonce":630132437}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6812","nonce":630132437},{"type":"v1","addr":"172.21.15.79:6813","nonce":630132437}]},"public_addr":"172.21.15.79:6809/630132437","cluster_addr":"172.21.15.79:6811/630132437","heartbeat_back_addr":"172.21.15.79:6815/630132437","heartbeat_front_addr":"172.21.15.79:6813/630132437","state":["exists","up"]},{"osd":5,"uuid":"a7b98502-2023-4f0a-bfa0-6840cc495e0c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":40,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6808","nonce":2135406915},{"type":"v1","addr":"172.21.15.134:6809","nonce":2135406915}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6810","nonce":2135406915},{"type":"v1","addr":"172.21.15.134:6811","nonce":2135406915}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6814","nonce":2135406915},{"type":"v1","addr":"172.21.15.134:6815","nonce":2135406915}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6812","nonce":2135406915},{"type":"v1","addr":"172.21.15.134:6813","nonce":2135406915}]},"public_addr":"172.21.15.134:6809/2135406915","cluster_addr":"172.21.15.134:6811/2135406915","heartbeat_back_addr":"172.21.15.134:6815/2135406915","heartbeat_front_addr":"172.21.15.134:6813/2135406915","state":["exists","up"]},{"osd":6,"uuid":"634c6562-d683-41c8-8242-fc848c38ce1f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6818","nonce":1400680396},{"type":"v1","addr":"172.21.15.22:6819","nonce":1400680396}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6820","nonce":1400680396},{"type":"v1","addr":"172.21.15.22:6821","nonce":1400680396}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6824","nonce":1400680396},{"type":"v1","addr":"172.21.15.22:6825","nonce":1400680396}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6822","nonce":1400680396},{"type":"v1","addr":"172.21.15.22:6823","nonce":1400680396}]},"public_addr":"172.21.15.22:6819/1400680396","cluster_addr":"172.21.15.22:6821/1400680396","heartbeat_back_addr":"172.21.15.22:6825/1400680396","heartbeat_front_addr":"172.21.15.22:6823/1400680396","state":["exists","up"]},{"osd":7,"uuid":"504eb4a8-6731-45ad-932c-1b38ef157871","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":53,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6816","nonce":1287105314},{"type":"v1","addr":"172.21.15.79:6817","nonce":1287105314}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6818","nonce":1287105314},{"type":"v1","addr":"172.21.15.79:6819","nonce":1287105314}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6822","nonce":1287105314},{"type":"v1","addr":"172.21.15.79:6823","nonce":1287105314}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6820","nonce":1287105314},{"type":"v1","addr":"172.21.15.79:6821","nonce":1287105314}]},"public_addr":"172.21.15.79:6817/1287105314","cluster_addr":"172.21.15.79:6819/1287105314","heartbeat_back_addr":"172.21.15.79:6823/1287105314","heartbeat_front_addr":"172.21.15.79:6821/1287105314","state":["exists","up"]},{"osd":8,"uuid":"20aa66c1-03d6-4cb8-a096-0eae41979c17","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":62,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6816","nonce":2265650388},{"type":"v1","addr":"172.21.15.134:6817","nonce":2265650388}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6818","nonce":2265650388},{"type":"v1","addr":"172.21.15.134:6819","nonce":2265650388}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6822","nonce":2265650388},{"type":"v1","addr":"172.21.15.134:6823","nonce":2265650388}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6820","nonce":2265650388},{"type":"v1","addr":"172.21.15.134:6821","nonce":2265650388}]},"public_addr":"172.21.15.134:6817/2265650388","cluster_addr":"172.21.15.134:6819/2265650388","heartbeat_back_addr":"172.21.15.134:6823/2265650388","heartbeat_front_addr":"172.21.15.134:6821/2265650388","state":["exists","up"]},{"osd":9,"uuid":"e1ae4020-7c2c-4334-a753-ee6b12d98584","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":70,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6826","nonce":3762222259},{"type":"v1","addr":"172.21.15.22:6827","nonce":3762222259}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6828","nonce":3762222259},{"type":"v1","addr":"172.21.15.22:6829","nonce":3762222259}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6832","nonce":3762222259},{"type":"v1","addr":"172.21.15.22:6833","nonce":3762222259}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6830","nonce":3762222259},{"type":"v1","addr":"172.21.15.22:6831","nonce":3762222259}]},"public_addr":"172.21.15.22:6827/3762222259","cluster_addr":"172.21.15.22:6829/3762222259","heartbeat_back_addr":"172.21.15.22:6833/3762222259","heartbeat_front_addr":"172.21.15.22:6831/3762222259","state":["exists","up"]},{"osd":10,"uuid":"f5198d52-df48-4679-a860-4dcb440b7a9f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":77,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6824","nonce":1746167904},{"type":"v1","addr":"172.21.15.79:6825","nonce":1746167904}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6826","nonce":1746167904},{"type":"v1","addr":"172.21.15.79:6827","nonce":1746167904}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6830","nonce":1746167904},{"type":"v1","addr":"172.21.15.79:6831","nonce":1746167904}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6828","nonce":1746167904},{"type":"v1","addr":"172.21.15.79:6829","nonce":1746167904}]},"public_addr":"172.21.15.79:6825/1746167904","cluster_addr":"172.21.15.79:6827/1746167904","heartbeat_back_addr":"172.21.15.79:6831/1746167904","heartbeat_front_addr":"172.21.15.79:6829/1746167904","state":["exists","up"]},{"osd":11,"uuid":"1427bca7-3414-4468-9246-e999156bc2ae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":85,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6824","nonce":4271716428},{"type":"v1","addr":"172.21.15.134:6825","nonce":4271716428}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6826","nonce":4271716428},{"type":"v1","addr":"172.21.15.134:6827","nonce":4271716428}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6830","nonce":4271716428},{"type":"v1","addr":"172.21.15.134:6831","nonce":4271716428}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6828","nonce":4271716428},{"type":"v1","addr":"172.21.15.134:6829","nonce":4271716428}]},"public_addr":"172.21.15.134:6825/4271716428","cluster_addr":"172.21.15.134:6827/4271716428","heartbeat_back_addr":"172.21.15.134:6831/4271716428","heartbeat_front_addr":"172.21.15.134:6829/4271716428","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T17:57:21.225125+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T17:57:55.197254+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T17:58:27.825136+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T17:59:04.319893+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T17:59:38.407752+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T18:00:12.118805+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T18:00:47.635214+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T18:01:21.231055+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T18:01:55.817861+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T18:02:31.680160+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T18:03:06.783398+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T18:03:41.307347+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.8","mappings":[{"from":2,"to":11}]},{"pgid":"2.1d","mappings":[{"from":8,"to":11}]},{"pgid":"3.3","mappings":[{"from":4,"to":9}]},{"pgid":"3.c","mappings":[{"from":3,"to":9}]},{"pgid":"3.21","mappings":[{"from":4,"to":8}]},{"pgid":"3.25","mappings":[{"from":5,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.22:0/3521924283":"2024-03-29T17:55:10.425504+0000","172.21.15.22:6801/730635939":"2024-03-29T17:55:10.425504+0000","172.21.15.22:0/2471229459":"2024-03-29T17:54:34.004559+0000","172.21.15.22:6800/3166828081":"2024-03-29T17:54:34.004559+0000","172.21.15.22:6800/730635939":"2024-03-29T17:55:10.425504+0000","172.21.15.22:0/1229047849":"2024-03-29T17:54:34.004559+0000","172.21.15.22:6801/3166828081":"2024-03-29T17:54:34.004559+0000","172.21.15.22:0/2717818856":"2024-03-29T17:55:10.425504+0000","172.21.15.22:0/2157570333":"2024-03-29T17:54:34.004559+0000","172.21.15.22:0/1006589530":"2024-03-29T17:55:10.425504+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-28T18:07:20.509 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:20.509+0000 7ff09aa00640 1 -- 172.21.15.22:0/2805852466 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7ff07c0a2880 msgr2=0x7ff07c0a4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:07:20.509 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:20.509+0000 7ff09aa00640 1 --2- 172.21.15.22:0/2805852466 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7ff07c0a2880 0x7ff07c0a4d40 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7ff0880095f0 tx=0x7ff088005f90 comp rx=0 tx=0).stop 2024-03-28T18:07:20.509 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:20.509+0000 7ff09aa00640 1 -- 172.21.15.22:0/2805852466 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff0941052a0 msgr2=0x7ff0941a0f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:07:20.509 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:20.509+0000 7ff09aa00640 1 --2- 172.21.15.22:0/2805852466 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff0941052a0 0x7ff0941a0f40 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7ff08c00eb10 tx=0x7ff08c00c2d0 comp rx=0 tx=0).stop 2024-03-28T18:07:20.509 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:20.509+0000 7ff09aa00640 1 -- 172.21.15.22:0/2805852466 shutdown_connections 2024-03-28T18:07:20.509 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:20.509+0000 7ff09aa00640 1 --2- 172.21.15.22:0/2805852466 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7ff07c0a2880 0x7ff07c0a4d40 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:07:20.509 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:20.509+0000 7ff09aa00640 1 --2- 172.21.15.22:0/2805852466 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff0941052a0 0x7ff0941a0f40 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:07:20.509 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:20.509+0000 7ff09aa00640 1 --2- 172.21.15.22:0/2805852466 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff0941048e0 0x7ff094199ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:07:20.509 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:20.509+0000 7ff09aa00640 1 --2- 172.21.15.22:0/2805852466 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ff0941036f0 0x7ff094199990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:07:20.510 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:20.509+0000 7ff09aa00640 1 -- 172.21.15.22:0/2805852466 >> 172.21.15.22:0/2805852466 conn(0x7ff0940fec80 msgr2=0x7ff094107be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:07:20.510 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:20.510+0000 7ff09aa00640 1 -- 172.21.15.22:0/2805852466 shutdown_connections 2024-03-28T18:07:20.510 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:20.510+0000 7ff09aa00640 1 -- 172.21.15.22:0/2805852466 wait complete. 2024-03-28T18:07:20.529 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-28T17:58:33.168872+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-28T18:04:47.004797+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '89', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-03-28T18:04:48.465502+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '90', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-28T18:07:20.529 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-28T18:07:20.735 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:20.735+0000 7f9846c00640 1 -- 172.21.15.22:0/2484134420 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9840102000 msgr2=0x7f98401043f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:07:20.735 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:20.735+0000 7f9846c00640 1 --2- 172.21.15.22:0/2484134420 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9840102000 0x7f98401043f0 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f9834009ca0 tx=0x7f983402c0f0 comp rx=0 tx=0).stop 2024-03-28T18:07:20.735 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:20.735+0000 7f9846c00640 1 -- 172.21.15.22:0/2484134420 shutdown_connections 2024-03-28T18:07:20.735 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:20.735+0000 7f9846c00640 1 --2- 172.21.15.22:0/2484134420 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f9840107260 0x7f98401096b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:07:20.735 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:20.735+0000 7f9846c00640 1 --2- 172.21.15.22:0/2484134420 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9840104930 0x7f9840106d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:07:20.735 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:20.735+0000 7f9846c00640 1 --2- 172.21.15.22:0/2484134420 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9840102000 0x7f98401043f0 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:07:20.736 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:20.735+0000 7f9846c00640 1 -- 172.21.15.22:0/2484134420 >> 172.21.15.22:0/2484134420 conn(0x7f98400fba10 msgr2=0x7f98400fde30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:07:20.736 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:20.735+0000 7f9846c00640 1 -- 172.21.15.22:0/2484134420 shutdown_connections 2024-03-28T18:07:20.736 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:20.735+0000 7f9846c00640 1 -- 172.21.15.22:0/2484134420 wait complete. 2024-03-28T18:07:20.737 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:20.736+0000 7f9846c00640 1 Processor -- start 2024-03-28T18:07:20.737 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:20.736+0000 7f9846c00640 1 -- start start 2024-03-28T18:07:20.737 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:20.737+0000 7f9846c00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9840102000 0x7f9840199ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:07:20.737 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:20.737+0000 7f9846c00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9840104930 0x7f984019a010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:07:20.737 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:20.737+0000 7f9846c00640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f9840107260 0x7f98401a1080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:07:20.737 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:20.737+0000 7f9846c00640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f984010d200 con 0x7f9840104930 2024-03-28T18:07:20.737 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:20.737+0000 7f9846c00640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f984010d080 con 0x7f9840107260 2024-03-28T18:07:20.738 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:20.737+0000 7f9846c00640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f984010d380 con 0x7f9840102000 2024-03-28T18:07:20.738 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:20.737+0000 7f9844e00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9840104930 0x7f984019a010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:07:20.738 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:20.737+0000 7f9844e00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9840104930 0x7f984019a010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:55520/0 (socket says 172.21.15.22:55520) 2024-03-28T18:07:20.738 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:20.737+0000 7f9844e00640 1 -- 172.21.15.22:0/1096579762 learned_addr learned my addr 172.21.15.22:0/1096579762 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-28T18:07:20.738 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:20.737+0000 7f9845800640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9840102000 0x7f9840199ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:07:20.738 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:20.737+0000 7f9846200640 1 --2- 172.21.15.22:0/1096579762 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f9840107260 0x7f98401a1080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:07:20.738 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:20.738+0000 7f9844e00640 1 -- 172.21.15.22:0/1096579762 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9840102000 msgr2=0x7f9840199ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:07:20.738 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:20.738+0000 7f9844e00640 1 --2- 172.21.15.22:0/1096579762 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9840102000 0x7f9840199ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:07:20.738 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:20.738+0000 7f9844e00640 1 -- 172.21.15.22:0/1096579762 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f9840107260 msgr2=0x7f98401a1080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:07:20.738 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:20.738+0000 7f9844e00640 1 --2- 172.21.15.22:0/1096579762 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f9840107260 0x7f98401a1080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:07:20.738 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:20.738+0000 7f9844e00640 1 -- 172.21.15.22:0/1096579762 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f98401a16f0 con 0x7f9840104930 2024-03-28T18:07:20.738 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:20.738+0000 7f9846200640 1 --2- 172.21.15.22:0/1096579762 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f9840107260 0x7f98401a1080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-28T18:07:20.739 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:20.738+0000 7f9844e00640 1 --2- 172.21.15.22:0/1096579762 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9840104930 0x7f984019a010 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f982c00e9f0 tx=0x7f982c00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:07:20.739 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:20.738+0000 7f983e000640 1 -- 172.21.15.22:0/1096579762 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f982c00ccc0 con 0x7f9840104930 2024-03-28T18:07:20.739 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:20.738+0000 7f983e000640 1 -- 172.21.15.22:0/1096579762 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f982c0054f0 con 0x7f9840104930 2024-03-28T18:07:20.739 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:20.738+0000 7f9846c00640 1 -- 172.21.15.22:0/1096579762 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f98401a19e0 con 0x7f9840104930 2024-03-28T18:07:20.739 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:20.738+0000 7f983e000640 1 -- 172.21.15.22:0/1096579762 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f982c010690 con 0x7f9840104930 2024-03-28T18:07:20.739 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:20.738+0000 7f9846c00640 1 -- 172.21.15.22:0/1096579762 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f98401a1ee0 con 0x7f9840104930 2024-03-28T18:07:20.740 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:20.740+0000 7f9846c00640 1 -- 172.21.15.22:0/1096579762 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9840108db0 con 0x7f9840104930 2024-03-28T18:07:20.741 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:20.742+0000 7f983e000640 1 -- 172.21.15.22:0/1096579762 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 18) v1 ==== 137290+0+0 (secure 0 0 0) 0x7f982c004e80 con 0x7f9840104930 2024-03-28T18:07:20.742 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:20.743+0000 7f983e000640 1 --2- 172.21.15.22:0/1096579762 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7f98280a2800 0x7f98280a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:07:20.743 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:20.743+0000 7f983e000640 1 -- 172.21.15.22:0/1096579762 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 8153+0+0 (secure 0 0 0) 0x7f982c0cee60 con 0x7f9840104930 2024-03-28T18:07:20.743 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:20.743+0000 7f9845800640 1 --2- 172.21.15.22:0/1096579762 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7f98280a2800 0x7f98280a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:07:20.743 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:20.744+0000 7f9845800640 1 --2- 172.21.15.22:0/1096579762 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7f98280a2800 0x7f98280a4cc0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f9834006010 tx=0x7f98340325d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:07:20.747 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:20.748+0000 7f983e000640 1 -- 172.21.15.22:0/1096579762 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f982c09bf00 con 0x7f9840104930 2024-03-28T18:07:20.781 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:07:20 smithi022 ceph-mon[31985]: from='client.15438 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-28T18:07:20.781 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:07:20 smithi022 ceph-mon[31985]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T18:07:20.781 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:07:20 smithi022 ceph-mon[31985]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T18:07:20.781 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:07:20 smithi022 ceph-mon[31985]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T18:07:20.781 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:07:20 smithi022 ceph-mon[31985]: from='client.? 172.21.15.22:0/2805852466' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-28T18:07:20.954 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:20.955+0000 7f9846c00640 1 -- 172.21.15.22:0/1096579762 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f98400622a0 con 0x7f9840104930 2024-03-28T18:07:20.955 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:20.955+0000 7f983e000640 1 -- 172.21.15.22:0/1096579762 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v92) v1 ==== 93+0+10 (secure 0 0 0) 0x7f982c09b800 con 0x7f9840104930 2024-03-28T18:07:20.955 INFO:teuthology.orchestra.run.smithi022.stdout:pg_num: 1 2024-03-28T18:07:20.958 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:20.958+0000 7f9846c00640 1 -- 172.21.15.22:0/1096579762 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7f98280a2800 msgr2=0x7f98280a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:07:20.958 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:20.958+0000 7f9846c00640 1 --2- 172.21.15.22:0/1096579762 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7f98280a2800 0x7f98280a4cc0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f9834006010 tx=0x7f98340325d0 comp rx=0 tx=0).stop 2024-03-28T18:07:20.958 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:20.958+0000 7f9846c00640 1 -- 172.21.15.22:0/1096579762 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9840104930 msgr2=0x7f984019a010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:07:20.958 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:20.958+0000 7f9846c00640 1 --2- 172.21.15.22:0/1096579762 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9840104930 0x7f984019a010 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f982c00e9f0 tx=0x7f982c00eeb0 comp rx=0 tx=0).stop 2024-03-28T18:07:20.958 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:20.958+0000 7f9846c00640 1 -- 172.21.15.22:0/1096579762 shutdown_connections 2024-03-28T18:07:20.958 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:20.958+0000 7f9846c00640 1 --2- 172.21.15.22:0/1096579762 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7f98280a2800 0x7f98280a4cc0 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:07:20.958 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:20.958+0000 7f9846c00640 1 --2- 172.21.15.22:0/1096579762 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f9840107260 0x7f98401a1080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:07:20.958 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:20.958+0000 7f9846c00640 1 --2- 172.21.15.22:0/1096579762 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9840104930 0x7f984019a010 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:07:20.958 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:20.958+0000 7f9846c00640 1 --2- 172.21.15.22:0/1096579762 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9840102000 0x7f9840199ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:07:20.958 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:20.958+0000 7f9846c00640 1 -- 172.21.15.22:0/1096579762 >> 172.21.15.22:0/1096579762 conn(0x7f98400fba10 msgr2=0x7f9840101710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:07:20.958 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:20.958+0000 7f9846c00640 1 -- 172.21.15.22:0/1096579762 shutdown_connections 2024-03-28T18:07:20.959 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:20.959+0000 7f9846c00640 1 -- 172.21.15.22:0/1096579762 wait complete. 2024-03-28T18:07:20.977 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-28T18:07:21.175 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:07:20 smithi134 ceph-mon[35037]: from='client.15438 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-28T18:07:21.175 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:07:20 smithi134 ceph-mon[35037]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T18:07:21.175 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:07:20 smithi134 ceph-mon[35037]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T18:07:21.175 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:07:20 smithi134 ceph-mon[35037]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T18:07:21.175 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:07:20 smithi134 ceph-mon[35037]: from='client.? 172.21.15.22:0/2805852466' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-28T18:07:21.182 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:07:20 smithi079 ceph-mon[35312]: from='client.15438 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-28T18:07:21.182 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:07:20 smithi079 ceph-mon[35312]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T18:07:21.183 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:07:20 smithi079 ceph-mon[35312]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T18:07:21.183 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:07:20 smithi079 ceph-mon[35312]: from='mgr.14152 172.21.15.22:0/879975530' entity='mgr.x' 2024-03-28T18:07:21.183 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:07:20 smithi079 ceph-mon[35312]: from='client.? 172.21.15.22:0/2805852466' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-28T18:07:21.199 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:21.200+0000 7fd731a00640 1 -- 172.21.15.22:0/149849338 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd72c1049b0 msgr2=0x7fd72c104e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:07:21.200 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:21.200+0000 7fd731a00640 1 --2- 172.21.15.22:0/149849338 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd72c1049b0 0x7fd72c104e30 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fd718009900 tx=0x7fd71802bc30 comp rx=0 tx=0).stop 2024-03-28T18:07:21.200 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:21.200+0000 7fd731a00640 1 -- 172.21.15.22:0/149849338 shutdown_connections 2024-03-28T18:07:21.200 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:21.200+0000 7fd731a00640 1 --2- 172.21.15.22:0/149849338 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd72c105370 0x7fd72c10bc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:07:21.200 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:21.200+0000 7fd731a00640 1 --2- 172.21.15.22:0/149849338 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd72c1049b0 0x7fd72c104e30 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:07:21.200 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:21.200+0000 7fd731a00640 1 --2- 172.21.15.22:0/149849338 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd72c1037c0 0x7fd72c103be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:07:21.200 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:21.200+0000 7fd731a00640 1 -- 172.21.15.22:0/149849338 >> 172.21.15.22:0/149849338 conn(0x7fd72c0fed50 msgr2=0x7fd72c101170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:07:21.200 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:21.200+0000 7fd731a00640 1 -- 172.21.15.22:0/149849338 shutdown_connections 2024-03-28T18:07:21.200 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:21.200+0000 7fd731a00640 1 -- 172.21.15.22:0/149849338 wait complete. 2024-03-28T18:07:21.201 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:21.201+0000 7fd731a00640 1 Processor -- start 2024-03-28T18:07:21.201 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:21.201+0000 7fd731a00640 1 -- start start 2024-03-28T18:07:21.202 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:21.202+0000 7fd731a00640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd72c1037c0 0x7fd72c199a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:07:21.202 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:21.202+0000 7fd731a00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd72c1049b0 0x7fd72c199fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:07:21.202 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:21.202+0000 7fd731a00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd72c105370 0x7fd72c1a1040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:07:21.203 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:21.202+0000 7fd731a00640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd72c10f470 con 0x7fd72c105370 2024-03-28T18:07:21.203 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:21.202+0000 7fd731a00640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd72c10f2f0 con 0x7fd72c1037c0 2024-03-28T18:07:21.203 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:21.202+0000 7fd731a00640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd72c10f5f0 con 0x7fd72c1049b0 2024-03-28T18:07:21.203 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:21.202+0000 7fd731000640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd72c105370 0x7fd72c1a1040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:07:21.203 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:21.202+0000 7fd731000640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd72c105370 0x7fd72c1a1040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:55538/0 (socket says 172.21.15.22:55538) 2024-03-28T18:07:21.203 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:21.202+0000 7fd731000640 1 -- 172.21.15.22:0/3359946719 learned_addr learned my addr 172.21.15.22:0/3359946719 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-28T18:07:21.203 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:21.202+0000 7fd72be00640 1 --2- 172.21.15.22:0/3359946719 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd72c1037c0 0x7fd72c199a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:07:21.203 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:21.203+0000 7fd72b400640 1 --2- 172.21.15.22:0/3359946719 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd72c1049b0 0x7fd72c199fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:07:21.203 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:21.203+0000 7fd731000640 1 -- 172.21.15.22:0/3359946719 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd72c1049b0 msgr2=0x7fd72c199fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:07:21.203 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:21.203+0000 7fd731000640 1 --2- 172.21.15.22:0/3359946719 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd72c1049b0 0x7fd72c199fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:07:21.203 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:21.203+0000 7fd731000640 1 -- 172.21.15.22:0/3359946719 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd72c1037c0 msgr2=0x7fd72c199a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:07:21.203 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:21.203+0000 7fd731000640 1 --2- 172.21.15.22:0/3359946719 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd72c1037c0 0x7fd72c199a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:07:21.203 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:21.203+0000 7fd731000640 1 -- 172.21.15.22:0/3359946719 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd72c1a16b0 con 0x7fd72c105370 2024-03-28T18:07:21.203 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:21.203+0000 7fd731000640 1 --2- 172.21.15.22:0/3359946719 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd72c105370 0x7fd72c1a1040 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7fd72400eb10 tx=0x7fd72400c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:07:21.204 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:21.203+0000 7fd728c00640 1 -- 172.21.15.22:0/3359946719 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd724004b20 con 0x7fd72c105370 2024-03-28T18:07:21.204 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:21.203+0000 7fd728c00640 1 -- 172.21.15.22:0/3359946719 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7fd724004cc0 con 0x7fd72c105370 2024-03-28T18:07:21.204 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:21.203+0000 7fd731a00640 1 -- 172.21.15.22:0/3359946719 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd72c1a19a0 con 0x7fd72c105370 2024-03-28T18:07:21.204 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:21.203+0000 7fd728c00640 1 -- 172.21.15.22:0/3359946719 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7240106c0 con 0x7fd72c105370 2024-03-28T18:07:21.204 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:21.204+0000 7fd731a00640 1 -- 172.21.15.22:0/3359946719 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd72c1a1e70 con 0x7fd72c105370 2024-03-28T18:07:21.205 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:21.205+0000 7fd731a00640 1 -- 172.21.15.22:0/3359946719 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd72c068800 con 0x7fd72c105370 2024-03-28T18:07:21.206 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:21.206+0000 7fd728c00640 1 -- 172.21.15.22:0/3359946719 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 18) v1 ==== 137290+0+0 (secure 0 0 0) 0x7fd724004e70 con 0x7fd72c105370 2024-03-28T18:07:21.207 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:21.208+0000 7fd728c00640 1 --2- 172.21.15.22:0/3359946719 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7fd7100a2800 0x7fd7100a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:07:21.208 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:21.208+0000 7fd728c00640 1 -- 172.21.15.22:0/3359946719 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 8153+0+0 (secure 0 0 0) 0x7fd7240cf1b0 con 0x7fd72c105370 2024-03-28T18:07:21.208 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:21.208+0000 7fd72be00640 1 --2- 172.21.15.22:0/3359946719 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7fd7100a2800 0x7fd7100a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:07:21.209 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:21.209+0000 7fd72be00640 1 --2- 172.21.15.22:0/3359946719 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7fd7100a2800 0x7fd7100a4cc0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7fd72c104810 tx=0x7fd71c00b390 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:07:21.211 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:21.212+0000 7fd728c00640 1 -- 172.21.15.22:0/3359946719 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fd72409c250 con 0x7fd72c105370 2024-03-28T18:07:21.416 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:21.416+0000 7fd731a00640 1 -- 172.21.15.22:0/3359946719 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fd72c02d070 con 0x7fd72c105370 2024-03-28T18:07:21.417 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:21.417+0000 7fd728c00640 1 -- 172.21.15.22:0/3359946719 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v92) v1 ==== 104+0+11 (secure 0 0 0) 0x7fd72409bb50 con 0x7fd72c105370 2024-03-28T18:07:21.417 INFO:teuthology.orchestra.run.smithi022.stdout:pg_num: 32 2024-03-28T18:07:21.420 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:21.420+0000 7fd731a00640 1 -- 172.21.15.22:0/3359946719 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7fd7100a2800 msgr2=0x7fd7100a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:07:21.420 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:21.420+0000 7fd731a00640 1 --2- 172.21.15.22:0/3359946719 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7fd7100a2800 0x7fd7100a4cc0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7fd72c104810 tx=0x7fd71c00b390 comp rx=0 tx=0).stop 2024-03-28T18:07:21.420 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:21.420+0000 7fd731a00640 1 -- 172.21.15.22:0/3359946719 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd72c105370 msgr2=0x7fd72c1a1040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:07:21.420 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:21.420+0000 7fd731a00640 1 --2- 172.21.15.22:0/3359946719 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd72c105370 0x7fd72c1a1040 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7fd72400eb10 tx=0x7fd72400c2d0 comp rx=0 tx=0).stop 2024-03-28T18:07:21.420 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:21.420+0000 7fd731a00640 1 -- 172.21.15.22:0/3359946719 shutdown_connections 2024-03-28T18:07:21.420 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:21.420+0000 7fd731a00640 1 --2- 172.21.15.22:0/3359946719 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7fd7100a2800 0x7fd7100a4cc0 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:07:21.421 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:21.420+0000 7fd731a00640 1 --2- 172.21.15.22:0/3359946719 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd72c105370 0x7fd72c1a1040 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:07:21.421 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:21.420+0000 7fd731a00640 1 --2- 172.21.15.22:0/3359946719 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd72c1049b0 0x7fd72c199fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:07:21.421 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:21.420+0000 7fd731a00640 1 --2- 172.21.15.22:0/3359946719 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd72c1037c0 0x7fd72c199a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:07:21.421 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:21.420+0000 7fd731a00640 1 -- 172.21.15.22:0/3359946719 >> 172.21.15.22:0/3359946719 conn(0x7fd72c0fed50 msgr2=0x7fd72c107d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:07:21.421 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:21.421+0000 7fd731a00640 1 -- 172.21.15.22:0/3359946719 shutdown_connections 2024-03-28T18:07:21.421 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:21.421+0000 7fd731a00640 1 -- 172.21.15.22:0/3359946719 wait complete. 2024-03-28T18:07:21.440 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-28T18:07:21.674 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:21.674+0000 7f0761600640 1 -- 172.21.15.22:0/2150899631 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f075c1028a0 msgr2=0x7f075c102d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:07:21.674 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:21.674+0000 7f0761600640 1 --2- 172.21.15.22:0/2150899631 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f075c1028a0 0x7f075c102d20 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f0744009900 tx=0x7f074402bc40 comp rx=0 tx=0).stop 2024-03-28T18:07:21.674 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:21.674+0000 7f0761600640 1 -- 172.21.15.22:0/2150899631 shutdown_connections 2024-03-28T18:07:21.674 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:21.674+0000 7f0761600640 1 --2- 172.21.15.22:0/2150899631 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f075c103260 0x7f075c109af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:07:21.674 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:21.674+0000 7f0761600640 1 --2- 172.21.15.22:0/2150899631 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f075c1028a0 0x7f075c102d20 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:07:21.674 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:21.674+0000 7f0761600640 1 --2- 172.21.15.22:0/2150899631 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f075c1016b0 0x7f075c101ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:07:21.675 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:21.674+0000 7f0761600640 1 -- 172.21.15.22:0/2150899631 >> 172.21.15.22:0/2150899631 conn(0x7f075c0fcc40 msgr2=0x7f075c0ff060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:07:21.675 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:21.674+0000 7f0761600640 1 -- 172.21.15.22:0/2150899631 shutdown_connections 2024-03-28T18:07:21.675 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:21.674+0000 7f0761600640 1 -- 172.21.15.22:0/2150899631 wait complete. 2024-03-28T18:07:21.676 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:21.675+0000 7f0761600640 1 Processor -- start 2024-03-28T18:07:21.676 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:21.675+0000 7f0761600640 1 -- start start 2024-03-28T18:07:21.676 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:21.676+0000 7f0761600640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f075c1016b0 0x7f075c199c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:07:21.676 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:21.676+0000 7f0761600640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f075c1028a0 0x7f075c19a180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:07:21.676 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:21.676+0000 7f0761600640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f075c103260 0x7f075c1a11b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:07:21.676 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:21.676+0000 7f0761600640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f075c10d360 con 0x7f075c103260 2024-03-28T18:07:21.676 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:21.676+0000 7f0761600640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f075c10d1e0 con 0x7f075c1028a0 2024-03-28T18:07:21.676 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:21.676+0000 7f0761600640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f075c10d4e0 con 0x7f075c1016b0 2024-03-28T18:07:21.676 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:21.676+0000 7f075b400640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f075c103260 0x7f075c1a11b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:07:21.676 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:21.676+0000 7f075b400640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f075c103260 0x7f075c1a11b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:55566/0 (socket says 172.21.15.22:55566) 2024-03-28T18:07:21.676 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:21.676+0000 7f075b400640 1 -- 172.21.15.22:0/2388986592 learned_addr learned my addr 172.21.15.22:0/2388986592 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-28T18:07:21.676 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:21.676+0000 7f0752000640 1 --2- 172.21.15.22:0/2388986592 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f075c1028a0 0x7f075c19a180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:07:21.677 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:21.676+0000 7f075aa00640 1 --2- 172.21.15.22:0/2388986592 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f075c1016b0 0x7f075c199c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:07:21.677 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:21.677+0000 7f075b400640 1 -- 172.21.15.22:0/2388986592 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f075c1016b0 msgr2=0x7f075c199c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:07:21.678 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:21.677+0000 7f075b400640 1 --2- 172.21.15.22:0/2388986592 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f075c1016b0 0x7f075c199c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:07:21.678 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:21.677+0000 7f075b400640 1 -- 172.21.15.22:0/2388986592 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f075c1028a0 msgr2=0x7f075c19a180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:07:21.678 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:21.677+0000 7f075b400640 1 --2- 172.21.15.22:0/2388986592 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f075c1028a0 0x7f075c19a180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:07:21.678 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:21.677+0000 7f075b400640 1 -- 172.21.15.22:0/2388986592 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f075c1a1850 con 0x7f075c103260 2024-03-28T18:07:21.678 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:21.677+0000 7f075b400640 1 --2- 172.21.15.22:0/2388986592 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f075c103260 0x7f075c1a11b0 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f075400eaa0 tx=0x7f075400ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:07:21.678 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:21.677+0000 7f0753e00640 1 -- 172.21.15.22:0/2388986592 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f075400cdb0 con 0x7f075c103260 2024-03-28T18:07:21.678 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:21.677+0000 7f0753e00640 1 -- 172.21.15.22:0/2388986592 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f0754005020 con 0x7f075c103260 2024-03-28T18:07:21.678 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:21.677+0000 7f0761600640 1 -- 172.21.15.22:0/2388986592 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f075c1a1b40 con 0x7f075c103260 2024-03-28T18:07:21.678 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:21.677+0000 7f0753e00640 1 -- 172.21.15.22:0/2388986592 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07540116b0 con 0x7f075c103260 2024-03-28T18:07:21.678 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:21.678+0000 7f0761600640 1 -- 172.21.15.22:0/2388986592 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f075c1a2040 con 0x7f075c103260 2024-03-28T18:07:21.679 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:21.679+0000 7f0761600640 1 -- 172.21.15.22:0/2388986592 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f075c1a2210 con 0x7f075c103260 2024-03-28T18:07:21.680 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:21.681+0000 7f0753e00640 1 -- 172.21.15.22:0/2388986592 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 18) v1 ==== 137290+0+0 (secure 0 0 0) 0x7f07540051d0 con 0x7f075c103260 2024-03-28T18:07:21.681 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:21.682+0000 7f0753e00640 1 --2- 172.21.15.22:0/2388986592 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7f07400a2800 0x7f07400a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:07:21.682 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:21.682+0000 7f0753e00640 1 -- 172.21.15.22:0/2388986592 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 8153+0+0 (secure 0 0 0) 0x7f07540ced20 con 0x7f075c103260 2024-03-28T18:07:21.682 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:21.682+0000 7f075aa00640 1 --2- 172.21.15.22:0/2388986592 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7f07400a2800 0x7f07400a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:07:21.682 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:21.683+0000 7f075aa00640 1 --2- 172.21.15.22:0/2388986592 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7f07400a2800 0x7f07400a4cc0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f074c005fd0 tx=0x7f074c005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:07:21.686 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:21.686+0000 7f0753e00640 1 -- 172.21.15.22:0/2388986592 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f0754016030 con 0x7f075c103260 2024-03-28T18:07:21.892 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:21.892+0000 7f0761600640 1 -- 172.21.15.22:0/2388986592 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f075c1a23e0 con 0x7f075c103260 2024-03-28T18:07:21.893 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:21.893+0000 7f0753e00640 1 -- 172.21.15.22:0/2388986592 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v92) v1 ==== 100+0+11 (secure 0 0 0) 0x7f075409bdc0 con 0x7f075c103260 2024-03-28T18:07:21.893 INFO:teuthology.orchestra.run.smithi022.stdout:pg_num: 64 2024-03-28T18:07:21.896 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:21.896+0000 7f0761600640 1 -- 172.21.15.22:0/2388986592 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7f07400a2800 msgr2=0x7f07400a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:07:21.896 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:21.896+0000 7f0761600640 1 --2- 172.21.15.22:0/2388986592 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7f07400a2800 0x7f07400a4cc0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f074c005fd0 tx=0x7f074c005850 comp rx=0 tx=0).stop 2024-03-28T18:07:21.896 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:21.896+0000 7f0761600640 1 -- 172.21.15.22:0/2388986592 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f075c103260 msgr2=0x7f075c1a11b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:07:21.896 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:21.896+0000 7f0761600640 1 --2- 172.21.15.22:0/2388986592 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f075c103260 0x7f075c1a11b0 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f075400eaa0 tx=0x7f075400ef60 comp rx=0 tx=0).stop 2024-03-28T18:07:21.896 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:21.896+0000 7f0761600640 1 -- 172.21.15.22:0/2388986592 shutdown_connections 2024-03-28T18:07:21.896 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:21.896+0000 7f0761600640 1 --2- 172.21.15.22:0/2388986592 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7f07400a2800 0x7f07400a4cc0 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:07:21.896 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:21.896+0000 7f0761600640 1 --2- 172.21.15.22:0/2388986592 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f075c103260 0x7f075c1a11b0 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:07:21.896 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:21.896+0000 7f0761600640 1 --2- 172.21.15.22:0/2388986592 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f075c1028a0 0x7f075c19a180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:07:21.897 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:21.896+0000 7f0761600640 1 --2- 172.21.15.22:0/2388986592 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f075c1016b0 0x7f075c199c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:07:21.897 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:21.896+0000 7f0761600640 1 -- 172.21.15.22:0/2388986592 >> 172.21.15.22:0/2388986592 conn(0x7f075c0fcc40 msgr2=0x7f075c0738d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:07:21.897 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:21.896+0000 7f0761600640 1 -- 172.21.15.22:0/2388986592 shutdown_connections 2024-03-28T18:07:21.897 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:21.896+0000 7f0761600640 1 -- 172.21.15.22:0/2388986592 wait complete. 2024-03-28T18:07:21.915 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-28T18:07:22.000 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:07:21 smithi022 ceph-mon[31985]: pgmap v447: 97 pgs: 97 active+clean; 649 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 1.7 KiB/s wr, 12 op/s 2024-03-28T18:07:22.001 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:07:21 smithi022 ceph-mon[31985]: from='client.? 172.21.15.22:0/1096579762' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-28T18:07:22.001 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:07:21 smithi022 ceph-mon[31985]: from='client.? 172.21.15.22:0/3359946719' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-28T18:07:22.146 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:22.140+0000 7fdc73200640 1 -- 172.21.15.22:0/666965461 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fdc74101f70 msgr2=0x7fdc74104360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:07:22.146 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:22.140+0000 7fdc73200640 1 --2- 172.21.15.22:0/666965461 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fdc74101f70 0x7fdc74104360 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7fdc64009900 tx=0x7fdc6402bc30 comp rx=0 tx=0).stop 2024-03-28T18:07:22.146 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:22.140+0000 7fdc73200640 1 -- 172.21.15.22:0/666965461 shutdown_connections 2024-03-28T18:07:22.146 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:22.140+0000 7fdc73200640 1 --2- 172.21.15.22:0/666965461 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fdc741071d0 0x7fdc74109620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:07:22.146 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:22.140+0000 7fdc73200640 1 --2- 172.21.15.22:0/666965461 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdc741048a0 0x7fdc74106c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:07:22.147 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:22.140+0000 7fdc73200640 1 --2- 172.21.15.22:0/666965461 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fdc74101f70 0x7fdc74104360 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:07:22.147 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:22.140+0000 7fdc73200640 1 -- 172.21.15.22:0/666965461 >> 172.21.15.22:0/666965461 conn(0x7fdc740fb980 msgr2=0x7fdc740fdda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:07:22.147 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:22.140+0000 7fdc73200640 1 -- 172.21.15.22:0/666965461 shutdown_connections 2024-03-28T18:07:22.147 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:22.140+0000 7fdc73200640 1 -- 172.21.15.22:0/666965461 wait complete. 2024-03-28T18:07:22.147 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:22.141+0000 7fdc73200640 1 Processor -- start 2024-03-28T18:07:22.147 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:22.141+0000 7fdc73200640 1 -- start start 2024-03-28T18:07:22.147 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:22.141+0000 7fdc73200640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fdc74101f70 0x7fdc74199a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:07:22.147 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:22.141+0000 7fdc73200640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdc741048a0 0x7fdc74199f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:07:22.147 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:22.142+0000 7fdc73200640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fdc741071d0 0x7fdc741a0fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:07:22.148 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:22.142+0000 7fdc73200640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc7410d1c0 con 0x7fdc74101f70 2024-03-28T18:07:22.148 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:22.142+0000 7fdc73200640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc7410d040 con 0x7fdc741071d0 2024-03-28T18:07:22.148 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:22.142+0000 7fdc73200640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc7410d340 con 0x7fdc741048a0 2024-03-28T18:07:22.148 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:22.142+0000 7fdc71e00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fdc74101f70 0x7fdc74199a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:07:22.148 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:22.142+0000 7fdc71e00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fdc74101f70 0x7fdc74199a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:55582/0 (socket says 172.21.15.22:55582) 2024-03-28T18:07:22.148 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:22.142+0000 7fdc71e00640 1 -- 172.21.15.22:0/2735544693 learned_addr learned my addr 172.21.15.22:0/2735544693 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-28T18:07:22.148 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:22.142+0000 7fdc72800640 1 --2- 172.21.15.22:0/2735544693 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fdc741071d0 0x7fdc741a0fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:07:22.148 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:22.142+0000 7fdc69400640 1 --2- 172.21.15.22:0/2735544693 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdc741048a0 0x7fdc74199f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:07:22.148 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:22.142+0000 7fdc71e00640 1 -- 172.21.15.22:0/2735544693 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdc741048a0 msgr2=0x7fdc74199f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:07:22.149 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:22.142+0000 7fdc71e00640 1 --2- 172.21.15.22:0/2735544693 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdc741048a0 0x7fdc74199f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:07:22.149 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:22.142+0000 7fdc71e00640 1 -- 172.21.15.22:0/2735544693 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fdc741071d0 msgr2=0x7fdc741a0fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:07:22.149 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:22.142+0000 7fdc71e00640 1 --2- 172.21.15.22:0/2735544693 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fdc741071d0 0x7fdc741a0fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:07:22.149 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:22.142+0000 7fdc71e00640 1 -- 172.21.15.22:0/2735544693 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdc741a1630 con 0x7fdc74101f70 2024-03-28T18:07:22.149 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:22.143+0000 7fdc71e00640 1 --2- 172.21.15.22:0/2735544693 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fdc74101f70 0x7fdc74199a10 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7fdc64000c00 tx=0x7fdc64004e20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:07:22.149 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:22.143+0000 7fdc6b400640 1 -- 172.21.15.22:0/2735544693 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc64043020 con 0x7fdc74101f70 2024-03-28T18:07:22.149 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:22.143+0000 7fdc6b400640 1 -- 172.21.15.22:0/2735544693 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7fdc64005460 con 0x7fdc74101f70 2024-03-28T18:07:22.149 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:22.143+0000 7fdc6b400640 1 -- 172.21.15.22:0/2735544693 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc64003d00 con 0x7fdc74101f70 2024-03-28T18:07:22.149 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:22.143+0000 7fdc73200640 1 -- 172.21.15.22:0/2735544693 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdc741a18c0 con 0x7fdc74101f70 2024-03-28T18:07:22.149 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:22.143+0000 7fdc73200640 1 -- 172.21.15.22:0/2735544693 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdc741a1dc0 con 0x7fdc74101f70 2024-03-28T18:07:22.149 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:22.144+0000 7fdc73200640 1 -- 172.21.15.22:0/2735544693 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdc74108d20 con 0x7fdc74101f70 2024-03-28T18:07:22.150 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:22.146+0000 7fdc6b400640 1 -- 172.21.15.22:0/2735544693 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 18) v1 ==== 137290+0+0 (secure 0 0 0) 0x7fdc64046020 con 0x7fdc74101f70 2024-03-28T18:07:22.151 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:22.148+0000 7fdc6b400640 1 --2- 172.21.15.22:0/2735544693 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7fdc580a2800 0x7fdc580a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:07:22.151 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:22.149+0000 7fdc6b400640 1 -- 172.21.15.22:0/2735544693 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 8153+0+0 (secure 0 0 0) 0x7fdc640f0d00 con 0x7fdc74101f70 2024-03-28T18:07:22.151 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:22.149+0000 7fdc69400640 1 --2- 172.21.15.22:0/2735544693 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7fdc580a2800 0x7fdc580a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:07:22.151 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:22.149+0000 7fdc69400640 1 --2- 172.21.15.22:0/2735544693 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7fdc580a2800 0x7fdc580a4cc0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7fdc5c005fd0 tx=0x7fdc5c005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:07:22.152 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:22.153+0000 7fdc6b400640 1 -- 172.21.15.22:0/2735544693 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fdc6404d020 con 0x7fdc74101f70 2024-03-28T18:07:22.174 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:07:21 smithi134 ceph-mon[35037]: pgmap v447: 97 pgs: 97 active+clean; 649 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 1.7 KiB/s wr, 12 op/s 2024-03-28T18:07:22.175 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:07:21 smithi134 ceph-mon[35037]: from='client.? 172.21.15.22:0/1096579762' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-28T18:07:22.175 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:07:21 smithi134 ceph-mon[35037]: from='client.? 172.21.15.22:0/3359946719' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-28T18:07:22.182 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:07:21 smithi079 ceph-mon[35312]: pgmap v447: 97 pgs: 97 active+clean; 649 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 1.7 KiB/s wr, 12 op/s 2024-03-28T18:07:22.182 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:07:21 smithi079 ceph-mon[35312]: from='client.? 172.21.15.22:0/1096579762' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-28T18:07:22.182 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:07:21 smithi079 ceph-mon[35312]: from='client.? 172.21.15.22:0/3359946719' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-28T18:07:22.396 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:22.396+0000 7fdc73200640 1 -- 172.21.15.22:0/2735544693 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fdc740622a0 con 0x7fdc74101f70 2024-03-28T18:07:22.398 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:22.398+0000 7fdc6b400640 1 -- 172.21.15.22:0/2735544693 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) v1 ==== 94+0+11452 (secure 0 0 0) 0x7fdc640bdda0 con 0x7fdc74101f70 2024-03-28T18:07:22.398 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-03-28T18:07:22.398 INFO:teuthology.orchestra.run.smithi022.stdout:{"epoch":27,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14544,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6835/3718238950","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6834","nonce":3718238950},{"type":"v1","addr":"172.21.15.22:6835","nonce":3718238950}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14562,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6837/3676939386","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6836","nonce":3676939386},{"type":"v1","addr":"172.21.15.22:6837","nonce":3676939386}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":27,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-28T18:04:48.991622+0000","modified":"2024-03-28T18:07:13.521383+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":34553,"mds_1":34511,"mds_2":34475,"mds_3":34442,"mds_4":24502},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14640":{"gid":14640,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.22:6839/3979998310","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6838","nonce":3979998310},{"type":"v1","addr":"172.21.15.22:6839","nonce":3979998310}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_14655":{"gid":14655,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.22:6841/3069056661","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6840","nonce":3069056661},{"type":"v1","addr":"172.21.15.22:6841","nonce":3069056661}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24412":{"gid":24412,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.134:6833/3176125362","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6832","nonce":3176125362},{"type":"v1","addr":"172.21.15.134:6833","nonce":3176125362}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24463":{"gid":24463,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.134:6835/3370549207","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6834","nonce":3370549207},{"type":"v1","addr":"172.21.15.134:6835","nonce":3370549207}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24478":{"gid":24478,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.134:6837/899825581","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6836","nonce":899825581},{"type":"v1","addr":"172.21.15.134:6837","nonce":899825581}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24502":{"gid":24502,"name":"i","rank":4,"incarnation":24,"state":"up:active","state_seq":7,"addr":"172.21.15.134:6839/3400263217","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6838","nonce":3400263217},{"type":"v1","addr":"172.21.15.134:6839","nonce":3400263217}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_34442":{"gid":34442,"name":"k","rank":3,"incarnation":22,"state":"up:active","state_seq":14,"addr":"172.21.15.79:6833/856896997","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6832","nonce":856896997},{"type":"v1","addr":"172.21.15.79:6833","nonce":856896997}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_34475":{"gid":34475,"name":"e","rank":2,"incarnation":20,"state":"up:active","state_seq":11,"addr":"172.21.15.79:6835/3365773431","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6834","nonce":3365773431},{"type":"v1","addr":"172.21.15.79:6835","nonce":3365773431}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_34511":{"gid":34511,"name":"h","rank":1,"incarnation":18,"state":"up:active","state_seq":8,"addr":"172.21.15.79:6837/1787903366","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6836","nonce":1787903366},{"type":"v1","addr":"172.21.15.79:6837","nonce":1787903366}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_34553":{"gid":34553,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.79:6839/2441542443","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6838","nonce":2441542443},{"type":"v1","addr":"172.21.15.79:6839","nonce":2441542443}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":34553,"qdb_cluster":[24502,34442,34475,34511,34553]},"id":1}]} 2024-03-28T18:07:22.398 INFO:teuthology.orchestra.run.smithi022.stderr:dumped fsmap epoch 27 2024-03-28T18:07:22.400 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:22.400+0000 7fdc73200640 1 -- 172.21.15.22:0/2735544693 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7fdc580a2800 msgr2=0x7fdc580a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:07:22.400 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:22.400+0000 7fdc73200640 1 --2- 172.21.15.22:0/2735544693 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7fdc580a2800 0x7fdc580a4cc0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7fdc5c005fd0 tx=0x7fdc5c005850 comp rx=0 tx=0).stop 2024-03-28T18:07:22.401 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:22.400+0000 7fdc73200640 1 -- 172.21.15.22:0/2735544693 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fdc74101f70 msgr2=0x7fdc74199a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:07:22.401 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:22.400+0000 7fdc73200640 1 --2- 172.21.15.22:0/2735544693 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fdc74101f70 0x7fdc74199a10 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7fdc64000c00 tx=0x7fdc64004e20 comp rx=0 tx=0).stop 2024-03-28T18:07:22.401 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:22.401+0000 7fdc73200640 1 -- 172.21.15.22:0/2735544693 shutdown_connections 2024-03-28T18:07:22.401 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:22.401+0000 7fdc73200640 1 --2- 172.21.15.22:0/2735544693 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7fdc580a2800 0x7fdc580a4cc0 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:07:22.401 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:22.401+0000 7fdc73200640 1 --2- 172.21.15.22:0/2735544693 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fdc741071d0 0x7fdc741a0fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:07:22.401 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:22.401+0000 7fdc73200640 1 --2- 172.21.15.22:0/2735544693 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdc741048a0 0x7fdc74199f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:07:22.401 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:22.401+0000 7fdc73200640 1 --2- 172.21.15.22:0/2735544693 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fdc74101f70 0x7fdc74199a10 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:07:22.401 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:22.401+0000 7fdc73200640 1 -- 172.21.15.22:0/2735544693 >> 172.21.15.22:0/2735544693 conn(0x7fdc740fb980 msgr2=0x7fdc741016f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:07:22.401 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:22.401+0000 7fdc73200640 1 -- 172.21.15.22:0/2735544693 shutdown_connections 2024-03-28T18:07:22.401 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:22.401+0000 7fdc73200640 1 -- 172.21.15.22:0/2735544693 wait complete. 2024-03-28T18:07:22.419 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-28T18:07:22.647 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:22.647+0000 7f9e01200640 1 -- 172.21.15.22:0/2718310874 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9dfc1071d0 msgr2=0x7f9dfc109620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:07:22.647 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:22.647+0000 7f9e01200640 1 --2- 172.21.15.22:0/2718310874 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9dfc1071d0 0x7f9dfc109620 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f9df4009900 tx=0x7f9df402bc30 comp rx=0 tx=0).stop 2024-03-28T18:07:22.647 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:22.647+0000 7f9e01200640 1 -- 172.21.15.22:0/2718310874 shutdown_connections 2024-03-28T18:07:22.647 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:22.647+0000 7f9e01200640 1 --2- 172.21.15.22:0/2718310874 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9dfc1071d0 0x7f9dfc109620 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:07:22.647 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:22.647+0000 7f9e01200640 1 --2- 172.21.15.22:0/2718310874 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f9dfc1048a0 0x7f9dfc106c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:07:22.647 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:22.647+0000 7f9e01200640 1 --2- 172.21.15.22:0/2718310874 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9dfc101f70 0x7f9dfc104360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:07:22.647 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:22.647+0000 7f9e01200640 1 -- 172.21.15.22:0/2718310874 >> 172.21.15.22:0/2718310874 conn(0x7f9dfc0fb980 msgr2=0x7f9dfc0fdda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:07:22.647 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:22.647+0000 7f9e01200640 1 -- 172.21.15.22:0/2718310874 shutdown_connections 2024-03-28T18:07:22.647 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:22.647+0000 7f9e01200640 1 -- 172.21.15.22:0/2718310874 wait complete. 2024-03-28T18:07:22.649 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:22.648+0000 7f9e01200640 1 Processor -- start 2024-03-28T18:07:22.649 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:22.648+0000 7f9e01200640 1 -- start start 2024-03-28T18:07:22.649 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:22.648+0000 7f9e01200640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9dfc101f70 0x7f9dfc199a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:07:22.650 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:22.648+0000 7f9e01200640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9dfc1048a0 0x7f9dfc199f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:07:22.650 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:22.648+0000 7f9e01200640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f9dfc1071d0 0x7f9dfc1a0fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:07:22.650 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:22.648+0000 7f9e01200640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9dfc10d170 con 0x7f9dfc101f70 2024-03-28T18:07:22.650 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:22.648+0000 7f9e01200640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9dfc10cff0 con 0x7f9dfc1071d0 2024-03-28T18:07:22.650 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:22.648+0000 7f9e01200640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9dfc10d2f0 con 0x7f9dfc1048a0 2024-03-28T18:07:22.650 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:22.649+0000 7f9dfb400640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9dfc101f70 0x7f9dfc199a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:07:22.650 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:22.649+0000 7f9dfb400640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9dfc101f70 0x7f9dfc199a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:55608/0 (socket says 172.21.15.22:55608) 2024-03-28T18:07:22.650 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:22.649+0000 7f9dfb400640 1 -- 172.21.15.22:0/470254142 learned_addr learned my addr 172.21.15.22:0/470254142 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-28T18:07:22.650 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:22.649+0000 7f9dfbe00640 1 --2- 172.21.15.22:0/470254142 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f9dfc1071d0 0x7f9dfc1a0fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:07:22.650 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:22.649+0000 7f9dfaa00640 1 --2- 172.21.15.22:0/470254142 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9dfc1048a0 0x7f9dfc199f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:07:22.650 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:22.649+0000 7f9dfb400640 1 -- 172.21.15.22:0/470254142 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9dfc1048a0 msgr2=0x7f9dfc199f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:07:22.650 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:22.649+0000 7f9dfb400640 1 --2- 172.21.15.22:0/470254142 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9dfc1048a0 0x7f9dfc199f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:07:22.650 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:22.649+0000 7f9dfb400640 1 -- 172.21.15.22:0/470254142 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f9dfc1071d0 msgr2=0x7f9dfc1a0fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:07:22.650 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:22.649+0000 7f9dfb400640 1 --2- 172.21.15.22:0/470254142 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f9dfc1071d0 0x7f9dfc1a0fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:07:22.651 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:22.649+0000 7f9dfb400640 1 -- 172.21.15.22:0/470254142 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9dfc1a1620 con 0x7f9dfc101f70 2024-03-28T18:07:22.651 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:22.649+0000 7f9dfaa00640 1 --2- 172.21.15.22:0/470254142 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9dfc1048a0 0x7f9dfc199f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T18:07:22.651 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:22.649+0000 7f9dfb400640 1 --2- 172.21.15.22:0/470254142 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9dfc101f70 0x7f9dfc199a00 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f9dec00d770 tx=0x7f9dec00dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:07:22.651 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:22.650+0000 7f9df3e00640 1 -- 172.21.15.22:0/470254142 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9dec017070 con 0x7f9dfc101f70 2024-03-28T18:07:22.651 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:22.650+0000 7f9e01200640 1 -- 172.21.15.22:0/470254142 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9dfc1a1910 con 0x7f9dfc101f70 2024-03-28T18:07:22.651 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:22.650+0000 7f9df3e00640 1 -- 172.21.15.22:0/470254142 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f9dec0054f0 con 0x7f9dfc101f70 2024-03-28T18:07:22.651 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:22.650+0000 7f9df3e00640 1 -- 172.21.15.22:0/470254142 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9dec004d60 con 0x7f9dfc101f70 2024-03-28T18:07:22.651 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:22.650+0000 7f9e01200640 1 -- 172.21.15.22:0/470254142 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9dfc1a1e10 con 0x7f9dfc101f70 2024-03-28T18:07:22.651 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:22.651+0000 7f9e01200640 1 -- 172.21.15.22:0/470254142 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9dfc108cc0 con 0x7f9dfc101f70 2024-03-28T18:07:22.653 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:22.653+0000 7f9df3e00640 1 -- 172.21.15.22:0/470254142 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 18) v1 ==== 137290+0+0 (secure 0 0 0) 0x7f9dec004f00 con 0x7f9dfc101f70 2024-03-28T18:07:22.654 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:22.654+0000 7f9df3e00640 1 --2- 172.21.15.22:0/470254142 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7f9ddc0a2800 0x7f9ddc0a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:07:22.654 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:22.655+0000 7f9df3e00640 1 -- 172.21.15.22:0/470254142 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 8153+0+0 (secure 0 0 0) 0x7f9dec0ce810 con 0x7f9dfc101f70 2024-03-28T18:07:22.654 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:22.655+0000 7f9dfaa00640 1 --2- 172.21.15.22:0/470254142 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7f9ddc0a2800 0x7f9ddc0a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:07:22.655 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:22.656+0000 7f9dfaa00640 1 --2- 172.21.15.22:0/470254142 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7f9ddc0a2800 0x7f9ddc0a4cc0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f9dfc19af10 tx=0x7f9de4009500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:07:22.657 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:22.657+0000 7f9df3e00640 1 -- 172.21.15.22:0/470254142 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f9dec09b950 con 0x7f9dfc101f70 2024-03-28T18:07:22.864 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:22.865+0000 7f9e01200640 1 -- 172.21.15.22:0/470254142 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f9dfc0622a0 con 0x7f9dfc101f70 2024-03-28T18:07:22.866 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:22.867+0000 7f9df3e00640 1 -- 172.21.15.22:0/470254142 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v92) v1 ==== 74+0+22522 (secure 0 0 0) 0x7f9dec09b250 con 0x7f9dfc101f70 2024-03-28T18:07:22.867 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-03-28T18:07:22.867 INFO:teuthology.orchestra.run.smithi022.stdout:{"epoch":92,"fsid":"fd220748-ed2b-11ee-95d0-87774f69a715","created":"2024-03-28T17:54:00.440279+0000","modified":"2024-03-28T18:05:11.725341+0000","last_up_change":"2024-03-28T18:03:43.220095+0000","last_in_change":"2024-03-28T18:03:18.526399+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-28T17:58:33.168872+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-28T18:04:47.004797+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-28T18:04:48.465502+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"90","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"5d708e06-f05b-40ba-bbcf-258ddc4f43d7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6802","nonce":2186290846},{"type":"v1","addr":"172.21.15.22:6803","nonce":2186290846}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6804","nonce":2186290846},{"type":"v1","addr":"172.21.15.22:6805","nonce":2186290846}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6808","nonce":2186290846},{"type":"v1","addr":"172.21.15.22:6809","nonce":2186290846}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6806","nonce":2186290846},{"type":"v1","addr":"172.21.15.22:6807","nonce":2186290846}]},"public_addr":"172.21.15.22:6803/2186290846","cluster_addr":"172.21.15.22:6805/2186290846","heartbeat_back_addr":"172.21.15.22:6809/2186290846","heartbeat_front_addr":"172.21.15.22:6807/2186290846","state":["exists","up"]},{"osd":1,"uuid":"68ad09bb-9256-409a-9867-4e14a2cb94aa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6800","nonce":2835549681},{"type":"v1","addr":"172.21.15.79:6801","nonce":2835549681}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6802","nonce":2835549681},{"type":"v1","addr":"172.21.15.79:6803","nonce":2835549681}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6806","nonce":2835549681},{"type":"v1","addr":"172.21.15.79:6807","nonce":2835549681}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6804","nonce":2835549681},{"type":"v1","addr":"172.21.15.79:6805","nonce":2835549681}]},"public_addr":"172.21.15.79:6801/2835549681","cluster_addr":"172.21.15.79:6803/2835549681","heartbeat_back_addr":"172.21.15.79:6807/2835549681","heartbeat_front_addr":"172.21.15.79:6805/2835549681","state":["exists","up"]},{"osd":2,"uuid":"ed11a5a7-004d-41f3-a5fc-eb2d603bd33f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6800","nonce":470588085},{"type":"v1","addr":"172.21.15.134:6801","nonce":470588085}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6802","nonce":470588085},{"type":"v1","addr":"172.21.15.134:6803","nonce":470588085}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6806","nonce":470588085},{"type":"v1","addr":"172.21.15.134:6807","nonce":470588085}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6804","nonce":470588085},{"type":"v1","addr":"172.21.15.134:6805","nonce":470588085}]},"public_addr":"172.21.15.134:6801/470588085","cluster_addr":"172.21.15.134:6803/470588085","heartbeat_back_addr":"172.21.15.134:6807/470588085","heartbeat_front_addr":"172.21.15.134:6805/470588085","state":["exists","up"]},{"osd":3,"uuid":"81a6c547-2a50-44c3-84d7-23ab6831790c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6810","nonce":1702441854},{"type":"v1","addr":"172.21.15.22:6811","nonce":1702441854}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6812","nonce":1702441854},{"type":"v1","addr":"172.21.15.22:6813","nonce":1702441854}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6816","nonce":1702441854},{"type":"v1","addr":"172.21.15.22:6817","nonce":1702441854}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6814","nonce":1702441854},{"type":"v1","addr":"172.21.15.22:6815","nonce":1702441854}]},"public_addr":"172.21.15.22:6811/1702441854","cluster_addr":"172.21.15.22:6813/1702441854","heartbeat_back_addr":"172.21.15.22:6817/1702441854","heartbeat_front_addr":"172.21.15.22:6815/1702441854","state":["exists","up"]},{"osd":4,"uuid":"d23a254e-7182-4471-8bca-7eb684ec618e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6808","nonce":630132437},{"type":"v1","addr":"172.21.15.79:6809","nonce":630132437}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6810","nonce":630132437},{"type":"v1","addr":"172.21.15.79:6811","nonce":630132437}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6814","nonce":630132437},{"type":"v1","addr":"172.21.15.79:6815","nonce":630132437}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6812","nonce":630132437},{"type":"v1","addr":"172.21.15.79:6813","nonce":630132437}]},"public_addr":"172.21.15.79:6809/630132437","cluster_addr":"172.21.15.79:6811/630132437","heartbeat_back_addr":"172.21.15.79:6815/630132437","heartbeat_front_addr":"172.21.15.79:6813/630132437","state":["exists","up"]},{"osd":5,"uuid":"a7b98502-2023-4f0a-bfa0-6840cc495e0c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":40,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6808","nonce":2135406915},{"type":"v1","addr":"172.21.15.134:6809","nonce":2135406915}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6810","nonce":2135406915},{"type":"v1","addr":"172.21.15.134:6811","nonce":2135406915}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6814","nonce":2135406915},{"type":"v1","addr":"172.21.15.134:6815","nonce":2135406915}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6812","nonce":2135406915},{"type":"v1","addr":"172.21.15.134:6813","nonce":2135406915}]},"public_addr":"172.21.15.134:6809/2135406915","cluster_addr":"172.21.15.134:6811/2135406915","heartbeat_back_addr":"172.21.15.134:6815/2135406915","heartbeat_front_addr":"172.21.15.134:6813/2135406915","state":["exists","up"]},{"osd":6,"uuid":"634c6562-d683-41c8-8242-fc848c38ce1f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6818","nonce":1400680396},{"type":"v1","addr":"172.21.15.22:6819","nonce":1400680396}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6820","nonce":1400680396},{"type":"v1","addr":"172.21.15.22:6821","nonce":1400680396}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6824","nonce":1400680396},{"type":"v1","addr":"172.21.15.22:6825","nonce":1400680396}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6822","nonce":1400680396},{"type":"v1","addr":"172.21.15.22:6823","nonce":1400680396}]},"public_addr":"172.21.15.22:6819/1400680396","cluster_addr":"172.21.15.22:6821/1400680396","heartbeat_back_addr":"172.21.15.22:6825/1400680396","heartbeat_front_addr":"172.21.15.22:6823/1400680396","state":["exists","up"]},{"osd":7,"uuid":"504eb4a8-6731-45ad-932c-1b38ef157871","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":53,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6816","nonce":1287105314},{"type":"v1","addr":"172.21.15.79:6817","nonce":1287105314}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6818","nonce":1287105314},{"type":"v1","addr":"172.21.15.79:6819","nonce":1287105314}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6822","nonce":1287105314},{"type":"v1","addr":"172.21.15.79:6823","nonce":1287105314}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6820","nonce":1287105314},{"type":"v1","addr":"172.21.15.79:6821","nonce":1287105314}]},"public_addr":"172.21.15.79:6817/1287105314","cluster_addr":"172.21.15.79:6819/1287105314","heartbeat_back_addr":"172.21.15.79:6823/1287105314","heartbeat_front_addr":"172.21.15.79:6821/1287105314","state":["exists","up"]},{"osd":8,"uuid":"20aa66c1-03d6-4cb8-a096-0eae41979c17","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":62,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6816","nonce":2265650388},{"type":"v1","addr":"172.21.15.134:6817","nonce":2265650388}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6818","nonce":2265650388},{"type":"v1","addr":"172.21.15.134:6819","nonce":2265650388}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6822","nonce":2265650388},{"type":"v1","addr":"172.21.15.134:6823","nonce":2265650388}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6820","nonce":2265650388},{"type":"v1","addr":"172.21.15.134:6821","nonce":2265650388}]},"public_addr":"172.21.15.134:6817/2265650388","cluster_addr":"172.21.15.134:6819/2265650388","heartbeat_back_addr":"172.21.15.134:6823/2265650388","heartbeat_front_addr":"172.21.15.134:6821/2265650388","state":["exists","up"]},{"osd":9,"uuid":"e1ae4020-7c2c-4334-a753-ee6b12d98584","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":70,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6826","nonce":3762222259},{"type":"v1","addr":"172.21.15.22:6827","nonce":3762222259}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6828","nonce":3762222259},{"type":"v1","addr":"172.21.15.22:6829","nonce":3762222259}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6832","nonce":3762222259},{"type":"v1","addr":"172.21.15.22:6833","nonce":3762222259}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6830","nonce":3762222259},{"type":"v1","addr":"172.21.15.22:6831","nonce":3762222259}]},"public_addr":"172.21.15.22:6827/3762222259","cluster_addr":"172.21.15.22:6829/3762222259","heartbeat_back_addr":"172.21.15.22:6833/3762222259","heartbeat_front_addr":"172.21.15.22:6831/3762222259","state":["exists","up"]},{"osd":10,"uuid":"f5198d52-df48-4679-a860-4dcb440b7a9f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":77,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6824","nonce":1746167904},{"type":"v1","addr":"172.21.15.79:6825","nonce":1746167904}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6826","nonce":1746167904},{"type":"v1","addr":"172.21.15.79:6827","nonce":1746167904}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6830","nonce":1746167904},{"type":"v1","addr":"172.21.15.79:6831","nonce":1746167904}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6828","nonce":1746167904},{"type":"v1","addr":"172.21.15.79:6829","nonce":1746167904}]},"public_addr":"172.21.15.79:6825/1746167904","cluster_addr":"172.21.15.79:6827/1746167904","heartbeat_back_addr":"172.21.15.79:6831/1746167904","heartbeat_front_addr":"172.21.15.79:6829/1746167904","state":["exists","up"]},{"osd":11,"uuid":"1427bca7-3414-4468-9246-e999156bc2ae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":85,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6824","nonce":4271716428},{"type":"v1","addr":"172.21.15.134:6825","nonce":4271716428}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6826","nonce":4271716428},{"type":"v1","addr":"172.21.15.134:6827","nonce":4271716428}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6830","nonce":4271716428},{"type":"v1","addr":"172.21.15.134:6831","nonce":4271716428}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6828","nonce":4271716428},{"type":"v1","addr":"172.21.15.134:6829","nonce":4271716428}]},"public_addr":"172.21.15.134:6825/4271716428","cluster_addr":"172.21.15.134:6827/4271716428","heartbeat_back_addr":"172.21.15.134:6831/4271716428","heartbeat_front_addr":"172.21.15.134:6829/4271716428","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T17:57:21.225125+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T17:57:55.197254+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T17:58:27.825136+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T17:59:04.319893+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T17:59:38.407752+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T18:00:12.118805+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T18:00:47.635214+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T18:01:21.231055+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T18:01:55.817861+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T18:02:31.680160+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T18:03:06.783398+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T18:03:41.307347+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.8","mappings":[{"from":2,"to":11}]},{"pgid":"2.1d","mappings":[{"from":8,"to":11}]},{"pgid":"3.3","mappings":[{"from":4,"to":9}]},{"pgid":"3.c","mappings":[{"from":3,"to":9}]},{"pgid":"3.21","mappings":[{"from":4,"to":8}]},{"pgid":"3.25","mappings":[{"from":5,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.22:0/3521924283":"2024-03-29T17:55:10.425504+0000","172.21.15.22:6801/730635939":"2024-03-29T17:55:10.425504+0000","172.21.15.22:0/2471229459":"2024-03-29T17:54:34.004559+0000","172.21.15.22:6800/3166828081":"2024-03-29T17:54:34.004559+0000","172.21.15.22:6800/730635939":"2024-03-29T17:55:10.425504+0000","172.21.15.22:0/1229047849":"2024-03-29T17:54:34.004559+0000","172.21.15.22:6801/3166828081":"2024-03-29T17:54:34.004559+0000","172.21.15.22:0/2717818856":"2024-03-29T17:55:10.425504+0000","172.21.15.22:0/2157570333":"2024-03-29T17:54:34.004559+0000","172.21.15.22:0/1006589530":"2024-03-29T17:55:10.425504+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-28T18:07:22.870 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:22.870+0000 7f9e01200640 1 -- 172.21.15.22:0/470254142 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7f9ddc0a2800 msgr2=0x7f9ddc0a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:07:22.870 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:22.870+0000 7f9e01200640 1 --2- 172.21.15.22:0/470254142 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7f9ddc0a2800 0x7f9ddc0a4cc0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f9dfc19af10 tx=0x7f9de4009500 comp rx=0 tx=0).stop 2024-03-28T18:07:22.870 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:22.870+0000 7f9e01200640 1 -- 172.21.15.22:0/470254142 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9dfc101f70 msgr2=0x7f9dfc199a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:07:22.870 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:22.870+0000 7f9e01200640 1 --2- 172.21.15.22:0/470254142 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9dfc101f70 0x7f9dfc199a00 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f9dec00d770 tx=0x7f9dec00dc30 comp rx=0 tx=0).stop 2024-03-28T18:07:22.870 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:22.870+0000 7f9e01200640 1 -- 172.21.15.22:0/470254142 shutdown_connections 2024-03-28T18:07:22.870 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:22.870+0000 7f9e01200640 1 --2- 172.21.15.22:0/470254142 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7f9ddc0a2800 0x7f9ddc0a4cc0 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:07:22.870 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:22.870+0000 7f9e01200640 1 --2- 172.21.15.22:0/470254142 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f9dfc1071d0 0x7f9dfc1a0fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:07:22.870 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:22.870+0000 7f9e01200640 1 --2- 172.21.15.22:0/470254142 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9dfc1048a0 0x7f9dfc199f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:07:22.870 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:22.870+0000 7f9e01200640 1 --2- 172.21.15.22:0/470254142 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9dfc101f70 0x7f9dfc199a00 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:07:22.871 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:22.870+0000 7f9e01200640 1 -- 172.21.15.22:0/470254142 >> 172.21.15.22:0/470254142 conn(0x7f9dfc0fb980 msgr2=0x7f9dfc101680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:07:22.871 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:22.870+0000 7f9e01200640 1 -- 172.21.15.22:0/470254142 shutdown_connections 2024-03-28T18:07:22.871 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:22.870+0000 7f9e01200640 1 -- 172.21.15.22:0/470254142 wait complete. 2024-03-28T18:07:22.890 DEBUG:tasks.fwd_scrub:Yielding 2024-03-28T18:07:22.891 INFO:teuthology.run_tasks:Running task exec... 2024-03-28T18:07:22.902 INFO:teuthology.task.exec:Executing custom commands... 2024-03-28T18:07:22.902 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi022.front.sepia.ceph.com 2024-03-28T18:07:22.902 DEBUG:teuthology.orchestra.run.smithi022:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph mgr module enable snap_schedule' 2024-03-28T18:07:22.903 INFO:tasks.fwd_scrub.fs.[cephfs]:start scrubbing fs: cephfs 2024-03-28T18:07:22.903 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-03-28T18:07:22.903 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-28T18:07:22.903 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-03-28T18:07:22.977 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:07:22 smithi022 ceph-mon[31985]: from='client.? 172.21.15.22:0/2388986592' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-28T18:07:22.977 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:07:22 smithi022 ceph-mon[31985]: from='client.? 172.21.15.22:0/2735544693' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-28T18:07:23.106 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:23.106+0000 7f2d1f200640 1 -- 172.21.15.22:0/3649078075 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2d180756e0 msgr2=0x7f2d18073b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:07:23.106 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:23.106+0000 7f2d1f200640 1 --2- 172.21.15.22:0/3649078075 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2d180756e0 0x7f2d18073b30 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f2d0c009900 tx=0x7f2d0c02bc30 comp rx=0 tx=0).stop 2024-03-28T18:07:23.106 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:23.106+0000 7f2d1f200640 1 -- 172.21.15.22:0/3649078075 shutdown_connections 2024-03-28T18:07:23.106 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:23.106+0000 7f2d1f200640 1 --2- 172.21.15.22:0/3649078075 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2d181093d0 0x7f2d1810b860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:07:23.106 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:23.106+0000 7f2d1f200640 1 --2- 172.21.15.22:0/3649078075 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f2d18074100 0x7f2d18074580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:07:23.107 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:23.106+0000 7f2d1f200640 1 --2- 172.21.15.22:0/3649078075 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2d180756e0 0x7f2d18073b30 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:07:23.107 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:23.106+0000 7f2d1f200640 1 -- 172.21.15.22:0/3649078075 >> 172.21.15.22:0/3649078075 conn(0x7f2d180fca80 msgr2=0x7f2d180feec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:07:23.107 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:23.106+0000 7f2d1f200640 1 -- 172.21.15.22:0/3649078075 shutdown_connections 2024-03-28T18:07:23.107 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:23.106+0000 7f2d1f200640 1 -- 172.21.15.22:0/3649078075 wait complete. 2024-03-28T18:07:23.108 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:23.107+0000 7f2d1f200640 1 Processor -- start 2024-03-28T18:07:23.109 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:23.108+0000 7f2d1f200640 1 -- start start 2024-03-28T18:07:23.109 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:23.108+0000 7f2d1f200640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2d18074100 0x7f2d181999c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:07:23.109 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:23.108+0000 7f2d1f200640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2d180756e0 0x7f2d18199f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:07:23.109 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:23.108+0000 7f2d1f200640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f2d181093d0 0x7f2d181a0f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:07:23.109 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:23.108+0000 7f2d1f200640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d1810f3b0 con 0x7f2d18074100 2024-03-28T18:07:23.109 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:23.108+0000 7f2d1f200640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d1810f230 con 0x7f2d181093d0 2024-03-28T18:07:23.109 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:23.108+0000 7f2d1f200640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d1810f530 con 0x7f2d180756e0 2024-03-28T18:07:23.109 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:23.108+0000 7f2d1de00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2d18074100 0x7f2d181999c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:07:23.109 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:23.108+0000 7f2d1de00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2d18074100 0x7f2d181999c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:55618/0 (socket says 172.21.15.22:55618) 2024-03-28T18:07:23.109 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:23.108+0000 7f2d1de00640 1 -- 172.21.15.22:0/662263897 learned_addr learned my addr 172.21.15.22:0/662263897 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-28T18:07:23.109 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:23.108+0000 7f2d1e800640 1 --2- 172.21.15.22:0/662263897 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f2d181093d0 0x7f2d181a0f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:07:23.109 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:23.108+0000 7f2d1d400640 1 --2- 172.21.15.22:0/662263897 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2d180756e0 0x7f2d18199f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:07:23.109 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:23.109+0000 7f2d1de00640 1 -- 172.21.15.22:0/662263897 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2d180756e0 msgr2=0x7f2d18199f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:07:23.110 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:23.109+0000 7f2d1de00640 1 --2- 172.21.15.22:0/662263897 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2d180756e0 0x7f2d18199f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:07:23.110 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:23.109+0000 7f2d1de00640 1 -- 172.21.15.22:0/662263897 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f2d181093d0 msgr2=0x7f2d181a0f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:07:23.110 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:23.109+0000 7f2d1de00640 1 --2- 172.21.15.22:0/662263897 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f2d181093d0 0x7f2d181a0f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:07:23.110 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:23.109+0000 7f2d1de00640 1 -- 172.21.15.22:0/662263897 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d181a15e0 con 0x7f2d18074100 2024-03-28T18:07:23.110 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:23.109+0000 7f2d1d400640 1 --2- 172.21.15.22:0/662263897 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2d180756e0 0x7f2d18199f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T18:07:23.110 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:23.109+0000 7f2d1e800640 1 --2- 172.21.15.22:0/662263897 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f2d181093d0 0x7f2d181a0f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-28T18:07:23.110 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:23.109+0000 7f2d1de00640 1 --2- 172.21.15.22:0/662263897 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2d18074100 0x7f2d181999c0 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f2d0c000c00 tx=0x7f2d0c004e20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:07:23.111 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:23.109+0000 7f2d12a00640 1 -- 172.21.15.22:0/662263897 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d0c043020 con 0x7f2d18074100 2024-03-28T18:07:23.111 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:23.109+0000 7f2d12a00640 1 -- 172.21.15.22:0/662263897 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f2d0c005460 con 0x7f2d18074100 2024-03-28T18:07:23.111 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:23.109+0000 7f2d12a00640 1 -- 172.21.15.22:0/662263897 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d0c003d00 con 0x7f2d18074100 2024-03-28T18:07:23.111 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:23.109+0000 7f2d1f200640 1 -- 172.21.15.22:0/662263897 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2d181a1870 con 0x7f2d18074100 2024-03-28T18:07:23.111 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:23.110+0000 7f2d1f200640 1 -- 172.21.15.22:0/662263897 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2d181a1d70 con 0x7f2d18074100 2024-03-28T18:07:23.111 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:23.111+0000 7f2d1f200640 1 -- 172.21.15.22:0/662263897 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2d18108100 con 0x7f2d18074100 2024-03-28T18:07:23.114 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:23.115+0000 7f2d12a00640 1 -- 172.21.15.22:0/662263897 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 18) v1 ==== 137290+0+0 (secure 0 0 0) 0x7f2d0c046020 con 0x7f2d18074100 2024-03-28T18:07:23.116 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:23.116+0000 7f2d12a00640 1 --2- 172.21.15.22:0/662263897 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7f2d000a2870 0x7f2d000a4d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:07:23.116 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:23.116+0000 7f2d12a00640 1 -- 172.21.15.22:0/662263897 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 8153+0+0 (secure 0 0 0) 0x7f2d0c0f0d40 con 0x7f2d18074100 2024-03-28T18:07:23.116 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:23.116+0000 7f2d1d400640 1 --2- 172.21.15.22:0/662263897 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7f2d000a2870 0x7f2d000a4d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:07:23.116 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:23.117+0000 7f2d1d400640 1 --2- 172.21.15.22:0/662263897 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7f2d000a2870 0x7f2d000a4d30 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f2d1819aed0 tx=0x7f2d08005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:07:23.120 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:23.121+0000 7f2d12a00640 1 -- 172.21.15.22:0/662263897 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f2d0c04d020 con 0x7f2d18074100 2024-03-28T18:07:23.121 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:23.122+0000 7ffb7e800640 1 -- 172.21.15.22:0/518113129 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ffb78100ac0 msgr2=0x7ffb78100ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:07:23.121 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:23.122+0000 7ffb7e800640 1 --2- 172.21.15.22:0/518113129 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ffb78100ac0 0x7ffb78100ee0 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7ffb68009900 tx=0x7ffb6802bc30 comp rx=0 tx=0).stop 2024-03-28T18:07:23.122 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:23.122+0000 7ffb7e800640 1 -- 172.21.15.22:0/518113129 shutdown_connections 2024-03-28T18:07:23.122 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:23.122+0000 7ffb7e800640 1 --2- 172.21.15.22:0/518113129 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ffb78102190 0x7ffb78108a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:07:23.122 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:23.122+0000 7ffb7e800640 1 --2- 172.21.15.22:0/518113129 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ffb781017d0 0x7ffb78101c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:07:23.122 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:23.122+0000 7ffb7e800640 1 --2- 172.21.15.22:0/518113129 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ffb78100ac0 0x7ffb78100ee0 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:07:23.122 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:23.122+0000 7ffb7e800640 1 -- 172.21.15.22:0/518113129 >> 172.21.15.22:0/518113129 conn(0x7ffb780fb290 msgr2=0x7ffb780fd6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:07:23.122 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:23.122+0000 7ffb7e800640 1 -- 172.21.15.22:0/518113129 shutdown_connections 2024-03-28T18:07:23.122 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:23.122+0000 7ffb7e800640 1 -- 172.21.15.22:0/518113129 wait complete. 2024-03-28T18:07:23.123 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:23.123+0000 7ffb7e800640 1 Processor -- start 2024-03-28T18:07:23.123 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:23.123+0000 7ffb7e800640 1 -- start start 2024-03-28T18:07:23.123 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:23.123+0000 7ffb7e800640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ffb78100ac0 0x7ffb78199c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:07:23.123 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:23.123+0000 7ffb7e800640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ffb781017d0 0x7ffb7819a1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:07:23.123 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:23.123+0000 7ffb7e800640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ffb78102190 0x7ffb781a1230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:07:23.123 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:23.123+0000 7ffb7e800640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb7810c280 con 0x7ffb78102190 2024-03-28T18:07:23.123 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:23.123+0000 7ffb7e800640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb7810c100 con 0x7ffb78100ac0 2024-03-28T18:07:23.123 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:23.123+0000 7ffb7e800640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb7810c400 con 0x7ffb781017d0 2024-03-28T18:07:23.125 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:23.124+0000 7ffb77e00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ffb78102190 0x7ffb781a1230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:07:23.125 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:23.124+0000 7ffb77e00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ffb78102190 0x7ffb781a1230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:55640/0 (socket says 172.21.15.22:55640) 2024-03-28T18:07:23.125 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:23.124+0000 7ffb77e00640 1 -- 172.21.15.22:0/2562049588 learned_addr learned my addr 172.21.15.22:0/2562049588 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-28T18:07:23.125 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:23.124+0000 7ffb77400640 1 --2- 172.21.15.22:0/2562049588 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ffb78100ac0 0x7ffb78199c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:07:23.125 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:23.124+0000 7ffb76a00640 1 --2- 172.21.15.22:0/2562049588 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ffb781017d0 0x7ffb7819a1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:07:23.125 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:23.124+0000 7ffb77e00640 1 -- 172.21.15.22:0/2562049588 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ffb781017d0 msgr2=0x7ffb7819a1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:07:23.125 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:23.124+0000 7ffb77e00640 1 --2- 172.21.15.22:0/2562049588 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ffb781017d0 0x7ffb7819a1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:07:23.125 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:23.124+0000 7ffb77e00640 1 -- 172.21.15.22:0/2562049588 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ffb78100ac0 msgr2=0x7ffb78199c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:07:23.125 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:23.124+0000 7ffb77e00640 1 --2- 172.21.15.22:0/2562049588 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ffb78100ac0 0x7ffb78199c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:07:23.125 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:23.124+0000 7ffb77e00640 1 -- 172.21.15.22:0/2562049588 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb781a18d0 con 0x7ffb78102190 2024-03-28T18:07:23.125 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:23.125+0000 7ffb77e00640 1 --2- 172.21.15.22:0/2562049588 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ffb78102190 0x7ffb781a1230 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7ffb7000cad0 tx=0x7ffb7000cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:07:23.126 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:23.125+0000 7ffb6fe00640 1 -- 172.21.15.22:0/2562049588 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb7000ae20 con 0x7ffb78102190 2024-03-28T18:07:23.126 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:23.125+0000 7ffb6fe00640 1 -- 172.21.15.22:0/2562049588 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7ffb70005020 con 0x7ffb78102190 2024-03-28T18:07:23.126 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:23.125+0000 7ffb6fe00640 1 -- 172.21.15.22:0/2562049588 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb7000f690 con 0x7ffb78102190 2024-03-28T18:07:23.126 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:23.125+0000 7ffb7e800640 1 -- 172.21.15.22:0/2562049588 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffb781a1bc0 con 0x7ffb78102190 2024-03-28T18:07:23.126 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:23.125+0000 7ffb7e800640 1 -- 172.21.15.22:0/2562049588 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffb781a2090 con 0x7ffb78102190 2024-03-28T18:07:23.129 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:23.129+0000 7ffb81571740 1 Processor -- start 2024-03-28T18:07:23.129 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:23.130+0000 7ffb6fe00640 1 -- 172.21.15.22:0/2562049588 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 18) v1 ==== 137290+0+0 (secure 0 0 0) 0x7ffb700051d0 con 0x7ffb78102190 2024-03-28T18:07:23.129 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:23.130+0000 7ffb81571740 1 -- start start 2024-03-28T18:07:23.131 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:23.130+0000 7ffb81571740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561acfaf80a0 0x561acfaf84c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:07:23.131 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:23.130+0000 7ffb81571740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561acfaf8a90 0x561acfafd590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:07:23.131 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:23.130+0000 7ffb81571740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x561acfafdc00 0x561acfb00090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:07:23.131 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:23.130+0000 7ffb81571740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x561acfa5deb0 con 0x561acfaf80a0 2024-03-28T18:07:23.131 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:23.130+0000 7ffb81571740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x561acfa06a60 con 0x561acfafdc00 2024-03-28T18:07:23.131 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:23.130+0000 7ffb81571740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x561acfa59f50 con 0x561acfaf8a90 2024-03-28T18:07:23.131 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:23.131+0000 7ffb76a00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561acfaf80a0 0x561acfaf84c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:07:23.132 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:23.131+0000 7ffb76a00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561acfaf80a0 0x561acfaf84c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:55654/0 (socket says 172.21.15.22:55654) 2024-03-28T18:07:23.132 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:23.131+0000 7ffb76a00640 1 -- 172.21.15.22:0/2426646342 learned_addr learned my addr 172.21.15.22:0/2426646342 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-28T18:07:23.132 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:23.131+0000 7ffb76a00640 1 --2- 172.21.15.22:0/2426646342 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x561acfafdc00 0x561acfb00090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:07:23.132 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:23.131+0000 7ffb77400640 1 --2- 172.21.15.22:0/2426646342 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561acfaf8a90 0x561acfafd590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:07:23.132 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:23.131+0000 7ffb76a00640 1 -- 172.21.15.22:0/2426646342 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561acfaf8a90 msgr2=0x561acfafd590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T18:07:23.132 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:23.131+0000 7ffb76a00640 1 --2- 172.21.15.22:0/2426646342 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561acfaf8a90 0x561acfafd590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:07:23.132 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:23.131+0000 7ffb76a00640 1 -- 172.21.15.22:0/2426646342 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x561acfafdc00 msgr2=0x561acfb00090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T18:07:23.132 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:23.131+0000 7ffb76a00640 1 --2- 172.21.15.22:0/2426646342 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x561acfafdc00 0x561acfb00090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:07:23.132 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:23.131+0000 7ffb76a00640 1 -- 172.21.15.22:0/2426646342 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561acf83a7e0 con 0x561acfaf80a0 2024-03-28T18:07:23.132 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:23.132+0000 7ffb6fe00640 1 --2- 172.21.15.22:0/2562049588 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7ffb5c0a2800 0x7ffb5c0a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:07:23.132 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:23.132+0000 7ffb76a00640 1 --2- 172.21.15.22:0/2426646342 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561acfaf80a0 0x561acfaf84c0 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7ffb64009fa0 tx=0x7ffb640097b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=449bd5da937beda2 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:07:23.136 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:23.132+0000 7ffb6fe00640 1 -- 172.21.15.22:0/2562049588 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 8153+0+0 (secure 0 0 0) 0x7ffb700cea90 con 0x7ffb78102190 2024-03-28T18:07:23.136 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:23.132+0000 7ffb77400640 1 --2- 172.21.15.22:0/2562049588 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7ffb5c0a2800 0x7ffb5c0a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:07:23.136 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:23.132+0000 7ffb6cc00640 1 -- 172.21.15.22:0/2426646342 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb64010490 con 0x561acfaf80a0 2024-03-28T18:07:23.136 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:23.132+0000 7ffb6cc00640 1 -- 172.21.15.22:0/2426646342 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7ffb64010b00 con 0x561acfaf80a0 2024-03-28T18:07:23.136 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:23.132+0000 7ffb6cc00640 1 -- 172.21.15.22:0/2426646342 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb6400f970 con 0x561acfaf80a0 2024-03-28T18:07:23.136 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:23.133+0000 7ffb81571740 1 -- 172.21.15.22:0/2426646342 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561acfaf80a0 msgr2=0x561acfaf84c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:07:23.136 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:23.133+0000 7ffb81571740 1 --2- 172.21.15.22:0/2426646342 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561acfaf80a0 0x561acfaf84c0 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7ffb64009fa0 tx=0x7ffb640097b0 comp rx=0 tx=0).stop 2024-03-28T18:07:23.136 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:23.133+0000 7ffb77400640 1 --2- 172.21.15.22:0/2562049588 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7ffb5c0a2800 0x7ffb5c0a4cc0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7ffb6802c170 tx=0x7ffb6802c820 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:07:23.136 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:23.133+0000 7ffb81571740 1 -- 172.21.15.22:0/2426646342 shutdown_connections 2024-03-28T18:07:23.136 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:23.133+0000 7ffb81571740 1 --2- 172.21.15.22:0/2426646342 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x561acfafdc00 0x561acfb00090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:07:23.136 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:23.133+0000 7ffb81571740 1 --2- 172.21.15.22:0/2426646342 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561acfaf8a90 0x561acfafd590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:07:23.136 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:23.133+0000 7ffb81571740 1 --2- 172.21.15.22:0/2426646342 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561acfaf80a0 0x561acfaf84c0 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:07:23.137 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:23.133+0000 7ffb81571740 1 -- 172.21.15.22:0/2426646342 >> 172.21.15.22:0/2426646342 conn(0x561acfaf3610 msgr2=0x561acfaf3a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:07:23.137 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:23.133+0000 7ffb81571740 1 -- 172.21.15.22:0/2426646342 shutdown_connections 2024-03-28T18:07:23.137 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:23.133+0000 7ffb81571740 1 -- 172.21.15.22:0/2426646342 wait complete. 2024-03-28T18:07:23.137 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:23.134+0000 7ffb81571740 1 -- start start 2024-03-28T18:07:23.137 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:23.134+0000 7ffb81571740 1 Processor -- start 2024-03-28T18:07:23.137 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:23.134+0000 7ffb81571740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561acfaf80a0 0x561acfb022b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:07:23.137 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:23.134+0000 7ffb81571740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x561acfaf8a90 0x561acfb027f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:07:23.137 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:23.134+0000 7ffb81571740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561acfafdc00 0x561acfb09830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:07:23.137 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:23.134+0000 7ffb81571740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x561acfb00bb0 con 0x561acfafdc00 2024-03-28T18:07:23.137 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:23.134+0000 7ffb81571740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x561acfa19eb0 con 0x561acfaf8a90 2024-03-28T18:07:23.137 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:23.134+0000 7ffb81571740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x561acfb00d30 con 0x561acfaf80a0 2024-03-28T18:07:23.137 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:23.135+0000 7ffb81571740 10 client.0 _ll_register_callbacks cb 0x561acfa0e150 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-28T18:07:23.137 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:23.135+0000 7ffb77e00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561acfafdc00 0x561acfb09830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:07:23.137 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:23.135+0000 7ffb77e00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561acfafdc00 0x561acfb09830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:55664/0 (socket says 172.21.15.22:55664) 2024-03-28T18:07:23.137 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:23.135+0000 7ffb77e00640 1 -- 172.21.15.22:0/3889246515 learned_addr learned my addr 172.21.15.22:0/3889246515 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-28T18:07:23.138 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:23.135+0000 7ffb76a00640 1 --2- 172.21.15.22:0/3889246515 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x561acfaf8a90 0x561acfb027f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:07:23.138 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:23.135+0000 7ffb77400640 1 --2- 172.21.15.22:0/3889246515 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561acfaf80a0 0x561acfb022b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:07:23.138 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:23.135+0000 7ffb77e00640 1 -- 172.21.15.22:0/3889246515 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561acfaf80a0 msgr2=0x561acfb022b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T18:07:23.138 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:23.135+0000 7ffb77e00640 1 --2- 172.21.15.22:0/3889246515 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561acfaf80a0 0x561acfb022b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:07:23.138 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:23.135+0000 7ffb77e00640 1 -- 172.21.15.22:0/3889246515 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x561acfaf8a90 msgr2=0x561acfb027f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T18:07:23.138 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:23.135+0000 7ffb77e00640 1 --2- 172.21.15.22:0/3889246515 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x561acfaf8a90 0x561acfb027f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:07:23.138 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:23.135+0000 7ffb77e00640 1 -- 172.21.15.22:0/3889246515 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x561acf89af70 con 0x561acfafdc00 2024-03-28T18:07:23.138 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:23.135+0000 7ffb77400640 1 --2- 172.21.15.22:0/3889246515 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561acfaf80a0 0x561acfb022b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T18:07:23.138 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:23.135+0000 7ffb76a00640 1 --2- 172.21.15.22:0/3889246515 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x561acfaf8a90 0x561acfb027f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-28T18:07:23.138 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:23.135+0000 7ffb77e00640 1 --2- 172.21.15.22:0/3889246515 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561acfafdc00 0x561acfb09830 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7ffb70097e00 tx=0x7ffb700055d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9e7e0d04d11bc090 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:07:23.138 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:23.135+0000 7ffb61600640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-03-28T18:07:23.138 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:23.136+0000 7ffb61600640 1 -- 172.21.15.22:0/3889246515 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb7001d030 con 0x561acfafdc00 2024-03-28T18:07:23.138 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:23.136+0000 7ffb81571740 1 -- 172.21.15.22:0/3889246515 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7ffb64010b00 con 0x561acfafdc00 2024-03-28T18:07:23.138 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:23.136+0000 7ffb61600640 1 -- 172.21.15.22:0/3889246515 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 8153+0+0 (secure 0 0 0) 0x7ffb7009f320 con 0x561acfafdc00 2024-03-28T18:07:23.138 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:23.136+0000 7ffb61600640 1 -- 172.21.15.22:0/3889246515 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7ffb7009f580 con 0x561acfafdc00 2024-03-28T18:07:23.139 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:23.136+0000 7ffb81571740 10 client.15492 fetch_fsmap learned FSMap version 27 2024-03-28T18:07:23.139 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:23.136+0000 7ffb81571740 1 -- 172.21.15.22:0/3889246515 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x561acf732170 con 0x561acfafdc00 2024-03-28T18:07:23.139 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:23.136+0000 7ffb61600640 1 -- 172.21.15.22:0/3889246515 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7ffb64010b00 con 0x561acfafdc00 2024-03-28T18:07:23.139 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:23.136+0000 7ffb61600640 1 -- 172.21.15.22:0/3889246515 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 27) v1 ==== 7934+0+0 (secure 0 0 0) 0x7ffb70097180 con 0x561acfafdc00 2024-03-28T18:07:23.140 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:23.136+0000 7ffb81571740 10 client.15492 fetch_fsmap finished waiting for FSMap version 27 2024-03-28T18:07:23.140 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:23.136+0000 7ffb81571740 10 client.15492 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-28T18:07:23.140 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:23.137+0000 7ffb81571740 20 client.15492 populate_metadata read hostname 'smithi022' 2024-03-28T18:07:23.141 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:23.137+0000 7ffb81571740 1 --2- 172.21.15.22:0/3889246515 >> [v2:172.21.15.79:6838/2441542443,v1:172.21.15.79:6839/2441542443] conn(0x561acfb0e140 0x561acfb10590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:07:23.141 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:23.137+0000 7ffb81571740 4 client.15492 mds_command: new command op to 34553 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-28T18:07:23.141 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:23.137+0000 7ffb81571740 1 -- 172.21.15.22:0/3889246515 --> [v2:172.21.15.79:6838/2441542443,v1:172.21.15.79:6839/2441542443] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x561acfa6b3f0 con 0x561acfb0e140 2024-03-28T18:07:23.141 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:23.137+0000 7ffb77400640 1 --2- 172.21.15.22:0/3889246515 >> [v2:172.21.15.79:6838/2441542443,v1:172.21.15.79:6839/2441542443] conn(0x561acfb0e140 0x561acfb10590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:07:23.141 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:23.138+0000 7ffb77400640 1 --2- 172.21.15.22:0/3889246515 >> [v2:172.21.15.79:6838/2441542443,v1:172.21.15.79:6839/2441542443] conn(0x561acfb0e140 0x561acfb10590 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=36a7617d73462223 server_cookie=3a8bb462e9b18cf8 in_seq=0 out_seq=0 2024-03-28T18:07:23.141 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:23.138+0000 7ffb61600640 10 client.15492 ms_handle_connect on v2:172.21.15.79:6838/2441542443 2024-03-28T18:07:23.141 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:23.139+0000 7ffb61600640 1 -- 172.21.15.22:0/3889246515 <== mds.0 v2:172.21.15.79:6838/2441542443 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x561acfa6b3f0 con 0x561acfb0e140 2024-03-28T18:07:23.141 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:23.139+0000 7ffb61600640 10 client.15492 handle_command_reply: tid=0 2024-03-28T18:07:23.141 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:23.139+0000 7ffb81571740 1 client.15492 shutdown 2024-03-28T18:07:23.141 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:23.139+0000 7ffb81571740 1 -- 172.21.15.22:0/3889246515 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561acfafdc00 msgr2=0x561acfb09830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:07:23.141 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:23.139+0000 7ffb81571740 1 --2- 172.21.15.22:0/3889246515 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561acfafdc00 0x561acfb09830 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7ffb70097e00 tx=0x7ffb700055d0 comp rx=0 tx=0).stop 2024-03-28T18:07:23.141 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:23.139+0000 7ffb81571740 1 -- 172.21.15.22:0/3889246515 shutdown_connections 2024-03-28T18:07:23.141 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:23.139+0000 7ffb81571740 1 --2- 172.21.15.22:0/3889246515 >> [v2:172.21.15.79:6838/2441542443,v1:172.21.15.79:6839/2441542443] conn(0x561acfb0e140 0x561acfb10590 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:07:23.141 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:23.139+0000 7ffb81571740 1 --2- 172.21.15.22:0/3889246515 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561acfafdc00 0x561acfb09830 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:07:23.141 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:23.139+0000 7ffb81571740 1 --2- 172.21.15.22:0/3889246515 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x561acfaf8a90 0x561acfb027f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:07:23.142 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:23.139+0000 7ffb81571740 1 --2- 172.21.15.22:0/3889246515 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561acfaf80a0 0x561acfb022b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:07:23.142 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:23.139+0000 7ffb61600640 0 client.15492 ms_handle_reset on v2:172.21.15.79:6838/2441542443 2024-03-28T18:07:23.142 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:23.139+0000 7ffb81571740 1 -- 172.21.15.22:0/3889246515 >> 172.21.15.22:0/3889246515 conn(0x561acfaf3610 msgr2=0x561acfaf2830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:07:23.142 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:23.139+0000 7ffb81571740 1 -- 172.21.15.22:0/3889246515 shutdown_connections 2024-03-28T18:07:23.142 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:23.140+0000 7ffb81571740 1 -- 172.21.15.22:0/3889246515 wait complete. 2024-03-28T18:07:23.142 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:23.140+0000 7ffb81571740 20 client.15492 trim_cache size 0 max 16384 2024-03-28T18:07:23.158 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:23.157+0000 7ffb81571740 1 Processor -- start 2024-03-28T18:07:23.158 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:23.157+0000 7ffb81571740 1 -- start start 2024-03-28T18:07:23.158 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:23.158+0000 7ffb81571740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x561acfaf80a0 0x561acfaf9500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:07:23.158 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:23.158+0000 7ffb81571740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561acfaf8a90 0x561acfb097f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:07:23.158 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:23.158+0000 7ffb81571740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561acfafdc00 0x561acfaf9a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:07:23.158 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:23.158+0000 7ffb81571740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb781aaab0 con 0x561acfaf8a90 2024-03-28T18:07:23.158 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:23.158+0000 7ffb81571740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb781ac430 con 0x561acfaf80a0 2024-03-28T18:07:23.158 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:23.158+0000 7ffb81571740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb781ab110 con 0x561acfafdc00 2024-03-28T18:07:23.158 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:23.158+0000 7ffb76a00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561acfaf8a90 0x561acfb097f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:07:23.159 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:23.158+0000 7ffb76a00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561acfaf8a90 0x561acfb097f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:55672/0 (socket says 172.21.15.22:55672) 2024-03-28T18:07:23.159 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:23.158+0000 7ffb76a00640 1 -- 172.21.15.22:0/2468143461 learned_addr learned my addr 172.21.15.22:0/2468143461 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-28T18:07:23.159 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:23.158+0000 7ffb77400640 1 --2- 172.21.15.22:0/2468143461 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x561acfaf80a0 0x561acfaf9500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:07:23.159 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:23.158+0000 7ffb77e00640 1 --2- 172.21.15.22:0/2468143461 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561acfafdc00 0x561acfaf9a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:07:23.159 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:23.158+0000 7ffb76a00640 1 -- 172.21.15.22:0/2468143461 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561acfafdc00 msgr2=0x561acfaf9a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T18:07:23.159 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:23.158+0000 7ffb76a00640 1 --2- 172.21.15.22:0/2468143461 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561acfafdc00 0x561acfaf9a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:07:23.159 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:23.158+0000 7ffb76a00640 1 -- 172.21.15.22:0/2468143461 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x561acfaf80a0 msgr2=0x561acfaf9500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T18:07:23.159 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:23.158+0000 7ffb76a00640 1 --2- 172.21.15.22:0/2468143461 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x561acfaf80a0 0x561acfaf9500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:07:23.159 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:23.158+0000 7ffb76a00640 1 -- 172.21.15.22:0/2468143461 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561acfaeec20 con 0x561acfaf8a90 2024-03-28T18:07:23.159 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:23.158+0000 7ffb77e00640 1 --2- 172.21.15.22:0/2468143461 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561acfafdc00 0x561acfaf9a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T18:07:23.162 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:23.159+0000 7ffb76a00640 1 --2- 172.21.15.22:0/2468143461 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561acfaf8a90 0x561acfb097f0 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7ffb64009fa0 tx=0x7ffb640098f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dcf4e91832165db8 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:07:23.162 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:23.159+0000 7ffb63e00640 1 -- 172.21.15.22:0/2468143461 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb640130d0 con 0x561acfaf8a90 2024-03-28T18:07:23.162 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:23.159+0000 7ffb63e00640 1 -- 172.21.15.22:0/2468143461 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7ffb64005570 con 0x561acfaf8a90 2024-03-28T18:07:23.162 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:23.159+0000 7ffb63e00640 1 -- 172.21.15.22:0/2468143461 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb64003e50 con 0x561acfaf8a90 2024-03-28T18:07:23.162 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:23.160+0000 7ffb81571740 1 -- 172.21.15.22:0/2468143461 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561acfaf8a90 msgr2=0x561acfb097f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:07:23.162 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:23.160+0000 7ffb81571740 1 --2- 172.21.15.22:0/2468143461 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561acfaf8a90 0x561acfb097f0 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7ffb64009fa0 tx=0x7ffb640098f0 comp rx=0 tx=0).stop 2024-03-28T18:07:23.162 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:23.160+0000 7ffb81571740 1 -- 172.21.15.22:0/2468143461 shutdown_connections 2024-03-28T18:07:23.162 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:23.160+0000 7ffb81571740 1 --2- 172.21.15.22:0/2468143461 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561acfafdc00 0x561acfaf9a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:07:23.162 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:23.160+0000 7ffb81571740 1 --2- 172.21.15.22:0/2468143461 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561acfaf8a90 0x561acfb097f0 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:07:23.162 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:23.160+0000 7ffb81571740 1 --2- 172.21.15.22:0/2468143461 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x561acfaf80a0 0x561acfaf9500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:07:23.162 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:23.160+0000 7ffb81571740 1 -- 172.21.15.22:0/2468143461 >> 172.21.15.22:0/2468143461 conn(0x561acfaf3610 msgr2=0x561acfaf7100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:07:23.162 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:23.160+0000 7ffb81571740 1 -- 172.21.15.22:0/2468143461 shutdown_connections 2024-03-28T18:07:23.162 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:23.160+0000 7ffb81571740 1 -- 172.21.15.22:0/2468143461 wait complete. 2024-03-28T18:07:23.163 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:23.160+0000 7ffb81571740 1 -- start start 2024-03-28T18:07:23.165 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:23.161+0000 7ffb81571740 1 Processor -- start 2024-03-28T18:07:23.165 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:23.161+0000 7ffb81571740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561acfaf80a0 0x561acfb074f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:07:23.165 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:23.161+0000 7ffb81571740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x561acfaf8a90 0x561acfb07a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:07:23.165 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:23.161+0000 7ffb81571740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561acfafdc00 0x561acfb08a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:07:23.165 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:23.161+0000 7ffb81571740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb781abdd0 con 0x561acfafdc00 2024-03-28T18:07:23.166 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:23.161+0000 7ffb81571740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb781ab770 con 0x561acfaf8a90 2024-03-28T18:07:23.166 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:23.161+0000 7ffb81571740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb781a8c70 con 0x561acfaf80a0 2024-03-28T18:07:23.166 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:23.161+0000 7ffb81571740 10 client.0 _ll_register_callbacks cb 0x7ffb700054c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-28T18:07:23.166 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:23.161+0000 7ffb77e00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561acfafdc00 0x561acfb08a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:07:23.166 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:23.161+0000 7ffb77e00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561acfafdc00 0x561acfb08a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:55688/0 (socket says 172.21.15.22:55688) 2024-03-28T18:07:23.166 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:23.161+0000 7ffb77e00640 1 -- 172.21.15.22:0/1730127637 learned_addr learned my addr 172.21.15.22:0/1730127637 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-28T18:07:23.166 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:23.161+0000 7ffb76a00640 1 --2- 172.21.15.22:0/1730127637 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x561acfaf8a90 0x561acfb07a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:07:23.166 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:23.162+0000 7ffb77400640 1 --2- 172.21.15.22:0/1730127637 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561acfaf80a0 0x561acfb074f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:07:23.166 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:23.162+0000 7ffb77e00640 1 -- 172.21.15.22:0/1730127637 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561acfaf80a0 msgr2=0x561acfb074f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T18:07:23.166 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:23.162+0000 7ffb77e00640 1 --2- 172.21.15.22:0/1730127637 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561acfaf80a0 0x561acfb074f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:07:23.166 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:23.162+0000 7ffb77e00640 1 -- 172.21.15.22:0/1730127637 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x561acfaf8a90 msgr2=0x561acfb07a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T18:07:23.166 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:23.162+0000 7ffb77e00640 1 --2- 172.21.15.22:0/1730127637 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x561acfaf8a90 0x561acfb07a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:07:23.166 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:23.162+0000 7ffb77e00640 1 -- 172.21.15.22:0/1730127637 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x561acfb09030 con 0x561acfafdc00 2024-03-28T18:07:23.166 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:23.162+0000 7ffb76a00640 1 --2- 172.21.15.22:0/1730127637 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x561acfaf8a90 0x561acfb07a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-28T18:07:23.167 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:23.162+0000 7ffb77e00640 1 --2- 172.21.15.22:0/1730127637 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561acfafdc00 0x561acfb08a40 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7ffb7009b130 tx=0x7ffb7009a4e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c87a7b46a0801402 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:07:23.167 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:23.162+0000 7ffb61600640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-03-28T18:07:23.169 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:23.162+0000 7ffb61600640 1 -- 172.21.15.22:0/1730127637 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb7001d070 con 0x561acfafdc00 2024-03-28T18:07:23.169 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:23.163+0000 7ffb61600640 1 -- 172.21.15.22:0/1730127637 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 8153+0+0 (secure 0 0 0) 0x7ffb700bb300 con 0x561acfafdc00 2024-03-28T18:07:23.169 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:23.163+0000 7ffb81571740 1 -- 172.21.15.22:0/1730127637 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7ffb64005570 con 0x561acfafdc00 2024-03-28T18:07:23.169 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:23.163+0000 7ffb61600640 1 -- 172.21.15.22:0/1730127637 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7ffb700976d0 con 0x561acfafdc00 2024-03-28T18:07:23.169 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:23.163+0000 7ffb81571740 10 client.15498 fetch_fsmap learned FSMap version 27 2024-03-28T18:07:23.169 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:23.163+0000 7ffb81571740 1 -- 172.21.15.22:0/1730127637 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x561acfb69fd0 con 0x561acfafdc00 2024-03-28T18:07:23.169 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:23.163+0000 7ffb61600640 1 -- 172.21.15.22:0/1730127637 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7ffb64005570 con 0x561acfafdc00 2024-03-28T18:07:23.169 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:23.163+0000 7ffb61600640 1 -- 172.21.15.22:0/1730127637 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 27) v1 ==== 7934+0+0 (secure 0 0 0) 0x7ffb7007c030 con 0x561acfafdc00 2024-03-28T18:07:23.169 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:23.163+0000 7ffb81571740 10 client.15498 fetch_fsmap finished waiting for FSMap version 27 2024-03-28T18:07:23.169 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:23.163+0000 7ffb81571740 10 client.15498 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-28T18:07:23.169 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:23.163+0000 7ffb81571740 20 client.15498 populate_metadata read hostname 'smithi022' 2024-03-28T18:07:23.169 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:23.163+0000 7ffb81571740 1 --2- 172.21.15.22:0/1730127637 >> [v2:172.21.15.79:6838/2441542443,v1:172.21.15.79:6839/2441542443] conn(0x561acfb6a880 0x561acfb6cce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:07:23.169 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:23.163+0000 7ffb81571740 4 client.15498 mds_command: new command op to 34553 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-03-28T18:07:23.169 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:23.163+0000 7ffb81571740 1 -- 172.21.15.22:0/1730127637 --> [v2:172.21.15.79:6838/2441542443,v1:172.21.15.79:6839/2441542443] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x561acfa65740 con 0x561acfb6a880 2024-03-28T18:07:23.170 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:23.164+0000 7ffb77400640 1 --2- 172.21.15.22:0/1730127637 >> [v2:172.21.15.79:6838/2441542443,v1:172.21.15.79:6839/2441542443] conn(0x561acfb6a880 0x561acfb6cce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:07:23.170 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:23.164+0000 7ffb77400640 1 --2- 172.21.15.22:0/1730127637 >> [v2:172.21.15.79:6838/2441542443,v1:172.21.15.79:6839/2441542443] conn(0x561acfb6a880 0x561acfb6cce0 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=26b86e6991d09203 server_cookie=5d57e2d7747d3f23 in_seq=0 out_seq=0 2024-03-28T18:07:23.170 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:23.164+0000 7ffb61600640 10 client.15498 ms_handle_connect on v2:172.21.15.79:6838/2441542443 2024-03-28T18:07:23.170 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:23.166+0000 7ffb61600640 1 -- 172.21.15.22:0/1730127637 <== mds.0 v2:172.21.15.79:6838/2441542443 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x561acfa65740 con 0x561acfb6a880 2024-03-28T18:07:23.170 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:23.166+0000 7ffb61600640 10 client.15498 handle_command_reply: tid=0 2024-03-28T18:07:23.170 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:23.166+0000 7ffb81571740 1 client.15498 shutdown 2024-03-28T18:07:23.170 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:23.167+0000 7ffb81571740 1 -- 172.21.15.22:0/1730127637 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561acfafdc00 msgr2=0x561acfb08a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:07:23.170 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:23.167+0000 7ffb81571740 1 --2- 172.21.15.22:0/1730127637 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561acfafdc00 0x561acfb08a40 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7ffb7009b130 tx=0x7ffb7009a4e0 comp rx=0 tx=0).stop 2024-03-28T18:07:23.170 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:23.167+0000 7ffb81571740 1 -- 172.21.15.22:0/1730127637 shutdown_connections 2024-03-28T18:07:23.170 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:23.167+0000 7ffb81571740 1 --2- 172.21.15.22:0/1730127637 >> [v2:172.21.15.79:6838/2441542443,v1:172.21.15.79:6839/2441542443] conn(0x561acfb6a880 0x561acfb6cce0 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:07:23.170 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:23.167+0000 7ffb81571740 1 --2- 172.21.15.22:0/1730127637 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561acfafdc00 0x561acfb08a40 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:07:23.170 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:23.167+0000 7ffb81571740 1 --2- 172.21.15.22:0/1730127637 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x561acfaf8a90 0x561acfb07a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:07:23.170 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:23.167+0000 7ffb81571740 1 --2- 172.21.15.22:0/1730127637 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561acfaf80a0 0x561acfb074f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:07:23.170 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:23.167+0000 7ffb81571740 1 -- 172.21.15.22:0/1730127637 >> 172.21.15.22:0/1730127637 conn(0x561acfaf3610 msgr2=0x561acfaf1fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:07:23.170 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:23.167+0000 7ffb61600640 0 client.15498 ms_handle_reset on v2:172.21.15.79:6838/2441542443 2024-03-28T18:07:23.171 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:23.167+0000 7ffb81571740 1 -- 172.21.15.22:0/1730127637 shutdown_connections 2024-03-28T18:07:23.171 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:23.167+0000 7ffb81571740 1 -- 172.21.15.22:0/1730127637 wait complete. 2024-03-28T18:07:23.171 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:23.167+0000 7ffb81571740 20 client.15498 trim_cache size 0 max 16384 2024-03-28T18:07:23.171 INFO:teuthology.orchestra.run.smithi022.stdout:{ 2024-03-28T18:07:23.171 INFO:teuthology.orchestra.run.smithi022.stdout: "return_code": 0, 2024-03-28T18:07:23.171 INFO:teuthology.orchestra.run.smithi022.stdout: "scrub_tag": "c652d562-2df9-4f99-a9ab-11652cd9ac08", 2024-03-28T18:07:23.171 INFO:teuthology.orchestra.run.smithi022.stdout: "mode": "asynchronous" 2024-03-28T18:07:23.171 INFO:teuthology.orchestra.run.smithi022.stdout:} 2024-03-28T18:07:23.172 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:23.168+0000 7ffb7e800640 1 -- 172.21.15.22:0/2562049588 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7ffb5c0a2800 msgr2=0x7ffb5c0a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:07:23.172 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:23.168+0000 7ffb7e800640 1 --2- 172.21.15.22:0/2562049588 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7ffb5c0a2800 0x7ffb5c0a4cc0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7ffb6802c170 tx=0x7ffb6802c820 comp rx=0 tx=0).stop 2024-03-28T18:07:23.172 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:23.168+0000 7ffb7e800640 1 -- 172.21.15.22:0/2562049588 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ffb78102190 msgr2=0x7ffb781a1230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:07:23.172 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:23.168+0000 7ffb7e800640 1 --2- 172.21.15.22:0/2562049588 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ffb78102190 0x7ffb781a1230 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7ffb7000cad0 tx=0x7ffb7000cf90 comp rx=0 tx=0).stop 2024-03-28T18:07:23.172 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:23.168+0000 7ffb7e800640 1 -- 172.21.15.22:0/2562049588 shutdown_connections 2024-03-28T18:07:23.172 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:23.168+0000 7ffb7e800640 1 --2- 172.21.15.22:0/2562049588 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7ffb5c0a2800 0x7ffb5c0a4cc0 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:07:23.172 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:23.168+0000 7ffb7e800640 1 --2- 172.21.15.22:0/2562049588 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ffb78102190 0x7ffb781a1230 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:07:23.172 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:23.168+0000 7ffb7e800640 1 --2- 172.21.15.22:0/2562049588 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ffb781017d0 0x7ffb7819a1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:07:23.172 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:23.168+0000 7ffb7e800640 1 --2- 172.21.15.22:0/2562049588 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ffb78100ac0 0x7ffb78199c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:07:23.172 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:23.168+0000 7ffb7e800640 1 -- 172.21.15.22:0/2562049588 >> 172.21.15.22:0/2562049588 conn(0x7ffb780fb290 msgr2=0x7ffb780fcdc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:07:23.173 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:23.168+0000 7ffb7e800640 1 -- 172.21.15.22:0/2562049588 shutdown_connections 2024-03-28T18:07:23.173 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:23.168+0000 7ffb7e800640 1 -- 172.21.15.22:0/2562049588 wait complete. 2024-03-28T18:07:23.174 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:07:22 smithi134 ceph-mon[35037]: from='client.? 172.21.15.22:0/2388986592' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-28T18:07:23.174 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:07:22 smithi134 ceph-mon[35037]: from='client.? 172.21.15.22:0/2735544693' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-28T18:07:23.182 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:07:22 smithi079 ceph-mon[35312]: from='client.? 172.21.15.22:0/2388986592' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-28T18:07:23.182 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:07:22 smithi079 ceph-mon[35312]: from='client.? 172.21.15.22:0/2735544693' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-28T18:07:23.185 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-28T18:07:23.185 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-28T18:07:23.386 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:23.386+0000 7f2d1f200640 1 -- 172.21.15.22:0/662263897 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "snap_schedule"} v 0) v1 -- 0x7f2d180622a0 con 0x7f2d18074100 2024-03-28T18:07:23.452 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:23.452+0000 7f44ec800640 1 -- 172.21.15.22:0/3187377521 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f44e4076060 msgr2=0x7f44e4076480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:07:23.452 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:23.452+0000 7f44ec800640 1 --2- 172.21.15.22:0/3187377521 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f44e4076060 0x7f44e4076480 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f44d8009900 tx=0x7f44d802bc30 comp rx=0 tx=0).stop 2024-03-28T18:07:23.452 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:23.452+0000 7f44ec800640 1 -- 172.21.15.22:0/3187377521 shutdown_connections 2024-03-28T18:07:23.453 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:23.452+0000 7f44ec800640 1 --2- 172.21.15.22:0/3187377521 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f44e4077c40 0x7f44e40784e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:07:23.453 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:23.452+0000 7f44ec800640 1 --2- 172.21.15.22:0/3187377521 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f44e40772a0 0x7f44e4077700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:07:23.453 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:23.452+0000 7f44ec800640 1 --2- 172.21.15.22:0/3187377521 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f44e4076060 0x7f44e4076480 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:07:23.453 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:23.452+0000 7f44ec800640 1 -- 172.21.15.22:0/3187377521 >> 172.21.15.22:0/3187377521 conn(0x7f44e40fee50 msgr2=0x7f44e4101290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:07:23.453 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:23.453+0000 7f44ec800640 1 -- 172.21.15.22:0/3187377521 shutdown_connections 2024-03-28T18:07:23.453 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:23.453+0000 7f44ec800640 1 -- 172.21.15.22:0/3187377521 wait complete. 2024-03-28T18:07:23.454 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:23.453+0000 7f44ec800640 1 Processor -- start 2024-03-28T18:07:23.454 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:23.454+0000 7f44ec800640 1 -- start start 2024-03-28T18:07:23.454 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:23.454+0000 7f44ec800640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f44e4076060 0x7f44e419dff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:07:23.454 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:23.454+0000 7f44ec800640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f44e40772a0 0x7f44e419e530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:07:23.454 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:23.454+0000 7f44ec800640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f44e4077c40 0x7f44e41a55a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:07:23.454 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:23.454+0000 7f44ec800640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44e4113980 con 0x7f44e40772a0 2024-03-28T18:07:23.454 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:23.454+0000 7f44ec800640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44e4113800 con 0x7f44e4077c40 2024-03-28T18:07:23.455 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:23.454+0000 7f44ec800640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44e4113b00 con 0x7f44e4076060 2024-03-28T18:07:23.455 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:23.454+0000 7f44e9400640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f44e40772a0 0x7f44e419e530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:07:23.455 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:23.454+0000 7f44e9400640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f44e40772a0 0x7f44e419e530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:55712/0 (socket says 172.21.15.22:55712) 2024-03-28T18:07:23.455 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:23.454+0000 7f44e9400640 1 -- 172.21.15.22:0/1337294189 learned_addr learned my addr 172.21.15.22:0/1337294189 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-28T18:07:23.455 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:23.454+0000 7f44ea800640 1 --2- 172.21.15.22:0/1337294189 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f44e4077c40 0x7f44e41a55a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:07:23.455 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:23.455+0000 7f44e9e00640 1 --2- 172.21.15.22:0/1337294189 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f44e4076060 0x7f44e419dff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:07:23.456 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:23.455+0000 7f44e9400640 1 -- 172.21.15.22:0/1337294189 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f44e4076060 msgr2=0x7f44e419dff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:07:23.456 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:23.455+0000 7f44e9400640 1 --2- 172.21.15.22:0/1337294189 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f44e4076060 0x7f44e419dff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:07:23.456 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:23.455+0000 7f44e9400640 1 -- 172.21.15.22:0/1337294189 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f44e4077c40 msgr2=0x7f44e41a55a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:07:23.456 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:23.455+0000 7f44e9400640 1 --2- 172.21.15.22:0/1337294189 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f44e4077c40 0x7f44e41a55a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:07:23.456 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:23.455+0000 7f44e9400640 1 -- 172.21.15.22:0/1337294189 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f44e41a5c10 con 0x7f44e40772a0 2024-03-28T18:07:23.456 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:23.455+0000 7f44e9400640 1 --2- 172.21.15.22:0/1337294189 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f44e40772a0 0x7f44e419e530 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f44d400d770 tx=0x7f44d400dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:07:23.456 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:23.455+0000 7f44dea00640 1 -- 172.21.15.22:0/1337294189 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44d4017070 con 0x7f44e40772a0 2024-03-28T18:07:23.456 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:23.455+0000 7f44dea00640 1 -- 172.21.15.22:0/1337294189 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f44d40054f0 con 0x7f44e40772a0 2024-03-28T18:07:23.456 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:23.456+0000 7f44dea00640 1 -- 172.21.15.22:0/1337294189 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44d4004d60 con 0x7f44e40772a0 2024-03-28T18:07:23.456 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:23.456+0000 7f44ec800640 1 -- 172.21.15.22:0/1337294189 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f44e41a5f00 con 0x7f44e40772a0 2024-03-28T18:07:23.456 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:23.456+0000 7f44ec800640 1 -- 172.21.15.22:0/1337294189 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f44e41a63c0 con 0x7f44e40772a0 2024-03-28T18:07:23.459 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:23.460+0000 7f44ef453740 1 Processor -- start 2024-03-28T18:07:23.460 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:23.460+0000 7f44ef453740 1 -- start start 2024-03-28T18:07:23.461 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:23.460+0000 7f44dea00640 1 -- 172.21.15.22:0/1337294189 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 18) v1 ==== 137290+0+0 (secure 0 0 0) 0x7f44d4004f00 con 0x7f44e40772a0 2024-03-28T18:07:23.461 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:23.461+0000 7f44ef453740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55ae865ede80 0x55ae865ee2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:07:23.461 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:23.461+0000 7f44ef453740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ae865ee870 0x55ae865f3370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:07:23.461 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:23.461+0000 7f44ef453740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ae865f39e0 0x55ae865f5e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:07:23.461 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:23.461+0000 7f44ef453740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ae86553250 con 0x55ae865ee870 2024-03-28T18:07:23.461 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:23.461+0000 7f44ef453740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ae864fba30 con 0x55ae865ede80 2024-03-28T18:07:23.461 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:23.461+0000 7f44ef453740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ae8654f2f0 con 0x55ae865f39e0 2024-03-28T18:07:23.462 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:23.461+0000 7f44e9e00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ae865ee870 0x55ae865f3370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:07:23.462 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:23.461+0000 7f44e9e00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ae865ee870 0x55ae865f3370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:55714/0 (socket says 172.21.15.22:55714) 2024-03-28T18:07:23.462 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:23.461+0000 7f44e9e00640 1 -- 172.21.15.22:0/4214469169 learned_addr learned my addr 172.21.15.22:0/4214469169 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-28T18:07:23.462 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:23.461+0000 7f44ea800640 1 --2- 172.21.15.22:0/4214469169 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55ae865ede80 0x55ae865ee2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:07:23.462 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:23.461+0000 7f44e9400640 1 --2- 172.21.15.22:0/4214469169 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ae865f39e0 0x55ae865f5e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:07:23.462 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:23.461+0000 7f44e9e00640 1 -- 172.21.15.22:0/4214469169 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ae865f39e0 msgr2=0x55ae865f5e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T18:07:23.462 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:23.461+0000 7f44e9e00640 1 --2- 172.21.15.22:0/4214469169 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ae865f39e0 0x55ae865f5e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:07:23.462 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:23.462+0000 7f44e9e00640 1 -- 172.21.15.22:0/4214469169 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55ae865ede80 msgr2=0x55ae865ee2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T18:07:23.462 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:23.462+0000 7f44e9e00640 1 --2- 172.21.15.22:0/4214469169 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55ae865ede80 0x55ae865ee2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:07:23.462 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:23.462+0000 7f44e9e00640 1 -- 172.21.15.22:0/4214469169 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ae864e6690 con 0x55ae865ee870 2024-03-28T18:07:23.464 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:23.462+0000 7f44e9e00640 1 --2- 172.21.15.22:0/4214469169 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ae865ee870 0x55ae865f3370 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f44d802c240 tx=0x7f44d8005d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f1aa69d18f6c9515 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:07:23.464 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:23.462+0000 7f44d3000640 1 -- 172.21.15.22:0/4214469169 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44d8044020 con 0x55ae865ee870 2024-03-28T18:07:23.464 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:23.462+0000 7f44dea00640 1 --2- 172.21.15.22:0/1337294189 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7f44cc0a2880 0x7f44cc0a4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:07:23.464 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:23.462+0000 7f44d3000640 1 -- 172.21.15.22:0/4214469169 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f44d802cba0 con 0x55ae865ee870 2024-03-28T18:07:23.464 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:23.462+0000 7f44d3000640 1 -- 172.21.15.22:0/4214469169 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44d8004920 con 0x55ae865ee870 2024-03-28T18:07:23.464 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:23.463+0000 7f44dea00640 1 -- 172.21.15.22:0/1337294189 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 8153+0+0 (secure 0 0 0) 0x7f44d40ce400 con 0x7f44e40772a0 2024-03-28T18:07:23.464 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:23.463+0000 7f44ea800640 1 --2- 172.21.15.22:0/1337294189 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7f44cc0a2880 0x7f44cc0a4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:07:23.464 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:23.463+0000 7f44ef453740 1 -- 172.21.15.22:0/4214469169 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ae865ee870 msgr2=0x55ae865f3370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:07:23.464 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:23.463+0000 7f44ef453740 1 --2- 172.21.15.22:0/4214469169 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ae865ee870 0x55ae865f3370 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f44d802c240 tx=0x7f44d8005d50 comp rx=0 tx=0).stop 2024-03-28T18:07:23.464 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:23.463+0000 7f44ef453740 1 -- 172.21.15.22:0/4214469169 shutdown_connections 2024-03-28T18:07:23.464 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:23.463+0000 7f44ef453740 1 --2- 172.21.15.22:0/4214469169 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ae865f39e0 0x55ae865f5e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:07:23.464 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:23.463+0000 7f44ef453740 1 --2- 172.21.15.22:0/4214469169 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ae865ee870 0x55ae865f3370 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:07:23.464 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:23.463+0000 7f44ef453740 1 --2- 172.21.15.22:0/4214469169 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55ae865ede80 0x55ae865ee2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:07:23.465 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:23.463+0000 7f44ef453740 1 -- 172.21.15.22:0/4214469169 >> 172.21.15.22:0/4214469169 conn(0x55ae865b6200 msgr2=0x55ae865e97f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:07:23.465 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:23.463+0000 7f44ea800640 1 --2- 172.21.15.22:0/1337294189 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7f44cc0a2880 0x7f44cc0a4d40 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x55ae865ef0d0 tx=0x7f44e00096a0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:07:23.465 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:23.464+0000 7f44ef453740 1 -- 172.21.15.22:0/4214469169 shutdown_connections 2024-03-28T18:07:23.465 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:23.464+0000 7f44ef453740 1 -- 172.21.15.22:0/4214469169 wait complete. 2024-03-28T18:07:23.465 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:23.464+0000 7f44ef453740 1 -- start start 2024-03-28T18:07:23.467 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:23.465+0000 7f44ef453740 1 Processor -- start 2024-03-28T18:07:23.467 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:23.465+0000 7f44ef453740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55ae865ede80 0x55ae865f7da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:07:23.467 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:23.465+0000 7f44ef453740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ae865ee870 0x55ae865f82e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:07:23.467 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:23.465+0000 7f44ef453740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ae865f39e0 0x55ae865ff320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:07:23.467 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:23.465+0000 7f44ef453740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ae865f6930 con 0x55ae865ee870 2024-03-28T18:07:23.467 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:23.465+0000 7f44ef453740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ae8650ee80 con 0x55ae865ede80 2024-03-28T18:07:23.467 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:23.465+0000 7f44ef453740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ae865f6ab0 con 0x55ae865f39e0 2024-03-28T18:07:23.468 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:23.465+0000 7f44ef453740 10 client.0 _ll_register_callbacks cb 0x55ae86521130 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-28T18:07:23.468 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:23.466+0000 7f44e9400640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ae865ee870 0x55ae865f82e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:07:23.468 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:23.466+0000 7f44e9e00640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55ae865ede80 0x55ae865f7da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:07:23.468 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:23.466+0000 7f44e9e00640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55ae865ede80 0x55ae865f7da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.22:59400/0 (socket says 172.21.15.22:59400) 2024-03-28T18:07:23.468 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:23.466+0000 7f44e9e00640 1 -- 172.21.15.22:0/501638849 learned_addr learned my addr 172.21.15.22:0/501638849 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-28T18:07:23.468 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:23.466+0000 7f44ea800640 1 --2- 172.21.15.22:0/501638849 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ae865f39e0 0x55ae865ff320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:07:23.468 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:23.466+0000 7f44e9e00640 1 -- 172.21.15.22:0/501638849 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ae865f39e0 msgr2=0x55ae865ff320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T18:07:23.468 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:23.466+0000 7f44e9e00640 1 --2- 172.21.15.22:0/501638849 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ae865f39e0 0x55ae865ff320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:07:23.468 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:23.466+0000 7f44e9e00640 1 -- 172.21.15.22:0/501638849 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ae865ee870 msgr2=0x55ae865f82e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T18:07:23.468 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:23.466+0000 7f44e9e00640 1 --2- 172.21.15.22:0/501638849 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ae865ee870 0x55ae865f82e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:07:23.468 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:23.466+0000 7f44e9e00640 1 -- 172.21.15.22:0/501638849 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55ae863317d0 con 0x55ae865ede80 2024-03-28T18:07:23.468 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:23.466+0000 7f44e9400640 1 --2- 172.21.15.22:0/501638849 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ae865ee870 0x55ae865f82e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-28T18:07:23.468 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:23.466+0000 7f44e9e00640 1 --2- 172.21.15.22:0/501638849 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55ae865ede80 0x55ae865f7da0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f44d802c170 tx=0x7f44d8003820 comp rx=0 tx=0).ready entity=mon.2 client_cookie=df37fe252d401ccc server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:07:23.468 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:23.466+0000 7f44c7400640 10 client.0 ms_handle_connect on v2:172.21.15.79:3300/0 2024-03-28T18:07:23.469 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:23.467+0000 7f44c7400640 1 -- 172.21.15.22:0/501638849 <== mon.2 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44d8044040 con 0x55ae865ede80 2024-03-28T18:07:23.469 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:23.467+0000 7f44ef453740 1 -- 172.21.15.22:0/501638849 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f44d802cba0 con 0x55ae865ede80 2024-03-28T18:07:23.469 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:23.467+0000 7f44c7400640 1 -- 172.21.15.22:0/501638849 <== mon.2 v2:172.21.15.79:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 8153+0+0 (secure 0 0 0) 0x7f44d8003c20 con 0x55ae865ede80 2024-03-28T18:07:23.469 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:23.467+0000 7f44c7400640 1 -- 172.21.15.22:0/501638849 <== mon.2 v2:172.21.15.79:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f44d8003e20 con 0x55ae865ede80 2024-03-28T18:07:23.469 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:23.467+0000 7f44ef453740 10 client.35369 fetch_fsmap learned FSMap version 27 2024-03-28T18:07:23.469 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:23.467+0000 7f44ef453740 1 -- 172.21.15.22:0/501638849 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55ae862673f0 con 0x55ae865ede80 2024-03-28T18:07:23.471 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:23.468+0000 7f44c7400640 1 -- 172.21.15.22:0/501638849 <== mon.2 v2:172.21.15.79:3300/0 4 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f44d802cba0 con 0x55ae865ede80 2024-03-28T18:07:23.471 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:23.468+0000 7f44c7400640 1 -- 172.21.15.22:0/501638849 <== mon.2 v2:172.21.15.79:3300/0 5 ==== fsmap(e 27) v1 ==== 7934+0+0 (secure 0 0 0) 0x7f44d804b460 con 0x55ae865ede80 2024-03-28T18:07:23.471 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:23.468+0000 7f44ef453740 10 client.35369 fetch_fsmap finished waiting for FSMap version 27 2024-03-28T18:07:23.471 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:23.468+0000 7f44ef453740 10 client.35369 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-28T18:07:23.471 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:23.468+0000 7f44ef453740 20 client.35369 populate_metadata read hostname 'smithi022' 2024-03-28T18:07:23.471 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:23.468+0000 7f44ef453740 1 --2- 172.21.15.22:0/501638849 >> [v2:172.21.15.79:6838/2441542443,v1:172.21.15.79:6839/2441542443] conn(0x55ae86603810 0x55ae86605c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:07:23.471 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:23.468+0000 7f44ef453740 4 client.35369 mds_command: new command op to 34553 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-28T18:07:23.471 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:23.468+0000 7f44ef453740 1 -- 172.21.15.22:0/501638849 --> [v2:172.21.15.79:6838/2441542443,v1:172.21.15.79:6839/2441542443] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55ae86560790 con 0x55ae86603810 2024-03-28T18:07:23.472 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:23.469+0000 7f44e9400640 1 --2- 172.21.15.22:0/501638849 >> [v2:172.21.15.79:6838/2441542443,v1:172.21.15.79:6839/2441542443] conn(0x55ae86603810 0x55ae86605c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:07:23.472 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:23.469+0000 7f44e9400640 1 --2- 172.21.15.22:0/501638849 >> [v2:172.21.15.79:6838/2441542443,v1:172.21.15.79:6839/2441542443] conn(0x55ae86603810 0x55ae86605c00 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4259ce9d22b15b0a server_cookie=6bebc830599a6677 in_seq=0 out_seq=0 2024-03-28T18:07:23.472 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:23.469+0000 7f44c7400640 10 client.35369 ms_handle_connect on v2:172.21.15.79:6838/2441542443 2024-03-28T18:07:23.472 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:23.471+0000 7f44c7400640 1 -- 172.21.15.22:0/501638849 <== mds.0 v2:172.21.15.79:6838/2441542443 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x55ae86560790 con 0x55ae86603810 2024-03-28T18:07:23.472 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:23.471+0000 7f44c7400640 10 client.35369 handle_command_reply: tid=0 2024-03-28T18:07:23.472 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:23.471+0000 7f44ef453740 1 client.35369 shutdown 2024-03-28T18:07:23.472 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:23.471+0000 7f44ef453740 1 -- 172.21.15.22:0/501638849 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55ae865ede80 msgr2=0x55ae865f7da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:07:23.472 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:23.471+0000 7f44ef453740 1 --2- 172.21.15.22:0/501638849 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55ae865ede80 0x55ae865f7da0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f44d802c170 tx=0x7f44d8003820 comp rx=0 tx=0).stop 2024-03-28T18:07:23.472 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:23.471+0000 7f44ef453740 1 -- 172.21.15.22:0/501638849 shutdown_connections 2024-03-28T18:07:23.472 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:23.471+0000 7f44ef453740 1 --2- 172.21.15.22:0/501638849 >> [v2:172.21.15.79:6838/2441542443,v1:172.21.15.79:6839/2441542443] conn(0x55ae86603810 0x55ae86605c00 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:07:23.472 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:23.471+0000 7f44ef453740 1 --2- 172.21.15.22:0/501638849 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ae865f39e0 0x55ae865ff320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:07:23.472 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:23.471+0000 7f44c7400640 0 client.35369 ms_handle_reset on v2:172.21.15.79:6838/2441542443 2024-03-28T18:07:23.472 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:23.471+0000 7f44ef453740 1 --2- 172.21.15.22:0/501638849 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ae865ee870 0x55ae865f82e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:07:23.472 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:23.471+0000 7f44ef453740 1 --2- 172.21.15.22:0/501638849 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55ae865ede80 0x55ae865f7da0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:07:23.472 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:23.471+0000 7f44ef453740 1 -- 172.21.15.22:0/501638849 >> 172.21.15.22:0/501638849 conn(0x55ae865b6200 msgr2=0x55ae865e8a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:07:23.473 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:23.471+0000 7f44ef453740 1 -- 172.21.15.22:0/501638849 shutdown_connections 2024-03-28T18:07:23.473 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:23.471+0000 7f44ef453740 1 -- 172.21.15.22:0/501638849 wait complete. 2024-03-28T18:07:23.473 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:23.472+0000 7f44ef453740 20 client.35369 trim_cache size 0 max 16384 2024-03-28T18:07:23.509 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:23.509+0000 7f44ef453740 1 Processor -- start 2024-03-28T18:07:23.509 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:23.509+0000 7f44ef453740 1 -- start start 2024-03-28T18:07:23.510 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:23.509+0000 7f44ef453740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ae865ede80 0x55ae865ed500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:07:23.510 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:23.509+0000 7f44ef453740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ae865ee870 0x55ae865ff2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:07:23.510 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:23.509+0000 7f44ef453740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55ae865f39e0 0x55ae8660a2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:07:23.510 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:23.509+0000 7f44ef453740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44e41aed20 con 0x55ae865ee870 2024-03-28T18:07:23.510 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:23.510+0000 7f44ef453740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44e41b06a0 con 0x55ae865f39e0 2024-03-28T18:07:23.510 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:23.510+0000 7f44ef453740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44e41af380 con 0x55ae865ede80 2024-03-28T18:07:23.510 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:23.510+0000 7f44ea800640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55ae865f39e0 0x55ae8660a2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:07:23.510 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:23.510+0000 7f44e9400640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ae865ee870 0x55ae865ff2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:07:23.510 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:23.510+0000 7f44ea800640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55ae865f39e0 0x55ae8660a2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.22:59412/0 (socket says 172.21.15.22:59412) 2024-03-28T18:07:23.510 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:23.510+0000 7f44ea800640 1 -- 172.21.15.22:0/2795184017 learned_addr learned my addr 172.21.15.22:0/2795184017 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-28T18:07:23.510 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:23.510+0000 7f44e9e00640 1 --2- 172.21.15.22:0/2795184017 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ae865ede80 0x55ae865ed500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:07:23.511 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:23.510+0000 7f44ea800640 1 -- 172.21.15.22:0/2795184017 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ae865ede80 msgr2=0x55ae865ed500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T18:07:23.511 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:23.510+0000 7f44ea800640 1 --2- 172.21.15.22:0/2795184017 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ae865ede80 0x55ae865ed500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:07:23.511 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:23.510+0000 7f44ea800640 1 -- 172.21.15.22:0/2795184017 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ae865ee870 msgr2=0x55ae865ff2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T18:07:23.511 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:23.510+0000 7f44ea800640 1 --2- 172.21.15.22:0/2795184017 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ae865ee870 0x55ae865ff2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:07:23.511 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:23.510+0000 7f44ea800640 1 -- 172.21.15.22:0/2795184017 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ae862270f0 con 0x55ae865f39e0 2024-03-28T18:07:23.511 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:23.511+0000 7f44ea800640 1 --2- 172.21.15.22:0/2795184017 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55ae865f39e0 0x55ae8660a2e0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f44e001f5f0 tx=0x7f44e0016000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d15fc32b86911454 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:07:23.512 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:23.511+0000 7f44d2600640 1 -- 172.21.15.22:0/2795184017 <== mon.2 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44e002d070 con 0x55ae865f39e0 2024-03-28T18:07:23.512 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:23.511+0000 7f44d2600640 1 -- 172.21.15.22:0/2795184017 <== mon.2 v2:172.21.15.79:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f44e00288e0 con 0x55ae865f39e0 2024-03-28T18:07:23.512 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:23.511+0000 7f44d2600640 1 -- 172.21.15.22:0/2795184017 <== mon.2 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44e001c830 con 0x55ae865f39e0 2024-03-28T18:07:23.513 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:23.512+0000 7f44ef453740 1 -- 172.21.15.22:0/2795184017 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55ae865f39e0 msgr2=0x55ae8660a2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:07:23.513 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:23.512+0000 7f44ef453740 1 --2- 172.21.15.22:0/2795184017 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55ae865f39e0 0x55ae8660a2e0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f44e001f5f0 tx=0x7f44e0016000 comp rx=0 tx=0).stop 2024-03-28T18:07:23.513 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:23.513+0000 7f44ef453740 1 -- 172.21.15.22:0/2795184017 shutdown_connections 2024-03-28T18:07:23.513 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:23.513+0000 7f44ef453740 1 --2- 172.21.15.22:0/2795184017 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55ae865f39e0 0x55ae8660a2e0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:07:23.513 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:23.513+0000 7f44ef453740 1 --2- 172.21.15.22:0/2795184017 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ae865ee870 0x55ae865ff2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:07:23.513 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:23.513+0000 7f44ef453740 1 --2- 172.21.15.22:0/2795184017 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ae865ede80 0x55ae865ed500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:07:23.513 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:23.513+0000 7f44ef453740 1 -- 172.21.15.22:0/2795184017 >> 172.21.15.22:0/2795184017 conn(0x55ae865b6200 msgr2=0x55ae865e8a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:07:23.513 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:23.513+0000 7f44ef453740 1 -- 172.21.15.22:0/2795184017 shutdown_connections 2024-03-28T18:07:23.513 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:23.513+0000 7f44ef453740 1 -- 172.21.15.22:0/2795184017 wait complete. 2024-03-28T18:07:23.513 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:23.513+0000 7f44ef453740 1 -- start start 2024-03-28T18:07:23.513 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:23.513+0000 7f44ef453740 1 Processor -- start 2024-03-28T18:07:23.515 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:23.514+0000 7f44ef453740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ae865ede80 0x55ae865fcc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:07:23.515 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:23.514+0000 7f44ef453740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55ae865ee870 0x55ae865fd150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:07:23.515 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:23.514+0000 7f44ef453740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ae865f39e0 0x55ae865fe160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:07:23.515 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:23.514+0000 7f44ef453740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44e41b0040 con 0x55ae865f39e0 2024-03-28T18:07:23.516 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:23.514+0000 7f44ef453740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44e41af9e0 con 0x55ae865ee870 2024-03-28T18:07:23.516 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:23.514+0000 7f44ef453740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44e41acf30 con 0x55ae865ede80 2024-03-28T18:07:23.516 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:23.514+0000 7f44ef453740 10 client.0 _ll_register_callbacks cb 0x7f44e0009ae0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-28T18:07:23.516 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:23.514+0000 7f44ea800640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ae865f39e0 0x55ae865fe160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:07:23.516 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:23.514+0000 7f44ea800640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ae865f39e0 0x55ae865fe160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:55750/0 (socket says 172.21.15.22:55750) 2024-03-28T18:07:23.516 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:23.514+0000 7f44e9400640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55ae865ee870 0x55ae865fd150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:07:23.516 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:23.514+0000 7f44ea800640 1 -- 172.21.15.22:0/3634462246 learned_addr learned my addr 172.21.15.22:0/3634462246 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-28T18:07:23.516 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:23.514+0000 7f44e9e00640 1 --2- 172.21.15.22:0/3634462246 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ae865ede80 0x55ae865fcc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:07:23.516 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:23.515+0000 7f44ea800640 1 -- 172.21.15.22:0/3634462246 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ae865ede80 msgr2=0x55ae865fcc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T18:07:23.516 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:23.515+0000 7f44ea800640 1 --2- 172.21.15.22:0/3634462246 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ae865ede80 0x55ae865fcc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:07:23.516 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:23.515+0000 7f44ea800640 1 -- 172.21.15.22:0/3634462246 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55ae865ee870 msgr2=0x55ae865fd150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T18:07:23.516 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:23.515+0000 7f44ea800640 1 --2- 172.21.15.22:0/3634462246 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55ae865ee870 0x55ae865fd150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:07:23.516 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:23.515+0000 7f44ea800640 1 -- 172.21.15.22:0/3634462246 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55ae865fe750 con 0x55ae865f39e0 2024-03-28T18:07:23.516 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:23.515+0000 7f44e9400640 1 --2- 172.21.15.22:0/3634462246 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55ae865ee870 0x55ae865fd150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-28T18:07:23.517 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:23.515+0000 7f44ea800640 1 --2- 172.21.15.22:0/3634462246 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ae865f39e0 0x55ae865fe160 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f44e0005cb0 tx=0x7f44e0004830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e65db6f48bbcd27 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:07:23.517 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:23.515+0000 7f44c7400640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-03-28T18:07:23.517 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:23.515+0000 7f44c7400640 1 -- 172.21.15.22:0/3634462246 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44e002d030 con 0x55ae865f39e0 2024-03-28T18:07:23.517 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:23.515+0000 7f44c7400640 1 -- 172.21.15.22:0/3634462246 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 8153+0+0 (secure 0 0 0) 0x7f44e0025040 con 0x55ae865f39e0 2024-03-28T18:07:23.517 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:23.515+0000 7f44ef453740 1 -- 172.21.15.22:0/3634462246 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f44e00288e0 con 0x55ae865f39e0 2024-03-28T18:07:23.517 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:23.516+0000 7f44c7400640 1 -- 172.21.15.22:0/3634462246 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f44e0028a90 con 0x55ae865f39e0 2024-03-28T18:07:23.517 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:23.516+0000 7f44ef453740 10 client.15516 fetch_fsmap learned FSMap version 27 2024-03-28T18:07:23.517 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:23.516+0000 7f44ef453740 1 -- 172.21.15.22:0/3634462246 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55ae8665f0f0 con 0x55ae865f39e0 2024-03-28T18:07:23.519 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:23.516+0000 7f44c7400640 1 -- 172.21.15.22:0/3634462246 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f44e00288e0 con 0x55ae865f39e0 2024-03-28T18:07:23.519 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:23.516+0000 7f44c7400640 1 -- 172.21.15.22:0/3634462246 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 27) v1 ==== 7934+0+0 (secure 0 0 0) 0x7f44e003c020 con 0x55ae865f39e0 2024-03-28T18:07:23.519 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:23.516+0000 7f44ef453740 10 client.15516 fetch_fsmap finished waiting for FSMap version 27 2024-03-28T18:07:23.519 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:23.516+0000 7f44ef453740 10 client.15516 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-28T18:07:23.519 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:23.516+0000 7f44ef453740 20 client.15516 populate_metadata read hostname 'smithi022' 2024-03-28T18:07:23.519 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:23.517+0000 7f44ef453740 1 --2- 172.21.15.22:0/3634462246 >> [v2:172.21.15.79:6838/2441542443,v1:172.21.15.79:6839/2441542443] conn(0x55ae8665f9c0 0x55ae86661de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:07:23.519 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:23.517+0000 7f44ef453740 4 client.15516 mds_command: new command op to 34553 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-28T18:07:23.519 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:23.517+0000 7f44ef453740 1 -- 172.21.15.22:0/3634462246 --> [v2:172.21.15.79:6838/2441542443,v1:172.21.15.79:6839/2441542443] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55ae8655aae0 con 0x55ae8665f9c0 2024-03-28T18:07:23.519 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:23.517+0000 7f44e9e00640 1 --2- 172.21.15.22:0/3634462246 >> [v2:172.21.15.79:6838/2441542443,v1:172.21.15.79:6839/2441542443] conn(0x55ae8665f9c0 0x55ae86661de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:07:23.519 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:23.518+0000 7f44e9e00640 1 --2- 172.21.15.22:0/3634462246 >> [v2:172.21.15.79:6838/2441542443,v1:172.21.15.79:6839/2441542443] conn(0x55ae8665f9c0 0x55ae86661de0 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5e46ce1ffbf35c8d server_cookie=b35faa304dc0c8b4 in_seq=0 out_seq=0 2024-03-28T18:07:23.519 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:23.518+0000 7f44c7400640 10 client.15516 ms_handle_connect on v2:172.21.15.79:6838/2441542443 2024-03-28T18:07:23.520 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:23.518+0000 7f44c7400640 1 -- 172.21.15.22:0/3634462246 <== mds.0 v2:172.21.15.79:6838/2441542443 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x55ae8655aae0 con 0x55ae8665f9c0 2024-03-28T18:07:23.520 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:23.518+0000 7f44c7400640 10 client.15516 handle_command_reply: tid=0 2024-03-28T18:07:23.520 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:23.518+0000 7f44ef453740 1 client.15516 shutdown 2024-03-28T18:07:23.520 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:23.518+0000 7f44ef453740 1 -- 172.21.15.22:0/3634462246 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ae865f39e0 msgr2=0x55ae865fe160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:07:23.520 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:23.518+0000 7f44ef453740 1 --2- 172.21.15.22:0/3634462246 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ae865f39e0 0x55ae865fe160 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f44e0005cb0 tx=0x7f44e0004830 comp rx=0 tx=0).stop 2024-03-28T18:07:23.520 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:23.518+0000 7f44ef453740 1 -- 172.21.15.22:0/3634462246 shutdown_connections 2024-03-28T18:07:23.520 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:23.518+0000 7f44ef453740 1 --2- 172.21.15.22:0/3634462246 >> [v2:172.21.15.79:6838/2441542443,v1:172.21.15.79:6839/2441542443] conn(0x55ae8665f9c0 0x55ae86661de0 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:07:23.520 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:23.518+0000 7f44ef453740 1 --2- 172.21.15.22:0/3634462246 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ae865f39e0 0x55ae865fe160 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:07:23.520 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:23.518+0000 7f44ef453740 1 --2- 172.21.15.22:0/3634462246 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55ae865ee870 0x55ae865fd150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:07:23.520 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:23.518+0000 7f44ef453740 1 --2- 172.21.15.22:0/3634462246 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ae865ede80 0x55ae865fcc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:07:23.520 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:23.518+0000 7f44ef453740 1 -- 172.21.15.22:0/3634462246 >> 172.21.15.22:0/3634462246 conn(0x55ae865b6200 msgr2=0x55ae865ef100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:07:23.520 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:23.519+0000 7f44c7400640 0 client.15516 ms_handle_reset on v2:172.21.15.79:6838/2441542443 2024-03-28T18:07:23.520 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:23.519+0000 7f44ef453740 1 -- 172.21.15.22:0/3634462246 shutdown_connections 2024-03-28T18:07:23.520 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:23.519+0000 7f44ef453740 1 -- 172.21.15.22:0/3634462246 wait complete. 2024-03-28T18:07:23.520 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:23.519+0000 7f44ef453740 20 client.15516 trim_cache size 0 max 16384 2024-03-28T18:07:23.521 INFO:teuthology.orchestra.run.smithi022.stdout:{ 2024-03-28T18:07:23.521 INFO:teuthology.orchestra.run.smithi022.stdout: "status": "IDLE (waiting for more scrubs)", 2024-03-28T18:07:23.521 INFO:teuthology.orchestra.run.smithi022.stdout: "scrubs": { 2024-03-28T18:07:23.521 INFO:teuthology.orchestra.run.smithi022.stdout: "c652d562-2df9-4f99-a9ab-11652cd9ac08": { 2024-03-28T18:07:23.521 INFO:teuthology.orchestra.run.smithi022.stdout: "path": "/", 2024-03-28T18:07:23.521 INFO:teuthology.orchestra.run.smithi022.stdout: "tag": "c652d562-2df9-4f99-a9ab-11652cd9ac08", 2024-03-28T18:07:23.521 INFO:teuthology.orchestra.run.smithi022.stdout: "options": "recursive,force" 2024-03-28T18:07:23.521 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-03-28T18:07:23.521 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-03-28T18:07:23.521 INFO:teuthology.orchestra.run.smithi022.stdout:} 2024-03-28T18:07:23.522 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:23.519+0000 7f44ec800640 1 -- 172.21.15.22:0/1337294189 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7f44cc0a2880 msgr2=0x7f44cc0a4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:07:23.522 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:23.519+0000 7f44ec800640 1 --2- 172.21.15.22:0/1337294189 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7f44cc0a2880 0x7f44cc0a4d40 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x55ae865ef0d0 tx=0x7f44e00096a0 comp rx=0 tx=0).stop 2024-03-28T18:07:23.522 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:23.519+0000 7f44ec800640 1 -- 172.21.15.22:0/1337294189 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f44e40772a0 msgr2=0x7f44e419e530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:07:23.522 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:23.519+0000 7f44ec800640 1 --2- 172.21.15.22:0/1337294189 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f44e40772a0 0x7f44e419e530 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f44d400d770 tx=0x7f44d400dc30 comp rx=0 tx=0).stop 2024-03-28T18:07:23.522 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:23.520+0000 7f44ec800640 1 -- 172.21.15.22:0/1337294189 shutdown_connections 2024-03-28T18:07:23.523 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:23.520+0000 7f44ec800640 1 --2- 172.21.15.22:0/1337294189 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7f44cc0a2880 0x7f44cc0a4d40 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:07:23.523 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:23.520+0000 7f44ec800640 1 --2- 172.21.15.22:0/1337294189 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f44e4077c40 0x7f44e41a55a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:07:23.523 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:23.520+0000 7f44ec800640 1 --2- 172.21.15.22:0/1337294189 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f44e40772a0 0x7f44e419e530 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:07:23.523 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:23.520+0000 7f44ec800640 1 --2- 172.21.15.22:0/1337294189 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f44e4076060 0x7f44e419dff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:07:23.523 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:23.520+0000 7f44ec800640 1 -- 172.21.15.22:0/1337294189 >> 172.21.15.22:0/1337294189 conn(0x7f44e40fee50 msgr2=0x7f44e4100570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:07:23.523 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:23.520+0000 7f44ec800640 1 -- 172.21.15.22:0/1337294189 shutdown_connections 2024-03-28T18:07:23.523 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:23.520+0000 7f44ec800640 1 -- 172.21.15.22:0/1337294189 wait complete. 2024-03-28T18:07:23.537 INFO:tasks.cephfs.filesystem:scrub status for tag:c652d562-2df9-4f99-a9ab-11652cd9ac08 - {'path': '/', 'tag': 'c652d562-2df9-4f99-a9ab-11652cd9ac08', 'options': 'recursive,force'} 2024-03-28T18:07:23.727 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:23.727+0000 7f2d12a00640 1 -- 172.21.15.22:0/662263897 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "snap_schedule"}]=0 v19) v1 ==== 92+0+0 (secure 0 0 0) 0x7f2d0c0bdde0 con 0x7f2d18074100 2024-03-28T18:07:23.729 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:23.730+0000 7f2d1f200640 1 -- 172.21.15.22:0/662263897 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7f2d000a2870 msgr2=0x7f2d000a4d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:07:23.730 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:23.730+0000 7f2d1f200640 1 --2- 172.21.15.22:0/662263897 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7f2d000a2870 0x7f2d000a4d30 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f2d1819aed0 tx=0x7f2d08005f90 comp rx=0 tx=0).stop 2024-03-28T18:07:23.730 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:23.730+0000 7f2d1f200640 1 -- 172.21.15.22:0/662263897 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2d18074100 msgr2=0x7f2d181999c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:07:23.730 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:23.730+0000 7f2d1f200640 1 --2- 172.21.15.22:0/662263897 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2d18074100 0x7f2d181999c0 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f2d0c000c00 tx=0x7f2d0c004e20 comp rx=0 tx=0).stop 2024-03-28T18:07:23.730 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:23.730+0000 7f2d1f200640 1 -- 172.21.15.22:0/662263897 shutdown_connections 2024-03-28T18:07:23.730 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:23.730+0000 7f2d1f200640 1 --2- 172.21.15.22:0/662263897 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7f2d000a2870 0x7f2d000a4d30 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:07:23.730 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:23.730+0000 7f2d1f200640 1 --2- 172.21.15.22:0/662263897 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f2d181093d0 0x7f2d181a0f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:07:23.730 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:23.730+0000 7f2d1f200640 1 --2- 172.21.15.22:0/662263897 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2d180756e0 0x7f2d18199f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:07:23.730 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:23.730+0000 7f2d1f200640 1 --2- 172.21.15.22:0/662263897 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2d18074100 0x7f2d181999c0 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:07:23.730 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:23.730+0000 7f2d1f200640 1 -- 172.21.15.22:0/662263897 >> 172.21.15.22:0/662263897 conn(0x7f2d180fca80 msgr2=0x7f2d181052d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:07:23.731 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:23.730+0000 7f2d1f200640 1 -- 172.21.15.22:0/662263897 shutdown_connections 2024-03-28T18:07:23.731 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:23.730+0000 7f2d1f200640 1 -- 172.21.15.22:0/662263897 wait complete. 2024-03-28T18:07:23.756 DEBUG:teuthology.orchestra.run.smithi022:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph config set mgr mgr/snap_schedule/allow_m_granularity true' 2024-03-28T18:07:23.821 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:07:23 smithi022 ceph-mon[31985]: pgmap v448: 97 pgs: 97 active+clean; 649 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 1.7 KiB/s wr, 12 op/s 2024-03-28T18:07:23.821 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:07:23 smithi022 ceph-mon[31985]: from='client.? 172.21.15.22:0/470254142' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-28T18:07:23.821 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:07:23 smithi022 ceph-mon[31985]: scrub queued for path: / 2024-03-28T18:07:23.821 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:07:23 smithi022 ceph-mon[31985]: scrub summary: idle+waiting paths [/] 2024-03-28T18:07:23.821 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:07:23 smithi022 ceph-mon[31985]: scrub summary: active paths [/] 2024-03-28T18:07:23.821 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:07:23 smithi022 ceph-mon[31985]: scrub summary: idle+waiting paths [/] 2024-03-28T18:07:23.821 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:07:23 smithi022 ceph-mon[31985]: from='client.? 172.21.15.22:0/662263897' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "snap_schedule"}]: dispatch 2024-03-28T18:07:23.946 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:23.946+0000 7f974a000640 1 -- 172.21.15.22:0/3256412713 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9744076060 msgr2=0x7f9744076480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:07:23.946 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:23.946+0000 7f974a000640 1 --2- 172.21.15.22:0/3256412713 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9744076060 0x7f9744076480 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f9734009900 tx=0x7f973402bc30 comp rx=0 tx=0).stop 2024-03-28T18:07:23.946 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:23.946+0000 7f974a000640 1 -- 172.21.15.22:0/3256412713 shutdown_connections 2024-03-28T18:07:23.946 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:23.946+0000 7f974a000640 1 --2- 172.21.15.22:0/3256412713 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f9744077c40 0x7f97440784e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:07:23.946 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:23.946+0000 7f974a000640 1 --2- 172.21.15.22:0/3256412713 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f97440772a0 0x7f9744077700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:07:23.946 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:23.946+0000 7f974a000640 1 --2- 172.21.15.22:0/3256412713 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9744076060 0x7f9744076480 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:07:23.947 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:23.946+0000 7f974a000640 1 -- 172.21.15.22:0/3256412713 >> 172.21.15.22:0/3256412713 conn(0x7f97440fee60 msgr2=0x7f97441012a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:07:23.947 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:23.946+0000 7f974a000640 1 -- 172.21.15.22:0/3256412713 shutdown_connections 2024-03-28T18:07:23.947 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:23.946+0000 7f974a000640 1 -- 172.21.15.22:0/3256412713 wait complete. 2024-03-28T18:07:23.948 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:23.947+0000 7f974a000640 1 Processor -- start 2024-03-28T18:07:23.948 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:23.947+0000 7f974a000640 1 -- start start 2024-03-28T18:07:23.948 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:23.948+0000 7f974a000640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9744076060 0x7f974419dfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:07:23.948 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:23.948+0000 7f974a000640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f97440772a0 0x7f974419e4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:07:23.948 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:23.948+0000 7f974a000640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f9744077c40 0x7f97441a5500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:07:23.948 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:23.948+0000 7f974a000640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9744113990 con 0x7f9744076060 2024-03-28T18:07:23.948 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:23.948+0000 7f974a000640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9744113810 con 0x7f9744077c40 2024-03-28T18:07:23.948 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:23.948+0000 7f974a000640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9744113b10 con 0x7f97440772a0 2024-03-28T18:07:23.949 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:23.948+0000 7f9743400640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9744076060 0x7f974419dfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:07:23.949 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:23.948+0000 7f9743400640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9744076060 0x7f974419dfa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:55782/0 (socket says 172.21.15.22:55782) 2024-03-28T18:07:23.949 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:23.948+0000 7f9743400640 1 -- 172.21.15.22:0/2494355278 learned_addr learned my addr 172.21.15.22:0/2494355278 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-28T18:07:23.949 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:23.948+0000 7f9743e00640 1 --2- 172.21.15.22:0/2494355278 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f9744077c40 0x7f97441a5500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:07:23.949 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:23.948+0000 7f9742a00640 1 --2- 172.21.15.22:0/2494355278 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f97440772a0 0x7f974419e4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:07:23.949 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:23.949+0000 7f9743400640 1 -- 172.21.15.22:0/2494355278 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f97440772a0 msgr2=0x7f974419e4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:07:23.949 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:23.949+0000 7f9743400640 1 --2- 172.21.15.22:0/2494355278 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f97440772a0 0x7f974419e4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:07:23.949 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:23.949+0000 7f9743400640 1 -- 172.21.15.22:0/2494355278 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f9744077c40 msgr2=0x7f97441a5500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:07:23.949 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:23.949+0000 7f9743400640 1 --2- 172.21.15.22:0/2494355278 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f9744077c40 0x7f97441a5500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:07:23.949 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:23.949+0000 7f9743400640 1 -- 172.21.15.22:0/2494355278 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f97441a5b70 con 0x7f9744076060 2024-03-28T18:07:23.950 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:23.949+0000 7f9743400640 1 --2- 172.21.15.22:0/2494355278 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9744076060 0x7f974419dfa0 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f9734000c00 tx=0x7f9734004e20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:07:23.950 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:23.949+0000 7f973be00640 1 -- 172.21.15.22:0/2494355278 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9734043020 con 0x7f9744076060 2024-03-28T18:07:23.950 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:23.949+0000 7f973be00640 1 -- 172.21.15.22:0/2494355278 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f9734005460 con 0x7f9744076060 2024-03-28T18:07:23.950 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:23.949+0000 7f973be00640 1 -- 172.21.15.22:0/2494355278 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9734003d00 con 0x7f9744076060 2024-03-28T18:07:23.950 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:23.949+0000 7f974a000640 1 -- 172.21.15.22:0/2494355278 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f97441a5e00 con 0x7f9744076060 2024-03-28T18:07:23.950 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:23.950+0000 7f974a000640 1 -- 172.21.15.22:0/2494355278 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f97441a61b0 con 0x7f9744076060 2024-03-28T18:07:23.951 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:23.951+0000 7f974a000640 1 -- 172.21.15.22:0/2494355278 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9744110260 con 0x7f9744076060 2024-03-28T18:07:23.952 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:23.952+0000 7f973be00640 1 -- 172.21.15.22:0/2494355278 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 19) v1 ==== 137307+0+0 (secure 0 0 0) 0x7f9734046020 con 0x7f9744076060 2024-03-28T18:07:23.953 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:23.954+0000 7f973be00640 1 --2- 172.21.15.22:0/2494355278 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7f97280a28f0 0x7f97280a4db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:07:23.954 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:23.954+0000 7f973be00640 1 -- 172.21.15.22:0/2494355278 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 8153+0+0 (secure 0 0 0) 0x7f97340f0c80 con 0x7f9744076060 2024-03-28T18:07:23.954 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:23.954+0000 7f9742a00640 1 -- 172.21.15.22:0/2494355278 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7f97280a28f0 msgr2=0x7f97280a4db0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.22:6800/4201048392 2024-03-28T18:07:23.954 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:23.954+0000 7f9742a00640 1 --2- 172.21.15.22:0/2494355278 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7f97280a28f0 0x7f97280a4db0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-28T18:07:23.958 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:23.958+0000 7f973be00640 1 -- 172.21.15.22:0/2494355278 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f973404d020 con 0x7f9744076060 2024-03-28T18:07:24.154 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:24.155+0000 7f9742a00640 1 -- 172.21.15.22:0/2494355278 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7f97280a28f0 msgr2=0x7f97280a4db0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.22:6800/4201048392 2024-03-28T18:07:24.155 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:24.155+0000 7f9742a00640 1 --2- 172.21.15.22:0/2494355278 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7f97280a28f0 0x7f97280a4db0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-28T18:07:24.174 INFO:journalctl@ceph.mgr.z.smithi134.stdout:Mar 28 18:07:23 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mgr-z[37033]: ignoring --setuser ceph since I am not root 2024-03-28T18:07:24.174 INFO:journalctl@ceph.mgr.z.smithi134.stdout:Mar 28 18:07:23 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mgr-z[37033]: ignoring --setgroup ceph since I am not root 2024-03-28T18:07:24.175 INFO:journalctl@ceph.mgr.z.smithi134.stdout:Mar 28 18:07:23 smithi134 ceph-mgr[37044]: -- 172.21.15.134:0/1749822997 <== mon.1 v2:172.21.15.134:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55ea05b9e5a0 con 0x55ea05b72800 2024-03-28T18:07:24.175 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:07:23 smithi134 ceph-mon[35037]: pgmap v448: 97 pgs: 97 active+clean; 649 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 1.7 KiB/s wr, 12 op/s 2024-03-28T18:07:24.175 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:07:23 smithi134 ceph-mon[35037]: from='client.? 172.21.15.22:0/470254142' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-28T18:07:24.175 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:07:23 smithi134 ceph-mon[35037]: scrub queued for path: / 2024-03-28T18:07:24.175 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:07:23 smithi134 ceph-mon[35037]: scrub summary: idle+waiting paths [/] 2024-03-28T18:07:24.175 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:07:23 smithi134 ceph-mon[35037]: scrub summary: active paths [/] 2024-03-28T18:07:24.175 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:07:23 smithi134 ceph-mon[35037]: scrub summary: idle+waiting paths [/] 2024-03-28T18:07:24.175 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:07:23 smithi134 ceph-mon[35037]: from='client.? 172.21.15.22:0/662263897' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "snap_schedule"}]: dispatch 2024-03-28T18:07:24.182 INFO:journalctl@ceph.mgr.y.smithi079.stdout:Mar 28 18:07:23 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mgr-y[36699]: ignoring --setuser ceph since I am not root 2024-03-28T18:07:24.183 INFO:journalctl@ceph.mgr.y.smithi079.stdout:Mar 28 18:07:23 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mgr-y[36699]: ignoring --setgroup ceph since I am not root 2024-03-28T18:07:24.183 INFO:journalctl@ceph.mgr.y.smithi079.stdout:Mar 28 18:07:23 smithi079 ceph-mgr[36703]: -- 172.21.15.79:0/984092825 <== mon.1 v2:172.21.15.134:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55aaef9785a0 con 0x55aaef94cc00 2024-03-28T18:07:24.183 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:07:23 smithi079 ceph-mon[35312]: pgmap v448: 97 pgs: 97 active+clean; 649 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 1.7 KiB/s wr, 12 op/s 2024-03-28T18:07:24.183 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:07:23 smithi079 ceph-mon[35312]: from='client.? 172.21.15.22:0/470254142' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-28T18:07:24.183 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:07:23 smithi079 ceph-mon[35312]: scrub queued for path: / 2024-03-28T18:07:24.184 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:07:23 smithi079 ceph-mon[35312]: scrub summary: idle+waiting paths [/] 2024-03-28T18:07:24.184 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:07:23 smithi079 ceph-mon[35312]: scrub summary: active paths [/] 2024-03-28T18:07:24.184 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:07:23 smithi079 ceph-mon[35312]: scrub summary: idle+waiting paths [/] 2024-03-28T18:07:24.184 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:07:23 smithi079 ceph-mon[35312]: from='client.? 172.21.15.22:0/662263897' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "snap_schedule"}]: dispatch 2024-03-28T18:07:24.190 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:24.191+0000 7f974a000640 1 -- 172.21.15.22:0/2494355278 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}] v 0) v1 -- 0x7f97440622a0 con 0x7f9744076060 2024-03-28T18:07:24.195 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:24.196+0000 7f973be00640 1 -- 172.21.15.22:0/2494355278 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}]=0 v31)=0 v31) v1 ==== 138+0+0 (secure 0 0 0) 0x7f97340bdd20 con 0x7f9744076060 2024-03-28T18:07:24.200 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:24.200+0000 7f974a000640 1 -- 172.21.15.22:0/2494355278 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7f97280a28f0 msgr2=0x7f97280a4db0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-28T18:07:24.200 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:24.200+0000 7f974a000640 1 --2- 172.21.15.22:0/2494355278 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7f97280a28f0 0x7f97280a4db0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:07:24.200 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:24.200+0000 7f974a000640 1 -- 172.21.15.22:0/2494355278 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9744076060 msgr2=0x7f974419dfa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:07:24.200 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:24.200+0000 7f974a000640 1 --2- 172.21.15.22:0/2494355278 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9744076060 0x7f974419dfa0 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f9734000c00 tx=0x7f9734004e20 comp rx=0 tx=0).stop 2024-03-28T18:07:24.200 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:24.200+0000 7f974a000640 1 -- 172.21.15.22:0/2494355278 shutdown_connections 2024-03-28T18:07:24.200 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:24.200+0000 7f974a000640 1 --2- 172.21.15.22:0/2494355278 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7f97280a28f0 0x7f97280a4db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:07:24.201 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:24.200+0000 7f974a000640 1 --2- 172.21.15.22:0/2494355278 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f9744077c40 0x7f97441a5500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:07:24.201 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:24.200+0000 7f974a000640 1 --2- 172.21.15.22:0/2494355278 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f97440772a0 0x7f974419e4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:07:24.201 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:24.200+0000 7f974a000640 1 --2- 172.21.15.22:0/2494355278 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9744076060 0x7f974419dfa0 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:07:24.201 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:24.200+0000 7f974a000640 1 -- 172.21.15.22:0/2494355278 >> 172.21.15.22:0/2494355278 conn(0x7f97440fee60 msgr2=0x7f9744100960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:07:24.201 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:24.200+0000 7f974a000640 1 -- 172.21.15.22:0/2494355278 shutdown_connections 2024-03-28T18:07:24.201 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:24.200+0000 7f974a000640 1 -- 172.21.15.22:0/2494355278 wait complete. 2024-03-28T18:07:24.215 INFO:journalctl@ceph.mgr.x.smithi022.stdout:Mar 28 18:07:23 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mgr-x[32211]: ignoring --setuser ceph since I am not root 2024-03-28T18:07:24.215 INFO:journalctl@ceph.mgr.x.smithi022.stdout:Mar 28 18:07:23 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mgr-x[32211]: ignoring --setgroup ceph since I am not root 2024-03-28T18:07:24.218 DEBUG:teuthology.orchestra.run.smithi022:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph config set mgr mgr/snap_schedule/dump_on_update true' 2024-03-28T18:07:24.409 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:24.409+0000 7f9c87e00640 1 -- 172.21.15.22:0/4023400845 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9c801028c0 msgr2=0x7f9c80102d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:07:24.409 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:24.409+0000 7f9c87e00640 1 --2- 172.21.15.22:0/4023400845 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9c801028c0 0x7f9c80102d40 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f9c74009ca0 tx=0x7f9c7402c0f0 comp rx=0 tx=0).stop 2024-03-28T18:07:24.409 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:24.409+0000 7f9c87e00640 1 -- 172.21.15.22:0/4023400845 shutdown_connections 2024-03-28T18:07:24.409 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:24.409+0000 7f9c87e00640 1 --2- 172.21.15.22:0/4023400845 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9c80103280 0x7f9c80109b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:07:24.409 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:24.409+0000 7f9c87e00640 1 --2- 172.21.15.22:0/4023400845 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9c801028c0 0x7f9c80102d40 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:07:24.409 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:24.409+0000 7f9c87e00640 1 --2- 172.21.15.22:0/4023400845 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f9c801016d0 0x7f9c80101af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:07:24.409 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:24.409+0000 7f9c87e00640 1 -- 172.21.15.22:0/4023400845 >> 172.21.15.22:0/4023400845 conn(0x7f9c800fcc60 msgr2=0x7f9c800ff080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:07:24.409 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:24.409+0000 7f9c87e00640 1 -- 172.21.15.22:0/4023400845 shutdown_connections 2024-03-28T18:07:24.409 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:24.409+0000 7f9c87e00640 1 -- 172.21.15.22:0/4023400845 wait complete. 2024-03-28T18:07:24.411 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:24.410+0000 7f9c87e00640 1 Processor -- start 2024-03-28T18:07:24.411 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:24.410+0000 7f9c87e00640 1 -- start start 2024-03-28T18:07:24.412 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:24.410+0000 7f9c87e00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9c801016d0 0x7f9c801978c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:07:24.412 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:24.410+0000 7f9c87e00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9c801028c0 0x7f9c80197e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:07:24.412 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:24.410+0000 7f9c87e00640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f9c80103280 0x7f9c8019ee70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:07:24.412 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:24.410+0000 7f9c87e00640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c8010d330 con 0x7f9c801016d0 2024-03-28T18:07:24.412 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:24.410+0000 7f9c87e00640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c8010d1b0 con 0x7f9c80103280 2024-03-28T18:07:24.412 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:24.410+0000 7f9c87e00640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c8010d4b0 con 0x7f9c801028c0 2024-03-28T18:07:24.412 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:24.411+0000 7f9c85400640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9c801016d0 0x7f9c801978c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:07:24.412 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:24.411+0000 7f9c85400640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9c801016d0 0x7f9c801978c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:55800/0 (socket says 172.21.15.22:55800) 2024-03-28T18:07:24.412 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:24.411+0000 7f9c85400640 1 -- 172.21.15.22:0/1275896374 learned_addr learned my addr 172.21.15.22:0/1275896374 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-28T18:07:24.412 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:24.411+0000 7f9c85e00640 1 --2- 172.21.15.22:0/1275896374 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f9c80103280 0x7f9c8019ee70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:07:24.412 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:24.411+0000 7f9c84a00640 1 --2- 172.21.15.22:0/1275896374 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9c801028c0 0x7f9c80197e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:07:24.412 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:24.411+0000 7f9c85400640 1 -- 172.21.15.22:0/1275896374 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9c801028c0 msgr2=0x7f9c80197e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:07:24.412 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:24.411+0000 7f9c85400640 1 --2- 172.21.15.22:0/1275896374 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9c801028c0 0x7f9c80197e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:07:24.412 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:24.411+0000 7f9c85400640 1 -- 172.21.15.22:0/1275896374 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f9c80103280 msgr2=0x7f9c8019ee70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:07:24.412 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:24.411+0000 7f9c85400640 1 --2- 172.21.15.22:0/1275896374 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f9c80103280 0x7f9c8019ee70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:07:24.413 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:24.411+0000 7f9c85400640 1 -- 172.21.15.22:0/1275896374 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9c8019f4e0 con 0x7f9c801016d0 2024-03-28T18:07:24.413 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:24.411+0000 7f9c85400640 1 --2- 172.21.15.22:0/1275896374 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9c801016d0 0x7f9c801978c0 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f9c6c00d770 tx=0x7f9c6c00dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:07:24.413 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:24.412+0000 7f9c7e000640 1 -- 172.21.15.22:0/1275896374 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c6c017070 con 0x7f9c801016d0 2024-03-28T18:07:24.413 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:24.412+0000 7f9c7e000640 1 -- 172.21.15.22:0/1275896374 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f9c6c009c30 con 0x7f9c801016d0 2024-03-28T18:07:24.413 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:24.412+0000 7f9c87e00640 1 -- 172.21.15.22:0/1275896374 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9c8019f7d0 con 0x7f9c801016d0 2024-03-28T18:07:24.413 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:24.412+0000 7f9c7e000640 1 -- 172.21.15.22:0/1275896374 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c6c003e10 con 0x7f9c801016d0 2024-03-28T18:07:24.413 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:24.412+0000 7f9c87e00640 1 -- 172.21.15.22:0/1275896374 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9c8019fd40 con 0x7f9c801016d0 2024-03-28T18:07:24.413 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:24.413+0000 7f9c87e00640 1 -- 172.21.15.22:0/1275896374 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9c80108920 con 0x7f9c801016d0 2024-03-28T18:07:24.415 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:24.415+0000 7f9c7e000640 1 -- 172.21.15.22:0/1275896374 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 19) v1 ==== 137307+0+0 (secure 0 0 0) 0x7f9c6c004000 con 0x7f9c801016d0 2024-03-28T18:07:24.419 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:24.419+0000 7f9c7e000640 1 --2- 172.21.15.22:0/1275896374 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7f9c680a2970 0x7f9c680a4e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:07:24.419 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:24.419+0000 7f9c84a00640 1 -- 172.21.15.22:0/1275896374 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7f9c680a2970 msgr2=0x7f9c680a4e30 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.22:6800/4201048392 2024-03-28T18:07:24.419 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:24.419+0000 7f9c7e000640 1 -- 172.21.15.22:0/1275896374 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 8153+0+0 (secure 0 0 0) 0x7f9c6c0ce7c0 con 0x7f9c801016d0 2024-03-28T18:07:24.419 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:24.419+0000 7f9c84a00640 1 --2- 172.21.15.22:0/1275896374 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7f9c680a2970 0x7f9c680a4e30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-28T18:07:24.421 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:24.421+0000 7f9c7e000640 1 -- 172.21.15.22:0/1275896374 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f9c6c09b8e0 con 0x7f9c801016d0 2024-03-28T18:07:24.619 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:24.620+0000 7f9c84a00640 1 -- 172.21.15.22:0/1275896374 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7f9c680a2970 msgr2=0x7f9c680a4e30 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.22:6800/4201048392 2024-03-28T18:07:24.620 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:24.620+0000 7f9c84a00640 1 --2- 172.21.15.22:0/1275896374 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7f9c680a2970 0x7f9c680a4e30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-28T18:07:24.624 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:24.624+0000 7f9c87e00640 1 -- 172.21.15.22:0/1275896374 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/dump_on_update}] v 0) v1 -- 0x7f9c8002d070 con 0x7f9c801016d0 2024-03-28T18:07:24.631 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:24.632+0000 7f9c7e000640 1 -- 172.21.15.22:0/1275896374 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/dump_on_update}]=0 v32)=0 v32) v1 ==== 133+0+0 (secure 0 0 0) 0x7f9c6c09b1e0 con 0x7f9c801016d0 2024-03-28T18:07:24.635 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:24.635+0000 7f9c87e00640 1 -- 172.21.15.22:0/1275896374 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7f9c680a2970 msgr2=0x7f9c680a4e30 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-28T18:07:24.635 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:24.635+0000 7f9c87e00640 1 --2- 172.21.15.22:0/1275896374 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7f9c680a2970 0x7f9c680a4e30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:07:24.635 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:24.635+0000 7f9c87e00640 1 -- 172.21.15.22:0/1275896374 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9c801016d0 msgr2=0x7f9c801978c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:07:24.635 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:24.635+0000 7f9c87e00640 1 --2- 172.21.15.22:0/1275896374 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9c801016d0 0x7f9c801978c0 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f9c6c00d770 tx=0x7f9c6c00dc30 comp rx=0 tx=0).stop 2024-03-28T18:07:24.635 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:24.635+0000 7f9c87e00640 1 -- 172.21.15.22:0/1275896374 shutdown_connections 2024-03-28T18:07:24.635 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:24.635+0000 7f9c87e00640 1 --2- 172.21.15.22:0/1275896374 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7f9c680a2970 0x7f9c680a4e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:07:24.635 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:24.635+0000 7f9c87e00640 1 --2- 172.21.15.22:0/1275896374 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f9c80103280 0x7f9c8019ee70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:07:24.636 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:24.635+0000 7f9c87e00640 1 --2- 172.21.15.22:0/1275896374 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9c801028c0 0x7f9c80197e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:07:24.636 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:24.635+0000 7f9c87e00640 1 --2- 172.21.15.22:0/1275896374 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9c801016d0 0x7f9c801978c0 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:07:24.636 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:24.635+0000 7f9c87e00640 1 -- 172.21.15.22:0/1275896374 >> 172.21.15.22:0/1275896374 conn(0x7f9c800fcc60 msgr2=0x7f9c80105af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:07:24.636 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:24.635+0000 7f9c87e00640 1 -- 172.21.15.22:0/1275896374 shutdown_connections 2024-03-28T18:07:24.636 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:24.635+0000 7f9c87e00640 1 -- 172.21.15.22:0/1275896374 wait complete. 2024-03-28T18:07:24.652 DEBUG:teuthology.orchestra.run.smithi022:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule add --fs=cephfs --path=/ --snap_schedule=1m' 2024-03-28T18:07:24.853 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:24.853+0000 7f7aba200640 1 -- 172.21.15.22:0/3846203267 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7ab4104910 msgr2=0x7f7ab4104d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:07:24.853 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:24.853+0000 7f7aba200640 1 --2- 172.21.15.22:0/3846203267 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7ab4104910 0x7f7ab4104d90 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f7aa0009900 tx=0x7f7aa002bc40 comp rx=0 tx=0).stop 2024-03-28T18:07:24.853 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:24.853+0000 7f7aba200640 1 -- 172.21.15.22:0/3846203267 shutdown_connections 2024-03-28T18:07:24.854 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:24.853+0000 7f7aba200640 1 --2- 172.21.15.22:0/3846203267 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7ab41052d0 0x7f7ab410bb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:07:24.854 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:24.853+0000 7f7aba200640 1 --2- 172.21.15.22:0/3846203267 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7ab4104910 0x7f7ab4104d90 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:07:24.854 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:24.853+0000 7f7aba200640 1 --2- 172.21.15.22:0/3846203267 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f7ab4103720 0x7f7ab4103b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:07:24.854 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:24.853+0000 7f7aba200640 1 -- 172.21.15.22:0/3846203267 >> 172.21.15.22:0/3846203267 conn(0x7f7ab40fecb0 msgr2=0x7f7ab41010d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:07:24.854 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:24.853+0000 7f7aba200640 1 -- 172.21.15.22:0/3846203267 shutdown_connections 2024-03-28T18:07:24.854 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:24.853+0000 7f7aba200640 1 -- 172.21.15.22:0/3846203267 wait complete. 2024-03-28T18:07:24.856 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:24.855+0000 7f7aba200640 1 Processor -- start 2024-03-28T18:07:24.857 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:24.855+0000 7f7aba200640 1 -- start start 2024-03-28T18:07:24.857 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:24.855+0000 7f7aba200640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7ab4103720 0x7f7ab41999e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:07:24.857 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:24.855+0000 7f7aba200640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f7ab4104910 0x7f7ab4199f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:07:24.857 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:24.855+0000 7f7aba200640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7ab41052d0 0x7f7ab41a0f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:07:24.857 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:24.855+0000 7f7aba200640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ab410f3d0 con 0x7f7ab4103720 2024-03-28T18:07:24.857 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:24.855+0000 7f7aba200640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ab410f250 con 0x7f7ab4104910 2024-03-28T18:07:24.857 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:24.855+0000 7f7aba200640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ab410f550 con 0x7f7ab41052d0 2024-03-28T18:07:24.857 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:24.856+0000 7f7ab8e00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7ab4103720 0x7f7ab41999e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:07:24.857 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:24.856+0000 7f7ab8e00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7ab4103720 0x7f7ab41999e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:55806/0 (socket says 172.21.15.22:55806) 2024-03-28T18:07:24.857 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:24.856+0000 7f7ab8e00640 1 -- 172.21.15.22:0/4196571610 learned_addr learned my addr 172.21.15.22:0/4196571610 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-28T18:07:24.857 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:24.856+0000 7f7ab3e00640 1 --2- 172.21.15.22:0/4196571610 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f7ab4104910 0x7f7ab4199f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:07:24.857 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:24.856+0000 7f7ab9800640 1 --2- 172.21.15.22:0/4196571610 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7ab41052d0 0x7f7ab41a0f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:07:24.858 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:24.856+0000 7f7ab8e00640 1 -- 172.21.15.22:0/4196571610 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7ab41052d0 msgr2=0x7f7ab41a0f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:07:24.858 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:24.856+0000 7f7ab8e00640 1 --2- 172.21.15.22:0/4196571610 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7ab41052d0 0x7f7ab41a0f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:07:24.858 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:24.856+0000 7f7ab8e00640 1 -- 172.21.15.22:0/4196571610 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f7ab4104910 msgr2=0x7f7ab4199f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:07:24.858 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:24.856+0000 7f7ab8e00640 1 --2- 172.21.15.22:0/4196571610 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f7ab4104910 0x7f7ab4199f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:07:24.858 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:24.856+0000 7f7ab8e00640 1 -- 172.21.15.22:0/4196571610 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ab41a1600 con 0x7f7ab4103720 2024-03-28T18:07:24.858 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:24.856+0000 7f7ab8e00640 1 --2- 172.21.15.22:0/4196571610 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7ab4103720 0x7f7ab41999e0 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f7aa800e9c0 tx=0x7f7aa800ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:07:24.858 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:24.857+0000 7f7ab1600640 1 -- 172.21.15.22:0/4196571610 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7aa800cd80 con 0x7f7ab4103720 2024-03-28T18:07:24.858 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:24.857+0000 7f7ab1600640 1 -- 172.21.15.22:0/4196571610 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f7aa80035f0 con 0x7f7ab4103720 2024-03-28T18:07:24.858 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:24.857+0000 7f7ab1600640 1 -- 172.21.15.22:0/4196571610 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7aa8010690 con 0x7f7ab4103720 2024-03-28T18:07:24.858 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:24.857+0000 7f7aba200640 1 -- 172.21.15.22:0/4196571610 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7ab41a18f0 con 0x7f7ab4103720 2024-03-28T18:07:24.858 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:24.857+0000 7f7aba200640 1 -- 172.21.15.22:0/4196571610 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7ab41a1e60 con 0x7f7ab4103720 2024-03-28T18:07:24.859 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:24.858+0000 7f7aba200640 1 -- 172.21.15.22:0/4196571610 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7ab410a9e0 con 0x7f7ab4103720 2024-03-28T18:07:24.861 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:24.862+0000 7f7ab1600640 1 -- 172.21.15.22:0/4196571610 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 19) v1 ==== 137307+0+0 (secure 0 0 0) 0x7f7aa80037a0 con 0x7f7ab4103720 2024-03-28T18:07:24.863 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:24.864+0000 7f7ab1600640 1 --2- 172.21.15.22:0/4196571610 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7f7a9c0a28c0 0x7f7a9c0a4d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:07:24.863 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:24.864+0000 7f7ab3e00640 1 -- 172.21.15.22:0/4196571610 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7f7a9c0a28c0 msgr2=0x7f7a9c0a4d80 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.22:6800/4201048392 2024-03-28T18:07:24.863 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:24.864+0000 7f7ab1600640 1 -- 172.21.15.22:0/4196571610 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 8153+0+0 (secure 0 0 0) 0x7f7aa80cef70 con 0x7f7ab4103720 2024-03-28T18:07:24.864 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:24.864+0000 7f7ab3e00640 1 --2- 172.21.15.22:0/4196571610 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7f7a9c0a28c0 0x7f7a9c0a4d80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-28T18:07:24.865 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:24.865+0000 7f7ab1600640 1 -- 172.21.15.22:0/4196571610 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f7aa809c0a0 con 0x7f7ab4103720 2024-03-28T18:07:24.916 INFO:journalctl@ceph.mgr.x.smithi022.stdout:Mar 28 18:07:24 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mgr-x[32211]: 2024-03-28T18:07:24.604+0000 7f07c85c6200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-03-28T18:07:24.916 INFO:journalctl@ceph.mgr.x.smithi022.stdout:Mar 28 18:07:24 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mgr-x[32211]: 2024-03-28T18:07:24.736+0000 7f07c85c6200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-03-28T18:07:24.916 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:07:24 smithi022 ceph-mon[31985]: from='client.? 172.21.15.22:0/662263897' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "snap_schedule"}]': finished 2024-03-28T18:07:24.916 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:07:24 smithi022 ceph-mon[31985]: mgrmap e19: x(active, since 12m), standbys: z, y 2024-03-28T18:07:24.917 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:07:24 smithi022 ceph-mon[31985]: from='client.? 172.21.15.22:0/2494355278' entity='client.admin' 2024-03-28T18:07:24.917 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:07:24 smithi022 ceph-mon[31985]: scrub summary: idle 2024-03-28T18:07:24.917 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:07:24 smithi022 ceph-mon[31985]: from='client.? 172.21.15.22:0/1275896374' entity='client.admin' 2024-03-28T18:07:24.924 INFO:journalctl@ceph.mgr.z.smithi134.stdout:Mar 28 18:07:24 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mgr-z[37033]: 2024-03-28T18:07:24.593+0000 7ff03b0c1200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-03-28T18:07:24.924 INFO:journalctl@ceph.mgr.z.smithi134.stdout:Mar 28 18:07:24 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mgr-z[37033]: 2024-03-28T18:07:24.760+0000 7ff03b0c1200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-03-28T18:07:24.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:07:24 smithi134 ceph-mon[35037]: from='client.? 172.21.15.22:0/662263897' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "snap_schedule"}]': finished 2024-03-28T18:07:24.925 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:07:24 smithi134 ceph-mon[35037]: mgrmap e19: x(active, since 12m), standbys: z, y 2024-03-28T18:07:24.925 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:07:24 smithi134 ceph-mon[35037]: from='client.? 172.21.15.22:0/2494355278' entity='client.admin' 2024-03-28T18:07:24.925 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:07:24 smithi134 ceph-mon[35037]: scrub summary: idle 2024-03-28T18:07:24.925 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:07:24 smithi134 ceph-mon[35037]: from='client.? 172.21.15.22:0/1275896374' entity='client.admin' 2024-03-28T18:07:25.050 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:07:24 smithi079 ceph-mon[35312]: from='client.? 172.21.15.22:0/662263897' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "snap_schedule"}]': finished 2024-03-28T18:07:25.050 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:07:24 smithi079 ceph-mon[35312]: mgrmap e19: x(active, since 12m), standbys: z, y 2024-03-28T18:07:25.051 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:07:24 smithi079 ceph-mon[35312]: from='client.? 172.21.15.22:0/2494355278' entity='client.admin' 2024-03-28T18:07:25.051 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:07:24 smithi079 ceph-mon[35312]: scrub summary: idle 2024-03-28T18:07:25.051 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:07:24 smithi079 ceph-mon[35312]: from='client.? 172.21.15.22:0/1275896374' entity='client.admin' 2024-03-28T18:07:25.051 INFO:journalctl@ceph.mgr.y.smithi079.stdout:Mar 28 18:07:24 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mgr-y[36699]: 2024-03-28T18:07:24.892+0000 7f7645a57200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-03-28T18:07:25.051 INFO:journalctl@ceph.mgr.y.smithi079.stdout:Mar 28 18:07:25 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mgr-y[36699]: 2024-03-28T18:07:25.049+0000 7f7645a57200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-03-28T18:07:25.064 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:25.065+0000 7f7ab3e00640 1 -- 172.21.15.22:0/4196571610 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7f7a9c0a28c0 msgr2=0x7f7a9c0a4d80 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.22:6800/4201048392 2024-03-28T18:07:25.064 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:25.065+0000 7f7ab3e00640 1 --2- 172.21.15.22:0/4196571610 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7f7a9c0a28c0 0x7f7a9c0a4d80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-28T18:07:25.083 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:25.083+0000 7f7aba200640 1 -- 172.21.15.22:0/4196571610 --> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}) v1 -- 0x7f7ab4061ae0 con 0x7f7a9c0a28c0 2024-03-28T18:07:25.466 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:25.466+0000 7f7ab3e00640 1 -- 172.21.15.22:0/4196571610 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7f7a9c0a28c0 msgr2=0x7f7a9c0a4d80 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.22:6800/4201048392 2024-03-28T18:07:25.466 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:25.466+0000 7f7ab3e00640 1 --2- 172.21.15.22:0/4196571610 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7f7a9c0a28c0 0x7f7a9c0a4d80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-28T18:07:25.868 INFO:journalctl@ceph.mgr.z.smithi134.stdout:Mar 28 18:07:25 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mgr-z[37033]: 2024-03-28T18:07:25.562+0000 7ff03b0c1200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-03-28T18:07:25.868 INFO:journalctl@ceph.mgr.z.smithi134.stdout:Mar 28 18:07:25 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mgr-z[37033]: 2024-03-28T18:07:25.686+0000 7ff03b0c1200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-03-28T18:07:25.868 INFO:journalctl@ceph.mgr.z.smithi134.stdout:Mar 28 18:07:25 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mgr-z[37033]: 2024-03-28T18:07:25.867+0000 7ff03b0c1200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-03-28T18:07:25.880 INFO:journalctl@ceph.mgr.x.smithi022.stdout:Mar 28 18:07:25 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mgr-x[32211]: 2024-03-28T18:07:25.572+0000 7f07c85c6200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-03-28T18:07:25.880 INFO:journalctl@ceph.mgr.x.smithi022.stdout:Mar 28 18:07:25 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mgr-x[32211]: 2024-03-28T18:07:25.698+0000 7f07c85c6200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-03-28T18:07:26.174 INFO:journalctl@ceph.mgr.z.smithi134.stdout:Mar 28 18:07:25 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mgr-z[37033]: 2024-03-28T18:07:25.995+0000 7ff03b0c1200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-03-28T18:07:26.174 INFO:journalctl@ceph.mgr.z.smithi134.stdout:Mar 28 18:07:26 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mgr-z[37033]: 2024-03-28T18:07:26.114+0000 7ff03b0c1200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-03-28T18:07:26.267 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:26.267+0000 7f7ab3e00640 1 -- 172.21.15.22:0/4196571610 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7f7a9c0a28c0 msgr2=0x7f7a9c0a4d80 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.22:6800/4201048392 2024-03-28T18:07:26.267 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:26.267+0000 7f7ab3e00640 1 --2- 172.21.15.22:0/4196571610 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7f7a9c0a28c0 0x7f7a9c0a4d80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-28T18:07:26.281 INFO:journalctl@ceph.mgr.x.smithi022.stdout:Mar 28 18:07:25 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mgr-x[32211]: 2024-03-28T18:07:25.880+0000 7f07c85c6200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-03-28T18:07:26.282 INFO:journalctl@ceph.mgr.x.smithi022.stdout:Mar 28 18:07:26 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mgr-x[32211]: 2024-03-28T18:07:26.009+0000 7f07c85c6200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-03-28T18:07:26.282 INFO:journalctl@ceph.mgr.x.smithi022.stdout:Mar 28 18:07:26 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mgr-x[32211]: 2024-03-28T18:07:26.123+0000 7f07c85c6200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-03-28T18:07:26.432 INFO:journalctl@ceph.mgr.y.smithi079.stdout:Mar 28 18:07:26 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mgr-y[36699]: 2024-03-28T18:07:26.173+0000 7f7645a57200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-03-28T18:07:26.432 INFO:journalctl@ceph.mgr.y.smithi079.stdout:Mar 28 18:07:26 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mgr-y[36699]: 2024-03-28T18:07:26.367+0000 7f7645a57200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-03-28T18:07:26.729 INFO:journalctl@ceph.mgr.x.smithi022.stdout:Mar 28 18:07:26 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mgr-x[32211]: 2024-03-28T18:07:26.467+0000 7f07c85c6200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-03-28T18:07:26.730 INFO:journalctl@ceph.mgr.x.smithi022.stdout:Mar 28 18:07:26 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mgr-x[32211]: 2024-03-28T18:07:26.613+0000 7f07c85c6200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-03-28T18:07:26.730 INFO:journalctl@ceph.mgr.x.smithi022.stdout:Mar 28 18:07:26 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mgr-x[32211]: 2024-03-28T18:07:26.729+0000 7f07c85c6200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-03-28T18:07:26.745 INFO:journalctl@ceph.mgr.z.smithi134.stdout:Mar 28 18:07:26 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mgr-z[37033]: 2024-03-28T18:07:26.479+0000 7ff03b0c1200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-03-28T18:07:26.746 INFO:journalctl@ceph.mgr.z.smithi134.stdout:Mar 28 18:07:26 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mgr-z[37033]: 2024-03-28T18:07:26.626+0000 7ff03b0c1200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-03-28T18:07:26.932 INFO:journalctl@ceph.mgr.y.smithi079.stdout:Mar 28 18:07:26 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mgr-y[36699]: 2024-03-28T18:07:26.622+0000 7f7645a57200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-03-28T18:07:26.932 INFO:journalctl@ceph.mgr.y.smithi079.stdout:Mar 28 18:07:26 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mgr-y[36699]: 2024-03-28T18:07:26.812+0000 7f7645a57200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-03-28T18:07:27.032 INFO:journalctl@ceph.mgr.x.smithi022.stdout:Mar 28 18:07:26 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mgr-x[32211]: 2024-03-28T18:07:26.858+0000 7f07c85c6200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-03-28T18:07:27.126 INFO:journalctl@ceph.mgr.z.smithi134.stdout:Mar 28 18:07:26 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mgr-z[37033]: 2024-03-28T18:07:26.745+0000 7ff03b0c1200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-03-28T18:07:27.126 INFO:journalctl@ceph.mgr.z.smithi134.stdout:Mar 28 18:07:26 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mgr-z[37033]: 2024-03-28T18:07:26.880+0000 7ff03b0c1200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-03-28T18:07:27.424 INFO:journalctl@ceph.mgr.z.smithi134.stdout:Mar 28 18:07:27 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mgr-z[37033]: 2024-03-28T18:07:27.125+0000 7ff03b0c1200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-03-28T18:07:27.432 INFO:journalctl@ceph.mgr.y.smithi079.stdout:Mar 28 18:07:26 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mgr-y[36699]: 2024-03-28T18:07:26.956+0000 7f7645a57200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-03-28T18:07:27.532 INFO:journalctl@ceph.mgr.x.smithi022.stdout:Mar 28 18:07:27 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mgr-x[32211]: 2024-03-28T18:07:27.083+0000 7f07c85c6200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-03-28T18:07:27.814 INFO:journalctl@ceph.mgr.y.smithi079.stdout:Mar 28 18:07:27 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mgr-y[36699]: 2024-03-28T18:07:27.450+0000 7f7645a57200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-03-28T18:07:27.815 INFO:journalctl@ceph.mgr.y.smithi079.stdout:Mar 28 18:07:27 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mgr-y[36699]: 2024-03-28T18:07:27.658+0000 7f7645a57200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-03-28T18:07:27.815 INFO:journalctl@ceph.mgr.y.smithi079.stdout:Mar 28 18:07:27 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mgr-y[36699]: 2024-03-28T18:07:27.813+0000 7f7645a57200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-03-28T18:07:27.869 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:27.869+0000 7f7ab3e00640 1 -- 172.21.15.22:0/4196571610 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7f7a9c0a28c0 msgr2=0x7f7a9c0a4d80 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.22:6800/4201048392 2024-03-28T18:07:27.869 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:27.869+0000 7f7ab3e00640 1 --2- 172.21.15.22:0/4196571610 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7f7a9c0a28c0 0x7f7a9c0a4d80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2024-03-28T18:07:28.182 INFO:journalctl@ceph.mgr.y.smithi079.stdout:Mar 28 18:07:27 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mgr-y[36699]: 2024-03-28T18:07:27.974+0000 7f7645a57200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-03-28T18:07:28.682 INFO:journalctl@ceph.mgr.y.smithi079.stdout:Mar 28 18:07:28 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mgr-y[36699]: 2024-03-28T18:07:28.251+0000 7f7645a57200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-03-28T18:07:28.781 INFO:journalctl@ceph.mgr.x.smithi022.stdout:Mar 28 18:07:28 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mgr-x[32211]: 2024-03-28T18:07:28.404+0000 7f07c85c6200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-03-28T18:07:28.782 INFO:journalctl@ceph.mgr.x.smithi022.stdout:Mar 28 18:07:28 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mgr-x[32211]: 2024-03-28T18:07:28.585+0000 7f07c85c6200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-03-28T18:07:28.924 INFO:journalctl@ceph.mgr.z.smithi134.stdout:Mar 28 18:07:28 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mgr-z[37033]: 2024-03-28T18:07:28.549+0000 7ff03b0c1200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-03-28T18:07:28.924 INFO:journalctl@ceph.mgr.z.smithi134.stdout:Mar 28 18:07:28 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mgr-z[37033]: 2024-03-28T18:07:28.759+0000 7ff03b0c1200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-03-28T18:07:29.281 INFO:journalctl@ceph.mgr.x.smithi022.stdout:Mar 28 18:07:29 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mgr-x[32211]: 2024-03-28T18:07:29.016+0000 7f07c85c6200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-03-28T18:07:29.282 INFO:journalctl@ceph.mgr.x.smithi022.stdout:Mar 28 18:07:29 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mgr-x[32211]: 2024-03-28T18:07:29.140+0000 7f07c85c6200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-03-28T18:07:29.632 INFO:journalctl@ceph.mgr.z.smithi134.stdout:Mar 28 18:07:29 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mgr-z[37033]: 2024-03-28T18:07:29.213+0000 7ff03b0c1200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-03-28T18:07:29.632 INFO:journalctl@ceph.mgr.z.smithi134.stdout:Mar 28 18:07:29 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mgr-z[37033]: 2024-03-28T18:07:29.365+0000 7ff03b0c1200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-03-28T18:07:29.669 INFO:journalctl@ceph.mgr.x.smithi022.stdout:Mar 28 18:07:29 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mgr-x[32211]: 2024-03-28T18:07:29.408+0000 7f07c85c6200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-03-28T18:07:29.887 INFO:journalctl@ceph.mgr.z.smithi134.stdout:Mar 28 18:07:29 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mgr-z[37033]: 2024-03-28T18:07:29.632+0000 7ff03b0c1200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-03-28T18:07:30.032 INFO:journalctl@ceph.mgr.x.smithi022.stdout:Mar 28 18:07:29 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mgr-x[32211]: 2024-03-28T18:07:29.669+0000 7f07c85c6200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-03-28T18:07:30.174 INFO:journalctl@ceph.mgr.z.smithi134.stdout:Mar 28 18:07:29 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mgr-z[37033]: 2024-03-28T18:07:29.887+0000 7ff03b0c1200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-03-28T18:07:30.391 INFO:journalctl@ceph.mgr.x.smithi022.stdout:Mar 28 18:07:30 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mgr-x[32211]: 2024-03-28T18:07:30.064+0000 7f07c85c6200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-03-28T18:07:30.432 INFO:journalctl@ceph.mgr.y.smithi079.stdout:Mar 28 18:07:30 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mgr-y[36699]: 2024-03-28T18:07:30.144+0000 7f7645a57200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-03-28T18:07:30.432 INFO:journalctl@ceph.mgr.y.smithi079.stdout:Mar 28 18:07:30 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mgr-y[36699]: 2024-03-28T18:07:30.374+0000 7f7645a57200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-03-28T18:07:30.609 INFO:journalctl@ceph.mgr.z.smithi134.stdout:Mar 28 18:07:30 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mgr-z[37033]: 2024-03-28T18:07:30.277+0000 7ff03b0c1200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-03-28T18:07:30.709 INFO:journalctl@ceph.mgr.x.smithi022.stdout:Mar 28 18:07:30 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mgr-x[32211]: 2024-03-28T18:07:30.391+0000 7f07c85c6200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-03-28T18:07:30.920 INFO:journalctl@ceph.mgr.z.smithi134.stdout:Mar 28 18:07:30 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mgr-z[37033]: 2024-03-28T18:07:30.608+0000 7ff03b0c1200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-03-28T18:07:31.031 INFO:journalctl@ceph.mgr.x.smithi022.stdout:Mar 28 18:07:30 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mgr-x[32211]: 2024-03-28T18:07:30.708+0000 7f07c85c6200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-03-28T18:07:31.070 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:31.070+0000 7f7ab3e00640 1 -- 172.21.15.22:0/4196571610 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7f7a9c0a28c0 msgr2=0x7f7a9c0a4d80 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.22:6800/4201048392 2024-03-28T18:07:31.070 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:31.070+0000 7f7ab3e00640 1 --2- 172.21.15.22:0/4196571610 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7f7a9c0a28c0 0x7f7a9c0a4d80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 6.400000 2024-03-28T18:07:31.174 INFO:journalctl@ceph.mgr.z.smithi134.stdout:Mar 28 18:07:30 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mgr-z[37033]: 2024-03-28T18:07:30.919+0000 7ff03b0c1200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-03-28T18:07:31.183 INFO:journalctl@ceph.mgr.y.smithi079.stdout:Mar 28 18:07:30 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mgr-y[36699]: 2024-03-28T18:07:30.883+0000 7f7645a57200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-03-28T18:07:31.183 INFO:journalctl@ceph.mgr.y.smithi079.stdout:Mar 28 18:07:31 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mgr-y[36699]: 2024-03-28T18:07:31.051+0000 7f7645a57200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-03-28T18:07:31.659 INFO:journalctl@ceph.mgr.y.smithi079.stdout:Mar 28 18:07:31 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mgr-y[36699]: 2024-03-28T18:07:31.334+0000 7f7645a57200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-03-28T18:07:32.281 INFO:journalctl@ceph.mgr.x.smithi022.stdout:Mar 28 18:07:31 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mgr-x[32211]: 2024-03-28T18:07:31.784+0000 7f07c85c6200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-03-28T18:07:32.424 INFO:journalctl@ceph.mgr.z.smithi134.stdout:Mar 28 18:07:32 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mgr-z[37033]: 2024-03-28T18:07:32.035+0000 7ff03b0c1200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-03-28T18:07:32.432 INFO:journalctl@ceph.mgr.y.smithi079.stdout:Mar 28 18:07:31 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mgr-y[36699]: 2024-03-28T18:07:31.659+0000 7f7645a57200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-03-28T18:07:32.432 INFO:journalctl@ceph.mgr.y.smithi079.stdout:Mar 28 18:07:32 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mgr-y[36699]: 2024-03-28T18:07:32.094+0000 7f7645a57200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-03-28T18:07:32.781 INFO:journalctl@ceph.mgr.x.smithi022.stdout:Mar 28 18:07:32 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mgr-x[32211]: 2024-03-28T18:07:32.509+0000 7f07c85c6200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-03-28T18:07:32.782 INFO:journalctl@ceph.mgr.x.smithi022.stdout:Mar 28 18:07:32 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mgr-x[32211]: 2024-03-28T18:07:32.646+0000 7f07c85c6200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-03-28T18:07:32.932 INFO:journalctl@ceph.mgr.y.smithi079.stdout:Mar 28 18:07:32 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mgr-y[36699]: 2024-03-28T18:07:32.499+0000 7f7645a57200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-03-28T18:07:32.978 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:32.978+0000 7f7ab1600640 1 -- 172.21.15.22:0/4196571610 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mgrmap(e 20) v1 ==== 136975+0+0 (secure 0 0 0) 0x7f7aa8098010 con 0x7f7ab4103720 2024-03-28T18:07:32.978 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:32.979+0000 7f7ab1600640 1 -- 172.21.15.22:0/4196571610 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7f7a9c0a28c0 msgr2=0x7f7a9c0a4d80 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-28T18:07:32.979 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:32.979+0000 7f7ab1600640 1 --2- 172.21.15.22:0/4196571610 >> [v2:172.21.15.22:6800/4201048392,v1:172.21.15.22:6801/4201048392] conn(0x7f7a9c0a28c0 0x7f7a9c0a4d80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:07:33.007 INFO:journalctl@ceph.mgr.z.smithi134.stdout:Mar 28 18:07:32 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mgr-z[37033]: 2024-03-28T18:07:32.756+0000 7ff03b0c1200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-03-28T18:07:33.007 INFO:journalctl@ceph.mgr.z.smithi134.stdout:Mar 28 18:07:32 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mgr-z[37033]: 2024-03-28T18:07:32.891+0000 7ff03b0c1200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-03-28T18:07:33.009 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:07:33 smithi134 ceph-mon[35037]: Active manager daemon x restarted 2024-03-28T18:07:33.009 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:07:33 smithi134 ceph-mon[35037]: Activating manager daemon x 2024-03-28T18:07:33.009 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:07:33 smithi134 ceph-mon[35037]: osdmap e93: 12 total, 12 up, 12 in 2024-03-28T18:07:33.009 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:07:33 smithi134 ceph-mon[35037]: mgrmap e20: x(active, starting, since 0.0321396s), standbys: z, y 2024-03-28T18:07:33.009 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:07:33 smithi134 ceph-mon[35037]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-28T18:07:33.009 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:07:33 smithi134 ceph-mon[35037]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-28T18:07:33.009 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:07:33 smithi134 ceph-mon[35037]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-28T18:07:33.010 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:07:33 smithi134 ceph-mon[35037]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-03-28T18:07:33.010 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:07:33 smithi134 ceph-mon[35037]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-03-28T18:07:33.010 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:07:33 smithi134 ceph-mon[35037]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-03-28T18:07:33.010 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:07:33 smithi134 ceph-mon[35037]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-03-28T18:07:33.010 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:07:33 smithi134 ceph-mon[35037]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-03-28T18:07:33.010 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:07:33 smithi134 ceph-mon[35037]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-03-28T18:07:33.010 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:07:33 smithi134 ceph-mon[35037]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-03-28T18:07:33.010 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:07:33 smithi134 ceph-mon[35037]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-03-28T18:07:33.010 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:07:33 smithi134 ceph-mon[35037]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-03-28T18:07:33.010 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:07:33 smithi134 ceph-mon[35037]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-03-28T18:07:33.010 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:07:33 smithi134 ceph-mon[35037]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-03-28T18:07:33.010 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:07:33 smithi134 ceph-mon[35037]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-03-28T18:07:33.010 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:07:33 smithi134 ceph-mon[35037]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-03-28T18:07:33.010 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:07:33 smithi134 ceph-mon[35037]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-03-28T18:07:33.011 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:07:33 smithi134 ceph-mon[35037]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-03-28T18:07:33.011 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:07:33 smithi134 ceph-mon[35037]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-28T18:07:33.011 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:07:33 smithi134 ceph-mon[35037]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-28T18:07:33.011 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:07:33 smithi134 ceph-mon[35037]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-28T18:07:33.011 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:07:33 smithi134 ceph-mon[35037]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-28T18:07:33.011 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:07:33 smithi134 ceph-mon[35037]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-28T18:07:33.011 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:07:33 smithi134 ceph-mon[35037]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-28T18:07:33.193 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:07:33 smithi079 ceph-mon[35312]: Active manager daemon x restarted 2024-03-28T18:07:33.193 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:07:33 smithi079 ceph-mon[35312]: Activating manager daemon x 2024-03-28T18:07:33.193 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:07:33 smithi079 ceph-mon[35312]: osdmap e93: 12 total, 12 up, 12 in 2024-03-28T18:07:33.193 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:07:33 smithi079 ceph-mon[35312]: mgrmap e20: x(active, starting, since 0.0321396s), standbys: z, y 2024-03-28T18:07:33.194 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:07:33 smithi079 ceph-mon[35312]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-28T18:07:33.194 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:07:33 smithi079 ceph-mon[35312]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-28T18:07:33.194 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:07:33 smithi079 ceph-mon[35312]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-28T18:07:33.194 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:07:33 smithi079 ceph-mon[35312]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-03-28T18:07:33.194 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:07:33 smithi079 ceph-mon[35312]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-03-28T18:07:33.194 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:07:33 smithi079 ceph-mon[35312]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-03-28T18:07:33.194 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:07:33 smithi079 ceph-mon[35312]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-03-28T18:07:33.194 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:07:33 smithi079 ceph-mon[35312]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-03-28T18:07:33.194 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:07:33 smithi079 ceph-mon[35312]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-03-28T18:07:33.194 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:07:33 smithi079 ceph-mon[35312]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-03-28T18:07:33.194 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:07:33 smithi079 ceph-mon[35312]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-03-28T18:07:33.195 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:07:33 smithi079 ceph-mon[35312]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-03-28T18:07:33.195 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:07:33 smithi079 ceph-mon[35312]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-03-28T18:07:33.195 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:07:33 smithi079 ceph-mon[35312]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-03-28T18:07:33.195 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:07:33 smithi079 ceph-mon[35312]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-03-28T18:07:33.195 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:07:33 smithi079 ceph-mon[35312]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-03-28T18:07:33.195 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:07:33 smithi079 ceph-mon[35312]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-03-28T18:07:33.195 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:07:33 smithi079 ceph-mon[35312]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-03-28T18:07:33.195 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:07:33 smithi079 ceph-mon[35312]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-28T18:07:33.195 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:07:33 smithi079 ceph-mon[35312]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-28T18:07:33.195 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:07:33 smithi079 ceph-mon[35312]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-28T18:07:33.195 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:07:33 smithi079 ceph-mon[35312]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-28T18:07:33.195 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:07:33 smithi079 ceph-mon[35312]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-28T18:07:33.195 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:07:33 smithi079 ceph-mon[35312]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-28T18:07:33.195 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:07:33 smithi079 ceph-mon[35312]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-28T18:07:33.195 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:07:33 smithi079 ceph-mon[35312]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-28T18:07:33.196 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:07:33 smithi079 ceph-mon[35312]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-28T18:07:33.196 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:07:33 smithi079 ceph-mon[35312]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-28T18:07:33.196 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:07:33 smithi079 ceph-mon[35312]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-03-28T18:07:33.196 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:07:33 smithi079 ceph-mon[35312]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-28T18:07:33.196 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:07:33 smithi079 ceph-mon[35312]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-03-28T18:07:33.196 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:07:33 smithi079 ceph-mon[35312]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-03-28T18:07:33.196 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:07:33 smithi079 ceph-mon[35312]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-03-28T18:07:33.196 INFO:journalctl@ceph.mgr.y.smithi079.stdout:Mar 28 18:07:32 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mgr-y[36699]: 2024-03-28T18:07:32.933+0000 7f7645a57200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-03-28T18:07:33.282 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:07:33 smithi022 ceph-mon[31985]: Active manager daemon x restarted 2024-03-28T18:07:33.282 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:07:33 smithi022 ceph-mon[31985]: Activating manager daemon x 2024-03-28T18:07:33.282 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:07:33 smithi022 ceph-mon[31985]: osdmap e93: 12 total, 12 up, 12 in 2024-03-28T18:07:33.282 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:07:33 smithi022 ceph-mon[31985]: mgrmap e20: x(active, starting, since 0.0321396s), standbys: z, y 2024-03-28T18:07:33.282 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:07:33 smithi022 ceph-mon[31985]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-28T18:07:33.282 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:07:33 smithi022 ceph-mon[31985]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-28T18:07:33.282 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:07:33 smithi022 ceph-mon[31985]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-28T18:07:33.282 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:07:33 smithi022 ceph-mon[31985]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-03-28T18:07:33.282 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:07:33 smithi022 ceph-mon[31985]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-03-28T18:07:33.282 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:07:33 smithi022 ceph-mon[31985]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-03-28T18:07:33.282 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:07:33 smithi022 ceph-mon[31985]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-03-28T18:07:33.283 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:07:33 smithi022 ceph-mon[31985]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-03-28T18:07:33.283 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:07:33 smithi022 ceph-mon[31985]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-03-28T18:07:33.283 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:07:33 smithi022 ceph-mon[31985]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-03-28T18:07:33.283 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:07:33 smithi022 ceph-mon[31985]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-03-28T18:07:33.283 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:07:33 smithi022 ceph-mon[31985]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-03-28T18:07:33.283 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:07:33 smithi022 ceph-mon[31985]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-03-28T18:07:33.283 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:07:33 smithi022 ceph-mon[31985]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-03-28T18:07:33.283 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:07:33 smithi022 ceph-mon[31985]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-03-28T18:07:33.283 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:07:33 smithi022 ceph-mon[31985]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-03-28T18:07:33.283 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:07:33 smithi022 ceph-mon[31985]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-03-28T18:07:33.283 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:07:33 smithi022 ceph-mon[31985]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-03-28T18:07:33.283 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:07:33 smithi022 ceph-mon[31985]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-28T18:07:33.283 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:07:33 smithi022 ceph-mon[31985]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-28T18:07:33.283 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:07:33 smithi022 ceph-mon[31985]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-28T18:07:33.283 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:07:33 smithi022 ceph-mon[31985]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-28T18:07:33.284 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:07:33 smithi022 ceph-mon[31985]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-28T18:07:33.284 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:07:33 smithi022 ceph-mon[31985]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-28T18:07:33.284 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:07:33 smithi022 ceph-mon[31985]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-28T18:07:33.284 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:07:33 smithi022 ceph-mon[31985]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-28T18:07:33.284 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:07:33 smithi022 ceph-mon[31985]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-28T18:07:33.284 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:07:33 smithi022 ceph-mon[31985]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-28T18:07:33.284 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:07:33 smithi022 ceph-mon[31985]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-03-28T18:07:33.284 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:07:33 smithi022 ceph-mon[31985]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-28T18:07:33.284 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:07:33 smithi022 ceph-mon[31985]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-03-28T18:07:33.284 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:07:33 smithi022 ceph-mon[31985]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-03-28T18:07:33.284 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:07:33 smithi022 ceph-mon[31985]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-03-28T18:07:33.284 INFO:journalctl@ceph.mgr.x.smithi022.stdout:Mar 28 18:07:32 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mgr-x[32211]: 2024-03-28T18:07:32.944+0000 7f07c85c6200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-03-28T18:07:33.424 INFO:journalctl@ceph.mgr.z.smithi134.stdout:Mar 28 18:07:33 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mgr-z[37033]: 2024-03-28T18:07:33.218+0000 7ff03b0c1200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-03-28T18:07:33.425 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:07:33 smithi134 ceph-mon[35037]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-28T18:07:33.425 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:07:33 smithi134 ceph-mon[35037]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-28T18:07:33.425 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:07:33 smithi134 ceph-mon[35037]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-28T18:07:33.425 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:07:33 smithi134 ceph-mon[35037]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-28T18:07:33.425 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:07:33 smithi134 ceph-mon[35037]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-03-28T18:07:33.425 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:07:33 smithi134 ceph-mon[35037]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-28T18:07:33.426 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:07:33 smithi134 ceph-mon[35037]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-03-28T18:07:33.426 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:07:33 smithi134 ceph-mon[35037]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-03-28T18:07:33.426 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:07:33 smithi134 ceph-mon[35037]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-03-28T18:07:33.783 INFO:journalctl@ceph.mgr.x.smithi022.stdout:Mar 28 18:07:33 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mgr-x[32211]: 2024-03-28T18:07:33.347+0000 7f071ae00700 -1 client.0 error registering admin socket command: (17) File exists 2024-03-28T18:07:33.783 INFO:journalctl@ceph.mgr.x.smithi022.stdout:Mar 28 18:07:33 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mgr-x[32211]: 2024-03-28T18:07:33.347+0000 7f071ae00700 -1 client.0 error registering admin socket command: (17) File exists 2024-03-28T18:07:33.783 INFO:journalctl@ceph.mgr.x.smithi022.stdout:Mar 28 18:07:33 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mgr-x[32211]: 2024-03-28T18:07:33.347+0000 7f071ae00700 -1 client.0 error registering admin socket command: (17) File exists 2024-03-28T18:07:33.783 INFO:journalctl@ceph.mgr.x.smithi022.stdout:Mar 28 18:07:33 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mgr-x[32211]: 2024-03-28T18:07:33.347+0000 7f071ae00700 -1 client.0 error registering admin socket command: (17) File exists 2024-03-28T18:07:33.783 INFO:journalctl@ceph.mgr.x.smithi022.stdout:Mar 28 18:07:33 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mgr-x[32211]: 2024-03-28T18:07:33.347+0000 7f071ae00700 -1 client.0 error registering admin socket command: (17) File exists 2024-03-28T18:07:33.783 INFO:journalctl@ceph.mgr.x.smithi022.stdout:Mar 28 18:07:33 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mgr-x[32211]: 2024-03-28T18:07:33.348+0000 7f071ea00700 -1 client.0 error registering admin socket command: (17) File exists 2024-03-28T18:07:33.783 INFO:journalctl@ceph.mgr.x.smithi022.stdout:Mar 28 18:07:33 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mgr-x[32211]: 2024-03-28T18:07:33.348+0000 7f071ea00700 -1 client.0 error registering admin socket command: (17) File exists 2024-03-28T18:07:33.784 INFO:journalctl@ceph.mgr.x.smithi022.stdout:Mar 28 18:07:33 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mgr-x[32211]: 2024-03-28T18:07:33.348+0000 7f071ea00700 -1 client.0 error registering admin socket command: (17) File exists 2024-03-28T18:07:33.784 INFO:journalctl@ceph.mgr.x.smithi022.stdout:Mar 28 18:07:33 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mgr-x[32211]: 2024-03-28T18:07:33.348+0000 7f071ea00700 -1 client.0 error registering admin socket command: (17) File exists 2024-03-28T18:07:33.784 INFO:journalctl@ceph.mgr.x.smithi022.stdout:Mar 28 18:07:33 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mgr-x[32211]: 2024-03-28T18:07:33.348+0000 7f071ea00700 -1 client.0 error registering admin socket command: (17) File exists 2024-03-28T18:07:34.046 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:34.046+0000 7f7ab1600640 1 -- 172.21.15.22:0/4196571610 <== mon.0 v2:172.21.15.22:3300/0 8 ==== mgrmap(e 21) v1 ==== 137102+0+0 (secure 0 0 0) 0x7f7aa80ccaf0 con 0x7f7ab4103720 2024-03-28T18:07:34.048 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:34.048+0000 7f7ab1600640 1 --2- 172.21.15.22:0/4196571610 >> [v2:172.21.15.22:6800/1357365082,v1:172.21.15.22:6801/1357365082] conn(0x7f7a9c051640 0x7f7a9c051a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:07:34.048 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:34.048+0000 7f7ab1600640 1 -- 172.21.15.22:0/4196571610 --> [v2:172.21.15.22:6800/1357365082,v1:172.21.15.22:6801/1357365082] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}) v1 -- 0x7f7a9c00fde0 con 0x7f7a9c051640 2024-03-28T18:07:34.055 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:34.055+0000 7f7ab3e00640 1 --2- 172.21.15.22:0/4196571610 >> [v2:172.21.15.22:6800/1357365082,v1:172.21.15.22:6801/1357365082] conn(0x7f7a9c051640 0x7f7a9c051a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:07:34.060 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:34.060+0000 7f7ab3e00640 1 --2- 172.21.15.22:0/4196571610 >> [v2:172.21.15.22:6800/1357365082,v1:172.21.15.22:6801/1357365082] conn(0x7f7a9c051640 0x7f7a9c051a20 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f7aa0000c00 tx=0x7f7aa002c830 comp rx=0 tx=0).ready entity=mgr.15522 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:07:34.169 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:34.169+0000 7f7ab1600640 1 -- 172.21.15.22:0/4196571610 <== mgr.15522 v2:172.21.15.22:6800/1357365082 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+24 (secure 0 0 0) 0x7f7a9c00fde0 con 0x7f7a9c051640 2024-03-28T18:07:34.169 INFO:teuthology.orchestra.run.smithi022.stdout:Schedule set for path / 2024-03-28T18:07:34.172 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:34.172+0000 7f7aba200640 1 -- 172.21.15.22:0/4196571610 >> [v2:172.21.15.22:6800/1357365082,v1:172.21.15.22:6801/1357365082] conn(0x7f7a9c051640 msgr2=0x7f7a9c051a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:07:34.172 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:34.172+0000 7f7aba200640 1 --2- 172.21.15.22:0/4196571610 >> [v2:172.21.15.22:6800/1357365082,v1:172.21.15.22:6801/1357365082] conn(0x7f7a9c051640 0x7f7a9c051a20 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f7aa0000c00 tx=0x7f7aa002c830 comp rx=0 tx=0).stop 2024-03-28T18:07:34.172 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:34.172+0000 7f7aba200640 1 -- 172.21.15.22:0/4196571610 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7ab4103720 msgr2=0x7f7ab41999e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:07:34.172 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:34.172+0000 7f7aba200640 1 --2- 172.21.15.22:0/4196571610 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7ab4103720 0x7f7ab41999e0 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f7aa800e9c0 tx=0x7f7aa800ee80 comp rx=0 tx=0).stop 2024-03-28T18:07:34.172 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:34.172+0000 7f7ab9800640 1 -- 172.21.15.22:0/4196571610 reap_dead start 2024-03-28T18:07:34.172 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:34.172+0000 7f7aba200640 1 -- 172.21.15.22:0/4196571610 shutdown_connections 2024-03-28T18:07:34.172 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:34.172+0000 7f7aba200640 1 -- 172.21.15.22:0/4196571610 >> 172.21.15.22:0/4196571610 conn(0x7f7ab40fecb0 msgr2=0x7f7ab4107bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:07:34.172 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:34.172+0000 7f7aba200640 1 -- 172.21.15.22:0/4196571610 shutdown_connections 2024-03-28T18:07:34.172 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:34.172+0000 7f7aba200640 1 -- 172.21.15.22:0/4196571610 wait complete. 2024-03-28T18:07:34.189 DEBUG:teuthology.orchestra.run.smithi022:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule retention add --fs=cephfs --path=/ --retention-spec-or-period=6m3h' 2024-03-28T18:07:34.281 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:07:34 smithi022 ceph-mon[31985]: Manager daemon x is now available 2024-03-28T18:07:34.281 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:07:34 smithi022 ceph-mon[31985]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T18:07:34.281 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:07:34 smithi022 ceph-mon[31985]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T18:07:34.282 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:07:34 smithi022 ceph-mon[31985]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-03-28T18:07:34.282 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:07:34 smithi022 ceph-mon[31985]: from='mgr.15522 ' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-03-28T18:07:34.282 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:07:34 smithi022 ceph-mon[31985]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-03-28T18:07:34.282 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:07:34 smithi022 ceph-mon[31985]: from='mgr.15522 ' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-03-28T18:07:34.282 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:07:34 smithi022 ceph-mon[31985]: Standby manager daemon z restarted 2024-03-28T18:07:34.282 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:07:34 smithi022 ceph-mon[31985]: Standby manager daemon z started 2024-03-28T18:07:34.282 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:07:34 smithi022 ceph-mon[31985]: from='mgr.? 172.21.15.134:0/355725715' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-03-28T18:07:34.282 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:07:34 smithi022 ceph-mon[31985]: from='mgr.? 172.21.15.134:0/355725715' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-03-28T18:07:34.282 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:07:34 smithi022 ceph-mon[31985]: from='mgr.? 172.21.15.134:0/355725715' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-03-28T18:07:34.282 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:07:34 smithi022 ceph-mon[31985]: from='mgr.? 172.21.15.134:0/355725715' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-03-28T18:07:34.381 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:34.381+0000 7fe6c2400640 1 -- 172.21.15.22:0/1649520069 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe6bc1021e0 msgr2=0x7fe6bc1045d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:07:34.381 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:34.381+0000 7fe6c2400640 1 --2- 172.21.15.22:0/1649520069 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe6bc1021e0 0x7fe6bc1045d0 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7fe6ac009900 tx=0x7fe6ac02bc30 comp rx=0 tx=0).stop 2024-03-28T18:07:34.381 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:34.381+0000 7fe6c2400640 1 -- 172.21.15.22:0/1649520069 shutdown_connections 2024-03-28T18:07:34.381 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:34.381+0000 7fe6c2400640 1 --2- 172.21.15.22:0/1649520069 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe6bc107440 0x7fe6bc109830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:07:34.381 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:34.381+0000 7fe6c2400640 1 --2- 172.21.15.22:0/1649520069 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fe6bc104b10 0x7fe6bc106f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:07:34.381 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:34.381+0000 7fe6c2400640 1 --2- 172.21.15.22:0/1649520069 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe6bc1021e0 0x7fe6bc1045d0 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:07:34.381 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:34.381+0000 7fe6c2400640 1 -- 172.21.15.22:0/1649520069 >> 172.21.15.22:0/1649520069 conn(0x7fe6bc0fbb30 msgr2=0x7fe6bc0fdf70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:07:34.381 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:34.381+0000 7fe6c2400640 1 -- 172.21.15.22:0/1649520069 shutdown_connections 2024-03-28T18:07:34.381 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:34.381+0000 7fe6c2400640 1 -- 172.21.15.22:0/1649520069 wait complete. 2024-03-28T18:07:34.386 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:34.382+0000 7fe6c2400640 1 Processor -- start 2024-03-28T18:07:34.386 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:34.382+0000 7fe6c2400640 1 -- start start 2024-03-28T18:07:34.386 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:34.382+0000 7fe6c2400640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe6bc1021e0 0x7fe6bc101b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:07:34.386 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:34.382+0000 7fe6c2400640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe6bc104b10 0x7fe6bc100190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:07:34.386 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:34.382+0000 7fe6c2400640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fe6bc107440 0x7fe6bc1006d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:07:34.386 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:34.382+0000 7fe6c2400640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6bc10d390 con 0x7fe6bc1021e0 2024-03-28T18:07:34.386 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:34.382+0000 7fe6c2400640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6bc10d210 con 0x7fe6bc107440 2024-03-28T18:07:34.386 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:34.382+0000 7fe6c2400640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6bc10d510 con 0x7fe6bc104b10 2024-03-28T18:07:34.386 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:34.383+0000 7fe6bb400640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe6bc1021e0 0x7fe6bc101b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:07:34.386 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:34.383+0000 7fe6bb400640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe6bc1021e0 0x7fe6bc101b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:39838/0 (socket says 172.21.15.22:39838) 2024-03-28T18:07:34.386 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:34.383+0000 7fe6bb400640 1 -- 172.21.15.22:0/1628478587 learned_addr learned my addr 172.21.15.22:0/1628478587 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-28T18:07:34.386 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:34.383+0000 7fe6baa00640 1 --2- 172.21.15.22:0/1628478587 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe6bc104b10 0x7fe6bc100190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:07:34.387 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:34.383+0000 7fe6bb400640 1 -- 172.21.15.22:0/1628478587 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe6bc104b10 msgr2=0x7fe6bc100190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:07:34.387 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:34.383+0000 7fe6bb400640 1 --2- 172.21.15.22:0/1628478587 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe6bc104b10 0x7fe6bc100190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:07:34.387 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:34.383+0000 7fe6bb400640 1 -- 172.21.15.22:0/1628478587 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fe6bc107440 msgr2=0x7fe6bc1006d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:07:34.387 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:34.383+0000 7fe6bb400640 1 --2- 172.21.15.22:0/1628478587 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fe6bc107440 0x7fe6bc1006d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:07:34.387 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:34.383+0000 7fe6bb400640 1 -- 172.21.15.22:0/1628478587 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe6bc100f60 con 0x7fe6bc1021e0 2024-03-28T18:07:34.387 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:34.384+0000 7fe6bb400640 1 --2- 172.21.15.22:0/1628478587 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe6bc1021e0 0x7fe6bc101b50 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7fe6ac000c00 tx=0x7fe6ac005e40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:07:34.387 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:34.384+0000 7fe6b3e00640 1 -- 172.21.15.22:0/1628478587 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6ac043020 con 0x7fe6bc1021e0 2024-03-28T18:07:34.387 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:34.384+0000 7fe6b3e00640 1 -- 172.21.15.22:0/1628478587 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7fe6ac005470 con 0x7fe6bc1021e0 2024-03-28T18:07:34.387 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:34.384+0000 7fe6c2400640 1 -- 172.21.15.22:0/1628478587 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe6bc197600 con 0x7fe6bc1021e0 2024-03-28T18:07:34.387 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:34.384+0000 7fe6b3e00640 1 -- 172.21.15.22:0/1628478587 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6ac003c90 con 0x7fe6bc1021e0 2024-03-28T18:07:34.387 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:34.384+0000 7fe6c2400640 1 -- 172.21.15.22:0/1628478587 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe6bc197ba0 con 0x7fe6bc1021e0 2024-03-28T18:07:34.388 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:34.385+0000 7fe6c2400640 1 -- 172.21.15.22:0/1628478587 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe6bc0fd620 con 0x7fe6bc1021e0 2024-03-28T18:07:34.388 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:34.388+0000 7fe6b3e00640 1 -- 172.21.15.22:0/1628478587 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 21) v1 ==== 137102+0+0 (secure 0 0 0) 0x7fe6ac046020 con 0x7fe6bc1021e0 2024-03-28T18:07:34.389 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:34.390+0000 7fe6b3e00640 1 --2- 172.21.15.22:0/1628478587 >> [v2:172.21.15.22:6800/1357365082,v1:172.21.15.22:6801/1357365082] conn(0x7fe6a00a2730 0x7fe6a00a4bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:07:34.390 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:34.390+0000 7fe6b3e00640 1 -- 172.21.15.22:0/1628478587 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 8411+0+0 (secure 0 0 0) 0x7fe6ac0f0870 con 0x7fe6bc1021e0 2024-03-28T18:07:34.390 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:34.390+0000 7fe6baa00640 1 --2- 172.21.15.22:0/1628478587 >> [v2:172.21.15.22:6800/1357365082,v1:172.21.15.22:6801/1357365082] conn(0x7fe6a00a2730 0x7fe6a00a4bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:07:34.391 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:34.391+0000 7fe6baa00640 1 --2- 172.21.15.22:0/1628478587 >> [v2:172.21.15.22:6800/1357365082,v1:172.21.15.22:6801/1357365082] conn(0x7fe6a00a2730 0x7fe6a00a4bf0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fe6bc1018c0 tx=0x7fe6a8005f90 comp rx=0 tx=0).ready entity=mgr.15522 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:07:34.394 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:34.395+0000 7fe6b3e00640 1 -- 172.21.15.22:0/1628478587 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fe6ac04d020 con 0x7fe6bc1021e0 2024-03-28T18:07:34.397 INFO:journalctl@ceph.mgr.y.smithi079.stdout:Mar 28 18:07:34 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mgr-y[36699]: 2024-03-28T18:07:34.397+0000 7f7645a57200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-03-28T18:07:34.398 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:07:34 smithi079 ceph-mon[35312]: Manager daemon x is now available 2024-03-28T18:07:34.398 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:07:34 smithi079 ceph-mon[35312]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T18:07:34.398 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:07:34 smithi079 ceph-mon[35312]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T18:07:34.398 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:07:34 smithi079 ceph-mon[35312]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-03-28T18:07:34.398 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:07:34 smithi079 ceph-mon[35312]: from='mgr.15522 ' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-03-28T18:07:34.398 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:07:34 smithi079 ceph-mon[35312]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-03-28T18:07:34.398 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:07:34 smithi079 ceph-mon[35312]: from='mgr.15522 ' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-03-28T18:07:34.399 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:07:34 smithi079 ceph-mon[35312]: Standby manager daemon z restarted 2024-03-28T18:07:34.399 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:07:34 smithi079 ceph-mon[35312]: Standby manager daemon z started 2024-03-28T18:07:34.399 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:07:34 smithi079 ceph-mon[35312]: from='mgr.? 172.21.15.134:0/355725715' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-03-28T18:07:34.399 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:07:34 smithi079 ceph-mon[35312]: from='mgr.? 172.21.15.134:0/355725715' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-03-28T18:07:34.399 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:07:34 smithi079 ceph-mon[35312]: from='mgr.? 172.21.15.134:0/355725715' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-03-28T18:07:34.399 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:07:34 smithi079 ceph-mon[35312]: from='mgr.? 172.21.15.134:0/355725715' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-03-28T18:07:34.424 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:07:34 smithi134 ceph-mon[35037]: Manager daemon x is now available 2024-03-28T18:07:34.424 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:07:34 smithi134 ceph-mon[35037]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T18:07:34.424 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:07:34 smithi134 ceph-mon[35037]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T18:07:34.425 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:07:34 smithi134 ceph-mon[35037]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-03-28T18:07:34.425 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:07:34 smithi134 ceph-mon[35037]: from='mgr.15522 ' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-03-28T18:07:34.425 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:07:34 smithi134 ceph-mon[35037]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-03-28T18:07:34.425 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:07:34 smithi134 ceph-mon[35037]: from='mgr.15522 ' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-03-28T18:07:34.425 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:07:34 smithi134 ceph-mon[35037]: Standby manager daemon z restarted 2024-03-28T18:07:34.425 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:07:34 smithi134 ceph-mon[35037]: Standby manager daemon z started 2024-03-28T18:07:34.425 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:07:34 smithi134 ceph-mon[35037]: from='mgr.? 172.21.15.134:0/355725715' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-03-28T18:07:34.425 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:07:34 smithi134 ceph-mon[35037]: from='mgr.? 172.21.15.134:0/355725715' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-03-28T18:07:34.425 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:07:34 smithi134 ceph-mon[35037]: from='mgr.? 172.21.15.134:0/355725715' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-03-28T18:07:34.425 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:07:34 smithi134 ceph-mon[35037]: from='mgr.? 172.21.15.134:0/355725715' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-03-28T18:07:34.625 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:34.625+0000 7fe6c2400640 1 -- 172.21.15.22:0/1628478587 --> [v2:172.21.15.22:6800/1357365082,v1:172.21.15.22:6801/1357365082] -- mgr_command(tid 0: {"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6m3h", "target": ["mon-mgr", ""]}) v1 -- 0x7fe6bc061c40 con 0x7fe6a00a2730 2024-03-28T18:07:34.752 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:34.752+0000 7fe6b3e00640 1 -- 172.21.15.22:0/1628478587 <== mgr.15522 v2:172.21.15.22:6800/1357365082 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+26 (secure 0 0 0) 0x7fe6bc061c40 con 0x7fe6a00a2730 2024-03-28T18:07:34.752 INFO:teuthology.orchestra.run.smithi022.stdout:Retention added to path / 2024-03-28T18:07:34.754 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:34.754+0000 7fe6c2400640 1 -- 172.21.15.22:0/1628478587 >> [v2:172.21.15.22:6800/1357365082,v1:172.21.15.22:6801/1357365082] conn(0x7fe6a00a2730 msgr2=0x7fe6a00a4bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:07:34.754 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:34.754+0000 7fe6c2400640 1 --2- 172.21.15.22:0/1628478587 >> [v2:172.21.15.22:6800/1357365082,v1:172.21.15.22:6801/1357365082] conn(0x7fe6a00a2730 0x7fe6a00a4bf0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fe6bc1018c0 tx=0x7fe6a8005f90 comp rx=0 tx=0).stop 2024-03-28T18:07:34.754 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:34.754+0000 7fe6c2400640 1 -- 172.21.15.22:0/1628478587 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe6bc1021e0 msgr2=0x7fe6bc101b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:07:34.755 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:34.754+0000 7fe6c2400640 1 --2- 172.21.15.22:0/1628478587 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe6bc1021e0 0x7fe6bc101b50 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7fe6ac000c00 tx=0x7fe6ac005e40 comp rx=0 tx=0).stop 2024-03-28T18:07:34.755 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:34.754+0000 7fe6c2400640 1 -- 172.21.15.22:0/1628478587 shutdown_connections 2024-03-28T18:07:34.755 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:34.754+0000 7fe6c2400640 1 --2- 172.21.15.22:0/1628478587 >> [v2:172.21.15.22:6800/1357365082,v1:172.21.15.22:6801/1357365082] conn(0x7fe6a00a2730 0x7fe6a00a4bf0 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:07:34.755 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:34.754+0000 7fe6c2400640 1 --2- 172.21.15.22:0/1628478587 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fe6bc107440 0x7fe6bc1006d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:07:34.755 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:34.754+0000 7fe6c2400640 1 --2- 172.21.15.22:0/1628478587 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe6bc104b10 0x7fe6bc100190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:07:34.755 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:34.754+0000 7fe6c2400640 1 --2- 172.21.15.22:0/1628478587 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe6bc1021e0 0x7fe6bc101b50 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:07:34.755 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:34.754+0000 7fe6c2400640 1 -- 172.21.15.22:0/1628478587 >> 172.21.15.22:0/1628478587 conn(0x7fe6bc0fbb30 msgr2=0x7fe6bc102b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:07:34.755 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:34.754+0000 7fe6c2400640 1 -- 172.21.15.22:0/1628478587 shutdown_connections 2024-03-28T18:07:34.755 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:34.754+0000 7fe6c2400640 1 -- 172.21.15.22:0/1628478587 wait complete. 2024-03-28T18:07:34.770 DEBUG:teuthology.orchestra.run.smithi022:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule status --fs=cephfs --path=/' 2024-03-28T18:07:34.965 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:34.965+0000 7f5fc3a00640 1 -- 172.21.15.22:0/2271362328 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f5fbc103900 msgr2=0x7f5fbc103d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:07:34.966 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:34.965+0000 7f5fc3a00640 1 --2- 172.21.15.22:0/2271362328 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f5fbc103900 0x7f5fbc103d20 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f5fb0009900 tx=0x7f5fb002bc30 comp rx=0 tx=0).stop 2024-03-28T18:07:34.966 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:34.965+0000 7f5fc3a00640 1 -- 172.21.15.22:0/2271362328 shutdown_connections 2024-03-28T18:07:34.966 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:34.965+0000 7f5fc3a00640 1 --2- 172.21.15.22:0/2271362328 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5fbc1054b0 0x7f5fbc10bd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:07:34.966 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:34.965+0000 7f5fc3a00640 1 --2- 172.21.15.22:0/2271362328 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5fbc104af0 0x7f5fbc104f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:07:34.966 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:34.965+0000 7f5fc3a00640 1 --2- 172.21.15.22:0/2271362328 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f5fbc103900 0x7f5fbc103d20 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:07:34.966 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:34.966+0000 7f5fc3a00640 1 -- 172.21.15.22:0/2271362328 >> 172.21.15.22:0/2271362328 conn(0x7f5fbc0fee90 msgr2=0x7f5fbc1012b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:07:34.966 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:34.966+0000 7f5fc3a00640 1 -- 172.21.15.22:0/2271362328 shutdown_connections 2024-03-28T18:07:34.966 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:34.966+0000 7f5fc3a00640 1 -- 172.21.15.22:0/2271362328 wait complete. 2024-03-28T18:07:34.968 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:34.966+0000 7f5fc3a00640 1 Processor -- start 2024-03-28T18:07:34.968 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:34.967+0000 7f5fc3a00640 1 -- start start 2024-03-28T18:07:34.968 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:34.967+0000 7f5fc3a00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5fbc103900 0x7f5fbc199bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:07:34.968 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:34.967+0000 7f5fc3a00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5fbc104af0 0x7f5fbc19a110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:07:34.969 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:34.967+0000 7f5fc3a00640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f5fbc1054b0 0x7f5fbc1a1180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:07:34.969 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:34.967+0000 7f5fc3a00640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5fbc10f5b0 con 0x7f5fbc104af0 2024-03-28T18:07:34.969 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:34.967+0000 7f5fc3a00640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5fbc10f430 con 0x7f5fbc1054b0 2024-03-28T18:07:34.969 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:34.967+0000 7f5fc3a00640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5fbc10f730 con 0x7f5fbc103900 2024-03-28T18:07:34.969 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:34.967+0000 7f5fbbe00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5fbc104af0 0x7f5fbc19a110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:07:34.969 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:34.967+0000 7f5fbbe00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5fbc104af0 0x7f5fbc19a110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:39868/0 (socket says 172.21.15.22:39868) 2024-03-28T18:07:34.969 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:34.967+0000 7f5fbbe00640 1 -- 172.21.15.22:0/196765836 learned_addr learned my addr 172.21.15.22:0/196765836 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-28T18:07:34.969 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:34.967+0000 7f5fc1a00640 1 --2- 172.21.15.22:0/196765836 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f5fbc1054b0 0x7f5fbc1a1180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:07:34.969 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:34.967+0000 7f5fc1000640 1 --2- 172.21.15.22:0/196765836 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5fbc103900 0x7f5fbc199bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:07:34.969 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:34.968+0000 7f5fbbe00640 1 -- 172.21.15.22:0/196765836 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5fbc103900 msgr2=0x7f5fbc199bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:07:34.969 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:34.968+0000 7f5fbbe00640 1 --2- 172.21.15.22:0/196765836 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5fbc103900 0x7f5fbc199bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:07:34.969 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:34.968+0000 7f5fbbe00640 1 -- 172.21.15.22:0/196765836 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f5fbc1054b0 msgr2=0x7f5fbc1a1180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:07:34.969 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:34.968+0000 7f5fbbe00640 1 --2- 172.21.15.22:0/196765836 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f5fbc1054b0 0x7f5fbc1a1180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:07:34.969 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:34.968+0000 7f5fbbe00640 1 -- 172.21.15.22:0/196765836 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5fbc1a17f0 con 0x7f5fbc104af0 2024-03-28T18:07:34.970 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:34.968+0000 7f5fbbe00640 1 --2- 172.21.15.22:0/196765836 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5fbc104af0 0x7f5fbc19a110 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f5fa800e9f0 tx=0x7f5fa800edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:07:34.970 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:34.968+0000 7f5fb9600640 1 -- 172.21.15.22:0/196765836 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5fa800cd80 con 0x7f5fbc104af0 2024-03-28T18:07:34.970 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:34.968+0000 7f5fb9600640 1 -- 172.21.15.22:0/196765836 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f5fa80054f0 con 0x7f5fbc104af0 2024-03-28T18:07:34.970 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:34.968+0000 7f5fc3a00640 1 -- 172.21.15.22:0/196765836 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5fbc1a1ae0 con 0x7f5fbc104af0 2024-03-28T18:07:34.970 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:34.968+0000 7f5fb9600640 1 -- 172.21.15.22:0/196765836 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5fa8010690 con 0x7f5fbc104af0 2024-03-28T18:07:34.970 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:34.969+0000 7f5fc3a00640 1 -- 172.21.15.22:0/196765836 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5fbc1a1fe0 con 0x7f5fbc104af0 2024-03-28T18:07:34.970 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:34.970+0000 7f5fc3a00640 1 -- 172.21.15.22:0/196765836 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5fbc10abc0 con 0x7f5fbc104af0 2024-03-28T18:07:34.971 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:34.972+0000 7f5fb9600640 1 -- 172.21.15.22:0/196765836 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 21) v1 ==== 137102+0+0 (secure 0 0 0) 0x7f5fa8004f20 con 0x7f5fbc104af0 2024-03-28T18:07:34.973 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:34.973+0000 7f5fb9600640 1 --2- 172.21.15.22:0/196765836 >> [v2:172.21.15.22:6800/1357365082,v1:172.21.15.22:6801/1357365082] conn(0x7f5fa40a26b0 0x7f5fa40a4b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:07:34.973 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:34.974+0000 7f5fb9600640 1 -- 172.21.15.22:0/196765836 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 8411+0+0 (secure 0 0 0) 0x7f5fa80ceae0 con 0x7f5fbc104af0 2024-03-28T18:07:34.973 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:34.974+0000 7f5fc1000640 1 --2- 172.21.15.22:0/196765836 >> [v2:172.21.15.22:6800/1357365082,v1:172.21.15.22:6801/1357365082] conn(0x7f5fa40a26b0 0x7f5fa40a4b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:07:34.974 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:34.975+0000 7f5fc1000640 1 --2- 172.21.15.22:0/196765836 >> [v2:172.21.15.22:6800/1357365082,v1:172.21.15.22:6801/1357365082] conn(0x7f5fa40a26b0 0x7f5fa40a4b70 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f5fb002c170 tx=0x7f5fb0005a20 comp rx=0 tx=0).ready entity=mgr.15522 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:07:34.980 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:34.980+0000 7f5fb9600640 1 -- 172.21.15.22:0/196765836 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f5fa809bdc0 con 0x7f5fbc104af0 2024-03-28T18:07:35.063 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:35.063+0000 7f5fb9600640 1 -- 172.21.15.22:0/196765836 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mgrmap(e 22) v1 ==== 137410+0+0 (secure 0 0 0) 0x7f5fa8097d30 con 0x7f5fbc104af0 2024-03-28T18:07:35.175 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:07:35 smithi134 ceph-mon[35037]: mgrmap e21: x(active, since 1.09762s), standbys: z, y 2024-03-28T18:07:35.175 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:07:35 smithi134 ceph-mon[35037]: [28/Mar/2024:18:07:34] ENGINE Bus STARTING 2024-03-28T18:07:35.175 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:07:35 smithi134 ceph-mon[35037]: [28/Mar/2024:18:07:34] ENGINE Serving on http://172.21.15.22:8765 2024-03-28T18:07:35.214 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:35.214+0000 7f5fc3a00640 1 -- 172.21.15.22:0/196765836 --> [v2:172.21.15.22:6800/1357365082,v1:172.21.15.22:6801/1357365082] -- mgr_command(tid 0: {"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}) v1 -- 0x7f5fbc061ae0 con 0x7f5fa40a26b0 2024-03-28T18:07:35.227 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:35.227+0000 7f5fb9600640 1 -- 172.21.15.22:0/196765836 <== mgr.15522 v2:172.21.15.22:6800/1357365082 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+298 (secure 0 0 0) 0x7f5fbc061ae0 con 0x7f5fa40a26b0 2024-03-28T18:07:35.227 INFO:teuthology.orchestra.run.smithi022.stdout:{"fs": "cephfs", "subvol": null, "group": null, "path": "/", "rel_path": "/", "schedule": "1m", "retention": {"m": 6, "h": 3}, "start": "2024-03-28T00:00:00", "created": "2024-03-28T18:07:34", "first": null, "last": null, "last_pruned": null, "created_count": 0, "pruned_count": 0, "active": true} 2024-03-28T18:07:35.229 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:35.229+0000 7f5fc3a00640 1 -- 172.21.15.22:0/196765836 >> [v2:172.21.15.22:6800/1357365082,v1:172.21.15.22:6801/1357365082] conn(0x7f5fa40a26b0 msgr2=0x7f5fa40a4b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:07:35.229 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:35.229+0000 7f5fc3a00640 1 --2- 172.21.15.22:0/196765836 >> [v2:172.21.15.22:6800/1357365082,v1:172.21.15.22:6801/1357365082] conn(0x7f5fa40a26b0 0x7f5fa40a4b70 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f5fb002c170 tx=0x7f5fb0005a20 comp rx=0 tx=0).stop 2024-03-28T18:07:35.229 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:35.229+0000 7f5fc3a00640 1 -- 172.21.15.22:0/196765836 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5fbc104af0 msgr2=0x7f5fbc19a110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:07:35.229 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:35.229+0000 7f5fc3a00640 1 --2- 172.21.15.22:0/196765836 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5fbc104af0 0x7f5fbc19a110 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f5fa800e9f0 tx=0x7f5fa800edf0 comp rx=0 tx=0).stop 2024-03-28T18:07:35.230 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:35.230+0000 7f5fc3a00640 1 -- 172.21.15.22:0/196765836 shutdown_connections 2024-03-28T18:07:35.230 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:35.230+0000 7f5fc3a00640 1 --2- 172.21.15.22:0/196765836 >> [v2:172.21.15.22:6800/1357365082,v1:172.21.15.22:6801/1357365082] conn(0x7f5fa40a26b0 0x7f5fa40a4b70 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:07:35.230 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:35.230+0000 7f5fc3a00640 1 --2- 172.21.15.22:0/196765836 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f5fbc1054b0 0x7f5fbc1a1180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:07:35.230 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:35.230+0000 7f5fc3a00640 1 --2- 172.21.15.22:0/196765836 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5fbc104af0 0x7f5fbc19a110 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:07:35.230 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:35.230+0000 7f5fc3a00640 1 --2- 172.21.15.22:0/196765836 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5fbc103900 0x7f5fbc199bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:07:35.230 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:35.230+0000 7f5fc3a00640 1 -- 172.21.15.22:0/196765836 >> 172.21.15.22:0/196765836 conn(0x7f5fbc0fee90 msgr2=0x7f5fbc107d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:07:35.230 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:35.230+0000 7f5fc3a00640 1 -- 172.21.15.22:0/196765836 shutdown_connections 2024-03-28T18:07:35.230 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:35.230+0000 7f5fc3a00640 1 -- 172.21.15.22:0/196765836 wait complete. 2024-03-28T18:07:35.253 DEBUG:teuthology.orchestra.run.smithi022:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true' 2024-03-28T18:07:35.281 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:07:35 smithi022 ceph-mon[31985]: mgrmap e21: x(active, since 1.09762s), standbys: z, y 2024-03-28T18:07:35.281 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:07:35 smithi022 ceph-mon[31985]: [28/Mar/2024:18:07:34] ENGINE Bus STARTING 2024-03-28T18:07:35.281 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:07:35 smithi022 ceph-mon[31985]: [28/Mar/2024:18:07:34] ENGINE Serving on http://172.21.15.22:8765 2024-03-28T18:07:35.294 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:07:35 smithi079 ceph-mon[35312]: mgrmap e21: x(active, since 1.09762s), standbys: z, y 2024-03-28T18:07:35.294 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:07:35 smithi079 ceph-mon[35312]: [28/Mar/2024:18:07:34] ENGINE Bus STARTING 2024-03-28T18:07:35.295 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:07:35 smithi079 ceph-mon[35312]: [28/Mar/2024:18:07:34] ENGINE Serving on http://172.21.15.22:8765 2024-03-28T18:07:35.454 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:35.454+0000 7fe364400640 1 -- 172.21.15.22:0/3643754933 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe35c075660 msgr2=0x7fe35c073ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:07:35.454 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:35.454+0000 7fe364400640 1 --2- 172.21.15.22:0/3643754933 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe35c075660 0x7fe35c073ab0 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7fe350009900 tx=0x7fe35002bcd0 comp rx=0 tx=0).stop 2024-03-28T18:07:35.454 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:35.454+0000 7fe364400640 1 -- 172.21.15.22:0/3643754933 shutdown_connections 2024-03-28T18:07:35.454 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:35.454+0000 7fe364400640 1 --2- 172.21.15.22:0/3643754933 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fe35c109580 0x7fe35c10ba10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:07:35.454 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:35.454+0000 7fe364400640 1 --2- 172.21.15.22:0/3643754933 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe35c074080 0x7fe35c0744e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:07:35.455 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:35.454+0000 7fe364400640 1 --2- 172.21.15.22:0/3643754933 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe35c075660 0x7fe35c073ab0 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:07:35.455 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:35.454+0000 7fe364400640 1 -- 172.21.15.22:0/3643754933 >> 172.21.15.22:0/3643754933 conn(0x7fe35c0fcc60 msgr2=0x7fe35c0ff080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:07:35.455 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:35.454+0000 7fe364400640 1 -- 172.21.15.22:0/3643754933 shutdown_connections 2024-03-28T18:07:35.455 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:35.454+0000 7fe364400640 1 -- 172.21.15.22:0/3643754933 wait complete. 2024-03-28T18:07:35.457 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:35.455+0000 7fe364400640 1 Processor -- start 2024-03-28T18:07:35.457 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:35.455+0000 7fe364400640 1 -- start start 2024-03-28T18:07:35.457 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:35.455+0000 7fe364400640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe35c074080 0x7fe35c199b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:07:35.457 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:35.455+0000 7fe364400640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe35c075660 0x7fe35c19a0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:07:35.457 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:35.456+0000 7fe364400640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fe35c109580 0x7fe35c1a1110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:07:35.457 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:35.456+0000 7fe364400640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe35c10f590 con 0x7fe35c074080 2024-03-28T18:07:35.457 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:35.456+0000 7fe364400640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe35c10f410 con 0x7fe35c109580 2024-03-28T18:07:35.457 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:35.456+0000 7fe364400640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe35c10f710 con 0x7fe35c075660 2024-03-28T18:07:35.457 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:35.456+0000 7fe361a00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe35c074080 0x7fe35c199b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:07:35.458 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:35.456+0000 7fe362400640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fe35c109580 0x7fe35c1a1110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:07:35.458 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:35.456+0000 7fe362400640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fe35c109580 0x7fe35c1a1110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.22:54360/0 (socket says 172.21.15.22:54360) 2024-03-28T18:07:35.458 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:35.456+0000 7fe362400640 1 -- 172.21.15.22:0/1750471653 learned_addr learned my addr 172.21.15.22:0/1750471653 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-28T18:07:35.458 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:35.456+0000 7fe361000640 1 --2- 172.21.15.22:0/1750471653 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe35c075660 0x7fe35c19a0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:07:35.458 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:35.456+0000 7fe362400640 1 -- 172.21.15.22:0/1750471653 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe35c075660 msgr2=0x7fe35c19a0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:07:35.458 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:35.456+0000 7fe362400640 1 --2- 172.21.15.22:0/1750471653 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe35c075660 0x7fe35c19a0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:07:35.458 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:35.456+0000 7fe362400640 1 -- 172.21.15.22:0/1750471653 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe35c074080 msgr2=0x7fe35c199b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:07:35.458 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:35.456+0000 7fe362400640 1 --2- 172.21.15.22:0/1750471653 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe35c074080 0x7fe35c199b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:07:35.458 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:35.456+0000 7fe362400640 1 -- 172.21.15.22:0/1750471653 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe35c1a1780 con 0x7fe35c109580 2024-03-28T18:07:35.458 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:35.456+0000 7fe361a00640 1 --2- 172.21.15.22:0/1750471653 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe35c074080 0x7fe35c199b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-28T18:07:35.458 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:35.457+0000 7fe362400640 1 --2- 172.21.15.22:0/1750471653 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fe35c109580 0x7fe35c1a1110 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fe35400efd0 tx=0x7fe35400c6a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:07:35.458 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:35.457+0000 7fe35a000640 1 -- 172.21.15.22:0/1750471653 <== mon.2 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe354003b70 con 0x7fe35c109580 2024-03-28T18:07:35.458 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:35.457+0000 7fe364400640 1 -- 172.21.15.22:0/1750471653 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe35c1a1a70 con 0x7fe35c109580 2024-03-28T18:07:35.458 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:35.457+0000 7fe35a000640 1 -- 172.21.15.22:0/1750471653 <== mon.2 v2:172.21.15.79:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7fe354003d10 con 0x7fe35c109580 2024-03-28T18:07:35.458 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:35.457+0000 7fe35a000640 1 -- 172.21.15.22:0/1750471653 <== mon.2 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3540109f0 con 0x7fe35c109580 2024-03-28T18:07:35.459 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:35.457+0000 7fe364400640 1 -- 172.21.15.22:0/1750471653 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe35c1a1f70 con 0x7fe35c109580 2024-03-28T18:07:35.459 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:35.458+0000 7fe364400640 1 -- 172.21.15.22:0/1750471653 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe35c108340 con 0x7fe35c109580 2024-03-28T18:07:35.459 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:35.460+0000 7fe35a000640 1 -- 172.21.15.22:0/1750471653 <== mon.2 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 22) v1 ==== 137410+0+0 (secure 0 0 0) 0x7fe354010b90 con 0x7fe35c109580 2024-03-28T18:07:35.461 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:35.461+0000 7fe35a000640 1 --2- 172.21.15.22:0/1750471653 >> [v2:172.21.15.22:6800/1357365082,v1:172.21.15.22:6801/1357365082] conn(0x7fe3440a2a10 0x7fe3440a4ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:07:35.461 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:35.461+0000 7fe35a000640 1 -- 172.21.15.22:0/1750471653 <== mon.2 v2:172.21.15.79:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 8411+0+0 (secure 0 0 0) 0x7fe3540cf580 con 0x7fe35c109580 2024-03-28T18:07:35.461 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:35.462+0000 7fe361a00640 1 --2- 172.21.15.22:0/1750471653 >> [v2:172.21.15.22:6800/1357365082,v1:172.21.15.22:6801/1357365082] conn(0x7fe3440a2a10 0x7fe3440a4ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:07:35.462 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:35.462+0000 7fe361a00640 1 --2- 172.21.15.22:0/1750471653 >> [v2:172.21.15.22:6800/1357365082,v1:172.21.15.22:6801/1357365082] conn(0x7fe3440a2a10 0x7fe3440a4ed0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fe350000c00 tx=0x7fe35002c8c0 comp rx=0 tx=0).ready entity=mgr.15522 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:07:35.471 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:35.471+0000 7fe35a000640 1 -- 172.21.15.22:0/1750471653 <== mon.2 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fe35409c2f0 con 0x7fe35c109580 2024-03-28T18:07:35.555 INFO:journalctl@ceph.mgr.y.smithi079.stdout:Mar 28 18:07:35 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mgr-y[36699]: 2024-03-28T18:07:35.383+0000 7f7645a57200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-03-28T18:07:35.703 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:35.703+0000 7fe364400640 1 -- 172.21.15.22:0/1750471653 --> [v2:172.21.15.22:6800/1357365082,v1:172.21.15.22:6801/1357365082] -- mgr_command(tid 0: {"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}) v1 -- 0x7fe35c061ae0 con 0x7fe3440a2a10 2024-03-28T18:07:35.712 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:35.713+0000 7fe35a000640 1 -- 172.21.15.22:0/1750471653 <== mgr.15522 v2:172.21.15.22:6800/1357365082 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+10 (secure 0 0 0) 0x7fe35c061ae0 con 0x7fe3440a2a10 2024-03-28T18:07:35.712 INFO:teuthology.orchestra.run.smithi022.stdout:/ 1m 6m3h 2024-03-28T18:07:35.715 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:35.715+0000 7fe364400640 1 -- 172.21.15.22:0/1750471653 >> [v2:172.21.15.22:6800/1357365082,v1:172.21.15.22:6801/1357365082] conn(0x7fe3440a2a10 msgr2=0x7fe3440a4ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:07:35.715 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:35.715+0000 7fe364400640 1 --2- 172.21.15.22:0/1750471653 >> [v2:172.21.15.22:6800/1357365082,v1:172.21.15.22:6801/1357365082] conn(0x7fe3440a2a10 0x7fe3440a4ed0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fe350000c00 tx=0x7fe35002c8c0 comp rx=0 tx=0).stop 2024-03-28T18:07:35.715 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:35.715+0000 7fe364400640 1 -- 172.21.15.22:0/1750471653 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fe35c109580 msgr2=0x7fe35c1a1110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:07:35.715 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:35.715+0000 7fe364400640 1 --2- 172.21.15.22:0/1750471653 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fe35c109580 0x7fe35c1a1110 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fe35400efd0 tx=0x7fe35400c6a0 comp rx=0 tx=0).stop 2024-03-28T18:07:35.715 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:35.715+0000 7fe364400640 1 -- 172.21.15.22:0/1750471653 shutdown_connections 2024-03-28T18:07:35.715 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:35.715+0000 7fe364400640 1 --2- 172.21.15.22:0/1750471653 >> [v2:172.21.15.22:6800/1357365082,v1:172.21.15.22:6801/1357365082] conn(0x7fe3440a2a10 0x7fe3440a4ed0 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:07:35.716 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:35.715+0000 7fe364400640 1 --2- 172.21.15.22:0/1750471653 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fe35c109580 0x7fe35c1a1110 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:07:35.716 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:35.715+0000 7fe364400640 1 --2- 172.21.15.22:0/1750471653 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe35c075660 0x7fe35c19a0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:07:35.716 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:35.715+0000 7fe364400640 1 --2- 172.21.15.22:0/1750471653 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe35c074080 0x7fe35c199b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:07:35.716 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:35.715+0000 7fe364400640 1 -- 172.21.15.22:0/1750471653 >> 172.21.15.22:0/1750471653 conn(0x7fe35c0fcc60 msgr2=0x7fe35c105510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:07:35.716 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:35.716+0000 7fe364400640 1 -- 172.21.15.22:0/1750471653 shutdown_connections 2024-03-28T18:07:35.716 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:35.716+0000 7fe364400640 1 -- 172.21.15.22:0/1750471653 wait complete. 2024-03-28T18:07:35.736 DEBUG:teuthology.orchestra.run.smithi022:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'date +%s > START_TIME' 2024-03-28T18:07:35.856 INFO:teuthology.run_tasks:Running task full_sequential_finally... 2024-03-28T18:07:35.867 INFO:teuthology.run_tasks:Running task workunit... 2024-03-28T18:07:35.884 INFO:tasks.workunit:Pulling workunits from ref 3be9a5c9bc793e11e2800a8c0c696e8b46742033 2024-03-28T18:07:35.885 INFO:tasks.workunit:Making a separate scratch dir for every client... 2024-03-28T18:07:35.885 INFO:tasks.workunit:timeout=3h 2024-03-28T18:07:35.885 INFO:tasks.workunit:cleanup=True 2024-03-28T18:07:35.885 DEBUG:teuthology.orchestra.run.smithi022:> stat -- /home/ubuntu/cephtest/mnt.0 2024-03-28T18:07:35.928 INFO:teuthology.orchestra.run.smithi022.stdout: File: /home/ubuntu/cephtest/mnt.0 2024-03-28T18:07:35.928 INFO:teuthology.orchestra.run.smithi022.stdout: Size: 0 Blocks: 1 IO Block: 4194304 directory 2024-03-28T18:07:35.928 INFO:teuthology.orchestra.run.smithi022.stdout:Device: 76h/118d Inode: 1 Links: 2 2024-03-28T18:07:35.928 INFO:teuthology.orchestra.run.smithi022.stdout:Access: (1777/drwxrwxrwt) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-28T18:07:35.928 INFO:teuthology.orchestra.run.smithi022.stdout:Context: system_u:object_r:fusefs_t:s0 2024-03-28T18:07:35.929 INFO:teuthology.orchestra.run.smithi022.stdout:Access: 2024-03-28 18:04:56.164567127 +0000 2024-03-28T18:07:35.929 INFO:teuthology.orchestra.run.smithi022.stdout:Modify: 2024-03-28 18:04:56.164567127 +0000 2024-03-28T18:07:35.929 INFO:teuthology.orchestra.run.smithi022.stdout:Change: 2024-03-28 18:07:13.022672048 +0000 2024-03-28T18:07:35.929 INFO:teuthology.orchestra.run.smithi022.stdout: Birth: - 2024-03-28T18:07:35.929 INFO:tasks.workunit:Did not need to create dir /home/ubuntu/cephtest/mnt.0 2024-03-28T18:07:35.929 DEBUG:teuthology.orchestra.run.smithi022:> cd -- /home/ubuntu/cephtest/mnt.0 && sudo install -d -m 0755 --owner=ubuntu -- client.0 2024-03-28T18:07:35.932 INFO:journalctl@ceph.mgr.y.smithi079.stdout:Mar 28 18:07:35 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mgr-y[36699]: 2024-03-28T18:07:35.556+0000 7f7645a57200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-03-28T18:07:36.062 DEBUG:teuthology.orchestra.run.smithi022:> rm -rf /home/ubuntu/cephtest/clone.client.0 && git clone https://git.ceph.com/ceph-ci.git /home/ubuntu/cephtest/clone.client.0 && cd /home/ubuntu/cephtest/clone.client.0 && git checkout 3be9a5c9bc793e11e2800a8c0c696e8b46742033 2024-03-28T18:07:36.112 INFO:tasks.workunit.client.0.smithi022.stderr:Cloning into '/home/ubuntu/cephtest/clone.client.0'... 2024-03-28T18:07:36.424 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:07:36 smithi134 ceph-mon[35037]: pgmap v4: 97 pgs: 97 active+clean; 1.1 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-28T18:07:36.424 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:07:36 smithi134 ceph-mon[35037]: [28/Mar/2024:18:07:34] ENGINE Serving on https://172.21.15.22:7150 2024-03-28T18:07:36.424 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:07:36 smithi134 ceph-mon[35037]: [28/Mar/2024:18:07:35] ENGINE Bus STARTED 2024-03-28T18:07:36.424 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:07:36 smithi134 ceph-mon[35037]: mgrmap e22: x(active, since 2s), standbys: z, y 2024-03-28T18:07:36.425 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:07:36 smithi134 ceph-mon[35037]: from='client.15600 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-03-28T18:07:36.425 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:07:36 smithi134 ceph-mon[35037]: from='client.35465 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-03-28T18:07:36.425 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:07:36 smithi134 ceph-mon[35037]: Standby manager daemon y restarted 2024-03-28T18:07:36.425 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:07:36 smithi134 ceph-mon[35037]: Standby manager daemon y started 2024-03-28T18:07:36.425 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:07:36 smithi134 ceph-mon[35037]: from='mgr.? 172.21.15.79:0/1176589613' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-03-28T18:07:36.425 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:07:36 smithi134 ceph-mon[35037]: from='mgr.? 172.21.15.79:0/1176589613' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-03-28T18:07:36.425 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:07:36 smithi134 ceph-mon[35037]: from='mgr.? 172.21.15.79:0/1176589613' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-03-28T18:07:36.425 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:07:36 smithi134 ceph-mon[35037]: from='mgr.? 172.21.15.79:0/1176589613' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-03-28T18:07:36.432 INFO:journalctl@ceph.mgr.y.smithi079.stdout:Mar 28 18:07:35 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mgr-y[36699]: 2024-03-28T18:07:35.964+0000 7f7645a57200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-03-28T18:07:36.432 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:07:36 smithi079 ceph-mon[35312]: pgmap v4: 97 pgs: 97 active+clean; 1.1 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-28T18:07:36.432 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:07:36 smithi079 ceph-mon[35312]: [28/Mar/2024:18:07:34] ENGINE Serving on https://172.21.15.22:7150 2024-03-28T18:07:36.432 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:07:36 smithi079 ceph-mon[35312]: [28/Mar/2024:18:07:35] ENGINE Bus STARTED 2024-03-28T18:07:36.432 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:07:36 smithi079 ceph-mon[35312]: mgrmap e22: x(active, since 2s), standbys: z, y 2024-03-28T18:07:36.433 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:07:36 smithi079 ceph-mon[35312]: from='client.15600 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-03-28T18:07:36.433 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:07:36 smithi079 ceph-mon[35312]: from='client.35465 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-03-28T18:07:36.433 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:07:36 smithi079 ceph-mon[35312]: Standby manager daemon y restarted 2024-03-28T18:07:36.433 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:07:36 smithi079 ceph-mon[35312]: Standby manager daemon y started 2024-03-28T18:07:36.433 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:07:36 smithi079 ceph-mon[35312]: from='mgr.? 172.21.15.79:0/1176589613' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-03-28T18:07:36.433 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:07:36 smithi079 ceph-mon[35312]: from='mgr.? 172.21.15.79:0/1176589613' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-03-28T18:07:36.433 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:07:36 smithi079 ceph-mon[35312]: from='mgr.? 172.21.15.79:0/1176589613' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-03-28T18:07:36.433 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:07:36 smithi079 ceph-mon[35312]: from='mgr.? 172.21.15.79:0/1176589613' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-03-28T18:07:36.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:07:36 smithi022 ceph-mon[31985]: pgmap v4: 97 pgs: 97 active+clean; 1.1 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-28T18:07:36.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:07:36 smithi022 ceph-mon[31985]: [28/Mar/2024:18:07:34] ENGINE Serving on https://172.21.15.22:7150 2024-03-28T18:07:36.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:07:36 smithi022 ceph-mon[31985]: [28/Mar/2024:18:07:35] ENGINE Bus STARTED 2024-03-28T18:07:36.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:07:36 smithi022 ceph-mon[31985]: mgrmap e22: x(active, since 2s), standbys: z, y 2024-03-28T18:07:36.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:07:36 smithi022 ceph-mon[31985]: from='client.15600 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-03-28T18:07:36.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:07:36 smithi022 ceph-mon[31985]: from='client.35465 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-03-28T18:07:36.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:07:36 smithi022 ceph-mon[31985]: Standby manager daemon y restarted 2024-03-28T18:07:36.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:07:36 smithi022 ceph-mon[31985]: Standby manager daemon y started 2024-03-28T18:07:36.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:07:36 smithi022 ceph-mon[31985]: from='mgr.? 172.21.15.79:0/1176589613' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-03-28T18:07:36.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:07:36 smithi022 ceph-mon[31985]: from='mgr.? 172.21.15.79:0/1176589613' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-03-28T18:07:36.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:07:36 smithi022 ceph-mon[31985]: from='mgr.? 172.21.15.79:0/1176589613' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-03-28T18:07:36.533 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:07:36 smithi022 ceph-mon[31985]: from='mgr.? 172.21.15.79:0/1176589613' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-03-28T18:07:37.425 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:07:37 smithi134 ceph-mon[35037]: mgrmap e23: x(active, since 3s), standbys: z, y 2024-03-28T18:07:37.426 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:07:37 smithi134 ceph-mon[35037]: from='mgr.15522 ' entity='mgr.x' 2024-03-28T18:07:37.426 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:07:37 smithi134 ceph-mon[35037]: from='mgr.15522 ' entity='mgr.x' 2024-03-28T18:07:37.426 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:07:37 smithi134 ceph-mon[35037]: from='mgr.15522 ' entity='mgr.x' 2024-03-28T18:07:37.426 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:07:37 smithi134 ceph-mon[35037]: from='mgr.15522 ' entity='mgr.x' 2024-03-28T18:07:37.426 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:07:37 smithi134 ceph-mon[35037]: from='mgr.15522 ' entity='mgr.x' 2024-03-28T18:07:37.426 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:07:37 smithi134 ceph-mon[35037]: from='mgr.15522 ' entity='mgr.x' 2024-03-28T18:07:37.432 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:07:37 smithi079 ceph-mon[35312]: mgrmap e23: x(active, since 3s), standbys: z, y 2024-03-28T18:07:37.432 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:07:37 smithi079 ceph-mon[35312]: from='mgr.15522 ' entity='mgr.x' 2024-03-28T18:07:37.432 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:07:37 smithi079 ceph-mon[35312]: from='mgr.15522 ' entity='mgr.x' 2024-03-28T18:07:37.432 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:07:37 smithi079 ceph-mon[35312]: from='mgr.15522 ' entity='mgr.x' 2024-03-28T18:07:37.432 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:07:37 smithi079 ceph-mon[35312]: from='mgr.15522 ' entity='mgr.x' 2024-03-28T18:07:37.432 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:07:37 smithi079 ceph-mon[35312]: from='mgr.15522 ' entity='mgr.x' 2024-03-28T18:07:37.433 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:07:37 smithi079 ceph-mon[35312]: from='mgr.15522 ' entity='mgr.x' 2024-03-28T18:07:37.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:07:37 smithi022 ceph-mon[31985]: mgrmap e23: x(active, since 3s), standbys: z, y 2024-03-28T18:07:37.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:07:37 smithi022 ceph-mon[31985]: from='mgr.15522 ' entity='mgr.x' 2024-03-28T18:07:37.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:07:37 smithi022 ceph-mon[31985]: from='mgr.15522 ' entity='mgr.x' 2024-03-28T18:07:37.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:07:37 smithi022 ceph-mon[31985]: from='mgr.15522 ' entity='mgr.x' 2024-03-28T18:07:37.533 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:07:37 smithi022 ceph-mon[31985]: from='mgr.15522 ' entity='mgr.x' 2024-03-28T18:07:37.533 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:07:37 smithi022 ceph-mon[31985]: from='mgr.15522 ' entity='mgr.x' 2024-03-28T18:07:37.533 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:07:37 smithi022 ceph-mon[31985]: from='mgr.15522 ' entity='mgr.x' 2024-03-28T18:07:39.032 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:07:38 smithi022 ceph-mon[31985]: pgmap v5: 97 pgs: 97 active+clean; 1.1 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-28T18:07:39.032 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:07:38 smithi022 ceph-mon[31985]: mgrmap e24: x(active, since 4s), standbys: z, y 2024-03-28T18:07:39.032 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:07:38 smithi022 ceph-mon[31985]: from='mgr.15522 ' entity='mgr.x' 2024-03-28T18:07:39.033 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:07:38 smithi022 ceph-mon[31985]: from='mgr.15522 ' entity='mgr.x' 2024-03-28T18:07:39.033 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:07:38 smithi022 ceph-mon[31985]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-28T18:07:39.033 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:07:38 smithi022 ceph-mon[31985]: from='mgr.15522 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-28T18:07:39.033 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:07:38 smithi022 ceph-mon[31985]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-28T18:07:39.033 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:07:38 smithi022 ceph-mon[31985]: from='mgr.15522 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-28T18:07:39.033 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:07:38 smithi022 ceph-mon[31985]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-03-28T18:07:39.033 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:07:38 smithi022 ceph-mon[31985]: from='mgr.15522 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-03-28T18:07:39.033 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:07:38 smithi022 ceph-mon[31985]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-03-28T18:07:39.033 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:07:38 smithi022 ceph-mon[31985]: from='mgr.15522 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-03-28T18:07:39.033 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:07:38 smithi022 ceph-mon[31985]: from='mgr.15522 ' entity='mgr.x' 2024-03-28T18:07:39.033 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:07:38 smithi022 ceph-mon[31985]: from='mgr.15522 ' entity='mgr.x' 2024-03-28T18:07:39.033 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:07:38 smithi022 ceph-mon[31985]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-28T18:07:39.033 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:07:38 smithi022 ceph-mon[31985]: from='mgr.15522 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-28T18:07:39.033 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:07:38 smithi022 ceph-mon[31985]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-28T18:07:39.034 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:07:38 smithi022 ceph-mon[31985]: from='mgr.15522 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-28T18:07:39.034 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:07:38 smithi022 ceph-mon[31985]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-03-28T18:07:39.034 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:07:38 smithi022 ceph-mon[31985]: from='mgr.15522 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-03-28T18:07:39.034 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:07:38 smithi022 ceph-mon[31985]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-03-28T18:07:39.034 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:07:38 smithi022 ceph-mon[31985]: from='mgr.15522 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-03-28T18:07:39.175 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:07:38 smithi134 ceph-mon[35037]: pgmap v5: 97 pgs: 97 active+clean; 1.1 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-28T18:07:39.175 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:07:38 smithi134 ceph-mon[35037]: mgrmap e24: x(active, since 4s), standbys: z, y 2024-03-28T18:07:39.175 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:07:38 smithi134 ceph-mon[35037]: from='mgr.15522 ' entity='mgr.x' 2024-03-28T18:07:39.175 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:07:38 smithi134 ceph-mon[35037]: from='mgr.15522 ' entity='mgr.x' 2024-03-28T18:07:39.176 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:07:38 smithi134 ceph-mon[35037]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-28T18:07:39.176 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:07:38 smithi134 ceph-mon[35037]: from='mgr.15522 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-28T18:07:39.176 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:07:38 smithi134 ceph-mon[35037]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-28T18:07:39.176 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:07:38 smithi134 ceph-mon[35037]: from='mgr.15522 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-28T18:07:39.176 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:07:38 smithi134 ceph-mon[35037]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-03-28T18:07:39.176 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:07:38 smithi134 ceph-mon[35037]: from='mgr.15522 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-03-28T18:07:39.176 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:07:38 smithi134 ceph-mon[35037]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-03-28T18:07:39.176 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:07:38 smithi134 ceph-mon[35037]: from='mgr.15522 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-03-28T18:07:39.176 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:07:38 smithi134 ceph-mon[35037]: from='mgr.15522 ' entity='mgr.x' 2024-03-28T18:07:39.176 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:07:38 smithi134 ceph-mon[35037]: from='mgr.15522 ' entity='mgr.x' 2024-03-28T18:07:39.176 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:07:38 smithi134 ceph-mon[35037]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-28T18:07:39.176 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:07:38 smithi134 ceph-mon[35037]: from='mgr.15522 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-28T18:07:39.176 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:07:38 smithi134 ceph-mon[35037]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-28T18:07:39.176 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:07:38 smithi134 ceph-mon[35037]: from='mgr.15522 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-28T18:07:39.176 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:07:38 smithi134 ceph-mon[35037]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-03-28T18:07:39.176 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:07:38 smithi134 ceph-mon[35037]: from='mgr.15522 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-03-28T18:07:39.177 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:07:38 smithi134 ceph-mon[35037]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-03-28T18:07:39.177 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:07:38 smithi134 ceph-mon[35037]: from='mgr.15522 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-03-28T18:07:39.182 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:07:38 smithi079 ceph-mon[35312]: pgmap v5: 97 pgs: 97 active+clean; 1.1 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-28T18:07:39.183 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:07:38 smithi079 ceph-mon[35312]: mgrmap e24: x(active, since 4s), standbys: z, y 2024-03-28T18:07:39.183 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:07:38 smithi079 ceph-mon[35312]: from='mgr.15522 ' entity='mgr.x' 2024-03-28T18:07:39.183 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:07:38 smithi079 ceph-mon[35312]: from='mgr.15522 ' entity='mgr.x' 2024-03-28T18:07:39.183 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:07:38 smithi079 ceph-mon[35312]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-28T18:07:39.183 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:07:38 smithi079 ceph-mon[35312]: from='mgr.15522 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-28T18:07:39.183 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:07:38 smithi079 ceph-mon[35312]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-28T18:07:39.183 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:07:38 smithi079 ceph-mon[35312]: from='mgr.15522 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-28T18:07:39.183 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:07:38 smithi079 ceph-mon[35312]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-03-28T18:07:39.183 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:07:38 smithi079 ceph-mon[35312]: from='mgr.15522 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-03-28T18:07:39.183 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:07:38 smithi079 ceph-mon[35312]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-03-28T18:07:39.183 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:07:38 smithi079 ceph-mon[35312]: from='mgr.15522 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-03-28T18:07:39.183 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:07:38 smithi079 ceph-mon[35312]: from='mgr.15522 ' entity='mgr.x' 2024-03-28T18:07:39.183 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:07:38 smithi079 ceph-mon[35312]: from='mgr.15522 ' entity='mgr.x' 2024-03-28T18:07:39.183 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:07:38 smithi079 ceph-mon[35312]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-28T18:07:39.184 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:07:38 smithi079 ceph-mon[35312]: from='mgr.15522 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-28T18:07:39.184 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:07:38 smithi079 ceph-mon[35312]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-28T18:07:39.184 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:07:38 smithi079 ceph-mon[35312]: from='mgr.15522 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-28T18:07:39.184 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:07:38 smithi079 ceph-mon[35312]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-03-28T18:07:39.184 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:07:38 smithi079 ceph-mon[35312]: from='mgr.15522 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-03-28T18:07:39.184 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:07:38 smithi079 ceph-mon[35312]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-03-28T18:07:39.184 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:07:38 smithi079 ceph-mon[35312]: from='mgr.15522 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-03-28T18:07:39.933 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:07:39 smithi079 ceph-mon[35312]: Adjusting osd_memory_target on smithi134 to 176.8M 2024-03-28T18:07:39.933 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:07:39 smithi079 ceph-mon[35312]: Unable to set osd_memory_target on smithi134 to 185393868: error parsing value: Value '185393868' is below minimum 939524096 2024-03-28T18:07:39.933 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:07:39 smithi079 ceph-mon[35312]: Adjusting osd_memory_target on smithi022 to 177.8M 2024-03-28T18:07:39.933 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:07:39 smithi079 ceph-mon[35312]: Unable to set osd_memory_target on smithi022 to 186497740: error parsing value: Value '186497740' is below minimum 939524096 2024-03-28T18:07:39.933 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:07:39 smithi079 ceph-mon[35312]: from='mgr.15522 ' entity='mgr.x' 2024-03-28T18:07:39.933 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:07:39 smithi079 ceph-mon[35312]: from='mgr.15522 ' entity='mgr.x' 2024-03-28T18:07:39.933 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:07:39 smithi079 ceph-mon[35312]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-28T18:07:39.933 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:07:39 smithi079 ceph-mon[35312]: from='mgr.15522 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-28T18:07:39.933 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:07:39 smithi079 ceph-mon[35312]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-28T18:07:39.933 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:07:39 smithi079 ceph-mon[35312]: from='mgr.15522 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-28T18:07:39.933 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:07:39 smithi079 ceph-mon[35312]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-03-28T18:07:39.934 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:07:39 smithi079 ceph-mon[35312]: from='mgr.15522 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-03-28T18:07:39.934 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:07:39 smithi079 ceph-mon[35312]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-03-28T18:07:39.934 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:07:39 smithi079 ceph-mon[35312]: from='mgr.15522 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-03-28T18:07:39.934 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:07:39 smithi079 ceph-mon[35312]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T18:07:39.934 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:07:39 smithi079 ceph-mon[35312]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T18:07:40.032 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:07:39 smithi022 ceph-mon[31985]: Adjusting osd_memory_target on smithi134 to 176.8M 2024-03-28T18:07:40.032 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:07:39 smithi022 ceph-mon[31985]: Unable to set osd_memory_target on smithi134 to 185393868: error parsing value: Value '185393868' is below minimum 939524096 2024-03-28T18:07:40.032 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:07:39 smithi022 ceph-mon[31985]: Adjusting osd_memory_target on smithi022 to 177.8M 2024-03-28T18:07:40.032 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:07:39 smithi022 ceph-mon[31985]: Unable to set osd_memory_target on smithi022 to 186497740: error parsing value: Value '186497740' is below minimum 939524096 2024-03-28T18:07:40.032 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:07:39 smithi022 ceph-mon[31985]: from='mgr.15522 ' entity='mgr.x' 2024-03-28T18:07:40.032 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:07:39 smithi022 ceph-mon[31985]: from='mgr.15522 ' entity='mgr.x' 2024-03-28T18:07:40.032 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:07:39 smithi022 ceph-mon[31985]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-28T18:07:40.032 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:07:39 smithi022 ceph-mon[31985]: from='mgr.15522 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-28T18:07:40.033 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:07:39 smithi022 ceph-mon[31985]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-28T18:07:40.033 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:07:39 smithi022 ceph-mon[31985]: from='mgr.15522 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-28T18:07:40.033 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:07:39 smithi022 ceph-mon[31985]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-03-28T18:07:40.033 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:07:39 smithi022 ceph-mon[31985]: from='mgr.15522 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-03-28T18:07:40.033 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:07:39 smithi022 ceph-mon[31985]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-03-28T18:07:40.033 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:07:39 smithi022 ceph-mon[31985]: from='mgr.15522 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-03-28T18:07:40.033 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:07:39 smithi022 ceph-mon[31985]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T18:07:40.033 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:07:39 smithi022 ceph-mon[31985]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T18:07:40.175 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:07:39 smithi134 ceph-mon[35037]: Adjusting osd_memory_target on smithi134 to 176.8M 2024-03-28T18:07:40.175 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:07:39 smithi134 ceph-mon[35037]: Unable to set osd_memory_target on smithi134 to 185393868: error parsing value: Value '185393868' is below minimum 939524096 2024-03-28T18:07:40.175 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:07:39 smithi134 ceph-mon[35037]: Adjusting osd_memory_target on smithi022 to 177.8M 2024-03-28T18:07:40.175 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:07:39 smithi134 ceph-mon[35037]: Unable to set osd_memory_target on smithi022 to 186497740: error parsing value: Value '186497740' is below minimum 939524096 2024-03-28T18:07:40.175 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:07:39 smithi134 ceph-mon[35037]: from='mgr.15522 ' entity='mgr.x' 2024-03-28T18:07:40.175 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:07:39 smithi134 ceph-mon[35037]: from='mgr.15522 ' entity='mgr.x' 2024-03-28T18:07:40.175 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:07:39 smithi134 ceph-mon[35037]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-28T18:07:40.175 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:07:39 smithi134 ceph-mon[35037]: from='mgr.15522 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-28T18:07:40.175 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:07:39 smithi134 ceph-mon[35037]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-28T18:07:40.175 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:07:39 smithi134 ceph-mon[35037]: from='mgr.15522 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-28T18:07:40.175 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:07:39 smithi134 ceph-mon[35037]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-03-28T18:07:40.175 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:07:39 smithi134 ceph-mon[35037]: from='mgr.15522 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-03-28T18:07:40.176 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:07:39 smithi134 ceph-mon[35037]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-03-28T18:07:40.176 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:07:39 smithi134 ceph-mon[35037]: from='mgr.15522 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-03-28T18:07:40.176 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:07:39 smithi134 ceph-mon[35037]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T18:07:40.176 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:07:39 smithi134 ceph-mon[35037]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T18:07:41.031 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:07:40 smithi022 ceph-mon[31985]: Adjusting osd_memory_target on smithi079 to 176.7M 2024-03-28T18:07:41.032 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:07:40 smithi022 ceph-mon[31985]: Unable to set osd_memory_target on smithi079 to 185386700: error parsing value: Value '185386700' is below minimum 939524096 2024-03-28T18:07:41.032 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:07:40 smithi022 ceph-mon[31985]: Updating smithi022:/etc/ceph/ceph.conf 2024-03-28T18:07:41.032 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:07:40 smithi022 ceph-mon[31985]: Updating smithi079:/etc/ceph/ceph.conf 2024-03-28T18:07:41.032 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:07:40 smithi022 ceph-mon[31985]: Updating smithi134:/etc/ceph/ceph.conf 2024-03-28T18:07:41.032 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:07:40 smithi022 ceph-mon[31985]: pgmap v6: 97 pgs: 97 active+clean; 1.1 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-28T18:07:41.032 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:07:40 smithi022 ceph-mon[31985]: Updating smithi079:/var/lib/ceph/fd220748-ed2b-11ee-95d0-87774f69a715/config/ceph.conf 2024-03-28T18:07:41.032 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:07:40 smithi022 ceph-mon[31985]: Updating smithi134:/var/lib/ceph/fd220748-ed2b-11ee-95d0-87774f69a715/config/ceph.conf 2024-03-28T18:07:41.032 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:07:40 smithi022 ceph-mon[31985]: Updating smithi022:/var/lib/ceph/fd220748-ed2b-11ee-95d0-87774f69a715/config/ceph.conf 2024-03-28T18:07:41.174 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:07:40 smithi134 ceph-mon[35037]: Adjusting osd_memory_target on smithi079 to 176.7M 2024-03-28T18:07:41.174 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:07:40 smithi134 ceph-mon[35037]: Unable to set osd_memory_target on smithi079 to 185386700: error parsing value: Value '185386700' is below minimum 939524096 2024-03-28T18:07:41.175 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:07:40 smithi134 ceph-mon[35037]: Updating smithi022:/etc/ceph/ceph.conf 2024-03-28T18:07:41.175 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:07:40 smithi134 ceph-mon[35037]: Updating smithi079:/etc/ceph/ceph.conf 2024-03-28T18:07:41.175 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:07:40 smithi134 ceph-mon[35037]: Updating smithi134:/etc/ceph/ceph.conf 2024-03-28T18:07:41.175 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:07:40 smithi134 ceph-mon[35037]: pgmap v6: 97 pgs: 97 active+clean; 1.1 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-28T18:07:41.175 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:07:40 smithi134 ceph-mon[35037]: Updating smithi079:/var/lib/ceph/fd220748-ed2b-11ee-95d0-87774f69a715/config/ceph.conf 2024-03-28T18:07:41.175 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:07:40 smithi134 ceph-mon[35037]: Updating smithi134:/var/lib/ceph/fd220748-ed2b-11ee-95d0-87774f69a715/config/ceph.conf 2024-03-28T18:07:41.175 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:07:40 smithi134 ceph-mon[35037]: Updating smithi022:/var/lib/ceph/fd220748-ed2b-11ee-95d0-87774f69a715/config/ceph.conf 2024-03-28T18:07:41.182 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:07:40 smithi079 ceph-mon[35312]: Adjusting osd_memory_target on smithi079 to 176.7M 2024-03-28T18:07:41.182 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:07:40 smithi079 ceph-mon[35312]: Unable to set osd_memory_target on smithi079 to 185386700: error parsing value: Value '185386700' is below minimum 939524096 2024-03-28T18:07:41.183 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:07:40 smithi079 ceph-mon[35312]: Updating smithi022:/etc/ceph/ceph.conf 2024-03-28T18:07:41.183 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:07:40 smithi079 ceph-mon[35312]: Updating smithi079:/etc/ceph/ceph.conf 2024-03-28T18:07:41.183 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:07:40 smithi079 ceph-mon[35312]: Updating smithi134:/etc/ceph/ceph.conf 2024-03-28T18:07:41.183 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:07:40 smithi079 ceph-mon[35312]: pgmap v6: 97 pgs: 97 active+clean; 1.1 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-28T18:07:41.183 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:07:40 smithi079 ceph-mon[35312]: Updating smithi079:/var/lib/ceph/fd220748-ed2b-11ee-95d0-87774f69a715/config/ceph.conf 2024-03-28T18:07:41.183 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:07:40 smithi079 ceph-mon[35312]: Updating smithi134:/var/lib/ceph/fd220748-ed2b-11ee-95d0-87774f69a715/config/ceph.conf 2024-03-28T18:07:41.183 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:07:40 smithi079 ceph-mon[35312]: Updating smithi022:/var/lib/ceph/fd220748-ed2b-11ee-95d0-87774f69a715/config/ceph.conf 2024-03-28T18:07:42.031 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:07:41 smithi022 ceph-mon[31985]: Updating smithi079:/etc/ceph/ceph.client.admin.keyring 2024-03-28T18:07:42.031 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:07:41 smithi022 ceph-mon[31985]: Updating smithi134:/etc/ceph/ceph.client.admin.keyring 2024-03-28T18:07:42.032 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:07:41 smithi022 ceph-mon[31985]: Updating smithi022:/etc/ceph/ceph.client.admin.keyring 2024-03-28T18:07:42.174 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:07:41 smithi134 ceph-mon[35037]: Updating smithi079:/etc/ceph/ceph.client.admin.keyring 2024-03-28T18:07:42.175 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:07:41 smithi134 ceph-mon[35037]: Updating smithi134:/etc/ceph/ceph.client.admin.keyring 2024-03-28T18:07:42.175 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:07:41 smithi134 ceph-mon[35037]: Updating smithi022:/etc/ceph/ceph.client.admin.keyring 2024-03-28T18:07:42.182 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:07:41 smithi079 ceph-mon[35312]: Updating smithi079:/etc/ceph/ceph.client.admin.keyring 2024-03-28T18:07:42.182 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:07:41 smithi079 ceph-mon[35312]: Updating smithi134:/etc/ceph/ceph.client.admin.keyring 2024-03-28T18:07:42.182 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:07:41 smithi079 ceph-mon[35312]: Updating smithi022:/etc/ceph/ceph.client.admin.keyring 2024-03-28T18:07:43.032 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:07:42 smithi022 ceph-mon[31985]: pgmap v7: 97 pgs: 97 active+clean; 1.3 MiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 68 KiB/s rd, 133 KiB/s wr, 80 op/s 2024-03-28T18:07:43.032 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:07:42 smithi022 ceph-mon[31985]: Updating smithi079:/var/lib/ceph/fd220748-ed2b-11ee-95d0-87774f69a715/config/ceph.client.admin.keyring 2024-03-28T18:07:43.032 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:07:42 smithi022 ceph-mon[31985]: Updating smithi134:/var/lib/ceph/fd220748-ed2b-11ee-95d0-87774f69a715/config/ceph.client.admin.keyring 2024-03-28T18:07:43.032 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:07:42 smithi022 ceph-mon[31985]: Updating smithi022:/var/lib/ceph/fd220748-ed2b-11ee-95d0-87774f69a715/config/ceph.client.admin.keyring 2024-03-28T18:07:43.033 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:07:42 smithi022 ceph-mon[31985]: from='mgr.15522 ' entity='mgr.x' 2024-03-28T18:07:43.033 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:07:42 smithi022 ceph-mon[31985]: from='mgr.15522 ' entity='mgr.x' 2024-03-28T18:07:43.033 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:07:42 smithi022 ceph-mon[31985]: from='mgr.15522 ' entity='mgr.x' 2024-03-28T18:07:43.033 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:07:42 smithi022 ceph-mon[31985]: from='mgr.15522 ' entity='mgr.x' 2024-03-28T18:07:43.033 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:07:42 smithi022 ceph-mon[31985]: from='mgr.15522 ' entity='mgr.x' 2024-03-28T18:07:43.033 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:07:42 smithi022 ceph-mon[31985]: from='mgr.15522 ' entity='mgr.x' 2024-03-28T18:07:43.033 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:07:42 smithi022 ceph-mon[31985]: from='mgr.15522 ' entity='mgr.x' 2024-03-28T18:07:43.033 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:07:42 smithi022 ceph-mon[31985]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T18:07:43.033 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:07:42 smithi022 ceph-mon[31985]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T18:07:43.033 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:07:42 smithi022 ceph-mon[31985]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T18:07:43.033 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:07:42 smithi022 ceph-mon[31985]: from='mgr.15522 ' entity='mgr.x' 2024-03-28T18:07:43.175 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:07:42 smithi134 ceph-mon[35037]: pgmap v7: 97 pgs: 97 active+clean; 1.3 MiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 68 KiB/s rd, 133 KiB/s wr, 80 op/s 2024-03-28T18:07:43.175 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:07:42 smithi134 ceph-mon[35037]: Updating smithi079:/var/lib/ceph/fd220748-ed2b-11ee-95d0-87774f69a715/config/ceph.client.admin.keyring 2024-03-28T18:07:43.175 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:07:42 smithi134 ceph-mon[35037]: Updating smithi134:/var/lib/ceph/fd220748-ed2b-11ee-95d0-87774f69a715/config/ceph.client.admin.keyring 2024-03-28T18:07:43.175 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:07:42 smithi134 ceph-mon[35037]: Updating smithi022:/var/lib/ceph/fd220748-ed2b-11ee-95d0-87774f69a715/config/ceph.client.admin.keyring 2024-03-28T18:07:43.175 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:07:42 smithi134 ceph-mon[35037]: from='mgr.15522 ' entity='mgr.x' 2024-03-28T18:07:43.175 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:07:42 smithi134 ceph-mon[35037]: from='mgr.15522 ' entity='mgr.x' 2024-03-28T18:07:43.175 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:07:42 smithi134 ceph-mon[35037]: from='mgr.15522 ' entity='mgr.x' 2024-03-28T18:07:43.175 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:07:42 smithi134 ceph-mon[35037]: from='mgr.15522 ' entity='mgr.x' 2024-03-28T18:07:43.175 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:07:42 smithi134 ceph-mon[35037]: from='mgr.15522 ' entity='mgr.x' 2024-03-28T18:07:43.175 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:07:42 smithi134 ceph-mon[35037]: from='mgr.15522 ' entity='mgr.x' 2024-03-28T18:07:43.176 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:07:42 smithi134 ceph-mon[35037]: from='mgr.15522 ' entity='mgr.x' 2024-03-28T18:07:43.176 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:07:42 smithi134 ceph-mon[35037]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T18:07:43.176 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:07:42 smithi134 ceph-mon[35037]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T18:07:43.176 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:07:42 smithi134 ceph-mon[35037]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T18:07:43.176 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:07:42 smithi134 ceph-mon[35037]: from='mgr.15522 ' entity='mgr.x' 2024-03-28T18:07:43.183 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:07:42 smithi079 ceph-mon[35312]: pgmap v7: 97 pgs: 97 active+clean; 1.3 MiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 68 KiB/s rd, 133 KiB/s wr, 80 op/s 2024-03-28T18:07:43.183 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:07:42 smithi079 ceph-mon[35312]: Updating smithi079:/var/lib/ceph/fd220748-ed2b-11ee-95d0-87774f69a715/config/ceph.client.admin.keyring 2024-03-28T18:07:43.183 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:07:42 smithi079 ceph-mon[35312]: Updating smithi134:/var/lib/ceph/fd220748-ed2b-11ee-95d0-87774f69a715/config/ceph.client.admin.keyring 2024-03-28T18:07:43.183 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:07:42 smithi079 ceph-mon[35312]: Updating smithi022:/var/lib/ceph/fd220748-ed2b-11ee-95d0-87774f69a715/config/ceph.client.admin.keyring 2024-03-28T18:07:43.183 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:07:42 smithi079 ceph-mon[35312]: from='mgr.15522 ' entity='mgr.x' 2024-03-28T18:07:43.183 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:07:42 smithi079 ceph-mon[35312]: from='mgr.15522 ' entity='mgr.x' 2024-03-28T18:07:43.183 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:07:42 smithi079 ceph-mon[35312]: from='mgr.15522 ' entity='mgr.x' 2024-03-28T18:07:43.183 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:07:42 smithi079 ceph-mon[35312]: from='mgr.15522 ' entity='mgr.x' 2024-03-28T18:07:43.183 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:07:42 smithi079 ceph-mon[35312]: from='mgr.15522 ' entity='mgr.x' 2024-03-28T18:07:43.183 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:07:42 smithi079 ceph-mon[35312]: from='mgr.15522 ' entity='mgr.x' 2024-03-28T18:07:43.183 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:07:42 smithi079 ceph-mon[35312]: from='mgr.15522 ' entity='mgr.x' 2024-03-28T18:07:43.183 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:07:42 smithi079 ceph-mon[35312]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T18:07:43.183 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:07:42 smithi079 ceph-mon[35312]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T18:07:43.184 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:07:42 smithi079 ceph-mon[35312]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T18:07:43.184 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:07:42 smithi079 ceph-mon[35312]: from='mgr.15522 ' entity='mgr.x' 2024-03-28T18:07:45.032 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:07:44 smithi022 ceph-mon[31985]: pgmap v8: 97 pgs: 97 active+clean; 1.3 MiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 103 KiB/s wr, 62 op/s 2024-03-28T18:07:45.174 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:07:44 smithi134 ceph-mon[35037]: pgmap v8: 97 pgs: 97 active+clean; 1.3 MiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 103 KiB/s wr, 62 op/s 2024-03-28T18:07:45.182 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:07:44 smithi079 ceph-mon[35312]: pgmap v8: 97 pgs: 97 active+clean; 1.3 MiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 103 KiB/s wr, 62 op/s 2024-03-28T18:07:47.032 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:07:46 smithi022 ceph-mon[31985]: pgmap v9: 97 pgs: 97 active+clean; 1.3 MiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 84 KiB/s wr, 55 op/s 2024-03-28T18:07:47.174 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:07:46 smithi134 ceph-mon[35037]: pgmap v9: 97 pgs: 97 active+clean; 1.3 MiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 84 KiB/s wr, 55 op/s 2024-03-28T18:07:47.182 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:07:46 smithi079 ceph-mon[35312]: pgmap v9: 97 pgs: 97 active+clean; 1.3 MiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 84 KiB/s wr, 55 op/s 2024-03-28T18:07:49.174 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:07:48 smithi134 ceph-mon[35037]: pgmap v10: 97 pgs: 97 active+clean; 1.3 MiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 77 KiB/s wr, 50 op/s 2024-03-28T18:07:49.182 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:07:48 smithi079 ceph-mon[35312]: pgmap v10: 97 pgs: 97 active+clean; 1.3 MiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 77 KiB/s wr, 50 op/s 2024-03-28T18:07:49.281 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:07:48 smithi022 ceph-mon[31985]: pgmap v10: 97 pgs: 97 active+clean; 1.3 MiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 77 KiB/s wr, 50 op/s 2024-03-28T18:07:51.174 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:07:50 smithi134 ceph-mon[35037]: pgmap v11: 97 pgs: 97 active+clean; 1.3 MiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 77 KiB/s wr, 50 op/s 2024-03-28T18:07:51.182 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:07:50 smithi079 ceph-mon[35312]: pgmap v11: 97 pgs: 97 active+clean; 1.3 MiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 77 KiB/s wr, 50 op/s 2024-03-28T18:07:51.281 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:07:50 smithi022 ceph-mon[31985]: pgmap v11: 97 pgs: 97 active+clean; 1.3 MiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 77 KiB/s wr, 50 op/s 2024-03-28T18:07:53.085 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:07:52 smithi022 ceph-mon[31985]: pgmap v12: 97 pgs: 97 active+clean; 1.3 MiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 77 KiB/s wr, 54 op/s 2024-03-28T18:07:53.174 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:07:52 smithi134 ceph-mon[35037]: pgmap v12: 97 pgs: 97 active+clean; 1.3 MiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 77 KiB/s wr, 54 op/s 2024-03-28T18:07:53.182 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:07:52 smithi079 ceph-mon[35312]: pgmap v12: 97 pgs: 97 active+clean; 1.3 MiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 77 KiB/s wr, 54 op/s 2024-03-28T18:07:53.537 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-28T18:07:53.538 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-28T18:07:53.757 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:53.756+0000 7fba11800640 1 -- 172.21.15.22:0/608555502 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fba0c103840 msgr2=0x7fba0c103cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:07:53.757 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:53.756+0000 7fba11800640 1 --2- 172.21.15.22:0/608555502 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fba0c103840 0x7fba0c103cc0 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7fb9f4009930 tx=0x7fb9f402bc30 comp rx=0 tx=0).stop 2024-03-28T18:07:53.757 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:53.756+0000 7fba11800640 1 -- 172.21.15.22:0/608555502 shutdown_connections 2024-03-28T18:07:53.757 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:53.756+0000 7fba11800640 1 --2- 172.21.15.22:0/608555502 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fba0c104200 0x7fba0c110710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:07:53.757 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:53.756+0000 7fba11800640 1 --2- 172.21.15.22:0/608555502 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fba0c103840 0x7fba0c103cc0 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:07:53.757 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:53.756+0000 7fba11800640 1 --2- 172.21.15.22:0/608555502 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fba0c109850 0x7fba0c109c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:07:53.758 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:53.756+0000 7fba11800640 1 -- 172.21.15.22:0/608555502 >> 172.21.15.22:0/608555502 conn(0x7fba0c0ff310 msgr2=0x7fba0c101730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:07:53.758 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:53.756+0000 7fba11800640 1 -- 172.21.15.22:0/608555502 shutdown_connections 2024-03-28T18:07:53.758 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:53.756+0000 7fba11800640 1 -- 172.21.15.22:0/608555502 wait complete. 2024-03-28T18:07:53.759 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:53.757+0000 7fba11800640 1 Processor -- start 2024-03-28T18:07:53.759 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:53.757+0000 7fba11800640 1 -- start start 2024-03-28T18:07:53.759 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:53.758+0000 7fba11800640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fba0c103840 0x7fba0c19a1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:07:53.760 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:53.758+0000 7fba11800640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fba0c104200 0x7fba0c19a700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:07:53.760 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:53.758+0000 7fba11800640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fba0c109850 0x7fba0c19ea30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:07:53.760 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:53.758+0000 7fba11800640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba0c113c80 con 0x7fba0c104200 2024-03-28T18:07:53.760 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:53.758+0000 7fba11800640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba0c113b00 con 0x7fba0c109850 2024-03-28T18:07:53.760 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:53.758+0000 7fba11800640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba0c113e00 con 0x7fba0c103840 2024-03-28T18:07:53.760 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:53.758+0000 7fba03e00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fba0c104200 0x7fba0c19a700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:07:53.760 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:53.758+0000 7fba03e00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fba0c104200 0x7fba0c19a700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:49974/0 (socket says 172.21.15.22:49974) 2024-03-28T18:07:53.760 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:53.758+0000 7fba03e00640 1 -- 172.21.15.22:0/3497474030 learned_addr learned my addr 172.21.15.22:0/3497474030 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-28T18:07:53.760 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:53.758+0000 7fba0b400640 1 --2- 172.21.15.22:0/3497474030 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fba0c109850 0x7fba0c19ea30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:07:53.760 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:53.758+0000 7fba0aa00640 1 --2- 172.21.15.22:0/3497474030 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fba0c103840 0x7fba0c19a1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:07:53.760 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:53.759+0000 7fba03e00640 1 -- 172.21.15.22:0/3497474030 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fba0c103840 msgr2=0x7fba0c19a1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:07:53.760 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:53.759+0000 7fba03e00640 1 --2- 172.21.15.22:0/3497474030 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fba0c103840 0x7fba0c19a1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:07:53.760 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:53.759+0000 7fba03e00640 1 -- 172.21.15.22:0/3497474030 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fba0c109850 msgr2=0x7fba0c19ea30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:07:53.760 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:53.759+0000 7fba03e00640 1 --2- 172.21.15.22:0/3497474030 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fba0c109850 0x7fba0c19ea30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:07:53.761 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:53.759+0000 7fba03e00640 1 -- 172.21.15.22:0/3497474030 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fba0c19f160 con 0x7fba0c104200 2024-03-28T18:07:53.761 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:53.759+0000 7fba0aa00640 1 --2- 172.21.15.22:0/3497474030 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fba0c103840 0x7fba0c19a1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-28T18:07:53.761 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:53.759+0000 7fba03e00640 1 --2- 172.21.15.22:0/3497474030 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fba0c104200 0x7fba0c19a700 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7fb9f4000c00 tx=0x7fb9f4004e40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:07:53.761 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:53.759+0000 7fba03400640 1 -- 172.21.15.22:0/3497474030 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9f4043020 con 0x7fba0c104200 2024-03-28T18:07:53.762 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:53.759+0000 7fba03400640 1 -- 172.21.15.22:0/3497474030 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7fb9f402c5d0 con 0x7fba0c104200 2024-03-28T18:07:53.762 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:53.759+0000 7fba11800640 1 -- 172.21.15.22:0/3497474030 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fba0c19f3f0 con 0x7fba0c104200 2024-03-28T18:07:53.762 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:53.759+0000 7fba03400640 1 -- 172.21.15.22:0/3497474030 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9f4003a80 con 0x7fba0c104200 2024-03-28T18:07:53.762 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:53.760+0000 7fba11800640 1 -- 172.21.15.22:0/3497474030 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fba0c1a6ce0 con 0x7fba0c104200 2024-03-28T18:07:53.764 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:53.764+0000 7fba14588740 1 Processor -- start 2024-03-28T18:07:53.765 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:53.764+0000 7fba14588740 1 -- start start 2024-03-28T18:07:53.765 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:53.764+0000 7fba03400640 1 -- 172.21.15.22:0/3497474030 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 24) v1 ==== 137466+0+0 (secure 0 0 0) 0x7fb9f4046020 con 0x7fba0c104200 2024-03-28T18:07:53.766 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:53.764+0000 7fba14588740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561340056c00 0x561340055950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:07:53.766 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:53.764+0000 7fba14588740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x561340055f20 0x5613400563a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:07:53.766 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:53.764+0000 7fba14588740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56134005f290 0x561340061680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:07:53.766 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:53.764+0000 7fba14588740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x56133ffba250 con 0x56134005f290 2024-03-28T18:07:53.766 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:53.764+0000 7fba14588740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x56133ff62a30 con 0x561340055f20 2024-03-28T18:07:53.766 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:53.764+0000 7fba14588740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x56133ffb62f0 con 0x561340056c00 2024-03-28T18:07:53.766 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:53.765+0000 7fba03e00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56134005f290 0x561340061680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:07:53.766 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:53.765+0000 7fba0aa00640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x561340055f20 0x5613400563a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:07:53.766 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:53.765+0000 7fba03e00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56134005f290 0x561340061680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:49980/0 (socket says 172.21.15.22:49980) 2024-03-28T18:07:53.766 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:53.765+0000 7fba03e00640 1 -- 172.21.15.22:0/1400798678 learned_addr learned my addr 172.21.15.22:0/1400798678 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-28T18:07:53.766 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:53.765+0000 7fba0b400640 1 --2- 172.21.15.22:0/1400798678 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561340056c00 0x561340055950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:07:53.768 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:53.765+0000 7fba03e00640 1 -- 172.21.15.22:0/1400798678 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561340056c00 msgr2=0x561340055950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T18:07:53.768 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:53.765+0000 7fba03e00640 1 --2- 172.21.15.22:0/1400798678 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561340056c00 0x561340055950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:07:53.768 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:53.765+0000 7fba03e00640 1 -- 172.21.15.22:0/1400798678 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x561340055f20 msgr2=0x5613400563a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T18:07:53.768 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:53.765+0000 7fba03e00640 1 --2- 172.21.15.22:0/1400798678 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x561340055f20 0x5613400563a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:07:53.768 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:53.765+0000 7fba03e00640 1 -- 172.21.15.22:0/1400798678 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56133ff4d690 con 0x56134005f290 2024-03-28T18:07:53.768 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:53.766+0000 7fba03e00640 1 --2- 172.21.15.22:0/1400798678 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56134005f290 0x561340061680 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7fb9f40ee610 tx=0x7fb9f40f4e60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=425dc2412985450 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:07:53.768 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:53.766+0000 7fba03400640 1 --2- 172.21.15.22:0/3497474030 >> [v2:172.21.15.22:6800/1357365082,v1:172.21.15.22:6801/1357365082] conn(0x7fb9f00a2a80 0x7fb9f00a4f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:07:53.768 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:53.766+0000 7fb9fbe00640 1 -- 172.21.15.22:0/1400798678 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9f40b9770 con 0x56134005f290 2024-03-28T18:07:53.769 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:53.766+0000 7fb9fbe00640 1 -- 172.21.15.22:0/1400798678 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7fb9f40fe070 con 0x56134005f290 2024-03-28T18:07:53.769 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:53.766+0000 7fb9fbe00640 1 -- 172.21.15.22:0/1400798678 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9f40bcc50 con 0x56134005f290 2024-03-28T18:07:53.769 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:53.766+0000 7fba0b400640 1 --2- 172.21.15.22:0/3497474030 >> [v2:172.21.15.22:6800/1357365082,v1:172.21.15.22:6801/1357365082] conn(0x7fb9f00a2a80 0x7fb9f00a4f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:07:53.769 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:53.766+0000 7fba03400640 1 -- 172.21.15.22:0/3497474030 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 8411+0+0 (secure 0 0 0) 0x7fb9f40f0e60 con 0x7fba0c104200 2024-03-28T18:07:53.769 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:53.767+0000 7fba14588740 1 -- 172.21.15.22:0/1400798678 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56134005f290 msgr2=0x561340061680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:07:53.769 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:53.767+0000 7fba14588740 1 --2- 172.21.15.22:0/1400798678 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56134005f290 0x561340061680 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7fb9f40ee610 tx=0x7fb9f40f4e60 comp rx=0 tx=0).stop 2024-03-28T18:07:53.769 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:53.767+0000 7fba0b400640 1 --2- 172.21.15.22:0/3497474030 >> [v2:172.21.15.22:6800/1357365082,v1:172.21.15.22:6801/1357365082] conn(0x7fb9f00a2a80 0x7fb9f00a4f40 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fba04005fd0 tx=0x7fba04009500 comp rx=0 tx=0).ready entity=mgr.15522 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:07:53.769 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:53.767+0000 7fba14588740 1 -- 172.21.15.22:0/1400798678 shutdown_connections 2024-03-28T18:07:53.769 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:53.767+0000 7fba14588740 1 --2- 172.21.15.22:0/1400798678 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56134005f290 0x561340061680 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:07:53.769 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:53.767+0000 7fba14588740 1 --2- 172.21.15.22:0/1400798678 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x561340055f20 0x5613400563a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:07:53.769 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:53.767+0000 7fba14588740 1 --2- 172.21.15.22:0/1400798678 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561340056c00 0x561340055950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:07:53.770 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:53.767+0000 7fba14588740 1 -- 172.21.15.22:0/1400798678 >> 172.21.15.22:0/1400798678 conn(0x56134001d200 msgr2=0x561340050970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:07:53.770 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:53.768+0000 7fba14588740 1 -- 172.21.15.22:0/1400798678 shutdown_connections 2024-03-28T18:07:53.770 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:53.768+0000 7fba14588740 1 -- 172.21.15.22:0/1400798678 wait complete. 2024-03-28T18:07:53.777 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:53.768+0000 7fba14588740 1 -- start start 2024-03-28T18:07:53.777 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:53.768+0000 7fba14588740 1 Processor -- start 2024-03-28T18:07:53.777 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:53.769+0000 7fba14588740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561340055f20 0x56134005e930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:07:53.777 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:53.769+0000 7fba14588740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561340056c00 0x56134005bcc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:07:53.777 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:53.769+0000 7fba14588740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56134005f290 0x56134005c200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:07:53.777 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:53.769+0000 7fba14588740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x561340061ec0 con 0x561340056c00 2024-03-28T18:07:53.777 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:53.769+0000 7fba14588740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x56133ff75e80 con 0x56134005f290 2024-03-28T18:07:53.777 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:53.769+0000 7fba14588740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x561340062040 con 0x561340055f20 2024-03-28T18:07:53.777 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:53.769+0000 7fba14588740 10 client.0 _ll_register_callbacks cb 0x56133ff88130 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-28T18:07:53.777 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:53.769+0000 7fba03e00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561340056c00 0x56134005bcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:07:53.777 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:53.769+0000 7fba03e00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561340056c00 0x56134005bcc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:49984/0 (socket says 172.21.15.22:49984) 2024-03-28T18:07:53.778 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:53.769+0000 7fba03e00640 1 -- 172.21.15.22:0/2381361932 learned_addr learned my addr 172.21.15.22:0/2381361932 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-28T18:07:53.778 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:53.769+0000 7fba0b400640 1 --2- 172.21.15.22:0/2381361932 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56134005f290 0x56134005c200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:07:53.778 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:53.769+0000 7fba0aa00640 1 --2- 172.21.15.22:0/2381361932 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561340055f20 0x56134005e930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:07:53.778 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:53.769+0000 7fba03e00640 1 -- 172.21.15.22:0/2381361932 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561340055f20 msgr2=0x56134005e930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T18:07:53.778 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:53.770+0000 7fba03e00640 1 --2- 172.21.15.22:0/2381361932 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561340055f20 0x56134005e930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:07:53.778 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:53.770+0000 7fba03e00640 1 -- 172.21.15.22:0/2381361932 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56134005f290 msgr2=0x56134005c200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T18:07:53.778 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:53.770+0000 7fba03e00640 1 --2- 172.21.15.22:0/2381361932 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56134005f290 0x56134005c200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:07:53.778 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:53.770+0000 7fba03e00640 1 -- 172.21.15.22:0/2381361932 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56133fd987d0 con 0x561340056c00 2024-03-28T18:07:53.778 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:53.770+0000 7fba03e00640 1 --2- 172.21.15.22:0/2381361932 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561340056c00 0x56134005bcc0 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7fb9f40ee750 tx=0x7fb9f4004fb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=22ce928844b29a50 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:07:53.778 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:53.770+0000 7fb9f8c00640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-03-28T18:07:53.778 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:53.770+0000 7fb9f8c00640 1 -- 172.21.15.22:0/2381361932 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9f40bce30 con 0x561340056c00 2024-03-28T18:07:53.778 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:53.770+0000 7fb9f8c00640 1 -- 172.21.15.22:0/2381361932 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 8411+0+0 (secure 0 0 0) 0x7fb9f402cab0 con 0x561340056c00 2024-03-28T18:07:53.778 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:53.770+0000 7fba14588740 1 -- 172.21.15.22:0/2381361932 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fb9f40fe070 con 0x561340056c00 2024-03-28T18:07:53.779 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:53.770+0000 7fb9f8c00640 1 -- 172.21.15.22:0/2381361932 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb9f40bd010 con 0x561340056c00 2024-03-28T18:07:53.779 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:53.770+0000 7fba14588740 10 client.15624 fetch_fsmap learned FSMap version 27 2024-03-28T18:07:53.779 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:53.770+0000 7fba14588740 1 -- 172.21.15.22:0/2381361932 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x56133fcce3f0 con 0x561340056c00 2024-03-28T18:07:53.779 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:53.771+0000 7fb9f8c00640 1 -- 172.21.15.22:0/2381361932 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7fb9f40fe070 con 0x561340056c00 2024-03-28T18:07:53.779 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:53.771+0000 7fb9f8c00640 1 -- 172.21.15.22:0/2381361932 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 27) v1 ==== 7934+0+0 (secure 0 0 0) 0x7fb9f40ee810 con 0x561340056c00 2024-03-28T18:07:53.779 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:53.771+0000 7fba14588740 10 client.15624 fetch_fsmap finished waiting for FSMap version 27 2024-03-28T18:07:53.779 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:53.771+0000 7fba14588740 10 client.15624 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-28T18:07:53.779 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:53.771+0000 7fba14588740 20 client.15624 populate_metadata read hostname 'smithi022' 2024-03-28T18:07:53.779 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:53.771+0000 7fba14588740 1 --2- 172.21.15.22:0/2381361932 >> [v2:172.21.15.79:6838/2441542443,v1:172.21.15.79:6839/2441542443] conn(0x56134006ed20 0x561340071110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:07:53.779 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:53.771+0000 7fba14588740 4 client.15624 mds_command: new command op to 34553 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-28T18:07:53.779 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:53.771+0000 7fba14588740 1 -- 172.21.15.22:0/2381361932 --> [v2:172.21.15.79:6838/2441542443,v1:172.21.15.79:6839/2441542443] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56133ffc7790 con 0x56134006ed20 2024-03-28T18:07:53.779 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:53.772+0000 7fba0aa00640 1 --2- 172.21.15.22:0/2381361932 >> [v2:172.21.15.79:6838/2441542443,v1:172.21.15.79:6839/2441542443] conn(0x56134006ed20 0x561340071110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:07:53.779 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:53.772+0000 7fba0aa00640 1 --2- 172.21.15.22:0/2381361932 >> [v2:172.21.15.79:6838/2441542443,v1:172.21.15.79:6839/2441542443] conn(0x56134006ed20 0x561340071110 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=124ec6779a54d979 server_cookie=b92c8e055390a939 in_seq=0 out_seq=0 2024-03-28T18:07:53.779 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:53.772+0000 7fb9f8c00640 10 client.15624 ms_handle_connect on v2:172.21.15.79:6838/2441542443 2024-03-28T18:07:53.779 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:53.774+0000 7fb9f8c00640 1 -- 172.21.15.22:0/2381361932 <== mds.0 v2:172.21.15.79:6838/2441542443 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x56133ffc7790 con 0x56134006ed20 2024-03-28T18:07:53.780 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:53.774+0000 7fb9f8c00640 10 client.15624 handle_command_reply: tid=0 2024-03-28T18:07:53.780 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:53.774+0000 7fba14588740 1 client.15624 shutdown 2024-03-28T18:07:53.780 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:53.774+0000 7fba14588740 1 -- 172.21.15.22:0/2381361932 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561340056c00 msgr2=0x56134005bcc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:07:53.780 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:53.774+0000 7fba14588740 1 --2- 172.21.15.22:0/2381361932 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561340056c00 0x56134005bcc0 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7fb9f40ee750 tx=0x7fb9f4004fb0 comp rx=0 tx=0).stop 2024-03-28T18:07:53.780 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:53.774+0000 7fba14588740 1 -- 172.21.15.22:0/2381361932 shutdown_connections 2024-03-28T18:07:53.780 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:53.774+0000 7fba14588740 1 --2- 172.21.15.22:0/2381361932 >> [v2:172.21.15.79:6838/2441542443,v1:172.21.15.79:6839/2441542443] conn(0x56134006ed20 0x561340071110 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:07:53.780 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:53.774+0000 7fba14588740 1 --2- 172.21.15.22:0/2381361932 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56134005f290 0x56134005c200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:07:53.780 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:53.774+0000 7fba14588740 1 --2- 172.21.15.22:0/2381361932 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561340056c00 0x56134005bcc0 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:07:53.780 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:53.774+0000 7fba14588740 1 --2- 172.21.15.22:0/2381361932 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561340055f20 0x56134005e930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:07:53.780 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:53.774+0000 7fb9f8c00640 0 client.15624 ms_handle_reset on v2:172.21.15.79:6838/2441542443 2024-03-28T18:07:53.780 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:53.774+0000 7fba14588740 1 -- 172.21.15.22:0/2381361932 >> 172.21.15.22:0/2381361932 conn(0x56134001d200 msgr2=0x5613400526c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:07:53.780 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:53.775+0000 7fba14588740 1 -- 172.21.15.22:0/2381361932 shutdown_connections 2024-03-28T18:07:53.780 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:53.775+0000 7fba14588740 1 -- 172.21.15.22:0/2381361932 wait complete. 2024-03-28T18:07:53.780 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:53.775+0000 7fba14588740 20 client.15624 trim_cache size 0 max 16384 2024-03-28T18:07:53.793 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:53.792+0000 7fba14588740 1 Processor -- start 2024-03-28T18:07:53.793 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:53.792+0000 7fba14588740 1 -- start start 2024-03-28T18:07:53.794 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:53.793+0000 7fba14588740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561340055f20 0x56134005e3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:07:53.794 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:53.793+0000 7fba14588740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561340056c00 0x56134005e930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:07:53.794 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:53.793+0000 7fba14588740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56134005f290 0x56134004f910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:07:53.794 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:53.793+0000 7fba14588740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba0c1af210 con 0x561340055f20 2024-03-28T18:07:53.794 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:53.793+0000 7fba14588740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba0c1b0b90 con 0x56134005f290 2024-03-28T18:07:53.794 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:53.793+0000 7fba14588740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba0c1af870 con 0x561340056c00 2024-03-28T18:07:53.794 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:53.793+0000 7fba0aa00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561340055f20 0x56134005e3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:07:53.794 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:53.793+0000 7fba0aa00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561340055f20 0x56134005e3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:49988/0 (socket says 172.21.15.22:49988) 2024-03-28T18:07:53.794 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:53.793+0000 7fba0aa00640 1 -- 172.21.15.22:0/2868594881 learned_addr learned my addr 172.21.15.22:0/2868594881 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-28T18:07:53.795 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:53.793+0000 7fba0b400640 1 --2- 172.21.15.22:0/2868594881 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56134005f290 0x56134004f910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:07:53.795 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:53.794+0000 7fba0aa00640 1 -- 172.21.15.22:0/2868594881 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561340056c00 msgr2=0x56134005e930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T18:07:53.795 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:53.794+0000 7fba03e00640 1 --2- 172.21.15.22:0/2868594881 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561340056c00 0x56134005e930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:07:53.795 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:53.794+0000 7fba0aa00640 1 --2- 172.21.15.22:0/2868594881 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561340056c00 0x56134005e930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:07:53.796 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:53.794+0000 7fba0aa00640 1 -- 172.21.15.22:0/2868594881 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56134005f290 msgr2=0x56134004f910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T18:07:53.796 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:53.794+0000 7fba0aa00640 1 --2- 172.21.15.22:0/2868594881 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56134005f290 0x56134004f910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:07:53.796 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:53.794+0000 7fba0aa00640 1 -- 172.21.15.22:0/2868594881 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56133fc8e0f0 con 0x561340055f20 2024-03-28T18:07:53.796 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:53.794+0000 7fba03e00640 1 --2- 172.21.15.22:0/2868594881 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561340056c00 0x56134005e930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T18:07:53.796 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:53.794+0000 7fba0b400640 1 --2- 172.21.15.22:0/2868594881 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56134005f290 0x56134004f910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-28T18:07:53.796 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:53.794+0000 7fba0aa00640 1 --2- 172.21.15.22:0/2868594881 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561340055f20 0x56134005e3f0 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x561340071710 tx=0x7fb9fc005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3ac84425f8c3e494 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:07:53.796 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:53.794+0000 7fb9fb400640 1 -- 172.21.15.22:0/2868594881 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9fc024070 con 0x561340055f20 2024-03-28T18:07:53.796 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:53.794+0000 7fb9fb400640 1 -- 172.21.15.22:0/2868594881 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7fb9fc020c40 con 0x561340055f20 2024-03-28T18:07:53.796 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:53.794+0000 7fb9fb400640 1 -- 172.21.15.22:0/2868594881 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9fc012870 con 0x561340055f20 2024-03-28T18:07:53.797 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:53.795+0000 7fba14588740 1 -- 172.21.15.22:0/2868594881 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561340055f20 msgr2=0x56134005e3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:07:53.797 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:53.795+0000 7fba14588740 1 --2- 172.21.15.22:0/2868594881 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561340055f20 0x56134005e3f0 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x561340071710 tx=0x7fb9fc005f90 comp rx=0 tx=0).stop 2024-03-28T18:07:53.797 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:53.795+0000 7fba14588740 1 -- 172.21.15.22:0/2868594881 shutdown_connections 2024-03-28T18:07:53.797 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:53.795+0000 7fba14588740 1 --2- 172.21.15.22:0/2868594881 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56134005f290 0x56134004f910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:07:53.797 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:53.795+0000 7fba14588740 1 --2- 172.21.15.22:0/2868594881 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561340056c00 0x56134005e930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:07:53.797 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:53.795+0000 7fba14588740 1 --2- 172.21.15.22:0/2868594881 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561340055f20 0x56134005e3f0 unknown :-1 s=CLOSED pgs=470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:07:53.797 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:53.795+0000 7fba14588740 1 -- 172.21.15.22:0/2868594881 >> 172.21.15.22:0/2868594881 conn(0x56134001d200 msgr2=0x56134004f200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:07:53.797 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:53.796+0000 7fba14588740 1 -- 172.21.15.22:0/2868594881 shutdown_connections 2024-03-28T18:07:53.797 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:53.796+0000 7fba14588740 1 -- 172.21.15.22:0/2868594881 wait complete. 2024-03-28T18:07:53.798 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:53.796+0000 7fba14588740 1 -- start start 2024-03-28T18:07:53.798 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:53.797+0000 7fba14588740 1 Processor -- start 2024-03-28T18:07:53.799 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:53.797+0000 7fba14588740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561340056c00 0x561340064260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:07:53.799 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:53.797+0000 7fba14588740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56134005f290 0x5613400647a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:07:53.799 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:53.797+0000 7fba14588740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561340064ce0 0x5613400ca370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:07:53.800 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:53.797+0000 7fba14588740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba0c1b0530 con 0x561340056c00 2024-03-28T18:07:53.800 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:53.797+0000 7fba14588740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba0c1afed0 con 0x56134005f290 2024-03-28T18:07:53.800 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:53.797+0000 7fba14588740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba0c1ad3d0 con 0x561340064ce0 2024-03-28T18:07:53.800 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:53.797+0000 7fba14588740 10 client.0 _ll_register_callbacks cb 0x7fb9fc009550 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-28T18:07:53.800 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:53.798+0000 7fba0aa00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561340056c00 0x561340064260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:07:53.800 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:53.798+0000 7fba0aa00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561340056c00 0x561340064260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:49992/0 (socket says 172.21.15.22:49992) 2024-03-28T18:07:53.800 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:53.798+0000 7fba0aa00640 1 -- 172.21.15.22:0/3852193446 learned_addr learned my addr 172.21.15.22:0/3852193446 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-28T18:07:53.800 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:53.798+0000 7fba03e00640 1 --2- 172.21.15.22:0/3852193446 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56134005f290 0x5613400647a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:07:53.800 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:53.798+0000 7fba0b400640 1 --2- 172.21.15.22:0/3852193446 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561340064ce0 0x5613400ca370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:07:53.800 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:53.798+0000 7fba0aa00640 1 -- 172.21.15.22:0/3852193446 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561340064ce0 msgr2=0x5613400ca370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T18:07:53.800 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:53.798+0000 7fba0aa00640 1 --2- 172.21.15.22:0/3852193446 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561340064ce0 0x5613400ca370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:07:53.800 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:53.798+0000 7fba0aa00640 1 -- 172.21.15.22:0/3852193446 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56134005f290 msgr2=0x5613400647a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T18:07:53.800 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:53.798+0000 7fba0aa00640 1 --2- 172.21.15.22:0/3852193446 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56134005f290 0x5613400647a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:07:53.800 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:53.798+0000 7fba0aa00640 1 -- 172.21.15.22:0/3852193446 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x561340059cb0 con 0x561340056c00 2024-03-28T18:07:53.801 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:53.798+0000 7fba03e00640 1 --2- 172.21.15.22:0/3852193446 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56134005f290 0x5613400647a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-28T18:07:53.801 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:53.799+0000 7fba0aa00640 1 --2- 172.21.15.22:0/3852193446 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561340056c00 0x561340064260 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7fb9fc005d00 tx=0x7fb9fc009fa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2047693bdf7e821b server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:07:53.801 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:53.799+0000 7fb9f8c00640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-03-28T18:07:53.801 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:53.799+0000 7fb9f8c00640 1 -- 172.21.15.22:0/3852193446 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9fc02d030 con 0x561340056c00 2024-03-28T18:07:53.801 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:53.799+0000 7fba14588740 1 -- 172.21.15.22:0/3852193446 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fb9fc020c40 con 0x561340056c00 2024-03-28T18:07:53.801 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:53.799+0000 7fb9f8c00640 1 -- 172.21.15.22:0/3852193446 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 8411+0+0 (secure 0 0 0) 0x7fb9fc003b60 con 0x561340056c00 2024-03-28T18:07:53.803 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:53.799+0000 7fb9f8c00640 1 -- 172.21.15.22:0/3852193446 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb9fc037050 con 0x561340056c00 2024-03-28T18:07:53.803 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:53.799+0000 7fba14588740 10 client.15630 fetch_fsmap learned FSMap version 27 2024-03-28T18:07:53.803 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:53.799+0000 7fba14588740 1 -- 172.21.15.22:0/3852193446 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x5613400cb890 con 0x561340056c00 2024-03-28T18:07:53.803 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:53.800+0000 7fb9f8c00640 1 -- 172.21.15.22:0/3852193446 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7fb9fc020c40 con 0x561340056c00 2024-03-28T18:07:53.803 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:53.800+0000 7fb9f8c00640 1 -- 172.21.15.22:0/3852193446 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 27) v1 ==== 7934+0+0 (secure 0 0 0) 0x7fb9fc004730 con 0x561340056c00 2024-03-28T18:07:53.803 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:53.800+0000 7fba14588740 10 client.15630 fetch_fsmap finished waiting for FSMap version 27 2024-03-28T18:07:53.803 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:53.800+0000 7fba14588740 10 client.15630 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-28T18:07:53.804 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:53.800+0000 7fba14588740 20 client.15630 populate_metadata read hostname 'smithi022' 2024-03-28T18:07:53.804 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:53.800+0000 7fba14588740 1 --2- 172.21.15.22:0/3852193446 >> [v2:172.21.15.79:6838/2441542443,v1:172.21.15.79:6839/2441542443] conn(0x5613400cbf20 0x5613400ce340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:07:53.804 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:53.800+0000 7fba14588740 4 client.15630 mds_command: new command op to 34553 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-28T18:07:53.804 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:53.800+0000 7fba14588740 1 -- 172.21.15.22:0/3852193446 --> [v2:172.21.15.79:6838/2441542443,v1:172.21.15.79:6839/2441542443] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x56133ffc1ae0 con 0x5613400cbf20 2024-03-28T18:07:53.804 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:53.800+0000 7fba03e00640 1 --2- 172.21.15.22:0/3852193446 >> [v2:172.21.15.79:6838/2441542443,v1:172.21.15.79:6839/2441542443] conn(0x5613400cbf20 0x5613400ce340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:07:53.804 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:53.801+0000 7fba03e00640 1 --2- 172.21.15.22:0/3852193446 >> [v2:172.21.15.79:6838/2441542443,v1:172.21.15.79:6839/2441542443] conn(0x5613400cbf20 0x5613400ce340 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8d00323e5bc6ddbe server_cookie=4ae86a5415990bb7 in_seq=0 out_seq=0 2024-03-28T18:07:53.804 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:53.801+0000 7fb9f8c00640 10 client.15630 ms_handle_connect on v2:172.21.15.79:6838/2441542443 2024-03-28T18:07:53.804 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:53.801+0000 7fb9f8c00640 1 -- 172.21.15.22:0/3852193446 <== mds.0 v2:172.21.15.79:6838/2441542443 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x56133ffc1ae0 con 0x5613400cbf20 2024-03-28T18:07:53.804 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:53.801+0000 7fb9f8c00640 10 client.15630 handle_command_reply: tid=0 2024-03-28T18:07:53.804 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:53.801+0000 7fba14588740 1 client.15630 shutdown 2024-03-28T18:07:53.804 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:53.801+0000 7fba14588740 1 -- 172.21.15.22:0/3852193446 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561340056c00 msgr2=0x561340064260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:07:53.804 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:53.801+0000 7fba14588740 1 --2- 172.21.15.22:0/3852193446 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561340056c00 0x561340064260 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7fb9fc005d00 tx=0x7fb9fc009fa0 comp rx=0 tx=0).stop 2024-03-28T18:07:53.804 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:53.802+0000 7fba14588740 1 -- 172.21.15.22:0/3852193446 shutdown_connections 2024-03-28T18:07:53.804 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:53.802+0000 7fba14588740 1 --2- 172.21.15.22:0/3852193446 >> [v2:172.21.15.79:6838/2441542443,v1:172.21.15.79:6839/2441542443] conn(0x5613400cbf20 0x5613400ce340 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:07:53.805 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:53.802+0000 7fba14588740 1 --2- 172.21.15.22:0/3852193446 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561340064ce0 0x5613400ca370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:07:53.805 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:53.802+0000 7fba14588740 1 --2- 172.21.15.22:0/3852193446 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56134005f290 0x5613400647a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:07:53.805 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:53.802+0000 7fba14588740 1 --2- 172.21.15.22:0/3852193446 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561340056c00 0x561340064260 unknown :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:07:53.805 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:53.802+0000 7fba14588740 1 -- 172.21.15.22:0/3852193446 >> 172.21.15.22:0/3852193446 conn(0x56134001d200 msgr2=0x56134004fa90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:07:53.805 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:53.802+0000 7fb9f8c00640 0 client.15630 ms_handle_reset on v2:172.21.15.79:6838/2441542443 2024-03-28T18:07:53.805 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:53.802+0000 7fba14588740 1 -- 172.21.15.22:0/3852193446 shutdown_connections 2024-03-28T18:07:53.805 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:53.802+0000 7fba14588740 1 -- 172.21.15.22:0/3852193446 wait complete. 2024-03-28T18:07:53.805 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:53.802+0000 7fba14588740 20 client.15630 trim_cache size 0 max 16384 2024-03-28T18:07:53.805 INFO:teuthology.orchestra.run.smithi022.stdout:{ 2024-03-28T18:07:53.805 INFO:teuthology.orchestra.run.smithi022.stdout: "status": "no active scrubs running", 2024-03-28T18:07:53.805 INFO:teuthology.orchestra.run.smithi022.stdout: "scrubs": {} 2024-03-28T18:07:53.806 INFO:teuthology.orchestra.run.smithi022.stdout:} 2024-03-28T18:07:53.806 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:53.803+0000 7fba11800640 1 -- 172.21.15.22:0/3497474030 >> [v2:172.21.15.22:6800/1357365082,v1:172.21.15.22:6801/1357365082] conn(0x7fb9f00a2a80 msgr2=0x7fb9f00a4f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:07:53.806 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:53.803+0000 7fba11800640 1 --2- 172.21.15.22:0/3497474030 >> [v2:172.21.15.22:6800/1357365082,v1:172.21.15.22:6801/1357365082] conn(0x7fb9f00a2a80 0x7fb9f00a4f40 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fba04005fd0 tx=0x7fba04009500 comp rx=0 tx=0).stop 2024-03-28T18:07:53.806 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:53.803+0000 7fba11800640 1 -- 172.21.15.22:0/3497474030 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fba0c104200 msgr2=0x7fba0c19a700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:07:53.806 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:53.803+0000 7fba11800640 1 --2- 172.21.15.22:0/3497474030 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fba0c104200 0x7fba0c19a700 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7fb9f4000c00 tx=0x7fb9f4004e40 comp rx=0 tx=0).stop 2024-03-28T18:07:53.806 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:53.803+0000 7fba11800640 1 -- 172.21.15.22:0/3497474030 shutdown_connections 2024-03-28T18:07:53.806 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:53.803+0000 7fba11800640 1 --2- 172.21.15.22:0/3497474030 >> [v2:172.21.15.22:6800/1357365082,v1:172.21.15.22:6801/1357365082] conn(0x7fb9f00a2a80 0x7fb9f00a4f40 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:07:53.807 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:53.803+0000 7fba11800640 1 --2- 172.21.15.22:0/3497474030 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fba0c109850 0x7fba0c19ea30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:07:53.807 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:53.803+0000 7fba11800640 1 --2- 172.21.15.22:0/3497474030 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fba0c104200 0x7fba0c19a700 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:07:53.807 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:53.803+0000 7fba11800640 1 --2- 172.21.15.22:0/3497474030 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fba0c103840 0x7fba0c19a1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:07:53.807 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:53.803+0000 7fba11800640 1 -- 172.21.15.22:0/3497474030 >> 172.21.15.22:0/3497474030 conn(0x7fba0c0ff310 msgr2=0x7fba0c0ffaa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:07:53.807 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:53.803+0000 7fba11800640 1 -- 172.21.15.22:0/3497474030 shutdown_connections 2024-03-28T18:07:53.807 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:53.804+0000 7fba11800640 1 -- 172.21.15.22:0/3497474030 wait complete. 2024-03-28T18:07:53.829 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-03-28T18:07:53.830 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-28T18:07:54.034 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:54.033+0000 7f9ded200640 1 -- 172.21.15.22:0/3550312351 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f9de81014b0 msgr2=0x7f9de8101930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:07:54.034 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:54.033+0000 7f9ded200640 1 --2- 172.21.15.22:0/3550312351 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f9de81014b0 0x7f9de8101930 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f9dd4009930 tx=0x7f9dd402bc20 comp rx=0 tx=0).stop 2024-03-28T18:07:54.034 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:54.033+0000 7f9ded200640 1 -- 172.21.15.22:0/3550312351 shutdown_connections 2024-03-28T18:07:54.034 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:54.033+0000 7f9ded200640 1 --2- 172.21.15.22:0/3550312351 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9de8101e70 0x7f9de810e330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:07:54.034 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:54.033+0000 7f9ded200640 1 --2- 172.21.15.22:0/3550312351 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f9de81014b0 0x7f9de8101930 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:07:54.034 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:54.033+0000 7f9ded200640 1 --2- 172.21.15.22:0/3550312351 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9de8107570 0x7f9de8107950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:07:54.034 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:54.033+0000 7f9ded200640 1 -- 172.21.15.22:0/3550312351 >> 172.21.15.22:0/3550312351 conn(0x7f9de80fcf80 msgr2=0x7f9de80ff3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:07:54.034 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:54.033+0000 7f9ded200640 1 -- 172.21.15.22:0/3550312351 shutdown_connections 2024-03-28T18:07:54.034 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:54.033+0000 7f9ded200640 1 -- 172.21.15.22:0/3550312351 wait complete. 2024-03-28T18:07:54.037 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:54.034+0000 7f9ded200640 1 Processor -- start 2024-03-28T18:07:54.037 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:54.034+0000 7f9ded200640 1 -- start start 2024-03-28T18:07:54.037 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:54.035+0000 7f9ded200640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f9de81014b0 0x7f9de8108340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:07:54.037 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:54.035+0000 7f9ded200640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9de8101e70 0x7f9de8108880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:07:54.037 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:54.035+0000 7f9ded200640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9de8107570 0x7f9de8108dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:07:54.037 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:54.035+0000 7f9ded200640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9de8111880 con 0x7f9de8101e70 2024-03-28T18:07:54.037 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:54.035+0000 7f9ded200640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9de8111700 con 0x7f9de81014b0 2024-03-28T18:07:54.037 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:54.035+0000 7f9ded200640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9de8111a00 con 0x7f9de8107570 2024-03-28T18:07:54.038 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:54.035+0000 7f9de6a00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9de8101e70 0x7f9de8108880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:07:54.038 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:54.035+0000 7f9de6a00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9de8101e70 0x7f9de8108880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:50016/0 (socket says 172.21.15.22:50016) 2024-03-28T18:07:54.038 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:54.035+0000 7f9de6a00640 1 -- 172.21.15.22:0/3347287825 learned_addr learned my addr 172.21.15.22:0/3347287825 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-28T18:07:54.038 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:54.035+0000 7f9de7400640 1 --2- 172.21.15.22:0/3347287825 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f9de81014b0 0x7f9de8108340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:07:54.038 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:54.035+0000 7f9de7e00640 1 --2- 172.21.15.22:0/3347287825 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9de8107570 0x7f9de8108dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:07:54.038 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:54.035+0000 7f9de6a00640 1 -- 172.21.15.22:0/3347287825 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9de8107570 msgr2=0x7f9de8108dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:07:54.038 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:54.035+0000 7f9de6a00640 1 --2- 172.21.15.22:0/3347287825 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9de8107570 0x7f9de8108dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:07:54.038 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:54.035+0000 7f9de6a00640 1 -- 172.21.15.22:0/3347287825 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f9de81014b0 msgr2=0x7f9de8108340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:07:54.038 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:54.035+0000 7f9de6a00640 1 --2- 172.21.15.22:0/3347287825 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f9de81014b0 0x7f9de8108340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:07:54.038 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:54.035+0000 7f9de6a00640 1 -- 172.21.15.22:0/3347287825 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9de81aa600 con 0x7f9de8101e70 2024-03-28T18:07:54.038 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:54.036+0000 7f9de7e00640 1 --2- 172.21.15.22:0/3347287825 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9de8107570 0x7f9de8108dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T18:07:54.038 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:54.036+0000 7f9de6a00640 1 --2- 172.21.15.22:0/3347287825 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9de8101e70 0x7f9de8108880 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f9dd4036040 tx=0x7f9dd40045e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:07:54.038 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:54.036+0000 7f9ddfe00640 1 -- 172.21.15.22:0/3347287825 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9dd4043020 con 0x7f9de8101e70 2024-03-28T18:07:54.039 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:54.036+0000 7f9ded200640 1 -- 172.21.15.22:0/3347287825 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9de81aa890 con 0x7f9de8101e70 2024-03-28T18:07:54.039 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:54.036+0000 7f9ddfe00640 1 -- 172.21.15.22:0/3347287825 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f9dd402c5c0 con 0x7f9de8101e70 2024-03-28T18:07:54.039 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:54.036+0000 7f9ddfe00640 1 -- 172.21.15.22:0/3347287825 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9dd40039f0 con 0x7f9de8101e70 2024-03-28T18:07:54.039 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:54.036+0000 7f9ded200640 1 -- 172.21.15.22:0/3347287825 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9de81aae30 con 0x7f9de8101e70 2024-03-28T18:07:54.039 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:54.038+0000 7f9ded200640 1 -- 172.21.15.22:0/3347287825 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9de8102bd0 con 0x7f9de8101e70 2024-03-28T18:07:54.040 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:54.040+0000 7f9ddfe00640 1 -- 172.21.15.22:0/3347287825 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 24) v1 ==== 137466+0+0 (secure 0 0 0) 0x7f9dd4046020 con 0x7f9de8101e70 2024-03-28T18:07:54.042 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:54.041+0000 7f9ddfe00640 1 --2- 172.21.15.22:0/3347287825 >> [v2:172.21.15.22:6800/1357365082,v1:172.21.15.22:6801/1357365082] conn(0x7f9dcc0a2a00 0x7f9dcc0a4ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:07:54.042 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:54.041+0000 7f9ddfe00640 1 -- 172.21.15.22:0/3347287825 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 8411+0+0 (secure 0 0 0) 0x7f9dd40f1120 con 0x7f9de8101e70 2024-03-28T18:07:54.042 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:54.041+0000 7f9de7400640 1 --2- 172.21.15.22:0/3347287825 >> [v2:172.21.15.22:6800/1357365082,v1:172.21.15.22:6801/1357365082] conn(0x7f9dcc0a2a00 0x7f9dcc0a4ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:07:54.043 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:54.042+0000 7f9de7400640 1 --2- 172.21.15.22:0/3347287825 >> [v2:172.21.15.22:6800/1357365082,v1:172.21.15.22:6801/1357365082] conn(0x7f9dcc0a2a00 0x7f9dcc0a4ec0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f9dd8005fd0 tx=0x7f9dd800b390 comp rx=0 tx=0).ready entity=mgr.15522 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:07:54.045 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:54.044+0000 7f9ddfe00640 1 -- 172.21.15.22:0/3347287825 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f9dd404d070 con 0x7f9de8101e70 2024-03-28T18:07:54.283 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:54.282+0000 7f9ded200640 1 -- 172.21.15.22:0/3347287825 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f9de80683b0 con 0x7f9de8101e70 2024-03-28T18:07:54.284 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:54.284+0000 7f9ddfe00640 1 -- 172.21.15.22:0/3347287825 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) v1 ==== 94+0+11452 (secure 0 0 0) 0x7f9dd40bde30 con 0x7f9de8101e70 2024-03-28T18:07:54.285 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-03-28T18:07:54.285 INFO:teuthology.orchestra.run.smithi022.stdout:{"epoch":27,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14544,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6835/3718238950","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6834","nonce":3718238950},{"type":"v1","addr":"172.21.15.22:6835","nonce":3718238950}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14562,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6837/3676939386","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6836","nonce":3676939386},{"type":"v1","addr":"172.21.15.22:6837","nonce":3676939386}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":27,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-28T18:04:48.991622+0000","modified":"2024-03-28T18:07:13.521383+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":34553,"mds_1":34511,"mds_2":34475,"mds_3":34442,"mds_4":24502},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14640":{"gid":14640,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.22:6839/3979998310","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6838","nonce":3979998310},{"type":"v1","addr":"172.21.15.22:6839","nonce":3979998310}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_14655":{"gid":14655,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.22:6841/3069056661","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6840","nonce":3069056661},{"type":"v1","addr":"172.21.15.22:6841","nonce":3069056661}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24412":{"gid":24412,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.134:6833/3176125362","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6832","nonce":3176125362},{"type":"v1","addr":"172.21.15.134:6833","nonce":3176125362}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24463":{"gid":24463,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.134:6835/3370549207","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6834","nonce":3370549207},{"type":"v1","addr":"172.21.15.134:6835","nonce":3370549207}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24478":{"gid":24478,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.134:6837/899825581","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6836","nonce":899825581},{"type":"v1","addr":"172.21.15.134:6837","nonce":899825581}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24502":{"gid":24502,"name":"i","rank":4,"incarnation":24,"state":"up:active","state_seq":7,"addr":"172.21.15.134:6839/3400263217","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6838","nonce":3400263217},{"type":"v1","addr":"172.21.15.134:6839","nonce":3400263217}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_34442":{"gid":34442,"name":"k","rank":3,"incarnation":22,"state":"up:active","state_seq":14,"addr":"172.21.15.79:6833/856896997","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6832","nonce":856896997},{"type":"v1","addr":"172.21.15.79:6833","nonce":856896997}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_34475":{"gid":34475,"name":"e","rank":2,"incarnation":20,"state":"up:active","state_seq":11,"addr":"172.21.15.79:6835/3365773431","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6834","nonce":3365773431},{"type":"v1","addr":"172.21.15.79:6835","nonce":3365773431}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_34511":{"gid":34511,"name":"h","rank":1,"incarnation":18,"state":"up:active","state_seq":8,"addr":"172.21.15.79:6837/1787903366","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6836","nonce":1787903366},{"type":"v1","addr":"172.21.15.79:6837","nonce":1787903366}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_34553":{"gid":34553,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.79:6839/2441542443","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6838","nonce":2441542443},{"type":"v1","addr":"172.21.15.79:6839","nonce":2441542443}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":34553,"qdb_cluster":[24502,34442,34475,34511,34553]},"id":1}]} 2024-03-28T18:07:54.285 INFO:teuthology.orchestra.run.smithi022.stderr:dumped fsmap epoch 27 2024-03-28T18:07:54.287 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:54.286+0000 7f9ded200640 1 -- 172.21.15.22:0/3347287825 >> [v2:172.21.15.22:6800/1357365082,v1:172.21.15.22:6801/1357365082] conn(0x7f9dcc0a2a00 msgr2=0x7f9dcc0a4ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:07:54.287 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:54.286+0000 7f9ded200640 1 --2- 172.21.15.22:0/3347287825 >> [v2:172.21.15.22:6800/1357365082,v1:172.21.15.22:6801/1357365082] conn(0x7f9dcc0a2a00 0x7f9dcc0a4ec0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f9dd8005fd0 tx=0x7f9dd800b390 comp rx=0 tx=0).stop 2024-03-28T18:07:54.287 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:54.286+0000 7f9ded200640 1 -- 172.21.15.22:0/3347287825 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9de8101e70 msgr2=0x7f9de8108880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:07:54.287 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:54.286+0000 7f9ded200640 1 --2- 172.21.15.22:0/3347287825 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9de8101e70 0x7f9de8108880 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f9dd4036040 tx=0x7f9dd40045e0 comp rx=0 tx=0).stop 2024-03-28T18:07:54.287 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:54.286+0000 7f9ded200640 1 -- 172.21.15.22:0/3347287825 shutdown_connections 2024-03-28T18:07:54.287 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:54.286+0000 7f9ded200640 1 --2- 172.21.15.22:0/3347287825 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9de8107570 0x7f9de8108dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:07:54.287 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:54.286+0000 7f9ded200640 1 --2- 172.21.15.22:0/3347287825 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9de8101e70 0x7f9de8108880 unknown :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:07:54.287 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:54.286+0000 7f9ded200640 1 --2- 172.21.15.22:0/3347287825 >> [v2:172.21.15.22:6800/1357365082,v1:172.21.15.22:6801/1357365082] conn(0x7f9dcc0a2a00 0x7f9dcc0a4ec0 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:07:54.288 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:54.286+0000 7f9ded200640 1 --2- 172.21.15.22:0/3347287825 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f9de81014b0 0x7f9de8108340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:07:54.288 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:54.286+0000 7f9ded200640 1 -- 172.21.15.22:0/3347287825 >> 172.21.15.22:0/3347287825 conn(0x7f9de80fcf80 msgr2=0x7f9de810d010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:07:54.288 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:54.286+0000 7f9ded200640 1 -- 172.21.15.22:0/3347287825 shutdown_connections 2024-03-28T18:07:54.288 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:54.286+0000 7f9ded200640 1 -- 172.21.15.22:0/3347287825 wait complete. 2024-03-28T18:07:54.307 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-28T18:07:54.307 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2024-03-28T18:07:54.510 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:54.509+0000 7f7072000640 1 -- 172.21.15.22:0/3703854543 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f706c10bce0 msgr2=0x7f706c10e150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:07:54.510 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:54.509+0000 7f7072000640 1 --2- 172.21.15.22:0/3703854543 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f706c10bce0 0x7f706c10e150 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f7064009ca0 tx=0x7f706402c180 comp rx=0 tx=0).stop 2024-03-28T18:07:54.510 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:54.509+0000 7f7072000640 1 -- 172.21.15.22:0/3703854543 shutdown_connections 2024-03-28T18:07:54.510 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:54.509+0000 7f7072000640 1 --2- 172.21.15.22:0/3703854543 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f706c10bce0 0x7f706c10e150 unknown :-1 s=CLOSED pgs=473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:07:54.510 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:54.509+0000 7f7072000640 1 --2- 172.21.15.22:0/3703854543 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f706c103230 0x7f706c10b710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:07:54.510 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:54.509+0000 7f7072000640 1 --2- 172.21.15.22:0/3703854543 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f706c102910 0x7f706c102cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:07:54.511 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:54.509+0000 7f7072000640 1 -- 172.21.15.22:0/3703854543 >> 172.21.15.22:0/3703854543 conn(0x7f706c077700 msgr2=0x7f706c077b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:07:54.511 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:54.509+0000 7f7072000640 1 -- 172.21.15.22:0/3703854543 shutdown_connections 2024-03-28T18:07:54.511 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:54.509+0000 7f7072000640 1 -- 172.21.15.22:0/3703854543 wait complete. 2024-03-28T18:07:54.513 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:54.510+0000 7f7072000640 1 Processor -- start 2024-03-28T18:07:54.513 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:54.510+0000 7f7072000640 1 -- start start 2024-03-28T18:07:54.513 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:54.511+0000 7f7072000640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f706c102910 0x7f706c100560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:07:54.513 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:54.511+0000 7f7072000640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f706c103230 0x7f706c100aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:07:54.513 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:54.511+0000 7f7072000640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f706c10bce0 0x7f706c100fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:07:54.513 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:54.511+0000 7f7072000640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f706c111a80 con 0x7f706c103230 2024-03-28T18:07:54.513 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:54.511+0000 7f7072000640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f706c111900 con 0x7f706c10bce0 2024-03-28T18:07:54.513 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:54.511+0000 7f7072000640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f706c111c00 con 0x7f706c102910 2024-03-28T18:07:54.514 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:54.511+0000 7f706aa00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f706c103230 0x7f706c100aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:07:54.514 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:54.511+0000 7f706aa00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f706c103230 0x7f706c100aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:50048/0 (socket says 172.21.15.22:50048) 2024-03-28T18:07:54.514 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:54.511+0000 7f706aa00640 1 -- 172.21.15.22:0/3851341459 learned_addr learned my addr 172.21.15.22:0/3851341459 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-28T18:07:54.514 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:54.511+0000 7f706be00640 1 --2- 172.21.15.22:0/3851341459 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f706c10bce0 0x7f706c100fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:07:54.514 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:54.511+0000 7f706b400640 1 --2- 172.21.15.22:0/3851341459 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f706c102910 0x7f706c100560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:07:54.514 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:54.512+0000 7f706aa00640 1 -- 172.21.15.22:0/3851341459 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f706c102910 msgr2=0x7f706c100560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:07:54.514 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:54.512+0000 7f706aa00640 1 --2- 172.21.15.22:0/3851341459 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f706c102910 0x7f706c100560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:07:54.514 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:54.512+0000 7f706aa00640 1 -- 172.21.15.22:0/3851341459 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f706c10bce0 msgr2=0x7f706c100fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:07:54.514 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:54.512+0000 7f706aa00640 1 --2- 172.21.15.22:0/3851341459 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f706c10bce0 0x7f706c100fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:07:54.514 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:54.512+0000 7f706aa00640 1 -- 172.21.15.22:0/3851341459 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f706c1a64c0 con 0x7f706c103230 2024-03-28T18:07:54.514 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:54.512+0000 7f706aa00640 1 --2- 172.21.15.22:0/3851341459 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f706c103230 0x7f706c100aa0 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f705800d770 tx=0x7f705800dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:07:54.514 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:54.512+0000 7f7063e00640 1 -- 172.21.15.22:0/3851341459 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7058017070 con 0x7f706c103230 2024-03-28T18:07:54.514 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:54.512+0000 7f7063e00640 1 -- 172.21.15.22:0/3851341459 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f70580054f0 con 0x7f706c103230 2024-03-28T18:07:54.514 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:54.512+0000 7f7063e00640 1 -- 172.21.15.22:0/3851341459 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7058004d60 con 0x7f706c103230 2024-03-28T18:07:54.515 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:54.512+0000 7f7072000640 1 -- 172.21.15.22:0/3851341459 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f706c1a67b0 con 0x7f706c103230 2024-03-28T18:07:54.515 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:54.512+0000 7f7072000640 1 -- 172.21.15.22:0/3851341459 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f706c1a6d00 con 0x7f706c103230 2024-03-28T18:07:54.518 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:54.516+0000 7f7063e00640 1 -- 172.21.15.22:0/3851341459 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 24) v1 ==== 137466+0+0 (secure 0 0 0) 0x7f7058004f00 con 0x7f706c103230 2024-03-28T18:07:54.518 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:54.516+0000 7f7074dc5740 1 Processor -- start 2024-03-28T18:07:54.519 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:54.516+0000 7f7074dc5740 1 -- start start 2024-03-28T18:07:54.519 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:54.516+0000 7f7074dc5740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55dcf4efa6d0 0x55dcf4efaab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:07:54.519 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:54.516+0000 7f7074dc5740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55dcf4efb080 0x55dcf4f03630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:07:54.519 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:54.516+0000 7f7074dc5740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55dcf4f03c00 0x55dcf4f06050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:07:54.519 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:54.516+0000 7f7074dc5740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dcf4e5e2c0 con 0x55dcf4efa6d0 2024-03-28T18:07:54.519 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:54.516+0000 7f7074dc5740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dcf4e06c90 con 0x55dcf4f03c00 2024-03-28T18:07:54.519 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:54.516+0000 7f7074dc5740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dcf4e5a360 con 0x55dcf4efb080 2024-03-28T18:07:54.519 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:54.517+0000 7f706be00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55dcf4efa6d0 0x55dcf4efaab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:07:54.519 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:54.517+0000 7f706be00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55dcf4efa6d0 0x55dcf4efaab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:50062/0 (socket says 172.21.15.22:50062) 2024-03-28T18:07:54.519 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:54.517+0000 7f706be00640 1 -- 172.21.15.22:0/2145791246 learned_addr learned my addr 172.21.15.22:0/2145791246 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-28T18:07:54.519 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:54.517+0000 7f706aa00640 1 --2- 172.21.15.22:0/2145791246 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55dcf4f03c00 0x55dcf4f06050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:07:54.520 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:54.517+0000 7f706b400640 1 --2- 172.21.15.22:0/2145791246 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55dcf4efb080 0x55dcf4f03630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:07:54.520 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:54.517+0000 7f706be00640 1 -- 172.21.15.22:0/2145791246 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55dcf4efb080 msgr2=0x55dcf4f03630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T18:07:54.520 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:54.517+0000 7f706be00640 1 --2- 172.21.15.22:0/2145791246 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55dcf4efb080 0x55dcf4f03630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:07:54.520 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:54.517+0000 7f706be00640 1 -- 172.21.15.22:0/2145791246 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55dcf4f03c00 msgr2=0x55dcf4f06050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T18:07:54.520 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:54.517+0000 7f706be00640 1 --2- 172.21.15.22:0/2145791246 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55dcf4f03c00 0x55dcf4f06050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:07:54.520 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:54.517+0000 7f706be00640 1 -- 172.21.15.22:0/2145791246 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55dcf4e2b080 con 0x55dcf4efa6d0 2024-03-28T18:07:54.520 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:54.517+0000 7f7063e00640 1 --2- 172.21.15.22:0/3851341459 >> [v2:172.21.15.22:6800/1357365082,v1:172.21.15.22:6801/1357365082] conn(0x7f70500a2a80 0x7f70500a4f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:07:54.520 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:54.517+0000 7f706be00640 1 --2- 172.21.15.22:0/2145791246 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55dcf4efa6d0 0x55dcf4efaab0 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f7064037040 tx=0x7f7064005f80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fc960d2ecc22b1d0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:07:54.520 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:54.518+0000 7f7063e00640 1 -- 172.21.15.22:0/3851341459 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 8411+0+0 (secure 0 0 0) 0x7f70580ce990 con 0x7f706c103230 2024-03-28T18:07:54.520 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:54.518+0000 7f706b400640 1 --2- 172.21.15.22:0/3851341459 >> [v2:172.21.15.22:6800/1357365082,v1:172.21.15.22:6801/1357365082] conn(0x7f70500a2a80 0x7f70500a4f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:07:54.520 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:54.518+0000 7f7057e00640 1 -- 172.21.15.22:0/2145791246 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7064045020 con 0x55dcf4efa6d0 2024-03-28T18:07:54.520 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:54.518+0000 7f7057e00640 1 -- 172.21.15.22:0/2145791246 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f706403a040 con 0x55dcf4efa6d0 2024-03-28T18:07:54.520 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:54.518+0000 7f7057e00640 1 -- 172.21.15.22:0/2145791246 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7064039040 con 0x55dcf4efa6d0 2024-03-28T18:07:54.524 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:54.519+0000 7f7074dc5740 1 -- 172.21.15.22:0/2145791246 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55dcf4efa6d0 msgr2=0x55dcf4efaab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:07:54.524 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:54.519+0000 7f706b400640 1 --2- 172.21.15.22:0/3851341459 >> [v2:172.21.15.22:6800/1357365082,v1:172.21.15.22:6801/1357365082] conn(0x7f70500a2a80 0x7f70500a4f40 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x55dcf4ef5e10 tx=0x7f705c005f90 comp rx=0 tx=0).ready entity=mgr.15522 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:07:54.524 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:54.519+0000 7f7074dc5740 1 --2- 172.21.15.22:0/2145791246 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55dcf4efa6d0 0x55dcf4efaab0 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f7064037040 tx=0x7f7064005f80 comp rx=0 tx=0).stop 2024-03-28T18:07:54.525 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:54.519+0000 7f7074dc5740 1 -- 172.21.15.22:0/2145791246 shutdown_connections 2024-03-28T18:07:54.525 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:54.519+0000 7f7074dc5740 1 --2- 172.21.15.22:0/2145791246 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55dcf4f03c00 0x55dcf4f06050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:07:54.525 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:54.519+0000 7f7074dc5740 1 --2- 172.21.15.22:0/2145791246 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55dcf4efb080 0x55dcf4f03630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:07:54.525 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:54.519+0000 7f7074dc5740 1 --2- 172.21.15.22:0/2145791246 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55dcf4efa6d0 0x55dcf4efaab0 unknown :-1 s=CLOSED pgs=475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:07:54.525 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:54.519+0000 7f7074dc5740 1 -- 172.21.15.22:0/2145791246 >> 172.21.15.22:0/2145791246 conn(0x55dcf4ef4da0 msgr2=0x55dcf4ef5180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:07:54.525 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:54.519+0000 7f7074dc5740 1 -- 172.21.15.22:0/2145791246 shutdown_connections 2024-03-28T18:07:54.525 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:54.519+0000 7f7074dc5740 1 -- 172.21.15.22:0/2145791246 wait complete. 2024-03-28T18:07:54.525 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:54.519+0000 7f7074dc5740 1 -- start start 2024-03-28T18:07:54.525 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:54.520+0000 7f7074dc5740 1 Processor -- start 2024-03-28T18:07:54.525 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:54.520+0000 7f7074dc5740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55dcf4efa6d0 0x55dcf4f03570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:07:54.525 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:54.520+0000 7f7074dc5740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55dcf4efb080 0x55dcf4f00900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:07:54.525 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:54.520+0000 7f7074dc5740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55dcf4f03c00 0x55dcf4f00f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:07:54.525 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:54.520+0000 7f7074dc5740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dcf4f06b10 con 0x55dcf4efa6d0 2024-03-28T18:07:54.525 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:54.520+0000 7f7074dc5740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dcf4e1a0d0 con 0x55dcf4efb080 2024-03-28T18:07:54.526 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:54.520+0000 7f7074dc5740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dcf4f06c90 con 0x55dcf4f03c00 2024-03-28T18:07:54.526 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:54.520+0000 7f7074dc5740 10 client.0 _ll_register_callbacks cb 0x55dcf4e1ee80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-28T18:07:54.526 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:54.520+0000 7f706b400640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55dcf4efa6d0 0x55dcf4f03570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:07:54.526 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:54.520+0000 7f706b400640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55dcf4efa6d0 0x55dcf4f03570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:50076/0 (socket says 172.21.15.22:50076) 2024-03-28T18:07:54.526 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:54.520+0000 7f706b400640 1 -- 172.21.15.22:0/2617003666 learned_addr learned my addr 172.21.15.22:0/2617003666 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-28T18:07:54.526 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:54.520+0000 7f706aa00640 1 --2- 172.21.15.22:0/2617003666 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55dcf4efb080 0x55dcf4f00900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:07:54.526 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:54.521+0000 7f706be00640 1 --2- 172.21.15.22:0/2617003666 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55dcf4f03c00 0x55dcf4f00f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:07:54.526 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:54.521+0000 7f706b400640 1 -- 172.21.15.22:0/2617003666 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55dcf4f03c00 msgr2=0x55dcf4f00f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T18:07:54.526 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:54.521+0000 7f706b400640 1 --2- 172.21.15.22:0/2617003666 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55dcf4f03c00 0x55dcf4f00f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:07:54.526 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:54.521+0000 7f706b400640 1 -- 172.21.15.22:0/2617003666 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55dcf4efb080 msgr2=0x55dcf4f00900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T18:07:54.526 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:54.521+0000 7f706b400640 1 --2- 172.21.15.22:0/2617003666 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55dcf4efb080 0x55dcf4f00900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:07:54.526 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:54.521+0000 7f706b400640 1 -- 172.21.15.22:0/2617003666 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55dcf4b723f0 con 0x55dcf4efa6d0 2024-03-28T18:07:54.526 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:54.521+0000 7f706aa00640 1 --2- 172.21.15.22:0/2617003666 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55dcf4efb080 0x55dcf4f00900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-28T18:07:54.526 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:54.521+0000 7f706b400640 1 --2- 172.21.15.22:0/2617003666 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55dcf4efa6d0 0x55dcf4f03570 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f705c011c40 tx=0x7f705c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d78a4f22147d36c server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:07:54.527 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:54.521+0000 7f7054c00640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-03-28T18:07:54.527 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:54.522+0000 7f7054c00640 1 -- 172.21.15.22:0/2617003666 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f705c010ce0 con 0x55dcf4efa6d0 2024-03-28T18:07:54.527 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:54.522+0000 7f7054c00640 1 -- 172.21.15.22:0/2617003666 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 8411+0+0 (secure 0 0 0) 0x7f705c005020 con 0x55dcf4efa6d0 2024-03-28T18:07:54.527 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:54.522+0000 7f7074dc5740 1 -- 172.21.15.22:0/2617003666 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f706403a040 con 0x55dcf4efa6d0 2024-03-28T18:07:54.527 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:54.522+0000 7f7054c00640 1 -- 172.21.15.22:0/2617003666 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f705c005220 con 0x55dcf4efa6d0 2024-03-28T18:07:54.527 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:54.522+0000 7f7074dc5740 10 client.15648 fetch_fsmap learned FSMap version 27 2024-03-28T18:07:54.527 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:54.522+0000 7f7074dc5740 1 -- 172.21.15.22:0/2617003666 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55dcf4c3c7d0 con 0x55dcf4efa6d0 2024-03-28T18:07:54.527 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:54.522+0000 7f7054c00640 1 -- 172.21.15.22:0/2617003666 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f706403a040 con 0x55dcf4efa6d0 2024-03-28T18:07:54.527 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:54.522+0000 7f7054c00640 1 -- 172.21.15.22:0/2617003666 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 27) v1 ==== 7934+0+0 (secure 0 0 0) 0x7f705c030050 con 0x55dcf4efa6d0 2024-03-28T18:07:54.527 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:54.522+0000 7f7074dc5740 10 client.15648 fetch_fsmap finished waiting for FSMap version 27 2024-03-28T18:07:54.527 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:54.522+0000 7f7074dc5740 10 client.15648 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.i 2024-03-28T18:07:54.527 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:54.523+0000 7f7074dc5740 20 client.15648 populate_metadata read hostname 'smithi022' 2024-03-28T18:07:54.527 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:54.523+0000 7f7074dc5740 1 --2- 172.21.15.22:0/2617003666 >> [v2:172.21.15.134:6838/3400263217,v1:172.21.15.134:6839/3400263217] conn(0x55dcf4f13a60 0x55dcf4f15e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:07:54.527 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:54.523+0000 7f7074dc5740 4 client.15648 mds_command: new command op to 24502 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-28T18:07:54.528 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:54.523+0000 7f7074dc5740 1 -- 172.21.15.22:0/2617003666 --> [v2:172.21.15.134:6838/3400263217,v1:172.21.15.134:6839/3400263217] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55dcf4e6b800 con 0x55dcf4f13a60 2024-03-28T18:07:54.528 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:54.523+0000 7f706aa00640 1 --2- 172.21.15.22:0/2617003666 >> [v2:172.21.15.134:6838/3400263217,v1:172.21.15.134:6839/3400263217] conn(0x55dcf4f13a60 0x55dcf4f15e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:07:54.529 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:54.524+0000 7f706aa00640 1 --2- 172.21.15.22:0/2617003666 >> [v2:172.21.15.134:6838/3400263217,v1:172.21.15.134:6839/3400263217] conn(0x55dcf4f13a60 0x55dcf4f15e70 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=e31d4a82239bcaad server_cookie=1dbfed17cfce311b in_seq=0 out_seq=0 2024-03-28T18:07:54.529 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:54.524+0000 7f7054c00640 10 client.15648 ms_handle_connect on v2:172.21.15.134:6838/3400263217 2024-03-28T18:07:54.529 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:54.526+0000 7f7054c00640 1 -- 172.21.15.22:0/2617003666 <== mds.4 v2:172.21.15.134:6838/3400263217 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x55dcf4e6b800 con 0x55dcf4f13a60 2024-03-28T18:07:54.529 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:54.526+0000 7f7054c00640 10 client.15648 handle_command_reply: tid=0 2024-03-28T18:07:54.529 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:54.526+0000 7f7074dc5740 1 client.15648 shutdown 2024-03-28T18:07:54.529 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:54.526+0000 7f7074dc5740 1 -- 172.21.15.22:0/2617003666 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55dcf4efa6d0 msgr2=0x55dcf4f03570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:07:54.529 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:54.526+0000 7f7074dc5740 1 --2- 172.21.15.22:0/2617003666 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55dcf4efa6d0 0x55dcf4f03570 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f705c011c40 tx=0x7f705c0046c0 comp rx=0 tx=0).stop 2024-03-28T18:07:54.529 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:54.526+0000 7f7074dc5740 1 -- 172.21.15.22:0/2617003666 shutdown_connections 2024-03-28T18:07:54.529 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:54.526+0000 7f7074dc5740 1 --2- 172.21.15.22:0/2617003666 >> [v2:172.21.15.134:6838/3400263217,v1:172.21.15.134:6839/3400263217] conn(0x55dcf4f13a60 0x55dcf4f15e70 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:07:54.529 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:54.526+0000 7f7074dc5740 1 --2- 172.21.15.22:0/2617003666 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55dcf4f03c00 0x55dcf4f00f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:07:54.530 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:54.526+0000 7f7074dc5740 1 --2- 172.21.15.22:0/2617003666 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55dcf4efb080 0x55dcf4f00900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:07:54.530 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:54.526+0000 7f7074dc5740 1 --2- 172.21.15.22:0/2617003666 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55dcf4efa6d0 0x55dcf4f03570 unknown :-1 s=CLOSED pgs=476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:07:54.530 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:54.526+0000 7f7054c00640 0 client.15648 ms_handle_reset on v2:172.21.15.134:6838/3400263217 2024-03-28T18:07:54.530 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:54.526+0000 7f7074dc5740 1 -- 172.21.15.22:0/2617003666 >> 172.21.15.22:0/2617003666 conn(0x55dcf4ef4da0 msgr2=0x55dcf4ef6af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:07:54.530 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:54.526+0000 7f7074dc5740 1 -- 172.21.15.22:0/2617003666 shutdown_connections 2024-03-28T18:07:54.530 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:54.526+0000 7f7074dc5740 1 -- 172.21.15.22:0/2617003666 wait complete. 2024-03-28T18:07:54.530 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:54.526+0000 7f7074dc5740 20 client.15648 trim_cache size 0 max 16384 2024-03-28T18:07:54.541 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:54.540+0000 7f7074dc5740 1 Processor -- start 2024-03-28T18:07:54.541 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:54.540+0000 7f7074dc5740 1 -- start start 2024-03-28T18:07:54.541 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:54.540+0000 7f7074dc5740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55dcf4efa6d0 0x55dcf4f001e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:07:54.541 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:54.540+0000 7f7074dc5740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55dcf4efb080 0x55dcf4f03030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:07:54.541 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:54.540+0000 7f7074dc5740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55dcf4f03c00 0x55dcf4f03570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:07:54.541 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:54.540+0000 7f7074dc5740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f706c1af090 con 0x55dcf4f03c00 2024-03-28T18:07:54.541 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:54.540+0000 7f7074dc5740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f706c1b0a10 con 0x55dcf4efa6d0 2024-03-28T18:07:54.541 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:54.540+0000 7f7074dc5740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f706c1af6f0 con 0x55dcf4efb080 2024-03-28T18:07:54.543 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:54.541+0000 7f706be00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55dcf4f03c00 0x55dcf4f03570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:07:54.543 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:54.541+0000 7f706b400640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55dcf4efa6d0 0x55dcf4f001e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:07:54.543 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:54.541+0000 7f706be00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55dcf4f03c00 0x55dcf4f03570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:50088/0 (socket says 172.21.15.22:50088) 2024-03-28T18:07:54.543 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:54.541+0000 7f706be00640 1 -- 172.21.15.22:0/2552650571 learned_addr learned my addr 172.21.15.22:0/2552650571 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-28T18:07:54.543 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:54.541+0000 7f706be00640 1 -- 172.21.15.22:0/2552650571 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55dcf4efb080 msgr2=0x55dcf4f03030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T18:07:54.543 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:54.541+0000 7f706be00640 1 --2- 172.21.15.22:0/2552650571 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55dcf4efb080 0x55dcf4f03030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:07:54.543 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:54.541+0000 7f706be00640 1 -- 172.21.15.22:0/2552650571 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55dcf4efa6d0 msgr2=0x55dcf4f001e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T18:07:54.544 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:54.541+0000 7f706be00640 1 --2- 172.21.15.22:0/2552650571 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55dcf4efa6d0 0x55dcf4f001e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:07:54.544 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:54.541+0000 7f706be00640 1 -- 172.21.15.22:0/2552650571 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55dcf4ef48d0 con 0x55dcf4f03c00 2024-03-28T18:07:54.544 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:54.541+0000 7f706be00640 1 --2- 172.21.15.22:0/2552650571 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55dcf4f03c00 0x55dcf4f03570 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f7064046000 tx=0x7f7064005df0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=81c73b9ef71ba7fa server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:07:54.544 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:54.542+0000 7f7057400640 1 -- 172.21.15.22:0/2552650571 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f706404e070 con 0x55dcf4f03c00 2024-03-28T18:07:54.544 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:54.542+0000 7f7057400640 1 -- 172.21.15.22:0/2552650571 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f7064032400 con 0x55dcf4f03c00 2024-03-28T18:07:54.544 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:54.542+0000 7f7057400640 1 -- 172.21.15.22:0/2552650571 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7064045050 con 0x55dcf4f03c00 2024-03-28T18:07:54.544 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:54.542+0000 7f7074dc5740 1 -- 172.21.15.22:0/2552650571 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55dcf4f03c00 msgr2=0x55dcf4f03570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:07:54.544 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:54.542+0000 7f7074dc5740 1 --2- 172.21.15.22:0/2552650571 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55dcf4f03c00 0x55dcf4f03570 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f7064046000 tx=0x7f7064005df0 comp rx=0 tx=0).stop 2024-03-28T18:07:54.544 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:54.542+0000 7f7074dc5740 1 -- 172.21.15.22:0/2552650571 shutdown_connections 2024-03-28T18:07:54.544 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:54.542+0000 7f7074dc5740 1 --2- 172.21.15.22:0/2552650571 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55dcf4f03c00 0x55dcf4f03570 unknown :-1 s=CLOSED pgs=477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:07:54.544 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:54.543+0000 7f7074dc5740 1 --2- 172.21.15.22:0/2552650571 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55dcf4efb080 0x55dcf4f03030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:07:54.544 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:54.543+0000 7f7074dc5740 1 --2- 172.21.15.22:0/2552650571 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55dcf4efa6d0 0x55dcf4f001e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:07:54.544 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:54.543+0000 7f7074dc5740 1 -- 172.21.15.22:0/2552650571 >> 172.21.15.22:0/2552650571 conn(0x55dcf4ef4da0 msgr2=0x55dcf4ef3eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:07:54.544 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:54.543+0000 7f7074dc5740 1 -- 172.21.15.22:0/2552650571 shutdown_connections 2024-03-28T18:07:54.545 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:54.543+0000 7f7074dc5740 1 -- 172.21.15.22:0/2552650571 wait complete. 2024-03-28T18:07:54.547 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:54.543+0000 7f7074dc5740 1 -- start start 2024-03-28T18:07:54.547 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:54.543+0000 7f7074dc5740 1 Processor -- start 2024-03-28T18:07:54.547 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:54.544+0000 7f7074dc5740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55dcf4efa6d0 0x55dcf4f0be00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:07:54.547 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:54.544+0000 7f7074dc5740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55dcf4efb080 0x55dcf4f09190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:07:54.547 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:54.544+0000 7f7074dc5740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55dcf4f03c00 0x55dcf4f098d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:07:54.548 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:54.544+0000 7f7074dc5740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f706c1b03b0 con 0x55dcf4f03c00 2024-03-28T18:07:54.548 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:54.544+0000 7f7074dc5740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f706c1afd50 con 0x55dcf4efb080 2024-03-28T18:07:54.548 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:54.544+0000 7f7074dc5740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f706c1ad250 con 0x55dcf4efa6d0 2024-03-28T18:07:54.548 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:54.544+0000 7f7074dc5740 10 client.0 _ll_register_callbacks cb 0x7f705c009490 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-28T18:07:54.548 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:54.544+0000 7f706be00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55dcf4f03c00 0x55dcf4f098d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:07:54.548 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:54.544+0000 7f706be00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55dcf4f03c00 0x55dcf4f098d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:50104/0 (socket says 172.21.15.22:50104) 2024-03-28T18:07:54.548 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:54.544+0000 7f706be00640 1 -- 172.21.15.22:0/3378670456 learned_addr learned my addr 172.21.15.22:0/3378670456 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-28T18:07:54.548 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:54.544+0000 7f706aa00640 1 --2- 172.21.15.22:0/3378670456 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55dcf4efb080 0x55dcf4f09190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:07:54.548 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:54.544+0000 7f706b400640 1 --2- 172.21.15.22:0/3378670456 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55dcf4efa6d0 0x55dcf4f0be00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:07:54.548 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:54.544+0000 7f706be00640 1 -- 172.21.15.22:0/3378670456 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55dcf4efa6d0 msgr2=0x55dcf4f0be00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T18:07:54.548 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:54.544+0000 7f706be00640 1 --2- 172.21.15.22:0/3378670456 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55dcf4efa6d0 0x55dcf4f0be00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:07:54.548 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:54.544+0000 7f706be00640 1 -- 172.21.15.22:0/3378670456 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55dcf4efb080 msgr2=0x55dcf4f09190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T18:07:54.548 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:54.544+0000 7f706be00640 1 --2- 172.21.15.22:0/3378670456 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55dcf4efb080 0x55dcf4f09190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:07:54.548 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:54.544+0000 7f706be00640 1 -- 172.21.15.22:0/3378670456 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55dcf4f6e310 con 0x55dcf4f03c00 2024-03-28T18:07:54.549 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:54.544+0000 7f706aa00640 1 --2- 172.21.15.22:0/3378670456 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55dcf4efb080 0x55dcf4f09190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T18:07:54.549 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:54.545+0000 7f706be00640 1 --2- 172.21.15.22:0/3378670456 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55dcf4f03c00 0x55dcf4f098d0 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f7064009bf0 tx=0x7f706402c180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6a512c46452011f server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:07:54.549 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:54.545+0000 7f7054c00640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-03-28T18:07:54.549 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:54.545+0000 7f7054c00640 1 -- 172.21.15.22:0/3378670456 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f706404e040 con 0x55dcf4f03c00 2024-03-28T18:07:54.549 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:54.545+0000 7f7054c00640 1 -- 172.21.15.22:0/3378670456 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 8411+0+0 (secure 0 0 0) 0x7f7064032be0 con 0x55dcf4f03c00 2024-03-28T18:07:54.549 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:54.545+0000 7f7074dc5740 1 -- 172.21.15.22:0/3378670456 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f7064032400 con 0x55dcf4f03c00 2024-03-28T18:07:54.549 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:54.545+0000 7f7054c00640 1 -- 172.21.15.22:0/3378670456 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7064045020 con 0x55dcf4f03c00 2024-03-28T18:07:54.549 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:54.545+0000 7f7074dc5740 10 client.15654 fetch_fsmap learned FSMap version 27 2024-03-28T18:07:54.549 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:54.545+0000 7f7074dc5740 1 -- 172.21.15.22:0/3378670456 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55dcf4f6f4b0 con 0x55dcf4f03c00 2024-03-28T18:07:54.549 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:54.546+0000 7f7054c00640 1 -- 172.21.15.22:0/3378670456 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f7064032400 con 0x55dcf4f03c00 2024-03-28T18:07:54.549 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:54.546+0000 7f7054c00640 1 -- 172.21.15.22:0/3378670456 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 27) v1 ==== 7934+0+0 (secure 0 0 0) 0x7f70640414c0 con 0x55dcf4f03c00 2024-03-28T18:07:54.549 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:54.546+0000 7f7074dc5740 10 client.15654 fetch_fsmap finished waiting for FSMap version 27 2024-03-28T18:07:54.549 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:54.546+0000 7f7074dc5740 10 client.15654 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.i 2024-03-28T18:07:54.549 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:54.546+0000 7f7074dc5740 20 client.15654 populate_metadata read hostname 'smithi022' 2024-03-28T18:07:54.549 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:54.546+0000 7f7074dc5740 1 --2- 172.21.15.22:0/3378670456 >> [v2:172.21.15.134:6838/3400263217,v1:172.21.15.134:6839/3400263217] conn(0x55dcf4f6f680 0x55dcf4f71a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:07:54.550 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:54.546+0000 7f7074dc5740 4 client.15654 mds_command: new command op to 24502 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-28T18:07:54.550 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:54.546+0000 7f7074dc5740 1 -- 172.21.15.22:0/3378670456 --> [v2:172.21.15.134:6838/3400263217,v1:172.21.15.134:6839/3400263217] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55dcf4e65b50 con 0x55dcf4f6f680 2024-03-28T18:07:54.550 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:54.546+0000 7f706b400640 1 --2- 172.21.15.22:0/3378670456 >> [v2:172.21.15.134:6838/3400263217,v1:172.21.15.134:6839/3400263217] conn(0x55dcf4f6f680 0x55dcf4f71a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:07:54.551 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:54.547+0000 7f706b400640 1 --2- 172.21.15.22:0/3378670456 >> [v2:172.21.15.134:6838/3400263217,v1:172.21.15.134:6839/3400263217] conn(0x55dcf4f6f680 0x55dcf4f71a70 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=69597b343deaed56 server_cookie=817cee015be99fa7 in_seq=0 out_seq=0 2024-03-28T18:07:54.551 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:54.547+0000 7f7054c00640 10 client.15654 ms_handle_connect on v2:172.21.15.134:6838/3400263217 2024-03-28T18:07:54.551 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:54.548+0000 7f7054c00640 1 -- 172.21.15.22:0/3378670456 <== mds.4 v2:172.21.15.134:6838/3400263217 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55dcf4e65b50 con 0x55dcf4f6f680 2024-03-28T18:07:54.551 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:54.548+0000 7f7054c00640 10 client.15654 handle_command_reply: tid=0 2024-03-28T18:07:54.551 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:54.548+0000 7f7074dc5740 1 client.15654 shutdown 2024-03-28T18:07:54.551 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:54.548+0000 7f7074dc5740 1 -- 172.21.15.22:0/3378670456 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55dcf4f03c00 msgr2=0x55dcf4f098d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:07:54.551 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:54.548+0000 7f7074dc5740 1 --2- 172.21.15.22:0/3378670456 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55dcf4f03c00 0x55dcf4f098d0 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f7064009bf0 tx=0x7f706402c180 comp rx=0 tx=0).stop 2024-03-28T18:07:54.552 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:54.548+0000 7f7074dc5740 1 -- 172.21.15.22:0/3378670456 shutdown_connections 2024-03-28T18:07:54.552 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:54.548+0000 7f7074dc5740 1 --2- 172.21.15.22:0/3378670456 >> [v2:172.21.15.134:6838/3400263217,v1:172.21.15.134:6839/3400263217] conn(0x55dcf4f6f680 0x55dcf4f71a70 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:07:54.552 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:54.548+0000 7f7074dc5740 1 --2- 172.21.15.22:0/3378670456 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55dcf4f03c00 0x55dcf4f098d0 unknown :-1 s=CLOSED pgs=478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:07:54.552 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:54.548+0000 7f7074dc5740 1 --2- 172.21.15.22:0/3378670456 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55dcf4efb080 0x55dcf4f09190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:07:54.552 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:54.548+0000 7f7074dc5740 1 --2- 172.21.15.22:0/3378670456 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55dcf4efa6d0 0x55dcf4f0be00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:07:54.552 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:54.548+0000 7f7074dc5740 1 -- 172.21.15.22:0/3378670456 >> 172.21.15.22:0/3378670456 conn(0x55dcf4ef4da0 msgr2=0x55dcf4efb460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:07:54.552 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:54.548+0000 7f7054c00640 0 client.15654 ms_handle_reset on v2:172.21.15.134:6838/3400263217 2024-03-28T18:07:54.552 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:54.548+0000 7f7074dc5740 1 -- 172.21.15.22:0/3378670456 shutdown_connections 2024-03-28T18:07:54.552 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:54.548+0000 7f7074dc5740 1 -- 172.21.15.22:0/3378670456 wait complete. 2024-03-28T18:07:54.552 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:54.548+0000 7f7074dc5740 20 client.15654 trim_cache size 0 max 16384 2024-03-28T18:07:54.552 INFO:teuthology.orchestra.run.smithi022.stdout:[] 2024-03-28T18:07:54.553 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:54.549+0000 7f7072000640 1 -- 172.21.15.22:0/3851341459 >> [v2:172.21.15.22:6800/1357365082,v1:172.21.15.22:6801/1357365082] conn(0x7f70500a2a80 msgr2=0x7f70500a4f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:07:54.553 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:54.549+0000 7f7072000640 1 --2- 172.21.15.22:0/3851341459 >> [v2:172.21.15.22:6800/1357365082,v1:172.21.15.22:6801/1357365082] conn(0x7f70500a2a80 0x7f70500a4f40 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x55dcf4ef5e10 tx=0x7f705c005f90 comp rx=0 tx=0).stop 2024-03-28T18:07:54.553 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:54.549+0000 7f7072000640 1 -- 172.21.15.22:0/3851341459 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f706c103230 msgr2=0x7f706c100aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:07:54.553 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:54.549+0000 7f7072000640 1 --2- 172.21.15.22:0/3851341459 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f706c103230 0x7f706c100aa0 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f705800d770 tx=0x7f705800dc30 comp rx=0 tx=0).stop 2024-03-28T18:07:54.554 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:54.549+0000 7f7072000640 1 -- 172.21.15.22:0/3851341459 shutdown_connections 2024-03-28T18:07:54.554 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:54.549+0000 7f7072000640 1 --2- 172.21.15.22:0/3851341459 >> [v2:172.21.15.22:6800/1357365082,v1:172.21.15.22:6801/1357365082] conn(0x7f70500a2a80 0x7f70500a4f40 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:07:54.554 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:54.549+0000 7f7072000640 1 --2- 172.21.15.22:0/3851341459 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f706c10bce0 0x7f706c100fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:07:54.554 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:54.549+0000 7f7072000640 1 --2- 172.21.15.22:0/3851341459 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f706c103230 0x7f706c100aa0 unknown :-1 s=CLOSED pgs=474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:07:54.554 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:54.549+0000 7f7072000640 1 --2- 172.21.15.22:0/3851341459 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f706c102910 0x7f706c100560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:07:54.554 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:54.549+0000 7f7072000640 1 -- 172.21.15.22:0/3851341459 >> 172.21.15.22:0/3851341459 conn(0x7f706c077700 msgr2=0x7f706c0fe8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:07:54.554 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:54.549+0000 7f7072000640 1 -- 172.21.15.22:0/3851341459 shutdown_connections 2024-03-28T18:07:54.554 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:54.549+0000 7f7072000640 1 -- 172.21.15.22:0/3851341459 wait complete. 2024-03-28T18:07:54.565 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-28T18:07:54.565 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2024-03-28T18:07:54.758 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:54.757+0000 7f5ffec00640 1 -- 172.21.15.22:0/652882907 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5ff8112510 msgr2=0x7f5ff8114900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:07:54.758 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:54.757+0000 7f5ffec00640 1 --2- 172.21.15.22:0/652882907 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5ff8112510 0x7f5ff8114900 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f5ff0009ca0 tx=0x7f5ff002c180 comp rx=0 tx=0).stop 2024-03-28T18:07:54.759 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:54.757+0000 7f5ffec00640 1 -- 172.21.15.22:0/652882907 shutdown_connections 2024-03-28T18:07:54.759 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:54.757+0000 7f5ffec00640 1 --2- 172.21.15.22:0/652882907 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5ff8112510 0x7f5ff8114900 unknown :-1 s=CLOSED pgs=479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:07:54.759 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:54.757+0000 7f5ffec00640 1 --2- 172.21.15.22:0/652882907 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f5ff8074360 0x7f5ff8111fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:07:54.759 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:54.757+0000 7f5ffec00640 1 --2- 172.21.15.22:0/652882907 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5ff8073a40 0x7f5ff8073e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:07:54.759 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:54.757+0000 7f5ffec00640 1 -- 172.21.15.22:0/652882907 >> 172.21.15.22:0/652882907 conn(0x7f5ff80fd010 msgr2=0x7f5ff80ff430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:07:54.759 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:54.757+0000 7f5ffec00640 1 -- 172.21.15.22:0/652882907 shutdown_connections 2024-03-28T18:07:54.759 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:54.757+0000 7f5ffec00640 1 -- 172.21.15.22:0/652882907 wait complete. 2024-03-28T18:07:54.761 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:54.758+0000 7f5ffec00640 1 Processor -- start 2024-03-28T18:07:54.761 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:54.758+0000 7f5ffec00640 1 -- start start 2024-03-28T18:07:54.761 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:54.759+0000 7f5ffec00640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f5ff8073a40 0x7f5ff819e3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:07:54.761 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:54.759+0000 7f5ffec00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5ff8074360 0x7f5ff819e920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:07:54.761 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:54.759+0000 7f5ffec00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5ff8112510 0x7f5ff81a2ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:07:54.761 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:54.759+0000 7f5ffec00640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ff8117ee0 con 0x7f5ff8112510 2024-03-28T18:07:54.762 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:54.759+0000 7f5ffec00640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ff8117d60 con 0x7f5ff8073a40 2024-03-28T18:07:54.762 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:54.759+0000 7f5ffec00640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ff8118060 con 0x7f5ff8074360 2024-03-28T18:07:54.762 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:54.759+0000 7f5ffe200640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5ff8112510 0x7f5ff81a2ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:07:54.762 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:54.759+0000 7f5ffe200640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5ff8112510 0x7f5ff81a2ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:50112/0 (socket says 172.21.15.22:50112) 2024-03-28T18:07:54.762 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:54.759+0000 7f5ffe200640 1 -- 172.21.15.22:0/4220417590 learned_addr learned my addr 172.21.15.22:0/4220417590 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-28T18:07:54.762 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:54.759+0000 7f5ffd800640 1 --2- 172.21.15.22:0/4220417590 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f5ff8073a40 0x7f5ff819e3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:07:54.762 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:54.759+0000 7f5ffce00640 1 --2- 172.21.15.22:0/4220417590 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5ff8074360 0x7f5ff819e920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:07:54.762 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:54.759+0000 7f5ffe200640 1 -- 172.21.15.22:0/4220417590 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5ff8074360 msgr2=0x7f5ff819e920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:07:54.762 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:54.759+0000 7f5ffe200640 1 --2- 172.21.15.22:0/4220417590 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5ff8074360 0x7f5ff819e920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:07:54.762 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:54.759+0000 7f5ffe200640 1 -- 172.21.15.22:0/4220417590 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f5ff8073a40 msgr2=0x7f5ff819e3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:07:54.762 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:54.759+0000 7f5ffe200640 1 --2- 172.21.15.22:0/4220417590 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f5ff8073a40 0x7f5ff819e3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:07:54.762 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:54.759+0000 7f5ffe200640 1 -- 172.21.15.22:0/4220417590 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ff81a33d0 con 0x7f5ff8112510 2024-03-28T18:07:54.762 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:54.759+0000 7f5ffce00640 1 --2- 172.21.15.22:0/4220417590 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5ff8074360 0x7f5ff819e920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T18:07:54.762 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:54.760+0000 7f5ffe200640 1 --2- 172.21.15.22:0/4220417590 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5ff8112510 0x7f5ff81a2ca0 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f5ff0005e30 tx=0x7f5ff0034e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:07:54.763 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:54.760+0000 7f5ff6000640 1 -- 172.21.15.22:0/4220417590 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ff0039050 con 0x7f5ff8112510 2024-03-28T18:07:54.763 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:54.760+0000 7f5ff6000640 1 -- 172.21.15.22:0/4220417590 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f5ff003a040 con 0x7f5ff8112510 2024-03-28T18:07:54.763 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:54.760+0000 7f5ff6000640 1 -- 172.21.15.22:0/4220417590 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ff0032ad0 con 0x7f5ff8112510 2024-03-28T18:07:54.763 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:54.760+0000 7f5ffec00640 1 -- 172.21.15.22:0/4220417590 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5ff81a3660 con 0x7f5ff8112510 2024-03-28T18:07:54.763 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:54.760+0000 7f5ffec00640 1 -- 172.21.15.22:0/4220417590 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5ff81aaf50 con 0x7f5ff8112510 2024-03-28T18:07:54.765 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:54.764+0000 7f6002ff4740 1 Processor -- start 2024-03-28T18:07:54.765 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:54.765+0000 7f5ff6000640 1 -- 172.21.15.22:0/4220417590 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 24) v1 ==== 137466+0+0 (secure 0 0 0) 0x7f5ff0004610 con 0x7f5ff8112510 2024-03-28T18:07:54.765 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:54.765+0000 7f6002ff4740 1 -- start start 2024-03-28T18:07:54.766 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:54.765+0000 7f6002ff4740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55f068c8d8b0 0x55f068c8dc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:07:54.767 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:54.765+0000 7f6002ff4740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f068c8e260 0x55f068c96810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:07:54.767 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:54.765+0000 7f6002ff4740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f068c96de0 0x55f068c99230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:07:54.767 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:54.765+0000 7f6002ff4740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f068bf12c0 con 0x55f068c96de0 2024-03-28T18:07:54.767 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:54.765+0000 7f6002ff4740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f068b99c90 con 0x55f068c8d8b0 2024-03-28T18:07:54.767 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:54.765+0000 7f6002ff4740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f068bed360 con 0x55f068c8e260 2024-03-28T18:07:54.767 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:54.765+0000 7f5ffce00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f068c96de0 0x55f068c99230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:07:54.767 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:54.766+0000 7f5ffce00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f068c96de0 0x55f068c99230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:50122/0 (socket says 172.21.15.22:50122) 2024-03-28T18:07:54.767 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:54.766+0000 7f5ffce00640 1 -- 172.21.15.22:0/767696092 learned_addr learned my addr 172.21.15.22:0/767696092 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-28T18:07:54.767 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:54.766+0000 7f5ffce00640 1 --2- 172.21.15.22:0/767696092 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55f068c8d8b0 0x55f068c8dc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:07:54.767 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:54.766+0000 7f5ffd800640 1 --2- 172.21.15.22:0/767696092 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f068c8e260 0x55f068c96810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:07:54.767 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:54.766+0000 7f5ffce00640 1 -- 172.21.15.22:0/767696092 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f068c8e260 msgr2=0x55f068c96810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T18:07:54.767 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:54.766+0000 7f5ffce00640 1 --2- 172.21.15.22:0/767696092 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f068c8e260 0x55f068c96810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:07:54.767 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:54.766+0000 7f5ffce00640 1 -- 172.21.15.22:0/767696092 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55f068c8d8b0 msgr2=0x55f068c8dc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T18:07:54.767 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:54.766+0000 7f5ffce00640 1 --2- 172.21.15.22:0/767696092 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55f068c8d8b0 0x55f068c8dc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:07:54.768 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:54.766+0000 7f5ffce00640 1 -- 172.21.15.22:0/767696092 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f068bbe080 con 0x55f068c96de0 2024-03-28T18:07:54.772 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:54.766+0000 7f5ffce00640 1 --2- 172.21.15.22:0/767696092 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f068c96de0 0x55f068c99230 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f5fe40148b0 tx=0x7f5fe40095f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bfbd8863e9976ac6 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:07:54.772 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:54.766+0000 7f5ff6000640 1 --2- 172.21.15.22:0/4220417590 >> [v2:172.21.15.22:6800/1357365082,v1:172.21.15.22:6801/1357365082] conn(0x7f5fe00a2a80 0x7f5fe00a4f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:07:54.772 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:54.767+0000 7f5ff6000640 1 -- 172.21.15.22:0/4220417590 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 8411+0+0 (secure 0 0 0) 0x7f5ff00f1330 con 0x7f5ff8112510 2024-03-28T18:07:54.772 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:54.767+0000 7f5feaa00640 1 -- 172.21.15.22:0/767696092 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5fe4014e30 con 0x55f068c96de0 2024-03-28T18:07:54.772 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:54.767+0000 7f5feaa00640 1 -- 172.21.15.22:0/767696092 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f5fe400f760 con 0x55f068c96de0 2024-03-28T18:07:54.772 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:54.767+0000 7f5feaa00640 1 -- 172.21.15.22:0/767696092 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5fe4003850 con 0x55f068c96de0 2024-03-28T18:07:54.772 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:54.767+0000 7f5ffd800640 1 --2- 172.21.15.22:0/4220417590 >> [v2:172.21.15.22:6800/1357365082,v1:172.21.15.22:6801/1357365082] conn(0x7f5fe00a2a80 0x7f5fe00a4f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:07:54.772 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:54.768+0000 7f6002ff4740 1 -- 172.21.15.22:0/767696092 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f068c96de0 msgr2=0x55f068c99230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:07:54.772 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:54.768+0000 7f5ffd800640 1 --2- 172.21.15.22:0/4220417590 >> [v2:172.21.15.22:6800/1357365082,v1:172.21.15.22:6801/1357365082] conn(0x7f5fe00a2a80 0x7f5fe00a4f40 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x55f068c88ff0 tx=0x7f5fec00b390 comp rx=0 tx=0).ready entity=mgr.15522 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:07:54.772 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:54.768+0000 7f6002ff4740 1 --2- 172.21.15.22:0/767696092 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f068c96de0 0x55f068c99230 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f5fe40148b0 tx=0x7f5fe40095f0 comp rx=0 tx=0).stop 2024-03-28T18:07:54.772 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:54.768+0000 7f6002ff4740 1 -- 172.21.15.22:0/767696092 shutdown_connections 2024-03-28T18:07:54.772 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:54.768+0000 7f6002ff4740 1 --2- 172.21.15.22:0/767696092 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f068c96de0 0x55f068c99230 unknown :-1 s=CLOSED pgs=481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:07:54.773 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:54.768+0000 7f6002ff4740 1 --2- 172.21.15.22:0/767696092 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f068c8e260 0x55f068c96810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:07:54.773 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:54.768+0000 7f6002ff4740 1 --2- 172.21.15.22:0/767696092 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55f068c8d8b0 0x55f068c8dc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:07:54.773 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:54.768+0000 7f6002ff4740 1 -- 172.21.15.22:0/767696092 >> 172.21.15.22:0/767696092 conn(0x55f068c87f80 msgr2=0x55f068c88360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:07:54.773 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:54.768+0000 7f6002ff4740 1 -- 172.21.15.22:0/767696092 shutdown_connections 2024-03-28T18:07:54.773 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:54.768+0000 7f6002ff4740 1 -- 172.21.15.22:0/767696092 wait complete. 2024-03-28T18:07:54.773 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:54.768+0000 7f6002ff4740 1 -- start start 2024-03-28T18:07:54.773 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:54.768+0000 7f6002ff4740 1 Processor -- start 2024-03-28T18:07:54.773 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:54.769+0000 7f6002ff4740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55f068c8d8b0 0x55f068c96700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:07:54.773 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:54.769+0000 7f6002ff4740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f068c8e260 0x55f068c93a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:07:54.773 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:54.769+0000 7f6002ff4740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f068c96de0 0x55f068c94030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:07:54.773 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:54.769+0000 7f6002ff4740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f068c99d50 con 0x55f068c96de0 2024-03-28T18:07:54.773 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:54.769+0000 7f6002ff4740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f068bad0d0 con 0x55f068c8d8b0 2024-03-28T18:07:54.773 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:54.769+0000 7f6002ff4740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f068c99ed0 con 0x55f068c8e260 2024-03-28T18:07:54.773 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:54.769+0000 7f6002ff4740 10 client.0 _ll_register_callbacks cb 0x55f068bb1e80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-28T18:07:54.773 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:54.769+0000 7f5ffe200640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f068c96de0 0x55f068c94030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:07:54.774 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:54.769+0000 7f5ffe200640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f068c96de0 0x55f068c94030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:50130/0 (socket says 172.21.15.22:50130) 2024-03-28T18:07:54.774 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:54.769+0000 7f5ffe200640 1 -- 172.21.15.22:0/1881983774 learned_addr learned my addr 172.21.15.22:0/1881983774 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-28T18:07:54.774 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:54.769+0000 7f5ffd800640 1 --2- 172.21.15.22:0/1881983774 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55f068c8d8b0 0x55f068c96700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:07:54.774 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:54.770+0000 7f5ffce00640 1 --2- 172.21.15.22:0/1881983774 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f068c8e260 0x55f068c93a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:07:54.774 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:54.770+0000 7f5ffe200640 1 -- 172.21.15.22:0/1881983774 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f068c8e260 msgr2=0x55f068c93a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T18:07:54.774 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:54.770+0000 7f5ffe200640 1 --2- 172.21.15.22:0/1881983774 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f068c8e260 0x55f068c93a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:07:54.774 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:54.770+0000 7f5ffe200640 1 -- 172.21.15.22:0/1881983774 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55f068c8d8b0 msgr2=0x55f068c96700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T18:07:54.774 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:54.770+0000 7f5ffe200640 1 --2- 172.21.15.22:0/1881983774 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55f068c8d8b0 0x55f068c96700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:07:54.774 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:54.770+0000 7f5ffe200640 1 -- 172.21.15.22:0/1881983774 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55f0689053f0 con 0x55f068c96de0 2024-03-28T18:07:54.774 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:54.770+0000 7f5ffce00640 1 --2- 172.21.15.22:0/1881983774 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f068c8e260 0x55f068c93a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T18:07:54.774 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:54.770+0000 7f5ffe200640 1 --2- 172.21.15.22:0/1881983774 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f068c96de0 0x55f068c94030 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f5ff00bd6a0 tx=0x7f5ff00be0a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=570b81830c9ad8c0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:07:54.774 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:54.770+0000 7f5fdf400640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-03-28T18:07:54.774 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:54.770+0000 7f5fdf400640 1 -- 172.21.15.22:0/1881983774 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ff00c6140 con 0x55f068c96de0 2024-03-28T18:07:54.774 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:54.770+0000 7f5fdf400640 1 -- 172.21.15.22:0/1881983774 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 8411+0+0 (secure 0 0 0) 0x7f5ff0032c70 con 0x55f068c96de0 2024-03-28T18:07:54.775 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:54.770+0000 7f6002ff4740 1 -- 172.21.15.22:0/1881983774 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f5fe400f760 con 0x55f068c96de0 2024-03-28T18:07:54.775 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:54.770+0000 7f5fdf400640 1 -- 172.21.15.22:0/1881983774 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5ff00eece0 con 0x55f068c96de0 2024-03-28T18:07:54.775 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:54.770+0000 7f6002ff4740 10 client.15666 fetch_fsmap learned FSMap version 27 2024-03-28T18:07:54.775 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:54.770+0000 7f6002ff4740 1 -- 172.21.15.22:0/1881983774 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55f0689cf7d0 con 0x55f068c96de0 2024-03-28T18:07:54.775 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:54.771+0000 7f5fdf400640 1 -- 172.21.15.22:0/1881983774 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f5fe400f760 con 0x55f068c96de0 2024-03-28T18:07:54.775 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:54.771+0000 7f5fdf400640 1 -- 172.21.15.22:0/1881983774 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 27) v1 ==== 7934+0+0 (secure 0 0 0) 0x7f5ff00c5cb0 con 0x55f068c96de0 2024-03-28T18:07:54.775 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:54.771+0000 7f6002ff4740 10 client.15666 fetch_fsmap finished waiting for FSMap version 27 2024-03-28T18:07:54.775 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:54.771+0000 7f6002ff4740 10 client.15666 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.k 2024-03-28T18:07:54.775 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:54.771+0000 7f6002ff4740 20 client.15666 populate_metadata read hostname 'smithi022' 2024-03-28T18:07:54.775 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:54.771+0000 7f6002ff4740 1 --2- 172.21.15.22:0/1881983774 >> [v2:172.21.15.79:6832/856896997,v1:172.21.15.79:6833/856896997] conn(0x55f068ca6bc0 0x55f068ca8fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:07:54.775 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:54.771+0000 7f6002ff4740 4 client.15666 mds_command: new command op to 34442 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-28T18:07:54.775 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:54.771+0000 7f6002ff4740 1 -- 172.21.15.22:0/1881983774 --> [v2:172.21.15.79:6832/856896997,v1:172.21.15.79:6833/856896997] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55f068bfe800 con 0x55f068ca6bc0 2024-03-28T18:07:54.776 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:54.771+0000 7f5ffd800640 1 --2- 172.21.15.22:0/1881983774 >> [v2:172.21.15.79:6832/856896997,v1:172.21.15.79:6833/856896997] conn(0x55f068ca6bc0 0x55f068ca8fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:07:54.777 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:54.772+0000 7f5ffd800640 1 --2- 172.21.15.22:0/1881983774 >> [v2:172.21.15.79:6832/856896997,v1:172.21.15.79:6833/856896997] conn(0x55f068ca6bc0 0x55f068ca8fe0 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=67f2194eeef070b4 server_cookie=5c5d576bf6b8ee8f in_seq=0 out_seq=0 2024-03-28T18:07:54.777 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:54.772+0000 7f5fdf400640 10 client.15666 ms_handle_connect on v2:172.21.15.79:6832/856896997 2024-03-28T18:07:54.777 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:54.774+0000 7f5fdf400640 1 -- 172.21.15.22:0/1881983774 <== mds.3 v2:172.21.15.79:6832/856896997 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x55f068bfe800 con 0x55f068ca6bc0 2024-03-28T18:07:54.777 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:54.774+0000 7f5fdf400640 10 client.15666 handle_command_reply: tid=0 2024-03-28T18:07:54.777 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:54.774+0000 7f6002ff4740 1 client.15666 shutdown 2024-03-28T18:07:54.777 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:54.774+0000 7f6002ff4740 1 -- 172.21.15.22:0/1881983774 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f068c96de0 msgr2=0x55f068c94030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:07:54.777 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:54.774+0000 7f6002ff4740 1 --2- 172.21.15.22:0/1881983774 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f068c96de0 0x55f068c94030 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f5ff00bd6a0 tx=0x7f5ff00be0a0 comp rx=0 tx=0).stop 2024-03-28T18:07:54.777 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:54.774+0000 7f6002ff4740 1 -- 172.21.15.22:0/1881983774 shutdown_connections 2024-03-28T18:07:54.777 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:54.774+0000 7f6002ff4740 1 --2- 172.21.15.22:0/1881983774 >> [v2:172.21.15.79:6832/856896997,v1:172.21.15.79:6833/856896997] conn(0x55f068ca6bc0 0x55f068ca8fe0 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:07:54.777 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:54.774+0000 7f6002ff4740 1 --2- 172.21.15.22:0/1881983774 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f068c96de0 0x55f068c94030 unknown :-1 s=CLOSED pgs=482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:07:54.777 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:54.774+0000 7f6002ff4740 1 --2- 172.21.15.22:0/1881983774 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f068c8e260 0x55f068c93a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:07:54.777 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:54.774+0000 7f6002ff4740 1 --2- 172.21.15.22:0/1881983774 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55f068c8d8b0 0x55f068c96700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:07:54.777 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:54.774+0000 7f6002ff4740 1 -- 172.21.15.22:0/1881983774 >> 172.21.15.22:0/1881983774 conn(0x55f068c87f80 msgr2=0x55f068c89cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:07:54.777 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:54.774+0000 7f5fdf400640 0 client.15666 ms_handle_reset on v2:172.21.15.79:6832/856896997 2024-03-28T18:07:54.777 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:54.774+0000 7f6002ff4740 1 -- 172.21.15.22:0/1881983774 shutdown_connections 2024-03-28T18:07:54.778 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:54.774+0000 7f6002ff4740 1 -- 172.21.15.22:0/1881983774 wait complete. 2024-03-28T18:07:54.778 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:54.775+0000 7f6002ff4740 20 client.15666 trim_cache size 0 max 16384 2024-03-28T18:07:54.790 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:54.788+0000 7f6002ff4740 1 Processor -- start 2024-03-28T18:07:54.790 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:54.788+0000 7f6002ff4740 1 -- start start 2024-03-28T18:07:54.790 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:54.788+0000 7f6002ff4740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f068c8d8b0 0x55f068c961c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:07:54.790 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:54.788+0000 7f6002ff4740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55f068c8e260 0x55f068c96700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:07:54.790 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:54.788+0000 7f6002ff4740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f068c96de0 0x55f068c90cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:07:54.790 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:54.788+0000 7f6002ff4740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ff81b3440 con 0x55f068c96de0 2024-03-28T18:07:54.790 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:54.788+0000 7f6002ff4740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ff81b4dc0 con 0x55f068c8e260 2024-03-28T18:07:54.790 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:54.788+0000 7f6002ff4740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ff81b3aa0 con 0x55f068c8d8b0 2024-03-28T18:07:54.790 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:54.789+0000 7f5ffe200640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f068c96de0 0x55f068c90cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:07:54.790 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:54.789+0000 7f5ffe200640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f068c96de0 0x55f068c90cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:50138/0 (socket says 172.21.15.22:50138) 2024-03-28T18:07:54.791 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:54.789+0000 7f5ffe200640 1 -- 172.21.15.22:0/1352608348 learned_addr learned my addr 172.21.15.22:0/1352608348 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-28T18:07:54.791 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:54.789+0000 7f5ffce00640 1 --2- 172.21.15.22:0/1352608348 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55f068c8e260 0x55f068c96700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:07:54.791 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:54.789+0000 7f5ffe200640 1 -- 172.21.15.22:0/1352608348 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f068c8d8b0 msgr2=0x55f068c961c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T18:07:54.791 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:54.789+0000 7f5ffe200640 1 --2- 172.21.15.22:0/1352608348 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f068c8d8b0 0x55f068c961c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:07:54.791 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:54.789+0000 7f5ffe200640 1 -- 172.21.15.22:0/1352608348 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55f068c8e260 msgr2=0x55f068c96700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T18:07:54.791 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:54.789+0000 7f5ffe200640 1 --2- 172.21.15.22:0/1352608348 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55f068c8e260 0x55f068c96700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:07:54.791 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:54.789+0000 7f5ffe200640 1 -- 172.21.15.22:0/1352608348 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f068c92360 con 0x55f068c96de0 2024-03-28T18:07:54.791 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:54.789+0000 7f5ffe200640 1 --2- 172.21.15.22:0/1352608348 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f068c96de0 0x55f068c90cf0 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f5ff0004af0 tx=0x7f5ff00dc2a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6b1381af6f02014f server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:07:54.794 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:54.790+0000 7f5fea000640 1 -- 172.21.15.22:0/1352608348 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ff00bd4e0 con 0x55f068c96de0 2024-03-28T18:07:54.794 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:54.790+0000 7f5fea000640 1 -- 172.21.15.22:0/1352608348 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f5ff00c6140 con 0x55f068c96de0 2024-03-28T18:07:54.794 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:54.790+0000 7f5fea000640 1 -- 172.21.15.22:0/1352608348 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ff00bd770 con 0x55f068c96de0 2024-03-28T18:07:54.794 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:54.790+0000 7f6002ff4740 1 -- 172.21.15.22:0/1352608348 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f068c96de0 msgr2=0x55f068c90cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:07:54.794 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:54.790+0000 7f6002ff4740 1 --2- 172.21.15.22:0/1352608348 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f068c96de0 0x55f068c90cf0 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f5ff0004af0 tx=0x7f5ff00dc2a0 comp rx=0 tx=0).stop 2024-03-28T18:07:54.795 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:54.790+0000 7f6002ff4740 1 -- 172.21.15.22:0/1352608348 shutdown_connections 2024-03-28T18:07:54.795 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:54.790+0000 7f6002ff4740 1 --2- 172.21.15.22:0/1352608348 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f068c96de0 0x55f068c90cf0 unknown :-1 s=CLOSED pgs=483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:07:54.795 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:54.790+0000 7f6002ff4740 1 --2- 172.21.15.22:0/1352608348 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55f068c8e260 0x55f068c96700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:07:54.795 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:54.790+0000 7f6002ff4740 1 --2- 172.21.15.22:0/1352608348 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f068c8d8b0 0x55f068c961c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:07:54.795 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:54.790+0000 7f6002ff4740 1 -- 172.21.15.22:0/1352608348 >> 172.21.15.22:0/1352608348 conn(0x55f068c87f80 msgr2=0x55f068c87160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:07:54.795 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:54.791+0000 7f6002ff4740 1 -- 172.21.15.22:0/1352608348 shutdown_connections 2024-03-28T18:07:54.795 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:54.791+0000 7f6002ff4740 1 -- 172.21.15.22:0/1352608348 wait complete. 2024-03-28T18:07:54.795 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:54.791+0000 7f6002ff4740 1 -- start start 2024-03-28T18:07:54.795 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:54.791+0000 7f6002ff4740 1 Processor -- start 2024-03-28T18:07:54.795 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:54.791+0000 7f6002ff4740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f068c8d8b0 0x55f068c9ce40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:07:54.795 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:54.792+0000 7f6002ff4740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55f068c8e260 0x55f068c9d380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:07:54.795 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:54.792+0000 7f6002ff4740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f068c96de0 0x55f068cfb500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:07:54.795 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:54.792+0000 7f6002ff4740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ff81b4760 con 0x55f068c8d8b0 2024-03-28T18:07:54.796 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:54.792+0000 7f6002ff4740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ff81b4100 con 0x55f068c8e260 2024-03-28T18:07:54.796 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:54.792+0000 7f6002ff4740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ff81b1600 con 0x55f068c96de0 2024-03-28T18:07:54.796 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:54.792+0000 7f6002ff4740 10 client.0 _ll_register_callbacks cb 0x7f5ff00ee9d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-28T18:07:54.796 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:54.792+0000 7f5ffd800640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f068c8d8b0 0x55f068c9ce40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:07:54.796 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:54.792+0000 7f5ffd800640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f068c8d8b0 0x55f068c9ce40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:50140/0 (socket says 172.21.15.22:50140) 2024-03-28T18:07:54.796 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:54.792+0000 7f5ffd800640 1 -- 172.21.15.22:0/654970435 learned_addr learned my addr 172.21.15.22:0/654970435 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-28T18:07:54.796 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:54.792+0000 7f5ffce00640 1 --2- 172.21.15.22:0/654970435 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55f068c8e260 0x55f068c9d380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:07:54.796 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:54.792+0000 7f5ffd800640 1 -- 172.21.15.22:0/654970435 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f068c96de0 msgr2=0x55f068cfb500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T18:07:54.796 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:54.792+0000 7f5ffd800640 1 --2- 172.21.15.22:0/654970435 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f068c96de0 0x55f068cfb500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:07:54.796 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:54.792+0000 7f5ffd800640 1 -- 172.21.15.22:0/654970435 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55f068c8e260 msgr2=0x55f068c9d380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T18:07:54.796 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:54.792+0000 7f5ffd800640 1 --2- 172.21.15.22:0/654970435 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55f068c8e260 0x55f068c9d380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:07:54.796 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:54.792+0000 7f5ffd800640 1 -- 172.21.15.22:0/654970435 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55f068cfbaf0 con 0x55f068c8d8b0 2024-03-28T18:07:54.796 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:54.792+0000 7f5ffd800640 1 --2- 172.21.15.22:0/654970435 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f068c8d8b0 0x55f068c9ce40 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x55f068c8d380 tx=0x7f5fec0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=775eb536781d07f1 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:07:54.796 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:54.792+0000 7f5fdf400640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-03-28T18:07:54.796 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:54.793+0000 7f5fdf400640 1 -- 172.21.15.22:0/654970435 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5fec02d060 con 0x55f068c8d8b0 2024-03-28T18:07:54.797 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:54.793+0000 7f5fdf400640 1 -- 172.21.15.22:0/654970435 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 8411+0+0 (secure 0 0 0) 0x7f5fec028880 con 0x55f068c8d8b0 2024-03-28T18:07:54.797 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:54.793+0000 7f6002ff4740 1 -- 172.21.15.22:0/654970435 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f5ff00c6140 con 0x55f068c8d8b0 2024-03-28T18:07:54.797 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:54.793+0000 7f5fdf400640 1 -- 172.21.15.22:0/654970435 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5fec028a80 con 0x55f068c8d8b0 2024-03-28T18:07:54.797 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:54.793+0000 7f6002ff4740 10 client.15672 fetch_fsmap learned FSMap version 27 2024-03-28T18:07:54.797 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:54.793+0000 7f6002ff4740 1 -- 172.21.15.22:0/654970435 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55f068d02ef0 con 0x55f068c8d8b0 2024-03-28T18:07:54.797 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:54.793+0000 7f5fdf400640 1 -- 172.21.15.22:0/654970435 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f5ff00c6140 con 0x55f068c8d8b0 2024-03-28T18:07:54.797 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:54.793+0000 7f5fdf400640 1 -- 172.21.15.22:0/654970435 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 27) v1 ==== 7934+0+0 (secure 0 0 0) 0x7f5fec030030 con 0x55f068c8d8b0 2024-03-28T18:07:54.797 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:54.794+0000 7f6002ff4740 10 client.15672 fetch_fsmap finished waiting for FSMap version 27 2024-03-28T18:07:54.797 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:54.794+0000 7f6002ff4740 10 client.15672 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.k 2024-03-28T18:07:54.797 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:54.794+0000 7f6002ff4740 20 client.15672 populate_metadata read hostname 'smithi022' 2024-03-28T18:07:54.797 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:54.794+0000 7f6002ff4740 1 --2- 172.21.15.22:0/654970435 >> [v2:172.21.15.79:6832/856896997,v1:172.21.15.79:6833/856896997] conn(0x55f068d03600 0x55f068d05a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:07:54.797 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:54.794+0000 7f6002ff4740 4 client.15672 mds_command: new command op to 34442 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-28T18:07:54.797 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:54.794+0000 7f6002ff4740 1 -- 172.21.15.22:0/654970435 --> [v2:172.21.15.79:6832/856896997,v1:172.21.15.79:6833/856896997] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55f068bf8b50 con 0x55f068d03600 2024-03-28T18:07:54.799 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:54.794+0000 7f5ffce00640 1 --2- 172.21.15.22:0/654970435 >> [v2:172.21.15.79:6832/856896997,v1:172.21.15.79:6833/856896997] conn(0x55f068d03600 0x55f068d05a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:07:54.799 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:54.795+0000 7f5ffce00640 1 --2- 172.21.15.22:0/654970435 >> [v2:172.21.15.79:6832/856896997,v1:172.21.15.79:6833/856896997] conn(0x55f068d03600 0x55f068d05a20 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=7287ca5897cd84ce server_cookie=87498ff9bff2a0ef in_seq=0 out_seq=0 2024-03-28T18:07:54.799 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:54.795+0000 7f5fdf400640 10 client.15672 ms_handle_connect on v2:172.21.15.79:6832/856896997 2024-03-28T18:07:54.799 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:54.795+0000 7f5fdf400640 1 -- 172.21.15.22:0/654970435 <== mds.3 v2:172.21.15.79:6832/856896997 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55f068bf8b50 con 0x55f068d03600 2024-03-28T18:07:54.799 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:54.795+0000 7f5fdf400640 10 client.15672 handle_command_reply: tid=0 2024-03-28T18:07:54.799 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:54.795+0000 7f6002ff4740 1 client.15672 shutdown 2024-03-28T18:07:54.799 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:54.795+0000 7f6002ff4740 1 -- 172.21.15.22:0/654970435 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f068c8d8b0 msgr2=0x55f068c9ce40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:07:54.799 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:54.795+0000 7f6002ff4740 1 --2- 172.21.15.22:0/654970435 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f068c8d8b0 0x55f068c9ce40 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x55f068c8d380 tx=0x7f5fec0046c0 comp rx=0 tx=0).stop 2024-03-28T18:07:54.799 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:54.795+0000 7f6002ff4740 1 -- 172.21.15.22:0/654970435 shutdown_connections 2024-03-28T18:07:54.799 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:54.795+0000 7f6002ff4740 1 --2- 172.21.15.22:0/654970435 >> [v2:172.21.15.79:6832/856896997,v1:172.21.15.79:6833/856896997] conn(0x55f068d03600 0x55f068d05a20 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:07:54.800 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:54.795+0000 7f6002ff4740 1 --2- 172.21.15.22:0/654970435 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f068c96de0 0x55f068cfb500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:07:54.800 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:54.795+0000 7f6002ff4740 1 --2- 172.21.15.22:0/654970435 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55f068c8e260 0x55f068c9d380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:07:54.800 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:54.795+0000 7f6002ff4740 1 --2- 172.21.15.22:0/654970435 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f068c8d8b0 0x55f068c9ce40 unknown :-1 s=CLOSED pgs=484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:07:54.800 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:54.795+0000 7f5fdf400640 0 client.15672 ms_handle_reset on v2:172.21.15.79:6832/856896997 2024-03-28T18:07:54.800 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:54.795+0000 7f6002ff4740 1 -- 172.21.15.22:0/654970435 >> 172.21.15.22:0/654970435 conn(0x55f068c87f80 msgr2=0x55f068c8e690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:07:54.800 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:54.796+0000 7f6002ff4740 1 -- 172.21.15.22:0/654970435 shutdown_connections 2024-03-28T18:07:54.800 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:54.796+0000 7f6002ff4740 1 -- 172.21.15.22:0/654970435 wait complete. 2024-03-28T18:07:54.800 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:54.796+0000 7f6002ff4740 20 client.15672 trim_cache size 0 max 16384 2024-03-28T18:07:54.800 INFO:teuthology.orchestra.run.smithi022.stdout:[] 2024-03-28T18:07:54.801 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:54.796+0000 7f5ffec00640 1 -- 172.21.15.22:0/4220417590 >> [v2:172.21.15.22:6800/1357365082,v1:172.21.15.22:6801/1357365082] conn(0x7f5fe00a2a80 msgr2=0x7f5fe00a4f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:07:54.801 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:54.796+0000 7f5ffec00640 1 --2- 172.21.15.22:0/4220417590 >> [v2:172.21.15.22:6800/1357365082,v1:172.21.15.22:6801/1357365082] conn(0x7f5fe00a2a80 0x7f5fe00a4f40 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x55f068c88ff0 tx=0x7f5fec00b390 comp rx=0 tx=0).stop 2024-03-28T18:07:54.801 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:54.796+0000 7f5ffec00640 1 -- 172.21.15.22:0/4220417590 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5ff8112510 msgr2=0x7f5ff81a2ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:07:54.801 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:54.796+0000 7f5ffec00640 1 --2- 172.21.15.22:0/4220417590 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5ff8112510 0x7f5ff81a2ca0 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f5ff0005e30 tx=0x7f5ff0034e30 comp rx=0 tx=0).stop 2024-03-28T18:07:54.801 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:54.796+0000 7f5ffec00640 1 -- 172.21.15.22:0/4220417590 shutdown_connections 2024-03-28T18:07:54.801 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:54.797+0000 7f5ffec00640 1 --2- 172.21.15.22:0/4220417590 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5ff8112510 0x7f5ff81a2ca0 unknown :-1 s=CLOSED pgs=480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:07:54.801 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:54.797+0000 7f5ffec00640 1 --2- 172.21.15.22:0/4220417590 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5ff8074360 0x7f5ff819e920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:07:54.801 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:54.797+0000 7f5ffec00640 1 --2- 172.21.15.22:0/4220417590 >> [v2:172.21.15.22:6800/1357365082,v1:172.21.15.22:6801/1357365082] conn(0x7f5fe00a2a80 0x7f5fe00a4f40 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:07:54.801 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:54.797+0000 7f5ffec00640 1 --2- 172.21.15.22:0/4220417590 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f5ff8073a40 0x7f5ff819e3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:07:54.802 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:54.797+0000 7f5ffec00640 1 -- 172.21.15.22:0/4220417590 >> 172.21.15.22:0/4220417590 conn(0x7f5ff80fd010 msgr2=0x7f5ff80fdce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:07:54.802 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:54.797+0000 7f5ffec00640 1 -- 172.21.15.22:0/4220417590 shutdown_connections 2024-03-28T18:07:54.802 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:54.797+0000 7f5ffec00640 1 -- 172.21.15.22:0/4220417590 wait complete. 2024-03-28T18:07:54.813 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-28T18:07:54.813 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-03-28T18:07:54.875 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:07:54 smithi022 ceph-mon[31985]: pgmap v13: 97 pgs: 97 active+clean; 1.3 MiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 597 B/s wr, 8 op/s 2024-03-28T18:07:54.875 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:07:54 smithi022 ceph-mon[31985]: from='client.? 172.21.15.22:0/3347287825' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-28T18:07:55.009 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:55.008+0000 7fd456600640 1 -- 172.21.15.22:0/572604589 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd4500764f0 msgr2=0x7fd450111fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:07:55.009 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:55.008+0000 7fd456600640 1 --2- 172.21.15.22:0/572604589 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd4500764f0 0x7fd450111fa0 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7fd43c009900 tx=0x7fd43c02bc30 comp rx=0 tx=0).stop 2024-03-28T18:07:55.009 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:55.008+0000 7fd456600640 1 -- 172.21.15.22:0/572604589 shutdown_connections 2024-03-28T18:07:55.009 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:55.008+0000 7fd456600640 1 --2- 172.21.15.22:0/572604589 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd4501124e0 0x7fd4501148d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:07:55.009 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:55.008+0000 7fd456600640 1 --2- 172.21.15.22:0/572604589 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd4500764f0 0x7fd450111fa0 unknown :-1 s=CLOSED pgs=485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:07:55.009 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:55.008+0000 7fd456600640 1 --2- 172.21.15.22:0/572604589 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd450075bd0 0x7fd450075fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:07:55.009 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:55.008+0000 7fd456600640 1 -- 172.21.15.22:0/572604589 >> 172.21.15.22:0/572604589 conn(0x7fd4500ff310 msgr2=0x7fd450101730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:07:55.009 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:55.008+0000 7fd456600640 1 -- 172.21.15.22:0/572604589 shutdown_connections 2024-03-28T18:07:55.009 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:55.008+0000 7fd456600640 1 -- 172.21.15.22:0/572604589 wait complete. 2024-03-28T18:07:55.011 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:55.009+0000 7fd456600640 1 Processor -- start 2024-03-28T18:07:55.011 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:55.009+0000 7fd456600640 1 -- start start 2024-03-28T18:07:55.012 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:55.009+0000 7fd456600640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd450075bd0 0x7fd45019e550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:07:55.012 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:55.009+0000 7fd456600640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd4500764f0 0x7fd45019ea90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:07:55.012 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:55.009+0000 7fd456600640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd4501124e0 0x7fd4501a2e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:07:55.012 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:55.009+0000 7fd456600640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd450118010 con 0x7fd450075bd0 2024-03-28T18:07:55.012 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:55.009+0000 7fd456600640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd450117e90 con 0x7fd4501124e0 2024-03-28T18:07:55.012 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:55.009+0000 7fd456600640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd450118190 con 0x7fd4500764f0 2024-03-28T18:07:55.012 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:55.010+0000 7fd44f400640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd450075bd0 0x7fd45019e550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:07:55.012 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:55.010+0000 7fd44f400640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd450075bd0 0x7fd45019e550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:50158/0 (socket says 172.21.15.22:50158) 2024-03-28T18:07:55.012 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:55.010+0000 7fd44f400640 1 -- 172.21.15.22:0/3066751926 learned_addr learned my addr 172.21.15.22:0/3066751926 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-28T18:07:55.012 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:55.010+0000 7fd44fe00640 1 --2- 172.21.15.22:0/3066751926 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd4501124e0 0x7fd4501a2e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:07:55.012 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:55.010+0000 7fd44ea00640 1 --2- 172.21.15.22:0/3066751926 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd4500764f0 0x7fd45019ea90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:07:55.012 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:55.010+0000 7fd44f400640 1 -- 172.21.15.22:0/3066751926 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd4500764f0 msgr2=0x7fd45019ea90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:07:55.012 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:55.010+0000 7fd44f400640 1 --2- 172.21.15.22:0/3066751926 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd4500764f0 0x7fd45019ea90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:07:55.013 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:55.010+0000 7fd44f400640 1 -- 172.21.15.22:0/3066751926 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd4501124e0 msgr2=0x7fd4501a2e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:07:55.013 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:55.010+0000 7fd44f400640 1 --2- 172.21.15.22:0/3066751926 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd4501124e0 0x7fd4501a2e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:07:55.013 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:55.010+0000 7fd44f400640 1 -- 172.21.15.22:0/3066751926 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd4501a3540 con 0x7fd450075bd0 2024-03-28T18:07:55.013 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:55.010+0000 7fd44f400640 1 --2- 172.21.15.22:0/3066751926 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd450075bd0 0x7fd45019e550 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7fd44000d950 tx=0x7fd44000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:07:55.013 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:55.011+0000 7fd447e00640 1 -- 172.21.15.22:0/3066751926 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd440017030 con 0x7fd450075bd0 2024-03-28T18:07:55.013 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:55.011+0000 7fd447e00640 1 -- 172.21.15.22:0/3066751926 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7fd440004020 con 0x7fd450075bd0 2024-03-28T18:07:55.013 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:55.011+0000 7fd447e00640 1 -- 172.21.15.22:0/3066751926 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4400047a0 con 0x7fd450075bd0 2024-03-28T18:07:55.013 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:55.011+0000 7fd456600640 1 -- 172.21.15.22:0/3066751926 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd4501a3830 con 0x7fd450075bd0 2024-03-28T18:07:55.013 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:55.011+0000 7fd456600640 1 -- 172.21.15.22:0/3066751926 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd4501ab0c0 con 0x7fd450075bd0 2024-03-28T18:07:55.015 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:55.014+0000 7fd4593d2740 1 Processor -- start 2024-03-28T18:07:55.015 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:55.014+0000 7fd4593d2740 1 -- start start 2024-03-28T18:07:55.015 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:55.014+0000 7fd447e00640 1 -- 172.21.15.22:0/3066751926 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 24) v1 ==== 137466+0+0 (secure 0 0 0) 0x7fd4400041d0 con 0x7fd450075bd0 2024-03-28T18:07:55.015 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:55.014+0000 7fd4593d2740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e57acd36d0 0x55e57acd3ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:07:55.015 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:55.014+0000 7fd4593d2740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55e57acd4080 0x55e57acdc630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:07:55.015 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:55.014+0000 7fd4593d2740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e57acdcc00 0x55e57acdf050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:07:55.015 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:55.014+0000 7fd4593d2740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e57ac372c0 con 0x55e57acdcc00 2024-03-28T18:07:55.015 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:55.014+0000 7fd4593d2740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e57abdfc90 con 0x55e57acd4080 2024-03-28T18:07:55.015 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:55.014+0000 7fd4593d2740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e57ac33360 con 0x55e57acd36d0 2024-03-28T18:07:55.017 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:55.015+0000 7fd44ea00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e57acdcc00 0x55e57acdf050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:07:55.017 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:55.015+0000 7fd44ea00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e57acdcc00 0x55e57acdf050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:50170/0 (socket says 172.21.15.22:50170) 2024-03-28T18:07:55.017 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:55.015+0000 7fd44ea00640 1 -- 172.21.15.22:0/1711894078 learned_addr learned my addr 172.21.15.22:0/1711894078 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-28T18:07:55.017 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:55.015+0000 7fd44f400640 1 --2- 172.21.15.22:0/1711894078 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55e57acd4080 0x55e57acdc630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:07:55.017 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:55.015+0000 7fd44fe00640 1 --2- 172.21.15.22:0/1711894078 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e57acd36d0 0x55e57acd3ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:07:55.017 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:55.015+0000 7fd44ea00640 1 -- 172.21.15.22:0/1711894078 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e57acd36d0 msgr2=0x55e57acd3ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T18:07:55.017 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:55.015+0000 7fd44ea00640 1 --2- 172.21.15.22:0/1711894078 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e57acd36d0 0x55e57acd3ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:07:55.017 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:55.015+0000 7fd44ea00640 1 -- 172.21.15.22:0/1711894078 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55e57acd4080 msgr2=0x55e57acdc630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T18:07:55.017 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:55.015+0000 7fd44ea00640 1 --2- 172.21.15.22:0/1711894078 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55e57acd4080 0x55e57acdc630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:07:55.017 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:55.015+0000 7fd44ea00640 1 -- 172.21.15.22:0/1711894078 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e57ac04080 con 0x55e57acdcc00 2024-03-28T18:07:55.017 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:55.016+0000 7fd44ea00640 1 --2- 172.21.15.22:0/1711894078 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e57acdcc00 0x55e57acdf050 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7fd43c009900 tx=0x7fd43c005dd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=626ba4a3085ead51 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:07:55.017 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:55.016+0000 7fd447e00640 1 --2- 172.21.15.22:0/3066751926 >> [v2:172.21.15.22:6800/1357365082,v1:172.21.15.22:6801/1357365082] conn(0x7fd4340a2a00 0x7fd4340a4ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:07:55.018 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:55.016+0000 7fd447e00640 1 -- 172.21.15.22:0/3066751926 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 8411+0+0 (secure 0 0 0) 0x7fd4400ceae0 con 0x7fd450075bd0 2024-03-28T18:07:55.018 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:55.016+0000 7fd444c00640 1 -- 172.21.15.22:0/1711894078 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd43c02ca10 con 0x55e57acdcc00 2024-03-28T18:07:55.018 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:55.016+0000 7fd444c00640 1 -- 172.21.15.22:0/1711894078 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7fd43c02cbb0 con 0x55e57acdcc00 2024-03-28T18:07:55.018 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:55.016+0000 7fd444c00640 1 -- 172.21.15.22:0/1711894078 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd43c004920 con 0x55e57acdcc00 2024-03-28T18:07:55.018 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:55.016+0000 7fd44fe00640 1 --2- 172.21.15.22:0/3066751926 >> [v2:172.21.15.22:6800/1357365082,v1:172.21.15.22:6801/1357365082] conn(0x7fd4340a2a00 0x7fd4340a4ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:07:55.021 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:55.017+0000 7fd4593d2740 1 -- 172.21.15.22:0/1711894078 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e57acdcc00 msgr2=0x55e57acdf050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:07:55.021 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:55.017+0000 7fd4593d2740 1 --2- 172.21.15.22:0/1711894078 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e57acdcc00 0x55e57acdf050 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7fd43c009900 tx=0x7fd43c005dd0 comp rx=0 tx=0).stop 2024-03-28T18:07:55.021 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:55.017+0000 7fd44fe00640 1 --2- 172.21.15.22:0/3066751926 >> [v2:172.21.15.22:6800/1357365082,v1:172.21.15.22:6801/1357365082] conn(0x7fd4340a2a00 0x7fd4340a4ec0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fd45006ab50 tx=0x7fd448009760 comp rx=0 tx=0).ready entity=mgr.15522 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:07:55.021 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:55.017+0000 7fd4593d2740 1 -- 172.21.15.22:0/1711894078 shutdown_connections 2024-03-28T18:07:55.021 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:55.017+0000 7fd4593d2740 1 --2- 172.21.15.22:0/1711894078 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e57acdcc00 0x55e57acdf050 unknown :-1 s=CLOSED pgs=487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:07:55.022 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:55.017+0000 7fd4593d2740 1 --2- 172.21.15.22:0/1711894078 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55e57acd4080 0x55e57acdc630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:07:55.022 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:55.017+0000 7fd4593d2740 1 --2- 172.21.15.22:0/1711894078 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e57acd36d0 0x55e57acd3ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:07:55.022 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:55.017+0000 7fd4593d2740 1 -- 172.21.15.22:0/1711894078 >> 172.21.15.22:0/1711894078 conn(0x55e57accdda0 msgr2=0x55e57acce180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:07:55.022 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:55.017+0000 7fd4593d2740 1 -- 172.21.15.22:0/1711894078 shutdown_connections 2024-03-28T18:07:55.022 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:55.017+0000 7fd4593d2740 1 -- 172.21.15.22:0/1711894078 wait complete. 2024-03-28T18:07:55.022 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:55.018+0000 7fd4593d2740 1 -- start start 2024-03-28T18:07:55.022 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:55.018+0000 7fd4593d2740 1 Processor -- start 2024-03-28T18:07:55.022 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:55.018+0000 7fd4593d2740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55e57acd36d0 0x55e57acdc5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:07:55.022 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:55.018+0000 7fd4593d2740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e57acd4080 0x55e57acd9930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:07:55.022 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:55.018+0000 7fd4593d2740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e57acdcc00 0x55e57acda070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:07:55.022 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:55.019+0000 7fd4593d2740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e57acdfb10 con 0x55e57acd4080 2024-03-28T18:07:55.022 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:55.019+0000 7fd4593d2740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e57abf30d0 con 0x55e57acd36d0 2024-03-28T18:07:55.022 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:55.019+0000 7fd4593d2740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e57acdfc90 con 0x55e57acdcc00 2024-03-28T18:07:55.023 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:55.019+0000 7fd4593d2740 10 client.0 _ll_register_callbacks cb 0x55e57abf7e80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-28T18:07:55.023 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:55.019+0000 7fd44ea00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e57acd4080 0x55e57acd9930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:07:55.023 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:55.019+0000 7fd44ea00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e57acd4080 0x55e57acd9930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:50180/0 (socket says 172.21.15.22:50180) 2024-03-28T18:07:55.023 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:55.019+0000 7fd44ea00640 1 -- 172.21.15.22:0/2319305581 learned_addr learned my addr 172.21.15.22:0/2319305581 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-28T18:07:55.023 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:55.019+0000 7fd44f400640 1 --2- 172.21.15.22:0/2319305581 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55e57acd36d0 0x55e57acdc5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:07:55.023 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:55.019+0000 7fd44fe00640 1 --2- 172.21.15.22:0/2319305581 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e57acdcc00 0x55e57acda070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:07:55.023 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:55.019+0000 7fd44ea00640 1 -- 172.21.15.22:0/2319305581 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e57acdcc00 msgr2=0x55e57acda070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T18:07:55.023 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:55.019+0000 7fd44ea00640 1 --2- 172.21.15.22:0/2319305581 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e57acdcc00 0x55e57acda070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:07:55.023 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:55.019+0000 7fd44ea00640 1 -- 172.21.15.22:0/2319305581 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55e57acd36d0 msgr2=0x55e57acdc5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T18:07:55.023 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:55.019+0000 7fd44ea00640 1 --2- 172.21.15.22:0/2319305581 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55e57acd36d0 0x55e57acdc5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:07:55.023 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:55.019+0000 7fd44ea00640 1 -- 172.21.15.22:0/2319305581 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55e57a94b3f0 con 0x55e57acd4080 2024-03-28T18:07:55.023 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:55.020+0000 7fd44ea00640 1 --2- 172.21.15.22:0/2319305581 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e57acd4080 0x55e57acd9930 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7fd43c02c170 tx=0x7fd43c005fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=99bb619053198e81 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:07:55.023 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:55.020+0000 7fd439600640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-03-28T18:07:55.023 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:55.020+0000 7fd439600640 1 -- 172.21.15.22:0/2319305581 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd43c04c030 con 0x55e57acd4080 2024-03-28T18:07:55.024 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:55.020+0000 7fd439600640 1 -- 172.21.15.22:0/2319305581 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 8411+0+0 (secure 0 0 0) 0x7fd43c0409b0 con 0x55e57acd4080 2024-03-28T18:07:55.024 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:55.020+0000 7fd4593d2740 1 -- 172.21.15.22:0/2319305581 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fd43c02cbb0 con 0x55e57acd4080 2024-03-28T18:07:55.024 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:55.020+0000 7fd439600640 1 -- 172.21.15.22:0/2319305581 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd43c056050 con 0x55e57acd4080 2024-03-28T18:07:55.024 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:55.020+0000 7fd4593d2740 10 client.15684 fetch_fsmap learned FSMap version 27 2024-03-28T18:07:55.024 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:55.020+0000 7fd4593d2740 1 -- 172.21.15.22:0/2319305581 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55e57aa157d0 con 0x55e57acd4080 2024-03-28T18:07:55.024 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:55.020+0000 7fd439600640 1 -- 172.21.15.22:0/2319305581 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7fd43c02cbb0 con 0x55e57acd4080 2024-03-28T18:07:55.024 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:55.020+0000 7fd439600640 1 -- 172.21.15.22:0/2319305581 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 27) v1 ==== 7934+0+0 (secure 0 0 0) 0x7fd43c034c40 con 0x55e57acd4080 2024-03-28T18:07:55.024 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:55.021+0000 7fd4593d2740 10 client.15684 fetch_fsmap finished waiting for FSMap version 27 2024-03-28T18:07:55.024 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:55.021+0000 7fd4593d2740 10 client.15684 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.e 2024-03-28T18:07:55.024 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:55.021+0000 7fd4593d2740 20 client.15684 populate_metadata read hostname 'smithi022' 2024-03-28T18:07:55.024 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:55.021+0000 7fd4593d2740 1 --2- 172.21.15.22:0/2319305581 >> [v2:172.21.15.79:6834/3365773431,v1:172.21.15.79:6835/3365773431] conn(0x55e57acec920 0x55e57aceed10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:07:55.024 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:55.021+0000 7fd4593d2740 4 client.15684 mds_command: new command op to 34475 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-28T18:07:55.024 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:55.021+0000 7fd4593d2740 1 -- 172.21.15.22:0/2319305581 --> [v2:172.21.15.79:6834/3365773431,v1:172.21.15.79:6835/3365773431] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55e57ac44800 con 0x55e57acec920 2024-03-28T18:07:55.026 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:55.021+0000 7fd44f400640 1 --2- 172.21.15.22:0/2319305581 >> [v2:172.21.15.79:6834/3365773431,v1:172.21.15.79:6835/3365773431] conn(0x55e57acec920 0x55e57aceed10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:07:55.026 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:55.022+0000 7fd44f400640 1 --2- 172.21.15.22:0/2319305581 >> [v2:172.21.15.79:6834/3365773431,v1:172.21.15.79:6835/3365773431] conn(0x55e57acec920 0x55e57aceed10 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=ddd63ec4c22c1b server_cookie=8a137e77635c0e38 in_seq=0 out_seq=0 2024-03-28T18:07:55.026 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:55.022+0000 7fd439600640 10 client.15684 ms_handle_connect on v2:172.21.15.79:6834/3365773431 2024-03-28T18:07:55.026 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:55.023+0000 7fd439600640 1 -- 172.21.15.22:0/2319305581 <== mds.2 v2:172.21.15.79:6834/3365773431 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x55e57ac44800 con 0x55e57acec920 2024-03-28T18:07:55.026 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:55.023+0000 7fd439600640 10 client.15684 handle_command_reply: tid=0 2024-03-28T18:07:55.026 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:55.023+0000 7fd4593d2740 1 client.15684 shutdown 2024-03-28T18:07:55.026 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:55.023+0000 7fd4593d2740 1 -- 172.21.15.22:0/2319305581 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e57acd4080 msgr2=0x55e57acd9930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:07:55.026 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:55.023+0000 7fd4593d2740 1 --2- 172.21.15.22:0/2319305581 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e57acd4080 0x55e57acd9930 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7fd43c02c170 tx=0x7fd43c005fd0 comp rx=0 tx=0).stop 2024-03-28T18:07:55.026 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:55.024+0000 7fd4593d2740 1 -- 172.21.15.22:0/2319305581 shutdown_connections 2024-03-28T18:07:55.026 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:55.024+0000 7fd4593d2740 1 --2- 172.21.15.22:0/2319305581 >> [v2:172.21.15.79:6834/3365773431,v1:172.21.15.79:6835/3365773431] conn(0x55e57acec920 0x55e57aceed10 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:07:55.026 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:55.024+0000 7fd4593d2740 1 --2- 172.21.15.22:0/2319305581 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e57acdcc00 0x55e57acda070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:07:55.026 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:55.024+0000 7fd4593d2740 1 --2- 172.21.15.22:0/2319305581 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e57acd4080 0x55e57acd9930 unknown :-1 s=CLOSED pgs=488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:07:55.027 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:55.024+0000 7fd4593d2740 1 --2- 172.21.15.22:0/2319305581 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55e57acd36d0 0x55e57acdc5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:07:55.027 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:55.024+0000 7fd4593d2740 1 -- 172.21.15.22:0/2319305581 >> 172.21.15.22:0/2319305581 conn(0x55e57accdda0 msgr2=0x55e57accfaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:07:55.027 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:55.024+0000 7fd439600640 0 client.15684 ms_handle_reset on v2:172.21.15.79:6834/3365773431 2024-03-28T18:07:55.027 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:55.024+0000 7fd4593d2740 1 -- 172.21.15.22:0/2319305581 shutdown_connections 2024-03-28T18:07:55.027 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:55.024+0000 7fd4593d2740 1 -- 172.21.15.22:0/2319305581 wait complete. 2024-03-28T18:07:55.027 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:55.024+0000 7fd4593d2740 20 client.15684 trim_cache size 0 max 16384 2024-03-28T18:07:55.039 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:55.037+0000 7fd4593d2740 1 Processor -- start 2024-03-28T18:07:55.039 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:55.037+0000 7fd4593d2740 1 -- start start 2024-03-28T18:07:55.039 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:55.038+0000 7fd4593d2740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e57acd36d0 0x55e57acdc060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:07:55.039 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:55.038+0000 7fd4593d2740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55e57acd4080 0x55e57acdc5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:07:55.039 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:55.038+0000 7fd4593d2740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e57acdcc00 0x55e57acd6b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:07:55.039 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:55.038+0000 7fd4593d2740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4501b3620 con 0x55e57acd36d0 2024-03-28T18:07:55.039 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:55.038+0000 7fd4593d2740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4501b4fa0 con 0x55e57acd4080 2024-03-28T18:07:55.039 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:55.038+0000 7fd4593d2740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4501b3c80 con 0x55e57acdcc00 2024-03-28T18:07:55.039 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:55.038+0000 7fd44f400640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e57acd36d0 0x55e57acdc060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:07:55.040 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:55.038+0000 7fd44f400640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e57acd36d0 0x55e57acdc060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:50192/0 (socket says 172.21.15.22:50192) 2024-03-28T18:07:55.040 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:55.038+0000 7fd44f400640 1 -- 172.21.15.22:0/1085970379 learned_addr learned my addr 172.21.15.22:0/1085970379 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-28T18:07:55.040 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:55.038+0000 7fd44ea00640 1 --2- 172.21.15.22:0/1085970379 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55e57acd4080 0x55e57acdc5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:07:55.040 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:55.038+0000 7fd44fe00640 1 --2- 172.21.15.22:0/1085970379 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e57acdcc00 0x55e57acd6b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:07:55.040 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:55.038+0000 7fd44f400640 1 -- 172.21.15.22:0/1085970379 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e57acdcc00 msgr2=0x55e57acd6b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T18:07:55.040 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:55.038+0000 7fd44f400640 1 --2- 172.21.15.22:0/1085970379 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e57acdcc00 0x55e57acd6b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:07:55.040 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:55.038+0000 7fd44f400640 1 -- 172.21.15.22:0/1085970379 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55e57acd4080 msgr2=0x55e57acdc5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T18:07:55.040 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:55.038+0000 7fd44f400640 1 --2- 172.21.15.22:0/1085970379 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55e57acd4080 0x55e57acdc5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:07:55.040 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:55.038+0000 7fd44f400640 1 -- 172.21.15.22:0/1085970379 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e57acda3e0 con 0x55e57acd36d0 2024-03-28T18:07:55.040 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:55.039+0000 7fd44f400640 1 --2- 172.21.15.22:0/1085970379 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e57acd36d0 0x55e57acdc060 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7fd440097440 tx=0x7fd4400d24e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3f6b99f564e3e5d1 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:07:55.044 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:55.039+0000 7fd43be00640 1 -- 172.21.15.22:0/1085970379 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd44009ab20 con 0x55e57acd36d0 2024-03-28T18:07:55.044 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:55.039+0000 7fd43be00640 1 -- 172.21.15.22:0/1085970379 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7fd44009b6e0 con 0x55e57acd36d0 2024-03-28T18:07:55.044 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:55.039+0000 7fd43be00640 1 -- 172.21.15.22:0/1085970379 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd44009a940 con 0x55e57acd36d0 2024-03-28T18:07:55.044 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:55.040+0000 7fd4593d2740 1 -- 172.21.15.22:0/1085970379 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e57acd36d0 msgr2=0x55e57acdc060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:07:55.044 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:55.040+0000 7fd4593d2740 1 --2- 172.21.15.22:0/1085970379 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e57acd36d0 0x55e57acdc060 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7fd440097440 tx=0x7fd4400d24e0 comp rx=0 tx=0).stop 2024-03-28T18:07:55.044 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:55.040+0000 7fd4593d2740 1 -- 172.21.15.22:0/1085970379 shutdown_connections 2024-03-28T18:07:55.044 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:55.040+0000 7fd4593d2740 1 --2- 172.21.15.22:0/1085970379 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e57acdcc00 0x55e57acd6b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:07:55.044 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:55.040+0000 7fd4593d2740 1 --2- 172.21.15.22:0/1085970379 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55e57acd4080 0x55e57acdc5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:07:55.044 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:55.040+0000 7fd4593d2740 1 --2- 172.21.15.22:0/1085970379 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e57acd36d0 0x55e57acdc060 unknown :-1 s=CLOSED pgs=489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:07:55.044 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:55.040+0000 7fd4593d2740 1 -- 172.21.15.22:0/1085970379 >> 172.21.15.22:0/1085970379 conn(0x55e57accdda0 msgr2=0x55e57accceb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:07:55.044 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:55.040+0000 7fd4593d2740 1 -- 172.21.15.22:0/1085970379 shutdown_connections 2024-03-28T18:07:55.044 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:55.040+0000 7fd4593d2740 1 -- 172.21.15.22:0/1085970379 wait complete. 2024-03-28T18:07:55.045 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:55.040+0000 7fd4593d2740 1 -- start start 2024-03-28T18:07:55.045 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:55.040+0000 7fd4593d2740 1 Processor -- start 2024-03-28T18:07:55.045 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:55.041+0000 7fd4593d2740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e57acd36d0 0x55e57acf18b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:07:55.045 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:55.041+0000 7fd4593d2740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55e57acd4080 0x55e57acefc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:07:55.045 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:55.041+0000 7fd4593d2740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e57acdcc00 0x55e57acf0190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:07:55.045 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:55.041+0000 7fd4593d2740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4501b4940 con 0x55e57acd36d0 2024-03-28T18:07:55.045 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:55.041+0000 7fd4593d2740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4501b42e0 con 0x55e57acd4080 2024-03-28T18:07:55.045 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:55.041+0000 7fd4593d2740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4501b17e0 con 0x55e57acdcc00 2024-03-28T18:07:55.045 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:55.041+0000 7fd4593d2740 10 client.0 _ll_register_callbacks cb 0x7fd4400cc010 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-28T18:07:55.045 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:55.041+0000 7fd44f400640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e57acd36d0 0x55e57acf18b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:07:55.045 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:55.041+0000 7fd44f400640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e57acd36d0 0x55e57acf18b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:50206/0 (socket says 172.21.15.22:50206) 2024-03-28T18:07:55.045 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:55.041+0000 7fd44f400640 1 -- 172.21.15.22:0/3307302710 learned_addr learned my addr 172.21.15.22:0/3307302710 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-28T18:07:55.045 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:55.041+0000 7fd44ea00640 1 --2- 172.21.15.22:0/3307302710 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55e57acd4080 0x55e57acefc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:07:55.045 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:55.041+0000 7fd44fe00640 1 --2- 172.21.15.22:0/3307302710 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e57acdcc00 0x55e57acf0190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:07:55.045 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:55.041+0000 7fd44f400640 1 -- 172.21.15.22:0/3307302710 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e57acdcc00 msgr2=0x55e57acf0190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T18:07:55.046 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:55.041+0000 7fd44f400640 1 --2- 172.21.15.22:0/3307302710 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e57acdcc00 0x55e57acf0190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:07:55.046 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:55.041+0000 7fd44f400640 1 -- 172.21.15.22:0/3307302710 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55e57acd4080 msgr2=0x55e57acefc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T18:07:55.046 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:55.041+0000 7fd44f400640 1 --2- 172.21.15.22:0/3307302710 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55e57acd4080 0x55e57acefc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:07:55.046 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:55.041+0000 7fd44f400640 1 -- 172.21.15.22:0/3307302710 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55e57acf07a0 con 0x55e57acd36d0 2024-03-28T18:07:55.046 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:55.041+0000 7fd44fe00640 1 --2- 172.21.15.22:0/3307302710 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e57acdcc00 0x55e57acf0190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T18:07:55.046 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:55.042+0000 7fd44f400640 1 --2- 172.21.15.22:0/3307302710 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e57acd36d0 0x55e57acf18b0 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7fd4400a3250 tx=0x7fd440005190 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4826ee5b473ac7e9 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:07:55.046 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:55.042+0000 7fd439600640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-03-28T18:07:55.046 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:55.042+0000 7fd439600640 1 -- 172.21.15.22:0/3307302710 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd44009a940 con 0x55e57acd36d0 2024-03-28T18:07:55.046 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:55.042+0000 7fd439600640 1 -- 172.21.15.22:0/3307302710 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 8411+0+0 (secure 0 0 0) 0x7fd4400d8040 con 0x55e57acd36d0 2024-03-28T18:07:55.046 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:55.042+0000 7fd4593d2740 1 -- 172.21.15.22:0/3307302710 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fd44009b6e0 con 0x55e57acd36d0 2024-03-28T18:07:55.046 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:55.042+0000 7fd439600640 1 -- 172.21.15.22:0/3307302710 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd440065050 con 0x55e57acd36d0 2024-03-28T18:07:55.046 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:55.042+0000 7fd4593d2740 10 client.15690 fetch_fsmap learned FSMap version 27 2024-03-28T18:07:55.046 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:55.042+0000 7fd4593d2740 1 -- 172.21.15.22:0/3307302710 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55e57ad48cb0 con 0x55e57acd36d0 2024-03-28T18:07:55.046 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:55.042+0000 7fd439600640 1 -- 172.21.15.22:0/3307302710 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7fd44009b6e0 con 0x55e57acd36d0 2024-03-28T18:07:55.047 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:55.043+0000 7fd439600640 1 -- 172.21.15.22:0/3307302710 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 27) v1 ==== 7934+0+0 (secure 0 0 0) 0x7fd4400cc430 con 0x55e57acd36d0 2024-03-28T18:07:55.047 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:55.043+0000 7fd4593d2740 10 client.15690 fetch_fsmap finished waiting for FSMap version 27 2024-03-28T18:07:55.047 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:55.043+0000 7fd4593d2740 10 client.15690 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.e 2024-03-28T18:07:55.047 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:55.043+0000 7fd4593d2740 20 client.15690 populate_metadata read hostname 'smithi022' 2024-03-28T18:07:55.047 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:55.043+0000 7fd4593d2740 1 --2- 172.21.15.22:0/3307302710 >> [v2:172.21.15.79:6834/3365773431,v1:172.21.15.79:6835/3365773431] conn(0x55e57ad492d0 0x55e57ad4b6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:07:55.047 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:55.043+0000 7fd4593d2740 4 client.15690 mds_command: new command op to 34475 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-28T18:07:55.047 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:55.043+0000 7fd4593d2740 1 -- 172.21.15.22:0/3307302710 --> [v2:172.21.15.79:6834/3365773431,v1:172.21.15.79:6835/3365773431] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55e57ac3eb50 con 0x55e57ad492d0 2024-03-28T18:07:55.047 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:55.043+0000 7fd44ea00640 1 --2- 172.21.15.22:0/3307302710 >> [v2:172.21.15.79:6834/3365773431,v1:172.21.15.79:6835/3365773431] conn(0x55e57ad492d0 0x55e57ad4b6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:07:55.048 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:55.044+0000 7fd44ea00640 1 --2- 172.21.15.22:0/3307302710 >> [v2:172.21.15.79:6834/3365773431,v1:172.21.15.79:6835/3365773431] conn(0x55e57ad492d0 0x55e57ad4b6c0 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=469d7c82003dd29c server_cookie=95c29b6e5fccde9a in_seq=0 out_seq=0 2024-03-28T18:07:55.048 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:55.044+0000 7fd439600640 10 client.15690 ms_handle_connect on v2:172.21.15.79:6834/3365773431 2024-03-28T18:07:55.048 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:55.044+0000 7fd439600640 1 -- 172.21.15.22:0/3307302710 <== mds.2 v2:172.21.15.79:6834/3365773431 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55e57ac3eb50 con 0x55e57ad492d0 2024-03-28T18:07:55.048 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:55.044+0000 7fd439600640 10 client.15690 handle_command_reply: tid=0 2024-03-28T18:07:55.048 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:55.044+0000 7fd4593d2740 1 client.15690 shutdown 2024-03-28T18:07:55.048 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:55.044+0000 7fd4593d2740 1 -- 172.21.15.22:0/3307302710 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e57acd36d0 msgr2=0x55e57acf18b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:07:55.049 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:55.044+0000 7fd4593d2740 1 --2- 172.21.15.22:0/3307302710 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e57acd36d0 0x55e57acf18b0 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7fd4400a3250 tx=0x7fd440005190 comp rx=0 tx=0).stop 2024-03-28T18:07:55.049 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:55.044+0000 7fd4593d2740 1 -- 172.21.15.22:0/3307302710 shutdown_connections 2024-03-28T18:07:55.049 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:55.044+0000 7fd4593d2740 1 --2- 172.21.15.22:0/3307302710 >> [v2:172.21.15.79:6834/3365773431,v1:172.21.15.79:6835/3365773431] conn(0x55e57ad492d0 0x55e57ad4b6c0 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:07:55.049 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:55.044+0000 7fd4593d2740 1 --2- 172.21.15.22:0/3307302710 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e57acdcc00 0x55e57acf0190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:07:55.049 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:55.044+0000 7fd4593d2740 1 --2- 172.21.15.22:0/3307302710 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55e57acd4080 0x55e57acefc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:07:55.049 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:55.044+0000 7fd439600640 0 client.15690 ms_handle_reset on v2:172.21.15.79:6834/3365773431 2024-03-28T18:07:55.049 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:55.044+0000 7fd4593d2740 1 --2- 172.21.15.22:0/3307302710 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e57acd36d0 0x55e57acf18b0 unknown :-1 s=CLOSED pgs=490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:07:55.049 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:55.044+0000 7fd4593d2740 1 -- 172.21.15.22:0/3307302710 >> 172.21.15.22:0/3307302710 conn(0x55e57accdda0 msgr2=0x55e57acd4460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:07:55.049 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:55.045+0000 7fd4593d2740 1 -- 172.21.15.22:0/3307302710 shutdown_connections 2024-03-28T18:07:55.049 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:55.045+0000 7fd4593d2740 1 -- 172.21.15.22:0/3307302710 wait complete. 2024-03-28T18:07:55.049 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:55.045+0000 7fd4593d2740 20 client.15690 trim_cache size 0 max 16384 2024-03-28T18:07:55.049 INFO:teuthology.orchestra.run.smithi022.stdout:[] 2024-03-28T18:07:55.050 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:55.045+0000 7fd456600640 1 -- 172.21.15.22:0/3066751926 >> [v2:172.21.15.22:6800/1357365082,v1:172.21.15.22:6801/1357365082] conn(0x7fd4340a2a00 msgr2=0x7fd4340a4ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:07:55.050 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:55.045+0000 7fd456600640 1 --2- 172.21.15.22:0/3066751926 >> [v2:172.21.15.22:6800/1357365082,v1:172.21.15.22:6801/1357365082] conn(0x7fd4340a2a00 0x7fd4340a4ec0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fd45006ab50 tx=0x7fd448009760 comp rx=0 tx=0).stop 2024-03-28T18:07:55.050 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:55.045+0000 7fd456600640 1 -- 172.21.15.22:0/3066751926 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd450075bd0 msgr2=0x7fd45019e550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:07:55.050 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:55.045+0000 7fd456600640 1 --2- 172.21.15.22:0/3066751926 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd450075bd0 0x7fd45019e550 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7fd44000d950 tx=0x7fd44000de10 comp rx=0 tx=0).stop 2024-03-28T18:07:55.050 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:55.045+0000 7fd456600640 1 -- 172.21.15.22:0/3066751926 shutdown_connections 2024-03-28T18:07:55.051 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:55.045+0000 7fd456600640 1 --2- 172.21.15.22:0/3066751926 >> [v2:172.21.15.22:6800/1357365082,v1:172.21.15.22:6801/1357365082] conn(0x7fd4340a2a00 0x7fd4340a4ec0 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:07:55.051 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:55.045+0000 7fd456600640 1 --2- 172.21.15.22:0/3066751926 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd4501124e0 0x7fd4501a2e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:07:55.051 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:55.045+0000 7fd456600640 1 --2- 172.21.15.22:0/3066751926 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd4500764f0 0x7fd45019ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:07:55.051 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:55.045+0000 7fd456600640 1 --2- 172.21.15.22:0/3066751926 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd450075bd0 0x7fd45019e550 unknown :-1 s=CLOSED pgs=486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:07:55.051 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:55.045+0000 7fd456600640 1 -- 172.21.15.22:0/3066751926 >> 172.21.15.22:0/3066751926 conn(0x7fd4500ff310 msgr2=0x7fd450114070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:07:55.051 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:55.046+0000 7fd456600640 1 -- 172.21.15.22:0/3066751926 shutdown_connections 2024-03-28T18:07:55.051 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:55.046+0000 7fd456600640 1 -- 172.21.15.22:0/3066751926 wait complete. 2024-03-28T18:07:55.061 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-28T18:07:55.061 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-03-28T18:07:55.174 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:07:54 smithi134 ceph-mon[35037]: pgmap v13: 97 pgs: 97 active+clean; 1.3 MiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 597 B/s wr, 8 op/s 2024-03-28T18:07:55.175 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:07:54 smithi134 ceph-mon[35037]: from='client.? 172.21.15.22:0/3347287825' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-28T18:07:55.182 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:07:54 smithi079 ceph-mon[35312]: pgmap v13: 97 pgs: 97 active+clean; 1.3 MiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 597 B/s wr, 8 op/s 2024-03-28T18:07:55.182 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:07:54 smithi079 ceph-mon[35312]: from='client.? 172.21.15.22:0/3347287825' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-28T18:07:55.261 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:55.260+0000 7f40e6400640 1 -- 172.21.15.22:0/3793492024 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f40e00688a0 msgr2=0x7f40e0068d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:07:55.261 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:55.260+0000 7f40e6400640 1 --2- 172.21.15.22:0/3793492024 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f40e00688a0 0x7f40e0068d20 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f40c8009890 tx=0x7f40c802bb70 comp rx=0 tx=0).stop 2024-03-28T18:07:55.261 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:55.260+0000 7f40e6400640 1 -- 172.21.15.22:0/3793492024 shutdown_connections 2024-03-28T18:07:55.261 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:55.260+0000 7f40e6400640 1 --2- 172.21.15.22:0/3793492024 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f40e0069260 0x7f40e010e500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:07:55.261 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:55.260+0000 7f40e6400640 1 --2- 172.21.15.22:0/3793492024 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f40e00688a0 0x7f40e0068d20 unknown :-1 s=CLOSED pgs=491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:07:55.261 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:55.260+0000 7f40e6400640 1 --2- 172.21.15.22:0/3793492024 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f40e01076f0 0x7f40e0107ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:07:55.261 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:55.260+0000 7f40e6400640 1 -- 172.21.15.22:0/3793492024 >> 172.21.15.22:0/3793492024 conn(0x7f40e00776a0 msgr2=0x7f40e0077ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:07:55.261 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:55.260+0000 7f40e6400640 1 -- 172.21.15.22:0/3793492024 shutdown_connections 2024-03-28T18:07:55.261 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:55.260+0000 7f40e6400640 1 -- 172.21.15.22:0/3793492024 wait complete. 2024-03-28T18:07:55.263 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:55.261+0000 7f40e6400640 1 Processor -- start 2024-03-28T18:07:55.263 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:55.261+0000 7f40e6400640 1 -- start start 2024-03-28T18:07:55.263 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:55.262+0000 7f40e6400640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f40e00688a0 0x7f40e019a110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:07:55.263 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:55.262+0000 7f40e6400640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f40e0069260 0x7f40e019a650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:07:55.263 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:55.262+0000 7f40e6400640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f40e01076f0 0x7f40e019e9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:07:55.263 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:55.262+0000 7f40e6400640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40e0111a10 con 0x7f40e00688a0 2024-03-28T18:07:55.263 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:55.262+0000 7f40e6400640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40e0111890 con 0x7f40e0069260 2024-03-28T18:07:55.263 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:55.262+0000 7f40e6400640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40e0111b90 con 0x7f40e01076f0 2024-03-28T18:07:55.264 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:55.262+0000 7f40df400640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f40e00688a0 0x7f40e019a110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:07:55.264 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:55.262+0000 7f40df400640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f40e00688a0 0x7f40e019a110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:50218/0 (socket says 172.21.15.22:50218) 2024-03-28T18:07:55.264 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:55.262+0000 7f40df400640 1 -- 172.21.15.22:0/35191753 learned_addr learned my addr 172.21.15.22:0/35191753 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-28T18:07:55.264 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:55.262+0000 7f40dea00640 1 --2- 172.21.15.22:0/35191753 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f40e0069260 0x7f40e019a650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:07:55.264 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:55.262+0000 7f40dfe00640 1 --2- 172.21.15.22:0/35191753 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f40e01076f0 0x7f40e019e9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:07:55.264 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:55.262+0000 7f40df400640 1 -- 172.21.15.22:0/35191753 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f40e01076f0 msgr2=0x7f40e019e9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:07:55.264 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:55.262+0000 7f40df400640 1 --2- 172.21.15.22:0/35191753 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f40e01076f0 0x7f40e019e9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:07:55.264 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:55.263+0000 7f40df400640 1 -- 172.21.15.22:0/35191753 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f40e0069260 msgr2=0x7f40e019a650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:07:55.264 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:55.263+0000 7f40df400640 1 --2- 172.21.15.22:0/35191753 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f40e0069260 0x7f40e019a650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:07:55.264 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:55.263+0000 7f40df400640 1 -- 172.21.15.22:0/35191753 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f40e019f100 con 0x7f40e00688a0 2024-03-28T18:07:55.265 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:55.263+0000 7f40df400640 1 --2- 172.21.15.22:0/35191753 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f40e00688a0 0x7f40e019a110 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f40d000c420 tx=0x7f40d0009f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:07:55.265 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:55.263+0000 7f40d7e00640 1 -- 172.21.15.22:0/35191753 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40d0003970 con 0x7f40e00688a0 2024-03-28T18:07:55.265 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:55.263+0000 7f40e6400640 1 -- 172.21.15.22:0/35191753 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f40e019f3f0 con 0x7f40e00688a0 2024-03-28T18:07:55.265 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:55.263+0000 7f40d7e00640 1 -- 172.21.15.22:0/35191753 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f40d0004020 con 0x7f40e00688a0 2024-03-28T18:07:55.265 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:55.263+0000 7f40d7e00640 1 -- 172.21.15.22:0/35191753 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40d000ba30 con 0x7f40e00688a0 2024-03-28T18:07:55.265 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:55.263+0000 7f40e6400640 1 -- 172.21.15.22:0/35191753 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f40e01a6c80 con 0x7f40e00688a0 2024-03-28T18:07:55.267 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:55.266+0000 7f40d7e00640 1 -- 172.21.15.22:0/35191753 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 24) v1 ==== 137466+0+0 (secure 0 0 0) 0x7f40d001d020 con 0x7f40e00688a0 2024-03-28T18:07:55.267 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:55.266+0000 7f40e9079740 1 Processor -- start 2024-03-28T18:07:55.267 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:55.266+0000 7f40e9079740 1 -- start start 2024-03-28T18:07:55.269 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:55.267+0000 7f40e9079740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560408b246d0 0x560408b24ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:07:55.269 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:55.267+0000 7f40e9079740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560408b25080 0x560408b2d630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:07:55.269 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:55.267+0000 7f40e9079740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x560408b2dc00 0x560408b30050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:07:55.269 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:55.267+0000 7f40e9079740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x560408a882c0 con 0x560408b25080 2024-03-28T18:07:55.269 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:55.267+0000 7f40e9079740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x560408a30c90 con 0x560408b2dc00 2024-03-28T18:07:55.269 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:55.267+0000 7f40e9079740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x560408a84360 con 0x560408b246d0 2024-03-28T18:07:55.269 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:55.267+0000 7f40df400640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560408b25080 0x560408b2d630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:07:55.269 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:55.267+0000 7f40df400640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560408b25080 0x560408b2d630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:50234/0 (socket says 172.21.15.22:50234) 2024-03-28T18:07:55.269 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:55.267+0000 7f40df400640 1 -- 172.21.15.22:0/726567001 learned_addr learned my addr 172.21.15.22:0/726567001 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-28T18:07:55.269 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:55.267+0000 7f40dea00640 1 --2- 172.21.15.22:0/726567001 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x560408b2dc00 0x560408b30050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:07:55.269 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:55.267+0000 7f40dfe00640 1 --2- 172.21.15.22:0/726567001 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560408b246d0 0x560408b24ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:07:55.270 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:55.268+0000 7f40df400640 1 -- 172.21.15.22:0/726567001 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560408b246d0 msgr2=0x560408b24ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T18:07:55.270 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:55.268+0000 7f40df400640 1 --2- 172.21.15.22:0/726567001 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560408b246d0 0x560408b24ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:07:55.270 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:55.268+0000 7f40df400640 1 -- 172.21.15.22:0/726567001 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x560408b2dc00 msgr2=0x560408b30050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T18:07:55.270 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:55.268+0000 7f40df400640 1 --2- 172.21.15.22:0/726567001 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x560408b2dc00 0x560408b30050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:07:55.270 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:55.268+0000 7f40df400640 1 -- 172.21.15.22:0/726567001 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560408a55080 con 0x560408b25080 2024-03-28T18:07:55.270 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:55.268+0000 7f40dea00640 1 --2- 172.21.15.22:0/726567001 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x560408b2dc00 0x560408b30050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-28T18:07:55.270 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:55.268+0000 7f40df400640 1 --2- 172.21.15.22:0/726567001 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560408b25080 0x560408b2d630 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f40d00cc530 tx=0x7f40d00d71a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=74f2b244ff629726 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:07:55.270 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:55.268+0000 7f40d7e00640 1 --2- 172.21.15.22:0/35191753 >> [v2:172.21.15.22:6800/1357365082,v1:172.21.15.22:6801/1357365082] conn(0x7f40c40a2a00 0x7f40c40a4ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:07:55.270 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:55.268+0000 7f40d4c00640 1 -- 172.21.15.22:0/726567001 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40d00e2070 con 0x560408b25080 2024-03-28T18:07:55.270 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:55.268+0000 7f40d7e00640 1 -- 172.21.15.22:0/35191753 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 8411+0+0 (secure 0 0 0) 0x7f40d00cee90 con 0x7f40e00688a0 2024-03-28T18:07:55.270 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:55.268+0000 7f40d4c00640 1 -- 172.21.15.22:0/726567001 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f40d00045d0 con 0x560408b25080 2024-03-28T18:07:55.270 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:55.268+0000 7f40d4c00640 1 -- 172.21.15.22:0/726567001 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40d009ad50 con 0x560408b25080 2024-03-28T18:07:55.270 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:55.268+0000 7f40dfe00640 1 --2- 172.21.15.22:0/35191753 >> [v2:172.21.15.22:6800/1357365082,v1:172.21.15.22:6801/1357365082] conn(0x7f40c40a2a00 0x7f40c40a4ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:07:55.274 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:55.269+0000 7f40e9079740 1 -- 172.21.15.22:0/726567001 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560408b25080 msgr2=0x560408b2d630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:07:55.274 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:55.269+0000 7f40dfe00640 1 --2- 172.21.15.22:0/35191753 >> [v2:172.21.15.22:6800/1357365082,v1:172.21.15.22:6801/1357365082] conn(0x7f40c40a2a00 0x7f40c40a4ec0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f40d8005fd0 tx=0x7f40d8005850 comp rx=0 tx=0).ready entity=mgr.15522 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:07:55.274 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:55.269+0000 7f40e9079740 1 --2- 172.21.15.22:0/726567001 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560408b25080 0x560408b2d630 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f40d00cc530 tx=0x7f40d00d71a0 comp rx=0 tx=0).stop 2024-03-28T18:07:55.274 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:55.269+0000 7f40e9079740 1 -- 172.21.15.22:0/726567001 shutdown_connections 2024-03-28T18:07:55.274 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:55.269+0000 7f40e9079740 1 --2- 172.21.15.22:0/726567001 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x560408b2dc00 0x560408b30050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:07:55.274 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:55.269+0000 7f40e9079740 1 --2- 172.21.15.22:0/726567001 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560408b25080 0x560408b2d630 unknown :-1 s=CLOSED pgs=493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:07:55.274 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:55.269+0000 7f40e9079740 1 --2- 172.21.15.22:0/726567001 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560408b246d0 0x560408b24ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:07:55.274 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:55.269+0000 7f40e9079740 1 -- 172.21.15.22:0/726567001 >> 172.21.15.22:0/726567001 conn(0x560408b1eda0 msgr2=0x560408b1f180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:07:55.274 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:55.269+0000 7f40e9079740 1 -- 172.21.15.22:0/726567001 shutdown_connections 2024-03-28T18:07:55.275 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:55.269+0000 7f40e9079740 1 -- 172.21.15.22:0/726567001 wait complete. 2024-03-28T18:07:55.275 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:55.270+0000 7f40e9079740 1 -- start start 2024-03-28T18:07:55.275 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:55.270+0000 7f40e9079740 1 Processor -- start 2024-03-28T18:07:55.275 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:55.270+0000 7f40e9079740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x560408b246d0 0x560408b2d4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:07:55.275 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:55.270+0000 7f40e9079740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560408b25080 0x560408b2a860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:07:55.275 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:55.270+0000 7f40e9079740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560408b2dc00 0x560408b2adc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:07:55.275 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:55.270+0000 7f40e9079740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x560408b309f0 con 0x560408b25080 2024-03-28T18:07:55.275 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:55.270+0000 7f40e9079740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x560408a440d0 con 0x560408b246d0 2024-03-28T18:07:55.275 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:55.270+0000 7f40e9079740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x560408b30b70 con 0x560408b2dc00 2024-03-28T18:07:55.275 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:55.270+0000 7f40e9079740 10 client.0 _ll_register_callbacks cb 0x560408a48e80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-28T18:07:55.275 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:55.271+0000 7f40dea00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560408b25080 0x560408b2a860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:07:55.275 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:55.271+0000 7f40dea00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560408b25080 0x560408b2a860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:50246/0 (socket says 172.21.15.22:50246) 2024-03-28T18:07:55.275 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:55.271+0000 7f40dea00640 1 -- 172.21.15.22:0/2857937410 learned_addr learned my addr 172.21.15.22:0/2857937410 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-28T18:07:55.275 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:55.271+0000 7f40df400640 1 --2- 172.21.15.22:0/2857937410 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x560408b246d0 0x560408b2d4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:07:55.275 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:55.271+0000 7f40dfe00640 1 --2- 172.21.15.22:0/2857937410 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560408b2dc00 0x560408b2adc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:07:55.276 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:55.271+0000 7f40dea00640 1 -- 172.21.15.22:0/2857937410 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560408b2dc00 msgr2=0x560408b2adc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T18:07:55.276 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:55.271+0000 7f40dea00640 1 --2- 172.21.15.22:0/2857937410 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560408b2dc00 0x560408b2adc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:07:55.276 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:55.271+0000 7f40dea00640 1 -- 172.21.15.22:0/2857937410 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x560408b246d0 msgr2=0x560408b2d4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T18:07:55.276 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:55.271+0000 7f40dea00640 1 --2- 172.21.15.22:0/2857937410 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x560408b246d0 0x560408b2d4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:07:55.276 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:55.271+0000 7f40dea00640 1 -- 172.21.15.22:0/2857937410 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56040879c3f0 con 0x560408b25080 2024-03-28T18:07:55.276 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:55.271+0000 7f40dfe00640 1 --2- 172.21.15.22:0/2857937410 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560408b2dc00 0x560408b2adc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T18:07:55.276 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:55.271+0000 7f40dea00640 1 --2- 172.21.15.22:0/2857937410 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560408b25080 0x560408b2a860 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f40c80057c0 tx=0x7f40c8034670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9d727066929b9fd2 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:07:55.276 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:55.271+0000 7f40cd600640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-03-28T18:07:55.276 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:55.272+0000 7f40cd600640 1 -- 172.21.15.22:0/2857937410 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40c8044020 con 0x560408b25080 2024-03-28T18:07:55.276 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:55.272+0000 7f40cd600640 1 -- 172.21.15.22:0/2857937410 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 8411+0+0 (secure 0 0 0) 0x7f40c8004240 con 0x560408b25080 2024-03-28T18:07:55.276 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:55.272+0000 7f40e9079740 1 -- 172.21.15.22:0/2857937410 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x560408a95800 con 0x560408b25080 2024-03-28T18:07:55.276 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:55.272+0000 7f40cd600640 1 -- 172.21.15.22:0/2857937410 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f40c804f050 con 0x560408b25080 2024-03-28T18:07:55.276 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:55.272+0000 7f40e9079740 10 client.15702 fetch_fsmap learned FSMap version 27 2024-03-28T18:07:55.276 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:55.272+0000 7f40e9079740 1 -- 172.21.15.22:0/2857937410 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x5604088667d0 con 0x560408b25080 2024-03-28T18:07:55.276 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:55.272+0000 7f40cd600640 1 -- 172.21.15.22:0/2857937410 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x560408a95800 con 0x560408b25080 2024-03-28T18:07:55.276 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:55.272+0000 7f40cd600640 1 -- 172.21.15.22:0/2857937410 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 27) v1 ==== 7934+0+0 (secure 0 0 0) 0x7f40c802c770 con 0x560408b25080 2024-03-28T18:07:55.277 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:55.272+0000 7f40e9079740 10 client.15702 fetch_fsmap finished waiting for FSMap version 27 2024-03-28T18:07:55.277 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:55.272+0000 7f40e9079740 10 client.15702 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-03-28T18:07:55.277 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:55.272+0000 7f40e9079740 20 client.15702 populate_metadata read hostname 'smithi022' 2024-03-28T18:07:55.277 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:55.272+0000 7f40e9079740 1 --2- 172.21.15.22:0/2857937410 >> [v2:172.21.15.79:6836/1787903366,v1:172.21.15.79:6837/1787903366] conn(0x560408b3d8f0 0x560408b3fd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:07:55.277 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:55.272+0000 7f40e9079740 4 client.15702 mds_command: new command op to 34511 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-28T18:07:55.277 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:55.272+0000 7f40e9079740 1 -- 172.21.15.22:0/2857937410 --> [v2:172.21.15.79:6836/1787903366,v1:172.21.15.79:6837/1787903366] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x560408a8fb50 con 0x560408b3d8f0 2024-03-28T18:07:55.277 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:55.273+0000 7f40df400640 1 --2- 172.21.15.22:0/2857937410 >> [v2:172.21.15.79:6836/1787903366,v1:172.21.15.79:6837/1787903366] conn(0x560408b3d8f0 0x560408b3fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:07:55.277 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:55.273+0000 7f40df400640 1 --2- 172.21.15.22:0/2857937410 >> [v2:172.21.15.79:6836/1787903366,v1:172.21.15.79:6837/1787903366] conn(0x560408b3d8f0 0x560408b3fd50 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=34f998dc095f7d3b server_cookie=24e779731e0f9d65 in_seq=0 out_seq=0 2024-03-28T18:07:55.277 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:55.273+0000 7f40cd600640 10 client.15702 ms_handle_connect on v2:172.21.15.79:6836/1787903366 2024-03-28T18:07:55.278 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:55.275+0000 7f40cd600640 1 -- 172.21.15.22:0/2857937410 <== mds.1 v2:172.21.15.79:6836/1787903366 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x560408a8fb50 con 0x560408b3d8f0 2024-03-28T18:07:55.278 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:55.275+0000 7f40cd600640 10 client.15702 handle_command_reply: tid=0 2024-03-28T18:07:55.278 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:55.275+0000 7f40e9079740 1 client.15702 shutdown 2024-03-28T18:07:55.279 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:55.275+0000 7f40e9079740 1 -- 172.21.15.22:0/2857937410 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560408b25080 msgr2=0x560408b2a860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:07:55.279 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:55.275+0000 7f40e9079740 1 --2- 172.21.15.22:0/2857937410 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560408b25080 0x560408b2a860 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f40c80057c0 tx=0x7f40c8034670 comp rx=0 tx=0).stop 2024-03-28T18:07:55.279 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:55.275+0000 7f40e9079740 1 -- 172.21.15.22:0/2857937410 shutdown_connections 2024-03-28T18:07:55.279 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:55.275+0000 7f40e9079740 1 --2- 172.21.15.22:0/2857937410 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560408b2dc00 0x560408b2adc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:07:55.279 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:55.275+0000 7f40e9079740 1 --2- 172.21.15.22:0/2857937410 >> [v2:172.21.15.79:6836/1787903366,v1:172.21.15.79:6837/1787903366] conn(0x560408b3d8f0 0x560408b3fd50 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:07:55.279 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:55.275+0000 7f40e9079740 1 --2- 172.21.15.22:0/2857937410 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560408b25080 0x560408b2a860 unknown :-1 s=CLOSED pgs=494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:07:55.279 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:55.275+0000 7f40e9079740 1 --2- 172.21.15.22:0/2857937410 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x560408b246d0 0x560408b2d4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:07:55.279 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:55.275+0000 7f40cd600640 0 client.15702 ms_handle_reset on v2:172.21.15.79:6836/1787903366 2024-03-28T18:07:55.279 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:55.275+0000 7f40e9079740 1 -- 172.21.15.22:0/2857937410 >> 172.21.15.22:0/2857937410 conn(0x560408b1eda0 msgr2=0x560408b20af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:07:55.279 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:55.275+0000 7f40e9079740 1 -- 172.21.15.22:0/2857937410 shutdown_connections 2024-03-28T18:07:55.279 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:55.275+0000 7f40e9079740 1 -- 172.21.15.22:0/2857937410 wait complete. 2024-03-28T18:07:55.279 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:55.275+0000 7f40e9079740 20 client.15702 trim_cache size 0 max 16384 2024-03-28T18:07:55.290 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:55.289+0000 7f40e9079740 1 Processor -- start 2024-03-28T18:07:55.290 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:55.289+0000 7f40e9079740 1 -- start start 2024-03-28T18:07:55.290 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:55.289+0000 7f40e9079740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560408b246d0 0x560408b2cf90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:07:55.290 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:55.289+0000 7f40e9079740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560408b25080 0x560408b2d4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:07:55.290 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:55.289+0000 7f40e9079740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x560408b2dc00 0x560408b27ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:07:55.290 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:55.289+0000 7f40e9079740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40e01af250 con 0x560408b25080 2024-03-28T18:07:55.290 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:55.289+0000 7f40e9079740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40e01b0bd0 con 0x560408b2dc00 2024-03-28T18:07:55.290 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:55.289+0000 7f40e9079740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40e01af8b0 con 0x560408b246d0 2024-03-28T18:07:55.290 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:55.289+0000 7f40dea00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560408b25080 0x560408b2d4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:07:55.290 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:55.289+0000 7f40dea00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560408b25080 0x560408b2d4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:50262/0 (socket says 172.21.15.22:50262) 2024-03-28T18:07:55.290 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:55.289+0000 7f40dea00640 1 -- 172.21.15.22:0/2495824714 learned_addr learned my addr 172.21.15.22:0/2495824714 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-28T18:07:55.291 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:55.289+0000 7f40dfe00640 1 --2- 172.21.15.22:0/2495824714 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x560408b2dc00 0x560408b27ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:07:55.293 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:55.290+0000 7f40df400640 1 --2- 172.21.15.22:0/2495824714 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560408b246d0 0x560408b2cf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:07:55.293 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:55.290+0000 7f40dea00640 1 -- 172.21.15.22:0/2495824714 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560408b246d0 msgr2=0x560408b2cf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T18:07:55.293 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:55.290+0000 7f40dea00640 1 --2- 172.21.15.22:0/2495824714 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560408b246d0 0x560408b2cf90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:07:55.293 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:55.290+0000 7f40dea00640 1 -- 172.21.15.22:0/2495824714 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x560408b2dc00 msgr2=0x560408b27ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T18:07:55.293 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:55.290+0000 7f40dea00640 1 --2- 172.21.15.22:0/2495824714 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x560408b2dc00 0x560408b27ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:07:55.294 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:55.290+0000 7f40dea00640 1 -- 172.21.15.22:0/2495824714 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560408b2fff0 con 0x560408b25080 2024-03-28T18:07:55.294 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:55.290+0000 7f40dea00640 1 --2- 172.21.15.22:0/2495824714 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560408b25080 0x560408b2d4d0 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f40c802cbc0 tx=0x7f40c8034de0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a01afb4096615390 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:07:55.294 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:55.290+0000 7f40cfe00640 1 -- 172.21.15.22:0/2495824714 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40c8058070 con 0x560408b25080 2024-03-28T18:07:55.294 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:55.290+0000 7f40cfe00640 1 -- 172.21.15.22:0/2495824714 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f40c803a040 con 0x560408b25080 2024-03-28T18:07:55.294 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:55.290+0000 7f40cfe00640 1 -- 172.21.15.22:0/2495824714 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40c8044040 con 0x560408b25080 2024-03-28T18:07:55.294 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:55.291+0000 7f40e9079740 1 -- 172.21.15.22:0/2495824714 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560408b25080 msgr2=0x560408b2d4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:07:55.294 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:55.291+0000 7f40e9079740 1 --2- 172.21.15.22:0/2495824714 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560408b25080 0x560408b2d4d0 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f40c802cbc0 tx=0x7f40c8034de0 comp rx=0 tx=0).stop 2024-03-28T18:07:55.294 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:55.291+0000 7f40e9079740 1 -- 172.21.15.22:0/2495824714 shutdown_connections 2024-03-28T18:07:55.294 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:55.291+0000 7f40e9079740 1 --2- 172.21.15.22:0/2495824714 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x560408b2dc00 0x560408b27ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:07:55.294 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:55.291+0000 7f40e9079740 1 --2- 172.21.15.22:0/2495824714 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560408b25080 0x560408b2d4d0 unknown :-1 s=CLOSED pgs=495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:07:55.294 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:55.291+0000 7f40e9079740 1 --2- 172.21.15.22:0/2495824714 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560408b246d0 0x560408b2cf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:07:55.294 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:55.291+0000 7f40e9079740 1 -- 172.21.15.22:0/2495824714 >> 172.21.15.22:0/2495824714 conn(0x560408b1eda0 msgr2=0x560408b1deb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:07:55.294 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:55.291+0000 7f40e9079740 1 -- 172.21.15.22:0/2495824714 shutdown_connections 2024-03-28T18:07:55.294 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:55.291+0000 7f40e9079740 1 -- 172.21.15.22:0/2495824714 wait complete. 2024-03-28T18:07:55.294 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:55.291+0000 7f40e9079740 1 -- start start 2024-03-28T18:07:55.295 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:55.291+0000 7f40e9079740 1 Processor -- start 2024-03-28T18:07:55.295 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:55.292+0000 7f40e9079740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560408b246d0 0x560408b31580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:07:55.295 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:55.292+0000 7f40e9079740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x560408b25080 0x560408b31ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:07:55.295 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:55.292+0000 7f40e9079740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560408b2dc00 0x560408b981e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:07:55.295 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:55.292+0000 7f40e9079740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40e01b0570 con 0x560408b2dc00 2024-03-28T18:07:55.295 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:55.292+0000 7f40e9079740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40e01aff10 con 0x560408b25080 2024-03-28T18:07:55.295 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:55.292+0000 7f40e9079740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40e01ad410 con 0x560408b246d0 2024-03-28T18:07:55.295 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:55.292+0000 7f40e9079740 10 client.0 _ll_register_callbacks cb 0x7f40c80096b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-28T18:07:55.295 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:55.292+0000 7f40dfe00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560408b2dc00 0x560408b981e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:07:55.295 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:55.292+0000 7f40dfe00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560408b2dc00 0x560408b981e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:50266/0 (socket says 172.21.15.22:50266) 2024-03-28T18:07:55.295 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:55.292+0000 7f40dfe00640 1 -- 172.21.15.22:0/3405527139 learned_addr learned my addr 172.21.15.22:0/3405527139 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-28T18:07:55.295 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:55.292+0000 7f40dea00640 1 --2- 172.21.15.22:0/3405527139 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x560408b25080 0x560408b31ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:07:55.295 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:55.292+0000 7f40df400640 1 --2- 172.21.15.22:0/3405527139 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560408b246d0 0x560408b31580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:07:55.295 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:55.292+0000 7f40dfe00640 1 -- 172.21.15.22:0/3405527139 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560408b246d0 msgr2=0x560408b31580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T18:07:55.295 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:55.292+0000 7f40dfe00640 1 --2- 172.21.15.22:0/3405527139 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560408b246d0 0x560408b31580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:07:55.296 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:55.292+0000 7f40dfe00640 1 -- 172.21.15.22:0/3405527139 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x560408b25080 msgr2=0x560408b31ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T18:07:55.296 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:55.292+0000 7f40dfe00640 1 --2- 172.21.15.22:0/3405527139 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x560408b25080 0x560408b31ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:07:55.296 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:55.293+0000 7f40dfe00640 1 -- 172.21.15.22:0/3405527139 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x560408b98720 con 0x560408b2dc00 2024-03-28T18:07:55.296 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:55.293+0000 7f40dea00640 1 --2- 172.21.15.22:0/3405527139 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x560408b25080 0x560408b31ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T18:07:55.296 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:55.293+0000 7f40dfe00640 1 --2- 172.21.15.22:0/3405527139 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560408b2dc00 0x560408b981e0 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f40d8010f40 tx=0x7f40d8004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c1fce50c7c522db server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:07:55.296 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:55.293+0000 7f40cd600640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-03-28T18:07:55.298 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:55.293+0000 7f40cd600640 1 -- 172.21.15.22:0/3405527139 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40d8020840 con 0x560408b2dc00 2024-03-28T18:07:55.298 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:55.293+0000 7f40cd600640 1 -- 172.21.15.22:0/3405527139 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 8411+0+0 (secure 0 0 0) 0x7f40d80209e0 con 0x560408b2dc00 2024-03-28T18:07:55.298 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:55.293+0000 7f40e9079740 1 -- 172.21.15.22:0/3405527139 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f40c803a040 con 0x560408b2dc00 2024-03-28T18:07:55.298 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:55.293+0000 7f40cd600640 1 -- 172.21.15.22:0/3405527139 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f40d8020be0 con 0x560408b2dc00 2024-03-28T18:07:55.298 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:55.293+0000 7f40e9079740 10 client.15708 fetch_fsmap learned FSMap version 27 2024-03-28T18:07:55.298 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:55.293+0000 7f40e9079740 1 -- 172.21.15.22:0/3405527139 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x560408b998f0 con 0x560408b2dc00 2024-03-28T18:07:55.298 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:55.294+0000 7f40cd600640 1 -- 172.21.15.22:0/3405527139 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f40c803a040 con 0x560408b2dc00 2024-03-28T18:07:55.298 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:55.294+0000 7f40cd600640 1 -- 172.21.15.22:0/3405527139 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 27) v1 ==== 7934+0+0 (secure 0 0 0) 0x7f40d8031020 con 0x560408b2dc00 2024-03-28T18:07:55.298 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:55.294+0000 7f40e9079740 10 client.15708 fetch_fsmap finished waiting for FSMap version 27 2024-03-28T18:07:55.299 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:55.294+0000 7f40e9079740 10 client.15708 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-03-28T18:07:55.299 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:55.294+0000 7f40e9079740 20 client.15708 populate_metadata read hostname 'smithi022' 2024-03-28T18:07:55.299 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:55.294+0000 7f40e9079740 1 --2- 172.21.15.22:0/3405527139 >> [v2:172.21.15.79:6836/1787903366,v1:172.21.15.79:6837/1787903366] conn(0x560408b9a080 0x560408b9c4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:07:55.299 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:55.294+0000 7f40e9079740 4 client.15708 mds_command: new command op to 34511 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-28T18:07:55.299 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:55.294+0000 7f40e9079740 1 -- 172.21.15.22:0/3405527139 --> [v2:172.21.15.79:6836/1787903366,v1:172.21.15.79:6837/1787903366] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x560408a0d500 con 0x560408b9a080 2024-03-28T18:07:55.299 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:55.294+0000 7f40df400640 1 --2- 172.21.15.22:0/3405527139 >> [v2:172.21.15.79:6836/1787903366,v1:172.21.15.79:6837/1787903366] conn(0x560408b9a080 0x560408b9c4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:07:55.299 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:55.295+0000 7f40df400640 1 --2- 172.21.15.22:0/3405527139 >> [v2:172.21.15.79:6836/1787903366,v1:172.21.15.79:6837/1787903366] conn(0x560408b9a080 0x560408b9c4a0 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=2849465ba42fd592 server_cookie=983ae6eda534f27e in_seq=0 out_seq=0 2024-03-28T18:07:55.299 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:55.295+0000 7f40cd600640 10 client.15708 ms_handle_connect on v2:172.21.15.79:6836/1787903366 2024-03-28T18:07:55.299 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:55.296+0000 7f40cd600640 1 -- 172.21.15.22:0/3405527139 <== mds.1 v2:172.21.15.79:6836/1787903366 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x560408a0d500 con 0x560408b9a080 2024-03-28T18:07:55.299 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:55.296+0000 7f40cd600640 10 client.15708 handle_command_reply: tid=0 2024-03-28T18:07:55.299 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:55.296+0000 7f40e9079740 1 client.15708 shutdown 2024-03-28T18:07:55.299 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:55.296+0000 7f40e9079740 1 -- 172.21.15.22:0/3405527139 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560408b2dc00 msgr2=0x560408b981e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:07:55.299 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:55.296+0000 7f40e9079740 1 --2- 172.21.15.22:0/3405527139 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560408b2dc00 0x560408b981e0 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f40d8010f40 tx=0x7f40d8004730 comp rx=0 tx=0).stop 2024-03-28T18:07:55.299 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:55.296+0000 7f40e9079740 1 -- 172.21.15.22:0/3405527139 shutdown_connections 2024-03-28T18:07:55.299 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:55.296+0000 7f40e9079740 1 --2- 172.21.15.22:0/3405527139 >> [v2:172.21.15.79:6836/1787903366,v1:172.21.15.79:6837/1787903366] conn(0x560408b9a080 0x560408b9c4a0 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:07:55.299 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:55.296+0000 7f40e9079740 1 --2- 172.21.15.22:0/3405527139 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560408b2dc00 0x560408b981e0 unknown :-1 s=CLOSED pgs=496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:07:55.300 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:55.296+0000 7f40e9079740 1 --2- 172.21.15.22:0/3405527139 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x560408b25080 0x560408b31ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:07:55.300 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:55.296+0000 7f40e9079740 1 --2- 172.21.15.22:0/3405527139 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560408b246d0 0x560408b31580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:07:55.300 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:55.296+0000 7f40e9079740 1 -- 172.21.15.22:0/3405527139 >> 172.21.15.22:0/3405527139 conn(0x560408b1eda0 msgr2=0x560408b25460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:07:55.300 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:55.296+0000 7f40cd600640 0 client.15708 ms_handle_reset on v2:172.21.15.79:6836/1787903366 2024-03-28T18:07:55.300 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:55.296+0000 7f40e9079740 1 -- 172.21.15.22:0/3405527139 shutdown_connections 2024-03-28T18:07:55.300 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:55.296+0000 7f40e9079740 1 -- 172.21.15.22:0/3405527139 wait complete. 2024-03-28T18:07:55.300 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:55.296+0000 7f40e9079740 20 client.15708 trim_cache size 0 max 16384 2024-03-28T18:07:55.300 INFO:teuthology.orchestra.run.smithi022.stdout:[] 2024-03-28T18:07:55.301 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:55.297+0000 7f40e6400640 1 -- 172.21.15.22:0/35191753 >> [v2:172.21.15.22:6800/1357365082,v1:172.21.15.22:6801/1357365082] conn(0x7f40c40a2a00 msgr2=0x7f40c40a4ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:07:55.301 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:55.297+0000 7f40e6400640 1 --2- 172.21.15.22:0/35191753 >> [v2:172.21.15.22:6800/1357365082,v1:172.21.15.22:6801/1357365082] conn(0x7f40c40a2a00 0x7f40c40a4ec0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f40d8005fd0 tx=0x7f40d8005850 comp rx=0 tx=0).stop 2024-03-28T18:07:55.301 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:55.297+0000 7f40e6400640 1 -- 172.21.15.22:0/35191753 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f40e00688a0 msgr2=0x7f40e019a110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:07:55.301 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:55.297+0000 7f40e6400640 1 --2- 172.21.15.22:0/35191753 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f40e00688a0 0x7f40e019a110 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f40d000c420 tx=0x7f40d0009f90 comp rx=0 tx=0).stop 2024-03-28T18:07:55.301 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:55.297+0000 7f40e6400640 1 -- 172.21.15.22:0/35191753 shutdown_connections 2024-03-28T18:07:55.301 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:55.297+0000 7f40e6400640 1 --2- 172.21.15.22:0/35191753 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f40e01076f0 0x7f40e019e9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:07:55.301 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:55.297+0000 7f40e6400640 1 --2- 172.21.15.22:0/35191753 >> [v2:172.21.15.22:6800/1357365082,v1:172.21.15.22:6801/1357365082] conn(0x7f40c40a2a00 0x7f40c40a4ec0 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:07:55.301 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:55.297+0000 7f40e6400640 1 --2- 172.21.15.22:0/35191753 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f40e0069260 0x7f40e019a650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:07:55.301 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:55.297+0000 7f40e6400640 1 --2- 172.21.15.22:0/35191753 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f40e00688a0 0x7f40e019a110 unknown :-1 s=CLOSED pgs=492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:07:55.301 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:55.297+0000 7f40e6400640 1 -- 172.21.15.22:0/35191753 >> 172.21.15.22:0/35191753 conn(0x7f40e00776a0 msgr2=0x7f40e00fe390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:07:55.301 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:55.297+0000 7f40e6400640 1 -- 172.21.15.22:0/35191753 shutdown_connections 2024-03-28T18:07:55.301 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:55.297+0000 7f40e6400640 1 -- 172.21.15.22:0/35191753 wait complete. 2024-03-28T18:07:55.314 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-28T18:07:55.314 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-03-28T18:07:55.524 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:55.523+0000 7ff1afa00640 1 -- 172.21.15.22:0/673316942 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ff1a8104200 msgr2=0x7ff1a8110710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:07:55.524 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:55.523+0000 7ff1afa00640 1 --2- 172.21.15.22:0/673316942 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ff1a8104200 0x7ff1a8110710 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7ff1a0009ca0 tx=0x7ff1a002c100 comp rx=0 tx=0).stop 2024-03-28T18:07:55.524 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:55.523+0000 7ff1afa00640 1 -- 172.21.15.22:0/673316942 shutdown_connections 2024-03-28T18:07:55.524 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:55.523+0000 7ff1afa00640 1 --2- 172.21.15.22:0/673316942 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ff1a8104200 0x7ff1a8110710 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:07:55.524 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:55.523+0000 7ff1afa00640 1 --2- 172.21.15.22:0/673316942 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff1a8103840 0x7ff1a8103cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:07:55.524 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:55.523+0000 7ff1afa00640 1 --2- 172.21.15.22:0/673316942 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff1a8109850 0x7ff1a8109c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:07:55.525 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:55.523+0000 7ff1afa00640 1 -- 172.21.15.22:0/673316942 >> 172.21.15.22:0/673316942 conn(0x7ff1a80ff310 msgr2=0x7ff1a8101730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:07:55.525 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:55.524+0000 7ff1afa00640 1 -- 172.21.15.22:0/673316942 shutdown_connections 2024-03-28T18:07:55.525 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:55.524+0000 7ff1afa00640 1 -- 172.21.15.22:0/673316942 wait complete. 2024-03-28T18:07:55.527 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:55.524+0000 7ff1afa00640 1 Processor -- start 2024-03-28T18:07:55.527 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:55.525+0000 7ff1afa00640 1 -- start start 2024-03-28T18:07:55.527 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:55.525+0000 7ff1afa00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff1a8103840 0x7ff1a819a0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:07:55.527 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:55.525+0000 7ff1afa00640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ff1a8104200 0x7ff1a819a610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:07:55.527 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:55.525+0000 7ff1afa00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff1a8109850 0x7ff1a819e990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:07:55.527 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:55.525+0000 7ff1afa00640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1a8113c30 con 0x7ff1a8103840 2024-03-28T18:07:55.527 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:55.525+0000 7ff1afa00640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1a8113ab0 con 0x7ff1a8104200 2024-03-28T18:07:55.527 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:55.525+0000 7ff1afa00640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1a8113db0 con 0x7ff1a8109850 2024-03-28T18:07:55.527 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:55.525+0000 7ff1ad000640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff1a8103840 0x7ff1a819a0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:07:55.527 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:55.525+0000 7ff1ad000640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff1a8103840 0x7ff1a819a0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:50286/0 (socket says 172.21.15.22:50286) 2024-03-28T18:07:55.527 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:55.525+0000 7ff1ad000640 1 -- 172.21.15.22:0/212249984 learned_addr learned my addr 172.21.15.22:0/212249984 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-28T18:07:55.528 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:55.525+0000 7ff1a7e00640 1 --2- 172.21.15.22:0/212249984 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ff1a8104200 0x7ff1a819a610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:07:55.528 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:55.525+0000 7ff1ada00640 1 --2- 172.21.15.22:0/212249984 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff1a8109850 0x7ff1a819e990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:07:55.528 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:55.526+0000 7ff1ad000640 1 -- 172.21.15.22:0/212249984 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff1a8109850 msgr2=0x7ff1a819e990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:07:55.528 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:55.526+0000 7ff1ad000640 1 --2- 172.21.15.22:0/212249984 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff1a8109850 0x7ff1a819e990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:07:55.528 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:55.526+0000 7ff1ad000640 1 -- 172.21.15.22:0/212249984 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ff1a8104200 msgr2=0x7ff1a819a610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:07:55.528 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:55.526+0000 7ff1ad000640 1 --2- 172.21.15.22:0/212249984 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ff1a8104200 0x7ff1a819a610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:07:55.528 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:55.526+0000 7ff1ad000640 1 -- 172.21.15.22:0/212249984 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff1a819f0c0 con 0x7ff1a8103840 2024-03-28T18:07:55.528 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:55.526+0000 7ff1ada00640 1 --2- 172.21.15.22:0/212249984 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff1a8109850 0x7ff1a819e990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T18:07:55.528 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:55.526+0000 7ff1a7e00640 1 --2- 172.21.15.22:0/212249984 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ff1a8104200 0x7ff1a819a610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-28T18:07:55.528 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:55.526+0000 7ff1ad000640 1 --2- 172.21.15.22:0/212249984 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff1a8103840 0x7ff1a819a0d0 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7ff19c00e9f0 tx=0x7ff19c00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:07:55.528 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:55.526+0000 7ff1a5600640 1 -- 172.21.15.22:0/212249984 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff19c00cd80 con 0x7ff1a8103840 2024-03-28T18:07:55.528 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:55.526+0000 7ff1afa00640 1 -- 172.21.15.22:0/212249984 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff1a819f3b0 con 0x7ff1a8103840 2024-03-28T18:07:55.528 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:55.526+0000 7ff1a5600640 1 -- 172.21.15.22:0/212249984 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7ff19c010900 con 0x7ff1a8103840 2024-03-28T18:07:55.528 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:55.526+0000 7ff1a5600640 1 -- 172.21.15.22:0/212249984 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff19c0189e0 con 0x7ff1a8103840 2024-03-28T18:07:55.528 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:55.526+0000 7ff1afa00640 1 -- 172.21.15.22:0/212249984 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff1a81a6c40 con 0x7ff1a8103840 2024-03-28T18:07:55.530 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:55.529+0000 7ff1b27ec740 1 Processor -- start 2024-03-28T18:07:55.530 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:55.529+0000 7ff1b27ec740 1 -- start start 2024-03-28T18:07:55.532 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:55.530+0000 7ff1b27ec740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56224466e6d0 0x56224466eab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:07:55.532 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:55.530+0000 7ff1b27ec740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56224466f080 0x562244677630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:07:55.532 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:55.530+0000 7ff1a5600640 1 -- 172.21.15.22:0/212249984 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 24) v1 ==== 137466+0+0 (secure 0 0 0) 0x7ff19c010ab0 con 0x7ff1a8103840 2024-03-28T18:07:55.532 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:55.530+0000 7ff1b27ec740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562244677c00 0x56224467a050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:07:55.532 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:55.530+0000 7ff1b27ec740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x5622445d22c0 con 0x562244677c00 2024-03-28T18:07:55.532 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:55.530+0000 7ff1b27ec740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x56224457ac90 con 0x56224466f080 2024-03-28T18:07:55.532 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:55.530+0000 7ff1b27ec740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x5622445ce360 con 0x56224466e6d0 2024-03-28T18:07:55.532 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:55.530+0000 7ff1a7e00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562244677c00 0x56224467a050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:07:55.532 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:55.530+0000 7ff1ad000640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56224466f080 0x562244677630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:07:55.532 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:55.530+0000 7ff1a7e00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562244677c00 0x56224467a050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:50296/0 (socket says 172.21.15.22:50296) 2024-03-28T18:07:55.532 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:55.530+0000 7ff1ad000640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56224466f080 0x562244677630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.22:36966/0 (socket says 172.21.15.22:36966) 2024-03-28T18:07:55.532 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:55.530+0000 7ff1a7e00640 1 -- 172.21.15.22:0/2452025825 learned_addr learned my addr 172.21.15.22:0/2452025825 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-28T18:07:55.532 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:55.530+0000 7ff1ada00640 1 --2- 172.21.15.22:0/2452025825 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56224466e6d0 0x56224466eab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:07:55.533 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:55.531+0000 7ff1a7e00640 1 -- 172.21.15.22:0/2452025825 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56224466e6d0 msgr2=0x56224466eab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T18:07:55.533 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:55.531+0000 7ff1a7e00640 1 --2- 172.21.15.22:0/2452025825 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56224466e6d0 0x56224466eab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:07:55.533 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:55.531+0000 7ff1a7e00640 1 -- 172.21.15.22:0/2452025825 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56224466f080 msgr2=0x562244677630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T18:07:55.533 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:55.531+0000 7ff1a7e00640 1 --2- 172.21.15.22:0/2452025825 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56224466f080 0x562244677630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:07:55.533 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:55.531+0000 7ff1a7e00640 1 -- 172.21.15.22:0/2452025825 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56224459f080 con 0x562244677c00 2024-03-28T18:07:55.533 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:55.531+0000 7ff1ada00640 1 --2- 172.21.15.22:0/2452025825 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56224466e6d0 0x56224466eab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-28T18:07:55.533 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:55.531+0000 7ff1ad000640 1 --2- 172.21.15.22:0/2452025825 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56224466f080 0x562244677630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-28T18:07:55.533 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:55.531+0000 7ff1a7e00640 1 --2- 172.21.15.22:0/2452025825 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562244677c00 0x56224467a050 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7ff194010450 tx=0x7ff194010910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8baa1d01ec06c362 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:07:55.533 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:55.531+0000 7ff19a000640 1 -- 172.21.15.22:0/2452025825 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff194011760 con 0x562244677c00 2024-03-28T18:07:55.533 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:55.531+0000 7ff19a000640 1 -- 172.21.15.22:0/2452025825 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7ff194011dd0 con 0x562244677c00 2024-03-28T18:07:55.533 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:55.531+0000 7ff19a000640 1 -- 172.21.15.22:0/2452025825 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff19401a380 con 0x562244677c00 2024-03-28T18:07:55.537 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:55.531+0000 7ff1a5600640 1 --2- 172.21.15.22:0/212249984 >> [v2:172.21.15.22:6800/1357365082,v1:172.21.15.22:6801/1357365082] conn(0x7ff1900a2a00 0x7ff1900a4ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:07:55.537 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:55.532+0000 7ff1a5600640 1 -- 172.21.15.22:0/212249984 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 8411+0+0 (secure 0 0 0) 0x7ff19c0cf360 con 0x7ff1a8103840 2024-03-28T18:07:55.537 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:55.532+0000 7ff1ada00640 1 --2- 172.21.15.22:0/212249984 >> [v2:172.21.15.22:6800/1357365082,v1:172.21.15.22:6801/1357365082] conn(0x7ff1900a2a00 0x7ff1900a4ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:07:55.537 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:55.532+0000 7ff1b27ec740 1 -- 172.21.15.22:0/2452025825 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562244677c00 msgr2=0x56224467a050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:07:55.537 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:55.532+0000 7ff1b27ec740 1 --2- 172.21.15.22:0/2452025825 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562244677c00 0x56224467a050 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7ff194010450 tx=0x7ff194010910 comp rx=0 tx=0).stop 2024-03-28T18:07:55.537 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:55.532+0000 7ff1b27ec740 1 -- 172.21.15.22:0/2452025825 shutdown_connections 2024-03-28T18:07:55.537 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:55.532+0000 7ff1b27ec740 1 --2- 172.21.15.22:0/2452025825 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562244677c00 0x56224467a050 unknown :-1 s=CLOSED pgs=498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:07:55.537 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:55.532+0000 7ff1b27ec740 1 --2- 172.21.15.22:0/2452025825 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56224466f080 0x562244677630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:07:55.538 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:55.532+0000 7ff1b27ec740 1 --2- 172.21.15.22:0/2452025825 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56224466e6d0 0x56224466eab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:07:55.538 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:55.532+0000 7ff1b27ec740 1 -- 172.21.15.22:0/2452025825 >> 172.21.15.22:0/2452025825 conn(0x562244668da0 msgr2=0x562244669180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:07:55.538 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:55.532+0000 7ff1b27ec740 1 -- 172.21.15.22:0/2452025825 shutdown_connections 2024-03-28T18:07:55.538 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:55.532+0000 7ff1b27ec740 1 -- 172.21.15.22:0/2452025825 wait complete. 2024-03-28T18:07:55.538 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:55.533+0000 7ff1ada00640 1 --2- 172.21.15.22:0/212249984 >> [v2:172.21.15.22:6800/1357365082,v1:172.21.15.22:6801/1357365082] conn(0x7ff1900a2a00 0x7ff1900a4ec0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7ff1a002c710 tx=0x7ff1a0004a40 comp rx=0 tx=0).ready entity=mgr.15522 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:07:55.538 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:55.533+0000 7ff1b27ec740 1 -- start start 2024-03-28T18:07:55.538 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:55.533+0000 7ff1b27ec740 1 Processor -- start 2024-03-28T18:07:55.538 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:55.533+0000 7ff1b27ec740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56224466e6d0 0x5622446774d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:07:55.538 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:55.533+0000 7ff1b27ec740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56224466f080 0x562244674860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:07:55.538 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:55.533+0000 7ff1b27ec740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562244677c00 0x562244674dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:07:55.538 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:55.533+0000 7ff1b27ec740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x56224467ab70 con 0x56224466e6d0 2024-03-28T18:07:55.538 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:55.534+0000 7ff1b27ec740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x56224458e0d0 con 0x56224466f080 2024-03-28T18:07:55.538 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:55.534+0000 7ff1b27ec740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x56224467acf0 con 0x562244677c00 2024-03-28T18:07:55.538 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:55.534+0000 7ff1b27ec740 10 client.0 _ll_register_callbacks cb 0x562244592e80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-28T18:07:55.538 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:55.534+0000 7ff1ad000640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56224466e6d0 0x5622446774d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:07:55.539 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:55.534+0000 7ff1ad000640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56224466e6d0 0x5622446774d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:50298/0 (socket says 172.21.15.22:50298) 2024-03-28T18:07:55.539 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:55.534+0000 7ff1a7e00640 1 --2- 172.21.15.22:0/4269424024 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56224466f080 0x562244674860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:07:55.539 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:55.534+0000 7ff1ad000640 1 -- 172.21.15.22:0/4269424024 learned_addr learned my addr 172.21.15.22:0/4269424024 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-28T18:07:55.539 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:55.534+0000 7ff1ada00640 1 --2- 172.21.15.22:0/4269424024 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562244677c00 0x562244674dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:07:55.539 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:55.534+0000 7ff1ad000640 1 -- 172.21.15.22:0/4269424024 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562244677c00 msgr2=0x562244674dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T18:07:55.539 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:55.534+0000 7ff1ad000640 1 --2- 172.21.15.22:0/4269424024 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562244677c00 0x562244674dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:07:55.539 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:55.534+0000 7ff1ad000640 1 -- 172.21.15.22:0/4269424024 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56224466f080 msgr2=0x562244674860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T18:07:55.539 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:55.534+0000 7ff1ad000640 1 --2- 172.21.15.22:0/4269424024 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56224466f080 0x562244674860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:07:55.539 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:55.534+0000 7ff1ad000640 1 -- 172.21.15.22:0/4269424024 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5622442e63f0 con 0x56224466e6d0 2024-03-28T18:07:55.539 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:55.534+0000 7ff1ada00640 1 --2- 172.21.15.22:0/4269424024 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562244677c00 0x562244674dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T18:07:55.539 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:55.534+0000 7ff1a7e00640 1 --2- 172.21.15.22:0/4269424024 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56224466f080 0x562244674860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-28T18:07:55.539 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:55.534+0000 7ff1ad000640 1 --2- 172.21.15.22:0/4269424024 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56224466e6d0 0x5622446774d0 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7ff19c00c6a0 tx=0x7ff19c01f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e8ce7724f2acf2d server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:07:55.539 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:55.534+0000 7ff18ea00640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-03-28T18:07:55.539 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:55.535+0000 7ff18ea00640 1 -- 172.21.15.22:0/4269424024 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff19c09b630 con 0x56224466e6d0 2024-03-28T18:07:55.539 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:55.535+0000 7ff18ea00640 1 -- 172.21.15.22:0/4269424024 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 8411+0+0 (secure 0 0 0) 0x7ff19c0cceb0 con 0x56224466e6d0 2024-03-28T18:07:55.540 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:55.535+0000 7ff1b27ec740 1 -- 172.21.15.22:0/4269424024 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7ff194011dd0 con 0x56224466e6d0 2024-03-28T18:07:55.540 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:55.535+0000 7ff18ea00640 1 -- 172.21.15.22:0/4269424024 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff19c09b1c0 con 0x56224466e6d0 2024-03-28T18:07:55.540 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:55.535+0000 7ff1b27ec740 10 client.15720 fetch_fsmap learned FSMap version 27 2024-03-28T18:07:55.540 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:55.535+0000 7ff1b27ec740 1 -- 172.21.15.22:0/4269424024 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x5622443b07d0 con 0x56224466e6d0 2024-03-28T18:07:55.540 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:55.535+0000 7ff18ea00640 1 -- 172.21.15.22:0/4269424024 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7ff194011dd0 con 0x56224466e6d0 2024-03-28T18:07:55.540 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:55.535+0000 7ff18ea00640 1 -- 172.21.15.22:0/4269424024 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 27) v1 ==== 7934+0+0 (secure 0 0 0) 0x7ff19c09cc20 con 0x56224466e6d0 2024-03-28T18:07:55.540 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:55.535+0000 7ff1b27ec740 10 client.15720 fetch_fsmap finished waiting for FSMap version 27 2024-03-28T18:07:55.540 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:55.535+0000 7ff1b27ec740 10 client.15720 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-28T18:07:55.540 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:55.535+0000 7ff1b27ec740 20 client.15720 populate_metadata read hostname 'smithi022' 2024-03-28T18:07:55.540 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:55.536+0000 7ff1b27ec740 1 --2- 172.21.15.22:0/4269424024 >> [v2:172.21.15.79:6838/2441542443,v1:172.21.15.79:6839/2441542443] conn(0x562244687990 0x562244689da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:07:55.540 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:55.536+0000 7ff1b27ec740 4 client.15720 mds_command: new command op to 34553 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-28T18:07:55.540 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:55.536+0000 7ff1b27ec740 1 -- 172.21.15.22:0/4269424024 --> [v2:172.21.15.79:6838/2441542443,v1:172.21.15.79:6839/2441542443] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5622445df800 con 0x562244687990 2024-03-28T18:07:55.540 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:55.536+0000 7ff1a7e00640 1 --2- 172.21.15.22:0/4269424024 >> [v2:172.21.15.79:6838/2441542443,v1:172.21.15.79:6839/2441542443] conn(0x562244687990 0x562244689da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:07:55.540 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:55.537+0000 7ff1a7e00640 1 --2- 172.21.15.22:0/4269424024 >> [v2:172.21.15.79:6838/2441542443,v1:172.21.15.79:6839/2441542443] conn(0x562244687990 0x562244689da0 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=60530c53d751f5af server_cookie=fa499230ff172808 in_seq=0 out_seq=0 2024-03-28T18:07:55.540 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:55.537+0000 7ff18ea00640 10 client.15720 ms_handle_connect on v2:172.21.15.79:6838/2441542443 2024-03-28T18:07:55.541 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:55.538+0000 7ff18ea00640 1 -- 172.21.15.22:0/4269424024 <== mds.0 v2:172.21.15.79:6838/2441542443 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x5622445df800 con 0x562244687990 2024-03-28T18:07:55.542 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:55.538+0000 7ff18ea00640 10 client.15720 handle_command_reply: tid=0 2024-03-28T18:07:55.542 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:55.538+0000 7ff1b27ec740 1 client.15720 shutdown 2024-03-28T18:07:55.542 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:55.538+0000 7ff1b27ec740 1 -- 172.21.15.22:0/4269424024 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56224466e6d0 msgr2=0x5622446774d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:07:55.542 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:55.538+0000 7ff1b27ec740 1 --2- 172.21.15.22:0/4269424024 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56224466e6d0 0x5622446774d0 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7ff19c00c6a0 tx=0x7ff19c01f000 comp rx=0 tx=0).stop 2024-03-28T18:07:55.542 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:55.538+0000 7ff1b27ec740 1 -- 172.21.15.22:0/4269424024 shutdown_connections 2024-03-28T18:07:55.542 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:55.538+0000 7ff1b27ec740 1 --2- 172.21.15.22:0/4269424024 >> [v2:172.21.15.79:6838/2441542443,v1:172.21.15.79:6839/2441542443] conn(0x562244687990 0x562244689da0 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:07:55.542 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:55.538+0000 7ff1b27ec740 1 --2- 172.21.15.22:0/4269424024 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562244677c00 0x562244674dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:07:55.542 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:55.538+0000 7ff1b27ec740 1 --2- 172.21.15.22:0/4269424024 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56224466f080 0x562244674860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:07:55.542 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:55.538+0000 7ff18ea00640 0 client.15720 ms_handle_reset on v2:172.21.15.79:6838/2441542443 2024-03-28T18:07:55.542 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:55.538+0000 7ff1b27ec740 1 --2- 172.21.15.22:0/4269424024 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56224466e6d0 0x5622446774d0 unknown :-1 s=CLOSED pgs=499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:07:55.542 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:55.538+0000 7ff1b27ec740 1 -- 172.21.15.22:0/4269424024 >> 172.21.15.22:0/4269424024 conn(0x562244668da0 msgr2=0x56224466aaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:07:55.542 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:55.538+0000 7ff1b27ec740 1 -- 172.21.15.22:0/4269424024 shutdown_connections 2024-03-28T18:07:55.542 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:55.538+0000 7ff1b27ec740 1 -- 172.21.15.22:0/4269424024 wait complete. 2024-03-28T18:07:55.543 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:55.539+0000 7ff1b27ec740 20 client.15720 trim_cache size 0 max 16384 2024-03-28T18:07:55.553 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:55.552+0000 7ff1b27ec740 1 Processor -- start 2024-03-28T18:07:55.553 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:55.552+0000 7ff1b27ec740 1 -- start start 2024-03-28T18:07:55.553 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:55.552+0000 7ff1b27ec740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56224466e6d0 0x562244676f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:07:55.553 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:55.552+0000 7ff1b27ec740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56224466f080 0x5622446774d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:07:55.553 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:55.552+0000 7ff1b27ec740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562244677c00 0x562244671b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:07:55.553 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:55.552+0000 7ff1b27ec740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1a81af220 con 0x56224466e6d0 2024-03-28T18:07:55.554 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:55.552+0000 7ff1b27ec740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1a81b0ba0 con 0x56224466f080 2024-03-28T18:07:55.554 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:55.552+0000 7ff1b27ec740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1a81af880 con 0x562244677c00 2024-03-28T18:07:55.554 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:55.553+0000 7ff1ad000640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56224466e6d0 0x562244676f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:07:55.554 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:55.553+0000 7ff1ad000640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56224466e6d0 0x562244676f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:50306/0 (socket says 172.21.15.22:50306) 2024-03-28T18:07:55.554 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:55.553+0000 7ff1ad000640 1 -- 172.21.15.22:0/1127113579 learned_addr learned my addr 172.21.15.22:0/1127113579 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-28T18:07:55.554 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:55.553+0000 7ff1a7e00640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56224466f080 0x5622446774d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:07:55.557 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:55.553+0000 7ff1ada00640 1 --2- 172.21.15.22:0/1127113579 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562244677c00 0x562244671b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:07:55.557 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:55.553+0000 7ff1ad000640 1 -- 172.21.15.22:0/1127113579 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562244677c00 msgr2=0x562244671b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T18:07:55.557 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:55.553+0000 7ff1ad000640 1 --2- 172.21.15.22:0/1127113579 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562244677c00 0x562244671b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:07:55.557 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:55.553+0000 7ff1ad000640 1 -- 172.21.15.22:0/1127113579 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56224466f080 msgr2=0x5622446774d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T18:07:55.557 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:55.553+0000 7ff1ad000640 1 --2- 172.21.15.22:0/1127113579 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56224466f080 0x5622446774d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:07:55.557 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:55.553+0000 7ff1ad000640 1 -- 172.21.15.22:0/1127113579 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562244669fd0 con 0x56224466e6d0 2024-03-28T18:07:55.557 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:55.553+0000 7ff1a7e00640 1 --2- 172.21.15.22:0/1127113579 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56224466f080 0x5622446774d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-28T18:07:55.558 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:55.553+0000 7ff1ad000640 1 --2- 172.21.15.22:0/1127113579 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56224466e6d0 0x562244676f90 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7ff19c09b7f0 tx=0x7ff19c00e940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d9b063cd301271b0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:07:55.558 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:55.554+0000 7ff199600640 1 -- 172.21.15.22:0/1127113579 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff19c0b4070 con 0x56224466e6d0 2024-03-28T18:07:55.558 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:55.554+0000 7ff199600640 1 -- 172.21.15.22:0/1127113579 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7ff19c09cd90 con 0x56224466e6d0 2024-03-28T18:07:55.558 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:55.554+0000 7ff199600640 1 -- 172.21.15.22:0/1127113579 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff19c09b3a0 con 0x56224466e6d0 2024-03-28T18:07:55.558 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:55.554+0000 7ff1b27ec740 1 -- 172.21.15.22:0/1127113579 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56224466e6d0 msgr2=0x562244676f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:07:55.558 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:55.554+0000 7ff1b27ec740 1 --2- 172.21.15.22:0/1127113579 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56224466e6d0 0x562244676f90 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7ff19c09b7f0 tx=0x7ff19c00e940 comp rx=0 tx=0).stop 2024-03-28T18:07:55.558 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:55.554+0000 7ff1b27ec740 1 -- 172.21.15.22:0/1127113579 shutdown_connections 2024-03-28T18:07:55.558 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:55.554+0000 7ff1b27ec740 1 --2- 172.21.15.22:0/1127113579 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562244677c00 0x562244671b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:07:55.558 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:55.554+0000 7ff1b27ec740 1 --2- 172.21.15.22:0/1127113579 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56224466f080 0x5622446774d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:07:55.558 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:55.554+0000 7ff1b27ec740 1 --2- 172.21.15.22:0/1127113579 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56224466e6d0 0x562244676f90 unknown :-1 s=CLOSED pgs=500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:07:55.558 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:55.554+0000 7ff1b27ec740 1 -- 172.21.15.22:0/1127113579 >> 172.21.15.22:0/1127113579 conn(0x562244668da0 msgr2=0x56224466aa80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:07:55.558 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:55.554+0000 7ff1b27ec740 1 -- 172.21.15.22:0/1127113579 shutdown_connections 2024-03-28T18:07:55.558 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:55.554+0000 7ff1b27ec740 1 -- 172.21.15.22:0/1127113579 wait complete. 2024-03-28T18:07:55.558 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:55.555+0000 7ff1b27ec740 1 -- start start 2024-03-28T18:07:55.558 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:55.555+0000 7ff1b27ec740 1 Processor -- start 2024-03-28T18:07:55.559 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:55.555+0000 7ff1b27ec740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56224466e6d0 0x56224467f990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:07:55.559 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:55.555+0000 7ff1b27ec740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56224466f080 0x56224467bd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:07:55.559 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:55.555+0000 7ff1b27ec740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562244677c00 0x56224467c250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:07:55.559 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:55.555+0000 7ff1b27ec740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1a81b0540 con 0x56224466f080 2024-03-28T18:07:55.559 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:55.555+0000 7ff1b27ec740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1a81afee0 con 0x56224466e6d0 2024-03-28T18:07:55.559 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:55.555+0000 7ff1b27ec740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1a81ad3e0 con 0x562244677c00 2024-03-28T18:07:55.559 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:55.556+0000 7ff1b27ec740 10 client.0 _ll_register_callbacks cb 0x7ff19c010da0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-28T18:07:55.559 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:55.556+0000 7ff1ad000640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56224466e6d0 0x56224467f990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:07:55.559 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:55.556+0000 7ff1a7e00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56224466f080 0x56224467bd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:07:55.559 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:55.556+0000 7ff1a7e00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56224466f080 0x56224467bd10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:50312/0 (socket says 172.21.15.22:50312) 2024-03-28T18:07:55.559 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:55.556+0000 7ff1a7e00640 1 -- 172.21.15.22:0/403346509 learned_addr learned my addr 172.21.15.22:0/403346509 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-28T18:07:55.559 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:55.556+0000 7ff1ad000640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56224466e6d0 0x56224467f990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.22:36998/0 (socket says 172.21.15.22:36998) 2024-03-28T18:07:55.559 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:55.556+0000 7ff1ada00640 1 --2- 172.21.15.22:0/403346509 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562244677c00 0x56224467c250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:07:55.559 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:55.556+0000 7ff1a7e00640 1 -- 172.21.15.22:0/403346509 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562244677c00 msgr2=0x56224467c250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T18:07:55.559 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:55.556+0000 7ff1a7e00640 1 --2- 172.21.15.22:0/403346509 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562244677c00 0x56224467c250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:07:55.560 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:55.556+0000 7ff1a7e00640 1 -- 172.21.15.22:0/403346509 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56224466e6d0 msgr2=0x56224467f990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T18:07:55.560 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:55.556+0000 7ff1a7e00640 1 --2- 172.21.15.22:0/403346509 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56224466e6d0 0x56224467f990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:07:55.560 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:55.556+0000 7ff1a7e00640 1 -- 172.21.15.22:0/403346509 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56224467c790 con 0x56224466f080 2024-03-28T18:07:55.560 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:55.556+0000 7ff1ad000640 1 --2- 172.21.15.22:0/403346509 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56224466e6d0 0x56224467f990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-28T18:07:55.560 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:55.556+0000 7ff1a7e00640 1 --2- 172.21.15.22:0/403346509 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56224466f080 0x56224467bd10 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7ff194010e10 tx=0x7ff19400bba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2af72ee2e87265fd server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:07:55.560 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:55.556+0000 7ff18ea00640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-03-28T18:07:55.560 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:55.557+0000 7ff18ea00640 1 -- 172.21.15.22:0/403346509 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff19402f070 con 0x56224466f080 2024-03-28T18:07:55.560 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:55.557+0000 7ff18ea00640 1 -- 172.21.15.22:0/403346509 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 8411+0+0 (secure 0 0 0) 0x7ff19401c700 con 0x56224466f080 2024-03-28T18:07:55.560 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:55.557+0000 7ff1b27ec740 1 -- 172.21.15.22:0/403346509 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7ff19c09cd90 con 0x56224466f080 2024-03-28T18:07:55.562 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:55.557+0000 7ff18ea00640 1 -- 172.21.15.22:0/403346509 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff19403a050 con 0x56224466f080 2024-03-28T18:07:55.562 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:55.557+0000 7ff1b27ec740 10 client.15726 fetch_fsmap learned FSMap version 27 2024-03-28T18:07:55.562 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:55.557+0000 7ff1b27ec740 1 -- 172.21.15.22:0/403346509 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x5622446e3ed0 con 0x56224466f080 2024-03-28T18:07:55.562 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:55.557+0000 7ff18ea00640 1 -- 172.21.15.22:0/403346509 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7ff19c09cd90 con 0x56224466f080 2024-03-28T18:07:55.562 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:55.557+0000 7ff18ea00640 1 -- 172.21.15.22:0/403346509 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 27) v1 ==== 7934+0+0 (secure 0 0 0) 0x7ff19402ba40 con 0x56224466f080 2024-03-28T18:07:55.562 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:55.557+0000 7ff1b27ec740 10 client.15726 fetch_fsmap finished waiting for FSMap version 27 2024-03-28T18:07:55.562 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:55.557+0000 7ff1b27ec740 10 client.15726 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-28T18:07:55.562 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:55.557+0000 7ff1b27ec740 20 client.15726 populate_metadata read hostname 'smithi022' 2024-03-28T18:07:55.562 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:55.557+0000 7ff1b27ec740 1 --2- 172.21.15.22:0/403346509 >> [v2:172.21.15.79:6838/2441542443,v1:172.21.15.79:6839/2441542443] conn(0x5622446e4340 0x5622446e6760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:07:55.563 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:55.557+0000 7ff1b27ec740 4 client.15726 mds_command: new command op to 34553 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-28T18:07:55.563 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:55.558+0000 7ff1b27ec740 1 -- 172.21.15.22:0/403346509 --> [v2:172.21.15.79:6838/2441542443,v1:172.21.15.79:6839/2441542443] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x5622445d9b50 con 0x5622446e4340 2024-03-28T18:07:55.563 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:55.558+0000 7ff1ad000640 1 --2- 172.21.15.22:0/403346509 >> [v2:172.21.15.79:6838/2441542443,v1:172.21.15.79:6839/2441542443] conn(0x5622446e4340 0x5622446e6760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:07:55.563 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:55.558+0000 7ff1ad000640 1 --2- 172.21.15.22:0/403346509 >> [v2:172.21.15.79:6838/2441542443,v1:172.21.15.79:6839/2441542443] conn(0x5622446e4340 0x5622446e6760 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=93687b9f13787bb5 server_cookie=48ad6304bbcb3df2 in_seq=0 out_seq=0 2024-03-28T18:07:55.563 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:55.558+0000 7ff18ea00640 10 client.15726 ms_handle_connect on v2:172.21.15.79:6838/2441542443 2024-03-28T18:07:55.563 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:55.559+0000 7ff18ea00640 1 -- 172.21.15.22:0/403346509 <== mds.0 v2:172.21.15.79:6838/2441542443 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x5622445d9b50 con 0x5622446e4340 2024-03-28T18:07:55.563 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:55.559+0000 7ff18ea00640 10 client.15726 handle_command_reply: tid=0 2024-03-28T18:07:55.563 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:55.559+0000 7ff1b27ec740 1 client.15726 shutdown 2024-03-28T18:07:55.563 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:55.559+0000 7ff1b27ec740 1 -- 172.21.15.22:0/403346509 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56224466f080 msgr2=0x56224467bd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:07:55.563 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:55.559+0000 7ff1b27ec740 1 --2- 172.21.15.22:0/403346509 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56224466f080 0x56224467bd10 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7ff194010e10 tx=0x7ff19400bba0 comp rx=0 tx=0).stop 2024-03-28T18:07:55.563 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:55.559+0000 7ff1b27ec740 1 -- 172.21.15.22:0/403346509 shutdown_connections 2024-03-28T18:07:55.563 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:55.559+0000 7ff1b27ec740 1 --2- 172.21.15.22:0/403346509 >> [v2:172.21.15.79:6838/2441542443,v1:172.21.15.79:6839/2441542443] conn(0x5622446e4340 0x5622446e6760 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:07:55.563 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:55.559+0000 7ff1b27ec740 1 --2- 172.21.15.22:0/403346509 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562244677c00 0x56224467c250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:07:55.563 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:55.559+0000 7ff1b27ec740 1 --2- 172.21.15.22:0/403346509 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56224466f080 0x56224467bd10 unknown :-1 s=CLOSED pgs=501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:07:55.564 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:55.559+0000 7ff1b27ec740 1 --2- 172.21.15.22:0/403346509 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56224466e6d0 0x56224467f990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:07:55.564 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:55.559+0000 7ff1b27ec740 1 -- 172.21.15.22:0/403346509 >> 172.21.15.22:0/403346509 conn(0x562244668da0 msgr2=0x562244667620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:07:55.564 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:55.559+0000 7ff18ea00640 0 client.15726 ms_handle_reset on v2:172.21.15.79:6838/2441542443 2024-03-28T18:07:55.564 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:55.559+0000 7ff1b27ec740 1 -- 172.21.15.22:0/403346509 shutdown_connections 2024-03-28T18:07:55.564 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:55.559+0000 7ff1b27ec740 1 -- 172.21.15.22:0/403346509 wait complete. 2024-03-28T18:07:55.564 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:55.559+0000 7ff1b27ec740 20 client.15726 trim_cache size 0 max 16384 2024-03-28T18:07:55.564 INFO:teuthology.orchestra.run.smithi022.stdout:[] 2024-03-28T18:07:55.565 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:55.560+0000 7ff1afa00640 1 -- 172.21.15.22:0/212249984 >> [v2:172.21.15.22:6800/1357365082,v1:172.21.15.22:6801/1357365082] conn(0x7ff1900a2a00 msgr2=0x7ff1900a4ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:07:55.565 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:55.560+0000 7ff1afa00640 1 --2- 172.21.15.22:0/212249984 >> [v2:172.21.15.22:6800/1357365082,v1:172.21.15.22:6801/1357365082] conn(0x7ff1900a2a00 0x7ff1900a4ec0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7ff1a002c710 tx=0x7ff1a0004a40 comp rx=0 tx=0).stop 2024-03-28T18:07:55.565 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:55.560+0000 7ff1afa00640 1 -- 172.21.15.22:0/212249984 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff1a8103840 msgr2=0x7ff1a819a0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:07:55.565 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:55.560+0000 7ff1afa00640 1 --2- 172.21.15.22:0/212249984 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff1a8103840 0x7ff1a819a0d0 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7ff19c00e9f0 tx=0x7ff19c00eeb0 comp rx=0 tx=0).stop 2024-03-28T18:07:55.565 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:55.560+0000 7ff1afa00640 1 -- 172.21.15.22:0/212249984 shutdown_connections 2024-03-28T18:07:55.565 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:55.560+0000 7ff1afa00640 1 --2- 172.21.15.22:0/212249984 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff1a8109850 0x7ff1a819e990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:07:55.565 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:55.560+0000 7ff1afa00640 1 --2- 172.21.15.22:0/212249984 >> [v2:172.21.15.22:6800/1357365082,v1:172.21.15.22:6801/1357365082] conn(0x7ff1900a2a00 0x7ff1900a4ec0 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:07:55.565 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:55.560+0000 7ff1afa00640 1 --2- 172.21.15.22:0/212249984 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ff1a8104200 0x7ff1a819a610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:07:55.565 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:55.560+0000 7ff1afa00640 1 --2- 172.21.15.22:0/212249984 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff1a8103840 0x7ff1a819a0d0 unknown :-1 s=CLOSED pgs=497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:07:55.565 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:55.560+0000 7ff1afa00640 1 -- 172.21.15.22:0/212249984 >> 172.21.15.22:0/212249984 conn(0x7ff1a80ff310 msgr2=0x7ff1a80ff950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:07:55.565 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:55.560+0000 7ff1afa00640 1 -- 172.21.15.22:0/212249984 shutdown_connections 2024-03-28T18:07:55.565 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:55.560+0000 7ff1afa00640 1 -- 172.21.15.22:0/212249984 wait complete. 2024-03-28T18:07:56.577 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-03-28T18:07:56.577 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-28T18:07:56.577 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-03-28T18:07:56.819 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:56.817+0000 7f79c3800640 1 -- 172.21.15.22:0/566744833 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f79bc109850 msgr2=0x7f79bc109c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:07:56.819 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:56.817+0000 7f79c3800640 1 --2- 172.21.15.22:0/566744833 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f79bc109850 0x7f79bc109c30 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f79b0009930 tx=0x7f79b002bc20 comp rx=0 tx=0).stop 2024-03-28T18:07:56.819 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:56.818+0000 7f79c3800640 1 -- 172.21.15.22:0/566744833 shutdown_connections 2024-03-28T18:07:56.819 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:56.818+0000 7f79c3800640 1 --2- 172.21.15.22:0/566744833 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f79bc104200 0x7f79bc110710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:07:56.819 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:56.818+0000 7f79c3800640 1 --2- 172.21.15.22:0/566744833 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f79bc103840 0x7f79bc103cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:07:56.819 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:56.818+0000 7f79c3800640 1 --2- 172.21.15.22:0/566744833 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f79bc109850 0x7f79bc109c30 unknown :-1 s=CLOSED pgs=502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:07:56.819 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:56.818+0000 7f79c3800640 1 -- 172.21.15.22:0/566744833 >> 172.21.15.22:0/566744833 conn(0x7f79bc0ff350 msgr2=0x7f79bc101770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:07:56.819 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:56.818+0000 7f79c3800640 1 -- 172.21.15.22:0/566744833 shutdown_connections 2024-03-28T18:07:56.819 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:56.818+0000 7f79c3800640 1 -- 172.21.15.22:0/566744833 wait complete. 2024-03-28T18:07:56.821 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:56.819+0000 7f79c3800640 1 Processor -- start 2024-03-28T18:07:56.822 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:56.819+0000 7f79c3800640 1 -- start start 2024-03-28T18:07:56.822 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:56.819+0000 7f79c3800640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f79bc103840 0x7f79bc19a190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:07:56.822 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:56.819+0000 7f79c3800640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f79bc104200 0x7f79bc19a6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:07:56.822 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:56.819+0000 7f79c3800640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f79bc109850 0x7f79bc19ea50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:07:56.822 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:56.819+0000 7f79c3800640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79bc113c50 con 0x7f79bc103840 2024-03-28T18:07:56.822 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:56.819+0000 7f79c3800640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79bc113ad0 con 0x7f79bc109850 2024-03-28T18:07:56.822 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:56.819+0000 7f79c3800640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79bc113dd0 con 0x7f79bc104200 2024-03-28T18:07:56.822 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:56.819+0000 7f79c0e00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f79bc103840 0x7f79bc19a190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:07:56.822 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:56.820+0000 7f79c0e00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f79bc103840 0x7f79bc19a190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:50330/0 (socket says 172.21.15.22:50330) 2024-03-28T18:07:56.822 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:56.820+0000 7f79c0e00640 1 -- 172.21.15.22:0/1829866630 learned_addr learned my addr 172.21.15.22:0/1829866630 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-28T18:07:56.822 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:56.820+0000 7f79c1800640 1 --2- 172.21.15.22:0/1829866630 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f79bc109850 0x7f79bc19ea50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:07:56.822 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:56.820+0000 7f79bbe00640 1 --2- 172.21.15.22:0/1829866630 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f79bc104200 0x7f79bc19a6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:07:56.822 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:56.820+0000 7f79c0e00640 1 -- 172.21.15.22:0/1829866630 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f79bc104200 msgr2=0x7f79bc19a6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:07:56.823 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:56.820+0000 7f79c0e00640 1 --2- 172.21.15.22:0/1829866630 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f79bc104200 0x7f79bc19a6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:07:56.823 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:56.820+0000 7f79c0e00640 1 -- 172.21.15.22:0/1829866630 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f79bc109850 msgr2=0x7f79bc19ea50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:07:56.823 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:56.820+0000 7f79c0e00640 1 --2- 172.21.15.22:0/1829866630 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f79bc109850 0x7f79bc19ea50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:07:56.823 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:56.820+0000 7f79c0e00640 1 -- 172.21.15.22:0/1829866630 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f79bc19f180 con 0x7f79bc103840 2024-03-28T18:07:56.823 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:56.820+0000 7f79bbe00640 1 --2- 172.21.15.22:0/1829866630 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f79bc104200 0x7f79bc19a6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T18:07:56.823 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:56.820+0000 7f79c1800640 1 --2- 172.21.15.22:0/1829866630 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f79bc109850 0x7f79bc19ea50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-28T18:07:56.823 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:56.820+0000 7f79c0e00640 1 --2- 172.21.15.22:0/1829866630 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f79bc103840 0x7f79bc19a190 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f79b0000c00 tx=0x7f79b0005020 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:07:56.823 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:56.821+0000 7f79b9600640 1 -- 172.21.15.22:0/1829866630 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79b0043020 con 0x7f79bc103840 2024-03-28T18:07:56.823 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:56.821+0000 7f79b9600640 1 -- 172.21.15.22:0/1829866630 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f79b0005420 con 0x7f79bc103840 2024-03-28T18:07:56.823 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:56.821+0000 7f79c3800640 1 -- 172.21.15.22:0/1829866630 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f79bc19f410 con 0x7f79bc103840 2024-03-28T18:07:56.823 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:56.821+0000 7f79b9600640 1 -- 172.21.15.22:0/1829866630 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79b0003bf0 con 0x7f79bc103840 2024-03-28T18:07:56.823 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:56.821+0000 7f79c3800640 1 -- 172.21.15.22:0/1829866630 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f79bc1a6d00 con 0x7f79bc103840 2024-03-28T18:07:56.825 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:56.824+0000 7f79b9600640 1 -- 172.21.15.22:0/1829866630 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 24) v1 ==== 137466+0+0 (secure 0 0 0) 0x7f79b0046020 con 0x7f79bc103840 2024-03-28T18:07:56.825 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:56.824+0000 7f79c6438740 1 Processor -- start 2024-03-28T18:07:56.825 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:56.824+0000 7f79c6438740 1 -- start start 2024-03-28T18:07:56.826 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:56.825+0000 7f79c6438740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a297fa4080 0x55a297fa4460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:07:56.826 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:56.825+0000 7f79c6438740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55a297fa49a0 0x55a297fad040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:07:56.827 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:56.825+0000 7f79c6438740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a297fad580 0x55a297faf970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:07:56.827 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:56.825+0000 7f79c6438740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a297f08eb0 con 0x55a297fad580 2024-03-28T18:07:56.827 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:56.825+0000 7f79c6438740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a297eb1a60 con 0x55a297fa49a0 2024-03-28T18:07:56.827 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:56.825+0000 7f79c6438740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a297f04f50 con 0x55a297fa4080 2024-03-28T18:07:56.827 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:56.825+0000 7f79bbe00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a297fad580 0x55a297faf970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:07:56.827 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:56.825+0000 7f79bbe00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a297fad580 0x55a297faf970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:50336/0 (socket says 172.21.15.22:50336) 2024-03-28T18:07:56.827 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:56.825+0000 7f79bbe00640 1 -- 172.21.15.22:0/733708903 learned_addr learned my addr 172.21.15.22:0/733708903 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-28T18:07:56.827 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:56.825+0000 7f79c0e00640 1 --2- 172.21.15.22:0/733708903 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55a297fa49a0 0x55a297fad040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:07:56.827 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:56.825+0000 7f79c1800640 1 --2- 172.21.15.22:0/733708903 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a297fa4080 0x55a297fa4460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:07:56.827 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:56.826+0000 7f79bbe00640 1 -- 172.21.15.22:0/733708903 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a297fa4080 msgr2=0x55a297fa4460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T18:07:56.827 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:56.826+0000 7f79bbe00640 1 --2- 172.21.15.22:0/733708903 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a297fa4080 0x55a297fa4460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:07:56.827 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:56.826+0000 7f79bbe00640 1 -- 172.21.15.22:0/733708903 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55a297fa49a0 msgr2=0x55a297fad040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T18:07:56.827 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:56.826+0000 7f79bbe00640 1 --2- 172.21.15.22:0/733708903 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55a297fa49a0 0x55a297fad040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:07:56.827 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:56.826+0000 7f79bbe00640 1 -- 172.21.15.22:0/733708903 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a297ce57e0 con 0x55a297fad580 2024-03-28T18:07:56.828 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:56.826+0000 7f79b9600640 1 --2- 172.21.15.22:0/1829866630 >> [v2:172.21.15.22:6800/1357365082,v1:172.21.15.22:6801/1357365082] conn(0x7f79a40a2a80 0x7f79a40a4f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:07:56.828 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:56.826+0000 7f79c1800640 1 --2- 172.21.15.22:0/733708903 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a297fa4080 0x55a297fa4460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-28T18:07:56.828 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:56.826+0000 7f79b9600640 1 -- 172.21.15.22:0/1829866630 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 8411+0+0 (secure 0 0 0) 0x7f79b00f1060 con 0x7f79bc103840 2024-03-28T18:07:56.828 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:56.826+0000 7f79bbe00640 1 --2- 172.21.15.22:0/733708903 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a297fad580 0x55a297faf970 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f79a800bbe0 tx=0x7f79a800d710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6243803e722478f5 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:07:56.828 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:56.826+0000 7f79c1800640 1 --2- 172.21.15.22:0/1829866630 >> [v2:172.21.15.22:6800/1357365082,v1:172.21.15.22:6801/1357365082] conn(0x7f79a40a2a80 0x7f79a40a4f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:07:56.831 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:56.826+0000 7f79ae000640 1 -- 172.21.15.22:0/733708903 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79a8017620 con 0x55a297fad580 2024-03-28T18:07:56.831 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:56.826+0000 7f79ae000640 1 -- 172.21.15.22:0/733708903 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f79a8017c90 con 0x55a297fad580 2024-03-28T18:07:56.831 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:56.826+0000 7f79ae000640 1 -- 172.21.15.22:0/733708903 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79a800fd30 con 0x55a297fad580 2024-03-28T18:07:56.831 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:56.827+0000 7f79c6438740 1 -- 172.21.15.22:0/733708903 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a297fad580 msgr2=0x55a297faf970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:07:56.831 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:56.827+0000 7f79c6438740 1 --2- 172.21.15.22:0/733708903 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a297fad580 0x55a297faf970 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f79a800bbe0 tx=0x7f79a800d710 comp rx=0 tx=0).stop 2024-03-28T18:07:56.831 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:56.827+0000 7f79c1800640 1 --2- 172.21.15.22:0/1829866630 >> [v2:172.21.15.22:6800/1357365082,v1:172.21.15.22:6801/1357365082] conn(0x7f79a40a2a80 0x7f79a40a4f40 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f79b4005fd0 tx=0x7f79b4009990 comp rx=0 tx=0).ready entity=mgr.15522 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:07:56.831 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:56.827+0000 7f79c6438740 1 -- 172.21.15.22:0/733708903 shutdown_connections 2024-03-28T18:07:56.831 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:56.827+0000 7f79c6438740 1 --2- 172.21.15.22:0/733708903 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a297fad580 0x55a297faf970 unknown :-1 s=CLOSED pgs=504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:07:56.831 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:56.827+0000 7f79c6438740 1 --2- 172.21.15.22:0/733708903 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55a297fa49a0 0x55a297fad040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:07:56.831 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:56.827+0000 7f79c6438740 1 --2- 172.21.15.22:0/733708903 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a297fa4080 0x55a297fa4460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:07:56.831 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:56.827+0000 7f79c6438740 1 -- 172.21.15.22:0/733708903 >> 172.21.15.22:0/733708903 conn(0x55a297f9e7b0 msgr2=0x55a297f9eb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:07:56.831 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:56.827+0000 7f79c6438740 1 -- 172.21.15.22:0/733708903 shutdown_connections 2024-03-28T18:07:56.831 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:56.827+0000 7f79c6438740 1 -- 172.21.15.22:0/733708903 wait complete. 2024-03-28T18:07:56.832 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:56.828+0000 7f79c6438740 1 -- start start 2024-03-28T18:07:56.832 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:56.828+0000 7f79c6438740 1 Processor -- start 2024-03-28T18:07:56.832 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:56.828+0000 7f79c6438740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a297fa4080 0x55a297face30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:07:56.832 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:56.829+0000 7f79c6438740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55a297fa49a0 0x55a297faa1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:07:56.832 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:56.829+0000 7f79c6438740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a297fad580 0x55a297faa700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:07:56.832 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:56.829+0000 7f79c6438740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a297fb0370 con 0x55a297fad580 2024-03-28T18:07:56.832 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:56.829+0000 7f79c6438740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a297ec4eb0 con 0x55a297fa49a0 2024-03-28T18:07:56.832 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:56.829+0000 7f79c6438740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a297fb04f0 con 0x55a297fa4080 2024-03-28T18:07:56.832 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:56.829+0000 7f79c6438740 10 client.0 _ll_register_callbacks cb 0x55a297eb9150 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-28T18:07:56.832 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:56.829+0000 7f79c1800640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a297fad580 0x55a297faa700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:07:56.832 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:56.829+0000 7f79c1800640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a297fad580 0x55a297faa700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:50340/0 (socket says 172.21.15.22:50340) 2024-03-28T18:07:56.832 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:56.829+0000 7f79c1800640 1 -- 172.21.15.22:0/2680556704 learned_addr learned my addr 172.21.15.22:0/2680556704 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-28T18:07:56.832 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:56.829+0000 7f79bbe00640 1 --2- 172.21.15.22:0/2680556704 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55a297fa49a0 0x55a297faa1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:07:56.832 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:56.829+0000 7f79c0e00640 1 --2- 172.21.15.22:0/2680556704 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a297fa4080 0x55a297face30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:07:56.833 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:56.829+0000 7f79c1800640 1 -- 172.21.15.22:0/2680556704 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a297fa4080 msgr2=0x55a297face30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T18:07:56.833 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:56.829+0000 7f79c1800640 1 --2- 172.21.15.22:0/2680556704 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a297fa4080 0x55a297face30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:07:56.833 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:56.829+0000 7f79c1800640 1 -- 172.21.15.22:0/2680556704 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55a297fa49a0 msgr2=0x55a297faa1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T18:07:56.833 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:56.829+0000 7f79c1800640 1 --2- 172.21.15.22:0/2680556704 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55a297fa49a0 0x55a297faa1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:07:56.833 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:56.829+0000 7f79c1800640 1 -- 172.21.15.22:0/2680556704 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55a297d45f70 con 0x55a297fad580 2024-03-28T18:07:56.833 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:56.829+0000 7f79c0e00640 1 --2- 172.21.15.22:0/2680556704 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a297fa4080 0x55a297face30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-28T18:07:56.833 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:56.830+0000 7f79c1800640 1 --2- 172.21.15.22:0/2680556704 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a297fad580 0x55a297faa700 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f79b401d8d0 tx=0x7f79b4004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9c7fb4f61686ded4 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:07:56.833 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:56.830+0000 7f79a2a00640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-03-28T18:07:56.833 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:56.830+0000 7f79a2a00640 1 -- 172.21.15.22:0/2680556704 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79b401e6d0 con 0x55a297fad580 2024-03-28T18:07:56.833 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:56.830+0000 7f79a2a00640 1 -- 172.21.15.22:0/2680556704 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 8411+0+0 (secure 0 0 0) 0x7f79b401e870 con 0x55a297fad580 2024-03-28T18:07:56.833 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:56.830+0000 7f79c6438740 1 -- 172.21.15.22:0/2680556704 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f79a8017c90 con 0x55a297fad580 2024-03-28T18:07:56.838 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:56.830+0000 7f79a2a00640 1 -- 172.21.15.22:0/2680556704 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f79b401ea70 con 0x55a297fad580 2024-03-28T18:07:56.838 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:56.830+0000 7f79c6438740 10 client.15738 fetch_fsmap learned FSMap version 27 2024-03-28T18:07:56.838 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:56.830+0000 7f79c6438740 1 -- 172.21.15.22:0/2680556704 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55a297bdd170 con 0x55a297fad580 2024-03-28T18:07:56.839 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:56.831+0000 7f79a2a00640 1 -- 172.21.15.22:0/2680556704 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f79a8017c90 con 0x55a297fad580 2024-03-28T18:07:56.839 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:56.831+0000 7f79a2a00640 1 -- 172.21.15.22:0/2680556704 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 27) v1 ==== 7934+0+0 (secure 0 0 0) 0x7f79b400f040 con 0x55a297fad580 2024-03-28T18:07:56.839 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:56.831+0000 7f79c6438740 10 client.15738 fetch_fsmap finished waiting for FSMap version 27 2024-03-28T18:07:56.839 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:56.831+0000 7f79c6438740 10 client.15738 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-28T18:07:56.839 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:56.831+0000 7f79c6438740 20 client.15738 populate_metadata read hostname 'smithi022' 2024-03-28T18:07:56.839 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:56.831+0000 7f79c6438740 1 --2- 172.21.15.22:0/2680556704 >> [v2:172.21.15.79:6838/2441542443,v1:172.21.15.79:6839/2441542443] conn(0x55a297fbd520 0x55a297fbf980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:07:56.839 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:56.831+0000 7f79c6438740 4 client.15738 mds_command: new command op to 34553 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-28T18:07:56.839 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:56.831+0000 7f79c6438740 1 -- 172.21.15.22:0/2680556704 --> [v2:172.21.15.79:6838/2441542443,v1:172.21.15.79:6839/2441542443] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55a297f163f0 con 0x55a297fbd520 2024-03-28T18:07:56.839 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:56.832+0000 7f79c0e00640 1 --2- 172.21.15.22:0/2680556704 >> [v2:172.21.15.79:6838/2441542443,v1:172.21.15.79:6839/2441542443] conn(0x55a297fbd520 0x55a297fbf980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:07:56.839 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:56.832+0000 7f79c0e00640 1 --2- 172.21.15.22:0/2680556704 >> [v2:172.21.15.79:6838/2441542443,v1:172.21.15.79:6839/2441542443] conn(0x55a297fbd520 0x55a297fbf980 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fcc63cff828835c6 server_cookie=ca1122c92dc448dc in_seq=0 out_seq=0 2024-03-28T18:07:56.839 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:56.832+0000 7f79a2a00640 10 client.15738 ms_handle_connect on v2:172.21.15.79:6838/2441542443 2024-03-28T18:07:56.839 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:56.833+0000 7f79a2a00640 1 -- 172.21.15.22:0/2680556704 <== mds.0 v2:172.21.15.79:6838/2441542443 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x55a297f163f0 con 0x55a297fbd520 2024-03-28T18:07:56.839 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:56.834+0000 7f79a2a00640 10 client.15738 handle_command_reply: tid=0 2024-03-28T18:07:56.839 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:56.834+0000 7f79c6438740 1 client.15738 shutdown 2024-03-28T18:07:56.839 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:56.834+0000 7f79c6438740 1 -- 172.21.15.22:0/2680556704 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a297fad580 msgr2=0x55a297faa700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:07:56.840 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:56.834+0000 7f79c6438740 1 --2- 172.21.15.22:0/2680556704 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a297fad580 0x55a297faa700 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f79b401d8d0 tx=0x7f79b4004a40 comp rx=0 tx=0).stop 2024-03-28T18:07:56.840 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:56.834+0000 7f79c6438740 1 -- 172.21.15.22:0/2680556704 shutdown_connections 2024-03-28T18:07:56.840 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:56.834+0000 7f79c6438740 1 --2- 172.21.15.22:0/2680556704 >> [v2:172.21.15.79:6838/2441542443,v1:172.21.15.79:6839/2441542443] conn(0x55a297fbd520 0x55a297fbf980 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:07:56.840 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:56.834+0000 7f79c6438740 1 --2- 172.21.15.22:0/2680556704 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a297fad580 0x55a297faa700 unknown :-1 s=CLOSED pgs=505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:07:56.840 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:56.834+0000 7f79c6438740 1 --2- 172.21.15.22:0/2680556704 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55a297fa49a0 0x55a297faa1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:07:56.840 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:56.834+0000 7f79c6438740 1 --2- 172.21.15.22:0/2680556704 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a297fa4080 0x55a297face30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:07:56.840 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:56.834+0000 7f79c6438740 1 -- 172.21.15.22:0/2680556704 >> 172.21.15.22:0/2680556704 conn(0x55a297f9e7b0 msgr2=0x55a297fa0500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:07:56.840 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:56.834+0000 7f79a2a00640 0 client.15738 ms_handle_reset on v2:172.21.15.79:6838/2441542443 2024-03-28T18:07:56.840 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:56.834+0000 7f79c6438740 1 -- 172.21.15.22:0/2680556704 shutdown_connections 2024-03-28T18:07:56.840 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:56.834+0000 7f79c6438740 1 -- 172.21.15.22:0/2680556704 wait complete. 2024-03-28T18:07:56.840 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:56.834+0000 7f79c6438740 20 client.15738 trim_cache size 0 max 16384 2024-03-28T18:07:56.853 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:56.852+0000 7f79c6438740 1 Processor -- start 2024-03-28T18:07:56.853 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:56.853+0000 7f79c6438740 1 -- start start 2024-03-28T18:07:56.853 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:56.853+0000 7f79c6438740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55a297f9e7b0 0x55a297fa55d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:07:56.853 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:56.853+0000 7f79c6438740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a297fa4080 0x55a297fac8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:07:56.854 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:56.853+0000 7f79c6438740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a297fa49a0 0x55a297face30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:07:56.854 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:56.853+0000 7f79c6438740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79bc1af180 con 0x55a297fa49a0 2024-03-28T18:07:56.854 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:56.853+0000 7f79c6438740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79bc1b0b00 con 0x55a297f9e7b0 2024-03-28T18:07:56.854 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:56.853+0000 7f79c6438740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79bc1af7e0 con 0x55a297fa4080 2024-03-28T18:07:56.855 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:56.853+0000 7f79c1800640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a297fa49a0 0x55a297face30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:07:56.855 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:56.853+0000 7f79c1800640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a297fa49a0 0x55a297face30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:50356/0 (socket says 172.21.15.22:50356) 2024-03-28T18:07:56.855 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:56.853+0000 7f79c1800640 1 -- 172.21.15.22:0/1265034179 learned_addr learned my addr 172.21.15.22:0/1265034179 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-28T18:07:56.855 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:56.853+0000 7f79c0e00640 1 --2- 172.21.15.22:0/1265034179 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55a297f9e7b0 0x55a297fa55d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:07:56.855 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:56.853+0000 7f79bbe00640 1 --2- 172.21.15.22:0/1265034179 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a297fa4080 0x55a297fac8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:07:56.855 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:56.854+0000 7f79c1800640 1 -- 172.21.15.22:0/1265034179 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a297fa4080 msgr2=0x55a297fac8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T18:07:56.855 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:56.854+0000 7f79c1800640 1 --2- 172.21.15.22:0/1265034179 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a297fa4080 0x55a297fac8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:07:56.855 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:56.854+0000 7f79c1800640 1 -- 172.21.15.22:0/1265034179 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55a297f9e7b0 msgr2=0x55a297fa55d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T18:07:56.855 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:56.854+0000 7f79c1800640 1 --2- 172.21.15.22:0/1265034179 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55a297f9e7b0 0x55a297fa55d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:07:56.855 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:56.854+0000 7f79c1800640 1 -- 172.21.15.22:0/1265034179 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a297fafce0 con 0x55a297fa49a0 2024-03-28T18:07:56.856 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:56.854+0000 7f79c0e00640 1 --2- 172.21.15.22:0/1265034179 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55a297f9e7b0 0x55a297fa55d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-28T18:07:56.856 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:56.854+0000 7f79bbe00640 1 --2- 172.21.15.22:0/1265034179 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a297fa4080 0x55a297fac8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-28T18:07:56.856 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:56.855+0000 7f79c1800640 1 --2- 172.21.15.22:0/1265034179 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a297fa49a0 0x55a297face30 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f79b401e010 tx=0x7f79b401d530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bc02985bacb9eca3 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:07:56.857 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:56.855+0000 7f79ad600640 1 -- 172.21.15.22:0/1265034179 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79b4034070 con 0x55a297fa49a0 2024-03-28T18:07:56.857 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:56.855+0000 7f79ad600640 1 -- 172.21.15.22:0/1265034179 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f79b4027cb0 con 0x55a297fa49a0 2024-03-28T18:07:56.857 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:56.855+0000 7f79ad600640 1 -- 172.21.15.22:0/1265034179 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79b40278d0 con 0x55a297fa49a0 2024-03-28T18:07:56.857 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:56.856+0000 7f79c6438740 1 -- 172.21.15.22:0/1265034179 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a297fa49a0 msgr2=0x55a297face30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:07:56.857 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:56.856+0000 7f79c6438740 1 --2- 172.21.15.22:0/1265034179 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a297fa49a0 0x55a297face30 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f79b401e010 tx=0x7f79b401d530 comp rx=0 tx=0).stop 2024-03-28T18:07:56.857 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:56.856+0000 7f79c6438740 1 -- 172.21.15.22:0/1265034179 shutdown_connections 2024-03-28T18:07:56.857 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:56.856+0000 7f79c6438740 1 --2- 172.21.15.22:0/1265034179 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a297fa49a0 0x55a297face30 unknown :-1 s=CLOSED pgs=506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:07:56.857 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:56.856+0000 7f79c6438740 1 --2- 172.21.15.22:0/1265034179 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a297fa4080 0x55a297fac8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:07:56.857 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:56.856+0000 7f79c6438740 1 --2- 172.21.15.22:0/1265034179 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55a297f9e7b0 0x55a297fa55d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:07:56.857 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:56.856+0000 7f79c6438740 1 -- 172.21.15.22:0/1265034179 >> 172.21.15.22:0/1265034179 conn(0x55a297faa860 msgr2=0x55a297fa4ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:07:56.857 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:56.856+0000 7f79c6438740 1 -- 172.21.15.22:0/1265034179 shutdown_connections 2024-03-28T18:07:56.857 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:56.856+0000 7f79c6438740 1 -- 172.21.15.22:0/1265034179 wait complete. 2024-03-28T18:07:56.860 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:56.856+0000 7f79c6438740 1 -- start start 2024-03-28T18:07:56.860 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:56.857+0000 7f79c6438740 1 Processor -- start 2024-03-28T18:07:56.860 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:56.857+0000 7f79c6438740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a297f9e7b0 0x55a297fb2770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:07:56.860 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:56.857+0000 7f79c6438740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a297fa4080 0x55a297fb2cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:07:56.860 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:56.857+0000 7f79c6438740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55a297fa49a0 0x55a297fb31f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:07:56.860 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:56.857+0000 7f79c6438740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79bc1b04a0 con 0x55a297fa4080 2024-03-28T18:07:56.860 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:56.857+0000 7f79c6438740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79bc1afe40 con 0x55a297fa49a0 2024-03-28T18:07:56.860 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:56.857+0000 7f79c6438740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79bc1ad340 con 0x55a297f9e7b0 2024-03-28T18:07:56.860 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:56.857+0000 7f79c6438740 10 client.0 _ll_register_callbacks cb 0x7f79b4009be0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-28T18:07:56.860 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:56.857+0000 7f79bbe00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a297fa4080 0x55a297fb2cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:07:56.860 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:56.857+0000 7f79bbe00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a297fa4080 0x55a297fb2cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:50370/0 (socket says 172.21.15.22:50370) 2024-03-28T18:07:56.860 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:56.857+0000 7f79bbe00640 1 -- 172.21.15.22:0/2251488246 learned_addr learned my addr 172.21.15.22:0/2251488246 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-28T18:07:56.861 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:56.857+0000 7f79c1800640 1 --2- 172.21.15.22:0/2251488246 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55a297fa49a0 0x55a297fb31f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:07:56.861 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:56.858+0000 7f79c0e00640 1 --2- 172.21.15.22:0/2251488246 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a297f9e7b0 0x55a297fb2770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:07:56.861 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:56.858+0000 7f79bbe00640 1 -- 172.21.15.22:0/2251488246 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a297f9e7b0 msgr2=0x55a297fb2770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T18:07:56.861 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:56.858+0000 7f79bbe00640 1 --2- 172.21.15.22:0/2251488246 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a297f9e7b0 0x55a297fb2770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:07:56.861 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:56.858+0000 7f79bbe00640 1 -- 172.21.15.22:0/2251488246 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55a297fa49a0 msgr2=0x55a297fb31f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T18:07:56.861 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:56.858+0000 7f79bbe00640 1 --2- 172.21.15.22:0/2251488246 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55a297fa49a0 0x55a297fb31f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:07:56.861 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:56.858+0000 7f79bbe00640 1 -- 172.21.15.22:0/2251488246 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55a298019020 con 0x55a297fa4080 2024-03-28T18:07:56.861 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:56.858+0000 7f79bbe00640 1 --2- 172.21.15.22:0/2251488246 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a297fa4080 0x55a297fb2cb0 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x55a297f9f940 tx=0x7f79a800df80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=72ff2fa9d32b4ef server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:07:56.861 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:56.858+0000 7f79a2a00640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-03-28T18:07:56.861 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:56.858+0000 7f79a2a00640 1 -- 172.21.15.22:0/2251488246 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79a800f6c0 con 0x55a297fa4080 2024-03-28T18:07:56.861 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:56.858+0000 7f79a2a00640 1 -- 172.21.15.22:0/2251488246 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 8411+0+0 (secure 0 0 0) 0x7f79a8005030 con 0x55a297fa4080 2024-03-28T18:07:56.861 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:56.858+0000 7f79c6438740 1 -- 172.21.15.22:0/2251488246 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f79b4027cb0 con 0x55a297fa4080 2024-03-28T18:07:56.861 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:56.859+0000 7f79a2a00640 1 -- 172.21.15.22:0/2251488246 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f79a8005250 con 0x55a297fa4080 2024-03-28T18:07:56.861 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:56.859+0000 7f79c6438740 10 client.15744 fetch_fsmap learned FSMap version 27 2024-03-28T18:07:56.862 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:56.859+0000 7f79c6438740 1 -- 172.21.15.22:0/2251488246 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55a29801a310 con 0x55a297fa4080 2024-03-28T18:07:56.862 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:56.859+0000 7f79a2a00640 1 -- 172.21.15.22:0/2251488246 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f79b4027cb0 con 0x55a297fa4080 2024-03-28T18:07:56.862 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:56.859+0000 7f79a2a00640 1 -- 172.21.15.22:0/2251488246 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 27) v1 ==== 7934+0+0 (secure 0 0 0) 0x7f79a8004bc0 con 0x55a297fa4080 2024-03-28T18:07:56.862 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:56.859+0000 7f79c6438740 10 client.15744 fetch_fsmap finished waiting for FSMap version 27 2024-03-28T18:07:56.862 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:56.859+0000 7f79c6438740 10 client.15744 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-28T18:07:56.862 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:56.859+0000 7f79c6438740 20 client.15744 populate_metadata read hostname 'smithi022' 2024-03-28T18:07:56.862 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:56.859+0000 7f79c6438740 1 --2- 172.21.15.22:0/2251488246 >> [v2:172.21.15.79:6838/2441542443,v1:172.21.15.79:6839/2441542443] conn(0x55a297fad580 0x55a29801c8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:07:56.862 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:56.859+0000 7f79c6438740 4 client.15744 mds_command: new command op to 34553 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-03-28T18:07:56.862 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:56.859+0000 7f79c6438740 1 -- 172.21.15.22:0/2251488246 --> [v2:172.21.15.79:6838/2441542443,v1:172.21.15.79:6839/2441542443] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x55a297f10740 con 0x55a297fad580 2024-03-28T18:07:56.863 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:56.860+0000 7f79c0e00640 1 --2- 172.21.15.22:0/2251488246 >> [v2:172.21.15.79:6838/2441542443,v1:172.21.15.79:6839/2441542443] conn(0x55a297fad580 0x55a29801c8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:07:56.864 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:56.860+0000 7f79c0e00640 1 --2- 172.21.15.22:0/2251488246 >> [v2:172.21.15.79:6838/2441542443,v1:172.21.15.79:6839/2441542443] conn(0x55a297fad580 0x55a29801c8e0 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bc4cd4933effbb7a server_cookie=c5d9b8109a2301b4 in_seq=0 out_seq=0 2024-03-28T18:07:56.864 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:56.860+0000 7f79a2a00640 10 client.15744 ms_handle_connect on v2:172.21.15.79:6838/2441542443 2024-03-28T18:07:56.864 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:56.862+0000 7f79a2a00640 1 -- 172.21.15.22:0/2251488246 <== mds.0 v2:172.21.15.79:6838/2441542443 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x55a297f10740 con 0x55a297fad580 2024-03-28T18:07:56.864 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:56.862+0000 7f79a2a00640 10 client.15744 handle_command_reply: tid=0 2024-03-28T18:07:56.864 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:56.862+0000 7f79c6438740 1 client.15744 shutdown 2024-03-28T18:07:56.864 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:56.862+0000 7f79c6438740 1 -- 172.21.15.22:0/2251488246 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a297fa4080 msgr2=0x55a297fb2cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:07:56.864 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:56.862+0000 7f79c6438740 1 --2- 172.21.15.22:0/2251488246 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a297fa4080 0x55a297fb2cb0 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x55a297f9f940 tx=0x7f79a800df80 comp rx=0 tx=0).stop 2024-03-28T18:07:56.864 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:56.862+0000 7f79c6438740 1 -- 172.21.15.22:0/2251488246 shutdown_connections 2024-03-28T18:07:56.864 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:56.862+0000 7f79c6438740 1 --2- 172.21.15.22:0/2251488246 >> [v2:172.21.15.79:6838/2441542443,v1:172.21.15.79:6839/2441542443] conn(0x55a297fad580 0x55a29801c8e0 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:07:56.864 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:56.862+0000 7f79c6438740 1 --2- 172.21.15.22:0/2251488246 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55a297fa49a0 0x55a297fb31f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:07:56.864 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:56.862+0000 7f79c6438740 1 --2- 172.21.15.22:0/2251488246 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a297fa4080 0x55a297fb2cb0 unknown :-1 s=CLOSED pgs=507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:07:56.864 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:56.862+0000 7f79c6438740 1 --2- 172.21.15.22:0/2251488246 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a297f9e7b0 0x55a297fb2770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:07:56.864 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:56.862+0000 7f79a2a00640 0 client.15744 ms_handle_reset on v2:172.21.15.79:6838/2441542443 2024-03-28T18:07:56.864 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:56.862+0000 7f79c6438740 1 -- 172.21.15.22:0/2251488246 >> 172.21.15.22:0/2251488246 conn(0x55a297faa860 msgr2=0x55a297fa5750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:07:56.864 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:56.862+0000 7f79c6438740 1 -- 172.21.15.22:0/2251488246 shutdown_connections 2024-03-28T18:07:56.865 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:56.862+0000 7f79c6438740 1 -- 172.21.15.22:0/2251488246 wait complete. 2024-03-28T18:07:56.865 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:56.863+0000 7f79c6438740 20 client.15744 trim_cache size 0 max 16384 2024-03-28T18:07:56.865 INFO:teuthology.orchestra.run.smithi022.stdout:{ 2024-03-28T18:07:56.865 INFO:teuthology.orchestra.run.smithi022.stdout: "return_code": 0, 2024-03-28T18:07:56.865 INFO:teuthology.orchestra.run.smithi022.stdout: "scrub_tag": "d363fb2a-8987-4dd0-a598-698886fa3467", 2024-03-28T18:07:56.865 INFO:teuthology.orchestra.run.smithi022.stdout: "mode": "asynchronous" 2024-03-28T18:07:56.865 INFO:teuthology.orchestra.run.smithi022.stdout:} 2024-03-28T18:07:56.866 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:56.863+0000 7f79c3800640 1 -- 172.21.15.22:0/1829866630 >> [v2:172.21.15.22:6800/1357365082,v1:172.21.15.22:6801/1357365082] conn(0x7f79a40a2a80 msgr2=0x7f79a40a4f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:07:56.866 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:56.863+0000 7f79c3800640 1 --2- 172.21.15.22:0/1829866630 >> [v2:172.21.15.22:6800/1357365082,v1:172.21.15.22:6801/1357365082] conn(0x7f79a40a2a80 0x7f79a40a4f40 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f79b4005fd0 tx=0x7f79b4009990 comp rx=0 tx=0).stop 2024-03-28T18:07:56.866 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:56.863+0000 7f79c3800640 1 -- 172.21.15.22:0/1829866630 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f79bc103840 msgr2=0x7f79bc19a190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:07:56.866 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:56.863+0000 7f79c3800640 1 --2- 172.21.15.22:0/1829866630 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f79bc103840 0x7f79bc19a190 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f79b0000c00 tx=0x7f79b0005020 comp rx=0 tx=0).stop 2024-03-28T18:07:56.866 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:56.863+0000 7f79c3800640 1 -- 172.21.15.22:0/1829866630 shutdown_connections 2024-03-28T18:07:56.866 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:56.863+0000 7f79c3800640 1 --2- 172.21.15.22:0/1829866630 >> [v2:172.21.15.22:6800/1357365082,v1:172.21.15.22:6801/1357365082] conn(0x7f79a40a2a80 0x7f79a40a4f40 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:07:56.866 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:56.863+0000 7f79c3800640 1 --2- 172.21.15.22:0/1829866630 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f79bc109850 0x7f79bc19ea50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:07:56.866 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:56.863+0000 7f79c3800640 1 --2- 172.21.15.22:0/1829866630 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f79bc104200 0x7f79bc19a6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:07:56.866 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:56.863+0000 7f79c3800640 1 --2- 172.21.15.22:0/1829866630 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f79bc103840 0x7f79bc19a190 unknown :-1 s=CLOSED pgs=503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:07:56.867 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:56.863+0000 7f79c3800640 1 -- 172.21.15.22:0/1829866630 >> 172.21.15.22:0/1829866630 conn(0x7f79bc0ff350 msgr2=0x7f79bc0ffa70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:07:56.867 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:56.864+0000 7f79c3800640 1 -- 172.21.15.22:0/1829866630 shutdown_connections 2024-03-28T18:07:56.867 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:56.864+0000 7f79c3800640 1 -- 172.21.15.22:0/1829866630 wait complete. 2024-03-28T18:07:56.883 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-28T18:07:56.883 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-28T18:07:56.964 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:07:56 smithi022 ceph-mon[31985]: pgmap v14: 97 pgs: 97 active+clean; 1.3 MiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 597 B/s wr, 12 op/s 2024-03-28T18:07:57.109 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:57.108+0000 7f8512200640 1 -- 172.21.15.22:0/3353829560 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f850c101610 msgr2=0x7f850c101a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:07:57.109 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:57.108+0000 7f8512200640 1 --2- 172.21.15.22:0/3353829560 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f850c101610 0x7f850c101a90 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f84f8009900 tx=0x7f84f802bc40 comp rx=0 tx=0).stop 2024-03-28T18:07:57.110 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:57.108+0000 7f8512200640 1 -- 172.21.15.22:0/3353829560 shutdown_connections 2024-03-28T18:07:57.110 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:57.108+0000 7f8512200640 1 --2- 172.21.15.22:0/3353829560 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f850c101fd0 0x7f850c10e4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:07:57.110 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:57.108+0000 7f8512200640 1 --2- 172.21.15.22:0/3353829560 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f850c101610 0x7f850c101a90 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:07:57.110 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:57.108+0000 7f8512200640 1 --2- 172.21.15.22:0/3353829560 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f850c1076b0 0x7f850c107a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:07:57.110 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:57.108+0000 7f8512200640 1 -- 172.21.15.22:0/3353829560 >> 172.21.15.22:0/3353829560 conn(0x7f850c0fd100 msgr2=0x7f850c0ff520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:07:57.110 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:57.108+0000 7f8512200640 1 -- 172.21.15.22:0/3353829560 shutdown_connections 2024-03-28T18:07:57.110 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:57.108+0000 7f8512200640 1 -- 172.21.15.22:0/3353829560 wait complete. 2024-03-28T18:07:57.113 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:57.109+0000 7f8512200640 1 Processor -- start 2024-03-28T18:07:57.113 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:57.110+0000 7f8512200640 1 -- start start 2024-03-28T18:07:57.113 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:57.110+0000 7f8512200640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f850c101610 0x7f850c197ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:07:57.113 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:57.110+0000 7f8512200640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f850c101fd0 0x7f850c198410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:07:57.113 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:57.110+0000 7f8512200640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f850c1076b0 0x7f850c19c790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:07:57.113 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:57.110+0000 7f8512200640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f850c111a10 con 0x7f850c1076b0 2024-03-28T18:07:57.113 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:57.110+0000 7f8512200640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f850c111890 con 0x7f850c101610 2024-03-28T18:07:57.113 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:57.110+0000 7f8512200640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f850c111b90 con 0x7f850c101fd0 2024-03-28T18:07:57.114 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:57.110+0000 7f850be00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f850c1076b0 0x7f850c19c790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:07:57.114 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:57.111+0000 7f850b400640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f850c101610 0x7f850c197ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:07:57.114 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:57.111+0000 7f850be00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f850c1076b0 0x7f850c19c790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:55002/0 (socket says 172.21.15.22:55002) 2024-03-28T18:07:57.114 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:57.111+0000 7f850be00640 1 -- 172.21.15.22:0/485726337 learned_addr learned my addr 172.21.15.22:0/485726337 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-28T18:07:57.114 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:57.111+0000 7f850aa00640 1 --2- 172.21.15.22:0/485726337 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f850c101fd0 0x7f850c198410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:07:57.114 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:57.111+0000 7f850be00640 1 -- 172.21.15.22:0/485726337 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f850c101fd0 msgr2=0x7f850c198410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:07:57.114 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:57.111+0000 7f850be00640 1 --2- 172.21.15.22:0/485726337 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f850c101fd0 0x7f850c198410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:07:57.114 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:57.111+0000 7f850be00640 1 -- 172.21.15.22:0/485726337 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f850c101610 msgr2=0x7f850c197ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:07:57.114 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:57.111+0000 7f850be00640 1 --2- 172.21.15.22:0/485726337 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f850c101610 0x7f850c197ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:07:57.114 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:57.111+0000 7f850be00640 1 -- 172.21.15.22:0/485726337 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f850c19cec0 con 0x7f850c1076b0 2024-03-28T18:07:57.114 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:57.111+0000 7f850aa00640 1 --2- 172.21.15.22:0/485726337 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f850c101fd0 0x7f850c198410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-28T18:07:57.114 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:57.111+0000 7f850be00640 1 --2- 172.21.15.22:0/485726337 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f850c1076b0 0x7f850c19c790 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f850400e970 tx=0x7f850400ed70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:07:57.114 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:57.112+0000 7f8503e00640 1 -- 172.21.15.22:0/485726337 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f850400ccc0 con 0x7f850c1076b0 2024-03-28T18:07:57.114 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:57.112+0000 7f8503e00640 1 -- 172.21.15.22:0/485726337 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f850400ce60 con 0x7f850c1076b0 2024-03-28T18:07:57.114 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:57.112+0000 7f8512200640 1 -- 172.21.15.22:0/485726337 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f850c19d1b0 con 0x7f850c1076b0 2024-03-28T18:07:57.115 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:57.112+0000 7f8503e00640 1 -- 172.21.15.22:0/485726337 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8504010430 con 0x7f850c1076b0 2024-03-28T18:07:57.115 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:57.112+0000 7f8512200640 1 -- 172.21.15.22:0/485726337 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f850c1a4a40 con 0x7f850c1076b0 2024-03-28T18:07:57.117 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:57.115+0000 7f8503e00640 1 -- 172.21.15.22:0/485726337 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 24) v1 ==== 137466+0+0 (secure 0 0 0) 0x7f8504005000 con 0x7f850c1076b0 2024-03-28T18:07:57.117 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:57.116+0000 7f8514ffa740 1 Processor -- start 2024-03-28T18:07:57.117 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:57.116+0000 7f8514ffa740 1 -- start start 2024-03-28T18:07:57.117 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:57.116+0000 7f8514ffa740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f7170fcc00 0x55f7170fb950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:07:57.117 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:57.116+0000 7f8514ffa740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f7170fbf20 0x55f7170fc3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:07:57.117 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:57.116+0000 7f8514ffa740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55f7171096a0 0x55f71710ba90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:07:57.118 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:57.116+0000 7f8514ffa740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f717060250 con 0x55f7170fcc00 2024-03-28T18:07:57.118 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:57.116+0000 7f8514ffa740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f717008a30 con 0x55f7171096a0 2024-03-28T18:07:57.118 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:57.116+0000 7f8514ffa740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f71705c2f0 con 0x55f7170fbf20 2024-03-28T18:07:57.118 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:57.116+0000 7f850aa00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f7170fcc00 0x55f7170fb950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:07:57.118 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:57.116+0000 7f850aa00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f7170fcc00 0x55f7170fb950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:55014/0 (socket says 172.21.15.22:55014) 2024-03-28T18:07:57.118 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:57.116+0000 7f850aa00640 1 -- 172.21.15.22:0/2147773839 learned_addr learned my addr 172.21.15.22:0/2147773839 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-28T18:07:57.118 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:57.117+0000 7f850aa00640 1 --2- 172.21.15.22:0/2147773839 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55f7171096a0 0x55f71710ba90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:07:57.118 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:57.117+0000 7f850b400640 1 --2- 172.21.15.22:0/2147773839 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f7170fbf20 0x55f7170fc3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:07:57.123 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:57.117+0000 7f850aa00640 1 -- 172.21.15.22:0/2147773839 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f7170fbf20 msgr2=0x55f7170fc3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T18:07:57.123 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:57.117+0000 7f850aa00640 1 --2- 172.21.15.22:0/2147773839 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f7170fbf20 0x55f7170fc3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:07:57.123 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:57.117+0000 7f850aa00640 1 -- 172.21.15.22:0/2147773839 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55f7171096a0 msgr2=0x55f71710ba90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T18:07:57.123 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:57.117+0000 7f850aa00640 1 --2- 172.21.15.22:0/2147773839 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55f7171096a0 0x55f71710ba90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:07:57.124 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:57.117+0000 7f850aa00640 1 -- 172.21.15.22:0/2147773839 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f716ff3690 con 0x55f7170fcc00 2024-03-28T18:07:57.124 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:57.117+0000 7f850b400640 1 --2- 172.21.15.22:0/2147773839 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f7170fbf20 0x55f7170fc3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-28T18:07:57.124 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:57.117+0000 7f850aa00640 1 --2- 172.21.15.22:0/2147773839 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f7170fcc00 0x55f7170fb950 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f84f802c250 tx=0x7f84f8005bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a50ee33a54971343 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:07:57.124 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:57.118+0000 7f84f7e00640 1 -- 172.21.15.22:0/2147773839 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84f804a070 con 0x55f7170fcc00 2024-03-28T18:07:57.124 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:57.118+0000 7f84f7e00640 1 -- 172.21.15.22:0/2147773839 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f84f8005500 con 0x55f7170fcc00 2024-03-28T18:07:57.124 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:57.118+0000 7f84f7e00640 1 -- 172.21.15.22:0/2147773839 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84f8004b00 con 0x55f7170fcc00 2024-03-28T18:07:57.124 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:57.118+0000 7f8503e00640 1 --2- 172.21.15.22:0/485726337 >> [v2:172.21.15.22:6800/1357365082,v1:172.21.15.22:6801/1357365082] conn(0x7f84f00a2a00 0x7f84f00a4ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:07:57.124 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:57.118+0000 7f8503e00640 1 -- 172.21.15.22:0/485726337 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 8411+0+0 (secure 0 0 0) 0x7f85040cf4a0 con 0x7f850c1076b0 2024-03-28T18:07:57.124 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:57.118+0000 7f850b400640 1 --2- 172.21.15.22:0/485726337 >> [v2:172.21.15.22:6800/1357365082,v1:172.21.15.22:6801/1357365082] conn(0x7f84f00a2a00 0x7f84f00a4ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:07:57.124 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:57.118+0000 7f8514ffa740 1 -- 172.21.15.22:0/2147773839 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f7170fcc00 msgr2=0x55f7170fb950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:07:57.124 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:57.118+0000 7f8514ffa740 1 --2- 172.21.15.22:0/2147773839 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f7170fcc00 0x55f7170fb950 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f84f802c250 tx=0x7f84f8005bc0 comp rx=0 tx=0).stop 2024-03-28T18:07:57.124 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:57.119+0000 7f8514ffa740 1 -- 172.21.15.22:0/2147773839 shutdown_connections 2024-03-28T18:07:57.124 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:57.119+0000 7f8514ffa740 1 --2- 172.21.15.22:0/2147773839 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55f7171096a0 0x55f71710ba90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:07:57.124 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:57.119+0000 7f8514ffa740 1 --2- 172.21.15.22:0/2147773839 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f7170fbf20 0x55f7170fc3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:07:57.125 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:57.119+0000 7f8514ffa740 1 --2- 172.21.15.22:0/2147773839 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f7170fcc00 0x55f7170fb950 unknown :-1 s=CLOSED pgs=509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:07:57.125 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:57.119+0000 7f8514ffa740 1 -- 172.21.15.22:0/2147773839 >> 172.21.15.22:0/2147773839 conn(0x55f7170c3200 msgr2=0x55f7170f6970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:07:57.125 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:57.119+0000 7f8514ffa740 1 -- 172.21.15.22:0/2147773839 shutdown_connections 2024-03-28T18:07:57.125 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:57.119+0000 7f8514ffa740 1 -- 172.21.15.22:0/2147773839 wait complete. 2024-03-28T18:07:57.125 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:57.119+0000 7f850b400640 1 --2- 172.21.15.22:0/485726337 >> [v2:172.21.15.22:6800/1357365082,v1:172.21.15.22:6801/1357365082] conn(0x7f84f00a2a00 0x7f84f00a4ec0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x55f7170fd150 tx=0x7f84fc009670 comp rx=0 tx=0).ready entity=mgr.15522 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:07:57.125 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:57.119+0000 7f8514ffa740 1 -- start start 2024-03-28T18:07:57.125 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:57.119+0000 7f8514ffa740 1 Processor -- start 2024-03-28T18:07:57.125 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:57.120+0000 7f8514ffa740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f7170fbf20 0x55f717108d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:07:57.125 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:57.120+0000 7f8514ffa740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55f7170fcc00 0x55f7171060f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:07:57.125 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:57.120+0000 7f8514ffa740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f7171096a0 0x55f717106630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:07:57.125 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:57.120+0000 7f8514ffa740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f71710c4b0 con 0x55f7171096a0 2024-03-28T18:07:57.125 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:57.120+0000 7f8514ffa740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f71701be80 con 0x55f7170fcc00 2024-03-28T18:07:57.125 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:57.120+0000 7f8514ffa740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f71710c630 con 0x55f7170fbf20 2024-03-28T18:07:57.126 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:57.120+0000 7f8514ffa740 10 client.0 _ll_register_callbacks cb 0x55f71702e130 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-28T18:07:57.126 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:57.120+0000 7f850be00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f7171096a0 0x55f717106630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:07:57.126 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:57.120+0000 7f850be00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f7171096a0 0x55f717106630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:55028/0 (socket says 172.21.15.22:55028) 2024-03-28T18:07:57.126 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:57.120+0000 7f850be00640 1 -- 172.21.15.22:0/1556119565 learned_addr learned my addr 172.21.15.22:0/1556119565 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-28T18:07:57.126 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:57.120+0000 7f850aa00640 1 --2- 172.21.15.22:0/1556119565 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55f7170fcc00 0x55f7171060f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:07:57.126 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:57.121+0000 7f850b400640 1 --2- 172.21.15.22:0/1556119565 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f7170fbf20 0x55f717108d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:07:57.126 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:57.121+0000 7f850be00640 1 -- 172.21.15.22:0/1556119565 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f7170fbf20 msgr2=0x55f717108d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T18:07:57.126 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:57.121+0000 7f850be00640 1 --2- 172.21.15.22:0/1556119565 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f7170fbf20 0x55f717108d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:07:57.126 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:57.121+0000 7f850be00640 1 -- 172.21.15.22:0/1556119565 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55f7170fcc00 msgr2=0x55f7171060f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T18:07:57.126 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:57.121+0000 7f850be00640 1 --2- 172.21.15.22:0/1556119565 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55f7170fcc00 0x55f7171060f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:07:57.126 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:57.121+0000 7f850be00640 1 -- 172.21.15.22:0/1556119565 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55f716e3e7d0 con 0x55f7171096a0 2024-03-28T18:07:57.126 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:57.121+0000 7f850aa00640 1 --2- 172.21.15.22:0/1556119565 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55f7170fcc00 0x55f7171060f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-28T18:07:57.126 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:57.121+0000 7f850be00640 1 --2- 172.21.15.22:0/1556119565 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f7171096a0 0x55f717106630 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f850409b6a0 tx=0x7f850409c1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2bf2d33a7648aeca server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:07:57.127 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:57.121+0000 7f84f4c00640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-03-28T18:07:57.127 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:57.121+0000 7f84f4c00640 1 -- 172.21.15.22:0/1556119565 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85040a4140 con 0x55f7171096a0 2024-03-28T18:07:57.127 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:57.121+0000 7f84f4c00640 1 -- 172.21.15.22:0/1556119565 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 8411+0+0 (secure 0 0 0) 0x7f85040ccdf0 con 0x55f7171096a0 2024-03-28T18:07:57.127 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:57.121+0000 7f8514ffa740 1 -- 172.21.15.22:0/1556119565 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f84f8005500 con 0x55f7171096a0 2024-03-28T18:07:57.127 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:57.122+0000 7f84f4c00640 1 -- 172.21.15.22:0/1556119565 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f85040ccff0 con 0x55f7171096a0 2024-03-28T18:07:57.127 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:57.122+0000 7f8514ffa740 10 client.15756 fetch_fsmap learned FSMap version 27 2024-03-28T18:07:57.127 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:57.122+0000 7f8514ffa740 1 -- 172.21.15.22:0/1556119565 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55f716d743f0 con 0x55f7171096a0 2024-03-28T18:07:57.127 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:57.122+0000 7f84f4c00640 1 -- 172.21.15.22:0/1556119565 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f84f8005500 con 0x55f7171096a0 2024-03-28T18:07:57.127 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:57.122+0000 7f84f4c00640 1 -- 172.21.15.22:0/1556119565 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 27) v1 ==== 7934+0+0 (secure 0 0 0) 0x7f85040a3cb0 con 0x55f7171096a0 2024-03-28T18:07:57.127 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:57.122+0000 7f8514ffa740 10 client.15756 fetch_fsmap finished waiting for FSMap version 27 2024-03-28T18:07:57.127 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:57.122+0000 7f8514ffa740 10 client.15756 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-28T18:07:57.127 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:57.122+0000 7f8514ffa740 20 client.15756 populate_metadata read hostname 'smithi022' 2024-03-28T18:07:57.127 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:57.122+0000 7f8514ffa740 1 --2- 172.21.15.22:0/1556119565 >> [v2:172.21.15.79:6838/2441542443,v1:172.21.15.79:6839/2441542443] conn(0x55f7171195a0 0x55f71711b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:07:57.127 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:57.122+0000 7f8514ffa740 4 client.15756 mds_command: new command op to 34553 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-28T18:07:57.127 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:57.122+0000 7f8514ffa740 1 -- 172.21.15.22:0/1556119565 --> [v2:172.21.15.79:6838/2441542443,v1:172.21.15.79:6839/2441542443] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55f71706d790 con 0x55f7171195a0 2024-03-28T18:07:57.128 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:57.123+0000 7f850b400640 1 --2- 172.21.15.22:0/1556119565 >> [v2:172.21.15.79:6838/2441542443,v1:172.21.15.79:6839/2441542443] conn(0x55f7171195a0 0x55f71711b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:07:57.129 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:57.123+0000 7f850b400640 1 --2- 172.21.15.22:0/1556119565 >> [v2:172.21.15.79:6838/2441542443,v1:172.21.15.79:6839/2441542443] conn(0x55f7171195a0 0x55f71711b990 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8bdc1c0e8ffde2b2 server_cookie=fa192eb42e31b05a in_seq=0 out_seq=0 2024-03-28T18:07:57.129 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:57.123+0000 7f84f4c00640 10 client.15756 ms_handle_connect on v2:172.21.15.79:6838/2441542443 2024-03-28T18:07:57.129 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:57.125+0000 7f84f4c00640 1 -- 172.21.15.22:0/1556119565 <== mds.0 v2:172.21.15.79:6838/2441542443 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x55f71706d790 con 0x55f7171195a0 2024-03-28T18:07:57.129 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:57.125+0000 7f84f4c00640 10 client.15756 handle_command_reply: tid=0 2024-03-28T18:07:57.129 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:57.125+0000 7f8514ffa740 1 client.15756 shutdown 2024-03-28T18:07:57.129 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:57.125+0000 7f8514ffa740 1 -- 172.21.15.22:0/1556119565 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f7171096a0 msgr2=0x55f717106630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:07:57.129 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:57.125+0000 7f8514ffa740 1 --2- 172.21.15.22:0/1556119565 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f7171096a0 0x55f717106630 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f850409b6a0 tx=0x7f850409c1e0 comp rx=0 tx=0).stop 2024-03-28T18:07:57.129 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:57.125+0000 7f8514ffa740 1 -- 172.21.15.22:0/1556119565 shutdown_connections 2024-03-28T18:07:57.129 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:57.125+0000 7f8514ffa740 1 --2- 172.21.15.22:0/1556119565 >> [v2:172.21.15.79:6838/2441542443,v1:172.21.15.79:6839/2441542443] conn(0x55f7171195a0 0x55f71711b990 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:07:57.130 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:57.125+0000 7f8514ffa740 1 --2- 172.21.15.22:0/1556119565 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f7171096a0 0x55f717106630 unknown :-1 s=CLOSED pgs=510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:07:57.130 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:57.125+0000 7f8514ffa740 1 --2- 172.21.15.22:0/1556119565 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55f7170fcc00 0x55f7171060f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:07:57.130 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:57.125+0000 7f84f4c00640 0 client.15756 ms_handle_reset on v2:172.21.15.79:6838/2441542443 2024-03-28T18:07:57.130 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:57.125+0000 7f8514ffa740 1 --2- 172.21.15.22:0/1556119565 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f7170fbf20 0x55f717108d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:07:57.130 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:57.125+0000 7f8514ffa740 1 -- 172.21.15.22:0/1556119565 >> 172.21.15.22:0/1556119565 conn(0x55f7170c3200 msgr2=0x55f7170f7e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:07:57.130 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:57.126+0000 7f8514ffa740 1 -- 172.21.15.22:0/1556119565 shutdown_connections 2024-03-28T18:07:57.130 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:57.126+0000 7f8514ffa740 1 -- 172.21.15.22:0/1556119565 wait complete. 2024-03-28T18:07:57.130 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:57.126+0000 7f8514ffa740 20 client.15756 trim_cache size 0 max 16384 2024-03-28T18:07:57.144 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:57.143+0000 7f8514ffa740 1 Processor -- start 2024-03-28T18:07:57.144 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:57.144+0000 7f8514ffa740 1 -- start start 2024-03-28T18:07:57.144 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:57.144+0000 7f8514ffa740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f7170fbf20 0x55f7170f8460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:07:57.145 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:57.144+0000 7f8514ffa740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f7170fcc00 0x55f717108820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:07:57.145 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:57.144+0000 7f8514ffa740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55f7171096a0 0x55f717108d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:07:57.145 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:57.144+0000 7f8514ffa740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f850c1aad10 con 0x55f7170fcc00 2024-03-28T18:07:57.145 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:57.144+0000 7f8514ffa740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f850c1ac690 con 0x55f7171096a0 2024-03-28T18:07:57.145 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:57.144+0000 7f8514ffa740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f850c1ab370 con 0x55f7170fbf20 2024-03-28T18:07:57.146 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:57.144+0000 7f850be00640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55f7171096a0 0x55f717108d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:07:57.146 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:57.144+0000 7f850aa00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f7170fcc00 0x55f717108820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:07:57.146 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:57.144+0000 7f850be00640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55f7171096a0 0x55f717108d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.22:46186/0 (socket says 172.21.15.22:46186) 2024-03-28T18:07:57.146 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:57.144+0000 7f850be00640 1 -- 172.21.15.22:0/1141329735 learned_addr learned my addr 172.21.15.22:0/1141329735 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-28T18:07:57.147 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:57.145+0000 7f850b400640 1 --2- 172.21.15.22:0/1141329735 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f7170fbf20 0x55f7170f8460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:07:57.147 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:57.145+0000 7f850be00640 1 -- 172.21.15.22:0/1141329735 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f7170fbf20 msgr2=0x55f7170f8460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T18:07:57.147 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:57.145+0000 7f850be00640 1 --2- 172.21.15.22:0/1141329735 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f7170fbf20 0x55f7170f8460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:07:57.147 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:57.145+0000 7f850be00640 1 -- 172.21.15.22:0/1141329735 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f7170fcc00 msgr2=0x55f717108820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T18:07:57.147 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:57.145+0000 7f850be00640 1 --2- 172.21.15.22:0/1141329735 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f7170fcc00 0x55f717108820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:07:57.147 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:57.145+0000 7f850be00640 1 -- 172.21.15.22:0/1141329735 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f716d340f0 con 0x55f7171096a0 2024-03-28T18:07:57.147 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:57.145+0000 7f850b400640 1 --2- 172.21.15.22:0/1141329735 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f7170fbf20 0x55f7170f8460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T18:07:57.147 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:57.145+0000 7f850aa00640 1 --2- 172.21.15.22:0/1141329735 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f7170fcc00 0x55f717108820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-28T18:07:57.147 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:57.145+0000 7f850be00640 1 --2- 172.21.15.22:0/1141329735 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55f7171096a0 0x55f717108d60 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f8504097e00 tx=0x7f850409b4e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e70887d8a5fecd33 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:07:57.147 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:57.145+0000 7f84f7400640 1 -- 172.21.15.22:0/1141329735 <== mon.2 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85040863a0 con 0x55f7171096a0 2024-03-28T18:07:57.147 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:57.146+0000 7f84f7400640 1 -- 172.21.15.22:0/1141329735 <== mon.2 v2:172.21.15.79:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f85040a4140 con 0x55f7171096a0 2024-03-28T18:07:57.147 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:57.146+0000 7f84f7400640 1 -- 172.21.15.22:0/1141329735 <== mon.2 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85040a3cb0 con 0x55f7171096a0 2024-03-28T18:07:57.150 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:57.146+0000 7f8514ffa740 1 -- 172.21.15.22:0/1141329735 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55f7171096a0 msgr2=0x55f717108d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:07:57.150 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:57.146+0000 7f8514ffa740 1 --2- 172.21.15.22:0/1141329735 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55f7171096a0 0x55f717108d60 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f8504097e00 tx=0x7f850409b4e0 comp rx=0 tx=0).stop 2024-03-28T18:07:57.150 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:57.146+0000 7f8514ffa740 1 -- 172.21.15.22:0/1141329735 shutdown_connections 2024-03-28T18:07:57.150 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:57.146+0000 7f8514ffa740 1 --2- 172.21.15.22:0/1141329735 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55f7171096a0 0x55f717108d60 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:07:57.151 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:57.146+0000 7f8514ffa740 1 --2- 172.21.15.22:0/1141329735 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f7170fcc00 0x55f717108820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:07:57.151 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:57.146+0000 7f8514ffa740 1 --2- 172.21.15.22:0/1141329735 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f7170fbf20 0x55f7170f8460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:07:57.151 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:57.146+0000 7f8514ffa740 1 -- 172.21.15.22:0/1141329735 >> 172.21.15.22:0/1141329735 conn(0x55f7170c3200 msgr2=0x55f7170f7d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:07:57.151 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:57.146+0000 7f8514ffa740 1 -- 172.21.15.22:0/1141329735 shutdown_connections 2024-03-28T18:07:57.151 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:57.146+0000 7f8514ffa740 1 -- 172.21.15.22:0/1141329735 wait complete. 2024-03-28T18:07:57.151 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:57.147+0000 7f8514ffa740 1 -- start start 2024-03-28T18:07:57.151 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:57.147+0000 7f8514ffa740 1 Processor -- start 2024-03-28T18:07:57.151 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:57.147+0000 7f8514ffa740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f7170fbf20 0x55f7171000e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:07:57.151 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:57.147+0000 7f8514ffa740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f7170fcc00 0x55f717100620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:07:57.151 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:57.147+0000 7f8514ffa740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55f7171096a0 0x55f71710f9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:07:57.151 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:57.147+0000 7f8514ffa740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f850c1ac030 con 0x55f7170fbf20 2024-03-28T18:07:57.151 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:57.147+0000 7f8514ffa740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f850c1ab9d0 con 0x55f7171096a0 2024-03-28T18:07:57.151 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:57.147+0000 7f8514ffa740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f850c1a8ed0 con 0x55f7170fcc00 2024-03-28T18:07:57.151 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:57.147+0000 7f8514ffa740 10 client.0 _ll_register_callbacks cb 0x7f85040cc9d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-28T18:07:57.152 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:57.148+0000 7f850b400640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f7170fbf20 0x55f7171000e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:07:57.152 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:57.148+0000 7f850b400640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f7170fbf20 0x55f7171000e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:55036/0 (socket says 172.21.15.22:55036) 2024-03-28T18:07:57.152 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:57.148+0000 7f850b400640 1 -- 172.21.15.22:0/1790546848 learned_addr learned my addr 172.21.15.22:0/1790546848 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-28T18:07:57.152 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:57.148+0000 7f850be00640 1 --2- 172.21.15.22:0/1790546848 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55f7171096a0 0x55f71710f9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:07:57.152 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:57.148+0000 7f850b400640 1 -- 172.21.15.22:0/1790546848 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f7170fcc00 msgr2=0x55f717100620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T18:07:57.152 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:57.148+0000 7f850b400640 1 --2- 172.21.15.22:0/1790546848 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f7170fcc00 0x55f717100620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:07:57.152 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:57.148+0000 7f850b400640 1 -- 172.21.15.22:0/1790546848 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55f7171096a0 msgr2=0x55f71710f9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T18:07:57.152 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:57.148+0000 7f850b400640 1 --2- 172.21.15.22:0/1790546848 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55f7171096a0 0x55f71710f9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:07:57.152 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:57.148+0000 7f850b400640 1 -- 172.21.15.22:0/1790546848 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55f71710ff30 con 0x55f7170fbf20 2024-03-28T18:07:57.152 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:57.148+0000 7f850be00640 1 --2- 172.21.15.22:0/1790546848 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55f7171096a0 0x55f71710f9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-28T18:07:57.152 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:57.148+0000 7f850b400640 1 --2- 172.21.15.22:0/1790546848 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f7170fbf20 0x55f7171000e0 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x55f71711c1c0 tx=0x7f84fc004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=72e8e58f64cc8818 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:07:57.152 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:57.148+0000 7f84f4c00640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-03-28T18:07:57.152 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:57.149+0000 7f84f4c00640 1 -- 172.21.15.22:0/1790546848 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84fc02a270 con 0x55f7170fbf20 2024-03-28T18:07:57.152 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:57.149+0000 7f84f4c00640 1 -- 172.21.15.22:0/1790546848 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 8411+0+0 (secure 0 0 0) 0x7f84fc01c7d0 con 0x55f7170fbf20 2024-03-28T18:07:57.152 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:57.149+0000 7f8514ffa740 1 -- 172.21.15.22:0/1790546848 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f85040a4140 con 0x55f7170fbf20 2024-03-28T18:07:57.153 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:57.149+0000 7f84f4c00640 1 -- 172.21.15.22:0/1790546848 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f84fc038050 con 0x55f7170fbf20 2024-03-28T18:07:57.153 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:57.149+0000 7f8514ffa740 10 client.15762 fetch_fsmap learned FSMap version 27 2024-03-28T18:07:57.153 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:57.149+0000 7f8514ffa740 1 -- 172.21.15.22:0/1790546848 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55f717179db0 con 0x55f7170fbf20 2024-03-28T18:07:57.153 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:57.149+0000 7f84f4c00640 1 -- 172.21.15.22:0/1790546848 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f85040a4140 con 0x55f7170fbf20 2024-03-28T18:07:57.153 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:57.149+0000 7f84f4c00640 1 -- 172.21.15.22:0/1790546848 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 27) v1 ==== 7934+0+0 (secure 0 0 0) 0x7f84fc00f040 con 0x55f7170fbf20 2024-03-28T18:07:57.153 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:57.149+0000 7f8514ffa740 10 client.15762 fetch_fsmap finished waiting for FSMap version 27 2024-03-28T18:07:57.153 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:57.150+0000 7f8514ffa740 10 client.15762 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-28T18:07:57.153 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:57.150+0000 7f8514ffa740 20 client.15762 populate_metadata read hostname 'smithi022' 2024-03-28T18:07:57.153 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:57.150+0000 7f8514ffa740 1 --2- 172.21.15.22:0/1790546848 >> [v2:172.21.15.79:6838/2441542443,v1:172.21.15.79:6839/2441542443] conn(0x55f71717a4e0 0x55f71717c900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:07:57.153 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:57.150+0000 7f8514ffa740 4 client.15762 mds_command: new command op to 34553 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-28T18:07:57.153 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:57.150+0000 7f8514ffa740 1 -- 172.21.15.22:0/1790546848 --> [v2:172.21.15.79:6838/2441542443,v1:172.21.15.79:6839/2441542443] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55f7171069c0 con 0x55f71717a4e0 2024-03-28T18:07:57.153 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:57.150+0000 7f850aa00640 1 --2- 172.21.15.22:0/1790546848 >> [v2:172.21.15.79:6838/2441542443,v1:172.21.15.79:6839/2441542443] conn(0x55f71717a4e0 0x55f71717c900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:07:57.154 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:57.151+0000 7f850aa00640 1 --2- 172.21.15.22:0/1790546848 >> [v2:172.21.15.79:6838/2441542443,v1:172.21.15.79:6839/2441542443] conn(0x55f71717a4e0 0x55f71717c900 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6697731aa7af86a3 server_cookie=60093576f906c281 in_seq=0 out_seq=0 2024-03-28T18:07:57.155 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:57.151+0000 7f84f4c00640 10 client.15762 ms_handle_connect on v2:172.21.15.79:6838/2441542443 2024-03-28T18:07:57.155 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:57.151+0000 7f84f4c00640 1 -- 172.21.15.22:0/1790546848 <== mds.0 v2:172.21.15.79:6838/2441542443 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x55f7171069c0 con 0x55f71717a4e0 2024-03-28T18:07:57.155 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:57.151+0000 7f84f4c00640 10 client.15762 handle_command_reply: tid=0 2024-03-28T18:07:57.155 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:57.151+0000 7f8514ffa740 1 client.15762 shutdown 2024-03-28T18:07:57.155 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:57.151+0000 7f8514ffa740 1 -- 172.21.15.22:0/1790546848 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f7170fbf20 msgr2=0x55f7171000e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:07:57.155 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:57.151+0000 7f8514ffa740 1 --2- 172.21.15.22:0/1790546848 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f7170fbf20 0x55f7171000e0 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x55f71711c1c0 tx=0x7f84fc004a40 comp rx=0 tx=0).stop 2024-03-28T18:07:57.155 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:57.151+0000 7f8514ffa740 1 -- 172.21.15.22:0/1790546848 shutdown_connections 2024-03-28T18:07:57.155 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:57.151+0000 7f8514ffa740 1 --2- 172.21.15.22:0/1790546848 >> [v2:172.21.15.79:6838/2441542443,v1:172.21.15.79:6839/2441542443] conn(0x55f71717a4e0 0x55f71717c900 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:07:57.155 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:57.151+0000 7f8514ffa740 1 --2- 172.21.15.22:0/1790546848 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55f7171096a0 0x55f71710f9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:07:57.155 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:57.151+0000 7f8514ffa740 1 --2- 172.21.15.22:0/1790546848 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f7170fcc00 0x55f717100620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:07:57.155 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:57.151+0000 7f84f4c00640 0 client.15762 ms_handle_reset on v2:172.21.15.79:6838/2441542443 2024-03-28T18:07:57.155 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:57.151+0000 7f8514ffa740 1 --2- 172.21.15.22:0/1790546848 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f7170fbf20 0x55f7171000e0 unknown :-1 s=CLOSED pgs=511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:07:57.155 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:57.151+0000 7f8514ffa740 1 -- 172.21.15.22:0/1790546848 >> 172.21.15.22:0/1790546848 conn(0x55f7170c3200 msgr2=0x55f7170f5a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:07:57.156 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:57.151+0000 7f8514ffa740 1 -- 172.21.15.22:0/1790546848 shutdown_connections 2024-03-28T18:07:57.156 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:57.152+0000 7f8514ffa740 1 -- 172.21.15.22:0/1790546848 wait complete. 2024-03-28T18:07:57.156 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:57.152+0000 7f8514ffa740 20 client.15762 trim_cache size 0 max 16384 2024-03-28T18:07:57.156 INFO:teuthology.orchestra.run.smithi022.stdout:{ 2024-03-28T18:07:57.156 INFO:teuthology.orchestra.run.smithi022.stdout: "status": "IDLE (waiting for more scrubs)", 2024-03-28T18:07:57.156 INFO:teuthology.orchestra.run.smithi022.stdout: "scrubs": { 2024-03-28T18:07:57.156 INFO:teuthology.orchestra.run.smithi022.stdout: "d363fb2a-8987-4dd0-a598-698886fa3467": { 2024-03-28T18:07:57.156 INFO:teuthology.orchestra.run.smithi022.stdout: "path": "/", 2024-03-28T18:07:57.156 INFO:teuthology.orchestra.run.smithi022.stdout: "tag": "d363fb2a-8987-4dd0-a598-698886fa3467", 2024-03-28T18:07:57.156 INFO:teuthology.orchestra.run.smithi022.stdout: "options": "recursive,force" 2024-03-28T18:07:57.156 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-03-28T18:07:57.157 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-03-28T18:07:57.157 INFO:teuthology.orchestra.run.smithi022.stdout:} 2024-03-28T18:07:57.157 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:57.152+0000 7f8512200640 1 -- 172.21.15.22:0/485726337 >> [v2:172.21.15.22:6800/1357365082,v1:172.21.15.22:6801/1357365082] conn(0x7f84f00a2a00 msgr2=0x7f84f00a4ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:07:57.157 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:57.152+0000 7f8512200640 1 --2- 172.21.15.22:0/485726337 >> [v2:172.21.15.22:6800/1357365082,v1:172.21.15.22:6801/1357365082] conn(0x7f84f00a2a00 0x7f84f00a4ec0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x55f7170fd150 tx=0x7f84fc009670 comp rx=0 tx=0).stop 2024-03-28T18:07:57.157 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:57.152+0000 7f8512200640 1 -- 172.21.15.22:0/485726337 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f850c1076b0 msgr2=0x7f850c19c790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:07:57.157 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:57.152+0000 7f8512200640 1 --2- 172.21.15.22:0/485726337 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f850c1076b0 0x7f850c19c790 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f850400e970 tx=0x7f850400ed70 comp rx=0 tx=0).stop 2024-03-28T18:07:57.158 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:57.152+0000 7f8512200640 1 -- 172.21.15.22:0/485726337 shutdown_connections 2024-03-28T18:07:57.158 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:57.152+0000 7f8512200640 1 --2- 172.21.15.22:0/485726337 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f850c1076b0 0x7f850c19c790 unknown :-1 s=CLOSED pgs=508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:07:57.158 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:57.152+0000 7f8512200640 1 --2- 172.21.15.22:0/485726337 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f850c101fd0 0x7f850c198410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:07:57.158 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:57.152+0000 7f8512200640 1 --2- 172.21.15.22:0/485726337 >> [v2:172.21.15.22:6800/1357365082,v1:172.21.15.22:6801/1357365082] conn(0x7f84f00a2a00 0x7f84f00a4ec0 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:07:57.158 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:57.152+0000 7f8512200640 1 --2- 172.21.15.22:0/485726337 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f850c101610 0x7f850c197ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:07:57.158 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:57.152+0000 7f8512200640 1 -- 172.21.15.22:0/485726337 >> 172.21.15.22:0/485726337 conn(0x7f850c0fd100 msgr2=0x7f850c0fe7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:07:57.158 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:57.153+0000 7f8512200640 1 -- 172.21.15.22:0/485726337 shutdown_connections 2024-03-28T18:07:57.158 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:07:57.153+0000 7f8512200640 1 -- 172.21.15.22:0/485726337 wait complete. 2024-03-28T18:07:57.170 INFO:tasks.cephfs.filesystem:scrub status for tag:d363fb2a-8987-4dd0-a598-698886fa3467 - {'path': '/', 'tag': 'd363fb2a-8987-4dd0-a598-698886fa3467', 'options': 'recursive,force'} 2024-03-28T18:07:57.174 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:07:56 smithi134 ceph-mon[35037]: pgmap v14: 97 pgs: 97 active+clean; 1.3 MiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 597 B/s wr, 12 op/s 2024-03-28T18:07:57.182 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:07:56 smithi079 ceph-mon[35312]: pgmap v14: 97 pgs: 97 active+clean; 1.3 MiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 597 B/s wr, 12 op/s 2024-03-28T18:07:58.174 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:07:57 smithi134 ceph-mon[35037]: scrub queued for path: / 2024-03-28T18:07:58.175 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:07:57 smithi134 ceph-mon[35037]: scrub summary: idle+waiting paths [/] 2024-03-28T18:07:58.175 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:07:57 smithi134 ceph-mon[35037]: scrub summary: active paths [/] 2024-03-28T18:07:58.175 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:07:57 smithi134 ceph-mon[35037]: scrub summary: idle+waiting paths [/] 2024-03-28T18:07:58.182 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:07:57 smithi079 ceph-mon[35312]: scrub queued for path: / 2024-03-28T18:07:58.182 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:07:57 smithi079 ceph-mon[35312]: scrub summary: idle+waiting paths [/] 2024-03-28T18:07:58.182 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:07:57 smithi079 ceph-mon[35312]: scrub summary: active paths [/] 2024-03-28T18:07:58.182 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:07:57 smithi079 ceph-mon[35312]: scrub summary: idle+waiting paths [/] 2024-03-28T18:07:58.282 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:07:57 smithi022 ceph-mon[31985]: scrub queued for path: / 2024-03-28T18:07:58.282 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:07:57 smithi022 ceph-mon[31985]: scrub summary: idle+waiting paths [/] 2024-03-28T18:07:58.282 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:07:57 smithi022 ceph-mon[31985]: scrub summary: active paths [/] 2024-03-28T18:07:58.282 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:07:57 smithi022 ceph-mon[31985]: scrub summary: idle+waiting paths [/] 2024-03-28T18:07:59.077 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:07:58 smithi134 ceph-mon[35037]: pgmap v15: 97 pgs: 97 active+clean; 1.3 MiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 511 B/s wr, 8 op/s 2024-03-28T18:07:59.182 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:07:58 smithi079 ceph-mon[35312]: pgmap v15: 97 pgs: 97 active+clean; 1.3 MiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 511 B/s wr, 8 op/s 2024-03-28T18:07:59.281 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:07:58 smithi022 ceph-mon[31985]: pgmap v15: 97 pgs: 97 active+clean; 1.3 MiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 511 B/s wr, 8 op/s 2024-03-28T18:08:00.101 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:07:59 smithi134 ceph-mon[35037]: scrub summary: idle 2024-03-28T18:08:00.182 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:07:59 smithi079 ceph-mon[35312]: scrub summary: idle 2024-03-28T18:08:00.282 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:07:59 smithi022 ceph-mon[31985]: scrub summary: idle 2024-03-28T18:08:01.032 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:08:00 smithi022 ceph-mon[31985]: pgmap v16: 97 pgs: 97 active+clean; 1.3 MiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 511 B/s wr, 8 op/s 2024-03-28T18:08:01.032 INFO:journalctl@ceph.mgr.x.smithi022.stdout:Mar 28 18:08:00 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mgr-x[32211]: 2024-03-28T18:08:00.776+0000 7f0701e00700 -1 client.0 error registering admin socket command: (17) File exists 2024-03-28T18:08:01.032 INFO:journalctl@ceph.mgr.x.smithi022.stdout:Mar 28 18:08:00 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mgr-x[32211]: 2024-03-28T18:08:00.776+0000 7f0701e00700 -1 client.0 error registering admin socket command: (17) File exists 2024-03-28T18:08:01.032 INFO:journalctl@ceph.mgr.x.smithi022.stdout:Mar 28 18:08:00 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mgr-x[32211]: 2024-03-28T18:08:00.776+0000 7f0701e00700 -1 client.0 error registering admin socket command: (17) File exists 2024-03-28T18:08:01.032 INFO:journalctl@ceph.mgr.x.smithi022.stdout:Mar 28 18:08:00 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mgr-x[32211]: 2024-03-28T18:08:00.776+0000 7f0701e00700 -1 client.0 error registering admin socket command: (17) File exists 2024-03-28T18:08:01.032 INFO:journalctl@ceph.mgr.x.smithi022.stdout:Mar 28 18:08:00 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mgr-x[32211]: 2024-03-28T18:08:00.776+0000 7f0701e00700 -1 client.0 error registering admin socket command: (17) File exists 2024-03-28T18:08:01.174 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:08:00 smithi134 ceph-mon[35037]: pgmap v16: 97 pgs: 97 active+clean; 1.3 MiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 511 B/s wr, 8 op/s 2024-03-28T18:08:01.182 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:08:00 smithi079 ceph-mon[35312]: pgmap v16: 97 pgs: 97 active+clean; 1.3 MiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 511 B/s wr, 8 op/s 2024-03-28T18:08:03.174 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:08:02 smithi134 ceph-mon[35037]: pgmap v17: 97 pgs: 97 active+clean; 1.3 MiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 6.9 KiB/s rd, 511 B/s wr, 12 op/s 2024-03-28T18:08:03.175 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:08:02 smithi134 ceph-mon[35037]: mgrmap e25: x(active, since 28s), standbys: z, y 2024-03-28T18:08:03.182 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:08:02 smithi079 ceph-mon[35312]: pgmap v17: 97 pgs: 97 active+clean; 1.3 MiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 6.9 KiB/s rd, 511 B/s wr, 12 op/s 2024-03-28T18:08:03.182 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:08:02 smithi079 ceph-mon[35312]: mgrmap e25: x(active, since 28s), standbys: z, y 2024-03-28T18:08:03.282 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:08:02 smithi022 ceph-mon[31985]: pgmap v17: 97 pgs: 97 active+clean; 1.3 MiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 6.9 KiB/s rd, 511 B/s wr, 12 op/s 2024-03-28T18:08:03.282 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:08:02 smithi022 ceph-mon[31985]: mgrmap e25: x(active, since 28s), standbys: z, y 2024-03-28T18:08:05.032 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:08:04 smithi022 ceph-mon[31985]: pgmap v18: 97 pgs: 97 active+clean; 1.3 MiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 8 op/s 2024-03-28T18:08:05.174 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:08:04 smithi134 ceph-mon[35037]: pgmap v18: 97 pgs: 97 active+clean; 1.3 MiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 8 op/s 2024-03-28T18:08:05.182 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:08:04 smithi079 ceph-mon[35312]: pgmap v18: 97 pgs: 97 active+clean; 1.3 MiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 8 op/s 2024-03-28T18:08:07.175 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:08:06 smithi134 ceph-mon[35037]: pgmap v19: 97 pgs: 97 active+clean; 1.3 MiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 22 KiB/s wr, 28 op/s 2024-03-28T18:08:07.182 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:08:06 smithi079 ceph-mon[35312]: pgmap v19: 97 pgs: 97 active+clean; 1.3 MiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 22 KiB/s wr, 28 op/s 2024-03-28T18:08:07.281 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:08:06 smithi022 ceph-mon[31985]: pgmap v19: 97 pgs: 97 active+clean; 1.3 MiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 22 KiB/s wr, 28 op/s 2024-03-28T18:08:09.174 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:08:08 smithi134 ceph-mon[35037]: pgmap v20: 97 pgs: 97 active+clean; 1.3 MiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-03-28T18:08:09.182 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:08:08 smithi079 ceph-mon[35312]: pgmap v20: 97 pgs: 97 active+clean; 1.3 MiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-03-28T18:08:09.281 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:08:08 smithi022 ceph-mon[31985]: pgmap v20: 97 pgs: 97 active+clean; 1.3 MiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-03-28T18:08:11.130 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:08:10 smithi134 ceph-mon[35037]: pgmap v21: 97 pgs: 97 active+clean; 1.3 MiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-03-28T18:08:11.182 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:08:10 smithi079 ceph-mon[35312]: pgmap v21: 97 pgs: 97 active+clean; 1.3 MiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-03-28T18:08:11.282 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:08:10 smithi022 ceph-mon[31985]: pgmap v21: 97 pgs: 97 active+clean; 1.3 MiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-03-28T18:08:13.174 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:08:12 smithi134 ceph-mon[35037]: pgmap v22: 97 pgs: 97 active+clean; 1.3 MiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 23 KiB/s wr, 28 op/s 2024-03-28T18:08:13.182 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:08:12 smithi079 ceph-mon[35312]: pgmap v22: 97 pgs: 97 active+clean; 1.3 MiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 23 KiB/s wr, 28 op/s 2024-03-28T18:08:13.281 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:08:12 smithi022 ceph-mon[31985]: pgmap v22: 97 pgs: 97 active+clean; 1.3 MiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 23 KiB/s wr, 28 op/s 2024-03-28T18:08:15.174 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:08:14 smithi134 ceph-mon[35037]: pgmap v23: 97 pgs: 97 active+clean; 1.3 MiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 23 KiB/s wr, 23 op/s 2024-03-28T18:08:15.182 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:08:14 smithi079 ceph-mon[35312]: pgmap v23: 97 pgs: 97 active+clean; 1.3 MiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 23 KiB/s wr, 23 op/s 2024-03-28T18:08:15.281 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:08:14 smithi022 ceph-mon[31985]: pgmap v23: 97 pgs: 97 active+clean; 1.3 MiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 23 KiB/s wr, 23 op/s 2024-03-28T18:08:16.174 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:08:15 smithi134 ceph-mon[35037]: pgmap v24: 97 pgs: 97 active+clean; 1.3 MiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 23 KiB/s wr, 28 op/s 2024-03-28T18:08:16.182 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:08:15 smithi079 ceph-mon[35312]: pgmap v24: 97 pgs: 97 active+clean; 1.3 MiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 23 KiB/s wr, 28 op/s 2024-03-28T18:08:16.281 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:08:15 smithi022 ceph-mon[31985]: pgmap v24: 97 pgs: 97 active+clean; 1.3 MiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 23 KiB/s wr, 28 op/s 2024-03-28T18:08:18.425 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:08:18 smithi134 ceph-mon[35037]: pgmap v25: 97 pgs: 97 active+clean; 1.3 MiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 170 B/s wr, 8 op/s 2024-03-28T18:08:18.432 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:08:18 smithi079 ceph-mon[35312]: pgmap v25: 97 pgs: 97 active+clean; 1.3 MiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 170 B/s wr, 8 op/s 2024-03-28T18:08:18.531 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:08:18 smithi022 ceph-mon[31985]: pgmap v25: 97 pgs: 97 active+clean; 1.3 MiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 170 B/s wr, 8 op/s 2024-03-28T18:08:20.424 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:08:20 smithi134 ceph-mon[35037]: pgmap v26: 97 pgs: 97 active+clean; 1.3 MiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 170 B/s wr, 8 op/s 2024-03-28T18:08:20.432 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:08:20 smithi079 ceph-mon[35312]: pgmap v26: 97 pgs: 97 active+clean; 1.3 MiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 170 B/s wr, 8 op/s 2024-03-28T18:08:20.531 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:08:20 smithi022 ceph-mon[31985]: pgmap v26: 97 pgs: 97 active+clean; 1.3 MiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 170 B/s wr, 8 op/s 2024-03-28T18:08:22.425 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:08:22 smithi134 ceph-mon[35037]: pgmap v27: 97 pgs: 97 active+clean; 1.3 MiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 170 B/s wr, 12 op/s 2024-03-28T18:08:22.432 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:08:22 smithi079 ceph-mon[35312]: pgmap v27: 97 pgs: 97 active+clean; 1.3 MiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 170 B/s wr, 12 op/s 2024-03-28T18:08:22.531 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:08:22 smithi022 ceph-mon[31985]: pgmap v27: 97 pgs: 97 active+clean; 1.3 MiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 170 B/s wr, 12 op/s 2024-03-28T18:08:24.426 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:08:24 smithi134 ceph-mon[35037]: pgmap v28: 97 pgs: 97 active+clean; 1.3 MiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 85 B/s wr, 8 op/s 2024-03-28T18:08:24.432 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:08:24 smithi079 ceph-mon[35312]: pgmap v28: 97 pgs: 97 active+clean; 1.3 MiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 85 B/s wr, 8 op/s 2024-03-28T18:08:24.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:08:24 smithi022 ceph-mon[31985]: pgmap v28: 97 pgs: 97 active+clean; 1.3 MiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 85 B/s wr, 8 op/s 2024-03-28T18:08:26.424 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:08:26 smithi134 ceph-mon[35037]: pgmap v29: 97 pgs: 97 active+clean; 1.3 MiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 85 B/s wr, 12 op/s 2024-03-28T18:08:26.432 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:08:26 smithi079 ceph-mon[35312]: pgmap v29: 97 pgs: 97 active+clean; 1.3 MiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 85 B/s wr, 12 op/s 2024-03-28T18:08:26.531 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:08:26 smithi022 ceph-mon[31985]: pgmap v29: 97 pgs: 97 active+clean; 1.3 MiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 85 B/s wr, 12 op/s 2024-03-28T18:08:27.171 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-28T18:08:27.171 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-28T18:08:27.369 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:27.369+0000 7f628d400640 1 -- 172.21.15.22:0/3592798757 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f62881002e0 msgr2=0x7f62881006c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:08:27.369 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:27.369+0000 7f628d400640 1 --2- 172.21.15.22:0/3592798757 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f62881002e0 0x7f62881006c0 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f6278009900 tx=0x7f627802bc30 comp rx=0 tx=0).stop 2024-03-28T18:08:27.369 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:27.369+0000 7f628d400640 1 -- 172.21.15.22:0/3592798757 shutdown_connections 2024-03-28T18:08:27.369 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:27.369+0000 7f628d400640 1 --2- 172.21.15.22:0/3592798757 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f628810de90 0x7f6288110280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:08:27.369 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:27.369+0000 7f628d400640 1 --2- 172.21.15.22:0/3592798757 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6288100c90 0x7f628810d8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:08:27.369 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:27.369+0000 7f628d400640 1 --2- 172.21.15.22:0/3592798757 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f62881002e0 0x7f62881006c0 unknown :-1 s=CLOSED pgs=512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:08:27.369 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:27.369+0000 7f628d400640 1 -- 172.21.15.22:0/3592798757 >> 172.21.15.22:0/3592798757 conn(0x7f62880fbef0 msgr2=0x7f62880fe310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:08:27.369 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:27.369+0000 7f628d400640 1 -- 172.21.15.22:0/3592798757 shutdown_connections 2024-03-28T18:08:27.370 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:27.369+0000 7f628d400640 1 -- 172.21.15.22:0/3592798757 wait complete. 2024-03-28T18:08:27.370 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:27.371+0000 7f628d400640 1 Processor -- start 2024-03-28T18:08:27.370 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:27.371+0000 7f628d400640 1 -- start start 2024-03-28T18:08:27.371 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:27.371+0000 7f628d400640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f62881002e0 0x7f628819a000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:08:27.371 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:27.371+0000 7f628d400640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f6288100c90 0x7f628819a540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:08:27.372 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:27.371+0000 7f628d400640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f628810de90 0x7f628819e8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:08:27.372 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:27.371+0000 7f628d400640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6288113b60 con 0x7f628810de90 2024-03-28T18:08:27.372 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:27.371+0000 7f628d400640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62881139e0 con 0x7f6288100c90 2024-03-28T18:08:27.372 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:27.371+0000 7f628d400640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6288113ce0 con 0x7f62881002e0 2024-03-28T18:08:27.372 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:27.372+0000 7f628ca00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f628810de90 0x7f628819e8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:08:27.372 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:27.372+0000 7f628ca00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f628810de90 0x7f628819e8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:37778/0 (socket says 172.21.15.22:37778) 2024-03-28T18:08:27.372 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:27.372+0000 7f628ca00640 1 -- 172.21.15.22:0/3884468643 learned_addr learned my addr 172.21.15.22:0/3884468643 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-28T18:08:27.372 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:27.372+0000 7f6287e00640 1 --2- 172.21.15.22:0/3884468643 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f62881002e0 0x7f628819a000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:08:27.372 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:27.372+0000 7f6287400640 1 --2- 172.21.15.22:0/3884468643 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f6288100c90 0x7f628819a540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:08:27.373 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:27.372+0000 7f628ca00640 1 -- 172.21.15.22:0/3884468643 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f62881002e0 msgr2=0x7f628819a000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:08:27.373 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:27.372+0000 7f628ca00640 1 --2- 172.21.15.22:0/3884468643 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f62881002e0 0x7f628819a000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:08:27.373 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:27.372+0000 7f628ca00640 1 -- 172.21.15.22:0/3884468643 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f6288100c90 msgr2=0x7f628819a540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:08:27.373 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:27.372+0000 7f628ca00640 1 --2- 172.21.15.22:0/3884468643 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f6288100c90 0x7f628819a540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:08:27.374 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:27.372+0000 7f628ca00640 1 -- 172.21.15.22:0/3884468643 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f628819eff0 con 0x7f628810de90 2024-03-28T18:08:27.374 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:27.372+0000 7f6287400640 1 --2- 172.21.15.22:0/3884468643 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f6288100c90 0x7f628819a540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-28T18:08:27.374 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:27.373+0000 7f628ca00640 1 --2- 172.21.15.22:0/3884468643 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f628810de90 0x7f628819e8c0 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f6280009fd0 tx=0x7f628000ee60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:08:27.374 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:27.373+0000 7f6284c00640 1 -- 172.21.15.22:0/3884468643 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6280004d20 con 0x7f628810de90 2024-03-28T18:08:27.374 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:27.373+0000 7f6284c00640 1 -- 172.21.15.22:0/3884468643 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f6280005020 con 0x7f628810de90 2024-03-28T18:08:27.374 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:27.373+0000 7f6284c00640 1 -- 172.21.15.22:0/3884468643 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62800116d0 con 0x7f628810de90 2024-03-28T18:08:27.374 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:27.373+0000 7f628d400640 1 -- 172.21.15.22:0/3884468643 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f628819f2e0 con 0x7f628810de90 2024-03-28T18:08:27.374 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:27.373+0000 7f628d400640 1 -- 172.21.15.22:0/3884468643 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f62881a6b70 con 0x7f628810de90 2024-03-28T18:08:27.376 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:27.377+0000 7f6291816740 1 Processor -- start 2024-03-28T18:08:27.377 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:27.378+0000 7f6284c00640 1 -- 172.21.15.22:0/3884468643 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 25) v1 ==== 137523+0+0 (secure 0 0 0) 0x7f62800051d0 con 0x7f628810de90 2024-03-28T18:08:27.377 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:27.378+0000 7f6291816740 1 -- start start 2024-03-28T18:08:27.378 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:27.378+0000 7f6291816740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f08020add0 0x55f080209b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:08:27.378 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:27.378+0000 7f6291816740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55f08020a0f0 0x55f08020a570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:08:27.378 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:27.378+0000 7f6291816740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f08021bc70 0x55f08021e060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:08:27.378 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:27.378+0000 7f6291816740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f08016e250 con 0x55f08021bc70 2024-03-28T18:08:27.378 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:27.378+0000 7f6291816740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f080116a30 con 0x55f08020a0f0 2024-03-28T18:08:27.378 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:27.378+0000 7f6291816740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f08016a2f0 con 0x55f08020add0 2024-03-28T18:08:27.379 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:27.379+0000 7f6287400640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f08021bc70 0x55f08021e060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:08:27.379 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:27.379+0000 7f6287e00640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55f08020a0f0 0x55f08020a570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:08:27.379 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:27.379+0000 7f6287e00640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55f08020a0f0 0x55f08020a570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.22:47070/0 (socket says 172.21.15.22:47070) 2024-03-28T18:08:27.380 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:27.379+0000 7f6287e00640 1 -- 172.21.15.22:0/2675380579 learned_addr learned my addr 172.21.15.22:0/2675380579 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-28T18:08:27.380 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:27.379+0000 7f6287400640 1 --2- 172.21.15.22:0/2675380579 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f08020add0 0x55f080209b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:08:27.380 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:27.379+0000 7f6284c00640 1 --2- 172.21.15.22:0/3884468643 >> [v2:172.21.15.22:6800/1357365082,v1:172.21.15.22:6801/1357365082] conn(0x7f626c0a2b10 0x7f626c0a4fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:08:27.380 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:27.379+0000 7f6284c00640 1 -- 172.21.15.22:0/3884468643 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 8411+0+0 (secure 0 0 0) 0x7f62800cf270 con 0x7f628810de90 2024-03-28T18:08:27.380 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:27.379+0000 7f6287e00640 1 -- 172.21.15.22:0/2675380579 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f08020add0 msgr2=0x55f080209b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T18:08:27.380 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:27.379+0000 7f6287e00640 1 --2- 172.21.15.22:0/2675380579 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f08020add0 0x55f080209b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:08:27.380 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:27.379+0000 7f6287e00640 1 -- 172.21.15.22:0/2675380579 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f08021bc70 msgr2=0x55f08021e060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T18:08:27.380 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:27.379+0000 7f6287e00640 1 --2- 172.21.15.22:0/2675380579 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f08021bc70 0x55f08021e060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:08:27.380 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:27.379+0000 7f6287e00640 1 -- 172.21.15.22:0/2675380579 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f080101690 con 0x55f08020a0f0 2024-03-28T18:08:27.380 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:27.380+0000 7f628ca00640 1 --2- 172.21.15.22:0/3884468643 >> [v2:172.21.15.22:6800/1357365082,v1:172.21.15.22:6801/1357365082] conn(0x7f626c0a2b10 0x7f626c0a4fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:08:27.380 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:27.380+0000 7f6287e00640 1 --2- 172.21.15.22:0/2675380579 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55f08020a0f0 0x55f08020a570 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f6278000c00 tx=0x7f62780040c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a2c78f192bebdc63 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:08:27.381 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:27.380+0000 7f628ca00640 1 --2- 172.21.15.22:0/3884468643 >> [v2:172.21.15.22:6800/1357365082,v1:172.21.15.22:6801/1357365082] conn(0x7f626c0a2b10 0x7f626c0a4fd0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f62800cc8b0 tx=0x7f6280004a40 comp rx=0 tx=0).ready entity=mgr.15522 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:08:27.382 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:27.380+0000 7f627d200640 1 -- 172.21.15.22:0/2675380579 <== mon.2 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62780042b0 con 0x55f08020a0f0 2024-03-28T18:08:27.382 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:27.381+0000 7f627d200640 1 -- 172.21.15.22:0/2675380579 <== mon.2 v2:172.21.15.79:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f6278004450 con 0x55f08020a0f0 2024-03-28T18:08:27.382 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:27.381+0000 7f627d200640 1 -- 172.21.15.22:0/2675380579 <== mon.2 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62780046c0 con 0x55f08020a0f0 2024-03-28T18:08:27.382 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:27.381+0000 7f6291816740 1 -- 172.21.15.22:0/2675380579 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55f08020a0f0 msgr2=0x55f08020a570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:08:27.382 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:27.381+0000 7f6291816740 1 --2- 172.21.15.22:0/2675380579 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55f08020a0f0 0x55f08020a570 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f6278000c00 tx=0x7f62780040c0 comp rx=0 tx=0).stop 2024-03-28T18:08:27.382 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:27.381+0000 7f6291816740 1 -- 172.21.15.22:0/2675380579 shutdown_connections 2024-03-28T18:08:27.382 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:27.381+0000 7f6291816740 1 --2- 172.21.15.22:0/2675380579 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f08021bc70 0x55f08021e060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:08:27.382 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:27.381+0000 7f6291816740 1 --2- 172.21.15.22:0/2675380579 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55f08020a0f0 0x55f08020a570 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:08:27.382 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:27.381+0000 7f6291816740 1 --2- 172.21.15.22:0/2675380579 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f08020add0 0x55f080209b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:08:27.382 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:27.381+0000 7f6291816740 1 -- 172.21.15.22:0/2675380579 >> 172.21.15.22:0/2675380579 conn(0x55f0801d1200 msgr2=0x55f080204b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:08:27.382 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:27.382+0000 7f6291816740 1 -- 172.21.15.22:0/2675380579 shutdown_connections 2024-03-28T18:08:27.382 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:27.382+0000 7f6291816740 1 -- 172.21.15.22:0/2675380579 wait complete. 2024-03-28T18:08:27.382 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:27.382+0000 7f6291816740 1 -- start start 2024-03-28T18:08:27.382 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:27.382+0000 7f6291816740 1 Processor -- start 2024-03-28T18:08:27.385 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:27.383+0000 7f6291816740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55f08020a0f0 0x55f08021b2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:08:27.385 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:27.383+0000 7f6291816740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f08020add0 0x55f080218670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:08:27.385 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:27.383+0000 7f6291816740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f08021bc70 0x55f080218bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:08:27.385 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:27.383+0000 7f6291816740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f08021ea80 con 0x55f08021bc70 2024-03-28T18:08:27.385 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:27.383+0000 7f6291816740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f080129e80 con 0x55f08020a0f0 2024-03-28T18:08:27.385 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:27.383+0000 7f6291816740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f08021ec00 con 0x55f08020add0 2024-03-28T18:08:27.385 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:27.383+0000 7f6291816740 10 client.0 _ll_register_callbacks cb 0x55f08013c130 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-28T18:08:27.385 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:27.383+0000 7f6287400640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f08021bc70 0x55f080218bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:08:27.385 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:27.383+0000 7f6287400640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f08021bc70 0x55f080218bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:37792/0 (socket says 172.21.15.22:37792) 2024-03-28T18:08:27.385 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:27.383+0000 7f6287e00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f08020add0 0x55f080218670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:08:27.385 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:27.383+0000 7f6287400640 1 -- 172.21.15.22:0/167441454 learned_addr learned my addr 172.21.15.22:0/167441454 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-28T18:08:27.385 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:27.383+0000 7f6287400640 1 --2- 172.21.15.22:0/167441454 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55f08020a0f0 0x55f08021b2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:08:27.386 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:27.384+0000 7f6287400640 1 -- 172.21.15.22:0/167441454 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f08020add0 msgr2=0x55f080218670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T18:08:27.386 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:27.384+0000 7f6287400640 1 --2- 172.21.15.22:0/167441454 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f08020add0 0x55f080218670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:08:27.386 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:27.384+0000 7f6287400640 1 -- 172.21.15.22:0/167441454 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55f08020a0f0 msgr2=0x55f08021b2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T18:08:27.386 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:27.384+0000 7f6287400640 1 --2- 172.21.15.22:0/167441454 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55f08020a0f0 0x55f08021b2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:08:27.386 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:27.384+0000 7f6287400640 1 -- 172.21.15.22:0/167441454 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55f07ff4c7d0 con 0x55f08021bc70 2024-03-28T18:08:27.386 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:27.384+0000 7f6287e00640 1 --2- 172.21.15.22:0/167441454 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f08020add0 0x55f080218670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-28T18:08:27.386 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:27.384+0000 7f6287400640 1 --2- 172.21.15.22:0/167441454 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f08021bc70 0x55f080218bb0 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f627400f3f0 tx=0x7f6274004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=92329c00a877bb26 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:08:27.386 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:27.384+0000 7f6271600640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-03-28T18:08:27.386 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:27.385+0000 7f6271600640 1 -- 172.21.15.22:0/167441454 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6274022440 con 0x55f08021bc70 2024-03-28T18:08:27.386 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:27.385+0000 7f6271600640 1 -- 172.21.15.22:0/167441454 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 8411+0+0 (secure 0 0 0) 0x7f62740225e0 con 0x55f08021bc70 2024-03-28T18:08:27.386 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:27.385+0000 7f6291816740 1 -- 172.21.15.22:0/167441454 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f6278004450 con 0x55f08021bc70 2024-03-28T18:08:27.386 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:27.385+0000 7f6271600640 1 -- 172.21.15.22:0/167441454 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f62740227e0 con 0x55f08021bc70 2024-03-28T18:08:27.386 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:27.385+0000 7f6291816740 10 client.15780 fetch_fsmap learned FSMap version 27 2024-03-28T18:08:27.386 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:27.385+0000 7f6291816740 1 -- 172.21.15.22:0/167441454 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55f07fe823f0 con 0x55f08021bc70 2024-03-28T18:08:27.386 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:27.385+0000 7f6271600640 1 -- 172.21.15.22:0/167441454 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f6278004450 con 0x55f08021bc70 2024-03-28T18:08:27.387 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:27.386+0000 7f6271600640 1 -- 172.21.15.22:0/167441454 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 27) v1 ==== 7934+0+0 (secure 0 0 0) 0x7f6274020020 con 0x55f08021bc70 2024-03-28T18:08:27.387 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:27.386+0000 7f6291816740 10 client.15780 fetch_fsmap finished waiting for FSMap version 27 2024-03-28T18:08:27.388 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:27.386+0000 7f6291816740 10 client.15780 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-28T18:08:27.388 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:27.386+0000 7f6291816740 20 client.15780 populate_metadata read hostname 'smithi022' 2024-03-28T18:08:27.388 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:27.386+0000 7f6291816740 1 --2- 172.21.15.22:0/167441454 >> [v2:172.21.15.79:6838/2441542443,v1:172.21.15.79:6839/2441542443] conn(0x55f08022bad0 0x55f08022dec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:08:27.388 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:27.386+0000 7f6291816740 4 client.15780 mds_command: new command op to 34553 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-28T18:08:27.388 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:27.386+0000 7f6291816740 1 -- 172.21.15.22:0/167441454 --> [v2:172.21.15.79:6838/2441542443,v1:172.21.15.79:6839/2441542443] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55f08017b790 con 0x55f08022bad0 2024-03-28T18:08:27.388 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:27.386+0000 7f6287e00640 1 --2- 172.21.15.22:0/167441454 >> [v2:172.21.15.79:6838/2441542443,v1:172.21.15.79:6839/2441542443] conn(0x55f08022bad0 0x55f08022dec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:08:27.388 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:27.387+0000 7f6287e00640 1 --2- 172.21.15.22:0/167441454 >> [v2:172.21.15.79:6838/2441542443,v1:172.21.15.79:6839/2441542443] conn(0x55f08022bad0 0x55f08022dec0 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=377a5da7a74cc6c5 server_cookie=bc2f364d8e72291f in_seq=0 out_seq=0 2024-03-28T18:08:27.388 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:27.387+0000 7f6271600640 10 client.15780 ms_handle_connect on v2:172.21.15.79:6838/2441542443 2024-03-28T18:08:27.389 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:27.389+0000 7f6271600640 1 -- 172.21.15.22:0/167441454 <== mds.0 v2:172.21.15.79:6838/2441542443 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x55f08017b790 con 0x55f08022bad0 2024-03-28T18:08:27.389 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:27.389+0000 7f6271600640 10 client.15780 handle_command_reply: tid=0 2024-03-28T18:08:27.389 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:27.389+0000 7f6291816740 1 client.15780 shutdown 2024-03-28T18:08:27.389 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:27.389+0000 7f6291816740 1 -- 172.21.15.22:0/167441454 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f08021bc70 msgr2=0x55f080218bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:08:27.389 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:27.389+0000 7f6291816740 1 --2- 172.21.15.22:0/167441454 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f08021bc70 0x55f080218bb0 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f627400f3f0 tx=0x7f6274004730 comp rx=0 tx=0).stop 2024-03-28T18:08:27.389 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:27.389+0000 7f6291816740 1 -- 172.21.15.22:0/167441454 shutdown_connections 2024-03-28T18:08:27.390 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:27.389+0000 7f6291816740 1 --2- 172.21.15.22:0/167441454 >> [v2:172.21.15.79:6838/2441542443,v1:172.21.15.79:6839/2441542443] conn(0x55f08022bad0 0x55f08022dec0 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:08:27.390 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:27.389+0000 7f6291816740 1 --2- 172.21.15.22:0/167441454 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f08021bc70 0x55f080218bb0 unknown :-1 s=CLOSED pgs=514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:08:27.390 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:27.389+0000 7f6291816740 1 --2- 172.21.15.22:0/167441454 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f08020add0 0x55f080218670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:08:27.390 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:27.389+0000 7f6291816740 1 --2- 172.21.15.22:0/167441454 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55f08020a0f0 0x55f08021b2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:08:27.390 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:27.389+0000 7f6291816740 1 -- 172.21.15.22:0/167441454 >> 172.21.15.22:0/167441454 conn(0x55f0801d1200 msgr2=0x55f080206890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:08:27.390 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:27.389+0000 7f6271600640 0 client.15780 ms_handle_reset on v2:172.21.15.79:6838/2441542443 2024-03-28T18:08:27.390 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:27.389+0000 7f6291816740 1 -- 172.21.15.22:0/167441454 shutdown_connections 2024-03-28T18:08:27.390 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:27.390+0000 7f6291816740 1 -- 172.21.15.22:0/167441454 wait complete. 2024-03-28T18:08:27.390 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:27.390+0000 7f6291816740 20 client.15780 trim_cache size 0 max 16384 2024-03-28T18:08:27.407 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:27.407+0000 7f6291816740 1 Processor -- start 2024-03-28T18:08:27.407 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:27.407+0000 7f6291816740 1 -- start start 2024-03-28T18:08:27.407 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:27.407+0000 7f6291816740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f08020a0f0 0x55f080217f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:08:27.407 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:27.407+0000 7f6291816740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f08020add0 0x55f08021ada0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:08:27.408 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:27.407+0000 7f6291816740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55f08021bc70 0x55f08021b2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:08:27.408 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:27.407+0000 7f6291816740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62881aab40 con 0x55f08020a0f0 2024-03-28T18:08:27.408 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:27.407+0000 7f6291816740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62881ac4c0 con 0x55f08021bc70 2024-03-28T18:08:27.408 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:27.407+0000 7f6291816740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62881ab1a0 con 0x55f08020add0 2024-03-28T18:08:27.408 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:27.407+0000 7f6287400640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f08020a0f0 0x55f080217f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:08:27.408 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:27.407+0000 7f6287400640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f08020a0f0 0x55f080217f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:37802/0 (socket says 172.21.15.22:37802) 2024-03-28T18:08:27.408 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:27.407+0000 7f6287400640 1 -- 172.21.15.22:0/616895161 learned_addr learned my addr 172.21.15.22:0/616895161 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-28T18:08:27.408 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:27.408+0000 7f6287400640 1 --2- 172.21.15.22:0/616895161 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55f08021bc70 0x55f08021b2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:08:27.408 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:27.408+0000 7f6287400640 1 -- 172.21.15.22:0/616895161 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f08020add0 msgr2=0x55f08021ada0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T18:08:27.408 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:27.408+0000 7f6287e00640 1 --2- 172.21.15.22:0/616895161 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f08020add0 0x55f08021ada0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:08:27.408 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:27.408+0000 7f6287400640 1 --2- 172.21.15.22:0/616895161 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f08020add0 0x55f08021ada0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:08:27.408 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:27.408+0000 7f6287400640 1 -- 172.21.15.22:0/616895161 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55f08021bc70 msgr2=0x55f08021b2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T18:08:27.408 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:27.408+0000 7f6287400640 1 --2- 172.21.15.22:0/616895161 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55f08021bc70 0x55f08021b2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:08:27.408 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:27.408+0000 7f6287400640 1 -- 172.21.15.22:0/616895161 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f07fe420f0 con 0x55f08020a0f0 2024-03-28T18:08:27.408 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:27.408+0000 7f6287e00640 1 --2- 172.21.15.22:0/616895161 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f08020add0 0x55f08021ada0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T18:08:27.410 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:27.408+0000 7f6287400640 1 --2- 172.21.15.22:0/616895161 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f08020a0f0 0x55f080217f70 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f6274013d70 tx=0x7f6274016040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8880317e8e6461a0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:08:27.410 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:27.408+0000 7f6273e00640 1 -- 172.21.15.22:0/616895161 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f627400c8d0 con 0x55f08020a0f0 2024-03-28T18:08:27.410 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:27.409+0000 7f6273e00640 1 -- 172.21.15.22:0/616895161 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f627400c3d0 con 0x55f08020a0f0 2024-03-28T18:08:27.410 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:27.409+0000 7f6273e00640 1 -- 172.21.15.22:0/616895161 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6274022da0 con 0x55f08020a0f0 2024-03-28T18:08:27.410 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:27.409+0000 7f6291816740 1 -- 172.21.15.22:0/616895161 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f08020a0f0 msgr2=0x55f080217f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:08:27.410 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:27.409+0000 7f6291816740 1 --2- 172.21.15.22:0/616895161 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f08020a0f0 0x55f080217f70 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f6274013d70 tx=0x7f6274016040 comp rx=0 tx=0).stop 2024-03-28T18:08:27.410 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:27.409+0000 7f6291816740 1 -- 172.21.15.22:0/616895161 shutdown_connections 2024-03-28T18:08:27.410 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:27.409+0000 7f6291816740 1 --2- 172.21.15.22:0/616895161 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55f08021bc70 0x55f08021b2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:08:27.410 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:27.409+0000 7f6291816740 1 --2- 172.21.15.22:0/616895161 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f08020add0 0x55f08021ada0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:08:27.410 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:27.409+0000 7f6291816740 1 --2- 172.21.15.22:0/616895161 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f08020a0f0 0x55f080217f70 unknown :-1 s=CLOSED pgs=515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:08:27.410 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:27.409+0000 7f6291816740 1 -- 172.21.15.22:0/616895161 >> 172.21.15.22:0/616895161 conn(0x55f0801d1200 msgr2=0x55f080203330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:08:27.410 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:27.409+0000 7f6291816740 1 -- 172.21.15.22:0/616895161 shutdown_connections 2024-03-28T18:08:27.410 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:27.410+0000 7f6291816740 1 -- 172.21.15.22:0/616895161 wait complete. 2024-03-28T18:08:27.411 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:27.410+0000 7f6291816740 1 -- start start 2024-03-28T18:08:27.411 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:27.410+0000 7f6291816740 1 Processor -- start 2024-03-28T18:08:27.412 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:27.411+0000 7f6291816740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f08020a0f0 0x55f080214fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:08:27.412 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:27.411+0000 7f6291816740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f08020add0 0x55f080215500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:08:27.412 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:27.411+0000 7f6291816740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55f08021bc70 0x55f0802862b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:08:27.412 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:27.411+0000 7f6291816740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62881abe60 con 0x55f08020add0 2024-03-28T18:08:27.412 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:27.411+0000 7f6291816740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62881ab800 con 0x55f08021bc70 2024-03-28T18:08:27.413 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:27.411+0000 7f6291816740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62881a8d40 con 0x55f08020a0f0 2024-03-28T18:08:27.413 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:27.411+0000 7f6291816740 10 client.0 _ll_register_callbacks cb 0x7f6274013460 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-28T18:08:27.413 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:27.411+0000 7f6287e00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f08020add0 0x55f080215500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:08:27.413 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:27.411+0000 7f6287400640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55f08021bc70 0x55f0802862b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:08:27.413 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:27.411+0000 7f6287400640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55f08021bc70 0x55f0802862b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.22:47100/0 (socket says 172.21.15.22:47100) 2024-03-28T18:08:27.413 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:27.411+0000 7f6287400640 1 -- 172.21.15.22:0/4088809339 learned_addr learned my addr 172.21.15.22:0/4088809339 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-28T18:08:27.413 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:27.411+0000 7f6287400640 1 --2- 172.21.15.22:0/4088809339 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f08020a0f0 0x55f080214fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:08:27.413 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:27.412+0000 7f6287e00640 1 -- 172.21.15.22:0/4088809339 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f08020a0f0 msgr2=0x55f080214fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T18:08:27.413 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:27.412+0000 7f6287e00640 1 --2- 172.21.15.22:0/4088809339 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f08020a0f0 0x55f080214fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:08:27.413 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:27.412+0000 7f6287e00640 1 -- 172.21.15.22:0/4088809339 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55f08021bc70 msgr2=0x55f0802862b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T18:08:27.413 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:27.412+0000 7f6287e00640 1 --2- 172.21.15.22:0/4088809339 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55f08021bc70 0x55f0802862b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:08:27.413 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:27.412+0000 7f6287e00640 1 -- 172.21.15.22:0/4088809339 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55f080215db0 con 0x55f08020add0 2024-03-28T18:08:27.413 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:27.412+0000 7f6287400640 1 --2- 172.21.15.22:0/4088809339 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55f08021bc70 0x55f0802862b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-28T18:08:27.413 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:27.412+0000 7f6287e00640 1 --2- 172.21.15.22:0/4088809339 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f08020add0 0x55f080215500 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f6278036040 tx=0x7f627802c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4576d312625c870d server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:08:27.413 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:27.412+0000 7f6271600640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-03-28T18:08:27.414 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:27.412+0000 7f6271600640 1 -- 172.21.15.22:0/4088809339 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6278005850 con 0x55f08020add0 2024-03-28T18:08:27.414 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:27.412+0000 7f6271600640 1 -- 172.21.15.22:0/4088809339 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 8411+0+0 (secure 0 0 0) 0x7f6278005040 con 0x55f08020add0 2024-03-28T18:08:27.414 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:27.412+0000 7f6291816740 1 -- 172.21.15.22:0/4088809339 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f627400c3d0 con 0x55f08020add0 2024-03-28T18:08:27.414 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:27.413+0000 7f6271600640 1 -- 172.21.15.22:0/4088809339 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6278005260 con 0x55f08020add0 2024-03-28T18:08:27.414 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:27.413+0000 7f6291816740 10 client.15786 fetch_fsmap learned FSMap version 27 2024-03-28T18:08:27.414 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:27.413+0000 7f6291816740 1 -- 172.21.15.22:0/4088809339 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55f0802877f0 con 0x55f08020add0 2024-03-28T18:08:27.416 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:27.413+0000 7f6271600640 1 -- 172.21.15.22:0/4088809339 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f627400c3d0 con 0x55f08020add0 2024-03-28T18:08:27.416 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:27.413+0000 7f6271600640 1 -- 172.21.15.22:0/4088809339 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 27) v1 ==== 7934+0+0 (secure 0 0 0) 0x7f62780035f0 con 0x55f08020add0 2024-03-28T18:08:27.416 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:27.413+0000 7f6291816740 10 client.15786 fetch_fsmap finished waiting for FSMap version 27 2024-03-28T18:08:27.416 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:27.413+0000 7f6291816740 10 client.15786 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-28T18:08:27.416 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:27.413+0000 7f6291816740 20 client.15786 populate_metadata read hostname 'smithi022' 2024-03-28T18:08:27.416 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:27.413+0000 7f6291816740 1 --2- 172.21.15.22:0/4088809339 >> [v2:172.21.15.79:6838/2441542443,v1:172.21.15.79:6839/2441542443] conn(0x55f080287b30 0x55f080289f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:08:27.416 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:27.413+0000 7f6291816740 4 client.15786 mds_command: new command op to 34553 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-28T18:08:27.416 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:27.413+0000 7f6291816740 1 -- 172.21.15.22:0/4088809339 --> [v2:172.21.15.79:6838/2441542443,v1:172.21.15.79:6839/2441542443] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55f080175ae0 con 0x55f080287b30 2024-03-28T18:08:27.416 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:27.414+0000 7f6287400640 1 --2- 172.21.15.22:0/4088809339 >> [v2:172.21.15.79:6838/2441542443,v1:172.21.15.79:6839/2441542443] conn(0x55f080287b30 0x55f080289f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:08:27.416 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:27.415+0000 7f6287400640 1 --2- 172.21.15.22:0/4088809339 >> [v2:172.21.15.79:6838/2441542443,v1:172.21.15.79:6839/2441542443] conn(0x55f080287b30 0x55f080289f20 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ab3da0fe893581e8 server_cookie=3fa9849aa5dc6a68 in_seq=0 out_seq=0 2024-03-28T18:08:27.416 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:27.415+0000 7f6271600640 10 client.15786 ms_handle_connect on v2:172.21.15.79:6838/2441542443 2024-03-28T18:08:27.416 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:27.415+0000 7f6271600640 1 -- 172.21.15.22:0/4088809339 <== mds.0 v2:172.21.15.79:6838/2441542443 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x55f080175ae0 con 0x55f080287b30 2024-03-28T18:08:27.417 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:27.415+0000 7f6271600640 10 client.15786 handle_command_reply: tid=0 2024-03-28T18:08:27.417 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:27.415+0000 7f6291816740 1 client.15786 shutdown 2024-03-28T18:08:27.417 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:27.415+0000 7f6291816740 1 -- 172.21.15.22:0/4088809339 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f08020add0 msgr2=0x55f080215500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:08:27.417 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:27.415+0000 7f6291816740 1 --2- 172.21.15.22:0/4088809339 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f08020add0 0x55f080215500 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f6278036040 tx=0x7f627802c9f0 comp rx=0 tx=0).stop 2024-03-28T18:08:27.417 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:27.415+0000 7f6291816740 1 -- 172.21.15.22:0/4088809339 shutdown_connections 2024-03-28T18:08:27.417 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:27.415+0000 7f6291816740 1 --2- 172.21.15.22:0/4088809339 >> [v2:172.21.15.79:6838/2441542443,v1:172.21.15.79:6839/2441542443] conn(0x55f080287b30 0x55f080289f20 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:08:27.417 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:27.415+0000 7f6291816740 1 --2- 172.21.15.22:0/4088809339 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55f08021bc70 0x55f0802862b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:08:27.417 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:27.415+0000 7f6291816740 1 --2- 172.21.15.22:0/4088809339 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f08020add0 0x55f080215500 unknown :-1 s=CLOSED pgs=516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:08:27.417 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:27.415+0000 7f6291816740 1 --2- 172.21.15.22:0/4088809339 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f08020a0f0 0x55f080214fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:08:27.417 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:27.415+0000 7f6291816740 1 -- 172.21.15.22:0/4088809339 >> 172.21.15.22:0/4088809339 conn(0x55f0801d1200 msgr2=0x55f080205eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:08:27.417 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:27.415+0000 7f6271600640 0 client.15786 ms_handle_reset on v2:172.21.15.79:6838/2441542443 2024-03-28T18:08:27.417 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:27.415+0000 7f6291816740 1 -- 172.21.15.22:0/4088809339 shutdown_connections 2024-03-28T18:08:27.417 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:27.416+0000 7f6291816740 1 -- 172.21.15.22:0/4088809339 wait complete. 2024-03-28T18:08:27.417 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:27.416+0000 7f6291816740 20 client.15786 trim_cache size 0 max 16384 2024-03-28T18:08:27.418 INFO:teuthology.orchestra.run.smithi022.stdout:{ 2024-03-28T18:08:27.418 INFO:teuthology.orchestra.run.smithi022.stdout: "status": "no active scrubs running", 2024-03-28T18:08:27.418 INFO:teuthology.orchestra.run.smithi022.stdout: "scrubs": {} 2024-03-28T18:08:27.418 INFO:teuthology.orchestra.run.smithi022.stdout:} 2024-03-28T18:08:27.419 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:27.416+0000 7f628d400640 1 -- 172.21.15.22:0/3884468643 >> [v2:172.21.15.22:6800/1357365082,v1:172.21.15.22:6801/1357365082] conn(0x7f626c0a2b10 msgr2=0x7f626c0a4fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:08:27.419 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:27.416+0000 7f628d400640 1 --2- 172.21.15.22:0/3884468643 >> [v2:172.21.15.22:6800/1357365082,v1:172.21.15.22:6801/1357365082] conn(0x7f626c0a2b10 0x7f626c0a4fd0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f62800cc8b0 tx=0x7f6280004a40 comp rx=0 tx=0).stop 2024-03-28T18:08:27.419 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:27.416+0000 7f628d400640 1 -- 172.21.15.22:0/3884468643 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f628810de90 msgr2=0x7f628819e8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:08:27.419 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:27.416+0000 7f628d400640 1 --2- 172.21.15.22:0/3884468643 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f628810de90 0x7f628819e8c0 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f6280009fd0 tx=0x7f628000ee60 comp rx=0 tx=0).stop 2024-03-28T18:08:27.419 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:27.417+0000 7f628d400640 1 -- 172.21.15.22:0/3884468643 shutdown_connections 2024-03-28T18:08:27.419 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:27.417+0000 7f628d400640 1 --2- 172.21.15.22:0/3884468643 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f628810de90 0x7f628819e8c0 unknown :-1 s=CLOSED pgs=513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:08:27.419 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:27.417+0000 7f628d400640 1 --2- 172.21.15.22:0/3884468643 >> [v2:172.21.15.22:6800/1357365082,v1:172.21.15.22:6801/1357365082] conn(0x7f626c0a2b10 0x7f626c0a4fd0 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:08:27.419 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:27.417+0000 7f628d400640 1 --2- 172.21.15.22:0/3884468643 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f6288100c90 0x7f628819a540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:08:27.419 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:27.417+0000 7f628d400640 1 --2- 172.21.15.22:0/3884468643 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f62881002e0 0x7f628819a000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:08:27.419 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:27.417+0000 7f628d400640 1 -- 172.21.15.22:0/3884468643 >> 172.21.15.22:0/3884468643 conn(0x7f62880fbef0 msgr2=0x7f62880fe2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:08:27.419 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:27.417+0000 7f628d400640 1 -- 172.21.15.22:0/3884468643 shutdown_connections 2024-03-28T18:08:27.419 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:27.417+0000 7f628d400640 1 -- 172.21.15.22:0/3884468643 wait complete. 2024-03-28T18:08:27.436 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-03-28T18:08:27.436 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-28T18:08:27.636 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:27.636+0000 7f55db200640 1 -- 172.21.15.22:0/3280576706 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f55dc0743f0 msgr2=0x7f55dc10d790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:08:27.636 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:27.636+0000 7f55db200640 1 --2- 172.21.15.22:0/3280576706 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f55dc0743f0 0x7f55dc10d790 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f55c8009860 tx=0x7f55c802bc10 comp rx=0 tx=0).stop 2024-03-28T18:08:27.636 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:27.636+0000 7f55db200640 1 -- 172.21.15.22:0/3280576706 shutdown_connections 2024-03-28T18:08:27.636 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:27.636+0000 7f55db200640 1 --2- 172.21.15.22:0/3280576706 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f55dc10dcd0 0x7f55dc110120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:08:27.636 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:27.636+0000 7f55db200640 1 --2- 172.21.15.22:0/3280576706 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f55dc0743f0 0x7f55dc10d790 unknown :-1 s=CLOSED pgs=517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:08:27.636 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:27.636+0000 7f55db200640 1 --2- 172.21.15.22:0/3280576706 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f55dc073ad0 0x7f55dc073eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:08:27.636 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:27.636+0000 7f55db200640 1 -- 172.21.15.22:0/3280576706 >> 172.21.15.22:0/3280576706 conn(0x7f55dc0fcfa0 msgr2=0x7f55dc0ff3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:08:27.636 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:27.636+0000 7f55db200640 1 -- 172.21.15.22:0/3280576706 shutdown_connections 2024-03-28T18:08:27.636 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:27.636+0000 7f55db200640 1 -- 172.21.15.22:0/3280576706 wait complete. 2024-03-28T18:08:27.637 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:27.637+0000 7f55db200640 1 Processor -- start 2024-03-28T18:08:27.637 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:27.638+0000 7f55db200640 1 -- start start 2024-03-28T18:08:27.638 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:27.638+0000 7f55db200640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f55dc073ad0 0x7f55dc199fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:08:27.638 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:27.638+0000 7f55db200640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f55dc0743f0 0x7f55dc19a520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:08:27.638 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:27.638+0000 7f55db200640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f55dc10dcd0 0x7f55dc19e8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:08:27.638 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:27.638+0000 7f55db200640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55dc113a40 con 0x7f55dc073ad0 2024-03-28T18:08:27.639 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:27.638+0000 7f55db200640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55dc1138c0 con 0x7f55dc0743f0 2024-03-28T18:08:27.639 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:27.638+0000 7f55db200640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55dc113bc0 con 0x7f55dc10dcd0 2024-03-28T18:08:27.639 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:27.638+0000 7f55d9e00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f55dc073ad0 0x7f55dc199fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:08:27.639 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:27.638+0000 7f55d9400640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f55dc0743f0 0x7f55dc19a520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:08:27.639 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:27.639+0000 7f55d9400640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f55dc0743f0 0x7f55dc19a520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.22:47108/0 (socket says 172.21.15.22:47108) 2024-03-28T18:08:27.639 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:27.639+0000 7f55d9400640 1 -- 172.21.15.22:0/1885078804 learned_addr learned my addr 172.21.15.22:0/1885078804 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-28T18:08:27.639 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:27.639+0000 7f55da800640 1 --2- 172.21.15.22:0/1885078804 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f55dc10dcd0 0x7f55dc19e8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:08:27.640 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:27.639+0000 7f55d9400640 1 -- 172.21.15.22:0/1885078804 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f55dc10dcd0 msgr2=0x7f55dc19e8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:08:27.640 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:27.639+0000 7f55d9400640 1 --2- 172.21.15.22:0/1885078804 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f55dc10dcd0 0x7f55dc19e8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:08:27.640 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:27.639+0000 7f55d9400640 1 -- 172.21.15.22:0/1885078804 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f55dc073ad0 msgr2=0x7f55dc199fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:08:27.640 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:27.639+0000 7f55d9400640 1 --2- 172.21.15.22:0/1885078804 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f55dc073ad0 0x7f55dc199fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:08:27.640 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:27.639+0000 7f55d9400640 1 -- 172.21.15.22:0/1885078804 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f55dc19efd0 con 0x7f55dc0743f0 2024-03-28T18:08:27.640 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:27.639+0000 7f55d9e00640 1 --2- 172.21.15.22:0/1885078804 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f55dc073ad0 0x7f55dc199fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-28T18:08:27.640 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:27.640+0000 7f55d9400640 1 --2- 172.21.15.22:0/1885078804 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f55dc0743f0 0x7f55dc19a520 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f55c8000c00 tx=0x7f55c8004530 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:08:27.640 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:27.640+0000 7f55d2a00640 1 -- 172.21.15.22:0/1885078804 <== mon.2 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55c8042020 con 0x7f55dc0743f0 2024-03-28T18:08:27.641 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:27.640+0000 7f55db200640 1 -- 172.21.15.22:0/1885078804 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f55dc19f260 con 0x7f55dc0743f0 2024-03-28T18:08:27.641 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:27.640+0000 7f55d2a00640 1 -- 172.21.15.22:0/1885078804 <== mon.2 v2:172.21.15.79:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f55c802ca80 con 0x7f55dc0743f0 2024-03-28T18:08:27.641 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:27.640+0000 7f55d2a00640 1 -- 172.21.15.22:0/1885078804 <== mon.2 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55c8039040 con 0x7f55dc0743f0 2024-03-28T18:08:27.641 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:27.640+0000 7f55db200640 1 -- 172.21.15.22:0/1885078804 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f55dc1a6b50 con 0x7f55dc0743f0 2024-03-28T18:08:27.641 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:27.641+0000 7f55db200640 1 -- 172.21.15.22:0/1885078804 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f55dc0683b0 con 0x7f55dc0743f0 2024-03-28T18:08:27.644 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:27.645+0000 7f55d2a00640 1 -- 172.21.15.22:0/1885078804 <== mon.2 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 25) v1 ==== 137523+0+0 (secure 0 0 0) 0x7f55c8045020 con 0x7f55dc0743f0 2024-03-28T18:08:27.646 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:27.647+0000 7f55d2a00640 1 --2- 172.21.15.22:0/1885078804 >> [v2:172.21.15.22:6800/1357365082,v1:172.21.15.22:6801/1357365082] conn(0x7f55c00a2a90 0x7f55c00a4f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:08:27.647 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:27.647+0000 7f55d2a00640 1 -- 172.21.15.22:0/1885078804 <== mon.2 v2:172.21.15.79:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 8411+0+0 (secure 0 0 0) 0x7f55c80f0bf0 con 0x7f55dc0743f0 2024-03-28T18:08:27.647 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:27.647+0000 7f55d9e00640 1 --2- 172.21.15.22:0/1885078804 >> [v2:172.21.15.22:6800/1357365082,v1:172.21.15.22:6801/1357365082] conn(0x7f55c00a2a90 0x7f55c00a4f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:08:27.647 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:27.648+0000 7f55d9e00640 1 --2- 172.21.15.22:0/1885078804 >> [v2:172.21.15.22:6800/1357365082,v1:172.21.15.22:6801/1357365082] conn(0x7f55c00a2a90 0x7f55c00a4f50 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f55cc009fc0 tx=0x7f55cc005f90 comp rx=0 tx=0).ready entity=mgr.15522 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:08:27.656 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:27.656+0000 7f55d2a00640 1 -- 172.21.15.22:0/1885078804 <== mon.2 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f55c804c020 con 0x7f55dc0743f0 2024-03-28T18:08:27.952 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:27.952+0000 7f55db200640 1 -- 172.21.15.22:0/1885078804 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f55dc108e50 con 0x7f55dc0743f0 2024-03-28T18:08:27.953 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:27.954+0000 7f55d2a00640 1 -- 172.21.15.22:0/1885078804 <== mon.2 v2:172.21.15.79:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) v1 ==== 94+0+11452 (secure 0 0 0) 0x7f55c80bd820 con 0x7f55dc0743f0 2024-03-28T18:08:27.954 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-03-28T18:08:27.954 INFO:teuthology.orchestra.run.smithi022.stdout:{"epoch":27,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14544,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6835/3718238950","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6834","nonce":3718238950},{"type":"v1","addr":"172.21.15.22:6835","nonce":3718238950}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14562,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6837/3676939386","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6836","nonce":3676939386},{"type":"v1","addr":"172.21.15.22:6837","nonce":3676939386}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":27,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-28T18:04:48.991622+0000","modified":"2024-03-28T18:07:13.521383+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":34553,"mds_1":34511,"mds_2":34475,"mds_3":34442,"mds_4":24502},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14640":{"gid":14640,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.22:6839/3979998310","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6838","nonce":3979998310},{"type":"v1","addr":"172.21.15.22:6839","nonce":3979998310}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_14655":{"gid":14655,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.22:6841/3069056661","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6840","nonce":3069056661},{"type":"v1","addr":"172.21.15.22:6841","nonce":3069056661}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24412":{"gid":24412,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.134:6833/3176125362","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6832","nonce":3176125362},{"type":"v1","addr":"172.21.15.134:6833","nonce":3176125362}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24463":{"gid":24463,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.134:6835/3370549207","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6834","nonce":3370549207},{"type":"v1","addr":"172.21.15.134:6835","nonce":3370549207}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24478":{"gid":24478,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.134:6837/899825581","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6836","nonce":899825581},{"type":"v1","addr":"172.21.15.134:6837","nonce":899825581}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24502":{"gid":24502,"name":"i","rank":4,"incarnation":24,"state":"up:active","state_seq":7,"addr":"172.21.15.134:6839/3400263217","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6838","nonce":3400263217},{"type":"v1","addr":"172.21.15.134:6839","nonce":3400263217}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_34442":{"gid":34442,"name":"k","rank":3,"incarnation":22,"state":"up:active","state_seq":14,"addr":"172.21.15.79:6833/856896997","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6832","nonce":856896997},{"type":"v1","addr":"172.21.15.79:6833","nonce":856896997}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_34475":{"gid":34475,"name":"e","rank":2,"incarnation":20,"state":"up:active","state_seq":11,"addr":"172.21.15.79:6835/3365773431","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6834","nonce":3365773431},{"type":"v1","addr":"172.21.15.79:6835","nonce":3365773431}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_34511":{"gid":34511,"name":"h","rank":1,"incarnation":18,"state":"up:active","state_seq":8,"addr":"172.21.15.79:6837/1787903366","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6836","nonce":1787903366},{"type":"v1","addr":"172.21.15.79:6837","nonce":1787903366}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_34553":{"gid":34553,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.79:6839/2441542443","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6838","nonce":2441542443},{"type":"v1","addr":"172.21.15.79:6839","nonce":2441542443}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":34553,"qdb_cluster":[24502,34442,34475,34511,34553]},"id":1}]} 2024-03-28T18:08:27.955 INFO:teuthology.orchestra.run.smithi022.stderr:dumped fsmap epoch 27 2024-03-28T18:08:27.956 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:27.956+0000 7f55db200640 1 -- 172.21.15.22:0/1885078804 >> [v2:172.21.15.22:6800/1357365082,v1:172.21.15.22:6801/1357365082] conn(0x7f55c00a2a90 msgr2=0x7f55c00a4f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:08:27.956 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:27.956+0000 7f55db200640 1 --2- 172.21.15.22:0/1885078804 >> [v2:172.21.15.22:6800/1357365082,v1:172.21.15.22:6801/1357365082] conn(0x7f55c00a2a90 0x7f55c00a4f50 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f55cc009fc0 tx=0x7f55cc005f90 comp rx=0 tx=0).stop 2024-03-28T18:08:27.956 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:27.956+0000 7f55db200640 1 -- 172.21.15.22:0/1885078804 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f55dc0743f0 msgr2=0x7f55dc19a520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:08:27.956 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:27.956+0000 7f55db200640 1 --2- 172.21.15.22:0/1885078804 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f55dc0743f0 0x7f55dc19a520 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f55c8000c00 tx=0x7f55c8004530 comp rx=0 tx=0).stop 2024-03-28T18:08:27.956 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:27.957+0000 7f55db200640 1 -- 172.21.15.22:0/1885078804 shutdown_connections 2024-03-28T18:08:27.957 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:27.957+0000 7f55db200640 1 --2- 172.21.15.22:0/1885078804 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f55dc10dcd0 0x7f55dc19e8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:08:27.957 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:27.957+0000 7f55db200640 1 --2- 172.21.15.22:0/1885078804 >> [v2:172.21.15.22:6800/1357365082,v1:172.21.15.22:6801/1357365082] conn(0x7f55c00a2a90 0x7f55c00a4f50 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:08:27.957 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:27.957+0000 7f55db200640 1 --2- 172.21.15.22:0/1885078804 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f55dc0743f0 0x7f55dc19a520 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:08:27.957 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:27.957+0000 7f55db200640 1 --2- 172.21.15.22:0/1885078804 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f55dc073ad0 0x7f55dc199fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:08:27.957 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:27.957+0000 7f55db200640 1 -- 172.21.15.22:0/1885078804 >> 172.21.15.22:0/1885078804 conn(0x7f55dc0fcfa0 msgr2=0x7f55dc0fddb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:08:27.957 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:27.957+0000 7f55db200640 1 -- 172.21.15.22:0/1885078804 shutdown_connections 2024-03-28T18:08:27.957 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:27.957+0000 7f55db200640 1 -- 172.21.15.22:0/1885078804 wait complete. 2024-03-28T18:08:27.972 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-28T18:08:27.973 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2024-03-28T18:08:28.216 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:28.216+0000 7fb0e1c00640 1 -- 172.21.15.22:0/1787214872 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb0dc10ddd0 msgr2=0x7fb0dc1101c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:08:28.216 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:28.216+0000 7fb0e1c00640 1 --2- 172.21.15.22:0/1787214872 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb0dc10ddd0 0x7fb0dc1101c0 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7fb0d4009ca0 tx=0x7fb0d402c180 comp rx=0 tx=0).stop 2024-03-28T18:08:28.217 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:28.216+0000 7fb0e1c00640 1 -- 172.21.15.22:0/1787214872 shutdown_connections 2024-03-28T18:08:28.217 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:28.216+0000 7fb0e1c00640 1 --2- 172.21.15.22:0/1787214872 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb0dc10ddd0 0x7fb0dc1101c0 unknown :-1 s=CLOSED pgs=518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:08:28.217 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:28.216+0000 7fb0e1c00640 1 --2- 172.21.15.22:0/1787214872 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb0dc074370 0x7fb0dc10d890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:08:28.217 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:28.216+0000 7fb0e1c00640 1 --2- 172.21.15.22:0/1787214872 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fb0dc073a50 0x7fb0dc073e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:08:28.217 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:28.217+0000 7fb0e1c00640 1 -- 172.21.15.22:0/1787214872 >> 172.21.15.22:0/1787214872 conn(0x7fb0dc0fd040 msgr2=0x7fb0dc0ff460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:08:28.217 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:28.217+0000 7fb0e1c00640 1 -- 172.21.15.22:0/1787214872 shutdown_connections 2024-03-28T18:08:28.217 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:28.217+0000 7fb0e1c00640 1 -- 172.21.15.22:0/1787214872 wait complete. 2024-03-28T18:08:28.218 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:28.218+0000 7fb0e1c00640 1 Processor -- start 2024-03-28T18:08:28.219 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:28.218+0000 7fb0e1c00640 1 -- start start 2024-03-28T18:08:28.219 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:28.218+0000 7fb0e1c00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb0dc073a50 0x7fb0dc199f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:08:28.219 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:28.218+0000 7fb0e1c00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb0dc074370 0x7fb0dc19a4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:08:28.219 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:28.218+0000 7fb0e1c00640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fb0dc10ddd0 0x7fb0dc19e840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:08:28.219 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:28.218+0000 7fb0e1c00640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0dc113ac0 con 0x7fb0dc073a50 2024-03-28T18:08:28.219 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:28.218+0000 7fb0e1c00640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0dc113940 con 0x7fb0dc10ddd0 2024-03-28T18:08:28.219 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:28.218+0000 7fb0e1c00640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0dc113c40 con 0x7fb0dc074370 2024-03-28T18:08:28.219 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:28.218+0000 7fb0dbe00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb0dc073a50 0x7fb0dc199f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:08:28.219 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:28.218+0000 7fb0dbe00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb0dc073a50 0x7fb0dc199f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:37840/0 (socket says 172.21.15.22:37840) 2024-03-28T18:08:28.219 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:28.218+0000 7fb0dbe00640 1 -- 172.21.15.22:0/2044080486 learned_addr learned my addr 172.21.15.22:0/2044080486 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-28T18:08:28.219 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:28.219+0000 7fb0db400640 1 --2- 172.21.15.22:0/2044080486 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb0dc074370 0x7fb0dc19a4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:08:28.219 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:28.219+0000 7fb0e1200640 1 --2- 172.21.15.22:0/2044080486 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fb0dc10ddd0 0x7fb0dc19e840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:08:28.219 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:28.219+0000 7fb0dbe00640 1 -- 172.21.15.22:0/2044080486 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb0dc074370 msgr2=0x7fb0dc19a4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:08:28.219 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:28.219+0000 7fb0dbe00640 1 --2- 172.21.15.22:0/2044080486 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb0dc074370 0x7fb0dc19a4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:08:28.220 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:28.219+0000 7fb0dbe00640 1 -- 172.21.15.22:0/2044080486 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fb0dc10ddd0 msgr2=0x7fb0dc19e840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:08:28.220 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:28.219+0000 7fb0dbe00640 1 --2- 172.21.15.22:0/2044080486 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fb0dc10ddd0 0x7fb0dc19e840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:08:28.220 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:28.219+0000 7fb0dbe00640 1 -- 172.21.15.22:0/2044080486 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb0dc19ef70 con 0x7fb0dc073a50 2024-03-28T18:08:28.221 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:28.219+0000 7fb0dbe00640 1 --2- 172.21.15.22:0/2044080486 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb0dc073a50 0x7fb0dc199f80 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7fb0cc00e9f0 tx=0x7fb0cc00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:08:28.221 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:28.220+0000 7fb0d8c00640 1 -- 172.21.15.22:0/2044080486 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0cc00cd80 con 0x7fb0dc073a50 2024-03-28T18:08:28.221 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:28.220+0000 7fb0d8c00640 1 -- 172.21.15.22:0/2044080486 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7fb0cc010900 con 0x7fb0dc073a50 2024-03-28T18:08:28.221 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:28.220+0000 7fb0e1c00640 1 -- 172.21.15.22:0/2044080486 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb0dc19f260 con 0x7fb0dc073a50 2024-03-28T18:08:28.221 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:28.220+0000 7fb0d8c00640 1 -- 172.21.15.22:0/2044080486 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0cc0189e0 con 0x7fb0dc073a50 2024-03-28T18:08:28.221 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:28.220+0000 7fb0e1c00640 1 -- 172.21.15.22:0/2044080486 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb0dc1a6af0 con 0x7fb0dc073a50 2024-03-28T18:08:28.223 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:28.224+0000 7fb0e5f72740 1 Processor -- start 2024-03-28T18:08:28.224 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:28.224+0000 7fb0e5f72740 1 -- start start 2024-03-28T18:08:28.224 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:28.224+0000 7fb0d8c00640 1 -- 172.21.15.22:0/2044080486 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 25) v1 ==== 137523+0+0 (secure 0 0 0) 0x7fb0cc010ab0 con 0x7fb0dc073a50 2024-03-28T18:08:28.224 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:28.224+0000 7fb0e5f72740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ee887df8b0 0x55ee887dfc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:08:28.224 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:28.224+0000 7fb0e5f72740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55ee887e0260 0x55ee887e8810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:08:28.224 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:28.224+0000 7fb0e5f72740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ee887e8de0 0x55ee887eb230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:08:28.224 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:28.224+0000 7fb0e5f72740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ee887432c0 con 0x55ee887df8b0 2024-03-28T18:08:28.224 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:28.224+0000 7fb0e5f72740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ee886ebc90 con 0x55ee887e0260 2024-03-28T18:08:28.224 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:28.224+0000 7fb0e5f72740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ee8873f360 con 0x55ee887e8de0 2024-03-28T18:08:28.225 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:28.225+0000 7fb0e1200640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ee887df8b0 0x55ee887dfc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:08:28.225 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:28.225+0000 7fb0e1200640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ee887df8b0 0x55ee887dfc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:37854/0 (socket says 172.21.15.22:37854) 2024-03-28T18:08:28.225 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:28.225+0000 7fb0e1200640 1 -- 172.21.15.22:0/2067148887 learned_addr learned my addr 172.21.15.22:0/2067148887 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-28T18:08:28.225 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:28.225+0000 7fb0dbe00640 1 --2- 172.21.15.22:0/2067148887 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55ee887e0260 0x55ee887e8810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:08:28.225 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:28.225+0000 7fb0e1200640 1 -- 172.21.15.22:0/2067148887 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ee887e8de0 msgr2=0x55ee887eb230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T18:08:28.226 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:28.225+0000 7fb0e1200640 1 --2- 172.21.15.22:0/2067148887 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ee887e8de0 0x55ee887eb230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:08:28.226 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:28.225+0000 7fb0e1200640 1 -- 172.21.15.22:0/2067148887 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55ee887e0260 msgr2=0x55ee887e8810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T18:08:28.226 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:28.225+0000 7fb0e1200640 1 --2- 172.21.15.22:0/2067148887 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55ee887e0260 0x55ee887e8810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:08:28.226 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:28.225+0000 7fb0e1200640 1 -- 172.21.15.22:0/2067148887 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ee88710080 con 0x55ee887df8b0 2024-03-28T18:08:28.226 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:28.226+0000 7fb0e1200640 1 --2- 172.21.15.22:0/2067148887 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ee887df8b0 0x55ee887dfc90 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7fb0d4005e30 tx=0x7fb0d4005fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6e88dcc2fc8952fe server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:08:28.227 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:28.226+0000 7fb0d1600640 1 -- 172.21.15.22:0/2067148887 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0d4043020 con 0x55ee887df8b0 2024-03-28T18:08:28.227 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:28.226+0000 7fb0d1600640 1 -- 172.21.15.22:0/2067148887 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7fb0d4003e40 con 0x55ee887df8b0 2024-03-28T18:08:28.227 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:28.226+0000 7fb0d1600640 1 -- 172.21.15.22:0/2067148887 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0d4039040 con 0x55ee887df8b0 2024-03-28T18:08:28.228 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:28.226+0000 7fb0d8c00640 1 --2- 172.21.15.22:0/2044080486 >> [v2:172.21.15.22:6800/1357365082,v1:172.21.15.22:6801/1357365082] conn(0x7fb0c00a2b10 0x7fb0c00a4fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:08:28.228 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:28.227+0000 7fb0d8c00640 1 -- 172.21.15.22:0/2044080486 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 8411+0+0 (secure 0 0 0) 0x7fb0cc0cf4d0 con 0x7fb0dc073a50 2024-03-28T18:08:28.228 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:28.227+0000 7fb0dbe00640 1 --2- 172.21.15.22:0/2044080486 >> [v2:172.21.15.22:6800/1357365082,v1:172.21.15.22:6801/1357365082] conn(0x7fb0c00a2b10 0x7fb0c00a4fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:08:28.228 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:28.227+0000 7fb0e5f72740 1 -- 172.21.15.22:0/2067148887 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ee887df8b0 msgr2=0x55ee887dfc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:08:28.228 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:28.227+0000 7fb0e5f72740 1 --2- 172.21.15.22:0/2067148887 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ee887df8b0 0x55ee887dfc90 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7fb0d4005e30 tx=0x7fb0d4005fd0 comp rx=0 tx=0).stop 2024-03-28T18:08:28.228 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:28.227+0000 7fb0e5f72740 1 -- 172.21.15.22:0/2067148887 shutdown_connections 2024-03-28T18:08:28.228 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:28.227+0000 7fb0e5f72740 1 --2- 172.21.15.22:0/2067148887 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ee887e8de0 0x55ee887eb230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:08:28.228 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:28.227+0000 7fb0e5f72740 1 --2- 172.21.15.22:0/2067148887 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55ee887e0260 0x55ee887e8810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:08:28.228 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:28.227+0000 7fb0e5f72740 1 --2- 172.21.15.22:0/2067148887 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ee887df8b0 0x55ee887dfc90 unknown :-1 s=CLOSED pgs=520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:08:28.228 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:28.227+0000 7fb0e5f72740 1 -- 172.21.15.22:0/2067148887 >> 172.21.15.22:0/2067148887 conn(0x55ee887d9f80 msgr2=0x55ee887da360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:08:28.228 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:28.227+0000 7fb0e5f72740 1 -- 172.21.15.22:0/2067148887 shutdown_connections 2024-03-28T18:08:28.228 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:28.228+0000 7fb0e5f72740 1 -- 172.21.15.22:0/2067148887 wait complete. 2024-03-28T18:08:28.228 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:28.228+0000 7fb0dbe00640 1 --2- 172.21.15.22:0/2044080486 >> [v2:172.21.15.22:6800/1357365082,v1:172.21.15.22:6801/1357365082] conn(0x7fb0c00a2b10 0x7fb0c00a4fd0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fb0cc0ccb10 tx=0x7fb0cc000bc0 comp rx=0 tx=0).ready entity=mgr.15522 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:08:28.230 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:28.228+0000 7fb0e5f72740 1 -- start start 2024-03-28T18:08:28.230 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:28.229+0000 7fb0e5f72740 1 Processor -- start 2024-03-28T18:08:28.230 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:28.229+0000 7fb0e5f72740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ee887df8b0 0x55ee887e86b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:08:28.230 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:28.229+0000 7fb0e5f72740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ee887e0260 0x55ee887e5a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:08:28.230 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:28.229+0000 7fb0e5f72740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55ee887e8de0 0x55ee887e5fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:08:28.230 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:28.229+0000 7fb0e5f72740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ee887ebd40 con 0x55ee887df8b0 2024-03-28T18:08:28.230 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:28.229+0000 7fb0e5f72740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ee886ff0d0 con 0x55ee887e8de0 2024-03-28T18:08:28.230 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:28.229+0000 7fb0e5f72740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ee887ebec0 con 0x55ee887e0260 2024-03-28T18:08:28.230 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:28.229+0000 7fb0e5f72740 10 client.0 _ll_register_callbacks cb 0x55ee88703e80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-28T18:08:28.230 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:28.230+0000 7fb0e1200640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ee887df8b0 0x55ee887e86b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:08:28.230 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:28.230+0000 7fb0e1200640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ee887df8b0 0x55ee887e86b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:37864/0 (socket says 172.21.15.22:37864) 2024-03-28T18:08:28.231 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:28.230+0000 7fb0e1200640 1 -- 172.21.15.22:0/3789985343 learned_addr learned my addr 172.21.15.22:0/3789985343 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-28T18:08:28.231 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:28.230+0000 7fb0db400640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ee887e0260 0x55ee887e5a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:08:28.231 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:28.230+0000 7fb0e1200640 1 --2- 172.21.15.22:0/3789985343 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55ee887e8de0 0x55ee887e5fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:08:28.231 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:28.230+0000 7fb0e1200640 1 -- 172.21.15.22:0/3789985343 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ee887e0260 msgr2=0x55ee887e5a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T18:08:28.231 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:28.230+0000 7fb0e1200640 1 --2- 172.21.15.22:0/3789985343 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ee887e0260 0x55ee887e5a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:08:28.231 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:28.230+0000 7fb0e1200640 1 -- 172.21.15.22:0/3789985343 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55ee887e8de0 msgr2=0x55ee887e5fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T18:08:28.231 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:28.230+0000 7fb0e1200640 1 --2- 172.21.15.22:0/3789985343 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55ee887e8de0 0x55ee887e5fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:08:28.231 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:28.230+0000 7fb0e1200640 1 -- 172.21.15.22:0/3789985343 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55ee884573f0 con 0x55ee887df8b0 2024-03-28T18:08:28.232 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:28.231+0000 7fb0e1200640 1 --2- 172.21.15.22:0/3789985343 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ee887df8b0 0x55ee887e86b0 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7fb0d4009430 tx=0x7fb0d40095c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c5c184e930f701b0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:08:28.233 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:28.231+0000 7fb0c6000640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-03-28T18:08:28.233 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:28.231+0000 7fb0c6000640 1 -- 172.21.15.22:0/3789985343 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0d4039070 con 0x55ee887df8b0 2024-03-28T18:08:28.233 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:28.231+0000 7fb0c6000640 1 -- 172.21.15.22:0/3789985343 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 8411+0+0 (secure 0 0 0) 0x7fb0d40042d0 con 0x55ee887df8b0 2024-03-28T18:08:28.233 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:28.231+0000 7fb0e5f72740 1 -- 172.21.15.22:0/3789985343 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55ee88750800 con 0x55ee887df8b0 2024-03-28T18:08:28.233 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:28.232+0000 7fb0c6000640 1 -- 172.21.15.22:0/3789985343 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb0d4043070 con 0x55ee887df8b0 2024-03-28T18:08:28.233 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:28.232+0000 7fb0e5f72740 10 client.15804 fetch_fsmap learned FSMap version 27 2024-03-28T18:08:28.233 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:28.232+0000 7fb0e5f72740 1 -- 172.21.15.22:0/3789985343 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55ee885217d0 con 0x55ee887df8b0 2024-03-28T18:08:28.233 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:28.232+0000 7fb0c6000640 1 -- 172.21.15.22:0/3789985343 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7fb0d40044d0 con 0x55ee887df8b0 2024-03-28T18:08:28.233 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:28.232+0000 7fb0c6000640 1 -- 172.21.15.22:0/3789985343 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 27) v1 ==== 7934+0+0 (secure 0 0 0) 0x7fb0d403f6d0 con 0x55ee887df8b0 2024-03-28T18:08:28.233 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:28.232+0000 7fb0e5f72740 10 client.15804 fetch_fsmap finished waiting for FSMap version 27 2024-03-28T18:08:28.233 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:28.232+0000 7fb0e5f72740 10 client.15804 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.i 2024-03-28T18:08:28.233 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:28.232+0000 7fb0e5f72740 20 client.15804 populate_metadata read hostname 'smithi022' 2024-03-28T18:08:28.233 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:28.232+0000 7fb0e5f72740 1 --2- 172.21.15.22:0/3789985343 >> [v2:172.21.15.134:6838/3400263217,v1:172.21.15.134:6839/3400263217] conn(0x55ee888012b0 0x55ee888036a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:08:28.233 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:28.232+0000 7fb0e5f72740 4 client.15804 mds_command: new command op to 24502 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-28T18:08:28.233 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:28.232+0000 7fb0e5f72740 1 -- 172.21.15.22:0/3789985343 --> [v2:172.21.15.134:6838/3400263217,v1:172.21.15.134:6839/3400263217] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55ee88750800 con 0x55ee888012b0 2024-03-28T18:08:28.234 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:28.233+0000 7fb0db400640 1 --2- 172.21.15.22:0/3789985343 >> [v2:172.21.15.134:6838/3400263217,v1:172.21.15.134:6839/3400263217] conn(0x55ee888012b0 0x55ee888036a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:08:28.234 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:28.234+0000 7fb0c6000640 10 client.15804 ms_handle_connect on v2:172.21.15.134:6838/3400263217 2024-03-28T18:08:28.234 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:28.234+0000 7fb0db400640 1 --2- 172.21.15.22:0/3789985343 >> [v2:172.21.15.134:6838/3400263217,v1:172.21.15.134:6839/3400263217] conn(0x55ee888012b0 0x55ee888036a0 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=acba307b34a3b2eb server_cookie=c23bb28bb1a2568c in_seq=0 out_seq=0 2024-03-28T18:08:28.236 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:28.236+0000 7fb0c6000640 1 -- 172.21.15.22:0/3789985343 <== mds.4 v2:172.21.15.134:6838/3400263217 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x55ee88750800 con 0x55ee888012b0 2024-03-28T18:08:28.236 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:28.236+0000 7fb0c6000640 10 client.15804 handle_command_reply: tid=0 2024-03-28T18:08:28.236 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:28.236+0000 7fb0e5f72740 1 client.15804 shutdown 2024-03-28T18:08:28.236 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:28.236+0000 7fb0e5f72740 1 -- 172.21.15.22:0/3789985343 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ee887df8b0 msgr2=0x55ee887e86b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:08:28.236 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:28.236+0000 7fb0e5f72740 1 --2- 172.21.15.22:0/3789985343 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ee887df8b0 0x55ee887e86b0 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7fb0d4009430 tx=0x7fb0d40095c0 comp rx=0 tx=0).stop 2024-03-28T18:08:28.237 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:28.237+0000 7fb0e5f72740 1 -- 172.21.15.22:0/3789985343 shutdown_connections 2024-03-28T18:08:28.237 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:28.237+0000 7fb0e5f72740 1 --2- 172.21.15.22:0/3789985343 >> [v2:172.21.15.134:6838/3400263217,v1:172.21.15.134:6839/3400263217] conn(0x55ee888012b0 0x55ee888036a0 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:08:28.237 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:28.237+0000 7fb0e5f72740 1 --2- 172.21.15.22:0/3789985343 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55ee887e8de0 0x55ee887e5fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:08:28.237 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:28.237+0000 7fb0e5f72740 1 --2- 172.21.15.22:0/3789985343 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ee887e0260 0x55ee887e5a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:08:28.237 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:28.237+0000 7fb0e5f72740 1 --2- 172.21.15.22:0/3789985343 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ee887df8b0 0x55ee887e86b0 unknown :-1 s=CLOSED pgs=521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:08:28.237 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:28.237+0000 7fb0e5f72740 1 -- 172.21.15.22:0/3789985343 >> 172.21.15.22:0/3789985343 conn(0x55ee887d9f80 msgr2=0x55ee887dbcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:08:28.237 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:28.237+0000 7fb0c6000640 0 client.15804 ms_handle_reset on v2:172.21.15.134:6838/3400263217 2024-03-28T18:08:28.237 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:28.237+0000 7fb0e5f72740 1 -- 172.21.15.22:0/3789985343 shutdown_connections 2024-03-28T18:08:28.237 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:28.237+0000 7fb0e5f72740 1 -- 172.21.15.22:0/3789985343 wait complete. 2024-03-28T18:08:28.237 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:28.237+0000 7fb0e5f72740 20 client.15804 trim_cache size 0 max 16384 2024-03-28T18:08:28.252 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:28.252+0000 7fb0e5f72740 1 Processor -- start 2024-03-28T18:08:28.252 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:28.252+0000 7fb0e5f72740 1 -- start start 2024-03-28T18:08:28.253 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:28.253+0000 7fb0e5f72740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55ee887df8b0 0x55ee887d50a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:08:28.253 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:28.253+0000 7fb0e5f72740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ee887e0260 0x55ee887e8170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:08:28.253 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:28.253+0000 7fb0e5f72740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ee887e8de0 0x55ee887e86b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:08:28.253 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:28.253+0000 7fb0e5f72740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0dc1aabc0 con 0x55ee887e0260 2024-03-28T18:08:28.253 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:28.253+0000 7fb0e5f72740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0dc1ac540 con 0x55ee887df8b0 2024-03-28T18:08:28.254 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:28.253+0000 7fb0e5f72740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0dc1ab220 con 0x55ee887e8de0 2024-03-28T18:08:28.254 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:28.253+0000 7fb0db400640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ee887e0260 0x55ee887e8170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:08:28.254 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:28.253+0000 7fb0db400640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ee887e0260 0x55ee887e8170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:37874/0 (socket says 172.21.15.22:37874) 2024-03-28T18:08:28.254 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:28.253+0000 7fb0db400640 1 -- 172.21.15.22:0/1981550307 learned_addr learned my addr 172.21.15.22:0/1981550307 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-28T18:08:28.254 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:28.253+0000 7fb0e1200640 1 --2- 172.21.15.22:0/1981550307 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55ee887df8b0 0x55ee887d50a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:08:28.254 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:28.253+0000 7fb0e1200640 1 --2- 172.21.15.22:0/1981550307 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ee887e8de0 0x55ee887e86b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:08:28.254 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:28.253+0000 7fb0db400640 1 -- 172.21.15.22:0/1981550307 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ee887e8de0 msgr2=0x55ee887e86b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T18:08:28.254 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:28.253+0000 7fb0db400640 1 --2- 172.21.15.22:0/1981550307 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ee887e8de0 0x55ee887e86b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:08:28.254 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:28.253+0000 7fb0db400640 1 -- 172.21.15.22:0/1981550307 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55ee887df8b0 msgr2=0x55ee887d50a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T18:08:28.254 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:28.253+0000 7fb0db400640 1 --2- 172.21.15.22:0/1981550307 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55ee887df8b0 0x55ee887d50a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:08:28.254 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:28.253+0000 7fb0db400640 1 -- 172.21.15.22:0/1981550307 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ee887e6310 con 0x55ee887e0260 2024-03-28T18:08:28.254 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:28.254+0000 7fb0e1200640 1 --2- 172.21.15.22:0/1981550307 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ee887e8de0 0x55ee887e86b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T18:08:28.254 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:28.254+0000 7fb0db400640 1 --2- 172.21.15.22:0/1981550307 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ee887e0260 0x55ee887e8170 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x55ee887ebf10 tx=0x7fb0c8005d40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eb8e03c242fd1fd5 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:08:28.256 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:28.254+0000 7fb0d0c00640 1 -- 172.21.15.22:0/1981550307 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0c8004230 con 0x55ee887e0260 2024-03-28T18:08:28.256 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:28.254+0000 7fb0d0c00640 1 -- 172.21.15.22:0/1981550307 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7fb0c80043d0 con 0x55ee887e0260 2024-03-28T18:08:28.256 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:28.254+0000 7fb0d0c00640 1 -- 172.21.15.22:0/1981550307 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0c80117a0 con 0x55ee887e0260 2024-03-28T18:08:28.256 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:28.255+0000 7fb0e5f72740 1 -- 172.21.15.22:0/1981550307 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ee887e0260 msgr2=0x55ee887e8170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:08:28.256 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:28.255+0000 7fb0e5f72740 1 --2- 172.21.15.22:0/1981550307 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ee887e0260 0x55ee887e8170 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x55ee887ebf10 tx=0x7fb0c8005d40 comp rx=0 tx=0).stop 2024-03-28T18:08:28.256 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:28.255+0000 7fb0e5f72740 1 -- 172.21.15.22:0/1981550307 shutdown_connections 2024-03-28T18:08:28.256 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:28.255+0000 7fb0e5f72740 1 --2- 172.21.15.22:0/1981550307 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ee887e8de0 0x55ee887e86b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:08:28.256 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:28.255+0000 7fb0e5f72740 1 --2- 172.21.15.22:0/1981550307 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ee887e0260 0x55ee887e8170 unknown :-1 s=CLOSED pgs=522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:08:28.256 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:28.255+0000 7fb0e5f72740 1 --2- 172.21.15.22:0/1981550307 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55ee887df8b0 0x55ee887d50a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:08:28.256 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:28.255+0000 7fb0e5f72740 1 -- 172.21.15.22:0/1981550307 >> 172.21.15.22:0/1981550307 conn(0x55ee887d9f80 msgr2=0x55ee887d9160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:08:28.256 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:28.255+0000 7fb0e5f72740 1 -- 172.21.15.22:0/1981550307 shutdown_connections 2024-03-28T18:08:28.256 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:28.255+0000 7fb0e5f72740 1 -- 172.21.15.22:0/1981550307 wait complete. 2024-03-28T18:08:28.256 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:28.256+0000 7fb0e5f72740 1 -- start start 2024-03-28T18:08:28.256 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:28.256+0000 7fb0e5f72740 1 Processor -- start 2024-03-28T18:08:28.259 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:28.256+0000 7fb0e5f72740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55ee887df8b0 0x55ee88805d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:08:28.259 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:28.256+0000 7fb0e5f72740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ee887e0260 0x55ee88806240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:08:28.259 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:28.257+0000 7fb0e5f72740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ee887e8de0 0x55ee888041f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:08:28.259 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:28.257+0000 7fb0e5f72740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0dc1abee0 con 0x55ee887e0260 2024-03-28T18:08:28.259 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:28.257+0000 7fb0e5f72740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0dc1ab880 con 0x55ee887df8b0 2024-03-28T18:08:28.259 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:28.257+0000 7fb0e5f72740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0dc1a8dc0 con 0x55ee887e8de0 2024-03-28T18:08:28.259 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:28.257+0000 7fb0e5f72740 10 client.0 _ll_register_callbacks cb 0x7fb0cc005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-28T18:08:28.259 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:28.257+0000 7fb0db400640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ee887e0260 0x55ee88806240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:08:28.259 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:28.257+0000 7fb0db400640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ee887e0260 0x55ee88806240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:37884/0 (socket says 172.21.15.22:37884) 2024-03-28T18:08:28.259 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:28.257+0000 7fb0db400640 1 -- 172.21.15.22:0/2933364111 learned_addr learned my addr 172.21.15.22:0/2933364111 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-28T18:08:28.259 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:28.257+0000 7fb0e1200640 1 --2- 172.21.15.22:0/2933364111 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55ee887df8b0 0x55ee88805d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:08:28.259 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:28.257+0000 7fb0e1200640 1 --2- 172.21.15.22:0/2933364111 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ee887e8de0 0x55ee888041f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:08:28.260 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:28.257+0000 7fb0db400640 1 -- 172.21.15.22:0/2933364111 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ee887e8de0 msgr2=0x55ee888041f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T18:08:28.260 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:28.257+0000 7fb0db400640 1 --2- 172.21.15.22:0/2933364111 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ee887e8de0 0x55ee888041f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:08:28.260 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:28.257+0000 7fb0db400640 1 -- 172.21.15.22:0/2933364111 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55ee887df8b0 msgr2=0x55ee88805d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T18:08:28.260 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:28.257+0000 7fb0db400640 1 --2- 172.21.15.22:0/2933364111 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55ee887df8b0 0x55ee88805d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:08:28.260 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:28.258+0000 7fb0db400640 1 -- 172.21.15.22:0/2933364111 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55ee88804930 con 0x55ee887e0260 2024-03-28T18:08:28.260 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:28.258+0000 7fb0e1200640 1 --2- 172.21.15.22:0/2933364111 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ee887e8de0 0x55ee888041f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-28T18:08:28.260 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:28.258+0000 7fb0db400640 1 --2- 172.21.15.22:0/2933364111 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ee887e0260 0x55ee88806240 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7fb0c8000c00 tx=0x7fb0c8003f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=526b84f3581e6407 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:08:28.260 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:28.258+0000 7fb0c6000640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-03-28T18:08:28.260 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:28.258+0000 7fb0c6000640 1 -- 172.21.15.22:0/2933364111 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0c8003a00 con 0x55ee887e0260 2024-03-28T18:08:28.260 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:28.258+0000 7fb0c6000640 1 -- 172.21.15.22:0/2933364111 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 8411+0+0 (secure 0 0 0) 0x7fb0c8003ba0 con 0x55ee887e0260 2024-03-28T18:08:28.260 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:28.258+0000 7fb0e5f72740 1 -- 172.21.15.22:0/2933364111 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fb0c80043d0 con 0x55ee887e0260 2024-03-28T18:08:28.260 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:28.259+0000 7fb0c6000640 1 -- 172.21.15.22:0/2933364111 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb0c8035050 con 0x55ee887e0260 2024-03-28T18:08:28.260 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:28.259+0000 7fb0e5f72740 10 client.15810 fetch_fsmap learned FSMap version 27 2024-03-28T18:08:28.260 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:28.259+0000 7fb0e5f72740 1 -- 172.21.15.22:0/2933364111 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55ee8885d160 con 0x55ee887e0260 2024-03-28T18:08:28.260 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:28.259+0000 7fb0c6000640 1 -- 172.21.15.22:0/2933364111 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7fb0c80043d0 con 0x55ee887e0260 2024-03-28T18:08:28.262 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:28.259+0000 7fb0c6000640 1 -- 172.21.15.22:0/2933364111 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 27) v1 ==== 7934+0+0 (secure 0 0 0) 0x7fb0c80233f0 con 0x55ee887e0260 2024-03-28T18:08:28.262 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:28.259+0000 7fb0e5f72740 10 client.15810 fetch_fsmap finished waiting for FSMap version 27 2024-03-28T18:08:28.262 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:28.259+0000 7fb0e5f72740 10 client.15810 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.i 2024-03-28T18:08:28.262 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:28.259+0000 7fb0e5f72740 20 client.15810 populate_metadata read hostname 'smithi022' 2024-03-28T18:08:28.262 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:28.259+0000 7fb0e5f72740 1 --2- 172.21.15.22:0/2933364111 >> [v2:172.21.15.134:6838/3400263217,v1:172.21.15.134:6839/3400263217] conn(0x55ee8885d7e0 0x55ee8885fc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:08:28.262 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:28.259+0000 7fb0e5f72740 4 client.15810 mds_command: new command op to 24502 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-28T18:08:28.263 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:28.259+0000 7fb0e5f72740 1 -- 172.21.15.22:0/2933364111 --> [v2:172.21.15.134:6838/3400263217,v1:172.21.15.134:6839/3400263217] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55ee8874ab50 con 0x55ee8885d7e0 2024-03-28T18:08:28.263 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:28.260+0000 7fb0e1200640 1 --2- 172.21.15.22:0/2933364111 >> [v2:172.21.15.134:6838/3400263217,v1:172.21.15.134:6839/3400263217] conn(0x55ee8885d7e0 0x55ee8885fc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:08:28.263 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:28.261+0000 7fb0e1200640 1 --2- 172.21.15.22:0/2933364111 >> [v2:172.21.15.134:6838/3400263217,v1:172.21.15.134:6839/3400263217] conn(0x55ee8885d7e0 0x55ee8885fc00 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=5b43721664e1e237 server_cookie=28a2700b5e4af2ce in_seq=0 out_seq=0 2024-03-28T18:08:28.263 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:28.261+0000 7fb0c6000640 10 client.15810 ms_handle_connect on v2:172.21.15.134:6838/3400263217 2024-03-28T18:08:28.263 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:28.261+0000 7fb0c6000640 1 -- 172.21.15.22:0/2933364111 <== mds.4 v2:172.21.15.134:6838/3400263217 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55ee8874ab50 con 0x55ee8885d7e0 2024-03-28T18:08:28.263 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:28.261+0000 7fb0c6000640 10 client.15810 handle_command_reply: tid=0 2024-03-28T18:08:28.263 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:28.261+0000 7fb0e5f72740 1 client.15810 shutdown 2024-03-28T18:08:28.263 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:28.262+0000 7fb0e5f72740 1 -- 172.21.15.22:0/2933364111 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ee887e0260 msgr2=0x55ee88806240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:08:28.263 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:28.262+0000 7fb0e5f72740 1 --2- 172.21.15.22:0/2933364111 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ee887e0260 0x55ee88806240 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7fb0c8000c00 tx=0x7fb0c8003f90 comp rx=0 tx=0).stop 2024-03-28T18:08:28.263 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:28.262+0000 7fb0e5f72740 1 -- 172.21.15.22:0/2933364111 shutdown_connections 2024-03-28T18:08:28.263 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:28.262+0000 7fb0e5f72740 1 --2- 172.21.15.22:0/2933364111 >> [v2:172.21.15.134:6838/3400263217,v1:172.21.15.134:6839/3400263217] conn(0x55ee8885d7e0 0x55ee8885fc00 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:08:28.263 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:28.262+0000 7fb0e5f72740 1 --2- 172.21.15.22:0/2933364111 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ee887e8de0 0x55ee888041f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:08:28.263 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:28.262+0000 7fb0e5f72740 1 --2- 172.21.15.22:0/2933364111 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ee887e0260 0x55ee88806240 unknown :-1 s=CLOSED pgs=523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:08:28.263 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:28.262+0000 7fb0e5f72740 1 --2- 172.21.15.22:0/2933364111 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55ee887df8b0 0x55ee88805d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:08:28.263 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:28.262+0000 7fb0e5f72740 1 -- 172.21.15.22:0/2933364111 >> 172.21.15.22:0/2933364111 conn(0x55ee887d9f80 msgr2=0x55ee887d9160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:08:28.264 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:28.262+0000 7fb0c6000640 0 client.15810 ms_handle_reset on v2:172.21.15.134:6838/3400263217 2024-03-28T18:08:28.264 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:28.262+0000 7fb0e5f72740 1 -- 172.21.15.22:0/2933364111 shutdown_connections 2024-03-28T18:08:28.264 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:28.262+0000 7fb0e5f72740 1 -- 172.21.15.22:0/2933364111 wait complete. 2024-03-28T18:08:28.264 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:28.262+0000 7fb0e5f72740 20 client.15810 trim_cache size 0 max 16384 2024-03-28T18:08:28.264 INFO:teuthology.orchestra.run.smithi022.stdout:[] 2024-03-28T18:08:28.265 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:28.263+0000 7fb0e1c00640 1 -- 172.21.15.22:0/2044080486 >> [v2:172.21.15.22:6800/1357365082,v1:172.21.15.22:6801/1357365082] conn(0x7fb0c00a2b10 msgr2=0x7fb0c00a4fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:08:28.265 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:28.263+0000 7fb0e1c00640 1 --2- 172.21.15.22:0/2044080486 >> [v2:172.21.15.22:6800/1357365082,v1:172.21.15.22:6801/1357365082] conn(0x7fb0c00a2b10 0x7fb0c00a4fd0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fb0cc0ccb10 tx=0x7fb0cc000bc0 comp rx=0 tx=0).stop 2024-03-28T18:08:28.265 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:28.263+0000 7fb0e1c00640 1 -- 172.21.15.22:0/2044080486 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb0dc073a50 msgr2=0x7fb0dc199f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:08:28.265 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:28.263+0000 7fb0e1c00640 1 --2- 172.21.15.22:0/2044080486 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb0dc073a50 0x7fb0dc199f80 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7fb0cc00e9f0 tx=0x7fb0cc00eeb0 comp rx=0 tx=0).stop 2024-03-28T18:08:28.265 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:28.263+0000 7fb0e1c00640 1 -- 172.21.15.22:0/2044080486 shutdown_connections 2024-03-28T18:08:28.265 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:28.263+0000 7fb0e1c00640 1 --2- 172.21.15.22:0/2044080486 >> [v2:172.21.15.22:6800/1357365082,v1:172.21.15.22:6801/1357365082] conn(0x7fb0c00a2b10 0x7fb0c00a4fd0 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:08:28.265 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:28.263+0000 7fb0e1c00640 1 --2- 172.21.15.22:0/2044080486 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fb0dc10ddd0 0x7fb0dc19e840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:08:28.265 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:28.263+0000 7fb0e1c00640 1 --2- 172.21.15.22:0/2044080486 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb0dc074370 0x7fb0dc19a4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:08:28.265 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:28.263+0000 7fb0e1c00640 1 --2- 172.21.15.22:0/2044080486 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb0dc073a50 0x7fb0dc199f80 unknown :-1 s=CLOSED pgs=519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:08:28.265 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:28.263+0000 7fb0e1c00640 1 -- 172.21.15.22:0/2044080486 >> 172.21.15.22:0/2044080486 conn(0x7fb0dc0fd040 msgr2=0x7fb0dc0fddb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:08:28.266 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:28.263+0000 7fb0e1c00640 1 -- 172.21.15.22:0/2044080486 shutdown_connections 2024-03-28T18:08:28.266 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:28.263+0000 7fb0e1c00640 1 -- 172.21.15.22:0/2044080486 wait complete. 2024-03-28T18:08:28.277 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:08:28 smithi022 ceph-mon[31985]: pgmap v30: 97 pgs: 97 active+clean; 1.3 MiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-28T18:08:28.277 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:08:28 smithi022 ceph-mon[31985]: from='client.? 172.21.15.22:0/1885078804' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-28T18:08:28.279 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-28T18:08:28.279 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2024-03-28T18:08:28.424 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:08:28 smithi134 ceph-mon[35037]: pgmap v30: 97 pgs: 97 active+clean; 1.3 MiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-28T18:08:28.424 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:08:28 smithi134 ceph-mon[35037]: from='client.? 172.21.15.22:0/1885078804' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-28T18:08:28.432 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:08:28 smithi079 ceph-mon[35312]: pgmap v30: 97 pgs: 97 active+clean; 1.3 MiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-28T18:08:28.432 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:08:28 smithi079 ceph-mon[35312]: from='client.? 172.21.15.22:0/1885078804' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-28T18:08:28.490 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:28.490+0000 7fc381600640 1 -- 172.21.15.22:0/943517383 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc37c10ddb0 msgr2=0x7fc37c1101c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:08:28.490 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:28.490+0000 7fc381600640 1 --2- 172.21.15.22:0/943517383 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc37c10ddb0 0x7fc37c1101c0 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7fc374009ca0 tx=0x7fc37402c190 comp rx=0 tx=0).stop 2024-03-28T18:08:28.490 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:28.491+0000 7fc381600640 1 -- 172.21.15.22:0/943517383 shutdown_connections 2024-03-28T18:08:28.490 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:28.491+0000 7fc381600640 1 --2- 172.21.15.22:0/943517383 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc37c10ddb0 0x7fc37c1101c0 unknown :-1 s=CLOSED pgs=524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:08:28.490 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:28.491+0000 7fc381600640 1 --2- 172.21.15.22:0/943517383 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc37c074390 0x7fc37c10d870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:08:28.490 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:28.491+0000 7fc381600640 1 --2- 172.21.15.22:0/943517383 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc37c073a70 0x7fc37c073e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:08:28.490 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:28.491+0000 7fc381600640 1 -- 172.21.15.22:0/943517383 >> 172.21.15.22:0/943517383 conn(0x7fc37c0fd040 msgr2=0x7fc37c0ff460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:08:28.491 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:28.491+0000 7fc381600640 1 -- 172.21.15.22:0/943517383 shutdown_connections 2024-03-28T18:08:28.491 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:28.491+0000 7fc381600640 1 -- 172.21.15.22:0/943517383 wait complete. 2024-03-28T18:08:28.491 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:28.492+0000 7fc381600640 1 Processor -- start 2024-03-28T18:08:28.492 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:28.492+0000 7fc381600640 1 -- start start 2024-03-28T18:08:28.493 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:28.493+0000 7fc381600640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc37c073a70 0x7fc37c199fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:08:28.493 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:28.493+0000 7fc381600640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc37c074390 0x7fc37c19a4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:08:28.493 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:28.493+0000 7fc381600640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc37c10ddb0 0x7fc37c19e860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:08:28.493 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:28.493+0000 7fc381600640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc37c113ae0 con 0x7fc37c074390 2024-03-28T18:08:28.493 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:28.493+0000 7fc381600640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc37c113960 con 0x7fc37c073a70 2024-03-28T18:08:28.493 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:28.493+0000 7fc381600640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc37c113c60 con 0x7fc37c10ddb0 2024-03-28T18:08:28.493 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:28.493+0000 7fc37be00640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc37c073a70 0x7fc37c199fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:08:28.493 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:28.493+0000 7fc37b400640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc37c074390 0x7fc37c19a4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:08:28.493 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:28.493+0000 7fc37be00640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc37c073a70 0x7fc37c199fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.22:47174/0 (socket says 172.21.15.22:47174) 2024-03-28T18:08:28.494 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:28.493+0000 7fc37be00640 1 -- 172.21.15.22:0/1336328280 learned_addr learned my addr 172.21.15.22:0/1336328280 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-28T18:08:28.495 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:28.494+0000 7fc380c00640 1 --2- 172.21.15.22:0/1336328280 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc37c10ddb0 0x7fc37c19e860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:08:28.495 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:28.494+0000 7fc37b400640 1 -- 172.21.15.22:0/1336328280 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc37c10ddb0 msgr2=0x7fc37c19e860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:08:28.495 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:28.494+0000 7fc37b400640 1 --2- 172.21.15.22:0/1336328280 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc37c10ddb0 0x7fc37c19e860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:08:28.495 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:28.494+0000 7fc37b400640 1 -- 172.21.15.22:0/1336328280 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc37c073a70 msgr2=0x7fc37c199fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:08:28.495 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:28.494+0000 7fc37b400640 1 --2- 172.21.15.22:0/1336328280 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc37c073a70 0x7fc37c199fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:08:28.495 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:28.494+0000 7fc37b400640 1 -- 172.21.15.22:0/1336328280 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc37c19ef90 con 0x7fc37c074390 2024-03-28T18:08:28.495 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:28.494+0000 7fc37be00640 1 --2- 172.21.15.22:0/1336328280 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc37c073a70 0x7fc37c199fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-28T18:08:28.495 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:28.494+0000 7fc380c00640 1 --2- 172.21.15.22:0/1336328280 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc37c10ddb0 0x7fc37c19e860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-28T18:08:28.495 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:28.494+0000 7fc37b400640 1 --2- 172.21.15.22:0/1336328280 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc37c074390 0x7fc37c19a4e0 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7fc36800e9f0 tx=0x7fc36800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:08:28.495 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:28.495+0000 7fc378c00640 1 -- 172.21.15.22:0/1336328280 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc36800ccc0 con 0x7fc37c074390 2024-03-28T18:08:28.495 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:28.495+0000 7fc381600640 1 -- 172.21.15.22:0/1336328280 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc37c19f280 con 0x7fc37c074390 2024-03-28T18:08:28.495 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:28.495+0000 7fc378c00640 1 -- 172.21.15.22:0/1336328280 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7fc368010900 con 0x7fc37c074390 2024-03-28T18:08:28.495 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:28.495+0000 7fc378c00640 1 -- 172.21.15.22:0/1336328280 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc368018960 con 0x7fc37c074390 2024-03-28T18:08:28.496 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:28.495+0000 7fc381600640 1 -- 172.21.15.22:0/1336328280 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc37c1a6b10 con 0x7fc37c074390 2024-03-28T18:08:28.498 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:28.499+0000 7fc38599d740 1 Processor -- start 2024-03-28T18:08:28.498 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:28.499+0000 7fc38599d740 1 -- start start 2024-03-28T18:08:28.499 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:28.499+0000 7fc38599d740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55d0ac5198b0 0x55d0ac519c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:08:28.500 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:28.499+0000 7fc38599d740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d0ac51a260 0x55d0ac522810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:08:28.500 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:28.499+0000 7fc38599d740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d0ac522de0 0x55d0ac525230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:08:28.500 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:28.499+0000 7fc38599d740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d0ac47d2c0 con 0x55d0ac51a260 2024-03-28T18:08:28.500 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:28.499+0000 7fc38599d740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d0ac425c90 con 0x55d0ac5198b0 2024-03-28T18:08:28.500 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:28.499+0000 7fc38599d740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d0ac479360 con 0x55d0ac522de0 2024-03-28T18:08:28.500 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:28.500+0000 7fc380c00640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55d0ac5198b0 0x55d0ac519c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:08:28.500 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:28.500+0000 7fc380c00640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55d0ac5198b0 0x55d0ac519c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.22:47190/0 (socket says 172.21.15.22:47190) 2024-03-28T18:08:28.500 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:28.500+0000 7fc380c00640 1 -- 172.21.15.22:0/3872690251 learned_addr learned my addr 172.21.15.22:0/3872690251 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-28T18:08:28.500 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:28.500+0000 7fc380c00640 1 -- 172.21.15.22:0/3872690251 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d0ac522de0 msgr2=0x55d0ac525230 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-28T18:08:28.500 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:28.500+0000 7fc380c00640 1 --2- 172.21.15.22:0/3872690251 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d0ac522de0 0x55d0ac525230 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:08:28.500 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:28.500+0000 7fc380c00640 1 -- 172.21.15.22:0/3872690251 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d0ac51a260 msgr2=0x55d0ac522810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T18:08:28.500 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:28.500+0000 7fc380c00640 1 --2- 172.21.15.22:0/3872690251 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d0ac51a260 0x55d0ac522810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:08:28.500 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:28.500+0000 7fc380c00640 1 -- 172.21.15.22:0/3872690251 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d0ac44a080 con 0x55d0ac5198b0 2024-03-28T18:08:28.501 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:28.501+0000 7fc380c00640 1 --2- 172.21.15.22:0/3872690251 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55d0ac5198b0 0x55d0ac519c90 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fc37c0749c0 tx=0x7fc374009880 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9c7d44dcd6756af2 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:08:28.501 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:28.501+0000 7fc371200640 1 -- 172.21.15.22:0/3872690251 <== mon.2 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc374039040 con 0x55d0ac5198b0 2024-03-28T18:08:28.501 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:28.501+0000 7fc371200640 1 -- 172.21.15.22:0/3872690251 <== mon.2 v2:172.21.15.79:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7fc374003e40 con 0x55d0ac5198b0 2024-03-28T18:08:28.501 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:28.501+0000 7fc371200640 1 -- 172.21.15.22:0/3872690251 <== mon.2 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc374032870 con 0x55d0ac5198b0 2024-03-28T18:08:28.501 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:28.502+0000 7fc378c00640 1 -- 172.21.15.22:0/1336328280 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 25) v1 ==== 137523+0+0 (secure 0 0 0) 0x7fc368010ab0 con 0x7fc37c074390 2024-03-28T18:08:28.502 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:28.502+0000 7fc38599d740 1 -- 172.21.15.22:0/3872690251 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55d0ac5198b0 msgr2=0x55d0ac519c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:08:28.502 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:28.502+0000 7fc38599d740 1 --2- 172.21.15.22:0/3872690251 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55d0ac5198b0 0x55d0ac519c90 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fc37c0749c0 tx=0x7fc374009880 comp rx=0 tx=0).stop 2024-03-28T18:08:28.502 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:28.502+0000 7fc38599d740 1 -- 172.21.15.22:0/3872690251 shutdown_connections 2024-03-28T18:08:28.503 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:28.502+0000 7fc38599d740 1 --2- 172.21.15.22:0/3872690251 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d0ac522de0 0x55d0ac525230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:08:28.503 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:28.502+0000 7fc38599d740 1 --2- 172.21.15.22:0/3872690251 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d0ac51a260 0x55d0ac522810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:08:28.503 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:28.502+0000 7fc38599d740 1 --2- 172.21.15.22:0/3872690251 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55d0ac5198b0 0x55d0ac519c90 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:08:28.503 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:28.502+0000 7fc38599d740 1 -- 172.21.15.22:0/3872690251 >> 172.21.15.22:0/3872690251 conn(0x55d0ac513f80 msgr2=0x55d0ac514360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:08:28.503 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:28.502+0000 7fc38599d740 1 -- 172.21.15.22:0/3872690251 shutdown_connections 2024-03-28T18:08:28.503 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:28.502+0000 7fc38599d740 1 -- 172.21.15.22:0/3872690251 wait complete. 2024-03-28T18:08:28.503 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:28.503+0000 7fc38599d740 1 -- start start 2024-03-28T18:08:28.503 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:28.503+0000 7fc38599d740 1 Processor -- start 2024-03-28T18:08:28.504 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:28.503+0000 7fc38599d740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d0ac5198b0 0x55d0ac5226b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:08:28.504 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:28.503+0000 7fc38599d740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d0ac51a260 0x55d0ac51fa40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:08:28.504 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:28.503+0000 7fc38599d740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55d0ac522de0 0x55d0ac51ffa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:08:28.504 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:28.503+0000 7fc38599d740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d0ac525cf0 con 0x55d0ac51a260 2024-03-28T18:08:28.504 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:28.503+0000 7fc38599d740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d0ac4390d0 con 0x55d0ac522de0 2024-03-28T18:08:28.504 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:28.503+0000 7fc38599d740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d0ac525e70 con 0x55d0ac5198b0 2024-03-28T18:08:28.505 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:28.504+0000 7fc38599d740 10 client.0 _ll_register_callbacks cb 0x55d0ac43de80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-28T18:08:28.505 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:28.504+0000 7fc378c00640 1 --2- 172.21.15.22:0/1336328280 >> [v2:172.21.15.22:6800/1357365082,v1:172.21.15.22:6801/1357365082] conn(0x7fc3600a2a20 0x7fc3600a4ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:08:28.505 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:28.504+0000 7fc37be00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d0ac51a260 0x55d0ac51fa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:08:28.505 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:28.504+0000 7fc37b400640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55d0ac522de0 0x55d0ac51ffa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:08:28.505 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:28.504+0000 7fc37be00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d0ac51a260 0x55d0ac51fa40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:37920/0 (socket says 172.21.15.22:37920) 2024-03-28T18:08:28.505 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:28.504+0000 7fc37be00640 1 -- 172.21.15.22:0/1195634197 learned_addr learned my addr 172.21.15.22:0/1195634197 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-28T18:08:28.505 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:28.504+0000 7fc380c00640 1 --2- 172.21.15.22:0/1195634197 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d0ac5198b0 0x55d0ac5226b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:08:28.505 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:28.504+0000 7fc380c00640 1 --2- 172.21.15.22:0/1336328280 >> [v2:172.21.15.22:6800/1357365082,v1:172.21.15.22:6801/1357365082] conn(0x7fc3600a2a20 0x7fc3600a4ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:08:28.506 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:28.505+0000 7fc37be00640 1 -- 172.21.15.22:0/1195634197 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d0ac5198b0 msgr2=0x55d0ac5226b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T18:08:28.506 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:28.505+0000 7fc37be00640 1 --2- 172.21.15.22:0/1195634197 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d0ac5198b0 0x55d0ac5226b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:08:28.506 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:28.505+0000 7fc37be00640 1 -- 172.21.15.22:0/1195634197 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55d0ac522de0 msgr2=0x55d0ac51ffa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T18:08:28.506 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:28.505+0000 7fc37be00640 1 --2- 172.21.15.22:0/1195634197 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55d0ac522de0 0x55d0ac51ffa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:08:28.506 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:28.505+0000 7fc37be00640 1 -- 172.21.15.22:0/1195634197 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55d0ac1913f0 con 0x55d0ac51a260 2024-03-28T18:08:28.506 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:28.505+0000 7fc378c00640 1 -- 172.21.15.22:0/1336328280 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 8411+0+0 (secure 0 0 0) 0x7fc3680cf3b0 con 0x7fc37c074390 2024-03-28T18:08:28.506 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:28.505+0000 7fc37be00640 1 --2- 172.21.15.22:0/1195634197 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d0ac51a260 0x55d0ac51fa40 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7fc36c0042f0 tx=0x7fc36c00bf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=11347f1fd63a728d server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:08:28.506 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:28.505+0000 7fc365600640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-03-28T18:08:28.506 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:28.506+0000 7fc365600640 1 -- 172.21.15.22:0/1195634197 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc36c005220 con 0x55d0ac51a260 2024-03-28T18:08:28.507 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:28.506+0000 7fc365600640 1 -- 172.21.15.22:0/1195634197 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 8411+0+0 (secure 0 0 0) 0x7fc36c0053c0 con 0x55d0ac51a260 2024-03-28T18:08:28.507 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:28.506+0000 7fc380c00640 1 --2- 172.21.15.22:0/1336328280 >> [v2:172.21.15.22:6800/1357365082,v1:172.21.15.22:6801/1357365082] conn(0x7fc3600a2a20 0x7fc3600a4ee0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fc374032580 tx=0x7fc37402c190 comp rx=0 tx=0).ready entity=mgr.15522 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:08:28.507 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:28.506+0000 7fc38599d740 1 -- 172.21.15.22:0/1195634197 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55d0ac48a800 con 0x55d0ac51a260 2024-03-28T18:08:28.507 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:28.506+0000 7fc365600640 1 -- 172.21.15.22:0/1195634197 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc36c0055c0 con 0x55d0ac51a260 2024-03-28T18:08:28.507 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:28.507+0000 7fc38599d740 10 client.15819 fetch_fsmap learned FSMap version 27 2024-03-28T18:08:28.507 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:28.507+0000 7fc38599d740 1 -- 172.21.15.22:0/1195634197 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55d0ac25b7d0 con 0x55d0ac51a260 2024-03-28T18:08:28.508 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:28.507+0000 7fc365600640 1 -- 172.21.15.22:0/1195634197 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x55d0ac48a800 con 0x55d0ac51a260 2024-03-28T18:08:28.508 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:28.507+0000 7fc365600640 1 -- 172.21.15.22:0/1195634197 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 27) v1 ==== 7934+0+0 (secure 0 0 0) 0x7fc36c011ad0 con 0x55d0ac51a260 2024-03-28T18:08:28.508 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:28.508+0000 7fc38599d740 10 client.15819 fetch_fsmap finished waiting for FSMap version 27 2024-03-28T18:08:28.508 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:28.508+0000 7fc38599d740 10 client.15819 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.k 2024-03-28T18:08:28.508 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:28.508+0000 7fc38599d740 20 client.15819 populate_metadata read hostname 'smithi022' 2024-03-28T18:08:28.508 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:28.508+0000 7fc38599d740 1 --2- 172.21.15.22:0/1195634197 >> [v2:172.21.15.79:6832/856896997,v1:172.21.15.79:6833/856896997] conn(0x55d0ac53b4f0 0x55d0ac53d8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:08:28.508 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:28.508+0000 7fc38599d740 4 client.15819 mds_command: new command op to 34442 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-28T18:08:28.508 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:28.508+0000 7fc38599d740 1 -- 172.21.15.22:0/1195634197 --> [v2:172.21.15.79:6832/856896997,v1:172.21.15.79:6833/856896997] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55d0ac484b50 con 0x55d0ac53b4f0 2024-03-28T18:08:28.508 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:28.508+0000 7fc37b400640 1 --2- 172.21.15.22:0/1195634197 >> [v2:172.21.15.79:6832/856896997,v1:172.21.15.79:6833/856896997] conn(0x55d0ac53b4f0 0x55d0ac53d8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:08:28.509 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:28.509+0000 7fc37b400640 1 --2- 172.21.15.22:0/1195634197 >> [v2:172.21.15.79:6832/856896997,v1:172.21.15.79:6833/856896997] conn(0x55d0ac53b4f0 0x55d0ac53d8e0 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=945ec5e2e2aac05a server_cookie=e2d574eccc8fa08 in_seq=0 out_seq=0 2024-03-28T18:08:28.509 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:28.509+0000 7fc365600640 10 client.15819 ms_handle_connect on v2:172.21.15.79:6832/856896997 2024-03-28T18:08:28.510 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:28.510+0000 7fc365600640 1 -- 172.21.15.22:0/1195634197 <== mds.3 v2:172.21.15.79:6832/856896997 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x55d0ac484b50 con 0x55d0ac53b4f0 2024-03-28T18:08:28.510 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:28.510+0000 7fc365600640 10 client.15819 handle_command_reply: tid=0 2024-03-28T18:08:28.510 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:28.511+0000 7fc38599d740 1 client.15819 shutdown 2024-03-28T18:08:28.510 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:28.511+0000 7fc38599d740 1 -- 172.21.15.22:0/1195634197 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d0ac51a260 msgr2=0x55d0ac51fa40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:08:28.511 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:28.511+0000 7fc38599d740 1 --2- 172.21.15.22:0/1195634197 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d0ac51a260 0x55d0ac51fa40 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7fc36c0042f0 tx=0x7fc36c00bf90 comp rx=0 tx=0).stop 2024-03-28T18:08:28.511 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:28.511+0000 7fc38599d740 1 -- 172.21.15.22:0/1195634197 shutdown_connections 2024-03-28T18:08:28.511 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:28.511+0000 7fc38599d740 1 --2- 172.21.15.22:0/1195634197 >> [v2:172.21.15.79:6832/856896997,v1:172.21.15.79:6833/856896997] conn(0x55d0ac53b4f0 0x55d0ac53d8e0 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:08:28.511 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:28.511+0000 7fc365600640 0 client.15819 ms_handle_reset on v2:172.21.15.79:6832/856896997 2024-03-28T18:08:28.511 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:28.511+0000 7fc38599d740 1 --2- 172.21.15.22:0/1195634197 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55d0ac522de0 0x55d0ac51ffa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:08:28.511 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:28.511+0000 7fc38599d740 1 --2- 172.21.15.22:0/1195634197 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d0ac51a260 0x55d0ac51fa40 unknown :-1 s=CLOSED pgs=526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:08:28.511 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:28.511+0000 7fc38599d740 1 --2- 172.21.15.22:0/1195634197 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d0ac5198b0 0x55d0ac5226b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:08:28.511 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:28.511+0000 7fc38599d740 1 -- 172.21.15.22:0/1195634197 >> 172.21.15.22:0/1195634197 conn(0x55d0ac513f80 msgr2=0x55d0ac515cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:08:28.512 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:28.511+0000 7fc38599d740 1 -- 172.21.15.22:0/1195634197 shutdown_connections 2024-03-28T18:08:28.512 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:28.511+0000 7fc38599d740 1 -- 172.21.15.22:0/1195634197 wait complete. 2024-03-28T18:08:28.512 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:28.511+0000 7fc38599d740 20 client.15819 trim_cache size 0 max 16384 2024-03-28T18:08:28.526 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:28.526+0000 7fc38599d740 1 Processor -- start 2024-03-28T18:08:28.526 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:28.526+0000 7fc38599d740 1 -- start start 2024-03-28T18:08:28.526 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:28.526+0000 7fc38599d740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55d0ac5198b0 0x55d0ac522170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:08:28.526 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:28.526+0000 7fc38599d740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d0ac51a260 0x55d0ac5226b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:08:28.526 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:28.526+0000 7fc38599d740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d0ac522de0 0x55d0ac51cca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:08:28.526 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:28.526+0000 7fc38599d740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc37c1aab70 con 0x55d0ac522de0 2024-03-28T18:08:28.526 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:28.526+0000 7fc38599d740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc37c1ac4f0 con 0x55d0ac5198b0 2024-03-28T18:08:28.526 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:28.526+0000 7fc38599d740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc37c1ab1d0 con 0x55d0ac51a260 2024-03-28T18:08:28.528 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:28.527+0000 7fc37be00640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55d0ac5198b0 0x55d0ac522170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:08:28.528 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:28.527+0000 7fc380c00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d0ac522de0 0x55d0ac51cca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:08:28.528 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:28.527+0000 7fc380c00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d0ac522de0 0x55d0ac51cca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:37930/0 (socket says 172.21.15.22:37930) 2024-03-28T18:08:28.528 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:28.527+0000 7fc37be00640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55d0ac5198b0 0x55d0ac522170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.22:47212/0 (socket says 172.21.15.22:47212) 2024-03-28T18:08:28.528 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:28.527+0000 7fc380c00640 1 -- 172.21.15.22:0/3975286297 learned_addr learned my addr 172.21.15.22:0/3975286297 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-28T18:08:28.528 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:28.527+0000 7fc37b400640 1 --2- 172.21.15.22:0/3975286297 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d0ac51a260 0x55d0ac5226b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:08:28.528 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:28.527+0000 7fc380c00640 1 -- 172.21.15.22:0/3975286297 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d0ac51a260 msgr2=0x55d0ac5226b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T18:08:28.528 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:28.527+0000 7fc380c00640 1 --2- 172.21.15.22:0/3975286297 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d0ac51a260 0x55d0ac5226b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:08:28.528 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:28.527+0000 7fc380c00640 1 -- 172.21.15.22:0/3975286297 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55d0ac5198b0 msgr2=0x55d0ac522170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T18:08:28.528 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:28.527+0000 7fc380c00640 1 --2- 172.21.15.22:0/3975286297 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55d0ac5198b0 0x55d0ac522170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:08:28.528 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:28.527+0000 7fc380c00640 1 -- 172.21.15.22:0/3975286297 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d0ac53dec0 con 0x55d0ac522de0 2024-03-28T18:08:28.528 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:28.528+0000 7fc380c00640 1 --2- 172.21.15.22:0/3975286297 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d0ac522de0 0x55d0ac51cca0 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7fc3740325b0 tx=0x7fc37404a070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fc0bc4c6e11c8b32 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:08:28.529 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:28.528+0000 7fc367e00640 1 -- 172.21.15.22:0/3975286297 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc374058070 con 0x55d0ac522de0 2024-03-28T18:08:28.529 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:28.528+0000 7fc367e00640 1 -- 172.21.15.22:0/3975286297 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7fc374032d00 con 0x55d0ac522de0 2024-03-28T18:08:28.529 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:28.528+0000 7fc367e00640 1 -- 172.21.15.22:0/3975286297 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc37403f990 con 0x55d0ac522de0 2024-03-28T18:08:28.530 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:28.529+0000 7fc38599d740 1 -- 172.21.15.22:0/3975286297 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d0ac522de0 msgr2=0x55d0ac51cca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:08:28.530 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:28.529+0000 7fc38599d740 1 --2- 172.21.15.22:0/3975286297 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d0ac522de0 0x55d0ac51cca0 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7fc3740325b0 tx=0x7fc37404a070 comp rx=0 tx=0).stop 2024-03-28T18:08:28.530 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:28.529+0000 7fc38599d740 1 -- 172.21.15.22:0/3975286297 shutdown_connections 2024-03-28T18:08:28.530 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:28.529+0000 7fc38599d740 1 --2- 172.21.15.22:0/3975286297 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d0ac522de0 0x55d0ac51cca0 unknown :-1 s=CLOSED pgs=527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:08:28.530 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:28.529+0000 7fc38599d740 1 --2- 172.21.15.22:0/3975286297 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d0ac51a260 0x55d0ac5226b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:08:28.530 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:28.529+0000 7fc38599d740 1 --2- 172.21.15.22:0/3975286297 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55d0ac5198b0 0x55d0ac522170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:08:28.530 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:28.529+0000 7fc38599d740 1 -- 172.21.15.22:0/3975286297 >> 172.21.15.22:0/3975286297 conn(0x55d0ac513f80 msgr2=0x55d0ac515b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:08:28.530 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:28.529+0000 7fc38599d740 1 -- 172.21.15.22:0/3975286297 shutdown_connections 2024-03-28T18:08:28.530 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:28.529+0000 7fc38599d740 1 -- 172.21.15.22:0/3975286297 wait complete. 2024-03-28T18:08:28.530 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:28.530+0000 7fc38599d740 1 -- start start 2024-03-28T18:08:28.530 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:28.530+0000 7fc38599d740 1 Processor -- start 2024-03-28T18:08:28.532 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:28.531+0000 7fc38599d740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55d0ac5198b0 0x55d0ac52ab80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:08:28.532 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:28.531+0000 7fc38599d740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d0ac51a260 0x55d0ac526f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:08:28.532 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:28.531+0000 7fc38599d740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d0ac522de0 0x55d0ac527440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:08:28.532 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:28.531+0000 7fc38599d740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc37c1abe90 con 0x55d0ac522de0 2024-03-28T18:08:28.532 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:28.531+0000 7fc38599d740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc37c1ab830 con 0x55d0ac5198b0 2024-03-28T18:08:28.532 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:28.531+0000 7fc38599d740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc37c1a8d30 con 0x55d0ac51a260 2024-03-28T18:08:28.532 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:28.531+0000 7fc38599d740 10 client.0 _ll_register_callbacks cb 0x7fc36c005c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-28T18:08:28.532 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:28.531+0000 7fc380c00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d0ac522de0 0x55d0ac527440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:08:28.532 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:28.531+0000 7fc380c00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d0ac522de0 0x55d0ac527440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:37940/0 (socket says 172.21.15.22:37940) 2024-03-28T18:08:28.533 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:28.531+0000 7fc380c00640 1 -- 172.21.15.22:0/2041241206 learned_addr learned my addr 172.21.15.22:0/2041241206 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-28T18:08:28.533 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:28.531+0000 7fc37be00640 1 --2- 172.21.15.22:0/2041241206 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55d0ac5198b0 0x55d0ac52ab80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:08:28.533 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:28.531+0000 7fc37b400640 1 --2- 172.21.15.22:0/2041241206 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d0ac51a260 0x55d0ac526f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:08:28.533 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:28.532+0000 7fc37be00640 1 -- 172.21.15.22:0/2041241206 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d0ac51a260 msgr2=0x55d0ac526f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T18:08:28.533 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:28.532+0000 7fc37be00640 1 --2- 172.21.15.22:0/2041241206 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d0ac51a260 0x55d0ac526f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:08:28.533 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:28.532+0000 7fc37be00640 1 -- 172.21.15.22:0/2041241206 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d0ac522de0 msgr2=0x55d0ac527440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T18:08:28.533 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:28.532+0000 7fc37be00640 1 --2- 172.21.15.22:0/2041241206 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d0ac522de0 0x55d0ac527440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:08:28.533 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:28.532+0000 7fc37be00640 1 -- 172.21.15.22:0/2041241206 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55d0ac51b670 con 0x55d0ac5198b0 2024-03-28T18:08:28.533 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:28.532+0000 7fc37be00640 1 --2- 172.21.15.22:0/2041241206 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55d0ac5198b0 0x55d0ac52ab80 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fc36c011e70 tx=0x7fc36c023b70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=469db26c9be467e5 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:08:28.533 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:28.532+0000 7fc365600640 10 client.0 ms_handle_connect on v2:172.21.15.79:3300/0 2024-03-28T18:08:28.535 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:28.533+0000 7fc365600640 1 -- 172.21.15.22:0/2041241206 <== mon.2 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc36c028040 con 0x55d0ac5198b0 2024-03-28T18:08:28.535 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:28.533+0000 7fc365600640 1 -- 172.21.15.22:0/2041241206 <== mon.2 v2:172.21.15.79:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 8411+0+0 (secure 0 0 0) 0x7fc36c011650 con 0x55d0ac5198b0 2024-03-28T18:08:28.535 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:28.533+0000 7fc38599d740 1 -- 172.21.15.22:0/2041241206 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fc374032d00 con 0x55d0ac5198b0 2024-03-28T18:08:28.535 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:28.533+0000 7fc365600640 1 -- 172.21.15.22:0/2041241206 <== mon.2 v2:172.21.15.79:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc36c03b050 con 0x55d0ac5198b0 2024-03-28T18:08:28.535 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:28.533+0000 7fc38599d740 10 client.35663 fetch_fsmap learned FSMap version 27 2024-03-28T18:08:28.535 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:28.533+0000 7fc38599d740 1 -- 172.21.15.22:0/2041241206 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55d0ac5977c0 con 0x55d0ac5198b0 2024-03-28T18:08:28.535 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:28.534+0000 7fc365600640 1 -- 172.21.15.22:0/2041241206 <== mon.2 v2:172.21.15.79:3300/0 4 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7fc374032d00 con 0x55d0ac5198b0 2024-03-28T18:08:28.535 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:28.534+0000 7fc365600640 1 -- 172.21.15.22:0/2041241206 <== mon.2 v2:172.21.15.79:3300/0 5 ==== fsmap(e 27) v1 ==== 7934+0+0 (secure 0 0 0) 0x7fc36c019920 con 0x55d0ac5198b0 2024-03-28T18:08:28.535 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:28.534+0000 7fc38599d740 10 client.35663 fetch_fsmap finished waiting for FSMap version 27 2024-03-28T18:08:28.535 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:28.534+0000 7fc38599d740 10 client.35663 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.k 2024-03-28T18:08:28.535 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:28.534+0000 7fc38599d740 20 client.35663 populate_metadata read hostname 'smithi022' 2024-03-28T18:08:28.535 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:28.534+0000 7fc38599d740 1 --2- 172.21.15.22:0/2041241206 >> [v2:172.21.15.79:6832/856896997,v1:172.21.15.79:6833/856896997] conn(0x55d0ac597e30 0x55d0ac59a250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:08:28.535 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:28.534+0000 7fc38599d740 4 client.35663 mds_command: new command op to 34442 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-28T18:08:28.536 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:28.534+0000 7fc38599d740 1 -- 172.21.15.22:0/2041241206 --> [v2:172.21.15.79:6832/856896997,v1:172.21.15.79:6833/856896997] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55d0ac402500 con 0x55d0ac597e30 2024-03-28T18:08:28.536 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:28.535+0000 7fc37b400640 1 --2- 172.21.15.22:0/2041241206 >> [v2:172.21.15.79:6832/856896997,v1:172.21.15.79:6833/856896997] conn(0x55d0ac597e30 0x55d0ac59a250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:08:28.536 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:28.535+0000 7fc37b400640 1 --2- 172.21.15.22:0/2041241206 >> [v2:172.21.15.79:6832/856896997,v1:172.21.15.79:6833/856896997] conn(0x55d0ac597e30 0x55d0ac59a250 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=a8b82c7a04020385 server_cookie=7fa345d371ef34ce in_seq=0 out_seq=0 2024-03-28T18:08:28.536 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:28.535+0000 7fc365600640 10 client.35663 ms_handle_connect on v2:172.21.15.79:6832/856896997 2024-03-28T18:08:28.537 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:28.536+0000 7fc365600640 1 -- 172.21.15.22:0/2041241206 <== mds.3 v2:172.21.15.79:6832/856896997 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55d0ac402500 con 0x55d0ac597e30 2024-03-28T18:08:28.537 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:28.536+0000 7fc365600640 10 client.35663 handle_command_reply: tid=0 2024-03-28T18:08:28.537 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:28.536+0000 7fc38599d740 1 client.35663 shutdown 2024-03-28T18:08:28.537 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:28.536+0000 7fc38599d740 1 -- 172.21.15.22:0/2041241206 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55d0ac5198b0 msgr2=0x55d0ac52ab80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:08:28.537 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:28.536+0000 7fc38599d740 1 --2- 172.21.15.22:0/2041241206 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55d0ac5198b0 0x55d0ac52ab80 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fc36c011e70 tx=0x7fc36c023b70 comp rx=0 tx=0).stop 2024-03-28T18:08:28.537 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:28.536+0000 7fc38599d740 1 -- 172.21.15.22:0/2041241206 shutdown_connections 2024-03-28T18:08:28.537 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:28.536+0000 7fc38599d740 1 --2- 172.21.15.22:0/2041241206 >> [v2:172.21.15.79:6832/856896997,v1:172.21.15.79:6833/856896997] conn(0x55d0ac597e30 0x55d0ac59a250 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:08:28.537 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:28.536+0000 7fc38599d740 1 --2- 172.21.15.22:0/2041241206 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d0ac522de0 0x55d0ac527440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:08:28.537 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:28.536+0000 7fc38599d740 1 --2- 172.21.15.22:0/2041241206 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d0ac51a260 0x55d0ac526f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:08:28.537 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:28.536+0000 7fc365600640 0 client.35663 ms_handle_reset on v2:172.21.15.79:6832/856896997 2024-03-28T18:08:28.537 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:28.536+0000 7fc38599d740 1 --2- 172.21.15.22:0/2041241206 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55d0ac5198b0 0x55d0ac52ab80 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:08:28.538 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:28.536+0000 7fc38599d740 1 -- 172.21.15.22:0/2041241206 >> 172.21.15.22:0/2041241206 conn(0x55d0ac513f80 msgr2=0x55d0ac515260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:08:28.538 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:28.536+0000 7fc38599d740 1 -- 172.21.15.22:0/2041241206 shutdown_connections 2024-03-28T18:08:28.538 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:28.536+0000 7fc38599d740 1 -- 172.21.15.22:0/2041241206 wait complete. 2024-03-28T18:08:28.538 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:28.536+0000 7fc38599d740 20 client.35663 trim_cache size 0 max 16384 2024-03-28T18:08:28.538 INFO:teuthology.orchestra.run.smithi022.stdout:[] 2024-03-28T18:08:28.539 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:28.537+0000 7fc381600640 1 -- 172.21.15.22:0/1336328280 >> [v2:172.21.15.22:6800/1357365082,v1:172.21.15.22:6801/1357365082] conn(0x7fc3600a2a20 msgr2=0x7fc3600a4ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:08:28.539 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:28.537+0000 7fc381600640 1 --2- 172.21.15.22:0/1336328280 >> [v2:172.21.15.22:6800/1357365082,v1:172.21.15.22:6801/1357365082] conn(0x7fc3600a2a20 0x7fc3600a4ee0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fc374032580 tx=0x7fc37402c190 comp rx=0 tx=0).stop 2024-03-28T18:08:28.539 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:28.537+0000 7fc381600640 1 -- 172.21.15.22:0/1336328280 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc37c074390 msgr2=0x7fc37c19a4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:08:28.539 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:28.537+0000 7fc381600640 1 --2- 172.21.15.22:0/1336328280 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc37c074390 0x7fc37c19a4e0 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7fc36800e9f0 tx=0x7fc36800eeb0 comp rx=0 tx=0).stop 2024-03-28T18:08:28.539 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:28.537+0000 7fc381600640 1 -- 172.21.15.22:0/1336328280 shutdown_connections 2024-03-28T18:08:28.539 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:28.537+0000 7fc381600640 1 --2- 172.21.15.22:0/1336328280 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc37c10ddb0 0x7fc37c19e860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:08:28.539 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:28.537+0000 7fc381600640 1 --2- 172.21.15.22:0/1336328280 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc37c074390 0x7fc37c19a4e0 unknown :-1 s=CLOSED pgs=525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:08:28.539 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:28.537+0000 7fc381600640 1 --2- 172.21.15.22:0/1336328280 >> [v2:172.21.15.22:6800/1357365082,v1:172.21.15.22:6801/1357365082] conn(0x7fc3600a2a20 0x7fc3600a4ee0 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:08:28.539 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:28.537+0000 7fc381600640 1 --2- 172.21.15.22:0/1336328280 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc37c073a70 0x7fc37c199fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:08:28.540 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:28.537+0000 7fc381600640 1 -- 172.21.15.22:0/1336328280 >> 172.21.15.22:0/1336328280 conn(0x7fc37c0fd040 msgr2=0x7fc37c0fdd70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:08:28.540 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:28.538+0000 7fc381600640 1 -- 172.21.15.22:0/1336328280 shutdown_connections 2024-03-28T18:08:28.540 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:28.538+0000 7fc381600640 1 -- 172.21.15.22:0/1336328280 wait complete. 2024-03-28T18:08:28.560 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-28T18:08:28.561 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-03-28T18:08:28.771 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:28.771+0000 7f80bac00640 1 -- 172.21.15.22:0/3231531976 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f80b4100c80 msgr2=0x7f80b410d940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:08:28.771 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:28.771+0000 7f80bac00640 1 --2- 172.21.15.22:0/3231531976 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f80b4100c80 0x7f80b410d940 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f80a0009900 tx=0x7f80a002bc30 comp rx=0 tx=0).stop 2024-03-28T18:08:28.771 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:28.771+0000 7f80bac00640 1 -- 172.21.15.22:0/3231531976 shutdown_connections 2024-03-28T18:08:28.771 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:28.771+0000 7f80bac00640 1 --2- 172.21.15.22:0/3231531976 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f80b410de80 0x7f80b4110270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:08:28.771 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:28.771+0000 7f80bac00640 1 --2- 172.21.15.22:0/3231531976 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f80b4100c80 0x7f80b410d940 unknown :-1 s=CLOSED pgs=528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:08:28.771 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:28.771+0000 7f80bac00640 1 --2- 172.21.15.22:0/3231531976 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f80b41002d0 0x7f80b41006b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:08:28.771 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:28.771+0000 7f80bac00640 1 -- 172.21.15.22:0/3231531976 >> 172.21.15.22:0/3231531976 conn(0x7f80b40fbef0 msgr2=0x7f80b40fe310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:08:28.771 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:28.771+0000 7f80bac00640 1 -- 172.21.15.22:0/3231531976 shutdown_connections 2024-03-28T18:08:28.772 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:28.771+0000 7f80bac00640 1 -- 172.21.15.22:0/3231531976 wait complete. 2024-03-28T18:08:28.773 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:28.772+0000 7f80bac00640 1 Processor -- start 2024-03-28T18:08:28.773 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:28.772+0000 7f80bac00640 1 -- start start 2024-03-28T18:08:28.773 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:28.773+0000 7f80bac00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f80b41002d0 0x7f80b419a120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:08:28.773 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:28.773+0000 7f80bac00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f80b4100c80 0x7f80b419a660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:08:28.773 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:28.773+0000 7f80bac00640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f80b410de80 0x7f80b419e990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:08:28.773 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:28.773+0000 7f80bac00640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80b4113b60 con 0x7f80b4100c80 2024-03-28T18:08:28.773 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:28.773+0000 7f80bac00640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80b41139e0 con 0x7f80b410de80 2024-03-28T18:08:28.773 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:28.773+0000 7f80bac00640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80b4113ce0 con 0x7f80b41002d0 2024-03-28T18:08:28.773 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:28.773+0000 7f80b8e00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f80b4100c80 0x7f80b419a660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:08:28.773 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:28.773+0000 7f80b8e00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f80b4100c80 0x7f80b419a660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:37956/0 (socket says 172.21.15.22:37956) 2024-03-28T18:08:28.773 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:28.773+0000 7f80b8e00640 1 -- 172.21.15.22:0/3430334210 learned_addr learned my addr 172.21.15.22:0/3430334210 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-28T18:08:28.774 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:28.773+0000 7f80b9800640 1 --2- 172.21.15.22:0/3430334210 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f80b41002d0 0x7f80b419a120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:08:28.774 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:28.773+0000 7f80ba200640 1 --2- 172.21.15.22:0/3430334210 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f80b410de80 0x7f80b419e990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:08:28.774 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:28.774+0000 7f80b8e00640 1 -- 172.21.15.22:0/3430334210 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f80b41002d0 msgr2=0x7f80b419a120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:08:28.775 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:28.774+0000 7f80b8e00640 1 --2- 172.21.15.22:0/3430334210 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f80b41002d0 0x7f80b419a120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:08:28.775 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:28.774+0000 7f80b8e00640 1 -- 172.21.15.22:0/3430334210 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f80b410de80 msgr2=0x7f80b419e990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:08:28.775 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:28.774+0000 7f80b8e00640 1 --2- 172.21.15.22:0/3430334210 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f80b410de80 0x7f80b419e990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:08:28.775 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:28.774+0000 7f80b8e00640 1 -- 172.21.15.22:0/3430334210 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f80b419f0c0 con 0x7f80b4100c80 2024-03-28T18:08:28.775 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:28.774+0000 7f80b8e00640 1 --2- 172.21.15.22:0/3430334210 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f80b4100c80 0x7f80b419a660 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7f80a0000c00 tx=0x7f80a00045e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:08:28.775 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:28.774+0000 7f80b2000640 1 -- 172.21.15.22:0/3430334210 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80a0043020 con 0x7f80b4100c80 2024-03-28T18:08:28.775 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:28.775+0000 7f80b2000640 1 -- 172.21.15.22:0/3430334210 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f80a002caa0 con 0x7f80b4100c80 2024-03-28T18:08:28.775 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:28.775+0000 7f80bac00640 1 -- 172.21.15.22:0/3430334210 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f80b419f350 con 0x7f80b4100c80 2024-03-28T18:08:28.775 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:28.775+0000 7f80b2000640 1 -- 172.21.15.22:0/3430334210 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80a0003ac0 con 0x7f80b4100c80 2024-03-28T18:08:28.775 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:28.775+0000 7f80bac00640 1 -- 172.21.15.22:0/3430334210 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f80b41a6c40 con 0x7f80b4100c80 2024-03-28T18:08:28.779 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:28.780+0000 7f80bef81740 1 Processor -- start 2024-03-28T18:08:28.779 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:28.780+0000 7f80bef81740 1 -- start start 2024-03-28T18:08:28.780 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:28.780+0000 7f80bef81740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f9b212d8b0 0x55f9b212dc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:08:28.780 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:28.780+0000 7f80bef81740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f9b212e260 0x55f9b2136810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:08:28.780 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:28.780+0000 7f80bef81740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55f9b2136de0 0x55f9b2139230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:08:28.780 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:28.781+0000 7f80bef81740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f9b20912c0 con 0x55f9b212e260 2024-03-28T18:08:28.780 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:28.781+0000 7f80bef81740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f9b2039c90 con 0x55f9b2136de0 2024-03-28T18:08:28.780 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:28.781+0000 7f80bef81740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f9b208d360 con 0x55f9b212d8b0 2024-03-28T18:08:28.781 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:28.781+0000 7f80b9800640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f9b212e260 0x55f9b2136810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:08:28.782 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:28.781+0000 7f80b9800640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f9b212e260 0x55f9b2136810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:37970/0 (socket says 172.21.15.22:37970) 2024-03-28T18:08:28.782 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:28.781+0000 7f80b9800640 1 -- 172.21.15.22:0/2254013150 learned_addr learned my addr 172.21.15.22:0/2254013150 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-28T18:08:28.782 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:28.781+0000 7f80ba200640 1 --2- 172.21.15.22:0/2254013150 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f9b212d8b0 0x55f9b212dc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:08:28.782 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:28.781+0000 7f80b2000640 1 -- 172.21.15.22:0/3430334210 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 25) v1 ==== 137523+0+0 (secure 0 0 0) 0x7f80a0046020 con 0x7f80b4100c80 2024-03-28T18:08:28.782 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:28.782+0000 7f80b9800640 1 -- 172.21.15.22:0/2254013150 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f9b212d8b0 msgr2=0x55f9b212dc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T18:08:28.782 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:28.782+0000 7f80b9800640 1 --2- 172.21.15.22:0/2254013150 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f9b212d8b0 0x55f9b212dc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:08:28.782 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:28.782+0000 7f80b9800640 1 -- 172.21.15.22:0/2254013150 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55f9b2136de0 msgr2=0x55f9b2139230 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-03-28T18:08:28.782 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:28.782+0000 7f80b9800640 1 --2- 172.21.15.22:0/2254013150 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55f9b2136de0 0x55f9b2139230 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:08:28.782 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:28.782+0000 7f80b9800640 1 -- 172.21.15.22:0/2254013150 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f9b205e080 con 0x55f9b212e260 2024-03-28T18:08:28.783 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:28.782+0000 7f80b9800640 1 --2- 172.21.15.22:0/2254013150 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f9b212e260 0x55f9b2136810 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7f80a8009cf0 tx=0x7f80a8011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e3f25e87ea72659e server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:08:28.783 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:28.783+0000 7f80a6a00640 1 -- 172.21.15.22:0/2254013150 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80a8004230 con 0x55f9b212e260 2024-03-28T18:08:28.783 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:28.783+0000 7f80a6a00640 1 -- 172.21.15.22:0/2254013150 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f80a80043d0 con 0x55f9b212e260 2024-03-28T18:08:28.783 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:28.783+0000 7f80a6a00640 1 -- 172.21.15.22:0/2254013150 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80a8019850 con 0x55f9b212e260 2024-03-28T18:08:28.783 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:28.784+0000 7f80b2000640 1 --2- 172.21.15.22:0/3430334210 >> [v2:172.21.15.22:6800/1357365082,v1:172.21.15.22:6801/1357365082] conn(0x7f809c0a2b10 0x7f809c0a4fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:08:28.784 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:28.784+0000 7f80b2000640 1 -- 172.21.15.22:0/3430334210 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 8411+0+0 (secure 0 0 0) 0x7f80a00f0f60 con 0x7f80b4100c80 2024-03-28T18:08:28.784 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:28.784+0000 7f80bef81740 1 -- 172.21.15.22:0/2254013150 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f9b212e260 msgr2=0x55f9b2136810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:08:28.784 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:28.784+0000 7f80bef81740 1 --2- 172.21.15.22:0/2254013150 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f9b212e260 0x55f9b2136810 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7f80a8009cf0 tx=0x7f80a8011810 comp rx=0 tx=0).stop 2024-03-28T18:08:28.785 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:28.784+0000 7f80ba200640 1 --2- 172.21.15.22:0/3430334210 >> [v2:172.21.15.22:6800/1357365082,v1:172.21.15.22:6801/1357365082] conn(0x7f809c0a2b10 0x7f809c0a4fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:08:28.785 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:28.785+0000 7f80bef81740 1 -- 172.21.15.22:0/2254013150 shutdown_connections 2024-03-28T18:08:28.785 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:28.785+0000 7f80bef81740 1 --2- 172.21.15.22:0/2254013150 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55f9b2136de0 0x55f9b2139230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:08:28.785 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:28.785+0000 7f80bef81740 1 --2- 172.21.15.22:0/2254013150 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f9b212e260 0x55f9b2136810 unknown :-1 s=CLOSED pgs=530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:08:28.785 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:28.785+0000 7f80bef81740 1 --2- 172.21.15.22:0/2254013150 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f9b212d8b0 0x55f9b212dc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:08:28.785 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:28.785+0000 7f80bef81740 1 -- 172.21.15.22:0/2254013150 >> 172.21.15.22:0/2254013150 conn(0x55f9b2127f80 msgr2=0x55f9b2128360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:08:28.785 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:28.785+0000 7f80bef81740 1 -- 172.21.15.22:0/2254013150 shutdown_connections 2024-03-28T18:08:28.785 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:28.785+0000 7f80bef81740 1 -- 172.21.15.22:0/2254013150 wait complete. 2024-03-28T18:08:28.785 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:28.785+0000 7f80ba200640 1 --2- 172.21.15.22:0/3430334210 >> [v2:172.21.15.22:6800/1357365082,v1:172.21.15.22:6801/1357365082] conn(0x7f809c0a2b10 0x7f809c0a4fd0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f80b40698e0 tx=0x7f80ac009600 comp rx=0 tx=0).ready entity=mgr.15522 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:08:28.786 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:28.785+0000 7f80bef81740 1 -- start start 2024-03-28T18:08:28.787 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:28.786+0000 7f80bef81740 1 Processor -- start 2024-03-28T18:08:28.787 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:28.786+0000 7f80bef81740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f9b212d8b0 0x55f9b2136670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:08:28.787 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:28.786+0000 7f80bef81740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f9b212e260 0x55f9b2133a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:08:28.787 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:28.786+0000 7f80bef81740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55f9b2136de0 0x55f9b2133f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:08:28.787 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:28.786+0000 7f80bef81740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f9b2139d50 con 0x55f9b212d8b0 2024-03-28T18:08:28.787 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:28.786+0000 7f80bef81740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f9b204d0d0 con 0x55f9b2136de0 2024-03-28T18:08:28.787 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:28.786+0000 7f80bef81740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f9b2139ed0 con 0x55f9b212e260 2024-03-28T18:08:28.787 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:28.787+0000 7f80bef81740 10 client.0 _ll_register_callbacks cb 0x55f9b2051e80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-28T18:08:28.787 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:28.787+0000 7f80b9800640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f9b212d8b0 0x55f9b2136670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:08:28.787 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:28.787+0000 7f80b9800640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f9b212d8b0 0x55f9b2136670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:37980/0 (socket says 172.21.15.22:37980) 2024-03-28T18:08:28.787 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:28.787+0000 7f80b9800640 1 -- 172.21.15.22:0/1893089599 learned_addr learned my addr 172.21.15.22:0/1893089599 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-28T18:08:28.788 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:28.787+0000 7f80b9800640 1 -- 172.21.15.22:0/1893089599 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f9b212e260 msgr2=0x55f9b2133a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T18:08:28.788 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:28.787+0000 7f80b9800640 1 --2- 172.21.15.22:0/1893089599 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f9b212e260 0x55f9b2133a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:08:28.788 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:28.787+0000 7f80ba200640 1 --2- 172.21.15.22:0/1893089599 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55f9b2136de0 0x55f9b2133f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:08:28.788 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:28.787+0000 7f80b9800640 1 -- 172.21.15.22:0/1893089599 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55f9b2136de0 msgr2=0x55f9b2133f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T18:08:28.788 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:28.787+0000 7f80b9800640 1 --2- 172.21.15.22:0/1893089599 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55f9b2136de0 0x55f9b2133f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:08:28.789 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:28.787+0000 7f80b9800640 1 -- 172.21.15.22:0/1893089599 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55f9b1da53f0 con 0x55f9b212d8b0 2024-03-28T18:08:28.789 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:28.787+0000 7f80ba200640 1 --2- 172.21.15.22:0/1893089599 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55f9b2136de0 0x55f9b2133f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T18:08:28.789 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:28.788+0000 7f80b9800640 1 --2- 172.21.15.22:0/1893089599 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f9b212d8b0 0x55f9b2136670 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f80a8009380 tx=0x7f80a8003d90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8bc40e7f3a19889e server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:08:28.789 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:28.788+0000 7f809b400640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-03-28T18:08:28.789 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:28.788+0000 7f809b400640 1 -- 172.21.15.22:0/1893089599 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80a8027070 con 0x55f9b212d8b0 2024-03-28T18:08:28.789 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:28.788+0000 7f809b400640 1 -- 172.21.15.22:0/1893089599 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 8411+0+0 (secure 0 0 0) 0x7f80a8005240 con 0x55f9b212d8b0 2024-03-28T18:08:28.789 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:28.788+0000 7f80bef81740 1 -- 172.21.15.22:0/1893089599 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f80a80043d0 con 0x55f9b212d8b0 2024-03-28T18:08:28.789 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:28.788+0000 7f809b400640 1 -- 172.21.15.22:0/1893089599 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f80a8033050 con 0x55f9b212d8b0 2024-03-28T18:08:28.789 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:28.789+0000 7f80bef81740 10 client.15837 fetch_fsmap learned FSMap version 27 2024-03-28T18:08:28.789 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:28.789+0000 7f80bef81740 1 -- 172.21.15.22:0/1893089599 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55f9b1e6f7d0 con 0x55f9b212d8b0 2024-03-28T18:08:28.790 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:28.789+0000 7f809b400640 1 -- 172.21.15.22:0/1893089599 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f80a80043d0 con 0x55f9b212d8b0 2024-03-28T18:08:28.790 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:28.790+0000 7f809b400640 1 -- 172.21.15.22:0/1893089599 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 27) v1 ==== 7934+0+0 (secure 0 0 0) 0x7f80a8004b90 con 0x55f9b212d8b0 2024-03-28T18:08:28.790 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:28.790+0000 7f80bef81740 10 client.15837 fetch_fsmap finished waiting for FSMap version 27 2024-03-28T18:08:28.790 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:28.790+0000 7f80bef81740 10 client.15837 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.e 2024-03-28T18:08:28.790 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:28.790+0000 7f80bef81740 20 client.15837 populate_metadata read hostname 'smithi022' 2024-03-28T18:08:28.790 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:28.790+0000 7f80bef81740 1 --2- 172.21.15.22:0/1893089599 >> [v2:172.21.15.79:6834/3365773431,v1:172.21.15.79:6835/3365773431] conn(0x55f9b2146b40 0x55f9b2148f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:08:28.790 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:28.790+0000 7f80bef81740 4 client.15837 mds_command: new command op to 34475 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-28T18:08:28.790 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:28.790+0000 7f80bef81740 1 -- 172.21.15.22:0/1893089599 --> [v2:172.21.15.79:6834/3365773431,v1:172.21.15.79:6835/3365773431] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55f9b209e800 con 0x55f9b2146b40 2024-03-28T18:08:28.791 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:28.791+0000 7f80b8e00640 1 --2- 172.21.15.22:0/1893089599 >> [v2:172.21.15.79:6834/3365773431,v1:172.21.15.79:6835/3365773431] conn(0x55f9b2146b40 0x55f9b2148f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:08:28.791 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:28.791+0000 7f80b8e00640 1 --2- 172.21.15.22:0/1893089599 >> [v2:172.21.15.79:6834/3365773431,v1:172.21.15.79:6835/3365773431] conn(0x55f9b2146b40 0x55f9b2148f30 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=62557b8b2aef844a server_cookie=4ed5fe524b0112d0 in_seq=0 out_seq=0 2024-03-28T18:08:28.791 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:28.791+0000 7f809b400640 10 client.15837 ms_handle_connect on v2:172.21.15.79:6834/3365773431 2024-03-28T18:08:28.792 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:28.792+0000 7f809b400640 1 -- 172.21.15.22:0/1893089599 <== mds.2 v2:172.21.15.79:6834/3365773431 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x55f9b209e800 con 0x55f9b2146b40 2024-03-28T18:08:28.792 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:28.792+0000 7f809b400640 10 client.15837 handle_command_reply: tid=0 2024-03-28T18:08:28.792 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:28.792+0000 7f80bef81740 1 client.15837 shutdown 2024-03-28T18:08:28.793 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:28.792+0000 7f80bef81740 1 -- 172.21.15.22:0/1893089599 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f9b212d8b0 msgr2=0x55f9b2136670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:08:28.793 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:28.792+0000 7f80bef81740 1 --2- 172.21.15.22:0/1893089599 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f9b212d8b0 0x55f9b2136670 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f80a8009380 tx=0x7f80a8003d90 comp rx=0 tx=0).stop 2024-03-28T18:08:28.793 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:28.793+0000 7f80bef81740 1 -- 172.21.15.22:0/1893089599 shutdown_connections 2024-03-28T18:08:28.793 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:28.793+0000 7f80bef81740 1 --2- 172.21.15.22:0/1893089599 >> [v2:172.21.15.79:6834/3365773431,v1:172.21.15.79:6835/3365773431] conn(0x55f9b2146b40 0x55f9b2148f30 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:08:28.793 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:28.793+0000 7f80bef81740 1 --2- 172.21.15.22:0/1893089599 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55f9b2136de0 0x55f9b2133f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:08:28.793 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:28.793+0000 7f80bef81740 1 --2- 172.21.15.22:0/1893089599 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f9b212e260 0x55f9b2133a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:08:28.793 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:28.793+0000 7f80bef81740 1 --2- 172.21.15.22:0/1893089599 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f9b212d8b0 0x55f9b2136670 unknown :-1 s=CLOSED pgs=531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:08:28.793 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:28.793+0000 7f80bef81740 1 -- 172.21.15.22:0/1893089599 >> 172.21.15.22:0/1893089599 conn(0x55f9b2127f80 msgr2=0x55f9b2129cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:08:28.793 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:28.793+0000 7f809b400640 0 client.15837 ms_handle_reset on v2:172.21.15.79:6834/3365773431 2024-03-28T18:08:28.793 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:28.793+0000 7f80bef81740 1 -- 172.21.15.22:0/1893089599 shutdown_connections 2024-03-28T18:08:28.793 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:28.793+0000 7f80bef81740 1 -- 172.21.15.22:0/1893089599 wait complete. 2024-03-28T18:08:28.793 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:28.793+0000 7f80bef81740 20 client.15837 trim_cache size 0 max 16384 2024-03-28T18:08:28.806 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:28.806+0000 7f80bef81740 1 Processor -- start 2024-03-28T18:08:28.806 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:28.806+0000 7f80bef81740 1 -- start start 2024-03-28T18:08:28.807 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:28.807+0000 7f80bef81740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f9b212d8b0 0x55f9b21332e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:08:28.807 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:28.807+0000 7f80bef81740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f9b212e260 0x55f9b21268d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:08:28.807 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:28.807+0000 7f80bef81740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55f9b2136de0 0x55f9b2136130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:08:28.807 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:28.807+0000 7f80bef81740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80b41af090 con 0x55f9b212d8b0 2024-03-28T18:08:28.807 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:28.807+0000 7f80bef81740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80b41b0a10 con 0x55f9b2136de0 2024-03-28T18:08:28.807 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:28.807+0000 7f80bef81740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80b41af6f0 con 0x55f9b212e260 2024-03-28T18:08:28.807 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:28.807+0000 7f80b9800640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f9b212d8b0 0x55f9b21332e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:08:28.807 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:28.807+0000 7f80b9800640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f9b212d8b0 0x55f9b21332e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:37988/0 (socket says 172.21.15.22:37988) 2024-03-28T18:08:28.808 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:28.807+0000 7f80b9800640 1 -- 172.21.15.22:0/2377745432 learned_addr learned my addr 172.21.15.22:0/2377745432 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-28T18:08:28.808 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:28.807+0000 7f80b8e00640 1 --2- 172.21.15.22:0/2377745432 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f9b212e260 0x55f9b21268d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:08:28.808 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:28.808+0000 7f80ba200640 1 --2- 172.21.15.22:0/2377745432 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55f9b2136de0 0x55f9b2136130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:08:28.809 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:28.808+0000 7f80b9800640 1 -- 172.21.15.22:0/2377745432 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f9b212e260 msgr2=0x55f9b21268d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T18:08:28.809 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:28.808+0000 7f80b9800640 1 --2- 172.21.15.22:0/2377745432 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f9b212e260 0x55f9b21268d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:08:28.809 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:28.808+0000 7f80b9800640 1 -- 172.21.15.22:0/2377745432 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55f9b2136de0 msgr2=0x55f9b2136130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T18:08:28.809 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:28.808+0000 7f80b9800640 1 --2- 172.21.15.22:0/2377745432 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55f9b2136de0 0x55f9b2136130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:08:28.809 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:28.808+0000 7f80b9800640 1 -- 172.21.15.22:0/2377745432 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f9b212f600 con 0x55f9b212d8b0 2024-03-28T18:08:28.809 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:28.808+0000 7f80b8e00640 1 --2- 172.21.15.22:0/2377745432 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f9b212e260 0x55f9b21268d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-28T18:08:28.809 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:28.808+0000 7f80ba200640 1 --2- 172.21.15.22:0/2377745432 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55f9b2136de0 0x55f9b2136130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-28T18:08:28.809 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:28.809+0000 7f80b9800640 1 --2- 172.21.15.22:0/2377745432 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f9b212d8b0 0x55f9b21332e0 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f80a8001eb0 tx=0x7f80a80096c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a12fd63092716e0c server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:08:28.809 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:28.809+0000 7f80a6000640 1 -- 172.21.15.22:0/2377745432 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80a8030020 con 0x55f9b212d8b0 2024-03-28T18:08:28.809 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:28.809+0000 7f80a6000640 1 -- 172.21.15.22:0/2377745432 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f80a8009b70 con 0x55f9b212d8b0 2024-03-28T18:08:28.809 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:28.809+0000 7f80a6000640 1 -- 172.21.15.22:0/2377745432 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80a8027070 con 0x55f9b212d8b0 2024-03-28T18:08:28.810 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:28.810+0000 7f80bef81740 1 -- 172.21.15.22:0/2377745432 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f9b212d8b0 msgr2=0x55f9b21332e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:08:28.810 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:28.810+0000 7f80bef81740 1 --2- 172.21.15.22:0/2377745432 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f9b212d8b0 0x55f9b21332e0 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f80a8001eb0 tx=0x7f80a80096c0 comp rx=0 tx=0).stop 2024-03-28T18:08:28.810 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:28.810+0000 7f80bef81740 1 -- 172.21.15.22:0/2377745432 shutdown_connections 2024-03-28T18:08:28.810 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:28.810+0000 7f80bef81740 1 --2- 172.21.15.22:0/2377745432 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55f9b2136de0 0x55f9b2136130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:08:28.811 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:28.810+0000 7f80bef81740 1 --2- 172.21.15.22:0/2377745432 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f9b212e260 0x55f9b21268d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:08:28.811 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:28.810+0000 7f80bef81740 1 --2- 172.21.15.22:0/2377745432 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f9b212d8b0 0x55f9b21332e0 unknown :-1 s=CLOSED pgs=532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:08:28.811 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:28.810+0000 7f80bef81740 1 -- 172.21.15.22:0/2377745432 >> 172.21.15.22:0/2377745432 conn(0x55f9b2127f80 msgr2=0x55f9b212eef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:08:28.811 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:28.810+0000 7f80bef81740 1 -- 172.21.15.22:0/2377745432 shutdown_connections 2024-03-28T18:08:28.811 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:28.810+0000 7f80bef81740 1 -- 172.21.15.22:0/2377745432 wait complete. 2024-03-28T18:08:28.811 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:28.811+0000 7f80bef81740 1 -- start start 2024-03-28T18:08:28.811 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:28.811+0000 7f80bef81740 1 Processor -- start 2024-03-28T18:08:28.813 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:28.812+0000 7f80bef81740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f9b212d8b0 0x55f9b213a820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:08:28.813 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:28.812+0000 7f80bef81740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55f9b212e260 0x55f9b213ad60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:08:28.813 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:28.812+0000 7f80bef81740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f9b2136de0 0x55f9b213b2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:08:28.813 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:28.812+0000 7f80bef81740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80b41b03b0 con 0x55f9b212d8b0 2024-03-28T18:08:28.813 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:28.812+0000 7f80bef81740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80b41afd50 con 0x55f9b212e260 2024-03-28T18:08:28.813 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:28.812+0000 7f80bef81740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80b41ad270 con 0x55f9b2136de0 2024-03-28T18:08:28.813 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:28.812+0000 7f80bef81740 10 client.0 _ll_register_callbacks cb 0x7f80ac009a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-28T18:08:28.813 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:28.812+0000 7f80b9800640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f9b212d8b0 0x55f9b213a820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:08:28.813 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:28.812+0000 7f80b9800640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f9b212d8b0 0x55f9b213a820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:37994/0 (socket says 172.21.15.22:37994) 2024-03-28T18:08:28.813 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:28.812+0000 7f80b9800640 1 -- 172.21.15.22:0/3673263354 learned_addr learned my addr 172.21.15.22:0/3673263354 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-28T18:08:28.813 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:28.812+0000 7f80b8e00640 1 --2- 172.21.15.22:0/3673263354 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55f9b212e260 0x55f9b213ad60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:08:28.813 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:28.812+0000 7f80b9800640 1 -- 172.21.15.22:0/3673263354 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f9b2136de0 msgr2=0x55f9b213b2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T18:08:28.813 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:28.812+0000 7f80ba200640 1 --2- 172.21.15.22:0/3673263354 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f9b2136de0 0x55f9b213b2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:08:28.814 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:28.812+0000 7f80b9800640 1 --2- 172.21.15.22:0/3673263354 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f9b2136de0 0x55f9b213b2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:08:28.814 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:28.812+0000 7f80b9800640 1 -- 172.21.15.22:0/3673263354 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55f9b212e260 msgr2=0x55f9b213ad60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T18:08:28.814 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:28.812+0000 7f80b9800640 1 --2- 172.21.15.22:0/3673263354 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55f9b212e260 0x55f9b213ad60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:08:28.814 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:28.812+0000 7f80b9800640 1 -- 172.21.15.22:0/3673263354 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55f9b21a14a0 con 0x55f9b212d8b0 2024-03-28T18:08:28.814 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:28.812+0000 7f80b8e00640 1 --2- 172.21.15.22:0/3673263354 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55f9b212e260 0x55f9b213ad60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-28T18:08:28.814 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:28.812+0000 7f80ba200640 1 --2- 172.21.15.22:0/3673263354 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f9b2136de0 0x55f9b213b2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T18:08:28.814 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:28.813+0000 7f80b9800640 1 --2- 172.21.15.22:0/3673263354 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f9b212d8b0 0x55f9b213a820 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f80a8019650 tx=0x7f80a8009810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=504e269cb45a97b5 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:08:28.814 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:28.813+0000 7f809b400640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-03-28T18:08:28.814 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:28.813+0000 7f809b400640 1 -- 172.21.15.22:0/3673263354 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80a8030070 con 0x55f9b212d8b0 2024-03-28T18:08:28.814 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:28.813+0000 7f809b400640 1 -- 172.21.15.22:0/3673263354 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 8411+0+0 (secure 0 0 0) 0x7f80a80199b0 con 0x55f9b212d8b0 2024-03-28T18:08:28.814 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:28.813+0000 7f80bef81740 1 -- 172.21.15.22:0/3673263354 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f80a8009b70 con 0x55f9b212d8b0 2024-03-28T18:08:28.814 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:28.813+0000 7f809b400640 1 -- 172.21.15.22:0/3673263354 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f80a8027040 con 0x55f9b212d8b0 2024-03-28T18:08:28.814 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:28.813+0000 7f80bef81740 10 client.15843 fetch_fsmap learned FSMap version 27 2024-03-28T18:08:28.814 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:28.813+0000 7f80bef81740 1 -- 172.21.15.22:0/3673263354 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55f9b21a2780 con 0x55f9b212d8b0 2024-03-28T18:08:28.816 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:28.814+0000 7f809b400640 1 -- 172.21.15.22:0/3673263354 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f80a8009b70 con 0x55f9b212d8b0 2024-03-28T18:08:28.816 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:28.814+0000 7f809b400640 1 -- 172.21.15.22:0/3673263354 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 27) v1 ==== 7934+0+0 (secure 0 0 0) 0x7f80a8048050 con 0x55f9b212d8b0 2024-03-28T18:08:28.816 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:28.815+0000 7f80bef81740 10 client.15843 fetch_fsmap finished waiting for FSMap version 27 2024-03-28T18:08:28.816 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:28.815+0000 7f80bef81740 10 client.15843 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.e 2024-03-28T18:08:28.816 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:28.815+0000 7f80bef81740 20 client.15843 populate_metadata read hostname 'smithi022' 2024-03-28T18:08:28.816 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:28.815+0000 7f80bef81740 1 --2- 172.21.15.22:0/3673263354 >> [v2:172.21.15.79:6834/3365773431,v1:172.21.15.79:6835/3365773431] conn(0x55f9b21a2d30 0x55f9b21a5120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:08:28.816 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:28.815+0000 7f80bef81740 4 client.15843 mds_command: new command op to 34475 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-28T18:08:28.816 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:28.815+0000 7f80bef81740 1 -- 172.21.15.22:0/3673263354 --> [v2:172.21.15.79:6834/3365773431,v1:172.21.15.79:6835/3365773431] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55f9b2098b50 con 0x55f9b21a2d30 2024-03-28T18:08:28.817 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:28.815+0000 7f80b8e00640 1 --2- 172.21.15.22:0/3673263354 >> [v2:172.21.15.79:6834/3365773431,v1:172.21.15.79:6835/3365773431] conn(0x55f9b21a2d30 0x55f9b21a5120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:08:28.817 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:28.816+0000 7f80b8e00640 1 --2- 172.21.15.22:0/3673263354 >> [v2:172.21.15.79:6834/3365773431,v1:172.21.15.79:6835/3365773431] conn(0x55f9b21a2d30 0x55f9b21a5120 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=b5ec84971e3a6072 server_cookie=943e7ac252e36754 in_seq=0 out_seq=0 2024-03-28T18:08:28.817 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:28.816+0000 7f809b400640 10 client.15843 ms_handle_connect on v2:172.21.15.79:6834/3365773431 2024-03-28T18:08:28.817 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:28.816+0000 7f809b400640 1 -- 172.21.15.22:0/3673263354 <== mds.2 v2:172.21.15.79:6834/3365773431 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55f9b2098b50 con 0x55f9b21a2d30 2024-03-28T18:08:28.817 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:28.816+0000 7f809b400640 10 client.15843 handle_command_reply: tid=0 2024-03-28T18:08:28.817 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:28.816+0000 7f80bef81740 1 client.15843 shutdown 2024-03-28T18:08:28.817 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:28.816+0000 7f80bef81740 1 -- 172.21.15.22:0/3673263354 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f9b212d8b0 msgr2=0x55f9b213a820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:08:28.817 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:28.816+0000 7f80bef81740 1 --2- 172.21.15.22:0/3673263354 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f9b212d8b0 0x55f9b213a820 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f80a8019650 tx=0x7f80a8009810 comp rx=0 tx=0).stop 2024-03-28T18:08:28.817 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:28.816+0000 7f80bef81740 1 -- 172.21.15.22:0/3673263354 shutdown_connections 2024-03-28T18:08:28.817 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:28.816+0000 7f80bef81740 1 --2- 172.21.15.22:0/3673263354 >> [v2:172.21.15.79:6834/3365773431,v1:172.21.15.79:6835/3365773431] conn(0x55f9b21a2d30 0x55f9b21a5120 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:08:28.817 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:28.816+0000 7f80bef81740 1 --2- 172.21.15.22:0/3673263354 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f9b2136de0 0x55f9b213b2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:08:28.817 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:28.816+0000 7f80bef81740 1 --2- 172.21.15.22:0/3673263354 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55f9b212e260 0x55f9b213ad60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:08:28.818 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:28.816+0000 7f80bef81740 1 --2- 172.21.15.22:0/3673263354 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f9b212d8b0 0x55f9b213a820 unknown :-1 s=CLOSED pgs=533 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:08:28.818 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:28.816+0000 7f80bef81740 1 -- 172.21.15.22:0/3673263354 >> 172.21.15.22:0/3673263354 conn(0x55f9b2127f80 msgr2=0x55f9b214d680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:08:28.818 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:28.816+0000 7f809b400640 0 client.15843 ms_handle_reset on v2:172.21.15.79:6834/3365773431 2024-03-28T18:08:28.818 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:28.816+0000 7f80bef81740 1 -- 172.21.15.22:0/3673263354 shutdown_connections 2024-03-28T18:08:28.818 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:28.816+0000 7f80bef81740 1 -- 172.21.15.22:0/3673263354 wait complete. 2024-03-28T18:08:28.818 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:28.817+0000 7f80bef81740 20 client.15843 trim_cache size 0 max 16384 2024-03-28T18:08:28.818 INFO:teuthology.orchestra.run.smithi022.stdout:[] 2024-03-28T18:08:28.819 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:28.817+0000 7f80bac00640 1 -- 172.21.15.22:0/3430334210 >> [v2:172.21.15.22:6800/1357365082,v1:172.21.15.22:6801/1357365082] conn(0x7f809c0a2b10 msgr2=0x7f809c0a4fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:08:28.819 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:28.817+0000 7f80bac00640 1 --2- 172.21.15.22:0/3430334210 >> [v2:172.21.15.22:6800/1357365082,v1:172.21.15.22:6801/1357365082] conn(0x7f809c0a2b10 0x7f809c0a4fd0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f80b40698e0 tx=0x7f80ac009600 comp rx=0 tx=0).stop 2024-03-28T18:08:28.819 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:28.817+0000 7f80bac00640 1 -- 172.21.15.22:0/3430334210 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f80b4100c80 msgr2=0x7f80b419a660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:08:28.819 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:28.817+0000 7f80bac00640 1 --2- 172.21.15.22:0/3430334210 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f80b4100c80 0x7f80b419a660 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7f80a0000c00 tx=0x7f80a00045e0 comp rx=0 tx=0).stop 2024-03-28T18:08:28.819 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:28.818+0000 7f80bac00640 1 -- 172.21.15.22:0/3430334210 shutdown_connections 2024-03-28T18:08:28.819 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:28.818+0000 7f80bac00640 1 --2- 172.21.15.22:0/3430334210 >> [v2:172.21.15.22:6800/1357365082,v1:172.21.15.22:6801/1357365082] conn(0x7f809c0a2b10 0x7f809c0a4fd0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:08:28.819 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:28.818+0000 7f80bac00640 1 --2- 172.21.15.22:0/3430334210 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f80b410de80 0x7f80b419e990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:08:28.819 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:28.818+0000 7f80bac00640 1 --2- 172.21.15.22:0/3430334210 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f80b4100c80 0x7f80b419a660 unknown :-1 s=CLOSED pgs=529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:08:28.819 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:28.818+0000 7f80bac00640 1 --2- 172.21.15.22:0/3430334210 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f80b41002d0 0x7f80b419a120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:08:28.820 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:28.818+0000 7f80bac00640 1 -- 172.21.15.22:0/3430334210 >> 172.21.15.22:0/3430334210 conn(0x7f80b40fbef0 msgr2=0x7f80b40fe2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:08:28.820 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:28.818+0000 7f80bac00640 1 -- 172.21.15.22:0/3430334210 shutdown_connections 2024-03-28T18:08:28.820 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:28.818+0000 7f80bac00640 1 -- 172.21.15.22:0/3430334210 wait complete. 2024-03-28T18:08:28.835 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-28T18:08:28.835 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-03-28T18:08:29.052 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:29.052+0000 7f25c4c00640 1 -- 172.21.15.22:0/3509210130 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f25c01026d0 msgr2=0x7f25c0102ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:08:29.052 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:29.052+0000 7f25c4c00640 1 --2- 172.21.15.22:0/3509210130 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f25c01026d0 0x7f25c0102ab0 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f25b0009900 tx=0x7f25b002bcc0 comp rx=0 tx=0).stop 2024-03-28T18:08:29.052 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:29.052+0000 7f25c4c00640 1 -- 172.21.15.22:0/3509210130 shutdown_connections 2024-03-28T18:08:29.052 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:29.052+0000 7f25c4c00640 1 --2- 172.21.15.22:0/3509210130 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f25c010baf0 0x7f25c010df90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:08:29.052 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:29.052+0000 7f25c4c00640 1 --2- 172.21.15.22:0/3509210130 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f25c0102ff0 0x7f25c010b4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:08:29.052 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:29.052+0000 7f25c4c00640 1 --2- 172.21.15.22:0/3509210130 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f25c01026d0 0x7f25c0102ab0 unknown :-1 s=CLOSED pgs=534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:08:29.052 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:29.052+0000 7f25c4c00640 1 -- 172.21.15.22:0/3509210130 >> 172.21.15.22:0/3509210130 conn(0x7f25c00fbef0 msgr2=0x7f25c00fe310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:08:29.052 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:29.052+0000 7f25c4c00640 1 -- 172.21.15.22:0/3509210130 shutdown_connections 2024-03-28T18:08:29.052 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:29.052+0000 7f25c4c00640 1 -- 172.21.15.22:0/3509210130 wait complete. 2024-03-28T18:08:29.054 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:29.053+0000 7f25c4c00640 1 Processor -- start 2024-03-28T18:08:29.054 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:29.053+0000 7f25c4c00640 1 -- start start 2024-03-28T18:08:29.054 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:29.053+0000 7f25c4c00640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f25c01026d0 0x7f25c019a020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:08:29.054 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:29.053+0000 7f25c4c00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f25c0102ff0 0x7f25c019a560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:08:29.054 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:29.054+0000 7f25c4c00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f25c010baf0 0x7f25c019e8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:08:29.054 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:29.054+0000 7f25c4c00640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25c01118c0 con 0x7f25c010baf0 2024-03-28T18:08:29.054 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:29.054+0000 7f25c4c00640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25c0111740 con 0x7f25c01026d0 2024-03-28T18:08:29.054 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:29.054+0000 7f25c4c00640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25c0111a40 con 0x7f25c0102ff0 2024-03-28T18:08:29.054 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:29.054+0000 7f25bfe00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f25c010baf0 0x7f25c019e8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:08:29.054 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:29.054+0000 7f25bfe00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f25c010baf0 0x7f25c019e8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:38004/0 (socket says 172.21.15.22:38004) 2024-03-28T18:08:29.055 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:29.054+0000 7f25bfe00640 1 -- 172.21.15.22:0/4093754848 learned_addr learned my addr 172.21.15.22:0/4093754848 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-28T18:08:29.055 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:29.054+0000 7f25bf400640 1 --2- 172.21.15.22:0/4093754848 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f25c01026d0 0x7f25c019a020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:08:29.055 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:29.054+0000 7f25bea00640 1 --2- 172.21.15.22:0/4093754848 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f25c0102ff0 0x7f25c019a560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:08:29.055 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:29.054+0000 7f25bfe00640 1 -- 172.21.15.22:0/4093754848 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f25c0102ff0 msgr2=0x7f25c019a560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:08:29.056 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:29.054+0000 7f25bfe00640 1 --2- 172.21.15.22:0/4093754848 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f25c0102ff0 0x7f25c019a560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:08:29.056 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:29.055+0000 7f25bfe00640 1 -- 172.21.15.22:0/4093754848 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f25c01026d0 msgr2=0x7f25c019a020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:08:29.056 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:29.055+0000 7f25bfe00640 1 --2- 172.21.15.22:0/4093754848 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f25c01026d0 0x7f25c019a020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:08:29.056 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:29.055+0000 7f25bfe00640 1 -- 172.21.15.22:0/4093754848 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f25c019f010 con 0x7f25c010baf0 2024-03-28T18:08:29.056 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:29.055+0000 7f25bfe00640 1 --2- 172.21.15.22:0/4093754848 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f25c010baf0 0x7f25c019e8e0 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7f25b800eb10 tx=0x7f25b800c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:08:29.056 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:29.055+0000 7f25b7e00640 1 -- 172.21.15.22:0/4093754848 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25b8004b20 con 0x7f25c010baf0 2024-03-28T18:08:29.056 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:29.055+0000 7f25b7e00640 1 -- 172.21.15.22:0/4093754848 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f25b8004cc0 con 0x7f25c010baf0 2024-03-28T18:08:29.056 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:29.055+0000 7f25c4c00640 1 -- 172.21.15.22:0/4093754848 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f25c019f300 con 0x7f25c010baf0 2024-03-28T18:08:29.056 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:29.055+0000 7f25b7e00640 1 -- 172.21.15.22:0/4093754848 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25b8010700 con 0x7f25c010baf0 2024-03-28T18:08:29.056 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:29.056+0000 7f25c4c00640 1 -- 172.21.15.22:0/4093754848 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f25c01a6b90 con 0x7f25c010baf0 2024-03-28T18:08:29.059 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:29.060+0000 7f25c8df4740 1 Processor -- start 2024-03-28T18:08:29.060 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:29.060+0000 7f25b7e00640 1 -- 172.21.15.22:0/4093754848 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 25) v1 ==== 137523+0+0 (secure 0 0 0) 0x7f25b8004e70 con 0x7f25c010baf0 2024-03-28T18:08:29.060 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:29.060+0000 7f25c8df4740 1 -- start start 2024-03-28T18:08:29.061 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:29.061+0000 7f25c8df4740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55dec52d88b0 0x55dec52d8c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:08:29.061 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:29.061+0000 7f25c8df4740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55dec52d9260 0x55dec52e1810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:08:29.061 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:29.061+0000 7f25c8df4740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55dec52e1de0 0x55dec52e4230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:08:29.061 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:29.061+0000 7f25c8df4740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dec523c2c0 con 0x55dec52d88b0 2024-03-28T18:08:29.061 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:29.061+0000 7f25c8df4740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dec51e4c90 con 0x55dec52d9260 2024-03-28T18:08:29.061 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:29.061+0000 7f25c8df4740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dec5238360 con 0x55dec52e1de0 2024-03-28T18:08:29.061 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:29.061+0000 7f25bea00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55dec52d88b0 0x55dec52d8c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:08:29.061 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:29.061+0000 7f25bea00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55dec52d88b0 0x55dec52d8c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:38016/0 (socket says 172.21.15.22:38016) 2024-03-28T18:08:29.062 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:29.061+0000 7f25bea00640 1 -- 172.21.15.22:0/2908767713 learned_addr learned my addr 172.21.15.22:0/2908767713 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-28T18:08:29.062 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:29.062+0000 7f25bf400640 1 --2- 172.21.15.22:0/2908767713 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55dec52d9260 0x55dec52e1810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:08:29.062 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:29.062+0000 7f25bea00640 1 -- 172.21.15.22:0/2908767713 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55dec52e1de0 msgr2=0x55dec52e4230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T18:08:29.062 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:29.062+0000 7f25bea00640 1 --2- 172.21.15.22:0/2908767713 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55dec52e1de0 0x55dec52e4230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:08:29.062 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:29.062+0000 7f25bea00640 1 -- 172.21.15.22:0/2908767713 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55dec52d9260 msgr2=0x55dec52e1810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T18:08:29.062 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:29.062+0000 7f25bea00640 1 --2- 172.21.15.22:0/2908767713 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55dec52d9260 0x55dec52e1810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:08:29.063 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:29.062+0000 7f25b7e00640 1 --2- 172.21.15.22:0/4093754848 >> [v2:172.21.15.22:6800/1357365082,v1:172.21.15.22:6801/1357365082] conn(0x7f25a40a2b10 0x7f25a40a4fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:08:29.063 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:29.062+0000 7f25bea00640 1 -- 172.21.15.22:0/2908767713 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55dec5209080 con 0x55dec52d88b0 2024-03-28T18:08:29.063 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:29.062+0000 7f25b7e00640 1 -- 172.21.15.22:0/4093754848 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 8411+0+0 (secure 0 0 0) 0x7f25b80cf5c0 con 0x7f25c010baf0 2024-03-28T18:08:29.063 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:29.063+0000 7f25bea00640 1 --2- 172.21.15.22:0/4093754848 >> [v2:172.21.15.22:6800/1357365082,v1:172.21.15.22:6801/1357365082] conn(0x7f25a40a2b10 0x7f25a40a4fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:08:29.063 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:29.063+0000 7f25bea00640 1 --2- 172.21.15.22:0/2908767713 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55dec52d88b0 0x55dec52d8c90 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7f25ac00b7b0 tx=0x7f25ac00bc70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3c98d513b9391f3 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:08:29.063 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:29.063+0000 7f25abe00640 1 -- 172.21.15.22:0/2908767713 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25ac010630 con 0x55dec52d88b0 2024-03-28T18:08:29.063 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:29.063+0000 7f25abe00640 1 -- 172.21.15.22:0/2908767713 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f25ac016040 con 0x55dec52d88b0 2024-03-28T18:08:29.064 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:29.064+0000 7f25abe00640 1 -- 172.21.15.22:0/2908767713 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25ac00f820 con 0x55dec52d88b0 2024-03-28T18:08:29.064 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:29.064+0000 7f25c8df4740 1 -- 172.21.15.22:0/2908767713 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55dec52d88b0 msgr2=0x55dec52d8c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:08:29.064 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:29.064+0000 7f25bea00640 1 --2- 172.21.15.22:0/4093754848 >> [v2:172.21.15.22:6800/1357365082,v1:172.21.15.22:6801/1357365082] conn(0x7f25a40a2b10 0x7f25a40a4fd0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f25ac01f000 tx=0x7f25ac01ead0 comp rx=0 tx=0).ready entity=mgr.15522 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:08:29.064 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:29.064+0000 7f25c8df4740 1 --2- 172.21.15.22:0/2908767713 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55dec52d88b0 0x55dec52d8c90 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7f25ac00b7b0 tx=0x7f25ac00bc70 comp rx=0 tx=0).stop 2024-03-28T18:08:29.065 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:29.065+0000 7f25c8df4740 1 -- 172.21.15.22:0/2908767713 shutdown_connections 2024-03-28T18:08:29.065 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:29.065+0000 7f25c8df4740 1 --2- 172.21.15.22:0/2908767713 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55dec52e1de0 0x55dec52e4230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:08:29.065 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:29.065+0000 7f25c8df4740 1 --2- 172.21.15.22:0/2908767713 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55dec52d9260 0x55dec52e1810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:08:29.065 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:29.065+0000 7f25c8df4740 1 --2- 172.21.15.22:0/2908767713 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55dec52d88b0 0x55dec52d8c90 unknown :-1 s=CLOSED pgs=536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:08:29.065 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:29.065+0000 7f25c8df4740 1 -- 172.21.15.22:0/2908767713 >> 172.21.15.22:0/2908767713 conn(0x55dec52d2f80 msgr2=0x55dec52d3360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:08:29.065 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:29.065+0000 7f25c8df4740 1 -- 172.21.15.22:0/2908767713 shutdown_connections 2024-03-28T18:08:29.065 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:29.065+0000 7f25c8df4740 1 -- 172.21.15.22:0/2908767713 wait complete. 2024-03-28T18:08:29.065 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:29.065+0000 7f25c8df4740 1 -- start start 2024-03-28T18:08:29.066 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:29.066+0000 7f25c8df4740 1 Processor -- start 2024-03-28T18:08:29.066 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:29.066+0000 7f25c8df4740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55dec52d88b0 0x55dec52e15d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:08:29.067 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:29.066+0000 7f25c8df4740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55dec52d9260 0x55dec52de960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:08:29.067 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:29.066+0000 7f25c8df4740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55dec52e1de0 0x55dec52deea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:08:29.067 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:29.066+0000 7f25c8df4740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dec52e4da0 con 0x55dec52d9260 2024-03-28T18:08:29.067 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:29.066+0000 7f25c8df4740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dec51f80d0 con 0x55dec52d88b0 2024-03-28T18:08:29.067 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:29.066+0000 7f25c8df4740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dec52e4f20 con 0x55dec52e1de0 2024-03-28T18:08:29.067 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:29.067+0000 7f25c8df4740 10 client.0 _ll_register_callbacks cb 0x55dec51fce80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-28T18:08:29.067 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:29.067+0000 7f25bea00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55dec52d9260 0x55dec52de960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:08:29.067 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:29.067+0000 7f25bea00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55dec52d9260 0x55dec52de960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:38026/0 (socket says 172.21.15.22:38026) 2024-03-28T18:08:29.067 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:29.067+0000 7f25bea00640 1 -- 172.21.15.22:0/1894488146 learned_addr learned my addr 172.21.15.22:0/1894488146 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-28T18:08:29.067 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:29.067+0000 7f25bf400640 1 --2- 172.21.15.22:0/1894488146 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55dec52d88b0 0x55dec52e15d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:08:29.068 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:29.067+0000 7f25bfe00640 1 --2- 172.21.15.22:0/1894488146 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55dec52e1de0 0x55dec52deea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:08:29.069 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:29.067+0000 7f25bea00640 1 -- 172.21.15.22:0/1894488146 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55dec52e1de0 msgr2=0x55dec52deea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T18:08:29.069 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:29.067+0000 7f25bea00640 1 --2- 172.21.15.22:0/1894488146 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55dec52e1de0 0x55dec52deea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:08:29.069 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:29.067+0000 7f25bea00640 1 -- 172.21.15.22:0/1894488146 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55dec52d88b0 msgr2=0x55dec52e15d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T18:08:29.069 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:29.067+0000 7f25bea00640 1 --2- 172.21.15.22:0/1894488146 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55dec52d88b0 0x55dec52e15d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:08:29.069 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:29.067+0000 7f25bea00640 1 -- 172.21.15.22:0/1894488146 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55dec4f503f0 con 0x55dec52d9260 2024-03-28T18:08:29.069 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:29.068+0000 7f25bf400640 1 --2- 172.21.15.22:0/1894488146 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55dec52d88b0 0x55dec52e15d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-28T18:08:29.069 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:29.068+0000 7f25bea00640 1 --2- 172.21.15.22:0/1894488146 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55dec52d9260 0x55dec52de960 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7f25ac00b320 tx=0x7f25ac015000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3815034176be3242 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:08:29.069 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:29.068+0000 7f25a8c00640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-03-28T18:08:29.069 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:29.068+0000 7f25a8c00640 1 -- 172.21.15.22:0/1894488146 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25ac01e8e0 con 0x55dec52d9260 2024-03-28T18:08:29.069 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:29.068+0000 7f25c8df4740 1 -- 172.21.15.22:0/1894488146 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f25ac016040 con 0x55dec52d9260 2024-03-28T18:08:29.069 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:29.068+0000 7f25a8c00640 1 -- 172.21.15.22:0/1894488146 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 8411+0+0 (secure 0 0 0) 0x7f25ac005040 con 0x55dec52d9260 2024-03-28T18:08:29.070 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:29.069+0000 7f25a8c00640 1 -- 172.21.15.22:0/1894488146 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f25ac005240 con 0x55dec52d9260 2024-03-28T18:08:29.070 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:29.069+0000 7f25c8df4740 10 client.15855 fetch_fsmap learned FSMap version 27 2024-03-28T18:08:29.070 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:29.069+0000 7f25c8df4740 1 -- 172.21.15.22:0/1894488146 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55dec501a7d0 con 0x55dec52d9260 2024-03-28T18:08:29.070 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:29.069+0000 7f25a8c00640 1 -- 172.21.15.22:0/1894488146 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f25ac016040 con 0x55dec52d9260 2024-03-28T18:08:29.071 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:29.070+0000 7f25a8c00640 1 -- 172.21.15.22:0/1894488146 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 27) v1 ==== 7934+0+0 (secure 0 0 0) 0x7f25ac01c070 con 0x55dec52d9260 2024-03-28T18:08:29.071 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:29.070+0000 7f25c8df4740 10 client.15855 fetch_fsmap finished waiting for FSMap version 27 2024-03-28T18:08:29.071 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:29.070+0000 7f25c8df4740 10 client.15855 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-03-28T18:08:29.071 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:29.070+0000 7f25c8df4740 20 client.15855 populate_metadata read hostname 'smithi022' 2024-03-28T18:08:29.071 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:29.070+0000 7f25c8df4740 1 --2- 172.21.15.22:0/1894488146 >> [v2:172.21.15.79:6836/1787903366,v1:172.21.15.79:6837/1787903366] conn(0x55dec52fa2f0 0x55dec52fc6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:08:29.071 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:29.070+0000 7f25c8df4740 4 client.15855 mds_command: new command op to 34511 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-28T18:08:29.071 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:29.070+0000 7f25c8df4740 1 -- 172.21.15.22:0/1894488146 --> [v2:172.21.15.79:6836/1787903366,v1:172.21.15.79:6837/1787903366] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55dec5249800 con 0x55dec52fa2f0 2024-03-28T18:08:29.071 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:29.070+0000 7f25bf400640 1 --2- 172.21.15.22:0/1894488146 >> [v2:172.21.15.79:6836/1787903366,v1:172.21.15.79:6837/1787903366] conn(0x55dec52fa2f0 0x55dec52fc6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:08:29.071 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:29.071+0000 7f25bf400640 1 --2- 172.21.15.22:0/1894488146 >> [v2:172.21.15.79:6836/1787903366,v1:172.21.15.79:6837/1787903366] conn(0x55dec52fa2f0 0x55dec52fc6e0 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=94ae4fdc2d430da5 server_cookie=b6c123ad2918e14e in_seq=0 out_seq=0 2024-03-28T18:08:29.071 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:29.071+0000 7f25a8c00640 10 client.15855 ms_handle_connect on v2:172.21.15.79:6836/1787903366 2024-03-28T18:08:29.072 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:29.072+0000 7f25a8c00640 1 -- 172.21.15.22:0/1894488146 <== mds.1 v2:172.21.15.79:6836/1787903366 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x55dec5249800 con 0x55dec52fa2f0 2024-03-28T18:08:29.072 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:29.072+0000 7f25a8c00640 10 client.15855 handle_command_reply: tid=0 2024-03-28T18:08:29.072 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:29.072+0000 7f25c8df4740 1 client.15855 shutdown 2024-03-28T18:08:29.072 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:29.073+0000 7f25c8df4740 1 -- 172.21.15.22:0/1894488146 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55dec52d9260 msgr2=0x55dec52de960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:08:29.072 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:29.073+0000 7f25c8df4740 1 --2- 172.21.15.22:0/1894488146 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55dec52d9260 0x55dec52de960 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7f25ac00b320 tx=0x7f25ac015000 comp rx=0 tx=0).stop 2024-03-28T18:08:29.073 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:29.073+0000 7f25c8df4740 1 -- 172.21.15.22:0/1894488146 shutdown_connections 2024-03-28T18:08:29.073 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:29.073+0000 7f25c8df4740 1 --2- 172.21.15.22:0/1894488146 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55dec52e1de0 0x55dec52deea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:08:29.073 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:29.073+0000 7f25c8df4740 1 --2- 172.21.15.22:0/1894488146 >> [v2:172.21.15.79:6836/1787903366,v1:172.21.15.79:6837/1787903366] conn(0x55dec52fa2f0 0x55dec52fc6e0 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:08:29.073 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:29.073+0000 7f25c8df4740 1 --2- 172.21.15.22:0/1894488146 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55dec52d9260 0x55dec52de960 unknown :-1 s=CLOSED pgs=537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:08:29.073 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:29.073+0000 7f25c8df4740 1 --2- 172.21.15.22:0/1894488146 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55dec52d88b0 0x55dec52e15d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:08:29.073 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:29.073+0000 7f25c8df4740 1 -- 172.21.15.22:0/1894488146 >> 172.21.15.22:0/1894488146 conn(0x55dec52d2f80 msgr2=0x55dec52d4cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:08:29.073 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:29.073+0000 7f25a8c00640 0 client.15855 ms_handle_reset on v2:172.21.15.79:6836/1787903366 2024-03-28T18:08:29.073 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:29.073+0000 7f25c8df4740 1 -- 172.21.15.22:0/1894488146 shutdown_connections 2024-03-28T18:08:29.074 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:29.073+0000 7f25c8df4740 1 -- 172.21.15.22:0/1894488146 wait complete. 2024-03-28T18:08:29.074 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:29.073+0000 7f25c8df4740 20 client.15855 trim_cache size 0 max 16384 2024-03-28T18:08:29.088 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:29.088+0000 7f25c8df4740 1 Processor -- start 2024-03-28T18:08:29.088 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:29.088+0000 7f25c8df4740 1 -- start start 2024-03-28T18:08:29.088 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:29.088+0000 7f25c8df4740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55dec52d88b0 0x55dec52e1090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:08:29.088 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:29.088+0000 7f25c8df4740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55dec52e1de0 0x55dec52e15d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:08:29.088 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:29.088+0000 7f25c8df4740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55dec5301e90 0x55dec52ed630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:08:29.088 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:29.088+0000 7f25c8df4740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25c01aac20 con 0x55dec5301e90 2024-03-28T18:08:29.088 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:29.088+0000 7f25c8df4740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25c01ac5a0 con 0x55dec52d88b0 2024-03-28T18:08:29.088 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:29.088+0000 7f25c8df4740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25c01ab280 con 0x55dec52e1de0 2024-03-28T18:08:29.090 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:29.088+0000 7f25bfe00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55dec5301e90 0x55dec52ed630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:08:29.090 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:29.089+0000 7f25bfe00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55dec5301e90 0x55dec52ed630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:38036/0 (socket says 172.21.15.22:38036) 2024-03-28T18:08:29.090 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:29.089+0000 7f25bfe00640 1 -- 172.21.15.22:0/704507393 learned_addr learned my addr 172.21.15.22:0/704507393 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-28T18:08:29.090 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:29.089+0000 7f25bf400640 1 --2- 172.21.15.22:0/704507393 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55dec52d88b0 0x55dec52e1090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:08:29.090 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:29.089+0000 7f25bea00640 1 --2- 172.21.15.22:0/704507393 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55dec52e1de0 0x55dec52e15d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:08:29.090 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:29.089+0000 7f25bfe00640 1 -- 172.21.15.22:0/704507393 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55dec52e1de0 msgr2=0x55dec52e15d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T18:08:29.090 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:29.089+0000 7f25bfe00640 1 --2- 172.21.15.22:0/704507393 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55dec52e1de0 0x55dec52e15d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:08:29.090 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:29.089+0000 7f25bfe00640 1 -- 172.21.15.22:0/704507393 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55dec52d88b0 msgr2=0x55dec52e1090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T18:08:29.090 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:29.089+0000 7f25bfe00640 1 --2- 172.21.15.22:0/704507393 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55dec52d88b0 0x55dec52e1090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:08:29.091 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:29.089+0000 7f25bfe00640 1 -- 172.21.15.22:0/704507393 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55dec52d2db0 con 0x55dec5301e90 2024-03-28T18:08:29.091 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:29.089+0000 7f25bea00640 1 --2- 172.21.15.22:0/704507393 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55dec52e1de0 0x55dec52e15d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-28T18:08:29.091 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:29.089+0000 7f25bf400640 1 --2- 172.21.15.22:0/704507393 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55dec52d88b0 0x55dec52e1090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-28T18:08:29.091 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:29.089+0000 7f25bfe00640 1 --2- 172.21.15.22:0/704507393 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55dec5301e90 0x55dec52ed630 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7f25b8005820 tx=0x7f25b809b5e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cd9660c1e0fea173 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:08:29.091 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:29.090+0000 7f25ab400640 1 -- 172.21.15.22:0/704507393 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25b80b3070 con 0x55dec5301e90 2024-03-28T18:08:29.091 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:29.090+0000 7f25ab400640 1 -- 172.21.15.22:0/704507393 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f25b809b9e0 con 0x55dec5301e90 2024-03-28T18:08:29.091 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:29.090+0000 7f25ab400640 1 -- 172.21.15.22:0/704507393 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25b809bea0 con 0x55dec5301e90 2024-03-28T18:08:29.091 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:29.090+0000 7f25c8df4740 1 -- 172.21.15.22:0/704507393 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55dec5301e90 msgr2=0x55dec52ed630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:08:29.091 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:29.090+0000 7f25c8df4740 1 --2- 172.21.15.22:0/704507393 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55dec5301e90 0x55dec52ed630 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7f25b8005820 tx=0x7f25b809b5e0 comp rx=0 tx=0).stop 2024-03-28T18:08:29.091 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:29.091+0000 7f25c8df4740 1 -- 172.21.15.22:0/704507393 shutdown_connections 2024-03-28T18:08:29.091 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:29.091+0000 7f25c8df4740 1 --2- 172.21.15.22:0/704507393 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55dec5301e90 0x55dec52ed630 unknown :-1 s=CLOSED pgs=538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:08:29.091 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:29.091+0000 7f25c8df4740 1 --2- 172.21.15.22:0/704507393 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55dec52e1de0 0x55dec52e15d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:08:29.091 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:29.091+0000 7f25c8df4740 1 --2- 172.21.15.22:0/704507393 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55dec52d88b0 0x55dec52e1090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:08:29.092 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:29.091+0000 7f25c8df4740 1 -- 172.21.15.22:0/704507393 >> 172.21.15.22:0/704507393 conn(0x55dec52d2f80 msgr2=0x55dec52d2160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:08:29.094 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:29.091+0000 7f25c8df4740 1 -- 172.21.15.22:0/704507393 shutdown_connections 2024-03-28T18:08:29.094 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:29.091+0000 7f25c8df4740 1 -- 172.21.15.22:0/704507393 wait complete. 2024-03-28T18:08:29.094 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:29.091+0000 7f25c8df4740 1 -- start start 2024-03-28T18:08:29.094 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:29.091+0000 7f25c8df4740 1 Processor -- start 2024-03-28T18:08:29.094 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:29.092+0000 7f25c8df4740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55dec52d88b0 0x55dec52e7140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:08:29.094 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:29.092+0000 7f25c8df4740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55dec52e1de0 0x55dec52e7680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:08:29.094 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:29.092+0000 7f25c8df4740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55dec5301e90 0x55dec52e7bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:08:29.095 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:29.092+0000 7f25c8df4740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25c01abf40 con 0x55dec52e1de0 2024-03-28T18:08:29.095 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:29.092+0000 7f25c8df4740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25c01ab8e0 con 0x55dec52d88b0 2024-03-28T18:08:29.095 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:29.092+0000 7f25c8df4740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25c01a8e00 con 0x55dec5301e90 2024-03-28T18:08:29.095 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:29.092+0000 7f25c8df4740 10 client.0 _ll_register_callbacks cb 0x7f25b0009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-28T18:08:29.095 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:29.092+0000 7f25bea00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55dec52e1de0 0x55dec52e7680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:08:29.095 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:29.092+0000 7f25bea00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55dec52e1de0 0x55dec52e7680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:38048/0 (socket says 172.21.15.22:38048) 2024-03-28T18:08:29.095 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:29.092+0000 7f25bea00640 1 -- 172.21.15.22:0/2358094649 learned_addr learned my addr 172.21.15.22:0/2358094649 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-28T18:08:29.095 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:29.092+0000 7f25bf400640 1 --2- 172.21.15.22:0/2358094649 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55dec52d88b0 0x55dec52e7140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:08:29.095 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:29.093+0000 7f25bfe00640 1 --2- 172.21.15.22:0/2358094649 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55dec5301e90 0x55dec52e7bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:08:29.095 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:29.093+0000 7f25bea00640 1 -- 172.21.15.22:0/2358094649 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55dec5301e90 msgr2=0x55dec52e7bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T18:08:29.095 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:29.093+0000 7f25bea00640 1 --2- 172.21.15.22:0/2358094649 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55dec5301e90 0x55dec52e7bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:08:29.095 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:29.093+0000 7f25bea00640 1 -- 172.21.15.22:0/2358094649 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55dec52d88b0 msgr2=0x55dec52e7140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T18:08:29.095 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:29.093+0000 7f25bea00640 1 --2- 172.21.15.22:0/2358094649 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55dec52d88b0 0x55dec52e7140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:08:29.095 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:29.093+0000 7f25bea00640 1 -- 172.21.15.22:0/2358094649 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55dec5355a90 con 0x55dec52e1de0 2024-03-28T18:08:29.096 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:29.093+0000 7f25bf400640 1 --2- 172.21.15.22:0/2358094649 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55dec52d88b0 0x55dec52e7140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-28T18:08:29.096 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:29.093+0000 7f25bea00640 1 --2- 172.21.15.22:0/2358094649 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55dec52e1de0 0x55dec52e7680 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7f25ac00bcb0 tx=0x7f25ac0051d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=89858aa0168c4adb server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:08:29.096 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:29.093+0000 7f25a8c00640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-03-28T18:08:29.096 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:29.093+0000 7f25a8c00640 1 -- 172.21.15.22:0/2358094649 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25ac004140 con 0x55dec52e1de0 2024-03-28T18:08:29.096 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:29.094+0000 7f25a8c00640 1 -- 172.21.15.22:0/2358094649 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 8411+0+0 (secure 0 0 0) 0x7f25ac00faa0 con 0x55dec52e1de0 2024-03-28T18:08:29.096 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:29.094+0000 7f25c8df4740 1 -- 172.21.15.22:0/2358094649 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f25b809b9e0 con 0x55dec52e1de0 2024-03-28T18:08:29.096 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:29.094+0000 7f25a8c00640 1 -- 172.21.15.22:0/2358094649 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f25ac0055f0 con 0x55dec52e1de0 2024-03-28T18:08:29.096 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:29.094+0000 7f25c8df4740 10 client.15861 fetch_fsmap learned FSMap version 27 2024-03-28T18:08:29.096 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:29.094+0000 7f25c8df4740 1 -- 172.21.15.22:0/2358094649 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55dec5356d90 con 0x55dec52e1de0 2024-03-28T18:08:29.096 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:29.094+0000 7f25a8c00640 1 -- 172.21.15.22:0/2358094649 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f25b809b9e0 con 0x55dec52e1de0 2024-03-28T18:08:29.096 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:29.094+0000 7f25a8c00640 1 -- 172.21.15.22:0/2358094649 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 27) v1 ==== 7934+0+0 (secure 0 0 0) 0x7f25ac01c040 con 0x55dec52e1de0 2024-03-28T18:08:29.097 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:29.094+0000 7f25c8df4740 10 client.15861 fetch_fsmap finished waiting for FSMap version 27 2024-03-28T18:08:29.097 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:29.094+0000 7f25c8df4740 10 client.15861 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-03-28T18:08:29.097 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:29.094+0000 7f25c8df4740 20 client.15861 populate_metadata read hostname 'smithi022' 2024-03-28T18:08:29.097 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:29.094+0000 7f25c8df4740 1 --2- 172.21.15.22:0/2358094649 >> [v2:172.21.15.79:6836/1787903366,v1:172.21.15.79:6837/1787903366] conn(0x55dec5357390 0x55dec53597b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:08:29.097 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:29.094+0000 7f25c8df4740 4 client.15861 mds_command: new command op to 34511 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-28T18:08:29.097 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:29.094+0000 7f25c8df4740 1 -- 172.21.15.22:0/2358094649 --> [v2:172.21.15.79:6836/1787903366,v1:172.21.15.79:6837/1787903366] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55dec5243b50 con 0x55dec5357390 2024-03-28T18:08:29.098 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:29.095+0000 7f25bf400640 1 --2- 172.21.15.22:0/2358094649 >> [v2:172.21.15.79:6836/1787903366,v1:172.21.15.79:6837/1787903366] conn(0x55dec5357390 0x55dec53597b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:08:29.098 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:29.096+0000 7f25bf400640 1 --2- 172.21.15.22:0/2358094649 >> [v2:172.21.15.79:6836/1787903366,v1:172.21.15.79:6837/1787903366] conn(0x55dec5357390 0x55dec53597b0 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=49fdd15fc583b699 server_cookie=4bac1d2372aaeafb in_seq=0 out_seq=0 2024-03-28T18:08:29.098 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:29.096+0000 7f25a8c00640 10 client.15861 ms_handle_connect on v2:172.21.15.79:6836/1787903366 2024-03-28T18:08:29.098 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:29.096+0000 7f25a8c00640 1 -- 172.21.15.22:0/2358094649 <== mds.1 v2:172.21.15.79:6836/1787903366 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55dec5243b50 con 0x55dec5357390 2024-03-28T18:08:29.098 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:29.096+0000 7f25a8c00640 10 client.15861 handle_command_reply: tid=0 2024-03-28T18:08:29.098 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:29.096+0000 7f25c8df4740 1 client.15861 shutdown 2024-03-28T18:08:29.098 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:29.096+0000 7f25c8df4740 1 -- 172.21.15.22:0/2358094649 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55dec52e1de0 msgr2=0x55dec52e7680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:08:29.099 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:29.096+0000 7f25c8df4740 1 --2- 172.21.15.22:0/2358094649 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55dec52e1de0 0x55dec52e7680 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7f25ac00bcb0 tx=0x7f25ac0051d0 comp rx=0 tx=0).stop 2024-03-28T18:08:29.099 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:29.097+0000 7f25c8df4740 1 -- 172.21.15.22:0/2358094649 shutdown_connections 2024-03-28T18:08:29.099 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:29.097+0000 7f25c8df4740 1 --2- 172.21.15.22:0/2358094649 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55dec5301e90 0x55dec52e7bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:08:29.099 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:29.097+0000 7f25c8df4740 1 --2- 172.21.15.22:0/2358094649 >> [v2:172.21.15.79:6836/1787903366,v1:172.21.15.79:6837/1787903366] conn(0x55dec5357390 0x55dec53597b0 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:08:29.099 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:29.097+0000 7f25c8df4740 1 --2- 172.21.15.22:0/2358094649 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55dec52e1de0 0x55dec52e7680 unknown :-1 s=CLOSED pgs=539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:08:29.099 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:29.097+0000 7f25c8df4740 1 --2- 172.21.15.22:0/2358094649 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55dec52d88b0 0x55dec52e7140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:08:29.099 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:29.097+0000 7f25c8df4740 1 -- 172.21.15.22:0/2358094649 >> 172.21.15.22:0/2358094649 conn(0x55dec52d2f80 msgr2=0x55dec52d9640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:08:29.099 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:29.097+0000 7f25a8c00640 0 client.15861 ms_handle_reset on v2:172.21.15.79:6836/1787903366 2024-03-28T18:08:29.099 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:29.097+0000 7f25c8df4740 1 -- 172.21.15.22:0/2358094649 shutdown_connections 2024-03-28T18:08:29.099 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:29.097+0000 7f25c8df4740 1 -- 172.21.15.22:0/2358094649 wait complete. 2024-03-28T18:08:29.099 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:29.097+0000 7f25c8df4740 20 client.15861 trim_cache size 0 max 16384 2024-03-28T18:08:29.100 INFO:teuthology.orchestra.run.smithi022.stdout:[] 2024-03-28T18:08:29.100 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:29.097+0000 7f25c4c00640 1 -- 172.21.15.22:0/4093754848 >> [v2:172.21.15.22:6800/1357365082,v1:172.21.15.22:6801/1357365082] conn(0x7f25a40a2b10 msgr2=0x7f25a40a4fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:08:29.100 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:29.097+0000 7f25c4c00640 1 --2- 172.21.15.22:0/4093754848 >> [v2:172.21.15.22:6800/1357365082,v1:172.21.15.22:6801/1357365082] conn(0x7f25a40a2b10 0x7f25a40a4fd0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f25ac01f000 tx=0x7f25ac01ead0 comp rx=0 tx=0).stop 2024-03-28T18:08:29.101 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:29.097+0000 7f25c4c00640 1 -- 172.21.15.22:0/4093754848 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f25c010baf0 msgr2=0x7f25c019e8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:08:29.101 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:29.097+0000 7f25c4c00640 1 --2- 172.21.15.22:0/4093754848 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f25c010baf0 0x7f25c019e8e0 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7f25b800eb10 tx=0x7f25b800c2d0 comp rx=0 tx=0).stop 2024-03-28T18:08:29.101 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:29.098+0000 7f25c4c00640 1 -- 172.21.15.22:0/4093754848 shutdown_connections 2024-03-28T18:08:29.101 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:29.098+0000 7f25c4c00640 1 --2- 172.21.15.22:0/4093754848 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f25c010baf0 0x7f25c019e8e0 unknown :-1 s=CLOSED pgs=535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:08:29.101 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:29.098+0000 7f25c4c00640 1 --2- 172.21.15.22:0/4093754848 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f25c0102ff0 0x7f25c019a560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:08:29.101 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:29.098+0000 7f25c4c00640 1 --2- 172.21.15.22:0/4093754848 >> [v2:172.21.15.22:6800/1357365082,v1:172.21.15.22:6801/1357365082] conn(0x7f25a40a2b10 0x7f25a40a4fd0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:08:29.101 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:29.098+0000 7f25c4c00640 1 --2- 172.21.15.22:0/4093754848 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f25c01026d0 0x7f25c019a020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:08:29.101 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:29.098+0000 7f25c4c00640 1 -- 172.21.15.22:0/4093754848 >> 172.21.15.22:0/4093754848 conn(0x7f25c00fbef0 msgr2=0x7f25c00fcb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:08:29.101 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:29.098+0000 7f25c4c00640 1 -- 172.21.15.22:0/4093754848 shutdown_connections 2024-03-28T18:08:29.101 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:29.098+0000 7f25c4c00640 1 -- 172.21.15.22:0/4093754848 wait complete. 2024-03-28T18:08:29.121 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-28T18:08:29.121 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-03-28T18:08:29.325 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:29.325+0000 7fa2f4c00640 1 -- 172.21.15.22:0/2134942083 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa2f0109850 msgr2=0x7fa2f0109c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:08:29.325 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:29.325+0000 7fa2f4c00640 1 --2- 172.21.15.22:0/2134942083 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa2f0109850 0x7fa2f0109c30 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7fa2e0009930 tx=0x7fa2e002bc30 comp rx=0 tx=0).stop 2024-03-28T18:08:29.325 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:29.325+0000 7fa2f4c00640 1 -- 172.21.15.22:0/2134942083 shutdown_connections 2024-03-28T18:08:29.325 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:29.325+0000 7fa2f4c00640 1 --2- 172.21.15.22:0/2134942083 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa2f0104200 0x7fa2f0110710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:08:29.325 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:29.325+0000 7fa2f4c00640 1 --2- 172.21.15.22:0/2134942083 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa2f0103840 0x7fa2f0103cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:08:29.325 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:29.325+0000 7fa2f4c00640 1 --2- 172.21.15.22:0/2134942083 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa2f0109850 0x7fa2f0109c30 unknown :-1 s=CLOSED pgs=540 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:08:29.325 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:29.325+0000 7fa2f4c00640 1 -- 172.21.15.22:0/2134942083 >> 172.21.15.22:0/2134942083 conn(0x7fa2f00ff350 msgr2=0x7fa2f0101770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:08:29.325 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:29.325+0000 7fa2f4c00640 1 -- 172.21.15.22:0/2134942083 shutdown_connections 2024-03-28T18:08:29.325 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:29.325+0000 7fa2f4c00640 1 -- 172.21.15.22:0/2134942083 wait complete. 2024-03-28T18:08:29.327 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:29.326+0000 7fa2f4c00640 1 Processor -- start 2024-03-28T18:08:29.327 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:29.326+0000 7fa2f4c00640 1 -- start start 2024-03-28T18:08:29.327 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:29.327+0000 7fa2f4c00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa2f0103840 0x7fa2f019a160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:08:29.327 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:29.327+0000 7fa2f4c00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa2f0104200 0x7fa2f019a6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:08:29.327 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:29.327+0000 7fa2f4c00640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa2f0109850 0x7fa2f019ea20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:08:29.327 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:29.327+0000 7fa2f4c00640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2f0113c80 con 0x7fa2f0103840 2024-03-28T18:08:29.327 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:29.327+0000 7fa2f4c00640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2f0113b00 con 0x7fa2f0109850 2024-03-28T18:08:29.327 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:29.327+0000 7fa2f4c00640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2f0113e00 con 0x7fa2f0104200 2024-03-28T18:08:29.327 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:29.327+0000 7fa2ede00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa2f0103840 0x7fa2f019a160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:08:29.328 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:29.327+0000 7fa2ede00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa2f0103840 0x7fa2f019a160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:38076/0 (socket says 172.21.15.22:38076) 2024-03-28T18:08:29.328 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:29.327+0000 7fa2ede00640 1 -- 172.21.15.22:0/389321241 learned_addr learned my addr 172.21.15.22:0/389321241 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-28T18:08:29.328 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:29.327+0000 7fa2ee800640 1 --2- 172.21.15.22:0/389321241 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa2f0109850 0x7fa2f019ea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:08:29.328 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:29.327+0000 7fa2ed400640 1 --2- 172.21.15.22:0/389321241 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa2f0104200 0x7fa2f019a6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:08:29.328 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:29.327+0000 7fa2ede00640 1 -- 172.21.15.22:0/389321241 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa2f0104200 msgr2=0x7fa2f019a6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:08:29.328 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:29.327+0000 7fa2ede00640 1 --2- 172.21.15.22:0/389321241 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa2f0104200 0x7fa2f019a6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:08:29.328 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:29.327+0000 7fa2ede00640 1 -- 172.21.15.22:0/389321241 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa2f0109850 msgr2=0x7fa2f019ea20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:08:29.328 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:29.327+0000 7fa2ede00640 1 --2- 172.21.15.22:0/389321241 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa2f0109850 0x7fa2f019ea20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:08:29.328 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:29.327+0000 7fa2ede00640 1 -- 172.21.15.22:0/389321241 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa2f019f150 con 0x7fa2f0103840 2024-03-28T18:08:29.328 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:29.327+0000 7fa2ed400640 1 --2- 172.21.15.22:0/389321241 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa2f0104200 0x7fa2f019a6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-28T18:08:29.329 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:29.328+0000 7fa2ede00640 1 --2- 172.21.15.22:0/389321241 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa2f0103840 0x7fa2f019a160 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7fa2e0000c00 tx=0x7fa2e00045e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:08:29.329 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:29.328+0000 7fa2e6a00640 1 -- 172.21.15.22:0/389321241 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2e0043020 con 0x7fa2f0103840 2024-03-28T18:08:29.329 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:29.328+0000 7fa2e6a00640 1 -- 172.21.15.22:0/389321241 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7fa2e002caa0 con 0x7fa2f0103840 2024-03-28T18:08:29.329 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:29.328+0000 7fa2e6a00640 1 -- 172.21.15.22:0/389321241 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2e0003ae0 con 0x7fa2f0103840 2024-03-28T18:08:29.329 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:29.328+0000 7fa2f4c00640 1 -- 172.21.15.22:0/389321241 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa2f019f3e0 con 0x7fa2f0103840 2024-03-28T18:08:29.329 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:29.328+0000 7fa2f4c00640 1 -- 172.21.15.22:0/389321241 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa2f01a6cd0 con 0x7fa2f0103840 2024-03-28T18:08:29.333 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:29.332+0000 7fa2f787e740 1 Processor -- start 2024-03-28T18:08:29.333 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:29.332+0000 7fa2f787e740 1 -- start start 2024-03-28T18:08:29.333 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:29.333+0000 7fa2e6a00640 1 -- 172.21.15.22:0/389321241 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 25) v1 ==== 137523+0+0 (secure 0 0 0) 0x7fa2e0046020 con 0x7fa2f0103840 2024-03-28T18:08:29.333 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:29.333+0000 7fa2f787e740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d1030c06d0 0x55d1030c0ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:08:29.333 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:29.333+0000 7fa2f787e740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d1030c1080 0x55d1030c9630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:08:29.334 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:29.333+0000 7fa2f787e740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55d1030c9c00 0x55d1030cc050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:08:29.334 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:29.333+0000 7fa2f787e740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d1030242c0 con 0x55d1030c1080 2024-03-28T18:08:29.334 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:29.333+0000 7fa2f787e740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d102fccc90 con 0x55d1030c9c00 2024-03-28T18:08:29.334 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:29.333+0000 7fa2f787e740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d103020360 con 0x55d1030c06d0 2024-03-28T18:08:29.334 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:29.333+0000 7fa2ede00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d1030c1080 0x55d1030c9630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:08:29.334 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:29.333+0000 7fa2ede00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d1030c1080 0x55d1030c9630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:38084/0 (socket says 172.21.15.22:38084) 2024-03-28T18:08:29.334 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:29.333+0000 7fa2ede00640 1 -- 172.21.15.22:0/1853889283 learned_addr learned my addr 172.21.15.22:0/1853889283 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-28T18:08:29.334 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:29.333+0000 7fa2ed400640 1 --2- 172.21.15.22:0/1853889283 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55d1030c9c00 0x55d1030cc050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:08:29.334 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:29.334+0000 7fa2ee800640 1 --2- 172.21.15.22:0/1853889283 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d1030c06d0 0x55d1030c0ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:08:29.334 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:29.334+0000 7fa2ede00640 1 -- 172.21.15.22:0/1853889283 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d1030c06d0 msgr2=0x55d1030c0ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T18:08:29.334 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:29.334+0000 7fa2ede00640 1 --2- 172.21.15.22:0/1853889283 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d1030c06d0 0x55d1030c0ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:08:29.334 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:29.334+0000 7fa2ede00640 1 -- 172.21.15.22:0/1853889283 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55d1030c9c00 msgr2=0x55d1030cc050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T18:08:29.334 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:29.334+0000 7fa2ede00640 1 --2- 172.21.15.22:0/1853889283 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55d1030c9c00 0x55d1030cc050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:08:29.334 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:29.334+0000 7fa2ede00640 1 -- 172.21.15.22:0/1853889283 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d102ff1080 con 0x55d1030c1080 2024-03-28T18:08:29.334 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:29.334+0000 7fa2ed400640 1 --2- 172.21.15.22:0/1853889283 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55d1030c9c00 0x55d1030cc050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-28T18:08:29.336 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:29.334+0000 7fa2ede00640 1 --2- 172.21.15.22:0/1853889283 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d1030c1080 0x55d1030c9630 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7fa2e00ee790 tx=0x7fa2e00f90e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8018ecff487d4da6 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:08:29.336 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:29.334+0000 7fa2db000640 1 -- 172.21.15.22:0/1853889283 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2e00f9cf0 con 0x55d1030c1080 2024-03-28T18:08:29.336 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:29.334+0000 7fa2db000640 1 -- 172.21.15.22:0/1853889283 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7fa2e00fe070 con 0x55d1030c1080 2024-03-28T18:08:29.336 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:29.334+0000 7fa2db000640 1 -- 172.21.15.22:0/1853889283 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2e00f1750 con 0x55d1030c1080 2024-03-28T18:08:29.336 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:29.335+0000 7fa2e6a00640 1 --2- 172.21.15.22:0/389321241 >> [v2:172.21.15.22:6800/1357365082,v1:172.21.15.22:6801/1357365082] conn(0x7fa2d40a2b10 0x7fa2d40a4fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:08:29.336 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:29.335+0000 7fa2e6a00640 1 -- 172.21.15.22:0/389321241 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 8411+0+0 (secure 0 0 0) 0x7fa2e00f0f30 con 0x7fa2f0103840 2024-03-28T18:08:29.336 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:29.335+0000 7fa2ee800640 1 --2- 172.21.15.22:0/389321241 >> [v2:172.21.15.22:6800/1357365082,v1:172.21.15.22:6801/1357365082] conn(0x7fa2d40a2b10 0x7fa2d40a4fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:08:29.336 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:29.335+0000 7fa2f787e740 1 -- 172.21.15.22:0/1853889283 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d1030c1080 msgr2=0x55d1030c9630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:08:29.336 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:29.335+0000 7fa2f787e740 1 --2- 172.21.15.22:0/1853889283 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d1030c1080 0x55d1030c9630 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7fa2e00ee790 tx=0x7fa2e00f90e0 comp rx=0 tx=0).stop 2024-03-28T18:08:29.336 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:29.335+0000 7fa2f787e740 1 -- 172.21.15.22:0/1853889283 shutdown_connections 2024-03-28T18:08:29.337 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:29.335+0000 7fa2f787e740 1 --2- 172.21.15.22:0/1853889283 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55d1030c9c00 0x55d1030cc050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:08:29.337 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:29.335+0000 7fa2f787e740 1 --2- 172.21.15.22:0/1853889283 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d1030c1080 0x55d1030c9630 unknown :-1 s=CLOSED pgs=542 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:08:29.337 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:29.335+0000 7fa2f787e740 1 --2- 172.21.15.22:0/1853889283 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d1030c06d0 0x55d1030c0ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:08:29.337 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:29.335+0000 7fa2f787e740 1 -- 172.21.15.22:0/1853889283 >> 172.21.15.22:0/1853889283 conn(0x55d1030bada0 msgr2=0x55d1030bb180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:08:29.337 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:29.336+0000 7fa2f787e740 1 -- 172.21.15.22:0/1853889283 shutdown_connections 2024-03-28T18:08:29.337 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:29.336+0000 7fa2ee800640 1 --2- 172.21.15.22:0/389321241 >> [v2:172.21.15.22:6800/1357365082,v1:172.21.15.22:6801/1357365082] conn(0x7fa2d40a2b10 0x7fa2d40a4fd0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fa2e8005fd0 tx=0x7fa2e8009600 comp rx=0 tx=0).ready entity=mgr.15522 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:08:29.337 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:29.336+0000 7fa2f787e740 1 -- 172.21.15.22:0/1853889283 wait complete. 2024-03-28T18:08:29.337 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:29.336+0000 7fa2f787e740 1 -- start start 2024-03-28T18:08:29.337 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:29.336+0000 7fa2f787e740 1 Processor -- start 2024-03-28T18:08:29.340 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:29.337+0000 7fa2f787e740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d1030c06d0 0x55d1030c9580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:08:29.340 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:29.337+0000 7fa2f787e740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d1030c1080 0x55d1030c6910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:08:29.340 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:29.337+0000 7fa2f787e740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55d1030c9c00 0x55d1030c6f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:08:29.340 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:29.337+0000 7fa2f787e740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d1030cc930 con 0x55d1030c06d0 2024-03-28T18:08:29.340 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:29.337+0000 7fa2f787e740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d102fe00d0 con 0x55d1030c9c00 2024-03-28T18:08:29.340 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:29.337+0000 7fa2f787e740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d1030ccab0 con 0x55d1030c1080 2024-03-28T18:08:29.340 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:29.337+0000 7fa2f787e740 10 client.0 _ll_register_callbacks cb 0x55d102fe4e80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-28T18:08:29.340 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:29.337+0000 7fa2ede00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d1030c06d0 0x55d1030c9580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:08:29.341 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:29.337+0000 7fa2ede00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d1030c06d0 0x55d1030c9580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:38094/0 (socket says 172.21.15.22:38094) 2024-03-28T18:08:29.341 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:29.337+0000 7fa2ede00640 1 -- 172.21.15.22:0/3711535011 learned_addr learned my addr 172.21.15.22:0/3711535011 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-28T18:08:29.341 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:29.337+0000 7fa2ed400640 1 --2- 172.21.15.22:0/3711535011 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d1030c1080 0x55d1030c6910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:08:29.341 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:29.337+0000 7fa2ee800640 1 --2- 172.21.15.22:0/3711535011 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55d1030c9c00 0x55d1030c6f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:08:29.341 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:29.337+0000 7fa2ede00640 1 -- 172.21.15.22:0/3711535011 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d1030c1080 msgr2=0x55d1030c6910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T18:08:29.341 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:29.337+0000 7fa2ede00640 1 --2- 172.21.15.22:0/3711535011 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d1030c1080 0x55d1030c6910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:08:29.341 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:29.337+0000 7fa2ede00640 1 -- 172.21.15.22:0/3711535011 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55d1030c9c00 msgr2=0x55d1030c6f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T18:08:29.341 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:29.337+0000 7fa2ede00640 1 --2- 172.21.15.22:0/3711535011 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55d1030c9c00 0x55d1030c6f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:08:29.341 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:29.337+0000 7fa2ede00640 1 -- 172.21.15.22:0/3711535011 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55d102d383f0 con 0x55d1030c06d0 2024-03-28T18:08:29.341 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:29.337+0000 7fa2ee800640 1 --2- 172.21.15.22:0/3711535011 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55d1030c9c00 0x55d1030c6f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T18:08:29.341 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:29.338+0000 7fa2ede00640 1 --2- 172.21.15.22:0/3711535011 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d1030c06d0 0x55d1030c9580 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7fa2e00bd290 tx=0x7fa2e00035f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fd689ae2c782c4b0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:08:29.341 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:29.338+0000 7fa2cf400640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-03-28T18:08:29.341 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:29.338+0000 7fa2cf400640 1 -- 172.21.15.22:0/3711535011 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2e00b9830 con 0x55d1030c06d0 2024-03-28T18:08:29.341 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:29.338+0000 7fa2cf400640 1 -- 172.21.15.22:0/3711535011 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 8411+0+0 (secure 0 0 0) 0x7fa2e00bb890 con 0x55d1030c06d0 2024-03-28T18:08:29.341 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:29.338+0000 7fa2f787e740 1 -- 172.21.15.22:0/3711535011 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fa2e00fe070 con 0x55d1030c06d0 2024-03-28T18:08:29.342 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:29.338+0000 7fa2cf400640 1 -- 172.21.15.22:0/3711535011 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa2e00bcef0 con 0x55d1030c06d0 2024-03-28T18:08:29.342 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:29.338+0000 7fa2f787e740 10 client.15873 fetch_fsmap learned FSMap version 27 2024-03-28T18:08:29.342 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:29.338+0000 7fa2f787e740 1 -- 172.21.15.22:0/3711535011 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55d102e027d0 con 0x55d1030c06d0 2024-03-28T18:08:29.342 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:29.339+0000 7fa2cf400640 1 -- 172.21.15.22:0/3711535011 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7fa2e00fe070 con 0x55d1030c06d0 2024-03-28T18:08:29.342 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:29.339+0000 7fa2cf400640 1 -- 172.21.15.22:0/3711535011 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 27) v1 ==== 7934+0+0 (secure 0 0 0) 0x7fa2e0005330 con 0x55d1030c06d0 2024-03-28T18:08:29.342 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:29.339+0000 7fa2f787e740 10 client.15873 fetch_fsmap finished waiting for FSMap version 27 2024-03-28T18:08:29.342 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:29.339+0000 7fa2f787e740 10 client.15873 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-28T18:08:29.342 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:29.339+0000 7fa2f787e740 20 client.15873 populate_metadata read hostname 'smithi022' 2024-03-28T18:08:29.342 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:29.339+0000 7fa2f787e740 1 --2- 172.21.15.22:0/3711535011 >> [v2:172.21.15.79:6838/2441542443,v1:172.21.15.79:6839/2441542443] conn(0x55d1030d96c0 0x55d1030dbab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:08:29.342 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:29.339+0000 7fa2f787e740 4 client.15873 mds_command: new command op to 34553 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-28T18:08:29.342 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:29.339+0000 7fa2f787e740 1 -- 172.21.15.22:0/3711535011 --> [v2:172.21.15.79:6838/2441542443,v1:172.21.15.79:6839/2441542443] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55d103031800 con 0x55d1030d96c0 2024-03-28T18:08:29.342 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:29.340+0000 7fa2ed400640 1 --2- 172.21.15.22:0/3711535011 >> [v2:172.21.15.79:6838/2441542443,v1:172.21.15.79:6839/2441542443] conn(0x55d1030d96c0 0x55d1030dbab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:08:29.342 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:29.340+0000 7fa2ed400640 1 --2- 172.21.15.22:0/3711535011 >> [v2:172.21.15.79:6838/2441542443,v1:172.21.15.79:6839/2441542443] conn(0x55d1030d96c0 0x55d1030dbab0 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=41a3af68bc1477a2 server_cookie=b5a7a3cf4b4b5001 in_seq=0 out_seq=0 2024-03-28T18:08:29.342 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:29.340+0000 7fa2cf400640 10 client.15873 ms_handle_connect on v2:172.21.15.79:6838/2441542443 2024-03-28T18:08:29.343 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:29.341+0000 7fa2cf400640 1 -- 172.21.15.22:0/3711535011 <== mds.0 v2:172.21.15.79:6838/2441542443 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x55d103031800 con 0x55d1030d96c0 2024-03-28T18:08:29.344 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:29.341+0000 7fa2cf400640 10 client.15873 handle_command_reply: tid=0 2024-03-28T18:08:29.344 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:29.342+0000 7fa2f787e740 1 client.15873 shutdown 2024-03-28T18:08:29.344 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:29.342+0000 7fa2f787e740 1 -- 172.21.15.22:0/3711535011 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d1030c06d0 msgr2=0x55d1030c9580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:08:29.344 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:29.342+0000 7fa2f787e740 1 --2- 172.21.15.22:0/3711535011 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d1030c06d0 0x55d1030c9580 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7fa2e00bd290 tx=0x7fa2e00035f0 comp rx=0 tx=0).stop 2024-03-28T18:08:29.344 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:29.342+0000 7fa2f787e740 1 -- 172.21.15.22:0/3711535011 shutdown_connections 2024-03-28T18:08:29.344 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:29.342+0000 7fa2f787e740 1 --2- 172.21.15.22:0/3711535011 >> [v2:172.21.15.79:6838/2441542443,v1:172.21.15.79:6839/2441542443] conn(0x55d1030d96c0 0x55d1030dbab0 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:08:29.344 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:29.342+0000 7fa2f787e740 1 --2- 172.21.15.22:0/3711535011 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55d1030c9c00 0x55d1030c6f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:08:29.344 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:29.342+0000 7fa2f787e740 1 --2- 172.21.15.22:0/3711535011 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d1030c1080 0x55d1030c6910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:08:29.344 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:29.342+0000 7fa2f787e740 1 --2- 172.21.15.22:0/3711535011 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d1030c06d0 0x55d1030c9580 unknown :-1 s=CLOSED pgs=543 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:08:29.344 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:29.342+0000 7fa2cf400640 0 client.15873 ms_handle_reset on v2:172.21.15.79:6838/2441542443 2024-03-28T18:08:29.344 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:29.342+0000 7fa2f787e740 1 -- 172.21.15.22:0/3711535011 >> 172.21.15.22:0/3711535011 conn(0x55d1030bada0 msgr2=0x55d1030bcaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:08:29.344 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:29.342+0000 7fa2f787e740 1 -- 172.21.15.22:0/3711535011 shutdown_connections 2024-03-28T18:08:29.344 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:29.342+0000 7fa2f787e740 1 -- 172.21.15.22:0/3711535011 wait complete. 2024-03-28T18:08:29.344 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:29.342+0000 7fa2f787e740 20 client.15873 trim_cache size 0 max 16384 2024-03-28T18:08:29.356 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:29.355+0000 7fa2f787e740 1 Processor -- start 2024-03-28T18:08:29.356 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:29.356+0000 7fa2f787e740 1 -- start start 2024-03-28T18:08:29.356 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:29.356+0000 7fa2f787e740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d1030c06d0 0x55d1030ba0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:08:29.356 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:29.356+0000 7fa2f787e740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d1030c1080 0x55d1030c9040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:08:29.357 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:29.356+0000 7fa2f787e740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55d1030c9c00 0x55d1030c9580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:08:29.357 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:29.356+0000 7fa2f787e740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2f01af190 con 0x55d1030c1080 2024-03-28T18:08:29.357 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:29.356+0000 7fa2f787e740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2f01b0b10 con 0x55d1030c9c00 2024-03-28T18:08:29.357 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:29.356+0000 7fa2f787e740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2f01af7f0 con 0x55d1030c06d0 2024-03-28T18:08:29.357 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:29.356+0000 7fa2ed400640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d1030c1080 0x55d1030c9040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:08:29.357 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:29.356+0000 7fa2ed400640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d1030c1080 0x55d1030c9040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:38096/0 (socket says 172.21.15.22:38096) 2024-03-28T18:08:29.357 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:29.356+0000 7fa2ed400640 1 -- 172.21.15.22:0/4030517238 learned_addr learned my addr 172.21.15.22:0/4030517238 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-28T18:08:29.357 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:29.356+0000 7fa2ee800640 1 --2- 172.21.15.22:0/4030517238 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55d1030c9c00 0x55d1030c9580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:08:29.357 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:29.356+0000 7fa2ede00640 1 --2- 172.21.15.22:0/4030517238 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d1030c06d0 0x55d1030ba0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:08:29.357 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:29.357+0000 7fa2ed400640 1 -- 172.21.15.22:0/4030517238 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d1030c06d0 msgr2=0x55d1030ba0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T18:08:29.357 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:29.357+0000 7fa2ed400640 1 --2- 172.21.15.22:0/4030517238 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d1030c06d0 0x55d1030ba0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:08:29.357 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:29.357+0000 7fa2ed400640 1 -- 172.21.15.22:0/4030517238 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55d1030c9c00 msgr2=0x55d1030c9580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T18:08:29.357 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:29.357+0000 7fa2ed400640 1 --2- 172.21.15.22:0/4030517238 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55d1030c9c00 0x55d1030c9580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:08:29.357 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:29.357+0000 7fa2ed400640 1 -- 172.21.15.22:0/4030517238 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d1030c7290 con 0x55d1030c1080 2024-03-28T18:08:29.357 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:29.357+0000 7fa2ed400640 1 --2- 172.21.15.22:0/4030517238 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d1030c1080 0x55d1030c9040 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x55d1030c6170 tx=0x7fa2dc01ea50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e6c3e7d6f447c7fe server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:08:29.360 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:29.357+0000 7fa2da600640 1 -- 172.21.15.22:0/4030517238 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2dc004050 con 0x55d1030c1080 2024-03-28T18:08:29.360 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:29.357+0000 7fa2da600640 1 -- 172.21.15.22:0/4030517238 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7fa2dc0041f0 con 0x55d1030c1080 2024-03-28T18:08:29.360 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:29.357+0000 7fa2da600640 1 -- 172.21.15.22:0/4030517238 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2dc014850 con 0x55d1030c1080 2024-03-28T18:08:29.360 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:29.358+0000 7fa2f787e740 1 -- 172.21.15.22:0/4030517238 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d1030c1080 msgr2=0x55d1030c9040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:08:29.360 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:29.358+0000 7fa2f787e740 1 --2- 172.21.15.22:0/4030517238 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d1030c1080 0x55d1030c9040 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x55d1030c6170 tx=0x7fa2dc01ea50 comp rx=0 tx=0).stop 2024-03-28T18:08:29.360 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:29.358+0000 7fa2f787e740 1 -- 172.21.15.22:0/4030517238 shutdown_connections 2024-03-28T18:08:29.361 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:29.358+0000 7fa2f787e740 1 --2- 172.21.15.22:0/4030517238 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55d1030c9c00 0x55d1030c9580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:08:29.361 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:29.358+0000 7fa2f787e740 1 --2- 172.21.15.22:0/4030517238 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d1030c1080 0x55d1030c9040 unknown :-1 s=CLOSED pgs=544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:08:29.361 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:29.358+0000 7fa2f787e740 1 --2- 172.21.15.22:0/4030517238 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d1030c06d0 0x55d1030ba0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:08:29.361 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:29.358+0000 7fa2f787e740 1 -- 172.21.15.22:0/4030517238 >> 172.21.15.22:0/4030517238 conn(0x55d1030bada0 msgr2=0x55d1030c1d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:08:29.361 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:29.358+0000 7fa2f787e740 1 -- 172.21.15.22:0/4030517238 shutdown_connections 2024-03-28T18:08:29.361 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:29.358+0000 7fa2f787e740 1 -- 172.21.15.22:0/4030517238 wait complete. 2024-03-28T18:08:29.361 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:29.358+0000 7fa2f787e740 1 -- start start 2024-03-28T18:08:29.361 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:29.359+0000 7fa2f787e740 1 Processor -- start 2024-03-28T18:08:29.361 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:29.359+0000 7fa2f787e740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55d1030c06d0 0x55d1030d16e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:08:29.361 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:29.359+0000 7fa2f787e740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d1030c1080 0x55d1030cfa80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:08:29.361 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:29.359+0000 7fa2f787e740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d1030c9c00 0x55d1030cffc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:08:29.361 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:29.359+0000 7fa2f787e740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2f01b04b0 con 0x55d1030c9c00 2024-03-28T18:08:29.361 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:29.359+0000 7fa2f787e740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2f01afe50 con 0x55d1030c06d0 2024-03-28T18:08:29.361 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:29.359+0000 7fa2f787e740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2f01ad350 con 0x55d1030c1080 2024-03-28T18:08:29.361 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:29.359+0000 7fa2f787e740 10 client.0 _ll_register_callbacks cb 0x7fa2e00f4da0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-28T18:08:29.362 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:29.359+0000 7fa2ee800640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d1030c9c00 0x55d1030cffc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:08:29.362 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:29.359+0000 7fa2ee800640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d1030c9c00 0x55d1030cffc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:38112/0 (socket says 172.21.15.22:38112) 2024-03-28T18:08:29.362 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:29.359+0000 7fa2ee800640 1 -- 172.21.15.22:0/449262460 learned_addr learned my addr 172.21.15.22:0/449262460 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-28T18:08:29.362 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:29.359+0000 7fa2ede00640 1 --2- 172.21.15.22:0/449262460 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55d1030c06d0 0x55d1030d16e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:08:29.362 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:29.360+0000 7fa2ed400640 1 --2- 172.21.15.22:0/449262460 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d1030c1080 0x55d1030cfa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:08:29.362 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:29.360+0000 7fa2ede00640 1 -- 172.21.15.22:0/449262460 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d1030c1080 msgr2=0x55d1030cfa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T18:08:29.362 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:29.360+0000 7fa2ede00640 1 --2- 172.21.15.22:0/449262460 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d1030c1080 0x55d1030cfa80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:08:29.362 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:29.360+0000 7fa2ede00640 1 -- 172.21.15.22:0/449262460 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d1030c9c00 msgr2=0x55d1030cffc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T18:08:29.362 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:29.360+0000 7fa2ede00640 1 --2- 172.21.15.22:0/449262460 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d1030c9c00 0x55d1030cffc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:08:29.362 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:29.360+0000 7fa2ede00640 1 -- 172.21.15.22:0/449262460 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55d1030d0500 con 0x55d1030c06d0 2024-03-28T18:08:29.362 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:29.360+0000 7fa2ee800640 1 --2- 172.21.15.22:0/449262460 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d1030c9c00 0x55d1030cffc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-28T18:08:29.362 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:29.360+0000 7fa2ede00640 1 --2- 172.21.15.22:0/449262460 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55d1030c06d0 0x55d1030d16e0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fa2e0005ec0 tx=0x7fa2e00f1230 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9492768be5024c69 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:08:29.362 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:29.360+0000 7fa2cf400640 10 client.0 ms_handle_connect on v2:172.21.15.79:3300/0 2024-03-28T18:08:29.362 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:29.360+0000 7fa2cf400640 1 -- 172.21.15.22:0/449262460 <== mon.2 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2e00bd0d0 con 0x55d1030c06d0 2024-03-28T18:08:29.362 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:29.360+0000 7fa2cf400640 1 -- 172.21.15.22:0/449262460 <== mon.2 v2:172.21.15.79:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 8411+0+0 (secure 0 0 0) 0x7fa2e00f4110 con 0x55d1030c06d0 2024-03-28T18:08:29.363 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:29.360+0000 7fa2f787e740 1 -- 172.21.15.22:0/449262460 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fa2dc0041f0 con 0x55d1030c06d0 2024-03-28T18:08:29.363 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:29.361+0000 7fa2cf400640 1 -- 172.21.15.22:0/449262460 <== mon.2 v2:172.21.15.79:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa2e00bd360 con 0x55d1030c06d0 2024-03-28T18:08:29.363 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:29.361+0000 7fa2f787e740 10 client.35708 fetch_fsmap learned FSMap version 27 2024-03-28T18:08:29.363 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:29.361+0000 7fa2f787e740 1 -- 172.21.15.22:0/449262460 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55d1030d0500 con 0x55d1030c06d0 2024-03-28T18:08:29.364 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:29.361+0000 7fa2cf400640 1 -- 172.21.15.22:0/449262460 <== mon.2 v2:172.21.15.79:3300/0 4 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7fa2dc0041f0 con 0x55d1030c06d0 2024-03-28T18:08:29.364 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:29.361+0000 7fa2cf400640 1 -- 172.21.15.22:0/449262460 <== mon.2 v2:172.21.15.79:3300/0 5 ==== fsmap(e 27) v1 ==== 7934+0+0 (secure 0 0 0) 0x7fa2e00c58a0 con 0x55d1030c06d0 2024-03-28T18:08:29.364 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:29.361+0000 7fa2f787e740 10 client.35708 fetch_fsmap finished waiting for FSMap version 27 2024-03-28T18:08:29.365 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:29.361+0000 7fa2f787e740 10 client.35708 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-28T18:08:29.365 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:29.361+0000 7fa2f787e740 20 client.35708 populate_metadata read hostname 'smithi022' 2024-03-28T18:08:29.365 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:29.361+0000 7fa2f787e740 1 --2- 172.21.15.22:0/449262460 >> [v2:172.21.15.79:6838/2441542443,v1:172.21.15.79:6839/2441542443] conn(0x55d103135d70 0x55d103138190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:08:29.365 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:29.361+0000 7fa2f787e740 4 client.35708 mds_command: new command op to 34553 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-28T18:08:29.365 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:29.361+0000 7fa2f787e740 1 -- 172.21.15.22:0/449262460 --> [v2:172.21.15.79:6838/2441542443,v1:172.21.15.79:6839/2441542443] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55d10302bb50 con 0x55d103135d70 2024-03-28T18:08:29.365 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:29.362+0000 7fa2ed400640 1 --2- 172.21.15.22:0/449262460 >> [v2:172.21.15.79:6838/2441542443,v1:172.21.15.79:6839/2441542443] conn(0x55d103135d70 0x55d103138190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:08:29.365 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:29.362+0000 7fa2ed400640 1 --2- 172.21.15.22:0/449262460 >> [v2:172.21.15.79:6838/2441542443,v1:172.21.15.79:6839/2441542443] conn(0x55d103135d70 0x55d103138190 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=98d5417e4e2b1261 server_cookie=129ede202d96ea00 in_seq=0 out_seq=0 2024-03-28T18:08:29.365 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:29.362+0000 7fa2cf400640 10 client.35708 ms_handle_connect on v2:172.21.15.79:6838/2441542443 2024-03-28T18:08:29.365 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:29.363+0000 7fa2cf400640 1 -- 172.21.15.22:0/449262460 <== mds.0 v2:172.21.15.79:6838/2441542443 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55d10302bb50 con 0x55d103135d70 2024-03-28T18:08:29.365 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:29.363+0000 7fa2cf400640 10 client.35708 handle_command_reply: tid=0 2024-03-28T18:08:29.365 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:29.363+0000 7fa2f787e740 1 client.35708 shutdown 2024-03-28T18:08:29.365 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:29.363+0000 7fa2f787e740 1 -- 172.21.15.22:0/449262460 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55d1030c06d0 msgr2=0x55d1030d16e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:08:29.365 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:29.363+0000 7fa2f787e740 1 --2- 172.21.15.22:0/449262460 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55d1030c06d0 0x55d1030d16e0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fa2e0005ec0 tx=0x7fa2e00f1230 comp rx=0 tx=0).stop 2024-03-28T18:08:29.365 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:29.363+0000 7fa2f787e740 1 -- 172.21.15.22:0/449262460 shutdown_connections 2024-03-28T18:08:29.365 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:29.363+0000 7fa2f787e740 1 --2- 172.21.15.22:0/449262460 >> [v2:172.21.15.79:6838/2441542443,v1:172.21.15.79:6839/2441542443] conn(0x55d103135d70 0x55d103138190 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:08:29.365 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:29.363+0000 7fa2f787e740 1 --2- 172.21.15.22:0/449262460 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d1030c9c00 0x55d1030cffc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:08:29.366 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:29.363+0000 7fa2f787e740 1 --2- 172.21.15.22:0/449262460 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d1030c1080 0x55d1030cfa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:08:29.366 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:29.363+0000 7fa2cf400640 0 client.35708 ms_handle_reset on v2:172.21.15.79:6838/2441542443 2024-03-28T18:08:29.366 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:29.363+0000 7fa2f787e740 1 --2- 172.21.15.22:0/449262460 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55d1030c06d0 0x55d1030d16e0 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:08:29.366 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:29.363+0000 7fa2f787e740 1 -- 172.21.15.22:0/449262460 >> 172.21.15.22:0/449262460 conn(0x55d1030bada0 msgr2=0x55d1030e0190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:08:29.366 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:29.363+0000 7fa2f787e740 1 -- 172.21.15.22:0/449262460 shutdown_connections 2024-03-28T18:08:29.366 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:29.363+0000 7fa2f787e740 1 -- 172.21.15.22:0/449262460 wait complete. 2024-03-28T18:08:29.366 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:29.363+0000 7fa2f787e740 20 client.35708 trim_cache size 0 max 16384 2024-03-28T18:08:29.366 INFO:teuthology.orchestra.run.smithi022.stdout:[] 2024-03-28T18:08:29.367 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:29.364+0000 7fa2f4c00640 1 -- 172.21.15.22:0/389321241 >> [v2:172.21.15.22:6800/1357365082,v1:172.21.15.22:6801/1357365082] conn(0x7fa2d40a2b10 msgr2=0x7fa2d40a4fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:08:29.367 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:29.364+0000 7fa2f4c00640 1 --2- 172.21.15.22:0/389321241 >> [v2:172.21.15.22:6800/1357365082,v1:172.21.15.22:6801/1357365082] conn(0x7fa2d40a2b10 0x7fa2d40a4fd0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fa2e8005fd0 tx=0x7fa2e8009600 comp rx=0 tx=0).stop 2024-03-28T18:08:29.367 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:29.364+0000 7fa2f4c00640 1 -- 172.21.15.22:0/389321241 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa2f0103840 msgr2=0x7fa2f019a160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:08:29.367 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:29.364+0000 7fa2f4c00640 1 --2- 172.21.15.22:0/389321241 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa2f0103840 0x7fa2f019a160 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7fa2e0000c00 tx=0x7fa2e00045e0 comp rx=0 tx=0).stop 2024-03-28T18:08:29.367 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:29.364+0000 7fa2f4c00640 1 -- 172.21.15.22:0/389321241 shutdown_connections 2024-03-28T18:08:29.367 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:29.364+0000 7fa2f4c00640 1 --2- 172.21.15.22:0/389321241 >> [v2:172.21.15.22:6800/1357365082,v1:172.21.15.22:6801/1357365082] conn(0x7fa2d40a2b10 0x7fa2d40a4fd0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:08:29.367 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:29.364+0000 7fa2f4c00640 1 --2- 172.21.15.22:0/389321241 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa2f0109850 0x7fa2f019ea20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:08:29.367 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:29.364+0000 7fa2f4c00640 1 --2- 172.21.15.22:0/389321241 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa2f0104200 0x7fa2f019a6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:08:29.367 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:29.364+0000 7fa2f4c00640 1 --2- 172.21.15.22:0/389321241 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa2f0103840 0x7fa2f019a160 unknown :-1 s=CLOSED pgs=541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:08:29.368 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:29.364+0000 7fa2f4c00640 1 -- 172.21.15.22:0/389321241 >> 172.21.15.22:0/389321241 conn(0x7fa2f00ff350 msgr2=0x7fa2f00ffaa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:08:29.368 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:29.364+0000 7fa2f4c00640 1 -- 172.21.15.22:0/389321241 shutdown_connections 2024-03-28T18:08:29.368 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:29.364+0000 7fa2f4c00640 1 -- 172.21.15.22:0/389321241 wait complete. 2024-03-28T18:08:30.380 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-03-28T18:08:30.380 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-28T18:08:30.380 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-03-28T18:08:30.424 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:08:30 smithi134 ceph-mon[35037]: pgmap v31: 97 pgs: 97 active+clean; 1.3 MiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-28T18:08:30.432 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:08:30 smithi079 ceph-mon[35312]: pgmap v31: 97 pgs: 97 active+clean; 1.3 MiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-28T18:08:30.454 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:08:30 smithi022 ceph-mon[31985]: pgmap v31: 97 pgs: 97 active+clean; 1.3 MiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-28T18:08:30.623 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:30.623+0000 7f14e7200640 1 -- 172.21.15.22:0/2124836925 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f14e810ddd0 msgr2=0x7f14e81101c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:08:30.623 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:30.623+0000 7f14e7200640 1 --2- 172.21.15.22:0/2124836925 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f14e810ddd0 0x7f14e81101c0 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7f14e0009ca0 tx=0x7f14e002c190 comp rx=0 tx=0).stop 2024-03-28T18:08:30.623 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:30.623+0000 7f14e7200640 1 -- 172.21.15.22:0/2124836925 shutdown_connections 2024-03-28T18:08:30.623 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:30.623+0000 7f14e7200640 1 --2- 172.21.15.22:0/2124836925 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f14e810ddd0 0x7f14e81101c0 unknown :-1 s=CLOSED pgs=545 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:08:30.623 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:30.623+0000 7f14e7200640 1 --2- 172.21.15.22:0/2124836925 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f14e8074370 0x7f14e810d890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:08:30.624 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:30.623+0000 7f14e7200640 1 --2- 172.21.15.22:0/2124836925 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f14e8073a50 0x7f14e8073e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:08:30.624 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:30.623+0000 7f14e7200640 1 -- 172.21.15.22:0/2124836925 >> 172.21.15.22:0/2124836925 conn(0x7f14e80fd040 msgr2=0x7f14e80ff460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:08:30.624 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:30.623+0000 7f14e7200640 1 -- 172.21.15.22:0/2124836925 shutdown_connections 2024-03-28T18:08:30.624 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:30.623+0000 7f14e7200640 1 -- 172.21.15.22:0/2124836925 wait complete. 2024-03-28T18:08:30.625 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:30.624+0000 7f14e7200640 1 Processor -- start 2024-03-28T18:08:30.626 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:30.624+0000 7f14e7200640 1 -- start start 2024-03-28T18:08:30.626 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:30.625+0000 7f14e7200640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f14e8073a50 0x7f14e819a040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:08:30.626 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:30.625+0000 7f14e7200640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f14e8074370 0x7f14e819a580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:08:30.626 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:30.625+0000 7f14e7200640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f14e810ddd0 0x7f14e819e900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:08:30.626 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:30.625+0000 7f14e7200640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14e8113ac0 con 0x7f14e810ddd0 2024-03-28T18:08:30.626 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:30.625+0000 7f14e7200640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14e8113940 con 0x7f14e8073a50 2024-03-28T18:08:30.626 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:30.625+0000 7f14e7200640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14e8113c40 con 0x7f14e8074370 2024-03-28T18:08:30.626 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:30.625+0000 7f14e6800640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f14e810ddd0 0x7f14e819e900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:08:30.626 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:30.625+0000 7f14e5e00640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f14e8073a50 0x7f14e819a040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:08:30.626 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:30.625+0000 7f14e6800640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f14e810ddd0 0x7f14e819e900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:38124/0 (socket says 172.21.15.22:38124) 2024-03-28T18:08:30.626 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:30.625+0000 7f14e6800640 1 -- 172.21.15.22:0/2131267614 learned_addr learned my addr 172.21.15.22:0/2131267614 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-28T18:08:30.626 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:30.625+0000 7f14e5400640 1 --2- 172.21.15.22:0/2131267614 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f14e8074370 0x7f14e819a580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:08:30.626 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:30.626+0000 7f14e6800640 1 -- 172.21.15.22:0/2131267614 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f14e8074370 msgr2=0x7f14e819a580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:08:30.626 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:30.626+0000 7f14e6800640 1 --2- 172.21.15.22:0/2131267614 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f14e8074370 0x7f14e819a580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:08:30.627 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:30.626+0000 7f14e6800640 1 -- 172.21.15.22:0/2131267614 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f14e8073a50 msgr2=0x7f14e819a040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:08:30.627 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:30.626+0000 7f14e6800640 1 --2- 172.21.15.22:0/2131267614 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f14e8073a50 0x7f14e819a040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:08:30.627 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:30.626+0000 7f14e6800640 1 -- 172.21.15.22:0/2131267614 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f14e819f030 con 0x7f14e810ddd0 2024-03-28T18:08:30.627 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:30.626+0000 7f14e6800640 1 --2- 172.21.15.22:0/2131267614 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f14e810ddd0 0x7f14e819e900 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7f14e0005cc0 tx=0x7f14e0034b40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:08:30.627 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:30.626+0000 7f14dea00640 1 -- 172.21.15.22:0/2131267614 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14e0039040 con 0x7f14e810ddd0 2024-03-28T18:08:30.628 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:30.626+0000 7f14dea00640 1 -- 172.21.15.22:0/2131267614 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f14e0003b70 con 0x7f14e810ddd0 2024-03-28T18:08:30.628 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:30.626+0000 7f14dea00640 1 -- 172.21.15.22:0/2131267614 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14e0032a10 con 0x7f14e810ddd0 2024-03-28T18:08:30.628 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:30.626+0000 7f14e7200640 1 -- 172.21.15.22:0/2131267614 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f14e819f2c0 con 0x7f14e810ddd0 2024-03-28T18:08:30.628 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:30.627+0000 7f14e7200640 1 -- 172.21.15.22:0/2131267614 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f14e81a6bb0 con 0x7f14e810ddd0 2024-03-28T18:08:30.630 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:30.630+0000 7f14efdfc740 1 Processor -- start 2024-03-28T18:08:30.631 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:30.632+0000 7f14dea00640 1 -- 172.21.15.22:0/2131267614 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 25) v1 ==== 137523+0+0 (secure 0 0 0) 0x7f14e00045a0 con 0x7f14e810ddd0 2024-03-28T18:08:30.631 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:30.632+0000 7f14efdfc740 1 -- start start 2024-03-28T18:08:30.632 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:30.632+0000 7f14efdfc740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5588eeb46a80 0x5588eeb46e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:08:30.632 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:30.632+0000 7f14efdfc740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5588eeb473a0 0x5588eeb4fa40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:08:30.632 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:30.632+0000 7f14efdfc740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5588eeb4ff80 0x5588eeb52370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:08:30.633 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:30.632+0000 7f14efdfc740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x5588eeaa2eb0 con 0x5588eeb4ff80 2024-03-28T18:08:30.633 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:30.632+0000 7f14efdfc740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x5588eea4ba60 con 0x5588eeb473a0 2024-03-28T18:08:30.633 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:30.632+0000 7f14efdfc740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x5588eea9ef50 con 0x5588eeb46a80 2024-03-28T18:08:30.633 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:30.633+0000 7f14e5400640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5588eeb4ff80 0x5588eeb52370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:08:30.633 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:30.633+0000 7f14e5400640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5588eeb4ff80 0x5588eeb52370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:38136/0 (socket says 172.21.15.22:38136) 2024-03-28T18:08:30.633 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:30.633+0000 7f14e5400640 1 -- 172.21.15.22:0/2625265872 learned_addr learned my addr 172.21.15.22:0/2625265872 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-28T18:08:30.633 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:30.633+0000 7f14e5e00640 1 --2- 172.21.15.22:0/2625265872 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5588eeb473a0 0x5588eeb4fa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:08:30.633 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:30.633+0000 7f14e5400640 1 --2- 172.21.15.22:0/2625265872 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5588eeb46a80 0x5588eeb46e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:08:30.634 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:30.633+0000 7f14e5400640 1 -- 172.21.15.22:0/2625265872 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5588eeb46a80 msgr2=0x5588eeb46e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T18:08:30.634 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:30.633+0000 7f14e5400640 1 --2- 172.21.15.22:0/2625265872 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5588eeb46a80 0x5588eeb46e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:08:30.634 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:30.633+0000 7f14e5400640 1 -- 172.21.15.22:0/2625265872 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5588eeb473a0 msgr2=0x5588eeb4fa40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T18:08:30.634 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:30.633+0000 7f14e5400640 1 --2- 172.21.15.22:0/2625265872 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5588eeb473a0 0x5588eeb4fa40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:08:30.634 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:30.633+0000 7f14e5400640 1 -- 172.21.15.22:0/2625265872 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5588ee87f7e0 con 0x5588eeb4ff80 2024-03-28T18:08:30.634 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:30.634+0000 7f14e5400640 1 --2- 172.21.15.22:0/2625265872 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5588eeb4ff80 0x5588eeb52370 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7f14d40058c0 tx=0x7f14d4005850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=644b0c0764dabe24 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:08:30.635 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:30.634+0000 7f14dea00640 1 --2- 172.21.15.22:0/2131267614 >> [v2:172.21.15.22:6800/1357365082,v1:172.21.15.22:6801/1357365082] conn(0x7f14cc0a2a90 0x7f14cc0a4f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:08:30.635 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:30.634+0000 7f14dea00640 1 -- 172.21.15.22:0/2131267614 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 8411+0+0 (secure 0 0 0) 0x7f14e00f14c0 con 0x7f14e810ddd0 2024-03-28T18:08:30.635 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:30.634+0000 7f14e5e00640 1 --2- 172.21.15.22:0/2131267614 >> [v2:172.21.15.22:6800/1357365082,v1:172.21.15.22:6801/1357365082] conn(0x7f14cc0a2a90 0x7f14cc0a4f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:08:30.635 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:30.634+0000 7f14d3400640 1 -- 172.21.15.22:0/2625265872 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14d400f7c0 con 0x5588eeb4ff80 2024-03-28T18:08:30.635 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:30.634+0000 7f14d3400640 1 -- 172.21.15.22:0/2625265872 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f14d400fe30 con 0x5588eeb4ff80 2024-03-28T18:08:30.635 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:30.634+0000 7f14d3400640 1 -- 172.21.15.22:0/2625265872 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14d400e3b0 con 0x5588eeb4ff80 2024-03-28T18:08:30.636 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:30.635+0000 7f14efdfc740 1 -- 172.21.15.22:0/2625265872 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5588eeb4ff80 msgr2=0x5588eeb52370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:08:30.636 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:30.635+0000 7f14efdfc740 1 --2- 172.21.15.22:0/2625265872 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5588eeb4ff80 0x5588eeb52370 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7f14d40058c0 tx=0x7f14d4005850 comp rx=0 tx=0).stop 2024-03-28T18:08:30.636 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:30.635+0000 7f14e5e00640 1 --2- 172.21.15.22:0/2131267614 >> [v2:172.21.15.22:6800/1357365082,v1:172.21.15.22:6801/1357365082] conn(0x7f14cc0a2a90 0x7f14cc0a4f50 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x5588eeb46550 tx=0x7f14d8005f90 comp rx=0 tx=0).ready entity=mgr.15522 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:08:30.636 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:30.635+0000 7f14efdfc740 1 -- 172.21.15.22:0/2625265872 shutdown_connections 2024-03-28T18:08:30.636 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:30.635+0000 7f14efdfc740 1 --2- 172.21.15.22:0/2625265872 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5588eeb4ff80 0x5588eeb52370 unknown :-1 s=CLOSED pgs=547 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:08:30.636 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:30.635+0000 7f14efdfc740 1 --2- 172.21.15.22:0/2625265872 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5588eeb473a0 0x5588eeb4fa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:08:30.636 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:30.635+0000 7f14efdfc740 1 --2- 172.21.15.22:0/2625265872 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5588eeb46a80 0x5588eeb46e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:08:30.636 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:30.635+0000 7f14efdfc740 1 -- 172.21.15.22:0/2625265872 >> 172.21.15.22:0/2625265872 conn(0x5588eeb411b0 msgr2=0x5588eeb41590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:08:30.637 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:30.635+0000 7f14efdfc740 1 -- 172.21.15.22:0/2625265872 shutdown_connections 2024-03-28T18:08:30.637 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:30.635+0000 7f14efdfc740 1 -- 172.21.15.22:0/2625265872 wait complete. 2024-03-28T18:08:30.637 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:30.636+0000 7f14efdfc740 1 -- start start 2024-03-28T18:08:30.637 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:30.636+0000 7f14efdfc740 1 Processor -- start 2024-03-28T18:08:30.638 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:30.637+0000 7f14efdfc740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5588eeb46a80 0x5588eeb4f910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:08:30.638 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:30.637+0000 7f14efdfc740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5588eeb473a0 0x5588eeb4cca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:08:30.638 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:30.637+0000 7f14efdfc740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5588eeb4ff80 0x5588eeb4d220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:08:30.638 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:30.637+0000 7f14efdfc740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x5588eeb52d70 con 0x5588eeb473a0 2024-03-28T18:08:30.638 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:30.637+0000 7f14efdfc740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x5588eea5eeb0 con 0x5588eeb46a80 2024-03-28T18:08:30.638 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:30.637+0000 7f14efdfc740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x5588eeb52ef0 con 0x5588eeb4ff80 2024-03-28T18:08:30.638 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:30.637+0000 7f14efdfc740 10 client.0 _ll_register_callbacks cb 0x5588eea53150 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-28T18:08:30.638 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:30.638+0000 7f14e5e00640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5588eeb46a80 0x5588eeb4f910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:08:30.639 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:30.638+0000 7f14e5e00640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5588eeb46a80 0x5588eeb4f910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.22:47384/0 (socket says 172.21.15.22:47384) 2024-03-28T18:08:30.639 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:30.638+0000 7f14e5e00640 1 -- 172.21.15.22:0/1655597374 learned_addr learned my addr 172.21.15.22:0/1655597374 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-28T18:08:30.639 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:30.638+0000 7f14e5400640 1 --2- 172.21.15.22:0/1655597374 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5588eeb473a0 0x5588eeb4cca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:08:30.639 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:30.638+0000 7f14e6800640 1 --2- 172.21.15.22:0/1655597374 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5588eeb4ff80 0x5588eeb4d220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:08:30.639 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:30.638+0000 7f14e5e00640 1 -- 172.21.15.22:0/1655597374 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5588eeb4ff80 msgr2=0x5588eeb4d220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T18:08:30.639 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:30.638+0000 7f14e5e00640 1 --2- 172.21.15.22:0/1655597374 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5588eeb4ff80 0x5588eeb4d220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:08:30.639 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:30.638+0000 7f14e5e00640 1 -- 172.21.15.22:0/1655597374 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5588eeb473a0 msgr2=0x5588eeb4cca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T18:08:30.639 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:30.638+0000 7f14e5e00640 1 --2- 172.21.15.22:0/1655597374 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5588eeb473a0 0x5588eeb4cca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:08:30.639 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:30.638+0000 7f14e5e00640 1 -- 172.21.15.22:0/1655597374 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5588ee8dff70 con 0x5588eeb46a80 2024-03-28T18:08:30.639 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:30.638+0000 7f14e6800640 1 --2- 172.21.15.22:0/1655597374 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5588eeb4ff80 0x5588eeb4d220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-28T18:08:30.641 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:30.639+0000 7f14e5e00640 1 --2- 172.21.15.22:0/1655597374 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5588eeb46a80 0x5588eeb4f910 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f14d8009730 tx=0x7f14d8004730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=655421eb57aebf1c server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:08:30.641 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:30.639+0000 7f14c7e00640 10 client.0 ms_handle_connect on v2:172.21.15.79:3300/0 2024-03-28T18:08:30.641 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:30.639+0000 7f14c7e00640 1 -- 172.21.15.22:0/1655597374 <== mon.2 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14d801f640 con 0x5588eeb46a80 2024-03-28T18:08:30.641 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:30.639+0000 7f14efdfc740 1 -- 172.21.15.22:0/1655597374 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f14d400fe30 con 0x5588eeb46a80 2024-03-28T18:08:30.641 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:30.639+0000 7f14c7e00640 1 -- 172.21.15.22:0/1655597374 <== mon.2 v2:172.21.15.79:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 8411+0+0 (secure 0 0 0) 0x7f14d801f7e0 con 0x5588eeb46a80 2024-03-28T18:08:30.641 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:30.640+0000 7f14c7e00640 1 -- 172.21.15.22:0/1655597374 <== mon.2 v2:172.21.15.79:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f14d801f9e0 con 0x5588eeb46a80 2024-03-28T18:08:30.641 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:30.640+0000 7f14efdfc740 10 client.35720 fetch_fsmap learned FSMap version 27 2024-03-28T18:08:30.641 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:30.640+0000 7f14efdfc740 1 -- 172.21.15.22:0/1655597374 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x5588ee777170 con 0x5588eeb46a80 2024-03-28T18:08:30.641 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:30.640+0000 7f14c7e00640 1 -- 172.21.15.22:0/1655597374 <== mon.2 v2:172.21.15.79:3300/0 4 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f14d400fe30 con 0x5588eeb46a80 2024-03-28T18:08:30.641 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:30.640+0000 7f14c7e00640 1 -- 172.21.15.22:0/1655597374 <== mon.2 v2:172.21.15.79:3300/0 5 ==== fsmap(e 27) v1 ==== 7934+0+0 (secure 0 0 0) 0x7f14d8027070 con 0x5588eeb46a80 2024-03-28T18:08:30.641 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:30.640+0000 7f14efdfc740 10 client.35720 fetch_fsmap finished waiting for FSMap version 27 2024-03-28T18:08:30.641 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:30.640+0000 7f14efdfc740 10 client.35720 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-28T18:08:30.641 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:30.641+0000 7f14efdfc740 20 client.35720 populate_metadata read hostname 'smithi022' 2024-03-28T18:08:30.642 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:30.641+0000 7f14efdfc740 1 --2- 172.21.15.22:0/1655597374 >> [v2:172.21.15.79:6838/2441542443,v1:172.21.15.79:6839/2441542443] conn(0x5588eeb60070 0x5588eeb624b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:08:30.642 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:30.641+0000 7f14efdfc740 4 client.35720 mds_command: new command op to 34553 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-28T18:08:30.642 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:30.641+0000 7f14efdfc740 1 -- 172.21.15.22:0/1655597374 --> [v2:172.21.15.79:6838/2441542443,v1:172.21.15.79:6839/2441542443] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5588eeab03f0 con 0x5588eeb60070 2024-03-28T18:08:30.642 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:30.641+0000 7f14e5400640 1 --2- 172.21.15.22:0/1655597374 >> [v2:172.21.15.79:6838/2441542443,v1:172.21.15.79:6839/2441542443] conn(0x5588eeb60070 0x5588eeb624b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:08:30.642 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:30.642+0000 7f14e5400640 1 --2- 172.21.15.22:0/1655597374 >> [v2:172.21.15.79:6838/2441542443,v1:172.21.15.79:6839/2441542443] conn(0x5588eeb60070 0x5588eeb624b0 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ab1047fa478c78ec server_cookie=f59a297de1cf467c in_seq=0 out_seq=0 2024-03-28T18:08:30.642 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:30.642+0000 7f14c7e00640 10 client.35720 ms_handle_connect on v2:172.21.15.79:6838/2441542443 2024-03-28T18:08:30.643 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:30.643+0000 7f14c7e00640 1 -- 172.21.15.22:0/1655597374 <== mds.0 v2:172.21.15.79:6838/2441542443 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x5588eeab03f0 con 0x5588eeb60070 2024-03-28T18:08:30.644 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:30.643+0000 7f14c7e00640 10 client.35720 handle_command_reply: tid=0 2024-03-28T18:08:30.644 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:30.643+0000 7f14efdfc740 1 client.35720 shutdown 2024-03-28T18:08:30.644 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:30.644+0000 7f14efdfc740 1 -- 172.21.15.22:0/1655597374 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5588eeb46a80 msgr2=0x5588eeb4f910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:08:30.644 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:30.644+0000 7f14efdfc740 1 --2- 172.21.15.22:0/1655597374 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5588eeb46a80 0x5588eeb4f910 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f14d8009730 tx=0x7f14d8004730 comp rx=0 tx=0).stop 2024-03-28T18:08:30.644 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:30.644+0000 7f14efdfc740 1 -- 172.21.15.22:0/1655597374 shutdown_connections 2024-03-28T18:08:30.644 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:30.644+0000 7f14efdfc740 1 --2- 172.21.15.22:0/1655597374 >> [v2:172.21.15.79:6838/2441542443,v1:172.21.15.79:6839/2441542443] conn(0x5588eeb60070 0x5588eeb624b0 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:08:30.644 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:30.644+0000 7f14efdfc740 1 --2- 172.21.15.22:0/1655597374 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5588eeb4ff80 0x5588eeb4d220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:08:30.644 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:30.644+0000 7f14efdfc740 1 --2- 172.21.15.22:0/1655597374 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5588eeb473a0 0x5588eeb4cca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:08:30.644 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:30.644+0000 7f14efdfc740 1 --2- 172.21.15.22:0/1655597374 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5588eeb46a80 0x5588eeb4f910 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:08:30.644 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:30.644+0000 7f14efdfc740 1 -- 172.21.15.22:0/1655597374 >> 172.21.15.22:0/1655597374 conn(0x5588eeb411b0 msgr2=0x5588eeb42f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:08:30.644 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:30.644+0000 7f14c7e00640 0 client.35720 ms_handle_reset on v2:172.21.15.79:6838/2441542443 2024-03-28T18:08:30.644 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:30.644+0000 7f14efdfc740 1 -- 172.21.15.22:0/1655597374 shutdown_connections 2024-03-28T18:08:30.644 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:30.644+0000 7f14efdfc740 1 -- 172.21.15.22:0/1655597374 wait complete. 2024-03-28T18:08:30.644 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:30.644+0000 7f14efdfc740 20 client.35720 trim_cache size 0 max 16384 2024-03-28T18:08:30.670 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:30.670+0000 7f14efdfc740 1 Processor -- start 2024-03-28T18:08:30.670 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:30.671+0000 7f14efdfc740 1 -- start start 2024-03-28T18:08:30.671 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:30.671+0000 7f14efdfc740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5588eeb46a80 0x5588eeb4f3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:08:30.671 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:30.671+0000 7f14efdfc740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5588eeb473a0 0x5588eeb4f910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:08:30.672 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:30.671+0000 7f14efdfc740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5588eeb4ff80 0x5588eeb5b680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:08:30.672 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:30.671+0000 7f14efdfc740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14e81aac20 con 0x5588eeb473a0 2024-03-28T18:08:30.672 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:30.671+0000 7f14efdfc740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14e81ac5a0 con 0x5588eeb46a80 2024-03-28T18:08:30.672 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:30.671+0000 7f14efdfc740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14e81ab280 con 0x5588eeb4ff80 2024-03-28T18:08:30.672 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:30.671+0000 7f14e5400640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5588eeb473a0 0x5588eeb4f910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:08:30.672 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:30.671+0000 7f14e5400640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5588eeb473a0 0x5588eeb4f910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:38154/0 (socket says 172.21.15.22:38154) 2024-03-28T18:08:30.672 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:30.671+0000 7f14e5400640 1 -- 172.21.15.22:0/1493167476 learned_addr learned my addr 172.21.15.22:0/1493167476 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-28T18:08:30.672 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:30.671+0000 7f14e5e00640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5588eeb46a80 0x5588eeb4f3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:08:30.672 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:30.672+0000 7f14e6800640 1 --2- 172.21.15.22:0/1493167476 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5588eeb4ff80 0x5588eeb5b680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:08:30.672 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:30.672+0000 7f14e5400640 1 -- 172.21.15.22:0/1493167476 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5588eeb4ff80 msgr2=0x5588eeb5b680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T18:08:30.672 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:30.672+0000 7f14e5400640 1 --2- 172.21.15.22:0/1493167476 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5588eeb4ff80 0x5588eeb5b680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:08:30.672 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:30.672+0000 7f14e5400640 1 -- 172.21.15.22:0/1493167476 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5588eeb46a80 msgr2=0x5588eeb4f3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T18:08:30.672 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:30.672+0000 7f14e5400640 1 --2- 172.21.15.22:0/1493167476 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5588eeb46a80 0x5588eeb4f3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:08:30.672 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:30.672+0000 7f14e5400640 1 -- 172.21.15.22:0/1493167476 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5588eeb4b1f0 con 0x5588eeb473a0 2024-03-28T18:08:30.672 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:30.672+0000 7f14e5400640 1 --2- 172.21.15.22:0/1493167476 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5588eeb473a0 0x5588eeb4f910 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f14d4000c00 tx=0x7f14d4005fb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9f6f4351059e9925 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:08:30.674 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:30.672+0000 7f14d2a00640 1 -- 172.21.15.22:0/1493167476 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14d402d0d0 con 0x5588eeb473a0 2024-03-28T18:08:30.674 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:30.672+0000 7f14d2a00640 1 -- 172.21.15.22:0/1493167476 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f14d402d270 con 0x5588eeb473a0 2024-03-28T18:08:30.674 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:30.673+0000 7f14d2a00640 1 -- 172.21.15.22:0/1493167476 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14d400f790 con 0x5588eeb473a0 2024-03-28T18:08:30.674 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:30.673+0000 7f14efdfc740 1 -- 172.21.15.22:0/1493167476 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5588eeb473a0 msgr2=0x5588eeb4f910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:08:30.674 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:30.673+0000 7f14efdfc740 1 --2- 172.21.15.22:0/1493167476 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5588eeb473a0 0x5588eeb4f910 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f14d4000c00 tx=0x7f14d4005fb0 comp rx=0 tx=0).stop 2024-03-28T18:08:30.674 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:30.673+0000 7f14efdfc740 1 -- 172.21.15.22:0/1493167476 shutdown_connections 2024-03-28T18:08:30.674 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:30.673+0000 7f14efdfc740 1 --2- 172.21.15.22:0/1493167476 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5588eeb4ff80 0x5588eeb5b680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:08:30.674 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:30.673+0000 7f14efdfc740 1 --2- 172.21.15.22:0/1493167476 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5588eeb473a0 0x5588eeb4f910 unknown :-1 s=CLOSED pgs=548 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:08:30.674 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:30.674+0000 7f14efdfc740 1 --2- 172.21.15.22:0/1493167476 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5588eeb46a80 0x5588eeb4f3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:08:30.674 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:30.674+0000 7f14efdfc740 1 -- 172.21.15.22:0/1493167476 >> 172.21.15.22:0/1493167476 conn(0x5588eeb411b0 msgr2=0x5588eeb42e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:08:30.674 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:30.674+0000 7f14efdfc740 1 -- 172.21.15.22:0/1493167476 shutdown_connections 2024-03-28T18:08:30.674 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:30.674+0000 7f14efdfc740 1 -- 172.21.15.22:0/1493167476 wait complete. 2024-03-28T18:08:30.674 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:30.674+0000 7f14efdfc740 1 -- start start 2024-03-28T18:08:30.677 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:30.675+0000 7f14efdfc740 1 Processor -- start 2024-03-28T18:08:30.677 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:30.675+0000 7f14efdfc740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5588eeb46a80 0x5588eeb55380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:08:30.677 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:30.675+0000 7f14efdfc740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5588eeb473a0 0x5588eeb558c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:08:30.677 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:30.675+0000 7f14efdfc740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5588eeb4ff80 0x5588eebb5920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:08:30.677 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:30.675+0000 7f14efdfc740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14e81abf40 con 0x5588eeb473a0 2024-03-28T18:08:30.677 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:30.675+0000 7f14efdfc740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14e81ab8e0 con 0x5588eeb4ff80 2024-03-28T18:08:30.678 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:30.675+0000 7f14efdfc740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14e81a8e00 con 0x5588eeb46a80 2024-03-28T18:08:30.678 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:30.675+0000 7f14efdfc740 10 client.0 _ll_register_callbacks cb 0x7f14d8009ad0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-28T18:08:30.678 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:30.675+0000 7f14e5400640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5588eeb473a0 0x5588eeb558c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:08:30.678 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:30.676+0000 7f14e5400640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5588eeb473a0 0x5588eeb558c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:38158/0 (socket says 172.21.15.22:38158) 2024-03-28T18:08:30.678 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:30.676+0000 7f14e5400640 1 -- 172.21.15.22:0/3135422155 learned_addr learned my addr 172.21.15.22:0/3135422155 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-28T18:08:30.678 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:30.676+0000 7f14e6800640 1 --2- 172.21.15.22:0/3135422155 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5588eeb4ff80 0x5588eebb5920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:08:30.678 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:30.676+0000 7f14e5e00640 1 --2- 172.21.15.22:0/3135422155 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5588eeb46a80 0x5588eeb55380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:08:30.678 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:30.676+0000 7f14e5400640 1 -- 172.21.15.22:0/3135422155 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5588eeb46a80 msgr2=0x5588eeb55380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T18:08:30.678 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:30.676+0000 7f14e5400640 1 --2- 172.21.15.22:0/3135422155 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5588eeb46a80 0x5588eeb55380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:08:30.678 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:30.676+0000 7f14e5400640 1 -- 172.21.15.22:0/3135422155 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5588eeb4ff80 msgr2=0x5588eebb5920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T18:08:30.678 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:30.676+0000 7f14e5400640 1 --2- 172.21.15.22:0/3135422155 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5588eeb4ff80 0x5588eebb5920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:08:30.678 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:30.676+0000 7f14e5400640 1 -- 172.21.15.22:0/3135422155 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5588eebb5f90 con 0x5588eeb473a0 2024-03-28T18:08:30.678 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:30.676+0000 7f14e5400640 1 --2- 172.21.15.22:0/3135422155 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5588eeb473a0 0x5588eeb558c0 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7f14d4011770 tx=0x7f14d4009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4f07d611dad06ede server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:08:30.679 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:30.676+0000 7f14c7e00640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-03-28T18:08:30.679 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:30.677+0000 7f14c7e00640 1 -- 172.21.15.22:0/3135422155 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14d4005580 con 0x5588eeb473a0 2024-03-28T18:08:30.679 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:30.677+0000 7f14c7e00640 1 -- 172.21.15.22:0/3135422155 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 8411+0+0 (secure 0 0 0) 0x7f14d40050c0 con 0x5588eeb473a0 2024-03-28T18:08:30.679 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:30.677+0000 7f14efdfc740 1 -- 172.21.15.22:0/3135422155 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f14d402d270 con 0x5588eeb473a0 2024-03-28T18:08:30.679 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:30.677+0000 7f14c7e00640 1 -- 172.21.15.22:0/3135422155 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f14d403b050 con 0x5588eeb473a0 2024-03-28T18:08:30.679 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:30.677+0000 7f14efdfc740 10 client.15897 fetch_fsmap learned FSMap version 27 2024-03-28T18:08:30.679 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:30.677+0000 7f14efdfc740 1 -- 172.21.15.22:0/3135422155 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x5588eebbd420 con 0x5588eeb473a0 2024-03-28T18:08:30.679 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:30.677+0000 7f14c7e00640 1 -- 172.21.15.22:0/3135422155 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f14d402d270 con 0x5588eeb473a0 2024-03-28T18:08:30.679 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:30.677+0000 7f14c7e00640 1 -- 172.21.15.22:0/3135422155 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 27) v1 ==== 7934+0+0 (secure 0 0 0) 0x7f14d401d020 con 0x5588eeb473a0 2024-03-28T18:08:30.679 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:30.678+0000 7f14efdfc740 10 client.15897 fetch_fsmap finished waiting for FSMap version 27 2024-03-28T18:08:30.679 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:30.678+0000 7f14efdfc740 10 client.15897 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-28T18:08:30.679 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:30.678+0000 7f14efdfc740 20 client.15897 populate_metadata read hostname 'smithi022' 2024-03-28T18:08:30.679 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:30.678+0000 7f14efdfc740 1 --2- 172.21.15.22:0/3135422155 >> [v2:172.21.15.79:6838/2441542443,v1:172.21.15.79:6839/2441542443] conn(0x5588eebbd900 0x5588eebbfd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:08:30.679 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:30.678+0000 7f14efdfc740 4 client.15897 mds_command: new command op to 34553 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-03-28T18:08:30.679 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:30.678+0000 7f14efdfc740 1 -- 172.21.15.22:0/3135422155 --> [v2:172.21.15.79:6838/2441542443,v1:172.21.15.79:6839/2441542443] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x5588eeaaa740 con 0x5588eebbd900 2024-03-28T18:08:30.680 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:30.678+0000 7f14e5e00640 1 --2- 172.21.15.22:0/3135422155 >> [v2:172.21.15.79:6838/2441542443,v1:172.21.15.79:6839/2441542443] conn(0x5588eebbd900 0x5588eebbfd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:08:30.680 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:30.679+0000 7f14e5e00640 1 --2- 172.21.15.22:0/3135422155 >> [v2:172.21.15.79:6838/2441542443,v1:172.21.15.79:6839/2441542443] conn(0x5588eebbd900 0x5588eebbfd20 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e5e85a57db68a1c1 server_cookie=fa2617a2cd1f10d8 in_seq=0 out_seq=0 2024-03-28T18:08:30.680 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:30.679+0000 7f14c7e00640 10 client.15897 ms_handle_connect on v2:172.21.15.79:6838/2441542443 2024-03-28T18:08:30.680 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:30.681+0000 7f14c7e00640 1 -- 172.21.15.22:0/3135422155 <== mds.0 v2:172.21.15.79:6838/2441542443 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x5588eeaaa740 con 0x5588eebbd900 2024-03-28T18:08:30.680 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:30.681+0000 7f14c7e00640 10 client.15897 handle_command_reply: tid=0 2024-03-28T18:08:30.681 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:30.681+0000 7f14efdfc740 1 client.15897 shutdown 2024-03-28T18:08:30.681 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:30.681+0000 7f14efdfc740 1 -- 172.21.15.22:0/3135422155 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5588eeb473a0 msgr2=0x5588eeb558c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:08:30.681 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:30.681+0000 7f14efdfc740 1 --2- 172.21.15.22:0/3135422155 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5588eeb473a0 0x5588eeb558c0 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7f14d4011770 tx=0x7f14d4009780 comp rx=0 tx=0).stop 2024-03-28T18:08:30.682 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:30.681+0000 7f14efdfc740 1 -- 172.21.15.22:0/3135422155 shutdown_connections 2024-03-28T18:08:30.682 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:30.681+0000 7f14efdfc740 1 --2- 172.21.15.22:0/3135422155 >> [v2:172.21.15.79:6838/2441542443,v1:172.21.15.79:6839/2441542443] conn(0x5588eebbd900 0x5588eebbfd20 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:08:30.682 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:30.681+0000 7f14efdfc740 1 --2- 172.21.15.22:0/3135422155 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5588eeb4ff80 0x5588eebb5920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:08:30.682 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:30.681+0000 7f14efdfc740 1 --2- 172.21.15.22:0/3135422155 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5588eeb473a0 0x5588eeb558c0 unknown :-1 s=CLOSED pgs=549 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:08:30.682 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:30.681+0000 7f14c7e00640 0 client.15897 ms_handle_reset on v2:172.21.15.79:6838/2441542443 2024-03-28T18:08:30.682 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:30.681+0000 7f14efdfc740 1 --2- 172.21.15.22:0/3135422155 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5588eeb46a80 0x5588eeb55380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:08:30.682 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:30.681+0000 7f14efdfc740 1 -- 172.21.15.22:0/3135422155 >> 172.21.15.22:0/3135422155 conn(0x5588eeb411b0 msgr2=0x5588eeb478c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:08:30.682 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:30.681+0000 7f14efdfc740 1 -- 172.21.15.22:0/3135422155 shutdown_connections 2024-03-28T18:08:30.682 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:30.682+0000 7f14efdfc740 1 -- 172.21.15.22:0/3135422155 wait complete. 2024-03-28T18:08:30.682 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:30.682+0000 7f14efdfc740 20 client.15897 trim_cache size 0 max 16384 2024-03-28T18:08:30.683 INFO:teuthology.orchestra.run.smithi022.stdout:{ 2024-03-28T18:08:30.683 INFO:teuthology.orchestra.run.smithi022.stdout: "return_code": 0, 2024-03-28T18:08:30.683 INFO:teuthology.orchestra.run.smithi022.stdout: "scrub_tag": "f001b158-4bbe-44d6-976f-e2ad983361d7", 2024-03-28T18:08:30.683 INFO:teuthology.orchestra.run.smithi022.stdout: "mode": "asynchronous" 2024-03-28T18:08:30.683 INFO:teuthology.orchestra.run.smithi022.stdout:} 2024-03-28T18:08:30.683 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:30.683+0000 7f14e7200640 1 -- 172.21.15.22:0/2131267614 >> [v2:172.21.15.22:6800/1357365082,v1:172.21.15.22:6801/1357365082] conn(0x7f14cc0a2a90 msgr2=0x7f14cc0a4f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:08:30.683 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:30.683+0000 7f14e7200640 1 --2- 172.21.15.22:0/2131267614 >> [v2:172.21.15.22:6800/1357365082,v1:172.21.15.22:6801/1357365082] conn(0x7f14cc0a2a90 0x7f14cc0a4f50 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x5588eeb46550 tx=0x7f14d8005f90 comp rx=0 tx=0).stop 2024-03-28T18:08:30.684 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:30.683+0000 7f14e7200640 1 -- 172.21.15.22:0/2131267614 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f14e810ddd0 msgr2=0x7f14e819e900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:08:30.684 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:30.683+0000 7f14e7200640 1 --2- 172.21.15.22:0/2131267614 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f14e810ddd0 0x7f14e819e900 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7f14e0005cc0 tx=0x7f14e0034b40 comp rx=0 tx=0).stop 2024-03-28T18:08:30.684 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:30.683+0000 7f14e7200640 1 -- 172.21.15.22:0/2131267614 shutdown_connections 2024-03-28T18:08:30.684 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:30.683+0000 7f14e7200640 1 --2- 172.21.15.22:0/2131267614 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f14e810ddd0 0x7f14e819e900 unknown :-1 s=CLOSED pgs=546 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:08:30.684 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:30.683+0000 7f14e7200640 1 --2- 172.21.15.22:0/2131267614 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f14e8074370 0x7f14e819a580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:08:30.684 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:30.683+0000 7f14e7200640 1 --2- 172.21.15.22:0/2131267614 >> [v2:172.21.15.22:6800/1357365082,v1:172.21.15.22:6801/1357365082] conn(0x7f14cc0a2a90 0x7f14cc0a4f50 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:08:30.684 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:30.683+0000 7f14e7200640 1 --2- 172.21.15.22:0/2131267614 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f14e8073a50 0x7f14e819a040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:08:30.684 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:30.683+0000 7f14e7200640 1 -- 172.21.15.22:0/2131267614 >> 172.21.15.22:0/2131267614 conn(0x7f14e80fd040 msgr2=0x7f14e80fddb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:08:30.684 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:30.683+0000 7f14e7200640 1 -- 172.21.15.22:0/2131267614 shutdown_connections 2024-03-28T18:08:30.684 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:30.683+0000 7f14e7200640 1 -- 172.21.15.22:0/2131267614 wait complete. 2024-03-28T18:08:30.698 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-28T18:08:30.698 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-28T18:08:30.879 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:30.879+0000 7fb096800640 1 -- 172.21.15.22:0/4107701665 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fb090109730 msgr2=0x7fb090109b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:08:30.879 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:30.879+0000 7fb096800640 1 --2- 172.21.15.22:0/4107701665 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fb090109730 0x7fb090109b10 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fb084009900 tx=0x7fb08402bc50 comp rx=0 tx=0).stop 2024-03-28T18:08:30.879 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:30.879+0000 7fb096800640 1 -- 172.21.15.22:0/4107701665 shutdown_connections 2024-03-28T18:08:30.880 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:30.879+0000 7fb096800640 1 --2- 172.21.15.22:0/4107701665 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb0901040e0 0x7fb0901105f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:08:30.880 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:30.879+0000 7fb096800640 1 --2- 172.21.15.22:0/4107701665 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb090103720 0x7fb090103ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:08:30.880 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:30.879+0000 7fb096800640 1 --2- 172.21.15.22:0/4107701665 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fb090109730 0x7fb090109b10 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:08:30.880 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:30.879+0000 7fb096800640 1 -- 172.21.15.22:0/4107701665 >> 172.21.15.22:0/4107701665 conn(0x7fb0900ff1b0 msgr2=0x7fb0901015d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:08:30.880 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:30.879+0000 7fb096800640 1 -- 172.21.15.22:0/4107701665 shutdown_connections 2024-03-28T18:08:30.880 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:30.879+0000 7fb096800640 1 -- 172.21.15.22:0/4107701665 wait complete. 2024-03-28T18:08:30.882 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:30.881+0000 7fb096800640 1 Processor -- start 2024-03-28T18:08:30.882 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:30.881+0000 7fb096800640 1 -- start start 2024-03-28T18:08:30.882 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:30.881+0000 7fb096800640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb090103720 0x7fb09019a060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:08:30.882 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:30.881+0000 7fb096800640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb0901040e0 0x7fb09019a5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:08:30.882 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:30.881+0000 7fb096800640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fb090109730 0x7fb09019e920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:08:30.882 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:30.881+0000 7fb096800640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb090113b60 con 0x7fb090103720 2024-03-28T18:08:30.882 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:30.881+0000 7fb096800640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0901139e0 con 0x7fb090109730 2024-03-28T18:08:30.882 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:30.881+0000 7fb096800640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb090113ce0 con 0x7fb0901040e0 2024-03-28T18:08:30.882 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:30.882+0000 7fb095400640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb090103720 0x7fb09019a060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:08:30.883 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:30.882+0000 7fb095400640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb090103720 0x7fb09019a060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:38168/0 (socket says 172.21.15.22:38168) 2024-03-28T18:08:30.883 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:30.882+0000 7fb095400640 1 -- 172.21.15.22:0/1566201941 learned_addr learned my addr 172.21.15.22:0/1566201941 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-28T18:08:30.883 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:30.882+0000 7fb095e00640 1 --2- 172.21.15.22:0/1566201941 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fb090109730 0x7fb09019e920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:08:30.883 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:30.882+0000 7fb094a00640 1 --2- 172.21.15.22:0/1566201941 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb0901040e0 0x7fb09019a5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:08:30.883 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:30.882+0000 7fb095400640 1 -- 172.21.15.22:0/1566201941 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb0901040e0 msgr2=0x7fb09019a5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:08:30.883 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:30.882+0000 7fb095400640 1 --2- 172.21.15.22:0/1566201941 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb0901040e0 0x7fb09019a5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:08:30.883 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:30.882+0000 7fb095400640 1 -- 172.21.15.22:0/1566201941 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fb090109730 msgr2=0x7fb09019e920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:08:30.883 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:30.882+0000 7fb095400640 1 --2- 172.21.15.22:0/1566201941 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fb090109730 0x7fb09019e920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:08:30.883 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:30.882+0000 7fb095400640 1 -- 172.21.15.22:0/1566201941 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb09019f050 con 0x7fb090103720 2024-03-28T18:08:30.883 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:30.882+0000 7fb094a00640 1 --2- 172.21.15.22:0/1566201941 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb0901040e0 0x7fb09019a5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T18:08:30.884 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:30.883+0000 7fb095400640 1 --2- 172.21.15.22:0/1566201941 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb090103720 0x7fb09019a060 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7fb084005ab0 tx=0x7fb08402c6d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:08:30.884 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:30.883+0000 7fb08e000640 1 -- 172.21.15.22:0/1566201941 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb084043020 con 0x7fb090103720 2024-03-28T18:08:30.884 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:30.883+0000 7fb096800640 1 -- 172.21.15.22:0/1566201941 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb09019f2e0 con 0x7fb090103720 2024-03-28T18:08:30.884 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:30.883+0000 7fb08e000640 1 -- 172.21.15.22:0/1566201941 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7fb084005470 con 0x7fb090103720 2024-03-28T18:08:30.884 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:30.883+0000 7fb08e000640 1 -- 172.21.15.22:0/1566201941 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb084003bf0 con 0x7fb090103720 2024-03-28T18:08:30.884 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:30.883+0000 7fb096800640 1 -- 172.21.15.22:0/1566201941 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb0901a6bd0 con 0x7fb090103720 2024-03-28T18:08:30.887 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:30.887+0000 7fb09ab69740 1 Processor -- start 2024-03-28T18:08:30.887 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:30.887+0000 7fb09ab69740 1 -- start start 2024-03-28T18:08:30.888 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:30.888+0000 7fb09ab69740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55d6e6894dd0 0x55d6e6893b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:08:30.888 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:30.888+0000 7fb09ab69740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d6e68940f0 0x55d6e6894570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:08:30.888 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:30.888+0000 7fb09ab69740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d6e689d460 0x55d6e689f850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:08:30.888 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:30.888+0000 7fb09ab69740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d6e67f8250 con 0x55d6e68940f0 2024-03-28T18:08:30.888 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:30.888+0000 7fb09ab69740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d6e67a0a30 con 0x55d6e6894dd0 2024-03-28T18:08:30.888 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:30.888+0000 7fb09ab69740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d6e67f42f0 con 0x55d6e689d460 2024-03-28T18:08:30.888 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:30.888+0000 7fb095e00640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55d6e6894dd0 0x55d6e6893b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:08:30.889 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:30.888+0000 7fb095e00640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55d6e6894dd0 0x55d6e6893b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.22:47430/0 (socket says 172.21.15.22:47430) 2024-03-28T18:08:30.889 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:30.888+0000 7fb095e00640 1 -- 172.21.15.22:0/2306912274 learned_addr learned my addr 172.21.15.22:0/2306912274 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-28T18:08:30.889 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:30.888+0000 7fb094a00640 1 --2- 172.21.15.22:0/2306912274 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d6e689d460 0x55d6e689f850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:08:30.889 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:30.888+0000 7fb08e000640 1 -- 172.21.15.22:0/1566201941 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 25) v1 ==== 137523+0+0 (secure 0 0 0) 0x7fb084046020 con 0x7fb090103720 2024-03-28T18:08:30.889 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:30.889+0000 7fb095e00640 1 -- 172.21.15.22:0/2306912274 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d6e689d460 msgr2=0x55d6e689f850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T18:08:30.889 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:30.889+0000 7fb095e00640 1 --2- 172.21.15.22:0/2306912274 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d6e689d460 0x55d6e689f850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:08:30.889 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:30.889+0000 7fb095e00640 1 -- 172.21.15.22:0/2306912274 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d6e68940f0 msgr2=0x55d6e6894570 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-28T18:08:30.890 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:30.889+0000 7fb095400640 1 --2- 172.21.15.22:0/2306912274 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d6e68940f0 0x55d6e6894570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:08:30.891 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:30.889+0000 7fb095e00640 1 --2- 172.21.15.22:0/2306912274 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d6e68940f0 0x55d6e6894570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:08:30.891 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:30.889+0000 7fb095e00640 1 -- 172.21.15.22:0/2306912274 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d6e678b690 con 0x55d6e6894dd0 2024-03-28T18:08:30.891 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:30.889+0000 7fb095400640 1 --2- 172.21.15.22:0/2306912274 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d6e68940f0 0x55d6e6894570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T18:08:30.891 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:30.890+0000 7fb095e00640 1 --2- 172.21.15.22:0/2306912274 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55d6e6894dd0 0x55d6e6893b20 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fb088009f50 tx=0x7fb0880118d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ac0a7368141651b9 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:08:30.891 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:30.890+0000 7fb08e000640 1 --2- 172.21.15.22:0/1566201941 >> [v2:172.21.15.22:6800/1357365082,v1:172.21.15.22:6801/1357365082] conn(0x7fb0780a2b10 0x7fb0780a4fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:08:30.891 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:30.890+0000 7fb082600640 1 -- 172.21.15.22:0/2306912274 <== mon.2 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb088004d20 con 0x55d6e6894dd0 2024-03-28T18:08:30.891 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:30.890+0000 7fb08e000640 1 -- 172.21.15.22:0/1566201941 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 8411+0+0 (secure 0 0 0) 0x7fb0840f0fd0 con 0x7fb090103720 2024-03-28T18:08:30.891 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:30.890+0000 7fb082600640 1 -- 172.21.15.22:0/2306912274 <== mon.2 v2:172.21.15.79:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7fb088004050 con 0x55d6e6894dd0 2024-03-28T18:08:30.891 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:30.890+0000 7fb095400640 1 --2- 172.21.15.22:0/1566201941 >> [v2:172.21.15.22:6800/1357365082,v1:172.21.15.22:6801/1357365082] conn(0x7fb0780a2b10 0x7fb0780a4fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:08:30.891 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:30.891+0000 7fb095400640 1 --2- 172.21.15.22:0/1566201941 >> [v2:172.21.15.22:6800/1357365082,v1:172.21.15.22:6801/1357365082] conn(0x7fb0780a2b10 0x7fb0780a4fd0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fb0840ee780 tx=0x7fb084009260 comp rx=0 tx=0).ready entity=mgr.15522 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:08:30.891 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:30.891+0000 7fb082600640 1 -- 172.21.15.22:0/2306912274 <== mon.2 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb088005260 con 0x55d6e6894dd0 2024-03-28T18:08:30.892 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:30.891+0000 7fb09ab69740 1 -- 172.21.15.22:0/2306912274 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55d6e6894dd0 msgr2=0x55d6e6893b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:08:30.892 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:30.891+0000 7fb09ab69740 1 --2- 172.21.15.22:0/2306912274 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55d6e6894dd0 0x55d6e6893b20 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fb088009f50 tx=0x7fb0880118d0 comp rx=0 tx=0).stop 2024-03-28T18:08:30.892 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:30.891+0000 7fb09ab69740 1 -- 172.21.15.22:0/2306912274 shutdown_connections 2024-03-28T18:08:30.892 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:30.891+0000 7fb09ab69740 1 --2- 172.21.15.22:0/2306912274 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d6e689d460 0x55d6e689f850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:08:30.893 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:30.891+0000 7fb09ab69740 1 --2- 172.21.15.22:0/2306912274 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d6e68940f0 0x55d6e6894570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:08:30.893 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:30.891+0000 7fb09ab69740 1 --2- 172.21.15.22:0/2306912274 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55d6e6894dd0 0x55d6e6893b20 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:08:30.893 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:30.891+0000 7fb09ab69740 1 -- 172.21.15.22:0/2306912274 >> 172.21.15.22:0/2306912274 conn(0x55d6e685b200 msgr2=0x55d6e688eb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:08:30.893 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:30.892+0000 7fb09ab69740 1 -- 172.21.15.22:0/2306912274 shutdown_connections 2024-03-28T18:08:30.893 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:30.892+0000 7fb09ab69740 1 -- 172.21.15.22:0/2306912274 wait complete. 2024-03-28T18:08:30.893 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:30.892+0000 7fb09ab69740 1 -- start start 2024-03-28T18:08:30.895 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:30.893+0000 7fb09ab69740 1 Processor -- start 2024-03-28T18:08:30.895 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:30.893+0000 7fb09ab69740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d6e68940f0 0x55d6e689ca40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:08:30.895 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:30.894+0000 7fb09ab69740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55d6e6894dd0 0x55d6e6899dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:08:30.895 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:30.894+0000 7fb09ab69740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d6e689d460 0x55d6e689a310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:08:30.895 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:30.894+0000 7fb09ab69740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d6e68a0320 con 0x55d6e689d460 2024-03-28T18:08:30.895 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:30.894+0000 7fb09ab69740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d6e67b3e80 con 0x55d6e6894dd0 2024-03-28T18:08:30.895 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:30.894+0000 7fb09ab69740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d6e68a04a0 con 0x55d6e68940f0 2024-03-28T18:08:30.895 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:30.894+0000 7fb09ab69740 10 client.0 _ll_register_callbacks cb 0x55d6e67c6130 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-28T18:08:30.896 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:30.894+0000 7fb095e00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d6e68940f0 0x55d6e689ca40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:08:30.896 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:30.894+0000 7fb095e00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d6e68940f0 0x55d6e689ca40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.22:36094/0 (socket says 172.21.15.22:36094) 2024-03-28T18:08:30.896 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:30.894+0000 7fb095e00640 1 -- 172.21.15.22:0/3779116461 learned_addr learned my addr 172.21.15.22:0/3779116461 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-28T18:08:30.896 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:30.894+0000 7fb095e00640 1 --2- 172.21.15.22:0/3779116461 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d6e689d460 0x55d6e689a310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:08:30.896 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:30.894+0000 7fb094a00640 1 --2- 172.21.15.22:0/3779116461 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55d6e6894dd0 0x55d6e6899dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:08:30.896 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:30.895+0000 7fb094a00640 1 -- 172.21.15.22:0/3779116461 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d6e68940f0 msgr2=0x55d6e689ca40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T18:08:30.896 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:30.895+0000 7fb094a00640 1 --2- 172.21.15.22:0/3779116461 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d6e68940f0 0x55d6e689ca40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:08:30.896 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:30.895+0000 7fb094a00640 1 -- 172.21.15.22:0/3779116461 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d6e689d460 msgr2=0x55d6e689a310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T18:08:30.896 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:30.895+0000 7fb094a00640 1 --2- 172.21.15.22:0/3779116461 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d6e689d460 0x55d6e689a310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:08:30.896 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:30.895+0000 7fb094a00640 1 -- 172.21.15.22:0/3779116461 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55d6e65d67d0 con 0x55d6e6894dd0 2024-03-28T18:08:30.896 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:30.895+0000 7fb095e00640 1 --2- 172.21.15.22:0/3779116461 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d6e689d460 0x55d6e689a310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-28T18:08:30.896 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:30.895+0000 7fb094a00640 1 --2- 172.21.15.22:0/3779116461 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55d6e6894dd0 0x55d6e6899dd0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fb07c012440 tx=0x7fb07c012900 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2caa9aaf3678f4b3 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:08:30.897 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:30.895+0000 7fb076a00640 10 client.0 ms_handle_connect on v2:172.21.15.79:3300/0 2024-03-28T18:08:30.898 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:30.896+0000 7fb076a00640 1 -- 172.21.15.22:0/3779116461 <== mon.2 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb07c004460 con 0x55d6e6894dd0 2024-03-28T18:08:30.898 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:30.896+0000 7fb09ab69740 1 -- 172.21.15.22:0/3779116461 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fb088004050 con 0x55d6e6894dd0 2024-03-28T18:08:30.898 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:30.896+0000 7fb076a00640 1 -- 172.21.15.22:0/3779116461 <== mon.2 v2:172.21.15.79:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 8411+0+0 (secure 0 0 0) 0x7fb07c0035f0 con 0x55d6e6894dd0 2024-03-28T18:08:30.899 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:30.896+0000 7fb076a00640 1 -- 172.21.15.22:0/3779116461 <== mon.2 v2:172.21.15.79:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb07c0037f0 con 0x55d6e6894dd0 2024-03-28T18:08:30.899 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:30.896+0000 7fb09ab69740 10 client.35738 fetch_fsmap learned FSMap version 27 2024-03-28T18:08:30.899 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:30.896+0000 7fb09ab69740 1 -- 172.21.15.22:0/3779116461 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55d6e650c3f0 con 0x55d6e6894dd0 2024-03-28T18:08:30.899 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:30.897+0000 7fb076a00640 1 -- 172.21.15.22:0/3779116461 <== mon.2 v2:172.21.15.79:3300/0 4 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7fb088004050 con 0x55d6e6894dd0 2024-03-28T18:08:30.899 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:30.898+0000 7fb076a00640 1 -- 172.21.15.22:0/3779116461 <== mon.2 v2:172.21.15.79:3300/0 5 ==== fsmap(e 27) v1 ==== 7934+0+0 (secure 0 0 0) 0x7fb07c022030 con 0x55d6e6894dd0 2024-03-28T18:08:30.899 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:30.898+0000 7fb09ab69740 10 client.35738 fetch_fsmap finished waiting for FSMap version 27 2024-03-28T18:08:30.899 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:30.898+0000 7fb09ab69740 10 client.35738 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-28T18:08:30.899 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:30.898+0000 7fb09ab69740 20 client.35738 populate_metadata read hostname 'smithi022' 2024-03-28T18:08:30.899 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:30.898+0000 7fb09ab69740 1 --2- 172.21.15.22:0/3779116461 >> [v2:172.21.15.79:6838/2441542443,v1:172.21.15.79:6839/2441542443] conn(0x55d6e68b5970 0x55d6e68b7d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:08:30.899 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:30.898+0000 7fb09ab69740 4 client.35738 mds_command: new command op to 34553 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-28T18:08:30.899 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:30.898+0000 7fb09ab69740 1 -- 172.21.15.22:0/3779116461 --> [v2:172.21.15.79:6838/2441542443,v1:172.21.15.79:6839/2441542443] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55d6e6805790 con 0x55d6e68b5970 2024-03-28T18:08:30.899 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:30.898+0000 7fb095e00640 1 --2- 172.21.15.22:0/3779116461 >> [v2:172.21.15.79:6838/2441542443,v1:172.21.15.79:6839/2441542443] conn(0x55d6e68b5970 0x55d6e68b7d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:08:30.900 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:30.899+0000 7fb095e00640 1 --2- 172.21.15.22:0/3779116461 >> [v2:172.21.15.79:6838/2441542443,v1:172.21.15.79:6839/2441542443] conn(0x55d6e68b5970 0x55d6e68b7d60 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f3ff072d0867e66c server_cookie=78a649d1f7d990b4 in_seq=0 out_seq=0 2024-03-28T18:08:30.900 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:30.899+0000 7fb076a00640 10 client.35738 ms_handle_connect on v2:172.21.15.79:6838/2441542443 2024-03-28T18:08:30.901 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:30.900+0000 7fb076a00640 1 -- 172.21.15.22:0/3779116461 <== mds.0 v2:172.21.15.79:6838/2441542443 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x55d6e6805790 con 0x55d6e68b5970 2024-03-28T18:08:30.901 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:30.900+0000 7fb076a00640 10 client.35738 handle_command_reply: tid=0 2024-03-28T18:08:30.901 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:30.900+0000 7fb09ab69740 1 client.35738 shutdown 2024-03-28T18:08:30.901 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:30.900+0000 7fb09ab69740 1 -- 172.21.15.22:0/3779116461 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55d6e6894dd0 msgr2=0x55d6e6899dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:08:30.901 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:30.900+0000 7fb09ab69740 1 --2- 172.21.15.22:0/3779116461 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55d6e6894dd0 0x55d6e6899dd0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fb07c012440 tx=0x7fb07c012900 comp rx=0 tx=0).stop 2024-03-28T18:08:30.901 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:30.900+0000 7fb09ab69740 1 -- 172.21.15.22:0/3779116461 shutdown_connections 2024-03-28T18:08:30.901 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:30.900+0000 7fb09ab69740 1 --2- 172.21.15.22:0/3779116461 >> [v2:172.21.15.79:6838/2441542443,v1:172.21.15.79:6839/2441542443] conn(0x55d6e68b5970 0x55d6e68b7d60 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:08:30.901 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:30.900+0000 7fb09ab69740 1 --2- 172.21.15.22:0/3779116461 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d6e689d460 0x55d6e689a310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:08:30.901 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:30.900+0000 7fb09ab69740 1 --2- 172.21.15.22:0/3779116461 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55d6e6894dd0 0x55d6e6899dd0 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:08:30.902 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:30.900+0000 7fb09ab69740 1 --2- 172.21.15.22:0/3779116461 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d6e68940f0 0x55d6e689ca40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:08:30.902 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:30.900+0000 7fb076a00640 0 client.35738 ms_handle_reset on v2:172.21.15.79:6838/2441542443 2024-03-28T18:08:30.902 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:30.900+0000 7fb09ab69740 1 -- 172.21.15.22:0/3779116461 >> 172.21.15.22:0/3779116461 conn(0x55d6e685b200 msgr2=0x55d6e6890890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:08:30.902 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:30.900+0000 7fb09ab69740 1 -- 172.21.15.22:0/3779116461 shutdown_connections 2024-03-28T18:08:30.902 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:30.900+0000 7fb09ab69740 1 -- 172.21.15.22:0/3779116461 wait complete. 2024-03-28T18:08:30.902 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:30.900+0000 7fb09ab69740 20 client.35738 trim_cache size 0 max 16384 2024-03-28T18:08:30.933 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:30.933+0000 7fb09ab69740 1 Processor -- start 2024-03-28T18:08:30.933 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:30.933+0000 7fb09ab69740 1 -- start start 2024-03-28T18:08:30.933 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:30.933+0000 7fb09ab69740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d6e68940f0 0x55d6e689e850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:08:30.933 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:30.933+0000 7fb09ab69740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d6e6894dd0 0x55d6e689c500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:08:30.933 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:30.933+0000 7fb09ab69740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55d6e689d460 0x55d6e689ca40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:08:30.933 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:30.933+0000 7fb09ab69740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb07000e3e0 con 0x55d6e68940f0 2024-03-28T18:08:30.933 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:30.933+0000 7fb09ab69740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb07000fd60 con 0x55d6e689d460 2024-03-28T18:08:30.934 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:30.933+0000 7fb09ab69740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb07000ea40 con 0x55d6e6894dd0 2024-03-28T18:08:30.934 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:30.933+0000 7fb095e00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d6e68940f0 0x55d6e689e850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:08:30.934 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:30.934+0000 7fb095e00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d6e68940f0 0x55d6e689e850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:38192/0 (socket says 172.21.15.22:38192) 2024-03-28T18:08:30.934 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:30.934+0000 7fb095e00640 1 -- 172.21.15.22:0/2226454282 learned_addr learned my addr 172.21.15.22:0/2226454282 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-28T18:08:30.935 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:30.934+0000 7fb094a00640 1 --2- 172.21.15.22:0/2226454282 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d6e6894dd0 0x55d6e689c500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:08:30.935 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:30.934+0000 7fb095e00640 1 --2- 172.21.15.22:0/2226454282 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55d6e689d460 0x55d6e689ca40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:08:30.935 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:30.934+0000 7fb095e00640 1 -- 172.21.15.22:0/2226454282 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d6e6894dd0 msgr2=0x55d6e689c500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T18:08:30.935 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:30.934+0000 7fb095e00640 1 --2- 172.21.15.22:0/2226454282 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d6e6894dd0 0x55d6e689c500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:08:30.935 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:30.934+0000 7fb095e00640 1 -- 172.21.15.22:0/2226454282 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55d6e689d460 msgr2=0x55d6e689ca40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T18:08:30.935 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:30.934+0000 7fb095e00640 1 --2- 172.21.15.22:0/2226454282 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55d6e689d460 0x55d6e689ca40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:08:30.935 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:30.934+0000 7fb095e00640 1 -- 172.21.15.22:0/2226454282 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d6e64cc0f0 con 0x55d6e68940f0 2024-03-28T18:08:30.935 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:30.934+0000 7fb095e00640 1 --2- 172.21.15.22:0/2226454282 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d6e68940f0 0x55d6e689e850 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7fb0880099e0 tx=0x7fb0880047d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=31fa7b061f5c584c server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:08:30.935 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:30.935+0000 7fb081c00640 1 -- 172.21.15.22:0/2226454282 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb088003f00 con 0x55d6e68940f0 2024-03-28T18:08:30.935 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:30.935+0000 7fb081c00640 1 -- 172.21.15.22:0/2226454282 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7fb08802b070 con 0x55d6e68940f0 2024-03-28T18:08:30.935 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:30.935+0000 7fb081c00640 1 -- 172.21.15.22:0/2226454282 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb088019430 con 0x55d6e68940f0 2024-03-28T18:08:30.936 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:30.936+0000 7fb09ab69740 1 -- 172.21.15.22:0/2226454282 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d6e68940f0 msgr2=0x55d6e689e850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:08:30.937 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:30.936+0000 7fb09ab69740 1 --2- 172.21.15.22:0/2226454282 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d6e68940f0 0x55d6e689e850 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7fb0880099e0 tx=0x7fb0880047d0 comp rx=0 tx=0).stop 2024-03-28T18:08:30.937 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:30.936+0000 7fb09ab69740 1 -- 172.21.15.22:0/2226454282 shutdown_connections 2024-03-28T18:08:30.937 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:30.936+0000 7fb09ab69740 1 --2- 172.21.15.22:0/2226454282 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55d6e689d460 0x55d6e689ca40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:08:30.937 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:30.936+0000 7fb09ab69740 1 --2- 172.21.15.22:0/2226454282 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d6e6894dd0 0x55d6e689c500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:08:30.937 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:30.936+0000 7fb09ab69740 1 --2- 172.21.15.22:0/2226454282 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d6e68940f0 0x55d6e689e850 unknown :-1 s=CLOSED pgs=551 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:08:30.937 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:30.936+0000 7fb09ab69740 1 -- 172.21.15.22:0/2226454282 >> 172.21.15.22:0/2226454282 conn(0x55d6e685b200 msgr2=0x55d6e688e670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:08:30.937 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:30.936+0000 7fb09ab69740 1 -- 172.21.15.22:0/2226454282 shutdown_connections 2024-03-28T18:08:30.937 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:30.936+0000 7fb09ab69740 1 -- 172.21.15.22:0/2226454282 wait complete. 2024-03-28T18:08:30.937 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:30.937+0000 7fb09ab69740 1 -- start start 2024-03-28T18:08:30.937 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:30.937+0000 7fb09ab69740 1 Processor -- start 2024-03-28T18:08:30.939 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:30.937+0000 7fb09ab69740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55d6e68940f0 0x55d6e689f680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:08:30.939 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:30.937+0000 7fb09ab69740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d6e6894dd0 0x55d6e689ddc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:08:30.939 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:30.937+0000 7fb09ab69740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d6e689d460 0x55d6e689e300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:08:30.939 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:30.937+0000 7fb09ab69740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb07000f700 con 0x55d6e6894dd0 2024-03-28T18:08:30.939 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:30.937+0000 7fb09ab69740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb07000f0a0 con 0x55d6e68940f0 2024-03-28T18:08:30.939 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:30.937+0000 7fb09ab69740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb07000c5a0 con 0x55d6e689d460 2024-03-28T18:08:30.939 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:30.938+0000 7fb09ab69740 10 client.0 _ll_register_callbacks cb 0x7fb07c009550 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-28T18:08:30.939 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:30.938+0000 7fb095e00640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55d6e68940f0 0x55d6e689f680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:08:30.939 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:30.938+0000 7fb094a00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d6e6894dd0 0x55d6e689ddc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:08:30.939 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:30.938+0000 7fb095e00640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55d6e68940f0 0x55d6e689f680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.22:47446/0 (socket says 172.21.15.22:47446) 2024-03-28T18:08:30.940 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:30.938+0000 7fb095e00640 1 -- 172.21.15.22:0/3943435824 learned_addr learned my addr 172.21.15.22:0/3943435824 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-28T18:08:30.940 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:30.938+0000 7fb095e00640 1 --2- 172.21.15.22:0/3943435824 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d6e689d460 0x55d6e689e300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:08:30.940 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:30.938+0000 7fb094a00640 1 -- 172.21.15.22:0/3943435824 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d6e689d460 msgr2=0x55d6e689e300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T18:08:30.940 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:30.938+0000 7fb094a00640 1 --2- 172.21.15.22:0/3943435824 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d6e689d460 0x55d6e689e300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:08:30.940 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:30.938+0000 7fb094a00640 1 -- 172.21.15.22:0/3943435824 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55d6e68940f0 msgr2=0x55d6e689f680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T18:08:30.940 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:30.938+0000 7fb094a00640 1 --2- 172.21.15.22:0/3943435824 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55d6e68940f0 0x55d6e689f680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:08:30.940 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:30.938+0000 7fb094a00640 1 -- 172.21.15.22:0/3943435824 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55d6e689e840 con 0x55d6e6894dd0 2024-03-28T18:08:30.940 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:30.939+0000 7fb094a00640 1 --2- 172.21.15.22:0/3943435824 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d6e6894dd0 0x55d6e689ddc0 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7fb07c0107b0 tx=0x7fb07c005d40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=56ea6fab96ea69d3 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:08:30.940 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:30.939+0000 7fb076a00640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-03-28T18:08:30.940 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:30.939+0000 7fb076a00640 1 -- 172.21.15.22:0/3943435824 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb07c000e60 con 0x55d6e6894dd0 2024-03-28T18:08:30.940 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:30.939+0000 7fb076a00640 1 -- 172.21.15.22:0/3943435824 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 8411+0+0 (secure 0 0 0) 0x7fb07c0052d0 con 0x55d6e6894dd0 2024-03-28T18:08:30.940 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:30.939+0000 7fb09ab69740 1 -- 172.21.15.22:0/3943435824 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fb08802b070 con 0x55d6e6894dd0 2024-03-28T18:08:30.940 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:30.939+0000 7fb076a00640 1 -- 172.21.15.22:0/3943435824 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb07c012280 con 0x55d6e6894dd0 2024-03-28T18:08:30.940 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:30.939+0000 7fb09ab69740 10 client.15912 fetch_fsmap learned FSMap version 27 2024-03-28T18:08:30.940 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:30.939+0000 7fb09ab69740 1 -- 172.21.15.22:0/3943435824 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55d6e68add50 con 0x55d6e6894dd0 2024-03-28T18:08:30.942 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:30.940+0000 7fb076a00640 1 -- 172.21.15.22:0/3943435824 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7fb08802b070 con 0x55d6e6894dd0 2024-03-28T18:08:30.942 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:30.940+0000 7fb076a00640 1 -- 172.21.15.22:0/3943435824 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 27) v1 ==== 7934+0+0 (secure 0 0 0) 0x7fb07c034020 con 0x55d6e6894dd0 2024-03-28T18:08:30.942 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:30.940+0000 7fb09ab69740 10 client.15912 fetch_fsmap finished waiting for FSMap version 27 2024-03-28T18:08:30.942 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:30.940+0000 7fb09ab69740 10 client.15912 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-28T18:08:30.942 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:30.940+0000 7fb09ab69740 20 client.15912 populate_metadata read hostname 'smithi022' 2024-03-28T18:08:30.942 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:30.940+0000 7fb09ab69740 1 --2- 172.21.15.22:0/3943435824 >> [v2:172.21.15.79:6838/2441542443,v1:172.21.15.79:6839/2441542443] conn(0x55d6e6911590 0x55d6e69139b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:08:30.943 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:30.940+0000 7fb09ab69740 4 client.15912 mds_command: new command op to 34553 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-28T18:08:30.943 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:30.940+0000 7fb09ab69740 1 -- 172.21.15.22:0/3943435824 --> [v2:172.21.15.79:6838/2441542443,v1:172.21.15.79:6839/2441542443] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55d6e67ffae0 con 0x55d6e6911590 2024-03-28T18:08:30.943 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:30.941+0000 7fb095e00640 1 --2- 172.21.15.22:0/3943435824 >> [v2:172.21.15.79:6838/2441542443,v1:172.21.15.79:6839/2441542443] conn(0x55d6e6911590 0x55d6e69139b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:08:30.943 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:30.941+0000 7fb095e00640 1 --2- 172.21.15.22:0/3943435824 >> [v2:172.21.15.79:6838/2441542443,v1:172.21.15.79:6839/2441542443] conn(0x55d6e6911590 0x55d6e69139b0 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c98f8b85b7e70454 server_cookie=50a0b3278a225938 in_seq=0 out_seq=0 2024-03-28T18:08:30.943 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:30.941+0000 7fb076a00640 10 client.15912 ms_handle_connect on v2:172.21.15.79:6838/2441542443 2024-03-28T18:08:30.943 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:30.942+0000 7fb076a00640 1 -- 172.21.15.22:0/3943435824 <== mds.0 v2:172.21.15.79:6838/2441542443 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x55d6e67ffae0 con 0x55d6e6911590 2024-03-28T18:08:30.943 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:30.942+0000 7fb076a00640 10 client.15912 handle_command_reply: tid=0 2024-03-28T18:08:30.943 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:30.942+0000 7fb09ab69740 1 client.15912 shutdown 2024-03-28T18:08:30.943 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:30.942+0000 7fb09ab69740 1 -- 172.21.15.22:0/3943435824 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d6e6894dd0 msgr2=0x55d6e689ddc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:08:30.943 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:30.942+0000 7fb09ab69740 1 --2- 172.21.15.22:0/3943435824 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d6e6894dd0 0x55d6e689ddc0 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7fb07c0107b0 tx=0x7fb07c005d40 comp rx=0 tx=0).stop 2024-03-28T18:08:30.943 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:30.942+0000 7fb09ab69740 1 -- 172.21.15.22:0/3943435824 shutdown_connections 2024-03-28T18:08:30.943 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:30.942+0000 7fb09ab69740 1 --2- 172.21.15.22:0/3943435824 >> [v2:172.21.15.79:6838/2441542443,v1:172.21.15.79:6839/2441542443] conn(0x55d6e6911590 0x55d6e69139b0 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:08:30.943 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:30.942+0000 7fb09ab69740 1 --2- 172.21.15.22:0/3943435824 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d6e689d460 0x55d6e689e300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:08:30.943 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:30.942+0000 7fb09ab69740 1 --2- 172.21.15.22:0/3943435824 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d6e6894dd0 0x55d6e689ddc0 unknown :-1 s=CLOSED pgs=552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:08:30.943 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:30.942+0000 7fb09ab69740 1 --2- 172.21.15.22:0/3943435824 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55d6e68940f0 0x55d6e689f680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:08:30.944 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:30.942+0000 7fb09ab69740 1 -- 172.21.15.22:0/3943435824 >> 172.21.15.22:0/3943435824 conn(0x55d6e685b200 msgr2=0x55d6e688fd70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:08:30.944 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:30.942+0000 7fb076a00640 0 client.15912 ms_handle_reset on v2:172.21.15.79:6838/2441542443 2024-03-28T18:08:30.944 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:30.942+0000 7fb09ab69740 1 -- 172.21.15.22:0/3943435824 shutdown_connections 2024-03-28T18:08:30.944 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:30.942+0000 7fb09ab69740 1 -- 172.21.15.22:0/3943435824 wait complete. 2024-03-28T18:08:30.944 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:30.943+0000 7fb09ab69740 20 client.15912 trim_cache size 0 max 16384 2024-03-28T18:08:30.944 INFO:teuthology.orchestra.run.smithi022.stdout:{ 2024-03-28T18:08:30.944 INFO:teuthology.orchestra.run.smithi022.stdout: "status": "IDLE (waiting for more scrubs)", 2024-03-28T18:08:30.944 INFO:teuthology.orchestra.run.smithi022.stdout: "scrubs": { 2024-03-28T18:08:30.944 INFO:teuthology.orchestra.run.smithi022.stdout: "f001b158-4bbe-44d6-976f-e2ad983361d7": { 2024-03-28T18:08:30.944 INFO:teuthology.orchestra.run.smithi022.stdout: "path": "/", 2024-03-28T18:08:30.945 INFO:teuthology.orchestra.run.smithi022.stdout: "tag": "f001b158-4bbe-44d6-976f-e2ad983361d7", 2024-03-28T18:08:30.945 INFO:teuthology.orchestra.run.smithi022.stdout: "options": "recursive,force" 2024-03-28T18:08:30.945 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-03-28T18:08:30.945 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-03-28T18:08:30.945 INFO:teuthology.orchestra.run.smithi022.stdout:} 2024-03-28T18:08:30.945 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:30.943+0000 7fb096800640 1 -- 172.21.15.22:0/1566201941 >> [v2:172.21.15.22:6800/1357365082,v1:172.21.15.22:6801/1357365082] conn(0x7fb0780a2b10 msgr2=0x7fb0780a4fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:08:30.946 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:30.943+0000 7fb096800640 1 --2- 172.21.15.22:0/1566201941 >> [v2:172.21.15.22:6800/1357365082,v1:172.21.15.22:6801/1357365082] conn(0x7fb0780a2b10 0x7fb0780a4fd0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fb0840ee780 tx=0x7fb084009260 comp rx=0 tx=0).stop 2024-03-28T18:08:30.946 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:30.943+0000 7fb096800640 1 -- 172.21.15.22:0/1566201941 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb090103720 msgr2=0x7fb09019a060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:08:30.946 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:30.943+0000 7fb096800640 1 --2- 172.21.15.22:0/1566201941 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb090103720 0x7fb09019a060 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7fb084005ab0 tx=0x7fb08402c6d0 comp rx=0 tx=0).stop 2024-03-28T18:08:30.946 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:30.944+0000 7fb096800640 1 -- 172.21.15.22:0/1566201941 shutdown_connections 2024-03-28T18:08:30.946 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:30.944+0000 7fb096800640 1 --2- 172.21.15.22:0/1566201941 >> [v2:172.21.15.22:6800/1357365082,v1:172.21.15.22:6801/1357365082] conn(0x7fb0780a2b10 0x7fb0780a4fd0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:08:30.946 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:30.944+0000 7fb096800640 1 --2- 172.21.15.22:0/1566201941 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fb090109730 0x7fb09019e920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:08:30.946 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:30.944+0000 7fb096800640 1 --2- 172.21.15.22:0/1566201941 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb0901040e0 0x7fb09019a5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:08:30.946 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:30.944+0000 7fb096800640 1 --2- 172.21.15.22:0/1566201941 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb090103720 0x7fb09019a060 unknown :-1 s=CLOSED pgs=550 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:08:30.946 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:30.944+0000 7fb096800640 1 -- 172.21.15.22:0/1566201941 >> 172.21.15.22:0/1566201941 conn(0x7fb0900ff1b0 msgr2=0x7fb0900ff790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:08:30.946 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:30.944+0000 7fb096800640 1 -- 172.21.15.22:0/1566201941 shutdown_connections 2024-03-28T18:08:30.946 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:08:30.944+0000 7fb096800640 1 -- 172.21.15.22:0/1566201941 wait complete. 2024-03-28T18:08:30.965 INFO:tasks.cephfs.filesystem:scrub status for tag:f001b158-4bbe-44d6-976f-e2ad983361d7 - {'path': '/', 'tag': 'f001b158-4bbe-44d6-976f-e2ad983361d7', 'options': 'recursive,force'} 2024-03-28T18:08:32.424 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:08:32 smithi134 ceph-mon[35037]: scrub queued for path: / 2024-03-28T18:08:32.425 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:08:32 smithi134 ceph-mon[35037]: scrub summary: idle+waiting paths [/] 2024-03-28T18:08:32.425 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:08:32 smithi134 ceph-mon[35037]: scrub summary: active paths [/] 2024-03-28T18:08:32.425 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:08:32 smithi134 ceph-mon[35037]: scrub summary: idle+waiting paths [/] 2024-03-28T18:08:32.425 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:08:32 smithi134 ceph-mon[35037]: pgmap v32: 97 pgs: 97 active+clean; 1.3 MiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-03-28T18:08:32.432 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:08:32 smithi079 ceph-mon[35312]: scrub queued for path: / 2024-03-28T18:08:32.432 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:08:32 smithi079 ceph-mon[35312]: scrub summary: idle+waiting paths [/] 2024-03-28T18:08:32.432 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:08:32 smithi079 ceph-mon[35312]: scrub summary: active paths [/] 2024-03-28T18:08:32.432 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:08:32 smithi079 ceph-mon[35312]: scrub summary: idle+waiting paths [/] 2024-03-28T18:08:32.432 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:08:32 smithi079 ceph-mon[35312]: pgmap v32: 97 pgs: 97 active+clean; 1.3 MiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-03-28T18:08:32.531 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:08:32 smithi022 ceph-mon[31985]: scrub queued for path: / 2024-03-28T18:08:32.531 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:08:32 smithi022 ceph-mon[31985]: scrub summary: idle+waiting paths [/] 2024-03-28T18:08:32.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:08:32 smithi022 ceph-mon[31985]: scrub summary: active paths [/] 2024-03-28T18:08:32.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:08:32 smithi022 ceph-mon[31985]: scrub summary: idle+waiting paths [/] 2024-03-28T18:08:32.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:08:32 smithi022 ceph-mon[31985]: pgmap v32: 97 pgs: 97 active+clean; 1.3 MiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-03-28T18:08:34.425 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:08:34 smithi134 ceph-mon[35037]: pgmap v33: 97 pgs: 97 active+clean; 1.3 MiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-28T18:08:34.432 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:08:34 smithi079 ceph-mon[35312]: pgmap v33: 97 pgs: 97 active+clean; 1.3 MiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-28T18:08:34.531 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:08:34 smithi022 ceph-mon[31985]: pgmap v33: 97 pgs: 97 active+clean; 1.3 MiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-28T18:08:35.424 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:08:35 smithi134 ceph-mon[35037]: scrub summary: idle 2024-03-28T18:08:35.432 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:08:35 smithi079 ceph-mon[35312]: scrub summary: idle 2024-03-28T18:08:35.531 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:08:35 smithi022 ceph-mon[31985]: scrub summary: idle 2024-03-28T18:08:36.424 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:08:36 smithi134 ceph-mon[35037]: pgmap v34: 97 pgs: 97 active+clean; 1.3 MiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 12 op/s 2024-03-28T18:08:36.432 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:08:36 smithi079 ceph-mon[35312]: pgmap v34: 97 pgs: 97 active+clean; 1.3 MiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 12 op/s 2024-03-28T18:08:36.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:08:36 smithi022 ceph-mon[31985]: pgmap v34: 97 pgs: 97 active+clean; 1.3 MiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 12 op/s 2024-03-28T18:08:38.424 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:08:38 smithi134 ceph-mon[35037]: pgmap v35: 97 pgs: 97 active+clean; 1.3 MiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 8 op/s 2024-03-28T18:08:38.432 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:08:38 smithi079 ceph-mon[35312]: pgmap v35: 97 pgs: 97 active+clean; 1.3 MiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 8 op/s 2024-03-28T18:08:38.531 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:08:38 smithi022 ceph-mon[31985]: pgmap v35: 97 pgs: 97 active+clean; 1.3 MiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 8 op/s 2024-03-28T18:08:40.424 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:08:40 smithi134 ceph-mon[35037]: pgmap v36: 97 pgs: 97 active+clean; 1.3 MiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 8 op/s 2024-03-28T18:08:40.432 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:08:40 smithi079 ceph-mon[35312]: pgmap v36: 97 pgs: 97 active+clean; 1.3 MiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 8 op/s 2024-03-28T18:08:40.531 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:08:40 smithi022 ceph-mon[31985]: pgmap v36: 97 pgs: 97 active+clean; 1.3 MiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 8 op/s 2024-03-28T18:08:42.424 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:08:42 smithi134 ceph-mon[35037]: pgmap v37: 97 pgs: 97 active+clean; 1.3 MiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 12 op/s 2024-03-28T18:08:42.432 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:08:42 smithi079 ceph-mon[35312]: pgmap v37: 97 pgs: 97 active+clean; 1.3 MiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 12 op/s 2024-03-28T18:08:42.531 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:08:42 smithi022 ceph-mon[31985]: pgmap v37: 97 pgs: 97 active+clean; 1.3 MiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 12 op/s 2024-03-28T18:08:43.424 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:08:43 smithi134 ceph-mon[35037]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T18:08:43.432 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:08:43 smithi079 ceph-mon[35312]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T18:08:43.531 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:08:43 smithi022 ceph-mon[31985]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T18:08:44.424 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:08:44 smithi134 ceph-mon[35037]: pgmap v38: 97 pgs: 97 active+clean; 1.3 MiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 8 op/s 2024-03-28T18:08:44.432 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:08:44 smithi079 ceph-mon[35312]: pgmap v38: 97 pgs: 97 active+clean; 1.3 MiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 8 op/s 2024-03-28T18:08:44.531 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:08:44 smithi022 ceph-mon[31985]: pgmap v38: 97 pgs: 97 active+clean; 1.3 MiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 8 op/s 2024-03-28T18:08:45.424 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:08:45 smithi134 ceph-mon[35037]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T18:08:45.425 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:08:45 smithi134 ceph-mon[35037]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T18:08:45.425 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:08:45 smithi134 ceph-mon[35037]: from='mgr.15522 ' entity='mgr.x' 2024-03-28T18:08:45.432 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:08:45 smithi079 ceph-mon[35312]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T18:08:45.432 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:08:45 smithi079 ceph-mon[35312]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T18:08:45.432 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:08:45 smithi079 ceph-mon[35312]: from='mgr.15522 ' entity='mgr.x' 2024-03-28T18:08:45.531 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:08:45 smithi022 ceph-mon[31985]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T18:08:45.531 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:08:45 smithi022 ceph-mon[31985]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T18:08:45.531 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:08:45 smithi022 ceph-mon[31985]: from='mgr.15522 ' entity='mgr.x' 2024-03-28T18:08:46.674 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:08:46 smithi134 ceph-mon[35037]: pgmap v39: 97 pgs: 97 active+clean; 1.3 MiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 12 op/s 2024-03-28T18:08:46.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:08:46 smithi079 ceph-mon[35312]: pgmap v39: 97 pgs: 97 active+clean; 1.3 MiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 12 op/s 2024-03-28T18:08:46.781 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:08:46 smithi022 ceph-mon[31985]: pgmap v39: 97 pgs: 97 active+clean; 1.3 MiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 12 op/s 2024-03-28T18:08:48.674 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:08:48 smithi134 ceph-mon[35037]: pgmap v40: 97 pgs: 97 active+clean; 1.3 MiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-28T18:08:48.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:08:48 smithi079 ceph-mon[35312]: pgmap v40: 97 pgs: 97 active+clean; 1.3 MiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-28T18:08:48.781 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:08:48 smithi022 ceph-mon[31985]: pgmap v40: 97 pgs: 97 active+clean; 1.3 MiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-28T18:08:50.674 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:08:50 smithi134 ceph-mon[35037]: pgmap v41: 97 pgs: 97 active+clean; 1.3 MiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-28T18:08:50.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:08:50 smithi079 ceph-mon[35312]: pgmap v41: 97 pgs: 97 active+clean; 1.3 MiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-28T18:08:50.781 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:08:50 smithi022 ceph-mon[31985]: pgmap v41: 97 pgs: 97 active+clean; 1.3 MiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-28T18:08:52.674 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:08:52 smithi134 ceph-mon[35037]: pgmap v42: 97 pgs: 97 active+clean; 1.3 MiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-03-28T18:08:52.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:08:52 smithi079 ceph-mon[35312]: pgmap v42: 97 pgs: 97 active+clean; 1.3 MiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-03-28T18:08:52.781 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:08:52 smithi022 ceph-mon[31985]: pgmap v42: 97 pgs: 97 active+clean; 1.3 MiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-03-28T18:08:54.674 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:08:54 smithi134 ceph-mon[35037]: pgmap v43: 97 pgs: 97 active+clean; 1.3 MiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-28T18:08:54.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:08:54 smithi079 ceph-mon[35312]: pgmap v43: 97 pgs: 97 active+clean; 1.3 MiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-28T18:08:54.781 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:08:54 smithi022 ceph-mon[31985]: pgmap v43: 97 pgs: 97 active+clean; 1.3 MiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-28T18:08:56.674 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:08:56 smithi134 ceph-mon[35037]: pgmap v44: 97 pgs: 97 active+clean; 1.3 MiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-03-28T18:08:56.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:08:56 smithi079 ceph-mon[35312]: pgmap v44: 97 pgs: 97 active+clean; 1.3 MiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-03-28T18:08:56.781 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:08:56 smithi022 ceph-mon[31985]: pgmap v44: 97 pgs: 97 active+clean; 1.3 MiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-03-28T18:08:58.674 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:08:58 smithi134 ceph-mon[35037]: pgmap v45: 97 pgs: 97 active+clean; 1.3 MiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-28T18:08:58.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:08:58 smithi079 ceph-mon[35312]: pgmap v45: 97 pgs: 97 active+clean; 1.3 MiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-28T18:08:58.781 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:08:58 smithi022 ceph-mon[31985]: pgmap v45: 97 pgs: 97 active+clean; 1.3 MiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-28T18:09:00.162 INFO:tasks.workunit.client.0.smithi022.stderr:Updating files: 59% (7326/12271) Updating files: 60% (7363/12271) Updating files: 61% (7486/12271) Updating files: 62% (7609/12271) Updating files: 63% (7731/12271) Updating files: 64% (7854/12271) Updating files: 65% (7977/12271) Updating files: 66% (8099/12271) Updating files: 67% (8222/12271) Updating files: 68% (8345/12271) Updating files: 69% (8467/12271) Updating files: 70% (8590/12271) Updating files: 71% (8713/12271) Updating files: 72% (8836/12271) Updating files: 73% (8958/12271) Updating files: 74% (9081/12271) Updating files: 75% (9204/12271) Updating files: 76% (9326/12271) Updating files: 77% (9449/12271) Updating files: 78% (9572/12271) Updating files: 79% (9695/12271) Updating files: 80% (9817/12271) Updating files: 81% (9940/12271) Updating files: 82% (10063/12271) Updating files: 83% (10185/12271) Updating files: 84% (10308/12271) Updating files: 85% (10431/12271) Updating files: 86% (10554/12271) Updating files: 87% (10676/12271) Updating files: 88% (10799/12271) Updating files: 89% (10922/12271) Updating files: 90% (11044/12271) Updating files: 91% (11167/12271) Updating files: 92% (11290/12271) Updating files: 93% (11413/12271) Updating files: 94% (11535/12271) Updating files: 95% (11658/12271) Updating files: 96% (11781/12271) Updating files: 97% (11903/12271) Updating files: 98% (12026/12271) Updating files: 99% (12149/12271) Updating files: 99% (12202/12271) Updating files: 100% (12271/12271) Updating files: 100% (12271/12271), done. 2024-03-28T18:09:00.566 INFO:tasks.workunit.client.0.smithi022.stderr:Note: switching to '3be9a5c9bc793e11e2800a8c0c696e8b46742033'. 2024-03-28T18:09:00.566 INFO:tasks.workunit.client.0.smithi022.stderr: 2024-03-28T18:09:00.566 INFO:tasks.workunit.client.0.smithi022.stderr:You are in 'detached HEAD' state. You can look around, make experimental 2024-03-28T18:09:00.566 INFO:tasks.workunit.client.0.smithi022.stderr:changes and commit them, and you can discard any commits you make in this 2024-03-28T18:09:00.566 INFO:tasks.workunit.client.0.smithi022.stderr:state without impacting any branches by switching back to a branch. 2024-03-28T18:09:00.566 INFO:tasks.workunit.client.0.smithi022.stderr: 2024-03-28T18:09:00.566 INFO:tasks.workunit.client.0.smithi022.stderr:If you want to create a new branch to retain commits you create, you may 2024-03-28T18:09:00.566 INFO:tasks.workunit.client.0.smithi022.stderr:do so (now or later) by using -c with the switch command. Example: 2024-03-28T18:09:00.566 INFO:tasks.workunit.client.0.smithi022.stderr: 2024-03-28T18:09:00.566 INFO:tasks.workunit.client.0.smithi022.stderr: git switch -c 2024-03-28T18:09:00.566 INFO:tasks.workunit.client.0.smithi022.stderr: 2024-03-28T18:09:00.566 INFO:tasks.workunit.client.0.smithi022.stderr:Or undo this operation with: 2024-03-28T18:09:00.566 INFO:tasks.workunit.client.0.smithi022.stderr: 2024-03-28T18:09:00.567 INFO:tasks.workunit.client.0.smithi022.stderr: git switch - 2024-03-28T18:09:00.567 INFO:tasks.workunit.client.0.smithi022.stderr: 2024-03-28T18:09:00.567 INFO:tasks.workunit.client.0.smithi022.stderr:Turn off this advice by setting config variable advice.detachedHead to false 2024-03-28T18:09:00.567 INFO:tasks.workunit.client.0.smithi022.stderr: 2024-03-28T18:09:00.567 INFO:tasks.workunit.client.0.smithi022.stderr:HEAD is now at 3be9a5c9bc7 dummy commit #1 2024-03-28T18:09:00.575 DEBUG:teuthology.orchestra.run.smithi022:> cd -- /home/ubuntu/cephtest/clone.client.0/qa/workunits && if test -e Makefile ; then make ; fi && find -executable -type f -printf '%P\0' >/home/ubuntu/cephtest/workunits.list.client.0 2024-03-28T18:09:00.633 INFO:tasks.workunit.client.0.smithi022.stdout:for d in direct_io fs ; do ( cd $d ; make all ) ; done 2024-03-28T18:09:00.639 INFO:tasks.workunit.client.0.smithi022.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-03-28T18:09:00.639 INFO:tasks.workunit.client.0.smithi022.stdout:cc -Wall -Wextra -D_GNU_SOURCE direct_io_test.c -o direct_io_test 2024-03-28T18:09:00.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:09:00 smithi079 ceph-mon[35312]: pgmap v46: 97 pgs: 97 active+clean; 1.3 MiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-28T18:09:00.781 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:09:00 smithi022 ceph-mon[31985]: pgmap v46: 97 pgs: 97 active+clean; 1.3 MiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-28T18:09:00.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:09:00 smithi134 ceph-mon[35037]: pgmap v46: 97 pgs: 97 active+clean; 1.3 MiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-28T18:09:00.966 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-28T18:09:00.966 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-28T18:09:01.205 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:01.205+0000 7f2a60600640 1 -- 172.21.15.22:0/2110141932 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f2a58109850 msgr2=0x7f2a58109c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:09:01.205 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:01.205+0000 7f2a60600640 1 --2- 172.21.15.22:0/2110141932 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f2a58109850 0x7f2a58109c30 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f2a48009ca0 tx=0x7f2a4802c0f0 comp rx=0 tx=0).stop 2024-03-28T18:09:01.205 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:01.205+0000 7f2a60600640 1 -- 172.21.15.22:0/2110141932 shutdown_connections 2024-03-28T18:09:01.205 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:01.205+0000 7f2a60600640 1 --2- 172.21.15.22:0/2110141932 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2a58104200 0x7f2a58110710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:09:01.205 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:01.205+0000 7f2a60600640 1 --2- 172.21.15.22:0/2110141932 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2a58103840 0x7f2a58103cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:09:01.205 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:01.205+0000 7f2a60600640 1 --2- 172.21.15.22:0/2110141932 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f2a58109850 0x7f2a58109c30 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:09:01.205 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:01.205+0000 7f2a60600640 1 -- 172.21.15.22:0/2110141932 >> 172.21.15.22:0/2110141932 conn(0x7f2a580ff350 msgr2=0x7f2a58101770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:09:01.205 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:01.205+0000 7f2a60600640 1 -- 172.21.15.22:0/2110141932 shutdown_connections 2024-03-28T18:09:01.205 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:01.205+0000 7f2a60600640 1 -- 172.21.15.22:0/2110141932 wait complete. 2024-03-28T18:09:01.206 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:01.206+0000 7f2a60600640 1 Processor -- start 2024-03-28T18:09:01.206 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:01.206+0000 7f2a60600640 1 -- start start 2024-03-28T18:09:01.207 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:01.207+0000 7f2a60600640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2a58103840 0x7f2a5819a1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:09:01.207 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:01.207+0000 7f2a60600640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2a58104200 0x7f2a5819a730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:09:01.207 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:01.207+0000 7f2a60600640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f2a58109850 0x7f2a5819eab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:09:01.207 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:01.207+0000 7f2a60600640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a58113c30 con 0x7f2a58103840 2024-03-28T18:09:01.208 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:01.207+0000 7f2a60600640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a58113ab0 con 0x7f2a58109850 2024-03-28T18:09:01.208 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:01.207+0000 7f2a60600640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a58113db0 con 0x7f2a58104200 2024-03-28T18:09:01.208 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:01.207+0000 7f2a5dc00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2a58103840 0x7f2a5819a1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:09:01.208 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:01.207+0000 7f2a5dc00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2a58103840 0x7f2a5819a1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:54412/0 (socket says 172.21.15.22:54412) 2024-03-28T18:09:01.208 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:01.207+0000 7f2a5dc00640 1 -- 172.21.15.22:0/1381615903 learned_addr learned my addr 172.21.15.22:0/1381615903 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-28T18:09:01.208 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:01.207+0000 7f2a5e600640 1 --2- 172.21.15.22:0/1381615903 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f2a58109850 0x7f2a5819eab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:09:01.208 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:01.208+0000 7f2a5d200640 1 --2- 172.21.15.22:0/1381615903 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2a58104200 0x7f2a5819a730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:09:01.208 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:01.208+0000 7f2a5dc00640 1 -- 172.21.15.22:0/1381615903 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2a58104200 msgr2=0x7f2a5819a730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:09:01.208 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:01.208+0000 7f2a5dc00640 1 --2- 172.21.15.22:0/1381615903 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2a58104200 0x7f2a5819a730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:09:01.208 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:01.208+0000 7f2a5dc00640 1 -- 172.21.15.22:0/1381615903 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f2a58109850 msgr2=0x7f2a5819eab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:09:01.208 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:01.208+0000 7f2a5dc00640 1 --2- 172.21.15.22:0/1381615903 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f2a58109850 0x7f2a5819eab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:09:01.208 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:01.208+0000 7f2a5dc00640 1 -- 172.21.15.22:0/1381615903 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2a5819f1e0 con 0x7f2a58103840 2024-03-28T18:09:01.209 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:01.208+0000 7f2a5dc00640 1 --2- 172.21.15.22:0/1381615903 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2a58103840 0x7f2a5819a1f0 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7f2a48000f80 tx=0x7f2a48034cf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:09:01.209 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:01.209+0000 7f2a56000640 1 -- 172.21.15.22:0/1381615903 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a48039050 con 0x7f2a58103840 2024-03-28T18:09:01.209 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:01.209+0000 7f2a60600640 1 -- 172.21.15.22:0/1381615903 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2a5819f470 con 0x7f2a58103840 2024-03-28T18:09:01.209 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:01.209+0000 7f2a56000640 1 -- 172.21.15.22:0/1381615903 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f2a4803a040 con 0x7f2a58103840 2024-03-28T18:09:01.209 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:01.209+0000 7f2a56000640 1 -- 172.21.15.22:0/1381615903 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a48004590 con 0x7f2a58103840 2024-03-28T18:09:01.209 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:01.209+0000 7f2a60600640 1 -- 172.21.15.22:0/1381615903 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2a581a6d60 con 0x7f2a58103840 2024-03-28T18:09:01.213 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:01.212+0000 7f2a6325d740 1 Processor -- start 2024-03-28T18:09:01.213 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:01.212+0000 7f2a6325d740 1 -- start start 2024-03-28T18:09:01.213 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:01.212+0000 7f2a56000640 1 -- 172.21.15.22:0/1381615903 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 25) v1 ==== 137523+0+0 (secure 0 0 0) 0x7f2a48003970 con 0x7f2a58103840 2024-03-28T18:09:01.213 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:01.212+0000 7f2a6325d740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555f27b83c00 0x555f27b82950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:09:01.213 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:01.212+0000 7f2a6325d740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x555f27b82f20 0x555f27b833a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:09:01.213 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:01.212+0000 7f2a6325d740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x555f27b8c290 0x555f27b8e680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:09:01.213 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:01.212+0000 7f2a6325d740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x555f27ae7250 con 0x555f27b82f20 2024-03-28T18:09:01.213 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:01.212+0000 7f2a6325d740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x555f27a8fa30 con 0x555f27b8c290 2024-03-28T18:09:01.213 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:01.212+0000 7f2a6325d740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x555f27ae32f0 con 0x555f27b83c00 2024-03-28T18:09:01.213 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:01.213+0000 7f2a5dc00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x555f27b82f20 0x555f27b833a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:09:01.213 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:01.213+0000 7f2a5d200640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x555f27b8c290 0x555f27b8e680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:09:01.213 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:01.213+0000 7f2a5dc00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x555f27b82f20 0x555f27b833a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:54416/0 (socket says 172.21.15.22:54416) 2024-03-28T18:09:01.213 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:01.213+0000 7f2a5dc00640 1 -- 172.21.15.22:0/1199780441 learned_addr learned my addr 172.21.15.22:0/1199780441 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-28T18:09:01.214 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:01.213+0000 7f2a5e600640 1 --2- 172.21.15.22:0/1199780441 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555f27b83c00 0x555f27b82950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:09:01.214 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:01.213+0000 7f2a5d200640 1 -- 172.21.15.22:0/1199780441 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555f27b83c00 msgr2=0x555f27b82950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T18:09:01.214 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:01.213+0000 7f2a5d200640 1 --2- 172.21.15.22:0/1199780441 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555f27b83c00 0x555f27b82950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:09:01.214 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:01.213+0000 7f2a5d200640 1 -- 172.21.15.22:0/1199780441 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x555f27b82f20 msgr2=0x555f27b833a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T18:09:01.214 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:01.213+0000 7f2a5d200640 1 --2- 172.21.15.22:0/1199780441 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x555f27b82f20 0x555f27b833a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:09:01.214 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:01.213+0000 7f2a5d200640 1 -- 172.21.15.22:0/1199780441 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555f27a7a690 con 0x555f27b8c290 2024-03-28T18:09:01.217 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:01.214+0000 7f2a5d200640 1 --2- 172.21.15.22:0/1199780441 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x555f27b8c290 0x555f27b8e680 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f2a440058c0 tx=0x7f2a4400e350 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e7f67ea6b0138fb3 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:09:01.217 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:01.214+0000 7f2a56000640 1 --2- 172.21.15.22:0/1381615903 >> [v2:172.21.15.22:6800/1357365082,v1:172.21.15.22:6801/1357365082] conn(0x7f2a3c0a2b10 0x7f2a3c0a4fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:09:01.217 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:01.214+0000 7f2a4ea00640 1 -- 172.21.15.22:0/1199780441 <== mon.2 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a44004020 con 0x555f27b8c290 2024-03-28T18:09:01.217 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:01.214+0000 7f2a4ea00640 1 -- 172.21.15.22:0/1199780441 <== mon.2 v2:172.21.15.79:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f2a440041c0 con 0x555f27b8c290 2024-03-28T18:09:01.217 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:01.214+0000 7f2a4ea00640 1 -- 172.21.15.22:0/1199780441 <== mon.2 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a440106e0 con 0x555f27b8c290 2024-03-28T18:09:01.217 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:01.214+0000 7f2a56000640 1 -- 172.21.15.22:0/1381615903 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 8411+0+0 (secure 0 0 0) 0x7f2a480f16a0 con 0x7f2a58103840 2024-03-28T18:09:01.217 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:01.214+0000 7f2a5e600640 1 --2- 172.21.15.22:0/1381615903 >> [v2:172.21.15.22:6800/1357365082,v1:172.21.15.22:6801/1357365082] conn(0x7f2a3c0a2b10 0x7f2a3c0a4fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:09:01.217 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:01.215+0000 7f2a6325d740 1 -- 172.21.15.22:0/1199780441 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x555f27b8c290 msgr2=0x555f27b8e680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:09:01.217 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:01.215+0000 7f2a6325d740 1 --2- 172.21.15.22:0/1199780441 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x555f27b8c290 0x555f27b8e680 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f2a440058c0 tx=0x7f2a4400e350 comp rx=0 tx=0).stop 2024-03-28T18:09:01.218 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:01.215+0000 7f2a5e600640 1 --2- 172.21.15.22:0/1381615903 >> [v2:172.21.15.22:6800/1357365082,v1:172.21.15.22:6801/1357365082] conn(0x7f2a3c0a2b10 0x7f2a3c0a4fd0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f2a50009c70 tx=0x7f2a50005f90 comp rx=0 tx=0).ready entity=mgr.15522 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:09:01.218 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:01.215+0000 7f2a6325d740 1 -- 172.21.15.22:0/1199780441 shutdown_connections 2024-03-28T18:09:01.218 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:01.215+0000 7f2a6325d740 1 --2- 172.21.15.22:0/1199780441 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x555f27b8c290 0x555f27b8e680 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:09:01.218 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:01.215+0000 7f2a6325d740 1 --2- 172.21.15.22:0/1199780441 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x555f27b82f20 0x555f27b833a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:09:01.218 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:01.215+0000 7f2a6325d740 1 --2- 172.21.15.22:0/1199780441 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555f27b83c00 0x555f27b82950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:09:01.218 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:01.215+0000 7f2a6325d740 1 -- 172.21.15.22:0/1199780441 >> 172.21.15.22:0/1199780441 conn(0x555f27b4a200 msgr2=0x555f27b7d970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:09:01.218 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:01.215+0000 7f2a6325d740 1 -- 172.21.15.22:0/1199780441 shutdown_connections 2024-03-28T18:09:01.218 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:01.215+0000 7f2a6325d740 1 -- 172.21.15.22:0/1199780441 wait complete. 2024-03-28T18:09:01.218 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:01.215+0000 7f2a6325d740 1 -- start start 2024-03-28T18:09:01.218 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:01.216+0000 7f2a6325d740 1 Processor -- start 2024-03-28T18:09:01.218 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:01.216+0000 7f2a6325d740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555f27b82f20 0x555f27b8b8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:09:01.218 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:01.216+0000 7f2a6325d740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x555f27b83c00 0x555f27b88c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:09:01.218 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:01.216+0000 7f2a6325d740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x555f27b8c290 0x555f27b89170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:09:01.218 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:01.216+0000 7f2a6325d740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x555f27b8f100 con 0x555f27b83c00 2024-03-28T18:09:01.219 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:01.216+0000 7f2a6325d740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x555f27aa2e80 con 0x555f27b8c290 2024-03-28T18:09:01.219 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:01.216+0000 7f2a6325d740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x555f27b8f280 con 0x555f27b82f20 2024-03-28T18:09:01.219 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:01.216+0000 7f2a6325d740 10 client.0 _ll_register_callbacks cb 0x555f27ab5130 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-28T18:09:01.219 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:01.216+0000 7f2a5d200640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x555f27b83c00 0x555f27b88c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:09:01.219 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:01.217+0000 7f2a5e600640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x555f27b8c290 0x555f27b89170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:09:01.219 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:01.217+0000 7f2a5d200640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x555f27b83c00 0x555f27b88c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:54420/0 (socket says 172.21.15.22:54420) 2024-03-28T18:09:01.219 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:01.217+0000 7f2a5d200640 1 -- 172.21.15.22:0/277134551 learned_addr learned my addr 172.21.15.22:0/277134551 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-28T18:09:01.219 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:01.217+0000 7f2a5dc00640 1 --2- 172.21.15.22:0/277134551 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555f27b82f20 0x555f27b8b8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:09:01.219 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:01.217+0000 7f2a5d200640 1 -- 172.21.15.22:0/277134551 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555f27b82f20 msgr2=0x555f27b8b8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T18:09:01.219 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:01.217+0000 7f2a5d200640 1 --2- 172.21.15.22:0/277134551 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555f27b82f20 0x555f27b8b8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:09:01.219 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:01.217+0000 7f2a5d200640 1 -- 172.21.15.22:0/277134551 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x555f27b8c290 msgr2=0x555f27b89170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T18:09:01.219 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:01.217+0000 7f2a5d200640 1 --2- 172.21.15.22:0/277134551 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x555f27b8c290 0x555f27b89170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:09:01.219 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:01.217+0000 7f2a5d200640 1 -- 172.21.15.22:0/277134551 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x555f278c57d0 con 0x555f27b83c00 2024-03-28T18:09:01.219 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:01.217+0000 7f2a5e600640 1 --2- 172.21.15.22:0/277134551 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x555f27b8c290 0x555f27b89170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-28T18:09:01.219 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:01.217+0000 7f2a5d200640 1 --2- 172.21.15.22:0/277134551 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x555f27b83c00 0x555f27b88c30 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7f2a44003f30 tx=0x7f2a44003d90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=921f2f1ab2208e96 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:09:01.220 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:01.217+0000 7f2a43400640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-03-28T18:09:01.222 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:01.218+0000 7f2a43400640 1 -- 172.21.15.22:0/277134551 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a44029060 con 0x555f27b83c00 2024-03-28T18:09:01.222 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:01.218+0000 7f2a43400640 1 -- 172.21.15.22:0/277134551 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 8411+0+0 (secure 0 0 0) 0x7f2a44005040 con 0x555f27b83c00 2024-03-28T18:09:01.222 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:01.218+0000 7f2a6325d740 1 -- 172.21.15.22:0/277134551 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f2a440041c0 con 0x555f27b83c00 2024-03-28T18:09:01.222 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:01.218+0000 7f2a43400640 1 -- 172.21.15.22:0/277134551 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2a4400c710 con 0x555f27b83c00 2024-03-28T18:09:01.222 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:01.218+0000 7f2a6325d740 10 client.15924 fetch_fsmap learned FSMap version 27 2024-03-28T18:09:01.222 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:01.218+0000 7f2a6325d740 1 -- 172.21.15.22:0/277134551 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x555f277fb3f0 con 0x555f27b83c00 2024-03-28T18:09:01.222 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:01.218+0000 7f2a43400640 1 -- 172.21.15.22:0/277134551 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f2a440041c0 con 0x555f27b83c00 2024-03-28T18:09:01.222 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:01.219+0000 7f2a43400640 1 -- 172.21.15.22:0/277134551 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 27) v1 ==== 7934+0+0 (secure 0 0 0) 0x7f2a4402c020 con 0x555f27b83c00 2024-03-28T18:09:01.222 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:01.219+0000 7f2a6325d740 10 client.15924 fetch_fsmap finished waiting for FSMap version 27 2024-03-28T18:09:01.222 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:01.219+0000 7f2a6325d740 10 client.15924 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-28T18:09:01.222 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:01.219+0000 7f2a6325d740 20 client.15924 populate_metadata read hostname 'smithi022' 2024-03-28T18:09:01.222 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:01.219+0000 7f2a6325d740 1 --2- 172.21.15.22:0/277134551 >> [v2:172.21.15.79:6838/2441542443,v1:172.21.15.79:6839/2441542443] conn(0x555f27b9bfe0 0x555f27b9e3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:09:01.223 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:01.219+0000 7f2a6325d740 4 client.15924 mds_command: new command op to 34553 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-28T18:09:01.223 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:01.219+0000 7f2a6325d740 1 -- 172.21.15.22:0/277134551 --> [v2:172.21.15.79:6838/2441542443,v1:172.21.15.79:6839/2441542443] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x555f27af4790 con 0x555f27b9bfe0 2024-03-28T18:09:01.223 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:01.219+0000 7f2a5dc00640 1 --2- 172.21.15.22:0/277134551 >> [v2:172.21.15.79:6838/2441542443,v1:172.21.15.79:6839/2441542443] conn(0x555f27b9bfe0 0x555f27b9e3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:09:01.223 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:01.220+0000 7f2a5dc00640 1 --2- 172.21.15.22:0/277134551 >> [v2:172.21.15.79:6838/2441542443,v1:172.21.15.79:6839/2441542443] conn(0x555f27b9bfe0 0x555f27b9e3d0 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=afee3ea4c4ef4c59 server_cookie=a792b8a604e90b10 in_seq=0 out_seq=0 2024-03-28T18:09:01.223 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:01.220+0000 7f2a43400640 10 client.15924 ms_handle_connect on v2:172.21.15.79:6838/2441542443 2024-03-28T18:09:01.223 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:01.221+0000 7f2a43400640 1 -- 172.21.15.22:0/277134551 <== mds.0 v2:172.21.15.79:6838/2441542443 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x555f27af4790 con 0x555f27b9bfe0 2024-03-28T18:09:01.223 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:01.221+0000 7f2a43400640 10 client.15924 handle_command_reply: tid=0 2024-03-28T18:09:01.223 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:01.221+0000 7f2a6325d740 1 client.15924 shutdown 2024-03-28T18:09:01.223 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:01.221+0000 7f2a6325d740 1 -- 172.21.15.22:0/277134551 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x555f27b83c00 msgr2=0x555f27b88c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:09:01.223 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:01.221+0000 7f2a6325d740 1 --2- 172.21.15.22:0/277134551 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x555f27b83c00 0x555f27b88c30 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7f2a44003f30 tx=0x7f2a44003d90 comp rx=0 tx=0).stop 2024-03-28T18:09:01.223 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:01.221+0000 7f2a6325d740 1 -- 172.21.15.22:0/277134551 shutdown_connections 2024-03-28T18:09:01.223 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:01.221+0000 7f2a6325d740 1 --2- 172.21.15.22:0/277134551 >> [v2:172.21.15.79:6838/2441542443,v1:172.21.15.79:6839/2441542443] conn(0x555f27b9bfe0 0x555f27b9e3d0 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:09:01.223 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:01.221+0000 7f2a6325d740 1 --2- 172.21.15.22:0/277134551 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x555f27b8c290 0x555f27b89170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:09:01.223 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:01.221+0000 7f2a6325d740 1 --2- 172.21.15.22:0/277134551 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x555f27b83c00 0x555f27b88c30 unknown :-1 s=CLOSED pgs=554 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:09:01.223 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:01.221+0000 7f2a6325d740 1 --2- 172.21.15.22:0/277134551 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555f27b82f20 0x555f27b8b8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:09:01.223 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:01.221+0000 7f2a43400640 0 client.15924 ms_handle_reset on v2:172.21.15.79:6838/2441542443 2024-03-28T18:09:01.224 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:01.221+0000 7f2a6325d740 1 -- 172.21.15.22:0/277134551 >> 172.21.15.22:0/277134551 conn(0x555f27b4a200 msgr2=0x555f27b7f6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:09:01.224 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:01.221+0000 7f2a6325d740 1 -- 172.21.15.22:0/277134551 shutdown_connections 2024-03-28T18:09:01.224 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:01.221+0000 7f2a6325d740 1 -- 172.21.15.22:0/277134551 wait complete. 2024-03-28T18:09:01.224 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:01.221+0000 7f2a6325d740 20 client.15924 trim_cache size 0 max 16384 2024-03-28T18:09:01.239 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:01.239+0000 7f2a6325d740 1 Processor -- start 2024-03-28T18:09:01.239 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:01.239+0000 7f2a6325d740 1 -- start start 2024-03-28T18:09:01.241 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:01.240+0000 7f2a6325d740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555f27b82f20 0x555f27b87b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:09:01.241 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:01.240+0000 7f2a6325d740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x555f27b83c00 0x555f27b8b360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:09:01.241 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:01.240+0000 7f2a6325d740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x555f27b8c290 0x555f27b8b8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:09:01.241 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:01.240+0000 7f2a6325d740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a581af1b0 con 0x555f27b83c00 2024-03-28T18:09:01.241 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:01.240+0000 7f2a6325d740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a581b0b30 con 0x555f27b8c290 2024-03-28T18:09:01.241 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:01.240+0000 7f2a6325d740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a581af810 con 0x555f27b82f20 2024-03-28T18:09:01.241 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:01.240+0000 7f2a5d200640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x555f27b83c00 0x555f27b8b360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:09:01.241 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:01.240+0000 7f2a5d200640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x555f27b83c00 0x555f27b8b360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:54436/0 (socket says 172.21.15.22:54436) 2024-03-28T18:09:01.241 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:01.240+0000 7f2a5d200640 1 -- 172.21.15.22:0/4174977832 learned_addr learned my addr 172.21.15.22:0/4174977832 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-28T18:09:01.241 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:01.240+0000 7f2a5e600640 1 --2- 172.21.15.22:0/4174977832 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x555f27b8c290 0x555f27b8b8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:09:01.241 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:01.240+0000 7f2a5dc00640 1 --2- 172.21.15.22:0/4174977832 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555f27b82f20 0x555f27b87b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:09:01.242 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:01.240+0000 7f2a5d200640 1 -- 172.21.15.22:0/4174977832 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555f27b82f20 msgr2=0x555f27b87b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T18:09:01.242 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:01.240+0000 7f2a5d200640 1 --2- 172.21.15.22:0/4174977832 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555f27b82f20 0x555f27b87b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:09:01.242 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:01.240+0000 7f2a5d200640 1 -- 172.21.15.22:0/4174977832 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x555f27b8c290 msgr2=0x555f27b8b8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T18:09:01.242 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:01.240+0000 7f2a5d200640 1 --2- 172.21.15.22:0/4174977832 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x555f27b8c290 0x555f27b8b8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:09:01.242 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:01.240+0000 7f2a5d200640 1 -- 172.21.15.22:0/4174977832 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555f277bb0f0 con 0x555f27b83c00 2024-03-28T18:09:01.242 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:01.241+0000 7f2a5e600640 1 --2- 172.21.15.22:0/4174977832 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x555f27b8c290 0x555f27b8b8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-28T18:09:01.242 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:01.241+0000 7f2a5dc00640 1 --2- 172.21.15.22:0/4174977832 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555f27b82f20 0x555f27b87b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T18:09:01.242 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:01.241+0000 7f2a5d200640 1 --2- 172.21.15.22:0/4174977832 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x555f27b83c00 0x555f27b8b360 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f2a44003d20 tx=0x7f2a44004580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d9e9deab2a8f9017 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:09:01.242 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:01.241+0000 7f2a4e000640 1 -- 172.21.15.22:0/4174977832 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a4401e030 con 0x555f27b83c00 2024-03-28T18:09:01.242 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:01.241+0000 7f2a4e000640 1 -- 172.21.15.22:0/4174977832 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f2a44004b30 con 0x555f27b83c00 2024-03-28T18:09:01.242 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:01.241+0000 7f2a4e000640 1 -- 172.21.15.22:0/4174977832 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a44029070 con 0x555f27b83c00 2024-03-28T18:09:01.245 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:01.242+0000 7f2a6325d740 1 -- 172.21.15.22:0/4174977832 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x555f27b83c00 msgr2=0x555f27b8b360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:09:01.245 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:01.242+0000 7f2a6325d740 1 --2- 172.21.15.22:0/4174977832 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x555f27b83c00 0x555f27b8b360 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f2a44003d20 tx=0x7f2a44004580 comp rx=0 tx=0).stop 2024-03-28T18:09:01.245 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:01.242+0000 7f2a6325d740 1 -- 172.21.15.22:0/4174977832 shutdown_connections 2024-03-28T18:09:01.245 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:01.242+0000 7f2a6325d740 1 --2- 172.21.15.22:0/4174977832 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x555f27b8c290 0x555f27b8b8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:09:01.245 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:01.242+0000 7f2a6325d740 1 --2- 172.21.15.22:0/4174977832 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x555f27b83c00 0x555f27b8b360 unknown :-1 s=CLOSED pgs=555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:09:01.245 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:01.242+0000 7f2a6325d740 1 --2- 172.21.15.22:0/4174977832 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555f27b82f20 0x555f27b87b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:09:01.245 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:01.242+0000 7f2a6325d740 1 -- 172.21.15.22:0/4174977832 >> 172.21.15.22:0/4174977832 conn(0x555f27b4a200 msgr2=0x555f27b873f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:09:01.245 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:01.242+0000 7f2a6325d740 1 -- 172.21.15.22:0/4174977832 shutdown_connections 2024-03-28T18:09:01.245 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:01.242+0000 7f2a6325d740 1 -- 172.21.15.22:0/4174977832 wait complete. 2024-03-28T18:09:01.245 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:01.242+0000 7f2a6325d740 1 -- start start 2024-03-28T18:09:01.246 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:01.243+0000 7f2a6325d740 1 Processor -- start 2024-03-28T18:09:01.246 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:01.243+0000 7f2a6325d740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x555f27b82f20 0x555f27b868a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:09:01.246 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:01.243+0000 7f2a6325d740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555f27b83c00 0x555f27b852d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:09:01.246 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:01.243+0000 7f2a6325d740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x555f27b8c290 0x555f27b858c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:09:01.246 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:01.243+0000 7f2a6325d740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a581b04d0 con 0x555f27b8c290 2024-03-28T18:09:01.246 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:01.243+0000 7f2a6325d740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a581afe70 con 0x555f27b82f20 2024-03-28T18:09:01.246 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:01.243+0000 7f2a6325d740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a581ad390 con 0x555f27b83c00 2024-03-28T18:09:01.246 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:01.243+0000 7f2a6325d740 10 client.0 _ll_register_callbacks cb 0x7f2a44014070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-28T18:09:01.246 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:01.243+0000 7f2a5e600640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x555f27b8c290 0x555f27b858c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:09:01.246 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:01.243+0000 7f2a5e600640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x555f27b8c290 0x555f27b858c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:54450/0 (socket says 172.21.15.22:54450) 2024-03-28T18:09:01.246 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:01.243+0000 7f2a5e600640 1 -- 172.21.15.22:0/3374558973 learned_addr learned my addr 172.21.15.22:0/3374558973 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-28T18:09:01.246 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:01.243+0000 7f2a5dc00640 1 --2- 172.21.15.22:0/3374558973 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x555f27b82f20 0x555f27b868a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:09:01.246 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:01.244+0000 7f2a5d200640 1 --2- 172.21.15.22:0/3374558973 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555f27b83c00 0x555f27b852d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:09:01.246 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:01.244+0000 7f2a5e600640 1 -- 172.21.15.22:0/3374558973 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555f27b83c00 msgr2=0x555f27b852d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T18:09:01.247 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:01.244+0000 7f2a5e600640 1 --2- 172.21.15.22:0/3374558973 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555f27b83c00 0x555f27b852d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:09:01.247 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:01.244+0000 7f2a5e600640 1 -- 172.21.15.22:0/3374558973 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x555f27b82f20 msgr2=0x555f27b868a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T18:09:01.247 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:01.244+0000 7f2a5e600640 1 --2- 172.21.15.22:0/3374558973 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x555f27b82f20 0x555f27b868a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:09:01.247 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:01.244+0000 7f2a5e600640 1 -- 172.21.15.22:0/3374558973 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x555f27b860f0 con 0x555f27b8c290 2024-03-28T18:09:01.247 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:01.244+0000 7f2a5e600640 1 --2- 172.21.15.22:0/3374558973 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x555f27b8c290 0x555f27b858c0 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7f2a50003760 tx=0x7f2a500046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d1063e0e9d92c296 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:09:01.247 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:01.244+0000 7f2a43400640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-03-28T18:09:01.247 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:01.244+0000 7f2a43400640 1 -- 172.21.15.22:0/3374558973 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a5002e060 con 0x555f27b8c290 2024-03-28T18:09:01.247 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:01.245+0000 7f2a43400640 1 -- 172.21.15.22:0/3374558973 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 8411+0+0 (secure 0 0 0) 0x7f2a5001f8c0 con 0x555f27b8c290 2024-03-28T18:09:01.247 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:01.245+0000 7f2a6325d740 1 -- 172.21.15.22:0/3374558973 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f2a44004b30 con 0x555f27b8c290 2024-03-28T18:09:01.247 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:01.245+0000 7f2a43400640 1 -- 172.21.15.22:0/3374558973 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2a5001fac0 con 0x555f27b8c290 2024-03-28T18:09:01.247 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:01.245+0000 7f2a6325d740 10 client.15930 fetch_fsmap learned FSMap version 27 2024-03-28T18:09:01.247 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:01.245+0000 7f2a6325d740 1 -- 172.21.15.22:0/3374558973 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x555f27bf7650 con 0x555f27b8c290 2024-03-28T18:09:01.247 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:01.245+0000 7f2a43400640 1 -- 172.21.15.22:0/3374558973 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f2a44004b30 con 0x555f27b8c290 2024-03-28T18:09:01.247 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:01.245+0000 7f2a43400640 1 -- 172.21.15.22:0/3374558973 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 27) v1 ==== 7934+0+0 (secure 0 0 0) 0x7f2a50031020 con 0x555f27b8c290 2024-03-28T18:09:01.247 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:01.245+0000 7f2a6325d740 10 client.15930 fetch_fsmap finished waiting for FSMap version 27 2024-03-28T18:09:01.248 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:01.245+0000 7f2a6325d740 10 client.15930 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-28T18:09:01.248 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:01.245+0000 7f2a6325d740 20 client.15930 populate_metadata read hostname 'smithi022' 2024-03-28T18:09:01.248 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:01.245+0000 7f2a6325d740 1 --2- 172.21.15.22:0/3374558973 >> [v2:172.21.15.79:6838/2441542443,v1:172.21.15.79:6839/2441542443] conn(0x555f27bf7de0 0x555f27bfa200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:09:01.248 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:01.245+0000 7f2a6325d740 4 client.15930 mds_command: new command op to 34553 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-28T18:09:01.248 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:01.245+0000 7f2a6325d740 1 -- 172.21.15.22:0/3374558973 --> [v2:172.21.15.79:6838/2441542443,v1:172.21.15.79:6839/2441542443] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x555f27aeeae0 con 0x555f27bf7de0 2024-03-28T18:09:01.249 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:01.246+0000 7f2a5dc00640 1 --2- 172.21.15.22:0/3374558973 >> [v2:172.21.15.79:6838/2441542443,v1:172.21.15.79:6839/2441542443] conn(0x555f27bf7de0 0x555f27bfa200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:09:01.249 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:01.247+0000 7f2a5dc00640 1 --2- 172.21.15.22:0/3374558973 >> [v2:172.21.15.79:6838/2441542443,v1:172.21.15.79:6839/2441542443] conn(0x555f27bf7de0 0x555f27bfa200 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=adf8679766d337ef server_cookie=5f9540a5013a8756 in_seq=0 out_seq=0 2024-03-28T18:09:01.249 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:01.247+0000 7f2a43400640 10 client.15930 ms_handle_connect on v2:172.21.15.79:6838/2441542443 2024-03-28T18:09:01.249 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:01.247+0000 7f2a43400640 1 -- 172.21.15.22:0/3374558973 <== mds.0 v2:172.21.15.79:6838/2441542443 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x555f27aeeae0 con 0x555f27bf7de0 2024-03-28T18:09:01.249 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:01.247+0000 7f2a43400640 10 client.15930 handle_command_reply: tid=0 2024-03-28T18:09:01.249 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:01.247+0000 7f2a6325d740 1 client.15930 shutdown 2024-03-28T18:09:01.249 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:01.247+0000 7f2a6325d740 1 -- 172.21.15.22:0/3374558973 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x555f27b8c290 msgr2=0x555f27b858c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:09:01.250 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:01.247+0000 7f2a6325d740 1 --2- 172.21.15.22:0/3374558973 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x555f27b8c290 0x555f27b858c0 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7f2a50003760 tx=0x7f2a500046c0 comp rx=0 tx=0).stop 2024-03-28T18:09:01.250 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:01.247+0000 7f2a6325d740 1 -- 172.21.15.22:0/3374558973 shutdown_connections 2024-03-28T18:09:01.250 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:01.247+0000 7f2a6325d740 1 --2- 172.21.15.22:0/3374558973 >> [v2:172.21.15.79:6838/2441542443,v1:172.21.15.79:6839/2441542443] conn(0x555f27bf7de0 0x555f27bfa200 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:09:01.250 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:01.247+0000 7f2a6325d740 1 --2- 172.21.15.22:0/3374558973 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x555f27b8c290 0x555f27b858c0 unknown :-1 s=CLOSED pgs=556 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:09:01.250 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:01.247+0000 7f2a6325d740 1 --2- 172.21.15.22:0/3374558973 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555f27b83c00 0x555f27b852d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:09:01.250 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:01.247+0000 7f2a43400640 0 client.15930 ms_handle_reset on v2:172.21.15.79:6838/2441542443 2024-03-28T18:09:01.250 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:01.247+0000 7f2a6325d740 1 --2- 172.21.15.22:0/3374558973 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x555f27b82f20 0x555f27b868a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:09:01.250 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:01.247+0000 7f2a6325d740 1 -- 172.21.15.22:0/3374558973 >> 172.21.15.22:0/3374558973 conn(0x555f27b4a200 msgr2=0x555f27b87c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:09:01.250 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:01.247+0000 7f2a6325d740 1 -- 172.21.15.22:0/3374558973 shutdown_connections 2024-03-28T18:09:01.250 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:01.247+0000 7f2a6325d740 1 -- 172.21.15.22:0/3374558973 wait complete. 2024-03-28T18:09:01.250 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:01.248+0000 7f2a6325d740 20 client.15930 trim_cache size 0 max 16384 2024-03-28T18:09:01.250 INFO:teuthology.orchestra.run.smithi022.stdout:{ 2024-03-28T18:09:01.251 INFO:teuthology.orchestra.run.smithi022.stdout: "status": "no active scrubs running", 2024-03-28T18:09:01.251 INFO:teuthology.orchestra.run.smithi022.stdout: "scrubs": {} 2024-03-28T18:09:01.251 INFO:teuthology.orchestra.run.smithi022.stdout:} 2024-03-28T18:09:01.251 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:01.248+0000 7f2a60600640 1 -- 172.21.15.22:0/1381615903 >> [v2:172.21.15.22:6800/1357365082,v1:172.21.15.22:6801/1357365082] conn(0x7f2a3c0a2b10 msgr2=0x7f2a3c0a4fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:09:01.251 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:01.248+0000 7f2a60600640 1 --2- 172.21.15.22:0/1381615903 >> [v2:172.21.15.22:6800/1357365082,v1:172.21.15.22:6801/1357365082] conn(0x7f2a3c0a2b10 0x7f2a3c0a4fd0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f2a50009c70 tx=0x7f2a50005f90 comp rx=0 tx=0).stop 2024-03-28T18:09:01.251 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:01.248+0000 7f2a60600640 1 -- 172.21.15.22:0/1381615903 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2a58103840 msgr2=0x7f2a5819a1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:09:01.251 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:01.248+0000 7f2a60600640 1 --2- 172.21.15.22:0/1381615903 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2a58103840 0x7f2a5819a1f0 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7f2a48000f80 tx=0x7f2a48034cf0 comp rx=0 tx=0).stop 2024-03-28T18:09:01.252 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:01.248+0000 7f2a60600640 1 -- 172.21.15.22:0/1381615903 shutdown_connections 2024-03-28T18:09:01.252 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:01.248+0000 7f2a60600640 1 --2- 172.21.15.22:0/1381615903 >> [v2:172.21.15.22:6800/1357365082,v1:172.21.15.22:6801/1357365082] conn(0x7f2a3c0a2b10 0x7f2a3c0a4fd0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:09:01.252 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:01.248+0000 7f2a60600640 1 --2- 172.21.15.22:0/1381615903 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f2a58109850 0x7f2a5819eab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:09:01.252 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:01.248+0000 7f2a60600640 1 --2- 172.21.15.22:0/1381615903 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2a58104200 0x7f2a5819a730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:09:01.252 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:01.248+0000 7f2a60600640 1 --2- 172.21.15.22:0/1381615903 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2a58103840 0x7f2a5819a1f0 unknown :-1 s=CLOSED pgs=553 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:09:01.252 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:01.248+0000 7f2a60600640 1 -- 172.21.15.22:0/1381615903 >> 172.21.15.22:0/1381615903 conn(0x7f2a580ff350 msgr2=0x7f2a580ffb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:09:01.252 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:01.248+0000 7f2a60600640 1 -- 172.21.15.22:0/1381615903 shutdown_connections 2024-03-28T18:09:01.252 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:01.248+0000 7f2a60600640 1 -- 172.21.15.22:0/1381615903 wait complete. 2024-03-28T18:09:01.267 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-03-28T18:09:01.267 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-28T18:09:01.472 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:01.472+0000 7f02d8c00640 1 -- 172.21.15.22:0/3097061874 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f02d4109840 msgr2=0x7f02d4109c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:09:01.473 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:01.472+0000 7f02d8c00640 1 --2- 172.21.15.22:0/3097061874 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f02d4109840 0x7f02d4109c20 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7f02c4009930 tx=0x7f02c402bc20 comp rx=0 tx=0).stop 2024-03-28T18:09:01.473 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:01.473+0000 7f02d8c00640 1 -- 172.21.15.22:0/3097061874 shutdown_connections 2024-03-28T18:09:01.473 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:01.473+0000 7f02d8c00640 1 --2- 172.21.15.22:0/3097061874 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f02d41041f0 0x7f02d4110700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:09:01.473 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:01.473+0000 7f02d8c00640 1 --2- 172.21.15.22:0/3097061874 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f02d4103830 0x7f02d4103cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:09:01.473 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:01.473+0000 7f02d8c00640 1 --2- 172.21.15.22:0/3097061874 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f02d4109840 0x7f02d4109c20 unknown :-1 s=CLOSED pgs=557 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:09:01.473 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:01.473+0000 7f02d8c00640 1 -- 172.21.15.22:0/3097061874 >> 172.21.15.22:0/3097061874 conn(0x7f02d40ff300 msgr2=0x7f02d4101720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:09:01.473 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:01.473+0000 7f02d8c00640 1 -- 172.21.15.22:0/3097061874 shutdown_connections 2024-03-28T18:09:01.473 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:01.473+0000 7f02d8c00640 1 -- 172.21.15.22:0/3097061874 wait complete. 2024-03-28T18:09:01.474 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:01.474+0000 7f02d8c00640 1 Processor -- start 2024-03-28T18:09:01.474 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:01.474+0000 7f02d8c00640 1 -- start start 2024-03-28T18:09:01.475 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:01.474+0000 7f02d8c00640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f02d4103830 0x7f02d419a1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:09:01.475 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:01.474+0000 7f02d8c00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f02d41041f0 0x7f02d419a700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:09:01.475 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:01.475+0000 7f02d8c00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f02d4109840 0x7f02d419ea30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:09:01.475 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:01.475+0000 7f02d8c00640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02d4113c40 con 0x7f02d4109840 2024-03-28T18:09:01.475 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:01.475+0000 7f02d8c00640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02d4113ac0 con 0x7f02d4103830 2024-03-28T18:09:01.475 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:01.475+0000 7f02d8c00640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02d4113dc0 con 0x7f02d41041f0 2024-03-28T18:09:01.475 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:01.475+0000 7f02d2800640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f02d4109840 0x7f02d419ea30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:09:01.475 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:01.475+0000 7f02d2800640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f02d4109840 0x7f02d419ea30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:54474/0 (socket says 172.21.15.22:54474) 2024-03-28T18:09:01.475 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:01.475+0000 7f02d2800640 1 -- 172.21.15.22:0/2398483308 learned_addr learned my addr 172.21.15.22:0/2398483308 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-28T18:09:01.475 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:01.475+0000 7f02d1e00640 1 --2- 172.21.15.22:0/2398483308 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f02d4103830 0x7f02d419a1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:09:01.475 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:01.475+0000 7f02d1400640 1 --2- 172.21.15.22:0/2398483308 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f02d41041f0 0x7f02d419a700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:09:01.476 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:01.475+0000 7f02d2800640 1 -- 172.21.15.22:0/2398483308 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f02d41041f0 msgr2=0x7f02d419a700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:09:01.476 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:01.475+0000 7f02d2800640 1 --2- 172.21.15.22:0/2398483308 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f02d41041f0 0x7f02d419a700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:09:01.476 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:01.475+0000 7f02d2800640 1 -- 172.21.15.22:0/2398483308 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f02d4103830 msgr2=0x7f02d419a1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:09:01.476 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:01.475+0000 7f02d2800640 1 --2- 172.21.15.22:0/2398483308 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f02d4103830 0x7f02d419a1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:09:01.476 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:01.475+0000 7f02d2800640 1 -- 172.21.15.22:0/2398483308 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f02d419f160 con 0x7f02d4109840 2024-03-28T18:09:01.477 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:01.476+0000 7f02d2800640 1 --2- 172.21.15.22:0/2398483308 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f02d4109840 0x7f02d419ea30 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f02cc00eaa0 tx=0x7f02cc00ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:09:01.477 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:01.476+0000 7f02caa00640 1 -- 172.21.15.22:0/2398483308 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02cc00cdb0 con 0x7f02d4109840 2024-03-28T18:09:01.477 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:01.476+0000 7f02d8c00640 1 -- 172.21.15.22:0/2398483308 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f02d419f450 con 0x7f02d4109840 2024-03-28T18:09:01.477 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:01.476+0000 7f02caa00640 1 -- 172.21.15.22:0/2398483308 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f02cc005020 con 0x7f02d4109840 2024-03-28T18:09:01.477 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:01.476+0000 7f02caa00640 1 -- 172.21.15.22:0/2398483308 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02cc0116b0 con 0x7f02d4109840 2024-03-28T18:09:01.477 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:01.476+0000 7f02d8c00640 1 -- 172.21.15.22:0/2398483308 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f02d41a6ce0 con 0x7f02d4109840 2024-03-28T18:09:01.477 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:01.478+0000 7f02d8c00640 1 -- 172.21.15.22:0/2398483308 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f02d40688e0 con 0x7f02d4109840 2024-03-28T18:09:01.479 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:01.480+0000 7f02caa00640 1 -- 172.21.15.22:0/2398483308 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 25) v1 ==== 137523+0+0 (secure 0 0 0) 0x7f02cc0051d0 con 0x7f02d4109840 2024-03-28T18:09:01.481 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:01.481+0000 7f02caa00640 1 --2- 172.21.15.22:0/2398483308 >> [v2:172.21.15.22:6800/1357365082,v1:172.21.15.22:6801/1357365082] conn(0x7f02b80a2b10 0x7f02b80a4fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:09:01.481 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:01.481+0000 7f02caa00640 1 -- 172.21.15.22:0/2398483308 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 8411+0+0 (secure 0 0 0) 0x7f02cc0cf1f0 con 0x7f02d4109840 2024-03-28T18:09:01.481 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:01.481+0000 7f02d1e00640 1 --2- 172.21.15.22:0/2398483308 >> [v2:172.21.15.22:6800/1357365082,v1:172.21.15.22:6801/1357365082] conn(0x7f02b80a2b10 0x7f02b80a4fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:09:01.481 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:01.482+0000 7f02d1e00640 1 --2- 172.21.15.22:0/2398483308 >> [v2:172.21.15.22:6800/1357365082,v1:172.21.15.22:6801/1357365082] conn(0x7f02b80a2b10 0x7f02b80a4fd0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f02c4036040 tx=0x7f02c4000bc0 comp rx=0 tx=0).ready entity=mgr.15522 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:09:01.485 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:01.486+0000 7f02caa00640 1 -- 172.21.15.22:0/2398483308 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f02cc016030 con 0x7f02d4109840 2024-03-28T18:09:01.488 INFO:tasks.workunit.client.0.smithi022.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_sync_io.c -o test_sync_io 2024-03-28T18:09:01.719 INFO:tasks.workunit.client.0.smithi022.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_short_dio_read.c -o test_short_dio_read 2024-03-28T18:09:01.728 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:01.728+0000 7f02d8c00640 1 -- 172.21.15.22:0/2398483308 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f02d410b810 con 0x7f02d4109840 2024-03-28T18:09:01.730 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:01.730+0000 7f02caa00640 1 -- 172.21.15.22:0/2398483308 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) v1 ==== 94+0+11452 (secure 0 0 0) 0x7f02cc09b6c0 con 0x7f02d4109840 2024-03-28T18:09:01.730 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-03-28T18:09:01.730 INFO:teuthology.orchestra.run.smithi022.stdout:{"epoch":27,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14544,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6835/3718238950","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6834","nonce":3718238950},{"type":"v1","addr":"172.21.15.22:6835","nonce":3718238950}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14562,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6837/3676939386","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6836","nonce":3676939386},{"type":"v1","addr":"172.21.15.22:6837","nonce":3676939386}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":27,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-28T18:04:48.991622+0000","modified":"2024-03-28T18:07:13.521383+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":34553,"mds_1":34511,"mds_2":34475,"mds_3":34442,"mds_4":24502},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14640":{"gid":14640,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.22:6839/3979998310","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6838","nonce":3979998310},{"type":"v1","addr":"172.21.15.22:6839","nonce":3979998310}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_14655":{"gid":14655,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.22:6841/3069056661","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6840","nonce":3069056661},{"type":"v1","addr":"172.21.15.22:6841","nonce":3069056661}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24412":{"gid":24412,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.134:6833/3176125362","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6832","nonce":3176125362},{"type":"v1","addr":"172.21.15.134:6833","nonce":3176125362}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24463":{"gid":24463,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.134:6835/3370549207","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6834","nonce":3370549207},{"type":"v1","addr":"172.21.15.134:6835","nonce":3370549207}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24478":{"gid":24478,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.134:6837/899825581","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6836","nonce":899825581},{"type":"v1","addr":"172.21.15.134:6837","nonce":899825581}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24502":{"gid":24502,"name":"i","rank":4,"incarnation":24,"state":"up:active","state_seq":7,"addr":"172.21.15.134:6839/3400263217","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6838","nonce":3400263217},{"type":"v1","addr":"172.21.15.134:6839","nonce":3400263217}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_34442":{"gid":34442,"name":"k","rank":3,"incarnation":22,"state":"up:active","state_seq":14,"addr":"172.21.15.79:6833/856896997","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6832","nonce":856896997},{"type":"v1","addr":"172.21.15.79:6833","nonce":856896997}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_34475":{"gid":34475,"name":"e","rank":2,"incarnation":20,"state":"up:active","state_seq":11,"addr":"172.21.15.79:6835/3365773431","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6834","nonce":3365773431},{"type":"v1","addr":"172.21.15.79:6835","nonce":3365773431}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_34511":{"gid":34511,"name":"h","rank":1,"incarnation":18,"state":"up:active","state_seq":8,"addr":"172.21.15.79:6837/1787903366","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6836","nonce":1787903366},{"type":"v1","addr":"172.21.15.79:6837","nonce":1787903366}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_34553":{"gid":34553,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.79:6839/2441542443","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6838","nonce":2441542443},{"type":"v1","addr":"172.21.15.79:6839","nonce":2441542443}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":34553,"qdb_cluster":[24502,34442,34475,34511,34553]},"id":1}]} 2024-03-28T18:09:01.730 INFO:teuthology.orchestra.run.smithi022.stderr:dumped fsmap epoch 27 2024-03-28T18:09:01.732 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:01.732+0000 7f02d8c00640 1 -- 172.21.15.22:0/2398483308 >> [v2:172.21.15.22:6800/1357365082,v1:172.21.15.22:6801/1357365082] conn(0x7f02b80a2b10 msgr2=0x7f02b80a4fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:09:01.732 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:01.732+0000 7f02d8c00640 1 --2- 172.21.15.22:0/2398483308 >> [v2:172.21.15.22:6800/1357365082,v1:172.21.15.22:6801/1357365082] conn(0x7f02b80a2b10 0x7f02b80a4fd0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f02c4036040 tx=0x7f02c4000bc0 comp rx=0 tx=0).stop 2024-03-28T18:09:01.732 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:01.732+0000 7f02d8c00640 1 -- 172.21.15.22:0/2398483308 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f02d4109840 msgr2=0x7f02d419ea30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:09:01.732 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:01.732+0000 7f02d8c00640 1 --2- 172.21.15.22:0/2398483308 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f02d4109840 0x7f02d419ea30 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f02cc00eaa0 tx=0x7f02cc00ef60 comp rx=0 tx=0).stop 2024-03-28T18:09:01.733 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:01.732+0000 7f02d8c00640 1 -- 172.21.15.22:0/2398483308 shutdown_connections 2024-03-28T18:09:01.733 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:01.732+0000 7f02d8c00640 1 --2- 172.21.15.22:0/2398483308 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f02d4109840 0x7f02d419ea30 unknown :-1 s=CLOSED pgs=558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:09:01.733 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:01.732+0000 7f02d8c00640 1 --2- 172.21.15.22:0/2398483308 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f02d41041f0 0x7f02d419a700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:09:01.733 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:01.732+0000 7f02d8c00640 1 --2- 172.21.15.22:0/2398483308 >> [v2:172.21.15.22:6800/1357365082,v1:172.21.15.22:6801/1357365082] conn(0x7f02b80a2b10 0x7f02b80a4fd0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:09:01.733 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:01.732+0000 7f02d8c00640 1 --2- 172.21.15.22:0/2398483308 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f02d4103830 0x7f02d419a1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:09:01.733 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:01.733+0000 7f02d8c00640 1 -- 172.21.15.22:0/2398483308 >> 172.21.15.22:0/2398483308 conn(0x7f02d40ff300 msgr2=0x7f02d40ffb00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:09:01.733 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:01.733+0000 7f02d8c00640 1 -- 172.21.15.22:0/2398483308 shutdown_connections 2024-03-28T18:09:01.733 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:01.733+0000 7f02d8c00640 1 -- 172.21.15.22:0/2398483308 wait complete. 2024-03-28T18:09:01.753 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-28T18:09:01.753 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2024-03-28T18:09:01.854 INFO:tasks.workunit.client.0.smithi022.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-03-28T18:09:01.861 INFO:tasks.workunit.client.0.smithi022.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-03-28T18:09:01.862 INFO:tasks.workunit.client.0.smithi022.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_o_trunc.c -o test_o_trunc 2024-03-28T18:09:01.952 INFO:tasks.workunit.client.0.smithi022.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-03-28T18:09:01.961 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:01.961+0000 7f94cf200640 1 -- 172.21.15.22:0/2533545819 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f94d0109730 msgr2=0x7f94d0109b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:09:01.961 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:01.961+0000 7f94cf200640 1 --2- 172.21.15.22:0/2533545819 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f94d0109730 0x7f94d0109b10 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7f94c0009930 tx=0x7f94c002bc30 comp rx=0 tx=0).stop 2024-03-28T18:09:01.961 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:01.961+0000 7f94cf200640 1 -- 172.21.15.22:0/2533545819 shutdown_connections 2024-03-28T18:09:01.961 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:01.961+0000 7f94cf200640 1 --2- 172.21.15.22:0/2533545819 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f94d01040e0 0x7f94d01105f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:09:01.961 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:01.961+0000 7f94cf200640 1 --2- 172.21.15.22:0/2533545819 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f94d0103720 0x7f94d0103ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:09:01.961 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:01.961+0000 7f94cf200640 1 --2- 172.21.15.22:0/2533545819 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f94d0109730 0x7f94d0109b10 unknown :-1 s=CLOSED pgs=559 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:09:01.961 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:01.961+0000 7f94cf200640 1 -- 172.21.15.22:0/2533545819 >> 172.21.15.22:0/2533545819 conn(0x7f94d00ff1b0 msgr2=0x7f94d01015d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:09:01.961 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:01.961+0000 7f94cf200640 1 -- 172.21.15.22:0/2533545819 shutdown_connections 2024-03-28T18:09:01.961 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:01.961+0000 7f94cf200640 1 -- 172.21.15.22:0/2533545819 wait complete. 2024-03-28T18:09:01.962 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:01.962+0000 7f94cf200640 1 Processor -- start 2024-03-28T18:09:01.962 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:01.962+0000 7f94cf200640 1 -- start start 2024-03-28T18:09:01.962 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:01.963+0000 7f94cf200640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f94d0103720 0x7f94d019a360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:09:01.963 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:01.963+0000 7f94cf200640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f94d01040e0 0x7f94d019a8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:09:01.963 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:01.963+0000 7f94cf200640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f94d0109730 0x7f94d019ec20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:09:01.963 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:01.963+0000 7f94cf200640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94d0113b20 con 0x7f94d0103720 2024-03-28T18:09:01.964 DEBUG:teuthology.orchestra.run.smithi022:> set -ex 2024-03-28T18:09:01.964 DEBUG:teuthology.orchestra.run.smithi022:> dd if=/home/ubuntu/cephtest/workunits.list.client.0 of=/dev/stdout 2024-03-28T18:09:01.965 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:01.963+0000 7f94cf200640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94d01139a0 con 0x7f94d01040e0 2024-03-28T18:09:01.965 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:01.963+0000 7f94cf200640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94d0113ca0 con 0x7f94d0109730 2024-03-28T18:09:01.966 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:01.963+0000 7f94cde00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f94d0103720 0x7f94d019a360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:09:01.966 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:01.963+0000 7f94cde00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f94d0103720 0x7f94d019a360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:54490/0 (socket says 172.21.15.22:54490) 2024-03-28T18:09:01.966 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:01.963+0000 7f94cde00640 1 -- 172.21.15.22:0/2803155755 learned_addr learned my addr 172.21.15.22:0/2803155755 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-28T18:09:01.966 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:01.963+0000 7f94cd400640 1 --2- 172.21.15.22:0/2803155755 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f94d01040e0 0x7f94d019a8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:09:01.966 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:01.963+0000 7f94ce800640 1 --2- 172.21.15.22:0/2803155755 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f94d0109730 0x7f94d019ec20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:09:01.966 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:01.964+0000 7f94cde00640 1 -- 172.21.15.22:0/2803155755 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f94d0109730 msgr2=0x7f94d019ec20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:09:01.966 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:01.964+0000 7f94cde00640 1 --2- 172.21.15.22:0/2803155755 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f94d0109730 0x7f94d019ec20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:09:01.966 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:01.964+0000 7f94cde00640 1 -- 172.21.15.22:0/2803155755 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f94d01040e0 msgr2=0x7f94d019a8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:09:01.966 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:01.964+0000 7f94cde00640 1 --2- 172.21.15.22:0/2803155755 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f94d01040e0 0x7f94d019a8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:09:01.966 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:01.964+0000 7f94cde00640 1 -- 172.21.15.22:0/2803155755 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f94d019f350 con 0x7f94d0103720 2024-03-28T18:09:01.966 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:01.964+0000 7f94cde00640 1 --2- 172.21.15.22:0/2803155755 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f94d0103720 0x7f94d019a360 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7f94c0000c00 tx=0x7f94c0005020 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:09:01.966 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:01.964+0000 7f94c6a00640 1 -- 172.21.15.22:0/2803155755 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94c0043020 con 0x7f94d0103720 2024-03-28T18:09:01.966 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:01.964+0000 7f94c6a00640 1 -- 172.21.15.22:0/2803155755 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f94c0005420 con 0x7f94d0103720 2024-03-28T18:09:01.966 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:01.964+0000 7f94c6a00640 1 -- 172.21.15.22:0/2803155755 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94c0003bf0 con 0x7f94d0103720 2024-03-28T18:09:01.966 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:01.964+0000 7f94cf200640 1 -- 172.21.15.22:0/2803155755 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f94d019f5e0 con 0x7f94d0103720 2024-03-28T18:09:01.967 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:01.964+0000 7f94cf200640 1 -- 172.21.15.22:0/2803155755 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f94d01a6ed0 con 0x7f94d0103720 2024-03-28T18:09:01.967 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:01.968+0000 7f94d7b86740 1 Processor -- start 2024-03-28T18:09:01.967 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:01.968+0000 7f94d7b86740 1 -- start start 2024-03-28T18:09:01.973 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:01.968+0000 7f94c6a00640 1 -- 172.21.15.22:0/2803155755 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 25) v1 ==== 137523+0+0 (secure 0 0 0) 0x7f94c0046020 con 0x7f94d0103720 2024-03-28T18:09:01.973 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:01.968+0000 7f94d7b86740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d3b56758b0 0x55d3b5675c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:09:01.973 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:01.968+0000 7f94d7b86740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55d3b5676260 0x55d3b567e810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:09:01.973 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:01.968+0000 7f94d7b86740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d3b567ede0 0x55d3b5681230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:09:01.973 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:01.968+0000 7f94d7b86740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d3b55d92c0 con 0x55d3b56758b0 2024-03-28T18:09:01.973 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:01.968+0000 7f94d7b86740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d3b5581c90 con 0x55d3b5676260 2024-03-28T18:09:01.973 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:01.968+0000 7f94d7b86740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d3b55d5360 con 0x55d3b567ede0 2024-03-28T18:09:01.973 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:01.968+0000 7f94ce800640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d3b56758b0 0x55d3b5675c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:09:01.974 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:01.969+0000 7f94cde00640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55d3b5676260 0x55d3b567e810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:09:01.974 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:01.969+0000 7f94ce800640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d3b56758b0 0x55d3b5675c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:54498/0 (socket says 172.21.15.22:54498) 2024-03-28T18:09:01.974 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:01.969+0000 7f94cde00640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55d3b5676260 0x55d3b567e810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.22:35712/0 (socket says 172.21.15.22:35712) 2024-03-28T18:09:01.974 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:01.969+0000 7f94ce800640 1 -- 172.21.15.22:0/1065748174 learned_addr learned my addr 172.21.15.22:0/1065748174 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-28T18:09:01.974 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:01.969+0000 7f94cd400640 1 --2- 172.21.15.22:0/1065748174 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d3b567ede0 0x55d3b5681230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:09:01.974 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:01.969+0000 7f94cde00640 1 -- 172.21.15.22:0/1065748174 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d3b567ede0 msgr2=0x55d3b5681230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T18:09:01.974 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:01.969+0000 7f94cde00640 1 --2- 172.21.15.22:0/1065748174 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d3b567ede0 0x55d3b5681230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:09:01.974 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:01.969+0000 7f94cde00640 1 -- 172.21.15.22:0/1065748174 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d3b56758b0 msgr2=0x55d3b5675c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T18:09:01.974 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:01.969+0000 7f94cde00640 1 --2- 172.21.15.22:0/1065748174 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d3b56758b0 0x55d3b5675c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:09:01.974 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:01.969+0000 7f94cde00640 1 -- 172.21.15.22:0/1065748174 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d3b55a6080 con 0x55d3b5676260 2024-03-28T18:09:01.974 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:01.969+0000 7f94cde00640 1 --2- 172.21.15.22:0/1065748174 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55d3b5676260 0x55d3b567e810 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f94c00ee810 tx=0x7f94c00f9270 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bce46a637c7a7a35 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:09:01.974 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:01.970+0000 7f94c6a00640 1 --2- 172.21.15.22:0/2803155755 >> [v2:172.21.15.22:6800/1357365082,v1:172.21.15.22:6801/1357365082] conn(0x7f94b40a2b10 0x7f94b40a4fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:09:01.974 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:01.970+0000 7f94bb400640 1 -- 172.21.15.22:0/1065748174 <== mon.2 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94c00f9a70 con 0x55d3b5676260 2024-03-28T18:09:01.975 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:01.970+0000 7f94bb400640 1 -- 172.21.15.22:0/1065748174 <== mon.2 v2:172.21.15.79:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f94c00fe030 con 0x55d3b5676260 2024-03-28T18:09:01.975 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:01.970+0000 7f94bb400640 1 -- 172.21.15.22:0/1065748174 <== mon.2 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94c008b5c0 con 0x55d3b5676260 2024-03-28T18:09:01.975 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:01.970+0000 7f94c6a00640 1 -- 172.21.15.22:0/2803155755 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 8411+0+0 (secure 0 0 0) 0x7f94c00f10c0 con 0x7f94d0103720 2024-03-28T18:09:01.975 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:01.970+0000 7f94ce800640 1 --2- 172.21.15.22:0/2803155755 >> [v2:172.21.15.22:6800/1357365082,v1:172.21.15.22:6801/1357365082] conn(0x7f94b40a2b10 0x7f94b40a4fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:09:01.975 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:01.971+0000 7f94ce800640 1 --2- 172.21.15.22:0/2803155755 >> [v2:172.21.15.22:6800/1357365082,v1:172.21.15.22:6801/1357365082] conn(0x7f94b40a2b10 0x7f94b40a4fd0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f94c8009370 tx=0x7f94c8009820 comp rx=0 tx=0).ready entity=mgr.15522 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:09:01.975 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:01.971+0000 7f94d7b86740 1 -- 172.21.15.22:0/1065748174 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55d3b5676260 msgr2=0x55d3b567e810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:09:01.975 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:01.971+0000 7f94d7b86740 1 --2- 172.21.15.22:0/1065748174 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55d3b5676260 0x55d3b567e810 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f94c00ee810 tx=0x7f94c00f9270 comp rx=0 tx=0).stop 2024-03-28T18:09:01.975 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:01.971+0000 7f94d7b86740 1 -- 172.21.15.22:0/1065748174 shutdown_connections 2024-03-28T18:09:01.975 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:01.971+0000 7f94d7b86740 1 --2- 172.21.15.22:0/1065748174 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d3b567ede0 0x55d3b5681230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:09:01.975 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:01.971+0000 7f94d7b86740 1 --2- 172.21.15.22:0/1065748174 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55d3b5676260 0x55d3b567e810 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:09:01.975 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:01.971+0000 7f94d7b86740 1 --2- 172.21.15.22:0/1065748174 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d3b56758b0 0x55d3b5675c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:09:01.975 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:01.971+0000 7f94d7b86740 1 -- 172.21.15.22:0/1065748174 >> 172.21.15.22:0/1065748174 conn(0x55d3b566ff80 msgr2=0x55d3b5670360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:09:01.975 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:01.971+0000 7f94d7b86740 1 -- 172.21.15.22:0/1065748174 shutdown_connections 2024-03-28T18:09:01.975 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:01.971+0000 7f94d7b86740 1 -- 172.21.15.22:0/1065748174 wait complete. 2024-03-28T18:09:01.975 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:01.972+0000 7f94d7b86740 1 -- start start 2024-03-28T18:09:01.976 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:01.972+0000 7f94d7b86740 1 Processor -- start 2024-03-28T18:09:01.976 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:01.972+0000 7f94d7b86740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d3b56758b0 0x55d3b567e650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:09:01.976 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:01.972+0000 7f94d7b86740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55d3b5676260 0x55d3b567b9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:09:01.976 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:01.972+0000 7f94d7b86740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d3b567ede0 0x55d3b567bf20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:09:01.976 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:01.972+0000 7f94d7b86740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d3b5681b10 con 0x55d3b56758b0 2024-03-28T18:09:01.976 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:01.972+0000 7f94d7b86740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d3b55950d0 con 0x55d3b5676260 2024-03-28T18:09:01.976 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:01.972+0000 7f94d7b86740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d3b5681c90 con 0x55d3b567ede0 2024-03-28T18:09:01.976 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:01.973+0000 7f94d7b86740 10 client.0 _ll_register_callbacks cb 0x55d3b5599e80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-28T18:09:01.976 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:01.973+0000 7f94cd400640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55d3b5676260 0x55d3b567b9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:09:01.976 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:01.973+0000 7f94cde00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d3b56758b0 0x55d3b567e650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:09:01.976 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:01.973+0000 7f94cd400640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55d3b5676260 0x55d3b567b9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.22:35714/0 (socket says 172.21.15.22:35714) 2024-03-28T18:09:01.976 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:01.973+0000 7f94cd400640 1 -- 172.21.15.22:0/4043341676 learned_addr learned my addr 172.21.15.22:0/4043341676 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-28T18:09:01.976 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:01.973+0000 7f94ce800640 1 --2- 172.21.15.22:0/4043341676 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d3b567ede0 0x55d3b567bf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:09:01.976 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:01.973+0000 7f94cd400640 1 -- 172.21.15.22:0/4043341676 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d3b567ede0 msgr2=0x55d3b567bf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T18:09:01.977 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:01.973+0000 7f94cd400640 1 --2- 172.21.15.22:0/4043341676 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d3b567ede0 0x55d3b567bf20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:09:01.977 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:01.973+0000 7f94cd400640 1 -- 172.21.15.22:0/4043341676 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d3b56758b0 msgr2=0x55d3b567e650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T18:09:01.977 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:01.973+0000 7f94cd400640 1 --2- 172.21.15.22:0/4043341676 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d3b56758b0 0x55d3b567e650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:09:01.977 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:01.973+0000 7f94cd400640 1 -- 172.21.15.22:0/4043341676 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55d3b52ed3f0 con 0x55d3b5676260 2024-03-28T18:09:01.977 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:01.973+0000 7f94cde00640 1 --2- 172.21.15.22:0/4043341676 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d3b56758b0 0x55d3b567e650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-28T18:09:01.979 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:01.974+0000 7f94cd400640 1 --2- 172.21.15.22:0/4043341676 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55d3b5676260 0x55d3b567b9e0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f94bc00ba20 tx=0x7f94bc00bee0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9a08060e216d953c server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:09:01.979 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:01.974+0000 7f94afe00640 10 client.0 ms_handle_connect on v2:172.21.15.79:3300/0 2024-03-28T18:09:01.979 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:01.974+0000 7f94afe00640 1 -- 172.21.15.22:0/4043341676 <== mon.2 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94bc01b640 con 0x55d3b5676260 2024-03-28T18:09:01.979 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:01.974+0000 7f94d7b86740 1 -- 172.21.15.22:0/4043341676 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f94c00fe030 con 0x55d3b5676260 2024-03-28T18:09:01.979 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:01.974+0000 7f94afe00640 1 -- 172.21.15.22:0/4043341676 <== mon.2 v2:172.21.15.79:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 8411+0+0 (secure 0 0 0) 0x7f94bc01b7e0 con 0x55d3b5676260 2024-03-28T18:09:01.979 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:01.974+0000 7f94afe00640 1 -- 172.21.15.22:0/4043341676 <== mon.2 v2:172.21.15.79:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f94bc01b9e0 con 0x55d3b5676260 2024-03-28T18:09:01.979 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:01.974+0000 7f94d7b86740 10 client.35780 fetch_fsmap learned FSMap version 27 2024-03-28T18:09:01.979 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:01.974+0000 7f94d7b86740 1 -- 172.21.15.22:0/4043341676 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55d3b53b77d0 con 0x55d3b5676260 2024-03-28T18:09:01.979 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:01.975+0000 7f94afe00640 1 -- 172.21.15.22:0/4043341676 <== mon.2 v2:172.21.15.79:3300/0 4 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f94c00fe030 con 0x55d3b5676260 2024-03-28T18:09:01.979 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:01.975+0000 7f94afe00640 1 -- 172.21.15.22:0/4043341676 <== mon.2 v2:172.21.15.79:3300/0 5 ==== fsmap(e 27) v1 ==== 7934+0+0 (secure 0 0 0) 0x7f94bc023020 con 0x55d3b5676260 2024-03-28T18:09:01.979 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:01.975+0000 7f94d7b86740 10 client.35780 fetch_fsmap finished waiting for FSMap version 27 2024-03-28T18:09:01.979 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:01.975+0000 7f94d7b86740 10 client.35780 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.i 2024-03-28T18:09:01.980 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:01.975+0000 7f94d7b86740 20 client.35780 populate_metadata read hostname 'smithi022' 2024-03-28T18:09:01.980 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:01.975+0000 7f94d7b86740 1 --2- 172.21.15.22:0/4043341676 >> [v2:172.21.15.134:6838/3400263217,v1:172.21.15.134:6839/3400263217] conn(0x55d3b568e8a0 0x55d3b5690d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:09:01.980 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:01.975+0000 7f94d7b86740 4 client.35780 mds_command: new command op to 24502 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-28T18:09:01.980 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:01.975+0000 7f94d7b86740 1 -- 172.21.15.22:0/4043341676 --> [v2:172.21.15.134:6838/3400263217,v1:172.21.15.134:6839/3400263217] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55d3b55e6800 con 0x55d3b568e8a0 2024-03-28T18:09:01.980 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:01.976+0000 7f94cde00640 1 --2- 172.21.15.22:0/4043341676 >> [v2:172.21.15.134:6838/3400263217,v1:172.21.15.134:6839/3400263217] conn(0x55d3b568e8a0 0x55d3b5690d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:09:01.980 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:01.977+0000 7f94cde00640 1 --2- 172.21.15.22:0/4043341676 >> [v2:172.21.15.134:6838/3400263217,v1:172.21.15.134:6839/3400263217] conn(0x55d3b568e8a0 0x55d3b5690d00 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=a6302d0fb0feaca3 server_cookie=3c9d03bbde4d09d1 in_seq=0 out_seq=0 2024-03-28T18:09:01.980 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:01.977+0000 7f94afe00640 10 client.35780 ms_handle_connect on v2:172.21.15.134:6838/3400263217 2024-03-28T18:09:01.981 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:01.980+0000 7f94afe00640 1 -- 172.21.15.22:0/4043341676 <== mds.4 v2:172.21.15.134:6838/3400263217 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x55d3b55e6800 con 0x55d3b568e8a0 2024-03-28T18:09:01.981 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:01.980+0000 7f94afe00640 10 client.35780 handle_command_reply: tid=0 2024-03-28T18:09:01.981 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:01.980+0000 7f94d7b86740 1 client.35780 shutdown 2024-03-28T18:09:01.981 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:01.980+0000 7f94d7b86740 1 -- 172.21.15.22:0/4043341676 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55d3b5676260 msgr2=0x55d3b567b9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:09:01.981 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:01.980+0000 7f94d7b86740 1 --2- 172.21.15.22:0/4043341676 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55d3b5676260 0x55d3b567b9e0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f94bc00ba20 tx=0x7f94bc00bee0 comp rx=0 tx=0).stop 2024-03-28T18:09:01.982 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:01.980+0000 7f94d7b86740 1 -- 172.21.15.22:0/4043341676 shutdown_connections 2024-03-28T18:09:01.982 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:01.980+0000 7f94d7b86740 1 --2- 172.21.15.22:0/4043341676 >> [v2:172.21.15.134:6838/3400263217,v1:172.21.15.134:6839/3400263217] conn(0x55d3b568e8a0 0x55d3b5690d00 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:09:01.982 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:01.980+0000 7f94d7b86740 1 --2- 172.21.15.22:0/4043341676 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d3b567ede0 0x55d3b567bf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:09:01.982 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:01.980+0000 7f94d7b86740 1 --2- 172.21.15.22:0/4043341676 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55d3b5676260 0x55d3b567b9e0 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:09:01.982 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:01.980+0000 7f94d7b86740 1 --2- 172.21.15.22:0/4043341676 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d3b56758b0 0x55d3b567e650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:09:01.982 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:01.980+0000 7f94afe00640 0 client.35780 ms_handle_reset on v2:172.21.15.134:6838/3400263217 2024-03-28T18:09:01.982 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:01.980+0000 7f94d7b86740 1 -- 172.21.15.22:0/4043341676 >> 172.21.15.22:0/4043341676 conn(0x55d3b566ff80 msgr2=0x55d3b5671cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:09:01.982 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:01.980+0000 7f94d7b86740 1 -- 172.21.15.22:0/4043341676 shutdown_connections 2024-03-28T18:09:01.982 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:01.980+0000 7f94d7b86740 1 -- 172.21.15.22:0/4043341676 wait complete. 2024-03-28T18:09:01.982 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:01.981+0000 7f94d7b86740 20 client.35780 trim_cache size 0 max 16384 2024-03-28T18:09:01.996 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:01.996+0000 7f94d7b86740 1 Processor -- start 2024-03-28T18:09:01.996 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:01.996+0000 7f94d7b86740 1 -- start start 2024-03-28T18:09:01.998 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:01.997+0000 7f94d7b86740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55d3b56758b0 0x55d3b567e110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:09:01.998 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:01.997+0000 7f94d7b86740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d3b5676260 0x55d3b567e650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:09:01.998 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:01.997+0000 7f94d7b86740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d3b567ede0 0x55d3b5678c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:09:01.998 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:01.997+0000 7f94d7b86740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94d01af0e0 con 0x55d3b5676260 2024-03-28T18:09:01.998 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:01.997+0000 7f94d7b86740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94d01b0a60 con 0x55d3b56758b0 2024-03-28T18:09:01.998 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:01.997+0000 7f94d7b86740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94d01af740 con 0x55d3b567ede0 2024-03-28T18:09:01.998 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:01.997+0000 7f94cde00640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55d3b56758b0 0x55d3b567e110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:09:01.998 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:01.997+0000 7f94cd400640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d3b5676260 0x55d3b567e650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:09:01.998 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:01.997+0000 7f94cde00640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55d3b56758b0 0x55d3b567e110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.22:35724/0 (socket says 172.21.15.22:35724) 2024-03-28T18:09:01.999 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:01.997+0000 7f94cde00640 1 -- 172.21.15.22:0/1227866047 learned_addr learned my addr 172.21.15.22:0/1227866047 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-28T18:09:01.999 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:01.998+0000 7f94ce800640 1 --2- 172.21.15.22:0/1227866047 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d3b567ede0 0x55d3b5678c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:09:01.999 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:01.998+0000 7f94cde00640 1 -- 172.21.15.22:0/1227866047 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d3b567ede0 msgr2=0x55d3b5678c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T18:09:01.999 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:01.998+0000 7f94cde00640 1 --2- 172.21.15.22:0/1227866047 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d3b567ede0 0x55d3b5678c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:09:01.999 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:01.998+0000 7f94cde00640 1 -- 172.21.15.22:0/1227866047 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d3b5676260 msgr2=0x55d3b567e650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T18:09:01.999 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:01.998+0000 7f94cde00640 1 --2- 172.21.15.22:0/1227866047 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d3b5676260 0x55d3b567e650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:09:01.999 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:01.998+0000 7f94cde00640 1 -- 172.21.15.22:0/1227866047 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d3b566a570 con 0x55d3b56758b0 2024-03-28T18:09:01.999 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:01.998+0000 7f94ce800640 1 --2- 172.21.15.22:0/1227866047 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d3b567ede0 0x55d3b5678c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T18:09:01.999 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:01.998+0000 7f94cd400640 1 --2- 172.21.15.22:0/1227866047 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d3b5676260 0x55d3b567e650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-28T18:09:01.999 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:01.998+0000 7f94cde00640 1 --2- 172.21.15.22:0/1227866047 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55d3b56758b0 0x55d3b567e110 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f94c00bd6b0 tx=0x7f94c00f4bd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=65a5a649a5fb5e41 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:09:01.999 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:01.998+0000 7f94baa00640 1 -- 172.21.15.22:0/1227866047 <== mon.2 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94c008a530 con 0x55d3b56758b0 2024-03-28T18:09:01.999 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:01.998+0000 7f94baa00640 1 -- 172.21.15.22:0/1227866047 <== mon.2 v2:172.21.15.79:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f94c00fe040 con 0x55d3b56758b0 2024-03-28T18:09:01.999 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:01.998+0000 7f94baa00640 1 -- 172.21.15.22:0/1227866047 <== mon.2 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94c008a790 con 0x55d3b56758b0 2024-03-28T18:09:02.003 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:01.999+0000 7f94d7b86740 1 -- 172.21.15.22:0/1227866047 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55d3b56758b0 msgr2=0x55d3b567e110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:09:02.003 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:01.999+0000 7f94d7b86740 1 --2- 172.21.15.22:0/1227866047 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55d3b56758b0 0x55d3b567e110 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f94c00bd6b0 tx=0x7f94c00f4bd0 comp rx=0 tx=0).stop 2024-03-28T18:09:02.003 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:01.999+0000 7f94d7b86740 1 -- 172.21.15.22:0/1227866047 shutdown_connections 2024-03-28T18:09:02.003 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:01.999+0000 7f94d7b86740 1 --2- 172.21.15.22:0/1227866047 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d3b567ede0 0x55d3b5678c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:09:02.003 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:01.999+0000 7f94d7b86740 1 --2- 172.21.15.22:0/1227866047 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d3b5676260 0x55d3b567e650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:09:02.003 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:01.999+0000 7f94d7b86740 1 --2- 172.21.15.22:0/1227866047 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55d3b56758b0 0x55d3b567e110 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:09:02.003 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:01.999+0000 7f94d7b86740 1 -- 172.21.15.22:0/1227866047 >> 172.21.15.22:0/1227866047 conn(0x55d3b566ff80 msgr2=0x55d3b5671c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:09:02.003 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:01.999+0000 7f94d7b86740 1 -- 172.21.15.22:0/1227866047 shutdown_connections 2024-03-28T18:09:02.003 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:01.999+0000 7f94d7b86740 1 -- 172.21.15.22:0/1227866047 wait complete. 2024-03-28T18:09:02.003 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:02.000+0000 7f94d7b86740 1 -- start start 2024-03-28T18:09:02.003 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:02.000+0000 7f94d7b86740 1 Processor -- start 2024-03-28T18:09:02.004 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:02.000+0000 7f94d7b86740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d3b56758b0 0x55d3b5686a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:09:02.004 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:02.000+0000 7f94d7b86740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55d3b5676260 0x55d3b5682da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:09:02.004 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:02.000+0000 7f94d7b86740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d3b567ede0 0x55d3b56832e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:09:02.004 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:02.000+0000 7f94d7b86740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94d01b0400 con 0x55d3b567ede0 2024-03-28T18:09:02.004 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:02.000+0000 7f94d7b86740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94d01afda0 con 0x55d3b5676260 2024-03-28T18:09:02.004 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:02.000+0000 7f94d7b86740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94d01ad2e0 con 0x55d3b56758b0 2024-03-28T18:09:02.004 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:02.001+0000 7f94d7b86740 10 client.0 _ll_register_callbacks cb 0x7f94bc004160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-28T18:09:02.004 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:02.001+0000 7f94ce800640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d3b567ede0 0x55d3b56832e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:09:02.004 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:02.001+0000 7f94cd400640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55d3b5676260 0x55d3b5682da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:09:02.004 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:02.001+0000 7f94ce800640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d3b567ede0 0x55d3b56832e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:54514/0 (socket says 172.21.15.22:54514) 2024-03-28T18:09:02.004 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:02.001+0000 7f94cd400640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55d3b5676260 0x55d3b5682da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.22:35734/0 (socket says 172.21.15.22:35734) 2024-03-28T18:09:02.004 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:02.001+0000 7f94ce800640 1 -- 172.21.15.22:0/3793614812 learned_addr learned my addr 172.21.15.22:0/3793614812 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-28T18:09:02.004 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:02.001+0000 7f94cde00640 1 --2- 172.21.15.22:0/3793614812 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d3b56758b0 0x55d3b5686a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:09:02.005 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:02.001+0000 7f94cd400640 1 -- 172.21.15.22:0/3793614812 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d3b56758b0 msgr2=0x55d3b5686a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T18:09:02.005 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:02.001+0000 7f94cd400640 1 --2- 172.21.15.22:0/3793614812 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d3b56758b0 0x55d3b5686a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:09:02.005 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:02.001+0000 7f94cd400640 1 -- 172.21.15.22:0/3793614812 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d3b567ede0 msgr2=0x55d3b56832e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T18:09:02.005 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:02.001+0000 7f94cd400640 1 --2- 172.21.15.22:0/3793614812 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d3b567ede0 0x55d3b56832e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:09:02.005 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:02.001+0000 7f94cd400640 1 -- 172.21.15.22:0/3793614812 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55d3b5683920 con 0x55d3b5676260 2024-03-28T18:09:02.005 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:02.001+0000 7f94ce800640 1 --2- 172.21.15.22:0/3793614812 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d3b567ede0 0x55d3b56832e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-28T18:09:02.005 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:02.002+0000 7f94cd400640 1 --2- 172.21.15.22:0/3793614812 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55d3b5676260 0x55d3b5682da0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f94bc025d20 tx=0x7f94bc005e50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f5593afe94ff16de server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:09:02.005 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:02.002+0000 7f94afe00640 10 client.0 ms_handle_connect on v2:172.21.15.79:3300/0 2024-03-28T18:09:02.005 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:02.002+0000 7f94afe00640 1 -- 172.21.15.22:0/3793614812 <== mon.2 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94bc035060 con 0x55d3b5676260 2024-03-28T18:09:02.005 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:02.002+0000 7f94d7b86740 1 -- 172.21.15.22:0/3793614812 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f94c00fe040 con 0x55d3b5676260 2024-03-28T18:09:02.005 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:02.002+0000 7f94afe00640 1 -- 172.21.15.22:0/3793614812 <== mon.2 v2:172.21.15.79:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 8411+0+0 (secure 0 0 0) 0x7f94bc005040 con 0x55d3b5676260 2024-03-28T18:09:02.005 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:02.002+0000 7f94afe00640 1 -- 172.21.15.22:0/3793614812 <== mon.2 v2:172.21.15.79:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f94bc02b070 con 0x55d3b5676260 2024-03-28T18:09:02.005 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:02.002+0000 7f94d7b86740 10 client.35786 fetch_fsmap learned FSMap version 27 2024-03-28T18:09:02.005 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:02.003+0000 7f94d7b86740 1 -- 172.21.15.22:0/3793614812 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55d3b56eaf80 con 0x55d3b5676260 2024-03-28T18:09:02.005 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:02.003+0000 7f94afe00640 1 -- 172.21.15.22:0/3793614812 <== mon.2 v2:172.21.15.79:3300/0 4 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f94c00fe040 con 0x55d3b5676260 2024-03-28T18:09:02.006 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:02.003+0000 7f94afe00640 1 -- 172.21.15.22:0/3793614812 <== mon.2 v2:172.21.15.79:3300/0 5 ==== fsmap(e 27) v1 ==== 7934+0+0 (secure 0 0 0) 0x7f94bc038020 con 0x55d3b5676260 2024-03-28T18:09:02.006 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:02.003+0000 7f94d7b86740 10 client.35786 fetch_fsmap finished waiting for FSMap version 27 2024-03-28T18:09:02.006 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:02.003+0000 7f94d7b86740 10 client.35786 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.i 2024-03-28T18:09:02.006 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:02.003+0000 7f94d7b86740 20 client.35786 populate_metadata read hostname 'smithi022' 2024-03-28T18:09:02.006 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:02.003+0000 7f94d7b86740 1 --2- 172.21.15.22:0/3793614812 >> [v2:172.21.15.134:6838/3400263217,v1:172.21.15.134:6839/3400263217] conn(0x55d3b56eb680 0x55d3b56edaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:09:02.006 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:02.003+0000 7f94d7b86740 4 client.35786 mds_command: new command op to 24502 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-28T18:09:02.006 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:02.003+0000 7f94d7b86740 1 -- 172.21.15.22:0/3793614812 --> [v2:172.21.15.134:6838/3400263217,v1:172.21.15.134:6839/3400263217] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55d3b55e0b50 con 0x55d3b56eb680 2024-03-28T18:09:02.010 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:02.004+0000 7f94cde00640 1 --2- 172.21.15.22:0/3793614812 >> [v2:172.21.15.134:6838/3400263217,v1:172.21.15.134:6839/3400263217] conn(0x55d3b56eb680 0x55d3b56edaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:09:02.011 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:02.005+0000 7f94cde00640 1 --2- 172.21.15.22:0/3793614812 >> [v2:172.21.15.134:6838/3400263217,v1:172.21.15.134:6839/3400263217] conn(0x55d3b56eb680 0x55d3b56edaa0 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=22471f3b4d8ade83 server_cookie=9982e4535aaf23a5 in_seq=0 out_seq=0 2024-03-28T18:09:02.011 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:02.005+0000 7f94afe00640 10 client.35786 ms_handle_connect on v2:172.21.15.134:6838/3400263217 2024-03-28T18:09:02.011 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:02.005+0000 7f94afe00640 1 -- 172.21.15.22:0/3793614812 <== mds.4 v2:172.21.15.134:6838/3400263217 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55d3b55e0b50 con 0x55d3b56eb680 2024-03-28T18:09:02.011 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:02.005+0000 7f94afe00640 10 client.35786 handle_command_reply: tid=0 2024-03-28T18:09:02.011 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:02.005+0000 7f94d7b86740 1 client.35786 shutdown 2024-03-28T18:09:02.011 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:02.005+0000 7f94d7b86740 1 -- 172.21.15.22:0/3793614812 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55d3b5676260 msgr2=0x55d3b5682da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:09:02.011 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:02.005+0000 7f94d7b86740 1 --2- 172.21.15.22:0/3793614812 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55d3b5676260 0x55d3b5682da0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f94bc025d20 tx=0x7f94bc005e50 comp rx=0 tx=0).stop 2024-03-28T18:09:02.011 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:02.005+0000 7f94d7b86740 1 -- 172.21.15.22:0/3793614812 shutdown_connections 2024-03-28T18:09:02.011 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:02.006+0000 7f94d7b86740 1 --2- 172.21.15.22:0/3793614812 >> [v2:172.21.15.134:6838/3400263217,v1:172.21.15.134:6839/3400263217] conn(0x55d3b56eb680 0x55d3b56edaa0 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:09:02.011 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:02.006+0000 7f94d7b86740 1 --2- 172.21.15.22:0/3793614812 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d3b567ede0 0x55d3b56832e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:09:02.011 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:02.006+0000 7f94d7b86740 1 --2- 172.21.15.22:0/3793614812 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55d3b5676260 0x55d3b5682da0 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:09:02.011 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:02.006+0000 7f94d7b86740 1 --2- 172.21.15.22:0/3793614812 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d3b56758b0 0x55d3b5686a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:09:02.011 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:02.006+0000 7f94afe00640 0 client.35786 ms_handle_reset on v2:172.21.15.134:6838/3400263217 2024-03-28T18:09:02.011 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:02.006+0000 7f94d7b86740 1 -- 172.21.15.22:0/3793614812 >> 172.21.15.22:0/3793614812 conn(0x55d3b566ff80 msgr2=0x55d3b56713d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:09:02.012 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:02.006+0000 7f94d7b86740 1 -- 172.21.15.22:0/3793614812 shutdown_connections 2024-03-28T18:09:02.012 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:02.006+0000 7f94d7b86740 1 -- 172.21.15.22:0/3793614812 wait complete. 2024-03-28T18:09:02.012 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:02.006+0000 7f94d7b86740 20 client.35786 trim_cache size 0 max 16384 2024-03-28T18:09:02.012 INFO:teuthology.orchestra.run.smithi022.stdout:[] 2024-03-28T18:09:02.013 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:02.007+0000 7f94cf200640 1 -- 172.21.15.22:0/2803155755 >> [v2:172.21.15.22:6800/1357365082,v1:172.21.15.22:6801/1357365082] conn(0x7f94b40a2b10 msgr2=0x7f94b40a4fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:09:02.013 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:02.007+0000 7f94cf200640 1 --2- 172.21.15.22:0/2803155755 >> [v2:172.21.15.22:6800/1357365082,v1:172.21.15.22:6801/1357365082] conn(0x7f94b40a2b10 0x7f94b40a4fd0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f94c8009370 tx=0x7f94c8009820 comp rx=0 tx=0).stop 2024-03-28T18:09:02.013 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:02.007+0000 7f94cf200640 1 -- 172.21.15.22:0/2803155755 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f94d0103720 msgr2=0x7f94d019a360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:09:02.013 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:02.007+0000 7f94cf200640 1 --2- 172.21.15.22:0/2803155755 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f94d0103720 0x7f94d019a360 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7f94c0000c00 tx=0x7f94c0005020 comp rx=0 tx=0).stop 2024-03-28T18:09:02.013 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:02.007+0000 7f94cf200640 1 -- 172.21.15.22:0/2803155755 shutdown_connections 2024-03-28T18:09:02.013 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:02.007+0000 7f94cf200640 1 --2- 172.21.15.22:0/2803155755 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f94d0109730 0x7f94d019ec20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:09:02.013 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:02.007+0000 7f94cf200640 1 --2- 172.21.15.22:0/2803155755 >> [v2:172.21.15.22:6800/1357365082,v1:172.21.15.22:6801/1357365082] conn(0x7f94b40a2b10 0x7f94b40a4fd0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:09:02.013 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:02.007+0000 7f94cf200640 1 --2- 172.21.15.22:0/2803155755 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f94d01040e0 0x7f94d019a8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:09:02.013 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:02.007+0000 7f94cf200640 1 --2- 172.21.15.22:0/2803155755 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f94d0103720 0x7f94d019a360 unknown :-1 s=CLOSED pgs=560 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:09:02.014 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:02.007+0000 7f94cf200640 1 -- 172.21.15.22:0/2803155755 >> 172.21.15.22:0/2803155755 conn(0x7f94d00ff1b0 msgr2=0x7f94d00fff10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:09:02.014 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:02.007+0000 7f94cf200640 1 -- 172.21.15.22:0/2803155755 shutdown_connections 2024-03-28T18:09:02.014 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:02.007+0000 7f94cf200640 1 -- 172.21.15.22:0/2803155755 wait complete. 2024-03-28T18:09:02.015 INFO:tasks.workunit:Running workunits matching fs/norstats on client.0... 2024-03-28T18:09:02.017 INFO:tasks.workunit:Running workunit fs/norstats/kernel_untar_tar.sh... 2024-03-28T18:09:02.017 DEBUG:teuthology.orchestra.run.smithi022:workunit test fs/norstats/kernel_untar_tar.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=3be9a5c9bc793e11e2800a8c0c696e8b46742033 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/fs/norstats/kernel_untar_tar.sh 2024-03-28T18:09:02.023 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-28T18:09:02.023 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2024-03-28T18:09:02.142 INFO:tasks.workunit.client.0.smithi022.stderr:+ KERNEL=linux-4.0.5 2024-03-28T18:09:02.142 INFO:tasks.workunit.client.0.smithi022.stderr:+ wget -q http://download.ceph.com/qa/linux-4.0.5.tar.xz 2024-03-28T18:09:02.280 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:02.280+0000 7f0f09000640 1 -- 172.21.15.22:0/3167419299 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0f04100de0 msgr2=0x7f0f041052a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:09:02.280 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:02.280+0000 7f0f09000640 1 --2- 172.21.15.22:0/3167419299 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0f04100de0 0x7f0f041052a0 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7f0eec009900 tx=0x7f0eec02bd10 comp rx=0 tx=0).stop 2024-03-28T18:09:02.280 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:02.280+0000 7f0f09000640 1 -- 172.21.15.22:0/3167419299 shutdown_connections 2024-03-28T18:09:02.280 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:02.280+0000 7f0f09000640 1 --2- 172.21.15.22:0/3167419299 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f0f041058c0 0x7f0f04105d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:09:02.280 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:02.280+0000 7f0f09000640 1 --2- 172.21.15.22:0/3167419299 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0f04100de0 0x7f0f041052a0 unknown :-1 s=CLOSED pgs=561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:09:02.280 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:02.280+0000 7f0f09000640 1 --2- 172.21.15.22:0/3167419299 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0f041004c0 0x7f0f041008a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:09:02.280 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:02.280+0000 7f0f09000640 1 -- 172.21.15.22:0/3167419299 >> 172.21.15.22:0/3167419299 conn(0x7f0f040fbef0 msgr2=0x7f0f040fe310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:09:02.280 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:02.280+0000 7f0f09000640 1 -- 172.21.15.22:0/3167419299 shutdown_connections 2024-03-28T18:09:02.280 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:02.280+0000 7f0f09000640 1 -- 172.21.15.22:0/3167419299 wait complete. 2024-03-28T18:09:02.282 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:02.281+0000 7f0f09000640 1 Processor -- start 2024-03-28T18:09:02.282 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:02.281+0000 7f0f09000640 1 -- start start 2024-03-28T18:09:02.282 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:02.282+0000 7f0f09000640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f0f041004c0 0x7f0f0419e400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:09:02.282 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:02.282+0000 7f0f09000640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0f04100de0 0x7f0f0419e940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:09:02.282 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:02.282+0000 7f0f09000640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0f041058c0 0x7f0f041a2cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:09:02.282 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:02.282+0000 7f0f09000640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f04117f20 con 0x7f0f04100de0 2024-03-28T18:09:02.282 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:02.282+0000 7f0f09000640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f04117da0 con 0x7f0f041004c0 2024-03-28T18:09:02.282 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:02.282+0000 7f0f09000640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f041180a0 con 0x7f0f041058c0 2024-03-28T18:09:02.282 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:02.282+0000 7f0efbe00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0f04100de0 0x7f0f0419e940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:09:02.282 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:02.282+0000 7f0efbe00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0f04100de0 0x7f0f0419e940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:54540/0 (socket says 172.21.15.22:54540) 2024-03-28T18:09:02.282 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:02.282+0000 7f0f03400640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f0f041004c0 0x7f0f0419e400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:09:02.283 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:02.282+0000 7f0efbe00640 1 -- 172.21.15.22:0/1317656137 learned_addr learned my addr 172.21.15.22:0/1317656137 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-28T18:09:02.283 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:02.282+0000 7f0f03e00640 1 --2- 172.21.15.22:0/1317656137 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0f041058c0 0x7f0f041a2cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:09:02.284 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:02.283+0000 7f0efbe00640 1 -- 172.21.15.22:0/1317656137 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0f041058c0 msgr2=0x7f0f041a2cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:09:02.284 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:02.283+0000 7f0efbe00640 1 --2- 172.21.15.22:0/1317656137 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0f041058c0 0x7f0f041a2cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:09:02.284 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:02.283+0000 7f0efbe00640 1 -- 172.21.15.22:0/1317656137 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f0f041004c0 msgr2=0x7f0f0419e400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:09:02.284 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:02.283+0000 7f0efbe00640 1 --2- 172.21.15.22:0/1317656137 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f0f041004c0 0x7f0f0419e400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:09:02.284 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:02.283+0000 7f0efbe00640 1 -- 172.21.15.22:0/1317656137 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0f041a33f0 con 0x7f0f04100de0 2024-03-28T18:09:02.284 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:02.283+0000 7f0f03400640 1 --2- 172.21.15.22:0/1317656137 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f0f041004c0 0x7f0f0419e400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-28T18:09:02.284 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:02.283+0000 7f0efbe00640 1 --2- 172.21.15.22:0/1317656137 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0f04100de0 0x7f0f0419e940 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7f0eec000c00 tx=0x7f0eec004e40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:09:02.284 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:02.284+0000 7f0f00c00640 1 -- 172.21.15.22:0/1317656137 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0eec042020 con 0x7f0f04100de0 2024-03-28T18:09:02.284 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:02.284+0000 7f0f00c00640 1 -- 172.21.15.22:0/1317656137 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f0eec02c650 con 0x7f0f04100de0 2024-03-28T18:09:02.284 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:02.284+0000 7f0f09000640 1 -- 172.21.15.22:0/1317656137 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0f041a3680 con 0x7f0f04100de0 2024-03-28T18:09:02.284 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:02.284+0000 7f0f00c00640 1 -- 172.21.15.22:0/1317656137 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0eec003970 con 0x7f0f04100de0 2024-03-28T18:09:02.285 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:02.284+0000 7f0f09000640 1 -- 172.21.15.22:0/1317656137 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0f041aaf70 con 0x7f0f04100de0 2024-03-28T18:09:02.287 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:02.287+0000 7f0f0d226740 1 Processor -- start 2024-03-28T18:09:02.287 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:02.287+0000 7f0f0d226740 1 -- start start 2024-03-28T18:09:02.288 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:02.288+0000 7f0f0d226740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5616c04a78b0 0x5616c04a7c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:09:02.288 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:02.288+0000 7f0f0d226740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5616c04a8260 0x5616c04b0810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:09:02.288 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:02.288+0000 7f0f0d226740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5616c04b0de0 0x5616c04b3230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:09:02.288 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:02.288+0000 7f0f0d226740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x5616c040b2c0 con 0x5616c04b0de0 2024-03-28T18:09:02.289 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:02.288+0000 7f0f0d226740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x5616c03b3c90 con 0x5616c04a78b0 2024-03-28T18:09:02.289 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:02.288+0000 7f0f0d226740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x5616c0407360 con 0x5616c04a8260 2024-03-28T18:09:02.289 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:02.288+0000 7f0f03e00640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5616c04a78b0 0x5616c04a7c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:09:02.289 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:02.288+0000 7f0f00c00640 1 -- 172.21.15.22:0/1317656137 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 25) v1 ==== 137523+0+0 (secure 0 0 0) 0x7f0eec045020 con 0x7f0f04100de0 2024-03-28T18:09:02.289 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:02.288+0000 7f0f03e00640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5616c04a78b0 0x5616c04a7c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.22:35760/0 (socket says 172.21.15.22:35760) 2024-03-28T18:09:02.289 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:02.288+0000 7f0f03e00640 1 -- 172.21.15.22:0/2046355796 learned_addr learned my addr 172.21.15.22:0/2046355796 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-28T18:09:02.289 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:02.288+0000 7f0f03400640 1 --2- 172.21.15.22:0/2046355796 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5616c04a8260 0x5616c04b0810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:09:02.289 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:02.289+0000 7f0efbe00640 1 --2- 172.21.15.22:0/2046355796 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5616c04b0de0 0x5616c04b3230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:09:02.289 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:02.289+0000 7f0f03e00640 1 -- 172.21.15.22:0/2046355796 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5616c04a8260 msgr2=0x5616c04b0810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T18:09:02.289 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:02.289+0000 7f0f03e00640 1 --2- 172.21.15.22:0/2046355796 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5616c04a8260 0x5616c04b0810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:09:02.289 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:02.289+0000 7f0f03e00640 1 -- 172.21.15.22:0/2046355796 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5616c04b0de0 msgr2=0x5616c04b3230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T18:09:02.289 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:02.289+0000 7f0f03e00640 1 --2- 172.21.15.22:0/2046355796 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5616c04b0de0 0x5616c04b3230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:09:02.289 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:02.289+0000 7f0f03e00640 1 -- 172.21.15.22:0/2046355796 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5616c03d8080 con 0x5616c04a78b0 2024-03-28T18:09:02.289 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:02.289+0000 7f0efbe00640 1 --2- 172.21.15.22:0/2046355796 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5616c04b0de0 0x5616c04b3230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T18:09:02.291 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:02.289+0000 7f0f03e00640 1 --2- 172.21.15.22:0/2046355796 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5616c04a78b0 0x5616c04a7c90 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f0efc009f50 tx=0x7f0efc0118d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5f08926d5b69c43b server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:09:02.291 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:02.289+0000 7f0ef8c00640 1 -- 172.21.15.22:0/2046355796 <== mon.2 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0efc004d20 con 0x5616c04a78b0 2024-03-28T18:09:02.291 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:02.290+0000 7f0ef8c00640 1 -- 172.21.15.22:0/2046355796 <== mon.2 v2:172.21.15.79:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f0efc004050 con 0x5616c04a78b0 2024-03-28T18:09:02.291 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:02.290+0000 7f0ef8c00640 1 -- 172.21.15.22:0/2046355796 <== mon.2 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0efc005260 con 0x5616c04a78b0 2024-03-28T18:09:02.291 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:02.290+0000 7f0f00c00640 1 --2- 172.21.15.22:0/1317656137 >> [v2:172.21.15.22:6800/1357365082,v1:172.21.15.22:6801/1357365082] conn(0x7f0ee80a2b10 0x7f0ee80a4fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:09:02.291 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:02.290+0000 7f0f00c00640 1 -- 172.21.15.22:0/1317656137 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 8411+0+0 (secure 0 0 0) 0x7f0eec0f0a40 con 0x7f0f04100de0 2024-03-28T18:09:02.292 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:02.290+0000 7f0f03400640 1 --2- 172.21.15.22:0/1317656137 >> [v2:172.21.15.22:6800/1357365082,v1:172.21.15.22:6801/1357365082] conn(0x7f0ee80a2b10 0x7f0ee80a4fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:09:02.292 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:02.291+0000 7f0f0d226740 1 -- 172.21.15.22:0/2046355796 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5616c04a78b0 msgr2=0x5616c04a7c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:09:02.292 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:02.291+0000 7f0f0d226740 1 --2- 172.21.15.22:0/2046355796 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5616c04a78b0 0x5616c04a7c90 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f0efc009f50 tx=0x7f0efc0118d0 comp rx=0 tx=0).stop 2024-03-28T18:09:02.292 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:02.291+0000 7f0f0d226740 1 -- 172.21.15.22:0/2046355796 shutdown_connections 2024-03-28T18:09:02.292 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:02.291+0000 7f0f0d226740 1 --2- 172.21.15.22:0/2046355796 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5616c04b0de0 0x5616c04b3230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:09:02.292 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:02.291+0000 7f0f0d226740 1 --2- 172.21.15.22:0/2046355796 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5616c04a8260 0x5616c04b0810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:09:02.292 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:02.291+0000 7f0f0d226740 1 --2- 172.21.15.22:0/2046355796 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5616c04a78b0 0x5616c04a7c90 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:09:02.292 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:02.291+0000 7f0f0d226740 1 -- 172.21.15.22:0/2046355796 >> 172.21.15.22:0/2046355796 conn(0x5616c04a1f80 msgr2=0x5616c04a2360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:09:02.292 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:02.291+0000 7f0f03400640 1 --2- 172.21.15.22:0/1317656137 >> [v2:172.21.15.22:6800/1357365082,v1:172.21.15.22:6801/1357365082] conn(0x7f0ee80a2b10 0x7f0ee80a4fd0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x5616c04a2ff0 tx=0x7f0ef400b410 comp rx=0 tx=0).ready entity=mgr.15522 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:09:02.292 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:02.291+0000 7f0f0d226740 1 -- 172.21.15.22:0/2046355796 shutdown_connections 2024-03-28T18:09:02.292 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:02.291+0000 7f0f0d226740 1 -- 172.21.15.22:0/2046355796 wait complete. 2024-03-28T18:09:02.292 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:02.291+0000 7f0f0d226740 1 -- start start 2024-03-28T18:09:02.292 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:02.292+0000 7f0f0d226740 1 Processor -- start 2024-03-28T18:09:02.295 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:02.292+0000 7f0f0d226740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5616c04a78b0 0x5616c04b06f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:09:02.295 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:02.292+0000 7f0f0d226740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5616c04a8260 0x5616c04ada80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:09:02.296 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:02.292+0000 7f0f0d226740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5616c04b0de0 0x5616c04ae020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:09:02.296 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:02.292+0000 7f0f0d226740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x5616c04b3d50 con 0x5616c04a78b0 2024-03-28T18:09:02.296 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:02.292+0000 7f0f0d226740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x5616c03c70d0 con 0x5616c04b0de0 2024-03-28T18:09:02.296 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:02.292+0000 7f0f0d226740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x5616c04b3ed0 con 0x5616c04a8260 2024-03-28T18:09:02.296 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:02.292+0000 7f0f0d226740 10 client.0 _ll_register_callbacks cb 0x5616c03cbe80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-28T18:09:02.296 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:02.292+0000 7f0f03400640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5616c04a78b0 0x5616c04b06f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:09:02.296 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:02.292+0000 7f0f03400640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5616c04a78b0 0x5616c04b06f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:54564/0 (socket says 172.21.15.22:54564) 2024-03-28T18:09:02.296 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:02.292+0000 7f0f03400640 1 -- 172.21.15.22:0/2637483269 learned_addr learned my addr 172.21.15.22:0/2637483269 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-28T18:09:02.296 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:02.292+0000 7f0f03e00640 1 --2- 172.21.15.22:0/2637483269 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5616c04b0de0 0x5616c04ae020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:09:02.296 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:02.293+0000 7f0efbe00640 1 --2- 172.21.15.22:0/2637483269 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5616c04a8260 0x5616c04ada80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:09:02.296 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:02.293+0000 7f0f03400640 1 -- 172.21.15.22:0/2637483269 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5616c04a8260 msgr2=0x5616c04ada80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T18:09:02.296 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:02.293+0000 7f0f03400640 1 --2- 172.21.15.22:0/2637483269 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5616c04a8260 0x5616c04ada80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:09:02.296 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:02.293+0000 7f0f03400640 1 -- 172.21.15.22:0/2637483269 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5616c04b0de0 msgr2=0x5616c04ae020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T18:09:02.296 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:02.293+0000 7f0f03400640 1 --2- 172.21.15.22:0/2637483269 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5616c04b0de0 0x5616c04ae020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:09:02.297 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:02.293+0000 7f0f03400640 1 -- 172.21.15.22:0/2637483269 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5616c011f3f0 con 0x5616c04a78b0 2024-03-28T18:09:02.297 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:02.293+0000 7f0f03400640 1 --2- 172.21.15.22:0/2637483269 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5616c04a78b0 0x5616c04b06f0 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7f0ef4012010 tx=0x7f0ef40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=20a56eaf97767204 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:09:02.297 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:02.293+0000 7f0ef1600640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-03-28T18:09:02.297 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:02.294+0000 7f0ef1600640 1 -- 172.21.15.22:0/2637483269 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ef402c060 con 0x5616c04a78b0 2024-03-28T18:09:02.297 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:02.294+0000 7f0ef1600640 1 -- 172.21.15.22:0/2637483269 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 8411+0+0 (secure 0 0 0) 0x7f0ef4017030 con 0x5616c04a78b0 2024-03-28T18:09:02.297 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:02.294+0000 7f0f0d226740 1 -- 172.21.15.22:0/2637483269 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f0efc004050 con 0x5616c04a78b0 2024-03-28T18:09:02.297 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:02.294+0000 7f0ef1600640 1 -- 172.21.15.22:0/2637483269 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0ef401fbc0 con 0x5616c04a78b0 2024-03-28T18:09:02.297 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:02.294+0000 7f0f0d226740 10 client.15963 fetch_fsmap learned FSMap version 27 2024-03-28T18:09:02.297 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:02.294+0000 7f0f0d226740 1 -- 172.21.15.22:0/2637483269 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x5616c01e97d0 con 0x5616c04a78b0 2024-03-28T18:09:02.297 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:02.294+0000 7f0ef1600640 1 -- 172.21.15.22:0/2637483269 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f0efc004050 con 0x5616c04a78b0 2024-03-28T18:09:02.297 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:02.294+0000 7f0ef1600640 1 -- 172.21.15.22:0/2637483269 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 27) v1 ==== 7934+0+0 (secure 0 0 0) 0x7f0ef402f030 con 0x5616c04a78b0 2024-03-28T18:09:02.297 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:02.295+0000 7f0f0d226740 10 client.15963 fetch_fsmap finished waiting for FSMap version 27 2024-03-28T18:09:02.297 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:02.295+0000 7f0f0d226740 10 client.15963 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.k 2024-03-28T18:09:02.297 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:02.295+0000 7f0f0d226740 20 client.15963 populate_metadata read hostname 'smithi022' 2024-03-28T18:09:02.297 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:02.295+0000 7f0f0d226740 1 --2- 172.21.15.22:0/2637483269 >> [v2:172.21.15.79:6832/856896997,v1:172.21.15.79:6833/856896997] conn(0x5616c04c0c60 0x5616c04c3050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:09:02.298 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:02.295+0000 7f0f0d226740 4 client.15963 mds_command: new command op to 34442 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-28T18:09:02.298 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:02.295+0000 7f0f0d226740 1 -- 172.21.15.22:0/2637483269 --> [v2:172.21.15.79:6832/856896997,v1:172.21.15.79:6833/856896997] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5616c0418800 con 0x5616c04c0c60 2024-03-28T18:09:02.298 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:02.295+0000 7f0efbe00640 1 --2- 172.21.15.22:0/2637483269 >> [v2:172.21.15.79:6832/856896997,v1:172.21.15.79:6833/856896997] conn(0x5616c04c0c60 0x5616c04c3050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:09:02.298 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:02.296+0000 7f0ef1600640 10 client.15963 ms_handle_connect on v2:172.21.15.79:6832/856896997 2024-03-28T18:09:02.298 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:02.296+0000 7f0efbe00640 1 --2- 172.21.15.22:0/2637483269 >> [v2:172.21.15.79:6832/856896997,v1:172.21.15.79:6833/856896997] conn(0x5616c04c0c60 0x5616c04c3050 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=2f1d049c77c01e5d server_cookie=c1cbc13dc8cbab8d in_seq=0 out_seq=0 2024-03-28T18:09:02.299 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:02.297+0000 7f0ef1600640 1 -- 172.21.15.22:0/2637483269 <== mds.3 v2:172.21.15.79:6832/856896997 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x5616c0418800 con 0x5616c04c0c60 2024-03-28T18:09:02.299 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:02.297+0000 7f0ef1600640 10 client.15963 handle_command_reply: tid=0 2024-03-28T18:09:02.299 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:02.297+0000 7f0f0d226740 1 client.15963 shutdown 2024-03-28T18:09:02.299 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:02.297+0000 7f0f0d226740 1 -- 172.21.15.22:0/2637483269 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5616c04a78b0 msgr2=0x5616c04b06f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:09:02.299 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:02.297+0000 7f0f0d226740 1 --2- 172.21.15.22:0/2637483269 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5616c04a78b0 0x5616c04b06f0 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7f0ef4012010 tx=0x7f0ef40046c0 comp rx=0 tx=0).stop 2024-03-28T18:09:02.299 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:02.297+0000 7f0f0d226740 1 -- 172.21.15.22:0/2637483269 shutdown_connections 2024-03-28T18:09:02.299 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:02.297+0000 7f0f0d226740 1 --2- 172.21.15.22:0/2637483269 >> [v2:172.21.15.79:6832/856896997,v1:172.21.15.79:6833/856896997] conn(0x5616c04c0c60 0x5616c04c3050 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:09:02.299 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:02.297+0000 7f0f0d226740 1 --2- 172.21.15.22:0/2637483269 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5616c04b0de0 0x5616c04ae020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:09:02.300 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:02.297+0000 7f0f0d226740 1 --2- 172.21.15.22:0/2637483269 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5616c04a8260 0x5616c04ada80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:09:02.300 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:02.297+0000 7f0ef1600640 0 client.15963 ms_handle_reset on v2:172.21.15.79:6832/856896997 2024-03-28T18:09:02.300 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:02.297+0000 7f0f0d226740 1 --2- 172.21.15.22:0/2637483269 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5616c04a78b0 0x5616c04b06f0 unknown :-1 s=CLOSED pgs=563 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:09:02.300 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:02.297+0000 7f0f0d226740 1 -- 172.21.15.22:0/2637483269 >> 172.21.15.22:0/2637483269 conn(0x5616c04a1f80 msgr2=0x5616c04a3cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:09:02.300 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:02.298+0000 7f0f0d226740 1 -- 172.21.15.22:0/2637483269 shutdown_connections 2024-03-28T18:09:02.300 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:02.298+0000 7f0f0d226740 1 -- 172.21.15.22:0/2637483269 wait complete. 2024-03-28T18:09:02.300 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:02.298+0000 7f0f0d226740 20 client.15963 trim_cache size 0 max 16384 2024-03-28T18:09:02.313 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:02.312+0000 7f0f0d226740 1 Processor -- start 2024-03-28T18:09:02.313 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:02.313+0000 7f0f0d226740 1 -- start start 2024-03-28T18:09:02.313 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:02.313+0000 7f0f0d226740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5616c04a78b0 0x5616c04a1160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:09:02.313 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:02.313+0000 7f0f0d226740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5616c04a8260 0x5616c04b01b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:09:02.313 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:02.313+0000 7f0f0d226740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5616c04b0de0 0x5616c04b06f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:09:02.313 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:02.313+0000 7f0f0d226740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f041b33e0 con 0x5616c04a8260 2024-03-28T18:09:02.313 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:02.313+0000 7f0f0d226740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f041b4d60 con 0x5616c04b0de0 2024-03-28T18:09:02.313 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:02.313+0000 7f0f0d226740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f041b3a40 con 0x5616c04a78b0 2024-03-28T18:09:02.313 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:02.313+0000 7f0efbe00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5616c04a8260 0x5616c04b01b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:09:02.313 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:02.313+0000 7f0efbe00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5616c04a8260 0x5616c04b01b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:54568/0 (socket says 172.21.15.22:54568) 2024-03-28T18:09:02.313 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:02.313+0000 7f0efbe00640 1 -- 172.21.15.22:0/3748526875 learned_addr learned my addr 172.21.15.22:0/3748526875 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-28T18:09:02.315 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:02.313+0000 7f0f03400640 1 --2- 172.21.15.22:0/3748526875 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5616c04a78b0 0x5616c04a1160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:09:02.315 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:02.313+0000 7f0efbe00640 1 -- 172.21.15.22:0/3748526875 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5616c04a78b0 msgr2=0x5616c04a1160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T18:09:02.316 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:02.314+0000 7f0f03e00640 1 --2- 172.21.15.22:0/3748526875 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5616c04b0de0 0x5616c04b06f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:09:02.316 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:02.314+0000 7f0efbe00640 1 --2- 172.21.15.22:0/3748526875 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5616c04a78b0 0x5616c04a1160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:09:02.316 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:02.314+0000 7f0efbe00640 1 -- 172.21.15.22:0/3748526875 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5616c04b0de0 msgr2=0x5616c04b06f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T18:09:02.316 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:02.314+0000 7f0efbe00640 1 --2- 172.21.15.22:0/3748526875 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5616c04b0de0 0x5616c04b06f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:09:02.316 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:02.314+0000 7f0efbe00640 1 -- 172.21.15.22:0/3748526875 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5616c04c33c0 con 0x5616c04a8260 2024-03-28T18:09:02.316 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:02.314+0000 7f0f03400640 1 --2- 172.21.15.22:0/3748526875 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5616c04a78b0 0x5616c04a1160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T18:09:02.316 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:02.314+0000 7f0f03e00640 1 --2- 172.21.15.22:0/3748526875 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5616c04b0de0 0x5616c04b06f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T18:09:02.316 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:02.314+0000 7f0efbe00640 1 --2- 172.21.15.22:0/3748526875 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5616c04a8260 0x5616c04b01b0 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7f0eec0b99a0 tx=0x7f0eec004fb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=add36a716e9798cd server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:09:02.316 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:02.314+0000 7f0ef3e00640 1 -- 172.21.15.22:0/3748526875 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0eec0c1120 con 0x5616c04a8260 2024-03-28T18:09:02.316 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:02.314+0000 7f0ef3e00640 1 -- 172.21.15.22:0/3748526875 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f0eec0c5770 con 0x5616c04a8260 2024-03-28T18:09:02.316 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:02.314+0000 7f0ef3e00640 1 -- 172.21.15.22:0/3748526875 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0eec003b10 con 0x5616c04a8260 2024-03-28T18:09:02.316 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:02.315+0000 7f0f0d226740 1 -- 172.21.15.22:0/3748526875 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5616c04a8260 msgr2=0x5616c04b01b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:09:02.316 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:02.315+0000 7f0f0d226740 1 --2- 172.21.15.22:0/3748526875 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5616c04a8260 0x5616c04b01b0 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7f0eec0b99a0 tx=0x7f0eec004fb0 comp rx=0 tx=0).stop 2024-03-28T18:09:02.316 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:02.315+0000 7f0f0d226740 1 -- 172.21.15.22:0/3748526875 shutdown_connections 2024-03-28T18:09:02.317 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:02.315+0000 7f0f0d226740 1 --2- 172.21.15.22:0/3748526875 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5616c04b0de0 0x5616c04b06f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:09:02.317 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:02.315+0000 7f0f0d226740 1 --2- 172.21.15.22:0/3748526875 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5616c04a8260 0x5616c04b01b0 unknown :-1 s=CLOSED pgs=564 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:09:02.317 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:02.315+0000 7f0f0d226740 1 --2- 172.21.15.22:0/3748526875 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5616c04a78b0 0x5616c04a1160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:09:02.317 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:02.315+0000 7f0f0d226740 1 -- 172.21.15.22:0/3748526875 >> 172.21.15.22:0/3748526875 conn(0x5616c04a1f80 msgr2=0x5616c04a3280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:09:02.317 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:02.315+0000 7f0f0d226740 1 -- 172.21.15.22:0/3748526875 shutdown_connections 2024-03-28T18:09:02.317 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:02.315+0000 7f0f0d226740 1 -- 172.21.15.22:0/3748526875 wait complete. 2024-03-28T18:09:02.317 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:02.315+0000 7f0f0d226740 1 -- start start 2024-03-28T18:09:02.317 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:02.316+0000 7f0f0d226740 1 Processor -- start 2024-03-28T18:09:02.320 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:02.316+0000 7f0f0d226740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5616c04a78b0 0x5616c04b8da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:09:02.320 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:02.316+0000 7f0f0d226740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5616c04a8260 0x5616c04b7140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:09:02.320 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:02.316+0000 7f0f0d226740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5616c04b0de0 0x5616c04b7680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:09:02.320 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:02.316+0000 7f0f0d226740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f041b4700 con 0x5616c04b0de0 2024-03-28T18:09:02.320 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:02.316+0000 7f0f0d226740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f041b40a0 con 0x5616c04a78b0 2024-03-28T18:09:02.320 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:02.316+0000 7f0f0d226740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f041b15c0 con 0x5616c04a8260 2024-03-28T18:09:02.320 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:02.316+0000 7f0f0d226740 10 client.0 _ll_register_callbacks cb 0x7f0ef4005d30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-28T18:09:02.320 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:02.317+0000 7f0f03e00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5616c04b0de0 0x5616c04b7680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:09:02.320 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:02.317+0000 7f0f03400640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5616c04a78b0 0x5616c04b8da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:09:02.320 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:02.317+0000 7f0f03e00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5616c04b0de0 0x5616c04b7680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:54572/0 (socket says 172.21.15.22:54572) 2024-03-28T18:09:02.320 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:02.317+0000 7f0f03400640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5616c04a78b0 0x5616c04b8da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.22:35800/0 (socket says 172.21.15.22:35800) 2024-03-28T18:09:02.321 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:02.317+0000 7f0f03e00640 1 -- 172.21.15.22:0/206745707 learned_addr learned my addr 172.21.15.22:0/206745707 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-28T18:09:02.321 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:02.317+0000 7f0efbe00640 1 --2- 172.21.15.22:0/206745707 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5616c04a8260 0x5616c04b7140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:09:02.321 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:02.317+0000 7f0f03e00640 1 -- 172.21.15.22:0/206745707 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5616c04a8260 msgr2=0x5616c04b7140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T18:09:02.321 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:02.317+0000 7f0f03e00640 1 --2- 172.21.15.22:0/206745707 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5616c04a8260 0x5616c04b7140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:09:02.321 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:02.317+0000 7f0f03e00640 1 -- 172.21.15.22:0/206745707 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5616c04a78b0 msgr2=0x5616c04b8da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T18:09:02.321 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:02.317+0000 7f0f03e00640 1 --2- 172.21.15.22:0/206745707 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5616c04a78b0 0x5616c04b8da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:09:02.321 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:02.317+0000 7f0f03e00640 1 -- 172.21.15.22:0/206745707 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5616c04ae390 con 0x5616c04b0de0 2024-03-28T18:09:02.321 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:02.317+0000 7f0efbe00640 1 --2- 172.21.15.22:0/206745707 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5616c04a8260 0x5616c04b7140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-28T18:09:02.321 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:02.318+0000 7f0f03e00640 1 --2- 172.21.15.22:0/206745707 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5616c04b0de0 0x5616c04b7680 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7f0efc000f80 tx=0x7f0efc003ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ff759f36887d0313 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:09:02.321 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:02.318+0000 7f0ef1600640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-03-28T18:09:02.321 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:02.318+0000 7f0ef1600640 1 -- 172.21.15.22:0/206745707 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0efc028070 con 0x5616c04b0de0 2024-03-28T18:09:02.321 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:02.318+0000 7f0ef1600640 1 -- 172.21.15.22:0/206745707 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 8411+0+0 (secure 0 0 0) 0x7f0efc005450 con 0x5616c04b0de0 2024-03-28T18:09:02.321 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:02.318+0000 7f0f0d226740 1 -- 172.21.15.22:0/206745707 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f0eec0c5770 con 0x5616c04b0de0 2024-03-28T18:09:02.321 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:02.318+0000 7f0ef1600640 1 -- 172.21.15.22:0/206745707 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0efc005650 con 0x5616c04b0de0 2024-03-28T18:09:02.322 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:02.318+0000 7f0f0d226740 10 client.15969 fetch_fsmap learned FSMap version 27 2024-03-28T18:09:02.322 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:02.318+0000 7f0f0d226740 1 -- 172.21.15.22:0/206745707 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x5616c051d0f0 con 0x5616c04b0de0 2024-03-28T18:09:02.322 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:02.319+0000 7f0ef1600640 1 -- 172.21.15.22:0/206745707 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f0eec0c5770 con 0x5616c04b0de0 2024-03-28T18:09:02.322 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:02.319+0000 7f0ef1600640 1 -- 172.21.15.22:0/206745707 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 27) v1 ==== 7934+0+0 (secure 0 0 0) 0x7f0efc019430 con 0x5616c04b0de0 2024-03-28T18:09:02.322 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:02.319+0000 7f0f0d226740 10 client.15969 fetch_fsmap finished waiting for FSMap version 27 2024-03-28T18:09:02.322 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:02.319+0000 7f0f0d226740 10 client.15969 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.k 2024-03-28T18:09:02.322 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:02.319+0000 7f0f0d226740 20 client.15969 populate_metadata read hostname 'smithi022' 2024-03-28T18:09:02.322 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:02.319+0000 7f0f0d226740 1 --2- 172.21.15.22:0/206745707 >> [v2:172.21.15.79:6832/856896997,v1:172.21.15.79:6833/856896997] conn(0x5616c051d6d0 0x5616c051faf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:09:02.322 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:02.319+0000 7f0f0d226740 4 client.15969 mds_command: new command op to 34442 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-28T18:09:02.322 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:02.319+0000 7f0f0d226740 1 -- 172.21.15.22:0/206745707 --> [v2:172.21.15.79:6832/856896997,v1:172.21.15.79:6833/856896997] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x5616c0412b50 con 0x5616c051d6d0 2024-03-28T18:09:02.322 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:02.319+0000 7f0f03400640 1 --2- 172.21.15.22:0/206745707 >> [v2:172.21.15.79:6832/856896997,v1:172.21.15.79:6833/856896997] conn(0x5616c051d6d0 0x5616c051faf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:09:02.322 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:02.320+0000 7f0f03400640 1 --2- 172.21.15.22:0/206745707 >> [v2:172.21.15.79:6832/856896997,v1:172.21.15.79:6833/856896997] conn(0x5616c051d6d0 0x5616c051faf0 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=9a14187d5259b65e server_cookie=d487e74b7aa7e8d1 in_seq=0 out_seq=0 2024-03-28T18:09:02.322 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:02.320+0000 7f0ef1600640 10 client.15969 ms_handle_connect on v2:172.21.15.79:6832/856896997 2024-03-28T18:09:02.323 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:02.321+0000 7f0ef1600640 1 -- 172.21.15.22:0/206745707 <== mds.3 v2:172.21.15.79:6832/856896997 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x5616c0412b50 con 0x5616c051d6d0 2024-03-28T18:09:02.323 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:02.321+0000 7f0ef1600640 10 client.15969 handle_command_reply: tid=0 2024-03-28T18:09:02.324 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:02.321+0000 7f0f0d226740 1 client.15969 shutdown 2024-03-28T18:09:02.324 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:02.321+0000 7f0f0d226740 1 -- 172.21.15.22:0/206745707 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5616c04b0de0 msgr2=0x5616c04b7680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:09:02.324 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:02.321+0000 7f0f0d226740 1 --2- 172.21.15.22:0/206745707 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5616c04b0de0 0x5616c04b7680 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7f0efc000f80 tx=0x7f0efc003ce0 comp rx=0 tx=0).stop 2024-03-28T18:09:02.324 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:02.321+0000 7f0f0d226740 1 -- 172.21.15.22:0/206745707 shutdown_connections 2024-03-28T18:09:02.324 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:02.321+0000 7f0f0d226740 1 --2- 172.21.15.22:0/206745707 >> [v2:172.21.15.79:6832/856896997,v1:172.21.15.79:6833/856896997] conn(0x5616c051d6d0 0x5616c051faf0 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:09:02.324 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:02.321+0000 7f0f0d226740 1 --2- 172.21.15.22:0/206745707 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5616c04b0de0 0x5616c04b7680 unknown :-1 s=CLOSED pgs=565 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:09:02.324 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:02.321+0000 7f0f0d226740 1 --2- 172.21.15.22:0/206745707 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5616c04a8260 0x5616c04b7140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:09:02.324 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:02.321+0000 7f0f0d226740 1 --2- 172.21.15.22:0/206745707 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5616c04a78b0 0x5616c04b8da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:09:02.324 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:02.321+0000 7f0f0d226740 1 -- 172.21.15.22:0/206745707 >> 172.21.15.22:0/206745707 conn(0x5616c04a1f80 msgr2=0x5616c04a0fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:09:02.324 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:02.321+0000 7f0ef1600640 0 client.15969 ms_handle_reset on v2:172.21.15.79:6832/856896997 2024-03-28T18:09:02.324 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:02.321+0000 7f0f0d226740 1 -- 172.21.15.22:0/206745707 shutdown_connections 2024-03-28T18:09:02.324 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:02.321+0000 7f0f0d226740 1 -- 172.21.15.22:0/206745707 wait complete. 2024-03-28T18:09:02.324 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:02.321+0000 7f0f0d226740 20 client.15969 trim_cache size 0 max 16384 2024-03-28T18:09:02.325 INFO:teuthology.orchestra.run.smithi022.stdout:[] 2024-03-28T18:09:02.325 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:02.322+0000 7f0f09000640 1 -- 172.21.15.22:0/1317656137 >> [v2:172.21.15.22:6800/1357365082,v1:172.21.15.22:6801/1357365082] conn(0x7f0ee80a2b10 msgr2=0x7f0ee80a4fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:09:02.326 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:02.322+0000 7f0f09000640 1 --2- 172.21.15.22:0/1317656137 >> [v2:172.21.15.22:6800/1357365082,v1:172.21.15.22:6801/1357365082] conn(0x7f0ee80a2b10 0x7f0ee80a4fd0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x5616c04a2ff0 tx=0x7f0ef400b410 comp rx=0 tx=0).stop 2024-03-28T18:09:02.326 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:02.322+0000 7f0f09000640 1 -- 172.21.15.22:0/1317656137 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0f04100de0 msgr2=0x7f0f0419e940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:09:02.326 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:02.322+0000 7f0f09000640 1 --2- 172.21.15.22:0/1317656137 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0f04100de0 0x7f0f0419e940 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7f0eec000c00 tx=0x7f0eec004e40 comp rx=0 tx=0).stop 2024-03-28T18:09:02.326 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:02.322+0000 7f0f09000640 1 -- 172.21.15.22:0/1317656137 shutdown_connections 2024-03-28T18:09:02.326 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:02.322+0000 7f0f09000640 1 --2- 172.21.15.22:0/1317656137 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0f041058c0 0x7f0f041a2cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:09:02.326 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:02.322+0000 7f0f09000640 1 --2- 172.21.15.22:0/1317656137 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0f04100de0 0x7f0f0419e940 unknown :-1 s=CLOSED pgs=562 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:09:02.326 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:02.322+0000 7f0f09000640 1 --2- 172.21.15.22:0/1317656137 >> [v2:172.21.15.22:6800/1357365082,v1:172.21.15.22:6801/1357365082] conn(0x7f0ee80a2b10 0x7f0ee80a4fd0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:09:02.326 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:02.322+0000 7f0f09000640 1 --2- 172.21.15.22:0/1317656137 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f0f041004c0 0x7f0f0419e400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:09:02.326 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:02.322+0000 7f0f09000640 1 -- 172.21.15.22:0/1317656137 >> 172.21.15.22:0/1317656137 conn(0x7f0f040fbef0 msgr2=0x7f0f04103270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:09:02.326 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:02.322+0000 7f0f09000640 1 -- 172.21.15.22:0/1317656137 shutdown_connections 2024-03-28T18:09:02.326 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:02.322+0000 7f0f09000640 1 -- 172.21.15.22:0/1317656137 wait complete. 2024-03-28T18:09:02.339 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-28T18:09:02.339 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-03-28T18:09:02.628 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:02.628+0000 7fa755e00640 1 -- 172.21.15.22:0/2913580595 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa750107640 msgr2=0x7fa750107a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:09:02.628 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:02.628+0000 7fa755e00640 1 --2- 172.21.15.22:0/2913580595 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa750107640 0x7fa750107a20 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7fa740009930 tx=0x7fa74002bc30 comp rx=0 tx=0).stop 2024-03-28T18:09:02.628 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:02.628+0000 7fa755e00640 1 -- 172.21.15.22:0/2913580595 shutdown_connections 2024-03-28T18:09:02.628 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:02.628+0000 7fa755e00640 1 --2- 172.21.15.22:0/2913580595 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa750101ff0 0x7fa75010e500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:09:02.628 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:02.628+0000 7fa755e00640 1 --2- 172.21.15.22:0/2913580595 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa750101630 0x7fa750101ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:09:02.628 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:02.628+0000 7fa755e00640 1 --2- 172.21.15.22:0/2913580595 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa750107640 0x7fa750107a20 unknown :-1 s=CLOSED pgs=566 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:09:02.629 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:02.628+0000 7fa755e00640 1 -- 172.21.15.22:0/2913580595 >> 172.21.15.22:0/2913580595 conn(0x7fa750078aa0 msgr2=0x7fa750078eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:09:02.629 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:02.628+0000 7fa755e00640 1 -- 172.21.15.22:0/2913580595 shutdown_connections 2024-03-28T18:09:02.629 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:02.628+0000 7fa755e00640 1 -- 172.21.15.22:0/2913580595 wait complete. 2024-03-28T18:09:02.629 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:02.630+0000 7fa755e00640 1 Processor -- start 2024-03-28T18:09:02.629 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:02.630+0000 7fa755e00640 1 -- start start 2024-03-28T18:09:02.630 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:02.630+0000 7fa755e00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa750101630 0x7fa7500739a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:09:02.630 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:02.630+0000 7fa755e00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa750101ff0 0x7fa750073ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:09:02.630 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:02.630+0000 7fa755e00640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa750107640 0x7fa750074420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:09:02.630 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:02.630+0000 7fa755e00640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa750111a70 con 0x7fa750101ff0 2024-03-28T18:09:02.630 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:02.630+0000 7fa755e00640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7501118f0 con 0x7fa750107640 2024-03-28T18:09:02.630 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:02.630+0000 7fa755e00640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa750111bf0 con 0x7fa750101630 2024-03-28T18:09:02.632 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:02.631+0000 7fa74e000640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa750101ff0 0x7fa750073ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:09:02.632 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:02.631+0000 7fa74e000640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa750101ff0 0x7fa750073ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:54594/0 (socket says 172.21.15.22:54594) 2024-03-28T18:09:02.632 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:02.631+0000 7fa74e000640 1 -- 172.21.15.22:0/4159534876 learned_addr learned my addr 172.21.15.22:0/4159534876 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-28T18:09:02.632 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:02.631+0000 7fa74f400640 1 --2- 172.21.15.22:0/4159534876 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa750107640 0x7fa750074420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:09:02.632 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:02.631+0000 7fa74ea00640 1 --2- 172.21.15.22:0/4159534876 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa750101630 0x7fa7500739a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:09:02.632 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:02.631+0000 7fa74e000640 1 -- 172.21.15.22:0/4159534876 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa750101630 msgr2=0x7fa7500739a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:09:02.632 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:02.631+0000 7fa74e000640 1 --2- 172.21.15.22:0/4159534876 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa750101630 0x7fa7500739a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:09:02.632 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:02.631+0000 7fa74e000640 1 -- 172.21.15.22:0/4159534876 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa750107640 msgr2=0x7fa750074420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:09:02.632 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:02.631+0000 7fa74e000640 1 --2- 172.21.15.22:0/4159534876 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa750107640 0x7fa750074420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:09:02.632 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:02.631+0000 7fa74e000640 1 -- 172.21.15.22:0/4159534876 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa7501a6410 con 0x7fa750101ff0 2024-03-28T18:09:02.632 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:02.632+0000 7fa74e000640 1 --2- 172.21.15.22:0/4159534876 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa750101ff0 0x7fa750073ee0 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7fa73c00d950 tx=0x7fa73c00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:09:02.632 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:02.632+0000 7fa754a00640 1 -- 172.21.15.22:0/4159534876 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa73c017070 con 0x7fa750101ff0 2024-03-28T18:09:02.633 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:02.632+0000 7fa754a00640 1 -- 172.21.15.22:0/4159534876 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7fa73c009c30 con 0x7fa750101ff0 2024-03-28T18:09:02.633 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:02.632+0000 7fa755e00640 1 -- 172.21.15.22:0/4159534876 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa7501a6700 con 0x7fa750101ff0 2024-03-28T18:09:02.633 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:02.632+0000 7fa754a00640 1 -- 172.21.15.22:0/4159534876 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa73c00f6b0 con 0x7fa750101ff0 2024-03-28T18:09:02.633 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:02.632+0000 7fa755e00640 1 -- 172.21.15.22:0/4159534876 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa7501a6c50 con 0x7fa750101ff0 2024-03-28T18:09:02.635 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:02.635+0000 7fa758baf740 1 Processor -- start 2024-03-28T18:09:02.635 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:02.636+0000 7fa758baf740 1 -- start start 2024-03-28T18:09:02.637 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:02.636+0000 7fa758baf740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x558a219986d0 0x558a21998ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:09:02.637 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:02.636+0000 7fa758baf740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558a21999080 0x558a219a1630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:09:02.637 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:02.636+0000 7fa758baf740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558a219a1c00 0x558a219a4050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:09:02.637 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:02.636+0000 7fa758baf740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x558a218fc2c0 con 0x558a21999080 2024-03-28T18:09:02.637 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:02.636+0000 7fa758baf740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x558a218a4c90 con 0x558a219986d0 2024-03-28T18:09:02.637 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:02.636+0000 7fa758baf740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x558a218f8360 con 0x558a219a1c00 2024-03-28T18:09:02.637 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:02.636+0000 7fa74ea00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558a21999080 0x558a219a1630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:09:02.637 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:02.637+0000 7fa74ea00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558a21999080 0x558a219a1630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:54610/0 (socket says 172.21.15.22:54610) 2024-03-28T18:09:02.637 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:02.637+0000 7fa74ea00640 1 -- 172.21.15.22:0/3416930776 learned_addr learned my addr 172.21.15.22:0/3416930776 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-28T18:09:02.637 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:02.637+0000 7fa74f400640 1 --2- 172.21.15.22:0/3416930776 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x558a219986d0 0x558a21998ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:09:02.637 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:02.637+0000 7fa754a00640 1 -- 172.21.15.22:0/4159534876 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 25) v1 ==== 137523+0+0 (secure 0 0 0) 0x7fa73c010a50 con 0x7fa750101ff0 2024-03-28T18:09:02.637 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:02.637+0000 7fa74ea00640 1 -- 172.21.15.22:0/3416930776 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558a219a1c00 msgr2=0x558a219a4050 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-28T18:09:02.637 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:02.637+0000 7fa74ea00640 1 --2- 172.21.15.22:0/3416930776 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558a219a1c00 0x558a219a4050 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:09:02.637 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:02.637+0000 7fa74ea00640 1 -- 172.21.15.22:0/3416930776 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x558a219986d0 msgr2=0x558a21998ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T18:09:02.638 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:02.637+0000 7fa74ea00640 1 --2- 172.21.15.22:0/3416930776 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x558a219986d0 0x558a21998ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:09:02.638 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:02.637+0000 7fa74ea00640 1 -- 172.21.15.22:0/3416930776 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558a218c9080 con 0x558a21999080 2024-03-28T18:09:02.639 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:02.638+0000 7fa74ea00640 1 --2- 172.21.15.22:0/3416930776 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558a21999080 0x558a219a1630 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7fa740000c00 tx=0x7fa740005dc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3e75f277e92c420e server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:09:02.639 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:02.638+0000 7fa73be00640 1 -- 172.21.15.22:0/3416930776 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa740044020 con 0x558a21999080 2024-03-28T18:09:02.639 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:02.638+0000 7fa73be00640 1 -- 172.21.15.22:0/3416930776 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7fa740005510 con 0x558a21999080 2024-03-28T18:09:02.639 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:02.638+0000 7fa73be00640 1 -- 172.21.15.22:0/3416930776 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7400046c0 con 0x558a21999080 2024-03-28T18:09:02.639 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:02.638+0000 7fa754a00640 1 --2- 172.21.15.22:0/4159534876 >> [v2:172.21.15.22:6800/1357365082,v1:172.21.15.22:6801/1357365082] conn(0x7fa7340a2ac0 0x7fa7340a4f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:09:02.639 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:02.638+0000 7fa754a00640 1 -- 172.21.15.22:0/4159534876 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 8411+0+0 (secure 0 0 0) 0x7fa73c0ced60 con 0x7fa750101ff0 2024-03-28T18:09:02.640 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:02.639+0000 7fa74f400640 1 --2- 172.21.15.22:0/4159534876 >> [v2:172.21.15.22:6800/1357365082,v1:172.21.15.22:6801/1357365082] conn(0x7fa7340a2ac0 0x7fa7340a4f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:09:02.640 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:02.639+0000 7fa758baf740 1 -- 172.21.15.22:0/3416930776 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558a21999080 msgr2=0x558a219a1630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:09:02.640 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:02.639+0000 7fa758baf740 1 --2- 172.21.15.22:0/3416930776 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558a21999080 0x558a219a1630 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7fa740000c00 tx=0x7fa740005dc0 comp rx=0 tx=0).stop 2024-03-28T18:09:02.640 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:02.639+0000 7fa758baf740 1 -- 172.21.15.22:0/3416930776 shutdown_connections 2024-03-28T18:09:02.640 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:02.639+0000 7fa758baf740 1 --2- 172.21.15.22:0/3416930776 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558a219a1c00 0x558a219a4050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:09:02.640 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:02.639+0000 7fa758baf740 1 --2- 172.21.15.22:0/3416930776 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558a21999080 0x558a219a1630 unknown :-1 s=CLOSED pgs=568 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:09:02.640 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:02.639+0000 7fa758baf740 1 --2- 172.21.15.22:0/3416930776 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x558a219986d0 0x558a21998ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:09:02.640 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:02.639+0000 7fa758baf740 1 -- 172.21.15.22:0/3416930776 >> 172.21.15.22:0/3416930776 conn(0x558a21992da0 msgr2=0x558a21993180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:09:02.640 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:02.640+0000 7fa758baf740 1 -- 172.21.15.22:0/3416930776 shutdown_connections 2024-03-28T18:09:02.640 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:02.640+0000 7fa758baf740 1 -- 172.21.15.22:0/3416930776 wait complete. 2024-03-28T18:09:02.640 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:02.640+0000 7fa74f400640 1 --2- 172.21.15.22:0/4159534876 >> [v2:172.21.15.22:6800/1357365082,v1:172.21.15.22:6801/1357365082] conn(0x7fa7340a2ac0 0x7fa7340a4f80 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fa748005fd0 tx=0x7fa748009500 comp rx=0 tx=0).ready entity=mgr.15522 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:09:02.642 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:02.640+0000 7fa758baf740 1 -- start start 2024-03-28T18:09:02.642 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:02.641+0000 7fa758baf740 1 Processor -- start 2024-03-28T18:09:02.643 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:02.641+0000 7fa758baf740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558a219986d0 0x558a219a1490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:09:02.643 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:02.641+0000 7fa758baf740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x558a21999080 0x558a2199e820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:09:02.643 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:02.641+0000 7fa758baf740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558a219a1c00 0x558a2199ed60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:09:02.643 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:02.641+0000 7fa758baf740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x558a219a4b10 con 0x558a219986d0 2024-03-28T18:09:02.643 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:02.641+0000 7fa758baf740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x558a218b80d0 con 0x558a21999080 2024-03-28T18:09:02.643 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:02.641+0000 7fa758baf740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x558a219a4c90 con 0x558a219a1c00 2024-03-28T18:09:02.643 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:02.641+0000 7fa758baf740 10 client.0 _ll_register_callbacks cb 0x558a218bce80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-28T18:09:02.643 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:02.641+0000 7fa74ea00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558a219986d0 0x558a219a1490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:09:02.643 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:02.642+0000 7fa74ea00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558a219986d0 0x558a219a1490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:54626/0 (socket says 172.21.15.22:54626) 2024-03-28T18:09:02.643 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:02.642+0000 7fa74ea00640 1 -- 172.21.15.22:0/1353918083 learned_addr learned my addr 172.21.15.22:0/1353918083 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-28T18:09:02.643 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:02.642+0000 7fa74e000640 1 --2- 172.21.15.22:0/1353918083 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x558a21999080 0x558a2199e820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:09:02.643 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:02.642+0000 7fa74f400640 1 --2- 172.21.15.22:0/1353918083 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558a219a1c00 0x558a2199ed60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:09:02.643 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:02.642+0000 7fa74ea00640 1 -- 172.21.15.22:0/1353918083 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558a219a1c00 msgr2=0x558a2199ed60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T18:09:02.643 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:02.642+0000 7fa74ea00640 1 --2- 172.21.15.22:0/1353918083 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558a219a1c00 0x558a2199ed60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:09:02.643 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:02.642+0000 7fa74ea00640 1 -- 172.21.15.22:0/1353918083 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x558a21999080 msgr2=0x558a2199e820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T18:09:02.644 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:02.642+0000 7fa74ea00640 1 --2- 172.21.15.22:0/1353918083 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x558a21999080 0x558a2199e820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:09:02.644 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:02.642+0000 7fa74ea00640 1 -- 172.21.15.22:0/1353918083 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x558a216103f0 con 0x558a219986d0 2024-03-28T18:09:02.644 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:02.642+0000 7fa74ea00640 1 --2- 172.21.15.22:0/1353918083 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558a219986d0 0x558a219a1490 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7fa740006010 tx=0x7fa740005850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7bc26032bd6baeef server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:09:02.644 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:02.642+0000 7fa738c00640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-03-28T18:09:02.644 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:02.643+0000 7fa738c00640 1 -- 172.21.15.22:0/1353918083 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa74004d030 con 0x558a219986d0 2024-03-28T18:09:02.644 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:02.643+0000 7fa738c00640 1 -- 172.21.15.22:0/1353918083 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 8411+0+0 (secure 0 0 0) 0x7fa7400035f0 con 0x558a219986d0 2024-03-28T18:09:02.644 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:02.643+0000 7fa758baf740 1 -- 172.21.15.22:0/1353918083 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fa740005510 con 0x558a219986d0 2024-03-28T18:09:02.645 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:02.643+0000 7fa738c00640 1 -- 172.21.15.22:0/1353918083 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa74005a050 con 0x558a219986d0 2024-03-28T18:09:02.645 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:02.643+0000 7fa758baf740 10 client.15981 fetch_fsmap learned FSMap version 27 2024-03-28T18:09:02.645 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:02.643+0000 7fa758baf740 1 -- 172.21.15.22:0/1353918083 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x558a216da7d0 con 0x558a219986d0 2024-03-28T18:09:02.645 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:02.644+0000 7fa738c00640 1 -- 172.21.15.22:0/1353918083 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7fa740005510 con 0x558a219986d0 2024-03-28T18:09:02.645 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:02.644+0000 7fa738c00640 1 -- 172.21.15.22:0/1353918083 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 27) v1 ==== 7934+0+0 (secure 0 0 0) 0x7fa740040930 con 0x558a219986d0 2024-03-28T18:09:02.646 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:02.644+0000 7fa758baf740 10 client.15981 fetch_fsmap finished waiting for FSMap version 27 2024-03-28T18:09:02.646 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:02.644+0000 7fa758baf740 10 client.15981 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.e 2024-03-28T18:09:02.646 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:02.644+0000 7fa758baf740 20 client.15981 populate_metadata read hostname 'smithi022' 2024-03-28T18:09:02.646 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:02.644+0000 7fa758baf740 1 --2- 172.21.15.22:0/1353918083 >> [v2:172.21.15.79:6834/3365773431,v1:172.21.15.79:6835/3365773431] conn(0x558a219b1880 0x558a219b3c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:09:02.646 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:02.644+0000 7fa758baf740 4 client.15981 mds_command: new command op to 34475 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-28T18:09:02.646 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:02.644+0000 7fa758baf740 1 -- 172.21.15.22:0/1353918083 --> [v2:172.21.15.79:6834/3365773431,v1:172.21.15.79:6835/3365773431] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x558a21909800 con 0x558a219b1880 2024-03-28T18:09:02.646 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:02.645+0000 7fa74e000640 1 --2- 172.21.15.22:0/1353918083 >> [v2:172.21.15.79:6834/3365773431,v1:172.21.15.79:6835/3365773431] conn(0x558a219b1880 0x558a219b3c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:09:02.646 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:02.645+0000 7fa74e000640 1 --2- 172.21.15.22:0/1353918083 >> [v2:172.21.15.79:6834/3365773431,v1:172.21.15.79:6835/3365773431] conn(0x558a219b1880 0x558a219b3c70 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=762c0b28b272f650 server_cookie=1700af6d7ff81766 in_seq=0 out_seq=0 2024-03-28T18:09:02.646 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:02.645+0000 7fa738c00640 10 client.15981 ms_handle_connect on v2:172.21.15.79:6834/3365773431 2024-03-28T18:09:02.647 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:02.647+0000 7fa738c00640 1 -- 172.21.15.22:0/1353918083 <== mds.2 v2:172.21.15.79:6834/3365773431 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x558a21909800 con 0x558a219b1880 2024-03-28T18:09:02.647 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:02.647+0000 7fa738c00640 10 client.15981 handle_command_reply: tid=0 2024-03-28T18:09:02.647 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:02.647+0000 7fa758baf740 1 client.15981 shutdown 2024-03-28T18:09:02.648 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:02.647+0000 7fa758baf740 1 -- 172.21.15.22:0/1353918083 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558a219986d0 msgr2=0x558a219a1490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:09:02.648 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:02.647+0000 7fa758baf740 1 --2- 172.21.15.22:0/1353918083 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558a219986d0 0x558a219a1490 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7fa740006010 tx=0x7fa740005850 comp rx=0 tx=0).stop 2024-03-28T18:09:02.648 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:02.647+0000 7fa758baf740 1 -- 172.21.15.22:0/1353918083 shutdown_connections 2024-03-28T18:09:02.648 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:02.647+0000 7fa758baf740 1 --2- 172.21.15.22:0/1353918083 >> [v2:172.21.15.79:6834/3365773431,v1:172.21.15.79:6835/3365773431] conn(0x558a219b1880 0x558a219b3c70 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:09:02.648 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:02.647+0000 7fa758baf740 1 --2- 172.21.15.22:0/1353918083 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558a219a1c00 0x558a2199ed60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:09:02.648 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:02.647+0000 7fa758baf740 1 --2- 172.21.15.22:0/1353918083 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x558a21999080 0x558a2199e820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:09:02.648 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:02.647+0000 7fa758baf740 1 --2- 172.21.15.22:0/1353918083 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558a219986d0 0x558a219a1490 unknown :-1 s=CLOSED pgs=569 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:09:02.648 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:02.647+0000 7fa738c00640 0 client.15981 ms_handle_reset on v2:172.21.15.79:6834/3365773431 2024-03-28T18:09:02.648 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:02.647+0000 7fa758baf740 1 -- 172.21.15.22:0/1353918083 >> 172.21.15.22:0/1353918083 conn(0x558a21992da0 msgr2=0x558a21994af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:09:02.648 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:02.648+0000 7fa758baf740 1 -- 172.21.15.22:0/1353918083 shutdown_connections 2024-03-28T18:09:02.648 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:02.648+0000 7fa758baf740 1 -- 172.21.15.22:0/1353918083 wait complete. 2024-03-28T18:09:02.648 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:02.648+0000 7fa758baf740 20 client.15981 trim_cache size 0 max 16384 2024-03-28T18:09:02.662 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:02.662+0000 7fa758baf740 1 Processor -- start 2024-03-28T18:09:02.662 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:02.662+0000 7fa758baf740 1 -- start start 2024-03-28T18:09:02.662 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:02.662+0000 7fa758baf740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x558a219986d0 0x558a21999f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:09:02.662 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:02.662+0000 7fa758baf740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558a21999080 0x558a219a0f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:09:02.662 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:02.662+0000 7fa758baf740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558a219a1c00 0x558a219a1490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:09:02.662 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:02.662+0000 7fa758baf740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7501af270 con 0x558a21999080 2024-03-28T18:09:02.662 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:02.662+0000 7fa758baf740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7501b0bf0 con 0x558a219986d0 2024-03-28T18:09:02.662 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:02.662+0000 7fa758baf740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7501af8d0 con 0x558a219a1c00 2024-03-28T18:09:02.663 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:02.663+0000 7fa74e000640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558a21999080 0x558a219a0f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:09:02.664 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:02.663+0000 7fa74e000640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558a21999080 0x558a219a0f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:54632/0 (socket says 172.21.15.22:54632) 2024-03-28T18:09:02.664 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:02.663+0000 7fa74e000640 1 -- 172.21.15.22:0/41313718 learned_addr learned my addr 172.21.15.22:0/41313718 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-28T18:09:02.664 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:02.663+0000 7fa74ea00640 1 --2- 172.21.15.22:0/41313718 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x558a219986d0 0x558a21999f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:09:02.664 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:02.663+0000 7fa74f400640 1 --2- 172.21.15.22:0/41313718 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558a219a1c00 0x558a219a1490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:09:02.664 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:02.663+0000 7fa74e000640 1 -- 172.21.15.22:0/41313718 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558a219a1c00 msgr2=0x558a219a1490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T18:09:02.664 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:02.663+0000 7fa74e000640 1 --2- 172.21.15.22:0/41313718 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558a219a1c00 0x558a219a1490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:09:02.664 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:02.663+0000 7fa74e000640 1 -- 172.21.15.22:0/41313718 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x558a219986d0 msgr2=0x558a21999f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T18:09:02.664 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:02.663+0000 7fa74e000640 1 --2- 172.21.15.22:0/41313718 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x558a219986d0 0x558a21999f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:09:02.664 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:02.663+0000 7fa74e000640 1 -- 172.21.15.22:0/41313718 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558a2199f0b0 con 0x558a21999080 2024-03-28T18:09:02.664 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:02.664+0000 7fa74e000640 1 --2- 172.21.15.22:0/41313718 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558a21999080 0x558a219a0f50 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7fa73c09b110 tx=0x7fa73c0ce040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f2d0bb72804591cd server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:09:02.664 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:02.664+0000 7fa73b400640 1 -- 172.21.15.22:0/41313718 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa73c020040 con 0x558a21999080 2024-03-28T18:09:02.664 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:02.664+0000 7fa73b400640 1 -- 172.21.15.22:0/41313718 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7fa73c0a3920 con 0x558a21999080 2024-03-28T18:09:02.664 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:02.664+0000 7fa73b400640 1 -- 172.21.15.22:0/41313718 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa73c09a5d0 con 0x558a21999080 2024-03-28T18:09:02.665 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:02.665+0000 7fa758baf740 1 -- 172.21.15.22:0/41313718 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558a21999080 msgr2=0x558a219a0f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:09:02.665 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:02.665+0000 7fa758baf740 1 --2- 172.21.15.22:0/41313718 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558a21999080 0x558a219a0f50 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7fa73c09b110 tx=0x7fa73c0ce040 comp rx=0 tx=0).stop 2024-03-28T18:09:02.665 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:02.665+0000 7fa758baf740 1 -- 172.21.15.22:0/41313718 shutdown_connections 2024-03-28T18:09:02.665 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:02.665+0000 7fa758baf740 1 --2- 172.21.15.22:0/41313718 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558a219a1c00 0x558a219a1490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:09:02.665 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:02.665+0000 7fa758baf740 1 --2- 172.21.15.22:0/41313718 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558a21999080 0x558a219a0f50 unknown :-1 s=CLOSED pgs=570 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:09:02.666 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:02.665+0000 7fa758baf740 1 --2- 172.21.15.22:0/41313718 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x558a219986d0 0x558a21999f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:09:02.666 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:02.665+0000 7fa758baf740 1 -- 172.21.15.22:0/41313718 >> 172.21.15.22:0/41313718 conn(0x558a21992da0 msgr2=0x558a219ad7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:09:02.666 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:02.665+0000 7fa758baf740 1 -- 172.21.15.22:0/41313718 shutdown_connections 2024-03-28T18:09:02.666 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:02.665+0000 7fa758baf740 1 -- 172.21.15.22:0/41313718 wait complete. 2024-03-28T18:09:02.666 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:02.665+0000 7fa758baf740 1 -- start start 2024-03-28T18:09:02.666 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:02.666+0000 7fa758baf740 1 Processor -- start 2024-03-28T18:09:02.668 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:02.666+0000 7fa758baf740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558a219986d0 0x558a219b4c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:09:02.668 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:02.666+0000 7fa758baf740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x558a21999080 0x558a219b51a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:09:02.668 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:02.666+0000 7fa758baf740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558a219a1c00 0x558a219b6120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:09:02.668 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:02.666+0000 7fa758baf740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7501b0590 con 0x558a219986d0 2024-03-28T18:09:02.668 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:02.666+0000 7fa758baf740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7501aff30 con 0x558a21999080 2024-03-28T18:09:02.668 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:02.666+0000 7fa758baf740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7501ad450 con 0x558a219a1c00 2024-03-28T18:09:02.668 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:02.666+0000 7fa758baf740 10 client.0 _ll_register_callbacks cb 0x7fa748009b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-28T18:09:02.668 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:02.667+0000 7fa74ea00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558a219986d0 0x558a219b4c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:09:02.668 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:02.667+0000 7fa74ea00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558a219986d0 0x558a219b4c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:54648/0 (socket says 172.21.15.22:54648) 2024-03-28T18:09:02.669 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:02.667+0000 7fa74ea00640 1 -- 172.21.15.22:0/3006513282 learned_addr learned my addr 172.21.15.22:0/3006513282 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-28T18:09:02.669 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:02.667+0000 7fa74e000640 1 --2- 172.21.15.22:0/3006513282 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x558a21999080 0x558a219b51a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:09:02.669 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:02.667+0000 7fa74f400640 1 --2- 172.21.15.22:0/3006513282 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558a219a1c00 0x558a219b6120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:09:02.669 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:02.667+0000 7fa74ea00640 1 -- 172.21.15.22:0/3006513282 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558a219a1c00 msgr2=0x558a219b6120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T18:09:02.669 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:02.667+0000 7fa74ea00640 1 --2- 172.21.15.22:0/3006513282 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558a219a1c00 0x558a219b6120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:09:02.669 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:02.667+0000 7fa74ea00640 1 -- 172.21.15.22:0/3006513282 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x558a21999080 msgr2=0x558a219b51a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T18:09:02.669 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:02.667+0000 7fa74ea00640 1 --2- 172.21.15.22:0/3006513282 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x558a21999080 0x558a219b51a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:09:02.669 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:02.667+0000 7fa74ea00640 1 -- 172.21.15.22:0/3006513282 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x558a219b6810 con 0x558a219986d0 2024-03-28T18:09:02.669 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:02.667+0000 7fa74f400640 1 --2- 172.21.15.22:0/3006513282 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558a219a1c00 0x558a219b6120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T18:09:02.669 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:02.668+0000 7fa74ea00640 1 --2- 172.21.15.22:0/3006513282 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558a219986d0 0x558a219b4c60 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7fa74002c240 tx=0x7fa740005dc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1a56d5170c24ba51 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:09:02.669 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:02.668+0000 7fa738c00640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-03-28T18:09:02.669 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:02.668+0000 7fa738c00640 1 -- 172.21.15.22:0/3006513282 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa74004d070 con 0x558a219986d0 2024-03-28T18:09:02.669 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:02.668+0000 7fa738c00640 1 -- 172.21.15.22:0/3006513282 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 8411+0+0 (secure 0 0 0) 0x7fa740040880 con 0x558a219986d0 2024-03-28T18:09:02.669 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:02.668+0000 7fa758baf740 1 -- 172.21.15.22:0/3006513282 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fa73c0a3920 con 0x558a219986d0 2024-03-28T18:09:02.669 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:02.668+0000 7fa738c00640 1 -- 172.21.15.22:0/3006513282 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa740044040 con 0x558a219986d0 2024-03-28T18:09:02.669 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:02.668+0000 7fa758baf740 10 client.15987 fetch_fsmap learned FSMap version 27 2024-03-28T18:09:02.669 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:02.668+0000 7fa758baf740 1 -- 172.21.15.22:0/3006513282 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x558a21a0d9a0 con 0x558a219986d0 2024-03-28T18:09:02.671 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:02.669+0000 7fa738c00640 1 -- 172.21.15.22:0/3006513282 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7fa73c0a3920 con 0x558a219986d0 2024-03-28T18:09:02.671 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:02.669+0000 7fa738c00640 1 -- 172.21.15.22:0/3006513282 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 27) v1 ==== 7934+0+0 (secure 0 0 0) 0x7fa740049520 con 0x558a219986d0 2024-03-28T18:09:02.671 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:02.669+0000 7fa758baf740 10 client.15987 fetch_fsmap finished waiting for FSMap version 27 2024-03-28T18:09:02.671 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:02.669+0000 7fa758baf740 10 client.15987 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.e 2024-03-28T18:09:02.671 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:02.669+0000 7fa758baf740 20 client.15987 populate_metadata read hostname 'smithi022' 2024-03-28T18:09:02.672 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:02.669+0000 7fa758baf740 1 --2- 172.21.15.22:0/3006513282 >> [v2:172.21.15.79:6834/3365773431,v1:172.21.15.79:6835/3365773431] conn(0x558a21a0e030 0x558a21a10450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:09:02.672 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:02.669+0000 7fa758baf740 4 client.15987 mds_command: new command op to 34475 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-28T18:09:02.672 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:02.669+0000 7fa758baf740 1 -- 172.21.15.22:0/3006513282 --> [v2:172.21.15.79:6834/3365773431,v1:172.21.15.79:6835/3365773431] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x558a21903b50 con 0x558a21a0e030 2024-03-28T18:09:02.672 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:02.670+0000 7fa74e000640 1 --2- 172.21.15.22:0/3006513282 >> [v2:172.21.15.79:6834/3365773431,v1:172.21.15.79:6835/3365773431] conn(0x558a21a0e030 0x558a21a10450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:09:02.672 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:02.670+0000 7fa74e000640 1 --2- 172.21.15.22:0/3006513282 >> [v2:172.21.15.79:6834/3365773431,v1:172.21.15.79:6835/3365773431] conn(0x558a21a0e030 0x558a21a10450 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=e8dff3626a4b82fb server_cookie=6e7621d7e17909fd in_seq=0 out_seq=0 2024-03-28T18:09:02.672 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:02.670+0000 7fa738c00640 10 client.15987 ms_handle_connect on v2:172.21.15.79:6834/3365773431 2024-03-28T18:09:02.672 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:02.671+0000 7fa738c00640 1 -- 172.21.15.22:0/3006513282 <== mds.2 v2:172.21.15.79:6834/3365773431 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x558a21903b50 con 0x558a21a0e030 2024-03-28T18:09:02.672 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:02.671+0000 7fa738c00640 10 client.15987 handle_command_reply: tid=0 2024-03-28T18:09:02.672 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:02.671+0000 7fa758baf740 1 client.15987 shutdown 2024-03-28T18:09:02.672 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:02.671+0000 7fa758baf740 1 -- 172.21.15.22:0/3006513282 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558a219986d0 msgr2=0x558a219b4c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:09:02.672 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:02.671+0000 7fa758baf740 1 --2- 172.21.15.22:0/3006513282 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558a219986d0 0x558a219b4c60 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7fa74002c240 tx=0x7fa740005dc0 comp rx=0 tx=0).stop 2024-03-28T18:09:02.672 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:02.671+0000 7fa758baf740 1 -- 172.21.15.22:0/3006513282 shutdown_connections 2024-03-28T18:09:02.672 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:02.671+0000 7fa758baf740 1 --2- 172.21.15.22:0/3006513282 >> [v2:172.21.15.79:6834/3365773431,v1:172.21.15.79:6835/3365773431] conn(0x558a21a0e030 0x558a21a10450 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:09:02.672 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:02.671+0000 7fa758baf740 1 --2- 172.21.15.22:0/3006513282 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558a219a1c00 0x558a219b6120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:09:02.672 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:02.671+0000 7fa758baf740 1 --2- 172.21.15.22:0/3006513282 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x558a21999080 0x558a219b51a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:09:02.672 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:02.671+0000 7fa758baf740 1 --2- 172.21.15.22:0/3006513282 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558a219986d0 0x558a219b4c60 unknown :-1 s=CLOSED pgs=571 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:09:02.673 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:02.671+0000 7fa758baf740 1 -- 172.21.15.22:0/3006513282 >> 172.21.15.22:0/3006513282 conn(0x558a21992da0 msgr2=0x558a219acf40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:09:02.673 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:02.671+0000 7fa738c00640 0 client.15987 ms_handle_reset on v2:172.21.15.79:6834/3365773431 2024-03-28T18:09:02.673 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:02.671+0000 7fa758baf740 1 -- 172.21.15.22:0/3006513282 shutdown_connections 2024-03-28T18:09:02.673 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:02.672+0000 7fa758baf740 1 -- 172.21.15.22:0/3006513282 wait complete. 2024-03-28T18:09:02.673 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:02.672+0000 7fa758baf740 20 client.15987 trim_cache size 0 max 16384 2024-03-28T18:09:02.673 INFO:teuthology.orchestra.run.smithi022.stdout:[] 2024-03-28T18:09:02.674 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:02.672+0000 7fa755e00640 1 -- 172.21.15.22:0/4159534876 >> [v2:172.21.15.22:6800/1357365082,v1:172.21.15.22:6801/1357365082] conn(0x7fa7340a2ac0 msgr2=0x7fa7340a4f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:09:02.674 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:02.672+0000 7fa755e00640 1 --2- 172.21.15.22:0/4159534876 >> [v2:172.21.15.22:6800/1357365082,v1:172.21.15.22:6801/1357365082] conn(0x7fa7340a2ac0 0x7fa7340a4f80 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fa748005fd0 tx=0x7fa748009500 comp rx=0 tx=0).stop 2024-03-28T18:09:02.674 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:02.672+0000 7fa755e00640 1 -- 172.21.15.22:0/4159534876 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa750101ff0 msgr2=0x7fa750073ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:09:02.674 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:02.672+0000 7fa755e00640 1 --2- 172.21.15.22:0/4159534876 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa750101ff0 0x7fa750073ee0 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7fa73c00d950 tx=0x7fa73c00de10 comp rx=0 tx=0).stop 2024-03-28T18:09:02.674 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:02.672+0000 7fa755e00640 1 -- 172.21.15.22:0/4159534876 shutdown_connections 2024-03-28T18:09:02.674 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:02.672+0000 7fa755e00640 1 --2- 172.21.15.22:0/4159534876 >> [v2:172.21.15.22:6800/1357365082,v1:172.21.15.22:6801/1357365082] conn(0x7fa7340a2ac0 0x7fa7340a4f80 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:09:02.674 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:02.672+0000 7fa755e00640 1 --2- 172.21.15.22:0/4159534876 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa750107640 0x7fa750074420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:09:02.674 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:02.672+0000 7fa755e00640 1 --2- 172.21.15.22:0/4159534876 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa750101ff0 0x7fa750073ee0 unknown :-1 s=CLOSED pgs=567 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:09:02.674 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:02.672+0000 7fa755e00640 1 --2- 172.21.15.22:0/4159534876 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa750101630 0x7fa7500739a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:09:02.674 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:02.672+0000 7fa755e00640 1 -- 172.21.15.22:0/4159534876 >> 172.21.15.22:0/4159534876 conn(0x7fa750078aa0 msgr2=0x7fa750104080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:09:02.674 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:02.673+0000 7fa755e00640 1 -- 172.21.15.22:0/4159534876 shutdown_connections 2024-03-28T18:09:02.674 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:02.673+0000 7fa755e00640 1 -- 172.21.15.22:0/4159534876 wait complete. 2024-03-28T18:09:02.688 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:09:02 smithi022 ceph-mon[31985]: pgmap v47: 97 pgs: 97 active+clean; 1.3 MiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-03-28T18:09:02.688 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:09:02 smithi022 ceph-mon[31985]: from='client.? 172.21.15.22:0/2398483308' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-28T18:09:02.690 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-28T18:09:02.691 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-03-28T18:09:02.905 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:02.905+0000 7ff8bfe00640 1 -- 172.21.15.22:0/2104444759 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff8c0073ad0 msgr2=0x7ff8c0073eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:09:02.905 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:02.905+0000 7ff8bfe00640 1 --2- 172.21.15.22:0/2104444759 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff8c0073ad0 0x7ff8c0073eb0 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7ff8b0009900 tx=0x7ff8b002bcd0 comp rx=0 tx=0).stop 2024-03-28T18:09:02.905 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:02.905+0000 7ff8bfe00640 1 -- 172.21.15.22:0/2104444759 shutdown_connections 2024-03-28T18:09:02.905 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:02.905+0000 7ff8bfe00640 1 --2- 172.21.15.22:0/2104444759 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ff8c010dd50 0x7ff8c01101a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:09:02.906 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:02.905+0000 7ff8bfe00640 1 --2- 172.21.15.22:0/2104444759 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff8c00743f0 0x7ff8c010d810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:09:02.906 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:02.905+0000 7ff8bfe00640 1 --2- 172.21.15.22:0/2104444759 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff8c0073ad0 0x7ff8c0073eb0 unknown :-1 s=CLOSED pgs=572 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:09:02.906 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:02.905+0000 7ff8bfe00640 1 -- 172.21.15.22:0/2104444759 >> 172.21.15.22:0/2104444759 conn(0x7ff8c00fd040 msgr2=0x7ff8c00ff460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:09:02.906 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:02.906+0000 7ff8bfe00640 1 -- 172.21.15.22:0/2104444759 shutdown_connections 2024-03-28T18:09:02.906 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:02.906+0000 7ff8bfe00640 1 -- 172.21.15.22:0/2104444759 wait complete. 2024-03-28T18:09:02.907 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:02.907+0000 7ff8bfe00640 1 Processor -- start 2024-03-28T18:09:02.908 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:02.907+0000 7ff8bfe00640 1 -- start start 2024-03-28T18:09:02.908 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:02.907+0000 7ff8bfe00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff8c0073ad0 0x7ff8c019a010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:09:02.908 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:02.907+0000 7ff8bfe00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff8c00743f0 0x7ff8c019a550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:09:02.908 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:02.907+0000 7ff8bfe00640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ff8c010dd50 0x7ff8c019e8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:09:02.908 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:02.907+0000 7ff8bfe00640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8c0113b10 con 0x7ff8c0073ad0 2024-03-28T18:09:02.908 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:02.907+0000 7ff8bfe00640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8c0113990 con 0x7ff8c010dd50 2024-03-28T18:09:02.908 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:02.907+0000 7ff8bfe00640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8c0113c90 con 0x7ff8c00743f0 2024-03-28T18:09:02.908 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:02.907+0000 7ff8bea00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff8c0073ad0 0x7ff8c019a010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:09:02.908 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:02.908+0000 7ff8bea00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff8c0073ad0 0x7ff8c019a010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:54666/0 (socket says 172.21.15.22:54666) 2024-03-28T18:09:02.908 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:02.908+0000 7ff8bea00640 1 -- 172.21.15.22:0/1405814548 learned_addr learned my addr 172.21.15.22:0/1405814548 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-28T18:09:02.908 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:02.908+0000 7ff8bf400640 1 --2- 172.21.15.22:0/1405814548 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ff8c010dd50 0x7ff8c019e8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:09:02.908 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:02.908+0000 7ff8be000640 1 --2- 172.21.15.22:0/1405814548 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff8c00743f0 0x7ff8c019a550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:09:02.908 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:02.908+0000 7ff8bea00640 1 -- 172.21.15.22:0/1405814548 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff8c00743f0 msgr2=0x7ff8c019a550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:09:02.909 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:02.908+0000 7ff8bea00640 1 --2- 172.21.15.22:0/1405814548 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff8c00743f0 0x7ff8c019a550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:09:02.909 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:02.908+0000 7ff8bea00640 1 -- 172.21.15.22:0/1405814548 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ff8c010dd50 msgr2=0x7ff8c019e8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:09:02.909 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:02.908+0000 7ff8bea00640 1 --2- 172.21.15.22:0/1405814548 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ff8c010dd50 0x7ff8c019e8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:09:02.909 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:02.908+0000 7ff8bea00640 1 -- 172.21.15.22:0/1405814548 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff8c019f000 con 0x7ff8c0073ad0 2024-03-28T18:09:02.910 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:02.908+0000 7ff8bea00640 1 --2- 172.21.15.22:0/1405814548 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff8c0073ad0 0x7ff8c019a010 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7ff8b0000c00 tx=0x7ff8b00045e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:09:02.910 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:02.909+0000 7ff8b7400640 1 -- 172.21.15.22:0/1405814548 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8b0042020 con 0x7ff8c0073ad0 2024-03-28T18:09:02.910 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:02.909+0000 7ff8b7400640 1 -- 172.21.15.22:0/1405814548 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7ff8b0005330 con 0x7ff8c0073ad0 2024-03-28T18:09:02.910 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:02.909+0000 7ff8bfe00640 1 -- 172.21.15.22:0/1405814548 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff8c019f290 con 0x7ff8c0073ad0 2024-03-28T18:09:02.910 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:02.909+0000 7ff8b7400640 1 -- 172.21.15.22:0/1405814548 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8b0003ab0 con 0x7ff8c0073ad0 2024-03-28T18:09:02.910 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:02.909+0000 7ff8bfe00640 1 -- 172.21.15.22:0/1405814548 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff8c01a6b80 con 0x7ff8c0073ad0 2024-03-28T18:09:02.912 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:02.912+0000 7ff8b7400640 1 -- 172.21.15.22:0/1405814548 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 25) v1 ==== 137523+0+0 (secure 0 0 0) 0x7ff8b0045020 con 0x7ff8c0073ad0 2024-03-28T18:09:02.912 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:02.913+0000 7ff8c89e4740 1 Processor -- start 2024-03-28T18:09:02.912 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:02.913+0000 7ff8c89e4740 1 -- start start 2024-03-28T18:09:02.914 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:02.913+0000 7ff8c89e4740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e1b563b8b0 0x55e1b563bc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:09:02.914 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:02.913+0000 7ff8c89e4740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e1b563c260 0x55e1b5644810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:09:02.914 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:02.913+0000 7ff8c89e4740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55e1b5644de0 0x55e1b5647230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:09:02.914 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:02.913+0000 7ff8c89e4740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e1b559f2c0 con 0x55e1b563c260 2024-03-28T18:09:02.914 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:02.913+0000 7ff8c89e4740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e1b5547c90 con 0x55e1b5644de0 2024-03-28T18:09:02.914 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:02.913+0000 7ff8c89e4740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e1b559b360 con 0x55e1b563b8b0 2024-03-28T18:09:02.914 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:02.914+0000 7ff8bea00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e1b563c260 0x55e1b5644810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:09:02.914 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:02.914+0000 7ff8be000640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55e1b5644de0 0x55e1b5647230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:09:02.914 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:02.914+0000 7ff8bea00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e1b563c260 0x55e1b5644810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:54674/0 (socket says 172.21.15.22:54674) 2024-03-28T18:09:02.914 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:02.914+0000 7ff8bea00640 1 -- 172.21.15.22:0/2058827989 learned_addr learned my addr 172.21.15.22:0/2058827989 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-28T18:09:02.915 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:02.914+0000 7ff8bf400640 1 --2- 172.21.15.22:0/2058827989 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e1b563b8b0 0x55e1b563bc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:09:02.915 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:02.914+0000 7ff8be000640 1 -- 172.21.15.22:0/2058827989 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e1b563b8b0 msgr2=0x55e1b563bc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T18:09:02.915 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:02.914+0000 7ff8be000640 1 --2- 172.21.15.22:0/2058827989 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e1b563b8b0 0x55e1b563bc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:09:02.915 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:02.914+0000 7ff8be000640 1 -- 172.21.15.22:0/2058827989 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e1b563c260 msgr2=0x55e1b5644810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T18:09:02.915 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:02.914+0000 7ff8be000640 1 --2- 172.21.15.22:0/2058827989 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e1b563c260 0x55e1b5644810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:09:02.915 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:02.914+0000 7ff8be000640 1 -- 172.21.15.22:0/2058827989 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e1b556c080 con 0x55e1b5644de0 2024-03-28T18:09:02.915 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:02.914+0000 7ff8b7400640 1 --2- 172.21.15.22:0/1405814548 >> [v2:172.21.15.22:6800/1357365082,v1:172.21.15.22:6801/1357365082] conn(0x7ff8a40a2b10 0x7ff8a40a4fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:09:02.916 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:02.915+0000 7ff8be000640 1 --2- 172.21.15.22:0/2058827989 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55e1b5644de0 0x55e1b5647230 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7ff8ac00bbb0 tx=0x7ff8ac00f730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bbfea61b269ee47a server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:09:02.916 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:02.915+0000 7ff8b7400640 1 -- 172.21.15.22:0/1405814548 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 8411+0+0 (secure 0 0 0) 0x7ff8b00f0b80 con 0x7ff8c0073ad0 2024-03-28T18:09:02.916 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:02.915+0000 7ff8bf400640 1 --2- 172.21.15.22:0/1405814548 >> [v2:172.21.15.22:6800/1357365082,v1:172.21.15.22:6801/1357365082] conn(0x7ff8a40a2b10 0x7ff8a40a4fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:09:02.916 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:02.915+0000 7ff8abe00640 1 -- 172.21.15.22:0/2058827989 <== mon.2 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8ac015070 con 0x55e1b5644de0 2024-03-28T18:09:02.917 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:02.915+0000 7ff8abe00640 1 -- 172.21.15.22:0/2058827989 <== mon.2 v2:172.21.15.79:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7ff8ac0054f0 con 0x55e1b5644de0 2024-03-28T18:09:02.917 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:02.915+0000 7ff8abe00640 1 -- 172.21.15.22:0/2058827989 <== mon.2 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8ac011440 con 0x55e1b5644de0 2024-03-28T18:09:02.917 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:02.916+0000 7ff8c89e4740 1 -- 172.21.15.22:0/2058827989 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55e1b5644de0 msgr2=0x55e1b5647230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:09:02.917 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:02.916+0000 7ff8c89e4740 1 --2- 172.21.15.22:0/2058827989 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55e1b5644de0 0x55e1b5647230 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7ff8ac00bbb0 tx=0x7ff8ac00f730 comp rx=0 tx=0).stop 2024-03-28T18:09:02.917 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:02.916+0000 7ff8bf400640 1 --2- 172.21.15.22:0/1405814548 >> [v2:172.21.15.22:6800/1357365082,v1:172.21.15.22:6801/1357365082] conn(0x7ff8a40a2b10 0x7ff8a40a4fd0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7ff8b8005fd0 tx=0x7ff8b8009500 comp rx=0 tx=0).ready entity=mgr.15522 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:09:02.917 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:02.916+0000 7ff8c89e4740 1 -- 172.21.15.22:0/2058827989 shutdown_connections 2024-03-28T18:09:02.917 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:02.916+0000 7ff8c89e4740 1 --2- 172.21.15.22:0/2058827989 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55e1b5644de0 0x55e1b5647230 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:09:02.917 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:02.916+0000 7ff8c89e4740 1 --2- 172.21.15.22:0/2058827989 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e1b563c260 0x55e1b5644810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:09:02.917 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:02.916+0000 7ff8c89e4740 1 --2- 172.21.15.22:0/2058827989 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e1b563b8b0 0x55e1b563bc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:09:02.917 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:02.916+0000 7ff8c89e4740 1 -- 172.21.15.22:0/2058827989 >> 172.21.15.22:0/2058827989 conn(0x55e1b5635f80 msgr2=0x55e1b5636360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:09:02.917 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:02.916+0000 7ff8c89e4740 1 -- 172.21.15.22:0/2058827989 shutdown_connections 2024-03-28T18:09:02.917 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:02.916+0000 7ff8c89e4740 1 -- 172.21.15.22:0/2058827989 wait complete. 2024-03-28T18:09:02.917 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:02.917+0000 7ff8c89e4740 1 -- start start 2024-03-28T18:09:02.917 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:02.917+0000 7ff8c89e4740 1 Processor -- start 2024-03-28T18:09:02.920 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:02.917+0000 7ff8c89e4740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e1b563b8b0 0x55e1b5644710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:09:02.920 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:02.917+0000 7ff8c89e4740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e1b563c260 0x55e1b5641aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:09:02.920 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:02.917+0000 7ff8c89e4740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55e1b5644de0 0x55e1b56420b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:09:02.920 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:02.917+0000 7ff8c89e4740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e1b5647d50 con 0x55e1b563c260 2024-03-28T18:09:02.921 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:02.917+0000 7ff8c89e4740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e1b555b0d0 con 0x55e1b5644de0 2024-03-28T18:09:02.921 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:02.917+0000 7ff8c89e4740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e1b5647ed0 con 0x55e1b563b8b0 2024-03-28T18:09:02.921 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:02.918+0000 7ff8c89e4740 10 client.0 _ll_register_callbacks cb 0x55e1b555fe80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-28T18:09:02.921 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:02.918+0000 7ff8be000640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e1b563c260 0x55e1b5641aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:09:02.921 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:02.918+0000 7ff8be000640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e1b563c260 0x55e1b5641aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:54688/0 (socket says 172.21.15.22:54688) 2024-03-28T18:09:02.921 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:02.918+0000 7ff8be000640 1 -- 172.21.15.22:0/1467289840 learned_addr learned my addr 172.21.15.22:0/1467289840 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-28T18:09:02.921 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:02.918+0000 7ff8bf400640 1 --2- 172.21.15.22:0/1467289840 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55e1b5644de0 0x55e1b56420b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:09:02.921 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:02.918+0000 7ff8bea00640 1 --2- 172.21.15.22:0/1467289840 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e1b563b8b0 0x55e1b5644710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:09:02.921 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:02.918+0000 7ff8be000640 1 -- 172.21.15.22:0/1467289840 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e1b563b8b0 msgr2=0x55e1b5644710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T18:09:02.921 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:02.918+0000 7ff8be000640 1 --2- 172.21.15.22:0/1467289840 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e1b563b8b0 0x55e1b5644710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:09:02.921 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:02.918+0000 7ff8be000640 1 -- 172.21.15.22:0/1467289840 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55e1b5644de0 msgr2=0x55e1b56420b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T18:09:02.921 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:02.918+0000 7ff8be000640 1 --2- 172.21.15.22:0/1467289840 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55e1b5644de0 0x55e1b56420b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:09:02.921 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:02.918+0000 7ff8be000640 1 -- 172.21.15.22:0/1467289840 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55e1b52b33f0 con 0x55e1b563c260 2024-03-28T18:09:02.921 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:02.918+0000 7ff8bf400640 1 --2- 172.21.15.22:0/1467289840 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55e1b5644de0 0x55e1b56420b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-28T18:09:02.922 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:02.919+0000 7ff8be000640 1 --2- 172.21.15.22:0/1467289840 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e1b563c260 0x55e1b5641aa0 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7ff8ac005fa0 tx=0x7ff8ac005710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=784f6132e5b0f565 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:09:02.922 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:02.919+0000 7ff8a8c00640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-03-28T18:09:02.922 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:02.919+0000 7ff8a8c00640 1 -- 172.21.15.22:0/1467289840 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8ac028060 con 0x55e1b563c260 2024-03-28T18:09:02.922 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:02.919+0000 7ff8a8c00640 1 -- 172.21.15.22:0/1467289840 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 8411+0+0 (secure 0 0 0) 0x7ff8ac00bca0 con 0x55e1b563c260 2024-03-28T18:09:02.922 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:02.919+0000 7ff8c89e4740 1 -- 172.21.15.22:0/1467289840 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7ff8ac0054f0 con 0x55e1b563c260 2024-03-28T18:09:02.922 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:02.919+0000 7ff8a8c00640 1 -- 172.21.15.22:0/1467289840 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff8ac015070 con 0x55e1b563c260 2024-03-28T18:09:02.922 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:02.919+0000 7ff8c89e4740 10 client.15999 fetch_fsmap learned FSMap version 27 2024-03-28T18:09:02.922 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:02.919+0000 7ff8c89e4740 1 -- 172.21.15.22:0/1467289840 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55e1b537d7d0 con 0x55e1b563c260 2024-03-28T18:09:02.922 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:02.920+0000 7ff8a8c00640 1 -- 172.21.15.22:0/1467289840 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7ff8ac0054f0 con 0x55e1b563c260 2024-03-28T18:09:02.922 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:02.920+0000 7ff8a8c00640 1 -- 172.21.15.22:0/1467289840 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 27) v1 ==== 7934+0+0 (secure 0 0 0) 0x7ff8ac02b020 con 0x55e1b563c260 2024-03-28T18:09:02.922 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:02.920+0000 7ff8c89e4740 10 client.15999 fetch_fsmap finished waiting for FSMap version 27 2024-03-28T18:09:02.922 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:02.920+0000 7ff8c89e4740 10 client.15999 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-03-28T18:09:02.922 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:02.920+0000 7ff8c89e4740 20 client.15999 populate_metadata read hostname 'smithi022' 2024-03-28T18:09:02.922 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:02.920+0000 7ff8c89e4740 1 --2- 172.21.15.22:0/1467289840 >> [v2:172.21.15.79:6836/1787903366,v1:172.21.15.79:6837/1787903366] conn(0x55e1b5654c00 0x55e1b5656ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:09:02.923 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:02.920+0000 7ff8c89e4740 4 client.15999 mds_command: new command op to 34511 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-28T18:09:02.923 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:02.920+0000 7ff8c89e4740 1 -- 172.21.15.22:0/1467289840 --> [v2:172.21.15.79:6836/1787903366,v1:172.21.15.79:6837/1787903366] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55e1b55ac800 con 0x55e1b5654c00 2024-03-28T18:09:02.923 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:02.921+0000 7ff8bea00640 1 --2- 172.21.15.22:0/1467289840 >> [v2:172.21.15.79:6836/1787903366,v1:172.21.15.79:6837/1787903366] conn(0x55e1b5654c00 0x55e1b5656ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:09:02.924 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:02.921+0000 7ff8bea00640 1 --2- 172.21.15.22:0/1467289840 >> [v2:172.21.15.79:6836/1787903366,v1:172.21.15.79:6837/1787903366] conn(0x55e1b5654c00 0x55e1b5656ff0 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=c42c1d8fc7e453b1 server_cookie=7ae3f2ce2b2938a in_seq=0 out_seq=0 2024-03-28T18:09:02.924 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:02.921+0000 7ff8a8c00640 10 client.15999 ms_handle_connect on v2:172.21.15.79:6836/1787903366 2024-03-28T18:09:02.924 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:02.923+0000 7ff8a8c00640 1 -- 172.21.15.22:0/1467289840 <== mds.1 v2:172.21.15.79:6836/1787903366 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x55e1b55ac800 con 0x55e1b5654c00 2024-03-28T18:09:02.924 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:02.923+0000 7ff8a8c00640 10 client.15999 handle_command_reply: tid=0 2024-03-28T18:09:02.924 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:02.923+0000 7ff8c89e4740 1 client.15999 shutdown 2024-03-28T18:09:02.924 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:02.923+0000 7ff8c89e4740 1 -- 172.21.15.22:0/1467289840 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e1b563c260 msgr2=0x55e1b5641aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:09:02.924 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:02.923+0000 7ff8c89e4740 1 --2- 172.21.15.22:0/1467289840 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e1b563c260 0x55e1b5641aa0 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7ff8ac005fa0 tx=0x7ff8ac005710 comp rx=0 tx=0).stop 2024-03-28T18:09:02.924 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:02.923+0000 7ff8c89e4740 1 -- 172.21.15.22:0/1467289840 shutdown_connections 2024-03-28T18:09:02.924 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:02.923+0000 7ff8c89e4740 1 --2- 172.21.15.22:0/1467289840 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55e1b5644de0 0x55e1b56420b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:09:02.924 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:02.923+0000 7ff8c89e4740 1 --2- 172.21.15.22:0/1467289840 >> [v2:172.21.15.79:6836/1787903366,v1:172.21.15.79:6837/1787903366] conn(0x55e1b5654c00 0x55e1b5656ff0 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:09:02.925 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:02.923+0000 7ff8c89e4740 1 --2- 172.21.15.22:0/1467289840 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e1b563c260 0x55e1b5641aa0 unknown :-1 s=CLOSED pgs=574 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:09:02.925 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:02.923+0000 7ff8c89e4740 1 --2- 172.21.15.22:0/1467289840 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e1b563b8b0 0x55e1b5644710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:09:02.925 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:02.923+0000 7ff8c89e4740 1 -- 172.21.15.22:0/1467289840 >> 172.21.15.22:0/1467289840 conn(0x55e1b5635f80 msgr2=0x55e1b5637cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:09:02.925 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:02.923+0000 7ff8a8c00640 0 client.15999 ms_handle_reset on v2:172.21.15.79:6836/1787903366 2024-03-28T18:09:02.925 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:02.923+0000 7ff8c89e4740 1 -- 172.21.15.22:0/1467289840 shutdown_connections 2024-03-28T18:09:02.925 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:02.923+0000 7ff8c89e4740 1 -- 172.21.15.22:0/1467289840 wait complete. 2024-03-28T18:09:02.925 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:02.923+0000 7ff8c89e4740 20 client.15999 trim_cache size 0 max 16384 2024-03-28T18:09:02.925 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:09:02 smithi134 ceph-mon[35037]: pgmap v47: 97 pgs: 97 active+clean; 1.3 MiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-03-28T18:09:02.925 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:09:02 smithi134 ceph-mon[35037]: from='client.? 172.21.15.22:0/2398483308' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-28T18:09:02.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:09:02 smithi079 ceph-mon[35312]: pgmap v47: 97 pgs: 97 active+clean; 1.3 MiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-03-28T18:09:02.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:09:02 smithi079 ceph-mon[35312]: from='client.? 172.21.15.22:0/2398483308' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-28T18:09:02.940 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:02.941+0000 7ff8c89e4740 1 Processor -- start 2024-03-28T18:09:02.941 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:02.941+0000 7ff8c89e4740 1 -- start start 2024-03-28T18:09:02.942 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:02.941+0000 7ff8c89e4740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e1b563b8b0 0x55e1b56441d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:09:02.942 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:02.941+0000 7ff8c89e4740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55e1b563c260 0x55e1b5644710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:09:02.942 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:02.941+0000 7ff8c89e4740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e1b5644de0 0x55e1b563ed00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:09:02.942 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:02.941+0000 7ff8c89e4740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8c01aef50 con 0x55e1b563b8b0 2024-03-28T18:09:02.942 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:02.941+0000 7ff8c89e4740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8c01b08d0 con 0x55e1b563c260 2024-03-28T18:09:02.942 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:02.941+0000 7ff8c89e4740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8c01af5b0 con 0x55e1b5644de0 2024-03-28T18:09:02.942 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:02.942+0000 7ff8bea00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e1b563b8b0 0x55e1b56441d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:09:02.942 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:02.942+0000 7ff8bea00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e1b563b8b0 0x55e1b56441d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:54698/0 (socket says 172.21.15.22:54698) 2024-03-28T18:09:02.942 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:02.942+0000 7ff8bea00640 1 -- 172.21.15.22:0/1563849666 learned_addr learned my addr 172.21.15.22:0/1563849666 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-28T18:09:02.943 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:02.942+0000 7ff8be000640 1 --2- 172.21.15.22:0/1563849666 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55e1b563c260 0x55e1b5644710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:09:02.943 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:02.942+0000 7ff8bf400640 1 --2- 172.21.15.22:0/1563849666 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e1b5644de0 0x55e1b563ed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:09:02.943 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:02.942+0000 7ff8bea00640 1 -- 172.21.15.22:0/1563849666 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e1b5644de0 msgr2=0x55e1b563ed00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T18:09:02.943 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:02.942+0000 7ff8bea00640 1 --2- 172.21.15.22:0/1563849666 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e1b5644de0 0x55e1b563ed00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:09:02.943 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:02.942+0000 7ff8bea00640 1 -- 172.21.15.22:0/1563849666 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55e1b563c260 msgr2=0x55e1b5644710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T18:09:02.943 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:02.942+0000 7ff8bea00640 1 --2- 172.21.15.22:0/1563849666 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55e1b563c260 0x55e1b5644710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:09:02.943 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:02.942+0000 7ff8bea00640 1 -- 172.21.15.22:0/1563849666 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e1b5642420 con 0x55e1b563b8b0 2024-03-28T18:09:02.943 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:02.942+0000 7ff8be000640 1 --2- 172.21.15.22:0/1563849666 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55e1b563c260 0x55e1b5644710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-28T18:09:02.944 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:02.943+0000 7ff8bea00640 1 --2- 172.21.15.22:0/1563849666 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e1b563b8b0 0x55e1b56441d0 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7ff8b0005ab0 tx=0x7ff8b00f4690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ee16b8d47329f148 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:09:02.944 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:02.943+0000 7ff8ab400640 1 -- 172.21.15.22:0/1563849666 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8b00bcee0 con 0x55e1b563b8b0 2024-03-28T18:09:02.944 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:02.943+0000 7ff8ab400640 1 -- 172.21.15.22:0/1563849666 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7ff8b00dd0b0 con 0x55e1b563b8b0 2024-03-28T18:09:02.944 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:02.943+0000 7ff8ab400640 1 -- 172.21.15.22:0/1563849666 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8b00bca70 con 0x55e1b563b8b0 2024-03-28T18:09:02.945 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:02.944+0000 7ff8c89e4740 1 -- 172.21.15.22:0/1563849666 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e1b563b8b0 msgr2=0x55e1b56441d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:09:02.945 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:02.944+0000 7ff8c89e4740 1 --2- 172.21.15.22:0/1563849666 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e1b563b8b0 0x55e1b56441d0 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7ff8b0005ab0 tx=0x7ff8b00f4690 comp rx=0 tx=0).stop 2024-03-28T18:09:02.945 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:02.944+0000 7ff8c89e4740 1 -- 172.21.15.22:0/1563849666 shutdown_connections 2024-03-28T18:09:02.945 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:02.944+0000 7ff8c89e4740 1 --2- 172.21.15.22:0/1563849666 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e1b5644de0 0x55e1b563ed00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:09:02.945 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:02.944+0000 7ff8c89e4740 1 --2- 172.21.15.22:0/1563849666 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55e1b563c260 0x55e1b5644710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:09:02.945 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:02.944+0000 7ff8c89e4740 1 --2- 172.21.15.22:0/1563849666 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e1b563b8b0 0x55e1b56441d0 unknown :-1 s=CLOSED pgs=575 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:09:02.945 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:02.944+0000 7ff8c89e4740 1 -- 172.21.15.22:0/1563849666 >> 172.21.15.22:0/1563849666 conn(0x55e1b5635f80 msgr2=0x55e1b5635160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:09:02.945 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:02.944+0000 7ff8c89e4740 1 -- 172.21.15.22:0/1563849666 shutdown_connections 2024-03-28T18:09:02.945 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:02.944+0000 7ff8c89e4740 1 -- 172.21.15.22:0/1563849666 wait complete. 2024-03-28T18:09:02.945 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:02.944+0000 7ff8c89e4740 1 -- start start 2024-03-28T18:09:02.945 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:02.945+0000 7ff8c89e4740 1 Processor -- start 2024-03-28T18:09:02.948 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:02.945+0000 7ff8c89e4740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e1b563b8b0 0x55e1b564c7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:09:02.948 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:02.945+0000 7ff8c89e4740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e1b563c260 0x55e1b564cd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:09:02.948 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:02.945+0000 7ff8c89e4740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55e1b5644de0 0x55e1b564ad00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:09:02.948 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:02.945+0000 7ff8c89e4740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8c01b0270 con 0x55e1b563c260 2024-03-28T18:09:02.948 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:02.945+0000 7ff8c89e4740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8c01afc10 con 0x55e1b5644de0 2024-03-28T18:09:02.948 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:02.945+0000 7ff8c89e4740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8c01ad110 con 0x55e1b563b8b0 2024-03-28T18:09:02.948 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:02.945+0000 7ff8c89e4740 10 client.0 _ll_register_callbacks cb 0x7ff8b00f4560 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-28T18:09:02.948 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:02.946+0000 7ff8be000640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e1b563c260 0x55e1b564cd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:09:02.948 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:02.946+0000 7ff8be000640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e1b563c260 0x55e1b564cd10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:54706/0 (socket says 172.21.15.22:54706) 2024-03-28T18:09:02.948 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:02.946+0000 7ff8be000640 1 -- 172.21.15.22:0/877129544 learned_addr learned my addr 172.21.15.22:0/877129544 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-28T18:09:02.948 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:02.946+0000 7ff8bf400640 1 --2- 172.21.15.22:0/877129544 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55e1b5644de0 0x55e1b564ad00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:09:02.948 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:02.946+0000 7ff8bea00640 1 --2- 172.21.15.22:0/877129544 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e1b563b8b0 0x55e1b564c7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:09:02.949 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:02.946+0000 7ff8be000640 1 -- 172.21.15.22:0/877129544 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e1b563b8b0 msgr2=0x55e1b564c7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T18:09:02.949 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:02.946+0000 7ff8be000640 1 --2- 172.21.15.22:0/877129544 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e1b563b8b0 0x55e1b564c7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:09:02.949 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:02.946+0000 7ff8be000640 1 -- 172.21.15.22:0/877129544 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55e1b5644de0 msgr2=0x55e1b564ad00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T18:09:02.949 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:02.946+0000 7ff8be000640 1 --2- 172.21.15.22:0/877129544 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55e1b5644de0 0x55e1b564ad00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:09:02.949 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:02.946+0000 7ff8be000640 1 -- 172.21.15.22:0/877129544 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55e1b563c000 con 0x55e1b563c260 2024-03-28T18:09:02.949 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:02.946+0000 7ff8be000640 1 --2- 172.21.15.22:0/877129544 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e1b563c260 0x55e1b564cd10 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7ff8ac00f770 tx=0x7ff8ac00fd60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d2644f6525a30a7e server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:09:02.949 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:02.946+0000 7ff8a8c00640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-03-28T18:09:02.949 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:02.947+0000 7ff8a8c00640 1 -- 172.21.15.22:0/877129544 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8ac028040 con 0x55e1b563c260 2024-03-28T18:09:02.949 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:02.947+0000 7ff8a8c00640 1 -- 172.21.15.22:0/877129544 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 8411+0+0 (secure 0 0 0) 0x7ff8ac011440 con 0x55e1b563c260 2024-03-28T18:09:02.949 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:02.947+0000 7ff8c89e4740 1 -- 172.21.15.22:0/877129544 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55e1b55a6b50 con 0x55e1b563c260 2024-03-28T18:09:02.949 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:02.947+0000 7ff8a8c00640 1 -- 172.21.15.22:0/877129544 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff8ac011640 con 0x55e1b563c260 2024-03-28T18:09:02.949 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:02.947+0000 7ff8c89e4740 10 client.16005 fetch_fsmap learned FSMap version 27 2024-03-28T18:09:02.949 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:02.947+0000 7ff8c89e4740 1 -- 172.21.15.22:0/877129544 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55e1b56b92d0 con 0x55e1b563c260 2024-03-28T18:09:02.949 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:02.947+0000 7ff8a8c00640 1 -- 172.21.15.22:0/877129544 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x55e1b55a6b50 con 0x55e1b563c260 2024-03-28T18:09:02.949 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:02.948+0000 7ff8a8c00640 1 -- 172.21.15.22:0/877129544 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 27) v1 ==== 7934+0+0 (secure 0 0 0) 0x7ff8ac02d4d0 con 0x55e1b563c260 2024-03-28T18:09:02.950 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:02.948+0000 7ff8c89e4740 10 client.16005 fetch_fsmap finished waiting for FSMap version 27 2024-03-28T18:09:02.950 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:02.948+0000 7ff8c89e4740 10 client.16005 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-03-28T18:09:02.950 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:02.948+0000 7ff8c89e4740 20 client.16005 populate_metadata read hostname 'smithi022' 2024-03-28T18:09:02.950 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:02.948+0000 7ff8c89e4740 1 --2- 172.21.15.22:0/877129544 >> [v2:172.21.15.79:6836/1787903366,v1:172.21.15.79:6837/1787903366] conn(0x55e1b56b98e0 0x55e1b56bbd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:09:02.950 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:02.948+0000 7ff8c89e4740 4 client.16005 mds_command: new command op to 34511 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-28T18:09:02.950 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:02.948+0000 7ff8c89e4740 1 -- 172.21.15.22:0/877129544 --> [v2:172.21.15.79:6836/1787903366,v1:172.21.15.79:6837/1787903366] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55e1b5524500 con 0x55e1b56b98e0 2024-03-28T18:09:02.950 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:02.948+0000 7ff8bea00640 1 --2- 172.21.15.22:0/877129544 >> [v2:172.21.15.79:6836/1787903366,v1:172.21.15.79:6837/1787903366] conn(0x55e1b56b98e0 0x55e1b56bbd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:09:02.951 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:02.949+0000 7ff8bea00640 1 --2- 172.21.15.22:0/877129544 >> [v2:172.21.15.79:6836/1787903366,v1:172.21.15.79:6837/1787903366] conn(0x55e1b56b98e0 0x55e1b56bbd00 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=6d31d96a12abcee4 server_cookie=947a546c849250b5 in_seq=0 out_seq=0 2024-03-28T18:09:02.951 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:02.949+0000 7ff8a8c00640 10 client.16005 ms_handle_connect on v2:172.21.15.79:6836/1787903366 2024-03-28T18:09:02.951 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:02.949+0000 7ff8a8c00640 1 -- 172.21.15.22:0/877129544 <== mds.1 v2:172.21.15.79:6836/1787903366 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55e1b5524500 con 0x55e1b56b98e0 2024-03-28T18:09:02.951 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:02.949+0000 7ff8a8c00640 10 client.16005 handle_command_reply: tid=0 2024-03-28T18:09:02.951 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:02.949+0000 7ff8c89e4740 1 client.16005 shutdown 2024-03-28T18:09:02.951 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:02.949+0000 7ff8c89e4740 1 -- 172.21.15.22:0/877129544 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e1b563c260 msgr2=0x55e1b564cd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:09:02.951 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:02.949+0000 7ff8c89e4740 1 --2- 172.21.15.22:0/877129544 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e1b563c260 0x55e1b564cd10 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7ff8ac00f770 tx=0x7ff8ac00fd60 comp rx=0 tx=0).stop 2024-03-28T18:09:02.951 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:02.949+0000 7ff8c89e4740 1 -- 172.21.15.22:0/877129544 shutdown_connections 2024-03-28T18:09:02.952 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:02.949+0000 7ff8c89e4740 1 --2- 172.21.15.22:0/877129544 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55e1b5644de0 0x55e1b564ad00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:09:02.952 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:02.949+0000 7ff8c89e4740 1 --2- 172.21.15.22:0/877129544 >> [v2:172.21.15.79:6836/1787903366,v1:172.21.15.79:6837/1787903366] conn(0x55e1b56b98e0 0x55e1b56bbd00 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:09:02.952 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:02.949+0000 7ff8c89e4740 1 --2- 172.21.15.22:0/877129544 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e1b563c260 0x55e1b564cd10 unknown :-1 s=CLOSED pgs=576 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:09:02.952 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:02.949+0000 7ff8c89e4740 1 --2- 172.21.15.22:0/877129544 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e1b563b8b0 0x55e1b564c7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:09:02.952 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:02.949+0000 7ff8a8c00640 0 client.16005 ms_handle_reset on v2:172.21.15.79:6836/1787903366 2024-03-28T18:09:02.952 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:02.949+0000 7ff8c89e4740 1 -- 172.21.15.22:0/877129544 >> 172.21.15.22:0/877129544 conn(0x55e1b5635f80 msgr2=0x55e1b563c640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:09:02.952 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:02.950+0000 7ff8c89e4740 1 -- 172.21.15.22:0/877129544 shutdown_connections 2024-03-28T18:09:02.952 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:02.950+0000 7ff8c89e4740 1 -- 172.21.15.22:0/877129544 wait complete. 2024-03-28T18:09:02.952 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:02.950+0000 7ff8c89e4740 20 client.16005 trim_cache size 0 max 16384 2024-03-28T18:09:02.952 INFO:teuthology.orchestra.run.smithi022.stdout:[] 2024-03-28T18:09:02.953 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:02.950+0000 7ff8bfe00640 1 -- 172.21.15.22:0/1405814548 >> [v2:172.21.15.22:6800/1357365082,v1:172.21.15.22:6801/1357365082] conn(0x7ff8a40a2b10 msgr2=0x7ff8a40a4fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:09:02.953 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:02.950+0000 7ff8bfe00640 1 --2- 172.21.15.22:0/1405814548 >> [v2:172.21.15.22:6800/1357365082,v1:172.21.15.22:6801/1357365082] conn(0x7ff8a40a2b10 0x7ff8a40a4fd0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7ff8b8005fd0 tx=0x7ff8b8009500 comp rx=0 tx=0).stop 2024-03-28T18:09:02.953 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:02.950+0000 7ff8bfe00640 1 -- 172.21.15.22:0/1405814548 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff8c0073ad0 msgr2=0x7ff8c019a010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:09:02.953 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:02.950+0000 7ff8bfe00640 1 --2- 172.21.15.22:0/1405814548 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff8c0073ad0 0x7ff8c019a010 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7ff8b0000c00 tx=0x7ff8b00045e0 comp rx=0 tx=0).stop 2024-03-28T18:09:02.953 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:02.951+0000 7ff8bfe00640 1 -- 172.21.15.22:0/1405814548 shutdown_connections 2024-03-28T18:09:02.953 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:02.951+0000 7ff8bfe00640 1 --2- 172.21.15.22:0/1405814548 >> [v2:172.21.15.22:6800/1357365082,v1:172.21.15.22:6801/1357365082] conn(0x7ff8a40a2b10 0x7ff8a40a4fd0 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:09:02.953 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:02.951+0000 7ff8bfe00640 1 --2- 172.21.15.22:0/1405814548 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ff8c010dd50 0x7ff8c019e8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:09:02.953 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:02.951+0000 7ff8bfe00640 1 --2- 172.21.15.22:0/1405814548 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff8c00743f0 0x7ff8c019a550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:09:02.953 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:02.951+0000 7ff8bfe00640 1 --2- 172.21.15.22:0/1405814548 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff8c0073ad0 0x7ff8c019a010 unknown :-1 s=CLOSED pgs=573 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:09:02.954 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:02.951+0000 7ff8bfe00640 1 -- 172.21.15.22:0/1405814548 >> 172.21.15.22:0/1405814548 conn(0x7ff8c00fd040 msgr2=0x7ff8c00fe760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:09:02.954 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:02.951+0000 7ff8bfe00640 1 -- 172.21.15.22:0/1405814548 shutdown_connections 2024-03-28T18:09:02.954 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:02.951+0000 7ff8bfe00640 1 -- 172.21.15.22:0/1405814548 wait complete. 2024-03-28T18:09:02.968 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-28T18:09:02.968 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-03-28T18:09:03.188 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:03.188+0000 7f7e6a000640 1 -- 172.21.15.22:0/829622277 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7e64103840 msgr2=0x7f7e64103cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:09:03.188 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:03.188+0000 7f7e6a000640 1 --2- 172.21.15.22:0/829622277 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7e64103840 0x7f7e64103cc0 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7f7e50009930 tx=0x7f7e5002bc20 comp rx=0 tx=0).stop 2024-03-28T18:09:03.188 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:03.188+0000 7f7e6a000640 1 -- 172.21.15.22:0/829622277 shutdown_connections 2024-03-28T18:09:03.188 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:03.188+0000 7f7e6a000640 1 --2- 172.21.15.22:0/829622277 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f7e64104200 0x7f7e64110710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:09:03.188 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:03.188+0000 7f7e6a000640 1 --2- 172.21.15.22:0/829622277 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7e64103840 0x7f7e64103cc0 unknown :-1 s=CLOSED pgs=577 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:09:03.189 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:03.188+0000 7f7e6a000640 1 --2- 172.21.15.22:0/829622277 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7e64109850 0x7f7e64109c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:09:03.189 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:03.188+0000 7f7e6a000640 1 -- 172.21.15.22:0/829622277 >> 172.21.15.22:0/829622277 conn(0x7f7e640ff350 msgr2=0x7f7e64101770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:09:03.189 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:03.188+0000 7f7e6a000640 1 -- 172.21.15.22:0/829622277 shutdown_connections 2024-03-28T18:09:03.189 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:03.188+0000 7f7e6a000640 1 -- 172.21.15.22:0/829622277 wait complete. 2024-03-28T18:09:03.190 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:03.189+0000 7f7e6a000640 1 Processor -- start 2024-03-28T18:09:03.190 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:03.189+0000 7f7e6a000640 1 -- start start 2024-03-28T18:09:03.191 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:03.190+0000 7f7e6a000640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7e64103840 0x7f7e6419a1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:09:03.191 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:03.190+0000 7f7e6a000640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f7e64104200 0x7f7e6419a700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:09:03.191 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:03.190+0000 7f7e6a000640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7e64109850 0x7f7e6419ea80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:09:03.191 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:03.190+0000 7f7e6a000640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e64113c80 con 0x7f7e64109850 2024-03-28T18:09:03.191 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:03.190+0000 7f7e6a000640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e64113b00 con 0x7f7e64104200 2024-03-28T18:09:03.191 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:03.190+0000 7f7e6a000640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e64113e00 con 0x7f7e64103840 2024-03-28T18:09:03.191 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:03.190+0000 7f7e63e00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7e64109850 0x7f7e6419ea80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:09:03.191 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:03.190+0000 7f7e63e00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7e64109850 0x7f7e6419ea80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:54714/0 (socket says 172.21.15.22:54714) 2024-03-28T18:09:03.191 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:03.190+0000 7f7e63e00640 1 -- 172.21.15.22:0/1290860068 learned_addr learned my addr 172.21.15.22:0/1290860068 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-28T18:09:03.191 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:03.190+0000 7f7e62a00640 1 --2- 172.21.15.22:0/1290860068 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f7e64104200 0x7f7e6419a700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:09:03.191 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:03.191+0000 7f7e63400640 1 --2- 172.21.15.22:0/1290860068 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7e64103840 0x7f7e6419a1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:09:03.191 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:03.191+0000 7f7e63e00640 1 -- 172.21.15.22:0/1290860068 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7e64103840 msgr2=0x7f7e6419a1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:09:03.191 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:03.191+0000 7f7e63e00640 1 --2- 172.21.15.22:0/1290860068 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7e64103840 0x7f7e6419a1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:09:03.191 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:03.191+0000 7f7e63e00640 1 -- 172.21.15.22:0/1290860068 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f7e64104200 msgr2=0x7f7e6419a700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:09:03.192 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:03.191+0000 7f7e63e00640 1 --2- 172.21.15.22:0/1290860068 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f7e64104200 0x7f7e6419a700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:09:03.192 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:03.191+0000 7f7e63e00640 1 -- 172.21.15.22:0/1290860068 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7e6419f1b0 con 0x7f7e64109850 2024-03-28T18:09:03.192 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:03.191+0000 7f7e63e00640 1 --2- 172.21.15.22:0/1290860068 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7e64109850 0x7f7e6419ea80 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7f7e5c00eb10 tx=0x7f7e5c00c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:09:03.193 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:03.192+0000 7f7e5be00640 1 -- 172.21.15.22:0/1290860068 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e5c004b20 con 0x7f7e64109850 2024-03-28T18:09:03.193 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:03.192+0000 7f7e5be00640 1 -- 172.21.15.22:0/1290860068 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f7e5c004cc0 con 0x7f7e64109850 2024-03-28T18:09:03.193 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:03.192+0000 7f7e5be00640 1 -- 172.21.15.22:0/1290860068 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e5c010700 con 0x7f7e64109850 2024-03-28T18:09:03.193 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:03.192+0000 7f7e6a000640 1 -- 172.21.15.22:0/1290860068 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7e6419f4a0 con 0x7f7e64109850 2024-03-28T18:09:03.193 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:03.192+0000 7f7e6a000640 1 -- 172.21.15.22:0/1290860068 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7e641a6d30 con 0x7f7e64109850 2024-03-28T18:09:03.195 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:03.195+0000 7f7e5be00640 1 -- 172.21.15.22:0/1290860068 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 25) v1 ==== 137523+0+0 (secure 0 0 0) 0x7f7e5c004e70 con 0x7f7e64109850 2024-03-28T18:09:03.195 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:03.196+0000 7f7e6cc4e740 1 Processor -- start 2024-03-28T18:09:03.196 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:03.196+0000 7f7e6cc4e740 1 -- start start 2024-03-28T18:09:03.196 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:03.197+0000 7f7e6cc4e740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55bc0ae416d0 0x55bc0ae41ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:09:03.196 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:03.197+0000 7f7e6cc4e740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55bc0ae42080 0x55bc0ae4a630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:09:03.196 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:03.197+0000 7f7e6cc4e740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55bc0ae4ac00 0x55bc0ae4d050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:09:03.197 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:03.197+0000 7f7e6cc4e740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bc0ada52c0 con 0x55bc0ae4ac00 2024-03-28T18:09:03.197 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:03.197+0000 7f7e6cc4e740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bc0ad4dc90 con 0x55bc0ae416d0 2024-03-28T18:09:03.197 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:03.197+0000 7f7e6cc4e740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bc0ada1360 con 0x55bc0ae42080 2024-03-28T18:09:03.198 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:03.197+0000 7f7e62a00640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55bc0ae416d0 0x55bc0ae41ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:09:03.198 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:03.197+0000 7f7e62a00640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55bc0ae416d0 0x55bc0ae41ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.22:35938/0 (socket says 172.21.15.22:35938) 2024-03-28T18:09:03.198 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:03.197+0000 7f7e62a00640 1 -- 172.21.15.22:0/1738191869 learned_addr learned my addr 172.21.15.22:0/1738191869 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-28T18:09:03.198 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:03.197+0000 7f7e5be00640 1 --2- 172.21.15.22:0/1290860068 >> [v2:172.21.15.22:6800/1357365082,v1:172.21.15.22:6801/1357365082] conn(0x7f7e480a2a90 0x7f7e480a4f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:09:03.198 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:03.197+0000 7f7e62a00640 1 --2- 172.21.15.22:0/1738191869 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55bc0ae4ac00 0x55bc0ae4d050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:09:03.198 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:03.198+0000 7f7e63400640 1 --2- 172.21.15.22:0/1738191869 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55bc0ae42080 0x55bc0ae4a630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:09:03.198 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:03.198+0000 7f7e5be00640 1 -- 172.21.15.22:0/1290860068 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 8411+0+0 (secure 0 0 0) 0x7f7e5c0cf750 con 0x7f7e64109850 2024-03-28T18:09:03.198 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:03.198+0000 7f7e62a00640 1 -- 172.21.15.22:0/1738191869 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55bc0ae42080 msgr2=0x55bc0ae4a630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T18:09:03.198 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:03.198+0000 7f7e62a00640 1 --2- 172.21.15.22:0/1738191869 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55bc0ae42080 0x55bc0ae4a630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:09:03.199 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:03.198+0000 7f7e62a00640 1 -- 172.21.15.22:0/1738191869 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55bc0ae4ac00 msgr2=0x55bc0ae4d050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T18:09:03.199 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:03.198+0000 7f7e62a00640 1 --2- 172.21.15.22:0/1738191869 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55bc0ae4ac00 0x55bc0ae4d050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:09:03.199 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:03.198+0000 7f7e62a00640 1 -- 172.21.15.22:0/1738191869 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55bc0ad72080 con 0x55bc0ae416d0 2024-03-28T18:09:03.199 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:03.199+0000 7f7e63e00640 1 --2- 172.21.15.22:0/1290860068 >> [v2:172.21.15.22:6800/1357365082,v1:172.21.15.22:6801/1357365082] conn(0x7f7e480a2a90 0x7f7e480a4f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:09:03.199 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:03.199+0000 7f7e62a00640 1 --2- 172.21.15.22:0/1738191869 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55bc0ae416d0 0x55bc0ae41ab0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f7e50009900 tx=0x7f7e50005fd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a18e60ce8eb45a19 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:09:03.200 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:03.199+0000 7f7e4fe00640 1 -- 172.21.15.22:0/1738191869 <== mon.2 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e5004a070 con 0x55bc0ae416d0 2024-03-28T18:09:03.200 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:03.199+0000 7f7e63e00640 1 --2- 172.21.15.22:0/1290860068 >> [v2:172.21.15.22:6800/1357365082,v1:172.21.15.22:6801/1357365082] conn(0x7f7e480a2a90 0x7f7e480a4f50 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f7e5c0ccd90 tx=0x7f7e5c00c950 comp rx=0 tx=0).ready entity=mgr.15522 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:09:03.200 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:03.200+0000 7f7e4fe00640 1 -- 172.21.15.22:0/1738191869 <== mon.2 v2:172.21.15.79:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f7e5002ca90 con 0x55bc0ae416d0 2024-03-28T18:09:03.200 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:03.200+0000 7f7e4fe00640 1 -- 172.21.15.22:0/1738191869 <== mon.2 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e50003c80 con 0x55bc0ae416d0 2024-03-28T18:09:03.201 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:03.200+0000 7f7e6cc4e740 1 -- 172.21.15.22:0/1738191869 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55bc0ae416d0 msgr2=0x55bc0ae41ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:09:03.201 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:03.200+0000 7f7e6cc4e740 1 --2- 172.21.15.22:0/1738191869 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55bc0ae416d0 0x55bc0ae41ab0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f7e50009900 tx=0x7f7e50005fd0 comp rx=0 tx=0).stop 2024-03-28T18:09:03.201 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:03.201+0000 7f7e6cc4e740 1 -- 172.21.15.22:0/1738191869 shutdown_connections 2024-03-28T18:09:03.201 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:03.201+0000 7f7e6cc4e740 1 --2- 172.21.15.22:0/1738191869 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55bc0ae4ac00 0x55bc0ae4d050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:09:03.201 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:03.201+0000 7f7e6cc4e740 1 --2- 172.21.15.22:0/1738191869 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55bc0ae42080 0x55bc0ae4a630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:09:03.201 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:03.201+0000 7f7e6cc4e740 1 --2- 172.21.15.22:0/1738191869 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55bc0ae416d0 0x55bc0ae41ab0 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:09:03.201 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:03.201+0000 7f7e6cc4e740 1 -- 172.21.15.22:0/1738191869 >> 172.21.15.22:0/1738191869 conn(0x55bc0ae3bda0 msgr2=0x55bc0ae3c180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:09:03.201 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:03.201+0000 7f7e6cc4e740 1 -- 172.21.15.22:0/1738191869 shutdown_connections 2024-03-28T18:09:03.201 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:03.201+0000 7f7e6cc4e740 1 -- 172.21.15.22:0/1738191869 wait complete. 2024-03-28T18:09:03.201 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:03.201+0000 7f7e6cc4e740 1 -- start start 2024-03-28T18:09:03.202 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:03.201+0000 7f7e6cc4e740 1 Processor -- start 2024-03-28T18:09:03.203 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:03.202+0000 7f7e6cc4e740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55bc0ae416d0 0x55bc0ae4a510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:09:03.203 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:03.202+0000 7f7e6cc4e740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55bc0ae42080 0x55bc0ae478a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:09:03.203 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:03.202+0000 7f7e6cc4e740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55bc0ae4ac00 0x55bc0ae47e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:09:03.203 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:03.202+0000 7f7e6cc4e740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bc0ae4db10 con 0x55bc0ae4ac00 2024-03-28T18:09:03.203 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:03.202+0000 7f7e6cc4e740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bc0ad610d0 con 0x55bc0ae42080 2024-03-28T18:09:03.204 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:03.202+0000 7f7e6cc4e740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bc0ae4dc90 con 0x55bc0ae416d0 2024-03-28T18:09:03.204 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:03.202+0000 7f7e6cc4e740 10 client.0 _ll_register_callbacks cb 0x55bc0ad65e80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-28T18:09:03.204 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:03.202+0000 7f7e62a00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55bc0ae4ac00 0x55bc0ae47e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:09:03.204 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:03.202+0000 7f7e62a00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55bc0ae4ac00 0x55bc0ae47e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:54724/0 (socket says 172.21.15.22:54724) 2024-03-28T18:09:03.204 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:03.203+0000 7f7e63400640 1 --2- 172.21.15.22:0/3605410414 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55bc0ae42080 0x55bc0ae478a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:09:03.204 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:03.202+0000 7f7e62a00640 1 -- 172.21.15.22:0/3605410414 learned_addr learned my addr 172.21.15.22:0/3605410414 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-28T18:09:03.204 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:03.203+0000 7f7e62a00640 1 --2- 172.21.15.22:0/3605410414 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55bc0ae416d0 0x55bc0ae4a510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:09:03.204 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:03.203+0000 7f7e62a00640 1 -- 172.21.15.22:0/3605410414 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55bc0ae416d0 msgr2=0x55bc0ae4a510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T18:09:03.204 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:03.203+0000 7f7e62a00640 1 --2- 172.21.15.22:0/3605410414 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55bc0ae416d0 0x55bc0ae4a510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:09:03.204 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:03.203+0000 7f7e62a00640 1 -- 172.21.15.22:0/3605410414 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55bc0ae42080 msgr2=0x55bc0ae478a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T18:09:03.204 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:03.203+0000 7f7e62a00640 1 --2- 172.21.15.22:0/3605410414 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55bc0ae42080 0x55bc0ae478a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:09:03.204 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:03.203+0000 7f7e62a00640 1 -- 172.21.15.22:0/3605410414 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55bc0aab93f0 con 0x55bc0ae4ac00 2024-03-28T18:09:03.204 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:03.203+0000 7f7e63400640 1 --2- 172.21.15.22:0/3605410414 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55bc0ae42080 0x55bc0ae478a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-28T18:09:03.204 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:03.204+0000 7f7e62a00640 1 --2- 172.21.15.22:0/3605410414 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55bc0ae4ac00 0x55bc0ae47e40 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7f7e5002c160 tx=0x7f7e500043a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=78981594af034004 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:09:03.205 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:03.204+0000 7f7e4cc00640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-03-28T18:09:03.206 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:03.204+0000 7f7e4cc00640 1 -- 172.21.15.22:0/3605410414 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e5004a040 con 0x55bc0ae4ac00 2024-03-28T18:09:03.206 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:03.204+0000 7f7e4cc00640 1 -- 172.21.15.22:0/3605410414 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 8411+0+0 (secure 0 0 0) 0x7f7e5002c5c0 con 0x55bc0ae4ac00 2024-03-28T18:09:03.206 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:03.204+0000 7f7e6cc4e740 1 -- 172.21.15.22:0/3605410414 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f7e5002ca90 con 0x55bc0ae4ac00 2024-03-28T18:09:03.206 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:03.204+0000 7f7e4cc00640 1 -- 172.21.15.22:0/3605410414 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7e5002c7c0 con 0x55bc0ae4ac00 2024-03-28T18:09:03.206 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:03.204+0000 7f7e6cc4e740 10 client.16017 fetch_fsmap learned FSMap version 27 2024-03-28T18:09:03.207 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:03.204+0000 7f7e6cc4e740 1 -- 172.21.15.22:0/3605410414 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55bc0ab837d0 con 0x55bc0ae4ac00 2024-03-28T18:09:03.207 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:03.205+0000 7f7e4cc00640 1 -- 172.21.15.22:0/3605410414 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f7e5002ca90 con 0x55bc0ae4ac00 2024-03-28T18:09:03.207 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:03.205+0000 7f7e4cc00640 1 -- 172.21.15.22:0/3605410414 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 27) v1 ==== 7934+0+0 (secure 0 0 0) 0x7f7e50050040 con 0x55bc0ae4ac00 2024-03-28T18:09:03.207 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:03.205+0000 7f7e6cc4e740 10 client.16017 fetch_fsmap finished waiting for FSMap version 27 2024-03-28T18:09:03.207 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:03.205+0000 7f7e6cc4e740 10 client.16017 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-28T18:09:03.207 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:03.205+0000 7f7e6cc4e740 20 client.16017 populate_metadata read hostname 'smithi022' 2024-03-28T18:09:03.207 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:03.205+0000 7f7e6cc4e740 1 --2- 172.21.15.22:0/3605410414 >> [v2:172.21.15.79:6838/2441542443,v1:172.21.15.79:6839/2441542443] conn(0x55bc0ae5a910 0x55bc0ae5cd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:09:03.207 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:03.205+0000 7f7e6cc4e740 4 client.16017 mds_command: new command op to 34553 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-28T18:09:03.207 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:03.205+0000 7f7e6cc4e740 1 -- 172.21.15.22:0/3605410414 --> [v2:172.21.15.79:6838/2441542443,v1:172.21.15.79:6839/2441542443] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55bc0adb2800 con 0x55bc0ae5a910 2024-03-28T18:09:03.207 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:03.206+0000 7f7e63400640 1 --2- 172.21.15.22:0/3605410414 >> [v2:172.21.15.79:6838/2441542443,v1:172.21.15.79:6839/2441542443] conn(0x55bc0ae5a910 0x55bc0ae5cd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:09:03.207 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:03.207+0000 7f7e63400640 1 --2- 172.21.15.22:0/3605410414 >> [v2:172.21.15.79:6838/2441542443,v1:172.21.15.79:6839/2441542443] conn(0x55bc0ae5a910 0x55bc0ae5cd00 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=97d362e20eb6a29b server_cookie=6c8e1a7e98a41ba9 in_seq=0 out_seq=0 2024-03-28T18:09:03.207 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:03.207+0000 7f7e4cc00640 10 client.16017 ms_handle_connect on v2:172.21.15.79:6838/2441542443 2024-03-28T18:09:03.208 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:03.208+0000 7f7e4cc00640 1 -- 172.21.15.22:0/3605410414 <== mds.0 v2:172.21.15.79:6838/2441542443 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x55bc0adb2800 con 0x55bc0ae5a910 2024-03-28T18:09:03.208 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:03.208+0000 7f7e4cc00640 10 client.16017 handle_command_reply: tid=0 2024-03-28T18:09:03.209 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:03.208+0000 7f7e6cc4e740 1 client.16017 shutdown 2024-03-28T18:09:03.209 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:03.208+0000 7f7e6cc4e740 1 -- 172.21.15.22:0/3605410414 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55bc0ae4ac00 msgr2=0x55bc0ae47e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:09:03.209 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:03.208+0000 7f7e6cc4e740 1 --2- 172.21.15.22:0/3605410414 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55bc0ae4ac00 0x55bc0ae47e40 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7f7e5002c160 tx=0x7f7e500043a0 comp rx=0 tx=0).stop 2024-03-28T18:09:03.209 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:03.208+0000 7f7e6cc4e740 1 -- 172.21.15.22:0/3605410414 shutdown_connections 2024-03-28T18:09:03.209 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:03.208+0000 7f7e6cc4e740 1 --2- 172.21.15.22:0/3605410414 >> [v2:172.21.15.79:6838/2441542443,v1:172.21.15.79:6839/2441542443] conn(0x55bc0ae5a910 0x55bc0ae5cd00 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:09:03.209 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:03.208+0000 7f7e6cc4e740 1 --2- 172.21.15.22:0/3605410414 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55bc0ae4ac00 0x55bc0ae47e40 unknown :-1 s=CLOSED pgs=579 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:09:03.209 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:03.208+0000 7f7e6cc4e740 1 --2- 172.21.15.22:0/3605410414 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55bc0ae42080 0x55bc0ae478a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:09:03.209 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:03.208+0000 7f7e6cc4e740 1 --2- 172.21.15.22:0/3605410414 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55bc0ae416d0 0x55bc0ae4a510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:09:03.209 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:03.208+0000 7f7e4cc00640 0 client.16017 ms_handle_reset on v2:172.21.15.79:6838/2441542443 2024-03-28T18:09:03.209 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:03.208+0000 7f7e6cc4e740 1 -- 172.21.15.22:0/3605410414 >> 172.21.15.22:0/3605410414 conn(0x55bc0ae3bda0 msgr2=0x55bc0ae3daf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:09:03.209 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:03.208+0000 7f7e6cc4e740 1 -- 172.21.15.22:0/3605410414 shutdown_connections 2024-03-28T18:09:03.209 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:03.208+0000 7f7e6cc4e740 1 -- 172.21.15.22:0/3605410414 wait complete. 2024-03-28T18:09:03.209 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:03.209+0000 7f7e6cc4e740 20 client.16017 trim_cache size 0 max 16384 2024-03-28T18:09:03.229 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:03.229+0000 7f7e6cc4e740 1 Processor -- start 2024-03-28T18:09:03.229 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:03.229+0000 7f7e6cc4e740 1 -- start start 2024-03-28T18:09:03.230 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:03.229+0000 7f7e6cc4e740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55bc0ae416d0 0x55bc0ae46a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:09:03.230 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:03.230+0000 7f7e6cc4e740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55bc0ae42080 0x55bc0ae42bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:09:03.230 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:03.230+0000 7f7e6cc4e740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55bc0ae4ac00 0x55bc0ae43100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:09:03.231 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:03.230+0000 7f7e6cc4e740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e641af290 con 0x55bc0ae42080 2024-03-28T18:09:03.231 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:03.230+0000 7f7e6cc4e740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e641b0c10 con 0x55bc0ae416d0 2024-03-28T18:09:03.231 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:03.230+0000 7f7e6cc4e740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e641af8f0 con 0x55bc0ae4ac00 2024-03-28T18:09:03.231 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:03.230+0000 7f7e63400640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55bc0ae42080 0x55bc0ae42bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:09:03.231 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:03.230+0000 7f7e63400640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55bc0ae42080 0x55bc0ae42bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:54736/0 (socket says 172.21.15.22:54736) 2024-03-28T18:09:03.231 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:03.230+0000 7f7e63400640 1 -- 172.21.15.22:0/2015007202 learned_addr learned my addr 172.21.15.22:0/2015007202 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-28T18:09:03.231 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:03.230+0000 7f7e62a00640 1 --2- 172.21.15.22:0/2015007202 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55bc0ae416d0 0x55bc0ae46a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:09:03.231 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:03.230+0000 7f7e62a00640 1 --2- 172.21.15.22:0/2015007202 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55bc0ae4ac00 0x55bc0ae43100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:09:03.231 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:03.231+0000 7f7e63400640 1 -- 172.21.15.22:0/2015007202 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55bc0ae4ac00 msgr2=0x55bc0ae43100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T18:09:03.231 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:03.231+0000 7f7e63400640 1 --2- 172.21.15.22:0/2015007202 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55bc0ae4ac00 0x55bc0ae43100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:09:03.231 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:03.231+0000 7f7e63400640 1 -- 172.21.15.22:0/2015007202 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55bc0ae416d0 msgr2=0x55bc0ae46a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T18:09:03.231 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:03.231+0000 7f7e63400640 1 --2- 172.21.15.22:0/2015007202 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55bc0ae416d0 0x55bc0ae46a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:09:03.231 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:03.231+0000 7f7e63400640 1 -- 172.21.15.22:0/2015007202 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55bc0ae41e20 con 0x55bc0ae42080 2024-03-28T18:09:03.231 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:03.231+0000 7f7e63400640 1 --2- 172.21.15.22:0/2015007202 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55bc0ae42080 0x55bc0ae42bc0 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x55bc0ae3ce10 tx=0x7f7e54005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ebfc115133bd389c server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:09:03.232 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:03.231+0000 7f7e4f400640 1 -- 172.21.15.22:0/2015007202 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e540212e0 con 0x55bc0ae42080 2024-03-28T18:09:03.233 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:03.231+0000 7f7e4f400640 1 -- 172.21.15.22:0/2015007202 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f7e54025070 con 0x55bc0ae42080 2024-03-28T18:09:03.233 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:03.231+0000 7f7e4f400640 1 -- 172.21.15.22:0/2015007202 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e5401fc90 con 0x55bc0ae42080 2024-03-28T18:09:03.233 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:03.232+0000 7f7e6cc4e740 1 -- 172.21.15.22:0/2015007202 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55bc0ae42080 msgr2=0x55bc0ae42bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:09:03.233 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:03.232+0000 7f7e6cc4e740 1 --2- 172.21.15.22:0/2015007202 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55bc0ae42080 0x55bc0ae42bc0 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x55bc0ae3ce10 tx=0x7f7e54005f90 comp rx=0 tx=0).stop 2024-03-28T18:09:03.233 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:03.232+0000 7f7e6cc4e740 1 -- 172.21.15.22:0/2015007202 shutdown_connections 2024-03-28T18:09:03.233 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:03.232+0000 7f7e6cc4e740 1 --2- 172.21.15.22:0/2015007202 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55bc0ae4ac00 0x55bc0ae43100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:09:03.233 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:03.232+0000 7f7e6cc4e740 1 --2- 172.21.15.22:0/2015007202 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55bc0ae42080 0x55bc0ae42bc0 unknown :-1 s=CLOSED pgs=580 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:09:03.233 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:03.232+0000 7f7e6cc4e740 1 --2- 172.21.15.22:0/2015007202 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55bc0ae416d0 0x55bc0ae46a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:09:03.233 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:03.232+0000 7f7e6cc4e740 1 -- 172.21.15.22:0/2015007202 >> 172.21.15.22:0/2015007202 conn(0x55bc0ae3bda0 msgr2=0x55bc0ae424b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:09:03.233 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:03.233+0000 7f7e6cc4e740 1 -- 172.21.15.22:0/2015007202 shutdown_connections 2024-03-28T18:09:03.233 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:03.233+0000 7f7e6cc4e740 1 -- 172.21.15.22:0/2015007202 wait complete. 2024-03-28T18:09:03.233 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:03.233+0000 7f7e6cc4e740 1 -- start start 2024-03-28T18:09:03.233 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:03.233+0000 7f7e6cc4e740 1 Processor -- start 2024-03-28T18:09:03.236 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:03.234+0000 7f7e6cc4e740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55bc0ae416d0 0x55bc0ae5faa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:09:03.236 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:03.234+0000 7f7e6cc4e740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55bc0ae42080 0x55bc0ae5ce30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:09:03.236 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:03.234+0000 7f7e6cc4e740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55bc0ae4ac00 0x55bc0ae5d370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:09:03.236 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:03.234+0000 7f7e6cc4e740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e641b05b0 con 0x55bc0ae416d0 2024-03-28T18:09:03.236 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:03.234+0000 7f7e6cc4e740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e641aff50 con 0x55bc0ae42080 2024-03-28T18:09:03.236 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:03.234+0000 7f7e6cc4e740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e641ad450 con 0x55bc0ae4ac00 2024-03-28T18:09:03.236 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:03.234+0000 7f7e6cc4e740 10 client.0 _ll_register_callbacks cb 0x7f7e50048070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-28T18:09:03.236 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:03.234+0000 7f7e62a00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55bc0ae416d0 0x55bc0ae5faa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:09:03.236 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:03.234+0000 7f7e62a00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55bc0ae416d0 0x55bc0ae5faa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:54744/0 (socket says 172.21.15.22:54744) 2024-03-28T18:09:03.236 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:03.234+0000 7f7e62a00640 1 -- 172.21.15.22:0/3299542444 learned_addr learned my addr 172.21.15.22:0/3299542444 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-28T18:09:03.237 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:03.234+0000 7f7e63400640 1 --2- 172.21.15.22:0/3299542444 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55bc0ae42080 0x55bc0ae5ce30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:09:03.237 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:03.234+0000 7f7e62a00640 1 --2- 172.21.15.22:0/3299542444 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55bc0ae4ac00 0x55bc0ae5d370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:09:03.237 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:03.235+0000 7f7e62a00640 1 -- 172.21.15.22:0/3299542444 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55bc0ae4ac00 msgr2=0x55bc0ae5d370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T18:09:03.237 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:03.235+0000 7f7e62a00640 1 --2- 172.21.15.22:0/3299542444 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55bc0ae4ac00 0x55bc0ae5d370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:09:03.237 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:03.235+0000 7f7e62a00640 1 -- 172.21.15.22:0/3299542444 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55bc0ae42080 msgr2=0x55bc0ae5ce30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T18:09:03.237 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:03.235+0000 7f7e62a00640 1 --2- 172.21.15.22:0/3299542444 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55bc0ae42080 0x55bc0ae5ce30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:09:03.237 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:03.235+0000 7f7e62a00640 1 -- 172.21.15.22:0/3299542444 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55bc0ae5dba0 con 0x55bc0ae416d0 2024-03-28T18:09:03.237 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:03.235+0000 7f7e62a00640 1 --2- 172.21.15.22:0/3299542444 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55bc0ae416d0 0x55bc0ae5faa0 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x55bc0ae41280 tx=0x7f7e5002bc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=317e42e76abc837c server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:09:03.237 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:03.235+0000 7f7e4cc00640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-03-28T18:09:03.237 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:03.235+0000 7f7e4cc00640 1 -- 172.21.15.22:0/3299542444 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e500035f0 con 0x55bc0ae416d0 2024-03-28T18:09:03.237 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:03.235+0000 7f7e4cc00640 1 -- 172.21.15.22:0/3299542444 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 8411+0+0 (secure 0 0 0) 0x7f7e50003790 con 0x55bc0ae416d0 2024-03-28T18:09:03.237 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:03.235+0000 7f7e6cc4e740 1 -- 172.21.15.22:0/3299542444 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f7e54025070 con 0x55bc0ae416d0 2024-03-28T18:09:03.237 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:03.236+0000 7f7e4cc00640 1 -- 172.21.15.22:0/3299542444 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7e500523e0 con 0x55bc0ae416d0 2024-03-28T18:09:03.237 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:03.236+0000 7f7e6cc4e740 10 client.16023 fetch_fsmap learned FSMap version 27 2024-03-28T18:09:03.237 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:03.236+0000 7f7e6cc4e740 1 -- 172.21.15.22:0/3299542444 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55bc0aeb5e30 con 0x55bc0ae416d0 2024-03-28T18:09:03.238 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:03.236+0000 7f7e4cc00640 1 -- 172.21.15.22:0/3299542444 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f7e54025070 con 0x55bc0ae416d0 2024-03-28T18:09:03.238 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:03.236+0000 7f7e4cc00640 1 -- 172.21.15.22:0/3299542444 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 27) v1 ==== 7934+0+0 (secure 0 0 0) 0x7f7e50050070 con 0x55bc0ae416d0 2024-03-28T18:09:03.238 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:03.236+0000 7f7e6cc4e740 10 client.16023 fetch_fsmap finished waiting for FSMap version 27 2024-03-28T18:09:03.238 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:03.236+0000 7f7e6cc4e740 10 client.16023 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-28T18:09:03.238 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:03.236+0000 7f7e6cc4e740 20 client.16023 populate_metadata read hostname 'smithi022' 2024-03-28T18:09:03.238 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:03.236+0000 7f7e6cc4e740 1 --2- 172.21.15.22:0/3299542444 >> [v2:172.21.15.79:6838/2441542443,v1:172.21.15.79:6839/2441542443] conn(0x55bc0aeb62e0 0x55bc0aeb8700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:09:03.238 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:03.236+0000 7f7e6cc4e740 4 client.16023 mds_command: new command op to 34553 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-28T18:09:03.238 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:03.236+0000 7f7e6cc4e740 1 -- 172.21.15.22:0/3299542444 --> [v2:172.21.15.79:6838/2441542443,v1:172.21.15.79:6839/2441542443] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55bc0adacb50 con 0x55bc0aeb62e0 2024-03-28T18:09:03.239 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:03.237+0000 7f7e63400640 1 --2- 172.21.15.22:0/3299542444 >> [v2:172.21.15.79:6838/2441542443,v1:172.21.15.79:6839/2441542443] conn(0x55bc0aeb62e0 0x55bc0aeb8700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:09:03.239 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:03.237+0000 7f7e63400640 1 --2- 172.21.15.22:0/3299542444 >> [v2:172.21.15.79:6838/2441542443,v1:172.21.15.79:6839/2441542443] conn(0x55bc0aeb62e0 0x55bc0aeb8700 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5563fb87589e69db server_cookie=e2807cf43918b500 in_seq=0 out_seq=0 2024-03-28T18:09:03.239 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:03.237+0000 7f7e4cc00640 10 client.16023 ms_handle_connect on v2:172.21.15.79:6838/2441542443 2024-03-28T18:09:03.240 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:03.238+0000 7f7e4cc00640 1 -- 172.21.15.22:0/3299542444 <== mds.0 v2:172.21.15.79:6838/2441542443 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55bc0adacb50 con 0x55bc0aeb62e0 2024-03-28T18:09:03.240 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:03.238+0000 7f7e4cc00640 10 client.16023 handle_command_reply: tid=0 2024-03-28T18:09:03.240 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:03.238+0000 7f7e6cc4e740 1 client.16023 shutdown 2024-03-28T18:09:03.240 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:03.238+0000 7f7e6cc4e740 1 -- 172.21.15.22:0/3299542444 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55bc0ae416d0 msgr2=0x55bc0ae5faa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:09:03.240 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:03.238+0000 7f7e6cc4e740 1 --2- 172.21.15.22:0/3299542444 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55bc0ae416d0 0x55bc0ae5faa0 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x55bc0ae41280 tx=0x7f7e5002bc20 comp rx=0 tx=0).stop 2024-03-28T18:09:03.240 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:03.238+0000 7f7e6cc4e740 1 -- 172.21.15.22:0/3299542444 shutdown_connections 2024-03-28T18:09:03.240 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:03.238+0000 7f7e6cc4e740 1 --2- 172.21.15.22:0/3299542444 >> [v2:172.21.15.79:6838/2441542443,v1:172.21.15.79:6839/2441542443] conn(0x55bc0aeb62e0 0x55bc0aeb8700 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:09:03.240 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:03.238+0000 7f7e6cc4e740 1 --2- 172.21.15.22:0/3299542444 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55bc0ae4ac00 0x55bc0ae5d370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:09:03.240 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:03.238+0000 7f7e6cc4e740 1 --2- 172.21.15.22:0/3299542444 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55bc0ae42080 0x55bc0ae5ce30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:09:03.240 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:03.238+0000 7f7e6cc4e740 1 --2- 172.21.15.22:0/3299542444 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55bc0ae416d0 0x55bc0ae5faa0 unknown :-1 s=CLOSED pgs=581 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:09:03.240 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:03.238+0000 7f7e4cc00640 0 client.16023 ms_handle_reset on v2:172.21.15.79:6838/2441542443 2024-03-28T18:09:03.240 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:03.238+0000 7f7e6cc4e740 1 -- 172.21.15.22:0/3299542444 >> 172.21.15.22:0/3299542444 conn(0x55bc0ae3bda0 msgr2=0x55bc0ae3a620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:09:03.240 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:03.238+0000 7f7e6cc4e740 1 -- 172.21.15.22:0/3299542444 shutdown_connections 2024-03-28T18:09:03.240 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:03.238+0000 7f7e6cc4e740 1 -- 172.21.15.22:0/3299542444 wait complete. 2024-03-28T18:09:03.240 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:03.239+0000 7f7e6cc4e740 20 client.16023 trim_cache size 0 max 16384 2024-03-28T18:09:03.241 INFO:teuthology.orchestra.run.smithi022.stdout:[] 2024-03-28T18:09:03.241 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:03.239+0000 7f7e6a000640 1 -- 172.21.15.22:0/1290860068 >> [v2:172.21.15.22:6800/1357365082,v1:172.21.15.22:6801/1357365082] conn(0x7f7e480a2a90 msgr2=0x7f7e480a4f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:09:03.241 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:03.239+0000 7f7e6a000640 1 --2- 172.21.15.22:0/1290860068 >> [v2:172.21.15.22:6800/1357365082,v1:172.21.15.22:6801/1357365082] conn(0x7f7e480a2a90 0x7f7e480a4f50 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f7e5c0ccd90 tx=0x7f7e5c00c950 comp rx=0 tx=0).stop 2024-03-28T18:09:03.242 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:03.239+0000 7f7e6a000640 1 -- 172.21.15.22:0/1290860068 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7e64109850 msgr2=0x7f7e6419ea80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:09:03.242 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:03.239+0000 7f7e6a000640 1 --2- 172.21.15.22:0/1290860068 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7e64109850 0x7f7e6419ea80 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7f7e5c00eb10 tx=0x7f7e5c00c2d0 comp rx=0 tx=0).stop 2024-03-28T18:09:03.242 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:03.240+0000 7f7e6a000640 1 -- 172.21.15.22:0/1290860068 shutdown_connections 2024-03-28T18:09:03.242 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:03.240+0000 7f7e6a000640 1 --2- 172.21.15.22:0/1290860068 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7e64109850 0x7f7e6419ea80 unknown :-1 s=CLOSED pgs=578 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:09:03.242 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:03.240+0000 7f7e6a000640 1 --2- 172.21.15.22:0/1290860068 >> [v2:172.21.15.22:6800/1357365082,v1:172.21.15.22:6801/1357365082] conn(0x7f7e480a2a90 0x7f7e480a4f50 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:09:03.242 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:03.240+0000 7f7e6a000640 1 --2- 172.21.15.22:0/1290860068 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f7e64104200 0x7f7e6419a700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:09:03.242 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:03.240+0000 7f7e6a000640 1 --2- 172.21.15.22:0/1290860068 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7e64103840 0x7f7e6419a1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:09:03.242 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:03.240+0000 7f7e6a000640 1 -- 172.21.15.22:0/1290860068 >> 172.21.15.22:0/1290860068 conn(0x7f7e640ff350 msgr2=0x7f7e640ffb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:09:03.242 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:03.240+0000 7f7e6a000640 1 -- 172.21.15.22:0/1290860068 shutdown_connections 2024-03-28T18:09:03.242 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:03.240+0000 7f7e6a000640 1 -- 172.21.15.22:0/1290860068 wait complete. 2024-03-28T18:09:04.260 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-03-28T18:09:04.260 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-28T18:09:04.260 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-03-28T18:09:04.517 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:04.517+0000 7f70a5400640 1 -- 172.21.15.22:0/931663214 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f70a0109730 msgr2=0x7f70a0109b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:09:04.517 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:04.517+0000 7f70a5400640 1 --2- 172.21.15.22:0/931663214 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f70a0109730 0x7f70a0109b10 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f7090009900 tx=0x7f709002bc30 comp rx=0 tx=0).stop 2024-03-28T18:09:04.517 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:04.517+0000 7f70a5400640 1 -- 172.21.15.22:0/931663214 shutdown_connections 2024-03-28T18:09:04.517 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:04.517+0000 7f70a5400640 1 --2- 172.21.15.22:0/931663214 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f70a01040e0 0x7f70a01105f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:09:04.517 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:04.517+0000 7f70a5400640 1 --2- 172.21.15.22:0/931663214 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f70a0103720 0x7f70a0103ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:09:04.517 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:04.517+0000 7f70a5400640 1 --2- 172.21.15.22:0/931663214 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f70a0109730 0x7f70a0109b10 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:09:04.517 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:04.517+0000 7f70a5400640 1 -- 172.21.15.22:0/931663214 >> 172.21.15.22:0/931663214 conn(0x7f70a00ff1b0 msgr2=0x7f70a01015d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:09:04.517 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:04.517+0000 7f70a5400640 1 -- 172.21.15.22:0/931663214 shutdown_connections 2024-03-28T18:09:04.517 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:04.517+0000 7f70a5400640 1 -- 172.21.15.22:0/931663214 wait complete. 2024-03-28T18:09:04.518 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:04.518+0000 7f70a5400640 1 Processor -- start 2024-03-28T18:09:04.518 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:04.518+0000 7f70a5400640 1 -- start start 2024-03-28T18:09:04.519 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:04.519+0000 7f70a5400640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f70a0103720 0x7f70a019a360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:09:04.519 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:04.519+0000 7f70a5400640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f70a01040e0 0x7f70a019a8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:09:04.519 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:04.519+0000 7f70a5400640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f70a0109730 0x7f70a019ec20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:09:04.519 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:04.519+0000 7f70a5400640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70a0113b20 con 0x7f70a0109730 2024-03-28T18:09:04.519 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:04.519+0000 7f70a5400640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70a01139a0 con 0x7f70a0103720 2024-03-28T18:09:04.519 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:04.519+0000 7f70a5400640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70a0113ca0 con 0x7f70a01040e0 2024-03-28T18:09:04.519 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:04.519+0000 7f70a4a00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f70a0109730 0x7f70a019ec20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:09:04.519 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:04.519+0000 7f709fe00640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f70a0103720 0x7f70a019a360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:09:04.520 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:04.519+0000 7f709fe00640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f70a0103720 0x7f70a019a360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.22:35984/0 (socket says 172.21.15.22:35984) 2024-03-28T18:09:04.520 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:04.519+0000 7f709fe00640 1 -- 172.21.15.22:0/3607926390 learned_addr learned my addr 172.21.15.22:0/3607926390 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-28T18:09:04.520 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:04.520+0000 7f709f400640 1 --2- 172.21.15.22:0/3607926390 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f70a01040e0 0x7f70a019a8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:09:04.521 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:04.520+0000 7f709fe00640 1 -- 172.21.15.22:0/3607926390 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f70a01040e0 msgr2=0x7f70a019a8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:09:04.521 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:04.520+0000 7f709fe00640 1 --2- 172.21.15.22:0/3607926390 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f70a01040e0 0x7f70a019a8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:09:04.521 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:04.520+0000 7f709fe00640 1 -- 172.21.15.22:0/3607926390 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f70a0109730 msgr2=0x7f70a019ec20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:09:04.521 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:04.520+0000 7f709fe00640 1 --2- 172.21.15.22:0/3607926390 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f70a0109730 0x7f70a019ec20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:09:04.521 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:04.520+0000 7f709fe00640 1 -- 172.21.15.22:0/3607926390 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f70a019f350 con 0x7f70a0103720 2024-03-28T18:09:04.522 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:04.520+0000 7f70a4a00640 1 --2- 172.21.15.22:0/3607926390 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f70a0109730 0x7f70a019ec20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-28T18:09:04.522 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:04.520+0000 7f709fe00640 1 --2- 172.21.15.22:0/3607926390 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f70a0103720 0x7f70a019a360 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f7090000c00 tx=0x7f70900042f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:09:04.522 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:04.521+0000 7f709cc00640 1 -- 172.21.15.22:0/3607926390 <== mon.2 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7090043020 con 0x7f70a0103720 2024-03-28T18:09:04.522 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:04.521+0000 7f70a5400640 1 -- 172.21.15.22:0/3607926390 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f70a019f5e0 con 0x7f70a0103720 2024-03-28T18:09:04.522 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:04.521+0000 7f709cc00640 1 -- 172.21.15.22:0/3607926390 <== mon.2 v2:172.21.15.79:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f7090005470 con 0x7f70a0103720 2024-03-28T18:09:04.522 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:04.521+0000 7f709cc00640 1 -- 172.21.15.22:0/3607926390 <== mon.2 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7090003ab0 con 0x7f70a0103720 2024-03-28T18:09:04.522 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:04.521+0000 7f70a5400640 1 -- 172.21.15.22:0/3607926390 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f70a01a6ed0 con 0x7f70a0103720 2024-03-28T18:09:04.525 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:04.525+0000 7f70a981a740 1 Processor -- start 2024-03-28T18:09:04.525 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:04.525+0000 7f709cc00640 1 -- 172.21.15.22:0/3607926390 <== mon.2 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 25) v1 ==== 137523+0+0 (secure 0 0 0) 0x7f7090046020 con 0x7f70a0103720 2024-03-28T18:09:04.525 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:04.525+0000 7f70a981a740 1 -- start start 2024-03-28T18:09:04.525 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:04.526+0000 7f70a981a740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5636d2483290 0x5636d2483670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:09:04.526 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:04.526+0000 7f70a981a740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5636d2483bb0 0x5636d248c250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:09:04.526 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:04.526+0000 7f70a981a740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5636d248c790 0x5636d248eb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:09:04.526 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:04.526+0000 7f70a981a740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x5636d23e7eb0 con 0x5636d2483bb0 2024-03-28T18:09:04.526 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:04.526+0000 7f70a981a740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x5636d2390a60 con 0x5636d248c790 2024-03-28T18:09:04.526 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:04.526+0000 7f70a981a740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x5636d23e3f50 con 0x5636d2483290 2024-03-28T18:09:04.527 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:04.526+0000 7f709fe00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5636d2483bb0 0x5636d248c250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:09:04.527 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:04.526+0000 7f709fe00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5636d2483bb0 0x5636d248c250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:54780/0 (socket says 172.21.15.22:54780) 2024-03-28T18:09:04.527 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:04.526+0000 7f709fe00640 1 -- 172.21.15.22:0/1830913228 learned_addr learned my addr 172.21.15.22:0/1830913228 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-28T18:09:04.527 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:04.526+0000 7f70a4a00640 1 --2- 172.21.15.22:0/1830913228 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5636d2483290 0x5636d2483670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:09:04.527 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:04.526+0000 7f709f400640 1 --2- 172.21.15.22:0/1830913228 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5636d248c790 0x5636d248eb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:09:04.527 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:04.527+0000 7f709cc00640 1 --2- 172.21.15.22:0/3607926390 >> [v2:172.21.15.22:6800/1357365082,v1:172.21.15.22:6801/1357365082] conn(0x7f70840a2a90 0x7f70840a4f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:09:04.527 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:04.527+0000 7f709f400640 1 -- 172.21.15.22:0/1830913228 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5636d2483290 msgr2=0x5636d2483670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T18:09:04.527 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:04.527+0000 7f709f400640 1 --2- 172.21.15.22:0/1830913228 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5636d2483290 0x5636d2483670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:09:04.527 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:04.527+0000 7f709f400640 1 -- 172.21.15.22:0/1830913228 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5636d2483bb0 msgr2=0x5636d248c250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T18:09:04.528 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:04.527+0000 7f709f400640 1 --2- 172.21.15.22:0/1830913228 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5636d2483bb0 0x5636d248c250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:09:04.528 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:04.527+0000 7f709f400640 1 -- 172.21.15.22:0/1830913228 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5636d21c47e0 con 0x5636d248c790 2024-03-28T18:09:04.528 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:04.527+0000 7f709cc00640 1 -- 172.21.15.22:0/3607926390 <== mon.2 v2:172.21.15.79:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 8411+0+0 (secure 0 0 0) 0x7f70900f0fb0 con 0x7f70a0103720 2024-03-28T18:09:04.528 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:04.527+0000 7f70a4a00640 1 --2- 172.21.15.22:0/3607926390 >> [v2:172.21.15.22:6800/1357365082,v1:172.21.15.22:6801/1357365082] conn(0x7f70840a2a90 0x7f70840a4f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:09:04.529 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:04.528+0000 7f709f400640 1 --2- 172.21.15.22:0/1830913228 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5636d248c790 0x5636d248eb80 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f708c009bf0 tx=0x7f708c011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4f1133ab286ac821 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:09:04.529 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:04.528+0000 7f7095200640 1 -- 172.21.15.22:0/1830913228 <== mon.2 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f708c009da0 con 0x5636d248c790 2024-03-28T18:09:04.529 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:04.528+0000 7f7095200640 1 -- 172.21.15.22:0/1830913228 <== mon.2 v2:172.21.15.79:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f708c005020 con 0x5636d248c790 2024-03-28T18:09:04.529 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:04.528+0000 7f7095200640 1 -- 172.21.15.22:0/1830913228 <== mon.2 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f708c004bc0 con 0x5636d248c790 2024-03-28T18:09:04.529 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:04.529+0000 7f70a981a740 1 -- 172.21.15.22:0/1830913228 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5636d248c790 msgr2=0x5636d248eb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:09:04.530 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:04.529+0000 7f70a981a740 1 --2- 172.21.15.22:0/1830913228 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5636d248c790 0x5636d248eb80 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f708c009bf0 tx=0x7f708c011810 comp rx=0 tx=0).stop 2024-03-28T18:09:04.530 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:04.529+0000 7f70a4a00640 1 --2- 172.21.15.22:0/3607926390 >> [v2:172.21.15.22:6800/1357365082,v1:172.21.15.22:6801/1357365082] conn(0x7f70840a2a90 0x7f70840a4f50 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f7098005c40 tx=0x7f7098004a40 comp rx=0 tx=0).ready entity=mgr.15522 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:09:04.530 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:04.529+0000 7f70a981a740 1 -- 172.21.15.22:0/1830913228 shutdown_connections 2024-03-28T18:09:04.530 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:04.529+0000 7f70a981a740 1 --2- 172.21.15.22:0/1830913228 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5636d248c790 0x5636d248eb80 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:09:04.530 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:04.529+0000 7f70a981a740 1 --2- 172.21.15.22:0/1830913228 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5636d2483bb0 0x5636d248c250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:09:04.530 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:04.529+0000 7f70a981a740 1 --2- 172.21.15.22:0/1830913228 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5636d2483290 0x5636d2483670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:09:04.530 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:04.529+0000 7f70a981a740 1 -- 172.21.15.22:0/1830913228 >> 172.21.15.22:0/1830913228 conn(0x5636d247d9c0 msgr2=0x5636d247dda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:09:04.530 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:04.529+0000 7f70a981a740 1 -- 172.21.15.22:0/1830913228 shutdown_connections 2024-03-28T18:09:04.530 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:04.529+0000 7f70a981a740 1 -- 172.21.15.22:0/1830913228 wait complete. 2024-03-28T18:09:04.530 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:04.530+0000 7f70a981a740 1 -- start start 2024-03-28T18:09:04.532 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:04.530+0000 7f70a981a740 1 Processor -- start 2024-03-28T18:09:04.532 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:04.531+0000 7f70a981a740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5636d2483290 0x5636d248c090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:09:04.532 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:04.531+0000 7f70a981a740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5636d2483bb0 0x5636d2489420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:09:04.532 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:04.531+0000 7f70a981a740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5636d248c790 0x5636d2489960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:09:04.532 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:04.531+0000 7f70a981a740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x5636d248f580 con 0x5636d248c790 2024-03-28T18:09:04.532 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:04.531+0000 7f70a981a740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x5636d23a3eb0 con 0x5636d2483bb0 2024-03-28T18:09:04.532 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:04.531+0000 7f70a981a740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x5636d248f700 con 0x5636d2483290 2024-03-28T18:09:04.532 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:04.531+0000 7f70a981a740 10 client.0 _ll_register_callbacks cb 0x5636d2398150 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-28T18:09:04.532 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:04.531+0000 7f70a4a00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5636d248c790 0x5636d2489960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:09:04.533 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:04.531+0000 7f709f400640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5636d2483bb0 0x5636d2489420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:09:04.533 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:04.531+0000 7f70a4a00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5636d248c790 0x5636d2489960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:54794/0 (socket says 172.21.15.22:54794) 2024-03-28T18:09:04.533 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:04.531+0000 7f70a4a00640 1 -- 172.21.15.22:0/3636949219 learned_addr learned my addr 172.21.15.22:0/3636949219 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-28T18:09:04.533 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:04.531+0000 7f709fe00640 1 --2- 172.21.15.22:0/3636949219 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5636d2483290 0x5636d248c090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:09:04.533 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:04.532+0000 7f70a4a00640 1 -- 172.21.15.22:0/3636949219 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5636d2483290 msgr2=0x5636d248c090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T18:09:04.533 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:04.532+0000 7f70a4a00640 1 --2- 172.21.15.22:0/3636949219 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5636d2483290 0x5636d248c090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:09:04.533 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:04.532+0000 7f70a4a00640 1 -- 172.21.15.22:0/3636949219 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5636d2483bb0 msgr2=0x5636d2489420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T18:09:04.533 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:04.532+0000 7f70a4a00640 1 --2- 172.21.15.22:0/3636949219 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5636d2483bb0 0x5636d2489420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:09:04.533 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:04.532+0000 7f70a4a00640 1 -- 172.21.15.22:0/3636949219 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5636d2224f70 con 0x5636d248c790 2024-03-28T18:09:04.533 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:04.532+0000 7f70a4a00640 1 --2- 172.21.15.22:0/3636949219 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5636d248c790 0x5636d2489960 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7f709801f720 tx=0x7f7098004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e8ae64bb7ff8d182 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:09:04.533 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:04.532+0000 7f7089600640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-03-28T18:09:04.533 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:04.532+0000 7f7089600640 1 -- 172.21.15.22:0/3636949219 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7098004b20 con 0x5636d248c790 2024-03-28T18:09:04.533 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:04.532+0000 7f7089600640 1 -- 172.21.15.22:0/3636949219 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 8411+0+0 (secure 0 0 0) 0x7f7098004cc0 con 0x5636d248c790 2024-03-28T18:09:04.533 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:04.532+0000 7f70a981a740 1 -- 172.21.15.22:0/3636949219 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f708c005020 con 0x5636d248c790 2024-03-28T18:09:04.534 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:04.533+0000 7f7089600640 1 -- 172.21.15.22:0/3636949219 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f709801c5a0 con 0x5636d248c790 2024-03-28T18:09:04.535 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:04.533+0000 7f70a981a740 10 client.16035 fetch_fsmap learned FSMap version 27 2024-03-28T18:09:04.535 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:04.533+0000 7f70a981a740 1 -- 172.21.15.22:0/3636949219 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x5636d20bc170 con 0x5636d248c790 2024-03-28T18:09:04.535 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:04.533+0000 7f7089600640 1 -- 172.21.15.22:0/3636949219 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f708c005020 con 0x5636d248c790 2024-03-28T18:09:04.535 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:04.533+0000 7f7089600640 1 -- 172.21.15.22:0/3636949219 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 27) v1 ==== 7934+0+0 (secure 0 0 0) 0x7f7098030030 con 0x5636d248c790 2024-03-28T18:09:04.535 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:04.533+0000 7f70a981a740 10 client.16035 fetch_fsmap finished waiting for FSMap version 27 2024-03-28T18:09:04.535 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:04.533+0000 7f70a981a740 10 client.16035 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-28T18:09:04.535 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:04.533+0000 7f70a981a740 20 client.16035 populate_metadata read hostname 'smithi022' 2024-03-28T18:09:04.535 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:04.533+0000 7f70a981a740 1 --2- 172.21.15.22:0/3636949219 >> [v2:172.21.15.79:6838/2441542443,v1:172.21.15.79:6839/2441542443] conn(0x5636d249c800 0x5636d249ec60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:09:04.535 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:04.533+0000 7f70a981a740 4 client.16035 mds_command: new command op to 34553 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-28T18:09:04.535 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:04.534+0000 7f70a981a740 1 -- 172.21.15.22:0/3636949219 --> [v2:172.21.15.79:6838/2441542443,v1:172.21.15.79:6839/2441542443] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5636d23f53f0 con 0x5636d249c800 2024-03-28T18:09:04.535 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:04.534+0000 7f709fe00640 1 --2- 172.21.15.22:0/3636949219 >> [v2:172.21.15.79:6838/2441542443,v1:172.21.15.79:6839/2441542443] conn(0x5636d249c800 0x5636d249ec60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:09:04.535 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:04.535+0000 7f709fe00640 1 --2- 172.21.15.22:0/3636949219 >> [v2:172.21.15.79:6838/2441542443,v1:172.21.15.79:6839/2441542443] conn(0x5636d249c800 0x5636d249ec60 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f9065da13dc486c2 server_cookie=246174c862eec26d in_seq=0 out_seq=0 2024-03-28T18:09:04.535 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:04.535+0000 7f7089600640 10 client.16035 ms_handle_connect on v2:172.21.15.79:6838/2441542443 2024-03-28T18:09:04.537 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:04.536+0000 7f7089600640 1 -- 172.21.15.22:0/3636949219 <== mds.0 v2:172.21.15.79:6838/2441542443 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x5636d23f53f0 con 0x5636d249c800 2024-03-28T18:09:04.537 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:04.536+0000 7f7089600640 10 client.16035 handle_command_reply: tid=0 2024-03-28T18:09:04.537 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:04.536+0000 7f70a981a740 1 client.16035 shutdown 2024-03-28T18:09:04.537 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:04.536+0000 7f70a981a740 1 -- 172.21.15.22:0/3636949219 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5636d248c790 msgr2=0x5636d2489960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:09:04.537 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:04.536+0000 7f70a981a740 1 --2- 172.21.15.22:0/3636949219 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5636d248c790 0x5636d2489960 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7f709801f720 tx=0x7f7098004ab0 comp rx=0 tx=0).stop 2024-03-28T18:09:04.537 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:04.536+0000 7f70a981a740 1 -- 172.21.15.22:0/3636949219 shutdown_connections 2024-03-28T18:09:04.537 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:04.536+0000 7f70a981a740 1 --2- 172.21.15.22:0/3636949219 >> [v2:172.21.15.79:6838/2441542443,v1:172.21.15.79:6839/2441542443] conn(0x5636d249c800 0x5636d249ec60 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:09:04.537 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:04.536+0000 7f70a981a740 1 --2- 172.21.15.22:0/3636949219 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5636d248c790 0x5636d2489960 unknown :-1 s=CLOSED pgs=582 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:09:04.537 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:04.536+0000 7f70a981a740 1 --2- 172.21.15.22:0/3636949219 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5636d2483bb0 0x5636d2489420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:09:04.537 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:04.536+0000 7f70a981a740 1 --2- 172.21.15.22:0/3636949219 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5636d2483290 0x5636d248c090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:09:04.537 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:04.536+0000 7f70a981a740 1 -- 172.21.15.22:0/3636949219 >> 172.21.15.22:0/3636949219 conn(0x5636d247d9c0 msgr2=0x5636d247f710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:09:04.537 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:04.536+0000 7f7089600640 0 client.16035 ms_handle_reset on v2:172.21.15.79:6838/2441542443 2024-03-28T18:09:04.537 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:04.536+0000 7f70a981a740 1 -- 172.21.15.22:0/3636949219 shutdown_connections 2024-03-28T18:09:04.537 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:04.537+0000 7f70a981a740 1 -- 172.21.15.22:0/3636949219 wait complete. 2024-03-28T18:09:04.538 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:04.537+0000 7f70a981a740 20 client.16035 trim_cache size 0 max 16384 2024-03-28T18:09:04.560 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:04.561+0000 7f70a981a740 1 Processor -- start 2024-03-28T18:09:04.560 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:04.561+0000 7f70a981a740 1 -- start start 2024-03-28T18:09:04.562 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:04.561+0000 7f70a981a740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5636d2483290 0x5636d248bb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:09:04.562 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:04.561+0000 7f70a981a740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5636d2483bb0 0x5636d248c090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:09:04.562 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:04.561+0000 7f70a981a740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5636d248c790 0x5636d2497e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:09:04.562 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:04.561+0000 7f70a981a740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70a01aefd0 con 0x5636d2483290 2024-03-28T18:09:04.562 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:04.561+0000 7f70a981a740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70a01b0950 con 0x5636d2483bb0 2024-03-28T18:09:04.562 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:04.561+0000 7f70a981a740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70a01af630 con 0x5636d248c790 2024-03-28T18:09:04.562 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:04.561+0000 7f709fe00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5636d2483290 0x5636d248bb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:09:04.562 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:04.561+0000 7f709fe00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5636d2483290 0x5636d248bb50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:54802/0 (socket says 172.21.15.22:54802) 2024-03-28T18:09:04.562 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:04.562+0000 7f709fe00640 1 -- 172.21.15.22:0/1767536077 learned_addr learned my addr 172.21.15.22:0/1767536077 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-28T18:09:04.562 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:04.562+0000 7f709f400640 1 --2- 172.21.15.22:0/1767536077 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5636d2483bb0 0x5636d248c090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:09:04.562 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:04.562+0000 7f70a4a00640 1 --2- 172.21.15.22:0/1767536077 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5636d248c790 0x5636d2497e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:09:04.562 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:04.562+0000 7f709fe00640 1 -- 172.21.15.22:0/1767536077 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5636d248c790 msgr2=0x5636d2497e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T18:09:04.563 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:04.562+0000 7f709fe00640 1 --2- 172.21.15.22:0/1767536077 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5636d248c790 0x5636d2497e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:09:04.563 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:04.562+0000 7f709fe00640 1 -- 172.21.15.22:0/1767536077 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5636d2483bb0 msgr2=0x5636d248c090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T18:09:04.563 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:04.562+0000 7f709fe00640 1 --2- 172.21.15.22:0/1767536077 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5636d2483bb0 0x5636d248c090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:09:04.563 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:04.562+0000 7f709fe00640 1 -- 172.21.15.22:0/1767536077 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5636d24883f0 con 0x5636d2483290 2024-03-28T18:09:04.563 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:04.562+0000 7f709fe00640 1 --2- 172.21.15.22:0/1767536077 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5636d2483290 0x5636d248bb50 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7f70900b9f10 tx=0x7f70900f34c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d70caa6ca4e27796 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:09:04.565 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:04.563+0000 7f708be00640 1 -- 172.21.15.22:0/1767536077 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70900b97e0 con 0x5636d2483290 2024-03-28T18:09:04.565 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:04.563+0000 7f708be00640 1 -- 172.21.15.22:0/1767536077 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f7090004c70 con 0x5636d2483290 2024-03-28T18:09:04.565 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:04.563+0000 7f708be00640 1 -- 172.21.15.22:0/1767536077 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70900bcea0 con 0x5636d2483290 2024-03-28T18:09:04.566 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:04.563+0000 7f70a981a740 1 -- 172.21.15.22:0/1767536077 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5636d2483290 msgr2=0x5636d248bb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:09:04.566 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:04.563+0000 7f70a981a740 1 --2- 172.21.15.22:0/1767536077 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5636d2483290 0x5636d248bb50 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7f70900b9f10 tx=0x7f70900f34c0 comp rx=0 tx=0).stop 2024-03-28T18:09:04.566 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:04.563+0000 7f70a981a740 1 -- 172.21.15.22:0/1767536077 shutdown_connections 2024-03-28T18:09:04.566 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:04.563+0000 7f70a981a740 1 --2- 172.21.15.22:0/1767536077 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5636d248c790 0x5636d2497e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:09:04.566 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:04.563+0000 7f70a981a740 1 --2- 172.21.15.22:0/1767536077 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5636d2483bb0 0x5636d248c090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:09:04.566 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:04.563+0000 7f70a981a740 1 --2- 172.21.15.22:0/1767536077 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5636d2483290 0x5636d248bb50 unknown :-1 s=CLOSED pgs=583 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:09:04.566 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:04.563+0000 7f70a981a740 1 -- 172.21.15.22:0/1767536077 >> 172.21.15.22:0/1767536077 conn(0x5636d247d9c0 msgr2=0x5636d2484820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:09:04.566 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:04.564+0000 7f70a981a740 1 -- 172.21.15.22:0/1767536077 shutdown_connections 2024-03-28T18:09:04.566 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:04.564+0000 7f70a981a740 1 -- 172.21.15.22:0/1767536077 wait complete. 2024-03-28T18:09:04.566 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:04.564+0000 7f70a981a740 1 -- start start 2024-03-28T18:09:04.566 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:04.564+0000 7f70a981a740 1 Processor -- start 2024-03-28T18:09:04.566 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:04.565+0000 7f70a981a740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5636d2483290 0x5636d2494430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:09:04.567 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:04.565+0000 7f70a981a740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5636d2483bb0 0x5636d2491950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:09:04.567 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:04.565+0000 7f70a981a740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5636d248c790 0x5636d2491e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:09:04.567 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:04.565+0000 7f70a981a740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70a01b02f0 con 0x5636d2483290 2024-03-28T18:09:04.567 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:04.565+0000 7f70a981a740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70a01afc90 con 0x5636d248c790 2024-03-28T18:09:04.567 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:04.565+0000 7f70a981a740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70a01ad1b0 con 0x5636d2483bb0 2024-03-28T18:09:04.567 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:04.565+0000 7f70a981a740 10 client.0 _ll_register_callbacks cb 0x7f709800f990 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-28T18:09:04.567 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:04.565+0000 7f709fe00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5636d2483290 0x5636d2494430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:09:04.567 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:04.565+0000 7f709fe00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5636d2483290 0x5636d2494430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:54818/0 (socket says 172.21.15.22:54818) 2024-03-28T18:09:04.567 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:04.565+0000 7f709fe00640 1 -- 172.21.15.22:0/3047919531 learned_addr learned my addr 172.21.15.22:0/3047919531 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-28T18:09:04.567 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:04.565+0000 7f70a4a00640 1 --2- 172.21.15.22:0/3047919531 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5636d248c790 0x5636d2491e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:09:04.567 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:04.565+0000 7f709f400640 1 --2- 172.21.15.22:0/3047919531 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5636d2483bb0 0x5636d2491950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:09:04.567 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:04.565+0000 7f709fe00640 1 -- 172.21.15.22:0/3047919531 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5636d2483bb0 msgr2=0x5636d2491950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T18:09:04.567 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:04.565+0000 7f709fe00640 1 --2- 172.21.15.22:0/3047919531 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5636d2483bb0 0x5636d2491950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:09:04.567 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:04.565+0000 7f709fe00640 1 -- 172.21.15.22:0/3047919531 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5636d248c790 msgr2=0x5636d2491e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T18:09:04.567 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:04.565+0000 7f709fe00640 1 --2- 172.21.15.22:0/3047919531 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5636d248c790 0x5636d2491e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:09:04.568 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:04.566+0000 7f709fe00640 1 -- 172.21.15.22:0/3047919531 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5636d24923d0 con 0x5636d2483290 2024-03-28T18:09:04.568 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:04.566+0000 7f70a4a00640 1 --2- 172.21.15.22:0/3047919531 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5636d248c790 0x5636d2491e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-28T18:09:04.568 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:04.566+0000 7f709fe00640 1 --2- 172.21.15.22:0/3047919531 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5636d2483290 0x5636d2494430 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7f709002c210 tx=0x7f70900bcf30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b0cc867d43f3db52 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:09:04.568 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:04.566+0000 7f7089600640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-03-28T18:09:04.569 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:04.566+0000 7f7089600640 1 -- 172.21.15.22:0/3047919531 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70900bd080 con 0x5636d2483290 2024-03-28T18:09:04.569 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:04.566+0000 7f7089600640 1 -- 172.21.15.22:0/3047919531 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 8411+0+0 (secure 0 0 0) 0x7f70900f4a40 con 0x5636d2483290 2024-03-28T18:09:04.569 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:04.566+0000 7f70a981a740 1 -- 172.21.15.22:0/3047919531 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f7090004c70 con 0x5636d2483290 2024-03-28T18:09:04.570 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:04.566+0000 7f7089600640 1 -- 172.21.15.22:0/3047919531 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f70900bd310 con 0x5636d2483290 2024-03-28T18:09:04.570 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:04.566+0000 7f70a981a740 10 client.16041 fetch_fsmap learned FSMap version 27 2024-03-28T18:09:04.570 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:04.567+0000 7f70a981a740 1 -- 172.21.15.22:0/3047919531 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x5636d24f9e60 con 0x5636d2483290 2024-03-28T18:09:04.570 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:04.567+0000 7f7089600640 1 -- 172.21.15.22:0/3047919531 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f7090004c70 con 0x5636d2483290 2024-03-28T18:09:04.570 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:04.567+0000 7f7089600640 1 -- 172.21.15.22:0/3047919531 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 27) v1 ==== 7934+0+0 (secure 0 0 0) 0x7f709002cab0 con 0x5636d2483290 2024-03-28T18:09:04.570 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:04.567+0000 7f70a981a740 10 client.16041 fetch_fsmap finished waiting for FSMap version 27 2024-03-28T18:09:04.570 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:04.567+0000 7f70a981a740 10 client.16041 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-28T18:09:04.570 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:04.567+0000 7f70a981a740 20 client.16041 populate_metadata read hostname 'smithi022' 2024-03-28T18:09:04.570 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:04.567+0000 7f70a981a740 1 --2- 172.21.15.22:0/3047919531 >> [v2:172.21.15.79:6838/2441542443,v1:172.21.15.79:6839/2441542443] conn(0x5636d24fa480 0x5636d24fc870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:09:04.570 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:04.567+0000 7f70a981a740 4 client.16041 mds_command: new command op to 34553 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-03-28T18:09:04.570 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:04.567+0000 7f70a981a740 1 -- 172.21.15.22:0/3047919531 --> [v2:172.21.15.79:6838/2441542443,v1:172.21.15.79:6839/2441542443] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x5636d23ef740 con 0x5636d24fa480 2024-03-28T18:09:04.571 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:04.568+0000 7f709f400640 1 --2- 172.21.15.22:0/3047919531 >> [v2:172.21.15.79:6838/2441542443,v1:172.21.15.79:6839/2441542443] conn(0x5636d24fa480 0x5636d24fc870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:09:04.571 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:04.568+0000 7f709f400640 1 --2- 172.21.15.22:0/3047919531 >> [v2:172.21.15.79:6838/2441542443,v1:172.21.15.79:6839/2441542443] conn(0x5636d24fa480 0x5636d24fc870 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5b50644e88928e74 server_cookie=83f566d4b890c9e in_seq=0 out_seq=0 2024-03-28T18:09:04.571 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:04.568+0000 7f7089600640 10 client.16041 ms_handle_connect on v2:172.21.15.79:6838/2441542443 2024-03-28T18:09:04.572 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:04.571+0000 7f7089600640 1 -- 172.21.15.22:0/3047919531 <== mds.0 v2:172.21.15.79:6838/2441542443 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x5636d23ef740 con 0x5636d24fa480 2024-03-28T18:09:04.572 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:04.571+0000 7f7089600640 10 client.16041 handle_command_reply: tid=0 2024-03-28T18:09:04.572 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:04.571+0000 7f70a981a740 1 client.16041 shutdown 2024-03-28T18:09:04.572 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:04.572+0000 7f70a981a740 1 -- 172.21.15.22:0/3047919531 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5636d2483290 msgr2=0x5636d2494430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:09:04.572 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:04.572+0000 7f70a981a740 1 --2- 172.21.15.22:0/3047919531 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5636d2483290 0x5636d2494430 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7f709002c210 tx=0x7f70900bcf30 comp rx=0 tx=0).stop 2024-03-28T18:09:04.572 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:04.572+0000 7f70a981a740 1 -- 172.21.15.22:0/3047919531 shutdown_connections 2024-03-28T18:09:04.572 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:04.572+0000 7f70a981a740 1 --2- 172.21.15.22:0/3047919531 >> [v2:172.21.15.79:6838/2441542443,v1:172.21.15.79:6839/2441542443] conn(0x5636d24fa480 0x5636d24fc870 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:09:04.572 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:04.572+0000 7f70a981a740 1 --2- 172.21.15.22:0/3047919531 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5636d248c790 0x5636d2491e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:09:04.573 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:04.572+0000 7f70a981a740 1 --2- 172.21.15.22:0/3047919531 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5636d2483bb0 0x5636d2491950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:09:04.573 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:04.572+0000 7f70a981a740 1 --2- 172.21.15.22:0/3047919531 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5636d2483290 0x5636d2494430 unknown :-1 s=CLOSED pgs=584 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:09:04.573 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:04.572+0000 7f7089600640 0 client.16041 ms_handle_reset on v2:172.21.15.79:6838/2441542443 2024-03-28T18:09:04.573 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:04.572+0000 7f70a981a740 1 -- 172.21.15.22:0/3047919531 >> 172.21.15.22:0/3047919531 conn(0x5636d247d9c0 msgr2=0x5636d247eb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:09:04.573 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:04.572+0000 7f70a981a740 1 -- 172.21.15.22:0/3047919531 shutdown_connections 2024-03-28T18:09:04.573 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:04.572+0000 7f70a981a740 1 -- 172.21.15.22:0/3047919531 wait complete. 2024-03-28T18:09:04.573 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:04.572+0000 7f70a981a740 20 client.16041 trim_cache size 0 max 16384 2024-03-28T18:09:04.573 INFO:teuthology.orchestra.run.smithi022.stdout:{ 2024-03-28T18:09:04.573 INFO:teuthology.orchestra.run.smithi022.stdout: "return_code": 0, 2024-03-28T18:09:04.574 INFO:teuthology.orchestra.run.smithi022.stdout: "scrub_tag": "d281d108-fad4-4f95-99cc-457b98b3c446", 2024-03-28T18:09:04.574 INFO:teuthology.orchestra.run.smithi022.stdout: "mode": "asynchronous" 2024-03-28T18:09:04.574 INFO:teuthology.orchestra.run.smithi022.stdout:} 2024-03-28T18:09:04.574 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:04.573+0000 7f70a5400640 1 -- 172.21.15.22:0/3607926390 >> [v2:172.21.15.22:6800/1357365082,v1:172.21.15.22:6801/1357365082] conn(0x7f70840a2a90 msgr2=0x7f70840a4f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:09:04.575 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:04.573+0000 7f70a5400640 1 --2- 172.21.15.22:0/3607926390 >> [v2:172.21.15.22:6800/1357365082,v1:172.21.15.22:6801/1357365082] conn(0x7f70840a2a90 0x7f70840a4f50 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f7098005c40 tx=0x7f7098004a40 comp rx=0 tx=0).stop 2024-03-28T18:09:04.575 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:04.573+0000 7f70a5400640 1 -- 172.21.15.22:0/3607926390 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f70a0103720 msgr2=0x7f70a019a360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:09:04.575 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:04.573+0000 7f70a5400640 1 --2- 172.21.15.22:0/3607926390 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f70a0103720 0x7f70a019a360 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f7090000c00 tx=0x7f70900042f0 comp rx=0 tx=0).stop 2024-03-28T18:09:04.575 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:04.573+0000 7f70a5400640 1 -- 172.21.15.22:0/3607926390 shutdown_connections 2024-03-28T18:09:04.575 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:04.573+0000 7f70a5400640 1 --2- 172.21.15.22:0/3607926390 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f70a0109730 0x7f70a019ec20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:09:04.575 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:04.573+0000 7f70a5400640 1 --2- 172.21.15.22:0/3607926390 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f70a01040e0 0x7f70a019a8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:09:04.575 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:04.573+0000 7f70a5400640 1 --2- 172.21.15.22:0/3607926390 >> [v2:172.21.15.22:6800/1357365082,v1:172.21.15.22:6801/1357365082] conn(0x7f70840a2a90 0x7f70840a4f50 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:09:04.575 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:04.573+0000 7f70a5400640 1 --2- 172.21.15.22:0/3607926390 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f70a0103720 0x7f70a019a360 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:09:04.575 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:04.573+0000 7f70a5400640 1 -- 172.21.15.22:0/3607926390 >> 172.21.15.22:0/3607926390 conn(0x7f70a00ff1b0 msgr2=0x7f70a00ffb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:09:04.575 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:04.573+0000 7f70a5400640 1 -- 172.21.15.22:0/3607926390 shutdown_connections 2024-03-28T18:09:04.575 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:04.573+0000 7f70a5400640 1 -- 172.21.15.22:0/3607926390 wait complete. 2024-03-28T18:09:04.592 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-28T18:09:04.592 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-28T18:09:04.679 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:09:04 smithi022 ceph-mon[31985]: pgmap v48: 97 pgs: 97 active+clean; 1.3 MiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-28T18:09:04.825 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:04.825+0000 7f2bed600640 1 -- 172.21.15.22:0/2602129967 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f2be8069240 msgr2=0x7f2be810e4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:09:04.825 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:04.825+0000 7f2bed600640 1 --2- 172.21.15.22:0/2602129967 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f2be8069240 0x7f2be810e4e0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f2be0009ca0 tx=0x7f2be002c100 comp rx=0 tx=0).stop 2024-03-28T18:09:04.825 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:04.825+0000 7f2bed600640 1 -- 172.21.15.22:0/2602129967 shutdown_connections 2024-03-28T18:09:04.826 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:04.825+0000 7f2bed600640 1 --2- 172.21.15.22:0/2602129967 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f2be8069240 0x7f2be810e4e0 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:09:04.826 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:04.825+0000 7f2bed600640 1 --2- 172.21.15.22:0/2602129967 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2be8068880 0x7f2be8068d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:09:04.826 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:04.825+0000 7f2bed600640 1 --2- 172.21.15.22:0/2602129967 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2be81076b0 0x7f2be8107a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:09:04.826 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:04.825+0000 7f2bed600640 1 -- 172.21.15.22:0/2602129967 >> 172.21.15.22:0/2602129967 conn(0x7f2be80776a0 msgr2=0x7f2be8077ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:09:04.826 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:04.825+0000 7f2bed600640 1 -- 172.21.15.22:0/2602129967 shutdown_connections 2024-03-28T18:09:04.826 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:04.825+0000 7f2bed600640 1 -- 172.21.15.22:0/2602129967 wait complete. 2024-03-28T18:09:04.826 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:04.827+0000 7f2bed600640 1 Processor -- start 2024-03-28T18:09:04.827 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:04.827+0000 7f2bed600640 1 -- start start 2024-03-28T18:09:04.828 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:04.828+0000 7f2bed600640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2be8068880 0x7f2be8197e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:09:04.828 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:04.828+0000 7f2bed600640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2be8069240 0x7f2be81983a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:09:04.828 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:04.828+0000 7f2bed600640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f2be81076b0 0x7f2be819c720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:09:04.828 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:04.828+0000 7f2bed600640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2be81119c0 con 0x7f2be8068880 2024-03-28T18:09:04.828 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:04.828+0000 7f2bed600640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2be8111840 con 0x7f2be81076b0 2024-03-28T18:09:04.828 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:04.828+0000 7f2bed600640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2be8111b40 con 0x7f2be8069240 2024-03-28T18:09:04.829 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:04.828+0000 7f2be6a00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2be8068880 0x7f2be8197e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:09:04.829 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:04.828+0000 7f2be6a00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2be8068880 0x7f2be8197e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:54828/0 (socket says 172.21.15.22:54828) 2024-03-28T18:09:04.829 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:04.828+0000 7f2be6a00640 1 -- 172.21.15.22:0/3333609642 learned_addr learned my addr 172.21.15.22:0/3333609642 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-28T18:09:04.829 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:04.828+0000 7f2be7400640 1 --2- 172.21.15.22:0/3333609642 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f2be81076b0 0x7f2be819c720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:09:04.829 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:04.828+0000 7f2be6000640 1 --2- 172.21.15.22:0/3333609642 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2be8069240 0x7f2be81983a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:09:04.829 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:04.829+0000 7f2be6a00640 1 -- 172.21.15.22:0/3333609642 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2be8069240 msgr2=0x7f2be81983a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:09:04.829 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:04.829+0000 7f2be6a00640 1 --2- 172.21.15.22:0/3333609642 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2be8069240 0x7f2be81983a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:09:04.829 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:04.829+0000 7f2be6a00640 1 -- 172.21.15.22:0/3333609642 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f2be81076b0 msgr2=0x7f2be819c720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:09:04.829 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:04.829+0000 7f2be6a00640 1 --2- 172.21.15.22:0/3333609642 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f2be81076b0 0x7f2be819c720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:09:04.829 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:04.829+0000 7f2be6a00640 1 -- 172.21.15.22:0/3333609642 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2be819ce50 con 0x7f2be8068880 2024-03-28T18:09:04.830 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:04.829+0000 7f2be6a00640 1 --2- 172.21.15.22:0/3333609642 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2be8068880 0x7f2be8197e60 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7f2bd800e9c0 tx=0x7f2bd800ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:09:04.830 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:04.829+0000 7f2bdf400640 1 -- 172.21.15.22:0/3333609642 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2bd800cd80 con 0x7f2be8068880 2024-03-28T18:09:04.830 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:04.829+0000 7f2bdf400640 1 -- 172.21.15.22:0/3333609642 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f2bd80035f0 con 0x7f2be8068880 2024-03-28T18:09:04.830 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:04.829+0000 7f2bed600640 1 -- 172.21.15.22:0/3333609642 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2be819d140 con 0x7f2be8068880 2024-03-28T18:09:04.830 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:04.830+0000 7f2bdf400640 1 -- 172.21.15.22:0/3333609642 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2bd8010690 con 0x7f2be8068880 2024-03-28T18:09:04.830 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:04.830+0000 7f2bed600640 1 -- 172.21.15.22:0/3333609642 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2be81a49d0 con 0x7f2be8068880 2024-03-28T18:09:04.833 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:04.833+0000 7f2bdf400640 1 -- 172.21.15.22:0/3333609642 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 25) v1 ==== 137523+0+0 (secure 0 0 0) 0x7f2bd80037a0 con 0x7f2be8068880 2024-03-28T18:09:04.833 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:04.834+0000 7f2bf0264740 1 Processor -- start 2024-03-28T18:09:04.834 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:04.834+0000 7f2bf0264740 1 -- start start 2024-03-28T18:09:04.834 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:04.834+0000 7f2bf0264740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b2a07edc00 0x55b2a07ec950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:09:04.834 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:04.834+0000 7f2bf0264740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b2a07ecf20 0x55b2a07ed3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:09:04.834 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:04.834+0000 7f2bf0264740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55b2a07fa6a0 0x55b2a07fca90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:09:04.834 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:04.834+0000 7f2bf0264740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b2a0751250 con 0x55b2a07ecf20 2024-03-28T18:09:04.834 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:04.834+0000 7f2bf0264740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b2a06f9a30 con 0x55b2a07fa6a0 2024-03-28T18:09:04.834 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:04.834+0000 7f2bf0264740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b2a074d2f0 con 0x55b2a07edc00 2024-03-28T18:09:04.835 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:04.835+0000 7f2be6a00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b2a07ecf20 0x55b2a07ed3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:09:04.835 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:04.835+0000 7f2be6000640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55b2a07fa6a0 0x55b2a07fca90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:09:04.835 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:04.835+0000 7f2be6a00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b2a07ecf20 0x55b2a07ed3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:54830/0 (socket says 172.21.15.22:54830) 2024-03-28T18:09:04.835 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:04.835+0000 7f2be6a00640 1 -- 172.21.15.22:0/3457896948 learned_addr learned my addr 172.21.15.22:0/3457896948 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-28T18:09:04.836 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:04.835+0000 7f2be6000640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55b2a07fa6a0 0x55b2a07fca90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.22:36062/0 (socket says 172.21.15.22:36062) 2024-03-28T18:09:04.836 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:04.835+0000 7f2be7400640 1 --2- 172.21.15.22:0/3457896948 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b2a07edc00 0x55b2a07ec950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:09:04.836 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:04.835+0000 7f2be6000640 1 -- 172.21.15.22:0/3457896948 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b2a07edc00 msgr2=0x55b2a07ec950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T18:09:04.836 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:04.835+0000 7f2be6000640 1 --2- 172.21.15.22:0/3457896948 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b2a07edc00 0x55b2a07ec950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:09:04.836 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:04.835+0000 7f2be6000640 1 -- 172.21.15.22:0/3457896948 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b2a07ecf20 msgr2=0x55b2a07ed3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T18:09:04.836 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:04.835+0000 7f2be6000640 1 --2- 172.21.15.22:0/3457896948 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b2a07ecf20 0x55b2a07ed3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:09:04.836 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:04.835+0000 7f2be6000640 1 -- 172.21.15.22:0/3457896948 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b2a06e4690 con 0x55b2a07fa6a0 2024-03-28T18:09:04.836 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:04.835+0000 7f2be6a00640 1 --2- 172.21.15.22:0/3457896948 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b2a07ecf20 0x55b2a07ed3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-28T18:09:04.836 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:04.835+0000 7f2bdf400640 1 --2- 172.21.15.22:0/3333609642 >> [v2:172.21.15.22:6800/1357365082,v1:172.21.15.22:6801/1357365082] conn(0x7f2bcc0a2a90 0x7f2bcc0a4f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:09:04.836 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:04.836+0000 7f2bdf400640 1 -- 172.21.15.22:0/3333609642 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 8411+0+0 (secure 0 0 0) 0x7f2bd80cf540 con 0x7f2be8068880 2024-03-28T18:09:04.836 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:04.836+0000 7f2be7400640 1 --2- 172.21.15.22:0/3333609642 >> [v2:172.21.15.22:6800/1357365082,v1:172.21.15.22:6801/1357365082] conn(0x7f2bcc0a2a90 0x7f2bcc0a4f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:09:04.836 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:04.836+0000 7f2be6000640 1 --2- 172.21.15.22:0/3457896948 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55b2a07fa6a0 0x55b2a07fca90 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f2bd4009b40 tx=0x7f2bd40113e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=40701808fcb494a6 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:09:04.837 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:04.836+0000 7f2bd3400640 1 -- 172.21.15.22:0/3457896948 <== mon.2 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2bd4011e10 con 0x55b2a07fa6a0 2024-03-28T18:09:04.838 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:04.836+0000 7f2bd3400640 1 -- 172.21.15.22:0/3457896948 <== mon.2 v2:172.21.15.79:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f2bd4012920 con 0x55b2a07fa6a0 2024-03-28T18:09:04.838 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:04.836+0000 7f2bd3400640 1 -- 172.21.15.22:0/3457896948 <== mon.2 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2bd400fa90 con 0x55b2a07fa6a0 2024-03-28T18:09:04.838 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:04.837+0000 7f2bf0264740 1 -- 172.21.15.22:0/3457896948 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55b2a07fa6a0 msgr2=0x55b2a07fca90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:09:04.838 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:04.837+0000 7f2bf0264740 1 --2- 172.21.15.22:0/3457896948 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55b2a07fa6a0 0x55b2a07fca90 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f2bd4009b40 tx=0x7f2bd40113e0 comp rx=0 tx=0).stop 2024-03-28T18:09:04.838 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:04.837+0000 7f2be7400640 1 --2- 172.21.15.22:0/3333609642 >> [v2:172.21.15.22:6800/1357365082,v1:172.21.15.22:6801/1357365082] conn(0x7f2bcc0a2a90 0x7f2bcc0a4f50 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f2be0004af0 tx=0x7f2be0005c50 comp rx=0 tx=0).ready entity=mgr.15522 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:09:04.838 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:04.837+0000 7f2bf0264740 1 -- 172.21.15.22:0/3457896948 shutdown_connections 2024-03-28T18:09:04.838 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:04.837+0000 7f2bf0264740 1 --2- 172.21.15.22:0/3457896948 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55b2a07fa6a0 0x55b2a07fca90 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:09:04.838 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:04.837+0000 7f2bf0264740 1 --2- 172.21.15.22:0/3457896948 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b2a07ecf20 0x55b2a07ed3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:09:04.838 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:04.837+0000 7f2bf0264740 1 --2- 172.21.15.22:0/3457896948 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b2a07edc00 0x55b2a07ec950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:09:04.838 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:04.837+0000 7f2bf0264740 1 -- 172.21.15.22:0/3457896948 >> 172.21.15.22:0/3457896948 conn(0x55b2a07b4200 msgr2=0x55b2a07e7970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:09:04.838 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:04.837+0000 7f2bf0264740 1 -- 172.21.15.22:0/3457896948 shutdown_connections 2024-03-28T18:09:04.838 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:04.837+0000 7f2bf0264740 1 -- 172.21.15.22:0/3457896948 wait complete. 2024-03-28T18:09:04.838 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:04.838+0000 7f2bf0264740 1 -- start start 2024-03-28T18:09:04.839 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:04.838+0000 7f2bf0264740 1 Processor -- start 2024-03-28T18:09:04.841 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:04.839+0000 7f2bf0264740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b2a07ecf20 0x55b2a07f9d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:09:04.841 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:04.839+0000 7f2bf0264740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b2a07edc00 0x55b2a07f7110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:09:04.842 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:04.839+0000 7f2bf0264740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55b2a07fa6a0 0x55b2a07f7650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:09:04.842 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:04.839+0000 7f2bf0264740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b2a07fd510 con 0x55b2a07ecf20 2024-03-28T18:09:04.842 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:04.839+0000 7f2bf0264740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b2a070ce80 con 0x55b2a07fa6a0 2024-03-28T18:09:04.842 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:04.839+0000 7f2bf0264740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b2a07fd690 con 0x55b2a07edc00 2024-03-28T18:09:04.842 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:04.839+0000 7f2bf0264740 10 client.0 _ll_register_callbacks cb 0x55b2a071f130 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-28T18:09:04.842 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:04.839+0000 7f2be6a00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b2a07ecf20 0x55b2a07f9d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:09:04.842 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:04.839+0000 7f2be7400640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55b2a07fa6a0 0x55b2a07f7650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:09:04.842 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:04.839+0000 7f2be6a00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b2a07ecf20 0x55b2a07f9d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:54834/0 (socket says 172.21.15.22:54834) 2024-03-28T18:09:04.842 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:04.839+0000 7f2be6a00640 1 -- 172.21.15.22:0/3248199392 learned_addr learned my addr 172.21.15.22:0/3248199392 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-28T18:09:04.842 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:04.839+0000 7f2be6000640 1 --2- 172.21.15.22:0/3248199392 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b2a07edc00 0x55b2a07f7110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:09:04.842 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:04.840+0000 7f2be6a00640 1 -- 172.21.15.22:0/3248199392 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b2a07edc00 msgr2=0x55b2a07f7110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T18:09:04.842 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:04.840+0000 7f2be6a00640 1 --2- 172.21.15.22:0/3248199392 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b2a07edc00 0x55b2a07f7110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:09:04.842 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:04.840+0000 7f2be6a00640 1 -- 172.21.15.22:0/3248199392 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55b2a07fa6a0 msgr2=0x55b2a07f7650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T18:09:04.842 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:04.840+0000 7f2be6a00640 1 --2- 172.21.15.22:0/3248199392 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55b2a07fa6a0 0x55b2a07f7650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:09:04.843 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:04.840+0000 7f2be6a00640 1 -- 172.21.15.22:0/3248199392 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55b2a052f7d0 con 0x55b2a07ecf20 2024-03-28T18:09:04.843 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:04.840+0000 7f2be6a00640 1 --2- 172.21.15.22:0/3248199392 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b2a07ecf20 0x55b2a07f9d80 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7f2bd8018000 tx=0x7f2bd8003db0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=53501bee137438da server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:09:04.843 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:04.840+0000 7f2bc7e00640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-03-28T18:09:04.843 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:04.840+0000 7f2bc7e00640 1 -- 172.21.15.22:0/3248199392 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2bd809b500 con 0x55b2a07ecf20 2024-03-28T18:09:04.843 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:04.840+0000 7f2bc7e00640 1 -- 172.21.15.22:0/3248199392 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 8411+0+0 (secure 0 0 0) 0x7f2bd80cce60 con 0x55b2a07ecf20 2024-03-28T18:09:04.843 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:04.840+0000 7f2bf0264740 1 -- 172.21.15.22:0/3248199392 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f2bd4012920 con 0x55b2a07ecf20 2024-03-28T18:09:04.843 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:04.841+0000 7f2bc7e00640 1 -- 172.21.15.22:0/3248199392 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2bd809b320 con 0x55b2a07ecf20 2024-03-28T18:09:04.843 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:04.841+0000 7f2bf0264740 10 client.16053 fetch_fsmap learned FSMap version 27 2024-03-28T18:09:04.843 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:04.841+0000 7f2bf0264740 1 -- 172.21.15.22:0/3248199392 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55b2a04653f0 con 0x55b2a07ecf20 2024-03-28T18:09:04.843 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:04.841+0000 7f2bc7e00640 1 -- 172.21.15.22:0/3248199392 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f2bd4012920 con 0x55b2a07ecf20 2024-03-28T18:09:04.843 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:04.841+0000 7f2bc7e00640 1 -- 172.21.15.22:0/3248199392 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 27) v1 ==== 7934+0+0 (secure 0 0 0) 0x7f2bd80cf080 con 0x55b2a07ecf20 2024-03-28T18:09:04.843 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:04.841+0000 7f2bf0264740 10 client.16053 fetch_fsmap finished waiting for FSMap version 27 2024-03-28T18:09:04.843 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:04.841+0000 7f2bf0264740 10 client.16053 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-28T18:09:04.843 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:04.841+0000 7f2bf0264740 20 client.16053 populate_metadata read hostname 'smithi022' 2024-03-28T18:09:04.843 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:04.841+0000 7f2bf0264740 1 --2- 172.21.15.22:0/3248199392 >> [v2:172.21.15.79:6838/2441542443,v1:172.21.15.79:6839/2441542443] conn(0x55b2a080a610 0x55b2a080ca00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:09:04.844 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:04.841+0000 7f2bf0264740 4 client.16053 mds_command: new command op to 34553 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-28T18:09:04.844 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:04.841+0000 7f2bf0264740 1 -- 172.21.15.22:0/3248199392 --> [v2:172.21.15.79:6838/2441542443,v1:172.21.15.79:6839/2441542443] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55b2a075e790 con 0x55b2a080a610 2024-03-28T18:09:04.844 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:04.842+0000 7f2be6000640 1 --2- 172.21.15.22:0/3248199392 >> [v2:172.21.15.79:6838/2441542443,v1:172.21.15.79:6839/2441542443] conn(0x55b2a080a610 0x55b2a080ca00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:09:04.845 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:04.843+0000 7f2be6000640 1 --2- 172.21.15.22:0/3248199392 >> [v2:172.21.15.79:6838/2441542443,v1:172.21.15.79:6839/2441542443] conn(0x55b2a080a610 0x55b2a080ca00 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7f5d78312587ddb3 server_cookie=1c4c841655b610af in_seq=0 out_seq=0 2024-03-28T18:09:04.845 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:04.843+0000 7f2bc7e00640 10 client.16053 ms_handle_connect on v2:172.21.15.79:6838/2441542443 2024-03-28T18:09:04.845 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:04.843+0000 7f2bc7e00640 1 -- 172.21.15.22:0/3248199392 <== mds.0 v2:172.21.15.79:6838/2441542443 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x55b2a075e790 con 0x55b2a080a610 2024-03-28T18:09:04.845 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:04.843+0000 7f2bc7e00640 10 client.16053 handle_command_reply: tid=0 2024-03-28T18:09:04.845 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:04.844+0000 7f2bf0264740 1 client.16053 shutdown 2024-03-28T18:09:04.845 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:04.844+0000 7f2bf0264740 1 -- 172.21.15.22:0/3248199392 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b2a07ecf20 msgr2=0x55b2a07f9d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:09:04.846 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:04.844+0000 7f2bf0264740 1 --2- 172.21.15.22:0/3248199392 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b2a07ecf20 0x55b2a07f9d80 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7f2bd8018000 tx=0x7f2bd8003db0 comp rx=0 tx=0).stop 2024-03-28T18:09:04.846 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:04.844+0000 7f2bf0264740 1 -- 172.21.15.22:0/3248199392 shutdown_connections 2024-03-28T18:09:04.846 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:04.844+0000 7f2bf0264740 1 --2- 172.21.15.22:0/3248199392 >> [v2:172.21.15.79:6838/2441542443,v1:172.21.15.79:6839/2441542443] conn(0x55b2a080a610 0x55b2a080ca00 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:09:04.846 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:04.844+0000 7f2bf0264740 1 --2- 172.21.15.22:0/3248199392 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55b2a07fa6a0 0x55b2a07f7650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:09:04.846 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:04.844+0000 7f2bf0264740 1 --2- 172.21.15.22:0/3248199392 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b2a07edc00 0x55b2a07f7110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:09:04.846 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:04.844+0000 7f2bf0264740 1 --2- 172.21.15.22:0/3248199392 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b2a07ecf20 0x55b2a07f9d80 unknown :-1 s=CLOSED pgs=586 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:09:04.846 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:04.844+0000 7f2bc7e00640 0 client.16053 ms_handle_reset on v2:172.21.15.79:6838/2441542443 2024-03-28T18:09:04.846 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:04.844+0000 7f2bf0264740 1 -- 172.21.15.22:0/3248199392 >> 172.21.15.22:0/3248199392 conn(0x55b2a07b4200 msgr2=0x55b2a07e96c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:09:04.846 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:04.844+0000 7f2bf0264740 1 -- 172.21.15.22:0/3248199392 shutdown_connections 2024-03-28T18:09:04.846 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:04.844+0000 7f2bf0264740 1 -- 172.21.15.22:0/3248199392 wait complete. 2024-03-28T18:09:04.846 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:04.844+0000 7f2bf0264740 20 client.16053 trim_cache size 0 max 16384 2024-03-28T18:09:04.862 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:04.862+0000 7f2bf0264740 1 Processor -- start 2024-03-28T18:09:04.862 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:04.862+0000 7f2bf0264740 1 -- start start 2024-03-28T18:09:04.862 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:04.862+0000 7f2bf0264740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55b2a07ecf20 0x55b2a07f97d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:09:04.862 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:04.863+0000 7f2bf0264740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b2a07edc00 0x55b2a07f9d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:09:04.862 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:04.863+0000 7f2bf0264740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b2a07fa6a0 0x55b2a07fba90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:09:04.862 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:04.863+0000 7f2bf0264740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2be81aad60 con 0x55b2a07edc00 2024-03-28T18:09:04.863 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:04.863+0000 7f2bf0264740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2be81ac6e0 con 0x55b2a07ecf20 2024-03-28T18:09:04.863 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:04.863+0000 7f2bf0264740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2be81ab3c0 con 0x55b2a07fa6a0 2024-03-28T18:09:04.864 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:04.863+0000 7f2be6000640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b2a07edc00 0x55b2a07f9d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:09:04.864 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:04.863+0000 7f2be6000640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b2a07edc00 0x55b2a07f9d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:54846/0 (socket says 172.21.15.22:54846) 2024-03-28T18:09:04.864 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:04.863+0000 7f2be6000640 1 -- 172.21.15.22:0/3890791987 learned_addr learned my addr 172.21.15.22:0/3890791987 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-28T18:09:04.864 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:04.863+0000 7f2be6a00640 1 --2- 172.21.15.22:0/3890791987 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55b2a07ecf20 0x55b2a07f97d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:09:04.864 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:04.863+0000 7f2be7400640 1 --2- 172.21.15.22:0/3890791987 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b2a07fa6a0 0x55b2a07fba90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:09:04.864 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:04.863+0000 7f2be6000640 1 -- 172.21.15.22:0/3890791987 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b2a07fa6a0 msgr2=0x55b2a07fba90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T18:09:04.864 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:04.863+0000 7f2be6000640 1 --2- 172.21.15.22:0/3890791987 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b2a07fa6a0 0x55b2a07fba90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:09:04.864 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:04.863+0000 7f2be6000640 1 -- 172.21.15.22:0/3890791987 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55b2a07ecf20 msgr2=0x55b2a07f97d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T18:09:04.865 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:04.863+0000 7f2be6000640 1 --2- 172.21.15.22:0/3890791987 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55b2a07ecf20 0x55b2a07f97d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:09:04.865 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:04.863+0000 7f2be6000640 1 -- 172.21.15.22:0/3890791987 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b2a04250f0 con 0x55b2a07edc00 2024-03-28T18:09:04.865 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:04.863+0000 7f2be7400640 1 --2- 172.21.15.22:0/3890791987 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b2a07fa6a0 0x55b2a07fba90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T18:09:04.865 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:04.864+0000 7f2be6000640 1 --2- 172.21.15.22:0/3890791987 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b2a07edc00 0x55b2a07f9d10 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7f2bd4009490 tx=0x7f2bd4009500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=767d74fbf32286a1 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:09:04.865 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:04.864+0000 7f2bd2a00640 1 -- 172.21.15.22:0/3890791987 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2bd402c250 con 0x55b2a07edc00 2024-03-28T18:09:04.865 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:04.864+0000 7f2bd2a00640 1 -- 172.21.15.22:0/3890791987 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f2bd4030070 con 0x55b2a07edc00 2024-03-28T18:09:04.865 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:04.864+0000 7f2bd2a00640 1 -- 172.21.15.22:0/3890791987 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2bd4005330 con 0x55b2a07edc00 2024-03-28T18:09:04.867 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:04.865+0000 7f2bf0264740 1 -- 172.21.15.22:0/3890791987 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b2a07edc00 msgr2=0x55b2a07f9d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:09:04.868 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:04.865+0000 7f2bf0264740 1 --2- 172.21.15.22:0/3890791987 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b2a07edc00 0x55b2a07f9d10 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7f2bd4009490 tx=0x7f2bd4009500 comp rx=0 tx=0).stop 2024-03-28T18:09:04.868 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:04.865+0000 7f2bf0264740 1 -- 172.21.15.22:0/3890791987 shutdown_connections 2024-03-28T18:09:04.868 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:04.865+0000 7f2bf0264740 1 --2- 172.21.15.22:0/3890791987 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b2a07fa6a0 0x55b2a07fba90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:09:04.868 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:04.865+0000 7f2bf0264740 1 --2- 172.21.15.22:0/3890791987 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b2a07edc00 0x55b2a07f9d10 unknown :-1 s=CLOSED pgs=587 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:09:04.868 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:04.865+0000 7f2bf0264740 1 --2- 172.21.15.22:0/3890791987 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55b2a07ecf20 0x55b2a07f97d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:09:04.868 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:04.865+0000 7f2bf0264740 1 -- 172.21.15.22:0/3890791987 >> 172.21.15.22:0/3890791987 conn(0x55b2a07b4200 msgr2=0x55b2a07f59e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:09:04.868 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:04.865+0000 7f2bf0264740 1 -- 172.21.15.22:0/3890791987 shutdown_connections 2024-03-28T18:09:04.868 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:04.865+0000 7f2bf0264740 1 -- 172.21.15.22:0/3890791987 wait complete. 2024-03-28T18:09:04.868 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:04.865+0000 7f2bf0264740 1 -- start start 2024-03-28T18:09:04.868 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:04.866+0000 7f2bf0264740 1 Processor -- start 2024-03-28T18:09:04.868 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:04.866+0000 7f2bf0264740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55b2a07ecf20 0x55b2a07fe950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:09:04.868 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:04.866+0000 7f2bf0264740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b2a07edc00 0x55b2a07fee90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:09:04.868 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:04.866+0000 7f2bf0264740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b2a07fa6a0 0x55b2a085de10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:09:04.869 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:04.866+0000 7f2bf0264740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2be81ac080 con 0x55b2a07edc00 2024-03-28T18:09:04.869 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:04.866+0000 7f2bf0264740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2be81aba20 con 0x55b2a07ecf20 2024-03-28T18:09:04.869 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:04.866+0000 7f2bf0264740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2be81a8f20 con 0x55b2a07fa6a0 2024-03-28T18:09:04.869 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:04.866+0000 7f2bf0264740 10 client.0 _ll_register_callbacks cb 0x7f2bd8003ec0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-28T18:09:04.869 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:04.866+0000 7f2be6000640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b2a07edc00 0x55b2a07fee90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:09:04.869 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:04.866+0000 7f2be6000640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b2a07edc00 0x55b2a07fee90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:54852/0 (socket says 172.21.15.22:54852) 2024-03-28T18:09:04.869 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:04.867+0000 7f2be6000640 1 -- 172.21.15.22:0/332280614 learned_addr learned my addr 172.21.15.22:0/332280614 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-28T18:09:04.869 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:04.867+0000 7f2be6a00640 1 --2- 172.21.15.22:0/332280614 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55b2a07ecf20 0x55b2a07fe950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:09:04.869 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:04.867+0000 7f2be7400640 1 --2- 172.21.15.22:0/332280614 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b2a07fa6a0 0x55b2a085de10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:09:04.869 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:04.867+0000 7f2be6000640 1 -- 172.21.15.22:0/332280614 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b2a07fa6a0 msgr2=0x55b2a085de10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T18:09:04.869 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:04.867+0000 7f2be6000640 1 --2- 172.21.15.22:0/332280614 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b2a07fa6a0 0x55b2a085de10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:09:04.869 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:04.867+0000 7f2be6000640 1 -- 172.21.15.22:0/332280614 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55b2a07ecf20 msgr2=0x55b2a07fe950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T18:09:04.869 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:04.867+0000 7f2be6000640 1 --2- 172.21.15.22:0/332280614 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55b2a07ecf20 0x55b2a07fe950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:09:04.869 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:04.867+0000 7f2be6000640 1 -- 172.21.15.22:0/332280614 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55b2a085e610 con 0x55b2a07edc00 2024-03-28T18:09:04.869 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:04.867+0000 7f2be7400640 1 --2- 172.21.15.22:0/332280614 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b2a07fa6a0 0x55b2a085de10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T18:09:04.870 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:04.867+0000 7f2be6000640 1 --2- 172.21.15.22:0/332280614 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b2a07edc00 0x55b2a07fee90 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7f2bd4009b10 tx=0x7f2bd4009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=70afd74e8e032406 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:09:04.870 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:04.867+0000 7f2bc7e00640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-03-28T18:09:04.870 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:04.868+0000 7f2bc7e00640 1 -- 172.21.15.22:0/332280614 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2bd4038020 con 0x55b2a07edc00 2024-03-28T18:09:04.870 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:04.868+0000 7f2bc7e00640 1 -- 172.21.15.22:0/332280614 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 8411+0+0 (secure 0 0 0) 0x7f2bd4004220 con 0x55b2a07edc00 2024-03-28T18:09:04.870 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:04.868+0000 7f2bf0264740 1 -- 172.21.15.22:0/332280614 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f2bd4030070 con 0x55b2a07edc00 2024-03-28T18:09:04.870 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:04.868+0000 7f2bc7e00640 1 -- 172.21.15.22:0/332280614 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2bd4004460 con 0x55b2a07edc00 2024-03-28T18:09:04.870 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:04.868+0000 7f2bf0264740 10 client.16059 fetch_fsmap learned FSMap version 27 2024-03-28T18:09:04.870 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:04.868+0000 7f2bf0264740 1 -- 172.21.15.22:0/332280614 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55b2a08668f0 con 0x55b2a07edc00 2024-03-28T18:09:04.872 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:04.868+0000 7f2bc7e00640 1 -- 172.21.15.22:0/332280614 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f2bd4030070 con 0x55b2a07edc00 2024-03-28T18:09:04.872 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:04.869+0000 7f2bc7e00640 1 -- 172.21.15.22:0/332280614 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 27) v1 ==== 7934+0+0 (secure 0 0 0) 0x7f2bd400fb10 con 0x55b2a07edc00 2024-03-28T18:09:04.872 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:04.869+0000 7f2bf0264740 10 client.16059 fetch_fsmap finished waiting for FSMap version 27 2024-03-28T18:09:04.872 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:04.869+0000 7f2bf0264740 10 client.16059 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-28T18:09:04.872 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:04.869+0000 7f2bf0264740 20 client.16059 populate_metadata read hostname 'smithi022' 2024-03-28T18:09:04.872 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:04.869+0000 7f2bf0264740 1 --2- 172.21.15.22:0/332280614 >> [v2:172.21.15.79:6838/2441542443,v1:172.21.15.79:6839/2441542443] conn(0x55b2a0866de0 0x55b2a0869200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:09:04.872 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:04.869+0000 7f2bf0264740 4 client.16059 mds_command: new command op to 34553 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-28T18:09:04.872 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:04.869+0000 7f2bf0264740 1 -- 172.21.15.22:0/332280614 --> [v2:172.21.15.79:6838/2441542443,v1:172.21.15.79:6839/2441542443] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55b2a0758ae0 con 0x55b2a0866de0 2024-03-28T18:09:04.872 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:04.869+0000 7f2be6a00640 1 --2- 172.21.15.22:0/332280614 >> [v2:172.21.15.79:6838/2441542443,v1:172.21.15.79:6839/2441542443] conn(0x55b2a0866de0 0x55b2a0869200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:09:04.872 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:04.870+0000 7f2be6a00640 1 --2- 172.21.15.22:0/332280614 >> [v2:172.21.15.79:6838/2441542443,v1:172.21.15.79:6839/2441542443] conn(0x55b2a0866de0 0x55b2a0869200 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3d3112a84cbc6046 server_cookie=476e6a29a63f3c5d in_seq=0 out_seq=0 2024-03-28T18:09:04.872 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:04.870+0000 7f2bc7e00640 10 client.16059 ms_handle_connect on v2:172.21.15.79:6838/2441542443 2024-03-28T18:09:04.873 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:04.870+0000 7f2bc7e00640 1 -- 172.21.15.22:0/332280614 <== mds.0 v2:172.21.15.79:6838/2441542443 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x55b2a0758ae0 con 0x55b2a0866de0 2024-03-28T18:09:04.873 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:04.870+0000 7f2bc7e00640 10 client.16059 handle_command_reply: tid=0 2024-03-28T18:09:04.873 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:04.870+0000 7f2bf0264740 1 client.16059 shutdown 2024-03-28T18:09:04.873 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:04.870+0000 7f2bf0264740 1 -- 172.21.15.22:0/332280614 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b2a07edc00 msgr2=0x55b2a07fee90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:09:04.873 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:04.870+0000 7f2bf0264740 1 --2- 172.21.15.22:0/332280614 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b2a07edc00 0x55b2a07fee90 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7f2bd4009b10 tx=0x7f2bd4009840 comp rx=0 tx=0).stop 2024-03-28T18:09:04.873 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:04.870+0000 7f2bf0264740 1 -- 172.21.15.22:0/332280614 shutdown_connections 2024-03-28T18:09:04.873 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:04.870+0000 7f2bf0264740 1 --2- 172.21.15.22:0/332280614 >> [v2:172.21.15.79:6838/2441542443,v1:172.21.15.79:6839/2441542443] conn(0x55b2a0866de0 0x55b2a0869200 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:09:04.873 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:04.870+0000 7f2bf0264740 1 --2- 172.21.15.22:0/332280614 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b2a07fa6a0 0x55b2a085de10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:09:04.873 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:04.870+0000 7f2bf0264740 1 --2- 172.21.15.22:0/332280614 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b2a07edc00 0x55b2a07fee90 unknown :-1 s=CLOSED pgs=588 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:09:04.873 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:04.870+0000 7f2bf0264740 1 --2- 172.21.15.22:0/332280614 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55b2a07ecf20 0x55b2a07fe950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:09:04.873 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:04.870+0000 7f2bc7e00640 0 client.16059 ms_handle_reset on v2:172.21.15.79:6838/2441542443 2024-03-28T18:09:04.873 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:04.870+0000 7f2bf0264740 1 -- 172.21.15.22:0/332280614 >> 172.21.15.22:0/332280614 conn(0x55b2a07b4200 msgr2=0x55b2a07e8c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:09:04.873 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:04.871+0000 7f2bf0264740 1 -- 172.21.15.22:0/332280614 shutdown_connections 2024-03-28T18:09:04.873 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:04.871+0000 7f2bf0264740 1 -- 172.21.15.22:0/332280614 wait complete. 2024-03-28T18:09:04.873 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:04.871+0000 7f2bf0264740 20 client.16059 trim_cache size 0 max 16384 2024-03-28T18:09:04.874 INFO:teuthology.orchestra.run.smithi022.stdout:{ 2024-03-28T18:09:04.874 INFO:teuthology.orchestra.run.smithi022.stdout: "status": "IDLE (waiting for more scrubs)", 2024-03-28T18:09:04.874 INFO:teuthology.orchestra.run.smithi022.stdout: "scrubs": { 2024-03-28T18:09:04.874 INFO:teuthology.orchestra.run.smithi022.stdout: "d281d108-fad4-4f95-99cc-457b98b3c446": { 2024-03-28T18:09:04.874 INFO:teuthology.orchestra.run.smithi022.stdout: "path": "/", 2024-03-28T18:09:04.874 INFO:teuthology.orchestra.run.smithi022.stdout: "tag": "d281d108-fad4-4f95-99cc-457b98b3c446", 2024-03-28T18:09:04.874 INFO:teuthology.orchestra.run.smithi022.stdout: "options": "recursive,force" 2024-03-28T18:09:04.874 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-03-28T18:09:04.874 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-03-28T18:09:04.874 INFO:teuthology.orchestra.run.smithi022.stdout:} 2024-03-28T18:09:04.875 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:04.871+0000 7f2bed600640 1 -- 172.21.15.22:0/3333609642 >> [v2:172.21.15.22:6800/1357365082,v1:172.21.15.22:6801/1357365082] conn(0x7f2bcc0a2a90 msgr2=0x7f2bcc0a4f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:09:04.875 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:04.871+0000 7f2bed600640 1 --2- 172.21.15.22:0/3333609642 >> [v2:172.21.15.22:6800/1357365082,v1:172.21.15.22:6801/1357365082] conn(0x7f2bcc0a2a90 0x7f2bcc0a4f50 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f2be0004af0 tx=0x7f2be0005c50 comp rx=0 tx=0).stop 2024-03-28T18:09:04.875 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:04.871+0000 7f2bed600640 1 -- 172.21.15.22:0/3333609642 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2be8068880 msgr2=0x7f2be8197e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:09:04.875 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:04.871+0000 7f2bed600640 1 --2- 172.21.15.22:0/3333609642 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2be8068880 0x7f2be8197e60 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7f2bd800e9c0 tx=0x7f2bd800ee80 comp rx=0 tx=0).stop 2024-03-28T18:09:04.875 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:04.872+0000 7f2bed600640 1 -- 172.21.15.22:0/3333609642 shutdown_connections 2024-03-28T18:09:04.875 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:04.872+0000 7f2bed600640 1 --2- 172.21.15.22:0/3333609642 >> [v2:172.21.15.22:6800/1357365082,v1:172.21.15.22:6801/1357365082] conn(0x7f2bcc0a2a90 0x7f2bcc0a4f50 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:09:04.875 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:04.872+0000 7f2bed600640 1 --2- 172.21.15.22:0/3333609642 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f2be81076b0 0x7f2be819c720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:09:04.875 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:04.872+0000 7f2bed600640 1 --2- 172.21.15.22:0/3333609642 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2be8069240 0x7f2be81983a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:09:04.875 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:04.872+0000 7f2bed600640 1 --2- 172.21.15.22:0/3333609642 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2be8068880 0x7f2be8197e60 unknown :-1 s=CLOSED pgs=585 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:09:04.875 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:04.872+0000 7f2bed600640 1 -- 172.21.15.22:0/3333609642 >> 172.21.15.22:0/3333609642 conn(0x7f2be80776a0 msgr2=0x7f2be80ff010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:09:04.876 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:04.872+0000 7f2bed600640 1 -- 172.21.15.22:0/3333609642 shutdown_connections 2024-03-28T18:09:04.876 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:04.872+0000 7f2bed600640 1 -- 172.21.15.22:0/3333609642 wait complete. 2024-03-28T18:09:04.889 INFO:tasks.cephfs.filesystem:scrub status for tag:d281d108-fad4-4f95-99cc-457b98b3c446 - {'path': '/', 'tag': 'd281d108-fad4-4f95-99cc-457b98b3c446', 'options': 'recursive,force'} 2024-03-28T18:09:04.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:09:04 smithi134 ceph-mon[35037]: pgmap v48: 97 pgs: 97 active+clean; 1.3 MiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-28T18:09:04.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:09:04 smithi079 ceph-mon[35312]: pgmap v48: 97 pgs: 97 active+clean; 1.3 MiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-28T18:09:05.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:09:05 smithi022 ceph-mon[31985]: scrub queued for path: / 2024-03-28T18:09:05.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:09:05 smithi022 ceph-mon[31985]: scrub summary: idle+waiting paths [/] 2024-03-28T18:09:05.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:09:05 smithi022 ceph-mon[31985]: scrub summary: active paths [/] 2024-03-28T18:09:05.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:09:05 smithi022 ceph-mon[31985]: scrub summary: idle+waiting paths [/] 2024-03-28T18:09:05.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:09:05 smithi134 ceph-mon[35037]: scrub queued for path: / 2024-03-28T18:09:05.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:09:05 smithi134 ceph-mon[35037]: scrub summary: idle+waiting paths [/] 2024-03-28T18:09:05.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:09:05 smithi134 ceph-mon[35037]: scrub summary: active paths [/] 2024-03-28T18:09:05.925 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:09:05 smithi134 ceph-mon[35037]: scrub summary: idle+waiting paths [/] 2024-03-28T18:09:05.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:09:05 smithi079 ceph-mon[35312]: scrub queued for path: / 2024-03-28T18:09:05.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:09:05 smithi079 ceph-mon[35312]: scrub summary: idle+waiting paths [/] 2024-03-28T18:09:05.933 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:09:05 smithi079 ceph-mon[35312]: scrub summary: active paths [/] 2024-03-28T18:09:05.933 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:09:05 smithi079 ceph-mon[35312]: scrub summary: idle+waiting paths [/] 2024-03-28T18:09:06.781 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:09:06 smithi022 ceph-mon[31985]: pgmap v49: 97 pgs: 97 active+clean; 9.4 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 708 KiB/s wr, 29 op/s 2024-03-28T18:09:06.781 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:09:06 smithi022 ceph-mon[31985]: mgrmap e26: x(active, since 92s), standbys: z, y 2024-03-28T18:09:06.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:09:06 smithi134 ceph-mon[35037]: pgmap v49: 97 pgs: 97 active+clean; 9.4 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 708 KiB/s wr, 29 op/s 2024-03-28T18:09:06.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:09:06 smithi134 ceph-mon[35037]: mgrmap e26: x(active, since 92s), standbys: z, y 2024-03-28T18:09:06.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:09:06 smithi079 ceph-mon[35312]: pgmap v49: 97 pgs: 97 active+clean; 9.4 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 708 KiB/s wr, 29 op/s 2024-03-28T18:09:06.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:09:06 smithi079 ceph-mon[35312]: mgrmap e26: x(active, since 92s), standbys: z, y 2024-03-28T18:09:08.781 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:09:08 smithi022 ceph-mon[31985]: pgmap v50: 97 pgs: 97 active+clean; 9.4 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 708 KiB/s wr, 25 op/s 2024-03-28T18:09:08.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:09:08 smithi134 ceph-mon[35037]: pgmap v50: 97 pgs: 97 active+clean; 9.4 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 708 KiB/s wr, 25 op/s 2024-03-28T18:09:08.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:09:08 smithi079 ceph-mon[35312]: pgmap v50: 97 pgs: 97 active+clean; 9.4 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 708 KiB/s wr, 25 op/s 2024-03-28T18:09:09.781 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:09:09 smithi022 ceph-mon[31985]: scrub summary: idle 2024-03-28T18:09:09.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:09:09 smithi134 ceph-mon[35037]: scrub summary: idle 2024-03-28T18:09:09.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:09:09 smithi079 ceph-mon[35312]: scrub summary: idle 2024-03-28T18:09:10.781 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:09:10 smithi022 ceph-mon[31985]: pgmap v51: 97 pgs: 97 active+clean; 9.4 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 708 KiB/s wr, 25 op/s 2024-03-28T18:09:10.834 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:09:10 smithi079 ceph-mon[35312]: pgmap v51: 97 pgs: 97 active+clean; 9.4 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 708 KiB/s wr, 25 op/s 2024-03-28T18:09:10.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:09:10 smithi134 ceph-mon[35037]: pgmap v51: 97 pgs: 97 active+clean; 9.4 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 708 KiB/s wr, 25 op/s 2024-03-28T18:09:12.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:09:12 smithi022 ceph-mon[31985]: pgmap v52: 97 pgs: 97 active+clean; 41 MiB data, 506 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 3.4 MiB/s wr, 31 op/s 2024-03-28T18:09:12.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:09:12 smithi134 ceph-mon[35037]: pgmap v52: 97 pgs: 97 active+clean; 41 MiB data, 506 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 3.4 MiB/s wr, 31 op/s 2024-03-28T18:09:12.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:09:12 smithi079 ceph-mon[35312]: pgmap v52: 97 pgs: 97 active+clean; 41 MiB data, 506 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 3.4 MiB/s wr, 31 op/s 2024-03-28T18:09:14.534 INFO:tasks.workunit.client.0.smithi022.stderr:+ mkdir untar_tar 2024-03-28T18:09:14.551 INFO:tasks.workunit.client.0.smithi022.stderr:+ cd untar_tar 2024-03-28T18:09:14.557 INFO:tasks.workunit.client.0.smithi022.stderr:+ tar Jxvf ../linux-4.0.5.tar.xz linux-4.0.5/Documentation/ 2024-03-28T18:09:14.602 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/ 2024-03-28T18:09:14.613 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/00-INDEX 2024-03-28T18:09:14.623 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/ABI/ 2024-03-28T18:09:14.636 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/ABI/README 2024-03-28T18:09:14.645 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/ABI/obsolete/ 2024-03-28T18:09:14.655 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/ABI/obsolete/proc-sys-vm-nr_pdflush_threads 2024-03-28T18:09:14.664 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/ABI/obsolete/sysfs-bus-usb 2024-03-28T18:09:14.680 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/ABI/obsolete/sysfs-class-rfkill 2024-03-28T18:09:14.695 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/ABI/obsolete/sysfs-driver-hid-roccat-koneplus 2024-03-28T18:09:14.709 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/ABI/obsolete/sysfs-driver-hid-roccat-kovaplus 2024-03-28T18:09:14.723 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/ABI/obsolete/sysfs-driver-hid-roccat-pyra 2024-03-28T18:09:14.740 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/ABI/removed/ 2024-03-28T18:09:14.754 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/ABI/removed/devfs 2024-03-28T18:09:14.763 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/ABI/removed/dv1394 2024-03-28T18:09:14.779 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/ABI/removed/ip_queue 2024-03-28T18:09:14.781 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:09:14 smithi022 ceph-mon[31985]: pgmap v53: 97 pgs: 97 active+clean; 41 MiB data, 506 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 3.4 MiB/s wr, 26 op/s 2024-03-28T18:09:14.795 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/ABI/removed/net_dma 2024-03-28T18:09:14.808 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/ABI/removed/o2cb 2024-03-28T18:09:14.822 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/ABI/removed/raw1394 2024-03-28T18:09:14.836 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/ABI/removed/video1394 2024-03-28T18:09:14.850 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/ABI/stable/ 2024-03-28T18:09:14.866 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/ABI/stable/firewire-cdev 2024-03-28T18:09:14.876 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/ABI/stable/o2cb 2024-03-28T18:09:14.893 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/ABI/stable/syscalls 2024-03-28T18:09:14.907 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/ABI/stable/sysfs-acpi-pmprofile 2024-03-28T18:09:14.920 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/ABI/stable/sysfs-bus-firewire 2024-03-28T18:09:14.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:09:14 smithi134 ceph-mon[35037]: pgmap v53: 97 pgs: 97 active+clean; 41 MiB data, 506 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 3.4 MiB/s wr, 26 op/s 2024-03-28T18:09:14.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:09:14 smithi079 ceph-mon[35312]: pgmap v53: 97 pgs: 97 active+clean; 41 MiB data, 506 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 3.4 MiB/s wr, 26 op/s 2024-03-28T18:09:14.934 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/ABI/stable/sysfs-bus-usb 2024-03-28T18:09:14.950 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/ABI/stable/sysfs-bus-xen-backend 2024-03-28T18:09:14.963 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/ABI/stable/sysfs-class-backlight 2024-03-28T18:09:14.977 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/ABI/stable/sysfs-class-rfkill 2024-03-28T18:09:14.990 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/ABI/stable/sysfs-class-tpm 2024-03-28T18:09:15.005 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/ABI/stable/sysfs-class-ubi 2024-03-28T18:09:15.020 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/ABI/stable/sysfs-class-udc 2024-03-28T18:09:15.034 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/ABI/stable/sysfs-devices-node 2024-03-28T18:09:15.049 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/ABI/stable/sysfs-devices-system-cpu 2024-03-28T18:09:15.063 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/ABI/stable/sysfs-devices-system-xen_memory 2024-03-28T18:09:15.077 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/ABI/stable/sysfs-driver-ib_srp 2024-03-28T18:09:15.093 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/ABI/stable/sysfs-driver-qla2xxx 2024-03-28T18:09:15.107 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/ABI/stable/sysfs-driver-usb-usbtmc 2024-03-28T18:09:15.121 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/ABI/stable/sysfs-driver-w1_ds28e04 2024-03-28T18:09:15.135 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/ABI/stable/sysfs-firmware-efi-vars 2024-03-28T18:09:15.149 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/ABI/stable/sysfs-firmware-opal-dump 2024-03-28T18:09:15.163 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/ABI/stable/sysfs-firmware-opal-elog 2024-03-28T18:09:15.177 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/ABI/stable/sysfs-module 2024-03-28T18:09:15.192 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/ABI/stable/sysfs-transport-srp 2024-03-28T18:09:15.206 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/ABI/stable/thermal-notification 2024-03-28T18:09:15.219 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/ABI/stable/vdso 2024-03-28T18:09:15.234 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/ABI/testing/ 2024-03-28T18:09:15.249 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/ABI/testing/configfs-spear-pcie-gadget 2024-03-28T18:09:15.257 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/ABI/testing/configfs-usb-gadget 2024-03-28T18:09:15.274 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/ABI/testing/configfs-usb-gadget-acm 2024-03-28T18:09:15.290 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/ABI/testing/configfs-usb-gadget-ecm 2024-03-28T18:09:15.306 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/ABI/testing/configfs-usb-gadget-eem 2024-03-28T18:09:15.320 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/ABI/testing/configfs-usb-gadget-ffs 2024-03-28T18:09:15.335 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/ABI/testing/configfs-usb-gadget-hid 2024-03-28T18:09:15.350 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/ABI/testing/configfs-usb-gadget-loopback 2024-03-28T18:09:15.364 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/ABI/testing/configfs-usb-gadget-mass-storage 2024-03-28T18:09:15.379 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/ABI/testing/configfs-usb-gadget-midi 2024-03-28T18:09:15.393 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/ABI/testing/configfs-usb-gadget-ncm 2024-03-28T18:09:15.407 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/ABI/testing/configfs-usb-gadget-obex 2024-03-28T18:09:15.421 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/ABI/testing/configfs-usb-gadget-phonet 2024-03-28T18:09:15.436 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/ABI/testing/configfs-usb-gadget-rndis 2024-03-28T18:09:15.450 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/ABI/testing/configfs-usb-gadget-serial 2024-03-28T18:09:15.464 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/ABI/testing/configfs-usb-gadget-sourcesink 2024-03-28T18:09:15.478 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/ABI/testing/configfs-usb-gadget-subset 2024-03-28T18:09:15.491 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/ABI/testing/configfs-usb-gadget-uac1 2024-03-28T18:09:15.507 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/ABI/testing/configfs-usb-gadget-uac2 2024-03-28T18:09:15.520 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/ABI/testing/configfs-usb-gadget-uvc 2024-03-28T18:09:15.534 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/ABI/testing/debugfs-driver-genwqe 2024-03-28T18:09:15.547 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/ABI/testing/debugfs-ec 2024-03-28T18:09:15.560 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/ABI/testing/debugfs-ideapad 2024-03-28T18:09:15.574 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/ABI/testing/debugfs-olpc 2024-03-28T18:09:15.588 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/ABI/testing/debugfs-pfo-nx-crypto 2024-03-28T18:09:15.604 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/ABI/testing/debugfs-pktcdvd 2024-03-28T18:09:15.617 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/ABI/testing/dev-kmsg 2024-03-28T18:09:15.631 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/ABI/testing/evm 2024-03-28T18:09:15.644 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/ABI/testing/ima_policy 2024-03-28T18:09:15.658 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/ABI/testing/procfs-diskstats 2024-03-28T18:09:15.673 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/ABI/testing/pstore 2024-03-28T18:09:15.686 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-ata 2024-03-28T18:09:15.700 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-block 2024-03-28T18:09:15.714 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-block-bcache 2024-03-28T18:09:15.729 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-block-dm 2024-03-28T18:09:15.744 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-block-rssd 2024-03-28T18:09:15.759 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-block-zram 2024-03-28T18:09:15.774 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-bus-acpi 2024-03-28T18:09:15.790 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-bus-amba 2024-03-28T18:09:15.805 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-bus-bcma 2024-03-28T18:09:15.819 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-bus-coresight-devices-etb10 2024-03-28T18:09:15.834 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-bus-coresight-devices-etm3x 2024-03-28T18:09:15.849 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-bus-coresight-devices-funnel 2024-03-28T18:09:15.862 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-bus-coresight-devices-tmc 2024-03-28T18:09:15.876 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-bus-css 2024-03-28T18:09:15.890 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-bus-event_source-devices-events 2024-03-28T18:09:15.904 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-bus-event_source-devices-format 2024-03-28T18:09:15.918 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-bus-event_source-devices-hv_24x7 2024-03-28T18:09:15.931 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-bus-event_source-devices-hv_gpci 2024-03-28T18:09:15.945 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-bus-fcoe 2024-03-28T18:09:15.958 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-bus-hsi 2024-03-28T18:09:15.972 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-bus-i2c-devices-fsa9480 2024-03-28T18:09:15.987 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-bus-i2c-devices-hm6352 2024-03-28T18:09:16.001 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-bus-i2c-devices-lm3533 2024-03-28T18:09:16.015 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-bus-iio 2024-03-28T18:09:16.037 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-bus-iio-accel-bmc150 2024-03-28T18:09:16.051 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-bus-iio-frequency-ad9523 2024-03-28T18:09:16.066 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-bus-iio-frequency-adf4350 2024-03-28T18:09:16.081 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-bus-iio-gyro-bmg160 2024-03-28T18:09:16.094 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-bus-iio-light-lm3533-als 2024-03-28T18:09:16.108 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-bus-iio-mpu6050 2024-03-28T18:09:16.122 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-bus-iio-proximity-as3935 2024-03-28T18:09:16.136 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-bus-iio-trigger-sysfs 2024-03-28T18:09:16.150 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-bus-mdio 2024-03-28T18:09:16.165 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-bus-media 2024-03-28T18:09:16.178 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-bus-mei 2024-03-28T18:09:16.192 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-bus-pci 2024-03-28T18:09:16.208 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-bus-pci-devices-cciss 2024-03-28T18:09:16.223 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-bus-pci-drivers-ehci_hcd 2024-03-28T18:09:16.236 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-bus-platform 2024-03-28T18:09:16.252 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-bus-rbd 2024-03-28T18:09:16.267 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-bus-rpmsg 2024-03-28T18:09:16.282 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-bus-umc 2024-03-28T18:09:16.296 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-bus-usb 2024-03-28T18:09:16.312 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-bus-usb-devices-usbsevseg 2024-03-28T18:09:16.326 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-bus-usb-lvstest 2024-03-28T18:09:16.340 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-c2port 2024-03-28T18:09:16.354 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-cfq-target-latency 2024-03-28T18:09:16.367 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-class 2024-03-28T18:09:16.380 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-class-backlight-driver-adp8870 2024-03-28T18:09:16.393 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-class-backlight-driver-lm3533 2024-03-28T18:09:16.410 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-class-bdi 2024-03-28T18:09:16.423 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-class-cxl 2024-03-28T18:09:16.439 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-class-devfreq 2024-03-28T18:09:16.454 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-class-extcon 2024-03-28T18:09:16.468 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-class-iommu 2024-03-28T18:09:16.481 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-class-iommu-amd-iommu 2024-03-28T18:09:16.494 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-class-iommu-intel-iommu 2024-03-28T18:09:16.508 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-class-lcd 2024-03-28T18:09:16.522 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-class-led 2024-03-28T18:09:16.535 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-class-led-driver-lm3533 2024-03-28T18:09:16.549 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-class-leds-gt683r 2024-03-28T18:09:16.563 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-class-mei 2024-03-28T18:09:16.578 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-class-mic.txt 2024-03-28T18:09:16.592 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-class-mtd 2024-03-28T18:09:16.607 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-class-net 2024-03-28T18:09:16.621 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-class-net-batman-adv 2024-03-28T18:09:16.635 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-class-net-cdc_ncm 2024-03-28T18:09:16.648 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-class-net-grcan 2024-03-28T18:09:16.663 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-class-net-mesh 2024-03-28T18:09:16.677 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-class-net-queues 2024-03-28T18:09:16.691 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-class-net-statistics 2024-03-28T18:09:16.707 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-class-pktcdvd 2024-03-28T18:09:16.722 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-class-power 2024-03-28T18:09:16.739 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-class-powercap 2024-03-28T18:09:16.752 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-class-pwm 2024-03-28T18:09:16.767 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-class-rc 2024-03-28T18:09:16.780 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-class-regulator 2024-03-28T18:09:16.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:09:16 smithi022 ceph-mon[31985]: pgmap v54: 97 pgs: 97 active+clean; 69 MiB data, 594 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 5.7 MiB/s wr, 32 op/s 2024-03-28T18:09:16.796 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-class-rtc-rtc0-device-rtc_calibration 2024-03-28T18:09:16.809 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-class-scsi_host 2024-03-28T18:09:16.823 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-class-uwb_rc 2024-03-28T18:09:16.836 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-class-uwb_rc-wusbhc 2024-03-28T18:09:16.850 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-dev 2024-03-28T18:09:16.864 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-devices 2024-03-28T18:09:16.878 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-devices-edac 2024-03-28T18:09:16.894 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-devices-firmware_node 2024-03-28T18:09:16.909 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-devices-lpss_ltr 2024-03-28T18:09:16.923 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-devices-memory 2024-03-28T18:09:16.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:09:16 smithi134 ceph-mon[35037]: pgmap v54: 97 pgs: 97 active+clean; 69 MiB data, 594 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 5.7 MiB/s wr, 32 op/s 2024-03-28T18:09:16.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:09:16 smithi079 ceph-mon[35312]: pgmap v54: 97 pgs: 97 active+clean; 69 MiB data, 594 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 5.7 MiB/s wr, 32 op/s 2024-03-28T18:09:16.937 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-devices-mmc 2024-03-28T18:09:16.951 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-devices-online 2024-03-28T18:09:16.966 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-devices-platform-_UDC_-gadget 2024-03-28T18:09:16.980 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-devices-platform-docg3 2024-03-28T18:09:16.994 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-devices-platform-sh_mobile_lcdc_fb 2024-03-28T18:09:17.009 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-devices-power 2024-03-28T18:09:17.024 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-devices-power_resources_D0 2024-03-28T18:09:17.038 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-devices-power_resources_D1 2024-03-28T18:09:17.052 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-devices-power_resources_D2 2024-03-28T18:09:17.069 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-devices-power_resources_D3hot 2024-03-28T18:09:17.083 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-devices-power_resources_wakeup 2024-03-28T18:09:17.099 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-devices-power_state 2024-03-28T18:09:17.115 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-devices-real_power_state 2024-03-28T18:09:17.130 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-devices-resource_in_use 2024-03-28T18:09:17.143 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-devices-soc 2024-03-28T18:09:17.158 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-devices-sun 2024-03-28T18:09:17.172 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-devices-system-cpu 2024-03-28T18:09:17.186 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-devices-system-ibm-rtl 2024-03-28T18:09:17.201 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-devices-system-xen_cpu 2024-03-28T18:09:17.215 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-driver-genwqe 2024-03-28T18:09:17.228 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-driver-hid 2024-03-28T18:09:17.243 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-driver-hid-lenovo 2024-03-28T18:09:17.260 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-driver-hid-logitech-lg4ff 2024-03-28T18:09:17.275 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-driver-hid-multitouch 2024-03-28T18:09:17.289 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-driver-hid-picolcd 2024-03-28T18:09:17.303 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-driver-hid-prodikeys 2024-03-28T18:09:17.318 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-driver-hid-roccat-arvo 2024-03-28T18:09:17.334 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-driver-hid-roccat-isku 2024-03-28T18:09:17.350 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-driver-hid-roccat-kone 2024-03-28T18:09:17.365 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-driver-hid-roccat-koneplus 2024-03-28T18:09:17.380 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-driver-hid-roccat-konepure 2024-03-28T18:09:17.394 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-driver-hid-roccat-kovaplus 2024-03-28T18:09:17.407 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-driver-hid-roccat-lua 2024-03-28T18:09:17.422 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-driver-hid-roccat-pyra 2024-03-28T18:09:17.469 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-driver-hid-roccat-ryos 2024-03-28T18:09:17.484 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-driver-hid-roccat-savu 2024-03-28T18:09:17.499 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-driver-hid-srws1 2024-03-28T18:09:17.515 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-driver-hid-wiimote 2024-03-28T18:09:17.530 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-driver-input-axp-pek 2024-03-28T18:09:17.542 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-driver-intel-rapid-start 2024-03-28T18:09:17.556 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-driver-pciback 2024-03-28T18:09:17.570 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-driver-ppi 2024-03-28T18:09:17.584 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-driver-samsung-laptop 2024-03-28T18:09:17.599 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-driver-sunxi-sid 2024-03-28T18:09:17.613 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-driver-tegra-fuse 2024-03-28T18:09:17.627 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-driver-toshiba_acpi 2024-03-28T18:09:17.642 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-driver-wacom 2024-03-28T18:09:17.656 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-driver-xen-blkback 2024-03-28T18:09:17.670 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-driver-xen-blkfront 2024-03-28T18:09:17.685 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-firmware-acpi 2024-03-28T18:09:17.698 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-firmware-dmi 2024-03-28T18:09:17.714 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-firmware-efi 2024-03-28T18:09:17.728 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-firmware-efi-runtime-map 2024-03-28T18:09:17.743 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-firmware-gsmi 2024-03-28T18:09:17.757 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-firmware-log 2024-03-28T18:09:17.771 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-firmware-memmap 2024-03-28T18:09:17.785 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-firmware-ofw 2024-03-28T18:09:17.800 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-firmware-sfi 2024-03-28T18:09:17.814 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-firmware-sgi_uv 2024-03-28T18:09:17.830 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-fs-ext4 2024-03-28T18:09:17.845 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-fs-f2fs 2024-03-28T18:09:17.860 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-fs-nilfs2 2024-03-28T18:09:17.874 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-fs-xfs 2024-03-28T18:09:17.888 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-gpio 2024-03-28T18:09:17.903 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-i2c-bmp085 2024-03-28T18:09:17.918 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-ibft 2024-03-28T18:09:17.931 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-kernel-boot_params 2024-03-28T18:09:17.946 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-kernel-fscaps 2024-03-28T18:09:17.961 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-kernel-iommu_groups 2024-03-28T18:09:17.976 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-kernel-livepatch 2024-03-28T18:09:17.990 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-kernel-mm 2024-03-28T18:09:18.006 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-kernel-mm-hugepages 2024-03-28T18:09:18.020 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-kernel-mm-ksm 2024-03-28T18:09:18.034 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-kernel-slab 2024-03-28T18:09:18.051 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-kernel-uids 2024-03-28T18:09:18.066 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-kernel-vmcoreinfo 2024-03-28T18:09:18.080 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-memory-page-offline 2024-03-28T18:09:18.093 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-module 2024-03-28T18:09:18.108 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-ocfs2 2024-03-28T18:09:18.122 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-platform-asus-laptop 2024-03-28T18:09:18.137 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-platform-asus-wmi 2024-03-28T18:09:18.150 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-platform-at91 2024-03-28T18:09:18.163 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-platform-brcmstb-gisb-arb 2024-03-28T18:09:18.177 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-platform-chipidea-usb-otg 2024-03-28T18:09:18.191 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-platform-eeepc-laptop 2024-03-28T18:09:18.203 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-platform-ideapad-laptop 2024-03-28T18:09:18.218 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-platform-kim 2024-03-28T18:09:18.232 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-platform-msi-laptop 2024-03-28T18:09:18.247 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-platform-tahvo-usb 2024-03-28T18:09:18.261 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-platform-ts5500 2024-03-28T18:09:18.275 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-power 2024-03-28T18:09:18.292 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-pps 2024-03-28T18:09:18.307 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-profiling 2024-03-28T18:09:18.321 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-ptp 2024-03-28T18:09:18.337 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-tty 2024-03-28T18:09:18.350 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-wusb_cbaf 2024-03-28T18:09:18.364 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/BUG-HUNTING 2024-03-28T18:09:18.387 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/Changes 2024-03-28T18:09:18.404 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/CodeOfConflict 2024-03-28T18:09:18.419 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/CodingStyle 2024-03-28T18:09:18.434 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/DMA-API-HOWTO.txt 2024-03-28T18:09:18.452 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/DMA-API.txt 2024-03-28T18:09:18.467 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/DMA-ISA-LPC.txt 2024-03-28T18:09:18.479 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/DMA-attributes.txt 2024-03-28T18:09:18.493 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/DocBook/ 2024-03-28T18:09:18.502 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/DocBook/.gitignore 2024-03-28T18:09:18.510 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/DocBook/80211.tmpl 2024-03-28T18:09:18.528 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/DocBook/Makefile 2024-03-28T18:09:18.543 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/DocBook/alsa-driver-api.tmpl 2024-03-28T18:09:18.556 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/DocBook/crypto-API.tmpl 2024-03-28T18:09:18.576 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/DocBook/debugobjects.tmpl 2024-03-28T18:09:18.592 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/DocBook/device-drivers.tmpl 2024-03-28T18:09:18.606 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/DocBook/deviceiobook.tmpl 2024-03-28T18:09:18.621 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/DocBook/drm.tmpl 2024-03-28T18:09:18.657 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/DocBook/filesystems.tmpl 2024-03-28T18:09:18.672 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/DocBook/gadget.tmpl 2024-03-28T18:09:18.689 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/DocBook/genericirq.tmpl 2024-03-28T18:09:18.703 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/DocBook/kernel-api.tmpl 2024-03-28T18:09:18.719 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/DocBook/kernel-hacking.tmpl 2024-03-28T18:09:18.737 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/DocBook/kernel-locking.tmpl 2024-03-28T18:09:18.757 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/DocBook/kgdb.tmpl 2024-03-28T18:09:18.775 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/DocBook/libata.tmpl 2024-03-28T18:09:18.781 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:09:18 smithi022 ceph-mon[31985]: pgmap v55: 97 pgs: 97 active+clean; 69 MiB data, 594 MiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 5.0 MiB/s wr, 10 op/s 2024-03-28T18:09:18.793 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/DocBook/librs.tmpl 2024-03-28T18:09:18.807 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/DocBook/lsm.tmpl 2024-03-28T18:09:18.821 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/DocBook/media/ 2024-03-28T18:09:18.831 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/DocBook/media/Makefile 2024-03-28T18:09:18.843 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/DocBook/media/bayer.png.b64 2024-03-28T18:09:18.863 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/DocBook/media/constraints.png.b64 2024-03-28T18:09:18.881 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/DocBook/media/crop.gif.b64 2024-03-28T18:09:18.894 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/DocBook/media/dvb/ 2024-03-28T18:09:18.906 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/DocBook/media/dvb/.gitignore 2024-03-28T18:09:18.915 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/DocBook/media/dvb/audio.xml 2024-03-28T18:09:18.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:09:18 smithi134 ceph-mon[35037]: pgmap v55: 97 pgs: 97 active+clean; 69 MiB data, 594 MiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 5.0 MiB/s wr, 10 op/s 2024-03-28T18:09:18.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:09:18 smithi079 ceph-mon[35312]: pgmap v55: 97 pgs: 97 active+clean; 69 MiB data, 594 MiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 5.0 MiB/s wr, 10 op/s 2024-03-28T18:09:18.934 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/DocBook/media/dvb/ca.xml 2024-03-28T18:09:18.952 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/DocBook/media/dvb/demux.xml 2024-03-28T18:09:18.972 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/DocBook/media/dvb/dvbapi.xml 2024-03-28T18:09:18.987 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/DocBook/media/dvb/dvbproperty.xml 2024-03-28T18:09:19.008 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/DocBook/media/dvb/dvbstb.pdf 2024-03-28T18:09:19.022 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/DocBook/media/dvb/examples.xml 2024-03-28T18:09:19.038 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/DocBook/media/dvb/frontend.xml 2024-03-28T18:09:19.059 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/DocBook/media/dvb/intro.xml 2024-03-28T18:09:19.076 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/DocBook/media/dvb/kdapi.xml 2024-03-28T18:09:19.100 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/DocBook/media/dvb/net.xml 2024-03-28T18:09:19.115 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/DocBook/media/dvb/video.xml 2024-03-28T18:09:19.137 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/DocBook/media/dvbstb.png.b64 2024-03-28T18:09:19.162 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/DocBook/media/fieldseq_bt.gif.b64 2024-03-28T18:09:19.180 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/DocBook/media/fieldseq_tb.gif.b64 2024-03-28T18:09:19.200 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/DocBook/media/nv12mt.gif.b64 2024-03-28T18:09:19.216 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/DocBook/media/nv12mt_example.gif.b64 2024-03-28T18:09:19.230 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/DocBook/media/pipeline.png.b64 2024-03-28T18:09:19.247 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/DocBook/media/selection.png.b64 2024-03-28T18:09:19.264 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/ 2024-03-28T18:09:19.275 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/.gitignore 2024-03-28T18:09:19.285 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/biblio.xml 2024-03-28T18:09:19.300 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/capture.c.xml 2024-03-28T18:09:19.318 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/common.xml 2024-03-28T18:09:19.337 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/compat.xml 2024-03-28T18:09:19.367 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/controls.xml 2024-03-28T18:09:19.408 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/crop.pdf 2024-03-28T18:09:19.424 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/dev-capture.xml 2024-03-28T18:09:19.439 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/dev-codec.xml 2024-03-28T18:09:19.453 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/dev-effect.xml 2024-03-28T18:09:19.469 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/dev-event.xml 2024-03-28T18:09:19.483 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/dev-osd.xml 2024-03-28T18:09:19.497 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/dev-output.xml 2024-03-28T18:09:19.512 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/dev-overlay.xml 2024-03-28T18:09:19.529 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/dev-radio.xml 2024-03-28T18:09:19.544 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/dev-raw-vbi.xml 2024-03-28T18:09:19.561 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/dev-rds.xml 2024-03-28T18:09:19.576 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/dev-sdr.xml 2024-03-28T18:09:19.592 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/dev-sliced-vbi.xml 2024-03-28T18:09:19.611 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/dev-subdev.xml 2024-03-28T18:09:19.630 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/dev-teletext.xml 2024-03-28T18:09:19.645 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/driver.xml 2024-03-28T18:09:19.661 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/fdl-appendix.xml 2024-03-28T18:09:19.681 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/fieldseq_bt.pdf 2024-03-28T18:09:19.695 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/fieldseq_tb.pdf 2024-03-28T18:09:19.709 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/func-close.xml 2024-03-28T18:09:19.724 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/func-ioctl.xml 2024-03-28T18:09:19.738 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/func-mmap.xml 2024-03-28T18:09:19.755 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/func-munmap.xml 2024-03-28T18:09:19.772 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/func-open.xml 2024-03-28T18:09:19.787 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/func-poll.xml 2024-03-28T18:09:19.802 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/func-read.xml 2024-03-28T18:09:19.816 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/func-select.xml 2024-03-28T18:09:19.829 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/func-write.xml 2024-03-28T18:09:19.845 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/gen-errors.xml 2024-03-28T18:09:19.859 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/io.xml 2024-03-28T18:09:19.880 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/keytable.c.xml 2024-03-28T18:09:19.896 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/libv4l.xml 2024-03-28T18:09:19.910 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/lirc_device_interface.xml 2024-03-28T18:09:19.927 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/media-controller.xml 2024-03-28T18:09:19.942 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/media-func-close.xml 2024-03-28T18:09:19.957 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/media-func-ioctl.xml 2024-03-28T18:09:19.974 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/media-func-open.xml 2024-03-28T18:09:19.989 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/media-ioc-device-info.xml 2024-03-28T18:09:20.005 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/media-ioc-enum-entities.xml 2024-03-28T18:09:20.022 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/media-ioc-enum-links.xml 2024-03-28T18:09:20.039 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/media-ioc-setup-link.xml 2024-03-28T18:09:20.054 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/pipeline.pdf 2024-03-28T18:09:20.071 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/pixfmt-grey.xml 2024-03-28T18:09:20.086 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/pixfmt-m420.xml 2024-03-28T18:09:20.101 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/pixfmt-nv12.xml 2024-03-28T18:09:20.118 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/pixfmt-nv12m.xml 2024-03-28T18:09:20.183 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/pixfmt-nv12mt.xml 2024-03-28T18:09:20.200 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/pixfmt-nv16.xml 2024-03-28T18:09:20.215 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/pixfmt-nv16m.xml 2024-03-28T18:09:20.231 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/pixfmt-nv24.xml 2024-03-28T18:09:20.246 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/pixfmt-packed-rgb.xml 2024-03-28T18:09:20.264 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/pixfmt-packed-yuv.xml 2024-03-28T18:09:20.280 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/pixfmt-sbggr16.xml 2024-03-28T18:09:20.297 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/pixfmt-sbggr8.xml 2024-03-28T18:09:20.313 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/pixfmt-sdr-cs08.xml 2024-03-28T18:09:20.328 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/pixfmt-sdr-cs14le.xml 2024-03-28T18:09:20.343 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/pixfmt-sdr-cu08.xml 2024-03-28T18:09:20.358 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/pixfmt-sdr-cu16le.xml 2024-03-28T18:09:20.374 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/pixfmt-sdr-ru12le.xml 2024-03-28T18:09:20.390 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/pixfmt-sgbrg8.xml 2024-03-28T18:09:20.405 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/pixfmt-sgrbg8.xml 2024-03-28T18:09:20.421 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/pixfmt-srggb10.xml 2024-03-28T18:09:20.437 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/pixfmt-srggb10alaw8.xml 2024-03-28T18:09:20.453 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/pixfmt-srggb10dpcm8.xml 2024-03-28T18:09:20.468 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/pixfmt-srggb10p.xml 2024-03-28T18:09:20.484 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/pixfmt-srggb12.xml 2024-03-28T18:09:20.499 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/pixfmt-srggb8.xml 2024-03-28T18:09:20.517 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/pixfmt-uv8.xml 2024-03-28T18:09:20.535 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/pixfmt-uyvy.xml 2024-03-28T18:09:20.549 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/pixfmt-vyuy.xml 2024-03-28T18:09:20.565 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/pixfmt-y10.xml 2024-03-28T18:09:20.582 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/pixfmt-y10b.xml 2024-03-28T18:09:20.596 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/pixfmt-y12.xml 2024-03-28T18:09:20.611 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/pixfmt-y16.xml 2024-03-28T18:09:20.626 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/pixfmt-y41p.xml 2024-03-28T18:09:20.641 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/pixfmt-yuv410.xml 2024-03-28T18:09:20.654 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/pixfmt-yuv411p.xml 2024-03-28T18:09:20.669 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/pixfmt-yuv420.xml 2024-03-28T18:09:20.685 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/pixfmt-yuv420m.xml 2024-03-28T18:09:20.701 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/pixfmt-yuv422p.xml 2024-03-28T18:09:20.715 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/pixfmt-yuyv.xml 2024-03-28T18:09:20.731 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/pixfmt-yvu420m.xml 2024-03-28T18:09:20.746 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/pixfmt-yvyu.xml 2024-03-28T18:09:20.761 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/pixfmt.xml 2024-03-28T18:09:20.781 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:09:20 smithi022 ceph-mon[31985]: pgmap v56: 97 pgs: 97 active+clean; 69 MiB data, 594 MiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 5.0 MiB/s wr, 10 op/s 2024-03-28T18:09:20.786 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/planar-apis.xml 2024-03-28T18:09:20.801 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/remote_controllers.xml 2024-03-28T18:09:20.818 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/selection-api.xml 2024-03-28T18:09:20.835 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/selections-common.xml 2024-03-28T18:09:20.851 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/subdev-formats.xml 2024-03-28T18:09:20.883 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/subdev-image-processing-crop.dia 2024-03-28T18:09:20.904 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/subdev-image-processing-crop.svg 2024-03-28T18:09:20.919 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/subdev-image-processing-full.dia 2024-03-28T18:09:20.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:09:20 smithi134 ceph-mon[35037]: pgmap v56: 97 pgs: 97 active+clean; 69 MiB data, 594 MiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 5.0 MiB/s wr, 10 op/s 2024-03-28T18:09:20.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:09:20 smithi079 ceph-mon[35312]: pgmap v56: 97 pgs: 97 active+clean; 69 MiB data, 594 MiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 5.0 MiB/s wr, 10 op/s 2024-03-28T18:09:20.940 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/subdev-image-processing-full.svg 2024-03-28T18:09:20.956 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/subdev-image-processing-scaling-multi-source.dia 2024-03-28T18:09:20.977 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/subdev-image-processing-scaling-multi-source.svg 2024-03-28T18:09:20.993 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/v4l2.xml 2024-03-28T18:09:21.009 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/v4l2grab.c.xml 2024-03-28T18:09:21.025 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/vbi_525.pdf 2024-03-28T18:09:21.040 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/vbi_625.pdf 2024-03-28T18:09:21.055 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/vbi_hsync.pdf 2024-03-28T18:09:21.070 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/vidioc-create-bufs.xml 2024-03-28T18:09:21.087 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/vidioc-cropcap.xml 2024-03-28T18:09:21.103 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/vidioc-dbg-g-chip-info.xml 2024-03-28T18:09:21.123 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/vidioc-dbg-g-register.xml 2024-03-28T18:09:21.171 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/vidioc-decoder-cmd.xml 2024-03-28T18:09:21.205 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/vidioc-dqevent.xml 2024-03-28T18:09:21.224 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/vidioc-dv-timings-cap.xml 2024-03-28T18:09:21.241 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/vidioc-encoder-cmd.xml 2024-03-28T18:09:21.258 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/vidioc-enum-dv-timings.xml 2024-03-28T18:09:21.273 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/vidioc-enum-fmt.xml 2024-03-28T18:09:21.290 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/vidioc-enum-frameintervals.xml 2024-03-28T18:09:21.307 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/vidioc-enum-framesizes.xml 2024-03-28T18:09:21.323 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/vidioc-enum-freq-bands.xml 2024-03-28T18:09:21.337 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/vidioc-enumaudio.xml 2024-03-28T18:09:21.353 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/vidioc-enumaudioout.xml 2024-03-28T18:09:21.368 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/vidioc-enuminput.xml 2024-03-28T18:09:21.384 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/vidioc-enumoutput.xml 2024-03-28T18:09:21.399 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/vidioc-enumstd.xml 2024-03-28T18:09:21.415 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/vidioc-expbuf.xml 2024-03-28T18:09:21.430 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/vidioc-g-audio.xml 2024-03-28T18:09:21.447 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/vidioc-g-audioout.xml 2024-03-28T18:09:21.462 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/vidioc-g-crop.xml 2024-03-28T18:09:21.476 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/vidioc-g-ctrl.xml 2024-03-28T18:09:21.491 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/vidioc-g-dv-timings.xml 2024-03-28T18:09:21.508 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/vidioc-g-edid.xml 2024-03-28T18:09:21.525 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/vidioc-g-enc-index.xml 2024-03-28T18:09:21.543 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/vidioc-g-ext-ctrls.xml 2024-03-28T18:09:21.561 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/vidioc-g-fbuf.xml 2024-03-28T18:09:21.578 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/vidioc-g-fmt.xml 2024-03-28T18:09:21.593 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/vidioc-g-frequency.xml 2024-03-28T18:09:21.607 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/vidioc-g-input.xml 2024-03-28T18:09:21.623 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/vidioc-g-jpegcomp.xml 2024-03-28T18:09:21.637 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/vidioc-g-modulator.xml 2024-03-28T18:09:21.653 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/vidioc-g-output.xml 2024-03-28T18:09:21.668 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/vidioc-g-parm.xml 2024-03-28T18:09:21.684 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/vidioc-g-priority.xml 2024-03-28T18:09:21.699 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/vidioc-g-selection.xml 2024-03-28T18:09:21.717 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/vidioc-g-sliced-vbi-cap.xml 2024-03-28T18:09:21.735 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/vidioc-g-std.xml 2024-03-28T18:09:21.749 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/vidioc-g-tuner.xml 2024-03-28T18:09:21.766 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/vidioc-log-status.xml 2024-03-28T18:09:21.781 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/vidioc-overlay.xml 2024-03-28T18:09:21.796 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/vidioc-prepare-buf.xml 2024-03-28T18:09:21.811 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/vidioc-qbuf.xml 2024-03-28T18:09:21.827 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/vidioc-query-dv-timings.xml 2024-03-28T18:09:21.842 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/vidioc-querybuf.xml 2024-03-28T18:09:21.884 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/vidioc-querycap.xml 2024-03-28T18:09:21.902 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/vidioc-queryctrl.xml 2024-03-28T18:09:21.919 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/vidioc-querystd.xml 2024-03-28T18:09:21.935 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/vidioc-reqbufs.xml 2024-03-28T18:09:21.951 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/vidioc-s-hw-freq-seek.xml 2024-03-28T18:09:21.967 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/vidioc-streamon.xml 2024-03-28T18:09:21.982 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/vidioc-subdev-enum-frame-interval.xml 2024-03-28T18:09:21.997 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/vidioc-subdev-enum-frame-size.xml 2024-03-28T18:09:22.013 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/vidioc-subdev-enum-mbus-code.xml 2024-03-28T18:09:22.028 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/vidioc-subdev-g-crop.xml 2024-03-28T18:09:22.045 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/vidioc-subdev-g-fmt.xml 2024-03-28T18:09:22.062 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/vidioc-subdev-g-frame-interval.xml 2024-03-28T18:09:22.078 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/vidioc-subdev-g-selection.xml 2024-03-28T18:09:22.091 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/vidioc-subscribe-event.xml 2024-03-28T18:09:22.106 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/DocBook/media/vbi_525.gif.b64 2024-03-28T18:09:22.176 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/DocBook/media/vbi_625.gif.b64 2024-03-28T18:09:22.220 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/DocBook/media/vbi_hsync.gif.b64 2024-03-28T18:09:22.235 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/DocBook/media_api.tmpl 2024-03-28T18:09:22.253 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/DocBook/mtdnand.tmpl 2024-03-28T18:09:22.273 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/DocBook/networking.tmpl 2024-03-28T18:09:22.287 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/DocBook/rapidio.tmpl 2024-03-28T18:09:22.301 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/DocBook/regulator.tmpl 2024-03-28T18:09:22.315 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/DocBook/s390-drivers.tmpl 2024-03-28T18:09:22.330 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/DocBook/scsi.tmpl 2024-03-28T18:09:22.344 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/DocBook/sh.tmpl 2024-03-28T18:09:22.358 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/DocBook/stylesheet.xsl 2024-03-28T18:09:22.372 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/DocBook/tracepoint.tmpl 2024-03-28T18:09:22.385 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/DocBook/uio-howto.tmpl 2024-03-28T18:09:22.404 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/DocBook/usb.tmpl 2024-03-28T18:09:22.422 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/DocBook/w1.tmpl 2024-03-28T18:09:22.436 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/DocBook/writing-an-alsa-driver.tmpl 2024-03-28T18:09:22.479 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/DocBook/writing_musb_glue_layer.tmpl 2024-03-28T18:09:22.496 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/DocBook/writing_usb_driver.tmpl 2024-03-28T18:09:22.513 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/DocBook/z8530book.tmpl 2024-03-28T18:09:22.531 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/EDID/ 2024-03-28T18:09:22.545 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/EDID/1024x768.S 2024-03-28T18:09:22.553 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/EDID/1280x1024.S 2024-03-28T18:09:22.569 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/EDID/1600x1200.S 2024-03-28T18:09:22.582 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/EDID/1680x1050.S 2024-03-28T18:09:22.595 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/EDID/1920x1080.S 2024-03-28T18:09:22.608 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/EDID/800x600.S 2024-03-28T18:09:22.622 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/EDID/HOWTO.txt 2024-03-28T18:09:22.637 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/EDID/Makefile 2024-03-28T18:09:22.652 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/EDID/edid.S 2024-03-28T18:09:22.667 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/EDID/hex 2024-03-28T18:09:22.679 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/HOWTO 2024-03-28T18:09:22.699 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/IPMI.txt 2024-03-28T18:09:22.716 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/IRQ-affinity.txt 2024-03-28T18:09:22.730 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/IRQ-domain.txt 2024-03-28T18:09:22.744 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/IRQ.txt 2024-03-28T18:09:22.757 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/Intel-IOMMU.txt 2024-03-28T18:09:22.772 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/Makefile 2024-03-28T18:09:22.781 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:09:22 smithi022 ceph-mon[31985]: pgmap v57: 97 pgs: 97 active+clean; 82 MiB data, 644 MiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 6.1 MiB/s wr, 67 op/s 2024-03-28T18:09:22.784 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/ManagementStyle 2024-03-28T18:09:22.799 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/PCI/ 2024-03-28T18:09:22.809 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/PCI/00-INDEX 2024-03-28T18:09:22.817 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/PCI/MSI-HOWTO.txt 2024-03-28T18:09:22.834 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/PCI/PCIEBUS-HOWTO.txt 2024-03-28T18:09:22.849 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/PCI/pci-error-recovery.txt 2024-03-28T18:09:22.865 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/PCI/pci-iov-howto.txt 2024-03-28T18:09:22.881 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/PCI/pci.txt 2024-03-28T18:09:22.897 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/PCI/pcieaer-howto.txt 2024-03-28T18:09:22.912 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/RCU/ 2024-03-28T18:09:22.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:09:22 smithi134 ceph-mon[35037]: pgmap v57: 97 pgs: 97 active+clean; 82 MiB data, 644 MiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 6.1 MiB/s wr, 67 op/s 2024-03-28T18:09:22.927 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/RCU/00-INDEX 2024-03-28T18:09:22.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:09:22 smithi079 ceph-mon[35312]: pgmap v57: 97 pgs: 97 active+clean; 82 MiB data, 644 MiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 6.1 MiB/s wr, 67 op/s 2024-03-28T18:09:22.935 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/RCU/NMI-RCU.txt 2024-03-28T18:09:22.951 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/RCU/RTFP.txt 2024-03-28T18:09:22.975 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/RCU/UP.txt 2024-03-28T18:09:22.990 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/RCU/arrayRCU.txt 2024-03-28T18:09:23.003 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/RCU/checklist.txt 2024-03-28T18:09:23.019 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/RCU/listRCU.txt 2024-03-28T18:09:23.034 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/RCU/lockdep-splat.txt 2024-03-28T18:09:23.046 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/RCU/lockdep.txt 2024-03-28T18:09:23.061 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/RCU/rcu.txt 2024-03-28T18:09:23.074 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/RCU/rcu_dereference.txt 2024-03-28T18:09:23.090 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/RCU/rcubarrier.txt 2024-03-28T18:09:23.104 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/RCU/rculist_nulls.txt 2024-03-28T18:09:23.119 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/RCU/rcuref.txt 2024-03-28T18:09:23.142 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/RCU/stallwarn.txt 2024-03-28T18:09:23.207 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/RCU/torture.txt 2024-03-28T18:09:23.237 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/RCU/trace.txt 2024-03-28T18:09:23.253 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/RCU/whatisRCU.txt 2024-03-28T18:09:23.272 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/SAK.txt 2024-03-28T18:09:23.293 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/SM501.txt 2024-03-28T18:09:23.309 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/SecurityBugs 2024-03-28T18:09:23.322 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/SubmitChecklist 2024-03-28T18:09:23.335 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/SubmittingDrivers 2024-03-28T18:09:23.349 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/SubmittingPatches 2024-03-28T18:09:23.367 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/VGA-softcursor.txt 2024-03-28T18:09:23.380 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/accounting/ 2024-03-28T18:09:23.389 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/accounting/.gitignore 2024-03-28T18:09:23.397 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/accounting/Makefile 2024-03-28T18:09:23.410 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/accounting/cgroupstats.txt 2024-03-28T18:09:23.423 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/accounting/delay-accounting.txt 2024-03-28T18:09:23.436 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/accounting/getdelays.c 2024-03-28T18:09:23.452 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/accounting/taskstats-struct.txt 2024-03-28T18:09:23.465 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/accounting/taskstats.txt 2024-03-28T18:09:23.481 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/acpi/ 2024-03-28T18:09:23.496 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/acpi/apei/ 2024-03-28T18:09:23.504 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/acpi/apei/einj.txt 2024-03-28T18:09:23.513 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/acpi/apei/output_format.txt 2024-03-28T18:09:23.532 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/acpi/debug.txt 2024-03-28T18:09:23.552 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/acpi/dsdt-override.txt 2024-03-28T18:09:23.566 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/acpi/enumeration.txt 2024-03-28T18:09:23.584 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/acpi/gpio-properties.txt 2024-03-28T18:09:23.597 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/acpi/initrd_table_override.txt 2024-03-28T18:09:23.612 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/acpi/method-customizing.txt 2024-03-28T18:09:23.626 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/acpi/method-tracing.txt 2024-03-28T18:09:23.639 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/acpi/namespace.txt 2024-03-28T18:09:23.655 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/acpi/scan_handlers.txt 2024-03-28T18:09:23.670 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/acpi/video_extension.txt 2024-03-28T18:09:23.684 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/aoe/ 2024-03-28T18:09:23.700 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/aoe/aoe.txt 2024-03-28T18:09:23.709 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/aoe/autoload.sh 2024-03-28T18:09:23.724 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/aoe/status.sh 2024-03-28T18:09:23.738 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/aoe/todo.txt 2024-03-28T18:09:23.751 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/aoe/udev-install.sh 2024-03-28T18:09:23.764 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/aoe/udev.txt 2024-03-28T18:09:23.779 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/applying-patches.txt 2024-03-28T18:09:23.798 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/arm/ 2024-03-28T18:09:23.809 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/arm/00-INDEX 2024-03-28T18:09:23.817 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/arm/Atmel/ 2024-03-28T18:09:23.827 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/arm/Atmel/README 2024-03-28T18:09:23.835 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/arm/Booting 2024-03-28T18:09:23.855 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/arm/CCN.txt 2024-03-28T18:09:23.869 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/arm/IXP4xx 2024-03-28T18:09:23.883 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/arm/Interrupts 2024-03-28T18:09:23.899 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/arm/Makefile 2024-03-28T18:09:23.914 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/arm/Marvell/ 2024-03-28T18:09:23.924 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/arm/Marvell/README 2024-03-28T18:09:23.934 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/arm/Netwinder 2024-03-28T18:09:23.956 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/arm/OMAP/ 2024-03-28T18:09:23.969 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/arm/OMAP/DSS 2024-03-28T18:09:23.978 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/arm/OMAP/omap_pm 2024-03-28T18:09:23.994 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/arm/Porting 2024-03-28T18:09:24.010 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/arm/README 2024-03-28T18:09:24.025 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/arm/SA1100/ 2024-03-28T18:09:24.037 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/arm/SA1100/ADSBitsy 2024-03-28T18:09:24.048 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/arm/SA1100/Assabet 2024-03-28T18:09:24.064 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/arm/SA1100/Brutus 2024-03-28T18:09:24.080 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/arm/SA1100/CERF 2024-03-28T18:09:24.094 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/arm/SA1100/FreeBird 2024-03-28T18:09:24.109 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/arm/SA1100/GraphicsClient 2024-03-28T18:09:24.123 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/arm/SA1100/GraphicsMaster 2024-03-28T18:09:24.138 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/arm/SA1100/HUW_WEBPANEL 2024-03-28T18:09:24.207 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/arm/SA1100/Itsy 2024-03-28T18:09:24.232 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/arm/SA1100/LART 2024-03-28T18:09:24.246 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/arm/SA1100/PLEB 2024-03-28T18:09:24.260 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/arm/SA1100/Pangolin 2024-03-28T18:09:24.274 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/arm/SA1100/Tifon 2024-03-28T18:09:24.288 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/arm/SA1100/Victor 2024-03-28T18:09:24.302 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/arm/SA1100/Yopy 2024-03-28T18:09:24.314 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/arm/SA1100/empeg 2024-03-28T18:09:24.328 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/arm/SA1100/nanoEngine 2024-03-28T18:09:24.344 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/arm/SA1100/serial_UART 2024-03-28T18:09:24.358 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/arm/SH-Mobile/ 2024-03-28T18:09:24.373 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/arm/SH-Mobile/.gitignore 2024-03-28T18:09:24.380 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/arm/SH-Mobile/Makefile 2024-03-28T18:09:24.397 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/arm/SH-Mobile/vrl4.c 2024-03-28T18:09:24.413 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/arm/SH-Mobile/zboot-rom-mmcif.txt 2024-03-28T18:09:24.429 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/arm/SH-Mobile/zboot-rom-sdhi.txt 2024-03-28T18:09:24.443 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/arm/SPEAr/ 2024-03-28T18:09:24.459 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/arm/SPEAr/overview.txt 2024-03-28T18:09:24.468 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/arm/Samsung-S3C24XX/ 2024-03-28T18:09:24.484 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/arm/Samsung-S3C24XX/CPUfreq.txt 2024-03-28T18:09:24.494 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/arm/Samsung-S3C24XX/EB2410ITX.txt 2024-03-28T18:09:24.509 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/arm/Samsung-S3C24XX/GPIO.txt 2024-03-28T18:09:24.524 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/arm/Samsung-S3C24XX/H1940.txt 2024-03-28T18:09:24.539 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/arm/Samsung-S3C24XX/NAND.txt 2024-03-28T18:09:24.554 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/arm/Samsung-S3C24XX/Overview.txt 2024-03-28T18:09:24.571 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/arm/Samsung-S3C24XX/S3C2412.txt 2024-03-28T18:09:24.585 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/arm/Samsung-S3C24XX/S3C2413.txt 2024-03-28T18:09:24.599 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/arm/Samsung-S3C24XX/SMDK2440.txt 2024-03-28T18:09:24.613 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/arm/Samsung-S3C24XX/Suspend.txt 2024-03-28T18:09:24.628 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/arm/Samsung-S3C24XX/USB-Host.txt 2024-03-28T18:09:24.643 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/arm/Samsung/ 2024-03-28T18:09:24.659 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/arm/Samsung/GPIO.txt 2024-03-28T18:09:24.667 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/arm/Samsung/Overview.txt 2024-03-28T18:09:24.684 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/arm/Samsung/clksrc-change-registers.awk 2024-03-28T18:09:24.699 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/arm/Setup 2024-03-28T18:09:24.719 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/arm/VFP/ 2024-03-28T18:09:24.731 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/arm/VFP/release-notes.txt 2024-03-28T18:09:24.740 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/arm/cluster-pm-race-avoidance.txt 2024-03-28T18:09:24.762 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/arm/firmware.txt 2024-03-28T18:09:24.777 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/arm/kernel_mode_neon.txt 2024-03-28T18:09:24.781 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:09:24 smithi022 ceph-mon[31985]: pgmap v58: 97 pgs: 97 active+clean; 82 MiB data, 644 MiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 3.4 MiB/s wr, 61 op/s 2024-03-28T18:09:24.792 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/arm/kernel_user_helpers.txt 2024-03-28T18:09:24.808 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/arm/mem_alignment 2024-03-28T18:09:24.823 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/arm/memory.txt 2024-03-28T18:09:24.837 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/arm/msm/ 2024-03-28T18:09:24.848 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/arm/msm/gpiomux.txt 2024-03-28T18:09:24.857 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/arm/nwfpe/ 2024-03-28T18:09:24.873 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/arm/nwfpe/NOTES 2024-03-28T18:09:24.884 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/arm/nwfpe/README 2024-03-28T18:09:24.899 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/arm/nwfpe/README.FPE 2024-03-28T18:09:24.917 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/arm/nwfpe/TODO 2024-03-28T18:09:24.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:09:24 smithi134 ceph-mon[35037]: pgmap v58: 97 pgs: 97 active+clean; 82 MiB data, 644 MiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 3.4 MiB/s wr, 61 op/s 2024-03-28T18:09:24.932 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/arm/pxa/ 2024-03-28T18:09:24.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:09:24 smithi079 ceph-mon[35312]: pgmap v58: 97 pgs: 97 active+clean; 82 MiB data, 644 MiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 3.4 MiB/s wr, 61 op/s 2024-03-28T18:09:24.950 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/arm/pxa/mfp.txt 2024-03-28T18:09:24.961 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/arm/sti/ 2024-03-28T18:09:24.978 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/arm/sti/overview.txt 2024-03-28T18:09:24.987 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/arm/sti/stih407-overview.txt 2024-03-28T18:09:25.002 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/arm/sti/stih415-overview.txt 2024-03-28T18:09:25.017 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/arm/sti/stih416-overview.txt 2024-03-28T18:09:25.031 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/arm/sti/stih418-overview.txt 2024-03-28T18:09:25.046 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/arm/sunxi/ 2024-03-28T18:09:25.062 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/arm/sunxi/README 2024-03-28T18:09:25.070 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/arm/sunxi/clocks.txt 2024-03-28T18:09:25.086 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/arm/swp_emulation 2024-03-28T18:09:25.104 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/arm/tcm.txt 2024-03-28T18:09:25.117 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/arm/uefi.txt 2024-03-28T18:09:25.132 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/arm/vlocks.txt 2024-03-28T18:09:25.147 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/arm64/ 2024-03-28T18:09:25.202 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/arm64/booting.txt 2024-03-28T18:09:25.212 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/arm64/legacy_instructions.txt 2024-03-28T18:09:25.243 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/arm64/memory.txt 2024-03-28T18:09:25.257 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/arm64/tagged-pointers.txt 2024-03-28T18:09:25.273 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/assoc_array.txt 2024-03-28T18:09:25.291 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/atomic_ops.txt 2024-03-28T18:09:25.307 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/auxdisplay/ 2024-03-28T18:09:25.318 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/auxdisplay/.gitignore 2024-03-28T18:09:25.326 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/auxdisplay/Makefile 2024-03-28T18:09:25.340 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/auxdisplay/cfag12864b 2024-03-28T18:09:25.353 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/auxdisplay/cfag12864b-example.c 2024-03-28T18:09:25.368 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/auxdisplay/ks0108 2024-03-28T18:09:25.382 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/backlight/ 2024-03-28T18:09:25.398 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/backlight/lp855x-driver.txt 2024-03-28T18:09:25.406 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/bad_memory.txt 2024-03-28T18:09:25.427 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/basic_profiling.txt 2024-03-28T18:09:25.440 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/bcache.txt 2024-03-28T18:09:25.456 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/binfmt_misc.txt 2024-03-28T18:09:25.469 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/blackfin/ 2024-03-28T18:09:25.478 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/blackfin/00-INDEX 2024-03-28T18:09:25.486 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/blackfin/Makefile 2024-03-28T18:09:25.499 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/blackfin/bfin-gpio-notes.txt 2024-03-28T18:09:25.513 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/blackfin/bfin-spi-notes.txt 2024-03-28T18:09:25.526 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/blackfin/gptimers-example.c 2024-03-28T18:09:25.540 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/block/ 2024-03-28T18:09:25.554 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/block/00-INDEX 2024-03-28T18:09:25.563 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/block/biodoc.txt 2024-03-28T18:09:25.585 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/block/biovecs.txt 2024-03-28T18:09:25.604 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/block/capability.txt 2024-03-28T18:09:25.620 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/block/cfq-iosched.txt 2024-03-28T18:09:25.635 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/block/cmdline-partition.txt 2024-03-28T18:09:25.650 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/block/data-integrity.txt 2024-03-28T18:09:25.663 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/block/deadline-iosched.txt 2024-03-28T18:09:25.677 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/block/ioprio.txt 2024-03-28T18:09:25.693 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/block/null_blk.txt 2024-03-28T18:09:25.706 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/block/queue-sysfs.txt 2024-03-28T18:09:25.720 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/block/request.txt 2024-03-28T18:09:25.734 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/block/stat.txt 2024-03-28T18:09:25.747 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/block/switching-sched.txt 2024-03-28T18:09:25.761 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/block/writeback_cache_control.txt 2024-03-28T18:09:25.775 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/blockdev/ 2024-03-28T18:09:25.789 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/blockdev/00-INDEX 2024-03-28T18:09:25.796 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/blockdev/README.DAC960 2024-03-28T18:09:25.815 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/blockdev/cciss.txt 2024-03-28T18:09:25.830 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/blockdev/cpqarray.txt 2024-03-28T18:09:25.845 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/blockdev/drbd/ 2024-03-28T18:09:25.855 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/blockdev/drbd/DRBD-8.3-data-packets.svg 2024-03-28T18:09:25.866 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/blockdev/drbd/DRBD-data-packets.svg 2024-03-28T18:09:25.883 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/blockdev/drbd/README.txt 2024-03-28T18:09:25.901 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/blockdev/drbd/conn-states-8.dot 2024-03-28T18:09:25.917 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/blockdev/drbd/data-structure-v9.txt 2024-03-28T18:09:25.931 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/blockdev/drbd/disk-states-8.dot 2024-03-28T18:09:25.944 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/blockdev/drbd/drbd-connection-state-overview.dot 2024-03-28T18:09:25.960 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/blockdev/drbd/node-states-8.dot 2024-03-28T18:09:25.974 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/blockdev/floppy.txt 2024-03-28T18:09:25.994 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/blockdev/mflash.txt 2024-03-28T18:09:26.009 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/blockdev/nbd.txt 2024-03-28T18:09:26.023 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/blockdev/paride.txt 2024-03-28T18:09:26.040 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/blockdev/ramdisk.txt 2024-03-28T18:09:26.054 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/blockdev/zram.txt 2024-03-28T18:09:26.069 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/braille-console.txt 2024-03-28T18:09:26.085 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/bt8xxgpio.txt 2024-03-28T18:09:26.100 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/btmrvl.txt 2024-03-28T18:09:26.112 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/bus-devices/ 2024-03-28T18:09:26.123 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/bus-devices/ti-gpmc.txt 2024-03-28T18:09:26.132 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/bus-virt-phys-mapping.txt 2024-03-28T18:09:26.152 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/cachetlb.txt 2024-03-28T18:09:26.171 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/cdrom/ 2024-03-28T18:09:26.214 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/cdrom/00-INDEX 2024-03-28T18:09:26.227 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/cdrom/Makefile 2024-03-28T18:09:26.243 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/cdrom/cdrom-standard.tex 2024-03-28T18:09:26.265 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/cdrom/ide-cd 2024-03-28T18:09:26.282 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/cdrom/packet-writing.txt 2024-03-28T18:09:26.295 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/cgroups/ 2024-03-28T18:09:26.308 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/cgroups/00-INDEX 2024-03-28T18:09:26.316 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/cgroups/blkio-controller.txt 2024-03-28T18:09:26.334 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/cgroups/cgroups.txt 2024-03-28T18:09:26.350 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/cgroups/cpuacct.txt 2024-03-28T18:09:26.364 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/cgroups/cpusets.txt 2024-03-28T18:09:26.384 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/cgroups/devices.txt 2024-03-28T18:09:26.398 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/cgroups/freezer-subsystem.txt 2024-03-28T18:09:26.413 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/cgroups/hugetlb.txt 2024-03-28T18:09:26.427 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/cgroups/memcg_test.txt 2024-03-28T18:09:26.443 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/cgroups/memory.txt 2024-03-28T18:09:26.461 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/cgroups/net_cls.txt 2024-03-28T18:09:26.474 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/cgroups/net_prio.txt 2024-03-28T18:09:26.487 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/cgroups/unified-hierarchy.txt 2024-03-28T18:09:26.504 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/circular-buffers.txt 2024-03-28T18:09:26.523 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/clk.txt 2024-03-28T18:09:26.538 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/coccinelle.txt 2024-03-28T18:09:26.554 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/connector/ 2024-03-28T18:09:26.564 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/connector/.gitignore 2024-03-28T18:09:26.572 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/connector/Makefile 2024-03-28T18:09:26.585 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/connector/cn_test.c 2024-03-28T18:09:26.598 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/connector/connector.txt 2024-03-28T18:09:26.612 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/connector/ucon.c 2024-03-28T18:09:26.626 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/console/ 2024-03-28T18:09:26.640 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/console/console.txt 2024-03-28T18:09:26.649 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/cpu-freq/ 2024-03-28T18:09:26.664 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/cpu-freq/amd-powernow.txt 2024-03-28T18:09:26.673 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/cpu-freq/boost.txt 2024-03-28T18:09:26.687 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/cpu-freq/core.txt 2024-03-28T18:09:26.702 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/cpu-freq/cpu-drivers.txt 2024-03-28T18:09:26.718 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/cpu-freq/cpufreq-nforce2.txt 2024-03-28T18:09:26.734 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/cpu-freq/cpufreq-stats.txt 2024-03-28T18:09:26.747 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/cpu-freq/governors.txt 2024-03-28T18:09:26.762 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/cpu-freq/index.txt 2024-03-28T18:09:26.776 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/cpu-freq/intel-pstate.txt 2024-03-28T18:09:26.791 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/cpu-freq/pcc-cpufreq.txt 2024-03-28T18:09:26.807 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/cpu-freq/user-guide.txt 2024-03-28T18:09:26.821 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/cpu-hotplug.txt 2024-03-28T18:09:26.838 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/cpu-load.txt 2024-03-28T18:09:26.853 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/cpuidle/ 2024-03-28T18:09:26.862 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/cpuidle/core.txt 2024-03-28T18:09:26.871 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/cpuidle/driver.txt 2024-03-28T18:09:26.885 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/cpuidle/governor.txt 2024-03-28T18:09:26.898 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/cpuidle/sysfs.txt 2024-03-28T18:09:26.914 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/cputopology.txt 2024-03-28T18:09:26.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:09:26 smithi134 ceph-mon[35037]: pgmap v59: 97 pgs: 97 active+clean; 86 MiB data, 668 MiB used, 1.0 TiB / 1.0 TiB avail; 146 KiB/s rd, 3.8 MiB/s wr, 117 op/s 2024-03-28T18:09:26.930 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/crc32.txt 2024-03-28T18:09:26.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:09:26 smithi079 ceph-mon[35312]: pgmap v59: 97 pgs: 97 active+clean; 86 MiB data, 668 MiB used, 1.0 TiB / 1.0 TiB avail; 146 KiB/s rd, 3.8 MiB/s wr, 117 op/s 2024-03-28T18:09:26.946 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/cris/ 2024-03-28T18:09:26.955 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/cris/README 2024-03-28T18:09:26.964 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/crypto/ 2024-03-28T18:09:26.979 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/crypto/api-intro.txt 2024-03-28T18:09:26.987 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/crypto/asymmetric-keys.txt 2024-03-28T18:09:27.006 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/crypto/async-tx-api.txt 2024-03-28T18:09:27.021 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/crypto/crypto-API-userspace.txt 2024-03-28T18:09:27.031 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:09:26 smithi022 ceph-mon[31985]: pgmap v59: 97 pgs: 97 active+clean; 86 MiB data, 668 MiB used, 1.0 TiB / 1.0 TiB avail; 146 KiB/s rd, 3.8 MiB/s wr, 117 op/s 2024-03-28T18:09:27.036 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/crypto/descore-readme.txt 2024-03-28T18:09:27.050 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/dcdbas.txt 2024-03-28T18:09:27.071 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/debugging-modules.txt 2024-03-28T18:09:27.084 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/debugging-via-ohci1394.txt 2024-03-28T18:09:27.097 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/dell_rbu.txt 2024-03-28T18:09:27.109 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/development-process/ 2024-03-28T18:09:27.119 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/development-process/1.Intro 2024-03-28T18:09:27.130 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/development-process/2.Process 2024-03-28T18:09:27.146 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/development-process/3.Early-stage 2024-03-28T18:09:27.163 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/development-process/4.Coding 2024-03-28T18:09:27.227 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/development-process/5.Posting 2024-03-28T18:09:27.249 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/development-process/6.Followthrough 2024-03-28T18:09:27.265 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/development-process/7.AdvancedTopics 2024-03-28T18:09:27.280 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/development-process/8.Conclusion 2024-03-28T18:09:27.292 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/device-mapper/ 2024-03-28T18:09:27.308 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/device-mapper/cache-policies.txt 2024-03-28T18:09:27.317 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/device-mapper/cache.txt 2024-03-28T18:09:27.333 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/device-mapper/delay.txt 2024-03-28T18:09:27.349 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/device-mapper/dm-crypt.txt 2024-03-28T18:09:27.363 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/device-mapper/dm-flakey.txt 2024-03-28T18:09:27.377 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/device-mapper/dm-io.txt 2024-03-28T18:09:27.390 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/device-mapper/dm-log.txt 2024-03-28T18:09:27.405 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/device-mapper/dm-queue-length.txt 2024-03-28T18:09:27.418 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/device-mapper/dm-raid.txt 2024-03-28T18:09:27.433 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/device-mapper/dm-service-time.txt 2024-03-28T18:09:27.448 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/device-mapper/dm-uevent.txt 2024-03-28T18:09:27.463 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/device-mapper/era.txt 2024-03-28T18:09:27.476 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/device-mapper/kcopyd.txt 2024-03-28T18:09:27.491 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/device-mapper/linear.txt 2024-03-28T18:09:27.505 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/device-mapper/persistent-data.txt 2024-03-28T18:09:27.519 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/device-mapper/snapshot.txt 2024-03-28T18:09:27.535 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/device-mapper/statistics.txt 2024-03-28T18:09:27.550 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/device-mapper/striped.txt 2024-03-28T18:09:27.565 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/device-mapper/switch.txt 2024-03-28T18:09:27.580 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/device-mapper/thin-provisioning.txt 2024-03-28T18:09:27.595 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/device-mapper/verity.txt 2024-03-28T18:09:27.610 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/device-mapper/zero.txt 2024-03-28T18:09:27.624 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devices.txt 2024-03-28T18:09:27.659 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/ 2024-03-28T18:09:27.671 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/00-INDEX 2024-03-28T18:09:27.679 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/ 2024-03-28T18:09:27.691 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/ABI.txt 2024-03-28T18:09:27.700 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/arc/ 2024-03-28T18:09:27.711 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/arc/interrupts.txt 2024-03-28T18:09:27.720 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/arc/pmu.txt 2024-03-28T18:09:27.736 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/ 2024-03-28T18:09:27.754 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/adapteva.txt 2024-03-28T18:09:27.763 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/altera/ 2024-03-28T18:09:27.778 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/altera/socfpga-clk-manager.txt 2024-03-28T18:09:27.789 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/altera/socfpga-sdram-edac.txt 2024-03-28T18:09:27.807 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/altera/socfpga-system.txt 2024-03-28T18:09:27.823 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/amlogic.txt 2024-03-28T18:09:27.844 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/arch_timer.txt 2024-03-28T18:09:27.860 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/arm-boards 2024-03-28T18:09:27.876 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/armada-370-xp-pmsu.txt 2024-03-28T18:09:27.890 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/armada-370-xp.txt 2024-03-28T18:09:27.905 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/armada-375.txt 2024-03-28T18:09:27.920 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/armada-380-mpcore-soc-ctrl.txt 2024-03-28T18:09:27.934 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/armada-38x.txt 2024-03-28T18:09:27.951 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/armada-cpu-reset.txt 2024-03-28T18:09:27.966 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/armadeus.txt 2024-03-28T18:09:27.981 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/atmel-at91.txt 2024-03-28T18:09:27.996 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/atmel-pmc.txt 2024-03-28T18:09:28.011 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/axxia.txt 2024-03-28T18:09:28.026 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/bcm/ 2024-03-28T18:09:28.038 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/bcm/bcm11351.txt 2024-03-28T18:09:28.048 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/bcm/bcm21664.txt 2024-03-28T18:09:28.064 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/bcm/bcm63138.txt 2024-03-28T18:09:28.081 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/bcm/brcm,bcm11351-cpu-method 2024-03-28T18:09:28.095 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/bcm/cygnus.txt 2024-03-28T18:09:28.112 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/bcm/kona-resetmgr.txt 2024-03-28T18:09:28.126 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/bcm/kona-timer.txt 2024-03-28T18:09:28.142 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/bcm/kona-wdt.txt 2024-03-28T18:09:28.156 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/bcm2835.txt 2024-03-28T18:09:28.186 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/bcm4708.txt 2024-03-28T18:09:28.236 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/brcm-brcmstb.txt 2024-03-28T18:09:28.253 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/calxeda.txt 2024-03-28T18:09:28.268 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/calxeda/ 2024-03-28T18:09:28.279 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/calxeda/combophy.txt 2024-03-28T18:09:28.289 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/calxeda/l2ecc.txt 2024-03-28T18:09:28.306 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/calxeda/mem-ctrlr.txt 2024-03-28T18:09:28.322 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/cavium-thunder.txt 2024-03-28T18:09:28.343 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/cci.txt 2024-03-28T18:09:28.358 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/ccn.txt 2024-03-28T18:09:28.373 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/coherency-fabric.txt 2024-03-28T18:09:28.389 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/coresight.txt 2024-03-28T18:09:28.405 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/cpu-enable-method/ 2024-03-28T18:09:28.417 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/cpu-enable-method/marvell,berlin-smp 2024-03-28T18:09:28.426 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/cpus.txt 2024-03-28T18:09:28.447 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/davinci.txt 2024-03-28T18:09:28.462 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/davinci/ 2024-03-28T18:09:28.473 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/davinci/cp-intc.txt 2024-03-28T18:09:28.485 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/digicolor.txt 2024-03-28T18:09:28.506 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/exynos/ 2024-03-28T18:09:28.520 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/exynos/power_domain.txt 2024-03-28T18:09:28.529 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/exynos/smp-sysram.txt 2024-03-28T18:09:28.546 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/firmware/ 2024-03-28T18:09:28.564 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/firmware/tlm,trusted-foundations.txt 2024-03-28T18:09:28.576 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/fsl.txt 2024-03-28T18:09:28.599 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/fw-cfg.txt 2024-03-28T18:09:28.615 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/geniatech.txt 2024-03-28T18:09:28.632 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/gic-v3.txt 2024-03-28T18:09:28.646 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/gic.txt 2024-03-28T18:09:28.663 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/global_timer.txt 2024-03-28T18:09:28.677 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/hisilicon/ 2024-03-28T18:09:28.689 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/hisilicon/hisilicon.txt 2024-03-28T18:09:28.698 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/idle-states.txt 2024-03-28T18:09:28.723 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/insignal-boards.txt 2024-03-28T18:09:28.742 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/keystone/ 2024-03-28T18:09:28.753 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/keystone/keystone.txt 2024-03-28T18:09:28.763 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/kirkwood.txt 2024-03-28T18:09:28.785 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/l2cc.txt 2024-03-28T18:09:28.801 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/lpc32xx-mic.txt 2024-03-28T18:09:28.815 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/lpc32xx.txt 2024-03-28T18:09:28.829 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/marvell,berlin.txt 2024-03-28T18:09:28.845 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/marvell,dove.txt 2024-03-28T18:09:28.860 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/marvell,kirkwood.txt 2024-03-28T18:09:28.874 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/mediatek.txt 2024-03-28T18:09:28.891 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/mediatek/ 2024-03-28T18:09:28.906 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/mediatek/mediatek,sysirq.txt 2024-03-28T18:09:28.915 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/moxart.txt 2024-03-28T18:09:28.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:09:28 smithi134 ceph-mon[35037]: pgmap v60: 97 pgs: 97 active+clean; 86 MiB data, 668 MiB used, 1.0 TiB / 1.0 TiB avail; 143 KiB/s rd, 1.5 MiB/s wr, 112 op/s 2024-03-28T18:09:28.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:09:28 smithi079 ceph-mon[35312]: pgmap v60: 97 pgs: 97 active+clean; 86 MiB data, 668 MiB used, 1.0 TiB / 1.0 TiB avail; 143 KiB/s rd, 1.5 MiB/s wr, 112 op/s 2024-03-28T18:09:28.937 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/mrvl/ 2024-03-28T18:09:28.949 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/mrvl/feroceon.txt 2024-03-28T18:09:28.959 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/mrvl/intc.txt 2024-03-28T18:09:28.974 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/mrvl/mrvl.txt 2024-03-28T18:09:28.989 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/mrvl/tauros2.txt 2024-03-28T18:09:29.003 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/mrvl/timer.txt 2024-03-28T18:09:29.019 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/msm/ 2024-03-28T18:09:29.031 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:09:28 smithi022 ceph-mon[31985]: pgmap v60: 97 pgs: 97 active+clean; 86 MiB data, 668 MiB used, 1.0 TiB / 1.0 TiB avail; 143 KiB/s rd, 1.5 MiB/s wr, 112 op/s 2024-03-28T18:09:29.036 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/msm/qcom,kpss-acc.txt 2024-03-28T18:09:29.048 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/msm/qcom,saw2.txt 2024-03-28T18:09:29.067 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/msm/ssbi.txt 2024-03-28T18:09:29.086 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/msm/timer.txt 2024-03-28T18:09:29.102 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/mvebu-system-controller.txt 2024-03-28T18:09:29.124 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/nspire.txt 2024-03-28T18:09:29.141 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/olimex.txt 2024-03-28T18:09:29.157 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/omap/ 2024-03-28T18:09:29.170 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/omap/counter.txt 2024-03-28T18:09:29.180 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/omap/crossbar.txt 2024-03-28T18:09:29.271 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/omap/dmm.txt 2024-03-28T18:09:29.299 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/omap/dsp.txt 2024-03-28T18:09:29.314 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/omap/intc.txt 2024-03-28T18:09:29.331 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/omap/iva.txt 2024-03-28T18:09:29.346 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/omap/l3-noc.txt 2024-03-28T18:09:29.361 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/omap/mpu.txt 2024-03-28T18:09:29.377 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/omap/omap.txt 2024-03-28T18:09:29.394 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/omap/prcm.txt 2024-03-28T18:09:29.409 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/omap/timer.txt 2024-03-28T18:09:29.424 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/picoxcell.txt 2024-03-28T18:09:29.446 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/pmu.txt 2024-03-28T18:09:29.462 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/primecell.txt 2024-03-28T18:09:29.479 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/psci.txt 2024-03-28T18:09:29.495 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/rockchip.txt 2024-03-28T18:09:29.511 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/rockchip/ 2024-03-28T18:09:29.524 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/rockchip/pmu-sram.txt 2024-03-28T18:09:29.536 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/rockchip/pmu.txt 2024-03-28T18:09:29.559 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/rockchip/smp-sram.txt 2024-03-28T18:09:29.574 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/rtsm-dcscb.txt 2024-03-28T18:09:29.595 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/samsung-boards.txt 2024-03-28T18:09:29.611 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/samsung/ 2024-03-28T18:09:29.623 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/samsung/exynos-adc.txt 2024-03-28T18:09:29.632 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/samsung/exynos-chipid.txt 2024-03-28T18:09:29.649 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/samsung/interrupt-combiner.txt 2024-03-28T18:09:29.667 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/samsung/pmu.txt 2024-03-28T18:09:29.682 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/samsung/sysreg.txt 2024-03-28T18:09:29.698 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/shmobile.txt 2024-03-28T18:09:29.720 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/sirf.txt 2024-03-28T18:09:29.736 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/spear-misc.txt 2024-03-28T18:09:29.750 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/spear-timer.txt 2024-03-28T18:09:29.765 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/spear.txt 2024-03-28T18:09:29.779 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/spear/ 2024-03-28T18:09:29.791 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/spear/shirq.txt 2024-03-28T18:09:29.800 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/sprd.txt 2024-03-28T18:09:29.821 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/ste-nomadik.txt 2024-03-28T18:09:29.838 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/ste-u300.txt 2024-03-28T18:09:29.853 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/sti.txt 2024-03-28T18:09:29.869 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/sunxi.txt 2024-03-28T18:09:29.884 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/tegra.txt 2024-03-28T18:09:29.900 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/tegra/ 2024-03-28T18:09:29.912 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/tegra/nvidia,tegra20-ahb.txt 2024-03-28T18:09:29.922 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/tegra/nvidia,tegra20-emc.txt 2024-03-28T18:09:29.940 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/tegra/nvidia,tegra20-flowctrl.txt 2024-03-28T18:09:29.955 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/tegra/nvidia,tegra20-mc.txt 2024-03-28T18:09:29.970 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/tegra/nvidia,tegra20-pmc.txt 2024-03-28T18:09:29.987 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/tegra/nvidia,tegra30-mc.txt 2024-03-28T18:09:30.002 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/topology.txt 2024-03-28T18:09:30.025 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/twd.txt 2024-03-28T18:09:30.042 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/ux500/ 2024-03-28T18:09:30.053 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/ux500/power_domain.txt 2024-03-28T18:09:30.063 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/versatile-fpga-irq.txt 2024-03-28T18:09:30.084 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/versatile-sysreg.txt 2024-03-28T18:09:30.100 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/vexpress-scc.txt 2024-03-28T18:09:30.115 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/vexpress-sysreg.txt 2024-03-28T18:09:30.132 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/vexpress.txt 2024-03-28T18:09:30.148 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/vic.txt 2024-03-28T18:09:30.163 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/vt8500.txt 2024-03-28T18:09:30.178 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/vt8500/ 2024-03-28T18:09:30.192 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/vt8500/via,vt8500-intc.txt 2024-03-28T18:09:30.235 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/vt8500/via,vt8500-pmc.txt 2024-03-28T18:09:30.275 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/vt8500/via,vt8500-timer.txt 2024-03-28T18:09:30.292 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/xen.txt 2024-03-28T18:09:30.317 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/xilinx.txt 2024-03-28T18:09:30.334 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/ata/ 2024-03-28T18:09:30.352 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/ata/ahci-platform.txt 2024-03-28T18:09:30.361 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/ata/ahci-st.txt 2024-03-28T18:09:30.378 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/ata/apm-xgene.txt 2024-03-28T18:09:30.394 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/ata/atmel-at91_cf.txt 2024-03-28T18:09:30.410 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/ata/cavium-compact-flash.txt 2024-03-28T18:09:30.428 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/ata/exynos-sata.txt 2024-03-28T18:09:30.443 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/ata/fsl-sata.txt 2024-03-28T18:09:30.459 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/ata/imx-pata.txt 2024-03-28T18:09:30.475 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/ata/imx-sata.txt 2024-03-28T18:09:30.490 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/ata/marvell.txt 2024-03-28T18:09:30.505 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/ata/pata-arasan.txt 2024-03-28T18:09:30.521 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/ata/qcom-sata.txt 2024-03-28T18:09:30.535 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/ata/sata_highbank.txt 2024-03-28T18:09:30.553 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/ata/sata_rcar.txt 2024-03-28T18:09:30.569 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/ata/tegra-sata.txt 2024-03-28T18:09:30.585 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/btmrvl.txt 2024-03-28T18:09:30.606 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/bus/ 2024-03-28T18:09:30.618 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/bus/bcma.txt 2024-03-28T18:09:30.628 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/bus/brcm,gisb-arb.txt 2024-03-28T18:09:30.643 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/bus/imx-weim.txt 2024-03-28T18:09:30.659 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/bus/mvebu-mbus.txt 2024-03-28T18:09:30.674 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/bus/omap-ocp2scp.txt 2024-03-28T18:09:30.689 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/bus/ti-gpmc.txt 2024-03-28T18:09:30.704 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/c6x/ 2024-03-28T18:09:30.719 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/c6x/clocks.txt 2024-03-28T18:09:30.728 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/c6x/dscr.txt 2024-03-28T18:09:30.744 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/c6x/emifa.txt 2024-03-28T18:09:30.759 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/c6x/interrupt.txt 2024-03-28T18:09:30.775 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/c6x/soc.txt 2024-03-28T18:09:30.790 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/c6x/timer64.txt 2024-03-28T18:09:30.805 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/chosen.txt 2024-03-28T18:09:30.826 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/ 2024-03-28T18:09:30.840 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/alphascale,acc.txt 2024-03-28T18:09:30.850 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/altr_socfpga.txt 2024-03-28T18:09:30.866 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/arm-integrator.txt 2024-03-28T18:09:30.880 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/at91-clock.txt 2024-03-28T18:09:30.898 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/axi-clkgen.txt 2024-03-28T18:09:30.915 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/bcm-cygnus-clock.txt 2024-03-28T18:09:30.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:09:30 smithi134 ceph-mon[35037]: pgmap v61: 97 pgs: 97 active+clean; 86 MiB data, 668 MiB used, 1.0 TiB / 1.0 TiB avail; 143 KiB/s rd, 1.5 MiB/s wr, 112 op/s 2024-03-28T18:09:30.931 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/bcm-kona-clock.txt 2024-03-28T18:09:30.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:09:30 smithi079 ceph-mon[35312]: pgmap v61: 97 pgs: 97 active+clean; 86 MiB data, 668 MiB used, 1.0 TiB / 1.0 TiB avail; 143 KiB/s rd, 1.5 MiB/s wr, 112 op/s 2024-03-28T18:09:30.946 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/calxeda.txt 2024-03-28T18:09:30.962 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/clk-exynos-audss.txt 2024-03-28T18:09:30.978 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/clk-palmas-clk32kg-clocks.txt 2024-03-28T18:09:30.993 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/clk-s5pv210-audss.txt 2024-03-28T18:09:31.008 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/clock-bindings.txt 2024-03-28T18:09:31.025 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/clps711x-clock.txt 2024-03-28T18:09:31.031 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:09:30 smithi022 ceph-mon[31985]: pgmap v61: 97 pgs: 97 active+clean; 86 MiB data, 668 MiB used, 1.0 TiB / 1.0 TiB avail; 143 KiB/s rd, 1.5 MiB/s wr, 112 op/s 2024-03-28T18:09:31.040 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/efm32-clock.txt 2024-03-28T18:09:31.055 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/emev2-clock.txt 2024-03-28T18:09:31.070 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/exynos3250-clock.txt 2024-03-28T18:09:31.085 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/exynos4-clock.txt 2024-03-28T18:09:31.100 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/exynos4415-clock.txt 2024-03-28T18:09:31.116 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/exynos5250-clock.txt 2024-03-28T18:09:31.131 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/exynos5260-clock.txt 2024-03-28T18:09:31.148 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/exynos5410-clock.txt 2024-03-28T18:09:31.163 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/exynos5420-clock.txt 2024-03-28T18:09:31.178 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/exynos5440-clock.txt 2024-03-28T18:09:31.192 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/exynos7-clock.txt 2024-03-28T18:09:31.208 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/fixed-clock.txt 2024-03-28T18:09:31.259 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/fixed-factor-clock.txt 2024-03-28T18:09:31.276 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/gpio-gate-clock.txt 2024-03-28T18:09:31.293 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/hi3620-clock.txt 2024-03-28T18:09:31.306 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/hix5hd2-clock.txt 2024-03-28T18:09:31.321 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/imx1-clock.txt 2024-03-28T18:09:31.336 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/imx21-clock.txt 2024-03-28T18:09:31.350 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/imx23-clock.txt 2024-03-28T18:09:31.365 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/imx25-clock.txt 2024-03-28T18:09:31.381 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/imx27-clock.txt 2024-03-28T18:09:31.397 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/imx28-clock.txt 2024-03-28T18:09:31.411 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/imx31-clock.txt 2024-03-28T18:09:31.426 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/imx35-clock.txt 2024-03-28T18:09:31.441 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/imx5-clock.txt 2024-03-28T18:09:31.457 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/imx6q-clock.txt 2024-03-28T18:09:31.471 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/imx6sl-clock.txt 2024-03-28T18:09:31.487 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/imx6sx-clock.txt 2024-03-28T18:09:31.502 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/keystone-gate.txt 2024-03-28T18:09:31.517 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/keystone-pll.txt 2024-03-28T18:09:31.533 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/lsi,axm5516-clks.txt 2024-03-28T18:09:31.549 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/marvell,mmp2.txt 2024-03-28T18:09:31.564 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/marvell,pxa168.txt 2024-03-28T18:09:31.579 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/marvell,pxa910.txt 2024-03-28T18:09:31.594 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/maxim,max77686.txt 2024-03-28T18:09:31.610 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/maxim,max77802.txt 2024-03-28T18:09:31.624 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/moxa,moxart-clock.txt 2024-03-28T18:09:31.640 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/mvebu-core-clock.txt 2024-03-28T18:09:31.655 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/mvebu-corediv-clock.txt 2024-03-28T18:09:31.670 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/mvebu-cpu-clock.txt 2024-03-28T18:09:31.685 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/mvebu-gated-clock.txt 2024-03-28T18:09:31.701 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/nspire-clock.txt 2024-03-28T18:09:31.716 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/nvidia,tegra114-car.txt 2024-03-28T18:09:31.731 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/nvidia,tegra124-car.txt 2024-03-28T18:09:31.747 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/nvidia,tegra20-car.txt 2024-03-28T18:09:31.763 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/nvidia,tegra30-car.txt 2024-03-28T18:09:31.778 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/prima2-clock.txt 2024-03-28T18:09:31.794 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/pxa-clock.txt 2024-03-28T18:09:31.810 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/qcom,gcc.txt 2024-03-28T18:09:31.826 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/qcom,lcc.txt 2024-03-28T18:09:31.842 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/qcom,mmcc.txt 2024-03-28T18:09:31.858 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/qoriq-clock.txt 2024-03-28T18:09:31.873 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/renesas,cpg-div6-clocks.txt 2024-03-28T18:09:31.888 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/renesas,cpg-mstp-clocks.txt 2024-03-28T18:09:31.905 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/renesas,r8a73a4-cpg-clocks.txt 2024-03-28T18:09:31.921 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/renesas,r8a7740-cpg-clocks.txt 2024-03-28T18:09:31.935 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/renesas,r8a7779-cpg-clocks.txt 2024-03-28T18:09:31.950 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/renesas,rcar-gen2-cpg-clocks.txt 2024-03-28T18:09:31.968 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/renesas,rz-cpg-clocks.txt 2024-03-28T18:09:31.983 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/renesas,sh73a0-cpg-clocks.txt 2024-03-28T18:09:31.999 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/rockchip,rk3188-cru.txt 2024-03-28T18:09:32.014 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/rockchip,rk3288-cru.txt 2024-03-28T18:09:32.030 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/rockchip.txt 2024-03-28T18:09:32.046 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/samsung,s3c2410-clock.txt 2024-03-28T18:09:32.061 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/samsung,s3c2412-clock.txt 2024-03-28T18:09:32.075 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/samsung,s3c2443-clock.txt 2024-03-28T18:09:32.093 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/samsung,s3c64xx-clock.txt 2024-03-28T18:09:32.113 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/samsung,s5pv210-clock.txt 2024-03-28T18:09:32.129 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/silabs,si5351.txt 2024-03-28T18:09:32.146 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/silabs,si570.txt 2024-03-28T18:09:32.162 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/st,nomadik.txt 2024-03-28T18:09:32.177 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/st/ 2024-03-28T18:09:32.190 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/st/st,clkgen-divmux.txt 2024-03-28T18:09:32.200 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/st/st,clkgen-mux.txt 2024-03-28T18:09:32.278 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/st/st,clkgen-pll.txt 2024-03-28T18:09:32.302 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/st/st,clkgen-prediv.txt 2024-03-28T18:09:32.319 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/st/st,clkgen-vcc.txt 2024-03-28T18:09:32.335 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/st/st,clkgen.txt 2024-03-28T18:09:32.351 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/st/st,flexgen.txt 2024-03-28T18:09:32.368 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/st/st,quadfs.txt 2024-03-28T18:09:32.384 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/ste-u300-syscon-clock.txt 2024-03-28T18:09:32.408 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/sunxi.txt 2024-03-28T18:09:32.426 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/ti,cdce706.txt 2024-03-28T18:09:32.442 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/ti-keystone-pllctrl.txt 2024-03-28T18:09:32.458 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/ti/ 2024-03-28T18:09:32.470 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/ti/apll.txt 2024-03-28T18:09:32.480 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/ti/autoidle.txt 2024-03-28T18:09:32.496 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/ti/clockdomain.txt 2024-03-28T18:09:32.513 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/ti/composite.txt 2024-03-28T18:09:32.529 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/ti/divider.txt 2024-03-28T18:09:32.548 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/ti/dpll.txt 2024-03-28T18:09:32.563 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/ti/dra7-atl.txt 2024-03-28T18:09:32.580 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/ti/fapll.txt 2024-03-28T18:09:32.596 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/ti/fixed-factor-clock.txt 2024-03-28T18:09:32.612 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/ti/gate.txt 2024-03-28T18:09:32.628 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/ti/interface.txt 2024-03-28T18:09:32.643 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/ti/mux.txt 2024-03-28T18:09:32.659 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/vf610-clock.txt 2024-03-28T18:09:32.679 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/vt8500.txt 2024-03-28T18:09:32.697 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/xgene.txt 2024-03-28T18:09:32.713 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/zynq-7000.txt 2024-03-28T18:09:32.728 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/cpufreq/ 2024-03-28T18:09:32.746 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/cpufreq/arm_big_little_dt.txt 2024-03-28T18:09:32.756 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/cpufreq/cpufreq-dt.txt 2024-03-28T18:09:32.773 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/cpufreq/cpufreq-exynos5440.txt 2024-03-28T18:09:32.789 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/cpufreq/cpufreq-spear.txt 2024-03-28T18:09:32.804 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/crypto/ 2024-03-28T18:09:32.823 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/crypto/amd-ccp.txt 2024-03-28T18:09:32.832 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/crypto/atmel-crypto.txt 2024-03-28T18:09:32.848 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/crypto/fsl-dcp.txt 2024-03-28T18:09:32.864 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/crypto/fsl-imx-sahara.txt 2024-03-28T18:09:32.879 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/crypto/fsl-sec2.txt 2024-03-28T18:09:32.894 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/crypto/fsl-sec4.txt 2024-03-28T18:09:32.911 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/crypto/fsl-sec6.txt 2024-03-28T18:09:32.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:09:32 smithi134 ceph-mon[35037]: pgmap v62: 97 pgs: 97 active+clean; 89 MiB data, 687 MiB used, 1.0 TiB / 1.0 TiB avail; 260 KiB/s rd, 1.7 MiB/s wr, 166 op/s 2024-03-28T18:09:32.929 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/crypto/mv_cesa.txt 2024-03-28T18:09:32.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:09:32 smithi079 ceph-mon[35312]: pgmap v62: 97 pgs: 97 active+clean; 89 MiB data, 687 MiB used, 1.0 TiB / 1.0 TiB avail; 260 KiB/s rd, 1.7 MiB/s wr, 166 op/s 2024-03-28T18:09:32.944 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/crypto/omap-aes.txt 2024-03-28T18:09:32.960 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/crypto/omap-des.txt 2024-03-28T18:09:32.975 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/crypto/omap-sham.txt 2024-03-28T18:09:32.990 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/crypto/picochip-spacc.txt 2024-03-28T18:09:33.007 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/crypto/qcom-qce.txt 2024-03-28T18:09:33.023 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/crypto/samsung-sss.txt 2024-03-28T18:09:33.031 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:09:32 smithi022 ceph-mon[31985]: pgmap v62: 97 pgs: 97 active+clean; 89 MiB data, 687 MiB used, 1.0 TiB / 1.0 TiB avail; 260 KiB/s rd, 1.7 MiB/s wr, 166 op/s 2024-03-28T18:09:33.037 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/devfreq/ 2024-03-28T18:09:33.054 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/devfreq/event/ 2024-03-28T18:09:33.061 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/devfreq/event/exynos-ppmu.txt 2024-03-28T18:09:33.071 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/dma/ 2024-03-28T18:09:33.100 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/dma/arm-pl330.txt 2024-03-28T18:09:33.111 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/dma/atmel-dma.txt 2024-03-28T18:09:33.130 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/dma/atmel-xdma.txt 2024-03-28T18:09:33.147 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/dma/bcm2835-dma.txt 2024-03-28T18:09:33.162 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/dma/dma.txt 2024-03-28T18:09:33.180 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/dma/fsl-edma.txt 2024-03-28T18:09:33.196 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/dma/fsl-imx-dma.txt 2024-03-28T18:09:33.226 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/dma/fsl-imx-sdma.txt 2024-03-28T18:09:33.271 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/dma/fsl-mxs-dma.txt 2024-03-28T18:09:33.300 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/dma/img-mdc-dma.txt 2024-03-28T18:09:33.315 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/dma/k3dma.txt 2024-03-28T18:09:33.330 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/dma/mmp-dma.txt 2024-03-28T18:09:33.349 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/dma/moxa,moxart-dma.txt 2024-03-28T18:09:33.363 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/dma/mpc512x-dma.txt 2024-03-28T18:09:33.377 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/dma/mv-xor.txt 2024-03-28T18:09:33.392 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/dma/nbpfaxi.txt 2024-03-28T18:09:33.406 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/dma/qcom_adm.txt 2024-03-28T18:09:33.419 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/dma/qcom_bam_dma.txt 2024-03-28T18:09:33.433 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/dma/rcar-audmapp.txt 2024-03-28T18:09:33.446 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/dma/renesas,rcar-dmac.txt 2024-03-28T18:09:33.461 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/dma/shdma.txt 2024-03-28T18:09:33.476 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/dma/sirfsoc-dma.txt 2024-03-28T18:09:33.491 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/dma/snps-dma.txt 2024-03-28T18:09:33.504 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/dma/ste-coh901318.txt 2024-03-28T18:09:33.517 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/dma/ste-dma40.txt 2024-03-28T18:09:33.535 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/dma/sun6i-dma.txt 2024-03-28T18:09:33.549 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/dma/tegra20-apbdma.txt 2024-03-28T18:09:33.563 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/dma/ti-edma.txt 2024-03-28T18:09:33.579 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/dma/xilinx/ 2024-03-28T18:09:33.591 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/dma/xilinx/xilinx_dma.txt 2024-03-28T18:09:33.602 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/dma/xilinx/xilinx_vdma.txt 2024-03-28T18:09:33.619 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/drm/ 2024-03-28T18:09:33.646 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/drm/armada/ 2024-03-28T18:09:33.655 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/drm/armada/marvell,dove-lcd.txt 2024-03-28T18:09:33.664 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/drm/atmel/ 2024-03-28T18:09:33.685 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/drm/atmel/hlcdc-dc.txt 2024-03-28T18:09:33.696 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/drm/bridge/ 2024-03-28T18:09:33.719 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/drm/bridge/dw_hdmi.txt 2024-03-28T18:09:33.731 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/drm/i2c/ 2024-03-28T18:09:33.753 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/drm/i2c/tda998x.txt 2024-03-28T18:09:33.762 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/drm/imx/ 2024-03-28T18:09:33.782 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/drm/imx/fsl-imx-drm.txt 2024-03-28T18:09:33.792 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/drm/imx/hdmi.txt 2024-03-28T18:09:33.812 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/drm/imx/ldb.txt 2024-03-28T18:09:33.828 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/drm/msm/ 2024-03-28T18:09:33.847 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/drm/msm/gpu.txt 2024-03-28T18:09:33.856 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/drm/msm/hdmi.txt 2024-03-28T18:09:33.877 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/drm/msm/mdp.txt 2024-03-28T18:09:33.895 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/drm/tilcdc/ 2024-03-28T18:09:33.920 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/drm/tilcdc/panel.txt 2024-03-28T18:09:33.932 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/drm/tilcdc/slave.txt 2024-03-28T18:09:33.949 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/drm/tilcdc/tfp410.txt 2024-03-28T18:09:33.963 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/drm/tilcdc/tilcdc.txt 2024-03-28T18:09:33.979 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/eeprom.txt 2024-03-28T18:09:34.004 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/extcon/ 2024-03-28T18:09:34.018 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/extcon/extcon-palmas.txt 2024-03-28T18:09:34.029 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/extcon/extcon-rt8973a.txt 2024-03-28T18:09:34.045 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/extcon/extcon-sm5502.txt 2024-03-28T18:09:34.063 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/fb/ 2024-03-28T18:09:34.083 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/fb/mxsfb.txt 2024-03-28T18:09:34.092 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/fb/sm501fb.txt 2024-03-28T18:09:34.107 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/fpga/ 2024-03-28T18:09:34.124 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/fpga/altera-socfpga-fpga-mgr.txt 2024-03-28T18:09:34.134 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/fuse/ 2024-03-28T18:09:34.152 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/fuse/nvidia,tegra20-fuse.txt 2024-03-28T18:09:34.161 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/gpio/ 2024-03-28T18:09:34.178 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/gpio/8xxx_gpio.txt 2024-03-28T18:09:34.189 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/gpio/abilis,tb10x-gpio.txt 2024-03-28T18:09:34.206 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/gpio/cavium-octeon-gpio.txt 2024-03-28T18:09:34.223 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/gpio/cirrus,clps711x-mctrl-gpio.txt 2024-03-28T18:09:34.287 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/gpio/fsl-imx-gpio.txt 2024-03-28T18:09:34.326 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/gpio/fujitsu,mb86s70-gpio.txt 2024-03-28T18:09:34.339 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/gpio/gpio-74x164.txt 2024-03-28T18:09:34.353 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/gpio/gpio-74xx-mmio.txt 2024-03-28T18:09:34.368 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/gpio/gpio-adnp.txt 2024-03-28T18:09:34.384 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/gpio/gpio-bcm-kona.txt 2024-03-28T18:09:34.399 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/gpio/gpio-clps711x.txt 2024-03-28T18:09:34.414 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/gpio/gpio-davinci.txt 2024-03-28T18:09:34.430 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/gpio/gpio-dsp-keystone.txt 2024-03-28T18:09:34.445 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/gpio/gpio-fan.txt 2024-03-28T18:09:34.461 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/gpio/gpio-grgpio.txt 2024-03-28T18:09:34.476 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/gpio/gpio-lp3943.txt 2024-03-28T18:09:34.490 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/gpio/gpio-max732x.txt 2024-03-28T18:09:34.505 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/gpio/gpio-mcp23s08.txt 2024-03-28T18:09:34.521 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/gpio/gpio-mm-lantiq.txt 2024-03-28T18:09:34.536 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/gpio/gpio-msm.txt 2024-03-28T18:09:34.552 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/gpio/gpio-mvebu.txt 2024-03-28T18:09:34.566 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/gpio/gpio-mxs.txt 2024-03-28T18:09:34.584 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/gpio/gpio-nmk.txt 2024-03-28T18:09:34.600 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/gpio/gpio-omap.txt 2024-03-28T18:09:34.614 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/gpio/gpio-palmas.txt 2024-03-28T18:09:34.630 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/gpio/gpio-pca953x.txt 2024-03-28T18:09:34.645 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/gpio/gpio-pcf857x.txt 2024-03-28T18:09:34.661 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/gpio/gpio-poweroff.txt 2024-03-28T18:09:34.676 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/gpio/gpio-restart.txt 2024-03-28T18:09:34.690 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/gpio/gpio-samsung.txt 2024-03-28T18:09:34.708 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/gpio/gpio-stericsson-coh901.txt 2024-03-28T18:09:34.724 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/gpio/gpio-stmpe.txt 2024-03-28T18:09:34.740 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/gpio/gpio-stp-xway.txt 2024-03-28T18:09:34.755 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/gpio/gpio-sx150x.txt 2024-03-28T18:09:34.771 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/gpio/gpio-twl4030.txt 2024-03-28T18:09:34.786 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/gpio/gpio-tz1090-pdc.txt 2024-03-28T18:09:34.802 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/gpio/gpio-tz1090.txt 2024-03-28T18:09:34.817 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/gpio/gpio-vf610.txt 2024-03-28T18:09:34.833 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/gpio/gpio-xgene-sb.txt 2024-03-28T18:09:34.848 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/gpio/gpio-xgene.txt 2024-03-28T18:09:34.864 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/gpio/gpio-xilinx.txt 2024-03-28T18:09:34.881 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/gpio/gpio-zevio.txt 2024-03-28T18:09:34.890 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-28T18:09:34.890 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-28T18:09:34.899 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/gpio/gpio-zynq.txt 2024-03-28T18:09:34.911 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/gpio/gpio.txt 2024-03-28T18:09:34.923 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/gpio/gpio_atmel.txt 2024-03-28T18:09:34.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:09:34 smithi134 ceph-mon[35037]: pgmap v63: 97 pgs: 97 active+clean; 89 MiB data, 687 MiB used, 1.0 TiB / 1.0 TiB avail; 211 KiB/s rd, 642 KiB/s wr, 109 op/s 2024-03-28T18:09:34.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:09:34 smithi079 ceph-mon[35312]: pgmap v63: 97 pgs: 97 active+clean; 89 MiB data, 687 MiB used, 1.0 TiB / 1.0 TiB avail; 211 KiB/s rd, 642 KiB/s wr, 109 op/s 2024-03-28T18:09:34.937 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/gpio/gpio_lpc32xx.txt 2024-03-28T18:09:34.950 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/gpio/moxa,moxart-gpio.txt 2024-03-28T18:09:34.959 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/gpio/mrvl-gpio.txt 2024-03-28T18:09:34.973 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/gpio/nvidia,tegra20-gpio.txt 2024-03-28T18:09:34.987 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/gpio/pl061-gpio.txt 2024-03-28T18:09:34.989 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:09:34 smithi022 ceph-mon[31985]: pgmap v63: 97 pgs: 97 active+clean; 89 MiB data, 687 MiB used, 1.0 TiB / 1.0 TiB avail; 211 KiB/s rd, 642 KiB/s wr, 109 op/s 2024-03-28T18:09:35.002 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/gpio/renesas,gpio-rcar.txt 2024-03-28T18:09:35.012 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/gpio/snps-dwapb-gpio.txt 2024-03-28T18:09:35.023 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/gpio/sodaville.txt 2024-03-28T18:09:35.036 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/gpio/spear_spics.txt 2024-03-28T18:09:35.050 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/gpu/ 2024-03-28T18:09:35.065 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/gpu/nvidia,gk20a.txt 2024-03-28T18:09:35.073 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/gpu/nvidia,tegra20-host1x.txt 2024-03-28T18:09:35.091 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/gpu/samsung-g2d.txt 2024-03-28T18:09:35.107 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/gpu/samsung-rotator.txt 2024-03-28T18:09:35.124 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/gpu/st,stih4xx.txt 2024-03-28T18:09:35.137 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/graph.txt 2024-03-28T18:09:35.141 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:35.141+0000 7f2e5e400640 1 -- 172.21.15.22:0/1269358550 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f2e581075c0 msgr2=0x7f2e581079a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:09:35.141 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:35.141+0000 7f2e5e400640 1 --2- 172.21.15.22:0/1269358550 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f2e581075c0 0x7f2e581079a0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f2e40009930 tx=0x7f2e4002bc20 comp rx=0 tx=0).stop 2024-03-28T18:09:35.141 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:35.141+0000 7f2e5e400640 1 -- 172.21.15.22:0/1269358550 shutdown_connections 2024-03-28T18:09:35.141 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:35.141+0000 7f2e5e400640 1 --2- 172.21.15.22:0/1269358550 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2e58101ec0 0x7f2e5810e380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:09:35.141 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:35.141+0000 7f2e5e400640 1 --2- 172.21.15.22:0/1269358550 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2e58101500 0x7f2e58101980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:09:35.141 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:35.141+0000 7f2e5e400640 1 --2- 172.21.15.22:0/1269358550 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f2e581075c0 0x7f2e581079a0 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:09:35.141 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:35.141+0000 7f2e5e400640 1 -- 172.21.15.22:0/1269358550 >> 172.21.15.22:0/1269358550 conn(0x7f2e580fd010 msgr2=0x7f2e580ff430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:09:35.142 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:35.141+0000 7f2e5e400640 1 -- 172.21.15.22:0/1269358550 shutdown_connections 2024-03-28T18:09:35.142 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:35.141+0000 7f2e5e400640 1 -- 172.21.15.22:0/1269358550 wait complete. 2024-03-28T18:09:35.143 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:35.142+0000 7f2e5e400640 1 Processor -- start 2024-03-28T18:09:35.144 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:35.142+0000 7f2e5e400640 1 -- start start 2024-03-28T18:09:35.144 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:35.143+0000 7f2e5e400640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2e58101500 0x7f2e58199fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:09:35.144 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:35.143+0000 7f2e5e400640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2e58101ec0 0x7f2e5819a500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:09:35.144 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:35.143+0000 7f2e5e400640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f2e581075c0 0x7f2e5819e880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:09:35.144 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:35.143+0000 7f2e5e400640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e581118c0 con 0x7f2e58101500 2024-03-28T18:09:35.144 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:35.143+0000 7f2e5e400640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e58111740 con 0x7f2e581075c0 2024-03-28T18:09:35.144 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:35.143+0000 7f2e5e400640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e58111a40 con 0x7f2e58101ec0 2024-03-28T18:09:35.144 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:35.143+0000 7f2e5d000640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2e58101500 0x7f2e58199fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:09:35.144 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:35.143+0000 7f2e5d000640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2e58101500 0x7f2e58199fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:35912/0 (socket says 172.21.15.22:35912) 2024-03-28T18:09:35.144 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:35.143+0000 7f2e5d000640 1 -- 172.21.15.22:0/3431071085 learned_addr learned my addr 172.21.15.22:0/3431071085 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-28T18:09:35.144 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:35.143+0000 7f2e5da00640 1 --2- 172.21.15.22:0/3431071085 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f2e581075c0 0x7f2e5819e880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:09:35.144 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:35.143+0000 7f2e4fe00640 1 --2- 172.21.15.22:0/3431071085 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2e58101ec0 0x7f2e5819a500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:09:35.144 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:35.144+0000 7f2e5d000640 1 -- 172.21.15.22:0/3431071085 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2e58101ec0 msgr2=0x7f2e5819a500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:09:35.144 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:35.144+0000 7f2e5d000640 1 --2- 172.21.15.22:0/3431071085 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2e58101ec0 0x7f2e5819a500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:09:35.145 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:35.144+0000 7f2e5d000640 1 -- 172.21.15.22:0/3431071085 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f2e581075c0 msgr2=0x7f2e5819e880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:09:35.145 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:35.144+0000 7f2e5d000640 1 --2- 172.21.15.22:0/3431071085 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f2e581075c0 0x7f2e5819e880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:09:35.145 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:35.144+0000 7f2e5d000640 1 -- 172.21.15.22:0/3431071085 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2e5819efb0 con 0x7f2e58101500 2024-03-28T18:09:35.145 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:35.144+0000 7f2e5d000640 1 --2- 172.21.15.22:0/3431071085 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2e58101500 0x7f2e58199fc0 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7f2e40000c00 tx=0x7f2e400045e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:09:35.145 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:35.144+0000 7f2e56000640 1 -- 172.21.15.22:0/3431071085 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e40043020 con 0x7f2e58101500 2024-03-28T18:09:35.146 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:35.144+0000 7f2e5e400640 1 -- 172.21.15.22:0/3431071085 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2e5819f240 con 0x7f2e58101500 2024-03-28T18:09:35.146 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:35.145+0000 7f2e56000640 1 -- 172.21.15.22:0/3431071085 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f2e4002ca90 con 0x7f2e58101500 2024-03-28T18:09:35.146 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:35.145+0000 7f2e5e400640 1 -- 172.21.15.22:0/3431071085 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2e581a6b30 con 0x7f2e58101500 2024-03-28T18:09:35.146 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:35.145+0000 7f2e56000640 1 -- 172.21.15.22:0/3431071085 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e40003a80 con 0x7f2e58101500 2024-03-28T18:09:35.148 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:35.148+0000 7f2e627eb740 1 Processor -- start 2024-03-28T18:09:35.148 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:35.148+0000 7f2e56000640 1 -- 172.21.15.22:0/3431071085 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 26) v1 ==== 137421+0+0 (secure 0 0 0) 0x7f2e40046020 con 0x7f2e58101500 2024-03-28T18:09:35.148 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:35.149+0000 7f2e627eb740 1 -- start start 2024-03-28T18:09:35.150 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:35.149+0000 7f2e627eb740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55913def2dd0 0x55913def1b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:09:35.150 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:35.149+0000 7f2e627eb740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55913def20f0 0x55913def2570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:09:35.150 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:35.149+0000 7f2e627eb740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55913defb460 0x55913defd850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:09:35.150 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:35.149+0000 7f2e627eb740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x55913de56250 con 0x55913def20f0 2024-03-28T18:09:35.151 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:35.149+0000 7f2e627eb740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x55913ddfea30 con 0x55913defb460 2024-03-28T18:09:35.151 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:35.149+0000 7f2e627eb740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55913de522f0 con 0x55913def2dd0 2024-03-28T18:09:35.151 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:35.149+0000 7f2e5d000640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55913def20f0 0x55913def2570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:09:35.151 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:35.149+0000 7f2e4fe00640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55913defb460 0x55913defd850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:09:35.151 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:35.149+0000 7f2e4fe00640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55913defb460 0x55913defd850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.22:57542/0 (socket says 172.21.15.22:57542) 2024-03-28T18:09:35.151 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:35.150+0000 7f2e4fe00640 1 -- 172.21.15.22:0/1821414632 learned_addr learned my addr 172.21.15.22:0/1821414632 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-28T18:09:35.151 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:35.150+0000 7f2e5da00640 1 --2- 172.21.15.22:0/1821414632 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55913def2dd0 0x55913def1b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:09:35.151 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:35.150+0000 7f2e4fe00640 1 -- 172.21.15.22:0/1821414632 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55913def2dd0 msgr2=0x55913def1b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T18:09:35.151 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:35.150+0000 7f2e4fe00640 1 --2- 172.21.15.22:0/1821414632 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55913def2dd0 0x55913def1b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:09:35.151 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:35.150+0000 7f2e4fe00640 1 -- 172.21.15.22:0/1821414632 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55913def20f0 msgr2=0x55913def2570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T18:09:35.151 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:35.150+0000 7f2e4fe00640 1 --2- 172.21.15.22:0/1821414632 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55913def20f0 0x55913def2570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:09:35.151 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:35.150+0000 7f2e4fe00640 1 -- 172.21.15.22:0/1821414632 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55913dde9690 con 0x55913defb460 2024-03-28T18:09:35.152 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:35.150+0000 7f2e4fe00640 1 --2- 172.21.15.22:0/1821414632 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55913defb460 0x55913defd850 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f2e48009c20 tx=0x7f2e480117b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3e307355c5071bf3 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:09:35.152 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:35.150+0000 7f2e56000640 1 --2- 172.21.15.22:0/3431071085 >> [v2:172.21.15.22:6800/1357365082,v1:172.21.15.22:6801/1357365082] conn(0x7f2e380a2970 0x7f2e380a4e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:09:35.152 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:35.151+0000 7f2e56000640 1 -- 172.21.15.22:0/3431071085 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 8411+0+0 (secure 0 0 0) 0x7f2e400f0db0 con 0x7f2e58101500 2024-03-28T18:09:35.152 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:35.151+0000 7f2e4e000640 1 -- 172.21.15.22:0/1821414632 <== mon.2 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e48012450 con 0x55913defb460 2024-03-28T18:09:35.152 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:35.151+0000 7f2e4e000640 1 -- 172.21.15.22:0/1821414632 <== mon.2 v2:172.21.15.79:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f2e48012ac0 con 0x55913defb460 2024-03-28T18:09:35.152 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:35.151+0000 7f2e4e000640 1 -- 172.21.15.22:0/1821414632 <== mon.2 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e4800fc30 con 0x55913defb460 2024-03-28T18:09:35.152 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:35.151+0000 7f2e5da00640 1 --2- 172.21.15.22:0/3431071085 >> [v2:172.21.15.22:6800/1357365082,v1:172.21.15.22:6801/1357365082] conn(0x7f2e380a2970 0x7f2e380a4e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:09:35.153 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:35.151+0000 7f2e627eb740 1 -- 172.21.15.22:0/1821414632 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55913defb460 msgr2=0x55913defd850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:09:35.153 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:35.151+0000 7f2e627eb740 1 --2- 172.21.15.22:0/1821414632 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55913defb460 0x55913defd850 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f2e48009c20 tx=0x7f2e480117b0 comp rx=0 tx=0).stop 2024-03-28T18:09:35.153 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:35.152+0000 7f2e5da00640 1 --2- 172.21.15.22:0/3431071085 >> [v2:172.21.15.22:6800/1357365082,v1:172.21.15.22:6801/1357365082] conn(0x7f2e380a2970 0x7f2e380a4e30 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f2e50005fd0 tx=0x7f2e500096a0 comp rx=0 tx=0).ready entity=mgr.15522 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:09:35.153 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:35.152+0000 7f2e627eb740 1 -- 172.21.15.22:0/1821414632 shutdown_connections 2024-03-28T18:09:35.153 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:35.152+0000 7f2e627eb740 1 --2- 172.21.15.22:0/1821414632 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55913defb460 0x55913defd850 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:09:35.153 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:35.152+0000 7f2e627eb740 1 --2- 172.21.15.22:0/1821414632 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55913def20f0 0x55913def2570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:09:35.153 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:35.152+0000 7f2e627eb740 1 --2- 172.21.15.22:0/1821414632 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55913def2dd0 0x55913def1b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:09:35.154 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:35.152+0000 7f2e627eb740 1 -- 172.21.15.22:0/1821414632 >> 172.21.15.22:0/1821414632 conn(0x55913deb9200 msgr2=0x55913deecb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:09:35.154 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:35.152+0000 7f2e627eb740 1 -- 172.21.15.22:0/1821414632 shutdown_connections 2024-03-28T18:09:35.154 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:35.152+0000 7f2e627eb740 1 -- 172.21.15.22:0/1821414632 wait complete. 2024-03-28T18:09:35.154 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/hid/ 2024-03-28T18:09:35.157 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:35.152+0000 7f2e627eb740 1 -- start start 2024-03-28T18:09:35.157 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:35.153+0000 7f2e627eb740 1 Processor -- start 2024-03-28T18:09:35.157 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:35.153+0000 7f2e627eb740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55913def20f0 0x55913defab20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:09:35.157 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:35.153+0000 7f2e627eb740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55913def2dd0 0x55913def7eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:09:35.157 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:35.153+0000 7f2e627eb740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55913defb460 0x55913def83f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:09:35.157 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:35.153+0000 7f2e627eb740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x55913defe2d0 con 0x55913def20f0 2024-03-28T18:09:35.157 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:35.153+0000 7f2e627eb740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x55913de11e80 con 0x55913def2dd0 2024-03-28T18:09:35.158 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:35.153+0000 7f2e627eb740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55913defe450 con 0x55913defb460 2024-03-28T18:09:35.158 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:35.153+0000 7f2e627eb740 10 client.0 _ll_register_callbacks cb 0x55913de24130 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-28T18:09:35.158 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:35.154+0000 7f2e4fe00640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55913def2dd0 0x55913def7eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:09:35.158 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:35.154+0000 7f2e4fe00640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55913def2dd0 0x55913def7eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.22:57544/0 (socket says 172.21.15.22:57544) 2024-03-28T18:09:35.158 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:35.154+0000 7f2e4fe00640 1 -- 172.21.15.22:0/755693557 learned_addr learned my addr 172.21.15.22:0/755693557 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-28T18:09:35.158 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:35.154+0000 7f2e5d000640 1 --2- 172.21.15.22:0/755693557 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55913def20f0 0x55913defab20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:09:35.158 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:35.154+0000 7f2e4fe00640 1 -- 172.21.15.22:0/755693557 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55913defb460 msgr2=0x55913def83f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T18:09:35.158 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:35.154+0000 7f2e5da00640 1 --2- 172.21.15.22:0/755693557 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55913defb460 0x55913def83f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:09:35.158 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:35.154+0000 7f2e4fe00640 1 --2- 172.21.15.22:0/755693557 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55913defb460 0x55913def83f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:09:35.158 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:35.154+0000 7f2e4fe00640 1 -- 172.21.15.22:0/755693557 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55913def20f0 msgr2=0x55913defab20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T18:09:35.158 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:35.155+0000 7f2e4fe00640 1 --2- 172.21.15.22:0/755693557 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55913def20f0 0x55913defab20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:09:35.159 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:35.155+0000 7f2e4fe00640 1 -- 172.21.15.22:0/755693557 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55913dc347d0 con 0x55913def2dd0 2024-03-28T18:09:35.159 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:35.155+0000 7f2e5da00640 1 --2- 172.21.15.22:0/755693557 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55913defb460 0x55913def83f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T18:09:35.159 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:35.155+0000 7f2e5d000640 1 --2- 172.21.15.22:0/755693557 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55913def20f0 0x55913defab20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-28T18:09:35.159 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:35.155+0000 7f2e4fe00640 1 --2- 172.21.15.22:0/755693557 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55913def2dd0 0x55913def7eb0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f2e48000c00 tx=0x7f2e48005760 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1dba1fc3560e9928 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:09:35.159 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:35.155+0000 7f2e46a00640 10 client.0 ms_handle_connect on v2:172.21.15.79:3300/0 2024-03-28T18:09:35.159 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:35.155+0000 7f2e46a00640 1 -- 172.21.15.22:0/755693557 <== mon.2 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e48028070 con 0x55913def2dd0 2024-03-28T18:09:35.159 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:35.155+0000 7f2e46a00640 1 -- 172.21.15.22:0/755693557 <== mon.2 v2:172.21.15.79:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 8411+0+0 (secure 0 0 0) 0x7f2e48004450 con 0x55913def2dd0 2024-03-28T18:09:35.159 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:35.156+0000 7f2e627eb740 1 -- 172.21.15.22:0/755693557 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f2e48012ac0 con 0x55913def2dd0 2024-03-28T18:09:35.159 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:35.156+0000 7f2e46a00640 1 -- 172.21.15.22:0/755693557 <== mon.2 v2:172.21.15.79:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2e48034050 con 0x55913def2dd0 2024-03-28T18:09:35.159 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:35.156+0000 7f2e627eb740 10 client.35903 fetch_fsmap learned FSMap version 27 2024-03-28T18:09:35.159 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:35.156+0000 7f2e627eb740 1 -- 172.21.15.22:0/755693557 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55913db6a3f0 con 0x55913def2dd0 2024-03-28T18:09:35.159 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:35.156+0000 7f2e46a00640 1 -- 172.21.15.22:0/755693557 <== mon.2 v2:172.21.15.79:3300/0 4 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f2e48012ac0 con 0x55913def2dd0 2024-03-28T18:09:35.159 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:35.156+0000 7f2e46a00640 1 -- 172.21.15.22:0/755693557 <== mon.2 v2:172.21.15.79:3300/0 5 ==== fsmap(e 27) v1 ==== 7934+0+0 (secure 0 0 0) 0x7f2e4800fc80 con 0x55913def2dd0 2024-03-28T18:09:35.159 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:35.156+0000 7f2e627eb740 10 client.35903 fetch_fsmap finished waiting for FSMap version 27 2024-03-28T18:09:35.159 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:35.156+0000 7f2e627eb740 10 client.35903 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-28T18:09:35.160 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:35.156+0000 7f2e627eb740 20 client.35903 populate_metadata read hostname 'smithi022' 2024-03-28T18:09:35.160 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:35.156+0000 7f2e627eb740 1 --2- 172.21.15.22:0/755693557 >> [v2:172.21.15.79:6838/2441542443,v1:172.21.15.79:6839/2441542443] conn(0x55913df0b280 0x55913df0d670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:09:35.160 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:35.157+0000 7f2e627eb740 4 client.35903 mds_command: new command op to 34553 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-28T18:09:35.160 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:35.157+0000 7f2e627eb740 1 -- 172.21.15.22:0/755693557 --> [v2:172.21.15.79:6838/2441542443,v1:172.21.15.79:6839/2441542443] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55913de63790 con 0x55913df0b280 2024-03-28T18:09:35.160 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:35.157+0000 7f2e5d000640 1 --2- 172.21.15.22:0/755693557 >> [v2:172.21.15.79:6838/2441542443,v1:172.21.15.79:6839/2441542443] conn(0x55913df0b280 0x55913df0d670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:09:35.160 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:35.157+0000 7f2e5d000640 1 --2- 172.21.15.22:0/755693557 >> [v2:172.21.15.79:6838/2441542443,v1:172.21.15.79:6839/2441542443] conn(0x55913df0b280 0x55913df0d670 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3a448f8eed486b67 server_cookie=26ff08d0c73be911 in_seq=0 out_seq=0 2024-03-28T18:09:35.160 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:35.157+0000 7f2e46a00640 10 client.35903 ms_handle_connect on v2:172.21.15.79:6838/2441542443 2024-03-28T18:09:35.161 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:35.158+0000 7f2e46a00640 1 -- 172.21.15.22:0/755693557 <== mds.0 v2:172.21.15.79:6838/2441542443 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x55913de63790 con 0x55913df0b280 2024-03-28T18:09:35.161 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:35.158+0000 7f2e46a00640 10 client.35903 handle_command_reply: tid=0 2024-03-28T18:09:35.161 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:35.158+0000 7f2e627eb740 1 client.35903 shutdown 2024-03-28T18:09:35.161 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:35.159+0000 7f2e627eb740 1 -- 172.21.15.22:0/755693557 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55913def2dd0 msgr2=0x55913def7eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:09:35.161 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:35.159+0000 7f2e627eb740 1 --2- 172.21.15.22:0/755693557 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55913def2dd0 0x55913def7eb0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f2e48000c00 tx=0x7f2e48005760 comp rx=0 tx=0).stop 2024-03-28T18:09:35.161 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:35.159+0000 7f2e627eb740 1 -- 172.21.15.22:0/755693557 shutdown_connections 2024-03-28T18:09:35.161 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:35.159+0000 7f2e627eb740 1 --2- 172.21.15.22:0/755693557 >> [v2:172.21.15.79:6838/2441542443,v1:172.21.15.79:6839/2441542443] conn(0x55913df0b280 0x55913df0d670 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:09:35.162 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:35.159+0000 7f2e627eb740 1 --2- 172.21.15.22:0/755693557 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55913defb460 0x55913def83f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:09:35.162 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:35.159+0000 7f2e627eb740 1 --2- 172.21.15.22:0/755693557 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55913def2dd0 0x55913def7eb0 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:09:35.162 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:35.159+0000 7f2e627eb740 1 --2- 172.21.15.22:0/755693557 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55913def20f0 0x55913defab20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:09:35.162 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:35.159+0000 7f2e46a00640 0 client.35903 ms_handle_reset on v2:172.21.15.79:6838/2441542443 2024-03-28T18:09:35.162 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:35.159+0000 7f2e627eb740 1 -- 172.21.15.22:0/755693557 >> 172.21.15.22:0/755693557 conn(0x55913deb9200 msgr2=0x55913deee890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:09:35.162 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:35.159+0000 7f2e627eb740 1 -- 172.21.15.22:0/755693557 shutdown_connections 2024-03-28T18:09:35.162 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:35.159+0000 7f2e627eb740 1 -- 172.21.15.22:0/755693557 wait complete. 2024-03-28T18:09:35.162 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:35.159+0000 7f2e627eb740 20 client.35903 trim_cache size 0 max 16384 2024-03-28T18:09:35.162 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/hid/hid-over-i2c.txt 2024-03-28T18:09:35.164 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/hsi/ 2024-03-28T18:09:35.177 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/hsi/client-devices.txt 2024-03-28T18:09:35.180 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:35.180+0000 7f2e627eb740 1 Processor -- start 2024-03-28T18:09:35.180 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:35.180+0000 7f2e627eb740 1 -- start start 2024-03-28T18:09:35.181 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:35.181+0000 7f2e627eb740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55913def20f0 0x55913df07720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:09:35.181 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:35.181+0000 7f2e627eb740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55913def2dd0 0x55913defa5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:09:35.181 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:35.181+0000 7f2e627eb740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55913defb460 0x55913defab20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:09:35.181 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:35.181+0000 7f2e627eb740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e581af020 con 0x55913defb460 2024-03-28T18:09:35.181 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:35.181+0000 7f2e627eb740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e581b09a0 con 0x55913def2dd0 2024-03-28T18:09:35.181 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:35.181+0000 7f2e627eb740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e581af680 con 0x55913def20f0 2024-03-28T18:09:35.181 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:35.181+0000 7f2e5da00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55913defb460 0x55913defab20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:09:35.181 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:35.181+0000 7f2e5da00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55913defb460 0x55913defab20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:35942/0 (socket says 172.21.15.22:35942) 2024-03-28T18:09:35.181 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:35.181+0000 7f2e5da00640 1 -- 172.21.15.22:0/2900992166 learned_addr learned my addr 172.21.15.22:0/2900992166 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-28T18:09:35.182 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:35.181+0000 7f2e5da00640 1 -- 172.21.15.22:0/2900992166 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55913def20f0 msgr2=0x55913df07720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T18:09:35.182 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:35.181+0000 7f2e5da00640 1 --2- 172.21.15.22:0/2900992166 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55913def20f0 0x55913df07720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:09:35.182 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:35.181+0000 7f2e4fe00640 1 --2- 172.21.15.22:0/2900992166 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55913def2dd0 0x55913defa5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:09:35.182 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:35.181+0000 7f2e5da00640 1 -- 172.21.15.22:0/2900992166 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55913def2dd0 msgr2=0x55913defa5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T18:09:35.182 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:35.181+0000 7f2e5da00640 1 --2- 172.21.15.22:0/2900992166 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55913def2dd0 0x55913defa5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:09:35.183 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:35.181+0000 7f2e5da00640 1 -- 172.21.15.22:0/2900992166 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55913db2a0f0 con 0x55913defb460 2024-03-28T18:09:35.183 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:35.182+0000 7f2e4fe00640 1 --2- 172.21.15.22:0/2900992166 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55913def2dd0 0x55913defa5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T18:09:35.183 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:35.182+0000 7f2e5da00640 1 --2- 172.21.15.22:0/2900992166 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55913defb460 0x55913defab20 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7f2e5001e520 tx=0x7f2e50016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=327dc9a8768e3b4c server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:09:35.183 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:35.182+0000 7f2e4d600640 1 -- 172.21.15.22:0/2900992166 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e5001e0d0 con 0x55913defb460 2024-03-28T18:09:35.183 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/hsi/nokia-modem.txt 2024-03-28T18:09:35.185 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:35.182+0000 7f2e4d600640 1 -- 172.21.15.22:0/2900992166 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f2e5001e270 con 0x55913defb460 2024-03-28T18:09:35.185 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:35.182+0000 7f2e4d600640 1 -- 172.21.15.22:0/2900992166 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e50011660 con 0x55913defb460 2024-03-28T18:09:35.185 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:35.183+0000 7f2e627eb740 1 -- 172.21.15.22:0/2900992166 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55913defb460 msgr2=0x55913defab20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:09:35.185 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:35.183+0000 7f2e627eb740 1 --2- 172.21.15.22:0/2900992166 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55913defb460 0x55913defab20 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7f2e5001e520 tx=0x7f2e50016000 comp rx=0 tx=0).stop 2024-03-28T18:09:35.185 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:35.183+0000 7f2e627eb740 1 -- 172.21.15.22:0/2900992166 shutdown_connections 2024-03-28T18:09:35.185 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:35.183+0000 7f2e627eb740 1 --2- 172.21.15.22:0/2900992166 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55913defb460 0x55913defab20 unknown :-1 s=CLOSED pgs=590 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:09:35.185 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:35.183+0000 7f2e627eb740 1 --2- 172.21.15.22:0/2900992166 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55913def2dd0 0x55913defa5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:09:35.185 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:35.183+0000 7f2e627eb740 1 --2- 172.21.15.22:0/2900992166 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55913def20f0 0x55913df07720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:09:35.185 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:35.183+0000 7f2e627eb740 1 -- 172.21.15.22:0/2900992166 >> 172.21.15.22:0/2900992166 conn(0x55913deb9200 msgr2=0x55913def6780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:09:35.185 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:35.183+0000 7f2e627eb740 1 -- 172.21.15.22:0/2900992166 shutdown_connections 2024-03-28T18:09:35.186 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:35.183+0000 7f2e627eb740 1 -- 172.21.15.22:0/2900992166 wait complete. 2024-03-28T18:09:35.186 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:35.184+0000 7f2e627eb740 1 -- start start 2024-03-28T18:09:35.186 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:35.184+0000 7f2e627eb740 1 Processor -- start 2024-03-28T18:09:35.186 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:35.184+0000 7f2e627eb740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55913def20f0 0x55913def4720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:09:35.186 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:35.184+0000 7f2e627eb740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55913def2dd0 0x55913def4c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:09:35.186 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:35.185+0000 7f2e627eb740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55913defb460 0x55913def51a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:09:35.186 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:35.185+0000 7f2e627eb740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e581b0340 con 0x55913defb460 2024-03-28T18:09:35.186 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:35.185+0000 7f2e627eb740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e581afce0 con 0x55913def20f0 2024-03-28T18:09:35.186 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:35.185+0000 7f2e627eb740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e581ad1e0 con 0x55913def2dd0 2024-03-28T18:09:35.186 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:35.185+0000 7f2e627eb740 10 client.0 _ll_register_callbacks cb 0x7f2e400f4730 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-28T18:09:35.186 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:35.185+0000 7f2e5da00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55913defb460 0x55913def51a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:09:35.186 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:35.185+0000 7f2e5da00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55913defb460 0x55913def51a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:35948/0 (socket says 172.21.15.22:35948) 2024-03-28T18:09:35.186 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:35.185+0000 7f2e5d000640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55913def20f0 0x55913def4720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:09:35.186 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:35.185+0000 7f2e5da00640 1 -- 172.21.15.22:0/1497274580 learned_addr learned my addr 172.21.15.22:0/1497274580 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-28T18:09:35.186 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:35.185+0000 7f2e4fe00640 1 --2- 172.21.15.22:0/1497274580 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55913def2dd0 0x55913def4c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:09:35.187 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:35.185+0000 7f2e5da00640 1 -- 172.21.15.22:0/1497274580 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55913def2dd0 msgr2=0x55913def4c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T18:09:35.187 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:35.185+0000 7f2e5da00640 1 --2- 172.21.15.22:0/1497274580 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55913def2dd0 0x55913def4c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:09:35.187 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:35.185+0000 7f2e5da00640 1 -- 172.21.15.22:0/1497274580 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55913def20f0 msgr2=0x55913def4720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T18:09:35.187 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:35.185+0000 7f2e5da00640 1 --2- 172.21.15.22:0/1497274580 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55913def20f0 0x55913def4720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:09:35.187 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:35.185+0000 7f2e5da00640 1 -- 172.21.15.22:0/1497274580 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55913df658a0 con 0x55913defb460 2024-03-28T18:09:35.187 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:35.185+0000 7f2e5d000640 1 --2- 172.21.15.22:0/1497274580 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55913def20f0 0x55913def4720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-28T18:09:35.189 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:35.186+0000 7f2e5da00640 1 --2- 172.21.15.22:0/1497274580 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55913defb460 0x55913def51a0 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7f2e50018040 tx=0x7f2e5001e160 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3ccb682529180d00 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:09:35.189 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:35.186+0000 7f2e46a00640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-03-28T18:09:35.189 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:35.186+0000 7f2e46a00640 1 -- 172.21.15.22:0/1497274580 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e5001c610 con 0x55913defb460 2024-03-28T18:09:35.190 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:35.186+0000 7f2e627eb740 1 -- 172.21.15.22:0/1497274580 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f2e5001e270 con 0x55913defb460 2024-03-28T18:09:35.190 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:35.186+0000 7f2e46a00640 1 -- 172.21.15.22:0/1497274580 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 8411+0+0 (secure 0 0 0) 0x7f2e5001c7b0 con 0x55913defb460 2024-03-28T18:09:35.190 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:35.186+0000 7f2e46a00640 1 -- 172.21.15.22:0/1497274580 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2e5001c9b0 con 0x55913defb460 2024-03-28T18:09:35.190 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:35.186+0000 7f2e627eb740 10 client.16077 fetch_fsmap learned FSMap version 27 2024-03-28T18:09:35.190 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:35.186+0000 7f2e627eb740 1 -- 172.21.15.22:0/1497274580 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55913df66ae0 con 0x55913defb460 2024-03-28T18:09:35.190 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:35.187+0000 7f2e46a00640 1 -- 172.21.15.22:0/1497274580 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f2e5001e270 con 0x55913defb460 2024-03-28T18:09:35.190 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:35.187+0000 7f2e46a00640 1 -- 172.21.15.22:0/1497274580 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 27) v1 ==== 7934+0+0 (secure 0 0 0) 0x7f2e50039020 con 0x55913defb460 2024-03-28T18:09:35.190 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:35.187+0000 7f2e627eb740 10 client.16077 fetch_fsmap finished waiting for FSMap version 27 2024-03-28T18:09:35.190 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:35.187+0000 7f2e627eb740 10 client.16077 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-28T18:09:35.190 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:35.187+0000 7f2e627eb740 20 client.16077 populate_metadata read hostname 'smithi022' 2024-03-28T18:09:35.190 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:35.187+0000 7f2e627eb740 1 --2- 172.21.15.22:0/1497274580 >> [v2:172.21.15.79:6838/2441542443,v1:172.21.15.79:6839/2441542443] conn(0x55913df67110 0x55913df69500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:09:35.190 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:35.187+0000 7f2e627eb740 4 client.16077 mds_command: new command op to 34553 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-28T18:09:35.190 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:35.187+0000 7f2e627eb740 1 -- 172.21.15.22:0/1497274580 --> [v2:172.21.15.79:6838/2441542443,v1:172.21.15.79:6839/2441542443] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55913de5dae0 con 0x55913df67110 2024-03-28T18:09:35.190 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:35.187+0000 7f2e5d000640 1 --2- 172.21.15.22:0/1497274580 >> [v2:172.21.15.79:6838/2441542443,v1:172.21.15.79:6839/2441542443] conn(0x55913df67110 0x55913df69500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:09:35.191 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:35.188+0000 7f2e5d000640 1 --2- 172.21.15.22:0/1497274580 >> [v2:172.21.15.79:6838/2441542443,v1:172.21.15.79:6839/2441542443] conn(0x55913df67110 0x55913df69500 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d1f2e6af7c45e513 server_cookie=87b05f1779a6da05 in_seq=0 out_seq=0 2024-03-28T18:09:35.191 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:35.188+0000 7f2e46a00640 10 client.16077 ms_handle_connect on v2:172.21.15.79:6838/2441542443 2024-03-28T18:09:35.191 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:35.188+0000 7f2e46a00640 1 -- 172.21.15.22:0/1497274580 <== mds.0 v2:172.21.15.79:6838/2441542443 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x55913de5dae0 con 0x55913df67110 2024-03-28T18:09:35.191 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:35.188+0000 7f2e46a00640 10 client.16077 handle_command_reply: tid=0 2024-03-28T18:09:35.191 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:35.188+0000 7f2e627eb740 1 client.16077 shutdown 2024-03-28T18:09:35.191 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:35.188+0000 7f2e627eb740 1 -- 172.21.15.22:0/1497274580 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55913defb460 msgr2=0x55913def51a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:09:35.191 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:35.188+0000 7f2e627eb740 1 --2- 172.21.15.22:0/1497274580 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55913defb460 0x55913def51a0 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7f2e50018040 tx=0x7f2e5001e160 comp rx=0 tx=0).stop 2024-03-28T18:09:35.191 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:35.188+0000 7f2e627eb740 1 -- 172.21.15.22:0/1497274580 shutdown_connections 2024-03-28T18:09:35.191 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:35.188+0000 7f2e627eb740 1 --2- 172.21.15.22:0/1497274580 >> [v2:172.21.15.79:6838/2441542443,v1:172.21.15.79:6839/2441542443] conn(0x55913df67110 0x55913df69500 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:09:35.191 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:35.188+0000 7f2e627eb740 1 --2- 172.21.15.22:0/1497274580 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55913defb460 0x55913def51a0 unknown :-1 s=CLOSED pgs=591 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:09:35.191 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:35.188+0000 7f2e627eb740 1 --2- 172.21.15.22:0/1497274580 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55913def2dd0 0x55913def4c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:09:35.191 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:35.188+0000 7f2e627eb740 1 --2- 172.21.15.22:0/1497274580 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55913def20f0 0x55913def4720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:09:35.191 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:35.188+0000 7f2e46a00640 0 client.16077 ms_handle_reset on v2:172.21.15.79:6838/2441542443 2024-03-28T18:09:35.191 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:35.188+0000 7f2e627eb740 1 -- 172.21.15.22:0/1497274580 >> 172.21.15.22:0/1497274580 conn(0x55913deb9200 msgr2=0x55913deede70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:09:35.191 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:35.189+0000 7f2e627eb740 1 -- 172.21.15.22:0/1497274580 shutdown_connections 2024-03-28T18:09:35.191 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:35.189+0000 7f2e627eb740 1 -- 172.21.15.22:0/1497274580 wait complete. 2024-03-28T18:09:35.192 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:35.189+0000 7f2e627eb740 20 client.16077 trim_cache size 0 max 16384 2024-03-28T18:09:35.192 INFO:teuthology.orchestra.run.smithi022.stdout:{ 2024-03-28T18:09:35.192 INFO:teuthology.orchestra.run.smithi022.stdout: "status": "no active scrubs running", 2024-03-28T18:09:35.192 INFO:teuthology.orchestra.run.smithi022.stdout: "scrubs": {} 2024-03-28T18:09:35.192 INFO:teuthology.orchestra.run.smithi022.stdout:} 2024-03-28T18:09:35.193 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:35.189+0000 7f2e5e400640 1 -- 172.21.15.22:0/3431071085 >> [v2:172.21.15.22:6800/1357365082,v1:172.21.15.22:6801/1357365082] conn(0x7f2e380a2970 msgr2=0x7f2e380a4e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:09:35.193 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:35.189+0000 7f2e5e400640 1 --2- 172.21.15.22:0/3431071085 >> [v2:172.21.15.22:6800/1357365082,v1:172.21.15.22:6801/1357365082] conn(0x7f2e380a2970 0x7f2e380a4e30 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f2e50005fd0 tx=0x7f2e500096a0 comp rx=0 tx=0).stop 2024-03-28T18:09:35.193 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:35.190+0000 7f2e5e400640 1 -- 172.21.15.22:0/3431071085 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2e58101500 msgr2=0x7f2e58199fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:09:35.193 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:35.190+0000 7f2e5e400640 1 --2- 172.21.15.22:0/3431071085 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2e58101500 0x7f2e58199fc0 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7f2e40000c00 tx=0x7f2e400045e0 comp rx=0 tx=0).stop 2024-03-28T18:09:35.193 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:35.190+0000 7f2e5e400640 1 -- 172.21.15.22:0/3431071085 shutdown_connections 2024-03-28T18:09:35.193 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:35.190+0000 7f2e5e400640 1 --2- 172.21.15.22:0/3431071085 >> [v2:172.21.15.22:6800/1357365082,v1:172.21.15.22:6801/1357365082] conn(0x7f2e380a2970 0x7f2e380a4e30 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:09:35.193 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:35.190+0000 7f2e5e400640 1 --2- 172.21.15.22:0/3431071085 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f2e581075c0 0x7f2e5819e880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:09:35.193 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:35.190+0000 7f2e5e400640 1 --2- 172.21.15.22:0/3431071085 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2e58101ec0 0x7f2e5819a500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:09:35.193 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:35.190+0000 7f2e5e400640 1 --2- 172.21.15.22:0/3431071085 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2e58101500 0x7f2e58199fc0 unknown :-1 s=CLOSED pgs=589 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:09:35.193 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:35.190+0000 7f2e5e400640 1 -- 172.21.15.22:0/3431071085 >> 172.21.15.22:0/3431071085 conn(0x7f2e580fd010 msgr2=0x7f2e580fe5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:09:35.193 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:35.190+0000 7f2e5e400640 1 -- 172.21.15.22:0/3431071085 shutdown_connections 2024-03-28T18:09:35.193 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:35.190+0000 7f2e5e400640 1 -- 172.21.15.22:0/3431071085 wait complete. 2024-03-28T18:09:35.194 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/hsi/omap-ssi.txt 2024-03-28T18:09:35.202 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/hwmon/ 2024-03-28T18:09:35.208 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-03-28T18:09:35.208 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-28T18:09:35.217 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/hwmon/ads1015.txt 2024-03-28T18:09:35.226 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/hwmon/g762.txt 2024-03-28T18:09:35.243 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/hwmon/ibmpowernv.txt 2024-03-28T18:09:35.282 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/hwmon/lm90.txt 2024-03-28T18:09:35.301 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/hwmon/ltc2978.txt 2024-03-28T18:09:35.315 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/hwmon/ntc_thermistor.txt 2024-03-28T18:09:35.327 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/hwmon/pwm-fan.txt 2024-03-28T18:09:35.341 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/hwmon/vexpress.txt 2024-03-28T18:09:35.355 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/hwrng/ 2024-03-28T18:09:35.370 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/hwrng/atmel-trng.txt 2024-03-28T18:09:35.378 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/hwrng/omap_rng.txt 2024-03-28T18:09:35.394 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/hwrng/timeriomem_rng.txt 2024-03-28T18:09:35.409 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/i2c/ 2024-03-28T18:09:35.427 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/i2c/brcm,bcm2835-i2c.txt 2024-03-28T18:09:35.438 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/i2c/brcm,iproc-i2c.txt 2024-03-28T18:09:35.448 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:35.448+0000 7f5d16400640 1 -- 172.21.15.22:0/3588633898 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5d10068800 msgr2=0x7f5d10068c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:09:35.448 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:35.448+0000 7f5d16400640 1 --2- 172.21.15.22:0/3588633898 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5d10068800 0x7f5d10068c80 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7f5cfc009930 tx=0x7f5cfc02bc30 comp rx=0 tx=0).stop 2024-03-28T18:09:35.448 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:35.448+0000 7f5d16400640 1 -- 172.21.15.22:0/3588633898 shutdown_connections 2024-03-28T18:09:35.449 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:35.448+0000 7f5d16400640 1 --2- 172.21.15.22:0/3588633898 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f5d100691c0 0x7f5d1010e310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:09:35.449 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:35.448+0000 7f5d16400640 1 --2- 172.21.15.22:0/3588633898 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5d10068800 0x7f5d10068c80 unknown :-1 s=CLOSED pgs=592 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:09:35.449 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:35.448+0000 7f5d16400640 1 --2- 172.21.15.22:0/3588633898 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5d10107590 0x7f5d10107970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:09:35.449 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:35.448+0000 7f5d16400640 1 -- 172.21.15.22:0/3588633898 >> 172.21.15.22:0/3588633898 conn(0x7f5d10075fb0 msgr2=0x7f5d100763c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:09:35.449 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:35.449+0000 7f5d16400640 1 -- 172.21.15.22:0/3588633898 shutdown_connections 2024-03-28T18:09:35.449 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:35.449+0000 7f5d16400640 1 -- 172.21.15.22:0/3588633898 wait complete. 2024-03-28T18:09:35.449 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:35.450+0000 7f5d16400640 1 Processor -- start 2024-03-28T18:09:35.450 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:35.450+0000 7f5d16400640 1 -- start start 2024-03-28T18:09:35.451 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:35.451+0000 7f5d16400640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f5d10068800 0x7f5d10199fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:09:35.451 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:35.451+0000 7f5d16400640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5d100691c0 0x7f5d1019a510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:09:35.451 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:35.451+0000 7f5d16400640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5d10107590 0x7f5d1019e890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:09:35.451 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:35.451+0000 7f5d16400640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d101118b0 con 0x7f5d100691c0 2024-03-28T18:09:35.451 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:35.451+0000 7f5d16400640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d10111730 con 0x7f5d10068800 2024-03-28T18:09:35.451 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:35.451+0000 7f5d16400640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d10111a30 con 0x7f5d10107590 2024-03-28T18:09:35.451 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:35.451+0000 7f5d15000640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f5d10068800 0x7f5d10199fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:09:35.451 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:35.451+0000 7f5d0fe00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5d100691c0 0x7f5d1019a510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:09:35.451 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:35.451+0000 7f5d15000640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f5d10068800 0x7f5d10199fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.22:57592/0 (socket says 172.21.15.22:57592) 2024-03-28T18:09:35.451 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:35.451+0000 7f5d15000640 1 -- 172.21.15.22:0/868073401 learned_addr learned my addr 172.21.15.22:0/868073401 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-28T18:09:35.451 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:35.451+0000 7f5d15a00640 1 --2- 172.21.15.22:0/868073401 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5d10107590 0x7f5d1019e890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:09:35.452 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:35.451+0000 7f5d15000640 1 -- 172.21.15.22:0/868073401 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5d10107590 msgr2=0x7f5d1019e890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:09:35.452 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:35.451+0000 7f5d15000640 1 --2- 172.21.15.22:0/868073401 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5d10107590 0x7f5d1019e890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:09:35.453 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:35.451+0000 7f5d15000640 1 -- 172.21.15.22:0/868073401 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5d100691c0 msgr2=0x7f5d1019a510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:09:35.453 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:35.451+0000 7f5d15000640 1 --2- 172.21.15.22:0/868073401 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5d100691c0 0x7f5d1019a510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:09:35.453 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:35.452+0000 7f5d15000640 1 -- 172.21.15.22:0/868073401 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d1019efc0 con 0x7f5d10068800 2024-03-28T18:09:35.453 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:35.452+0000 7f5d0fe00640 1 --2- 172.21.15.22:0/868073401 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5d100691c0 0x7f5d1019a510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-28T18:09:35.453 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:35.452+0000 7f5d15a00640 1 --2- 172.21.15.22:0/868073401 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5d10107590 0x7f5d1019e890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-28T18:09:35.453 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:35.452+0000 7f5d15000640 1 --2- 172.21.15.22:0/868073401 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f5d10068800 0x7f5d10199fd0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f5d0400d770 tx=0x7f5d0400dc30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:09:35.453 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:35.452+0000 7f5d0d600640 1 -- 172.21.15.22:0/868073401 <== mon.2 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d04017070 con 0x7f5d10068800 2024-03-28T18:09:35.453 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:35.452+0000 7f5d16400640 1 -- 172.21.15.22:0/868073401 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5d1019f2b0 con 0x7f5d10068800 2024-03-28T18:09:35.453 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:35.452+0000 7f5d0d600640 1 -- 172.21.15.22:0/868073401 <== mon.2 v2:172.21.15.79:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f5d040054f0 con 0x7f5d10068800 2024-03-28T18:09:35.453 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:35.453+0000 7f5d16400640 1 -- 172.21.15.22:0/868073401 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5d101a6b40 con 0x7f5d10068800 2024-03-28T18:09:35.453 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:35.453+0000 7f5d0d600640 1 -- 172.21.15.22:0/868073401 <== mon.2 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d04020060 con 0x7f5d10068800 2024-03-28T18:09:35.454 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/i2c/i2c-arb-gpio-challenge.txt 2024-03-28T18:09:35.454 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:35.454+0000 7f5d16400640 1 -- 172.21.15.22:0/868073401 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5d100683b0 con 0x7f5d10068800 2024-03-28T18:09:35.456 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:35.457+0000 7f5d0d600640 1 -- 172.21.15.22:0/868073401 <== mon.2 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 26) v1 ==== 137421+0+0 (secure 0 0 0) 0x7f5d04009cc0 con 0x7f5d10068800 2024-03-28T18:09:35.457 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:35.458+0000 7f5d0d600640 1 --2- 172.21.15.22:0/868073401 >> [v2:172.21.15.22:6800/1357365082,v1:172.21.15.22:6801/1357365082] conn(0x7f5cf80a2970 0x7f5cf80a4e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:09:35.458 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:35.458+0000 7f5d0d600640 1 -- 172.21.15.22:0/868073401 <== mon.2 v2:172.21.15.79:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 8411+0+0 (secure 0 0 0) 0x7f5d040cec10 con 0x7f5d10068800 2024-03-28T18:09:35.458 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:35.458+0000 7f5d0fe00640 1 --2- 172.21.15.22:0/868073401 >> [v2:172.21.15.22:6800/1357365082,v1:172.21.15.22:6801/1357365082] conn(0x7f5cf80a2970 0x7f5cf80a4e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:09:35.458 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:35.459+0000 7f5d0fe00640 1 --2- 172.21.15.22:0/868073401 >> [v2:172.21.15.22:6800/1357365082,v1:172.21.15.22:6801/1357365082] conn(0x7f5cf80a2970 0x7f5cf80a4e30 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f5cfc036040 tx=0x7f5cfc000bc0 comp rx=0 tx=0).ready entity=mgr.15522 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:09:35.462 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/i2c/i2c-at91.txt 2024-03-28T18:09:35.464 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:35.465+0000 7f5d0d600640 1 -- 172.21.15.22:0/868073401 <== mon.2 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f5d0409b980 con 0x7f5d10068800 2024-03-28T18:09:35.472 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/i2c/i2c-axxia.txt 2024-03-28T18:09:35.482 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/i2c/i2c-bcm-kona.txt 2024-03-28T18:09:35.494 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/i2c/i2c-cadence.txt 2024-03-28T18:09:35.506 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/i2c/i2c-cbus-gpio.txt 2024-03-28T18:09:35.518 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/i2c/i2c-cros-ec-tunnel.txt 2024-03-28T18:09:35.530 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/i2c/i2c-davinci.txt 2024-03-28T18:09:35.545 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/i2c/i2c-designware.txt 2024-03-28T18:09:35.558 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/i2c/i2c-efm32.txt 2024-03-28T18:09:35.572 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/i2c/i2c-exynos5.txt 2024-03-28T18:09:35.587 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/i2c/i2c-gpio.txt 2024-03-28T18:09:35.600 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/i2c/i2c-hix5hd2.txt 2024-03-28T18:09:35.614 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/i2c/i2c-img-scb.txt 2024-03-28T18:09:35.631 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/i2c/i2c-imx.txt 2024-03-28T18:09:35.645 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/i2c/i2c-meson.txt 2024-03-28T18:09:35.659 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/i2c/i2c-mpc.txt 2024-03-28T18:09:35.674 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/i2c/i2c-mux-gpio.txt 2024-03-28T18:09:35.689 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/i2c/i2c-mux-pca954x.txt 2024-03-28T18:09:35.703 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/i2c/i2c-mux-pinctrl.txt 2024-03-28T18:09:35.716 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/i2c/i2c-mux.txt 2024-03-28T18:09:35.730 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/i2c/i2c-mv64xxx.txt 2024-03-28T18:09:35.737 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:35.737+0000 7f5d16400640 1 -- 172.21.15.22:0/868073401 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f5d101a6d70 con 0x7f5d10068800 2024-03-28T18:09:35.739 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:35.739+0000 7f5d0d600640 1 -- 172.21.15.22:0/868073401 <== mon.2 v2:172.21.15.79:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) v1 ==== 94+0+11452 (secure 0 0 0) 0x7f5d0409b280 con 0x7f5d10068800 2024-03-28T18:09:35.739 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-03-28T18:09:35.739 INFO:teuthology.orchestra.run.smithi022.stdout:{"epoch":27,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14544,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6835/3718238950","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6834","nonce":3718238950},{"type":"v1","addr":"172.21.15.22:6835","nonce":3718238950}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14562,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6837/3676939386","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6836","nonce":3676939386},{"type":"v1","addr":"172.21.15.22:6837","nonce":3676939386}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":27,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-28T18:04:48.991622+0000","modified":"2024-03-28T18:07:13.521383+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":34553,"mds_1":34511,"mds_2":34475,"mds_3":34442,"mds_4":24502},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14640":{"gid":14640,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.22:6839/3979998310","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6838","nonce":3979998310},{"type":"v1","addr":"172.21.15.22:6839","nonce":3979998310}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_14655":{"gid":14655,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.22:6841/3069056661","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6840","nonce":3069056661},{"type":"v1","addr":"172.21.15.22:6841","nonce":3069056661}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24412":{"gid":24412,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.134:6833/3176125362","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6832","nonce":3176125362},{"type":"v1","addr":"172.21.15.134:6833","nonce":3176125362}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24463":{"gid":24463,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.134:6835/3370549207","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6834","nonce":3370549207},{"type":"v1","addr":"172.21.15.134:6835","nonce":3370549207}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24478":{"gid":24478,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.134:6837/899825581","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6836","nonce":899825581},{"type":"v1","addr":"172.21.15.134:6837","nonce":899825581}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24502":{"gid":24502,"name":"i","rank":4,"incarnation":24,"state":"up:active","state_seq":7,"addr":"172.21.15.134:6839/3400263217","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6838","nonce":3400263217},{"type":"v1","addr":"172.21.15.134:6839","nonce":3400263217}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_34442":{"gid":34442,"name":"k","rank":3,"incarnation":22,"state":"up:active","state_seq":14,"addr":"172.21.15.79:6833/856896997","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6832","nonce":856896997},{"type":"v1","addr":"172.21.15.79:6833","nonce":856896997}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_34475":{"gid":34475,"name":"e","rank":2,"incarnation":20,"state":"up:active","state_seq":11,"addr":"172.21.15.79:6835/3365773431","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6834","nonce":3365773431},{"type":"v1","addr":"172.21.15.79:6835","nonce":3365773431}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_34511":{"gid":34511,"name":"h","rank":1,"incarnation":18,"state":"up:active","state_seq":8,"addr":"172.21.15.79:6837/1787903366","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6836","nonce":1787903366},{"type":"v1","addr":"172.21.15.79:6837","nonce":1787903366}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_34553":{"gid":34553,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.79:6839/2441542443","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6838","nonce":2441542443},{"type":"v1","addr":"172.21.15.79:6839","nonce":2441542443}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":34553,"qdb_cluster":[24502,34442,34475,34511,34553]},"id":1}]} 2024-03-28T18:09:35.739 INFO:teuthology.orchestra.run.smithi022.stderr:dumped fsmap epoch 27 2024-03-28T18:09:35.741 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:35.742+0000 7f5d16400640 1 -- 172.21.15.22:0/868073401 >> [v2:172.21.15.22:6800/1357365082,v1:172.21.15.22:6801/1357365082] conn(0x7f5cf80a2970 msgr2=0x7f5cf80a4e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:09:35.741 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:35.742+0000 7f5d16400640 1 --2- 172.21.15.22:0/868073401 >> [v2:172.21.15.22:6800/1357365082,v1:172.21.15.22:6801/1357365082] conn(0x7f5cf80a2970 0x7f5cf80a4e30 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f5cfc036040 tx=0x7f5cfc000bc0 comp rx=0 tx=0).stop 2024-03-28T18:09:35.741 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:35.742+0000 7f5d16400640 1 -- 172.21.15.22:0/868073401 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f5d10068800 msgr2=0x7f5d10199fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:09:35.742 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:35.742+0000 7f5d16400640 1 --2- 172.21.15.22:0/868073401 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f5d10068800 0x7f5d10199fd0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f5d0400d770 tx=0x7f5d0400dc30 comp rx=0 tx=0).stop 2024-03-28T18:09:35.742 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:35.742+0000 7f5d16400640 1 -- 172.21.15.22:0/868073401 shutdown_connections 2024-03-28T18:09:35.742 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:35.742+0000 7f5d16400640 1 --2- 172.21.15.22:0/868073401 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5d10107590 0x7f5d1019e890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:09:35.742 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:35.742+0000 7f5d16400640 1 --2- 172.21.15.22:0/868073401 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5d100691c0 0x7f5d1019a510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:09:35.742 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:35.742+0000 7f5d16400640 1 --2- 172.21.15.22:0/868073401 >> [v2:172.21.15.22:6800/1357365082,v1:172.21.15.22:6801/1357365082] conn(0x7f5cf80a2970 0x7f5cf80a4e30 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:09:35.742 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:35.742+0000 7f5d16400640 1 --2- 172.21.15.22:0/868073401 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f5d10068800 0x7f5d10199fd0 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:09:35.742 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:35.742+0000 7f5d16400640 1 -- 172.21.15.22:0/868073401 >> 172.21.15.22:0/868073401 conn(0x7f5d10075fb0 msgr2=0x7f5d100fefa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:09:35.742 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:35.742+0000 7f5d16400640 1 -- 172.21.15.22:0/868073401 shutdown_connections 2024-03-28T18:09:35.742 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:35.742+0000 7f5d16400640 1 -- 172.21.15.22:0/868073401 wait complete. 2024-03-28T18:09:35.744 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/i2c/i2c-mxs.txt 2024-03-28T18:09:35.756 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/i2c/i2c-nomadik.txt 2024-03-28T18:09:35.763 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-28T18:09:35.763 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2024-03-28T18:09:35.766 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/i2c/i2c-ocores.txt 2024-03-28T18:09:35.778 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/i2c/i2c-octeon.txt 2024-03-28T18:09:35.789 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/i2c/i2c-omap.txt 2024-03-28T18:09:35.800 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/i2c/i2c-opal.txt 2024-03-28T18:09:35.812 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/i2c/i2c-pnx.txt 2024-03-28T18:09:35.822 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/i2c/i2c-pxa-pci-ce4100.txt 2024-03-28T18:09:35.834 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/i2c/i2c-pxa.txt 2024-03-28T18:09:35.850 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/i2c/i2c-rcar.txt 2024-03-28T18:09:35.864 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/i2c/i2c-riic.txt 2024-03-28T18:09:35.877 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/i2c/i2c-rk3x.txt 2024-03-28T18:09:35.889 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/i2c/i2c-s3c2410.txt 2024-03-28T18:09:35.902 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/i2c/i2c-sh_mobile.txt 2024-03-28T18:09:35.915 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/i2c/i2c-sirf.txt 2024-03-28T18:09:35.929 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/i2c/i2c-st-ddci2c.txt 2024-03-28T18:09:35.942 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/i2c/i2c-st.txt 2024-03-28T18:09:35.954 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/i2c/i2c-sunxi-p2wi.txt 2024-03-28T18:09:35.969 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/i2c/i2c-versatile.txt 2024-03-28T18:09:35.985 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/i2c/i2c-vt8500.txt 2024-03-28T18:09:35.998 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/i2c/i2c-xiic.txt 2024-03-28T18:09:36.009 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.009+0000 7fc18be00640 1 -- 172.21.15.22:0/2132640053 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc18c101530 msgr2=0x7fc18c1019b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:09:36.009 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.009+0000 7fc18be00640 1 --2- 172.21.15.22:0/2132640053 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc18c101530 0x7fc18c1019b0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fc174009cd0 tx=0x7fc17402c0e0 comp rx=0 tx=0).stop 2024-03-28T18:09:36.009 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.009+0000 7fc18be00640 1 -- 172.21.15.22:0/2132640053 shutdown_connections 2024-03-28T18:09:36.009 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.009+0000 7fc18be00640 1 --2- 172.21.15.22:0/2132640053 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc18c101ef0 0x7fc18c10e3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:09:36.009 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.009+0000 7fc18be00640 1 --2- 172.21.15.22:0/2132640053 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc18c101530 0x7fc18c1019b0 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:09:36.009 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.009+0000 7fc18be00640 1 --2- 172.21.15.22:0/2132640053 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc18c1075f0 0x7fc18c1079d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:09:36.009 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.009+0000 7fc18be00640 1 -- 172.21.15.22:0/2132640053 >> 172.21.15.22:0/2132640053 conn(0x7fc18c0fd040 msgr2=0x7fc18c0ff460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:09:36.009 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.009+0000 7fc18be00640 1 -- 172.21.15.22:0/2132640053 shutdown_connections 2024-03-28T18:09:36.010 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.009+0000 7fc18be00640 1 -- 172.21.15.22:0/2132640053 wait complete. 2024-03-28T18:09:36.010 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/i2c/ina209.txt 2024-03-28T18:09:36.010 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.011+0000 7fc18be00640 1 Processor -- start 2024-03-28T18:09:36.010 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.011+0000 7fc18be00640 1 -- start start 2024-03-28T18:09:36.012 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.012+0000 7fc18be00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc18c101530 0x7fc18c199fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:09:36.012 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.012+0000 7fc18be00640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc18c101ef0 0x7fc18c19a500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:09:36.012 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.012+0000 7fc18be00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc18c1075f0 0x7fc18c19e880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:09:36.012 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.012+0000 7fc18be00640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc18c1118c0 con 0x7fc18c101530 2024-03-28T18:09:36.012 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.012+0000 7fc18be00640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc18c111740 con 0x7fc18c101ef0 2024-03-28T18:09:36.012 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.012+0000 7fc18be00640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc18c111a40 con 0x7fc18c1075f0 2024-03-28T18:09:36.013 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.012+0000 7fc18aa00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc18c101530 0x7fc18c199fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:09:36.013 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.012+0000 7fc18aa00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc18c101530 0x7fc18c199fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:35974/0 (socket says 172.21.15.22:35974) 2024-03-28T18:09:36.013 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.012+0000 7fc18aa00640 1 -- 172.21.15.22:0/3853924037 learned_addr learned my addr 172.21.15.22:0/3853924037 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-28T18:09:36.013 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.012+0000 7fc18a000640 1 --2- 172.21.15.22:0/3853924037 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc18c101ef0 0x7fc18c19a500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:09:36.013 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.012+0000 7fc18b400640 1 --2- 172.21.15.22:0/3853924037 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc18c1075f0 0x7fc18c19e880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:09:36.013 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.012+0000 7fc18aa00640 1 -- 172.21.15.22:0/3853924037 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc18c1075f0 msgr2=0x7fc18c19e880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:09:36.013 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.012+0000 7fc18aa00640 1 --2- 172.21.15.22:0/3853924037 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc18c1075f0 0x7fc18c19e880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:09:36.013 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.012+0000 7fc18aa00640 1 -- 172.21.15.22:0/3853924037 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc18c101ef0 msgr2=0x7fc18c19a500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:09:36.013 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.012+0000 7fc18aa00640 1 --2- 172.21.15.22:0/3853924037 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc18c101ef0 0x7fc18c19a500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:09:36.013 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.012+0000 7fc18aa00640 1 -- 172.21.15.22:0/3853924037 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc18c19efb0 con 0x7fc18c101530 2024-03-28T18:09:36.014 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.013+0000 7fc18aa00640 1 --2- 172.21.15.22:0/3853924037 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc18c101530 0x7fc18c199fc0 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7fc18400e9f0 tx=0x7fc18400edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:09:36.014 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.013+0000 7fc183400640 1 -- 172.21.15.22:0/3853924037 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc18400cd80 con 0x7fc18c101530 2024-03-28T18:09:36.014 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.013+0000 7fc18be00640 1 -- 172.21.15.22:0/3853924037 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc18c19f2a0 con 0x7fc18c101530 2024-03-28T18:09:36.014 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.013+0000 7fc18be00640 1 -- 172.21.15.22:0/3853924037 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc18c1a6b30 con 0x7fc18c101530 2024-03-28T18:09:36.014 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.014+0000 7fc183400640 1 -- 172.21.15.22:0/3853924037 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7fc1840054f0 con 0x7fc18c101530 2024-03-28T18:09:36.014 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.014+0000 7fc183400640 1 -- 172.21.15.22:0/3853924037 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc184010690 con 0x7fc18c101530 2024-03-28T18:09:36.016 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.017+0000 7fc183400640 1 -- 172.21.15.22:0/3853924037 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 26) v1 ==== 137421+0+0 (secure 0 0 0) 0x7fc1840046c0 con 0x7fc18c101530 2024-03-28T18:09:36.017 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.017+0000 7fc194394740 1 Processor -- start 2024-03-28T18:09:36.017 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.017+0000 7fc194394740 1 -- start start 2024-03-28T18:09:36.018 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.017+0000 7fc194394740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562e4b40f8b0 0x562e4b40fc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:09:36.018 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.017+0000 7fc194394740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x562e4b410260 0x562e4b418810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:09:36.018 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.017+0000 7fc194394740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562e4b418de0 0x562e4b41b230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:09:36.018 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.017+0000 7fc194394740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x562e4b3732c0 con 0x562e4b418de0 2024-03-28T18:09:36.018 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.017+0000 7fc194394740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x562e4b31bc90 con 0x562e4b410260 2024-03-28T18:09:36.018 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.017+0000 7fc194394740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x562e4b36f360 con 0x562e4b40f8b0 2024-03-28T18:09:36.018 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.018+0000 7fc18aa00640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x562e4b410260 0x562e4b418810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:09:36.018 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.018+0000 7fc18a000640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562e4b418de0 0x562e4b41b230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:09:36.018 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.018+0000 7fc18a000640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562e4b418de0 0x562e4b41b230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:35978/0 (socket says 172.21.15.22:35978) 2024-03-28T18:09:36.018 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.018+0000 7fc18aa00640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x562e4b410260 0x562e4b418810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.22:57620/0 (socket says 172.21.15.22:57620) 2024-03-28T18:09:36.018 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.018+0000 7fc18a000640 1 -- 172.21.15.22:0/2389221740 learned_addr learned my addr 172.21.15.22:0/2389221740 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-28T18:09:36.018 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.018+0000 7fc18b400640 1 --2- 172.21.15.22:0/2389221740 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562e4b40f8b0 0x562e4b40fc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:09:36.020 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.018+0000 7fc18a000640 1 -- 172.21.15.22:0/2389221740 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562e4b40f8b0 msgr2=0x562e4b40fc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T18:09:36.020 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.018+0000 7fc18a000640 1 --2- 172.21.15.22:0/2389221740 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562e4b40f8b0 0x562e4b40fc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:09:36.020 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.018+0000 7fc18a000640 1 -- 172.21.15.22:0/2389221740 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x562e4b410260 msgr2=0x562e4b418810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T18:09:36.020 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.018+0000 7fc18a000640 1 --2- 172.21.15.22:0/2389221740 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x562e4b410260 0x562e4b418810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:09:36.020 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.019+0000 7fc18a000640 1 -- 172.21.15.22:0/2389221740 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562e4b340080 con 0x562e4b418de0 2024-03-28T18:09:36.020 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.019+0000 7fc18aa00640 1 --2- 172.21.15.22:0/2389221740 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x562e4b410260 0x562e4b418810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-28T18:09:36.020 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.019+0000 7fc183400640 1 --2- 172.21.15.22:0/3853924037 >> [v2:172.21.15.22:6800/1357365082,v1:172.21.15.22:6801/1357365082] conn(0x7fc1700a29f0 0x7fc1700a4eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:09:36.020 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.019+0000 7fc18a000640 1 --2- 172.21.15.22:0/2389221740 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562e4b418de0 0x562e4b41b230 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7fc174004af0 tx=0x7fc174009880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1844010b50318a5e server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:09:36.020 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.019+0000 7fc183400640 1 -- 172.21.15.22:0/3853924037 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 8411+0+0 (secure 0 0 0) 0x7fc1840cf0f0 con 0x7fc18c101530 2024-03-28T18:09:36.020 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.019+0000 7fc18b400640 1 --2- 172.21.15.22:0/3853924037 >> [v2:172.21.15.22:6800/1357365082,v1:172.21.15.22:6801/1357365082] conn(0x7fc1700a29f0 0x7fc1700a4eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:09:36.020 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.019+0000 7fc17b400640 1 -- 172.21.15.22:0/2389221740 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc174045020 con 0x562e4b418de0 2024-03-28T18:09:36.020 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.019+0000 7fc17b400640 1 -- 172.21.15.22:0/2389221740 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7fc17403a040 con 0x562e4b418de0 2024-03-28T18:09:36.021 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.019+0000 7fc17b400640 1 -- 172.21.15.22:0/2389221740 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc174039040 con 0x562e4b418de0 2024-03-28T18:09:36.021 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/i2c/ina2xx.txt 2024-03-28T18:09:36.023 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.021+0000 7fc18b400640 1 --2- 172.21.15.22:0/3853924037 >> [v2:172.21.15.22:6800/1357365082,v1:172.21.15.22:6801/1357365082] conn(0x7fc1700a29f0 0x7fc1700a4eb0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fc17c00bca0 tx=0x7fc17c00b450 comp rx=0 tx=0).ready entity=mgr.15522 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:09:36.023 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.021+0000 7fc194394740 1 -- 172.21.15.22:0/2389221740 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562e4b418de0 msgr2=0x562e4b41b230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:09:36.023 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.021+0000 7fc194394740 1 --2- 172.21.15.22:0/2389221740 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562e4b418de0 0x562e4b41b230 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7fc174004af0 tx=0x7fc174009880 comp rx=0 tx=0).stop 2024-03-28T18:09:36.023 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.021+0000 7fc194394740 1 -- 172.21.15.22:0/2389221740 shutdown_connections 2024-03-28T18:09:36.023 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.021+0000 7fc194394740 1 --2- 172.21.15.22:0/2389221740 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562e4b418de0 0x562e4b41b230 unknown :-1 s=CLOSED pgs=594 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:09:36.023 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.021+0000 7fc194394740 1 --2- 172.21.15.22:0/2389221740 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x562e4b410260 0x562e4b418810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:09:36.023 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.021+0000 7fc194394740 1 --2- 172.21.15.22:0/2389221740 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562e4b40f8b0 0x562e4b40fc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:09:36.023 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.021+0000 7fc194394740 1 -- 172.21.15.22:0/2389221740 >> 172.21.15.22:0/2389221740 conn(0x562e4b409f80 msgr2=0x562e4b40a360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:09:36.023 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.021+0000 7fc194394740 1 -- 172.21.15.22:0/2389221740 shutdown_connections 2024-03-28T18:09:36.023 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.021+0000 7fc194394740 1 -- 172.21.15.22:0/2389221740 wait complete. 2024-03-28T18:09:36.024 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.022+0000 7fc194394740 1 -- start start 2024-03-28T18:09:36.024 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.022+0000 7fc194394740 1 Processor -- start 2024-03-28T18:09:36.024 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.022+0000 7fc194394740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562e4b40f8b0 0x562e4b418720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:09:36.024 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.022+0000 7fc194394740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562e4b410260 0x562e4b415ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:09:36.024 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.022+0000 7fc194394740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x562e4b418de0 0x562e4b4160c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:09:36.024 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.022+0000 7fc194394740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x562e4b41bcf0 con 0x562e4b410260 2024-03-28T18:09:36.024 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.022+0000 7fc194394740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x562e4b32f0d0 con 0x562e4b418de0 2024-03-28T18:09:36.024 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.022+0000 7fc194394740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x562e4b41be70 con 0x562e4b40f8b0 2024-03-28T18:09:36.024 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.022+0000 7fc194394740 10 client.0 _ll_register_callbacks cb 0x562e4b333e80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-28T18:09:36.024 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.023+0000 7fc18a000640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562e4b410260 0x562e4b415ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:09:36.024 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.023+0000 7fc18b400640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x562e4b418de0 0x562e4b4160c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:09:36.024 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.023+0000 7fc18a000640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562e4b410260 0x562e4b415ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:35992/0 (socket says 172.21.15.22:35992) 2024-03-28T18:09:36.024 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.023+0000 7fc18a000640 1 -- 172.21.15.22:0/818753794 learned_addr learned my addr 172.21.15.22:0/818753794 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-28T18:09:36.024 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.023+0000 7fc18aa00640 1 --2- 172.21.15.22:0/818753794 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562e4b40f8b0 0x562e4b418720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:09:36.024 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.023+0000 7fc18a000640 1 -- 172.21.15.22:0/818753794 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562e4b40f8b0 msgr2=0x562e4b418720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T18:09:36.025 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.023+0000 7fc18a000640 1 --2- 172.21.15.22:0/818753794 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562e4b40f8b0 0x562e4b418720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:09:36.025 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.023+0000 7fc18a000640 1 -- 172.21.15.22:0/818753794 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x562e4b418de0 msgr2=0x562e4b4160c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T18:09:36.025 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.023+0000 7fc18a000640 1 --2- 172.21.15.22:0/818753794 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x562e4b418de0 0x562e4b4160c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:09:36.025 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.023+0000 7fc18a000640 1 -- 172.21.15.22:0/818753794 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x562e4b0873f0 con 0x562e4b410260 2024-03-28T18:09:36.025 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.023+0000 7fc18b400640 1 --2- 172.21.15.22:0/818753794 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x562e4b418de0 0x562e4b4160c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-28T18:09:36.027 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.024+0000 7fc18a000640 1 --2- 172.21.15.22:0/818753794 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562e4b410260 0x562e4b415ab0 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7fc17402c620 tx=0x7fc17402c7e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b99d363e9a36b6cc server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:09:36.027 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.024+0000 7fc16fe00640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-03-28T18:09:36.027 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.024+0000 7fc16fe00640 1 -- 172.21.15.22:0/818753794 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc174050070 con 0x562e4b410260 2024-03-28T18:09:36.027 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.024+0000 7fc16fe00640 1 -- 172.21.15.22:0/818753794 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 8411+0+0 (secure 0 0 0) 0x7fc1740325d0 con 0x562e4b410260 2024-03-28T18:09:36.027 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.024+0000 7fc194394740 1 -- 172.21.15.22:0/818753794 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fc17403a040 con 0x562e4b410260 2024-03-28T18:09:36.027 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.024+0000 7fc16fe00640 1 -- 172.21.15.22:0/818753794 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc17405b050 con 0x562e4b410260 2024-03-28T18:09:36.027 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.024+0000 7fc194394740 10 client.16095 fetch_fsmap learned FSMap version 27 2024-03-28T18:09:36.027 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.024+0000 7fc194394740 1 -- 172.21.15.22:0/818753794 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x562e4b1517d0 con 0x562e4b410260 2024-03-28T18:09:36.027 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.025+0000 7fc16fe00640 1 -- 172.21.15.22:0/818753794 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7fc17403a040 con 0x562e4b410260 2024-03-28T18:09:36.027 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.025+0000 7fc16fe00640 1 -- 172.21.15.22:0/818753794 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 27) v1 ==== 7934+0+0 (secure 0 0 0) 0x7fc17404c440 con 0x562e4b410260 2024-03-28T18:09:36.027 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.025+0000 7fc194394740 10 client.16095 fetch_fsmap finished waiting for FSMap version 27 2024-03-28T18:09:36.028 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.025+0000 7fc194394740 10 client.16095 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.i 2024-03-28T18:09:36.028 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.025+0000 7fc194394740 20 client.16095 populate_metadata read hostname 'smithi022' 2024-03-28T18:09:36.028 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.025+0000 7fc194394740 1 --2- 172.21.15.22:0/818753794 >> [v2:172.21.15.134:6838/3400263217,v1:172.21.15.134:6839/3400263217] conn(0x562e4b428ab0 0x562e4b42aea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:09:36.028 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.025+0000 7fc194394740 4 client.16095 mds_command: new command op to 24502 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-28T18:09:36.028 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.025+0000 7fc194394740 1 -- 172.21.15.22:0/818753794 --> [v2:172.21.15.134:6838/3400263217,v1:172.21.15.134:6839/3400263217] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x562e4b380800 con 0x562e4b428ab0 2024-03-28T18:09:36.028 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.026+0000 7fc18aa00640 1 --2- 172.21.15.22:0/818753794 >> [v2:172.21.15.134:6838/3400263217,v1:172.21.15.134:6839/3400263217] conn(0x562e4b428ab0 0x562e4b42aea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:09:36.028 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.027+0000 7fc18aa00640 1 --2- 172.21.15.22:0/818753794 >> [v2:172.21.15.134:6838/3400263217,v1:172.21.15.134:6839/3400263217] conn(0x562e4b428ab0 0x562e4b42aea0 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=f491f004e005bd73 server_cookie=5f8d756c81fc211f in_seq=0 out_seq=0 2024-03-28T18:09:36.028 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.027+0000 7fc16fe00640 10 client.16095 ms_handle_connect on v2:172.21.15.134:6838/3400263217 2024-03-28T18:09:36.029 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.028+0000 7fc16fe00640 1 -- 172.21.15.22:0/818753794 <== mds.4 v2:172.21.15.134:6838/3400263217 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x562e4b380800 con 0x562e4b428ab0 2024-03-28T18:09:36.029 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.028+0000 7fc16fe00640 10 client.16095 handle_command_reply: tid=0 2024-03-28T18:09:36.029 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.028+0000 7fc194394740 1 client.16095 shutdown 2024-03-28T18:09:36.029 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.028+0000 7fc194394740 1 -- 172.21.15.22:0/818753794 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562e4b410260 msgr2=0x562e4b415ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:09:36.030 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.028+0000 7fc194394740 1 --2- 172.21.15.22:0/818753794 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562e4b410260 0x562e4b415ab0 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7fc17402c620 tx=0x7fc17402c7e0 comp rx=0 tx=0).stop 2024-03-28T18:09:36.030 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.028+0000 7fc194394740 1 -- 172.21.15.22:0/818753794 shutdown_connections 2024-03-28T18:09:36.030 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.028+0000 7fc194394740 1 --2- 172.21.15.22:0/818753794 >> [v2:172.21.15.134:6838/3400263217,v1:172.21.15.134:6839/3400263217] conn(0x562e4b428ab0 0x562e4b42aea0 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:09:36.030 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.028+0000 7fc194394740 1 --2- 172.21.15.22:0/818753794 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x562e4b418de0 0x562e4b4160c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:09:36.030 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.028+0000 7fc194394740 1 --2- 172.21.15.22:0/818753794 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562e4b410260 0x562e4b415ab0 unknown :-1 s=CLOSED pgs=595 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:09:36.030 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.028+0000 7fc194394740 1 --2- 172.21.15.22:0/818753794 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562e4b40f8b0 0x562e4b418720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:09:36.030 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.028+0000 7fc16fe00640 0 client.16095 ms_handle_reset on v2:172.21.15.134:6838/3400263217 2024-03-28T18:09:36.030 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.028+0000 7fc194394740 1 -- 172.21.15.22:0/818753794 >> 172.21.15.22:0/818753794 conn(0x562e4b409f80 msgr2=0x562e4b40bcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:09:36.030 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.029+0000 7fc194394740 1 -- 172.21.15.22:0/818753794 shutdown_connections 2024-03-28T18:09:36.030 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.029+0000 7fc194394740 1 -- 172.21.15.22:0/818753794 wait complete. 2024-03-28T18:09:36.030 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.029+0000 7fc194394740 20 client.16095 trim_cache size 0 max 16384 2024-03-28T18:09:36.031 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/i2c/max6697.txt 2024-03-28T18:09:36.038 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/i2c/nvidia,tegra20-i2c.txt 2024-03-28T18:09:36.044 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.045+0000 7fc194394740 1 Processor -- start 2024-03-28T18:09:36.044 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.045+0000 7fc194394740 1 -- start start 2024-03-28T18:09:36.045 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.045+0000 7fc194394740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x562e4b40f8b0 0x562e4b40ba50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:09:36.045 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.045+0000 7fc194394740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562e4b410260 0x562e4b4181e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:09:36.045 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.045+0000 7fc194394740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562e4b418de0 0x562e4b418720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:09:36.045 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.045+0000 7fc194394740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc168016b60 con 0x562e4b410260 2024-03-28T18:09:36.045 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.045+0000 7fc194394740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1680184e0 con 0x562e4b40f8b0 2024-03-28T18:09:36.045 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.045+0000 7fc194394740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1680171c0 con 0x562e4b418de0 2024-03-28T18:09:36.046 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.045+0000 7fc18a000640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562e4b410260 0x562e4b4181e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:09:36.046 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.045+0000 7fc18aa00640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x562e4b40f8b0 0x562e4b40ba50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:09:36.046 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.045+0000 7fc18a000640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562e4b410260 0x562e4b4181e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:36004/0 (socket says 172.21.15.22:36004) 2024-03-28T18:09:36.047 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.045+0000 7fc18a000640 1 -- 172.21.15.22:0/1054214070 learned_addr learned my addr 172.21.15.22:0/1054214070 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-28T18:09:36.047 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.046+0000 7fc18b400640 1 --2- 172.21.15.22:0/1054214070 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562e4b418de0 0x562e4b418720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:09:36.047 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.046+0000 7fc18a000640 1 -- 172.21.15.22:0/1054214070 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562e4b418de0 msgr2=0x562e4b418720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T18:09:36.047 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.046+0000 7fc18a000640 1 --2- 172.21.15.22:0/1054214070 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562e4b418de0 0x562e4b418720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:09:36.047 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.046+0000 7fc18a000640 1 -- 172.21.15.22:0/1054214070 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x562e4b40f8b0 msgr2=0x562e4b40ba50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T18:09:36.047 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.046+0000 7fc18a000640 1 --2- 172.21.15.22:0/1054214070 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x562e4b40f8b0 0x562e4b40ba50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:09:36.047 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.046+0000 7fc18a000640 1 -- 172.21.15.22:0/1054214070 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562e4b4166c0 con 0x562e4b410260 2024-03-28T18:09:36.047 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.046+0000 7fc18aa00640 1 --2- 172.21.15.22:0/1054214070 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x562e4b40f8b0 0x562e4b40ba50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-28T18:09:36.047 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.046+0000 7fc18a000640 1 --2- 172.21.15.22:0/1054214070 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562e4b410260 0x562e4b4181e0 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7fc174002230 tx=0x7fc1740093e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=de8c5e3da8d06cb1 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:09:36.047 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.046+0000 7fc17aa00640 1 -- 172.21.15.22:0/1054214070 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc174050040 con 0x562e4b410260 2024-03-28T18:09:36.047 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.046+0000 7fc17aa00640 1 -- 172.21.15.22:0/1054214070 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7fc17404c950 con 0x562e4b410260 2024-03-28T18:09:36.047 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.047+0000 7fc17aa00640 1 -- 172.21.15.22:0/1054214070 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc174039040 con 0x562e4b410260 2024-03-28T18:09:36.050 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.047+0000 7fc194394740 1 -- 172.21.15.22:0/1054214070 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562e4b410260 msgr2=0x562e4b4181e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:09:36.050 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.047+0000 7fc194394740 1 --2- 172.21.15.22:0/1054214070 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562e4b410260 0x562e4b4181e0 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7fc174002230 tx=0x7fc1740093e0 comp rx=0 tx=0).stop 2024-03-28T18:09:36.050 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.047+0000 7fc194394740 1 -- 172.21.15.22:0/1054214070 shutdown_connections 2024-03-28T18:09:36.050 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.047+0000 7fc194394740 1 --2- 172.21.15.22:0/1054214070 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562e4b418de0 0x562e4b418720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:09:36.050 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.047+0000 7fc194394740 1 --2- 172.21.15.22:0/1054214070 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562e4b410260 0x562e4b4181e0 unknown :-1 s=CLOSED pgs=596 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:09:36.050 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.047+0000 7fc194394740 1 --2- 172.21.15.22:0/1054214070 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x562e4b40f8b0 0x562e4b40ba50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:09:36.050 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.047+0000 7fc194394740 1 -- 172.21.15.22:0/1054214070 >> 172.21.15.22:0/1054214070 conn(0x562e4b409f80 msgr2=0x562e4b40b340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:09:36.050 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.047+0000 7fc194394740 1 -- 172.21.15.22:0/1054214070 shutdown_connections 2024-03-28T18:09:36.050 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.048+0000 7fc194394740 1 -- 172.21.15.22:0/1054214070 wait complete. 2024-03-28T18:09:36.051 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.048+0000 7fc194394740 1 -- start start 2024-03-28T18:09:36.051 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.048+0000 7fc194394740 1 Processor -- start 2024-03-28T18:09:36.051 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.049+0000 7fc194394740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562e4b40f8b0 0x562e4b420cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:09:36.051 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.049+0000 7fc194394740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x562e4b410260 0x562e4b41f090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:09:36.051 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.049+0000 7fc194394740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562e4b418de0 0x562e4b41f5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:09:36.051 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.049+0000 7fc194394740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc168017e80 con 0x562e4b40f8b0 2024-03-28T18:09:36.051 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.049+0000 7fc194394740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc168017820 con 0x562e4b410260 2024-03-28T18:09:36.051 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.049+0000 7fc194394740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc168014d20 con 0x562e4b418de0 2024-03-28T18:09:36.051 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.049+0000 7fc194394740 10 client.0 _ll_register_callbacks cb 0x7fc1840d2b10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-28T18:09:36.051 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.049+0000 7fc18aa00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562e4b40f8b0 0x562e4b420cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:09:36.051 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.049+0000 7fc18aa00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562e4b40f8b0 0x562e4b420cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:36016/0 (socket says 172.21.15.22:36016) 2024-03-28T18:09:36.051 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.049+0000 7fc18aa00640 1 -- 172.21.15.22:0/1483497771 learned_addr learned my addr 172.21.15.22:0/1483497771 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-28T18:09:36.051 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.049+0000 7fc18a000640 1 --2- 172.21.15.22:0/1483497771 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x562e4b410260 0x562e4b41f090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:09:36.052 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.049+0000 7fc18b400640 1 --2- 172.21.15.22:0/1483497771 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562e4b418de0 0x562e4b41f5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:09:36.052 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.049+0000 7fc18aa00640 1 -- 172.21.15.22:0/1483497771 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562e4b418de0 msgr2=0x562e4b41f5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T18:09:36.052 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.049+0000 7fc18aa00640 1 --2- 172.21.15.22:0/1483497771 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562e4b418de0 0x562e4b41f5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:09:36.052 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.049+0000 7fc18aa00640 1 -- 172.21.15.22:0/1483497771 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x562e4b410260 msgr2=0x562e4b41f090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T18:09:36.052 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.049+0000 7fc18aa00640 1 --2- 172.21.15.22:0/1483497771 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x562e4b410260 0x562e4b41f090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:09:36.052 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.049+0000 7fc18aa00640 1 -- 172.21.15.22:0/1483497771 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x562e4b41fc40 con 0x562e4b40f8b0 2024-03-28T18:09:36.052 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.050+0000 7fc18aa00640 1 --2- 172.21.15.22:0/1483497771 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562e4b40f8b0 0x562e4b420cf0 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7fc1840cc910 tx=0x7fc1840d2c40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9abb32fc05361b1a server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:09:36.052 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.050+0000 7fc16fe00640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-03-28T18:09:36.052 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.050+0000 7fc16fe00640 1 -- 172.21.15.22:0/1483497771 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc18409b590 con 0x562e4b40f8b0 2024-03-28T18:09:36.052 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.050+0000 7fc16fe00640 1 -- 172.21.15.22:0/1483497771 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 8411+0+0 (secure 0 0 0) 0x7fc184099ac0 con 0x562e4b40f8b0 2024-03-28T18:09:36.052 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.050+0000 7fc194394740 1 -- 172.21.15.22:0/1483497771 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fc17404c950 con 0x562e4b40f8b0 2024-03-28T18:09:36.052 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.050+0000 7fc16fe00640 1 -- 172.21.15.22:0/1483497771 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc18409b120 con 0x562e4b40f8b0 2024-03-28T18:09:36.052 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.050+0000 7fc194394740 10 client.16101 fetch_fsmap learned FSMap version 27 2024-03-28T18:09:36.052 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.050+0000 7fc194394740 1 -- 172.21.15.22:0/1483497771 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x562e4b484f40 con 0x562e4b40f8b0 2024-03-28T18:09:36.053 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/i2c/qcom,i2c-qup.txt 2024-03-28T18:09:36.054 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.051+0000 7fc16fe00640 1 -- 172.21.15.22:0/1483497771 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7fc17404c950 con 0x562e4b40f8b0 2024-03-28T18:09:36.054 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.051+0000 7fc16fe00640 1 -- 172.21.15.22:0/1483497771 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 27) v1 ==== 7934+0+0 (secure 0 0 0) 0x7fc1840ccb30 con 0x562e4b40f8b0 2024-03-28T18:09:36.055 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.051+0000 7fc194394740 10 client.16101 fetch_fsmap finished waiting for FSMap version 27 2024-03-28T18:09:36.055 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.051+0000 7fc194394740 10 client.16101 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.i 2024-03-28T18:09:36.055 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.051+0000 7fc194394740 20 client.16101 populate_metadata read hostname 'smithi022' 2024-03-28T18:09:36.055 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.051+0000 7fc194394740 1 --2- 172.21.15.22:0/1483497771 >> [v2:172.21.15.134:6838/3400263217,v1:172.21.15.134:6839/3400263217] conn(0x562e4b4855c0 0x562e4b4879e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:09:36.055 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.052+0000 7fc194394740 4 client.16101 mds_command: new command op to 24502 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-28T18:09:36.055 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.052+0000 7fc194394740 1 -- 172.21.15.22:0/1483497771 --> [v2:172.21.15.134:6838/3400263217,v1:172.21.15.134:6839/3400263217] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x562e4b37ab50 con 0x562e4b4855c0 2024-03-28T18:09:36.055 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.052+0000 7fc18a000640 1 --2- 172.21.15.22:0/1483497771 >> [v2:172.21.15.134:6838/3400263217,v1:172.21.15.134:6839/3400263217] conn(0x562e4b4855c0 0x562e4b4879e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:09:36.055 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.053+0000 7fc18a000640 1 --2- 172.21.15.22:0/1483497771 >> [v2:172.21.15.134:6838/3400263217,v1:172.21.15.134:6839/3400263217] conn(0x562e4b4855c0 0x562e4b4879e0 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=e03b51e72c1299ea server_cookie=e45b5469be9c1f5e in_seq=0 out_seq=0 2024-03-28T18:09:36.055 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.053+0000 7fc16fe00640 10 client.16101 ms_handle_connect on v2:172.21.15.134:6838/3400263217 2024-03-28T18:09:36.055 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.054+0000 7fc16fe00640 1 -- 172.21.15.22:0/1483497771 <== mds.4 v2:172.21.15.134:6838/3400263217 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x562e4b37ab50 con 0x562e4b4855c0 2024-03-28T18:09:36.055 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.054+0000 7fc16fe00640 10 client.16101 handle_command_reply: tid=0 2024-03-28T18:09:36.055 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.054+0000 7fc194394740 1 client.16101 shutdown 2024-03-28T18:09:36.055 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.054+0000 7fc194394740 1 -- 172.21.15.22:0/1483497771 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562e4b40f8b0 msgr2=0x562e4b420cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:09:36.056 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.054+0000 7fc194394740 1 --2- 172.21.15.22:0/1483497771 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562e4b40f8b0 0x562e4b420cf0 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7fc1840cc910 tx=0x7fc1840d2c40 comp rx=0 tx=0).stop 2024-03-28T18:09:36.056 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.054+0000 7fc194394740 1 -- 172.21.15.22:0/1483497771 shutdown_connections 2024-03-28T18:09:36.056 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.054+0000 7fc194394740 1 --2- 172.21.15.22:0/1483497771 >> [v2:172.21.15.134:6838/3400263217,v1:172.21.15.134:6839/3400263217] conn(0x562e4b4855c0 0x562e4b4879e0 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:09:36.056 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.054+0000 7fc194394740 1 --2- 172.21.15.22:0/1483497771 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562e4b418de0 0x562e4b41f5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:09:36.056 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.054+0000 7fc194394740 1 --2- 172.21.15.22:0/1483497771 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x562e4b410260 0x562e4b41f090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:09:36.056 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.054+0000 7fc16fe00640 0 client.16101 ms_handle_reset on v2:172.21.15.134:6838/3400263217 2024-03-28T18:09:36.056 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.054+0000 7fc194394740 1 --2- 172.21.15.22:0/1483497771 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562e4b40f8b0 0x562e4b420cf0 unknown :-1 s=CLOSED pgs=597 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:09:36.056 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.054+0000 7fc194394740 1 -- 172.21.15.22:0/1483497771 >> 172.21.15.22:0/1483497771 conn(0x562e4b409f80 msgr2=0x562e4b40bbd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:09:36.056 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.054+0000 7fc194394740 1 -- 172.21.15.22:0/1483497771 shutdown_connections 2024-03-28T18:09:36.056 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.054+0000 7fc194394740 1 -- 172.21.15.22:0/1483497771 wait complete. 2024-03-28T18:09:36.056 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.054+0000 7fc194394740 20 client.16101 trim_cache size 0 max 16384 2024-03-28T18:09:36.057 INFO:teuthology.orchestra.run.smithi022.stdout:[] 2024-03-28T18:09:36.057 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.055+0000 7fc18be00640 1 -- 172.21.15.22:0/3853924037 >> [v2:172.21.15.22:6800/1357365082,v1:172.21.15.22:6801/1357365082] conn(0x7fc1700a29f0 msgr2=0x7fc1700a4eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:09:36.057 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.055+0000 7fc18be00640 1 --2- 172.21.15.22:0/3853924037 >> [v2:172.21.15.22:6800/1357365082,v1:172.21.15.22:6801/1357365082] conn(0x7fc1700a29f0 0x7fc1700a4eb0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fc17c00bca0 tx=0x7fc17c00b450 comp rx=0 tx=0).stop 2024-03-28T18:09:36.058 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.055+0000 7fc18be00640 1 -- 172.21.15.22:0/3853924037 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc18c101530 msgr2=0x7fc18c199fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:09:36.058 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.055+0000 7fc18be00640 1 --2- 172.21.15.22:0/3853924037 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc18c101530 0x7fc18c199fc0 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7fc18400e9f0 tx=0x7fc18400edf0 comp rx=0 tx=0).stop 2024-03-28T18:09:36.058 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.055+0000 7fc18be00640 1 -- 172.21.15.22:0/3853924037 shutdown_connections 2024-03-28T18:09:36.058 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.055+0000 7fc18be00640 1 --2- 172.21.15.22:0/3853924037 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc18c1075f0 0x7fc18c19e880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:09:36.058 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.055+0000 7fc18be00640 1 --2- 172.21.15.22:0/3853924037 >> [v2:172.21.15.22:6800/1357365082,v1:172.21.15.22:6801/1357365082] conn(0x7fc1700a29f0 0x7fc1700a4eb0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:09:36.058 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.055+0000 7fc18be00640 1 --2- 172.21.15.22:0/3853924037 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc18c101ef0 0x7fc18c19a500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:09:36.058 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.055+0000 7fc18be00640 1 --2- 172.21.15.22:0/3853924037 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc18c101530 0x7fc18c199fc0 unknown :-1 s=CLOSED pgs=593 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:09:36.058 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.055+0000 7fc18be00640 1 -- 172.21.15.22:0/3853924037 >> 172.21.15.22:0/3853924037 conn(0x7fc18c0fd040 msgr2=0x7fc18c0fe500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:09:36.058 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.055+0000 7fc18be00640 1 -- 172.21.15.22:0/3853924037 shutdown_connections 2024-03-28T18:09:36.058 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.055+0000 7fc18be00640 1 -- 172.21.15.22:0/3853924037 wait complete. 2024-03-28T18:09:36.060 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/i2c/ti,bq32k.txt 2024-03-28T18:09:36.070 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/i2c/trivial-devices.txt 2024-03-28T18:09:36.076 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-28T18:09:36.077 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2024-03-28T18:09:36.083 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/iio/ 2024-03-28T18:09:36.099 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/iio/accel/ 2024-03-28T18:09:36.105 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/iio/accel/bma180.txt 2024-03-28T18:09:36.111 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/iio/adc/ 2024-03-28T18:09:36.129 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/iio/adc/at91_adc.txt 2024-03-28T18:09:36.135 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/iio/adc/cc10001_adc.txt 2024-03-28T18:09:36.150 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/iio/adc/max1027-adc.txt 2024-03-28T18:09:36.164 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/iio/adc/nuvoton-nau7802.txt 2024-03-28T18:09:36.178 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/iio/adc/qcom,spmi-iadc.txt 2024-03-28T18:09:36.193 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/iio/adc/qcom,spmi-vadc.txt 2024-03-28T18:09:36.208 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/iio/adc/rockchip-saradc.txt 2024-03-28T18:09:36.223 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/iio/adc/twl4030-madc.txt 2024-03-28T18:09:36.246 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/iio/adc/vf610-adc.txt 2024-03-28T18:09:36.296 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/iio/adc/xilinx-xadc.txt 2024-03-28T18:09:36.313 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.313+0000 7f4524e00640 1 -- 172.21.15.22:0/1342680649 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4520109850 msgr2=0x7f4520109c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:09:36.313 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.313+0000 7f4524e00640 1 --2- 172.21.15.22:0/1342680649 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4520109850 0x7f4520109c30 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7f4510009930 tx=0x7f451002bc30 comp rx=0 tx=0).stop 2024-03-28T18:09:36.313 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.313+0000 7f4524e00640 1 -- 172.21.15.22:0/1342680649 shutdown_connections 2024-03-28T18:09:36.313 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.313+0000 7f4524e00640 1 --2- 172.21.15.22:0/1342680649 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4520104200 0x7f4520110710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:09:36.313 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.313+0000 7f4524e00640 1 --2- 172.21.15.22:0/1342680649 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f4520103840 0x7f4520103cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:09:36.313 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.313+0000 7f4524e00640 1 --2- 172.21.15.22:0/1342680649 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4520109850 0x7f4520109c30 unknown :-1 s=CLOSED pgs=598 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:09:36.314 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.313+0000 7f4524e00640 1 -- 172.21.15.22:0/1342680649 >> 172.21.15.22:0/1342680649 conn(0x7f45200ff310 msgr2=0x7f4520101730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:09:36.314 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.313+0000 7f4524e00640 1 -- 172.21.15.22:0/1342680649 shutdown_connections 2024-03-28T18:09:36.314 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.313+0000 7f4524e00640 1 -- 172.21.15.22:0/1342680649 wait complete. 2024-03-28T18:09:36.314 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/iio/dac/ 2024-03-28T18:09:36.315 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.314+0000 7f4524e00640 1 Processor -- start 2024-03-28T18:09:36.315 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.315+0000 7f4524e00640 1 -- start start 2024-03-28T18:09:36.315 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.315+0000 7f4524e00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4520103840 0x7f452019a180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:09:36.315 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.315+0000 7f4524e00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4520104200 0x7f452019a6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:09:36.315 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.315+0000 7f4524e00640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f4520109850 0x7f452019ea40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:09:36.315 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.315+0000 7f4524e00640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4520113c80 con 0x7f4520103840 2024-03-28T18:09:36.315 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.315+0000 7f4524e00640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4520113b00 con 0x7f4520109850 2024-03-28T18:09:36.315 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.315+0000 7f4524e00640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4520113e00 con 0x7f4520104200 2024-03-28T18:09:36.317 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.315+0000 7f451de00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4520103840 0x7f452019a180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:09:36.317 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.316+0000 7f451de00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4520103840 0x7f452019a180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:36046/0 (socket says 172.21.15.22:36046) 2024-03-28T18:09:36.317 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.316+0000 7f451de00640 1 -- 172.21.15.22:0/1844794507 learned_addr learned my addr 172.21.15.22:0/1844794507 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-28T18:09:36.317 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.316+0000 7f451e800640 1 --2- 172.21.15.22:0/1844794507 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f4520109850 0x7f452019ea40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:09:36.317 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.316+0000 7f451d400640 1 --2- 172.21.15.22:0/1844794507 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4520104200 0x7f452019a6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:09:36.317 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.316+0000 7f451de00640 1 -- 172.21.15.22:0/1844794507 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4520104200 msgr2=0x7f452019a6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:09:36.317 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.316+0000 7f451de00640 1 --2- 172.21.15.22:0/1844794507 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4520104200 0x7f452019a6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:09:36.317 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.316+0000 7f451de00640 1 -- 172.21.15.22:0/1844794507 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f4520109850 msgr2=0x7f452019ea40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:09:36.318 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.316+0000 7f451de00640 1 --2- 172.21.15.22:0/1844794507 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f4520109850 0x7f452019ea40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:09:36.318 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.316+0000 7f451de00640 1 -- 172.21.15.22:0/1844794507 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f452019f170 con 0x7f4520103840 2024-03-28T18:09:36.318 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.316+0000 7f451d400640 1 --2- 172.21.15.22:0/1844794507 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4520104200 0x7f452019a6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T18:09:36.318 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.316+0000 7f451e800640 1 --2- 172.21.15.22:0/1844794507 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f4520109850 0x7f452019ea40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-28T18:09:36.318 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.317+0000 7f451de00640 1 --2- 172.21.15.22:0/1844794507 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4520103840 0x7f452019a180 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7f4510000c00 tx=0x7f45100045e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:09:36.318 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.317+0000 7f4516a00640 1 -- 172.21.15.22:0/1844794507 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4510043020 con 0x7f4520103840 2024-03-28T18:09:36.318 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.317+0000 7f4516a00640 1 -- 172.21.15.22:0/1844794507 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f451002caa0 con 0x7f4520103840 2024-03-28T18:09:36.318 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.317+0000 7f4524e00640 1 -- 172.21.15.22:0/1844794507 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f452019f400 con 0x7f4520103840 2024-03-28T18:09:36.318 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.317+0000 7f4516a00640 1 -- 172.21.15.22:0/1844794507 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4510003b10 con 0x7f4520103840 2024-03-28T18:09:36.318 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.317+0000 7f4524e00640 1 -- 172.21.15.22:0/1844794507 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f45201a6cf0 con 0x7f4520103840 2024-03-28T18:09:36.320 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.321+0000 7f4527a6f740 1 Processor -- start 2024-03-28T18:09:36.320 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.321+0000 7f4516a00640 1 -- 172.21.15.22:0/1844794507 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 26) v1 ==== 137421+0+0 (secure 0 0 0) 0x7f4510046020 con 0x7f4520103840 2024-03-28T18:09:36.321 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.321+0000 7f4527a6f740 1 -- start start 2024-03-28T18:09:36.322 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.321+0000 7f4527a6f740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5599837036d0 0x559983703ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:09:36.322 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.321+0000 7f4527a6f740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559983704080 0x55998370c630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:09:36.322 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.321+0000 7f4527a6f740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55998370cc00 0x55998370f050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:09:36.322 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.321+0000 7f4527a6f740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x5599836672c0 con 0x5599837036d0 2024-03-28T18:09:36.322 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.321+0000 7f4527a6f740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x55998360fc90 con 0x55998370cc00 2024-03-28T18:09:36.322 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.321+0000 7f4527a6f740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x559983663360 con 0x559983704080 2024-03-28T18:09:36.323 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.322+0000 7f451e800640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5599837036d0 0x559983703ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:09:36.323 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.322+0000 7f451e800640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5599837036d0 0x559983703ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:36054/0 (socket says 172.21.15.22:36054) 2024-03-28T18:09:36.323 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.322+0000 7f451e800640 1 -- 172.21.15.22:0/4020122963 learned_addr learned my addr 172.21.15.22:0/4020122963 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-28T18:09:36.323 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.322+0000 7f451d400640 1 --2- 172.21.15.22:0/4020122963 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55998370cc00 0x55998370f050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:09:36.323 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.322+0000 7f451de00640 1 --2- 172.21.15.22:0/4020122963 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559983704080 0x55998370c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:09:36.323 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.322+0000 7f451e800640 1 -- 172.21.15.22:0/4020122963 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559983704080 msgr2=0x55998370c630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T18:09:36.323 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.322+0000 7f451e800640 1 --2- 172.21.15.22:0/4020122963 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559983704080 0x55998370c630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:09:36.323 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.322+0000 7f451e800640 1 -- 172.21.15.22:0/4020122963 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55998370cc00 msgr2=0x55998370f050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T18:09:36.323 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.322+0000 7f451e800640 1 --2- 172.21.15.22:0/4020122963 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55998370cc00 0x55998370f050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:09:36.323 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.322+0000 7f451e800640 1 -- 172.21.15.22:0/4020122963 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559983634080 con 0x5599837036d0 2024-03-28T18:09:36.323 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.322+0000 7f451e800640 1 --2- 172.21.15.22:0/4020122963 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5599837036d0 0x559983703ab0 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7f4518010710 tx=0x7f4518010bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=92b27c05bd114dc1 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:09:36.323 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.323+0000 7f4516a00640 1 --2- 172.21.15.22:0/1844794507 >> [v2:172.21.15.22:6800/1357365082,v1:172.21.15.22:6801/1357365082] conn(0x7f45040a29f0 0x7f45040a4eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:09:36.325 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.323+0000 7f450b400640 1 -- 172.21.15.22:0/4020122963 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f451800e8d0 con 0x5599837036d0 2024-03-28T18:09:36.325 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.323+0000 7f4516a00640 1 -- 172.21.15.22:0/1844794507 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 8411+0+0 (secure 0 0 0) 0x7f45100f0e80 con 0x7f4520103840 2024-03-28T18:09:36.325 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.323+0000 7f450b400640 1 -- 172.21.15.22:0/4020122963 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f4518012450 con 0x5599837036d0 2024-03-28T18:09:36.326 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.323+0000 7f450b400640 1 -- 172.21.15.22:0/4020122963 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f451801c650 con 0x5599837036d0 2024-03-28T18:09:36.326 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.323+0000 7f451de00640 1 --2- 172.21.15.22:0/1844794507 >> [v2:172.21.15.22:6800/1357365082,v1:172.21.15.22:6801/1357365082] conn(0x7f45040a29f0 0x7f45040a4eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:09:36.326 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.324+0000 7f4527a6f740 1 -- 172.21.15.22:0/4020122963 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5599837036d0 msgr2=0x559983703ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:09:36.326 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.324+0000 7f451de00640 1 --2- 172.21.15.22:0/1844794507 >> [v2:172.21.15.22:6800/1357365082,v1:172.21.15.22:6801/1357365082] conn(0x7f45040a29f0 0x7f45040a4eb0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x5599837031a0 tx=0x7f4510009260 comp rx=0 tx=0).ready entity=mgr.15522 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:09:36.326 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.324+0000 7f4527a6f740 1 --2- 172.21.15.22:0/4020122963 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5599837036d0 0x559983703ab0 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7f4518010710 tx=0x7f4518010bd0 comp rx=0 tx=0).stop 2024-03-28T18:09:36.326 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.324+0000 7f4527a6f740 1 -- 172.21.15.22:0/4020122963 shutdown_connections 2024-03-28T18:09:36.326 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.324+0000 7f4527a6f740 1 --2- 172.21.15.22:0/4020122963 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55998370cc00 0x55998370f050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:09:36.326 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.324+0000 7f4527a6f740 1 --2- 172.21.15.22:0/4020122963 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559983704080 0x55998370c630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:09:36.326 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.324+0000 7f4527a6f740 1 --2- 172.21.15.22:0/4020122963 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5599837036d0 0x559983703ab0 unknown :-1 s=CLOSED pgs=600 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:09:36.326 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.324+0000 7f4527a6f740 1 -- 172.21.15.22:0/4020122963 >> 172.21.15.22:0/4020122963 conn(0x5599836fdda0 msgr2=0x5599836fe180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:09:36.326 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.324+0000 7f4527a6f740 1 -- 172.21.15.22:0/4020122963 shutdown_connections 2024-03-28T18:09:36.326 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.324+0000 7f4527a6f740 1 -- 172.21.15.22:0/4020122963 wait complete. 2024-03-28T18:09:36.326 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.325+0000 7f4527a6f740 1 -- start start 2024-03-28T18:09:36.327 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.325+0000 7f4527a6f740 1 Processor -- start 2024-03-28T18:09:36.327 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.325+0000 7f4527a6f740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5599837036d0 0x55998370c5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:09:36.327 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.325+0000 7f4527a6f740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x559983704080 0x559983709980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:09:36.327 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.325+0000 7f4527a6f740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55998370cc00 0x55998370a0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:09:36.327 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.325+0000 7f4527a6f740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x55998370fb70 con 0x5599837036d0 2024-03-28T18:09:36.327 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.326+0000 7f4527a6f740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x5599836230d0 con 0x559983704080 2024-03-28T18:09:36.327 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.326+0000 7f4527a6f740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55998370fcf0 con 0x55998370cc00 2024-03-28T18:09:36.327 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.326+0000 7f4527a6f740 10 client.0 _ll_register_callbacks cb 0x559983627e80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-28T18:09:36.327 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.326+0000 7f451e800640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5599837036d0 0x55998370c5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:09:36.327 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.326+0000 7f451d400640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x559983704080 0x559983709980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:09:36.327 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.326+0000 7f451e800640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5599837036d0 0x55998370c5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:36060/0 (socket says 172.21.15.22:36060) 2024-03-28T18:09:36.327 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.326+0000 7f451e800640 1 -- 172.21.15.22:0/975925394 learned_addr learned my addr 172.21.15.22:0/975925394 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-28T18:09:36.328 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.326+0000 7f451d400640 1 -- 172.21.15.22:0/975925394 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55998370cc00 msgr2=0x55998370a0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T18:09:36.328 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.326+0000 7f451d400640 1 --2- 172.21.15.22:0/975925394 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55998370cc00 0x55998370a0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:09:36.328 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.326+0000 7f451d400640 1 -- 172.21.15.22:0/975925394 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5599837036d0 msgr2=0x55998370c5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T18:09:36.328 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.326+0000 7f451d400640 1 --2- 172.21.15.22:0/975925394 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5599837036d0 0x55998370c5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:09:36.328 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.326+0000 7f451d400640 1 -- 172.21.15.22:0/975925394 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55998337b3f0 con 0x559983704080 2024-03-28T18:09:36.328 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.326+0000 7f451e800640 1 --2- 172.21.15.22:0/975925394 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5599837036d0 0x55998370c5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-28T18:09:36.329 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/iio/dac/ad7303.txt 2024-03-28T18:09:36.331 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.327+0000 7f451d400640 1 --2- 172.21.15.22:0/975925394 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x559983704080 0x559983709980 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f450c0136e0 tx=0x7f450c013ba0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a13d714b8cbf14e8 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:09:36.331 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.327+0000 7f44ffe00640 10 client.0 ms_handle_connect on v2:172.21.15.79:3300/0 2024-03-28T18:09:36.332 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.327+0000 7f44ffe00640 1 -- 172.21.15.22:0/975925394 <== mon.2 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f450c0119c0 con 0x559983704080 2024-03-28T18:09:36.332 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.327+0000 7f44ffe00640 1 -- 172.21.15.22:0/975925394 <== mon.2 v2:172.21.15.79:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 8411+0+0 (secure 0 0 0) 0x7f450c011b60 con 0x559983704080 2024-03-28T18:09:36.332 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.327+0000 7f4527a6f740 1 -- 172.21.15.22:0/975925394 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f4518012450 con 0x559983704080 2024-03-28T18:09:36.332 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.327+0000 7f44ffe00640 1 -- 172.21.15.22:0/975925394 <== mon.2 v2:172.21.15.79:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f450c011d60 con 0x559983704080 2024-03-28T18:09:36.332 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.327+0000 7f4527a6f740 10 client.35942 fetch_fsmap learned FSMap version 27 2024-03-28T18:09:36.332 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.327+0000 7f4527a6f740 1 -- 172.21.15.22:0/975925394 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x5599834457d0 con 0x559983704080 2024-03-28T18:09:36.332 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.328+0000 7f44ffe00640 1 -- 172.21.15.22:0/975925394 <== mon.2 v2:172.21.15.79:3300/0 4 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f4518012450 con 0x559983704080 2024-03-28T18:09:36.332 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.328+0000 7f44ffe00640 1 -- 172.21.15.22:0/975925394 <== mon.2 v2:172.21.15.79:3300/0 5 ==== fsmap(e 27) v1 ==== 7934+0+0 (secure 0 0 0) 0x7f450c024050 con 0x559983704080 2024-03-28T18:09:36.332 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.328+0000 7f4527a6f740 10 client.35942 fetch_fsmap finished waiting for FSMap version 27 2024-03-28T18:09:36.332 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.328+0000 7f4527a6f740 10 client.35942 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.k 2024-03-28T18:09:36.332 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.328+0000 7f4527a6f740 20 client.35942 populate_metadata read hostname 'smithi022' 2024-03-28T18:09:36.332 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.328+0000 7f4527a6f740 1 --2- 172.21.15.22:0/975925394 >> [v2:172.21.15.79:6832/856896997,v1:172.21.15.79:6833/856896997] conn(0x55998371cb10 0x55998371ef50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:09:36.332 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.328+0000 7f4527a6f740 4 client.35942 mds_command: new command op to 34442 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-28T18:09:36.332 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.328+0000 7f4527a6f740 1 -- 172.21.15.22:0/975925394 --> [v2:172.21.15.79:6832/856896997,v1:172.21.15.79:6833/856896997] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x559983674800 con 0x55998371cb10 2024-03-28T18:09:36.332 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.328+0000 7f451e800640 1 --2- 172.21.15.22:0/975925394 >> [v2:172.21.15.79:6832/856896997,v1:172.21.15.79:6833/856896997] conn(0x55998371cb10 0x55998371ef50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:09:36.332 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.329+0000 7f451e800640 1 --2- 172.21.15.22:0/975925394 >> [v2:172.21.15.79:6832/856896997,v1:172.21.15.79:6833/856896997] conn(0x55998371cb10 0x55998371ef50 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=eb68e4a23954c06f server_cookie=a1bf9000c81294bf in_seq=0 out_seq=0 2024-03-28T18:09:36.333 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.329+0000 7f44ffe00640 10 client.35942 ms_handle_connect on v2:172.21.15.79:6832/856896997 2024-03-28T18:09:36.333 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.330+0000 7f44ffe00640 1 -- 172.21.15.22:0/975925394 <== mds.3 v2:172.21.15.79:6832/856896997 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x559983674800 con 0x55998371cb10 2024-03-28T18:09:36.333 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.330+0000 7f44ffe00640 10 client.35942 handle_command_reply: tid=0 2024-03-28T18:09:36.333 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.330+0000 7f4527a6f740 1 client.35942 shutdown 2024-03-28T18:09:36.333 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.331+0000 7f4527a6f740 1 -- 172.21.15.22:0/975925394 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x559983704080 msgr2=0x559983709980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:09:36.333 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.331+0000 7f4527a6f740 1 --2- 172.21.15.22:0/975925394 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x559983704080 0x559983709980 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f450c0136e0 tx=0x7f450c013ba0 comp rx=0 tx=0).stop 2024-03-28T18:09:36.333 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.331+0000 7f4527a6f740 1 -- 172.21.15.22:0/975925394 shutdown_connections 2024-03-28T18:09:36.333 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.331+0000 7f4527a6f740 1 --2- 172.21.15.22:0/975925394 >> [v2:172.21.15.79:6832/856896997,v1:172.21.15.79:6833/856896997] conn(0x55998371cb10 0x55998371ef50 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:09:36.333 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.331+0000 7f4527a6f740 1 --2- 172.21.15.22:0/975925394 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55998370cc00 0x55998370a0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:09:36.333 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.331+0000 7f4527a6f740 1 --2- 172.21.15.22:0/975925394 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x559983704080 0x559983709980 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:09:36.333 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.331+0000 7f4527a6f740 1 --2- 172.21.15.22:0/975925394 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5599837036d0 0x55998370c5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:09:36.333 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.331+0000 7f44ffe00640 0 client.35942 ms_handle_reset on v2:172.21.15.79:6832/856896997 2024-03-28T18:09:36.333 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.331+0000 7f4527a6f740 1 -- 172.21.15.22:0/975925394 >> 172.21.15.22:0/975925394 conn(0x5599836fdda0 msgr2=0x5599836ffaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:09:36.333 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.331+0000 7f4527a6f740 1 -- 172.21.15.22:0/975925394 shutdown_connections 2024-03-28T18:09:36.333 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.331+0000 7f4527a6f740 1 -- 172.21.15.22:0/975925394 wait complete. 2024-03-28T18:09:36.334 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.331+0000 7f4527a6f740 20 client.35942 trim_cache size 0 max 16384 2024-03-28T18:09:36.334 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/iio/dac/max5821.txt 2024-03-28T18:09:36.342 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/iio/frequency/ 2024-03-28T18:09:36.345 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.345+0000 7f4527a6f740 1 Processor -- start 2024-03-28T18:09:36.345 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.345+0000 7f4527a6f740 1 -- start start 2024-03-28T18:09:36.345 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.346+0000 7f4527a6f740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5599837036d0 0x55998370c0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:09:36.345 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.346+0000 7f4527a6f740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559983704080 0x55998370c5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:09:36.345 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.346+0000 7f4527a6f740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55998370cc00 0x559983706be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:09:36.345 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.346+0000 7f4527a6f740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45201af260 con 0x559983704080 2024-03-28T18:09:36.346 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.346+0000 7f4527a6f740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45201b0be0 con 0x55998370cc00 2024-03-28T18:09:36.346 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.346+0000 7f4527a6f740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45201af8c0 con 0x5599837036d0 2024-03-28T18:09:36.348 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.346+0000 7f451d400640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559983704080 0x55998370c5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:09:36.348 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.346+0000 7f451d400640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559983704080 0x55998370c5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:36076/0 (socket says 172.21.15.22:36076) 2024-03-28T18:09:36.348 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.346+0000 7f451d400640 1 -- 172.21.15.22:0/559363890 learned_addr learned my addr 172.21.15.22:0/559363890 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-28T18:09:36.348 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.346+0000 7f451e800640 1 --2- 172.21.15.22:0/559363890 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55998370cc00 0x559983706be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:09:36.348 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.346+0000 7f451e800640 1 --2- 172.21.15.22:0/559363890 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5599837036d0 0x55998370c0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:09:36.348 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.346+0000 7f451d400640 1 -- 172.21.15.22:0/559363890 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5599837036d0 msgr2=0x55998370c0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T18:09:36.348 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.346+0000 7f451d400640 1 --2- 172.21.15.22:0/559363890 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5599837036d0 0x55998370c0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:09:36.348 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.346+0000 7f451d400640 1 -- 172.21.15.22:0/559363890 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55998370cc00 msgr2=0x559983706be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T18:09:36.348 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.346+0000 7f451d400640 1 --2- 172.21.15.22:0/559363890 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55998370cc00 0x559983706be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:09:36.348 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.346+0000 7f451d400640 1 -- 172.21.15.22:0/559363890 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55998370f470 con 0x559983704080 2024-03-28T18:09:36.348 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.346+0000 7f451e800640 1 --2- 172.21.15.22:0/559363890 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55998370cc00 0x559983706be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-28T18:09:36.349 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.347+0000 7f451d400640 1 --2- 172.21.15.22:0/559363890 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559983704080 0x55998370c5f0 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7f450c01c660 tx=0x7f450c01c4c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d7640e5ff64ab52d server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:09:36.349 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.347+0000 7f450aa00640 1 -- 172.21.15.22:0/559363890 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f450c000e60 con 0x559983704080 2024-03-28T18:09:36.349 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.347+0000 7f450aa00640 1 -- 172.21.15.22:0/559363890 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f450c005b90 con 0x559983704080 2024-03-28T18:09:36.349 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.347+0000 7f450aa00640 1 -- 172.21.15.22:0/559363890 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f450c003d20 con 0x559983704080 2024-03-28T18:09:36.349 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.348+0000 7f4527a6f740 1 -- 172.21.15.22:0/559363890 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559983704080 msgr2=0x55998370c5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:09:36.349 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.348+0000 7f4527a6f740 1 --2- 172.21.15.22:0/559363890 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559983704080 0x55998370c5f0 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7f450c01c660 tx=0x7f450c01c4c0 comp rx=0 tx=0).stop 2024-03-28T18:09:36.349 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.348+0000 7f4527a6f740 1 -- 172.21.15.22:0/559363890 shutdown_connections 2024-03-28T18:09:36.349 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.348+0000 7f4527a6f740 1 --2- 172.21.15.22:0/559363890 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55998370cc00 0x559983706be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:09:36.349 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.348+0000 7f4527a6f740 1 --2- 172.21.15.22:0/559363890 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559983704080 0x55998370c5f0 unknown :-1 s=CLOSED pgs=601 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:09:36.349 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.348+0000 7f4527a6f740 1 --2- 172.21.15.22:0/559363890 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5599837036d0 0x55998370c0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:09:36.349 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.348+0000 7f4527a6f740 1 -- 172.21.15.22:0/559363890 >> 172.21.15.22:0/559363890 conn(0x5599836fdda0 msgr2=0x5599837089b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:09:36.349 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.348+0000 7f4527a6f740 1 -- 172.21.15.22:0/559363890 shutdown_connections 2024-03-28T18:09:36.349 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.348+0000 7f4527a6f740 1 -- 172.21.15.22:0/559363890 wait complete. 2024-03-28T18:09:36.349 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.348+0000 7f4527a6f740 1 -- start start 2024-03-28T18:09:36.349 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.348+0000 7f4527a6f740 1 Processor -- start 2024-03-28T18:09:36.352 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.349+0000 7f4527a6f740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5599837036d0 0x5599837124c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:09:36.352 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.349+0000 7f4527a6f740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559983704080 0x559983712a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:09:36.352 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.349+0000 7f4527a6f740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55998370cc00 0x559983712f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:09:36.352 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.349+0000 7f4527a6f740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45201b0580 con 0x559983704080 2024-03-28T18:09:36.352 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.349+0000 7f4527a6f740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45201aff20 con 0x5599837036d0 2024-03-28T18:09:36.352 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.349+0000 7f4527a6f740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45201ad420 con 0x55998370cc00 2024-03-28T18:09:36.352 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.349+0000 7f4527a6f740 10 client.0 _ll_register_callbacks cb 0x7f450c009d10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-28T18:09:36.352 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.349+0000 7f451e800640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5599837036d0 0x5599837124c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:09:36.352 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.349+0000 7f451d400640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559983704080 0x559983712a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:09:36.353 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.349+0000 7f451e800640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5599837036d0 0x5599837124c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.22:57704/0 (socket says 172.21.15.22:57704) 2024-03-28T18:09:36.353 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.349+0000 7f451e800640 1 -- 172.21.15.22:0/1985742031 learned_addr learned my addr 172.21.15.22:0/1985742031 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-28T18:09:36.353 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.350+0000 7f451e800640 1 --2- 172.21.15.22:0/1985742031 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55998370cc00 0x559983712f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:09:36.353 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.350+0000 7f451e800640 1 -- 172.21.15.22:0/1985742031 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55998370cc00 msgr2=0x559983712f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T18:09:36.353 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.350+0000 7f451e800640 1 --2- 172.21.15.22:0/1985742031 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55998370cc00 0x559983712f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:09:36.353 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.350+0000 7f451e800640 1 -- 172.21.15.22:0/1985742031 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559983704080 msgr2=0x559983712a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T18:09:36.353 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.350+0000 7f451e800640 1 --2- 172.21.15.22:0/1985742031 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559983704080 0x559983712a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:09:36.353 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.350+0000 7f451e800640 1 -- 172.21.15.22:0/1985742031 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x559983777520 con 0x5599837036d0 2024-03-28T18:09:36.353 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.350+0000 7f451d400640 1 --2- 172.21.15.22:0/1985742031 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559983704080 0x559983712a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-28T18:09:36.353 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.350+0000 7f451e800640 1 --2- 172.21.15.22:0/1985742031 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5599837036d0 0x5599837124c0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f4518009d40 tx=0x7f451801e040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1e21d32984d5a70e server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:09:36.353 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.350+0000 7f44ffe00640 10 client.0 ms_handle_connect on v2:172.21.15.79:3300/0 2024-03-28T18:09:36.353 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.350+0000 7f44ffe00640 1 -- 172.21.15.22:0/1985742031 <== mon.2 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f451802c110 con 0x5599837036d0 2024-03-28T18:09:36.353 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.350+0000 7f4527a6f740 1 -- 172.21.15.22:0/1985742031 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f450c005b90 con 0x5599837036d0 2024-03-28T18:09:36.353 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.351+0000 7f44ffe00640 1 -- 172.21.15.22:0/1985742031 <== mon.2 v2:172.21.15.79:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 8411+0+0 (secure 0 0 0) 0x7f451802c2b0 con 0x5599837036d0 2024-03-28T18:09:36.353 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.351+0000 7f44ffe00640 1 -- 172.21.15.22:0/1985742031 <== mon.2 v2:172.21.15.79:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4518012a10 con 0x5599837036d0 2024-03-28T18:09:36.354 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.351+0000 7f4527a6f740 10 client.35948 fetch_fsmap learned FSMap version 27 2024-03-28T18:09:36.354 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.351+0000 7f4527a6f740 1 -- 172.21.15.22:0/1985742031 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x559983778800 con 0x5599837036d0 2024-03-28T18:09:36.354 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.351+0000 7f44ffe00640 1 -- 172.21.15.22:0/1985742031 <== mon.2 v2:172.21.15.79:3300/0 4 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f450c005b90 con 0x5599837036d0 2024-03-28T18:09:36.354 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.351+0000 7f44ffe00640 1 -- 172.21.15.22:0/1985742031 <== mon.2 v2:172.21.15.79:3300/0 5 ==== fsmap(e 27) v1 ==== 7934+0+0 (secure 0 0 0) 0x7f451800f040 con 0x5599837036d0 2024-03-28T18:09:36.354 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.352+0000 7f4527a6f740 10 client.35948 fetch_fsmap finished waiting for FSMap version 27 2024-03-28T18:09:36.354 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.352+0000 7f4527a6f740 10 client.35948 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.k 2024-03-28T18:09:36.354 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.352+0000 7f4527a6f740 20 client.35948 populate_metadata read hostname 'smithi022' 2024-03-28T18:09:36.354 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.352+0000 7f4527a6f740 1 --2- 172.21.15.22:0/1985742031 >> [v2:172.21.15.79:6832/856896997,v1:172.21.15.79:6833/856896997] conn(0x559983778c90 0x55998377b080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:09:36.354 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.352+0000 7f4527a6f740 4 client.35948 mds_command: new command op to 34442 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-28T18:09:36.354 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.352+0000 7f4527a6f740 1 -- 172.21.15.22:0/1985742031 --> [v2:172.21.15.79:6832/856896997,v1:172.21.15.79:6833/856896997] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55998366eb50 con 0x559983778c90 2024-03-28T18:09:36.354 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.352+0000 7f451d400640 1 --2- 172.21.15.22:0/1985742031 >> [v2:172.21.15.79:6832/856896997,v1:172.21.15.79:6833/856896997] conn(0x559983778c90 0x55998377b080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:09:36.355 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.353+0000 7f451d400640 1 --2- 172.21.15.22:0/1985742031 >> [v2:172.21.15.79:6832/856896997,v1:172.21.15.79:6833/856896997] conn(0x559983778c90 0x55998377b080 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=cbf4acdfa933b8cd server_cookie=ed93601916ca580e in_seq=0 out_seq=0 2024-03-28T18:09:36.356 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.353+0000 7f44ffe00640 10 client.35948 ms_handle_connect on v2:172.21.15.79:6832/856896997 2024-03-28T18:09:36.356 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.353+0000 7f44ffe00640 1 -- 172.21.15.22:0/1985742031 <== mds.3 v2:172.21.15.79:6832/856896997 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55998366eb50 con 0x559983778c90 2024-03-28T18:09:36.356 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.353+0000 7f44ffe00640 10 client.35948 handle_command_reply: tid=0 2024-03-28T18:09:36.356 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.353+0000 7f4527a6f740 1 client.35948 shutdown 2024-03-28T18:09:36.356 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.353+0000 7f4527a6f740 1 -- 172.21.15.22:0/1985742031 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5599837036d0 msgr2=0x5599837124c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:09:36.356 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.353+0000 7f4527a6f740 1 --2- 172.21.15.22:0/1985742031 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5599837036d0 0x5599837124c0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f4518009d40 tx=0x7f451801e040 comp rx=0 tx=0).stop 2024-03-28T18:09:36.356 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.353+0000 7f4527a6f740 1 -- 172.21.15.22:0/1985742031 shutdown_connections 2024-03-28T18:09:36.356 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.353+0000 7f4527a6f740 1 --2- 172.21.15.22:0/1985742031 >> [v2:172.21.15.79:6832/856896997,v1:172.21.15.79:6833/856896997] conn(0x559983778c90 0x55998377b080 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:09:36.356 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.353+0000 7f4527a6f740 1 --2- 172.21.15.22:0/1985742031 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55998370cc00 0x559983712f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:09:36.356 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.353+0000 7f4527a6f740 1 --2- 172.21.15.22:0/1985742031 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559983704080 0x559983712a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:09:36.356 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.353+0000 7f4527a6f740 1 --2- 172.21.15.22:0/1985742031 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5599837036d0 0x5599837124c0 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:09:36.356 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.353+0000 7f4527a6f740 1 -- 172.21.15.22:0/1985742031 >> 172.21.15.22:0/1985742031 conn(0x5599836fdda0 msgr2=0x5599836fc620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:09:36.356 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.353+0000 7f44ffe00640 0 client.35948 ms_handle_reset on v2:172.21.15.79:6832/856896997 2024-03-28T18:09:36.356 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.354+0000 7f4527a6f740 1 -- 172.21.15.22:0/1985742031 shutdown_connections 2024-03-28T18:09:36.357 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.354+0000 7f4527a6f740 1 -- 172.21.15.22:0/1985742031 wait complete. 2024-03-28T18:09:36.357 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.354+0000 7f4527a6f740 20 client.35948 trim_cache size 0 max 16384 2024-03-28T18:09:36.357 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/iio/frequency/adf4350.txt 2024-03-28T18:09:36.357 INFO:teuthology.orchestra.run.smithi022.stdout:[] 2024-03-28T18:09:36.358 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.355+0000 7f4524e00640 1 -- 172.21.15.22:0/1844794507 >> [v2:172.21.15.22:6800/1357365082,v1:172.21.15.22:6801/1357365082] conn(0x7f45040a29f0 msgr2=0x7f45040a4eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:09:36.358 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.355+0000 7f4524e00640 1 --2- 172.21.15.22:0/1844794507 >> [v2:172.21.15.22:6800/1357365082,v1:172.21.15.22:6801/1357365082] conn(0x7f45040a29f0 0x7f45040a4eb0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x5599837031a0 tx=0x7f4510009260 comp rx=0 tx=0).stop 2024-03-28T18:09:36.358 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.355+0000 7f4524e00640 1 -- 172.21.15.22:0/1844794507 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4520103840 msgr2=0x7f452019a180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:09:36.358 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.355+0000 7f4524e00640 1 --2- 172.21.15.22:0/1844794507 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4520103840 0x7f452019a180 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7f4510000c00 tx=0x7f45100045e0 comp rx=0 tx=0).stop 2024-03-28T18:09:36.358 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.355+0000 7f4524e00640 1 -- 172.21.15.22:0/1844794507 shutdown_connections 2024-03-28T18:09:36.358 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.355+0000 7f4524e00640 1 --2- 172.21.15.22:0/1844794507 >> [v2:172.21.15.22:6800/1357365082,v1:172.21.15.22:6801/1357365082] conn(0x7f45040a29f0 0x7f45040a4eb0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:09:36.358 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.355+0000 7f4524e00640 1 --2- 172.21.15.22:0/1844794507 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f4520109850 0x7f452019ea40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:09:36.358 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.355+0000 7f4524e00640 1 --2- 172.21.15.22:0/1844794507 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4520104200 0x7f452019a6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:09:36.358 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.355+0000 7f4524e00640 1 --2- 172.21.15.22:0/1844794507 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4520103840 0x7f452019a180 unknown :-1 s=CLOSED pgs=599 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:09:36.359 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.355+0000 7f4524e00640 1 -- 172.21.15.22:0/1844794507 >> 172.21.15.22:0/1844794507 conn(0x7f45200ff310 msgr2=0x7f45200ffaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:09:36.359 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.355+0000 7f4524e00640 1 -- 172.21.15.22:0/1844794507 shutdown_connections 2024-03-28T18:09:36.359 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.355+0000 7f4524e00640 1 -- 172.21.15.22:0/1844794507 wait complete. 2024-03-28T18:09:36.359 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/iio/humidity/ 2024-03-28T18:09:36.371 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/iio/humidity/dht11.txt 2024-03-28T18:09:36.372 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-28T18:09:36.373 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-03-28T18:09:36.379 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/iio/iio-bindings.txt 2024-03-28T18:09:36.399 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/iio/light/ 2024-03-28T18:09:36.408 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/iio/light/apds9300.txt 2024-03-28T18:09:36.416 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/iio/light/cm36651.txt 2024-03-28T18:09:36.427 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/iio/light/gp2ap020a00f.txt 2024-03-28T18:09:36.439 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/iio/light/tsl2563.txt 2024-03-28T18:09:36.449 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/iio/magnetometer/ 2024-03-28T18:09:36.464 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/iio/magnetometer/ak8975.txt 2024-03-28T18:09:36.470 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/iio/magnetometer/hmc5843.txt 2024-03-28T18:09:36.484 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/iio/proximity/ 2024-03-28T18:09:36.501 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/iio/proximity/as3935.txt 2024-03-28T18:09:36.510 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/iio/sensorhub.txt 2024-03-28T18:09:36.531 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/iio/st-sensors.txt 2024-03-28T18:09:36.546 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/input/ 2024-03-28T18:09:36.563 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/input/ads7846.txt 2024-03-28T18:09:36.573 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/input/atmel,maxtouch.txt 2024-03-28T18:09:36.589 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/input/cap11xx.txt 2024-03-28T18:09:36.604 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/input/clps711x-keypad.txt 2024-03-28T18:09:36.612 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.612+0000 7f2cebc00640 1 -- 172.21.15.22:0/209490104 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2ce4104200 msgr2=0x7f2ce4110710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:09:36.612 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.612+0000 7f2cebc00640 1 --2- 172.21.15.22:0/209490104 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2ce4104200 0x7f2ce4110710 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7f2cdc009cd0 tx=0x7f2cdc02c0e0 comp rx=0 tx=0).stop 2024-03-28T18:09:36.612 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.612+0000 7f2cebc00640 1 -- 172.21.15.22:0/209490104 shutdown_connections 2024-03-28T18:09:36.612 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.612+0000 7f2cebc00640 1 --2- 172.21.15.22:0/209490104 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2ce4104200 0x7f2ce4110710 unknown :-1 s=CLOSED pgs=602 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:09:36.612 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.612+0000 7f2cebc00640 1 --2- 172.21.15.22:0/209490104 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2ce4103840 0x7f2ce4103cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:09:36.612 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.612+0000 7f2cebc00640 1 --2- 172.21.15.22:0/209490104 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f2ce4109850 0x7f2ce4109c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:09:36.612 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.612+0000 7f2cebc00640 1 -- 172.21.15.22:0/209490104 >> 172.21.15.22:0/209490104 conn(0x7f2ce40ff310 msgr2=0x7f2ce4101730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:09:36.612 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.612+0000 7f2cebc00640 1 -- 172.21.15.22:0/209490104 shutdown_connections 2024-03-28T18:09:36.613 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.612+0000 7f2cebc00640 1 -- 172.21.15.22:0/209490104 wait complete. 2024-03-28T18:09:36.614 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.613+0000 7f2cebc00640 1 Processor -- start 2024-03-28T18:09:36.614 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.613+0000 7f2cebc00640 1 -- start start 2024-03-28T18:09:36.614 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.614+0000 7f2cebc00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2ce4103840 0x7f2ce419a150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:09:36.614 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.614+0000 7f2cebc00640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f2ce4104200 0x7f2ce419a690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:09:36.614 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.614+0000 7f2cebc00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2ce4109850 0x7f2ce419ea10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:09:36.614 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.614+0000 7f2cebc00640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ce4113c30 con 0x7f2ce4109850 2024-03-28T18:09:36.614 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.614+0000 7f2cebc00640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ce4113ab0 con 0x7f2ce4104200 2024-03-28T18:09:36.614 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.614+0000 7f2cebc00640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ce4113db0 con 0x7f2ce4103840 2024-03-28T18:09:36.614 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.614+0000 7f2ce9c00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2ce4109850 0x7f2ce419ea10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:09:36.615 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.614+0000 7f2ce9c00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2ce4109850 0x7f2ce419ea10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:36102/0 (socket says 172.21.15.22:36102) 2024-03-28T18:09:36.615 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.614+0000 7f2ce9c00640 1 -- 172.21.15.22:0/2090656502 learned_addr learned my addr 172.21.15.22:0/2090656502 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-28T18:09:36.615 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.614+0000 7f2ce3e00640 1 --2- 172.21.15.22:0/2090656502 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f2ce4104200 0x7f2ce419a690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:09:36.615 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.614+0000 7f2ce9200640 1 --2- 172.21.15.22:0/2090656502 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2ce4103840 0x7f2ce419a150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:09:36.616 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.615+0000 7f2ce9c00640 1 -- 172.21.15.22:0/2090656502 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2ce4103840 msgr2=0x7f2ce419a150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:09:36.616 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.615+0000 7f2ce9c00640 1 --2- 172.21.15.22:0/2090656502 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2ce4103840 0x7f2ce419a150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:09:36.616 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.615+0000 7f2ce9c00640 1 -- 172.21.15.22:0/2090656502 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f2ce4104200 msgr2=0x7f2ce419a690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:09:36.616 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.615+0000 7f2ce9c00640 1 --2- 172.21.15.22:0/2090656502 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f2ce4104200 0x7f2ce419a690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:09:36.616 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.615+0000 7f2ce9c00640 1 -- 172.21.15.22:0/2090656502 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2ce419f140 con 0x7f2ce4109850 2024-03-28T18:09:36.616 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.615+0000 7f2ce9c00640 1 --2- 172.21.15.22:0/2090656502 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2ce4109850 0x7f2ce419ea10 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7f2cdc005e30 tx=0x7f2cdc034cf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:09:36.616 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.615+0000 7f2ce1600640 1 -- 172.21.15.22:0/2090656502 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2cdc039050 con 0x7f2ce4109850 2024-03-28T18:09:36.616 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.615+0000 7f2ce1600640 1 -- 172.21.15.22:0/2090656502 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f2cdc03a040 con 0x7f2ce4109850 2024-03-28T18:09:36.616 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.615+0000 7f2cebc00640 1 -- 172.21.15.22:0/2090656502 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2ce419f3d0 con 0x7f2ce4109850 2024-03-28T18:09:36.616 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.615+0000 7f2ce1600640 1 -- 172.21.15.22:0/2090656502 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2cdc004620 con 0x7f2ce4109850 2024-03-28T18:09:36.616 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.616+0000 7f2cebc00640 1 -- 172.21.15.22:0/2090656502 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2ce41a6cc0 con 0x7f2ce4109850 2024-03-28T18:09:36.617 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/input/cros-ec-keyb.txt 2024-03-28T18:09:36.619 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.620+0000 7f2cee881740 1 Processor -- start 2024-03-28T18:09:36.620 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.620+0000 7f2ce1600640 1 -- 172.21.15.22:0/2090656502 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 26) v1 ==== 137421+0+0 (secure 0 0 0) 0x7f2cdc003970 con 0x7f2ce4109850 2024-03-28T18:09:36.620 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.620+0000 7f2cee881740 1 -- start start 2024-03-28T18:09:36.622 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.621+0000 7f2cee881740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5643bba5b6d0 0x5643bba5bab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:09:36.622 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.621+0000 7f2cee881740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5643bba5c080 0x5643bba64630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:09:36.622 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.621+0000 7f2cee881740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5643bba64c00 0x5643bba67050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:09:36.622 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.621+0000 7f2cee881740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x5643bb9bf2c0 con 0x5643bba5c080 2024-03-28T18:09:36.622 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.621+0000 7f2cee881740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x5643bb967c90 con 0x5643bba5b6d0 2024-03-28T18:09:36.622 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.621+0000 7f2cee881740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x5643bb9bb360 con 0x5643bba64c00 2024-03-28T18:09:36.622 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.621+0000 7f2ce9200640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5643bba5c080 0x5643bba64630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:09:36.622 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.621+0000 7f2ce9200640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5643bba5c080 0x5643bba64630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:36108/0 (socket says 172.21.15.22:36108) 2024-03-28T18:09:36.622 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.621+0000 7f2ce9200640 1 -- 172.21.15.22:0/3303211453 learned_addr learned my addr 172.21.15.22:0/3303211453 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-28T18:09:36.622 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.621+0000 7f2ce3e00640 1 --2- 172.21.15.22:0/3303211453 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5643bba5b6d0 0x5643bba5bab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:09:36.622 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.621+0000 7f2ce9200640 1 -- 172.21.15.22:0/3303211453 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5643bba64c00 msgr2=0x5643bba67050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T18:09:36.622 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.621+0000 7f2ce9200640 1 --2- 172.21.15.22:0/3303211453 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5643bba64c00 0x5643bba67050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:09:36.623 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.621+0000 7f2ce9200640 1 -- 172.21.15.22:0/3303211453 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5643bba5b6d0 msgr2=0x5643bba5bab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T18:09:36.623 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.621+0000 7f2ce9200640 1 --2- 172.21.15.22:0/3303211453 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5643bba5b6d0 0x5643bba5bab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:09:36.623 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.621+0000 7f2ce9200640 1 -- 172.21.15.22:0/3303211453 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5643bb98c080 con 0x5643bba5c080 2024-03-28T18:09:36.623 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.621+0000 7f2ce3e00640 1 --2- 172.21.15.22:0/3303211453 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5643bba5b6d0 0x5643bba5bab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-28T18:09:36.623 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.622+0000 7f2ce9200640 1 --2- 172.21.15.22:0/3303211453 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5643bba5c080 0x5643bba64630 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7f2cd8009bf0 tx=0x7f2cd80117b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4fa58413386f3ec9 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:09:36.623 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.622+0000 7f2ce1600640 1 --2- 172.21.15.22:0/2090656502 >> [v2:172.21.15.22:6800/1357365082,v1:172.21.15.22:6801/1357365082] conn(0x7f2ccc0a2970 0x7f2ccc0a4e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:09:36.623 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.622+0000 7f2cd2000640 1 -- 172.21.15.22:0/3303211453 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2cd8012450 con 0x5643bba5c080 2024-03-28T18:09:36.623 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.622+0000 7f2cd2000640 1 -- 172.21.15.22:0/3303211453 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f2cd8012ac0 con 0x5643bba5c080 2024-03-28T18:09:36.623 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.622+0000 7f2cd2000640 1 -- 172.21.15.22:0/3303211453 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2cd800fb50 con 0x5643bba5c080 2024-03-28T18:09:36.623 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.622+0000 7f2ce1600640 1 -- 172.21.15.22:0/2090656502 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 8411+0+0 (secure 0 0 0) 0x7f2cdc0f1380 con 0x7f2ce4109850 2024-03-28T18:09:36.623 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.622+0000 7f2ce3e00640 1 --2- 172.21.15.22:0/2090656502 >> [v2:172.21.15.22:6800/1357365082,v1:172.21.15.22:6801/1357365082] conn(0x7f2ccc0a2970 0x7f2ccc0a4e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:09:36.626 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.623+0000 7f2cee881740 1 -- 172.21.15.22:0/3303211453 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5643bba5c080 msgr2=0x5643bba64630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:09:36.626 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.623+0000 7f2ce3e00640 1 --2- 172.21.15.22:0/2090656502 >> [v2:172.21.15.22:6800/1357365082,v1:172.21.15.22:6801/1357365082] conn(0x7f2ccc0a2970 0x7f2ccc0a4e30 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f2cd400b620 tx=0x7f2cd400b5b0 comp rx=0 tx=0).ready entity=mgr.15522 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:09:36.626 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.623+0000 7f2cee881740 1 --2- 172.21.15.22:0/3303211453 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5643bba5c080 0x5643bba64630 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7f2cd8009bf0 tx=0x7f2cd80117b0 comp rx=0 tx=0).stop 2024-03-28T18:09:36.626 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.623+0000 7f2cee881740 1 -- 172.21.15.22:0/3303211453 shutdown_connections 2024-03-28T18:09:36.626 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.623+0000 7f2cee881740 1 --2- 172.21.15.22:0/3303211453 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5643bba64c00 0x5643bba67050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:09:36.626 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.623+0000 7f2cee881740 1 --2- 172.21.15.22:0/3303211453 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5643bba5c080 0x5643bba64630 unknown :-1 s=CLOSED pgs=604 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:09:36.626 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.623+0000 7f2cee881740 1 --2- 172.21.15.22:0/3303211453 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5643bba5b6d0 0x5643bba5bab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:09:36.626 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.623+0000 7f2cee881740 1 -- 172.21.15.22:0/3303211453 >> 172.21.15.22:0/3303211453 conn(0x5643bba55da0 msgr2=0x5643bba56180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:09:36.626 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.623+0000 7f2cee881740 1 -- 172.21.15.22:0/3303211453 shutdown_connections 2024-03-28T18:09:36.626 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.623+0000 7f2cee881740 1 -- 172.21.15.22:0/3303211453 wait complete. 2024-03-28T18:09:36.626 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.624+0000 7f2cee881740 1 -- start start 2024-03-28T18:09:36.626 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.624+0000 7f2cee881740 1 Processor -- start 2024-03-28T18:09:36.627 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.625+0000 7f2cee881740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5643bba5b6d0 0x5643bba64590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:09:36.627 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.625+0000 7f2cee881740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5643bba5c080 0x5643bba61920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:09:36.627 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.625+0000 7f2cee881740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5643bba64c00 0x5643bba61f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:09:36.627 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.625+0000 7f2cee881740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x5643bba67b70 con 0x5643bba5b6d0 2024-03-28T18:09:36.627 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.625+0000 7f2cee881740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x5643bb97b0d0 con 0x5643bba64c00 2024-03-28T18:09:36.627 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.625+0000 7f2cee881740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x5643bba67cf0 con 0x5643bba5c080 2024-03-28T18:09:36.627 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.625+0000 7f2cee881740 10 client.0 _ll_register_callbacks cb 0x5643bb97fe80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-28T18:09:36.627 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.625+0000 7f2ce9200640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5643bba5b6d0 0x5643bba64590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:09:36.627 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.625+0000 7f2ce9200640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5643bba5b6d0 0x5643bba64590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:36116/0 (socket says 172.21.15.22:36116) 2024-03-28T18:09:36.627 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.625+0000 7f2ce9200640 1 -- 172.21.15.22:0/3626577879 learned_addr learned my addr 172.21.15.22:0/3626577879 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-28T18:09:36.627 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.625+0000 7f2ce9c00640 1 --2- 172.21.15.22:0/3626577879 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5643bba64c00 0x5643bba61f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:09:36.627 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.625+0000 7f2ce3e00640 1 --2- 172.21.15.22:0/3626577879 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5643bba5c080 0x5643bba61920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:09:36.627 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.625+0000 7f2ce9200640 1 -- 172.21.15.22:0/3626577879 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5643bba5c080 msgr2=0x5643bba61920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T18:09:36.627 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.625+0000 7f2ce9200640 1 --2- 172.21.15.22:0/3626577879 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5643bba5c080 0x5643bba61920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:09:36.627 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.626+0000 7f2ce9200640 1 -- 172.21.15.22:0/3626577879 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5643bba64c00 msgr2=0x5643bba61f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T18:09:36.628 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.626+0000 7f2ce9200640 1 --2- 172.21.15.22:0/3626577879 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5643bba64c00 0x5643bba61f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:09:36.628 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.626+0000 7f2ce9200640 1 -- 172.21.15.22:0/3626577879 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5643bb6d33f0 con 0x5643bba5b6d0 2024-03-28T18:09:36.628 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.626+0000 7f2ce9200640 1 --2- 172.21.15.22:0/3626577879 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5643bba5b6d0 0x5643bba64590 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7f2cd8009920 tx=0x7f2cd8009950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=42b882e83453da19 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:09:36.628 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.626+0000 7f2cc6a00640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-03-28T18:09:36.628 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.626+0000 7f2cc6a00640 1 -- 172.21.15.22:0/3626577879 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2cd8028030 con 0x5643bba5b6d0 2024-03-28T18:09:36.628 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.626+0000 7f2cc6a00640 1 -- 172.21.15.22:0/3626577879 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 8411+0+0 (secure 0 0 0) 0x7f2cd8012660 con 0x5643bba5b6d0 2024-03-28T18:09:36.628 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.626+0000 7f2cee881740 1 -- 172.21.15.22:0/3626577879 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f2cd8012ac0 con 0x5643bba5b6d0 2024-03-28T18:09:36.629 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.626+0000 7f2cc6a00640 1 -- 172.21.15.22:0/3626577879 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2cd8032050 con 0x5643bba5b6d0 2024-03-28T18:09:36.629 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.627+0000 7f2cee881740 10 client.16131 fetch_fsmap learned FSMap version 27 2024-03-28T18:09:36.629 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.627+0000 7f2cee881740 1 -- 172.21.15.22:0/3626577879 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x5643bb79d7d0 con 0x5643bba5b6d0 2024-03-28T18:09:36.629 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.627+0000 7f2cc6a00640 1 -- 172.21.15.22:0/3626577879 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f2cd8012ac0 con 0x5643bba5b6d0 2024-03-28T18:09:36.629 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.627+0000 7f2cc6a00640 1 -- 172.21.15.22:0/3626577879 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 27) v1 ==== 7934+0+0 (secure 0 0 0) 0x7f2cd8005320 con 0x5643bba5b6d0 2024-03-28T18:09:36.629 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.627+0000 7f2cee881740 10 client.16131 fetch_fsmap finished waiting for FSMap version 27 2024-03-28T18:09:36.629 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.627+0000 7f2cee881740 10 client.16131 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.e 2024-03-28T18:09:36.630 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.627+0000 7f2cee881740 20 client.16131 populate_metadata read hostname 'smithi022' 2024-03-28T18:09:36.630 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.627+0000 7f2cee881740 1 --2- 172.21.15.22:0/3626577879 >> [v2:172.21.15.79:6834/3365773431,v1:172.21.15.79:6835/3365773431] conn(0x5643bba74ad0 0x5643bba76ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:09:36.630 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.627+0000 7f2cee881740 4 client.16131 mds_command: new command op to 34475 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-28T18:09:36.630 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.627+0000 7f2cee881740 1 -- 172.21.15.22:0/3626577879 --> [v2:172.21.15.79:6834/3365773431,v1:172.21.15.79:6835/3365773431] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5643bb9cc800 con 0x5643bba74ad0 2024-03-28T18:09:36.630 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.628+0000 7f2ce3e00640 1 --2- 172.21.15.22:0/3626577879 >> [v2:172.21.15.79:6834/3365773431,v1:172.21.15.79:6835/3365773431] conn(0x5643bba74ad0 0x5643bba76ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:09:36.630 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/input/e3x0-button.txt 2024-03-28T18:09:36.631 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.628+0000 7f2ce3e00640 1 --2- 172.21.15.22:0/3626577879 >> [v2:172.21.15.79:6834/3365773431,v1:172.21.15.79:6835/3365773431] conn(0x5643bba74ad0 0x5643bba76ec0 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=a840b172bd839c5f server_cookie=2b12f12ca4e42bd8 in_seq=0 out_seq=0 2024-03-28T18:09:36.631 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.628+0000 7f2cc6a00640 10 client.16131 ms_handle_connect on v2:172.21.15.79:6834/3365773431 2024-03-28T18:09:36.631 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.629+0000 7f2cc6a00640 1 -- 172.21.15.22:0/3626577879 <== mds.2 v2:172.21.15.79:6834/3365773431 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x5643bb9cc800 con 0x5643bba74ad0 2024-03-28T18:09:36.631 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.629+0000 7f2cc6a00640 10 client.16131 handle_command_reply: tid=0 2024-03-28T18:09:36.632 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.630+0000 7f2cee881740 1 client.16131 shutdown 2024-03-28T18:09:36.632 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.630+0000 7f2cee881740 1 -- 172.21.15.22:0/3626577879 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5643bba5b6d0 msgr2=0x5643bba64590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:09:36.632 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.630+0000 7f2cee881740 1 --2- 172.21.15.22:0/3626577879 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5643bba5b6d0 0x5643bba64590 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7f2cd8009920 tx=0x7f2cd8009950 comp rx=0 tx=0).stop 2024-03-28T18:09:36.632 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.630+0000 7f2cee881740 1 -- 172.21.15.22:0/3626577879 shutdown_connections 2024-03-28T18:09:36.632 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.630+0000 7f2cee881740 1 --2- 172.21.15.22:0/3626577879 >> [v2:172.21.15.79:6834/3365773431,v1:172.21.15.79:6835/3365773431] conn(0x5643bba74ad0 0x5643bba76ec0 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:09:36.632 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.630+0000 7f2cee881740 1 --2- 172.21.15.22:0/3626577879 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5643bba64c00 0x5643bba61f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:09:36.632 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.630+0000 7f2cee881740 1 --2- 172.21.15.22:0/3626577879 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5643bba5c080 0x5643bba61920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:09:36.632 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.630+0000 7f2cee881740 1 --2- 172.21.15.22:0/3626577879 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5643bba5b6d0 0x5643bba64590 unknown :-1 s=CLOSED pgs=605 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:09:36.632 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.630+0000 7f2cee881740 1 -- 172.21.15.22:0/3626577879 >> 172.21.15.22:0/3626577879 conn(0x5643bba55da0 msgr2=0x5643bba57af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:09:36.632 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.630+0000 7f2cc6a00640 0 client.16131 ms_handle_reset on v2:172.21.15.79:6834/3365773431 2024-03-28T18:09:36.632 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.630+0000 7f2cee881740 1 -- 172.21.15.22:0/3626577879 shutdown_connections 2024-03-28T18:09:36.632 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.630+0000 7f2cee881740 1 -- 172.21.15.22:0/3626577879 wait complete. 2024-03-28T18:09:36.632 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.630+0000 7f2cee881740 20 client.16131 trim_cache size 0 max 16384 2024-03-28T18:09:36.637 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/input/elan_i2c.txt 2024-03-28T18:09:36.645 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.645+0000 7f2cee881740 1 Processor -- start 2024-03-28T18:09:36.645 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.645+0000 7f2cee881740 1 -- start start 2024-03-28T18:09:36.646 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.646+0000 7f2cee881740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5643bba5b6d0 0x5643bba64050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:09:36.646 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.646+0000 7f2cee881740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5643bba5c080 0x5643bba64590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:09:36.646 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.646+0000 7f2cee881740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5643bba64c00 0x5643bba5eba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:09:36.646 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.646+0000 7f2cee881740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ce41af2d0 con 0x5643bba5b6d0 2024-03-28T18:09:36.646 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.646+0000 7f2cee881740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ce41b0c50 con 0x5643bba64c00 2024-03-28T18:09:36.647 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.646+0000 7f2cee881740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ce41af930 con 0x5643bba5c080 2024-03-28T18:09:36.647 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.646+0000 7f2ce9200640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5643bba5b6d0 0x5643bba64050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:09:36.647 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.646+0000 7f2ce9200640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5643bba5b6d0 0x5643bba64050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:36124/0 (socket says 172.21.15.22:36124) 2024-03-28T18:09:36.647 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.646+0000 7f2ce9200640 1 -- 172.21.15.22:0/1513447561 learned_addr learned my addr 172.21.15.22:0/1513447561 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-28T18:09:36.647 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.646+0000 7f2ce9c00640 1 --2- 172.21.15.22:0/1513447561 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5643bba64c00 0x5643bba5eba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:09:36.647 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.646+0000 7f2ce3e00640 1 --2- 172.21.15.22:0/1513447561 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5643bba5c080 0x5643bba64590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:09:36.647 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.646+0000 7f2ce9200640 1 -- 172.21.15.22:0/1513447561 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5643bba5c080 msgr2=0x5643bba64590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T18:09:36.647 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.646+0000 7f2ce9200640 1 --2- 172.21.15.22:0/1513447561 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5643bba5c080 0x5643bba64590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:09:36.647 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.647+0000 7f2ce9200640 1 -- 172.21.15.22:0/1513447561 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5643bba64c00 msgr2=0x5643bba5eba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T18:09:36.647 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.647+0000 7f2ce9200640 1 --2- 172.21.15.22:0/1513447561 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5643bba64c00 0x5643bba5eba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:09:36.647 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.647+0000 7f2ce9200640 1 -- 172.21.15.22:0/1513447561 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5643bba51510 con 0x5643bba5b6d0 2024-03-28T18:09:36.647 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.647+0000 7f2ce9200640 1 --2- 172.21.15.22:0/1513447561 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5643bba5b6d0 0x5643bba64050 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7f2cd8024040 tx=0x7f2cd8004680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d449796a76620287 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:09:36.648 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.647+0000 7f2cd1600640 1 -- 172.21.15.22:0/1513447561 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2cd803a020 con 0x5643bba5b6d0 2024-03-28T18:09:36.649 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.647+0000 7f2cd1600640 1 -- 172.21.15.22:0/1513447561 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f2cd8011cb0 con 0x5643bba5b6d0 2024-03-28T18:09:36.649 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.647+0000 7f2cd1600640 1 -- 172.21.15.22:0/1513447561 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2cd8028070 con 0x5643bba5b6d0 2024-03-28T18:09:36.649 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.648+0000 7f2cee881740 1 -- 172.21.15.22:0/1513447561 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5643bba5b6d0 msgr2=0x5643bba64050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:09:36.649 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.648+0000 7f2cee881740 1 --2- 172.21.15.22:0/1513447561 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5643bba5b6d0 0x5643bba64050 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7f2cd8024040 tx=0x7f2cd8004680 comp rx=0 tx=0).stop 2024-03-28T18:09:36.649 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.648+0000 7f2cee881740 1 -- 172.21.15.22:0/1513447561 shutdown_connections 2024-03-28T18:09:36.649 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.648+0000 7f2cee881740 1 --2- 172.21.15.22:0/1513447561 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5643bba64c00 0x5643bba5eba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:09:36.649 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.648+0000 7f2cee881740 1 --2- 172.21.15.22:0/1513447561 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5643bba5c080 0x5643bba64590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:09:36.649 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.648+0000 7f2cee881740 1 --2- 172.21.15.22:0/1513447561 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5643bba5b6d0 0x5643bba64050 unknown :-1 s=CLOSED pgs=606 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:09:36.649 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.648+0000 7f2cee881740 1 -- 172.21.15.22:0/1513447561 >> 172.21.15.22:0/1513447561 conn(0x5643bba55da0 msgr2=0x5643bba57a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:09:36.649 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.648+0000 7f2cee881740 1 -- 172.21.15.22:0/1513447561 shutdown_connections 2024-03-28T18:09:36.649 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.648+0000 7f2cee881740 1 -- 172.21.15.22:0/1513447561 wait complete. 2024-03-28T18:09:36.649 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.649+0000 7f2cee881740 1 -- start start 2024-03-28T18:09:36.650 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/input/elants_i2c.txt 2024-03-28T18:09:36.652 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.649+0000 7f2cee881740 1 Processor -- start 2024-03-28T18:09:36.652 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.649+0000 7f2cee881740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5643bba5b6d0 0x5643bba6a2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:09:36.652 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.649+0000 7f2cee881740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5643bba5c080 0x5643bba6a7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:09:36.652 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.649+0000 7f2cee881740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5643bba64c00 0x5643bba6ad30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:09:36.652 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.649+0000 7f2cee881740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ce41b05f0 con 0x5643bba64c00 2024-03-28T18:09:36.652 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.649+0000 7f2cee881740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ce41aff90 con 0x5643bba5c080 2024-03-28T18:09:36.652 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.649+0000 7f2cee881740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ce41ad4b0 con 0x5643bba5b6d0 2024-03-28T18:09:36.652 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.649+0000 7f2cee881740 10 client.0 _ll_register_callbacks cb 0x7f2cd8009a70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-28T18:09:36.652 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.650+0000 7f2ce3e00640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5643bba5c080 0x5643bba6a7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:09:36.653 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.650+0000 7f2ce9c00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5643bba64c00 0x5643bba6ad30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:09:36.653 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.650+0000 7f2ce3e00640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5643bba5c080 0x5643bba6a7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.22:57750/0 (socket says 172.21.15.22:57750) 2024-03-28T18:09:36.653 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.650+0000 7f2ce3e00640 1 -- 172.21.15.22:0/3576537282 learned_addr learned my addr 172.21.15.22:0/3576537282 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-28T18:09:36.653 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.650+0000 7f2ce9200640 1 --2- 172.21.15.22:0/3576537282 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5643bba5b6d0 0x5643bba6a2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:09:36.653 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.650+0000 7f2ce3e00640 1 -- 172.21.15.22:0/3576537282 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5643bba5b6d0 msgr2=0x5643bba6a2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T18:09:36.653 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.650+0000 7f2ce3e00640 1 --2- 172.21.15.22:0/3576537282 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5643bba5b6d0 0x5643bba6a2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:09:36.653 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.650+0000 7f2ce3e00640 1 -- 172.21.15.22:0/3576537282 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5643bba64c00 msgr2=0x5643bba6ad30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T18:09:36.653 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.650+0000 7f2ce3e00640 1 --2- 172.21.15.22:0/3576537282 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5643bba64c00 0x5643bba6ad30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:09:36.653 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.650+0000 7f2ce3e00640 1 -- 172.21.15.22:0/3576537282 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5643bbacf060 con 0x5643bba5c080 2024-03-28T18:09:36.653 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.650+0000 7f2ce9200640 1 --2- 172.21.15.22:0/3576537282 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5643bba5b6d0 0x5643bba6a2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-28T18:09:36.653 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.650+0000 7f2ce9c00640 1 --2- 172.21.15.22:0/3576537282 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5643bba64c00 0x5643bba6ad30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-28T18:09:36.653 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.651+0000 7f2ce3e00640 1 --2- 172.21.15.22:0/3576537282 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5643bba5c080 0x5643bba6a7f0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x5643bba56be0 tx=0x7f2cd40046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b6a64629e1fa8eaf server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:09:36.653 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.651+0000 7f2cc6a00640 10 client.0 ms_handle_connect on v2:172.21.15.79:3300/0 2024-03-28T18:09:36.653 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.651+0000 7f2cc6a00640 1 -- 172.21.15.22:0/3576537282 <== mon.2 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2cd4020b40 con 0x5643bba5c080 2024-03-28T18:09:36.653 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.651+0000 7f2cc6a00640 1 -- 172.21.15.22:0/3576537282 <== mon.2 v2:172.21.15.79:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 8411+0+0 (secure 0 0 0) 0x7f2cd4020ce0 con 0x5643bba5c080 2024-03-28T18:09:36.654 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.651+0000 7f2cee881740 1 -- 172.21.15.22:0/3576537282 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f2cd8011cb0 con 0x5643bba5c080 2024-03-28T18:09:36.654 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.651+0000 7f2cc6a00640 1 -- 172.21.15.22:0/3576537282 <== mon.2 v2:172.21.15.79:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2cd4038050 con 0x5643bba5c080 2024-03-28T18:09:36.654 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.652+0000 7f2cee881740 10 client.35966 fetch_fsmap learned FSMap version 27 2024-03-28T18:09:36.654 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.652+0000 7f2cee881740 1 -- 172.21.15.22:0/3576537282 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x5643bbad0310 con 0x5643bba5c080 2024-03-28T18:09:36.654 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.652+0000 7f2cc6a00640 1 -- 172.21.15.22:0/3576537282 <== mon.2 v2:172.21.15.79:3300/0 4 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f2cd8011cb0 con 0x5643bba5c080 2024-03-28T18:09:36.654 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.652+0000 7f2cc6a00640 1 -- 172.21.15.22:0/3576537282 <== mon.2 v2:172.21.15.79:3300/0 5 ==== fsmap(e 27) v1 ==== 7934+0+0 (secure 0 0 0) 0x7f2cd401f520 con 0x5643bba5c080 2024-03-28T18:09:36.654 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.652+0000 7f2cee881740 10 client.35966 fetch_fsmap finished waiting for FSMap version 27 2024-03-28T18:09:36.654 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.652+0000 7f2cee881740 10 client.35966 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.e 2024-03-28T18:09:36.654 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.652+0000 7f2cee881740 20 client.35966 populate_metadata read hostname 'smithi022' 2024-03-28T18:09:36.654 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.652+0000 7f2cee881740 1 --2- 172.21.15.22:0/3576537282 >> [v2:172.21.15.79:6834/3365773431,v1:172.21.15.79:6835/3365773431] conn(0x5643bbad0a20 0x5643bbad2e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:09:36.654 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.652+0000 7f2cee881740 4 client.35966 mds_command: new command op to 34475 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-28T18:09:36.654 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.652+0000 7f2cee881740 1 -- 172.21.15.22:0/3576537282 --> [v2:172.21.15.79:6834/3365773431,v1:172.21.15.79:6835/3365773431] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x5643bb9c6b50 con 0x5643bbad0a20 2024-03-28T18:09:36.655 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.653+0000 7f2ce9200640 1 --2- 172.21.15.22:0/3576537282 >> [v2:172.21.15.79:6834/3365773431,v1:172.21.15.79:6835/3365773431] conn(0x5643bbad0a20 0x5643bbad2e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:09:36.656 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.653+0000 7f2ce9200640 1 --2- 172.21.15.22:0/3576537282 >> [v2:172.21.15.79:6834/3365773431,v1:172.21.15.79:6835/3365773431] conn(0x5643bbad0a20 0x5643bbad2e40 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=4efb144bcff81995 server_cookie=552b85b063ff547b in_seq=0 out_seq=0 2024-03-28T18:09:36.656 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.653+0000 7f2cc6a00640 10 client.35966 ms_handle_connect on v2:172.21.15.79:6834/3365773431 2024-03-28T18:09:36.656 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.654+0000 7f2cc6a00640 1 -- 172.21.15.22:0/3576537282 <== mds.2 v2:172.21.15.79:6834/3365773431 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x5643bb9c6b50 con 0x5643bbad0a20 2024-03-28T18:09:36.656 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.654+0000 7f2cc6a00640 10 client.35966 handle_command_reply: tid=0 2024-03-28T18:09:36.656 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.654+0000 7f2cee881740 1 client.35966 shutdown 2024-03-28T18:09:36.656 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.654+0000 7f2cee881740 1 -- 172.21.15.22:0/3576537282 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5643bba5c080 msgr2=0x5643bba6a7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:09:36.656 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.654+0000 7f2cee881740 1 --2- 172.21.15.22:0/3576537282 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5643bba5c080 0x5643bba6a7f0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x5643bba56be0 tx=0x7f2cd40046c0 comp rx=0 tx=0).stop 2024-03-28T18:09:36.656 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.654+0000 7f2cee881740 1 -- 172.21.15.22:0/3576537282 shutdown_connections 2024-03-28T18:09:36.656 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.654+0000 7f2cee881740 1 --2- 172.21.15.22:0/3576537282 >> [v2:172.21.15.79:6834/3365773431,v1:172.21.15.79:6835/3365773431] conn(0x5643bbad0a20 0x5643bbad2e40 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:09:36.656 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.654+0000 7f2cee881740 1 --2- 172.21.15.22:0/3576537282 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5643bba64c00 0x5643bba6ad30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:09:36.656 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.654+0000 7f2cee881740 1 --2- 172.21.15.22:0/3576537282 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5643bba5c080 0x5643bba6a7f0 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:09:36.656 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.654+0000 7f2cee881740 1 --2- 172.21.15.22:0/3576537282 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5643bba5b6d0 0x5643bba6a2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:09:36.656 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.654+0000 7f2cee881740 1 -- 172.21.15.22:0/3576537282 >> 172.21.15.22:0/3576537282 conn(0x5643bba55da0 msgr2=0x5643bba54620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:09:36.656 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.654+0000 7f2cc6a00640 0 client.35966 ms_handle_reset on v2:172.21.15.79:6834/3365773431 2024-03-28T18:09:36.656 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.654+0000 7f2cee881740 1 -- 172.21.15.22:0/3576537282 shutdown_connections 2024-03-28T18:09:36.657 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.654+0000 7f2cee881740 1 -- 172.21.15.22:0/3576537282 wait complete. 2024-03-28T18:09:36.657 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.654+0000 7f2cee881740 20 client.35966 trim_cache size 0 max 16384 2024-03-28T18:09:36.657 INFO:teuthology.orchestra.run.smithi022.stdout:[] 2024-03-28T18:09:36.657 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.655+0000 7f2cebc00640 1 -- 172.21.15.22:0/2090656502 >> [v2:172.21.15.22:6800/1357365082,v1:172.21.15.22:6801/1357365082] conn(0x7f2ccc0a2970 msgr2=0x7f2ccc0a4e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:09:36.657 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.655+0000 7f2cebc00640 1 --2- 172.21.15.22:0/2090656502 >> [v2:172.21.15.22:6800/1357365082,v1:172.21.15.22:6801/1357365082] conn(0x7f2ccc0a2970 0x7f2ccc0a4e30 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f2cd400b620 tx=0x7f2cd400b5b0 comp rx=0 tx=0).stop 2024-03-28T18:09:36.658 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.655+0000 7f2cebc00640 1 -- 172.21.15.22:0/2090656502 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2ce4109850 msgr2=0x7f2ce419ea10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:09:36.658 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.655+0000 7f2cebc00640 1 --2- 172.21.15.22:0/2090656502 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2ce4109850 0x7f2ce419ea10 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7f2cdc005e30 tx=0x7f2cdc034cf0 comp rx=0 tx=0).stop 2024-03-28T18:09:36.658 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.655+0000 7f2cebc00640 1 -- 172.21.15.22:0/2090656502 shutdown_connections 2024-03-28T18:09:36.658 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.655+0000 7f2cebc00640 1 --2- 172.21.15.22:0/2090656502 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2ce4109850 0x7f2ce419ea10 unknown :-1 s=CLOSED pgs=603 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:09:36.658 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.655+0000 7f2cebc00640 1 --2- 172.21.15.22:0/2090656502 >> [v2:172.21.15.22:6800/1357365082,v1:172.21.15.22:6801/1357365082] conn(0x7f2ccc0a2970 0x7f2ccc0a4e30 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:09:36.658 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.655+0000 7f2cebc00640 1 --2- 172.21.15.22:0/2090656502 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f2ce4104200 0x7f2ce419a690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:09:36.658 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.655+0000 7f2cebc00640 1 --2- 172.21.15.22:0/2090656502 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2ce4103840 0x7f2ce419a150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:09:36.658 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.655+0000 7f2cebc00640 1 -- 172.21.15.22:0/2090656502 >> 172.21.15.22:0/2090656502 conn(0x7f2ce40ff310 msgr2=0x7f2ce40ff990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:09:36.658 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.655+0000 7f2cebc00640 1 -- 172.21.15.22:0/2090656502 shutdown_connections 2024-03-28T18:09:36.658 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.655+0000 7f2cebc00640 1 -- 172.21.15.22:0/2090656502 wait complete. 2024-03-28T18:09:36.658 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/input/fsl-mma8450.txt 2024-03-28T18:09:36.668 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/input/gpio-beeper.txt 2024-03-28T18:09:36.672 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:09:36 smithi022 ceph-mon[31985]: pgmap v64: 97 pgs: 97 active+clean; 91 MiB data, 694 MiB used, 1.0 TiB / 1.0 TiB avail; 318 KiB/s rd, 800 KiB/s wr, 161 op/s 2024-03-28T18:09:36.672 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:09:36 smithi022 ceph-mon[31985]: from='client.? 172.21.15.22:0/868073401' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-28T18:09:36.676 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-28T18:09:36.677 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-03-28T18:09:36.680 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/input/gpio-keys-polled.txt 2024-03-28T18:09:36.693 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/input/gpio-keys.txt 2024-03-28T18:09:36.705 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/input/gpio-matrix-keypad.txt 2024-03-28T18:09:36.716 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/input/imx-keypad.txt 2024-03-28T18:09:36.725 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/input/input-reset.txt 2024-03-28T18:09:36.736 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/input/lpc32xx-key.txt 2024-03-28T18:09:36.747 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/input/matrix-keymap.txt 2024-03-28T18:09:36.761 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/input/nvidia,tegra20-kbc.txt 2024-03-28T18:09:36.776 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/input/omap-keypad.txt 2024-03-28T18:09:36.789 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/input/ps2keyb-mouse-apbps2.txt 2024-03-28T18:09:36.802 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/input/pwm-beeper.txt 2024-03-28T18:09:36.817 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/input/pxa27x-keypad.txt 2024-03-28T18:09:36.831 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/input/qcom,pm8xxx-keypad.txt 2024-03-28T18:09:36.847 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/input/qcom,pm8xxx-pwrkey.txt 2024-03-28T18:09:36.862 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/input/qcom,pm8xxx-vib.txt 2024-03-28T18:09:36.877 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/input/regulator-haptic.txt 2024-03-28T18:09:36.890 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/input/rotary-encoder.txt 2024-03-28T18:09:36.902 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/input/samsung-keypad.txt 2024-03-28T18:09:36.912 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.912+0000 7fc0db000640 1 -- 172.21.15.22:0/30759739 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc0d4104200 msgr2=0x7fc0d4110710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:09:36.912 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.912+0000 7fc0db000640 1 --2- 172.21.15.22:0/30759739 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc0d4104200 0x7fc0d4110710 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7fc0cc009cd0 tx=0x7fc0cc02c0e0 comp rx=0 tx=0).stop 2024-03-28T18:09:36.912 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.912+0000 7fc0db000640 1 -- 172.21.15.22:0/30759739 shutdown_connections 2024-03-28T18:09:36.912 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.912+0000 7fc0db000640 1 --2- 172.21.15.22:0/30759739 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc0d4104200 0x7fc0d4110710 unknown :-1 s=CLOSED pgs=607 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:09:36.912 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.912+0000 7fc0db000640 1 --2- 172.21.15.22:0/30759739 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc0d4103840 0x7fc0d4103cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:09:36.912 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.912+0000 7fc0db000640 1 --2- 172.21.15.22:0/30759739 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc0d4109850 0x7fc0d4109c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:09:36.912 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.912+0000 7fc0db000640 1 -- 172.21.15.22:0/30759739 >> 172.21.15.22:0/30759739 conn(0x7fc0d40ff2f0 msgr2=0x7fc0d4101710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:09:36.912 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.912+0000 7fc0db000640 1 -- 172.21.15.22:0/30759739 shutdown_connections 2024-03-28T18:09:36.912 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.912+0000 7fc0db000640 1 -- 172.21.15.22:0/30759739 wait complete. 2024-03-28T18:09:36.913 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/input/spear-keyboard.txt 2024-03-28T18:09:36.914 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.913+0000 7fc0db000640 1 Processor -- start 2024-03-28T18:09:36.914 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.914+0000 7fc0db000640 1 -- start start 2024-03-28T18:09:36.914 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.914+0000 7fc0db000640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc0d4103840 0x7fc0d419a0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:09:36.914 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.914+0000 7fc0db000640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc0d4104200 0x7fc0d419a5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:09:36.914 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.914+0000 7fc0db000640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc0d4109850 0x7fc0d419e970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:09:36.914 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.914+0000 7fc0db000640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0d4113c30 con 0x7fc0d4103840 2024-03-28T18:09:36.914 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.914+0000 7fc0db000640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0d4113ab0 con 0x7fc0d4109850 2024-03-28T18:09:36.915 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.914+0000 7fc0db000640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0d4113db0 con 0x7fc0d4104200 2024-03-28T18:09:36.915 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.914+0000 7fc0d3e00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc0d4103840 0x7fc0d419a0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:09:36.915 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.914+0000 7fc0d3e00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc0d4103840 0x7fc0d419a0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:36146/0 (socket says 172.21.15.22:36146) 2024-03-28T18:09:36.915 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.914+0000 7fc0d3e00640 1 -- 172.21.15.22:0/3644885839 learned_addr learned my addr 172.21.15.22:0/3644885839 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-28T18:09:36.915 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.915+0000 7fc0d9000640 1 --2- 172.21.15.22:0/3644885839 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc0d4109850 0x7fc0d419e970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:09:36.915 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.915+0000 7fc0d3400640 1 --2- 172.21.15.22:0/3644885839 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc0d4104200 0x7fc0d419a5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:09:36.916 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.915+0000 7fc0d3e00640 1 -- 172.21.15.22:0/3644885839 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc0d4104200 msgr2=0x7fc0d419a5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:09:36.916 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.915+0000 7fc0d3e00640 1 --2- 172.21.15.22:0/3644885839 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc0d4104200 0x7fc0d419a5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:09:36.916 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.915+0000 7fc0d3e00640 1 -- 172.21.15.22:0/3644885839 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc0d4109850 msgr2=0x7fc0d419e970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:09:36.916 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.915+0000 7fc0d3e00640 1 --2- 172.21.15.22:0/3644885839 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc0d4109850 0x7fc0d419e970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:09:36.916 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.915+0000 7fc0d3e00640 1 -- 172.21.15.22:0/3644885839 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc0d419f0a0 con 0x7fc0d4103840 2024-03-28T18:09:36.916 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.915+0000 7fc0d3e00640 1 --2- 172.21.15.22:0/3644885839 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc0d4103840 0x7fc0d419a0b0 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7fc0c400e9f0 tx=0x7fc0c400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:09:36.916 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.916+0000 7fc0d0c00640 1 -- 172.21.15.22:0/3644885839 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0c400cd80 con 0x7fc0d4103840 2024-03-28T18:09:36.916 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.916+0000 7fc0db000640 1 -- 172.21.15.22:0/3644885839 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc0d419f390 con 0x7fc0d4103840 2024-03-28T18:09:36.916 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.916+0000 7fc0d0c00640 1 -- 172.21.15.22:0/3644885839 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7fc0c4010900 con 0x7fc0d4103840 2024-03-28T18:09:36.916 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.916+0000 7fc0d0c00640 1 -- 172.21.15.22:0/3644885839 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0c40189e0 con 0x7fc0d4103840 2024-03-28T18:09:36.916 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.916+0000 7fc0db000640 1 -- 172.21.15.22:0/3644885839 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc0d41a6c20 con 0x7fc0d4103840 2024-03-28T18:09:36.919 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.919+0000 7fc0d0c00640 1 -- 172.21.15.22:0/3644885839 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 26) v1 ==== 137421+0+0 (secure 0 0 0) 0x7fc0c4010ab0 con 0x7fc0d4103840 2024-03-28T18:09:36.919 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.919+0000 7fc0ddd68740 1 Processor -- start 2024-03-28T18:09:36.919 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.920+0000 7fc0ddd68740 1 -- start start 2024-03-28T18:09:36.921 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.920+0000 7fc0ddd68740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f8cf1186d0 0x55f8cf118ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:09:36.921 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.920+0000 7fc0ddd68740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f8cf119080 0x55f8cf121630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:09:36.921 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.920+0000 7fc0ddd68740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55f8cf121c00 0x55f8cf124050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:09:36.921 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.920+0000 7fc0ddd68740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f8cf07c2c0 con 0x55f8cf1186d0 2024-03-28T18:09:36.921 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.920+0000 7fc0ddd68740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f8cf024c90 con 0x55f8cf121c00 2024-03-28T18:09:36.921 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.920+0000 7fc0ddd68740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f8cf078360 con 0x55f8cf119080 2024-03-28T18:09:36.921 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.920+0000 7fc0d9000640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f8cf1186d0 0x55f8cf118ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:09:36.921 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.920+0000 7fc0d9000640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f8cf1186d0 0x55f8cf118ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:36150/0 (socket says 172.21.15.22:36150) 2024-03-28T18:09:36.921 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.920+0000 7fc0d9000640 1 -- 172.21.15.22:0/2347066953 learned_addr learned my addr 172.21.15.22:0/2347066953 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-28T18:09:36.922 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.920+0000 7fc0d3400640 1 --2- 172.21.15.22:0/2347066953 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55f8cf121c00 0x55f8cf124050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:09:36.922 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.921+0000 7fc0d3e00640 1 --2- 172.21.15.22:0/2347066953 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f8cf119080 0x55f8cf121630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:09:36.922 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.921+0000 7fc0d9000640 1 -- 172.21.15.22:0/2347066953 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f8cf119080 msgr2=0x55f8cf121630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T18:09:36.922 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.921+0000 7fc0d9000640 1 --2- 172.21.15.22:0/2347066953 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f8cf119080 0x55f8cf121630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:09:36.922 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.921+0000 7fc0d9000640 1 -- 172.21.15.22:0/2347066953 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55f8cf121c00 msgr2=0x55f8cf124050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T18:09:36.922 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.921+0000 7fc0d9000640 1 --2- 172.21.15.22:0/2347066953 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55f8cf121c00 0x55f8cf124050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:09:36.922 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.921+0000 7fc0d9000640 1 -- 172.21.15.22:0/2347066953 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f8cf049080 con 0x55f8cf1186d0 2024-03-28T18:09:36.922 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.921+0000 7fc0d0c00640 1 --2- 172.21.15.22:0/3644885839 >> [v2:172.21.15.22:6800/1357365082,v1:172.21.15.22:6801/1357365082] conn(0x7fc0b80a29f0 0x7fc0b80a4eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:09:36.922 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.921+0000 7fc0d9000640 1 --2- 172.21.15.22:0/2347066953 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f8cf1186d0 0x55f8cf118ab0 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7fc0cc004af0 tx=0x7fc0cc009910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e1382346d44aff08 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:09:36.922 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.922+0000 7fc0d3e00640 1 --2- 172.21.15.22:0/3644885839 >> [v2:172.21.15.22:6800/1357365082,v1:172.21.15.22:6801/1357365082] conn(0x7fc0b80a29f0 0x7fc0b80a4eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:09:36.922 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.922+0000 7fc0d0c00640 1 -- 172.21.15.22:0/3644885839 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 8411+0+0 (secure 0 0 0) 0x7fc0c40cf110 con 0x7fc0d4103840 2024-03-28T18:09:36.923 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.922+0000 7fc0c9600640 1 -- 172.21.15.22:0/2347066953 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0cc045020 con 0x55f8cf1186d0 2024-03-28T18:09:36.923 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.922+0000 7fc0c9600640 1 -- 172.21.15.22:0/2347066953 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7fc0cc03a040 con 0x55f8cf1186d0 2024-03-28T18:09:36.923 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.922+0000 7fc0c9600640 1 -- 172.21.15.22:0/2347066953 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0cc039040 con 0x55f8cf1186d0 2024-03-28T18:09:36.923 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.923+0000 7fc0d3e00640 1 --2- 172.21.15.22:0/3644885839 >> [v2:172.21.15.22:6800/1357365082,v1:172.21.15.22:6801/1357365082] conn(0x7fc0b80a29f0 0x7fc0b80a4eb0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x55f8cf1181a0 tx=0x7fc0c400cf20 comp rx=0 tx=0).ready entity=mgr.15522 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:09:36.923 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.923+0000 7fc0ddd68740 1 -- 172.21.15.22:0/2347066953 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f8cf1186d0 msgr2=0x55f8cf118ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:09:36.924 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.923+0000 7fc0ddd68740 1 --2- 172.21.15.22:0/2347066953 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f8cf1186d0 0x55f8cf118ab0 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7fc0cc004af0 tx=0x7fc0cc009910 comp rx=0 tx=0).stop 2024-03-28T18:09:36.924 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.923+0000 7fc0ddd68740 1 -- 172.21.15.22:0/2347066953 shutdown_connections 2024-03-28T18:09:36.924 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.923+0000 7fc0ddd68740 1 --2- 172.21.15.22:0/2347066953 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55f8cf121c00 0x55f8cf124050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:09:36.924 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.923+0000 7fc0ddd68740 1 --2- 172.21.15.22:0/2347066953 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f8cf119080 0x55f8cf121630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:09:36.924 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.923+0000 7fc0ddd68740 1 --2- 172.21.15.22:0/2347066953 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f8cf1186d0 0x55f8cf118ab0 unknown :-1 s=CLOSED pgs=609 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:09:36.924 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.923+0000 7fc0ddd68740 1 -- 172.21.15.22:0/2347066953 >> 172.21.15.22:0/2347066953 conn(0x55f8cf112da0 msgr2=0x55f8cf113180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:09:36.924 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.923+0000 7fc0ddd68740 1 -- 172.21.15.22:0/2347066953 shutdown_connections 2024-03-28T18:09:36.924 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.923+0000 7fc0ddd68740 1 -- 172.21.15.22:0/2347066953 wait complete. 2024-03-28T18:09:36.924 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/input/st-keyscan.txt 2024-03-28T18:09:36.926 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.924+0000 7fc0ddd68740 1 -- start start 2024-03-28T18:09:36.926 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.924+0000 7fc0ddd68740 1 Processor -- start 2024-03-28T18:09:36.926 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.925+0000 7fc0ddd68740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f8cf1186d0 0x55f8cf1214a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:09:36.926 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.925+0000 7fc0ddd68740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55f8cf119080 0x55f8cf11e830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:09:36.926 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.925+0000 7fc0ddd68740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f8cf121c00 0x55f8cf11ed70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:09:36.926 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.925+0000 7fc0ddd68740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f8cf124b10 con 0x55f8cf1186d0 2024-03-28T18:09:36.926 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.925+0000 7fc0ddd68740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f8cf0380d0 con 0x55f8cf119080 2024-03-28T18:09:36.927 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.925+0000 7fc0ddd68740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f8cf124c90 con 0x55f8cf121c00 2024-03-28T18:09:36.927 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.925+0000 7fc0ddd68740 10 client.0 _ll_register_callbacks cb 0x55f8cf03ce80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-28T18:09:36.927 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.925+0000 7fc0d3400640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55f8cf119080 0x55f8cf11e830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:09:36.927 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.925+0000 7fc0d3400640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55f8cf119080 0x55f8cf11e830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.22:57788/0 (socket says 172.21.15.22:57788) 2024-03-28T18:09:36.927 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.925+0000 7fc0d3400640 1 -- 172.21.15.22:0/4178235787 learned_addr learned my addr 172.21.15.22:0/4178235787 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-28T18:09:36.927 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.925+0000 7fc0d9000640 1 --2- 172.21.15.22:0/4178235787 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f8cf1186d0 0x55f8cf1214a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:09:36.927 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.925+0000 7fc0d9000640 1 --2- 172.21.15.22:0/4178235787 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f8cf121c00 0x55f8cf11ed70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:09:36.927 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.926+0000 7fc0d3400640 1 -- 172.21.15.22:0/4178235787 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f8cf121c00 msgr2=0x55f8cf11ed70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T18:09:36.927 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.926+0000 7fc0d3400640 1 --2- 172.21.15.22:0/4178235787 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f8cf121c00 0x55f8cf11ed70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:09:36.927 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.926+0000 7fc0d3400640 1 -- 172.21.15.22:0/4178235787 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f8cf1186d0 msgr2=0x55f8cf1214a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T18:09:36.927 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.926+0000 7fc0d3400640 1 --2- 172.21.15.22:0/4178235787 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f8cf1186d0 0x55f8cf1214a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:09:36.927 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.926+0000 7fc0d3400640 1 -- 172.21.15.22:0/4178235787 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55f8ced903f0 con 0x55f8cf119080 2024-03-28T18:09:36.927 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.926+0000 7fc0d3400640 1 --2- 172.21.15.22:0/4178235787 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55f8cf119080 0x55f8cf11e830 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fc0c000ce00 tx=0x7fc0c00039c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=33a6c99d4762984f server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:09:36.927 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.926+0000 7fc0be000640 10 client.0 ms_handle_connect on v2:172.21.15.79:3300/0 2024-03-28T18:09:36.927 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.926+0000 7fc0be000640 1 -- 172.21.15.22:0/4178235787 <== mon.2 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0c0011970 con 0x55f8cf119080 2024-03-28T18:09:36.927 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.927+0000 7fc0be000640 1 -- 172.21.15.22:0/4178235787 <== mon.2 v2:172.21.15.79:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 8411+0+0 (secure 0 0 0) 0x7fc0c0011b10 con 0x55f8cf119080 2024-03-28T18:09:36.928 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.927+0000 7fc0ddd68740 1 -- 172.21.15.22:0/4178235787 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fc0cc03a040 con 0x55f8cf119080 2024-03-28T18:09:36.928 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:09:36 smithi134 ceph-mon[35037]: pgmap v64: 97 pgs: 97 active+clean; 91 MiB data, 694 MiB used, 1.0 TiB / 1.0 TiB avail; 318 KiB/s rd, 800 KiB/s wr, 161 op/s 2024-03-28T18:09:36.928 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:09:36 smithi134 ceph-mon[35037]: from='client.? 172.21.15.22:0/868073401' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-28T18:09:36.929 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.927+0000 7fc0be000640 1 -- 172.21.15.22:0/4178235787 <== mon.2 v2:172.21.15.79:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc0c0011d10 con 0x55f8cf119080 2024-03-28T18:09:36.929 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.927+0000 7fc0ddd68740 10 client.35978 fetch_fsmap learned FSMap version 27 2024-03-28T18:09:36.929 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.927+0000 7fc0ddd68740 1 -- 172.21.15.22:0/4178235787 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55f8cee5a7d0 con 0x55f8cf119080 2024-03-28T18:09:36.930 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.927+0000 7fc0be000640 1 -- 172.21.15.22:0/4178235787 <== mon.2 v2:172.21.15.79:3300/0 4 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7fc0cc03a040 con 0x55f8cf119080 2024-03-28T18:09:36.930 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.928+0000 7fc0be000640 1 -- 172.21.15.22:0/4178235787 <== mon.2 v2:172.21.15.79:3300/0 5 ==== fsmap(e 27) v1 ==== 7934+0+0 (secure 0 0 0) 0x7fc0c0018070 con 0x55f8cf119080 2024-03-28T18:09:36.930 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.928+0000 7fc0ddd68740 10 client.35978 fetch_fsmap finished waiting for FSMap version 27 2024-03-28T18:09:36.930 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.928+0000 7fc0ddd68740 10 client.35978 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-03-28T18:09:36.930 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.928+0000 7fc0ddd68740 20 client.35978 populate_metadata read hostname 'smithi022' 2024-03-28T18:09:36.930 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.928+0000 7fc0ddd68740 1 --2- 172.21.15.22:0/4178235787 >> [v2:172.21.15.79:6836/1787903366,v1:172.21.15.79:6837/1787903366] conn(0x55f8cf131a20 0x55f8cf133e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:09:36.930 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.928+0000 7fc0ddd68740 4 client.35978 mds_command: new command op to 34511 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-28T18:09:36.930 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.928+0000 7fc0ddd68740 1 -- 172.21.15.22:0/4178235787 --> [v2:172.21.15.79:6836/1787903366,v1:172.21.15.79:6837/1787903366] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55f8cf089800 con 0x55f8cf131a20 2024-03-28T18:09:36.930 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.928+0000 7fc0d9000640 1 --2- 172.21.15.22:0/4178235787 >> [v2:172.21.15.79:6836/1787903366,v1:172.21.15.79:6837/1787903366] conn(0x55f8cf131a20 0x55f8cf133e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:09:36.930 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.929+0000 7fc0d9000640 1 --2- 172.21.15.22:0/4178235787 >> [v2:172.21.15.79:6836/1787903366,v1:172.21.15.79:6837/1787903366] conn(0x55f8cf131a20 0x55f8cf133e10 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=d702a2a7800227f4 server_cookie=d8ada1a2fbea90dd in_seq=0 out_seq=0 2024-03-28T18:09:36.930 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.929+0000 7fc0be000640 10 client.35978 ms_handle_connect on v2:172.21.15.79:6836/1787903366 2024-03-28T18:09:36.931 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.930+0000 7fc0be000640 1 -- 172.21.15.22:0/4178235787 <== mds.1 v2:172.21.15.79:6836/1787903366 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x55f8cf089800 con 0x55f8cf131a20 2024-03-28T18:09:36.931 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.930+0000 7fc0be000640 10 client.35978 handle_command_reply: tid=0 2024-03-28T18:09:36.931 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.930+0000 7fc0ddd68740 1 client.35978 shutdown 2024-03-28T18:09:36.931 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.930+0000 7fc0ddd68740 1 -- 172.21.15.22:0/4178235787 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55f8cf119080 msgr2=0x55f8cf11e830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:09:36.932 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.930+0000 7fc0ddd68740 1 --2- 172.21.15.22:0/4178235787 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55f8cf119080 0x55f8cf11e830 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fc0c000ce00 tx=0x7fc0c00039c0 comp rx=0 tx=0).stop 2024-03-28T18:09:36.932 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.931+0000 7fc0ddd68740 1 -- 172.21.15.22:0/4178235787 shutdown_connections 2024-03-28T18:09:36.932 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.931+0000 7fc0ddd68740 1 --2- 172.21.15.22:0/4178235787 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f8cf121c00 0x55f8cf11ed70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:09:36.932 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.931+0000 7fc0ddd68740 1 --2- 172.21.15.22:0/4178235787 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55f8cf119080 0x55f8cf11e830 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:09:36.932 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.931+0000 7fc0ddd68740 1 --2- 172.21.15.22:0/4178235787 >> [v2:172.21.15.79:6836/1787903366,v1:172.21.15.79:6837/1787903366] conn(0x55f8cf131a20 0x55f8cf133e10 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:09:36.932 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.931+0000 7fc0ddd68740 1 --2- 172.21.15.22:0/4178235787 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f8cf1186d0 0x55f8cf1214a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:09:36.932 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.931+0000 7fc0be000640 0 client.35978 ms_handle_reset on v2:172.21.15.79:6836/1787903366 2024-03-28T18:09:36.932 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.931+0000 7fc0ddd68740 1 -- 172.21.15.22:0/4178235787 >> 172.21.15.22:0/4178235787 conn(0x55f8cf112da0 msgr2=0x55f8cf114af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:09:36.932 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.931+0000 7fc0ddd68740 1 -- 172.21.15.22:0/4178235787 shutdown_connections 2024-03-28T18:09:36.932 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.931+0000 7fc0ddd68740 1 -- 172.21.15.22:0/4178235787 wait complete. 2024-03-28T18:09:36.932 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.931+0000 7fc0ddd68740 20 client.35978 trim_cache size 0 max 16384 2024-03-28T18:09:36.933 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:09:36 smithi079 ceph-mon[35312]: pgmap v64: 97 pgs: 97 active+clean; 91 MiB data, 694 MiB used, 1.0 TiB / 1.0 TiB avail; 318 KiB/s rd, 800 KiB/s wr, 161 op/s 2024-03-28T18:09:36.933 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:09:36 smithi079 ceph-mon[35312]: from='client.? 172.21.15.22:0/868073401' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-28T18:09:36.933 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/input/stmpe-keypad.txt 2024-03-28T18:09:36.943 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/input/sun4i-lradc-keys.txt 2024-03-28T18:09:36.946 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.946+0000 7fc0ddd68740 1 Processor -- start 2024-03-28T18:09:36.946 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.946+0000 7fc0ddd68740 1 -- start start 2024-03-28T18:09:36.947 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.947+0000 7fc0ddd68740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f8cf112da0 0x55f8cf11d790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:09:36.947 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.947+0000 7fc0ddd68740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f8cf1186d0 0x55f8cf120f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:09:36.947 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.947+0000 7fc0ddd68740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55f8cf119080 0x55f8cf1214a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:09:36.947 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.947+0000 7fc0ddd68740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0d41af270 con 0x55f8cf1186d0 2024-03-28T18:09:36.948 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.947+0000 7fc0ddd68740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0d41b0bf0 con 0x55f8cf119080 2024-03-28T18:09:36.948 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.947+0000 7fc0ddd68740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0d41af8d0 con 0x55f8cf112da0 2024-03-28T18:09:36.948 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.947+0000 7fc0d3400640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f8cf1186d0 0x55f8cf120f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:09:36.948 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.947+0000 7fc0d3400640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f8cf1186d0 0x55f8cf120f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:36162/0 (socket says 172.21.15.22:36162) 2024-03-28T18:09:36.948 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.947+0000 7fc0d3400640 1 -- 172.21.15.22:0/1618592730 learned_addr learned my addr 172.21.15.22:0/1618592730 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-28T18:09:36.948 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.947+0000 7fc0d9000640 1 --2- 172.21.15.22:0/1618592730 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f8cf112da0 0x55f8cf11d790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:09:36.948 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.947+0000 7fc0d9000640 1 --2- 172.21.15.22:0/1618592730 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55f8cf119080 0x55f8cf1214a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:09:36.948 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.948+0000 7fc0d3400640 1 -- 172.21.15.22:0/1618592730 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f8cf112da0 msgr2=0x55f8cf11d790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T18:09:36.948 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.948+0000 7fc0d3400640 1 --2- 172.21.15.22:0/1618592730 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f8cf112da0 0x55f8cf11d790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:09:36.948 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.948+0000 7fc0d3400640 1 -- 172.21.15.22:0/1618592730 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55f8cf119080 msgr2=0x55f8cf1214a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T18:09:36.948 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.948+0000 7fc0d3400640 1 --2- 172.21.15.22:0/1618592730 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55f8cf119080 0x55f8cf1214a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:09:36.948 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.948+0000 7fc0d3400640 1 -- 172.21.15.22:0/1618592730 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f8cf117760 con 0x55f8cf1186d0 2024-03-28T18:09:36.950 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.948+0000 7fc0d3400640 1 --2- 172.21.15.22:0/1618592730 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f8cf1186d0 0x55f8cf120f60 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7fc0c000d010 tx=0x7fc0c000c6b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3d04bd07101a3c8f server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:09:36.950 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.948+0000 7fc0c8c00640 1 -- 172.21.15.22:0/1618592730 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0c0030070 con 0x55f8cf1186d0 2024-03-28T18:09:36.950 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.949+0000 7fc0c8c00640 1 -- 172.21.15.22:0/1618592730 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7fc0c00197c0 con 0x55f8cf1186d0 2024-03-28T18:09:36.950 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.949+0000 7fc0c8c00640 1 -- 172.21.15.22:0/1618592730 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0c0027030 con 0x55f8cf1186d0 2024-03-28T18:09:36.950 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.949+0000 7fc0ddd68740 1 -- 172.21.15.22:0/1618592730 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f8cf1186d0 msgr2=0x55f8cf120f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:09:36.950 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.949+0000 7fc0ddd68740 1 --2- 172.21.15.22:0/1618592730 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f8cf1186d0 0x55f8cf120f60 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7fc0c000d010 tx=0x7fc0c000c6b0 comp rx=0 tx=0).stop 2024-03-28T18:09:36.950 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.949+0000 7fc0ddd68740 1 -- 172.21.15.22:0/1618592730 shutdown_connections 2024-03-28T18:09:36.950 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.949+0000 7fc0ddd68740 1 --2- 172.21.15.22:0/1618592730 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55f8cf119080 0x55f8cf1214a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:09:36.950 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.949+0000 7fc0ddd68740 1 --2- 172.21.15.22:0/1618592730 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f8cf1186d0 0x55f8cf120f60 unknown :-1 s=CLOSED pgs=610 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:09:36.950 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.949+0000 7fc0ddd68740 1 --2- 172.21.15.22:0/1618592730 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f8cf112da0 0x55f8cf11d790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:09:36.950 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.949+0000 7fc0ddd68740 1 -- 172.21.15.22:0/1618592730 >> 172.21.15.22:0/1618592730 conn(0x55f8cf11eed0 msgr2=0x55f8cf114a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:09:36.950 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.950+0000 7fc0ddd68740 1 -- 172.21.15.22:0/1618592730 shutdown_connections 2024-03-28T18:09:36.951 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.950+0000 7fc0ddd68740 1 -- 172.21.15.22:0/1618592730 wait complete. 2024-03-28T18:09:36.951 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.950+0000 7fc0ddd68740 1 -- start start 2024-03-28T18:09:36.951 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.950+0000 7fc0ddd68740 1 Processor -- start 2024-03-28T18:09:36.953 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.951+0000 7fc0ddd68740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55f8cf112da0 0x55f8cf129bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:09:36.953 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.951+0000 7fc0ddd68740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f8cf1186d0 0x55f8cf125f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:09:36.953 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.951+0000 7fc0ddd68740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f8cf119080 0x55f8cf1264b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:09:36.953 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.951+0000 7fc0ddd68740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0d41b0590 con 0x55f8cf1186d0 2024-03-28T18:09:36.953 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.951+0000 7fc0ddd68740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0d41aff30 con 0x55f8cf112da0 2024-03-28T18:09:36.953 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.951+0000 7fc0ddd68740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0d41ad430 con 0x55f8cf119080 2024-03-28T18:09:36.953 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.951+0000 7fc0ddd68740 10 client.0 _ll_register_callbacks cb 0x7fc0c00040a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-28T18:09:36.953 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.951+0000 7fc0d3400640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f8cf1186d0 0x55f8cf125f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:09:36.953 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.951+0000 7fc0d3400640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f8cf1186d0 0x55f8cf125f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:36168/0 (socket says 172.21.15.22:36168) 2024-03-28T18:09:36.953 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.951+0000 7fc0d3400640 1 -- 172.21.15.22:0/3862850198 learned_addr learned my addr 172.21.15.22:0/3862850198 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-28T18:09:36.953 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.951+0000 7fc0d9000640 1 --2- 172.21.15.22:0/3862850198 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55f8cf112da0 0x55f8cf129bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:09:36.954 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.951+0000 7fc0d9000640 1 --2- 172.21.15.22:0/3862850198 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f8cf119080 0x55f8cf1264b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:09:36.954 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.951+0000 7fc0d3400640 1 -- 172.21.15.22:0/3862850198 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f8cf119080 msgr2=0x55f8cf1264b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T18:09:36.954 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.951+0000 7fc0d3400640 1 --2- 172.21.15.22:0/3862850198 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f8cf119080 0x55f8cf1264b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:09:36.954 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.951+0000 7fc0d3400640 1 -- 172.21.15.22:0/3862850198 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55f8cf112da0 msgr2=0x55f8cf129bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T18:09:36.954 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.951+0000 7fc0d3400640 1 --2- 172.21.15.22:0/3862850198 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55f8cf112da0 0x55f8cf129bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:09:36.954 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.951+0000 7fc0d3400640 1 -- 172.21.15.22:0/3862850198 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55f8cf126c60 con 0x55f8cf1186d0 2024-03-28T18:09:36.954 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.952+0000 7fc0d3400640 1 --2- 172.21.15.22:0/3862850198 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f8cf1186d0 0x55f8cf125f70 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7fc0c0019790 tx=0x7fc0c00193e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=de3c397b0ed644a6 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:09:36.954 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.952+0000 7fc0be000640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-03-28T18:09:36.954 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.952+0000 7fc0be000640 1 -- 172.21.15.22:0/3862850198 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0c0030070 con 0x55f8cf1186d0 2024-03-28T18:09:36.954 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.952+0000 7fc0be000640 1 -- 172.21.15.22:0/3862850198 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 8411+0+0 (secure 0 0 0) 0x7fc0c0011770 con 0x55f8cf1186d0 2024-03-28T18:09:36.954 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.952+0000 7fc0ddd68740 1 -- 172.21.15.22:0/3862850198 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fc0c00197c0 con 0x55f8cf1186d0 2024-03-28T18:09:36.954 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.952+0000 7fc0be000640 1 -- 172.21.15.22:0/3862850198 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc0c0027040 con 0x55f8cf1186d0 2024-03-28T18:09:36.954 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.952+0000 7fc0ddd68740 10 client.16155 fetch_fsmap learned FSMap version 27 2024-03-28T18:09:36.954 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.952+0000 7fc0ddd68740 1 -- 172.21.15.22:0/3862850198 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55f8cf18e270 con 0x55f8cf1186d0 2024-03-28T18:09:36.955 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.953+0000 7fc0be000640 1 -- 172.21.15.22:0/3862850198 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7fc0c00197c0 con 0x55f8cf1186d0 2024-03-28T18:09:36.955 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.953+0000 7fc0be000640 1 -- 172.21.15.22:0/3862850198 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 27) v1 ==== 7934+0+0 (secure 0 0 0) 0x7fc0c0018040 con 0x55f8cf1186d0 2024-03-28T18:09:36.955 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.953+0000 7fc0ddd68740 10 client.16155 fetch_fsmap finished waiting for FSMap version 27 2024-03-28T18:09:36.955 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.953+0000 7fc0ddd68740 10 client.16155 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-03-28T18:09:36.955 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.953+0000 7fc0ddd68740 20 client.16155 populate_metadata read hostname 'smithi022' 2024-03-28T18:09:36.955 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.953+0000 7fc0ddd68740 1 --2- 172.21.15.22:0/3862850198 >> [v2:172.21.15.79:6836/1787903366,v1:172.21.15.79:6837/1787903366] conn(0x55f8cf121c00 0x55f8cf190930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:09:36.955 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.953+0000 7fc0ddd68740 4 client.16155 mds_command: new command op to 34511 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-28T18:09:36.955 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.953+0000 7fc0ddd68740 1 -- 172.21.15.22:0/3862850198 --> [v2:172.21.15.79:6836/1787903366,v1:172.21.15.79:6837/1787903366] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55f8cf083b50 con 0x55f8cf121c00 2024-03-28T18:09:36.955 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.953+0000 7fc0d9000640 1 --2- 172.21.15.22:0/3862850198 >> [v2:172.21.15.79:6836/1787903366,v1:172.21.15.79:6837/1787903366] conn(0x55f8cf121c00 0x55f8cf190930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:09:36.955 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.954+0000 7fc0d9000640 1 --2- 172.21.15.22:0/3862850198 >> [v2:172.21.15.79:6836/1787903366,v1:172.21.15.79:6837/1787903366] conn(0x55f8cf121c00 0x55f8cf190930 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=d5f05469d2ae1094 server_cookie=469c0d7ac402e3ef in_seq=0 out_seq=0 2024-03-28T18:09:36.955 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.954+0000 7fc0be000640 10 client.16155 ms_handle_connect on v2:172.21.15.79:6836/1787903366 2024-03-28T18:09:36.956 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/input/tca8418_keypad.txt 2024-03-28T18:09:36.957 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.954+0000 7fc0be000640 1 -- 172.21.15.22:0/3862850198 <== mds.1 v2:172.21.15.79:6836/1787903366 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55f8cf083b50 con 0x55f8cf121c00 2024-03-28T18:09:36.957 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.954+0000 7fc0be000640 10 client.16155 handle_command_reply: tid=0 2024-03-28T18:09:36.957 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.954+0000 7fc0ddd68740 1 client.16155 shutdown 2024-03-28T18:09:36.957 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.954+0000 7fc0ddd68740 1 -- 172.21.15.22:0/3862850198 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f8cf1186d0 msgr2=0x55f8cf125f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:09:36.957 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.954+0000 7fc0ddd68740 1 --2- 172.21.15.22:0/3862850198 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f8cf1186d0 0x55f8cf125f70 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7fc0c0019790 tx=0x7fc0c00193e0 comp rx=0 tx=0).stop 2024-03-28T18:09:36.957 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.955+0000 7fc0ddd68740 1 -- 172.21.15.22:0/3862850198 shutdown_connections 2024-03-28T18:09:36.957 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.955+0000 7fc0ddd68740 1 --2- 172.21.15.22:0/3862850198 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f8cf119080 0x55f8cf1264b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:09:36.957 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.955+0000 7fc0ddd68740 1 --2- 172.21.15.22:0/3862850198 >> [v2:172.21.15.79:6836/1787903366,v1:172.21.15.79:6837/1787903366] conn(0x55f8cf121c00 0x55f8cf190930 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:09:36.957 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.955+0000 7fc0ddd68740 1 --2- 172.21.15.22:0/3862850198 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f8cf1186d0 0x55f8cf125f70 unknown :-1 s=CLOSED pgs=611 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:09:36.957 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.955+0000 7fc0ddd68740 1 --2- 172.21.15.22:0/3862850198 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55f8cf112da0 0x55f8cf129bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:09:36.957 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.955+0000 7fc0ddd68740 1 -- 172.21.15.22:0/3862850198 >> 172.21.15.22:0/3862850198 conn(0x55f8cf11eed0 msgr2=0x55f8cf111620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:09:36.957 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.955+0000 7fc0be000640 0 client.16155 ms_handle_reset on v2:172.21.15.79:6836/1787903366 2024-03-28T18:09:36.958 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.955+0000 7fc0ddd68740 1 -- 172.21.15.22:0/3862850198 shutdown_connections 2024-03-28T18:09:36.958 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.955+0000 7fc0ddd68740 1 -- 172.21.15.22:0/3862850198 wait complete. 2024-03-28T18:09:36.958 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.955+0000 7fc0ddd68740 20 client.16155 trim_cache size 0 max 16384 2024-03-28T18:09:36.958 INFO:teuthology.orchestra.run.smithi022.stdout:[] 2024-03-28T18:09:36.959 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.956+0000 7fc0db000640 1 -- 172.21.15.22:0/3644885839 >> [v2:172.21.15.22:6800/1357365082,v1:172.21.15.22:6801/1357365082] conn(0x7fc0b80a29f0 msgr2=0x7fc0b80a4eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:09:36.959 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.956+0000 7fc0db000640 1 --2- 172.21.15.22:0/3644885839 >> [v2:172.21.15.22:6800/1357365082,v1:172.21.15.22:6801/1357365082] conn(0x7fc0b80a29f0 0x7fc0b80a4eb0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x55f8cf1181a0 tx=0x7fc0c400cf20 comp rx=0 tx=0).stop 2024-03-28T18:09:36.959 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.956+0000 7fc0db000640 1 -- 172.21.15.22:0/3644885839 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc0d4103840 msgr2=0x7fc0d419a0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:09:36.959 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.956+0000 7fc0db000640 1 --2- 172.21.15.22:0/3644885839 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc0d4103840 0x7fc0d419a0b0 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7fc0c400e9f0 tx=0x7fc0c400eeb0 comp rx=0 tx=0).stop 2024-03-28T18:09:36.959 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.956+0000 7fc0db000640 1 -- 172.21.15.22:0/3644885839 shutdown_connections 2024-03-28T18:09:36.959 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.956+0000 7fc0db000640 1 --2- 172.21.15.22:0/3644885839 >> [v2:172.21.15.22:6800/1357365082,v1:172.21.15.22:6801/1357365082] conn(0x7fc0b80a29f0 0x7fc0b80a4eb0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:09:36.959 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.956+0000 7fc0db000640 1 --2- 172.21.15.22:0/3644885839 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc0d4109850 0x7fc0d419e970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:09:36.959 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.956+0000 7fc0db000640 1 --2- 172.21.15.22:0/3644885839 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc0d4104200 0x7fc0d419a5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:09:36.959 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.956+0000 7fc0db000640 1 --2- 172.21.15.22:0/3644885839 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc0d4103840 0x7fc0d419a0b0 unknown :-1 s=CLOSED pgs=608 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:09:36.959 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.956+0000 7fc0db000640 1 -- 172.21.15.22:0/3644885839 >> 172.21.15.22:0/3644885839 conn(0x7fc0d40ff2f0 msgr2=0x7fc0d40ff910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:09:36.959 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.956+0000 7fc0db000640 1 -- 172.21.15.22:0/3644885839 shutdown_connections 2024-03-28T18:09:36.959 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:36.956+0000 7fc0db000640 1 -- 172.21.15.22:0/3644885839 wait complete. 2024-03-28T18:09:36.962 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/input/ti,drv260x.txt 2024-03-28T18:09:36.973 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-28T18:09:36.973 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-03-28T18:09:36.974 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/input/ti,drv2667.txt 2024-03-28T18:09:36.983 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/input/ti,nspire-keypad.txt 2024-03-28T18:09:36.997 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/input/ti,palmas-pwrbutton.txt 2024-03-28T18:09:37.010 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/input/touchscreen/ 2024-03-28T18:09:37.020 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/input/touchscreen/auo_pixcir_ts.txt 2024-03-28T18:09:37.024 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/input/touchscreen/bu21013.txt 2024-03-28T18:09:37.038 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/input/touchscreen/edt-ft5x06.txt 2024-03-28T18:09:37.047 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/input/touchscreen/egalax-ts.txt 2024-03-28T18:09:37.061 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/input/touchscreen/lpc32xx-tsc.txt 2024-03-28T18:09:37.076 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/input/touchscreen/mms114.txt 2024-03-28T18:09:37.090 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/input/touchscreen/pixcir_i2c_ts.txt 2024-03-28T18:09:37.102 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/input/touchscreen/sitronix-st1232.txt 2024-03-28T18:09:37.114 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/input/touchscreen/stmpe.txt 2024-03-28T18:09:37.128 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/input/touchscreen/sun4i.txt 2024-03-28T18:09:37.146 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/input/touchscreen/ti-tsc-adc.txt 2024-03-28T18:09:37.159 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/input/touchscreen/touchscreen.txt 2024-03-28T18:09:37.174 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/input/touchscreen/tsc2005.txt 2024-03-28T18:09:37.190 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/input/touchscreen/tsc2007.txt 2024-03-28T18:09:37.206 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/input/touchscreen/zforce_ts.txt 2024-03-28T18:09:37.218 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/input/tps65218-pwrbutton.txt 2024-03-28T18:09:37.222 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:37.222+0000 7f8ea2a00640 1 -- 172.21.15.22:0/3810511213 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8e9c068880 msgr2=0x7f8e9c068c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:09:37.222 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:37.222+0000 7f8ea2a00640 1 --2- 172.21.15.22:0/3810511213 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8e9c068880 0x7f8e9c068c60 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7f8e8c009900 tx=0x7f8e8c02bc30 comp rx=0 tx=0).stop 2024-03-28T18:09:37.222 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:37.222+0000 7f8ea2a00640 1 -- 172.21.15.22:0/3810511213 shutdown_connections 2024-03-28T18:09:37.222 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:37.222+0000 7f8ea2a00640 1 --2- 172.21.15.22:0/3810511213 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8e9c10e040 0x7f8e9c110430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:09:37.222 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:37.222+0000 7f8ea2a00640 1 --2- 172.21.15.22:0/3810511213 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f8e9c0691a0 0x7f8e9c10db00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:09:37.222 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:37.222+0000 7f8ea2a00640 1 --2- 172.21.15.22:0/3810511213 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8e9c068880 0x7f8e9c068c60 unknown :-1 s=CLOSED pgs=612 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:09:37.222 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:37.222+0000 7f8ea2a00640 1 -- 172.21.15.22:0/3810511213 >> 172.21.15.22:0/3810511213 conn(0x7f8e9c0776a0 msgr2=0x7f8e9c077ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:09:37.222 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:37.222+0000 7f8ea2a00640 1 -- 172.21.15.22:0/3810511213 shutdown_connections 2024-03-28T18:09:37.222 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:37.222+0000 7f8ea2a00640 1 -- 172.21.15.22:0/3810511213 wait complete. 2024-03-28T18:09:37.225 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:37.223+0000 7f8ea2a00640 1 Processor -- start 2024-03-28T18:09:37.225 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:37.223+0000 7f8ea2a00640 1 -- start start 2024-03-28T18:09:37.225 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:37.223+0000 7f8ea2a00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8e9c068880 0x7f8e9c19a130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:09:37.225 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:37.223+0000 7f8ea2a00640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f8e9c0691a0 0x7f8e9c19a670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:09:37.225 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:37.224+0000 7f8ea2a00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8e9c10e040 0x7f8e9c19e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:09:37.225 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:37.224+0000 7f8ea2a00640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e9c113bd0 con 0x7f8e9c068880 2024-03-28T18:09:37.225 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:37.224+0000 7f8ea2a00640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e9c113a50 con 0x7f8e9c0691a0 2024-03-28T18:09:37.226 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:37.224+0000 7f8ea2a00640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e9c113d50 con 0x7f8e9c10e040 2024-03-28T18:09:37.226 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:37.224+0000 7f8e9be00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8e9c068880 0x7f8e9c19a130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:09:37.226 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:37.224+0000 7f8e9be00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8e9c068880 0x7f8e9c19a130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:46536/0 (socket says 172.21.15.22:46536) 2024-03-28T18:09:37.226 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:37.224+0000 7f8e9be00640 1 -- 172.21.15.22:0/1105939788 learned_addr learned my addr 172.21.15.22:0/1105939788 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-28T18:09:37.226 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:37.224+0000 7f8e93e00640 1 --2- 172.21.15.22:0/1105939788 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f8e9c0691a0 0x7f8e9c19a670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:09:37.226 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:37.224+0000 7f8ea0a00640 1 --2- 172.21.15.22:0/1105939788 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8e9c10e040 0x7f8e9c19e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:09:37.226 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:37.224+0000 7f8e9be00640 1 -- 172.21.15.22:0/1105939788 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8e9c10e040 msgr2=0x7f8e9c19e9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:09:37.226 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:37.224+0000 7f8e9be00640 1 --2- 172.21.15.22:0/1105939788 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8e9c10e040 0x7f8e9c19e9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:09:37.226 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:37.224+0000 7f8e9be00640 1 -- 172.21.15.22:0/1105939788 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f8e9c0691a0 msgr2=0x7f8e9c19a670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:09:37.226 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:37.224+0000 7f8e9be00640 1 --2- 172.21.15.22:0/1105939788 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f8e9c0691a0 0x7f8e9c19a670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:09:37.226 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:37.224+0000 7f8e9be00640 1 -- 172.21.15.22:0/1105939788 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8e9c19f120 con 0x7f8e9c068880 2024-03-28T18:09:37.226 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:37.224+0000 7f8e93e00640 1 --2- 172.21.15.22:0/1105939788 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f8e9c0691a0 0x7f8e9c19a670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-28T18:09:37.226 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:37.225+0000 7f8e9be00640 1 --2- 172.21.15.22:0/1105939788 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8e9c068880 0x7f8e9c19a130 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7f8e8c005940 tx=0x7f8e8c004bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:09:37.226 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:37.225+0000 7f8e99600640 1 -- 172.21.15.22:0/1105939788 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e8c043020 con 0x7f8e9c068880 2024-03-28T18:09:37.227 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:37.225+0000 7f8e99600640 1 -- 172.21.15.22:0/1105939788 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f8e8c02c5d0 con 0x7f8e9c068880 2024-03-28T18:09:37.227 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:37.225+0000 7f8ea2a00640 1 -- 172.21.15.22:0/1105939788 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8e9c19f3b0 con 0x7f8e9c068880 2024-03-28T18:09:37.227 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:37.225+0000 7f8e99600640 1 -- 172.21.15.22:0/1105939788 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e8c003a80 con 0x7f8e9c068880 2024-03-28T18:09:37.227 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:37.225+0000 7f8ea2a00640 1 -- 172.21.15.22:0/1105939788 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8e9c1a6ca0 con 0x7f8e9c068880 2024-03-28T18:09:37.228 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:37.229+0000 7f8ea5322740 1 Processor -- start 2024-03-28T18:09:37.228 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:37.229+0000 7f8e99600640 1 -- 172.21.15.22:0/1105939788 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 26) v1 ==== 137421+0+0 (secure 0 0 0) 0x7f8e8c046020 con 0x7f8e9c068880 2024-03-28T18:09:37.229 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:37.229+0000 7f8ea5322740 1 -- start start 2024-03-28T18:09:37.230 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:37.229+0000 7f8ea5322740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560e32dc16d0 0x560e32dc1ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:09:37.230 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:37.229+0000 7f8ea5322740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560e32dc2080 0x560e32dca630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:09:37.230 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:37.229+0000 7f8ea5322740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x560e32dcac00 0x560e32dcd050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:09:37.230 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:37.229+0000 7f8ea5322740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x560e32d252c0 con 0x560e32dc2080 2024-03-28T18:09:37.231 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:37.229+0000 7f8ea5322740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x560e32ccdc90 con 0x560e32dcac00 2024-03-28T18:09:37.231 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:37.229+0000 7f8ea5322740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x560e32d21360 con 0x560e32dc16d0 2024-03-28T18:09:37.231 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:37.230+0000 7f8e9be00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560e32dc2080 0x560e32dca630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:09:37.231 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:37.230+0000 7f8e93e00640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x560e32dcac00 0x560e32dcd050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:09:37.231 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:37.230+0000 7f8e93e00640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x560e32dcac00 0x560e32dcd050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.22:41026/0 (socket says 172.21.15.22:41026) 2024-03-28T18:09:37.231 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:37.230+0000 7f8e9be00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560e32dc2080 0x560e32dca630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:46546/0 (socket says 172.21.15.22:46546) 2024-03-28T18:09:37.231 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:37.230+0000 7f8e93e00640 1 -- 172.21.15.22:0/3349622727 learned_addr learned my addr 172.21.15.22:0/3349622727 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-28T18:09:37.231 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:37.230+0000 7f8ea0a00640 1 --2- 172.21.15.22:0/3349622727 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560e32dc16d0 0x560e32dc1ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:09:37.231 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:37.230+0000 7f8e93e00640 1 -- 172.21.15.22:0/3349622727 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560e32dc16d0 msgr2=0x560e32dc1ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T18:09:37.231 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:37.230+0000 7f8e93e00640 1 --2- 172.21.15.22:0/3349622727 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560e32dc16d0 0x560e32dc1ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:09:37.231 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:37.230+0000 7f8e93e00640 1 -- 172.21.15.22:0/3349622727 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560e32dc2080 msgr2=0x560e32dca630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T18:09:37.231 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:37.230+0000 7f8e93e00640 1 --2- 172.21.15.22:0/3349622727 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560e32dc2080 0x560e32dca630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:09:37.231 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:37.230+0000 7f8e93e00640 1 -- 172.21.15.22:0/3349622727 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560e32cf2080 con 0x560e32dcac00 2024-03-28T18:09:37.232 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:37.231+0000 7f8e93e00640 1 --2- 172.21.15.22:0/3349622727 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x560e32dcac00 0x560e32dcd050 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f8e840114f0 tx=0x7f8e840119b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=aba6bf27f209841 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:09:37.232 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:37.231+0000 7f8e99600640 1 --2- 172.21.15.22:0/1105939788 >> [v2:172.21.15.22:6800/1357365082,v1:172.21.15.22:6801/1357365082] conn(0x7f8e800a2970 0x7f8e800a4e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:09:37.233 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:37.231+0000 7f8e91200640 1 -- 172.21.15.22:0/3349622727 <== mon.2 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e8401e070 con 0x560e32dcac00 2024-03-28T18:09:37.233 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:37.231+0000 7f8e99600640 1 -- 172.21.15.22:0/1105939788 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 8411+0+0 (secure 0 0 0) 0x7f8e8c0f08a0 con 0x7f8e9c068880 2024-03-28T18:09:37.233 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:37.231+0000 7f8e91200640 1 -- 172.21.15.22:0/3349622727 <== mon.2 v2:172.21.15.79:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f8e840035f0 con 0x560e32dcac00 2024-03-28T18:09:37.233 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:37.231+0000 7f8e91200640 1 -- 172.21.15.22:0/3349622727 <== mon.2 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e8400f820 con 0x560e32dcac00 2024-03-28T18:09:37.233 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:37.231+0000 7f8ea0a00640 1 --2- 172.21.15.22:0/1105939788 >> [v2:172.21.15.22:6800/1357365082,v1:172.21.15.22:6801/1357365082] conn(0x7f8e800a2970 0x7f8e800a4e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:09:37.233 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:37.232+0000 7f8ea5322740 1 -- 172.21.15.22:0/3349622727 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x560e32dcac00 msgr2=0x560e32dcd050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:09:37.233 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:37.232+0000 7f8ea0a00640 1 --2- 172.21.15.22:0/1105939788 >> [v2:172.21.15.22:6800/1357365082,v1:172.21.15.22:6801/1357365082] conn(0x7f8e800a2970 0x7f8e800a4e30 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f8e9c06abd0 tx=0x7f8e94009890 comp rx=0 tx=0).ready entity=mgr.15522 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:09:37.233 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:37.232+0000 7f8ea5322740 1 --2- 172.21.15.22:0/3349622727 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x560e32dcac00 0x560e32dcd050 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f8e840114f0 tx=0x7f8e840119b0 comp rx=0 tx=0).stop 2024-03-28T18:09:37.233 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:37.232+0000 7f8ea5322740 1 -- 172.21.15.22:0/3349622727 shutdown_connections 2024-03-28T18:09:37.234 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:37.232+0000 7f8ea5322740 1 --2- 172.21.15.22:0/3349622727 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x560e32dcac00 0x560e32dcd050 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:09:37.234 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:37.232+0000 7f8ea5322740 1 --2- 172.21.15.22:0/3349622727 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560e32dc2080 0x560e32dca630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:09:37.234 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:37.232+0000 7f8ea5322740 1 --2- 172.21.15.22:0/3349622727 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560e32dc16d0 0x560e32dc1ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:09:37.234 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:37.232+0000 7f8ea5322740 1 -- 172.21.15.22:0/3349622727 >> 172.21.15.22:0/3349622727 conn(0x560e32dbbda0 msgr2=0x560e32dbc180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:09:37.234 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:37.232+0000 7f8ea5322740 1 -- 172.21.15.22:0/3349622727 shutdown_connections 2024-03-28T18:09:37.234 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:37.232+0000 7f8ea5322740 1 -- 172.21.15.22:0/3349622727 wait complete. 2024-03-28T18:09:37.234 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/input/twl4030-keypad.txt 2024-03-28T18:09:37.237 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:37.233+0000 7f8ea5322740 1 -- start start 2024-03-28T18:09:37.237 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:37.233+0000 7f8ea5322740 1 Processor -- start 2024-03-28T18:09:37.237 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:37.234+0000 7f8ea5322740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560e32dc16d0 0x560e32dca560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:09:37.237 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:37.234+0000 7f8ea5322740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560e32dc2080 0x560e32dc78f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:09:37.238 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:37.234+0000 7f8ea5322740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x560e32dcac00 0x560e32dc7f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:09:37.238 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:37.234+0000 7f8ea5322740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x560e32dcdb70 con 0x560e32dc2080 2024-03-28T18:09:37.238 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:37.234+0000 7f8ea5322740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x560e32ce10d0 con 0x560e32dcac00 2024-03-28T18:09:37.238 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:37.234+0000 7f8ea5322740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x560e32dcdcf0 con 0x560e32dc16d0 2024-03-28T18:09:37.238 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:37.234+0000 7f8ea5322740 10 client.0 _ll_register_callbacks cb 0x560e32ce5e80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-28T18:09:37.238 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:37.234+0000 7f8ea0a00640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x560e32dcac00 0x560e32dc7f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:09:37.238 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:37.234+0000 7f8ea0a00640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x560e32dcac00 0x560e32dc7f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.22:41034/0 (socket says 172.21.15.22:41034) 2024-03-28T18:09:37.238 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:37.234+0000 7f8ea0a00640 1 -- 172.21.15.22:0/3387448381 learned_addr learned my addr 172.21.15.22:0/3387448381 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-28T18:09:37.238 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:37.234+0000 7f8e93e00640 1 --2- 172.21.15.22:0/3387448381 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560e32dc2080 0x560e32dc78f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:09:37.238 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:37.234+0000 7f8e9be00640 1 --2- 172.21.15.22:0/3387448381 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560e32dc16d0 0x560e32dca560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:09:37.238 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:37.235+0000 7f8ea0a00640 1 -- 172.21.15.22:0/3387448381 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560e32dc16d0 msgr2=0x560e32dca560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T18:09:37.238 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:37.235+0000 7f8ea0a00640 1 --2- 172.21.15.22:0/3387448381 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560e32dc16d0 0x560e32dca560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:09:37.239 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:37.235+0000 7f8ea0a00640 1 -- 172.21.15.22:0/3387448381 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560e32dc2080 msgr2=0x560e32dc78f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T18:09:37.239 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:37.235+0000 7f8ea0a00640 1 --2- 172.21.15.22:0/3387448381 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560e32dc2080 0x560e32dc78f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:09:37.239 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:37.235+0000 7f8ea0a00640 1 -- 172.21.15.22:0/3387448381 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x560e32a393f0 con 0x560e32dcac00 2024-03-28T18:09:37.239 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:37.235+0000 7f8ea0a00640 1 --2- 172.21.15.22:0/3387448381 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x560e32dcac00 0x560e32dc7f00 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f8e94012c10 tx=0x7f8e94004a40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=985c065321d8f677 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:09:37.239 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:37.235+0000 7f8e89600640 10 client.0 ms_handle_connect on v2:172.21.15.79:3300/0 2024-03-28T18:09:37.239 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:37.235+0000 7f8e89600640 1 -- 172.21.15.22:0/3387448381 <== mon.2 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e94012e40 con 0x560e32dcac00 2024-03-28T18:09:37.239 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:37.236+0000 7f8e89600640 1 -- 172.21.15.22:0/3387448381 <== mon.2 v2:172.21.15.79:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 8411+0+0 (secure 0 0 0) 0x7f8e94004b20 con 0x560e32dcac00 2024-03-28T18:09:37.239 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:37.236+0000 7f8ea5322740 1 -- 172.21.15.22:0/3387448381 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f8e840035f0 con 0x560e32dcac00 2024-03-28T18:09:37.239 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:37.236+0000 7f8e89600640 1 -- 172.21.15.22:0/3387448381 <== mon.2 v2:172.21.15.79:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8e94004d20 con 0x560e32dcac00 2024-03-28T18:09:37.239 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:37.236+0000 7f8ea5322740 10 client.35996 fetch_fsmap learned FSMap version 27 2024-03-28T18:09:37.239 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:37.236+0000 7f8ea5322740 1 -- 172.21.15.22:0/3387448381 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x560e32b037d0 con 0x560e32dcac00 2024-03-28T18:09:37.239 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:37.236+0000 7f8e89600640 1 -- 172.21.15.22:0/3387448381 <== mon.2 v2:172.21.15.79:3300/0 4 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f8e840035f0 con 0x560e32dcac00 2024-03-28T18:09:37.239 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:37.237+0000 7f8e89600640 1 -- 172.21.15.22:0/3387448381 <== mon.2 v2:172.21.15.79:3300/0 5 ==== fsmap(e 27) v1 ==== 7934+0+0 (secure 0 0 0) 0x7f8e9402f050 con 0x560e32dcac00 2024-03-28T18:09:37.239 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:37.237+0000 7f8ea5322740 10 client.35996 fetch_fsmap finished waiting for FSMap version 27 2024-03-28T18:09:37.239 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:37.237+0000 7f8ea5322740 10 client.35996 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-28T18:09:37.240 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:37.237+0000 7f8ea5322740 20 client.35996 populate_metadata read hostname 'smithi022' 2024-03-28T18:09:37.240 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:37.237+0000 7f8ea5322740 1 --2- 172.21.15.22:0/3387448381 >> [v2:172.21.15.79:6838/2441542443,v1:172.21.15.79:6839/2441542443] conn(0x560e32ddaa70 0x560e32ddce60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:09:37.240 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:37.237+0000 7f8ea5322740 4 client.35996 mds_command: new command op to 34553 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-28T18:09:37.240 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:37.237+0000 7f8ea5322740 1 -- 172.21.15.22:0/3387448381 --> [v2:172.21.15.79:6838/2441542443,v1:172.21.15.79:6839/2441542443] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x560e32d32800 con 0x560e32ddaa70 2024-03-28T18:09:37.240 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:37.237+0000 7f8e9be00640 1 --2- 172.21.15.22:0/3387448381 >> [v2:172.21.15.79:6838/2441542443,v1:172.21.15.79:6839/2441542443] conn(0x560e32ddaa70 0x560e32ddce60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:09:37.240 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:37.238+0000 7f8e9be00640 1 --2- 172.21.15.22:0/3387448381 >> [v2:172.21.15.79:6838/2441542443,v1:172.21.15.79:6839/2441542443] conn(0x560e32ddaa70 0x560e32ddce60 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4b99a6927aa871fe server_cookie=a401ff446e364b13 in_seq=0 out_seq=0 2024-03-28T18:09:37.240 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:37.238+0000 7f8e89600640 10 client.35996 ms_handle_connect on v2:172.21.15.79:6838/2441542443 2024-03-28T18:09:37.241 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:37.239+0000 7f8e89600640 1 -- 172.21.15.22:0/3387448381 <== mds.0 v2:172.21.15.79:6838/2441542443 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x560e32d32800 con 0x560e32ddaa70 2024-03-28T18:09:37.241 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:37.239+0000 7f8e89600640 10 client.35996 handle_command_reply: tid=0 2024-03-28T18:09:37.241 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:37.239+0000 7f8ea5322740 1 client.35996 shutdown 2024-03-28T18:09:37.241 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:37.239+0000 7f8ea5322740 1 -- 172.21.15.22:0/3387448381 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x560e32dcac00 msgr2=0x560e32dc7f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:09:37.241 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:37.239+0000 7f8ea5322740 1 --2- 172.21.15.22:0/3387448381 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x560e32dcac00 0x560e32dc7f00 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f8e94012c10 tx=0x7f8e94004a40 comp rx=0 tx=0).stop 2024-03-28T18:09:37.241 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:37.239+0000 7f8ea5322740 1 -- 172.21.15.22:0/3387448381 shutdown_connections 2024-03-28T18:09:37.242 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:37.239+0000 7f8ea5322740 1 --2- 172.21.15.22:0/3387448381 >> [v2:172.21.15.79:6838/2441542443,v1:172.21.15.79:6839/2441542443] conn(0x560e32ddaa70 0x560e32ddce60 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:09:37.242 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:37.239+0000 7f8ea5322740 1 --2- 172.21.15.22:0/3387448381 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x560e32dcac00 0x560e32dc7f00 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:09:37.242 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:37.239+0000 7f8ea5322740 1 --2- 172.21.15.22:0/3387448381 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560e32dc2080 0x560e32dc78f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:09:37.242 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:37.239+0000 7f8e89600640 0 client.35996 ms_handle_reset on v2:172.21.15.79:6838/2441542443 2024-03-28T18:09:37.242 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:37.239+0000 7f8ea5322740 1 --2- 172.21.15.22:0/3387448381 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560e32dc16d0 0x560e32dca560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:09:37.242 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:37.239+0000 7f8ea5322740 1 -- 172.21.15.22:0/3387448381 >> 172.21.15.22:0/3387448381 conn(0x560e32dbbda0 msgr2=0x560e32dbdaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:09:37.242 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:37.239+0000 7f8ea5322740 1 -- 172.21.15.22:0/3387448381 shutdown_connections 2024-03-28T18:09:37.242 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:37.239+0000 7f8ea5322740 1 -- 172.21.15.22:0/3387448381 wait complete. 2024-03-28T18:09:37.242 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:37.239+0000 7f8ea5322740 20 client.35996 trim_cache size 0 max 16384 2024-03-28T18:09:37.243 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/input/twl4030-pwrbutton.txt 2024-03-28T18:09:37.253 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:37.253+0000 7f8ea5322740 1 Processor -- start 2024-03-28T18:09:37.254 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:37.254+0000 7f8ea5322740 1 -- start start 2024-03-28T18:09:37.254 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:37.254+0000 7f8ea5322740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x560e32dc16d0 0x560e32dca020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:09:37.254 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:37.254+0000 7f8ea5322740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560e32dc2080 0x560e32dca560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:09:37.254 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:37.254+0000 7f8ea5322740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560e32dcac00 0x560e32dc4b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:09:37.254 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:37.254+0000 7f8ea5322740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e9c1af190 con 0x560e32dc2080 2024-03-28T18:09:37.254 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:37.254+0000 7f8ea5322740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e9c1b0b10 con 0x560e32dc16d0 2024-03-28T18:09:37.254 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:37.254+0000 7f8ea5322740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e9c1af7f0 con 0x560e32dcac00 2024-03-28T18:09:37.256 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:37.254+0000 7f8e93e00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560e32dc2080 0x560e32dca560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:09:37.256 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:37.254+0000 7f8e93e00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560e32dc2080 0x560e32dca560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:46562/0 (socket says 172.21.15.22:46562) 2024-03-28T18:09:37.256 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:37.254+0000 7f8e93e00640 1 -- 172.21.15.22:0/3067655510 learned_addr learned my addr 172.21.15.22:0/3067655510 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-28T18:09:37.257 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:37.254+0000 7f8e9be00640 1 --2- 172.21.15.22:0/3067655510 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x560e32dc16d0 0x560e32dca020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:09:37.257 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:37.255+0000 7f8ea0a00640 1 --2- 172.21.15.22:0/3067655510 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560e32dcac00 0x560e32dc4b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:09:37.257 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:37.255+0000 7f8e93e00640 1 -- 172.21.15.22:0/3067655510 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560e32dcac00 msgr2=0x560e32dc4b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T18:09:37.257 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:37.255+0000 7f8e93e00640 1 --2- 172.21.15.22:0/3067655510 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560e32dcac00 0x560e32dc4b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:09:37.257 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:37.255+0000 7f8e93e00640 1 -- 172.21.15.22:0/3067655510 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x560e32dc16d0 msgr2=0x560e32dca020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T18:09:37.257 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:37.255+0000 7f8e93e00640 1 --2- 172.21.15.22:0/3067655510 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x560e32dc16d0 0x560e32dca020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:09:37.257 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:37.255+0000 7f8e93e00640 1 -- 172.21.15.22:0/3067655510 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560e32dbc6e0 con 0x560e32dc2080 2024-03-28T18:09:37.257 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:37.255+0000 7f8ea0a00640 1 --2- 172.21.15.22:0/3067655510 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560e32dcac00 0x560e32dc4b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T18:09:37.257 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:37.255+0000 7f8e9be00640 1 --2- 172.21.15.22:0/3067655510 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x560e32dc16d0 0x560e32dca020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-28T18:09:37.257 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:37.255+0000 7f8e93e00640 1 --2- 172.21.15.22:0/3067655510 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560e32dc2080 0x560e32dca560 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7f8e840058c0 tx=0x7f8e84009ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e98e7ff7eeb0a765 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:09:37.257 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:37.255+0000 7f8e8be00640 1 -- 172.21.15.22:0/3067655510 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e8401e040 con 0x560e32dc2080 2024-03-28T18:09:37.257 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:37.255+0000 7f8e8be00640 1 -- 172.21.15.22:0/3067655510 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f8e84028070 con 0x560e32dc2080 2024-03-28T18:09:37.258 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:37.255+0000 7f8e8be00640 1 -- 172.21.15.22:0/3067655510 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e8400f5c0 con 0x560e32dc2080 2024-03-28T18:09:37.258 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:37.256+0000 7f8ea5322740 1 -- 172.21.15.22:0/3067655510 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560e32dc2080 msgr2=0x560e32dca560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:09:37.258 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:37.256+0000 7f8ea5322740 1 --2- 172.21.15.22:0/3067655510 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560e32dc2080 0x560e32dca560 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7f8e840058c0 tx=0x7f8e84009ce0 comp rx=0 tx=0).stop 2024-03-28T18:09:37.258 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:37.256+0000 7f8ea5322740 1 -- 172.21.15.22:0/3067655510 shutdown_connections 2024-03-28T18:09:37.258 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:37.256+0000 7f8ea5322740 1 --2- 172.21.15.22:0/3067655510 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560e32dcac00 0x560e32dc4b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:09:37.258 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:37.256+0000 7f8ea5322740 1 --2- 172.21.15.22:0/3067655510 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560e32dc2080 0x560e32dca560 unknown :-1 s=CLOSED pgs=614 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:09:37.258 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:37.256+0000 7f8ea5322740 1 --2- 172.21.15.22:0/3067655510 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x560e32dc16d0 0x560e32dca020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:09:37.258 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:37.256+0000 7f8ea5322740 1 -- 172.21.15.22:0/3067655510 >> 172.21.15.22:0/3067655510 conn(0x560e32dbbda0 msgr2=0x560e32dbaeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:09:37.258 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:37.256+0000 7f8ea5322740 1 -- 172.21.15.22:0/3067655510 shutdown_connections 2024-03-28T18:09:37.258 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:37.256+0000 7f8ea5322740 1 -- 172.21.15.22:0/3067655510 wait complete. 2024-03-28T18:09:37.258 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:37.257+0000 7f8ea5322740 1 -- start start 2024-03-28T18:09:37.258 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:37.257+0000 7f8ea5322740 1 Processor -- start 2024-03-28T18:09:37.261 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:37.257+0000 7f8ea5322740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560e32dc16d0 0x560e32dd28d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:09:37.261 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:37.257+0000 7f8ea5322740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560e32dc2080 0x560e32dd2e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:09:37.261 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:37.257+0000 7f8ea5322740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x560e32dcac00 0x560e32dd0ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:09:37.261 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:37.257+0000 7f8ea5322740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e9c1b04b0 con 0x560e32dc16d0 2024-03-28T18:09:37.261 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:37.257+0000 7f8ea5322740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e9c1afe50 con 0x560e32dcac00 2024-03-28T18:09:37.261 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:37.257+0000 7f8ea5322740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e9c1ad350 con 0x560e32dc2080 2024-03-28T18:09:37.261 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:37.257+0000 7f8ea5322740 10 client.0 _ll_register_callbacks cb 0x7f8e94009f10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-28T18:09:37.261 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:37.258+0000 7f8e9be00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560e32dc16d0 0x560e32dd28d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:09:37.262 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:37.258+0000 7f8e9be00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560e32dc16d0 0x560e32dd28d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:46570/0 (socket says 172.21.15.22:46570) 2024-03-28T18:09:37.262 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:37.258+0000 7f8e9be00640 1 -- 172.21.15.22:0/1991219465 learned_addr learned my addr 172.21.15.22:0/1991219465 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-28T18:09:37.262 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:37.258+0000 7f8ea0a00640 1 --2- 172.21.15.22:0/1991219465 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x560e32dcac00 0x560e32dd0ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:09:37.262 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:37.258+0000 7f8e93e00640 1 --2- 172.21.15.22:0/1991219465 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560e32dc2080 0x560e32dd2e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:09:37.262 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:37.258+0000 7f8e9be00640 1 -- 172.21.15.22:0/1991219465 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560e32dc2080 msgr2=0x560e32dd2e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T18:09:37.262 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:37.258+0000 7f8e9be00640 1 --2- 172.21.15.22:0/1991219465 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560e32dc2080 0x560e32dd2e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:09:37.262 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:37.258+0000 7f8e9be00640 1 -- 172.21.15.22:0/1991219465 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x560e32dcac00 msgr2=0x560e32dd0ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T18:09:37.262 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:37.258+0000 7f8e9be00640 1 --2- 172.21.15.22:0/1991219465 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x560e32dcac00 0x560e32dd0ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:09:37.262 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:37.258+0000 7f8e9be00640 1 -- 172.21.15.22:0/1991219465 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x560e32dd16c0 con 0x560e32dc16d0 2024-03-28T18:09:37.262 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:37.258+0000 7f8ea0a00640 1 --2- 172.21.15.22:0/1991219465 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x560e32dcac00 0x560e32dd0ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-28T18:09:37.262 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:37.258+0000 7f8e93e00640 1 --2- 172.21.15.22:0/1991219465 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560e32dc2080 0x560e32dd2e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T18:09:37.263 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:37.258+0000 7f8e9be00640 1 --2- 172.21.15.22:0/1991219465 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560e32dc16d0 0x560e32dd28d0 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7f8e8c0bcd20 tx=0x7f8e8c005fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f8643e7298f27585 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:09:37.263 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:37.258+0000 7f8e89600640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-03-28T18:09:37.263 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:37.259+0000 7f8e89600640 1 -- 172.21.15.22:0/1991219465 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e8c0bd720 con 0x560e32dc16d0 2024-03-28T18:09:37.263 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:37.259+0000 7f8e89600640 1 -- 172.21.15.22:0/1991219465 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 8411+0+0 (secure 0 0 0) 0x7f8e8c0b9690 con 0x560e32dc16d0 2024-03-28T18:09:37.263 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:37.259+0000 7f8ea5322740 1 -- 172.21.15.22:0/1991219465 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f8e84028070 con 0x560e32dc16d0 2024-03-28T18:09:37.263 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:37.259+0000 7f8e89600640 1 -- 172.21.15.22:0/1991219465 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8e8c0bd020 con 0x560e32dc16d0 2024-03-28T18:09:37.263 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:37.259+0000 7f8ea5322740 10 client.16173 fetch_fsmap learned FSMap version 27 2024-03-28T18:09:37.263 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:37.259+0000 7f8ea5322740 1 -- 172.21.15.22:0/1991219465 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x560e32e36ea0 con 0x560e32dc16d0 2024-03-28T18:09:37.263 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:37.259+0000 7f8e89600640 1 -- 172.21.15.22:0/1991219465 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f8e84028070 con 0x560e32dc16d0 2024-03-28T18:09:37.263 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:37.259+0000 7f8e89600640 1 -- 172.21.15.22:0/1991219465 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 27) v1 ==== 7934+0+0 (secure 0 0 0) 0x7f8e8c0bd280 con 0x560e32dc16d0 2024-03-28T18:09:37.263 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:37.259+0000 7f8ea5322740 10 client.16173 fetch_fsmap finished waiting for FSMap version 27 2024-03-28T18:09:37.263 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:37.259+0000 7f8ea5322740 10 client.16173 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-28T18:09:37.263 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:37.259+0000 7f8ea5322740 20 client.16173 populate_metadata read hostname 'smithi022' 2024-03-28T18:09:37.264 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:37.260+0000 7f8ea5322740 1 --2- 172.21.15.22:0/1991219465 >> [v2:172.21.15.79:6838/2441542443,v1:172.21.15.79:6839/2441542443] conn(0x560e32e375a0 0x560e32e399c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:09:37.264 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:37.260+0000 7f8ea5322740 4 client.16173 mds_command: new command op to 34553 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-28T18:09:37.264 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:37.260+0000 7f8ea5322740 1 -- 172.21.15.22:0/1991219465 --> [v2:172.21.15.79:6838/2441542443,v1:172.21.15.79:6839/2441542443] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x560e32d2cb50 con 0x560e32e375a0 2024-03-28T18:09:37.264 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:37.260+0000 7f8e93e00640 1 --2- 172.21.15.22:0/1991219465 >> [v2:172.21.15.79:6838/2441542443,v1:172.21.15.79:6839/2441542443] conn(0x560e32e375a0 0x560e32e399c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:09:37.264 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:37.260+0000 7f8e93e00640 1 --2- 172.21.15.22:0/1991219465 >> [v2:172.21.15.79:6838/2441542443,v1:172.21.15.79:6839/2441542443] conn(0x560e32e375a0 0x560e32e399c0 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b67954f9d59146a1 server_cookie=559a9bcfdc6dd888 in_seq=0 out_seq=0 2024-03-28T18:09:37.264 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:37.260+0000 7f8e89600640 10 client.16173 ms_handle_connect on v2:172.21.15.79:6838/2441542443 2024-03-28T18:09:37.265 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:37.263+0000 7f8e89600640 1 -- 172.21.15.22:0/1991219465 <== mds.0 v2:172.21.15.79:6838/2441542443 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x560e32d2cb50 con 0x560e32e375a0 2024-03-28T18:09:37.265 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:37.263+0000 7f8e89600640 10 client.16173 handle_command_reply: tid=0 2024-03-28T18:09:37.265 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:37.263+0000 7f8ea5322740 1 client.16173 shutdown 2024-03-28T18:09:37.265 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:37.263+0000 7f8ea5322740 1 -- 172.21.15.22:0/1991219465 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560e32dc16d0 msgr2=0x560e32dd28d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:09:37.265 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:37.263+0000 7f8ea5322740 1 --2- 172.21.15.22:0/1991219465 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560e32dc16d0 0x560e32dd28d0 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7f8e8c0bcd20 tx=0x7f8e8c005fd0 comp rx=0 tx=0).stop 2024-03-28T18:09:37.265 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:37.263+0000 7f8ea5322740 1 -- 172.21.15.22:0/1991219465 shutdown_connections 2024-03-28T18:09:37.265 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:37.263+0000 7f8ea5322740 1 --2- 172.21.15.22:0/1991219465 >> [v2:172.21.15.79:6838/2441542443,v1:172.21.15.79:6839/2441542443] conn(0x560e32e375a0 0x560e32e399c0 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:09:37.266 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:37.263+0000 7f8ea5322740 1 --2- 172.21.15.22:0/1991219465 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x560e32dcac00 0x560e32dd0ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:09:37.266 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:37.263+0000 7f8ea5322740 1 --2- 172.21.15.22:0/1991219465 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560e32dc2080 0x560e32dd2e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:09:37.266 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:37.263+0000 7f8ea5322740 1 --2- 172.21.15.22:0/1991219465 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560e32dc16d0 0x560e32dd28d0 unknown :-1 s=CLOSED pgs=615 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:09:37.266 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:37.263+0000 7f8e89600640 0 client.16173 ms_handle_reset on v2:172.21.15.79:6838/2441542443 2024-03-28T18:09:37.266 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:37.263+0000 7f8ea5322740 1 -- 172.21.15.22:0/1991219465 >> 172.21.15.22:0/1991219465 conn(0x560e32dbbda0 msgr2=0x560e32dbd0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:09:37.266 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:37.263+0000 7f8ea5322740 1 -- 172.21.15.22:0/1991219465 shutdown_connections 2024-03-28T18:09:37.266 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:37.263+0000 7f8ea5322740 1 -- 172.21.15.22:0/1991219465 wait complete. 2024-03-28T18:09:37.266 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:37.263+0000 7f8ea5322740 20 client.16173 trim_cache size 0 max 16384 2024-03-28T18:09:37.266 INFO:teuthology.orchestra.run.smithi022.stdout:[] 2024-03-28T18:09:37.267 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:37.264+0000 7f8ea2a00640 1 -- 172.21.15.22:0/1105939788 >> [v2:172.21.15.22:6800/1357365082,v1:172.21.15.22:6801/1357365082] conn(0x7f8e800a2970 msgr2=0x7f8e800a4e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:09:37.267 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:37.264+0000 7f8ea2a00640 1 --2- 172.21.15.22:0/1105939788 >> [v2:172.21.15.22:6800/1357365082,v1:172.21.15.22:6801/1357365082] conn(0x7f8e800a2970 0x7f8e800a4e30 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f8e9c06abd0 tx=0x7f8e94009890 comp rx=0 tx=0).stop 2024-03-28T18:09:37.267 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:37.264+0000 7f8ea2a00640 1 -- 172.21.15.22:0/1105939788 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8e9c068880 msgr2=0x7f8e9c19a130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:09:37.267 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:37.264+0000 7f8ea2a00640 1 --2- 172.21.15.22:0/1105939788 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8e9c068880 0x7f8e9c19a130 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7f8e8c005940 tx=0x7f8e8c004bd0 comp rx=0 tx=0).stop 2024-03-28T18:09:37.267 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:37.264+0000 7f8ea2a00640 1 -- 172.21.15.22:0/1105939788 shutdown_connections 2024-03-28T18:09:37.267 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:37.264+0000 7f8ea2a00640 1 --2- 172.21.15.22:0/1105939788 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8e9c10e040 0x7f8e9c19e9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:09:37.267 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:37.264+0000 7f8ea2a00640 1 --2- 172.21.15.22:0/1105939788 >> [v2:172.21.15.22:6800/1357365082,v1:172.21.15.22:6801/1357365082] conn(0x7f8e800a2970 0x7f8e800a4e30 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:09:37.267 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:37.264+0000 7f8ea2a00640 1 --2- 172.21.15.22:0/1105939788 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f8e9c0691a0 0x7f8e9c19a670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:09:37.268 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:37.264+0000 7f8ea2a00640 1 --2- 172.21.15.22:0/1105939788 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8e9c068880 0x7f8e9c19a130 unknown :-1 s=CLOSED pgs=613 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:09:37.268 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:37.264+0000 7f8ea2a00640 1 -- 172.21.15.22:0/1105939788 >> 172.21.15.22:0/1105939788 conn(0x7f8e9c0776a0 msgr2=0x7f8e9c10f570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:09:37.268 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:37.264+0000 7f8ea2a00640 1 -- 172.21.15.22:0/1105939788 shutdown_connections 2024-03-28T18:09:37.268 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:37.264+0000 7f8ea2a00640 1 -- 172.21.15.22:0/1105939788 wait complete. 2024-03-28T18:09:37.293 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/interrupt-controller/ 2024-03-28T18:09:37.323 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/interrupt-controller/abilis,tb10x-ictl.txt 2024-03-28T18:09:37.332 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/interrupt-controller/allwinner,sun4i-ic.txt 2024-03-28T18:09:37.350 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/interrupt-controller/allwinner,sun67i-sc-nmi.txt 2024-03-28T18:09:37.367 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/interrupt-controller/atmel,aic.txt 2024-03-28T18:09:37.382 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/interrupt-controller/brcm,bcm2835-armctrl-ic.txt 2024-03-28T18:09:37.398 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/interrupt-controller/brcm,bcm7120-l2-intc.txt 2024-03-28T18:09:37.413 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/interrupt-controller/brcm,l2-intc.txt 2024-03-28T18:09:37.430 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/interrupt-controller/cirrus,clps711x-intc.txt 2024-03-28T18:09:37.446 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/interrupt-controller/digicolor-ic.txt 2024-03-28T18:09:37.463 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/interrupt-controller/interrupts.txt 2024-03-28T18:09:37.480 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/interrupt-controller/lsi,zevio-intc.txt 2024-03-28T18:09:37.497 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/interrupt-controller/marvell,armada-370-xp-mpic.txt 2024-03-28T18:09:37.513 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/interrupt-controller/marvell,orion-intc.txt 2024-03-28T18:09:37.532 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/interrupt-controller/mips-gic.txt 2024-03-28T18:09:37.554 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/interrupt-controller/opencores,or1k-pic.txt 2024-03-28T18:09:37.570 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/interrupt-controller/renesas,intc-irqpin.txt 2024-03-28T18:09:37.586 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/interrupt-controller/renesas,irqc.txt 2024-03-28T18:09:37.601 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/interrupt-controller/samsung,s3c24xx-irq.txt 2024-03-28T18:09:37.618 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/interrupt-controller/snps,dw-apb-ictl.txt 2024-03-28T18:09:37.633 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/interrupt-controller/ti,keystone-irq.txt 2024-03-28T18:09:37.648 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/interrupt-controller/ti,omap-intc-irq.txt 2024-03-28T18:09:37.665 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/iommu/ 2024-03-28T18:09:37.682 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/iommu/arm,smmu.txt 2024-03-28T18:09:37.692 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/iommu/iommu.txt 2024-03-28T18:09:37.710 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/iommu/nvidia,tegra20-gart.txt 2024-03-28T18:09:37.726 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/iommu/nvidia,tegra30-smmu.txt 2024-03-28T18:09:37.742 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/iommu/renesas,ipmmu-vmsa.txt 2024-03-28T18:09:37.759 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/iommu/rockchip,iommu.txt 2024-03-28T18:09:37.772 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/iommu/samsung,sysmmu.txt 2024-03-28T18:09:37.789 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/iommu/ti,omap-iommu.txt 2024-03-28T18:09:37.803 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/leds/ 2024-03-28T18:09:37.820 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/leds/common.txt 2024-03-28T18:09:37.832 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/leds/leds-gpio.txt 2024-03-28T18:09:37.849 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/leds/leds-lp55xx.txt 2024-03-28T18:09:37.866 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/leds/leds-lp8860.txt 2024-03-28T18:09:37.883 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/leds/leds-ns2.txt 2024-03-28T18:09:37.898 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/leds/leds-pwm.txt 2024-03-28T18:09:37.915 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/leds/pca963x.txt 2024-03-28T18:09:37.930 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/leds/register-bit-led.txt 2024-03-28T18:09:37.947 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/leds/tca6507.txt 2024-03-28T18:09:37.963 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/lpddr2/ 2024-03-28T18:09:37.981 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/lpddr2/lpddr2-timings.txt 2024-03-28T18:09:37.991 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/lpddr2/lpddr2.txt 2024-03-28T18:09:38.007 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/mailbox/ 2024-03-28T18:09:38.026 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/mailbox/altera-mailbox.txt 2024-03-28T18:09:38.034 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/mailbox/mailbox.txt 2024-03-28T18:09:38.051 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/mailbox/omap-mailbox.txt 2024-03-28T18:09:38.070 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/marvell.txt 2024-03-28T18:09:38.094 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/media/ 2024-03-28T18:09:38.107 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/media/atmel-isi.txt 2024-03-28T18:09:38.117 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/media/coda.txt 2024-03-28T18:09:38.135 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/media/exynos-fimc-lite.txt 2024-03-28T18:09:38.150 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/media/exynos-jpeg-codec.txt 2024-03-28T18:09:38.169 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/media/exynos4-fimc-is.txt 2024-03-28T18:09:38.186 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/media/exynos5-gsc.txt 2024-03-28T18:09:38.202 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/media/gpio-ir-receiver.txt 2024-03-28T18:09:38.219 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/media/hix5hd2-ir.txt 2024-03-28T18:09:38.234 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/media/i2c/ 2024-03-28T18:09:38.261 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/media/i2c/adv7343.txt 2024-03-28T18:09:38.282 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-03-28T18:09:38.282 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-28T18:09:38.282 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-03-28T18:09:38.294 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/media/i2c/adv7604.txt 2024-03-28T18:09:38.326 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/media/i2c/mt9m111.txt 2024-03-28T18:09:38.338 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/media/i2c/mt9p031.txt 2024-03-28T18:09:38.350 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/media/i2c/nokia,smia.txt 2024-03-28T18:09:38.364 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/media/i2c/ths8200.txt 2024-03-28T18:09:38.378 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/media/i2c/tvp514x.txt 2024-03-28T18:09:38.393 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/media/i2c/tvp7002.txt 2024-03-28T18:09:38.408 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/media/img-ir-rev1.txt 2024-03-28T18:09:38.426 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/media/meson-ir.txt 2024-03-28T18:09:38.440 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/media/pxa-camera.txt 2024-03-28T18:09:38.454 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/media/rcar_vin.txt 2024-03-28T18:09:38.469 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/media/renesas,vsp1.txt 2024-03-28T18:09:38.483 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/media/s5p-mfc.txt 2024-03-28T18:09:38.498 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/media/samsung-fimc.txt 2024-03-28T18:09:38.513 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/media/samsung-mipi-csis.txt 2024-03-28T18:09:38.527 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/media/samsung-s5c73m3.txt 2024-03-28T18:09:38.535 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:38.535+0000 7f0b4da00640 1 -- 172.21.15.22:0/4155071359 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0b4810bb60 msgr2=0x7f0b4810df90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:09:38.535 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:38.535+0000 7f0b4da00640 1 --2- 172.21.15.22:0/4155071359 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0b4810bb60 0x7f0b4810df90 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7f0b40009ca0 tx=0x7f0b4002c180 comp rx=0 tx=0).stop 2024-03-28T18:09:38.535 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:38.535+0000 7f0b4da00640 1 -- 172.21.15.22:0/4155071359 shutdown_connections 2024-03-28T18:09:38.535 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:38.535+0000 7f0b4da00640 1 --2- 172.21.15.22:0/4155071359 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0b4810bb60 0x7f0b4810df90 unknown :-1 s=CLOSED pgs=616 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:09:38.535 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:38.535+0000 7f0b4da00640 1 --2- 172.21.15.22:0/4155071359 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0b481030b0 0x7f0b4810b590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:09:38.535 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:38.535+0000 7f0b4da00640 1 --2- 172.21.15.22:0/4155071359 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f0b48102790 0x7f0b48102b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:09:38.535 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:38.535+0000 7f0b4da00640 1 -- 172.21.15.22:0/4155071359 >> 172.21.15.22:0/4155071359 conn(0x7f0b480fbef0 msgr2=0x7f0b480fe310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:09:38.535 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:38.535+0000 7f0b4da00640 1 -- 172.21.15.22:0/4155071359 shutdown_connections 2024-03-28T18:09:38.535 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:38.535+0000 7f0b4da00640 1 -- 172.21.15.22:0/4155071359 wait complete. 2024-03-28T18:09:38.537 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:38.536+0000 7f0b4da00640 1 Processor -- start 2024-03-28T18:09:38.537 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:38.536+0000 7f0b4da00640 1 -- start start 2024-03-28T18:09:38.537 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:38.537+0000 7f0b4da00640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f0b48102790 0x7f0b48199f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:09:38.537 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:38.537+0000 7f0b4da00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0b481030b0 0x7f0b4819a4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:09:38.537 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:38.537+0000 7f0b4da00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0b4810bb60 0x7f0b4819e840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:09:38.537 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:38.537+0000 7f0b4da00640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b481118a0 con 0x7f0b481030b0 2024-03-28T18:09:38.537 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:38.537+0000 7f0b4da00640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b48111720 con 0x7f0b48102790 2024-03-28T18:09:38.537 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:38.537+0000 7f0b4da00640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b48111a20 con 0x7f0b4810bb60 2024-03-28T18:09:38.537 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:38.537+0000 7f0b47400640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0b481030b0 0x7f0b4819a4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:09:38.538 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:38.537+0000 7f0b47e00640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f0b48102790 0x7f0b48199f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:09:38.538 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:38.537+0000 7f0b47400640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0b481030b0 0x7f0b4819a4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:46584/0 (socket says 172.21.15.22:46584) 2024-03-28T18:09:38.538 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:38.537+0000 7f0b47400640 1 -- 172.21.15.22:0/1259619197 learned_addr learned my addr 172.21.15.22:0/1259619197 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-28T18:09:38.538 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:38.537+0000 7f0b4d000640 1 --2- 172.21.15.22:0/1259619197 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0b4810bb60 0x7f0b4819e840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:09:38.538 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:38.537+0000 7f0b47400640 1 -- 172.21.15.22:0/1259619197 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0b4810bb60 msgr2=0x7f0b4819e840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:09:38.538 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:38.537+0000 7f0b47400640 1 --2- 172.21.15.22:0/1259619197 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0b4810bb60 0x7f0b4819e840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:09:38.538 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:38.537+0000 7f0b47400640 1 -- 172.21.15.22:0/1259619197 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f0b48102790 msgr2=0x7f0b48199f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:09:38.538 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:38.537+0000 7f0b47400640 1 --2- 172.21.15.22:0/1259619197 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f0b48102790 0x7f0b48199f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:09:38.538 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:38.538+0000 7f0b47400640 1 -- 172.21.15.22:0/1259619197 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0b4819ef70 con 0x7f0b481030b0 2024-03-28T18:09:38.539 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:38.538+0000 7f0b47400640 1 --2- 172.21.15.22:0/1259619197 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0b481030b0 0x7f0b4819a4c0 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7f0b3400d950 tx=0x7f0b3400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:09:38.539 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:38.538+0000 7f0b44c00640 1 -- 172.21.15.22:0/1259619197 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b34017070 con 0x7f0b481030b0 2024-03-28T18:09:38.539 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:38.538+0000 7f0b44c00640 1 -- 172.21.15.22:0/1259619197 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f0b34009c30 con 0x7f0b481030b0 2024-03-28T18:09:38.539 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:38.538+0000 7f0b4da00640 1 -- 172.21.15.22:0/1259619197 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0b4819f260 con 0x7f0b481030b0 2024-03-28T18:09:38.539 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:38.538+0000 7f0b44c00640 1 -- 172.21.15.22:0/1259619197 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b3400f6b0 con 0x7f0b481030b0 2024-03-28T18:09:38.539 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:38.538+0000 7f0b4da00640 1 -- 172.21.15.22:0/1259619197 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0b481a6af0 con 0x7f0b481030b0 2024-03-28T18:09:38.540 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/media/samsung-s5k5baf.txt 2024-03-28T18:09:38.542 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:38.542+0000 7f0b51c66740 1 Processor -- start 2024-03-28T18:09:38.542 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:38.542+0000 7f0b51c66740 1 -- start start 2024-03-28T18:09:38.543 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:38.543+0000 7f0b51c66740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560fd02bc290 0x560fd02bc670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:09:38.543 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:38.543+0000 7f0b51c66740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560fd02bcbb0 0x560fd02c5250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:09:38.543 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:38.543+0000 7f0b51c66740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x560fd02c5790 0x560fd02c7b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:09:38.543 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:38.543+0000 7f0b51c66740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x560fd0220eb0 con 0x560fd02bc290 2024-03-28T18:09:38.543 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:38.543+0000 7f0b51c66740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x560fd01c9a60 con 0x560fd02c5790 2024-03-28T18:09:38.543 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:38.543+0000 7f0b51c66740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x560fd021cf50 con 0x560fd02bcbb0 2024-03-28T18:09:38.543 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:38.543+0000 7f0b44c00640 1 -- 172.21.15.22:0/1259619197 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 26) v1 ==== 137421+0+0 (secure 0 0 0) 0x7f0b34010a50 con 0x7f0b481030b0 2024-03-28T18:09:38.543 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:38.543+0000 7f0b4d000640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560fd02bc290 0x560fd02bc670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:09:38.544 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:38.543+0000 7f0b4d000640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560fd02bc290 0x560fd02bc670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:46590/0 (socket says 172.21.15.22:46590) 2024-03-28T18:09:38.544 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:38.543+0000 7f0b4d000640 1 -- 172.21.15.22:0/4203760545 learned_addr learned my addr 172.21.15.22:0/4203760545 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-28T18:09:38.544 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:38.543+0000 7f0b47e00640 1 --2- 172.21.15.22:0/4203760545 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560fd02bcbb0 0x560fd02c5250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:09:38.544 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:38.543+0000 7f0b47400640 1 --2- 172.21.15.22:0/4203760545 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x560fd02c5790 0x560fd02c7b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:09:38.546 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:38.544+0000 7f0b4d000640 1 -- 172.21.15.22:0/4203760545 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560fd02bcbb0 msgr2=0x560fd02c5250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T18:09:38.546 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:38.544+0000 7f0b4d000640 1 --2- 172.21.15.22:0/4203760545 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560fd02bcbb0 0x560fd02c5250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:09:38.546 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:38.544+0000 7f0b4d000640 1 -- 172.21.15.22:0/4203760545 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x560fd02c5790 msgr2=0x560fd02c7b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T18:09:38.546 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:38.544+0000 7f0b4d000640 1 --2- 172.21.15.22:0/4203760545 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x560fd02c5790 0x560fd02c7b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:09:38.546 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:38.544+0000 7f0b4d000640 1 -- 172.21.15.22:0/4203760545 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560fcfffd7e0 con 0x560fd02bc290 2024-03-28T18:09:38.546 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:38.544+0000 7f0b4d000640 1 --2- 172.21.15.22:0/4203760545 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560fd02bc290 0x560fd02bc670 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7f0b40005e30 tx=0x7f0b40009880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=826d6820afeb0c39 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:09:38.546 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:38.544+0000 7f0b3d600640 1 -- 172.21.15.22:0/4203760545 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b40044020 con 0x560fd02bc290 2024-03-28T18:09:38.546 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:38.545+0000 7f0b3d600640 1 -- 172.21.15.22:0/4203760545 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f0b40003e40 con 0x560fd02bc290 2024-03-28T18:09:38.546 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:38.545+0000 7f0b3d600640 1 -- 172.21.15.22:0/4203760545 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b40039040 con 0x560fd02bc290 2024-03-28T18:09:38.546 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:38.545+0000 7f0b44c00640 1 --2- 172.21.15.22:0/1259619197 >> [v2:172.21.15.22:6800/1357365082,v1:172.21.15.22:6801/1357365082] conn(0x7f0b2c0a2970 0x7f0b2c0a4e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:09:38.546 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:38.545+0000 7f0b44c00640 1 -- 172.21.15.22:0/1259619197 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 8411+0+0 (secure 0 0 0) 0x7f0b340ce9c0 con 0x7f0b481030b0 2024-03-28T18:09:38.546 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:38.545+0000 7f0b47e00640 1 --2- 172.21.15.22:0/1259619197 >> [v2:172.21.15.22:6800/1357365082,v1:172.21.15.22:6801/1357365082] conn(0x7f0b2c0a2970 0x7f0b2c0a4e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:09:38.547 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:38.545+0000 7f0b51c66740 1 -- 172.21.15.22:0/4203760545 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560fd02bc290 msgr2=0x560fd02bc670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:09:38.547 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:38.545+0000 7f0b51c66740 1 --2- 172.21.15.22:0/4203760545 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560fd02bc290 0x560fd02bc670 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7f0b40005e30 tx=0x7f0b40009880 comp rx=0 tx=0).stop 2024-03-28T18:09:38.547 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:38.546+0000 7f0b51c66740 1 -- 172.21.15.22:0/4203760545 shutdown_connections 2024-03-28T18:09:38.547 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:38.546+0000 7f0b51c66740 1 --2- 172.21.15.22:0/4203760545 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x560fd02c5790 0x560fd02c7b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:09:38.547 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:38.546+0000 7f0b51c66740 1 --2- 172.21.15.22:0/4203760545 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560fd02bcbb0 0x560fd02c5250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:09:38.547 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:38.546+0000 7f0b51c66740 1 --2- 172.21.15.22:0/4203760545 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560fd02bc290 0x560fd02bc670 unknown :-1 s=CLOSED pgs=618 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:09:38.547 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:38.546+0000 7f0b51c66740 1 -- 172.21.15.22:0/4203760545 >> 172.21.15.22:0/4203760545 conn(0x560fd02b69c0 msgr2=0x560fd02b6da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:09:38.547 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:38.546+0000 7f0b51c66740 1 -- 172.21.15.22:0/4203760545 shutdown_connections 2024-03-28T18:09:38.547 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:38.546+0000 7f0b51c66740 1 -- 172.21.15.22:0/4203760545 wait complete. 2024-03-28T18:09:38.547 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:38.546+0000 7f0b47e00640 1 --2- 172.21.15.22:0/1259619197 >> [v2:172.21.15.22:6800/1357365082,v1:172.21.15.22:6801/1357365082] conn(0x7f0b2c0a2970 0x7f0b2c0a4e30 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x560fd02bbd60 tx=0x7f0b38005f90 comp rx=0 tx=0).ready entity=mgr.15522 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:09:38.547 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:38.546+0000 7f0b51c66740 1 -- start start 2024-03-28T18:09:38.550 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:38.547+0000 7f0b51c66740 1 Processor -- start 2024-03-28T18:09:38.550 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:38.547+0000 7f0b51c66740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x560fd02bc290 0x560fd02c5080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:09:38.550 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:38.547+0000 7f0b51c66740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560fd02bcbb0 0x560fd02c2410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:09:38.550 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:38.547+0000 7f0b51c66740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560fd02c5790 0x560fd02c2950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:09:38.550 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:38.547+0000 7f0b51c66740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x560fd02c8520 con 0x560fd02bcbb0 2024-03-28T18:09:38.550 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:38.547+0000 7f0b51c66740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x560fd01dceb0 con 0x560fd02bc290 2024-03-28T18:09:38.551 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:38.547+0000 7f0b51c66740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x560fd02c86a0 con 0x560fd02c5790 2024-03-28T18:09:38.551 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:38.547+0000 7f0b51c66740 10 client.0 _ll_register_callbacks cb 0x560fd01d1150 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-28T18:09:38.551 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:38.547+0000 7f0b47400640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560fd02bcbb0 0x560fd02c2410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:09:38.551 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:38.547+0000 7f0b47400640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560fd02bcbb0 0x560fd02c2410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:46606/0 (socket says 172.21.15.22:46606) 2024-03-28T18:09:38.551 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:38.547+0000 7f0b47400640 1 -- 172.21.15.22:0/4226740915 learned_addr learned my addr 172.21.15.22:0/4226740915 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-28T18:09:38.551 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:38.547+0000 7f0b47e00640 1 --2- 172.21.15.22:0/4226740915 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x560fd02bc290 0x560fd02c5080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:09:38.551 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:38.548+0000 7f0b4d000640 1 --2- 172.21.15.22:0/4226740915 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560fd02c5790 0x560fd02c2950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:09:38.551 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:38.548+0000 7f0b47400640 1 -- 172.21.15.22:0/4226740915 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560fd02c5790 msgr2=0x560fd02c2950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T18:09:38.551 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:38.548+0000 7f0b47400640 1 --2- 172.21.15.22:0/4226740915 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560fd02c5790 0x560fd02c2950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:09:38.551 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:38.548+0000 7f0b47400640 1 -- 172.21.15.22:0/4226740915 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x560fd02bc290 msgr2=0x560fd02c5080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T18:09:38.551 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:38.548+0000 7f0b47400640 1 --2- 172.21.15.22:0/4226740915 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x560fd02bc290 0x560fd02c5080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:09:38.551 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:38.548+0000 7f0b47400640 1 -- 172.21.15.22:0/4226740915 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x560fd005df70 con 0x560fd02bcbb0 2024-03-28T18:09:38.551 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:38.548+0000 7f0b47400640 1 --2- 172.21.15.22:0/4226740915 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560fd02bcbb0 0x560fd02c2410 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7f0b3409ad90 tx=0x7f0b3409af20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e15ff1f5be904ac7 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:09:38.551 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:38.548+0000 7f0b32000640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-03-28T18:09:38.551 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:38.548+0000 7f0b32000640 1 -- 172.21.15.22:0/4226740915 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b34020040 con 0x560fd02bcbb0 2024-03-28T18:09:38.552 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:38.548+0000 7f0b32000640 1 -- 172.21.15.22:0/4226740915 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 8411+0+0 (secure 0 0 0) 0x7f0b340b9b90 con 0x560fd02bcbb0 2024-03-28T18:09:38.552 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:38.548+0000 7f0b51c66740 1 -- 172.21.15.22:0/4226740915 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x560fd022e3f0 con 0x560fd02bcbb0 2024-03-28T18:09:38.552 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:38.549+0000 7f0b32000640 1 -- 172.21.15.22:0/4226740915 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0b34009de0 con 0x560fd02bcbb0 2024-03-28T18:09:38.552 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:38.549+0000 7f0b51c66740 10 client.16185 fetch_fsmap learned FSMap version 27 2024-03-28T18:09:38.552 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:38.549+0000 7f0b51c66740 1 -- 172.21.15.22:0/4226740915 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x560fcfef5170 con 0x560fd02bcbb0 2024-03-28T18:09:38.552 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:38.549+0000 7f0b32000640 1 -- 172.21.15.22:0/4226740915 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x560fd022e3f0 con 0x560fd02bcbb0 2024-03-28T18:09:38.552 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:38.549+0000 7f0b32000640 1 -- 172.21.15.22:0/4226740915 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 27) v1 ==== 7934+0+0 (secure 0 0 0) 0x7f0b34096de0 con 0x560fd02bcbb0 2024-03-28T18:09:38.552 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:38.549+0000 7f0b51c66740 10 client.16185 fetch_fsmap finished waiting for FSMap version 27 2024-03-28T18:09:38.552 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:38.549+0000 7f0b51c66740 10 client.16185 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-28T18:09:38.552 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:38.549+0000 7f0b51c66740 20 client.16185 populate_metadata read hostname 'smithi022' 2024-03-28T18:09:38.552 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:38.549+0000 7f0b51c66740 1 --2- 172.21.15.22:0/4226740915 >> [v2:172.21.15.79:6838/2441542443,v1:172.21.15.79:6839/2441542443] conn(0x560fd02d5670 0x560fd02d7a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:09:38.552 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:38.549+0000 7f0b51c66740 4 client.16185 mds_command: new command op to 34553 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-28T18:09:38.552 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:38.549+0000 7f0b51c66740 1 -- 172.21.15.22:0/4226740915 --> [v2:172.21.15.79:6838/2441542443,v1:172.21.15.79:6839/2441542443] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x560fd0228740 con 0x560fd02d5670 2024-03-28T18:09:38.552 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:38.550+0000 7f0b47e00640 1 --2- 172.21.15.22:0/4226740915 >> [v2:172.21.15.79:6838/2441542443,v1:172.21.15.79:6839/2441542443] conn(0x560fd02d5670 0x560fd02d7a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:09:38.553 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/media/samsung-s5k6a3.txt 2024-03-28T18:09:38.554 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:38.550+0000 7f0b47e00640 1 --2- 172.21.15.22:0/4226740915 >> [v2:172.21.15.79:6838/2441542443,v1:172.21.15.79:6839/2441542443] conn(0x560fd02d5670 0x560fd02d7a60 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=729af8176dc17b18 server_cookie=148bf88310a13598 in_seq=0 out_seq=0 2024-03-28T18:09:38.554 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:38.550+0000 7f0b32000640 10 client.16185 ms_handle_connect on v2:172.21.15.79:6838/2441542443 2024-03-28T18:09:38.554 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:38.551+0000 7f0b32000640 1 -- 172.21.15.22:0/4226740915 <== mds.0 v2:172.21.15.79:6838/2441542443 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x560fd0228740 con 0x560fd02d5670 2024-03-28T18:09:38.554 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:38.551+0000 7f0b32000640 10 client.16185 handle_command_reply: tid=0 2024-03-28T18:09:38.554 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:38.551+0000 7f0b51c66740 1 client.16185 shutdown 2024-03-28T18:09:38.554 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:38.552+0000 7f0b51c66740 1 -- 172.21.15.22:0/4226740915 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560fd02bcbb0 msgr2=0x560fd02c2410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:09:38.554 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:38.552+0000 7f0b51c66740 1 --2- 172.21.15.22:0/4226740915 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560fd02bcbb0 0x560fd02c2410 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7f0b3409ad90 tx=0x7f0b3409af20 comp rx=0 tx=0).stop 2024-03-28T18:09:38.554 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:38.552+0000 7f0b51c66740 1 -- 172.21.15.22:0/4226740915 shutdown_connections 2024-03-28T18:09:38.554 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:38.552+0000 7f0b51c66740 1 --2- 172.21.15.22:0/4226740915 >> [v2:172.21.15.79:6838/2441542443,v1:172.21.15.79:6839/2441542443] conn(0x560fd02d5670 0x560fd02d7a60 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:09:38.554 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:38.552+0000 7f0b51c66740 1 --2- 172.21.15.22:0/4226740915 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560fd02c5790 0x560fd02c2950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:09:38.554 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:38.552+0000 7f0b51c66740 1 --2- 172.21.15.22:0/4226740915 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560fd02bcbb0 0x560fd02c2410 unknown :-1 s=CLOSED pgs=619 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:09:38.554 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:38.552+0000 7f0b32000640 0 client.16185 ms_handle_reset on v2:172.21.15.79:6838/2441542443 2024-03-28T18:09:38.555 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:38.552+0000 7f0b51c66740 1 --2- 172.21.15.22:0/4226740915 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x560fd02bc290 0x560fd02c5080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:09:38.555 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:38.552+0000 7f0b51c66740 1 -- 172.21.15.22:0/4226740915 >> 172.21.15.22:0/4226740915 conn(0x560fd02b69c0 msgr2=0x560fd02b8710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:09:38.555 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:38.552+0000 7f0b51c66740 1 -- 172.21.15.22:0/4226740915 shutdown_connections 2024-03-28T18:09:38.555 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:38.552+0000 7f0b51c66740 1 -- 172.21.15.22:0/4226740915 wait complete. 2024-03-28T18:09:38.555 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:38.553+0000 7f0b51c66740 20 client.16185 trim_cache size 0 max 16384 2024-03-28T18:09:38.560 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/media/sh_mobile_ceu.txt 2024-03-28T18:09:38.570 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/media/si4713.txt 2024-03-28T18:09:38.572 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:38.572+0000 7f0b51c66740 1 Processor -- start 2024-03-28T18:09:38.572 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:38.572+0000 7f0b51c66740 1 -- start start 2024-03-28T18:09:38.573 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:38.573+0000 7f0b51c66740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560fd02bc290 0x560fd02c4b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:09:38.573 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:38.573+0000 7f0b51c66740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x560fd02bcbb0 0x560fd02c5080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:09:38.573 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:38.573+0000 7f0b51c66740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560fd02c5790 0x560fd02b83d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:09:38.574 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:38.573+0000 7f0b51c66740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b481aeff0 con 0x560fd02bc290 2024-03-28T18:09:38.574 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:38.573+0000 7f0b51c66740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b481b0970 con 0x560fd02bcbb0 2024-03-28T18:09:38.574 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:38.573+0000 7f0b51c66740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b481af650 con 0x560fd02c5790 2024-03-28T18:09:38.574 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:38.573+0000 7f0b47e00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560fd02bc290 0x560fd02c4b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:09:38.574 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:38.573+0000 7f0b47400640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x560fd02bcbb0 0x560fd02c5080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:09:38.574 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:38.573+0000 7f0b47400640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x560fd02bcbb0 0x560fd02c5080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.22:41082/0 (socket says 172.21.15.22:41082) 2024-03-28T18:09:38.574 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:38.573+0000 7f0b47400640 1 -- 172.21.15.22:0/3916622340 learned_addr learned my addr 172.21.15.22:0/3916622340 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-28T18:09:38.574 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:38.573+0000 7f0b4d000640 1 --2- 172.21.15.22:0/3916622340 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560fd02c5790 0x560fd02b83d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:09:38.574 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:38.574+0000 7f0b47400640 1 -- 172.21.15.22:0/3916622340 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560fd02c5790 msgr2=0x560fd02b83d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T18:09:38.574 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:38.574+0000 7f0b47400640 1 --2- 172.21.15.22:0/3916622340 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560fd02c5790 0x560fd02b83d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:09:38.574 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:38.574+0000 7f0b47400640 1 -- 172.21.15.22:0/3916622340 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560fd02bc290 msgr2=0x560fd02c4b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T18:09:38.574 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:38.574+0000 7f0b47400640 1 --2- 172.21.15.22:0/3916622340 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560fd02bc290 0x560fd02c4b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:09:38.574 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:38.574+0000 7f0b47400640 1 -- 172.21.15.22:0/3916622340 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560fd02b0c40 con 0x560fd02bcbb0 2024-03-28T18:09:38.575 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:38.574+0000 7f0b47400640 1 --2- 172.21.15.22:0/3916622340 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x560fd02bcbb0 0x560fd02c5080 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f0b34097180 tx=0x7f0b3409af90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1d3aa7ddf6db7fce server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:09:38.579 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:38.579+0000 7f0b3cc00640 1 -- 172.21.15.22:0/3916622340 <== mon.2 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b34020070 con 0x560fd02bcbb0 2024-03-28T18:09:38.579 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:38.579+0000 7f0b3cc00640 1 -- 172.21.15.22:0/3916622340 <== mon.2 v2:172.21.15.79:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f0b3409b090 con 0x560fd02bcbb0 2024-03-28T18:09:38.579 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:38.580+0000 7f0b3cc00640 1 -- 172.21.15.22:0/3916622340 <== mon.2 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b340b9b90 con 0x560fd02bcbb0 2024-03-28T18:09:38.580 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:38.580+0000 7f0b51c66740 1 -- 172.21.15.22:0/3916622340 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x560fd02bcbb0 msgr2=0x560fd02c5080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:09:38.580 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:38.580+0000 7f0b51c66740 1 --2- 172.21.15.22:0/3916622340 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x560fd02bcbb0 0x560fd02c5080 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f0b34097180 tx=0x7f0b3409af90 comp rx=0 tx=0).stop 2024-03-28T18:09:38.580 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:38.580+0000 7f0b51c66740 1 -- 172.21.15.22:0/3916622340 shutdown_connections 2024-03-28T18:09:38.581 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:38.580+0000 7f0b51c66740 1 --2- 172.21.15.22:0/3916622340 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560fd02c5790 0x560fd02b83d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:09:38.581 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:38.580+0000 7f0b51c66740 1 --2- 172.21.15.22:0/3916622340 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x560fd02bcbb0 0x560fd02c5080 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:09:38.581 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:38.580+0000 7f0b51c66740 1 --2- 172.21.15.22:0/3916622340 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560fd02bc290 0x560fd02c4b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:09:38.581 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:38.581+0000 7f0b51c66740 1 -- 172.21.15.22:0/3916622340 >> 172.21.15.22:0/3916622340 conn(0x560fd02b69c0 msgr2=0x560fd02b7cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:09:38.581 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:38.581+0000 7f0b51c66740 1 -- 172.21.15.22:0/3916622340 shutdown_connections 2024-03-28T18:09:38.581 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:38.581+0000 7f0b51c66740 1 -- 172.21.15.22:0/3916622340 wait complete. 2024-03-28T18:09:38.581 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:38.581+0000 7f0b51c66740 1 -- start start 2024-03-28T18:09:38.582 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:38.581+0000 7f0b51c66740 1 Processor -- start 2024-03-28T18:09:38.582 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/media/st-rc.txt 2024-03-28T18:09:38.584 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:38.582+0000 7f0b51c66740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560fd02bc290 0x560fd02bf6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:09:38.584 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:38.582+0000 7f0b51c66740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560fd02bcbb0 0x560fd02bfc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:09:38.584 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:38.582+0000 7f0b51c66740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x560fd02c5790 0x560fd0328e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:09:38.584 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:38.582+0000 7f0b51c66740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b481b0310 con 0x560fd02bcbb0 2024-03-28T18:09:38.584 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:38.582+0000 7f0b51c66740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b481afcb0 con 0x560fd02c5790 2024-03-28T18:09:38.584 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:38.582+0000 7f0b51c66740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b481ad1d0 con 0x560fd02bc290 2024-03-28T18:09:38.584 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:38.582+0000 7f0b51c66740 10 client.0 _ll_register_callbacks cb 0x7f0b3400df40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-28T18:09:38.584 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:38.582+0000 7f0b47400640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560fd02bcbb0 0x560fd02bfc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:09:38.584 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:38.582+0000 7f0b47400640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560fd02bcbb0 0x560fd02bfc20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:46624/0 (socket says 172.21.15.22:46624) 2024-03-28T18:09:38.584 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:38.582+0000 7f0b47400640 1 -- 172.21.15.22:0/138944578 learned_addr learned my addr 172.21.15.22:0/138944578 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-28T18:09:38.584 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:38.582+0000 7f0b47e00640 1 --2- 172.21.15.22:0/138944578 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560fd02bc290 0x560fd02bf6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:09:38.584 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:38.583+0000 7f0b4d000640 1 --2- 172.21.15.22:0/138944578 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x560fd02c5790 0x560fd0328e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:09:38.585 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:38.583+0000 7f0b47400640 1 -- 172.21.15.22:0/138944578 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560fd02bc290 msgr2=0x560fd02bf6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T18:09:38.585 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:38.583+0000 7f0b47400640 1 --2- 172.21.15.22:0/138944578 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560fd02bc290 0x560fd02bf6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:09:38.585 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:38.583+0000 7f0b47400640 1 -- 172.21.15.22:0/138944578 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x560fd02c5790 msgr2=0x560fd0328e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T18:09:38.585 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:38.583+0000 7f0b47400640 1 --2- 172.21.15.22:0/138944578 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x560fd02c5790 0x560fd0328e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:09:38.585 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:38.583+0000 7f0b47400640 1 -- 172.21.15.22:0/138944578 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x560fd0329660 con 0x560fd02bcbb0 2024-03-28T18:09:38.585 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:38.583+0000 7f0b4d000640 1 --2- 172.21.15.22:0/138944578 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x560fd02c5790 0x560fd0328e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-28T18:09:38.585 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:38.584+0000 7f0b47400640 1 --2- 172.21.15.22:0/138944578 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560fd02bcbb0 0x560fd02bfc20 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7f0b34097a60 tx=0x7f0b3409af20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5464043f93171604 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:09:38.585 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:38.584+0000 7f0b32000640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-03-28T18:09:38.585 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:38.584+0000 7f0b32000640 1 -- 172.21.15.22:0/138944578 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b34020070 con 0x560fd02bcbb0 2024-03-28T18:09:38.585 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:38.584+0000 7f0b32000640 1 -- 172.21.15.22:0/138944578 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 8411+0+0 (secure 0 0 0) 0x7f0b34099f50 con 0x560fd02bcbb0 2024-03-28T18:09:38.585 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:38.584+0000 7f0b51c66740 1 -- 172.21.15.22:0/138944578 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f0b3409b090 con 0x560fd02bcbb0 2024-03-28T18:09:38.585 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:38.584+0000 7f0b32000640 1 -- 172.21.15.22:0/138944578 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0b3409f1e0 con 0x560fd02bcbb0 2024-03-28T18:09:38.585 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:38.584+0000 7f0b51c66740 10 client.16191 fetch_fsmap learned FSMap version 27 2024-03-28T18:09:38.585 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:38.584+0000 7f0b51c66740 1 -- 172.21.15.22:0/138944578 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x560fd03319d0 con 0x560fd02bcbb0 2024-03-28T18:09:38.585 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:38.584+0000 7f0b32000640 1 -- 172.21.15.22:0/138944578 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f0b3409b090 con 0x560fd02bcbb0 2024-03-28T18:09:38.586 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:38.585+0000 7f0b32000640 1 -- 172.21.15.22:0/138944578 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 27) v1 ==== 7934+0+0 (secure 0 0 0) 0x7f0b3409a150 con 0x560fd02bcbb0 2024-03-28T18:09:38.586 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:38.585+0000 7f0b51c66740 10 client.16191 fetch_fsmap finished waiting for FSMap version 27 2024-03-28T18:09:38.586 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:38.585+0000 7f0b51c66740 10 client.16191 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-28T18:09:38.587 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:38.585+0000 7f0b51c66740 20 client.16191 populate_metadata read hostname 'smithi022' 2024-03-28T18:09:38.587 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:38.585+0000 7f0b51c66740 1 --2- 172.21.15.22:0/138944578 >> [v2:172.21.15.79:6838/2441542443,v1:172.21.15.79:6839/2441542443] conn(0x560fd0331f60 0x560fd0334380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:09:38.587 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:38.585+0000 7f0b51c66740 4 client.16191 mds_command: new command op to 34553 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-03-28T18:09:38.587 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:38.585+0000 7f0b51c66740 1 -- 172.21.15.22:0/138944578 --> [v2:172.21.15.79:6838/2441542443,v1:172.21.15.79:6839/2441542443] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x560fd01a6a40 con 0x560fd0331f60 2024-03-28T18:09:38.587 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:38.585+0000 7f0b47e00640 1 --2- 172.21.15.22:0/138944578 >> [v2:172.21.15.79:6838/2441542443,v1:172.21.15.79:6839/2441542443] conn(0x560fd0331f60 0x560fd0334380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:09:38.587 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:38.586+0000 7f0b47e00640 1 --2- 172.21.15.22:0/138944578 >> [v2:172.21.15.79:6838/2441542443,v1:172.21.15.79:6839/2441542443] conn(0x560fd0331f60 0x560fd0334380 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bca38ffe39f9a9c1 server_cookie=3b24f7b50371f949 in_seq=0 out_seq=0 2024-03-28T18:09:38.587 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:38.586+0000 7f0b32000640 10 client.16191 ms_handle_connect on v2:172.21.15.79:6838/2441542443 2024-03-28T18:09:38.588 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:38.587+0000 7f0b32000640 1 -- 172.21.15.22:0/138944578 <== mds.0 v2:172.21.15.79:6838/2441542443 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x560fd01a6a40 con 0x560fd0331f60 2024-03-28T18:09:38.588 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:38.587+0000 7f0b32000640 10 client.16191 handle_command_reply: tid=0 2024-03-28T18:09:38.588 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:38.588+0000 7f0b51c66740 1 client.16191 shutdown 2024-03-28T18:09:38.588 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:38.588+0000 7f0b51c66740 1 -- 172.21.15.22:0/138944578 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560fd02bcbb0 msgr2=0x560fd02bfc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:09:38.588 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:38.588+0000 7f0b51c66740 1 --2- 172.21.15.22:0/138944578 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560fd02bcbb0 0x560fd02bfc20 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7f0b34097a60 tx=0x7f0b3409af20 comp rx=0 tx=0).stop 2024-03-28T18:09:38.589 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:38.588+0000 7f0b51c66740 1 -- 172.21.15.22:0/138944578 shutdown_connections 2024-03-28T18:09:38.589 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:38.588+0000 7f0b51c66740 1 --2- 172.21.15.22:0/138944578 >> [v2:172.21.15.79:6838/2441542443,v1:172.21.15.79:6839/2441542443] conn(0x560fd0331f60 0x560fd0334380 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:09:38.589 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:38.588+0000 7f0b51c66740 1 --2- 172.21.15.22:0/138944578 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x560fd02c5790 0x560fd0328e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:09:38.589 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:38.588+0000 7f0b51c66740 1 --2- 172.21.15.22:0/138944578 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560fd02bcbb0 0x560fd02bfc20 unknown :-1 s=CLOSED pgs=620 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:09:38.589 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:38.588+0000 7f0b51c66740 1 --2- 172.21.15.22:0/138944578 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560fd02bc290 0x560fd02bf6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:09:38.589 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:38.588+0000 7f0b51c66740 1 -- 172.21.15.22:0/138944578 >> 172.21.15.22:0/138944578 conn(0x560fd02b69c0 msgr2=0x560fd02b5a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:09:38.589 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:38.588+0000 7f0b32000640 0 client.16191 ms_handle_reset on v2:172.21.15.79:6838/2441542443 2024-03-28T18:09:38.589 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:38.588+0000 7f0b51c66740 1 -- 172.21.15.22:0/138944578 shutdown_connections 2024-03-28T18:09:38.589 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:38.588+0000 7f0b51c66740 1 -- 172.21.15.22:0/138944578 wait complete. 2024-03-28T18:09:38.589 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:38.588+0000 7f0b51c66740 20 client.16191 trim_cache size 0 max 16384 2024-03-28T18:09:38.590 INFO:teuthology.orchestra.run.smithi022.stdout:{ 2024-03-28T18:09:38.590 INFO:teuthology.orchestra.run.smithi022.stdout: "return_code": 0, 2024-03-28T18:09:38.590 INFO:teuthology.orchestra.run.smithi022.stdout: "scrub_tag": "9e8f4632-ca47-4611-891b-c1d73717ee96", 2024-03-28T18:09:38.590 INFO:teuthology.orchestra.run.smithi022.stdout: "mode": "asynchronous" 2024-03-28T18:09:38.590 INFO:teuthology.orchestra.run.smithi022.stdout:} 2024-03-28T18:09:38.591 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:38.589+0000 7f0b4da00640 1 -- 172.21.15.22:0/1259619197 >> [v2:172.21.15.22:6800/1357365082,v1:172.21.15.22:6801/1357365082] conn(0x7f0b2c0a2970 msgr2=0x7f0b2c0a4e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:09:38.591 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:38.589+0000 7f0b4da00640 1 --2- 172.21.15.22:0/1259619197 >> [v2:172.21.15.22:6800/1357365082,v1:172.21.15.22:6801/1357365082] conn(0x7f0b2c0a2970 0x7f0b2c0a4e30 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x560fd02bbd60 tx=0x7f0b38005f90 comp rx=0 tx=0).stop 2024-03-28T18:09:38.591 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:38.589+0000 7f0b4da00640 1 -- 172.21.15.22:0/1259619197 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0b481030b0 msgr2=0x7f0b4819a4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:09:38.591 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:38.589+0000 7f0b4da00640 1 --2- 172.21.15.22:0/1259619197 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0b481030b0 0x7f0b4819a4c0 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7f0b3400d950 tx=0x7f0b3400de10 comp rx=0 tx=0).stop 2024-03-28T18:09:38.591 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:38.589+0000 7f0b4da00640 1 -- 172.21.15.22:0/1259619197 shutdown_connections 2024-03-28T18:09:38.591 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:38.589+0000 7f0b4da00640 1 --2- 172.21.15.22:0/1259619197 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0b4810bb60 0x7f0b4819e840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:09:38.591 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:38.589+0000 7f0b4da00640 1 --2- 172.21.15.22:0/1259619197 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0b481030b0 0x7f0b4819a4c0 unknown :-1 s=CLOSED pgs=617 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:09:38.591 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:38.589+0000 7f0b4da00640 1 --2- 172.21.15.22:0/1259619197 >> [v2:172.21.15.22:6800/1357365082,v1:172.21.15.22:6801/1357365082] conn(0x7f0b2c0a2970 0x7f0b2c0a4e30 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:09:38.591 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:38.589+0000 7f0b4da00640 1 --2- 172.21.15.22:0/1259619197 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f0b48102790 0x7f0b48199f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:09:38.591 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:38.589+0000 7f0b4da00640 1 -- 172.21.15.22:0/1259619197 >> 172.21.15.22:0/1259619197 conn(0x7f0b480fbef0 msgr2=0x7f0b4810cdd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:09:38.591 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:38.589+0000 7f0b4da00640 1 -- 172.21.15.22:0/1259619197 shutdown_connections 2024-03-28T18:09:38.591 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:38.589+0000 7f0b4da00640 1 -- 172.21.15.22:0/1259619197 wait complete. 2024-03-28T18:09:38.593 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/media/sunxi-ir.txt 2024-03-28T18:09:38.608 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-28T18:09:38.608 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-28T18:09:38.609 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/media/ti-am437x-vpfe.txt 2024-03-28T18:09:38.623 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/media/video-interfaces.txt 2024-03-28T18:09:38.637 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/memory-controllers/ 2024-03-28T18:09:38.652 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/memory-controllers/fsl/ 2024-03-28T18:09:38.656 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/memory-controllers/fsl/ifc.txt 2024-03-28T18:09:38.663 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/memory-controllers/mvebu-devbus.txt 2024-03-28T18:09:38.687 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/memory-controllers/mvebu-sdram-controller.txt 2024-03-28T18:09:38.693 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:09:38 smithi022 ceph-mon[31985]: pgmap v65: 97 pgs: 97 active+clean; 91 MiB data, 694 MiB used, 1.0 TiB / 1.0 TiB avail; 223 KiB/s rd, 408 KiB/s wr, 106 op/s 2024-03-28T18:09:38.705 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/memory-controllers/nvidia,tegra-mc.txt 2024-03-28T18:09:38.718 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/memory-controllers/renesas-memory-controllers.txt 2024-03-28T18:09:38.732 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/memory-controllers/synopsys.txt 2024-03-28T18:09:38.748 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/memory-controllers/ti-aemif.txt 2024-03-28T18:09:38.763 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/memory-controllers/ti/ 2024-03-28T18:09:38.773 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/memory-controllers/ti/emif.txt 2024-03-28T18:09:38.782 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/metag/ 2024-03-28T18:09:38.805 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/metag/meta-intc.txt 2024-03-28T18:09:38.814 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/metag/meta.txt 2024-03-28T18:09:38.831 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/metag/pdc-intc.txt 2024-03-28T18:09:38.845 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/mfd/ 2024-03-28T18:09:38.854 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:38.854+0000 7ff62aa00640 1 -- 172.21.15.22:0/2837121660 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff62410cf80 msgr2=0x7ff624114b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:09:38.854 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:38.854+0000 7ff62aa00640 1 --2- 172.21.15.22:0/2837121660 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff62410cf80 0x7ff624114b50 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7ff61c009ca0 tx=0x7ff61c02c100 comp rx=0 tx=0).stop 2024-03-28T18:09:38.854 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:38.854+0000 7ff62aa00640 1 -- 172.21.15.22:0/2837121660 shutdown_connections 2024-03-28T18:09:38.854 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:38.854+0000 7ff62aa00640 1 --2- 172.21.15.22:0/2837121660 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff62410cf80 0x7ff624114b50 unknown :-1 s=CLOSED pgs=621 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:09:38.854 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:38.854+0000 7ff62aa00640 1 --2- 172.21.15.22:0/2837121660 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ff624108860 0x7ff62410c8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:09:38.855 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:38.854+0000 7ff62aa00640 1 --2- 172.21.15.22:0/2837121660 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff624107eb0 0x7ff624108290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:09:38.855 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:38.854+0000 7ff62aa00640 1 -- 172.21.15.22:0/2837121660 >> 172.21.15.22:0/2837121660 conn(0x7ff624075be0 msgr2=0x7ff624078000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:09:38.855 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:38.854+0000 7ff62aa00640 1 -- 172.21.15.22:0/2837121660 shutdown_connections 2024-03-28T18:09:38.855 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:38.854+0000 7ff62aa00640 1 -- 172.21.15.22:0/2837121660 wait complete. 2024-03-28T18:09:38.856 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:38.855+0000 7ff62aa00640 1 Processor -- start 2024-03-28T18:09:38.856 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:38.856+0000 7ff62aa00640 1 -- start start 2024-03-28T18:09:38.856 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:38.856+0000 7ff62aa00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff624107eb0 0x7ff62419e520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:09:38.856 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:38.856+0000 7ff62aa00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff62410cf80 0x7ff62419ea60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:09:38.856 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:38.856+0000 7ff62aa00640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ff6241a2de0 0x7ff6241a3290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:09:38.856 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:38.856+0000 7ff62aa00640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff624118020 con 0x7ff624107eb0 2024-03-28T18:09:38.857 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:38.856+0000 7ff62aa00640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff624117ea0 con 0x7ff6241a2de0 2024-03-28T18:09:38.857 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:38.856+0000 7ff62aa00640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6241181a0 con 0x7ff62410cf80 2024-03-28T18:09:38.857 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:38.856+0000 7ff623e00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff624107eb0 0x7ff62419e520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:09:38.857 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:38.856+0000 7ff623e00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff624107eb0 0x7ff62419e520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:46648/0 (socket says 172.21.15.22:46648) 2024-03-28T18:09:38.857 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:38.856+0000 7ff623e00640 1 -- 172.21.15.22:0/1597709685 learned_addr learned my addr 172.21.15.22:0/1597709685 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-28T18:09:38.857 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:38.857+0000 7ff61be00640 1 --2- 172.21.15.22:0/1597709685 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff62410cf80 0x7ff62419ea60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:09:38.857 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:38.857+0000 7ff628a00640 1 --2- 172.21.15.22:0/1597709685 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ff6241a2de0 0x7ff6241a3290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:09:38.858 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:38.857+0000 7ff623e00640 1 -- 172.21.15.22:0/1597709685 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff62410cf80 msgr2=0x7ff62419ea60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:09:38.858 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:38.857+0000 7ff623e00640 1 --2- 172.21.15.22:0/1597709685 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff62410cf80 0x7ff62419ea60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:09:38.858 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:38.857+0000 7ff623e00640 1 -- 172.21.15.22:0/1597709685 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ff6241a2de0 msgr2=0x7ff6241a3290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:09:38.858 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:38.857+0000 7ff623e00640 1 --2- 172.21.15.22:0/1597709685 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ff6241a2de0 0x7ff6241a3290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:09:38.858 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:38.857+0000 7ff623e00640 1 -- 172.21.15.22:0/1597709685 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff6241a38d0 con 0x7ff624107eb0 2024-03-28T18:09:38.858 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:38.857+0000 7ff61be00640 1 --2- 172.21.15.22:0/1597709685 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff62410cf80 0x7ff62419ea60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-28T18:09:38.858 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:38.857+0000 7ff623e00640 1 --2- 172.21.15.22:0/1597709685 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff624107eb0 0x7ff62419e520 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7ff61400e9f0 tx=0x7ff61400edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:09:38.859 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:38.858+0000 7ff621600640 1 -- 172.21.15.22:0/1597709685 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff61400cd80 con 0x7ff624107eb0 2024-03-28T18:09:38.859 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:38.858+0000 7ff62aa00640 1 -- 172.21.15.22:0/1597709685 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff6241a3bc0 con 0x7ff624107eb0 2024-03-28T18:09:38.859 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:38.858+0000 7ff621600640 1 -- 172.21.15.22:0/1597709685 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7ff6140054f0 con 0x7ff624107eb0 2024-03-28T18:09:38.859 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:38.858+0000 7ff621600640 1 -- 172.21.15.22:0/1597709685 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff614010690 con 0x7ff624107eb0 2024-03-28T18:09:38.859 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:38.858+0000 7ff62aa00640 1 -- 172.21.15.22:0/1597709685 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff6241ab360 con 0x7ff624107eb0 2024-03-28T18:09:38.861 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:38.862+0000 7ff621600640 1 -- 172.21.15.22:0/1597709685 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 26) v1 ==== 137421+0+0 (secure 0 0 0) 0x7ff614004f20 con 0x7ff624107eb0 2024-03-28T18:09:38.861 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:38.862+0000 7ff62d79c740 1 Processor -- start 2024-03-28T18:09:38.862 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:38.862+0000 7ff62d79c740 1 -- start start 2024-03-28T18:09:38.862 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/mfd/88pm860x.txt 2024-03-28T18:09:38.863 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:38.862+0000 7ff62d79c740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55609038dc00 0x55609038c950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:09:38.863 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:38.863+0000 7ff62d79c740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55609038cf20 0x55609038d3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:09:38.863 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:38.863+0000 7ff62d79c740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556090396290 0x556090398680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:09:38.863 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:38.863+0000 7ff62d79c740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x5560902f1250 con 0x556090396290 2024-03-28T18:09:38.863 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:38.863+0000 7ff62d79c740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x556090299a30 con 0x55609038dc00 2024-03-28T18:09:38.863 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:38.863+0000 7ff62d79c740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x5560902ed2f0 con 0x55609038cf20 2024-03-28T18:09:38.863 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:38.863+0000 7ff61be00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556090396290 0x556090398680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:09:38.863 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:38.863+0000 7ff61be00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556090396290 0x556090398680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:46664/0 (socket says 172.21.15.22:46664) 2024-03-28T18:09:38.863 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:38.863+0000 7ff61be00640 1 -- 172.21.15.22:0/2555528395 learned_addr learned my addr 172.21.15.22:0/2555528395 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-28T18:09:38.863 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:38.863+0000 7ff628a00640 1 --2- 172.21.15.22:0/2555528395 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55609038dc00 0x55609038c950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:09:38.863 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:38.863+0000 7ff623e00640 1 --2- 172.21.15.22:0/2555528395 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55609038cf20 0x55609038d3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:09:38.865 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:38.864+0000 7ff61be00640 1 -- 172.21.15.22:0/2555528395 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55609038cf20 msgr2=0x55609038d3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T18:09:38.865 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:38.864+0000 7ff61be00640 1 --2- 172.21.15.22:0/2555528395 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55609038cf20 0x55609038d3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:09:38.865 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:38.864+0000 7ff61be00640 1 -- 172.21.15.22:0/2555528395 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55609038dc00 msgr2=0x55609038c950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T18:09:38.865 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:38.864+0000 7ff61be00640 1 --2- 172.21.15.22:0/2555528395 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55609038dc00 0x55609038c950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:09:38.865 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:38.864+0000 7ff61be00640 1 -- 172.21.15.22:0/2555528395 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556090284690 con 0x556090396290 2024-03-28T18:09:38.865 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:38.864+0000 7ff628a00640 1 --2- 172.21.15.22:0/2555528395 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55609038dc00 0x55609038c950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-28T18:09:38.865 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:38.864+0000 7ff621600640 1 --2- 172.21.15.22:0/1597709685 >> [v2:172.21.15.22:6800/1357365082,v1:172.21.15.22:6801/1357365082] conn(0x7ff6080a2970 0x7ff6080a4e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:09:38.865 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:38.864+0000 7ff61be00640 1 --2- 172.21.15.22:0/2555528395 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556090396290 0x556090398680 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7ff60c009c30 tx=0x7ff60c0117b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d8c1151c75a11e90 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:09:38.865 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:38.864+0000 7ff621600640 1 -- 172.21.15.22:0/1597709685 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 8411+0+0 (secure 0 0 0) 0x7ff6140cf2f0 con 0x7ff624107eb0 2024-03-28T18:09:38.865 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:38.864+0000 7ff628a00640 1 --2- 172.21.15.22:0/1597709685 >> [v2:172.21.15.22:6800/1357365082,v1:172.21.15.22:6801/1357365082] conn(0x7ff6080a2970 0x7ff6080a4e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:09:38.865 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:38.864+0000 7ff619200640 1 -- 172.21.15.22:0/2555528395 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff60c004020 con 0x556090396290 2024-03-28T18:09:38.865 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:38.864+0000 7ff619200640 1 -- 172.21.15.22:0/2555528395 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7ff60c0041c0 con 0x556090396290 2024-03-28T18:09:38.865 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:38.865+0000 7ff619200640 1 -- 172.21.15.22:0/2555528395 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff60c0126f0 con 0x556090396290 2024-03-28T18:09:38.866 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:38.865+0000 7ff62d79c740 1 -- 172.21.15.22:0/2555528395 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556090396290 msgr2=0x556090398680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:09:38.866 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:38.865+0000 7ff628a00640 1 --2- 172.21.15.22:0/1597709685 >> [v2:172.21.15.22:6800/1357365082,v1:172.21.15.22:6801/1357365082] conn(0x7ff6080a2970 0x7ff6080a4e30 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7ff61c004af0 tx=0x7ff61c005c50 comp rx=0 tx=0).ready entity=mgr.15522 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:09:38.867 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:38.865+0000 7ff62d79c740 1 --2- 172.21.15.22:0/2555528395 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556090396290 0x556090398680 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7ff60c009c30 tx=0x7ff60c0117b0 comp rx=0 tx=0).stop 2024-03-28T18:09:38.867 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:38.866+0000 7ff62d79c740 1 -- 172.21.15.22:0/2555528395 shutdown_connections 2024-03-28T18:09:38.867 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:38.866+0000 7ff62d79c740 1 --2- 172.21.15.22:0/2555528395 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556090396290 0x556090398680 unknown :-1 s=CLOSED pgs=623 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:09:38.867 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:38.866+0000 7ff62d79c740 1 --2- 172.21.15.22:0/2555528395 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55609038cf20 0x55609038d3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:09:38.867 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:38.866+0000 7ff62d79c740 1 --2- 172.21.15.22:0/2555528395 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55609038dc00 0x55609038c950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:09:38.867 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:38.866+0000 7ff62d79c740 1 -- 172.21.15.22:0/2555528395 >> 172.21.15.22:0/2555528395 conn(0x556090354200 msgr2=0x556090387970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:09:38.867 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:38.866+0000 7ff62d79c740 1 -- 172.21.15.22:0/2555528395 shutdown_connections 2024-03-28T18:09:38.867 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:38.866+0000 7ff62d79c740 1 -- 172.21.15.22:0/2555528395 wait complete. 2024-03-28T18:09:38.867 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:38.866+0000 7ff62d79c740 1 -- start start 2024-03-28T18:09:38.867 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:38.867+0000 7ff62d79c740 1 Processor -- start 2024-03-28T18:09:38.869 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:38.867+0000 7ff62d79c740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55609038cf20 0x556090395940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:09:38.869 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:38.867+0000 7ff62d79c740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55609038dc00 0x556090392cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:09:38.869 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:38.867+0000 7ff62d79c740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556090396290 0x556090393210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:09:38.869 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:38.867+0000 7ff62d79c740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x556090399100 con 0x556090396290 2024-03-28T18:09:38.869 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:38.867+0000 7ff62d79c740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x5560902ace80 con 0x55609038cf20 2024-03-28T18:09:38.869 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:38.867+0000 7ff62d79c740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x556090399280 con 0x55609038dc00 2024-03-28T18:09:38.869 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:38.867+0000 7ff62d79c740 10 client.0 _ll_register_callbacks cb 0x5560902bf130 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-28T18:09:38.869 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:38.868+0000 7ff628a00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556090396290 0x556090393210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:09:38.869 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:38.868+0000 7ff628a00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556090396290 0x556090393210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:46676/0 (socket says 172.21.15.22:46676) 2024-03-28T18:09:38.869 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:38.868+0000 7ff628a00640 1 -- 172.21.15.22:0/1608248811 learned_addr learned my addr 172.21.15.22:0/1608248811 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-28T18:09:38.870 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:38.868+0000 7ff623e00640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55609038cf20 0x556090395940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:09:38.870 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:38.868+0000 7ff61be00640 1 --2- 172.21.15.22:0/1608248811 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55609038dc00 0x556090392cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:09:38.870 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:38.868+0000 7ff628a00640 1 -- 172.21.15.22:0/1608248811 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55609038dc00 msgr2=0x556090392cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T18:09:38.870 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:38.868+0000 7ff628a00640 1 --2- 172.21.15.22:0/1608248811 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55609038dc00 0x556090392cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:09:38.870 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:38.868+0000 7ff628a00640 1 -- 172.21.15.22:0/1608248811 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55609038cf20 msgr2=0x556090395940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T18:09:38.870 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:38.868+0000 7ff628a00640 1 --2- 172.21.15.22:0/1608248811 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55609038cf20 0x556090395940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:09:38.870 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:38.868+0000 7ff628a00640 1 -- 172.21.15.22:0/1608248811 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5560900cf7d0 con 0x556090396290 2024-03-28T18:09:38.870 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:38.868+0000 7ff623e00640 1 --2- 172.21.15.22:0/1608248811 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55609038cf20 0x556090395940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-28T18:09:38.870 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:38.869+0000 7ff628a00640 1 --2- 172.21.15.22:0/1608248811 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556090396290 0x556090393210 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7ff61c009ca0 tx=0x7ff61c02c100 comp rx=0 tx=0).ready entity=mon.0 client_cookie=12112a51819e443e server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:09:38.870 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:38.869+0000 7ff611600640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-03-28T18:09:38.870 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/mfd/ab8500.txt 2024-03-28T18:09:38.872 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:38.869+0000 7ff611600640 1 -- 172.21.15.22:0/1608248811 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff61c039040 con 0x556090396290 2024-03-28T18:09:38.872 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:38.869+0000 7ff611600640 1 -- 172.21.15.22:0/1608248811 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 8411+0+0 (secure 0 0 0) 0x7ff61c004250 con 0x556090396290 2024-03-28T18:09:38.872 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:38.869+0000 7ff62d79c740 1 -- 172.21.15.22:0/1608248811 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7ff60c0041c0 con 0x556090396290 2024-03-28T18:09:38.872 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:38.870+0000 7ff611600640 1 -- 172.21.15.22:0/1608248811 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff61c004450 con 0x556090396290 2024-03-28T18:09:38.872 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:38.870+0000 7ff62d79c740 10 client.16203 fetch_fsmap learned FSMap version 27 2024-03-28T18:09:38.872 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:38.870+0000 7ff62d79c740 1 -- 172.21.15.22:0/1608248811 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x5560900053f0 con 0x556090396290 2024-03-28T18:09:38.872 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:38.870+0000 7ff611600640 1 -- 172.21.15.22:0/1608248811 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7ff60c0041c0 con 0x556090396290 2024-03-28T18:09:38.872 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:38.871+0000 7ff611600640 1 -- 172.21.15.22:0/1608248811 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 27) v1 ==== 7934+0+0 (secure 0 0 0) 0x7ff61c03f040 con 0x556090396290 2024-03-28T18:09:38.872 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:38.871+0000 7ff62d79c740 10 client.16203 fetch_fsmap finished waiting for FSMap version 27 2024-03-28T18:09:38.872 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:38.871+0000 7ff62d79c740 10 client.16203 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-28T18:09:38.872 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:38.871+0000 7ff62d79c740 20 client.16203 populate_metadata read hostname 'smithi022' 2024-03-28T18:09:38.872 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:38.871+0000 7ff62d79c740 1 --2- 172.21.15.22:0/1608248811 >> [v2:172.21.15.79:6838/2441542443,v1:172.21.15.79:6839/2441542443] conn(0x5560903a61c0 0x5560903a8600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:09:38.873 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:38.871+0000 7ff62d79c740 4 client.16203 mds_command: new command op to 34553 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-28T18:09:38.873 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:38.871+0000 7ff62d79c740 1 -- 172.21.15.22:0/1608248811 --> [v2:172.21.15.79:6838/2441542443,v1:172.21.15.79:6839/2441542443] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5560902fe790 con 0x5560903a61c0 2024-03-28T18:09:38.873 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:38.872+0000 7ff623e00640 1 --2- 172.21.15.22:0/1608248811 >> [v2:172.21.15.79:6838/2441542443,v1:172.21.15.79:6839/2441542443] conn(0x5560903a61c0 0x5560903a8600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:09:38.873 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:38.873+0000 7ff623e00640 1 --2- 172.21.15.22:0/1608248811 >> [v2:172.21.15.79:6838/2441542443,v1:172.21.15.79:6839/2441542443] conn(0x5560903a61c0 0x5560903a8600 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1bcb24187c3156be server_cookie=388ed9c572bbc0e7 in_seq=0 out_seq=0 2024-03-28T18:09:38.873 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:38.873+0000 7ff611600640 10 client.16203 ms_handle_connect on v2:172.21.15.79:6838/2441542443 2024-03-28T18:09:38.875 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:38.875+0000 7ff611600640 1 -- 172.21.15.22:0/1608248811 <== mds.0 v2:172.21.15.79:6838/2441542443 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x5560902fe790 con 0x5560903a61c0 2024-03-28T18:09:38.875 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:38.875+0000 7ff611600640 10 client.16203 handle_command_reply: tid=0 2024-03-28T18:09:38.875 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:38.875+0000 7ff62d79c740 1 client.16203 shutdown 2024-03-28T18:09:38.875 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:38.875+0000 7ff62d79c740 1 -- 172.21.15.22:0/1608248811 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556090396290 msgr2=0x556090393210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:09:38.875 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:38.875+0000 7ff62d79c740 1 --2- 172.21.15.22:0/1608248811 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556090396290 0x556090393210 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7ff61c009ca0 tx=0x7ff61c02c100 comp rx=0 tx=0).stop 2024-03-28T18:09:38.876 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:38.875+0000 7ff62d79c740 1 -- 172.21.15.22:0/1608248811 shutdown_connections 2024-03-28T18:09:38.876 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:38.875+0000 7ff62d79c740 1 --2- 172.21.15.22:0/1608248811 >> [v2:172.21.15.79:6838/2441542443,v1:172.21.15.79:6839/2441542443] conn(0x5560903a61c0 0x5560903a8600 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:09:38.876 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:38.875+0000 7ff62d79c740 1 --2- 172.21.15.22:0/1608248811 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556090396290 0x556090393210 unknown :-1 s=CLOSED pgs=624 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:09:38.876 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:38.875+0000 7ff62d79c740 1 --2- 172.21.15.22:0/1608248811 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55609038dc00 0x556090392cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:09:38.876 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:38.875+0000 7ff62d79c740 1 --2- 172.21.15.22:0/1608248811 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55609038cf20 0x556090395940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:09:38.876 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:38.875+0000 7ff62d79c740 1 -- 172.21.15.22:0/1608248811 >> 172.21.15.22:0/1608248811 conn(0x556090354200 msgr2=0x5560903896c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:09:38.876 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:38.876+0000 7ff611600640 0 client.16203 ms_handle_reset on v2:172.21.15.79:6838/2441542443 2024-03-28T18:09:38.876 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:38.876+0000 7ff62d79c740 1 -- 172.21.15.22:0/1608248811 shutdown_connections 2024-03-28T18:09:38.876 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:38.876+0000 7ff62d79c740 1 -- 172.21.15.22:0/1608248811 wait complete. 2024-03-28T18:09:38.876 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:38.876+0000 7ff62d79c740 20 client.16203 trim_cache size 0 max 16384 2024-03-28T18:09:38.882 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/mfd/arizona.txt 2024-03-28T18:09:38.892 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/mfd/as3711.txt 2024-03-28T18:09:38.897 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:38.897+0000 7ff62d79c740 1 Processor -- start 2024-03-28T18:09:38.897 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:38.897+0000 7ff62d79c740 1 -- start start 2024-03-28T18:09:38.898 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:38.898+0000 7ff62d79c740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55609038cf20 0x556090395400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:09:38.898 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:38.898+0000 7ff62d79c740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55609038dc00 0x556090395940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:09:38.898 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:38.898+0000 7ff62d79c740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x556090396290 0x556090397680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:09:38.898 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:38.898+0000 7ff62d79c740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6241b35c0 con 0x55609038dc00 2024-03-28T18:09:38.898 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:38.898+0000 7ff62d79c740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6241b4f40 con 0x556090396290 2024-03-28T18:09:38.898 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:38.898+0000 7ff62d79c740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6241b3c20 con 0x55609038cf20 2024-03-28T18:09:38.898 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:38.898+0000 7ff61be00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55609038dc00 0x556090395940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:09:38.898 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:38.898+0000 7ff61be00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55609038dc00 0x556090395940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:46690/0 (socket says 172.21.15.22:46690) 2024-03-28T18:09:38.898 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:38.898+0000 7ff61be00640 1 -- 172.21.15.22:0/531365319 learned_addr learned my addr 172.21.15.22:0/531365319 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-28T18:09:38.898 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:38.898+0000 7ff628a00640 1 --2- 172.21.15.22:0/531365319 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x556090396290 0x556090397680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:09:38.899 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:38.899+0000 7ff623e00640 1 --2- 172.21.15.22:0/531365319 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55609038cf20 0x556090395400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:09:38.899 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:38.899+0000 7ff61be00640 1 -- 172.21.15.22:0/531365319 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55609038cf20 msgr2=0x556090395400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T18:09:38.900 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:38.899+0000 7ff61be00640 1 --2- 172.21.15.22:0/531365319 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55609038cf20 0x556090395400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:09:38.900 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:38.899+0000 7ff61be00640 1 -- 172.21.15.22:0/531365319 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x556090396290 msgr2=0x556090397680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T18:09:38.900 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:38.899+0000 7ff61be00640 1 --2- 172.21.15.22:0/531365319 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x556090396290 0x556090397680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:09:38.900 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:38.899+0000 7ff61be00640 1 -- 172.21.15.22:0/531365319 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55608ffc50f0 con 0x55609038dc00 2024-03-28T18:09:38.900 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:38.899+0000 7ff61be00640 1 --2- 172.21.15.22:0/531365319 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55609038dc00 0x556090395940 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7ff60c009c30 tx=0x7ff60c0095b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5cf72162d586b593 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:09:38.900 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:38.900+0000 7ff613e00640 1 -- 172.21.15.22:0/531365319 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff60c026070 con 0x55609038dc00 2024-03-28T18:09:38.900 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:38.900+0000 7ff613e00640 1 -- 172.21.15.22:0/531365319 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7ff60c005500 con 0x55609038dc00 2024-03-28T18:09:38.900 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:38.900+0000 7ff613e00640 1 -- 172.21.15.22:0/531365319 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff60c004ce0 con 0x55609038dc00 2024-03-28T18:09:38.901 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:38.901+0000 7ff62d79c740 1 -- 172.21.15.22:0/531365319 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55609038dc00 msgr2=0x556090395940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:09:38.901 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:38.901+0000 7ff62d79c740 1 --2- 172.21.15.22:0/531365319 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55609038dc00 0x556090395940 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7ff60c009c30 tx=0x7ff60c0095b0 comp rx=0 tx=0).stop 2024-03-28T18:09:38.901 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:38.901+0000 7ff62d79c740 1 -- 172.21.15.22:0/531365319 shutdown_connections 2024-03-28T18:09:38.901 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:38.901+0000 7ff62d79c740 1 --2- 172.21.15.22:0/531365319 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x556090396290 0x556090397680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:09:38.901 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:38.901+0000 7ff62d79c740 1 --2- 172.21.15.22:0/531365319 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55609038dc00 0x556090395940 unknown :-1 s=CLOSED pgs=625 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:09:38.901 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:38.901+0000 7ff62d79c740 1 --2- 172.21.15.22:0/531365319 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55609038cf20 0x556090395400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:09:38.901 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:38.901+0000 7ff62d79c740 1 -- 172.21.15.22:0/531365319 >> 172.21.15.22:0/531365319 conn(0x556090354200 msgr2=0x5560903915c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:09:38.901 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:38.901+0000 7ff62d79c740 1 -- 172.21.15.22:0/531365319 shutdown_connections 2024-03-28T18:09:38.902 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:38.901+0000 7ff62d79c740 1 -- 172.21.15.22:0/531365319 wait complete. 2024-03-28T18:09:38.902 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:38.901+0000 7ff62d79c740 1 -- start start 2024-03-28T18:09:38.902 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:38.902+0000 7ff62d79c740 1 Processor -- start 2024-03-28T18:09:38.902 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:38.902+0000 7ff62d79c740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55609038cf20 0x55609038f540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:09:38.902 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:38.902+0000 7ff62d79c740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55609038dc00 0x55609038fa80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:09:38.902 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:38.902+0000 7ff62d79c740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556090396290 0x55609038ffc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:09:38.902 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:38.902+0000 7ff62d79c740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6241b48e0 con 0x55609038cf20 2024-03-28T18:09:38.903 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:38.902+0000 7ff62d79c740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6241b4280 con 0x55609038dc00 2024-03-28T18:09:38.903 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:38.902+0000 7ff62d79c740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6241b1780 con 0x556090396290 2024-03-28T18:09:38.903 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:38.902+0000 7ff62d79c740 10 client.0 _ll_register_callbacks cb 0x7ff614005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-28T18:09:38.903 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/mfd/as3722.txt 2024-03-28T18:09:38.904 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:38.903+0000 7ff61be00640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55609038dc00 0x55609038fa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:09:38.904 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:38.903+0000 7ff61be00640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55609038dc00 0x55609038fa80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.22:41118/0 (socket says 172.21.15.22:41118) 2024-03-28T18:09:38.904 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:38.903+0000 7ff61be00640 1 -- 172.21.15.22:0/2718314261 learned_addr learned my addr 172.21.15.22:0/2718314261 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-28T18:09:38.904 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:38.903+0000 7ff623e00640 1 --2- 172.21.15.22:0/2718314261 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55609038cf20 0x55609038f540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:09:38.904 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:38.903+0000 7ff628a00640 1 --2- 172.21.15.22:0/2718314261 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556090396290 0x55609038ffc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:09:38.904 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:38.903+0000 7ff61be00640 1 -- 172.21.15.22:0/2718314261 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556090396290 msgr2=0x55609038ffc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T18:09:38.904 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:38.904+0000 7ff61be00640 1 --2- 172.21.15.22:0/2718314261 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556090396290 0x55609038ffc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:09:38.904 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:38.904+0000 7ff61be00640 1 -- 172.21.15.22:0/2718314261 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55609038cf20 msgr2=0x55609038f540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T18:09:38.904 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:38.904+0000 7ff61be00640 1 --2- 172.21.15.22:0/2718314261 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55609038cf20 0x55609038f540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:09:38.905 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:38.904+0000 7ff61be00640 1 -- 172.21.15.22:0/2718314261 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x556090400b10 con 0x55609038dc00 2024-03-28T18:09:38.905 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:38.904+0000 7ff623e00640 1 --2- 172.21.15.22:0/2718314261 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55609038cf20 0x55609038f540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-28T18:09:38.905 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:38.904+0000 7ff61be00640 1 --2- 172.21.15.22:0/2718314261 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55609038dc00 0x55609038fa80 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7ff60c0117f0 tx=0x7ff60c0056b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3c3b22b02bd9199a server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:09:38.905 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:38.904+0000 7ff611600640 10 client.0 ms_handle_connect on v2:172.21.15.79:3300/0 2024-03-28T18:09:38.906 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:38.905+0000 7ff611600640 1 -- 172.21.15.22:0/2718314261 <== mon.2 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff60c026030 con 0x55609038dc00 2024-03-28T18:09:38.906 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:38.905+0000 7ff611600640 1 -- 172.21.15.22:0/2718314261 <== mon.2 v2:172.21.15.79:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 8411+0+0 (secure 0 0 0) 0x7ff60c004e70 con 0x55609038dc00 2024-03-28T18:09:38.906 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:38.905+0000 7ff62d79c740 1 -- 172.21.15.22:0/2718314261 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7ff60c005500 con 0x55609038dc00 2024-03-28T18:09:38.906 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:38.905+0000 7ff611600640 1 -- 172.21.15.22:0/2718314261 <== mon.2 v2:172.21.15.79:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff60c039050 con 0x55609038dc00 2024-03-28T18:09:38.906 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:38.905+0000 7ff62d79c740 10 client.36038 fetch_fsmap learned FSMap version 27 2024-03-28T18:09:38.906 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:38.905+0000 7ff62d79c740 1 -- 172.21.15.22:0/2718314261 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x556090401de0 con 0x55609038dc00 2024-03-28T18:09:38.907 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:38.906+0000 7ff611600640 1 -- 172.21.15.22:0/2718314261 <== mon.2 v2:172.21.15.79:3300/0 4 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7ff60c005500 con 0x55609038dc00 2024-03-28T18:09:38.907 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:38.906+0000 7ff611600640 1 -- 172.21.15.22:0/2718314261 <== mon.2 v2:172.21.15.79:3300/0 5 ==== fsmap(e 27) v1 ==== 7934+0+0 (secure 0 0 0) 0x7ff60c02a410 con 0x55609038dc00 2024-03-28T18:09:38.907 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:38.906+0000 7ff62d79c740 10 client.36038 fetch_fsmap finished waiting for FSMap version 27 2024-03-28T18:09:38.907 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:38.906+0000 7ff62d79c740 10 client.36038 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-28T18:09:38.907 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:38.906+0000 7ff62d79c740 20 client.36038 populate_metadata read hostname 'smithi022' 2024-03-28T18:09:38.907 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:38.906+0000 7ff62d79c740 1 --2- 172.21.15.22:0/2718314261 >> [v2:172.21.15.79:6838/2441542443,v1:172.21.15.79:6839/2441542443] conn(0x5560904022c0 0x5560904046e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:09:38.907 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:38.906+0000 7ff62d79c740 4 client.36038 mds_command: new command op to 34553 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-28T18:09:38.907 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:38.906+0000 7ff62d79c740 1 -- 172.21.15.22:0/2718314261 --> [v2:172.21.15.79:6838/2441542443,v1:172.21.15.79:6839/2441542443] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x5560902f8ae0 con 0x5560904022c0 2024-03-28T18:09:38.907 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:38.906+0000 7ff623e00640 1 --2- 172.21.15.22:0/2718314261 >> [v2:172.21.15.79:6838/2441542443,v1:172.21.15.79:6839/2441542443] conn(0x5560904022c0 0x5560904046e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:09:38.908 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:38.907+0000 7ff623e00640 1 --2- 172.21.15.22:0/2718314261 >> [v2:172.21.15.79:6838/2441542443,v1:172.21.15.79:6839/2441542443] conn(0x5560904022c0 0x5560904046e0 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ce51a232e99c5fdd server_cookie=7309c32f3fb5787a in_seq=0 out_seq=0 2024-03-28T18:09:38.908 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:38.907+0000 7ff611600640 10 client.36038 ms_handle_connect on v2:172.21.15.79:6838/2441542443 2024-03-28T18:09:38.908 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:38.908+0000 7ff611600640 1 -- 172.21.15.22:0/2718314261 <== mds.0 v2:172.21.15.79:6838/2441542443 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+267 (crc 0 0 0) 0x5560902f8ae0 con 0x5560904022c0 2024-03-28T18:09:38.909 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:38.908+0000 7ff611600640 10 client.36038 handle_command_reply: tid=0 2024-03-28T18:09:38.909 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:38.908+0000 7ff62d79c740 1 client.36038 shutdown 2024-03-28T18:09:38.909 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:38.908+0000 7ff62d79c740 1 -- 172.21.15.22:0/2718314261 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55609038dc00 msgr2=0x55609038fa80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:09:38.909 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:38.908+0000 7ff62d79c740 1 --2- 172.21.15.22:0/2718314261 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55609038dc00 0x55609038fa80 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7ff60c0117f0 tx=0x7ff60c0056b0 comp rx=0 tx=0).stop 2024-03-28T18:09:38.909 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:38.908+0000 7ff62d79c740 1 -- 172.21.15.22:0/2718314261 shutdown_connections 2024-03-28T18:09:38.909 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:38.908+0000 7ff62d79c740 1 --2- 172.21.15.22:0/2718314261 >> [v2:172.21.15.79:6838/2441542443,v1:172.21.15.79:6839/2441542443] conn(0x5560904022c0 0x5560904046e0 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:09:38.909 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:38.908+0000 7ff62d79c740 1 --2- 172.21.15.22:0/2718314261 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556090396290 0x55609038ffc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:09:38.909 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:38.908+0000 7ff62d79c740 1 --2- 172.21.15.22:0/2718314261 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55609038dc00 0x55609038fa80 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:09:38.909 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:38.908+0000 7ff62d79c740 1 --2- 172.21.15.22:0/2718314261 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55609038cf20 0x55609038f540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:09:38.909 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:38.908+0000 7ff611600640 0 client.36038 ms_handle_reset on v2:172.21.15.79:6838/2441542443 2024-03-28T18:09:38.909 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:38.908+0000 7ff62d79c740 1 -- 172.21.15.22:0/2718314261 >> 172.21.15.22:0/2718314261 conn(0x556090354200 msgr2=0x556090391a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:09:38.909 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:38.908+0000 7ff62d79c740 1 -- 172.21.15.22:0/2718314261 shutdown_connections 2024-03-28T18:09:38.909 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:38.908+0000 7ff62d79c740 1 -- 172.21.15.22:0/2718314261 wait complete. 2024-03-28T18:09:38.909 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:38.908+0000 7ff62d79c740 20 client.36038 trim_cache size 0 max 16384 2024-03-28T18:09:38.910 INFO:teuthology.orchestra.run.smithi022.stdout:{ 2024-03-28T18:09:38.910 INFO:teuthology.orchestra.run.smithi022.stdout: "status": "scrub active (411 inodes in the stack)", 2024-03-28T18:09:38.910 INFO:teuthology.orchestra.run.smithi022.stdout: "scrubs": { 2024-03-28T18:09:38.910 INFO:teuthology.orchestra.run.smithi022.stdout: "9e8f4632-ca47-4611-891b-c1d73717ee96": { 2024-03-28T18:09:38.910 INFO:teuthology.orchestra.run.smithi022.stdout: "path": "/", 2024-03-28T18:09:38.910 INFO:teuthology.orchestra.run.smithi022.stdout: "tag": "9e8f4632-ca47-4611-891b-c1d73717ee96", 2024-03-28T18:09:38.910 INFO:teuthology.orchestra.run.smithi022.stdout: "options": "recursive,force" 2024-03-28T18:09:38.910 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-03-28T18:09:38.910 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-03-28T18:09:38.910 INFO:teuthology.orchestra.run.smithi022.stdout:} 2024-03-28T18:09:38.911 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:38.909+0000 7ff62aa00640 1 -- 172.21.15.22:0/1597709685 >> [v2:172.21.15.22:6800/1357365082,v1:172.21.15.22:6801/1357365082] conn(0x7ff6080a2970 msgr2=0x7ff6080a4e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:09:38.911 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:38.909+0000 7ff62aa00640 1 --2- 172.21.15.22:0/1597709685 >> [v2:172.21.15.22:6800/1357365082,v1:172.21.15.22:6801/1357365082] conn(0x7ff6080a2970 0x7ff6080a4e30 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7ff61c004af0 tx=0x7ff61c005c50 comp rx=0 tx=0).stop 2024-03-28T18:09:38.911 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:38.909+0000 7ff62aa00640 1 -- 172.21.15.22:0/1597709685 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff624107eb0 msgr2=0x7ff62419e520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:09:38.911 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:38.909+0000 7ff62aa00640 1 --2- 172.21.15.22:0/1597709685 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff624107eb0 0x7ff62419e520 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7ff61400e9f0 tx=0x7ff61400edf0 comp rx=0 tx=0).stop 2024-03-28T18:09:38.911 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:38.909+0000 7ff62aa00640 1 -- 172.21.15.22:0/1597709685 shutdown_connections 2024-03-28T18:09:38.911 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:38.909+0000 7ff62aa00640 1 --2- 172.21.15.22:0/1597709685 >> [v2:172.21.15.22:6800/1357365082,v1:172.21.15.22:6801/1357365082] conn(0x7ff6080a2970 0x7ff6080a4e30 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:09:38.912 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:38.909+0000 7ff62aa00640 1 --2- 172.21.15.22:0/1597709685 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ff6241a2de0 0x7ff6241a3290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:09:38.912 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:38.909+0000 7ff62aa00640 1 --2- 172.21.15.22:0/1597709685 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff62410cf80 0x7ff62419ea60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:09:38.912 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:38.909+0000 7ff62aa00640 1 --2- 172.21.15.22:0/1597709685 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff624107eb0 0x7ff62419e520 unknown :-1 s=CLOSED pgs=622 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:09:38.912 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:38.909+0000 7ff62aa00640 1 -- 172.21.15.22:0/1597709685 >> 172.21.15.22:0/1597709685 conn(0x7ff624075be0 msgr2=0x7ff624077ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:09:38.912 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:38.909+0000 7ff62aa00640 1 -- 172.21.15.22:0/1597709685 shutdown_connections 2024-03-28T18:09:38.912 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:09:38.909+0000 7ff62aa00640 1 -- 172.21.15.22:0/1597709685 wait complete. 2024-03-28T18:09:38.914 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/mfd/atmel-gpbr.txt 2024-03-28T18:09:38.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:09:38 smithi134 ceph-mon[35037]: pgmap v65: 97 pgs: 97 active+clean; 91 MiB data, 694 MiB used, 1.0 TiB / 1.0 TiB avail; 223 KiB/s rd, 408 KiB/s wr, 106 op/s 2024-03-28T18:09:38.926 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/mfd/atmel-hlcdc.txt 2024-03-28T18:09:38.929 INFO:tasks.cephfs.filesystem:scrub status for tag:9e8f4632-ca47-4611-891b-c1d73717ee96 - {'path': '/', 'tag': '9e8f4632-ca47-4611-891b-c1d73717ee96', 'options': 'recursive,force'} 2024-03-28T18:09:38.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:09:38 smithi079 ceph-mon[35312]: pgmap v65: 97 pgs: 97 active+clean; 91 MiB data, 694 MiB used, 1.0 TiB / 1.0 TiB avail; 223 KiB/s rd, 408 KiB/s wr, 106 op/s 2024-03-28T18:09:38.938 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/mfd/atmel-matrix.txt 2024-03-28T18:09:38.949 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/mfd/atmel-smc.txt 2024-03-28T18:09:38.961 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/mfd/bcm590xx.txt 2024-03-28T18:09:38.975 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/mfd/bfticu.txt 2024-03-28T18:09:38.989 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/mfd/cros-ec.txt 2024-03-28T18:09:39.004 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/mfd/da9052-i2c.txt 2024-03-28T18:09:39.019 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/mfd/da9055.txt 2024-03-28T18:09:39.034 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/mfd/da9063.txt 2024-03-28T18:09:39.049 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/mfd/hi6421.txt 2024-03-28T18:09:39.063 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/mfd/lp3943.txt 2024-03-28T18:09:39.079 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/mfd/max14577.txt 2024-03-28T18:09:39.096 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/mfd/max77686.txt 2024-03-28T18:09:39.111 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/mfd/max77693.txt 2024-03-28T18:09:39.128 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/mfd/max8925.txt 2024-03-28T18:09:39.143 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/mfd/max8998.txt 2024-03-28T18:09:39.160 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/mfd/mc13xxx.txt 2024-03-28T18:09:39.175 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/mfd/omap-usb-host.txt 2024-03-28T18:09:39.192 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/mfd/omap-usb-tll.txt 2024-03-28T18:09:39.207 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/mfd/palmas.txt 2024-03-28T18:09:39.223 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/mfd/qcom,spmi-pmic.txt 2024-03-28T18:09:39.241 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/mfd/qcom-pm8xxx.txt 2024-03-28T18:09:39.257 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/mfd/qcom-rpm.txt 2024-03-28T18:09:39.314 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/mfd/qriox.txt 2024-03-28T18:09:39.332 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/mfd/rk808.txt 2024-03-28T18:09:39.342 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/mfd/rn5t618.txt 2024-03-28T18:09:39.358 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/mfd/s2mpa01.txt 2024-03-28T18:09:39.373 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/mfd/s2mps11.txt 2024-03-28T18:09:39.388 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/mfd/stmpe.txt 2024-03-28T18:09:39.402 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/mfd/sun6i-prcm.txt 2024-03-28T18:09:39.418 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/mfd/syscon.txt 2024-03-28T18:09:39.433 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/mfd/tc3589x.txt 2024-03-28T18:09:39.448 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/mfd/ti-keystone-devctrl.txt 2024-03-28T18:09:39.465 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/mfd/tps6507x.txt 2024-03-28T18:09:39.481 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/mfd/tps65910.txt 2024-03-28T18:09:39.497 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/mfd/twl-familly.txt 2024-03-28T18:09:39.512 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/mfd/twl4030-audio.txt 2024-03-28T18:09:39.528 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/mfd/twl4030-power.txt 2024-03-28T18:09:39.546 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/mfd/twl6040.txt 2024-03-28T18:09:39.565 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/mipi/ 2024-03-28T18:09:39.584 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/mipi/dsi/ 2024-03-28T18:09:39.593 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/mipi/dsi/mipi-dsi-bus.txt 2024-03-28T18:09:39.602 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/mipi/nvidia,tegra114-mipi.txt 2024-03-28T18:09:39.632 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/mips/ 2024-03-28T18:09:39.714 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/mips/brcm/ 2024-03-28T18:09:39.723 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/mips/brcm/bcm3384-intc.txt 2024-03-28T18:09:39.733 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/mips/brcm/bmips.txt 2024-03-28T18:09:39.751 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/mips/brcm/cm-dsl.txt 2024-03-28T18:09:39.766 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/mips/brcm/usb.txt 2024-03-28T18:09:39.781 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/mips/cavium/ 2024-03-28T18:09:39.798 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/mips/cavium/bootbus.txt 2024-03-28T18:09:39.808 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/mips/cavium/cib.txt 2024-03-28T18:09:39.827 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/mips/cavium/ciu.txt 2024-03-28T18:09:39.845 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/mips/cavium/ciu2.txt 2024-03-28T18:09:39.860 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/mips/cavium/dma-engine.txt 2024-03-28T18:09:39.877 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/mips/cavium/uctl.txt 2024-03-28T18:09:39.894 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/mips/cpu_irq.txt 2024-03-28T18:09:39.917 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/mips/ralink.txt 2024-03-28T18:09:39.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:09:39 smithi134 ceph-mon[35037]: scrub queued for path: / 2024-03-28T18:09:39.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:09:39 smithi134 ceph-mon[35037]: scrub summary: idle+waiting paths [/] 2024-03-28T18:09:39.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:09:39 smithi134 ceph-mon[35037]: scrub summary: active paths [/] 2024-03-28T18:09:39.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:09:39 smithi134 ceph-mon[35037]: scrub summary: idle+waiting paths [/] 2024-03-28T18:09:39.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:09:39 smithi134 ceph-mon[35037]: scrub summary: idle 2024-03-28T18:09:39.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:09:39 smithi079 ceph-mon[35312]: scrub queued for path: / 2024-03-28T18:09:39.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:09:39 smithi079 ceph-mon[35312]: scrub summary: idle+waiting paths [/] 2024-03-28T18:09:39.933 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:09:39 smithi079 ceph-mon[35312]: scrub summary: active paths [/] 2024-03-28T18:09:39.933 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:09:39 smithi079 ceph-mon[35312]: scrub summary: idle+waiting paths [/] 2024-03-28T18:09:39.933 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:09:39 smithi079 ceph-mon[35312]: scrub summary: idle 2024-03-28T18:09:39.934 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/misc/ 2024-03-28T18:09:39.953 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/misc/allwinner,sunxi-sid.txt 2024-03-28T18:09:39.965 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/misc/arm-charlcd.txt 2024-03-28T18:09:39.985 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/misc/at25.txt 2024-03-28T18:09:40.004 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/misc/atmel-ssc.txt 2024-03-28T18:09:40.020 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/misc/bmp085.txt 2024-03-28T18:09:40.031 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:09:39 smithi022 ceph-mon[31985]: scrub queued for path: / 2024-03-28T18:09:40.032 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:09:39 smithi022 ceph-mon[31985]: scrub summary: idle+waiting paths [/] 2024-03-28T18:09:40.032 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:09:39 smithi022 ceph-mon[31985]: scrub summary: active paths [/] 2024-03-28T18:09:40.032 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:09:39 smithi022 ceph-mon[31985]: scrub summary: idle+waiting paths [/] 2024-03-28T18:09:40.032 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:09:39 smithi022 ceph-mon[31985]: scrub summary: idle 2024-03-28T18:09:40.035 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/misc/fsl,qoriq-mc.txt 2024-03-28T18:09:40.053 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/misc/ifm-csi.txt 2024-03-28T18:09:40.068 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/misc/lis302.txt 2024-03-28T18:09:40.084 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/misc/nvidia,tegra20-apbmisc.txt 2024-03-28T18:09:40.102 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/misc/smc.txt 2024-03-28T18:09:40.117 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/misc/sram.txt 2024-03-28T18:09:40.133 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/misc/ti,dac7512.txt 2024-03-28T18:09:40.185 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/mmc/ 2024-03-28T18:09:40.185 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/mmc/arasan,sdhci.txt 2024-03-28T18:09:40.185 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/mmc/atmel-hsmci.txt 2024-03-28T18:09:40.195 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/mmc/brcm,bcm2835-sdhci.txt 2024-03-28T18:09:40.210 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/mmc/davinci_mmc.txt 2024-03-28T18:09:40.226 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/mmc/exynos-dw-mshc.txt 2024-03-28T18:09:40.243 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/mmc/fsl-esdhc.txt 2024-03-28T18:09:40.257 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/mmc/fsl-imx-esdhc.txt 2024-03-28T18:09:40.282 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/mmc/fsl-imx-mmc.txt 2024-03-28T18:09:40.336 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/mmc/img-dw-mshc.txt 2024-03-28T18:09:40.361 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/mmc/k3-dw-mshc.txt 2024-03-28T18:09:40.377 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/mmc/kona-sdhci.txt 2024-03-28T18:09:40.392 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/mmc/mmc-pwrseq-emmc.txt 2024-03-28T18:09:40.408 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/mmc/mmc-pwrseq-simple.txt 2024-03-28T18:09:40.423 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/mmc/mmc-spi-slot.txt 2024-03-28T18:09:40.440 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/mmc/mmc.txt 2024-03-28T18:09:40.455 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/mmc/mmci.txt 2024-03-28T18:09:40.471 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/mmc/moxa,moxart-mmc.txt 2024-03-28T18:09:40.487 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/mmc/mxs-mmc.txt 2024-03-28T18:09:40.503 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/mmc/nvidia,tegra20-sdhci.txt 2024-03-28T18:09:40.517 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/mmc/orion-sdio.txt 2024-03-28T18:09:40.534 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/mmc/pxa-mmc.txt 2024-03-28T18:09:40.551 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/mmc/renesas,mmcif.txt 2024-03-28T18:09:40.567 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/mmc/rockchip-dw-mshc.txt 2024-03-28T18:09:40.582 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/mmc/samsung-sdhci.txt 2024-03-28T18:09:40.597 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/mmc/sdhci-dove.txt 2024-03-28T18:09:40.613 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/mmc/sdhci-fujitsu.txt 2024-03-28T18:09:40.629 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/mmc/sdhci-msm.txt 2024-03-28T18:09:40.646 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/mmc/sdhci-pxa.txt 2024-03-28T18:09:40.661 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/mmc/sdhci-sirf.txt 2024-03-28T18:09:40.677 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/mmc/sdhci-spear.txt 2024-03-28T18:09:40.692 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/mmc/sdhci-st.txt 2024-03-28T18:09:40.707 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/mmc/socfpga-dw-mshc.txt 2024-03-28T18:09:40.725 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/mmc/sunxi-mmc.txt 2024-03-28T18:09:40.739 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/mmc/synopsys-dw-mshc.txt 2024-03-28T18:09:40.759 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/mmc/ti-omap-hsmmc.txt 2024-03-28T18:09:40.774 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/mmc/ti-omap.txt 2024-03-28T18:09:40.788 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/mmc/tmio_mmc.txt 2024-03-28T18:09:40.804 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/mmc/usdhi6rol0.txt 2024-03-28T18:09:40.821 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/mmc/vt8500-sdmmc.txt 2024-03-28T18:09:40.837 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/mtd/ 2024-03-28T18:09:40.856 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/mtd/arm-versatile.txt 2024-03-28T18:09:40.865 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/mtd/atmel-dataflash.txt 2024-03-28T18:09:40.883 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/mtd/atmel-nand.txt 2024-03-28T18:09:40.897 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/mtd/davinci-nand.txt 2024-03-28T18:09:40.914 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/mtd/denali-nand.txt 2024-03-28T18:09:40.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:09:40 smithi134 ceph-mon[35037]: pgmap v66: 97 pgs: 97 active+clean; 91 MiB data, 694 MiB used, 1.0 TiB / 1.0 TiB avail; 223 KiB/s rd, 408 KiB/s wr, 106 op/s 2024-03-28T18:09:40.925 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:09:40 smithi134 ceph-mon[35037]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-28T18:09:40.929 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/mtd/diskonchip.txt 2024-03-28T18:09:40.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:09:40 smithi079 ceph-mon[35312]: pgmap v66: 97 pgs: 97 active+clean; 91 MiB data, 694 MiB used, 1.0 TiB / 1.0 TiB avail; 223 KiB/s rd, 408 KiB/s wr, 106 op/s 2024-03-28T18:09:40.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:09:40 smithi079 ceph-mon[35312]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-28T18:09:40.944 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/mtd/elm.txt 2024-03-28T18:09:40.959 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/mtd/flctl-nand.txt 2024-03-28T18:09:40.975 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/mtd/fsl-quadspi.txt 2024-03-28T18:09:40.990 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/mtd/fsl-upm-nand.txt 2024-03-28T18:09:41.006 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/mtd/fsmc-nand.txt 2024-03-28T18:09:41.022 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/mtd/gpio-control-nand.txt 2024-03-28T18:09:41.032 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:09:40 smithi022 ceph-mon[31985]: pgmap v66: 97 pgs: 97 active+clean; 91 MiB data, 694 MiB used, 1.0 TiB / 1.0 TiB avail; 223 KiB/s rd, 408 KiB/s wr, 106 op/s 2024-03-28T18:09:41.032 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:09:40 smithi022 ceph-mon[31985]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-28T18:09:41.038 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/mtd/gpmc-nand.txt 2024-03-28T18:09:41.053 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/mtd/gpmc-nor.txt 2024-03-28T18:09:41.069 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/mtd/gpmc-onenand.txt 2024-03-28T18:09:41.084 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/mtd/gpmi-nand.txt 2024-03-28T18:09:41.101 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/mtd/hisi504-nand.txt 2024-03-28T18:09:41.114 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/mtd/lpc32xx-mlc.txt 2024-03-28T18:09:41.132 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/mtd/lpc32xx-slc.txt 2024-03-28T18:09:41.148 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/mtd/m25p80.txt 2024-03-28T18:09:41.164 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/mtd/mtd-physmap.txt 2024-03-28T18:09:41.178 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/mtd/mxc-nand.txt 2024-03-28T18:09:41.195 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/mtd/nand.txt 2024-03-28T18:09:41.209 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/mtd/orion-nand.txt 2024-03-28T18:09:41.223 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/mtd/partition.txt 2024-03-28T18:09:41.241 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/mtd/pxa3xx-nand.txt 2024-03-28T18:09:41.256 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/mtd/spear_smi.txt 2024-03-28T18:09:41.272 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/mtd/st-fsm.txt 2024-03-28T18:09:41.289 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/mtd/sunxi-nand.txt 2024-03-28T18:09:41.332 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/ 2024-03-28T18:09:41.358 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/allwinner,sun4i-emac.txt 2024-03-28T18:09:41.368 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/allwinner,sun4i-mdio.txt 2024-03-28T18:09:41.385 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/allwinner,sun7i-a20-gmac.txt 2024-03-28T18:09:41.400 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/altera_tse.txt 2024-03-28T18:09:41.416 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/amd-xgbe-phy.txt 2024-03-28T18:09:41.432 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/amd-xgbe.txt 2024-03-28T18:09:41.449 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/apm-xgene-enet.txt 2024-03-28T18:09:41.465 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/arc_emac.txt 2024-03-28T18:09:41.481 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/broadcom-bcm87xx.txt 2024-03-28T18:09:41.497 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/broadcom-bcmgenet.txt 2024-03-28T18:09:41.514 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/broadcom-mdio-unimac.txt 2024-03-28T18:09:41.531 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/broadcom-sf2.txt 2024-03-28T18:09:41.546 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/broadcom-systemport.txt 2024-03-28T18:09:41.562 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/calxeda-xgmac.txt 2024-03-28T18:09:41.578 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/can/ 2024-03-28T18:09:41.589 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/can/atmel-can.txt 2024-03-28T18:09:41.600 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/can/c_can.txt 2024-03-28T18:09:41.616 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/can/cc770.txt 2024-03-28T18:09:41.633 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/can/fsl-flexcan.txt 2024-03-28T18:09:41.648 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/can/grcan.txt 2024-03-28T18:09:41.664 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/can/m_can.txt 2024-03-28T18:09:41.681 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/can/microchip,mcp251x.txt 2024-03-28T18:09:41.697 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/can/mpc5xxx-mscan.txt 2024-03-28T18:09:41.714 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/can/rcar_can.txt 2024-03-28T18:09:41.730 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/can/sja1000.txt 2024-03-28T18:09:41.748 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/can/xilinx_can.txt 2024-03-28T18:09:41.765 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/cavium-mdio.txt 2024-03-28T18:09:41.786 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/cavium-mix.txt 2024-03-28T18:09:41.804 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/cavium-pip.txt 2024-03-28T18:09:41.822 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/cdns-emac.txt 2024-03-28T18:09:41.837 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/cpsw-phy-sel.txt 2024-03-28T18:09:41.854 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/cpsw.txt 2024-03-28T18:09:41.870 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/davicom-dm9000.txt 2024-03-28T18:09:41.886 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/davinci-mdio.txt 2024-03-28T18:09:41.904 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/davinci_emac.txt 2024-03-28T18:09:41.919 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/dsa/ 2024-03-28T18:09:41.931 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/dsa/dsa.txt 2024-03-28T18:09:41.941 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/emac_rockchip.txt 2024-03-28T18:09:41.962 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/ethernet.txt 2024-03-28T18:09:41.978 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/fixed-link.txt 2024-03-28T18:09:41.996 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/fsl-fec.txt 2024-03-28T18:09:42.010 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/fsl-tsec-phy.txt 2024-03-28T18:09:42.027 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/gpmc-eth.txt 2024-03-28T18:09:42.044 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/hisilicon-hip04-net.txt 2024-03-28T18:09:42.060 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/hisilicon-hix5hd2-gmac.txt 2024-03-28T18:09:42.075 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/ieee802154/ 2024-03-28T18:09:42.087 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/ieee802154/at86rf230.txt 2024-03-28T18:09:42.097 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/ieee802154/cc2520.txt 2024-03-28T18:09:42.115 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/keystone-netcp.txt 2024-03-28T18:09:42.136 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/lpc-eth.txt 2024-03-28T18:09:42.153 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/macb.txt 2024-03-28T18:09:42.171 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/marvell-armada-370-neta.txt 2024-03-28T18:09:42.187 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/marvell-orion-mdio.txt 2024-03-28T18:09:42.203 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/marvell-orion-net.txt 2024-03-28T18:09:42.221 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/marvell-pp2.txt 2024-03-28T18:09:42.237 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/marvell-pxa168.txt 2024-03-28T18:09:42.254 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/mdio-gpio.txt 2024-03-28T18:09:42.271 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/mdio-mux-gpio.txt 2024-03-28T18:09:42.310 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/mdio-mux-mmioreg.txt 2024-03-28T18:09:42.353 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/mdio-mux.txt 2024-03-28T18:09:42.383 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/meson-dwmac.txt 2024-03-28T18:09:42.401 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/micrel-ks8851.txt 2024-03-28T18:09:42.417 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/micrel-ksz90x1.txt 2024-03-28T18:09:42.432 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/micrel.txt 2024-03-28T18:09:42.448 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/moxa,moxart-mac.txt 2024-03-28T18:09:42.464 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/nfc/ 2024-03-28T18:09:42.478 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/nfc/pn544.txt 2024-03-28T18:09:42.488 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/nfc/st21nfca.txt 2024-03-28T18:09:42.506 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/nfc/st21nfcb.txt 2024-03-28T18:09:42.522 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/nfc/trf7970a.txt 2024-03-28T18:09:42.537 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/opencores-ethoc.txt 2024-03-28T18:09:42.561 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/phy.txt 2024-03-28T18:09:42.577 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/qca-qca7000-spi.txt 2024-03-28T18:09:42.595 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/rockchip-dwmac.txt 2024-03-28T18:09:42.613 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/samsung-sxgbe.txt 2024-03-28T18:09:42.626 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/sh_eth.txt 2024-03-28T18:09:42.642 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/smsc-lan91c111.txt 2024-03-28T18:09:42.659 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/smsc911x.txt 2024-03-28T18:09:42.675 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/socfpga-dwmac.txt 2024-03-28T18:09:42.690 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/sti-dwmac.txt 2024-03-28T18:09:42.707 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/stmmac.txt 2024-03-28T18:09:42.723 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/via-rhine.txt 2024-03-28T18:09:42.740 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/via-velocity.txt 2024-03-28T18:09:42.755 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/wireless/ 2024-03-28T18:09:42.769 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/wireless/brcm,bcm43xx-fmac.txt 2024-03-28T18:09:42.779 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/wireless/qcom,ath10k.txt 2024-03-28T18:09:42.798 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/wireless/ti,wl1251.txt 2024-03-28T18:09:42.816 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/nios2/ 2024-03-28T18:09:42.839 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/nios2/nios2.txt 2024-03-28T18:09:42.849 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/nios2/timer.txt 2024-03-28T18:09:42.867 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/nvec/ 2024-03-28T18:09:42.885 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/nvec/nvidia,nvec.txt 2024-03-28T18:09:42.894 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/open-pic.txt 2024-03-28T18:09:42.918 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/panel/ 2024-03-28T18:09:42.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:09:42 smithi134 ceph-mon[35037]: pgmap v67: 97 pgs: 97 active+clean; 92 MiB data, 697 MiB used, 1.0 TiB / 1.0 TiB avail; 340 KiB/s rd, 581 KiB/s wr, 166 op/s 2024-03-28T18:09:42.931 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/panel/auo,b101aw03.txt 2024-03-28T18:09:42.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:09:42 smithi079 ceph-mon[35312]: pgmap v67: 97 pgs: 97 active+clean; 92 MiB data, 697 MiB used, 1.0 TiB / 1.0 TiB avail; 340 KiB/s rd, 581 KiB/s wr, 166 op/s 2024-03-28T18:09:42.940 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/panel/auo,b101xtn01.txt 2024-03-28T18:09:42.956 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/panel/auo,b116xw03.txt 2024-03-28T18:09:42.972 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/panel/auo,b133htn01.txt 2024-03-28T18:09:42.989 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/panel/auo,b133xtn01.txt 2024-03-28T18:09:43.004 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/panel/avic,tm070ddh03.txt 2024-03-28T18:09:43.020 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/panel/chunghwa,claa101wa01a.txt 2024-03-28T18:09:43.031 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:09:42 smithi022 ceph-mon[31985]: pgmap v67: 97 pgs: 97 active+clean; 92 MiB data, 697 MiB used, 1.0 TiB / 1.0 TiB avail; 340 KiB/s rd, 581 KiB/s wr, 166 op/s 2024-03-28T18:09:43.034 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/panel/chunghwa,claa101wb03.txt 2024-03-28T18:09:43.050 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/panel/edt,et057090dhu.txt 2024-03-28T18:09:43.066 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/panel/edt,et070080dh6.txt 2024-03-28T18:09:43.083 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/panel/edt,etm0700g0dh6.txt 2024-03-28T18:09:43.098 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/panel/foxlink,fl500wvr00-a0t.txt 2024-03-28T18:09:43.113 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/panel/giantplus,gpg482739qs5.txt 2024-03-28T18:09:43.128 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/panel/hannstar,hsd070pww1.txt 2024-03-28T18:09:43.143 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/panel/hit,tx23d38vm0caa.txt 2024-03-28T18:09:43.158 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/panel/innolux,g121i1-l01.txt 2024-03-28T18:09:43.174 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/panel/innolux,n116bge.txt 2024-03-28T18:09:43.189 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/panel/innolux,n156bge-l21.txt 2024-03-28T18:09:43.205 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/panel/lg,ld070wx3-sl01.txt 2024-03-28T18:09:43.220 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/panel/lg,lh500wx1-sd03.txt 2024-03-28T18:09:43.236 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/panel/lg,lp129qe.txt 2024-03-28T18:09:43.255 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/panel/panasonic,vvx10f004b00.txt 2024-03-28T18:09:43.271 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/panel/samsung,ld9040.txt 2024-03-28T18:09:43.286 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/panel/samsung,ltn101nt05.txt 2024-03-28T18:09:43.302 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/panel/samsung,s6e8aa0.txt 2024-03-28T18:09:43.319 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/panel/sharp,lq101r1sx01.txt 2024-03-28T18:09:43.369 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/panel/simple-panel.txt 2024-03-28T18:09:43.387 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/pci/ 2024-03-28T18:09:43.408 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/pci/83xx-512x-pci.txt 2024-03-28T18:09:43.418 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/pci/designware-pcie.txt 2024-03-28T18:09:43.438 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/pci/fsl,imx6q-pcie.txt 2024-03-28T18:09:43.454 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/pci/fsl,pci.txt 2024-03-28T18:09:43.469 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/pci/host-generic-pci.txt 2024-03-28T18:09:43.485 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/pci/layerscape-pci.txt 2024-03-28T18:09:43.501 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/pci/mvebu-pci.txt 2024-03-28T18:09:43.519 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/pci/nvidia,tegra20-pcie.txt 2024-03-28T18:09:43.535 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/pci/pci-keystone.txt 2024-03-28T18:09:43.550 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/pci/pci-rcar-gen2.txt 2024-03-28T18:09:43.565 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/pci/pci.txt 2024-03-28T18:09:43.581 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/pci/ralink,rt3883-pci.txt 2024-03-28T18:09:43.597 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/pci/rcar-pci.txt 2024-03-28T18:09:43.615 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/pci/samsung,exynos5440-pcie.txt 2024-03-28T18:09:43.631 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/pci/spear13xx-pcie.txt 2024-03-28T18:09:43.645 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/pci/ti-pci.txt 2024-03-28T18:09:43.660 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/pci/v3-v360epc-pci.txt 2024-03-28T18:09:43.676 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/pci/versatile.txt 2024-03-28T18:09:43.693 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/pci/xgene-pci.txt 2024-03-28T18:09:43.708 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/pci/xilinx-pcie.txt 2024-03-28T18:09:43.724 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/phy/ 2024-03-28T18:09:43.742 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/phy/apm-xgene-phy.txt 2024-03-28T18:09:43.752 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/phy/bcm-phy.txt 2024-03-28T18:09:43.770 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/phy/berlin-sata-phy.txt 2024-03-28T18:09:43.785 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/phy/berlin-usb-phy.txt 2024-03-28T18:09:43.800 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/phy/hix5hd2-phy.txt 2024-03-28T18:09:43.815 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/phy/phy-bindings.txt 2024-03-28T18:09:43.830 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/phy/phy-miphy28lp.txt 2024-03-28T18:09:43.846 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/phy/phy-miphy365x.txt 2024-03-28T18:09:43.862 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/phy/phy-mvebu.txt 2024-03-28T18:09:43.878 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/phy/phy-stih407-usb.txt 2024-03-28T18:09:43.893 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/phy/phy-stih41x-usb.txt 2024-03-28T18:09:43.910 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/phy/qcom-apq8064-sata-phy.txt 2024-03-28T18:09:43.925 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/phy/qcom-dwc3-usb-phy.txt 2024-03-28T18:09:43.942 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/phy/qcom-ipq806x-sata-phy.txt 2024-03-28T18:09:43.957 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/phy/rcar-gen2-phy.txt 2024-03-28T18:09:43.973 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/phy/rockchip-usb-phy.txt 2024-03-28T18:09:43.990 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/phy/samsung-phy.txt 2024-03-28T18:09:44.007 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/phy/st-spear-miphy.txt 2024-03-28T18:09:44.023 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/phy/sun4i-usb-phy.txt 2024-03-28T18:09:44.039 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/phy/ti-phy.txt 2024-03-28T18:09:44.055 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/pinctrl/ 2024-03-28T18:09:44.073 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/pinctrl/abilis,tb10x-iomux.txt 2024-03-28T18:09:44.083 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/pinctrl/allwinner,sunxi-pinctrl.txt 2024-03-28T18:09:44.100 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/pinctrl/atmel,at91-pinctrl.txt 2024-03-28T18:09:44.116 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/pinctrl/brcm,bcm11351-pinctrl.txt 2024-03-28T18:09:44.133 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/pinctrl/brcm,bcm2835-gpio.txt 2024-03-28T18:09:44.149 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/pinctrl/fsl,imx-pinctrl.txt 2024-03-28T18:09:44.165 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/pinctrl/fsl,imx25-pinctrl.txt 2024-03-28T18:09:44.180 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/pinctrl/fsl,imx27-pinctrl.txt 2024-03-28T18:09:44.195 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/pinctrl/fsl,imx35-pinctrl.txt 2024-03-28T18:09:44.210 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/pinctrl/fsl,imx51-pinctrl.txt 2024-03-28T18:09:44.226 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/pinctrl/fsl,imx53-pinctrl.txt 2024-03-28T18:09:44.241 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/pinctrl/fsl,imx6dl-pinctrl.txt 2024-03-28T18:09:44.257 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/pinctrl/fsl,imx6q-pinctrl.txt 2024-03-28T18:09:44.271 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/pinctrl/fsl,imx6sl-pinctrl.txt 2024-03-28T18:09:44.287 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/pinctrl/fsl,imx6sx-pinctrl.txt 2024-03-28T18:09:44.302 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/pinctrl/fsl,mxs-pinctrl.txt 2024-03-28T18:09:44.329 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/pinctrl/fsl,vf610-pinctrl.txt 2024-03-28T18:09:44.376 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/pinctrl/img,tz1090-pdc-pinctrl.txt 2024-03-28T18:09:44.396 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/pinctrl/img,tz1090-pinctrl.txt 2024-03-28T18:09:44.413 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/pinctrl/lantiq,falcon-pinumx.txt 2024-03-28T18:09:44.429 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/pinctrl/lantiq,xway-pinumx.txt 2024-03-28T18:09:44.444 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/pinctrl/marvell,armada-370-pinctrl.txt 2024-03-28T18:09:44.461 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/pinctrl/marvell,armada-375-pinctrl.txt 2024-03-28T18:09:44.475 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/pinctrl/marvell,armada-38x-pinctrl.txt 2024-03-28T18:09:44.489 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/pinctrl/marvell,armada-xp-pinctrl.txt 2024-03-28T18:09:44.506 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/pinctrl/marvell,dove-pinctrl.txt 2024-03-28T18:09:44.522 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/pinctrl/marvell,kirkwood-pinctrl.txt 2024-03-28T18:09:44.539 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/pinctrl/marvell,mvebu-pinctrl.txt 2024-03-28T18:09:44.556 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/pinctrl/marvell,orion-pinctrl.txt 2024-03-28T18:09:44.571 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/pinctrl/meson,pinctrl.txt 2024-03-28T18:09:44.586 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/pinctrl/nvidia,tegra114-pinmux.txt 2024-03-28T18:09:44.602 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/pinctrl/nvidia,tegra124-pinmux.txt 2024-03-28T18:09:44.618 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/pinctrl/nvidia,tegra124-xusb-padctl.txt 2024-03-28T18:09:44.637 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/pinctrl/nvidia,tegra20-pinmux.txt 2024-03-28T18:09:44.650 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/pinctrl/nvidia,tegra30-pinmux.txt 2024-03-28T18:09:44.691 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/pinctrl/pinctrl-bindings.txt 2024-03-28T18:09:44.707 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/pinctrl/pinctrl-palmas.txt 2024-03-28T18:09:44.723 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/pinctrl/pinctrl-single.txt 2024-03-28T18:09:44.738 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/pinctrl/pinctrl-sirf.txt 2024-03-28T18:09:44.755 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/pinctrl/pinctrl-st.txt 2024-03-28T18:09:44.773 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/pinctrl/pinctrl-vt8500.txt 2024-03-28T18:09:44.787 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/pinctrl/pinctrl_spear.txt 2024-03-28T18:09:44.801 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/pinctrl/qcom,apq8064-pinctrl.txt 2024-03-28T18:09:44.812 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/pinctrl/qcom,apq8084-pinctrl.txt 2024-03-28T18:09:44.822 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/pinctrl/qcom,ipq8064-pinctrl.txt 2024-03-28T18:09:44.832 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/pinctrl/qcom,msm8916-pinctrl.txt 2024-03-28T18:09:44.843 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/pinctrl/qcom,msm8960-pinctrl.txt 2024-03-28T18:09:44.853 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/pinctrl/qcom,msm8974-pinctrl.txt 2024-03-28T18:09:44.863 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/pinctrl/qcom,pmic-gpio.txt 2024-03-28T18:09:44.872 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/pinctrl/qcom,pmic-mpp.txt 2024-03-28T18:09:44.881 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/pinctrl/renesas,pfc-pinctrl.txt 2024-03-28T18:09:44.891 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/pinctrl/rockchip,pinctrl.txt 2024-03-28T18:09:44.900 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/pinctrl/samsung-pinctrl.txt 2024-03-28T18:09:44.910 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/pinctrl/ste,abx500.txt 2024-03-28T18:09:44.920 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/pinctrl/ste,nomadik.txt 2024-03-28T18:09:44.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:09:44 smithi134 ceph-mon[35037]: pgmap v68: 97 pgs: 97 active+clean; 92 MiB data, 697 MiB used, 1.0 TiB / 1.0 TiB avail; 223 KiB/s rd, 330 KiB/s wr, 112 op/s 2024-03-28T18:09:44.928 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/pinctrl/ti,omap-pinctrl.txt 2024-03-28T18:09:44.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:09:44 smithi079 ceph-mon[35312]: pgmap v68: 97 pgs: 97 active+clean; 92 MiB data, 697 MiB used, 1.0 TiB / 1.0 TiB avail; 223 KiB/s rd, 330 KiB/s wr, 112 op/s 2024-03-28T18:09:44.937 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/pinctrl/xlnx,zynq-pinctrl.txt 2024-03-28T18:09:44.946 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/power/ 2024-03-28T18:09:44.960 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/power/bq2415x.txt 2024-03-28T18:09:44.965 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/power/isp1704.txt 2024-03-28T18:09:44.977 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/power/ltc2941.txt 2024-03-28T18:09:44.987 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/power/opp.txt 2024-03-28T18:09:45.000 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/power/power-controller.txt 2024-03-28T18:09:45.014 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/power/power_domain.txt 2024-03-28T18:09:45.029 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/power/renesas,sysc-rmobile.txt 2024-03-28T18:09:45.031 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:09:44 smithi022 ceph-mon[31985]: pgmap v68: 97 pgs: 97 active+clean; 92 MiB data, 697 MiB used, 1.0 TiB / 1.0 TiB avail; 223 KiB/s rd, 330 KiB/s wr, 112 op/s 2024-03-28T18:09:45.044 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/power/reset/ 2024-03-28T18:09:45.056 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/power/reset/keystone-reset.txt 2024-03-28T18:09:45.064 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/power/reset/ltc2952-poweroff.txt 2024-03-28T18:09:45.079 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/power/reset/st-reset.txt 2024-03-28T18:09:45.095 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/power/reset/syscon-reboot.txt 2024-03-28T18:09:45.109 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/power/rockchip-io-domain.txt 2024-03-28T18:09:45.129 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/power/rx51-battery.txt 2024-03-28T18:09:45.143 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/power/twl-charger.txt 2024-03-28T18:09:45.158 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/power_supply/ 2024-03-28T18:09:45.171 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/power_supply/ab8500/ 2024-03-28T18:09:45.177 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/power_supply/ab8500/btemp.txt 2024-03-28T18:09:45.185 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/power_supply/ab8500/chargalg.txt 2024-03-28T18:09:45.202 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/power_supply/ab8500/charger.txt 2024-03-28T18:09:45.217 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/power_supply/ab8500/fg.txt 2024-03-28T18:09:45.229 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/power_supply/axxia-reset.txt 2024-03-28T18:09:45.248 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/power_supply/charger-manager.txt 2024-03-28T18:09:45.262 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/power_supply/gpio-charger.txt 2024-03-28T18:09:45.275 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/power_supply/imx-snvs-poweroff.txt 2024-03-28T18:09:45.288 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/power_supply/lp8727_charger.txt 2024-03-28T18:09:45.300 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/power_supply/max17042_battery.txt 2024-03-28T18:09:45.314 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/power_supply/max8925_batter.txt 2024-03-28T18:09:45.331 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/power_supply/msm-poweroff.txt 2024-03-28T18:09:45.377 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/power_supply/olpc_battery.txt 2024-03-28T18:09:45.399 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/power_supply/power_supply.txt 2024-03-28T18:09:45.411 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/power_supply/qnap-poweroff.txt 2024-03-28T18:09:45.424 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/power_supply/restart-poweroff.txt 2024-03-28T18:09:45.436 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/power_supply/sbs_sbs-battery.txt 2024-03-28T18:09:45.446 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/power_supply/ti,bq24735.txt 2024-03-28T18:09:45.457 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/power_supply/tps65090.txt 2024-03-28T18:09:45.465 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/powerpc/ 2024-03-28T18:09:45.476 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/powerpc/4xx/ 2024-03-28T18:09:45.480 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/powerpc/4xx/akebono.txt 2024-03-28T18:09:45.485 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/powerpc/4xx/cpm.txt 2024-03-28T18:09:45.498 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/powerpc/4xx/emac.txt 2024-03-28T18:09:45.510 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/powerpc/4xx/hsta.txt 2024-03-28T18:09:45.522 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/powerpc/4xx/ndfc.txt 2024-03-28T18:09:45.537 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/powerpc/4xx/ppc440spe-adma.txt 2024-03-28T18:09:45.553 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/powerpc/4xx/reboot.txt 2024-03-28T18:09:45.566 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/powerpc/fsl/ 2024-03-28T18:09:45.581 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/powerpc/fsl/board.txt 2024-03-28T18:09:45.591 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/powerpc/fsl/cache_sram.txt 2024-03-28T18:09:45.608 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/powerpc/fsl/ccf.txt 2024-03-28T18:09:45.624 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/powerpc/fsl/cpm_qe/ 2024-03-28T18:09:45.636 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/powerpc/fsl/cpm_qe/cpm.txt 2024-03-28T18:09:45.644 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/powerpc/fsl/cpm_qe/cpm/ 2024-03-28T18:09:45.655 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/powerpc/fsl/cpm_qe/cpm/brg.txt 2024-03-28T18:09:45.664 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/powerpc/fsl/cpm_qe/cpm/i2c.txt 2024-03-28T18:09:45.676 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/powerpc/fsl/cpm_qe/cpm/pic.txt 2024-03-28T18:09:45.689 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/powerpc/fsl/cpm_qe/cpm/usb.txt 2024-03-28T18:09:45.701 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/powerpc/fsl/cpm_qe/gpio.txt 2024-03-28T18:09:45.722 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/powerpc/fsl/cpm_qe/network.txt 2024-03-28T18:09:45.736 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/powerpc/fsl/cpm_qe/qe.txt 2024-03-28T18:09:45.749 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/powerpc/fsl/cpm_qe/qe/ 2024-03-28T18:09:45.761 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/powerpc/fsl/cpm_qe/qe/firmware.txt 2024-03-28T18:09:45.770 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/powerpc/fsl/cpm_qe/qe/par_io.txt 2024-03-28T18:09:45.785 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/powerpc/fsl/cpm_qe/qe/pincfg.txt 2024-03-28T18:09:45.802 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/powerpc/fsl/cpm_qe/qe/ucc.txt 2024-03-28T18:09:45.819 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/powerpc/fsl/cpm_qe/qe/usb.txt 2024-03-28T18:09:45.834 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/powerpc/fsl/cpm_qe/serial.txt 2024-03-28T18:09:45.853 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/powerpc/fsl/cpus.txt 2024-03-28T18:09:45.875 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/powerpc/fsl/dcsr.txt 2024-03-28T18:09:45.892 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/powerpc/fsl/diu.txt 2024-03-28T18:09:45.906 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/powerpc/fsl/dma.txt 2024-03-28T18:09:45.919 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/powerpc/fsl/ecm.txt 2024-03-28T18:09:45.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:09:45 smithi134 ceph-mon[35037]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-28T18:09:45.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:09:45 smithi134 ceph-mon[35037]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T18:09:45.932 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/powerpc/fsl/fman.txt 2024-03-28T18:09:45.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:09:45 smithi079 ceph-mon[35312]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-28T18:09:45.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:09:45 smithi079 ceph-mon[35312]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T18:09:45.944 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/powerpc/fsl/gtm.txt 2024-03-28T18:09:45.958 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/powerpc/fsl/guts.txt 2024-03-28T18:09:45.975 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/powerpc/fsl/interlaken-lac.txt 2024-03-28T18:09:45.992 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/powerpc/fsl/l2cache.txt 2024-03-28T18:09:46.006 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/powerpc/fsl/lbc.txt 2024-03-28T18:09:46.020 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/powerpc/fsl/mcm.txt 2024-03-28T18:09:46.031 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:09:45 smithi022 ceph-mon[31985]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-28T18:09:46.032 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:09:45 smithi022 ceph-mon[31985]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T18:09:46.035 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/powerpc/fsl/mcu-mpc8349emitx.txt 2024-03-28T18:09:46.049 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/powerpc/fsl/mem-ctrlr.txt 2024-03-28T18:09:46.064 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/powerpc/fsl/mpc5121-psc.txt 2024-03-28T18:09:46.078 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/powerpc/fsl/mpc5200.txt 2024-03-28T18:09:46.093 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/powerpc/fsl/mpic-msgr.txt 2024-03-28T18:09:46.109 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/powerpc/fsl/mpic-timer.txt 2024-03-28T18:09:46.126 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/powerpc/fsl/mpic.txt 2024-03-28T18:09:46.143 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/powerpc/fsl/msi-pic.txt 2024-03-28T18:09:46.158 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/powerpc/fsl/pamu.txt 2024-03-28T18:09:46.175 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/powerpc/fsl/pmc.txt 2024-03-28T18:09:46.191 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/powerpc/fsl/raideng.txt 2024-03-28T18:09:46.206 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/powerpc/fsl/srio-rmu.txt 2024-03-28T18:09:46.222 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/powerpc/fsl/srio.txt 2024-03-28T18:09:46.237 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/powerpc/nintendo/ 2024-03-28T18:09:46.251 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/powerpc/nintendo/gamecube.txt 2024-03-28T18:09:46.257 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/powerpc/nintendo/wii.txt 2024-03-28T18:09:46.269 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/pps/ 2024-03-28T18:09:46.293 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/pps/pps-gpio.txt 2024-03-28T18:09:46.300 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/pwm/ 2024-03-28T18:09:46.315 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/pwm/atmel-hlcdc-pwm.txt 2024-03-28T18:09:46.322 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/pwm/atmel-pwm.txt 2024-03-28T18:09:46.337 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/pwm/atmel-tcb-pwm.txt 2024-03-28T18:09:46.392 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/pwm/bcm-kona-pwm.txt 2024-03-28T18:09:46.414 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/pwm/cirrus,clps711x-pwm.txt 2024-03-28T18:09:46.430 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/pwm/img-pwm.txt 2024-03-28T18:09:46.446 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/pwm/imx-pwm.txt 2024-03-28T18:09:46.462 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/pwm/lpc32xx-pwm.txt 2024-03-28T18:09:46.477 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/pwm/mxs-pwm.txt 2024-03-28T18:09:46.493 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/pwm/nvidia,tegra20-pwm.txt 2024-03-28T18:09:46.509 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/pwm/nxp,pca9685-pwm.txt 2024-03-28T18:09:46.523 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/pwm/pwm-bcm2835.txt 2024-03-28T18:09:46.538 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/pwm/pwm-fsl-ftm.txt 2024-03-28T18:09:46.553 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/pwm/pwm-lp3943.txt 2024-03-28T18:09:46.570 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/pwm/pwm-rockchip.txt 2024-03-28T18:09:46.585 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/pwm/pwm-samsung.txt 2024-03-28T18:09:46.599 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/pwm/pwm-st.txt 2024-03-28T18:09:46.613 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/pwm/pwm-sun4i.txt 2024-03-28T18:09:46.631 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/pwm/pwm-tiecap.txt 2024-03-28T18:09:46.648 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/pwm/pwm-tiehrpwm.txt 2024-03-28T18:09:46.664 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/pwm/pwm-tipwmss.txt 2024-03-28T18:09:46.677 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/pwm/pwm.txt 2024-03-28T18:09:46.694 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/pwm/pxa-pwm.txt 2024-03-28T18:09:46.709 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/pwm/renesas,tpu-pwm.txt 2024-03-28T18:09:46.724 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/pwm/spear-pwm.txt 2024-03-28T18:09:46.741 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/pwm/ti,twl-pwm.txt 2024-03-28T18:09:46.756 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/pwm/ti,twl-pwmled.txt 2024-03-28T18:09:46.772 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/pwm/vt8500-pwm.txt 2024-03-28T18:09:46.788 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/regmap/ 2024-03-28T18:09:46.804 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/regmap/regmap.txt 2024-03-28T18:09:46.813 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/regulator/ 2024-03-28T18:09:46.832 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/regulator/88pm800.txt 2024-03-28T18:09:46.841 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/regulator/88pm860x.txt 2024-03-28T18:09:46.857 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/regulator/act8865-regulator.txt 2024-03-28T18:09:46.872 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/regulator/anatop-regulator.txt 2024-03-28T18:09:46.887 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/regulator/as3722-regulator.txt 2024-03-28T18:09:46.899 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/regulator/da9210.txt 2024-03-28T18:09:46.916 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/regulator/da9211.txt 2024-03-28T18:09:46.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:09:46 smithi134 ceph-mon[35037]: pgmap v69: 97 pgs: 97 active+clean; 94 MiB data, 709 MiB used, 1.0 TiB / 1.0 TiB avail; 356 KiB/s rd, 511 KiB/s wr, 166 op/s 2024-03-28T18:09:46.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:09:46 smithi134 ceph-mon[35037]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T18:09:46.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:09:46 smithi134 ceph-mon[35037]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T18:09:46.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:09:46 smithi134 ceph-mon[35037]: from='mgr.15522 ' entity='mgr.x' 2024-03-28T18:09:46.932 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/regulator/fan53555.txt 2024-03-28T18:09:46.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:09:46 smithi079 ceph-mon[35312]: pgmap v69: 97 pgs: 97 active+clean; 94 MiB data, 709 MiB used, 1.0 TiB / 1.0 TiB avail; 356 KiB/s rd, 511 KiB/s wr, 166 op/s 2024-03-28T18:09:46.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:09:46 smithi079 ceph-mon[35312]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T18:09:46.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:09:46 smithi079 ceph-mon[35312]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T18:09:46.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:09:46 smithi079 ceph-mon[35312]: from='mgr.15522 ' entity='mgr.x' 2024-03-28T18:09:46.947 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/regulator/fixed-regulator.txt 2024-03-28T18:09:46.964 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/regulator/gpio-regulator.txt 2024-03-28T18:09:46.980 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/regulator/isl9305.txt 2024-03-28T18:09:46.996 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/regulator/lp872x.txt 2024-03-28T18:09:47.012 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/regulator/ltc3589.txt 2024-03-28T18:09:47.029 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/regulator/max1586-regulator.txt 2024-03-28T18:09:47.032 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:09:46 smithi022 ceph-mon[31985]: pgmap v69: 97 pgs: 97 active+clean; 94 MiB data, 709 MiB used, 1.0 TiB / 1.0 TiB avail; 356 KiB/s rd, 511 KiB/s wr, 166 op/s 2024-03-28T18:09:47.032 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:09:46 smithi022 ceph-mon[31985]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T18:09:47.032 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:09:46 smithi022 ceph-mon[31985]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T18:09:47.032 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:09:46 smithi022 ceph-mon[31985]: from='mgr.15522 ' entity='mgr.x' 2024-03-28T18:09:47.045 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/regulator/max77802.txt 2024-03-28T18:09:47.062 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/regulator/max8660.txt 2024-03-28T18:09:47.079 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/regulator/max8907.txt 2024-03-28T18:09:47.095 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/regulator/max8925-regulator.txt 2024-03-28T18:09:47.112 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/regulator/max8952.txt 2024-03-28T18:09:47.128 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/regulator/max8973-regulator.txt 2024-03-28T18:09:47.145 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/regulator/max8997-regulator.txt 2024-03-28T18:09:47.161 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/regulator/mt6397-regulator.txt 2024-03-28T18:09:47.177 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/regulator/palmas-pmic.txt 2024-03-28T18:09:47.193 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/regulator/pbias-regulator.txt 2024-03-28T18:09:47.211 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/regulator/pfuze100.txt 2024-03-28T18:09:47.227 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/regulator/pwm-regulator.txt 2024-03-28T18:09:47.242 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/regulator/regulator.txt 2024-03-28T18:09:47.259 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/regulator/s5m8767-regulator.txt 2024-03-28T18:09:47.275 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/regulator/sky81452-regulator.txt 2024-03-28T18:09:47.290 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/regulator/ti-abb-regulator.txt 2024-03-28T18:09:47.305 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/regulator/tps51632-regulator.txt 2024-03-28T18:09:47.321 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/regulator/tps62360-regulator.txt 2024-03-28T18:09:47.340 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/regulator/tps65090.txt 2024-03-28T18:09:47.379 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/regulator/tps65217.txt 2024-03-28T18:09:47.399 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/regulator/tps65218.txt 2024-03-28T18:09:47.415 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/regulator/tps6586x.txt 2024-03-28T18:09:47.432 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/regulator/twl-regulator.txt 2024-03-28T18:09:47.449 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/regulator/vexpress.txt 2024-03-28T18:09:47.465 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/reserved-memory/ 2024-03-28T18:09:47.486 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/reserved-memory/reserved-memory.txt 2024-03-28T18:09:47.495 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/reset/ 2024-03-28T18:09:47.516 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/reset/allwinner,sunxi-clock-reset.txt 2024-03-28T18:09:47.524 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/reset/fsl,imx-src.txt 2024-03-28T18:09:47.542 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/reset/reset.txt 2024-03-28T18:09:47.559 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/reset/sirf,rstc.txt 2024-03-28T18:09:47.574 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/reset/socfpga-reset.txt 2024-03-28T18:09:47.589 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/reset/st,sti-picophyreset.txt 2024-03-28T18:09:47.605 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/reset/st,sti-powerdown.txt 2024-03-28T18:09:47.622 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/reset/st,sti-softreset.txt 2024-03-28T18:09:47.638 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/resource-names.txt 2024-03-28T18:09:47.659 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/rng/ 2024-03-28T18:09:47.672 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/rng/apm,rng.txt 2024-03-28T18:09:47.681 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/rng/brcm,bcm2835.txt 2024-03-28T18:09:47.697 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/rng/qcom,prng.txt 2024-03-28T18:09:47.713 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/rtc/ 2024-03-28T18:09:47.733 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/rtc/armada-380-rtc.txt 2024-03-28T18:09:47.742 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/rtc/atmel,at91rm9200-rtc.txt 2024-03-28T18:09:47.760 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/rtc/atmel,at91sam9-rtc.txt 2024-03-28T18:09:47.776 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/rtc/dallas,ds1339.txt 2024-03-28T18:09:47.791 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/rtc/dw-apb.txt 2024-03-28T18:09:47.807 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/rtc/haoyu,hym8563.txt 2024-03-28T18:09:47.823 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/rtc/imxdi-rtc.txt 2024-03-28T18:09:47.842 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/rtc/isil,isl12057.txt 2024-03-28T18:09:47.858 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/rtc/lpc32xx-rtc.txt 2024-03-28T18:09:47.874 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/rtc/maxim,ds1742.txt 2024-03-28T18:09:47.891 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/rtc/moxa,moxart-rtc.txt 2024-03-28T18:09:47.906 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/rtc/nvidia,tegra20-rtc.txt 2024-03-28T18:09:47.923 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/rtc/nxp,rtc-2123.txt 2024-03-28T18:09:47.939 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/rtc/olpc-xo1-rtc.txt 2024-03-28T18:09:47.955 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/rtc/orion-rtc.txt 2024-03-28T18:09:47.973 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/rtc/pxa-rtc.txt 2024-03-28T18:09:47.990 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/rtc/rtc-cmos.txt 2024-03-28T18:09:48.004 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/rtc/rtc-omap.txt 2024-03-28T18:09:48.021 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/rtc/rtc-opal.txt 2024-03-28T18:09:48.034 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/rtc/rtc-palmas.txt 2024-03-28T18:09:48.050 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/rtc/s3c-rtc.txt 2024-03-28T18:09:48.068 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/rtc/sa1100-rtc.txt 2024-03-28T18:09:48.084 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/rtc/snvs-rtc.txt 2024-03-28T18:09:48.100 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/rtc/spear-rtc.txt 2024-03-28T18:09:48.116 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/rtc/stmp3xxx-rtc.txt 2024-03-28T18:09:48.132 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/rtc/sun6i-rtc.txt 2024-03-28T18:09:48.146 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/rtc/sunxi-rtc.txt 2024-03-28T18:09:48.161 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/rtc/twl-rtc.txt 2024-03-28T18:09:48.176 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/rtc/via,vt8500-rtc.txt 2024-03-28T18:09:48.194 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/rtc/xgene-rtc.txt 2024-03-28T18:09:48.211 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/security/ 2024-03-28T18:09:48.230 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/security/tpm/ 2024-03-28T18:09:48.238 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/security/tpm/st33zp24-i2c.txt 2024-03-28T18:09:48.248 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/serial/ 2024-03-28T18:09:48.272 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/serial/8250.txt 2024-03-28T18:09:48.282 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/serial/altera_jtaguart.txt 2024-03-28T18:09:48.301 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/serial/altera_uart.txt 2024-03-28T18:09:48.316 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/serial/arc-uart.txt 2024-03-28T18:09:48.331 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/serial/atmel-usart.txt 2024-03-28T18:09:48.406 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/serial/axis,etraxfs-uart.txt 2024-03-28T18:09:48.433 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/serial/bcm63xx-uart.txt 2024-03-28T18:09:48.448 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/serial/cavium-uart.txt 2024-03-28T18:09:48.463 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/serial/cdns,uart.txt 2024-03-28T18:09:48.478 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/serial/cirrus,clps711x-uart.txt 2024-03-28T18:09:48.493 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/serial/digicolor-usart.txt 2024-03-28T18:09:48.508 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/serial/efm32-uart.txt 2024-03-28T18:09:48.523 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/serial/fsl-imx-uart.txt 2024-03-28T18:09:48.538 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/serial/fsl-lpuart.txt 2024-03-28T18:09:48.553 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/serial/fsl-mxs-auart.txt 2024-03-28T18:09:48.570 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/serial/lantiq_asc.txt 2024-03-28T18:09:48.586 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/serial/maxim,max310x.txt 2024-03-28T18:09:48.601 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/serial/mrvl,pxa-ssp.txt 2024-03-28T18:09:48.617 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/serial/mrvl-serial.txt 2024-03-28T18:09:48.632 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/serial/mtk-uart.txt 2024-03-28T18:09:48.649 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/serial/nvidia,tegra20-hsuart.txt 2024-03-28T18:09:48.663 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/serial/nxp,sc16is7xx.txt 2024-03-28T18:09:48.682 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/serial/nxp-lpc32xx-hsuart.txt 2024-03-28T18:09:48.697 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/serial/omap_serial.txt 2024-03-28T18:09:48.712 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/serial/pl011.txt 2024-03-28T18:09:48.727 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/serial/qca,ar9330-uart.txt 2024-03-28T18:09:48.743 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/serial/qcom,msm-uart.txt 2024-03-28T18:09:48.759 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/serial/qcom,msm-uartdm.txt 2024-03-28T18:09:48.775 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/serial/renesas,sci-serial.txt 2024-03-28T18:09:48.791 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/serial/rs485.txt 2024-03-28T18:09:48.806 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/serial/samsung_uart.txt 2024-03-28T18:09:48.822 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/serial/sirf-uart.txt 2024-03-28T18:09:48.839 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/serial/snps-dw-apb-uart.txt 2024-03-28T18:09:48.857 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/serial/sprd-uart.txt 2024-03-28T18:09:48.873 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/serial/st-asc.txt 2024-03-28T18:09:48.888 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/serial/vt8500-uart.txt 2024-03-28T18:09:48.904 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/serio/ 2024-03-28T18:09:48.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:09:48 smithi134 ceph-mon[35037]: pgmap v70: 97 pgs: 97 active+clean; 94 MiB data, 709 MiB used, 1.0 TiB / 1.0 TiB avail; 250 KiB/s rd, 353 KiB/s wr, 114 op/s 2024-03-28T18:09:48.925 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/serio/allwinner,sun4i-ps2.txt 2024-03-28T18:09:48.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:09:48 smithi079 ceph-mon[35312]: pgmap v70: 97 pgs: 97 active+clean; 94 MiB data, 709 MiB used, 1.0 TiB / 1.0 TiB avail; 250 KiB/s rd, 353 KiB/s wr, 114 op/s 2024-03-28T18:09:48.937 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/serio/altera_ps2.txt 2024-03-28T18:09:48.963 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/serio/olpc,ap-sp.txt 2024-03-28T18:09:48.980 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/serio/snps-arc_ps2.txt 2024-03-28T18:09:48.997 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/soc/ 2024-03-28T18:09:49.015 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/soc/fsl/ 2024-03-28T18:09:49.022 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/soc/fsl/bman-portals.txt 2024-03-28T18:09:49.031 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:09:48 smithi022 ceph-mon[31985]: pgmap v70: 97 pgs: 97 active+clean; 94 MiB data, 709 MiB used, 1.0 TiB / 1.0 TiB avail; 250 KiB/s rd, 353 KiB/s wr, 114 op/s 2024-03-28T18:09:49.032 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/soc/fsl/bman.txt 2024-03-28T18:09:49.050 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/soc/fsl/qman-portals.txt 2024-03-28T18:09:49.068 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/soc/fsl/qman.txt 2024-03-28T18:09:49.084 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/soc/qcom/ 2024-03-28T18:09:49.103 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/soc/qcom/qcom,gsbi.txt 2024-03-28T18:09:49.114 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/soc/ti/ 2024-03-28T18:09:49.133 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/soc/ti/keystone-navigator-dma.txt 2024-03-28T18:09:49.145 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/soc/ti/keystone-navigator-qmss.txt 2024-03-28T18:09:49.164 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/ 2024-03-28T18:09:49.189 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/adi,adau1701.txt 2024-03-28T18:09:49.198 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/adi,axi-i2s.txt 2024-03-28T18:09:49.216 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/adi,axi-spdif-tx.txt 2024-03-28T18:09:49.231 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/adi,ssm2602.txt 2024-03-28T18:09:49.246 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/ak4104.txt 2024-03-28T18:09:49.261 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/ak4554.c 2024-03-28T18:09:49.275 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/ak4642.txt 2024-03-28T18:09:49.294 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/ak5386.txt 2024-03-28T18:09:49.311 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/alc5623.txt 2024-03-28T18:09:49.326 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/alc5632.txt 2024-03-28T18:09:49.342 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/armada-370db-audio.txt 2024-03-28T18:09:49.360 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/arndale.txt 2024-03-28T18:09:49.396 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/atmel-at91sam9g20ek-wm8731-audio.txt 2024-03-28T18:09:49.422 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/atmel-sam9x5-wm8731-audio.txt 2024-03-28T18:09:49.436 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/atmel-wm8904.txt 2024-03-28T18:09:49.452 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/atmel_ac97c.txt 2024-03-28T18:09:49.468 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/bcm2835-i2s.txt 2024-03-28T18:09:49.484 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/cdns,xtfpga-i2s.txt 2024-03-28T18:09:49.500 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/cs35l32.txt 2024-03-28T18:09:49.516 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/cs4265.txt 2024-03-28T18:09:49.533 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/cs4270.txt 2024-03-28T18:09:49.551 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/cs4271.txt 2024-03-28T18:09:49.566 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/cs42l52.txt 2024-03-28T18:09:49.581 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/cs42l56.txt 2024-03-28T18:09:49.597 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/cs42l73.txt 2024-03-28T18:09:49.613 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/cs42xx8.txt 2024-03-28T18:09:49.629 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/da9055.txt 2024-03-28T18:09:49.648 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/davinci-evm-audio.txt 2024-03-28T18:09:49.660 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/davinci-mcasp-audio.txt 2024-03-28T18:09:49.678 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/designware-i2s.txt 2024-03-28T18:09:49.698 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/es8328.txt 2024-03-28T18:09:49.736 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/eukrea-tlv320.txt 2024-03-28T18:09:49.752 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/fsl,asrc.txt 2024-03-28T18:09:49.775 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/fsl,esai.txt 2024-03-28T18:09:49.794 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/fsl,spdif.txt 2024-03-28T18:09:49.809 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/fsl,ssi.txt 2024-03-28T18:09:49.833 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/fsl-asoc-card.txt 2024-03-28T18:09:49.845 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/fsl-sai.txt 2024-03-28T18:09:49.863 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/hdmi.txt 2024-03-28T18:09:49.882 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/imx-audio-es8328.txt 2024-03-28T18:09:49.900 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/imx-audio-sgtl5000.txt 2024-03-28T18:09:49.912 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/imx-audio-spdif.txt 2024-03-28T18:09:49.926 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/imx-audio-wm8962.txt 2024-03-28T18:09:49.940 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/imx-audmux.txt 2024-03-28T18:09:49.956 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/ingenic,jz4740-i2s.txt 2024-03-28T18:09:49.972 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/max98090.txt 2024-03-28T18:09:49.988 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/max98095.txt 2024-03-28T18:09:50.210 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/max98357a.txt 2024-03-28T18:09:50.226 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/mrvl,pxa-ssp.txt 2024-03-28T18:09:50.242 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/mrvl,pxa2xx-pcm.txt 2024-03-28T18:09:50.259 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/mvebu-audio.txt 2024-03-28T18:09:50.275 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/mxs-audio-sgtl5000.txt 2024-03-28T18:09:50.290 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/mxs-saif.txt 2024-03-28T18:09:50.311 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/nokia,rx51.txt 2024-03-28T18:09:50.329 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/nvidia,tegra-audio-alc5632.txt 2024-03-28T18:09:50.344 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/nvidia,tegra-audio-max98090.txt 2024-03-28T18:09:50.371 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/nvidia,tegra-audio-rt5640.txt 2024-03-28T18:09:50.402 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/nvidia,tegra-audio-rt5677.txt 2024-03-28T18:09:50.419 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/nvidia,tegra-audio-trimslice.txt 2024-03-28T18:09:50.435 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/nvidia,tegra-audio-wm8753.txt 2024-03-28T18:09:50.449 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/nvidia,tegra-audio-wm8903.txt 2024-03-28T18:09:50.464 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/nvidia,tegra-audio-wm9712.txt 2024-03-28T18:09:50.479 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/nvidia,tegra20-ac97.txt 2024-03-28T18:09:50.497 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/nvidia,tegra20-das.txt 2024-03-28T18:09:50.512 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/nvidia,tegra20-i2s.txt 2024-03-28T18:09:50.527 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/nvidia,tegra30-ahub.txt 2024-03-28T18:09:50.542 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/nvidia,tegra30-hda.txt 2024-03-28T18:09:50.559 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/nvidia,tegra30-i2s.txt 2024-03-28T18:09:50.576 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/omap-abe-twl6040.txt 2024-03-28T18:09:50.592 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/omap-dmic.txt 2024-03-28T18:09:50.613 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/omap-mcbsp.txt 2024-03-28T18:09:50.628 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/omap-mcpdm.txt 2024-03-28T18:09:50.643 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/omap-twl4030.txt 2024-03-28T18:09:50.659 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/pcm1792a.txt 2024-03-28T18:09:50.674 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/pcm512x.txt 2024-03-28T18:09:50.690 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/renesas,fsi.txt 2024-03-28T18:09:50.707 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/renesas,rsnd.txt 2024-03-28T18:09:50.723 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/rockchip-i2s.txt 2024-03-28T18:09:50.738 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/rt5631.txt 2024-03-28T18:09:50.755 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/rt5640.txt 2024-03-28T18:09:50.769 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/rt5677.txt 2024-03-28T18:09:50.785 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/samsung,odroidx2-max98090.txt 2024-03-28T18:09:50.800 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/samsung,smdk-wm8994.txt 2024-03-28T18:09:50.816 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/samsung-i2s.txt 2024-03-28T18:09:50.832 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/sgtl5000.txt 2024-03-28T18:09:50.848 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/simple-card.txt 2024-03-28T18:09:50.863 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/sirf-audio-codec.txt 2024-03-28T18:09:50.878 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/sirf-audio-port.txt 2024-03-28T18:09:50.893 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/sirf-audio.txt 2024-03-28T18:09:50.908 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/sirf-usp.txt 2024-03-28T18:09:50.923 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/snow.txt 2024-03-28T18:09:50.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:09:50 smithi134 ceph-mon[35037]: pgmap v71: 97 pgs: 97 active+clean; 94 MiB data, 709 MiB used, 1.0 TiB / 1.0 TiB avail; 250 KiB/s rd, 354 KiB/s wr, 114 op/s 2024-03-28T18:09:50.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:09:50 smithi134 ceph-mon[35037]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-28T18:09:50.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:09:50 smithi079 ceph-mon[35312]: pgmap v71: 97 pgs: 97 active+clean; 94 MiB data, 709 MiB used, 1.0 TiB / 1.0 TiB avail; 250 KiB/s rd, 354 KiB/s wr, 114 op/s 2024-03-28T18:09:50.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:09:50 smithi079 ceph-mon[35312]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-28T18:09:50.939 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/soc-ac97link.txt 2024-03-28T18:09:50.954 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/spdif-receiver.txt 2024-03-28T18:09:50.969 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/spdif-transmitter.txt 2024-03-28T18:09:50.985 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/ssm2518.txt 2024-03-28T18:09:51.001 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/ssm4567.txt 2024-03-28T18:09:51.017 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/st,sta32x.txt 2024-03-28T18:09:51.032 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:09:50 smithi022 ceph-mon[31985]: pgmap v71: 97 pgs: 97 active+clean; 94 MiB data, 709 MiB used, 1.0 TiB / 1.0 TiB avail; 250 KiB/s rd, 354 KiB/s wr, 114 op/s 2024-03-28T18:09:51.032 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:09:50 smithi022 ceph-mon[31985]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-28T18:09:51.033 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/st,sta350.txt 2024-03-28T18:09:51.050 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/tas2552.txt 2024-03-28T18:09:51.064 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/tdm-slot.txt 2024-03-28T18:09:51.078 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/ti,pcm1681.txt 2024-03-28T18:09:51.094 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/ti,tas5086.txt 2024-03-28T18:09:51.111 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/tlv320aic31xx.txt 2024-03-28T18:09:51.125 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/tlv320aic32x4.txt 2024-03-28T18:09:51.139 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/tlv320aic3x.txt 2024-03-28T18:09:51.155 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/tpa6130a2.txt 2024-03-28T18:09:51.171 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/ts3a227e.txt 2024-03-28T18:09:51.186 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/ux500-mop500.txt 2024-03-28T18:09:51.204 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/ux500-msp.txt 2024-03-28T18:09:51.220 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/widgets.txt 2024-03-28T18:09:51.242 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/wm8510.txt 2024-03-28T18:09:51.258 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/wm8523.txt 2024-03-28T18:09:51.275 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/wm8580.txt 2024-03-28T18:09:51.291 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/wm8711.txt 2024-03-28T18:09:51.306 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/wm8728.txt 2024-03-28T18:09:51.325 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/wm8731.txt 2024-03-28T18:09:51.340 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/wm8737.txt 2024-03-28T18:09:51.357 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/wm8741.txt 2024-03-28T18:09:51.386 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/wm8750.txt 2024-03-28T18:09:51.402 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/wm8753.txt 2024-03-28T18:09:51.418 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/wm8770.txt 2024-03-28T18:09:51.433 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/wm8776.txt 2024-03-28T18:09:51.451 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/wm8804.txt 2024-03-28T18:09:51.466 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/wm8903.txt 2024-03-28T18:09:51.483 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/wm8904.txt 2024-03-28T18:09:51.497 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/wm8960.txt 2024-03-28T18:09:51.511 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/wm8962.txt 2024-03-28T18:09:51.525 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/wm8994.txt 2024-03-28T18:09:51.540 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/spi/ 2024-03-28T18:09:51.558 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/spi/brcm,bcm2835-spi.txt 2024-03-28T18:09:51.568 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/spi/efm32-spi.txt 2024-03-28T18:09:51.585 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/spi/fsl-imx-cspi.txt 2024-03-28T18:09:51.602 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/spi/fsl-spi.txt 2024-03-28T18:09:51.617 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/spi/mxs-spi.txt 2024-03-28T18:09:51.632 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/spi/nvidia,tegra114-spi.txt 2024-03-28T18:09:51.647 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/spi/nvidia,tegra20-sflash.txt 2024-03-28T18:09:51.662 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/spi/nvidia,tegra20-slink.txt 2024-03-28T18:09:51.677 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/spi/omap-spi.txt 2024-03-28T18:09:51.693 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/spi/qcom,spi-qup.txt 2024-03-28T18:09:51.710 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/spi/sh-hspi.txt 2024-03-28T18:09:51.723 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/spi/sh-msiof.txt 2024-03-28T18:09:51.739 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/spi/snps,dw-apb-ssi.txt 2024-03-28T18:09:51.753 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/spi/spi-bus.txt 2024-03-28T18:09:51.768 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/spi/spi-cadence.txt 2024-03-28T18:09:51.784 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/spi/spi-davinci.txt 2024-03-28T18:09:51.801 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/spi/spi-dw.txt 2024-03-28T18:09:51.820 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/spi/spi-fsl-dspi.txt 2024-03-28T18:09:51.836 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/spi/spi-gpio.txt 2024-03-28T18:09:51.852 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/spi/spi-img-spfi.txt 2024-03-28T18:09:51.867 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/spi/spi-meson.txt 2024-03-28T18:09:51.881 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/spi/spi-octeon.txt 2024-03-28T18:09:51.898 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/spi/spi-orion.txt 2024-03-28T18:09:51.914 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/spi/spi-rockchip.txt 2024-03-28T18:09:51.930 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/spi/spi-rspi.txt 2024-03-28T18:09:51.946 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/spi/spi-samsung.txt 2024-03-28T18:09:51.962 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/spi/spi-sc18is602.txt 2024-03-28T18:09:51.980 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/spi/spi-sirf.txt 2024-03-28T18:09:51.994 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/spi/spi-st-ssc.txt 2024-03-28T18:09:52.009 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/spi/spi-sun4i.txt 2024-03-28T18:09:52.025 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/spi/spi-sun6i.txt 2024-03-28T18:09:52.041 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/spi/spi-xtensa-xtfpga.txt 2024-03-28T18:09:52.057 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/spi/spi_altera.txt 2024-03-28T18:09:52.071 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/spi/spi_atmel.txt 2024-03-28T18:09:52.086 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/spi/spi_oc_tiny.txt 2024-03-28T18:09:52.103 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/spi/spi_pl022.txt 2024-03-28T18:09:52.119 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/spi/ti_qspi.txt 2024-03-28T18:09:52.134 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/spmi/ 2024-03-28T18:09:52.153 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/spmi/qcom,spmi-pmic-arb.txt 2024-03-28T18:09:52.162 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/spmi/spmi.txt 2024-03-28T18:09:52.179 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/staging/ 2024-03-28T18:09:52.197 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/staging/iio/ 2024-03-28T18:09:52.204 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/staging/iio/adc/ 2024-03-28T18:09:52.211 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/staging/iio/adc/lpc32xx-adc.txt 2024-03-28T18:09:52.221 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/staging/iio/adc/mxs-lradc.txt 2024-03-28T18:09:52.243 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/staging/iio/adc/spear-adc.txt 2024-03-28T18:09:52.260 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/submitting-patches.txt 2024-03-28T18:09:52.294 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/thermal/ 2024-03-28T18:09:52.310 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/thermal/armada-thermal.txt 2024-03-28T18:09:52.319 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/thermal/db8500-thermal.txt 2024-03-28T18:09:52.338 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/thermal/dove-thermal.txt 2024-03-28T18:09:52.354 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/thermal/exynos-thermal.txt 2024-03-28T18:09:52.371 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/thermal/imx-thermal.txt 2024-03-28T18:09:52.392 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/thermal/kirkwood-thermal.txt 2024-03-28T18:09:52.435 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/thermal/rcar-thermal.txt 2024-03-28T18:09:52.459 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/thermal/rockchip-thermal.txt 2024-03-28T18:09:52.474 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/thermal/spear-thermal.txt 2024-03-28T18:09:52.490 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/thermal/st-thermal.txt 2024-03-28T18:09:52.505 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/thermal/tegra-soctherm.txt 2024-03-28T18:09:52.522 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/thermal/thermal.txt 2024-03-28T18:09:52.540 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/thermal/ti_soc_thermal.txt 2024-03-28T18:09:52.554 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/timer/ 2024-03-28T18:09:52.573 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/timer/allwinner,sun4i-timer.txt 2024-03-28T18:09:52.582 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/timer/allwinner,sun5i-a13-hstimer.txt 2024-03-28T18:09:52.600 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/timer/amlogic,meson6-timer.txt 2024-03-28T18:09:52.617 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/timer/arm,sp804.txt 2024-03-28T18:09:52.632 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/timer/brcm,bcm2835-system-timer.txt 2024-03-28T18:09:52.649 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/timer/cadence,ttc-timer.txt 2024-03-28T18:09:52.667 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/timer/cirrus,clps711x-timer.txt 2024-03-28T18:09:52.682 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/timer/digicolor-timer.txt 2024-03-28T18:09:52.699 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/timer/energymicro,efm32-timer.txt 2024-03-28T18:09:52.714 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/timer/fsl,ftm-timer.txt 2024-03-28T18:09:52.730 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/timer/fsl,imxgpt.txt 2024-03-28T18:09:52.745 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/timer/lsi,zevio-timer.txt 2024-03-28T18:09:52.762 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/timer/marvell,armada-370-xp-timer.txt 2024-03-28T18:09:52.779 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/timer/marvell,orion-timer.txt 2024-03-28T18:09:52.795 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/timer/mediatek,mtk-timer.txt 2024-03-28T18:09:52.811 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/timer/moxa,moxart-timer.txt 2024-03-28T18:09:52.827 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/timer/nvidia,tegra20-timer.txt 2024-03-28T18:09:52.843 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/timer/nvidia,tegra30-timer.txt 2024-03-28T18:09:52.859 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/timer/renesas,cmt.txt 2024-03-28T18:09:52.874 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/timer/renesas,mtu2.txt 2024-03-28T18:09:52.891 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/timer/renesas,tmu.txt 2024-03-28T18:09:52.908 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/timer/rockchip,rk3288-timer.txt 2024-03-28T18:09:52.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:09:52 smithi134 ceph-mon[35037]: pgmap v72: 97 pgs: 97 active+clean; 96 MiB data, 722 MiB used, 1.0 TiB / 1.0 TiB avail; 373 KiB/s rd, 545 KiB/s wr, 171 op/s 2024-03-28T18:09:52.925 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/timer/samsung,exynos4210-mct.txt 2024-03-28T18:09:52.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:09:52 smithi079 ceph-mon[35312]: pgmap v72: 97 pgs: 97 active+clean; 96 MiB data, 722 MiB used, 1.0 TiB / 1.0 TiB avail; 373 KiB/s rd, 545 KiB/s wr, 171 op/s 2024-03-28T18:09:52.941 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/timer/stericsson-u300-apptimer.txt 2024-03-28T18:09:52.958 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/timer/ti,keystone-timer.txt 2024-03-28T18:09:52.978 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/ufs/ 2024-03-28T18:09:52.997 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/ufs/ufshcd-pltfrm.txt 2024-03-28T18:09:53.008 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/unittest.txt 2024-03-28T18:09:53.031 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/usb/ 2024-03-28T18:09:53.031 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:09:52 smithi022 ceph-mon[31985]: pgmap v72: 97 pgs: 97 active+clean; 96 MiB data, 722 MiB used, 1.0 TiB / 1.0 TiB avail; 373 KiB/s rd, 545 KiB/s wr, 171 op/s 2024-03-28T18:09:53.044 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/usb/am33xx-usb.txt 2024-03-28T18:09:53.057 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/usb/atmel-usb.txt 2024-03-28T18:09:53.073 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/usb/ci-hdrc-imx.txt 2024-03-28T18:09:53.089 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/usb/ci-hdrc-qcom.txt 2024-03-28T18:09:53.103 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/usb/ci-hdrc-usb2.txt 2024-03-28T18:09:53.118 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/usb/ci-hdrc-zevio.txt 2024-03-28T18:09:53.135 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/usb/dwc2.txt 2024-03-28T18:09:53.153 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/usb/dwc3-st.txt 2024-03-28T18:09:53.168 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/usb/dwc3.txt 2024-03-28T18:09:53.184 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/usb/ehci-omap.txt 2024-03-28T18:09:53.199 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/usb/ehci-orion.txt 2024-03-28T18:09:53.215 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/usb/ehci-st.txt 2024-03-28T18:09:53.230 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/usb/exynos-usb.txt 2024-03-28T18:09:53.248 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/usb/fsl-usb.txt 2024-03-28T18:09:53.263 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/usb/generic.txt 2024-03-28T18:09:53.278 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/usb/gr-udc.txt 2024-03-28T18:09:53.294 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/usb/isp1301.txt 2024-03-28T18:09:53.309 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/usb/keystone-phy.txt 2024-03-28T18:09:53.327 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/usb/keystone-usb.txt 2024-03-28T18:09:53.344 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/usb/lpc32xx-udc.txt 2024-03-28T18:09:53.360 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/usb/msm-hsusb.txt 2024-03-28T18:09:53.374 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/usb/mxs-phy.txt 2024-03-28T18:09:53.389 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/usb/nvidia,tegra20-ehci.txt 2024-03-28T18:09:53.442 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/usb/nvidia,tegra20-usb-phy.txt 2024-03-28T18:09:53.464 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/usb/ohci-nxp.txt 2024-03-28T18:09:53.480 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/usb/ohci-omap3.txt 2024-03-28T18:09:53.495 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/usb/ohci-st.txt 2024-03-28T18:09:53.511 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/usb/omap-usb.txt 2024-03-28T18:09:53.527 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/usb/pxa-usb.txt 2024-03-28T18:09:53.546 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/usb/qcom,dwc3.txt 2024-03-28T18:09:53.563 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/usb/renesas_usbhs.txt 2024-03-28T18:09:53.578 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/usb/samsung-hsotg.txt 2024-03-28T18:09:53.594 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/usb/samsung-usbphy.txt 2024-03-28T18:09:53.609 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/usb/spear-usb.txt 2024-03-28T18:09:53.624 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/usb/twlxxxx-usb.txt 2024-03-28T18:09:53.640 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/usb/udc-xilinx.txt 2024-03-28T18:09:53.657 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/usb/usb-ehci.txt 2024-03-28T18:09:53.675 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/usb/usb-nop-xceiv.txt 2024-03-28T18:09:53.690 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/usb/usb-ohci.txt 2024-03-28T18:09:53.706 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/usb/usb-uhci.txt 2024-03-28T18:09:53.723 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/usb/usb-xhci.txt 2024-03-28T18:09:53.738 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/usb/usb3503.txt 2024-03-28T18:09:53.755 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/usb/usbmisc-imx.txt 2024-03-28T18:09:53.771 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/usb/ux500-usb.txt 2024-03-28T18:09:53.786 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/vendor-prefixes.txt 2024-03-28T18:09:53.820 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/video/ 2024-03-28T18:09:53.836 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/video/adi,adv7123.txt 2024-03-28T18:09:53.846 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/video/adi,adv7511.txt 2024-03-28T18:09:53.863 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/video/analog-tv-connector.txt 2024-03-28T18:09:53.879 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/video/arm,pl11x.txt 2024-03-28T18:09:53.897 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/video/atmel,lcdc.txt 2024-03-28T18:09:53.915 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/video/backlight/ 2024-03-28T18:09:53.927 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/video/backlight/88pm860x.txt 2024-03-28T18:09:53.936 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/video/backlight/gpio-backlight.txt 2024-03-28T18:09:53.953 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/video/backlight/lp855x.txt 2024-03-28T18:09:53.968 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/video/backlight/max8925-backlight.txt 2024-03-28T18:09:53.985 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/video/backlight/pwm-backlight.txt 2024-03-28T18:09:54.001 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/video/backlight/tps65217-backlight.txt 2024-03-28T18:09:54.017 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/video/bridge/ 2024-03-28T18:09:54.036 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/video/bridge/ps8622.txt 2024-03-28T18:09:54.045 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/video/bridge/ptn3460.txt 2024-03-28T18:09:54.064 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/video/cirrus,clps711x-fb.txt 2024-03-28T18:09:54.090 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/video/display-timing.txt 2024-03-28T18:09:54.106 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/video/dvi-connector.txt 2024-03-28T18:09:54.121 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/video/dw_hdmi-rockchip.txt 2024-03-28T18:09:54.138 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/video/exynos7-decon.txt 2024-03-28T18:09:54.155 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/video/exynos_dp.txt 2024-03-28T18:09:54.171 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/video/exynos_dsim.txt 2024-03-28T18:09:54.186 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/video/exynos_hdmi.txt 2024-03-28T18:09:54.202 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/video/exynos_hdmiddc.txt 2024-03-28T18:09:54.219 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/video/exynos_hdmiphy.txt 2024-03-28T18:09:54.234 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/video/exynos_mixer.txt 2024-03-28T18:09:54.248 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/video/fsl,imx-fb.txt 2024-03-28T18:09:54.263 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/video/hdmi-connector.txt 2024-03-28T18:09:54.278 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/video/lgphilips,lb035q02.txt 2024-03-28T18:09:54.302 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/video/panel-dpi.txt 2024-03-28T18:09:54.325 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/video/panel-dsi-cm.txt 2024-03-28T18:09:54.339 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/video/renesas,du.txt 2024-03-28T18:09:54.354 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/video/rockchip-drm.txt 2024-03-28T18:09:54.370 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/video/rockchip-vop.txt 2024-03-28T18:09:54.385 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/video/samsung-fimd.txt 2024-03-28T18:09:54.417 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/video/sharp,ls037v7dw01.txt 2024-03-28T18:09:54.436 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/video/simple-framebuffer-sunxi.txt 2024-03-28T18:09:54.452 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/video/simple-framebuffer.txt 2024-03-28T18:09:54.468 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/video/sony,acx565akm.txt 2024-03-28T18:09:54.485 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/video/ssd1289fb.txt 2024-03-28T18:09:54.501 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/video/ssd1307fb.txt 2024-03-28T18:09:54.516 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/video/thine,thc63lvdm83d 2024-03-28T18:09:54.531 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/video/ti,dra7-dss.txt 2024-03-28T18:09:54.877 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/video/ti,omap-dss.txt 2024-03-28T18:09:54.895 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/video/ti,omap2-dss.txt 2024-03-28T18:09:54.915 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/video/ti,omap3-dss.txt 2024-03-28T18:09:54.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:09:54 smithi134 ceph-mon[35037]: pgmap v73: 97 pgs: 97 active+clean; 96 MiB data, 722 MiB used, 1.0 TiB / 1.0 TiB avail; 256 KiB/s rd, 372 KiB/s wr, 111 op/s 2024-03-28T18:09:54.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:09:54 smithi079 ceph-mon[35312]: pgmap v73: 97 pgs: 97 active+clean; 96 MiB data, 722 MiB used, 1.0 TiB / 1.0 TiB avail; 256 KiB/s rd, 372 KiB/s wr, 111 op/s 2024-03-28T18:09:54.939 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/video/ti,omap4-dss.txt 2024-03-28T18:09:54.959 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/video/ti,omap5-dss.txt 2024-03-28T18:09:54.977 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/video/ti,opa362.txt 2024-03-28T18:09:54.992 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/video/ti,tfp410.txt 2024-03-28T18:09:55.003 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/video/ti,tpd12s015.txt 2024-03-28T18:09:55.020 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/video/toppoly,td028ttec1.txt 2024-03-28T18:09:55.031 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:09:54 smithi022 ceph-mon[31985]: pgmap v73: 97 pgs: 97 active+clean; 96 MiB data, 722 MiB used, 1.0 TiB / 1.0 TiB avail; 256 KiB/s rd, 372 KiB/s wr, 111 op/s 2024-03-28T18:09:55.044 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/video/tpo,td043mtea1.txt 2024-03-28T18:09:55.060 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/video/vga-connector.txt 2024-03-28T18:09:55.076 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/video/via,vt8500-fb.txt 2024-03-28T18:09:55.091 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/video/wm,prizm-ge-rops.txt 2024-03-28T18:09:55.106 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/video/wm,wm8505-fb.txt 2024-03-28T18:09:55.120 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/virtio/ 2024-03-28T18:09:55.141 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/virtio/mmio.txt 2024-03-28T18:09:55.151 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/w1/ 2024-03-28T18:09:55.174 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/w1/fsl-imx-owire.txt 2024-03-28T18:09:55.183 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/w1/omap-hdq.txt 2024-03-28T18:09:55.200 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/w1/w1-gpio.txt 2024-03-28T18:09:55.216 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/watchdog/ 2024-03-28T18:09:55.235 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/watchdog/atmel-at91rm9200-wdt.txt 2024-03-28T18:09:55.246 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/watchdog/atmel-wdt.txt 2024-03-28T18:09:55.264 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/watchdog/brcm,bcm2835-pm-wdog.txt 2024-03-28T18:09:55.279 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/watchdog/cadence-wdt.txt 2024-03-28T18:09:55.296 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/watchdog/davinci-wdt.txt 2024-03-28T18:09:55.310 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/watchdog/dw_wdt.txt 2024-03-28T18:09:55.330 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/watchdog/fsl-imx-wdt.txt 2024-03-28T18:09:55.347 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/watchdog/gpio-wdt.txt 2024-03-28T18:09:55.364 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/watchdog/imgpdc-wdt.txt 2024-03-28T18:09:55.380 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/watchdog/ingenic,jz4740-wdt.txt 2024-03-28T18:09:55.396 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/watchdog/marvel.txt 2024-03-28T18:09:55.411 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/watchdog/men-a021-wdt.txt 2024-03-28T18:09:55.462 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/watchdog/meson6-wdt.txt 2024-03-28T18:09:55.483 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/watchdog/moxa,moxart-watchdog.txt 2024-03-28T18:09:55.498 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/watchdog/mtk-wdt.txt 2024-03-28T18:09:55.512 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/watchdog/of-xilinx-wdt.txt 2024-03-28T18:09:55.528 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/watchdog/omap-wdt.txt 2024-03-28T18:09:55.544 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/watchdog/pnx4008-wdt.txt 2024-03-28T18:09:55.560 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/watchdog/qca-ar7130-wdt.txt 2024-03-28T18:09:55.576 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/watchdog/qcom-wdt.txt 2024-03-28T18:09:55.591 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/watchdog/rt2880-wdt.txt 2024-03-28T18:09:55.608 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/watchdog/samsung-wdt.txt 2024-03-28T18:09:55.623 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/watchdog/sirfsoc_wdt.txt 2024-03-28T18:09:55.638 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/watchdog/stericsson-coh901327.txt 2024-03-28T18:09:55.654 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/watchdog/sunxi-wdt.txt 2024-03-28T18:09:55.669 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/watchdog/twl4030-wdt.txt 2024-03-28T18:09:55.685 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/x86/ 2024-03-28T18:09:55.702 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/x86/ce4100.txt 2024-03-28T18:09:55.712 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/x86/interrupt.txt 2024-03-28T18:09:55.728 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/x86/timer.txt 2024-03-28T18:09:55.745 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/xilinx.txt 2024-03-28T18:09:55.767 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/xillybus/ 2024-03-28T18:09:55.780 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/bindings/xillybus/xillybus.txt 2024-03-28T18:09:55.789 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/booting-without-of.txt 2024-03-28T18:09:55.823 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/changesets.txt 2024-03-28T18:09:55.840 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/dynamic-resolution-notes.txt 2024-03-28T18:09:55.854 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/of_selftest.txt 2024-03-28T18:09:55.869 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/overlay-notes.txt 2024-03-28T18:09:55.884 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/todo.txt 2024-03-28T18:09:55.898 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/devicetree/usage-model.txt 2024-03-28T18:09:55.916 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/digsig.txt 2024-03-28T18:09:55.936 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/dma-buf-sharing.txt 2024-03-28T18:09:55.953 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/dmaengine/ 2024-03-28T18:09:55.964 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/dmaengine/00-INDEX 2024-03-28T18:09:55.971 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/dmaengine/client.txt 2024-03-28T18:09:55.988 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/dmaengine/dmatest.txt 2024-03-28T18:09:56.002 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/dmaengine/provider.txt 2024-03-28T18:09:56.018 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/dontdiff 2024-03-28T18:09:56.036 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/driver-model/ 2024-03-28T18:09:56.047 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/driver-model/binding.txt 2024-03-28T18:09:56.055 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/driver-model/bus.txt 2024-03-28T18:09:56.072 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/driver-model/class.txt 2024-03-28T18:09:56.087 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/driver-model/design-patterns.txt 2024-03-28T18:09:56.104 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/driver-model/device.txt 2024-03-28T18:09:56.114 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/driver-model/devres.txt 2024-03-28T18:09:56.131 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/driver-model/driver.txt 2024-03-28T18:09:56.147 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/driver-model/overview.txt 2024-03-28T18:09:56.160 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/driver-model/platform.txt 2024-03-28T18:09:56.174 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:09:55 smithi134 ceph-mon[35037]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-28T18:09:56.176 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/driver-model/porting.txt 2024-03-28T18:09:56.182 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:09:55 smithi079 ceph-mon[35312]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-28T18:09:56.192 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/dvb/ 2024-03-28T18:09:56.209 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/dvb/README.dvb-usb 2024-03-28T18:09:56.217 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/dvb/avermedia.txt 2024-03-28T18:09:56.235 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/dvb/bt8xx.txt 2024-03-28T18:09:56.250 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/dvb/cards.txt 2024-03-28T18:09:56.265 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/dvb/ci.txt 2024-03-28T18:09:56.281 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:09:55 smithi022 ceph-mon[31985]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-28T18:09:56.282 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/dvb/contributors.txt 2024-03-28T18:09:56.297 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/dvb/faq.txt 2024-03-28T18:09:56.312 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/dvb/get_dvb_firmware 2024-03-28T18:09:56.330 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/dvb/lmedm04.txt 2024-03-28T18:09:56.345 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/dvb/opera-firmware.txt 2024-03-28T18:09:56.359 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/dvb/readme.txt 2024-03-28T18:09:56.373 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/dvb/technisat.txt 2024-03-28T18:09:56.388 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/dvb/ttusb-dec.txt 2024-03-28T18:09:56.405 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/dvb/udev.txt 2024-03-28T18:09:56.419 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/dynamic-debug-howto.txt 2024-03-28T18:09:56.489 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/early-userspace/ 2024-03-28T18:09:56.503 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/early-userspace/README 2024-03-28T18:09:56.510 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/early-userspace/buffer-format.txt 2024-03-28T18:09:56.527 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/edac.txt 2024-03-28T18:09:56.549 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/efi-stub.txt 2024-03-28T18:09:56.564 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/eisa.txt 2024-03-28T18:09:56.578 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/email-clients.txt 2024-03-28T18:09:56.592 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/extcon/ 2024-03-28T18:09:56.603 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/extcon/porting-android-switch-class 2024-03-28T18:09:56.611 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/fault-injection/ 2024-03-28T18:09:56.628 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/fault-injection/fault-injection.txt 2024-03-28T18:09:56.637 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/fault-injection/notifier-error-inject.txt 2024-03-28T18:09:56.654 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/fault-injection/provoke-crashes.txt 2024-03-28T18:09:56.669 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/fb/ 2024-03-28T18:09:56.684 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/fb/00-INDEX 2024-03-28T18:09:56.693 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/fb/api.txt 2024-03-28T18:09:56.710 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/fb/arkfb.txt 2024-03-28T18:09:56.727 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/fb/aty128fb.txt 2024-03-28T18:09:56.741 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/fb/cirrusfb.txt 2024-03-28T18:09:56.755 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/fb/cmap_xfbdev.txt 2024-03-28T18:09:56.771 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/fb/deferred_io.txt 2024-03-28T18:09:56.786 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/fb/efifb.txt 2024-03-28T18:09:56.801 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/fb/ep93xx-fb.txt 2024-03-28T18:09:56.815 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/fb/fbcon.txt 2024-03-28T18:09:56.833 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/fb/framebuffer.txt 2024-03-28T18:09:56.848 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/fb/gxfb.txt 2024-03-28T18:09:56.862 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/fb/intel810.txt 2024-03-28T18:09:56.877 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/fb/intelfb.txt 2024-03-28T18:09:56.892 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/fb/internals.txt 2024-03-28T18:09:56.907 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/fb/lxfb.txt 2024-03-28T18:09:56.922 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/fb/matroxfb.txt 2024-03-28T18:09:56.940 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/fb/metronomefb.txt 2024-03-28T18:09:56.954 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/fb/modedb.txt 2024-03-28T18:09:56.969 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/fb/pvr2fb.txt 2024-03-28T18:09:56.983 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/fb/pxafb.txt 2024-03-28T18:09:56.997 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/fb/s3fb.txt 2024-03-28T18:09:57.011 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/fb/sa1100fb.txt 2024-03-28T18:09:57.026 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/fb/sh7760fb.txt 2024-03-28T18:09:57.040 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/fb/sisfb.txt 2024-03-28T18:09:57.055 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/fb/sm501.txt 2024-03-28T18:09:57.070 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/fb/sstfb.txt 2024-03-28T18:09:57.086 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/fb/tgafb.txt 2024-03-28T18:09:57.101 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/fb/tridentfb.txt 2024-03-28T18:09:57.117 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/fb/udlfb.txt 2024-03-28T18:09:57.130 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/fb/uvesafb.txt 2024-03-28T18:09:57.145 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/fb/vesafb.txt 2024-03-28T18:09:57.160 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/fb/viafb.modes 2024-03-28T18:09:57.174 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:09:56 smithi134 ceph-mon[35037]: pgmap v74: 97 pgs: 97 active+clean; 98 MiB data, 756 MiB used, 1.0 TiB / 1.0 TiB avail; 386 KiB/s rd, 563 KiB/s wr, 165 op/s 2024-03-28T18:09:57.177 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/fb/viafb.txt 2024-03-28T18:09:57.182 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:09:56 smithi079 ceph-mon[35312]: pgmap v74: 97 pgs: 97 active+clean; 98 MiB data, 756 MiB used, 1.0 TiB / 1.0 TiB avail; 386 KiB/s rd, 563 KiB/s wr, 165 op/s 2024-03-28T18:09:57.193 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/fb/vt8623fb.txt 2024-03-28T18:09:57.207 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/filesystems/ 2024-03-28T18:09:57.226 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/filesystems/.gitignore 2024-03-28T18:09:57.237 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/filesystems/00-INDEX 2024-03-28T18:09:57.255 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/filesystems/9p.txt 2024-03-28T18:09:57.268 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/filesystems/Locking 2024-03-28T18:09:57.281 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:09:56 smithi022 ceph-mon[31985]: pgmap v74: 97 pgs: 97 active+clean; 98 MiB data, 756 MiB used, 1.0 TiB / 1.0 TiB avail; 386 KiB/s rd, 563 KiB/s wr, 165 op/s 2024-03-28T18:09:57.287 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/filesystems/Makefile 2024-03-28T18:09:57.301 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/filesystems/adfs.txt 2024-03-28T18:09:57.319 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/filesystems/affs.txt 2024-03-28T18:09:57.335 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/filesystems/afs.txt 2024-03-28T18:09:57.351 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/filesystems/autofs4-mount-control.txt 2024-03-28T18:09:57.367 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/filesystems/autofs4.txt 2024-03-28T18:09:57.386 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/filesystems/automount-support.txt 2024-03-28T18:09:57.402 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/filesystems/befs.txt 2024-03-28T18:09:57.418 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/filesystems/bfs.txt 2024-03-28T18:09:57.436 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/filesystems/btrfs.txt 2024-03-28T18:09:57.481 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/filesystems/caching/ 2024-03-28T18:09:57.504 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/filesystems/caching/backend-api.txt 2024-03-28T18:09:57.515 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/filesystems/caching/cachefiles.txt 2024-03-28T18:09:57.533 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/filesystems/caching/fscache.txt 2024-03-28T18:09:57.552 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/filesystems/caching/netfs-api.txt 2024-03-28T18:09:57.572 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/filesystems/caching/object.txt 2024-03-28T18:09:57.590 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/filesystems/caching/operations.txt 2024-03-28T18:09:57.604 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/filesystems/ceph.txt 2024-03-28T18:09:57.625 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/filesystems/cifs/ 2024-03-28T18:09:57.636 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/filesystems/cifs/AUTHORS 2024-03-28T18:09:57.643 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/filesystems/cifs/CHANGES 2024-03-28T18:09:57.665 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/filesystems/cifs/README 2024-03-28T18:09:57.687 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/filesystems/cifs/TODO 2024-03-28T18:09:57.702 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/filesystems/cifs/cifs.txt 2024-03-28T18:09:57.718 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/filesystems/cifs/winucase_convert.pl 2024-03-28T18:09:57.735 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/filesystems/coda.txt 2024-03-28T18:09:57.760 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/filesystems/configfs/ 2024-03-28T18:09:57.772 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/filesystems/configfs/Makefile 2024-03-28T18:09:57.780 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/filesystems/configfs/configfs.txt 2024-03-28T18:09:57.801 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/filesystems/configfs/configfs_example_explicit.c 2024-03-28T18:09:57.819 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/filesystems/configfs/configfs_example_macros.c 2024-03-28T18:09:57.836 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/filesystems/cramfs.txt 2024-03-28T18:09:57.856 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/filesystems/dax.txt 2024-03-28T18:09:57.872 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/filesystems/debugfs.txt 2024-03-28T18:09:57.885 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/filesystems/devpts.txt 2024-03-28T18:09:57.901 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/filesystems/directory-locking 2024-03-28T18:09:57.916 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/filesystems/dlmfs.txt 2024-03-28T18:09:57.930 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/filesystems/dnotify.txt 2024-03-28T18:09:57.945 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/filesystems/dnotify_test.c 2024-03-28T18:09:57.959 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/filesystems/ecryptfs.txt 2024-03-28T18:09:57.973 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/filesystems/efivarfs.txt 2024-03-28T18:09:57.987 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/filesystems/exofs.txt 2024-03-28T18:09:58.003 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/filesystems/ext2.txt 2024-03-28T18:09:58.020 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/filesystems/ext3.txt 2024-03-28T18:09:58.035 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/filesystems/ext4.txt 2024-03-28T18:09:58.053 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/filesystems/f2fs.txt 2024-03-28T18:09:58.069 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/filesystems/fiemap.txt 2024-03-28T18:09:58.084 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/filesystems/files.txt 2024-03-28T18:09:58.100 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/filesystems/fuse.txt 2024-03-28T18:09:58.116 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/filesystems/gfs2-glocks.txt 2024-03-28T18:09:58.131 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/filesystems/gfs2-uevents.txt 2024-03-28T18:09:58.144 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/filesystems/gfs2.txt 2024-03-28T18:09:58.158 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/filesystems/hfs.txt 2024-03-28T18:09:58.173 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/filesystems/hfsplus.txt 2024-03-28T18:09:58.187 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/filesystems/hpfs.txt 2024-03-28T18:09:58.203 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/filesystems/inotify.txt 2024-03-28T18:09:58.218 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/filesystems/isofs.txt 2024-03-28T18:09:58.234 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/filesystems/jfs.txt 2024-03-28T18:09:58.248 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/filesystems/locks.txt 2024-03-28T18:09:58.262 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/filesystems/logfs.txt 2024-03-28T18:09:58.279 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/filesystems/mandatory-locking.txt 2024-03-28T18:09:58.294 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/filesystems/ncpfs.txt 2024-03-28T18:09:58.309 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/filesystems/nfs/ 2024-03-28T18:09:58.321 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/filesystems/nfs/00-INDEX 2024-03-28T18:09:58.331 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/filesystems/nfs/Exporting 2024-03-28T18:09:58.348 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/filesystems/nfs/fault_injection.txt 2024-03-28T18:09:58.363 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/filesystems/nfs/idmapper.txt 2024-03-28T18:09:58.379 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/filesystems/nfs/knfsd-stats.txt 2024-03-28T18:09:58.392 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/filesystems/nfs/nfs-rdma.txt 2024-03-28T18:09:58.411 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/filesystems/nfs/nfs.txt 2024-03-28T18:09:58.427 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/filesystems/nfs/nfs41-server.txt 2024-03-28T18:09:58.456 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/filesystems/nfs/nfsd-admin-interfaces.txt 2024-03-28T18:09:58.499 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/filesystems/nfs/nfsroot.txt 2024-03-28T18:09:58.519 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/filesystems/nfs/pnfs-block-server.txt 2024-03-28T18:09:58.533 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/filesystems/nfs/pnfs.txt 2024-03-28T18:09:58.549 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/filesystems/nfs/rpc-cache.txt 2024-03-28T18:09:58.566 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/filesystems/nfs/rpc-server-gss.txt 2024-03-28T18:09:58.582 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/filesystems/nilfs2.txt 2024-03-28T18:09:58.605 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/filesystems/ntfs.txt 2024-03-28T18:09:58.621 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/filesystems/ocfs2.txt 2024-03-28T18:09:58.635 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/filesystems/omfs.txt 2024-03-28T18:09:58.650 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/filesystems/overlayfs.txt 2024-03-28T18:09:58.665 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/filesystems/path-lookup.txt 2024-03-28T18:09:58.682 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/filesystems/pohmelfs/ 2024-03-28T18:09:58.693 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/filesystems/pohmelfs/design_notes.txt 2024-03-28T18:09:58.703 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/filesystems/pohmelfs/info.txt 2024-03-28T18:09:58.718 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/filesystems/pohmelfs/network_protocol.txt 2024-03-28T18:09:58.736 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/filesystems/porting 2024-03-28T18:09:58.760 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/filesystems/proc.txt 2024-03-28T18:09:58.786 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/filesystems/qnx6.txt 2024-03-28T18:09:58.801 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/filesystems/quota.txt 2024-03-28T18:09:58.817 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/filesystems/ramfs-rootfs-initramfs.txt 2024-03-28T18:09:58.832 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/filesystems/relay.txt 2024-03-28T18:09:58.849 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/filesystems/romfs.txt 2024-03-28T18:09:58.865 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/filesystems/seq_file.txt 2024-03-28T18:09:58.883 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/filesystems/sharedsubtree.txt 2024-03-28T18:09:58.901 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/filesystems/spufs.txt 2024-03-28T18:09:58.920 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/filesystems/squashfs.txt 2024-03-28T18:09:58.934 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/filesystems/sysfs-pci.txt 2024-03-28T18:09:58.948 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/filesystems/sysfs-tagging.txt 2024-03-28T18:09:58.962 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/filesystems/sysfs.txt 2024-03-28T18:09:58.981 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/filesystems/sysv-fs.txt 2024-03-28T18:09:58.998 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/filesystems/tmpfs.txt 2024-03-28T18:09:59.012 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/filesystems/ubifs.txt 2024-03-28T18:09:59.027 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/filesystems/udf.txt 2024-03-28T18:09:59.044 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/filesystems/ufs.txt 2024-03-28T18:09:59.059 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/filesystems/vfat.txt 2024-03-28T18:09:59.076 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/filesystems/vfs.txt 2024-03-28T18:09:59.098 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/filesystems/xfs-delayed-logging-design.txt 2024-03-28T18:09:59.121 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/filesystems/xfs-self-describing-metadata.txt 2024-03-28T18:09:59.137 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/filesystems/xfs.txt 2024-03-28T18:09:59.154 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/firmware_class/ 2024-03-28T18:09:59.170 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/firmware_class/README 2024-03-28T18:09:59.174 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:09:58 smithi134 ceph-mon[35037]: pgmap v75: 97 pgs: 97 active+clean; 98 MiB data, 756 MiB used, 1.0 TiB / 1.0 TiB avail; 252 KiB/s rd, 382 KiB/s wr, 112 op/s 2024-03-28T18:09:59.179 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/firmware_class/hotplug-script 2024-03-28T18:09:59.182 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:09:58 smithi079 ceph-mon[35312]: pgmap v75: 97 pgs: 97 active+clean; 98 MiB data, 756 MiB used, 1.0 TiB / 1.0 TiB avail; 252 KiB/s rd, 382 KiB/s wr, 112 op/s 2024-03-28T18:09:59.195 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/flexible-arrays.txt 2024-03-28T18:09:59.217 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/fmc/ 2024-03-28T18:09:59.229 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/fmc/00-INDEX 2024-03-28T18:09:59.236 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/fmc/API.txt 2024-03-28T18:09:59.253 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/fmc/FMC-and-SDB.txt 2024-03-28T18:09:59.268 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/fmc/carrier.txt 2024-03-28T18:09:59.281 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:09:58 smithi022 ceph-mon[31985]: pgmap v75: 97 pgs: 97 active+clean; 98 MiB data, 756 MiB used, 1.0 TiB / 1.0 TiB avail; 252 KiB/s rd, 382 KiB/s wr, 112 op/s 2024-03-28T18:09:59.288 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/fmc/fmc-chardev.txt 2024-03-28T18:09:59.301 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/fmc/fmc-fakedev.txt 2024-03-28T18:09:59.315 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/fmc/fmc-trivial.txt 2024-03-28T18:09:59.328 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/fmc/fmc-write-eeprom.txt 2024-03-28T18:09:59.343 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/fmc/identifiers.txt 2024-03-28T18:09:59.360 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/fmc/mezzanine.txt 2024-03-28T18:09:59.376 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/fmc/parameters.txt 2024-03-28T18:09:59.390 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/frv/ 2024-03-28T18:09:59.411 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/frv/README.txt 2024-03-28T18:09:59.418 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/frv/atomic-ops.txt 2024-03-28T18:09:59.435 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/frv/booting.txt 2024-03-28T18:09:59.449 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/frv/clock.txt 2024-03-28T18:09:59.509 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/frv/configuring.txt 2024-03-28T18:09:59.533 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/frv/features.txt 2024-03-28T18:09:59.548 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/frv/gdbinit 2024-03-28T18:09:59.563 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/frv/gdbstub.txt 2024-03-28T18:09:59.577 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/frv/kernel-ABI.txt 2024-03-28T18:09:59.593 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/frv/mmu-layout.txt 2024-03-28T18:09:59.608 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/futex-requeue-pi.txt 2024-03-28T18:09:59.628 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/gcov.txt 2024-03-28T18:09:59.642 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/gdb-kernel-debugging.txt 2024-03-28T18:09:59.657 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/gpio/ 2024-03-28T18:09:59.668 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/gpio/00-INDEX 2024-03-28T18:09:59.675 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/gpio/board.txt 2024-03-28T18:09:59.692 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/gpio/consumer.txt 2024-03-28T18:09:59.708 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/gpio/driver.txt 2024-03-28T18:09:59.727 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/gpio/gpio-legacy.txt 2024-03-28T18:09:59.745 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/gpio/gpio.txt 2024-03-28T18:09:59.761 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/gpio/sysfs.txt 2024-03-28T18:09:59.777 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/hid/ 2024-03-28T18:09:59.792 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/hid/hid-sensor.txt 2024-03-28T18:09:59.803 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/hid/hid-transport.txt 2024-03-28T18:09:59.828 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/hid/hiddev.txt 2024-03-28T18:09:59.845 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/hid/hidraw.txt 2024-03-28T18:09:59.863 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/hid/uhid.txt 2024-03-28T18:09:59.879 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/highuid.txt 2024-03-28T18:09:59.900 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/hsi.txt 2024-03-28T18:09:59.915 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/hw_random.txt 2024-03-28T18:09:59.931 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/hwmon/ 2024-03-28T18:09:59.942 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/hwmon/ab8500 2024-03-28T18:09:59.951 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/hwmon/abituguru 2024-03-28T18:09:59.966 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/hwmon/abituguru-datasheet 2024-03-28T18:09:59.984 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/hwmon/abituguru3 2024-03-28T18:09:59.995 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/hwmon/abx500 2024-03-28T18:10:00.009 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/hwmon/acpi_power_meter 2024-03-28T18:10:00.023 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/hwmon/ad7314 2024-03-28T18:10:00.057 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/hwmon/adc128d818 2024-03-28T18:10:00.071 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/hwmon/adm1021 2024-03-28T18:10:00.089 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/hwmon/adm1025 2024-03-28T18:10:00.104 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/hwmon/adm1026 2024-03-28T18:10:00.118 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/hwmon/adm1031 2024-03-28T18:10:00.133 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/hwmon/adm1275 2024-03-28T18:10:00.248 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/hwmon/adm9240 2024-03-28T18:10:00.437 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/hwmon/ads1015 2024-03-28T18:10:00.551 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/hwmon/ads7828 2024-03-28T18:10:00.583 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/hwmon/adt7410 2024-03-28T18:10:00.598 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/hwmon/adt7411 2024-03-28T18:10:00.612 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/hwmon/adt7462 2024-03-28T18:10:00.635 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/hwmon/adt7470 2024-03-28T18:10:00.641 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/hwmon/adt7475 2024-03-28T18:10:00.657 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/hwmon/amc6821 2024-03-28T18:10:00.672 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/hwmon/asb100 2024-03-28T18:10:00.686 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/hwmon/asc7621 2024-03-28T18:10:00.700 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/hwmon/coretemp 2024-03-28T18:10:00.715 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/hwmon/da9052 2024-03-28T18:10:00.730 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/hwmon/da9055 2024-03-28T18:10:00.746 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/hwmon/dme1737 2024-03-28T18:10:00.762 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/hwmon/ds1621 2024-03-28T18:10:00.777 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/hwmon/ds620 2024-03-28T18:10:00.794 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/hwmon/emc1403 2024-03-28T18:10:00.807 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/hwmon/emc2103 2024-03-28T18:10:00.822 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/hwmon/emc6w201 2024-03-28T18:10:00.836 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/hwmon/f71805f 2024-03-28T18:10:00.850 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/hwmon/f71882fg 2024-03-28T18:10:00.866 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/hwmon/fam15h_power 2024-03-28T18:10:00.882 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/hwmon/g760a 2024-03-28T18:10:00.896 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/hwmon/g762 2024-03-28T18:10:00.913 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/hwmon/gl518sm 2024-03-28T18:10:00.927 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/hwmon/hih6130 2024-03-28T18:10:00.942 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/hwmon/htu21 2024-03-28T18:10:00.956 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/hwmon/hwmon-kernel-api.txt 2024-03-28T18:10:00.972 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/hwmon/ibmaem 2024-03-28T18:10:00.986 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/hwmon/ibmpowernv 2024-03-28T18:10:01.001 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/hwmon/ina209 2024-03-28T18:10:01.015 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/hwmon/ina2xx 2024-03-28T18:10:01.030 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/hwmon/it87 2024-03-28T18:10:01.045 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/hwmon/jc42 2024-03-28T18:10:01.060 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/hwmon/k10temp 2024-03-28T18:10:01.075 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/hwmon/k8temp 2024-03-28T18:10:01.089 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/hwmon/lineage-pem 2024-03-28T18:10:01.105 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/hwmon/lm25066 2024-03-28T18:10:01.119 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/hwmon/lm63 2024-03-28T18:10:01.132 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/hwmon/lm70 2024-03-28T18:10:01.148 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/hwmon/lm73 2024-03-28T18:10:01.162 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/hwmon/lm75 2024-03-28T18:10:01.174 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:10:00 smithi134 ceph-mon[35037]: pgmap v76: 97 pgs: 97 active+clean; 98 MiB data, 756 MiB used, 1.0 TiB / 1.0 TiB avail; 252 KiB/s rd, 382 KiB/s wr, 112 op/s 2024-03-28T18:10:01.174 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:10:00 smithi134 ceph-mon[35037]: overall HEALTH_OK 2024-03-28T18:10:01.177 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/hwmon/lm77 2024-03-28T18:10:01.182 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:10:00 smithi079 ceph-mon[35312]: pgmap v76: 97 pgs: 97 active+clean; 98 MiB data, 756 MiB used, 1.0 TiB / 1.0 TiB avail; 252 KiB/s rd, 382 KiB/s wr, 112 op/s 2024-03-28T18:10:01.182 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:10:00 smithi079 ceph-mon[35312]: overall HEALTH_OK 2024-03-28T18:10:01.193 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/hwmon/lm78 2024-03-28T18:10:01.207 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/hwmon/lm80 2024-03-28T18:10:01.223 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/hwmon/lm83 2024-03-28T18:10:01.237 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/hwmon/lm85 2024-03-28T18:10:01.253 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/hwmon/lm87 2024-03-28T18:10:01.268 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/hwmon/lm90 2024-03-28T18:10:01.281 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:10:00 smithi022 ceph-mon[31985]: pgmap v76: 97 pgs: 97 active+clean; 98 MiB data, 756 MiB used, 1.0 TiB / 1.0 TiB avail; 252 KiB/s rd, 382 KiB/s wr, 112 op/s 2024-03-28T18:10:01.282 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:10:00 smithi022 ceph-mon[31985]: overall HEALTH_OK 2024-03-28T18:10:01.284 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/hwmon/lm92 2024-03-28T18:10:01.299 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/hwmon/lm93 2024-03-28T18:10:01.314 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/hwmon/lm95234 2024-03-28T18:10:01.329 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/hwmon/lm95245 2024-03-28T18:10:01.342 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/hwmon/ltc2945 2024-03-28T18:10:01.355 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/hwmon/ltc2978 2024-03-28T18:10:01.370 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/hwmon/ltc4151 2024-03-28T18:10:01.383 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/hwmon/ltc4215 2024-03-28T18:10:01.398 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/hwmon/ltc4245 2024-03-28T18:10:01.419 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/hwmon/ltc4260 2024-03-28T18:10:01.438 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/hwmon/ltc4261 2024-03-28T18:10:01.454 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/hwmon/max16064 2024-03-28T18:10:01.477 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/hwmon/max16065 2024-03-28T18:10:01.517 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/hwmon/max1619 2024-03-28T18:10:01.544 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/hwmon/max1668 2024-03-28T18:10:01.559 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/hwmon/max197 2024-03-28T18:10:01.572 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/hwmon/max34440 2024-03-28T18:10:01.586 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/hwmon/max6639 2024-03-28T18:10:01.601 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/hwmon/max6642 2024-03-28T18:10:01.615 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/hwmon/max6650 2024-03-28T18:10:01.631 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/hwmon/max6697 2024-03-28T18:10:01.645 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/hwmon/max8688 2024-03-28T18:10:01.659 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/hwmon/mc13783-adc 2024-03-28T18:10:01.673 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/hwmon/mcp3021 2024-03-28T18:10:01.689 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/hwmon/menf21bmc 2024-03-28T18:10:01.706 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/hwmon/nct6683 2024-03-28T18:10:01.721 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/hwmon/nct6775 2024-03-28T18:10:01.737 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/hwmon/nct7802 2024-03-28T18:10:01.752 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/hwmon/ntc_thermistor 2024-03-28T18:10:01.769 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/hwmon/pc87360 2024-03-28T18:10:01.783 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/hwmon/pc87427 2024-03-28T18:10:01.797 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/hwmon/pcf8591 2024-03-28T18:10:01.814 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/hwmon/pmbus 2024-03-28T18:10:01.826 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/hwmon/pmbus-core 2024-03-28T18:10:01.841 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/hwmon/powr1220 2024-03-28T18:10:01.855 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/hwmon/pwm-fan 2024-03-28T18:10:01.870 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/hwmon/sch5627 2024-03-28T18:10:01.884 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/hwmon/sch5636 2024-03-28T18:10:01.899 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/hwmon/sht15 2024-03-28T18:10:01.913 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/hwmon/sht21 2024-03-28T18:10:01.929 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/hwmon/shtc1 2024-03-28T18:10:01.944 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/hwmon/sis5595 2024-03-28T18:10:01.957 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/hwmon/smm665 2024-03-28T18:10:01.975 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/hwmon/smsc47b397 2024-03-28T18:10:01.990 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/hwmon/smsc47m1 2024-03-28T18:10:02.004 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/hwmon/smsc47m192 2024-03-28T18:10:02.017 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/hwmon/submitting-patches 2024-03-28T18:10:02.032 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/hwmon/sysfs-interface 2024-03-28T18:10:02.050 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/hwmon/thmc50 2024-03-28T18:10:02.064 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/hwmon/tmp102 2024-03-28T18:10:02.078 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/hwmon/tmp103 2024-03-28T18:10:02.092 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/hwmon/tmp401 2024-03-28T18:10:02.106 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/hwmon/tmp421 2024-03-28T18:10:02.122 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/hwmon/tps40422 2024-03-28T18:10:02.136 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/hwmon/twl4030-madc-hwmon 2024-03-28T18:10:02.150 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/hwmon/ucd9000 2024-03-28T18:10:02.167 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/hwmon/ucd9200 2024-03-28T18:10:02.181 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/hwmon/userspace-tools 2024-03-28T18:10:02.197 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/hwmon/vexpress 2024-03-28T18:10:02.212 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/hwmon/via686a 2024-03-28T18:10:02.227 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/hwmon/vt1211 2024-03-28T18:10:02.243 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/hwmon/w83627ehf 2024-03-28T18:10:02.259 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/hwmon/w83627hf 2024-03-28T18:10:02.274 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/hwmon/w83781d 2024-03-28T18:10:02.292 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/hwmon/w83791d 2024-03-28T18:10:02.309 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/hwmon/w83792d 2024-03-28T18:10:02.325 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/hwmon/w83793 2024-03-28T18:10:02.338 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/hwmon/w83795 2024-03-28T18:10:02.354 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/hwmon/w83l785ts 2024-03-28T18:10:02.367 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/hwmon/w83l786ng 2024-03-28T18:10:02.382 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/hwmon/wm831x 2024-03-28T18:10:02.397 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/hwmon/wm8350 2024-03-28T18:10:02.411 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/hwmon/zl6100 2024-03-28T18:10:02.427 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/hwspinlock.txt 2024-03-28T18:10:02.449 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/i2c/ 2024-03-28T18:10:02.463 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/i2c/busses/ 2024-03-28T18:10:02.469 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/i2c/busses/i2c-ali1535 2024-03-28T18:10:02.485 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/i2c/busses/i2c-ali1563 2024-03-28T18:10:02.559 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/i2c/busses/i2c-ali15x3 2024-03-28T18:10:02.580 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/i2c/busses/i2c-amd756 2024-03-28T18:10:02.595 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/i2c/busses/i2c-amd8111 2024-03-28T18:10:02.609 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/i2c/busses/i2c-diolan-u2c 2024-03-28T18:10:02.624 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/i2c/busses/i2c-i801 2024-03-28T18:10:02.641 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/i2c/busses/i2c-ismt 2024-03-28T18:10:02.657 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/i2c/busses/i2c-nforce2 2024-03-28T18:10:02.671 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/i2c/busses/i2c-ocores 2024-03-28T18:10:02.687 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/i2c/busses/i2c-parport 2024-03-28T18:10:02.703 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/i2c/busses/i2c-parport-light 2024-03-28T18:10:02.718 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/i2c/busses/i2c-pca-isa 2024-03-28T18:10:02.733 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/i2c/busses/i2c-piix4 2024-03-28T18:10:02.749 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/i2c/busses/i2c-sis5595 2024-03-28T18:10:02.764 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/i2c/busses/i2c-sis630 2024-03-28T18:10:02.780 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/i2c/busses/i2c-sis96x 2024-03-28T18:10:02.793 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/i2c/busses/i2c-taos-evm 2024-03-28T18:10:02.809 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/i2c/busses/i2c-via 2024-03-28T18:10:02.825 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/i2c/busses/i2c-viapro 2024-03-28T18:10:02.840 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/i2c/busses/scx200_acb 2024-03-28T18:10:02.855 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/i2c/dev-interface 2024-03-28T18:10:02.875 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/i2c/fault-codes 2024-03-28T18:10:02.892 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/i2c/functionality 2024-03-28T18:10:02.907 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/i2c/i2c-protocol 2024-03-28T18:10:02.923 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/i2c/i2c-stub 2024-03-28T18:10:02.936 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/i2c/instantiating-devices 2024-03-28T18:10:02.953 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/i2c/muxes/ 2024-03-28T18:10:02.965 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/i2c/muxes/i2c-mux-gpio 2024-03-28T18:10:02.975 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/i2c/old-module-parameters 2024-03-28T18:10:02.996 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/i2c/smbus-protocol 2024-03-28T18:10:03.013 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/i2c/summary 2024-03-28T18:10:03.027 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/i2c/ten-bit-addresses 2024-03-28T18:10:03.041 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/i2c/upgrading-clients 2024-03-28T18:10:03.057 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/i2c/writing-clients 2024-03-28T18:10:03.073 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/i2o/ 2024-03-28T18:10:03.089 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/i2o/README 2024-03-28T18:10:03.096 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/i2o/ioctl 2024-03-28T18:10:03.115 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/ia64/ 2024-03-28T18:10:03.132 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/ia64/.gitignore 2024-03-28T18:10:03.139 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/ia64/IRQ-redir.txt 2024-03-28T18:10:03.157 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/ia64/Makefile 2024-03-28T18:10:03.172 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/ia64/README 2024-03-28T18:10:03.174 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:10:02 smithi134 ceph-mon[35037]: pgmap v77: 97 pgs: 97 active+clean; 102 MiB data, 778 MiB used, 1.0 TiB / 1.0 TiB avail; 377 KiB/s rd, 678 KiB/s wr, 168 op/s 2024-03-28T18:10:03.182 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:10:02 smithi079 ceph-mon[35312]: pgmap v77: 97 pgs: 97 active+clean; 102 MiB data, 778 MiB used, 1.0 TiB / 1.0 TiB avail; 377 KiB/s rd, 678 KiB/s wr, 168 op/s 2024-03-28T18:10:03.187 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/ia64/aliasing-test.c 2024-03-28T18:10:03.202 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/ia64/aliasing.txt 2024-03-28T18:10:03.217 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/ia64/efirtc.txt 2024-03-28T18:10:03.234 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/ia64/err_inject.txt 2024-03-28T18:10:03.251 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/ia64/fsys.txt 2024-03-28T18:10:03.268 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/ia64/mca.txt 2024-03-28T18:10:03.281 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:10:02 smithi022 ceph-mon[31985]: pgmap v77: 97 pgs: 97 active+clean; 102 MiB data, 778 MiB used, 1.0 TiB / 1.0 TiB avail; 377 KiB/s rd, 678 KiB/s wr, 168 op/s 2024-03-28T18:10:03.285 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/ia64/serial.txt 2024-03-28T18:10:03.300 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/ia64/xen.txt 2024-03-28T18:10:03.315 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/ide/ 2024-03-28T18:10:03.331 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/ide/00-INDEX 2024-03-28T18:10:03.339 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/ide/ChangeLog.ide-cd.1994-2004 2024-03-28T18:10:03.358 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/ide/ChangeLog.ide-floppy.1996-2002 2024-03-28T18:10:03.375 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/ide/ChangeLog.ide-tape.1995-2002 2024-03-28T18:10:03.391 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/ide/ide-tape.txt 2024-03-28T18:10:03.408 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/ide/ide.txt 2024-03-28T18:10:03.424 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/ide/warm-plug-howto.txt 2024-03-28T18:10:03.440 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/infiniband/ 2024-03-28T18:10:03.459 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/infiniband/core_locking.txt 2024-03-28T18:10:03.467 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/infiniband/ipoib.txt 2024-03-28T18:10:03.492 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/infiniband/sysfs.txt 2024-03-28T18:10:03.538 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/infiniband/user_mad.txt 2024-03-28T18:10:03.564 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/infiniband/user_verbs.txt 2024-03-28T18:10:03.577 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/init.txt 2024-03-28T18:10:03.597 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/initrd.txt 2024-03-28T18:10:03.614 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/input/ 2024-03-28T18:10:03.624 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/input/alps.txt 2024-03-28T18:10:03.633 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/input/amijoy.txt 2024-03-28T18:10:03.649 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/input/appletouch.txt 2024-03-28T18:10:03.664 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/input/atarikbd.txt 2024-03-28T18:10:03.684 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/input/bcm5974.txt 2024-03-28T18:10:03.697 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/input/cd32.txt 2024-03-28T18:10:03.712 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/input/cma3000_d0x.txt 2024-03-28T18:10:03.726 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/input/cs461x.txt 2024-03-28T18:10:03.740 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/input/edt-ft5x06.txt 2024-03-28T18:10:03.754 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/input/elantech.txt 2024-03-28T18:10:03.771 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/input/event-codes.txt 2024-03-28T18:10:03.786 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/input/ff.txt 2024-03-28T18:10:03.801 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/input/gamepad.txt 2024-03-28T18:10:03.818 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/input/gameport-programming.txt 2024-03-28T18:10:03.832 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/input/gpio-tilt.txt 2024-03-28T18:10:03.846 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/input/iforce-protocol.txt 2024-03-28T18:10:03.861 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/input/input-programming.txt 2024-03-28T18:10:03.876 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/input/input.txt 2024-03-28T18:10:03.891 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/input/interactive.fig 2024-03-28T18:10:03.906 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/input/joystick-api.txt 2024-03-28T18:10:03.926 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/input/joystick-parport.txt 2024-03-28T18:10:03.943 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/input/joystick.txt 2024-03-28T18:10:03.961 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/input/multi-touch-protocol.txt 2024-03-28T18:10:03.978 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/input/notifier.txt 2024-03-28T18:10:03.993 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/input/ntrig.txt 2024-03-28T18:10:04.008 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/input/rotary-encoder.txt 2024-03-28T18:10:04.022 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/input/sentelic.txt 2024-03-28T18:10:04.040 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/input/shape.fig 2024-03-28T18:10:04.053 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/input/walkera0701.txt 2024-03-28T18:10:04.067 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/input/xpad.txt 2024-03-28T18:10:04.084 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/input/yealink.txt 2024-03-28T18:10:04.101 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/intel_txt.txt 2024-03-28T18:10:04.121 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/io-mapping.txt 2024-03-28T18:10:04.140 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/io_ordering.txt 2024-03-28T18:10:04.155 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/ioctl/ 2024-03-28T18:10:04.165 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/ioctl/00-INDEX 2024-03-28T18:10:04.174 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/ioctl/botching-up-ioctls.txt 2024-03-28T18:10:04.191 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/ioctl/cdrom.txt 2024-03-28T18:10:04.209 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/ioctl/hdio.txt 2024-03-28T18:10:04.228 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/ioctl/ioctl-decoding.txt 2024-03-28T18:10:04.244 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/ioctl/ioctl-number.txt 2024-03-28T18:10:04.263 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/iostats.txt 2024-03-28T18:10:04.281 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:10:03 smithi022 ceph-mon[31985]: pgmap v78: 97 pgs: 97 active+clean; 102 MiB data, 778 MiB used, 1.0 TiB / 1.0 TiB avail; 254 KiB/s rd, 487 KiB/s wr, 110 op/s 2024-03-28T18:10:04.283 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/irqflags-tracing.txt 2024-03-28T18:10:04.323 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/isapnp.txt 2024-03-28T18:10:04.342 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/isdn/ 2024-03-28T18:10:04.355 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/isdn/00-INDEX 2024-03-28T18:10:04.396 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/isdn/CREDITS 2024-03-28T18:10:04.408 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/isdn/HiSax.cert 2024-03-28T18:10:04.424 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:10:03 smithi134 ceph-mon[35037]: pgmap v78: 97 pgs: 97 active+clean; 102 MiB data, 778 MiB used, 1.0 TiB / 1.0 TiB avail; 254 KiB/s rd, 487 KiB/s wr, 110 op/s 2024-03-28T18:10:04.427 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/isdn/INTERFACE 2024-03-28T18:10:04.432 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:10:03 smithi079 ceph-mon[35312]: pgmap v78: 97 pgs: 97 active+clean; 102 MiB data, 778 MiB used, 1.0 TiB / 1.0 TiB avail; 254 KiB/s rd, 487 KiB/s wr, 110 op/s 2024-03-28T18:10:04.461 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/isdn/INTERFACE.CAPI 2024-03-28T18:10:04.482 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/isdn/INTERFACE.fax 2024-03-28T18:10:04.517 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/isdn/README 2024-03-28T18:10:04.540 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/isdn/README.FAQ 2024-03-28T18:10:04.563 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/isdn/README.HiSax 2024-03-28T18:10:04.576 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/isdn/README.act2000 2024-03-28T18:10:04.593 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/isdn/README.audio 2024-03-28T18:10:04.621 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/isdn/README.avmb1 2024-03-28T18:10:04.634 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/isdn/README.concap 2024-03-28T18:10:04.652 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/isdn/README.diversion 2024-03-28T18:10:04.674 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/isdn/README.fax 2024-03-28T18:10:04.689 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/isdn/README.gigaset 2024-03-28T18:10:04.705 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/isdn/README.hfc-pci 2024-03-28T18:10:04.721 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/isdn/README.hysdn 2024-03-28T18:10:04.736 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/isdn/README.icn 2024-03-28T18:10:04.747 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/isdn/README.mISDN 2024-03-28T18:10:04.759 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/isdn/README.pcbit 2024-03-28T18:10:04.773 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/isdn/README.sc 2024-03-28T18:10:04.794 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/isdn/README.syncppp 2024-03-28T18:10:04.811 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/isdn/README.x25 2024-03-28T18:10:04.826 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/isdn/syncPPP.FAQ 2024-03-28T18:10:04.843 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/ja_JP/ 2024-03-28T18:10:04.863 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/ja_JP/HOWTO 2024-03-28T18:10:04.877 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/ja_JP/SubmitChecklist 2024-03-28T18:10:04.892 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/ja_JP/SubmittingPatches 2024-03-28T18:10:04.952 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/ja_JP/stable_api_nonsense.txt 2024-03-28T18:10:04.975 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/ja_JP/stable_kernel_rules.txt 2024-03-28T18:10:04.986 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/java.txt 2024-03-28T18:10:05.006 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/kasan.txt 2024-03-28T18:10:05.024 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/kbuild/ 2024-03-28T18:10:05.035 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/kbuild/00-INDEX 2024-03-28T18:10:05.042 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/kbuild/headers_install.txt 2024-03-28T18:10:05.051 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/kbuild/kbuild.txt 2024-03-28T18:10:05.067 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/kbuild/kconfig-language.txt 2024-03-28T18:10:05.081 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/kbuild/kconfig.txt 2024-03-28T18:10:05.096 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/kbuild/makefiles.txt 2024-03-28T18:10:05.118 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/kbuild/modules.txt 2024-03-28T18:10:05.135 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/kdump/ 2024-03-28T18:10:05.150 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/kdump/gdbmacros.txt 2024-03-28T18:10:05.158 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/kdump/kdump.txt 2024-03-28T18:10:05.178 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/kernel-doc-nano-HOWTO.txt 2024-03-28T18:10:05.198 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/kernel-docs.txt 2024-03-28T18:10:05.216 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/kernel-parameters.txt 2024-03-28T18:10:05.245 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/kernel-per-CPU-kthreads.txt 2024-03-28T18:10:05.260 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/kmemcheck.txt 2024-03-28T18:10:05.277 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/kmemleak.txt 2024-03-28T18:10:05.293 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/ko_KR/ 2024-03-28T18:10:05.304 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/ko_KR/HOWTO 2024-03-28T18:10:05.318 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/ko_KR/stable_api_nonsense.txt 2024-03-28T18:10:05.336 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/kobject.txt 2024-03-28T18:10:05.357 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/kprobes.txt 2024-03-28T18:10:05.375 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/kref.txt 2024-03-28T18:10:05.388 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/kselftest.txt 2024-03-28T18:10:05.402 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/laptops/ 2024-03-28T18:10:05.413 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/laptops/.gitignore 2024-03-28T18:10:05.420 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/laptops/00-INDEX 2024-03-28T18:10:05.438 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/laptops/Makefile 2024-03-28T18:10:05.455 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/laptops/asus-laptop.txt 2024-03-28T18:10:05.469 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/laptops/disk-shock-protection.txt 2024-03-28T18:10:05.488 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/laptops/dslm.c 2024-03-28T18:10:05.504 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/laptops/freefall.c 2024-03-28T18:10:05.532 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/laptops/laptop-mode.txt 2024-03-28T18:10:05.551 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/laptops/sony-laptop.txt 2024-03-28T18:10:05.567 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/laptops/sonypi.txt 2024-03-28T18:10:05.581 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/laptops/thinkpad-acpi.txt 2024-03-28T18:10:05.601 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/laptops/toshiba_haps.txt 2024-03-28T18:10:05.615 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/ldm.txt 2024-03-28T18:10:05.634 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/leds/ 2024-03-28T18:10:05.646 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/leds/00-INDEX 2024-03-28T18:10:05.654 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/leds/leds-blinkm.txt 2024-03-28T18:10:05.670 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/leds/leds-class.txt 2024-03-28T18:10:05.684 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/leds/leds-lm3556.txt 2024-03-28T18:10:05.700 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/leds/leds-lp3944.txt 2024-03-28T18:10:05.714 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/leds/leds-lp5521.txt 2024-03-28T18:10:05.727 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/leds/leds-lp5523.txt 2024-03-28T18:10:05.745 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/leds/leds-lp5562.txt 2024-03-28T18:10:05.760 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/leds/leds-lp55xx.txt 2024-03-28T18:10:05.775 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/leds/ledtrig-oneshot.txt 2024-03-28T18:10:05.788 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/leds/ledtrig-transient.txt 2024-03-28T18:10:05.804 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/local_ops.txt 2024-03-28T18:10:05.831 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/locking/ 2024-03-28T18:10:05.843 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/locking/00-INDEX 2024-03-28T18:10:05.851 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/locking/lglock.txt 2024-03-28T18:10:05.867 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/locking/lockdep-design.txt 2024-03-28T18:10:05.882 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/locking/lockstat.txt 2024-03-28T18:10:05.901 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/locking/locktorture.txt 2024-03-28T18:10:05.917 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/locking/mutex-design.txt 2024-03-28T18:10:05.932 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/locking/rt-mutex-design.txt 2024-03-28T18:10:05.952 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/locking/rt-mutex.txt 2024-03-28T18:10:05.966 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/locking/spinlocks.txt 2024-03-28T18:10:05.982 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/locking/ww-mutex-design.txt 2024-03-28T18:10:05.998 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/lockup-watchdogs.txt 2024-03-28T18:10:06.021 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/logo.gif 2024-03-28T18:10:06.043 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/logo.txt 2024-03-28T18:10:06.058 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/lzo.txt 2024-03-28T18:10:06.074 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/m68k/ 2024-03-28T18:10:06.085 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/m68k/00-INDEX 2024-03-28T18:10:06.093 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/m68k/README.buddha 2024-03-28T18:10:06.111 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/m68k/kernel-options.txt 2024-03-28T18:10:06.132 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/magic-number.txt 2024-03-28T18:10:06.152 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/mailbox.txt 2024-03-28T18:10:06.172 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/md.txt 2024-03-28T18:10:06.189 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/media-framework.txt 2024-03-28T18:10:06.203 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/memory-barriers.txt 2024-03-28T18:10:06.231 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/memory-devices/ 2024-03-28T18:10:06.242 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/memory-devices/ti-emif.txt 2024-03-28T18:10:06.251 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/memory-hotplug.txt 2024-03-28T18:10:06.273 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/metag/ 2024-03-28T18:10:06.283 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/metag/00-INDEX 2024-03-28T18:10:06.291 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/metag/kernel-ABI.txt 2024-03-28T18:10:06.309 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/mic/ 2024-03-28T18:10:06.326 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/mic/Makefile 2024-03-28T18:10:06.334 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/mic/mic_overview.txt 2024-03-28T18:10:06.352 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/mic/mpssd/ 2024-03-28T18:10:06.362 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/mic/mpssd/.gitignore 2024-03-28T18:10:06.371 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/mic/mpssd/Makefile 2024-03-28T18:10:06.387 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/mic/mpssd/micctrl 2024-03-28T18:10:06.402 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/mic/mpssd/mpss 2024-03-28T18:10:06.417 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/mic/mpssd/mpssd.c 2024-03-28T18:10:06.424 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:10:06 smithi134 ceph-mon[35037]: pgmap v79: 97 pgs: 97 active+clean; 105 MiB data, 795 MiB used, 1.0 TiB / 1.0 TiB avail; 445 KiB/s rd, 836 KiB/s wr, 177 op/s 2024-03-28T18:10:06.432 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:10:06 smithi079 ceph-mon[35312]: pgmap v79: 97 pgs: 97 active+clean; 105 MiB data, 795 MiB used, 1.0 TiB / 1.0 TiB avail; 445 KiB/s rd, 836 KiB/s wr, 177 op/s 2024-03-28T18:10:06.439 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/mic/mpssd/mpssd.h 2024-03-28T18:10:06.454 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/mic/mpssd/sysfs.c 2024-03-28T18:10:06.469 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/mips/ 2024-03-28T18:10:06.491 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/mips/00-INDEX 2024-03-28T18:10:06.498 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/mips/AU1xxx_IDE.README 2024-03-28T18:10:06.523 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/misc-devices/ 2024-03-28T18:10:06.531 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:10:06 smithi022 ceph-mon[31985]: pgmap v79: 97 pgs: 97 active+clean; 105 MiB data, 795 MiB used, 1.0 TiB / 1.0 TiB avail; 445 KiB/s rd, 836 KiB/s wr, 177 op/s 2024-03-28T18:10:06.539 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/misc-devices/Makefile 2024-03-28T18:10:06.547 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/misc-devices/ad525x_dpot.txt 2024-03-28T18:10:06.563 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/misc-devices/apds990x.txt 2024-03-28T18:10:06.577 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/misc-devices/bh1770glc.txt 2024-03-28T18:10:06.593 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/misc-devices/c2port.txt 2024-03-28T18:10:06.607 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/misc-devices/eeprom 2024-03-28T18:10:06.621 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/misc-devices/ics932s401 2024-03-28T18:10:06.636 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/misc-devices/isl29003 2024-03-28T18:10:06.653 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/misc-devices/lis3lv02d 2024-03-28T18:10:06.667 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/misc-devices/max6875 2024-03-28T18:10:06.683 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/misc-devices/mei/ 2024-03-28T18:10:06.695 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/misc-devices/mei/.gitignore 2024-03-28T18:10:06.703 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/misc-devices/mei/Makefile 2024-03-28T18:10:06.720 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/misc-devices/mei/TODO 2024-03-28T18:10:06.737 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/misc-devices/mei/mei-amt-version.c 2024-03-28T18:10:06.756 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/misc-devices/mei/mei-client-bus.txt 2024-03-28T18:10:06.772 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/misc-devices/mei/mei.txt 2024-03-28T18:10:06.788 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/misc-devices/spear-pcie-gadget.txt 2024-03-28T18:10:06.808 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/mmc/ 2024-03-28T18:10:06.823 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/mmc/00-INDEX 2024-03-28T18:10:06.832 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/mmc/mmc-async-req.txt 2024-03-28T18:10:06.851 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/mmc/mmc-dev-attrs.txt 2024-03-28T18:10:06.865 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/mmc/mmc-dev-parts.txt 2024-03-28T18:10:06.879 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/mn10300/ 2024-03-28T18:10:06.894 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/mn10300/ABI.txt 2024-03-28T18:10:06.903 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/mn10300/compartmentalisation.txt 2024-03-28T18:10:06.918 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/module-signing.txt 2024-03-28T18:10:06.938 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/mono.txt 2024-03-28T18:10:06.952 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/mtd/ 2024-03-28T18:10:06.963 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/mtd/nand/ 2024-03-28T18:10:06.969 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/mtd/nand/pxa3xx-nand.txt 2024-03-28T18:10:06.978 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/mtd/nand_ecc.txt 2024-03-28T18:10:07.004 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/mtd/spi-nor.txt 2024-03-28T18:10:07.020 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/namespaces/ 2024-03-28T18:10:07.038 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/namespaces/compatibility-list.txt 2024-03-28T18:10:07.045 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/namespaces/resource-control.txt 2024-03-28T18:10:07.063 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/netlabel/ 2024-03-28T18:10:07.080 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/netlabel/00-INDEX 2024-03-28T18:10:07.088 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/netlabel/cipso_ipv4.txt 2024-03-28T18:10:07.103 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/netlabel/draft-ietf-cipso-ipsecurity-01.txt 2024-03-28T18:10:07.121 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/netlabel/introduction.txt 2024-03-28T18:10:07.135 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/netlabel/lsm_interface.txt 2024-03-28T18:10:07.150 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/networking/ 2024-03-28T18:10:07.168 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/networking/00-INDEX 2024-03-28T18:10:07.177 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/networking/3c509.txt 2024-03-28T18:10:07.194 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/networking/6pack.txt 2024-03-28T18:10:07.211 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/networking/LICENSE.qla3xxx 2024-03-28T18:10:07.227 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/networking/LICENSE.qlcnic 2024-03-28T18:10:07.244 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/networking/LICENSE.qlge 2024-03-28T18:10:07.261 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/networking/Makefile 2024-03-28T18:10:07.275 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/networking/PLIP.txt 2024-03-28T18:10:07.291 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/networking/README.ipw2100 2024-03-28T18:10:07.307 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/networking/README.ipw2200 2024-03-28T18:10:07.322 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/networking/README.sb1000 2024-03-28T18:10:07.337 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/networking/alias.txt 2024-03-28T18:10:07.352 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/networking/altera_tse.txt 2024-03-28T18:10:07.368 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/networking/arcnet-hardware.txt 2024-03-28T18:10:07.398 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/networking/arcnet.txt 2024-03-28T18:10:07.415 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/networking/atm.txt 2024-03-28T18:10:07.432 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/networking/ax25.txt 2024-03-28T18:10:07.447 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/networking/batman-adv.txt 2024-03-28T18:10:07.462 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/networking/baycom.txt 2024-03-28T18:10:07.478 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/networking/bonding.txt 2024-03-28T18:10:07.516 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/networking/bridge.txt 2024-03-28T18:10:07.533 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/networking/caif/ 2024-03-28T18:10:07.542 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/networking/caif/Linux-CAIF.txt 2024-03-28T18:10:07.552 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/networking/caif/README 2024-03-28T18:10:07.567 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/networking/caif/spi_porting.txt 2024-03-28T18:10:07.582 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/networking/can.txt 2024-03-28T18:10:07.608 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/networking/cdc_mbim.txt 2024-03-28T18:10:07.627 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/networking/cops.txt 2024-03-28T18:10:07.641 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/networking/cs89x0.txt 2024-03-28T18:10:07.659 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/networking/cxacru-cf.py 2024-03-28T18:10:07.675 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/networking/cxacru.txt 2024-03-28T18:10:07.690 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/networking/cxgb.txt 2024-03-28T18:10:07.705 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/networking/dccp.txt 2024-03-28T18:10:07.722 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/networking/dctcp.txt 2024-03-28T18:10:07.737 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/networking/de4x5.txt 2024-03-28T18:10:07.754 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/networking/decnet.txt 2024-03-28T18:10:07.770 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/networking/dl2k.txt 2024-03-28T18:10:07.787 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/networking/dm9000.txt 2024-03-28T18:10:07.802 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/networking/dmfe.txt 2024-03-28T18:10:07.817 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/networking/dns_resolver.txt 2024-03-28T18:10:07.831 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/networking/driver.txt 2024-03-28T18:10:07.849 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/networking/e100.txt 2024-03-28T18:10:07.864 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/networking/e1000.txt 2024-03-28T18:10:07.882 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/networking/e1000e.txt 2024-03-28T18:10:07.898 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/networking/eql.txt 2024-03-28T18:10:07.915 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/networking/fib_trie.txt 2024-03-28T18:10:07.932 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/networking/filter.txt 2024-03-28T18:10:07.951 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/networking/fore200e.txt 2024-03-28T18:10:07.966 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/networking/framerelay.txt 2024-03-28T18:10:07.981 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/networking/gen_stats.txt 2024-03-28T18:10:07.996 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/networking/generic-hdlc.txt 2024-03-28T18:10:08.013 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/networking/generic_netlink.txt 2024-03-28T18:10:08.028 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/networking/gianfar.txt 2024-03-28T18:10:08.043 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/networking/i40e.txt 2024-03-28T18:10:08.058 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/networking/i40evf.txt 2024-03-28T18:10:08.073 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/networking/ieee802154.txt 2024-03-28T18:10:08.087 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/networking/igb.txt 2024-03-28T18:10:08.104 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/networking/igbvf.txt 2024-03-28T18:10:08.118 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/networking/ip-sysctl.txt 2024-03-28T18:10:08.141 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/networking/ip_dynaddr.txt 2024-03-28T18:10:08.156 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/networking/ipddp.txt 2024-03-28T18:10:08.173 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/networking/iphase.txt 2024-03-28T18:10:08.191 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/networking/ipsec.txt 2024-03-28T18:10:08.207 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/networking/ipv6.txt 2024-03-28T18:10:08.224 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/networking/ipvlan.txt 2024-03-28T18:10:08.239 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/networking/ipvs-sysctl.txt 2024-03-28T18:10:08.255 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/networking/irda.txt 2024-03-28T18:10:08.269 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/networking/ixgb.txt 2024-03-28T18:10:08.287 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/networking/ixgbe.txt 2024-03-28T18:10:08.304 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/networking/ixgbevf.txt 2024-03-28T18:10:08.319 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/networking/l2tp.txt 2024-03-28T18:10:08.333 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/networking/lapb-module.txt 2024-03-28T18:10:08.348 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/networking/ltpc.txt 2024-03-28T18:10:08.364 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/networking/mac80211-auth-assoc-deauth.txt 2024-03-28T18:10:08.380 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/networking/mac80211-injection.txt 2024-03-28T18:10:08.394 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/networking/mac80211_hwsim/ 2024-03-28T18:10:08.405 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/networking/mac80211_hwsim/README 2024-03-28T18:10:08.414 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/networking/mac80211_hwsim/hostapd.conf 2024-03-28T18:10:08.424 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:10:08 smithi134 ceph-mon[35037]: pgmap v80: 97 pgs: 97 active+clean; 105 MiB data, 795 MiB used, 1.0 TiB / 1.0 TiB avail; 316 KiB/s rd, 645 KiB/s wr, 123 op/s 2024-03-28T18:10:08.431 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/networking/mac80211_hwsim/wpa_supplicant.conf 2024-03-28T18:10:08.432 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:10:08 smithi079 ceph-mon[35312]: pgmap v80: 97 pgs: 97 active+clean; 105 MiB data, 795 MiB used, 1.0 TiB / 1.0 TiB avail; 316 KiB/s rd, 645 KiB/s wr, 123 op/s 2024-03-28T18:10:08.446 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/networking/multiqueue.txt 2024-03-28T18:10:08.466 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/networking/netconsole.txt 2024-03-28T18:10:08.483 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/networking/netdev-FAQ.txt 2024-03-28T18:10:08.498 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/networking/netdev-features.txt 2024-03-28T18:10:08.527 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/networking/netdevices.txt 2024-03-28T18:10:08.531 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:10:08 smithi022 ceph-mon[31985]: pgmap v80: 97 pgs: 97 active+clean; 105 MiB data, 795 MiB used, 1.0 TiB / 1.0 TiB avail; 316 KiB/s rd, 645 KiB/s wr, 123 op/s 2024-03-28T18:10:08.568 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/networking/netif-msg.txt 2024-03-28T18:10:08.596 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/networking/netlink_mmap.txt 2024-03-28T18:10:08.611 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/networking/nf_conntrack-sysctl.txt 2024-03-28T18:10:08.624 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/networking/nfc.txt 2024-03-28T18:10:08.639 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/networking/openvswitch.txt 2024-03-28T18:10:08.654 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/networking/operstates.txt 2024-03-28T18:10:08.669 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/networking/packet_mmap.txt 2024-03-28T18:10:08.688 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/networking/phonet.txt 2024-03-28T18:10:08.706 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/networking/phy.txt 2024-03-28T18:10:08.724 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/networking/pktgen.txt 2024-03-28T18:10:08.741 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/networking/policy-routing.txt 2024-03-28T18:10:08.758 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/networking/ppp_generic.txt 2024-03-28T18:10:08.772 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/networking/proc_net_tcp.txt 2024-03-28T18:10:08.787 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/networking/radiotap-headers.txt 2024-03-28T18:10:08.802 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/networking/ray_cs.txt 2024-03-28T18:10:08.819 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/networking/rds.txt 2024-03-28T18:10:08.835 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/networking/regulatory.txt 2024-03-28T18:10:08.849 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/networking/rxrpc.txt 2024-03-28T18:10:08.870 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/networking/s2io.txt 2024-03-28T18:10:08.885 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/networking/scaling.txt 2024-03-28T18:10:08.903 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/networking/sctp.txt 2024-03-28T18:10:08.918 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/networking/secid.txt 2024-03-28T18:10:08.930 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-28T18:10:08.930 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-28T18:10:08.940 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/networking/skfp.txt 2024-03-28T18:10:08.947 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/networking/smc9.txt 2024-03-28T18:10:08.961 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/networking/spider_net.txt 2024-03-28T18:10:08.972 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/networking/stmmac.txt 2024-03-28T18:10:08.983 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/networking/switchdev.txt 2024-03-28T18:10:08.993 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/networking/tc-actions-env-rules.txt 2024-03-28T18:10:09.004 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/networking/tcp-thin.txt 2024-03-28T18:10:09.015 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/networking/tcp.txt 2024-03-28T18:10:09.027 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/networking/team.txt 2024-03-28T18:10:09.038 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/networking/timestamping.txt 2024-03-28T18:10:09.054 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/networking/timestamping/ 2024-03-28T18:10:09.062 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/networking/timestamping/.gitignore 2024-03-28T18:10:09.068 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/networking/timestamping/Makefile 2024-03-28T18:10:09.081 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/networking/timestamping/hwtstamp_config.c 2024-03-28T18:10:09.094 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/networking/timestamping/timestamping.c 2024-03-28T18:10:09.108 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/networking/timestamping/txtimestamp.c 2024-03-28T18:10:09.125 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/networking/tlan.txt 2024-03-28T18:10:09.146 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/networking/tproxy.txt 2024-03-28T18:10:09.160 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/networking/tuntap.txt 2024-03-28T18:10:09.174 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/networking/udplite.txt 2024-03-28T18:10:09.186 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/networking/vortex.txt 2024-03-28T18:10:09.189 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:09.189+0000 7f26d2a00640 1 -- 172.21.15.22:0/1593733451 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f26cc101ef0 msgr2=0x7f26cc10e3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:10:09.189 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:09.189+0000 7f26d2a00640 1 --2- 172.21.15.22:0/1593733451 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f26cc101ef0 0x7f26cc10e3b0 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7f26c4009ca0 tx=0x7f26c402c100 comp rx=0 tx=0).stop 2024-03-28T18:10:09.189 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:09.189+0000 7f26d2a00640 1 -- 172.21.15.22:0/1593733451 shutdown_connections 2024-03-28T18:10:09.189 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:09.189+0000 7f26d2a00640 1 --2- 172.21.15.22:0/1593733451 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f26cc101ef0 0x7f26cc10e3b0 unknown :-1 s=CLOSED pgs=626 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:10:09.189 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:09.189+0000 7f26d2a00640 1 --2- 172.21.15.22:0/1593733451 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f26cc101530 0x7f26cc1019b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:10:09.189 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:09.189+0000 7f26d2a00640 1 --2- 172.21.15.22:0/1593733451 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f26cc1075f0 0x7f26cc1079d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:10:09.189 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:09.189+0000 7f26d2a00640 1 -- 172.21.15.22:0/1593733451 >> 172.21.15.22:0/1593733451 conn(0x7f26cc0fd040 msgr2=0x7f26cc0ff460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:10:09.189 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:09.189+0000 7f26d2a00640 1 -- 172.21.15.22:0/1593733451 shutdown_connections 2024-03-28T18:10:09.189 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:09.189+0000 7f26d2a00640 1 -- 172.21.15.22:0/1593733451 wait complete. 2024-03-28T18:10:09.191 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:09.190+0000 7f26d2a00640 1 Processor -- start 2024-03-28T18:10:09.191 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:09.190+0000 7f26d2a00640 1 -- start start 2024-03-28T18:10:09.191 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:09.191+0000 7f26d2a00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f26cc101530 0x7f26cc199ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:10:09.191 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:09.191+0000 7f26d2a00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f26cc101ef0 0x7f26cc19a400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:10:09.191 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:09.191+0000 7f26d2a00640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f26cc1075f0 0x7f26cc19e780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:10:09.191 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:09.191+0000 7f26d2a00640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26cc1118c0 con 0x7f26cc101ef0 2024-03-28T18:10:09.192 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:09.191+0000 7f26d2a00640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26cc111740 con 0x7f26cc1075f0 2024-03-28T18:10:09.192 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:09.191+0000 7f26d2a00640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26cc111a40 con 0x7f26cc101530 2024-03-28T18:10:09.192 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:09.191+0000 7f26d0c00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f26cc101ef0 0x7f26cc19a400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:10:09.192 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:09.191+0000 7f26d0c00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f26cc101ef0 0x7f26cc19a400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:45792/0 (socket says 172.21.15.22:45792) 2024-03-28T18:10:09.192 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:09.191+0000 7f26d0c00640 1 -- 172.21.15.22:0/3854944229 learned_addr learned my addr 172.21.15.22:0/3854944229 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-28T18:10:09.192 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:09.191+0000 7f26d2000640 1 --2- 172.21.15.22:0/3854944229 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f26cc1075f0 0x7f26cc19e780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:10:09.192 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:09.191+0000 7f26d1600640 1 --2- 172.21.15.22:0/3854944229 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f26cc101530 0x7f26cc199ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:10:09.192 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:09.192+0000 7f26d0c00640 1 -- 172.21.15.22:0/3854944229 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f26cc101530 msgr2=0x7f26cc199ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:10:09.192 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:09.192+0000 7f26d0c00640 1 --2- 172.21.15.22:0/3854944229 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f26cc101530 0x7f26cc199ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:10:09.192 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:09.192+0000 7f26d0c00640 1 -- 172.21.15.22:0/3854944229 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f26cc1075f0 msgr2=0x7f26cc19e780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:10:09.192 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:09.192+0000 7f26d0c00640 1 --2- 172.21.15.22:0/3854944229 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f26cc1075f0 0x7f26cc19e780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:10:09.192 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:09.192+0000 7f26d0c00640 1 -- 172.21.15.22:0/3854944229 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f26cc19eeb0 con 0x7f26cc101ef0 2024-03-28T18:10:09.193 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:09.192+0000 7f26d0c00640 1 --2- 172.21.15.22:0/3854944229 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f26cc101ef0 0x7f26cc19a400 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7f26b800d770 tx=0x7f26b800dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:10:09.193 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:09.192+0000 7f26ca000640 1 -- 172.21.15.22:0/3854944229 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26b8017070 con 0x7f26cc101ef0 2024-03-28T18:10:09.193 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:09.192+0000 7f26ca000640 1 -- 172.21.15.22:0/3854944229 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f26b80054f0 con 0x7f26cc101ef0 2024-03-28T18:10:09.194 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:09.192+0000 7f26d2a00640 1 -- 172.21.15.22:0/3854944229 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f26cc19f1a0 con 0x7f26cc101ef0 2024-03-28T18:10:09.194 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:09.192+0000 7f26ca000640 1 -- 172.21.15.22:0/3854944229 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26b8004d60 con 0x7f26cc101ef0 2024-03-28T18:10:09.194 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:09.193+0000 7f26d2a00640 1 -- 172.21.15.22:0/3854944229 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f26cc1a6a30 con 0x7f26cc101ef0 2024-03-28T18:10:09.196 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:09.196+0000 7f26ca000640 1 -- 172.21.15.22:0/3854944229 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 26) v1 ==== 137421+0+0 (secure 0 0 0) 0x7f26b8004f00 con 0x7f26cc101ef0 2024-03-28T18:10:09.196 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:09.196+0000 7f26d6db7740 1 Processor -- start 2024-03-28T18:10:09.196 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:09.197+0000 7f26d6db7740 1 -- start start 2024-03-28T18:10:09.197 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/networking/vxge.txt 2024-03-28T18:10:09.198 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:09.197+0000 7f26d6db7740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559b337f3dd0 0x559b337f2b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:10:09.198 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:09.197+0000 7f26d6db7740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x559b337f30f0 0x559b337f3570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:10:09.199 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:09.197+0000 7f26d6db7740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559b337fc460 0x559b337fe850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:10:09.199 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:09.197+0000 7f26d6db7740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x559b33757250 con 0x559b337f3dd0 2024-03-28T18:10:09.199 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:09.197+0000 7f26d6db7740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x559b336ffa30 con 0x559b337f30f0 2024-03-28T18:10:09.199 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:09.197+0000 7f26d6db7740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x559b337532f0 con 0x559b337fc460 2024-03-28T18:10:09.199 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:09.197+0000 7f26d2000640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559b337f3dd0 0x559b337f2b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:10:09.199 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:09.197+0000 7f26d1600640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x559b337f30f0 0x559b337f3570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:10:09.199 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:09.197+0000 7f26d2000640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559b337f3dd0 0x559b337f2b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:45800/0 (socket says 172.21.15.22:45800) 2024-03-28T18:10:09.199 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:09.197+0000 7f26d2000640 1 -- 172.21.15.22:0/1164623401 learned_addr learned my addr 172.21.15.22:0/1164623401 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-28T18:10:09.199 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:09.198+0000 7f26d0c00640 1 --2- 172.21.15.22:0/1164623401 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559b337fc460 0x559b337fe850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:10:09.199 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:09.198+0000 7f26d1600640 1 -- 172.21.15.22:0/1164623401 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559b337fc460 msgr2=0x559b337fe850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T18:10:09.199 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:09.198+0000 7f26d1600640 1 --2- 172.21.15.22:0/1164623401 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559b337fc460 0x559b337fe850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:10:09.199 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:09.198+0000 7f26d1600640 1 -- 172.21.15.22:0/1164623401 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559b337f3dd0 msgr2=0x559b337f2b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T18:10:09.199 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:09.198+0000 7f26d1600640 1 --2- 172.21.15.22:0/1164623401 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559b337f3dd0 0x559b337f2b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:10:09.199 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:09.198+0000 7f26d1600640 1 -- 172.21.15.22:0/1164623401 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559b336ea690 con 0x559b337f30f0 2024-03-28T18:10:09.200 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:09.198+0000 7f26d2000640 1 --2- 172.21.15.22:0/1164623401 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559b337f3dd0 0x559b337f2b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-28T18:10:09.200 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:09.198+0000 7f26ca000640 1 --2- 172.21.15.22:0/3854944229 >> [v2:172.21.15.22:6800/1357365082,v1:172.21.15.22:6801/1357365082] conn(0x7f26b40a2970 0x7f26b40a4e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:10:09.200 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:09.198+0000 7f26d1600640 1 --2- 172.21.15.22:0/1164623401 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x559b337f30f0 0x559b337f3570 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f26c0009b70 tx=0x7f26c00117b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=52fda56f48fb3c75 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:10:09.200 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:09.198+0000 7f26ca000640 1 -- 172.21.15.22:0/3854944229 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 8411+0+0 (secure 0 0 0) 0x7f26b80ce8d0 con 0x7f26cc101ef0 2024-03-28T18:10:09.200 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:09.199+0000 7f26d2000640 1 --2- 172.21.15.22:0/3854944229 >> [v2:172.21.15.22:6800/1357365082,v1:172.21.15.22:6801/1357365082] conn(0x7f26b40a2970 0x7f26b40a4e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:10:09.200 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:09.199+0000 7f26be600640 1 -- 172.21.15.22:0/1164623401 <== mon.2 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26c0009e60 con 0x559b337f30f0 2024-03-28T18:10:09.200 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:09.199+0000 7f26be600640 1 -- 172.21.15.22:0/1164623401 <== mon.2 v2:172.21.15.79:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f26c0012920 con 0x559b337f30f0 2024-03-28T18:10:09.200 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:09.199+0000 7f26be600640 1 -- 172.21.15.22:0/1164623401 <== mon.2 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26c000fb00 con 0x559b337f30f0 2024-03-28T18:10:09.203 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:09.200+0000 7f26d6db7740 1 -- 172.21.15.22:0/1164623401 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x559b337f30f0 msgr2=0x559b337f3570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:10:09.203 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:09.200+0000 7f26d2000640 1 --2- 172.21.15.22:0/3854944229 >> [v2:172.21.15.22:6800/1357365082,v1:172.21.15.22:6801/1357365082] conn(0x7f26b40a2970 0x7f26b40a4e30 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f26c4004af0 tx=0x7f26c4005d80 comp rx=0 tx=0).ready entity=mgr.15522 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:10:09.203 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:09.200+0000 7f26d6db7740 1 --2- 172.21.15.22:0/1164623401 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x559b337f30f0 0x559b337f3570 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f26c0009b70 tx=0x7f26c00117b0 comp rx=0 tx=0).stop 2024-03-28T18:10:09.203 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:09.200+0000 7f26d6db7740 1 -- 172.21.15.22:0/1164623401 shutdown_connections 2024-03-28T18:10:09.203 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:09.200+0000 7f26d6db7740 1 --2- 172.21.15.22:0/1164623401 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559b337fc460 0x559b337fe850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:10:09.203 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:09.200+0000 7f26d6db7740 1 --2- 172.21.15.22:0/1164623401 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x559b337f30f0 0x559b337f3570 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:10:09.203 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:09.200+0000 7f26d6db7740 1 --2- 172.21.15.22:0/1164623401 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559b337f3dd0 0x559b337f2b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:10:09.203 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:09.200+0000 7f26d6db7740 1 -- 172.21.15.22:0/1164623401 >> 172.21.15.22:0/1164623401 conn(0x559b337ba200 msgr2=0x559b337edb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:10:09.203 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:09.200+0000 7f26d6db7740 1 -- 172.21.15.22:0/1164623401 shutdown_connections 2024-03-28T18:10:09.203 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:09.200+0000 7f26d6db7740 1 -- 172.21.15.22:0/1164623401 wait complete. 2024-03-28T18:10:09.203 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:09.201+0000 7f26d6db7740 1 -- start start 2024-03-28T18:10:09.203 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:09.201+0000 7f26d6db7740 1 Processor -- start 2024-03-28T18:10:09.203 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:09.201+0000 7f26d6db7740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x559b337f30f0 0x559b337fbbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:10:09.204 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:09.201+0000 7f26d6db7740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559b337f3dd0 0x559b337f8f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:10:09.204 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:09.201+0000 7f26d6db7740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559b337fc460 0x559b337f94a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:10:09.204 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:09.201+0000 7f26d6db7740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x559b337ff2d0 con 0x559b337f3dd0 2024-03-28T18:10:09.204 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:09.201+0000 7f26d6db7740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x559b33712e80 con 0x559b337f30f0 2024-03-28T18:10:09.204 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:09.201+0000 7f26d6db7740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x559b337ff450 con 0x559b337fc460 2024-03-28T18:10:09.204 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:09.201+0000 7f26d6db7740 10 client.0 _ll_register_callbacks cb 0x559b33725130 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-28T18:10:09.204 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:09.202+0000 7f26d1600640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x559b337f30f0 0x559b337fbbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:10:09.204 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:09.202+0000 7f26d1600640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x559b337f30f0 0x559b337fbbd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.22:52524/0 (socket says 172.21.15.22:52524) 2024-03-28T18:10:09.204 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:09.202+0000 7f26d1600640 1 -- 172.21.15.22:0/1797061227 learned_addr learned my addr 172.21.15.22:0/1797061227 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-28T18:10:09.204 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:09.202+0000 7f26d0c00640 1 --2- 172.21.15.22:0/1797061227 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559b337f3dd0 0x559b337f8f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:10:09.204 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:09.202+0000 7f26d2000640 1 --2- 172.21.15.22:0/1797061227 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559b337fc460 0x559b337f94a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:10:09.204 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:09.202+0000 7f26d1600640 1 -- 172.21.15.22:0/1797061227 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559b337fc460 msgr2=0x559b337f94a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T18:10:09.204 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:09.202+0000 7f26d1600640 1 --2- 172.21.15.22:0/1797061227 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559b337fc460 0x559b337f94a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:10:09.204 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:09.202+0000 7f26d1600640 1 -- 172.21.15.22:0/1797061227 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559b337f3dd0 msgr2=0x559b337f8f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T18:10:09.205 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:09.202+0000 7f26d1600640 1 --2- 172.21.15.22:0/1797061227 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559b337f3dd0 0x559b337f8f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:10:09.205 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:09.202+0000 7f26d1600640 1 -- 172.21.15.22:0/1797061227 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x559b335357d0 con 0x559b337f30f0 2024-03-28T18:10:09.205 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:09.202+0000 7f26d0c00640 1 --2- 172.21.15.22:0/1797061227 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559b337f3dd0 0x559b337f8f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-28T18:10:09.205 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:09.203+0000 7f26d1600640 1 --2- 172.21.15.22:0/1797061227 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x559b337f30f0 0x559b337fbbd0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f26c0000c00 tx=0x7f26c0012660 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f1e0d592f84bbfbc server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:10:09.205 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:09.203+0000 7f26b2a00640 10 client.0 ms_handle_connect on v2:172.21.15.79:3300/0 2024-03-28T18:10:09.205 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:09.203+0000 7f26b2a00640 1 -- 172.21.15.22:0/1797061227 <== mon.2 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26c0004030 con 0x559b337f30f0 2024-03-28T18:10:09.205 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:09.203+0000 7f26b2a00640 1 -- 172.21.15.22:0/1797061227 <== mon.2 v2:172.21.15.79:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 8411+0+0 (secure 0 0 0) 0x7f26c00041d0 con 0x559b337f30f0 2024-03-28T18:10:09.205 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:09.203+0000 7f26d6db7740 1 -- 172.21.15.22:0/1797061227 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f26c0012920 con 0x559b337f30f0 2024-03-28T18:10:09.206 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:09.203+0000 7f26b2a00640 1 -- 172.21.15.22:0/1797061227 <== mon.2 v2:172.21.15.79:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) v2 ==== 24+0+0 (secure 0 0 0) 0x7f26c0033050 con 0x559b337f30f0 2024-03-28T18:10:09.206 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:09.203+0000 7f26d6db7740 10 client.36050 fetch_fsmap learned FSMap version 31 2024-03-28T18:10:09.207 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:09.203+0000 7f26d6db7740 1 -- 172.21.15.22:0/1797061227 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) v3 -- 0x559b3346b3f0 con 0x559b337f30f0 2024-03-28T18:10:09.207 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:09.204+0000 7f26b2a00640 1 -- 172.21.15.22:0/1797061227 <== mon.2 v2:172.21.15.79:3300/0 4 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f26c0012920 con 0x559b337f30f0 2024-03-28T18:10:09.207 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:09.204+0000 7f26b2a00640 1 -- 172.21.15.22:0/1797061227 <== mon.2 v2:172.21.15.79:3300/0 5 ==== fsmap(e 31) v1 ==== 7950+0+0 (secure 0 0 0) 0x7f26c000f870 con 0x559b337f30f0 2024-03-28T18:10:09.207 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:09.204+0000 7f26d6db7740 10 client.36050 fetch_fsmap finished waiting for FSMap version 31 2024-03-28T18:10:09.207 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:09.204+0000 7f26d6db7740 10 client.36050 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-28T18:10:09.207 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:09.204+0000 7f26d6db7740 20 client.36050 populate_metadata read hostname 'smithi022' 2024-03-28T18:10:09.207 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:09.204+0000 7f26d6db7740 1 --2- 172.21.15.22:0/1797061227 >> [v2:172.21.15.79:6838/2441542443,v1:172.21.15.79:6839/2441542443] conn(0x559b3380c3c0 0x559b3380e7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:10:09.207 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:09.204+0000 7f26d6db7740 4 client.36050 mds_command: new command op to 34553 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-28T18:10:09.207 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:09.204+0000 7f26d6db7740 1 -- 172.21.15.22:0/1797061227 --> [v2:172.21.15.79:6838/2441542443,v1:172.21.15.79:6839/2441542443] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x559b33764790 con 0x559b3380c3c0 2024-03-28T18:10:09.207 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:09.205+0000 7f26d0c00640 1 --2- 172.21.15.22:0/1797061227 >> [v2:172.21.15.79:6838/2441542443,v1:172.21.15.79:6839/2441542443] conn(0x559b3380c3c0 0x559b3380e7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:10:09.207 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:09.205+0000 7f26d0c00640 1 --2- 172.21.15.22:0/1797061227 >> [v2:172.21.15.79:6838/2441542443,v1:172.21.15.79:6839/2441542443] conn(0x559b3380c3c0 0x559b3380e7b0 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dce706e193511669 server_cookie=b216730ff761367d in_seq=0 out_seq=0 2024-03-28T18:10:09.207 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:09.205+0000 7f26b2a00640 10 client.36050 ms_handle_connect on v2:172.21.15.79:6838/2441542443 2024-03-28T18:10:09.208 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/networking/vxlan.txt 2024-03-28T18:10:09.208 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:09.206+0000 7f26b2a00640 1 -- 172.21.15.22:0/1797061227 <== mds.0 v2:172.21.15.79:6838/2441542443 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x559b33764790 con 0x559b3380c3c0 2024-03-28T18:10:09.209 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:09.206+0000 7f26b2a00640 10 client.36050 handle_command_reply: tid=0 2024-03-28T18:10:09.209 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:09.206+0000 7f26d6db7740 1 client.36050 shutdown 2024-03-28T18:10:09.209 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:09.206+0000 7f26d6db7740 1 -- 172.21.15.22:0/1797061227 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x559b337f30f0 msgr2=0x559b337fbbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:10:09.209 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:09.206+0000 7f26d6db7740 1 --2- 172.21.15.22:0/1797061227 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x559b337f30f0 0x559b337fbbd0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f26c0000c00 tx=0x7f26c0012660 comp rx=0 tx=0).stop 2024-03-28T18:10:09.209 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:09.206+0000 7f26d6db7740 1 -- 172.21.15.22:0/1797061227 shutdown_connections 2024-03-28T18:10:09.209 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:09.206+0000 7f26d6db7740 1 --2- 172.21.15.22:0/1797061227 >> [v2:172.21.15.79:6838/2441542443,v1:172.21.15.79:6839/2441542443] conn(0x559b3380c3c0 0x559b3380e7b0 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:10:09.209 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:09.206+0000 7f26d6db7740 1 --2- 172.21.15.22:0/1797061227 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559b337fc460 0x559b337f94a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:10:09.209 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:09.206+0000 7f26d6db7740 1 --2- 172.21.15.22:0/1797061227 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559b337f3dd0 0x559b337f8f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:10:09.209 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:09.206+0000 7f26b2a00640 0 client.36050 ms_handle_reset on v2:172.21.15.79:6838/2441542443 2024-03-28T18:10:09.209 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:09.206+0000 7f26d6db7740 1 --2- 172.21.15.22:0/1797061227 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x559b337f30f0 0x559b337fbbd0 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:10:09.209 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:09.206+0000 7f26d6db7740 1 -- 172.21.15.22:0/1797061227 >> 172.21.15.22:0/1797061227 conn(0x559b337ba200 msgr2=0x559b337ef890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:10:09.209 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:09.207+0000 7f26d6db7740 1 -- 172.21.15.22:0/1797061227 shutdown_connections 2024-03-28T18:10:09.209 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:09.207+0000 7f26d6db7740 1 -- 172.21.15.22:0/1797061227 wait complete. 2024-03-28T18:10:09.209 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:09.207+0000 7f26d6db7740 20 client.36050 trim_cache size 0 max 16384 2024-03-28T18:10:09.215 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/networking/x25-iface.txt 2024-03-28T18:10:09.224 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/networking/x25.txt 2024-03-28T18:10:09.227 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:09.228+0000 7f26d6db7740 1 Processor -- start 2024-03-28T18:10:09.227 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:09.228+0000 7f26d6db7740 1 -- start start 2024-03-28T18:10:09.229 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:09.228+0000 7f26d6db7740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559b337f30f0 0x559b337fb690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:10:09.229 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:09.228+0000 7f26d6db7740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559b337f3dd0 0x559b337fbbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:10:09.229 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:09.228+0000 7f26d6db7740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x559b337fc460 0x559b33807bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:10:09.229 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:09.228+0000 7f26d6db7740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26cc1aef40 con 0x559b337f3dd0 2024-03-28T18:10:09.229 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:09.228+0000 7f26d6db7740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26cc1b0980 con 0x559b337fc460 2024-03-28T18:10:09.229 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:09.228+0000 7f26d6db7740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26cc1af5a0 con 0x559b337f30f0 2024-03-28T18:10:09.229 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:09.228+0000 7f26d0c00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559b337f3dd0 0x559b337fbbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:10:09.229 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:09.228+0000 7f26d0c00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559b337f3dd0 0x559b337fbbd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:45822/0 (socket says 172.21.15.22:45822) 2024-03-28T18:10:09.229 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:09.228+0000 7f26d0c00640 1 -- 172.21.15.22:0/1831937212 learned_addr learned my addr 172.21.15.22:0/1831937212 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-28T18:10:09.229 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:09.229+0000 7f26d2000640 1 --2- 172.21.15.22:0/1831937212 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x559b337fc460 0x559b33807bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:10:09.229 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:09.229+0000 7f26d1600640 1 --2- 172.21.15.22:0/1831937212 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559b337f30f0 0x559b337fb690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:10:09.229 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:09.229+0000 7f26d0c00640 1 -- 172.21.15.22:0/1831937212 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559b337f30f0 msgr2=0x559b337fb690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T18:10:09.229 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:09.229+0000 7f26d0c00640 1 --2- 172.21.15.22:0/1831937212 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559b337f30f0 0x559b337fb690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:10:09.229 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:09.229+0000 7f26d0c00640 1 -- 172.21.15.22:0/1831937212 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x559b337fc460 msgr2=0x559b33807bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T18:10:09.229 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:09.229+0000 7f26d0c00640 1 --2- 172.21.15.22:0/1831937212 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x559b337fc460 0x559b33807bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:10:09.230 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:09.229+0000 7f26d0c00640 1 -- 172.21.15.22:0/1831937212 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559b3342b0f0 con 0x559b337f3dd0 2024-03-28T18:10:09.230 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:09.229+0000 7f26d0c00640 1 --2- 172.21.15.22:0/1831937212 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559b337f3dd0 0x559b337fbbd0 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7f26b8097910 tx=0x7f26b80d0d20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a91077624bdc62ef server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:10:09.231 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:09.229+0000 7f26bdc00640 1 -- 172.21.15.22:0/1831937212 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26b8020040 con 0x559b337f3dd0 2024-03-28T18:10:09.231 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:09.230+0000 7f26bdc00640 1 -- 172.21.15.22:0/1831937212 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f26b80046c0 con 0x559b337f3dd0 2024-03-28T18:10:09.231 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:09.230+0000 7f26bdc00640 1 -- 172.21.15.22:0/1831937212 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26b8099050 con 0x559b337f3dd0 2024-03-28T18:10:09.231 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:09.230+0000 7f26d6db7740 1 -- 172.21.15.22:0/1831937212 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559b337f3dd0 msgr2=0x559b337fbbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:10:09.231 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:09.230+0000 7f26d6db7740 1 --2- 172.21.15.22:0/1831937212 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559b337f3dd0 0x559b337fbbd0 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7f26b8097910 tx=0x7f26b80d0d20 comp rx=0 tx=0).stop 2024-03-28T18:10:09.231 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:09.230+0000 7f26d6db7740 1 -- 172.21.15.22:0/1831937212 shutdown_connections 2024-03-28T18:10:09.231 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:09.230+0000 7f26d6db7740 1 --2- 172.21.15.22:0/1831937212 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x559b337fc460 0x559b33807bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:10:09.231 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:09.230+0000 7f26d6db7740 1 --2- 172.21.15.22:0/1831937212 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559b337f3dd0 0x559b337fbbd0 unknown :-1 s=CLOSED pgs=628 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:10:09.231 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:09.230+0000 7f26d6db7740 1 --2- 172.21.15.22:0/1831937212 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559b337f30f0 0x559b337fb690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:10:09.231 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:09.230+0000 7f26d6db7740 1 -- 172.21.15.22:0/1831937212 >> 172.21.15.22:0/1831937212 conn(0x559b337ba200 msgr2=0x559b337f7830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:10:09.231 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:09.231+0000 7f26d6db7740 1 -- 172.21.15.22:0/1831937212 shutdown_connections 2024-03-28T18:10:09.231 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:09.231+0000 7f26d6db7740 1 -- 172.21.15.22:0/1831937212 wait complete. 2024-03-28T18:10:09.231 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:09.231+0000 7f26d6db7740 1 -- start start 2024-03-28T18:10:09.232 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:09.231+0000 7f26d6db7740 1 Processor -- start 2024-03-28T18:10:09.234 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:09.232+0000 7f26d6db7740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559b337f30f0 0x559b337f61e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:10:09.234 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:09.232+0000 7f26d6db7740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x559b337f3dd0 0x559b337f6720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:10:09.234 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:09.232+0000 7f26d6db7740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559b337fc460 0x559b337f6c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:10:09.234 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:09.232+0000 7f26d6db7740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26cc1b0260 con 0x559b337f30f0 2024-03-28T18:10:09.234 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:09.232+0000 7f26d6db7740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26cc1afc00 con 0x559b337f3dd0 2024-03-28T18:10:09.234 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:09.232+0000 7f26d6db7740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26cc1ad100 con 0x559b337fc460 2024-03-28T18:10:09.234 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:09.232+0000 7f26d6db7740 10 client.0 _ll_register_callbacks cb 0x7f26c0005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-28T18:10:09.235 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:09.232+0000 7f26d1600640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559b337f30f0 0x559b337f61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:10:09.235 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:09.232+0000 7f26d1600640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559b337f30f0 0x559b337f61e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:45836/0 (socket says 172.21.15.22:45836) 2024-03-28T18:10:09.235 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:09.232+0000 7f26d0c00640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x559b337f3dd0 0x559b337f6720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:10:09.235 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:09.232+0000 7f26d1600640 1 -- 172.21.15.22:0/4252556775 learned_addr learned my addr 172.21.15.22:0/4252556775 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-28T18:10:09.235 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:09.232+0000 7f26d2000640 1 --2- 172.21.15.22:0/4252556775 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559b337fc460 0x559b337f6c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:10:09.235 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:09.232+0000 7f26d1600640 1 -- 172.21.15.22:0/4252556775 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559b337fc460 msgr2=0x559b337f6c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T18:10:09.235 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:09.232+0000 7f26d1600640 1 --2- 172.21.15.22:0/4252556775 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559b337fc460 0x559b337f6c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:10:09.235 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:09.232+0000 7f26d1600640 1 -- 172.21.15.22:0/4252556775 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x559b337f3dd0 msgr2=0x559b337f6720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T18:10:09.235 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:09.232+0000 7f26d1600640 1 --2- 172.21.15.22:0/4252556775 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x559b337f3dd0 0x559b337f6720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:10:09.235 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:09.232+0000 7f26d1600640 1 -- 172.21.15.22:0/4252556775 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x559b337fced0 con 0x559b337f30f0 2024-03-28T18:10:09.235 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:09.232+0000 7f26d0c00640 1 --2- 172.21.15.22:0/4252556775 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x559b337f3dd0 0x559b337f6720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-28T18:10:09.235 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:09.233+0000 7f26d1600640 1 --2- 172.21.15.22:0/4252556775 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559b337f30f0 0x559b337f61e0 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7f26c0001eb0 tx=0x7f26c000fec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=741f000e6dff2f0c server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:10:09.235 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:09.233+0000 7f26b2a00640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-03-28T18:10:09.235 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:09.233+0000 7f26b2a00640 1 -- 172.21.15.22:0/4252556775 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26c003c060 con 0x559b337f30f0 2024-03-28T18:10:09.236 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:09.233+0000 7f26b2a00640 1 -- 172.21.15.22:0/4252556775 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 8411+0+0 (secure 0 0 0) 0x7f26c000f920 con 0x559b337f30f0 2024-03-28T18:10:09.236 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:09.233+0000 7f26d6db7740 1 -- 172.21.15.22:0/4252556775 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x559b3375eae0 con 0x559b337f30f0 2024-03-28T18:10:09.236 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:09.233+0000 7f26b2a00640 1 -- 172.21.15.22:0/4252556775 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) v2 ==== 24+0+0 (secure 0 0 0) 0x7f26c0009780 con 0x559b337f30f0 2024-03-28T18:10:09.236 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:09.234+0000 7f26d6db7740 10 client.16227 fetch_fsmap learned FSMap version 31 2024-03-28T18:10:09.236 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:09.234+0000 7f26d6db7740 1 -- 172.21.15.22:0/4252556775 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) v3 -- 0x559b33867ed0 con 0x559b337f30f0 2024-03-28T18:10:09.236 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:09.234+0000 7f26b2a00640 1 -- 172.21.15.22:0/4252556775 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x559b3375eae0 con 0x559b337f30f0 2024-03-28T18:10:09.236 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:09.234+0000 7f26b2a00640 1 -- 172.21.15.22:0/4252556775 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 31) v1 ==== 7950+0+0 (secure 0 0 0) 0x7f26c003f020 con 0x559b337f30f0 2024-03-28T18:10:09.236 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:09.234+0000 7f26d6db7740 10 client.16227 fetch_fsmap finished waiting for FSMap version 31 2024-03-28T18:10:09.236 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:09.234+0000 7f26d6db7740 10 client.16227 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-28T18:10:09.236 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:09.234+0000 7f26d6db7740 20 client.16227 populate_metadata read hostname 'smithi022' 2024-03-28T18:10:09.236 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:09.234+0000 7f26d6db7740 1 --2- 172.21.15.22:0/4252556775 >> [v2:172.21.15.79:6838/2441542443,v1:172.21.15.79:6839/2441542443] conn(0x559b338685e0 0x559b3386aa00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:10:09.236 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:09.234+0000 7f26d6db7740 4 client.16227 mds_command: new command op to 34553 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-28T18:10:09.236 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:09.234+0000 7f26d6db7740 1 -- 172.21.15.22:0/4252556775 --> [v2:172.21.15.79:6838/2441542443,v1:172.21.15.79:6839/2441542443] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x559b336dc3a0 con 0x559b338685e0 2024-03-28T18:10:09.236 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:09.235+0000 7f26d0c00640 1 --2- 172.21.15.22:0/4252556775 >> [v2:172.21.15.79:6838/2441542443,v1:172.21.15.79:6839/2441542443] conn(0x559b338685e0 0x559b3386aa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:10:09.237 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:09.235+0000 7f26d0c00640 1 --2- 172.21.15.22:0/4252556775 >> [v2:172.21.15.79:6838/2441542443,v1:172.21.15.79:6839/2441542443] conn(0x559b338685e0 0x559b3386aa00 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f3f2cb84c257afd0 server_cookie=14355ed826f648d1 in_seq=0 out_seq=0 2024-03-28T18:10:09.237 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:09.235+0000 7f26b2a00640 10 client.16227 ms_handle_connect on v2:172.21.15.79:6838/2441542443 2024-03-28T18:10:09.237 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:09.235+0000 7f26b2a00640 1 -- 172.21.15.22:0/4252556775 <== mds.0 v2:172.21.15.79:6838/2441542443 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x559b336dc3a0 con 0x559b338685e0 2024-03-28T18:10:09.237 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:09.235+0000 7f26b2a00640 10 client.16227 handle_command_reply: tid=0 2024-03-28T18:10:09.237 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:09.235+0000 7f26d6db7740 1 client.16227 shutdown 2024-03-28T18:10:09.238 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/networking/xfrm_proc.txt 2024-03-28T18:10:09.238 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:09.236+0000 7f26d6db7740 1 -- 172.21.15.22:0/4252556775 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559b337f30f0 msgr2=0x559b337f61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:10:09.238 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:09.236+0000 7f26d6db7740 1 --2- 172.21.15.22:0/4252556775 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559b337f30f0 0x559b337f61e0 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7f26c0001eb0 tx=0x7f26c000fec0 comp rx=0 tx=0).stop 2024-03-28T18:10:09.239 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:09.236+0000 7f26d6db7740 1 -- 172.21.15.22:0/4252556775 shutdown_connections 2024-03-28T18:10:09.239 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:09.236+0000 7f26d6db7740 1 --2- 172.21.15.22:0/4252556775 >> [v2:172.21.15.79:6838/2441542443,v1:172.21.15.79:6839/2441542443] conn(0x559b338685e0 0x559b3386aa00 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:10:09.239 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:09.236+0000 7f26d6db7740 1 --2- 172.21.15.22:0/4252556775 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559b337fc460 0x559b337f6c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:10:09.239 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:09.236+0000 7f26d6db7740 1 --2- 172.21.15.22:0/4252556775 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x559b337f3dd0 0x559b337f6720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:10:09.239 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:09.236+0000 7f26d6db7740 1 --2- 172.21.15.22:0/4252556775 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559b337f30f0 0x559b337f61e0 unknown :-1 s=CLOSED pgs=629 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:10:09.239 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:09.236+0000 7f26b2a00640 0 client.16227 ms_handle_reset on v2:172.21.15.79:6838/2441542443 2024-03-28T18:10:09.239 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:09.236+0000 7f26d6db7740 1 -- 172.21.15.22:0/4252556775 >> 172.21.15.22:0/4252556775 conn(0x559b337ba200 msgr2=0x559b337eeec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:10:09.239 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:09.236+0000 7f26d6db7740 1 -- 172.21.15.22:0/4252556775 shutdown_connections 2024-03-28T18:10:09.239 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:09.236+0000 7f26d6db7740 1 -- 172.21.15.22:0/4252556775 wait complete. 2024-03-28T18:10:09.239 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:09.236+0000 7f26d6db7740 20 client.16227 trim_cache size 0 max 16384 2024-03-28T18:10:09.239 INFO:teuthology.orchestra.run.smithi022.stdout:{ 2024-03-28T18:10:09.239 INFO:teuthology.orchestra.run.smithi022.stdout: "status": "no active scrubs running", 2024-03-28T18:10:09.240 INFO:teuthology.orchestra.run.smithi022.stdout: "scrubs": {} 2024-03-28T18:10:09.240 INFO:teuthology.orchestra.run.smithi022.stdout:} 2024-03-28T18:10:09.240 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:09.237+0000 7f26d2a00640 1 -- 172.21.15.22:0/3854944229 >> [v2:172.21.15.22:6800/1357365082,v1:172.21.15.22:6801/1357365082] conn(0x7f26b40a2970 msgr2=0x7f26b40a4e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:10:09.240 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:09.237+0000 7f26d2a00640 1 --2- 172.21.15.22:0/3854944229 >> [v2:172.21.15.22:6800/1357365082,v1:172.21.15.22:6801/1357365082] conn(0x7f26b40a2970 0x7f26b40a4e30 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f26c4004af0 tx=0x7f26c4005d80 comp rx=0 tx=0).stop 2024-03-28T18:10:09.240 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:09.237+0000 7f26d2a00640 1 -- 172.21.15.22:0/3854944229 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f26cc101ef0 msgr2=0x7f26cc19a400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:10:09.240 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:09.237+0000 7f26d2a00640 1 --2- 172.21.15.22:0/3854944229 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f26cc101ef0 0x7f26cc19a400 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7f26b800d770 tx=0x7f26b800dc30 comp rx=0 tx=0).stop 2024-03-28T18:10:09.241 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:09.237+0000 7f26d2a00640 1 -- 172.21.15.22:0/3854944229 shutdown_connections 2024-03-28T18:10:09.241 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:09.237+0000 7f26d2a00640 1 --2- 172.21.15.22:0/3854944229 >> [v2:172.21.15.22:6800/1357365082,v1:172.21.15.22:6801/1357365082] conn(0x7f26b40a2970 0x7f26b40a4e30 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:10:09.241 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:09.237+0000 7f26d2a00640 1 --2- 172.21.15.22:0/3854944229 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f26cc1075f0 0x7f26cc19e780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:10:09.241 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:09.237+0000 7f26d2a00640 1 --2- 172.21.15.22:0/3854944229 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f26cc101ef0 0x7f26cc19a400 unknown :-1 s=CLOSED pgs=627 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:10:09.241 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:09.237+0000 7f26d2a00640 1 --2- 172.21.15.22:0/3854944229 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f26cc101530 0x7f26cc199ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:10:09.241 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:09.237+0000 7f26d2a00640 1 -- 172.21.15.22:0/3854944229 >> 172.21.15.22:0/3854944229 conn(0x7f26cc0fd040 msgr2=0x7f26cc0fe490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:10:09.241 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:09.237+0000 7f26d2a00640 1 -- 172.21.15.22:0/3854944229 shutdown_connections 2024-03-28T18:10:09.241 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:09.237+0000 7f26d2a00640 1 -- 172.21.15.22:0/3854944229 wait complete. 2024-03-28T18:10:09.245 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/networking/xfrm_sync.txt 2024-03-28T18:10:09.255 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/networking/xfrm_sysctl.txt 2024-03-28T18:10:09.255 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-03-28T18:10:09.256 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-28T18:10:09.268 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/networking/z8530drv.txt 2024-03-28T18:10:09.283 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/nfc/ 2024-03-28T18:10:09.300 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/nfc/nfc-hci.txt 2024-03-28T18:10:09.305 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/nfc/nfc-pn544.txt 2024-03-28T18:10:09.317 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/nios2/ 2024-03-28T18:10:09.329 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/nios2/README 2024-03-28T18:10:09.332 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/nommu-mmap.txt 2024-03-28T18:10:09.352 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/numastat.txt 2024-03-28T18:10:09.364 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/oops-tracing.txt 2024-03-28T18:10:09.376 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/padata.txt 2024-03-28T18:10:09.388 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/parisc/ 2024-03-28T18:10:09.397 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/parisc/00-INDEX 2024-03-28T18:10:09.405 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/parisc/debugging 2024-03-28T18:10:09.419 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/parisc/registers 2024-03-28T18:10:09.437 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/parport-lowlevel.txt 2024-03-28T18:10:09.459 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/parport.txt 2024-03-28T18:10:09.477 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/pcmcia/ 2024-03-28T18:10:09.488 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/pcmcia/.gitignore 2024-03-28T18:10:09.494 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/pcmcia/Makefile 2024-03-28T18:10:09.500 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:09.500+0000 7fd20e800640 1 -- 172.21.15.22:0/3396998028 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd208075bd0 msgr2=0x7fd208075fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:10:09.500 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:09.500+0000 7fd20e800640 1 --2- 172.21.15.22:0/3396998028 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd208075bd0 0x7fd208075fb0 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7fd1fc009900 tx=0x7fd1fc02bc30 comp rx=0 tx=0).stop 2024-03-28T18:10:09.500 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:09.501+0000 7fd20e800640 1 -- 172.21.15.22:0/3396998028 shutdown_connections 2024-03-28T18:10:09.500 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:09.501+0000 7fd20e800640 1 --2- 172.21.15.22:0/3396998028 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd208112320 0x7fd208114710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:10:09.500 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:09.501+0000 7fd20e800640 1 --2- 172.21.15.22:0/3396998028 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd2080764f0 0x7fd208111de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:10:09.501 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:09.501+0000 7fd20e800640 1 --2- 172.21.15.22:0/3396998028 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd208075bd0 0x7fd208075fb0 unknown :-1 s=CLOSED pgs=630 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:10:09.501 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:09.501+0000 7fd20e800640 1 -- 172.21.15.22:0/3396998028 >> 172.21.15.22:0/3396998028 conn(0x7fd2080ff130 msgr2=0x7fd208101550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:10:09.501 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:09.501+0000 7fd20e800640 1 -- 172.21.15.22:0/3396998028 shutdown_connections 2024-03-28T18:10:09.501 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:09.501+0000 7fd20e800640 1 -- 172.21.15.22:0/3396998028 wait complete. 2024-03-28T18:10:09.501 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:09.502+0000 7fd20e800640 1 Processor -- start 2024-03-28T18:10:09.501 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:09.502+0000 7fd20e800640 1 -- start start 2024-03-28T18:10:09.503 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:09.502+0000 7fd20e800640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd208075bd0 0x7fd20819e480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:10:09.503 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:09.502+0000 7fd20e800640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd2080764f0 0x7fd20819e9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:10:09.503 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:09.502+0000 7fd20e800640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd208112320 0x7fd2081a2cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:10:09.503 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:09.502+0000 7fd20e800640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd208117ec0 con 0x7fd208112320 2024-03-28T18:10:09.503 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:09.502+0000 7fd20e800640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd208117d40 con 0x7fd208075bd0 2024-03-28T18:10:09.503 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:09.502+0000 7fd20e800640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd208118040 con 0x7fd2080764f0 2024-03-28T18:10:09.503 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:09.502+0000 7fd20de00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd208112320 0x7fd2081a2cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:10:09.503 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:09.503+0000 7fd20de00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd208112320 0x7fd2081a2cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:45850/0 (socket says 172.21.15.22:45850) 2024-03-28T18:10:09.503 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:09.503+0000 7fd20de00640 1 -- 172.21.15.22:0/1169249648 learned_addr learned my addr 172.21.15.22:0/1169249648 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-28T18:10:09.503 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:09.503+0000 7fd20d400640 1 --2- 172.21.15.22:0/1169249648 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd208075bd0 0x7fd20819e480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:10:09.503 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:09.503+0000 7fd20ca00640 1 --2- 172.21.15.22:0/1169249648 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd2080764f0 0x7fd20819e9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:10:09.504 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:09.503+0000 7fd20de00640 1 -- 172.21.15.22:0/1169249648 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd2080764f0 msgr2=0x7fd20819e9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:10:09.504 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:09.503+0000 7fd20de00640 1 --2- 172.21.15.22:0/1169249648 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd2080764f0 0x7fd20819e9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:10:09.504 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:09.503+0000 7fd20de00640 1 -- 172.21.15.22:0/1169249648 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd208075bd0 msgr2=0x7fd20819e480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:10:09.504 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:09.503+0000 7fd20de00640 1 --2- 172.21.15.22:0/1169249648 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd208075bd0 0x7fd20819e480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:10:09.504 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:09.503+0000 7fd20de00640 1 -- 172.21.15.22:0/1169249648 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd2081a3420 con 0x7fd208112320 2024-03-28T18:10:09.504 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:09.503+0000 7fd20ca00640 1 --2- 172.21.15.22:0/1169249648 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd2080764f0 0x7fd20819e9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T18:10:09.504 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:09.503+0000 7fd20d400640 1 --2- 172.21.15.22:0/1169249648 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd208075bd0 0x7fd20819e480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-28T18:10:09.504 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:09.503+0000 7fd20de00640 1 --2- 172.21.15.22:0/1169249648 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd208112320 0x7fd2081a2cf0 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7fd20000eb40 tx=0x7fd20000c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:10:09.505 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:09.504+0000 7fd206000640 1 -- 172.21.15.22:0/1169249648 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd200004b20 con 0x7fd208112320 2024-03-28T18:10:09.505 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:09.504+0000 7fd206000640 1 -- 172.21.15.22:0/1169249648 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7fd200004cc0 con 0x7fd208112320 2024-03-28T18:10:09.505 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:09.504+0000 7fd206000640 1 -- 172.21.15.22:0/1169249648 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd200010850 con 0x7fd208112320 2024-03-28T18:10:09.505 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:09.504+0000 7fd20e800640 1 -- 172.21.15.22:0/1169249648 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd2081a3710 con 0x7fd208112320 2024-03-28T18:10:09.505 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:09.504+0000 7fd20e800640 1 -- 172.21.15.22:0/1169249648 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd2081aafa0 con 0x7fd208112320 2024-03-28T18:10:09.505 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/pcmcia/crc32hash.c 2024-03-28T18:10:09.506 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:09.505+0000 7fd20e800640 1 -- 172.21.15.22:0/1169249648 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd208068800 con 0x7fd208112320 2024-03-28T18:10:09.507 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:09.507+0000 7fd206000640 1 -- 172.21.15.22:0/1169249648 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 26) v1 ==== 137421+0+0 (secure 0 0 0) 0x7fd200005000 con 0x7fd208112320 2024-03-28T18:10:09.509 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:09.509+0000 7fd206000640 1 --2- 172.21.15.22:0/1169249648 >> [v2:172.21.15.22:6800/1357365082,v1:172.21.15.22:6801/1357365082] conn(0x7fd1f00a29f0 0x7fd1f00a4eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:10:09.509 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:09.509+0000 7fd206000640 1 -- 172.21.15.22:0/1169249648 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 8411+0+0 (secure 0 0 0) 0x7fd2000cf500 con 0x7fd208112320 2024-03-28T18:10:09.509 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:09.509+0000 7fd20d400640 1 --2- 172.21.15.22:0/1169249648 >> [v2:172.21.15.22:6800/1357365082,v1:172.21.15.22:6801/1357365082] conn(0x7fd1f00a29f0 0x7fd1f00a4eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:10:09.510 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:09.510+0000 7fd20d400640 1 --2- 172.21.15.22:0/1169249648 >> [v2:172.21.15.22:6800/1357365082,v1:172.21.15.22:6801/1357365082] conn(0x7fd1f00a29f0 0x7fd1f00a4eb0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fd1fc005ab0 tx=0x7fd1fc005a20 comp rx=0 tx=0).ready entity=mgr.15522 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:10:09.513 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/pcmcia/devicetable.txt 2024-03-28T18:10:09.514 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:09.515+0000 7fd206000640 1 -- 172.21.15.22:0/1169249648 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fd20009c2d0 con 0x7fd208112320 2024-03-28T18:10:09.522 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/pcmcia/driver-changes.txt 2024-03-28T18:10:09.534 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/pcmcia/driver.txt 2024-03-28T18:10:09.586 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/pcmcia/locking.txt 2024-03-28T18:10:09.609 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/percpu-rw-semaphore.txt 2024-03-28T18:10:09.626 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/phy.txt 2024-03-28T18:10:09.640 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/phy/ 2024-03-28T18:10:09.649 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/phy/samsung-usb2.txt 2024-03-28T18:10:09.656 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/pi-futex.txt 2024-03-28T18:10:09.676 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/pinctrl.txt 2024-03-28T18:10:09.696 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/platform/ 2024-03-28T18:10:09.707 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/platform/x86-laptop-drivers.txt 2024-03-28T18:10:09.714 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/pnp.txt 2024-03-28T18:10:09.734 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/power/ 2024-03-28T18:10:09.745 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/power/00-INDEX 2024-03-28T18:10:09.752 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/power/apm-acpi.txt 2024-03-28T18:10:09.769 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/power/basic-pm-debugging.txt 2024-03-28T18:10:09.785 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/power/charger-manager.txt 2024-03-28T18:10:09.797 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:09.797+0000 7fd20e800640 1 -- 172.21.15.22:0/1169249648 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd20802d070 con 0x7fd208112320 2024-03-28T18:10:09.798 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:09.799+0000 7fd206000640 1 -- 172.21.15.22:0/1169249648 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 31 v31) v1 ==== 94+0+11459 (secure 0 0 0) 0x7fd20009bbd0 con 0x7fd208112320 2024-03-28T18:10:09.799 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/power/devices.txt 2024-03-28T18:10:09.799 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-03-28T18:10:09.799 INFO:teuthology.orchestra.run.smithi022.stdout:{"epoch":31,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14544,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6835/3718238950","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6834","nonce":3718238950},{"type":"v1","addr":"172.21.15.22:6835","nonce":3718238950}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14562,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6837/3676939386","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6836","nonce":3676939386},{"type":"v1","addr":"172.21.15.22:6837","nonce":3676939386}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":31,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-28T18:04:48.991622+0000","modified":"2024-03-28T18:09:54.295972+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":34553,"mds_1":34511,"mds_2":34475,"mds_3":34442,"mds_4":24502},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14640":{"gid":14640,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.22:6839/3979998310","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6838","nonce":3979998310},{"type":"v1","addr":"172.21.15.22:6839","nonce":3979998310}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_14655":{"gid":14655,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.22:6841/3069056661","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6840","nonce":3069056661},{"type":"v1","addr":"172.21.15.22:6841","nonce":3069056661}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24412":{"gid":24412,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.134:6833/3176125362","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6832","nonce":3176125362},{"type":"v1","addr":"172.21.15.134:6833","nonce":3176125362}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24463":{"gid":24463,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.134:6835/3370549207","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6834","nonce":3370549207},{"type":"v1","addr":"172.21.15.134:6835","nonce":3370549207}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24478":{"gid":24478,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.134:6837/899825581","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6836","nonce":899825581},{"type":"v1","addr":"172.21.15.134:6837","nonce":899825581}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24502":{"gid":24502,"name":"i","rank":4,"incarnation":24,"state":"up:active","state_seq":7,"addr":"172.21.15.134:6839/3400263217","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6838","nonce":3400263217},{"type":"v1","addr":"172.21.15.134:6839","nonce":3400263217}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_34442":{"gid":34442,"name":"k","rank":3,"incarnation":22,"state":"up:active","state_seq":14,"addr":"172.21.15.79:6833/856896997","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6832","nonce":856896997},{"type":"v1","addr":"172.21.15.79:6833","nonce":856896997}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_34475":{"gid":34475,"name":"e","rank":2,"incarnation":20,"state":"up:active","state_seq":11,"addr":"172.21.15.79:6835/3365773431","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6834","nonce":3365773431},{"type":"v1","addr":"172.21.15.79:6835","nonce":3365773431}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_34511":{"gid":34511,"name":"h","rank":1,"incarnation":18,"state":"up:active","state_seq":8,"addr":"172.21.15.79:6837/1787903366","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6836","nonce":1787903366},{"type":"v1","addr":"172.21.15.79:6837","nonce":1787903366}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_34553":{"gid":34553,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.79:6839/2441542443","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6838","nonce":2441542443},{"type":"v1","addr":"172.21.15.79:6839","nonce":2441542443}]},"join_fscid":-1,"export_targets":[1,2,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":34553,"qdb_cluster":[24502,34442,34475,34511,34553]},"id":1}]} 2024-03-28T18:10:09.799 INFO:teuthology.orchestra.run.smithi022.stderr:dumped fsmap epoch 31 2024-03-28T18:10:09.803 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:09.803+0000 7fd20e800640 1 -- 172.21.15.22:0/1169249648 >> [v2:172.21.15.22:6800/1357365082,v1:172.21.15.22:6801/1357365082] conn(0x7fd1f00a29f0 msgr2=0x7fd1f00a4eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:10:09.803 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:09.803+0000 7fd20e800640 1 --2- 172.21.15.22:0/1169249648 >> [v2:172.21.15.22:6800/1357365082,v1:172.21.15.22:6801/1357365082] conn(0x7fd1f00a29f0 0x7fd1f00a4eb0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fd1fc005ab0 tx=0x7fd1fc005a20 comp rx=0 tx=0).stop 2024-03-28T18:10:09.803 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:09.803+0000 7fd20e800640 1 -- 172.21.15.22:0/1169249648 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd208112320 msgr2=0x7fd2081a2cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:10:09.803 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:09.803+0000 7fd20e800640 1 --2- 172.21.15.22:0/1169249648 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd208112320 0x7fd2081a2cf0 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7fd20000eb40 tx=0x7fd20000c2d0 comp rx=0 tx=0).stop 2024-03-28T18:10:09.804 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:09.804+0000 7fd20e800640 1 -- 172.21.15.22:0/1169249648 shutdown_connections 2024-03-28T18:10:09.804 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:09.804+0000 7fd20e800640 1 --2- 172.21.15.22:0/1169249648 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd208112320 0x7fd2081a2cf0 unknown :-1 s=CLOSED pgs=631 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:10:09.804 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:09.804+0000 7fd20e800640 1 --2- 172.21.15.22:0/1169249648 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd2080764f0 0x7fd20819e9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:10:09.804 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:09.804+0000 7fd20e800640 1 --2- 172.21.15.22:0/1169249648 >> [v2:172.21.15.22:6800/1357365082,v1:172.21.15.22:6801/1357365082] conn(0x7fd1f00a29f0 0x7fd1f00a4eb0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:10:09.804 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:09.804+0000 7fd20e800640 1 --2- 172.21.15.22:0/1169249648 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd208075bd0 0x7fd20819e480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:10:09.804 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:09.804+0000 7fd20e800640 1 -- 172.21.15.22:0/1169249648 >> 172.21.15.22:0/1169249648 conn(0x7fd2080ff130 msgr2=0x7fd208113a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:10:09.804 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:09.804+0000 7fd20e800640 1 -- 172.21.15.22:0/1169249648 shutdown_connections 2024-03-28T18:10:09.804 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:09.804+0000 7fd20e800640 1 -- 172.21.15.22:0/1169249648 wait complete. 2024-03-28T18:10:09.811 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/power/drivers-testing.txt 2024-03-28T18:10:09.821 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/power/freezing-of-tasks.txt 2024-03-28T18:10:09.823 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-28T18:10:09.824 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2024-03-28T18:10:09.834 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/power/interface.txt 2024-03-28T18:10:09.844 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/power/notifiers.txt 2024-03-28T18:10:09.857 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/power/opp.txt 2024-03-28T18:10:09.871 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/power/pci.txt 2024-03-28T18:10:09.887 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/power/pm_qos_interface.txt 2024-03-28T18:10:09.900 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/power/power_supply_class.txt 2024-03-28T18:10:09.914 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/power/powercap/ 2024-03-28T18:10:09.925 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/power/powercap/powercap.txt 2024-03-28T18:10:09.932 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/power/regulator/ 2024-03-28T18:10:09.946 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/power/regulator/consumer.txt 2024-03-28T18:10:09.955 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/power/regulator/design.txt 2024-03-28T18:10:09.971 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/power/regulator/machine.txt 2024-03-28T18:10:09.984 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/power/regulator/overview.txt 2024-03-28T18:10:10.001 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/power/regulator/regulator.txt 2024-03-28T18:10:10.015 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/power/runtime_pm.txt 2024-03-28T18:10:10.040 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/power/s2ram.txt 2024-03-28T18:10:10.054 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/power/states.txt 2024-03-28T18:10:10.065 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/power/suspend-and-cpuhotplug.txt 2024-03-28T18:10:10.076 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:10.076+0000 7f4320a00640 1 -- 172.21.15.22:0/2162427097 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f431c1030f0 msgr2=0x7f431c10b5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:10:10.076 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:10.076+0000 7f4320a00640 1 --2- 172.21.15.22:0/2162427097 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f431c1030f0 0x7f431c10b5d0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f4304009860 tx=0x7f430402bc10 comp rx=0 tx=0).stop 2024-03-28T18:10:10.077 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:10.077+0000 7f4320a00640 1 -- 172.21.15.22:0/2162427097 shutdown_connections 2024-03-28T18:10:10.077 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:10.077+0000 7f4320a00640 1 --2- 172.21.15.22:0/2162427097 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f431c10bba0 0x7f431c10e020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:10:10.077 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:10.077+0000 7f4320a00640 1 --2- 172.21.15.22:0/2162427097 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f431c1030f0 0x7f431c10b5d0 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:10:10.077 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:10.077+0000 7f4320a00640 1 --2- 172.21.15.22:0/2162427097 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f431c1027d0 0x7f431c102bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:10:10.077 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:10.077+0000 7f4320a00640 1 -- 172.21.15.22:0/2162427097 >> 172.21.15.22:0/2162427097 conn(0x7f431c0fbfb0 msgr2=0x7f431c0fe3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:10:10.077 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:10.077+0000 7f4320a00640 1 -- 172.21.15.22:0/2162427097 shutdown_connections 2024-03-28T18:10:10.077 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:10.077+0000 7f4320a00640 1 -- 172.21.15.22:0/2162427097 wait complete. 2024-03-28T18:10:10.077 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/power/suspend-and-interrupts.txt 2024-03-28T18:10:10.078 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:10.078+0000 7f4320a00640 1 Processor -- start 2024-03-28T18:10:10.078 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:10.078+0000 7f4320a00640 1 -- start start 2024-03-28T18:10:10.078 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:10.078+0000 7f4320a00640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f431c1027d0 0x7f431c100420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:10:10.078 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:10.079+0000 7f4320a00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f431c1030f0 0x7f431c100960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:10:10.078 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:10.079+0000 7f4320a00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f431c10bba0 0x7f431c100ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:10:10.079 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:10.079+0000 7f4320a00640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f431c1119b0 con 0x7f431c1030f0 2024-03-28T18:10:10.079 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:10.079+0000 7f4320a00640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f431c111830 con 0x7f431c1027d0 2024-03-28T18:10:10.079 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:10.079+0000 7f4320a00640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f431c111b30 con 0x7f431c10bba0 2024-03-28T18:10:10.080 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:10.079+0000 7f431b400640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f431c1027d0 0x7f431c100420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:10:10.080 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:10.079+0000 7f431b400640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f431c1027d0 0x7f431c100420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.22:52586/0 (socket says 172.21.15.22:52586) 2024-03-28T18:10:10.080 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:10.079+0000 7f431b400640 1 -- 172.21.15.22:0/182835476 learned_addr learned my addr 172.21.15.22:0/182835476 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-28T18:10:10.080 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:10.079+0000 7f4312a00640 1 --2- 172.21.15.22:0/182835476 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f431c1030f0 0x7f431c100960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:10:10.080 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:10.079+0000 7f431be00640 1 --2- 172.21.15.22:0/182835476 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f431c10bba0 0x7f431c100ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:10:10.080 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:10.079+0000 7f431b400640 1 -- 172.21.15.22:0/182835476 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f431c10bba0 msgr2=0x7f431c100ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:10:10.080 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:10.079+0000 7f431b400640 1 --2- 172.21.15.22:0/182835476 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f431c10bba0 0x7f431c100ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:10:10.080 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:10.079+0000 7f431b400640 1 -- 172.21.15.22:0/182835476 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f431c1030f0 msgr2=0x7f431c100960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:10:10.080 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:10.079+0000 7f431b400640 1 --2- 172.21.15.22:0/182835476 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f431c1030f0 0x7f431c100960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:10:10.081 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:10.079+0000 7f431b400640 1 -- 172.21.15.22:0/182835476 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f431c1a6370 con 0x7f431c1027d0 2024-03-28T18:10:10.081 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:10.080+0000 7f431b400640 1 --2- 172.21.15.22:0/182835476 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f431c1027d0 0x7f431c100420 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f430c00c420 tx=0x7f430c009f90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:10:10.081 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:10.080+0000 7f4318c00640 1 -- 172.21.15.22:0/182835476 <== mon.2 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f430c003970 con 0x7f431c1027d0 2024-03-28T18:10:10.081 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:10.080+0000 7f4318c00640 1 -- 172.21.15.22:0/182835476 <== mon.2 v2:172.21.15.79:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f430c004020 con 0x7f431c1027d0 2024-03-28T18:10:10.081 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:10.080+0000 7f4320a00640 1 -- 172.21.15.22:0/182835476 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f431c1a6660 con 0x7f431c1027d0 2024-03-28T18:10:10.081 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:10.080+0000 7f4318c00640 1 -- 172.21.15.22:0/182835476 <== mon.2 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f430c00ba30 con 0x7f431c1027d0 2024-03-28T18:10:10.081 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:10.080+0000 7f4320a00640 1 -- 172.21.15.22:0/182835476 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f431c1a6bd0 con 0x7f431c1027d0 2024-03-28T18:10:10.083 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:10.084+0000 7f4324dc8740 1 Processor -- start 2024-03-28T18:10:10.084 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:10.084+0000 7f4324dc8740 1 -- start start 2024-03-28T18:10:10.085 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:10.084+0000 7f4324dc8740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55aabc98a8b0 0x55aabc98ac90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:10:10.085 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:10.084+0000 7f4324dc8740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55aabc98b260 0x55aabc993810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:10:10.085 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:10.084+0000 7f4318c00640 1 -- 172.21.15.22:0/182835476 <== mon.2 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 26) v1 ==== 137421+0+0 (secure 0 0 0) 0x7f430c01d020 con 0x7f431c1027d0 2024-03-28T18:10:10.085 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:10.084+0000 7f4324dc8740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55aabc993de0 0x55aabc996230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:10:10.085 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:10.084+0000 7f4324dc8740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aabc8ee2c0 con 0x55aabc993de0 2024-03-28T18:10:10.085 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:10.084+0000 7f4324dc8740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aabc896c90 con 0x55aabc98a8b0 2024-03-28T18:10:10.085 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:10.084+0000 7f4324dc8740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aabc8ea360 con 0x55aabc98b260 2024-03-28T18:10:10.085 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:10.085+0000 7f4312a00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55aabc993de0 0x55aabc996230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:10:10.085 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:10.085+0000 7f431be00640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55aabc98a8b0 0x55aabc98ac90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:10:10.086 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:10.085+0000 7f431be00640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55aabc98a8b0 0x55aabc98ac90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.22:52600/0 (socket says 172.21.15.22:52600) 2024-03-28T18:10:10.086 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:10.085+0000 7f4312a00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55aabc993de0 0x55aabc996230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:45884/0 (socket says 172.21.15.22:45884) 2024-03-28T18:10:10.086 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:10.085+0000 7f431be00640 1 -- 172.21.15.22:0/2431917992 learned_addr learned my addr 172.21.15.22:0/2431917992 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-28T18:10:10.086 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:10.085+0000 7f431b400640 1 --2- 172.21.15.22:0/2431917992 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55aabc98b260 0x55aabc993810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:10:10.086 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:10.085+0000 7f4312a00640 1 -- 172.21.15.22:0/2431917992 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55aabc98b260 msgr2=0x55aabc993810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T18:10:10.086 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:10.085+0000 7f4312a00640 1 --2- 172.21.15.22:0/2431917992 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55aabc98b260 0x55aabc993810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:10:10.086 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:10.085+0000 7f4312a00640 1 -- 172.21.15.22:0/2431917992 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55aabc98a8b0 msgr2=0x55aabc98ac90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T18:10:10.086 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:10.085+0000 7f4312a00640 1 --2- 172.21.15.22:0/2431917992 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55aabc98a8b0 0x55aabc98ac90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:10:10.086 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:10.085+0000 7f4312a00640 1 -- 172.21.15.22:0/2431917992 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55aabc8bb080 con 0x55aabc993de0 2024-03-28T18:10:10.087 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:10.086+0000 7f4312a00640 1 --2- 172.21.15.22:0/2431917992 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55aabc993de0 0x55aabc996230 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7f430402c150 tx=0x7f4304009560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d061d4184bd33ed2 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:10:10.087 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:10.086+0000 7f430be00640 1 -- 172.21.15.22:0/2431917992 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4304043020 con 0x55aabc993de0 2024-03-28T18:10:10.087 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:10.086+0000 7f430be00640 1 -- 172.21.15.22:0/2431917992 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f4304034d80 con 0x55aabc993de0 2024-03-28T18:10:10.087 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:10.086+0000 7f430be00640 1 -- 172.21.15.22:0/2431917992 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4304039040 con 0x55aabc993de0 2024-03-28T18:10:10.087 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:10.086+0000 7f4318c00640 1 --2- 172.21.15.22:0/182835476 >> [v2:172.21.15.22:6800/1357365082,v1:172.21.15.22:6801/1357365082] conn(0x7f43000a2a20 0x7f43000a4ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:10:10.087 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:10.087+0000 7f4318c00640 1 -- 172.21.15.22:0/182835476 <== mon.2 v2:172.21.15.79:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 8411+0+0 (secure 0 0 0) 0x7f430c0ced00 con 0x7f431c1027d0 2024-03-28T18:10:10.088 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:10.087+0000 7f431be00640 1 --2- 172.21.15.22:0/182835476 >> [v2:172.21.15.22:6800/1357365082,v1:172.21.15.22:6801/1357365082] conn(0x7f43000a2a20 0x7f43000a4ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:10:10.088 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:10.087+0000 7f4324dc8740 1 -- 172.21.15.22:0/2431917992 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55aabc993de0 msgr2=0x55aabc996230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:10:10.088 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:10.087+0000 7f4324dc8740 1 --2- 172.21.15.22:0/2431917992 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55aabc993de0 0x55aabc996230 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7f430402c150 tx=0x7f4304009560 comp rx=0 tx=0).stop 2024-03-28T18:10:10.088 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/power/swsusp-and-swap-files.txt 2024-03-28T18:10:10.091 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:10.087+0000 7f4324dc8740 1 -- 172.21.15.22:0/2431917992 shutdown_connections 2024-03-28T18:10:10.091 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:10.087+0000 7f4324dc8740 1 --2- 172.21.15.22:0/2431917992 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55aabc993de0 0x55aabc996230 unknown :-1 s=CLOSED pgs=632 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:10:10.091 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:10.087+0000 7f4324dc8740 1 --2- 172.21.15.22:0/2431917992 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55aabc98b260 0x55aabc993810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:10:10.091 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:10.087+0000 7f4324dc8740 1 --2- 172.21.15.22:0/2431917992 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55aabc98a8b0 0x55aabc98ac90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:10:10.092 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:10.087+0000 7f4324dc8740 1 -- 172.21.15.22:0/2431917992 >> 172.21.15.22:0/2431917992 conn(0x55aabc984f80 msgr2=0x55aabc985360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:10:10.092 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:10.087+0000 7f4324dc8740 1 -- 172.21.15.22:0/2431917992 shutdown_connections 2024-03-28T18:10:10.092 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:10.088+0000 7f4324dc8740 1 -- 172.21.15.22:0/2431917992 wait complete. 2024-03-28T18:10:10.092 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:10.088+0000 7f431be00640 1 --2- 172.21.15.22:0/182835476 >> [v2:172.21.15.22:6800/1357365082,v1:172.21.15.22:6801/1357365082] conn(0x7f43000a2a20 0x7f43000a4ee0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f431400bb30 tx=0x7f4314008040 comp rx=0 tx=0).ready entity=mgr.15522 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:10:10.092 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:10.088+0000 7f4324dc8740 1 -- start start 2024-03-28T18:10:10.092 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:10.088+0000 7f4324dc8740 1 Processor -- start 2024-03-28T18:10:10.092 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:10.089+0000 7f4324dc8740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55aabc98a8b0 0x55aabc993710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:10:10.092 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:10.089+0000 7f4324dc8740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55aabc98b260 0x55aabc990aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:10:10.092 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:10.089+0000 7f4324dc8740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55aabc993de0 0x55aabc9910b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:10:10.092 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:10.089+0000 7f4324dc8740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aabc996cf0 con 0x55aabc98b260 2024-03-28T18:10:10.092 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:10.089+0000 7f4324dc8740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aabc8aa0d0 con 0x55aabc993de0 2024-03-28T18:10:10.092 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:10.089+0000 7f4324dc8740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aabc996e70 con 0x55aabc98a8b0 2024-03-28T18:10:10.092 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:10.089+0000 7f4324dc8740 10 client.0 _ll_register_callbacks cb 0x55aabc8aee80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-28T18:10:10.092 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:10.089+0000 7f4312a00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55aabc98b260 0x55aabc990aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:10:10.093 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:10.089+0000 7f431be00640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55aabc993de0 0x55aabc9910b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:10:10.093 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:10.089+0000 7f4312a00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55aabc98b260 0x55aabc990aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:45894/0 (socket says 172.21.15.22:45894) 2024-03-28T18:10:10.093 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:10.089+0000 7f4312a00640 1 -- 172.21.15.22:0/3181323489 learned_addr learned my addr 172.21.15.22:0/3181323489 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-28T18:10:10.093 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:10.090+0000 7f431b400640 1 --2- 172.21.15.22:0/3181323489 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55aabc98a8b0 0x55aabc993710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:10:10.093 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:10.090+0000 7f4312a00640 1 -- 172.21.15.22:0/3181323489 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55aabc98a8b0 msgr2=0x55aabc993710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T18:10:10.093 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:10.090+0000 7f4312a00640 1 --2- 172.21.15.22:0/3181323489 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55aabc98a8b0 0x55aabc993710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:10:10.093 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:10.090+0000 7f4312a00640 1 -- 172.21.15.22:0/3181323489 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55aabc993de0 msgr2=0x55aabc9910b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T18:10:10.093 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:10.090+0000 7f4312a00640 1 --2- 172.21.15.22:0/3181323489 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55aabc993de0 0x55aabc9910b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:10:10.093 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:10.090+0000 7f4312a00640 1 -- 172.21.15.22:0/3181323489 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55aabc6023f0 con 0x55aabc98b260 2024-03-28T18:10:10.093 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:10.090+0000 7f4312a00640 1 --2- 172.21.15.22:0/3181323489 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55aabc98b260 0x55aabc990aa0 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7f4304034820 tx=0x7f4304034850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bb60652f7da75b9f server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:10:10.093 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:10.090+0000 7f4308c00640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-03-28T18:10:10.093 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:10.090+0000 7f4308c00640 1 -- 172.21.15.22:0/3181323489 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f430404c030 con 0x55aabc98b260 2024-03-28T18:10:10.093 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:10.091+0000 7f4308c00640 1 -- 172.21.15.22:0/3181323489 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 8411+0+0 (secure 0 0 0) 0x7f4304034b80 con 0x55aabc98b260 2024-03-28T18:10:10.093 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:10.091+0000 7f4324dc8740 1 -- 172.21.15.22:0/3181323489 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f4304034d80 con 0x55aabc98b260 2024-03-28T18:10:10.094 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:10.091+0000 7f4308c00640 1 -- 172.21.15.22:0/3181323489 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4304059050 con 0x55aabc98b260 2024-03-28T18:10:10.094 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:10.091+0000 7f4324dc8740 10 client.16245 fetch_fsmap learned FSMap version 31 2024-03-28T18:10:10.094 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:10.091+0000 7f4324dc8740 1 -- 172.21.15.22:0/3181323489 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) v3 -- 0x55aabc6cc7d0 con 0x55aabc98b260 2024-03-28T18:10:10.094 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:10.091+0000 7f4308c00640 1 -- 172.21.15.22:0/3181323489 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f4304034d80 con 0x55aabc98b260 2024-03-28T18:10:10.094 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:10.091+0000 7f4308c00640 1 -- 172.21.15.22:0/3181323489 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 31) v1 ==== 7950+0+0 (secure 0 0 0) 0x7f4304003710 con 0x55aabc98b260 2024-03-28T18:10:10.094 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:10.092+0000 7f4324dc8740 10 client.16245 fetch_fsmap finished waiting for FSMap version 31 2024-03-28T18:10:10.094 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:10.092+0000 7f4324dc8740 10 client.16245 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.i 2024-03-28T18:10:10.094 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:10.092+0000 7f4324dc8740 20 client.16245 populate_metadata read hostname 'smithi022' 2024-03-28T18:10:10.094 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:10.092+0000 7f4324dc8740 1 --2- 172.21.15.22:0/3181323489 >> [v2:172.21.15.134:6838/3400263217,v1:172.21.15.134:6839/3400263217] conn(0x55aabc9a3a90 0x55aabc9a5e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:10:10.094 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:10.092+0000 7f4324dc8740 4 client.16245 mds_command: new command op to 24502 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-28T18:10:10.094 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:10.092+0000 7f4324dc8740 1 -- 172.21.15.22:0/3181323489 --> [v2:172.21.15.134:6838/3400263217,v1:172.21.15.134:6839/3400263217] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55aabc8fb800 con 0x55aabc9a3a90 2024-03-28T18:10:10.095 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:10.093+0000 7f431b400640 1 --2- 172.21.15.22:0/3181323489 >> [v2:172.21.15.134:6838/3400263217,v1:172.21.15.134:6839/3400263217] conn(0x55aabc9a3a90 0x55aabc9a5e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:10:10.096 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:10.094+0000 7f431b400640 1 --2- 172.21.15.22:0/3181323489 >> [v2:172.21.15.134:6838/3400263217,v1:172.21.15.134:6839/3400263217] conn(0x55aabc9a3a90 0x55aabc9a5e80 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=5f8310abad6d44f server_cookie=560a545928278d3a in_seq=0 out_seq=0 2024-03-28T18:10:10.096 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:10.094+0000 7f4308c00640 10 client.16245 ms_handle_connect on v2:172.21.15.134:6838/3400263217 2024-03-28T18:10:10.096 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:10.095+0000 7f4308c00640 1 -- 172.21.15.22:0/3181323489 <== mds.4 v2:172.21.15.134:6838/3400263217 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x55aabc8fb800 con 0x55aabc9a3a90 2024-03-28T18:10:10.096 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:10.095+0000 7f4308c00640 10 client.16245 handle_command_reply: tid=0 2024-03-28T18:10:10.096 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:10.095+0000 7f4324dc8740 1 client.16245 shutdown 2024-03-28T18:10:10.096 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:10.095+0000 7f4324dc8740 1 -- 172.21.15.22:0/3181323489 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55aabc98b260 msgr2=0x55aabc990aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:10:10.096 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:10.095+0000 7f4324dc8740 1 --2- 172.21.15.22:0/3181323489 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55aabc98b260 0x55aabc990aa0 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7f4304034820 tx=0x7f4304034850 comp rx=0 tx=0).stop 2024-03-28T18:10:10.096 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:10.095+0000 7f4324dc8740 1 -- 172.21.15.22:0/3181323489 shutdown_connections 2024-03-28T18:10:10.096 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:10.095+0000 7f4324dc8740 1 --2- 172.21.15.22:0/3181323489 >> [v2:172.21.15.134:6838/3400263217,v1:172.21.15.134:6839/3400263217] conn(0x55aabc9a3a90 0x55aabc9a5e80 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:10:10.096 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:10.095+0000 7f4324dc8740 1 --2- 172.21.15.22:0/3181323489 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55aabc993de0 0x55aabc9910b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:10:10.096 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:10.095+0000 7f4324dc8740 1 --2- 172.21.15.22:0/3181323489 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55aabc98b260 0x55aabc990aa0 unknown :-1 s=CLOSED pgs=633 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:10:10.096 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:10.095+0000 7f4324dc8740 1 --2- 172.21.15.22:0/3181323489 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55aabc98a8b0 0x55aabc993710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:10:10.096 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:10.095+0000 7f4308c00640 0 client.16245 ms_handle_reset on v2:172.21.15.134:6838/3400263217 2024-03-28T18:10:10.096 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:10.095+0000 7f4324dc8740 1 -- 172.21.15.22:0/3181323489 >> 172.21.15.22:0/3181323489 conn(0x55aabc984f80 msgr2=0x55aabc986cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:10:10.097 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:10.096+0000 7f4324dc8740 1 -- 172.21.15.22:0/3181323489 shutdown_connections 2024-03-28T18:10:10.097 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:10.096+0000 7f4324dc8740 1 -- 172.21.15.22:0/3181323489 wait complete. 2024-03-28T18:10:10.097 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:10.096+0000 7f4324dc8740 20 client.16245 trim_cache size 0 max 16384 2024-03-28T18:10:10.097 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/power/swsusp-dmcrypt.txt 2024-03-28T18:10:10.105 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/power/swsusp.txt 2024-03-28T18:10:10.111 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:10.112+0000 7f4324dc8740 1 Processor -- start 2024-03-28T18:10:10.111 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:10.112+0000 7f4324dc8740 1 -- start start 2024-03-28T18:10:10.113 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:10.112+0000 7f4324dc8740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55aabc98a8b0 0x55aabc9843a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:10:10.113 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:10.112+0000 7f4324dc8740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55aabc98b260 0x55aabc9931d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:10:10.113 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:10.112+0000 7f4324dc8740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55aabc993de0 0x55aabc993710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:10:10.113 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:10.112+0000 7f4324dc8740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f431c1af0a0 con 0x55aabc98a8b0 2024-03-28T18:10:10.113 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:10.112+0000 7f4324dc8740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f431c1b0ae0 con 0x55aabc993de0 2024-03-28T18:10:10.113 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:10.112+0000 7f4324dc8740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f431c1af700 con 0x55aabc98b260 2024-03-28T18:10:10.113 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:10.112+0000 7f431b400640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55aabc98a8b0 0x55aabc9843a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:10:10.113 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:10.113+0000 7f431be00640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55aabc993de0 0x55aabc993710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:10:10.113 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:10.113+0000 7f431be00640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55aabc993de0 0x55aabc993710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.22:52628/0 (socket says 172.21.15.22:52628) 2024-03-28T18:10:10.113 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:10.113+0000 7f431b400640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55aabc98a8b0 0x55aabc9843a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:45902/0 (socket says 172.21.15.22:45902) 2024-03-28T18:10:10.113 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:10.113+0000 7f431be00640 1 -- 172.21.15.22:0/2864738327 learned_addr learned my addr 172.21.15.22:0/2864738327 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-28T18:10:10.114 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:10.113+0000 7f4312a00640 1 --2- 172.21.15.22:0/2864738327 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55aabc98b260 0x55aabc9931d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:10:10.114 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:10.113+0000 7f431be00640 1 -- 172.21.15.22:0/2864738327 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55aabc98b260 msgr2=0x55aabc9931d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T18:10:10.114 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:10.113+0000 7f431be00640 1 --2- 172.21.15.22:0/2864738327 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55aabc98b260 0x55aabc9931d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:10:10.114 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:10.113+0000 7f431be00640 1 -- 172.21.15.22:0/2864738327 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55aabc98a8b0 msgr2=0x55aabc9843a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T18:10:10.114 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:10.113+0000 7f431be00640 1 --2- 172.21.15.22:0/2864738327 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55aabc98a8b0 0x55aabc9843a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:10:10.114 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:10.113+0000 7f431be00640 1 -- 172.21.15.22:0/2864738327 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55aabc9913e0 con 0x55aabc993de0 2024-03-28T18:10:10.114 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:10.113+0000 7f431b400640 1 --2- 172.21.15.22:0/2864738327 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55aabc98a8b0 0x55aabc9843a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-28T18:10:10.114 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:10.113+0000 7f431be00640 1 --2- 172.21.15.22:0/2864738327 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55aabc993de0 0x55aabc993710 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f431400b2f0 tx=0x7f4314013000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f9eb551baa97c096 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:10:10.115 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:10.114+0000 7f430b400640 1 -- 172.21.15.22:0/2864738327 <== mon.2 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4314026070 con 0x55aabc993de0 2024-03-28T18:10:10.115 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:10.114+0000 7f430b400640 1 -- 172.21.15.22:0/2864738327 <== mon.2 v2:172.21.15.79:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f431401d040 con 0x55aabc993de0 2024-03-28T18:10:10.115 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:10.114+0000 7f430b400640 1 -- 172.21.15.22:0/2864738327 <== mon.2 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f431401c040 con 0x55aabc993de0 2024-03-28T18:10:10.115 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:10.115+0000 7f4324dc8740 1 -- 172.21.15.22:0/2864738327 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55aabc993de0 msgr2=0x55aabc993710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:10:10.115 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:10.115+0000 7f4324dc8740 1 --2- 172.21.15.22:0/2864738327 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55aabc993de0 0x55aabc993710 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f431400b2f0 tx=0x7f4314013000 comp rx=0 tx=0).stop 2024-03-28T18:10:10.115 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:10.115+0000 7f4324dc8740 1 -- 172.21.15.22:0/2864738327 shutdown_connections 2024-03-28T18:10:10.115 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:10.115+0000 7f4324dc8740 1 --2- 172.21.15.22:0/2864738327 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55aabc993de0 0x55aabc993710 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:10:10.115 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:10.115+0000 7f4324dc8740 1 --2- 172.21.15.22:0/2864738327 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55aabc98b260 0x55aabc9931d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:10:10.116 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:10.115+0000 7f4324dc8740 1 --2- 172.21.15.22:0/2864738327 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55aabc98a8b0 0x55aabc9843a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:10:10.116 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:10.115+0000 7f4324dc8740 1 -- 172.21.15.22:0/2864738327 >> 172.21.15.22:0/2864738327 conn(0x55aabc984f80 msgr2=0x55aabc98bed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:10:10.116 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:10.115+0000 7f4324dc8740 1 -- 172.21.15.22:0/2864738327 shutdown_connections 2024-03-28T18:10:10.116 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:10.115+0000 7f4324dc8740 1 -- 172.21.15.22:0/2864738327 wait complete. 2024-03-28T18:10:10.116 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/power/tricks.txt 2024-03-28T18:10:10.118 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:10.116+0000 7f4324dc8740 1 -- start start 2024-03-28T18:10:10.119 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:10.116+0000 7f4324dc8740 1 Processor -- start 2024-03-28T18:10:10.119 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:10.116+0000 7f4324dc8740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55aabc98a8b0 0x55aabc9a7a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:10:10.119 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:10.116+0000 7f4324dc8740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55aabc98b260 0x55aabc9a7f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:10:10.119 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:10.116+0000 7f4324dc8740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55aabc993de0 0x55aabc9fefd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:10:10.119 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:10.116+0000 7f4324dc8740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f431c1b03c0 con 0x55aabc98b260 2024-03-28T18:10:10.119 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:10.116+0000 7f4324dc8740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f431c1afd60 con 0x55aabc993de0 2024-03-28T18:10:10.119 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:10.116+0000 7f4324dc8740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f431c1ad2c0 con 0x55aabc98a8b0 2024-03-28T18:10:10.119 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:10.116+0000 7f4324dc8740 10 client.0 _ll_register_callbacks cb 0x7f431400b1c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-28T18:10:10.119 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:10.117+0000 7f4312a00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55aabc98b260 0x55aabc9a7f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:10:10.119 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:10.117+0000 7f4312a00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55aabc98b260 0x55aabc9a7f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:45904/0 (socket says 172.21.15.22:45904) 2024-03-28T18:10:10.119 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:10.117+0000 7f4312a00640 1 -- 172.21.15.22:0/1143433848 learned_addr learned my addr 172.21.15.22:0/1143433848 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-28T18:10:10.119 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:10.117+0000 7f431b400640 1 --2- 172.21.15.22:0/1143433848 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55aabc98a8b0 0x55aabc9a7a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:10:10.119 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:10.117+0000 7f431be00640 1 --2- 172.21.15.22:0/1143433848 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55aabc993de0 0x55aabc9fefd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:10:10.119 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:10.117+0000 7f4312a00640 1 -- 172.21.15.22:0/1143433848 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55aabc98a8b0 msgr2=0x55aabc9a7a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T18:10:10.120 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:10.117+0000 7f4312a00640 1 --2- 172.21.15.22:0/1143433848 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55aabc98a8b0 0x55aabc9a7a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:10:10.120 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:10.117+0000 7f4312a00640 1 -- 172.21.15.22:0/1143433848 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55aabc993de0 msgr2=0x55aabc9fefd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T18:10:10.120 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:10.117+0000 7f4312a00640 1 --2- 172.21.15.22:0/1143433848 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55aabc993de0 0x55aabc9fefd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:10:10.120 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:10.117+0000 7f4312a00640 1 -- 172.21.15.22:0/1143433848 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55aabc9ff590 con 0x55aabc98b260 2024-03-28T18:10:10.120 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:10.117+0000 7f431b400640 1 --2- 172.21.15.22:0/1143433848 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55aabc98a8b0 0x55aabc9a7a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-28T18:10:10.120 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:10.117+0000 7f431be00640 1 --2- 172.21.15.22:0/1143433848 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55aabc993de0 0x55aabc9fefd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-28T18:10:10.120 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:10.117+0000 7f4312a00640 1 --2- 172.21.15.22:0/1143433848 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55aabc98b260 0x55aabc9a7f50 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x55aabc992070 tx=0x7f4304004500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2078c5ad30114f9d server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:10:10.120 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:10.118+0000 7f4308c00640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-03-28T18:10:10.120 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:10.118+0000 7f4308c00640 1 -- 172.21.15.22:0/1143433848 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f430404c040 con 0x55aabc98b260 2024-03-28T18:10:10.120 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:10.118+0000 7f4308c00640 1 -- 172.21.15.22:0/1143433848 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 8411+0+0 (secure 0 0 0) 0x7f4304034cc0 con 0x55aabc98b260 2024-03-28T18:10:10.120 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:10.118+0000 7f4324dc8740 1 -- 172.21.15.22:0/1143433848 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f431401d040 con 0x55aabc98b260 2024-03-28T18:10:10.120 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:10.118+0000 7f4308c00640 1 -- 172.21.15.22:0/1143433848 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4304065050 con 0x55aabc98b260 2024-03-28T18:10:10.120 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:10.118+0000 7f4324dc8740 10 client.16251 fetch_fsmap learned FSMap version 31 2024-03-28T18:10:10.120 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:10.118+0000 7f4324dc8740 1 -- 172.21.15.22:0/1143433848 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) v3 -- 0x55aabca00870 con 0x55aabc98b260 2024-03-28T18:10:10.121 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:10.118+0000 7f4308c00640 1 -- 172.21.15.22:0/1143433848 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f431401d040 con 0x55aabc98b260 2024-03-28T18:10:10.121 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:10.119+0000 7f4308c00640 1 -- 172.21.15.22:0/1143433848 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 31) v1 ==== 7950+0+0 (secure 0 0 0) 0x7f430402c5b0 con 0x55aabc98b260 2024-03-28T18:10:10.121 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:10.119+0000 7f4324dc8740 10 client.16251 fetch_fsmap finished waiting for FSMap version 31 2024-03-28T18:10:10.121 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:10.119+0000 7f4324dc8740 10 client.16251 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.i 2024-03-28T18:10:10.121 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:10.119+0000 7f4324dc8740 20 client.16251 populate_metadata read hostname 'smithi022' 2024-03-28T18:10:10.121 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:10.119+0000 7f4324dc8740 1 --2- 172.21.15.22:0/1143433848 >> [v2:172.21.15.134:6838/3400263217,v1:172.21.15.134:6839/3400263217] conn(0x55aabca00ee0 0x55aabca03300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:10:10.121 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:10.119+0000 7f4324dc8740 4 client.16251 mds_command: new command op to 24502 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-28T18:10:10.121 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:10.119+0000 7f4324dc8740 1 -- 172.21.15.22:0/1143433848 --> [v2:172.21.15.134:6838/3400263217,v1:172.21.15.134:6839/3400263217] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55aabc8f5b50 con 0x55aabca00ee0 2024-03-28T18:10:10.122 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:10.119+0000 7f431b400640 1 --2- 172.21.15.22:0/1143433848 >> [v2:172.21.15.134:6838/3400263217,v1:172.21.15.134:6839/3400263217] conn(0x55aabca00ee0 0x55aabca03300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:10:10.122 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:10.120+0000 7f431b400640 1 --2- 172.21.15.22:0/1143433848 >> [v2:172.21.15.134:6838/3400263217,v1:172.21.15.134:6839/3400263217] conn(0x55aabca00ee0 0x55aabca03300 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=1f3a4b27db4f933f server_cookie=bddab1c34a313cf2 in_seq=0 out_seq=0 2024-03-28T18:10:10.123 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:10.120+0000 7f4308c00640 10 client.16251 ms_handle_connect on v2:172.21.15.134:6838/3400263217 2024-03-28T18:10:10.123 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:10.121+0000 7f4308c00640 1 -- 172.21.15.22:0/1143433848 <== mds.4 v2:172.21.15.134:6838/3400263217 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55aabc8f5b50 con 0x55aabca00ee0 2024-03-28T18:10:10.123 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:10.121+0000 7f4308c00640 10 client.16251 handle_command_reply: tid=0 2024-03-28T18:10:10.123 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:10.121+0000 7f4324dc8740 1 client.16251 shutdown 2024-03-28T18:10:10.123 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:10.121+0000 7f4324dc8740 1 -- 172.21.15.22:0/1143433848 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55aabc98b260 msgr2=0x55aabc9a7f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:10:10.123 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:10.121+0000 7f4324dc8740 1 --2- 172.21.15.22:0/1143433848 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55aabc98b260 0x55aabc9a7f50 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x55aabc992070 tx=0x7f4304004500 comp rx=0 tx=0).stop 2024-03-28T18:10:10.123 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:10.121+0000 7f4324dc8740 1 -- 172.21.15.22:0/1143433848 shutdown_connections 2024-03-28T18:10:10.123 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:10.121+0000 7f4324dc8740 1 --2- 172.21.15.22:0/1143433848 >> [v2:172.21.15.134:6838/3400263217,v1:172.21.15.134:6839/3400263217] conn(0x55aabca00ee0 0x55aabca03300 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:10:10.123 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:10.121+0000 7f4324dc8740 1 --2- 172.21.15.22:0/1143433848 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55aabc993de0 0x55aabc9fefd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:10:10.123 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:10.121+0000 7f4324dc8740 1 --2- 172.21.15.22:0/1143433848 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55aabc98b260 0x55aabc9a7f50 unknown :-1 s=CLOSED pgs=634 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:10:10.123 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:10.121+0000 7f4324dc8740 1 --2- 172.21.15.22:0/1143433848 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55aabc98a8b0 0x55aabc9a7a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:10:10.123 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:10.121+0000 7f4308c00640 0 client.16251 ms_handle_reset on v2:172.21.15.134:6838/3400263217 2024-03-28T18:10:10.123 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:10.121+0000 7f4324dc8740 1 -- 172.21.15.22:0/1143433848 >> 172.21.15.22:0/1143433848 conn(0x55aabc984f80 msgr2=0x55aabc986060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:10:10.123 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:10.121+0000 7f4324dc8740 1 -- 172.21.15.22:0/1143433848 shutdown_connections 2024-03-28T18:10:10.124 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:10.121+0000 7f4324dc8740 1 -- 172.21.15.22:0/1143433848 wait complete. 2024-03-28T18:10:10.124 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:10.122+0000 7f4324dc8740 20 client.16251 trim_cache size 0 max 16384 2024-03-28T18:10:10.124 INFO:teuthology.orchestra.run.smithi022.stdout:[] 2024-03-28T18:10:10.125 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:10.122+0000 7f4320a00640 1 -- 172.21.15.22:0/182835476 >> [v2:172.21.15.22:6800/1357365082,v1:172.21.15.22:6801/1357365082] conn(0x7f43000a2a20 msgr2=0x7f43000a4ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:10:10.125 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:10.122+0000 7f4320a00640 1 --2- 172.21.15.22:0/182835476 >> [v2:172.21.15.22:6800/1357365082,v1:172.21.15.22:6801/1357365082] conn(0x7f43000a2a20 0x7f43000a4ee0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f431400bb30 tx=0x7f4314008040 comp rx=0 tx=0).stop 2024-03-28T18:10:10.125 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:10.122+0000 7f4320a00640 1 -- 172.21.15.22:0/182835476 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f431c1027d0 msgr2=0x7f431c100420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:10:10.125 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:10.122+0000 7f4320a00640 1 --2- 172.21.15.22:0/182835476 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f431c1027d0 0x7f431c100420 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f430c00c420 tx=0x7f430c009f90 comp rx=0 tx=0).stop 2024-03-28T18:10:10.125 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:10.122+0000 7f4320a00640 1 -- 172.21.15.22:0/182835476 shutdown_connections 2024-03-28T18:10:10.125 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:10.122+0000 7f4320a00640 1 --2- 172.21.15.22:0/182835476 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f431c10bba0 0x7f431c100ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:10:10.125 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:10.122+0000 7f4320a00640 1 --2- 172.21.15.22:0/182835476 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f431c1030f0 0x7f431c100960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:10:10.125 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:10.122+0000 7f4320a00640 1 --2- 172.21.15.22:0/182835476 >> [v2:172.21.15.22:6800/1357365082,v1:172.21.15.22:6801/1357365082] conn(0x7f43000a2a20 0x7f43000a4ee0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:10:10.125 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:10.122+0000 7f4320a00640 1 --2- 172.21.15.22:0/182835476 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f431c1027d0 0x7f431c100420 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:10:10.125 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:10.122+0000 7f4320a00640 1 -- 172.21.15.22:0/182835476 >> 172.21.15.22:0/182835476 conn(0x7f431c0fbfb0 msgr2=0x7f431c0fd740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:10:10.125 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:10.123+0000 7f4320a00640 1 -- 172.21.15.22:0/182835476 shutdown_connections 2024-03-28T18:10:10.125 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:10.123+0000 7f4320a00640 1 -- 172.21.15.22:0/182835476 wait complete. 2024-03-28T18:10:10.126 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/power/userland-swsusp.txt 2024-03-28T18:10:10.133 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/power/video.txt 2024-03-28T18:10:10.143 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-28T18:10:10.143 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2024-03-28T18:10:10.146 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/powerpc/ 2024-03-28T18:10:10.158 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/powerpc/00-INDEX 2024-03-28T18:10:10.164 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/powerpc/bootwrapper.txt 2024-03-28T18:10:10.175 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/powerpc/cpu_families.txt 2024-03-28T18:10:10.185 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/powerpc/cpu_features.txt 2024-03-28T18:10:10.195 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/powerpc/cxl.txt 2024-03-28T18:10:10.208 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/powerpc/eeh-pci-error-recovery.txt 2024-03-28T18:10:10.221 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/powerpc/firmware-assisted-dump.txt 2024-03-28T18:10:10.234 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/powerpc/hvcs.txt 2024-03-28T18:10:10.246 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/powerpc/mpc52xx.txt 2024-03-28T18:10:10.257 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/powerpc/pmu-ebb.txt 2024-03-28T18:10:10.271 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/powerpc/ptrace.txt 2024-03-28T18:10:10.283 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/powerpc/qe_firmware.txt 2024-03-28T18:10:10.299 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/powerpc/transactional_memory.txt 2024-03-28T18:10:10.314 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/pps/ 2024-03-28T18:10:10.329 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/pps/pps.txt 2024-03-28T18:10:10.337 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/prctl/ 2024-03-28T18:10:10.354 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/prctl/.gitignore 2024-03-28T18:10:10.359 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/prctl/Makefile 2024-03-28T18:10:10.374 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/prctl/disable-tsc-ctxt-sw-stress-test.c 2024-03-28T18:10:10.375 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:10.376+0000 7f3805800640 1 -- 172.21.15.22:0/235387541 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3800101530 msgr2=0x7f38001019b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:10:10.376 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:10.376+0000 7f3805800640 1 --2- 172.21.15.22:0/235387541 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3800101530 0x7f38001019b0 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7f37e8009900 tx=0x7f37e802bc30 comp rx=0 tx=0).stop 2024-03-28T18:10:10.376 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:10.376+0000 7f3805800640 1 -- 172.21.15.22:0/235387541 shutdown_connections 2024-03-28T18:10:10.376 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:10.376+0000 7f3805800640 1 --2- 172.21.15.22:0/235387541 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3800101ef0 0x7f380010e3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:10:10.376 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:10.376+0000 7f3805800640 1 --2- 172.21.15.22:0/235387541 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3800101530 0x7f38001019b0 unknown :-1 s=CLOSED pgs=635 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:10:10.376 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:10.376+0000 7f3805800640 1 --2- 172.21.15.22:0/235387541 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f38001075f0 0x7f38001079d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:10:10.376 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:10.376+0000 7f3805800640 1 -- 172.21.15.22:0/235387541 >> 172.21.15.22:0/235387541 conn(0x7f38000fd040 msgr2=0x7f38000ff460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:10:10.376 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:10.376+0000 7f3805800640 1 -- 172.21.15.22:0/235387541 shutdown_connections 2024-03-28T18:10:10.376 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:10.376+0000 7f3805800640 1 -- 172.21.15.22:0/235387541 wait complete. 2024-03-28T18:10:10.377 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:10.377+0000 7f3805800640 1 Processor -- start 2024-03-28T18:10:10.377 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:10.377+0000 7f3805800640 1 -- start start 2024-03-28T18:10:10.378 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:10.378+0000 7f3805800640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3800101530 0x7f380019a0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:10:10.378 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:10.378+0000 7f3805800640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f3800101ef0 0x7f380019a5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:10:10.378 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:10.378+0000 7f3805800640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f38001075f0 0x7f380019e960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:10:10.379 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:10.378+0000 7f3805800640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3800075460 con 0x7f3800101530 2024-03-28T18:10:10.379 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:10.378+0000 7f3805800640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38000752e0 con 0x7f3800101ef0 2024-03-28T18:10:10.379 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:10.378+0000 7f3805800640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38000755e0 con 0x7f38001075f0 2024-03-28T18:10:10.379 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:10.378+0000 7f37ffe00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3800101530 0x7f380019a0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:10:10.379 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:10.378+0000 7f37ffe00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3800101530 0x7f380019a0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:45926/0 (socket says 172.21.15.22:45926) 2024-03-28T18:10:10.379 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:10.378+0000 7f37ffe00640 1 -- 172.21.15.22:0/481830179 learned_addr learned my addr 172.21.15.22:0/481830179 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-28T18:10:10.379 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:10.378+0000 7f37f7400640 1 --2- 172.21.15.22:0/481830179 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f3800101ef0 0x7f380019a5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:10:10.379 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:10.378+0000 7f3804e00640 1 --2- 172.21.15.22:0/481830179 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f38001075f0 0x7f380019e960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:10:10.379 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:10.379+0000 7f37ffe00640 1 -- 172.21.15.22:0/481830179 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f38001075f0 msgr2=0x7f380019e960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:10:10.379 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:10.379+0000 7f37ffe00640 1 --2- 172.21.15.22:0/481830179 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f38001075f0 0x7f380019e960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:10:10.379 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:10.379+0000 7f37ffe00640 1 -- 172.21.15.22:0/481830179 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f3800101ef0 msgr2=0x7f380019a5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:10:10.379 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:10.379+0000 7f37ffe00640 1 --2- 172.21.15.22:0/481830179 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f3800101ef0 0x7f380019a5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:10:10.379 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:10.379+0000 7f37ffe00640 1 -- 172.21.15.22:0/481830179 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f380019f090 con 0x7f3800101530 2024-03-28T18:10:10.379 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:10.379+0000 7f37ffe00640 1 --2- 172.21.15.22:0/481830179 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3800101530 0x7f380019a0a0 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7f37f000ea80 tx=0x7f37f000ef40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:10:10.380 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:10.379+0000 7f37fd600640 1 -- 172.21.15.22:0/481830179 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37f000ce40 con 0x7f3800101530 2024-03-28T18:10:10.380 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:10.379+0000 7f37fd600640 1 -- 172.21.15.22:0/481830179 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f37f0005020 con 0x7f3800101530 2024-03-28T18:10:10.380 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:10.379+0000 7f3805800640 1 -- 172.21.15.22:0/481830179 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f380019f380 con 0x7f3800101530 2024-03-28T18:10:10.380 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:10.379+0000 7f37fd600640 1 -- 172.21.15.22:0/481830179 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37f00116d0 con 0x7f3800101530 2024-03-28T18:10:10.380 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:10.379+0000 7f3805800640 1 -- 172.21.15.22:0/481830179 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f38001a6c10 con 0x7f3800101530 2024-03-28T18:10:10.382 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:10.383+0000 7f37fd600640 1 -- 172.21.15.22:0/481830179 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 26) v1 ==== 137421+0+0 (secure 0 0 0) 0x7f37f00051d0 con 0x7f3800101530 2024-03-28T18:10:10.383 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/prctl/disable-tsc-on-off-stress-test.c 2024-03-28T18:10:10.383 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:10.383+0000 7f3809bd8740 1 Processor -- start 2024-03-28T18:10:10.384 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:10.383+0000 7f3809bd8740 1 -- start start 2024-03-28T18:10:10.384 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:10.384+0000 7f3809bd8740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560db13578b0 0x560db1357c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:10:10.384 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:10.384+0000 7f3809bd8740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560db1358260 0x560db1360810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:10:10.384 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:10.384+0000 7f3809bd8740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x560db1360de0 0x560db1363230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:10:10.384 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:10.384+0000 7f3809bd8740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x560db12bb2c0 con 0x560db13578b0 2024-03-28T18:10:10.384 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:10.384+0000 7f3809bd8740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x560db1263c90 con 0x560db1360de0 2024-03-28T18:10:10.384 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:10.384+0000 7f3809bd8740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x560db12b7360 con 0x560db1358260 2024-03-28T18:10:10.384 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:10.384+0000 7f37f7400640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x560db1360de0 0x560db1363230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:10:10.384 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:10.384+0000 7f37f7400640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x560db1360de0 0x560db1363230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.22:52662/0 (socket says 172.21.15.22:52662) 2024-03-28T18:10:10.385 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:10.384+0000 7f37f7400640 1 -- 172.21.15.22:0/2401073011 learned_addr learned my addr 172.21.15.22:0/2401073011 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-28T18:10:10.385 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:10.384+0000 7f3804e00640 1 --2- 172.21.15.22:0/2401073011 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560db13578b0 0x560db1357c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:10:10.385 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:10.385+0000 7f37ffe00640 1 --2- 172.21.15.22:0/2401073011 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560db1358260 0x560db1360810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:10:10.386 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:10.385+0000 7f37f7400640 1 -- 172.21.15.22:0/2401073011 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560db1358260 msgr2=0x560db1360810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T18:10:10.386 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:10.385+0000 7f37f7400640 1 --2- 172.21.15.22:0/2401073011 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560db1358260 0x560db1360810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:10:10.386 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:10.385+0000 7f37f7400640 1 -- 172.21.15.22:0/2401073011 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560db13578b0 msgr2=0x560db1357c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T18:10:10.386 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:10.385+0000 7f37f7400640 1 --2- 172.21.15.22:0/2401073011 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560db13578b0 0x560db1357c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:10:10.386 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:10.385+0000 7f37f7400640 1 -- 172.21.15.22:0/2401073011 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560db1288080 con 0x560db1360de0 2024-03-28T18:10:10.386 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:10.385+0000 7f37fd600640 1 --2- 172.21.15.22:0/481830179 >> [v2:172.21.15.22:6800/1357365082,v1:172.21.15.22:6801/1357365082] conn(0x7f37e40a2970 0x7f37e40a4e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:10:10.386 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:10.385+0000 7f37f7400640 1 --2- 172.21.15.22:0/2401073011 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x560db1360de0 0x560db1363230 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f37e8009900 tx=0x7f37e802c5d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5d0c1b9f17a400a6 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:10:10.386 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:10.385+0000 7f37fd600640 1 -- 172.21.15.22:0/481830179 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 8411+0+0 (secure 0 0 0) 0x7f37f00cf0f0 con 0x7f3800101530 2024-03-28T18:10:10.386 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:10.386+0000 7f3804e00640 1 --2- 172.21.15.22:0/481830179 >> [v2:172.21.15.22:6800/1357365082,v1:172.21.15.22:6801/1357365082] conn(0x7f37e40a2970 0x7f37e40a4e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:10:10.387 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:10.386+0000 7f37f5600640 1 -- 172.21.15.22:0/2401073011 <== mon.2 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37e802c810 con 0x560db1360de0 2024-03-28T18:10:10.387 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:10.386+0000 7f37f5600640 1 -- 172.21.15.22:0/2401073011 <== mon.2 v2:172.21.15.79:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f37e802c9b0 con 0x560db1360de0 2024-03-28T18:10:10.387 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:10.386+0000 7f37f5600640 1 -- 172.21.15.22:0/2401073011 <== mon.2 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37e80046c0 con 0x560db1360de0 2024-03-28T18:10:10.388 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:10.387+0000 7f3809bd8740 1 -- 172.21.15.22:0/2401073011 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x560db1360de0 msgr2=0x560db1363230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:10:10.389 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:10.387+0000 7f3809bd8740 1 --2- 172.21.15.22:0/2401073011 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x560db1360de0 0x560db1363230 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f37e8009900 tx=0x7f37e802c5d0 comp rx=0 tx=0).stop 2024-03-28T18:10:10.389 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:10.387+0000 7f3804e00640 1 --2- 172.21.15.22:0/481830179 >> [v2:172.21.15.22:6800/1357365082,v1:172.21.15.22:6801/1357365082] conn(0x7f37e40a2970 0x7f37e40a4e30 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f37f800cc20 tx=0x7f37f8005ec0 comp rx=0 tx=0).ready entity=mgr.15522 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:10:10.389 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:10.387+0000 7f3809bd8740 1 -- 172.21.15.22:0/2401073011 shutdown_connections 2024-03-28T18:10:10.389 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:10.387+0000 7f3809bd8740 1 --2- 172.21.15.22:0/2401073011 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x560db1360de0 0x560db1363230 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:10:10.389 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:10.387+0000 7f3809bd8740 1 --2- 172.21.15.22:0/2401073011 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560db1358260 0x560db1360810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:10:10.389 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:10.387+0000 7f3809bd8740 1 --2- 172.21.15.22:0/2401073011 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560db13578b0 0x560db1357c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:10:10.389 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:10.387+0000 7f3809bd8740 1 -- 172.21.15.22:0/2401073011 >> 172.21.15.22:0/2401073011 conn(0x560db1351f80 msgr2=0x560db1352360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:10:10.389 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:10.387+0000 7f3809bd8740 1 -- 172.21.15.22:0/2401073011 shutdown_connections 2024-03-28T18:10:10.389 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:10.387+0000 7f3809bd8740 1 -- 172.21.15.22:0/2401073011 wait complete. 2024-03-28T18:10:10.389 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:10.388+0000 7f3809bd8740 1 -- start start 2024-03-28T18:10:10.389 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:10.388+0000 7f3809bd8740 1 Processor -- start 2024-03-28T18:10:10.389 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:10.388+0000 7f3809bd8740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560db13578b0 0x560db1360620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:10:10.389 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:10.388+0000 7f3809bd8740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x560db1358260 0x560db135d9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:10:10.390 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:10.389+0000 7f3809bd8740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560db1360de0 0x560db135def0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:10:10.390 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:10.389+0000 7f3809bd8740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x560db1374d50 con 0x560db13578b0 2024-03-28T18:10:10.390 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:10.389+0000 7f3809bd8740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x560db12770d0 con 0x560db1358260 2024-03-28T18:10:10.390 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:10.389+0000 7f3809bd8740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x560db1374ed0 con 0x560db1360de0 2024-03-28T18:10:10.390 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:10.389+0000 7f3809bd8740 10 client.0 _ll_register_callbacks cb 0x560db127be80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-28T18:10:10.390 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:10.389+0000 7f37ffe00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560db13578b0 0x560db1360620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:10:10.390 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:10.389+0000 7f37ffe00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560db13578b0 0x560db1360620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:45948/0 (socket says 172.21.15.22:45948) 2024-03-28T18:10:10.390 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:10.389+0000 7f37ffe00640 1 -- 172.21.15.22:0/4243735460 learned_addr learned my addr 172.21.15.22:0/4243735460 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-28T18:10:10.390 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:10.389+0000 7f37f7400640 1 --2- 172.21.15.22:0/4243735460 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x560db1358260 0x560db135d9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:10:10.390 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:10.389+0000 7f3804e00640 1 --2- 172.21.15.22:0/4243735460 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560db1360de0 0x560db135def0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:10:10.392 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:10.389+0000 7f37ffe00640 1 -- 172.21.15.22:0/4243735460 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560db1360de0 msgr2=0x560db135def0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T18:10:10.392 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:10.389+0000 7f37ffe00640 1 --2- 172.21.15.22:0/4243735460 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560db1360de0 0x560db135def0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:10:10.392 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:10.389+0000 7f37ffe00640 1 -- 172.21.15.22:0/4243735460 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x560db1358260 msgr2=0x560db135d9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T18:10:10.393 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:10.389+0000 7f37ffe00640 1 --2- 172.21.15.22:0/4243735460 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x560db1358260 0x560db135d9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:10:10.393 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:10.389+0000 7f37ffe00640 1 -- 172.21.15.22:0/4243735460 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x560db0fcf3f0 con 0x560db13578b0 2024-03-28T18:10:10.393 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:10.390+0000 7f37ffe00640 1 --2- 172.21.15.22:0/4243735460 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560db13578b0 0x560db1360620 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7f37f000c700 tx=0x7f37f009b590 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c5b8641f828c6ae8 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:10:10.393 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:10.390+0000 7f37ee000640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-03-28T18:10:10.393 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:10.390+0000 7f37ee000640 1 -- 172.21.15.22:0/4243735460 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37f001f040 con 0x560db13578b0 2024-03-28T18:10:10.393 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:10.390+0000 7f37ee000640 1 -- 172.21.15.22:0/4243735460 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 8411+0+0 (secure 0 0 0) 0x7f37f0097e30 con 0x560db13578b0 2024-03-28T18:10:10.393 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:10.390+0000 7f3809bd8740 1 -- 172.21.15.22:0/4243735460 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f37e802c9b0 con 0x560db13578b0 2024-03-28T18:10:10.393 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:10.390+0000 7f37ee000640 1 -- 172.21.15.22:0/4243735460 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) v2 ==== 24+0+0 (secure 0 0 0) 0x7f37f009bec0 con 0x560db13578b0 2024-03-28T18:10:10.393 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:10.390+0000 7f3809bd8740 10 client.16263 fetch_fsmap learned FSMap version 31 2024-03-28T18:10:10.393 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:10.390+0000 7f3809bd8740 1 -- 172.21.15.22:0/4243735460 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) v3 -- 0x560db10997d0 con 0x560db13578b0 2024-03-28T18:10:10.393 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:10.391+0000 7f37ee000640 1 -- 172.21.15.22:0/4243735460 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f37e802c9b0 con 0x560db13578b0 2024-03-28T18:10:10.393 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:10.391+0000 7f37ee000640 1 -- 172.21.15.22:0/4243735460 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 31) v1 ==== 7950+0+0 (secure 0 0 0) 0x7f37f000f040 con 0x560db13578b0 2024-03-28T18:10:10.393 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:10.391+0000 7f3809bd8740 10 client.16263 fetch_fsmap finished waiting for FSMap version 31 2024-03-28T18:10:10.393 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:10.391+0000 7f3809bd8740 10 client.16263 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.k 2024-03-28T18:10:10.393 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:10.391+0000 7f3809bd8740 20 client.16263 populate_metadata read hostname 'smithi022' 2024-03-28T18:10:10.393 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:10.391+0000 7f3809bd8740 1 --2- 172.21.15.22:0/4243735460 >> [v2:172.21.15.79:6832/856896997,v1:172.21.15.79:6833/856896997] conn(0x560db1381af0 0x560db1383f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:10:10.394 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:10.391+0000 7f3809bd8740 4 client.16263 mds_command: new command op to 34442 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-28T18:10:10.394 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:10.391+0000 7f3809bd8740 1 -- 172.21.15.22:0/4243735460 --> [v2:172.21.15.79:6832/856896997,v1:172.21.15.79:6833/856896997] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x560db12c8800 con 0x560db1381af0 2024-03-28T18:10:10.394 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:10.391+0000 7f37f7400640 1 --2- 172.21.15.22:0/4243735460 >> [v2:172.21.15.79:6832/856896997,v1:172.21.15.79:6833/856896997] conn(0x560db1381af0 0x560db1383f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:10:10.394 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:10.392+0000 7f37f7400640 1 --2- 172.21.15.22:0/4243735460 >> [v2:172.21.15.79:6832/856896997,v1:172.21.15.79:6833/856896997] conn(0x560db1381af0 0x560db1383f40 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=2525bed18d8f0676 server_cookie=ffd1e96f5d256146 in_seq=0 out_seq=0 2024-03-28T18:10:10.394 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:10.392+0000 7f37ee000640 10 client.16263 ms_handle_connect on v2:172.21.15.79:6832/856896997 2024-03-28T18:10:10.394 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/prctl/disable-tsc-test.c 2024-03-28T18:10:10.395 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:10.393+0000 7f37ee000640 1 -- 172.21.15.22:0/4243735460 <== mds.3 v2:172.21.15.79:6832/856896997 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x560db12c8800 con 0x560db1381af0 2024-03-28T18:10:10.395 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:10.393+0000 7f37ee000640 10 client.16263 handle_command_reply: tid=0 2024-03-28T18:10:10.395 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:10.393+0000 7f3809bd8740 1 client.16263 shutdown 2024-03-28T18:10:10.395 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:10.393+0000 7f3809bd8740 1 -- 172.21.15.22:0/4243735460 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560db13578b0 msgr2=0x560db1360620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:10:10.395 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:10.393+0000 7f3809bd8740 1 --2- 172.21.15.22:0/4243735460 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560db13578b0 0x560db1360620 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7f37f000c700 tx=0x7f37f009b590 comp rx=0 tx=0).stop 2024-03-28T18:10:10.396 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:10.393+0000 7f3809bd8740 1 -- 172.21.15.22:0/4243735460 shutdown_connections 2024-03-28T18:10:10.396 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:10.393+0000 7f3809bd8740 1 --2- 172.21.15.22:0/4243735460 >> [v2:172.21.15.79:6832/856896997,v1:172.21.15.79:6833/856896997] conn(0x560db1381af0 0x560db1383f40 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:10:10.396 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:10.393+0000 7f3809bd8740 1 --2- 172.21.15.22:0/4243735460 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560db1360de0 0x560db135def0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:10:10.396 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:10.393+0000 7f3809bd8740 1 --2- 172.21.15.22:0/4243735460 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x560db1358260 0x560db135d9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:10:10.396 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:10.393+0000 7f3809bd8740 1 --2- 172.21.15.22:0/4243735460 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560db13578b0 0x560db1360620 unknown :-1 s=CLOSED pgs=637 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:10:10.396 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:10.393+0000 7f37ee000640 0 client.16263 ms_handle_reset on v2:172.21.15.79:6832/856896997 2024-03-28T18:10:10.396 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:10.393+0000 7f3809bd8740 1 -- 172.21.15.22:0/4243735460 >> 172.21.15.22:0/4243735460 conn(0x560db1351f80 msgr2=0x560db1353cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:10:10.396 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:10.394+0000 7f3809bd8740 1 -- 172.21.15.22:0/4243735460 shutdown_connections 2024-03-28T18:10:10.396 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:10.394+0000 7f3809bd8740 1 -- 172.21.15.22:0/4243735460 wait complete. 2024-03-28T18:10:10.396 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:10.394+0000 7f3809bd8740 20 client.16263 trim_cache size 0 max 16384 2024-03-28T18:10:10.402 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/prctl/no_new_privs.txt 2024-03-28T18:10:10.409 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:10.409+0000 7f3809bd8740 1 Processor -- start 2024-03-28T18:10:10.409 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:10.410+0000 7f3809bd8740 1 -- start start 2024-03-28T18:10:10.411 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:10.410+0000 7f3809bd8740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x560db13578b0 0x560db1359110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:10:10.411 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:10.410+0000 7f3809bd8740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560db1358260 0x560db13600e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:10:10.411 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:10.410+0000 7f3809bd8740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560db1360de0 0x560db1360620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:10:10.411 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:10.410+0000 7f3809bd8740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38000731f0 con 0x560db1358260 2024-03-28T18:10:10.411 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:10.410+0000 7f3809bd8740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38001a8140 con 0x560db13578b0 2024-03-28T18:10:10.411 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:10.410+0000 7f3809bd8740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38001a6de0 con 0x560db1360de0 2024-03-28T18:10:10.411 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:10.410+0000 7f37f7400640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560db1358260 0x560db13600e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:10:10.411 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:10.410+0000 7f37ffe00640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x560db13578b0 0x560db1359110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:10:10.411 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:10.410+0000 7f37ffe00640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x560db13578b0 0x560db1359110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.22:52670/0 (socket says 172.21.15.22:52670) 2024-03-28T18:10:10.412 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:10.410+0000 7f37ffe00640 1 -- 172.21.15.22:0/3165610776 learned_addr learned my addr 172.21.15.22:0/3165610776 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-28T18:10:10.412 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:10.411+0000 7f3804e00640 1 --2- 172.21.15.22:0/3165610776 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560db1360de0 0x560db1360620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:10:10.412 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:10.411+0000 7f37ffe00640 1 -- 172.21.15.22:0/3165610776 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560db1360de0 msgr2=0x560db1360620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T18:10:10.412 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:10.411+0000 7f37ffe00640 1 --2- 172.21.15.22:0/3165610776 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560db1360de0 0x560db1360620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:10:10.412 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:10.411+0000 7f37ffe00640 1 -- 172.21.15.22:0/3165610776 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560db1358260 msgr2=0x560db13600e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T18:10:10.412 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:10.411+0000 7f37ffe00640 1 --2- 172.21.15.22:0/3165610776 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560db1358260 0x560db13600e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:10:10.412 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:10.411+0000 7f37ffe00640 1 -- 172.21.15.22:0/3165610776 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560db1358000 con 0x560db13578b0 2024-03-28T18:10:10.412 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:10.411+0000 7f37f7400640 1 --2- 172.21.15.22:0/3165610776 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560db1358260 0x560db13600e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-28T18:10:10.412 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:10.411+0000 7f37ffe00640 1 --2- 172.21.15.22:0/3165610776 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x560db13578b0 0x560db1359110 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f37f0099a70 tx=0x7f37f009ad50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f1b778f341f15af7 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:10:10.412 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:10.411+0000 7f37f4c00640 1 -- 172.21.15.22:0/3165610776 <== mon.2 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37f001f070 con 0x560db13578b0 2024-03-28T18:10:10.412 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:10.411+0000 7f37f4c00640 1 -- 172.21.15.22:0/3165610776 <== mon.2 v2:172.21.15.79:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f37f009b7c0 con 0x560db13578b0 2024-03-28T18:10:10.412 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:10.411+0000 7f37f4c00640 1 -- 172.21.15.22:0/3165610776 <== mon.2 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37f009cb70 con 0x560db13578b0 2024-03-28T18:10:10.413 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:10.412+0000 7f3809bd8740 1 -- 172.21.15.22:0/3165610776 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x560db13578b0 msgr2=0x560db1359110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:10:10.413 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:10.412+0000 7f3809bd8740 1 --2- 172.21.15.22:0/3165610776 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x560db13578b0 0x560db1359110 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f37f0099a70 tx=0x7f37f009ad50 comp rx=0 tx=0).stop 2024-03-28T18:10:10.413 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:10.412+0000 7f3809bd8740 1 -- 172.21.15.22:0/3165610776 shutdown_connections 2024-03-28T18:10:10.413 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:10.412+0000 7f3809bd8740 1 --2- 172.21.15.22:0/3165610776 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560db1360de0 0x560db1360620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:10:10.414 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:10.412+0000 7f3809bd8740 1 --2- 172.21.15.22:0/3165610776 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560db1358260 0x560db13600e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:10:10.414 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:10.412+0000 7f3809bd8740 1 --2- 172.21.15.22:0/3165610776 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x560db13578b0 0x560db1359110 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:10:10.414 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:10.412+0000 7f3809bd8740 1 -- 172.21.15.22:0/3165610776 >> 172.21.15.22:0/3165610776 conn(0x560db1351f80 msgr2=0x560db1353880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:10:10.414 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:10.412+0000 7f3809bd8740 1 -- 172.21.15.22:0/3165610776 shutdown_connections 2024-03-28T18:10:10.414 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:10.412+0000 7f3809bd8740 1 -- 172.21.15.22:0/3165610776 wait complete. 2024-03-28T18:10:10.414 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/prctl/seccomp_filter.txt 2024-03-28T18:10:10.418 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:10.413+0000 7f3809bd8740 1 -- start start 2024-03-28T18:10:10.418 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:10.413+0000 7f3809bd8740 1 Processor -- start 2024-03-28T18:10:10.418 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:10.413+0000 7f3809bd8740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x560db13578b0 0x560db1375d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:10:10.418 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:10.413+0000 7f3809bd8740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560db1358260 0x560db1376290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:10:10.418 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:10.413+0000 7f3809bd8740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560db1360de0 0x560db13deb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:10:10.418 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:10.413+0000 7f3809bd8740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38001a7a20 con 0x560db1358260 2024-03-28T18:10:10.418 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:10.413+0000 7f3809bd8740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38001a73c0 con 0x560db13578b0 2024-03-28T18:10:10.418 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:10.413+0000 7f3809bd8740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38000713f0 con 0x560db1360de0 2024-03-28T18:10:10.418 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:10.414+0000 7f3809bd8740 10 client.0 _ll_register_callbacks cb 0x7f37f0009270 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-28T18:10:10.419 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:10.414+0000 7f37ffe00640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x560db13578b0 0x560db1375d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:10:10.419 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:10.414+0000 7f37f7400640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560db1358260 0x560db1376290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:10:10.419 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:10.414+0000 7f37ffe00640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x560db13578b0 0x560db1375d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.22:52684/0 (socket says 172.21.15.22:52684) 2024-03-28T18:10:10.419 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:10.414+0000 7f37ffe00640 1 -- 172.21.15.22:0/3886767646 learned_addr learned my addr 172.21.15.22:0/3886767646 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-28T18:10:10.419 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:10.414+0000 7f3804e00640 1 --2- 172.21.15.22:0/3886767646 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560db1360de0 0x560db13deb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:10:10.419 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:10.414+0000 7f37f7400640 1 -- 172.21.15.22:0/3886767646 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560db1360de0 msgr2=0x560db13deb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T18:10:10.419 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:10.414+0000 7f37f7400640 1 --2- 172.21.15.22:0/3886767646 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560db1360de0 0x560db13deb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:10:10.419 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:10.414+0000 7f37f7400640 1 -- 172.21.15.22:0/3886767646 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x560db13578b0 msgr2=0x560db1375d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T18:10:10.419 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:10.414+0000 7f37f7400640 1 --2- 172.21.15.22:0/3886767646 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x560db13578b0 0x560db1375d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:10:10.419 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:10.414+0000 7f37f7400640 1 -- 172.21.15.22:0/3886767646 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x560db13df170 con 0x560db1358260 2024-03-28T18:10:10.419 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:10.414+0000 7f37ffe00640 1 --2- 172.21.15.22:0/3886767646 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x560db13578b0 0x560db1375d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-28T18:10:10.419 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:10.415+0000 7f37f7400640 1 --2- 172.21.15.22:0/3886767646 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560db1358260 0x560db1376290 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7f37e8000c00 tx=0x7f37e8005350 comp rx=0 tx=0).ready entity=mon.0 client_cookie=af30a73d09a74f86 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:10:10.419 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:10.415+0000 7f37ee000640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-03-28T18:10:10.420 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:10.415+0000 7f37ee000640 1 -- 172.21.15.22:0/3886767646 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37e8005500 con 0x560db1358260 2024-03-28T18:10:10.420 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:10.415+0000 7f37ee000640 1 -- 172.21.15.22:0/3886767646 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 8411+0+0 (secure 0 0 0) 0x7f37e804e830 con 0x560db1358260 2024-03-28T18:10:10.420 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:10.415+0000 7f3809bd8740 1 -- 172.21.15.22:0/3886767646 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f37f009b7c0 con 0x560db1358260 2024-03-28T18:10:10.420 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:10.415+0000 7f37ee000640 1 -- 172.21.15.22:0/3886767646 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) v2 ==== 24+0+0 (secure 0 0 0) 0x7f37e805c050 con 0x560db1358260 2024-03-28T18:10:10.420 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:10.415+0000 7f3809bd8740 10 client.16269 fetch_fsmap learned FSMap version 31 2024-03-28T18:10:10.420 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:10.415+0000 7f3809bd8740 1 -- 172.21.15.22:0/3886767646 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) v3 -- 0x560db13e64f0 con 0x560db1358260 2024-03-28T18:10:10.420 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:10.416+0000 7f37ee000640 1 -- 172.21.15.22:0/3886767646 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f37f009b7c0 con 0x560db1358260 2024-03-28T18:10:10.420 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:10.416+0000 7f37ee000640 1 -- 172.21.15.22:0/3886767646 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 31) v1 ==== 7950+0+0 (secure 0 0 0) 0x7f37e8004b50 con 0x560db1358260 2024-03-28T18:10:10.420 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:10.416+0000 7f3809bd8740 10 client.16269 fetch_fsmap finished waiting for FSMap version 31 2024-03-28T18:10:10.420 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:10.416+0000 7f3809bd8740 10 client.16269 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.k 2024-03-28T18:10:10.420 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:10.416+0000 7f3809bd8740 20 client.16269 populate_metadata read hostname 'smithi022' 2024-03-28T18:10:10.420 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:10.416+0000 7f3809bd8740 1 --2- 172.21.15.22:0/3886767646 >> [v2:172.21.15.79:6832/856896997,v1:172.21.15.79:6833/856896997] conn(0x560db13e68a0 0x560db13e8cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:10:10.420 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:10.416+0000 7f3809bd8740 4 client.16269 mds_command: new command op to 34442 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-28T18:10:10.420 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:10.416+0000 7f3809bd8740 1 -- 172.21.15.22:0/3886767646 --> [v2:172.21.15.79:6832/856896997,v1:172.21.15.79:6833/856896997] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x560db12c2b50 con 0x560db13e68a0 2024-03-28T18:10:10.420 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:10.416+0000 7f37ffe00640 1 --2- 172.21.15.22:0/3886767646 >> [v2:172.21.15.79:6832/856896997,v1:172.21.15.79:6833/856896997] conn(0x560db13e68a0 0x560db13e8cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:10:10.421 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:10.417+0000 7f37ffe00640 1 --2- 172.21.15.22:0/3886767646 >> [v2:172.21.15.79:6832/856896997,v1:172.21.15.79:6833/856896997] conn(0x560db13e68a0 0x560db13e8cc0 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=db1da0b7ffd20e6f server_cookie=422b118c2a6f00f0 in_seq=0 out_seq=0 2024-03-28T18:10:10.421 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:10.417+0000 7f37ee000640 10 client.16269 ms_handle_connect on v2:172.21.15.79:6832/856896997 2024-03-28T18:10:10.421 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:10.417+0000 7f37ee000640 1 -- 172.21.15.22:0/3886767646 <== mds.3 v2:172.21.15.79:6832/856896997 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x560db12c2b50 con 0x560db13e68a0 2024-03-28T18:10:10.421 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:10.417+0000 7f37ee000640 10 client.16269 handle_command_reply: tid=0 2024-03-28T18:10:10.421 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:10.417+0000 7f3809bd8740 1 client.16269 shutdown 2024-03-28T18:10:10.421 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:10.418+0000 7f3809bd8740 1 -- 172.21.15.22:0/3886767646 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560db1358260 msgr2=0x560db1376290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:10:10.421 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:10.418+0000 7f3809bd8740 1 --2- 172.21.15.22:0/3886767646 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560db1358260 0x560db1376290 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7f37e8000c00 tx=0x7f37e8005350 comp rx=0 tx=0).stop 2024-03-28T18:10:10.421 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:10.418+0000 7f3809bd8740 1 -- 172.21.15.22:0/3886767646 shutdown_connections 2024-03-28T18:10:10.421 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:10.418+0000 7f3809bd8740 1 --2- 172.21.15.22:0/3886767646 >> [v2:172.21.15.79:6832/856896997,v1:172.21.15.79:6833/856896997] conn(0x560db13e68a0 0x560db13e8cc0 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:10:10.421 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:10.418+0000 7f3809bd8740 1 --2- 172.21.15.22:0/3886767646 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560db1360de0 0x560db13deb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:10:10.421 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:10.418+0000 7f3809bd8740 1 --2- 172.21.15.22:0/3886767646 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560db1358260 0x560db1376290 unknown :-1 s=CLOSED pgs=638 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:10:10.421 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:10.418+0000 7f3809bd8740 1 --2- 172.21.15.22:0/3886767646 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x560db13578b0 0x560db1375d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:10:10.421 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:10.418+0000 7f3809bd8740 1 -- 172.21.15.22:0/3886767646 >> 172.21.15.22:0/3886767646 conn(0x560db1351f80 msgr2=0x560db1352f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:10:10.421 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:10.418+0000 7f37ee000640 0 client.16269 ms_handle_reset on v2:172.21.15.79:6832/856896997 2024-03-28T18:10:10.422 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:10.418+0000 7f3809bd8740 1 -- 172.21.15.22:0/3886767646 shutdown_connections 2024-03-28T18:10:10.422 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:10.418+0000 7f3809bd8740 1 -- 172.21.15.22:0/3886767646 wait complete. 2024-03-28T18:10:10.422 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:10.418+0000 7f3809bd8740 20 client.16269 trim_cache size 0 max 16384 2024-03-28T18:10:10.422 INFO:teuthology.orchestra.run.smithi022.stdout:[] 2024-03-28T18:10:10.423 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:10.419+0000 7f3805800640 1 -- 172.21.15.22:0/481830179 >> [v2:172.21.15.22:6800/1357365082,v1:172.21.15.22:6801/1357365082] conn(0x7f37e40a2970 msgr2=0x7f37e40a4e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:10:10.423 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:10.419+0000 7f3805800640 1 --2- 172.21.15.22:0/481830179 >> [v2:172.21.15.22:6800/1357365082,v1:172.21.15.22:6801/1357365082] conn(0x7f37e40a2970 0x7f37e40a4e30 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f37f800cc20 tx=0x7f37f8005ec0 comp rx=0 tx=0).stop 2024-03-28T18:10:10.423 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:10.419+0000 7f3805800640 1 -- 172.21.15.22:0/481830179 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3800101530 msgr2=0x7f380019a0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:10:10.423 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:10.419+0000 7f3805800640 1 --2- 172.21.15.22:0/481830179 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3800101530 0x7f380019a0a0 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7f37f000ea80 tx=0x7f37f000ef40 comp rx=0 tx=0).stop 2024-03-28T18:10:10.423 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:10.419+0000 7f3805800640 1 -- 172.21.15.22:0/481830179 shutdown_connections 2024-03-28T18:10:10.423 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:10.419+0000 7f3805800640 1 --2- 172.21.15.22:0/481830179 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f38001075f0 0x7f380019e960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:10:10.423 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:10.419+0000 7f3805800640 1 --2- 172.21.15.22:0/481830179 >> [v2:172.21.15.22:6800/1357365082,v1:172.21.15.22:6801/1357365082] conn(0x7f37e40a2970 0x7f37e40a4e30 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:10:10.423 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:10.419+0000 7f3805800640 1 --2- 172.21.15.22:0/481830179 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f3800101ef0 0x7f380019a5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:10:10.423 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:10.419+0000 7f3805800640 1 --2- 172.21.15.22:0/481830179 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3800101530 0x7f380019a0a0 unknown :-1 s=CLOSED pgs=636 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:10:10.423 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:10.419+0000 7f3805800640 1 -- 172.21.15.22:0/481830179 >> 172.21.15.22:0/481830179 conn(0x7f38000fd040 msgr2=0x7f38000fe660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:10:10.424 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:10.419+0000 7f3805800640 1 -- 172.21.15.22:0/481830179 shutdown_connections 2024-03-28T18:10:10.424 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:10.419+0000 7f3805800640 1 -- 172.21.15.22:0/481830179 wait complete. 2024-03-28T18:10:10.424 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/preempt-locking.txt 2024-03-28T18:10:10.425 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:10:10 smithi134 ceph-mon[35037]: pgmap v81: 97 pgs: 97 active+clean; 105 MiB data, 795 MiB used, 1.0 TiB / 1.0 TiB avail; 316 KiB/s rd, 645 KiB/s wr, 123 op/s 2024-03-28T18:10:10.425 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:10:10 smithi134 ceph-mon[35037]: from='client.? 172.21.15.22:0/1169249648' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-28T18:10:10.433 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:10:10 smithi079 ceph-mon[35312]: pgmap v81: 97 pgs: 97 active+clean; 105 MiB data, 795 MiB used, 1.0 TiB / 1.0 TiB avail; 316 KiB/s rd, 645 KiB/s wr, 123 op/s 2024-03-28T18:10:10.433 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:10:10 smithi079 ceph-mon[35312]: from='client.? 172.21.15.22:0/1169249648' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-28T18:10:10.434 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:10:10 smithi022 ceph-mon[31985]: pgmap v81: 97 pgs: 97 active+clean; 105 MiB data, 795 MiB used, 1.0 TiB / 1.0 TiB avail; 316 KiB/s rd, 645 KiB/s wr, 123 op/s 2024-03-28T18:10:10.435 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:10:10 smithi022 ceph-mon[31985]: from='client.? 172.21.15.22:0/1169249648' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-28T18:10:10.437 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-28T18:10:10.437 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-03-28T18:10:10.438 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/printk-formats.txt 2024-03-28T18:10:10.449 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/pti/ 2024-03-28T18:10:10.459 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/pti/pti_intel_mid.txt 2024-03-28T18:10:10.466 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/ptp/ 2024-03-28T18:10:10.479 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/ptp/.gitignore 2024-03-28T18:10:10.483 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/ptp/Makefile 2024-03-28T18:10:10.495 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/ptp/ptp.txt 2024-03-28T18:10:10.507 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/ptp/testptp.c 2024-03-28T18:10:10.520 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/ptp/testptp.mk 2024-03-28T18:10:10.536 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/pwm.txt 2024-03-28T18:10:10.609 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/ramoops.txt 2024-03-28T18:10:10.626 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/rapidio/ 2024-03-28T18:10:10.636 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/rapidio/rapidio.txt 2024-03-28T18:10:10.646 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/rapidio/sysfs.txt 2024-03-28T18:10:10.661 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/rapidio/tsi721.txt 2024-03-28T18:10:10.672 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/rbtree.txt 2024-03-28T18:10:10.678 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:10.678+0000 7fae4ce00640 1 -- 172.21.15.22:0/2344599844 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fae481002e0 msgr2=0x7fae481006c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:10:10.678 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:10.678+0000 7fae4ce00640 1 --2- 172.21.15.22:0/2344599844 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fae481002e0 0x7fae481006c0 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7fae40009900 tx=0x7fae4002bc40 comp rx=0 tx=0).stop 2024-03-28T18:10:10.678 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:10.678+0000 7fae4ce00640 1 -- 172.21.15.22:0/2344599844 shutdown_connections 2024-03-28T18:10:10.678 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:10.678+0000 7fae4ce00640 1 --2- 172.21.15.22:0/2344599844 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fae4810de90 0x7fae48110280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:10:10.678 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:10.678+0000 7fae4ce00640 1 --2- 172.21.15.22:0/2344599844 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fae48100c90 0x7fae4810d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:10:10.678 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:10.678+0000 7fae4ce00640 1 --2- 172.21.15.22:0/2344599844 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fae481002e0 0x7fae481006c0 unknown :-1 s=CLOSED pgs=639 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:10:10.678 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:10.678+0000 7fae4ce00640 1 -- 172.21.15.22:0/2344599844 >> 172.21.15.22:0/2344599844 conn(0x7fae480fbf00 msgr2=0x7fae480fe320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:10:10.679 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:10.678+0000 7fae4ce00640 1 -- 172.21.15.22:0/2344599844 shutdown_connections 2024-03-28T18:10:10.679 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:10.678+0000 7fae4ce00640 1 -- 172.21.15.22:0/2344599844 wait complete. 2024-03-28T18:10:10.680 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:10.679+0000 7fae4ce00640 1 Processor -- start 2024-03-28T18:10:10.680 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:10.680+0000 7fae4ce00640 1 -- start start 2024-03-28T18:10:10.681 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:10.680+0000 7fae4ce00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fae481002e0 0x7fae4819a140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:10:10.681 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:10.680+0000 7fae4ce00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fae48100c90 0x7fae4819a680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:10:10.681 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:10.680+0000 7fae4ce00640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fae4810de90 0x7fae4819e9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:10:10.681 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:10.680+0000 7fae4ce00640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae48113b70 con 0x7fae481002e0 2024-03-28T18:10:10.681 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:10.680+0000 7fae4ce00640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae481139f0 con 0x7fae4810de90 2024-03-28T18:10:10.681 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:10.680+0000 7fae4ce00640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae48113cf0 con 0x7fae48100c90 2024-03-28T18:10:10.681 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:10.680+0000 7fae47400640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fae481002e0 0x7fae4819a140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:10:10.681 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:10.680+0000 7fae47400640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fae481002e0 0x7fae4819a140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:45990/0 (socket says 172.21.15.22:45990) 2024-03-28T18:10:10.681 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:10.680+0000 7fae47400640 1 -- 172.21.15.22:0/2531667224 learned_addr learned my addr 172.21.15.22:0/2531667224 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-28T18:10:10.681 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:10.680+0000 7fae47e00640 1 --2- 172.21.15.22:0/2531667224 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fae4810de90 0x7fae4819e9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:10:10.681 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:10.680+0000 7fae46a00640 1 --2- 172.21.15.22:0/2531667224 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fae48100c90 0x7fae4819a680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:10:10.681 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:10.681+0000 7fae47400640 1 -- 172.21.15.22:0/2531667224 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fae48100c90 msgr2=0x7fae4819a680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:10:10.681 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:10.681+0000 7fae47400640 1 --2- 172.21.15.22:0/2531667224 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fae48100c90 0x7fae4819a680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:10:10.681 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:10.681+0000 7fae47400640 1 -- 172.21.15.22:0/2531667224 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fae4810de90 msgr2=0x7fae4819e9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:10:10.682 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:10.681+0000 7fae47400640 1 --2- 172.21.15.22:0/2531667224 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fae4810de90 0x7fae4819e9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:10:10.682 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:10.681+0000 7fae47400640 1 -- 172.21.15.22:0/2531667224 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae4819f0e0 con 0x7fae481002e0 2024-03-28T18:10:10.682 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:10.681+0000 7fae47e00640 1 --2- 172.21.15.22:0/2531667224 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fae4810de90 0x7fae4819e9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-28T18:10:10.682 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:10.681+0000 7fae47400640 1 --2- 172.21.15.22:0/2531667224 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fae481002e0 0x7fae4819a140 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7fae40000c00 tx=0x7fae400045e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:10:10.683 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:10.682+0000 7fae3fe00640 1 -- 172.21.15.22:0/2531667224 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae40043020 con 0x7fae481002e0 2024-03-28T18:10:10.683 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:10.682+0000 7fae3fe00640 1 -- 172.21.15.22:0/2531667224 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7fae4002cab0 con 0x7fae481002e0 2024-03-28T18:10:10.683 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:10.682+0000 7fae3fe00640 1 -- 172.21.15.22:0/2531667224 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae40003a10 con 0x7fae481002e0 2024-03-28T18:10:10.683 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:10.682+0000 7fae4ce00640 1 -- 172.21.15.22:0/2531667224 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fae4819f370 con 0x7fae481002e0 2024-03-28T18:10:10.683 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:10.682+0000 7fae4ce00640 1 -- 172.21.15.22:0/2531667224 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fae481a6c60 con 0x7fae481002e0 2024-03-28T18:10:10.685 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:10.685+0000 7fae511d6740 1 Processor -- start 2024-03-28T18:10:10.685 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:10.686+0000 7fae511d6740 1 -- start start 2024-03-28T18:10:10.685 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/remoteproc.txt 2024-03-28T18:10:10.686 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:10.686+0000 7fae3fe00640 1 -- 172.21.15.22:0/2531667224 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 26) v1 ==== 137421+0+0 (secure 0 0 0) 0x7fae40046020 con 0x7fae481002e0 2024-03-28T18:10:10.686 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:10.686+0000 7fae511d6740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55caf5ab48b0 0x55caf5ab4c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:10:10.686 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:10.686+0000 7fae511d6740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55caf5ab5260 0x55caf5abd810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:10:10.686 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:10.686+0000 7fae511d6740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55caf5abdde0 0x55caf5ac0230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:10:10.686 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:10.686+0000 7fae511d6740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x55caf5a182c0 con 0x55caf5abdde0 2024-03-28T18:10:10.686 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:10.686+0000 7fae511d6740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x55caf59c0c90 con 0x55caf5ab48b0 2024-03-28T18:10:10.686 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:10.686+0000 7fae511d6740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55caf5a14360 con 0x55caf5ab5260 2024-03-28T18:10:10.688 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:10.686+0000 7fae47e00640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55caf5ab48b0 0x55caf5ab4c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:10:10.688 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:10.687+0000 7fae47e00640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55caf5ab48b0 0x55caf5ab4c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.22:52704/0 (socket says 172.21.15.22:52704) 2024-03-28T18:10:10.688 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:10.687+0000 7fae47e00640 1 -- 172.21.15.22:0/1470048494 learned_addr learned my addr 172.21.15.22:0/1470048494 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-28T18:10:10.688 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:10.687+0000 7fae46a00640 1 --2- 172.21.15.22:0/1470048494 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55caf5abdde0 0x55caf5ac0230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:10:10.688 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:10.687+0000 7fae47e00640 1 -- 172.21.15.22:0/1470048494 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55caf5ab5260 msgr2=0x55caf5abd810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T18:10:10.688 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:10.687+0000 7fae47e00640 1 --2- 172.21.15.22:0/1470048494 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55caf5ab5260 0x55caf5abd810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:10:10.688 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:10.687+0000 7fae47e00640 1 -- 172.21.15.22:0/1470048494 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55caf5abdde0 msgr2=0x55caf5ac0230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T18:10:10.688 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:10.687+0000 7fae47e00640 1 --2- 172.21.15.22:0/1470048494 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55caf5abdde0 0x55caf5ac0230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:10:10.688 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:10.687+0000 7fae47e00640 1 -- 172.21.15.22:0/1470048494 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55caf59e5080 con 0x55caf5ab48b0 2024-03-28T18:10:10.688 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:10.687+0000 7fae47e00640 1 --2- 172.21.15.22:0/1470048494 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55caf5ab48b0 0x55caf5ab4c90 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fae38011790 tx=0x7fae38011c50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=68944bb21d9baaf8 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:10:10.688 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:10.688+0000 7fae3cc00640 1 -- 172.21.15.22:0/1470048494 <== mon.2 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae38003c30 con 0x55caf5ab48b0 2024-03-28T18:10:10.688 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:10.688+0000 7fae3fe00640 1 --2- 172.21.15.22:0/2531667224 >> [v2:172.21.15.22:6800/1357365082,v1:172.21.15.22:6801/1357365082] conn(0x7fae2c0a29f0 0x7fae2c0a4eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:10:10.689 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:10.688+0000 7fae3fe00640 1 -- 172.21.15.22:0/2531667224 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 8411+0+0 (secure 0 0 0) 0x7fae400f0d20 con 0x7fae481002e0 2024-03-28T18:10:10.689 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:10.688+0000 7fae3cc00640 1 -- 172.21.15.22:0/1470048494 <== mon.2 v2:172.21.15.79:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7fae38003dd0 con 0x55caf5ab48b0 2024-03-28T18:10:10.689 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:10.688+0000 7fae3cc00640 1 -- 172.21.15.22:0/1470048494 <== mon.2 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae380193e0 con 0x55caf5ab48b0 2024-03-28T18:10:10.689 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:10.688+0000 7fae47400640 1 --2- 172.21.15.22:0/2531667224 >> [v2:172.21.15.22:6800/1357365082,v1:172.21.15.22:6801/1357365082] conn(0x7fae2c0a29f0 0x7fae2c0a4eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:10:10.691 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:10.689+0000 7fae511d6740 1 -- 172.21.15.22:0/1470048494 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55caf5ab48b0 msgr2=0x55caf5ab4c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:10:10.691 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:10.689+0000 7fae47400640 1 --2- 172.21.15.22:0/2531667224 >> [v2:172.21.15.22:6800/1357365082,v1:172.21.15.22:6801/1357365082] conn(0x7fae2c0a29f0 0x7fae2c0a4eb0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x55caf5ab4380 tx=0x7fae400dd040 comp rx=0 tx=0).ready entity=mgr.15522 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:10:10.691 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:10.689+0000 7fae511d6740 1 --2- 172.21.15.22:0/1470048494 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55caf5ab48b0 0x55caf5ab4c90 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fae38011790 tx=0x7fae38011c50 comp rx=0 tx=0).stop 2024-03-28T18:10:10.691 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:10.689+0000 7fae511d6740 1 -- 172.21.15.22:0/1470048494 shutdown_connections 2024-03-28T18:10:10.691 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:10.689+0000 7fae511d6740 1 --2- 172.21.15.22:0/1470048494 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55caf5abdde0 0x55caf5ac0230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:10:10.691 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:10.689+0000 7fae511d6740 1 --2- 172.21.15.22:0/1470048494 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55caf5ab5260 0x55caf5abd810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:10:10.691 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:10.689+0000 7fae511d6740 1 --2- 172.21.15.22:0/1470048494 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55caf5ab48b0 0x55caf5ab4c90 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:10:10.691 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:10.689+0000 7fae511d6740 1 -- 172.21.15.22:0/1470048494 >> 172.21.15.22:0/1470048494 conn(0x55caf5aaef80 msgr2=0x55caf5aaf360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:10:10.691 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:10.689+0000 7fae511d6740 1 -- 172.21.15.22:0/1470048494 shutdown_connections 2024-03-28T18:10:10.691 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:10.689+0000 7fae511d6740 1 -- 172.21.15.22:0/1470048494 wait complete. 2024-03-28T18:10:10.692 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:10.690+0000 7fae511d6740 1 -- start start 2024-03-28T18:10:10.692 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:10.690+0000 7fae511d6740 1 Processor -- start 2024-03-28T18:10:10.692 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:10.691+0000 7fae511d6740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55caf5ab48b0 0x55caf5abd660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:10:10.692 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:10.691+0000 7fae511d6740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55caf5ab5260 0x55caf5aba9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:10:10.692 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:10.691+0000 7fae511d6740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55caf5abdde0 0x55caf5abaf30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:10:10.692 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:10.691+0000 7fae511d6740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x55caf5ac0d50 con 0x55caf5ab48b0 2024-03-28T18:10:10.692 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:10.691+0000 7fae511d6740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x55caf59d40d0 con 0x55caf5abdde0 2024-03-28T18:10:10.692 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:10.691+0000 7fae511d6740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55caf5ac0ed0 con 0x55caf5ab5260 2024-03-28T18:10:10.692 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:10.691+0000 7fae511d6740 10 client.0 _ll_register_callbacks cb 0x55caf59d8e80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-28T18:10:10.692 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:10.691+0000 7fae47e00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55caf5ab48b0 0x55caf5abd660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:10:10.692 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:10.691+0000 7fae47e00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55caf5ab48b0 0x55caf5abd660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:46020/0 (socket says 172.21.15.22:46020) 2024-03-28T18:10:10.692 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:10.691+0000 7fae47e00640 1 -- 172.21.15.22:0/3569023584 learned_addr learned my addr 172.21.15.22:0/3569023584 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-28T18:10:10.692 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:10.691+0000 7fae46a00640 1 --2- 172.21.15.22:0/3569023584 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55caf5ab5260 0x55caf5aba9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:10:10.692 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:10.691+0000 7fae47e00640 1 --2- 172.21.15.22:0/3569023584 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55caf5abdde0 0x55caf5abaf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:10:10.695 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:10.692+0000 7fae47e00640 1 -- 172.21.15.22:0/3569023584 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55caf5ab5260 msgr2=0x55caf5aba9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T18:10:10.695 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:10.692+0000 7fae47e00640 1 --2- 172.21.15.22:0/3569023584 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55caf5ab5260 0x55caf5aba9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:10:10.695 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:10.692+0000 7fae47e00640 1 -- 172.21.15.22:0/3569023584 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55caf5abdde0 msgr2=0x55caf5abaf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T18:10:10.695 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:10.692+0000 7fae47e00640 1 --2- 172.21.15.22:0/3569023584 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55caf5abdde0 0x55caf5abaf30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:10:10.695 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:10.692+0000 7fae47e00640 1 -- 172.21.15.22:0/3569023584 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55caf572c3f0 con 0x55caf5ab48b0 2024-03-28T18:10:10.695 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:10.692+0000 7fae47e00640 1 --2- 172.21.15.22:0/3569023584 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55caf5ab48b0 0x55caf5abd660 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7fae38004440 tx=0x7fae38004540 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9f15c6bb20ffd36d server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:10:10.695 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:10.692+0000 7fae35600640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-03-28T18:10:10.695 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:10.693+0000 7fae35600640 1 -- 172.21.15.22:0/3569023584 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae3802f060 con 0x55caf5ab48b0 2024-03-28T18:10:10.695 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:10.693+0000 7fae35600640 1 -- 172.21.15.22:0/3569023584 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 8411+0+0 (secure 0 0 0) 0x7fae3800fb20 con 0x55caf5ab48b0 2024-03-28T18:10:10.695 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:10.693+0000 7fae511d6740 1 -- 172.21.15.22:0/3569023584 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fae38003dd0 con 0x55caf5ab48b0 2024-03-28T18:10:10.695 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:10.693+0000 7fae35600640 1 -- 172.21.15.22:0/3569023584 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) v2 ==== 24+0+0 (secure 0 0 0) 0x7fae3801f040 con 0x55caf5ab48b0 2024-03-28T18:10:10.695 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:10.693+0000 7fae511d6740 10 client.16281 fetch_fsmap learned FSMap version 31 2024-03-28T18:10:10.695 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:10.693+0000 7fae511d6740 1 -- 172.21.15.22:0/3569023584 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) v3 -- 0x55caf57f67d0 con 0x55caf5ab48b0 2024-03-28T18:10:10.696 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:10.693+0000 7fae35600640 1 -- 172.21.15.22:0/3569023584 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7fae38003dd0 con 0x55caf5ab48b0 2024-03-28T18:10:10.696 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:10.693+0000 7fae35600640 1 -- 172.21.15.22:0/3569023584 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 31) v1 ==== 7950+0+0 (secure 0 0 0) 0x7fae38032070 con 0x55caf5ab48b0 2024-03-28T18:10:10.696 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:10.694+0000 7fae511d6740 10 client.16281 fetch_fsmap finished waiting for FSMap version 31 2024-03-28T18:10:10.696 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:10.694+0000 7fae511d6740 10 client.16281 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.e 2024-03-28T18:10:10.696 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:10.694+0000 7fae511d6740 20 client.16281 populate_metadata read hostname 'smithi022' 2024-03-28T18:10:10.696 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:10.694+0000 7fae511d6740 1 --2- 172.21.15.22:0/3569023584 >> [v2:172.21.15.79:6834/3365773431,v1:172.21.15.79:6835/3365773431] conn(0x55caf5acdac0 0x55caf5acfeb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:10:10.696 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:10.694+0000 7fae511d6740 4 client.16281 mds_command: new command op to 34475 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-28T18:10:10.696 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:10.694+0000 7fae511d6740 1 -- 172.21.15.22:0/3569023584 --> [v2:172.21.15.79:6834/3365773431,v1:172.21.15.79:6835/3365773431] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55caf5a25800 con 0x55caf5acdac0 2024-03-28T18:10:10.696 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:10.694+0000 7fae46a00640 1 --2- 172.21.15.22:0/3569023584 >> [v2:172.21.15.79:6834/3365773431,v1:172.21.15.79:6835/3365773431] conn(0x55caf5acdac0 0x55caf5acfeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:10:10.696 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:10.695+0000 7fae46a00640 1 --2- 172.21.15.22:0/3569023584 >> [v2:172.21.15.79:6834/3365773431,v1:172.21.15.79:6835/3365773431] conn(0x55caf5acdac0 0x55caf5acfeb0 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=ceb591a02a045cc5 server_cookie=32ac9dcf7aa783c2 in_seq=0 out_seq=0 2024-03-28T18:10:10.696 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:10.695+0000 7fae35600640 10 client.16281 ms_handle_connect on v2:172.21.15.79:6834/3365773431 2024-03-28T18:10:10.697 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:10.696+0000 7fae35600640 1 -- 172.21.15.22:0/3569023584 <== mds.2 v2:172.21.15.79:6834/3365773431 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x55caf5a25800 con 0x55caf5acdac0 2024-03-28T18:10:10.697 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:10.696+0000 7fae35600640 10 client.16281 handle_command_reply: tid=0 2024-03-28T18:10:10.697 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:10.696+0000 7fae511d6740 1 client.16281 shutdown 2024-03-28T18:10:10.698 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:10.696+0000 7fae511d6740 1 -- 172.21.15.22:0/3569023584 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55caf5ab48b0 msgr2=0x55caf5abd660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:10:10.698 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:10.696+0000 7fae511d6740 1 --2- 172.21.15.22:0/3569023584 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55caf5ab48b0 0x55caf5abd660 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7fae38004440 tx=0x7fae38004540 comp rx=0 tx=0).stop 2024-03-28T18:10:10.698 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:10.696+0000 7fae511d6740 1 -- 172.21.15.22:0/3569023584 shutdown_connections 2024-03-28T18:10:10.698 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:10.696+0000 7fae511d6740 1 --2- 172.21.15.22:0/3569023584 >> [v2:172.21.15.79:6834/3365773431,v1:172.21.15.79:6835/3365773431] conn(0x55caf5acdac0 0x55caf5acfeb0 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:10:10.698 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:10.696+0000 7fae511d6740 1 --2- 172.21.15.22:0/3569023584 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55caf5abdde0 0x55caf5abaf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:10:10.698 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:10.696+0000 7fae511d6740 1 --2- 172.21.15.22:0/3569023584 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55caf5ab5260 0x55caf5aba9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:10:10.698 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:10.696+0000 7fae511d6740 1 --2- 172.21.15.22:0/3569023584 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55caf5ab48b0 0x55caf5abd660 unknown :-1 s=CLOSED pgs=641 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:10:10.698 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:10.696+0000 7fae511d6740 1 -- 172.21.15.22:0/3569023584 >> 172.21.15.22:0/3569023584 conn(0x55caf5aaef80 msgr2=0x55caf5ab0cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:10:10.698 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:10.696+0000 7fae35600640 0 client.16281 ms_handle_reset on v2:172.21.15.79:6834/3365773431 2024-03-28T18:10:10.698 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:10.696+0000 7fae511d6740 1 -- 172.21.15.22:0/3569023584 shutdown_connections 2024-03-28T18:10:10.698 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:10.696+0000 7fae511d6740 1 -- 172.21.15.22:0/3569023584 wait complete. 2024-03-28T18:10:10.698 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:10.697+0000 7fae511d6740 20 client.16281 trim_cache size 0 max 16384 2024-03-28T18:10:10.698 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/rfkill.txt 2024-03-28T18:10:10.706 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/robust-futex-ABI.txt 2024-03-28T18:10:10.713 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:10.714+0000 7fae511d6740 1 Processor -- start 2024-03-28T18:10:10.714 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:10.714+0000 7fae511d6740 1 -- start start 2024-03-28T18:10:10.714 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:10.714+0000 7fae511d6740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55caf5ab48b0 0x55caf5abd120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:10:10.714 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:10.714+0000 7fae511d6740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55caf5ab5260 0x55caf5abd660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:10:10.714 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:10.714+0000 7fae511d6740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55caf5abdde0 0x55caf5ab7cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:10:10.714 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:10.714+0000 7fae511d6740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae481af030 con 0x55caf5ab48b0 2024-03-28T18:10:10.714 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:10.714+0000 7fae511d6740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae481b0a70 con 0x55caf5abdde0 2024-03-28T18:10:10.714 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:10.714+0000 7fae511d6740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae481af690 con 0x55caf5ab5260 2024-03-28T18:10:10.715 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:10.714+0000 7fae47e00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55caf5ab48b0 0x55caf5abd120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:10:10.715 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:10.715+0000 7fae47e00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55caf5ab48b0 0x55caf5abd120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:46036/0 (socket says 172.21.15.22:46036) 2024-03-28T18:10:10.715 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:10.715+0000 7fae47e00640 1 -- 172.21.15.22:0/1237724334 learned_addr learned my addr 172.21.15.22:0/1237724334 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-28T18:10:10.715 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:10.715+0000 7fae46a00640 1 --2- 172.21.15.22:0/1237724334 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55caf5ab5260 0x55caf5abd660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:10:10.716 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:10.715+0000 7fae47e00640 1 --2- 172.21.15.22:0/1237724334 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55caf5abdde0 0x55caf5ab7cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:10:10.716 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:10.715+0000 7fae47e00640 1 -- 172.21.15.22:0/1237724334 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55caf5ab5260 msgr2=0x55caf5abd660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T18:10:10.716 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:10.715+0000 7fae47e00640 1 --2- 172.21.15.22:0/1237724334 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55caf5ab5260 0x55caf5abd660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:10:10.716 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:10.715+0000 7fae47e00640 1 -- 172.21.15.22:0/1237724334 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55caf5abdde0 msgr2=0x55caf5ab7cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T18:10:10.716 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:10.715+0000 7fae47e00640 1 --2- 172.21.15.22:0/1237724334 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55caf5abdde0 0x55caf5ab7cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:10:10.716 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:10.715+0000 7fae47e00640 1 -- 172.21.15.22:0/1237724334 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55caf5ad0310 con 0x55caf5ab48b0 2024-03-28T18:10:10.716 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/robust-futexes.txt 2024-03-28T18:10:10.717 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:10.716+0000 7fae47e00640 1 --2- 172.21.15.22:0/1237724334 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55caf5ab48b0 0x55caf5abd120 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7fae38006010 tx=0x7fae380021d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=835bd4ddf67b7475 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:10:10.717 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:10.716+0000 7fae37e00640 1 -- 172.21.15.22:0/1237724334 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae3802f070 con 0x55caf5ab48b0 2024-03-28T18:10:10.717 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:10.716+0000 7fae37e00640 1 -- 172.21.15.22:0/1237724334 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7fae3803b040 con 0x55caf5ab48b0 2024-03-28T18:10:10.717 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:10.716+0000 7fae37e00640 1 -- 172.21.15.22:0/1237724334 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae3800f920 con 0x55caf5ab48b0 2024-03-28T18:10:10.717 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:10.717+0000 7fae511d6740 1 -- 172.21.15.22:0/1237724334 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55caf5ab48b0 msgr2=0x55caf5abd120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:10:10.717 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:10.717+0000 7fae511d6740 1 --2- 172.21.15.22:0/1237724334 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55caf5ab48b0 0x55caf5abd120 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7fae38006010 tx=0x7fae380021d0 comp rx=0 tx=0).stop 2024-03-28T18:10:10.718 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:10.717+0000 7fae511d6740 1 -- 172.21.15.22:0/1237724334 shutdown_connections 2024-03-28T18:10:10.718 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:10.717+0000 7fae511d6740 1 --2- 172.21.15.22:0/1237724334 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55caf5abdde0 0x55caf5ab7cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:10:10.718 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:10.717+0000 7fae511d6740 1 --2- 172.21.15.22:0/1237724334 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55caf5ab5260 0x55caf5abd660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:10:10.718 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:10.717+0000 7fae511d6740 1 --2- 172.21.15.22:0/1237724334 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55caf5ab48b0 0x55caf5abd120 unknown :-1 s=CLOSED pgs=642 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:10:10.718 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:10.717+0000 7fae511d6740 1 -- 172.21.15.22:0/1237724334 >> 172.21.15.22:0/1237724334 conn(0x55caf5aaef80 msgr2=0x55caf5aae160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:10:10.718 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:10.717+0000 7fae511d6740 1 -- 172.21.15.22:0/1237724334 shutdown_connections 2024-03-28T18:10:10.718 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:10.717+0000 7fae511d6740 1 -- 172.21.15.22:0/1237724334 wait complete. 2024-03-28T18:10:10.718 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:10.718+0000 7fae511d6740 1 -- start start 2024-03-28T18:10:10.721 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:10.718+0000 7fae511d6740 1 Processor -- start 2024-03-28T18:10:10.721 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:10.719+0000 7fae511d6740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55caf5ab48b0 0x55caf5ac1870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:10:10.721 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:10.719+0000 7fae511d6740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55caf5ab5260 0x55caf5ac1db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:10:10.721 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:10.719+0000 7fae511d6740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55caf5abdde0 0x55caf5b28160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:10:10.721 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:10.719+0000 7fae511d6740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae481b0350 con 0x55caf5ab5260 2024-03-28T18:10:10.721 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:10.719+0000 7fae511d6740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae481afcf0 con 0x55caf5ab48b0 2024-03-28T18:10:10.721 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:10.719+0000 7fae511d6740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae481ad210 con 0x55caf5abdde0 2024-03-28T18:10:10.721 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:10.719+0000 7fae511d6740 10 client.0 _ll_register_callbacks cb 0x7fae30004120 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-28T18:10:10.721 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:10.719+0000 7fae46a00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55caf5ab5260 0x55caf5ac1db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:10:10.721 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:10.719+0000 7fae47e00640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55caf5ab48b0 0x55caf5ac1870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:10:10.722 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:10.719+0000 7fae46a00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55caf5ab5260 0x55caf5ac1db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:46050/0 (socket says 172.21.15.22:46050) 2024-03-28T18:10:10.722 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:10.719+0000 7fae46a00640 1 -- 172.21.15.22:0/1664997026 learned_addr learned my addr 172.21.15.22:0/1664997026 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-28T18:10:10.722 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:10.719+0000 7fae47e00640 1 --2- 172.21.15.22:0/1664997026 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55caf5abdde0 0x55caf5b28160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:10:10.722 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:10.719+0000 7fae46a00640 1 -- 172.21.15.22:0/1664997026 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55caf5abdde0 msgr2=0x55caf5b28160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T18:10:10.722 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:10.719+0000 7fae46a00640 1 --2- 172.21.15.22:0/1664997026 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55caf5abdde0 0x55caf5b28160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:10:10.722 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:10.719+0000 7fae46a00640 1 -- 172.21.15.22:0/1664997026 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55caf5ab48b0 msgr2=0x55caf5ac1870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T18:10:10.722 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:10.719+0000 7fae46a00640 1 --2- 172.21.15.22:0/1664997026 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55caf5ab48b0 0x55caf5ac1870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:10:10.722 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:10.719+0000 7fae46a00640 1 -- 172.21.15.22:0/1664997026 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55caf5ac2680 con 0x55caf5ab5260 2024-03-28T18:10:10.722 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:10.719+0000 7fae47e00640 1 --2- 172.21.15.22:0/1664997026 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55caf5ab48b0 0x55caf5ac1870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-28T18:10:10.722 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:10.720+0000 7fae46a00640 1 --2- 172.21.15.22:0/1664997026 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55caf5ab5260 0x55caf5ac1db0 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7fae30004650 tx=0x7fae30020bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=32bc715a987acb00 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:10:10.722 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:10.720+0000 7fae35600640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-03-28T18:10:10.722 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:10.720+0000 7fae35600640 1 -- 172.21.15.22:0/1664997026 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae300202e0 con 0x55caf5ab5260 2024-03-28T18:10:10.722 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:10.720+0000 7fae511d6740 1 -- 172.21.15.22:0/1664997026 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fae3803b040 con 0x55caf5ab5260 2024-03-28T18:10:10.722 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:10.720+0000 7fae35600640 1 -- 172.21.15.22:0/1664997026 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 8411+0+0 (secure 0 0 0) 0x7fae30003b20 con 0x55caf5ab5260 2024-03-28T18:10:10.723 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:10.721+0000 7fae35600640 1 -- 172.21.15.22:0/1664997026 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) v2 ==== 24+0+0 (secure 0 0 0) 0x7fae3002f050 con 0x55caf5ab5260 2024-03-28T18:10:10.723 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:10.721+0000 7fae511d6740 10 client.16287 fetch_fsmap learned FSMap version 31 2024-03-28T18:10:10.723 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:10.721+0000 7fae511d6740 1 -- 172.21.15.22:0/1664997026 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) v3 -- 0x55caf5b29830 con 0x55caf5ab5260 2024-03-28T18:10:10.723 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:10.721+0000 7fae35600640 1 -- 172.21.15.22:0/1664997026 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7fae3803b040 con 0x55caf5ab5260 2024-03-28T18:10:10.723 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:10.721+0000 7fae35600640 1 -- 172.21.15.22:0/1664997026 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 31) v1 ==== 7950+0+0 (secure 0 0 0) 0x7fae30005020 con 0x55caf5ab5260 2024-03-28T18:10:10.723 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:10.721+0000 7fae511d6740 10 client.16287 fetch_fsmap finished waiting for FSMap version 31 2024-03-28T18:10:10.723 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:10.721+0000 7fae511d6740 10 client.16287 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.e 2024-03-28T18:10:10.723 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:10.721+0000 7fae511d6740 20 client.16287 populate_metadata read hostname 'smithi022' 2024-03-28T18:10:10.723 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:10.721+0000 7fae511d6740 1 --2- 172.21.15.22:0/1664997026 >> [v2:172.21.15.79:6834/3365773431,v1:172.21.15.79:6835/3365773431] conn(0x55caf5b29f90 0x55caf5b2c3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:10:10.723 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:10.721+0000 7fae511d6740 4 client.16287 mds_command: new command op to 34475 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-28T18:10:10.723 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:10.721+0000 7fae511d6740 1 -- 172.21.15.22:0/1664997026 --> [v2:172.21.15.79:6834/3365773431,v1:172.21.15.79:6835/3365773431] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55caf5a1fb50 con 0x55caf5b29f90 2024-03-28T18:10:10.725 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:10.722+0000 7fae47e00640 1 --2- 172.21.15.22:0/1664997026 >> [v2:172.21.15.79:6834/3365773431,v1:172.21.15.79:6835/3365773431] conn(0x55caf5b29f90 0x55caf5b2c3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:10:10.725 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:10.722+0000 7fae47e00640 1 --2- 172.21.15.22:0/1664997026 >> [v2:172.21.15.79:6834/3365773431,v1:172.21.15.79:6835/3365773431] conn(0x55caf5b29f90 0x55caf5b2c3b0 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=171276909e029096 server_cookie=14af894fce01f7ff in_seq=0 out_seq=0 2024-03-28T18:10:10.725 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:10.722+0000 7fae35600640 10 client.16287 ms_handle_connect on v2:172.21.15.79:6834/3365773431 2024-03-28T18:10:10.725 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:10.723+0000 7fae35600640 1 -- 172.21.15.22:0/1664997026 <== mds.2 v2:172.21.15.79:6834/3365773431 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55caf5a1fb50 con 0x55caf5b29f90 2024-03-28T18:10:10.725 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:10.723+0000 7fae35600640 10 client.16287 handle_command_reply: tid=0 2024-03-28T18:10:10.725 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:10.723+0000 7fae511d6740 1 client.16287 shutdown 2024-03-28T18:10:10.725 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:10.723+0000 7fae511d6740 1 -- 172.21.15.22:0/1664997026 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55caf5ab5260 msgr2=0x55caf5ac1db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:10:10.725 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:10.723+0000 7fae511d6740 1 --2- 172.21.15.22:0/1664997026 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55caf5ab5260 0x55caf5ac1db0 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7fae30004650 tx=0x7fae30020bb0 comp rx=0 tx=0).stop 2024-03-28T18:10:10.725 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:10.723+0000 7fae511d6740 1 -- 172.21.15.22:0/1664997026 shutdown_connections 2024-03-28T18:10:10.725 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:10.723+0000 7fae511d6740 1 --2- 172.21.15.22:0/1664997026 >> [v2:172.21.15.79:6834/3365773431,v1:172.21.15.79:6835/3365773431] conn(0x55caf5b29f90 0x55caf5b2c3b0 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:10:10.725 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:10.723+0000 7fae511d6740 1 --2- 172.21.15.22:0/1664997026 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55caf5abdde0 0x55caf5b28160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:10:10.725 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:10.723+0000 7fae511d6740 1 --2- 172.21.15.22:0/1664997026 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55caf5ab5260 0x55caf5ac1db0 unknown :-1 s=CLOSED pgs=643 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:10:10.725 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:10.723+0000 7fae511d6740 1 --2- 172.21.15.22:0/1664997026 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55caf5ab48b0 0x55caf5ac1870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:10:10.725 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:10.723+0000 7fae35600640 0 client.16287 ms_handle_reset on v2:172.21.15.79:6834/3365773431 2024-03-28T18:10:10.725 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:10.723+0000 7fae511d6740 1 -- 172.21.15.22:0/1664997026 >> 172.21.15.22:0/1664997026 conn(0x55caf5aaef80 msgr2=0x55caf5ab5640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:10:10.726 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:10.723+0000 7fae511d6740 1 -- 172.21.15.22:0/1664997026 shutdown_connections 2024-03-28T18:10:10.726 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:10.723+0000 7fae511d6740 1 -- 172.21.15.22:0/1664997026 wait complete. 2024-03-28T18:10:10.726 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:10.723+0000 7fae511d6740 20 client.16287 trim_cache size 0 max 16384 2024-03-28T18:10:10.726 INFO:teuthology.orchestra.run.smithi022.stdout:[] 2024-03-28T18:10:10.727 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:10.724+0000 7fae4ce00640 1 -- 172.21.15.22:0/2531667224 >> [v2:172.21.15.22:6800/1357365082,v1:172.21.15.22:6801/1357365082] conn(0x7fae2c0a29f0 msgr2=0x7fae2c0a4eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:10:10.727 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:10.724+0000 7fae4ce00640 1 --2- 172.21.15.22:0/2531667224 >> [v2:172.21.15.22:6800/1357365082,v1:172.21.15.22:6801/1357365082] conn(0x7fae2c0a29f0 0x7fae2c0a4eb0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x55caf5ab4380 tx=0x7fae400dd040 comp rx=0 tx=0).stop 2024-03-28T18:10:10.727 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:10.724+0000 7fae4ce00640 1 -- 172.21.15.22:0/2531667224 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fae481002e0 msgr2=0x7fae4819a140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:10:10.727 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:10.724+0000 7fae4ce00640 1 --2- 172.21.15.22:0/2531667224 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fae481002e0 0x7fae4819a140 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7fae40000c00 tx=0x7fae400045e0 comp rx=0 tx=0).stop 2024-03-28T18:10:10.727 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:10.724+0000 7fae4ce00640 1 -- 172.21.15.22:0/2531667224 shutdown_connections 2024-03-28T18:10:10.727 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:10.724+0000 7fae4ce00640 1 --2- 172.21.15.22:0/2531667224 >> [v2:172.21.15.22:6800/1357365082,v1:172.21.15.22:6801/1357365082] conn(0x7fae2c0a29f0 0x7fae2c0a4eb0 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:10:10.727 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:10.724+0000 7fae4ce00640 1 --2- 172.21.15.22:0/2531667224 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fae4810de90 0x7fae4819e9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:10:10.727 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:10.724+0000 7fae4ce00640 1 --2- 172.21.15.22:0/2531667224 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fae48100c90 0x7fae4819a680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:10:10.727 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:10.724+0000 7fae4ce00640 1 --2- 172.21.15.22:0/2531667224 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fae481002e0 0x7fae4819a140 unknown :-1 s=CLOSED pgs=640 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:10:10.727 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:10.724+0000 7fae4ce00640 1 -- 172.21.15.22:0/2531667224 >> 172.21.15.22:0/2531667224 conn(0x7fae480fbf00 msgr2=0x7fae480fe2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:10:10.727 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:10.724+0000 7fae4ce00640 1 -- 172.21.15.22:0/2531667224 shutdown_connections 2024-03-28T18:10:10.727 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:10.724+0000 7fae4ce00640 1 -- 172.21.15.22:0/2531667224 wait complete. 2024-03-28T18:10:10.728 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/rpmsg.txt 2024-03-28T18:10:10.733 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/rtc.txt 2024-03-28T18:10:10.742 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-28T18:10:10.743 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-03-28T18:10:10.745 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/s390/ 2024-03-28T18:10:10.753 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/s390/00-INDEX 2024-03-28T18:10:10.757 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/s390/3270.ChangeLog 2024-03-28T18:10:10.767 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/s390/3270.txt 2024-03-28T18:10:10.776 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/s390/CommonIO 2024-03-28T18:10:10.787 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/s390/DASD 2024-03-28T18:10:10.797 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/s390/Debugging390.txt 2024-03-28T18:10:10.818 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/s390/cds.txt 2024-03-28T18:10:10.832 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/s390/config3270.sh 2024-03-28T18:10:10.845 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/s390/driver-model.txt 2024-03-28T18:10:10.858 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/s390/kvm.txt 2024-03-28T18:10:10.885 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/s390/monreader.txt 2024-03-28T18:10:10.888 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/s390/qeth.txt 2024-03-28T18:10:10.902 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/s390/s390dbf.txt 2024-03-28T18:10:10.918 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/s390/zfcpdump.txt 2024-03-28T18:10:10.934 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/scheduler/ 2024-03-28T18:10:10.949 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/scheduler/00-INDEX 2024-03-28T18:10:10.957 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/scheduler/completion.txt 2024-03-28T18:10:10.974 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/scheduler/sched-arch.txt 2024-03-28T18:10:10.985 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/scheduler/sched-bwc.txt 2024-03-28T18:10:10.986 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:10.986+0000 7f174bc00640 1 -- 172.21.15.22:0/605545602 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1744103840 msgr2=0x7f1744103cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:10:10.986 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:10.986+0000 7f174bc00640 1 --2- 172.21.15.22:0/605545602 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1744103840 0x7f1744103cc0 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7f1730009930 tx=0x7f173002bc30 comp rx=0 tx=0).stop 2024-03-28T18:10:10.986 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:10.986+0000 7f174bc00640 1 -- 172.21.15.22:0/605545602 shutdown_connections 2024-03-28T18:10:10.986 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:10.986+0000 7f174bc00640 1 --2- 172.21.15.22:0/605545602 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f1744104200 0x7f1744110710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:10:10.986 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:10.986+0000 7f174bc00640 1 --2- 172.21.15.22:0/605545602 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1744103840 0x7f1744103cc0 unknown :-1 s=CLOSED pgs=644 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:10:10.986 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:10.986+0000 7f174bc00640 1 --2- 172.21.15.22:0/605545602 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1744109850 0x7f1744109c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:10:10.986 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:10.986+0000 7f174bc00640 1 -- 172.21.15.22:0/605545602 >> 172.21.15.22:0/605545602 conn(0x7f17440ff310 msgr2=0x7f1744101730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:10:10.987 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:10.986+0000 7f174bc00640 1 -- 172.21.15.22:0/605545602 shutdown_connections 2024-03-28T18:10:10.987 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:10.987+0000 7f174bc00640 1 -- 172.21.15.22:0/605545602 wait complete. 2024-03-28T18:10:10.987 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:10.988+0000 7f174bc00640 1 Processor -- start 2024-03-28T18:10:10.987 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:10.988+0000 7f174bc00640 1 -- start start 2024-03-28T18:10:10.989 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:10.988+0000 7f174bc00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1744103840 0x7f174419a1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:10:10.989 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:10.988+0000 7f174bc00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1744104200 0x7f174419a6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:10:10.989 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:10.988+0000 7f174bc00640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f1744109850 0x7f174419ea60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:10:10.989 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:10.988+0000 7f174bc00640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1744113c80 con 0x7f1744104200 2024-03-28T18:10:10.989 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:10.988+0000 7f174bc00640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1744113b00 con 0x7f1744109850 2024-03-28T18:10:10.989 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:10.988+0000 7f174bc00640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1744113e00 con 0x7f1744103840 2024-03-28T18:10:10.989 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:10.989+0000 7f1749c00640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f1744109850 0x7f174419ea60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:10:10.989 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:10.989+0000 7f1743e00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1744104200 0x7f174419a6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:10:10.989 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:10.989+0000 7f1749c00640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f1744109850 0x7f174419ea60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.22:52742/0 (socket says 172.21.15.22:52742) 2024-03-28T18:10:10.989 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:10.989+0000 7f1749c00640 1 -- 172.21.15.22:0/315555845 learned_addr learned my addr 172.21.15.22:0/315555845 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-28T18:10:10.990 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:10.989+0000 7f1749200640 1 --2- 172.21.15.22:0/315555845 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1744103840 0x7f174419a1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:10:10.990 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:10.989+0000 7f1749c00640 1 -- 172.21.15.22:0/315555845 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1744103840 msgr2=0x7f174419a1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:10:10.990 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:10.989+0000 7f1749c00640 1 --2- 172.21.15.22:0/315555845 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1744103840 0x7f174419a1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:10:10.990 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:10.989+0000 7f1749c00640 1 -- 172.21.15.22:0/315555845 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1744104200 msgr2=0x7f174419a6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:10:10.990 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:10.989+0000 7f1749c00640 1 --2- 172.21.15.22:0/315555845 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1744104200 0x7f174419a6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:10:10.990 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:10.989+0000 7f1749c00640 1 -- 172.21.15.22:0/315555845 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f174419f190 con 0x7f1744109850 2024-03-28T18:10:10.990 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:10.989+0000 7f1743e00640 1 --2- 172.21.15.22:0/315555845 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1744104200 0x7f174419a6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-28T18:10:10.991 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:10.990+0000 7f1749c00640 1 --2- 172.21.15.22:0/315555845 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f1744109850 0x7f174419ea60 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f173c00eb10 tx=0x7f173c00c2d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:10:10.991 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:10.990+0000 7f1741600640 1 -- 172.21.15.22:0/315555845 <== mon.2 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f173c004b20 con 0x7f1744109850 2024-03-28T18:10:10.991 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:10.990+0000 7f1741600640 1 -- 172.21.15.22:0/315555845 <== mon.2 v2:172.21.15.79:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f173c004cc0 con 0x7f1744109850 2024-03-28T18:10:10.991 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:10.990+0000 7f1741600640 1 -- 172.21.15.22:0/315555845 <== mon.2 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f173c010700 con 0x7f1744109850 2024-03-28T18:10:10.991 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:10.990+0000 7f174bc00640 1 -- 172.21.15.22:0/315555845 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f174419f480 con 0x7f1744109850 2024-03-28T18:10:10.991 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:10.990+0000 7f174bc00640 1 -- 172.21.15.22:0/315555845 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f17441a6d10 con 0x7f1744109850 2024-03-28T18:10:10.993 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:10.993+0000 7f174e9cf740 1 Processor -- start 2024-03-28T18:10:10.993 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:10.994+0000 7f1741600640 1 -- 172.21.15.22:0/315555845 <== mon.2 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 26) v1 ==== 137421+0+0 (secure 0 0 0) 0x7f173c004e70 con 0x7f1744109850 2024-03-28T18:10:10.994 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:10.994+0000 7f174e9cf740 1 -- start start 2024-03-28T18:10:10.994 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:10.994+0000 7f174e9cf740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ac120286d0 0x55ac12028ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:10:10.994 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:10.994+0000 7f174e9cf740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ac12029080 0x55ac12031630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:10:10.995 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:10.995+0000 7f174e9cf740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55ac12031c00 0x55ac12034050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:10:10.995 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:10.995+0000 7f174e9cf740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ac11f8c2c0 con 0x55ac120286d0 2024-03-28T18:10:10.995 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:10.995+0000 7f174e9cf740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ac11f34c90 con 0x55ac12031c00 2024-03-28T18:10:10.995 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:10.995+0000 7f174e9cf740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ac11f88360 con 0x55ac12029080 2024-03-28T18:10:10.995 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/scheduler/sched-deadline.txt 2024-03-28T18:10:10.996 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:10.995+0000 7f1743e00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ac120286d0 0x55ac12028ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:10:10.996 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:10.995+0000 7f1743e00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ac120286d0 0x55ac12028ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:46076/0 (socket says 172.21.15.22:46076) 2024-03-28T18:10:10.996 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:10.995+0000 7f1743e00640 1 -- 172.21.15.22:0/1760833195 learned_addr learned my addr 172.21.15.22:0/1760833195 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-28T18:10:10.997 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:10.995+0000 7f1743e00640 1 --2- 172.21.15.22:0/1760833195 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55ac12031c00 0x55ac12034050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:10:10.997 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:10.995+0000 7f1749200640 1 --2- 172.21.15.22:0/1760833195 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ac12029080 0x55ac12031630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:10:10.997 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:10.996+0000 7f1743e00640 1 -- 172.21.15.22:0/1760833195 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ac12029080 msgr2=0x55ac12031630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T18:10:10.997 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:10.996+0000 7f1743e00640 1 --2- 172.21.15.22:0/1760833195 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ac12029080 0x55ac12031630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:10:10.997 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:10.996+0000 7f1743e00640 1 -- 172.21.15.22:0/1760833195 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55ac12031c00 msgr2=0x55ac12034050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T18:10:10.997 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:10.996+0000 7f1743e00640 1 --2- 172.21.15.22:0/1760833195 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55ac12031c00 0x55ac12034050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:10:10.997 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:10.996+0000 7f1743e00640 1 -- 172.21.15.22:0/1760833195 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ac11f59080 con 0x55ac120286d0 2024-03-28T18:10:10.997 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:10.996+0000 7f1741600640 1 --2- 172.21.15.22:0/315555845 >> [v2:172.21.15.22:6800/1357365082,v1:172.21.15.22:6801/1357365082] conn(0x7f172c0a29f0 0x7f172c0a4eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:10:10.997 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:10.996+0000 7f1743e00640 1 --2- 172.21.15.22:0/1760833195 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ac120286d0 0x55ac12028ab0 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7f1730009900 tx=0x7f1730005fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d086612d0a958464 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:10:10.997 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:10.996+0000 7f1741600640 1 -- 172.21.15.22:0/315555845 <== mon.2 v2:172.21.15.79:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 8411+0+0 (secure 0 0 0) 0x7f173c0cf3c0 con 0x7f1744109850 2024-03-28T18:10:10.997 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:10.996+0000 7f1749200640 1 --2- 172.21.15.22:0/315555845 >> [v2:172.21.15.22:6800/1357365082,v1:172.21.15.22:6801/1357365082] conn(0x7f172c0a29f0 0x7f172c0a4eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:10:10.997 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:10.996+0000 7f1736000640 1 -- 172.21.15.22:0/1760833195 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17300041e0 con 0x55ac120286d0 2024-03-28T18:10:10.997 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:10.997+0000 7f1736000640 1 -- 172.21.15.22:0/1760833195 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f1730004380 con 0x55ac120286d0 2024-03-28T18:10:10.997 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:10.997+0000 7f1736000640 1 -- 172.21.15.22:0/1760833195 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f173002ca90 con 0x55ac120286d0 2024-03-28T18:10:11.001 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:10.998+0000 7f174e9cf740 1 -- 172.21.15.22:0/1760833195 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ac120286d0 msgr2=0x55ac12028ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:10:11.001 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:10.998+0000 7f1749200640 1 --2- 172.21.15.22:0/315555845 >> [v2:172.21.15.22:6800/1357365082,v1:172.21.15.22:6801/1357365082] conn(0x7f172c0a29f0 0x7f172c0a4eb0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x55ac12023e10 tx=0x7f173800b4d0 comp rx=0 tx=0).ready entity=mgr.15522 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:10:11.001 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:10.998+0000 7f174e9cf740 1 --2- 172.21.15.22:0/1760833195 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ac120286d0 0x55ac12028ab0 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7f1730009900 tx=0x7f1730005fd0 comp rx=0 tx=0).stop 2024-03-28T18:10:11.002 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:10.998+0000 7f174e9cf740 1 -- 172.21.15.22:0/1760833195 shutdown_connections 2024-03-28T18:10:11.002 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:10.998+0000 7f174e9cf740 1 --2- 172.21.15.22:0/1760833195 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55ac12031c00 0x55ac12034050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:10:11.002 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:10.998+0000 7f174e9cf740 1 --2- 172.21.15.22:0/1760833195 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ac12029080 0x55ac12031630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:10:11.002 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:10.998+0000 7f174e9cf740 1 --2- 172.21.15.22:0/1760833195 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ac120286d0 0x55ac12028ab0 unknown :-1 s=CLOSED pgs=645 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:10:11.002 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:10.998+0000 7f174e9cf740 1 -- 172.21.15.22:0/1760833195 >> 172.21.15.22:0/1760833195 conn(0x55ac12022da0 msgr2=0x55ac12023180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:10:11.002 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:10.998+0000 7f174e9cf740 1 -- 172.21.15.22:0/1760833195 shutdown_connections 2024-03-28T18:10:11.002 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:10.998+0000 7f174e9cf740 1 -- 172.21.15.22:0/1760833195 wait complete. 2024-03-28T18:10:11.002 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:10.999+0000 7f174e9cf740 1 -- start start 2024-03-28T18:10:11.002 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:10.999+0000 7f174e9cf740 1 Processor -- start 2024-03-28T18:10:11.006 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:11.000+0000 7f174e9cf740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55ac120286d0 0x55ac120314b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:10:11.006 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:11.000+0000 7f174e9cf740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ac12029080 0x55ac1202e840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:10:11.006 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:11.000+0000 7f174e9cf740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ac12031c00 0x55ac1202ed80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:10:11.006 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:11.000+0000 7f174e9cf740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ac12034b10 con 0x55ac12031c00 2024-03-28T18:10:11.006 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:11.000+0000 7f174e9cf740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ac11f480d0 con 0x55ac120286d0 2024-03-28T18:10:11.006 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:11.000+0000 7f174e9cf740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ac12034c90 con 0x55ac12029080 2024-03-28T18:10:11.006 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:11.000+0000 7f174e9cf740 10 client.0 _ll_register_callbacks cb 0x55ac11f4ce80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-28T18:10:11.007 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:11.000+0000 7f1749200640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55ac120286d0 0x55ac120314b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:10:11.007 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:11.000+0000 7f1749200640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55ac120286d0 0x55ac120314b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.22:52750/0 (socket says 172.21.15.22:52750) 2024-03-28T18:10:11.007 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:11.000+0000 7f1749200640 1 -- 172.21.15.22:0/3901581767 learned_addr learned my addr 172.21.15.22:0/3901581767 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-28T18:10:11.007 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:11.000+0000 7f1749c00640 1 --2- 172.21.15.22:0/3901581767 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ac12031c00 0x55ac1202ed80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:10:11.007 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:11.000+0000 7f1743e00640 1 --2- 172.21.15.22:0/3901581767 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ac12029080 0x55ac1202e840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:10:11.007 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:11.001+0000 7f1749200640 1 -- 172.21.15.22:0/3901581767 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ac12029080 msgr2=0x55ac1202e840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T18:10:11.007 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:11.001+0000 7f1749200640 1 --2- 172.21.15.22:0/3901581767 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ac12029080 0x55ac1202e840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:10:11.007 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:11.001+0000 7f1749200640 1 -- 172.21.15.22:0/3901581767 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ac12031c00 msgr2=0x55ac1202ed80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T18:10:11.007 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:11.001+0000 7f1749200640 1 --2- 172.21.15.22:0/3901581767 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ac12031c00 0x55ac1202ed80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:10:11.007 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:11.001+0000 7f1749200640 1 -- 172.21.15.22:0/3901581767 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55ac11ca03f0 con 0x55ac120286d0 2024-03-28T18:10:11.007 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:11.001+0000 7f1749200640 1 --2- 172.21.15.22:0/3901581767 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55ac120286d0 0x55ac120314b0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f1738010680 tx=0x7f17380046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=39c79cb458e4683e server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:10:11.007 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:11.001+0000 7f172aa00640 10 client.0 ms_handle_connect on v2:172.21.15.79:3300/0 2024-03-28T18:10:11.007 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:11.001+0000 7f172aa00640 1 -- 172.21.15.22:0/3901581767 <== mon.2 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f173801f8c0 con 0x55ac120286d0 2024-03-28T18:10:11.007 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:11.001+0000 7f172aa00640 1 -- 172.21.15.22:0/3901581767 <== mon.2 v2:172.21.15.79:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 8411+0+0 (secure 0 0 0) 0x7f173801fa60 con 0x55ac120286d0 2024-03-28T18:10:11.007 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:11.001+0000 7f174e9cf740 1 -- 172.21.15.22:0/3901581767 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f1730004380 con 0x55ac120286d0 2024-03-28T18:10:11.008 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:11.002+0000 7f172aa00640 1 -- 172.21.15.22:0/3901581767 <== mon.2 v2:172.21.15.79:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) v2 ==== 24+0+0 (secure 0 0 0) 0x7f173801fc60 con 0x55ac120286d0 2024-03-28T18:10:11.008 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:11.002+0000 7f174e9cf740 10 client.36128 fetch_fsmap learned FSMap version 31 2024-03-28T18:10:11.008 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:11.002+0000 7f174e9cf740 1 -- 172.21.15.22:0/3901581767 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) v3 -- 0x55ac11d6a7d0 con 0x55ac120286d0 2024-03-28T18:10:11.008 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:11.002+0000 7f172aa00640 1 -- 172.21.15.22:0/3901581767 <== mon.2 v2:172.21.15.79:3300/0 4 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f1730004380 con 0x55ac120286d0 2024-03-28T18:10:11.008 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:11.002+0000 7f172aa00640 1 -- 172.21.15.22:0/3901581767 <== mon.2 v2:172.21.15.79:3300/0 5 ==== fsmap(e 31) v1 ==== 7950+0+0 (secure 0 0 0) 0x7f173802f020 con 0x55ac120286d0 2024-03-28T18:10:11.008 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:11.002+0000 7f174e9cf740 10 client.36128 fetch_fsmap finished waiting for FSMap version 31 2024-03-28T18:10:11.008 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:11.002+0000 7f174e9cf740 10 client.36128 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-03-28T18:10:11.008 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:11.002+0000 7f174e9cf740 20 client.36128 populate_metadata read hostname 'smithi022' 2024-03-28T18:10:11.008 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:11.002+0000 7f174e9cf740 1 --2- 172.21.15.22:0/3901581767 >> [v2:172.21.15.79:6836/1787903366,v1:172.21.15.79:6837/1787903366] conn(0x55ac120419d0 0x55ac12043dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:10:11.008 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:11.002+0000 7f174e9cf740 4 client.36128 mds_command: new command op to 34511 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-28T18:10:11.008 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:11.002+0000 7f174e9cf740 1 -- 172.21.15.22:0/3901581767 --> [v2:172.21.15.79:6836/1787903366,v1:172.21.15.79:6837/1787903366] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55ac11f99800 con 0x55ac120419d0 2024-03-28T18:10:11.008 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:11.003+0000 7f1743e00640 1 --2- 172.21.15.22:0/3901581767 >> [v2:172.21.15.79:6836/1787903366,v1:172.21.15.79:6837/1787903366] conn(0x55ac120419d0 0x55ac12043dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:10:11.008 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:11.003+0000 7f1743e00640 1 --2- 172.21.15.22:0/3901581767 >> [v2:172.21.15.79:6836/1787903366,v1:172.21.15.79:6837/1787903366] conn(0x55ac120419d0 0x55ac12043dc0 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=c5627b65f3a03871 server_cookie=fcb0f11316867f79 in_seq=0 out_seq=0 2024-03-28T18:10:11.008 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:11.003+0000 7f172aa00640 10 client.36128 ms_handle_connect on v2:172.21.15.79:6836/1787903366 2024-03-28T18:10:11.008 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:11.004+0000 7f172aa00640 1 -- 172.21.15.22:0/3901581767 <== mds.1 v2:172.21.15.79:6836/1787903366 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x55ac11f99800 con 0x55ac120419d0 2024-03-28T18:10:11.009 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:11.004+0000 7f172aa00640 10 client.36128 handle_command_reply: tid=0 2024-03-28T18:10:11.009 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:11.005+0000 7f174e9cf740 1 client.36128 shutdown 2024-03-28T18:10:11.009 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:11.005+0000 7f174e9cf740 1 -- 172.21.15.22:0/3901581767 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55ac120286d0 msgr2=0x55ac120314b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:10:11.009 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:11.005+0000 7f174e9cf740 1 --2- 172.21.15.22:0/3901581767 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55ac120286d0 0x55ac120314b0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f1738010680 tx=0x7f17380046c0 comp rx=0 tx=0).stop 2024-03-28T18:10:11.009 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:11.005+0000 7f174e9cf740 1 -- 172.21.15.22:0/3901581767 shutdown_connections 2024-03-28T18:10:11.009 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:11.005+0000 7f174e9cf740 1 --2- 172.21.15.22:0/3901581767 >> [v2:172.21.15.79:6836/1787903366,v1:172.21.15.79:6837/1787903366] conn(0x55ac120419d0 0x55ac12043dc0 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:10:11.009 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:11.005+0000 7f174e9cf740 1 --2- 172.21.15.22:0/3901581767 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ac12031c00 0x55ac1202ed80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:10:11.009 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:11.005+0000 7f174e9cf740 1 --2- 172.21.15.22:0/3901581767 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ac12029080 0x55ac1202e840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:10:11.009 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:11.005+0000 7f174e9cf740 1 --2- 172.21.15.22:0/3901581767 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55ac120286d0 0x55ac120314b0 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:10:11.009 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:11.005+0000 7f174e9cf740 1 -- 172.21.15.22:0/3901581767 >> 172.21.15.22:0/3901581767 conn(0x55ac12022da0 msgr2=0x55ac12024af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:10:11.009 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:11.005+0000 7f172aa00640 0 client.36128 ms_handle_reset on v2:172.21.15.79:6836/1787903366 2024-03-28T18:10:11.009 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:11.005+0000 7f174e9cf740 1 -- 172.21.15.22:0/3901581767 shutdown_connections 2024-03-28T18:10:11.009 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:11.005+0000 7f174e9cf740 1 -- 172.21.15.22:0/3901581767 wait complete. 2024-03-28T18:10:11.009 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:11.005+0000 7f174e9cf740 20 client.36128 trim_cache size 0 max 16384 2024-03-28T18:10:11.010 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/scheduler/sched-design-CFS.txt 2024-03-28T18:10:11.014 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/scheduler/sched-domains.txt 2024-03-28T18:10:11.021 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:11.022+0000 7f174e9cf740 1 Processor -- start 2024-03-28T18:10:11.021 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:11.022+0000 7f174e9cf740 1 -- start start 2024-03-28T18:10:11.023 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:11.022+0000 7f174e9cf740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55ac120286d0 0x55ac12030f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:10:11.023 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:11.022+0000 7f174e9cf740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ac12029080 0x55ac120314b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:10:11.023 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:11.022+0000 7f174e9cf740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ac12031c00 0x55ac1202bac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:10:11.023 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:11.022+0000 7f174e9cf740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17441af220 con 0x55ac12031c00 2024-03-28T18:10:11.023 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:11.022+0000 7f174e9cf740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17441b0c60 con 0x55ac120286d0 2024-03-28T18:10:11.023 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:11.022+0000 7f174e9cf740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17441af880 con 0x55ac12029080 2024-03-28T18:10:11.023 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:11.022+0000 7f1749c00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ac12031c00 0x55ac1202bac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:10:11.023 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:11.023+0000 7f1749c00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ac12031c00 0x55ac1202bac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:46090/0 (socket says 172.21.15.22:46090) 2024-03-28T18:10:11.023 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:11.023+0000 7f1749c00640 1 -- 172.21.15.22:0/1182772017 learned_addr learned my addr 172.21.15.22:0/1182772017 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-28T18:10:11.023 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:11.023+0000 7f1749200640 1 --2- 172.21.15.22:0/1182772017 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55ac120286d0 0x55ac12030f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:10:11.023 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:11.023+0000 7f1743e00640 1 --2- 172.21.15.22:0/1182772017 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ac12029080 0x55ac120314b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:10:11.024 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:11.023+0000 7f1749c00640 1 -- 172.21.15.22:0/1182772017 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ac12029080 msgr2=0x55ac120314b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T18:10:11.024 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:11.023+0000 7f1749c00640 1 --2- 172.21.15.22:0/1182772017 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ac12029080 0x55ac120314b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:10:11.024 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:11.023+0000 7f1749c00640 1 -- 172.21.15.22:0/1182772017 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55ac120286d0 msgr2=0x55ac12030f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T18:10:11.024 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:11.023+0000 7f1749c00640 1 --2- 172.21.15.22:0/1182772017 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55ac120286d0 0x55ac12030f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:10:11.024 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:11.023+0000 7f1749c00640 1 -- 172.21.15.22:0/1182772017 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ac12028100 con 0x55ac12031c00 2024-03-28T18:10:11.024 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:11.023+0000 7f1749c00640 1 --2- 172.21.15.22:0/1182772017 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ac12031c00 0x55ac1202bac0 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7f173c005930 tx=0x7f173c09b990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=724a6d4f9f7504f5 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:10:11.024 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:11.024+0000 7f1735600640 1 -- 172.21.15.22:0/1182772017 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f173c0b3070 con 0x55ac12031c00 2024-03-28T18:10:11.025 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:11.024+0000 7f1735600640 1 -- 172.21.15.22:0/1182772017 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f173c09ba90 con 0x55ac12031c00 2024-03-28T18:10:11.025 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:11.024+0000 7f1735600640 1 -- 172.21.15.22:0/1182772017 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f173c09bcf0 con 0x55ac12031c00 2024-03-28T18:10:11.025 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/scheduler/sched-nice-design.txt 2024-03-28T18:10:11.026 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:11.025+0000 7f174e9cf740 1 -- 172.21.15.22:0/1182772017 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ac12031c00 msgr2=0x55ac1202bac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:10:11.026 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:11.025+0000 7f174e9cf740 1 --2- 172.21.15.22:0/1182772017 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ac12031c00 0x55ac1202bac0 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7f173c005930 tx=0x7f173c09b990 comp rx=0 tx=0).stop 2024-03-28T18:10:11.026 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:11.025+0000 7f174e9cf740 1 -- 172.21.15.22:0/1182772017 shutdown_connections 2024-03-28T18:10:11.027 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:11.025+0000 7f174e9cf740 1 --2- 172.21.15.22:0/1182772017 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ac12031c00 0x55ac1202bac0 unknown :-1 s=CLOSED pgs=646 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:10:11.027 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:11.025+0000 7f174e9cf740 1 --2- 172.21.15.22:0/1182772017 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ac12029080 0x55ac120314b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:10:11.027 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:11.025+0000 7f174e9cf740 1 --2- 172.21.15.22:0/1182772017 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55ac120286d0 0x55ac12030f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:10:11.027 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:11.025+0000 7f174e9cf740 1 -- 172.21.15.22:0/1182772017 >> 172.21.15.22:0/1182772017 conn(0x55ac12022da0 msgr2=0x55ac12024a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:10:11.027 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:11.025+0000 7f174e9cf740 1 -- 172.21.15.22:0/1182772017 shutdown_connections 2024-03-28T18:10:11.027 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:11.025+0000 7f174e9cf740 1 -- 172.21.15.22:0/1182772017 wait complete. 2024-03-28T18:10:11.027 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:11.025+0000 7f174e9cf740 1 -- start start 2024-03-28T18:10:11.027 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:11.026+0000 7f174e9cf740 1 Processor -- start 2024-03-28T18:10:11.027 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:11.026+0000 7f174e9cf740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ac120286d0 0x55ac12035950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:10:11.027 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:11.026+0000 7f174e9cf740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55ac12029080 0x55ac12035e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:10:11.027 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:11.026+0000 7f174e9cf740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ac12031c00 0x55ac1209c230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:10:11.027 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:11.026+0000 7f174e9cf740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17441b0540 con 0x55ac12031c00 2024-03-28T18:10:11.027 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:11.026+0000 7f174e9cf740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17441afee0 con 0x55ac12029080 2024-03-28T18:10:11.027 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:11.026+0000 7f174e9cf740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17441ad3e0 con 0x55ac120286d0 2024-03-28T18:10:11.027 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:11.027+0000 7f174e9cf740 10 client.0 _ll_register_callbacks cb 0x7f173800bea0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-28T18:10:11.027 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:11.027+0000 7f1743e00640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55ac12029080 0x55ac12035e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:10:11.029 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:11.027+0000 7f1743e00640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55ac12029080 0x55ac12035e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.22:52772/0 (socket says 172.21.15.22:52772) 2024-03-28T18:10:11.029 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:11.027+0000 7f1743e00640 1 -- 172.21.15.22:0/3278709156 learned_addr learned my addr 172.21.15.22:0/3278709156 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-28T18:10:11.029 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:11.027+0000 7f1749c00640 1 --2- 172.21.15.22:0/3278709156 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ac12031c00 0x55ac1209c230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:10:11.029 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:11.027+0000 7f1749200640 1 --2- 172.21.15.22:0/3278709156 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ac120286d0 0x55ac12035950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:10:11.029 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:11.027+0000 7f1743e00640 1 -- 172.21.15.22:0/3278709156 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ac120286d0 msgr2=0x55ac12035950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T18:10:11.029 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:11.027+0000 7f1743e00640 1 --2- 172.21.15.22:0/3278709156 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ac120286d0 0x55ac12035950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:10:11.029 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:11.027+0000 7f1743e00640 1 -- 172.21.15.22:0/3278709156 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ac12031c00 msgr2=0x55ac1209c230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T18:10:11.029 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:11.027+0000 7f1743e00640 1 --2- 172.21.15.22:0/3278709156 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ac12031c00 0x55ac1209c230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:10:11.030 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:11.027+0000 7f1743e00640 1 -- 172.21.15.22:0/3278709156 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55ac1209c770 con 0x55ac12029080 2024-03-28T18:10:11.030 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:11.028+0000 7f1743e00640 1 --2- 172.21.15.22:0/3278709156 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55ac12029080 0x55ac12035e90 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f173002c210 tx=0x7f17300056d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=55c3a692e8ccbe10 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:10:11.030 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:11.028+0000 7f172aa00640 10 client.0 ms_handle_connect on v2:172.21.15.79:3300/0 2024-03-28T18:10:11.030 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:11.028+0000 7f172aa00640 1 -- 172.21.15.22:0/3278709156 <== mon.2 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1730004360 con 0x55ac12029080 2024-03-28T18:10:11.030 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:11.028+0000 7f172aa00640 1 -- 172.21.15.22:0/3278709156 <== mon.2 v2:172.21.15.79:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 8411+0+0 (secure 0 0 0) 0x7f17300051b0 con 0x55ac12029080 2024-03-28T18:10:11.030 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:11.028+0000 7f174e9cf740 1 -- 172.21.15.22:0/3278709156 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f173c09ba90 con 0x55ac12029080 2024-03-28T18:10:11.030 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:11.028+0000 7f172aa00640 1 -- 172.21.15.22:0/3278709156 <== mon.2 v2:172.21.15.79:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) v2 ==== 24+0+0 (secure 0 0 0) 0x7f17300053d0 con 0x55ac12029080 2024-03-28T18:10:11.030 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:11.028+0000 7f174e9cf740 10 client.36134 fetch_fsmap learned FSMap version 31 2024-03-28T18:10:11.030 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:11.028+0000 7f174e9cf740 1 -- 172.21.15.22:0/3278709156 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) v3 -- 0x55ac1209d970 con 0x55ac12029080 2024-03-28T18:10:11.030 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:11.029+0000 7f172aa00640 1 -- 172.21.15.22:0/3278709156 <== mon.2 v2:172.21.15.79:3300/0 4 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f173c09ba90 con 0x55ac12029080 2024-03-28T18:10:11.030 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:11.029+0000 7f172aa00640 1 -- 172.21.15.22:0/3278709156 <== mon.2 v2:172.21.15.79:3300/0 5 ==== fsmap(e 31) v1 ==== 7950+0+0 (secure 0 0 0) 0x7f173005f020 con 0x55ac12029080 2024-03-28T18:10:11.030 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:11.029+0000 7f174e9cf740 10 client.36134 fetch_fsmap finished waiting for FSMap version 31 2024-03-28T18:10:11.030 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:11.029+0000 7f174e9cf740 10 client.36134 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-03-28T18:10:11.030 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:11.029+0000 7f174e9cf740 20 client.36134 populate_metadata read hostname 'smithi022' 2024-03-28T18:10:11.030 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:11.029+0000 7f174e9cf740 1 --2- 172.21.15.22:0/3278709156 >> [v2:172.21.15.79:6836/1787903366,v1:172.21.15.79:6837/1787903366] conn(0x55ac1209dba0 0x55ac1209ff90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:10:11.031 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:11.029+0000 7f174e9cf740 4 client.36134 mds_command: new command op to 34511 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-28T18:10:11.031 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:11.029+0000 7f174e9cf740 1 -- 172.21.15.22:0/3278709156 --> [v2:172.21.15.79:6836/1787903366,v1:172.21.15.79:6837/1787903366] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55ac1202f110 con 0x55ac1209dba0 2024-03-28T18:10:11.031 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:11.029+0000 7f1749200640 1 --2- 172.21.15.22:0/3278709156 >> [v2:172.21.15.79:6836/1787903366,v1:172.21.15.79:6837/1787903366] conn(0x55ac1209dba0 0x55ac1209ff90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:10:11.032 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:11.030+0000 7f1749200640 1 --2- 172.21.15.22:0/3278709156 >> [v2:172.21.15.79:6836/1787903366,v1:172.21.15.79:6837/1787903366] conn(0x55ac1209dba0 0x55ac1209ff90 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=2724de7f2e83f929 server_cookie=5e9c73c76879f5d0 in_seq=0 out_seq=0 2024-03-28T18:10:11.032 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:11.030+0000 7f172aa00640 10 client.36134 ms_handle_connect on v2:172.21.15.79:6836/1787903366 2024-03-28T18:10:11.032 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:11.030+0000 7f172aa00640 1 -- 172.21.15.22:0/3278709156 <== mds.1 v2:172.21.15.79:6836/1787903366 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55ac1202f110 con 0x55ac1209dba0 2024-03-28T18:10:11.032 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:11.031+0000 7f172aa00640 10 client.36134 handle_command_reply: tid=0 2024-03-28T18:10:11.032 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:11.031+0000 7f174e9cf740 1 client.36134 shutdown 2024-03-28T18:10:11.032 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:11.031+0000 7f174e9cf740 1 -- 172.21.15.22:0/3278709156 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55ac12029080 msgr2=0x55ac12035e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:10:11.032 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:11.031+0000 7f174e9cf740 1 --2- 172.21.15.22:0/3278709156 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55ac12029080 0x55ac12035e90 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f173002c210 tx=0x7f17300056d0 comp rx=0 tx=0).stop 2024-03-28T18:10:11.032 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:11.031+0000 7f174e9cf740 1 -- 172.21.15.22:0/3278709156 shutdown_connections 2024-03-28T18:10:11.033 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:11.031+0000 7f174e9cf740 1 --2- 172.21.15.22:0/3278709156 >> [v2:172.21.15.79:6836/1787903366,v1:172.21.15.79:6837/1787903366] conn(0x55ac1209dba0 0x55ac1209ff90 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:10:11.033 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:11.031+0000 7f174e9cf740 1 --2- 172.21.15.22:0/3278709156 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ac12031c00 0x55ac1209c230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:10:11.033 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:11.031+0000 7f174e9cf740 1 --2- 172.21.15.22:0/3278709156 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55ac12029080 0x55ac12035e90 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:10:11.033 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:11.031+0000 7f174e9cf740 1 --2- 172.21.15.22:0/3278709156 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ac120286d0 0x55ac12035950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:10:11.033 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:11.031+0000 7f172aa00640 0 client.36134 ms_handle_reset on v2:172.21.15.79:6836/1787903366 2024-03-28T18:10:11.033 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:11.031+0000 7f174e9cf740 1 -- 172.21.15.22:0/3278709156 >> 172.21.15.22:0/3278709156 conn(0x55ac12022da0 msgr2=0x55ac12029460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:10:11.033 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:11.031+0000 7f174e9cf740 1 -- 172.21.15.22:0/3278709156 shutdown_connections 2024-03-28T18:10:11.033 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:11.031+0000 7f174e9cf740 1 -- 172.21.15.22:0/3278709156 wait complete. 2024-03-28T18:10:11.033 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:11.031+0000 7f174e9cf740 20 client.36134 trim_cache size 0 max 16384 2024-03-28T18:10:11.033 INFO:teuthology.orchestra.run.smithi022.stdout:[] 2024-03-28T18:10:11.034 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:11.032+0000 7f174bc00640 1 -- 172.21.15.22:0/315555845 >> [v2:172.21.15.22:6800/1357365082,v1:172.21.15.22:6801/1357365082] conn(0x7f172c0a29f0 msgr2=0x7f172c0a4eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:10:11.034 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:11.032+0000 7f174bc00640 1 --2- 172.21.15.22:0/315555845 >> [v2:172.21.15.22:6800/1357365082,v1:172.21.15.22:6801/1357365082] conn(0x7f172c0a29f0 0x7f172c0a4eb0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x55ac12023e10 tx=0x7f173800b4d0 comp rx=0 tx=0).stop 2024-03-28T18:10:11.034 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:11.032+0000 7f174bc00640 1 -- 172.21.15.22:0/315555845 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f1744109850 msgr2=0x7f174419ea60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:10:11.034 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:11.032+0000 7f174bc00640 1 --2- 172.21.15.22:0/315555845 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f1744109850 0x7f174419ea60 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f173c00eb10 tx=0x7f173c00c2d0 comp rx=0 tx=0).stop 2024-03-28T18:10:11.035 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:11.032+0000 7f174bc00640 1 -- 172.21.15.22:0/315555845 shutdown_connections 2024-03-28T18:10:11.035 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:11.032+0000 7f174bc00640 1 --2- 172.21.15.22:0/315555845 >> [v2:172.21.15.22:6800/1357365082,v1:172.21.15.22:6801/1357365082] conn(0x7f172c0a29f0 0x7f172c0a4eb0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:10:11.035 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:11.032+0000 7f174bc00640 1 --2- 172.21.15.22:0/315555845 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f1744109850 0x7f174419ea60 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:10:11.035 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:11.032+0000 7f174bc00640 1 --2- 172.21.15.22:0/315555845 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1744104200 0x7f174419a6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:10:11.035 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:11.032+0000 7f174bc00640 1 --2- 172.21.15.22:0/315555845 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1744103840 0x7f174419a1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:10:11.035 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:11.032+0000 7f174bc00640 1 -- 172.21.15.22:0/315555845 >> 172.21.15.22:0/315555845 conn(0x7f17440ff310 msgr2=0x7f17440ffab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:10:11.035 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:11.032+0000 7f174bc00640 1 -- 172.21.15.22:0/315555845 shutdown_connections 2024-03-28T18:10:11.035 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:11.032+0000 7f174bc00640 1 -- 172.21.15.22:0/315555845 wait complete. 2024-03-28T18:10:11.035 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/scheduler/sched-rt-group.txt 2024-03-28T18:10:11.043 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/scheduler/sched-stats.txt 2024-03-28T18:10:11.051 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-28T18:10:11.051 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-03-28T18:10:11.054 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/scsi/ 2024-03-28T18:10:11.064 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/scsi/00-INDEX 2024-03-28T18:10:11.068 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/scsi/53c700.txt 2024-03-28T18:10:11.080 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/scsi/BusLogic.txt 2024-03-28T18:10:11.091 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/scsi/ChangeLog.1992-1997 2024-03-28T18:10:11.101 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/scsi/ChangeLog.arcmsr 2024-03-28T18:10:11.111 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/scsi/ChangeLog.ips 2024-03-28T18:10:11.122 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/scsi/ChangeLog.lpfc 2024-03-28T18:10:11.138 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/scsi/ChangeLog.megaraid 2024-03-28T18:10:11.150 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/scsi/ChangeLog.megaraid_sas 2024-03-28T18:10:11.166 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/scsi/ChangeLog.ncr53c8xx 2024-03-28T18:10:11.180 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/scsi/ChangeLog.sym53c8xx 2024-03-28T18:10:11.195 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/scsi/ChangeLog.sym53c8xx_2 2024-03-28T18:10:11.210 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/scsi/FlashPoint.txt 2024-03-28T18:10:11.223 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/scsi/LICENSE.FlashPoint 2024-03-28T18:10:11.237 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/scsi/LICENSE.qla2xxx 2024-03-28T18:10:11.253 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/scsi/LICENSE.qla4xxx 2024-03-28T18:10:11.266 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/scsi/Mylex.txt 2024-03-28T18:10:11.271 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:11.271+0000 7f58d1e00640 1 -- 172.21.15.22:0/3951478227 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f58cc1064a0 msgr2=0x7f58cc106880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:10:11.271 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:11.271+0000 7f58d1e00640 1 --2- 172.21.15.22:0/3951478227 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f58cc1064a0 0x7f58cc106880 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f58c0009900 tx=0x7f58c002bc40 comp rx=0 tx=0).stop 2024-03-28T18:10:11.271 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:11.271+0000 7f58d1e00640 1 -- 172.21.15.22:0/3951478227 shutdown_connections 2024-03-28T18:10:11.271 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:11.271+0000 7f58d1e00640 1 --2- 172.21.15.22:0/3951478227 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f58cc100dc0 0x7f58cc10d2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:10:11.272 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:11.271+0000 7f58d1e00640 1 --2- 172.21.15.22:0/3951478227 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f58cc100400 0x7f58cc100880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:10:11.272 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:11.271+0000 7f58d1e00640 1 --2- 172.21.15.22:0/3951478227 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f58cc1064a0 0x7f58cc106880 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:10:11.272 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:11.271+0000 7f58d1e00640 1 -- 172.21.15.22:0/3951478227 >> 172.21.15.22:0/3951478227 conn(0x7f58cc0fbef0 msgr2=0x7f58cc0fe310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:10:11.272 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:11.272+0000 7f58d1e00640 1 -- 172.21.15.22:0/3951478227 shutdown_connections 2024-03-28T18:10:11.272 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:11.272+0000 7f58d1e00640 1 -- 172.21.15.22:0/3951478227 wait complete. 2024-03-28T18:10:11.273 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:11.272+0000 7f58d1e00640 1 Processor -- start 2024-03-28T18:10:11.273 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:11.273+0000 7f58d1e00640 1 -- start start 2024-03-28T18:10:11.273 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:11.273+0000 7f58d1e00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f58cc100400 0x7f58cc102060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:10:11.273 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:11.273+0000 7f58d1e00640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f58cc100dc0 0x7f58cc1025a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:10:11.273 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:11.273+0000 7f58d1e00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f58cc1064a0 0x7f58cc102ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:10:11.273 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:11.273+0000 7f58d1e00640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58cc110810 con 0x7f58cc100400 2024-03-28T18:10:11.273 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:11.273+0000 7f58d1e00640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58cc110690 con 0x7f58cc100dc0 2024-03-28T18:10:11.273 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:11.273+0000 7f58d1e00640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58cc110990 con 0x7f58cc1064a0 2024-03-28T18:10:11.274 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:11.273+0000 7f58d0a00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f58cc100400 0x7f58cc102060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:10:11.274 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:11.273+0000 7f58d0a00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f58cc100400 0x7f58cc102060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:46112/0 (socket says 172.21.15.22:46112) 2024-03-28T18:10:11.274 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:11.273+0000 7f58cbe00640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f58cc100dc0 0x7f58cc1025a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:10:11.274 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:11.273+0000 7f58d0a00640 1 -- 172.21.15.22:0/2846192617 learned_addr learned my addr 172.21.15.22:0/2846192617 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-28T18:10:11.274 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:11.274+0000 7f58d1400640 1 --2- 172.21.15.22:0/2846192617 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f58cc1064a0 0x7f58cc102ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:10:11.275 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:11.274+0000 7f58d0a00640 1 -- 172.21.15.22:0/2846192617 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f58cc1064a0 msgr2=0x7f58cc102ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:10:11.275 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:11.274+0000 7f58d0a00640 1 --2- 172.21.15.22:0/2846192617 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f58cc1064a0 0x7f58cc102ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:10:11.275 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:11.274+0000 7f58d0a00640 1 -- 172.21.15.22:0/2846192617 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f58cc100dc0 msgr2=0x7f58cc1025a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:10:11.275 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:11.274+0000 7f58d0a00640 1 --2- 172.21.15.22:0/2846192617 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f58cc100dc0 0x7f58cc1025a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:10:11.275 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:11.274+0000 7f58d0a00640 1 -- 172.21.15.22:0/2846192617 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f58cc1a62e0 con 0x7f58cc100400 2024-03-28T18:10:11.275 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:11.274+0000 7f58d0a00640 1 --2- 172.21.15.22:0/2846192617 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f58cc100400 0x7f58cc102060 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7f58c0000c00 tx=0x7f58c0004430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:10:11.275 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:11.275+0000 7f58c9600640 1 -- 172.21.15.22:0/2846192617 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58c0043020 con 0x7f58cc100400 2024-03-28T18:10:11.275 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:11.275+0000 7f58d1e00640 1 -- 172.21.15.22:0/2846192617 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f58cc1a6570 con 0x7f58cc100400 2024-03-28T18:10:11.275 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:11.275+0000 7f58c9600640 1 -- 172.21.15.22:0/2846192617 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f58c0005470 con 0x7f58cc100400 2024-03-28T18:10:11.276 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:11.275+0000 7f58c9600640 1 -- 172.21.15.22:0/2846192617 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58c0003ad0 con 0x7f58cc100400 2024-03-28T18:10:11.276 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:11.275+0000 7f58d1e00640 1 -- 172.21.15.22:0/2846192617 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f58cc1a6b10 con 0x7f58cc100400 2024-03-28T18:10:11.276 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/scsi/NinjaSCSI.txt 2024-03-28T18:10:11.278 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:11.279+0000 7f58d61c1740 1 Processor -- start 2024-03-28T18:10:11.278 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:11.279+0000 7f58d61c1740 1 -- start start 2024-03-28T18:10:11.280 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:11.279+0000 7f58d61c1740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56455e5ce8b0 0x56455e5cec90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:10:11.280 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:11.279+0000 7f58d61c1740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56455e5cf260 0x56455e5d7810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:10:11.280 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:11.279+0000 7f58d61c1740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56455e5d7de0 0x56455e5da230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:10:11.280 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:11.279+0000 7f58d61c1740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x56455e5322c0 con 0x56455e5ce8b0 2024-03-28T18:10:11.280 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:11.279+0000 7f58d61c1740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x56455e4dac90 con 0x56455e5d7de0 2024-03-28T18:10:11.280 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:11.279+0000 7f58d61c1740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x56455e52e360 con 0x56455e5cf260 2024-03-28T18:10:11.280 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:11.279+0000 7f58c9600640 1 -- 172.21.15.22:0/2846192617 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 26) v1 ==== 137421+0+0 (secure 0 0 0) 0x7f58c0046020 con 0x7f58cc100400 2024-03-28T18:10:11.280 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:11.280+0000 7f58cbe00640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56455e5d7de0 0x56455e5da230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:10:11.281 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:11.280+0000 7f58d1400640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56455e5ce8b0 0x56455e5cec90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:10:11.281 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:11.280+0000 7f58cbe00640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56455e5d7de0 0x56455e5da230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.22:52796/0 (socket says 172.21.15.22:52796) 2024-03-28T18:10:11.281 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:11.280+0000 7f58cbe00640 1 -- 172.21.15.22:0/2948347433 learned_addr learned my addr 172.21.15.22:0/2948347433 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-28T18:10:11.281 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:11.280+0000 7f58d0a00640 1 --2- 172.21.15.22:0/2948347433 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56455e5cf260 0x56455e5d7810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:10:11.281 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:11.280+0000 7f58cbe00640 1 -- 172.21.15.22:0/2948347433 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56455e5cf260 msgr2=0x56455e5d7810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T18:10:11.281 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:11.280+0000 7f58cbe00640 1 --2- 172.21.15.22:0/2948347433 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56455e5cf260 0x56455e5d7810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:10:11.281 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:11.280+0000 7f58cbe00640 1 -- 172.21.15.22:0/2948347433 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56455e5ce8b0 msgr2=0x56455e5cec90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T18:10:11.281 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:11.280+0000 7f58cbe00640 1 --2- 172.21.15.22:0/2948347433 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56455e5ce8b0 0x56455e5cec90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:10:11.281 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:11.280+0000 7f58cbe00640 1 -- 172.21.15.22:0/2948347433 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56455e4ff080 con 0x56455e5d7de0 2024-03-28T18:10:11.281 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:11.280+0000 7f58d0a00640 1 --2- 172.21.15.22:0/2948347433 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56455e5cf260 0x56455e5d7810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T18:10:11.281 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:11.280+0000 7f58d1400640 1 --2- 172.21.15.22:0/2948347433 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56455e5ce8b0 0x56455e5cec90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-28T18:10:11.281 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:11.281+0000 7f58cbe00640 1 --2- 172.21.15.22:0/2948347433 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56455e5d7de0 0x56455e5da230 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f58b800aca0 tx=0x7f58b800f790 comp rx=0 tx=0).ready entity=mon.2 client_cookie=291da5600e329b39 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:10:11.283 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:11.281+0000 7f58bdc00640 1 -- 172.21.15.22:0/2948347433 <== mon.2 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58b801d070 con 0x56455e5d7de0 2024-03-28T18:10:11.283 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:11.281+0000 7f58bdc00640 1 -- 172.21.15.22:0/2948347433 <== mon.2 v2:172.21.15.79:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f58b8018b10 con 0x56455e5d7de0 2024-03-28T18:10:11.283 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:11.281+0000 7f58bdc00640 1 -- 172.21.15.22:0/2948347433 <== mon.2 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58b8010bf0 con 0x56455e5d7de0 2024-03-28T18:10:11.283 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:11.281+0000 7f58c9600640 1 --2- 172.21.15.22:0/2846192617 >> [v2:172.21.15.22:6800/1357365082,v1:172.21.15.22:6801/1357365082] conn(0x7f58b40a2970 0x7f58b40a4e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:10:11.283 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:11.282+0000 7f58c9600640 1 -- 172.21.15.22:0/2846192617 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 8411+0+0 (secure 0 0 0) 0x7f58c00f0ec0 con 0x7f58cc100400 2024-03-28T18:10:11.283 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:11.282+0000 7f58d1400640 1 --2- 172.21.15.22:0/2846192617 >> [v2:172.21.15.22:6800/1357365082,v1:172.21.15.22:6801/1357365082] conn(0x7f58b40a2970 0x7f58b40a4e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:10:11.283 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:11.282+0000 7f58d61c1740 1 -- 172.21.15.22:0/2948347433 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56455e5d7de0 msgr2=0x56455e5da230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:10:11.283 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:11.282+0000 7f58d61c1740 1 --2- 172.21.15.22:0/2948347433 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56455e5d7de0 0x56455e5da230 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f58b800aca0 tx=0x7f58b800f790 comp rx=0 tx=0).stop 2024-03-28T18:10:11.283 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:11.282+0000 7f58d61c1740 1 -- 172.21.15.22:0/2948347433 shutdown_connections 2024-03-28T18:10:11.284 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:11.282+0000 7f58d61c1740 1 --2- 172.21.15.22:0/2948347433 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56455e5d7de0 0x56455e5da230 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:10:11.284 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:11.282+0000 7f58d61c1740 1 --2- 172.21.15.22:0/2948347433 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56455e5cf260 0x56455e5d7810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:10:11.284 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:11.282+0000 7f58d61c1740 1 --2- 172.21.15.22:0/2948347433 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56455e5ce8b0 0x56455e5cec90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:10:11.284 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:11.282+0000 7f58d61c1740 1 -- 172.21.15.22:0/2948347433 >> 172.21.15.22:0/2948347433 conn(0x56455e5c8f80 msgr2=0x56455e5c9360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:10:11.284 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:11.282+0000 7f58d61c1740 1 -- 172.21.15.22:0/2948347433 shutdown_connections 2024-03-28T18:10:11.284 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:11.283+0000 7f58d61c1740 1 -- 172.21.15.22:0/2948347433 wait complete. 2024-03-28T18:10:11.284 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:11.283+0000 7f58d1400640 1 --2- 172.21.15.22:0/2846192617 >> [v2:172.21.15.22:6800/1357365082,v1:172.21.15.22:6801/1357365082] conn(0x7f58b40a2970 0x7f58b40a4e30 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f58c4005fd0 tx=0x7f58c4009500 comp rx=0 tx=0).ready entity=mgr.15522 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:10:11.284 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:11.283+0000 7f58d61c1740 1 -- start start 2024-03-28T18:10:11.284 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:11.283+0000 7f58d61c1740 1 Processor -- start 2024-03-28T18:10:11.286 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:11.284+0000 7f58d61c1740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56455e5ce8b0 0x56455e5d7680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:10:11.286 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:11.284+0000 7f58d61c1740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56455e5cf260 0x56455e5d4a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:10:11.286 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:11.284+0000 7f58d61c1740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56455e5d7de0 0x56455e5d4f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:10:11.286 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:11.284+0000 7f58d61c1740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x56455e5dad50 con 0x56455e5d7de0 2024-03-28T18:10:11.286 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:11.284+0000 7f58d61c1740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x56455e4ee0d0 con 0x56455e5cf260 2024-03-28T18:10:11.286 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:11.284+0000 7f58d61c1740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x56455e5daed0 con 0x56455e5ce8b0 2024-03-28T18:10:11.286 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:11.284+0000 7f58d61c1740 10 client.0 _ll_register_callbacks cb 0x56455e4f2e80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-28T18:10:11.287 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:11.284+0000 7f58d1400640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56455e5d7de0 0x56455e5d4f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:10:11.287 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:11.284+0000 7f58cbe00640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56455e5cf260 0x56455e5d4a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:10:11.287 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:11.284+0000 7f58d1400640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56455e5d7de0 0x56455e5d4f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:46134/0 (socket says 172.21.15.22:46134) 2024-03-28T18:10:11.287 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:11.284+0000 7f58cbe00640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56455e5cf260 0x56455e5d4a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.22:52812/0 (socket says 172.21.15.22:52812) 2024-03-28T18:10:11.287 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:11.284+0000 7f58d1400640 1 -- 172.21.15.22:0/1023038954 learned_addr learned my addr 172.21.15.22:0/1023038954 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-28T18:10:11.287 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:11.284+0000 7f58d0a00640 1 --2- 172.21.15.22:0/1023038954 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56455e5ce8b0 0x56455e5d7680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:10:11.287 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:11.285+0000 7f58cbe00640 1 -- 172.21.15.22:0/1023038954 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56455e5ce8b0 msgr2=0x56455e5d7680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T18:10:11.287 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:11.285+0000 7f58cbe00640 1 --2- 172.21.15.22:0/1023038954 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56455e5ce8b0 0x56455e5d7680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:10:11.287 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:11.285+0000 7f58cbe00640 1 -- 172.21.15.22:0/1023038954 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56455e5d7de0 msgr2=0x56455e5d4f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T18:10:11.287 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:11.285+0000 7f58cbe00640 1 --2- 172.21.15.22:0/1023038954 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56455e5d7de0 0x56455e5d4f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:10:11.287 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:11.285+0000 7f58cbe00640 1 -- 172.21.15.22:0/1023038954 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56455e2463f0 con 0x56455e5cf260 2024-03-28T18:10:11.287 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:11.285+0000 7f58d1400640 1 --2- 172.21.15.22:0/1023038954 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56455e5d7de0 0x56455e5d4f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-28T18:10:11.287 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:11.285+0000 7f58cbe00640 1 --2- 172.21.15.22:0/1023038954 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56455e5cf260 0x56455e5d4a10 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f58b800a670 tx=0x7f58b800aa20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=63447d9cbe1512ce server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:10:11.287 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:11.285+0000 7f58b2000640 10 client.0 ms_handle_connect on v2:172.21.15.79:3300/0 2024-03-28T18:10:11.288 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:11.286+0000 7f58b2000640 1 -- 172.21.15.22:0/1023038954 <== mon.2 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58b8027020 con 0x56455e5cf260 2024-03-28T18:10:11.288 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:11.286+0000 7f58d61c1740 1 -- 172.21.15.22:0/1023038954 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f58b8018b10 con 0x56455e5cf260 2024-03-28T18:10:11.288 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:11.286+0000 7f58b2000640 1 -- 172.21.15.22:0/1023038954 <== mon.2 v2:172.21.15.79:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 8411+0+0 (secure 0 0 0) 0x7f58b8010b30 con 0x56455e5cf260 2024-03-28T18:10:11.288 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/scsi/aacraid.txt 2024-03-28T18:10:11.290 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:11.286+0000 7f58b2000640 1 -- 172.21.15.22:0/1023038954 <== mon.2 v2:172.21.15.79:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) v2 ==== 24+0+0 (secure 0 0 0) 0x7f58b8032050 con 0x56455e5cf260 2024-03-28T18:10:11.290 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:11.286+0000 7f58d61c1740 10 client.36146 fetch_fsmap learned FSMap version 31 2024-03-28T18:10:11.290 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:11.286+0000 7f58d61c1740 1 -- 172.21.15.22:0/1023038954 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) v3 -- 0x56455e3107d0 con 0x56455e5cf260 2024-03-28T18:10:11.290 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:11.287+0000 7f58b2000640 1 -- 172.21.15.22:0/1023038954 <== mon.2 v2:172.21.15.79:3300/0 4 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f58b8018b10 con 0x56455e5cf260 2024-03-28T18:10:11.290 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:11.287+0000 7f58b2000640 1 -- 172.21.15.22:0/1023038954 <== mon.2 v2:172.21.15.79:3300/0 5 ==== fsmap(e 31) v1 ==== 7950+0+0 (secure 0 0 0) 0x7f58b8005390 con 0x56455e5cf260 2024-03-28T18:10:11.290 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:11.287+0000 7f58d61c1740 10 client.36146 fetch_fsmap finished waiting for FSMap version 31 2024-03-28T18:10:11.290 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:11.287+0000 7f58d61c1740 10 client.36146 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-28T18:10:11.290 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:11.287+0000 7f58d61c1740 20 client.36146 populate_metadata read hostname 'smithi022' 2024-03-28T18:10:11.290 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:11.287+0000 7f58d61c1740 1 --2- 172.21.15.22:0/1023038954 >> [v2:172.21.15.79:6838/2441542443,v1:172.21.15.79:6839/2441542443] conn(0x56455e5e7a90 0x56455e5e9e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:10:11.290 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:11.287+0000 7f58d61c1740 4 client.36146 mds_command: new command op to 34553 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-28T18:10:11.291 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:11.287+0000 7f58d61c1740 1 -- 172.21.15.22:0/1023038954 --> [v2:172.21.15.79:6838/2441542443,v1:172.21.15.79:6839/2441542443] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56455e53f800 con 0x56455e5e7a90 2024-03-28T18:10:11.291 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:11.288+0000 7f58d0a00640 1 --2- 172.21.15.22:0/1023038954 >> [v2:172.21.15.79:6838/2441542443,v1:172.21.15.79:6839/2441542443] conn(0x56455e5e7a90 0x56455e5e9e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:10:11.291 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:11.288+0000 7f58b2000640 10 client.36146 ms_handle_connect on v2:172.21.15.79:6838/2441542443 2024-03-28T18:10:11.291 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:11.288+0000 7f58d0a00640 1 --2- 172.21.15.22:0/1023038954 >> [v2:172.21.15.79:6838/2441542443,v1:172.21.15.79:6839/2441542443] conn(0x56455e5e7a90 0x56455e5e9e80 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a96b85937e71d21c server_cookie=c8ff623849186ad9 in_seq=0 out_seq=0 2024-03-28T18:10:11.291 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:11.290+0000 7f58b2000640 1 -- 172.21.15.22:0/1023038954 <== mds.0 v2:172.21.15.79:6838/2441542443 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x56455e53f800 con 0x56455e5e7a90 2024-03-28T18:10:11.291 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:11.290+0000 7f58b2000640 10 client.36146 handle_command_reply: tid=0 2024-03-28T18:10:11.291 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:11.290+0000 7f58d61c1740 1 client.36146 shutdown 2024-03-28T18:10:11.291 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:11.290+0000 7f58d61c1740 1 -- 172.21.15.22:0/1023038954 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56455e5cf260 msgr2=0x56455e5d4a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:10:11.291 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:11.290+0000 7f58d61c1740 1 --2- 172.21.15.22:0/1023038954 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56455e5cf260 0x56455e5d4a10 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f58b800a670 tx=0x7f58b800aa20 comp rx=0 tx=0).stop 2024-03-28T18:10:11.291 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:11.290+0000 7f58d61c1740 1 -- 172.21.15.22:0/1023038954 shutdown_connections 2024-03-28T18:10:11.291 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:11.290+0000 7f58d61c1740 1 --2- 172.21.15.22:0/1023038954 >> [v2:172.21.15.79:6838/2441542443,v1:172.21.15.79:6839/2441542443] conn(0x56455e5e7a90 0x56455e5e9e80 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:10:11.291 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:11.290+0000 7f58d61c1740 1 --2- 172.21.15.22:0/1023038954 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56455e5d7de0 0x56455e5d4f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:10:11.291 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:11.290+0000 7f58d61c1740 1 --2- 172.21.15.22:0/1023038954 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56455e5cf260 0x56455e5d4a10 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:10:11.291 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:11.290+0000 7f58d61c1740 1 --2- 172.21.15.22:0/1023038954 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56455e5ce8b0 0x56455e5d7680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:10:11.291 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:11.290+0000 7f58d61c1740 1 -- 172.21.15.22:0/1023038954 >> 172.21.15.22:0/1023038954 conn(0x56455e5c8f80 msgr2=0x56455e5cacd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:10:11.291 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:11.290+0000 7f58b2000640 0 client.36146 ms_handle_reset on v2:172.21.15.79:6838/2441542443 2024-03-28T18:10:11.291 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:11.290+0000 7f58d61c1740 1 -- 172.21.15.22:0/1023038954 shutdown_connections 2024-03-28T18:10:11.292 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:11.290+0000 7f58d61c1740 1 -- 172.21.15.22:0/1023038954 wait complete. 2024-03-28T18:10:11.292 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:11.291+0000 7f58d61c1740 20 client.36146 trim_cache size 0 max 16384 2024-03-28T18:10:11.295 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/scsi/advansys.txt 2024-03-28T18:10:11.305 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/scsi/aha152x.txt 2024-03-28T18:10:11.306 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:11.307+0000 7f58d61c1740 1 Processor -- start 2024-03-28T18:10:11.307 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:11.307+0000 7f58d61c1740 1 -- start start 2024-03-28T18:10:11.308 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:11.307+0000 7f58d61c1740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56455e5ce8b0 0x56455e5d7140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:10:11.308 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:11.308+0000 7f58d61c1740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56455e5d7de0 0x56455e5d7680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:10:11.308 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:11.308+0000 7f58d61c1740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56455e5e0140 0x56455e5d1c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:10:11.308 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:11.308+0000 7f58d61c1740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58cc1aefa0 con 0x56455e5ce8b0 2024-03-28T18:10:11.308 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:11.308+0000 7f58d61c1740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58cc1b09e0 con 0x56455e5d7de0 2024-03-28T18:10:11.308 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:11.308+0000 7f58d61c1740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58cc1af600 con 0x56455e5e0140 2024-03-28T18:10:11.309 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:11.308+0000 7f58d0a00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56455e5ce8b0 0x56455e5d7140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:10:11.309 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:11.308+0000 7f58cbe00640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56455e5d7de0 0x56455e5d7680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:10:11.309 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:11.308+0000 7f58d0a00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56455e5ce8b0 0x56455e5d7140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:46144/0 (socket says 172.21.15.22:46144) 2024-03-28T18:10:11.309 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:11.308+0000 7f58d0a00640 1 -- 172.21.15.22:0/1882839081 learned_addr learned my addr 172.21.15.22:0/1882839081 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-28T18:10:11.309 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:11.308+0000 7f58d1400640 1 --2- 172.21.15.22:0/1882839081 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56455e5e0140 0x56455e5d1c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:10:11.309 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:11.308+0000 7f58cbe00640 1 -- 172.21.15.22:0/1882839081 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56455e5e0140 msgr2=0x56455e5d1c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T18:10:11.309 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:11.308+0000 7f58cbe00640 1 --2- 172.21.15.22:0/1882839081 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56455e5e0140 0x56455e5d1c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:10:11.309 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:11.308+0000 7f58cbe00640 1 -- 172.21.15.22:0/1882839081 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56455e5ce8b0 msgr2=0x56455e5d7140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T18:10:11.309 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:11.308+0000 7f58cbe00640 1 --2- 172.21.15.22:0/1882839081 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56455e5ce8b0 0x56455e5d7140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:10:11.309 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:11.308+0000 7f58cbe00640 1 -- 172.21.15.22:0/1882839081 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56455e5da1d0 con 0x56455e5d7de0 2024-03-28T18:10:11.309 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:11.308+0000 7f58d0a00640 1 --2- 172.21.15.22:0/1882839081 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56455e5ce8b0 0x56455e5d7140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-28T18:10:11.309 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:11.309+0000 7f58cbe00640 1 --2- 172.21.15.22:0/1882839081 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56455e5d7de0 0x56455e5d7680 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x56455e5d5aa0 tx=0x7f58b80049b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f2fddf6aa6b9dfab server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:10:11.311 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:11.309+0000 7f58bd200640 1 -- 172.21.15.22:0/1882839081 <== mon.2 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58b803a070 con 0x56455e5d7de0 2024-03-28T18:10:11.311 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:11.309+0000 7f58bd200640 1 -- 172.21.15.22:0/1882839081 <== mon.2 v2:172.21.15.79:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f58b8010d70 con 0x56455e5d7de0 2024-03-28T18:10:11.311 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:11.309+0000 7f58bd200640 1 -- 172.21.15.22:0/1882839081 <== mon.2 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58b8027070 con 0x56455e5d7de0 2024-03-28T18:10:11.311 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:11.310+0000 7f58d61c1740 1 -- 172.21.15.22:0/1882839081 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56455e5d7de0 msgr2=0x56455e5d7680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:10:11.311 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:11.310+0000 7f58d61c1740 1 --2- 172.21.15.22:0/1882839081 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56455e5d7de0 0x56455e5d7680 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x56455e5d5aa0 tx=0x7f58b80049b0 comp rx=0 tx=0).stop 2024-03-28T18:10:11.311 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:11.310+0000 7f58d61c1740 1 -- 172.21.15.22:0/1882839081 shutdown_connections 2024-03-28T18:10:11.311 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:11.310+0000 7f58d61c1740 1 --2- 172.21.15.22:0/1882839081 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56455e5e0140 0x56455e5d1c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:10:11.311 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:11.310+0000 7f58d61c1740 1 --2- 172.21.15.22:0/1882839081 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56455e5d7de0 0x56455e5d7680 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:10:11.311 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:11.310+0000 7f58d61c1740 1 --2- 172.21.15.22:0/1882839081 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56455e5ce8b0 0x56455e5d7140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:10:11.311 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:11.310+0000 7f58d61c1740 1 -- 172.21.15.22:0/1882839081 >> 172.21.15.22:0/1882839081 conn(0x56455e5c8f80 msgr2=0x56455e5c8160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:10:11.311 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:11.310+0000 7f58d61c1740 1 -- 172.21.15.22:0/1882839081 shutdown_connections 2024-03-28T18:10:11.311 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:11.310+0000 7f58d61c1740 1 -- 172.21.15.22:0/1882839081 wait complete. 2024-03-28T18:10:11.311 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:11.311+0000 7f58d61c1740 1 -- start start 2024-03-28T18:10:11.312 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:11.311+0000 7f58d61c1740 1 Processor -- start 2024-03-28T18:10:11.314 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:11.312+0000 7f58d61c1740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56455e5ce8b0 0x56455e5dd490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:10:11.314 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:11.312+0000 7f58d61c1740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56455e5d7de0 0x56455e5dd9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:10:11.314 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:11.312+0000 7f58d61c1740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56455e5e0140 0x56455e642200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:10:11.314 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:11.312+0000 7f58d61c1740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58cc1b02c0 con 0x56455e5e0140 2024-03-28T18:10:11.315 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:11.312+0000 7f58d61c1740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58cc1afc60 con 0x56455e5d7de0 2024-03-28T18:10:11.315 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:11.312+0000 7f58d61c1740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58cc1ad160 con 0x56455e5ce8b0 2024-03-28T18:10:11.315 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:11.312+0000 7f58d61c1740 10 client.0 _ll_register_callbacks cb 0x7f58b801b070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-28T18:10:11.315 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:11.312+0000 7f58d1400640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56455e5e0140 0x56455e642200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:10:11.315 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:11.312+0000 7f58cbe00640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56455e5d7de0 0x56455e5dd9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:10:11.315 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:11.312+0000 7f58cbe00640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56455e5d7de0 0x56455e5dd9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.22:52828/0 (socket says 172.21.15.22:52828) 2024-03-28T18:10:11.315 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:11.312+0000 7f58cbe00640 1 -- 172.21.15.22:0/1899131552 learned_addr learned my addr 172.21.15.22:0/1899131552 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-28T18:10:11.315 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:11.312+0000 7f58d0a00640 1 --2- 172.21.15.22:0/1899131552 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56455e5ce8b0 0x56455e5dd490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:10:11.315 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:11.312+0000 7f58d1400640 1 -- 172.21.15.22:0/1899131552 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56455e5ce8b0 msgr2=0x56455e5dd490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T18:10:11.315 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:11.312+0000 7f58d1400640 1 --2- 172.21.15.22:0/1899131552 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56455e5ce8b0 0x56455e5dd490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:10:11.315 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:11.312+0000 7f58d1400640 1 -- 172.21.15.22:0/1899131552 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56455e5d7de0 msgr2=0x56455e5dd9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T18:10:11.315 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:11.312+0000 7f58d1400640 1 --2- 172.21.15.22:0/1899131552 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56455e5d7de0 0x56455e5dd9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:10:11.315 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:11.312+0000 7f58d1400640 1 -- 172.21.15.22:0/1899131552 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56455e642740 con 0x56455e5e0140 2024-03-28T18:10:11.315 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:11.312+0000 7f58cbe00640 1 --2- 172.21.15.22:0/1899131552 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56455e5d7de0 0x56455e5dd9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-28T18:10:11.316 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:11.313+0000 7f58d1400640 1 --2- 172.21.15.22:0/1899131552 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56455e5e0140 0x56455e642200 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x56455e5d3ee0 tx=0x7f58c4004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=553694cc0fecc4f9 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:10:11.316 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:11.313+0000 7f58b2000640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-03-28T18:10:11.316 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:11.313+0000 7f58b2000640 1 -- 172.21.15.22:0/1899131552 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58c402d060 con 0x56455e5e0140 2024-03-28T18:10:11.316 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:11.313+0000 7f58b2000640 1 -- 172.21.15.22:0/1899131552 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 8411+0+0 (secure 0 0 0) 0x7f58c401eb00 con 0x56455e5e0140 2024-03-28T18:10:11.316 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:11.313+0000 7f58d61c1740 1 -- 172.21.15.22:0/1899131552 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f58b8010d70 con 0x56455e5e0140 2024-03-28T18:10:11.316 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:11.313+0000 7f58b2000640 1 -- 172.21.15.22:0/1899131552 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) v2 ==== 24+0+0 (secure 0 0 0) 0x7f58c401ed00 con 0x56455e5e0140 2024-03-28T18:10:11.316 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:11.314+0000 7f58d61c1740 10 client.16323 fetch_fsmap learned FSMap version 31 2024-03-28T18:10:11.316 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:11.314+0000 7f58d61c1740 1 -- 172.21.15.22:0/1899131552 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) v3 -- 0x56455e643940 con 0x56455e5e0140 2024-03-28T18:10:11.316 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:11.314+0000 7f58b2000640 1 -- 172.21.15.22:0/1899131552 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f58b8010d70 con 0x56455e5e0140 2024-03-28T18:10:11.316 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:11.314+0000 7f58b2000640 1 -- 172.21.15.22:0/1899131552 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 31) v1 ==== 7950+0+0 (secure 0 0 0) 0x7f58c4030020 con 0x56455e5e0140 2024-03-28T18:10:11.316 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:11.314+0000 7f58d61c1740 10 client.16323 fetch_fsmap finished waiting for FSMap version 31 2024-03-28T18:10:11.316 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:11.314+0000 7f58d61c1740 10 client.16323 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-28T18:10:11.316 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:11.314+0000 7f58d61c1740 20 client.16323 populate_metadata read hostname 'smithi022' 2024-03-28T18:10:11.316 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:11.314+0000 7f58d61c1740 1 --2- 172.21.15.22:0/1899131552 >> [v2:172.21.15.79:6838/2441542443,v1:172.21.15.79:6839/2441542443] conn(0x56455e644130 0x56455e646550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:10:11.316 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:11.314+0000 7f58d61c1740 4 client.16323 mds_command: new command op to 34553 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-28T18:10:11.317 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:11.314+0000 7f58d61c1740 1 -- 172.21.15.22:0/1899131552 --> [v2:172.21.15.79:6838/2441542443,v1:172.21.15.79:6839/2441542443] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x56455e539b50 con 0x56455e644130 2024-03-28T18:10:11.317 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:11.315+0000 7f58d0a00640 1 --2- 172.21.15.22:0/1899131552 >> [v2:172.21.15.79:6838/2441542443,v1:172.21.15.79:6839/2441542443] conn(0x56455e644130 0x56455e646550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:10:11.318 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:11.315+0000 7f58d0a00640 1 --2- 172.21.15.22:0/1899131552 >> [v2:172.21.15.79:6838/2441542443,v1:172.21.15.79:6839/2441542443] conn(0x56455e644130 0x56455e646550 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4468f930fddd67ef server_cookie=a7f637e90bcb1dd4 in_seq=0 out_seq=0 2024-03-28T18:10:11.318 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:11.315+0000 7f58b2000640 10 client.16323 ms_handle_connect on v2:172.21.15.79:6838/2441542443 2024-03-28T18:10:11.318 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:11.316+0000 7f58b2000640 1 -- 172.21.15.22:0/1899131552 <== mds.0 v2:172.21.15.79:6838/2441542443 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x56455e539b50 con 0x56455e644130 2024-03-28T18:10:11.318 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:11.316+0000 7f58b2000640 10 client.16323 handle_command_reply: tid=0 2024-03-28T18:10:11.318 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:11.316+0000 7f58d61c1740 1 client.16323 shutdown 2024-03-28T18:10:11.318 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:11.316+0000 7f58d61c1740 1 -- 172.21.15.22:0/1899131552 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56455e5e0140 msgr2=0x56455e642200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:10:11.318 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:11.316+0000 7f58d61c1740 1 --2- 172.21.15.22:0/1899131552 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56455e5e0140 0x56455e642200 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x56455e5d3ee0 tx=0x7f58c4004a40 comp rx=0 tx=0).stop 2024-03-28T18:10:11.318 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:11.316+0000 7f58d61c1740 1 -- 172.21.15.22:0/1899131552 shutdown_connections 2024-03-28T18:10:11.318 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:11.316+0000 7f58d61c1740 1 --2- 172.21.15.22:0/1899131552 >> [v2:172.21.15.79:6838/2441542443,v1:172.21.15.79:6839/2441542443] conn(0x56455e644130 0x56455e646550 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:10:11.318 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:11.316+0000 7f58d61c1740 1 --2- 172.21.15.22:0/1899131552 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56455e5e0140 0x56455e642200 unknown :-1 s=CLOSED pgs=648 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:10:11.318 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:11.316+0000 7f58d61c1740 1 --2- 172.21.15.22:0/1899131552 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56455e5d7de0 0x56455e5dd9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:10:11.318 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:11.316+0000 7f58d61c1740 1 --2- 172.21.15.22:0/1899131552 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56455e5ce8b0 0x56455e5dd490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:10:11.319 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:11.316+0000 7f58d61c1740 1 -- 172.21.15.22:0/1899131552 >> 172.21.15.22:0/1899131552 conn(0x56455e5c8f80 msgr2=0x56455e5c8160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:10:11.319 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:11.316+0000 7f58b2000640 0 client.16323 ms_handle_reset on v2:172.21.15.79:6838/2441542443 2024-03-28T18:10:11.319 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/scsi/aic79xx.txt 2024-03-28T18:10:11.319 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:11.316+0000 7f58d61c1740 1 -- 172.21.15.22:0/1899131552 shutdown_connections 2024-03-28T18:10:11.319 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:11.316+0000 7f58d61c1740 1 -- 172.21.15.22:0/1899131552 wait complete. 2024-03-28T18:10:11.319 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:11.317+0000 7f58d61c1740 20 client.16323 trim_cache size 0 max 16384 2024-03-28T18:10:11.320 INFO:teuthology.orchestra.run.smithi022.stdout:[] 2024-03-28T18:10:11.321 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:11.317+0000 7f58d1e00640 1 -- 172.21.15.22:0/2846192617 >> [v2:172.21.15.22:6800/1357365082,v1:172.21.15.22:6801/1357365082] conn(0x7f58b40a2970 msgr2=0x7f58b40a4e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:10:11.321 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:11.317+0000 7f58d1e00640 1 --2- 172.21.15.22:0/2846192617 >> [v2:172.21.15.22:6800/1357365082,v1:172.21.15.22:6801/1357365082] conn(0x7f58b40a2970 0x7f58b40a4e30 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f58c4005fd0 tx=0x7f58c4009500 comp rx=0 tx=0).stop 2024-03-28T18:10:11.321 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:11.317+0000 7f58d1e00640 1 -- 172.21.15.22:0/2846192617 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f58cc100400 msgr2=0x7f58cc102060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:10:11.321 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:11.317+0000 7f58d1e00640 1 --2- 172.21.15.22:0/2846192617 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f58cc100400 0x7f58cc102060 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7f58c0000c00 tx=0x7f58c0004430 comp rx=0 tx=0).stop 2024-03-28T18:10:11.321 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:11.318+0000 7f58d1e00640 1 -- 172.21.15.22:0/2846192617 shutdown_connections 2024-03-28T18:10:11.321 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:11.318+0000 7f58d1e00640 1 --2- 172.21.15.22:0/2846192617 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f58cc1064a0 0x7f58cc102ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:10:11.321 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:11.318+0000 7f58d1e00640 1 --2- 172.21.15.22:0/2846192617 >> [v2:172.21.15.22:6800/1357365082,v1:172.21.15.22:6801/1357365082] conn(0x7f58b40a2970 0x7f58b40a4e30 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:10:11.321 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:11.318+0000 7f58d1e00640 1 --2- 172.21.15.22:0/2846192617 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f58cc100dc0 0x7f58cc1025a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:10:11.321 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:11.318+0000 7f58d1e00640 1 --2- 172.21.15.22:0/2846192617 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f58cc100400 0x7f58cc102060 unknown :-1 s=CLOSED pgs=647 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:10:11.321 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:11.318+0000 7f58d1e00640 1 -- 172.21.15.22:0/2846192617 >> 172.21.15.22:0/2846192617 conn(0x7f58cc0fbef0 msgr2=0x7f58cc104340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:10:11.321 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:11.318+0000 7f58d1e00640 1 -- 172.21.15.22:0/2846192617 shutdown_connections 2024-03-28T18:10:11.321 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:11.318+0000 7f58d1e00640 1 -- 172.21.15.22:0/2846192617 wait complete. 2024-03-28T18:10:11.325 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/scsi/aic7xxx.txt 2024-03-28T18:10:11.336 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/scsi/arcmsr_spec.txt 2024-03-28T18:10:11.347 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/scsi/bfa.txt 2024-03-28T18:10:11.360 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/scsi/bnx2fc.txt 2024-03-28T18:10:11.373 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/scsi/cxgb3i.txt 2024-03-28T18:10:11.389 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/scsi/dc395x.txt 2024-03-28T18:10:11.404 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/scsi/dpti.txt 2024-03-28T18:10:11.419 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/scsi/dtc3x80.txt 2024-03-28T18:10:11.434 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/scsi/g_NCR5380.txt 2024-03-28T18:10:11.445 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/scsi/hpsa.txt 2024-03-28T18:10:11.467 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/scsi/hptiop.txt 2024-03-28T18:10:11.477 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/scsi/in2000.txt 2024-03-28T18:10:11.496 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/scsi/libsas.txt 2024-03-28T18:10:11.514 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/scsi/link_power_management_policy.txt 2024-03-28T18:10:11.529 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/scsi/lpfc.txt 2024-03-28T18:10:11.554 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/scsi/megaraid.txt 2024-03-28T18:10:11.597 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/scsi/ncr53c8xx.txt 2024-03-28T18:10:11.631 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/scsi/osd.txt 2024-03-28T18:10:11.647 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/scsi/osst.txt 2024-03-28T18:10:11.664 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/scsi/ppa.txt 2024-03-28T18:10:11.679 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/scsi/qlogicfas.txt 2024-03-28T18:10:11.693 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/scsi/scsi-changer.txt 2024-03-28T18:10:11.707 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/scsi/scsi-generic.txt 2024-03-28T18:10:11.724 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/scsi/scsi-parameters.txt 2024-03-28T18:10:11.738 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/scsi/scsi.txt 2024-03-28T18:10:11.754 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/scsi/scsi_eh.txt 2024-03-28T18:10:11.772 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/scsi/scsi_fc_transport.txt 2024-03-28T18:10:11.791 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/scsi/scsi_mid_low_api.txt 2024-03-28T18:10:11.814 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/scsi/scsi_transport_srp/ 2024-03-28T18:10:11.826 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/scsi/scsi_transport_srp/Makefile 2024-03-28T18:10:11.836 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/scsi/scsi_transport_srp/rport_state_diagram.dot 2024-03-28T18:10:11.853 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/scsi/st.txt 2024-03-28T18:10:11.876 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/scsi/sym53c500_cs.txt 2024-03-28T18:10:11.893 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/scsi/sym53c8xx_2.txt 2024-03-28T18:10:11.914 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/scsi/tmscsim.txt 2024-03-28T18:10:11.930 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/scsi/ufs.txt 2024-03-28T18:10:11.946 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/scsi/wd719x.txt 2024-03-28T18:10:11.961 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/security/ 2024-03-28T18:10:11.976 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/security/00-INDEX 2024-03-28T18:10:11.984 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/security/IMA-templates.txt 2024-03-28T18:10:12.003 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/security/LSM.txt 2024-03-28T18:10:12.018 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/security/SELinux.txt 2024-03-28T18:10:12.032 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/security/Smack.txt 2024-03-28T18:10:12.051 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/security/Yama.txt 2024-03-28T18:10:12.070 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/security/apparmor.txt 2024-03-28T18:10:12.085 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/security/credentials.txt 2024-03-28T18:10:12.104 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/security/keys-ecryptfs.txt 2024-03-28T18:10:12.119 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/security/keys-request-key.txt 2024-03-28T18:10:12.136 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/security/keys-trusted-encrypted.txt 2024-03-28T18:10:12.153 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/security/keys.txt 2024-03-28T18:10:12.175 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/security/tomoyo.txt 2024-03-28T18:10:12.189 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/serial-console.txt 2024-03-28T18:10:12.207 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/serial/ 2024-03-28T18:10:12.219 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/serial/00-INDEX 2024-03-28T18:10:12.227 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/serial/README.cycladesZ 2024-03-28T18:10:12.243 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/serial/driver 2024-03-28T18:10:12.261 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/serial/moxa-smartio 2024-03-28T18:10:12.279 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/serial/n_gsm.txt 2024-03-28T18:10:12.289 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/serial/rocket.txt 2024-03-28T18:10:12.304 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/serial/serial-rs485.txt 2024-03-28T18:10:12.319 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/serial/tty.txt 2024-03-28T18:10:12.334 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/sgi-ioc4.txt 2024-03-28T18:10:12.337 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-03-28T18:10:12.337 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-28T18:10:12.337 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-03-28T18:10:12.356 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/sh/ 2024-03-28T18:10:12.366 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/sh/new-machine.txt 2024-03-28T18:10:12.375 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/sh/register-banks.txt 2024-03-28T18:10:12.387 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/smsc_ece1099.txt 2024-03-28T18:10:12.401 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/sound/ 2024-03-28T18:10:12.409 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/sound/alsa/ 2024-03-28T18:10:12.415 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/sound/alsa/ALSA-Configuration.txt 2024-03-28T18:10:12.424 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:10:12 smithi134 ceph-mon[35037]: pgmap v82: 97 pgs: 97 active+clean; 111 MiB data, 809 MiB used, 1.0 TiB / 1.0 TiB avail; 571 KiB/s rd, 1.1 MiB/s wr, 187 op/s 2024-03-28T18:10:12.429 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:10:12 smithi022 ceph-mon[31985]: pgmap v82: 97 pgs: 97 active+clean; 111 MiB data, 809 MiB used, 1.0 TiB / 1.0 TiB avail; 571 KiB/s rd, 1.1 MiB/s wr, 187 op/s 2024-03-28T18:10:12.432 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:10:12 smithi079 ceph-mon[35312]: pgmap v82: 97 pgs: 97 active+clean; 111 MiB data, 809 MiB used, 1.0 TiB / 1.0 TiB avail; 571 KiB/s rd, 1.1 MiB/s wr, 187 op/s 2024-03-28T18:10:12.434 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/sound/alsa/Audigy-mixer.txt 2024-03-28T18:10:12.451 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/sound/alsa/Audiophile-Usb.txt 2024-03-28T18:10:12.467 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/sound/alsa/Bt87x.txt 2024-03-28T18:10:12.480 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/sound/alsa/CMIPCI.txt 2024-03-28T18:10:12.495 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/sound/alsa/Channel-Mapping-API.txt 2024-03-28T18:10:12.511 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/sound/alsa/ControlNames.txt 2024-03-28T18:10:12.525 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/sound/alsa/HD-Audio-Controls.txt 2024-03-28T18:10:12.539 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/sound/alsa/HD-Audio-Models.txt 2024-03-28T18:10:12.561 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/sound/alsa/HD-Audio.txt 2024-03-28T18:10:12.591 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:12.591+0000 7fe9dfe00640 1 -- 172.21.15.22:0/3049693237 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe9e01026d0 msgr2=0x7fe9e0102ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:10:12.591 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:12.591+0000 7fe9dfe00640 1 --2- 172.21.15.22:0/3049693237 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe9e01026d0 0x7fe9e0102ab0 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7fe9d0009900 tx=0x7fe9d002bcd0 comp rx=0 tx=0).stop 2024-03-28T18:10:12.592 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:12.591+0000 7fe9dfe00640 1 -- 172.21.15.22:0/3049693237 shutdown_connections 2024-03-28T18:10:12.592 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:12.591+0000 7fe9dfe00640 1 --2- 172.21.15.22:0/3049693237 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fe9e010baf0 0x7fe9e010df90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:10:12.592 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:12.591+0000 7fe9dfe00640 1 --2- 172.21.15.22:0/3049693237 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe9e0102ff0 0x7fe9e010b4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:10:12.592 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:12.591+0000 7fe9dfe00640 1 --2- 172.21.15.22:0/3049693237 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe9e01026d0 0x7fe9e0102ab0 unknown :-1 s=CLOSED pgs=649 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:10:12.592 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:12.591+0000 7fe9dfe00640 1 -- 172.21.15.22:0/3049693237 >> 172.21.15.22:0/3049693237 conn(0x7fe9e00fbef0 msgr2=0x7fe9e00fe310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:10:12.592 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:12.592+0000 7fe9dfe00640 1 -- 172.21.15.22:0/3049693237 shutdown_connections 2024-03-28T18:10:12.592 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:12.592+0000 7fe9dfe00640 1 -- 172.21.15.22:0/3049693237 wait complete. 2024-03-28T18:10:12.593 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:12.593+0000 7fe9dfe00640 1 Processor -- start 2024-03-28T18:10:12.593 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:12.593+0000 7fe9dfe00640 1 -- start start 2024-03-28T18:10:12.593 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:12.593+0000 7fe9dfe00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe9e01026d0 0x7fe9e019a050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:10:12.593 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:12.593+0000 7fe9dfe00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe9e0102ff0 0x7fe9e019a590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:10:12.593 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:12.593+0000 7fe9dfe00640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fe9e010baf0 0x7fe9e019e910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:10:12.593 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:12.593+0000 7fe9dfe00640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9e0111910 con 0x7fe9e0102ff0 2024-03-28T18:10:12.593 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:12.593+0000 7fe9dfe00640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9e0111790 con 0x7fe9e010baf0 2024-03-28T18:10:12.593 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:12.593+0000 7fe9dfe00640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9e0111a90 con 0x7fe9e01026d0 2024-03-28T18:10:12.594 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:12.594+0000 7fe9d6000640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe9e0102ff0 0x7fe9e019a590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:10:12.595 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:12.594+0000 7fe9d6000640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe9e0102ff0 0x7fe9e019a590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:46180/0 (socket says 172.21.15.22:46180) 2024-03-28T18:10:12.595 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:12.594+0000 7fe9d6000640 1 -- 172.21.15.22:0/1444360711 learned_addr learned my addr 172.21.15.22:0/1444360711 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-28T18:10:12.595 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:12.594+0000 7fe9df400640 1 --2- 172.21.15.22:0/1444360711 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fe9e010baf0 0x7fe9e019e910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:10:12.595 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:12.594+0000 7fe9dea00640 1 --2- 172.21.15.22:0/1444360711 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe9e01026d0 0x7fe9e019a050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:10:12.595 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:12.594+0000 7fe9d6000640 1 -- 172.21.15.22:0/1444360711 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe9e01026d0 msgr2=0x7fe9e019a050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:10:12.595 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:12.594+0000 7fe9d6000640 1 --2- 172.21.15.22:0/1444360711 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe9e01026d0 0x7fe9e019a050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:10:12.595 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:12.594+0000 7fe9d6000640 1 -- 172.21.15.22:0/1444360711 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fe9e010baf0 msgr2=0x7fe9e019e910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:10:12.595 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:12.594+0000 7fe9d6000640 1 --2- 172.21.15.22:0/1444360711 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fe9e010baf0 0x7fe9e019e910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:10:12.595 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:12.594+0000 7fe9d6000640 1 -- 172.21.15.22:0/1444360711 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe9e019f040 con 0x7fe9e0102ff0 2024-03-28T18:10:12.596 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:12.595+0000 7fe9d6000640 1 --2- 172.21.15.22:0/1444360711 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe9e0102ff0 0x7fe9e019a590 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7fe9c800d750 tx=0x7fe9c800dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:10:12.596 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:12.595+0000 7fe9d7e00640 1 -- 172.21.15.22:0/1444360711 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9c8017070 con 0x7fe9e0102ff0 2024-03-28T18:10:12.596 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:12.595+0000 7fe9d7e00640 1 -- 172.21.15.22:0/1444360711 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7fe9c8005020 con 0x7fe9e0102ff0 2024-03-28T18:10:12.596 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:12.595+0000 7fe9d7e00640 1 -- 172.21.15.22:0/1444360711 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9c8004d60 con 0x7fe9e0102ff0 2024-03-28T18:10:12.596 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:12.595+0000 7fe9dfe00640 1 -- 172.21.15.22:0/1444360711 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe9e019f330 con 0x7fe9e0102ff0 2024-03-28T18:10:12.596 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:12.595+0000 7fe9dfe00640 1 -- 172.21.15.22:0/1444360711 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe9e01a6bc0 con 0x7fe9e0102ff0 2024-03-28T18:10:12.598 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:12.599+0000 7fe9e899a740 1 Processor -- start 2024-03-28T18:10:12.598 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:12.599+0000 7fe9d7e00640 1 -- 172.21.15.22:0/1444360711 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 26) v1 ==== 137421+0+0 (secure 0 0 0) 0x7fe9c80051d0 con 0x7fe9e0102ff0 2024-03-28T18:10:12.598 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:12.599+0000 7fe9e899a740 1 -- start start 2024-03-28T18:10:12.599 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:12.599+0000 7fe9e899a740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56232f79e290 0x56232f79e670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:10:12.599 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:12.599+0000 7fe9e899a740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56232f79ebb0 0x56232f7a7250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:10:12.599 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:12.599+0000 7fe9e899a740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56232f7a7790 0x56232f7a9b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:10:12.599 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:12.599+0000 7fe9e899a740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x56232f702eb0 con 0x56232f79e290 2024-03-28T18:10:12.600 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:12.599+0000 7fe9e899a740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x56232f6aba60 con 0x56232f7a7790 2024-03-28T18:10:12.600 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:12.599+0000 7fe9e899a740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x56232f6fef50 con 0x56232f79ebb0 2024-03-28T18:10:12.600 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:12.600+0000 7fe9df400640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56232f79e290 0x56232f79e670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:10:12.600 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:12.600+0000 7fe9df400640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56232f79e290 0x56232f79e670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:46190/0 (socket says 172.21.15.22:46190) 2024-03-28T18:10:12.600 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:12.600+0000 7fe9df400640 1 -- 172.21.15.22:0/1382659646 learned_addr learned my addr 172.21.15.22:0/1382659646 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-28T18:10:12.600 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:12.600+0000 7fe9d6000640 1 --2- 172.21.15.22:0/1382659646 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56232f7a7790 0x56232f7a9b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:10:12.600 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:12.600+0000 7fe9dea00640 1 --2- 172.21.15.22:0/1382659646 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56232f79ebb0 0x56232f7a7250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:10:12.601 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:12.600+0000 7fe9df400640 1 -- 172.21.15.22:0/1382659646 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56232f79ebb0 msgr2=0x56232f7a7250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T18:10:12.601 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:12.600+0000 7fe9df400640 1 --2- 172.21.15.22:0/1382659646 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56232f79ebb0 0x56232f7a7250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:10:12.601 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:12.600+0000 7fe9df400640 1 -- 172.21.15.22:0/1382659646 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56232f7a7790 msgr2=0x56232f7a9b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T18:10:12.601 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:12.600+0000 7fe9df400640 1 --2- 172.21.15.22:0/1382659646 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56232f7a7790 0x56232f7a9b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:10:12.601 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:12.600+0000 7fe9df400640 1 -- 172.21.15.22:0/1382659646 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56232f4df7e0 con 0x56232f79e290 2024-03-28T18:10:12.601 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:12.601+0000 7fe9d7e00640 1 --2- 172.21.15.22:0/1444360711 >> [v2:172.21.15.22:6800/1357365082,v1:172.21.15.22:6801/1357365082] conn(0x7fe9c40a29f0 0x7fe9c40a4eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:10:12.601 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:12.601+0000 7fe9df400640 1 --2- 172.21.15.22:0/1382659646 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56232f79e290 0x56232f79e670 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7fe9d8009f90 tx=0x7fe9d8010860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=179f41dbd2bcf77a server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:10:12.602 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:12.601+0000 7fe9d7e00640 1 -- 172.21.15.22:0/1444360711 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 8411+0+0 (secure 0 0 0) 0x7fe9c80ce7a0 con 0x7fe9e0102ff0 2024-03-28T18:10:12.602 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:12.601+0000 7fe9dea00640 1 --2- 172.21.15.22:0/1444360711 >> [v2:172.21.15.22:6800/1357365082,v1:172.21.15.22:6801/1357365082] conn(0x7fe9c40a29f0 0x7fe9c40a4eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:10:12.602 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:12.601+0000 7fe9cfe00640 1 -- 172.21.15.22:0/1382659646 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9d8011700 con 0x56232f79e290 2024-03-28T18:10:12.602 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:12.601+0000 7fe9cfe00640 1 -- 172.21.15.22:0/1382659646 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7fe9d8011d70 con 0x56232f79e290 2024-03-28T18:10:12.602 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:12.601+0000 7fe9cfe00640 1 -- 172.21.15.22:0/1382659646 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9d8019e30 con 0x56232f79e290 2024-03-28T18:10:12.603 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:12.602+0000 7fe9dea00640 1 --2- 172.21.15.22:0/1444360711 >> [v2:172.21.15.22:6800/1357365082,v1:172.21.15.22:6801/1357365082] conn(0x7fe9c40a29f0 0x7fe9c40a4eb0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fe9d0006010 tx=0x7fe9d0005b50 comp rx=0 tx=0).ready entity=mgr.15522 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:10:12.603 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:12.602+0000 7fe9e899a740 1 -- 172.21.15.22:0/1382659646 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56232f79e290 msgr2=0x56232f79e670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:10:12.603 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:12.602+0000 7fe9e899a740 1 --2- 172.21.15.22:0/1382659646 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56232f79e290 0x56232f79e670 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7fe9d8009f90 tx=0x7fe9d8010860 comp rx=0 tx=0).stop 2024-03-28T18:10:12.603 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:12.603+0000 7fe9e899a740 1 -- 172.21.15.22:0/1382659646 shutdown_connections 2024-03-28T18:10:12.603 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:12.603+0000 7fe9e899a740 1 --2- 172.21.15.22:0/1382659646 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56232f7a7790 0x56232f7a9b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:10:12.603 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:12.603+0000 7fe9e899a740 1 --2- 172.21.15.22:0/1382659646 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56232f79ebb0 0x56232f7a7250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:10:12.603 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:12.603+0000 7fe9e899a740 1 --2- 172.21.15.22:0/1382659646 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56232f79e290 0x56232f79e670 unknown :-1 s=CLOSED pgs=651 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:10:12.603 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:12.603+0000 7fe9e899a740 1 -- 172.21.15.22:0/1382659646 >> 172.21.15.22:0/1382659646 conn(0x56232f7989c0 msgr2=0x56232f798da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:10:12.603 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:12.603+0000 7fe9e899a740 1 -- 172.21.15.22:0/1382659646 shutdown_connections 2024-03-28T18:10:12.603 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:12.603+0000 7fe9e899a740 1 -- 172.21.15.22:0/1382659646 wait complete. 2024-03-28T18:10:12.603 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:12.603+0000 7fe9e899a740 1 -- start start 2024-03-28T18:10:12.604 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/sound/alsa/Joystick.txt 2024-03-28T18:10:12.605 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:12.604+0000 7fe9e899a740 1 Processor -- start 2024-03-28T18:10:12.605 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:12.604+0000 7fe9e899a740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56232f79e290 0x56232f7a70e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:10:12.605 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:12.604+0000 7fe9e899a740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56232f79ebb0 0x56232f7a4470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:10:12.605 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:12.604+0000 7fe9e899a740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56232f7a7790 0x56232f7a49d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:10:12.605 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:12.604+0000 7fe9e899a740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x56232f7aa580 con 0x56232f79e290 2024-03-28T18:10:12.605 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:12.604+0000 7fe9e899a740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x56232f6beeb0 con 0x56232f79ebb0 2024-03-28T18:10:12.605 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:12.604+0000 7fe9e899a740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x56232f7aa700 con 0x56232f7a7790 2024-03-28T18:10:12.605 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:12.604+0000 7fe9e899a740 10 client.0 _ll_register_callbacks cb 0x56232f6b3150 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-28T18:10:12.606 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:12.604+0000 7fe9d6000640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56232f79ebb0 0x56232f7a4470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:10:12.606 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:12.605+0000 7fe9d6000640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56232f79ebb0 0x56232f7a4470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.22:52866/0 (socket says 172.21.15.22:52866) 2024-03-28T18:10:12.606 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:12.605+0000 7fe9d6000640 1 -- 172.21.15.22:0/1601695017 learned_addr learned my addr 172.21.15.22:0/1601695017 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-28T18:10:12.606 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:12.605+0000 7fe9dea00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56232f79e290 0x56232f7a70e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:10:12.606 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:12.605+0000 7fe9df400640 1 --2- 172.21.15.22:0/1601695017 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56232f7a7790 0x56232f7a49d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:10:12.606 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:12.605+0000 7fe9d6000640 1 -- 172.21.15.22:0/1601695017 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56232f7a7790 msgr2=0x56232f7a49d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T18:10:12.606 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:12.605+0000 7fe9d6000640 1 --2- 172.21.15.22:0/1601695017 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56232f7a7790 0x56232f7a49d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:10:12.606 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:12.605+0000 7fe9d6000640 1 -- 172.21.15.22:0/1601695017 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56232f79e290 msgr2=0x56232f7a70e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T18:10:12.606 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:12.605+0000 7fe9d6000640 1 --2- 172.21.15.22:0/1601695017 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56232f79e290 0x56232f7a70e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:10:12.606 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:12.605+0000 7fe9d6000640 1 -- 172.21.15.22:0/1601695017 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56232f53ff70 con 0x56232f79ebb0 2024-03-28T18:10:12.606 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:12.605+0000 7fe9d6000640 1 --2- 172.21.15.22:0/1601695017 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56232f79ebb0 0x56232f7a4470 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fe9c809ada0 tx=0x7fe9c80971a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=16f4658d329d1c9a server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:10:12.606 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:12.605+0000 7fe9ccc00640 10 client.0 ms_handle_connect on v2:172.21.15.79:3300/0 2024-03-28T18:10:12.608 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:12.606+0000 7fe9ccc00640 1 -- 172.21.15.22:0/1601695017 <== mon.2 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9c8020050 con 0x56232f79ebb0 2024-03-28T18:10:12.608 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:12.606+0000 7fe9ccc00640 1 -- 172.21.15.22:0/1601695017 <== mon.2 v2:172.21.15.79:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 8411+0+0 (secure 0 0 0) 0x7fe9c80cc180 con 0x56232f79ebb0 2024-03-28T18:10:12.608 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:12.606+0000 7fe9e899a740 1 -- 172.21.15.22:0/1601695017 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fe9d8011d70 con 0x56232f79ebb0 2024-03-28T18:10:12.608 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:12.606+0000 7fe9ccc00640 1 -- 172.21.15.22:0/1601695017 <== mon.2 v2:172.21.15.79:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe9c80cc380 con 0x56232f79ebb0 2024-03-28T18:10:12.608 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:12.606+0000 7fe9e899a740 10 client.36164 fetch_fsmap learned FSMap version 31 2024-03-28T18:10:12.608 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:12.606+0000 7fe9e899a740 1 -- 172.21.15.22:0/1601695017 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) v3 -- 0x56232f3d7170 con 0x56232f79ebb0 2024-03-28T18:10:12.608 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:12.607+0000 7fe9ccc00640 1 -- 172.21.15.22:0/1601695017 <== mon.2 v2:172.21.15.79:3300/0 4 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7fe9d8011d70 con 0x56232f79ebb0 2024-03-28T18:10:12.608 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:12.607+0000 7fe9ccc00640 1 -- 172.21.15.22:0/1601695017 <== mon.2 v2:172.21.15.79:3300/0 5 ==== fsmap(e 31) v1 ==== 7950+0+0 (secure 0 0 0) 0x7fe9c80ceb90 con 0x56232f79ebb0 2024-03-28T18:10:12.608 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:12.607+0000 7fe9e899a740 10 client.36164 fetch_fsmap finished waiting for FSMap version 31 2024-03-28T18:10:12.608 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:12.607+0000 7fe9e899a740 10 client.36164 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-28T18:10:12.608 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:12.607+0000 7fe9e899a740 20 client.36164 populate_metadata read hostname 'smithi022' 2024-03-28T18:10:12.608 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:12.607+0000 7fe9e899a740 1 --2- 172.21.15.22:0/1601695017 >> [v2:172.21.15.79:6838/2441542443,v1:172.21.15.79:6839/2441542443] conn(0x56232f7b7900 0x56232f7b9cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:10:12.608 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:12.607+0000 7fe9e899a740 4 client.36164 mds_command: new command op to 34553 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-28T18:10:12.608 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:12.607+0000 7fe9e899a740 1 -- 172.21.15.22:0/1601695017 --> [v2:172.21.15.79:6838/2441542443,v1:172.21.15.79:6839/2441542443] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56232f7103f0 con 0x56232f7b7900 2024-03-28T18:10:12.609 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:12.608+0000 7fe9dea00640 1 --2- 172.21.15.22:0/1601695017 >> [v2:172.21.15.79:6838/2441542443,v1:172.21.15.79:6839/2441542443] conn(0x56232f7b7900 0x56232f7b9cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:10:12.609 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:12.609+0000 7fe9dea00640 1 --2- 172.21.15.22:0/1601695017 >> [v2:172.21.15.79:6838/2441542443,v1:172.21.15.79:6839/2441542443] conn(0x56232f7b7900 0x56232f7b9cf0 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6e7c5f2294a3e10c server_cookie=669d52a82ac78e10 in_seq=0 out_seq=0 2024-03-28T18:10:12.609 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:12.609+0000 7fe9ccc00640 10 client.36164 ms_handle_connect on v2:172.21.15.79:6838/2441542443 2024-03-28T18:10:12.613 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:12.613+0000 7fe9ccc00640 1 -- 172.21.15.22:0/1601695017 <== mds.0 v2:172.21.15.79:6838/2441542443 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x56232f7103f0 con 0x56232f7b7900 2024-03-28T18:10:12.613 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:12.613+0000 7fe9ccc00640 10 client.36164 handle_command_reply: tid=0 2024-03-28T18:10:12.613 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:12.613+0000 7fe9e899a740 1 client.36164 shutdown 2024-03-28T18:10:12.613 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:12.613+0000 7fe9e899a740 1 -- 172.21.15.22:0/1601695017 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56232f79ebb0 msgr2=0x56232f7a4470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:10:12.613 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:12.613+0000 7fe9e899a740 1 --2- 172.21.15.22:0/1601695017 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56232f79ebb0 0x56232f7a4470 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fe9c809ada0 tx=0x7fe9c80971a0 comp rx=0 tx=0).stop 2024-03-28T18:10:12.613 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:12.613+0000 7fe9e899a740 1 -- 172.21.15.22:0/1601695017 shutdown_connections 2024-03-28T18:10:12.613 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:12.613+0000 7fe9e899a740 1 --2- 172.21.15.22:0/1601695017 >> [v2:172.21.15.79:6838/2441542443,v1:172.21.15.79:6839/2441542443] conn(0x56232f7b7900 0x56232f7b9cf0 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:10:12.613 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:12.613+0000 7fe9e899a740 1 --2- 172.21.15.22:0/1601695017 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56232f7a7790 0x56232f7a49d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:10:12.614 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:12.613+0000 7fe9e899a740 1 --2- 172.21.15.22:0/1601695017 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56232f79ebb0 0x56232f7a4470 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:10:12.614 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:12.613+0000 7fe9e899a740 1 --2- 172.21.15.22:0/1601695017 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56232f79e290 0x56232f7a70e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:10:12.614 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:12.613+0000 7fe9e899a740 1 -- 172.21.15.22:0/1601695017 >> 172.21.15.22:0/1601695017 conn(0x56232f7989c0 msgr2=0x56232f79a710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:10:12.614 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:12.613+0000 7fe9ccc00640 0 client.36164 ms_handle_reset on v2:172.21.15.79:6838/2441542443 2024-03-28T18:10:12.614 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:12.613+0000 7fe9e899a740 1 -- 172.21.15.22:0/1601695017 shutdown_connections 2024-03-28T18:10:12.614 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:12.613+0000 7fe9e899a740 1 -- 172.21.15.22:0/1601695017 wait complete. 2024-03-28T18:10:12.614 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:12.613+0000 7fe9e899a740 20 client.36164 trim_cache size 0 max 16384 2024-03-28T18:10:12.630 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/sound/alsa/MIXART.txt 2024-03-28T18:10:12.635 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:12.634+0000 7fe9e899a740 1 Processor -- start 2024-03-28T18:10:12.635 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:12.634+0000 7fe9e899a740 1 -- start start 2024-03-28T18:10:12.635 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:12.635+0000 7fe9e899a740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56232f79e290 0x56232f7a6ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:10:12.635 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:12.635+0000 7fe9e899a740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56232f79ebb0 0x56232f7a70e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:10:12.635 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:12.635+0000 7fe9e899a740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56232f7a7790 0x56232f79a3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:10:12.635 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:12.635+0000 7fe9e899a740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9e01af040 con 0x56232f7a7790 2024-03-28T18:10:12.635 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:12.635+0000 7fe9e899a740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9e01b0a80 con 0x56232f79e290 2024-03-28T18:10:12.635 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:12.635+0000 7fe9e899a740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9e01af6a0 con 0x56232f79ebb0 2024-03-28T18:10:12.635 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:12.635+0000 7fe9df400640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56232f7a7790 0x56232f79a3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:10:12.635 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:12.635+0000 7fe9dea00640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56232f79e290 0x56232f7a6ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:10:12.635 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:12.635+0000 7fe9df400640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56232f7a7790 0x56232f79a3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:46200/0 (socket says 172.21.15.22:46200) 2024-03-28T18:10:12.636 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:12.635+0000 7fe9dea00640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56232f79e290 0x56232f7a6ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.22:52874/0 (socket says 172.21.15.22:52874) 2024-03-28T18:10:12.636 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:12.635+0000 7fe9df400640 1 -- 172.21.15.22:0/683743643 learned_addr learned my addr 172.21.15.22:0/683743643 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-28T18:10:12.637 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:12.635+0000 7fe9d6000640 1 --2- 172.21.15.22:0/683743643 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56232f79ebb0 0x56232f7a70e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:10:12.637 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:12.636+0000 7fe9df400640 1 -- 172.21.15.22:0/683743643 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56232f79ebb0 msgr2=0x56232f7a70e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T18:10:12.637 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:12.636+0000 7fe9df400640 1 --2- 172.21.15.22:0/683743643 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56232f79ebb0 0x56232f7a70e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:10:12.637 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:12.636+0000 7fe9df400640 1 -- 172.21.15.22:0/683743643 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56232f79e290 msgr2=0x56232f7a6ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T18:10:12.637 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:12.636+0000 7fe9df400640 1 --2- 172.21.15.22:0/683743643 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56232f79e290 0x56232f7a6ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:10:12.637 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:12.636+0000 7fe9df400640 1 -- 172.21.15.22:0/683743643 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56232f798200 con 0x56232f7a7790 2024-03-28T18:10:12.637 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:12.636+0000 7fe9df400640 1 --2- 172.21.15.22:0/683743643 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56232f7a7790 0x56232f79a3d0 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7fe9d8009f90 tx=0x7fe9d8005c70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=40d8538171446f66 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:10:12.637 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:12.637+0000 7fe9cf400640 1 -- 172.21.15.22:0/683743643 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9d8003970 con 0x56232f7a7790 2024-03-28T18:10:12.637 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:12.637+0000 7fe9cf400640 1 -- 172.21.15.22:0/683743643 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7fe9d8011500 con 0x56232f7a7790 2024-03-28T18:10:12.637 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:12.637+0000 7fe9cf400640 1 -- 172.21.15.22:0/683743643 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9d80052a0 con 0x56232f7a7790 2024-03-28T18:10:12.639 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:12.637+0000 7fe9e899a740 1 -- 172.21.15.22:0/683743643 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56232f7a7790 msgr2=0x56232f79a3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:10:12.640 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:12.637+0000 7fe9e899a740 1 --2- 172.21.15.22:0/683743643 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56232f7a7790 0x56232f79a3d0 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7fe9d8009f90 tx=0x7fe9d8005c70 comp rx=0 tx=0).stop 2024-03-28T18:10:12.640 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:12.638+0000 7fe9e899a740 1 -- 172.21.15.22:0/683743643 shutdown_connections 2024-03-28T18:10:12.640 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:12.638+0000 7fe9e899a740 1 --2- 172.21.15.22:0/683743643 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56232f7a7790 0x56232f79a3d0 unknown :-1 s=CLOSED pgs=652 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:10:12.640 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:12.638+0000 7fe9e899a740 1 --2- 172.21.15.22:0/683743643 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56232f79ebb0 0x56232f7a70e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:10:12.640 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:12.638+0000 7fe9e899a740 1 --2- 172.21.15.22:0/683743643 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56232f79e290 0x56232f7a6ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:10:12.640 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:12.638+0000 7fe9e899a740 1 -- 172.21.15.22:0/683743643 >> 172.21.15.22:0/683743643 conn(0x56232f7989c0 msgr2=0x56232f799cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:10:12.640 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:12.638+0000 7fe9e899a740 1 -- 172.21.15.22:0/683743643 shutdown_connections 2024-03-28T18:10:12.640 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:12.638+0000 7fe9e899a740 1 -- 172.21.15.22:0/683743643 wait complete. 2024-03-28T18:10:12.640 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:12.638+0000 7fe9e899a740 1 -- start start 2024-03-28T18:10:12.640 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:12.638+0000 7fe9e899a740 1 Processor -- start 2024-03-28T18:10:12.640 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:12.639+0000 7fe9e899a740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56232f79e290 0x56232f7af3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:10:12.640 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:12.639+0000 7fe9e899a740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56232f79ebb0 0x56232f7a1570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:10:12.640 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:12.639+0000 7fe9e899a740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56232f7a7790 0x56232f7a1ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:10:12.640 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:12.639+0000 7fe9e899a740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9e01b0360 con 0x56232f79e290 2024-03-28T18:10:12.641 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:12.639+0000 7fe9e899a740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9e01afd00 con 0x56232f79ebb0 2024-03-28T18:10:12.641 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:12.639+0000 7fe9e899a740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9e01ad200 con 0x56232f7a7790 2024-03-28T18:10:12.641 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:12.639+0000 7fe9e899a740 10 client.0 _ll_register_callbacks cb 0x7fe9c80d4220 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-28T18:10:12.641 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:12.639+0000 7fe9dea00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56232f79e290 0x56232f7af3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:10:12.641 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:12.639+0000 7fe9dea00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56232f79e290 0x56232f7af3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:46216/0 (socket says 172.21.15.22:46216) 2024-03-28T18:10:12.641 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:12.639+0000 7fe9dea00640 1 -- 172.21.15.22:0/1862669073 learned_addr learned my addr 172.21.15.22:0/1862669073 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-28T18:10:12.641 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:12.639+0000 7fe9d6000640 1 --2- 172.21.15.22:0/1862669073 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56232f79ebb0 0x56232f7a1570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:10:12.641 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:12.639+0000 7fe9df400640 1 --2- 172.21.15.22:0/1862669073 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56232f7a7790 0x56232f7a1ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:10:12.641 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:12.640+0000 7fe9dea00640 1 -- 172.21.15.22:0/1862669073 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56232f7a7790 msgr2=0x56232f7a1ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T18:10:12.641 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:12.640+0000 7fe9dea00640 1 --2- 172.21.15.22:0/1862669073 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56232f7a7790 0x56232f7a1ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:10:12.641 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:12.640+0000 7fe9dea00640 1 -- 172.21.15.22:0/1862669073 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56232f79ebb0 msgr2=0x56232f7a1570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T18:10:12.641 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:12.640+0000 7fe9dea00640 1 --2- 172.21.15.22:0/1862669073 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56232f79ebb0 0x56232f7a1570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:10:12.641 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:12.640+0000 7fe9dea00640 1 -- 172.21.15.22:0/1862669073 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56232f7a1ff0 con 0x56232f79e290 2024-03-28T18:10:12.641 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:12.640+0000 7fe9dea00640 1 --2- 172.21.15.22:0/1862669073 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56232f79e290 0x56232f7af3d0 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7fe9d002c2e0 tx=0x7fe9d0009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=49312a8250f82140 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:10:12.641 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:12.640+0000 7fe9ccc00640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-03-28T18:10:12.642 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:12.640+0000 7fe9ccc00640 1 -- 172.21.15.22:0/1862669073 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9d0059060 con 0x56232f79e290 2024-03-28T18:10:12.642 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:12.640+0000 7fe9ccc00640 1 -- 172.21.15.22:0/1862669073 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 8411+0+0 (secure 0 0 0) 0x7fe9d00046c0 con 0x56232f79e290 2024-03-28T18:10:12.642 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:12.640+0000 7fe9e899a740 1 -- 172.21.15.22:0/1862669073 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fe9d8011500 con 0x56232f79e290 2024-03-28T18:10:12.643 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:12.641+0000 7fe9ccc00640 1 -- 172.21.15.22:0/1862669073 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe9d00048c0 con 0x56232f79e290 2024-03-28T18:10:12.643 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:12.641+0000 7fe9e899a740 10 client.16341 fetch_fsmap learned FSMap version 31 2024-03-28T18:10:12.643 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:12.641+0000 7fe9e899a740 1 -- 172.21.15.22:0/1862669073 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) v3 -- 0x56232f813ea0 con 0x56232f79e290 2024-03-28T18:10:12.643 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:12.641+0000 7fe9ccc00640 1 -- 172.21.15.22:0/1862669073 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7fe9d8011500 con 0x56232f79e290 2024-03-28T18:10:12.643 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/sound/alsa/OSS-Emulation.txt 2024-03-28T18:10:12.645 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:12.641+0000 7fe9ccc00640 1 -- 172.21.15.22:0/1862669073 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 31) v1 ==== 7950+0+0 (secure 0 0 0) 0x7fe9d005c020 con 0x56232f79e290 2024-03-28T18:10:12.645 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:12.641+0000 7fe9e899a740 10 client.16341 fetch_fsmap finished waiting for FSMap version 31 2024-03-28T18:10:12.645 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:12.641+0000 7fe9e899a740 10 client.16341 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-28T18:10:12.645 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:12.641+0000 7fe9e899a740 20 client.16341 populate_metadata read hostname 'smithi022' 2024-03-28T18:10:12.645 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:12.641+0000 7fe9e899a740 1 --2- 172.21.15.22:0/1862669073 >> [v2:172.21.15.79:6838/2441542443,v1:172.21.15.79:6839/2441542443] conn(0x56232f814670 0x56232f816a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:10:12.645 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:12.641+0000 7fe9e899a740 4 client.16341 mds_command: new command op to 34553 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-03-28T18:10:12.645 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:12.642+0000 7fe9e899a740 1 -- 172.21.15.22:0/1862669073 --> [v2:172.21.15.79:6838/2441542443,v1:172.21.15.79:6839/2441542443] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x56232f70a740 con 0x56232f814670 2024-03-28T18:10:12.645 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:12.642+0000 7fe9d6000640 1 --2- 172.21.15.22:0/1862669073 >> [v2:172.21.15.79:6838/2441542443,v1:172.21.15.79:6839/2441542443] conn(0x56232f814670 0x56232f816a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:10:12.645 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:12.642+0000 7fe9d6000640 1 --2- 172.21.15.22:0/1862669073 >> [v2:172.21.15.79:6838/2441542443,v1:172.21.15.79:6839/2441542443] conn(0x56232f814670 0x56232f816a90 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7bc4f3d48fb60020 server_cookie=44505e025f577f3f in_seq=0 out_seq=0 2024-03-28T18:10:12.645 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:12.642+0000 7fe9ccc00640 10 client.16341 ms_handle_connect on v2:172.21.15.79:6838/2441542443 2024-03-28T18:10:12.645 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:12.644+0000 7fe9ccc00640 1 -- 172.21.15.22:0/1862669073 <== mds.0 v2:172.21.15.79:6838/2441542443 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x56232f70a740 con 0x56232f814670 2024-03-28T18:10:12.645 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:12.644+0000 7fe9ccc00640 10 client.16341 handle_command_reply: tid=0 2024-03-28T18:10:12.645 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:12.644+0000 7fe9e899a740 1 client.16341 shutdown 2024-03-28T18:10:12.646 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:12.644+0000 7fe9e899a740 1 -- 172.21.15.22:0/1862669073 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56232f79e290 msgr2=0x56232f7af3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:10:12.646 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:12.644+0000 7fe9e899a740 1 --2- 172.21.15.22:0/1862669073 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56232f79e290 0x56232f7af3d0 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7fe9d002c2e0 tx=0x7fe9d0009260 comp rx=0 tx=0).stop 2024-03-28T18:10:12.646 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:12.644+0000 7fe9e899a740 1 -- 172.21.15.22:0/1862669073 shutdown_connections 2024-03-28T18:10:12.646 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:12.644+0000 7fe9e899a740 1 --2- 172.21.15.22:0/1862669073 >> [v2:172.21.15.79:6838/2441542443,v1:172.21.15.79:6839/2441542443] conn(0x56232f814670 0x56232f816a90 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:10:12.646 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:12.644+0000 7fe9e899a740 1 --2- 172.21.15.22:0/1862669073 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56232f7a7790 0x56232f7a1ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:10:12.646 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:12.644+0000 7fe9e899a740 1 --2- 172.21.15.22:0/1862669073 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56232f79ebb0 0x56232f7a1570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:10:12.646 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:12.645+0000 7fe9e899a740 1 --2- 172.21.15.22:0/1862669073 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56232f79e290 0x56232f7af3d0 unknown :-1 s=CLOSED pgs=653 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:10:12.646 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:12.645+0000 7fe9ccc00640 0 client.16341 ms_handle_reset on v2:172.21.15.79:6838/2441542443 2024-03-28T18:10:12.646 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:12.645+0000 7fe9e899a740 1 -- 172.21.15.22:0/1862669073 >> 172.21.15.22:0/1862669073 conn(0x56232f7989c0 msgr2=0x56232f797a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:10:12.646 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:12.645+0000 7fe9e899a740 1 -- 172.21.15.22:0/1862669073 shutdown_connections 2024-03-28T18:10:12.646 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:12.645+0000 7fe9e899a740 1 -- 172.21.15.22:0/1862669073 wait complete. 2024-03-28T18:10:12.646 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:12.645+0000 7fe9e899a740 20 client.16341 trim_cache size 0 max 16384 2024-03-28T18:10:12.647 INFO:teuthology.orchestra.run.smithi022.stdout:{ 2024-03-28T18:10:12.647 INFO:teuthology.orchestra.run.smithi022.stdout: "return_code": 0, 2024-03-28T18:10:12.647 INFO:teuthology.orchestra.run.smithi022.stdout: "scrub_tag": "6dada245-ed7b-4368-89c1-2734f3acce1b", 2024-03-28T18:10:12.647 INFO:teuthology.orchestra.run.smithi022.stdout: "mode": "asynchronous" 2024-03-28T18:10:12.647 INFO:teuthology.orchestra.run.smithi022.stdout:} 2024-03-28T18:10:12.648 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:12.646+0000 7fe9dfe00640 1 -- 172.21.15.22:0/1444360711 >> [v2:172.21.15.22:6800/1357365082,v1:172.21.15.22:6801/1357365082] conn(0x7fe9c40a29f0 msgr2=0x7fe9c40a4eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:10:12.648 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:12.646+0000 7fe9dfe00640 1 --2- 172.21.15.22:0/1444360711 >> [v2:172.21.15.22:6800/1357365082,v1:172.21.15.22:6801/1357365082] conn(0x7fe9c40a29f0 0x7fe9c40a4eb0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fe9d0006010 tx=0x7fe9d0005b50 comp rx=0 tx=0).stop 2024-03-28T18:10:12.648 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:12.646+0000 7fe9dfe00640 1 -- 172.21.15.22:0/1444360711 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe9e0102ff0 msgr2=0x7fe9e019a590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:10:12.648 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:12.646+0000 7fe9dfe00640 1 --2- 172.21.15.22:0/1444360711 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe9e0102ff0 0x7fe9e019a590 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7fe9c800d750 tx=0x7fe9c800dc10 comp rx=0 tx=0).stop 2024-03-28T18:10:12.648 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:12.646+0000 7fe9dfe00640 1 -- 172.21.15.22:0/1444360711 shutdown_connections 2024-03-28T18:10:12.648 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:12.646+0000 7fe9dfe00640 1 --2- 172.21.15.22:0/1444360711 >> [v2:172.21.15.22:6800/1357365082,v1:172.21.15.22:6801/1357365082] conn(0x7fe9c40a29f0 0x7fe9c40a4eb0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:10:12.648 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:12.646+0000 7fe9dfe00640 1 --2- 172.21.15.22:0/1444360711 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fe9e010baf0 0x7fe9e019e910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:10:12.648 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:12.646+0000 7fe9dfe00640 1 --2- 172.21.15.22:0/1444360711 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe9e0102ff0 0x7fe9e019a590 unknown :-1 s=CLOSED pgs=650 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:10:12.648 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:12.646+0000 7fe9dfe00640 1 --2- 172.21.15.22:0/1444360711 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe9e01026d0 0x7fe9e019a050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:10:12.648 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:12.646+0000 7fe9dfe00640 1 -- 172.21.15.22:0/1444360711 >> 172.21.15.22:0/1444360711 conn(0x7fe9e00fbef0 msgr2=0x7fe9e00fe2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:10:12.648 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:12.646+0000 7fe9dfe00640 1 -- 172.21.15.22:0/1444360711 shutdown_connections 2024-03-28T18:10:12.648 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:12.646+0000 7fe9dfe00640 1 -- 172.21.15.22:0/1444360711 wait complete. 2024-03-28T18:10:12.662 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/sound/alsa/Procfile.txt 2024-03-28T18:10:12.665 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-28T18:10:12.665 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-28T18:10:12.677 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/sound/alsa/README.maya44 2024-03-28T18:10:12.699 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/sound/alsa/SB-Live-mixer.txt 2024-03-28T18:10:12.711 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/sound/alsa/VIA82xx-mixer.txt 2024-03-28T18:10:12.728 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/sound/alsa/alsa-parameters.txt 2024-03-28T18:10:12.749 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/sound/alsa/compress_offload.txt 2024-03-28T18:10:12.770 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/sound/alsa/emu10k1-jack.txt 2024-03-28T18:10:12.788 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/sound/alsa/hda_codec.txt 2024-03-28T18:10:12.807 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/sound/alsa/hdspm.txt 2024-03-28T18:10:12.821 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/sound/alsa/powersave.txt 2024-03-28T18:10:12.837 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/sound/alsa/seq_oss.html 2024-03-28T18:10:12.851 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/sound/alsa/serial-u16550.txt 2024-03-28T18:10:12.866 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/sound/alsa/soc/ 2024-03-28T18:10:12.878 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/sound/alsa/soc/DAI.txt 2024-03-28T18:10:12.886 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/sound/alsa/soc/DPCM.txt 2024-03-28T18:10:12.901 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/sound/alsa/soc/clocking.txt 2024-03-28T18:10:12.913 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/sound/alsa/soc/codec.txt 2024-03-28T18:10:12.920 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:12.920+0000 7f76d3200640 1 -- 172.21.15.22:0/1116170017 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f76cc1091a0 msgr2=0x7f76cc1142b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:10:12.920 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:12.920+0000 7f76d3200640 1 --2- 172.21.15.22:0/1116170017 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f76cc1091a0 0x7f76cc1142b0 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7f76b8009900 tx=0x7f76b802bc30 comp rx=0 tx=0).stop 2024-03-28T18:10:12.921 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:12.921+0000 7f76d3200640 1 -- 172.21.15.22:0/1116170017 shutdown_connections 2024-03-28T18:10:12.921 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:12.921+0000 7f76d3200640 1 --2- 172.21.15.22:0/1116170017 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f76cc1147f0 0x7f76cc116be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:10:12.921 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:12.921+0000 7f76d3200640 1 --2- 172.21.15.22:0/1116170017 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f76cc1091a0 0x7f76cc1142b0 unknown :-1 s=CLOSED pgs=654 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:10:12.921 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:12.921+0000 7f76d3200640 1 --2- 172.21.15.22:0/1116170017 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f76cc108880 0x7f76cc108c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:10:12.921 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:12.921+0000 7f76d3200640 1 -- 172.21.15.22:0/1116170017 >> 172.21.15.22:0/1116170017 conn(0x7f76cc06c780 msgr2=0x7f76cc06cb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:10:12.921 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:12.921+0000 7f76d3200640 1 -- 172.21.15.22:0/1116170017 shutdown_connections 2024-03-28T18:10:12.921 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:12.921+0000 7f76d3200640 1 -- 172.21.15.22:0/1116170017 wait complete. 2024-03-28T18:10:12.921 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:12.922+0000 7f76d3200640 1 Processor -- start 2024-03-28T18:10:12.922 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:12.922+0000 7f76d3200640 1 -- start start 2024-03-28T18:10:12.923 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:12.922+0000 7f76d3200640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f76cc108880 0x7f76cc1a0740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:10:12.923 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:12.922+0000 7f76d3200640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f76cc1091a0 0x7f76cc1a0c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:10:12.923 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:12.922+0000 7f76d3200640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f76cc1147f0 0x7f76cc1a5000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:10:12.923 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:12.922+0000 7f76d3200640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76cc11a1e0 con 0x7f76cc108880 2024-03-28T18:10:12.923 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:12.922+0000 7f76d3200640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76cc11a060 con 0x7f76cc1091a0 2024-03-28T18:10:12.923 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:12.922+0000 7f76d3200640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76cc11a360 con 0x7f76cc1147f0 2024-03-28T18:10:12.923 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:12.923+0000 7f76cbe00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f76cc108880 0x7f76cc1a0740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:10:12.923 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:12.923+0000 7f76cbe00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f76cc108880 0x7f76cc1a0740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:46240/0 (socket says 172.21.15.22:46240) 2024-03-28T18:10:12.923 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:12.923+0000 7f76cbe00640 1 -- 172.21.15.22:0/1639963463 learned_addr learned my addr 172.21.15.22:0/1639963463 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-28T18:10:12.924 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:12.923+0000 7f76d1200640 1 --2- 172.21.15.22:0/1639963463 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f76cc1147f0 0x7f76cc1a5000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:10:12.924 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:12.923+0000 7f76cb400640 1 --2- 172.21.15.22:0/1639963463 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f76cc1091a0 0x7f76cc1a0c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:10:12.924 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:12.923+0000 7f76cbe00640 1 -- 172.21.15.22:0/1639963463 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f76cc1147f0 msgr2=0x7f76cc1a5000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:10:12.924 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:12.923+0000 7f76cbe00640 1 --2- 172.21.15.22:0/1639963463 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f76cc1147f0 0x7f76cc1a5000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:10:12.924 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:12.923+0000 7f76cbe00640 1 -- 172.21.15.22:0/1639963463 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f76cc1091a0 msgr2=0x7f76cc1a0c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:10:12.924 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:12.923+0000 7f76cbe00640 1 --2- 172.21.15.22:0/1639963463 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f76cc1091a0 0x7f76cc1a0c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:10:12.924 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:12.923+0000 7f76cbe00640 1 -- 172.21.15.22:0/1639963463 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f76cc1a5730 con 0x7f76cc108880 2024-03-28T18:10:12.924 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:12.924+0000 7f76cbe00640 1 --2- 172.21.15.22:0/1639963463 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f76cc108880 0x7f76cc1a0740 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7f76bc00e9f0 tx=0x7f76bc00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:10:12.925 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:12.924+0000 7f76c8c00640 1 -- 172.21.15.22:0/1639963463 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76bc00cd80 con 0x7f76cc108880 2024-03-28T18:10:12.925 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:12.924+0000 7f76c8c00640 1 -- 172.21.15.22:0/1639963463 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f76bc010900 con 0x7f76cc108880 2024-03-28T18:10:12.925 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:12.924+0000 7f76d3200640 1 -- 172.21.15.22:0/1639963463 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f76cc1a5a20 con 0x7f76cc108880 2024-03-28T18:10:12.925 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:12.924+0000 7f76c8c00640 1 -- 172.21.15.22:0/1639963463 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76bc0189e0 con 0x7f76cc108880 2024-03-28T18:10:12.925 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:12.924+0000 7f76d3200640 1 -- 172.21.15.22:0/1639963463 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f76cc1ad2b0 con 0x7f76cc108880 2024-03-28T18:10:12.925 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/sound/alsa/soc/dapm.txt 2024-03-28T18:10:12.929 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:12.928+0000 7f76d5ff0740 1 Processor -- start 2024-03-28T18:10:12.929 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:12.928+0000 7f76d5ff0740 1 -- start start 2024-03-28T18:10:12.929 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:12.929+0000 7f76d5ff0740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e854185ec0 0x55e8541862a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:10:12.929 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:12.929+0000 7f76d5ff0740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55e854186870 0x55e8541a2e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:10:12.929 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:12.929+0000 7f76d5ff0740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e8541a33c0 0x55e8541a57b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:10:12.929 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:12.929+0000 7f76d5ff0740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e8540ed250 con 0x55e854185ec0 2024-03-28T18:10:12.929 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:12.929+0000 7f76d5ff0740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e854095a30 con 0x55e854186870 2024-03-28T18:10:12.930 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:12.929+0000 7f76d5ff0740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e8540e92f0 con 0x55e8541a33c0 2024-03-28T18:10:12.930 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:12.929+0000 7f76d1200640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e854185ec0 0x55e8541862a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:10:12.930 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:12.929+0000 7f76d1200640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e854185ec0 0x55e8541862a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:46254/0 (socket says 172.21.15.22:46254) 2024-03-28T18:10:12.930 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:12.929+0000 7f76d1200640 1 -- 172.21.15.22:0/3106154768 learned_addr learned my addr 172.21.15.22:0/3106154768 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-28T18:10:12.930 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:12.929+0000 7f76cb400640 1 --2- 172.21.15.22:0/3106154768 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e8541a33c0 0x55e8541a57b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:10:12.931 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:12.930+0000 7f76cb400640 1 -- 172.21.15.22:0/3106154768 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55e854186870 msgr2=0x55e8541a2e80 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-28T18:10:12.931 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:12.930+0000 7f76cb400640 1 --2- 172.21.15.22:0/3106154768 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55e854186870 0x55e8541a2e80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:10:12.931 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:12.930+0000 7f76cb400640 1 -- 172.21.15.22:0/3106154768 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e854185ec0 msgr2=0x55e8541862a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T18:10:12.931 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:12.930+0000 7f76cb400640 1 --2- 172.21.15.22:0/3106154768 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e854185ec0 0x55e8541862a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:10:12.931 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:12.930+0000 7f76cb400640 1 -- 172.21.15.22:0/3106154768 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e854080690 con 0x55e8541a33c0 2024-03-28T18:10:12.931 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:12.930+0000 7f76d1200640 1 --2- 172.21.15.22:0/3106154768 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e854185ec0 0x55e8541862a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-28T18:10:12.931 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:12.930+0000 7f76c8c00640 1 -- 172.21.15.22:0/1639963463 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 26) v1 ==== 137421+0+0 (secure 0 0 0) 0x7f76bc010ab0 con 0x7f76cc108880 2024-03-28T18:10:12.931 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:12.930+0000 7f76cb400640 1 --2- 172.21.15.22:0/3106154768 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e8541a33c0 0x55e8541a57b0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f76b8009230 tx=0x7f76b8005be0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=53b12855d8db074e server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:10:12.931 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:12.931+0000 7f76c1600640 1 -- 172.21.15.22:0/3106154768 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76b8045020 con 0x55e8541a33c0 2024-03-28T18:10:12.931 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:12.931+0000 7f76c1600640 1 -- 172.21.15.22:0/3106154768 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f76b8005040 con 0x55e8541a33c0 2024-03-28T18:10:12.932 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:12.931+0000 7f76c1600640 1 -- 172.21.15.22:0/3106154768 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76b802cac0 con 0x55e8541a33c0 2024-03-28T18:10:12.933 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:12.932+0000 7f76d5ff0740 1 -- 172.21.15.22:0/3106154768 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e8541a33c0 msgr2=0x55e8541a57b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:10:12.933 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:12.932+0000 7f76d5ff0740 1 --2- 172.21.15.22:0/3106154768 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e8541a33c0 0x55e8541a57b0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f76b8009230 tx=0x7f76b8005be0 comp rx=0 tx=0).stop 2024-03-28T18:10:12.933 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:12.932+0000 7f76d5ff0740 1 -- 172.21.15.22:0/3106154768 shutdown_connections 2024-03-28T18:10:12.933 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:12.932+0000 7f76d5ff0740 1 --2- 172.21.15.22:0/3106154768 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e8541a33c0 0x55e8541a57b0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:10:12.933 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:12.932+0000 7f76d5ff0740 1 --2- 172.21.15.22:0/3106154768 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55e854186870 0x55e8541a2e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:10:12.933 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:12.932+0000 7f76d5ff0740 1 --2- 172.21.15.22:0/3106154768 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e854185ec0 0x55e8541862a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:10:12.933 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:12.932+0000 7f76d5ff0740 1 -- 172.21.15.22:0/3106154768 >> 172.21.15.22:0/3106154768 conn(0x55e854150200 msgr2=0x55e854184db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:10:12.933 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:12.932+0000 7f76d5ff0740 1 -- 172.21.15.22:0/3106154768 shutdown_connections 2024-03-28T18:10:12.933 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:12.932+0000 7f76c8c00640 1 --2- 172.21.15.22:0/1639963463 >> [v2:172.21.15.22:6800/1357365082,v1:172.21.15.22:6801/1357365082] conn(0x7f76b00a2920 0x7f76b00a4de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:10:12.933 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:12.932+0000 7f76d5ff0740 1 -- 172.21.15.22:0/3106154768 wait complete. 2024-03-28T18:10:12.933 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:12.933+0000 7f76d5ff0740 1 -- start start 2024-03-28T18:10:12.933 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:12.933+0000 7f76c8c00640 1 -- 172.21.15.22:0/1639963463 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 8411+0+0 (secure 0 0 0) 0x7f76bc0cf0e0 con 0x7f76cc108880 2024-03-28T18:10:12.934 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:12.933+0000 7f76cb400640 1 --2- 172.21.15.22:0/1639963463 >> [v2:172.21.15.22:6800/1357365082,v1:172.21.15.22:6801/1357365082] conn(0x7f76b00a2920 0x7f76b00a4de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:10:12.934 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:12.933+0000 7f76d5ff0740 1 Processor -- start 2024-03-28T18:10:12.935 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:12.934+0000 7f76d5ff0740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e854185ec0 0x55e85418f3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:10:12.935 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:12.934+0000 7f76d5ff0740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55e854186870 0x55e85418c7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:10:12.935 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:12.934+0000 7f76d5ff0740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e8541a33c0 0x55e85418cd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:10:12.935 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:12.934+0000 7f76d5ff0740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e8541a60d0 con 0x55e8541a33c0 2024-03-28T18:10:12.935 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:12.934+0000 7f76d5ff0740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e8540a8e80 con 0x55e854186870 2024-03-28T18:10:12.935 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:12.934+0000 7f76d5ff0740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e8541a6250 con 0x55e854185ec0 2024-03-28T18:10:12.935 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:12.934+0000 7f76d5ff0740 10 client.0 _ll_register_callbacks cb 0x55e8540bb130 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-28T18:10:12.935 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:12.934+0000 7f76d1200640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e8541a33c0 0x55e85418cd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:10:12.935 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:12.934+0000 7f76cb400640 1 --2- 172.21.15.22:0/1639963463 >> [v2:172.21.15.22:6800/1357365082,v1:172.21.15.22:6801/1357365082] conn(0x7f76b00a2920 0x7f76b00a4de0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f76b8005ab0 tx=0x7f76b802bc30 comp rx=0 tx=0).ready entity=mgr.15522 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:10:12.935 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:12.934+0000 7f76d1200640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e8541a33c0 0x55e85418cd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:46256/0 (socket says 172.21.15.22:46256) 2024-03-28T18:10:12.935 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:12.934+0000 7f76d1200640 1 -- 172.21.15.22:0/4237074216 learned_addr learned my addr 172.21.15.22:0/4237074216 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-28T18:10:12.935 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:12.935+0000 7f76cbe00640 1 --2- 172.21.15.22:0/4237074216 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e854185ec0 0x55e85418f3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:10:12.935 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:12.935+0000 7f76cb400640 1 --2- 172.21.15.22:0/4237074216 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55e854186870 0x55e85418c7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:10:12.936 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:12.935+0000 7f76d1200640 1 -- 172.21.15.22:0/4237074216 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e854185ec0 msgr2=0x55e85418f3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T18:10:12.936 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:12.935+0000 7f76d1200640 1 --2- 172.21.15.22:0/4237074216 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e854185ec0 0x55e85418f3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:10:12.936 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:12.935+0000 7f76d1200640 1 -- 172.21.15.22:0/4237074216 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55e854186870 msgr2=0x55e85418c7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T18:10:12.936 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:12.935+0000 7f76d1200640 1 --2- 172.21.15.22:0/4237074216 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55e854186870 0x55e85418c7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:10:12.936 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:12.935+0000 7f76d1200640 1 -- 172.21.15.22:0/4237074216 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55e853ecb7d0 con 0x55e8541a33c0 2024-03-28T18:10:12.936 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:12.935+0000 7f76cb400640 1 --2- 172.21.15.22:0/4237074216 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55e854186870 0x55e85418c7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T18:10:12.937 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:12.935+0000 7f76d1200640 1 --2- 172.21.15.22:0/4237074216 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e8541a33c0 0x55e85418cd00 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7f76c4011d10 tx=0x7f76c400f790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c28fe2f25ebcc803 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:10:12.937 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:12.936+0000 7f76b6000640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-03-28T18:10:12.937 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:12.936+0000 7f76b6000640 1 -- 172.21.15.22:0/4237074216 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76c401f060 con 0x55e8541a33c0 2024-03-28T18:10:12.938 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:12.936+0000 7f76b6000640 1 -- 172.21.15.22:0/4237074216 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 8411+0+0 (secure 0 0 0) 0x7f76c4017050 con 0x55e8541a33c0 2024-03-28T18:10:12.938 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:12.936+0000 7f76d5ff0740 1 -- 172.21.15.22:0/4237074216 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f76b8005040 con 0x55e8541a33c0 2024-03-28T18:10:12.938 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:12.936+0000 7f76b6000640 1 -- 172.21.15.22:0/4237074216 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) v2 ==== 24+0+0 (secure 0 0 0) 0x7f76c400fe20 con 0x55e8541a33c0 2024-03-28T18:10:12.938 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:12.936+0000 7f76d5ff0740 10 client.16353 fetch_fsmap learned FSMap version 31 2024-03-28T18:10:12.938 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:12.936+0000 7f76d5ff0740 1 -- 172.21.15.22:0/4237074216 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) v3 -- 0x55e853e013f0 con 0x55e8541a33c0 2024-03-28T18:10:12.938 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:12.937+0000 7f76b6000640 1 -- 172.21.15.22:0/4237074216 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f76b8005040 con 0x55e8541a33c0 2024-03-28T18:10:12.938 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:12.937+0000 7f76b6000640 1 -- 172.21.15.22:0/4237074216 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 31) v1 ==== 7950+0+0 (secure 0 0 0) 0x7f76c4022020 con 0x55e8541a33c0 2024-03-28T18:10:12.938 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:12.937+0000 7f76d5ff0740 10 client.16353 fetch_fsmap finished waiting for FSMap version 31 2024-03-28T18:10:12.938 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:12.937+0000 7f76d5ff0740 10 client.16353 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-28T18:10:12.938 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:12.937+0000 7f76d5ff0740 20 client.16353 populate_metadata read hostname 'smithi022' 2024-03-28T18:10:12.938 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:12.937+0000 7f76d5ff0740 1 --2- 172.21.15.22:0/4237074216 >> [v2:172.21.15.79:6838/2441542443,v1:172.21.15.79:6839/2441542443] conn(0x55e8541b3250 0x55e8541b5670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:10:12.938 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:12.937+0000 7f76d5ff0740 4 client.16353 mds_command: new command op to 34553 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-28T18:10:12.938 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:12.937+0000 7f76d5ff0740 1 -- 172.21.15.22:0/4237074216 --> [v2:172.21.15.79:6838/2441542443,v1:172.21.15.79:6839/2441542443] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55e8540fa790 con 0x55e8541b3250 2024-03-28T18:10:12.938 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:12.938+0000 7f76cbe00640 1 --2- 172.21.15.22:0/4237074216 >> [v2:172.21.15.79:6838/2441542443,v1:172.21.15.79:6839/2441542443] conn(0x55e8541b3250 0x55e8541b5670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:10:12.939 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/sound/alsa/soc/jack.txt 2024-03-28T18:10:12.939 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:12.939+0000 7f76cbe00640 1 --2- 172.21.15.22:0/4237074216 >> [v2:172.21.15.79:6838/2441542443,v1:172.21.15.79:6839/2441542443] conn(0x55e8541b3250 0x55e8541b5670 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4d39c4bae2616ed server_cookie=8fc4af09346e929d in_seq=0 out_seq=0 2024-03-28T18:10:12.939 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:12.939+0000 7f76b6000640 10 client.16353 ms_handle_connect on v2:172.21.15.79:6838/2441542443 2024-03-28T18:10:12.940 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:12.941+0000 7f76b6000640 1 -- 172.21.15.22:0/4237074216 <== mds.0 v2:172.21.15.79:6838/2441542443 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x55e8540fa790 con 0x55e8541b3250 2024-03-28T18:10:12.940 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:12.941+0000 7f76b6000640 10 client.16353 handle_command_reply: tid=0 2024-03-28T18:10:12.940 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:12.941+0000 7f76d5ff0740 1 client.16353 shutdown 2024-03-28T18:10:12.941 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:12.941+0000 7f76d5ff0740 1 -- 172.21.15.22:0/4237074216 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e8541a33c0 msgr2=0x55e85418cd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:10:12.941 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:12.941+0000 7f76d5ff0740 1 --2- 172.21.15.22:0/4237074216 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e8541a33c0 0x55e85418cd00 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7f76c4011d10 tx=0x7f76c400f790 comp rx=0 tx=0).stop 2024-03-28T18:10:12.941 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:12.941+0000 7f76d5ff0740 1 -- 172.21.15.22:0/4237074216 shutdown_connections 2024-03-28T18:10:12.942 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:12.941+0000 7f76d5ff0740 1 --2- 172.21.15.22:0/4237074216 >> [v2:172.21.15.79:6838/2441542443,v1:172.21.15.79:6839/2441542443] conn(0x55e8541b3250 0x55e8541b5670 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:10:12.942 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:12.941+0000 7f76d5ff0740 1 --2- 172.21.15.22:0/4237074216 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e8541a33c0 0x55e85418cd00 unknown :-1 s=CLOSED pgs=656 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:10:12.942 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:12.941+0000 7f76d5ff0740 1 --2- 172.21.15.22:0/4237074216 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55e854186870 0x55e85418c7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:10:12.942 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:12.941+0000 7f76d5ff0740 1 --2- 172.21.15.22:0/4237074216 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e854185ec0 0x55e85418f3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:10:12.942 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:12.941+0000 7f76d5ff0740 1 -- 172.21.15.22:0/4237074216 >> 172.21.15.22:0/4237074216 conn(0x55e854150200 msgr2=0x55e854183cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:10:12.942 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:12.941+0000 7f76b6000640 0 client.16353 ms_handle_reset on v2:172.21.15.79:6838/2441542443 2024-03-28T18:10:12.942 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:12.942+0000 7f76d5ff0740 1 -- 172.21.15.22:0/4237074216 shutdown_connections 2024-03-28T18:10:12.942 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:12.942+0000 7f76d5ff0740 1 -- 172.21.15.22:0/4237074216 wait complete. 2024-03-28T18:10:12.942 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:12.942+0000 7f76d5ff0740 20 client.16353 trim_cache size 0 max 16384 2024-03-28T18:10:12.948 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/sound/alsa/soc/machine.txt 2024-03-28T18:10:12.958 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/sound/alsa/soc/overview.txt 2024-03-28T18:10:12.965 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:12.965+0000 7f76d5ff0740 1 Processor -- start 2024-03-28T18:10:12.965 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:12.965+0000 7f76d5ff0740 1 -- start start 2024-03-28T18:10:12.965 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:12.966+0000 7f76d5ff0740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e854185ec0 0x55e8541848f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:10:12.965 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:12.966+0000 7f76d5ff0740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e854186870 0x55e854184e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:10:12.965 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:12.966+0000 7f76d5ff0740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55e8541a33c0 0x55e854189880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:10:12.965 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:12.966+0000 7f76d5ff0740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76cc1b35d0 con 0x55e854186870 2024-03-28T18:10:12.965 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:12.966+0000 7f76d5ff0740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76cc1b5010 con 0x55e8541a33c0 2024-03-28T18:10:12.965 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:12.966+0000 7f76d5ff0740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76cc1b3c30 con 0x55e854185ec0 2024-03-28T18:10:12.967 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:12.966+0000 7f76cb400640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e854186870 0x55e854184e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:10:12.967 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:12.966+0000 7f76cbe00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e854185ec0 0x55e8541848f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:10:12.967 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:12.966+0000 7f76cb400640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e854186870 0x55e854184e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:46264/0 (socket says 172.21.15.22:46264) 2024-03-28T18:10:12.967 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:12.966+0000 7f76cbe00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e854185ec0 0x55e8541848f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.22:45170/0 (socket says 172.21.15.22:45170) 2024-03-28T18:10:12.967 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:12.966+0000 7f76cb400640 1 -- 172.21.15.22:0/166825013 learned_addr learned my addr 172.21.15.22:0/166825013 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-28T18:10:12.967 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:12.966+0000 7f76d1200640 1 --2- 172.21.15.22:0/166825013 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55e8541a33c0 0x55e854189880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:10:12.967 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:12.967+0000 7f76cbe00640 1 -- 172.21.15.22:0/166825013 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55e8541a33c0 msgr2=0x55e854189880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T18:10:12.967 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:12.967+0000 7f76cbe00640 1 --2- 172.21.15.22:0/166825013 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55e8541a33c0 0x55e854189880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:10:12.967 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:12.967+0000 7f76cbe00640 1 -- 172.21.15.22:0/166825013 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e854186870 msgr2=0x55e854184e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T18:10:12.967 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:12.967+0000 7f76cbe00640 1 --2- 172.21.15.22:0/166825013 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e854186870 0x55e854184e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:10:12.967 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:12.967+0000 7f76cbe00640 1 -- 172.21.15.22:0/166825013 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e853dc10f0 con 0x55e854185ec0 2024-03-28T18:10:12.967 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:12.967+0000 7f76d1200640 1 --2- 172.21.15.22:0/166825013 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55e8541a33c0 0x55e854189880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-28T18:10:12.967 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:12.967+0000 7f76cbe00640 1 --2- 172.21.15.22:0/166825013 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e854185ec0 0x55e8541848f0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f76bc0cc8c0 tx=0x7f76bc01f000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=73415fb844e2d34d server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:10:12.968 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:12.967+0000 7f76c0c00640 1 -- 172.21.15.22:0/166825013 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76bc099b80 con 0x55e854185ec0 2024-03-28T18:10:12.968 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:12.967+0000 7f76c0c00640 1 -- 172.21.15.22:0/166825013 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f76bc097ef0 con 0x55e854185ec0 2024-03-28T18:10:12.968 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:12.967+0000 7f76c0c00640 1 -- 172.21.15.22:0/166825013 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76bc018de0 con 0x55e854185ec0 2024-03-28T18:10:12.968 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:12.968+0000 7f76d5ff0740 1 -- 172.21.15.22:0/166825013 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e854185ec0 msgr2=0x55e8541848f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:10:12.968 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:12.968+0000 7f76d5ff0740 1 --2- 172.21.15.22:0/166825013 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e854185ec0 0x55e8541848f0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f76bc0cc8c0 tx=0x7f76bc01f000 comp rx=0 tx=0).stop 2024-03-28T18:10:12.969 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:12.968+0000 7f76d5ff0740 1 -- 172.21.15.22:0/166825013 shutdown_connections 2024-03-28T18:10:12.969 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:12.968+0000 7f76d5ff0740 1 --2- 172.21.15.22:0/166825013 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55e8541a33c0 0x55e854189880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:10:12.969 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:12.968+0000 7f76d5ff0740 1 --2- 172.21.15.22:0/166825013 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e854186870 0x55e854184e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:10:12.969 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:12.968+0000 7f76d5ff0740 1 --2- 172.21.15.22:0/166825013 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e854185ec0 0x55e8541848f0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:10:12.969 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:12.968+0000 7f76d5ff0740 1 -- 172.21.15.22:0/166825013 >> 172.21.15.22:0/166825013 conn(0x55e854150200 msgr2=0x55e854187860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:10:12.969 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:12.969+0000 7f76d5ff0740 1 -- 172.21.15.22:0/166825013 shutdown_connections 2024-03-28T18:10:12.969 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:12.969+0000 7f76d5ff0740 1 -- 172.21.15.22:0/166825013 wait complete. 2024-03-28T18:10:12.969 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/sound/alsa/soc/platform.txt 2024-03-28T18:10:12.970 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:12.969+0000 7f76d5ff0740 1 -- start start 2024-03-28T18:10:12.970 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:12.969+0000 7f76d5ff0740 1 Processor -- start 2024-03-28T18:10:12.970 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:12.970+0000 7f76d5ff0740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e854185ec0 0x55e8541aaec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:10:12.970 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:12.970+0000 7f76d5ff0740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55e854186870 0x55e8541ab400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:10:12.970 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:12.970+0000 7f76d5ff0740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e8541a33c0 0x55e8541a86e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:10:12.970 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:12.970+0000 7f76d5ff0740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76cc1b48f0 con 0x55e854185ec0 2024-03-28T18:10:12.970 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:12.970+0000 7f76d5ff0740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76cc1b4290 con 0x55e854186870 2024-03-28T18:10:12.971 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:12.970+0000 7f76d5ff0740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76cc1b1790 con 0x55e8541a33c0 2024-03-28T18:10:12.971 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:12.970+0000 7f76d5ff0740 10 client.0 _ll_register_callbacks cb 0x7f76c4009cc0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-28T18:10:12.971 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:12.970+0000 7f76cbe00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e854185ec0 0x55e8541aaec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:10:12.971 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:12.970+0000 7f76cbe00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e854185ec0 0x55e8541aaec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:46270/0 (socket says 172.21.15.22:46270) 2024-03-28T18:10:12.971 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:12.970+0000 7f76cbe00640 1 -- 172.21.15.22:0/3969661961 learned_addr learned my addr 172.21.15.22:0/3969661961 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-28T18:10:12.971 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:12.970+0000 7f76cb400640 1 --2- 172.21.15.22:0/3969661961 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55e854186870 0x55e8541ab400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:10:12.971 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:12.970+0000 7f76d1200640 1 --2- 172.21.15.22:0/3969661961 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e8541a33c0 0x55e8541a86e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:10:12.972 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:12.971+0000 7f76cbe00640 1 -- 172.21.15.22:0/3969661961 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e8541a33c0 msgr2=0x55e8541a86e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T18:10:12.972 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:12.971+0000 7f76cbe00640 1 --2- 172.21.15.22:0/3969661961 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e8541a33c0 0x55e8541a86e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:10:12.972 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:12.971+0000 7f76cbe00640 1 -- 172.21.15.22:0/3969661961 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55e854186870 msgr2=0x55e8541ab400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T18:10:12.972 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:12.971+0000 7f76cbe00640 1 --2- 172.21.15.22:0/3969661961 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55e854186870 0x55e8541ab400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:10:12.972 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:12.971+0000 7f76cbe00640 1 -- 172.21.15.22:0/3969661961 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55e8541a8ee0 con 0x55e854185ec0 2024-03-28T18:10:12.972 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:12.971+0000 7f76cbe00640 1 --2- 172.21.15.22:0/3969661961 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e854185ec0 0x55e8541aaec0 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7f76bc09b810 tx=0x7f76bc09faa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=29bfb6cfabf7b49a server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:10:12.972 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:12.971+0000 7f76b6000640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-03-28T18:10:12.972 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:12.972+0000 7f76b6000640 1 -- 172.21.15.22:0/3969661961 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76bc0af060 con 0x55e854185ec0 2024-03-28T18:10:12.972 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:12.972+0000 7f76b6000640 1 -- 172.21.15.22:0/3969661961 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 8411+0+0 (secure 0 0 0) 0x7f76bc0afbb0 con 0x55e854185ec0 2024-03-28T18:10:12.972 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:12.972+0000 7f76d5ff0740 1 -- 172.21.15.22:0/3969661961 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f76bc097ef0 con 0x55e854185ec0 2024-03-28T18:10:12.973 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:12.972+0000 7f76b6000640 1 -- 172.21.15.22:0/3969661961 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) v2 ==== 24+0+0 (secure 0 0 0) 0x7f76bc099c40 con 0x55e854185ec0 2024-03-28T18:10:12.973 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:12.972+0000 7f76d5ff0740 10 client.16359 fetch_fsmap learned FSMap version 31 2024-03-28T18:10:12.973 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:12.972+0000 7f76d5ff0740 1 -- 172.21.15.22:0/3969661961 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) v3 -- 0x55e85420e750 con 0x55e854185ec0 2024-03-28T18:10:12.973 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:12.972+0000 7f76b6000640 1 -- 172.21.15.22:0/3969661961 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f76bc097ef0 con 0x55e854185ec0 2024-03-28T18:10:12.973 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:12.972+0000 7f76b6000640 1 -- 172.21.15.22:0/3969661961 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 31) v1 ==== 7950+0+0 (secure 0 0 0) 0x7f76bc010430 con 0x55e854185ec0 2024-03-28T18:10:12.973 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:12.972+0000 7f76d5ff0740 10 client.16359 fetch_fsmap finished waiting for FSMap version 31 2024-03-28T18:10:12.973 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:12.972+0000 7f76d5ff0740 10 client.16359 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-28T18:10:12.973 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:12.973+0000 7f76d5ff0740 20 client.16359 populate_metadata read hostname 'smithi022' 2024-03-28T18:10:12.973 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:12.973+0000 7f76d5ff0740 1 --2- 172.21.15.22:0/3969661961 >> [v2:172.21.15.79:6838/2441542443,v1:172.21.15.79:6839/2441542443] conn(0x55e85420ed80 0x55e8542111a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:10:12.974 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:12.973+0000 7f76d5ff0740 4 client.16359 mds_command: new command op to 34553 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-28T18:10:12.974 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:12.973+0000 7f76d5ff0740 1 -- 172.21.15.22:0/3969661961 --> [v2:172.21.15.79:6838/2441542443,v1:172.21.15.79:6839/2441542443] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55e8540f4ae0 con 0x55e85420ed80 2024-03-28T18:10:12.974 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:12.973+0000 7f76cb400640 1 --2- 172.21.15.22:0/3969661961 >> [v2:172.21.15.79:6838/2441542443,v1:172.21.15.79:6839/2441542443] conn(0x55e85420ed80 0x55e8542111a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:10:12.974 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:12.974+0000 7f76cb400640 1 --2- 172.21.15.22:0/3969661961 >> [v2:172.21.15.79:6838/2441542443,v1:172.21.15.79:6839/2441542443] conn(0x55e85420ed80 0x55e8542111a0 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8227bf58f669ea21 server_cookie=41e61836cae82d72 in_seq=0 out_seq=0 2024-03-28T18:10:12.975 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:12.974+0000 7f76b6000640 10 client.16359 ms_handle_connect on v2:172.21.15.79:6838/2441542443 2024-03-28T18:10:12.975 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:12.975+0000 7f76b6000640 1 -- 172.21.15.22:0/3969661961 <== mds.0 v2:172.21.15.79:6838/2441542443 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+267 (crc 0 0 0) 0x55e8540f4ae0 con 0x55e85420ed80 2024-03-28T18:10:12.975 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:12.975+0000 7f76b6000640 10 client.16359 handle_command_reply: tid=0 2024-03-28T18:10:12.975 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:12.975+0000 7f76d5ff0740 1 client.16359 shutdown 2024-03-28T18:10:12.975 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:12.975+0000 7f76d5ff0740 1 -- 172.21.15.22:0/3969661961 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e854185ec0 msgr2=0x55e8541aaec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:10:12.975 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:12.975+0000 7f76d5ff0740 1 --2- 172.21.15.22:0/3969661961 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e854185ec0 0x55e8541aaec0 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7f76bc09b810 tx=0x7f76bc09faa0 comp rx=0 tx=0).stop 2024-03-28T18:10:12.976 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:12.975+0000 7f76d5ff0740 1 -- 172.21.15.22:0/3969661961 shutdown_connections 2024-03-28T18:10:12.976 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:12.975+0000 7f76d5ff0740 1 --2- 172.21.15.22:0/3969661961 >> [v2:172.21.15.79:6838/2441542443,v1:172.21.15.79:6839/2441542443] conn(0x55e85420ed80 0x55e8542111a0 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:10:12.976 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:12.975+0000 7f76d5ff0740 1 --2- 172.21.15.22:0/3969661961 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e8541a33c0 0x55e8541a86e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:10:12.976 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:12.975+0000 7f76d5ff0740 1 --2- 172.21.15.22:0/3969661961 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55e854186870 0x55e8541ab400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:10:12.976 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:12.975+0000 7f76d5ff0740 1 --2- 172.21.15.22:0/3969661961 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e854185ec0 0x55e8541aaec0 unknown :-1 s=CLOSED pgs=657 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:10:12.976 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:12.975+0000 7f76d5ff0740 1 -- 172.21.15.22:0/3969661961 >> 172.21.15.22:0/3969661961 conn(0x55e854150200 msgr2=0x55e85418c780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:10:12.976 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:12.975+0000 7f76b6000640 0 client.16359 ms_handle_reset on v2:172.21.15.79:6838/2441542443 2024-03-28T18:10:12.976 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:12.976+0000 7f76d5ff0740 1 -- 172.21.15.22:0/3969661961 shutdown_connections 2024-03-28T18:10:12.976 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:12.976+0000 7f76d5ff0740 1 -- 172.21.15.22:0/3969661961 wait complete. 2024-03-28T18:10:12.976 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:12.976+0000 7f76d5ff0740 20 client.16359 trim_cache size 0 max 16384 2024-03-28T18:10:12.977 INFO:teuthology.orchestra.run.smithi022.stdout:{ 2024-03-28T18:10:12.977 INFO:teuthology.orchestra.run.smithi022.stdout: "status": "scrub active (273 inodes in the stack)", 2024-03-28T18:10:12.977 INFO:teuthology.orchestra.run.smithi022.stdout: "scrubs": { 2024-03-28T18:10:12.977 INFO:teuthology.orchestra.run.smithi022.stdout: "6dada245-ed7b-4368-89c1-2734f3acce1b": { 2024-03-28T18:10:12.977 INFO:teuthology.orchestra.run.smithi022.stdout: "path": "/", 2024-03-28T18:10:12.977 INFO:teuthology.orchestra.run.smithi022.stdout: "tag": "6dada245-ed7b-4368-89c1-2734f3acce1b", 2024-03-28T18:10:12.977 INFO:teuthology.orchestra.run.smithi022.stdout: "options": "recursive,force" 2024-03-28T18:10:12.977 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-03-28T18:10:12.977 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-03-28T18:10:12.977 INFO:teuthology.orchestra.run.smithi022.stdout:} 2024-03-28T18:10:12.978 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:12.977+0000 7f76d3200640 1 -- 172.21.15.22:0/1639963463 >> [v2:172.21.15.22:6800/1357365082,v1:172.21.15.22:6801/1357365082] conn(0x7f76b00a2920 msgr2=0x7f76b00a4de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:10:12.978 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:12.977+0000 7f76d3200640 1 --2- 172.21.15.22:0/1639963463 >> [v2:172.21.15.22:6800/1357365082,v1:172.21.15.22:6801/1357365082] conn(0x7f76b00a2920 0x7f76b00a4de0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f76b8005ab0 tx=0x7f76b802bc30 comp rx=0 tx=0).stop 2024-03-28T18:10:12.978 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:12.977+0000 7f76d3200640 1 -- 172.21.15.22:0/1639963463 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f76cc108880 msgr2=0x7f76cc1a0740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:10:12.978 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:12.977+0000 7f76d3200640 1 --2- 172.21.15.22:0/1639963463 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f76cc108880 0x7f76cc1a0740 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7f76bc00e9f0 tx=0x7f76bc00eeb0 comp rx=0 tx=0).stop 2024-03-28T18:10:12.978 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:12.977+0000 7f76d3200640 1 -- 172.21.15.22:0/1639963463 shutdown_connections 2024-03-28T18:10:12.978 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:12.977+0000 7f76d3200640 1 --2- 172.21.15.22:0/1639963463 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f76cc1147f0 0x7f76cc1a5000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:10:12.978 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:12.977+0000 7f76d3200640 1 --2- 172.21.15.22:0/1639963463 >> [v2:172.21.15.22:6800/1357365082,v1:172.21.15.22:6801/1357365082] conn(0x7f76b00a2920 0x7f76b00a4de0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:10:12.979 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:12.977+0000 7f76d3200640 1 --2- 172.21.15.22:0/1639963463 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f76cc1091a0 0x7f76cc1a0c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:10:12.979 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:12.977+0000 7f76d3200640 1 --2- 172.21.15.22:0/1639963463 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f76cc108880 0x7f76cc1a0740 unknown :-1 s=CLOSED pgs=655 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:10:12.979 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:12.977+0000 7f76d3200640 1 -- 172.21.15.22:0/1639963463 >> 172.21.15.22:0/1639963463 conn(0x7f76cc06c780 msgr2=0x7f76cc070360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:10:12.979 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:12.977+0000 7f76d3200640 1 -- 172.21.15.22:0/1639963463 shutdown_connections 2024-03-28T18:10:12.979 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:12.977+0000 7f76d3200640 1 -- 172.21.15.22:0/1639963463 wait complete. 2024-03-28T18:10:12.979 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/sound/alsa/soc/pops_clicks.txt 2024-03-28T18:10:12.989 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/sound/oss/ 2024-03-28T18:10:12.996 INFO:tasks.cephfs.filesystem:scrub status for tag:6dada245-ed7b-4368-89c1-2734f3acce1b - {'path': '/', 'tag': '6dada245-ed7b-4368-89c1-2734f3acce1b', 'options': 'recursive,force'} 2024-03-28T18:10:13.004 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/sound/oss/ALS 2024-03-28T18:10:13.014 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/sound/oss/AudioExcelDSP16 2024-03-28T18:10:13.041 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/sound/oss/CMI8330 2024-03-28T18:10:13.056 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/sound/oss/ESS 2024-03-28T18:10:13.076 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/sound/oss/ESS1868 2024-03-28T18:10:13.097 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/sound/oss/Introduction 2024-03-28T18:10:13.126 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/sound/oss/MultiSound 2024-03-28T18:10:13.147 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/sound/oss/OPL3 2024-03-28T18:10:13.166 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/sound/oss/Opti 2024-03-28T18:10:13.185 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/sound/oss/PAS16 2024-03-28T18:10:13.197 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/sound/oss/PSS 2024-03-28T18:10:13.213 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/sound/oss/PSS-updates 2024-03-28T18:10:13.225 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/sound/oss/README.OSS 2024-03-28T18:10:13.245 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/sound/oss/README.modules 2024-03-28T18:10:13.258 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/sound/oss/README.ymfsb 2024-03-28T18:10:13.271 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/sound/oss/SoundPro 2024-03-28T18:10:13.285 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/sound/oss/Soundblaster 2024-03-28T18:10:13.298 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/sound/oss/Tropez+ 2024-03-28T18:10:13.312 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/sound/oss/VIBRA16 2024-03-28T18:10:13.329 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/sound/oss/WaveArtist 2024-03-28T18:10:13.346 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/sound/oss/btaudio 2024-03-28T18:10:13.361 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/sound/oss/mwave 2024-03-28T18:10:13.376 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/sound/oss/oss-parameters.txt 2024-03-28T18:10:13.391 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/sound/oss/ultrasound 2024-03-28T18:10:13.407 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/sparse.txt 2024-03-28T18:10:13.438 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/spi/ 2024-03-28T18:10:13.450 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/spi/.gitignore 2024-03-28T18:10:13.458 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/spi/00-INDEX 2024-03-28T18:10:13.472 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/spi/Makefile 2024-03-28T18:10:13.486 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/spi/butterfly 2024-03-28T18:10:13.501 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/spi/ep93xx_spi 2024-03-28T18:10:13.515 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/spi/pxa2xx 2024-03-28T18:10:13.531 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/spi/spi-lm70llp 2024-03-28T18:10:13.545 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/spi/spi-sc18is602 2024-03-28T18:10:13.574 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/spi/spi-summary 2024-03-28T18:10:13.624 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/spi/spidev 2024-03-28T18:10:13.646 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/spi/spidev_fdx.c 2024-03-28T18:10:13.661 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/spi/spidev_test.c 2024-03-28T18:10:13.676 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/stable_api_nonsense.txt 2024-03-28T18:10:13.697 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/stable_kernel_rules.txt 2024-03-28T18:10:13.714 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/static-keys.txt 2024-03-28T18:10:13.728 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/svga.txt 2024-03-28T18:10:13.743 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/sysctl/ 2024-03-28T18:10:13.754 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/sysctl/00-INDEX 2024-03-28T18:10:13.761 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/sysctl/README 2024-03-28T18:10:13.776 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/sysctl/abi.txt 2024-03-28T18:10:13.789 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/sysctl/fs.txt 2024-03-28T18:10:13.804 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/sysctl/kernel.txt 2024-03-28T18:10:13.820 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/sysctl/net.txt 2024-03-28T18:10:13.836 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/sysctl/sunrpc.txt 2024-03-28T18:10:13.850 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/sysctl/vm.txt 2024-03-28T18:10:13.869 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/sysfs-rules.txt 2024-03-28T18:10:13.890 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/sysrq.txt 2024-03-28T18:10:13.907 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/target/ 2024-03-28T18:10:13.918 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/target/tcm_mod_builder.py 2024-03-28T18:10:13.930 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/target/tcm_mod_builder.txt 2024-03-28T18:10:13.946 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/target/tcmu-design.txt 2024-03-28T18:10:13.974 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/thermal/ 2024-03-28T18:10:13.990 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/thermal/cpu-cooling-api.txt 2024-03-28T18:10:14.000 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/thermal/exynos_thermal 2024-03-28T18:10:14.016 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/thermal/exynos_thermal_emulation 2024-03-28T18:10:14.030 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/thermal/intel_powerclamp.txt 2024-03-28T18:10:14.046 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/thermal/nouveau_thermal 2024-03-28T18:10:14.062 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/thermal/sysfs-api.txt 2024-03-28T18:10:14.079 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/thermal/x86_pkg_temperature_thermal 2024-03-28T18:10:14.095 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/this_cpu_ops.txt 2024-03-28T18:10:14.118 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/timers/ 2024-03-28T18:10:14.129 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/timers/.gitignore 2024-03-28T18:10:14.138 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/timers/00-INDEX 2024-03-28T18:10:14.157 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/timers/Makefile 2024-03-28T18:10:14.171 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/timers/NO_HZ.txt 2024-03-28T18:10:14.187 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/timers/highres.txt 2024-03-28T18:10:14.205 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/timers/hpet.txt 2024-03-28T18:10:14.220 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/timers/hpet_example.c 2024-03-28T18:10:14.234 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/timers/hrtimers.txt 2024-03-28T18:10:14.251 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/timers/timekeeping.txt 2024-03-28T18:10:14.267 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/timers/timer_stats.txt 2024-03-28T18:10:14.282 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/timers/timers-howto.txt 2024-03-28T18:10:14.302 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/tpm/ 2024-03-28T18:10:14.396 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/tpm/xen-tpmfront.txt 2024-03-28T18:10:14.411 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/trace/ 2024-03-28T18:10:14.425 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:10:14 smithi134 ceph-mon[35037]: scrub queued for path: / 2024-03-28T18:10:14.425 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:10:14 smithi134 ceph-mon[35037]: scrub summary: idle+waiting paths [/] 2024-03-28T18:10:14.426 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:10:14 smithi134 ceph-mon[35037]: scrub summary: active paths [/] 2024-03-28T18:10:14.426 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:10:14 smithi134 ceph-mon[35037]: scrub summary: active paths [/] 2024-03-28T18:10:14.426 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:10:14 smithi134 ceph-mon[35037]: scrub summary: active paths [/] 2024-03-28T18:10:14.426 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:10:14 smithi134 ceph-mon[35037]: scrub summary: idle+waiting paths [/] 2024-03-28T18:10:14.426 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:10:14 smithi134 ceph-mon[35037]: scrub summary: active paths [/] 2024-03-28T18:10:14.426 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:10:14 smithi134 ceph-mon[35037]: scrub summary: idle+waiting paths [/] 2024-03-28T18:10:14.426 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:10:14 smithi134 ceph-mon[35037]: scrub summary: idle+waiting paths [/] 2024-03-28T18:10:14.426 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:10:14 smithi134 ceph-mon[35037]: scrub summary: active paths [/] 2024-03-28T18:10:14.426 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:10:14 smithi134 ceph-mon[35037]: scrub summary: active paths [/] 2024-03-28T18:10:14.426 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:10:14 smithi134 ceph-mon[35037]: scrub summary: active paths [/] 2024-03-28T18:10:14.426 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:10:14 smithi134 ceph-mon[35037]: scrub summary: active paths [/] 2024-03-28T18:10:14.426 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:10:14 smithi134 ceph-mon[35037]: scrub summary: idle+waiting paths [/] 2024-03-28T18:10:14.426 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:10:14 smithi134 ceph-mon[35037]: scrub summary: idle+waiting paths [/] 2024-03-28T18:10:14.426 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:10:14 smithi134 ceph-mon[35037]: pgmap v83: 97 pgs: 97 active+clean; 111 MiB data, 809 MiB used, 1.0 TiB / 1.0 TiB avail; 447 KiB/s rd, 864 KiB/s wr, 131 op/s 2024-03-28T18:10:14.426 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:10:14 smithi134 ceph-mon[35037]: scrub summary: active paths [/] 2024-03-28T18:10:14.427 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:10:14 smithi134 ceph-mon[35037]: scrub summary: active paths [/] 2024-03-28T18:10:14.427 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:10:14 smithi134 ceph-mon[35037]: scrub summary: idle+waiting paths [/] 2024-03-28T18:10:14.427 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:10:14 smithi134 ceph-mon[35037]: scrub summary: active paths [/] 2024-03-28T18:10:14.427 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:10:14 smithi134 ceph-mon[35037]: scrub summary: idle+waiting paths [/] 2024-03-28T18:10:14.427 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:10:14 smithi134 ceph-mon[35037]: scrub summary: idle+waiting paths [/] 2024-03-28T18:10:14.427 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:10:14 smithi134 ceph-mon[35037]: scrub summary: active paths [/] 2024-03-28T18:10:14.427 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:10:14 smithi134 ceph-mon[35037]: scrub summary: idle+waiting paths [/] 2024-03-28T18:10:14.427 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:10:14 smithi134 ceph-mon[35037]: scrub summary: idle+waiting paths [/] 2024-03-28T18:10:14.427 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:10:14 smithi134 ceph-mon[35037]: scrub summary: active paths [/] 2024-03-28T18:10:14.427 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:10:14 smithi134 ceph-mon[35037]: scrub summary: idle+waiting paths [/] 2024-03-28T18:10:14.427 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:10:14 smithi134 ceph-mon[35037]: scrub summary: idle+waiting paths [/] 2024-03-28T18:10:14.427 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:10:14 smithi134 ceph-mon[35037]: scrub summary: idle+waiting paths [/] 2024-03-28T18:10:14.433 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:10:14 smithi079 ceph-mon[35312]: scrub queued for path: / 2024-03-28T18:10:14.433 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:10:14 smithi079 ceph-mon[35312]: scrub summary: idle+waiting paths [/] 2024-03-28T18:10:14.433 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:10:14 smithi079 ceph-mon[35312]: scrub summary: active paths [/] 2024-03-28T18:10:14.433 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:10:14 smithi079 ceph-mon[35312]: scrub summary: active paths [/] 2024-03-28T18:10:14.433 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:10:14 smithi079 ceph-mon[35312]: scrub summary: active paths [/] 2024-03-28T18:10:14.433 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:10:14 smithi079 ceph-mon[35312]: scrub summary: idle+waiting paths [/] 2024-03-28T18:10:14.433 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:10:14 smithi079 ceph-mon[35312]: scrub summary: active paths [/] 2024-03-28T18:10:14.433 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:10:14 smithi079 ceph-mon[35312]: scrub summary: idle+waiting paths [/] 2024-03-28T18:10:14.433 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:10:14 smithi079 ceph-mon[35312]: scrub summary: idle+waiting paths [/] 2024-03-28T18:10:14.433 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:10:14 smithi079 ceph-mon[35312]: scrub summary: active paths [/] 2024-03-28T18:10:14.433 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:10:14 smithi079 ceph-mon[35312]: scrub summary: active paths [/] 2024-03-28T18:10:14.434 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:10:14 smithi079 ceph-mon[35312]: scrub summary: active paths [/] 2024-03-28T18:10:14.434 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:10:14 smithi079 ceph-mon[35312]: scrub summary: active paths [/] 2024-03-28T18:10:14.434 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:10:14 smithi079 ceph-mon[35312]: scrub summary: idle+waiting paths [/] 2024-03-28T18:10:14.434 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:10:14 smithi079 ceph-mon[35312]: scrub summary: idle+waiting paths [/] 2024-03-28T18:10:14.434 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:10:14 smithi079 ceph-mon[35312]: pgmap v83: 97 pgs: 97 active+clean; 111 MiB data, 809 MiB used, 1.0 TiB / 1.0 TiB avail; 447 KiB/s rd, 864 KiB/s wr, 131 op/s 2024-03-28T18:10:14.434 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:10:14 smithi079 ceph-mon[35312]: scrub summary: active paths [/] 2024-03-28T18:10:14.434 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:10:14 smithi079 ceph-mon[35312]: scrub summary: active paths [/] 2024-03-28T18:10:14.434 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:10:14 smithi079 ceph-mon[35312]: scrub summary: idle+waiting paths [/] 2024-03-28T18:10:14.434 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:10:14 smithi079 ceph-mon[35312]: scrub summary: active paths [/] 2024-03-28T18:10:14.434 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:10:14 smithi079 ceph-mon[35312]: scrub summary: idle+waiting paths [/] 2024-03-28T18:10:14.434 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:10:14 smithi079 ceph-mon[35312]: scrub summary: idle+waiting paths [/] 2024-03-28T18:10:14.434 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:10:14 smithi079 ceph-mon[35312]: scrub summary: active paths [/] 2024-03-28T18:10:14.434 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:10:14 smithi079 ceph-mon[35312]: scrub summary: idle+waiting paths [/] 2024-03-28T18:10:14.434 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:10:14 smithi079 ceph-mon[35312]: scrub summary: idle+waiting paths [/] 2024-03-28T18:10:14.434 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:10:14 smithi079 ceph-mon[35312]: scrub summary: active paths [/] 2024-03-28T18:10:14.435 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:10:14 smithi079 ceph-mon[35312]: scrub summary: idle+waiting paths [/] 2024-03-28T18:10:14.435 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:10:14 smithi079 ceph-mon[35312]: scrub summary: idle+waiting paths [/] 2024-03-28T18:10:14.435 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:10:14 smithi079 ceph-mon[35312]: scrub summary: idle+waiting paths [/] 2024-03-28T18:10:14.505 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/trace/coresight.txt 2024-03-28T18:10:14.517 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/trace/events-kmem.txt 2024-03-28T18:10:14.533 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:10:14 smithi022 ceph-mon[31985]: scrub queued for path: / 2024-03-28T18:10:14.533 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:10:14 smithi022 ceph-mon[31985]: scrub summary: idle+waiting paths [/] 2024-03-28T18:10:14.533 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:10:14 smithi022 ceph-mon[31985]: scrub summary: active paths [/] 2024-03-28T18:10:14.533 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:10:14 smithi022 ceph-mon[31985]: scrub summary: active paths [/] 2024-03-28T18:10:14.533 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:10:14 smithi022 ceph-mon[31985]: scrub summary: active paths [/] 2024-03-28T18:10:14.533 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:10:14 smithi022 ceph-mon[31985]: scrub summary: idle+waiting paths [/] 2024-03-28T18:10:14.533 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:10:14 smithi022 ceph-mon[31985]: scrub summary: active paths [/] 2024-03-28T18:10:14.533 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:10:14 smithi022 ceph-mon[31985]: scrub summary: idle+waiting paths [/] 2024-03-28T18:10:14.533 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:10:14 smithi022 ceph-mon[31985]: scrub summary: idle+waiting paths [/] 2024-03-28T18:10:14.534 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:10:14 smithi022 ceph-mon[31985]: scrub summary: active paths [/] 2024-03-28T18:10:14.534 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:10:14 smithi022 ceph-mon[31985]: scrub summary: active paths [/] 2024-03-28T18:10:14.534 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:10:14 smithi022 ceph-mon[31985]: scrub summary: active paths [/] 2024-03-28T18:10:14.534 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:10:14 smithi022 ceph-mon[31985]: scrub summary: active paths [/] 2024-03-28T18:10:14.534 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:10:14 smithi022 ceph-mon[31985]: scrub summary: idle+waiting paths [/] 2024-03-28T18:10:14.534 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:10:14 smithi022 ceph-mon[31985]: scrub summary: idle+waiting paths [/] 2024-03-28T18:10:14.534 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:10:14 smithi022 ceph-mon[31985]: pgmap v83: 97 pgs: 97 active+clean; 111 MiB data, 809 MiB used, 1.0 TiB / 1.0 TiB avail; 447 KiB/s rd, 864 KiB/s wr, 131 op/s 2024-03-28T18:10:14.534 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:10:14 smithi022 ceph-mon[31985]: scrub summary: active paths [/] 2024-03-28T18:10:14.534 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:10:14 smithi022 ceph-mon[31985]: scrub summary: active paths [/] 2024-03-28T18:10:14.534 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:10:14 smithi022 ceph-mon[31985]: scrub summary: idle+waiting paths [/] 2024-03-28T18:10:14.534 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:10:14 smithi022 ceph-mon[31985]: scrub summary: active paths [/] 2024-03-28T18:10:14.534 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:10:14 smithi022 ceph-mon[31985]: scrub summary: idle+waiting paths [/] 2024-03-28T18:10:14.534 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:10:14 smithi022 ceph-mon[31985]: scrub summary: idle+waiting paths [/] 2024-03-28T18:10:14.534 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:10:14 smithi022 ceph-mon[31985]: scrub summary: active paths [/] 2024-03-28T18:10:14.534 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:10:14 smithi022 ceph-mon[31985]: scrub summary: idle+waiting paths [/] 2024-03-28T18:10:14.535 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:10:14 smithi022 ceph-mon[31985]: scrub summary: idle+waiting paths [/] 2024-03-28T18:10:14.535 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:10:14 smithi022 ceph-mon[31985]: scrub summary: active paths [/] 2024-03-28T18:10:14.535 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:10:14 smithi022 ceph-mon[31985]: scrub summary: idle+waiting paths [/] 2024-03-28T18:10:14.535 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:10:14 smithi022 ceph-mon[31985]: scrub summary: idle+waiting paths [/] 2024-03-28T18:10:14.535 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:10:14 smithi022 ceph-mon[31985]: scrub summary: idle+waiting paths [/] 2024-03-28T18:10:14.538 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/trace/events-nmi.txt 2024-03-28T18:10:14.558 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/trace/events-power.txt 2024-03-28T18:10:14.571 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/trace/events.txt 2024-03-28T18:10:14.621 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/trace/ftrace-design.txt 2024-03-28T18:10:14.642 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/trace/ftrace.txt 2024-03-28T18:10:14.672 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/trace/function-graph-fold.vim 2024-03-28T18:10:14.686 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/trace/kprobetrace.txt 2024-03-28T18:10:14.699 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/trace/mmiotrace.txt 2024-03-28T18:10:14.714 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/trace/postprocess/ 2024-03-28T18:10:14.725 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/trace/postprocess/trace-pagealloc-postprocess.pl 2024-03-28T18:10:14.735 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/trace/postprocess/trace-vmscan-postprocess.pl 2024-03-28T18:10:14.752 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/trace/ring-buffer-design.txt 2024-03-28T18:10:14.774 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/trace/tracepoint-analysis.txt 2024-03-28T18:10:14.790 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/trace/tracepoints.txt 2024-03-28T18:10:14.804 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/trace/uprobetracer.txt 2024-03-28T18:10:14.821 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/unaligned-memory-access.txt 2024-03-28T18:10:14.843 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/unicode.txt 2024-03-28T18:10:14.861 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/unshare.txt 2024-03-28T18:10:14.885 INFO:tasks.workunit.client.0.smithi022.stdout:linux-4.0.5/Documentation/usb/ 2024-03-28T18:10:15.183 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:10:14 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[64932]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-2145-g3be9a5c9/rpm/el8/BUILD/ceph-19.0.0-2145-g3be9a5c9/src/mds/Locker.cc: In function 'void Locker::set_xlocks_done(MutationImpl*, bool)' thread 7f538a000700 time 2024-03-28T18:10:14.896621+0000 2024-03-28T18:10:15.183 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:10:14 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[64932]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-2145-g3be9a5c9/rpm/el8/BUILD/ceph-19.0.0-2145-g3be9a5c9/src/mds/Locker.cc: 626: FAILED ceph_assert(obj->is_auth()) 2024-03-28T18:10:15.183 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:10:14 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[64932]: ceph version 19.0.0-2145-g3be9a5c9 (3be9a5c9bc793e11e2800a8c0c696e8b46742033) squid (dev) 2024-03-28T18:10:15.184 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:10:14 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[64932]: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x135) [0x7f53959194a1] 2024-03-28T18:10:15.184 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:10:14 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[64932]: 2: /usr/lib64/ceph/libceph-common.so.2(+0x2d2667) [0x7f5395919667] 2024-03-28T18:10:15.184 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:10:14 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[64932]: 3: (Locker::set_xlocks_done(MutationImpl*, bool)+0x493) [0x5564d3fa8223] 2024-03-28T18:10:15.184 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:10:14 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[64932]: 4: (Server::early_reply(boost::intrusive_ptr const&, CInode*, CDentry*)+0x3e6) [0x5564d3e225d6] 2024-03-28T18:10:15.184 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:10:14 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[64932]: 5: (Server::journal_and_reply(boost::intrusive_ptr const&, CInode*, CDentry*, LogEvent*, MDSLogContextBase*)+0xc1) [0x5564d3e22da1] 2024-03-28T18:10:15.184 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:10:14 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[64932]: 6: (Server::handle_client_mkdir(boost::intrusive_ptr const&)+0x5ba) [0x5564d3e586aa] 2024-03-28T18:10:15.184 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:10:14 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[64932]: 7: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0x733) [0x5564d3e75233] 2024-03-28T18:10:15.184 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:10:14 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[64932]: 8: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x5564d3e77197] 2024-03-28T18:10:15.184 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:10:14 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[64932]: 9: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x5564d3d7611b] 2024-03-28T18:10:15.184 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:10:14 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[64932]: 10: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x5564d3d766bc] 2024-03-28T18:10:15.184 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:10:14 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[64932]: 11: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bf) [0x5564d3d601ff] 2024-03-28T18:10:15.184 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:10:14 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[64932]: 12: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x478) [0x7f5395bab488] 2024-03-28T18:10:15.184 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:10:14 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[64932]: 13: (DispatchQueue::entry()+0x51f) [0x7f5395ba860f] 2024-03-28T18:10:15.184 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:10:14 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[64932]: 14: (DispatchQueue::DispatchThread::entry()+0x11) [0x7f5395c7be81] 2024-03-28T18:10:15.185 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:10:14 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[64932]: 15: /lib64/libpthread.so.0(+0x81ca) [0x7f53946961ca] 2024-03-28T18:10:15.185 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:10:14 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[64932]: 16: clone() 2024-03-28T18:10:15.185 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:10:14 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[64932]: 2024-03-28T18:10:14.897+0000 7f538a000700 -1 /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-2145-g3be9a5c9/rpm/el8/BUILD/ceph-19.0.0-2145-g3be9a5c9/src/mds/Locker.cc: In function 'void Locker::set_xlocks_done(MutationImpl*, bool)' thread 7f538a000700 time 2024-03-28T18:10:14.896621+0000 2024-03-28T18:10:15.185 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:10:14 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[64932]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-2145-g3be9a5c9/rpm/el8/BUILD/ceph-19.0.0-2145-g3be9a5c9/src/mds/Locker.cc: 626: FAILED ceph_assert(obj->is_auth()) 2024-03-28T18:10:15.185 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:10:14 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[64932]: 2024-03-28T18:10:15.185 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:10:14 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[64932]: ceph version 19.0.0-2145-g3be9a5c9 (3be9a5c9bc793e11e2800a8c0c696e8b46742033) squid (dev) 2024-03-28T18:10:15.186 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:10:14 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[64932]: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x135) [0x7f53959194a1] 2024-03-28T18:10:15.186 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:10:14 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[64932]: 2: /usr/lib64/ceph/libceph-common.so.2(+0x2d2667) [0x7f5395919667] 2024-03-28T18:10:15.186 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:10:14 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[64932]: 3: (Locker::set_xlocks_done(MutationImpl*, bool)+0x493) [0x5564d3fa8223] 2024-03-28T18:10:15.186 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:10:14 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[64932]: 4: (Server::early_reply(boost::intrusive_ptr const&, CInode*, CDentry*)+0x3e6) [0x5564d3e225d6] 2024-03-28T18:10:15.186 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:10:14 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[64932]: 5: (Server::journal_and_reply(boost::intrusive_ptr const&, CInode*, CDentry*, LogEvent*, MDSLogContextBase*)+0xc1) [0x5564d3e22da1] 2024-03-28T18:10:15.186 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:10:14 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[64932]: 6: (Server::handle_client_mkdir(boost::intrusive_ptr const&)+0x5ba) [0x5564d3e586aa] 2024-03-28T18:10:15.186 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:10:14 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[64932]: 7: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0x733) [0x5564d3e75233] 2024-03-28T18:10:15.186 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:10:14 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[64932]: 8: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x5564d3e77197] 2024-03-28T18:10:15.186 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:10:14 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[64932]: 9: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x5564d3d7611b] 2024-03-28T18:10:15.186 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:10:14 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[64932]: 10: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x5564d3d766bc] 2024-03-28T18:10:15.186 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:10:14 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[64932]: 11: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bf) [0x5564d3d601ff] 2024-03-28T18:10:15.186 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:10:14 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[64932]: 12: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x478) [0x7f5395bab488] 2024-03-28T18:10:15.186 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:10:14 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[64932]: 13: (DispatchQueue::entry()+0x51f) [0x7f5395ba860f] 2024-03-28T18:10:15.187 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:10:14 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[64932]: 14: (DispatchQueue::DispatchThread::entry()+0x11) [0x7f5395c7be81] 2024-03-28T18:10:15.187 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:10:14 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[64932]: 15: /lib64/libpthread.so.0(+0x81ca) [0x7f53946961ca] 2024-03-28T18:10:15.187 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:10:14 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[64932]: 16: clone() 2024-03-28T18:10:15.187 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:10:14 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[64932]: 2024-03-28T18:10:15.187 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:10:14 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[64932]: *** Caught signal (Aborted) ** 2024-03-28T18:10:15.187 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:10:14 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[64932]: in thread 7f538a000700 thread_name:ms_dispatch 2024-03-28T18:10:15.187 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:10:14 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[64932]: ceph version 19.0.0-2145-g3be9a5c9 (3be9a5c9bc793e11e2800a8c0c696e8b46742033) squid (dev) 2024-03-28T18:10:15.187 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:10:14 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[64932]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7f53946a0d20] 2024-03-28T18:10:15.187 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:10:14 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[64932]: 2: gsignal() 2024-03-28T18:10:15.187 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:10:14 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[64932]: 3: abort() 2024-03-28T18:10:15.187 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:10:14 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[64932]: 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x18f) [0x7f53959194fb] 2024-03-28T18:10:15.187 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:10:14 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[64932]: 5: /usr/lib64/ceph/libceph-common.so.2(+0x2d2667) [0x7f5395919667] 2024-03-28T18:10:15.187 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:10:14 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[64932]: 6: (Locker::set_xlocks_done(MutationImpl*, bool)+0x493) [0x5564d3fa8223] 2024-03-28T18:10:15.187 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:10:14 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[64932]: 7: (Server::early_reply(boost::intrusive_ptr const&, CInode*, CDentry*)+0x3e6) [0x5564d3e225d6] 2024-03-28T18:10:15.187 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:10:14 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[64932]: 8: (Server::journal_and_reply(boost::intrusive_ptr const&, CInode*, CDentry*, LogEvent*, MDSLogContextBase*)+0xc1) [0x5564d3e22da1] 2024-03-28T18:10:15.187 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:10:14 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[64932]: 9: (Server::handle_client_mkdir(boost::intrusive_ptr const&)+0x5ba) [0x5564d3e586aa] 2024-03-28T18:10:15.188 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:10:14 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[64932]: 10: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0x733) [0x5564d3e75233] 2024-03-28T18:10:15.188 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:10:14 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[64932]: 11: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x5564d3e77197] 2024-03-28T18:10:15.188 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:10:14 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[64932]: 12: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x5564d3d7611b] 2024-03-28T18:10:15.188 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:10:14 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[64932]: 13: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x5564d3d766bc] 2024-03-28T18:10:15.188 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:10:14 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[64932]: 14: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bf) [0x5564d3d601ff] 2024-03-28T18:10:15.188 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:10:14 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[64932]: 15: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x478) [0x7f5395bab488] 2024-03-28T18:10:15.188 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:10:14 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[64932]: 16: (DispatchQueue::entry()+0x51f) [0x7f5395ba860f] 2024-03-28T18:10:15.188 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:10:14 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[64932]: 17: (DispatchQueue::DispatchThread::entry()+0x11) [0x7f5395c7be81] 2024-03-28T18:10:15.188 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:10:14 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[64932]: 18: /lib64/libpthread.so.0(+0x81ca) [0x7f53946961ca] 2024-03-28T18:10:15.188 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:10:14 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[64932]: 19: clone() 2024-03-28T18:10:15.188 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:10:14 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[64932]: 2024-03-28T18:10:14.900+0000 7f538a000700 -1 *** Caught signal (Aborted) ** 2024-03-28T18:10:15.188 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:10:14 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[64932]: in thread 7f538a000700 thread_name:ms_dispatch 2024-03-28T18:10:15.188 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:10:14 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[64932]: 2024-03-28T18:10:15.188 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:10:14 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[64932]: ceph version 19.0.0-2145-g3be9a5c9 (3be9a5c9bc793e11e2800a8c0c696e8b46742033) squid (dev) 2024-03-28T18:10:15.189 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:10:14 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[64932]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7f53946a0d20] 2024-03-28T18:10:15.189 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:10:14 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[64932]: 2: gsignal() 2024-03-28T18:10:15.189 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:10:14 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[64932]: 3: abort() 2024-03-28T18:10:15.189 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:10:14 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[64932]: 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x18f) [0x7f53959194fb] 2024-03-28T18:10:15.189 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:10:14 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[64932]: 5: /usr/lib64/ceph/libceph-common.so.2(+0x2d2667) [0x7f5395919667] 2024-03-28T18:10:15.190 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:10:14 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[64932]: 6: (Locker::set_xlocks_done(MutationImpl*, bool)+0x493) [0x5564d3fa8223] 2024-03-28T18:10:15.190 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:10:14 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[64932]: 7: (Server::early_reply(boost::intrusive_ptr const&, CInode*, CDentry*)+0x3e6) [0x5564d3e225d6] 2024-03-28T18:10:15.190 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:10:14 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[64932]: 8: (Server::journal_and_reply(boost::intrusive_ptr const&, CInode*, CDentry*, LogEvent*, MDSLogContextBase*)+0xc1) [0x5564d3e22da1] 2024-03-28T18:10:15.190 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:10:14 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[64932]: 9: (Server::handle_client_mkdir(boost::intrusive_ptr const&)+0x5ba) [0x5564d3e586aa] 2024-03-28T18:10:15.190 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:10:14 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[64932]: 10: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0x733) [0x5564d3e75233] 2024-03-28T18:10:15.190 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:10:14 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[64932]: 11: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x5564d3e77197] 2024-03-28T18:10:15.190 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:10:14 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[64932]: 12: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x5564d3d7611b] 2024-03-28T18:10:15.190 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:10:14 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[64932]: 13: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x5564d3d766bc] 2024-03-28T18:10:15.190 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:10:14 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[64932]: 14: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bf) [0x5564d3d601ff] 2024-03-28T18:10:15.190 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:10:14 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[64932]: 15: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x478) [0x7f5395bab488] 2024-03-28T18:10:15.190 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:10:14 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[64932]: 16: (DispatchQueue::entry()+0x51f) [0x7f5395ba860f] 2024-03-28T18:10:15.191 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:10:14 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[64932]: 17: (DispatchQueue::DispatchThread::entry()+0x11) [0x7f5395c7be81] 2024-03-28T18:10:15.191 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:10:14 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[64932]: 18: /lib64/libpthread.so.0(+0x81ca) [0x7f53946961ca] 2024-03-28T18:10:15.191 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:10:14 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[64932]: 19: clone() 2024-03-28T18:10:15.191 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:10:14 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[64932]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-03-28T18:10:15.191 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:10:14 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[64932]: 2024-03-28T18:10:15.191 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:10:14 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[64932]: -1> 2024-03-28T18:10:14.897+0000 7f538a000700 -1 /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-2145-g3be9a5c9/rpm/el8/BUILD/ceph-19.0.0-2145-g3be9a5c9/src/mds/Locker.cc: In function 'void Locker::set_xlocks_done(MutationImpl*, bool)' thread 7f538a000700 time 2024-03-28T18:10:14.896621+0000 2024-03-28T18:10:15.191 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:10:14 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[64932]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-2145-g3be9a5c9/rpm/el8/BUILD/ceph-19.0.0-2145-g3be9a5c9/src/mds/Locker.cc: 626: FAILED ceph_assert(obj->is_auth()) 2024-03-28T18:10:15.191 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:10:14 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[64932]: 2024-03-28T18:10:15.191 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:10:14 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[64932]: ceph version 19.0.0-2145-g3be9a5c9 (3be9a5c9bc793e11e2800a8c0c696e8b46742033) squid (dev) 2024-03-28T18:10:15.191 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:10:14 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[64932]: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x135) [0x7f53959194a1] 2024-03-28T18:10:15.191 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:10:14 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[64932]: 2: /usr/lib64/ceph/libceph-common.so.2(+0x2d2667) [0x7f5395919667] 2024-03-28T18:10:15.191 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:10:14 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[64932]: 3: (Locker::set_xlocks_done(MutationImpl*, bool)+0x493) [0x5564d3fa8223] 2024-03-28T18:10:15.191 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:10:14 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[64932]: 4: (Server::early_reply(boost::intrusive_ptr const&, CInode*, CDentry*)+0x3e6) [0x5564d3e225d6] 2024-03-28T18:10:15.191 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:10:14 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[64932]: 5: (Server::journal_and_reply(boost::intrusive_ptr const&, CInode*, CDentry*, LogEvent*, MDSLogContextBase*)+0xc1) [0x5564d3e22da1] 2024-03-28T18:10:15.191 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:10:14 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[64932]: 6: (Server::handle_client_mkdir(boost::intrusive_ptr const&)+0x5ba) [0x5564d3e586aa] 2024-03-28T18:10:15.192 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:10:14 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[64932]: 7: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0x733) [0x5564d3e75233] 2024-03-28T18:10:15.192 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:10:14 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[64932]: 8: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x5564d3e77197] 2024-03-28T18:10:15.192 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:10:14 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[64932]: 9: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x5564d3d7611b] 2024-03-28T18:10:15.192 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:10:14 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[64932]: 10: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x5564d3d766bc] 2024-03-28T18:10:15.192 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:10:14 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[64932]: 11: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bf) [0x5564d3d601ff] 2024-03-28T18:10:15.192 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:10:14 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[64932]: 12: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x478) [0x7f5395bab488] 2024-03-28T18:10:15.192 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:10:14 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[64932]: 13: (DispatchQueue::entry()+0x51f) [0x7f5395ba860f] 2024-03-28T18:10:15.192 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:10:14 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[64932]: 14: (DispatchQueue::DispatchThread::entry()+0x11) [0x7f5395c7be81] 2024-03-28T18:10:15.192 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:10:14 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[64932]: 15: /lib64/libpthread.so.0(+0x81ca) [0x7f53946961ca] 2024-03-28T18:10:15.192 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:10:14 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[64932]: 16: clone() 2024-03-28T18:10:15.192 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:10:14 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[64932]: 2024-03-28T18:10:15.192 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:10:14 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[64932]: 0> 2024-03-28T18:10:14.900+0000 7f538a000700 -1 *** Caught signal (Aborted) ** 2024-03-28T18:10:15.192 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:10:14 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[64932]: in thread 7f538a000700 thread_name:ms_dispatch 2024-03-28T18:10:15.192 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:10:14 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[64932]: 2024-03-28T18:10:15.192 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:10:14 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[64932]: ceph version 19.0.0-2145-g3be9a5c9 (3be9a5c9bc793e11e2800a8c0c696e8b46742033) squid (dev) 2024-03-28T18:10:15.192 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:10:14 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[64932]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7f53946a0d20] 2024-03-28T18:10:15.192 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:10:14 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[64932]: 2: gsignal() 2024-03-28T18:10:15.193 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:10:14 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[64932]: 3: abort() 2024-03-28T18:10:15.193 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:10:14 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[64932]: 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x18f) [0x7f53959194fb] 2024-03-28T18:10:15.193 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:10:14 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[64932]: 5: /usr/lib64/ceph/libceph-common.so.2(+0x2d2667) [0x7f5395919667] 2024-03-28T18:10:15.193 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:10:14 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[64932]: 6: (Locker::set_xlocks_done(MutationImpl*, bool)+0x493) [0x5564d3fa8223] 2024-03-28T18:10:15.193 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:10:14 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[64932]: 7: (Server::early_reply(boost::intrusive_ptr const&, CInode*, CDentry*)+0x3e6) [0x5564d3e225d6] 2024-03-28T18:10:15.193 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:10:14 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[64932]: 8: (Server::journal_and_reply(boost::intrusive_ptr const&, CInode*, CDentry*, LogEvent*, MDSLogContextBase*)+0xc1) [0x5564d3e22da1] 2024-03-28T18:10:15.193 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:10:14 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[64932]: 9: (Server::handle_client_mkdir(boost::intrusive_ptr const&)+0x5ba) [0x5564d3e586aa] 2024-03-28T18:10:15.193 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:10:14 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[64932]: 10: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0x733) [0x5564d3e75233] 2024-03-28T18:10:15.193 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:10:14 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[64932]: 11: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x5564d3e77197] 2024-03-28T18:10:15.193 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:10:14 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[64932]: 12: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x5564d3d7611b] 2024-03-28T18:10:15.193 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:10:14 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[64932]: 13: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x5564d3d766bc] 2024-03-28T18:10:15.193 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:10:14 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[64932]: 14: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bf) [0x5564d3d601ff] 2024-03-28T18:10:15.193 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:10:14 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[64932]: 15: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x478) [0x7f5395bab488] 2024-03-28T18:10:15.193 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:10:14 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[64932]: 16: (DispatchQueue::entry()+0x51f) [0x7f5395ba860f] 2024-03-28T18:10:15.193 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:10:14 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[64932]: 17: (DispatchQueue::DispatchThread::entry()+0x11) [0x7f5395c7be81] 2024-03-28T18:10:15.193 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:10:14 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[64932]: 18: /lib64/libpthread.so.0(+0x81ca) [0x7f53946961ca] 2024-03-28T18:10:15.194 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:10:14 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[64932]: 19: clone() 2024-03-28T18:10:15.194 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:10:14 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[64932]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-03-28T18:10:15.194 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:10:14 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[64932]: 2024-03-28T18:10:15.194 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:10:14 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[64932]: -1> 2024-03-28T18:10:14.897+0000 7f538a000700 -1 /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-2145-g3be9a5c9/rpm/el8/BUILD/ceph-19.0.0-2145-g3be9a5c9/src/mds/Locker.cc: In function 'void Locker::set_xlocks_done(MutationImpl*, bool)' thread 7f538a000700 time 2024-03-28T18:10:14.896621+0000 2024-03-28T18:10:15.194 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:10:14 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[64932]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-2145-g3be9a5c9/rpm/el8/BUILD/ceph-19.0.0-2145-g3be9a5c9/src/mds/Locker.cc: 626: FAILED ceph_assert(obj->is_auth()) 2024-03-28T18:10:15.194 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:10:14 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[64932]: 2024-03-28T18:10:15.194 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:10:14 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[64932]: ceph version 19.0.0-2145-g3be9a5c9 (3be9a5c9bc793e11e2800a8c0c696e8b46742033) squid (dev) 2024-03-28T18:10:15.194 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:10:14 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[64932]: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x135) [0x7f53959194a1] 2024-03-28T18:10:15.194 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:10:14 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[64932]: 2: /usr/lib64/ceph/libceph-common.so.2(+0x2d2667) [0x7f5395919667] 2024-03-28T18:10:15.194 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:10:14 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[64932]: 3: (Locker::set_xlocks_done(MutationImpl*, bool)+0x493) [0x5564d3fa8223] 2024-03-28T18:10:15.194 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:10:14 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[64932]: 4: (Server::early_reply(boost::intrusive_ptr const&, CInode*, CDentry*)+0x3e6) [0x5564d3e225d6] 2024-03-28T18:10:15.194 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:10:14 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[64932]: 5: (Server::journal_and_reply(boost::intrusive_ptr const&, CInode*, CDentry*, LogEvent*, MDSLogContextBase*)+0xc1) [0x5564d3e22da1] 2024-03-28T18:10:15.194 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:10:14 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[64932]: 6: (Server::handle_client_mkdir(boost::intrusive_ptr const&)+0x5ba) [0x5564d3e586aa] 2024-03-28T18:10:15.194 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:10:14 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[64932]: 7: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0x733) [0x5564d3e75233] 2024-03-28T18:10:15.195 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:10:14 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[64932]: 8: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x5564d3e77197] 2024-03-28T18:10:15.195 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:10:14 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[64932]: 9: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x5564d3d7611b] 2024-03-28T18:10:15.195 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:10:14 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[64932]: 10: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x5564d3d766bc] 2024-03-28T18:10:15.195 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:10:14 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[64932]: 11: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bf) [0x5564d3d601ff] 2024-03-28T18:10:15.195 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:10:14 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[64932]: 12: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x478) [0x7f5395bab488] 2024-03-28T18:10:15.195 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:10:14 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[64932]: 13: (DispatchQueue::entry()+0x51f) [0x7f5395ba860f] 2024-03-28T18:10:15.195 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:10:14 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[64932]: 14: (DispatchQueue::DispatchThread::entry()+0x11) [0x7f5395c7be81] 2024-03-28T18:10:15.195 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:10:14 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[64932]: 15: /lib64/libpthread.so.0(+0x81ca) [0x7f53946961ca] 2024-03-28T18:10:15.195 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:10:14 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[64932]: 16: clone() 2024-03-28T18:10:15.195 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:10:14 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[64932]: 2024-03-28T18:10:15.195 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:10:14 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[64932]: 0> 2024-03-28T18:10:14.900+0000 7f538a000700 -1 *** Caught signal (Aborted) ** 2024-03-28T18:10:15.195 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:10:14 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[64932]: in thread 7f538a000700 thread_name:ms_dispatch 2024-03-28T18:10:15.195 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:10:14 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[64932]: 2024-03-28T18:10:15.195 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:10:14 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[64932]: ceph version 19.0.0-2145-g3be9a5c9 (3be9a5c9bc793e11e2800a8c0c696e8b46742033) squid (dev) 2024-03-28T18:10:15.196 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:10:14 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[64932]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7f53946a0d20] 2024-03-28T18:10:15.196 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:10:14 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[64932]: 2: gsignal() 2024-03-28T18:10:15.196 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:10:14 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[64932]: 3: abort() 2024-03-28T18:10:15.196 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:10:14 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[64932]: 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x18f) [0x7f53959194fb] 2024-03-28T18:10:15.196 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:10:14 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[64932]: 5: /usr/lib64/ceph/libceph-common.so.2(+0x2d2667) [0x7f5395919667] 2024-03-28T18:10:15.196 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:10:14 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[64932]: 6: (Locker::set_xlocks_done(MutationImpl*, bool)+0x493) [0x5564d3fa8223] 2024-03-28T18:10:15.196 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:10:14 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[64932]: 7: (Server::early_reply(boost::intrusive_ptr const&, CInode*, CDentry*)+0x3e6) [0x5564d3e225d6] 2024-03-28T18:10:15.196 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:10:14 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[64932]: 8: (Server::journal_and_reply(boost::intrusive_ptr const&, CInode*, CDentry*, LogEvent*, MDSLogContextBase*)+0xc1) [0x5564d3e22da1] 2024-03-28T18:10:15.196 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:10:14 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[64932]: 9: (Server::handle_client_mkdir(boost::intrusive_ptr const&)+0x5ba) [0x5564d3e586aa] 2024-03-28T18:10:15.196 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:10:14 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[64932]: 10: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0x733) [0x5564d3e75233] 2024-03-28T18:10:15.196 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:10:14 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[64932]: 11: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x5564d3e77197] 2024-03-28T18:10:15.196 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:10:14 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[64932]: 12: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x5564d3d7611b] 2024-03-28T18:10:15.196 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:10:14 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[64932]: 13: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x5564d3d766bc] 2024-03-28T18:10:15.196 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:10:14 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[64932]: 14: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bf) [0x5564d3d601ff] 2024-03-28T18:10:15.196 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:10:14 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[64932]: 15: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x478) [0x7f5395bab488] 2024-03-28T18:10:15.197 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:10:14 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[64932]: 16: (DispatchQueue::entry()+0x51f) [0x7f5395ba860f] 2024-03-28T18:10:15.197 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:10:14 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[64932]: 17: (DispatchQueue::DispatchThread::entry()+0x11) [0x7f5395c7be81] 2024-03-28T18:10:15.197 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:10:14 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[64932]: 18: /lib64/libpthread.so.0(+0x81ca) [0x7f53946961ca] 2024-03-28T18:10:15.197 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:10:14 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[64932]: 19: clone() 2024-03-28T18:10:15.197 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:10:14 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[64932]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-03-28T18:10:15.197 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:10:14 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[64932]: 2024-03-28T18:10:15.197 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:10:15 smithi079 podman[68941]: 2024-03-28 18:10:15.037377911 +0000 UTC m=+0.037835630 container died 6faabd85524be22b28f413c870b774965a25d51a2d6fc4b8068370cff2a2f2d8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.build-date=20240319, maintainer=Guillaume Abrioux , RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30) 2024-03-28T18:10:16.146 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:10:15 smithi079 podman[68941]: 2024-03-28 18:10:15.850444368 +0000 UTC m=+0.850902093 container remove 6faabd85524be22b28f413c870b774965a25d51a2d6fc4b8068370cff2a2f2d8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h, ceph=True, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240319, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_BRANCH=HEAD, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux ) 2024-03-28T18:10:16.146 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:10:15 smithi079 systemd[1]: ceph-fd220748-ed2b-11ee-95d0-87774f69a715@mds.h.service: Main process exited, code=exited, status=134/n/a 2024-03-28T18:10:16.424 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:10:16 smithi134 ceph-mon[35037]: pgmap v84: 97 pgs: 97 active+clean; 115 MiB data, 808 MiB used, 1.0 TiB / 1.0 TiB avail; 588 KiB/s rd, 1.3 MiB/s wr, 197 op/s 2024-03-28T18:10:16.432 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:10:16 smithi079 ceph-mon[35312]: pgmap v84: 97 pgs: 97 active+clean; 115 MiB data, 808 MiB used, 1.0 TiB / 1.0 TiB avail; 588 KiB/s rd, 1.3 MiB/s wr, 197 op/s 2024-03-28T18:10:16.432 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:10:16 smithi079 systemd[1]: ceph-fd220748-ed2b-11ee-95d0-87774f69a715@mds.h.service: Failed with result 'exit-code'. 2024-03-28T18:10:16.433 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:10:16 smithi079 systemd[1]: ceph-fd220748-ed2b-11ee-95d0-87774f69a715@mds.h.service: Consumed 3min 25.189s CPU time. 2024-03-28T18:10:16.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:10:16 smithi022 ceph-mon[31985]: pgmap v84: 97 pgs: 97 active+clean; 115 MiB data, 808 MiB used, 1.0 TiB / 1.0 TiB avail; 588 KiB/s rd, 1.3 MiB/s wr, 197 op/s 2024-03-28T18:10:18.425 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:10:18 smithi134 ceph-mon[35037]: pgmap v85: 97 pgs: 97 active+clean; 115 MiB data, 808 MiB used, 1.0 TiB / 1.0 TiB avail; 397 KiB/s rd, 951 KiB/s wr, 129 op/s 2024-03-28T18:10:18.432 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:10:18 smithi079 ceph-mon[35312]: pgmap v85: 97 pgs: 97 active+clean; 115 MiB data, 808 MiB used, 1.0 TiB / 1.0 TiB avail; 397 KiB/s rd, 951 KiB/s wr, 129 op/s 2024-03-28T18:10:18.531 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:10:18 smithi022 ceph-mon[31985]: pgmap v85: 97 pgs: 97 active+clean; 115 MiB data, 808 MiB used, 1.0 TiB / 1.0 TiB avail; 397 KiB/s rd, 951 KiB/s wr, 129 op/s 2024-03-28T18:10:20.424 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:10:20 smithi134 ceph-mon[35037]: pgmap v86: 97 pgs: 97 active+clean; 115 MiB data, 808 MiB used, 1.0 TiB / 1.0 TiB avail; 397 KiB/s rd, 951 KiB/s wr, 129 op/s 2024-03-28T18:10:20.425 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:10:20 smithi134 ceph-mon[35037]: scrub summary: idle 2024-03-28T18:10:20.425 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:10:20 smithi134 ceph-mon[35037]: scrub summary: idle 2024-03-28T18:10:20.425 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:10:20 smithi134 ceph-mon[35037]: scrub summary: idle 2024-03-28T18:10:20.425 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:10:20 smithi134 ceph-mon[35037]: scrub summary: idle 2024-03-28T18:10:20.432 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:10:20 smithi079 ceph-mon[35312]: pgmap v86: 97 pgs: 97 active+clean; 115 MiB data, 808 MiB used, 1.0 TiB / 1.0 TiB avail; 397 KiB/s rd, 951 KiB/s wr, 129 op/s 2024-03-28T18:10:20.432 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:10:20 smithi079 ceph-mon[35312]: scrub summary: idle 2024-03-28T18:10:20.432 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:10:20 smithi079 ceph-mon[35312]: scrub summary: idle 2024-03-28T18:10:20.432 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:10:20 smithi079 ceph-mon[35312]: scrub summary: idle 2024-03-28T18:10:20.432 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:10:20 smithi079 ceph-mon[35312]: scrub summary: idle 2024-03-28T18:10:20.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:10:20 smithi022 ceph-mon[31985]: pgmap v86: 97 pgs: 97 active+clean; 115 MiB data, 808 MiB used, 1.0 TiB / 1.0 TiB avail; 397 KiB/s rd, 951 KiB/s wr, 129 op/s 2024-03-28T18:10:20.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:10:20 smithi022 ceph-mon[31985]: scrub summary: idle 2024-03-28T18:10:20.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:10:20 smithi022 ceph-mon[31985]: scrub summary: idle 2024-03-28T18:10:20.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:10:20 smithi022 ceph-mon[31985]: scrub summary: idle 2024-03-28T18:10:20.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:10:20 smithi022 ceph-mon[31985]: scrub summary: idle 2024-03-28T18:10:22.424 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:10:22 smithi134 ceph-mon[35037]: pgmap v87: 97 pgs: 97 active+clean; 116 MiB data, 813 MiB used, 1.0 TiB / 1.0 TiB avail; 452 KiB/s rd, 1.0 MiB/s wr, 145 op/s 2024-03-28T18:10:22.432 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:10:22 smithi079 ceph-mon[35312]: pgmap v87: 97 pgs: 97 active+clean; 116 MiB data, 813 MiB used, 1.0 TiB / 1.0 TiB avail; 452 KiB/s rd, 1.0 MiB/s wr, 145 op/s 2024-03-28T18:10:22.531 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:10:22 smithi022 ceph-mon[31985]: pgmap v87: 97 pgs: 97 active+clean; 116 MiB data, 813 MiB used, 1.0 TiB / 1.0 TiB avail; 452 KiB/s rd, 1.0 MiB/s wr, 145 op/s 2024-03-28T18:10:24.424 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:10:24 smithi134 ceph-mon[35037]: pgmap v88: 97 pgs: 97 active+clean; 116 MiB data, 813 MiB used, 1.0 TiB / 1.0 TiB avail; 196 KiB/s rd, 518 KiB/s wr, 81 op/s 2024-03-28T18:10:24.432 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:10:24 smithi079 ceph-mon[35312]: pgmap v88: 97 pgs: 97 active+clean; 116 MiB data, 813 MiB used, 1.0 TiB / 1.0 TiB avail; 196 KiB/s rd, 518 KiB/s wr, 81 op/s 2024-03-28T18:10:24.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:10:24 smithi022 ceph-mon[31985]: pgmap v88: 97 pgs: 97 active+clean; 116 MiB data, 813 MiB used, 1.0 TiB / 1.0 TiB avail; 196 KiB/s rd, 518 KiB/s wr, 81 op/s 2024-03-28T18:10:26.433 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:10:26 smithi079 ceph-mon[35312]: pgmap v89: 97 pgs: 97 active+clean; 116 MiB data, 814 MiB used, 1.0 TiB / 1.0 TiB avail; 232 KiB/s rd, 552 KiB/s wr, 86 op/s 2024-03-28T18:10:26.531 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:10:26 smithi022 ceph-mon[31985]: pgmap v89: 97 pgs: 97 active+clean; 116 MiB data, 814 MiB used, 1.0 TiB / 1.0 TiB avail; 232 KiB/s rd, 552 KiB/s wr, 86 op/s 2024-03-28T18:10:26.674 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:10:26 smithi134 ceph-mon[35037]: pgmap v89: 97 pgs: 97 active+clean; 116 MiB data, 814 MiB used, 1.0 TiB / 1.0 TiB avail; 232 KiB/s rd, 552 KiB/s wr, 86 op/s 2024-03-28T18:10:26.832 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:10:26 smithi079 systemd[1]: ceph-fd220748-ed2b-11ee-95d0-87774f69a715@mds.h.service: Scheduled restart job, restart counter is at 1. 2024-03-28T18:10:26.832 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:10:26 smithi079 systemd[1]: Stopped Ceph mds.h for fd220748-ed2b-11ee-95d0-87774f69a715. 2024-03-28T18:10:26.832 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:10:26 smithi079 systemd[1]: ceph-fd220748-ed2b-11ee-95d0-87774f69a715@mds.h.service: Consumed 3min 25.189s CPU time. 2024-03-28T18:10:26.832 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:10:26 smithi079 systemd[1]: Starting Ceph mds.h for fd220748-ed2b-11ee-95d0-87774f69a715... 2024-03-28T18:10:27.138 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:10:26 smithi079 podman[69030]: 2024-03-28 18:10:26.738466261 +0000 UTC m=+0.039190826 image pull c664afe56bb01db4f9afa9532c20598e201e664f2e9ba2be50db0d52674a26d2 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc 2024-03-28T18:10:27.138 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:10:26 smithi079 podman[69030]: 2024-03-28 18:10:26.861797926 +0000 UTC m=+0.162522506 container create b14c9698ded52199d9da0731ae67bf00eebbde5d68374ad20645c8e74a333d28 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, ceph=True, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240319, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-28T18:10:27.433 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:10:27 smithi079 podman[69030]: 2024-03-28 18:10:27.138159616 +0000 UTC m=+0.438884203 container init b14c9698ded52199d9da0731ae67bf00eebbde5d68374ad20645c8e74a333d28 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, ceph=True, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.build-date=20240319, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=) 2024-03-28T18:10:27.433 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:10:27 smithi079 podman[69030]: 2024-03-28 18:10:27.14904155 +0000 UTC m=+0.449766096 container start b14c9698ded52199d9da0731ae67bf00eebbde5d68374ad20645c8e74a333d28 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.build-date=20240319, GIT_BRANCH=HEAD, ceph=True, org.label-schema.schema-version=1.0, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c) 2024-03-28T18:10:27.433 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:10:27 smithi079 ceph-mds[69048]: -- 172.21.15.79:0/838221518 <== mon.2 v2:172.21.15.79:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x5623816f5a40 con 0x5623816c8800 2024-03-28T18:10:27.433 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:10:27 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69044]: starting mds.h at 2024-03-28T18:10:27.433 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:10:27 smithi079 bash[69030]: b14c9698ded52199d9da0731ae67bf00eebbde5d68374ad20645c8e74a333d28 2024-03-28T18:10:27.433 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:10:27 smithi079 systemd[1]: Started Ceph mds.h for fd220748-ed2b-11ee-95d0-87774f69a715. 2024-03-28T18:10:28.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:10:28 smithi022 ceph-mon[31985]: pgmap v90: 97 pgs: 97 active+clean; 116 MiB data, 814 MiB used, 1.0 TiB / 1.0 TiB avail; 91 KiB/s rd, 116 KiB/s wr, 21 op/s 2024-03-28T18:10:28.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:10:28 smithi022 ceph-mon[31985]: daemon mds.h restarted 2024-03-28T18:10:28.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:10:28 smithi022 ceph-mon[31985]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-03-28T18:10:28.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:10:28 smithi022 ceph-mon[31985]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-03-28T18:10:28.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:10:28 smithi022 ceph-mon[31985]: osdmap e94: 12 total, 12 up, 12 in 2024-03-28T18:10:28.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:10:28 smithi022 ceph-mon[31985]: mds.? [v2:172.21.15.79:6836/3437531480,v1:172.21.15.79:6837/3437531480] up:boot 2024-03-28T18:10:28.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:10:28 smithi022 ceph-mon[31985]: Standby daemon mds.f assigned to filesystem cephfs as rank 1 2024-03-28T18:10:28.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:10:28 smithi022 ceph-mon[31985]: fsmap cephfs:4/5 {0=b=up:active,2=e=up:active,3=k=up:active,4=i=up:active} 5 up:standby-replay 3 up:standby, 1 failed 2024-03-28T18:10:28.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:10:28 smithi022 ceph-mon[31985]: fsmap cephfs:5/5 4 up:standby-replay 3 up:standby 1 up:replay 4 up:active 2024-03-28T18:10:28.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:10:28 smithi134 ceph-mon[35037]: pgmap v90: 97 pgs: 97 active+clean; 116 MiB data, 814 MiB used, 1.0 TiB / 1.0 TiB avail; 91 KiB/s rd, 116 KiB/s wr, 21 op/s 2024-03-28T18:10:28.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:10:28 smithi134 ceph-mon[35037]: daemon mds.h restarted 2024-03-28T18:10:28.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:10:28 smithi134 ceph-mon[35037]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-03-28T18:10:28.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:10:28 smithi134 ceph-mon[35037]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-03-28T18:10:28.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:10:28 smithi134 ceph-mon[35037]: osdmap e94: 12 total, 12 up, 12 in 2024-03-28T18:10:28.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:10:28 smithi134 ceph-mon[35037]: mds.? [v2:172.21.15.79:6836/3437531480,v1:172.21.15.79:6837/3437531480] up:boot 2024-03-28T18:10:28.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:10:28 smithi134 ceph-mon[35037]: Standby daemon mds.f assigned to filesystem cephfs as rank 1 2024-03-28T18:10:28.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:10:28 smithi134 ceph-mon[35037]: fsmap cephfs:4/5 {0=b=up:active,2=e=up:active,3=k=up:active,4=i=up:active} 5 up:standby-replay 3 up:standby, 1 failed 2024-03-28T18:10:28.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:10:28 smithi134 ceph-mon[35037]: fsmap cephfs:5/5 4 up:standby-replay 3 up:standby 1 up:replay 4 up:active 2024-03-28T18:10:28.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:10:28 smithi079 ceph-mon[35312]: pgmap v90: 97 pgs: 97 active+clean; 116 MiB data, 814 MiB used, 1.0 TiB / 1.0 TiB avail; 91 KiB/s rd, 116 KiB/s wr, 21 op/s 2024-03-28T18:10:28.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:10:28 smithi079 ceph-mon[35312]: daemon mds.h restarted 2024-03-28T18:10:28.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:10:28 smithi079 ceph-mon[35312]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-03-28T18:10:28.683 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:10:28 smithi079 ceph-mon[35312]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-03-28T18:10:28.683 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:10:28 smithi079 ceph-mon[35312]: osdmap e94: 12 total, 12 up, 12 in 2024-03-28T18:10:28.683 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:10:28 smithi079 ceph-mon[35312]: mds.? [v2:172.21.15.79:6836/3437531480,v1:172.21.15.79:6837/3437531480] up:boot 2024-03-28T18:10:28.683 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:10:28 smithi079 ceph-mon[35312]: Standby daemon mds.f assigned to filesystem cephfs as rank 1 2024-03-28T18:10:28.683 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:10:28 smithi079 ceph-mon[35312]: fsmap cephfs:4/5 {0=b=up:active,2=e=up:active,3=k=up:active,4=i=up:active} 5 up:standby-replay 3 up:standby, 1 failed 2024-03-28T18:10:28.683 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:10:28 smithi079 ceph-mon[35312]: fsmap cephfs:5/5 4 up:standby-replay 3 up:standby 1 up:replay 4 up:active 2024-03-28T18:10:29.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:10:29 smithi022 ceph-mon[31985]: mds.? [v2:172.21.15.134:6834/3370549207,v1:172.21.15.134:6835/3370549207] up:resolve 2024-03-28T18:10:29.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:10:29 smithi022 ceph-mon[31985]: fsmap cephfs:5/5 4 up:standby-replay 3 up:standby 1 up:resolve 4 up:active 2024-03-28T18:10:29.674 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:10:29 smithi134 ceph-mon[35037]: mds.? [v2:172.21.15.134:6834/3370549207,v1:172.21.15.134:6835/3370549207] up:resolve 2024-03-28T18:10:29.674 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:10:29 smithi134 ceph-mon[35037]: fsmap cephfs:5/5 4 up:standby-replay 3 up:standby 1 up:resolve 4 up:active 2024-03-28T18:10:29.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:10:29 smithi079 ceph-mon[35312]: mds.? [v2:172.21.15.134:6834/3370549207,v1:172.21.15.134:6835/3370549207] up:resolve 2024-03-28T18:10:29.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:10:29 smithi079 ceph-mon[35312]: fsmap cephfs:5/5 4 up:standby-replay 3 up:standby 1 up:resolve 4 up:active 2024-03-28T18:10:30.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:10:30 smithi022 ceph-mon[31985]: pgmap v92: 97 pgs: 97 active+clean; 116 MiB data, 814 MiB used, 1.0 TiB / 1.0 TiB avail; 109 KiB/s rd, 140 KiB/s wr, 25 op/s 2024-03-28T18:10:30.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:10:30 smithi022 ceph-mon[31985]: mds.? [v2:172.21.15.134:6834/3370549207,v1:172.21.15.134:6835/3370549207] up:reconnect 2024-03-28T18:10:30.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:10:30 smithi022 ceph-mon[31985]: fsmap cephfs:5/5 4 up:standby-replay 3 up:standby 1 up:reconnect 4 up:active 2024-03-28T18:10:30.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:10:30 smithi022 ceph-mon[31985]: reconnect by client.35222 172.21.15.22:0/3729116781 after 0.0129997 2024-03-28T18:10:30.576 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:10:30 smithi134 ceph-mon[35037]: pgmap v92: 97 pgs: 97 active+clean; 116 MiB data, 814 MiB used, 1.0 TiB / 1.0 TiB avail; 109 KiB/s rd, 140 KiB/s wr, 25 op/s 2024-03-28T18:10:30.577 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:10:30 smithi134 ceph-mon[35037]: mds.? [v2:172.21.15.134:6834/3370549207,v1:172.21.15.134:6835/3370549207] up:reconnect 2024-03-28T18:10:30.577 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:10:30 smithi134 ceph-mon[35037]: fsmap cephfs:5/5 4 up:standby-replay 3 up:standby 1 up:reconnect 4 up:active 2024-03-28T18:10:30.577 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:10:30 smithi134 ceph-mon[35037]: reconnect by client.35222 172.21.15.22:0/3729116781 after 0.0129997 2024-03-28T18:10:30.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:10:30 smithi079 ceph-mon[35312]: pgmap v92: 97 pgs: 97 active+clean; 116 MiB data, 814 MiB used, 1.0 TiB / 1.0 TiB avail; 109 KiB/s rd, 140 KiB/s wr, 25 op/s 2024-03-28T18:10:30.683 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:10:30 smithi079 ceph-mon[35312]: mds.? [v2:172.21.15.134:6834/3370549207,v1:172.21.15.134:6835/3370549207] up:reconnect 2024-03-28T18:10:30.683 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:10:30 smithi079 ceph-mon[35312]: fsmap cephfs:5/5 4 up:standby-replay 3 up:standby 1 up:reconnect 4 up:active 2024-03-28T18:10:30.683 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:10:30 smithi079 ceph-mon[35312]: reconnect by client.35222 172.21.15.22:0/3729116781 after 0.0129997 2024-03-28T18:10:31.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:10:31 smithi022 ceph-mon[31985]: mds.? [v2:172.21.15.134:6834/3370549207,v1:172.21.15.134:6835/3370549207] up:rejoin 2024-03-28T18:10:31.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:10:31 smithi022 ceph-mon[31985]: fsmap cephfs:5/5 4 up:standby-replay 3 up:standby 1 up:rejoin 4 up:active 2024-03-28T18:10:31.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:10:31 smithi022 ceph-mon[31985]: daemon mds.f is now active in filesystem cephfs as rank 1 2024-03-28T18:10:31.583 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:10:31 smithi134 ceph-mon[35037]: mds.? [v2:172.21.15.134:6834/3370549207,v1:172.21.15.134:6835/3370549207] up:rejoin 2024-03-28T18:10:31.583 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:10:31 smithi134 ceph-mon[35037]: fsmap cephfs:5/5 4 up:standby-replay 3 up:standby 1 up:rejoin 4 up:active 2024-03-28T18:10:31.583 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:10:31 smithi134 ceph-mon[35037]: daemon mds.f is now active in filesystem cephfs as rank 1 2024-03-28T18:10:31.584 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:10:31 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[65133]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-2145-g3be9a5c9/rpm/el8/BUILD/ceph-19.0.0-2145-g3be9a5c9/src/mds/Locker.cc: In function 'void Locker::set_xlocks_done(MutationImpl*, bool)' thread 7ff70f200700 time 2024-03-28T18:10:31.460784+0000 2024-03-28T18:10:31.584 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:10:31 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[65133]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-2145-g3be9a5c9/rpm/el8/BUILD/ceph-19.0.0-2145-g3be9a5c9/src/mds/Locker.cc: 626: FAILED ceph_assert(obj->is_auth()) 2024-03-28T18:10:31.584 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:10:31 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[65133]: ceph version 19.0.0-2145-g3be9a5c9 (3be9a5c9bc793e11e2800a8c0c696e8b46742033) squid (dev) 2024-03-28T18:10:31.584 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:10:31 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[65133]: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x135) [0x7ff71aadd4a1] 2024-03-28T18:10:31.584 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:10:31 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[65133]: 2: /usr/lib64/ceph/libceph-common.so.2(+0x2d2667) [0x7ff71aadd667] 2024-03-28T18:10:31.584 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:10:31 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[65133]: 3: (Locker::set_xlocks_done(MutationImpl*, bool)+0x493) [0x5629a6ae3223] 2024-03-28T18:10:31.584 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:10:31 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[65133]: 4: (Server::early_reply(boost::intrusive_ptr const&, CInode*, CDentry*)+0x3e6) [0x5629a695d5d6] 2024-03-28T18:10:31.584 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:10:31 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[65133]: 5: (Server::journal_and_reply(boost::intrusive_ptr const&, CInode*, CDentry*, LogEvent*, MDSLogContextBase*)+0xc1) [0x5629a695dda1] 2024-03-28T18:10:31.585 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:10:31 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[65133]: 6: (Server::handle_client_mkdir(boost::intrusive_ptr const&)+0x5ba) [0x5629a69936aa] 2024-03-28T18:10:31.585 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:10:31 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[65133]: 7: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0x733) [0x5629a69b0233] 2024-03-28T18:10:31.585 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:10:31 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[65133]: 8: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x5629a69b2197] 2024-03-28T18:10:31.585 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:10:31 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[65133]: 9: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x5629a68b111b] 2024-03-28T18:10:31.585 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:10:31 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[65133]: 10: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x5629a68b16bc] 2024-03-28T18:10:31.585 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:10:31 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[65133]: 11: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bf) [0x5629a689b1ff] 2024-03-28T18:10:31.585 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:10:31 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[65133]: 12: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x478) [0x7ff71ad6f488] 2024-03-28T18:10:31.585 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:10:31 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[65133]: 13: (DispatchQueue::entry()+0x51f) [0x7ff71ad6c60f] 2024-03-28T18:10:31.585 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:10:31 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[65133]: 14: (DispatchQueue::DispatchThread::entry()+0x11) [0x7ff71ae3fe81] 2024-03-28T18:10:31.585 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:10:31 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[65133]: 15: /lib64/libpthread.so.0(+0x81ca) [0x7ff71985a1ca] 2024-03-28T18:10:31.585 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:10:31 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[65133]: 16: clone() 2024-03-28T18:10:31.586 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:10:31 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[65133]: 2024-03-28T18:10:31.460+0000 7ff70f200700 -1 /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-2145-g3be9a5c9/rpm/el8/BUILD/ceph-19.0.0-2145-g3be9a5c9/src/mds/Locker.cc: In function 'void Locker::set_xlocks_done(MutationImpl*, bool)' thread 7ff70f200700 time 2024-03-28T18:10:31.460784+0000 2024-03-28T18:10:31.586 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:10:31 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[65133]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-2145-g3be9a5c9/rpm/el8/BUILD/ceph-19.0.0-2145-g3be9a5c9/src/mds/Locker.cc: 626: FAILED ceph_assert(obj->is_auth()) 2024-03-28T18:10:31.586 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:10:31 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[65133]: 2024-03-28T18:10:31.587 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:10:31 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[65133]: ceph version 19.0.0-2145-g3be9a5c9 (3be9a5c9bc793e11e2800a8c0c696e8b46742033) squid (dev) 2024-03-28T18:10:31.587 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:10:31 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[65133]: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x135) [0x7ff71aadd4a1] 2024-03-28T18:10:31.587 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:10:31 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[65133]: 2: /usr/lib64/ceph/libceph-common.so.2(+0x2d2667) [0x7ff71aadd667] 2024-03-28T18:10:31.587 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:10:31 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[65133]: 3: (Locker::set_xlocks_done(MutationImpl*, bool)+0x493) [0x5629a6ae3223] 2024-03-28T18:10:31.587 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:10:31 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[65133]: 4: (Server::early_reply(boost::intrusive_ptr const&, CInode*, CDentry*)+0x3e6) [0x5629a695d5d6] 2024-03-28T18:10:31.587 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:10:31 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[65133]: 5: (Server::journal_and_reply(boost::intrusive_ptr const&, CInode*, CDentry*, LogEvent*, MDSLogContextBase*)+0xc1) [0x5629a695dda1] 2024-03-28T18:10:31.587 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:10:31 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[65133]: 6: (Server::handle_client_mkdir(boost::intrusive_ptr const&)+0x5ba) [0x5629a69936aa] 2024-03-28T18:10:31.587 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:10:31 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[65133]: 7: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0x733) [0x5629a69b0233] 2024-03-28T18:10:31.587 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:10:31 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[65133]: 8: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x5629a69b2197] 2024-03-28T18:10:31.587 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:10:31 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[65133]: 9: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x5629a68b111b] 2024-03-28T18:10:31.587 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:10:31 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[65133]: 10: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x5629a68b16bc] 2024-03-28T18:10:31.587 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:10:31 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[65133]: 11: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bf) [0x5629a689b1ff] 2024-03-28T18:10:31.587 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:10:31 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[65133]: 12: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x478) [0x7ff71ad6f488] 2024-03-28T18:10:31.588 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:10:31 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[65133]: 13: (DispatchQueue::entry()+0x51f) [0x7ff71ad6c60f] 2024-03-28T18:10:31.588 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:10:31 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[65133]: 14: (DispatchQueue::DispatchThread::entry()+0x11) [0x7ff71ae3fe81] 2024-03-28T18:10:31.588 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:10:31 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[65133]: 15: /lib64/libpthread.so.0(+0x81ca) [0x7ff71985a1ca] 2024-03-28T18:10:31.588 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:10:31 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[65133]: 16: clone() 2024-03-28T18:10:31.588 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:10:31 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[65133]: 2024-03-28T18:10:31.588 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:10:31 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[65133]: *** Caught signal (Aborted) ** 2024-03-28T18:10:31.588 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:10:31 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[65133]: in thread 7ff70f200700 thread_name:ms_dispatch 2024-03-28T18:10:31.588 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:10:31 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[65133]: ceph version 19.0.0-2145-g3be9a5c9 (3be9a5c9bc793e11e2800a8c0c696e8b46742033) squid (dev) 2024-03-28T18:10:31.588 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:10:31 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[65133]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7ff719864d20] 2024-03-28T18:10:31.588 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:10:31 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[65133]: 2: gsignal() 2024-03-28T18:10:31.588 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:10:31 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[65133]: 3: abort() 2024-03-28T18:10:31.588 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:10:31 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[65133]: 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x18f) [0x7ff71aadd4fb] 2024-03-28T18:10:31.588 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:10:31 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[65133]: 5: /usr/lib64/ceph/libceph-common.so.2(+0x2d2667) [0x7ff71aadd667] 2024-03-28T18:10:31.588 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:10:31 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[65133]: 6: (Locker::set_xlocks_done(MutationImpl*, bool)+0x493) [0x5629a6ae3223] 2024-03-28T18:10:31.589 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:10:31 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[65133]: 7: (Server::early_reply(boost::intrusive_ptr const&, CInode*, CDentry*)+0x3e6) [0x5629a695d5d6] 2024-03-28T18:10:31.589 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:10:31 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[65133]: 8: (Server::journal_and_reply(boost::intrusive_ptr const&, CInode*, CDentry*, LogEvent*, MDSLogContextBase*)+0xc1) [0x5629a695dda1] 2024-03-28T18:10:31.589 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:10:31 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[65133]: 9: (Server::handle_client_mkdir(boost::intrusive_ptr const&)+0x5ba) [0x5629a69936aa] 2024-03-28T18:10:31.589 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:10:31 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[65133]: 10: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0x733) [0x5629a69b0233] 2024-03-28T18:10:31.589 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:10:31 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[65133]: 11: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x5629a69b2197] 2024-03-28T18:10:31.589 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:10:31 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[65133]: 12: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x5629a68b111b] 2024-03-28T18:10:31.589 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:10:31 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[65133]: 13: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x5629a68b16bc] 2024-03-28T18:10:31.589 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:10:31 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[65133]: 14: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bf) [0x5629a689b1ff] 2024-03-28T18:10:31.589 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:10:31 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[65133]: 15: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x478) [0x7ff71ad6f488] 2024-03-28T18:10:31.589 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:10:31 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[65133]: 16: (DispatchQueue::entry()+0x51f) [0x7ff71ad6c60f] 2024-03-28T18:10:31.589 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:10:31 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[65133]: 17: (DispatchQueue::DispatchThread::entry()+0x11) [0x7ff71ae3fe81] 2024-03-28T18:10:31.589 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:10:31 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[65133]: 18: /lib64/libpthread.so.0(+0x81ca) [0x7ff71985a1ca] 2024-03-28T18:10:31.589 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:10:31 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[65133]: 19: clone() 2024-03-28T18:10:31.589 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:10:31 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[65133]: 2024-03-28T18:10:31.463+0000 7ff70f200700 -1 *** Caught signal (Aborted) ** 2024-03-28T18:10:31.589 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:10:31 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[65133]: in thread 7ff70f200700 thread_name:ms_dispatch 2024-03-28T18:10:31.589 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:10:31 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[65133]: 2024-03-28T18:10:31.590 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:10:31 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[65133]: ceph version 19.0.0-2145-g3be9a5c9 (3be9a5c9bc793e11e2800a8c0c696e8b46742033) squid (dev) 2024-03-28T18:10:31.590 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:10:31 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[65133]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7ff719864d20] 2024-03-28T18:10:31.590 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:10:31 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[65133]: 2: gsignal() 2024-03-28T18:10:31.590 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:10:31 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[65133]: 3: abort() 2024-03-28T18:10:31.590 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:10:31 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[65133]: 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x18f) [0x7ff71aadd4fb] 2024-03-28T18:10:31.590 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:10:31 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[65133]: 5: /usr/lib64/ceph/libceph-common.so.2(+0x2d2667) [0x7ff71aadd667] 2024-03-28T18:10:31.590 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:10:31 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[65133]: 6: (Locker::set_xlocks_done(MutationImpl*, bool)+0x493) [0x5629a6ae3223] 2024-03-28T18:10:31.590 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:10:31 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[65133]: 7: (Server::early_reply(boost::intrusive_ptr const&, CInode*, CDentry*)+0x3e6) [0x5629a695d5d6] 2024-03-28T18:10:31.590 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:10:31 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[65133]: 8: (Server::journal_and_reply(boost::intrusive_ptr const&, CInode*, CDentry*, LogEvent*, MDSLogContextBase*)+0xc1) [0x5629a695dda1] 2024-03-28T18:10:31.590 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:10:31 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[65133]: 9: (Server::handle_client_mkdir(boost::intrusive_ptr const&)+0x5ba) [0x5629a69936aa] 2024-03-28T18:10:31.590 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:10:31 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[65133]: 10: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0x733) [0x5629a69b0233] 2024-03-28T18:10:31.590 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:10:31 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[65133]: 11: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x5629a69b2197] 2024-03-28T18:10:31.590 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:10:31 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[65133]: 12: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x5629a68b111b] 2024-03-28T18:10:31.590 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:10:31 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[65133]: 13: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x5629a68b16bc] 2024-03-28T18:10:31.591 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:10:31 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[65133]: 14: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bf) [0x5629a689b1ff] 2024-03-28T18:10:31.591 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:10:31 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[65133]: 15: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x478) [0x7ff71ad6f488] 2024-03-28T18:10:31.591 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:10:31 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[65133]: 16: (DispatchQueue::entry()+0x51f) [0x7ff71ad6c60f] 2024-03-28T18:10:31.591 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:10:31 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[65133]: 17: (DispatchQueue::DispatchThread::entry()+0x11) [0x7ff71ae3fe81] 2024-03-28T18:10:31.591 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:10:31 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[65133]: 18: /lib64/libpthread.so.0(+0x81ca) [0x7ff71985a1ca] 2024-03-28T18:10:31.591 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:10:31 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[65133]: 19: clone() 2024-03-28T18:10:31.591 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:10:31 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[65133]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-03-28T18:10:31.591 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:10:31 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[65133]: 2024-03-28T18:10:31.591 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:10:31 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[65133]: -1> 2024-03-28T18:10:31.460+0000 7ff70f200700 -1 /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-2145-g3be9a5c9/rpm/el8/BUILD/ceph-19.0.0-2145-g3be9a5c9/src/mds/Locker.cc: In function 'void Locker::set_xlocks_done(MutationImpl*, bool)' thread 7ff70f200700 time 2024-03-28T18:10:31.460784+0000 2024-03-28T18:10:31.591 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:10:31 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[65133]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-2145-g3be9a5c9/rpm/el8/BUILD/ceph-19.0.0-2145-g3be9a5c9/src/mds/Locker.cc: 626: FAILED ceph_assert(obj->is_auth()) 2024-03-28T18:10:31.591 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:10:31 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[65133]: 2024-03-28T18:10:31.591 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:10:31 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[65133]: ceph version 19.0.0-2145-g3be9a5c9 (3be9a5c9bc793e11e2800a8c0c696e8b46742033) squid (dev) 2024-03-28T18:10:31.591 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:10:31 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[65133]: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x135) [0x7ff71aadd4a1] 2024-03-28T18:10:31.591 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:10:31 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[65133]: 2: /usr/lib64/ceph/libceph-common.so.2(+0x2d2667) [0x7ff71aadd667] 2024-03-28T18:10:31.591 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:10:31 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[65133]: 3: (Locker::set_xlocks_done(MutationImpl*, bool)+0x493) [0x5629a6ae3223] 2024-03-28T18:10:31.592 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:10:31 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[65133]: 4: (Server::early_reply(boost::intrusive_ptr const&, CInode*, CDentry*)+0x3e6) [0x5629a695d5d6] 2024-03-28T18:10:31.592 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:10:31 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[65133]: 5: (Server::journal_and_reply(boost::intrusive_ptr const&, CInode*, CDentry*, LogEvent*, MDSLogContextBase*)+0xc1) [0x5629a695dda1] 2024-03-28T18:10:31.592 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:10:31 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[65133]: 6: (Server::handle_client_mkdir(boost::intrusive_ptr const&)+0x5ba) [0x5629a69936aa] 2024-03-28T18:10:31.592 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:10:31 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[65133]: 7: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0x733) [0x5629a69b0233] 2024-03-28T18:10:31.592 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:10:31 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[65133]: 8: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x5629a69b2197] 2024-03-28T18:10:31.592 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:10:31 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[65133]: 9: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x5629a68b111b] 2024-03-28T18:10:31.592 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:10:31 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[65133]: 10: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x5629a68b16bc] 2024-03-28T18:10:31.592 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:10:31 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[65133]: 11: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bf) [0x5629a689b1ff] 2024-03-28T18:10:31.592 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:10:31 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[65133]: 12: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x478) [0x7ff71ad6f488] 2024-03-28T18:10:31.592 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:10:31 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[65133]: 13: (DispatchQueue::entry()+0x51f) [0x7ff71ad6c60f] 2024-03-28T18:10:31.592 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:10:31 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[65133]: 14: (DispatchQueue::DispatchThread::entry()+0x11) [0x7ff71ae3fe81] 2024-03-28T18:10:31.592 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:10:31 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[65133]: 15: /lib64/libpthread.so.0(+0x81ca) [0x7ff71985a1ca] 2024-03-28T18:10:31.592 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:10:31 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[65133]: 16: clone() 2024-03-28T18:10:31.592 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:10:31 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[65133]: 2024-03-28T18:10:31.593 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:10:31 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[65133]: 0> 2024-03-28T18:10:31.463+0000 7ff70f200700 -1 *** Caught signal (Aborted) ** 2024-03-28T18:10:31.593 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:10:31 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[65133]: in thread 7ff70f200700 thread_name:ms_dispatch 2024-03-28T18:10:31.593 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:10:31 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[65133]: 2024-03-28T18:10:31.593 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:10:31 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[65133]: ceph version 19.0.0-2145-g3be9a5c9 (3be9a5c9bc793e11e2800a8c0c696e8b46742033) squid (dev) 2024-03-28T18:10:31.593 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:10:31 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[65133]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7ff719864d20] 2024-03-28T18:10:31.593 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:10:31 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[65133]: 2: gsignal() 2024-03-28T18:10:31.593 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:10:31 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[65133]: 3: abort() 2024-03-28T18:10:31.593 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:10:31 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[65133]: 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x18f) [0x7ff71aadd4fb] 2024-03-28T18:10:31.593 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:10:31 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[65133]: 5: /usr/lib64/ceph/libceph-common.so.2(+0x2d2667) [0x7ff71aadd667] 2024-03-28T18:10:31.593 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:10:31 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[65133]: 6: (Locker::set_xlocks_done(MutationImpl*, bool)+0x493) [0x5629a6ae3223] 2024-03-28T18:10:31.593 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:10:31 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[65133]: 7: (Server::early_reply(boost::intrusive_ptr const&, CInode*, CDentry*)+0x3e6) [0x5629a695d5d6] 2024-03-28T18:10:31.593 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:10:31 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[65133]: 8: (Server::journal_and_reply(boost::intrusive_ptr const&, CInode*, CDentry*, LogEvent*, MDSLogContextBase*)+0xc1) [0x5629a695dda1] 2024-03-28T18:10:31.593 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:10:31 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[65133]: 9: (Server::handle_client_mkdir(boost::intrusive_ptr const&)+0x5ba) [0x5629a69936aa] 2024-03-28T18:10:31.593 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:10:31 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[65133]: 10: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0x733) [0x5629a69b0233] 2024-03-28T18:10:31.593 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:10:31 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[65133]: 11: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x5629a69b2197] 2024-03-28T18:10:31.594 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:10:31 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[65133]: 12: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x5629a68b111b] 2024-03-28T18:10:31.594 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:10:31 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[65133]: 13: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x5629a68b16bc] 2024-03-28T18:10:31.594 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:10:31 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[65133]: 14: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bf) [0x5629a689b1ff] 2024-03-28T18:10:31.594 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:10:31 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[65133]: 15: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x478) [0x7ff71ad6f488] 2024-03-28T18:10:31.594 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:10:31 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[65133]: 16: (DispatchQueue::entry()+0x51f) [0x7ff71ad6c60f] 2024-03-28T18:10:31.594 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:10:31 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[65133]: 17: (DispatchQueue::DispatchThread::entry()+0x11) [0x7ff71ae3fe81] 2024-03-28T18:10:31.594 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:10:31 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[65133]: 18: /lib64/libpthread.so.0(+0x81ca) [0x7ff71985a1ca] 2024-03-28T18:10:31.594 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:10:31 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[65133]: 19: clone() 2024-03-28T18:10:31.594 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:10:31 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[65133]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-03-28T18:10:31.594 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:10:31 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[65133]: 2024-03-28T18:10:31.594 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:10:31 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[65133]: -4> 2024-03-28T18:10:31.460+0000 7ff70f200700 -1 /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-2145-g3be9a5c9/rpm/el8/BUILD/ceph-19.0.0-2145-g3be9a5c9/src/mds/Locker.cc: In function 'void Locker::set_xlocks_done(MutationImpl*, bool)' thread 7ff70f200700 time 2024-03-28T18:10:31.460784+0000 2024-03-28T18:10:31.594 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:10:31 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[65133]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-2145-g3be9a5c9/rpm/el8/BUILD/ceph-19.0.0-2145-g3be9a5c9/src/mds/Locker.cc: 626: FAILED ceph_assert(obj->is_auth()) 2024-03-28T18:10:31.594 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:10:31 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[65133]: 2024-03-28T18:10:31.594 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:10:31 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[65133]: ceph version 19.0.0-2145-g3be9a5c9 (3be9a5c9bc793e11e2800a8c0c696e8b46742033) squid (dev) 2024-03-28T18:10:31.594 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:10:31 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[65133]: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x135) [0x7ff71aadd4a1] 2024-03-28T18:10:31.595 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:10:31 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[65133]: 2: /usr/lib64/ceph/libceph-common.so.2(+0x2d2667) [0x7ff71aadd667] 2024-03-28T18:10:31.595 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:10:31 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[65133]: 3: (Locker::set_xlocks_done(MutationImpl*, bool)+0x493) [0x5629a6ae3223] 2024-03-28T18:10:31.595 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:10:31 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[65133]: 4: (Server::early_reply(boost::intrusive_ptr const&, CInode*, CDentry*)+0x3e6) [0x5629a695d5d6] 2024-03-28T18:10:31.595 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:10:31 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[65133]: 5: (Server::journal_and_reply(boost::intrusive_ptr const&, CInode*, CDentry*, LogEvent*, MDSLogContextBase*)+0xc1) [0x5629a695dda1] 2024-03-28T18:10:31.595 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:10:31 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[65133]: 6: (Server::handle_client_mkdir(boost::intrusive_ptr const&)+0x5ba) [0x5629a69936aa] 2024-03-28T18:10:31.595 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:10:31 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[65133]: 7: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0x733) [0x5629a69b0233] 2024-03-28T18:10:31.596 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:10:31 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[65133]: 8: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x5629a69b2197] 2024-03-28T18:10:31.596 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:10:31 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[65133]: 9: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x5629a68b111b] 2024-03-28T18:10:31.596 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:10:31 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[65133]: 10: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x5629a68b16bc] 2024-03-28T18:10:31.596 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:10:31 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[65133]: 11: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bf) [0x5629a689b1ff] 2024-03-28T18:10:31.596 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:10:31 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[65133]: 12: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x478) [0x7ff71ad6f488] 2024-03-28T18:10:31.596 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:10:31 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[65133]: 13: (DispatchQueue::entry()+0x51f) [0x7ff71ad6c60f] 2024-03-28T18:10:31.596 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:10:31 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[65133]: 14: (DispatchQueue::DispatchThread::entry()+0x11) [0x7ff71ae3fe81] 2024-03-28T18:10:31.596 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:10:31 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[65133]: 15: /lib64/libpthread.so.0(+0x81ca) [0x7ff71985a1ca] 2024-03-28T18:10:31.596 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:10:31 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[65133]: 16: clone() 2024-03-28T18:10:31.596 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:10:31 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[65133]: 2024-03-28T18:10:31.596 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:10:31 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[65133]: -3> 2024-03-28T18:10:31.463+0000 7ff70f200700 -1 *** Caught signal (Aborted) ** 2024-03-28T18:10:31.596 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:10:31 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[65133]: in thread 7ff70f200700 thread_name:ms_dispatch 2024-03-28T18:10:31.596 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:10:31 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[65133]: 2024-03-28T18:10:31.596 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:10:31 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[65133]: ceph version 19.0.0-2145-g3be9a5c9 (3be9a5c9bc793e11e2800a8c0c696e8b46742033) squid (dev) 2024-03-28T18:10:31.596 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:10:31 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[65133]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7ff719864d20] 2024-03-28T18:10:31.597 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:10:31 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[65133]: 2: gsignal() 2024-03-28T18:10:31.597 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:10:31 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[65133]: 3: abort() 2024-03-28T18:10:31.597 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:10:31 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[65133]: 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x18f) [0x7ff71aadd4fb] 2024-03-28T18:10:31.597 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:10:31 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[65133]: 5: /usr/lib64/ceph/libceph-common.so.2(+0x2d2667) [0x7ff71aadd667] 2024-03-28T18:10:31.597 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:10:31 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[65133]: 6: (Locker::set_xlocks_done(MutationImpl*, bool)+0x493) [0x5629a6ae3223] 2024-03-28T18:10:31.597 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:10:31 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[65133]: 7: (Server::early_reply(boost::intrusive_ptr const&, CInode*, CDentry*)+0x3e6) [0x5629a695d5d6] 2024-03-28T18:10:31.597 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:10:31 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[65133]: 8: (Server::journal_and_reply(boost::intrusive_ptr const&, CInode*, CDentry*, LogEvent*, MDSLogContextBase*)+0xc1) [0x5629a695dda1] 2024-03-28T18:10:31.597 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:10:31 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[65133]: 9: (Server::handle_client_mkdir(boost::intrusive_ptr const&)+0x5ba) [0x5629a69936aa] 2024-03-28T18:10:31.597 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:10:31 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[65133]: 10: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0x733) [0x5629a69b0233] 2024-03-28T18:10:31.597 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:10:31 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[65133]: 11: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x5629a69b2197] 2024-03-28T18:10:31.597 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:10:31 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[65133]: 12: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x5629a68b111b] 2024-03-28T18:10:31.597 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:10:31 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[65133]: 13: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x5629a68b16bc] 2024-03-28T18:10:31.597 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:10:31 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[65133]: 14: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bf) [0x5629a689b1ff] 2024-03-28T18:10:31.597 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:10:31 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[65133]: 15: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x478) [0x7ff71ad6f488] 2024-03-28T18:10:31.598 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:10:31 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[65133]: 16: (DispatchQueue::entry()+0x51f) [0x7ff71ad6c60f] 2024-03-28T18:10:31.598 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:10:31 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[65133]: 17: (DispatchQueue::DispatchThread::entry()+0x11) [0x7ff71ae3fe81] 2024-03-28T18:10:31.598 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:10:31 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[65133]: 18: /lib64/libpthread.so.0(+0x81ca) [0x7ff71985a1ca] 2024-03-28T18:10:31.598 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:10:31 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[65133]: 19: clone() 2024-03-28T18:10:31.598 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:10:31 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[65133]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-03-28T18:10:31.598 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:10:31 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[65133]: 2024-03-28T18:10:31.598 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:10:31 smithi134 podman[69652]: 2024-03-28 18:10:31.58418155 +0000 UTC m=+0.035438664 container died 7fe8640da6db60521ec0366a2d3b24b04b7295e51c6ba6786f37d5cb444425e0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240319, GIT_CLEAN=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30) 2024-03-28T18:10:31.683 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:10:31 smithi079 ceph-mon[35312]: mds.? [v2:172.21.15.134:6834/3370549207,v1:172.21.15.134:6835/3370549207] up:rejoin 2024-03-28T18:10:31.683 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:10:31 smithi079 ceph-mon[35312]: fsmap cephfs:5/5 4 up:standby-replay 3 up:standby 1 up:rejoin 4 up:active 2024-03-28T18:10:31.683 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:10:31 smithi079 ceph-mon[35312]: daemon mds.f is now active in filesystem cephfs as rank 1 2024-03-28T18:10:32.425 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:10:32 smithi134 ceph-mon[35037]: pgmap v93: 97 pgs: 97 active+clean; 116 MiB data, 814 MiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 41 KiB/s wr, 12 op/s 2024-03-28T18:10:32.425 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:10:32 smithi134 ceph-mon[35037]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-03-28T18:10:32.425 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:10:32 smithi134 ceph-mon[35037]: Cluster is now healthy 2024-03-28T18:10:32.425 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:10:32 smithi134 ceph-mon[35037]: mds.? [v2:172.21.15.134:6834/3370549207,v1:172.21.15.134:6835/3370549207] up:active 2024-03-28T18:10:32.425 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:10:32 smithi134 ceph-mon[35037]: fsmap cephfs:5 4 up:standby-replay 3 up:standby 5 up:active 2024-03-28T18:10:32.425 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:10:32 smithi134 ceph-mon[35037]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-28T18:10:32.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:10:32 smithi022 ceph-mon[31985]: pgmap v93: 97 pgs: 97 active+clean; 116 MiB data, 814 MiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 41 KiB/s wr, 12 op/s 2024-03-28T18:10:32.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:10:32 smithi022 ceph-mon[31985]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-03-28T18:10:32.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:10:32 smithi022 ceph-mon[31985]: Cluster is now healthy 2024-03-28T18:10:32.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:10:32 smithi022 ceph-mon[31985]: mds.? [v2:172.21.15.134:6834/3370549207,v1:172.21.15.134:6835/3370549207] up:active 2024-03-28T18:10:32.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:10:32 smithi022 ceph-mon[31985]: fsmap cephfs:5 4 up:standby-replay 3 up:standby 5 up:active 2024-03-28T18:10:32.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:10:32 smithi022 ceph-mon[31985]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-28T18:10:32.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:10:32 smithi079 ceph-mon[35312]: pgmap v93: 97 pgs: 97 active+clean; 116 MiB data, 814 MiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 41 KiB/s wr, 12 op/s 2024-03-28T18:10:32.683 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:10:32 smithi079 ceph-mon[35312]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-03-28T18:10:32.683 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:10:32 smithi079 ceph-mon[35312]: Cluster is now healthy 2024-03-28T18:10:32.683 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:10:32 smithi079 ceph-mon[35312]: mds.? [v2:172.21.15.134:6834/3370549207,v1:172.21.15.134:6835/3370549207] up:active 2024-03-28T18:10:32.683 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:10:32 smithi079 ceph-mon[35312]: fsmap cephfs:5 4 up:standby-replay 3 up:standby 5 up:active 2024-03-28T18:10:32.683 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:10:32 smithi079 ceph-mon[35312]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-28T18:10:32.924 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:10:32 smithi134 podman[69652]: 2024-03-28 18:10:32.537358559 +0000 UTC m=+0.988615709 container remove 7fe8640da6db60521ec0366a2d3b24b04b7295e51c6ba6786f37d5cb444425e0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240319, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, ceph=True) 2024-03-28T18:10:32.925 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:10:32 smithi134 systemd[1]: ceph-fd220748-ed2b-11ee-95d0-87774f69a715@mds.f.service: Main process exited, code=exited, status=134/n/a 2024-03-28T18:10:33.349 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:10:33 smithi134 systemd[1]: ceph-fd220748-ed2b-11ee-95d0-87774f69a715@mds.f.service: Failed with result 'exit-code'. 2024-03-28T18:10:33.349 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:10:33 smithi134 systemd[1]: ceph-fd220748-ed2b-11ee-95d0-87774f69a715@mds.f.service: Consumed 3.440s CPU time. 2024-03-28T18:10:34.432 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:10:34 smithi079 ceph-mon[35312]: pgmap v94: 97 pgs: 97 active+clean; 116 MiB data, 814 MiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 41 KiB/s wr, 12 op/s 2024-03-28T18:10:34.433 INFO:journalctl@ceph.mds.b.smithi079.stdout:Mar 28 18:10:34 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-b[65245]: 2024-03-28T18:10:34.077+0000 7f845f400700 -1 mds.pinger is_rank_lagging: rank=1 was never sent ping request. 2024-03-28T18:10:34.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:10:34 smithi022 ceph-mon[31985]: pgmap v94: 97 pgs: 97 active+clean; 116 MiB data, 814 MiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 41 KiB/s wr, 12 op/s 2024-03-28T18:10:34.607 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:10:34 smithi134 ceph-mon[35037]: pgmap v94: 97 pgs: 97 active+clean; 116 MiB data, 814 MiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 41 KiB/s wr, 12 op/s 2024-03-28T18:10:36.534 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:10:36 smithi022 ceph-mon[31985]: pgmap v95: 97 pgs: 97 active+clean; 117 MiB data, 799 MiB used, 1.0 TiB / 1.0 TiB avail; 112 KiB/s rd, 30 KiB/s wr, 28 op/s 2024-03-28T18:10:36.674 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:10:36 smithi134 ceph-mon[35037]: pgmap v95: 97 pgs: 97 active+clean; 117 MiB data, 799 MiB used, 1.0 TiB / 1.0 TiB avail; 112 KiB/s rd, 30 KiB/s wr, 28 op/s 2024-03-28T18:10:36.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:10:36 smithi079 ceph-mon[35312]: pgmap v95: 97 pgs: 97 active+clean; 117 MiB data, 799 MiB used, 1.0 TiB / 1.0 TiB avail; 112 KiB/s rd, 30 KiB/s wr, 28 op/s 2024-03-28T18:10:38.674 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:10:38 smithi134 ceph-mon[35037]: pgmap v96: 97 pgs: 97 active+clean; 117 MiB data, 799 MiB used, 1.0 TiB / 1.0 TiB avail; 112 KiB/s rd, 30 KiB/s wr, 28 op/s 2024-03-28T18:10:38.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:10:38 smithi079 ceph-mon[35312]: pgmap v96: 97 pgs: 97 active+clean; 117 MiB data, 799 MiB used, 1.0 TiB / 1.0 TiB avail; 112 KiB/s rd, 30 KiB/s wr, 28 op/s 2024-03-28T18:10:38.783 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:10:38 smithi022 ceph-mon[31985]: pgmap v96: 97 pgs: 97 active+clean; 117 MiB data, 799 MiB used, 1.0 TiB / 1.0 TiB avail; 112 KiB/s rd, 30 KiB/s wr, 28 op/s 2024-03-28T18:10:40.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:10:40 smithi134 ceph-mon[35037]: pgmap v97: 97 pgs: 97 active+clean; 117 MiB data, 799 MiB used, 1.0 TiB / 1.0 TiB avail; 94 KiB/s rd, 26 KiB/s wr, 23 op/s 2024-03-28T18:10:40.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:10:40 smithi079 ceph-mon[35312]: pgmap v97: 97 pgs: 97 active+clean; 117 MiB data, 799 MiB used, 1.0 TiB / 1.0 TiB avail; 94 KiB/s rd, 26 KiB/s wr, 23 op/s 2024-03-28T18:10:40.781 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:10:40 smithi022 ceph-mon[31985]: pgmap v97: 97 pgs: 97 active+clean; 117 MiB data, 799 MiB used, 1.0 TiB / 1.0 TiB avail; 94 KiB/s rd, 26 KiB/s wr, 23 op/s 2024-03-28T18:10:42.674 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:10:42 smithi134 ceph-mon[35037]: pgmap v98: 97 pgs: 97 active+clean; 117 MiB data, 799 MiB used, 1.0 TiB / 1.0 TiB avail; 95 KiB/s rd, 25 KiB/s wr, 27 op/s 2024-03-28T18:10:42.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:10:42 smithi079 ceph-mon[35312]: pgmap v98: 97 pgs: 97 active+clean; 117 MiB data, 799 MiB used, 1.0 TiB / 1.0 TiB avail; 95 KiB/s rd, 25 KiB/s wr, 27 op/s 2024-03-28T18:10:42.781 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:10:42 smithi022 ceph-mon[31985]: pgmap v98: 97 pgs: 97 active+clean; 117 MiB data, 799 MiB used, 1.0 TiB / 1.0 TiB avail; 95 KiB/s rd, 25 KiB/s wr, 27 op/s 2024-03-28T18:10:42.996 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-28T18:10:42.997 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-28T18:10:43.235 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:43.235+0000 7f63f5200640 1 -- 172.21.15.22:0/520946209 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f63f010cea0 msgr2=0x7f63f0114a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:10:43.236 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:43.235+0000 7f63f5200640 1 --2- 172.21.15.22:0/520946209 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f63f010cea0 0x7f63f0114a70 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7f63e8009ca0 tx=0x7f63e802c100 comp rx=0 tx=0).stop 2024-03-28T18:10:43.236 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:43.235+0000 7f63f5200640 1 -- 172.21.15.22:0/520946209 shutdown_connections 2024-03-28T18:10:43.236 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:43.235+0000 7f63f5200640 1 --2- 172.21.15.22:0/520946209 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f63f010cea0 0x7f63f0114a70 unknown :-1 s=CLOSED pgs=658 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:10:43.236 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:43.235+0000 7f63f5200640 1 --2- 172.21.15.22:0/520946209 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f63f0108800 0x7f63f010c7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:10:43.236 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:43.235+0000 7f63f5200640 1 --2- 172.21.15.22:0/520946209 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f63f0107e50 0x7f63f0108230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:10:43.236 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:43.235+0000 7f63f5200640 1 -- 172.21.15.22:0/520946209 >> 172.21.15.22:0/520946209 conn(0x7f63f00774c0 msgr2=0x7f63f01064c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:10:43.236 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:43.235+0000 7f63f5200640 1 -- 172.21.15.22:0/520946209 shutdown_connections 2024-03-28T18:10:43.236 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:43.236+0000 7f63f5200640 1 -- 172.21.15.22:0/520946209 wait complete. 2024-03-28T18:10:43.238 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:43.236+0000 7f63f5200640 1 Processor -- start 2024-03-28T18:10:43.239 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:43.237+0000 7f63f5200640 1 -- start start 2024-03-28T18:10:43.239 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:43.237+0000 7f63f5200640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f63f0107e50 0x7f63f019e3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:10:43.239 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:43.237+0000 7f63f5200640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f63f010cea0 0x7f63f019e900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:10:43.239 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:43.237+0000 7f63f5200640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f63f01a2c80 0x7f63f01a3130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:10:43.239 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:43.237+0000 7f63f5200640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63f0117f60 con 0x7f63f0107e50 2024-03-28T18:10:43.239 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:43.237+0000 7f63f5200640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63f0117de0 con 0x7f63f010cea0 2024-03-28T18:10:43.239 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:43.237+0000 7f63f5200640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63f01180e0 con 0x7f63f01a2c80 2024-03-28T18:10:43.239 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:43.237+0000 7f63ef400640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f63f0107e50 0x7f63f019e3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:10:43.239 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:43.237+0000 7f63ef400640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f63f0107e50 0x7f63f019e3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:34486/0 (socket says 172.21.15.22:34486) 2024-03-28T18:10:43.239 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:43.237+0000 7f63ef400640 1 -- 172.21.15.22:0/3262567993 learned_addr learned my addr 172.21.15.22:0/3262567993 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-28T18:10:43.239 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:43.238+0000 7f63efe00640 1 --2- 172.21.15.22:0/3262567993 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f63f01a2c80 0x7f63f01a3130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:10:43.239 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:43.238+0000 7f63e7e00640 1 --2- 172.21.15.22:0/3262567993 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f63f010cea0 0x7f63f019e900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:10:43.239 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:43.238+0000 7f63ef400640 1 -- 172.21.15.22:0/3262567993 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f63f01a2c80 msgr2=0x7f63f01a3130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:10:43.239 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:43.238+0000 7f63ef400640 1 --2- 172.21.15.22:0/3262567993 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f63f01a2c80 0x7f63f01a3130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:10:43.240 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:43.238+0000 7f63ef400640 1 -- 172.21.15.22:0/3262567993 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f63f010cea0 msgr2=0x7f63f019e900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:10:43.240 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:43.238+0000 7f63ef400640 1 --2- 172.21.15.22:0/3262567993 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f63f010cea0 0x7f63f019e900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:10:43.240 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:43.238+0000 7f63ef400640 1 -- 172.21.15.22:0/3262567993 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f63f01a3770 con 0x7f63f0107e50 2024-03-28T18:10:43.240 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:43.238+0000 7f63ef400640 1 --2- 172.21.15.22:0/3262567993 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f63f0107e50 0x7f63f019e3c0 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7f63e000d950 tx=0x7f63e000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:10:43.240 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:43.238+0000 7f63ecc00640 1 -- 172.21.15.22:0/3262567993 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63e0017070 con 0x7f63f0107e50 2024-03-28T18:10:43.240 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:43.238+0000 7f63ecc00640 1 -- 172.21.15.22:0/3262567993 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f63e0009c30 con 0x7f63f0107e50 2024-03-28T18:10:43.240 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:43.238+0000 7f63ecc00640 1 -- 172.21.15.22:0/3262567993 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63e000f6b0 con 0x7f63f0107e50 2024-03-28T18:10:43.240 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:43.238+0000 7f63f5200640 1 -- 172.21.15.22:0/3262567993 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f63f01a3a60 con 0x7f63f0107e50 2024-03-28T18:10:43.240 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:43.239+0000 7f63f5200640 1 -- 172.21.15.22:0/3262567993 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f63f01ab240 con 0x7f63f0107e50 2024-03-28T18:10:43.241 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:43.242+0000 7f63f9471740 1 Processor -- start 2024-03-28T18:10:43.242 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:43.242+0000 7f63f9471740 1 -- start start 2024-03-28T18:10:43.242 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:43.242+0000 7f63ecc00640 1 -- 172.21.15.22:0/3262567993 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 26) v1 ==== 137421+0+0 (secure 0 0 0) 0x7f63e0010a50 con 0x7f63f0107e50 2024-03-28T18:10:43.242 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:43.242+0000 7f63f9471740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x556c0c03bdd0 0x556c0c03ab20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:10:43.242 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:43.242+0000 7f63f9471740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556c0c03b0f0 0x556c0c03b570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:10:43.242 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:43.242+0000 7f63f9471740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556c0c044460 0x556c0c046850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:10:43.242 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:43.242+0000 7f63f9471740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x556c0bf9f250 con 0x556c0c03b0f0 2024-03-28T18:10:43.242 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:43.242+0000 7f63f9471740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x556c0bf47a30 con 0x556c0c03bdd0 2024-03-28T18:10:43.242 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:43.242+0000 7f63f9471740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x556c0bf9b2f0 con 0x556c0c044460 2024-03-28T18:10:43.244 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:43.242+0000 7f63ef400640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556c0c03b0f0 0x556c0c03b570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:10:43.244 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:43.242+0000 7f63ef400640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556c0c03b0f0 0x556c0c03b570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:34488/0 (socket says 172.21.15.22:34488) 2024-03-28T18:10:43.244 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:43.242+0000 7f63ef400640 1 -- 172.21.15.22:0/1321471715 learned_addr learned my addr 172.21.15.22:0/1321471715 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-28T18:10:43.244 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:43.243+0000 7f63e7e00640 1 --2- 172.21.15.22:0/1321471715 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556c0c044460 0x556c0c046850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:10:43.244 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:43.243+0000 7f63efe00640 1 --2- 172.21.15.22:0/1321471715 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x556c0c03bdd0 0x556c0c03ab20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:10:43.244 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:43.243+0000 7f63ef400640 1 -- 172.21.15.22:0/1321471715 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556c0c044460 msgr2=0x556c0c046850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T18:10:43.244 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:43.243+0000 7f63ef400640 1 --2- 172.21.15.22:0/1321471715 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556c0c044460 0x556c0c046850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:10:43.244 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:43.243+0000 7f63ef400640 1 -- 172.21.15.22:0/1321471715 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x556c0c03bdd0 msgr2=0x556c0c03ab20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T18:10:43.244 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:43.243+0000 7f63ef400640 1 --2- 172.21.15.22:0/1321471715 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x556c0c03bdd0 0x556c0c03ab20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:10:43.244 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:43.243+0000 7f63ef400640 1 -- 172.21.15.22:0/1321471715 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556c0bf32690 con 0x556c0c03b0f0 2024-03-28T18:10:43.245 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:43.243+0000 7f63ef400640 1 --2- 172.21.15.22:0/1321471715 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556c0c03b0f0 0x556c0c03b570 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7f63e00cc1a0 tx=0x7f63e00d7610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=929a0f4e365ffa1d server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:10:43.245 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:43.244+0000 7f63e4c00640 1 -- 172.21.15.22:0/1321471715 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63e00db660 con 0x556c0c03b0f0 2024-03-28T18:10:43.245 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:43.244+0000 7f63e4c00640 1 -- 172.21.15.22:0/1321471715 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f63e00e0070 con 0x556c0c03b0f0 2024-03-28T18:10:43.245 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:43.244+0000 7f63e4c00640 1 -- 172.21.15.22:0/1321471715 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63e00da9c0 con 0x556c0c03b0f0 2024-03-28T18:10:43.245 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:43.244+0000 7f63ecc00640 1 --2- 172.21.15.22:0/3262567993 >> [v2:172.21.15.22:6800/1357365082,v1:172.21.15.22:6801/1357365082] conn(0x7f63d40a29f0 0x7f63d40a4eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:10:43.245 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:43.244+0000 7f63ecc00640 1 -- 172.21.15.22:0/3262567993 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 8497+0+0 (secure 0 0 0) 0x7f63e00cead0 con 0x7f63f0107e50 2024-03-28T18:10:43.245 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:43.244+0000 7f63efe00640 1 --2- 172.21.15.22:0/3262567993 >> [v2:172.21.15.22:6800/1357365082,v1:172.21.15.22:6801/1357365082] conn(0x7f63d40a29f0 0x7f63d40a4eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:10:43.245 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:43.244+0000 7f63f9471740 1 -- 172.21.15.22:0/1321471715 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556c0c03b0f0 msgr2=0x556c0c03b570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:10:43.245 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:43.244+0000 7f63f9471740 1 --2- 172.21.15.22:0/1321471715 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556c0c03b0f0 0x556c0c03b570 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7f63e00cc1a0 tx=0x7f63e00d7610 comp rx=0 tx=0).stop 2024-03-28T18:10:43.249 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:43.245+0000 7f63f9471740 1 -- 172.21.15.22:0/1321471715 shutdown_connections 2024-03-28T18:10:43.249 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:43.245+0000 7f63f9471740 1 --2- 172.21.15.22:0/1321471715 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556c0c044460 0x556c0c046850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:10:43.249 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:43.245+0000 7f63f9471740 1 --2- 172.21.15.22:0/1321471715 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556c0c03b0f0 0x556c0c03b570 unknown :-1 s=CLOSED pgs=660 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:10:43.249 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:43.245+0000 7f63f9471740 1 --2- 172.21.15.22:0/1321471715 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x556c0c03bdd0 0x556c0c03ab20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:10:43.249 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:43.245+0000 7f63f9471740 1 -- 172.21.15.22:0/1321471715 >> 172.21.15.22:0/1321471715 conn(0x556c0c002200 msgr2=0x556c0c035b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:10:43.249 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:43.245+0000 7f63f9471740 1 -- 172.21.15.22:0/1321471715 shutdown_connections 2024-03-28T18:10:43.249 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:43.245+0000 7f63efe00640 1 --2- 172.21.15.22:0/3262567993 >> [v2:172.21.15.22:6800/1357365082,v1:172.21.15.22:6801/1357365082] conn(0x7f63d40a29f0 0x7f63d40a4eb0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f63e8004af0 tx=0x7f63e8004a40 comp rx=0 tx=0).ready entity=mgr.15522 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:10:43.249 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:43.245+0000 7f63f9471740 1 -- 172.21.15.22:0/1321471715 wait complete. 2024-03-28T18:10:43.250 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:43.245+0000 7f63f9471740 1 -- start start 2024-03-28T18:10:43.250 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:43.246+0000 7f63f9471740 1 Processor -- start 2024-03-28T18:10:43.250 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:43.246+0000 7f63f9471740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x556c0c03b0f0 0x556c0c043b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:10:43.250 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:43.246+0000 7f63f9471740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556c0c03bdd0 0x556c0c040e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:10:43.250 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:43.246+0000 7f63f9471740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556c0c044460 0x556c0c0413d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:10:43.250 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:43.246+0000 7f63f9471740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x556c0c047270 con 0x556c0c044460 2024-03-28T18:10:43.250 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:43.246+0000 7f63f9471740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x556c0bf5ae80 con 0x556c0c03b0f0 2024-03-28T18:10:43.250 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:43.246+0000 7f63f9471740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x556c0c0473f0 con 0x556c0c03bdd0 2024-03-28T18:10:43.250 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:43.246+0000 7f63f9471740 10 client.0 _ll_register_callbacks cb 0x556c0bf6d130 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-28T18:10:43.250 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:43.247+0000 7f63efe00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556c0c044460 0x556c0c0413d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:10:43.250 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:43.247+0000 7f63ef400640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x556c0c03b0f0 0x556c0c043b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:10:43.250 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:43.247+0000 7f63efe00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556c0c044460 0x556c0c0413d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:34494/0 (socket says 172.21.15.22:34494) 2024-03-28T18:10:43.250 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:43.247+0000 7f63efe00640 1 -- 172.21.15.22:0/1015732883 learned_addr learned my addr 172.21.15.22:0/1015732883 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-28T18:10:43.250 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:43.247+0000 7f63e7e00640 1 --2- 172.21.15.22:0/1015732883 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556c0c03bdd0 0x556c0c040e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:10:43.251 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:43.247+0000 7f63efe00640 1 -- 172.21.15.22:0/1015732883 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556c0c03bdd0 msgr2=0x556c0c040e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T18:10:43.251 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:43.247+0000 7f63efe00640 1 --2- 172.21.15.22:0/1015732883 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556c0c03bdd0 0x556c0c040e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:10:43.251 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:43.247+0000 7f63efe00640 1 -- 172.21.15.22:0/1015732883 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x556c0c03b0f0 msgr2=0x556c0c043b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T18:10:43.251 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:43.247+0000 7f63efe00640 1 --2- 172.21.15.22:0/1015732883 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x556c0c03b0f0 0x556c0c043b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:10:43.251 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:43.247+0000 7f63efe00640 1 -- 172.21.15.22:0/1015732883 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x556c0bd7d7d0 con 0x556c0c044460 2024-03-28T18:10:43.251 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:43.247+0000 7f63ef400640 1 --2- 172.21.15.22:0/1015732883 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x556c0c03b0f0 0x556c0c043b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-28T18:10:43.251 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:43.248+0000 7f63efe00640 1 --2- 172.21.15.22:0/1015732883 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556c0c044460 0x556c0c0413d0 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7f63e8036000 tx=0x7f63e802c100 comp rx=0 tx=0).ready entity=mon.0 client_cookie=89d53eacf6b8fdef server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:10:43.251 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:43.248+0000 7f63dd600640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-03-28T18:10:43.251 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:43.248+0000 7f63dd600640 1 -- 172.21.15.22:0/1015732883 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63e8050020 con 0x556c0c044460 2024-03-28T18:10:43.251 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:43.248+0000 7f63dd600640 1 -- 172.21.15.22:0/1015732883 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 8497+0+0 (secure 0 0 0) 0x7f63e8040660 con 0x556c0c044460 2024-03-28T18:10:43.251 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:43.248+0000 7f63f9471740 1 -- 172.21.15.22:0/1015732883 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f63e00e0070 con 0x556c0c044460 2024-03-28T18:10:43.251 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:43.248+0000 7f63dd600640 1 -- 172.21.15.22:0/1015732883 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) v2 ==== 24+0+0 (secure 0 0 0) 0x7f63e805b050 con 0x556c0c044460 2024-03-28T18:10:43.251 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:43.248+0000 7f63f9471740 10 client.16374 fetch_fsmap learned FSMap version 38 2024-03-28T18:10:43.251 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:43.248+0000 7f63f9471740 1 -- 172.21.15.22:0/1015732883 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) v3 -- 0x556c0bcb33f0 con 0x556c0c044460 2024-03-28T18:10:43.251 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:43.249+0000 7f63dd600640 1 -- 172.21.15.22:0/1015732883 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f63e00e0070 con 0x556c0c044460 2024-03-28T18:10:43.252 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:43.249+0000 7f63dd600640 1 -- 172.21.15.22:0/1015732883 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 38) v1 ==== 7950+0+0 (secure 0 0 0) 0x7f63e8034a40 con 0x556c0c044460 2024-03-28T18:10:43.252 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:43.249+0000 7f63f9471740 10 client.16374 fetch_fsmap finished waiting for FSMap version 38 2024-03-28T18:10:43.252 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:43.249+0000 7f63f9471740 10 client.16374 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-28T18:10:43.252 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:43.249+0000 7f63f9471740 20 client.16374 populate_metadata read hostname 'smithi022' 2024-03-28T18:10:43.252 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:43.249+0000 7f63f9471740 1 --2- 172.21.15.22:0/1015732883 >> [v2:172.21.15.79:6838/2441542443,v1:172.21.15.79:6839/2441542443] conn(0x556c0c054390 0x556c0c0567d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:10:43.252 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:43.249+0000 7f63f9471740 4 client.16374 mds_command: new command op to 34553 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-28T18:10:43.252 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:43.249+0000 7f63f9471740 1 -- 172.21.15.22:0/1015732883 --> [v2:172.21.15.79:6838/2441542443,v1:172.21.15.79:6839/2441542443] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x556c0bfac790 con 0x556c0c054390 2024-03-28T18:10:43.252 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:43.249+0000 7f63ef400640 1 --2- 172.21.15.22:0/1015732883 >> [v2:172.21.15.79:6838/2441542443,v1:172.21.15.79:6839/2441542443] conn(0x556c0c054390 0x556c0c0567d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:10:43.253 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:43.250+0000 7f63ef400640 1 --2- 172.21.15.22:0/1015732883 >> [v2:172.21.15.79:6838/2441542443,v1:172.21.15.79:6839/2441542443] conn(0x556c0c054390 0x556c0c0567d0 crc :-1 s=READY pgs=94 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6d6d593316429a3b server_cookie=b6d07a1186be4e9 in_seq=0 out_seq=0 2024-03-28T18:10:43.253 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:43.250+0000 7f63dd600640 10 client.16374 ms_handle_connect on v2:172.21.15.79:6838/2441542443 2024-03-28T18:10:43.253 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:43.252+0000 7f63dd600640 1 -- 172.21.15.22:0/1015732883 <== mds.0 v2:172.21.15.79:6838/2441542443 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x556c0bfac790 con 0x556c0c054390 2024-03-28T18:10:43.253 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:43.252+0000 7f63dd600640 10 client.16374 handle_command_reply: tid=0 2024-03-28T18:10:43.253 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:43.252+0000 7f63f9471740 1 client.16374 shutdown 2024-03-28T18:10:43.254 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:43.252+0000 7f63f9471740 1 -- 172.21.15.22:0/1015732883 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556c0c044460 msgr2=0x556c0c0413d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:10:43.254 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:43.252+0000 7f63f9471740 1 --2- 172.21.15.22:0/1015732883 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556c0c044460 0x556c0c0413d0 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7f63e8036000 tx=0x7f63e802c100 comp rx=0 tx=0).stop 2024-03-28T18:10:43.254 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:43.252+0000 7f63f9471740 1 -- 172.21.15.22:0/1015732883 shutdown_connections 2024-03-28T18:10:43.254 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:43.252+0000 7f63f9471740 1 --2- 172.21.15.22:0/1015732883 >> [v2:172.21.15.79:6838/2441542443,v1:172.21.15.79:6839/2441542443] conn(0x556c0c054390 0x556c0c0567d0 crc :-1 s=READY pgs=94 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:10:43.254 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:43.252+0000 7f63f9471740 1 --2- 172.21.15.22:0/1015732883 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556c0c044460 0x556c0c0413d0 unknown :-1 s=CLOSED pgs=661 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:10:43.254 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:43.252+0000 7f63f9471740 1 --2- 172.21.15.22:0/1015732883 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556c0c03bdd0 0x556c0c040e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:10:43.254 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:43.252+0000 7f63f9471740 1 --2- 172.21.15.22:0/1015732883 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x556c0c03b0f0 0x556c0c043b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:10:43.254 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:43.252+0000 7f63f9471740 1 -- 172.21.15.22:0/1015732883 >> 172.21.15.22:0/1015732883 conn(0x556c0c002200 msgr2=0x556c0c037890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:10:43.254 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:43.252+0000 7f63dd600640 0 client.16374 ms_handle_reset on v2:172.21.15.79:6838/2441542443 2024-03-28T18:10:43.254 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:43.253+0000 7f63f9471740 1 -- 172.21.15.22:0/1015732883 shutdown_connections 2024-03-28T18:10:43.254 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:43.253+0000 7f63f9471740 1 -- 172.21.15.22:0/1015732883 wait complete. 2024-03-28T18:10:43.254 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:43.253+0000 7f63f9471740 20 client.16374 trim_cache size 0 max 16384 2024-03-28T18:10:43.271 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:43.270+0000 7f63f9471740 1 Processor -- start 2024-03-28T18:10:43.271 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:43.270+0000 7f63f9471740 1 -- start start 2024-03-28T18:10:43.271 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:43.271+0000 7f63f9471740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556c0c03b0f0 0x556c0c03fef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:10:43.272 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:43.271+0000 7f63f9471740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x556c0c03bdd0 0x556c0c0435c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:10:43.272 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:43.271+0000 7f63f9471740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556c0c044460 0x556c0c043b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:10:43.272 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:43.271+0000 7f63f9471740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63f01b3490 con 0x556c0c044460 2024-03-28T18:10:43.272 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:43.271+0000 7f63f9471740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63f01b4ed0 con 0x556c0c03bdd0 2024-03-28T18:10:43.272 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:43.271+0000 7f63f9471740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63f01b3af0 con 0x556c0c03b0f0 2024-03-28T18:10:43.272 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:43.271+0000 7f63efe00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556c0c044460 0x556c0c043b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:10:43.272 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:43.271+0000 7f63efe00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556c0c044460 0x556c0c043b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:34498/0 (socket says 172.21.15.22:34498) 2024-03-28T18:10:43.272 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:43.271+0000 7f63efe00640 1 -- 172.21.15.22:0/2781501224 learned_addr learned my addr 172.21.15.22:0/2781501224 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-28T18:10:43.272 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:43.271+0000 7f63ef400640 1 --2- 172.21.15.22:0/2781501224 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556c0c03b0f0 0x556c0c03fef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:10:43.272 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:43.271+0000 7f63e7e00640 1 --2- 172.21.15.22:0/2781501224 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x556c0c03bdd0 0x556c0c0435c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:10:43.272 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:43.271+0000 7f63efe00640 1 -- 172.21.15.22:0/2781501224 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556c0c03b0f0 msgr2=0x556c0c03fef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T18:10:43.272 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:43.271+0000 7f63efe00640 1 --2- 172.21.15.22:0/2781501224 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556c0c03b0f0 0x556c0c03fef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:10:43.272 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:43.271+0000 7f63efe00640 1 -- 172.21.15.22:0/2781501224 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x556c0c03bdd0 msgr2=0x556c0c0435c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T18:10:43.272 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:43.271+0000 7f63efe00640 1 --2- 172.21.15.22:0/2781501224 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x556c0c03bdd0 0x556c0c0435c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:10:43.273 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:43.271+0000 7f63efe00640 1 -- 172.21.15.22:0/2781501224 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556c0bc730f0 con 0x556c0c044460 2024-03-28T18:10:43.273 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:43.272+0000 7f63e7e00640 1 --2- 172.21.15.22:0/2781501224 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x556c0c03bdd0 0x556c0c0435c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T18:10:43.273 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:43.272+0000 7f63efe00640 1 --2- 172.21.15.22:0/2781501224 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556c0c044460 0x556c0c043b00 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7f63e804c3e0 tx=0x7f63e8032da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6a4c4e2d46115549 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:10:43.276 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:43.272+0000 7f63dfe00640 1 -- 172.21.15.22:0/2781501224 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63e8064070 con 0x556c0c044460 2024-03-28T18:10:43.276 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:43.272+0000 7f63dfe00640 1 -- 172.21.15.22:0/2781501224 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f63e8047070 con 0x556c0c044460 2024-03-28T18:10:43.276 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:43.272+0000 7f63dfe00640 1 -- 172.21.15.22:0/2781501224 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63e8050070 con 0x556c0c044460 2024-03-28T18:10:43.276 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:43.273+0000 7f63f9471740 1 -- 172.21.15.22:0/2781501224 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556c0c044460 msgr2=0x556c0c043b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:10:43.276 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:43.273+0000 7f63f9471740 1 --2- 172.21.15.22:0/2781501224 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556c0c044460 0x556c0c043b00 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7f63e804c3e0 tx=0x7f63e8032da0 comp rx=0 tx=0).stop 2024-03-28T18:10:43.276 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:43.273+0000 7f63f9471740 1 -- 172.21.15.22:0/2781501224 shutdown_connections 2024-03-28T18:10:43.276 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:43.273+0000 7f63f9471740 1 --2- 172.21.15.22:0/2781501224 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556c0c044460 0x556c0c043b00 unknown :-1 s=CLOSED pgs=662 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:10:43.276 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:43.273+0000 7f63f9471740 1 --2- 172.21.15.22:0/2781501224 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x556c0c03bdd0 0x556c0c0435c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:10:43.277 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:43.273+0000 7f63f9471740 1 --2- 172.21.15.22:0/2781501224 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556c0c03b0f0 0x556c0c03fef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:10:43.277 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:43.273+0000 7f63f9471740 1 -- 172.21.15.22:0/2781501224 >> 172.21.15.22:0/2781501224 conn(0x556c0c002200 msgr2=0x556c0c03f7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:10:43.277 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:43.273+0000 7f63f9471740 1 -- 172.21.15.22:0/2781501224 shutdown_connections 2024-03-28T18:10:43.277 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:43.273+0000 7f63f9471740 1 -- 172.21.15.22:0/2781501224 wait complete. 2024-03-28T18:10:43.277 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:43.273+0000 7f63f9471740 1 -- start start 2024-03-28T18:10:43.277 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:43.274+0000 7f63f9471740 1 Processor -- start 2024-03-28T18:10:43.277 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:43.274+0000 7f63f9471740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556c0c03b0f0 0x556c0c03d850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:10:43.277 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:43.274+0000 7f63f9471740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556c0c03bdd0 0x556c0c03dd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:10:43.277 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:43.274+0000 7f63f9471740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x556c0c044460 0x556c0c03e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:10:43.277 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:43.274+0000 7f63f9471740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63f01b4870 con 0x556c0c03bdd0 2024-03-28T18:10:43.277 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:43.274+0000 7f63f9471740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63f01b4150 con 0x556c0c044460 2024-03-28T18:10:43.277 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:43.274+0000 7f63f9471740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63f01b16b0 con 0x556c0c03b0f0 2024-03-28T18:10:43.277 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:43.274+0000 7f63f9471740 10 client.0 _ll_register_callbacks cb 0x7f63e8037070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-28T18:10:43.277 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:43.274+0000 7f63e7e00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556c0c03bdd0 0x556c0c03dd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:10:43.277 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:43.274+0000 7f63e7e00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556c0c03bdd0 0x556c0c03dd90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:34504/0 (socket says 172.21.15.22:34504) 2024-03-28T18:10:43.278 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:43.274+0000 7f63e7e00640 1 -- 172.21.15.22:0/2747884261 learned_addr learned my addr 172.21.15.22:0/2747884261 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-28T18:10:43.278 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:43.274+0000 7f63ef400640 1 --2- 172.21.15.22:0/2747884261 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556c0c03b0f0 0x556c0c03d850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:10:43.278 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:43.275+0000 7f63efe00640 1 --2- 172.21.15.22:0/2747884261 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x556c0c044460 0x556c0c03e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:10:43.278 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:43.275+0000 7f63e7e00640 1 -- 172.21.15.22:0/2747884261 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556c0c03b0f0 msgr2=0x556c0c03d850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T18:10:43.278 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:43.275+0000 7f63e7e00640 1 --2- 172.21.15.22:0/2747884261 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556c0c03b0f0 0x556c0c03d850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:10:43.278 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:43.275+0000 7f63e7e00640 1 -- 172.21.15.22:0/2747884261 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x556c0c044460 msgr2=0x556c0c03e2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T18:10:43.278 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:43.275+0000 7f63e7e00640 1 --2- 172.21.15.22:0/2747884261 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x556c0c044460 0x556c0c03e2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:10:43.278 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:43.275+0000 7f63e7e00640 1 -- 172.21.15.22:0/2747884261 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x556c0c0aef30 con 0x556c0c03bdd0 2024-03-28T18:10:43.278 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:43.275+0000 7f63e7e00640 1 --2- 172.21.15.22:0/2747884261 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556c0c03bdd0 0x556c0c03dd90 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7f63f006abd0 tx=0x7f63d80138f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cb73168033cb622d server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:10:43.278 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:43.275+0000 7f63dd600640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-03-28T18:10:43.278 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:43.275+0000 7f63dd600640 1 -- 172.21.15.22:0/2747884261 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63d8005020 con 0x556c0c03bdd0 2024-03-28T18:10:43.278 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:43.275+0000 7f63dd600640 1 -- 172.21.15.22:0/2747884261 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 8497+0+0 (secure 0 0 0) 0x7f63d80051c0 con 0x556c0c03bdd0 2024-03-28T18:10:43.278 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:43.275+0000 7f63f9471740 1 -- 172.21.15.22:0/2747884261 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f63e8047070 con 0x556c0c03bdd0 2024-03-28T18:10:43.279 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:43.276+0000 7f63dd600640 1 -- 172.21.15.22:0/2747884261 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) v2 ==== 24+0+0 (secure 0 0 0) 0x7f63d8005400 con 0x556c0c03bdd0 2024-03-28T18:10:43.279 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:43.276+0000 7f63f9471740 10 client.16380 fetch_fsmap learned FSMap version 38 2024-03-28T18:10:43.279 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:43.276+0000 7f63f9471740 1 -- 172.21.15.22:0/2747884261 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) v3 -- 0x556c0c0b0240 con 0x556c0c03bdd0 2024-03-28T18:10:43.279 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:43.276+0000 7f63dd600640 1 -- 172.21.15.22:0/2747884261 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f63e8047070 con 0x556c0c03bdd0 2024-03-28T18:10:43.279 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:43.276+0000 7f63dd600640 1 -- 172.21.15.22:0/2747884261 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 38) v1 ==== 7950+0+0 (secure 0 0 0) 0x7f63d801d730 con 0x556c0c03bdd0 2024-03-28T18:10:43.279 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:43.276+0000 7f63f9471740 10 client.16380 fetch_fsmap finished waiting for FSMap version 38 2024-03-28T18:10:43.279 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:43.276+0000 7f63f9471740 10 client.16380 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-28T18:10:43.279 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:43.276+0000 7f63f9471740 20 client.16380 populate_metadata read hostname 'smithi022' 2024-03-28T18:10:43.279 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:43.276+0000 7f63f9471740 1 --2- 172.21.15.22:0/2747884261 >> [v2:172.21.15.79:6838/2441542443,v1:172.21.15.79:6839/2441542443] conn(0x556c0c0b0a40 0x556c0c0b2e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:10:43.279 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:43.276+0000 7f63f9471740 4 client.16380 mds_command: new command op to 34553 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-28T18:10:43.279 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:43.276+0000 7f63f9471740 1 -- 172.21.15.22:0/2747884261 --> [v2:172.21.15.79:6838/2441542443,v1:172.21.15.79:6839/2441542443] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x556c0bfa6ae0 con 0x556c0c0b0a40 2024-03-28T18:10:43.281 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:43.277+0000 7f63ef400640 1 --2- 172.21.15.22:0/2747884261 >> [v2:172.21.15.79:6838/2441542443,v1:172.21.15.79:6839/2441542443] conn(0x556c0c0b0a40 0x556c0c0b2e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:10:43.281 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:43.278+0000 7f63ef400640 1 --2- 172.21.15.22:0/2747884261 >> [v2:172.21.15.79:6838/2441542443,v1:172.21.15.79:6839/2441542443] conn(0x556c0c0b0a40 0x556c0c0b2e60 crc :-1 s=READY pgs=95 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ca568bc9468fa323 server_cookie=fda0a4e604d532c7 in_seq=0 out_seq=0 2024-03-28T18:10:43.281 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:43.278+0000 7f63dd600640 10 client.16380 ms_handle_connect on v2:172.21.15.79:6838/2441542443 2024-03-28T18:10:43.281 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:43.278+0000 7f63dd600640 1 -- 172.21.15.22:0/2747884261 <== mds.0 v2:172.21.15.79:6838/2441542443 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x556c0bfa6ae0 con 0x556c0c0b0a40 2024-03-28T18:10:43.281 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:43.278+0000 7f63dd600640 10 client.16380 handle_command_reply: tid=0 2024-03-28T18:10:43.281 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:43.278+0000 7f63f9471740 1 client.16380 shutdown 2024-03-28T18:10:43.281 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:43.278+0000 7f63f9471740 1 -- 172.21.15.22:0/2747884261 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556c0c03bdd0 msgr2=0x556c0c03dd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:10:43.281 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:43.278+0000 7f63f9471740 1 --2- 172.21.15.22:0/2747884261 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556c0c03bdd0 0x556c0c03dd90 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7f63f006abd0 tx=0x7f63d80138f0 comp rx=0 tx=0).stop 2024-03-28T18:10:43.281 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:43.278+0000 7f63f9471740 1 -- 172.21.15.22:0/2747884261 shutdown_connections 2024-03-28T18:10:43.281 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:43.278+0000 7f63f9471740 1 --2- 172.21.15.22:0/2747884261 >> [v2:172.21.15.79:6838/2441542443,v1:172.21.15.79:6839/2441542443] conn(0x556c0c0b0a40 0x556c0c0b2e60 crc :-1 s=READY pgs=95 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:10:43.281 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:43.278+0000 7f63f9471740 1 --2- 172.21.15.22:0/2747884261 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x556c0c044460 0x556c0c03e2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:10:43.281 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:43.278+0000 7f63f9471740 1 --2- 172.21.15.22:0/2747884261 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556c0c03bdd0 0x556c0c03dd90 unknown :-1 s=CLOSED pgs=663 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:10:43.282 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:43.278+0000 7f63f9471740 1 --2- 172.21.15.22:0/2747884261 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556c0c03b0f0 0x556c0c03d850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:10:43.282 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:43.278+0000 7f63dd600640 0 client.16380 ms_handle_reset on v2:172.21.15.79:6838/2441542443 2024-03-28T18:10:43.282 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:43.278+0000 7f63f9471740 1 -- 172.21.15.22:0/2747884261 >> 172.21.15.22:0/2747884261 conn(0x556c0c002200 msgr2=0x556c0c034330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:10:43.282 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:43.279+0000 7f63f9471740 1 -- 172.21.15.22:0/2747884261 shutdown_connections 2024-03-28T18:10:43.282 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:43.279+0000 7f63f9471740 1 -- 172.21.15.22:0/2747884261 wait complete. 2024-03-28T18:10:43.282 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:43.279+0000 7f63f9471740 20 client.16380 trim_cache size 0 max 16384 2024-03-28T18:10:43.282 INFO:teuthology.orchestra.run.smithi022.stdout:{ 2024-03-28T18:10:43.282 INFO:teuthology.orchestra.run.smithi022.stdout: "status": "no active scrubs running", 2024-03-28T18:10:43.282 INFO:teuthology.orchestra.run.smithi022.stdout: "scrubs": {} 2024-03-28T18:10:43.282 INFO:teuthology.orchestra.run.smithi022.stdout:} 2024-03-28T18:10:43.283 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:43.279+0000 7f63f5200640 1 -- 172.21.15.22:0/3262567993 >> [v2:172.21.15.22:6800/1357365082,v1:172.21.15.22:6801/1357365082] conn(0x7f63d40a29f0 msgr2=0x7f63d40a4eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:10:43.283 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:43.279+0000 7f63f5200640 1 --2- 172.21.15.22:0/3262567993 >> [v2:172.21.15.22:6800/1357365082,v1:172.21.15.22:6801/1357365082] conn(0x7f63d40a29f0 0x7f63d40a4eb0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f63e8004af0 tx=0x7f63e8004a40 comp rx=0 tx=0).stop 2024-03-28T18:10:43.283 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:43.279+0000 7f63f5200640 1 -- 172.21.15.22:0/3262567993 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f63f0107e50 msgr2=0x7f63f019e3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:10:43.284 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:43.279+0000 7f63f5200640 1 --2- 172.21.15.22:0/3262567993 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f63f0107e50 0x7f63f019e3c0 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7f63e000d950 tx=0x7f63e000de10 comp rx=0 tx=0).stop 2024-03-28T18:10:43.284 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:43.280+0000 7f63f5200640 1 -- 172.21.15.22:0/3262567993 shutdown_connections 2024-03-28T18:10:43.284 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:43.280+0000 7f63f5200640 1 --2- 172.21.15.22:0/3262567993 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f63f01a2c80 0x7f63f01a3130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:10:43.284 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:43.280+0000 7f63f5200640 1 --2- 172.21.15.22:0/3262567993 >> [v2:172.21.15.22:6800/1357365082,v1:172.21.15.22:6801/1357365082] conn(0x7f63d40a29f0 0x7f63d40a4eb0 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:10:43.284 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:43.280+0000 7f63f5200640 1 --2- 172.21.15.22:0/3262567993 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f63f010cea0 0x7f63f019e900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:10:43.284 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:43.280+0000 7f63f5200640 1 --2- 172.21.15.22:0/3262567993 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f63f0107e50 0x7f63f019e3c0 unknown :-1 s=CLOSED pgs=659 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:10:43.284 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:43.280+0000 7f63f5200640 1 -- 172.21.15.22:0/3262567993 >> 172.21.15.22:0/3262567993 conn(0x7f63f00774c0 msgr2=0x7f63f0077ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:10:43.284 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:43.280+0000 7f63f5200640 1 -- 172.21.15.22:0/3262567993 shutdown_connections 2024-03-28T18:10:43.284 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:43.280+0000 7f63f5200640 1 -- 172.21.15.22:0/3262567993 wait complete. 2024-03-28T18:10:43.296 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-03-28T18:10:43.296 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-28T18:10:43.513 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:43.513+0000 7fd9ad800640 1 -- 172.21.15.22:0/931487915 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd9a8108420 msgr2=0x7fd9a8114930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:10:43.513 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:43.513+0000 7fd9ad800640 1 --2- 172.21.15.22:0/931487915 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd9a8108420 0x7fd9a8114930 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7fd9a0009c10 tx=0x7fd9a002bee0 comp rx=0 tx=0).stop 2024-03-28T18:10:43.513 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:43.513+0000 7fd9ad800640 1 -- 172.21.15.22:0/931487915 shutdown_connections 2024-03-28T18:10:43.513 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:43.513+0000 7fd9ad800640 1 --2- 172.21.15.22:0/931487915 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd9a8108420 0x7fd9a8114930 unknown :-1 s=CLOSED pgs=664 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:10:43.513 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:43.513+0000 7fd9ad800640 1 --2- 172.21.15.22:0/931487915 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd9a8107a60 0x7fd9a8107ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:10:43.513 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:43.513+0000 7fd9ad800640 1 --2- 172.21.15.22:0/931487915 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd9a810da70 0x7fd9a810de50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:10:43.514 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:43.513+0000 7fd9ad800640 1 -- 172.21.15.22:0/931487915 >> 172.21.15.22:0/931487915 conn(0x7fd9a80771c0 msgr2=0x7fd9a80795e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:10:43.514 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:43.513+0000 7fd9ad800640 1 -- 172.21.15.22:0/931487915 shutdown_connections 2024-03-28T18:10:43.514 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:43.513+0000 7fd9ad800640 1 -- 172.21.15.22:0/931487915 wait complete. 2024-03-28T18:10:43.516 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:43.514+0000 7fd9ad800640 1 Processor -- start 2024-03-28T18:10:43.516 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:43.514+0000 7fd9ad800640 1 -- start start 2024-03-28T18:10:43.516 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:43.515+0000 7fd9ad800640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd9a8107a60 0x7fd9a819e2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:10:43.516 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:43.515+0000 7fd9ad800640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd9a8108420 0x7fd9a819e820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:10:43.516 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:43.515+0000 7fd9ad800640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd9a810da70 0x7fd9a81a2ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:10:43.516 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:43.515+0000 7fd9ad800640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9a8117ea0 con 0x7fd9a810da70 2024-03-28T18:10:43.516 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:43.515+0000 7fd9ad800640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9a8117d20 con 0x7fd9a8108420 2024-03-28T18:10:43.516 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:43.515+0000 7fd9ad800640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9a8118020 con 0x7fd9a8107a60 2024-03-28T18:10:43.516 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:43.515+0000 7fd9ace00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd9a810da70 0x7fd9a81a2ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:10:43.517 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:43.515+0000 7fd9a7e00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd9a8107a60 0x7fd9a819e2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:10:43.517 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:43.515+0000 7fd9ace00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd9a810da70 0x7fd9a81a2ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:34516/0 (socket says 172.21.15.22:34516) 2024-03-28T18:10:43.517 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:43.515+0000 7fd9ace00640 1 -- 172.21.15.22:0/813358153 learned_addr learned my addr 172.21.15.22:0/813358153 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-28T18:10:43.517 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:43.515+0000 7fd99fe00640 1 --2- 172.21.15.22:0/813358153 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd9a8108420 0x7fd9a819e820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:10:43.517 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:43.515+0000 7fd9ace00640 1 -- 172.21.15.22:0/813358153 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd9a8107a60 msgr2=0x7fd9a819e2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:10:43.517 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:43.515+0000 7fd9ace00640 1 --2- 172.21.15.22:0/813358153 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd9a8107a60 0x7fd9a819e2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:10:43.517 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:43.515+0000 7fd9ace00640 1 -- 172.21.15.22:0/813358153 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd9a8108420 msgr2=0x7fd9a819e820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:10:43.517 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:43.515+0000 7fd9ace00640 1 --2- 172.21.15.22:0/813358153 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd9a8108420 0x7fd9a819e820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:10:43.517 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:43.515+0000 7fd9ace00640 1 -- 172.21.15.22:0/813358153 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd9a81a32d0 con 0x7fd9a810da70 2024-03-28T18:10:43.517 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:43.515+0000 7fd99fe00640 1 --2- 172.21.15.22:0/813358153 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd9a8108420 0x7fd9a819e820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-28T18:10:43.517 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:43.516+0000 7fd9ace00640 1 --2- 172.21.15.22:0/813358153 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd9a810da70 0x7fd9a81a2ba0 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7fd9a002c420 tx=0x7fd9a0033730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:10:43.517 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:43.516+0000 7fd9a5600640 1 -- 172.21.15.22:0/813358153 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9a0043020 con 0x7fd9a810da70 2024-03-28T18:10:43.517 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:43.516+0000 7fd9a5600640 1 -- 172.21.15.22:0/813358153 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7fd9a0003b80 con 0x7fd9a810da70 2024-03-28T18:10:43.517 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:43.516+0000 7fd9ad800640 1 -- 172.21.15.22:0/813358153 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd9a81a3560 con 0x7fd9a810da70 2024-03-28T18:10:43.518 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:43.516+0000 7fd9a5600640 1 -- 172.21.15.22:0/813358153 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9a003e6b0 con 0x7fd9a810da70 2024-03-28T18:10:43.518 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:43.516+0000 7fd9ad800640 1 -- 172.21.15.22:0/813358153 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd9a81aae50 con 0x7fd9a810da70 2024-03-28T18:10:43.518 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:43.517+0000 7fd9ad800640 1 -- 172.21.15.22:0/813358153 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd9a8068800 con 0x7fd9a810da70 2024-03-28T18:10:43.519 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:43.519+0000 7fd9a5600640 1 -- 172.21.15.22:0/813358153 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 26) v1 ==== 137421+0+0 (secure 0 0 0) 0x7fd9a0046020 con 0x7fd9a810da70 2024-03-28T18:10:43.521 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:43.521+0000 7fd9a5600640 1 --2- 172.21.15.22:0/813358153 >> [v2:172.21.15.22:6800/1357365082,v1:172.21.15.22:6801/1357365082] conn(0x7fd98c0a29f0 0x7fd98c0a4eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:10:43.521 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:43.521+0000 7fd9a5600640 1 -- 172.21.15.22:0/813358153 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 8497+0+0 (secure 0 0 0) 0x7fd9a00f15b0 con 0x7fd9a810da70 2024-03-28T18:10:43.521 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:43.521+0000 7fd9a7e00640 1 --2- 172.21.15.22:0/813358153 >> [v2:172.21.15.22:6800/1357365082,v1:172.21.15.22:6801/1357365082] conn(0x7fd98c0a29f0 0x7fd98c0a4eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:10:43.522 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:43.522+0000 7fd9a7e00640 1 --2- 172.21.15.22:0/813358153 >> [v2:172.21.15.22:6800/1357365082,v1:172.21.15.22:6801/1357365082] conn(0x7fd98c0a29f0 0x7fd98c0a4eb0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fd998009b50 tx=0x7fd998005f90 comp rx=0 tx=0).ready entity=mgr.15522 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:10:43.525 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:43.526+0000 7fd9a5600640 1 -- 172.21.15.22:0/813358153 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fd9a004d020 con 0x7fd9a810da70 2024-03-28T18:10:43.606 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:10:43 smithi134 systemd[1]: ceph-fd220748-ed2b-11ee-95d0-87774f69a715@mds.f.service: Scheduled restart job, restart counter is at 1. 2024-03-28T18:10:43.606 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:10:43 smithi134 systemd[1]: Stopped Ceph mds.f for fd220748-ed2b-11ee-95d0-87774f69a715. 2024-03-28T18:10:43.606 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:10:43 smithi134 systemd[1]: ceph-fd220748-ed2b-11ee-95d0-87774f69a715@mds.f.service: Consumed 3.440s CPU time. 2024-03-28T18:10:43.606 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:10:43 smithi134 systemd[1]: Starting Ceph mds.f for fd220748-ed2b-11ee-95d0-87774f69a715... 2024-03-28T18:10:43.768 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:43.768+0000 7fd9ad800640 1 -- 172.21.15.22:0/813358153 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd9a81ab0f0 con 0x7fd9a810da70 2024-03-28T18:10:43.770 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:43.770+0000 7fd9a5600640 1 -- 172.21.15.22:0/813358153 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 38 v38) v1 ==== 94+0+11461 (secure 0 0 0) 0x7fd9a00be3d0 con 0x7fd9a810da70 2024-03-28T18:10:43.770 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-03-28T18:10:43.771 INFO:teuthology.orchestra.run.smithi022.stdout:{"epoch":38,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14544,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6835/3718238950","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6834","nonce":3718238950},{"type":"v1","addr":"172.21.15.22:6835","nonce":3718238950}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14562,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6837/3676939386","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6836","nonce":3676939386},{"type":"v1","addr":"172.21.15.22:6837","nonce":3676939386}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":38,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-28T18:04:48.991622+0000","modified":"2024-03-28T18:10:31.275543+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":94,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":34553,"mds_1":24463,"mds_2":34475,"mds_3":34442,"mds_4":24502},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14640":{"gid":14640,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.22:6839/3979998310","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6838","nonce":3979998310},{"type":"v1","addr":"172.21.15.22:6839","nonce":3979998310}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_14655":{"gid":14655,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.22:6841/3069056661","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6840","nonce":3069056661},{"type":"v1","addr":"172.21.15.22:6841","nonce":3069056661}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24412":{"gid":24412,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.134:6833/3176125362","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6832","nonce":3176125362},{"type":"v1","addr":"172.21.15.134:6833","nonce":3176125362}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24463":{"gid":24463,"name":"f","rank":1,"incarnation":33,"state":"up:active","state_seq":96,"addr":"172.21.15.134:6835/3370549207","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6834","nonce":3370549207},{"type":"v1","addr":"172.21.15.134:6835","nonce":3370549207}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24478":{"gid":24478,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.134:6837/899825581","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6836","nonce":899825581},{"type":"v1","addr":"172.21.15.134:6837","nonce":899825581}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24502":{"gid":24502,"name":"i","rank":4,"incarnation":24,"state":"up:active","state_seq":7,"addr":"172.21.15.134:6839/3400263217","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6838","nonce":3400263217},{"type":"v1","addr":"172.21.15.134:6839","nonce":3400263217}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_34442":{"gid":34442,"name":"k","rank":3,"incarnation":22,"state":"up:active","state_seq":14,"addr":"172.21.15.79:6833/856896997","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6832","nonce":856896997},{"type":"v1","addr":"172.21.15.79:6833","nonce":856896997}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_34475":{"gid":34475,"name":"e","rank":2,"incarnation":20,"state":"up:active","state_seq":11,"addr":"172.21.15.79:6835/3365773431","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6834","nonce":3365773431},{"type":"v1","addr":"172.21.15.79:6835","nonce":3365773431}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_34553":{"gid":34553,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.79:6839/2441542443","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6838","nonce":2441542443},{"type":"v1","addr":"172.21.15.79:6839","nonce":2441542443}]},"join_fscid":-1,"export_targets":[1,2,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_36188":{"gid":36188,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.79:6837/3437531480","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6836","nonce":3437531480},{"type":"v1","addr":"172.21.15.79:6837","nonce":3437531480}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":34553,"qdb_cluster":[34475,34553,24463,24502,34442]},"id":1}]} 2024-03-28T18:10:43.771 INFO:teuthology.orchestra.run.smithi022.stderr:dumped fsmap epoch 38 2024-03-28T18:10:43.773 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:43.773+0000 7fd9ad800640 1 -- 172.21.15.22:0/813358153 >> [v2:172.21.15.22:6800/1357365082,v1:172.21.15.22:6801/1357365082] conn(0x7fd98c0a29f0 msgr2=0x7fd98c0a4eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:10:43.773 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:43.773+0000 7fd9ad800640 1 --2- 172.21.15.22:0/813358153 >> [v2:172.21.15.22:6800/1357365082,v1:172.21.15.22:6801/1357365082] conn(0x7fd98c0a29f0 0x7fd98c0a4eb0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fd998009b50 tx=0x7fd998005f90 comp rx=0 tx=0).stop 2024-03-28T18:10:43.773 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:43.773+0000 7fd9ad800640 1 -- 172.21.15.22:0/813358153 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd9a810da70 msgr2=0x7fd9a81a2ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:10:43.773 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:43.773+0000 7fd9ad800640 1 --2- 172.21.15.22:0/813358153 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd9a810da70 0x7fd9a81a2ba0 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7fd9a002c420 tx=0x7fd9a0033730 comp rx=0 tx=0).stop 2024-03-28T18:10:43.773 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:43.773+0000 7fd9ad800640 1 -- 172.21.15.22:0/813358153 shutdown_connections 2024-03-28T18:10:43.773 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:43.773+0000 7fd9ad800640 1 --2- 172.21.15.22:0/813358153 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd9a810da70 0x7fd9a81a2ba0 unknown :-1 s=CLOSED pgs=665 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:10:43.773 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:43.773+0000 7fd9ad800640 1 --2- 172.21.15.22:0/813358153 >> [v2:172.21.15.22:6800/1357365082,v1:172.21.15.22:6801/1357365082] conn(0x7fd98c0a29f0 0x7fd98c0a4eb0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:10:43.773 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:43.773+0000 7fd9ad800640 1 --2- 172.21.15.22:0/813358153 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd9a8108420 0x7fd9a819e820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:10:43.773 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:43.773+0000 7fd9ad800640 1 --2- 172.21.15.22:0/813358153 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd9a8107a60 0x7fd9a819e2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:10:43.773 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:43.773+0000 7fd9ad800640 1 -- 172.21.15.22:0/813358153 >> 172.21.15.22:0/813358153 conn(0x7fd9a80771c0 msgr2=0x7fd9a8077700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:10:43.774 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:43.773+0000 7fd9ad800640 1 -- 172.21.15.22:0/813358153 shutdown_connections 2024-03-28T18:10:43.774 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:43.774+0000 7fd9ad800640 1 -- 172.21.15.22:0/813358153 wait complete. 2024-03-28T18:10:43.793 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-28T18:10:43.793 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-03-28T18:10:43.883 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:10:43 smithi134 podman[69755]: 2024-03-28 18:10:43.511670714 +0000 UTC m=+0.033191013 image pull c664afe56bb01db4f9afa9532c20598e201e664f2e9ba2be50db0d52674a26d2 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc 2024-03-28T18:10:43.883 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:10:43 smithi134 podman[69755]: 2024-03-28 18:10:43.638365658 +0000 UTC m=+0.159885981 container create d152d7307b05bc350e7621116317bf1237be480aada83ebe56fc7359227b8025 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f, GIT_BRANCH=HEAD, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240319, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, io.buildah.version=1.33.5, ceph=True) 2024-03-28T18:10:44.017 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.017+0000 7fb621000640 1 -- 172.21.15.22:0/3068294139 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fb61c101530 msgr2=0x7fb61c1019b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:10:44.017 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.017+0000 7fb621000640 1 --2- 172.21.15.22:0/3068294139 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fb61c101530 0x7fb61c1019b0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fb604009c10 tx=0x7fb60402bee0 comp rx=0 tx=0).stop 2024-03-28T18:10:44.017 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.017+0000 7fb621000640 1 -- 172.21.15.22:0/3068294139 shutdown_connections 2024-03-28T18:10:44.017 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.017+0000 7fb621000640 1 --2- 172.21.15.22:0/3068294139 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb61c101ef0 0x7fb61c10e3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:10:44.018 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.017+0000 7fb621000640 1 --2- 172.21.15.22:0/3068294139 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fb61c101530 0x7fb61c1019b0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:10:44.018 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.017+0000 7fb621000640 1 --2- 172.21.15.22:0/3068294139 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb61c1075f0 0x7fb61c1079d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:10:44.018 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.017+0000 7fb621000640 1 -- 172.21.15.22:0/3068294139 >> 172.21.15.22:0/3068294139 conn(0x7fb61c0fd000 msgr2=0x7fb61c0ff420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:10:44.018 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.017+0000 7fb621000640 1 -- 172.21.15.22:0/3068294139 shutdown_connections 2024-03-28T18:10:44.018 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.017+0000 7fb621000640 1 -- 172.21.15.22:0/3068294139 wait complete. 2024-03-28T18:10:44.020 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.018+0000 7fb621000640 1 Processor -- start 2024-03-28T18:10:44.021 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.018+0000 7fb621000640 1 -- start start 2024-03-28T18:10:44.021 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.019+0000 7fb621000640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb61c101530 0x7fb61c199fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:10:44.021 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.019+0000 7fb621000640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fb61c101ef0 0x7fb61c19a500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:10:44.021 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.019+0000 7fb621000640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb61c1075f0 0x7fb61c19e880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:10:44.021 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.019+0000 7fb621000640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb61c1118c0 con 0x7fb61c101530 2024-03-28T18:10:44.021 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.019+0000 7fb621000640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb61c111740 con 0x7fb61c101ef0 2024-03-28T18:10:44.021 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.019+0000 7fb621000640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb61c111a40 con 0x7fb61c1075f0 2024-03-28T18:10:44.022 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.019+0000 7fb61b400640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb61c101530 0x7fb61c199fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:10:44.022 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.019+0000 7fb61aa00640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fb61c101ef0 0x7fb61c19a500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:10:44.022 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.019+0000 7fb61b400640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb61c101530 0x7fb61c199fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:34520/0 (socket says 172.21.15.22:34520) 2024-03-28T18:10:44.022 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.019+0000 7fb61b400640 1 -- 172.21.15.22:0/1450212829 learned_addr learned my addr 172.21.15.22:0/1450212829 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-28T18:10:44.022 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.019+0000 7fb61be00640 1 --2- 172.21.15.22:0/1450212829 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb61c1075f0 0x7fb61c19e880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:10:44.022 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.020+0000 7fb61b400640 1 -- 172.21.15.22:0/1450212829 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb61c1075f0 msgr2=0x7fb61c19e880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:10:44.022 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.020+0000 7fb61b400640 1 --2- 172.21.15.22:0/1450212829 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb61c1075f0 0x7fb61c19e880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:10:44.022 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.020+0000 7fb61b400640 1 -- 172.21.15.22:0/1450212829 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fb61c101ef0 msgr2=0x7fb61c19a500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:10:44.022 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.020+0000 7fb61b400640 1 --2- 172.21.15.22:0/1450212829 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fb61c101ef0 0x7fb61c19a500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:10:44.022 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.020+0000 7fb61b400640 1 -- 172.21.15.22:0/1450212829 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb61c19efb0 con 0x7fb61c101530 2024-03-28T18:10:44.023 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.020+0000 7fb61b400640 1 --2- 172.21.15.22:0/1450212829 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb61c101530 0x7fb61c199fc0 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7fb60c00e9c0 tx=0x7fb60c00ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:10:44.023 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.020+0000 7fb613e00640 1 -- 172.21.15.22:0/1450212829 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb60c00ccc0 con 0x7fb61c101530 2024-03-28T18:10:44.023 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.020+0000 7fb621000640 1 -- 172.21.15.22:0/1450212829 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb61c19f2a0 con 0x7fb61c101530 2024-03-28T18:10:44.023 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.020+0000 7fb613e00640 1 -- 172.21.15.22:0/1450212829 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7fb60c0054f0 con 0x7fb61c101530 2024-03-28T18:10:44.023 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.020+0000 7fb621000640 1 -- 172.21.15.22:0/1450212829 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb61c1a6b30 con 0x7fb61c101530 2024-03-28T18:10:44.023 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.020+0000 7fb613e00640 1 -- 172.21.15.22:0/1450212829 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb60c018430 con 0x7fb61c101530 2024-03-28T18:10:44.025 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.023+0000 7fb625376740 1 Processor -- start 2024-03-28T18:10:44.025 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.024+0000 7fb613e00640 1 -- 172.21.15.22:0/1450212829 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 26) v1 ==== 137421+0+0 (secure 0 0 0) 0x7fb60c0046c0 con 0x7fb61c101530 2024-03-28T18:10:44.025 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.024+0000 7fb625376740 1 -- start start 2024-03-28T18:10:44.025 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.024+0000 7fb625376740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f5e593a8b0 0x55f5e593ac90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:10:44.026 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.024+0000 7fb625376740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f5e593b260 0x55f5e5943810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:10:44.026 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.024+0000 7fb625376740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55f5e5943de0 0x55f5e5946230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:10:44.026 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.024+0000 7fb625376740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f5e589e2c0 con 0x55f5e593b260 2024-03-28T18:10:44.026 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.024+0000 7fb625376740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f5e5846c90 con 0x55f5e5943de0 2024-03-28T18:10:44.026 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.024+0000 7fb625376740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f5e589a360 con 0x55f5e593a8b0 2024-03-28T18:10:44.026 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.024+0000 7fb61b400640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f5e593b260 0x55f5e5943810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:10:44.026 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.024+0000 7fb61b400640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f5e593b260 0x55f5e5943810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:34532/0 (socket says 172.21.15.22:34532) 2024-03-28T18:10:44.026 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.024+0000 7fb61b400640 1 -- 172.21.15.22:0/2142521004 learned_addr learned my addr 172.21.15.22:0/2142521004 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-28T18:10:44.026 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.025+0000 7fb61aa00640 1 --2- 172.21.15.22:0/2142521004 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55f5e5943de0 0x55f5e5946230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:10:44.026 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.025+0000 7fb61be00640 1 --2- 172.21.15.22:0/2142521004 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f5e593a8b0 0x55f5e593ac90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:10:44.027 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.025+0000 7fb61b400640 1 -- 172.21.15.22:0/2142521004 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f5e593a8b0 msgr2=0x55f5e593ac90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T18:10:44.027 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.025+0000 7fb61b400640 1 --2- 172.21.15.22:0/2142521004 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f5e593a8b0 0x55f5e593ac90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:10:44.027 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.025+0000 7fb61b400640 1 -- 172.21.15.22:0/2142521004 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55f5e5943de0 msgr2=0x55f5e5946230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T18:10:44.027 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.025+0000 7fb61b400640 1 --2- 172.21.15.22:0/2142521004 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55f5e5943de0 0x55f5e5946230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:10:44.027 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.025+0000 7fb61b400640 1 -- 172.21.15.22:0/2142521004 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f5e586b080 con 0x55f5e593b260 2024-03-28T18:10:44.027 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.025+0000 7fb613e00640 1 --2- 172.21.15.22:0/1450212829 >> [v2:172.21.15.22:6800/1357365082,v1:172.21.15.22:6801/1357365082] conn(0x7fb5fc0a2970 0x7fb5fc0a4e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:10:44.027 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.025+0000 7fb61b400640 1 --2- 172.21.15.22:0/2142521004 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f5e593b260 0x55f5e5943810 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7fb60c0cc9a0 tx=0x7fb60c0d7820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=428bdc3bce21668e server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:10:44.027 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.026+0000 7fb613e00640 1 -- 172.21.15.22:0/1450212829 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 8583+0+0 (secure 0 0 0) 0x7fb60c0cf320 con 0x7fb61c101530 2024-03-28T18:10:44.027 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.026+0000 7fb610c00640 1 -- 172.21.15.22:0/2142521004 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb60c097be0 con 0x55f5e593b260 2024-03-28T18:10:44.027 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.026+0000 7fb61be00640 1 --2- 172.21.15.22:0/1450212829 >> [v2:172.21.15.22:6800/1357365082,v1:172.21.15.22:6801/1357365082] conn(0x7fb5fc0a2970 0x7fb5fc0a4e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:10:44.027 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.026+0000 7fb610c00640 1 -- 172.21.15.22:0/2142521004 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7fb60c09c040 con 0x55f5e593b260 2024-03-28T18:10:44.027 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.026+0000 7fb610c00640 1 -- 172.21.15.22:0/2142521004 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb60c09b2a0 con 0x55f5e593b260 2024-03-28T18:10:44.033 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.026+0000 7fb625376740 1 -- 172.21.15.22:0/2142521004 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f5e593b260 msgr2=0x55f5e5943810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:10:44.034 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.026+0000 7fb61be00640 1 --2- 172.21.15.22:0/1450212829 >> [v2:172.21.15.22:6800/1357365082,v1:172.21.15.22:6801/1357365082] conn(0x7fb5fc0a2970 0x7fb5fc0a4e30 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fb614009cf0 tx=0x7fb614005f90 comp rx=0 tx=0).ready entity=mgr.15522 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:10:44.034 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.026+0000 7fb625376740 1 --2- 172.21.15.22:0/2142521004 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f5e593b260 0x55f5e5943810 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7fb60c0cc9a0 tx=0x7fb60c0d7820 comp rx=0 tx=0).stop 2024-03-28T18:10:44.034 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.027+0000 7fb625376740 1 -- 172.21.15.22:0/2142521004 shutdown_connections 2024-03-28T18:10:44.034 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.027+0000 7fb625376740 1 --2- 172.21.15.22:0/2142521004 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55f5e5943de0 0x55f5e5946230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:10:44.034 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.027+0000 7fb625376740 1 --2- 172.21.15.22:0/2142521004 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f5e593b260 0x55f5e5943810 unknown :-1 s=CLOSED pgs=667 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:10:44.034 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.027+0000 7fb625376740 1 --2- 172.21.15.22:0/2142521004 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f5e593a8b0 0x55f5e593ac90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:10:44.034 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.027+0000 7fb625376740 1 -- 172.21.15.22:0/2142521004 >> 172.21.15.22:0/2142521004 conn(0x55f5e5934f80 msgr2=0x55f5e5935360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:10:44.034 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.027+0000 7fb625376740 1 -- 172.21.15.22:0/2142521004 shutdown_connections 2024-03-28T18:10:44.034 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.027+0000 7fb625376740 1 -- 172.21.15.22:0/2142521004 wait complete. 2024-03-28T18:10:44.034 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.027+0000 7fb625376740 1 -- start start 2024-03-28T18:10:44.035 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.027+0000 7fb625376740 1 Processor -- start 2024-03-28T18:10:44.035 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.028+0000 7fb625376740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f5e593a8b0 0x55f5e59436f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:10:44.035 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.028+0000 7fb625376740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f5e593b260 0x55f5e5940a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:10:44.035 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.028+0000 7fb625376740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55f5e5943de0 0x55f5e5941020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:10:44.035 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.028+0000 7fb625376740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f5e5946d50 con 0x55f5e593a8b0 2024-03-28T18:10:44.035 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.028+0000 7fb625376740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f5e585a0d0 con 0x55f5e5943de0 2024-03-28T18:10:44.035 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.028+0000 7fb625376740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f5e5946ed0 con 0x55f5e593b260 2024-03-28T18:10:44.035 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.028+0000 7fb625376740 10 client.0 _ll_register_callbacks cb 0x55f5e585ee80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-28T18:10:44.035 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.028+0000 7fb61b400640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f5e593a8b0 0x55f5e59436f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:10:44.035 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.028+0000 7fb61b400640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f5e593a8b0 0x55f5e59436f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:34534/0 (socket says 172.21.15.22:34534) 2024-03-28T18:10:44.035 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.028+0000 7fb61b400640 1 -- 172.21.15.22:0/484016041 learned_addr learned my addr 172.21.15.22:0/484016041 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-28T18:10:44.036 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.028+0000 7fb61aa00640 1 --2- 172.21.15.22:0/484016041 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f5e593b260 0x55f5e5940a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:10:44.036 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.028+0000 7fb61be00640 1 --2- 172.21.15.22:0/484016041 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55f5e5943de0 0x55f5e5941020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:10:44.036 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.029+0000 7fb61b400640 1 -- 172.21.15.22:0/484016041 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f5e593b260 msgr2=0x55f5e5940a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T18:10:44.036 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.029+0000 7fb61b400640 1 --2- 172.21.15.22:0/484016041 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f5e593b260 0x55f5e5940a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:10:44.036 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.029+0000 7fb61b400640 1 -- 172.21.15.22:0/484016041 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55f5e5943de0 msgr2=0x55f5e5941020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T18:10:44.036 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.029+0000 7fb61b400640 1 --2- 172.21.15.22:0/484016041 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55f5e5943de0 0x55f5e5941020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:10:44.036 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.029+0000 7fb61b400640 1 -- 172.21.15.22:0/484016041 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55f5e55b23f0 con 0x55f5e593a8b0 2024-03-28T18:10:44.036 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.029+0000 7fb61b400640 1 --2- 172.21.15.22:0/484016041 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f5e593a8b0 0x55f5e59436f0 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7fb60c0ccae0 tx=0x7fb60c0cf620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c8d3977fd3b367d7 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:10:44.036 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.029+0000 7fb609600640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-03-28T18:10:44.037 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.029+0000 7fb609600640 1 -- 172.21.15.22:0/484016041 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb60c09b480 con 0x55f5e593a8b0 2024-03-28T18:10:44.037 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.029+0000 7fb609600640 1 -- 172.21.15.22:0/484016041 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 8583+0+0 (secure 0 0 0) 0x7fb60c09bdb0 con 0x55f5e593a8b0 2024-03-28T18:10:44.037 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.029+0000 7fb625376740 1 -- 172.21.15.22:0/484016041 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fb60c09c040 con 0x55f5e593a8b0 2024-03-28T18:10:44.037 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.030+0000 7fb609600640 1 -- 172.21.15.22:0/484016041 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb60c09b710 con 0x55f5e593a8b0 2024-03-28T18:10:44.037 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.030+0000 7fb625376740 10 client.16398 fetch_fsmap learned FSMap version 40 2024-03-28T18:10:44.037 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.030+0000 7fb625376740 1 -- 172.21.15.22:0/484016041 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) v3 -- 0x55f5e567c7d0 con 0x55f5e593a8b0 2024-03-28T18:10:44.037 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.030+0000 7fb609600640 1 -- 172.21.15.22:0/484016041 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7fb60c09c040 con 0x55f5e593a8b0 2024-03-28T18:10:44.037 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.030+0000 7fb609600640 1 -- 172.21.15.22:0/484016041 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 40) v1 ==== 7954+0+0 (secure 0 0 0) 0x7fb60c0a3c50 con 0x55f5e593a8b0 2024-03-28T18:10:44.037 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.030+0000 7fb625376740 10 client.16398 fetch_fsmap finished waiting for FSMap version 40 2024-03-28T18:10:44.037 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.030+0000 7fb625376740 10 client.16398 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-03-28T18:10:44.037 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.030+0000 7fb625376740 20 client.16398 populate_metadata read hostname 'smithi022' 2024-03-28T18:10:44.037 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.030+0000 7fb625376740 1 --2- 172.21.15.22:0/484016041 >> [v2:172.21.15.79:6836/3437531480,v1:172.21.15.79:6837/3437531480] conn(0x55f5e5953b20 0x55f5e5955f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:10:44.038 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.030+0000 7fb625376740 4 client.16398 mds_command: new command op to 36188 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-28T18:10:44.038 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.030+0000 7fb625376740 1 -- 172.21.15.22:0/484016041 --> [v2:172.21.15.79:6836/3437531480,v1:172.21.15.79:6837/3437531480] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55f5e58ab800 con 0x55f5e5953b20 2024-03-28T18:10:44.038 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.031+0000 7fb61aa00640 1 --2- 172.21.15.22:0/484016041 >> [v2:172.21.15.79:6836/3437531480,v1:172.21.15.79:6837/3437531480] conn(0x55f5e5953b20 0x55f5e5955f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:10:44.038 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.031+0000 7fb61aa00640 1 --2- 172.21.15.22:0/484016041 >> [v2:172.21.15.79:6836/3437531480,v1:172.21.15.79:6837/3437531480] conn(0x55f5e5953b20 0x55f5e5955f10 crc :-1 s=READY pgs=9 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=3b0bcd6c9b88e860 server_cookie=603c9b2b54ed8663 in_seq=0 out_seq=0 2024-03-28T18:10:44.038 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.031+0000 7fb609600640 10 client.16398 ms_handle_connect on v2:172.21.15.79:6836/3437531480 2024-03-28T18:10:44.038 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.033+0000 7fb609600640 1 -- 172.21.15.22:0/484016041 <== mds.1 v2:172.21.15.79:6836/3437531480 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x55f5e58ab800 con 0x55f5e5953b20 2024-03-28T18:10:44.038 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.033+0000 7fb609600640 10 client.16398 handle_command_reply: tid=0 2024-03-28T18:10:44.038 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.033+0000 7fb625376740 1 client.16398 shutdown 2024-03-28T18:10:44.038 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.033+0000 7fb625376740 1 -- 172.21.15.22:0/484016041 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f5e593a8b0 msgr2=0x55f5e59436f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:10:44.038 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.033+0000 7fb625376740 1 --2- 172.21.15.22:0/484016041 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f5e593a8b0 0x55f5e59436f0 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7fb60c0ccae0 tx=0x7fb60c0cf620 comp rx=0 tx=0).stop 2024-03-28T18:10:44.039 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.033+0000 7fb625376740 1 -- 172.21.15.22:0/484016041 shutdown_connections 2024-03-28T18:10:44.039 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.033+0000 7fb625376740 1 --2- 172.21.15.22:0/484016041 >> [v2:172.21.15.79:6836/3437531480,v1:172.21.15.79:6837/3437531480] conn(0x55f5e5953b20 0x55f5e5955f10 crc :-1 s=READY pgs=9 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:10:44.039 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.033+0000 7fb625376740 1 --2- 172.21.15.22:0/484016041 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55f5e5943de0 0x55f5e5941020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:10:44.039 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.033+0000 7fb625376740 1 --2- 172.21.15.22:0/484016041 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f5e593b260 0x55f5e5940a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:10:44.039 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.033+0000 7fb609600640 0 client.16398 ms_handle_reset on v2:172.21.15.79:6836/3437531480 2024-03-28T18:10:44.039 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.033+0000 7fb625376740 1 --2- 172.21.15.22:0/484016041 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f5e593a8b0 0x55f5e59436f0 unknown :-1 s=CLOSED pgs=668 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:10:44.039 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.033+0000 7fb625376740 1 -- 172.21.15.22:0/484016041 >> 172.21.15.22:0/484016041 conn(0x55f5e5934f80 msgr2=0x55f5e5936cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:10:44.039 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.033+0000 7fb625376740 1 -- 172.21.15.22:0/484016041 shutdown_connections 2024-03-28T18:10:44.039 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.033+0000 7fb625376740 1 -- 172.21.15.22:0/484016041 wait complete. 2024-03-28T18:10:44.039 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.033+0000 7fb625376740 20 client.16398 trim_cache size 0 max 16384 2024-03-28T18:10:44.049 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.047+0000 7fb625376740 1 Processor -- start 2024-03-28T18:10:44.050 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.047+0000 7fb625376740 1 -- start start 2024-03-28T18:10:44.050 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.047+0000 7fb625376740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f5e593a8b0 0x55f5e59431b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:10:44.050 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.047+0000 7fb625376740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55f5e593b260 0x55f5e59436f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:10:44.050 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.047+0000 7fb625376740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f5e5943de0 0x55f5e593dd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:10:44.050 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.047+0000 7fb625376740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb61c1af8d0 con 0x55f5e593a8b0 2024-03-28T18:10:44.050 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.047+0000 7fb625376740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb61c1b0cb0 con 0x55f5e593b260 2024-03-28T18:10:44.050 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.047+0000 7fb625376740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb61c1aff30 con 0x55f5e5943de0 2024-03-28T18:10:44.051 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.048+0000 7fb61b400640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f5e593a8b0 0x55f5e59431b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:10:44.051 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.048+0000 7fb61b400640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f5e593a8b0 0x55f5e59431b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:34536/0 (socket says 172.21.15.22:34536) 2024-03-28T18:10:44.051 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.048+0000 7fb61b400640 1 -- 172.21.15.22:0/2830667260 learned_addr learned my addr 172.21.15.22:0/2830667260 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-28T18:10:44.051 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.048+0000 7fb61aa00640 1 --2- 172.21.15.22:0/2830667260 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55f5e593b260 0x55f5e59436f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:10:44.051 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.048+0000 7fb61be00640 1 --2- 172.21.15.22:0/2830667260 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f5e5943de0 0x55f5e593dd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:10:44.051 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.048+0000 7fb61b400640 1 -- 172.21.15.22:0/2830667260 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f5e5943de0 msgr2=0x55f5e593dd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T18:10:44.051 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.048+0000 7fb61b400640 1 --2- 172.21.15.22:0/2830667260 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f5e5943de0 0x55f5e593dd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:10:44.051 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.048+0000 7fb61b400640 1 -- 172.21.15.22:0/2830667260 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55f5e593b260 msgr2=0x55f5e59436f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T18:10:44.051 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.048+0000 7fb61b400640 1 --2- 172.21.15.22:0/2830667260 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55f5e593b260 0x55f5e59436f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:10:44.051 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.048+0000 7fb61b400640 1 -- 172.21.15.22:0/2830667260 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f5e593a350 con 0x55f5e593a8b0 2024-03-28T18:10:44.051 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.048+0000 7fb61be00640 1 --2- 172.21.15.22:0/2830667260 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f5e5943de0 0x55f5e593dd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T18:10:44.052 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.048+0000 7fb61b400640 1 --2- 172.21.15.22:0/2830667260 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f5e593a8b0 0x55f5e59431b0 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7fb60c098310 tx=0x7fb60c0cf690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8b693ed22793f93f server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:10:44.052 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.048+0000 7fb60be00640 1 -- 172.21.15.22:0/2830667260 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb60c09b480 con 0x55f5e593a8b0 2024-03-28T18:10:44.052 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.049+0000 7fb60be00640 1 -- 172.21.15.22:0/2830667260 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7fb60c0ce070 con 0x55f5e593a8b0 2024-03-28T18:10:44.052 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.049+0000 7fb60be00640 1 -- 172.21.15.22:0/2830667260 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb60c09b2a0 con 0x55f5e593a8b0 2024-03-28T18:10:44.052 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.049+0000 7fb625376740 1 -- 172.21.15.22:0/2830667260 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f5e593a8b0 msgr2=0x55f5e59431b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:10:44.052 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.049+0000 7fb625376740 1 --2- 172.21.15.22:0/2830667260 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f5e593a8b0 0x55f5e59431b0 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7fb60c098310 tx=0x7fb60c0cf690 comp rx=0 tx=0).stop 2024-03-28T18:10:44.052 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.049+0000 7fb625376740 1 -- 172.21.15.22:0/2830667260 shutdown_connections 2024-03-28T18:10:44.052 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.049+0000 7fb625376740 1 --2- 172.21.15.22:0/2830667260 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f5e5943de0 0x55f5e593dd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:10:44.052 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.049+0000 7fb625376740 1 --2- 172.21.15.22:0/2830667260 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55f5e593b260 0x55f5e59436f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:10:44.052 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.049+0000 7fb625376740 1 --2- 172.21.15.22:0/2830667260 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f5e593a8b0 0x55f5e59431b0 unknown :-1 s=CLOSED pgs=669 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:10:44.052 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.049+0000 7fb625376740 1 -- 172.21.15.22:0/2830667260 >> 172.21.15.22:0/2830667260 conn(0x55f5e5934f80 msgr2=0x55f5e5936c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:10:44.052 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.049+0000 7fb625376740 1 -- 172.21.15.22:0/2830667260 shutdown_connections 2024-03-28T18:10:44.053 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.049+0000 7fb625376740 1 -- 172.21.15.22:0/2830667260 wait complete. 2024-03-28T18:10:44.053 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.050+0000 7fb625376740 1 -- start start 2024-03-28T18:10:44.053 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.050+0000 7fb625376740 1 Processor -- start 2024-03-28T18:10:44.057 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.050+0000 7fb625376740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f5e593a8b0 0x55f5e5957300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:10:44.057 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.050+0000 7fb625376740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f5e593b260 0x55f5e5957840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:10:44.057 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.050+0000 7fb625376740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55f5e5943de0 0x55f5e5957d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:10:44.058 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.050+0000 7fb625376740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb61c1ae050 con 0x55f5e593b260 2024-03-28T18:10:44.058 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.050+0000 7fb625376740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb61c1b0650 con 0x55f5e5943de0 2024-03-28T18:10:44.058 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.050+0000 7fb625376740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb61c1ad410 con 0x55f5e593a8b0 2024-03-28T18:10:44.058 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.050+0000 7fb625376740 10 client.0 _ll_register_callbacks cb 0x7fb60c09f070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-28T18:10:44.058 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.051+0000 7fb61aa00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f5e593b260 0x55f5e5957840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:10:44.058 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.051+0000 7fb61aa00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f5e593b260 0x55f5e5957840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:34552/0 (socket says 172.21.15.22:34552) 2024-03-28T18:10:44.058 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.051+0000 7fb61aa00640 1 -- 172.21.15.22:0/2092485126 learned_addr learned my addr 172.21.15.22:0/2092485126 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-28T18:10:44.058 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.051+0000 7fb61be00640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55f5e5943de0 0x55f5e5957d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:10:44.058 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.051+0000 7fb61b400640 1 --2- 172.21.15.22:0/2092485126 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f5e593a8b0 0x55f5e5957300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:10:44.058 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.051+0000 7fb61aa00640 1 -- 172.21.15.22:0/2092485126 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f5e593a8b0 msgr2=0x55f5e5957300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T18:10:44.059 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.051+0000 7fb61aa00640 1 --2- 172.21.15.22:0/2092485126 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f5e593a8b0 0x55f5e5957300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:10:44.059 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.051+0000 7fb61aa00640 1 -- 172.21.15.22:0/2092485126 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55f5e5943de0 msgr2=0x55f5e5957d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T18:10:44.059 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.051+0000 7fb61aa00640 1 --2- 172.21.15.22:0/2092485126 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55f5e5943de0 0x55f5e5957d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:10:44.059 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.051+0000 7fb61aa00640 1 -- 172.21.15.22:0/2092485126 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55f5e59306c0 con 0x55f5e593b260 2024-03-28T18:10:44.059 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.051+0000 7fb61aa00640 1 --2- 172.21.15.22:0/2092485126 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f5e593b260 0x55f5e5957840 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7fb604006010 tx=0x7fb6040334d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ba5411ffe1860051 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:10:44.059 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.051+0000 7fb609600640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-03-28T18:10:44.059 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.052+0000 7fb609600640 1 -- 172.21.15.22:0/2092485126 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb60404d070 con 0x55f5e593b260 2024-03-28T18:10:44.059 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.052+0000 7fb609600640 1 -- 172.21.15.22:0/2092485126 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 8583+0+0 (secure 0 0 0) 0x7fb6040337d0 con 0x55f5e593b260 2024-03-28T18:10:44.059 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.052+0000 7fb625376740 1 -- 172.21.15.22:0/2092485126 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fb60c0ce070 con 0x55f5e593b260 2024-03-28T18:10:44.059 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.052+0000 7fb609600640 1 -- 172.21.15.22:0/2092485126 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb604059050 con 0x55f5e593b260 2024-03-28T18:10:44.060 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.052+0000 7fb625376740 10 client.16404 fetch_fsmap learned FSMap version 40 2024-03-28T18:10:44.060 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.052+0000 7fb625376740 1 -- 172.21.15.22:0/2092485126 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) v3 -- 0x55f5e59b0190 con 0x55f5e593b260 2024-03-28T18:10:44.060 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.052+0000 7fb609600640 1 -- 172.21.15.22:0/2092485126 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7fb60c0ce070 con 0x55f5e593b260 2024-03-28T18:10:44.060 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.052+0000 7fb609600640 1 -- 172.21.15.22:0/2092485126 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 40) v1 ==== 7954+0+0 (secure 0 0 0) 0x7fb604004050 con 0x55f5e593b260 2024-03-28T18:10:44.060 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.052+0000 7fb625376740 10 client.16404 fetch_fsmap finished waiting for FSMap version 40 2024-03-28T18:10:44.060 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.053+0000 7fb625376740 10 client.16404 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-03-28T18:10:44.060 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.053+0000 7fb625376740 20 client.16404 populate_metadata read hostname 'smithi022' 2024-03-28T18:10:44.061 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.053+0000 7fb625376740 1 --2- 172.21.15.22:0/2092485126 >> [v2:172.21.15.79:6836/3437531480,v1:172.21.15.79:6837/3437531480] conn(0x55f5e59b08a0 0x55f5e59b2cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:10:44.061 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.053+0000 7fb625376740 4 client.16404 mds_command: new command op to 36188 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-28T18:10:44.061 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.053+0000 7fb625376740 1 -- 172.21.15.22:0/2092485126 --> [v2:172.21.15.79:6836/3437531480,v1:172.21.15.79:6837/3437531480] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55f5e58a5b50 con 0x55f5e59b08a0 2024-03-28T18:10:44.061 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.053+0000 7fb61b400640 1 --2- 172.21.15.22:0/2092485126 >> [v2:172.21.15.79:6836/3437531480,v1:172.21.15.79:6837/3437531480] conn(0x55f5e59b08a0 0x55f5e59b2cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:10:44.061 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.053+0000 7fb61b400640 1 --2- 172.21.15.22:0/2092485126 >> [v2:172.21.15.79:6836/3437531480,v1:172.21.15.79:6837/3437531480] conn(0x55f5e59b08a0 0x55f5e59b2cc0 crc :-1 s=READY pgs=10 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=7b4262bf19ba5089 server_cookie=d1e02887df8cb715 in_seq=0 out_seq=0 2024-03-28T18:10:44.061 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.054+0000 7fb609600640 10 client.16404 ms_handle_connect on v2:172.21.15.79:6836/3437531480 2024-03-28T18:10:44.061 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.054+0000 7fb609600640 1 -- 172.21.15.22:0/2092485126 <== mds.1 v2:172.21.15.79:6836/3437531480 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55f5e58a5b50 con 0x55f5e59b08a0 2024-03-28T18:10:44.061 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.054+0000 7fb609600640 10 client.16404 handle_command_reply: tid=0 2024-03-28T18:10:44.061 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.054+0000 7fb625376740 1 client.16404 shutdown 2024-03-28T18:10:44.061 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.054+0000 7fb625376740 1 -- 172.21.15.22:0/2092485126 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f5e593b260 msgr2=0x55f5e5957840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:10:44.061 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.054+0000 7fb625376740 1 --2- 172.21.15.22:0/2092485126 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f5e593b260 0x55f5e5957840 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7fb604006010 tx=0x7fb6040334d0 comp rx=0 tx=0).stop 2024-03-28T18:10:44.061 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.054+0000 7fb625376740 1 -- 172.21.15.22:0/2092485126 shutdown_connections 2024-03-28T18:10:44.061 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.054+0000 7fb625376740 1 --2- 172.21.15.22:0/2092485126 >> [v2:172.21.15.79:6836/3437531480,v1:172.21.15.79:6837/3437531480] conn(0x55f5e59b08a0 0x55f5e59b2cc0 crc :-1 s=READY pgs=10 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:10:44.062 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.054+0000 7fb625376740 1 --2- 172.21.15.22:0/2092485126 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55f5e5943de0 0x55f5e5957d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:10:44.062 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.054+0000 7fb625376740 1 --2- 172.21.15.22:0/2092485126 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f5e593b260 0x55f5e5957840 unknown :-1 s=CLOSED pgs=670 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:10:44.062 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.054+0000 7fb625376740 1 --2- 172.21.15.22:0/2092485126 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f5e593a8b0 0x55f5e5957300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:10:44.062 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.054+0000 7fb625376740 1 -- 172.21.15.22:0/2092485126 >> 172.21.15.22:0/2092485126 conn(0x55f5e5934f80 msgr2=0x55f5e59338d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:10:44.062 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.054+0000 7fb609600640 0 client.16404 ms_handle_reset on v2:172.21.15.79:6836/3437531480 2024-03-28T18:10:44.062 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.054+0000 7fb625376740 1 -- 172.21.15.22:0/2092485126 shutdown_connections 2024-03-28T18:10:44.062 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.054+0000 7fb625376740 1 -- 172.21.15.22:0/2092485126 wait complete. 2024-03-28T18:10:44.062 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.054+0000 7fb625376740 20 client.16404 trim_cache size 0 max 16384 2024-03-28T18:10:44.062 INFO:teuthology.orchestra.run.smithi022.stdout:[] 2024-03-28T18:10:44.063 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.055+0000 7fb621000640 1 -- 172.21.15.22:0/1450212829 >> [v2:172.21.15.22:6800/1357365082,v1:172.21.15.22:6801/1357365082] conn(0x7fb5fc0a2970 msgr2=0x7fb5fc0a4e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:10:44.063 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.055+0000 7fb621000640 1 --2- 172.21.15.22:0/1450212829 >> [v2:172.21.15.22:6800/1357365082,v1:172.21.15.22:6801/1357365082] conn(0x7fb5fc0a2970 0x7fb5fc0a4e30 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fb614009cf0 tx=0x7fb614005f90 comp rx=0 tx=0).stop 2024-03-28T18:10:44.063 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.055+0000 7fb621000640 1 -- 172.21.15.22:0/1450212829 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb61c101530 msgr2=0x7fb61c199fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:10:44.063 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.055+0000 7fb621000640 1 --2- 172.21.15.22:0/1450212829 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb61c101530 0x7fb61c199fc0 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7fb60c00e9c0 tx=0x7fb60c00ee80 comp rx=0 tx=0).stop 2024-03-28T18:10:44.063 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.055+0000 7fb621000640 1 -- 172.21.15.22:0/1450212829 shutdown_connections 2024-03-28T18:10:44.063 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.055+0000 7fb621000640 1 --2- 172.21.15.22:0/1450212829 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb61c1075f0 0x7fb61c19e880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:10:44.063 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.055+0000 7fb621000640 1 --2- 172.21.15.22:0/1450212829 >> [v2:172.21.15.22:6800/1357365082,v1:172.21.15.22:6801/1357365082] conn(0x7fb5fc0a2970 0x7fb5fc0a4e30 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:10:44.064 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.055+0000 7fb621000640 1 --2- 172.21.15.22:0/1450212829 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fb61c101ef0 0x7fb61c19a500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:10:44.064 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.055+0000 7fb621000640 1 --2- 172.21.15.22:0/1450212829 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb61c101530 0x7fb61c199fc0 unknown :-1 s=CLOSED pgs=666 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:10:44.064 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.055+0000 7fb621000640 1 -- 172.21.15.22:0/1450212829 >> 172.21.15.22:0/1450212829 conn(0x7fb61c0fd000 msgr2=0x7fb61c0fe5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:10:44.064 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.055+0000 7fb621000640 1 -- 172.21.15.22:0/1450212829 shutdown_connections 2024-03-28T18:10:44.064 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.055+0000 7fb621000640 1 -- 172.21.15.22:0/1450212829 wait complete. 2024-03-28T18:10:44.072 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-28T18:10:44.072 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2024-03-28T18:10:44.175 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:10:43 smithi134 podman[69755]: 2024-03-28 18:10:43.883488446 +0000 UTC m=+0.405008819 container init d152d7307b05bc350e7621116317bf1237be480aada83ebe56fc7359227b8025 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f, ceph=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240319, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-28T18:10:44.175 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:10:43 smithi134 podman[69755]: 2024-03-28 18:10:43.894180659 +0000 UTC m=+0.415701032 container start d152d7307b05bc350e7621116317bf1237be480aada83ebe56fc7359227b8025 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f, org.label-schema.license=GPLv2, GIT_CLEAN=True, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, ceph=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.build-date=20240319, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS) 2024-03-28T18:10:44.175 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:10:43 smithi134 ceph-mds[69773]: -- 172.21.15.134:0/911618062 <== mon.1 v2:172.21.15.134:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55f068a1fa40 con 0x55f0689f2800 2024-03-28T18:10:44.175 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:10:43 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[69769]: starting mds.f at 2024-03-28T18:10:44.175 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:10:44 smithi134 bash[69755]: d152d7307b05bc350e7621116317bf1237be480aada83ebe56fc7359227b8025 2024-03-28T18:10:44.175 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:10:44 smithi134 systemd[1]: Started Ceph mds.f for fd220748-ed2b-11ee-95d0-87774f69a715. 2024-03-28T18:10:44.302 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.302+0000 7f09a3e00640 1 -- 172.21.15.22:0/3210481649 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f09a4109730 msgr2=0x7f09a4109b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:10:44.302 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.302+0000 7f09a3e00640 1 --2- 172.21.15.22:0/3210481649 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f09a4109730 0x7f09a4109b10 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7f0994009cd0 tx=0x7f099402c0f0 comp rx=0 tx=0).stop 2024-03-28T18:10:44.302 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.302+0000 7f09a3e00640 1 -- 172.21.15.22:0/3210481649 shutdown_connections 2024-03-28T18:10:44.302 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.302+0000 7f09a3e00640 1 --2- 172.21.15.22:0/3210481649 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f09a41040e0 0x7f09a41105f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:10:44.302 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.302+0000 7f09a3e00640 1 --2- 172.21.15.22:0/3210481649 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f09a4103720 0x7f09a4103ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:10:44.303 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.302+0000 7f09a3e00640 1 --2- 172.21.15.22:0/3210481649 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f09a4109730 0x7f09a4109b10 unknown :-1 s=CLOSED pgs=671 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:10:44.303 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.302+0000 7f09a3e00640 1 -- 172.21.15.22:0/3210481649 >> 172.21.15.22:0/3210481649 conn(0x7f09a40ff1d0 msgr2=0x7f09a41015f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:10:44.303 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.302+0000 7f09a3e00640 1 -- 172.21.15.22:0/3210481649 shutdown_connections 2024-03-28T18:10:44.303 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.302+0000 7f09a3e00640 1 -- 172.21.15.22:0/3210481649 wait complete. 2024-03-28T18:10:44.304 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.303+0000 7f09a3e00640 1 Processor -- start 2024-03-28T18:10:44.305 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.303+0000 7f09a3e00640 1 -- start start 2024-03-28T18:10:44.305 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.304+0000 7f09a3e00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f09a4103720 0x7f09a419a070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:10:44.305 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.304+0000 7f09a3e00640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f09a41040e0 0x7f09a419a5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:10:44.305 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.304+0000 7f09a3e00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f09a4109730 0x7f09a419e930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:10:44.305 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.304+0000 7f09a3e00640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09a4113b10 con 0x7f09a4103720 2024-03-28T18:10:44.305 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.304+0000 7f09a3e00640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09a4113990 con 0x7f09a41040e0 2024-03-28T18:10:44.305 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.304+0000 7f09a3e00640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09a4113c90 con 0x7f09a4109730 2024-03-28T18:10:44.305 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.304+0000 7f09a2a00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f09a4103720 0x7f09a419a070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:10:44.305 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.304+0000 7f09a2a00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f09a4103720 0x7f09a419a070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:34566/0 (socket says 172.21.15.22:34566) 2024-03-28T18:10:44.305 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.304+0000 7f09a2a00640 1 -- 172.21.15.22:0/2610077547 learned_addr learned my addr 172.21.15.22:0/2610077547 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-28T18:10:44.305 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.304+0000 7f09a3400640 1 --2- 172.21.15.22:0/2610077547 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f09a4109730 0x7f09a419e930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:10:44.305 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.305+0000 7f09a2000640 1 --2- 172.21.15.22:0/2610077547 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f09a41040e0 0x7f09a419a5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:10:44.305 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.305+0000 7f09a2a00640 1 -- 172.21.15.22:0/2610077547 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f09a4109730 msgr2=0x7f09a419e930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:10:44.305 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.305+0000 7f09a2a00640 1 --2- 172.21.15.22:0/2610077547 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f09a4109730 0x7f09a419e930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:10:44.306 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.305+0000 7f09a2a00640 1 -- 172.21.15.22:0/2610077547 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f09a41040e0 msgr2=0x7f09a419a5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:10:44.306 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.305+0000 7f09a2a00640 1 --2- 172.21.15.22:0/2610077547 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f09a41040e0 0x7f09a419a5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:10:44.306 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.305+0000 7f09a2a00640 1 -- 172.21.15.22:0/2610077547 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f09a419f060 con 0x7f09a4103720 2024-03-28T18:10:44.306 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.305+0000 7f09a2000640 1 --2- 172.21.15.22:0/2610077547 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f09a41040e0 0x7f09a419a5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T18:10:44.306 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.305+0000 7f09a2a00640 1 --2- 172.21.15.22:0/2610077547 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f09a4103720 0x7f09a419a070 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7f0994000f80 tx=0x7f0994034cf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:10:44.306 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.306+0000 7f099b400640 1 -- 172.21.15.22:0/2610077547 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0994039050 con 0x7f09a4103720 2024-03-28T18:10:44.307 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.306+0000 7f099b400640 1 -- 172.21.15.22:0/2610077547 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f099403a040 con 0x7f09a4103720 2024-03-28T18:10:44.307 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.306+0000 7f09a3e00640 1 -- 172.21.15.22:0/2610077547 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f09a419f2f0 con 0x7f09a4103720 2024-03-28T18:10:44.307 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.306+0000 7f099b400640 1 -- 172.21.15.22:0/2610077547 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0994004760 con 0x7f09a4103720 2024-03-28T18:10:44.307 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.306+0000 7f09a3e00640 1 -- 172.21.15.22:0/2610077547 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f09a41a6be0 con 0x7f09a4103720 2024-03-28T18:10:44.309 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.310+0000 7f099b400640 1 -- 172.21.15.22:0/2610077547 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 26) v1 ==== 137421+0+0 (secure 0 0 0) 0x7f0994003970 con 0x7f09a4103720 2024-03-28T18:10:44.309 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.310+0000 7f09ac594740 1 Processor -- start 2024-03-28T18:10:44.310 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.310+0000 7f09ac594740 1 -- start start 2024-03-28T18:10:44.312 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.310+0000 7f09ac594740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560951b178b0 0x560951b17c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:10:44.312 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.311+0000 7f09ac594740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x560951b18260 0x560951b20810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:10:44.312 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.311+0000 7f09ac594740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560951b20de0 0x560951b23230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:10:44.312 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.311+0000 7f09ac594740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x560951a7b2c0 con 0x560951b178b0 2024-03-28T18:10:44.312 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.311+0000 7f09ac594740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x560951a23c90 con 0x560951b18260 2024-03-28T18:10:44.312 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.311+0000 7f09ac594740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x560951a77360 con 0x560951b20de0 2024-03-28T18:10:44.312 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.311+0000 7f09a3400640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560951b178b0 0x560951b17c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:10:44.313 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.311+0000 7f09a3400640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560951b178b0 0x560951b17c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:34570/0 (socket says 172.21.15.22:34570) 2024-03-28T18:10:44.313 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.311+0000 7f09a3400640 1 -- 172.21.15.22:0/1024087035 learned_addr learned my addr 172.21.15.22:0/1024087035 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-28T18:10:44.313 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.311+0000 7f09a2a00640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x560951b18260 0x560951b20810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:10:44.313 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.311+0000 7f09a2000640 1 --2- 172.21.15.22:0/1024087035 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560951b20de0 0x560951b23230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:10:44.313 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.311+0000 7f09a3400640 1 -- 172.21.15.22:0/1024087035 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560951b20de0 msgr2=0x560951b23230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T18:10:44.313 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.311+0000 7f09a3400640 1 --2- 172.21.15.22:0/1024087035 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560951b20de0 0x560951b23230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:10:44.313 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.311+0000 7f09a3400640 1 -- 172.21.15.22:0/1024087035 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x560951b18260 msgr2=0x560951b20810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T18:10:44.313 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.311+0000 7f09a3400640 1 --2- 172.21.15.22:0/1024087035 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x560951b18260 0x560951b20810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:10:44.313 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.311+0000 7f09a3400640 1 -- 172.21.15.22:0/1024087035 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560951a48080 con 0x560951b178b0 2024-03-28T18:10:44.313 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.312+0000 7f09a2000640 1 --2- 172.21.15.22:0/1024087035 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560951b20de0 0x560951b23230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-28T18:10:44.313 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.312+0000 7f099b400640 1 --2- 172.21.15.22:0/2610077547 >> [v2:172.21.15.22:6800/1357365082,v1:172.21.15.22:6801/1357365082] conn(0x7f09880a29f0 0x7f09880a4eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:10:44.313 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.312+0000 7f09a3400640 1 --2- 172.21.15.22:0/1024087035 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560951b178b0 0x560951b17c90 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7f099c00bba0 tx=0x7f099c00f730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e571700f5f25ced2 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:10:44.313 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.312+0000 7f099b400640 1 -- 172.21.15.22:0/2610077547 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 8583+0+0 (secure 0 0 0) 0x7f09940f1590 con 0x7f09a4103720 2024-03-28T18:10:44.313 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.312+0000 7f09a2a00640 1 --2- 172.21.15.22:0/2610077547 >> [v2:172.21.15.22:6800/1357365082,v1:172.21.15.22:6801/1357365082] conn(0x7f09880a29f0 0x7f09880a4eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:10:44.314 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.312+0000 7f098fe00640 1 -- 172.21.15.22:0/1024087035 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f099c015070 con 0x560951b178b0 2024-03-28T18:10:44.314 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.312+0000 7f098fe00640 1 -- 172.21.15.22:0/1024087035 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f099c0054f0 con 0x560951b178b0 2024-03-28T18:10:44.314 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.312+0000 7f098fe00640 1 -- 172.21.15.22:0/1024087035 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f099c011640 con 0x560951b178b0 2024-03-28T18:10:44.317 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.313+0000 7f09ac594740 1 -- 172.21.15.22:0/1024087035 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560951b178b0 msgr2=0x560951b17c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:10:44.318 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.313+0000 7f09ac594740 1 --2- 172.21.15.22:0/1024087035 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560951b178b0 0x560951b17c90 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7f099c00bba0 tx=0x7f099c00f730 comp rx=0 tx=0).stop 2024-03-28T18:10:44.318 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.313+0000 7f09a2a00640 1 --2- 172.21.15.22:0/2610077547 >> [v2:172.21.15.22:6800/1357365082,v1:172.21.15.22:6801/1357365082] conn(0x7f09880a29f0 0x7f09880a4eb0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x560951b17380 tx=0x7f09940f70e0 comp rx=0 tx=0).ready entity=mgr.15522 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:10:44.318 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.313+0000 7f09ac594740 1 -- 172.21.15.22:0/1024087035 shutdown_connections 2024-03-28T18:10:44.318 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.313+0000 7f09ac594740 1 --2- 172.21.15.22:0/1024087035 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560951b20de0 0x560951b23230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:10:44.318 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.313+0000 7f09ac594740 1 --2- 172.21.15.22:0/1024087035 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x560951b18260 0x560951b20810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:10:44.318 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.313+0000 7f09ac594740 1 --2- 172.21.15.22:0/1024087035 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560951b178b0 0x560951b17c90 unknown :-1 s=CLOSED pgs=673 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:10:44.318 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.313+0000 7f09ac594740 1 -- 172.21.15.22:0/1024087035 >> 172.21.15.22:0/1024087035 conn(0x560951b11f80 msgr2=0x560951b12360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:10:44.318 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.313+0000 7f09ac594740 1 -- 172.21.15.22:0/1024087035 shutdown_connections 2024-03-28T18:10:44.318 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.313+0000 7f09ac594740 1 -- 172.21.15.22:0/1024087035 wait complete. 2024-03-28T18:10:44.318 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.314+0000 7f09ac594740 1 -- start start 2024-03-28T18:10:44.318 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.314+0000 7f09ac594740 1 Processor -- start 2024-03-28T18:10:44.318 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.314+0000 7f09ac594740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560951b178b0 0x560951b206c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:10:44.318 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.314+0000 7f09ac594740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560951b18260 0x560951b1da50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:10:44.318 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.315+0000 7f09ac594740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x560951b20de0 0x560951b1dfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:10:44.319 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.315+0000 7f09ac594740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x560951b23d50 con 0x560951b178b0 2024-03-28T18:10:44.319 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.315+0000 7f09ac594740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x560951a370d0 con 0x560951b20de0 2024-03-28T18:10:44.319 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.315+0000 7f09ac594740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x560951b23ed0 con 0x560951b18260 2024-03-28T18:10:44.319 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.315+0000 7f09ac594740 10 client.0 _ll_register_callbacks cb 0x560951a3be80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-28T18:10:44.319 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.315+0000 7f09a3400640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560951b178b0 0x560951b206c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:10:44.319 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.315+0000 7f09a3400640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560951b178b0 0x560951b206c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:34584/0 (socket says 172.21.15.22:34584) 2024-03-28T18:10:44.319 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.315+0000 7f09a3400640 1 -- 172.21.15.22:0/2655327831 learned_addr learned my addr 172.21.15.22:0/2655327831 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-28T18:10:44.319 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.315+0000 7f09a2000640 1 --2- 172.21.15.22:0/2655327831 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560951b18260 0x560951b1da50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:10:44.319 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.315+0000 7f09a3400640 1 --2- 172.21.15.22:0/2655327831 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x560951b20de0 0x560951b1dfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:10:44.319 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.315+0000 7f09a3400640 1 -- 172.21.15.22:0/2655327831 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560951b18260 msgr2=0x560951b1da50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T18:10:44.319 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.315+0000 7f09a3400640 1 --2- 172.21.15.22:0/2655327831 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560951b18260 0x560951b1da50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:10:44.319 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.315+0000 7f09a3400640 1 -- 172.21.15.22:0/2655327831 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x560951b20de0 msgr2=0x560951b1dfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T18:10:44.319 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.315+0000 7f09a3400640 1 --2- 172.21.15.22:0/2655327831 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x560951b20de0 0x560951b1dfb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:10:44.319 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.315+0000 7f09a3400640 1 -- 172.21.15.22:0/2655327831 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56095178f3f0 con 0x560951b178b0 2024-03-28T18:10:44.319 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.316+0000 7f09a3400640 1 --2- 172.21.15.22:0/2655327831 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560951b178b0 0x560951b206c0 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7f099c000c00 tx=0x7f099c00b2e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=33f67f6b28754e87 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:10:44.320 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.316+0000 7f098cc00640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-03-28T18:10:44.320 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.316+0000 7f098cc00640 1 -- 172.21.15.22:0/2655327831 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f099c015070 con 0x560951b178b0 2024-03-28T18:10:44.320 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.316+0000 7f098cc00640 1 -- 172.21.15.22:0/2655327831 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 8583+0+0 (secure 0 0 0) 0x7f099c005000 con 0x560951b178b0 2024-03-28T18:10:44.320 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.316+0000 7f09ac594740 1 -- 172.21.15.22:0/2655327831 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f099c0054f0 con 0x560951b178b0 2024-03-28T18:10:44.320 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.316+0000 7f098cc00640 1 -- 172.21.15.22:0/2655327831 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7f099c00be90 con 0x560951b178b0 2024-03-28T18:10:44.320 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.316+0000 7f09ac594740 10 client.16416 fetch_fsmap learned FSMap version 40 2024-03-28T18:10:44.320 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.316+0000 7f09ac594740 1 -- 172.21.15.22:0/2655327831 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) v3 -- 0x5609518597d0 con 0x560951b178b0 2024-03-28T18:10:44.320 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.317+0000 7f098cc00640 1 -- 172.21.15.22:0/2655327831 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f099c0054f0 con 0x560951b178b0 2024-03-28T18:10:44.320 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.317+0000 7f098cc00640 1 -- 172.21.15.22:0/2655327831 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 40) v1 ==== 7954+0+0 (secure 0 0 0) 0x7f099c019410 con 0x560951b178b0 2024-03-28T18:10:44.320 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.317+0000 7f09ac594740 10 client.16416 fetch_fsmap finished waiting for FSMap version 40 2024-03-28T18:10:44.320 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.317+0000 7f09ac594740 10 client.16416 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.i 2024-03-28T18:10:44.320 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.317+0000 7f09ac594740 20 client.16416 populate_metadata read hostname 'smithi022' 2024-03-28T18:10:44.320 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.317+0000 7f09ac594740 1 --2- 172.21.15.22:0/2655327831 >> [v2:172.21.15.134:6838/3400263217,v1:172.21.15.134:6839/3400263217] conn(0x560951b30ab0 0x560951b32ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:10:44.320 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.317+0000 7f09ac594740 4 client.16416 mds_command: new command op to 24502 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-28T18:10:44.320 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.317+0000 7f09ac594740 1 -- 172.21.15.22:0/2655327831 --> [v2:172.21.15.134:6838/3400263217,v1:172.21.15.134:6839/3400263217] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x560951a88800 con 0x560951b30ab0 2024-03-28T18:10:44.321 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.317+0000 7f09a2000640 1 --2- 172.21.15.22:0/2655327831 >> [v2:172.21.15.134:6838/3400263217,v1:172.21.15.134:6839/3400263217] conn(0x560951b30ab0 0x560951b32ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:10:44.322 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.318+0000 7f09a2000640 1 --2- 172.21.15.22:0/2655327831 >> [v2:172.21.15.134:6838/3400263217,v1:172.21.15.134:6839/3400263217] conn(0x560951b30ab0 0x560951b32ea0 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=8be4c227ff7cf29a server_cookie=59de4f703ceaeacd in_seq=0 out_seq=0 2024-03-28T18:10:44.322 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.318+0000 7f098cc00640 10 client.16416 ms_handle_connect on v2:172.21.15.134:6838/3400263217 2024-03-28T18:10:44.322 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.319+0000 7f098cc00640 1 -- 172.21.15.22:0/2655327831 <== mds.4 v2:172.21.15.134:6838/3400263217 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x560951a88800 con 0x560951b30ab0 2024-03-28T18:10:44.322 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.319+0000 7f098cc00640 10 client.16416 handle_command_reply: tid=0 2024-03-28T18:10:44.322 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.320+0000 7f09ac594740 1 client.16416 shutdown 2024-03-28T18:10:44.322 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.320+0000 7f09ac594740 1 -- 172.21.15.22:0/2655327831 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560951b178b0 msgr2=0x560951b206c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:10:44.322 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.320+0000 7f09ac594740 1 --2- 172.21.15.22:0/2655327831 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560951b178b0 0x560951b206c0 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7f099c000c00 tx=0x7f099c00b2e0 comp rx=0 tx=0).stop 2024-03-28T18:10:44.322 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.320+0000 7f09ac594740 1 -- 172.21.15.22:0/2655327831 shutdown_connections 2024-03-28T18:10:44.322 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.320+0000 7f09ac594740 1 --2- 172.21.15.22:0/2655327831 >> [v2:172.21.15.134:6838/3400263217,v1:172.21.15.134:6839/3400263217] conn(0x560951b30ab0 0x560951b32ea0 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:10:44.322 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.320+0000 7f09ac594740 1 --2- 172.21.15.22:0/2655327831 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x560951b20de0 0x560951b1dfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:10:44.322 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.320+0000 7f09ac594740 1 --2- 172.21.15.22:0/2655327831 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560951b18260 0x560951b1da50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:10:44.322 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.320+0000 7f09ac594740 1 --2- 172.21.15.22:0/2655327831 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560951b178b0 0x560951b206c0 unknown :-1 s=CLOSED pgs=674 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:10:44.322 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.320+0000 7f09ac594740 1 -- 172.21.15.22:0/2655327831 >> 172.21.15.22:0/2655327831 conn(0x560951b11f80 msgr2=0x560951b13cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:10:44.322 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.320+0000 7f098cc00640 0 client.16416 ms_handle_reset on v2:172.21.15.134:6838/3400263217 2024-03-28T18:10:44.323 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.320+0000 7f09ac594740 1 -- 172.21.15.22:0/2655327831 shutdown_connections 2024-03-28T18:10:44.323 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.320+0000 7f09ac594740 1 -- 172.21.15.22:0/2655327831 wait complete. 2024-03-28T18:10:44.323 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.320+0000 7f09ac594740 20 client.16416 trim_cache size 0 max 16384 2024-03-28T18:10:44.334 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.334+0000 7f09ac594740 1 Processor -- start 2024-03-28T18:10:44.334 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.334+0000 7f09ac594740 1 -- start start 2024-03-28T18:10:44.334 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.334+0000 7f09ac594740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560951b178b0 0x560951b1ccc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:10:44.334 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.334+0000 7f09ac594740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x560951b18260 0x560951b20180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:10:44.334 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.334+0000 7f09ac594740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560951b20de0 0x560951b206c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:10:44.334 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.334+0000 7f09ac594740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09a41af880 con 0x560951b178b0 2024-03-28T18:10:44.334 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.334+0000 7f09ac594740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09a41b0c60 con 0x560951b18260 2024-03-28T18:10:44.335 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.334+0000 7f09ac594740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09a41afee0 con 0x560951b20de0 2024-03-28T18:10:44.335 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.334+0000 7f09a3400640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560951b178b0 0x560951b1ccc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:10:44.335 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.335+0000 7f09a3400640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560951b178b0 0x560951b1ccc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:34594/0 (socket says 172.21.15.22:34594) 2024-03-28T18:10:44.335 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.335+0000 7f09a3400640 1 -- 172.21.15.22:0/2023174211 learned_addr learned my addr 172.21.15.22:0/2023174211 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-28T18:10:44.335 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.335+0000 7f09a2000640 1 --2- 172.21.15.22:0/2023174211 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x560951b18260 0x560951b20180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:10:44.336 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.335+0000 7f09a3400640 1 -- 172.21.15.22:0/2023174211 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560951b20de0 msgr2=0x560951b206c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T18:10:44.336 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.335+0000 7f09a3400640 1 --2- 172.21.15.22:0/2023174211 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560951b20de0 0x560951b206c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:10:44.336 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.335+0000 7f09a3400640 1 -- 172.21.15.22:0/2023174211 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x560951b18260 msgr2=0x560951b20180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T18:10:44.336 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.335+0000 7f09a3400640 1 --2- 172.21.15.22:0/2023174211 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x560951b18260 0x560951b20180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:10:44.336 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.335+0000 7f09a3400640 1 -- 172.21.15.22:0/2023174211 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560951b33210 con 0x560951b178b0 2024-03-28T18:10:44.337 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.335+0000 7f09a3400640 1 --2- 172.21.15.22:0/2023174211 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560951b178b0 0x560951b1ccc0 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7f099c005880 tx=0x7f099c0199d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=18fe389dd07592f6 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:10:44.337 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.335+0000 7f098f400640 1 -- 172.21.15.22:0/2023174211 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f099c035030 con 0x560951b178b0 2024-03-28T18:10:44.337 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.335+0000 7f098f400640 1 -- 172.21.15.22:0/2023174211 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f099c01e030 con 0x560951b178b0 2024-03-28T18:10:44.337 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.335+0000 7f098f400640 1 -- 172.21.15.22:0/2023174211 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f099c00bc90 con 0x560951b178b0 2024-03-28T18:10:44.337 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.336+0000 7f09ac594740 1 -- 172.21.15.22:0/2023174211 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560951b178b0 msgr2=0x560951b1ccc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:10:44.337 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.336+0000 7f09ac594740 1 --2- 172.21.15.22:0/2023174211 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560951b178b0 0x560951b1ccc0 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7f099c005880 tx=0x7f099c0199d0 comp rx=0 tx=0).stop 2024-03-28T18:10:44.337 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.336+0000 7f09ac594740 1 -- 172.21.15.22:0/2023174211 shutdown_connections 2024-03-28T18:10:44.337 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.336+0000 7f09ac594740 1 --2- 172.21.15.22:0/2023174211 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560951b20de0 0x560951b206c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:10:44.337 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.336+0000 7f09ac594740 1 --2- 172.21.15.22:0/2023174211 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x560951b18260 0x560951b20180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:10:44.337 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.336+0000 7f09ac594740 1 --2- 172.21.15.22:0/2023174211 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560951b178b0 0x560951b1ccc0 unknown :-1 s=CLOSED pgs=675 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:10:44.337 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.336+0000 7f09ac594740 1 -- 172.21.15.22:0/2023174211 >> 172.21.15.22:0/2023174211 conn(0x560951b11f80 msgr2=0x560951b18ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:10:44.337 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.336+0000 7f09ac594740 1 -- 172.21.15.22:0/2023174211 shutdown_connections 2024-03-28T18:10:44.338 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.336+0000 7f09ac594740 1 -- 172.21.15.22:0/2023174211 wait complete. 2024-03-28T18:10:44.338 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.337+0000 7f09ac594740 1 -- start start 2024-03-28T18:10:44.338 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.337+0000 7f09ac594740 1 Processor -- start 2024-03-28T18:10:44.340 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.337+0000 7f09ac594740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x560951b178b0 0x560951b28b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:10:44.341 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.337+0000 7f09ac594740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560951b18260 0x560951b24f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:10:44.341 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.337+0000 7f09ac594740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560951b20de0 0x560951b254a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:10:44.341 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.337+0000 7f09ac594740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09a41ae000 con 0x560951b18260 2024-03-28T18:10:44.341 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.337+0000 7f09ac594740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09a41b0600 con 0x560951b178b0 2024-03-28T18:10:44.341 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.337+0000 7f09ac594740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09a41ad3c0 con 0x560951b20de0 2024-03-28T18:10:44.341 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.338+0000 7f09ac594740 10 client.0 _ll_register_callbacks cb 0x7f09940f4f10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-28T18:10:44.341 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.338+0000 7f09a2000640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560951b18260 0x560951b24f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:10:44.341 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.338+0000 7f09a2000640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560951b18260 0x560951b24f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:34608/0 (socket says 172.21.15.22:34608) 2024-03-28T18:10:44.341 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.338+0000 7f09a2000640 1 -- 172.21.15.22:0/2202685160 learned_addr learned my addr 172.21.15.22:0/2202685160 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-28T18:10:44.341 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.338+0000 7f09a3400640 1 --2- 172.21.15.22:0/2202685160 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x560951b178b0 0x560951b28b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:10:44.341 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.338+0000 7f09a2000640 1 -- 172.21.15.22:0/2202685160 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560951b20de0 msgr2=0x560951b254a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T18:10:44.341 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.338+0000 7f09a3400640 1 --2- 172.21.15.22:0/2202685160 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560951b20de0 0x560951b254a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:10:44.342 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.338+0000 7f09a2000640 1 --2- 172.21.15.22:0/2202685160 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560951b20de0 0x560951b254a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:10:44.342 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.338+0000 7f09a2000640 1 -- 172.21.15.22:0/2202685160 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x560951b178b0 msgr2=0x560951b28b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T18:10:44.342 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.338+0000 7f09a2000640 1 --2- 172.21.15.22:0/2202685160 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x560951b178b0 0x560951b28b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:10:44.342 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.338+0000 7f09a2000640 1 -- 172.21.15.22:0/2202685160 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x560951b259e0 con 0x560951b18260 2024-03-28T18:10:44.342 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.338+0000 7f09a3400640 1 --2- 172.21.15.22:0/2202685160 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560951b20de0 0x560951b254a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T18:10:44.342 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.338+0000 7f09a2000640 1 --2- 172.21.15.22:0/2202685160 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560951b18260 0x560951b24f60 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7f0990009840 tx=0x7f099001ba20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=45b745486f6f6e91 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:10:44.342 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.338+0000 7f098cc00640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-03-28T18:10:44.342 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.339+0000 7f098cc00640 1 -- 172.21.15.22:0/2202685160 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0990005440 con 0x560951b18260 2024-03-28T18:10:44.342 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.339+0000 7f098cc00640 1 -- 172.21.15.22:0/2202685160 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 8583+0+0 (secure 0 0 0) 0x7f0990004730 con 0x560951b18260 2024-03-28T18:10:44.342 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.339+0000 7f09ac594740 1 -- 172.21.15.22:0/2202685160 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f099c01e030 con 0x560951b18260 2024-03-28T18:10:44.342 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.339+0000 7f098cc00640 1 -- 172.21.15.22:0/2202685160 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7f09900055e0 con 0x560951b18260 2024-03-28T18:10:44.342 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.339+0000 7f09ac594740 10 client.16422 fetch_fsmap learned FSMap version 40 2024-03-28T18:10:44.343 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.339+0000 7f09ac594740 1 -- 172.21.15.22:0/2202685160 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) v3 -- 0x560951b8cd80 con 0x560951b18260 2024-03-28T18:10:44.343 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.339+0000 7f098cc00640 1 -- 172.21.15.22:0/2202685160 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f099c01e030 con 0x560951b18260 2024-03-28T18:10:44.343 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.339+0000 7f098cc00640 1 -- 172.21.15.22:0/2202685160 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 40) v1 ==== 7954+0+0 (secure 0 0 0) 0x7f099000f040 con 0x560951b18260 2024-03-28T18:10:44.343 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.340+0000 7f09ac594740 10 client.16422 fetch_fsmap finished waiting for FSMap version 40 2024-03-28T18:10:44.343 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.340+0000 7f09ac594740 10 client.16422 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.i 2024-03-28T18:10:44.343 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.340+0000 7f09ac594740 20 client.16422 populate_metadata read hostname 'smithi022' 2024-03-28T18:10:44.343 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.340+0000 7f09ac594740 1 --2- 172.21.15.22:0/2202685160 >> [v2:172.21.15.134:6838/3400263217,v1:172.21.15.134:6839/3400263217] conn(0x560951b8d4e0 0x560951b8f900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:10:44.343 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.340+0000 7f09ac594740 4 client.16422 mds_command: new command op to 24502 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-28T18:10:44.343 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.340+0000 7f09ac594740 1 -- 172.21.15.22:0/2202685160 --> [v2:172.21.15.134:6838/3400263217,v1:172.21.15.134:6839/3400263217] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x560951a82b50 con 0x560951b8d4e0 2024-03-28T18:10:44.343 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.340+0000 7f09a3400640 1 --2- 172.21.15.22:0/2202685160 >> [v2:172.21.15.134:6838/3400263217,v1:172.21.15.134:6839/3400263217] conn(0x560951b8d4e0 0x560951b8f900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:10:44.343 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.341+0000 7f09a3400640 1 --2- 172.21.15.22:0/2202685160 >> [v2:172.21.15.134:6838/3400263217,v1:172.21.15.134:6839/3400263217] conn(0x560951b8d4e0 0x560951b8f900 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=b9b884009ae576fc server_cookie=499376d33a6fa40f in_seq=0 out_seq=0 2024-03-28T18:10:44.343 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.341+0000 7f098cc00640 10 client.16422 ms_handle_connect on v2:172.21.15.134:6838/3400263217 2024-03-28T18:10:44.344 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.342+0000 7f098cc00640 1 -- 172.21.15.22:0/2202685160 <== mds.4 v2:172.21.15.134:6838/3400263217 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x560951a82b50 con 0x560951b8d4e0 2024-03-28T18:10:44.344 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.342+0000 7f098cc00640 10 client.16422 handle_command_reply: tid=0 2024-03-28T18:10:44.345 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.342+0000 7f09ac594740 1 client.16422 shutdown 2024-03-28T18:10:44.345 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.342+0000 7f09ac594740 1 -- 172.21.15.22:0/2202685160 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560951b18260 msgr2=0x560951b24f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:10:44.345 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.342+0000 7f09ac594740 1 --2- 172.21.15.22:0/2202685160 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560951b18260 0x560951b24f60 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7f0990009840 tx=0x7f099001ba20 comp rx=0 tx=0).stop 2024-03-28T18:10:44.345 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.342+0000 7f09ac594740 1 -- 172.21.15.22:0/2202685160 shutdown_connections 2024-03-28T18:10:44.345 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.342+0000 7f09ac594740 1 --2- 172.21.15.22:0/2202685160 >> [v2:172.21.15.134:6838/3400263217,v1:172.21.15.134:6839/3400263217] conn(0x560951b8d4e0 0x560951b8f900 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:10:44.345 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.342+0000 7f09ac594740 1 --2- 172.21.15.22:0/2202685160 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560951b20de0 0x560951b254a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:10:44.345 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.342+0000 7f09ac594740 1 --2- 172.21.15.22:0/2202685160 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560951b18260 0x560951b24f60 unknown :-1 s=CLOSED pgs=676 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:10:44.345 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.342+0000 7f09ac594740 1 --2- 172.21.15.22:0/2202685160 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x560951b178b0 0x560951b28b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:10:44.345 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.342+0000 7f098cc00640 0 client.16422 ms_handle_reset on v2:172.21.15.134:6838/3400263217 2024-03-28T18:10:44.345 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.342+0000 7f09ac594740 1 -- 172.21.15.22:0/2202685160 >> 172.21.15.22:0/2202685160 conn(0x560951b11f80 msgr2=0x560951b13220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:10:44.345 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.342+0000 7f09ac594740 1 -- 172.21.15.22:0/2202685160 shutdown_connections 2024-03-28T18:10:44.345 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.342+0000 7f09ac594740 1 -- 172.21.15.22:0/2202685160 wait complete. 2024-03-28T18:10:44.345 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.342+0000 7f09ac594740 20 client.16422 trim_cache size 0 max 16384 2024-03-28T18:10:44.346 INFO:teuthology.orchestra.run.smithi022.stdout:[] 2024-03-28T18:10:44.346 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.343+0000 7f09a3e00640 1 -- 172.21.15.22:0/2610077547 >> [v2:172.21.15.22:6800/1357365082,v1:172.21.15.22:6801/1357365082] conn(0x7f09880a29f0 msgr2=0x7f09880a4eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:10:44.346 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.343+0000 7f09a3e00640 1 --2- 172.21.15.22:0/2610077547 >> [v2:172.21.15.22:6800/1357365082,v1:172.21.15.22:6801/1357365082] conn(0x7f09880a29f0 0x7f09880a4eb0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x560951b17380 tx=0x7f09940f70e0 comp rx=0 tx=0).stop 2024-03-28T18:10:44.347 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.343+0000 7f09a3e00640 1 -- 172.21.15.22:0/2610077547 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f09a4103720 msgr2=0x7f09a419a070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:10:44.347 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.343+0000 7f09a3e00640 1 --2- 172.21.15.22:0/2610077547 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f09a4103720 0x7f09a419a070 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7f0994000f80 tx=0x7f0994034cf0 comp rx=0 tx=0).stop 2024-03-28T18:10:44.347 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.343+0000 7f09a3e00640 1 -- 172.21.15.22:0/2610077547 shutdown_connections 2024-03-28T18:10:44.347 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.343+0000 7f09a3e00640 1 --2- 172.21.15.22:0/2610077547 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f09a4109730 0x7f09a419e930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:10:44.347 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.343+0000 7f09a3e00640 1 --2- 172.21.15.22:0/2610077547 >> [v2:172.21.15.22:6800/1357365082,v1:172.21.15.22:6801/1357365082] conn(0x7f09880a29f0 0x7f09880a4eb0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:10:44.347 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.343+0000 7f09a3e00640 1 --2- 172.21.15.22:0/2610077547 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f09a41040e0 0x7f09a419a5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:10:44.347 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.343+0000 7f09a3e00640 1 --2- 172.21.15.22:0/2610077547 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f09a4103720 0x7f09a419a070 unknown :-1 s=CLOSED pgs=672 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:10:44.347 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.343+0000 7f09a3e00640 1 -- 172.21.15.22:0/2610077547 >> 172.21.15.22:0/2610077547 conn(0x7f09a40ff1d0 msgr2=0x7f09a40ffa70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:10:44.347 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.343+0000 7f09a3e00640 1 -- 172.21.15.22:0/2610077547 shutdown_connections 2024-03-28T18:10:44.347 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.343+0000 7f09a3e00640 1 -- 172.21.15.22:0/2610077547 wait complete. 2024-03-28T18:10:44.357 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:10:44 smithi022 ceph-mon[31985]: pgmap v99: 97 pgs: 97 active+clean; 117 MiB data, 799 MiB used, 1.0 TiB / 1.0 TiB avail; 92 KiB/s rd, 25 KiB/s wr, 22 op/s 2024-03-28T18:10:44.357 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:10:44 smithi022 ceph-mon[31985]: from='client.? 172.21.15.22:0/813358153' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-28T18:10:44.357 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:10:44 smithi022 ceph-mon[31985]: daemon mds.f restarted 2024-03-28T18:10:44.357 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:10:44 smithi022 ceph-mon[31985]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-03-28T18:10:44.357 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:10:44 smithi022 ceph-mon[31985]: osdmap e95: 12 total, 12 up, 12 in 2024-03-28T18:10:44.357 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:10:44 smithi022 ceph-mon[31985]: mds.? [v2:172.21.15.134:6834/3009142418,v1:172.21.15.134:6835/3009142418] up:boot 2024-03-28T18:10:44.358 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:10:44 smithi022 ceph-mon[31985]: Standby daemon mds.h assigned to filesystem cephfs as rank 1 2024-03-28T18:10:44.358 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:10:44 smithi022 ceph-mon[31985]: fsmap cephfs:4/5 {0=b=up:active,2=e=up:active,3=k=up:active,4=i=up:active} 5 up:standby-replay 3 up:standby, 1 failed 2024-03-28T18:10:44.358 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:10:44 smithi022 ceph-mon[31985]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-03-28T18:10:44.358 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:10:44 smithi022 ceph-mon[31985]: fsmap cephfs:5/5 4 up:standby-replay 3 up:standby 1 up:replay 4 up:active 2024-03-28T18:10:44.362 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-28T18:10:44.362 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2024-03-28T18:10:44.581 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.581+0000 7f202ba00640 1 -- 172.21.15.22:0/238011599 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2024107640 msgr2=0x7f2024107a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:10:44.581 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.581+0000 7f202ba00640 1 --2- 172.21.15.22:0/238011599 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2024107640 0x7f2024107a20 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7f2014009930 tx=0x7f201402bc30 comp rx=0 tx=0).stop 2024-03-28T18:10:44.581 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.581+0000 7f202ba00640 1 -- 172.21.15.22:0/238011599 shutdown_connections 2024-03-28T18:10:44.581 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.581+0000 7f202ba00640 1 --2- 172.21.15.22:0/238011599 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2024101ff0 0x7f202410e500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:10:44.582 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.581+0000 7f202ba00640 1 --2- 172.21.15.22:0/238011599 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f2024101630 0x7f2024101ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:10:44.582 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.581+0000 7f202ba00640 1 --2- 172.21.15.22:0/238011599 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2024107640 0x7f2024107a20 unknown :-1 s=CLOSED pgs=677 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:10:44.582 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.581+0000 7f202ba00640 1 -- 172.21.15.22:0/238011599 >> 172.21.15.22:0/238011599 conn(0x7f2024078aa0 msgr2=0x7f2024078eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:10:44.582 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.581+0000 7f202ba00640 1 -- 172.21.15.22:0/238011599 shutdown_connections 2024-03-28T18:10:44.582 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.581+0000 7f202ba00640 1 -- 172.21.15.22:0/238011599 wait complete. 2024-03-28T18:10:44.583 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.582+0000 7f202ba00640 1 Processor -- start 2024-03-28T18:10:44.583 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.583+0000 7f202ba00640 1 -- start start 2024-03-28T18:10:44.583 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.583+0000 7f202ba00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2024101630 0x7f202419a160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:10:44.583 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.583+0000 7f202ba00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2024101ff0 0x7f202419a6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:10:44.583 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.583+0000 7f202ba00640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f2024107640 0x7f202419e9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:10:44.584 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.583+0000 7f202ba00640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2024111a20 con 0x7f2024101630 2024-03-28T18:10:44.584 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.583+0000 7f202ba00640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20241118a0 con 0x7f2024107640 2024-03-28T18:10:44.584 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.583+0000 7f202ba00640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2024111ba0 con 0x7f2024101ff0 2024-03-28T18:10:44.584 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.583+0000 7f2029000640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2024101630 0x7f202419a160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:10:44.584 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.584+0000 7f2029000640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2024101630 0x7f202419a160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:34624/0 (socket says 172.21.15.22:34624) 2024-03-28T18:10:44.584 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.584+0000 7f2029000640 1 -- 172.21.15.22:0/2465530591 learned_addr learned my addr 172.21.15.22:0/2465530591 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-28T18:10:44.584 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.584+0000 7f2029a00640 1 --2- 172.21.15.22:0/2465530591 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f2024107640 0x7f202419e9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:10:44.584 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.584+0000 7f201be00640 1 --2- 172.21.15.22:0/2465530591 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2024101ff0 0x7f202419a6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:10:44.585 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.584+0000 7f2029000640 1 -- 172.21.15.22:0/2465530591 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2024101ff0 msgr2=0x7f202419a6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:10:44.585 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.584+0000 7f2029000640 1 --2- 172.21.15.22:0/2465530591 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2024101ff0 0x7f202419a6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:10:44.585 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.584+0000 7f2029000640 1 -- 172.21.15.22:0/2465530591 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f2024107640 msgr2=0x7f202419e9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:10:44.585 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.584+0000 7f2029000640 1 --2- 172.21.15.22:0/2465530591 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f2024107640 0x7f202419e9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:10:44.586 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.584+0000 7f2029000640 1 -- 172.21.15.22:0/2465530591 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f202419f100 con 0x7f2024101630 2024-03-28T18:10:44.586 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.584+0000 7f201be00640 1 --2- 172.21.15.22:0/2465530591 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2024101ff0 0x7f202419a6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-28T18:10:44.586 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.584+0000 7f2029000640 1 --2- 172.21.15.22:0/2465530591 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2024101630 0x7f202419a160 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7f2014000c00 tx=0x7f20140045e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:10:44.586 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.585+0000 7f2022000640 1 -- 172.21.15.22:0/2465530591 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2014043020 con 0x7f2024101630 2024-03-28T18:10:44.586 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.585+0000 7f2022000640 1 -- 172.21.15.22:0/2465530591 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f201402caa0 con 0x7f2024101630 2024-03-28T18:10:44.586 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.585+0000 7f2022000640 1 -- 172.21.15.22:0/2465530591 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2014003bf0 con 0x7f2024101630 2024-03-28T18:10:44.586 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.585+0000 7f202ba00640 1 -- 172.21.15.22:0/2465530591 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f202419f390 con 0x7f2024101630 2024-03-28T18:10:44.586 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.585+0000 7f202ba00640 1 -- 172.21.15.22:0/2465530591 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f20241a6c80 con 0x7f2024101630 2024-03-28T18:10:44.588 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.588+0000 7f2022000640 1 -- 172.21.15.22:0/2465530591 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 26) v1 ==== 137421+0+0 (secure 0 0 0) 0x7f2014046020 con 0x7f2024101630 2024-03-28T18:10:44.588 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.588+0000 7f202e835740 1 Processor -- start 2024-03-28T18:10:44.588 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.589+0000 7f202e835740 1 -- start start 2024-03-28T18:10:44.590 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.589+0000 7f202e835740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561ddf10c6d0 0x561ddf10cab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:10:44.590 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.589+0000 7f202e835740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561ddf10d080 0x561ddf115630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:10:44.590 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.589+0000 7f202e835740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x561ddf115c00 0x561ddf118050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:10:44.590 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.589+0000 7f202e835740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x561ddf0702c0 con 0x561ddf10c6d0 2024-03-28T18:10:44.590 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.589+0000 7f202e835740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x561ddf018c90 con 0x561ddf115c00 2024-03-28T18:10:44.591 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.589+0000 7f202e835740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x561ddf06c360 con 0x561ddf10d080 2024-03-28T18:10:44.591 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.589+0000 7f2029a00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561ddf10c6d0 0x561ddf10cab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:10:44.591 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.589+0000 7f2029a00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561ddf10c6d0 0x561ddf10cab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:34630/0 (socket says 172.21.15.22:34630) 2024-03-28T18:10:44.591 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.589+0000 7f2029a00640 1 -- 172.21.15.22:0/3639298293 learned_addr learned my addr 172.21.15.22:0/3639298293 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-28T18:10:44.591 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.589+0000 7f201be00640 1 --2- 172.21.15.22:0/3639298293 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x561ddf115c00 0x561ddf118050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:10:44.591 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.589+0000 7f2029000640 1 --2- 172.21.15.22:0/3639298293 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561ddf10d080 0x561ddf115630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:10:44.591 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.590+0000 7f2029a00640 1 -- 172.21.15.22:0/3639298293 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561ddf10d080 msgr2=0x561ddf115630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T18:10:44.591 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.590+0000 7f2029a00640 1 --2- 172.21.15.22:0/3639298293 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561ddf10d080 0x561ddf115630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:10:44.591 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.590+0000 7f2029a00640 1 -- 172.21.15.22:0/3639298293 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x561ddf115c00 msgr2=0x561ddf118050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T18:10:44.591 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.590+0000 7f2029a00640 1 --2- 172.21.15.22:0/3639298293 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x561ddf115c00 0x561ddf118050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:10:44.591 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.590+0000 7f2029a00640 1 -- 172.21.15.22:0/3639298293 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561ddf03d080 con 0x561ddf10c6d0 2024-03-28T18:10:44.591 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.590+0000 7f2029a00640 1 --2- 172.21.15.22:0/3639298293 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561ddf10c6d0 0x561ddf10cab0 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7f201c009f70 tx=0x7f201c010800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5ca2537457a222cb server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:10:44.591 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.590+0000 7f2022000640 1 --2- 172.21.15.22:0/2465530591 >> [v2:172.21.15.22:6800/1357365082,v1:172.21.15.22:6801/1357365082] conn(0x7f20080a29f0 0x7f20080a4eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:10:44.591 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.590+0000 7f2029000640 1 --2- 172.21.15.22:0/2465530591 >> [v2:172.21.15.22:6800/1357365082,v1:172.21.15.22:6801/1357365082] conn(0x7f20080a29f0 0x7f20080a4eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:10:44.591 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.590+0000 7f201a000640 1 -- 172.21.15.22:0/3639298293 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f201c016070 con 0x561ddf10c6d0 2024-03-28T18:10:44.592 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.591+0000 7f201a000640 1 -- 172.21.15.22:0/3639298293 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f201c004d20 con 0x561ddf10c6d0 2024-03-28T18:10:44.592 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.591+0000 7f201a000640 1 -- 172.21.15.22:0/3639298293 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f201c012910 con 0x561ddf10c6d0 2024-03-28T18:10:44.592 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.591+0000 7f2022000640 1 -- 172.21.15.22:0/2465530591 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 8583+0+0 (secure 0 0 0) 0x7f20140f0860 con 0x7f2024101630 2024-03-28T18:10:44.595 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.591+0000 7f202e835740 1 -- 172.21.15.22:0/3639298293 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561ddf10c6d0 msgr2=0x561ddf10cab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:10:44.595 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.591+0000 7f2029000640 1 --2- 172.21.15.22:0/2465530591 >> [v2:172.21.15.22:6800/1357365082,v1:172.21.15.22:6801/1357365082] conn(0x7f20080a29f0 0x7f20080a4eb0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x561ddf10c1a0 tx=0x7f201402bc30 comp rx=0 tx=0).ready entity=mgr.15522 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:10:44.595 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.591+0000 7f202e835740 1 --2- 172.21.15.22:0/3639298293 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561ddf10c6d0 0x561ddf10cab0 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7f201c009f70 tx=0x7f201c010800 comp rx=0 tx=0).stop 2024-03-28T18:10:44.595 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.591+0000 7f202e835740 1 -- 172.21.15.22:0/3639298293 shutdown_connections 2024-03-28T18:10:44.595 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.591+0000 7f202e835740 1 --2- 172.21.15.22:0/3639298293 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x561ddf115c00 0x561ddf118050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:10:44.596 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.591+0000 7f202e835740 1 --2- 172.21.15.22:0/3639298293 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561ddf10d080 0x561ddf115630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:10:44.596 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.591+0000 7f202e835740 1 --2- 172.21.15.22:0/3639298293 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561ddf10c6d0 0x561ddf10cab0 unknown :-1 s=CLOSED pgs=679 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:10:44.596 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.591+0000 7f202e835740 1 -- 172.21.15.22:0/3639298293 >> 172.21.15.22:0/3639298293 conn(0x561ddf106da0 msgr2=0x561ddf107180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:10:44.596 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.592+0000 7f202e835740 1 -- 172.21.15.22:0/3639298293 shutdown_connections 2024-03-28T18:10:44.596 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.592+0000 7f202e835740 1 -- 172.21.15.22:0/3639298293 wait complete. 2024-03-28T18:10:44.596 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.592+0000 7f202e835740 1 -- start start 2024-03-28T18:10:44.596 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.592+0000 7f202e835740 1 Processor -- start 2024-03-28T18:10:44.596 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.593+0000 7f202e835740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561ddf10c6d0 0x561ddf115550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:10:44.596 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.593+0000 7f202e835740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561ddf10d080 0x561ddf1128e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:10:44.596 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.593+0000 7f202e835740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x561ddf115c00 0x561ddf112ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:10:44.596 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.593+0000 7f202e835740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x561ddf118b70 con 0x561ddf10d080 2024-03-28T18:10:44.596 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.593+0000 7f202e835740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x561ddf02c0d0 con 0x561ddf115c00 2024-03-28T18:10:44.596 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.593+0000 7f202e835740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x561ddf118cf0 con 0x561ddf10c6d0 2024-03-28T18:10:44.596 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.593+0000 7f202e835740 10 client.0 _ll_register_callbacks cb 0x561ddf030e80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-28T18:10:44.597 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.593+0000 7f201be00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561ddf10d080 0x561ddf1128e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:10:44.597 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.593+0000 7f201be00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561ddf10d080 0x561ddf1128e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:34644/0 (socket says 172.21.15.22:34644) 2024-03-28T18:10:44.597 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.593+0000 7f201be00640 1 -- 172.21.15.22:0/2102995244 learned_addr learned my addr 172.21.15.22:0/2102995244 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-28T18:10:44.597 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.593+0000 7f2029a00640 1 --2- 172.21.15.22:0/2102995244 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x561ddf115c00 0x561ddf112ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:10:44.597 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.594+0000 7f201be00640 1 -- 172.21.15.22:0/2102995244 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561ddf10c6d0 msgr2=0x561ddf115550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T18:10:44.597 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.594+0000 7f2029a00640 1 --2- 172.21.15.22:0/2102995244 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561ddf10c6d0 0x561ddf115550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:10:44.597 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.594+0000 7f201be00640 1 --2- 172.21.15.22:0/2102995244 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561ddf10c6d0 0x561ddf115550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:10:44.597 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.594+0000 7f201be00640 1 -- 172.21.15.22:0/2102995244 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x561ddf115c00 msgr2=0x561ddf112ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T18:10:44.597 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.594+0000 7f201be00640 1 --2- 172.21.15.22:0/2102995244 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x561ddf115c00 0x561ddf112ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:10:44.597 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.594+0000 7f201be00640 1 -- 172.21.15.22:0/2102995244 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x561dded843f0 con 0x561ddf10d080 2024-03-28T18:10:44.597 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.594+0000 7f2029a00640 1 --2- 172.21.15.22:0/2102995244 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561ddf10c6d0 0x561ddf115550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T18:10:44.597 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.594+0000 7f201be00640 1 --2- 172.21.15.22:0/2102995244 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561ddf10d080 0x561ddf1128e0 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7f200c00bfb0 tx=0x7f200c0128e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7bd7b39c2d44c804 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:10:44.597 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.594+0000 7f2012a00640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-03-28T18:10:44.597 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.594+0000 7f2012a00640 1 -- 172.21.15.22:0/2102995244 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f200c0035f0 con 0x561ddf10d080 2024-03-28T18:10:44.598 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.594+0000 7f2012a00640 1 -- 172.21.15.22:0/2102995244 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 8583+0+0 (secure 0 0 0) 0x7f200c003790 con 0x561ddf10d080 2024-03-28T18:10:44.598 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.594+0000 7f202e835740 1 -- 172.21.15.22:0/2102995244 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f201c004d20 con 0x561ddf10d080 2024-03-28T18:10:44.598 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.594+0000 7f2012a00640 1 -- 172.21.15.22:0/2102995244 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7f200c003990 con 0x561ddf10d080 2024-03-28T18:10:44.598 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.595+0000 7f202e835740 10 client.16434 fetch_fsmap learned FSMap version 40 2024-03-28T18:10:44.598 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.595+0000 7f202e835740 1 -- 172.21.15.22:0/2102995244 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) v3 -- 0x561ddee4e7d0 con 0x561ddf10d080 2024-03-28T18:10:44.598 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.595+0000 7f2012a00640 1 -- 172.21.15.22:0/2102995244 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f201c004d20 con 0x561ddf10d080 2024-03-28T18:10:44.598 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.595+0000 7f2012a00640 1 -- 172.21.15.22:0/2102995244 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 40) v1 ==== 7954+0+0 (secure 0 0 0) 0x7f200c021030 con 0x561ddf10d080 2024-03-28T18:10:44.598 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.595+0000 7f202e835740 10 client.16434 fetch_fsmap finished waiting for FSMap version 40 2024-03-28T18:10:44.598 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.595+0000 7f202e835740 10 client.16434 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.k 2024-03-28T18:10:44.598 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.595+0000 7f202e835740 20 client.16434 populate_metadata read hostname 'smithi022' 2024-03-28T18:10:44.598 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.595+0000 7f202e835740 1 --2- 172.21.15.22:0/2102995244 >> [v2:172.21.15.79:6832/856896997,v1:172.21.15.79:6833/856896997] conn(0x561ddf125b70 0x561ddf127f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:10:44.598 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.595+0000 7f202e835740 4 client.16434 mds_command: new command op to 34442 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-28T18:10:44.598 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.595+0000 7f202e835740 1 -- 172.21.15.22:0/2102995244 --> [v2:172.21.15.79:6832/856896997,v1:172.21.15.79:6833/856896997] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x561ddf07d800 con 0x561ddf125b70 2024-03-28T18:10:44.598 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.596+0000 7f2029a00640 1 --2- 172.21.15.22:0/2102995244 >> [v2:172.21.15.79:6832/856896997,v1:172.21.15.79:6833/856896997] conn(0x561ddf125b70 0x561ddf127f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:10:44.600 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.596+0000 7f2029a00640 1 --2- 172.21.15.22:0/2102995244 >> [v2:172.21.15.79:6832/856896997,v1:172.21.15.79:6833/856896997] conn(0x561ddf125b70 0x561ddf127f60 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=7920cef2cd63efbc server_cookie=20e52348c34af17f in_seq=0 out_seq=0 2024-03-28T18:10:44.600 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.596+0000 7f2012a00640 10 client.16434 ms_handle_connect on v2:172.21.15.79:6832/856896997 2024-03-28T18:10:44.600 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.598+0000 7f2012a00640 1 -- 172.21.15.22:0/2102995244 <== mds.3 v2:172.21.15.79:6832/856896997 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x561ddf07d800 con 0x561ddf125b70 2024-03-28T18:10:44.600 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.598+0000 7f2012a00640 10 client.16434 handle_command_reply: tid=0 2024-03-28T18:10:44.600 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.598+0000 7f202e835740 1 client.16434 shutdown 2024-03-28T18:10:44.600 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.598+0000 7f202e835740 1 -- 172.21.15.22:0/2102995244 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561ddf10d080 msgr2=0x561ddf1128e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:10:44.600 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.598+0000 7f202e835740 1 --2- 172.21.15.22:0/2102995244 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561ddf10d080 0x561ddf1128e0 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7f200c00bfb0 tx=0x7f200c0128e0 comp rx=0 tx=0).stop 2024-03-28T18:10:44.600 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.598+0000 7f202e835740 1 -- 172.21.15.22:0/2102995244 shutdown_connections 2024-03-28T18:10:44.600 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.598+0000 7f202e835740 1 --2- 172.21.15.22:0/2102995244 >> [v2:172.21.15.79:6832/856896997,v1:172.21.15.79:6833/856896997] conn(0x561ddf125b70 0x561ddf127f60 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:10:44.600 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.598+0000 7f202e835740 1 --2- 172.21.15.22:0/2102995244 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x561ddf115c00 0x561ddf112ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:10:44.600 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.598+0000 7f202e835740 1 --2- 172.21.15.22:0/2102995244 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561ddf10d080 0x561ddf1128e0 unknown :-1 s=CLOSED pgs=680 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:10:44.601 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.598+0000 7f202e835740 1 --2- 172.21.15.22:0/2102995244 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561ddf10c6d0 0x561ddf115550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:10:44.601 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.598+0000 7f202e835740 1 -- 172.21.15.22:0/2102995244 >> 172.21.15.22:0/2102995244 conn(0x561ddf106da0 msgr2=0x561ddf108af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:10:44.601 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.598+0000 7f2012a00640 0 client.16434 ms_handle_reset on v2:172.21.15.79:6832/856896997 2024-03-28T18:10:44.601 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.598+0000 7f202e835740 1 -- 172.21.15.22:0/2102995244 shutdown_connections 2024-03-28T18:10:44.601 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.599+0000 7f202e835740 1 -- 172.21.15.22:0/2102995244 wait complete. 2024-03-28T18:10:44.601 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.599+0000 7f202e835740 20 client.16434 trim_cache size 0 max 16384 2024-03-28T18:10:44.612 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.612+0000 7f202e835740 1 Processor -- start 2024-03-28T18:10:44.612 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.613+0000 7f202e835740 1 -- start start 2024-03-28T18:10:44.614 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.613+0000 7f202e835740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561ddf10c6d0 0x561ddf101d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:10:44.614 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.613+0000 7f202e835740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x561ddf10d080 0x561ddf115010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:10:44.614 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.613+0000 7f202e835740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561ddf115c00 0x561ddf115550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:10:44.614 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.613+0000 7f202e835740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20241af9b0 con 0x561ddf115c00 2024-03-28T18:10:44.614 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.613+0000 7f202e835740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20241b0d90 con 0x561ddf10d080 2024-03-28T18:10:44.614 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.613+0000 7f202e835740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20241b0010 con 0x561ddf10c6d0 2024-03-28T18:10:44.614 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.613+0000 7f2029a00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561ddf115c00 0x561ddf115550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:10:44.614 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.613+0000 7f2029a00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561ddf115c00 0x561ddf115550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:34652/0 (socket says 172.21.15.22:34652) 2024-03-28T18:10:44.614 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.613+0000 7f2029a00640 1 -- 172.21.15.22:0/823797465 learned_addr learned my addr 172.21.15.22:0/823797465 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-28T18:10:44.614 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.613+0000 7f201be00640 1 --2- 172.21.15.22:0/823797465 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x561ddf10d080 0x561ddf115010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:10:44.615 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.614+0000 7f2029a00640 1 --2- 172.21.15.22:0/823797465 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561ddf10c6d0 0x561ddf101d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:10:44.615 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.614+0000 7f2029a00640 1 -- 172.21.15.22:0/823797465 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561ddf10c6d0 msgr2=0x561ddf101d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T18:10:44.615 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.614+0000 7f2029a00640 1 --2- 172.21.15.22:0/823797465 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561ddf10c6d0 0x561ddf101d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:10:44.615 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.614+0000 7f2029a00640 1 -- 172.21.15.22:0/823797465 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x561ddf10d080 msgr2=0x561ddf115010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T18:10:44.615 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.614+0000 7f2029a00640 1 --2- 172.21.15.22:0/823797465 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x561ddf10d080 0x561ddf115010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:10:44.615 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.614+0000 7f2029a00640 1 -- 172.21.15.22:0/823797465 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561ddf10c2b0 con 0x561ddf115c00 2024-03-28T18:10:44.615 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.614+0000 7f201be00640 1 --2- 172.21.15.22:0/823797465 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x561ddf10d080 0x561ddf115010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-28T18:10:44.615 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.614+0000 7f2029a00640 1 --2- 172.21.15.22:0/823797465 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561ddf115c00 0x561ddf115550 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7f201c012580 tx=0x7f201c010800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=202568a3ef5c8e08 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:10:44.617 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.615+0000 7f2019600640 1 -- 172.21.15.22:0/823797465 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f201c016040 con 0x561ddf115c00 2024-03-28T18:10:44.617 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.615+0000 7f2019600640 1 -- 172.21.15.22:0/823797465 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f201c003e40 con 0x561ddf115c00 2024-03-28T18:10:44.617 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.615+0000 7f2019600640 1 -- 172.21.15.22:0/823797465 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f201c02ac00 con 0x561ddf115c00 2024-03-28T18:10:44.617 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.615+0000 7f202e835740 1 -- 172.21.15.22:0/823797465 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561ddf115c00 msgr2=0x561ddf115550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:10:44.617 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.615+0000 7f202e835740 1 --2- 172.21.15.22:0/823797465 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561ddf115c00 0x561ddf115550 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7f201c012580 tx=0x7f201c010800 comp rx=0 tx=0).stop 2024-03-28T18:10:44.617 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.615+0000 7f202e835740 1 -- 172.21.15.22:0/823797465 shutdown_connections 2024-03-28T18:10:44.618 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.615+0000 7f202e835740 1 --2- 172.21.15.22:0/823797465 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561ddf115c00 0x561ddf115550 unknown :-1 s=CLOSED pgs=681 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:10:44.618 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.615+0000 7f202e835740 1 --2- 172.21.15.22:0/823797465 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x561ddf10d080 0x561ddf115010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:10:44.618 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.615+0000 7f202e835740 1 --2- 172.21.15.22:0/823797465 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561ddf10c6d0 0x561ddf101d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:10:44.618 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.615+0000 7f202e835740 1 -- 172.21.15.22:0/823797465 >> 172.21.15.22:0/823797465 conn(0x561ddf106da0 msgr2=0x561ddf108a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:10:44.618 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.615+0000 7f202e835740 1 -- 172.21.15.22:0/823797465 shutdown_connections 2024-03-28T18:10:44.618 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.615+0000 7f202e835740 1 -- 172.21.15.22:0/823797465 wait complete. 2024-03-28T18:10:44.618 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.616+0000 7f202e835740 1 -- start start 2024-03-28T18:10:44.618 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.616+0000 7f202e835740 1 Processor -- start 2024-03-28T18:10:44.618 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.616+0000 7f202e835740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x561ddf10c6d0 0x561ddf11b9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:10:44.618 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.616+0000 7f202e835740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561ddf10d080 0x561ddf11bf30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:10:44.618 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.616+0000 7f202e835740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561ddf115c00 0x561ddf180440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:10:44.618 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.617+0000 7f202e835740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20241ae130 con 0x561ddf115c00 2024-03-28T18:10:44.618 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.617+0000 7f202e835740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20241b0730 con 0x561ddf10c6d0 2024-03-28T18:10:44.619 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.617+0000 7f202e835740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20241ad4f0 con 0x561ddf10d080 2024-03-28T18:10:44.619 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.617+0000 7f202e835740 10 client.0 _ll_register_callbacks cb 0x7f200c00b8e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-28T18:10:44.619 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.617+0000 7f2029a00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561ddf115c00 0x561ddf180440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:10:44.619 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.617+0000 7f2029a00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561ddf115c00 0x561ddf180440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:34658/0 (socket says 172.21.15.22:34658) 2024-03-28T18:10:44.619 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.617+0000 7f2029a00640 1 -- 172.21.15.22:0/352893228 learned_addr learned my addr 172.21.15.22:0/352893228 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-28T18:10:44.619 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.617+0000 7f201be00640 1 --2- 172.21.15.22:0/352893228 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561ddf10d080 0x561ddf11bf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:10:44.619 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.617+0000 7f2029a00640 1 --2- 172.21.15.22:0/352893228 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x561ddf10c6d0 0x561ddf11b9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:10:44.619 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.618+0000 7f2029a00640 1 -- 172.21.15.22:0/352893228 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561ddf10d080 msgr2=0x561ddf11bf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T18:10:44.619 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.618+0000 7f2029a00640 1 --2- 172.21.15.22:0/352893228 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561ddf10d080 0x561ddf11bf30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:10:44.619 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.618+0000 7f2029a00640 1 -- 172.21.15.22:0/352893228 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x561ddf10c6d0 msgr2=0x561ddf11b9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T18:10:44.619 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.618+0000 7f2029a00640 1 --2- 172.21.15.22:0/352893228 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x561ddf10c6d0 0x561ddf11b9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:10:44.619 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.618+0000 7f2029a00640 1 -- 172.21.15.22:0/352893228 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x561ddf180980 con 0x561ddf115c00 2024-03-28T18:10:44.622 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.618+0000 7f2029a00640 1 --2- 172.21.15.22:0/352893228 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561ddf115c00 0x561ddf180440 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7f201c009ca0 tx=0x7f201c02d040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f4fc4bb7ebb0d89e server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:10:44.622 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.618+0000 7f2012a00640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-03-28T18:10:44.622 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.618+0000 7f2012a00640 1 -- 172.21.15.22:0/352893228 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f201c030020 con 0x561ddf115c00 2024-03-28T18:10:44.622 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.618+0000 7f2012a00640 1 -- 172.21.15.22:0/352893228 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 8583+0+0 (secure 0 0 0) 0x7f201c003970 con 0x561ddf115c00 2024-03-28T18:10:44.622 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.618+0000 7f202e835740 1 -- 172.21.15.22:0/352893228 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x561ddf077b50 con 0x561ddf115c00 2024-03-28T18:10:44.622 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.618+0000 7f2012a00640 1 -- 172.21.15.22:0/352893228 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7f201c016040 con 0x561ddf115c00 2024-03-28T18:10:44.622 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.618+0000 7f202e835740 10 client.16440 fetch_fsmap learned FSMap version 40 2024-03-28T18:10:44.622 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.619+0000 7f202e835740 1 -- 172.21.15.22:0/352893228 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) v3 -- 0x561ddf181bb0 con 0x561ddf115c00 2024-03-28T18:10:44.622 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.619+0000 7f2012a00640 1 -- 172.21.15.22:0/352893228 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x561ddf077b50 con 0x561ddf115c00 2024-03-28T18:10:44.622 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.619+0000 7f2012a00640 1 -- 172.21.15.22:0/352893228 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 40) v1 ==== 7954+0+0 (secure 0 0 0) 0x7f201c00f040 con 0x561ddf115c00 2024-03-28T18:10:44.622 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.619+0000 7f202e835740 10 client.16440 fetch_fsmap finished waiting for FSMap version 40 2024-03-28T18:10:44.623 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.619+0000 7f202e835740 10 client.16440 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.k 2024-03-28T18:10:44.623 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.619+0000 7f202e835740 20 client.16440 populate_metadata read hostname 'smithi022' 2024-03-28T18:10:44.623 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.619+0000 7f202e835740 1 --2- 172.21.15.22:0/352893228 >> [v2:172.21.15.79:6832/856896997,v1:172.21.15.79:6833/856896997] conn(0x561ddf181fe0 0x561ddf1843d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:10:44.623 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.619+0000 7f202e835740 4 client.16440 mds_command: new command op to 34442 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-28T18:10:44.623 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.619+0000 7f202e835740 1 -- 172.21.15.22:0/352893228 --> [v2:172.21.15.79:6832/856896997,v1:172.21.15.79:6833/856896997] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x561ddeff5500 con 0x561ddf181fe0 2024-03-28T18:10:44.623 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.620+0000 7f201be00640 1 --2- 172.21.15.22:0/352893228 >> [v2:172.21.15.79:6832/856896997,v1:172.21.15.79:6833/856896997] conn(0x561ddf181fe0 0x561ddf1843d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:10:44.623 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.620+0000 7f201be00640 1 --2- 172.21.15.22:0/352893228 >> [v2:172.21.15.79:6832/856896997,v1:172.21.15.79:6833/856896997] conn(0x561ddf181fe0 0x561ddf1843d0 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=68e17dde3498afff server_cookie=192d77720b44386e in_seq=0 out_seq=0 2024-03-28T18:10:44.623 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.620+0000 7f2012a00640 10 client.16440 ms_handle_connect on v2:172.21.15.79:6832/856896997 2024-03-28T18:10:44.623 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.621+0000 7f2012a00640 1 -- 172.21.15.22:0/352893228 <== mds.3 v2:172.21.15.79:6832/856896997 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x561ddeff5500 con 0x561ddf181fe0 2024-03-28T18:10:44.623 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.621+0000 7f2012a00640 10 client.16440 handle_command_reply: tid=0 2024-03-28T18:10:44.623 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.621+0000 7f202e835740 1 client.16440 shutdown 2024-03-28T18:10:44.623 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.621+0000 7f202e835740 1 -- 172.21.15.22:0/352893228 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561ddf115c00 msgr2=0x561ddf180440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:10:44.623 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.621+0000 7f202e835740 1 --2- 172.21.15.22:0/352893228 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561ddf115c00 0x561ddf180440 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7f201c009ca0 tx=0x7f201c02d040 comp rx=0 tx=0).stop 2024-03-28T18:10:44.623 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.621+0000 7f202e835740 1 -- 172.21.15.22:0/352893228 shutdown_connections 2024-03-28T18:10:44.623 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.621+0000 7f202e835740 1 --2- 172.21.15.22:0/352893228 >> [v2:172.21.15.79:6832/856896997,v1:172.21.15.79:6833/856896997] conn(0x561ddf181fe0 0x561ddf1843d0 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:10:44.624 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.621+0000 7f202e835740 1 --2- 172.21.15.22:0/352893228 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561ddf115c00 0x561ddf180440 unknown :-1 s=CLOSED pgs=682 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:10:44.624 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.621+0000 7f202e835740 1 --2- 172.21.15.22:0/352893228 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561ddf10d080 0x561ddf11bf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:10:44.624 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.621+0000 7f202e835740 1 --2- 172.21.15.22:0/352893228 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x561ddf10c6d0 0x561ddf11b9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:10:44.624 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.621+0000 7f2012a00640 0 client.16440 ms_handle_reset on v2:172.21.15.79:6832/856896997 2024-03-28T18:10:44.624 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.621+0000 7f202e835740 1 -- 172.21.15.22:0/352893228 >> 172.21.15.22:0/352893228 conn(0x561ddf106da0 msgr2=0x561ddf1081f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:10:44.624 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.621+0000 7f202e835740 1 -- 172.21.15.22:0/352893228 shutdown_connections 2024-03-28T18:10:44.624 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.621+0000 7f202e835740 1 -- 172.21.15.22:0/352893228 wait complete. 2024-03-28T18:10:44.624 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.621+0000 7f202e835740 20 client.16440 trim_cache size 0 max 16384 2024-03-28T18:10:44.624 INFO:teuthology.orchestra.run.smithi022.stdout:[] 2024-03-28T18:10:44.625 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.622+0000 7f202ba00640 1 -- 172.21.15.22:0/2465530591 >> [v2:172.21.15.22:6800/1357365082,v1:172.21.15.22:6801/1357365082] conn(0x7f20080a29f0 msgr2=0x7f20080a4eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:10:44.625 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.622+0000 7f202ba00640 1 --2- 172.21.15.22:0/2465530591 >> [v2:172.21.15.22:6800/1357365082,v1:172.21.15.22:6801/1357365082] conn(0x7f20080a29f0 0x7f20080a4eb0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x561ddf10c1a0 tx=0x7f201402bc30 comp rx=0 tx=0).stop 2024-03-28T18:10:44.625 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.622+0000 7f202ba00640 1 -- 172.21.15.22:0/2465530591 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2024101630 msgr2=0x7f202419a160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:10:44.625 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.622+0000 7f202ba00640 1 --2- 172.21.15.22:0/2465530591 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2024101630 0x7f202419a160 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7f2014000c00 tx=0x7f20140045e0 comp rx=0 tx=0).stop 2024-03-28T18:10:44.625 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.622+0000 7f202ba00640 1 -- 172.21.15.22:0/2465530591 shutdown_connections 2024-03-28T18:10:44.625 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.622+0000 7f202ba00640 1 --2- 172.21.15.22:0/2465530591 >> [v2:172.21.15.22:6800/1357365082,v1:172.21.15.22:6801/1357365082] conn(0x7f20080a29f0 0x7f20080a4eb0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:10:44.625 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.622+0000 7f202ba00640 1 --2- 172.21.15.22:0/2465530591 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f2024107640 0x7f202419e9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:10:44.626 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.622+0000 7f202ba00640 1 --2- 172.21.15.22:0/2465530591 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2024101ff0 0x7f202419a6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:10:44.626 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.622+0000 7f202ba00640 1 --2- 172.21.15.22:0/2465530591 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2024101630 0x7f202419a160 unknown :-1 s=CLOSED pgs=678 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:10:44.626 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.622+0000 7f202ba00640 1 -- 172.21.15.22:0/2465530591 >> 172.21.15.22:0/2465530591 conn(0x7f2024078aa0 msgr2=0x7f20241031c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:10:44.626 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.622+0000 7f202ba00640 1 -- 172.21.15.22:0/2465530591 shutdown_connections 2024-03-28T18:10:44.626 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.622+0000 7f202ba00640 1 -- 172.21.15.22:0/2465530591 wait complete. 2024-03-28T18:10:44.639 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-28T18:10:44.639 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-03-28T18:10:44.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:10:44 smithi134 ceph-mon[35037]: pgmap v99: 97 pgs: 97 active+clean; 117 MiB data, 799 MiB used, 1.0 TiB / 1.0 TiB avail; 92 KiB/s rd, 25 KiB/s wr, 22 op/s 2024-03-28T18:10:44.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:10:44 smithi134 ceph-mon[35037]: from='client.? 172.21.15.22:0/813358153' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-28T18:10:44.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:10:44 smithi134 ceph-mon[35037]: daemon mds.f restarted 2024-03-28T18:10:44.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:10:44 smithi134 ceph-mon[35037]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-03-28T18:10:44.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:10:44 smithi134 ceph-mon[35037]: osdmap e95: 12 total, 12 up, 12 in 2024-03-28T18:10:44.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:10:44 smithi134 ceph-mon[35037]: mds.? [v2:172.21.15.134:6834/3009142418,v1:172.21.15.134:6835/3009142418] up:boot 2024-03-28T18:10:44.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:10:44 smithi134 ceph-mon[35037]: Standby daemon mds.h assigned to filesystem cephfs as rank 1 2024-03-28T18:10:44.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:10:44 smithi134 ceph-mon[35037]: fsmap cephfs:4/5 {0=b=up:active,2=e=up:active,3=k=up:active,4=i=up:active} 5 up:standby-replay 3 up:standby, 1 failed 2024-03-28T18:10:44.676 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:10:44 smithi134 ceph-mon[35037]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-03-28T18:10:44.676 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:10:44 smithi134 ceph-mon[35037]: fsmap cephfs:5/5 4 up:standby-replay 3 up:standby 1 up:replay 4 up:active 2024-03-28T18:10:44.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:10:44 smithi079 ceph-mon[35312]: pgmap v99: 97 pgs: 97 active+clean; 117 MiB data, 799 MiB used, 1.0 TiB / 1.0 TiB avail; 92 KiB/s rd, 25 KiB/s wr, 22 op/s 2024-03-28T18:10:44.683 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:10:44 smithi079 ceph-mon[35312]: from='client.? 172.21.15.22:0/813358153' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-28T18:10:44.683 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:10:44 smithi079 ceph-mon[35312]: daemon mds.f restarted 2024-03-28T18:10:44.683 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:10:44 smithi079 ceph-mon[35312]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-03-28T18:10:44.683 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:10:44 smithi079 ceph-mon[35312]: osdmap e95: 12 total, 12 up, 12 in 2024-03-28T18:10:44.683 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:10:44 smithi079 ceph-mon[35312]: mds.? [v2:172.21.15.134:6834/3009142418,v1:172.21.15.134:6835/3009142418] up:boot 2024-03-28T18:10:44.683 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:10:44 smithi079 ceph-mon[35312]: Standby daemon mds.h assigned to filesystem cephfs as rank 1 2024-03-28T18:10:44.683 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:10:44 smithi079 ceph-mon[35312]: fsmap cephfs:4/5 {0=b=up:active,2=e=up:active,3=k=up:active,4=i=up:active} 5 up:standby-replay 3 up:standby, 1 failed 2024-03-28T18:10:44.683 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:10:44 smithi079 ceph-mon[35312]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-03-28T18:10:44.683 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:10:44 smithi079 ceph-mon[35312]: fsmap cephfs:5/5 4 up:standby-replay 3 up:standby 1 up:replay 4 up:active 2024-03-28T18:10:44.864 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.863+0000 7fea0a000640 1 -- 172.21.15.22:0/3870273710 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fea040764f0 msgr2=0x7fea04111fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:10:44.864 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.863+0000 7fea0a000640 1 --2- 172.21.15.22:0/3870273710 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fea040764f0 0x7fea04111fa0 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7fe9f0009900 tx=0x7fe9f002bc40 comp rx=0 tx=0).stop 2024-03-28T18:10:44.864 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.864+0000 7fea0a000640 1 -- 172.21.15.22:0/3870273710 shutdown_connections 2024-03-28T18:10:44.864 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.864+0000 7fea0a000640 1 --2- 172.21.15.22:0/3870273710 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fea041124e0 0x7fea041148d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:10:44.864 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.864+0000 7fea0a000640 1 --2- 172.21.15.22:0/3870273710 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fea040764f0 0x7fea04111fa0 unknown :-1 s=CLOSED pgs=683 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:10:44.864 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.864+0000 7fea0a000640 1 --2- 172.21.15.22:0/3870273710 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fea04075bd0 0x7fea04075fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:10:44.864 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.864+0000 7fea0a000640 1 -- 172.21.15.22:0/3870273710 >> 172.21.15.22:0/3870273710 conn(0x7fea040ff2f0 msgr2=0x7fea04101710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:10:44.864 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.864+0000 7fea0a000640 1 -- 172.21.15.22:0/3870273710 shutdown_connections 2024-03-28T18:10:44.864 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.864+0000 7fea0a000640 1 -- 172.21.15.22:0/3870273710 wait complete. 2024-03-28T18:10:44.865 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.865+0000 7fea0a000640 1 Processor -- start 2024-03-28T18:10:44.865 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.865+0000 7fea0a000640 1 -- start start 2024-03-28T18:10:44.866 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.866+0000 7fea0a000640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fea04075bd0 0x7fea0419e600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:10:44.866 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.866+0000 7fea0a000640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fea040764f0 0x7fea0419eb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:10:44.866 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.866+0000 7fea0a000640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fea041124e0 0x7fea041a2ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:10:44.866 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.866+0000 7fea0a000640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea04118030 con 0x7fea04075bd0 2024-03-28T18:10:44.867 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.866+0000 7fea0a000640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea04117eb0 con 0x7fea040764f0 2024-03-28T18:10:44.867 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.866+0000 7fea0a000640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea041181b0 con 0x7fea041124e0 2024-03-28T18:10:44.867 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.866+0000 7fea03400640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fea04075bd0 0x7fea0419e600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:10:44.867 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.866+0000 7fea02a00640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fea040764f0 0x7fea0419eb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:10:44.867 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.866+0000 7fea03400640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fea04075bd0 0x7fea0419e600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:34670/0 (socket says 172.21.15.22:34670) 2024-03-28T18:10:44.867 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.866+0000 7fea03400640 1 -- 172.21.15.22:0/2922082353 learned_addr learned my addr 172.21.15.22:0/2922082353 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-28T18:10:44.867 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.866+0000 7fea03e00640 1 --2- 172.21.15.22:0/2922082353 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fea041124e0 0x7fea041a2ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:10:44.867 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.867+0000 7fea02a00640 1 -- 172.21.15.22:0/2922082353 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fea041124e0 msgr2=0x7fea041a2ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:10:44.867 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.867+0000 7fea02a00640 1 --2- 172.21.15.22:0/2922082353 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fea041124e0 0x7fea041a2ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:10:44.867 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.867+0000 7fea02a00640 1 -- 172.21.15.22:0/2922082353 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fea04075bd0 msgr2=0x7fea0419e600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:10:44.867 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.867+0000 7fea02a00640 1 --2- 172.21.15.22:0/2922082353 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fea04075bd0 0x7fea0419e600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:10:44.867 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.867+0000 7fea02a00640 1 -- 172.21.15.22:0/2922082353 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fea041a35f0 con 0x7fea040764f0 2024-03-28T18:10:44.868 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.867+0000 7fea02a00640 1 --2- 172.21.15.22:0/2922082353 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fea040764f0 0x7fea0419eb40 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fe9f0009230 tx=0x7fe9f002ca50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:10:44.868 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.868+0000 7fe9fbe00640 1 -- 172.21.15.22:0/2922082353 <== mon.2 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9f0043020 con 0x7fea040764f0 2024-03-28T18:10:44.868 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.868+0000 7fea0a000640 1 -- 172.21.15.22:0/2922082353 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fea041a3880 con 0x7fea040764f0 2024-03-28T18:10:44.868 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.868+0000 7fe9fbe00640 1 -- 172.21.15.22:0/2922082353 <== mon.2 v2:172.21.15.79:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7fe9f00054f0 con 0x7fea040764f0 2024-03-28T18:10:44.868 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.868+0000 7fe9fbe00640 1 -- 172.21.15.22:0/2922082353 <== mon.2 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9f00039f0 con 0x7fea040764f0 2024-03-28T18:10:44.868 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.868+0000 7fea0a000640 1 -- 172.21.15.22:0/2922082353 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fea041ab170 con 0x7fea040764f0 2024-03-28T18:10:44.870 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.871+0000 7fea0cda3740 1 Processor -- start 2024-03-28T18:10:44.871 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.871+0000 7fea0cda3740 1 -- start start 2024-03-28T18:10:44.872 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.871+0000 7fea0cda3740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ce83df16d0 0x55ce83df1ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:10:44.872 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.872+0000 7fea0cda3740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ce83df2080 0x55ce83dfa630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:10:44.872 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.872+0000 7fea0cda3740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55ce83dfac00 0x55ce83dfd050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:10:44.872 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.872+0000 7fea0cda3740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ce83d552c0 con 0x55ce83df16d0 2024-03-28T18:10:44.872 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.872+0000 7fea0cda3740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ce83cfdc90 con 0x55ce83dfac00 2024-03-28T18:10:44.872 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.872+0000 7fea0cda3740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ce83d51360 con 0x55ce83df2080 2024-03-28T18:10:44.872 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.872+0000 7fea03e00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ce83df16d0 0x55ce83df1ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:10:44.872 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.872+0000 7fea03e00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ce83df16d0 0x55ce83df1ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:34684/0 (socket says 172.21.15.22:34684) 2024-03-28T18:10:44.873 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.872+0000 7fea03e00640 1 -- 172.21.15.22:0/3714313342 learned_addr learned my addr 172.21.15.22:0/3714313342 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-28T18:10:44.873 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.872+0000 7fea03400640 1 --2- 172.21.15.22:0/3714313342 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ce83df2080 0x55ce83dfa630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:10:44.873 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.872+0000 7fea03e00640 1 -- 172.21.15.22:0/3714313342 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ce83df2080 msgr2=0x55ce83dfa630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T18:10:44.873 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.872+0000 7fea03e00640 1 --2- 172.21.15.22:0/3714313342 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ce83df2080 0x55ce83dfa630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:10:44.873 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.872+0000 7fea03e00640 1 -- 172.21.15.22:0/3714313342 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55ce83dfac00 msgr2=0x55ce83dfd050 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-28T18:10:44.873 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.872+0000 7fea03e00640 1 --2- 172.21.15.22:0/3714313342 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55ce83dfac00 0x55ce83dfd050 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:10:44.873 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.872+0000 7fea03e00640 1 -- 172.21.15.22:0/3714313342 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ce83d22080 con 0x55ce83df16d0 2024-03-28T18:10:44.873 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.873+0000 7fe9fbe00640 1 -- 172.21.15.22:0/2922082353 <== mon.2 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 26) v1 ==== 137421+0+0 (secure 0 0 0) 0x7fe9f0046020 con 0x7fea040764f0 2024-03-28T18:10:44.875 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.873+0000 7fea03e00640 1 --2- 172.21.15.22:0/3714313342 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ce83df16d0 0x55ce83df1ab0 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7fe9fc009f00 tx=0x7fe9fc0107a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=74ff51287856ef76 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:10:44.875 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.873+0000 7fe9efe00640 1 -- 172.21.15.22:0/3714313342 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9fc003b70 con 0x55ce83df16d0 2024-03-28T18:10:44.875 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.873+0000 7fe9efe00640 1 -- 172.21.15.22:0/3714313342 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7fe9fc003d10 con 0x55ce83df16d0 2024-03-28T18:10:44.875 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.873+0000 7fe9efe00640 1 -- 172.21.15.22:0/3714313342 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9fc011a00 con 0x55ce83df16d0 2024-03-28T18:10:44.875 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.874+0000 7fea0cda3740 1 -- 172.21.15.22:0/3714313342 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ce83df16d0 msgr2=0x55ce83df1ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:10:44.875 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.874+0000 7fea0cda3740 1 --2- 172.21.15.22:0/3714313342 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ce83df16d0 0x55ce83df1ab0 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7fe9fc009f00 tx=0x7fe9fc0107a0 comp rx=0 tx=0).stop 2024-03-28T18:10:44.875 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.874+0000 7fea0cda3740 1 -- 172.21.15.22:0/3714313342 shutdown_connections 2024-03-28T18:10:44.875 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.874+0000 7fea0cda3740 1 --2- 172.21.15.22:0/3714313342 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55ce83dfac00 0x55ce83dfd050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:10:44.875 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.874+0000 7fea0cda3740 1 --2- 172.21.15.22:0/3714313342 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ce83df2080 0x55ce83dfa630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:10:44.875 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.874+0000 7fea0cda3740 1 --2- 172.21.15.22:0/3714313342 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ce83df16d0 0x55ce83df1ab0 unknown :-1 s=CLOSED pgs=684 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:10:44.875 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.874+0000 7fea0cda3740 1 -- 172.21.15.22:0/3714313342 >> 172.21.15.22:0/3714313342 conn(0x55ce83debda0 msgr2=0x55ce83dec180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:10:44.875 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.874+0000 7fea0cda3740 1 -- 172.21.15.22:0/3714313342 shutdown_connections 2024-03-28T18:10:44.875 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.874+0000 7fea0cda3740 1 -- 172.21.15.22:0/3714313342 wait complete. 2024-03-28T18:10:44.876 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.874+0000 7fe9fbe00640 1 --2- 172.21.15.22:0/2922082353 >> [v2:172.21.15.22:6800/1357365082,v1:172.21.15.22:6801/1357365082] conn(0x7fe9e80a2880 0x7fe9e80a4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:10:44.876 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.875+0000 7fea0cda3740 1 -- start start 2024-03-28T18:10:44.876 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.875+0000 7fe9fbe00640 1 -- 172.21.15.22:0/2922082353 <== mon.2 v2:172.21.15.79:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 8583+0+0 (secure 0 0 0) 0x7fe9f00f07b0 con 0x7fea040764f0 2024-03-28T18:10:44.876 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.875+0000 7fea03e00640 1 --2- 172.21.15.22:0/2922082353 >> [v2:172.21.15.22:6800/1357365082,v1:172.21.15.22:6801/1357365082] conn(0x7fe9e80a2880 0x7fe9e80a4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:10:44.876 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.875+0000 7fea0cda3740 1 Processor -- start 2024-03-28T18:10:44.879 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.876+0000 7fea0cda3740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ce83df16d0 0x55ce83dfa510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:10:44.879 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.876+0000 7fea03e00640 1 --2- 172.21.15.22:0/2922082353 >> [v2:172.21.15.22:6800/1357365082,v1:172.21.15.22:6801/1357365082] conn(0x7fe9e80a2880 0x7fe9e80a4d40 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fe9fc0048b0 tx=0x7fe9fc013040 comp rx=0 tx=0).ready entity=mgr.15522 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:10:44.879 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.876+0000 7fea0cda3740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55ce83df2080 0x55ce83df78a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:10:44.879 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.876+0000 7fea0cda3740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ce83dfac00 0x55ce83df7e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:10:44.879 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.876+0000 7fea0cda3740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ce83dfdb70 con 0x55ce83df16d0 2024-03-28T18:10:44.879 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.876+0000 7fea0cda3740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ce83d110d0 con 0x55ce83df2080 2024-03-28T18:10:44.879 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.876+0000 7fea0cda3740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ce83dfdcf0 con 0x55ce83dfac00 2024-03-28T18:10:44.879 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.876+0000 7fea0cda3740 10 client.0 _ll_register_callbacks cb 0x55ce83d15e80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-28T18:10:44.879 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.876+0000 7fea03400640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ce83df16d0 0x55ce83dfa510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:10:44.879 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.876+0000 7fea03400640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ce83df16d0 0x55ce83dfa510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:34698/0 (socket says 172.21.15.22:34698) 2024-03-28T18:10:44.879 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.876+0000 7fea03400640 1 -- 172.21.15.22:0/1058278566 learned_addr learned my addr 172.21.15.22:0/1058278566 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-28T18:10:44.879 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.876+0000 7fea02a00640 1 --2- 172.21.15.22:0/1058278566 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55ce83df2080 0x55ce83df78a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:10:44.880 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.876+0000 7fea03e00640 1 --2- 172.21.15.22:0/1058278566 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ce83dfac00 0x55ce83df7e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:10:44.880 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.876+0000 7fea03400640 1 -- 172.21.15.22:0/1058278566 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ce83dfac00 msgr2=0x55ce83df7e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T18:10:44.880 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.876+0000 7fea03400640 1 --2- 172.21.15.22:0/1058278566 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ce83dfac00 0x55ce83df7e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:10:44.880 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.876+0000 7fea03400640 1 -- 172.21.15.22:0/1058278566 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55ce83df2080 msgr2=0x55ce83df78a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T18:10:44.880 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.876+0000 7fea03400640 1 --2- 172.21.15.22:0/1058278566 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55ce83df2080 0x55ce83df78a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:10:44.880 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.876+0000 7fea03400640 1 -- 172.21.15.22:0/1058278566 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55ce83a693f0 con 0x55ce83df16d0 2024-03-28T18:10:44.880 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.876+0000 7fea03e00640 1 --2- 172.21.15.22:0/1058278566 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ce83dfac00 0x55ce83df7e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T18:10:44.880 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.877+0000 7fea03400640 1 --2- 172.21.15.22:0/1058278566 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ce83df16d0 0x55ce83dfa510 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7fe9f400bee0 tx=0x7fe9f4012820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7a85bfd4dbd5387e server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:10:44.880 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.877+0000 7fe9ecc00640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-03-28T18:10:44.880 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.877+0000 7fe9ecc00640 1 -- 172.21.15.22:0/1058278566 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9f4010960 con 0x55ce83df16d0 2024-03-28T18:10:44.880 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.877+0000 7fe9ecc00640 1 -- 172.21.15.22:0/1058278566 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 8583+0+0 (secure 0 0 0) 0x7fe9f4010b00 con 0x55ce83df16d0 2024-03-28T18:10:44.880 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.877+0000 7fea0cda3740 1 -- 172.21.15.22:0/1058278566 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fe9fc003d10 con 0x55ce83df16d0 2024-03-28T18:10:44.880 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.877+0000 7fe9ecc00640 1 -- 172.21.15.22:0/1058278566 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe9f4010d40 con 0x55ce83df16d0 2024-03-28T18:10:44.880 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.877+0000 7fea0cda3740 10 client.16452 fetch_fsmap learned FSMap version 40 2024-03-28T18:10:44.880 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.877+0000 7fea0cda3740 1 -- 172.21.15.22:0/1058278566 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) v3 -- 0x55ce83b337d0 con 0x55ce83df16d0 2024-03-28T18:10:44.881 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.878+0000 7fe9ecc00640 1 -- 172.21.15.22:0/1058278566 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7fe9fc003d10 con 0x55ce83df16d0 2024-03-28T18:10:44.881 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.878+0000 7fe9ecc00640 1 -- 172.21.15.22:0/1058278566 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 40) v1 ==== 7954+0+0 (secure 0 0 0) 0x7fe9f4023410 con 0x55ce83df16d0 2024-03-28T18:10:44.881 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.878+0000 7fea0cda3740 10 client.16452 fetch_fsmap finished waiting for FSMap version 40 2024-03-28T18:10:44.881 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.878+0000 7fea0cda3740 10 client.16452 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.e 2024-03-28T18:10:44.881 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.878+0000 7fea0cda3740 20 client.16452 populate_metadata read hostname 'smithi022' 2024-03-28T18:10:44.881 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.878+0000 7fea0cda3740 1 --2- 172.21.15.22:0/1058278566 >> [v2:172.21.15.79:6834/3365773431,v1:172.21.15.79:6835/3365773431] conn(0x55ce83e0aab0 0x55ce83e0cea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:10:44.881 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.878+0000 7fea0cda3740 4 client.16452 mds_command: new command op to 34475 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-28T18:10:44.881 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.878+0000 7fea0cda3740 1 -- 172.21.15.22:0/1058278566 --> [v2:172.21.15.79:6834/3365773431,v1:172.21.15.79:6835/3365773431] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55ce83d62800 con 0x55ce83e0aab0 2024-03-28T18:10:44.881 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.879+0000 7fea02a00640 1 --2- 172.21.15.22:0/1058278566 >> [v2:172.21.15.79:6834/3365773431,v1:172.21.15.79:6835/3365773431] conn(0x55ce83e0aab0 0x55ce83e0cea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:10:44.881 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.879+0000 7fea02a00640 1 --2- 172.21.15.22:0/1058278566 >> [v2:172.21.15.79:6834/3365773431,v1:172.21.15.79:6835/3365773431] conn(0x55ce83e0aab0 0x55ce83e0cea0 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=5d851e73a711900f server_cookie=2e6d4571d6201be0 in_seq=0 out_seq=0 2024-03-28T18:10:44.881 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.879+0000 7fe9ecc00640 10 client.16452 ms_handle_connect on v2:172.21.15.79:6834/3365773431 2024-03-28T18:10:44.882 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.882+0000 7fe9ecc00640 1 -- 172.21.15.22:0/1058278566 <== mds.2 v2:172.21.15.79:6834/3365773431 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x55ce83d62800 con 0x55ce83e0aab0 2024-03-28T18:10:44.882 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.882+0000 7fe9ecc00640 10 client.16452 handle_command_reply: tid=0 2024-03-28T18:10:44.883 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.882+0000 7fea0cda3740 1 client.16452 shutdown 2024-03-28T18:10:44.883 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.882+0000 7fea0cda3740 1 -- 172.21.15.22:0/1058278566 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ce83df16d0 msgr2=0x55ce83dfa510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:10:44.883 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.882+0000 7fea0cda3740 1 --2- 172.21.15.22:0/1058278566 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ce83df16d0 0x55ce83dfa510 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7fe9f400bee0 tx=0x7fe9f4012820 comp rx=0 tx=0).stop 2024-03-28T18:10:44.883 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.882+0000 7fea0cda3740 1 -- 172.21.15.22:0/1058278566 shutdown_connections 2024-03-28T18:10:44.883 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.882+0000 7fea0cda3740 1 --2- 172.21.15.22:0/1058278566 >> [v2:172.21.15.79:6834/3365773431,v1:172.21.15.79:6835/3365773431] conn(0x55ce83e0aab0 0x55ce83e0cea0 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:10:44.883 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.882+0000 7fea0cda3740 1 --2- 172.21.15.22:0/1058278566 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ce83dfac00 0x55ce83df7e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:10:44.883 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.882+0000 7fea0cda3740 1 --2- 172.21.15.22:0/1058278566 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55ce83df2080 0x55ce83df78a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:10:44.883 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.882+0000 7fea0cda3740 1 --2- 172.21.15.22:0/1058278566 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ce83df16d0 0x55ce83dfa510 unknown :-1 s=CLOSED pgs=685 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:10:44.883 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.882+0000 7fe9ecc00640 0 client.16452 ms_handle_reset on v2:172.21.15.79:6834/3365773431 2024-03-28T18:10:44.883 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.882+0000 7fea0cda3740 1 -- 172.21.15.22:0/1058278566 >> 172.21.15.22:0/1058278566 conn(0x55ce83debda0 msgr2=0x55ce83dedaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:10:44.883 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.882+0000 7fea0cda3740 1 -- 172.21.15.22:0/1058278566 shutdown_connections 2024-03-28T18:10:44.883 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.882+0000 7fea0cda3740 1 -- 172.21.15.22:0/1058278566 wait complete. 2024-03-28T18:10:44.883 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.882+0000 7fea0cda3740 20 client.16452 trim_cache size 0 max 16384 2024-03-28T18:10:44.896 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.896+0000 7fea0cda3740 1 Processor -- start 2024-03-28T18:10:44.896 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.896+0000 7fea0cda3740 1 -- start start 2024-03-28T18:10:44.896 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.896+0000 7fea0cda3740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ce83df16d0 0x55ce83df9fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:10:44.896 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.896+0000 7fea0cda3740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55ce83df2080 0x55ce83dfa510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:10:44.896 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.896+0000 7fea0cda3740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ce83dfac00 0x55ce83dead30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:10:44.896 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.896+0000 7fea0cda3740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea041b3da0 con 0x55ce83df16d0 2024-03-28T18:10:44.896 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.896+0000 7fea0cda3740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea041b5180 con 0x55ce83df2080 2024-03-28T18:10:44.896 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.896+0000 7fea0cda3740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea041b4400 con 0x55ce83dfac00 2024-03-28T18:10:44.898 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.897+0000 7fea03400640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ce83df16d0 0x55ce83df9fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:10:44.898 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.897+0000 7fea03400640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ce83df16d0 0x55ce83df9fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:34704/0 (socket says 172.21.15.22:34704) 2024-03-28T18:10:44.898 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.897+0000 7fea03400640 1 -- 172.21.15.22:0/549870117 learned_addr learned my addr 172.21.15.22:0/549870117 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-28T18:10:44.898 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.897+0000 7fea02a00640 1 --2- 172.21.15.22:0/549870117 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55ce83df2080 0x55ce83dfa510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:10:44.898 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.897+0000 7fea03e00640 1 --2- 172.21.15.22:0/549870117 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ce83dfac00 0x55ce83dead30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:10:44.898 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.897+0000 7fea03400640 1 -- 172.21.15.22:0/549870117 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ce83dfac00 msgr2=0x55ce83dead30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T18:10:44.898 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.897+0000 7fea03400640 1 --2- 172.21.15.22:0/549870117 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ce83dfac00 0x55ce83dead30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:10:44.899 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.897+0000 7fea03400640 1 -- 172.21.15.22:0/549870117 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55ce83df2080 msgr2=0x55ce83dfa510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T18:10:44.899 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.897+0000 7fea03400640 1 --2- 172.21.15.22:0/549870117 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55ce83df2080 0x55ce83dfa510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:10:44.899 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.897+0000 7fea03400640 1 -- 172.21.15.22:0/549870117 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ce83df8190 con 0x55ce83df16d0 2024-03-28T18:10:44.899 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.897+0000 7fea02a00640 1 --2- 172.21.15.22:0/549870117 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55ce83df2080 0x55ce83dfa510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-28T18:10:44.899 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.897+0000 7fea03400640 1 --2- 172.21.15.22:0/549870117 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ce83df16d0 0x55ce83df9fd0 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7fe9f40238d0 tx=0x7fe9f4005bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9bc35031f882781 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:10:44.899 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.898+0000 7fe9ef400640 1 -- 172.21.15.22:0/549870117 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9f4000e60 con 0x55ce83df16d0 2024-03-28T18:10:44.899 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.898+0000 7fe9ef400640 1 -- 172.21.15.22:0/549870117 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7fe9f4032070 con 0x55ce83df16d0 2024-03-28T18:10:44.899 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.898+0000 7fe9ef400640 1 -- 172.21.15.22:0/549870117 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9f4004a50 con 0x55ce83df16d0 2024-03-28T18:10:44.899 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.898+0000 7fea0cda3740 1 -- 172.21.15.22:0/549870117 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ce83df16d0 msgr2=0x55ce83df9fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:10:44.899 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.898+0000 7fea0cda3740 1 --2- 172.21.15.22:0/549870117 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ce83df16d0 0x55ce83df9fd0 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7fe9f40238d0 tx=0x7fe9f4005bf0 comp rx=0 tx=0).stop 2024-03-28T18:10:44.899 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.898+0000 7fea0cda3740 1 -- 172.21.15.22:0/549870117 shutdown_connections 2024-03-28T18:10:44.899 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.898+0000 7fea0cda3740 1 --2- 172.21.15.22:0/549870117 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ce83dfac00 0x55ce83dead30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:10:44.899 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.898+0000 7fea0cda3740 1 --2- 172.21.15.22:0/549870117 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55ce83df2080 0x55ce83dfa510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:10:44.899 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.898+0000 7fea0cda3740 1 --2- 172.21.15.22:0/549870117 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ce83df16d0 0x55ce83df9fd0 unknown :-1 s=CLOSED pgs=686 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:10:44.899 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.898+0000 7fea0cda3740 1 -- 172.21.15.22:0/549870117 >> 172.21.15.22:0/549870117 conn(0x55ce83debda0 msgr2=0x55ce83dea620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:10:44.899 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.899+0000 7fea0cda3740 1 -- 172.21.15.22:0/549870117 shutdown_connections 2024-03-28T18:10:44.900 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.899+0000 7fea0cda3740 1 -- 172.21.15.22:0/549870117 wait complete. 2024-03-28T18:10:44.902 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.899+0000 7fea0cda3740 1 -- start start 2024-03-28T18:10:44.902 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.899+0000 7fea0cda3740 1 Processor -- start 2024-03-28T18:10:44.902 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.900+0000 7fea0cda3740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55ce83df16d0 0x55ce83e02630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:10:44.902 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.900+0000 7fea0cda3740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ce83df2080 0x55ce83e02b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:10:44.902 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.900+0000 7fea0cda3740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ce83dfac00 0x55ce83dfeab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:10:44.902 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.900+0000 7fea0cda3740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea041b2520 con 0x55ce83dfac00 2024-03-28T18:10:44.903 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.900+0000 7fea0cda3740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea041b4b20 con 0x55ce83df16d0 2024-03-28T18:10:44.903 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.900+0000 7fea0cda3740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea041b1900 con 0x55ce83df2080 2024-03-28T18:10:44.903 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.900+0000 7fea0cda3740 10 client.0 _ll_register_callbacks cb 0x7fe9f400b810 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-28T18:10:44.903 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.900+0000 7fea03e00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ce83dfac00 0x55ce83dfeab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:10:44.903 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.900+0000 7fea03e00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ce83dfac00 0x55ce83dfeab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:34718/0 (socket says 172.21.15.22:34718) 2024-03-28T18:10:44.903 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.900+0000 7fea03e00640 1 -- 172.21.15.22:0/1226845718 learned_addr learned my addr 172.21.15.22:0/1226845718 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-28T18:10:44.903 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.900+0000 7fea03400640 1 --2- 172.21.15.22:0/1226845718 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55ce83df16d0 0x55ce83e02630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:10:44.903 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.900+0000 7fea02a00640 1 --2- 172.21.15.22:0/1226845718 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ce83df2080 0x55ce83e02b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:10:44.903 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.900+0000 7fea03e00640 1 -- 172.21.15.22:0/1226845718 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ce83df2080 msgr2=0x55ce83e02b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T18:10:44.903 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.900+0000 7fea03e00640 1 --2- 172.21.15.22:0/1226845718 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ce83df2080 0x55ce83e02b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:10:44.903 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.900+0000 7fea03e00640 1 -- 172.21.15.22:0/1226845718 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55ce83df16d0 msgr2=0x55ce83e02630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T18:10:44.903 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.900+0000 7fea03e00640 1 --2- 172.21.15.22:0/1226845718 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55ce83df16d0 0x55ce83e02630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:10:44.903 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.900+0000 7fea03e00640 1 -- 172.21.15.22:0/1226845718 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55ce83df1e20 con 0x55ce83dfac00 2024-03-28T18:10:44.904 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.901+0000 7fea03e00640 1 --2- 172.21.15.22:0/1226845718 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ce83dfac00 0x55ce83dfeab0 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7fe9fc005d70 tx=0x7fe9fc009500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=87362546cabc301d server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:10:44.904 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.901+0000 7fe9ecc00640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-03-28T18:10:44.904 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.901+0000 7fe9ecc00640 1 -- 172.21.15.22:0/1226845718 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9fc038060 con 0x55ce83dfac00 2024-03-28T18:10:44.904 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.901+0000 7fe9ecc00640 1 -- 172.21.15.22:0/1226845718 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 8583+0+0 (secure 0 0 0) 0x7fe9fc023030 con 0x55ce83dfac00 2024-03-28T18:10:44.904 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.901+0000 7fea0cda3740 1 -- 172.21.15.22:0/1226845718 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fe9f4032070 con 0x55ce83dfac00 2024-03-28T18:10:44.904 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.901+0000 7fe9ecc00640 1 -- 172.21.15.22:0/1226845718 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe9fc0116f0 con 0x55ce83dfac00 2024-03-28T18:10:44.904 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.901+0000 7fea0cda3740 10 client.16458 fetch_fsmap learned FSMap version 40 2024-03-28T18:10:44.904 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.901+0000 7fea0cda3740 1 -- 172.21.15.22:0/1226845718 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) v3 -- 0x55ce83e66be0 con 0x55ce83dfac00 2024-03-28T18:10:44.904 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.902+0000 7fe9ecc00640 1 -- 172.21.15.22:0/1226845718 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7fe9f4032070 con 0x55ce83dfac00 2024-03-28T18:10:44.904 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.902+0000 7fe9ecc00640 1 -- 172.21.15.22:0/1226845718 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 40) v1 ==== 7954+0+0 (secure 0 0 0) 0x7fe9fc005420 con 0x55ce83dfac00 2024-03-28T18:10:44.904 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.902+0000 7fea0cda3740 10 client.16458 fetch_fsmap finished waiting for FSMap version 40 2024-03-28T18:10:44.904 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.902+0000 7fea0cda3740 10 client.16458 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.e 2024-03-28T18:10:44.904 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.902+0000 7fea0cda3740 20 client.16458 populate_metadata read hostname 'smithi022' 2024-03-28T18:10:44.905 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.902+0000 7fea0cda3740 1 --2- 172.21.15.22:0/1226845718 >> [v2:172.21.15.79:6834/3365773431,v1:172.21.15.79:6835/3365773431] conn(0x55ce83e670c0 0x55ce83e694b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:10:44.905 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.902+0000 7fea0cda3740 4 client.16458 mds_command: new command op to 34475 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-28T18:10:44.905 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.902+0000 7fea0cda3740 1 -- 172.21.15.22:0/1226845718 --> [v2:172.21.15.79:6834/3365773431,v1:172.21.15.79:6835/3365773431] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55ce83d5cb50 con 0x55ce83e670c0 2024-03-28T18:10:44.905 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.902+0000 7fea03400640 1 --2- 172.21.15.22:0/1226845718 >> [v2:172.21.15.79:6834/3365773431,v1:172.21.15.79:6835/3365773431] conn(0x55ce83e670c0 0x55ce83e694b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:10:44.906 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.903+0000 7fea03400640 1 --2- 172.21.15.22:0/1226845718 >> [v2:172.21.15.79:6834/3365773431,v1:172.21.15.79:6835/3365773431] conn(0x55ce83e670c0 0x55ce83e694b0 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=7102e9f0288f2b8a server_cookie=5826865ab1e4c853 in_seq=0 out_seq=0 2024-03-28T18:10:44.906 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.903+0000 7fe9ecc00640 10 client.16458 ms_handle_connect on v2:172.21.15.79:6834/3365773431 2024-03-28T18:10:44.906 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.904+0000 7fe9ecc00640 1 -- 172.21.15.22:0/1226845718 <== mds.2 v2:172.21.15.79:6834/3365773431 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55ce83d5cb50 con 0x55ce83e670c0 2024-03-28T18:10:44.906 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.904+0000 7fe9ecc00640 10 client.16458 handle_command_reply: tid=0 2024-03-28T18:10:44.906 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.904+0000 7fea0cda3740 1 client.16458 shutdown 2024-03-28T18:10:44.906 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.904+0000 7fea0cda3740 1 -- 172.21.15.22:0/1226845718 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ce83dfac00 msgr2=0x55ce83dfeab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:10:44.906 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.904+0000 7fea0cda3740 1 --2- 172.21.15.22:0/1226845718 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ce83dfac00 0x55ce83dfeab0 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7fe9fc005d70 tx=0x7fe9fc009500 comp rx=0 tx=0).stop 2024-03-28T18:10:44.906 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.904+0000 7fea0cda3740 1 -- 172.21.15.22:0/1226845718 shutdown_connections 2024-03-28T18:10:44.906 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.904+0000 7fea0cda3740 1 --2- 172.21.15.22:0/1226845718 >> [v2:172.21.15.79:6834/3365773431,v1:172.21.15.79:6835/3365773431] conn(0x55ce83e670c0 0x55ce83e694b0 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:10:44.906 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.904+0000 7fea0cda3740 1 --2- 172.21.15.22:0/1226845718 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ce83dfac00 0x55ce83dfeab0 unknown :-1 s=CLOSED pgs=687 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:10:44.907 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.904+0000 7fea0cda3740 1 --2- 172.21.15.22:0/1226845718 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ce83df2080 0x55ce83e02b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:10:44.907 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.904+0000 7fea0cda3740 1 --2- 172.21.15.22:0/1226845718 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55ce83df16d0 0x55ce83e02630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:10:44.907 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.904+0000 7fe9ecc00640 0 client.16458 ms_handle_reset on v2:172.21.15.79:6834/3365773431 2024-03-28T18:10:44.907 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.904+0000 7fea0cda3740 1 -- 172.21.15.22:0/1226845718 >> 172.21.15.22:0/1226845718 conn(0x55ce83debda0 msgr2=0x55ce83ded910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:10:44.907 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.904+0000 7fea0cda3740 1 -- 172.21.15.22:0/1226845718 shutdown_connections 2024-03-28T18:10:44.907 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.904+0000 7fea0cda3740 1 -- 172.21.15.22:0/1226845718 wait complete. 2024-03-28T18:10:44.907 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.904+0000 7fea0cda3740 20 client.16458 trim_cache size 0 max 16384 2024-03-28T18:10:44.907 INFO:teuthology.orchestra.run.smithi022.stdout:[] 2024-03-28T18:10:44.908 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.905+0000 7fea0a000640 1 -- 172.21.15.22:0/2922082353 >> [v2:172.21.15.22:6800/1357365082,v1:172.21.15.22:6801/1357365082] conn(0x7fe9e80a2880 msgr2=0x7fe9e80a4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:10:44.908 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.905+0000 7fea0a000640 1 --2- 172.21.15.22:0/2922082353 >> [v2:172.21.15.22:6800/1357365082,v1:172.21.15.22:6801/1357365082] conn(0x7fe9e80a2880 0x7fe9e80a4d40 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fe9fc0048b0 tx=0x7fe9fc013040 comp rx=0 tx=0).stop 2024-03-28T18:10:44.908 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.905+0000 7fea0a000640 1 -- 172.21.15.22:0/2922082353 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fea040764f0 msgr2=0x7fea0419eb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:10:44.908 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.905+0000 7fea0a000640 1 --2- 172.21.15.22:0/2922082353 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fea040764f0 0x7fea0419eb40 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fe9f0009230 tx=0x7fe9f002ca50 comp rx=0 tx=0).stop 2024-03-28T18:10:44.908 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.905+0000 7fea0a000640 1 -- 172.21.15.22:0/2922082353 shutdown_connections 2024-03-28T18:10:44.908 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.905+0000 7fea0a000640 1 --2- 172.21.15.22:0/2922082353 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fea041124e0 0x7fea041a2ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:10:44.908 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.905+0000 7fea0a000640 1 --2- 172.21.15.22:0/2922082353 >> [v2:172.21.15.22:6800/1357365082,v1:172.21.15.22:6801/1357365082] conn(0x7fe9e80a2880 0x7fe9e80a4d40 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:10:44.909 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.905+0000 7fea0a000640 1 --2- 172.21.15.22:0/2922082353 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fea040764f0 0x7fea0419eb40 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:10:44.909 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.905+0000 7fea0a000640 1 --2- 172.21.15.22:0/2922082353 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fea04075bd0 0x7fea0419e600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:10:44.909 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.905+0000 7fea0a000640 1 -- 172.21.15.22:0/2922082353 >> 172.21.15.22:0/2922082353 conn(0x7fea040ff2f0 msgr2=0x7fea041016e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:10:44.909 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.905+0000 7fea0a000640 1 -- 172.21.15.22:0/2922082353 shutdown_connections 2024-03-28T18:10:44.909 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:44.905+0000 7fea0a000640 1 -- 172.21.15.22:0/2922082353 wait complete. 2024-03-28T18:10:44.923 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-28T18:10:44.923 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-03-28T18:10:45.155 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:45.155+0000 7f174bc00640 1 -- 172.21.15.22:0/1962777825 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1744101ff0 msgr2=0x7f174410e500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:10:45.155 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:45.155+0000 7f174bc00640 1 --2- 172.21.15.22:0/1962777825 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1744101ff0 0x7f174410e500 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7f173c009cd0 tx=0x7f173c02c0f0 comp rx=0 tx=0).stop 2024-03-28T18:10:45.155 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:45.155+0000 7f174bc00640 1 -- 172.21.15.22:0/1962777825 shutdown_connections 2024-03-28T18:10:45.156 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:45.155+0000 7f174bc00640 1 --2- 172.21.15.22:0/1962777825 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1744101ff0 0x7f174410e500 unknown :-1 s=CLOSED pgs=688 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:10:45.156 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:45.155+0000 7f174bc00640 1 --2- 172.21.15.22:0/1962777825 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f1744101630 0x7f1744101ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:10:45.156 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:45.155+0000 7f174bc00640 1 --2- 172.21.15.22:0/1962777825 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1744107640 0x7f1744107a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:10:45.156 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:45.155+0000 7f174bc00640 1 -- 172.21.15.22:0/1962777825 >> 172.21.15.22:0/1962777825 conn(0x7f1744078aa0 msgr2=0x7f1744078eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:10:45.156 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:45.155+0000 7f174bc00640 1 -- 172.21.15.22:0/1962777825 shutdown_connections 2024-03-28T18:10:45.156 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:45.155+0000 7f174bc00640 1 -- 172.21.15.22:0/1962777825 wait complete. 2024-03-28T18:10:45.158 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:45.156+0000 7f174bc00640 1 Processor -- start 2024-03-28T18:10:45.158 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:45.156+0000 7f174bc00640 1 -- start start 2024-03-28T18:10:45.158 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:45.157+0000 7f174bc00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1744101630 0x7f174419a0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:10:45.158 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:45.157+0000 7f174bc00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1744101ff0 0x7f174419a600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:10:45.158 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:45.157+0000 7f174bc00640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f1744107640 0x7f174419e980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:10:45.158 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:45.157+0000 7f174bc00640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17441119e0 con 0x7f1744101ff0 2024-03-28T18:10:45.158 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:45.157+0000 7f174bc00640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1744111860 con 0x7f1744107640 2024-03-28T18:10:45.158 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:45.157+0000 7f174bc00640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1744111b60 con 0x7f1744101630 2024-03-28T18:10:45.158 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:45.157+0000 7f1743e00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1744101ff0 0x7f174419a600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:10:45.159 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:45.157+0000 7f1743e00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1744101ff0 0x7f174419a600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:34736/0 (socket says 172.21.15.22:34736) 2024-03-28T18:10:45.159 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:45.157+0000 7f1743e00640 1 -- 172.21.15.22:0/2840604647 learned_addr learned my addr 172.21.15.22:0/2840604647 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-28T18:10:45.159 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:45.157+0000 7f1749c00640 1 --2- 172.21.15.22:0/2840604647 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f1744107640 0x7f174419e980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:10:45.159 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:45.157+0000 7f1749200640 1 --2- 172.21.15.22:0/2840604647 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1744101630 0x7f174419a0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:10:45.159 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:45.157+0000 7f1743e00640 1 -- 172.21.15.22:0/2840604647 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1744101630 msgr2=0x7f174419a0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:10:45.159 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:45.157+0000 7f1743e00640 1 --2- 172.21.15.22:0/2840604647 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1744101630 0x7f174419a0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:10:45.159 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:45.157+0000 7f1743e00640 1 -- 172.21.15.22:0/2840604647 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f1744107640 msgr2=0x7f174419e980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:10:45.159 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:45.157+0000 7f1743e00640 1 --2- 172.21.15.22:0/2840604647 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f1744107640 0x7f174419e980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:10:45.159 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:45.157+0000 7f1743e00640 1 -- 172.21.15.22:0/2840604647 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f174419f0b0 con 0x7f1744101ff0 2024-03-28T18:10:45.159 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:45.158+0000 7f1743e00640 1 --2- 172.21.15.22:0/2840604647 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1744101ff0 0x7f174419a600 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7f173000e9f0 tx=0x7f173000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:10:45.159 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:45.158+0000 7f1741600640 1 -- 172.21.15.22:0/2840604647 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f173000cd80 con 0x7f1744101ff0 2024-03-28T18:10:45.159 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:45.158+0000 7f174bc00640 1 -- 172.21.15.22:0/2840604647 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f174419f3a0 con 0x7f1744101ff0 2024-03-28T18:10:45.159 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:45.158+0000 7f1741600640 1 -- 172.21.15.22:0/2840604647 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f1730010900 con 0x7f1744101ff0 2024-03-28T18:10:45.159 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:45.158+0000 7f1741600640 1 -- 172.21.15.22:0/2840604647 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17300189e0 con 0x7f1744101ff0 2024-03-28T18:10:45.159 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:45.158+0000 7f174bc00640 1 -- 172.21.15.22:0/2840604647 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f17441a6c30 con 0x7f1744101ff0 2024-03-28T18:10:45.161 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:45.161+0000 7f174e9a3740 1 Processor -- start 2024-03-28T18:10:45.161 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:45.162+0000 7f174e9a3740 1 -- start start 2024-03-28T18:10:45.163 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:45.162+0000 7f1741600640 1 -- 172.21.15.22:0/2840604647 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 26) v1 ==== 137421+0+0 (secure 0 0 0) 0x7f1730010ab0 con 0x7f1744101ff0 2024-03-28T18:10:45.163 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:45.162+0000 7f174e9a3740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5563e38796d0 0x5563e3879ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:10:45.163 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:45.162+0000 7f174e9a3740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5563e387a080 0x5563e3882630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:10:45.163 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:45.162+0000 7f174e9a3740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5563e3882c00 0x5563e3885050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:10:45.163 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:45.162+0000 7f174e9a3740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x5563e37dd2c0 con 0x5563e387a080 2024-03-28T18:10:45.163 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:45.162+0000 7f174e9a3740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x5563e3785c90 con 0x5563e38796d0 2024-03-28T18:10:45.163 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:45.162+0000 7f174e9a3740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x5563e37d9360 con 0x5563e3882c00 2024-03-28T18:10:45.163 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:45.162+0000 7f1749200640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5563e387a080 0x5563e3882630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:10:45.163 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:45.162+0000 7f1749c00640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5563e38796d0 0x5563e3879ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:10:45.163 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:45.162+0000 7f1749200640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5563e387a080 0x5563e3882630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:34738/0 (socket says 172.21.15.22:34738) 2024-03-28T18:10:45.163 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:45.162+0000 7f1749200640 1 -- 172.21.15.22:0/2129288 learned_addr learned my addr 172.21.15.22:0/2129288 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-28T18:10:45.164 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:45.163+0000 7f1743e00640 1 --2- 172.21.15.22:0/2129288 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5563e3882c00 0x5563e3885050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:10:45.164 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:45.163+0000 7f1749200640 1 -- 172.21.15.22:0/2129288 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5563e3882c00 msgr2=0x5563e3885050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T18:10:45.164 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:45.163+0000 7f1749200640 1 --2- 172.21.15.22:0/2129288 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5563e3882c00 0x5563e3885050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:10:45.164 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:45.163+0000 7f1749200640 1 -- 172.21.15.22:0/2129288 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5563e38796d0 msgr2=0x5563e3879ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T18:10:45.164 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:45.163+0000 7f1749200640 1 --2- 172.21.15.22:0/2129288 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5563e38796d0 0x5563e3879ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:10:45.164 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:45.163+0000 7f1749200640 1 -- 172.21.15.22:0/2129288 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5563e37aa080 con 0x5563e387a080 2024-03-28T18:10:45.164 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:45.163+0000 7f1749c00640 1 --2- 172.21.15.22:0/2129288 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5563e38796d0 0x5563e3879ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-28T18:10:45.164 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:45.163+0000 7f1749200640 1 --2- 172.21.15.22:0/2129288 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5563e387a080 0x5563e3882630 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7f173800bc60 tx=0x7f173800f730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b9857a7bee45a755 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:10:45.167 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:45.164+0000 7f1736000640 1 -- 172.21.15.22:0/2129288 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1738015070 con 0x5563e387a080 2024-03-28T18:10:45.167 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:45.164+0000 7f1736000640 1 -- 172.21.15.22:0/2129288 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f1738011910 con 0x5563e387a080 2024-03-28T18:10:45.167 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:45.164+0000 7f1741600640 1 --2- 172.21.15.22:0/2840604647 >> [v2:172.21.15.22:6800/1357365082,v1:172.21.15.22:6801/1357365082] conn(0x7f172c0a29f0 0x7f172c0a4eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:10:45.167 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:45.164+0000 7f1736000640 1 -- 172.21.15.22:0/2129288 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17380108b0 con 0x5563e387a080 2024-03-28T18:10:45.167 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:45.164+0000 7f1741600640 1 -- 172.21.15.22:0/2840604647 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 8583+0+0 (secure 0 0 0) 0x7f17300cf2a0 con 0x7f1744101ff0 2024-03-28T18:10:45.167 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:45.164+0000 7f1749c00640 1 --2- 172.21.15.22:0/2840604647 >> [v2:172.21.15.22:6800/1357365082,v1:172.21.15.22:6801/1357365082] conn(0x7f172c0a29f0 0x7f172c0a4eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:10:45.167 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:45.164+0000 7f174e9a3740 1 -- 172.21.15.22:0/2129288 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5563e387a080 msgr2=0x5563e3882630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:10:45.168 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:45.164+0000 7f174e9a3740 1 --2- 172.21.15.22:0/2129288 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5563e387a080 0x5563e3882630 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7f173800bc60 tx=0x7f173800f730 comp rx=0 tx=0).stop 2024-03-28T18:10:45.168 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:45.165+0000 7f174e9a3740 1 -- 172.21.15.22:0/2129288 shutdown_connections 2024-03-28T18:10:45.168 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:45.165+0000 7f174e9a3740 1 --2- 172.21.15.22:0/2129288 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5563e3882c00 0x5563e3885050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:10:45.168 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:45.165+0000 7f174e9a3740 1 --2- 172.21.15.22:0/2129288 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5563e387a080 0x5563e3882630 unknown :-1 s=CLOSED pgs=690 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:10:45.168 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:45.165+0000 7f174e9a3740 1 --2- 172.21.15.22:0/2129288 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5563e38796d0 0x5563e3879ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:10:45.168 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:45.165+0000 7f174e9a3740 1 -- 172.21.15.22:0/2129288 >> 172.21.15.22:0/2129288 conn(0x5563e3873da0 msgr2=0x5563e3874180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:10:45.168 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:45.165+0000 7f174e9a3740 1 -- 172.21.15.22:0/2129288 shutdown_connections 2024-03-28T18:10:45.168 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:45.165+0000 7f1749c00640 1 --2- 172.21.15.22:0/2840604647 >> [v2:172.21.15.22:6800/1357365082,v1:172.21.15.22:6801/1357365082] conn(0x7f172c0a29f0 0x7f172c0a4eb0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f173c009bf0 tx=0x7f173c004a40 comp rx=0 tx=0).ready entity=mgr.15522 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:10:45.168 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:45.165+0000 7f174e9a3740 1 -- 172.21.15.22:0/2129288 wait complete. 2024-03-28T18:10:45.168 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:45.165+0000 7f174e9a3740 1 -- start start 2024-03-28T18:10:45.168 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:45.166+0000 7f174e9a3740 1 Processor -- start 2024-03-28T18:10:45.168 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:45.166+0000 7f174e9a3740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5563e38796d0 0x5563e3882500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:10:45.168 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:45.166+0000 7f174e9a3740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5563e387a080 0x5563e387f890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:10:45.168 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:45.166+0000 7f174e9a3740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5563e3882c00 0x5563e387fe10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:10:45.169 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:45.166+0000 7f174e9a3740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x5563e3885b70 con 0x5563e387a080 2024-03-28T18:10:45.169 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:45.166+0000 7f174e9a3740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x5563e37990d0 con 0x5563e3882c00 2024-03-28T18:10:45.169 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:45.166+0000 7f174e9a3740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x5563e3885cf0 con 0x5563e38796d0 2024-03-28T18:10:45.169 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:45.166+0000 7f174e9a3740 10 client.0 _ll_register_callbacks cb 0x5563e379de80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-28T18:10:45.169 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:45.166+0000 7f1743e00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5563e387a080 0x5563e387f890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:10:45.169 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:45.166+0000 7f1743e00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5563e387a080 0x5563e387f890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:34750/0 (socket says 172.21.15.22:34750) 2024-03-28T18:10:45.169 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:45.166+0000 7f1743e00640 1 -- 172.21.15.22:0/3413813302 learned_addr learned my addr 172.21.15.22:0/3413813302 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-28T18:10:45.169 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:45.166+0000 7f1749c00640 1 --2- 172.21.15.22:0/3413813302 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5563e3882c00 0x5563e387fe10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:10:45.169 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:45.167+0000 7f1749200640 1 --2- 172.21.15.22:0/3413813302 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5563e38796d0 0x5563e3882500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:10:45.169 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:45.167+0000 7f1743e00640 1 -- 172.21.15.22:0/3413813302 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5563e38796d0 msgr2=0x5563e3882500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T18:10:45.169 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:45.167+0000 7f1743e00640 1 --2- 172.21.15.22:0/3413813302 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5563e38796d0 0x5563e3882500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:10:45.169 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:45.167+0000 7f1743e00640 1 -- 172.21.15.22:0/3413813302 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5563e3882c00 msgr2=0x5563e387fe10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T18:10:45.169 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:45.167+0000 7f1743e00640 1 --2- 172.21.15.22:0/3413813302 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5563e3882c00 0x5563e387fe10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:10:45.169 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:45.167+0000 7f1743e00640 1 -- 172.21.15.22:0/3413813302 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5563e34f13f0 con 0x5563e387a080 2024-03-28T18:10:45.169 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:45.167+0000 7f1749c00640 1 --2- 172.21.15.22:0/3413813302 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5563e3882c00 0x5563e387fe10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-28T18:10:45.170 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:45.167+0000 7f1743e00640 1 --2- 172.21.15.22:0/3413813302 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5563e387a080 0x5563e387f890 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7f1730000c00 tx=0x7f1730097c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8c13ee56b0bb83bb server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:10:45.170 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:45.167+0000 7f172aa00640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-03-28T18:10:45.170 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:45.167+0000 7f172aa00640 1 -- 172.21.15.22:0/3413813302 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f173009bfc0 con 0x5563e387a080 2024-03-28T18:10:45.170 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:45.167+0000 7f174e9a3740 1 -- 172.21.15.22:0/3413813302 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f1738011910 con 0x5563e387a080 2024-03-28T18:10:45.170 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:45.167+0000 7f172aa00640 1 -- 172.21.15.22:0/3413813302 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 8583+0+0 (secure 0 0 0) 0x7f17300860b0 con 0x5563e387a080 2024-03-28T18:10:45.170 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:45.168+0000 7f172aa00640 1 -- 172.21.15.22:0/3413813302 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7f17300b4050 con 0x5563e387a080 2024-03-28T18:10:45.170 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:45.168+0000 7f174e9a3740 10 client.16470 fetch_fsmap learned FSMap version 40 2024-03-28T18:10:45.170 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:45.168+0000 7f174e9a3740 1 -- 172.21.15.22:0/3413813302 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) v3 -- 0x5563e35bb7d0 con 0x5563e387a080 2024-03-28T18:10:45.172 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:45.168+0000 7f172aa00640 1 -- 172.21.15.22:0/3413813302 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f1738011910 con 0x5563e387a080 2024-03-28T18:10:45.172 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:45.168+0000 7f172aa00640 1 -- 172.21.15.22:0/3413813302 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 40) v1 ==== 7954+0+0 (secure 0 0 0) 0x7f17300d2c40 con 0x5563e387a080 2024-03-28T18:10:45.172 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:45.168+0000 7f174e9a3740 10 client.16470 fetch_fsmap finished waiting for FSMap version 40 2024-03-28T18:10:45.172 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:45.168+0000 7f174e9a3740 10 client.16470 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-28T18:10:45.172 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:45.168+0000 7f174e9a3740 20 client.16470 populate_metadata read hostname 'smithi022' 2024-03-28T18:10:45.172 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:45.168+0000 7f174e9a3740 1 --2- 172.21.15.22:0/3413813302 >> [v2:172.21.15.79:6838/2441542443,v1:172.21.15.79:6839/2441542443] conn(0x5563e3892a70 0x5563e3894e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:10:45.172 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:45.168+0000 7f174e9a3740 4 client.16470 mds_command: new command op to 34553 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-28T18:10:45.172 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:45.168+0000 7f174e9a3740 1 -- 172.21.15.22:0/3413813302 --> [v2:172.21.15.79:6838/2441542443,v1:172.21.15.79:6839/2441542443] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5563e37ea800 con 0x5563e3892a70 2024-03-28T18:10:45.172 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:45.169+0000 7f1749200640 1 --2- 172.21.15.22:0/3413813302 >> [v2:172.21.15.79:6838/2441542443,v1:172.21.15.79:6839/2441542443] conn(0x5563e3892a70 0x5563e3894e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:10:45.173 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:45.170+0000 7f1749200640 1 --2- 172.21.15.22:0/3413813302 >> [v2:172.21.15.79:6838/2441542443,v1:172.21.15.79:6839/2441542443] conn(0x5563e3892a70 0x5563e3894e60 crc :-1 s=READY pgs=96 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8e2129b78df152e0 server_cookie=9ff6f3761bfa7362 in_seq=0 out_seq=0 2024-03-28T18:10:45.173 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:45.170+0000 7f172aa00640 10 client.16470 ms_handle_connect on v2:172.21.15.79:6838/2441542443 2024-03-28T18:10:45.173 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:45.171+0000 7f172aa00640 1 -- 172.21.15.22:0/3413813302 <== mds.0 v2:172.21.15.79:6838/2441542443 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x5563e37ea800 con 0x5563e3892a70 2024-03-28T18:10:45.173 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:45.171+0000 7f172aa00640 10 client.16470 handle_command_reply: tid=0 2024-03-28T18:10:45.173 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:45.171+0000 7f174e9a3740 1 client.16470 shutdown 2024-03-28T18:10:45.173 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:45.171+0000 7f174e9a3740 1 -- 172.21.15.22:0/3413813302 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5563e387a080 msgr2=0x5563e387f890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:10:45.173 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:45.171+0000 7f174e9a3740 1 --2- 172.21.15.22:0/3413813302 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5563e387a080 0x5563e387f890 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7f1730000c00 tx=0x7f1730097c20 comp rx=0 tx=0).stop 2024-03-28T18:10:45.173 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:45.172+0000 7f174e9a3740 1 -- 172.21.15.22:0/3413813302 shutdown_connections 2024-03-28T18:10:45.173 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:45.172+0000 7f174e9a3740 1 --2- 172.21.15.22:0/3413813302 >> [v2:172.21.15.79:6838/2441542443,v1:172.21.15.79:6839/2441542443] conn(0x5563e3892a70 0x5563e3894e60 crc :-1 s=READY pgs=96 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:10:45.173 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:45.172+0000 7f174e9a3740 1 --2- 172.21.15.22:0/3413813302 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5563e3882c00 0x5563e387fe10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:10:45.173 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:45.172+0000 7f174e9a3740 1 --2- 172.21.15.22:0/3413813302 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5563e387a080 0x5563e387f890 unknown :-1 s=CLOSED pgs=691 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:10:45.173 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:45.172+0000 7f174e9a3740 1 --2- 172.21.15.22:0/3413813302 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5563e38796d0 0x5563e3882500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:10:45.173 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:45.172+0000 7f172aa00640 0 client.16470 ms_handle_reset on v2:172.21.15.79:6838/2441542443 2024-03-28T18:10:45.173 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:45.172+0000 7f174e9a3740 1 -- 172.21.15.22:0/3413813302 >> 172.21.15.22:0/3413813302 conn(0x5563e3873da0 msgr2=0x5563e3875af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:10:45.173 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:45.172+0000 7f174e9a3740 1 -- 172.21.15.22:0/3413813302 shutdown_connections 2024-03-28T18:10:45.173 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:45.172+0000 7f174e9a3740 1 -- 172.21.15.22:0/3413813302 wait complete. 2024-03-28T18:10:45.174 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:45.172+0000 7f174e9a3740 20 client.16470 trim_cache size 0 max 16384 2024-03-28T18:10:45.186 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:45.186+0000 7f174e9a3740 1 Processor -- start 2024-03-28T18:10:45.186 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:45.186+0000 7f174e9a3740 1 -- start start 2024-03-28T18:10:45.187 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:45.186+0000 7f174e9a3740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5563e38796d0 0x5563e3881f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:10:45.187 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:45.187+0000 7f174e9a3740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5563e387a080 0x5563e3882490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:10:45.187 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:45.187+0000 7f174e9a3740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5563e3882c00 0x5563e387caf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:10:45.187 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:45.187+0000 7f174e9a3740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17441af9d0 con 0x5563e3882c00 2024-03-28T18:10:45.187 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:45.187+0000 7f174e9a3740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17441b0db0 con 0x5563e38796d0 2024-03-28T18:10:45.187 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:45.187+0000 7f174e9a3740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17441b0030 con 0x5563e387a080 2024-03-28T18:10:45.187 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:45.187+0000 7f1749200640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5563e38796d0 0x5563e3881f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:10:45.187 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:45.187+0000 7f1749200640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5563e38796d0 0x5563e3881f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.22:53578/0 (socket says 172.21.15.22:53578) 2024-03-28T18:10:45.187 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:45.187+0000 7f1749200640 1 -- 172.21.15.22:0/4185400497 learned_addr learned my addr 172.21.15.22:0/4185400497 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-28T18:10:45.187 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:45.187+0000 7f1749c00640 1 --2- 172.21.15.22:0/4185400497 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5563e3882c00 0x5563e387caf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:10:45.188 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:45.187+0000 7f1743e00640 1 --2- 172.21.15.22:0/4185400497 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5563e387a080 0x5563e3882490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:10:45.189 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:45.187+0000 7f1749200640 1 -- 172.21.15.22:0/4185400497 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5563e387a080 msgr2=0x5563e3882490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T18:10:45.189 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:45.187+0000 7f1749200640 1 --2- 172.21.15.22:0/4185400497 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5563e387a080 0x5563e3882490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:10:45.189 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:45.187+0000 7f1749200640 1 -- 172.21.15.22:0/4185400497 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5563e3882c00 msgr2=0x5563e387caf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T18:10:45.189 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:45.187+0000 7f1749200640 1 --2- 172.21.15.22:0/4185400497 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5563e3882c00 0x5563e387caf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:10:45.189 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:45.187+0000 7f1749200640 1 -- 172.21.15.22:0/4185400497 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5563e3879e20 con 0x5563e38796d0 2024-03-28T18:10:45.189 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:45.188+0000 7f1749200640 1 --2- 172.21.15.22:0/4185400497 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5563e38796d0 0x5563e3881f50 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f173800bc60 tx=0x7f1738005e90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=72456d4c8f017b12 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:10:45.189 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:45.188+0000 7f1735600640 1 -- 172.21.15.22:0/4185400497 <== mon.2 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1738015040 con 0x5563e38796d0 2024-03-28T18:10:45.189 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:45.188+0000 7f1735600640 1 -- 172.21.15.22:0/4185400497 <== mon.2 v2:172.21.15.79:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f17380270d0 con 0x5563e38796d0 2024-03-28T18:10:45.189 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:45.188+0000 7f1735600640 1 -- 172.21.15.22:0/4185400497 <== mon.2 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1738005560 con 0x5563e38796d0 2024-03-28T18:10:45.189 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:45.189+0000 7f174e9a3740 1 -- 172.21.15.22:0/4185400497 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5563e38796d0 msgr2=0x5563e3881f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:10:45.189 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:45.189+0000 7f174e9a3740 1 --2- 172.21.15.22:0/4185400497 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5563e38796d0 0x5563e3881f50 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f173800bc60 tx=0x7f1738005e90 comp rx=0 tx=0).stop 2024-03-28T18:10:45.189 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:45.189+0000 7f174e9a3740 1 -- 172.21.15.22:0/4185400497 shutdown_connections 2024-03-28T18:10:45.189 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:45.189+0000 7f174e9a3740 1 --2- 172.21.15.22:0/4185400497 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5563e3882c00 0x5563e387caf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:10:45.189 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:45.189+0000 7f174e9a3740 1 --2- 172.21.15.22:0/4185400497 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5563e387a080 0x5563e3882490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:10:45.190 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:45.189+0000 7f174e9a3740 1 --2- 172.21.15.22:0/4185400497 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5563e38796d0 0x5563e3881f50 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:10:45.190 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:45.189+0000 7f174e9a3740 1 -- 172.21.15.22:0/4185400497 >> 172.21.15.22:0/4185400497 conn(0x5563e3873da0 msgr2=0x5563e3872eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:10:45.190 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:45.189+0000 7f174e9a3740 1 -- 172.21.15.22:0/4185400497 shutdown_connections 2024-03-28T18:10:45.190 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:45.189+0000 7f174e9a3740 1 -- 172.21.15.22:0/4185400497 wait complete. 2024-03-28T18:10:45.192 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:45.189+0000 7f174e9a3740 1 -- start start 2024-03-28T18:10:45.193 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:45.190+0000 7f174e9a3740 1 Processor -- start 2024-03-28T18:10:45.193 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:45.190+0000 7f174e9a3740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5563e38796d0 0x5563e388a680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:10:45.193 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:45.190+0000 7f174e9a3740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5563e387a080 0x5563e388abc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:10:45.193 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:45.190+0000 7f174e9a3740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5563e3882c00 0x5563e3888b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:10:45.193 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:45.190+0000 7f174e9a3740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17441ae150 con 0x5563e38796d0 2024-03-28T18:10:45.193 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:45.190+0000 7f174e9a3740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17441b0750 con 0x5563e3882c00 2024-03-28T18:10:45.193 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:45.190+0000 7f174e9a3740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17441ad510 con 0x5563e387a080 2024-03-28T18:10:45.193 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:45.190+0000 7f174e9a3740 10 client.0 _ll_register_callbacks cb 0x7f1730010da0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-28T18:10:45.193 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:45.190+0000 7f1749200640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5563e38796d0 0x5563e388a680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:10:45.193 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:45.190+0000 7f1749200640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5563e38796d0 0x5563e388a680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:34772/0 (socket says 172.21.15.22:34772) 2024-03-28T18:10:45.193 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:45.190+0000 7f1749200640 1 -- 172.21.15.22:0/2160571751 learned_addr learned my addr 172.21.15.22:0/2160571751 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-28T18:10:45.193 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:45.191+0000 7f1749c00640 1 --2- 172.21.15.22:0/2160571751 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5563e3882c00 0x5563e3888b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:10:45.193 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:45.191+0000 7f1743e00640 1 --2- 172.21.15.22:0/2160571751 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5563e387a080 0x5563e388abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:10:45.194 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:45.191+0000 7f1749200640 1 -- 172.21.15.22:0/2160571751 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5563e387a080 msgr2=0x5563e388abc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T18:10:45.194 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:45.191+0000 7f1749200640 1 --2- 172.21.15.22:0/2160571751 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5563e387a080 0x5563e388abc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:10:45.194 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:45.191+0000 7f1749200640 1 -- 172.21.15.22:0/2160571751 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5563e3882c00 msgr2=0x5563e3888b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T18:10:45.194 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:45.191+0000 7f1749200640 1 --2- 172.21.15.22:0/2160571751 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5563e3882c00 0x5563e3888b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:10:45.194 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:45.191+0000 7f1749200640 1 -- 172.21.15.22:0/2160571751 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5563e38892b0 con 0x5563e38796d0 2024-03-28T18:10:45.194 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:45.191+0000 7f1749c00640 1 --2- 172.21.15.22:0/2160571751 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5563e3882c00 0x5563e3888b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-28T18:10:45.194 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:45.191+0000 7f1749200640 1 --2- 172.21.15.22:0/2160571751 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5563e38796d0 0x5563e388a680 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7f173800b650 tx=0x7f173800b400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7231fa30157a9f41 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:10:45.194 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:45.191+0000 7f172aa00640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-03-28T18:10:45.194 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:45.192+0000 7f172aa00640 1 -- 172.21.15.22:0/2160571751 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1738015040 con 0x5563e38796d0 2024-03-28T18:10:45.194 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:45.192+0000 7f172aa00640 1 -- 172.21.15.22:0/2160571751 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 8583+0+0 (secure 0 0 0) 0x7f17380054e0 con 0x5563e38796d0 2024-03-28T18:10:45.194 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:45.192+0000 7f174e9a3740 1 -- 172.21.15.22:0/2160571751 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f17380270d0 con 0x5563e38796d0 2024-03-28T18:10:45.194 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:45.192+0000 7f172aa00640 1 -- 172.21.15.22:0/2160571751 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1738027710 con 0x5563e38796d0 2024-03-28T18:10:45.194 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:45.192+0000 7f174e9a3740 10 client.16476 fetch_fsmap learned FSMap version 40 2024-03-28T18:10:45.194 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:45.192+0000 7f174e9a3740 1 -- 172.21.15.22:0/2160571751 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) v3 -- 0x5563e38eed90 con 0x5563e38796d0 2024-03-28T18:10:45.195 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:45.192+0000 7f172aa00640 1 -- 172.21.15.22:0/2160571751 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f17380270d0 con 0x5563e38796d0 2024-03-28T18:10:45.195 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:45.192+0000 7f172aa00640 1 -- 172.21.15.22:0/2160571751 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 40) v1 ==== 7954+0+0 (secure 0 0 0) 0x7f1738004c20 con 0x5563e38796d0 2024-03-28T18:10:45.195 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:45.193+0000 7f174e9a3740 10 client.16476 fetch_fsmap finished waiting for FSMap version 40 2024-03-28T18:10:45.195 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:45.193+0000 7f174e9a3740 10 client.16476 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-28T18:10:45.195 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:45.193+0000 7f174e9a3740 20 client.16476 populate_metadata read hostname 'smithi022' 2024-03-28T18:10:45.195 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:45.193+0000 7f174e9a3740 1 --2- 172.21.15.22:0/2160571751 >> [v2:172.21.15.79:6838/2441542443,v1:172.21.15.79:6839/2441542443] conn(0x5563e38ef230 0x5563e38f1650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:10:45.195 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:45.193+0000 7f174e9a3740 4 client.16476 mds_command: new command op to 34553 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-28T18:10:45.195 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:45.193+0000 7f174e9a3740 1 -- 172.21.15.22:0/2160571751 --> [v2:172.21.15.79:6838/2441542443,v1:172.21.15.79:6839/2441542443] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x5563e37e4b50 con 0x5563e38ef230 2024-03-28T18:10:45.195 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:45.193+0000 7f1743e00640 1 --2- 172.21.15.22:0/2160571751 >> [v2:172.21.15.79:6838/2441542443,v1:172.21.15.79:6839/2441542443] conn(0x5563e38ef230 0x5563e38f1650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:10:45.196 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:45.194+0000 7f1743e00640 1 --2- 172.21.15.22:0/2160571751 >> [v2:172.21.15.79:6838/2441542443,v1:172.21.15.79:6839/2441542443] conn(0x5563e38ef230 0x5563e38f1650 crc :-1 s=READY pgs=97 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6d3776b1f3c71dba server_cookie=8ff1c19251f351b in_seq=0 out_seq=0 2024-03-28T18:10:45.196 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:45.194+0000 7f172aa00640 10 client.16476 ms_handle_connect on v2:172.21.15.79:6838/2441542443 2024-03-28T18:10:45.196 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:45.194+0000 7f172aa00640 1 -- 172.21.15.22:0/2160571751 <== mds.0 v2:172.21.15.79:6838/2441542443 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x5563e37e4b50 con 0x5563e38ef230 2024-03-28T18:10:45.196 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:45.194+0000 7f172aa00640 10 client.16476 handle_command_reply: tid=0 2024-03-28T18:10:45.196 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:45.194+0000 7f174e9a3740 1 client.16476 shutdown 2024-03-28T18:10:45.197 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:45.194+0000 7f174e9a3740 1 -- 172.21.15.22:0/2160571751 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5563e38796d0 msgr2=0x5563e388a680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:10:45.197 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:45.194+0000 7f174e9a3740 1 --2- 172.21.15.22:0/2160571751 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5563e38796d0 0x5563e388a680 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7f173800b650 tx=0x7f173800b400 comp rx=0 tx=0).stop 2024-03-28T18:10:45.197 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:45.194+0000 7f174e9a3740 1 -- 172.21.15.22:0/2160571751 shutdown_connections 2024-03-28T18:10:45.197 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:45.194+0000 7f174e9a3740 1 --2- 172.21.15.22:0/2160571751 >> [v2:172.21.15.79:6838/2441542443,v1:172.21.15.79:6839/2441542443] conn(0x5563e38ef230 0x5563e38f1650 crc :-1 s=READY pgs=97 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:10:45.197 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:45.194+0000 7f174e9a3740 1 --2- 172.21.15.22:0/2160571751 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5563e3882c00 0x5563e3888b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:10:45.197 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:45.194+0000 7f174e9a3740 1 --2- 172.21.15.22:0/2160571751 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5563e387a080 0x5563e388abc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:10:45.197 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:45.194+0000 7f174e9a3740 1 --2- 172.21.15.22:0/2160571751 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5563e38796d0 0x5563e388a680 unknown :-1 s=CLOSED pgs=692 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:10:45.197 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:45.194+0000 7f172aa00640 0 client.16476 ms_handle_reset on v2:172.21.15.79:6838/2441542443 2024-03-28T18:10:45.197 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:45.194+0000 7f174e9a3740 1 -- 172.21.15.22:0/2160571751 >> 172.21.15.22:0/2160571751 conn(0x5563e3873da0 msgr2=0x5563e3872620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:10:45.197 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:45.195+0000 7f174e9a3740 1 -- 172.21.15.22:0/2160571751 shutdown_connections 2024-03-28T18:10:45.197 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:45.195+0000 7f174e9a3740 1 -- 172.21.15.22:0/2160571751 wait complete. 2024-03-28T18:10:45.197 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:45.195+0000 7f174e9a3740 20 client.16476 trim_cache size 0 max 16384 2024-03-28T18:10:45.198 INFO:teuthology.orchestra.run.smithi022.stdout:[] 2024-03-28T18:10:45.198 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:45.195+0000 7f174bc00640 1 -- 172.21.15.22:0/2840604647 >> [v2:172.21.15.22:6800/1357365082,v1:172.21.15.22:6801/1357365082] conn(0x7f172c0a29f0 msgr2=0x7f172c0a4eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:10:45.198 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:45.195+0000 7f174bc00640 1 --2- 172.21.15.22:0/2840604647 >> [v2:172.21.15.22:6800/1357365082,v1:172.21.15.22:6801/1357365082] conn(0x7f172c0a29f0 0x7f172c0a4eb0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f173c009bf0 tx=0x7f173c004a40 comp rx=0 tx=0).stop 2024-03-28T18:10:45.199 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:45.195+0000 7f174bc00640 1 -- 172.21.15.22:0/2840604647 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1744101ff0 msgr2=0x7f174419a600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:10:45.199 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:45.195+0000 7f174bc00640 1 --2- 172.21.15.22:0/2840604647 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1744101ff0 0x7f174419a600 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7f173000e9f0 tx=0x7f173000eeb0 comp rx=0 tx=0).stop 2024-03-28T18:10:45.199 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:45.195+0000 7f174bc00640 1 -- 172.21.15.22:0/2840604647 shutdown_connections 2024-03-28T18:10:45.199 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:45.195+0000 7f174bc00640 1 --2- 172.21.15.22:0/2840604647 >> [v2:172.21.15.22:6800/1357365082,v1:172.21.15.22:6801/1357365082] conn(0x7f172c0a29f0 0x7f172c0a4eb0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:10:45.199 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:45.195+0000 7f174bc00640 1 --2- 172.21.15.22:0/2840604647 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f1744107640 0x7f174419e980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:10:45.199 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:45.195+0000 7f174bc00640 1 --2- 172.21.15.22:0/2840604647 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1744101ff0 0x7f174419a600 unknown :-1 s=CLOSED pgs=689 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:10:45.199 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:45.195+0000 7f174bc00640 1 --2- 172.21.15.22:0/2840604647 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1744101630 0x7f174419a0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:10:45.199 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:45.195+0000 7f174bc00640 1 -- 172.21.15.22:0/2840604647 >> 172.21.15.22:0/2840604647 conn(0x7f1744078aa0 msgr2=0x7f1744103df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:10:45.199 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:45.196+0000 7f174bc00640 1 -- 172.21.15.22:0/2840604647 shutdown_connections 2024-03-28T18:10:45.199 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:45.196+0000 7f174bc00640 1 -- 172.21.15.22:0/2840604647 wait complete. 2024-03-28T18:10:46.215 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-03-28T18:10:46.216 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-28T18:10:46.216 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-03-28T18:10:46.460 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:46.460+0000 7f7facc00640 1 -- 172.21.15.22:0/3407176744 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7fa8109730 msgr2=0x7f7fa8109b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:10:46.460 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:46.460+0000 7f7facc00640 1 --2- 172.21.15.22:0/3407176744 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7fa8109730 0x7f7fa8109b10 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7f7f98009930 tx=0x7f7f9802bc30 comp rx=0 tx=0).stop 2024-03-28T18:10:46.460 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:46.460+0000 7f7facc00640 1 -- 172.21.15.22:0/3407176744 shutdown_connections 2024-03-28T18:10:46.460 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:46.460+0000 7f7facc00640 1 --2- 172.21.15.22:0/3407176744 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7fa81040e0 0x7f7fa81105f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:10:46.460 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:46.460+0000 7f7facc00640 1 --2- 172.21.15.22:0/3407176744 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f7fa8103720 0x7f7fa8103ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:10:46.460 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:46.460+0000 7f7facc00640 1 --2- 172.21.15.22:0/3407176744 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7fa8109730 0x7f7fa8109b10 unknown :-1 s=CLOSED pgs=693 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:10:46.461 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:46.460+0000 7f7facc00640 1 -- 172.21.15.22:0/3407176744 >> 172.21.15.22:0/3407176744 conn(0x7f7fa80ff1d0 msgr2=0x7f7fa81015f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:10:46.461 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:46.460+0000 7f7facc00640 1 -- 172.21.15.22:0/3407176744 shutdown_connections 2024-03-28T18:10:46.461 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:46.460+0000 7f7facc00640 1 -- 172.21.15.22:0/3407176744 wait complete. 2024-03-28T18:10:46.462 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:46.461+0000 7f7facc00640 1 Processor -- start 2024-03-28T18:10:46.462 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:46.461+0000 7f7facc00640 1 -- start start 2024-03-28T18:10:46.462 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:46.462+0000 7f7facc00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7fa8103720 0x7f7fa819a0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:10:46.462 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:46.462+0000 7f7facc00640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f7fa81040e0 0x7f7fa819a600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:10:46.462 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:46.462+0000 7f7facc00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7fa8109730 0x7f7fa819e980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:10:46.462 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:46.462+0000 7f7facc00640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7fa8113b80 con 0x7f7fa8109730 2024-03-28T18:10:46.462 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:46.462+0000 7f7facc00640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7fa8113a00 con 0x7f7fa81040e0 2024-03-28T18:10:46.462 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:46.462+0000 7f7facc00640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7fa8113d00 con 0x7f7fa8103720 2024-03-28T18:10:46.463 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:46.462+0000 7f7fa7e00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7fa8109730 0x7f7fa819e980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:10:46.463 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:46.462+0000 7f7fa7e00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7fa8109730 0x7f7fa819e980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:34786/0 (socket says 172.21.15.22:34786) 2024-03-28T18:10:46.463 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:46.462+0000 7f7fa7e00640 1 -- 172.21.15.22:0/310957274 learned_addr learned my addr 172.21.15.22:0/310957274 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-28T18:10:46.463 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:46.462+0000 7f7f9fe00640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f7fa81040e0 0x7f7fa819a600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:10:46.463 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:46.463+0000 7f7fa7400640 1 --2- 172.21.15.22:0/310957274 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7fa8103720 0x7f7fa819a0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:10:46.464 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:46.463+0000 7f7f9fe00640 1 -- 172.21.15.22:0/310957274 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7fa8103720 msgr2=0x7f7fa819a0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:10:46.464 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:46.463+0000 7f7f9fe00640 1 --2- 172.21.15.22:0/310957274 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7fa8103720 0x7f7fa819a0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:10:46.464 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:46.463+0000 7f7f9fe00640 1 -- 172.21.15.22:0/310957274 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7fa8109730 msgr2=0x7f7fa819e980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:10:46.464 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:46.463+0000 7f7f9fe00640 1 --2- 172.21.15.22:0/310957274 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7fa8109730 0x7f7fa819e980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:10:46.464 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:46.463+0000 7f7f9fe00640 1 -- 172.21.15.22:0/310957274 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7fa819f0b0 con 0x7f7fa81040e0 2024-03-28T18:10:46.464 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:46.463+0000 7f7fa7e00640 1 --2- 172.21.15.22:0/310957274 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7fa8109730 0x7f7fa819e980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-28T18:10:46.464 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:46.463+0000 7f7fa7400640 1 --2- 172.21.15.22:0/310957274 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7fa8103720 0x7f7fa819a0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-28T18:10:46.464 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:46.463+0000 7f7f9fe00640 1 --2- 172.21.15.22:0/310957274 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f7fa81040e0 0x7f7fa819a600 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f7f9000e9f0 tx=0x7f7f9000eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:10:46.464 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:46.464+0000 7f7fa4c00640 1 -- 172.21.15.22:0/310957274 <== mon.2 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f9000ccc0 con 0x7f7fa81040e0 2024-03-28T18:10:46.464 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:46.464+0000 7f7facc00640 1 -- 172.21.15.22:0/310957274 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7fa819f3a0 con 0x7f7fa81040e0 2024-03-28T18:10:46.464 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:46.464+0000 7f7fa4c00640 1 -- 172.21.15.22:0/310957274 <== mon.2 v2:172.21.15.79:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f7f90010900 con 0x7f7fa81040e0 2024-03-28T18:10:46.465 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:46.464+0000 7f7fa4c00640 1 -- 172.21.15.22:0/310957274 <== mon.2 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f90018960 con 0x7f7fa81040e0 2024-03-28T18:10:46.465 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:46.464+0000 7f7facc00640 1 -- 172.21.15.22:0/310957274 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7fa81a6c30 con 0x7f7fa81040e0 2024-03-28T18:10:46.467 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:46.467+0000 7f7fb0fdc740 1 Processor -- start 2024-03-28T18:10:46.467 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:46.467+0000 7f7fb0fdc740 1 -- start start 2024-03-28T18:10:46.468 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:46.468+0000 7f7fb0fdc740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5600e59d60a0 0x5600e59d6480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:10:46.468 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:46.468+0000 7f7fb0fdc740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5600e59d6a50 0x5600e59e7a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:10:46.468 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:46.468+0000 7f7fb0fdc740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5600e59e7fa0 0x5600e59ea390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:10:46.468 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:46.468+0000 7f7fb0fdc740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x5600e593aeb0 con 0x5600e59e7fa0 2024-03-28T18:10:46.468 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:46.468+0000 7f7fb0fdc740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x5600e58e3a60 con 0x5600e59d60a0 2024-03-28T18:10:46.469 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:46.468+0000 7f7fb0fdc740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x5600e5936f50 con 0x5600e59d6a50 2024-03-28T18:10:46.469 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:46.468+0000 7f7fa7e00640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5600e59d60a0 0x5600e59d6480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:10:46.469 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:46.468+0000 7f7fa7e00640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5600e59d60a0 0x5600e59d6480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.22:53614/0 (socket says 172.21.15.22:53614) 2024-03-28T18:10:46.469 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:46.468+0000 7f7fa7e00640 1 -- 172.21.15.22:0/2629609691 learned_addr learned my addr 172.21.15.22:0/2629609691 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-28T18:10:46.469 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:46.468+0000 7f7fa7400640 1 --2- 172.21.15.22:0/2629609691 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5600e59d6a50 0x5600e59e7a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:10:46.469 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:46.468+0000 7f7fa7e00640 1 -- 172.21.15.22:0/2629609691 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5600e59d6a50 msgr2=0x5600e59e7a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T18:10:46.469 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:46.468+0000 7f7fa7e00640 1 --2- 172.21.15.22:0/2629609691 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5600e59d6a50 0x5600e59e7a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:10:46.469 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:46.468+0000 7f7fa7e00640 1 -- 172.21.15.22:0/2629609691 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5600e59e7fa0 msgr2=0x5600e59ea390 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-28T18:10:46.469 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:46.468+0000 7f7fa7e00640 1 --2- 172.21.15.22:0/2629609691 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5600e59e7fa0 0x5600e59ea390 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:10:46.469 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:46.468+0000 7f7fa7e00640 1 -- 172.21.15.22:0/2629609691 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5600e57177e0 con 0x5600e59d60a0 2024-03-28T18:10:46.469 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:46.469+0000 7f7fa4c00640 1 -- 172.21.15.22:0/310957274 <== mon.2 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 26) v1 ==== 137421+0+0 (secure 0 0 0) 0x7f7f90010ab0 con 0x7f7fa81040e0 2024-03-28T18:10:46.469 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:46.469+0000 7f7fa7e00640 1 --2- 172.21.15.22:0/2629609691 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5600e59d60a0 0x5600e59d6480 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f7fa000e5e0 tx=0x7f7fa0009710 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cdc5eec5c4697926 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:10:46.473 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:46.469+0000 7f7f97e00640 1 -- 172.21.15.22:0/2629609691 <== mon.2 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7fa0017020 con 0x5600e59d60a0 2024-03-28T18:10:46.473 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:46.469+0000 7f7f97e00640 1 -- 172.21.15.22:0/2629609691 <== mon.2 v2:172.21.15.79:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f7fa0013da0 con 0x5600e59d60a0 2024-03-28T18:10:46.473 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:46.469+0000 7f7f97e00640 1 -- 172.21.15.22:0/2629609691 <== mon.2 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7fa0012360 con 0x5600e59d60a0 2024-03-28T18:10:46.473 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:46.470+0000 7f7fb0fdc740 1 -- 172.21.15.22:0/2629609691 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5600e59d60a0 msgr2=0x5600e59d6480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:10:46.473 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:46.470+0000 7f7fb0fdc740 1 --2- 172.21.15.22:0/2629609691 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5600e59d60a0 0x5600e59d6480 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f7fa000e5e0 tx=0x7f7fa0009710 comp rx=0 tx=0).stop 2024-03-28T18:10:46.473 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:46.470+0000 7f7fb0fdc740 1 -- 172.21.15.22:0/2629609691 shutdown_connections 2024-03-28T18:10:46.473 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:46.470+0000 7f7fb0fdc740 1 --2- 172.21.15.22:0/2629609691 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5600e59e7fa0 0x5600e59ea390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:10:46.473 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:46.470+0000 7f7fb0fdc740 1 --2- 172.21.15.22:0/2629609691 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5600e59d6a50 0x5600e59e7a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:10:46.473 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:46.470+0000 7f7fb0fdc740 1 --2- 172.21.15.22:0/2629609691 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5600e59d60a0 0x5600e59d6480 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:10:46.473 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:46.470+0000 7f7fb0fdc740 1 -- 172.21.15.22:0/2629609691 >> 172.21.15.22:0/2629609691 conn(0x5600e59d09c0 msgr2=0x5600e59d0da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:10:46.473 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:46.470+0000 7f7fb0fdc740 1 -- 172.21.15.22:0/2629609691 shutdown_connections 2024-03-28T18:10:46.473 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:46.470+0000 7f7fb0fdc740 1 -- 172.21.15.22:0/2629609691 wait complete. 2024-03-28T18:10:46.473 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:46.470+0000 7f7fa4c00640 1 --2- 172.21.15.22:0/310957274 >> [v2:172.21.15.22:6800/1357365082,v1:172.21.15.22:6801/1357365082] conn(0x7f7f8c0a2830 0x7f7f8c0a4cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:10:46.474 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:46.470+0000 7f7fa4c00640 1 -- 172.21.15.22:0/310957274 <== mon.2 v2:172.21.15.79:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 8583+0+0 (secure 0 0 0) 0x7f7f900cf2e0 con 0x7f7fa81040e0 2024-03-28T18:10:46.474 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:46.470+0000 7f7fb0fdc740 1 -- start start 2024-03-28T18:10:46.474 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:46.470+0000 7f7fa7400640 1 --2- 172.21.15.22:0/310957274 >> [v2:172.21.15.22:6800/1357365082,v1:172.21.15.22:6801/1357365082] conn(0x7f7f8c0a2830 0x7f7f8c0a4cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:10:46.474 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:46.471+0000 7f7fb0fdc740 1 Processor -- start 2024-03-28T18:10:46.474 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:46.471+0000 7f7fb0fdc740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5600e59d60a0 0x5600e59e7870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:10:46.474 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:46.471+0000 7f7fa7400640 1 --2- 172.21.15.22:0/310957274 >> [v2:172.21.15.22:6800/1357365082,v1:172.21.15.22:6801/1357365082] conn(0x7f7f8c0a2830 0x7f7f8c0a4cf0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f7f98036040 tx=0x7f7f98005fd0 comp rx=0 tx=0).ready entity=mgr.15522 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:10:46.474 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:46.471+0000 7f7fb0fdc740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5600e59d6a50 0x5600e59e4c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:10:46.474 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:46.471+0000 7f7fb0fdc740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5600e59e7fa0 0x5600e59e5140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:10:46.474 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:46.471+0000 7f7fb0fdc740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x5600e59ead60 con 0x5600e59d6a50 2024-03-28T18:10:46.474 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:46.471+0000 7f7fb0fdc740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x5600e58f6eb0 con 0x5600e59e7fa0 2024-03-28T18:10:46.474 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:46.471+0000 7f7fb0fdc740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x5600e59eaee0 con 0x5600e59d60a0 2024-03-28T18:10:46.474 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:46.472+0000 7f7fb0fdc740 10 client.0 _ll_register_callbacks cb 0x5600e58eb150 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-28T18:10:46.474 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:46.472+0000 7f7f9fe00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5600e59d6a50 0x5600e59e4c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:10:46.474 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:46.472+0000 7f7f9fe00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5600e59d6a50 0x5600e59e4c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:34802/0 (socket says 172.21.15.22:34802) 2024-03-28T18:10:46.475 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:46.472+0000 7f7f9fe00640 1 -- 172.21.15.22:0/2752225432 learned_addr learned my addr 172.21.15.22:0/2752225432 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-28T18:10:46.475 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:46.472+0000 7f7fa7e00640 1 --2- 172.21.15.22:0/2752225432 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5600e59e7fa0 0x5600e59e5140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:10:46.475 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:46.472+0000 7f7fa7400640 1 --2- 172.21.15.22:0/2752225432 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5600e59d60a0 0x5600e59e7870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:10:46.475 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:46.472+0000 7f7f9fe00640 1 -- 172.21.15.22:0/2752225432 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5600e59d60a0 msgr2=0x5600e59e7870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T18:10:46.475 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:46.472+0000 7f7f9fe00640 1 --2- 172.21.15.22:0/2752225432 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5600e59d60a0 0x5600e59e7870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:10:46.475 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:46.472+0000 7f7f9fe00640 1 -- 172.21.15.22:0/2752225432 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5600e59e7fa0 msgr2=0x5600e59e5140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T18:10:46.475 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:46.472+0000 7f7f9fe00640 1 --2- 172.21.15.22:0/2752225432 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5600e59e7fa0 0x5600e59e5140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:10:46.475 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:46.472+0000 7f7f9fe00640 1 -- 172.21.15.22:0/2752225432 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5600e5777f70 con 0x5600e59d6a50 2024-03-28T18:10:46.475 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:46.473+0000 7f7f9fe00640 1 --2- 172.21.15.22:0/2752225432 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5600e59d6a50 0x5600e59e4c00 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7f7f9009b830 tx=0x7f7f9009b4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=73a95146138a0f1e server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:10:46.475 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:46.473+0000 7f7f94c00640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-03-28T18:10:46.475 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:46.473+0000 7f7f94c00640 1 -- 172.21.15.22:0/2752225432 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f9009b8a0 con 0x5600e59d6a50 2024-03-28T18:10:46.475 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:46.473+0000 7f7f94c00640 1 -- 172.21.15.22:0/2752225432 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 8583+0+0 (secure 0 0 0) 0x7f7f9009cbd0 con 0x5600e59d6a50 2024-03-28T18:10:46.475 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:46.473+0000 7f7fb0fdc740 1 -- 172.21.15.22:0/2752225432 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f7fa0013da0 con 0x5600e59d6a50 2024-03-28T18:10:46.475 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:46.473+0000 7f7f94c00640 1 -- 172.21.15.22:0/2752225432 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7f900b9050 con 0x5600e59d6a50 2024-03-28T18:10:46.475 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:46.473+0000 7f7fb0fdc740 10 client.16485 fetch_fsmap learned FSMap version 42 2024-03-28T18:10:46.476 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:46.473+0000 7f7fb0fdc740 1 -- 172.21.15.22:0/2752225432 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) v3 -- 0x5600e560f170 con 0x5600e59d6a50 2024-03-28T18:10:46.477 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:46.473+0000 7f7f94c00640 1 -- 172.21.15.22:0/2752225432 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f7fa0013da0 con 0x5600e59d6a50 2024-03-28T18:10:46.477 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:46.473+0000 7f7f94c00640 1 -- 172.21.15.22:0/2752225432 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 42) v1 ==== 7954+0+0 (secure 0 0 0) 0x7f7f900a3cb0 con 0x5600e59d6a50 2024-03-28T18:10:46.478 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:46.474+0000 7f7fb0fdc740 10 client.16485 fetch_fsmap finished waiting for FSMap version 42 2024-03-28T18:10:46.478 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:46.474+0000 7f7fb0fdc740 10 client.16485 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-28T18:10:46.478 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:46.474+0000 7f7fb0fdc740 20 client.16485 populate_metadata read hostname 'smithi022' 2024-03-28T18:10:46.478 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:46.474+0000 7f7fb0fdc740 1 --2- 172.21.15.22:0/2752225432 >> [v2:172.21.15.79:6838/2441542443,v1:172.21.15.79:6839/2441542443] conn(0x5600e59f8020 0x5600e59fa440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:10:46.478 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:46.474+0000 7f7fb0fdc740 4 client.16485 mds_command: new command op to 34553 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-28T18:10:46.478 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:46.474+0000 7f7fb0fdc740 1 -- 172.21.15.22:0/2752225432 --> [v2:172.21.15.79:6838/2441542443,v1:172.21.15.79:6839/2441542443] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5600e59483f0 con 0x5600e59f8020 2024-03-28T18:10:46.478 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:46.474+0000 7f7fa7400640 1 --2- 172.21.15.22:0/2752225432 >> [v2:172.21.15.79:6838/2441542443,v1:172.21.15.79:6839/2441542443] conn(0x5600e59f8020 0x5600e59fa440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:10:46.478 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:46.475+0000 7f7fa7400640 1 --2- 172.21.15.22:0/2752225432 >> [v2:172.21.15.79:6838/2441542443,v1:172.21.15.79:6839/2441542443] conn(0x5600e59f8020 0x5600e59fa440 crc :-1 s=READY pgs=99 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=be78cfc461406cde server_cookie=b9ccf462164f961a in_seq=0 out_seq=0 2024-03-28T18:10:46.478 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:46.475+0000 7f7f94c00640 10 client.16485 ms_handle_connect on v2:172.21.15.79:6838/2441542443 2024-03-28T18:10:46.478 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:46.476+0000 7f7f94c00640 1 -- 172.21.15.22:0/2752225432 <== mds.0 v2:172.21.15.79:6838/2441542443 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x5600e59483f0 con 0x5600e59f8020 2024-03-28T18:10:46.478 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:46.476+0000 7f7f94c00640 10 client.16485 handle_command_reply: tid=0 2024-03-28T18:10:46.478 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:46.476+0000 7f7fb0fdc740 1 client.16485 shutdown 2024-03-28T18:10:46.478 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:46.476+0000 7f7fb0fdc740 1 -- 172.21.15.22:0/2752225432 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5600e59d6a50 msgr2=0x5600e59e4c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:10:46.478 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:46.476+0000 7f7fb0fdc740 1 --2- 172.21.15.22:0/2752225432 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5600e59d6a50 0x5600e59e4c00 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7f7f9009b830 tx=0x7f7f9009b4a0 comp rx=0 tx=0).stop 2024-03-28T18:10:46.479 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:46.476+0000 7f7fb0fdc740 1 -- 172.21.15.22:0/2752225432 shutdown_connections 2024-03-28T18:10:46.479 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:46.476+0000 7f7fb0fdc740 1 --2- 172.21.15.22:0/2752225432 >> [v2:172.21.15.79:6838/2441542443,v1:172.21.15.79:6839/2441542443] conn(0x5600e59f8020 0x5600e59fa440 crc :-1 s=READY pgs=99 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:10:46.479 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:46.476+0000 7f7fb0fdc740 1 --2- 172.21.15.22:0/2752225432 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5600e59e7fa0 0x5600e59e5140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:10:46.479 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:46.476+0000 7f7fb0fdc740 1 --2- 172.21.15.22:0/2752225432 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5600e59d6a50 0x5600e59e4c00 unknown :-1 s=CLOSED pgs=694 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:10:46.479 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:46.476+0000 7f7f94c00640 0 client.16485 ms_handle_reset on v2:172.21.15.79:6838/2441542443 2024-03-28T18:10:46.479 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:46.476+0000 7f7fb0fdc740 1 --2- 172.21.15.22:0/2752225432 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5600e59d60a0 0x5600e59e7870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:10:46.479 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:46.476+0000 7f7fb0fdc740 1 -- 172.21.15.22:0/2752225432 >> 172.21.15.22:0/2752225432 conn(0x5600e59d09c0 msgr2=0x5600e59d2710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:10:46.479 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:46.476+0000 7f7fb0fdc740 1 -- 172.21.15.22:0/2752225432 shutdown_connections 2024-03-28T18:10:46.479 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:46.476+0000 7f7fb0fdc740 1 -- 172.21.15.22:0/2752225432 wait complete. 2024-03-28T18:10:46.479 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:46.476+0000 7f7fb0fdc740 20 client.16485 trim_cache size 0 max 16384 2024-03-28T18:10:46.494 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:46.494+0000 7f7fb0fdc740 1 Processor -- start 2024-03-28T18:10:46.494 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:46.494+0000 7f7fb0fdc740 1 -- start start 2024-03-28T18:10:46.495 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:46.495+0000 7f7fb0fdc740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5600e59d60a0 0x5600e59d24f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:10:46.495 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:46.495+0000 7f7fb0fdc740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5600e59d6a50 0x5600e59e7330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:10:46.495 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:46.495+0000 7f7fb0fdc740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5600e59e7fa0 0x5600e59e7870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:10:46.495 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:46.495+0000 7f7fb0fdc740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7fa81ab490 con 0x5600e59e7fa0 2024-03-28T18:10:46.495 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:46.495+0000 7f7fb0fdc740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7fa81ac870 con 0x5600e59d60a0 2024-03-28T18:10:46.495 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:46.495+0000 7f7fb0fdc740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7fa81abaf0 con 0x5600e59d6a50 2024-03-28T18:10:46.496 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:46.495+0000 7f7fa7e00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5600e59e7fa0 0x5600e59e7870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:10:46.496 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:46.495+0000 7f7fa7e00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5600e59e7fa0 0x5600e59e7870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:34806/0 (socket says 172.21.15.22:34806) 2024-03-28T18:10:46.496 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:46.495+0000 7f7fa7e00640 1 -- 172.21.15.22:0/4269339171 learned_addr learned my addr 172.21.15.22:0/4269339171 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-28T18:10:46.496 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:46.495+0000 7f7fa7400640 1 --2- 172.21.15.22:0/4269339171 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5600e59d60a0 0x5600e59d24f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:10:46.496 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:46.495+0000 7f7f9fe00640 1 --2- 172.21.15.22:0/4269339171 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5600e59d6a50 0x5600e59e7330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:10:46.496 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:46.495+0000 7f7fa7e00640 1 -- 172.21.15.22:0/4269339171 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5600e59d6a50 msgr2=0x5600e59e7330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T18:10:46.496 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:46.495+0000 7f7fa7e00640 1 --2- 172.21.15.22:0/4269339171 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5600e59d6a50 0x5600e59e7330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:10:46.496 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:46.495+0000 7f7fa7e00640 1 -- 172.21.15.22:0/4269339171 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5600e59d60a0 msgr2=0x5600e59d24f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T18:10:46.496 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:46.495+0000 7f7fa7e00640 1 --2- 172.21.15.22:0/4269339171 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5600e59d60a0 0x5600e59d24f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:10:46.496 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:46.495+0000 7f7fa7e00640 1 -- 172.21.15.22:0/4269339171 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5600e59e3190 con 0x5600e59e7fa0 2024-03-28T18:10:46.496 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:46.496+0000 7f7fa7e00640 1 --2- 172.21.15.22:0/4269339171 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5600e59e7fa0 0x5600e59e7870 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7f7fa000b010 tx=0x7f7fa0004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6ea41aaaf516f02a server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:10:46.500 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:46.496+0000 7f7f97400640 1 -- 172.21.15.22:0/4269339171 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7fa002a070 con 0x5600e59e7fa0 2024-03-28T18:10:46.500 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:46.496+0000 7f7f97400640 1 -- 172.21.15.22:0/4269339171 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f7fa0007760 con 0x5600e59e7fa0 2024-03-28T18:10:46.500 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:46.496+0000 7f7f97400640 1 -- 172.21.15.22:0/4269339171 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7fa0017040 con 0x5600e59e7fa0 2024-03-28T18:10:46.500 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:46.497+0000 7f7fb0fdc740 1 -- 172.21.15.22:0/4269339171 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5600e59e7fa0 msgr2=0x5600e59e7870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:10:46.500 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:46.497+0000 7f7fb0fdc740 1 --2- 172.21.15.22:0/4269339171 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5600e59e7fa0 0x5600e59e7870 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7f7fa000b010 tx=0x7f7fa0004a40 comp rx=0 tx=0).stop 2024-03-28T18:10:46.500 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:46.497+0000 7f7fb0fdc740 1 -- 172.21.15.22:0/4269339171 shutdown_connections 2024-03-28T18:10:46.500 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:46.497+0000 7f7fb0fdc740 1 --2- 172.21.15.22:0/4269339171 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5600e59e7fa0 0x5600e59e7870 unknown :-1 s=CLOSED pgs=695 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:10:46.500 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:46.497+0000 7f7fb0fdc740 1 --2- 172.21.15.22:0/4269339171 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5600e59d6a50 0x5600e59e7330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:10:46.500 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:46.497+0000 7f7fb0fdc740 1 --2- 172.21.15.22:0/4269339171 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5600e59d60a0 0x5600e59d24f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:10:46.501 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:46.497+0000 7f7fb0fdc740 1 -- 172.21.15.22:0/4269339171 >> 172.21.15.22:0/4269339171 conn(0x5600e59d09c0 msgr2=0x5600e59d1de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:10:46.501 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:46.497+0000 7f7fb0fdc740 1 -- 172.21.15.22:0/4269339171 shutdown_connections 2024-03-28T18:10:46.501 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:46.497+0000 7f7fb0fdc740 1 -- 172.21.15.22:0/4269339171 wait complete. 2024-03-28T18:10:46.501 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:46.497+0000 7f7fb0fdc740 1 -- start start 2024-03-28T18:10:46.501 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:46.497+0000 7f7fb0fdc740 1 Processor -- start 2024-03-28T18:10:46.501 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:46.498+0000 7f7fb0fdc740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5600e59d60a0 0x5600e59ed140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:10:46.501 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:46.498+0000 7f7fb0fdc740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5600e59d6a50 0x5600e59ed680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:10:46.501 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:46.498+0000 7f7fb0fdc740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5600e59e7fa0 0x5600e59edbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:10:46.501 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:46.498+0000 7f7fb0fdc740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7fa81a9c10 con 0x5600e59d6a50 2024-03-28T18:10:46.501 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:46.498+0000 7f7fb0fdc740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7fa81ac210 con 0x5600e59e7fa0 2024-03-28T18:10:46.501 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:46.498+0000 7f7fb0fdc740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7fa81a8fd0 con 0x5600e59d60a0 2024-03-28T18:10:46.501 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:46.498+0000 7f7fb0fdc740 10 client.0 _ll_register_callbacks cb 0x7f7f9009b670 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-28T18:10:46.501 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:46.498+0000 7f7f9fe00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5600e59d6a50 0x5600e59ed680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:10:46.501 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:46.498+0000 7f7f9fe00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5600e59d6a50 0x5600e59ed680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:34818/0 (socket says 172.21.15.22:34818) 2024-03-28T18:10:46.501 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:46.498+0000 7f7f9fe00640 1 -- 172.21.15.22:0/1661391928 learned_addr learned my addr 172.21.15.22:0/1661391928 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-28T18:10:46.502 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:46.498+0000 7f7fa7e00640 1 --2- 172.21.15.22:0/1661391928 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5600e59e7fa0 0x5600e59edbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:10:46.502 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:46.498+0000 7f7fa7400640 1 --2- 172.21.15.22:0/1661391928 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5600e59d60a0 0x5600e59ed140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:10:46.502 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:46.499+0000 7f7f9fe00640 1 -- 172.21.15.22:0/1661391928 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5600e59d60a0 msgr2=0x5600e59ed140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T18:10:46.502 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:46.499+0000 7f7f9fe00640 1 --2- 172.21.15.22:0/1661391928 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5600e59d60a0 0x5600e59ed140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:10:46.502 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:46.499+0000 7f7f9fe00640 1 -- 172.21.15.22:0/1661391928 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5600e59e7fa0 msgr2=0x5600e59edbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T18:10:46.502 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:46.499+0000 7f7f9fe00640 1 --2- 172.21.15.22:0/1661391928 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5600e59e7fa0 0x5600e59edbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:10:46.502 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:46.499+0000 7f7f9fe00640 1 -- 172.21.15.22:0/1661391928 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5600e5a53a90 con 0x5600e59d6a50 2024-03-28T18:10:46.502 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:46.499+0000 7f7f9fe00640 1 --2- 172.21.15.22:0/1661391928 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5600e59d6a50 0x5600e59ed680 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7f7f90097150 tx=0x7f7f90097bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7a85790937869f2b server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:10:46.502 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:46.499+0000 7f7f94c00640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-03-28T18:10:46.502 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:46.499+0000 7f7f94c00640 1 -- 172.21.15.22:0/1661391928 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f90080060 con 0x5600e59d6a50 2024-03-28T18:10:46.502 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:46.499+0000 7f7f94c00640 1 -- 172.21.15.22:0/1661391928 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 8583+0+0 (secure 0 0 0) 0x7f7f90010430 con 0x5600e59d6a50 2024-03-28T18:10:46.502 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:46.499+0000 7f7fb0fdc740 1 -- 172.21.15.22:0/1661391928 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f7fa0007760 con 0x5600e59d6a50 2024-03-28T18:10:46.502 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:46.500+0000 7f7f94c00640 1 -- 172.21.15.22:0/1661391928 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7f9009b8a0 con 0x5600e59d6a50 2024-03-28T18:10:46.502 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:46.500+0000 7f7fb0fdc740 10 client.16491 fetch_fsmap learned FSMap version 42 2024-03-28T18:10:46.502 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:46.500+0000 7f7fb0fdc740 1 -- 172.21.15.22:0/1661391928 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) v3 -- 0x5600e5a54db0 con 0x5600e59d6a50 2024-03-28T18:10:46.503 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:46.500+0000 7f7f94c00640 1 -- 172.21.15.22:0/1661391928 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f7fa0007760 con 0x5600e59d6a50 2024-03-28T18:10:46.503 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:46.500+0000 7f7f94c00640 1 -- 172.21.15.22:0/1661391928 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 42) v1 ==== 7954+0+0 (secure 0 0 0) 0x7f7f90018b00 con 0x5600e59d6a50 2024-03-28T18:10:46.503 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:46.500+0000 7f7fb0fdc740 10 client.16491 fetch_fsmap finished waiting for FSMap version 42 2024-03-28T18:10:46.503 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:46.500+0000 7f7fb0fdc740 10 client.16491 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-28T18:10:46.503 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:46.500+0000 7f7fb0fdc740 20 client.16491 populate_metadata read hostname 'smithi022' 2024-03-28T18:10:46.503 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:46.500+0000 7f7fb0fdc740 1 --2- 172.21.15.22:0/1661391928 >> [v2:172.21.15.79:6838/2441542443,v1:172.21.15.79:6839/2441542443] conn(0x5600e5a55580 0x5600e5a579a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:10:46.503 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:46.500+0000 7f7fb0fdc740 4 client.16491 mds_command: new command op to 34553 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-03-28T18:10:46.503 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:46.500+0000 7f7fb0fdc740 1 -- 172.21.15.22:0/1661391928 --> [v2:172.21.15.79:6838/2441542443,v1:172.21.15.79:6839/2441542443] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x5600e5942740 con 0x5600e5a55580 2024-03-28T18:10:46.503 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:46.501+0000 7f7fa7400640 1 --2- 172.21.15.22:0/1661391928 >> [v2:172.21.15.79:6838/2441542443,v1:172.21.15.79:6839/2441542443] conn(0x5600e5a55580 0x5600e5a579a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:10:46.504 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:46.501+0000 7f7fa7400640 1 --2- 172.21.15.22:0/1661391928 >> [v2:172.21.15.79:6838/2441542443,v1:172.21.15.79:6839/2441542443] conn(0x5600e5a55580 0x5600e5a579a0 crc :-1 s=READY pgs=100 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=25bab24ff1372520 server_cookie=b43e07a9d7dcfb37 in_seq=0 out_seq=0 2024-03-28T18:10:46.504 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:46.501+0000 7f7f94c00640 10 client.16491 ms_handle_connect on v2:172.21.15.79:6838/2441542443 2024-03-28T18:10:46.504 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:46.503+0000 7f7f94c00640 1 -- 172.21.15.22:0/1661391928 <== mds.0 v2:172.21.15.79:6838/2441542443 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x5600e5942740 con 0x5600e5a55580 2024-03-28T18:10:46.504 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:46.503+0000 7f7f94c00640 10 client.16491 handle_command_reply: tid=0 2024-03-28T18:10:46.505 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:46.503+0000 7f7fb0fdc740 1 client.16491 shutdown 2024-03-28T18:10:46.505 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:46.504+0000 7f7fb0fdc740 1 -- 172.21.15.22:0/1661391928 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5600e59d6a50 msgr2=0x5600e59ed680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:10:46.505 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:46.504+0000 7f7fb0fdc740 1 --2- 172.21.15.22:0/1661391928 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5600e59d6a50 0x5600e59ed680 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7f7f90097150 tx=0x7f7f90097bf0 comp rx=0 tx=0).stop 2024-03-28T18:10:46.505 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:46.504+0000 7f7fb0fdc740 1 -- 172.21.15.22:0/1661391928 shutdown_connections 2024-03-28T18:10:46.505 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:46.504+0000 7f7fb0fdc740 1 --2- 172.21.15.22:0/1661391928 >> [v2:172.21.15.79:6838/2441542443,v1:172.21.15.79:6839/2441542443] conn(0x5600e5a55580 0x5600e5a579a0 crc :-1 s=READY pgs=100 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:10:46.505 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:46.504+0000 7f7fb0fdc740 1 --2- 172.21.15.22:0/1661391928 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5600e59e7fa0 0x5600e59edbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:10:46.505 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:46.504+0000 7f7fb0fdc740 1 --2- 172.21.15.22:0/1661391928 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5600e59d6a50 0x5600e59ed680 unknown :-1 s=CLOSED pgs=696 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:10:46.505 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:46.504+0000 7f7f94c00640 0 client.16491 ms_handle_reset on v2:172.21.15.79:6838/2441542443 2024-03-28T18:10:46.505 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:46.504+0000 7f7fb0fdc740 1 --2- 172.21.15.22:0/1661391928 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5600e59d60a0 0x5600e59ed140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:10:46.505 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:46.504+0000 7f7fb0fdc740 1 -- 172.21.15.22:0/1661391928 >> 172.21.15.22:0/1661391928 conn(0x5600e59d09c0 msgr2=0x5600e59cfa80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:10:46.505 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:46.504+0000 7f7fb0fdc740 1 -- 172.21.15.22:0/1661391928 shutdown_connections 2024-03-28T18:10:46.505 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:46.504+0000 7f7fb0fdc740 1 -- 172.21.15.22:0/1661391928 wait complete. 2024-03-28T18:10:46.505 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:46.504+0000 7f7fb0fdc740 20 client.16491 trim_cache size 0 max 16384 2024-03-28T18:10:46.506 INFO:teuthology.orchestra.run.smithi022.stdout:{ 2024-03-28T18:10:46.506 INFO:teuthology.orchestra.run.smithi022.stdout: "return_code": 0, 2024-03-28T18:10:46.506 INFO:teuthology.orchestra.run.smithi022.stdout: "scrub_tag": "4bbbd61d-29a3-4bf6-a4f2-d5783d585de3", 2024-03-28T18:10:46.506 INFO:teuthology.orchestra.run.smithi022.stdout: "mode": "asynchronous" 2024-03-28T18:10:46.506 INFO:teuthology.orchestra.run.smithi022.stdout:} 2024-03-28T18:10:46.507 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:46.505+0000 7f7facc00640 1 -- 172.21.15.22:0/310957274 >> [v2:172.21.15.22:6800/1357365082,v1:172.21.15.22:6801/1357365082] conn(0x7f7f8c0a2830 msgr2=0x7f7f8c0a4cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:10:46.507 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:46.505+0000 7f7facc00640 1 --2- 172.21.15.22:0/310957274 >> [v2:172.21.15.22:6800/1357365082,v1:172.21.15.22:6801/1357365082] conn(0x7f7f8c0a2830 0x7f7f8c0a4cf0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f7f98036040 tx=0x7f7f98005fd0 comp rx=0 tx=0).stop 2024-03-28T18:10:46.507 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:46.505+0000 7f7facc00640 1 -- 172.21.15.22:0/310957274 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f7fa81040e0 msgr2=0x7f7fa819a600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:10:46.507 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:46.505+0000 7f7facc00640 1 --2- 172.21.15.22:0/310957274 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f7fa81040e0 0x7f7fa819a600 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f7f9000e9f0 tx=0x7f7f9000eeb0 comp rx=0 tx=0).stop 2024-03-28T18:10:46.507 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:46.505+0000 7f7facc00640 1 -- 172.21.15.22:0/310957274 shutdown_connections 2024-03-28T18:10:46.507 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:46.505+0000 7f7facc00640 1 --2- 172.21.15.22:0/310957274 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7fa8109730 0x7f7fa819e980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:10:46.507 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:46.505+0000 7f7facc00640 1 --2- 172.21.15.22:0/310957274 >> [v2:172.21.15.22:6800/1357365082,v1:172.21.15.22:6801/1357365082] conn(0x7f7f8c0a2830 0x7f7f8c0a4cf0 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:10:46.507 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:46.505+0000 7f7facc00640 1 --2- 172.21.15.22:0/310957274 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f7fa81040e0 0x7f7fa819a600 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:10:46.507 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:46.505+0000 7f7facc00640 1 --2- 172.21.15.22:0/310957274 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7fa8103720 0x7f7fa819a0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:10:46.507 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:46.505+0000 7f7facc00640 1 -- 172.21.15.22:0/310957274 >> 172.21.15.22:0/310957274 conn(0x7f7fa80ff1d0 msgr2=0x7f7fa80ff9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:10:46.507 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:46.505+0000 7f7facc00640 1 -- 172.21.15.22:0/310957274 shutdown_connections 2024-03-28T18:10:46.507 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:46.505+0000 7f7facc00640 1 -- 172.21.15.22:0/310957274 wait complete. 2024-03-28T18:10:46.527 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-28T18:10:46.527 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-28T18:10:46.614 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:10:46 smithi022 ceph-mon[31985]: pgmap v101: 97 pgs: 97 active+clean; 117 MiB data, 799 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:10:46.615 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:10:46 smithi022 ceph-mon[31985]: mds.? [v2:172.21.15.79:6836/3437531480,v1:172.21.15.79:6837/3437531480] up:resolve 2024-03-28T18:10:46.615 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:10:46 smithi022 ceph-mon[31985]: fsmap cephfs:5/5 4 up:standby-replay 3 up:standby 1 up:resolve 4 up:active 2024-03-28T18:10:46.674 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:10:46 smithi134 ceph-mon[35037]: pgmap v101: 97 pgs: 97 active+clean; 117 MiB data, 799 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:10:46.674 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:10:46 smithi134 ceph-mon[35037]: mds.? [v2:172.21.15.79:6836/3437531480,v1:172.21.15.79:6837/3437531480] up:resolve 2024-03-28T18:10:46.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:10:46 smithi134 ceph-mon[35037]: fsmap cephfs:5/5 4 up:standby-replay 3 up:standby 1 up:resolve 4 up:active 2024-03-28T18:10:46.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:10:46 smithi079 ceph-mon[35312]: pgmap v101: 97 pgs: 97 active+clean; 117 MiB data, 799 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:10:46.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:10:46 smithi079 ceph-mon[35312]: mds.? [v2:172.21.15.79:6836/3437531480,v1:172.21.15.79:6837/3437531480] up:resolve 2024-03-28T18:10:46.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:10:46 smithi079 ceph-mon[35312]: fsmap cephfs:5/5 4 up:standby-replay 3 up:standby 1 up:resolve 4 up:active 2024-03-28T18:10:46.762 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:46.762+0000 7fc818e00640 1 -- 172.21.15.22:0/1792486412 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc814102ff0 msgr2=0x7fc81410b4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:10:46.762 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:46.762+0000 7fc818e00640 1 --2- 172.21.15.22:0/1792486412 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc814102ff0 0x7fc81410b4d0 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7fc800009900 tx=0x7fc80002bcc0 comp rx=0 tx=0).stop 2024-03-28T18:10:46.762 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:46.762+0000 7fc818e00640 1 -- 172.21.15.22:0/1792486412 shutdown_connections 2024-03-28T18:10:46.762 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:46.762+0000 7fc818e00640 1 --2- 172.21.15.22:0/1792486412 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc81410baf0 0x7fc81410df90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:10:46.762 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:46.762+0000 7fc818e00640 1 --2- 172.21.15.22:0/1792486412 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc814102ff0 0x7fc81410b4d0 unknown :-1 s=CLOSED pgs=697 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:10:46.762 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:46.762+0000 7fc818e00640 1 --2- 172.21.15.22:0/1792486412 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc8141026d0 0x7fc814102ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:10:46.762 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:46.762+0000 7fc818e00640 1 -- 172.21.15.22:0/1792486412 >> 172.21.15.22:0/1792486412 conn(0x7fc8140fbef0 msgr2=0x7fc8140fe310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:10:46.762 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:46.762+0000 7fc818e00640 1 -- 172.21.15.22:0/1792486412 shutdown_connections 2024-03-28T18:10:46.763 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:46.762+0000 7fc818e00640 1 -- 172.21.15.22:0/1792486412 wait complete. 2024-03-28T18:10:46.764 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:46.763+0000 7fc818e00640 1 Processor -- start 2024-03-28T18:10:46.764 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:46.763+0000 7fc818e00640 1 -- start start 2024-03-28T18:10:46.764 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:46.763+0000 7fc818e00640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc8141026d0 0x7fc81419a020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:10:46.764 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:46.763+0000 7fc818e00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc814102ff0 0x7fc81419a560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:10:46.765 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:46.764+0000 7fc818e00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc81410baf0 0x7fc81419e8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:10:46.765 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:46.764+0000 7fc818e00640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8141118c0 con 0x7fc814102ff0 2024-03-28T18:10:46.765 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:46.764+0000 7fc818e00640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc814111740 con 0x7fc8141026d0 2024-03-28T18:10:46.765 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:46.764+0000 7fc818e00640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc814111a40 con 0x7fc81410baf0 2024-03-28T18:10:46.765 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:46.764+0000 7fc812a00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc814102ff0 0x7fc81419a560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:10:46.765 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:46.764+0000 7fc813400640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc8141026d0 0x7fc81419a020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:10:46.765 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:46.764+0000 7fc812a00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc814102ff0 0x7fc81419a560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:34822/0 (socket says 172.21.15.22:34822) 2024-03-28T18:10:46.765 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:46.764+0000 7fc812a00640 1 -- 172.21.15.22:0/769375606 learned_addr learned my addr 172.21.15.22:0/769375606 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-28T18:10:46.765 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:46.764+0000 7fc813e00640 1 --2- 172.21.15.22:0/769375606 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc81410baf0 0x7fc81419e8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:10:46.765 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:46.764+0000 7fc812a00640 1 -- 172.21.15.22:0/769375606 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc81410baf0 msgr2=0x7fc81419e8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:10:46.765 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:46.764+0000 7fc812a00640 1 --2- 172.21.15.22:0/769375606 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc81410baf0 0x7fc81419e8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:10:46.765 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:46.764+0000 7fc812a00640 1 -- 172.21.15.22:0/769375606 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc8141026d0 msgr2=0x7fc81419a020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:10:46.765 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:46.764+0000 7fc812a00640 1 --2- 172.21.15.22:0/769375606 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc8141026d0 0x7fc81419a020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:10:46.765 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:46.764+0000 7fc812a00640 1 -- 172.21.15.22:0/769375606 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc81419f010 con 0x7fc814102ff0 2024-03-28T18:10:46.765 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:46.765+0000 7fc812a00640 1 --2- 172.21.15.22:0/769375606 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc814102ff0 0x7fc81419a560 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7fc800009230 tx=0x7fc800004e20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:10:46.766 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:46.765+0000 7fc80be00640 1 -- 172.21.15.22:0/769375606 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc800043020 con 0x7fc814102ff0 2024-03-28T18:10:46.766 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:46.765+0000 7fc818e00640 1 -- 172.21.15.22:0/769375606 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc81419f2a0 con 0x7fc814102ff0 2024-03-28T18:10:46.766 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:46.765+0000 7fc80be00640 1 -- 172.21.15.22:0/769375606 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7fc80002c660 con 0x7fc814102ff0 2024-03-28T18:10:46.766 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:46.765+0000 7fc80be00640 1 -- 172.21.15.22:0/769375606 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc800003af0 con 0x7fc814102ff0 2024-03-28T18:10:46.766 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:46.765+0000 7fc818e00640 1 -- 172.21.15.22:0/769375606 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc8141a6b90 con 0x7fc814102ff0 2024-03-28T18:10:46.768 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:46.768+0000 7fc81d083740 1 Processor -- start 2024-03-28T18:10:46.768 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:46.769+0000 7fc81d083740 1 -- start start 2024-03-28T18:10:46.769 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:46.769+0000 7fc81d083740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562d90142de0 0x562d90141b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:10:46.769 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:46.769+0000 7fc81d083740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562d90142100 0x562d90142580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:10:46.769 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:46.769+0000 7fc81d083740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x562d9014b470 0x562d9014d860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:10:46.769 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:46.769+0000 7fc81d083740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x562d900a6250 con 0x562d90142100 2024-03-28T18:10:46.769 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:46.769+0000 7fc81d083740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x562d9004ea30 con 0x562d9014b470 2024-03-28T18:10:46.769 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:46.769+0000 7fc81d083740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x562d900a22f0 con 0x562d90142de0 2024-03-28T18:10:46.770 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:46.769+0000 7fc813400640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562d90142100 0x562d90142580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:10:46.770 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:46.769+0000 7fc813e00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562d90142de0 0x562d90141b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:10:46.771 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:46.769+0000 7fc813400640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562d90142100 0x562d90142580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:34830/0 (socket says 172.21.15.22:34830) 2024-03-28T18:10:46.771 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:46.769+0000 7fc813400640 1 -- 172.21.15.22:0/2257157961 learned_addr learned my addr 172.21.15.22:0/2257157961 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-28T18:10:46.771 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:46.770+0000 7fc80be00640 1 -- 172.21.15.22:0/769375606 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 26) v1 ==== 137421+0+0 (secure 0 0 0) 0x7fc800046020 con 0x7fc814102ff0 2024-03-28T18:10:46.771 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:46.770+0000 7fc813e00640 1 -- 172.21.15.22:0/2257157961 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x562d9014b470 msgr2=0x562d9014d860 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-28T18:10:46.771 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:46.770+0000 7fc813e00640 1 --2- 172.21.15.22:0/2257157961 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x562d9014b470 0x562d9014d860 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:10:46.771 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:46.770+0000 7fc813e00640 1 -- 172.21.15.22:0/2257157961 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562d90142100 msgr2=0x562d90142580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T18:10:46.771 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:46.770+0000 7fc813e00640 1 --2- 172.21.15.22:0/2257157961 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562d90142100 0x562d90142580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:10:46.771 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:46.770+0000 7fc813e00640 1 -- 172.21.15.22:0/2257157961 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562d90039690 con 0x562d90142de0 2024-03-28T18:10:46.771 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:46.770+0000 7fc813400640 1 --2- 172.21.15.22:0/2257157961 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562d90142100 0x562d90142580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-28T18:10:46.771 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:46.770+0000 7fc813e00640 1 --2- 172.21.15.22:0/2257157961 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562d90142de0 0x562d90141b30 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fc80c009eb0 tx=0x7fc80c010740 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cf1397b7e2506816 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:10:46.771 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:46.771+0000 7fc808c00640 1 -- 172.21.15.22:0/2257157961 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc80c016070 con 0x562d90142de0 2024-03-28T18:10:46.771 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:46.771+0000 7fc808c00640 1 -- 172.21.15.22:0/2257157961 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7fc80c004d20 con 0x562d90142de0 2024-03-28T18:10:46.771 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:46.771+0000 7fc808c00640 1 -- 172.21.15.22:0/2257157961 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc80c012780 con 0x562d90142de0 2024-03-28T18:10:46.771 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:46.771+0000 7fc80be00640 1 --2- 172.21.15.22:0/769375606 >> [v2:172.21.15.22:6800/1357365082,v1:172.21.15.22:6801/1357365082] conn(0x7fc7f80a2970 0x7fc7f80a4e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:10:46.773 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:46.771+0000 7fc80be00640 1 -- 172.21.15.22:0/769375606 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 8583+0+0 (secure 0 0 0) 0x7fc8000f0d50 con 0x7fc814102ff0 2024-03-28T18:10:46.774 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:46.771+0000 7fc813400640 1 --2- 172.21.15.22:0/769375606 >> [v2:172.21.15.22:6800/1357365082,v1:172.21.15.22:6801/1357365082] conn(0x7fc7f80a2970 0x7fc7f80a4e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:10:46.774 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:46.771+0000 7fc81d083740 1 -- 172.21.15.22:0/2257157961 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562d90142de0 msgr2=0x562d90141b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:10:46.774 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:46.771+0000 7fc81d083740 1 --2- 172.21.15.22:0/2257157961 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562d90142de0 0x562d90141b30 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fc80c009eb0 tx=0x7fc80c010740 comp rx=0 tx=0).stop 2024-03-28T18:10:46.774 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:46.772+0000 7fc81d083740 1 -- 172.21.15.22:0/2257157961 shutdown_connections 2024-03-28T18:10:46.774 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:46.772+0000 7fc81d083740 1 --2- 172.21.15.22:0/2257157961 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x562d9014b470 0x562d9014d860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:10:46.774 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:46.772+0000 7fc81d083740 1 --2- 172.21.15.22:0/2257157961 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562d90142100 0x562d90142580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:10:46.774 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:46.772+0000 7fc81d083740 1 --2- 172.21.15.22:0/2257157961 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562d90142de0 0x562d90141b30 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:10:46.774 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:46.772+0000 7fc81d083740 1 -- 172.21.15.22:0/2257157961 >> 172.21.15.22:0/2257157961 conn(0x562d90109200 msgr2=0x562d9013cb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:10:46.774 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:46.772+0000 7fc81d083740 1 -- 172.21.15.22:0/2257157961 shutdown_connections 2024-03-28T18:10:46.774 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:46.772+0000 7fc81d083740 1 -- 172.21.15.22:0/2257157961 wait complete. 2024-03-28T18:10:46.774 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:46.772+0000 7fc813400640 1 --2- 172.21.15.22:0/769375606 >> [v2:172.21.15.22:6800/1357365082,v1:172.21.15.22:6801/1357365082] conn(0x7fc7f80a2970 0x7fc7f80a4e30 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x562d90143330 tx=0x7fc804005f90 comp rx=0 tx=0).ready entity=mgr.15522 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:10:46.774 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:46.773+0000 7fc81d083740 1 -- start start 2024-03-28T18:10:46.774 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:46.773+0000 7fc81d083740 1 Processor -- start 2024-03-28T18:10:46.774 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:46.773+0000 7fc81d083740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x562d90142100 0x562d9014abc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:10:46.775 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:46.773+0000 7fc81d083740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562d90142de0 0x562d90147f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:10:46.775 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:46.773+0000 7fc81d083740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562d9014b470 0x562d90148490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:10:46.775 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:46.773+0000 7fc81d083740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x562d9014e2e0 con 0x562d9014b470 2024-03-28T18:10:46.775 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:46.773+0000 7fc81d083740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x562d90061e80 con 0x562d90142100 2024-03-28T18:10:46.775 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:46.773+0000 7fc81d083740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x562d9014e460 con 0x562d90142de0 2024-03-28T18:10:46.775 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:46.774+0000 7fc81d083740 10 client.0 _ll_register_callbacks cb 0x562d90074130 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-28T18:10:46.775 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:46.774+0000 7fc813e00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562d9014b470 0x562d90148490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:10:46.775 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:46.774+0000 7fc813400640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x562d90142100 0x562d9014abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:10:46.775 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:46.774+0000 7fc812a00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562d90142de0 0x562d90147f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:10:46.775 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:46.774+0000 7fc813400640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x562d90142100 0x562d9014abc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.22:53676/0 (socket says 172.21.15.22:53676) 2024-03-28T18:10:46.775 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:46.774+0000 7fc813400640 1 -- 172.21.15.22:0/189518384 learned_addr learned my addr 172.21.15.22:0/189518384 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-28T18:10:46.778 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:46.774+0000 7fc813400640 1 -- 172.21.15.22:0/189518384 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562d90142de0 msgr2=0x562d90147f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T18:10:46.778 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:46.774+0000 7fc813400640 1 --2- 172.21.15.22:0/189518384 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562d90142de0 0x562d90147f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:10:46.778 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:46.774+0000 7fc813400640 1 -- 172.21.15.22:0/189518384 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562d9014b470 msgr2=0x562d90148490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T18:10:46.778 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:46.774+0000 7fc813400640 1 --2- 172.21.15.22:0/189518384 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562d9014b470 0x562d90148490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:10:46.778 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:46.774+0000 7fc813400640 1 -- 172.21.15.22:0/189518384 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x562d8fe847d0 con 0x562d90142100 2024-03-28T18:10:46.778 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:46.774+0000 7fc813e00640 1 --2- 172.21.15.22:0/189518384 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562d9014b470 0x562d90148490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-28T18:10:46.778 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:46.775+0000 7fc813400640 1 --2- 172.21.15.22:0/189518384 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x562d90142100 0x562d9014abc0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fc80401ea00 tx=0x7fc8040046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6eadf8f084aec0cb server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:10:46.778 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:46.775+0000 7fc7fd600640 10 client.0 ms_handle_connect on v2:172.21.15.79:3300/0 2024-03-28T18:10:46.778 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:46.775+0000 7fc7fd600640 1 -- 172.21.15.22:0/189518384 <== mon.2 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc804005020 con 0x562d90142100 2024-03-28T18:10:46.778 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:46.775+0000 7fc7fd600640 1 -- 172.21.15.22:0/189518384 <== mon.2 v2:172.21.15.79:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 8583+0+0 (secure 0 0 0) 0x7fc804018030 con 0x562d90142100 2024-03-28T18:10:46.778 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:46.775+0000 7fc81d083740 1 -- 172.21.15.22:0/189518384 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fc80c004d20 con 0x562d90142100 2024-03-28T18:10:46.778 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:46.776+0000 7fc7fd600640 1 -- 172.21.15.22:0/189518384 <== mon.2 v2:172.21.15.79:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc8040051c0 con 0x562d90142100 2024-03-28T18:10:46.779 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:46.776+0000 7fc81d083740 10 client.36326 fetch_fsmap learned FSMap version 42 2024-03-28T18:10:46.779 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:46.776+0000 7fc81d083740 1 -- 172.21.15.22:0/189518384 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) v3 -- 0x562d8fdba3f0 con 0x562d90142100 2024-03-28T18:10:46.779 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:46.776+0000 7fc7fd600640 1 -- 172.21.15.22:0/189518384 <== mon.2 v2:172.21.15.79:3300/0 4 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7fc80c004d20 con 0x562d90142100 2024-03-28T18:10:46.779 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:46.776+0000 7fc7fd600640 1 -- 172.21.15.22:0/189518384 <== mon.2 v2:172.21.15.79:3300/0 5 ==== fsmap(e 42) v1 ==== 7954+0+0 (secure 0 0 0) 0x7fc8040294d0 con 0x562d90142100 2024-03-28T18:10:46.779 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:46.776+0000 7fc81d083740 10 client.36326 fetch_fsmap finished waiting for FSMap version 42 2024-03-28T18:10:46.779 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:46.776+0000 7fc81d083740 10 client.36326 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-28T18:10:46.779 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:46.776+0000 7fc81d083740 20 client.36326 populate_metadata read hostname 'smithi022' 2024-03-28T18:10:46.779 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:46.776+0000 7fc81d083740 1 --2- 172.21.15.22:0/189518384 >> [v2:172.21.15.79:6838/2441542443,v1:172.21.15.79:6839/2441542443] conn(0x562d9015b530 0x562d9015d920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:10:46.779 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:46.776+0000 7fc81d083740 4 client.36326 mds_command: new command op to 34553 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-28T18:10:46.779 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:46.776+0000 7fc81d083740 1 -- 172.21.15.22:0/189518384 --> [v2:172.21.15.79:6838/2441542443,v1:172.21.15.79:6839/2441542443] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x562d900b3790 con 0x562d9015b530 2024-03-28T18:10:46.779 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:46.777+0000 7fc812a00640 1 --2- 172.21.15.22:0/189518384 >> [v2:172.21.15.79:6838/2441542443,v1:172.21.15.79:6839/2441542443] conn(0x562d9015b530 0x562d9015d920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:10:46.779 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:46.777+0000 7fc812a00640 1 --2- 172.21.15.22:0/189518384 >> [v2:172.21.15.79:6838/2441542443,v1:172.21.15.79:6839/2441542443] conn(0x562d9015b530 0x562d9015d920 crc :-1 s=READY pgs=101 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=65ceab9caf906169 server_cookie=bb50944fe1329c74 in_seq=0 out_seq=0 2024-03-28T18:10:46.779 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:46.777+0000 7fc7fd600640 10 client.36326 ms_handle_connect on v2:172.21.15.79:6838/2441542443 2024-03-28T18:10:46.779 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:46.778+0000 7fc7fd600640 1 -- 172.21.15.22:0/189518384 <== mds.0 v2:172.21.15.79:6838/2441542443 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x562d900b3790 con 0x562d9015b530 2024-03-28T18:10:46.779 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:46.778+0000 7fc7fd600640 10 client.36326 handle_command_reply: tid=0 2024-03-28T18:10:46.780 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:46.778+0000 7fc81d083740 1 client.36326 shutdown 2024-03-28T18:10:46.780 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:46.778+0000 7fc81d083740 1 -- 172.21.15.22:0/189518384 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x562d90142100 msgr2=0x562d9014abc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:10:46.780 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:46.778+0000 7fc81d083740 1 --2- 172.21.15.22:0/189518384 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x562d90142100 0x562d9014abc0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fc80401ea00 tx=0x7fc8040046c0 comp rx=0 tx=0).stop 2024-03-28T18:10:46.780 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:46.778+0000 7fc81d083740 1 -- 172.21.15.22:0/189518384 shutdown_connections 2024-03-28T18:10:46.781 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:46.778+0000 7fc81d083740 1 --2- 172.21.15.22:0/189518384 >> [v2:172.21.15.79:6838/2441542443,v1:172.21.15.79:6839/2441542443] conn(0x562d9015b530 0x562d9015d920 crc :-1 s=READY pgs=101 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:10:46.781 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:46.778+0000 7fc81d083740 1 --2- 172.21.15.22:0/189518384 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562d9014b470 0x562d90148490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:10:46.781 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:46.778+0000 7fc81d083740 1 --2- 172.21.15.22:0/189518384 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562d90142de0 0x562d90147f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:10:46.781 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:46.778+0000 7fc7fd600640 0 client.36326 ms_handle_reset on v2:172.21.15.79:6838/2441542443 2024-03-28T18:10:46.781 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:46.778+0000 7fc81d083740 1 --2- 172.21.15.22:0/189518384 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x562d90142100 0x562d9014abc0 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:10:46.781 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:46.778+0000 7fc81d083740 1 -- 172.21.15.22:0/189518384 >> 172.21.15.22:0/189518384 conn(0x562d90109200 msgr2=0x562d9013e8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:10:46.781 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:46.779+0000 7fc81d083740 1 -- 172.21.15.22:0/189518384 shutdown_connections 2024-03-28T18:10:46.781 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:46.779+0000 7fc81d083740 1 -- 172.21.15.22:0/189518384 wait complete. 2024-03-28T18:10:46.781 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:46.779+0000 7fc81d083740 20 client.36326 trim_cache size 0 max 16384 2024-03-28T18:10:46.798 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:46.798+0000 7fc81d083740 1 Processor -- start 2024-03-28T18:10:46.798 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:46.798+0000 7fc81d083740 1 -- start start 2024-03-28T18:10:46.798 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:46.798+0000 7fc81d083740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562d90142de0 0x562d9014a680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:10:46.798 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:46.798+0000 7fc81d083740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x562d9014b470 0x562d9014abc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:10:46.798 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:46.798+0000 7fc81d083740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562d901631e0 0x562d9014c860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:10:46.798 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:46.798+0000 7fc81d083740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8141af830 con 0x562d901631e0 2024-03-28T18:10:46.798 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:46.798+0000 7fc81d083740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8141b0c10 con 0x562d9014b470 2024-03-28T18:10:46.798 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:46.798+0000 7fc81d083740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8141afe90 con 0x562d90142de0 2024-03-28T18:10:46.800 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:46.799+0000 7fc812a00640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x562d9014b470 0x562d9014abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:10:46.800 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:46.799+0000 7fc813400640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562d90142de0 0x562d9014a680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:10:46.800 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:46.799+0000 7fc813e00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562d901631e0 0x562d9014c860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:10:46.800 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:46.799+0000 7fc812a00640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x562d9014b470 0x562d9014abc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.22:53680/0 (socket says 172.21.15.22:53680) 2024-03-28T18:10:46.800 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:46.799+0000 7fc812a00640 1 -- 172.21.15.22:0/103398781 learned_addr learned my addr 172.21.15.22:0/103398781 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-28T18:10:46.800 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:46.799+0000 7fc812a00640 1 -- 172.21.15.22:0/103398781 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562d90142de0 msgr2=0x562d9014a680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T18:10:46.800 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:46.799+0000 7fc812a00640 1 --2- 172.21.15.22:0/103398781 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562d90142de0 0x562d9014a680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:10:46.800 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:46.799+0000 7fc812a00640 1 -- 172.21.15.22:0/103398781 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562d901631e0 msgr2=0x562d9014c860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T18:10:46.800 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:46.799+0000 7fc812a00640 1 --2- 172.21.15.22:0/103398781 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562d901631e0 0x562d9014c860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:10:46.800 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:46.799+0000 7fc812a00640 1 -- 172.21.15.22:0/103398781 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562d8fd7a0f0 con 0x562d9014b470 2024-03-28T18:10:46.800 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:46.799+0000 7fc813400640 1 --2- 172.21.15.22:0/103398781 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562d90142de0 0x562d9014a680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-28T18:10:46.800 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:46.799+0000 7fc813e00640 1 --2- 172.21.15.22:0/103398781 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562d901631e0 0x562d9014c860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-28T18:10:46.800 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:46.799+0000 7fc812a00640 1 --2- 172.21.15.22:0/103398781 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x562d9014b470 0x562d9014abc0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fc80003e040 tx=0x7fc8000f0040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bb6ee5bd6da12eba server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:10:46.800 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:46.800+0000 7fc7ffe00640 1 -- 172.21.15.22:0/103398781 <== mon.2 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8000bd480 con 0x562d9014b470 2024-03-28T18:10:46.800 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:46.800+0000 7fc7ffe00640 1 -- 172.21.15.22:0/103398781 <== mon.2 v2:172.21.15.79:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7fc8000c15d0 con 0x562d9014b470 2024-03-28T18:10:46.801 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:46.800+0000 7fc7ffe00640 1 -- 172.21.15.22:0/103398781 <== mon.2 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc800003c90 con 0x562d9014b470 2024-03-28T18:10:46.803 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:46.801+0000 7fc81d083740 1 -- 172.21.15.22:0/103398781 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x562d9014b470 msgr2=0x562d9014abc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:10:46.803 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:46.801+0000 7fc81d083740 1 --2- 172.21.15.22:0/103398781 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x562d9014b470 0x562d9014abc0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fc80003e040 tx=0x7fc8000f0040 comp rx=0 tx=0).stop 2024-03-28T18:10:46.803 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:46.801+0000 7fc81d083740 1 -- 172.21.15.22:0/103398781 shutdown_connections 2024-03-28T18:10:46.803 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:46.801+0000 7fc81d083740 1 --2- 172.21.15.22:0/103398781 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562d901631e0 0x562d9014c860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:10:46.803 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:46.801+0000 7fc81d083740 1 --2- 172.21.15.22:0/103398781 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x562d9014b470 0x562d9014abc0 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:10:46.803 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:46.801+0000 7fc81d083740 1 --2- 172.21.15.22:0/103398781 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562d90142de0 0x562d9014a680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:10:46.803 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:46.801+0000 7fc81d083740 1 -- 172.21.15.22:0/103398781 >> 172.21.15.22:0/103398781 conn(0x562d90109200 msgr2=0x562d9013c680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:10:46.803 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:46.801+0000 7fc81d083740 1 -- 172.21.15.22:0/103398781 shutdown_connections 2024-03-28T18:10:46.803 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:46.801+0000 7fc81d083740 1 -- 172.21.15.22:0/103398781 wait complete. 2024-03-28T18:10:46.803 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:46.801+0000 7fc81d083740 1 -- start start 2024-03-28T18:10:46.803 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:46.802+0000 7fc81d083740 1 Processor -- start 2024-03-28T18:10:46.803 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:46.802+0000 7fc81d083740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562d90142de0 0x562d90156570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:10:46.803 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:46.802+0000 7fc81d083740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x562d9014b470 0x562d90156ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:10:46.804 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:46.802+0000 7fc81d083740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562d901631e0 0x562d90151a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:10:46.804 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:46.802+0000 7fc81d083740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8141adfb0 con 0x562d90142de0 2024-03-28T18:10:46.804 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:46.802+0000 7fc81d083740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8141b05b0 con 0x562d9014b470 2024-03-28T18:10:46.804 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:46.802+0000 7fc81d083740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8141ad370 con 0x562d901631e0 2024-03-28T18:10:46.804 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:46.802+0000 7fc81d083740 10 client.0 _ll_register_callbacks cb 0x7fc80401e890 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-28T18:10:46.804 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:46.802+0000 7fc813400640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562d90142de0 0x562d90156570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:10:46.804 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:46.802+0000 7fc813400640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562d90142de0 0x562d90156570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:34872/0 (socket says 172.21.15.22:34872) 2024-03-28T18:10:46.804 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:46.802+0000 7fc813400640 1 -- 172.21.15.22:0/4220369546 learned_addr learned my addr 172.21.15.22:0/4220369546 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-28T18:10:46.804 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:46.802+0000 7fc812a00640 1 --2- 172.21.15.22:0/4220369546 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x562d9014b470 0x562d90156ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:10:46.804 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:46.802+0000 7fc813e00640 1 --2- 172.21.15.22:0/4220369546 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562d901631e0 0x562d90151a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:10:46.804 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:46.803+0000 7fc813400640 1 -- 172.21.15.22:0/4220369546 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562d901631e0 msgr2=0x562d90151a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T18:10:46.804 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:46.803+0000 7fc813400640 1 --2- 172.21.15.22:0/4220369546 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562d901631e0 0x562d90151a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:10:46.804 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:46.803+0000 7fc813400640 1 -- 172.21.15.22:0/4220369546 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x562d9014b470 msgr2=0x562d90156ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T18:10:46.804 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:46.803+0000 7fc813400640 1 --2- 172.21.15.22:0/4220369546 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x562d9014b470 0x562d90156ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:10:46.804 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:46.803+0000 7fc813400640 1 -- 172.21.15.22:0/4220369546 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x562d901520d0 con 0x562d90142de0 2024-03-28T18:10:46.804 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:46.803+0000 7fc813e00640 1 --2- 172.21.15.22:0/4220369546 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562d901631e0 0x562d90151a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-28T18:10:46.804 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:46.803+0000 7fc813400640 1 --2- 172.21.15.22:0/4220369546 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562d90142de0 0x562d90156570 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7fc804029990 tx=0x7fc8040296d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=81ce15b2faaf2f6f server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:10:46.805 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:46.803+0000 7fc7fd600640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-03-28T18:10:46.807 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:46.803+0000 7fc7fd600640 1 -- 172.21.15.22:0/4220369546 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc80403f070 con 0x562d90142de0 2024-03-28T18:10:46.807 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:46.803+0000 7fc7fd600640 1 -- 172.21.15.22:0/4220369546 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 8583+0+0 (secure 0 0 0) 0x7fc804004730 con 0x562d90142de0 2024-03-28T18:10:46.807 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:46.803+0000 7fc81d083740 1 -- 172.21.15.22:0/4220369546 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fc8000c15d0 con 0x562d90142de0 2024-03-28T18:10:46.807 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:46.804+0000 7fc7fd600640 1 -- 172.21.15.22:0/4220369546 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc804029e60 con 0x562d90142de0 2024-03-28T18:10:46.807 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:46.804+0000 7fc81d083740 10 client.16509 fetch_fsmap learned FSMap version 42 2024-03-28T18:10:46.807 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:46.804+0000 7fc81d083740 1 -- 172.21.15.22:0/4220369546 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) v3 -- 0x562d901c0040 con 0x562d90142de0 2024-03-28T18:10:46.807 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:46.804+0000 7fc7fd600640 1 -- 172.21.15.22:0/4220369546 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7fc8000c15d0 con 0x562d90142de0 2024-03-28T18:10:46.808 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:46.804+0000 7fc7fd600640 1 -- 172.21.15.22:0/4220369546 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 42) v1 ==== 7954+0+0 (secure 0 0 0) 0x7fc804042050 con 0x562d90142de0 2024-03-28T18:10:46.808 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:46.804+0000 7fc81d083740 10 client.16509 fetch_fsmap finished waiting for FSMap version 42 2024-03-28T18:10:46.808 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:46.804+0000 7fc81d083740 10 client.16509 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-28T18:10:46.808 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:46.804+0000 7fc81d083740 20 client.16509 populate_metadata read hostname 'smithi022' 2024-03-28T18:10:46.808 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:46.804+0000 7fc81d083740 1 --2- 172.21.15.22:0/4220369546 >> [v2:172.21.15.79:6838/2441542443,v1:172.21.15.79:6839/2441542443] conn(0x562d901c06e0 0x562d901c2b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:10:46.808 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:46.804+0000 7fc81d083740 4 client.16509 mds_command: new command op to 34553 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-28T18:10:46.808 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:46.804+0000 7fc81d083740 1 -- 172.21.15.22:0/4220369546 --> [v2:172.21.15.79:6838/2441542443,v1:172.21.15.79:6839/2441542443] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x562d900adae0 con 0x562d901c06e0 2024-03-28T18:10:46.808 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:46.805+0000 7fc812a00640 1 --2- 172.21.15.22:0/4220369546 >> [v2:172.21.15.79:6838/2441542443,v1:172.21.15.79:6839/2441542443] conn(0x562d901c06e0 0x562d901c2b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:10:46.808 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:46.805+0000 7fc812a00640 1 --2- 172.21.15.22:0/4220369546 >> [v2:172.21.15.79:6838/2441542443,v1:172.21.15.79:6839/2441542443] conn(0x562d901c06e0 0x562d901c2b00 crc :-1 s=READY pgs=102 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2d9967302e2615f2 server_cookie=c08b6493fcc5e320 in_seq=0 out_seq=0 2024-03-28T18:10:46.808 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:46.805+0000 7fc7fd600640 10 client.16509 ms_handle_connect on v2:172.21.15.79:6838/2441542443 2024-03-28T18:10:46.808 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:46.806+0000 7fc7fd600640 1 -- 172.21.15.22:0/4220369546 <== mds.0 v2:172.21.15.79:6838/2441542443 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+267 (crc 0 0 0) 0x562d900adae0 con 0x562d901c06e0 2024-03-28T18:10:46.808 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:46.806+0000 7fc7fd600640 10 client.16509 handle_command_reply: tid=0 2024-03-28T18:10:46.808 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:46.806+0000 7fc81d083740 1 client.16509 shutdown 2024-03-28T18:10:46.808 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:46.806+0000 7fc81d083740 1 -- 172.21.15.22:0/4220369546 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562d90142de0 msgr2=0x562d90156570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:10:46.808 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:46.806+0000 7fc81d083740 1 --2- 172.21.15.22:0/4220369546 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562d90142de0 0x562d90156570 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7fc804029990 tx=0x7fc8040296d0 comp rx=0 tx=0).stop 2024-03-28T18:10:46.808 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:46.806+0000 7fc81d083740 1 -- 172.21.15.22:0/4220369546 shutdown_connections 2024-03-28T18:10:46.809 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:46.806+0000 7fc81d083740 1 --2- 172.21.15.22:0/4220369546 >> [v2:172.21.15.79:6838/2441542443,v1:172.21.15.79:6839/2441542443] conn(0x562d901c06e0 0x562d901c2b00 crc :-1 s=READY pgs=102 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:10:46.809 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:46.806+0000 7fc81d083740 1 --2- 172.21.15.22:0/4220369546 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562d901631e0 0x562d90151a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:10:46.809 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:46.806+0000 7fc81d083740 1 --2- 172.21.15.22:0/4220369546 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x562d9014b470 0x562d90156ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:10:46.809 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:46.806+0000 7fc81d083740 1 --2- 172.21.15.22:0/4220369546 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562d90142de0 0x562d90156570 unknown :-1 s=CLOSED pgs=699 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:10:46.809 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:46.806+0000 7fc7fd600640 0 client.16509 ms_handle_reset on v2:172.21.15.79:6838/2441542443 2024-03-28T18:10:46.809 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:46.806+0000 7fc81d083740 1 -- 172.21.15.22:0/4220369546 >> 172.21.15.22:0/4220369546 conn(0x562d90109200 msgr2=0x562d90147ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:10:46.809 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:46.806+0000 7fc81d083740 1 -- 172.21.15.22:0/4220369546 shutdown_connections 2024-03-28T18:10:46.809 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:46.806+0000 7fc81d083740 1 -- 172.21.15.22:0/4220369546 wait complete. 2024-03-28T18:10:46.809 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:46.806+0000 7fc81d083740 20 client.16509 trim_cache size 0 max 16384 2024-03-28T18:10:46.809 INFO:teuthology.orchestra.run.smithi022.stdout:{ 2024-03-28T18:10:46.809 INFO:teuthology.orchestra.run.smithi022.stdout: "status": "scrub active (347 inodes in the stack)", 2024-03-28T18:10:46.809 INFO:teuthology.orchestra.run.smithi022.stdout: "scrubs": { 2024-03-28T18:10:46.810 INFO:teuthology.orchestra.run.smithi022.stdout: "4bbbd61d-29a3-4bf6-a4f2-d5783d585de3": { 2024-03-28T18:10:46.810 INFO:teuthology.orchestra.run.smithi022.stdout: "path": "/", 2024-03-28T18:10:46.810 INFO:teuthology.orchestra.run.smithi022.stdout: "tag": "4bbbd61d-29a3-4bf6-a4f2-d5783d585de3", 2024-03-28T18:10:46.810 INFO:teuthology.orchestra.run.smithi022.stdout: "options": "recursive,force" 2024-03-28T18:10:46.810 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-03-28T18:10:46.810 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-03-28T18:10:46.810 INFO:teuthology.orchestra.run.smithi022.stdout:} 2024-03-28T18:10:46.811 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:46.807+0000 7fc818e00640 1 -- 172.21.15.22:0/769375606 >> [v2:172.21.15.22:6800/1357365082,v1:172.21.15.22:6801/1357365082] conn(0x7fc7f80a2970 msgr2=0x7fc7f80a4e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:10:46.811 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:46.807+0000 7fc818e00640 1 --2- 172.21.15.22:0/769375606 >> [v2:172.21.15.22:6800/1357365082,v1:172.21.15.22:6801/1357365082] conn(0x7fc7f80a2970 0x7fc7f80a4e30 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x562d90143330 tx=0x7fc804005f90 comp rx=0 tx=0).stop 2024-03-28T18:10:46.811 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:46.807+0000 7fc818e00640 1 -- 172.21.15.22:0/769375606 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc814102ff0 msgr2=0x7fc81419a560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:10:46.811 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:46.807+0000 7fc818e00640 1 --2- 172.21.15.22:0/769375606 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc814102ff0 0x7fc81419a560 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7fc800009230 tx=0x7fc800004e20 comp rx=0 tx=0).stop 2024-03-28T18:10:46.811 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:46.807+0000 7fc818e00640 1 -- 172.21.15.22:0/769375606 shutdown_connections 2024-03-28T18:10:46.811 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:46.807+0000 7fc818e00640 1 --2- 172.21.15.22:0/769375606 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc81410baf0 0x7fc81419e8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:10:46.811 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:46.807+0000 7fc818e00640 1 --2- 172.21.15.22:0/769375606 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc814102ff0 0x7fc81419a560 unknown :-1 s=CLOSED pgs=698 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:10:46.811 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:46.807+0000 7fc818e00640 1 --2- 172.21.15.22:0/769375606 >> [v2:172.21.15.22:6800/1357365082,v1:172.21.15.22:6801/1357365082] conn(0x7fc7f80a2970 0x7fc7f80a4e30 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:10:46.811 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:46.807+0000 7fc818e00640 1 --2- 172.21.15.22:0/769375606 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc8141026d0 0x7fc81419a020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:10:46.811 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:46.807+0000 7fc818e00640 1 -- 172.21.15.22:0/769375606 >> 172.21.15.22:0/769375606 conn(0x7fc8140fbef0 msgr2=0x7fc81410cd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:10:46.811 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:46.807+0000 7fc818e00640 1 -- 172.21.15.22:0/769375606 shutdown_connections 2024-03-28T18:10:46.811 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:10:46.807+0000 7fc818e00640 1 -- 172.21.15.22:0/769375606 wait complete. 2024-03-28T18:10:46.824 INFO:tasks.cephfs.filesystem:scrub status for tag:4bbbd61d-29a3-4bf6-a4f2-d5783d585de3 - {'path': '/', 'tag': '4bbbd61d-29a3-4bf6-a4f2-d5783d585de3', 'options': 'recursive,force'} 2024-03-28T18:10:47.674 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:10:47 smithi134 ceph-mon[35037]: reconnect by client.35222 172.21.15.22:0/3729116781 after 0.00299993 2024-03-28T18:10:47.674 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:10:47 smithi134 ceph-mon[35037]: mds.? [v2:172.21.15.79:6836/3437531480,v1:172.21.15.79:6837/3437531480] up:reconnect 2024-03-28T18:10:47.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:10:47 smithi134 ceph-mon[35037]: fsmap cephfs:5/5 4 up:standby-replay 3 up:standby 1 up:reconnect 4 up:active 2024-03-28T18:10:47.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:10:47 smithi134 ceph-mon[35037]: scrub queued for path: / 2024-03-28T18:10:47.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:10:47 smithi134 ceph-mon[35037]: scrub summary: idle+waiting paths [/] 2024-03-28T18:10:47.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:10:47 smithi134 ceph-mon[35037]: scrub summary: active paths [/] 2024-03-28T18:10:47.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:10:47 smithi134 ceph-mon[35037]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T18:10:47.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:10:47 smithi134 ceph-mon[35037]: mds.? [v2:172.21.15.79:6836/3437531480,v1:172.21.15.79:6837/3437531480] up:rejoin 2024-03-28T18:10:47.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:10:47 smithi134 ceph-mon[35037]: fsmap cephfs:5/5 4 up:standby-replay 3 up:standby 1 up:rejoin 4 up:active 2024-03-28T18:10:47.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:10:47 smithi079 ceph-mon[35312]: reconnect by client.35222 172.21.15.22:0/3729116781 after 0.00299993 2024-03-28T18:10:47.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:10:47 smithi079 ceph-mon[35312]: mds.? [v2:172.21.15.79:6836/3437531480,v1:172.21.15.79:6837/3437531480] up:reconnect 2024-03-28T18:10:47.683 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:10:47 smithi079 ceph-mon[35312]: fsmap cephfs:5/5 4 up:standby-replay 3 up:standby 1 up:reconnect 4 up:active 2024-03-28T18:10:47.683 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:10:47 smithi079 ceph-mon[35312]: scrub queued for path: / 2024-03-28T18:10:47.683 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:10:47 smithi079 ceph-mon[35312]: scrub summary: idle+waiting paths [/] 2024-03-28T18:10:47.683 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:10:47 smithi079 ceph-mon[35312]: scrub summary: active paths [/] 2024-03-28T18:10:47.683 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:10:47 smithi079 ceph-mon[35312]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T18:10:47.683 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:10:47 smithi079 ceph-mon[35312]: mds.? [v2:172.21.15.79:6836/3437531480,v1:172.21.15.79:6837/3437531480] up:rejoin 2024-03-28T18:10:47.683 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:10:47 smithi079 ceph-mon[35312]: fsmap cephfs:5/5 4 up:standby-replay 3 up:standby 1 up:rejoin 4 up:active 2024-03-28T18:10:47.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:10:47 smithi022 ceph-mon[31985]: reconnect by client.35222 172.21.15.22:0/3729116781 after 0.00299993 2024-03-28T18:10:47.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:10:47 smithi022 ceph-mon[31985]: mds.? [v2:172.21.15.79:6836/3437531480,v1:172.21.15.79:6837/3437531480] up:reconnect 2024-03-28T18:10:47.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:10:47 smithi022 ceph-mon[31985]: fsmap cephfs:5/5 4 up:standby-replay 3 up:standby 1 up:reconnect 4 up:active 2024-03-28T18:10:47.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:10:47 smithi022 ceph-mon[31985]: scrub queued for path: / 2024-03-28T18:10:47.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:10:47 smithi022 ceph-mon[31985]: scrub summary: idle+waiting paths [/] 2024-03-28T18:10:47.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:10:47 smithi022 ceph-mon[31985]: scrub summary: active paths [/] 2024-03-28T18:10:47.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:10:47 smithi022 ceph-mon[31985]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T18:10:47.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:10:47 smithi022 ceph-mon[31985]: mds.? [v2:172.21.15.79:6836/3437531480,v1:172.21.15.79:6837/3437531480] up:rejoin 2024-03-28T18:10:47.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:10:47 smithi022 ceph-mon[31985]: fsmap cephfs:5/5 4 up:standby-replay 3 up:standby 1 up:rejoin 4 up:active 2024-03-28T18:10:48.184 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:10:47 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69044]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-2145-g3be9a5c9/rpm/el8/BUILD/ceph-19.0.0-2145-g3be9a5c9/src/mds/Locker.cc: In function 'void Locker::set_xlocks_done(MutationImpl*, bool)' thread 7fef90600700 time 2024-03-28T18:10:47.919496+0000 2024-03-28T18:10:48.184 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:10:47 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69044]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-2145-g3be9a5c9/rpm/el8/BUILD/ceph-19.0.0-2145-g3be9a5c9/src/mds/Locker.cc: 626: FAILED ceph_assert(obj->is_auth()) 2024-03-28T18:10:48.184 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:10:47 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69044]: ceph version 19.0.0-2145-g3be9a5c9 (3be9a5c9bc793e11e2800a8c0c696e8b46742033) squid (dev) 2024-03-28T18:10:48.184 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:10:47 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69044]: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x135) [0x7fef9bfff4a1] 2024-03-28T18:10:48.184 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:10:47 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69044]: 2: /usr/lib64/ceph/libceph-common.so.2(+0x2d2667) [0x7fef9bfff667] 2024-03-28T18:10:48.184 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:10:47 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69044]: 3: (Locker::set_xlocks_done(MutationImpl*, bool)+0x493) [0x562380474223] 2024-03-28T18:10:48.184 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:10:47 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69044]: 4: (Server::early_reply(boost::intrusive_ptr const&, CInode*, CDentry*)+0x3e6) [0x5623802ee5d6] 2024-03-28T18:10:48.184 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:10:47 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69044]: 5: (Server::journal_and_reply(boost::intrusive_ptr const&, CInode*, CDentry*, LogEvent*, MDSLogContextBase*)+0xc1) [0x5623802eeda1] 2024-03-28T18:10:48.184 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:10:47 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69044]: 6: (Server::handle_client_mkdir(boost::intrusive_ptr const&)+0x5ba) [0x5623803246aa] 2024-03-28T18:10:48.184 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:10:47 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69044]: 7: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0x733) [0x562380341233] 2024-03-28T18:10:48.185 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:10:47 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69044]: 8: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x562380343197] 2024-03-28T18:10:48.185 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:10:47 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69044]: 9: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x56238024211b] 2024-03-28T18:10:48.185 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:10:47 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69044]: 10: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x5623802426bc] 2024-03-28T18:10:48.185 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:10:47 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69044]: 11: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bf) [0x56238022c1ff] 2024-03-28T18:10:48.185 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:10:47 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69044]: 12: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x478) [0x7fef9c291488] 2024-03-28T18:10:48.185 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:10:47 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69044]: 13: (DispatchQueue::entry()+0x51f) [0x7fef9c28e60f] 2024-03-28T18:10:48.185 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:10:47 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69044]: 14: (DispatchQueue::DispatchThread::entry()+0x11) [0x7fef9c361e81] 2024-03-28T18:10:48.185 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:10:47 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69044]: 15: /lib64/libpthread.so.0(+0x81ca) [0x7fef9ad7c1ca] 2024-03-28T18:10:48.185 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:10:47 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69044]: 16: clone() 2024-03-28T18:10:48.185 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:10:47 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69044]: *** Caught signal (Aborted) ** 2024-03-28T18:10:48.185 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:10:47 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69044]: in thread 7fef90600700 thread_name:ms_dispatch 2024-03-28T18:10:48.186 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:10:47 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69044]: 2024-03-28T18:10:47.921+0000 7fef90600700 -1 /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-2145-g3be9a5c9/rpm/el8/BUILD/ceph-19.0.0-2145-g3be9a5c9/src/mds/Locker.cc: In function 'void Locker::set_xlocks_done(MutationImpl*, bool)' thread 7fef90600700 time 2024-03-28T18:10:47.919496+0000 2024-03-28T18:10:48.186 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:10:47 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69044]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-2145-g3be9a5c9/rpm/el8/BUILD/ceph-19.0.0-2145-g3be9a5c9/src/mds/Locker.cc: 626: FAILED ceph_assert(obj->is_auth()) 2024-03-28T18:10:48.186 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:10:47 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69044]: 2024-03-28T18:10:48.186 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:10:47 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69044]: ceph version 19.0.0-2145-g3be9a5c9 (3be9a5c9bc793e11e2800a8c0c696e8b46742033) squid (dev) 2024-03-28T18:10:48.187 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:10:47 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69044]: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x135) [0x7fef9bfff4a1] 2024-03-28T18:10:48.187 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:10:47 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69044]: 2: /usr/lib64/ceph/libceph-common.so.2(+0x2d2667) [0x7fef9bfff667] 2024-03-28T18:10:48.187 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:10:47 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69044]: 3: (Locker::set_xlocks_done(MutationImpl*, bool)+0x493) [0x562380474223] 2024-03-28T18:10:48.187 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:10:47 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69044]: 4: (Server::early_reply(boost::intrusive_ptr const&, CInode*, CDentry*)+0x3e6) [0x5623802ee5d6] 2024-03-28T18:10:48.187 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:10:47 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69044]: 5: (Server::journal_and_reply(boost::intrusive_ptr const&, CInode*, CDentry*, LogEvent*, MDSLogContextBase*)+0xc1) [0x5623802eeda1] 2024-03-28T18:10:48.187 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:10:47 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69044]: 6: (Server::handle_client_mkdir(boost::intrusive_ptr const&)+0x5ba) [0x5623803246aa] 2024-03-28T18:10:48.187 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:10:47 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69044]: 7: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0x733) [0x562380341233] 2024-03-28T18:10:48.187 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:10:47 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69044]: 8: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x562380343197] 2024-03-28T18:10:48.187 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:10:47 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69044]: 9: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x56238024211b] 2024-03-28T18:10:48.187 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:10:47 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69044]: 10: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x5623802426bc] 2024-03-28T18:10:48.187 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:10:47 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69044]: 11: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bf) [0x56238022c1ff] 2024-03-28T18:10:48.187 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:10:47 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69044]: 12: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x478) [0x7fef9c291488] 2024-03-28T18:10:48.187 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:10:47 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69044]: 13: (DispatchQueue::entry()+0x51f) [0x7fef9c28e60f] 2024-03-28T18:10:48.187 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:10:47 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69044]: 14: (DispatchQueue::DispatchThread::entry()+0x11) [0x7fef9c361e81] 2024-03-28T18:10:48.188 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:10:47 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69044]: 15: /lib64/libpthread.so.0(+0x81ca) [0x7fef9ad7c1ca] 2024-03-28T18:10:48.188 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:10:47 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69044]: 16: clone() 2024-03-28T18:10:48.188 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:10:47 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69044]: 2024-03-28T18:10:48.188 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:10:47 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69044]: ceph version 19.0.0-2145-g3be9a5c9 (3be9a5c9bc793e11e2800a8c0c696e8b46742033) squid (dev) 2024-03-28T18:10:48.188 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:10:47 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69044]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7fef9ad86d20] 2024-03-28T18:10:48.188 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:10:47 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69044]: 2: gsignal() 2024-03-28T18:10:48.188 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:10:47 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69044]: 3: abort() 2024-03-28T18:10:48.188 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:10:47 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69044]: 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x18f) [0x7fef9bfff4fb] 2024-03-28T18:10:48.189 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:10:47 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69044]: 5: /usr/lib64/ceph/libceph-common.so.2(+0x2d2667) [0x7fef9bfff667] 2024-03-28T18:10:48.189 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:10:47 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69044]: 6: (Locker::set_xlocks_done(MutationImpl*, bool)+0x493) [0x562380474223] 2024-03-28T18:10:48.189 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:10:47 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69044]: 7: (Server::early_reply(boost::intrusive_ptr const&, CInode*, CDentry*)+0x3e6) [0x5623802ee5d6] 2024-03-28T18:10:48.189 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:10:47 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69044]: 8: (Server::journal_and_reply(boost::intrusive_ptr const&, CInode*, CDentry*, LogEvent*, MDSLogContextBase*)+0xc1) [0x5623802eeda1] 2024-03-28T18:10:48.189 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:10:47 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69044]: 9: (Server::handle_client_mkdir(boost::intrusive_ptr const&)+0x5ba) [0x5623803246aa] 2024-03-28T18:10:48.189 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:10:47 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69044]: 10: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0x733) [0x562380341233] 2024-03-28T18:10:48.189 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:10:47 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69044]: 11: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x562380343197] 2024-03-28T18:10:48.189 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:10:47 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69044]: 12: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x56238024211b] 2024-03-28T18:10:48.189 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:10:47 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69044]: 13: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x5623802426bc] 2024-03-28T18:10:48.189 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:10:47 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69044]: 14: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bf) [0x56238022c1ff] 2024-03-28T18:10:48.189 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:10:47 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69044]: 15: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x478) [0x7fef9c291488] 2024-03-28T18:10:48.189 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:10:47 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69044]: 16: (DispatchQueue::entry()+0x51f) [0x7fef9c28e60f] 2024-03-28T18:10:48.189 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:10:47 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69044]: 17: (DispatchQueue::DispatchThread::entry()+0x11) [0x7fef9c361e81] 2024-03-28T18:10:48.189 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:10:47 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69044]: 18: /lib64/libpthread.so.0(+0x81ca) [0x7fef9ad7c1ca] 2024-03-28T18:10:48.190 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:10:47 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69044]: 19: clone() 2024-03-28T18:10:48.190 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:10:47 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69044]: 2024-03-28T18:10:47.924+0000 7fef90600700 -1 *** Caught signal (Aborted) ** 2024-03-28T18:10:48.190 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:10:47 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69044]: in thread 7fef90600700 thread_name:ms_dispatch 2024-03-28T18:10:48.190 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:10:47 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69044]: 2024-03-28T18:10:48.190 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:10:47 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69044]: ceph version 19.0.0-2145-g3be9a5c9 (3be9a5c9bc793e11e2800a8c0c696e8b46742033) squid (dev) 2024-03-28T18:10:48.190 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:10:47 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69044]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7fef9ad86d20] 2024-03-28T18:10:48.190 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:10:47 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69044]: 2: gsignal() 2024-03-28T18:10:48.190 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:10:47 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69044]: 3: abort() 2024-03-28T18:10:48.190 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:10:47 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69044]: 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x18f) [0x7fef9bfff4fb] 2024-03-28T18:10:48.190 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:10:47 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69044]: 5: /usr/lib64/ceph/libceph-common.so.2(+0x2d2667) [0x7fef9bfff667] 2024-03-28T18:10:48.190 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:10:47 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69044]: 6: (Locker::set_xlocks_done(MutationImpl*, bool)+0x493) [0x562380474223] 2024-03-28T18:10:48.190 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:10:47 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69044]: 7: (Server::early_reply(boost::intrusive_ptr const&, CInode*, CDentry*)+0x3e6) [0x5623802ee5d6] 2024-03-28T18:10:48.190 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:10:47 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69044]: 8: (Server::journal_and_reply(boost::intrusive_ptr const&, CInode*, CDentry*, LogEvent*, MDSLogContextBase*)+0xc1) [0x5623802eeda1] 2024-03-28T18:10:48.190 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:10:47 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69044]: 9: (Server::handle_client_mkdir(boost::intrusive_ptr const&)+0x5ba) [0x5623803246aa] 2024-03-28T18:10:48.190 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:10:47 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69044]: 10: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0x733) [0x562380341233] 2024-03-28T18:10:48.191 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:10:47 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69044]: 11: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x562380343197] 2024-03-28T18:10:48.191 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:10:47 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69044]: 12: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x56238024211b] 2024-03-28T18:10:48.191 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:10:47 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69044]: 13: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x5623802426bc] 2024-03-28T18:10:48.191 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:10:47 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69044]: 14: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bf) [0x56238022c1ff] 2024-03-28T18:10:48.191 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:10:47 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69044]: 15: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x478) [0x7fef9c291488] 2024-03-28T18:10:48.191 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:10:47 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69044]: 16: (DispatchQueue::entry()+0x51f) [0x7fef9c28e60f] 2024-03-28T18:10:48.191 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:10:47 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69044]: 17: (DispatchQueue::DispatchThread::entry()+0x11) [0x7fef9c361e81] 2024-03-28T18:10:48.191 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:10:47 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69044]: 18: /lib64/libpthread.so.0(+0x81ca) [0x7fef9ad7c1ca] 2024-03-28T18:10:48.191 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:10:47 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69044]: 19: clone() 2024-03-28T18:10:48.191 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:10:47 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69044]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-03-28T18:10:48.191 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:10:47 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69044]: 2024-03-28T18:10:48.191 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:10:47 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69044]: -1> 2024-03-28T18:10:47.921+0000 7fef90600700 -1 /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-2145-g3be9a5c9/rpm/el8/BUILD/ceph-19.0.0-2145-g3be9a5c9/src/mds/Locker.cc: In function 'void Locker::set_xlocks_done(MutationImpl*, bool)' thread 7fef90600700 time 2024-03-28T18:10:47.919496+0000 2024-03-28T18:10:48.191 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:10:47 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69044]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-2145-g3be9a5c9/rpm/el8/BUILD/ceph-19.0.0-2145-g3be9a5c9/src/mds/Locker.cc: 626: FAILED ceph_assert(obj->is_auth()) 2024-03-28T18:10:48.191 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:10:47 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69044]: 2024-03-28T18:10:48.191 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:10:47 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69044]: ceph version 19.0.0-2145-g3be9a5c9 (3be9a5c9bc793e11e2800a8c0c696e8b46742033) squid (dev) 2024-03-28T18:10:48.192 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:10:47 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69044]: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x135) [0x7fef9bfff4a1] 2024-03-28T18:10:48.192 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:10:47 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69044]: 2: /usr/lib64/ceph/libceph-common.so.2(+0x2d2667) [0x7fef9bfff667] 2024-03-28T18:10:48.192 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:10:47 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69044]: 3: (Locker::set_xlocks_done(MutationImpl*, bool)+0x493) [0x562380474223] 2024-03-28T18:10:48.192 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:10:47 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69044]: 4: (Server::early_reply(boost::intrusive_ptr const&, CInode*, CDentry*)+0x3e6) [0x5623802ee5d6] 2024-03-28T18:10:48.192 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:10:47 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69044]: 5: (Server::journal_and_reply(boost::intrusive_ptr const&, CInode*, CDentry*, LogEvent*, MDSLogContextBase*)+0xc1) [0x5623802eeda1] 2024-03-28T18:10:48.192 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:10:47 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69044]: 6: (Server::handle_client_mkdir(boost::intrusive_ptr const&)+0x5ba) [0x5623803246aa] 2024-03-28T18:10:48.193 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:10:47 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69044]: 7: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0x733) [0x562380341233] 2024-03-28T18:10:48.193 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:10:47 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69044]: 8: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x562380343197] 2024-03-28T18:10:48.193 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:10:47 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69044]: 9: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x56238024211b] 2024-03-28T18:10:48.193 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:10:47 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69044]: 10: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x5623802426bc] 2024-03-28T18:10:48.193 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:10:47 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69044]: 11: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bf) [0x56238022c1ff] 2024-03-28T18:10:48.193 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:10:47 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69044]: 12: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x478) [0x7fef9c291488] 2024-03-28T18:10:48.193 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:10:47 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69044]: 13: (DispatchQueue::entry()+0x51f) [0x7fef9c28e60f] 2024-03-28T18:10:48.193 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:10:47 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69044]: 14: (DispatchQueue::DispatchThread::entry()+0x11) [0x7fef9c361e81] 2024-03-28T18:10:48.193 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:10:47 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69044]: 15: /lib64/libpthread.so.0(+0x81ca) [0x7fef9ad7c1ca] 2024-03-28T18:10:48.193 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:10:47 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69044]: 16: clone() 2024-03-28T18:10:48.193 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:10:47 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69044]: 2024-03-28T18:10:48.193 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:10:47 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69044]: 0> 2024-03-28T18:10:47.924+0000 7fef90600700 -1 *** Caught signal (Aborted) ** 2024-03-28T18:10:48.193 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:10:47 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69044]: in thread 7fef90600700 thread_name:ms_dispatch 2024-03-28T18:10:48.193 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:10:47 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69044]: 2024-03-28T18:10:48.194 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:10:47 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69044]: ceph version 19.0.0-2145-g3be9a5c9 (3be9a5c9bc793e11e2800a8c0c696e8b46742033) squid (dev) 2024-03-28T18:10:48.194 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:10:47 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69044]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7fef9ad86d20] 2024-03-28T18:10:48.194 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:10:47 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69044]: 2: gsignal() 2024-03-28T18:10:48.194 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:10:47 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69044]: 3: abort() 2024-03-28T18:10:48.194 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:10:47 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69044]: 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x18f) [0x7fef9bfff4fb] 2024-03-28T18:10:48.194 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:10:47 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69044]: 5: /usr/lib64/ceph/libceph-common.so.2(+0x2d2667) [0x7fef9bfff667] 2024-03-28T18:10:48.194 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:10:47 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69044]: 6: (Locker::set_xlocks_done(MutationImpl*, bool)+0x493) [0x562380474223] 2024-03-28T18:10:48.194 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:10:47 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69044]: 7: (Server::early_reply(boost::intrusive_ptr const&, CInode*, CDentry*)+0x3e6) [0x5623802ee5d6] 2024-03-28T18:10:48.194 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:10:47 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69044]: 8: (Server::journal_and_reply(boost::intrusive_ptr const&, CInode*, CDentry*, LogEvent*, MDSLogContextBase*)+0xc1) [0x5623802eeda1] 2024-03-28T18:10:48.194 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:10:47 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69044]: 9: (Server::handle_client_mkdir(boost::intrusive_ptr const&)+0x5ba) [0x5623803246aa] 2024-03-28T18:10:48.194 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:10:47 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69044]: 10: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0x733) [0x562380341233] 2024-03-28T18:10:48.194 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:10:47 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69044]: 11: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x562380343197] 2024-03-28T18:10:48.194 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:10:47 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69044]: 12: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x56238024211b] 2024-03-28T18:10:48.194 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:10:47 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69044]: 13: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x5623802426bc] 2024-03-28T18:10:48.195 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:10:47 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69044]: 14: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bf) [0x56238022c1ff] 2024-03-28T18:10:48.195 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:10:47 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69044]: 15: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x478) [0x7fef9c291488] 2024-03-28T18:10:48.195 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:10:47 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69044]: 16: (DispatchQueue::entry()+0x51f) [0x7fef9c28e60f] 2024-03-28T18:10:48.195 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:10:47 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69044]: 17: (DispatchQueue::DispatchThread::entry()+0x11) [0x7fef9c361e81] 2024-03-28T18:10:48.195 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:10:47 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69044]: 18: /lib64/libpthread.so.0(+0x81ca) [0x7fef9ad7c1ca] 2024-03-28T18:10:48.195 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:10:47 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69044]: 19: clone() 2024-03-28T18:10:48.195 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:10:47 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69044]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-03-28T18:10:48.195 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:10:47 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69044]: 2024-03-28T18:10:48.195 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:10:47 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69044]: -1> 2024-03-28T18:10:47.921+0000 7fef90600700 -1 /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-2145-g3be9a5c9/rpm/el8/BUILD/ceph-19.0.0-2145-g3be9a5c9/src/mds/Locker.cc: In function 'void Locker::set_xlocks_done(MutationImpl*, bool)' thread 7fef90600700 time 2024-03-28T18:10:47.919496+0000 2024-03-28T18:10:48.195 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:10:47 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69044]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-2145-g3be9a5c9/rpm/el8/BUILD/ceph-19.0.0-2145-g3be9a5c9/src/mds/Locker.cc: 626: FAILED ceph_assert(obj->is_auth()) 2024-03-28T18:10:48.195 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:10:47 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69044]: 2024-03-28T18:10:48.195 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:10:47 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69044]: ceph version 19.0.0-2145-g3be9a5c9 (3be9a5c9bc793e11e2800a8c0c696e8b46742033) squid (dev) 2024-03-28T18:10:48.195 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:10:47 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69044]: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x135) [0x7fef9bfff4a1] 2024-03-28T18:10:48.195 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:10:47 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69044]: 2: /usr/lib64/ceph/libceph-common.so.2(+0x2d2667) [0x7fef9bfff667] 2024-03-28T18:10:48.196 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:10:47 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69044]: 3: (Locker::set_xlocks_done(MutationImpl*, bool)+0x493) [0x562380474223] 2024-03-28T18:10:48.196 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:10:47 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69044]: 4: (Server::early_reply(boost::intrusive_ptr const&, CInode*, CDentry*)+0x3e6) [0x5623802ee5d6] 2024-03-28T18:10:48.196 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:10:47 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69044]: 5: (Server::journal_and_reply(boost::intrusive_ptr const&, CInode*, CDentry*, LogEvent*, MDSLogContextBase*)+0xc1) [0x5623802eeda1] 2024-03-28T18:10:48.196 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:10:47 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69044]: 6: (Server::handle_client_mkdir(boost::intrusive_ptr const&)+0x5ba) [0x5623803246aa] 2024-03-28T18:10:48.196 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:10:47 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69044]: 7: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0x733) [0x562380341233] 2024-03-28T18:10:48.196 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:10:47 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69044]: 8: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x562380343197] 2024-03-28T18:10:48.196 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:10:47 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69044]: 9: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x56238024211b] 2024-03-28T18:10:48.196 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:10:47 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69044]: 10: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x5623802426bc] 2024-03-28T18:10:48.196 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:10:47 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69044]: 11: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bf) [0x56238022c1ff] 2024-03-28T18:10:48.196 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:10:47 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69044]: 12: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x478) [0x7fef9c291488] 2024-03-28T18:10:48.196 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:10:47 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69044]: 13: (DispatchQueue::entry()+0x51f) [0x7fef9c28e60f] 2024-03-28T18:10:48.196 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:10:47 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69044]: 14: (DispatchQueue::DispatchThread::entry()+0x11) [0x7fef9c361e81] 2024-03-28T18:10:48.196 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:10:47 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69044]: 15: /lib64/libpthread.so.0(+0x81ca) [0x7fef9ad7c1ca] 2024-03-28T18:10:48.196 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:10:47 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69044]: 16: clone() 2024-03-28T18:10:48.196 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:10:47 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69044]: 2024-03-28T18:10:48.197 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:10:47 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69044]: 0> 2024-03-28T18:10:47.924+0000 7fef90600700 -1 *** Caught signal (Aborted) ** 2024-03-28T18:10:48.197 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:10:47 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69044]: in thread 7fef90600700 thread_name:ms_dispatch 2024-03-28T18:10:48.197 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:10:47 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69044]: 2024-03-28T18:10:48.197 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:10:47 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69044]: ceph version 19.0.0-2145-g3be9a5c9 (3be9a5c9bc793e11e2800a8c0c696e8b46742033) squid (dev) 2024-03-28T18:10:48.197 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:10:47 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69044]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7fef9ad86d20] 2024-03-28T18:10:48.197 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:10:47 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69044]: 2: gsignal() 2024-03-28T18:10:48.197 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:10:47 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69044]: 3: abort() 2024-03-28T18:10:48.197 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:10:47 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69044]: 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x18f) [0x7fef9bfff4fb] 2024-03-28T18:10:48.197 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:10:47 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69044]: 5: /usr/lib64/ceph/libceph-common.so.2(+0x2d2667) [0x7fef9bfff667] 2024-03-28T18:10:48.197 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:10:47 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69044]: 6: (Locker::set_xlocks_done(MutationImpl*, bool)+0x493) [0x562380474223] 2024-03-28T18:10:48.197 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:10:47 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69044]: 7: (Server::early_reply(boost::intrusive_ptr const&, CInode*, CDentry*)+0x3e6) [0x5623802ee5d6] 2024-03-28T18:10:48.197 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:10:47 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69044]: 8: (Server::journal_and_reply(boost::intrusive_ptr const&, CInode*, CDentry*, LogEvent*, MDSLogContextBase*)+0xc1) [0x5623802eeda1] 2024-03-28T18:10:48.197 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:10:47 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69044]: 9: (Server::handle_client_mkdir(boost::intrusive_ptr const&)+0x5ba) [0x5623803246aa] 2024-03-28T18:10:48.197 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:10:47 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69044]: 10: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0x733) [0x562380341233] 2024-03-28T18:10:48.198 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:10:47 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69044]: 11: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x562380343197] 2024-03-28T18:10:48.198 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:10:47 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69044]: 12: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x56238024211b] 2024-03-28T18:10:48.198 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:10:47 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69044]: 13: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x5623802426bc] 2024-03-28T18:10:48.198 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:10:47 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69044]: 14: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bf) [0x56238022c1ff] 2024-03-28T18:10:48.198 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:10:47 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69044]: 15: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x478) [0x7fef9c291488] 2024-03-28T18:10:48.199 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:10:47 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69044]: 16: (DispatchQueue::entry()+0x51f) [0x7fef9c28e60f] 2024-03-28T18:10:48.199 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:10:47 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69044]: 17: (DispatchQueue::DispatchThread::entry()+0x11) [0x7fef9c361e81] 2024-03-28T18:10:48.199 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:10:47 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69044]: 18: /lib64/libpthread.so.0(+0x81ca) [0x7fef9ad7c1ca] 2024-03-28T18:10:48.199 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:10:47 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69044]: 19: clone() 2024-03-28T18:10:48.199 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:10:47 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69044]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-03-28T18:10:48.199 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:10:47 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69044]: 2024-03-28T18:10:48.199 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:10:48 smithi079 podman[69179]: 2024-03-28 18:10:48.086730461 +0000 UTC m=+0.054035195 container died b14c9698ded52199d9da0731ae67bf00eebbde5d68374ad20645c8e74a333d28 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240319, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0) 2024-03-28T18:10:48.676 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:10:48 smithi134 ceph-mon[35037]: pgmap v102: 97 pgs: 97 active+clean; 117 MiB data, 799 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:10:48.676 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:10:48 smithi134 ceph-mon[35037]: daemon mds.h is now active in filesystem cephfs as rank 1 2024-03-28T18:10:48.676 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:10:48 smithi134 ceph-mon[35037]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-03-28T18:10:48.676 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:10:48 smithi134 ceph-mon[35037]: Cluster is now healthy 2024-03-28T18:10:48.676 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:10:48 smithi134 ceph-mon[35037]: scrub summary: active paths [/] 2024-03-28T18:10:48.676 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:10:48 smithi134 ceph-mon[35037]: mds.? [v2:172.21.15.79:6836/3437531480,v1:172.21.15.79:6837/3437531480] up:active 2024-03-28T18:10:48.676 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:10:48 smithi134 ceph-mon[35037]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-28T18:10:48.676 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:10:48 smithi134 ceph-mon[35037]: scrub summary: idle+waiting paths [/] 2024-03-28T18:10:48.676 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:10:48 smithi134 ceph-mon[35037]: scrub summary: active paths [/] 2024-03-28T18:10:48.677 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:10:48 smithi134 ceph-mon[35037]: scrub summary: idle+waiting paths [/] 2024-03-28T18:10:48.677 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:10:48 smithi134 ceph-mon[35037]: scrub summary: active paths [/] 2024-03-28T18:10:48.677 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:10:48 smithi134 ceph-mon[35037]: scrub summary: active paths [/] 2024-03-28T18:10:48.677 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:10:48 smithi134 ceph-mon[35037]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T18:10:48.677 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:10:48 smithi134 ceph-mon[35037]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T18:10:48.677 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:10:48 smithi134 ceph-mon[35037]: from='mgr.15522 ' entity='mgr.x' 2024-03-28T18:10:48.677 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:10:48 smithi134 ceph-mon[35037]: scrub summary: idle+waiting paths [/] 2024-03-28T18:10:48.677 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:10:48 smithi134 ceph-mon[35037]: scrub summary: active paths [/] 2024-03-28T18:10:48.677 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:10:48 smithi134 ceph-mon[35037]: scrub summary: idle+waiting paths [/] 2024-03-28T18:10:48.677 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:10:48 smithi134 ceph-mon[35037]: scrub summary: active paths [/] 2024-03-28T18:10:48.677 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:10:48 smithi134 ceph-mon[35037]: scrub summary: idle+waiting paths [/] 2024-03-28T18:10:48.677 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:10:48 smithi134 ceph-mon[35037]: scrub summary: active paths [/] 2024-03-28T18:10:48.677 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:10:48 smithi134 ceph-mon[35037]: scrub summary: idle+waiting paths [/] 2024-03-28T18:10:48.678 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:10:48 smithi134 ceph-mon[35037]: scrub summary: active paths [/] 2024-03-28T18:10:48.678 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:10:48 smithi134 ceph-mon[35037]: scrub summary: idle+waiting paths [/] 2024-03-28T18:10:48.678 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:10:48 smithi134 ceph-mon[35037]: scrub summary: active paths [/] 2024-03-28T18:10:48.678 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:10:48 smithi134 ceph-mon[35037]: scrub summary: idle+waiting paths [/] 2024-03-28T18:10:48.678 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:10:48 smithi134 ceph-mon[35037]: scrub summary: active paths [/] 2024-03-28T18:10:48.678 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:10:48 smithi134 ceph-mon[35037]: scrub summary: idle+waiting paths [/] 2024-03-28T18:10:48.678 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:10:48 smithi134 ceph-mon[35037]: scrub summary: active paths [/] 2024-03-28T18:10:48.678 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:10:48 smithi134 ceph-mon[35037]: scrub summary: idle+waiting paths [/] 2024-03-28T18:10:48.678 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:10:48 smithi134 ceph-mon[35037]: scrub summary: active paths [/] 2024-03-28T18:10:48.678 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:10:48 smithi134 ceph-mon[35037]: scrub summary: idle+waiting paths [/] 2024-03-28T18:10:48.678 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:10:48 smithi134 ceph-mon[35037]: scrub summary: active paths [/] 2024-03-28T18:10:48.678 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:10:48 smithi134 ceph-mon[35037]: scrub summary: idle+waiting paths [/] 2024-03-28T18:10:48.679 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:10:48 smithi134 ceph-mon[35037]: scrub summary: idle+waiting paths [/] 2024-03-28T18:10:48.684 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:10:48 smithi079 ceph-mon[35312]: pgmap v102: 97 pgs: 97 active+clean; 117 MiB data, 799 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:10:48.684 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:10:48 smithi079 ceph-mon[35312]: daemon mds.h is now active in filesystem cephfs as rank 1 2024-03-28T18:10:48.684 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:10:48 smithi079 ceph-mon[35312]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-03-28T18:10:48.684 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:10:48 smithi079 ceph-mon[35312]: Cluster is now healthy 2024-03-28T18:10:48.684 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:10:48 smithi079 ceph-mon[35312]: scrub summary: active paths [/] 2024-03-28T18:10:48.684 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:10:48 smithi079 ceph-mon[35312]: mds.? [v2:172.21.15.79:6836/3437531480,v1:172.21.15.79:6837/3437531480] up:active 2024-03-28T18:10:48.684 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:10:48 smithi079 ceph-mon[35312]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-28T18:10:48.685 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:10:48 smithi079 ceph-mon[35312]: scrub summary: idle+waiting paths [/] 2024-03-28T18:10:48.685 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:10:48 smithi079 ceph-mon[35312]: scrub summary: active paths [/] 2024-03-28T18:10:48.685 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:10:48 smithi079 ceph-mon[35312]: scrub summary: idle+waiting paths [/] 2024-03-28T18:10:48.685 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:10:48 smithi079 ceph-mon[35312]: scrub summary: active paths [/] 2024-03-28T18:10:48.685 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:10:48 smithi079 ceph-mon[35312]: scrub summary: active paths [/] 2024-03-28T18:10:48.685 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:10:48 smithi079 ceph-mon[35312]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T18:10:48.685 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:10:48 smithi079 ceph-mon[35312]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T18:10:48.685 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:10:48 smithi079 ceph-mon[35312]: from='mgr.15522 ' entity='mgr.x' 2024-03-28T18:10:48.685 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:10:48 smithi079 ceph-mon[35312]: scrub summary: idle+waiting paths [/] 2024-03-28T18:10:48.685 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:10:48 smithi079 ceph-mon[35312]: scrub summary: active paths [/] 2024-03-28T18:10:48.685 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:10:48 smithi079 ceph-mon[35312]: scrub summary: idle+waiting paths [/] 2024-03-28T18:10:48.685 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:10:48 smithi079 ceph-mon[35312]: scrub summary: active paths [/] 2024-03-28T18:10:48.685 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:10:48 smithi079 ceph-mon[35312]: scrub summary: idle+waiting paths [/] 2024-03-28T18:10:48.686 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:10:48 smithi079 ceph-mon[35312]: scrub summary: active paths [/] 2024-03-28T18:10:48.686 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:10:48 smithi079 ceph-mon[35312]: scrub summary: idle+waiting paths [/] 2024-03-28T18:10:48.686 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:10:48 smithi079 ceph-mon[35312]: scrub summary: active paths [/] 2024-03-28T18:10:48.686 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:10:48 smithi079 ceph-mon[35312]: scrub summary: idle+waiting paths [/] 2024-03-28T18:10:48.686 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:10:48 smithi079 ceph-mon[35312]: scrub summary: active paths [/] 2024-03-28T18:10:48.686 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:10:48 smithi079 ceph-mon[35312]: scrub summary: idle+waiting paths [/] 2024-03-28T18:10:48.686 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:10:48 smithi079 ceph-mon[35312]: scrub summary: active paths [/] 2024-03-28T18:10:48.686 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:10:48 smithi079 ceph-mon[35312]: scrub summary: idle+waiting paths [/] 2024-03-28T18:10:48.686 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:10:48 smithi079 ceph-mon[35312]: scrub summary: active paths [/] 2024-03-28T18:10:48.686 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:10:48 smithi079 ceph-mon[35312]: scrub summary: idle+waiting paths [/] 2024-03-28T18:10:48.686 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:10:48 smithi079 ceph-mon[35312]: scrub summary: active paths [/] 2024-03-28T18:10:48.686 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:10:48 smithi079 ceph-mon[35312]: scrub summary: idle+waiting paths [/] 2024-03-28T18:10:48.687 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:10:48 smithi079 ceph-mon[35312]: scrub summary: active paths [/] 2024-03-28T18:10:48.687 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:10:48 smithi079 ceph-mon[35312]: scrub summary: idle+waiting paths [/] 2024-03-28T18:10:48.687 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:10:48 smithi079 ceph-mon[35312]: scrub summary: idle+waiting paths [/] 2024-03-28T18:10:48.783 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:10:48 smithi022 ceph-mon[31985]: pgmap v102: 97 pgs: 97 active+clean; 117 MiB data, 799 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:10:48.783 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:10:48 smithi022 ceph-mon[31985]: daemon mds.h is now active in filesystem cephfs as rank 1 2024-03-28T18:10:48.783 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:10:48 smithi022 ceph-mon[31985]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-03-28T18:10:48.783 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:10:48 smithi022 ceph-mon[31985]: Cluster is now healthy 2024-03-28T18:10:48.783 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:10:48 smithi022 ceph-mon[31985]: scrub summary: active paths [/] 2024-03-28T18:10:48.783 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:10:48 smithi022 ceph-mon[31985]: mds.? [v2:172.21.15.79:6836/3437531480,v1:172.21.15.79:6837/3437531480] up:active 2024-03-28T18:10:48.783 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:10:48 smithi022 ceph-mon[31985]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-28T18:10:48.783 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:10:48 smithi022 ceph-mon[31985]: scrub summary: idle+waiting paths [/] 2024-03-28T18:10:48.783 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:10:48 smithi022 ceph-mon[31985]: scrub summary: active paths [/] 2024-03-28T18:10:48.784 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:10:48 smithi022 ceph-mon[31985]: scrub summary: idle+waiting paths [/] 2024-03-28T18:10:48.784 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:10:48 smithi022 ceph-mon[31985]: scrub summary: active paths [/] 2024-03-28T18:10:48.784 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:10:48 smithi022 ceph-mon[31985]: scrub summary: active paths [/] 2024-03-28T18:10:48.784 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:10:48 smithi022 ceph-mon[31985]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T18:10:48.784 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:10:48 smithi022 ceph-mon[31985]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T18:10:48.784 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:10:48 smithi022 ceph-mon[31985]: from='mgr.15522 ' entity='mgr.x' 2024-03-28T18:10:48.784 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:10:48 smithi022 ceph-mon[31985]: scrub summary: idle+waiting paths [/] 2024-03-28T18:10:48.784 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:10:48 smithi022 ceph-mon[31985]: scrub summary: active paths [/] 2024-03-28T18:10:48.784 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:10:48 smithi022 ceph-mon[31985]: scrub summary: idle+waiting paths [/] 2024-03-28T18:10:48.784 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:10:48 smithi022 ceph-mon[31985]: scrub summary: active paths [/] 2024-03-28T18:10:48.784 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:10:48 smithi022 ceph-mon[31985]: scrub summary: idle+waiting paths [/] 2024-03-28T18:10:48.784 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:10:48 smithi022 ceph-mon[31985]: scrub summary: active paths [/] 2024-03-28T18:10:48.785 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:10:48 smithi022 ceph-mon[31985]: scrub summary: idle+waiting paths [/] 2024-03-28T18:10:48.785 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:10:48 smithi022 ceph-mon[31985]: scrub summary: active paths [/] 2024-03-28T18:10:48.785 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:10:48 smithi022 ceph-mon[31985]: scrub summary: idle+waiting paths [/] 2024-03-28T18:10:48.785 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:10:48 smithi022 ceph-mon[31985]: scrub summary: active paths [/] 2024-03-28T18:10:48.785 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:10:48 smithi022 ceph-mon[31985]: scrub summary: idle+waiting paths [/] 2024-03-28T18:10:48.785 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:10:48 smithi022 ceph-mon[31985]: scrub summary: active paths [/] 2024-03-28T18:10:48.785 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:10:48 smithi022 ceph-mon[31985]: scrub summary: idle+waiting paths [/] 2024-03-28T18:10:48.785 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:10:48 smithi022 ceph-mon[31985]: scrub summary: active paths [/] 2024-03-28T18:10:48.785 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:10:48 smithi022 ceph-mon[31985]: scrub summary: idle+waiting paths [/] 2024-03-28T18:10:48.785 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:10:48 smithi022 ceph-mon[31985]: scrub summary: active paths [/] 2024-03-28T18:10:48.785 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:10:48 smithi022 ceph-mon[31985]: scrub summary: idle+waiting paths [/] 2024-03-28T18:10:48.785 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:10:48 smithi022 ceph-mon[31985]: scrub summary: active paths [/] 2024-03-28T18:10:48.785 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:10:48 smithi022 ceph-mon[31985]: scrub summary: idle+waiting paths [/] 2024-03-28T18:10:48.786 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:10:48 smithi022 ceph-mon[31985]: scrub summary: idle+waiting paths [/] 2024-03-28T18:10:49.078 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:10:48 smithi079 podman[69179]: 2024-03-28 18:10:48.831815248 +0000 UTC m=+0.799119978 container remove b14c9698ded52199d9da0731ae67bf00eebbde5d68374ad20645c8e74a333d28 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h, ceph=True, maintainer=Guillaume Abrioux , GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.build-date=20240319, org.label-schema.schema-version=1.0, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_BRANCH=HEAD) 2024-03-28T18:10:49.078 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:10:48 smithi079 systemd[1]: ceph-fd220748-ed2b-11ee-95d0-87774f69a715@mds.h.service: Main process exited, code=exited, status=134/n/a 2024-03-28T18:10:49.359 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:10:49 smithi079 systemd[1]: ceph-fd220748-ed2b-11ee-95d0-87774f69a715@mds.h.service: Failed with result 'exit-code'. 2024-03-28T18:10:49.359 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:10:49 smithi079 systemd[1]: ceph-fd220748-ed2b-11ee-95d0-87774f69a715@mds.h.service: Consumed 1.778s CPU time. 2024-03-28T18:10:49.359 INFO:journalctl@ceph.mds.b.smithi079.stdout:Mar 28 18:10:49 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-b[65245]: 2024-03-28T18:10:49.077+0000 7f845f400700 -1 mds.pinger is_rank_lagging: rank=1 was never sent ping request. 2024-03-28T18:10:49.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:10:49 smithi134 ceph-mon[35037]: scrub summary: active paths [/] 2024-03-28T18:10:49.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:10:49 smithi134 ceph-mon[35037]: scrub summary: idle+waiting paths [/] 2024-03-28T18:10:49.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:10:49 smithi134 ceph-mon[35037]: scrub summary: active paths [/] 2024-03-28T18:10:49.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:10:49 smithi134 ceph-mon[35037]: scrub summary: idle+waiting paths [/] 2024-03-28T18:10:49.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:10:49 smithi134 ceph-mon[35037]: scrub summary: active paths [/] 2024-03-28T18:10:49.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:10:49 smithi134 ceph-mon[35037]: scrub summary: idle+waiting paths [/] 2024-03-28T18:10:49.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:10:49 smithi079 ceph-mon[35312]: scrub summary: active paths [/] 2024-03-28T18:10:49.683 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:10:49 smithi079 ceph-mon[35312]: scrub summary: idle+waiting paths [/] 2024-03-28T18:10:49.683 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:10:49 smithi079 ceph-mon[35312]: scrub summary: active paths [/] 2024-03-28T18:10:49.683 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:10:49 smithi079 ceph-mon[35312]: scrub summary: idle+waiting paths [/] 2024-03-28T18:10:49.683 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:10:49 smithi079 ceph-mon[35312]: scrub summary: active paths [/] 2024-03-28T18:10:49.683 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:10:49 smithi079 ceph-mon[35312]: scrub summary: idle+waiting paths [/] 2024-03-28T18:10:49.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:10:49 smithi022 ceph-mon[31985]: scrub summary: active paths [/] 2024-03-28T18:10:49.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:10:49 smithi022 ceph-mon[31985]: scrub summary: idle+waiting paths [/] 2024-03-28T18:10:49.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:10:49 smithi022 ceph-mon[31985]: scrub summary: active paths [/] 2024-03-28T18:10:49.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:10:49 smithi022 ceph-mon[31985]: scrub summary: idle+waiting paths [/] 2024-03-28T18:10:49.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:10:49 smithi022 ceph-mon[31985]: scrub summary: active paths [/] 2024-03-28T18:10:49.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:10:49 smithi022 ceph-mon[31985]: scrub summary: idle+waiting paths [/] 2024-03-28T18:10:50.674 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:10:50 smithi134 ceph-mon[35037]: pgmap v103: 97 pgs: 97 active+clean; 117 MiB data, 799 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:10:50.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:10:50 smithi079 ceph-mon[35312]: pgmap v103: 97 pgs: 97 active+clean; 117 MiB data, 799 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:10:50.781 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:10:50 smithi022 ceph-mon[31985]: pgmap v103: 97 pgs: 97 active+clean; 117 MiB data, 799 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:10:52.432 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:10:52 smithi079 ceph-mon[35312]: pgmap v104: 97 pgs: 97 active+clean; 117 MiB data, 799 MiB used, 1.0 TiB / 1.0 TiB avail; 110 KiB/s rd, 5.4 KiB/s wr, 27 op/s 2024-03-28T18:10:52.674 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:10:52 smithi134 ceph-mon[35037]: pgmap v104: 97 pgs: 97 active+clean; 117 MiB data, 799 MiB used, 1.0 TiB / 1.0 TiB avail; 110 KiB/s rd, 5.4 KiB/s wr, 27 op/s 2024-03-28T18:10:52.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:10:52 smithi022 ceph-mon[31985]: pgmap v104: 97 pgs: 97 active+clean; 117 MiB data, 799 MiB used, 1.0 TiB / 1.0 TiB avail; 110 KiB/s rd, 5.4 KiB/s wr, 27 op/s 2024-03-28T18:10:54.674 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:10:54 smithi134 ceph-mon[35037]: pgmap v105: 97 pgs: 97 active+clean; 117 MiB data, 799 MiB used, 1.0 TiB / 1.0 TiB avail; 110 KiB/s rd, 5.4 KiB/s wr, 27 op/s 2024-03-28T18:10:54.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:10:54 smithi079 ceph-mon[35312]: pgmap v105: 97 pgs: 97 active+clean; 117 MiB data, 799 MiB used, 1.0 TiB / 1.0 TiB avail; 110 KiB/s rd, 5.4 KiB/s wr, 27 op/s 2024-03-28T18:10:54.781 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:10:54 smithi022 ceph-mon[31985]: pgmap v105: 97 pgs: 97 active+clean; 117 MiB data, 799 MiB used, 1.0 TiB / 1.0 TiB avail; 110 KiB/s rd, 5.4 KiB/s wr, 27 op/s 2024-03-28T18:10:56.674 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:10:56 smithi134 ceph-mon[35037]: pgmap v106: 97 pgs: 97 active+clean; 117 MiB data, 799 MiB used, 1.0 TiB / 1.0 TiB avail; 101 KiB/s rd, 5.0 KiB/s wr, 29 op/s 2024-03-28T18:10:56.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:10:56 smithi079 ceph-mon[35312]: pgmap v106: 97 pgs: 97 active+clean; 117 MiB data, 799 MiB used, 1.0 TiB / 1.0 TiB avail; 101 KiB/s rd, 5.0 KiB/s wr, 29 op/s 2024-03-28T18:10:56.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:10:56 smithi022 ceph-mon[31985]: pgmap v106: 97 pgs: 97 active+clean; 117 MiB data, 799 MiB used, 1.0 TiB / 1.0 TiB avail; 101 KiB/s rd, 5.0 KiB/s wr, 29 op/s 2024-03-28T18:10:58.674 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:10:58 smithi134 ceph-mon[35037]: pgmap v107: 97 pgs: 97 active+clean; 117 MiB data, 799 MiB used, 1.0 TiB / 1.0 TiB avail; 92 KiB/s rd, 4.7 KiB/s wr, 23 op/s 2024-03-28T18:10:58.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:10:58 smithi079 ceph-mon[35312]: pgmap v107: 97 pgs: 97 active+clean; 117 MiB data, 799 MiB used, 1.0 TiB / 1.0 TiB avail; 92 KiB/s rd, 4.7 KiB/s wr, 23 op/s 2024-03-28T18:10:58.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:10:58 smithi022 ceph-mon[31985]: pgmap v107: 97 pgs: 97 active+clean; 117 MiB data, 799 MiB used, 1.0 TiB / 1.0 TiB avail; 92 KiB/s rd, 4.7 KiB/s wr, 23 op/s 2024-03-28T18:10:59.862 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:10:59 smithi079 systemd[1]: ceph-fd220748-ed2b-11ee-95d0-87774f69a715@mds.h.service: Scheduled restart job, restart counter is at 2. 2024-03-28T18:10:59.862 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:10:59 smithi079 systemd[1]: Stopped Ceph mds.h for fd220748-ed2b-11ee-95d0-87774f69a715. 2024-03-28T18:10:59.862 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:10:59 smithi079 systemd[1]: ceph-fd220748-ed2b-11ee-95d0-87774f69a715@mds.h.service: Consumed 1.778s CPU time. 2024-03-28T18:10:59.862 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:10:59 smithi079 systemd[1]: Starting Ceph mds.h for fd220748-ed2b-11ee-95d0-87774f69a715... 2024-03-28T18:11:00.152 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:10:59 smithi079 podman[69306]: 2024-03-28 18:10:59.766342823 +0000 UTC m=+0.033563771 image pull c664afe56bb01db4f9afa9532c20598e201e664f2e9ba2be50db0d52674a26d2 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc 2024-03-28T18:11:00.152 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:10:59 smithi079 podman[69306]: 2024-03-28 18:10:59.890339527 +0000 UTC m=+0.157560507 container create 6dbaa33fd278e5ff68b86b0ad2174d690a95fbcb13c514fd0e9c5445b9fada74 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.build-date=20240319, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS) 2024-03-28T18:11:00.433 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:11:00 smithi079 podman[69306]: 2024-03-28 18:11:00.152104966 +0000 UTC m=+0.419325966 container init 6dbaa33fd278e5ff68b86b0ad2174d690a95fbcb13c514fd0e9c5445b9fada74 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.build-date=20240319) 2024-03-28T18:11:00.433 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:11:00 smithi079 podman[69306]: 2024-03-28 18:11:00.161801804 +0000 UTC m=+0.429022742 container start 6dbaa33fd278e5ff68b86b0ad2174d690a95fbcb13c514fd0e9c5445b9fada74 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, ceph=True, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.build-date=20240319, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, org.label-schema.vendor=CentOS) 2024-03-28T18:11:00.433 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:11:00 smithi079 ceph-mds[69325]: -- 172.21.15.79:0/1021643010 <== mon.2 v2:172.21.15.79:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55a0a6cd5a40 con 0x55a0a6ca8800 2024-03-28T18:11:00.433 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:11:00 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69321]: starting mds.h at 2024-03-28T18:11:00.433 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:11:00 smithi079 bash[69306]: 6dbaa33fd278e5ff68b86b0ad2174d690a95fbcb13c514fd0e9c5445b9fada74 2024-03-28T18:11:00.433 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:11:00 smithi079 systemd[1]: Started Ceph mds.h for fd220748-ed2b-11ee-95d0-87774f69a715. 2024-03-28T18:11:00.433 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:11:00 smithi079 ceph-mon[35312]: pgmap v108: 97 pgs: 97 active+clean; 117 MiB data, 799 MiB used, 1.0 TiB / 1.0 TiB avail; 92 KiB/s rd, 4.7 KiB/s wr, 23 op/s 2024-03-28T18:11:00.434 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:11:00 smithi079 ceph-mon[35312]: daemon mds.h restarted 2024-03-28T18:11:00.434 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:11:00 smithi079 ceph-mon[35312]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-03-28T18:11:00.434 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:11:00 smithi079 ceph-mon[35312]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-03-28T18:11:00.434 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:11:00 smithi079 ceph-mon[35312]: osdmap e96: 12 total, 12 up, 12 in 2024-03-28T18:11:00.434 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:11:00 smithi079 ceph-mon[35312]: mds.? [v2:172.21.15.79:6836/2541118119,v1:172.21.15.79:6837/2541118119] up:boot 2024-03-28T18:11:00.434 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:11:00 smithi079 ceph-mon[35312]: Standby daemon mds.f assigned to filesystem cephfs as rank 1 2024-03-28T18:11:00.434 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:11:00 smithi079 ceph-mon[35312]: fsmap cephfs:4/5 {0=b=up:active,2=e=up:active,3=k=up:active,4=i=up:active} 5 up:standby-replay 3 up:standby, 1 failed 2024-03-28T18:11:00.434 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:11:00 smithi079 ceph-mon[35312]: fsmap cephfs:5/5 4 up:standby-replay 3 up:standby 1 up:replay 4 up:active 2024-03-28T18:11:00.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:11:00 smithi134 ceph-mon[35037]: pgmap v108: 97 pgs: 97 active+clean; 117 MiB data, 799 MiB used, 1.0 TiB / 1.0 TiB avail; 92 KiB/s rd, 4.7 KiB/s wr, 23 op/s 2024-03-28T18:11:00.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:11:00 smithi134 ceph-mon[35037]: daemon mds.h restarted 2024-03-28T18:11:00.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:11:00 smithi134 ceph-mon[35037]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-03-28T18:11:00.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:11:00 smithi134 ceph-mon[35037]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-03-28T18:11:00.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:11:00 smithi134 ceph-mon[35037]: osdmap e96: 12 total, 12 up, 12 in 2024-03-28T18:11:00.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:11:00 smithi134 ceph-mon[35037]: mds.? [v2:172.21.15.79:6836/2541118119,v1:172.21.15.79:6837/2541118119] up:boot 2024-03-28T18:11:00.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:11:00 smithi134 ceph-mon[35037]: Standby daemon mds.f assigned to filesystem cephfs as rank 1 2024-03-28T18:11:00.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:11:00 smithi134 ceph-mon[35037]: fsmap cephfs:4/5 {0=b=up:active,2=e=up:active,3=k=up:active,4=i=up:active} 5 up:standby-replay 3 up:standby, 1 failed 2024-03-28T18:11:00.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:11:00 smithi134 ceph-mon[35037]: fsmap cephfs:5/5 4 up:standby-replay 3 up:standby 1 up:replay 4 up:active 2024-03-28T18:11:00.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:11:00 smithi022 ceph-mon[31985]: pgmap v108: 97 pgs: 97 active+clean; 117 MiB data, 799 MiB used, 1.0 TiB / 1.0 TiB avail; 92 KiB/s rd, 4.7 KiB/s wr, 23 op/s 2024-03-28T18:11:00.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:11:00 smithi022 ceph-mon[31985]: daemon mds.h restarted 2024-03-28T18:11:00.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:11:00 smithi022 ceph-mon[31985]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-03-28T18:11:00.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:11:00 smithi022 ceph-mon[31985]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-03-28T18:11:00.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:11:00 smithi022 ceph-mon[31985]: osdmap e96: 12 total, 12 up, 12 in 2024-03-28T18:11:00.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:11:00 smithi022 ceph-mon[31985]: mds.? [v2:172.21.15.79:6836/2541118119,v1:172.21.15.79:6837/2541118119] up:boot 2024-03-28T18:11:00.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:11:00 smithi022 ceph-mon[31985]: Standby daemon mds.f assigned to filesystem cephfs as rank 1 2024-03-28T18:11:00.783 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:11:00 smithi022 ceph-mon[31985]: fsmap cephfs:4/5 {0=b=up:active,2=e=up:active,3=k=up:active,4=i=up:active} 5 up:standby-replay 3 up:standby, 1 failed 2024-03-28T18:11:00.783 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:11:00 smithi022 ceph-mon[31985]: fsmap cephfs:5/5 4 up:standby-replay 3 up:standby 1 up:replay 4 up:active 2024-03-28T18:11:02.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:11:02 smithi134 ceph-mon[35037]: pgmap v110: 97 pgs: 97 active+clean; 117 MiB data, 799 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2024-03-28T18:11:02.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:11:02 smithi134 ceph-mon[35037]: mds.? [v2:172.21.15.134:6834/3009142418,v1:172.21.15.134:6835/3009142418] up:resolve 2024-03-28T18:11:02.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:11:02 smithi134 ceph-mon[35037]: fsmap cephfs:5/5 4 up:standby-replay 3 up:standby 1 up:resolve 4 up:active 2024-03-28T18:11:02.683 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:11:02 smithi079 ceph-mon[35312]: pgmap v110: 97 pgs: 97 active+clean; 117 MiB data, 799 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2024-03-28T18:11:02.683 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:11:02 smithi079 ceph-mon[35312]: mds.? [v2:172.21.15.134:6834/3009142418,v1:172.21.15.134:6835/3009142418] up:resolve 2024-03-28T18:11:02.683 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:11:02 smithi079 ceph-mon[35312]: fsmap cephfs:5/5 4 up:standby-replay 3 up:standby 1 up:resolve 4 up:active 2024-03-28T18:11:02.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:11:02 smithi022 ceph-mon[31985]: pgmap v110: 97 pgs: 97 active+clean; 117 MiB data, 799 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2024-03-28T18:11:02.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:11:02 smithi022 ceph-mon[31985]: mds.? [v2:172.21.15.134:6834/3009142418,v1:172.21.15.134:6835/3009142418] up:resolve 2024-03-28T18:11:02.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:11:02 smithi022 ceph-mon[31985]: fsmap cephfs:5/5 4 up:standby-replay 3 up:standby 1 up:resolve 4 up:active 2024-03-28T18:11:03.674 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:11:03 smithi134 ceph-mon[35037]: reconnect by client.35222 172.21.15.22:0/3729116781 after 0.00299993 2024-03-28T18:11:03.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:11:03 smithi134 ceph-mon[35037]: mds.? [v2:172.21.15.134:6834/3009142418,v1:172.21.15.134:6835/3009142418] up:reconnect 2024-03-28T18:11:03.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:11:03 smithi134 ceph-mon[35037]: fsmap cephfs:5/5 4 up:standby-replay 3 up:standby 1 up:reconnect 4 up:active 2024-03-28T18:11:03.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:11:03 smithi079 ceph-mon[35312]: reconnect by client.35222 172.21.15.22:0/3729116781 after 0.00299993 2024-03-28T18:11:03.683 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:11:03 smithi079 ceph-mon[35312]: mds.? [v2:172.21.15.134:6834/3009142418,v1:172.21.15.134:6835/3009142418] up:reconnect 2024-03-28T18:11:03.683 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:11:03 smithi079 ceph-mon[35312]: fsmap cephfs:5/5 4 up:standby-replay 3 up:standby 1 up:reconnect 4 up:active 2024-03-28T18:11:03.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:11:03 smithi022 ceph-mon[31985]: reconnect by client.35222 172.21.15.22:0/3729116781 after 0.00299993 2024-03-28T18:11:03.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:11:03 smithi022 ceph-mon[31985]: mds.? [v2:172.21.15.134:6834/3009142418,v1:172.21.15.134:6835/3009142418] up:reconnect 2024-03-28T18:11:03.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:11:03 smithi022 ceph-mon[31985]: fsmap cephfs:5/5 4 up:standby-replay 3 up:standby 1 up:reconnect 4 up:active 2024-03-28T18:11:04.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:11:04 smithi079 ceph-mon[35312]: pgmap v111: 97 pgs: 97 active+clean; 117 MiB data, 799 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2024-03-28T18:11:04.683 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:11:04 smithi079 ceph-mon[35312]: mds.? [v2:172.21.15.134:6834/3009142418,v1:172.21.15.134:6835/3009142418] up:rejoin 2024-03-28T18:11:04.683 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:11:04 smithi079 ceph-mon[35312]: fsmap cephfs:5/5 4 up:standby-replay 3 up:standby 1 up:rejoin 4 up:active 2024-03-28T18:11:04.683 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:11:04 smithi079 ceph-mon[35312]: daemon mds.f is now active in filesystem cephfs as rank 1 2024-03-28T18:11:04.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:11:04 smithi022 ceph-mon[31985]: pgmap v111: 97 pgs: 97 active+clean; 117 MiB data, 799 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2024-03-28T18:11:04.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:11:04 smithi022 ceph-mon[31985]: mds.? [v2:172.21.15.134:6834/3009142418,v1:172.21.15.134:6835/3009142418] up:rejoin 2024-03-28T18:11:04.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:11:04 smithi022 ceph-mon[31985]: fsmap cephfs:5/5 4 up:standby-replay 3 up:standby 1 up:rejoin 4 up:active 2024-03-28T18:11:04.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:11:04 smithi022 ceph-mon[31985]: daemon mds.f is now active in filesystem cephfs as rank 1 2024-03-28T18:11:04.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:11:04 smithi134 ceph-mon[35037]: pgmap v111: 97 pgs: 97 active+clean; 117 MiB data, 799 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2024-03-28T18:11:04.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:11:04 smithi134 ceph-mon[35037]: mds.? [v2:172.21.15.134:6834/3009142418,v1:172.21.15.134:6835/3009142418] up:rejoin 2024-03-28T18:11:04.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:11:04 smithi134 ceph-mon[35037]: fsmap cephfs:5/5 4 up:standby-replay 3 up:standby 1 up:rejoin 4 up:active 2024-03-28T18:11:04.925 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:11:04 smithi134 ceph-mon[35037]: daemon mds.f is now active in filesystem cephfs as rank 1 2024-03-28T18:11:04.925 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:11:04 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[69769]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-2145-g3be9a5c9/rpm/el8/BUILD/ceph-19.0.0-2145-g3be9a5c9/src/mds/Locker.cc: In function 'void Locker::set_xlocks_done(MutationImpl*, bool)' thread 7f3677200700 time 2024-03-28T18:11:04.533867+0000 2024-03-28T18:11:04.925 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:11:04 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[69769]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-2145-g3be9a5c9/rpm/el8/BUILD/ceph-19.0.0-2145-g3be9a5c9/src/mds/Locker.cc: 626: FAILED ceph_assert(obj->is_auth()) 2024-03-28T18:11:04.925 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:11:04 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[69769]: ceph version 19.0.0-2145-g3be9a5c9 (3be9a5c9bc793e11e2800a8c0c696e8b46742033) squid (dev) 2024-03-28T18:11:04.926 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:11:04 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[69769]: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x135) [0x7f3682a5a4a1] 2024-03-28T18:11:04.926 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:11:04 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[69769]: 2: /usr/lib64/ceph/libceph-common.so.2(+0x2d2667) [0x7f3682a5a667] 2024-03-28T18:11:04.926 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:11:04 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[69769]: 3: (Locker::set_xlocks_done(MutationImpl*, bool)+0x493) [0x55f066fe5223] 2024-03-28T18:11:04.926 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:11:04 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[69769]: 4: (Server::early_reply(boost::intrusive_ptr const&, CInode*, CDentry*)+0x3e6) [0x55f066e5f5d6] 2024-03-28T18:11:04.926 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:11:04 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[69769]: 5: (Server::journal_and_reply(boost::intrusive_ptr const&, CInode*, CDentry*, LogEvent*, MDSLogContextBase*)+0xc1) [0x55f066e5fda1] 2024-03-28T18:11:04.926 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:11:04 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[69769]: 6: (Server::handle_client_mkdir(boost::intrusive_ptr const&)+0x5ba) [0x55f066e956aa] 2024-03-28T18:11:04.926 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:11:04 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[69769]: 7: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0x733) [0x55f066eb2233] 2024-03-28T18:11:04.926 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:11:04 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[69769]: 8: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x55f066eb4197] 2024-03-28T18:11:04.926 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:11:04 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[69769]: 9: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x55f066db311b] 2024-03-28T18:11:04.926 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:11:04 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[69769]: 10: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x55f066db36bc] 2024-03-28T18:11:04.926 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:11:04 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[69769]: 11: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bf) [0x55f066d9d1ff] 2024-03-28T18:11:04.926 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:11:04 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[69769]: 12: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x478) [0x7f3682cec488] 2024-03-28T18:11:04.926 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:11:04 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[69769]: 13: (DispatchQueue::entry()+0x51f) [0x7f3682ce960f] 2024-03-28T18:11:04.926 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:11:04 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[69769]: 14: (DispatchQueue::DispatchThread::entry()+0x11) [0x7f3682dbce81] 2024-03-28T18:11:04.927 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:11:04 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[69769]: 15: /lib64/libpthread.so.0(+0x81ca) [0x7f36817d71ca] 2024-03-28T18:11:04.927 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:11:04 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[69769]: 16: clone() 2024-03-28T18:11:04.927 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:11:04 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[69769]: 2024-03-28T18:11:04.534+0000 7f3677200700 -1 /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-2145-g3be9a5c9/rpm/el8/BUILD/ceph-19.0.0-2145-g3be9a5c9/src/mds/Locker.cc: In function 'void Locker::set_xlocks_done(MutationImpl*, bool)' thread 7f3677200700 time 2024-03-28T18:11:04.533867+0000 2024-03-28T18:11:04.927 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:11:04 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[69769]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-2145-g3be9a5c9/rpm/el8/BUILD/ceph-19.0.0-2145-g3be9a5c9/src/mds/Locker.cc: 626: FAILED ceph_assert(obj->is_auth()) 2024-03-28T18:11:04.927 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:11:04 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[69769]: 2024-03-28T18:11:04.927 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:11:04 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[69769]: ceph version 19.0.0-2145-g3be9a5c9 (3be9a5c9bc793e11e2800a8c0c696e8b46742033) squid (dev) 2024-03-28T18:11:04.927 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:11:04 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[69769]: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x135) [0x7f3682a5a4a1] 2024-03-28T18:11:04.927 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:11:04 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[69769]: 2: /usr/lib64/ceph/libceph-common.so.2(+0x2d2667) [0x7f3682a5a667] 2024-03-28T18:11:04.927 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:11:04 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[69769]: 3: (Locker::set_xlocks_done(MutationImpl*, bool)+0x493) [0x55f066fe5223] 2024-03-28T18:11:04.928 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:11:04 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[69769]: 4: (Server::early_reply(boost::intrusive_ptr const&, CInode*, CDentry*)+0x3e6) [0x55f066e5f5d6] 2024-03-28T18:11:04.928 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:11:04 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[69769]: 5: (Server::journal_and_reply(boost::intrusive_ptr const&, CInode*, CDentry*, LogEvent*, MDSLogContextBase*)+0xc1) [0x55f066e5fda1] 2024-03-28T18:11:04.928 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:11:04 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[69769]: 6: (Server::handle_client_mkdir(boost::intrusive_ptr const&)+0x5ba) [0x55f066e956aa] 2024-03-28T18:11:04.928 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:11:04 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[69769]: 7: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0x733) [0x55f066eb2233] 2024-03-28T18:11:04.928 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:11:04 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[69769]: 8: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x55f066eb4197] 2024-03-28T18:11:04.928 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:11:04 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[69769]: 9: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x55f066db311b] 2024-03-28T18:11:04.928 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:11:04 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[69769]: 10: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x55f066db36bc] 2024-03-28T18:11:04.928 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:11:04 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[69769]: 11: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bf) [0x55f066d9d1ff] 2024-03-28T18:11:04.928 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:11:04 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[69769]: 12: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x478) [0x7f3682cec488] 2024-03-28T18:11:04.928 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:11:04 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[69769]: 13: (DispatchQueue::entry()+0x51f) [0x7f3682ce960f] 2024-03-28T18:11:04.928 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:11:04 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[69769]: 14: (DispatchQueue::DispatchThread::entry()+0x11) [0x7f3682dbce81] 2024-03-28T18:11:04.928 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:11:04 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[69769]: 15: /lib64/libpthread.so.0(+0x81ca) [0x7f36817d71ca] 2024-03-28T18:11:04.928 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:11:04 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[69769]: 16: clone() 2024-03-28T18:11:04.928 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:11:04 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[69769]: 2024-03-28T18:11:04.929 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:11:04 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[69769]: *** Caught signal (Aborted) ** 2024-03-28T18:11:04.929 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:11:04 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[69769]: in thread 7f3677200700 thread_name:ms_dispatch 2024-03-28T18:11:04.929 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:11:04 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[69769]: ceph version 19.0.0-2145-g3be9a5c9 (3be9a5c9bc793e11e2800a8c0c696e8b46742033) squid (dev) 2024-03-28T18:11:04.929 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:11:04 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[69769]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7f36817e1d20] 2024-03-28T18:11:04.929 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:11:04 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[69769]: 2: gsignal() 2024-03-28T18:11:04.929 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:11:04 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[69769]: 3: abort() 2024-03-28T18:11:04.929 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:11:04 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[69769]: 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x18f) [0x7f3682a5a4fb] 2024-03-28T18:11:04.929 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:11:04 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[69769]: 5: /usr/lib64/ceph/libceph-common.so.2(+0x2d2667) [0x7f3682a5a667] 2024-03-28T18:11:04.929 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:11:04 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[69769]: 6: (Locker::set_xlocks_done(MutationImpl*, bool)+0x493) [0x55f066fe5223] 2024-03-28T18:11:04.929 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:11:04 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[69769]: 7: (Server::early_reply(boost::intrusive_ptr const&, CInode*, CDentry*)+0x3e6) [0x55f066e5f5d6] 2024-03-28T18:11:04.929 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:11:04 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[69769]: 8: (Server::journal_and_reply(boost::intrusive_ptr const&, CInode*, CDentry*, LogEvent*, MDSLogContextBase*)+0xc1) [0x55f066e5fda1] 2024-03-28T18:11:04.930 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:11:04 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[69769]: 9: (Server::handle_client_mkdir(boost::intrusive_ptr const&)+0x5ba) [0x55f066e956aa] 2024-03-28T18:11:04.930 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:11:04 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[69769]: 10: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0x733) [0x55f066eb2233] 2024-03-28T18:11:04.930 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:11:04 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[69769]: 11: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x55f066eb4197] 2024-03-28T18:11:04.930 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:11:04 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[69769]: 12: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x55f066db311b] 2024-03-28T18:11:04.930 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:11:04 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[69769]: 13: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x55f066db36bc] 2024-03-28T18:11:04.930 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:11:04 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[69769]: 14: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bf) [0x55f066d9d1ff] 2024-03-28T18:11:04.930 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:11:04 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[69769]: 15: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x478) [0x7f3682cec488] 2024-03-28T18:11:04.930 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:11:04 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[69769]: 16: (DispatchQueue::entry()+0x51f) [0x7f3682ce960f] 2024-03-28T18:11:04.930 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:11:04 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[69769]: 17: (DispatchQueue::DispatchThread::entry()+0x11) [0x7f3682dbce81] 2024-03-28T18:11:04.930 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:11:04 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[69769]: 18: /lib64/libpthread.so.0(+0x81ca) [0x7f36817d71ca] 2024-03-28T18:11:04.930 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:11:04 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[69769]: 19: clone() 2024-03-28T18:11:04.930 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:11:04 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[69769]: 2024-03-28T18:11:04.537+0000 7f3677200700 -1 *** Caught signal (Aborted) ** 2024-03-28T18:11:04.930 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:11:04 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[69769]: in thread 7f3677200700 thread_name:ms_dispatch 2024-03-28T18:11:04.930 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:11:04 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[69769]: 2024-03-28T18:11:04.930 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:11:04 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[69769]: ceph version 19.0.0-2145-g3be9a5c9 (3be9a5c9bc793e11e2800a8c0c696e8b46742033) squid (dev) 2024-03-28T18:11:04.931 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:11:04 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[69769]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7f36817e1d20] 2024-03-28T18:11:04.931 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:11:04 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[69769]: 2: gsignal() 2024-03-28T18:11:04.931 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:11:04 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[69769]: 3: abort() 2024-03-28T18:11:04.931 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:11:04 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[69769]: 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x18f) [0x7f3682a5a4fb] 2024-03-28T18:11:04.931 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:11:04 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[69769]: 5: /usr/lib64/ceph/libceph-common.so.2(+0x2d2667) [0x7f3682a5a667] 2024-03-28T18:11:04.931 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:11:04 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[69769]: 6: (Locker::set_xlocks_done(MutationImpl*, bool)+0x493) [0x55f066fe5223] 2024-03-28T18:11:04.931 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:11:04 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[69769]: 7: (Server::early_reply(boost::intrusive_ptr const&, CInode*, CDentry*)+0x3e6) [0x55f066e5f5d6] 2024-03-28T18:11:04.931 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:11:04 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[69769]: 8: (Server::journal_and_reply(boost::intrusive_ptr const&, CInode*, CDentry*, LogEvent*, MDSLogContextBase*)+0xc1) [0x55f066e5fda1] 2024-03-28T18:11:04.931 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:11:04 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[69769]: 9: (Server::handle_client_mkdir(boost::intrusive_ptr const&)+0x5ba) [0x55f066e956aa] 2024-03-28T18:11:04.931 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:11:04 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[69769]: 10: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0x733) [0x55f066eb2233] 2024-03-28T18:11:04.932 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:11:04 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[69769]: 11: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x55f066eb4197] 2024-03-28T18:11:04.932 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:11:04 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[69769]: 12: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x55f066db311b] 2024-03-28T18:11:04.932 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:11:04 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[69769]: 13: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x55f066db36bc] 2024-03-28T18:11:04.932 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:11:04 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[69769]: 14: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bf) [0x55f066d9d1ff] 2024-03-28T18:11:04.932 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:11:04 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[69769]: 15: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x478) [0x7f3682cec488] 2024-03-28T18:11:04.932 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:11:04 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[69769]: 16: (DispatchQueue::entry()+0x51f) [0x7f3682ce960f] 2024-03-28T18:11:04.932 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:11:04 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[69769]: 17: (DispatchQueue::DispatchThread::entry()+0x11) [0x7f3682dbce81] 2024-03-28T18:11:04.932 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:11:04 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[69769]: 18: /lib64/libpthread.so.0(+0x81ca) [0x7f36817d71ca] 2024-03-28T18:11:04.932 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:11:04 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[69769]: 19: clone() 2024-03-28T18:11:04.932 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:11:04 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[69769]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-03-28T18:11:04.932 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:11:04 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[69769]: 2024-03-28T18:11:04.932 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:11:04 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[69769]: -1> 2024-03-28T18:11:04.534+0000 7f3677200700 -1 /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-2145-g3be9a5c9/rpm/el8/BUILD/ceph-19.0.0-2145-g3be9a5c9/src/mds/Locker.cc: In function 'void Locker::set_xlocks_done(MutationImpl*, bool)' thread 7f3677200700 time 2024-03-28T18:11:04.533867+0000 2024-03-28T18:11:04.932 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:11:04 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[69769]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-2145-g3be9a5c9/rpm/el8/BUILD/ceph-19.0.0-2145-g3be9a5c9/src/mds/Locker.cc: 626: FAILED ceph_assert(obj->is_auth()) 2024-03-28T18:11:04.932 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:11:04 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[69769]: 2024-03-28T18:11:04.933 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:11:04 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[69769]: ceph version 19.0.0-2145-g3be9a5c9 (3be9a5c9bc793e11e2800a8c0c696e8b46742033) squid (dev) 2024-03-28T18:11:04.933 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:11:04 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[69769]: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x135) [0x7f3682a5a4a1] 2024-03-28T18:11:04.933 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:11:04 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[69769]: 2: /usr/lib64/ceph/libceph-common.so.2(+0x2d2667) [0x7f3682a5a667] 2024-03-28T18:11:04.933 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:11:04 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[69769]: 3: (Locker::set_xlocks_done(MutationImpl*, bool)+0x493) [0x55f066fe5223] 2024-03-28T18:11:04.933 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:11:04 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[69769]: 4: (Server::early_reply(boost::intrusive_ptr const&, CInode*, CDentry*)+0x3e6) [0x55f066e5f5d6] 2024-03-28T18:11:04.933 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:11:04 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[69769]: 5: (Server::journal_and_reply(boost::intrusive_ptr const&, CInode*, CDentry*, LogEvent*, MDSLogContextBase*)+0xc1) [0x55f066e5fda1] 2024-03-28T18:11:04.933 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:11:04 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[69769]: 6: (Server::handle_client_mkdir(boost::intrusive_ptr const&)+0x5ba) [0x55f066e956aa] 2024-03-28T18:11:04.933 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:11:04 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[69769]: 7: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0x733) [0x55f066eb2233] 2024-03-28T18:11:04.934 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:11:04 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[69769]: 8: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x55f066eb4197] 2024-03-28T18:11:04.934 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:11:04 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[69769]: 9: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x55f066db311b] 2024-03-28T18:11:04.934 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:11:04 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[69769]: 10: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x55f066db36bc] 2024-03-28T18:11:04.934 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:11:04 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[69769]: 11: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bf) [0x55f066d9d1ff] 2024-03-28T18:11:04.934 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:11:04 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[69769]: 12: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x478) [0x7f3682cec488] 2024-03-28T18:11:04.934 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:11:04 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[69769]: 13: (DispatchQueue::entry()+0x51f) [0x7f3682ce960f] 2024-03-28T18:11:04.934 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:11:04 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[69769]: 14: (DispatchQueue::DispatchThread::entry()+0x11) [0x7f3682dbce81] 2024-03-28T18:11:04.934 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:11:04 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[69769]: 15: /lib64/libpthread.so.0(+0x81ca) [0x7f36817d71ca] 2024-03-28T18:11:04.934 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:11:04 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[69769]: 16: clone() 2024-03-28T18:11:04.934 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:11:04 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[69769]: 2024-03-28T18:11:04.934 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:11:04 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[69769]: 0> 2024-03-28T18:11:04.537+0000 7f3677200700 -1 *** Caught signal (Aborted) ** 2024-03-28T18:11:04.934 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:11:04 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[69769]: in thread 7f3677200700 thread_name:ms_dispatch 2024-03-28T18:11:04.934 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:11:04 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[69769]: 2024-03-28T18:11:04.934 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:11:04 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[69769]: ceph version 19.0.0-2145-g3be9a5c9 (3be9a5c9bc793e11e2800a8c0c696e8b46742033) squid (dev) 2024-03-28T18:11:04.935 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:11:04 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[69769]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7f36817e1d20] 2024-03-28T18:11:04.935 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:11:04 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[69769]: 2: gsignal() 2024-03-28T18:11:04.935 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:11:04 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[69769]: 3: abort() 2024-03-28T18:11:04.935 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:11:04 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[69769]: 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x18f) [0x7f3682a5a4fb] 2024-03-28T18:11:04.935 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:11:04 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[69769]: 5: /usr/lib64/ceph/libceph-common.so.2(+0x2d2667) [0x7f3682a5a667] 2024-03-28T18:11:04.935 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:11:04 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[69769]: 6: (Locker::set_xlocks_done(MutationImpl*, bool)+0x493) [0x55f066fe5223] 2024-03-28T18:11:04.935 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:11:04 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[69769]: 7: (Server::early_reply(boost::intrusive_ptr const&, CInode*, CDentry*)+0x3e6) [0x55f066e5f5d6] 2024-03-28T18:11:04.935 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:11:04 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[69769]: 8: (Server::journal_and_reply(boost::intrusive_ptr const&, CInode*, CDentry*, LogEvent*, MDSLogContextBase*)+0xc1) [0x55f066e5fda1] 2024-03-28T18:11:04.935 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:11:04 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[69769]: 9: (Server::handle_client_mkdir(boost::intrusive_ptr const&)+0x5ba) [0x55f066e956aa] 2024-03-28T18:11:04.935 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:11:04 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[69769]: 10: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0x733) [0x55f066eb2233] 2024-03-28T18:11:04.935 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:11:04 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[69769]: 11: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x55f066eb4197] 2024-03-28T18:11:04.935 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:11:04 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[69769]: 12: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x55f066db311b] 2024-03-28T18:11:04.935 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:11:04 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[69769]: 13: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x55f066db36bc] 2024-03-28T18:11:04.935 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:11:04 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[69769]: 14: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bf) [0x55f066d9d1ff] 2024-03-28T18:11:04.936 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:11:04 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[69769]: 15: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x478) [0x7f3682cec488] 2024-03-28T18:11:04.936 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:11:04 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[69769]: 16: (DispatchQueue::entry()+0x51f) [0x7f3682ce960f] 2024-03-28T18:11:04.936 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:11:04 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[69769]: 17: (DispatchQueue::DispatchThread::entry()+0x11) [0x7f3682dbce81] 2024-03-28T18:11:04.936 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:11:04 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[69769]: 18: /lib64/libpthread.so.0(+0x81ca) [0x7f36817d71ca] 2024-03-28T18:11:04.936 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:11:04 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[69769]: 19: clone() 2024-03-28T18:11:04.936 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:11:04 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[69769]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-03-28T18:11:04.936 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:11:04 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[69769]: 2024-03-28T18:11:04.936 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:11:04 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[69769]: -1> 2024-03-28T18:11:04.534+0000 7f3677200700 -1 /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-2145-g3be9a5c9/rpm/el8/BUILD/ceph-19.0.0-2145-g3be9a5c9/src/mds/Locker.cc: In function 'void Locker::set_xlocks_done(MutationImpl*, bool)' thread 7f3677200700 time 2024-03-28T18:11:04.533867+0000 2024-03-28T18:11:04.936 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:11:04 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[69769]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-2145-g3be9a5c9/rpm/el8/BUILD/ceph-19.0.0-2145-g3be9a5c9/src/mds/Locker.cc: 626: FAILED ceph_assert(obj->is_auth()) 2024-03-28T18:11:04.936 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:11:04 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[69769]: 2024-03-28T18:11:04.936 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:11:04 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[69769]: ceph version 19.0.0-2145-g3be9a5c9 (3be9a5c9bc793e11e2800a8c0c696e8b46742033) squid (dev) 2024-03-28T18:11:04.936 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:11:04 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[69769]: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x135) [0x7f3682a5a4a1] 2024-03-28T18:11:04.936 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:11:04 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[69769]: 2: /usr/lib64/ceph/libceph-common.so.2(+0x2d2667) [0x7f3682a5a667] 2024-03-28T18:11:04.936 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:11:04 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[69769]: 3: (Locker::set_xlocks_done(MutationImpl*, bool)+0x493) [0x55f066fe5223] 2024-03-28T18:11:04.936 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:11:04 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[69769]: 4: (Server::early_reply(boost::intrusive_ptr const&, CInode*, CDentry*)+0x3e6) [0x55f066e5f5d6] 2024-03-28T18:11:04.936 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:11:04 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[69769]: 5: (Server::journal_and_reply(boost::intrusive_ptr const&, CInode*, CDentry*, LogEvent*, MDSLogContextBase*)+0xc1) [0x55f066e5fda1] 2024-03-28T18:11:04.937 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:11:04 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[69769]: 6: (Server::handle_client_mkdir(boost::intrusive_ptr const&)+0x5ba) [0x55f066e956aa] 2024-03-28T18:11:04.937 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:11:04 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[69769]: 7: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0x733) [0x55f066eb2233] 2024-03-28T18:11:04.937 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:11:04 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[69769]: 8: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x55f066eb4197] 2024-03-28T18:11:04.937 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:11:04 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[69769]: 9: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x55f066db311b] 2024-03-28T18:11:04.937 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:11:04 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[69769]: 10: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x55f066db36bc] 2024-03-28T18:11:04.937 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:11:04 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[69769]: 11: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bf) [0x55f066d9d1ff] 2024-03-28T18:11:04.937 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:11:04 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[69769]: 12: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x478) [0x7f3682cec488] 2024-03-28T18:11:04.937 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:11:04 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[69769]: 13: (DispatchQueue::entry()+0x51f) [0x7f3682ce960f] 2024-03-28T18:11:04.938 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:11:04 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[69769]: 14: (DispatchQueue::DispatchThread::entry()+0x11) [0x7f3682dbce81] 2024-03-28T18:11:04.938 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:11:04 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[69769]: 15: /lib64/libpthread.so.0(+0x81ca) [0x7f36817d71ca] 2024-03-28T18:11:04.938 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:11:04 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[69769]: 16: clone() 2024-03-28T18:11:04.938 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:11:04 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[69769]: 2024-03-28T18:11:04.938 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:11:04 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[69769]: 0> 2024-03-28T18:11:04.537+0000 7f3677200700 -1 *** Caught signal (Aborted) ** 2024-03-28T18:11:04.938 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:11:04 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[69769]: in thread 7f3677200700 thread_name:ms_dispatch 2024-03-28T18:11:04.938 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:11:04 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[69769]: 2024-03-28T18:11:04.938 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:11:04 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[69769]: ceph version 19.0.0-2145-g3be9a5c9 (3be9a5c9bc793e11e2800a8c0c696e8b46742033) squid (dev) 2024-03-28T18:11:04.938 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:11:04 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[69769]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7f36817e1d20] 2024-03-28T18:11:04.938 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:11:04 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[69769]: 2: gsignal() 2024-03-28T18:11:04.938 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:11:04 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[69769]: 3: abort() 2024-03-28T18:11:04.938 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:11:04 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[69769]: 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x18f) [0x7f3682a5a4fb] 2024-03-28T18:11:04.938 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:11:04 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[69769]: 5: /usr/lib64/ceph/libceph-common.so.2(+0x2d2667) [0x7f3682a5a667] 2024-03-28T18:11:04.938 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:11:04 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[69769]: 6: (Locker::set_xlocks_done(MutationImpl*, bool)+0x493) [0x55f066fe5223] 2024-03-28T18:11:04.938 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:11:04 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[69769]: 7: (Server::early_reply(boost::intrusive_ptr const&, CInode*, CDentry*)+0x3e6) [0x55f066e5f5d6] 2024-03-28T18:11:04.939 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:11:04 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[69769]: 8: (Server::journal_and_reply(boost::intrusive_ptr const&, CInode*, CDentry*, LogEvent*, MDSLogContextBase*)+0xc1) [0x55f066e5fda1] 2024-03-28T18:11:04.939 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:11:04 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[69769]: 9: (Server::handle_client_mkdir(boost::intrusive_ptr const&)+0x5ba) [0x55f066e956aa] 2024-03-28T18:11:04.939 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:11:04 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[69769]: 10: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0x733) [0x55f066eb2233] 2024-03-28T18:11:04.939 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:11:04 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[69769]: 11: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x55f066eb4197] 2024-03-28T18:11:04.939 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:11:04 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[69769]: 12: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x55f066db311b] 2024-03-28T18:11:04.939 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:11:04 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[69769]: 13: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x55f066db36bc] 2024-03-28T18:11:04.939 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:11:04 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[69769]: 14: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bf) [0x55f066d9d1ff] 2024-03-28T18:11:04.939 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:11:04 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[69769]: 15: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x478) [0x7f3682cec488] 2024-03-28T18:11:04.939 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:11:04 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[69769]: 16: (DispatchQueue::entry()+0x51f) [0x7f3682ce960f] 2024-03-28T18:11:04.939 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:11:04 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[69769]: 17: (DispatchQueue::DispatchThread::entry()+0x11) [0x7f3682dbce81] 2024-03-28T18:11:04.939 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:11:04 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[69769]: 18: /lib64/libpthread.so.0(+0x81ca) [0x7f36817d71ca] 2024-03-28T18:11:04.939 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:11:04 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[69769]: 19: clone() 2024-03-28T18:11:04.939 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:11:04 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[69769]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-03-28T18:11:04.939 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:11:04 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[69769]: 2024-03-28T18:11:04.939 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:11:04 smithi134 podman[69944]: 2024-03-28 18:11:04.648925863 +0000 UTC m=+0.033662906 container died d152d7307b05bc350e7621116317bf1237be480aada83ebe56fc7359227b8025 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.license=GPLv2, org.label-schema.build-date=20240319, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-28T18:11:05.674 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:11:05 smithi134 podman[69944]: 2024-03-28 18:11:05.388068371 +0000 UTC m=+0.772805459 container remove d152d7307b05bc350e7621116317bf1237be480aada83ebe56fc7359227b8025 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f, ceph=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.build-date=20240319, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.schema-version=1.0) 2024-03-28T18:11:05.674 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:11:05 smithi134 systemd[1]: ceph-fd220748-ed2b-11ee-95d0-87774f69a715@mds.f.service: Main process exited, code=exited, status=134/n/a 2024-03-28T18:11:05.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:11:05 smithi134 ceph-mon[35037]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-03-28T18:11:05.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:11:05 smithi134 ceph-mon[35037]: Cluster is now healthy 2024-03-28T18:11:05.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:11:05 smithi134 ceph-mon[35037]: mds.? [v2:172.21.15.134:6834/3009142418,v1:172.21.15.134:6835/3009142418] up:active 2024-03-28T18:11:05.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:11:05 smithi134 ceph-mon[35037]: fsmap cephfs:5 4 up:standby-replay 3 up:standby 5 up:active 2024-03-28T18:11:05.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:11:05 smithi134 ceph-mon[35037]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-28T18:11:05.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:11:05 smithi134 ceph-mon[35037]: scrub summary: idle+waiting paths [/] 2024-03-28T18:11:05.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:11:05 smithi079 ceph-mon[35312]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-03-28T18:11:05.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:11:05 smithi079 ceph-mon[35312]: Cluster is now healthy 2024-03-28T18:11:05.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:11:05 smithi079 ceph-mon[35312]: mds.? [v2:172.21.15.134:6834/3009142418,v1:172.21.15.134:6835/3009142418] up:active 2024-03-28T18:11:05.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:11:05 smithi079 ceph-mon[35312]: fsmap cephfs:5 4 up:standby-replay 3 up:standby 5 up:active 2024-03-28T18:11:05.683 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:11:05 smithi079 ceph-mon[35312]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-28T18:11:05.683 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:11:05 smithi079 ceph-mon[35312]: scrub summary: idle+waiting paths [/] 2024-03-28T18:11:05.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:11:05 smithi022 ceph-mon[31985]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-03-28T18:11:05.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:11:05 smithi022 ceph-mon[31985]: Cluster is now healthy 2024-03-28T18:11:05.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:11:05 smithi022 ceph-mon[31985]: mds.? [v2:172.21.15.134:6834/3009142418,v1:172.21.15.134:6835/3009142418] up:active 2024-03-28T18:11:05.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:11:05 smithi022 ceph-mon[31985]: fsmap cephfs:5 4 up:standby-replay 3 up:standby 5 up:active 2024-03-28T18:11:05.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:11:05 smithi022 ceph-mon[31985]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-28T18:11:05.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:11:05 smithi022 ceph-mon[31985]: scrub summary: idle+waiting paths [/] 2024-03-28T18:11:06.174 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:11:05 smithi134 systemd[1]: ceph-fd220748-ed2b-11ee-95d0-87774f69a715@mds.f.service: Failed with result 'exit-code'. 2024-03-28T18:11:06.174 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:11:05 smithi134 systemd[1]: ceph-fd220748-ed2b-11ee-95d0-87774f69a715@mds.f.service: Consumed 1.854s CPU time. 2024-03-28T18:11:06.781 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:11:06 smithi022 ceph-mon[31985]: pgmap v112: 97 pgs: 97 active+clean; 117 MiB data, 799 MiB used, 1.0 TiB / 1.0 TiB avail; 9.0 KiB/s rd, 2.8 KiB/s wr, 12 op/s 2024-03-28T18:11:06.859 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:11:06 smithi134 ceph-mon[35037]: pgmap v112: 97 pgs: 97 active+clean; 117 MiB data, 799 MiB used, 1.0 TiB / 1.0 TiB avail; 9.0 KiB/s rd, 2.8 KiB/s wr, 12 op/s 2024-03-28T18:11:06.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:11:06 smithi079 ceph-mon[35312]: pgmap v112: 97 pgs: 97 active+clean; 117 MiB data, 799 MiB used, 1.0 TiB / 1.0 TiB avail; 9.0 KiB/s rd, 2.8 KiB/s wr, 12 op/s 2024-03-28T18:11:08.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:11:08 smithi022 ceph-mon[31985]: pgmap v113: 97 pgs: 97 active+clean; 117 MiB data, 799 MiB used, 1.0 TiB / 1.0 TiB avail; 9.0 KiB/s rd, 2.8 KiB/s wr, 12 op/s 2024-03-28T18:11:08.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:11:08 smithi134 ceph-mon[35037]: pgmap v113: 97 pgs: 97 active+clean; 117 MiB data, 799 MiB used, 1.0 TiB / 1.0 TiB avail; 9.0 KiB/s rd, 2.8 KiB/s wr, 12 op/s 2024-03-28T18:11:08.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:11:08 smithi079 ceph-mon[35312]: pgmap v113: 97 pgs: 97 active+clean; 117 MiB data, 799 MiB used, 1.0 TiB / 1.0 TiB avail; 9.0 KiB/s rd, 2.8 KiB/s wr, 12 op/s 2024-03-28T18:11:10.734 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:11:10 smithi079 ceph-mon[35312]: pgmap v114: 97 pgs: 97 active+clean; 117 MiB data, 799 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 2.8 KiB/s wr, 17 op/s 2024-03-28T18:11:10.781 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:11:10 smithi022 ceph-mon[31985]: pgmap v114: 97 pgs: 97 active+clean; 117 MiB data, 799 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 2.8 KiB/s wr, 17 op/s 2024-03-28T18:11:10.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:11:10 smithi134 ceph-mon[35037]: pgmap v114: 97 pgs: 97 active+clean; 117 MiB data, 799 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 2.8 KiB/s wr, 17 op/s 2024-03-28T18:11:12.781 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:11:12 smithi022 ceph-mon[31985]: pgmap v115: 97 pgs: 97 active+clean; 117 MiB data, 800 MiB used, 1.0 TiB / 1.0 TiB avail; 111 KiB/s rd, 30 KiB/s wr, 40 op/s 2024-03-28T18:11:12.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:11:12 smithi134 ceph-mon[35037]: pgmap v115: 97 pgs: 97 active+clean; 117 MiB data, 800 MiB used, 1.0 TiB / 1.0 TiB avail; 111 KiB/s rd, 30 KiB/s wr, 40 op/s 2024-03-28T18:11:12.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:11:12 smithi079 ceph-mon[35312]: pgmap v115: 97 pgs: 97 active+clean; 117 MiB data, 800 MiB used, 1.0 TiB / 1.0 TiB avail; 111 KiB/s rd, 30 KiB/s wr, 40 op/s 2024-03-28T18:11:14.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:11:14 smithi022 ceph-mon[31985]: pgmap v116: 97 pgs: 97 active+clean; 117 MiB data, 800 MiB used, 1.0 TiB / 1.0 TiB avail; 100 KiB/s rd, 27 KiB/s wr, 36 op/s 2024-03-28T18:11:14.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:11:14 smithi134 ceph-mon[35037]: pgmap v116: 97 pgs: 97 active+clean; 117 MiB data, 800 MiB used, 1.0 TiB / 1.0 TiB avail; 100 KiB/s rd, 27 KiB/s wr, 36 op/s 2024-03-28T18:11:14.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:11:14 smithi079 ceph-mon[35312]: pgmap v116: 97 pgs: 97 active+clean; 117 MiB data, 800 MiB used, 1.0 TiB / 1.0 TiB avail; 100 KiB/s rd, 27 KiB/s wr, 36 op/s 2024-03-28T18:11:16.301 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:11:15 smithi134 systemd[1]: ceph-fd220748-ed2b-11ee-95d0-87774f69a715@mds.f.service: Scheduled restart job, restart counter is at 2. 2024-03-28T18:11:16.301 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:11:15 smithi134 systemd[1]: Stopped Ceph mds.f for fd220748-ed2b-11ee-95d0-87774f69a715. 2024-03-28T18:11:16.301 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:11:15 smithi134 systemd[1]: ceph-fd220748-ed2b-11ee-95d0-87774f69a715@mds.f.service: Consumed 1.854s CPU time. 2024-03-28T18:11:16.301 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:11:15 smithi134 systemd[1]: Starting Ceph mds.f for fd220748-ed2b-11ee-95d0-87774f69a715... 2024-03-28T18:11:16.601 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:11:16 smithi134 ceph-mon[35037]: pgmap v117: 97 pgs: 97 active+clean; 117 MiB data, 800 MiB used, 1.0 TiB / 1.0 TiB avail; 103 KiB/s rd, 27 KiB/s wr, 41 op/s 2024-03-28T18:11:16.601 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:11:16 smithi134 podman[70049]: 2024-03-28 18:11:16.205814127 +0000 UTC m=+0.032877015 image pull c664afe56bb01db4f9afa9532c20598e201e664f2e9ba2be50db0d52674a26d2 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc 2024-03-28T18:11:16.601 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:11:16 smithi134 podman[70049]: 2024-03-28 18:11:16.313438483 +0000 UTC m=+0.140501400 container create 6b5902d9445efe5aad8c2117d7096ee69fe245063320d93d8eea88e5f822802a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f, GIT_BRANCH=HEAD, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240319, CEPH_POINT_RELEASE=, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, maintainer=Guillaume Abrioux , GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5) 2024-03-28T18:11:16.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:11:16 smithi022 ceph-mon[31985]: pgmap v117: 97 pgs: 97 active+clean; 117 MiB data, 800 MiB used, 1.0 TiB / 1.0 TiB avail; 103 KiB/s rd, 27 KiB/s wr, 41 op/s 2024-03-28T18:11:16.825 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-28T18:11:16.826 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-28T18:11:16.925 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:11:16 smithi134 podman[70049]: 2024-03-28 18:11:16.601373921 +0000 UTC m=+0.428436856 container init 6b5902d9445efe5aad8c2117d7096ee69fe245063320d93d8eea88e5f822802a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240319, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, ceph=True) 2024-03-28T18:11:16.925 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:11:16 smithi134 podman[70049]: 2024-03-28 18:11:16.610696704 +0000 UTC m=+0.437759601 container start 6b5902d9445efe5aad8c2117d7096ee69fe245063320d93d8eea88e5f822802a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f, ceph=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240319, GIT_CLEAN=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c) 2024-03-28T18:11:16.925 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:11:16 smithi134 ceph-mds[70068]: -- 172.21.15.134:0/2962658875 <== mon.1 v2:172.21.15.134:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55f4b609da40 con 0x55f4b6070800 2024-03-28T18:11:16.925 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:11:16 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[70064]: starting mds.f at 2024-03-28T18:11:16.926 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:11:16 smithi134 bash[70049]: 6b5902d9445efe5aad8c2117d7096ee69fe245063320d93d8eea88e5f822802a 2024-03-28T18:11:16.926 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:11:16 smithi134 systemd[1]: Started Ceph mds.f for fd220748-ed2b-11ee-95d0-87774f69a715. 2024-03-28T18:11:16.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:11:16 smithi079 ceph-mon[35312]: pgmap v117: 97 pgs: 97 active+clean; 117 MiB data, 800 MiB used, 1.0 TiB / 1.0 TiB avail; 103 KiB/s rd, 27 KiB/s wr, 41 op/s 2024-03-28T18:11:17.057 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:11:17.057+0000 7f557ea00640 1 -- 172.21.15.22:0/1632820562 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5578109710 msgr2=0x7f5578109af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:11:17.057 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:11:17.057+0000 7f557ea00640 1 --2- 172.21.15.22:0/1632820562 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5578109710 0x7f5578109af0 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7f556c009930 tx=0x7f556c02bc20 comp rx=0 tx=0).stop 2024-03-28T18:11:17.057 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:11:17.057+0000 7f557ea00640 1 -- 172.21.15.22:0/1632820562 shutdown_connections 2024-03-28T18:11:17.058 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:11:17.057+0000 7f557ea00640 1 --2- 172.21.15.22:0/1632820562 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f55781040c0 0x7f55781105d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:11:17.058 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:11:17.057+0000 7f557ea00640 1 --2- 172.21.15.22:0/1632820562 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5578103700 0x7f5578103b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:11:17.058 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:11:17.057+0000 7f557ea00640 1 --2- 172.21.15.22:0/1632820562 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5578109710 0x7f5578109af0 unknown :-1 s=CLOSED pgs=700 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:11:17.058 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:11:17.057+0000 7f557ea00640 1 -- 172.21.15.22:0/1632820562 >> 172.21.15.22:0/1632820562 conn(0x7f55780ff1b0 msgr2=0x7f55781015d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:11:17.058 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:11:17.057+0000 7f557ea00640 1 -- 172.21.15.22:0/1632820562 shutdown_connections 2024-03-28T18:11:17.058 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:11:17.057+0000 7f557ea00640 1 -- 172.21.15.22:0/1632820562 wait complete. 2024-03-28T18:11:17.058 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:11:17.059+0000 7f557ea00640 1 Processor -- start 2024-03-28T18:11:17.058 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:11:17.059+0000 7f557ea00640 1 -- start start 2024-03-28T18:11:17.060 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:11:17.059+0000 7f557ea00640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f5578103700 0x7f557819a070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:11:17.060 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:11:17.059+0000 7f557ea00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f55781040c0 0x7f557819a5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:11:17.060 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:11:17.059+0000 7f557ea00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5578109710 0x7f557819e930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:11:17.060 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:11:17.060+0000 7f557ea00640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5578113af0 con 0x7f5578109710 2024-03-28T18:11:17.060 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:11:17.060+0000 7f557ea00640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5578113970 con 0x7f5578103700 2024-03-28T18:11:17.060 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:11:17.060+0000 7f557ea00640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5578113c70 con 0x7f55781040c0 2024-03-28T18:11:17.060 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:11:17.060+0000 7f557e000640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5578109710 0x7f557819e930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:11:17.060 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:11:17.060+0000 7f557d600640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f5578103700 0x7f557819a070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:11:17.061 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:11:17.060+0000 7f557d600640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f5578103700 0x7f557819a070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.22:45388/0 (socket says 172.21.15.22:45388) 2024-03-28T18:11:17.061 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:11:17.060+0000 7f557d600640 1 -- 172.21.15.22:0/2798285649 learned_addr learned my addr 172.21.15.22:0/2798285649 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-28T18:11:17.061 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:11:17.060+0000 7f557cc00640 1 --2- 172.21.15.22:0/2798285649 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f55781040c0 0x7f557819a5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:11:17.061 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:11:17.060+0000 7f557e000640 1 -- 172.21.15.22:0/2798285649 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f55781040c0 msgr2=0x7f557819a5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:11:17.061 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:11:17.060+0000 7f557e000640 1 --2- 172.21.15.22:0/2798285649 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f55781040c0 0x7f557819a5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:11:17.061 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:11:17.060+0000 7f557e000640 1 -- 172.21.15.22:0/2798285649 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f5578103700 msgr2=0x7f557819a070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:11:17.061 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:11:17.060+0000 7f557e000640 1 --2- 172.21.15.22:0/2798285649 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f5578103700 0x7f557819a070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:11:17.061 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:11:17.060+0000 7f557e000640 1 -- 172.21.15.22:0/2798285649 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f557819f060 con 0x7f5578109710 2024-03-28T18:11:17.061 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:11:17.060+0000 7f557d600640 1 --2- 172.21.15.22:0/2798285649 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f5578103700 0x7f557819a070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-28T18:11:17.062 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:11:17.061+0000 7f557e000640 1 --2- 172.21.15.22:0/2798285649 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5578109710 0x7f557819e930 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7f557000ec50 tx=0x7f557000c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:11:17.062 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:11:17.061+0000 7f5576000640 1 -- 172.21.15.22:0/2798285649 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5570003b70 con 0x7f5578109710 2024-03-28T18:11:17.062 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:11:17.061+0000 7f5576000640 1 -- 172.21.15.22:0/2798285649 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f5570003d10 con 0x7f5578109710 2024-03-28T18:11:17.062 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:11:17.061+0000 7f557ea00640 1 -- 172.21.15.22:0/2798285649 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f557819f350 con 0x7f5578109710 2024-03-28T18:11:17.062 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:11:17.061+0000 7f5576000640 1 -- 172.21.15.22:0/2798285649 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5570010890 con 0x7f5578109710 2024-03-28T18:11:17.062 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:11:17.061+0000 7f557ea00640 1 -- 172.21.15.22:0/2798285649 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f55781a6be0 con 0x7f5578109710 2024-03-28T18:11:17.064 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:11:17.065+0000 7f5582c7d740 1 Processor -- start 2024-03-28T18:11:17.065 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:11:17.066+0000 7f5576000640 1 -- 172.21.15.22:0/2798285649 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 26) v1 ==== 137421+0+0 (secure 0 0 0) 0x7f5570004000 con 0x7f5578109710 2024-03-28T18:11:17.065 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:11:17.066+0000 7f5582c7d740 1 -- start start 2024-03-28T18:11:17.067 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:11:17.066+0000 7f5582c7d740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5604768d9dd0 0x5604768d8b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:11:17.067 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:11:17.066+0000 7f5582c7d740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5604768d90f0 0x5604768d9570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:11:17.067 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:11:17.066+0000 7f5582c7d740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5604768e2460 0x5604768e4850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:11:17.067 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:11:17.066+0000 7f5582c7d740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x56047683d250 con 0x5604768d9dd0 2024-03-28T18:11:17.067 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:11:17.066+0000 7f5582c7d740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x5604767e5a30 con 0x5604768d90f0 2024-03-28T18:11:17.067 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:11:17.066+0000 7f5582c7d740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x5604768392f0 con 0x5604768e2460 2024-03-28T18:11:17.067 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:11:17.066+0000 7f557cc00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5604768d9dd0 0x5604768d8b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:11:17.067 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:11:17.067+0000 7f557cc00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5604768d9dd0 0x5604768d8b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:50346/0 (socket says 172.21.15.22:50346) 2024-03-28T18:11:17.068 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:11:17.067+0000 7f557cc00640 1 -- 172.21.15.22:0/3321626714 learned_addr learned my addr 172.21.15.22:0/3321626714 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-28T18:11:17.068 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:11:17.067+0000 7f557d600640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5604768d90f0 0x5604768d9570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:11:17.068 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:11:17.067+0000 7f557cc00640 1 --2- 172.21.15.22:0/3321626714 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5604768e2460 0x5604768e4850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:11:17.068 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:11:17.067+0000 7f557d600640 1 -- 172.21.15.22:0/3321626714 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5604768e2460 msgr2=0x5604768e4850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T18:11:17.068 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:11:17.067+0000 7f557d600640 1 --2- 172.21.15.22:0/3321626714 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5604768e2460 0x5604768e4850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:11:17.068 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:11:17.067+0000 7f557d600640 1 -- 172.21.15.22:0/3321626714 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5604768d9dd0 msgr2=0x5604768d8b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T18:11:17.068 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:11:17.067+0000 7f557d600640 1 --2- 172.21.15.22:0/3321626714 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5604768d9dd0 0x5604768d8b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:11:17.068 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:11:17.067+0000 7f557d600640 1 -- 172.21.15.22:0/3321626714 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5604767d0690 con 0x5604768d90f0 2024-03-28T18:11:17.068 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:11:17.067+0000 7f557cc00640 1 --2- 172.21.15.22:0/3321626714 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5604768e2460 0x5604768e4850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T18:11:17.068 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:11:17.067+0000 7f5576000640 1 --2- 172.21.15.22:0/2798285649 >> [v2:172.21.15.22:6800/1357365082,v1:172.21.15.22:6801/1357365082] conn(0x7f55600a2970 0x7f55600a4e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:11:17.068 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:11:17.067+0000 7f5576000640 1 -- 172.21.15.22:0/2798285649 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 8755+0+0 (secure 0 0 0) 0x7f55700cf4c0 con 0x7f5578109710 2024-03-28T18:11:17.068 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:11:17.067+0000 7f557d600640 1 --2- 172.21.15.22:0/3321626714 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5604768d90f0 0x5604768d9570 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f556c006010 tx=0x7f556c005e60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=85693ef36ba52147 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:11:17.068 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:11:17.067+0000 7f557cc00640 1 --2- 172.21.15.22:0/2798285649 >> [v2:172.21.15.22:6800/1357365082,v1:172.21.15.22:6801/1357365082] conn(0x7f55600a2970 0x7f55600a4e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:11:17.072 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:11:17.068+0000 7f556a600640 1 -- 172.21.15.22:0/3321626714 <== mon.2 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f556c004030 con 0x5604768d90f0 2024-03-28T18:11:17.072 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:11:17.068+0000 7f556a600640 1 -- 172.21.15.22:0/3321626714 <== mon.2 v2:172.21.15.79:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f556c02ca90 con 0x5604768d90f0 2024-03-28T18:11:17.072 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:11:17.068+0000 7f556a600640 1 -- 172.21.15.22:0/3321626714 <== mon.2 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f556c003af0 con 0x5604768d90f0 2024-03-28T18:11:17.073 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:11:17.069+0000 7f5582c7d740 1 -- 172.21.15.22:0/3321626714 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5604768d90f0 msgr2=0x5604768d9570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:11:17.073 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:11:17.069+0000 7f5582c7d740 1 --2- 172.21.15.22:0/3321626714 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5604768d90f0 0x5604768d9570 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f556c006010 tx=0x7f556c005e60 comp rx=0 tx=0).stop 2024-03-28T18:11:17.073 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:11:17.069+0000 7f557cc00640 1 --2- 172.21.15.22:0/2798285649 >> [v2:172.21.15.22:6800/1357365082,v1:172.21.15.22:6801/1357365082] conn(0x7f55600a2970 0x7f55600a4e30 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f5564009fd0 tx=0x7f55640097a0 comp rx=0 tx=0).ready entity=mgr.15522 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:11:17.073 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:11:17.069+0000 7f5582c7d740 1 -- 172.21.15.22:0/3321626714 shutdown_connections 2024-03-28T18:11:17.073 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:11:17.069+0000 7f5582c7d740 1 --2- 172.21.15.22:0/3321626714 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5604768e2460 0x5604768e4850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:11:17.073 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:11:17.069+0000 7f5582c7d740 1 --2- 172.21.15.22:0/3321626714 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5604768d90f0 0x5604768d9570 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:11:17.073 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:11:17.069+0000 7f5582c7d740 1 --2- 172.21.15.22:0/3321626714 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5604768d9dd0 0x5604768d8b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:11:17.073 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:11:17.069+0000 7f5582c7d740 1 -- 172.21.15.22:0/3321626714 >> 172.21.15.22:0/3321626714 conn(0x5604768a0200 msgr2=0x5604768d3b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:11:17.073 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:11:17.069+0000 7f5582c7d740 1 -- 172.21.15.22:0/3321626714 shutdown_connections 2024-03-28T18:11:17.073 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:11:17.069+0000 7f5582c7d740 1 -- 172.21.15.22:0/3321626714 wait complete. 2024-03-28T18:11:17.073 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:11:17.069+0000 7f5582c7d740 1 -- start start 2024-03-28T18:11:17.073 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:11:17.070+0000 7f5582c7d740 1 Processor -- start 2024-03-28T18:11:17.073 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:11:17.070+0000 7f5582c7d740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5604768d90f0 0x5604768e1ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:11:17.073 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:11:17.070+0000 7f5582c7d740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5604768d9dd0 0x5604768dee70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:11:17.074 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:11:17.070+0000 7f5582c7d740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5604768e2460 0x5604768df3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:11:17.074 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:11:17.070+0000 7f5582c7d740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x5604768e5270 con 0x5604768e2460 2024-03-28T18:11:17.074 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:11:17.070+0000 7f5582c7d740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x5604767f8e80 con 0x5604768d90f0 2024-03-28T18:11:17.074 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:11:17.070+0000 7f5582c7d740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x5604768e53f0 con 0x5604768d9dd0 2024-03-28T18:11:17.074 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:11:17.070+0000 7f5582c7d740 10 client.0 _ll_register_callbacks cb 0x56047680b130 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-28T18:11:17.074 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:11:17.070+0000 7f557e000640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5604768e2460 0x5604768df3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:11:17.074 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:11:17.070+0000 7f557e000640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5604768e2460 0x5604768df3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:50348/0 (socket says 172.21.15.22:50348) 2024-03-28T18:11:17.074 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:11:17.070+0000 7f557e000640 1 -- 172.21.15.22:0/4005325386 learned_addr learned my addr 172.21.15.22:0/4005325386 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-28T18:11:17.074 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:11:17.070+0000 7f557d600640 1 --2- 172.21.15.22:0/4005325386 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5604768d90f0 0x5604768e1ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:11:17.074 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:11:17.071+0000 7f557cc00640 1 --2- 172.21.15.22:0/4005325386 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5604768d9dd0 0x5604768dee70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:11:17.074 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:11:17.071+0000 7f557e000640 1 -- 172.21.15.22:0/4005325386 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5604768d9dd0 msgr2=0x5604768dee70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T18:11:17.074 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:11:17.071+0000 7f557e000640 1 --2- 172.21.15.22:0/4005325386 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5604768d9dd0 0x5604768dee70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:11:17.074 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:11:17.071+0000 7f557e000640 1 -- 172.21.15.22:0/4005325386 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5604768d90f0 msgr2=0x5604768e1ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T18:11:17.074 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:11:17.071+0000 7f557e000640 1 --2- 172.21.15.22:0/4005325386 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5604768d90f0 0x5604768e1ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:11:17.074 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:11:17.071+0000 7f557e000640 1 -- 172.21.15.22:0/4005325386 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56047661b7d0 con 0x5604768e2460 2024-03-28T18:11:17.075 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:11:17.071+0000 7f557e000640 1 --2- 172.21.15.22:0/4005325386 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5604768e2460 0x5604768df3b0 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7f5570098400 tx=0x7f557009b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b7816d29c648d590 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:11:17.075 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:11:17.071+0000 7f555ea00640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-03-28T18:11:17.075 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:11:17.071+0000 7f555ea00640 1 -- 172.21.15.22:0/4005325386 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55700044f0 con 0x5604768e2460 2024-03-28T18:11:17.075 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:11:17.071+0000 7f555ea00640 1 -- 172.21.15.22:0/4005325386 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(97..97 src has 1..97) v4 ==== 8755+0+0 (secure 0 0 0) 0x7f5570004b40 con 0x5604768e2460 2024-03-28T18:11:17.075 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:11:17.071+0000 7f5582c7d740 1 -- 172.21.15.22:0/4005325386 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f556c02ca90 con 0x5604768e2460 2024-03-28T18:11:17.075 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:11:17.072+0000 7f555ea00640 1 -- 172.21.15.22:0/4005325386 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=53) v2 ==== 24+0+0 (secure 0 0 0) 0x7f557009c130 con 0x5604768e2460 2024-03-28T18:11:17.075 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:11:17.072+0000 7f5582c7d740 10 client.16524 fetch_fsmap learned FSMap version 53 2024-03-28T18:11:17.075 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:11:17.072+0000 7f5582c7d740 1 -- 172.21.15.22:0/4005325386 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=53,monmap=4+}) v3 -- 0x5604765513f0 con 0x5604768e2460 2024-03-28T18:11:17.075 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:11:17.072+0000 7f555ea00640 1 -- 172.21.15.22:0/4005325386 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f556c02ca90 con 0x5604768e2460 2024-03-28T18:11:17.075 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:11:17.072+0000 7f555ea00640 1 -- 172.21.15.22:0/4005325386 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 53) v1 ==== 7954+0+0 (secure 0 0 0) 0x7f55700860d0 con 0x5604768e2460 2024-03-28T18:11:17.075 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:11:17.072+0000 7f5582c7d740 10 client.16524 fetch_fsmap finished waiting for FSMap version 53 2024-03-28T18:11:17.075 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:11:17.072+0000 7f5582c7d740 10 client.16524 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-28T18:11:17.075 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:11:17.072+0000 7f5582c7d740 20 client.16524 populate_metadata read hostname 'smithi022' 2024-03-28T18:11:17.075 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:11:17.072+0000 7f5582c7d740 1 --2- 172.21.15.22:0/4005325386 >> [v2:172.21.15.79:6838/2441542443,v1:172.21.15.79:6839/2441542443] conn(0x5604768f2260 0x5604768f4650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:11:17.075 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:11:17.072+0000 7f5582c7d740 4 client.16524 mds_command: new command op to 34553 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-28T18:11:17.076 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:11:17.072+0000 7f5582c7d740 1 -- 172.21.15.22:0/4005325386 --> [v2:172.21.15.79:6838/2441542443,v1:172.21.15.79:6839/2441542443] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56047684a790 con 0x5604768f2260 2024-03-28T18:11:17.076 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:11:17.073+0000 7f557d600640 1 --2- 172.21.15.22:0/4005325386 >> [v2:172.21.15.79:6838/2441542443,v1:172.21.15.79:6839/2441542443] conn(0x5604768f2260 0x5604768f4650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:11:17.077 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:11:17.074+0000 7f557d600640 1 --2- 172.21.15.22:0/4005325386 >> [v2:172.21.15.79:6838/2441542443,v1:172.21.15.79:6839/2441542443] conn(0x5604768f2260 0x5604768f4650 crc :-1 s=READY pgs=116 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f4d7b1ada648f1ef server_cookie=544c41c5e3b4b24d in_seq=0 out_seq=0 2024-03-28T18:11:17.077 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:11:17.074+0000 7f555ea00640 10 client.16524 ms_handle_connect on v2:172.21.15.79:6838/2441542443 2024-03-28T18:11:17.077 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:11:17.075+0000 7f555ea00640 1 -- 172.21.15.22:0/4005325386 <== mds.0 v2:172.21.15.79:6838/2441542443 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x56047684a790 con 0x5604768f2260 2024-03-28T18:11:17.077 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:11:17.075+0000 7f555ea00640 10 client.16524 handle_command_reply: tid=0 2024-03-28T18:11:17.077 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:11:17.075+0000 7f5582c7d740 1 client.16524 shutdown 2024-03-28T18:11:17.077 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:11:17.075+0000 7f5582c7d740 1 -- 172.21.15.22:0/4005325386 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5604768e2460 msgr2=0x5604768df3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:11:17.077 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:11:17.075+0000 7f5582c7d740 1 --2- 172.21.15.22:0/4005325386 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5604768e2460 0x5604768df3b0 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7f5570098400 tx=0x7f557009b8f0 comp rx=0 tx=0).stop 2024-03-28T18:11:17.077 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:11:17.075+0000 7f5582c7d740 1 -- 172.21.15.22:0/4005325386 shutdown_connections 2024-03-28T18:11:17.077 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:11:17.075+0000 7f5582c7d740 1 --2- 172.21.15.22:0/4005325386 >> [v2:172.21.15.79:6838/2441542443,v1:172.21.15.79:6839/2441542443] conn(0x5604768f2260 0x5604768f4650 crc :-1 s=READY pgs=116 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:11:17.077 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:11:17.075+0000 7f5582c7d740 1 --2- 172.21.15.22:0/4005325386 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5604768e2460 0x5604768df3b0 unknown :-1 s=CLOSED pgs=702 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:11:17.078 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:11:17.075+0000 7f5582c7d740 1 --2- 172.21.15.22:0/4005325386 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5604768d9dd0 0x5604768dee70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:11:17.078 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:11:17.075+0000 7f5582c7d740 1 --2- 172.21.15.22:0/4005325386 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5604768d90f0 0x5604768e1ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:11:17.078 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:11:17.075+0000 7f555ea00640 0 client.16524 ms_handle_reset on v2:172.21.15.79:6838/2441542443 2024-03-28T18:11:17.078 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:11:17.075+0000 7f5582c7d740 1 -- 172.21.15.22:0/4005325386 >> 172.21.15.22:0/4005325386 conn(0x5604768a0200 msgr2=0x5604768d5890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:11:17.078 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:11:17.076+0000 7f5582c7d740 1 -- 172.21.15.22:0/4005325386 shutdown_connections 2024-03-28T18:11:17.078 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:11:17.076+0000 7f5582c7d740 1 -- 172.21.15.22:0/4005325386 wait complete. 2024-03-28T18:11:17.078 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:11:17.076+0000 7f5582c7d740 20 client.16524 trim_cache size 0 max 16384 2024-03-28T18:11:17.098 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:11:17.097+0000 7f5582c7d740 1 Processor -- start 2024-03-28T18:11:17.098 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:11:17.097+0000 7f5582c7d740 1 -- start start 2024-03-28T18:11:17.098 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:11:17.097+0000 7f5582c7d740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5604768d90f0 0x5604768e15a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:11:17.099 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:11:17.098+0000 7f5582c7d740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5604768d9dd0 0x5604768e1ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:11:17.099 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:11:17.098+0000 7f5582c7d740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5604768e2460 0x5604768e3850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:11:17.099 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:11:17.098+0000 7f5582c7d740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55781af870 con 0x5604768d90f0 2024-03-28T18:11:17.099 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:11:17.098+0000 7f5582c7d740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55781b0c50 con 0x5604768e2460 2024-03-28T18:11:17.099 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:11:17.098+0000 7f5582c7d740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55781afed0 con 0x5604768d9dd0 2024-03-28T18:11:17.099 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:11:17.098+0000 7f557d600640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5604768d90f0 0x5604768e15a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:11:17.099 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:11:17.098+0000 7f557d600640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5604768d90f0 0x5604768e15a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:50358/0 (socket says 172.21.15.22:50358) 2024-03-28T18:11:17.099 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:11:17.098+0000 7f557d600640 1 -- 172.21.15.22:0/3259149745 learned_addr learned my addr 172.21.15.22:0/3259149745 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-28T18:11:17.099 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:11:17.098+0000 7f557e000640 1 --2- 172.21.15.22:0/3259149745 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5604768e2460 0x5604768e3850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:11:17.099 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:11:17.098+0000 7f557cc00640 1 --2- 172.21.15.22:0/3259149745 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5604768d9dd0 0x5604768e1ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:11:17.099 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:11:17.098+0000 7f557d600640 1 -- 172.21.15.22:0/3259149745 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5604768d9dd0 msgr2=0x5604768e1ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T18:11:17.099 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:11:17.098+0000 7f557d600640 1 --2- 172.21.15.22:0/3259149745 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5604768d9dd0 0x5604768e1ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:11:17.100 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:11:17.098+0000 7f557d600640 1 -- 172.21.15.22:0/3259149745 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5604768e2460 msgr2=0x5604768e3850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T18:11:17.100 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:11:17.098+0000 7f557d600640 1 --2- 172.21.15.22:0/3259149745 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5604768e2460 0x5604768e3850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:11:17.100 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:11:17.098+0000 7f557d600640 1 -- 172.21.15.22:0/3259149745 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5604768f49c0 con 0x5604768d90f0 2024-03-28T18:11:17.100 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:11:17.099+0000 7f557d600640 1 --2- 172.21.15.22:0/3259149745 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5604768d90f0 0x5604768e15a0 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7f556c006010 tx=0x7f556c041040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c097a2fa443e604e server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:11:17.102 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:11:17.099+0000 7f5569c00640 1 -- 172.21.15.22:0/3259149745 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f556c02c9c0 con 0x5604768d90f0 2024-03-28T18:11:17.103 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:11:17.099+0000 7f5569c00640 1 -- 172.21.15.22:0/3259149745 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f556c03f910 con 0x5604768d90f0 2024-03-28T18:11:17.103 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:11:17.099+0000 7f5569c00640 1 -- 172.21.15.22:0/3259149745 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f556c0346a0 con 0x5604768d90f0 2024-03-28T18:11:17.103 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:11:17.100+0000 7f5582c7d740 1 -- 172.21.15.22:0/3259149745 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5604768d90f0 msgr2=0x5604768e15a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:11:17.103 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:11:17.100+0000 7f5582c7d740 1 --2- 172.21.15.22:0/3259149745 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5604768d90f0 0x5604768e15a0 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7f556c006010 tx=0x7f556c041040 comp rx=0 tx=0).stop 2024-03-28T18:11:17.103 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:11:17.100+0000 7f5582c7d740 1 -- 172.21.15.22:0/3259149745 shutdown_connections 2024-03-28T18:11:17.103 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:11:17.100+0000 7f5582c7d740 1 --2- 172.21.15.22:0/3259149745 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5604768e2460 0x5604768e3850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:11:17.103 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:11:17.100+0000 7f5582c7d740 1 --2- 172.21.15.22:0/3259149745 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5604768d9dd0 0x5604768e1ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:11:17.103 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:11:17.100+0000 7f5582c7d740 1 --2- 172.21.15.22:0/3259149745 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5604768d90f0 0x5604768e15a0 unknown :-1 s=CLOSED pgs=703 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:11:17.103 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:11:17.100+0000 7f5582c7d740 1 -- 172.21.15.22:0/3259149745 >> 172.21.15.22:0/3259149745 conn(0x5604768a0200 msgr2=0x5604768d3670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:11:17.103 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:11:17.100+0000 7f5582c7d740 1 -- 172.21.15.22:0/3259149745 shutdown_connections 2024-03-28T18:11:17.103 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:11:17.100+0000 7f5582c7d740 1 -- 172.21.15.22:0/3259149745 wait complete. 2024-03-28T18:11:17.103 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:11:17.101+0000 7f5582c7d740 1 -- start start 2024-03-28T18:11:17.103 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:11:17.101+0000 7f5582c7d740 1 Processor -- start 2024-03-28T18:11:17.103 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:11:17.101+0000 7f5582c7d740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5604768d90f0 0x5604768db740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:11:17.104 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:11:17.101+0000 7f5582c7d740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5604768d9dd0 0x5604768dbc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:11:17.104 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:11:17.101+0000 7f5582c7d740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5604768e2460 0x5604768dc1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:11:17.104 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:11:17.101+0000 7f5582c7d740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55781adff0 con 0x5604768d90f0 2024-03-28T18:11:17.104 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:11:17.101+0000 7f5582c7d740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55781b05f0 con 0x5604768d9dd0 2024-03-28T18:11:17.104 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:11:17.101+0000 7f5582c7d740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55781ad3b0 con 0x5604768e2460 2024-03-28T18:11:17.104 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:11:17.101+0000 7f5582c7d740 10 client.0 _ll_register_callbacks cb 0x7f5570004a40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-28T18:11:17.104 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:11:17.102+0000 7f557d600640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5604768d90f0 0x5604768db740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:11:17.104 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:11:17.102+0000 7f557cc00640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5604768d9dd0 0x5604768dbc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:11:17.104 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:11:17.102+0000 7f557d600640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5604768d90f0 0x5604768db740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:50366/0 (socket says 172.21.15.22:50366) 2024-03-28T18:11:17.104 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:11:17.102+0000 7f557d600640 1 -- 172.21.15.22:0/1805876641 learned_addr learned my addr 172.21.15.22:0/1805876641 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-28T18:11:17.104 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:11:17.102+0000 7f557e000640 1 --2- 172.21.15.22:0/1805876641 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5604768e2460 0x5604768dc1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:11:17.104 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:11:17.102+0000 7f557d600640 1 -- 172.21.15.22:0/1805876641 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5604768e2460 msgr2=0x5604768dc1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T18:11:17.104 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:11:17.102+0000 7f557d600640 1 --2- 172.21.15.22:0/1805876641 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5604768e2460 0x5604768dc1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:11:17.104 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:11:17.102+0000 7f557d600640 1 -- 172.21.15.22:0/1805876641 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5604768d9dd0 msgr2=0x5604768dbc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T18:11:17.104 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:11:17.102+0000 7f557d600640 1 --2- 172.21.15.22:0/1805876641 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5604768d9dd0 0x5604768dbc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:11:17.105 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:11:17.102+0000 7f557d600640 1 -- 172.21.15.22:0/1805876641 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5604765110f0 con 0x5604768d90f0 2024-03-28T18:11:17.105 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:11:17.102+0000 7f557d600640 1 --2- 172.21.15.22:0/1805876641 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5604768d90f0 0x5604768db740 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7f556c02bc60 tx=0x7f556c02c7c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=66082fd5aed9c51e server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:11:17.105 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:11:17.102+0000 7f555ea00640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-03-28T18:11:17.105 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:11:17.103+0000 7f555ea00640 1 -- 172.21.15.22:0/1805876641 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f556c062060 con 0x5604768d90f0 2024-03-28T18:11:17.105 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:11:17.103+0000 7f555ea00640 1 -- 172.21.15.22:0/1805876641 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(97..97 src has 1..97) v4 ==== 8755+0+0 (secure 0 0 0) 0x7f556c03f650 con 0x5604768d90f0 2024-03-28T18:11:17.105 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:11:17.103+0000 7f5582c7d740 1 -- 172.21.15.22:0/1805876641 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f556c03f910 con 0x5604768d90f0 2024-03-28T18:11:17.105 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:11:17.103+0000 7f555ea00640 1 -- 172.21.15.22:0/1805876641 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=53) v2 ==== 24+0+0 (secure 0 0 0) 0x7f556c069050 con 0x5604768d90f0 2024-03-28T18:11:17.105 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:11:17.103+0000 7f5582c7d740 10 client.16530 fetch_fsmap learned FSMap version 53 2024-03-28T18:11:17.105 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:11:17.103+0000 7f5582c7d740 1 -- 172.21.15.22:0/1805876641 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=53,monmap=4+}) v3 -- 0x56047694dc70 con 0x5604768d90f0 2024-03-28T18:11:17.107 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:11:17.103+0000 7f555ea00640 1 -- 172.21.15.22:0/1805876641 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f556c03f910 con 0x5604768d90f0 2024-03-28T18:11:17.107 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:11:17.103+0000 7f555ea00640 1 -- 172.21.15.22:0/1805876641 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 53) v1 ==== 7954+0+0 (secure 0 0 0) 0x7f556c053cd0 con 0x5604768d90f0 2024-03-28T18:11:17.107 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:11:17.104+0000 7f5582c7d740 10 client.16530 fetch_fsmap finished waiting for FSMap version 53 2024-03-28T18:11:17.107 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:11:17.104+0000 7f5582c7d740 10 client.16530 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-28T18:11:17.107 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:11:17.104+0000 7f5582c7d740 20 client.16530 populate_metadata read hostname 'smithi022' 2024-03-28T18:11:17.107 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:11:17.104+0000 7f5582c7d740 1 --2- 172.21.15.22:0/1805876641 >> [v2:172.21.15.79:6838/2441542443,v1:172.21.15.79:6839/2441542443] conn(0x56047694df90 0x560476950380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:11:17.107 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:11:17.104+0000 7f5582c7d740 4 client.16530 mds_command: new command op to 34553 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-28T18:11:17.107 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:11:17.104+0000 7f5582c7d740 1 -- 172.21.15.22:0/1805876641 --> [v2:172.21.15.79:6838/2441542443,v1:172.21.15.79:6839/2441542443] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x560476844ae0 con 0x56047694df90 2024-03-28T18:11:17.107 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:11:17.104+0000 7f557cc00640 1 --2- 172.21.15.22:0/1805876641 >> [v2:172.21.15.79:6838/2441542443,v1:172.21.15.79:6839/2441542443] conn(0x56047694df90 0x560476950380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:11:17.107 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:11:17.105+0000 7f557cc00640 1 --2- 172.21.15.22:0/1805876641 >> [v2:172.21.15.79:6838/2441542443,v1:172.21.15.79:6839/2441542443] conn(0x56047694df90 0x560476950380 crc :-1 s=READY pgs=117 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=87ce6a04e1ed8b4b server_cookie=f6804ab26752be15 in_seq=0 out_seq=0 2024-03-28T18:11:17.108 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:11:17.105+0000 7f555ea00640 10 client.16530 ms_handle_connect on v2:172.21.15.79:6838/2441542443 2024-03-28T18:11:17.108 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:11:17.105+0000 7f555ea00640 1 -- 172.21.15.22:0/1805876641 <== mds.0 v2:172.21.15.79:6838/2441542443 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x560476844ae0 con 0x56047694df90 2024-03-28T18:11:17.108 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:11:17.105+0000 7f555ea00640 10 client.16530 handle_command_reply: tid=0 2024-03-28T18:11:17.108 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:11:17.105+0000 7f5582c7d740 1 client.16530 shutdown 2024-03-28T18:11:17.108 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:11:17.106+0000 7f5582c7d740 1 -- 172.21.15.22:0/1805876641 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5604768d90f0 msgr2=0x5604768db740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:11:17.108 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:11:17.106+0000 7f5582c7d740 1 --2- 172.21.15.22:0/1805876641 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5604768d90f0 0x5604768db740 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7f556c02bc60 tx=0x7f556c02c7c0 comp rx=0 tx=0).stop 2024-03-28T18:11:17.108 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:11:17.106+0000 7f5582c7d740 1 -- 172.21.15.22:0/1805876641 shutdown_connections 2024-03-28T18:11:17.108 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:11:17.106+0000 7f5582c7d740 1 --2- 172.21.15.22:0/1805876641 >> [v2:172.21.15.79:6838/2441542443,v1:172.21.15.79:6839/2441542443] conn(0x56047694df90 0x560476950380 crc :-1 s=READY pgs=117 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:11:17.108 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:11:17.106+0000 7f5582c7d740 1 --2- 172.21.15.22:0/1805876641 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5604768e2460 0x5604768dc1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:11:17.108 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:11:17.106+0000 7f5582c7d740 1 --2- 172.21.15.22:0/1805876641 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5604768d9dd0 0x5604768dbc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:11:17.108 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:11:17.106+0000 7f5582c7d740 1 --2- 172.21.15.22:0/1805876641 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5604768d90f0 0x5604768db740 unknown :-1 s=CLOSED pgs=704 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:11:17.108 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:11:17.106+0000 7f555ea00640 0 client.16530 ms_handle_reset on v2:172.21.15.79:6838/2441542443 2024-03-28T18:11:17.108 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:11:17.106+0000 7f5582c7d740 1 -- 172.21.15.22:0/1805876641 >> 172.21.15.22:0/1805876641 conn(0x5604768a0200 msgr2=0x5604768dd340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:11:17.109 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:11:17.106+0000 7f5582c7d740 1 -- 172.21.15.22:0/1805876641 shutdown_connections 2024-03-28T18:11:17.109 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:11:17.106+0000 7f5582c7d740 1 -- 172.21.15.22:0/1805876641 wait complete. 2024-03-28T18:11:17.109 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:11:17.106+0000 7f5582c7d740 20 client.16530 trim_cache size 0 max 16384 2024-03-28T18:11:17.109 INFO:teuthology.orchestra.run.smithi022.stdout:{ 2024-03-28T18:11:17.109 INFO:teuthology.orchestra.run.smithi022.stdout: "status": "IDLE (waiting for more scrubs)", 2024-03-28T18:11:17.109 INFO:teuthology.orchestra.run.smithi022.stdout: "scrubs": { 2024-03-28T18:11:17.109 INFO:teuthology.orchestra.run.smithi022.stdout: "4bbbd61d-29a3-4bf6-a4f2-d5783d585de3": { 2024-03-28T18:11:17.109 INFO:teuthology.orchestra.run.smithi022.stdout: "path": "/", 2024-03-28T18:11:17.109 INFO:teuthology.orchestra.run.smithi022.stdout: "tag": "4bbbd61d-29a3-4bf6-a4f2-d5783d585de3", 2024-03-28T18:11:17.109 INFO:teuthology.orchestra.run.smithi022.stdout: "options": "recursive,force" 2024-03-28T18:11:17.110 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-03-28T18:11:17.110 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-03-28T18:11:17.110 INFO:teuthology.orchestra.run.smithi022.stdout:} 2024-03-28T18:11:17.110 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:11:17.107+0000 7f557ea00640 1 -- 172.21.15.22:0/2798285649 >> [v2:172.21.15.22:6800/1357365082,v1:172.21.15.22:6801/1357365082] conn(0x7f55600a2970 msgr2=0x7f55600a4e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:11:17.111 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:11:17.107+0000 7f557ea00640 1 --2- 172.21.15.22:0/2798285649 >> [v2:172.21.15.22:6800/1357365082,v1:172.21.15.22:6801/1357365082] conn(0x7f55600a2970 0x7f55600a4e30 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f5564009fd0 tx=0x7f55640097a0 comp rx=0 tx=0).stop 2024-03-28T18:11:17.111 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:11:17.107+0000 7f557ea00640 1 -- 172.21.15.22:0/2798285649 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5578109710 msgr2=0x7f557819e930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:11:17.111 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:11:17.107+0000 7f557ea00640 1 --2- 172.21.15.22:0/2798285649 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5578109710 0x7f557819e930 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7f557000ec50 tx=0x7f557000c6a0 comp rx=0 tx=0).stop 2024-03-28T18:11:17.111 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:11:17.107+0000 7f557ea00640 1 -- 172.21.15.22:0/2798285649 shutdown_connections 2024-03-28T18:11:17.111 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:11:17.107+0000 7f557ea00640 1 --2- 172.21.15.22:0/2798285649 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5578109710 0x7f557819e930 unknown :-1 s=CLOSED pgs=701 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:11:17.111 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:11:17.107+0000 7f557ea00640 1 --2- 172.21.15.22:0/2798285649 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f55781040c0 0x7f557819a5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:11:17.111 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:11:17.107+0000 7f557ea00640 1 --2- 172.21.15.22:0/2798285649 >> [v2:172.21.15.22:6800/1357365082,v1:172.21.15.22:6801/1357365082] conn(0x7f55600a2970 0x7f55600a4e30 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:11:17.111 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:11:17.107+0000 7f557ea00640 1 --2- 172.21.15.22:0/2798285649 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f5578103700 0x7f557819a070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:11:17.111 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:11:17.107+0000 7f557ea00640 1 -- 172.21.15.22:0/2798285649 >> 172.21.15.22:0/2798285649 conn(0x7f55780ff1b0 msgr2=0x7f55780ff920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:11:17.111 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:11:17.107+0000 7f557ea00640 1 -- 172.21.15.22:0/2798285649 shutdown_connections 2024-03-28T18:11:17.111 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:11:17.107+0000 7f557ea00640 1 -- 172.21.15.22:0/2798285649 wait complete. 2024-03-28T18:11:17.126 INFO:tasks.cephfs.filesystem:scrub status for tag:4bbbd61d-29a3-4bf6-a4f2-d5783d585de3 - {'path': '/', 'tag': '4bbbd61d-29a3-4bf6-a4f2-d5783d585de3', 'options': 'recursive,force'} 2024-03-28T18:11:17.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:11:17 smithi022 ceph-mon[31985]: daemon mds.f restarted 2024-03-28T18:11:17.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:11:17 smithi022 ceph-mon[31985]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-03-28T18:11:17.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:11:17 smithi022 ceph-mon[31985]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-03-28T18:11:17.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:11:17 smithi022 ceph-mon[31985]: osdmap e97: 12 total, 12 up, 12 in 2024-03-28T18:11:17.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:11:17 smithi022 ceph-mon[31985]: mds.? [v2:172.21.15.134:6834/568477027,v1:172.21.15.134:6835/568477027] up:boot 2024-03-28T18:11:17.785 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:11:17 smithi022 ceph-mon[31985]: Standby daemon mds.h assigned to filesystem cephfs as rank 1 2024-03-28T18:11:17.785 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:11:17 smithi022 ceph-mon[31985]: fsmap cephfs:4/5 {0=b=up:active,2=e=up:active,3=k=up:active,4=i=up:active} 5 up:standby-replay 3 up:standby, 1 failed 2024-03-28T18:11:17.785 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:11:17 smithi022 ceph-mon[31985]: fsmap cephfs:5/5 4 up:standby-replay 3 up:standby 1 up:replay 4 up:active 2024-03-28T18:11:17.861 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:11:17 smithi134 ceph-mon[35037]: daemon mds.f restarted 2024-03-28T18:11:17.861 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:11:17 smithi134 ceph-mon[35037]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-03-28T18:11:17.861 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:11:17 smithi134 ceph-mon[35037]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-03-28T18:11:17.861 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:11:17 smithi134 ceph-mon[35037]: osdmap e97: 12 total, 12 up, 12 in 2024-03-28T18:11:17.861 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:11:17 smithi134 ceph-mon[35037]: mds.? [v2:172.21.15.134:6834/568477027,v1:172.21.15.134:6835/568477027] up:boot 2024-03-28T18:11:17.861 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:11:17 smithi134 ceph-mon[35037]: Standby daemon mds.h assigned to filesystem cephfs as rank 1 2024-03-28T18:11:17.861 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:11:17 smithi134 ceph-mon[35037]: fsmap cephfs:4/5 {0=b=up:active,2=e=up:active,3=k=up:active,4=i=up:active} 5 up:standby-replay 3 up:standby, 1 failed 2024-03-28T18:11:17.861 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:11:17 smithi134 ceph-mon[35037]: fsmap cephfs:5/5 4 up:standby-replay 3 up:standby 1 up:replay 4 up:active 2024-03-28T18:11:17.933 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:11:17 smithi079 ceph-mon[35312]: daemon mds.f restarted 2024-03-28T18:11:17.933 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:11:17 smithi079 ceph-mon[35312]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-03-28T18:11:17.933 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:11:17 smithi079 ceph-mon[35312]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-03-28T18:11:17.933 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:11:17 smithi079 ceph-mon[35312]: osdmap e97: 12 total, 12 up, 12 in 2024-03-28T18:11:17.933 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:11:17 smithi079 ceph-mon[35312]: mds.? [v2:172.21.15.134:6834/568477027,v1:172.21.15.134:6835/568477027] up:boot 2024-03-28T18:11:17.933 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:11:17 smithi079 ceph-mon[35312]: Standby daemon mds.h assigned to filesystem cephfs as rank 1 2024-03-28T18:11:17.933 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:11:17 smithi079 ceph-mon[35312]: fsmap cephfs:4/5 {0=b=up:active,2=e=up:active,3=k=up:active,4=i=up:active} 5 up:standby-replay 3 up:standby, 1 failed 2024-03-28T18:11:17.933 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:11:17 smithi079 ceph-mon[35312]: fsmap cephfs:5/5 4 up:standby-replay 3 up:standby 1 up:replay 4 up:active 2024-03-28T18:11:18.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:11:18 smithi134 ceph-mon[35037]: pgmap v119: 97 pgs: 97 active+clean; 117 MiB data, 800 MiB used, 1.0 TiB / 1.0 TiB avail; 117 KiB/s rd, 29 KiB/s wr, 41 op/s 2024-03-28T18:11:18.925 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:11:18 smithi134 ceph-mon[35037]: mds.? [v2:172.21.15.79:6836/2541118119,v1:172.21.15.79:6837/2541118119] up:resolve 2024-03-28T18:11:18.925 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:11:18 smithi134 ceph-mon[35037]: fsmap cephfs:5/5 4 up:standby-replay 3 up:standby 1 up:resolve 4 up:active 2024-03-28T18:11:19.031 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:11:18 smithi022 ceph-mon[31985]: pgmap v119: 97 pgs: 97 active+clean; 117 MiB data, 800 MiB used, 1.0 TiB / 1.0 TiB avail; 117 KiB/s rd, 29 KiB/s wr, 41 op/s 2024-03-28T18:11:19.032 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:11:18 smithi022 ceph-mon[31985]: mds.? [v2:172.21.15.79:6836/2541118119,v1:172.21.15.79:6837/2541118119] up:resolve 2024-03-28T18:11:19.032 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:11:18 smithi022 ceph-mon[31985]: fsmap cephfs:5/5 4 up:standby-replay 3 up:standby 1 up:resolve 4 up:active 2024-03-28T18:11:19.183 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:11:18 smithi079 ceph-mon[35312]: pgmap v119: 97 pgs: 97 active+clean; 117 MiB data, 800 MiB used, 1.0 TiB / 1.0 TiB avail; 117 KiB/s rd, 29 KiB/s wr, 41 op/s 2024-03-28T18:11:19.183 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:11:18 smithi079 ceph-mon[35312]: mds.? [v2:172.21.15.79:6836/2541118119,v1:172.21.15.79:6837/2541118119] up:resolve 2024-03-28T18:11:19.183 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:11:18 smithi079 ceph-mon[35312]: fsmap cephfs:5/5 4 up:standby-replay 3 up:standby 1 up:resolve 4 up:active 2024-03-28T18:11:20.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:11:20 smithi022 ceph-mon[31985]: pgmap v120: 97 pgs: 97 active+clean; 117 MiB data, 800 MiB used, 1.0 TiB / 1.0 TiB avail; 116 KiB/s rd, 29 KiB/s wr, 40 op/s 2024-03-28T18:11:20.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:11:20 smithi022 ceph-mon[31985]: reconnect by client.35222 172.21.15.22:0/3729116781 after 0.00399991 2024-03-28T18:11:20.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:11:20 smithi022 ceph-mon[31985]: mds.? [v2:172.21.15.79:6836/2541118119,v1:172.21.15.79:6837/2541118119] up:reconnect 2024-03-28T18:11:20.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:11:20 smithi022 ceph-mon[31985]: fsmap cephfs:5/5 4 up:standby-replay 3 up:standby 1 up:reconnect 4 up:active 2024-03-28T18:11:20.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:11:20 smithi134 ceph-mon[35037]: pgmap v120: 97 pgs: 97 active+clean; 117 MiB data, 800 MiB used, 1.0 TiB / 1.0 TiB avail; 116 KiB/s rd, 29 KiB/s wr, 40 op/s 2024-03-28T18:11:20.925 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:11:20 smithi134 ceph-mon[35037]: reconnect by client.35222 172.21.15.22:0/3729116781 after 0.00399991 2024-03-28T18:11:20.925 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:11:20 smithi134 ceph-mon[35037]: mds.? [v2:172.21.15.79:6836/2541118119,v1:172.21.15.79:6837/2541118119] up:reconnect 2024-03-28T18:11:20.925 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:11:20 smithi134 ceph-mon[35037]: fsmap cephfs:5/5 4 up:standby-replay 3 up:standby 1 up:reconnect 4 up:active 2024-03-28T18:11:20.933 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:11:20 smithi079 ceph-mon[35312]: pgmap v120: 97 pgs: 97 active+clean; 117 MiB data, 800 MiB used, 1.0 TiB / 1.0 TiB avail; 116 KiB/s rd, 29 KiB/s wr, 40 op/s 2024-03-28T18:11:20.933 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:11:20 smithi079 ceph-mon[35312]: reconnect by client.35222 172.21.15.22:0/3729116781 after 0.00399991 2024-03-28T18:11:20.933 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:11:20 smithi079 ceph-mon[35312]: mds.? [v2:172.21.15.79:6836/2541118119,v1:172.21.15.79:6837/2541118119] up:reconnect 2024-03-28T18:11:20.933 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:11:20 smithi079 ceph-mon[35312]: fsmap cephfs:5/5 4 up:standby-replay 3 up:standby 1 up:reconnect 4 up:active 2024-03-28T18:11:21.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:11:21 smithi022 ceph-mon[31985]: mds.? [v2:172.21.15.79:6836/2541118119,v1:172.21.15.79:6837/2541118119] up:rejoin 2024-03-28T18:11:21.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:11:21 smithi022 ceph-mon[31985]: fsmap cephfs:5/5 4 up:standby-replay 3 up:standby 1 up:rejoin 4 up:active 2024-03-28T18:11:21.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:11:21 smithi022 ceph-mon[31985]: daemon mds.h is now active in filesystem cephfs as rank 1 2024-03-28T18:11:21.925 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:11:21 smithi134 ceph-mon[35037]: mds.? [v2:172.21.15.79:6836/2541118119,v1:172.21.15.79:6837/2541118119] up:rejoin 2024-03-28T18:11:21.925 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:11:21 smithi134 ceph-mon[35037]: fsmap cephfs:5/5 4 up:standby-replay 3 up:standby 1 up:rejoin 4 up:active 2024-03-28T18:11:21.925 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:11:21 smithi134 ceph-mon[35037]: daemon mds.h is now active in filesystem cephfs as rank 1 2024-03-28T18:11:21.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:11:21 smithi079 ceph-mon[35312]: mds.? [v2:172.21.15.79:6836/2541118119,v1:172.21.15.79:6837/2541118119] up:rejoin 2024-03-28T18:11:21.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:11:21 smithi079 ceph-mon[35312]: fsmap cephfs:5/5 4 up:standby-replay 3 up:standby 1 up:rejoin 4 up:active 2024-03-28T18:11:21.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:11:21 smithi079 ceph-mon[35312]: daemon mds.h is now active in filesystem cephfs as rank 1 2024-03-28T18:11:21.933 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:11:21 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69321]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-2145-g3be9a5c9/rpm/el8/BUILD/ceph-19.0.0-2145-g3be9a5c9/src/mds/Locker.cc: In function 'void Locker::set_xlocks_done(MutationImpl*, bool)' thread 7f50e0600700 time 2024-03-28T18:11:21.592061+0000 2024-03-28T18:11:21.933 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:11:21 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69321]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-2145-g3be9a5c9/rpm/el8/BUILD/ceph-19.0.0-2145-g3be9a5c9/src/mds/Locker.cc: 626: FAILED ceph_assert(obj->is_auth()) 2024-03-28T18:11:21.933 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:11:21 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69321]: ceph version 19.0.0-2145-g3be9a5c9 (3be9a5c9bc793e11e2800a8c0c696e8b46742033) squid (dev) 2024-03-28T18:11:21.933 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:11:21 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69321]: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x135) [0x7f50ebfd64a1] 2024-03-28T18:11:21.933 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:11:21 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69321]: 2: /usr/lib64/ceph/libceph-common.so.2(+0x2d2667) [0x7f50ebfd6667] 2024-03-28T18:11:21.934 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:11:21 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69321]: 3: (Locker::set_xlocks_done(MutationImpl*, bool)+0x493) [0x55a0a4ac5223] 2024-03-28T18:11:21.934 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:11:21 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69321]: 4: (Server::early_reply(boost::intrusive_ptr const&, CInode*, CDentry*)+0x3e6) [0x55a0a493f5d6] 2024-03-28T18:11:21.934 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:11:21 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69321]: 5: (Server::journal_and_reply(boost::intrusive_ptr const&, CInode*, CDentry*, LogEvent*, MDSLogContextBase*)+0xc1) [0x55a0a493fda1] 2024-03-28T18:11:21.934 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:11:21 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69321]: 6: (Server::handle_client_mkdir(boost::intrusive_ptr const&)+0x5ba) [0x55a0a49756aa] 2024-03-28T18:11:21.934 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:11:21 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69321]: 7: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0x733) [0x55a0a4992233] 2024-03-28T18:11:21.934 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:11:21 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69321]: 8: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x55a0a4994197] 2024-03-28T18:11:21.934 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:11:21 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69321]: 9: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x55a0a489311b] 2024-03-28T18:11:21.934 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:11:21 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69321]: 10: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x55a0a48936bc] 2024-03-28T18:11:21.934 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:11:21 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69321]: 11: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bf) [0x55a0a487d1ff] 2024-03-28T18:11:21.934 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:11:21 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69321]: 12: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x478) [0x7f50ec268488] 2024-03-28T18:11:21.934 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:11:21 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69321]: 13: (DispatchQueue::entry()+0x51f) [0x7f50ec26560f] 2024-03-28T18:11:21.934 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:11:21 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69321]: 14: (DispatchQueue::DispatchThread::entry()+0x11) [0x7f50ec338e81] 2024-03-28T18:11:21.935 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:11:21 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69321]: 15: /lib64/libpthread.so.0(+0x81ca) [0x7f50ead531ca] 2024-03-28T18:11:21.935 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:11:21 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69321]: 16: clone() 2024-03-28T18:11:21.935 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:11:21 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69321]: 2024-03-28T18:11:21.593+0000 7f50e0600700 -1 /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-2145-g3be9a5c9/rpm/el8/BUILD/ceph-19.0.0-2145-g3be9a5c9/src/mds/Locker.cc: In function 'void Locker::set_xlocks_done(MutationImpl*, bool)' thread 7f50e0600700 time 2024-03-28T18:11:21.592061+0000 2024-03-28T18:11:21.935 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:11:21 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69321]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-2145-g3be9a5c9/rpm/el8/BUILD/ceph-19.0.0-2145-g3be9a5c9/src/mds/Locker.cc: 626: FAILED ceph_assert(obj->is_auth()) 2024-03-28T18:11:21.935 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:11:21 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69321]: 2024-03-28T18:11:21.935 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:11:21 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69321]: ceph version 19.0.0-2145-g3be9a5c9 (3be9a5c9bc793e11e2800a8c0c696e8b46742033) squid (dev) 2024-03-28T18:11:21.935 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:11:21 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69321]: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x135) [0x7f50ebfd64a1] 2024-03-28T18:11:21.935 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:11:21 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69321]: 2: /usr/lib64/ceph/libceph-common.so.2(+0x2d2667) [0x7f50ebfd6667] 2024-03-28T18:11:21.936 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:11:21 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69321]: 3: (Locker::set_xlocks_done(MutationImpl*, bool)+0x493) [0x55a0a4ac5223] 2024-03-28T18:11:21.936 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:11:21 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69321]: 4: (Server::early_reply(boost::intrusive_ptr const&, CInode*, CDentry*)+0x3e6) [0x55a0a493f5d6] 2024-03-28T18:11:21.936 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:11:21 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69321]: 5: (Server::journal_and_reply(boost::intrusive_ptr const&, CInode*, CDentry*, LogEvent*, MDSLogContextBase*)+0xc1) [0x55a0a493fda1] 2024-03-28T18:11:21.936 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:11:21 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69321]: 6: (Server::handle_client_mkdir(boost::intrusive_ptr const&)+0x5ba) [0x55a0a49756aa] 2024-03-28T18:11:21.936 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:11:21 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69321]: 7: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0x733) [0x55a0a4992233] 2024-03-28T18:11:21.936 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:11:21 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69321]: 8: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x55a0a4994197] 2024-03-28T18:11:21.936 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:11:21 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69321]: 9: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x55a0a489311b] 2024-03-28T18:11:21.936 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:11:21 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69321]: 10: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x55a0a48936bc] 2024-03-28T18:11:21.936 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:11:21 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69321]: 11: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bf) [0x55a0a487d1ff] 2024-03-28T18:11:21.936 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:11:21 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69321]: 12: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x478) [0x7f50ec268488] 2024-03-28T18:11:21.936 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:11:21 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69321]: 13: (DispatchQueue::entry()+0x51f) [0x7f50ec26560f] 2024-03-28T18:11:21.936 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:11:21 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69321]: 14: (DispatchQueue::DispatchThread::entry()+0x11) [0x7f50ec338e81] 2024-03-28T18:11:21.937 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:11:21 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69321]: 15: /lib64/libpthread.so.0(+0x81ca) [0x7f50ead531ca] 2024-03-28T18:11:21.937 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:11:21 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69321]: 16: clone() 2024-03-28T18:11:21.937 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:11:21 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69321]: 2024-03-28T18:11:21.937 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:11:21 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69321]: *** Caught signal (Aborted) ** 2024-03-28T18:11:21.937 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:11:21 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69321]: in thread 7f50e0600700 thread_name:ms_dispatch 2024-03-28T18:11:21.937 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:11:21 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69321]: ceph version 19.0.0-2145-g3be9a5c9 (3be9a5c9bc793e11e2800a8c0c696e8b46742033) squid (dev) 2024-03-28T18:11:21.937 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:11:21 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69321]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7f50ead5dd20] 2024-03-28T18:11:21.938 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:11:21 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69321]: 2: gsignal() 2024-03-28T18:11:21.938 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:11:21 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69321]: 3: abort() 2024-03-28T18:11:21.938 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:11:21 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69321]: 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x18f) [0x7f50ebfd64fb] 2024-03-28T18:11:21.938 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:11:21 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69321]: 5: /usr/lib64/ceph/libceph-common.so.2(+0x2d2667) [0x7f50ebfd6667] 2024-03-28T18:11:21.938 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:11:21 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69321]: 6: (Locker::set_xlocks_done(MutationImpl*, bool)+0x493) [0x55a0a4ac5223] 2024-03-28T18:11:21.938 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:11:21 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69321]: 7: (Server::early_reply(boost::intrusive_ptr const&, CInode*, CDentry*)+0x3e6) [0x55a0a493f5d6] 2024-03-28T18:11:21.938 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:11:21 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69321]: 8: (Server::journal_and_reply(boost::intrusive_ptr const&, CInode*, CDentry*, LogEvent*, MDSLogContextBase*)+0xc1) [0x55a0a493fda1] 2024-03-28T18:11:21.938 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:11:21 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69321]: 9: (Server::handle_client_mkdir(boost::intrusive_ptr const&)+0x5ba) [0x55a0a49756aa] 2024-03-28T18:11:21.938 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:11:21 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69321]: 10: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0x733) [0x55a0a4992233] 2024-03-28T18:11:21.938 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:11:21 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69321]: 11: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x55a0a4994197] 2024-03-28T18:11:21.939 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:11:21 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69321]: 12: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x55a0a489311b] 2024-03-28T18:11:21.939 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:11:21 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69321]: 13: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x55a0a48936bc] 2024-03-28T18:11:21.939 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:11:21 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69321]: 14: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bf) [0x55a0a487d1ff] 2024-03-28T18:11:21.939 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:11:21 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69321]: 15: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x478) [0x7f50ec268488] 2024-03-28T18:11:21.939 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:11:21 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69321]: 16: (DispatchQueue::entry()+0x51f) [0x7f50ec26560f] 2024-03-28T18:11:21.939 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:11:21 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69321]: 17: (DispatchQueue::DispatchThread::entry()+0x11) [0x7f50ec338e81] 2024-03-28T18:11:21.939 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:11:21 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69321]: 18: /lib64/libpthread.so.0(+0x81ca) [0x7f50ead531ca] 2024-03-28T18:11:21.939 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:11:21 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69321]: 19: clone() 2024-03-28T18:11:21.939 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:11:21 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69321]: 2024-03-28T18:11:21.595+0000 7f50e0600700 -1 *** Caught signal (Aborted) ** 2024-03-28T18:11:21.939 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:11:21 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69321]: in thread 7f50e0600700 thread_name:ms_dispatch 2024-03-28T18:11:21.939 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:11:21 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69321]: 2024-03-28T18:11:21.939 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:11:21 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69321]: ceph version 19.0.0-2145-g3be9a5c9 (3be9a5c9bc793e11e2800a8c0c696e8b46742033) squid (dev) 2024-03-28T18:11:21.940 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:11:21 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69321]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7f50ead5dd20] 2024-03-28T18:11:21.940 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:11:21 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69321]: 2: gsignal() 2024-03-28T18:11:21.940 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:11:21 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69321]: 3: abort() 2024-03-28T18:11:21.940 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:11:21 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69321]: 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x18f) [0x7f50ebfd64fb] 2024-03-28T18:11:21.940 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:11:21 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69321]: 5: /usr/lib64/ceph/libceph-common.so.2(+0x2d2667) [0x7f50ebfd6667] 2024-03-28T18:11:21.940 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:11:21 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69321]: 6: (Locker::set_xlocks_done(MutationImpl*, bool)+0x493) [0x55a0a4ac5223] 2024-03-28T18:11:21.940 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:11:21 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69321]: 7: (Server::early_reply(boost::intrusive_ptr const&, CInode*, CDentry*)+0x3e6) [0x55a0a493f5d6] 2024-03-28T18:11:21.940 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:11:21 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69321]: 8: (Server::journal_and_reply(boost::intrusive_ptr const&, CInode*, CDentry*, LogEvent*, MDSLogContextBase*)+0xc1) [0x55a0a493fda1] 2024-03-28T18:11:21.940 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:11:21 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69321]: 9: (Server::handle_client_mkdir(boost::intrusive_ptr const&)+0x5ba) [0x55a0a49756aa] 2024-03-28T18:11:21.940 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:11:21 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69321]: 10: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0x733) [0x55a0a4992233] 2024-03-28T18:11:21.940 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:11:21 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69321]: 11: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x55a0a4994197] 2024-03-28T18:11:21.940 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:11:21 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69321]: 12: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x55a0a489311b] 2024-03-28T18:11:21.940 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:11:21 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69321]: 13: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x55a0a48936bc] 2024-03-28T18:11:21.941 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:11:21 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69321]: 14: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bf) [0x55a0a487d1ff] 2024-03-28T18:11:21.941 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:11:21 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69321]: 15: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x478) [0x7f50ec268488] 2024-03-28T18:11:21.941 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:11:21 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69321]: 16: (DispatchQueue::entry()+0x51f) [0x7f50ec26560f] 2024-03-28T18:11:21.941 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:11:21 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69321]: 17: (DispatchQueue::DispatchThread::entry()+0x11) [0x7f50ec338e81] 2024-03-28T18:11:21.941 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:11:21 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69321]: 18: /lib64/libpthread.so.0(+0x81ca) [0x7f50ead531ca] 2024-03-28T18:11:21.941 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:11:21 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69321]: 19: clone() 2024-03-28T18:11:21.941 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:11:21 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69321]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-03-28T18:11:21.941 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:11:21 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69321]: 2024-03-28T18:11:21.941 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:11:21 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69321]: -1> 2024-03-28T18:11:21.593+0000 7f50e0600700 -1 /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-2145-g3be9a5c9/rpm/el8/BUILD/ceph-19.0.0-2145-g3be9a5c9/src/mds/Locker.cc: In function 'void Locker::set_xlocks_done(MutationImpl*, bool)' thread 7f50e0600700 time 2024-03-28T18:11:21.592061+0000 2024-03-28T18:11:21.941 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:11:21 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69321]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-2145-g3be9a5c9/rpm/el8/BUILD/ceph-19.0.0-2145-g3be9a5c9/src/mds/Locker.cc: 626: FAILED ceph_assert(obj->is_auth()) 2024-03-28T18:11:21.941 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:11:21 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69321]: 2024-03-28T18:11:21.941 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:11:21 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69321]: ceph version 19.0.0-2145-g3be9a5c9 (3be9a5c9bc793e11e2800a8c0c696e8b46742033) squid (dev) 2024-03-28T18:11:21.942 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:11:21 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69321]: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x135) [0x7f50ebfd64a1] 2024-03-28T18:11:21.942 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:11:21 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69321]: 2: /usr/lib64/ceph/libceph-common.so.2(+0x2d2667) [0x7f50ebfd6667] 2024-03-28T18:11:21.942 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:11:21 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69321]: 3: (Locker::set_xlocks_done(MutationImpl*, bool)+0x493) [0x55a0a4ac5223] 2024-03-28T18:11:21.943 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:11:21 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69321]: 4: (Server::early_reply(boost::intrusive_ptr const&, CInode*, CDentry*)+0x3e6) [0x55a0a493f5d6] 2024-03-28T18:11:21.943 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:11:21 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69321]: 5: (Server::journal_and_reply(boost::intrusive_ptr const&, CInode*, CDentry*, LogEvent*, MDSLogContextBase*)+0xc1) [0x55a0a493fda1] 2024-03-28T18:11:21.943 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:11:21 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69321]: 6: (Server::handle_client_mkdir(boost::intrusive_ptr const&)+0x5ba) [0x55a0a49756aa] 2024-03-28T18:11:21.943 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:11:21 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69321]: 7: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0x733) [0x55a0a4992233] 2024-03-28T18:11:21.943 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:11:21 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69321]: 8: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x55a0a4994197] 2024-03-28T18:11:21.943 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:11:21 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69321]: 9: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x55a0a489311b] 2024-03-28T18:11:21.943 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:11:21 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69321]: 10: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x55a0a48936bc] 2024-03-28T18:11:21.943 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:11:21 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69321]: 11: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bf) [0x55a0a487d1ff] 2024-03-28T18:11:21.943 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:11:21 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69321]: 12: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x478) [0x7f50ec268488] 2024-03-28T18:11:21.943 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:11:21 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69321]: 13: (DispatchQueue::entry()+0x51f) [0x7f50ec26560f] 2024-03-28T18:11:21.943 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:11:21 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69321]: 14: (DispatchQueue::DispatchThread::entry()+0x11) [0x7f50ec338e81] 2024-03-28T18:11:21.944 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:11:21 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69321]: 15: /lib64/libpthread.so.0(+0x81ca) [0x7f50ead531ca] 2024-03-28T18:11:21.944 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:11:21 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69321]: 16: clone() 2024-03-28T18:11:21.944 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:11:21 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69321]: 2024-03-28T18:11:21.944 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:11:21 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69321]: 0> 2024-03-28T18:11:21.595+0000 7f50e0600700 -1 *** Caught signal (Aborted) ** 2024-03-28T18:11:21.944 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:11:21 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69321]: in thread 7f50e0600700 thread_name:ms_dispatch 2024-03-28T18:11:21.944 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:11:21 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69321]: 2024-03-28T18:11:21.944 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:11:21 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69321]: ceph version 19.0.0-2145-g3be9a5c9 (3be9a5c9bc793e11e2800a8c0c696e8b46742033) squid (dev) 2024-03-28T18:11:21.944 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:11:21 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69321]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7f50ead5dd20] 2024-03-28T18:11:21.944 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:11:21 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69321]: 2: gsignal() 2024-03-28T18:11:21.944 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:11:21 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69321]: 3: abort() 2024-03-28T18:11:21.944 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:11:21 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69321]: 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x18f) [0x7f50ebfd64fb] 2024-03-28T18:11:21.944 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:11:21 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69321]: 5: /usr/lib64/ceph/libceph-common.so.2(+0x2d2667) [0x7f50ebfd6667] 2024-03-28T18:11:21.944 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:11:21 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69321]: 6: (Locker::set_xlocks_done(MutationImpl*, bool)+0x493) [0x55a0a4ac5223] 2024-03-28T18:11:21.945 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:11:21 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69321]: 7: (Server::early_reply(boost::intrusive_ptr const&, CInode*, CDentry*)+0x3e6) [0x55a0a493f5d6] 2024-03-28T18:11:21.945 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:11:21 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69321]: 8: (Server::journal_and_reply(boost::intrusive_ptr const&, CInode*, CDentry*, LogEvent*, MDSLogContextBase*)+0xc1) [0x55a0a493fda1] 2024-03-28T18:11:21.945 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:11:21 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69321]: 9: (Server::handle_client_mkdir(boost::intrusive_ptr const&)+0x5ba) [0x55a0a49756aa] 2024-03-28T18:11:21.945 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:11:21 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69321]: 10: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0x733) [0x55a0a4992233] 2024-03-28T18:11:21.945 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:11:21 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69321]: 11: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x55a0a4994197] 2024-03-28T18:11:21.945 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:11:21 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69321]: 12: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x55a0a489311b] 2024-03-28T18:11:21.945 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:11:21 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69321]: 13: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x55a0a48936bc] 2024-03-28T18:11:21.945 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:11:21 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69321]: 14: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bf) [0x55a0a487d1ff] 2024-03-28T18:11:21.945 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:11:21 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69321]: 15: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x478) [0x7f50ec268488] 2024-03-28T18:11:21.945 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:11:21 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69321]: 16: (DispatchQueue::entry()+0x51f) [0x7f50ec26560f] 2024-03-28T18:11:21.945 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:11:21 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69321]: 17: (DispatchQueue::DispatchThread::entry()+0x11) [0x7f50ec338e81] 2024-03-28T18:11:21.945 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:11:21 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69321]: 18: /lib64/libpthread.so.0(+0x81ca) [0x7f50ead531ca] 2024-03-28T18:11:21.945 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:11:21 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69321]: 19: clone() 2024-03-28T18:11:21.946 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:11:21 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69321]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-03-28T18:11:21.946 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:11:21 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69321]: 2024-03-28T18:11:21.946 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:11:21 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69321]: -1> 2024-03-28T18:11:21.593+0000 7f50e0600700 -1 /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-2145-g3be9a5c9/rpm/el8/BUILD/ceph-19.0.0-2145-g3be9a5c9/src/mds/Locker.cc: In function 'void Locker::set_xlocks_done(MutationImpl*, bool)' thread 7f50e0600700 time 2024-03-28T18:11:21.592061+0000 2024-03-28T18:11:21.946 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:11:21 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69321]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-2145-g3be9a5c9/rpm/el8/BUILD/ceph-19.0.0-2145-g3be9a5c9/src/mds/Locker.cc: 626: FAILED ceph_assert(obj->is_auth()) 2024-03-28T18:11:21.946 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:11:21 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69321]: 2024-03-28T18:11:21.946 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:11:21 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69321]: ceph version 19.0.0-2145-g3be9a5c9 (3be9a5c9bc793e11e2800a8c0c696e8b46742033) squid (dev) 2024-03-28T18:11:21.946 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:11:21 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69321]: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x135) [0x7f50ebfd64a1] 2024-03-28T18:11:21.946 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:11:21 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69321]: 2: /usr/lib64/ceph/libceph-common.so.2(+0x2d2667) [0x7f50ebfd6667] 2024-03-28T18:11:21.946 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:11:21 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69321]: 3: (Locker::set_xlocks_done(MutationImpl*, bool)+0x493) [0x55a0a4ac5223] 2024-03-28T18:11:21.946 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:11:21 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69321]: 4: (Server::early_reply(boost::intrusive_ptr const&, CInode*, CDentry*)+0x3e6) [0x55a0a493f5d6] 2024-03-28T18:11:21.946 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:11:21 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69321]: 5: (Server::journal_and_reply(boost::intrusive_ptr const&, CInode*, CDentry*, LogEvent*, MDSLogContextBase*)+0xc1) [0x55a0a493fda1] 2024-03-28T18:11:21.947 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:11:21 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69321]: 6: (Server::handle_client_mkdir(boost::intrusive_ptr const&)+0x5ba) [0x55a0a49756aa] 2024-03-28T18:11:21.947 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:11:21 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69321]: 7: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0x733) [0x55a0a4992233] 2024-03-28T18:11:21.947 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:11:21 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69321]: 8: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x55a0a4994197] 2024-03-28T18:11:21.947 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:11:21 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69321]: 9: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x55a0a489311b] 2024-03-28T18:11:21.947 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:11:21 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69321]: 10: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x55a0a48936bc] 2024-03-28T18:11:21.947 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:11:21 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69321]: 11: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bf) [0x55a0a487d1ff] 2024-03-28T18:11:21.947 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:11:21 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69321]: 12: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x478) [0x7f50ec268488] 2024-03-28T18:11:21.947 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:11:21 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69321]: 13: (DispatchQueue::entry()+0x51f) [0x7f50ec26560f] 2024-03-28T18:11:21.947 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:11:21 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69321]: 14: (DispatchQueue::DispatchThread::entry()+0x11) [0x7f50ec338e81] 2024-03-28T18:11:21.947 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:11:21 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69321]: 15: /lib64/libpthread.so.0(+0x81ca) [0x7f50ead531ca] 2024-03-28T18:11:21.947 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:11:21 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69321]: 16: clone() 2024-03-28T18:11:21.947 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:11:21 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69321]: 2024-03-28T18:11:21.948 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:11:21 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69321]: 0> 2024-03-28T18:11:21.595+0000 7f50e0600700 -1 *** Caught signal (Aborted) ** 2024-03-28T18:11:21.948 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:11:21 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69321]: in thread 7f50e0600700 thread_name:ms_dispatch 2024-03-28T18:11:21.948 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:11:21 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69321]: 2024-03-28T18:11:21.948 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:11:21 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69321]: ceph version 19.0.0-2145-g3be9a5c9 (3be9a5c9bc793e11e2800a8c0c696e8b46742033) squid (dev) 2024-03-28T18:11:21.948 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:11:21 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69321]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7f50ead5dd20] 2024-03-28T18:11:21.948 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:11:21 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69321]: 2: gsignal() 2024-03-28T18:11:21.948 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:11:21 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69321]: 3: abort() 2024-03-28T18:11:21.948 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:11:21 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69321]: 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x18f) [0x7f50ebfd64fb] 2024-03-28T18:11:21.948 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:11:21 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69321]: 5: /usr/lib64/ceph/libceph-common.so.2(+0x2d2667) [0x7f50ebfd6667] 2024-03-28T18:11:21.948 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:11:21 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69321]: 6: (Locker::set_xlocks_done(MutationImpl*, bool)+0x493) [0x55a0a4ac5223] 2024-03-28T18:11:21.948 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:11:21 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69321]: 7: (Server::early_reply(boost::intrusive_ptr const&, CInode*, CDentry*)+0x3e6) [0x55a0a493f5d6] 2024-03-28T18:11:21.948 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:11:21 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69321]: 8: (Server::journal_and_reply(boost::intrusive_ptr const&, CInode*, CDentry*, LogEvent*, MDSLogContextBase*)+0xc1) [0x55a0a493fda1] 2024-03-28T18:11:21.949 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:11:21 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69321]: 9: (Server::handle_client_mkdir(boost::intrusive_ptr const&)+0x5ba) [0x55a0a49756aa] 2024-03-28T18:11:21.949 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:11:21 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69321]: 10: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0x733) [0x55a0a4992233] 2024-03-28T18:11:21.949 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:11:21 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69321]: 11: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x55a0a4994197] 2024-03-28T18:11:21.949 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:11:21 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69321]: 12: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x55a0a489311b] 2024-03-28T18:11:21.949 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:11:21 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69321]: 13: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x55a0a48936bc] 2024-03-28T18:11:21.949 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:11:21 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69321]: 14: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bf) [0x55a0a487d1ff] 2024-03-28T18:11:21.949 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:11:21 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69321]: 15: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x478) [0x7f50ec268488] 2024-03-28T18:11:21.949 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:11:21 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69321]: 16: (DispatchQueue::entry()+0x51f) [0x7f50ec26560f] 2024-03-28T18:11:21.949 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:11:21 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69321]: 17: (DispatchQueue::DispatchThread::entry()+0x11) [0x7f50ec338e81] 2024-03-28T18:11:21.949 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:11:21 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69321]: 18: /lib64/libpthread.so.0(+0x81ca) [0x7f50ead531ca] 2024-03-28T18:11:21.949 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:11:21 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69321]: 19: clone() 2024-03-28T18:11:21.949 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:11:21 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69321]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-03-28T18:11:21.949 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:11:21 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69321]: 2024-03-28T18:11:21.950 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:11:21 smithi079 podman[69360]: 2024-03-28 18:11:21.719067175 +0000 UTC m=+0.043293927 container died 6dbaa33fd278e5ff68b86b0ad2174d690a95fbcb13c514fd0e9c5445b9fada74 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.build-date=20240319, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5) 2024-03-28T18:11:22.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:11:22 smithi022 ceph-mon[31985]: pgmap v121: 97 pgs: 97 active+clean; 117 MiB data, 800 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 307 B/s wr, 10 op/s 2024-03-28T18:11:22.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:11:22 smithi022 ceph-mon[31985]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-03-28T18:11:22.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:11:22 smithi022 ceph-mon[31985]: Cluster is now healthy 2024-03-28T18:11:22.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:11:22 smithi022 ceph-mon[31985]: mds.? [v2:172.21.15.79:6836/2541118119,v1:172.21.15.79:6837/2541118119] up:active 2024-03-28T18:11:22.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:11:22 smithi022 ceph-mon[31985]: fsmap cephfs:5 4 up:standby-replay 3 up:standby 5 up:active 2024-03-28T18:11:22.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:11:22 smithi022 ceph-mon[31985]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-28T18:11:22.907 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:11:22 smithi079 podman[69360]: 2024-03-28 18:11:22.470312892 +0000 UTC m=+0.794539660 container remove 6dbaa33fd278e5ff68b86b0ad2174d690a95fbcb13c514fd0e9c5445b9fada74 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, ceph=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240319, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, maintainer=Guillaume Abrioux ) 2024-03-28T18:11:22.908 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:11:22 smithi079 systemd[1]: ceph-fd220748-ed2b-11ee-95d0-87774f69a715@mds.h.service: Main process exited, code=exited, status=134/n/a 2024-03-28T18:11:22.908 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:11:22 smithi079 ceph-mon[35312]: pgmap v121: 97 pgs: 97 active+clean; 117 MiB data, 800 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 307 B/s wr, 10 op/s 2024-03-28T18:11:22.908 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:11:22 smithi079 ceph-mon[35312]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-03-28T18:11:22.908 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:11:22 smithi079 ceph-mon[35312]: Cluster is now healthy 2024-03-28T18:11:22.908 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:11:22 smithi079 ceph-mon[35312]: mds.? [v2:172.21.15.79:6836/2541118119,v1:172.21.15.79:6837/2541118119] up:active 2024-03-28T18:11:22.908 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:11:22 smithi079 ceph-mon[35312]: fsmap cephfs:5 4 up:standby-replay 3 up:standby 5 up:active 2024-03-28T18:11:22.908 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:11:22 smithi079 ceph-mon[35312]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-28T18:11:22.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:11:22 smithi134 ceph-mon[35037]: pgmap v121: 97 pgs: 97 active+clean; 117 MiB data, 800 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 307 B/s wr, 10 op/s 2024-03-28T18:11:22.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:11:22 smithi134 ceph-mon[35037]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-03-28T18:11:22.925 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:11:22 smithi134 ceph-mon[35037]: Cluster is now healthy 2024-03-28T18:11:22.925 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:11:22 smithi134 ceph-mon[35037]: mds.? [v2:172.21.15.79:6836/2541118119,v1:172.21.15.79:6837/2541118119] up:active 2024-03-28T18:11:22.925 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:11:22 smithi134 ceph-mon[35037]: fsmap cephfs:5 4 up:standby-replay 3 up:standby 5 up:active 2024-03-28T18:11:22.925 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:11:22 smithi134 ceph-mon[35037]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-28T18:11:23.182 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:11:22 smithi079 systemd[1]: ceph-fd220748-ed2b-11ee-95d0-87774f69a715@mds.h.service: Failed with result 'exit-code'. 2024-03-28T18:11:23.182 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:11:22 smithi079 systemd[1]: ceph-fd220748-ed2b-11ee-95d0-87774f69a715@mds.h.service: Consumed 1.673s CPU time. 2024-03-28T18:11:24.433 INFO:journalctl@ceph.mds.b.smithi079.stdout:Mar 28 18:11:24 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-b[65245]: 2024-03-28T18:11:24.081+0000 7f845f400700 -1 mds.pinger is_rank_lagging: rank=1 was never sent ping request. 2024-03-28T18:11:24.925 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:11:24 smithi134 ceph-mon[35037]: pgmap v122: 97 pgs: 97 active+clean; 117 MiB data, 800 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 307 B/s wr, 10 op/s 2024-03-28T18:11:24.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:11:24 smithi079 ceph-mon[35312]: pgmap v122: 97 pgs: 97 active+clean; 117 MiB data, 800 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 307 B/s wr, 10 op/s 2024-03-28T18:11:25.031 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:11:24 smithi022 ceph-mon[31985]: pgmap v122: 97 pgs: 97 active+clean; 117 MiB data, 800 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 307 B/s wr, 10 op/s 2024-03-28T18:11:26.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:11:26 smithi134 ceph-mon[35037]: pgmap v123: 97 pgs: 97 active+clean; 117 MiB data, 800 MiB used, 1.0 TiB / 1.0 TiB avail; 109 KiB/s rd, 5.2 KiB/s wr, 26 op/s 2024-03-28T18:11:26.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:11:26 smithi079 ceph-mon[35312]: pgmap v123: 97 pgs: 97 active+clean; 117 MiB data, 800 MiB used, 1.0 TiB / 1.0 TiB avail; 109 KiB/s rd, 5.2 KiB/s wr, 26 op/s 2024-03-28T18:11:27.032 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:11:26 smithi022 ceph-mon[31985]: pgmap v123: 97 pgs: 97 active+clean; 117 MiB data, 800 MiB used, 1.0 TiB / 1.0 TiB avail; 109 KiB/s rd, 5.2 KiB/s wr, 26 op/s 2024-03-28T18:11:28.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:11:28 smithi134 ceph-mon[35037]: pgmap v124: 97 pgs: 97 active+clean; 117 MiB data, 800 MiB used, 1.0 TiB / 1.0 TiB avail; 105 KiB/s rd, 5.0 KiB/s wr, 25 op/s 2024-03-28T18:11:28.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:11:28 smithi079 ceph-mon[35312]: pgmap v124: 97 pgs: 97 active+clean; 117 MiB data, 800 MiB used, 1.0 TiB / 1.0 TiB avail; 105 KiB/s rd, 5.0 KiB/s wr, 25 op/s 2024-03-28T18:11:29.033 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:11:28 smithi022 ceph-mon[31985]: pgmap v124: 97 pgs: 97 active+clean; 117 MiB data, 800 MiB used, 1.0 TiB / 1.0 TiB avail; 105 KiB/s rd, 5.0 KiB/s wr, 25 op/s 2024-03-28T18:11:30.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:11:30 smithi134 ceph-mon[35037]: pgmap v125: 97 pgs: 97 active+clean; 117 MiB data, 800 MiB used, 1.0 TiB / 1.0 TiB avail; 92 KiB/s rd, 4.3 KiB/s wr, 24 op/s 2024-03-28T18:11:30.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:11:30 smithi079 ceph-mon[35312]: pgmap v125: 97 pgs: 97 active+clean; 117 MiB data, 800 MiB used, 1.0 TiB / 1.0 TiB avail; 92 KiB/s rd, 4.3 KiB/s wr, 24 op/s 2024-03-28T18:11:31.031 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:11:30 smithi022 ceph-mon[31985]: pgmap v125: 97 pgs: 97 active+clean; 117 MiB data, 800 MiB used, 1.0 TiB / 1.0 TiB avail; 92 KiB/s rd, 4.3 KiB/s wr, 24 op/s 2024-03-28T18:11:32.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:11:32 smithi134 ceph-mon[35037]: pgmap v126: 97 pgs: 97 active+clean; 117 MiB data, 800 MiB used, 1.0 TiB / 1.0 TiB avail; 92 KiB/s rd, 4.2 KiB/s wr, 23 op/s 2024-03-28T18:11:32.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:11:32 smithi079 ceph-mon[35312]: pgmap v126: 97 pgs: 97 active+clean; 117 MiB data, 800 MiB used, 1.0 TiB / 1.0 TiB avail; 92 KiB/s rd, 4.2 KiB/s wr, 23 op/s 2024-03-28T18:11:33.031 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:11:32 smithi022 ceph-mon[31985]: pgmap v126: 97 pgs: 97 active+clean; 117 MiB data, 800 MiB used, 1.0 TiB / 1.0 TiB avail; 92 KiB/s rd, 4.2 KiB/s wr, 23 op/s 2024-03-28T18:11:33.339 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:11:32 smithi079 systemd[1]: ceph-fd220748-ed2b-11ee-95d0-87774f69a715@mds.h.service: Scheduled restart job, restart counter is at 3. 2024-03-28T18:11:33.339 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:11:32 smithi079 systemd[1]: Stopped Ceph mds.h for fd220748-ed2b-11ee-95d0-87774f69a715. 2024-03-28T18:11:33.339 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:11:32 smithi079 systemd[1]: ceph-fd220748-ed2b-11ee-95d0-87774f69a715@mds.h.service: Consumed 1.673s CPU time. 2024-03-28T18:11:33.339 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:11:32 smithi079 systemd[1]: Starting Ceph mds.h for fd220748-ed2b-11ee-95d0-87774f69a715... 2024-03-28T18:11:33.625 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:11:33 smithi079 podman[69446]: 2024-03-28 18:11:33.245311291 +0000 UTC m=+0.044130759 image pull c664afe56bb01db4f9afa9532c20598e201e664f2e9ba2be50db0d52674a26d2 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc 2024-03-28T18:11:33.625 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:11:33 smithi079 podman[69446]: 2024-03-28 18:11:33.369984525 +0000 UTC m=+0.168803997 container create aa05077e4c57ade8832dc601ab80f5d0cb4bfa5115afa7c28cb3f99ecaada10b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, ceph=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240319) 2024-03-28T18:11:33.934 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:11:33 smithi079 podman[69446]: 2024-03-28 18:11:33.625440314 +0000 UTC m=+0.424259863 container init aa05077e4c57ade8832dc601ab80f5d0cb4bfa5115afa7c28cb3f99ecaada10b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.build-date=20240319, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_CLEAN=True, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0) 2024-03-28T18:11:33.934 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:11:33 smithi079 podman[69446]: 2024-03-28 18:11:33.63638074 +0000 UTC m=+0.435200212 container start aa05077e4c57ade8832dc601ab80f5d0cb4bfa5115afa7c28cb3f99ecaada10b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h, GIT_BRANCH=HEAD, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, io.buildah.version=1.33.5, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240319, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c) 2024-03-28T18:11:33.934 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:11:33 smithi079 ceph-mds[69464]: -- 172.21.15.79:0/1331400071 <== mon.2 v2:172.21.15.79:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x563337abba40 con 0x563337a8e800 2024-03-28T18:11:33.934 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:11:33 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69460]: starting mds.h at 2024-03-28T18:11:33.934 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:11:33 smithi079 bash[69446]: aa05077e4c57ade8832dc601ab80f5d0cb4bfa5115afa7c28cb3f99ecaada10b 2024-03-28T18:11:33.934 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:11:33 smithi079 systemd[1]: Started Ceph mds.h for fd220748-ed2b-11ee-95d0-87774f69a715. 2024-03-28T18:11:34.925 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:11:34 smithi134 ceph-mon[35037]: pgmap v127: 97 pgs: 97 active+clean; 117 MiB data, 800 MiB used, 1.0 TiB / 1.0 TiB avail; 90 KiB/s rd, 4.2 KiB/s wr, 21 op/s 2024-03-28T18:11:34.925 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:11:34 smithi134 ceph-mon[35037]: daemon mds.h restarted 2024-03-28T18:11:34.925 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:11:34 smithi134 ceph-mon[35037]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-03-28T18:11:34.925 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:11:34 smithi134 ceph-mon[35037]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-03-28T18:11:34.925 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:11:34 smithi134 ceph-mon[35037]: osdmap e98: 12 total, 12 up, 12 in 2024-03-28T18:11:34.925 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:11:34 smithi134 ceph-mon[35037]: mds.? [v2:172.21.15.79:6836/3952268652,v1:172.21.15.79:6837/3952268652] up:boot 2024-03-28T18:11:34.925 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:11:34 smithi134 ceph-mon[35037]: Standby daemon mds.f assigned to filesystem cephfs as rank 1 2024-03-28T18:11:34.925 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:11:34 smithi134 ceph-mon[35037]: fsmap cephfs:4/5 {0=b=up:active,2=e=up:active,3=k=up:active,4=i=up:active} 5 up:standby-replay 3 up:standby, 1 failed 2024-03-28T18:11:34.926 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:11:34 smithi134 ceph-mon[35037]: fsmap cephfs:5/5 4 up:standby-replay 3 up:standby 1 up:replay 4 up:active 2024-03-28T18:11:34.933 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:11:34 smithi079 ceph-mon[35312]: pgmap v127: 97 pgs: 97 active+clean; 117 MiB data, 800 MiB used, 1.0 TiB / 1.0 TiB avail; 90 KiB/s rd, 4.2 KiB/s wr, 21 op/s 2024-03-28T18:11:34.933 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:11:34 smithi079 ceph-mon[35312]: daemon mds.h restarted 2024-03-28T18:11:34.933 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:11:34 smithi079 ceph-mon[35312]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-03-28T18:11:34.933 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:11:34 smithi079 ceph-mon[35312]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-03-28T18:11:34.933 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:11:34 smithi079 ceph-mon[35312]: osdmap e98: 12 total, 12 up, 12 in 2024-03-28T18:11:34.933 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:11:34 smithi079 ceph-mon[35312]: mds.? [v2:172.21.15.79:6836/3952268652,v1:172.21.15.79:6837/3952268652] up:boot 2024-03-28T18:11:34.933 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:11:34 smithi079 ceph-mon[35312]: Standby daemon mds.f assigned to filesystem cephfs as rank 1 2024-03-28T18:11:34.933 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:11:34 smithi079 ceph-mon[35312]: fsmap cephfs:4/5 {0=b=up:active,2=e=up:active,3=k=up:active,4=i=up:active} 5 up:standby-replay 3 up:standby, 1 failed 2024-03-28T18:11:34.933 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:11:34 smithi079 ceph-mon[35312]: fsmap cephfs:5/5 4 up:standby-replay 3 up:standby 1 up:replay 4 up:active 2024-03-28T18:11:35.032 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:11:34 smithi022 ceph-mon[31985]: pgmap v127: 97 pgs: 97 active+clean; 117 MiB data, 800 MiB used, 1.0 TiB / 1.0 TiB avail; 90 KiB/s rd, 4.2 KiB/s wr, 21 op/s 2024-03-28T18:11:35.032 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:11:34 smithi022 ceph-mon[31985]: daemon mds.h restarted 2024-03-28T18:11:35.032 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:11:34 smithi022 ceph-mon[31985]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-03-28T18:11:35.032 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:11:34 smithi022 ceph-mon[31985]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-03-28T18:11:35.032 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:11:34 smithi022 ceph-mon[31985]: osdmap e98: 12 total, 12 up, 12 in 2024-03-28T18:11:35.032 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:11:34 smithi022 ceph-mon[31985]: mds.? [v2:172.21.15.79:6836/3952268652,v1:172.21.15.79:6837/3952268652] up:boot 2024-03-28T18:11:35.032 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:11:34 smithi022 ceph-mon[31985]: Standby daemon mds.f assigned to filesystem cephfs as rank 1 2024-03-28T18:11:35.032 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:11:34 smithi022 ceph-mon[31985]: fsmap cephfs:4/5 {0=b=up:active,2=e=up:active,3=k=up:active,4=i=up:active} 5 up:standby-replay 3 up:standby, 1 failed 2024-03-28T18:11:35.033 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:11:34 smithi022 ceph-mon[31985]: fsmap cephfs:5/5 4 up:standby-replay 3 up:standby 1 up:replay 4 up:active 2024-03-28T18:11:36.925 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:11:36 smithi134 ceph-mon[35037]: pgmap v129: 97 pgs: 97 active+clean; 117 MiB data, 800 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:11:36.925 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:11:36 smithi134 ceph-mon[35037]: mds.? [v2:172.21.15.134:6834/568477027,v1:172.21.15.134:6835/568477027] up:resolve 2024-03-28T18:11:36.925 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:11:36 smithi134 ceph-mon[35037]: fsmap cephfs:5/5 4 up:standby-replay 3 up:standby 1 up:resolve 4 up:active 2024-03-28T18:11:36.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:11:36 smithi079 ceph-mon[35312]: pgmap v129: 97 pgs: 97 active+clean; 117 MiB data, 800 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:11:36.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:11:36 smithi079 ceph-mon[35312]: mds.? [v2:172.21.15.134:6834/568477027,v1:172.21.15.134:6835/568477027] up:resolve 2024-03-28T18:11:36.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:11:36 smithi079 ceph-mon[35312]: fsmap cephfs:5/5 4 up:standby-replay 3 up:standby 1 up:resolve 4 up:active 2024-03-28T18:11:37.032 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:11:36 smithi022 ceph-mon[31985]: pgmap v129: 97 pgs: 97 active+clean; 117 MiB data, 800 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:11:37.032 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:11:36 smithi022 ceph-mon[31985]: mds.? [v2:172.21.15.134:6834/568477027,v1:172.21.15.134:6835/568477027] up:resolve 2024-03-28T18:11:37.032 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:11:36 smithi022 ceph-mon[31985]: fsmap cephfs:5/5 4 up:standby-replay 3 up:standby 1 up:resolve 4 up:active 2024-03-28T18:11:37.910 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:11:37 smithi134 ceph-mon[35037]: reconnect by client.35222 172.21.15.22:0/3729116781 after 0.00299993 2024-03-28T18:11:37.910 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:11:37 smithi134 ceph-mon[35037]: mds.? [v2:172.21.15.134:6834/568477027,v1:172.21.15.134:6835/568477027] up:reconnect 2024-03-28T18:11:37.910 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:11:37 smithi134 ceph-mon[35037]: fsmap cephfs:5/5 4 up:standby-replay 3 up:standby 1 up:reconnect 4 up:active 2024-03-28T18:11:37.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:11:37 smithi079 ceph-mon[35312]: reconnect by client.35222 172.21.15.22:0/3729116781 after 0.00299993 2024-03-28T18:11:37.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:11:37 smithi079 ceph-mon[35312]: mds.? [v2:172.21.15.134:6834/568477027,v1:172.21.15.134:6835/568477027] up:reconnect 2024-03-28T18:11:37.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:11:37 smithi079 ceph-mon[35312]: fsmap cephfs:5/5 4 up:standby-replay 3 up:standby 1 up:reconnect 4 up:active 2024-03-28T18:11:38.032 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:11:37 smithi022 ceph-mon[31985]: reconnect by client.35222 172.21.15.22:0/3729116781 after 0.00299993 2024-03-28T18:11:38.032 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:11:37 smithi022 ceph-mon[31985]: mds.? [v2:172.21.15.134:6834/568477027,v1:172.21.15.134:6835/568477027] up:reconnect 2024-03-28T18:11:38.032 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:11:37 smithi022 ceph-mon[31985]: fsmap cephfs:5/5 4 up:standby-replay 3 up:standby 1 up:reconnect 4 up:active 2024-03-28T18:11:38.175 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:11:37 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[70064]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-2145-g3be9a5c9/rpm/el8/BUILD/ceph-19.0.0-2145-g3be9a5c9/src/mds/Locker.cc: In function 'void Locker::set_xlocks_done(MutationImpl*, bool)' thread 7f3dff800700 time 2024-03-28T18:11:37.910813+0000 2024-03-28T18:11:38.176 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:11:37 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[70064]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-2145-g3be9a5c9/rpm/el8/BUILD/ceph-19.0.0-2145-g3be9a5c9/src/mds/Locker.cc: 626: FAILED ceph_assert(obj->is_auth()) 2024-03-28T18:11:38.176 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:11:37 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[70064]: ceph version 19.0.0-2145-g3be9a5c9 (3be9a5c9bc793e11e2800a8c0c696e8b46742033) squid (dev) 2024-03-28T18:11:38.176 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:11:37 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[70064]: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x135) [0x7f3e0b1c24a1] 2024-03-28T18:11:38.176 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:11:37 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[70064]: 2: /usr/lib64/ceph/libceph-common.so.2(+0x2d2667) [0x7f3e0b1c2667] 2024-03-28T18:11:38.176 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:11:37 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[70064]: 3: (Locker::set_xlocks_done(MutationImpl*, bool)+0x493) [0x55f4b3fc8223] 2024-03-28T18:11:38.176 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:11:37 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[70064]: 4: (Server::early_reply(boost::intrusive_ptr const&, CInode*, CDentry*)+0x3e6) [0x55f4b3e425d6] 2024-03-28T18:11:38.176 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:11:37 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[70064]: 5: (Server::journal_and_reply(boost::intrusive_ptr const&, CInode*, CDentry*, LogEvent*, MDSLogContextBase*)+0xc1) [0x55f4b3e42da1] 2024-03-28T18:11:38.176 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:11:37 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[70064]: 6: (Server::handle_client_mkdir(boost::intrusive_ptr const&)+0x5ba) [0x55f4b3e786aa] 2024-03-28T18:11:38.176 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:11:37 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[70064]: 7: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0x733) [0x55f4b3e95233] 2024-03-28T18:11:38.176 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:11:37 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[70064]: 8: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x55f4b3e97197] 2024-03-28T18:11:38.176 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:11:37 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[70064]: 9: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x55f4b3d9611b] 2024-03-28T18:11:38.177 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:11:37 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[70064]: 10: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x55f4b3d966bc] 2024-03-28T18:11:38.177 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:11:37 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[70064]: 11: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bf) [0x55f4b3d801ff] 2024-03-28T18:11:38.177 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:11:37 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[70064]: 12: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x478) [0x7f3e0b454488] 2024-03-28T18:11:38.177 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:11:37 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[70064]: 13: (DispatchQueue::entry()+0x51f) [0x7f3e0b45160f] 2024-03-28T18:11:38.177 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:11:37 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[70064]: 14: (DispatchQueue::DispatchThread::entry()+0x11) [0x7f3e0b524e81] 2024-03-28T18:11:38.177 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:11:37 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[70064]: 15: /lib64/libpthread.so.0(+0x81ca) [0x7f3e09f3f1ca] 2024-03-28T18:11:38.177 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:11:37 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[70064]: 16: clone() 2024-03-28T18:11:38.177 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:11:37 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[70064]: 2024-03-28T18:11:37.914+0000 7f3dff800700 -1 /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-2145-g3be9a5c9/rpm/el8/BUILD/ceph-19.0.0-2145-g3be9a5c9/src/mds/Locker.cc: In function 'void Locker::set_xlocks_done(MutationImpl*, bool)' thread 7f3dff800700 time 2024-03-28T18:11:37.910813+0000 2024-03-28T18:11:38.177 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:11:37 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[70064]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-2145-g3be9a5c9/rpm/el8/BUILD/ceph-19.0.0-2145-g3be9a5c9/src/mds/Locker.cc: 626: FAILED ceph_assert(obj->is_auth()) 2024-03-28T18:11:38.177 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:11:37 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[70064]: 2024-03-28T18:11:38.177 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:11:37 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[70064]: ceph version 19.0.0-2145-g3be9a5c9 (3be9a5c9bc793e11e2800a8c0c696e8b46742033) squid (dev) 2024-03-28T18:11:38.178 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:11:37 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[70064]: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x135) [0x7f3e0b1c24a1] 2024-03-28T18:11:38.178 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:11:37 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[70064]: 2: /usr/lib64/ceph/libceph-common.so.2(+0x2d2667) [0x7f3e0b1c2667] 2024-03-28T18:11:38.178 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:11:37 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[70064]: 3: (Locker::set_xlocks_done(MutationImpl*, bool)+0x493) [0x55f4b3fc8223] 2024-03-28T18:11:38.178 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:11:37 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[70064]: 4: (Server::early_reply(boost::intrusive_ptr const&, CInode*, CDentry*)+0x3e6) [0x55f4b3e425d6] 2024-03-28T18:11:38.178 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:11:37 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[70064]: 5: (Server::journal_and_reply(boost::intrusive_ptr const&, CInode*, CDentry*, LogEvent*, MDSLogContextBase*)+0xc1) [0x55f4b3e42da1] 2024-03-28T18:11:38.178 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:11:37 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[70064]: 6: (Server::handle_client_mkdir(boost::intrusive_ptr const&)+0x5ba) [0x55f4b3e786aa] 2024-03-28T18:11:38.178 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:11:37 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[70064]: 7: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0x733) [0x55f4b3e95233] 2024-03-28T18:11:38.178 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:11:37 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[70064]: 8: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x55f4b3e97197] 2024-03-28T18:11:38.178 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:11:37 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[70064]: 9: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x55f4b3d9611b] 2024-03-28T18:11:38.178 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:11:37 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[70064]: 10: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x55f4b3d966bc] 2024-03-28T18:11:38.178 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:11:37 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[70064]: 11: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bf) [0x55f4b3d801ff] 2024-03-28T18:11:38.178 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:11:37 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[70064]: 12: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x478) [0x7f3e0b454488] 2024-03-28T18:11:38.178 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:11:37 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[70064]: 13: (DispatchQueue::entry()+0x51f) [0x7f3e0b45160f] 2024-03-28T18:11:38.178 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:11:37 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[70064]: 14: (DispatchQueue::DispatchThread::entry()+0x11) [0x7f3e0b524e81] 2024-03-28T18:11:38.179 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:11:37 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[70064]: 15: /lib64/libpthread.so.0(+0x81ca) [0x7f3e09f3f1ca] 2024-03-28T18:11:38.179 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:11:37 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[70064]: 16: clone() 2024-03-28T18:11:38.179 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:11:37 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[70064]: 2024-03-28T18:11:38.179 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:11:37 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[70064]: *** Caught signal (Aborted) ** 2024-03-28T18:11:38.179 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:11:37 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[70064]: in thread 7f3dff800700 thread_name:ms_dispatch 2024-03-28T18:11:38.179 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:11:37 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[70064]: ceph version 19.0.0-2145-g3be9a5c9 (3be9a5c9bc793e11e2800a8c0c696e8b46742033) squid (dev) 2024-03-28T18:11:38.179 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:11:37 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[70064]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7f3e09f49d20] 2024-03-28T18:11:38.179 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:11:37 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[70064]: 2: gsignal() 2024-03-28T18:11:38.179 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:11:37 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[70064]: 3: abort() 2024-03-28T18:11:38.179 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:11:37 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[70064]: 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x18f) [0x7f3e0b1c24fb] 2024-03-28T18:11:38.179 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:11:37 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[70064]: 5: /usr/lib64/ceph/libceph-common.so.2(+0x2d2667) [0x7f3e0b1c2667] 2024-03-28T18:11:38.180 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:11:37 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[70064]: 6: (Locker::set_xlocks_done(MutationImpl*, bool)+0x493) [0x55f4b3fc8223] 2024-03-28T18:11:38.180 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:11:37 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[70064]: 7: (Server::early_reply(boost::intrusive_ptr const&, CInode*, CDentry*)+0x3e6) [0x55f4b3e425d6] 2024-03-28T18:11:38.180 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:11:37 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[70064]: 8: (Server::journal_and_reply(boost::intrusive_ptr const&, CInode*, CDentry*, LogEvent*, MDSLogContextBase*)+0xc1) [0x55f4b3e42da1] 2024-03-28T18:11:38.180 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:11:37 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[70064]: 9: (Server::handle_client_mkdir(boost::intrusive_ptr const&)+0x5ba) [0x55f4b3e786aa] 2024-03-28T18:11:38.180 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:11:37 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[70064]: 10: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0x733) [0x55f4b3e95233] 2024-03-28T18:11:38.180 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:11:37 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[70064]: 11: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x55f4b3e97197] 2024-03-28T18:11:38.180 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:11:37 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[70064]: 12: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x55f4b3d9611b] 2024-03-28T18:11:38.180 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:11:37 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[70064]: 13: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x55f4b3d966bc] 2024-03-28T18:11:38.180 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:11:37 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[70064]: 14: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bf) [0x55f4b3d801ff] 2024-03-28T18:11:38.180 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:11:37 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[70064]: 15: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x478) [0x7f3e0b454488] 2024-03-28T18:11:38.180 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:11:37 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[70064]: 16: (DispatchQueue::entry()+0x51f) [0x7f3e0b45160f] 2024-03-28T18:11:38.180 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:11:37 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[70064]: 17: (DispatchQueue::DispatchThread::entry()+0x11) [0x7f3e0b524e81] 2024-03-28T18:11:38.180 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:11:37 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[70064]: 18: /lib64/libpthread.so.0(+0x81ca) [0x7f3e09f3f1ca] 2024-03-28T18:11:38.180 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:11:37 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[70064]: 19: clone() 2024-03-28T18:11:38.180 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:11:37 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[70064]: 2024-03-28T18:11:37.918+0000 7f3dff800700 -1 *** Caught signal (Aborted) ** 2024-03-28T18:11:38.181 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:11:37 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[70064]: in thread 7f3dff800700 thread_name:ms_dispatch 2024-03-28T18:11:38.181 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:11:37 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[70064]: 2024-03-28T18:11:38.181 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:11:37 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[70064]: ceph version 19.0.0-2145-g3be9a5c9 (3be9a5c9bc793e11e2800a8c0c696e8b46742033) squid (dev) 2024-03-28T18:11:38.181 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:11:37 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[70064]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7f3e09f49d20] 2024-03-28T18:11:38.181 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:11:37 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[70064]: 2: gsignal() 2024-03-28T18:11:38.181 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:11:37 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[70064]: 3: abort() 2024-03-28T18:11:38.181 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:11:37 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[70064]: 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x18f) [0x7f3e0b1c24fb] 2024-03-28T18:11:38.181 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:11:37 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[70064]: 5: /usr/lib64/ceph/libceph-common.so.2(+0x2d2667) [0x7f3e0b1c2667] 2024-03-28T18:11:38.181 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:11:37 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[70064]: 6: (Locker::set_xlocks_done(MutationImpl*, bool)+0x493) [0x55f4b3fc8223] 2024-03-28T18:11:38.181 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:11:37 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[70064]: 7: (Server::early_reply(boost::intrusive_ptr const&, CInode*, CDentry*)+0x3e6) [0x55f4b3e425d6] 2024-03-28T18:11:38.181 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:11:37 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[70064]: 8: (Server::journal_and_reply(boost::intrusive_ptr const&, CInode*, CDentry*, LogEvent*, MDSLogContextBase*)+0xc1) [0x55f4b3e42da1] 2024-03-28T18:11:38.182 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:11:37 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[70064]: 9: (Server::handle_client_mkdir(boost::intrusive_ptr const&)+0x5ba) [0x55f4b3e786aa] 2024-03-28T18:11:38.182 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:11:37 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[70064]: 10: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0x733) [0x55f4b3e95233] 2024-03-28T18:11:38.182 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:11:37 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[70064]: 11: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x55f4b3e97197] 2024-03-28T18:11:38.182 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:11:37 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[70064]: 12: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x55f4b3d9611b] 2024-03-28T18:11:38.182 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:11:37 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[70064]: 13: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x55f4b3d966bc] 2024-03-28T18:11:38.182 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:11:37 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[70064]: 14: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bf) [0x55f4b3d801ff] 2024-03-28T18:11:38.182 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:11:37 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[70064]: 15: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x478) [0x7f3e0b454488] 2024-03-28T18:11:38.182 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:11:37 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[70064]: 16: (DispatchQueue::entry()+0x51f) [0x7f3e0b45160f] 2024-03-28T18:11:38.182 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:11:37 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[70064]: 17: (DispatchQueue::DispatchThread::entry()+0x11) [0x7f3e0b524e81] 2024-03-28T18:11:38.182 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:11:37 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[70064]: 18: /lib64/libpthread.so.0(+0x81ca) [0x7f3e09f3f1ca] 2024-03-28T18:11:38.182 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:11:37 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[70064]: 19: clone() 2024-03-28T18:11:38.182 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:11:37 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[70064]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-03-28T18:11:38.182 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:11:37 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[70064]: 2024-03-28T18:11:38.182 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:11:37 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[70064]: -1> 2024-03-28T18:11:37.914+0000 7f3dff800700 -1 /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-2145-g3be9a5c9/rpm/el8/BUILD/ceph-19.0.0-2145-g3be9a5c9/src/mds/Locker.cc: In function 'void Locker::set_xlocks_done(MutationImpl*, bool)' thread 7f3dff800700 time 2024-03-28T18:11:37.910813+0000 2024-03-28T18:11:38.183 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:11:37 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[70064]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-2145-g3be9a5c9/rpm/el8/BUILD/ceph-19.0.0-2145-g3be9a5c9/src/mds/Locker.cc: 626: FAILED ceph_assert(obj->is_auth()) 2024-03-28T18:11:38.183 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:11:37 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[70064]: 2024-03-28T18:11:38.183 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:11:37 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[70064]: ceph version 19.0.0-2145-g3be9a5c9 (3be9a5c9bc793e11e2800a8c0c696e8b46742033) squid (dev) 2024-03-28T18:11:38.183 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:11:37 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[70064]: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x135) [0x7f3e0b1c24a1] 2024-03-28T18:11:38.183 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:11:37 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[70064]: 2: /usr/lib64/ceph/libceph-common.so.2(+0x2d2667) [0x7f3e0b1c2667] 2024-03-28T18:11:38.183 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:11:37 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[70064]: 3: (Locker::set_xlocks_done(MutationImpl*, bool)+0x493) [0x55f4b3fc8223] 2024-03-28T18:11:38.183 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:11:37 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[70064]: 4: (Server::early_reply(boost::intrusive_ptr const&, CInode*, CDentry*)+0x3e6) [0x55f4b3e425d6] 2024-03-28T18:11:38.183 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:11:37 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[70064]: 5: (Server::journal_and_reply(boost::intrusive_ptr const&, CInode*, CDentry*, LogEvent*, MDSLogContextBase*)+0xc1) [0x55f4b3e42da1] 2024-03-28T18:11:38.183 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:11:37 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[70064]: 6: (Server::handle_client_mkdir(boost::intrusive_ptr const&)+0x5ba) [0x55f4b3e786aa] 2024-03-28T18:11:38.183 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:11:37 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[70064]: 7: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0x733) [0x55f4b3e95233] 2024-03-28T18:11:38.183 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:11:37 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[70064]: 8: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x55f4b3e97197] 2024-03-28T18:11:38.184 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:11:37 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[70064]: 9: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x55f4b3d9611b] 2024-03-28T18:11:38.184 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:11:37 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[70064]: 10: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x55f4b3d966bc] 2024-03-28T18:11:38.184 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:11:37 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[70064]: 11: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bf) [0x55f4b3d801ff] 2024-03-28T18:11:38.184 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:11:37 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[70064]: 12: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x478) [0x7f3e0b454488] 2024-03-28T18:11:38.184 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:11:37 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[70064]: 13: (DispatchQueue::entry()+0x51f) [0x7f3e0b45160f] 2024-03-28T18:11:38.184 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:11:37 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[70064]: 14: (DispatchQueue::DispatchThread::entry()+0x11) [0x7f3e0b524e81] 2024-03-28T18:11:38.184 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:11:37 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[70064]: 15: /lib64/libpthread.so.0(+0x81ca) [0x7f3e09f3f1ca] 2024-03-28T18:11:38.184 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:11:37 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[70064]: 16: clone() 2024-03-28T18:11:38.184 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:11:37 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[70064]: 2024-03-28T18:11:38.184 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:11:37 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[70064]: 0> 2024-03-28T18:11:37.918+0000 7f3dff800700 -1 *** Caught signal (Aborted) ** 2024-03-28T18:11:38.184 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:11:37 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[70064]: in thread 7f3dff800700 thread_name:ms_dispatch 2024-03-28T18:11:38.184 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:11:37 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[70064]: 2024-03-28T18:11:38.184 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:11:37 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[70064]: ceph version 19.0.0-2145-g3be9a5c9 (3be9a5c9bc793e11e2800a8c0c696e8b46742033) squid (dev) 2024-03-28T18:11:38.184 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:11:37 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[70064]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7f3e09f49d20] 2024-03-28T18:11:38.184 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:11:37 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[70064]: 2: gsignal() 2024-03-28T18:11:38.184 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:11:37 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[70064]: 3: abort() 2024-03-28T18:11:38.185 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:11:37 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[70064]: 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x18f) [0x7f3e0b1c24fb] 2024-03-28T18:11:38.185 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:11:37 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[70064]: 5: /usr/lib64/ceph/libceph-common.so.2(+0x2d2667) [0x7f3e0b1c2667] 2024-03-28T18:11:38.185 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:11:37 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[70064]: 6: (Locker::set_xlocks_done(MutationImpl*, bool)+0x493) [0x55f4b3fc8223] 2024-03-28T18:11:38.185 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:11:37 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[70064]: 7: (Server::early_reply(boost::intrusive_ptr const&, CInode*, CDentry*)+0x3e6) [0x55f4b3e425d6] 2024-03-28T18:11:38.185 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:11:37 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[70064]: 8: (Server::journal_and_reply(boost::intrusive_ptr const&, CInode*, CDentry*, LogEvent*, MDSLogContextBase*)+0xc1) [0x55f4b3e42da1] 2024-03-28T18:11:38.185 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:11:37 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[70064]: 9: (Server::handle_client_mkdir(boost::intrusive_ptr const&)+0x5ba) [0x55f4b3e786aa] 2024-03-28T18:11:38.185 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:11:37 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[70064]: 10: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0x733) [0x55f4b3e95233] 2024-03-28T18:11:38.185 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:11:37 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[70064]: 11: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x55f4b3e97197] 2024-03-28T18:11:38.185 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:11:37 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[70064]: 12: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x55f4b3d9611b] 2024-03-28T18:11:38.185 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:11:37 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[70064]: 13: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x55f4b3d966bc] 2024-03-28T18:11:38.185 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:11:37 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[70064]: 14: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bf) [0x55f4b3d801ff] 2024-03-28T18:11:38.186 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:11:37 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[70064]: 15: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x478) [0x7f3e0b454488] 2024-03-28T18:11:38.186 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:11:37 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[70064]: 16: (DispatchQueue::entry()+0x51f) [0x7f3e0b45160f] 2024-03-28T18:11:38.186 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:11:37 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[70064]: 17: (DispatchQueue::DispatchThread::entry()+0x11) [0x7f3e0b524e81] 2024-03-28T18:11:38.186 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:11:37 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[70064]: 18: /lib64/libpthread.so.0(+0x81ca) [0x7f3e09f3f1ca] 2024-03-28T18:11:38.186 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:11:37 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[70064]: 19: clone() 2024-03-28T18:11:38.186 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:11:37 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[70064]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-03-28T18:11:38.186 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:11:37 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[70064]: 2024-03-28T18:11:38.186 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:11:37 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[70064]: -1> 2024-03-28T18:11:37.914+0000 7f3dff800700 -1 /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-2145-g3be9a5c9/rpm/el8/BUILD/ceph-19.0.0-2145-g3be9a5c9/src/mds/Locker.cc: In function 'void Locker::set_xlocks_done(MutationImpl*, bool)' thread 7f3dff800700 time 2024-03-28T18:11:37.910813+0000 2024-03-28T18:11:38.186 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:11:37 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[70064]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-2145-g3be9a5c9/rpm/el8/BUILD/ceph-19.0.0-2145-g3be9a5c9/src/mds/Locker.cc: 626: FAILED ceph_assert(obj->is_auth()) 2024-03-28T18:11:38.186 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:11:37 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[70064]: 2024-03-28T18:11:38.186 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:11:37 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[70064]: ceph version 19.0.0-2145-g3be9a5c9 (3be9a5c9bc793e11e2800a8c0c696e8b46742033) squid (dev) 2024-03-28T18:11:38.186 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:11:37 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[70064]: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x135) [0x7f3e0b1c24a1] 2024-03-28T18:11:38.186 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:11:37 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[70064]: 2: /usr/lib64/ceph/libceph-common.so.2(+0x2d2667) [0x7f3e0b1c2667] 2024-03-28T18:11:38.186 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:11:37 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[70064]: 3: (Locker::set_xlocks_done(MutationImpl*, bool)+0x493) [0x55f4b3fc8223] 2024-03-28T18:11:38.186 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:11:37 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[70064]: 4: (Server::early_reply(boost::intrusive_ptr const&, CInode*, CDentry*)+0x3e6) [0x55f4b3e425d6] 2024-03-28T18:11:38.187 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:11:37 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[70064]: 5: (Server::journal_and_reply(boost::intrusive_ptr const&, CInode*, CDentry*, LogEvent*, MDSLogContextBase*)+0xc1) [0x55f4b3e42da1] 2024-03-28T18:11:38.187 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:11:37 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[70064]: 6: (Server::handle_client_mkdir(boost::intrusive_ptr const&)+0x5ba) [0x55f4b3e786aa] 2024-03-28T18:11:38.187 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:11:37 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[70064]: 7: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0x733) [0x55f4b3e95233] 2024-03-28T18:11:38.187 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:11:37 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[70064]: 8: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x55f4b3e97197] 2024-03-28T18:11:38.187 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:11:37 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[70064]: 9: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x55f4b3d9611b] 2024-03-28T18:11:38.187 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:11:37 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[70064]: 10: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x55f4b3d966bc] 2024-03-28T18:11:38.187 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:11:37 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[70064]: 11: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bf) [0x55f4b3d801ff] 2024-03-28T18:11:38.188 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:11:37 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[70064]: 12: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x478) [0x7f3e0b454488] 2024-03-28T18:11:38.188 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:11:37 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[70064]: 13: (DispatchQueue::entry()+0x51f) [0x7f3e0b45160f] 2024-03-28T18:11:38.188 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:11:37 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[70064]: 14: (DispatchQueue::DispatchThread::entry()+0x11) [0x7f3e0b524e81] 2024-03-28T18:11:38.188 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:11:37 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[70064]: 15: /lib64/libpthread.so.0(+0x81ca) [0x7f3e09f3f1ca] 2024-03-28T18:11:38.188 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:11:37 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[70064]: 16: clone() 2024-03-28T18:11:38.188 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:11:37 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[70064]: 2024-03-28T18:11:38.188 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:11:37 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[70064]: 0> 2024-03-28T18:11:37.918+0000 7f3dff800700 -1 *** Caught signal (Aborted) ** 2024-03-28T18:11:38.188 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:11:37 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[70064]: in thread 7f3dff800700 thread_name:ms_dispatch 2024-03-28T18:11:38.188 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:11:37 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[70064]: 2024-03-28T18:11:38.188 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:11:37 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[70064]: ceph version 19.0.0-2145-g3be9a5c9 (3be9a5c9bc793e11e2800a8c0c696e8b46742033) squid (dev) 2024-03-28T18:11:38.188 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:11:37 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[70064]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7f3e09f49d20] 2024-03-28T18:11:38.188 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:11:37 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[70064]: 2: gsignal() 2024-03-28T18:11:38.188 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:11:37 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[70064]: 3: abort() 2024-03-28T18:11:38.188 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:11:37 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[70064]: 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x18f) [0x7f3e0b1c24fb] 2024-03-28T18:11:38.189 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:11:37 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[70064]: 5: /usr/lib64/ceph/libceph-common.so.2(+0x2d2667) [0x7f3e0b1c2667] 2024-03-28T18:11:38.189 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:11:37 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[70064]: 6: (Locker::set_xlocks_done(MutationImpl*, bool)+0x493) [0x55f4b3fc8223] 2024-03-28T18:11:38.189 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:11:37 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[70064]: 7: (Server::early_reply(boost::intrusive_ptr const&, CInode*, CDentry*)+0x3e6) [0x55f4b3e425d6] 2024-03-28T18:11:38.189 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:11:37 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[70064]: 8: (Server::journal_and_reply(boost::intrusive_ptr const&, CInode*, CDentry*, LogEvent*, MDSLogContextBase*)+0xc1) [0x55f4b3e42da1] 2024-03-28T18:11:38.189 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:11:37 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[70064]: 9: (Server::handle_client_mkdir(boost::intrusive_ptr const&)+0x5ba) [0x55f4b3e786aa] 2024-03-28T18:11:38.189 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:11:37 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[70064]: 10: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0x733) [0x55f4b3e95233] 2024-03-28T18:11:38.189 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:11:37 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[70064]: 11: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x55f4b3e97197] 2024-03-28T18:11:38.189 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:11:37 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[70064]: 12: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x55f4b3d9611b] 2024-03-28T18:11:38.189 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:11:37 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[70064]: 13: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x55f4b3d966bc] 2024-03-28T18:11:38.189 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:11:37 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[70064]: 14: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bf) [0x55f4b3d801ff] 2024-03-28T18:11:38.189 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:11:37 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[70064]: 15: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x478) [0x7f3e0b454488] 2024-03-28T18:11:38.189 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:11:37 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[70064]: 16: (DispatchQueue::entry()+0x51f) [0x7f3e0b45160f] 2024-03-28T18:11:38.189 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:11:37 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[70064]: 17: (DispatchQueue::DispatchThread::entry()+0x11) [0x7f3e0b524e81] 2024-03-28T18:11:38.189 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:11:37 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[70064]: 18: /lib64/libpthread.so.0(+0x81ca) [0x7f3e09f3f1ca] 2024-03-28T18:11:38.189 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:11:37 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[70064]: 19: clone() 2024-03-28T18:11:38.190 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:11:37 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[70064]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-03-28T18:11:38.190 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:11:37 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[70064]: 2024-03-28T18:11:38.190 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:11:38 smithi134 podman[70106]: 2024-03-28 18:11:38.071904215 +0000 UTC m=+0.035064693 container died 6b5902d9445efe5aad8c2117d7096ee69fe245063320d93d8eea88e5f822802a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240319, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.license=GPLv2) 2024-03-28T18:11:38.829 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:11:38 smithi134 ceph-mon[35037]: pgmap v130: 97 pgs: 97 active+clean; 117 MiB data, 800 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:11:38.829 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:11:38 smithi134 ceph-mon[35037]: mds.? [v2:172.21.15.134:6834/568477027,v1:172.21.15.134:6835/568477027] up:rejoin 2024-03-28T18:11:38.829 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:11:38 smithi134 ceph-mon[35037]: fsmap cephfs:5/5 4 up:standby-replay 3 up:standby 1 up:rejoin 4 up:active 2024-03-28T18:11:38.829 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:11:38 smithi134 ceph-mon[35037]: daemon mds.f is now active in filesystem cephfs as rank 1 2024-03-28T18:11:38.829 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:11:38 smithi134 ceph-mon[35037]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-03-28T18:11:38.829 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:11:38 smithi134 ceph-mon[35037]: Cluster is now healthy 2024-03-28T18:11:38.829 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:11:38 smithi134 ceph-mon[35037]: mds.? [v2:172.21.15.134:6834/568477027,v1:172.21.15.134:6835/568477027] up:active 2024-03-28T18:11:38.829 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:11:38 smithi134 ceph-mon[35037]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-28T18:11:38.933 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:11:38 smithi079 ceph-mon[35312]: pgmap v130: 97 pgs: 97 active+clean; 117 MiB data, 800 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:11:38.933 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:11:38 smithi079 ceph-mon[35312]: mds.? [v2:172.21.15.134:6834/568477027,v1:172.21.15.134:6835/568477027] up:rejoin 2024-03-28T18:11:38.933 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:11:38 smithi079 ceph-mon[35312]: fsmap cephfs:5/5 4 up:standby-replay 3 up:standby 1 up:rejoin 4 up:active 2024-03-28T18:11:38.933 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:11:38 smithi079 ceph-mon[35312]: daemon mds.f is now active in filesystem cephfs as rank 1 2024-03-28T18:11:38.933 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:11:38 smithi079 ceph-mon[35312]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-03-28T18:11:38.933 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:11:38 smithi079 ceph-mon[35312]: Cluster is now healthy 2024-03-28T18:11:38.933 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:11:38 smithi079 ceph-mon[35312]: mds.? [v2:172.21.15.134:6834/568477027,v1:172.21.15.134:6835/568477027] up:active 2024-03-28T18:11:38.933 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:11:38 smithi079 ceph-mon[35312]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-28T18:11:39.032 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:11:38 smithi022 ceph-mon[31985]: pgmap v130: 97 pgs: 97 active+clean; 117 MiB data, 800 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:11:39.032 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:11:38 smithi022 ceph-mon[31985]: mds.? [v2:172.21.15.134:6834/568477027,v1:172.21.15.134:6835/568477027] up:rejoin 2024-03-28T18:11:39.032 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:11:38 smithi022 ceph-mon[31985]: fsmap cephfs:5/5 4 up:standby-replay 3 up:standby 1 up:rejoin 4 up:active 2024-03-28T18:11:39.032 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:11:38 smithi022 ceph-mon[31985]: daemon mds.f is now active in filesystem cephfs as rank 1 2024-03-28T18:11:39.032 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:11:38 smithi022 ceph-mon[31985]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-03-28T18:11:39.032 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:11:38 smithi022 ceph-mon[31985]: Cluster is now healthy 2024-03-28T18:11:39.032 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:11:38 smithi022 ceph-mon[31985]: mds.? [v2:172.21.15.134:6834/568477027,v1:172.21.15.134:6835/568477027] up:active 2024-03-28T18:11:39.033 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:11:38 smithi022 ceph-mon[31985]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-28T18:11:39.174 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:11:38 smithi134 podman[70106]: 2024-03-28 18:11:38.828579448 +0000 UTC m=+0.791739990 container remove 6b5902d9445efe5aad8c2117d7096ee69fe245063320d93d8eea88e5f822802a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, ceph=True, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.build-date=20240319, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-28T18:11:39.174 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:11:38 smithi134 systemd[1]: ceph-fd220748-ed2b-11ee-95d0-87774f69a715@mds.f.service: Main process exited, code=exited, status=134/n/a 2024-03-28T18:11:39.432 INFO:journalctl@ceph.mds.b.smithi079.stdout:Mar 28 18:11:39 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-b[65245]: 2024-03-28T18:11:39.081+0000 7f845f400700 -1 mds.pinger is_rank_lagging: rank=1 was never sent ping request. 2024-03-28T18:11:39.674 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:11:39 smithi134 systemd[1]: ceph-fd220748-ed2b-11ee-95d0-87774f69a715@mds.f.service: Failed with result 'exit-code'. 2024-03-28T18:11:39.675 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:11:39 smithi134 systemd[1]: ceph-fd220748-ed2b-11ee-95d0-87774f69a715@mds.f.service: Consumed 1.812s CPU time. 2024-03-28T18:11:40.925 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:11:40 smithi134 ceph-mon[35037]: pgmap v131: 97 pgs: 97 active+clean; 117 MiB data, 800 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 4.5 KiB/s wr, 20 op/s 2024-03-28T18:11:40.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:11:40 smithi079 ceph-mon[35312]: pgmap v131: 97 pgs: 97 active+clean; 117 MiB data, 800 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 4.5 KiB/s wr, 20 op/s 2024-03-28T18:11:41.031 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:11:40 smithi022 ceph-mon[31985]: pgmap v131: 97 pgs: 97 active+clean; 117 MiB data, 800 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 4.5 KiB/s wr, 20 op/s 2024-03-28T18:11:42.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:11:42 smithi134 ceph-mon[35037]: pgmap v132: 97 pgs: 97 active+clean; 117 MiB data, 800 MiB used, 1.0 TiB / 1.0 TiB avail; 109 KiB/s rd, 4.7 KiB/s wr, 26 op/s 2024-03-28T18:11:42.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:11:42 smithi079 ceph-mon[35312]: pgmap v132: 97 pgs: 97 active+clean; 117 MiB data, 800 MiB used, 1.0 TiB / 1.0 TiB avail; 109 KiB/s rd, 4.7 KiB/s wr, 26 op/s 2024-03-28T18:11:43.031 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:11:42 smithi022 ceph-mon[31985]: pgmap v132: 97 pgs: 97 active+clean; 117 MiB data, 800 MiB used, 1.0 TiB / 1.0 TiB avail; 109 KiB/s rd, 4.7 KiB/s wr, 26 op/s 2024-03-28T18:11:44.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:11:44 smithi134 ceph-mon[35037]: pgmap v133: 97 pgs: 97 active+clean; 117 MiB data, 800 MiB used, 1.0 TiB / 1.0 TiB avail; 109 KiB/s rd, 4.7 KiB/s wr, 26 op/s 2024-03-28T18:11:44.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:11:44 smithi079 ceph-mon[35312]: pgmap v133: 97 pgs: 97 active+clean; 117 MiB data, 800 MiB used, 1.0 TiB / 1.0 TiB avail; 109 KiB/s rd, 4.7 KiB/s wr, 26 op/s 2024-03-28T18:11:45.031 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:11:44 smithi022 ceph-mon[31985]: pgmap v133: 97 pgs: 97 active+clean; 117 MiB data, 800 MiB used, 1.0 TiB / 1.0 TiB avail; 109 KiB/s rd, 4.7 KiB/s wr, 26 op/s 2024-03-28T18:11:46.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:11:46 smithi134 ceph-mon[35037]: pgmap v134: 97 pgs: 97 active+clean; 117 MiB data, 800 MiB used, 1.0 TiB / 1.0 TiB avail; 98 KiB/s rd, 4.1 KiB/s wr, 27 op/s 2024-03-28T18:11:46.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:11:46 smithi079 ceph-mon[35312]: pgmap v134: 97 pgs: 97 active+clean; 117 MiB data, 800 MiB used, 1.0 TiB / 1.0 TiB avail; 98 KiB/s rd, 4.1 KiB/s wr, 27 op/s 2024-03-28T18:11:47.032 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:11:46 smithi022 ceph-mon[31985]: pgmap v134: 97 pgs: 97 active+clean; 117 MiB data, 800 MiB used, 1.0 TiB / 1.0 TiB avail; 98 KiB/s rd, 4.1 KiB/s wr, 27 op/s 2024-03-28T18:11:47.127 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-28T18:11:47.127 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-28T18:11:47.365 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:11:47.364+0000 7ff537200640 1 -- 172.21.15.22:0/4177492963 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff538108800 msgr2=0x7ff53810c7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:11:47.365 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:11:47.364+0000 7ff537200640 1 --2- 172.21.15.22:0/4177492963 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff538108800 0x7ff53810c7c0 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7ff524009900 tx=0x7ff52402bc50 comp rx=0 tx=0).stop 2024-03-28T18:11:47.365 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:11:47.364+0000 7ff537200640 1 -- 172.21.15.22:0/4177492963 shutdown_connections 2024-03-28T18:11:47.365 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:11:47.364+0000 7ff537200640 1 --2- 172.21.15.22:0/4177492963 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff53810cea0 0x7ff538114a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:11:47.365 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:11:47.364+0000 7ff537200640 1 --2- 172.21.15.22:0/4177492963 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff538108800 0x7ff53810c7c0 unknown :-1 s=CLOSED pgs=705 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:11:47.366 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:11:47.364+0000 7ff537200640 1 --2- 172.21.15.22:0/4177492963 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ff538107e50 0x7ff538108230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:11:47.366 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:11:47.364+0000 7ff537200640 1 -- 172.21.15.22:0/4177492963 >> 172.21.15.22:0/4177492963 conn(0x7ff5380774c0 msgr2=0x7ff5381064c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:11:47.366 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:11:47.364+0000 7ff537200640 1 -- 172.21.15.22:0/4177492963 shutdown_connections 2024-03-28T18:11:47.366 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:11:47.364+0000 7ff537200640 1 -- 172.21.15.22:0/4177492963 wait complete. 2024-03-28T18:11:47.367 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:11:47.365+0000 7ff537200640 1 Processor -- start 2024-03-28T18:11:47.368 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:11:47.365+0000 7ff537200640 1 -- start start 2024-03-28T18:11:47.368 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:11:47.366+0000 7ff537200640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff538107e50 0x7ff53819e400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:11:47.368 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:11:47.366+0000 7ff537200640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ff53810cea0 0x7ff53819e940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:11:47.368 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:11:47.366+0000 7ff537200640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff5381a2cc0 0x7ff5381a3170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:11:47.368 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:11:47.366+0000 7ff537200640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff538117f20 con 0x7ff5381a2cc0 2024-03-28T18:11:47.368 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:11:47.366+0000 7ff537200640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff538117da0 con 0x7ff53810cea0 2024-03-28T18:11:47.368 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:11:47.366+0000 7ff537200640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5381180a0 con 0x7ff538107e50 2024-03-28T18:11:47.368 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:11:47.366+0000 7ff536800640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff5381a2cc0 0x7ff5381a3170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:11:47.368 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:11:47.366+0000 7ff536800640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff5381a2cc0 0x7ff5381a3170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:44160/0 (socket says 172.21.15.22:44160) 2024-03-28T18:11:47.368 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:11:47.366+0000 7ff536800640 1 -- 172.21.15.22:0/461233731 learned_addr learned my addr 172.21.15.22:0/461233731 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-28T18:11:47.368 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:11:47.366+0000 7ff535400640 1 --2- 172.21.15.22:0/461233731 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ff53810cea0 0x7ff53819e940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:11:47.368 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:11:47.366+0000 7ff535e00640 1 --2- 172.21.15.22:0/461233731 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff538107e50 0x7ff53819e400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:11:47.368 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:11:47.367+0000 7ff536800640 1 -- 172.21.15.22:0/461233731 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff538107e50 msgr2=0x7ff53819e400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:11:47.369 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:11:47.367+0000 7ff536800640 1 --2- 172.21.15.22:0/461233731 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff538107e50 0x7ff53819e400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:11:47.369 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:11:47.367+0000 7ff536800640 1 -- 172.21.15.22:0/461233731 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ff53810cea0 msgr2=0x7ff53819e940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:11:47.369 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:11:47.367+0000 7ff536800640 1 --2- 172.21.15.22:0/461233731 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ff53810cea0 0x7ff53819e940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:11:47.369 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:11:47.367+0000 7ff536800640 1 -- 172.21.15.22:0/461233731 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff5381a37b0 con 0x7ff5381a2cc0 2024-03-28T18:11:47.369 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:11:47.367+0000 7ff535e00640 1 --2- 172.21.15.22:0/461233731 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff538107e50 0x7ff53819e400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-28T18:11:47.370 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:11:47.367+0000 7ff536800640 1 --2- 172.21.15.22:0/461233731 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff5381a2cc0 0x7ff5381a3170 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7ff53000e960 tx=0x7ff53000ee20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:11:47.370 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:11:47.368+0000 7ff52ea00640 1 -- 172.21.15.22:0/461233731 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff530004d20 con 0x7ff5381a2cc0 2024-03-28T18:11:47.370 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:11:47.368+0000 7ff52ea00640 1 -- 172.21.15.22:0/461233731 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7ff530005020 con 0x7ff5381a2cc0 2024-03-28T18:11:47.370 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:11:47.368+0000 7ff537200640 1 -- 172.21.15.22:0/461233731 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff5381a3aa0 con 0x7ff5381a2cc0 2024-03-28T18:11:47.370 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:11:47.368+0000 7ff52ea00640 1 -- 172.21.15.22:0/461233731 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5300116d0 con 0x7ff5381a2cc0 2024-03-28T18:11:47.370 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:11:47.368+0000 7ff537200640 1 -- 172.21.15.22:0/461233731 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff5381ab260 con 0x7ff5381a2cc0 2024-03-28T18:11:47.372 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:11:47.371+0000 7ff52ea00640 1 -- 172.21.15.22:0/461233731 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 26) v1 ==== 137421+0+0 (secure 0 0 0) 0x7ff5300051d0 con 0x7ff5381a2cc0 2024-03-28T18:11:47.372 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:11:47.371+0000 7ff53fb6e740 1 Processor -- start 2024-03-28T18:11:47.372 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:11:47.371+0000 7ff53fb6e740 1 -- start start 2024-03-28T18:11:47.374 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:11:47.372+0000 7ff53fb6e740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5644adc48dd0 0x5644adc47b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:11:47.374 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:11:47.372+0000 7ff53fb6e740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5644adc480f0 0x5644adc48570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:11:47.374 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:11:47.372+0000 7ff53fb6e740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5644adc51460 0x5644adc53850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:11:47.374 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:11:47.372+0000 7ff53fb6e740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x5644adbac250 con 0x5644adc51460 2024-03-28T18:11:47.374 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:11:47.372+0000 7ff53fb6e740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x5644adb54a30 con 0x5644adc48dd0 2024-03-28T18:11:47.374 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:11:47.372+0000 7ff53fb6e740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x5644adba82f0 con 0x5644adc480f0 2024-03-28T18:11:47.374 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:11:47.372+0000 7ff535400640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5644adc51460 0x5644adc53850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:11:47.374 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:11:47.372+0000 7ff535400640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5644adc51460 0x5644adc53850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:44168/0 (socket says 172.21.15.22:44168) 2024-03-28T18:11:47.375 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:11:47.372+0000 7ff535400640 1 -- 172.21.15.22:0/340826714 learned_addr learned my addr 172.21.15.22:0/340826714 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-28T18:11:47.375 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:11:47.372+0000 7ff535400640 1 --2- 172.21.15.22:0/340826714 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5644adc48dd0 0x5644adc47b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:11:47.375 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:11:47.372+0000 7ff535e00640 1 --2- 172.21.15.22:0/340826714 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5644adc480f0 0x5644adc48570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:11:47.375 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:11:47.373+0000 7ff535400640 1 -- 172.21.15.22:0/340826714 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5644adc480f0 msgr2=0x5644adc48570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T18:11:47.375 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:11:47.373+0000 7ff535400640 1 --2- 172.21.15.22:0/340826714 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5644adc480f0 0x5644adc48570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:11:47.375 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:11:47.373+0000 7ff535400640 1 -- 172.21.15.22:0/340826714 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5644adc48dd0 msgr2=0x5644adc47b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T18:11:47.375 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:11:47.373+0000 7ff535400640 1 --2- 172.21.15.22:0/340826714 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5644adc48dd0 0x5644adc47b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:11:47.375 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:11:47.373+0000 7ff535400640 1 -- 172.21.15.22:0/340826714 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5644adb3f690 con 0x5644adc51460 2024-03-28T18:11:47.375 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:11:47.373+0000 7ff535e00640 1 --2- 172.21.15.22:0/340826714 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5644adc480f0 0x5644adc48570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-28T18:11:47.375 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:11:47.373+0000 7ff52ea00640 1 --2- 172.21.15.22:0/461233731 >> [v2:172.21.15.22:6800/1357365082,v1:172.21.15.22:6801/1357365082] conn(0x7ff51c0a2970 0x7ff51c0a4e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:11:47.375 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:11:47.373+0000 7ff535400640 1 --2- 172.21.15.22:0/340826714 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5644adc51460 0x5644adc53850 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7ff524006010 tx=0x7ff52402cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aeca9dd6b55923fc server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:11:47.375 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:11:47.373+0000 7ff52ea00640 1 -- 172.21.15.22:0/461233731 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 8841+0+0 (secure 0 0 0) 0x7ff5300cf490 con 0x7ff5381a2cc0 2024-03-28T18:11:47.375 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:11:47.373+0000 7ff535e00640 1 --2- 172.21.15.22:0/461233731 >> [v2:172.21.15.22:6800/1357365082,v1:172.21.15.22:6801/1357365082] conn(0x7ff51c0a2970 0x7ff51c0a4e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:11:47.375 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:11:47.373+0000 7ff523400640 1 -- 172.21.15.22:0/340826714 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff52402c7f0 con 0x5644adc51460 2024-03-28T18:11:47.376 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:11:47.373+0000 7ff523400640 1 -- 172.21.15.22:0/340826714 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7ff5240035f0 con 0x5644adc51460 2024-03-28T18:11:47.376 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:11:47.373+0000 7ff523400640 1 -- 172.21.15.22:0/340826714 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff524034640 con 0x5644adc51460 2024-03-28T18:11:47.379 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:11:47.374+0000 7ff53fb6e740 1 -- 172.21.15.22:0/340826714 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5644adc51460 msgr2=0x5644adc53850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:11:47.380 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:11:47.374+0000 7ff535e00640 1 --2- 172.21.15.22:0/461233731 >> [v2:172.21.15.22:6800/1357365082,v1:172.21.15.22:6801/1357365082] conn(0x7ff51c0a2970 0x7ff51c0a4e30 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x5644adc49320 tx=0x7ff52800b7d0 comp rx=0 tx=0).ready entity=mgr.15522 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:11:47.380 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:11:47.374+0000 7ff53fb6e740 1 --2- 172.21.15.22:0/340826714 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5644adc51460 0x5644adc53850 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7ff524006010 tx=0x7ff52402cf90 comp rx=0 tx=0).stop 2024-03-28T18:11:47.380 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:11:47.374+0000 7ff53fb6e740 1 -- 172.21.15.22:0/340826714 shutdown_connections 2024-03-28T18:11:47.380 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:11:47.374+0000 7ff53fb6e740 1 --2- 172.21.15.22:0/340826714 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5644adc51460 0x5644adc53850 unknown :-1 s=CLOSED pgs=707 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:11:47.380 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:11:47.374+0000 7ff53fb6e740 1 --2- 172.21.15.22:0/340826714 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5644adc480f0 0x5644adc48570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:11:47.380 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:11:47.374+0000 7ff53fb6e740 1 --2- 172.21.15.22:0/340826714 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5644adc48dd0 0x5644adc47b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:11:47.380 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:11:47.374+0000 7ff53fb6e740 1 -- 172.21.15.22:0/340826714 >> 172.21.15.22:0/340826714 conn(0x5644adc0f200 msgr2=0x5644adc42b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:11:47.380 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:11:47.374+0000 7ff53fb6e740 1 -- 172.21.15.22:0/340826714 shutdown_connections 2024-03-28T18:11:47.380 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:11:47.374+0000 7ff53fb6e740 1 -- 172.21.15.22:0/340826714 wait complete. 2024-03-28T18:11:47.380 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:11:47.375+0000 7ff53fb6e740 1 -- start start 2024-03-28T18:11:47.380 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:11:47.375+0000 7ff53fb6e740 1 Processor -- start 2024-03-28T18:11:47.380 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:11:47.376+0000 7ff53fb6e740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5644adc480f0 0x5644adc50b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:11:47.380 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:11:47.376+0000 7ff53fb6e740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5644adc48dd0 0x5644adc4dee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:11:47.381 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:11:47.376+0000 7ff53fb6e740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5644adc51460 0x5644adc4e420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:11:47.381 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:11:47.376+0000 7ff53fb6e740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x5644adc54270 con 0x5644adc51460 2024-03-28T18:11:47.381 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:11:47.376+0000 7ff53fb6e740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x5644adb67e80 con 0x5644adc48dd0 2024-03-28T18:11:47.381 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:11:47.376+0000 7ff53fb6e740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x5644adc543f0 con 0x5644adc480f0 2024-03-28T18:11:47.381 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:11:47.376+0000 7ff53fb6e740 10 client.0 _ll_register_callbacks cb 0x5644adb7a130 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-28T18:11:47.381 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:11:47.376+0000 7ff536800640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5644adc51460 0x5644adc4e420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:11:47.381 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:11:47.376+0000 7ff536800640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5644adc51460 0x5644adc4e420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:44170/0 (socket says 172.21.15.22:44170) 2024-03-28T18:11:47.381 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:11:47.376+0000 7ff536800640 1 -- 172.21.15.22:0/1525925892 learned_addr learned my addr 172.21.15.22:0/1525925892 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-28T18:11:47.381 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:11:47.376+0000 7ff535e00640 1 --2- 172.21.15.22:0/1525925892 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5644adc480f0 0x5644adc50b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:11:47.381 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:11:47.376+0000 7ff535400640 1 --2- 172.21.15.22:0/1525925892 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5644adc48dd0 0x5644adc4dee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:11:47.381 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:11:47.376+0000 7ff536800640 1 -- 172.21.15.22:0/1525925892 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5644adc480f0 msgr2=0x5644adc50b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T18:11:47.381 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:11:47.376+0000 7ff536800640 1 --2- 172.21.15.22:0/1525925892 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5644adc480f0 0x5644adc50b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:11:47.381 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:11:47.376+0000 7ff536800640 1 -- 172.21.15.22:0/1525925892 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5644adc48dd0 msgr2=0x5644adc4dee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T18:11:47.381 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:11:47.376+0000 7ff536800640 1 --2- 172.21.15.22:0/1525925892 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5644adc48dd0 0x5644adc4dee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:11:47.382 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:11:47.376+0000 7ff536800640 1 -- 172.21.15.22:0/1525925892 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5644ad98a7d0 con 0x5644adc51460 2024-03-28T18:11:47.382 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:11:47.377+0000 7ff536800640 1 --2- 172.21.15.22:0/1525925892 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5644adc51460 0x5644adc4e420 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7ff5300a3bc0 tx=0x7ff5300055d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=64edc9285aad4d55 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:11:47.382 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:11:47.377+0000 7ff517e00640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-03-28T18:11:47.382 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:11:47.377+0000 7ff517e00640 1 -- 172.21.15.22:0/1525925892 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff53001f040 con 0x5644adc51460 2024-03-28T18:11:47.382 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:11:47.377+0000 7ff517e00640 1 -- 172.21.15.22:0/1525925892 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(98..98 src has 1..98) v4 ==== 8841+0+0 (secure 0 0 0) 0x7ff5300862e0 con 0x5644adc51460 2024-03-28T18:11:47.382 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:11:47.377+0000 7ff53fb6e740 1 -- 172.21.15.22:0/1525925892 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7ff5240035f0 con 0x5644adc51460 2024-03-28T18:11:47.382 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:11:47.377+0000 7ff517e00640 1 -- 172.21.15.22:0/1525925892 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=64) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff53009b950 con 0x5644adc51460 2024-03-28T18:11:47.382 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:11:47.377+0000 7ff53fb6e740 10 client.16545 fetch_fsmap learned FSMap version 64 2024-03-28T18:11:47.382 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:11:47.377+0000 7ff53fb6e740 1 -- 172.21.15.22:0/1525925892 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=64,monmap=4+}) v3 -- 0x5644ad8c03f0 con 0x5644adc51460 2024-03-28T18:11:47.382 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:11:47.377+0000 7ff517e00640 1 -- 172.21.15.22:0/1525925892 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7ff5240035f0 con 0x5644adc51460 2024-03-28T18:11:47.382 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:11:47.378+0000 7ff517e00640 1 -- 172.21.15.22:0/1525925892 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 64) v1 ==== 7950+0+0 (secure 0 0 0) 0x7ff53000f040 con 0x5644adc51460 2024-03-28T18:11:47.382 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:11:47.378+0000 7ff53fb6e740 10 client.16545 fetch_fsmap finished waiting for FSMap version 64 2024-03-28T18:11:47.382 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:11:47.378+0000 7ff53fb6e740 10 client.16545 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-28T18:11:47.382 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:11:47.378+0000 7ff53fb6e740 20 client.16545 populate_metadata read hostname 'smithi022' 2024-03-28T18:11:47.383 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:11:47.378+0000 7ff53fb6e740 1 --2- 172.21.15.22:0/1525925892 >> [v2:172.21.15.79:6838/2441542443,v1:172.21.15.79:6839/2441542443] conn(0x5644adc61360 0x5644adc63780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:11:47.383 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:11:47.378+0000 7ff53fb6e740 4 client.16545 mds_command: new command op to 34553 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-28T18:11:47.383 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:11:47.378+0000 7ff53fb6e740 1 -- 172.21.15.22:0/1525925892 --> [v2:172.21.15.79:6838/2441542443,v1:172.21.15.79:6839/2441542443] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5644adbb9790 con 0x5644adc61360 2024-03-28T18:11:47.383 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:11:47.378+0000 7ff535e00640 1 --2- 172.21.15.22:0/1525925892 >> [v2:172.21.15.79:6838/2441542443,v1:172.21.15.79:6839/2441542443] conn(0x5644adc61360 0x5644adc63780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:11:47.384 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:11:47.379+0000 7ff535e00640 1 --2- 172.21.15.22:0/1525925892 >> [v2:172.21.15.79:6838/2441542443,v1:172.21.15.79:6839/2441542443] conn(0x5644adc61360 0x5644adc63780 crc :-1 s=READY pgs=132 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d0c317f82dadc64 server_cookie=374a4741fdf01b17 in_seq=0 out_seq=0 2024-03-28T18:11:47.384 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:11:47.379+0000 7ff517e00640 10 client.16545 ms_handle_connect on v2:172.21.15.79:6838/2441542443 2024-03-28T18:11:47.384 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:11:47.381+0000 7ff517e00640 1 -- 172.21.15.22:0/1525925892 <== mds.0 v2:172.21.15.79:6838/2441542443 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x5644adbb9790 con 0x5644adc61360 2024-03-28T18:11:47.384 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:11:47.381+0000 7ff517e00640 10 client.16545 handle_command_reply: tid=0 2024-03-28T18:11:47.384 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:11:47.381+0000 7ff53fb6e740 1 client.16545 shutdown 2024-03-28T18:11:47.384 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:11:47.382+0000 7ff53fb6e740 1 -- 172.21.15.22:0/1525925892 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5644adc51460 msgr2=0x5644adc4e420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:11:47.384 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:11:47.382+0000 7ff53fb6e740 1 --2- 172.21.15.22:0/1525925892 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5644adc51460 0x5644adc4e420 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7ff5300a3bc0 tx=0x7ff5300055d0 comp rx=0 tx=0).stop 2024-03-28T18:11:47.385 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:11:47.382+0000 7ff53fb6e740 1 -- 172.21.15.22:0/1525925892 shutdown_connections 2024-03-28T18:11:47.385 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:11:47.382+0000 7ff53fb6e740 1 --2- 172.21.15.22:0/1525925892 >> [v2:172.21.15.79:6838/2441542443,v1:172.21.15.79:6839/2441542443] conn(0x5644adc61360 0x5644adc63780 crc :-1 s=READY pgs=132 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:11:47.385 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:11:47.382+0000 7ff53fb6e740 1 --2- 172.21.15.22:0/1525925892 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5644adc51460 0x5644adc4e420 unknown :-1 s=CLOSED pgs=708 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:11:47.385 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:11:47.382+0000 7ff53fb6e740 1 --2- 172.21.15.22:0/1525925892 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5644adc48dd0 0x5644adc4dee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:11:47.385 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:11:47.382+0000 7ff53fb6e740 1 --2- 172.21.15.22:0/1525925892 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5644adc480f0 0x5644adc50b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:11:47.385 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:11:47.382+0000 7ff517e00640 0 client.16545 ms_handle_reset on v2:172.21.15.79:6838/2441542443 2024-03-28T18:11:47.385 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:11:47.382+0000 7ff53fb6e740 1 -- 172.21.15.22:0/1525925892 >> 172.21.15.22:0/1525925892 conn(0x5644adc0f200 msgr2=0x5644adc44890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:11:47.385 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:11:47.382+0000 7ff53fb6e740 1 -- 172.21.15.22:0/1525925892 shutdown_connections 2024-03-28T18:11:47.385 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:11:47.382+0000 7ff53fb6e740 1 -- 172.21.15.22:0/1525925892 wait complete. 2024-03-28T18:11:47.385 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:11:47.382+0000 7ff53fb6e740 20 client.16545 trim_cache size 0 max 16384 2024-03-28T18:11:47.401 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:11:47.400+0000 7ff53fb6e740 1 Processor -- start 2024-03-28T18:11:47.401 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:11:47.400+0000 7ff53fb6e740 1 -- start start 2024-03-28T18:11:47.401 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:11:47.400+0000 7ff53fb6e740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5644adc480f0 0x5644adc50610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:11:47.401 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:11:47.400+0000 7ff53fb6e740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5644adc48dd0 0x5644adc50b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:11:47.401 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:11:47.400+0000 7ff53fb6e740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5644adc51460 0x5644adc52850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:11:47.401 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:11:47.400+0000 7ff53fb6e740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5381b3670 con 0x5644adc51460 2024-03-28T18:11:47.401 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:11:47.400+0000 7ff53fb6e740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5381b50b0 con 0x5644adc48dd0 2024-03-28T18:11:47.401 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:11:47.400+0000 7ff53fb6e740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5381b3cd0 con 0x5644adc480f0 2024-03-28T18:11:47.403 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:11:47.401+0000 7ff536800640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5644adc51460 0x5644adc52850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:11:47.403 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:11:47.401+0000 7ff536800640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5644adc51460 0x5644adc52850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:44184/0 (socket says 172.21.15.22:44184) 2024-03-28T18:11:47.403 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:11:47.401+0000 7ff535400640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5644adc48dd0 0x5644adc50b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:11:47.403 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:11:47.401+0000 7ff536800640 1 -- 172.21.15.22:0/1485906147 learned_addr learned my addr 172.21.15.22:0/1485906147 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-28T18:11:47.403 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:11:47.401+0000 7ff535e00640 1 --2- 172.21.15.22:0/1485906147 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5644adc480f0 0x5644adc50610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:11:47.403 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:11:47.401+0000 7ff536800640 1 -- 172.21.15.22:0/1485906147 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5644adc480f0 msgr2=0x5644adc50610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T18:11:47.403 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:11:47.401+0000 7ff536800640 1 --2- 172.21.15.22:0/1485906147 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5644adc480f0 0x5644adc50610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:11:47.403 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:11:47.401+0000 7ff536800640 1 -- 172.21.15.22:0/1485906147 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5644adc48dd0 msgr2=0x5644adc50b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T18:11:47.404 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:11:47.401+0000 7ff536800640 1 --2- 172.21.15.22:0/1485906147 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5644adc48dd0 0x5644adc50b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:11:47.404 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:11:47.401+0000 7ff536800640 1 -- 172.21.15.22:0/1485906147 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5644ad8800f0 con 0x5644adc51460 2024-03-28T18:11:47.404 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:11:47.401+0000 7ff536800640 1 --2- 172.21.15.22:0/1485906147 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5644adc51460 0x5644adc52850 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7ff530098320 tx=0x7ff5300a40f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=51ba90da180ec772 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:11:47.404 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:11:47.402+0000 7ff522a00640 1 -- 172.21.15.22:0/1485906147 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff53001f070 con 0x5644adc51460 2024-03-28T18:11:47.404 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:11:47.402+0000 7ff522a00640 1 -- 172.21.15.22:0/1485906147 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7ff53009faa0 con 0x5644adc51460 2024-03-28T18:11:47.404 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:11:47.402+0000 7ff522a00640 1 -- 172.21.15.22:0/1485906147 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff530086320 con 0x5644adc51460 2024-03-28T18:11:47.404 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:11:47.402+0000 7ff53fb6e740 1 -- 172.21.15.22:0/1485906147 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5644adc51460 msgr2=0x5644adc52850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:11:47.404 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:11:47.402+0000 7ff53fb6e740 1 --2- 172.21.15.22:0/1485906147 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5644adc51460 0x5644adc52850 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7ff530098320 tx=0x7ff5300a40f0 comp rx=0 tx=0).stop 2024-03-28T18:11:47.404 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:11:47.402+0000 7ff53fb6e740 1 -- 172.21.15.22:0/1485906147 shutdown_connections 2024-03-28T18:11:47.404 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:11:47.402+0000 7ff53fb6e740 1 --2- 172.21.15.22:0/1485906147 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5644adc51460 0x5644adc52850 unknown :-1 s=CLOSED pgs=709 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:11:47.404 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:11:47.402+0000 7ff53fb6e740 1 --2- 172.21.15.22:0/1485906147 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5644adc48dd0 0x5644adc50b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:11:47.404 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:11:47.402+0000 7ff53fb6e740 1 --2- 172.21.15.22:0/1485906147 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5644adc480f0 0x5644adc50610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:11:47.404 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:11:47.402+0000 7ff53fb6e740 1 -- 172.21.15.22:0/1485906147 >> 172.21.15.22:0/1485906147 conn(0x5644adc0f200 msgr2=0x5644adc4c7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:11:47.404 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:11:47.402+0000 7ff53fb6e740 1 -- 172.21.15.22:0/1485906147 shutdown_connections 2024-03-28T18:11:47.404 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:11:47.402+0000 7ff53fb6e740 1 -- 172.21.15.22:0/1485906147 wait complete. 2024-03-28T18:11:47.407 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:11:47.403+0000 7ff53fb6e740 1 -- start start 2024-03-28T18:11:47.407 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:11:47.403+0000 7ff53fb6e740 1 Processor -- start 2024-03-28T18:11:47.407 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:11:47.403+0000 7ff53fb6e740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5644adc480f0 0x5644adc4a860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:11:47.407 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:11:47.403+0000 7ff53fb6e740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5644adc48dd0 0x5644adc4ada0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:11:47.408 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:11:47.403+0000 7ff53fb6e740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5644adc51460 0x5644adcbbc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:11:47.408 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:11:47.403+0000 7ff53fb6e740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5381b4a50 con 0x5644adc51460 2024-03-28T18:11:47.408 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:11:47.403+0000 7ff53fb6e740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5381b4330 con 0x5644adc48dd0 2024-03-28T18:11:47.408 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:11:47.403+0000 7ff53fb6e740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5381b1870 con 0x5644adc480f0 2024-03-28T18:11:47.408 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:11:47.404+0000 7ff53fb6e740 10 client.0 _ll_register_callbacks cb 0x7ff5300057c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-28T18:11:47.408 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:11:47.404+0000 7ff536800640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5644adc51460 0x5644adcbbc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:11:47.408 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:11:47.404+0000 7ff536800640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5644adc51460 0x5644adcbbc50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:44194/0 (socket says 172.21.15.22:44194) 2024-03-28T18:11:47.408 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:11:47.404+0000 7ff536800640 1 -- 172.21.15.22:0/1382672085 learned_addr learned my addr 172.21.15.22:0/1382672085 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-28T18:11:47.408 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:11:47.404+0000 7ff535400640 1 --2- 172.21.15.22:0/1382672085 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5644adc48dd0 0x5644adc4ada0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:11:47.408 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:11:47.404+0000 7ff535e00640 1 --2- 172.21.15.22:0/1382672085 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5644adc480f0 0x5644adc4a860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:11:47.408 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:11:47.404+0000 7ff536800640 1 -- 172.21.15.22:0/1382672085 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5644adc480f0 msgr2=0x5644adc4a860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T18:11:47.408 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:11:47.404+0000 7ff536800640 1 --2- 172.21.15.22:0/1382672085 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5644adc480f0 0x5644adc4a860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:11:47.408 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:11:47.404+0000 7ff536800640 1 -- 172.21.15.22:0/1382672085 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5644adc48dd0 msgr2=0x5644adc4ada0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T18:11:47.408 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:11:47.404+0000 7ff536800640 1 --2- 172.21.15.22:0/1382672085 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5644adc48dd0 0x5644adc4ada0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:11:47.409 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:11:47.404+0000 7ff536800640 1 -- 172.21.15.22:0/1382672085 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5644adc4b650 con 0x5644adc51460 2024-03-28T18:11:47.409 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:11:47.404+0000 7ff535400640 1 --2- 172.21.15.22:0/1382672085 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5644adc48dd0 0x5644adc4ada0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-28T18:11:47.409 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:11:47.405+0000 7ff536800640 1 --2- 172.21.15.22:0/1382672085 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5644adc51460 0x5644adcbbc50 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7ff5300a3bc0 tx=0x7ff530000f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=da7b0dbd9f9da385 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:11:47.409 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:11:47.405+0000 7ff517e00640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-03-28T18:11:47.409 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:11:47.405+0000 7ff517e00640 1 -- 172.21.15.22:0/1382672085 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff53001f070 con 0x5644adc51460 2024-03-28T18:11:47.409 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:11:47.405+0000 7ff517e00640 1 -- 172.21.15.22:0/1382672085 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(98..98 src has 1..98) v4 ==== 8841+0+0 (secure 0 0 0) 0x7ff53009ada0 con 0x5644adc51460 2024-03-28T18:11:47.409 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:11:47.405+0000 7ff53fb6e740 1 -- 172.21.15.22:0/1382672085 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7ff53009faa0 con 0x5644adc51460 2024-03-28T18:11:47.409 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:11:47.405+0000 7ff517e00640 1 -- 172.21.15.22:0/1382672085 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=64) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff53009afe0 con 0x5644adc51460 2024-03-28T18:11:47.409 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:11:47.405+0000 7ff53fb6e740 10 client.16551 fetch_fsmap learned FSMap version 64 2024-03-28T18:11:47.409 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:11:47.405+0000 7ff53fb6e740 1 -- 172.21.15.22:0/1382672085 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=64,monmap=4+}) v3 -- 0x5644adcbd280 con 0x5644adc51460 2024-03-28T18:11:47.409 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:11:47.405+0000 7ff517e00640 1 -- 172.21.15.22:0/1382672085 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7ff53009faa0 con 0x5644adc51460 2024-03-28T18:11:47.409 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:11:47.405+0000 7ff517e00640 1 -- 172.21.15.22:0/1382672085 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 64) v1 ==== 7950+0+0 (secure 0 0 0) 0x7ff530082030 con 0x5644adc51460 2024-03-28T18:11:47.409 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:11:47.406+0000 7ff53fb6e740 10 client.16551 fetch_fsmap finished waiting for FSMap version 64 2024-03-28T18:11:47.409 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:11:47.406+0000 7ff53fb6e740 10 client.16551 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-28T18:11:47.409 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:11:47.406+0000 7ff53fb6e740 20 client.16551 populate_metadata read hostname 'smithi022' 2024-03-28T18:11:47.410 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:11:47.406+0000 7ff53fb6e740 1 --2- 172.21.15.22:0/1382672085 >> [v2:172.21.15.79:6838/2441542443,v1:172.21.15.79:6839/2441542443] conn(0x5644adcbd6f0 0x5644adcbfb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:11:47.410 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:11:47.406+0000 7ff53fb6e740 4 client.16551 mds_command: new command op to 34553 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-28T18:11:47.410 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:11:47.406+0000 7ff53fb6e740 1 -- 172.21.15.22:0/1382672085 --> [v2:172.21.15.79:6838/2441542443,v1:172.21.15.79:6839/2441542443] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x5644adbb3ae0 con 0x5644adcbd6f0 2024-03-28T18:11:47.410 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:11:47.406+0000 7ff535e00640 1 --2- 172.21.15.22:0/1382672085 >> [v2:172.21.15.79:6838/2441542443,v1:172.21.15.79:6839/2441542443] conn(0x5644adcbd6f0 0x5644adcbfb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:11:47.411 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:11:47.407+0000 7ff535e00640 1 --2- 172.21.15.22:0/1382672085 >> [v2:172.21.15.79:6838/2441542443,v1:172.21.15.79:6839/2441542443] conn(0x5644adcbd6f0 0x5644adcbfb10 crc :-1 s=READY pgs=133 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ecf3d3a643cce52c server_cookie=c0837812ec7582ce in_seq=0 out_seq=0 2024-03-28T18:11:47.411 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:11:47.407+0000 7ff517e00640 10 client.16551 ms_handle_connect on v2:172.21.15.79:6838/2441542443 2024-03-28T18:11:47.411 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:11:47.408+0000 7ff517e00640 1 -- 172.21.15.22:0/1382672085 <== mds.0 v2:172.21.15.79:6838/2441542443 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x5644adbb3ae0 con 0x5644adcbd6f0 2024-03-28T18:11:47.411 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:11:47.408+0000 7ff517e00640 10 client.16551 handle_command_reply: tid=0 2024-03-28T18:11:47.411 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:11:47.408+0000 7ff53fb6e740 1 client.16551 shutdown 2024-03-28T18:11:47.411 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:11:47.408+0000 7ff53fb6e740 1 -- 172.21.15.22:0/1382672085 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5644adc51460 msgr2=0x5644adcbbc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:11:47.411 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:11:47.408+0000 7ff53fb6e740 1 --2- 172.21.15.22:0/1382672085 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5644adc51460 0x5644adcbbc50 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7ff5300a3bc0 tx=0x7ff530000f40 comp rx=0 tx=0).stop 2024-03-28T18:11:47.411 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:11:47.408+0000 7ff53fb6e740 1 -- 172.21.15.22:0/1382672085 shutdown_connections 2024-03-28T18:11:47.411 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:11:47.408+0000 7ff53fb6e740 1 --2- 172.21.15.22:0/1382672085 >> [v2:172.21.15.79:6838/2441542443,v1:172.21.15.79:6839/2441542443] conn(0x5644adcbd6f0 0x5644adcbfb10 crc :-1 s=READY pgs=133 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:11:47.412 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:11:47.408+0000 7ff53fb6e740 1 --2- 172.21.15.22:0/1382672085 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5644adc51460 0x5644adcbbc50 unknown :-1 s=CLOSED pgs=710 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:11:47.412 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:11:47.408+0000 7ff53fb6e740 1 --2- 172.21.15.22:0/1382672085 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5644adc48dd0 0x5644adc4ada0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:11:47.412 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:11:47.408+0000 7ff53fb6e740 1 --2- 172.21.15.22:0/1382672085 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5644adc480f0 0x5644adc4a860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:11:47.412 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:11:47.408+0000 7ff517e00640 0 client.16551 ms_handle_reset on v2:172.21.15.79:6838/2441542443 2024-03-28T18:11:47.412 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:11:47.408+0000 7ff53fb6e740 1 -- 172.21.15.22:0/1382672085 >> 172.21.15.22:0/1382672085 conn(0x5644adc0f200 msgr2=0x5644adc4cc40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:11:47.412 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:11:47.408+0000 7ff53fb6e740 1 -- 172.21.15.22:0/1382672085 shutdown_connections 2024-03-28T18:11:47.412 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:11:47.408+0000 7ff53fb6e740 1 -- 172.21.15.22:0/1382672085 wait complete. 2024-03-28T18:11:47.412 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:11:47.408+0000 7ff53fb6e740 20 client.16551 trim_cache size 0 max 16384 2024-03-28T18:11:47.412 INFO:teuthology.orchestra.run.smithi022.stdout:{ 2024-03-28T18:11:47.412 INFO:teuthology.orchestra.run.smithi022.stdout: "status": "IDLE (waiting for more scrubs)", 2024-03-28T18:11:47.412 INFO:teuthology.orchestra.run.smithi022.stdout: "scrubs": { 2024-03-28T18:11:47.413 INFO:teuthology.orchestra.run.smithi022.stdout: "4bbbd61d-29a3-4bf6-a4f2-d5783d585de3": { 2024-03-28T18:11:47.413 INFO:teuthology.orchestra.run.smithi022.stdout: "path": "/", 2024-03-28T18:11:47.413 INFO:teuthology.orchestra.run.smithi022.stdout: "tag": "4bbbd61d-29a3-4bf6-a4f2-d5783d585de3", 2024-03-28T18:11:47.413 INFO:teuthology.orchestra.run.smithi022.stdout: "options": "recursive,force" 2024-03-28T18:11:47.413 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-03-28T18:11:47.413 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-03-28T18:11:47.413 INFO:teuthology.orchestra.run.smithi022.stdout:} 2024-03-28T18:11:47.414 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:11:47.409+0000 7ff537200640 1 -- 172.21.15.22:0/461233731 >> [v2:172.21.15.22:6800/1357365082,v1:172.21.15.22:6801/1357365082] conn(0x7ff51c0a2970 msgr2=0x7ff51c0a4e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:11:47.414 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:11:47.409+0000 7ff537200640 1 --2- 172.21.15.22:0/461233731 >> [v2:172.21.15.22:6800/1357365082,v1:172.21.15.22:6801/1357365082] conn(0x7ff51c0a2970 0x7ff51c0a4e30 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x5644adc49320 tx=0x7ff52800b7d0 comp rx=0 tx=0).stop 2024-03-28T18:11:47.414 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:11:47.409+0000 7ff537200640 1 -- 172.21.15.22:0/461233731 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff5381a2cc0 msgr2=0x7ff5381a3170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:11:47.414 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:11:47.409+0000 7ff537200640 1 --2- 172.21.15.22:0/461233731 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff5381a2cc0 0x7ff5381a3170 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7ff53000e960 tx=0x7ff53000ee20 comp rx=0 tx=0).stop 2024-03-28T18:11:47.414 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:11:47.409+0000 7ff537200640 1 -- 172.21.15.22:0/461233731 shutdown_connections 2024-03-28T18:11:47.414 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:11:47.409+0000 7ff537200640 1 --2- 172.21.15.22:0/461233731 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff5381a2cc0 0x7ff5381a3170 unknown :-1 s=CLOSED pgs=706 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:11:47.414 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:11:47.409+0000 7ff537200640 1 --2- 172.21.15.22:0/461233731 >> [v2:172.21.15.22:6800/1357365082,v1:172.21.15.22:6801/1357365082] conn(0x7ff51c0a2970 0x7ff51c0a4e30 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:11:47.414 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:11:47.409+0000 7ff537200640 1 --2- 172.21.15.22:0/461233731 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ff53810cea0 0x7ff53819e940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:11:47.414 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:11:47.409+0000 7ff537200640 1 --2- 172.21.15.22:0/461233731 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff538107e50 0x7ff53819e400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:11:47.414 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:11:47.409+0000 7ff537200640 1 -- 172.21.15.22:0/461233731 >> 172.21.15.22:0/461233731 conn(0x7ff5380774c0 msgr2=0x7ff538079110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:11:47.414 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:11:47.409+0000 7ff537200640 1 -- 172.21.15.22:0/461233731 shutdown_connections 2024-03-28T18:11:47.414 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:11:47.409+0000 7ff537200640 1 -- 172.21.15.22:0/461233731 wait complete. 2024-03-28T18:11:47.430 INFO:tasks.cephfs.filesystem:scrub status for tag:4bbbd61d-29a3-4bf6-a4f2-d5783d585de3 - {'path': '/', 'tag': '4bbbd61d-29a3-4bf6-a4f2-d5783d585de3', 'options': 'recursive,force'} 2024-03-28T18:11:48.925 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:11:48 smithi134 ceph-mon[35037]: pgmap v135: 97 pgs: 97 active+clean; 117 MiB data, 800 MiB used, 1.0 TiB / 1.0 TiB avail; 91 KiB/s rd, 3.9 KiB/s wr, 22 op/s 2024-03-28T18:11:48.925 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:11:48 smithi134 ceph-mon[35037]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T18:11:48.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:11:48 smithi079 ceph-mon[35312]: pgmap v135: 97 pgs: 97 active+clean; 117 MiB data, 800 MiB used, 1.0 TiB / 1.0 TiB avail; 91 KiB/s rd, 3.9 KiB/s wr, 22 op/s 2024-03-28T18:11:48.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:11:48 smithi079 ceph-mon[35312]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T18:11:49.032 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:11:48 smithi022 ceph-mon[31985]: pgmap v135: 97 pgs: 97 active+clean; 117 MiB data, 800 MiB used, 1.0 TiB / 1.0 TiB avail; 91 KiB/s rd, 3.9 KiB/s wr, 22 op/s 2024-03-28T18:11:49.032 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:11:48 smithi022 ceph-mon[31985]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T18:11:49.817 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:11:49 smithi134 systemd[1]: ceph-fd220748-ed2b-11ee-95d0-87774f69a715@mds.f.service: Scheduled restart job, restart counter is at 3. 2024-03-28T18:11:49.817 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:11:49 smithi134 systemd[1]: Stopped Ceph mds.f for fd220748-ed2b-11ee-95d0-87774f69a715. 2024-03-28T18:11:49.817 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:11:49 smithi134 systemd[1]: ceph-fd220748-ed2b-11ee-95d0-87774f69a715@mds.f.service: Consumed 1.812s CPU time. 2024-03-28T18:11:49.817 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:11:49 smithi134 systemd[1]: Starting Ceph mds.f for fd220748-ed2b-11ee-95d0-87774f69a715... 2024-03-28T18:11:50.141 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:11:49 smithi134 podman[70293]: 2024-03-28 18:11:49.72167975 +0000 UTC m=+0.044876849 image pull c664afe56bb01db4f9afa9532c20598e201e664f2e9ba2be50db0d52674a26d2 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc 2024-03-28T18:11:50.141 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:11:49 smithi134 podman[70293]: 2024-03-28 18:11:49.837389116 +0000 UTC m=+0.160586201 container create b193fd8b1c5f7f9ea7a2b686fef5ccc3478f80f9db24dc449834c386a4cbcc79 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f, io.buildah.version=1.33.5, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, org.label-schema.build-date=20240319, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2) 2024-03-28T18:11:50.425 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:11:50 smithi134 podman[70293]: 2024-03-28 18:11:50.141370694 +0000 UTC m=+0.464567797 container init b193fd8b1c5f7f9ea7a2b686fef5ccc3478f80f9db24dc449834c386a4cbcc79 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , ceph=True, GIT_BRANCH=HEAD, GIT_CLEAN=True, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, org.label-schema.schema-version=1.0, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.license=GPLv2, org.label-schema.build-date=20240319, CEPH_POINT_RELEASE=) 2024-03-28T18:11:50.425 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:11:50 smithi134 podman[70293]: 2024-03-28 18:11:50.152117284 +0000 UTC m=+0.475314372 container start b193fd8b1c5f7f9ea7a2b686fef5ccc3478f80f9db24dc449834c386a4cbcc79 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_CLEAN=True, maintainer=Guillaume Abrioux , RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, org.label-schema.license=GPLv2, org.label-schema.build-date=20240319, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, ceph=True, CEPH_POINT_RELEASE=) 2024-03-28T18:11:50.425 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:11:50 smithi134 ceph-mds[70323]: -- 172.21.15.134:0/523005239 <== mon.1 v2:172.21.15.134:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x561e61f17a40 con 0x561e61eeb000 2024-03-28T18:11:50.425 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:11:50 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[70308]: starting mds.f at 2024-03-28T18:11:50.425 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:11:50 smithi134 bash[70293]: b193fd8b1c5f7f9ea7a2b686fef5ccc3478f80f9db24dc449834c386a4cbcc79 2024-03-28T18:11:50.426 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:11:50 smithi134 systemd[1]: Started Ceph mds.f for fd220748-ed2b-11ee-95d0-87774f69a715. 2024-03-28T18:11:50.926 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:11:50 smithi134 ceph-mon[35037]: pgmap v136: 97 pgs: 97 active+clean; 117 MiB data, 800 MiB used, 1.0 TiB / 1.0 TiB avail; 92 KiB/s rd, 3.9 KiB/s wr, 25 op/s 2024-03-28T18:11:50.926 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:11:50 smithi134 ceph-mon[35037]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T18:11:50.926 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:11:50 smithi134 ceph-mon[35037]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T18:11:50.926 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:11:50 smithi134 ceph-mon[35037]: from='mgr.15522 ' entity='mgr.x' 2024-03-28T18:11:50.926 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:11:50 smithi134 ceph-mon[35037]: daemon mds.f restarted 2024-03-28T18:11:50.926 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:11:50 smithi134 ceph-mon[35037]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-03-28T18:11:50.926 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:11:50 smithi134 ceph-mon[35037]: osdmap e99: 12 total, 12 up, 12 in 2024-03-28T18:11:50.926 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:11:50 smithi134 ceph-mon[35037]: mds.? [v2:172.21.15.134:6834/2842919302,v1:172.21.15.134:6835/2842919302] up:boot 2024-03-28T18:11:50.926 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:11:50 smithi134 ceph-mon[35037]: Standby daemon mds.h assigned to filesystem cephfs as rank 1 2024-03-28T18:11:50.926 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:11:50 smithi134 ceph-mon[35037]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-03-28T18:11:50.927 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:11:50 smithi134 ceph-mon[35037]: fsmap cephfs:4/5 {0=b=up:active,2=e=up:active,3=k=up:active,4=i=up:active} 5 up:standby-replay 3 up:standby, 1 failed 2024-03-28T18:11:50.927 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:11:50 smithi134 ceph-mon[35037]: fsmap cephfs:5/5 4 up:standby-replay 3 up:standby 1 up:replay 4 up:active 2024-03-28T18:11:50.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:11:50 smithi079 ceph-mon[35312]: pgmap v136: 97 pgs: 97 active+clean; 117 MiB data, 800 MiB used, 1.0 TiB / 1.0 TiB avail; 92 KiB/s rd, 3.9 KiB/s wr, 25 op/s 2024-03-28T18:11:50.933 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:11:50 smithi079 ceph-mon[35312]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T18:11:50.933 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:11:50 smithi079 ceph-mon[35312]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T18:11:50.933 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:11:50 smithi079 ceph-mon[35312]: from='mgr.15522 ' entity='mgr.x' 2024-03-28T18:11:50.933 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:11:50 smithi079 ceph-mon[35312]: daemon mds.f restarted 2024-03-28T18:11:50.933 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:11:50 smithi079 ceph-mon[35312]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-03-28T18:11:50.933 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:11:50 smithi079 ceph-mon[35312]: osdmap e99: 12 total, 12 up, 12 in 2024-03-28T18:11:50.933 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:11:50 smithi079 ceph-mon[35312]: mds.? [v2:172.21.15.134:6834/2842919302,v1:172.21.15.134:6835/2842919302] up:boot 2024-03-28T18:11:50.933 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:11:50 smithi079 ceph-mon[35312]: Standby daemon mds.h assigned to filesystem cephfs as rank 1 2024-03-28T18:11:50.933 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:11:50 smithi079 ceph-mon[35312]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-03-28T18:11:50.933 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:11:50 smithi079 ceph-mon[35312]: fsmap cephfs:4/5 {0=b=up:active,2=e=up:active,3=k=up:active,4=i=up:active} 5 up:standby-replay 3 up:standby, 1 failed 2024-03-28T18:11:50.933 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:11:50 smithi079 ceph-mon[35312]: fsmap cephfs:5/5 4 up:standby-replay 3 up:standby 1 up:replay 4 up:active 2024-03-28T18:11:51.032 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:11:50 smithi022 ceph-mon[31985]: pgmap v136: 97 pgs: 97 active+clean; 117 MiB data, 800 MiB used, 1.0 TiB / 1.0 TiB avail; 92 KiB/s rd, 3.9 KiB/s wr, 25 op/s 2024-03-28T18:11:51.032 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:11:50 smithi022 ceph-mon[31985]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T18:11:51.032 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:11:50 smithi022 ceph-mon[31985]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T18:11:51.032 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:11:50 smithi022 ceph-mon[31985]: from='mgr.15522 ' entity='mgr.x' 2024-03-28T18:11:51.032 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:11:50 smithi022 ceph-mon[31985]: daemon mds.f restarted 2024-03-28T18:11:51.033 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:11:50 smithi022 ceph-mon[31985]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-03-28T18:11:51.033 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:11:50 smithi022 ceph-mon[31985]: osdmap e99: 12 total, 12 up, 12 in 2024-03-28T18:11:51.033 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:11:50 smithi022 ceph-mon[31985]: mds.? [v2:172.21.15.134:6834/2842919302,v1:172.21.15.134:6835/2842919302] up:boot 2024-03-28T18:11:51.033 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:11:50 smithi022 ceph-mon[31985]: Standby daemon mds.h assigned to filesystem cephfs as rank 1 2024-03-28T18:11:51.033 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:11:50 smithi022 ceph-mon[31985]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-03-28T18:11:51.033 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:11:50 smithi022 ceph-mon[31985]: fsmap cephfs:4/5 {0=b=up:active,2=e=up:active,3=k=up:active,4=i=up:active} 5 up:standby-replay 3 up:standby, 1 failed 2024-03-28T18:11:51.033 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:11:50 smithi022 ceph-mon[31985]: fsmap cephfs:5/5 4 up:standby-replay 3 up:standby 1 up:replay 4 up:active 2024-03-28T18:11:52.925 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:11:52 smithi134 ceph-mon[35037]: pgmap v138: 97 pgs: 97 active+clean; 117 MiB data, 800 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:11:52.925 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:11:52 smithi134 ceph-mon[35037]: mds.? [v2:172.21.15.79:6836/3952268652,v1:172.21.15.79:6837/3952268652] up:resolve 2024-03-28T18:11:52.925 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:11:52 smithi134 ceph-mon[35037]: fsmap cephfs:5/5 4 up:standby-replay 3 up:standby 1 up:resolve 4 up:active 2024-03-28T18:11:52.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:11:52 smithi079 ceph-mon[35312]: pgmap v138: 97 pgs: 97 active+clean; 117 MiB data, 800 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:11:52.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:11:52 smithi079 ceph-mon[35312]: mds.? [v2:172.21.15.79:6836/3952268652,v1:172.21.15.79:6837/3952268652] up:resolve 2024-03-28T18:11:52.933 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:11:52 smithi079 ceph-mon[35312]: fsmap cephfs:5/5 4 up:standby-replay 3 up:standby 1 up:resolve 4 up:active 2024-03-28T18:11:53.032 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:11:52 smithi022 ceph-mon[31985]: pgmap v138: 97 pgs: 97 active+clean; 117 MiB data, 800 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:11:53.032 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:11:52 smithi022 ceph-mon[31985]: mds.? [v2:172.21.15.79:6836/3952268652,v1:172.21.15.79:6837/3952268652] up:resolve 2024-03-28T18:11:53.032 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:11:52 smithi022 ceph-mon[31985]: fsmap cephfs:5/5 4 up:standby-replay 3 up:standby 1 up:resolve 4 up:active 2024-03-28T18:11:53.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:11:53 smithi079 ceph-mon[35312]: mds.? [v2:172.21.15.79:6836/3952268652,v1:172.21.15.79:6837/3952268652] up:reconnect 2024-03-28T18:11:53.933 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:11:53 smithi079 ceph-mon[35312]: fsmap cephfs:5/5 4 up:standby-replay 3 up:standby 1 up:reconnect 4 up:active 2024-03-28T18:11:53.933 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:11:53 smithi079 ceph-mon[35312]: reconnect by client.35222 172.21.15.22:0/3729116781 after 0.00399991 2024-03-28T18:11:54.032 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:11:53 smithi022 ceph-mon[31985]: mds.? [v2:172.21.15.79:6836/3952268652,v1:172.21.15.79:6837/3952268652] up:reconnect 2024-03-28T18:11:54.032 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:11:53 smithi022 ceph-mon[31985]: fsmap cephfs:5/5 4 up:standby-replay 3 up:standby 1 up:reconnect 4 up:active 2024-03-28T18:11:54.032 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:11:53 smithi022 ceph-mon[31985]: reconnect by client.35222 172.21.15.22:0/3729116781 after 0.00399991 2024-03-28T18:11:54.174 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:11:53 smithi134 ceph-mon[35037]: mds.? [v2:172.21.15.79:6836/3952268652,v1:172.21.15.79:6837/3952268652] up:reconnect 2024-03-28T18:11:54.175 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:11:53 smithi134 ceph-mon[35037]: fsmap cephfs:5/5 4 up:standby-replay 3 up:standby 1 up:reconnect 4 up:active 2024-03-28T18:11:54.175 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:11:53 smithi134 ceph-mon[35037]: reconnect by client.35222 172.21.15.22:0/3729116781 after 0.00399991 2024-03-28T18:11:55.032 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:11:54 smithi022 ceph-mon[31985]: pgmap v139: 97 pgs: 97 active+clean; 117 MiB data, 800 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:11:55.032 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:11:54 smithi022 ceph-mon[31985]: mds.? [v2:172.21.15.79:6836/3952268652,v1:172.21.15.79:6837/3952268652] up:rejoin 2024-03-28T18:11:55.032 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:11:54 smithi022 ceph-mon[31985]: fsmap cephfs:5/5 4 up:standby-replay 3 up:standby 1 up:rejoin 4 up:active 2024-03-28T18:11:55.032 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:11:54 smithi022 ceph-mon[31985]: daemon mds.h is now active in filesystem cephfs as rank 1 2024-03-28T18:11:55.174 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:11:54 smithi134 ceph-mon[35037]: pgmap v139: 97 pgs: 97 active+clean; 117 MiB data, 800 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:11:55.175 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:11:54 smithi134 ceph-mon[35037]: mds.? [v2:172.21.15.79:6836/3952268652,v1:172.21.15.79:6837/3952268652] up:rejoin 2024-03-28T18:11:55.175 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:11:54 smithi134 ceph-mon[35037]: fsmap cephfs:5/5 4 up:standby-replay 3 up:standby 1 up:rejoin 4 up:active 2024-03-28T18:11:55.175 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:11:54 smithi134 ceph-mon[35037]: daemon mds.h is now active in filesystem cephfs as rank 1 2024-03-28T18:11:55.183 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:11:54 smithi079 ceph-mon[35312]: pgmap v139: 97 pgs: 97 active+clean; 117 MiB data, 800 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:11:55.183 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:11:54 smithi079 ceph-mon[35312]: mds.? [v2:172.21.15.79:6836/3952268652,v1:172.21.15.79:6837/3952268652] up:rejoin 2024-03-28T18:11:55.183 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:11:54 smithi079 ceph-mon[35312]: fsmap cephfs:5/5 4 up:standby-replay 3 up:standby 1 up:rejoin 4 up:active 2024-03-28T18:11:55.183 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:11:54 smithi079 ceph-mon[35312]: daemon mds.h is now active in filesystem cephfs as rank 1 2024-03-28T18:11:55.183 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:11:54 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69460]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-2145-g3be9a5c9/rpm/el8/BUILD/ceph-19.0.0-2145-g3be9a5c9/src/mds/Locker.cc: In function 'void Locker::set_xlocks_done(MutationImpl*, bool)' thread 7f1143600700 time 2024-03-28T18:11:54.764776+0000 2024-03-28T18:11:55.183 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:11:54 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69460]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-2145-g3be9a5c9/rpm/el8/BUILD/ceph-19.0.0-2145-g3be9a5c9/src/mds/Locker.cc: 626: FAILED ceph_assert(obj->is_auth()) 2024-03-28T18:11:55.183 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:11:54 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69460]: ceph version 19.0.0-2145-g3be9a5c9 (3be9a5c9bc793e11e2800a8c0c696e8b46742033) squid (dev) 2024-03-28T18:11:55.184 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:11:54 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69460]: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x135) [0x7f114ef834a1] 2024-03-28T18:11:55.184 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:11:54 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69460]: 2: /usr/lib64/ceph/libceph-common.so.2(+0x2d2667) [0x7f114ef83667] 2024-03-28T18:11:55.184 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:11:54 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69460]: 3: (Locker::set_xlocks_done(MutationImpl*, bool)+0x493) [0x563335a7b223] 2024-03-28T18:11:55.184 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:11:54 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69460]: 4: (Server::early_reply(boost::intrusive_ptr const&, CInode*, CDentry*)+0x3e6) [0x5633358f55d6] 2024-03-28T18:11:55.184 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:11:54 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69460]: 5: (Server::journal_and_reply(boost::intrusive_ptr const&, CInode*, CDentry*, LogEvent*, MDSLogContextBase*)+0xc1) [0x5633358f5da1] 2024-03-28T18:11:55.184 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:11:54 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69460]: 6: (Server::handle_client_mkdir(boost::intrusive_ptr const&)+0x5ba) [0x56333592b6aa] 2024-03-28T18:11:55.184 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:11:54 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69460]: 7: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0x733) [0x563335948233] 2024-03-28T18:11:55.184 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:11:54 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69460]: 8: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x56333594a197] 2024-03-28T18:11:55.184 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:11:54 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69460]: 9: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x56333584911b] 2024-03-28T18:11:55.184 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:11:54 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69460]: 10: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x5633358496bc] 2024-03-28T18:11:55.184 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:11:54 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69460]: 11: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bf) [0x5633358331ff] 2024-03-28T18:11:55.184 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:11:54 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69460]: 12: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x478) [0x7f114f215488] 2024-03-28T18:11:55.184 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:11:54 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69460]: 13: (DispatchQueue::entry()+0x51f) [0x7f114f21260f] 2024-03-28T18:11:55.184 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:11:54 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69460]: 14: (DispatchQueue::DispatchThread::entry()+0x11) [0x7f114f2e5e81] 2024-03-28T18:11:55.184 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:11:54 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69460]: 15: /lib64/libpthread.so.0(+0x81ca) [0x7f114dd001ca] 2024-03-28T18:11:55.185 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:11:54 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69460]: 16: clone() 2024-03-28T18:11:55.185 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:11:54 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69460]: 2024-03-28T18:11:54.766+0000 7f1143600700 -1 /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-2145-g3be9a5c9/rpm/el8/BUILD/ceph-19.0.0-2145-g3be9a5c9/src/mds/Locker.cc: In function 'void Locker::set_xlocks_done(MutationImpl*, bool)' thread 7f1143600700 time 2024-03-28T18:11:54.764776+0000 2024-03-28T18:11:55.185 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:11:54 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69460]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-2145-g3be9a5c9/rpm/el8/BUILD/ceph-19.0.0-2145-g3be9a5c9/src/mds/Locker.cc: 626: FAILED ceph_assert(obj->is_auth()) 2024-03-28T18:11:55.185 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:11:54 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69460]: 2024-03-28T18:11:55.185 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:11:54 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69460]: ceph version 19.0.0-2145-g3be9a5c9 (3be9a5c9bc793e11e2800a8c0c696e8b46742033) squid (dev) 2024-03-28T18:11:55.185 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:11:54 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69460]: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x135) [0x7f114ef834a1] 2024-03-28T18:11:55.186 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:11:54 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69460]: 2: /usr/lib64/ceph/libceph-common.so.2(+0x2d2667) [0x7f114ef83667] 2024-03-28T18:11:55.186 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:11:54 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69460]: 3: (Locker::set_xlocks_done(MutationImpl*, bool)+0x493) [0x563335a7b223] 2024-03-28T18:11:55.186 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:11:54 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69460]: 4: (Server::early_reply(boost::intrusive_ptr const&, CInode*, CDentry*)+0x3e6) [0x5633358f55d6] 2024-03-28T18:11:55.186 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:11:54 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69460]: 5: (Server::journal_and_reply(boost::intrusive_ptr const&, CInode*, CDentry*, LogEvent*, MDSLogContextBase*)+0xc1) [0x5633358f5da1] 2024-03-28T18:11:55.186 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:11:54 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69460]: 6: (Server::handle_client_mkdir(boost::intrusive_ptr const&)+0x5ba) [0x56333592b6aa] 2024-03-28T18:11:55.186 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:11:54 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69460]: 7: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0x733) [0x563335948233] 2024-03-28T18:11:55.186 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:11:54 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69460]: 8: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x56333594a197] 2024-03-28T18:11:55.186 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:11:54 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69460]: 9: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x56333584911b] 2024-03-28T18:11:55.186 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:11:54 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69460]: 10: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x5633358496bc] 2024-03-28T18:11:55.186 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:11:54 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69460]: 11: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bf) [0x5633358331ff] 2024-03-28T18:11:55.186 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:11:54 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69460]: 12: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x478) [0x7f114f215488] 2024-03-28T18:11:55.186 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:11:54 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69460]: 13: (DispatchQueue::entry()+0x51f) [0x7f114f21260f] 2024-03-28T18:11:55.186 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:11:54 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69460]: 14: (DispatchQueue::DispatchThread::entry()+0x11) [0x7f114f2e5e81] 2024-03-28T18:11:55.189 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:11:54 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69460]: 15: /lib64/libpthread.so.0(+0x81ca) [0x7f114dd001ca] 2024-03-28T18:11:55.189 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:11:54 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69460]: 16: clone() 2024-03-28T18:11:55.189 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:11:54 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69460]: 2024-03-28T18:11:55.189 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:11:54 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69460]: *** Caught signal (Aborted) ** 2024-03-28T18:11:55.189 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:11:54 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69460]: in thread 7f1143600700 thread_name:ms_dispatch 2024-03-28T18:11:55.189 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:11:54 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69460]: ceph version 19.0.0-2145-g3be9a5c9 (3be9a5c9bc793e11e2800a8c0c696e8b46742033) squid (dev) 2024-03-28T18:11:55.189 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:11:54 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69460]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7f114dd0ad20] 2024-03-28T18:11:55.189 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:11:54 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69460]: 2: gsignal() 2024-03-28T18:11:55.189 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:11:54 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69460]: 3: abort() 2024-03-28T18:11:55.189 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:11:54 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69460]: 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x18f) [0x7f114ef834fb] 2024-03-28T18:11:55.190 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:11:54 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69460]: 5: /usr/lib64/ceph/libceph-common.so.2(+0x2d2667) [0x7f114ef83667] 2024-03-28T18:11:55.190 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:11:54 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69460]: 6: (Locker::set_xlocks_done(MutationImpl*, bool)+0x493) [0x563335a7b223] 2024-03-28T18:11:55.190 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:11:54 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69460]: 7: (Server::early_reply(boost::intrusive_ptr const&, CInode*, CDentry*)+0x3e6) [0x5633358f55d6] 2024-03-28T18:11:55.190 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:11:54 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69460]: 8: (Server::journal_and_reply(boost::intrusive_ptr const&, CInode*, CDentry*, LogEvent*, MDSLogContextBase*)+0xc1) [0x5633358f5da1] 2024-03-28T18:11:55.190 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:11:54 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69460]: 9: (Server::handle_client_mkdir(boost::intrusive_ptr const&)+0x5ba) [0x56333592b6aa] 2024-03-28T18:11:55.190 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:11:54 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69460]: 10: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0x733) [0x563335948233] 2024-03-28T18:11:55.190 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:11:54 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69460]: 11: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x56333594a197] 2024-03-28T18:11:55.190 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:11:54 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69460]: 12: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x56333584911b] 2024-03-28T18:11:55.190 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:11:54 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69460]: 13: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x5633358496bc] 2024-03-28T18:11:55.190 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:11:54 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69460]: 14: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bf) [0x5633358331ff] 2024-03-28T18:11:55.190 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:11:54 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69460]: 15: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x478) [0x7f114f215488] 2024-03-28T18:11:55.190 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:11:54 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69460]: 16: (DispatchQueue::entry()+0x51f) [0x7f114f21260f] 2024-03-28T18:11:55.190 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:11:54 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69460]: 17: (DispatchQueue::DispatchThread::entry()+0x11) [0x7f114f2e5e81] 2024-03-28T18:11:55.190 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:11:54 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69460]: 18: /lib64/libpthread.so.0(+0x81ca) [0x7f114dd001ca] 2024-03-28T18:11:55.190 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:11:54 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69460]: 19: clone() 2024-03-28T18:11:55.191 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:11:54 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69460]: 2024-03-28T18:11:54.769+0000 7f1143600700 -1 *** Caught signal (Aborted) ** 2024-03-28T18:11:55.191 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:11:54 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69460]: in thread 7f1143600700 thread_name:ms_dispatch 2024-03-28T18:11:55.191 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:11:54 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69460]: 2024-03-28T18:11:55.191 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:11:54 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69460]: ceph version 19.0.0-2145-g3be9a5c9 (3be9a5c9bc793e11e2800a8c0c696e8b46742033) squid (dev) 2024-03-28T18:11:55.191 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:11:54 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69460]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7f114dd0ad20] 2024-03-28T18:11:55.191 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:11:54 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69460]: 2: gsignal() 2024-03-28T18:11:55.191 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:11:54 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69460]: 3: abort() 2024-03-28T18:11:55.191 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:11:54 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69460]: 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x18f) [0x7f114ef834fb] 2024-03-28T18:11:55.191 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:11:54 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69460]: 5: /usr/lib64/ceph/libceph-common.so.2(+0x2d2667) [0x7f114ef83667] 2024-03-28T18:11:55.194 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:11:54 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69460]: 6: (Locker::set_xlocks_done(MutationImpl*, bool)+0x493) [0x563335a7b223] 2024-03-28T18:11:55.195 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:11:54 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69460]: 7: (Server::early_reply(boost::intrusive_ptr const&, CInode*, CDentry*)+0x3e6) [0x5633358f55d6] 2024-03-28T18:11:55.195 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:11:54 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69460]: 8: (Server::journal_and_reply(boost::intrusive_ptr const&, CInode*, CDentry*, LogEvent*, MDSLogContextBase*)+0xc1) [0x5633358f5da1] 2024-03-28T18:11:55.195 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:11:54 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69460]: 9: (Server::handle_client_mkdir(boost::intrusive_ptr const&)+0x5ba) [0x56333592b6aa] 2024-03-28T18:11:55.195 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:11:54 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69460]: 10: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0x733) [0x563335948233] 2024-03-28T18:11:55.195 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:11:54 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69460]: 11: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x56333594a197] 2024-03-28T18:11:55.195 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:11:54 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69460]: 12: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x56333584911b] 2024-03-28T18:11:55.195 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:11:54 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69460]: 13: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x5633358496bc] 2024-03-28T18:11:55.195 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:11:54 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69460]: 14: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bf) [0x5633358331ff] 2024-03-28T18:11:55.195 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:11:54 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69460]: 15: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x478) [0x7f114f215488] 2024-03-28T18:11:55.195 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:11:54 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69460]: 16: (DispatchQueue::entry()+0x51f) [0x7f114f21260f] 2024-03-28T18:11:55.195 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:11:54 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69460]: 17: (DispatchQueue::DispatchThread::entry()+0x11) [0x7f114f2e5e81] 2024-03-28T18:11:55.195 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:11:54 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69460]: 18: /lib64/libpthread.so.0(+0x81ca) [0x7f114dd001ca] 2024-03-28T18:11:55.195 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:11:54 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69460]: 19: clone() 2024-03-28T18:11:55.196 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:11:54 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69460]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-03-28T18:11:55.196 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:11:54 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69460]: 2024-03-28T18:11:55.196 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:11:54 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69460]: -1> 2024-03-28T18:11:54.766+0000 7f1143600700 -1 /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-2145-g3be9a5c9/rpm/el8/BUILD/ceph-19.0.0-2145-g3be9a5c9/src/mds/Locker.cc: In function 'void Locker::set_xlocks_done(MutationImpl*, bool)' thread 7f1143600700 time 2024-03-28T18:11:54.764776+0000 2024-03-28T18:11:55.196 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:11:54 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69460]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-2145-g3be9a5c9/rpm/el8/BUILD/ceph-19.0.0-2145-g3be9a5c9/src/mds/Locker.cc: 626: FAILED ceph_assert(obj->is_auth()) 2024-03-28T18:11:55.196 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:11:54 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69460]: 2024-03-28T18:11:55.196 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:11:54 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69460]: ceph version 19.0.0-2145-g3be9a5c9 (3be9a5c9bc793e11e2800a8c0c696e8b46742033) squid (dev) 2024-03-28T18:11:55.196 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:11:54 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69460]: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x135) [0x7f114ef834a1] 2024-03-28T18:11:55.196 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:11:54 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69460]: 2: /usr/lib64/ceph/libceph-common.so.2(+0x2d2667) [0x7f114ef83667] 2024-03-28T18:11:55.196 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:11:54 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69460]: 3: (Locker::set_xlocks_done(MutationImpl*, bool)+0x493) [0x563335a7b223] 2024-03-28T18:11:55.196 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:11:54 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69460]: 4: (Server::early_reply(boost::intrusive_ptr const&, CInode*, CDentry*)+0x3e6) [0x5633358f55d6] 2024-03-28T18:11:55.196 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:11:54 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69460]: 5: (Server::journal_and_reply(boost::intrusive_ptr const&, CInode*, CDentry*, LogEvent*, MDSLogContextBase*)+0xc1) [0x5633358f5da1] 2024-03-28T18:11:55.196 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:11:54 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69460]: 6: (Server::handle_client_mkdir(boost::intrusive_ptr const&)+0x5ba) [0x56333592b6aa] 2024-03-28T18:11:55.196 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:11:54 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69460]: 7: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0x733) [0x563335948233] 2024-03-28T18:11:55.196 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:11:54 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69460]: 8: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x56333594a197] 2024-03-28T18:11:55.196 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:11:54 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69460]: 9: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x56333584911b] 2024-03-28T18:11:55.197 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:11:54 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69460]: 10: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x5633358496bc] 2024-03-28T18:11:55.197 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:11:54 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69460]: 11: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bf) [0x5633358331ff] 2024-03-28T18:11:55.197 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:11:54 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69460]: 12: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x478) [0x7f114f215488] 2024-03-28T18:11:55.197 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:11:54 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69460]: 13: (DispatchQueue::entry()+0x51f) [0x7f114f21260f] 2024-03-28T18:11:55.197 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:11:54 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69460]: 14: (DispatchQueue::DispatchThread::entry()+0x11) [0x7f114f2e5e81] 2024-03-28T18:11:55.197 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:11:54 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69460]: 15: /lib64/libpthread.so.0(+0x81ca) [0x7f114dd001ca] 2024-03-28T18:11:55.197 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:11:54 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69460]: 16: clone() 2024-03-28T18:11:55.197 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:11:54 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69460]: 2024-03-28T18:11:55.197 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:11:54 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69460]: 0> 2024-03-28T18:11:54.769+0000 7f1143600700 -1 *** Caught signal (Aborted) ** 2024-03-28T18:11:55.197 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:11:54 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69460]: in thread 7f1143600700 thread_name:ms_dispatch 2024-03-28T18:11:55.197 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:11:54 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69460]: 2024-03-28T18:11:55.197 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:11:54 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69460]: ceph version 19.0.0-2145-g3be9a5c9 (3be9a5c9bc793e11e2800a8c0c696e8b46742033) squid (dev) 2024-03-28T18:11:55.197 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:11:54 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69460]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7f114dd0ad20] 2024-03-28T18:11:55.197 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:11:54 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69460]: 2: gsignal() 2024-03-28T18:11:55.197 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:11:54 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69460]: 3: abort() 2024-03-28T18:11:55.197 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:11:54 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69460]: 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x18f) [0x7f114ef834fb] 2024-03-28T18:11:55.198 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:11:54 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69460]: 5: /usr/lib64/ceph/libceph-common.so.2(+0x2d2667) [0x7f114ef83667] 2024-03-28T18:11:55.198 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:11:54 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69460]: 6: (Locker::set_xlocks_done(MutationImpl*, bool)+0x493) [0x563335a7b223] 2024-03-28T18:11:55.198 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:11:54 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69460]: 7: (Server::early_reply(boost::intrusive_ptr const&, CInode*, CDentry*)+0x3e6) [0x5633358f55d6] 2024-03-28T18:11:55.198 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:11:54 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69460]: 8: (Server::journal_and_reply(boost::intrusive_ptr const&, CInode*, CDentry*, LogEvent*, MDSLogContextBase*)+0xc1) [0x5633358f5da1] 2024-03-28T18:11:55.198 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:11:54 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69460]: 9: (Server::handle_client_mkdir(boost::intrusive_ptr const&)+0x5ba) [0x56333592b6aa] 2024-03-28T18:11:55.198 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:11:54 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69460]: 10: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0x733) [0x563335948233] 2024-03-28T18:11:55.198 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:11:54 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69460]: 11: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x56333594a197] 2024-03-28T18:11:55.198 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:11:54 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69460]: 12: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x56333584911b] 2024-03-28T18:11:55.198 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:11:54 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69460]: 13: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x5633358496bc] 2024-03-28T18:11:55.198 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:11:54 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69460]: 14: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bf) [0x5633358331ff] 2024-03-28T18:11:55.198 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:11:54 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69460]: 15: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x478) [0x7f114f215488] 2024-03-28T18:11:55.198 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:11:54 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69460]: 16: (DispatchQueue::entry()+0x51f) [0x7f114f21260f] 2024-03-28T18:11:55.198 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:11:54 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69460]: 17: (DispatchQueue::DispatchThread::entry()+0x11) [0x7f114f2e5e81] 2024-03-28T18:11:55.198 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:11:54 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69460]: 18: /lib64/libpthread.so.0(+0x81ca) [0x7f114dd001ca] 2024-03-28T18:11:55.199 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:11:54 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69460]: 19: clone() 2024-03-28T18:11:55.199 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:11:54 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69460]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-03-28T18:11:55.199 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:11:54 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69460]: 2024-03-28T18:11:55.199 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:11:54 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69460]: -1> 2024-03-28T18:11:54.766+0000 7f1143600700 -1 /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-2145-g3be9a5c9/rpm/el8/BUILD/ceph-19.0.0-2145-g3be9a5c9/src/mds/Locker.cc: In function 'void Locker::set_xlocks_done(MutationImpl*, bool)' thread 7f1143600700 time 2024-03-28T18:11:54.764776+0000 2024-03-28T18:11:55.199 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:11:54 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69460]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-2145-g3be9a5c9/rpm/el8/BUILD/ceph-19.0.0-2145-g3be9a5c9/src/mds/Locker.cc: 626: FAILED ceph_assert(obj->is_auth()) 2024-03-28T18:11:55.199 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:11:54 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69460]: 2024-03-28T18:11:55.199 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:11:54 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69460]: ceph version 19.0.0-2145-g3be9a5c9 (3be9a5c9bc793e11e2800a8c0c696e8b46742033) squid (dev) 2024-03-28T18:11:55.199 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:11:54 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69460]: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x135) [0x7f114ef834a1] 2024-03-28T18:11:55.199 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:11:54 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69460]: 2: /usr/lib64/ceph/libceph-common.so.2(+0x2d2667) [0x7f114ef83667] 2024-03-28T18:11:55.199 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:11:54 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69460]: 3: (Locker::set_xlocks_done(MutationImpl*, bool)+0x493) [0x563335a7b223] 2024-03-28T18:11:55.199 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:11:54 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69460]: 4: (Server::early_reply(boost::intrusive_ptr const&, CInode*, CDentry*)+0x3e6) [0x5633358f55d6] 2024-03-28T18:11:55.199 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:11:54 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69460]: 5: (Server::journal_and_reply(boost::intrusive_ptr const&, CInode*, CDentry*, LogEvent*, MDSLogContextBase*)+0xc1) [0x5633358f5da1] 2024-03-28T18:11:55.199 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:11:54 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69460]: 6: (Server::handle_client_mkdir(boost::intrusive_ptr const&)+0x5ba) [0x56333592b6aa] 2024-03-28T18:11:55.199 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:11:54 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69460]: 7: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0x733) [0x563335948233] 2024-03-28T18:11:55.199 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:11:54 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69460]: 8: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x56333594a197] 2024-03-28T18:11:55.200 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:11:54 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69460]: 9: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x56333584911b] 2024-03-28T18:11:55.200 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:11:54 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69460]: 10: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x5633358496bc] 2024-03-28T18:11:55.200 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:11:54 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69460]: 11: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bf) [0x5633358331ff] 2024-03-28T18:11:55.200 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:11:54 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69460]: 12: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x478) [0x7f114f215488] 2024-03-28T18:11:55.200 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:11:54 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69460]: 13: (DispatchQueue::entry()+0x51f) [0x7f114f21260f] 2024-03-28T18:11:55.200 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:11:54 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69460]: 14: (DispatchQueue::DispatchThread::entry()+0x11) [0x7f114f2e5e81] 2024-03-28T18:11:55.200 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:11:54 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69460]: 15: /lib64/libpthread.so.0(+0x81ca) [0x7f114dd001ca] 2024-03-28T18:11:55.200 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:11:54 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69460]: 16: clone() 2024-03-28T18:11:55.200 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:11:54 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69460]: 2024-03-28T18:11:55.200 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:11:54 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69460]: 0> 2024-03-28T18:11:54.769+0000 7f1143600700 -1 *** Caught signal (Aborted) ** 2024-03-28T18:11:55.200 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:11:54 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69460]: in thread 7f1143600700 thread_name:ms_dispatch 2024-03-28T18:11:55.200 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:11:54 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69460]: 2024-03-28T18:11:55.200 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:11:54 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69460]: ceph version 19.0.0-2145-g3be9a5c9 (3be9a5c9bc793e11e2800a8c0c696e8b46742033) squid (dev) 2024-03-28T18:11:55.200 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:11:54 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69460]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7f114dd0ad20] 2024-03-28T18:11:55.200 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:11:54 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69460]: 2: gsignal() 2024-03-28T18:11:55.200 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:11:54 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69460]: 3: abort() 2024-03-28T18:11:55.201 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:11:54 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69460]: 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x18f) [0x7f114ef834fb] 2024-03-28T18:11:55.201 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:11:54 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69460]: 5: /usr/lib64/ceph/libceph-common.so.2(+0x2d2667) [0x7f114ef83667] 2024-03-28T18:11:55.201 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:11:54 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69460]: 6: (Locker::set_xlocks_done(MutationImpl*, bool)+0x493) [0x563335a7b223] 2024-03-28T18:11:55.201 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:11:54 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69460]: 7: (Server::early_reply(boost::intrusive_ptr const&, CInode*, CDentry*)+0x3e6) [0x5633358f55d6] 2024-03-28T18:11:55.201 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:11:54 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69460]: 8: (Server::journal_and_reply(boost::intrusive_ptr const&, CInode*, CDentry*, LogEvent*, MDSLogContextBase*)+0xc1) [0x5633358f5da1] 2024-03-28T18:11:55.201 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:11:54 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69460]: 9: (Server::handle_client_mkdir(boost::intrusive_ptr const&)+0x5ba) [0x56333592b6aa] 2024-03-28T18:11:55.201 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:11:54 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69460]: 10: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0x733) [0x563335948233] 2024-03-28T18:11:55.201 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:11:54 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69460]: 11: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x56333594a197] 2024-03-28T18:11:55.201 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:11:54 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69460]: 12: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x56333584911b] 2024-03-28T18:11:55.201 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:11:54 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69460]: 13: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x5633358496bc] 2024-03-28T18:11:55.201 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:11:54 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69460]: 14: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bf) [0x5633358331ff] 2024-03-28T18:11:55.201 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:11:54 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69460]: 15: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x478) [0x7f114f215488] 2024-03-28T18:11:55.201 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:11:54 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69460]: 16: (DispatchQueue::entry()+0x51f) [0x7f114f21260f] 2024-03-28T18:11:55.201 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:11:54 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69460]: 17: (DispatchQueue::DispatchThread::entry()+0x11) [0x7f114f2e5e81] 2024-03-28T18:11:55.201 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:11:54 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69460]: 18: /lib64/libpthread.so.0(+0x81ca) [0x7f114dd001ca] 2024-03-28T18:11:55.201 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:11:54 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69460]: 19: clone() 2024-03-28T18:11:55.202 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:11:54 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69460]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-03-28T18:11:55.202 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:11:54 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69460]: 2024-03-28T18:11:55.202 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:11:54 smithi079 podman[69627]: 2024-03-28 18:11:54.924861121 +0000 UTC m=+0.044486290 container died aa05077e4c57ade8832dc601ab80f5d0cb4bfa5115afa7c28cb3f99ecaada10b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240319, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_BRANCH=HEAD, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS) 2024-03-28T18:11:55.933 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:11:55 smithi079 ceph-mon[35312]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-03-28T18:11:55.933 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:11:55 smithi079 ceph-mon[35312]: Cluster is now healthy 2024-03-28T18:11:55.933 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:11:55 smithi079 ceph-mon[35312]: mds.? [v2:172.21.15.79:6836/3952268652,v1:172.21.15.79:6837/3952268652] up:active 2024-03-28T18:11:55.933 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:11:55 smithi079 ceph-mon[35312]: fsmap cephfs:5 4 up:standby-replay 3 up:standby 5 up:active 2024-03-28T18:11:55.933 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:11:55 smithi079 ceph-mon[35312]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-28T18:11:55.933 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:11:55 smithi079 podman[69627]: 2024-03-28 18:11:55.551052777 +0000 UTC m=+0.670677930 container remove aa05077e4c57ade8832dc601ab80f5d0cb4bfa5115afa7c28cb3f99ecaada10b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240319, maintainer=Guillaume Abrioux , RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30) 2024-03-28T18:11:55.933 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:11:55 smithi079 systemd[1]: ceph-fd220748-ed2b-11ee-95d0-87774f69a715@mds.h.service: Main process exited, code=exited, status=134/n/a 2024-03-28T18:11:56.032 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:11:55 smithi022 ceph-mon[31985]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-03-28T18:11:56.032 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:11:55 smithi022 ceph-mon[31985]: Cluster is now healthy 2024-03-28T18:11:56.032 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:11:55 smithi022 ceph-mon[31985]: mds.? [v2:172.21.15.79:6836/3952268652,v1:172.21.15.79:6837/3952268652] up:active 2024-03-28T18:11:56.032 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:11:55 smithi022 ceph-mon[31985]: fsmap cephfs:5 4 up:standby-replay 3 up:standby 5 up:active 2024-03-28T18:11:56.032 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:11:55 smithi022 ceph-mon[31985]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-28T18:11:56.175 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:11:55 smithi134 ceph-mon[35037]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-03-28T18:11:56.175 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:11:55 smithi134 ceph-mon[35037]: Cluster is now healthy 2024-03-28T18:11:56.175 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:11:55 smithi134 ceph-mon[35037]: mds.? [v2:172.21.15.79:6836/3952268652,v1:172.21.15.79:6837/3952268652] up:active 2024-03-28T18:11:56.175 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:11:55 smithi134 ceph-mon[35037]: fsmap cephfs:5 4 up:standby-replay 3 up:standby 5 up:active 2024-03-28T18:11:56.175 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:11:55 smithi134 ceph-mon[35037]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-28T18:11:56.433 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:11:55 smithi079 systemd[1]: ceph-fd220748-ed2b-11ee-95d0-87774f69a715@mds.h.service: Failed with result 'exit-code'. 2024-03-28T18:11:56.433 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:11:55 smithi079 systemd[1]: ceph-fd220748-ed2b-11ee-95d0-87774f69a715@mds.h.service: Consumed 1.698s CPU time. 2024-03-28T18:11:57.032 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:11:56 smithi022 ceph-mon[31985]: pgmap v140: 97 pgs: 97 active+clean; 117 MiB data, 800 MiB used, 1.0 TiB / 1.0 TiB avail; 7.9 KiB/s rd, 2.5 KiB/s wr, 11 op/s 2024-03-28T18:11:57.174 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:11:56 smithi134 ceph-mon[35037]: pgmap v140: 97 pgs: 97 active+clean; 117 MiB data, 800 MiB used, 1.0 TiB / 1.0 TiB avail; 7.9 KiB/s rd, 2.5 KiB/s wr, 11 op/s 2024-03-28T18:11:57.182 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:11:56 smithi079 ceph-mon[35312]: pgmap v140: 97 pgs: 97 active+clean; 117 MiB data, 800 MiB used, 1.0 TiB / 1.0 TiB avail; 7.9 KiB/s rd, 2.5 KiB/s wr, 11 op/s 2024-03-28T18:11:58.933 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:11:58 smithi079 ceph-mon[35312]: pgmap v141: 97 pgs: 97 active+clean; 117 MiB data, 800 MiB used, 1.0 TiB / 1.0 TiB avail; 7.9 KiB/s rd, 2.5 KiB/s wr, 11 op/s 2024-03-28T18:11:59.032 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:11:58 smithi022 ceph-mon[31985]: pgmap v141: 97 pgs: 97 active+clean; 117 MiB data, 800 MiB used, 1.0 TiB / 1.0 TiB avail; 7.9 KiB/s rd, 2.5 KiB/s wr, 11 op/s 2024-03-28T18:11:59.174 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:11:58 smithi134 ceph-mon[35037]: pgmap v141: 97 pgs: 97 active+clean; 117 MiB data, 800 MiB used, 1.0 TiB / 1.0 TiB avail; 7.9 KiB/s rd, 2.5 KiB/s wr, 11 op/s 2024-03-28T18:12:00.980 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:12:00 smithi134 ceph-mon[35037]: pgmap v142: 97 pgs: 97 active+clean; 117 MiB data, 800 MiB used, 1.0 TiB / 1.0 TiB avail; 108 KiB/s rd, 4.8 KiB/s wr, 23 op/s 2024-03-28T18:12:01.032 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:12:00 smithi022 ceph-mon[31985]: pgmap v142: 97 pgs: 97 active+clean; 117 MiB data, 800 MiB used, 1.0 TiB / 1.0 TiB avail; 108 KiB/s rd, 4.8 KiB/s wr, 23 op/s 2024-03-28T18:12:01.182 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:12:00 smithi079 ceph-mon[35312]: pgmap v142: 97 pgs: 97 active+clean; 117 MiB data, 800 MiB used, 1.0 TiB / 1.0 TiB avail; 108 KiB/s rd, 4.8 KiB/s wr, 23 op/s 2024-03-28T18:12:02.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:12:02 smithi134 ceph-mon[35037]: pgmap v143: 97 pgs: 97 active+clean; 117 MiB data, 800 MiB used, 1.0 TiB / 1.0 TiB avail; 100 KiB/s rd, 4.4 KiB/s wr, 24 op/s 2024-03-28T18:12:03.031 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:12:02 smithi022 ceph-mon[31985]: pgmap v143: 97 pgs: 97 active+clean; 117 MiB data, 800 MiB used, 1.0 TiB / 1.0 TiB avail; 100 KiB/s rd, 4.4 KiB/s wr, 24 op/s 2024-03-28T18:12:03.182 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:12:02 smithi079 ceph-mon[35312]: pgmap v143: 97 pgs: 97 active+clean; 117 MiB data, 800 MiB used, 1.0 TiB / 1.0 TiB avail; 100 KiB/s rd, 4.4 KiB/s wr, 24 op/s 2024-03-28T18:12:05.031 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:12:04 smithi022 ceph-mon[31985]: pgmap v144: 97 pgs: 97 active+clean; 117 MiB data, 800 MiB used, 1.0 TiB / 1.0 TiB avail; 91 KiB/s rd, 4.0 KiB/s wr, 22 op/s 2024-03-28T18:12:05.174 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:12:04 smithi134 ceph-mon[35037]: pgmap v144: 97 pgs: 97 active+clean; 117 MiB data, 800 MiB used, 1.0 TiB / 1.0 TiB avail; 91 KiB/s rd, 4.0 KiB/s wr, 22 op/s 2024-03-28T18:12:05.182 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:12:04 smithi079 ceph-mon[35312]: pgmap v144: 97 pgs: 97 active+clean; 117 MiB data, 800 MiB used, 1.0 TiB / 1.0 TiB avail; 91 KiB/s rd, 4.0 KiB/s wr, 22 op/s 2024-03-28T18:12:06.588 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:12:06 smithi079 systemd[1]: ceph-fd220748-ed2b-11ee-95d0-87774f69a715@mds.h.service: Scheduled restart job, restart counter is at 4. 2024-03-28T18:12:06.588 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:12:06 smithi079 systemd[1]: Stopped Ceph mds.h for fd220748-ed2b-11ee-95d0-87774f69a715. 2024-03-28T18:12:06.588 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:12:06 smithi079 systemd[1]: ceph-fd220748-ed2b-11ee-95d0-87774f69a715@mds.h.service: Consumed 1.698s CPU time. 2024-03-28T18:12:06.588 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:12:06 smithi079 systemd[1]: Starting Ceph mds.h for fd220748-ed2b-11ee-95d0-87774f69a715... 2024-03-28T18:12:06.905 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:12:06 smithi079 ceph-mon[35312]: pgmap v145: 97 pgs: 97 active+clean; 117 MiB data, 800 MiB used, 1.0 TiB / 1.0 TiB avail; 94 KiB/s rd, 4.0 KiB/s wr, 27 op/s 2024-03-28T18:12:06.905 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:12:06 smithi079 podman[69728]: 2024-03-28 18:12:06.494681188 +0000 UTC m=+0.040923590 image pull c664afe56bb01db4f9afa9532c20598e201e664f2e9ba2be50db0d52674a26d2 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc 2024-03-28T18:12:06.905 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:12:06 smithi079 podman[69728]: 2024-03-28 18:12:06.634831336 +0000 UTC m=+0.181073758 container create 8c1d8d9a7546d8f8f2fd111916548d82a06ef058daf7d65dcc6a9ecd81578320 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, ceph=True, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240319, GIT_CLEAN=True, org.label-schema.schema-version=1.0, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-28T18:12:07.031 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:12:06 smithi022 ceph-mon[31985]: pgmap v145: 97 pgs: 97 active+clean; 117 MiB data, 800 MiB used, 1.0 TiB / 1.0 TiB avail; 94 KiB/s rd, 4.0 KiB/s wr, 27 op/s 2024-03-28T18:12:07.174 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:12:06 smithi134 ceph-mon[35037]: pgmap v145: 97 pgs: 97 active+clean; 117 MiB data, 800 MiB used, 1.0 TiB / 1.0 TiB avail; 94 KiB/s rd, 4.0 KiB/s wr, 27 op/s 2024-03-28T18:12:07.182 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:12:06 smithi079 podman[69728]: 2024-03-28 18:12:06.904919953 +0000 UTC m=+0.451162361 container init 8c1d8d9a7546d8f8f2fd111916548d82a06ef058daf7d65dcc6a9ecd81578320 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, ceph=True, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240319, maintainer=Guillaume Abrioux , GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=) 2024-03-28T18:12:07.182 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:12:06 smithi079 podman[69728]: 2024-03-28 18:12:06.916133458 +0000 UTC m=+0.462375850 container start 8c1d8d9a7546d8f8f2fd111916548d82a06ef058daf7d65dcc6a9ecd81578320 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h, org.label-schema.license=GPLv2, ceph=True, GIT_CLEAN=True, org.label-schema.schema-version=1.0, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.build-date=20240319, maintainer=Guillaume Abrioux , GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS) 2024-03-28T18:12:07.183 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:12:06 smithi079 ceph-mds[69745]: -- 172.21.15.79:0/1346053051 <== mon.2 v2:172.21.15.79:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55c654bd5a40 con 0x55c654ba9000 2024-03-28T18:12:07.183 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:12:06 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69741]: starting mds.h at 2024-03-28T18:12:07.183 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:12:07 smithi079 bash[69728]: 8c1d8d9a7546d8f8f2fd111916548d82a06ef058daf7d65dcc6a9ecd81578320 2024-03-28T18:12:07.183 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:12:07 smithi079 systemd[1]: Started Ceph mds.h for fd220748-ed2b-11ee-95d0-87774f69a715. 2024-03-28T18:12:08.032 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:12:07 smithi022 ceph-mon[31985]: daemon mds.h restarted 2024-03-28T18:12:08.032 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:12:07 smithi022 ceph-mon[31985]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-03-28T18:12:08.032 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:12:07 smithi022 ceph-mon[31985]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-03-28T18:12:08.032 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:12:07 smithi022 ceph-mon[31985]: osdmap e100: 12 total, 12 up, 12 in 2024-03-28T18:12:08.032 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:12:07 smithi022 ceph-mon[31985]: mds.? [v2:172.21.15.79:6836/3204076941,v1:172.21.15.79:6837/3204076941] up:boot 2024-03-28T18:12:08.032 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:12:07 smithi022 ceph-mon[31985]: Standby daemon mds.f assigned to filesystem cephfs as rank 1 2024-03-28T18:12:08.032 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:12:07 smithi022 ceph-mon[31985]: fsmap cephfs:4/5 {0=b=up:active,2=e=up:active,3=k=up:active,4=i=up:active} 5 up:standby-replay 3 up:standby, 1 failed 2024-03-28T18:12:08.032 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:12:07 smithi022 ceph-mon[31985]: fsmap cephfs:5/5 4 up:standby-replay 3 up:standby 1 up:replay 4 up:active 2024-03-28T18:12:08.175 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:12:07 smithi134 ceph-mon[35037]: daemon mds.h restarted 2024-03-28T18:12:08.175 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:12:07 smithi134 ceph-mon[35037]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-03-28T18:12:08.175 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:12:07 smithi134 ceph-mon[35037]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-03-28T18:12:08.175 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:12:07 smithi134 ceph-mon[35037]: osdmap e100: 12 total, 12 up, 12 in 2024-03-28T18:12:08.175 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:12:07 smithi134 ceph-mon[35037]: mds.? [v2:172.21.15.79:6836/3204076941,v1:172.21.15.79:6837/3204076941] up:boot 2024-03-28T18:12:08.175 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:12:07 smithi134 ceph-mon[35037]: Standby daemon mds.f assigned to filesystem cephfs as rank 1 2024-03-28T18:12:08.175 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:12:07 smithi134 ceph-mon[35037]: fsmap cephfs:4/5 {0=b=up:active,2=e=up:active,3=k=up:active,4=i=up:active} 5 up:standby-replay 3 up:standby, 1 failed 2024-03-28T18:12:08.175 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:12:07 smithi134 ceph-mon[35037]: fsmap cephfs:5/5 4 up:standby-replay 3 up:standby 1 up:replay 4 up:active 2024-03-28T18:12:08.182 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:12:07 smithi079 ceph-mon[35312]: daemon mds.h restarted 2024-03-28T18:12:08.183 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:12:07 smithi079 ceph-mon[35312]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-03-28T18:12:08.183 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:12:07 smithi079 ceph-mon[35312]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-03-28T18:12:08.183 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:12:07 smithi079 ceph-mon[35312]: osdmap e100: 12 total, 12 up, 12 in 2024-03-28T18:12:08.183 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:12:07 smithi079 ceph-mon[35312]: mds.? [v2:172.21.15.79:6836/3204076941,v1:172.21.15.79:6837/3204076941] up:boot 2024-03-28T18:12:08.183 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:12:07 smithi079 ceph-mon[35312]: Standby daemon mds.f assigned to filesystem cephfs as rank 1 2024-03-28T18:12:08.183 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:12:07 smithi079 ceph-mon[35312]: fsmap cephfs:4/5 {0=b=up:active,2=e=up:active,3=k=up:active,4=i=up:active} 5 up:standby-replay 3 up:standby, 1 failed 2024-03-28T18:12:08.183 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:12:07 smithi079 ceph-mon[35312]: fsmap cephfs:5/5 4 up:standby-replay 3 up:standby 1 up:replay 4 up:active 2024-03-28T18:12:09.032 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:12:08 smithi022 ceph-mon[31985]: pgmap v147: 97 pgs: 97 active+clean; 117 MiB data, 800 MiB used, 1.0 TiB / 1.0 TiB avail; 107 KiB/s rd, 2.3 KiB/s wr, 26 op/s 2024-03-28T18:12:09.034 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:12:08 smithi079 ceph-mon[35312]: pgmap v147: 97 pgs: 97 active+clean; 117 MiB data, 800 MiB used, 1.0 TiB / 1.0 TiB avail; 107 KiB/s rd, 2.3 KiB/s wr, 26 op/s 2024-03-28T18:12:09.174 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:12:08 smithi134 ceph-mon[35037]: pgmap v147: 97 pgs: 97 active+clean; 117 MiB data, 800 MiB used, 1.0 TiB / 1.0 TiB avail; 107 KiB/s rd, 2.3 KiB/s wr, 26 op/s 2024-03-28T18:12:10.032 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:12:09 smithi022 ceph-mon[31985]: mds.? [v2:172.21.15.134:6834/2842919302,v1:172.21.15.134:6835/2842919302] up:resolve 2024-03-28T18:12:10.032 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:12:09 smithi022 ceph-mon[31985]: fsmap cephfs:5/5 4 up:standby-replay 3 up:standby 1 up:resolve 4 up:active 2024-03-28T18:12:10.174 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:12:09 smithi134 ceph-mon[35037]: mds.? [v2:172.21.15.134:6834/2842919302,v1:172.21.15.134:6835/2842919302] up:resolve 2024-03-28T18:12:10.175 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:12:09 smithi134 ceph-mon[35037]: fsmap cephfs:5/5 4 up:standby-replay 3 up:standby 1 up:resolve 4 up:active 2024-03-28T18:12:10.182 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:12:09 smithi079 ceph-mon[35312]: mds.? [v2:172.21.15.134:6834/2842919302,v1:172.21.15.134:6835/2842919302] up:resolve 2024-03-28T18:12:10.182 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:12:09 smithi079 ceph-mon[35312]: fsmap cephfs:5/5 4 up:standby-replay 3 up:standby 1 up:resolve 4 up:active 2024-03-28T18:12:11.175 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:12:10 smithi134 ceph-mon[35037]: pgmap v148: 97 pgs: 97 active+clean; 117 MiB data, 800 MiB used, 1.0 TiB / 1.0 TiB avail; 8.9 KiB/s rd, 204 B/s wr, 15 op/s 2024-03-28T18:12:11.175 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:12:10 smithi134 ceph-mon[35037]: reconnect by client.35222 172.21.15.22:0/3729116781 after 0.0039999 2024-03-28T18:12:11.175 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:12:10 smithi134 ceph-mon[35037]: mds.? [v2:172.21.15.134:6834/2842919302,v1:172.21.15.134:6835/2842919302] up:reconnect 2024-03-28T18:12:11.175 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:12:10 smithi134 ceph-mon[35037]: fsmap cephfs:5/5 4 up:standby-replay 3 up:standby 1 up:reconnect 4 up:active 2024-03-28T18:12:11.182 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:12:10 smithi079 ceph-mon[35312]: pgmap v148: 97 pgs: 97 active+clean; 117 MiB data, 800 MiB used, 1.0 TiB / 1.0 TiB avail; 8.9 KiB/s rd, 204 B/s wr, 15 op/s 2024-03-28T18:12:11.182 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:12:10 smithi079 ceph-mon[35312]: reconnect by client.35222 172.21.15.22:0/3729116781 after 0.0039999 2024-03-28T18:12:11.183 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:12:10 smithi079 ceph-mon[35312]: mds.? [v2:172.21.15.134:6834/2842919302,v1:172.21.15.134:6835/2842919302] up:reconnect 2024-03-28T18:12:11.183 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:12:10 smithi079 ceph-mon[35312]: fsmap cephfs:5/5 4 up:standby-replay 3 up:standby 1 up:reconnect 4 up:active 2024-03-28T18:12:11.282 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:12:10 smithi022 ceph-mon[31985]: pgmap v148: 97 pgs: 97 active+clean; 117 MiB data, 800 MiB used, 1.0 TiB / 1.0 TiB avail; 8.9 KiB/s rd, 204 B/s wr, 15 op/s 2024-03-28T18:12:11.282 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:12:10 smithi022 ceph-mon[31985]: reconnect by client.35222 172.21.15.22:0/3729116781 after 0.0039999 2024-03-28T18:12:11.282 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:12:10 smithi022 ceph-mon[31985]: mds.? [v2:172.21.15.134:6834/2842919302,v1:172.21.15.134:6835/2842919302] up:reconnect 2024-03-28T18:12:11.282 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:12:10 smithi022 ceph-mon[31985]: fsmap cephfs:5/5 4 up:standby-replay 3 up:standby 1 up:reconnect 4 up:active 2024-03-28T18:12:12.174 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:12:11 smithi134 ceph-mon[35037]: mds.? [v2:172.21.15.134:6834/2842919302,v1:172.21.15.134:6835/2842919302] up:rejoin 2024-03-28T18:12:12.174 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:12:11 smithi134 ceph-mon[35037]: fsmap cephfs:5/5 4 up:standby-replay 3 up:standby 1 up:rejoin 4 up:active 2024-03-28T18:12:12.175 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:12:11 smithi134 ceph-mon[35037]: daemon mds.f is now active in filesystem cephfs as rank 1 2024-03-28T18:12:12.175 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:12:11 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[70308]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-2145-g3be9a5c9/rpm/el8/BUILD/ceph-19.0.0-2145-g3be9a5c9/src/mds/Locker.cc: In function 'void Locker::set_xlocks_done(MutationImpl*, bool)' thread 7fee7f200700 time 2024-03-28T18:12:11.876307+0000 2024-03-28T18:12:12.176 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:12:11 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[70308]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-2145-g3be9a5c9/rpm/el8/BUILD/ceph-19.0.0-2145-g3be9a5c9/src/mds/Locker.cc: 626: FAILED ceph_assert(obj->is_auth()) 2024-03-28T18:12:12.176 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:12:11 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[70308]: ceph version 19.0.0-2145-g3be9a5c9 (3be9a5c9bc793e11e2800a8c0c696e8b46742033) squid (dev) 2024-03-28T18:12:12.176 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:12:11 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[70308]: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x135) [0x7fee8ab544a1] 2024-03-28T18:12:12.176 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:12:11 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[70308]: 2: /usr/lib64/ceph/libceph-common.so.2(+0x2d2667) [0x7fee8ab54667] 2024-03-28T18:12:12.176 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:12:11 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[70308]: 3: (Locker::set_xlocks_done(MutationImpl*, bool)+0x493) [0x561e609d6223] 2024-03-28T18:12:12.176 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:12:11 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[70308]: 4: (Server::early_reply(boost::intrusive_ptr const&, CInode*, CDentry*)+0x3e6) [0x561e608505d6] 2024-03-28T18:12:12.176 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:12:11 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[70308]: 5: (Server::journal_and_reply(boost::intrusive_ptr const&, CInode*, CDentry*, LogEvent*, MDSLogContextBase*)+0xc1) [0x561e60850da1] 2024-03-28T18:12:12.176 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:12:11 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[70308]: 6: (Server::handle_client_mkdir(boost::intrusive_ptr const&)+0x5ba) [0x561e608866aa] 2024-03-28T18:12:12.176 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:12:11 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[70308]: 7: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0x733) [0x561e608a3233] 2024-03-28T18:12:12.176 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:12:11 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[70308]: 8: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x561e608a5197] 2024-03-28T18:12:12.176 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:12:11 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[70308]: 9: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x561e607a411b] 2024-03-28T18:12:12.176 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:12:11 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[70308]: 10: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x561e607a46bc] 2024-03-28T18:12:12.176 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:12:11 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[70308]: 11: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bf) [0x561e6078e1ff] 2024-03-28T18:12:12.177 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:12:11 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[70308]: 12: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x478) [0x7fee8ade6488] 2024-03-28T18:12:12.177 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:12:11 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[70308]: 13: (DispatchQueue::entry()+0x51f) [0x7fee8ade360f] 2024-03-28T18:12:12.177 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:12:11 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[70308]: 14: (DispatchQueue::DispatchThread::entry()+0x11) [0x7fee8aeb6e81] 2024-03-28T18:12:12.177 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:12:11 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[70308]: 15: /lib64/libpthread.so.0(+0x81ca) [0x7fee898d11ca] 2024-03-28T18:12:12.177 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:12:11 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[70308]: 16: clone() 2024-03-28T18:12:12.177 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:12:11 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[70308]: 2024-03-28T18:12:11.878+0000 7fee7f200700 -1 /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-2145-g3be9a5c9/rpm/el8/BUILD/ceph-19.0.0-2145-g3be9a5c9/src/mds/Locker.cc: In function 'void Locker::set_xlocks_done(MutationImpl*, bool)' thread 7fee7f200700 time 2024-03-28T18:12:11.876307+0000 2024-03-28T18:12:12.177 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:12:11 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[70308]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-2145-g3be9a5c9/rpm/el8/BUILD/ceph-19.0.0-2145-g3be9a5c9/src/mds/Locker.cc: 626: FAILED ceph_assert(obj->is_auth()) 2024-03-28T18:12:12.177 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:12:11 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[70308]: 2024-03-28T18:12:12.177 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:12:11 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[70308]: ceph version 19.0.0-2145-g3be9a5c9 (3be9a5c9bc793e11e2800a8c0c696e8b46742033) squid (dev) 2024-03-28T18:12:12.177 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:12:11 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[70308]: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x135) [0x7fee8ab544a1] 2024-03-28T18:12:12.178 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:12:11 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[70308]: 2: /usr/lib64/ceph/libceph-common.so.2(+0x2d2667) [0x7fee8ab54667] 2024-03-28T18:12:12.178 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:12:11 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[70308]: 3: (Locker::set_xlocks_done(MutationImpl*, bool)+0x493) [0x561e609d6223] 2024-03-28T18:12:12.178 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:12:11 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[70308]: 4: (Server::early_reply(boost::intrusive_ptr const&, CInode*, CDentry*)+0x3e6) [0x561e608505d6] 2024-03-28T18:12:12.178 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:12:11 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[70308]: 5: (Server::journal_and_reply(boost::intrusive_ptr const&, CInode*, CDentry*, LogEvent*, MDSLogContextBase*)+0xc1) [0x561e60850da1] 2024-03-28T18:12:12.178 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:12:11 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[70308]: 6: (Server::handle_client_mkdir(boost::intrusive_ptr const&)+0x5ba) [0x561e608866aa] 2024-03-28T18:12:12.178 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:12:11 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[70308]: 7: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0x733) [0x561e608a3233] 2024-03-28T18:12:12.178 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:12:11 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[70308]: 8: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x561e608a5197] 2024-03-28T18:12:12.178 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:12:11 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[70308]: 9: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x561e607a411b] 2024-03-28T18:12:12.178 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:12:11 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[70308]: 10: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x561e607a46bc] 2024-03-28T18:12:12.178 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:12:11 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[70308]: 11: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bf) [0x561e6078e1ff] 2024-03-28T18:12:12.178 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:12:11 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[70308]: 12: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x478) [0x7fee8ade6488] 2024-03-28T18:12:12.178 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:12:11 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[70308]: 13: (DispatchQueue::entry()+0x51f) [0x7fee8ade360f] 2024-03-28T18:12:12.178 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:12:11 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[70308]: 14: (DispatchQueue::DispatchThread::entry()+0x11) [0x7fee8aeb6e81] 2024-03-28T18:12:12.178 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:12:11 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[70308]: 15: /lib64/libpthread.so.0(+0x81ca) [0x7fee898d11ca] 2024-03-28T18:12:12.179 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:12:11 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[70308]: 16: clone() 2024-03-28T18:12:12.179 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:12:11 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[70308]: 2024-03-28T18:12:12.179 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:12:11 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[70308]: *** Caught signal (Aborted) ** 2024-03-28T18:12:12.179 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:12:11 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[70308]: in thread 7fee7f200700 thread_name:ms_dispatch 2024-03-28T18:12:12.179 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:12:11 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[70308]: ceph version 19.0.0-2145-g3be9a5c9 (3be9a5c9bc793e11e2800a8c0c696e8b46742033) squid (dev) 2024-03-28T18:12:12.179 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:12:11 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[70308]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7fee898dbd20] 2024-03-28T18:12:12.179 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:12:11 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[70308]: 2: gsignal() 2024-03-28T18:12:12.179 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:12:11 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[70308]: 3: abort() 2024-03-28T18:12:12.179 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:12:11 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[70308]: 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x18f) [0x7fee8ab544fb] 2024-03-28T18:12:12.179 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:12:11 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[70308]: 5: /usr/lib64/ceph/libceph-common.so.2(+0x2d2667) [0x7fee8ab54667] 2024-03-28T18:12:12.180 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:12:11 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[70308]: 6: (Locker::set_xlocks_done(MutationImpl*, bool)+0x493) [0x561e609d6223] 2024-03-28T18:12:12.180 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:12:11 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[70308]: 7: (Server::early_reply(boost::intrusive_ptr const&, CInode*, CDentry*)+0x3e6) [0x561e608505d6] 2024-03-28T18:12:12.180 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:12:11 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[70308]: 8: (Server::journal_and_reply(boost::intrusive_ptr const&, CInode*, CDentry*, LogEvent*, MDSLogContextBase*)+0xc1) [0x561e60850da1] 2024-03-28T18:12:12.180 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:12:11 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[70308]: 9: (Server::handle_client_mkdir(boost::intrusive_ptr const&)+0x5ba) [0x561e608866aa] 2024-03-28T18:12:12.180 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:12:11 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[70308]: 10: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0x733) [0x561e608a3233] 2024-03-28T18:12:12.180 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:12:11 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[70308]: 11: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x561e608a5197] 2024-03-28T18:12:12.180 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:12:11 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[70308]: 12: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x561e607a411b] 2024-03-28T18:12:12.180 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:12:11 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[70308]: 13: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x561e607a46bc] 2024-03-28T18:12:12.180 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:12:11 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[70308]: 14: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bf) [0x561e6078e1ff] 2024-03-28T18:12:12.180 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:12:11 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[70308]: 15: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x478) [0x7fee8ade6488] 2024-03-28T18:12:12.180 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:12:11 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[70308]: 16: (DispatchQueue::entry()+0x51f) [0x7fee8ade360f] 2024-03-28T18:12:12.180 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:12:11 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[70308]: 17: (DispatchQueue::DispatchThread::entry()+0x11) [0x7fee8aeb6e81] 2024-03-28T18:12:12.180 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:12:11 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[70308]: 18: /lib64/libpthread.so.0(+0x81ca) [0x7fee898d11ca] 2024-03-28T18:12:12.180 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:12:11 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[70308]: 19: clone() 2024-03-28T18:12:12.180 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:12:11 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[70308]: 2024-03-28T18:12:11.881+0000 7fee7f200700 -1 *** Caught signal (Aborted) ** 2024-03-28T18:12:12.181 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:12:11 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[70308]: in thread 7fee7f200700 thread_name:ms_dispatch 2024-03-28T18:12:12.181 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:12:11 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[70308]: 2024-03-28T18:12:12.181 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:12:11 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[70308]: ceph version 19.0.0-2145-g3be9a5c9 (3be9a5c9bc793e11e2800a8c0c696e8b46742033) squid (dev) 2024-03-28T18:12:12.181 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:12:11 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[70308]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7fee898dbd20] 2024-03-28T18:12:12.181 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:12:11 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[70308]: 2: gsignal() 2024-03-28T18:12:12.181 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:12:11 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[70308]: 3: abort() 2024-03-28T18:12:12.181 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:12:11 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[70308]: 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x18f) [0x7fee8ab544fb] 2024-03-28T18:12:12.181 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:12:11 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[70308]: 5: /usr/lib64/ceph/libceph-common.so.2(+0x2d2667) [0x7fee8ab54667] 2024-03-28T18:12:12.181 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:12:11 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[70308]: 6: (Locker::set_xlocks_done(MutationImpl*, bool)+0x493) [0x561e609d6223] 2024-03-28T18:12:12.181 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:12:11 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[70308]: 7: (Server::early_reply(boost::intrusive_ptr const&, CInode*, CDentry*)+0x3e6) [0x561e608505d6] 2024-03-28T18:12:12.181 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:12:11 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[70308]: 8: (Server::journal_and_reply(boost::intrusive_ptr const&, CInode*, CDentry*, LogEvent*, MDSLogContextBase*)+0xc1) [0x561e60850da1] 2024-03-28T18:12:12.182 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:12:11 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[70308]: 9: (Server::handle_client_mkdir(boost::intrusive_ptr const&)+0x5ba) [0x561e608866aa] 2024-03-28T18:12:12.182 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:12:11 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[70308]: 10: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0x733) [0x561e608a3233] 2024-03-28T18:12:12.182 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:12:11 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[70308]: 11: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x561e608a5197] 2024-03-28T18:12:12.182 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:12:11 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[70308]: 12: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x561e607a411b] 2024-03-28T18:12:12.182 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:12:11 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[70308]: 13: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x561e607a46bc] 2024-03-28T18:12:12.182 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:12:11 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[70308]: 14: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bf) [0x561e6078e1ff] 2024-03-28T18:12:12.182 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:12:11 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[70308]: 15: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x478) [0x7fee8ade6488] 2024-03-28T18:12:12.182 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:12:11 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[70308]: 16: (DispatchQueue::entry()+0x51f) [0x7fee8ade360f] 2024-03-28T18:12:12.182 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:12:11 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[70308]: 17: (DispatchQueue::DispatchThread::entry()+0x11) [0x7fee8aeb6e81] 2024-03-28T18:12:12.182 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:12:11 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[70308]: 18: /lib64/libpthread.so.0(+0x81ca) [0x7fee898d11ca] 2024-03-28T18:12:12.182 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:12:11 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[70308]: 19: clone() 2024-03-28T18:12:12.182 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:12:11 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[70308]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-03-28T18:12:12.182 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:12:11 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[70308]: 2024-03-28T18:12:12.182 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:12:11 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[70308]: -1> 2024-03-28T18:12:11.878+0000 7fee7f200700 -1 /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-2145-g3be9a5c9/rpm/el8/BUILD/ceph-19.0.0-2145-g3be9a5c9/src/mds/Locker.cc: In function 'void Locker::set_xlocks_done(MutationImpl*, bool)' thread 7fee7f200700 time 2024-03-28T18:12:11.876307+0000 2024-03-28T18:12:12.183 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:12:11 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[70308]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-2145-g3be9a5c9/rpm/el8/BUILD/ceph-19.0.0-2145-g3be9a5c9/src/mds/Locker.cc: 626: FAILED ceph_assert(obj->is_auth()) 2024-03-28T18:12:12.183 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:12:11 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[70308]: 2024-03-28T18:12:12.183 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:12:11 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[70308]: ceph version 19.0.0-2145-g3be9a5c9 (3be9a5c9bc793e11e2800a8c0c696e8b46742033) squid (dev) 2024-03-28T18:12:12.183 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:12:11 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[70308]: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x135) [0x7fee8ab544a1] 2024-03-28T18:12:12.183 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:12:11 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[70308]: 2: /usr/lib64/ceph/libceph-common.so.2(+0x2d2667) [0x7fee8ab54667] 2024-03-28T18:12:12.183 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:12:11 smithi079 ceph-mon[35312]: mds.? [v2:172.21.15.134:6834/2842919302,v1:172.21.15.134:6835/2842919302] up:rejoin 2024-03-28T18:12:12.183 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:12:11 smithi079 ceph-mon[35312]: fsmap cephfs:5/5 4 up:standby-replay 3 up:standby 1 up:rejoin 4 up:active 2024-03-28T18:12:12.183 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:12:11 smithi079 ceph-mon[35312]: daemon mds.f is now active in filesystem cephfs as rank 1 2024-03-28T18:12:12.184 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:12:11 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[70308]: 3: (Locker::set_xlocks_done(MutationImpl*, bool)+0x493) [0x561e609d6223] 2024-03-28T18:12:12.184 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:12:11 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[70308]: 4: (Server::early_reply(boost::intrusive_ptr const&, CInode*, CDentry*)+0x3e6) [0x561e608505d6] 2024-03-28T18:12:12.184 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:12:11 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[70308]: 5: (Server::journal_and_reply(boost::intrusive_ptr const&, CInode*, CDentry*, LogEvent*, MDSLogContextBase*)+0xc1) [0x561e60850da1] 2024-03-28T18:12:12.184 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:12:11 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[70308]: 6: (Server::handle_client_mkdir(boost::intrusive_ptr const&)+0x5ba) [0x561e608866aa] 2024-03-28T18:12:12.184 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:12:11 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[70308]: 7: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0x733) [0x561e608a3233] 2024-03-28T18:12:12.184 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:12:11 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[70308]: 8: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x561e608a5197] 2024-03-28T18:12:12.184 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:12:11 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[70308]: 9: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x561e607a411b] 2024-03-28T18:12:12.184 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:12:11 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[70308]: 10: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x561e607a46bc] 2024-03-28T18:12:12.184 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:12:11 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[70308]: 11: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bf) [0x561e6078e1ff] 2024-03-28T18:12:12.184 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:12:11 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[70308]: 12: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x478) [0x7fee8ade6488] 2024-03-28T18:12:12.184 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:12:11 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[70308]: 13: (DispatchQueue::entry()+0x51f) [0x7fee8ade360f] 2024-03-28T18:12:12.184 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:12:11 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[70308]: 14: (DispatchQueue::DispatchThread::entry()+0x11) [0x7fee8aeb6e81] 2024-03-28T18:12:12.184 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:12:11 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[70308]: 15: /lib64/libpthread.so.0(+0x81ca) [0x7fee898d11ca] 2024-03-28T18:12:12.185 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:12:11 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[70308]: 16: clone() 2024-03-28T18:12:12.185 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:12:11 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[70308]: 2024-03-28T18:12:12.185 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:12:11 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[70308]: 0> 2024-03-28T18:12:11.881+0000 7fee7f200700 -1 *** Caught signal (Aborted) ** 2024-03-28T18:12:12.185 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:12:11 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[70308]: in thread 7fee7f200700 thread_name:ms_dispatch 2024-03-28T18:12:12.185 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:12:11 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[70308]: 2024-03-28T18:12:12.185 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:12:11 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[70308]: ceph version 19.0.0-2145-g3be9a5c9 (3be9a5c9bc793e11e2800a8c0c696e8b46742033) squid (dev) 2024-03-28T18:12:12.185 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:12:11 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[70308]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7fee898dbd20] 2024-03-28T18:12:12.185 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:12:11 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[70308]: 2: gsignal() 2024-03-28T18:12:12.185 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:12:11 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[70308]: 3: abort() 2024-03-28T18:12:12.185 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:12:11 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[70308]: 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x18f) [0x7fee8ab544fb] 2024-03-28T18:12:12.185 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:12:11 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[70308]: 5: /usr/lib64/ceph/libceph-common.so.2(+0x2d2667) [0x7fee8ab54667] 2024-03-28T18:12:12.185 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:12:11 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[70308]: 6: (Locker::set_xlocks_done(MutationImpl*, bool)+0x493) [0x561e609d6223] 2024-03-28T18:12:12.185 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:12:11 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[70308]: 7: (Server::early_reply(boost::intrusive_ptr const&, CInode*, CDentry*)+0x3e6) [0x561e608505d6] 2024-03-28T18:12:12.186 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:12:11 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[70308]: 8: (Server::journal_and_reply(boost::intrusive_ptr const&, CInode*, CDentry*, LogEvent*, MDSLogContextBase*)+0xc1) [0x561e60850da1] 2024-03-28T18:12:12.186 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:12:11 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[70308]: 9: (Server::handle_client_mkdir(boost::intrusive_ptr const&)+0x5ba) [0x561e608866aa] 2024-03-28T18:12:12.186 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:12:11 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[70308]: 10: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0x733) [0x561e608a3233] 2024-03-28T18:12:12.186 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:12:11 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[70308]: 11: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x561e608a5197] 2024-03-28T18:12:12.186 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:12:11 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[70308]: 12: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x561e607a411b] 2024-03-28T18:12:12.186 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:12:11 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[70308]: 13: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x561e607a46bc] 2024-03-28T18:12:12.186 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:12:11 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[70308]: 14: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bf) [0x561e6078e1ff] 2024-03-28T18:12:12.186 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:12:11 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[70308]: 15: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x478) [0x7fee8ade6488] 2024-03-28T18:12:12.186 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:12:11 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[70308]: 16: (DispatchQueue::entry()+0x51f) [0x7fee8ade360f] 2024-03-28T18:12:12.186 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:12:11 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[70308]: 17: (DispatchQueue::DispatchThread::entry()+0x11) [0x7fee8aeb6e81] 2024-03-28T18:12:12.186 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:12:11 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[70308]: 18: /lib64/libpthread.so.0(+0x81ca) [0x7fee898d11ca] 2024-03-28T18:12:12.186 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:12:11 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[70308]: 19: clone() 2024-03-28T18:12:12.186 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:12:11 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[70308]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-03-28T18:12:12.187 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:12:11 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[70308]: 2024-03-28T18:12:12.187 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:12:11 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[70308]: -1> 2024-03-28T18:12:11.878+0000 7fee7f200700 -1 /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-2145-g3be9a5c9/rpm/el8/BUILD/ceph-19.0.0-2145-g3be9a5c9/src/mds/Locker.cc: In function 'void Locker::set_xlocks_done(MutationImpl*, bool)' thread 7fee7f200700 time 2024-03-28T18:12:11.876307+0000 2024-03-28T18:12:12.187 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:12:11 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[70308]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-2145-g3be9a5c9/rpm/el8/BUILD/ceph-19.0.0-2145-g3be9a5c9/src/mds/Locker.cc: 626: FAILED ceph_assert(obj->is_auth()) 2024-03-28T18:12:12.187 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:12:11 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[70308]: 2024-03-28T18:12:12.187 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:12:11 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[70308]: ceph version 19.0.0-2145-g3be9a5c9 (3be9a5c9bc793e11e2800a8c0c696e8b46742033) squid (dev) 2024-03-28T18:12:12.187 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:12:11 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[70308]: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x135) [0x7fee8ab544a1] 2024-03-28T18:12:12.187 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:12:11 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[70308]: 2: /usr/lib64/ceph/libceph-common.so.2(+0x2d2667) [0x7fee8ab54667] 2024-03-28T18:12:12.187 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:12:11 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[70308]: 3: (Locker::set_xlocks_done(MutationImpl*, bool)+0x493) [0x561e609d6223] 2024-03-28T18:12:12.187 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:12:11 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[70308]: 4: (Server::early_reply(boost::intrusive_ptr const&, CInode*, CDentry*)+0x3e6) [0x561e608505d6] 2024-03-28T18:12:12.187 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:12:11 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[70308]: 5: (Server::journal_and_reply(boost::intrusive_ptr const&, CInode*, CDentry*, LogEvent*, MDSLogContextBase*)+0xc1) [0x561e60850da1] 2024-03-28T18:12:12.188 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:12:11 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[70308]: 6: (Server::handle_client_mkdir(boost::intrusive_ptr const&)+0x5ba) [0x561e608866aa] 2024-03-28T18:12:12.188 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:12:11 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[70308]: 7: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0x733) [0x561e608a3233] 2024-03-28T18:12:12.188 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:12:11 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[70308]: 8: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x561e608a5197] 2024-03-28T18:12:12.188 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:12:11 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[70308]: 9: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x561e607a411b] 2024-03-28T18:12:12.188 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:12:11 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[70308]: 10: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x561e607a46bc] 2024-03-28T18:12:12.188 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:12:11 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[70308]: 11: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bf) [0x561e6078e1ff] 2024-03-28T18:12:12.188 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:12:11 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[70308]: 12: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x478) [0x7fee8ade6488] 2024-03-28T18:12:12.188 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:12:11 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[70308]: 13: (DispatchQueue::entry()+0x51f) [0x7fee8ade360f] 2024-03-28T18:12:12.188 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:12:11 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[70308]: 14: (DispatchQueue::DispatchThread::entry()+0x11) [0x7fee8aeb6e81] 2024-03-28T18:12:12.188 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:12:11 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[70308]: 15: /lib64/libpthread.so.0(+0x81ca) [0x7fee898d11ca] 2024-03-28T18:12:12.189 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:12:11 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[70308]: 16: clone() 2024-03-28T18:12:12.189 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:12:11 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[70308]: 2024-03-28T18:12:12.189 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:12:11 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[70308]: 0> 2024-03-28T18:12:11.881+0000 7fee7f200700 -1 *** Caught signal (Aborted) ** 2024-03-28T18:12:12.189 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:12:11 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[70308]: in thread 7fee7f200700 thread_name:ms_dispatch 2024-03-28T18:12:12.189 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:12:11 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[70308]: 2024-03-28T18:12:12.189 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:12:11 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[70308]: ceph version 19.0.0-2145-g3be9a5c9 (3be9a5c9bc793e11e2800a8c0c696e8b46742033) squid (dev) 2024-03-28T18:12:12.189 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:12:11 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[70308]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7fee898dbd20] 2024-03-28T18:12:12.189 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:12:11 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[70308]: 2: gsignal() 2024-03-28T18:12:12.189 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:12:11 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[70308]: 3: abort() 2024-03-28T18:12:12.189 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:12:11 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[70308]: 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x18f) [0x7fee8ab544fb] 2024-03-28T18:12:12.189 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:12:11 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[70308]: 5: /usr/lib64/ceph/libceph-common.so.2(+0x2d2667) [0x7fee8ab54667] 2024-03-28T18:12:12.189 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:12:11 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[70308]: 6: (Locker::set_xlocks_done(MutationImpl*, bool)+0x493) [0x561e609d6223] 2024-03-28T18:12:12.189 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:12:11 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[70308]: 7: (Server::early_reply(boost::intrusive_ptr const&, CInode*, CDentry*)+0x3e6) [0x561e608505d6] 2024-03-28T18:12:12.189 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:12:11 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[70308]: 8: (Server::journal_and_reply(boost::intrusive_ptr const&, CInode*, CDentry*, LogEvent*, MDSLogContextBase*)+0xc1) [0x561e60850da1] 2024-03-28T18:12:12.189 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:12:11 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[70308]: 9: (Server::handle_client_mkdir(boost::intrusive_ptr const&)+0x5ba) [0x561e608866aa] 2024-03-28T18:12:12.190 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:12:11 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[70308]: 10: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0x733) [0x561e608a3233] 2024-03-28T18:12:12.190 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:12:11 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[70308]: 11: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x561e608a5197] 2024-03-28T18:12:12.190 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:12:11 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[70308]: 12: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x561e607a411b] 2024-03-28T18:12:12.190 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:12:11 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[70308]: 13: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x561e607a46bc] 2024-03-28T18:12:12.190 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:12:11 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[70308]: 14: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bf) [0x561e6078e1ff] 2024-03-28T18:12:12.190 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:12:11 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[70308]: 15: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x478) [0x7fee8ade6488] 2024-03-28T18:12:12.190 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:12:11 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[70308]: 16: (DispatchQueue::entry()+0x51f) [0x7fee8ade360f] 2024-03-28T18:12:12.190 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:12:11 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[70308]: 17: (DispatchQueue::DispatchThread::entry()+0x11) [0x7fee8aeb6e81] 2024-03-28T18:12:12.190 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:12:11 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[70308]: 18: /lib64/libpthread.so.0(+0x81ca) [0x7fee898d11ca] 2024-03-28T18:12:12.190 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:12:11 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[70308]: 19: clone() 2024-03-28T18:12:12.190 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:12:11 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[70308]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-03-28T18:12:12.190 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:12:11 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[70308]: 2024-03-28T18:12:12.190 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:12:12 smithi134 podman[70393]: 2024-03-28 18:12:12.012891549 +0000 UTC m=+0.036302434 container died b193fd8b1c5f7f9ea7a2b686fef5ccc3478f80f9db24dc449834c386a4cbcc79 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.build-date=20240319, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30) 2024-03-28T18:12:12.281 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:12:11 smithi022 ceph-mon[31985]: mds.? [v2:172.21.15.134:6834/2842919302,v1:172.21.15.134:6835/2842919302] up:rejoin 2024-03-28T18:12:12.282 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:12:11 smithi022 ceph-mon[31985]: fsmap cephfs:5/5 4 up:standby-replay 3 up:standby 1 up:rejoin 4 up:active 2024-03-28T18:12:12.282 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:12:11 smithi022 ceph-mon[31985]: daemon mds.f is now active in filesystem cephfs as rank 1 2024-03-28T18:12:13.175 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:12:12 smithi134 podman[70393]: 2024-03-28 18:12:12.7283789 +0000 UTC m=+0.751789803 container remove b193fd8b1c5f7f9ea7a2b686fef5ccc3478f80f9db24dc449834c386a4cbcc79 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240319, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_CLEAN=True, ceph=True, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c) 2024-03-28T18:12:13.175 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:12:12 smithi134 systemd[1]: ceph-fd220748-ed2b-11ee-95d0-87774f69a715@mds.f.service: Main process exited, code=exited, status=134/n/a 2024-03-28T18:12:13.175 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:12:12 smithi134 ceph-mon[35037]: pgmap v149: 97 pgs: 97 active+clean; 117 MiB data, 801 MiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 204 B/s wr, 11 op/s 2024-03-28T18:12:13.175 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:12:12 smithi134 ceph-mon[35037]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-03-28T18:12:13.175 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:12:12 smithi134 ceph-mon[35037]: Cluster is now healthy 2024-03-28T18:12:13.175 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:12:12 smithi134 ceph-mon[35037]: mds.? [v2:172.21.15.134:6834/2842919302,v1:172.21.15.134:6835/2842919302] up:active 2024-03-28T18:12:13.175 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:12:12 smithi134 ceph-mon[35037]: fsmap cephfs:5 4 up:standby-replay 3 up:standby 5 up:active 2024-03-28T18:12:13.175 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:12:12 smithi134 ceph-mon[35037]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-28T18:12:13.182 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:12:12 smithi079 ceph-mon[35312]: pgmap v149: 97 pgs: 97 active+clean; 117 MiB data, 801 MiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 204 B/s wr, 11 op/s 2024-03-28T18:12:13.182 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:12:12 smithi079 ceph-mon[35312]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-03-28T18:12:13.183 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:12:12 smithi079 ceph-mon[35312]: Cluster is now healthy 2024-03-28T18:12:13.183 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:12:12 smithi079 ceph-mon[35312]: mds.? [v2:172.21.15.134:6834/2842919302,v1:172.21.15.134:6835/2842919302] up:active 2024-03-28T18:12:13.183 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:12:12 smithi079 ceph-mon[35312]: fsmap cephfs:5 4 up:standby-replay 3 up:standby 5 up:active 2024-03-28T18:12:13.183 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:12:12 smithi079 ceph-mon[35312]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-28T18:12:13.282 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:12:12 smithi022 ceph-mon[31985]: pgmap v149: 97 pgs: 97 active+clean; 117 MiB data, 801 MiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 204 B/s wr, 11 op/s 2024-03-28T18:12:13.282 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:12:12 smithi022 ceph-mon[31985]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-03-28T18:12:13.282 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:12:12 smithi022 ceph-mon[31985]: Cluster is now healthy 2024-03-28T18:12:13.282 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:12:12 smithi022 ceph-mon[31985]: mds.? [v2:172.21.15.134:6834/2842919302,v1:172.21.15.134:6835/2842919302] up:active 2024-03-28T18:12:13.282 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:12:12 smithi022 ceph-mon[31985]: fsmap cephfs:5 4 up:standby-replay 3 up:standby 5 up:active 2024-03-28T18:12:13.282 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:12:12 smithi022 ceph-mon[31985]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-28T18:12:13.674 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:12:13 smithi134 systemd[1]: ceph-fd220748-ed2b-11ee-95d0-87774f69a715@mds.f.service: Failed with result 'exit-code'. 2024-03-28T18:12:13.674 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:12:13 smithi134 systemd[1]: ceph-fd220748-ed2b-11ee-95d0-87774f69a715@mds.f.service: Consumed 1.829s CPU time. 2024-03-28T18:12:14.432 INFO:journalctl@ceph.mds.b.smithi079.stdout:Mar 28 18:12:14 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-b[65245]: 2024-03-28T18:12:14.084+0000 7f845f400700 -1 mds.pinger is_rank_lagging: rank=1 was never sent ping request. 2024-03-28T18:12:15.174 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:12:14 smithi134 ceph-mon[35037]: pgmap v150: 97 pgs: 97 active+clean; 117 MiB data, 801 MiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 204 B/s wr, 11 op/s 2024-03-28T18:12:15.182 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:12:14 smithi079 ceph-mon[35312]: pgmap v150: 97 pgs: 97 active+clean; 117 MiB data, 801 MiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 204 B/s wr, 11 op/s 2024-03-28T18:12:15.281 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:12:14 smithi022 ceph-mon[31985]: pgmap v150: 97 pgs: 97 active+clean; 117 MiB data, 801 MiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 204 B/s wr, 11 op/s 2024-03-28T18:12:17.174 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:12:16 smithi134 ceph-mon[35037]: pgmap v151: 97 pgs: 97 active+clean; 117 MiB data, 785 MiB used, 1.0 TiB / 1.0 TiB avail; 119 KiB/s rd, 31 KiB/s wr, 42 op/s 2024-03-28T18:12:17.182 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:12:16 smithi079 ceph-mon[35312]: pgmap v151: 97 pgs: 97 active+clean; 117 MiB data, 785 MiB used, 1.0 TiB / 1.0 TiB avail; 119 KiB/s rd, 31 KiB/s wr, 42 op/s 2024-03-28T18:12:17.282 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:12:16 smithi022 ceph-mon[31985]: pgmap v151: 97 pgs: 97 active+clean; 117 MiB data, 785 MiB used, 1.0 TiB / 1.0 TiB avail; 119 KiB/s rd, 31 KiB/s wr, 42 op/s 2024-03-28T18:12:17.431 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-28T18:12:17.432 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-28T18:12:17.678 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:12:17.677+0000 7f4c11000640 1 -- 172.21.15.22:0/1586358835 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4c0c108800 msgr2=0x7f4c0c10c7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:12:17.678 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:12:17.677+0000 7f4c11000640 1 --2- 172.21.15.22:0/1586358835 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4c0c108800 0x7f4c0c10c7c0 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7f4bf4009860 tx=0x7f4bf402bb70 comp rx=0 tx=0).stop 2024-03-28T18:12:17.678 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:12:17.678+0000 7f4c11000640 1 -- 172.21.15.22:0/1586358835 shutdown_connections 2024-03-28T18:12:17.678 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:12:17.678+0000 7f4c11000640 1 --2- 172.21.15.22:0/1586358835 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4c0c10cea0 0x7f4c0c114a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:12:17.678 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:12:17.678+0000 7f4c11000640 1 --2- 172.21.15.22:0/1586358835 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4c0c108800 0x7f4c0c10c7c0 unknown :-1 s=CLOSED pgs=711 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:12:17.678 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:12:17.678+0000 7f4c11000640 1 --2- 172.21.15.22:0/1586358835 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f4c0c107e50 0x7f4c0c108230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:12:17.678 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:12:17.678+0000 7f4c11000640 1 -- 172.21.15.22:0/1586358835 >> 172.21.15.22:0/1586358835 conn(0x7f4c0c0774c0 msgr2=0x7f4c0c1064c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:12:17.678 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:12:17.678+0000 7f4c11000640 1 -- 172.21.15.22:0/1586358835 shutdown_connections 2024-03-28T18:12:17.678 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:12:17.678+0000 7f4c11000640 1 -- 172.21.15.22:0/1586358835 wait complete. 2024-03-28T18:12:17.681 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:12:17.679+0000 7f4c11000640 1 Processor -- start 2024-03-28T18:12:17.681 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:12:17.679+0000 7f4c11000640 1 -- start start 2024-03-28T18:12:17.681 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:12:17.680+0000 7f4c11000640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4c0c107e50 0x7f4c0c19e480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:12:17.681 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:12:17.680+0000 7f4c11000640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f4c0c10cea0 0x7f4c0c19e9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:12:17.681 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:12:17.680+0000 7f4c11000640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4c0c1a2cf0 0x7f4c0c1a31a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:12:17.681 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:12:17.680+0000 7f4c11000640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c0c117f20 con 0x7f4c0c107e50 2024-03-28T18:12:17.681 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:12:17.680+0000 7f4c11000640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c0c117da0 con 0x7f4c0c10cea0 2024-03-28T18:12:17.681 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:12:17.680+0000 7f4c11000640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c0c1180a0 con 0x7f4c0c1a2cf0 2024-03-28T18:12:17.681 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:12:17.680+0000 7f4c0b400640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4c0c107e50 0x7f4c0c19e480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:12:17.681 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:12:17.680+0000 7f4c0b400640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4c0c107e50 0x7f4c0c19e480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:50402/0 (socket says 172.21.15.22:50402) 2024-03-28T18:12:17.681 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:12:17.680+0000 7f4c0b400640 1 -- 172.21.15.22:0/1869886502 learned_addr learned my addr 172.21.15.22:0/1869886502 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-28T18:12:17.681 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:12:17.680+0000 7f4c03e00640 1 --2- 172.21.15.22:0/1869886502 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f4c0c10cea0 0x7f4c0c19e9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:12:17.682 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:12:17.680+0000 7f4c0be00640 1 --2- 172.21.15.22:0/1869886502 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4c0c1a2cf0 0x7f4c0c1a31a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:12:17.682 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:12:17.680+0000 7f4c0b400640 1 -- 172.21.15.22:0/1869886502 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4c0c1a2cf0 msgr2=0x7f4c0c1a31a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:12:17.682 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:12:17.680+0000 7f4c0b400640 1 --2- 172.21.15.22:0/1869886502 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4c0c1a2cf0 0x7f4c0c1a31a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:12:17.682 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:12:17.680+0000 7f4c0b400640 1 -- 172.21.15.22:0/1869886502 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f4c0c10cea0 msgr2=0x7f4c0c19e9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:12:17.682 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:12:17.680+0000 7f4c0b400640 1 --2- 172.21.15.22:0/1869886502 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f4c0c10cea0 0x7f4c0c19e9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:12:17.682 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:12:17.680+0000 7f4c0b400640 1 -- 172.21.15.22:0/1869886502 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c0c1a37e0 con 0x7f4c0c107e50 2024-03-28T18:12:17.682 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:12:17.680+0000 7f4c0be00640 1 --2- 172.21.15.22:0/1869886502 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4c0c1a2cf0 0x7f4c0c1a31a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T18:12:17.682 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:12:17.681+0000 7f4c0b400640 1 --2- 172.21.15.22:0/1869886502 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4c0c107e50 0x7f4c0c19e480 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7f4bfc00e9c0 tx=0x7f4bfc00ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:12:17.682 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:12:17.681+0000 7f4c08c00640 1 -- 172.21.15.22:0/1869886502 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4bfc00cde0 con 0x7f4c0c107e50 2024-03-28T18:12:17.682 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:12:17.681+0000 7f4c08c00640 1 -- 172.21.15.22:0/1869886502 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f4bfc0035f0 con 0x7f4c0c107e50 2024-03-28T18:12:17.682 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:12:17.681+0000 7f4c11000640 1 -- 172.21.15.22:0/1869886502 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4c0c1a3ad0 con 0x7f4c0c107e50 2024-03-28T18:12:17.682 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:12:17.681+0000 7f4c08c00640 1 -- 172.21.15.22:0/1869886502 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4bfc010670 con 0x7f4c0c107e50 2024-03-28T18:12:17.682 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:12:17.681+0000 7f4c11000640 1 -- 172.21.15.22:0/1869886502 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4c0c1ab280 con 0x7f4c0c107e50 2024-03-28T18:12:17.684 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:12:17.684+0000 7f4c15369740 1 Processor -- start 2024-03-28T18:12:17.684 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:12:17.685+0000 7f4c15369740 1 -- start start 2024-03-28T18:12:17.684 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:12:17.685+0000 7f4c08c00640 1 -- 172.21.15.22:0/1869886502 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 26) v1 ==== 137421+0+0 (secure 0 0 0) 0x7f4bfc0037a0 con 0x7f4c0c107e50 2024-03-28T18:12:17.686 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:12:17.685+0000 7f4c15369740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558bbc6cbdd0 0x558bbc6cab20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:12:17.686 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:12:17.685+0000 7f4c15369740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558bbc6cb0f0 0x558bbc6cb570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:12:17.686 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:12:17.685+0000 7f4c15369740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x558bbc6d4460 0x558bbc6d6850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:12:17.686 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:12:17.685+0000 7f4c15369740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x558bbc62f250 con 0x558bbc6cbdd0 2024-03-28T18:12:17.686 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:12:17.685+0000 7f4c15369740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x558bbc5d7a30 con 0x558bbc6d4460 2024-03-28T18:12:17.686 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:12:17.685+0000 7f4c15369740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x558bbc62b2f0 con 0x558bbc6cb0f0 2024-03-28T18:12:17.686 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:12:17.685+0000 7f4c0be00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558bbc6cbdd0 0x558bbc6cab20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:12:17.686 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:12:17.686+0000 7f4c0be00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558bbc6cbdd0 0x558bbc6cab20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:50416/0 (socket says 172.21.15.22:50416) 2024-03-28T18:12:17.686 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:12:17.686+0000 7f4c0be00640 1 -- 172.21.15.22:0/106517336 learned_addr learned my addr 172.21.15.22:0/106517336 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-28T18:12:17.686 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:12:17.686+0000 7f4c03e00640 1 --2- 172.21.15.22:0/106517336 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x558bbc6d4460 0x558bbc6d6850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:12:17.686 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:12:17.686+0000 7f4c0b400640 1 --2- 172.21.15.22:0/106517336 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558bbc6cb0f0 0x558bbc6cb570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:12:17.687 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:12:17.686+0000 7f4c0be00640 1 -- 172.21.15.22:0/106517336 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558bbc6cb0f0 msgr2=0x558bbc6cb570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T18:12:17.687 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:12:17.686+0000 7f4c0be00640 1 --2- 172.21.15.22:0/106517336 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558bbc6cb0f0 0x558bbc6cb570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:12:17.687 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:12:17.686+0000 7f4c0be00640 1 -- 172.21.15.22:0/106517336 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x558bbc6d4460 msgr2=0x558bbc6d6850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T18:12:17.687 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:12:17.686+0000 7f4c0be00640 1 --2- 172.21.15.22:0/106517336 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x558bbc6d4460 0x558bbc6d6850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:12:17.687 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:12:17.686+0000 7f4c0be00640 1 -- 172.21.15.22:0/106517336 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558bbc5c2690 con 0x558bbc6cbdd0 2024-03-28T18:12:17.687 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:12:17.686+0000 7f4c03e00640 1 --2- 172.21.15.22:0/106517336 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x558bbc6d4460 0x558bbc6d6850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-28T18:12:17.687 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:12:17.686+0000 7f4c0be00640 1 --2- 172.21.15.22:0/106517336 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558bbc6cbdd0 0x558bbc6cab20 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7f4c04009ea0 tx=0x7f4c04009f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=52c8b92824c65913 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:12:17.687 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:12:17.686+0000 7f4c08c00640 1 --2- 172.21.15.22:0/1869886502 >> [v2:172.21.15.22:6800/1357365082,v1:172.21.15.22:6801/1357365082] conn(0x7f4bf00a29f0 0x7f4bf00a4eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:12:17.690 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:12:17.687+0000 7f4c00c00640 1 -- 172.21.15.22:0/106517336 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c04004d20 con 0x558bbc6cbdd0 2024-03-28T18:12:17.691 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:12:17.687+0000 7f4c00c00640 1 -- 172.21.15.22:0/106517336 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f4c040054f0 con 0x558bbc6cbdd0 2024-03-28T18:12:17.691 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:12:17.687+0000 7f4c00c00640 1 -- 172.21.15.22:0/106517336 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c040118d0 con 0x558bbc6cbdd0 2024-03-28T18:12:17.691 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:12:17.687+0000 7f4c08c00640 1 -- 172.21.15.22:0/1869886502 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(100..100 src has 1..100) v4 ==== 9013+0+0 (secure 0 0 0) 0x7f4bfc0cf490 con 0x7f4c0c107e50 2024-03-28T18:12:17.691 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:12:17.687+0000 7f4c0b400640 1 --2- 172.21.15.22:0/1869886502 >> [v2:172.21.15.22:6800/1357365082,v1:172.21.15.22:6801/1357365082] conn(0x7f4bf00a29f0 0x7f4bf00a4eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:12:17.691 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:12:17.687+0000 7f4c15369740 1 -- 172.21.15.22:0/106517336 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558bbc6cbdd0 msgr2=0x558bbc6cab20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:12:17.691 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:12:17.687+0000 7f4c15369740 1 --2- 172.21.15.22:0/106517336 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558bbc6cbdd0 0x558bbc6cab20 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7f4c04009ea0 tx=0x7f4c04009f90 comp rx=0 tx=0).stop 2024-03-28T18:12:17.691 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:12:17.688+0000 7f4c0b400640 1 --2- 172.21.15.22:0/1869886502 >> [v2:172.21.15.22:6800/1357365082,v1:172.21.15.22:6801/1357365082] conn(0x7f4bf00a29f0 0x7f4bf00a4eb0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x558bbc6cc320 tx=0x7f4bfc000bc0 comp rx=0 tx=0).ready entity=mgr.15522 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:12:17.691 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:12:17.688+0000 7f4c15369740 1 -- 172.21.15.22:0/106517336 shutdown_connections 2024-03-28T18:12:17.691 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:12:17.688+0000 7f4c15369740 1 --2- 172.21.15.22:0/106517336 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x558bbc6d4460 0x558bbc6d6850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:12:17.691 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:12:17.688+0000 7f4c15369740 1 --2- 172.21.15.22:0/106517336 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558bbc6cb0f0 0x558bbc6cb570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:12:17.691 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:12:17.688+0000 7f4c15369740 1 --2- 172.21.15.22:0/106517336 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558bbc6cbdd0 0x558bbc6cab20 unknown :-1 s=CLOSED pgs=713 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:12:17.691 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:12:17.688+0000 7f4c15369740 1 -- 172.21.15.22:0/106517336 >> 172.21.15.22:0/106517336 conn(0x558bbc692200 msgr2=0x558bbc6c5b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:12:17.692 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:12:17.688+0000 7f4c15369740 1 -- 172.21.15.22:0/106517336 shutdown_connections 2024-03-28T18:12:17.692 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:12:17.688+0000 7f4c15369740 1 -- 172.21.15.22:0/106517336 wait complete. 2024-03-28T18:12:17.692 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:12:17.688+0000 7f4c15369740 1 -- start start 2024-03-28T18:12:17.692 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:12:17.688+0000 7f4c15369740 1 Processor -- start 2024-03-28T18:12:17.692 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:12:17.689+0000 7f4c15369740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558bbc6cb0f0 0x558bbc6d3b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:12:17.692 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:12:17.689+0000 7f4c15369740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558bbc6cbdd0 0x558bbc6d0ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:12:17.692 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:12:17.689+0000 7f4c15369740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x558bbc6d4460 0x558bbc6d1430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:12:17.692 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:12:17.689+0000 7f4c15369740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x558bbc6d72d0 con 0x558bbc6cb0f0 2024-03-28T18:12:17.692 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:12:17.689+0000 7f4c15369740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x558bbc5eae80 con 0x558bbc6d4460 2024-03-28T18:12:17.692 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:12:17.689+0000 7f4c15369740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x558bbc6d7450 con 0x558bbc6cbdd0 2024-03-28T18:12:17.692 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:12:17.689+0000 7f4c15369740 10 client.0 _ll_register_callbacks cb 0x558bbc5fd130 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-28T18:12:17.692 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:12:17.689+0000 7f4c0be00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558bbc6cb0f0 0x558bbc6d3b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:12:17.692 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:12:17.689+0000 7f4c0be00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558bbc6cb0f0 0x558bbc6d3b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:50426/0 (socket says 172.21.15.22:50426) 2024-03-28T18:12:17.692 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:12:17.689+0000 7f4c0be00640 1 -- 172.21.15.22:0/3026152575 learned_addr learned my addr 172.21.15.22:0/3026152575 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-28T18:12:17.693 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:12:17.690+0000 7f4c03e00640 1 --2- 172.21.15.22:0/3026152575 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558bbc6cbdd0 0x558bbc6d0ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:12:17.693 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:12:17.690+0000 7f4c0be00640 1 --2- 172.21.15.22:0/3026152575 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x558bbc6d4460 0x558bbc6d1430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:12:17.693 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:12:17.690+0000 7f4c0be00640 1 -- 172.21.15.22:0/3026152575 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558bbc6cbdd0 msgr2=0x558bbc6d0ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T18:12:17.693 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:12:17.690+0000 7f4c0be00640 1 --2- 172.21.15.22:0/3026152575 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558bbc6cbdd0 0x558bbc6d0ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:12:17.693 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:12:17.690+0000 7f4c0be00640 1 -- 172.21.15.22:0/3026152575 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x558bbc6d4460 msgr2=0x558bbc6d1430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T18:12:17.693 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:12:17.690+0000 7f4c0be00640 1 --2- 172.21.15.22:0/3026152575 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x558bbc6d4460 0x558bbc6d1430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:12:17.693 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:12:17.690+0000 7f4c0be00640 1 -- 172.21.15.22:0/3026152575 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x558bbc40d7d0 con 0x558bbc6cb0f0 2024-03-28T18:12:17.693 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:12:17.690+0000 7f4c0be00640 1 --2- 172.21.15.22:0/3026152575 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558bbc6cb0f0 0x558bbc6d3b60 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7f4c040114f0 tx=0x7f4c04011480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1b9ac4382d25ffcd server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:12:17.693 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:12:17.690+0000 7f4bf9600640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-03-28T18:12:17.693 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:12:17.691+0000 7f4bf9600640 1 -- 172.21.15.22:0/3026152575 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c04021040 con 0x558bbc6cb0f0 2024-03-28T18:12:17.693 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:12:17.691+0000 7f4bf9600640 1 -- 172.21.15.22:0/3026152575 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(100..100 src has 1..100) v4 ==== 9013+0+0 (secure 0 0 0) 0x7f4c04004a40 con 0x558bbc6cb0f0 2024-03-28T18:12:17.693 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:12:17.691+0000 7f4c15369740 1 -- 172.21.15.22:0/3026152575 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f4c040054f0 con 0x558bbc6cb0f0 2024-03-28T18:12:17.693 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:12:17.691+0000 7f4bf9600640 1 -- 172.21.15.22:0/3026152575 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=78) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4c04004c40 con 0x558bbc6cb0f0 2024-03-28T18:12:17.693 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:12:17.691+0000 7f4c15369740 10 client.16572 fetch_fsmap learned FSMap version 78 2024-03-28T18:12:17.693 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:12:17.691+0000 7f4c15369740 1 -- 172.21.15.22:0/3026152575 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=78,monmap=4+}) v3 -- 0x558bbc3433f0 con 0x558bbc6cb0f0 2024-03-28T18:12:17.694 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:12:17.691+0000 7f4bf9600640 1 -- 172.21.15.22:0/3026152575 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f4c040054f0 con 0x558bbc6cb0f0 2024-03-28T18:12:17.695 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:12:17.691+0000 7f4bf9600640 1 -- 172.21.15.22:0/3026152575 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 78) v1 ==== 7950+0+0 (secure 0 0 0) 0x7f4c04027070 con 0x558bbc6cb0f0 2024-03-28T18:12:17.695 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:12:17.691+0000 7f4c15369740 10 client.16572 fetch_fsmap finished waiting for FSMap version 78 2024-03-28T18:12:17.695 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:12:17.691+0000 7f4c15369740 10 client.16572 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-28T18:12:17.695 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:12:17.692+0000 7f4c15369740 20 client.16572 populate_metadata read hostname 'smithi022' 2024-03-28T18:12:17.696 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:12:17.692+0000 7f4c15369740 1 --2- 172.21.15.22:0/3026152575 >> [v2:172.21.15.79:6838/2441542443,v1:172.21.15.79:6839/2441542443] conn(0x558bbc6e4210 0x558bbc6e6600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:12:17.696 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:12:17.692+0000 7f4c15369740 4 client.16572 mds_command: new command op to 34553 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-28T18:12:17.696 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:12:17.692+0000 7f4c15369740 1 -- 172.21.15.22:0/3026152575 --> [v2:172.21.15.79:6838/2441542443,v1:172.21.15.79:6839/2441542443] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x558bbc63c790 con 0x558bbc6e4210 2024-03-28T18:12:17.696 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:12:17.692+0000 7f4c03e00640 1 --2- 172.21.15.22:0/3026152575 >> [v2:172.21.15.79:6838/2441542443,v1:172.21.15.79:6839/2441542443] conn(0x558bbc6e4210 0x558bbc6e6600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:12:17.696 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:12:17.693+0000 7f4c03e00640 1 --2- 172.21.15.22:0/3026152575 >> [v2:172.21.15.79:6838/2441542443,v1:172.21.15.79:6839/2441542443] conn(0x558bbc6e4210 0x558bbc6e6600 crc :-1 s=READY pgs=147 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c625178441c4b3e4 server_cookie=38b69205517fc390 in_seq=0 out_seq=0 2024-03-28T18:12:17.696 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:12:17.693+0000 7f4bf9600640 10 client.16572 ms_handle_connect on v2:172.21.15.79:6838/2441542443 2024-03-28T18:12:17.696 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:12:17.694+0000 7f4bf9600640 1 -- 172.21.15.22:0/3026152575 <== mds.0 v2:172.21.15.79:6838/2441542443 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x558bbc63c790 con 0x558bbc6e4210 2024-03-28T18:12:17.696 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:12:17.694+0000 7f4bf9600640 10 client.16572 handle_command_reply: tid=0 2024-03-28T18:12:17.696 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:12:17.694+0000 7f4c15369740 1 client.16572 shutdown 2024-03-28T18:12:17.696 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:12:17.695+0000 7f4c15369740 1 -- 172.21.15.22:0/3026152575 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558bbc6cb0f0 msgr2=0x558bbc6d3b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:12:17.696 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:12:17.695+0000 7f4c15369740 1 --2- 172.21.15.22:0/3026152575 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558bbc6cb0f0 0x558bbc6d3b60 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7f4c040114f0 tx=0x7f4c04011480 comp rx=0 tx=0).stop 2024-03-28T18:12:17.696 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:12:17.695+0000 7f4c15369740 1 -- 172.21.15.22:0/3026152575 shutdown_connections 2024-03-28T18:12:17.696 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:12:17.695+0000 7f4c15369740 1 --2- 172.21.15.22:0/3026152575 >> [v2:172.21.15.79:6838/2441542443,v1:172.21.15.79:6839/2441542443] conn(0x558bbc6e4210 0x558bbc6e6600 crc :-1 s=READY pgs=147 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:12:17.696 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:12:17.695+0000 7f4c15369740 1 --2- 172.21.15.22:0/3026152575 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x558bbc6d4460 0x558bbc6d1430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:12:17.696 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:12:17.695+0000 7f4c15369740 1 --2- 172.21.15.22:0/3026152575 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558bbc6cbdd0 0x558bbc6d0ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:12:17.697 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:12:17.695+0000 7f4c15369740 1 --2- 172.21.15.22:0/3026152575 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558bbc6cb0f0 0x558bbc6d3b60 unknown :-1 s=CLOSED pgs=714 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:12:17.697 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:12:17.695+0000 7f4bf9600640 0 client.16572 ms_handle_reset on v2:172.21.15.79:6838/2441542443 2024-03-28T18:12:17.697 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:12:17.695+0000 7f4c15369740 1 -- 172.21.15.22:0/3026152575 >> 172.21.15.22:0/3026152575 conn(0x558bbc692200 msgr2=0x558bbc6c7890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:12:17.697 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:12:17.695+0000 7f4c15369740 1 -- 172.21.15.22:0/3026152575 shutdown_connections 2024-03-28T18:12:17.697 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:12:17.695+0000 7f4c15369740 1 -- 172.21.15.22:0/3026152575 wait complete. 2024-03-28T18:12:17.697 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:12:17.695+0000 7f4c15369740 20 client.16572 trim_cache size 0 max 16384 2024-03-28T18:12:17.714 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:12:17.713+0000 7f4c15369740 1 Processor -- start 2024-03-28T18:12:17.714 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:12:17.713+0000 7f4c15369740 1 -- start start 2024-03-28T18:12:17.714 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:12:17.713+0000 7f4c15369740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558bbc6cb0f0 0x558bbc6d3620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:12:17.714 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:12:17.713+0000 7f4c15369740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558bbc6cbdd0 0x558bbc6d3b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:12:17.715 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:12:17.713+0000 7f4c15369740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x558bbc6d4460 0x558bbc6d5850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:12:17.715 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:12:17.713+0000 7f4c15369740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c0c1b3b50 con 0x558bbc6cbdd0 2024-03-28T18:12:17.715 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:12:17.713+0000 7f4c15369740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c0c1b5590 con 0x558bbc6d4460 2024-03-28T18:12:17.715 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:12:17.713+0000 7f4c15369740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c0c1b41b0 con 0x558bbc6cb0f0 2024-03-28T18:12:17.715 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:12:17.714+0000 7f4c03e00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558bbc6cbdd0 0x558bbc6d3b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:12:17.715 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:12:17.714+0000 7f4c03e00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558bbc6cbdd0 0x558bbc6d3b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:50438/0 (socket says 172.21.15.22:50438) 2024-03-28T18:12:17.715 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:12:17.714+0000 7f4c03e00640 1 -- 172.21.15.22:0/3524238774 learned_addr learned my addr 172.21.15.22:0/3524238774 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-28T18:12:17.715 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:12:17.714+0000 7f4c0be00640 1 --2- 172.21.15.22:0/3524238774 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558bbc6cb0f0 0x558bbc6d3620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:12:17.715 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:12:17.714+0000 7f4c0be00640 1 --2- 172.21.15.22:0/3524238774 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x558bbc6d4460 0x558bbc6d5850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:12:17.715 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:12:17.714+0000 7f4c03e00640 1 -- 172.21.15.22:0/3524238774 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558bbc6cb0f0 msgr2=0x558bbc6d3620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T18:12:17.715 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:12:17.714+0000 7f4c03e00640 1 --2- 172.21.15.22:0/3524238774 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558bbc6cb0f0 0x558bbc6d3620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:12:17.715 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:12:17.714+0000 7f4c03e00640 1 -- 172.21.15.22:0/3524238774 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x558bbc6d4460 msgr2=0x558bbc6d5850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T18:12:17.715 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:12:17.714+0000 7f4c03e00640 1 --2- 172.21.15.22:0/3524238774 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x558bbc6d4460 0x558bbc6d5850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:12:17.715 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:12:17.714+0000 7f4c03e00640 1 -- 172.21.15.22:0/3524238774 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558bbc3030f0 con 0x558bbc6cbdd0 2024-03-28T18:12:17.716 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:12:17.714+0000 7f4c0be00640 1 --2- 172.21.15.22:0/3524238774 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x558bbc6d4460 0x558bbc6d5850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T18:12:17.716 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:12:17.714+0000 7f4c03e00640 1 --2- 172.21.15.22:0/3524238774 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558bbc6cbdd0 0x558bbc6d3b60 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7f4bf4009830 tx=0x7f4bf4005d20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d79dc691903b4837 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:12:17.716 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:12:17.715+0000 7f4bfbe00640 1 -- 172.21.15.22:0/3524238774 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4bf4047070 con 0x558bbc6cbdd0 2024-03-28T18:12:17.716 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:12:17.715+0000 7f4bfbe00640 1 -- 172.21.15.22:0/3524238774 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f4bf4043300 con 0x558bbc6cbdd0 2024-03-28T18:12:17.716 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:12:17.715+0000 7f4bfbe00640 1 -- 172.21.15.22:0/3524238774 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4bf4039040 con 0x558bbc6cbdd0 2024-03-28T18:12:17.719 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:12:17.715+0000 7f4c15369740 1 -- 172.21.15.22:0/3524238774 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558bbc6cbdd0 msgr2=0x558bbc6d3b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:12:17.719 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:12:17.715+0000 7f4c15369740 1 --2- 172.21.15.22:0/3524238774 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558bbc6cbdd0 0x558bbc6d3b60 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7f4bf4009830 tx=0x7f4bf4005d20 comp rx=0 tx=0).stop 2024-03-28T18:12:17.719 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:12:17.715+0000 7f4c15369740 1 -- 172.21.15.22:0/3524238774 shutdown_connections 2024-03-28T18:12:17.719 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:12:17.715+0000 7f4c15369740 1 --2- 172.21.15.22:0/3524238774 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x558bbc6d4460 0x558bbc6d5850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:12:17.719 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:12:17.715+0000 7f4c15369740 1 --2- 172.21.15.22:0/3524238774 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558bbc6cbdd0 0x558bbc6d3b60 unknown :-1 s=CLOSED pgs=715 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:12:17.719 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:12:17.715+0000 7f4c15369740 1 --2- 172.21.15.22:0/3524238774 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558bbc6cb0f0 0x558bbc6d3620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:12:17.719 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:12:17.715+0000 7f4c15369740 1 -- 172.21.15.22:0/3524238774 >> 172.21.15.22:0/3524238774 conn(0x558bbc692200 msgr2=0x558bbc6cf7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:12:17.719 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:12:17.715+0000 7f4c15369740 1 -- 172.21.15.22:0/3524238774 shutdown_connections 2024-03-28T18:12:17.719 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:12:17.715+0000 7f4c15369740 1 -- 172.21.15.22:0/3524238774 wait complete. 2024-03-28T18:12:17.719 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:12:17.716+0000 7f4c15369740 1 -- start start 2024-03-28T18:12:17.719 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:12:17.716+0000 7f4c15369740 1 Processor -- start 2024-03-28T18:12:17.720 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:12:17.716+0000 7f4c15369740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558bbc6cb0f0 0x558bbc6ce200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:12:17.720 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:12:17.716+0000 7f4c15369740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x558bbc6cbdd0 0x558bbc6ce740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:12:17.720 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:12:17.716+0000 7f4c15369740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558bbc6d4460 0x558bbc6d4e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:12:17.720 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:12:17.716+0000 7f4c15369740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c0c1b4f30 con 0x558bbc6d4460 2024-03-28T18:12:17.720 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:12:17.716+0000 7f4c15369740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c0c1b4810 con 0x558bbc6cbdd0 2024-03-28T18:12:17.720 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:12:17.716+0000 7f4c15369740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c0c1b1d10 con 0x558bbc6cb0f0 2024-03-28T18:12:17.720 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:12:17.717+0000 7f4c15369740 10 client.0 _ll_register_callbacks cb 0x7f4c04009cf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-28T18:12:17.720 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:12:17.717+0000 7f4c0be00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558bbc6d4460 0x558bbc6d4e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:12:17.720 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:12:17.717+0000 7f4c0be00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558bbc6d4460 0x558bbc6d4e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:50454/0 (socket says 172.21.15.22:50454) 2024-03-28T18:12:17.720 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:12:17.717+0000 7f4c0be00640 1 -- 172.21.15.22:0/2739451071 learned_addr learned my addr 172.21.15.22:0/2739451071 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-28T18:12:17.720 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:12:17.717+0000 7f4c03e00640 1 --2- 172.21.15.22:0/2739451071 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x558bbc6cbdd0 0x558bbc6ce740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:12:17.720 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:12:17.717+0000 7f4c0be00640 1 --2- 172.21.15.22:0/2739451071 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558bbc6cb0f0 0x558bbc6ce200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:12:17.720 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:12:17.717+0000 7f4c0be00640 1 -- 172.21.15.22:0/2739451071 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558bbc6cb0f0 msgr2=0x558bbc6ce200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T18:12:17.720 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:12:17.717+0000 7f4c0be00640 1 --2- 172.21.15.22:0/2739451071 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558bbc6cb0f0 0x558bbc6ce200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:12:17.721 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:12:17.717+0000 7f4c0be00640 1 -- 172.21.15.22:0/2739451071 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x558bbc6cbdd0 msgr2=0x558bbc6ce740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T18:12:17.721 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:12:17.717+0000 7f4c0be00640 1 --2- 172.21.15.22:0/2739451071 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x558bbc6cbdd0 0x558bbc6ce740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:12:17.721 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:12:17.717+0000 7f4c0be00640 1 -- 172.21.15.22:0/2739451071 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x558bbc6d5360 con 0x558bbc6d4460 2024-03-28T18:12:17.721 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:12:17.718+0000 7f4c0be00640 1 --2- 172.21.15.22:0/2739451071 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558bbc6d4460 0x558bbc6d4e20 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7f4c040114f0 tx=0x7f4c04011480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5e633fc012515690 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:12:17.721 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:12:17.718+0000 7f4bf9600640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-03-28T18:12:17.721 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:12:17.718+0000 7f4bf9600640 1 -- 172.21.15.22:0/2739451071 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c04029c90 con 0x558bbc6d4460 2024-03-28T18:12:17.721 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:12:17.718+0000 7f4c15369740 1 -- 172.21.15.22:0/2739451071 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f4bf4043300 con 0x558bbc6d4460 2024-03-28T18:12:17.721 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:12:17.718+0000 7f4bf9600640 1 -- 172.21.15.22:0/2739451071 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(100..100 src has 1..100) v4 ==== 9013+0+0 (secure 0 0 0) 0x7f4c040051e0 con 0x558bbc6d4460 2024-03-28T18:12:17.721 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:12:17.718+0000 7f4bf9600640 1 -- 172.21.15.22:0/2739451071 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=78) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4c04029e30 con 0x558bbc6d4460 2024-03-28T18:12:17.721 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:12:17.718+0000 7f4c15369740 10 client.16578 fetch_fsmap learned FSMap version 78 2024-03-28T18:12:17.721 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:12:17.718+0000 7f4c15369740 1 -- 172.21.15.22:0/2739451071 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=78,monmap=4+}) v3 -- 0x558bbc740040 con 0x558bbc6d4460 2024-03-28T18:12:17.721 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:12:17.719+0000 7f4bf9600640 1 -- 172.21.15.22:0/2739451071 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f4bf4043300 con 0x558bbc6d4460 2024-03-28T18:12:17.721 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:12:17.719+0000 7f4bf9600640 1 -- 172.21.15.22:0/2739451071 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 78) v1 ==== 7950+0+0 (secure 0 0 0) 0x7f4c04027070 con 0x558bbc6d4460 2024-03-28T18:12:17.721 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:12:17.719+0000 7f4c15369740 10 client.16578 fetch_fsmap finished waiting for FSMap version 78 2024-03-28T18:12:17.722 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:12:17.719+0000 7f4c15369740 10 client.16578 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-28T18:12:17.722 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:12:17.719+0000 7f4c15369740 20 client.16578 populate_metadata read hostname 'smithi022' 2024-03-28T18:12:17.722 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:12:17.719+0000 7f4c15369740 1 --2- 172.21.15.22:0/2739451071 >> [v2:172.21.15.79:6838/2441542443,v1:172.21.15.79:6839/2441542443] conn(0x558bbc740750 0x558bbc742b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:12:17.722 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:12:17.719+0000 7f4c15369740 4 client.16578 mds_command: new command op to 34553 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-28T18:12:17.722 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:12:17.719+0000 7f4c15369740 1 -- 172.21.15.22:0/2739451071 --> [v2:172.21.15.79:6838/2441542443,v1:172.21.15.79:6839/2441542443] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x558bbc636ae0 con 0x558bbc740750 2024-03-28T18:12:17.723 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:12:17.720+0000 7f4c03e00640 1 --2- 172.21.15.22:0/2739451071 >> [v2:172.21.15.79:6838/2441542443,v1:172.21.15.79:6839/2441542443] conn(0x558bbc740750 0x558bbc742b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:12:17.723 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:12:17.720+0000 7f4c03e00640 1 --2- 172.21.15.22:0/2739451071 >> [v2:172.21.15.79:6838/2441542443,v1:172.21.15.79:6839/2441542443] conn(0x558bbc740750 0x558bbc742b70 crc :-1 s=READY pgs=148 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8086a6570e0ce987 server_cookie=3a5df4a073579710 in_seq=0 out_seq=0 2024-03-28T18:12:17.723 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:12:17.720+0000 7f4bf9600640 10 client.16578 ms_handle_connect on v2:172.21.15.79:6838/2441542443 2024-03-28T18:12:17.723 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:12:17.721+0000 7f4bf9600640 1 -- 172.21.15.22:0/2739451071 <== mds.0 v2:172.21.15.79:6838/2441542443 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x558bbc636ae0 con 0x558bbc740750 2024-03-28T18:12:17.724 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:12:17.721+0000 7f4bf9600640 10 client.16578 handle_command_reply: tid=0 2024-03-28T18:12:17.724 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:12:17.721+0000 7f4c15369740 1 client.16578 shutdown 2024-03-28T18:12:17.724 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:12:17.721+0000 7f4c15369740 1 -- 172.21.15.22:0/2739451071 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558bbc6d4460 msgr2=0x558bbc6d4e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:12:17.724 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:12:17.721+0000 7f4c15369740 1 --2- 172.21.15.22:0/2739451071 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558bbc6d4460 0x558bbc6d4e20 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7f4c040114f0 tx=0x7f4c04011480 comp rx=0 tx=0).stop 2024-03-28T18:12:17.724 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:12:17.721+0000 7f4c15369740 1 -- 172.21.15.22:0/2739451071 shutdown_connections 2024-03-28T18:12:17.724 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:12:17.721+0000 7f4c15369740 1 --2- 172.21.15.22:0/2739451071 >> [v2:172.21.15.79:6838/2441542443,v1:172.21.15.79:6839/2441542443] conn(0x558bbc740750 0x558bbc742b70 crc :-1 s=READY pgs=148 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:12:17.724 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:12:17.721+0000 7f4c15369740 1 --2- 172.21.15.22:0/2739451071 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558bbc6d4460 0x558bbc6d4e20 unknown :-1 s=CLOSED pgs=716 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:12:17.724 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:12:17.721+0000 7f4c15369740 1 --2- 172.21.15.22:0/2739451071 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x558bbc6cbdd0 0x558bbc6ce740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:12:17.724 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:12:17.721+0000 7f4c15369740 1 --2- 172.21.15.22:0/2739451071 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558bbc6cb0f0 0x558bbc6ce200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:12:17.724 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:12:17.721+0000 7f4c15369740 1 -- 172.21.15.22:0/2739451071 >> 172.21.15.22:0/2739451071 conn(0x558bbc692200 msgr2=0x558bbc6c6f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:12:17.724 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:12:17.721+0000 7f4bf9600640 0 client.16578 ms_handle_reset on v2:172.21.15.79:6838/2441542443 2024-03-28T18:12:17.724 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:12:17.721+0000 7f4c15369740 1 -- 172.21.15.22:0/2739451071 shutdown_connections 2024-03-28T18:12:17.724 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:12:17.721+0000 7f4c15369740 1 -- 172.21.15.22:0/2739451071 wait complete. 2024-03-28T18:12:17.724 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:12:17.721+0000 7f4c15369740 20 client.16578 trim_cache size 0 max 16384 2024-03-28T18:12:17.725 INFO:teuthology.orchestra.run.smithi022.stdout:{ 2024-03-28T18:12:17.725 INFO:teuthology.orchestra.run.smithi022.stdout: "status": "IDLE (waiting for more scrubs)", 2024-03-28T18:12:17.725 INFO:teuthology.orchestra.run.smithi022.stdout: "scrubs": { 2024-03-28T18:12:17.725 INFO:teuthology.orchestra.run.smithi022.stdout: "4bbbd61d-29a3-4bf6-a4f2-d5783d585de3": { 2024-03-28T18:12:17.725 INFO:teuthology.orchestra.run.smithi022.stdout: "path": "/", 2024-03-28T18:12:17.725 INFO:teuthology.orchestra.run.smithi022.stdout: "tag": "4bbbd61d-29a3-4bf6-a4f2-d5783d585de3", 2024-03-28T18:12:17.725 INFO:teuthology.orchestra.run.smithi022.stdout: "options": "recursive,force" 2024-03-28T18:12:17.725 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-03-28T18:12:17.725 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-03-28T18:12:17.725 INFO:teuthology.orchestra.run.smithi022.stdout:} 2024-03-28T18:12:17.726 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:12:17.722+0000 7f4c11000640 1 -- 172.21.15.22:0/1869886502 >> [v2:172.21.15.22:6800/1357365082,v1:172.21.15.22:6801/1357365082] conn(0x7f4bf00a29f0 msgr2=0x7f4bf00a4eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:12:17.726 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:12:17.722+0000 7f4c11000640 1 --2- 172.21.15.22:0/1869886502 >> [v2:172.21.15.22:6800/1357365082,v1:172.21.15.22:6801/1357365082] conn(0x7f4bf00a29f0 0x7f4bf00a4eb0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x558bbc6cc320 tx=0x7f4bfc000bc0 comp rx=0 tx=0).stop 2024-03-28T18:12:17.726 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:12:17.722+0000 7f4c11000640 1 -- 172.21.15.22:0/1869886502 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4c0c107e50 msgr2=0x7f4c0c19e480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:12:17.726 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:12:17.722+0000 7f4c11000640 1 --2- 172.21.15.22:0/1869886502 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4c0c107e50 0x7f4c0c19e480 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7f4bfc00e9c0 tx=0x7f4bfc00ee80 comp rx=0 tx=0).stop 2024-03-28T18:12:17.726 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:12:17.722+0000 7f4c11000640 1 -- 172.21.15.22:0/1869886502 shutdown_connections 2024-03-28T18:12:17.726 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:12:17.722+0000 7f4c11000640 1 --2- 172.21.15.22:0/1869886502 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4c0c1a2cf0 0x7f4c0c1a31a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:12:17.727 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:12:17.722+0000 7f4c11000640 1 --2- 172.21.15.22:0/1869886502 >> [v2:172.21.15.22:6800/1357365082,v1:172.21.15.22:6801/1357365082] conn(0x7f4bf00a29f0 0x7f4bf00a4eb0 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:12:17.727 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:12:17.722+0000 7f4c11000640 1 --2- 172.21.15.22:0/1869886502 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f4c0c10cea0 0x7f4c0c19e9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:12:17.727 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:12:17.722+0000 7f4c11000640 1 --2- 172.21.15.22:0/1869886502 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4c0c107e50 0x7f4c0c19e480 unknown :-1 s=CLOSED pgs=712 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:12:17.727 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:12:17.722+0000 7f4c11000640 1 -- 172.21.15.22:0/1869886502 >> 172.21.15.22:0/1869886502 conn(0x7f4c0c0774c0 msgr2=0x7f4c0c079110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:12:17.727 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:12:17.722+0000 7f4c11000640 1 -- 172.21.15.22:0/1869886502 shutdown_connections 2024-03-28T18:12:17.727 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:12:17.722+0000 7f4c11000640 1 -- 172.21.15.22:0/1869886502 wait complete. 2024-03-28T18:12:17.739 INFO:tasks.cephfs.filesystem:scrub status for tag:4bbbd61d-29a3-4bf6-a4f2-d5783d585de3 - {'path': '/', 'tag': '4bbbd61d-29a3-4bf6-a4f2-d5783d585de3', 'options': 'recursive,force'} 2024-03-28T18:12:19.174 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:12:18 smithi134 ceph-mon[35037]: pgmap v152: 97 pgs: 97 active+clean; 117 MiB data, 785 MiB used, 1.0 TiB / 1.0 TiB avail; 118 KiB/s rd, 31 KiB/s wr, 41 op/s 2024-03-28T18:12:19.182 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:12:18 smithi079 ceph-mon[35312]: pgmap v152: 97 pgs: 97 active+clean; 117 MiB data, 785 MiB used, 1.0 TiB / 1.0 TiB avail; 118 KiB/s rd, 31 KiB/s wr, 41 op/s 2024-03-28T18:12:19.282 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:12:18 smithi022 ceph-mon[31985]: pgmap v152: 97 pgs: 97 active+clean; 117 MiB data, 785 MiB used, 1.0 TiB / 1.0 TiB avail; 118 KiB/s rd, 31 KiB/s wr, 41 op/s 2024-03-28T18:12:21.175 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:12:20 smithi134 ceph-mon[35037]: pgmap v153: 97 pgs: 97 active+clean; 117 MiB data, 785 MiB used, 1.0 TiB / 1.0 TiB avail; 102 KiB/s rd, 26 KiB/s wr, 39 op/s 2024-03-28T18:12:21.182 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:12:20 smithi079 ceph-mon[35312]: pgmap v153: 97 pgs: 97 active+clean; 117 MiB data, 785 MiB used, 1.0 TiB / 1.0 TiB avail; 102 KiB/s rd, 26 KiB/s wr, 39 op/s 2024-03-28T18:12:21.281 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:12:20 smithi022 ceph-mon[31985]: pgmap v153: 97 pgs: 97 active+clean; 117 MiB data, 785 MiB used, 1.0 TiB / 1.0 TiB avail; 102 KiB/s rd, 26 KiB/s wr, 39 op/s 2024-03-28T18:12:23.174 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:12:22 smithi134 ceph-mon[35037]: pgmap v154: 97 pgs: 97 active+clean; 117 MiB data, 785 MiB used, 1.0 TiB / 1.0 TiB avail; 99 KiB/s rd, 26 KiB/s wr, 35 op/s 2024-03-28T18:12:23.182 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:12:22 smithi079 ceph-mon[35312]: pgmap v154: 97 pgs: 97 active+clean; 117 MiB data, 785 MiB used, 1.0 TiB / 1.0 TiB avail; 99 KiB/s rd, 26 KiB/s wr, 35 op/s 2024-03-28T18:12:23.282 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:12:22 smithi022 ceph-mon[31985]: pgmap v154: 97 pgs: 97 active+clean; 117 MiB data, 785 MiB used, 1.0 TiB / 1.0 TiB avail; 99 KiB/s rd, 26 KiB/s wr, 35 op/s 2024-03-28T18:12:23.849 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:12:23 smithi134 systemd[1]: ceph-fd220748-ed2b-11ee-95d0-87774f69a715@mds.f.service: Scheduled restart job, restart counter is at 4. 2024-03-28T18:12:23.849 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:12:23 smithi134 systemd[1]: Stopped Ceph mds.f for fd220748-ed2b-11ee-95d0-87774f69a715. 2024-03-28T18:12:23.849 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:12:23 smithi134 systemd[1]: ceph-fd220748-ed2b-11ee-95d0-87774f69a715@mds.f.service: Consumed 1.829s CPU time. 2024-03-28T18:12:23.849 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:12:23 smithi134 systemd[1]: Starting Ceph mds.f for fd220748-ed2b-11ee-95d0-87774f69a715... 2024-03-28T18:12:24.166 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:12:23 smithi134 podman[70486]: 2024-03-28 18:12:23.754501794 +0000 UTC m=+0.033337077 image pull c664afe56bb01db4f9afa9532c20598e201e664f2e9ba2be50db0d52674a26d2 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc 2024-03-28T18:12:24.166 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:12:23 smithi134 podman[70486]: 2024-03-28 18:12:23.879142082 +0000 UTC m=+0.157977403 container create fd770f89545b6274c7fcb94b7a69b6b45042a9397d6080b451ade0b4b9397af3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f, org.label-schema.build-date=20240319, org.label-schema.vendor=CentOS, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0) 2024-03-28T18:12:24.166 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:12:24 smithi134 podman[70486]: 2024-03-28 18:12:24.165768711 +0000 UTC m=+0.444604039 container init fd770f89545b6274c7fcb94b7a69b6b45042a9397d6080b451ade0b4b9397af3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_CLEAN=True, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.build-date=20240319, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS) 2024-03-28T18:12:24.425 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:12:24 smithi134 podman[70486]: 2024-03-28 18:12:24.17618685 +0000 UTC m=+0.455022140 container start fd770f89545b6274c7fcb94b7a69b6b45042a9397d6080b451ade0b4b9397af3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, ceph=True, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, org.label-schema.build-date=20240319, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS) 2024-03-28T18:12:24.425 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:12:24 smithi134 ceph-mds[70505]: -- 172.21.15.134:0/3867345385 <== mon.1 v2:172.21.15.134:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55bce76bba40 con 0x55bce768f000 2024-03-28T18:12:24.425 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:12:24 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[70501]: starting mds.f at 2024-03-28T18:12:24.425 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:12:24 smithi134 bash[70486]: fd770f89545b6274c7fcb94b7a69b6b45042a9397d6080b451ade0b4b9397af3 2024-03-28T18:12:24.425 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:12:24 smithi134 systemd[1]: Started Ceph mds.f for fd220748-ed2b-11ee-95d0-87774f69a715. 2024-03-28T18:12:25.175 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:12:24 smithi134 ceph-mon[35037]: pgmap v155: 97 pgs: 97 active+clean; 117 MiB data, 785 MiB used, 1.0 TiB / 1.0 TiB avail; 99 KiB/s rd, 26 KiB/s wr, 35 op/s 2024-03-28T18:12:25.175 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:12:24 smithi134 ceph-mon[35037]: daemon mds.f restarted 2024-03-28T18:12:25.175 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:12:24 smithi134 ceph-mon[35037]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-03-28T18:12:25.175 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:12:24 smithi134 ceph-mon[35037]: osdmap e101: 12 total, 12 up, 12 in 2024-03-28T18:12:25.175 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:12:24 smithi134 ceph-mon[35037]: mds.? [v2:172.21.15.134:6834/726536645,v1:172.21.15.134:6835/726536645] up:boot 2024-03-28T18:12:25.175 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:12:24 smithi134 ceph-mon[35037]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-03-28T18:12:25.175 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:12:24 smithi134 ceph-mon[35037]: Standby daemon mds.h assigned to filesystem cephfs as rank 1 2024-03-28T18:12:25.176 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:12:24 smithi134 ceph-mon[35037]: fsmap cephfs:4/5 {0=b=up:active,2=e=up:active,3=k=up:active,4=i=up:active} 5 up:standby-replay 3 up:standby, 1 failed 2024-03-28T18:12:25.176 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:12:24 smithi134 ceph-mon[35037]: fsmap cephfs:5/5 4 up:standby-replay 3 up:standby 1 up:replay 4 up:active 2024-03-28T18:12:25.182 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:12:24 smithi079 ceph-mon[35312]: pgmap v155: 97 pgs: 97 active+clean; 117 MiB data, 785 MiB used, 1.0 TiB / 1.0 TiB avail; 99 KiB/s rd, 26 KiB/s wr, 35 op/s 2024-03-28T18:12:25.182 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:12:24 smithi079 ceph-mon[35312]: daemon mds.f restarted 2024-03-28T18:12:25.183 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:12:24 smithi079 ceph-mon[35312]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-03-28T18:12:25.183 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:12:24 smithi079 ceph-mon[35312]: osdmap e101: 12 total, 12 up, 12 in 2024-03-28T18:12:25.183 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:12:24 smithi079 ceph-mon[35312]: mds.? [v2:172.21.15.134:6834/726536645,v1:172.21.15.134:6835/726536645] up:boot 2024-03-28T18:12:25.183 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:12:24 smithi079 ceph-mon[35312]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-03-28T18:12:25.183 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:12:24 smithi079 ceph-mon[35312]: Standby daemon mds.h assigned to filesystem cephfs as rank 1 2024-03-28T18:12:25.183 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:12:24 smithi079 ceph-mon[35312]: fsmap cephfs:4/5 {0=b=up:active,2=e=up:active,3=k=up:active,4=i=up:active} 5 up:standby-replay 3 up:standby, 1 failed 2024-03-28T18:12:25.183 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:12:24 smithi079 ceph-mon[35312]: fsmap cephfs:5/5 4 up:standby-replay 3 up:standby 1 up:replay 4 up:active 2024-03-28T18:12:25.282 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:12:24 smithi022 ceph-mon[31985]: pgmap v155: 97 pgs: 97 active+clean; 117 MiB data, 785 MiB used, 1.0 TiB / 1.0 TiB avail; 99 KiB/s rd, 26 KiB/s wr, 35 op/s 2024-03-28T18:12:25.282 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:12:24 smithi022 ceph-mon[31985]: daemon mds.f restarted 2024-03-28T18:12:25.282 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:12:24 smithi022 ceph-mon[31985]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-03-28T18:12:25.282 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:12:24 smithi022 ceph-mon[31985]: osdmap e101: 12 total, 12 up, 12 in 2024-03-28T18:12:25.282 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:12:24 smithi022 ceph-mon[31985]: mds.? [v2:172.21.15.134:6834/726536645,v1:172.21.15.134:6835/726536645] up:boot 2024-03-28T18:12:25.282 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:12:24 smithi022 ceph-mon[31985]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-03-28T18:12:25.282 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:12:24 smithi022 ceph-mon[31985]: Standby daemon mds.h assigned to filesystem cephfs as rank 1 2024-03-28T18:12:25.282 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:12:24 smithi022 ceph-mon[31985]: fsmap cephfs:4/5 {0=b=up:active,2=e=up:active,3=k=up:active,4=i=up:active} 5 up:standby-replay 3 up:standby, 1 failed 2024-03-28T18:12:25.283 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:12:24 smithi022 ceph-mon[31985]: fsmap cephfs:5/5 4 up:standby-replay 3 up:standby 1 up:replay 4 up:active 2024-03-28T18:12:27.175 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:12:26 smithi134 ceph-mon[35037]: pgmap v157: 97 pgs: 97 active+clean; 117 MiB data, 785 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 10 op/s 2024-03-28T18:12:27.175 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:12:26 smithi134 ceph-mon[35037]: mds.? [v2:172.21.15.79:6836/3204076941,v1:172.21.15.79:6837/3204076941] up:resolve 2024-03-28T18:12:27.175 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:12:26 smithi134 ceph-mon[35037]: fsmap cephfs:5/5 4 up:standby-replay 3 up:standby 1 up:resolve 4 up:active 2024-03-28T18:12:27.182 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:12:26 smithi079 ceph-mon[35312]: pgmap v157: 97 pgs: 97 active+clean; 117 MiB data, 785 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 10 op/s 2024-03-28T18:12:27.182 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:12:26 smithi079 ceph-mon[35312]: mds.? [v2:172.21.15.79:6836/3204076941,v1:172.21.15.79:6837/3204076941] up:resolve 2024-03-28T18:12:27.182 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:12:26 smithi079 ceph-mon[35312]: fsmap cephfs:5/5 4 up:standby-replay 3 up:standby 1 up:resolve 4 up:active 2024-03-28T18:12:27.282 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:12:26 smithi022 ceph-mon[31985]: pgmap v157: 97 pgs: 97 active+clean; 117 MiB data, 785 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 10 op/s 2024-03-28T18:12:27.282 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:12:26 smithi022 ceph-mon[31985]: mds.? [v2:172.21.15.79:6836/3204076941,v1:172.21.15.79:6837/3204076941] up:resolve 2024-03-28T18:12:27.282 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:12:26 smithi022 ceph-mon[31985]: fsmap cephfs:5/5 4 up:standby-replay 3 up:standby 1 up:resolve 4 up:active 2024-03-28T18:12:28.174 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:12:27 smithi134 ceph-mon[35037]: reconnect by client.35222 172.21.15.22:0/3729116781 after 0.00399991 2024-03-28T18:12:28.175 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:12:27 smithi134 ceph-mon[35037]: mds.? [v2:172.21.15.79:6836/3204076941,v1:172.21.15.79:6837/3204076941] up:reconnect 2024-03-28T18:12:28.175 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:12:27 smithi134 ceph-mon[35037]: fsmap cephfs:5/5 4 up:standby-replay 3 up:standby 1 up:reconnect 4 up:active 2024-03-28T18:12:28.182 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:12:27 smithi079 ceph-mon[35312]: reconnect by client.35222 172.21.15.22:0/3729116781 after 0.00399991 2024-03-28T18:12:28.182 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:12:27 smithi079 ceph-mon[35312]: mds.? [v2:172.21.15.79:6836/3204076941,v1:172.21.15.79:6837/3204076941] up:reconnect 2024-03-28T18:12:28.182 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:12:27 smithi079 ceph-mon[35312]: fsmap cephfs:5/5 4 up:standby-replay 3 up:standby 1 up:reconnect 4 up:active 2024-03-28T18:12:28.282 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:12:27 smithi022 ceph-mon[31985]: reconnect by client.35222 172.21.15.22:0/3729116781 after 0.00399991 2024-03-28T18:12:28.282 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:12:27 smithi022 ceph-mon[31985]: mds.? [v2:172.21.15.79:6836/3204076941,v1:172.21.15.79:6837/3204076941] up:reconnect 2024-03-28T18:12:28.282 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:12:27 smithi022 ceph-mon[31985]: fsmap cephfs:5/5 4 up:standby-replay 3 up:standby 1 up:reconnect 4 up:active 2024-03-28T18:12:29.174 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:12:28 smithi134 ceph-mon[35037]: pgmap v158: 97 pgs: 97 active+clean; 117 MiB data, 785 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 10 op/s 2024-03-28T18:12:29.175 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:12:28 smithi134 ceph-mon[35037]: mds.? [v2:172.21.15.79:6836/3204076941,v1:172.21.15.79:6837/3204076941] up:rejoin 2024-03-28T18:12:29.175 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:12:28 smithi134 ceph-mon[35037]: fsmap cephfs:5/5 4 up:standby-replay 3 up:standby 1 up:rejoin 4 up:active 2024-03-28T18:12:29.175 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:12:28 smithi134 ceph-mon[35037]: daemon mds.h is now active in filesystem cephfs as rank 1 2024-03-28T18:12:29.182 INFO:journalctl@ceph.mds.b.smithi079.stdout:Mar 28 18:12:29 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-b[65245]: 2024-03-28T18:12:29.085+0000 7f845f400700 -1 mds.pinger is_rank_lagging: rank=1 was never sent ping request. 2024-03-28T18:12:29.183 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:12:28 smithi079 ceph-mon[35312]: pgmap v158: 97 pgs: 97 active+clean; 117 MiB data, 785 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 10 op/s 2024-03-28T18:12:29.183 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:12:28 smithi079 ceph-mon[35312]: mds.? [v2:172.21.15.79:6836/3204076941,v1:172.21.15.79:6837/3204076941] up:rejoin 2024-03-28T18:12:29.183 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:12:28 smithi079 ceph-mon[35312]: fsmap cephfs:5/5 4 up:standby-replay 3 up:standby 1 up:rejoin 4 up:active 2024-03-28T18:12:29.183 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:12:28 smithi079 ceph-mon[35312]: daemon mds.h is now active in filesystem cephfs as rank 1 2024-03-28T18:12:29.183 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:12:28 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69741]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-2145-g3be9a5c9/rpm/el8/BUILD/ceph-19.0.0-2145-g3be9a5c9/src/mds/Locker.cc: In function 'void Locker::set_xlocks_done(MutationImpl*, bool)' thread 7faf7b400700 time 2024-03-28T18:12:28.981251+0000 2024-03-28T18:12:29.183 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:12:28 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69741]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-2145-g3be9a5c9/rpm/el8/BUILD/ceph-19.0.0-2145-g3be9a5c9/src/mds/Locker.cc: 626: FAILED ceph_assert(obj->is_auth()) 2024-03-28T18:12:29.183 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:12:28 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69741]: ceph version 19.0.0-2145-g3be9a5c9 (3be9a5c9bc793e11e2800a8c0c696e8b46742033) squid (dev) 2024-03-28T18:12:29.184 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:12:28 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69741]: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x135) [0x7faf86da64a1] 2024-03-28T18:12:29.184 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:12:28 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69741]: 2: /usr/lib64/ceph/libceph-common.so.2(+0x2d2667) [0x7faf86da6667] 2024-03-28T18:12:29.184 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:12:28 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69741]: 3: (Locker::set_xlocks_done(MutationImpl*, bool)+0x493) [0x55c652b89223] 2024-03-28T18:12:29.184 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:12:28 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69741]: 4: (Server::early_reply(boost::intrusive_ptr const&, CInode*, CDentry*)+0x3e6) [0x55c652a035d6] 2024-03-28T18:12:29.184 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:12:28 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69741]: 5: (Server::journal_and_reply(boost::intrusive_ptr const&, CInode*, CDentry*, LogEvent*, MDSLogContextBase*)+0xc1) [0x55c652a03da1] 2024-03-28T18:12:29.184 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:12:28 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69741]: 6: (Server::handle_client_mkdir(boost::intrusive_ptr const&)+0x5ba) [0x55c652a396aa] 2024-03-28T18:12:29.184 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:12:28 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69741]: 7: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0x733) [0x55c652a56233] 2024-03-28T18:12:29.184 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:12:28 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69741]: 8: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x55c652a58197] 2024-03-28T18:12:29.184 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:12:28 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69741]: 9: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x55c65295711b] 2024-03-28T18:12:29.184 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:12:28 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69741]: 10: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x55c6529576bc] 2024-03-28T18:12:29.184 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:12:28 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69741]: 11: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bf) [0x55c6529411ff] 2024-03-28T18:12:29.184 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:12:28 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69741]: 12: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x478) [0x7faf87038488] 2024-03-28T18:12:29.184 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:12:28 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69741]: 13: (DispatchQueue::entry()+0x51f) [0x7faf8703560f] 2024-03-28T18:12:29.184 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:12:28 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69741]: 14: (DispatchQueue::DispatchThread::entry()+0x11) [0x7faf87108e81] 2024-03-28T18:12:29.184 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:12:28 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69741]: 15: /lib64/libpthread.so.0(+0x81ca) [0x7faf85b231ca] 2024-03-28T18:12:29.185 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:12:28 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69741]: 16: clone() 2024-03-28T18:12:29.185 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:12:28 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69741]: 2024-03-28T18:12:28.982+0000 7faf7b400700 -1 /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-2145-g3be9a5c9/rpm/el8/BUILD/ceph-19.0.0-2145-g3be9a5c9/src/mds/Locker.cc: In function 'void Locker::set_xlocks_done(MutationImpl*, bool)' thread 7faf7b400700 time 2024-03-28T18:12:28.981251+0000 2024-03-28T18:12:29.185 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:12:28 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69741]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-2145-g3be9a5c9/rpm/el8/BUILD/ceph-19.0.0-2145-g3be9a5c9/src/mds/Locker.cc: 626: FAILED ceph_assert(obj->is_auth()) 2024-03-28T18:12:29.185 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:12:28 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69741]: 2024-03-28T18:12:29.185 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:12:28 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69741]: ceph version 19.0.0-2145-g3be9a5c9 (3be9a5c9bc793e11e2800a8c0c696e8b46742033) squid (dev) 2024-03-28T18:12:29.185 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:12:28 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69741]: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x135) [0x7faf86da64a1] 2024-03-28T18:12:29.186 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:12:28 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69741]: 2: /usr/lib64/ceph/libceph-common.so.2(+0x2d2667) [0x7faf86da6667] 2024-03-28T18:12:29.186 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:12:28 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69741]: 3: (Locker::set_xlocks_done(MutationImpl*, bool)+0x493) [0x55c652b89223] 2024-03-28T18:12:29.186 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:12:28 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69741]: 4: (Server::early_reply(boost::intrusive_ptr const&, CInode*, CDentry*)+0x3e6) [0x55c652a035d6] 2024-03-28T18:12:29.186 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:12:28 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69741]: 5: (Server::journal_and_reply(boost::intrusive_ptr const&, CInode*, CDentry*, LogEvent*, MDSLogContextBase*)+0xc1) [0x55c652a03da1] 2024-03-28T18:12:29.186 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:12:28 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69741]: 6: (Server::handle_client_mkdir(boost::intrusive_ptr const&)+0x5ba) [0x55c652a396aa] 2024-03-28T18:12:29.186 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:12:28 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69741]: 7: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0x733) [0x55c652a56233] 2024-03-28T18:12:29.186 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:12:28 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69741]: 8: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x55c652a58197] 2024-03-28T18:12:29.186 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:12:28 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69741]: 9: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x55c65295711b] 2024-03-28T18:12:29.186 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:12:28 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69741]: 10: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x55c6529576bc] 2024-03-28T18:12:29.186 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:12:28 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69741]: 11: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bf) [0x55c6529411ff] 2024-03-28T18:12:29.186 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:12:28 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69741]: 12: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x478) [0x7faf87038488] 2024-03-28T18:12:29.186 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:12:28 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69741]: 13: (DispatchQueue::entry()+0x51f) [0x7faf8703560f] 2024-03-28T18:12:29.187 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:12:28 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69741]: 14: (DispatchQueue::DispatchThread::entry()+0x11) [0x7faf87108e81] 2024-03-28T18:12:29.187 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:12:28 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69741]: 15: /lib64/libpthread.so.0(+0x81ca) [0x7faf85b231ca] 2024-03-28T18:12:29.187 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:12:28 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69741]: 16: clone() 2024-03-28T18:12:29.187 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:12:28 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69741]: 2024-03-28T18:12:29.187 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:12:28 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69741]: *** Caught signal (Aborted) ** 2024-03-28T18:12:29.187 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:12:28 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69741]: in thread 7faf7b400700 thread_name:ms_dispatch 2024-03-28T18:12:29.187 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:12:28 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69741]: ceph version 19.0.0-2145-g3be9a5c9 (3be9a5c9bc793e11e2800a8c0c696e8b46742033) squid (dev) 2024-03-28T18:12:29.187 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:12:28 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69741]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7faf85b2dd20] 2024-03-28T18:12:29.187 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:12:28 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69741]: 2: gsignal() 2024-03-28T18:12:29.187 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:12:28 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69741]: 3: abort() 2024-03-28T18:12:29.187 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:12:28 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69741]: 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x18f) [0x7faf86da64fb] 2024-03-28T18:12:29.187 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:12:28 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69741]: 5: /usr/lib64/ceph/libceph-common.so.2(+0x2d2667) [0x7faf86da6667] 2024-03-28T18:12:29.188 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:12:28 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69741]: 6: (Locker::set_xlocks_done(MutationImpl*, bool)+0x493) [0x55c652b89223] 2024-03-28T18:12:29.188 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:12:28 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69741]: 7: (Server::early_reply(boost::intrusive_ptr const&, CInode*, CDentry*)+0x3e6) [0x55c652a035d6] 2024-03-28T18:12:29.188 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:12:28 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69741]: 8: (Server::journal_and_reply(boost::intrusive_ptr const&, CInode*, CDentry*, LogEvent*, MDSLogContextBase*)+0xc1) [0x55c652a03da1] 2024-03-28T18:12:29.188 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:12:28 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69741]: 9: (Server::handle_client_mkdir(boost::intrusive_ptr const&)+0x5ba) [0x55c652a396aa] 2024-03-28T18:12:29.188 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:12:28 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69741]: 10: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0x733) [0x55c652a56233] 2024-03-28T18:12:29.188 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:12:28 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69741]: 11: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x55c652a58197] 2024-03-28T18:12:29.188 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:12:28 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69741]: 12: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x55c65295711b] 2024-03-28T18:12:29.188 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:12:28 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69741]: 13: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x55c6529576bc] 2024-03-28T18:12:29.188 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:12:28 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69741]: 14: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bf) [0x55c6529411ff] 2024-03-28T18:12:29.188 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:12:28 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69741]: 15: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x478) [0x7faf87038488] 2024-03-28T18:12:29.188 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:12:28 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69741]: 16: (DispatchQueue::entry()+0x51f) [0x7faf8703560f] 2024-03-28T18:12:29.188 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:12:28 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69741]: 17: (DispatchQueue::DispatchThread::entry()+0x11) [0x7faf87108e81] 2024-03-28T18:12:29.188 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:12:28 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69741]: 18: /lib64/libpthread.so.0(+0x81ca) [0x7faf85b231ca] 2024-03-28T18:12:29.188 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:12:28 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69741]: 19: clone() 2024-03-28T18:12:29.189 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:12:28 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69741]: 2024-03-28T18:12:28.984+0000 7faf7b400700 -1 *** Caught signal (Aborted) ** 2024-03-28T18:12:29.189 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:12:28 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69741]: in thread 7faf7b400700 thread_name:ms_dispatch 2024-03-28T18:12:29.189 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:12:28 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69741]: 2024-03-28T18:12:29.189 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:12:28 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69741]: ceph version 19.0.0-2145-g3be9a5c9 (3be9a5c9bc793e11e2800a8c0c696e8b46742033) squid (dev) 2024-03-28T18:12:29.189 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:12:28 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69741]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7faf85b2dd20] 2024-03-28T18:12:29.189 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:12:28 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69741]: 2: gsignal() 2024-03-28T18:12:29.189 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:12:28 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69741]: 3: abort() 2024-03-28T18:12:29.189 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:12:28 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69741]: 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x18f) [0x7faf86da64fb] 2024-03-28T18:12:29.189 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:12:28 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69741]: 5: /usr/lib64/ceph/libceph-common.so.2(+0x2d2667) [0x7faf86da6667] 2024-03-28T18:12:29.190 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:12:28 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69741]: 6: (Locker::set_xlocks_done(MutationImpl*, bool)+0x493) [0x55c652b89223] 2024-03-28T18:12:29.190 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:12:28 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69741]: 7: (Server::early_reply(boost::intrusive_ptr const&, CInode*, CDentry*)+0x3e6) [0x55c652a035d6] 2024-03-28T18:12:29.190 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:12:28 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69741]: 8: (Server::journal_and_reply(boost::intrusive_ptr const&, CInode*, CDentry*, LogEvent*, MDSLogContextBase*)+0xc1) [0x55c652a03da1] 2024-03-28T18:12:29.190 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:12:28 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69741]: 9: (Server::handle_client_mkdir(boost::intrusive_ptr const&)+0x5ba) [0x55c652a396aa] 2024-03-28T18:12:29.190 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:12:28 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69741]: 10: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0x733) [0x55c652a56233] 2024-03-28T18:12:29.190 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:12:28 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69741]: 11: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x55c652a58197] 2024-03-28T18:12:29.191 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:12:28 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69741]: 12: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x55c65295711b] 2024-03-28T18:12:29.191 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:12:28 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69741]: 13: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x55c6529576bc] 2024-03-28T18:12:29.191 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:12:28 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69741]: 14: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bf) [0x55c6529411ff] 2024-03-28T18:12:29.191 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:12:28 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69741]: 15: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x478) [0x7faf87038488] 2024-03-28T18:12:29.191 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:12:28 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69741]: 16: (DispatchQueue::entry()+0x51f) [0x7faf8703560f] 2024-03-28T18:12:29.191 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:12:28 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69741]: 17: (DispatchQueue::DispatchThread::entry()+0x11) [0x7faf87108e81] 2024-03-28T18:12:29.191 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:12:28 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69741]: 18: /lib64/libpthread.so.0(+0x81ca) [0x7faf85b231ca] 2024-03-28T18:12:29.191 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:12:28 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69741]: 19: clone() 2024-03-28T18:12:29.191 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:12:28 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69741]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-03-28T18:12:29.191 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:12:28 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69741]: 2024-03-28T18:12:29.191 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:12:29 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69741]: -1> 2024-03-28T18:12:28.982+0000 7faf7b400700 -1 /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-2145-g3be9a5c9/rpm/el8/BUILD/ceph-19.0.0-2145-g3be9a5c9/src/mds/Locker.cc: In function 'void Locker::set_xlocks_done(MutationImpl*, bool)' thread 7faf7b400700 time 2024-03-28T18:12:28.981251+0000 2024-03-28T18:12:29.191 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:12:29 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69741]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-2145-g3be9a5c9/rpm/el8/BUILD/ceph-19.0.0-2145-g3be9a5c9/src/mds/Locker.cc: 626: FAILED ceph_assert(obj->is_auth()) 2024-03-28T18:12:29.191 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:12:29 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69741]: 2024-03-28T18:12:29.191 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:12:29 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69741]: ceph version 19.0.0-2145-g3be9a5c9 (3be9a5c9bc793e11e2800a8c0c696e8b46742033) squid (dev) 2024-03-28T18:12:29.191 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:12:29 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69741]: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x135) [0x7faf86da64a1] 2024-03-28T18:12:29.192 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:12:29 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69741]: 2: /usr/lib64/ceph/libceph-common.so.2(+0x2d2667) [0x7faf86da6667] 2024-03-28T18:12:29.192 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:12:29 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69741]: 3: (Locker::set_xlocks_done(MutationImpl*, bool)+0x493) [0x55c652b89223] 2024-03-28T18:12:29.192 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:12:29 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69741]: 4: (Server::early_reply(boost::intrusive_ptr const&, CInode*, CDentry*)+0x3e6) [0x55c652a035d6] 2024-03-28T18:12:29.192 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:12:29 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69741]: 5: (Server::journal_and_reply(boost::intrusive_ptr const&, CInode*, CDentry*, LogEvent*, MDSLogContextBase*)+0xc1) [0x55c652a03da1] 2024-03-28T18:12:29.192 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:12:29 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69741]: 6: (Server::handle_client_mkdir(boost::intrusive_ptr const&)+0x5ba) [0x55c652a396aa] 2024-03-28T18:12:29.192 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:12:29 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69741]: 7: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0x733) [0x55c652a56233] 2024-03-28T18:12:29.192 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:12:29 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69741]: 8: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x55c652a58197] 2024-03-28T18:12:29.192 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:12:29 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69741]: 9: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x55c65295711b] 2024-03-28T18:12:29.192 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:12:29 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69741]: 10: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x55c6529576bc] 2024-03-28T18:12:29.192 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:12:29 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69741]: 11: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bf) [0x55c6529411ff] 2024-03-28T18:12:29.192 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:12:29 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69741]: 12: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x478) [0x7faf87038488] 2024-03-28T18:12:29.192 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:12:29 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69741]: 13: (DispatchQueue::entry()+0x51f) [0x7faf8703560f] 2024-03-28T18:12:29.192 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:12:29 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69741]: 14: (DispatchQueue::DispatchThread::entry()+0x11) [0x7faf87108e81] 2024-03-28T18:12:29.192 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:12:29 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69741]: 15: /lib64/libpthread.so.0(+0x81ca) [0x7faf85b231ca] 2024-03-28T18:12:29.192 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:12:29 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69741]: 16: clone() 2024-03-28T18:12:29.193 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:12:29 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69741]: 2024-03-28T18:12:29.193 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:12:29 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69741]: 0> 2024-03-28T18:12:28.984+0000 7faf7b400700 -1 *** Caught signal (Aborted) ** 2024-03-28T18:12:29.193 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:12:29 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69741]: in thread 7faf7b400700 thread_name:ms_dispatch 2024-03-28T18:12:29.193 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:12:29 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69741]: 2024-03-28T18:12:29.193 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:12:29 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69741]: ceph version 19.0.0-2145-g3be9a5c9 (3be9a5c9bc793e11e2800a8c0c696e8b46742033) squid (dev) 2024-03-28T18:12:29.193 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:12:29 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69741]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7faf85b2dd20] 2024-03-28T18:12:29.193 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:12:29 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69741]: 2: gsignal() 2024-03-28T18:12:29.193 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:12:29 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69741]: 3: abort() 2024-03-28T18:12:29.193 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:12:29 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69741]: 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x18f) [0x7faf86da64fb] 2024-03-28T18:12:29.193 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:12:29 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69741]: 5: /usr/lib64/ceph/libceph-common.so.2(+0x2d2667) [0x7faf86da6667] 2024-03-28T18:12:29.193 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:12:29 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69741]: 6: (Locker::set_xlocks_done(MutationImpl*, bool)+0x493) [0x55c652b89223] 2024-03-28T18:12:29.193 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:12:29 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69741]: 7: (Server::early_reply(boost::intrusive_ptr const&, CInode*, CDentry*)+0x3e6) [0x55c652a035d6] 2024-03-28T18:12:29.193 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:12:29 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69741]: 8: (Server::journal_and_reply(boost::intrusive_ptr const&, CInode*, CDentry*, LogEvent*, MDSLogContextBase*)+0xc1) [0x55c652a03da1] 2024-03-28T18:12:29.193 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:12:29 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69741]: 9: (Server::handle_client_mkdir(boost::intrusive_ptr const&)+0x5ba) [0x55c652a396aa] 2024-03-28T18:12:29.193 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:12:29 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69741]: 10: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0x733) [0x55c652a56233] 2024-03-28T18:12:29.194 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:12:29 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69741]: 11: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x55c652a58197] 2024-03-28T18:12:29.194 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:12:29 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69741]: 12: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x55c65295711b] 2024-03-28T18:12:29.194 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:12:29 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69741]: 13: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x55c6529576bc] 2024-03-28T18:12:29.194 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:12:29 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69741]: 14: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bf) [0x55c6529411ff] 2024-03-28T18:12:29.194 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:12:29 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69741]: 15: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x478) [0x7faf87038488] 2024-03-28T18:12:29.194 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:12:29 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69741]: 16: (DispatchQueue::entry()+0x51f) [0x7faf8703560f] 2024-03-28T18:12:29.194 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:12:29 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69741]: 17: (DispatchQueue::DispatchThread::entry()+0x11) [0x7faf87108e81] 2024-03-28T18:12:29.194 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:12:29 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69741]: 18: /lib64/libpthread.so.0(+0x81ca) [0x7faf85b231ca] 2024-03-28T18:12:29.194 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:12:29 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69741]: 19: clone() 2024-03-28T18:12:29.194 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:12:29 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69741]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-03-28T18:12:29.194 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:12:29 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69741]: 2024-03-28T18:12:29.194 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:12:29 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69741]: -1> 2024-03-28T18:12:28.982+0000 7faf7b400700 -1 /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-2145-g3be9a5c9/rpm/el8/BUILD/ceph-19.0.0-2145-g3be9a5c9/src/mds/Locker.cc: In function 'void Locker::set_xlocks_done(MutationImpl*, bool)' thread 7faf7b400700 time 2024-03-28T18:12:28.981251+0000 2024-03-28T18:12:29.194 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:12:29 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69741]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-2145-g3be9a5c9/rpm/el8/BUILD/ceph-19.0.0-2145-g3be9a5c9/src/mds/Locker.cc: 626: FAILED ceph_assert(obj->is_auth()) 2024-03-28T18:12:29.195 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:12:29 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69741]: 2024-03-28T18:12:29.195 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:12:29 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69741]: ceph version 19.0.0-2145-g3be9a5c9 (3be9a5c9bc793e11e2800a8c0c696e8b46742033) squid (dev) 2024-03-28T18:12:29.195 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:12:29 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69741]: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x135) [0x7faf86da64a1] 2024-03-28T18:12:29.195 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:12:29 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69741]: 2: /usr/lib64/ceph/libceph-common.so.2(+0x2d2667) [0x7faf86da6667] 2024-03-28T18:12:29.195 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:12:29 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69741]: 3: (Locker::set_xlocks_done(MutationImpl*, bool)+0x493) [0x55c652b89223] 2024-03-28T18:12:29.195 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:12:29 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69741]: 4: (Server::early_reply(boost::intrusive_ptr const&, CInode*, CDentry*)+0x3e6) [0x55c652a035d6] 2024-03-28T18:12:29.195 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:12:29 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69741]: 5: (Server::journal_and_reply(boost::intrusive_ptr const&, CInode*, CDentry*, LogEvent*, MDSLogContextBase*)+0xc1) [0x55c652a03da1] 2024-03-28T18:12:29.195 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:12:29 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69741]: 6: (Server::handle_client_mkdir(boost::intrusive_ptr const&)+0x5ba) [0x55c652a396aa] 2024-03-28T18:12:29.195 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:12:29 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69741]: 7: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0x733) [0x55c652a56233] 2024-03-28T18:12:29.195 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:12:29 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69741]: 8: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x55c652a58197] 2024-03-28T18:12:29.195 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:12:29 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69741]: 9: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x55c65295711b] 2024-03-28T18:12:29.195 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:12:29 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69741]: 10: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x55c6529576bc] 2024-03-28T18:12:29.195 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:12:29 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69741]: 11: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bf) [0x55c6529411ff] 2024-03-28T18:12:29.195 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:12:29 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69741]: 12: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x478) [0x7faf87038488] 2024-03-28T18:12:29.195 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:12:29 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69741]: 13: (DispatchQueue::entry()+0x51f) [0x7faf8703560f] 2024-03-28T18:12:29.196 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:12:29 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69741]: 14: (DispatchQueue::DispatchThread::entry()+0x11) [0x7faf87108e81] 2024-03-28T18:12:29.196 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:12:29 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69741]: 15: /lib64/libpthread.so.0(+0x81ca) [0x7faf85b231ca] 2024-03-28T18:12:29.196 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:12:29 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69741]: 16: clone() 2024-03-28T18:12:29.196 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:12:29 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69741]: 2024-03-28T18:12:29.196 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:12:29 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69741]: 0> 2024-03-28T18:12:28.984+0000 7faf7b400700 -1 *** Caught signal (Aborted) ** 2024-03-28T18:12:29.196 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:12:29 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69741]: in thread 7faf7b400700 thread_name:ms_dispatch 2024-03-28T18:12:29.196 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:12:29 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69741]: 2024-03-28T18:12:29.196 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:12:29 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69741]: ceph version 19.0.0-2145-g3be9a5c9 (3be9a5c9bc793e11e2800a8c0c696e8b46742033) squid (dev) 2024-03-28T18:12:29.196 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:12:29 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69741]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7faf85b2dd20] 2024-03-28T18:12:29.196 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:12:29 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69741]: 2: gsignal() 2024-03-28T18:12:29.196 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:12:29 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69741]: 3: abort() 2024-03-28T18:12:29.196 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:12:29 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69741]: 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x18f) [0x7faf86da64fb] 2024-03-28T18:12:29.196 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:12:29 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69741]: 5: /usr/lib64/ceph/libceph-common.so.2(+0x2d2667) [0x7faf86da6667] 2024-03-28T18:12:29.196 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:12:29 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69741]: 6: (Locker::set_xlocks_done(MutationImpl*, bool)+0x493) [0x55c652b89223] 2024-03-28T18:12:29.196 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:12:29 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69741]: 7: (Server::early_reply(boost::intrusive_ptr const&, CInode*, CDentry*)+0x3e6) [0x55c652a035d6] 2024-03-28T18:12:29.197 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:12:29 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69741]: 8: (Server::journal_and_reply(boost::intrusive_ptr const&, CInode*, CDentry*, LogEvent*, MDSLogContextBase*)+0xc1) [0x55c652a03da1] 2024-03-28T18:12:29.197 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:12:29 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69741]: 9: (Server::handle_client_mkdir(boost::intrusive_ptr const&)+0x5ba) [0x55c652a396aa] 2024-03-28T18:12:29.197 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:12:29 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69741]: 10: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0x733) [0x55c652a56233] 2024-03-28T18:12:29.197 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:12:29 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69741]: 11: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x55c652a58197] 2024-03-28T18:12:29.197 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:12:29 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69741]: 12: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x55c65295711b] 2024-03-28T18:12:29.197 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:12:29 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69741]: 13: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x55c6529576bc] 2024-03-28T18:12:29.197 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:12:29 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69741]: 14: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bf) [0x55c6529411ff] 2024-03-28T18:12:29.197 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:12:29 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69741]: 15: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x478) [0x7faf87038488] 2024-03-28T18:12:29.197 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:12:29 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69741]: 16: (DispatchQueue::entry()+0x51f) [0x7faf8703560f] 2024-03-28T18:12:29.197 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:12:29 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69741]: 17: (DispatchQueue::DispatchThread::entry()+0x11) [0x7faf87108e81] 2024-03-28T18:12:29.197 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:12:29 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69741]: 18: /lib64/libpthread.so.0(+0x81ca) [0x7faf85b231ca] 2024-03-28T18:12:29.197 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:12:29 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69741]: 19: clone() 2024-03-28T18:12:29.197 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:12:29 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69741]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-03-28T18:12:29.197 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:12:29 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69741]: 2024-03-28T18:12:29.197 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:12:29 smithi079 podman[69780]: 2024-03-28 18:12:29.112103161 +0000 UTC m=+0.037322220 container died 8c1d8d9a7546d8f8f2fd111916548d82a06ef058daf7d65dcc6a9ecd81578320 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, GIT_CLEAN=True, ceph=True, org.label-schema.build-date=20240319, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0) 2024-03-28T18:12:29.282 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:12:28 smithi022 ceph-mon[31985]: pgmap v158: 97 pgs: 97 active+clean; 117 MiB data, 785 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 10 op/s 2024-03-28T18:12:29.282 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:12:28 smithi022 ceph-mon[31985]: mds.? [v2:172.21.15.79:6836/3204076941,v1:172.21.15.79:6837/3204076941] up:rejoin 2024-03-28T18:12:29.282 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:12:28 smithi022 ceph-mon[31985]: fsmap cephfs:5/5 4 up:standby-replay 3 up:standby 1 up:rejoin 4 up:active 2024-03-28T18:12:29.282 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:12:28 smithi022 ceph-mon[31985]: daemon mds.h is now active in filesystem cephfs as rank 1 2024-03-28T18:12:30.175 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:12:29 smithi134 ceph-mon[35037]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-03-28T18:12:30.175 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:12:29 smithi134 ceph-mon[35037]: Cluster is now healthy 2024-03-28T18:12:30.175 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:12:29 smithi134 ceph-mon[35037]: mds.? [v2:172.21.15.79:6836/3204076941,v1:172.21.15.79:6837/3204076941] up:active 2024-03-28T18:12:30.175 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:12:29 smithi134 ceph-mon[35037]: fsmap cephfs:5 4 up:standby-replay 3 up:standby 5 up:active 2024-03-28T18:12:30.175 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:12:29 smithi134 ceph-mon[35037]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-28T18:12:30.183 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:12:29 smithi079 ceph-mon[35312]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-03-28T18:12:30.183 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:12:29 smithi079 ceph-mon[35312]: Cluster is now healthy 2024-03-28T18:12:30.183 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:12:29 smithi079 ceph-mon[35312]: mds.? [v2:172.21.15.79:6836/3204076941,v1:172.21.15.79:6837/3204076941] up:active 2024-03-28T18:12:30.183 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:12:29 smithi079 ceph-mon[35312]: fsmap cephfs:5 4 up:standby-replay 3 up:standby 5 up:active 2024-03-28T18:12:30.183 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:12:29 smithi079 ceph-mon[35312]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-28T18:12:30.183 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:12:29 smithi079 podman[69780]: 2024-03-28 18:12:29.789690789 +0000 UTC m=+0.714909843 container remove 8c1d8d9a7546d8f8f2fd111916548d82a06ef058daf7d65dcc6a9ecd81578320 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.build-date=20240319, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-28T18:12:30.183 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:12:29 smithi079 systemd[1]: ceph-fd220748-ed2b-11ee-95d0-87774f69a715@mds.h.service: Main process exited, code=exited, status=134/n/a 2024-03-28T18:12:30.282 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:12:29 smithi022 ceph-mon[31985]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-03-28T18:12:30.282 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:12:29 smithi022 ceph-mon[31985]: Cluster is now healthy 2024-03-28T18:12:30.282 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:12:29 smithi022 ceph-mon[31985]: mds.? [v2:172.21.15.79:6836/3204076941,v1:172.21.15.79:6837/3204076941] up:active 2024-03-28T18:12:30.282 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:12:29 smithi022 ceph-mon[31985]: fsmap cephfs:5 4 up:standby-replay 3 up:standby 5 up:active 2024-03-28T18:12:30.282 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:12:29 smithi022 ceph-mon[31985]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-28T18:12:30.683 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:12:30 smithi079 systemd[1]: ceph-fd220748-ed2b-11ee-95d0-87774f69a715@mds.h.service: Failed with result 'exit-code'. 2024-03-28T18:12:30.683 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:12:30 smithi079 systemd[1]: ceph-fd220748-ed2b-11ee-95d0-87774f69a715@mds.h.service: Consumed 1.692s CPU time. 2024-03-28T18:12:31.183 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:12:30 smithi079 ceph-mon[35312]: pgmap v159: 97 pgs: 97 active+clean; 117 MiB data, 785 MiB used, 1.0 TiB / 1.0 TiB avail; 106 KiB/s rd, 4.8 KiB/s wr, 26 op/s 2024-03-28T18:12:31.282 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:12:30 smithi022 ceph-mon[31985]: pgmap v159: 97 pgs: 97 active+clean; 117 MiB data, 785 MiB used, 1.0 TiB / 1.0 TiB avail; 106 KiB/s rd, 4.8 KiB/s wr, 26 op/s 2024-03-28T18:12:31.424 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:12:30 smithi134 ceph-mon[35037]: pgmap v159: 97 pgs: 97 active+clean; 117 MiB data, 785 MiB used, 1.0 TiB / 1.0 TiB avail; 106 KiB/s rd, 4.8 KiB/s wr, 26 op/s 2024-03-28T18:12:33.282 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:12:32 smithi022 ceph-mon[31985]: pgmap v160: 97 pgs: 97 active+clean; 117 MiB data, 785 MiB used, 1.0 TiB / 1.0 TiB avail; 106 KiB/s rd, 4.8 KiB/s wr, 26 op/s 2024-03-28T18:12:33.424 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:12:32 smithi134 ceph-mon[35037]: pgmap v160: 97 pgs: 97 active+clean; 117 MiB data, 785 MiB used, 1.0 TiB / 1.0 TiB avail; 106 KiB/s rd, 4.8 KiB/s wr, 26 op/s 2024-03-28T18:12:33.432 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:12:32 smithi079 ceph-mon[35312]: pgmap v160: 97 pgs: 97 active+clean; 117 MiB data, 785 MiB used, 1.0 TiB / 1.0 TiB avail; 106 KiB/s rd, 4.8 KiB/s wr, 26 op/s 2024-03-28T18:12:35.282 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:12:34 smithi022 ceph-mon[31985]: pgmap v161: 97 pgs: 97 active+clean; 117 MiB data, 785 MiB used, 1.0 TiB / 1.0 TiB avail; 106 KiB/s rd, 4.8 KiB/s wr, 26 op/s 2024-03-28T18:12:35.424 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:12:34 smithi134 ceph-mon[35037]: pgmap v161: 97 pgs: 97 active+clean; 117 MiB data, 785 MiB used, 1.0 TiB / 1.0 TiB avail; 106 KiB/s rd, 4.8 KiB/s wr, 26 op/s 2024-03-28T18:12:35.432 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:12:34 smithi079 ceph-mon[35312]: pgmap v161: 97 pgs: 97 active+clean; 117 MiB data, 785 MiB used, 1.0 TiB / 1.0 TiB avail; 106 KiB/s rd, 4.8 KiB/s wr, 26 op/s 2024-03-28T18:12:36.282 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:12:35 smithi022 ceph-mon[31985]: pgmap v162: 97 pgs: 97 active+clean; 117 MiB data, 785 MiB used, 1.0 TiB / 1.0 TiB avail; 100 KiB/s rd, 4.4 KiB/s wr, 24 op/s 2024-03-28T18:12:36.424 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:12:35 smithi134 ceph-mon[35037]: pgmap v162: 97 pgs: 97 active+clean; 117 MiB data, 785 MiB used, 1.0 TiB / 1.0 TiB avail; 100 KiB/s rd, 4.4 KiB/s wr, 24 op/s 2024-03-28T18:12:36.432 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:12:35 smithi079 ceph-mon[35312]: pgmap v162: 97 pgs: 97 active+clean; 117 MiB data, 785 MiB used, 1.0 TiB / 1.0 TiB avail; 100 KiB/s rd, 4.4 KiB/s wr, 24 op/s 2024-03-28T18:12:38.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:12:38 smithi022 ceph-mon[31985]: pgmap v163: 97 pgs: 97 active+clean; 117 MiB data, 785 MiB used, 1.0 TiB / 1.0 TiB avail; 91 KiB/s rd, 4.0 KiB/s wr, 21 op/s 2024-03-28T18:12:38.674 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:12:38 smithi134 ceph-mon[35037]: pgmap v163: 97 pgs: 97 active+clean; 117 MiB data, 785 MiB used, 1.0 TiB / 1.0 TiB avail; 91 KiB/s rd, 4.0 KiB/s wr, 21 op/s 2024-03-28T18:12:38.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:12:38 smithi079 ceph-mon[35312]: pgmap v163: 97 pgs: 97 active+clean; 117 MiB data, 785 MiB used, 1.0 TiB / 1.0 TiB avail; 91 KiB/s rd, 4.0 KiB/s wr, 21 op/s 2024-03-28T18:12:40.455 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:12:40 smithi079 systemd[1]: ceph-fd220748-ed2b-11ee-95d0-87774f69a715@mds.h.service: Scheduled restart job, restart counter is at 5. 2024-03-28T18:12:40.455 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:12:40 smithi079 systemd[1]: Stopped Ceph mds.h for fd220748-ed2b-11ee-95d0-87774f69a715. 2024-03-28T18:12:40.455 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:12:40 smithi079 systemd[1]: ceph-fd220748-ed2b-11ee-95d0-87774f69a715@mds.h.service: Consumed 1.692s CPU time. 2024-03-28T18:12:40.455 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:12:40 smithi079 ceph-mon[35312]: pgmap v164: 97 pgs: 97 active+clean; 117 MiB data, 785 MiB used, 1.0 TiB / 1.0 TiB avail; 93 KiB/s rd, 4.0 KiB/s wr, 26 op/s 2024-03-28T18:12:40.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:12:40 smithi022 ceph-mon[31985]: pgmap v164: 97 pgs: 97 active+clean; 117 MiB data, 785 MiB used, 1.0 TiB / 1.0 TiB avail; 93 KiB/s rd, 4.0 KiB/s wr, 26 op/s 2024-03-28T18:12:40.674 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:12:40 smithi134 ceph-mon[35037]: pgmap v164: 97 pgs: 97 active+clean; 117 MiB data, 785 MiB used, 1.0 TiB / 1.0 TiB avail; 93 KiB/s rd, 4.0 KiB/s wr, 26 op/s 2024-03-28T18:12:40.856 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:12:40 smithi079 systemd[1]: Starting Ceph mds.h for fd220748-ed2b-11ee-95d0-87774f69a715... 2024-03-28T18:12:41.175 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:12:40 smithi079 podman[69863]: 2024-03-28 18:12:40.762213286 +0000 UTC m=+0.045747018 image pull c664afe56bb01db4f9afa9532c20598e201e664f2e9ba2be50db0d52674a26d2 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc 2024-03-28T18:12:41.175 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:12:40 smithi079 podman[69863]: 2024-03-28 18:12:40.889820739 +0000 UTC m=+0.173354490 container create becb7c966c54908099c6e9524f3fb51c4b51d6d91f38d13c856e803b956507c8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, ceph=True, GIT_BRANCH=HEAD, org.label-schema.build-date=20240319, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.license=GPLv2) 2024-03-28T18:12:41.433 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:12:41 smithi079 podman[69863]: 2024-03-28 18:12:41.176028627 +0000 UTC m=+0.459562348 container init becb7c966c54908099c6e9524f3fb51c4b51d6d91f38d13c856e803b956507c8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240319, io.buildah.version=1.33.5, ceph=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux ) 2024-03-28T18:12:41.433 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:12:41 smithi079 podman[69863]: 2024-03-28 18:12:41.183160526 +0000 UTC m=+0.466694257 container start becb7c966c54908099c6e9524f3fb51c4b51d6d91f38d13c856e803b956507c8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h, GIT_CLEAN=True, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.build-date=20240319, CEPH_POINT_RELEASE=, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, ceph=True) 2024-03-28T18:12:41.433 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:12:41 smithi079 ceph-mds[69882]: -- 172.21.15.79:0/1832497843 <== mon.2 v2:172.21.15.79:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x559c18949a40 con 0x559c1891c800 2024-03-28T18:12:41.434 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:12:41 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69878]: starting mds.h at 2024-03-28T18:12:41.434 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:12:41 smithi079 bash[69863]: becb7c966c54908099c6e9524f3fb51c4b51d6d91f38d13c856e803b956507c8 2024-03-28T18:12:41.434 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:12:41 smithi079 systemd[1]: Started Ceph mds.h for fd220748-ed2b-11ee-95d0-87774f69a715. 2024-03-28T18:12:42.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:12:42 smithi022 ceph-mon[31985]: pgmap v165: 97 pgs: 97 active+clean; 117 MiB data, 785 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 8 op/s 2024-03-28T18:12:42.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:12:42 smithi022 ceph-mon[31985]: daemon mds.h restarted 2024-03-28T18:12:42.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:12:42 smithi022 ceph-mon[31985]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-03-28T18:12:42.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:12:42 smithi022 ceph-mon[31985]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-03-28T18:12:42.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:12:42 smithi022 ceph-mon[31985]: osdmap e102: 12 total, 12 up, 12 in 2024-03-28T18:12:42.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:12:42 smithi022 ceph-mon[31985]: mds.? [v2:172.21.15.79:6836/1653690981,v1:172.21.15.79:6837/1653690981] up:boot 2024-03-28T18:12:42.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:12:42 smithi022 ceph-mon[31985]: Standby daemon mds.f assigned to filesystem cephfs as rank 1 2024-03-28T18:12:42.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:12:42 smithi022 ceph-mon[31985]: fsmap cephfs:4/5 {0=b=up:active,2=e=up:active,3=k=up:active,4=i=up:active} 5 up:standby-replay 3 up:standby, 1 failed 2024-03-28T18:12:42.533 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:12:42 smithi022 ceph-mon[31985]: fsmap cephfs:5/5 4 up:standby-replay 3 up:standby 1 up:replay 4 up:active 2024-03-28T18:12:42.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:12:42 smithi134 ceph-mon[35037]: pgmap v165: 97 pgs: 97 active+clean; 117 MiB data, 785 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 8 op/s 2024-03-28T18:12:42.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:12:42 smithi134 ceph-mon[35037]: daemon mds.h restarted 2024-03-28T18:12:42.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:12:42 smithi134 ceph-mon[35037]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-03-28T18:12:42.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:12:42 smithi134 ceph-mon[35037]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-03-28T18:12:42.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:12:42 smithi134 ceph-mon[35037]: osdmap e102: 12 total, 12 up, 12 in 2024-03-28T18:12:42.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:12:42 smithi134 ceph-mon[35037]: mds.? [v2:172.21.15.79:6836/1653690981,v1:172.21.15.79:6837/1653690981] up:boot 2024-03-28T18:12:42.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:12:42 smithi134 ceph-mon[35037]: Standby daemon mds.f assigned to filesystem cephfs as rank 1 2024-03-28T18:12:42.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:12:42 smithi134 ceph-mon[35037]: fsmap cephfs:4/5 {0=b=up:active,2=e=up:active,3=k=up:active,4=i=up:active} 5 up:standby-replay 3 up:standby, 1 failed 2024-03-28T18:12:42.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:12:42 smithi134 ceph-mon[35037]: fsmap cephfs:5/5 4 up:standby-replay 3 up:standby 1 up:replay 4 up:active 2024-03-28T18:12:42.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:12:42 smithi079 ceph-mon[35312]: pgmap v165: 97 pgs: 97 active+clean; 117 MiB data, 785 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 8 op/s 2024-03-28T18:12:42.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:12:42 smithi079 ceph-mon[35312]: daemon mds.h restarted 2024-03-28T18:12:42.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:12:42 smithi079 ceph-mon[35312]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-03-28T18:12:42.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:12:42 smithi079 ceph-mon[35312]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-03-28T18:12:42.683 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:12:42 smithi079 ceph-mon[35312]: osdmap e102: 12 total, 12 up, 12 in 2024-03-28T18:12:42.683 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:12:42 smithi079 ceph-mon[35312]: mds.? [v2:172.21.15.79:6836/1653690981,v1:172.21.15.79:6837/1653690981] up:boot 2024-03-28T18:12:42.683 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:12:42 smithi079 ceph-mon[35312]: Standby daemon mds.f assigned to filesystem cephfs as rank 1 2024-03-28T18:12:42.683 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:12:42 smithi079 ceph-mon[35312]: fsmap cephfs:4/5 {0=b=up:active,2=e=up:active,3=k=up:active,4=i=up:active} 5 up:standby-replay 3 up:standby, 1 failed 2024-03-28T18:12:42.683 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:12:42 smithi079 ceph-mon[35312]: fsmap cephfs:5/5 4 up:standby-replay 3 up:standby 1 up:replay 4 up:active 2024-03-28T18:12:43.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:12:43 smithi022 ceph-mon[31985]: mds.? [v2:172.21.15.134:6834/726536645,v1:172.21.15.134:6835/726536645] up:resolve 2024-03-28T18:12:43.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:12:43 smithi022 ceph-mon[31985]: fsmap cephfs:5/5 4 up:standby-replay 3 up:standby 1 up:resolve 4 up:active 2024-03-28T18:12:43.674 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:12:43 smithi134 ceph-mon[35037]: mds.? [v2:172.21.15.134:6834/726536645,v1:172.21.15.134:6835/726536645] up:resolve 2024-03-28T18:12:43.674 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:12:43 smithi134 ceph-mon[35037]: fsmap cephfs:5/5 4 up:standby-replay 3 up:standby 1 up:resolve 4 up:active 2024-03-28T18:12:43.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:12:43 smithi079 ceph-mon[35312]: mds.? [v2:172.21.15.134:6834/726536645,v1:172.21.15.134:6835/726536645] up:resolve 2024-03-28T18:12:43.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:12:43 smithi079 ceph-mon[35312]: fsmap cephfs:5/5 4 up:standby-replay 3 up:standby 1 up:resolve 4 up:active 2024-03-28T18:12:44.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:12:44 smithi022 ceph-mon[31985]: pgmap v167: 97 pgs: 97 active+clean; 117 MiB data, 785 MiB used, 1.0 TiB / 1.0 TiB avail; 7.8 KiB/s rd, 10 op/s 2024-03-28T18:12:44.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:12:44 smithi022 ceph-mon[31985]: reconnect by client.35222 172.21.15.22:0/3729116781 after 0.00299993 2024-03-28T18:12:44.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:12:44 smithi022 ceph-mon[31985]: mds.? [v2:172.21.15.134:6834/726536645,v1:172.21.15.134:6835/726536645] up:reconnect 2024-03-28T18:12:44.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:12:44 smithi022 ceph-mon[31985]: fsmap cephfs:5/5 4 up:standby-replay 3 up:standby 1 up:reconnect 4 up:active 2024-03-28T18:12:44.674 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:12:44 smithi134 ceph-mon[35037]: pgmap v167: 97 pgs: 97 active+clean; 117 MiB data, 785 MiB used, 1.0 TiB / 1.0 TiB avail; 7.8 KiB/s rd, 10 op/s 2024-03-28T18:12:44.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:12:44 smithi134 ceph-mon[35037]: reconnect by client.35222 172.21.15.22:0/3729116781 after 0.00299993 2024-03-28T18:12:44.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:12:44 smithi134 ceph-mon[35037]: mds.? [v2:172.21.15.134:6834/726536645,v1:172.21.15.134:6835/726536645] up:reconnect 2024-03-28T18:12:44.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:12:44 smithi134 ceph-mon[35037]: fsmap cephfs:5/5 4 up:standby-replay 3 up:standby 1 up:reconnect 4 up:active 2024-03-28T18:12:44.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:12:44 smithi079 ceph-mon[35312]: pgmap v167: 97 pgs: 97 active+clean; 117 MiB data, 785 MiB used, 1.0 TiB / 1.0 TiB avail; 7.8 KiB/s rd, 10 op/s 2024-03-28T18:12:44.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:12:44 smithi079 ceph-mon[35312]: reconnect by client.35222 172.21.15.22:0/3729116781 after 0.00299993 2024-03-28T18:12:44.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:12:44 smithi079 ceph-mon[35312]: mds.? [v2:172.21.15.134:6834/726536645,v1:172.21.15.134:6835/726536645] up:reconnect 2024-03-28T18:12:44.683 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:12:44 smithi079 ceph-mon[35312]: fsmap cephfs:5/5 4 up:standby-replay 3 up:standby 1 up:reconnect 4 up:active 2024-03-28T18:12:45.509 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:12:45 smithi134 ceph-mon[35037]: mds.? [v2:172.21.15.134:6834/726536645,v1:172.21.15.134:6835/726536645] up:rejoin 2024-03-28T18:12:45.510 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:12:45 smithi134 ceph-mon[35037]: fsmap cephfs:5/5 4 up:standby-replay 3 up:standby 1 up:rejoin 4 up:active 2024-03-28T18:12:45.510 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:12:45 smithi134 ceph-mon[35037]: daemon mds.f is now active in filesystem cephfs as rank 1 2024-03-28T18:12:45.510 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:12:45 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[70501]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-2145-g3be9a5c9/rpm/el8/BUILD/ceph-19.0.0-2145-g3be9a5c9/src/mds/Locker.cc: In function 'void Locker::set_xlocks_done(MutationImpl*, bool)' thread 7f49f7800700 time 2024-03-28T18:12:45.411239+0000 2024-03-28T18:12:45.510 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:12:45 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[70501]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-2145-g3be9a5c9/rpm/el8/BUILD/ceph-19.0.0-2145-g3be9a5c9/src/mds/Locker.cc: 626: FAILED ceph_assert(obj->is_auth()) 2024-03-28T18:12:45.511 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:12:45 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[70501]: ceph version 19.0.0-2145-g3be9a5c9 (3be9a5c9bc793e11e2800a8c0c696e8b46742033) squid (dev) 2024-03-28T18:12:45.511 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:12:45 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[70501]: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x135) [0x7f4a031184a1] 2024-03-28T18:12:45.511 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:12:45 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[70501]: 2: /usr/lib64/ceph/libceph-common.so.2(+0x2d2667) [0x7f4a03118667] 2024-03-28T18:12:45.511 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:12:45 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[70501]: 3: (Locker::set_xlocks_done(MutationImpl*, bool)+0x493) [0x55bce68c0223] 2024-03-28T18:12:45.511 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:12:45 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[70501]: 4: (Server::early_reply(boost::intrusive_ptr const&, CInode*, CDentry*)+0x3e6) [0x55bce673a5d6] 2024-03-28T18:12:45.511 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:12:45 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[70501]: 5: (Server::journal_and_reply(boost::intrusive_ptr const&, CInode*, CDentry*, LogEvent*, MDSLogContextBase*)+0xc1) [0x55bce673ada1] 2024-03-28T18:12:45.511 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:12:45 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[70501]: 6: (Server::handle_client_mkdir(boost::intrusive_ptr const&)+0x5ba) [0x55bce67706aa] 2024-03-28T18:12:45.511 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:12:45 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[70501]: 7: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0x733) [0x55bce678d233] 2024-03-28T18:12:45.511 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:12:45 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[70501]: 8: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x55bce678f197] 2024-03-28T18:12:45.511 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:12:45 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[70501]: 9: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x55bce668e11b] 2024-03-28T18:12:45.511 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:12:45 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[70501]: 10: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x55bce668e6bc] 2024-03-28T18:12:45.511 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:12:45 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[70501]: 11: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bf) [0x55bce66781ff] 2024-03-28T18:12:45.511 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:12:45 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[70501]: 12: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x478) [0x7f4a033aa488] 2024-03-28T18:12:45.512 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:12:45 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[70501]: 13: (DispatchQueue::entry()+0x51f) [0x7f4a033a760f] 2024-03-28T18:12:45.512 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:12:45 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[70501]: 14: (DispatchQueue::DispatchThread::entry()+0x11) [0x7f4a0347ae81] 2024-03-28T18:12:45.512 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:12:45 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[70501]: 15: /lib64/libpthread.so.0(+0x81ca) [0x7f4a01e951ca] 2024-03-28T18:12:45.512 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:12:45 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[70501]: 16: clone() 2024-03-28T18:12:45.512 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:12:45 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[70501]: 2024-03-28T18:12:45.414+0000 7f49f7800700 -1 /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-2145-g3be9a5c9/rpm/el8/BUILD/ceph-19.0.0-2145-g3be9a5c9/src/mds/Locker.cc: In function 'void Locker::set_xlocks_done(MutationImpl*, bool)' thread 7f49f7800700 time 2024-03-28T18:12:45.411239+0000 2024-03-28T18:12:45.512 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:12:45 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[70501]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-2145-g3be9a5c9/rpm/el8/BUILD/ceph-19.0.0-2145-g3be9a5c9/src/mds/Locker.cc: 626: FAILED ceph_assert(obj->is_auth()) 2024-03-28T18:12:45.512 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:12:45 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[70501]: 2024-03-28T18:12:45.512 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:12:45 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[70501]: ceph version 19.0.0-2145-g3be9a5c9 (3be9a5c9bc793e11e2800a8c0c696e8b46742033) squid (dev) 2024-03-28T18:12:45.512 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:12:45 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[70501]: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x135) [0x7f4a031184a1] 2024-03-28T18:12:45.512 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:12:45 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[70501]: 2: /usr/lib64/ceph/libceph-common.so.2(+0x2d2667) [0x7f4a03118667] 2024-03-28T18:12:45.512 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:12:45 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[70501]: 3: (Locker::set_xlocks_done(MutationImpl*, bool)+0x493) [0x55bce68c0223] 2024-03-28T18:12:45.512 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:12:45 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[70501]: 4: (Server::early_reply(boost::intrusive_ptr const&, CInode*, CDentry*)+0x3e6) [0x55bce673a5d6] 2024-03-28T18:12:45.512 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:12:45 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[70501]: 5: (Server::journal_and_reply(boost::intrusive_ptr const&, CInode*, CDentry*, LogEvent*, MDSLogContextBase*)+0xc1) [0x55bce673ada1] 2024-03-28T18:12:45.513 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:12:45 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[70501]: 6: (Server::handle_client_mkdir(boost::intrusive_ptr const&)+0x5ba) [0x55bce67706aa] 2024-03-28T18:12:45.513 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:12:45 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[70501]: 7: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0x733) [0x55bce678d233] 2024-03-28T18:12:45.513 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:12:45 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[70501]: 8: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x55bce678f197] 2024-03-28T18:12:45.513 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:12:45 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[70501]: 9: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x55bce668e11b] 2024-03-28T18:12:45.513 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:12:45 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[70501]: 10: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x55bce668e6bc] 2024-03-28T18:12:45.513 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:12:45 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[70501]: 11: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bf) [0x55bce66781ff] 2024-03-28T18:12:45.513 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:12:45 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[70501]: 12: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x478) [0x7f4a033aa488] 2024-03-28T18:12:45.513 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:12:45 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[70501]: 13: (DispatchQueue::entry()+0x51f) [0x7f4a033a760f] 2024-03-28T18:12:45.513 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:12:45 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[70501]: 14: (DispatchQueue::DispatchThread::entry()+0x11) [0x7f4a0347ae81] 2024-03-28T18:12:45.513 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:12:45 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[70501]: 15: /lib64/libpthread.so.0(+0x81ca) [0x7f4a01e951ca] 2024-03-28T18:12:45.513 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:12:45 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[70501]: 16: clone() 2024-03-28T18:12:45.513 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:12:45 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[70501]: 2024-03-28T18:12:45.513 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:12:45 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[70501]: *** Caught signal (Aborted) ** 2024-03-28T18:12:45.513 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:12:45 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[70501]: in thread 7f49f7800700 thread_name:ms_dispatch 2024-03-28T18:12:45.513 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:12:45 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[70501]: ceph version 19.0.0-2145-g3be9a5c9 (3be9a5c9bc793e11e2800a8c0c696e8b46742033) squid (dev) 2024-03-28T18:12:45.514 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:12:45 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[70501]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7f4a01e9fd20] 2024-03-28T18:12:45.515 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:12:45 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[70501]: 2: gsignal() 2024-03-28T18:12:45.515 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:12:45 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[70501]: 3: abort() 2024-03-28T18:12:45.515 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:12:45 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[70501]: 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x18f) [0x7f4a031184fb] 2024-03-28T18:12:45.515 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:12:45 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[70501]: 5: /usr/lib64/ceph/libceph-common.so.2(+0x2d2667) [0x7f4a03118667] 2024-03-28T18:12:45.515 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:12:45 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[70501]: 6: (Locker::set_xlocks_done(MutationImpl*, bool)+0x493) [0x55bce68c0223] 2024-03-28T18:12:45.515 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:12:45 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[70501]: 7: (Server::early_reply(boost::intrusive_ptr const&, CInode*, CDentry*)+0x3e6) [0x55bce673a5d6] 2024-03-28T18:12:45.515 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:12:45 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[70501]: 8: (Server::journal_and_reply(boost::intrusive_ptr const&, CInode*, CDentry*, LogEvent*, MDSLogContextBase*)+0xc1) [0x55bce673ada1] 2024-03-28T18:12:45.515 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:12:45 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[70501]: 9: (Server::handle_client_mkdir(boost::intrusive_ptr const&)+0x5ba) [0x55bce67706aa] 2024-03-28T18:12:45.515 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:12:45 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[70501]: 10: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0x733) [0x55bce678d233] 2024-03-28T18:12:45.515 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:12:45 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[70501]: 11: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x55bce678f197] 2024-03-28T18:12:45.515 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:12:45 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[70501]: 12: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x55bce668e11b] 2024-03-28T18:12:45.515 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:12:45 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[70501]: 13: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x55bce668e6bc] 2024-03-28T18:12:45.516 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:12:45 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[70501]: 14: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bf) [0x55bce66781ff] 2024-03-28T18:12:45.516 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:12:45 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[70501]: 15: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x478) [0x7f4a033aa488] 2024-03-28T18:12:45.516 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:12:45 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[70501]: 16: (DispatchQueue::entry()+0x51f) [0x7f4a033a760f] 2024-03-28T18:12:45.516 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:12:45 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[70501]: 17: (DispatchQueue::DispatchThread::entry()+0x11) [0x7f4a0347ae81] 2024-03-28T18:12:45.516 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:12:45 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[70501]: 18: /lib64/libpthread.so.0(+0x81ca) [0x7f4a01e951ca] 2024-03-28T18:12:45.516 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:12:45 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[70501]: 19: clone() 2024-03-28T18:12:45.516 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:12:45 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[70501]: 2024-03-28T18:12:45.420+0000 7f49f7800700 -1 *** Caught signal (Aborted) ** 2024-03-28T18:12:45.516 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:12:45 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[70501]: in thread 7f49f7800700 thread_name:ms_dispatch 2024-03-28T18:12:45.516 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:12:45 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[70501]: 2024-03-28T18:12:45.516 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:12:45 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[70501]: ceph version 19.0.0-2145-g3be9a5c9 (3be9a5c9bc793e11e2800a8c0c696e8b46742033) squid (dev) 2024-03-28T18:12:45.516 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:12:45 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[70501]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7f4a01e9fd20] 2024-03-28T18:12:45.516 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:12:45 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[70501]: 2: gsignal() 2024-03-28T18:12:45.516 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:12:45 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[70501]: 3: abort() 2024-03-28T18:12:45.516 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:12:45 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[70501]: 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x18f) [0x7f4a031184fb] 2024-03-28T18:12:45.516 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:12:45 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[70501]: 5: /usr/lib64/ceph/libceph-common.so.2(+0x2d2667) [0x7f4a03118667] 2024-03-28T18:12:45.517 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:12:45 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[70501]: 6: (Locker::set_xlocks_done(MutationImpl*, bool)+0x493) [0x55bce68c0223] 2024-03-28T18:12:45.517 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:12:45 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[70501]: 7: (Server::early_reply(boost::intrusive_ptr const&, CInode*, CDentry*)+0x3e6) [0x55bce673a5d6] 2024-03-28T18:12:45.517 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:12:45 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[70501]: 8: (Server::journal_and_reply(boost::intrusive_ptr const&, CInode*, CDentry*, LogEvent*, MDSLogContextBase*)+0xc1) [0x55bce673ada1] 2024-03-28T18:12:45.517 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:12:45 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[70501]: 9: (Server::handle_client_mkdir(boost::intrusive_ptr const&)+0x5ba) [0x55bce67706aa] 2024-03-28T18:12:45.517 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:12:45 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[70501]: 10: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0x733) [0x55bce678d233] 2024-03-28T18:12:45.517 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:12:45 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[70501]: 11: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x55bce678f197] 2024-03-28T18:12:45.517 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:12:45 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[70501]: 12: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x55bce668e11b] 2024-03-28T18:12:45.517 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:12:45 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[70501]: 13: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x55bce668e6bc] 2024-03-28T18:12:45.517 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:12:45 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[70501]: 14: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bf) [0x55bce66781ff] 2024-03-28T18:12:45.517 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:12:45 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[70501]: 15: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x478) [0x7f4a033aa488] 2024-03-28T18:12:45.517 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:12:45 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[70501]: 16: (DispatchQueue::entry()+0x51f) [0x7f4a033a760f] 2024-03-28T18:12:45.517 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:12:45 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[70501]: 17: (DispatchQueue::DispatchThread::entry()+0x11) [0x7f4a0347ae81] 2024-03-28T18:12:45.517 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:12:45 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[70501]: 18: /lib64/libpthread.so.0(+0x81ca) [0x7f4a01e951ca] 2024-03-28T18:12:45.517 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:12:45 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[70501]: 19: clone() 2024-03-28T18:12:45.517 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:12:45 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[70501]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-03-28T18:12:45.518 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:12:45 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[70501]: 2024-03-28T18:12:45.518 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:12:45 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[70501]: -1> 2024-03-28T18:12:45.414+0000 7f49f7800700 -1 /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-2145-g3be9a5c9/rpm/el8/BUILD/ceph-19.0.0-2145-g3be9a5c9/src/mds/Locker.cc: In function 'void Locker::set_xlocks_done(MutationImpl*, bool)' thread 7f49f7800700 time 2024-03-28T18:12:45.411239+0000 2024-03-28T18:12:45.518 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:12:45 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[70501]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-2145-g3be9a5c9/rpm/el8/BUILD/ceph-19.0.0-2145-g3be9a5c9/src/mds/Locker.cc: 626: FAILED ceph_assert(obj->is_auth()) 2024-03-28T18:12:45.518 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:12:45 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[70501]: 2024-03-28T18:12:45.518 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:12:45 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[70501]: ceph version 19.0.0-2145-g3be9a5c9 (3be9a5c9bc793e11e2800a8c0c696e8b46742033) squid (dev) 2024-03-28T18:12:45.518 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:12:45 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[70501]: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x135) [0x7f4a031184a1] 2024-03-28T18:12:45.518 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:12:45 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[70501]: 2: /usr/lib64/ceph/libceph-common.so.2(+0x2d2667) [0x7f4a03118667] 2024-03-28T18:12:45.518 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:12:45 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[70501]: 3: (Locker::set_xlocks_done(MutationImpl*, bool)+0x493) [0x55bce68c0223] 2024-03-28T18:12:45.518 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:12:45 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[70501]: 4: (Server::early_reply(boost::intrusive_ptr const&, CInode*, CDentry*)+0x3e6) [0x55bce673a5d6] 2024-03-28T18:12:45.518 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:12:45 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[70501]: 5: (Server::journal_and_reply(boost::intrusive_ptr const&, CInode*, CDentry*, LogEvent*, MDSLogContextBase*)+0xc1) [0x55bce673ada1] 2024-03-28T18:12:45.518 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:12:45 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[70501]: 6: (Server::handle_client_mkdir(boost::intrusive_ptr const&)+0x5ba) [0x55bce67706aa] 2024-03-28T18:12:45.518 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:12:45 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[70501]: 7: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0x733) [0x55bce678d233] 2024-03-28T18:12:45.518 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:12:45 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[70501]: 8: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x55bce678f197] 2024-03-28T18:12:45.518 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:12:45 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[70501]: 9: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x55bce668e11b] 2024-03-28T18:12:45.519 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:12:45 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[70501]: 10: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x55bce668e6bc] 2024-03-28T18:12:45.519 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:12:45 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[70501]: 11: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bf) [0x55bce66781ff] 2024-03-28T18:12:45.519 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:12:45 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[70501]: 12: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x478) [0x7f4a033aa488] 2024-03-28T18:12:45.519 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:12:45 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[70501]: 13: (DispatchQueue::entry()+0x51f) [0x7f4a033a760f] 2024-03-28T18:12:45.519 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:12:45 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[70501]: 14: (DispatchQueue::DispatchThread::entry()+0x11) [0x7f4a0347ae81] 2024-03-28T18:12:45.519 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:12:45 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[70501]: 15: /lib64/libpthread.so.0(+0x81ca) [0x7f4a01e951ca] 2024-03-28T18:12:45.519 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:12:45 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[70501]: 16: clone() 2024-03-28T18:12:45.519 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:12:45 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[70501]: 2024-03-28T18:12:45.519 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:12:45 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[70501]: 0> 2024-03-28T18:12:45.420+0000 7f49f7800700 -1 *** Caught signal (Aborted) ** 2024-03-28T18:12:45.519 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:12:45 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[70501]: in thread 7f49f7800700 thread_name:ms_dispatch 2024-03-28T18:12:45.519 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:12:45 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[70501]: 2024-03-28T18:12:45.519 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:12:45 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[70501]: ceph version 19.0.0-2145-g3be9a5c9 (3be9a5c9bc793e11e2800a8c0c696e8b46742033) squid (dev) 2024-03-28T18:12:45.519 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:12:45 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[70501]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7f4a01e9fd20] 2024-03-28T18:12:45.519 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:12:45 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[70501]: 2: gsignal() 2024-03-28T18:12:45.519 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:12:45 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[70501]: 3: abort() 2024-03-28T18:12:45.520 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:12:45 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[70501]: 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x18f) [0x7f4a031184fb] 2024-03-28T18:12:45.520 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:12:45 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[70501]: 5: /usr/lib64/ceph/libceph-common.so.2(+0x2d2667) [0x7f4a03118667] 2024-03-28T18:12:45.520 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:12:45 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[70501]: 6: (Locker::set_xlocks_done(MutationImpl*, bool)+0x493) [0x55bce68c0223] 2024-03-28T18:12:45.520 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:12:45 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[70501]: 7: (Server::early_reply(boost::intrusive_ptr const&, CInode*, CDentry*)+0x3e6) [0x55bce673a5d6] 2024-03-28T18:12:45.520 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:12:45 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[70501]: 8: (Server::journal_and_reply(boost::intrusive_ptr const&, CInode*, CDentry*, LogEvent*, MDSLogContextBase*)+0xc1) [0x55bce673ada1] 2024-03-28T18:12:45.520 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:12:45 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[70501]: 9: (Server::handle_client_mkdir(boost::intrusive_ptr const&)+0x5ba) [0x55bce67706aa] 2024-03-28T18:12:45.520 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:12:45 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[70501]: 10: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0x733) [0x55bce678d233] 2024-03-28T18:12:45.520 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:12:45 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[70501]: 11: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x55bce678f197] 2024-03-28T18:12:45.520 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:12:45 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[70501]: 12: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x55bce668e11b] 2024-03-28T18:12:45.520 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:12:45 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[70501]: 13: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x55bce668e6bc] 2024-03-28T18:12:45.520 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:12:45 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[70501]: 14: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bf) [0x55bce66781ff] 2024-03-28T18:12:45.520 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:12:45 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[70501]: 15: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x478) [0x7f4a033aa488] 2024-03-28T18:12:45.520 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:12:45 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[70501]: 16: (DispatchQueue::entry()+0x51f) [0x7f4a033a760f] 2024-03-28T18:12:45.520 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:12:45 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[70501]: 17: (DispatchQueue::DispatchThread::entry()+0x11) [0x7f4a0347ae81] 2024-03-28T18:12:45.520 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:12:45 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[70501]: 18: /lib64/libpthread.so.0(+0x81ca) [0x7f4a01e951ca] 2024-03-28T18:12:45.521 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:12:45 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[70501]: 19: clone() 2024-03-28T18:12:45.521 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:12:45 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[70501]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-03-28T18:12:45.521 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:12:45 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[70501]: 2024-03-28T18:12:45.521 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:12:45 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[70501]: -1> 2024-03-28T18:12:45.414+0000 7f49f7800700 -1 /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-2145-g3be9a5c9/rpm/el8/BUILD/ceph-19.0.0-2145-g3be9a5c9/src/mds/Locker.cc: In function 'void Locker::set_xlocks_done(MutationImpl*, bool)' thread 7f49f7800700 time 2024-03-28T18:12:45.411239+0000 2024-03-28T18:12:45.521 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:12:45 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[70501]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-2145-g3be9a5c9/rpm/el8/BUILD/ceph-19.0.0-2145-g3be9a5c9/src/mds/Locker.cc: 626: FAILED ceph_assert(obj->is_auth()) 2024-03-28T18:12:45.521 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:12:45 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[70501]: 2024-03-28T18:12:45.521 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:12:45 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[70501]: ceph version 19.0.0-2145-g3be9a5c9 (3be9a5c9bc793e11e2800a8c0c696e8b46742033) squid (dev) 2024-03-28T18:12:45.521 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:12:45 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[70501]: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x135) [0x7f4a031184a1] 2024-03-28T18:12:45.521 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:12:45 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[70501]: 2: /usr/lib64/ceph/libceph-common.so.2(+0x2d2667) [0x7f4a03118667] 2024-03-28T18:12:45.521 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:12:45 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[70501]: 3: (Locker::set_xlocks_done(MutationImpl*, bool)+0x493) [0x55bce68c0223] 2024-03-28T18:12:45.521 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:12:45 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[70501]: 4: (Server::early_reply(boost::intrusive_ptr const&, CInode*, CDentry*)+0x3e6) [0x55bce673a5d6] 2024-03-28T18:12:45.521 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:12:45 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[70501]: 5: (Server::journal_and_reply(boost::intrusive_ptr const&, CInode*, CDentry*, LogEvent*, MDSLogContextBase*)+0xc1) [0x55bce673ada1] 2024-03-28T18:12:45.521 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:12:45 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[70501]: 6: (Server::handle_client_mkdir(boost::intrusive_ptr const&)+0x5ba) [0x55bce67706aa] 2024-03-28T18:12:45.521 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:12:45 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[70501]: 7: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0x733) [0x55bce678d233] 2024-03-28T18:12:45.521 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:12:45 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[70501]: 8: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x55bce678f197] 2024-03-28T18:12:45.522 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:12:45 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[70501]: 9: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x55bce668e11b] 2024-03-28T18:12:45.522 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:12:45 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[70501]: 10: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x55bce668e6bc] 2024-03-28T18:12:45.522 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:12:45 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[70501]: 11: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bf) [0x55bce66781ff] 2024-03-28T18:12:45.522 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:12:45 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[70501]: 12: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x478) [0x7f4a033aa488] 2024-03-28T18:12:45.522 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:12:45 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[70501]: 13: (DispatchQueue::entry()+0x51f) [0x7f4a033a760f] 2024-03-28T18:12:45.522 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:12:45 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[70501]: 14: (DispatchQueue::DispatchThread::entry()+0x11) [0x7f4a0347ae81] 2024-03-28T18:12:45.522 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:12:45 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[70501]: 15: /lib64/libpthread.so.0(+0x81ca) [0x7f4a01e951ca] 2024-03-28T18:12:45.522 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:12:45 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[70501]: 16: clone() 2024-03-28T18:12:45.522 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:12:45 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[70501]: 2024-03-28T18:12:45.522 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:12:45 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[70501]: 0> 2024-03-28T18:12:45.420+0000 7f49f7800700 -1 *** Caught signal (Aborted) ** 2024-03-28T18:12:45.522 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:12:45 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[70501]: in thread 7f49f7800700 thread_name:ms_dispatch 2024-03-28T18:12:45.522 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:12:45 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[70501]: 2024-03-28T18:12:45.522 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:12:45 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[70501]: ceph version 19.0.0-2145-g3be9a5c9 (3be9a5c9bc793e11e2800a8c0c696e8b46742033) squid (dev) 2024-03-28T18:12:45.522 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:12:45 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[70501]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7f4a01e9fd20] 2024-03-28T18:12:45.522 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:12:45 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[70501]: 2: gsignal() 2024-03-28T18:12:45.523 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:12:45 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[70501]: 3: abort() 2024-03-28T18:12:45.523 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:12:45 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[70501]: 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x18f) [0x7f4a031184fb] 2024-03-28T18:12:45.523 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:12:45 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[70501]: 5: /usr/lib64/ceph/libceph-common.so.2(+0x2d2667) [0x7f4a03118667] 2024-03-28T18:12:45.523 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:12:45 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[70501]: 6: (Locker::set_xlocks_done(MutationImpl*, bool)+0x493) [0x55bce68c0223] 2024-03-28T18:12:45.523 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:12:45 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[70501]: 7: (Server::early_reply(boost::intrusive_ptr const&, CInode*, CDentry*)+0x3e6) [0x55bce673a5d6] 2024-03-28T18:12:45.523 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:12:45 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[70501]: 8: (Server::journal_and_reply(boost::intrusive_ptr const&, CInode*, CDentry*, LogEvent*, MDSLogContextBase*)+0xc1) [0x55bce673ada1] 2024-03-28T18:12:45.523 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:12:45 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[70501]: 9: (Server::handle_client_mkdir(boost::intrusive_ptr const&)+0x5ba) [0x55bce67706aa] 2024-03-28T18:12:45.523 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:12:45 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[70501]: 10: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0x733) [0x55bce678d233] 2024-03-28T18:12:45.523 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:12:45 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[70501]: 11: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x55bce678f197] 2024-03-28T18:12:45.523 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:12:45 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[70501]: 12: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x55bce668e11b] 2024-03-28T18:12:45.523 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:12:45 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[70501]: 13: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x55bce668e6bc] 2024-03-28T18:12:45.523 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:12:45 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[70501]: 14: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bf) [0x55bce66781ff] 2024-03-28T18:12:45.523 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:12:45 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[70501]: 15: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x478) [0x7f4a033aa488] 2024-03-28T18:12:45.523 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:12:45 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[70501]: 16: (DispatchQueue::entry()+0x51f) [0x7f4a033a760f] 2024-03-28T18:12:45.523 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:12:45 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[70501]: 17: (DispatchQueue::DispatchThread::entry()+0x11) [0x7f4a0347ae81] 2024-03-28T18:12:45.524 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:12:45 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[70501]: 18: /lib64/libpthread.so.0(+0x81ca) [0x7f4a01e951ca] 2024-03-28T18:12:45.524 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:12:45 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[70501]: 19: clone() 2024-03-28T18:12:45.524 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:12:45 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[70501]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-03-28T18:12:45.524 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:12:45 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[70501]: 2024-03-28T18:12:45.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:12:45 smithi079 ceph-mon[35312]: mds.? [v2:172.21.15.134:6834/726536645,v1:172.21.15.134:6835/726536645] up:rejoin 2024-03-28T18:12:45.683 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:12:45 smithi079 ceph-mon[35312]: fsmap cephfs:5/5 4 up:standby-replay 3 up:standby 1 up:rejoin 4 up:active 2024-03-28T18:12:45.683 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:12:45 smithi079 ceph-mon[35312]: daemon mds.f is now active in filesystem cephfs as rank 1 2024-03-28T18:12:45.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:12:45 smithi022 ceph-mon[31985]: mds.? [v2:172.21.15.134:6834/726536645,v1:172.21.15.134:6835/726536645] up:rejoin 2024-03-28T18:12:45.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:12:45 smithi022 ceph-mon[31985]: fsmap cephfs:5/5 4 up:standby-replay 3 up:standby 1 up:rejoin 4 up:active 2024-03-28T18:12:45.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:12:45 smithi022 ceph-mon[31985]: daemon mds.f is now active in filesystem cephfs as rank 1 2024-03-28T18:12:45.903 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:12:45 smithi134 podman[70542]: 2024-03-28 18:12:45.577082115 +0000 UTC m=+0.047319715 container died fd770f89545b6274c7fcb94b7a69b6b45042a9397d6080b451ade0b4b9397af3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f, org.label-schema.schema-version=1.0, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.build-date=20240319, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_CLEAN=True, ceph=True, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, GIT_BRANCH=HEAD) 2024-03-28T18:12:46.675 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:12:46 smithi134 podman[70542]: 2024-03-28 18:12:46.277275561 +0000 UTC m=+0.747513161 container remove fd770f89545b6274c7fcb94b7a69b6b45042a9397d6080b451ade0b4b9397af3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.vendor=CentOS, ceph=True, io.buildah.version=1.33.5, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240319, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-03-28T18:12:46.675 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:12:46 smithi134 systemd[1]: ceph-fd220748-ed2b-11ee-95d0-87774f69a715@mds.f.service: Main process exited, code=exited, status=134/n/a 2024-03-28T18:12:46.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:12:46 smithi134 ceph-mon[35037]: pgmap v168: 97 pgs: 97 active+clean; 117 MiB data, 785 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 204 B/s wr, 10 op/s 2024-03-28T18:12:46.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:12:46 smithi134 ceph-mon[35037]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-03-28T18:12:46.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:12:46 smithi134 ceph-mon[35037]: Cluster is now healthy 2024-03-28T18:12:46.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:12:46 smithi134 ceph-mon[35037]: mds.? [v2:172.21.15.134:6834/726536645,v1:172.21.15.134:6835/726536645] up:active 2024-03-28T18:12:46.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:12:46 smithi134 ceph-mon[35037]: fsmap cephfs:5 4 up:standby-replay 3 up:standby 5 up:active 2024-03-28T18:12:46.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:12:46 smithi134 ceph-mon[35037]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-28T18:12:46.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:12:46 smithi079 ceph-mon[35312]: pgmap v168: 97 pgs: 97 active+clean; 117 MiB data, 785 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 204 B/s wr, 10 op/s 2024-03-28T18:12:46.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:12:46 smithi079 ceph-mon[35312]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-03-28T18:12:46.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:12:46 smithi079 ceph-mon[35312]: Cluster is now healthy 2024-03-28T18:12:46.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:12:46 smithi079 ceph-mon[35312]: mds.? [v2:172.21.15.134:6834/726536645,v1:172.21.15.134:6835/726536645] up:active 2024-03-28T18:12:46.683 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:12:46 smithi079 ceph-mon[35312]: fsmap cephfs:5 4 up:standby-replay 3 up:standby 5 up:active 2024-03-28T18:12:46.683 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:12:46 smithi079 ceph-mon[35312]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-28T18:12:46.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:12:46 smithi022 ceph-mon[31985]: pgmap v168: 97 pgs: 97 active+clean; 117 MiB data, 785 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 204 B/s wr, 10 op/s 2024-03-28T18:12:46.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:12:46 smithi022 ceph-mon[31985]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-03-28T18:12:46.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:12:46 smithi022 ceph-mon[31985]: Cluster is now healthy 2024-03-28T18:12:46.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:12:46 smithi022 ceph-mon[31985]: mds.? [v2:172.21.15.134:6834/726536645,v1:172.21.15.134:6835/726536645] up:active 2024-03-28T18:12:46.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:12:46 smithi022 ceph-mon[31985]: fsmap cephfs:5 4 up:standby-replay 3 up:standby 5 up:active 2024-03-28T18:12:46.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:12:46 smithi022 ceph-mon[31985]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-28T18:12:47.174 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:12:46 smithi134 systemd[1]: ceph-fd220748-ed2b-11ee-95d0-87774f69a715@mds.f.service: Failed with result 'exit-code'. 2024-03-28T18:12:47.175 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:12:46 smithi134 systemd[1]: ceph-fd220748-ed2b-11ee-95d0-87774f69a715@mds.f.service: Consumed 2.022s CPU time. 2024-03-28T18:12:47.740 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-28T18:12:47.740 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-28T18:12:47.977 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:12:47.977+0000 7fdd9d400640 1 -- 172.21.15.22:0/1039006745 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fdd98101ef0 msgr2=0x7fdd9810e3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:12:47.977 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:12:47.977+0000 7fdd9d400640 1 --2- 172.21.15.22:0/1039006745 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fdd98101ef0 0x7fdd9810e3b0 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7fdd90009ca0 tx=0x7fdd9002c100 comp rx=0 tx=0).stop 2024-03-28T18:12:47.977 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:12:47.977+0000 7fdd9d400640 1 -- 172.21.15.22:0/1039006745 shutdown_connections 2024-03-28T18:12:47.977 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:12:47.977+0000 7fdd9d400640 1 --2- 172.21.15.22:0/1039006745 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fdd98101ef0 0x7fdd9810e3b0 unknown :-1 s=CLOSED pgs=717 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:12:47.977 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:12:47.977+0000 7fdd9d400640 1 --2- 172.21.15.22:0/1039006745 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fdd98101530 0x7fdd981019b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:12:47.977 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:12:47.977+0000 7fdd9d400640 1 --2- 172.21.15.22:0/1039006745 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdd981075f0 0x7fdd981079d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:12:47.977 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:12:47.977+0000 7fdd9d400640 1 -- 172.21.15.22:0/1039006745 >> 172.21.15.22:0/1039006745 conn(0x7fdd980fd000 msgr2=0x7fdd980ff420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:12:47.977 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:12:47.977+0000 7fdd9d400640 1 -- 172.21.15.22:0/1039006745 shutdown_connections 2024-03-28T18:12:47.978 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:12:47.977+0000 7fdd9d400640 1 -- 172.21.15.22:0/1039006745 wait complete. 2024-03-28T18:12:47.980 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:12:47.978+0000 7fdd9d400640 1 Processor -- start 2024-03-28T18:12:47.980 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:12:47.978+0000 7fdd9d400640 1 -- start start 2024-03-28T18:12:47.980 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:12:47.978+0000 7fdd9d400640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fdd98101530 0x7fdd98199e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:12:47.980 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:12:47.978+0000 7fdd9d400640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fdd98101ef0 0x7fdd9819a3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:12:47.980 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:12:47.978+0000 7fdd9d400640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdd981075f0 0x7fdd9819e720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:12:47.980 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:12:47.978+0000 7fdd9d400640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd981118c0 con 0x7fdd98101530 2024-03-28T18:12:47.980 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:12:47.978+0000 7fdd9d400640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd98111740 con 0x7fdd98101ef0 2024-03-28T18:12:47.980 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:12:47.978+0000 7fdd9d400640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd98111a40 con 0x7fdd981075f0 2024-03-28T18:12:47.980 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:12:47.979+0000 7fdd97e00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fdd98101530 0x7fdd98199e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:12:47.980 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:12:47.979+0000 7fdd97e00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fdd98101530 0x7fdd98199e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:53086/0 (socket says 172.21.15.22:53086) 2024-03-28T18:12:47.980 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:12:47.979+0000 7fdd97e00640 1 -- 172.21.15.22:0/1589273660 learned_addr learned my addr 172.21.15.22:0/1589273660 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-28T18:12:47.980 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:12:47.979+0000 7fdd97400640 1 --2- 172.21.15.22:0/1589273660 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fdd98101ef0 0x7fdd9819a3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:12:47.980 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:12:47.979+0000 7fdd9ca00640 1 --2- 172.21.15.22:0/1589273660 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdd981075f0 0x7fdd9819e720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:12:47.981 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:12:47.979+0000 7fdd97e00640 1 -- 172.21.15.22:0/1589273660 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdd981075f0 msgr2=0x7fdd9819e720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:12:47.981 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:12:47.979+0000 7fdd97e00640 1 --2- 172.21.15.22:0/1589273660 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdd981075f0 0x7fdd9819e720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:12:47.981 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:12:47.979+0000 7fdd97e00640 1 -- 172.21.15.22:0/1589273660 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fdd98101ef0 msgr2=0x7fdd9819a3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:12:47.981 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:12:47.979+0000 7fdd97e00640 1 --2- 172.21.15.22:0/1589273660 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fdd98101ef0 0x7fdd9819a3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:12:47.981 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:12:47.979+0000 7fdd97e00640 1 -- 172.21.15.22:0/1589273660 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd9819ee50 con 0x7fdd98101530 2024-03-28T18:12:47.981 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:12:47.979+0000 7fdd97e00640 1 --2- 172.21.15.22:0/1589273660 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fdd98101530 0x7fdd98199e60 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7fdd8800d770 tx=0x7fdd8800db70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:12:47.981 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:12:47.980+0000 7fdd94c00640 1 -- 172.21.15.22:0/1589273660 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd88017070 con 0x7fdd98101530 2024-03-28T18:12:47.981 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:12:47.980+0000 7fdd94c00640 1 -- 172.21.15.22:0/1589273660 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7fdd88009820 con 0x7fdd98101530 2024-03-28T18:12:47.981 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:12:47.980+0000 7fdd9d400640 1 -- 172.21.15.22:0/1589273660 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdd9819f140 con 0x7fdd98101530 2024-03-28T18:12:47.981 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:12:47.980+0000 7fdd94c00640 1 -- 172.21.15.22:0/1589273660 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd88004c30 con 0x7fdd98101530 2024-03-28T18:12:47.981 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:12:47.980+0000 7fdd9d400640 1 -- 172.21.15.22:0/1589273660 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdd981a69d0 con 0x7fdd98101530 2024-03-28T18:12:47.984 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:12:47.984+0000 7fdda17ee740 1 Processor -- start 2024-03-28T18:12:47.984 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:12:47.985+0000 7fdda17ee740 1 -- start start 2024-03-28T18:12:47.984 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:12:47.985+0000 7fdd94c00640 1 -- 172.21.15.22:0/1589273660 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 26) v1 ==== 137421+0+0 (secure 0 0 0) 0x7fdd880099d0 con 0x7fdd98101530 2024-03-28T18:12:47.987 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:12:47.985+0000 7fdda17ee740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a39d8d1dd0 0x55a39d8d0b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:12:47.987 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:12:47.985+0000 7fdda17ee740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55a39d8d10f0 0x55a39d8d1570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:12:47.987 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:12:47.985+0000 7fdda17ee740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a39d8da460 0x55a39d8dc850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:12:47.987 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:12:47.985+0000 7fdda17ee740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a39d835250 con 0x55a39d8da460 2024-03-28T18:12:47.987 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:12:47.985+0000 7fdda17ee740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a39d7dda30 con 0x55a39d8d10f0 2024-03-28T18:12:47.987 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:12:47.985+0000 7fdda17ee740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a39d8312f0 con 0x55a39d8d1dd0 2024-03-28T18:12:47.987 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:12:47.985+0000 7fdd97400640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a39d8da460 0x55a39d8dc850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:12:47.987 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:12:47.985+0000 7fdd97400640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a39d8da460 0x55a39d8dc850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:53096/0 (socket says 172.21.15.22:53096) 2024-03-28T18:12:47.987 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:12:47.985+0000 7fdd97400640 1 -- 172.21.15.22:0/1508884658 learned_addr learned my addr 172.21.15.22:0/1508884658 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-28T18:12:47.987 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:12:47.986+0000 7fdd97e00640 1 --2- 172.21.15.22:0/1508884658 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55a39d8d10f0 0x55a39d8d1570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:12:47.987 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:12:47.986+0000 7fdd9ca00640 1 --2- 172.21.15.22:0/1508884658 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a39d8d1dd0 0x55a39d8d0b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:12:47.987 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:12:47.986+0000 7fdd97400640 1 -- 172.21.15.22:0/1508884658 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a39d8d1dd0 msgr2=0x55a39d8d0b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T18:12:47.987 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:12:47.986+0000 7fdd97400640 1 --2- 172.21.15.22:0/1508884658 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a39d8d1dd0 0x55a39d8d0b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:12:47.988 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:12:47.986+0000 7fdd97400640 1 -- 172.21.15.22:0/1508884658 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55a39d8d10f0 msgr2=0x55a39d8d1570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T18:12:47.988 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:12:47.986+0000 7fdd97400640 1 --2- 172.21.15.22:0/1508884658 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55a39d8d10f0 0x55a39d8d1570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:12:47.988 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:12:47.986+0000 7fdd97400640 1 -- 172.21.15.22:0/1508884658 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a39d7c8690 con 0x55a39d8da460 2024-03-28T18:12:47.988 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:12:47.986+0000 7fdd97400640 1 --2- 172.21.15.22:0/1508884658 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a39d8da460 0x55a39d8dc850 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7fdd8400aca0 tx=0x7fdd8400f790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ae39810ac9aaae15 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:12:47.988 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:12:47.986+0000 7fdd8d200640 1 -- 172.21.15.22:0/1508884658 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd8401d070 con 0x55a39d8da460 2024-03-28T18:12:47.988 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:12:47.986+0000 7fdd8d200640 1 -- 172.21.15.22:0/1508884658 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7fdd84018b10 con 0x55a39d8da460 2024-03-28T18:12:47.988 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:12:47.987+0000 7fdd8d200640 1 -- 172.21.15.22:0/1508884658 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd84010bf0 con 0x55a39d8da460 2024-03-28T18:12:47.988 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:12:47.987+0000 7fdd94c00640 1 --2- 172.21.15.22:0/1589273660 >> [v2:172.21.15.22:6800/1357365082,v1:172.21.15.22:6801/1357365082] conn(0x7fdd7c0a2970 0x7fdd7c0a4e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:12:47.988 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:12:47.987+0000 7fdd94c00640 1 -- 172.21.15.22:0/1589273660 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 9185+0+0 (secure 0 0 0) 0x7fdd880ceb80 con 0x7fdd98101530 2024-03-28T18:12:47.988 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:12:47.987+0000 7fdd9ca00640 1 --2- 172.21.15.22:0/1589273660 >> [v2:172.21.15.22:6800/1357365082,v1:172.21.15.22:6801/1357365082] conn(0x7fdd7c0a2970 0x7fdd7c0a4e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:12:47.988 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:12:47.987+0000 7fdda17ee740 1 -- 172.21.15.22:0/1508884658 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a39d8da460 msgr2=0x55a39d8dc850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:12:47.988 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:12:47.987+0000 7fdda17ee740 1 --2- 172.21.15.22:0/1508884658 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a39d8da460 0x55a39d8dc850 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7fdd8400aca0 tx=0x7fdd8400f790 comp rx=0 tx=0).stop 2024-03-28T18:12:47.992 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:12:47.987+0000 7fdda17ee740 1 -- 172.21.15.22:0/1508884658 shutdown_connections 2024-03-28T18:12:47.992 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:12:47.987+0000 7fdda17ee740 1 --2- 172.21.15.22:0/1508884658 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a39d8da460 0x55a39d8dc850 unknown :-1 s=CLOSED pgs=719 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:12:47.992 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:12:47.987+0000 7fdda17ee740 1 --2- 172.21.15.22:0/1508884658 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55a39d8d10f0 0x55a39d8d1570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:12:47.992 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:12:47.987+0000 7fdda17ee740 1 --2- 172.21.15.22:0/1508884658 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a39d8d1dd0 0x55a39d8d0b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:12:47.993 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:12:47.987+0000 7fdda17ee740 1 -- 172.21.15.22:0/1508884658 >> 172.21.15.22:0/1508884658 conn(0x55a39d898200 msgr2=0x55a39d8cbb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:12:47.993 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:12:47.988+0000 7fdda17ee740 1 -- 172.21.15.22:0/1508884658 shutdown_connections 2024-03-28T18:12:47.993 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:12:47.988+0000 7fdda17ee740 1 -- 172.21.15.22:0/1508884658 wait complete. 2024-03-28T18:12:47.993 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:12:47.988+0000 7fdd9ca00640 1 --2- 172.21.15.22:0/1589273660 >> [v2:172.21.15.22:6800/1357365082,v1:172.21.15.22:6801/1357365082] conn(0x7fdd7c0a2970 0x7fdd7c0a4e30 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fdd90009430 tx=0x7fdd90009360 comp rx=0 tx=0).ready entity=mgr.15522 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:12:47.993 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:12:47.988+0000 7fdda17ee740 1 -- start start 2024-03-28T18:12:47.993 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:12:47.988+0000 7fdda17ee740 1 Processor -- start 2024-03-28T18:12:47.993 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:12:47.989+0000 7fdda17ee740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a39d8d10f0 0x55a39d8d9af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:12:47.993 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:12:47.989+0000 7fdda17ee740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55a39d8d1dd0 0x55a39d8d6e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:12:47.993 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:12:47.989+0000 7fdda17ee740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a39d8da460 0x55a39d8d73c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:12:47.993 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:12:47.989+0000 7fdda17ee740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a39d8dd2d0 con 0x55a39d8da460 2024-03-28T18:12:47.993 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:12:47.989+0000 7fdda17ee740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a39d7f0e80 con 0x55a39d8d1dd0 2024-03-28T18:12:47.993 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:12:47.989+0000 7fdda17ee740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a39d8dd450 con 0x55a39d8d10f0 2024-03-28T18:12:47.993 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:12:47.989+0000 7fdda17ee740 10 client.0 _ll_register_callbacks cb 0x55a39d803130 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-28T18:12:47.993 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:12:47.989+0000 7fdd9ca00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a39d8da460 0x55a39d8d73c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:12:47.993 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:12:47.989+0000 7fdd9ca00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a39d8da460 0x55a39d8d73c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:53106/0 (socket says 172.21.15.22:53106) 2024-03-28T18:12:47.994 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:12:47.989+0000 7fdd9ca00640 1 -- 172.21.15.22:0/4072523553 learned_addr learned my addr 172.21.15.22:0/4072523553 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-28T18:12:47.994 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:12:47.989+0000 7fdd97400640 1 --2- 172.21.15.22:0/4072523553 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55a39d8d1dd0 0x55a39d8d6e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:12:47.994 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:12:47.989+0000 7fdd97e00640 1 --2- 172.21.15.22:0/4072523553 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a39d8d10f0 0x55a39d8d9af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:12:47.994 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:12:47.990+0000 7fdd9ca00640 1 -- 172.21.15.22:0/4072523553 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a39d8d10f0 msgr2=0x55a39d8d9af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T18:12:47.994 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:12:47.990+0000 7fdd9ca00640 1 --2- 172.21.15.22:0/4072523553 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a39d8d10f0 0x55a39d8d9af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:12:47.994 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:12:47.990+0000 7fdd9ca00640 1 -- 172.21.15.22:0/4072523553 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55a39d8d1dd0 msgr2=0x55a39d8d6e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T18:12:47.994 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:12:47.990+0000 7fdd9ca00640 1 --2- 172.21.15.22:0/4072523553 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55a39d8d1dd0 0x55a39d8d6e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:12:47.994 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:12:47.990+0000 7fdd9ca00640 1 -- 172.21.15.22:0/4072523553 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55a39d6137d0 con 0x55a39d8da460 2024-03-28T18:12:47.994 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:12:47.990+0000 7fdd97e00640 1 --2- 172.21.15.22:0/4072523553 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a39d8d10f0 0x55a39d8d9af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-28T18:12:47.994 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:12:47.990+0000 7fdd9ca00640 1 --2- 172.21.15.22:0/4072523553 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a39d8da460 0x55a39d8d73c0 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7fdd90036000 tx=0x7fdd9002c100 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ec4a59d5d97a3e5f server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:12:47.994 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:12:47.990+0000 7fdd81600640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-03-28T18:12:47.994 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:12:47.990+0000 7fdd81600640 1 -- 172.21.15.22:0/4072523553 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd9004f020 con 0x55a39d8da460 2024-03-28T18:12:47.994 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:12:47.991+0000 7fdd81600640 1 -- 172.21.15.22:0/4072523553 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(102..102 src has 1..102) v4 ==== 9185+0+0 (secure 0 0 0) 0x7fdd90004100 con 0x55a39d8da460 2024-03-28T18:12:47.994 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:12:47.991+0000 7fdda17ee740 1 -- 172.21.15.22:0/4072523553 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fdd84018b10 con 0x55a39d8da460 2024-03-28T18:12:47.995 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:12:47.991+0000 7fdd81600640 1 -- 172.21.15.22:0/4072523553 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=92) v2 ==== 24+0+0 (secure 0 0 0) 0x7fdd90039040 con 0x55a39d8da460 2024-03-28T18:12:47.995 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:12:47.991+0000 7fdda17ee740 10 client.16596 fetch_fsmap learned FSMap version 92 2024-03-28T18:12:47.995 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:12:47.991+0000 7fdda17ee740 1 -- 172.21.15.22:0/4072523553 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=92,monmap=4+}) v3 -- 0x55a39d5493f0 con 0x55a39d8da460 2024-03-28T18:12:47.995 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:12:47.991+0000 7fdd81600640 1 -- 172.21.15.22:0/4072523553 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7fdd84018b10 con 0x55a39d8da460 2024-03-28T18:12:47.995 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:12:47.991+0000 7fdd81600640 1 -- 172.21.15.22:0/4072523553 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 92) v1 ==== 7946+0+0 (secure 0 0 0) 0x7fdd9003faa0 con 0x55a39d8da460 2024-03-28T18:12:47.995 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:12:47.991+0000 7fdda17ee740 10 client.16596 fetch_fsmap finished waiting for FSMap version 92 2024-03-28T18:12:47.995 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:12:47.991+0000 7fdda17ee740 10 client.16596 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-28T18:12:47.995 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:12:47.991+0000 7fdda17ee740 20 client.16596 populate_metadata read hostname 'smithi022' 2024-03-28T18:12:47.995 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:12:47.991+0000 7fdda17ee740 1 --2- 172.21.15.22:0/4072523553 >> [v2:172.21.15.79:6838/2441542443,v1:172.21.15.79:6839/2441542443] conn(0x55a39d8ea370 0x55a39d8ec780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:12:47.995 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:12:47.991+0000 7fdda17ee740 4 client.16596 mds_command: new command op to 34553 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-28T18:12:47.995 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:12:47.992+0000 7fdda17ee740 1 -- 172.21.15.22:0/4072523553 --> [v2:172.21.15.79:6838/2441542443,v1:172.21.15.79:6839/2441542443] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55a39d842790 con 0x55a39d8ea370 2024-03-28T18:12:47.995 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:12:47.992+0000 7fdd97e00640 1 --2- 172.21.15.22:0/4072523553 >> [v2:172.21.15.79:6838/2441542443,v1:172.21.15.79:6839/2441542443] conn(0x55a39d8ea370 0x55a39d8ec780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:12:47.996 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:12:47.993+0000 7fdd97e00640 1 --2- 172.21.15.22:0/4072523553 >> [v2:172.21.15.79:6838/2441542443,v1:172.21.15.79:6839/2441542443] conn(0x55a39d8ea370 0x55a39d8ec780 crc :-1 s=READY pgs=162 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e8824c46e2b62228 server_cookie=36cc694370f4b385 in_seq=0 out_seq=0 2024-03-28T18:12:47.997 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:12:47.993+0000 7fdd81600640 10 client.16596 ms_handle_connect on v2:172.21.15.79:6838/2441542443 2024-03-28T18:12:47.997 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:12:47.995+0000 7fdd81600640 1 -- 172.21.15.22:0/4072523553 <== mds.0 v2:172.21.15.79:6838/2441542443 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x55a39d842790 con 0x55a39d8ea370 2024-03-28T18:12:47.997 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:12:47.995+0000 7fdd81600640 10 client.16596 handle_command_reply: tid=0 2024-03-28T18:12:47.997 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:12:47.995+0000 7fdda17ee740 1 client.16596 shutdown 2024-03-28T18:12:47.997 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:12:47.995+0000 7fdda17ee740 1 -- 172.21.15.22:0/4072523553 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a39d8da460 msgr2=0x55a39d8d73c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:12:47.997 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:12:47.995+0000 7fdda17ee740 1 --2- 172.21.15.22:0/4072523553 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a39d8da460 0x55a39d8d73c0 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7fdd90036000 tx=0x7fdd9002c100 comp rx=0 tx=0).stop 2024-03-28T18:12:47.997 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:12:47.995+0000 7fdda17ee740 1 -- 172.21.15.22:0/4072523553 shutdown_connections 2024-03-28T18:12:47.997 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:12:47.995+0000 7fdda17ee740 1 --2- 172.21.15.22:0/4072523553 >> [v2:172.21.15.79:6838/2441542443,v1:172.21.15.79:6839/2441542443] conn(0x55a39d8ea370 0x55a39d8ec780 crc :-1 s=READY pgs=162 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:12:47.997 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:12:47.995+0000 7fdda17ee740 1 --2- 172.21.15.22:0/4072523553 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a39d8da460 0x55a39d8d73c0 unknown :-1 s=CLOSED pgs=720 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:12:47.997 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:12:47.995+0000 7fdda17ee740 1 --2- 172.21.15.22:0/4072523553 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55a39d8d1dd0 0x55a39d8d6e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:12:47.997 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:12:47.995+0000 7fdda17ee740 1 --2- 172.21.15.22:0/4072523553 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a39d8d10f0 0x55a39d8d9af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:12:47.997 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:12:47.995+0000 7fdd81600640 0 client.16596 ms_handle_reset on v2:172.21.15.79:6838/2441542443 2024-03-28T18:12:47.997 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:12:47.995+0000 7fdda17ee740 1 -- 172.21.15.22:0/4072523553 >> 172.21.15.22:0/4072523553 conn(0x55a39d898200 msgr2=0x55a39d8cd890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:12:47.997 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:12:47.995+0000 7fdda17ee740 1 -- 172.21.15.22:0/4072523553 shutdown_connections 2024-03-28T18:12:47.997 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:12:47.995+0000 7fdda17ee740 1 -- 172.21.15.22:0/4072523553 wait complete. 2024-03-28T18:12:47.998 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:12:47.996+0000 7fdda17ee740 20 client.16596 trim_cache size 0 max 16384 2024-03-28T18:12:48.015 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:12:48.013+0000 7fdda17ee740 1 Processor -- start 2024-03-28T18:12:48.015 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:12:48.013+0000 7fdda17ee740 1 -- start start 2024-03-28T18:12:48.015 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:12:48.014+0000 7fdda17ee740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a39d8d10f0 0x55a39d8d95b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:12:48.015 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:12:48.014+0000 7fdda17ee740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55a39d8d1dd0 0x55a39d8d9af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:12:48.015 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:12:48.014+0000 7fdda17ee740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a39d8da460 0x55a39d8db850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:12:48.015 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:12:48.014+0000 7fdda17ee740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd981af640 con 0x55a39d8d10f0 2024-03-28T18:12:48.015 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:12:48.014+0000 7fdda17ee740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd981b1050 con 0x55a39d8d1dd0 2024-03-28T18:12:48.015 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:12:48.014+0000 7fdda17ee740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd981afca0 con 0x55a39d8da460 2024-03-28T18:12:48.015 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:12:48.014+0000 7fdd97e00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a39d8d10f0 0x55a39d8d95b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:12:48.015 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:12:48.014+0000 7fdd97e00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a39d8d10f0 0x55a39d8d95b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:53112/0 (socket says 172.21.15.22:53112) 2024-03-28T18:12:48.015 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:12:48.014+0000 7fdd97e00640 1 -- 172.21.15.22:0/2838417878 learned_addr learned my addr 172.21.15.22:0/2838417878 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-28T18:12:48.015 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:12:48.014+0000 7fdd97400640 1 --2- 172.21.15.22:0/2838417878 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55a39d8d1dd0 0x55a39d8d9af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:12:48.015 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:12:48.014+0000 7fdd9ca00640 1 --2- 172.21.15.22:0/2838417878 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a39d8da460 0x55a39d8db850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:12:48.015 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:12:48.014+0000 7fdd97e00640 1 -- 172.21.15.22:0/2838417878 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a39d8da460 msgr2=0x55a39d8db850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T18:12:48.016 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:12:48.014+0000 7fdd97e00640 1 --2- 172.21.15.22:0/2838417878 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a39d8da460 0x55a39d8db850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:12:48.016 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:12:48.014+0000 7fdd97e00640 1 -- 172.21.15.22:0/2838417878 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55a39d8d1dd0 msgr2=0x55a39d8d9af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T18:12:48.016 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:12:48.014+0000 7fdd97e00640 1 --2- 172.21.15.22:0/2838417878 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55a39d8d1dd0 0x55a39d8d9af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:12:48.016 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:12:48.014+0000 7fdd97e00640 1 -- 172.21.15.22:0/2838417878 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a39d5090f0 con 0x55a39d8d10f0 2024-03-28T18:12:48.016 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:12:48.014+0000 7fdd9ca00640 1 --2- 172.21.15.22:0/2838417878 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a39d8da460 0x55a39d8db850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T18:12:48.016 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:12:48.015+0000 7fdd97e00640 1 --2- 172.21.15.22:0/2838417878 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a39d8d10f0 0x55a39d8d95b0 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7fdd8800dfd0 tx=0x7fdd880cee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b578b4f439803383 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:12:48.016 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:12:48.015+0000 7fdd83e00640 1 -- 172.21.15.22:0/2838417878 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd88020040 con 0x55a39d8d10f0 2024-03-28T18:12:48.016 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:12:48.015+0000 7fdd83e00640 1 -- 172.21.15.22:0/2838417878 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7fdd8809f090 con 0x55a39d8d10f0 2024-03-28T18:12:48.016 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:12:48.015+0000 7fdd83e00640 1 -- 172.21.15.22:0/2838417878 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd880975b0 con 0x55a39d8d10f0 2024-03-28T18:12:48.019 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:12:48.016+0000 7fdda17ee740 1 -- 172.21.15.22:0/2838417878 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a39d8d10f0 msgr2=0x55a39d8d95b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:12:48.019 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:12:48.016+0000 7fdda17ee740 1 --2- 172.21.15.22:0/2838417878 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a39d8d10f0 0x55a39d8d95b0 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7fdd8800dfd0 tx=0x7fdd880cee80 comp rx=0 tx=0).stop 2024-03-28T18:12:48.020 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:12:48.016+0000 7fdda17ee740 1 -- 172.21.15.22:0/2838417878 shutdown_connections 2024-03-28T18:12:48.020 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:12:48.016+0000 7fdda17ee740 1 --2- 172.21.15.22:0/2838417878 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a39d8da460 0x55a39d8db850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:12:48.020 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:12:48.016+0000 7fdda17ee740 1 --2- 172.21.15.22:0/2838417878 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55a39d8d1dd0 0x55a39d8d9af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:12:48.020 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:12:48.016+0000 7fdda17ee740 1 --2- 172.21.15.22:0/2838417878 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a39d8d10f0 0x55a39d8d95b0 unknown :-1 s=CLOSED pgs=721 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:12:48.020 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:12:48.016+0000 7fdda17ee740 1 -- 172.21.15.22:0/2838417878 >> 172.21.15.22:0/2838417878 conn(0x55a39d898200 msgr2=0x55a39d8d5750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:12:48.020 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:12:48.016+0000 7fdda17ee740 1 -- 172.21.15.22:0/2838417878 shutdown_connections 2024-03-28T18:12:48.020 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:12:48.016+0000 7fdda17ee740 1 -- 172.21.15.22:0/2838417878 wait complete. 2024-03-28T18:12:48.020 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:12:48.016+0000 7fdda17ee740 1 -- start start 2024-03-28T18:12:48.020 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:12:48.016+0000 7fdda17ee740 1 Processor -- start 2024-03-28T18:12:48.020 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:12:48.017+0000 7fdda17ee740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a39d8d10f0 0x55a39d8d3c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:12:48.020 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:12:48.017+0000 7fdda17ee740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a39d8d1dd0 0x55a39d8d41a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:12:48.020 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:12:48.017+0000 7fdda17ee740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55a39d8da460 0x55a39d944c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:12:48.020 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:12:48.017+0000 7fdda17ee740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd981b09f0 con 0x55a39d8d10f0 2024-03-28T18:12:48.020 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:12:48.017+0000 7fdda17ee740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd981b0300 con 0x55a39d8da460 2024-03-28T18:12:48.021 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:12:48.017+0000 7fdda17ee740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd981ad800 con 0x55a39d8d1dd0 2024-03-28T18:12:48.021 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:12:48.017+0000 7fdda17ee740 10 client.0 _ll_register_callbacks cb 0x7fdd90037070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-28T18:12:48.021 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:12:48.017+0000 7fdd97e00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a39d8d10f0 0x55a39d8d3c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:12:48.021 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:12:48.017+0000 7fdd97e00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a39d8d10f0 0x55a39d8d3c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:53116/0 (socket says 172.21.15.22:53116) 2024-03-28T18:12:48.021 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:12:48.017+0000 7fdd97e00640 1 -- 172.21.15.22:0/4048156698 learned_addr learned my addr 172.21.15.22:0/4048156698 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-28T18:12:48.021 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:12:48.017+0000 7fdd9ca00640 1 --2- 172.21.15.22:0/4048156698 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55a39d8da460 0x55a39d944c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:12:48.021 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:12:48.018+0000 7fdd97400640 1 --2- 172.21.15.22:0/4048156698 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a39d8d1dd0 0x55a39d8d41a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:12:48.021 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:12:48.018+0000 7fdd97e00640 1 -- 172.21.15.22:0/4048156698 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a39d8d1dd0 msgr2=0x55a39d8d41a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T18:12:48.021 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:12:48.018+0000 7fdd97e00640 1 --2- 172.21.15.22:0/4048156698 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a39d8d1dd0 0x55a39d8d41a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:12:48.021 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:12:48.018+0000 7fdd97e00640 1 -- 172.21.15.22:0/4048156698 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55a39d8da460 msgr2=0x55a39d944c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T18:12:48.021 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:12:48.018+0000 7fdd97e00640 1 --2- 172.21.15.22:0/4048156698 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55a39d8da460 0x55a39d944c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:12:48.021 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:12:48.018+0000 7fdd97e00640 1 -- 172.21.15.22:0/4048156698 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55a39d9451e0 con 0x55a39d8d10f0 2024-03-28T18:12:48.021 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:12:48.018+0000 7fdd97e00640 1 --2- 172.21.15.22:0/4048156698 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a39d8d10f0 0x55a39d8d3c60 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7fdd880cc630 tx=0x7fdd880a37f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=947311bacb5834df server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:12:48.021 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:12:48.018+0000 7fdd81600640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-03-28T18:12:48.022 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:12:48.018+0000 7fdd81600640 1 -- 172.21.15.22:0/4048156698 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd880b7020 con 0x55a39d8d10f0 2024-03-28T18:12:48.022 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:12:48.018+0000 7fdd81600640 1 -- 172.21.15.22:0/4048156698 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(102..102 src has 1..102) v4 ==== 9185+0+0 (secure 0 0 0) 0x7fdd880cc2b0 con 0x55a39d8d10f0 2024-03-28T18:12:48.022 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:12:48.018+0000 7fdda17ee740 1 -- 172.21.15.22:0/4048156698 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fdd8809f090 con 0x55a39d8d10f0 2024-03-28T18:12:48.022 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:12:48.018+0000 7fdd81600640 1 -- 172.21.15.22:0/4048156698 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=92) v2 ==== 24+0+0 (secure 0 0 0) 0x7fdd88020050 con 0x55a39d8d10f0 2024-03-28T18:12:48.022 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:12:48.019+0000 7fdda17ee740 10 client.16602 fetch_fsmap learned FSMap version 92 2024-03-28T18:12:48.022 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:12:48.019+0000 7fdda17ee740 1 -- 172.21.15.22:0/4048156698 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=92,monmap=4+}) v3 -- 0x55a39d946460 con 0x55a39d8d10f0 2024-03-28T18:12:48.022 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:12:48.019+0000 7fdd81600640 1 -- 172.21.15.22:0/4048156698 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7fdd8809f090 con 0x55a39d8d10f0 2024-03-28T18:12:48.022 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:12:48.019+0000 7fdd81600640 1 -- 172.21.15.22:0/4048156698 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 92) v1 ==== 7946+0+0 (secure 0 0 0) 0x7fdd880d5070 con 0x55a39d8d10f0 2024-03-28T18:12:48.022 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:12:48.019+0000 7fdda17ee740 10 client.16602 fetch_fsmap finished waiting for FSMap version 92 2024-03-28T18:12:48.022 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:12:48.019+0000 7fdda17ee740 10 client.16602 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-28T18:12:48.022 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:12:48.019+0000 7fdda17ee740 20 client.16602 populate_metadata read hostname 'smithi022' 2024-03-28T18:12:48.022 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:12:48.019+0000 7fdda17ee740 1 --2- 172.21.15.22:0/4048156698 >> [v2:172.21.15.79:6838/2441542443,v1:172.21.15.79:6839/2441542443] conn(0x55a39d946b90 0x55a39d948fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:12:48.022 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:12:48.019+0000 7fdda17ee740 4 client.16602 mds_command: new command op to 34553 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-28T18:12:48.022 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:12:48.019+0000 7fdda17ee740 1 -- 172.21.15.22:0/4048156698 --> [v2:172.21.15.79:6838/2441542443,v1:172.21.15.79:6839/2441542443] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55a39d83cae0 con 0x55a39d946b90 2024-03-28T18:12:48.023 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:12:48.020+0000 7fdd97400640 1 --2- 172.21.15.22:0/4048156698 >> [v2:172.21.15.79:6838/2441542443,v1:172.21.15.79:6839/2441542443] conn(0x55a39d946b90 0x55a39d948fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:12:48.024 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:12:48.020+0000 7fdd97400640 1 --2- 172.21.15.22:0/4048156698 >> [v2:172.21.15.79:6838/2441542443,v1:172.21.15.79:6839/2441542443] conn(0x55a39d946b90 0x55a39d948fb0 crc :-1 s=READY pgs=163 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7bbef0187a0fb7c2 server_cookie=a85bcd00a4733d87 in_seq=0 out_seq=0 2024-03-28T18:12:48.024 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:12:48.020+0000 7fdd81600640 10 client.16602 ms_handle_connect on v2:172.21.15.79:6838/2441542443 2024-03-28T18:12:48.024 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:12:48.021+0000 7fdd81600640 1 -- 172.21.15.22:0/4048156698 <== mds.0 v2:172.21.15.79:6838/2441542443 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x55a39d83cae0 con 0x55a39d946b90 2024-03-28T18:12:48.024 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:12:48.021+0000 7fdd81600640 10 client.16602 handle_command_reply: tid=0 2024-03-28T18:12:48.024 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:12:48.021+0000 7fdda17ee740 1 client.16602 shutdown 2024-03-28T18:12:48.024 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:12:48.021+0000 7fdda17ee740 1 -- 172.21.15.22:0/4048156698 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a39d8d10f0 msgr2=0x55a39d8d3c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:12:48.024 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:12:48.021+0000 7fdda17ee740 1 --2- 172.21.15.22:0/4048156698 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a39d8d10f0 0x55a39d8d3c60 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7fdd880cc630 tx=0x7fdd880a37f0 comp rx=0 tx=0).stop 2024-03-28T18:12:48.024 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:12:48.021+0000 7fdda17ee740 1 -- 172.21.15.22:0/4048156698 shutdown_connections 2024-03-28T18:12:48.024 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:12:48.021+0000 7fdda17ee740 1 --2- 172.21.15.22:0/4048156698 >> [v2:172.21.15.79:6838/2441542443,v1:172.21.15.79:6839/2441542443] conn(0x55a39d946b90 0x55a39d948fb0 crc :-1 s=READY pgs=163 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:12:48.024 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:12:48.021+0000 7fdda17ee740 1 --2- 172.21.15.22:0/4048156698 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55a39d8da460 0x55a39d944c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:12:48.024 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:12:48.021+0000 7fdda17ee740 1 --2- 172.21.15.22:0/4048156698 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a39d8d1dd0 0x55a39d8d41a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:12:48.024 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:12:48.021+0000 7fdda17ee740 1 --2- 172.21.15.22:0/4048156698 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a39d8d10f0 0x55a39d8d3c60 unknown :-1 s=CLOSED pgs=722 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:12:48.024 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:12:48.021+0000 7fdd81600640 0 client.16602 ms_handle_reset on v2:172.21.15.79:6838/2441542443 2024-03-28T18:12:48.024 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:12:48.021+0000 7fdda17ee740 1 -- 172.21.15.22:0/4048156698 >> 172.21.15.22:0/4048156698 conn(0x55a39d898200 msgr2=0x55a39d8cabc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:12:48.024 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:12:48.021+0000 7fdda17ee740 1 -- 172.21.15.22:0/4048156698 shutdown_connections 2024-03-28T18:12:48.025 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:12:48.021+0000 7fdda17ee740 1 -- 172.21.15.22:0/4048156698 wait complete. 2024-03-28T18:12:48.025 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:12:48.022+0000 7fdda17ee740 20 client.16602 trim_cache size 0 max 16384 2024-03-28T18:12:48.025 INFO:teuthology.orchestra.run.smithi022.stdout:{ 2024-03-28T18:12:48.025 INFO:teuthology.orchestra.run.smithi022.stdout: "status": "IDLE (waiting for more scrubs)", 2024-03-28T18:12:48.025 INFO:teuthology.orchestra.run.smithi022.stdout: "scrubs": { 2024-03-28T18:12:48.025 INFO:teuthology.orchestra.run.smithi022.stdout: "4bbbd61d-29a3-4bf6-a4f2-d5783d585de3": { 2024-03-28T18:12:48.025 INFO:teuthology.orchestra.run.smithi022.stdout: "path": "/", 2024-03-28T18:12:48.025 INFO:teuthology.orchestra.run.smithi022.stdout: "tag": "4bbbd61d-29a3-4bf6-a4f2-d5783d585de3", 2024-03-28T18:12:48.025 INFO:teuthology.orchestra.run.smithi022.stdout: "options": "recursive,force" 2024-03-28T18:12:48.025 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-03-28T18:12:48.025 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-03-28T18:12:48.025 INFO:teuthology.orchestra.run.smithi022.stdout:} 2024-03-28T18:12:48.026 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:12:48.022+0000 7fdd9d400640 1 -- 172.21.15.22:0/1589273660 >> [v2:172.21.15.22:6800/1357365082,v1:172.21.15.22:6801/1357365082] conn(0x7fdd7c0a2970 msgr2=0x7fdd7c0a4e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:12:48.026 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:12:48.022+0000 7fdd9d400640 1 --2- 172.21.15.22:0/1589273660 >> [v2:172.21.15.22:6800/1357365082,v1:172.21.15.22:6801/1357365082] conn(0x7fdd7c0a2970 0x7fdd7c0a4e30 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fdd90009430 tx=0x7fdd90009360 comp rx=0 tx=0).stop 2024-03-28T18:12:48.026 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:12:48.022+0000 7fdd9d400640 1 -- 172.21.15.22:0/1589273660 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fdd98101530 msgr2=0x7fdd98199e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:12:48.026 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:12:48.022+0000 7fdd9d400640 1 --2- 172.21.15.22:0/1589273660 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fdd98101530 0x7fdd98199e60 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7fdd8800d770 tx=0x7fdd8800db70 comp rx=0 tx=0).stop 2024-03-28T18:12:48.027 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:12:48.022+0000 7fdd9d400640 1 -- 172.21.15.22:0/1589273660 shutdown_connections 2024-03-28T18:12:48.027 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:12:48.022+0000 7fdd9d400640 1 --2- 172.21.15.22:0/1589273660 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdd981075f0 0x7fdd9819e720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:12:48.027 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:12:48.022+0000 7fdd9d400640 1 --2- 172.21.15.22:0/1589273660 >> [v2:172.21.15.22:6800/1357365082,v1:172.21.15.22:6801/1357365082] conn(0x7fdd7c0a2970 0x7fdd7c0a4e30 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:12:48.027 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:12:48.022+0000 7fdd9d400640 1 --2- 172.21.15.22:0/1589273660 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fdd98101ef0 0x7fdd9819a3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:12:48.027 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:12:48.022+0000 7fdd9d400640 1 --2- 172.21.15.22:0/1589273660 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fdd98101530 0x7fdd98199e60 unknown :-1 s=CLOSED pgs=718 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:12:48.027 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:12:48.022+0000 7fdd9d400640 1 -- 172.21.15.22:0/1589273660 >> 172.21.15.22:0/1589273660 conn(0x7fdd980fd000 msgr2=0x7fdd980fe480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:12:48.027 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:12:48.023+0000 7fdd9d400640 1 -- 172.21.15.22:0/1589273660 shutdown_connections 2024-03-28T18:12:48.027 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:12:48.023+0000 7fdd9d400640 1 -- 172.21.15.22:0/1589273660 wait complete. 2024-03-28T18:12:48.040 INFO:tasks.cephfs.filesystem:scrub status for tag:4bbbd61d-29a3-4bf6-a4f2-d5783d585de3 - {'path': '/', 'tag': '4bbbd61d-29a3-4bf6-a4f2-d5783d585de3', 'options': 'recursive,force'} 2024-03-28T18:12:48.674 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:12:48 smithi134 ceph-mon[35037]: pgmap v169: 97 pgs: 97 active+clean; 117 MiB data, 785 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 204 B/s wr, 10 op/s 2024-03-28T18:12:48.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:12:48 smithi079 ceph-mon[35312]: pgmap v169: 97 pgs: 97 active+clean; 117 MiB data, 785 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 204 B/s wr, 10 op/s 2024-03-28T18:12:48.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:12:48 smithi022 ceph-mon[31985]: pgmap v169: 97 pgs: 97 active+clean; 117 MiB data, 785 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 204 B/s wr, 10 op/s 2024-03-28T18:12:50.674 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:12:50 smithi134 ceph-mon[35037]: pgmap v170: 97 pgs: 97 active+clean; 117 MiB data, 785 MiB used, 1.0 TiB / 1.0 TiB avail; 109 KiB/s rd, 4.4 KiB/s wr, 26 op/s 2024-03-28T18:12:50.674 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:12:50 smithi134 ceph-mon[35037]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-28T18:12:50.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:12:50 smithi134 ceph-mon[35037]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T18:12:50.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:12:50 smithi079 ceph-mon[35312]: pgmap v170: 97 pgs: 97 active+clean; 117 MiB data, 785 MiB used, 1.0 TiB / 1.0 TiB avail; 109 KiB/s rd, 4.4 KiB/s wr, 26 op/s 2024-03-28T18:12:50.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:12:50 smithi079 ceph-mon[35312]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-28T18:12:50.683 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:12:50 smithi079 ceph-mon[35312]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T18:12:50.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:12:50 smithi022 ceph-mon[31985]: pgmap v170: 97 pgs: 97 active+clean; 117 MiB data, 785 MiB used, 1.0 TiB / 1.0 TiB avail; 109 KiB/s rd, 4.4 KiB/s wr, 26 op/s 2024-03-28T18:12:50.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:12:50 smithi022 ceph-mon[31985]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-28T18:12:50.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:12:50 smithi022 ceph-mon[31985]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T18:12:52.674 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:12:52 smithi134 ceph-mon[35037]: pgmap v171: 97 pgs: 97 active+clean; 117 MiB data, 785 MiB used, 1.0 TiB / 1.0 TiB avail; 109 KiB/s rd, 4.4 KiB/s wr, 26 op/s 2024-03-28T18:12:52.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:12:52 smithi134 ceph-mon[35037]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T18:12:52.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:12:52 smithi134 ceph-mon[35037]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T18:12:52.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:12:52 smithi134 ceph-mon[35037]: from='mgr.15522 ' entity='mgr.x' 2024-03-28T18:12:52.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:12:52 smithi079 ceph-mon[35312]: pgmap v171: 97 pgs: 97 active+clean; 117 MiB data, 785 MiB used, 1.0 TiB / 1.0 TiB avail; 109 KiB/s rd, 4.4 KiB/s wr, 26 op/s 2024-03-28T18:12:52.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:12:52 smithi079 ceph-mon[35312]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T18:12:52.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:12:52 smithi079 ceph-mon[35312]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T18:12:52.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:12:52 smithi079 ceph-mon[35312]: from='mgr.15522 ' entity='mgr.x' 2024-03-28T18:12:52.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:12:52 smithi022 ceph-mon[31985]: pgmap v171: 97 pgs: 97 active+clean; 117 MiB data, 785 MiB used, 1.0 TiB / 1.0 TiB avail; 109 KiB/s rd, 4.4 KiB/s wr, 26 op/s 2024-03-28T18:12:52.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:12:52 smithi022 ceph-mon[31985]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T18:12:52.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:12:52 smithi022 ceph-mon[31985]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T18:12:52.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:12:52 smithi022 ceph-mon[31985]: from='mgr.15522 ' entity='mgr.x' 2024-03-28T18:12:54.674 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:12:54 smithi134 ceph-mon[35037]: pgmap v172: 97 pgs: 97 active+clean; 117 MiB data, 785 MiB used, 1.0 TiB / 1.0 TiB avail; 92 KiB/s rd, 3.7 KiB/s wr, 22 op/s 2024-03-28T18:12:54.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:12:54 smithi079 ceph-mon[35312]: pgmap v172: 97 pgs: 97 active+clean; 117 MiB data, 785 MiB used, 1.0 TiB / 1.0 TiB avail; 92 KiB/s rd, 3.7 KiB/s wr, 22 op/s 2024-03-28T18:12:54.781 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:12:54 smithi022 ceph-mon[31985]: pgmap v172: 97 pgs: 97 active+clean; 117 MiB data, 785 MiB used, 1.0 TiB / 1.0 TiB avail; 92 KiB/s rd, 3.7 KiB/s wr, 22 op/s 2024-03-28T18:12:56.585 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:12:56 smithi022 ceph-mon[31985]: pgmap v173: 97 pgs: 97 active+clean; 117 MiB data, 785 MiB used, 1.0 TiB / 1.0 TiB avail; 93 KiB/s rd, 3.7 KiB/s wr, 26 op/s 2024-03-28T18:12:56.674 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:12:56 smithi134 ceph-mon[35037]: pgmap v173: 97 pgs: 97 active+clean; 117 MiB data, 785 MiB used, 1.0 TiB / 1.0 TiB avail; 93 KiB/s rd, 3.7 KiB/s wr, 26 op/s 2024-03-28T18:12:56.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:12:56 smithi079 ceph-mon[35312]: pgmap v173: 97 pgs: 97 active+clean; 117 MiB data, 785 MiB used, 1.0 TiB / 1.0 TiB avail; 93 KiB/s rd, 3.7 KiB/s wr, 26 op/s 2024-03-28T18:12:57.316 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:12:56 smithi134 systemd[1]: ceph-fd220748-ed2b-11ee-95d0-87774f69a715@mds.f.service: Scheduled restart job, restart counter is at 5. 2024-03-28T18:12:57.316 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:12:56 smithi134 systemd[1]: Stopped Ceph mds.f for fd220748-ed2b-11ee-95d0-87774f69a715. 2024-03-28T18:12:57.316 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:12:56 smithi134 systemd[1]: ceph-fd220748-ed2b-11ee-95d0-87774f69a715@mds.f.service: Consumed 2.022s CPU time. 2024-03-28T18:12:57.316 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:12:56 smithi134 systemd[1]: Starting Ceph mds.f for fd220748-ed2b-11ee-95d0-87774f69a715... 2024-03-28T18:12:57.674 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:12:57 smithi134 podman[70764]: 2024-03-28 18:12:57.221167792 +0000 UTC m=+0.032952715 image pull c664afe56bb01db4f9afa9532c20598e201e664f2e9ba2be50db0d52674a26d2 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc 2024-03-28T18:12:57.674 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:12:57 smithi134 podman[70764]: 2024-03-28 18:12:57.354321067 +0000 UTC m=+0.166106036 container create 647fd1aee601c2a626810f4519c48ece29371a7bbd06a71a03284376fc837cb6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240319, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , ceph=True, CEPH_POINT_RELEASE=, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.vendor=CentOS) 2024-03-28T18:12:57.924 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:12:57 smithi134 podman[70764]: 2024-03-28 18:12:57.674462367 +0000 UTC m=+0.486247326 container init 647fd1aee601c2a626810f4519c48ece29371a7bbd06a71a03284376fc837cb6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240319, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, ceph=True, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-03-28T18:12:57.924 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:12:57 smithi134 podman[70764]: 2024-03-28 18:12:57.684173613 +0000 UTC m=+0.495958546 container start 647fd1aee601c2a626810f4519c48ece29371a7bbd06a71a03284376fc837cb6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f, org.label-schema.build-date=20240319, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , ceph=True, GIT_CLEAN=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-28T18:12:57.924 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:12:57 smithi134 ceph-mds[70783]: -- 172.21.15.134:0/3347191419 <== mon.1 v2:172.21.15.134:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55839379da40 con 0x558393770800 2024-03-28T18:12:57.924 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:12:57 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[70779]: starting mds.f at 2024-03-28T18:12:57.925 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:12:57 smithi134 bash[70764]: 647fd1aee601c2a626810f4519c48ece29371a7bbd06a71a03284376fc837cb6 2024-03-28T18:12:57.925 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:12:57 smithi134 systemd[1]: Started Ceph mds.f for fd220748-ed2b-11ee-95d0-87774f69a715. 2024-03-28T18:12:58.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:12:58 smithi134 ceph-mon[35037]: pgmap v174: 97 pgs: 97 active+clean; 117 MiB data, 785 MiB used, 1.0 TiB / 1.0 TiB avail; 90 KiB/s rd, 3.5 KiB/s wr, 21 op/s 2024-03-28T18:12:58.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:12:58 smithi134 ceph-mon[35037]: daemon mds.f restarted 2024-03-28T18:12:58.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:12:58 smithi134 ceph-mon[35037]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-03-28T18:12:58.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:12:58 smithi134 ceph-mon[35037]: osdmap e103: 12 total, 12 up, 12 in 2024-03-28T18:12:58.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:12:58 smithi134 ceph-mon[35037]: mds.? [v2:172.21.15.134:6834/2867168769,v1:172.21.15.134:6835/2867168769] up:boot 2024-03-28T18:12:58.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:12:58 smithi134 ceph-mon[35037]: Standby daemon mds.h assigned to filesystem cephfs as rank 1 2024-03-28T18:12:58.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:12:58 smithi134 ceph-mon[35037]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-03-28T18:12:58.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:12:58 smithi134 ceph-mon[35037]: fsmap cephfs:4/5 {0=b=up:active,2=e=up:active,3=k=up:active,4=i=up:active} 5 up:standby-replay 3 up:standby, 1 failed 2024-03-28T18:12:58.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:12:58 smithi134 ceph-mon[35037]: fsmap cephfs:5/5 4 up:standby-replay 3 up:standby 1 up:replay 4 up:active 2024-03-28T18:12:58.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:12:58 smithi079 ceph-mon[35312]: pgmap v174: 97 pgs: 97 active+clean; 117 MiB data, 785 MiB used, 1.0 TiB / 1.0 TiB avail; 90 KiB/s rd, 3.5 KiB/s wr, 21 op/s 2024-03-28T18:12:58.683 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:12:58 smithi079 ceph-mon[35312]: daemon mds.f restarted 2024-03-28T18:12:58.683 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:12:58 smithi079 ceph-mon[35312]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-03-28T18:12:58.683 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:12:58 smithi079 ceph-mon[35312]: osdmap e103: 12 total, 12 up, 12 in 2024-03-28T18:12:58.683 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:12:58 smithi079 ceph-mon[35312]: mds.? [v2:172.21.15.134:6834/2867168769,v1:172.21.15.134:6835/2867168769] up:boot 2024-03-28T18:12:58.683 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:12:58 smithi079 ceph-mon[35312]: Standby daemon mds.h assigned to filesystem cephfs as rank 1 2024-03-28T18:12:58.683 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:12:58 smithi079 ceph-mon[35312]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-03-28T18:12:58.683 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:12:58 smithi079 ceph-mon[35312]: fsmap cephfs:4/5 {0=b=up:active,2=e=up:active,3=k=up:active,4=i=up:active} 5 up:standby-replay 3 up:standby, 1 failed 2024-03-28T18:12:58.683 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:12:58 smithi079 ceph-mon[35312]: fsmap cephfs:5/5 4 up:standby-replay 3 up:standby 1 up:replay 4 up:active 2024-03-28T18:12:58.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:12:58 smithi022 ceph-mon[31985]: pgmap v174: 97 pgs: 97 active+clean; 117 MiB data, 785 MiB used, 1.0 TiB / 1.0 TiB avail; 90 KiB/s rd, 3.5 KiB/s wr, 21 op/s 2024-03-28T18:12:58.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:12:58 smithi022 ceph-mon[31985]: daemon mds.f restarted 2024-03-28T18:12:58.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:12:58 smithi022 ceph-mon[31985]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-03-28T18:12:58.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:12:58 smithi022 ceph-mon[31985]: osdmap e103: 12 total, 12 up, 12 in 2024-03-28T18:12:58.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:12:58 smithi022 ceph-mon[31985]: mds.? [v2:172.21.15.134:6834/2867168769,v1:172.21.15.134:6835/2867168769] up:boot 2024-03-28T18:12:58.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:12:58 smithi022 ceph-mon[31985]: Standby daemon mds.h assigned to filesystem cephfs as rank 1 2024-03-28T18:12:58.783 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:12:58 smithi022 ceph-mon[31985]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-03-28T18:12:58.783 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:12:58 smithi022 ceph-mon[31985]: fsmap cephfs:4/5 {0=b=up:active,2=e=up:active,3=k=up:active,4=i=up:active} 5 up:standby-replay 3 up:standby, 1 failed 2024-03-28T18:12:58.783 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:12:58 smithi022 ceph-mon[31985]: fsmap cephfs:5/5 4 up:standby-replay 3 up:standby 1 up:replay 4 up:active 2024-03-28T18:13:00.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:13:00 smithi134 ceph-mon[35037]: pgmap v176: 97 pgs: 97 active+clean; 117 MiB data, 785 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 10 op/s 2024-03-28T18:13:00.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:13:00 smithi134 ceph-mon[35037]: mds.? [v2:172.21.15.79:6836/1653690981,v1:172.21.15.79:6837/1653690981] up:resolve 2024-03-28T18:13:00.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:13:00 smithi134 ceph-mon[35037]: fsmap cephfs:5/5 4 up:standby-replay 3 up:standby 1 up:resolve 4 up:active 2024-03-28T18:13:00.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:13:00 smithi079 ceph-mon[35312]: pgmap v176: 97 pgs: 97 active+clean; 117 MiB data, 785 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 10 op/s 2024-03-28T18:13:00.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:13:00 smithi079 ceph-mon[35312]: mds.? [v2:172.21.15.79:6836/1653690981,v1:172.21.15.79:6837/1653690981] up:resolve 2024-03-28T18:13:00.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:13:00 smithi079 ceph-mon[35312]: fsmap cephfs:5/5 4 up:standby-replay 3 up:standby 1 up:resolve 4 up:active 2024-03-28T18:13:00.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:13:00 smithi022 ceph-mon[31985]: pgmap v176: 97 pgs: 97 active+clean; 117 MiB data, 785 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 10 op/s 2024-03-28T18:13:00.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:13:00 smithi022 ceph-mon[31985]: mds.? [v2:172.21.15.79:6836/1653690981,v1:172.21.15.79:6837/1653690981] up:resolve 2024-03-28T18:13:00.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:13:00 smithi022 ceph-mon[31985]: fsmap cephfs:5/5 4 up:standby-replay 3 up:standby 1 up:resolve 4 up:active 2024-03-28T18:13:01.674 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:13:01 smithi134 ceph-mon[35037]: reconnect by client.35222 172.21.15.22:0/3729116781 after 0.00399991 2024-03-28T18:13:01.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:13:01 smithi134 ceph-mon[35037]: mds.? [v2:172.21.15.79:6836/1653690981,v1:172.21.15.79:6837/1653690981] up:reconnect 2024-03-28T18:13:01.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:13:01 smithi134 ceph-mon[35037]: fsmap cephfs:5/5 4 up:standby-replay 3 up:standby 1 up:reconnect 4 up:active 2024-03-28T18:13:01.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:13:01 smithi079 ceph-mon[35312]: reconnect by client.35222 172.21.15.22:0/3729116781 after 0.00399991 2024-03-28T18:13:01.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:13:01 smithi079 ceph-mon[35312]: mds.? [v2:172.21.15.79:6836/1653690981,v1:172.21.15.79:6837/1653690981] up:reconnect 2024-03-28T18:13:01.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:13:01 smithi079 ceph-mon[35312]: fsmap cephfs:5/5 4 up:standby-replay 3 up:standby 1 up:reconnect 4 up:active 2024-03-28T18:13:01.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:13:01 smithi022 ceph-mon[31985]: reconnect by client.35222 172.21.15.22:0/3729116781 after 0.00399991 2024-03-28T18:13:01.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:13:01 smithi022 ceph-mon[31985]: mds.? [v2:172.21.15.79:6836/1653690981,v1:172.21.15.79:6837/1653690981] up:reconnect 2024-03-28T18:13:01.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:13:01 smithi022 ceph-mon[31985]: fsmap cephfs:5/5 4 up:standby-replay 3 up:standby 1 up:reconnect 4 up:active 2024-03-28T18:13:02.674 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:13:02 smithi134 ceph-mon[35037]: pgmap v177: 97 pgs: 97 active+clean; 117 MiB data, 785 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 10 op/s 2024-03-28T18:13:02.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:13:02 smithi134 ceph-mon[35037]: mds.? [v2:172.21.15.79:6836/1653690981,v1:172.21.15.79:6837/1653690981] up:rejoin 2024-03-28T18:13:02.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:13:02 smithi134 ceph-mon[35037]: fsmap cephfs:5/5 4 up:standby-replay 3 up:standby 1 up:rejoin 4 up:active 2024-03-28T18:13:02.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:13:02 smithi134 ceph-mon[35037]: daemon mds.h is now active in filesystem cephfs as rank 1 2024-03-28T18:13:02.683 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:13:02 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69878]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-2145-g3be9a5c9/rpm/el8/BUILD/ceph-19.0.0-2145-g3be9a5c9/src/mds/Locker.cc: In function 'void Locker::set_xlocks_done(MutationImpl*, bool)' thread 7f52e3400700 time 2024-03-28T18:13:02.449116+0000 2024-03-28T18:13:02.683 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:13:02 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69878]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-2145-g3be9a5c9/rpm/el8/BUILD/ceph-19.0.0-2145-g3be9a5c9/src/mds/Locker.cc: 626: FAILED ceph_assert(obj->is_auth()) 2024-03-28T18:13:02.683 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:13:02 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69878]: ceph version 19.0.0-2145-g3be9a5c9 (3be9a5c9bc793e11e2800a8c0c696e8b46742033) squid (dev) 2024-03-28T18:13:02.683 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:13:02 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69878]: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x135) [0x7f52eecf64a1] 2024-03-28T18:13:02.683 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:13:02 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69878]: 2: /usr/lib64/ceph/libceph-common.so.2(+0x2d2667) [0x7f52eecf6667] 2024-03-28T18:13:02.683 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:13:02 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69878]: 3: (Locker::set_xlocks_done(MutationImpl*, bool)+0x493) [0x559c17a5d223] 2024-03-28T18:13:02.684 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:13:02 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69878]: 4: (Server::early_reply(boost::intrusive_ptr const&, CInode*, CDentry*)+0x3e6) [0x559c178d75d6] 2024-03-28T18:13:02.684 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:13:02 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69878]: 5: (Server::journal_and_reply(boost::intrusive_ptr const&, CInode*, CDentry*, LogEvent*, MDSLogContextBase*)+0xc1) [0x559c178d7da1] 2024-03-28T18:13:02.684 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:13:02 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69878]: 6: (Server::handle_client_mkdir(boost::intrusive_ptr const&)+0x5ba) [0x559c1790d6aa] 2024-03-28T18:13:02.684 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:13:02 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69878]: 7: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0x733) [0x559c1792a233] 2024-03-28T18:13:02.684 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:13:02 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69878]: 8: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x559c1792c197] 2024-03-28T18:13:02.684 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:13:02 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69878]: 9: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x559c1782b11b] 2024-03-28T18:13:02.684 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:13:02 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69878]: 10: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x559c1782b6bc] 2024-03-28T18:13:02.684 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:13:02 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69878]: 11: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bf) [0x559c178151ff] 2024-03-28T18:13:02.684 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:13:02 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69878]: 12: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x478) [0x7f52eef88488] 2024-03-28T18:13:02.684 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:13:02 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69878]: 13: (DispatchQueue::entry()+0x51f) [0x7f52eef8560f] 2024-03-28T18:13:02.684 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:13:02 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69878]: 14: (DispatchQueue::DispatchThread::entry()+0x11) [0x7f52ef058e81] 2024-03-28T18:13:02.684 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:13:02 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69878]: 15: /lib64/libpthread.so.0(+0x81ca) [0x7f52eda731ca] 2024-03-28T18:13:02.684 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:13:02 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69878]: 16: clone() 2024-03-28T18:13:02.685 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:13:02 smithi079 ceph-mon[35312]: pgmap v177: 97 pgs: 97 active+clean; 117 MiB data, 785 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 10 op/s 2024-03-28T18:13:02.685 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:13:02 smithi079 ceph-mon[35312]: mds.? [v2:172.21.15.79:6836/1653690981,v1:172.21.15.79:6837/1653690981] up:rejoin 2024-03-28T18:13:02.685 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:13:02 smithi079 ceph-mon[35312]: fsmap cephfs:5/5 4 up:standby-replay 3 up:standby 1 up:rejoin 4 up:active 2024-03-28T18:13:02.685 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:13:02 smithi079 ceph-mon[35312]: daemon mds.h is now active in filesystem cephfs as rank 1 2024-03-28T18:13:02.685 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:13:02 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69878]: 2024-03-28T18:13:02.449+0000 7f52e3400700 -1 /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-2145-g3be9a5c9/rpm/el8/BUILD/ceph-19.0.0-2145-g3be9a5c9/src/mds/Locker.cc: In function 'void Locker::set_xlocks_done(MutationImpl*, bool)' thread 7f52e3400700 time 2024-03-28T18:13:02.449116+0000 2024-03-28T18:13:02.685 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:13:02 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69878]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-2145-g3be9a5c9/rpm/el8/BUILD/ceph-19.0.0-2145-g3be9a5c9/src/mds/Locker.cc: 626: FAILED ceph_assert(obj->is_auth()) 2024-03-28T18:13:02.686 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:13:02 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69878]: 2024-03-28T18:13:02.686 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:13:02 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69878]: ceph version 19.0.0-2145-g3be9a5c9 (3be9a5c9bc793e11e2800a8c0c696e8b46742033) squid (dev) 2024-03-28T18:13:02.686 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:13:02 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69878]: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x135) [0x7f52eecf64a1] 2024-03-28T18:13:02.686 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:13:02 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69878]: 2: /usr/lib64/ceph/libceph-common.so.2(+0x2d2667) [0x7f52eecf6667] 2024-03-28T18:13:02.686 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:13:02 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69878]: 3: (Locker::set_xlocks_done(MutationImpl*, bool)+0x493) [0x559c17a5d223] 2024-03-28T18:13:02.686 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:13:02 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69878]: 4: (Server::early_reply(boost::intrusive_ptr const&, CInode*, CDentry*)+0x3e6) [0x559c178d75d6] 2024-03-28T18:13:02.686 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:13:02 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69878]: 5: (Server::journal_and_reply(boost::intrusive_ptr const&, CInode*, CDentry*, LogEvent*, MDSLogContextBase*)+0xc1) [0x559c178d7da1] 2024-03-28T18:13:02.686 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:13:02 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69878]: 6: (Server::handle_client_mkdir(boost::intrusive_ptr const&)+0x5ba) [0x559c1790d6aa] 2024-03-28T18:13:02.686 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:13:02 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69878]: 7: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0x733) [0x559c1792a233] 2024-03-28T18:13:02.686 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:13:02 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69878]: 8: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x559c1792c197] 2024-03-28T18:13:02.686 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:13:02 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69878]: 9: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x559c1782b11b] 2024-03-28T18:13:02.686 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:13:02 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69878]: 10: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x559c1782b6bc] 2024-03-28T18:13:02.686 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:13:02 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69878]: 11: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bf) [0x559c178151ff] 2024-03-28T18:13:02.687 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:13:02 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69878]: 12: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x478) [0x7f52eef88488] 2024-03-28T18:13:02.687 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:13:02 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69878]: 13: (DispatchQueue::entry()+0x51f) [0x7f52eef8560f] 2024-03-28T18:13:02.687 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:13:02 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69878]: 14: (DispatchQueue::DispatchThread::entry()+0x11) [0x7f52ef058e81] 2024-03-28T18:13:02.687 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:13:02 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69878]: 15: /lib64/libpthread.so.0(+0x81ca) [0x7f52eda731ca] 2024-03-28T18:13:02.687 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:13:02 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69878]: 16: clone() 2024-03-28T18:13:02.687 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:13:02 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69878]: 2024-03-28T18:13:02.687 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:13:02 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69878]: *** Caught signal (Aborted) ** 2024-03-28T18:13:02.687 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:13:02 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69878]: in thread 7f52e3400700 thread_name:ms_dispatch 2024-03-28T18:13:02.687 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:13:02 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69878]: ceph version 19.0.0-2145-g3be9a5c9 (3be9a5c9bc793e11e2800a8c0c696e8b46742033) squid (dev) 2024-03-28T18:13:02.687 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:13:02 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69878]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7f52eda7dd20] 2024-03-28T18:13:02.687 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:13:02 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69878]: 2: gsignal() 2024-03-28T18:13:02.687 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:13:02 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69878]: 3: abort() 2024-03-28T18:13:02.687 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:13:02 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69878]: 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x18f) [0x7f52eecf64fb] 2024-03-28T18:13:02.687 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:13:02 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69878]: 5: /usr/lib64/ceph/libceph-common.so.2(+0x2d2667) [0x7f52eecf6667] 2024-03-28T18:13:02.688 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:13:02 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69878]: 6: (Locker::set_xlocks_done(MutationImpl*, bool)+0x493) [0x559c17a5d223] 2024-03-28T18:13:02.688 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:13:02 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69878]: 7: (Server::early_reply(boost::intrusive_ptr const&, CInode*, CDentry*)+0x3e6) [0x559c178d75d6] 2024-03-28T18:13:02.688 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:13:02 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69878]: 8: (Server::journal_and_reply(boost::intrusive_ptr const&, CInode*, CDentry*, LogEvent*, MDSLogContextBase*)+0xc1) [0x559c178d7da1] 2024-03-28T18:13:02.688 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:13:02 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69878]: 9: (Server::handle_client_mkdir(boost::intrusive_ptr const&)+0x5ba) [0x559c1790d6aa] 2024-03-28T18:13:02.688 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:13:02 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69878]: 10: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0x733) [0x559c1792a233] 2024-03-28T18:13:02.688 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:13:02 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69878]: 11: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x559c1792c197] 2024-03-28T18:13:02.688 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:13:02 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69878]: 12: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x559c1782b11b] 2024-03-28T18:13:02.688 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:13:02 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69878]: 13: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x559c1782b6bc] 2024-03-28T18:13:02.688 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:13:02 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69878]: 14: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bf) [0x559c178151ff] 2024-03-28T18:13:02.688 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:13:02 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69878]: 15: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x478) [0x7f52eef88488] 2024-03-28T18:13:02.688 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:13:02 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69878]: 16: (DispatchQueue::entry()+0x51f) [0x7f52eef8560f] 2024-03-28T18:13:02.688 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:13:02 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69878]: 17: (DispatchQueue::DispatchThread::entry()+0x11) [0x7f52ef058e81] 2024-03-28T18:13:02.688 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:13:02 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69878]: 18: /lib64/libpthread.so.0(+0x81ca) [0x7f52eda731ca] 2024-03-28T18:13:02.688 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:13:02 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69878]: 19: clone() 2024-03-28T18:13:02.688 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:13:02 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69878]: 2024-03-28T18:13:02.452+0000 7f52e3400700 -1 *** Caught signal (Aborted) ** 2024-03-28T18:13:02.688 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:13:02 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69878]: in thread 7f52e3400700 thread_name:ms_dispatch 2024-03-28T18:13:02.689 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:13:02 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69878]: 2024-03-28T18:13:02.689 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:13:02 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69878]: ceph version 19.0.0-2145-g3be9a5c9 (3be9a5c9bc793e11e2800a8c0c696e8b46742033) squid (dev) 2024-03-28T18:13:02.689 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:13:02 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69878]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7f52eda7dd20] 2024-03-28T18:13:02.689 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:13:02 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69878]: 2: gsignal() 2024-03-28T18:13:02.689 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:13:02 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69878]: 3: abort() 2024-03-28T18:13:02.689 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:13:02 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69878]: 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x18f) [0x7f52eecf64fb] 2024-03-28T18:13:02.689 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:13:02 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69878]: 5: /usr/lib64/ceph/libceph-common.so.2(+0x2d2667) [0x7f52eecf6667] 2024-03-28T18:13:02.690 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:13:02 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69878]: 6: (Locker::set_xlocks_done(MutationImpl*, bool)+0x493) [0x559c17a5d223] 2024-03-28T18:13:02.690 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:13:02 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69878]: 7: (Server::early_reply(boost::intrusive_ptr const&, CInode*, CDentry*)+0x3e6) [0x559c178d75d6] 2024-03-28T18:13:02.690 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:13:02 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69878]: 8: (Server::journal_and_reply(boost::intrusive_ptr const&, CInode*, CDentry*, LogEvent*, MDSLogContextBase*)+0xc1) [0x559c178d7da1] 2024-03-28T18:13:02.690 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:13:02 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69878]: 9: (Server::handle_client_mkdir(boost::intrusive_ptr const&)+0x5ba) [0x559c1790d6aa] 2024-03-28T18:13:02.690 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:13:02 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69878]: 10: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0x733) [0x559c1792a233] 2024-03-28T18:13:02.690 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:13:02 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69878]: 11: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x559c1792c197] 2024-03-28T18:13:02.690 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:13:02 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69878]: 12: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x559c1782b11b] 2024-03-28T18:13:02.691 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:13:02 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69878]: 13: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x559c1782b6bc] 2024-03-28T18:13:02.691 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:13:02 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69878]: 14: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bf) [0x559c178151ff] 2024-03-28T18:13:02.691 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:13:02 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69878]: 15: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x478) [0x7f52eef88488] 2024-03-28T18:13:02.691 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:13:02 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69878]: 16: (DispatchQueue::entry()+0x51f) [0x7f52eef8560f] 2024-03-28T18:13:02.691 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:13:02 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69878]: 17: (DispatchQueue::DispatchThread::entry()+0x11) [0x7f52ef058e81] 2024-03-28T18:13:02.691 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:13:02 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69878]: 18: /lib64/libpthread.so.0(+0x81ca) [0x7f52eda731ca] 2024-03-28T18:13:02.691 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:13:02 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69878]: 19: clone() 2024-03-28T18:13:02.691 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:13:02 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69878]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-03-28T18:13:02.691 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:13:02 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69878]: 2024-03-28T18:13:02.691 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:13:02 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69878]: -1> 2024-03-28T18:13:02.449+0000 7f52e3400700 -1 /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-2145-g3be9a5c9/rpm/el8/BUILD/ceph-19.0.0-2145-g3be9a5c9/src/mds/Locker.cc: In function 'void Locker::set_xlocks_done(MutationImpl*, bool)' thread 7f52e3400700 time 2024-03-28T18:13:02.449116+0000 2024-03-28T18:13:02.691 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:13:02 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69878]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-2145-g3be9a5c9/rpm/el8/BUILD/ceph-19.0.0-2145-g3be9a5c9/src/mds/Locker.cc: 626: FAILED ceph_assert(obj->is_auth()) 2024-03-28T18:13:02.691 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:13:02 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69878]: 2024-03-28T18:13:02.691 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:13:02 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69878]: ceph version 19.0.0-2145-g3be9a5c9 (3be9a5c9bc793e11e2800a8c0c696e8b46742033) squid (dev) 2024-03-28T18:13:02.691 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:13:02 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69878]: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x135) [0x7f52eecf64a1] 2024-03-28T18:13:02.691 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:13:02 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69878]: 2: /usr/lib64/ceph/libceph-common.so.2(+0x2d2667) [0x7f52eecf6667] 2024-03-28T18:13:02.692 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:13:02 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69878]: 3: (Locker::set_xlocks_done(MutationImpl*, bool)+0x493) [0x559c17a5d223] 2024-03-28T18:13:02.692 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:13:02 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69878]: 4: (Server::early_reply(boost::intrusive_ptr const&, CInode*, CDentry*)+0x3e6) [0x559c178d75d6] 2024-03-28T18:13:02.692 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:13:02 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69878]: 5: (Server::journal_and_reply(boost::intrusive_ptr const&, CInode*, CDentry*, LogEvent*, MDSLogContextBase*)+0xc1) [0x559c178d7da1] 2024-03-28T18:13:02.692 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:13:02 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69878]: 6: (Server::handle_client_mkdir(boost::intrusive_ptr const&)+0x5ba) [0x559c1790d6aa] 2024-03-28T18:13:02.692 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:13:02 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69878]: 7: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0x733) [0x559c1792a233] 2024-03-28T18:13:02.692 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:13:02 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69878]: 8: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x559c1792c197] 2024-03-28T18:13:02.692 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:13:02 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69878]: 9: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x559c1782b11b] 2024-03-28T18:13:02.692 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:13:02 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69878]: 10: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x559c1782b6bc] 2024-03-28T18:13:02.692 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:13:02 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69878]: 11: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bf) [0x559c178151ff] 2024-03-28T18:13:02.692 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:13:02 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69878]: 12: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x478) [0x7f52eef88488] 2024-03-28T18:13:02.692 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:13:02 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69878]: 13: (DispatchQueue::entry()+0x51f) [0x7f52eef8560f] 2024-03-28T18:13:02.692 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:13:02 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69878]: 14: (DispatchQueue::DispatchThread::entry()+0x11) [0x7f52ef058e81] 2024-03-28T18:13:02.692 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:13:02 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69878]: 15: /lib64/libpthread.so.0(+0x81ca) [0x7f52eda731ca] 2024-03-28T18:13:02.692 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:13:02 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69878]: 16: clone() 2024-03-28T18:13:02.692 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:13:02 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69878]: 2024-03-28T18:13:02.693 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:13:02 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69878]: 0> 2024-03-28T18:13:02.452+0000 7f52e3400700 -1 *** Caught signal (Aborted) ** 2024-03-28T18:13:02.693 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:13:02 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69878]: in thread 7f52e3400700 thread_name:ms_dispatch 2024-03-28T18:13:02.693 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:13:02 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69878]: 2024-03-28T18:13:02.693 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:13:02 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69878]: ceph version 19.0.0-2145-g3be9a5c9 (3be9a5c9bc793e11e2800a8c0c696e8b46742033) squid (dev) 2024-03-28T18:13:02.693 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:13:02 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69878]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7f52eda7dd20] 2024-03-28T18:13:02.693 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:13:02 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69878]: 2: gsignal() 2024-03-28T18:13:02.693 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:13:02 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69878]: 3: abort() 2024-03-28T18:13:02.693 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:13:02 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69878]: 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x18f) [0x7f52eecf64fb] 2024-03-28T18:13:02.693 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:13:02 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69878]: 5: /usr/lib64/ceph/libceph-common.so.2(+0x2d2667) [0x7f52eecf6667] 2024-03-28T18:13:02.693 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:13:02 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69878]: 6: (Locker::set_xlocks_done(MutationImpl*, bool)+0x493) [0x559c17a5d223] 2024-03-28T18:13:02.693 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:13:02 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69878]: 7: (Server::early_reply(boost::intrusive_ptr const&, CInode*, CDentry*)+0x3e6) [0x559c178d75d6] 2024-03-28T18:13:02.693 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:13:02 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69878]: 8: (Server::journal_and_reply(boost::intrusive_ptr const&, CInode*, CDentry*, LogEvent*, MDSLogContextBase*)+0xc1) [0x559c178d7da1] 2024-03-28T18:13:02.693 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:13:02 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69878]: 9: (Server::handle_client_mkdir(boost::intrusive_ptr const&)+0x5ba) [0x559c1790d6aa] 2024-03-28T18:13:02.694 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:13:02 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69878]: 10: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0x733) [0x559c1792a233] 2024-03-28T18:13:02.694 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:13:02 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69878]: 11: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x559c1792c197] 2024-03-28T18:13:02.694 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:13:02 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69878]: 12: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x559c1782b11b] 2024-03-28T18:13:02.694 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:13:02 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69878]: 13: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x559c1782b6bc] 2024-03-28T18:13:02.694 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:13:02 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69878]: 14: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bf) [0x559c178151ff] 2024-03-28T18:13:02.694 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:13:02 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69878]: 15: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x478) [0x7f52eef88488] 2024-03-28T18:13:02.694 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:13:02 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69878]: 16: (DispatchQueue::entry()+0x51f) [0x7f52eef8560f] 2024-03-28T18:13:02.694 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:13:02 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69878]: 17: (DispatchQueue::DispatchThread::entry()+0x11) [0x7f52ef058e81] 2024-03-28T18:13:02.694 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:13:02 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69878]: 18: /lib64/libpthread.so.0(+0x81ca) [0x7f52eda731ca] 2024-03-28T18:13:02.694 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:13:02 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69878]: 19: clone() 2024-03-28T18:13:02.694 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:13:02 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69878]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-03-28T18:13:02.694 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:13:02 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69878]: 2024-03-28T18:13:02.694 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:13:02 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69878]: -1> 2024-03-28T18:13:02.449+0000 7f52e3400700 -1 /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-2145-g3be9a5c9/rpm/el8/BUILD/ceph-19.0.0-2145-g3be9a5c9/src/mds/Locker.cc: In function 'void Locker::set_xlocks_done(MutationImpl*, bool)' thread 7f52e3400700 time 2024-03-28T18:13:02.449116+0000 2024-03-28T18:13:02.694 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:13:02 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69878]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-2145-g3be9a5c9/rpm/el8/BUILD/ceph-19.0.0-2145-g3be9a5c9/src/mds/Locker.cc: 626: FAILED ceph_assert(obj->is_auth()) 2024-03-28T18:13:02.695 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:13:02 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69878]: 2024-03-28T18:13:02.695 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:13:02 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69878]: ceph version 19.0.0-2145-g3be9a5c9 (3be9a5c9bc793e11e2800a8c0c696e8b46742033) squid (dev) 2024-03-28T18:13:02.695 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:13:02 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69878]: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x135) [0x7f52eecf64a1] 2024-03-28T18:13:02.695 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:13:02 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69878]: 2: /usr/lib64/ceph/libceph-common.so.2(+0x2d2667) [0x7f52eecf6667] 2024-03-28T18:13:02.695 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:13:02 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69878]: 3: (Locker::set_xlocks_done(MutationImpl*, bool)+0x493) [0x559c17a5d223] 2024-03-28T18:13:02.695 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:13:02 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69878]: 4: (Server::early_reply(boost::intrusive_ptr const&, CInode*, CDentry*)+0x3e6) [0x559c178d75d6] 2024-03-28T18:13:02.695 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:13:02 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69878]: 5: (Server::journal_and_reply(boost::intrusive_ptr const&, CInode*, CDentry*, LogEvent*, MDSLogContextBase*)+0xc1) [0x559c178d7da1] 2024-03-28T18:13:02.695 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:13:02 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69878]: 6: (Server::handle_client_mkdir(boost::intrusive_ptr const&)+0x5ba) [0x559c1790d6aa] 2024-03-28T18:13:02.695 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:13:02 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69878]: 7: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0x733) [0x559c1792a233] 2024-03-28T18:13:02.695 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:13:02 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69878]: 8: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x559c1792c197] 2024-03-28T18:13:02.695 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:13:02 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69878]: 9: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x559c1782b11b] 2024-03-28T18:13:02.695 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:13:02 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69878]: 10: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x559c1782b6bc] 2024-03-28T18:13:02.695 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:13:02 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69878]: 11: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bf) [0x559c178151ff] 2024-03-28T18:13:02.695 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:13:02 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69878]: 12: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x478) [0x7f52eef88488] 2024-03-28T18:13:02.695 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:13:02 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69878]: 13: (DispatchQueue::entry()+0x51f) [0x7f52eef8560f] 2024-03-28T18:13:02.696 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:13:02 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69878]: 14: (DispatchQueue::DispatchThread::entry()+0x11) [0x7f52ef058e81] 2024-03-28T18:13:02.696 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:13:02 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69878]: 15: /lib64/libpthread.so.0(+0x81ca) [0x7f52eda731ca] 2024-03-28T18:13:02.696 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:13:02 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69878]: 16: clone() 2024-03-28T18:13:02.696 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:13:02 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69878]: 2024-03-28T18:13:02.696 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:13:02 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69878]: 0> 2024-03-28T18:13:02.452+0000 7f52e3400700 -1 *** Caught signal (Aborted) ** 2024-03-28T18:13:02.696 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:13:02 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69878]: in thread 7f52e3400700 thread_name:ms_dispatch 2024-03-28T18:13:02.696 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:13:02 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69878]: 2024-03-28T18:13:02.696 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:13:02 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69878]: ceph version 19.0.0-2145-g3be9a5c9 (3be9a5c9bc793e11e2800a8c0c696e8b46742033) squid (dev) 2024-03-28T18:13:02.696 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:13:02 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69878]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7f52eda7dd20] 2024-03-28T18:13:02.696 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:13:02 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69878]: 2: gsignal() 2024-03-28T18:13:02.696 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:13:02 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69878]: 3: abort() 2024-03-28T18:13:02.696 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:13:02 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69878]: 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x18f) [0x7f52eecf64fb] 2024-03-28T18:13:02.696 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:13:02 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69878]: 5: /usr/lib64/ceph/libceph-common.so.2(+0x2d2667) [0x7f52eecf6667] 2024-03-28T18:13:02.696 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:13:02 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69878]: 6: (Locker::set_xlocks_done(MutationImpl*, bool)+0x493) [0x559c17a5d223] 2024-03-28T18:13:02.696 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:13:02 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69878]: 7: (Server::early_reply(boost::intrusive_ptr const&, CInode*, CDentry*)+0x3e6) [0x559c178d75d6] 2024-03-28T18:13:02.697 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:13:02 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69878]: 8: (Server::journal_and_reply(boost::intrusive_ptr const&, CInode*, CDentry*, LogEvent*, MDSLogContextBase*)+0xc1) [0x559c178d7da1] 2024-03-28T18:13:02.697 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:13:02 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69878]: 9: (Server::handle_client_mkdir(boost::intrusive_ptr const&)+0x5ba) [0x559c1790d6aa] 2024-03-28T18:13:02.697 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:13:02 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69878]: 10: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0x733) [0x559c1792a233] 2024-03-28T18:13:02.697 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:13:02 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69878]: 11: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x559c1792c197] 2024-03-28T18:13:02.697 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:13:02 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69878]: 12: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x559c1782b11b] 2024-03-28T18:13:02.697 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:13:02 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69878]: 13: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x559c1782b6bc] 2024-03-28T18:13:02.697 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:13:02 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69878]: 14: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bf) [0x559c178151ff] 2024-03-28T18:13:02.697 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:13:02 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69878]: 15: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x478) [0x7f52eef88488] 2024-03-28T18:13:02.697 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:13:02 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69878]: 16: (DispatchQueue::entry()+0x51f) [0x7f52eef8560f] 2024-03-28T18:13:02.697 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:13:02 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69878]: 17: (DispatchQueue::DispatchThread::entry()+0x11) [0x7f52ef058e81] 2024-03-28T18:13:02.697 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:13:02 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69878]: 18: /lib64/libpthread.so.0(+0x81ca) [0x7f52eda731ca] 2024-03-28T18:13:02.697 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:13:02 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69878]: 19: clone() 2024-03-28T18:13:02.697 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:13:02 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69878]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-03-28T18:13:02.697 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:13:02 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h[69878]: 2024-03-28T18:13:02.697 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:13:02 smithi079 podman[70044]: 2024-03-28 18:13:02.581042329 +0000 UTC m=+0.039358215 container died becb7c966c54908099c6e9524f3fb51c4b51d6d91f38d13c856e803b956507c8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.build-date=20240319, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_CLEAN=True, GIT_BRANCH=HEAD, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True) 2024-03-28T18:13:02.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:13:02 smithi022 ceph-mon[31985]: pgmap v177: 97 pgs: 97 active+clean; 117 MiB data, 785 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 10 op/s 2024-03-28T18:13:02.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:13:02 smithi022 ceph-mon[31985]: mds.? [v2:172.21.15.79:6836/1653690981,v1:172.21.15.79:6837/1653690981] up:rejoin 2024-03-28T18:13:02.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:13:02 smithi022 ceph-mon[31985]: fsmap cephfs:5/5 4 up:standby-replay 3 up:standby 1 up:rejoin 4 up:active 2024-03-28T18:13:02.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:13:02 smithi022 ceph-mon[31985]: daemon mds.h is now active in filesystem cephfs as rank 1 2024-03-28T18:13:03.672 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:13:03 smithi079 podman[70044]: 2024-03-28 18:13:03.254138523 +0000 UTC m=+0.712454422 container remove becb7c966c54908099c6e9524f3fb51c4b51d6d91f38d13c856e803b956507c8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-h, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.build-date=20240319, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, ceph=True, GIT_CLEAN=True, io.buildah.version=1.33.5) 2024-03-28T18:13:03.672 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:13:03 smithi079 systemd[1]: ceph-fd220748-ed2b-11ee-95d0-87774f69a715@mds.h.service: Main process exited, code=exited, status=134/n/a 2024-03-28T18:13:03.672 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:13:03 smithi079 ceph-mon[35312]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-03-28T18:13:03.672 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:13:03 smithi079 ceph-mon[35312]: Cluster is now healthy 2024-03-28T18:13:03.672 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:13:03 smithi079 ceph-mon[35312]: mds.? [v2:172.21.15.79:6836/1653690981,v1:172.21.15.79:6837/1653690981] up:active 2024-03-28T18:13:03.672 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:13:03 smithi079 ceph-mon[35312]: fsmap cephfs:5 4 up:standby-replay 3 up:standby 5 up:active 2024-03-28T18:13:03.672 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:13:03 smithi079 ceph-mon[35312]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-28T18:13:03.674 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:13:03 smithi134 ceph-mon[35037]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-03-28T18:13:03.674 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:13:03 smithi134 ceph-mon[35037]: Cluster is now healthy 2024-03-28T18:13:03.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:13:03 smithi134 ceph-mon[35037]: mds.? [v2:172.21.15.79:6836/1653690981,v1:172.21.15.79:6837/1653690981] up:active 2024-03-28T18:13:03.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:13:03 smithi134 ceph-mon[35037]: fsmap cephfs:5 4 up:standby-replay 3 up:standby 5 up:active 2024-03-28T18:13:03.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:13:03 smithi134 ceph-mon[35037]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-28T18:13:03.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:13:03 smithi022 ceph-mon[31985]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-03-28T18:13:03.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:13:03 smithi022 ceph-mon[31985]: Cluster is now healthy 2024-03-28T18:13:03.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:13:03 smithi022 ceph-mon[31985]: mds.? [v2:172.21.15.79:6836/1653690981,v1:172.21.15.79:6837/1653690981] up:active 2024-03-28T18:13:03.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:13:03 smithi022 ceph-mon[31985]: fsmap cephfs:5 4 up:standby-replay 3 up:standby 5 up:active 2024-03-28T18:13:03.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:13:03 smithi022 ceph-mon[31985]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-28T18:13:04.089 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:13:03 smithi079 systemd[1]: ceph-fd220748-ed2b-11ee-95d0-87774f69a715@mds.h.service: Failed with result 'exit-code'. 2024-03-28T18:13:04.089 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:13:03 smithi079 systemd[1]: ceph-fd220748-ed2b-11ee-95d0-87774f69a715@mds.h.service: Consumed 1.637s CPU time. 2024-03-28T18:13:04.408 INFO:journalctl@ceph.mds.b.smithi079.stdout:Mar 28 18:13:04 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-b[65245]: 2024-03-28T18:13:04.088+0000 7f845f400700 -1 mds.pinger is_rank_lagging: rank=1 was never sent ping request. 2024-03-28T18:13:04.674 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:13:04 smithi134 ceph-mon[35037]: pgmap v178: 97 pgs: 97 active+clean; 117 MiB data, 785 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 10 op/s 2024-03-28T18:13:04.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:13:04 smithi079 ceph-mon[35312]: pgmap v178: 97 pgs: 97 active+clean; 117 MiB data, 785 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 10 op/s 2024-03-28T18:13:04.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:13:04 smithi022 ceph-mon[31985]: pgmap v178: 97 pgs: 97 active+clean; 117 MiB data, 785 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 10 op/s 2024-03-28T18:13:06.674 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:13:06 smithi134 ceph-mon[35037]: pgmap v179: 97 pgs: 97 active+clean; 117 MiB data, 786 MiB used, 1.0 TiB / 1.0 TiB avail; 120 KiB/s rd, 31 KiB/s wr, 43 op/s 2024-03-28T18:13:06.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:13:06 smithi079 ceph-mon[35312]: pgmap v179: 97 pgs: 97 active+clean; 117 MiB data, 786 MiB used, 1.0 TiB / 1.0 TiB avail; 120 KiB/s rd, 31 KiB/s wr, 43 op/s 2024-03-28T18:13:06.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:13:06 smithi022 ceph-mon[31985]: pgmap v179: 97 pgs: 97 active+clean; 117 MiB data, 786 MiB used, 1.0 TiB / 1.0 TiB avail; 120 KiB/s rd, 31 KiB/s wr, 43 op/s 2024-03-28T18:13:08.674 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:13:08 smithi134 ceph-mon[35037]: pgmap v180: 97 pgs: 97 active+clean; 117 MiB data, 786 MiB used, 1.0 TiB / 1.0 TiB avail; 120 KiB/s rd, 31 KiB/s wr, 43 op/s 2024-03-28T18:13:08.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:13:08 smithi079 ceph-mon[35312]: pgmap v180: 97 pgs: 97 active+clean; 117 MiB data, 786 MiB used, 1.0 TiB / 1.0 TiB avail; 120 KiB/s rd, 31 KiB/s wr, 43 op/s 2024-03-28T18:13:08.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:13:08 smithi022 ceph-mon[31985]: pgmap v180: 97 pgs: 97 active+clean; 117 MiB data, 786 MiB used, 1.0 TiB / 1.0 TiB avail; 120 KiB/s rd, 31 KiB/s wr, 43 op/s 2024-03-28T18:13:10.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:13:10 smithi022 ceph-mon[31985]: pgmap v181: 97 pgs: 97 active+clean; 117 MiB data, 786 MiB used, 1.0 TiB / 1.0 TiB avail; 108 KiB/s rd, 27 KiB/s wr, 43 op/s 2024-03-28T18:13:10.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:13:10 smithi022 ceph-mon[31985]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-28T18:13:10.925 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:13:10 smithi134 ceph-mon[35037]: pgmap v181: 97 pgs: 97 active+clean; 117 MiB data, 786 MiB used, 1.0 TiB / 1.0 TiB avail; 108 KiB/s rd, 27 KiB/s wr, 43 op/s 2024-03-28T18:13:10.925 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:13:10 smithi134 ceph-mon[35037]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-28T18:13:10.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:13:10 smithi079 ceph-mon[35312]: pgmap v181: 97 pgs: 97 active+clean; 117 MiB data, 786 MiB used, 1.0 TiB / 1.0 TiB avail; 108 KiB/s rd, 27 KiB/s wr, 43 op/s 2024-03-28T18:13:10.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:13:10 smithi079 ceph-mon[35312]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-28T18:13:12.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:13:12 smithi022 ceph-mon[31985]: pgmap v182: 97 pgs: 97 active+clean; 117 MiB data, 786 MiB used, 1.0 TiB / 1.0 TiB avail; 99 KiB/s rd, 26 KiB/s wr, 36 op/s 2024-03-28T18:13:12.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:13:12 smithi134 ceph-mon[35037]: pgmap v182: 97 pgs: 97 active+clean; 117 MiB data, 786 MiB used, 1.0 TiB / 1.0 TiB avail; 99 KiB/s rd, 26 KiB/s wr, 36 op/s 2024-03-28T18:13:12.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:13:12 smithi079 ceph-mon[35312]: pgmap v182: 97 pgs: 97 active+clean; 117 MiB data, 786 MiB used, 1.0 TiB / 1.0 TiB avail; 99 KiB/s rd, 26 KiB/s wr, 36 op/s 2024-03-28T18:13:14.433 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:13:13 smithi079 systemd[1]: ceph-fd220748-ed2b-11ee-95d0-87774f69a715@mds.h.service: Scheduled restart job, restart counter is at 6. 2024-03-28T18:13:14.433 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:13:13 smithi079 systemd[1]: Stopped Ceph mds.h for fd220748-ed2b-11ee-95d0-87774f69a715. 2024-03-28T18:13:14.433 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:13:13 smithi079 systemd[1]: ceph-fd220748-ed2b-11ee-95d0-87774f69a715@mds.h.service: Consumed 1.637s CPU time. 2024-03-28T18:13:14.433 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:13:13 smithi079 systemd[1]: ceph-fd220748-ed2b-11ee-95d0-87774f69a715@mds.h.service: Start request repeated too quickly. 2024-03-28T18:13:14.433 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:13:13 smithi079 systemd[1]: ceph-fd220748-ed2b-11ee-95d0-87774f69a715@mds.h.service: Failed with result 'exit-code'. 2024-03-28T18:13:14.433 INFO:journalctl@ceph.mds.h.smithi079.stdout:Mar 28 18:13:13 smithi079 systemd[1]: Failed to start Ceph mds.h for fd220748-ed2b-11ee-95d0-87774f69a715. 2024-03-28T18:13:14.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:13:14 smithi022 ceph-mon[31985]: pgmap v183: 97 pgs: 97 active+clean; 117 MiB data, 786 MiB used, 1.0 TiB / 1.0 TiB avail; 99 KiB/s rd, 26 KiB/s wr, 36 op/s 2024-03-28T18:13:14.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:13:14 smithi134 ceph-mon[35037]: pgmap v183: 97 pgs: 97 active+clean; 117 MiB data, 786 MiB used, 1.0 TiB / 1.0 TiB avail; 99 KiB/s rd, 26 KiB/s wr, 36 op/s 2024-03-28T18:13:14.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:13:14 smithi079 ceph-mon[35312]: pgmap v183: 97 pgs: 97 active+clean; 117 MiB data, 786 MiB used, 1.0 TiB / 1.0 TiB avail; 99 KiB/s rd, 26 KiB/s wr, 36 op/s 2024-03-28T18:13:16.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:13:16 smithi022 ceph-mon[31985]: pgmap v184: 97 pgs: 97 active+clean; 117 MiB data, 786 MiB used, 1.0 TiB / 1.0 TiB avail; 101 KiB/s rd, 26 KiB/s wr, 40 op/s 2024-03-28T18:13:16.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:13:16 smithi134 ceph-mon[35037]: pgmap v184: 97 pgs: 97 active+clean; 117 MiB data, 786 MiB used, 1.0 TiB / 1.0 TiB avail; 101 KiB/s rd, 26 KiB/s wr, 40 op/s 2024-03-28T18:13:16.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:13:16 smithi079 ceph-mon[35312]: pgmap v184: 97 pgs: 97 active+clean; 117 MiB data, 786 MiB used, 1.0 TiB / 1.0 TiB avail; 101 KiB/s rd, 26 KiB/s wr, 40 op/s 2024-03-28T18:13:18.041 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-28T18:13:18.041 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-28T18:13:18.271 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:13:18.271+0000 7fd1afe00640 1 -- 172.21.15.22:0/282621798 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd1b01064a0 msgr2=0x7fd1b0106880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:13:18.271 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:13:18.271+0000 7fd1afe00640 1 --2- 172.21.15.22:0/282621798 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd1b01064a0 0x7fd1b0106880 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fd1a0009900 tx=0x7fd1a002bc50 comp rx=0 tx=0).stop 2024-03-28T18:13:18.271 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:13:18.271+0000 7fd1afe00640 1 -- 172.21.15.22:0/282621798 shutdown_connections 2024-03-28T18:13:18.272 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:13:18.271+0000 7fd1afe00640 1 --2- 172.21.15.22:0/282621798 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd1b0100dc0 0x7fd1b010d2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:13:18.272 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:13:18.271+0000 7fd1afe00640 1 --2- 172.21.15.22:0/282621798 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd1b0100400 0x7fd1b0100880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:13:18.272 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:13:18.271+0000 7fd1afe00640 1 --2- 172.21.15.22:0/282621798 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd1b01064a0 0x7fd1b0106880 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:13:18.272 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:13:18.271+0000 7fd1afe00640 1 -- 172.21.15.22:0/282621798 >> 172.21.15.22:0/282621798 conn(0x7fd1b00fbef0 msgr2=0x7fd1b00fe310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:13:18.272 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:13:18.271+0000 7fd1afe00640 1 -- 172.21.15.22:0/282621798 shutdown_connections 2024-03-28T18:13:18.272 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:13:18.271+0000 7fd1afe00640 1 -- 172.21.15.22:0/282621798 wait complete. 2024-03-28T18:13:18.273 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:13:18.272+0000 7fd1afe00640 1 Processor -- start 2024-03-28T18:13:18.274 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:13:18.273+0000 7fd1afe00640 1 -- start start 2024-03-28T18:13:18.274 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:13:18.273+0000 7fd1afe00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd1b0100400 0x7fd1b019a030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:13:18.274 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:13:18.273+0000 7fd1afe00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd1b0100dc0 0x7fd1b019a570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:13:18.274 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:13:18.273+0000 7fd1afe00640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd1b01064a0 0x7fd1b019e8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:13:18.274 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:13:18.273+0000 7fd1afe00640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1b0110810 con 0x7fd1b0100400 2024-03-28T18:13:18.274 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:13:18.273+0000 7fd1afe00640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1b0110690 con 0x7fd1b01064a0 2024-03-28T18:13:18.274 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:13:18.273+0000 7fd1afe00640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1b0110990 con 0x7fd1b0100dc0 2024-03-28T18:13:18.274 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:13:18.273+0000 7fd1aea00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd1b0100400 0x7fd1b019a030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:13:18.274 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:13:18.274+0000 7fd1af400640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd1b01064a0 0x7fd1b019e8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:13:18.274 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:13:18.274+0000 7fd1aea00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd1b0100400 0x7fd1b019a030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:37918/0 (socket says 172.21.15.22:37918) 2024-03-28T18:13:18.274 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:13:18.274+0000 7fd1aea00640 1 -- 172.21.15.22:0/4255290993 learned_addr learned my addr 172.21.15.22:0/4255290993 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-28T18:13:18.274 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:13:18.274+0000 7fd1ae000640 1 --2- 172.21.15.22:0/4255290993 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd1b0100dc0 0x7fd1b019a570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:13:18.275 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:13:18.274+0000 7fd1aea00640 1 -- 172.21.15.22:0/4255290993 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd1b0100dc0 msgr2=0x7fd1b019a570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:13:18.275 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:13:18.274+0000 7fd1aea00640 1 --2- 172.21.15.22:0/4255290993 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd1b0100dc0 0x7fd1b019a570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:13:18.275 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:13:18.274+0000 7fd1aea00640 1 -- 172.21.15.22:0/4255290993 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd1b01064a0 msgr2=0x7fd1b019e8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:13:18.275 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:13:18.274+0000 7fd1aea00640 1 --2- 172.21.15.22:0/4255290993 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd1b01064a0 0x7fd1b019e8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:13:18.275 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:13:18.274+0000 7fd1aea00640 1 -- 172.21.15.22:0/4255290993 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd1b019f020 con 0x7fd1b0100400 2024-03-28T18:13:18.276 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:13:18.274+0000 7fd1aea00640 1 --2- 172.21.15.22:0/4255290993 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd1b0100400 0x7fd1b019a030 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7fd1a0000c00 tx=0x7fd1a0004430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:13:18.276 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:13:18.275+0000 7fd1a7400640 1 -- 172.21.15.22:0/4255290993 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1a0043020 con 0x7fd1b0100400 2024-03-28T18:13:18.276 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:13:18.275+0000 7fd1afe00640 1 -- 172.21.15.22:0/4255290993 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd1b019f2b0 con 0x7fd1b0100400 2024-03-28T18:13:18.276 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:13:18.275+0000 7fd1a7400640 1 -- 172.21.15.22:0/4255290993 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7fd1a0005470 con 0x7fd1b0100400 2024-03-28T18:13:18.276 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:13:18.275+0000 7fd1a7400640 1 -- 172.21.15.22:0/4255290993 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1a0003ad0 con 0x7fd1b0100400 2024-03-28T18:13:18.276 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:13:18.275+0000 7fd1afe00640 1 -- 172.21.15.22:0/4255290993 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd1b01a6ba0 con 0x7fd1b0100400 2024-03-28T18:13:18.278 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:13:18.278+0000 7fd1a7400640 1 -- 172.21.15.22:0/4255290993 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 26) v1 ==== 137421+0+0 (secure 0 0 0) 0x7fd1a0046020 con 0x7fd1b0100400 2024-03-28T18:13:18.278 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:13:18.278+0000 7fd1b8879740 1 Processor -- start 2024-03-28T18:13:18.278 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:13:18.279+0000 7fd1b8879740 1 -- start start 2024-03-28T18:13:18.280 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:13:18.279+0000 7fd1b8879740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55dc416e9dd0 0x55dc416e8b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:13:18.280 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:13:18.279+0000 7fd1b8879740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55dc416e90f0 0x55dc416e9570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:13:18.281 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:13:18.279+0000 7fd1b8879740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55dc416f2460 0x55dc416f4850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:13:18.281 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:13:18.279+0000 7fd1b8879740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dc4164d250 con 0x55dc416e9dd0 2024-03-28T18:13:18.281 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:13:18.279+0000 7fd1b8879740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dc415f5a30 con 0x55dc416f2460 2024-03-28T18:13:18.281 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:13:18.279+0000 7fd1b8879740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dc416492f0 con 0x55dc416e90f0 2024-03-28T18:13:18.281 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:13:18.279+0000 7fd1af400640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55dc416e9dd0 0x55dc416e8b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:13:18.281 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:13:18.279+0000 7fd1af400640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55dc416e9dd0 0x55dc416e8b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:37922/0 (socket says 172.21.15.22:37922) 2024-03-28T18:13:18.281 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:13:18.279+0000 7fd1af400640 1 -- 172.21.15.22:0/2008952792 learned_addr learned my addr 172.21.15.22:0/2008952792 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-28T18:13:18.281 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:13:18.279+0000 7fd1ae000640 1 --2- 172.21.15.22:0/2008952792 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55dc416f2460 0x55dc416f4850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:13:18.281 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:13:18.279+0000 7fd1aea00640 1 --2- 172.21.15.22:0/2008952792 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55dc416e90f0 0x55dc416e9570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:13:18.281 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:13:18.280+0000 7fd1a7400640 1 --2- 172.21.15.22:0/4255290993 >> [v2:172.21.15.22:6800/1357365082,v1:172.21.15.22:6801/1357365082] conn(0x7fd1940a2970 0x7fd1940a4e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:13:18.281 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:13:18.280+0000 7fd1af400640 1 -- 172.21.15.22:0/2008952792 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55dc416e90f0 msgr2=0x55dc416e9570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T18:13:18.281 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:13:18.280+0000 7fd1af400640 1 --2- 172.21.15.22:0/2008952792 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55dc416e90f0 0x55dc416e9570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:13:18.281 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:13:18.280+0000 7fd1af400640 1 -- 172.21.15.22:0/2008952792 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55dc416f2460 msgr2=0x55dc416f4850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T18:13:18.281 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:13:18.280+0000 7fd1af400640 1 --2- 172.21.15.22:0/2008952792 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55dc416f2460 0x55dc416f4850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:13:18.281 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:13:18.280+0000 7fd1af400640 1 -- 172.21.15.22:0/2008952792 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55dc415e0690 con 0x55dc416e9dd0 2024-03-28T18:13:18.282 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:13:18.280+0000 7fd1a7400640 1 -- 172.21.15.22:0/4255290993 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 9357+0+0 (secure 0 0 0) 0x7fd1a00f11e0 con 0x7fd1b0100400 2024-03-28T18:13:18.282 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:13:18.280+0000 7fd1af400640 1 --2- 172.21.15.22:0/4255290993 >> [v2:172.21.15.22:6800/1357365082,v1:172.21.15.22:6801/1357365082] conn(0x7fd1940a2970 0x7fd1940a4e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:13:18.282 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:13:18.280+0000 7fd1af400640 1 --2- 172.21.15.22:0/2008952792 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55dc416e9dd0 0x55dc416e8b20 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7fd1a8009f50 tx=0x7fd1a80118d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=745a84a7f40e1028 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:13:18.282 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:13:18.281+0000 7fd19be00640 1 -- 172.21.15.22:0/2008952792 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1a8003bf0 con 0x55dc416e9dd0 2024-03-28T18:13:18.282 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:13:18.281+0000 7fd19be00640 1 -- 172.21.15.22:0/2008952792 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7fd1a8003d90 con 0x55dc416e9dd0 2024-03-28T18:13:18.282 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:13:18.281+0000 7fd19be00640 1 -- 172.21.15.22:0/2008952792 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1a800fc80 con 0x55dc416e9dd0 2024-03-28T18:13:18.286 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:13:18.281+0000 7fd1b8879740 1 -- 172.21.15.22:0/2008952792 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55dc416e9dd0 msgr2=0x55dc416e8b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:13:18.286 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:13:18.281+0000 7fd1b8879740 1 --2- 172.21.15.22:0/2008952792 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55dc416e9dd0 0x55dc416e8b20 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7fd1a8009f50 tx=0x7fd1a80118d0 comp rx=0 tx=0).stop 2024-03-28T18:13:18.286 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:13:18.281+0000 7fd1af400640 1 --2- 172.21.15.22:0/4255290993 >> [v2:172.21.15.22:6800/1357365082,v1:172.21.15.22:6801/1357365082] conn(0x7fd1940a2970 0x7fd1940a4e30 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fd1a8010010 tx=0x7fd1a800ff00 comp rx=0 tx=0).ready entity=mgr.15522 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:13:18.286 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:13:18.282+0000 7fd1b8879740 1 -- 172.21.15.22:0/2008952792 shutdown_connections 2024-03-28T18:13:18.286 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:13:18.282+0000 7fd1b8879740 1 --2- 172.21.15.22:0/2008952792 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55dc416f2460 0x55dc416f4850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:13:18.286 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:13:18.282+0000 7fd1b8879740 1 --2- 172.21.15.22:0/2008952792 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55dc416e90f0 0x55dc416e9570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:13:18.286 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:13:18.282+0000 7fd1b8879740 1 --2- 172.21.15.22:0/2008952792 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55dc416e9dd0 0x55dc416e8b20 unknown :-1 s=CLOSED pgs=724 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:13:18.286 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:13:18.282+0000 7fd1b8879740 1 -- 172.21.15.22:0/2008952792 >> 172.21.15.22:0/2008952792 conn(0x55dc416b0200 msgr2=0x55dc416e3b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:13:18.286 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:13:18.282+0000 7fd1b8879740 1 -- 172.21.15.22:0/2008952792 shutdown_connections 2024-03-28T18:13:18.286 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:13:18.282+0000 7fd1b8879740 1 -- 172.21.15.22:0/2008952792 wait complete. 2024-03-28T18:13:18.286 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:13:18.282+0000 7fd1b8879740 1 -- start start 2024-03-28T18:13:18.286 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:13:18.282+0000 7fd1b8879740 1 Processor -- start 2024-03-28T18:13:18.286 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:13:18.283+0000 7fd1b8879740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55dc416e90f0 0x55dc416f1a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:13:18.286 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:13:18.283+0000 7fd1b8879740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55dc416e9dd0 0x55dc416eee00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:13:18.287 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:13:18.283+0000 7fd1b8879740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55dc416f2460 0x55dc416ef340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:13:18.287 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:13:18.283+0000 7fd1b8879740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dc416f52d0 con 0x55dc416e9dd0 2024-03-28T18:13:18.287 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:13:18.283+0000 7fd1b8879740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dc41608e80 con 0x55dc416e90f0 2024-03-28T18:13:18.287 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:13:18.283+0000 7fd1b8879740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dc416f5450 con 0x55dc416f2460 2024-03-28T18:13:18.287 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:13:18.283+0000 7fd1b8879740 10 client.0 _ll_register_callbacks cb 0x55dc4161b130 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-28T18:13:18.287 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:13:18.283+0000 7fd1ae000640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55dc416e9dd0 0x55dc416eee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:13:18.287 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:13:18.283+0000 7fd1ae000640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55dc416e9dd0 0x55dc416eee00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:37930/0 (socket says 172.21.15.22:37930) 2024-03-28T18:13:18.287 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:13:18.283+0000 7fd1ae000640 1 -- 172.21.15.22:0/2973264811 learned_addr learned my addr 172.21.15.22:0/2973264811 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-28T18:13:18.287 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:13:18.283+0000 7fd1aea00640 1 --2- 172.21.15.22:0/2973264811 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55dc416e90f0 0x55dc416f1a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:13:18.287 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:13:18.283+0000 7fd1af400640 1 --2- 172.21.15.22:0/2973264811 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55dc416f2460 0x55dc416ef340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:13:18.287 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:13:18.284+0000 7fd1ae000640 1 -- 172.21.15.22:0/2973264811 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55dc416f2460 msgr2=0x55dc416ef340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T18:13:18.287 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:13:18.284+0000 7fd1ae000640 1 --2- 172.21.15.22:0/2973264811 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55dc416f2460 0x55dc416ef340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:13:18.287 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:13:18.284+0000 7fd1ae000640 1 -- 172.21.15.22:0/2973264811 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55dc416e90f0 msgr2=0x55dc416f1a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T18:13:18.287 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:13:18.284+0000 7fd1ae000640 1 --2- 172.21.15.22:0/2973264811 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55dc416e90f0 0x55dc416f1a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:13:18.287 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:13:18.284+0000 7fd1ae000640 1 -- 172.21.15.22:0/2973264811 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55dc4142b7d0 con 0x55dc416e9dd0 2024-03-28T18:13:18.288 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:13:18.284+0000 7fd1af400640 1 --2- 172.21.15.22:0/2973264811 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55dc416f2460 0x55dc416ef340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T18:13:18.288 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:13:18.284+0000 7fd1aea00640 1 --2- 172.21.15.22:0/2973264811 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55dc416e90f0 0x55dc416f1a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-28T18:13:18.288 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:13:18.284+0000 7fd1ae000640 1 --2- 172.21.15.22:0/2973264811 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55dc416e9dd0 0x55dc416eee00 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7fd19c00bf60 tx=0x7fd19c00bf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b9c66eacbe0150c2 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:13:18.288 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:13:18.284+0000 7fd198c00640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-03-28T18:13:18.288 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:13:18.284+0000 7fd198c00640 1 -- 172.21.15.22:0/2973264811 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd19c004230 con 0x55dc416e9dd0 2024-03-28T18:13:18.288 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:13:18.284+0000 7fd198c00640 1 -- 172.21.15.22:0/2973264811 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(104..104 src has 1..104) v4 ==== 9357+0+0 (secure 0 0 0) 0x7fd19c0043d0 con 0x55dc416e9dd0 2024-03-28T18:13:18.288 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:13:18.284+0000 7fd1b8879740 1 -- 172.21.15.22:0/2973264811 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fd1a8003d90 con 0x55dc416e9dd0 2024-03-28T18:13:18.288 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:13:18.285+0000 7fd198c00640 1 -- 172.21.15.22:0/2973264811 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=102) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd19c0035f0 con 0x55dc416e9dd0 2024-03-28T18:13:18.288 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:13:18.285+0000 7fd1b8879740 10 client.16617 fetch_fsmap learned FSMap version 102 2024-03-28T18:13:18.288 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:13:18.285+0000 7fd1b8879740 1 -- 172.21.15.22:0/2973264811 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=102,monmap=4+}) v3 -- 0x55dc413613f0 con 0x55dc416e9dd0 2024-03-28T18:13:18.288 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:13:18.285+0000 7fd198c00640 1 -- 172.21.15.22:0/2973264811 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7fd1a8003d90 con 0x55dc416e9dd0 2024-03-28T18:13:18.288 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:13:18.285+0000 7fd198c00640 1 -- 172.21.15.22:0/2973264811 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 102) v1 ==== 7363+0+0 (secure 0 0 0) 0x7fd19c010ae0 con 0x55dc416e9dd0 2024-03-28T18:13:18.288 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:13:18.285+0000 7fd1b8879740 10 client.16617 fetch_fsmap finished waiting for FSMap version 102 2024-03-28T18:13:18.288 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:13:18.285+0000 7fd1b8879740 10 client.16617 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-28T18:13:18.288 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:13:18.285+0000 7fd1b8879740 20 client.16617 populate_metadata read hostname 'smithi022' 2024-03-28T18:13:18.289 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:13:18.285+0000 7fd1b8879740 1 --2- 172.21.15.22:0/2973264811 >> [v2:172.21.15.79:6838/2441542443,v1:172.21.15.79:6839/2441542443] conn(0x55dc417024a0 0x55dc41704940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:13:18.289 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:13:18.285+0000 7fd1b8879740 4 client.16617 mds_command: new command op to 34553 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-28T18:13:18.289 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:13:18.285+0000 7fd1b8879740 1 -- 172.21.15.22:0/2973264811 --> [v2:172.21.15.79:6838/2441542443,v1:172.21.15.79:6839/2441542443] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55dc4165a790 con 0x55dc417024a0 2024-03-28T18:13:18.289 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:13:18.286+0000 7fd1aea00640 1 --2- 172.21.15.22:0/2973264811 >> [v2:172.21.15.79:6838/2441542443,v1:172.21.15.79:6839/2441542443] conn(0x55dc417024a0 0x55dc41704940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:13:18.290 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:13:18.286+0000 7fd1aea00640 1 --2- 172.21.15.22:0/2973264811 >> [v2:172.21.15.79:6838/2441542443,v1:172.21.15.79:6839/2441542443] conn(0x55dc417024a0 0x55dc41704940 crc :-1 s=READY pgs=174 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ec9a8bf3fa4e155d server_cookie=e54619672546ca16 in_seq=0 out_seq=0 2024-03-28T18:13:18.290 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:13:18.286+0000 7fd198c00640 10 client.16617 ms_handle_connect on v2:172.21.15.79:6838/2441542443 2024-03-28T18:13:18.290 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:13:18.287+0000 7fd198c00640 1 -- 172.21.15.22:0/2973264811 <== mds.0 v2:172.21.15.79:6838/2441542443 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x55dc4165a790 con 0x55dc417024a0 2024-03-28T18:13:18.290 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:13:18.287+0000 7fd198c00640 10 client.16617 handle_command_reply: tid=0 2024-03-28T18:13:18.290 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:13:18.288+0000 7fd1b8879740 1 client.16617 shutdown 2024-03-28T18:13:18.290 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:13:18.288+0000 7fd1b8879740 1 -- 172.21.15.22:0/2973264811 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55dc416e9dd0 msgr2=0x55dc416eee00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:13:18.290 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:13:18.288+0000 7fd1b8879740 1 --2- 172.21.15.22:0/2973264811 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55dc416e9dd0 0x55dc416eee00 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7fd19c00bf60 tx=0x7fd19c00bf90 comp rx=0 tx=0).stop 2024-03-28T18:13:18.290 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:13:18.288+0000 7fd1b8879740 1 -- 172.21.15.22:0/2973264811 shutdown_connections 2024-03-28T18:13:18.291 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:13:18.288+0000 7fd1b8879740 1 --2- 172.21.15.22:0/2973264811 >> [v2:172.21.15.79:6838/2441542443,v1:172.21.15.79:6839/2441542443] conn(0x55dc417024a0 0x55dc41704940 crc :-1 s=READY pgs=174 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:13:18.291 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:13:18.288+0000 7fd1b8879740 1 --2- 172.21.15.22:0/2973264811 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55dc416f2460 0x55dc416ef340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:13:18.291 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:13:18.288+0000 7fd1b8879740 1 --2- 172.21.15.22:0/2973264811 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55dc416e9dd0 0x55dc416eee00 unknown :-1 s=CLOSED pgs=725 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:13:18.291 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:13:18.288+0000 7fd1b8879740 1 --2- 172.21.15.22:0/2973264811 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55dc416e90f0 0x55dc416f1a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:13:18.291 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:13:18.288+0000 7fd198c00640 0 client.16617 ms_handle_reset on v2:172.21.15.79:6838/2441542443 2024-03-28T18:13:18.291 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:13:18.288+0000 7fd1b8879740 1 -- 172.21.15.22:0/2973264811 >> 172.21.15.22:0/2973264811 conn(0x55dc416b0200 msgr2=0x55dc416e5890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:13:18.291 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:13:18.288+0000 7fd1b8879740 1 -- 172.21.15.22:0/2973264811 shutdown_connections 2024-03-28T18:13:18.291 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:13:18.288+0000 7fd1b8879740 1 -- 172.21.15.22:0/2973264811 wait complete. 2024-03-28T18:13:18.291 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:13:18.288+0000 7fd1b8879740 20 client.16617 trim_cache size 0 max 16384 2024-03-28T18:13:18.305 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:13:18.306+0000 7fd1b8879740 1 Processor -- start 2024-03-28T18:13:18.306 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:13:18.306+0000 7fd1b8879740 1 -- start start 2024-03-28T18:13:18.307 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:13:18.306+0000 7fd1b8879740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55dc416e90f0 0x55dc416f1530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:13:18.307 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:13:18.306+0000 7fd1b8879740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55dc416e9dd0 0x55dc416f1a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:13:18.307 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:13:18.306+0000 7fd1b8879740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55dc416f2460 0x55dc416f3850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:13:18.307 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:13:18.306+0000 7fd1b8879740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1b01b1230 con 0x55dc416e9dd0 2024-03-28T18:13:18.307 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:13:18.306+0000 7fd1b8879740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1b01af8b0 con 0x55dc416e90f0 2024-03-28T18:13:18.307 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:13:18.306+0000 7fd1b8879740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1b01b0bd0 con 0x55dc416f2460 2024-03-28T18:13:18.307 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:13:18.307+0000 7fd1aea00640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55dc416e90f0 0x55dc416f1530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:13:18.307 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:13:18.307+0000 7fd1ae000640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55dc416e9dd0 0x55dc416f1a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:13:18.307 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:13:18.307+0000 7fd1aea00640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55dc416e90f0 0x55dc416f1530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.22:42848/0 (socket says 172.21.15.22:42848) 2024-03-28T18:13:18.307 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:13:18.307+0000 7fd1aea00640 1 -- 172.21.15.22:0/376133981 learned_addr learned my addr 172.21.15.22:0/376133981 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-28T18:13:18.307 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:13:18.307+0000 7fd1af400640 1 --2- 172.21.15.22:0/376133981 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55dc416f2460 0x55dc416f3850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:13:18.307 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:13:18.307+0000 7fd1aea00640 1 -- 172.21.15.22:0/376133981 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55dc416f2460 msgr2=0x55dc416f3850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T18:13:18.307 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:13:18.307+0000 7fd1aea00640 1 --2- 172.21.15.22:0/376133981 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55dc416f2460 0x55dc416f3850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:13:18.307 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:13:18.307+0000 7fd1aea00640 1 -- 172.21.15.22:0/376133981 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55dc416e9dd0 msgr2=0x55dc416f1a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T18:13:18.308 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:13:18.307+0000 7fd1aea00640 1 --2- 172.21.15.22:0/376133981 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55dc416e9dd0 0x55dc416f1a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:13:18.308 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:13:18.307+0000 7fd1aea00640 1 -- 172.21.15.22:0/376133981 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55dc413210f0 con 0x55dc416e90f0 2024-03-28T18:13:18.309 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:13:18.308+0000 7fd1aea00640 1 --2- 172.21.15.22:0/376133981 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55dc416e90f0 0x55dc416f1530 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fd1a00b9a10 tx=0x7fd1a00a8040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a2de493d53be16e2 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:13:18.309 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:13:18.308+0000 7fd19b400640 1 -- 172.21.15.22:0/376133981 <== mon.2 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1a00bd0f0 con 0x55dc416e90f0 2024-03-28T18:13:18.309 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:13:18.308+0000 7fd19b400640 1 -- 172.21.15.22:0/376133981 <== mon.2 v2:172.21.15.79:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7fd1a00bdcb0 con 0x55dc416e90f0 2024-03-28T18:13:18.309 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:13:18.308+0000 7fd19b400640 1 -- 172.21.15.22:0/376133981 <== mon.2 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1a00bcf10 con 0x55dc416e90f0 2024-03-28T18:13:18.309 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:13:18.308+0000 7fd1b8879740 1 -- 172.21.15.22:0/376133981 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55dc416e90f0 msgr2=0x55dc416f1530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:13:18.309 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:13:18.308+0000 7fd1b8879740 1 --2- 172.21.15.22:0/376133981 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55dc416e90f0 0x55dc416f1530 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fd1a00b9a10 tx=0x7fd1a00a8040 comp rx=0 tx=0).stop 2024-03-28T18:13:18.309 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:13:18.309+0000 7fd1b8879740 1 -- 172.21.15.22:0/376133981 shutdown_connections 2024-03-28T18:13:18.309 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:13:18.309+0000 7fd1b8879740 1 --2- 172.21.15.22:0/376133981 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55dc416f2460 0x55dc416f3850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:13:18.309 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:13:18.309+0000 7fd1b8879740 1 --2- 172.21.15.22:0/376133981 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55dc416e9dd0 0x55dc416f1a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:13:18.310 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:13:18.309+0000 7fd1b8879740 1 --2- 172.21.15.22:0/376133981 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55dc416e90f0 0x55dc416f1530 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:13:18.310 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:13:18.309+0000 7fd1b8879740 1 -- 172.21.15.22:0/376133981 >> 172.21.15.22:0/376133981 conn(0x55dc416b0200 msgr2=0x55dc416e3670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:13:18.310 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:13:18.309+0000 7fd1b8879740 1 -- 172.21.15.22:0/376133981 shutdown_connections 2024-03-28T18:13:18.310 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:13:18.309+0000 7fd1b8879740 1 -- 172.21.15.22:0/376133981 wait complete. 2024-03-28T18:13:18.310 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:13:18.309+0000 7fd1b8879740 1 -- start start 2024-03-28T18:13:18.310 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:13:18.309+0000 7fd1b8879740 1 Processor -- start 2024-03-28T18:13:18.313 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:13:18.310+0000 7fd1b8879740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55dc416e90f0 0x55dc416ec6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:13:18.313 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:13:18.310+0000 7fd1b8879740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55dc416e9dd0 0x55dc416eb090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:13:18.314 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:13:18.310+0000 7fd1b8879740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55dc416f2460 0x55dc416eb5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:13:18.314 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:13:18.310+0000 7fd1b8879740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1b01b0570 con 0x55dc416e90f0 2024-03-28T18:13:18.314 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:13:18.310+0000 7fd1b8879740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1b01aff10 con 0x55dc416e9dd0 2024-03-28T18:13:18.314 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:13:18.310+0000 7fd1b8879740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1b01ada90 con 0x55dc416f2460 2024-03-28T18:13:18.314 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:13:18.310+0000 7fd1b8879740 10 client.0 _ll_register_callbacks cb 0x7fd19c00b790 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-28T18:13:18.314 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:13:18.310+0000 7fd1aea00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55dc416e90f0 0x55dc416ec6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:13:18.314 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:13:18.310+0000 7fd1aea00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55dc416e90f0 0x55dc416ec6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:37952/0 (socket says 172.21.15.22:37952) 2024-03-28T18:13:18.314 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:13:18.310+0000 7fd1aea00640 1 -- 172.21.15.22:0/3492965851 learned_addr learned my addr 172.21.15.22:0/3492965851 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-28T18:13:18.314 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:13:18.310+0000 7fd1ae000640 1 --2- 172.21.15.22:0/3492965851 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55dc416e9dd0 0x55dc416eb090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:13:18.314 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:13:18.310+0000 7fd1af400640 1 --2- 172.21.15.22:0/3492965851 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55dc416f2460 0x55dc416eb5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:13:18.314 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:13:18.311+0000 7fd1aea00640 1 -- 172.21.15.22:0/3492965851 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55dc416f2460 msgr2=0x55dc416eb5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T18:13:18.314 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:13:18.311+0000 7fd1aea00640 1 --2- 172.21.15.22:0/3492965851 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55dc416f2460 0x55dc416eb5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:13:18.314 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:13:18.311+0000 7fd1aea00640 1 -- 172.21.15.22:0/3492965851 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55dc416e9dd0 msgr2=0x55dc416eb090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T18:13:18.315 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:13:18.311+0000 7fd1aea00640 1 --2- 172.21.15.22:0/3492965851 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55dc416e9dd0 0x55dc416eb090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:13:18.315 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:13:18.311+0000 7fd1aea00640 1 -- 172.21.15.22:0/3492965851 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55dc416ebb40 con 0x55dc416e90f0 2024-03-28T18:13:18.315 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:13:18.311+0000 7fd1ae000640 1 --2- 172.21.15.22:0/3492965851 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55dc416e9dd0 0x55dc416eb090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-28T18:13:18.315 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:13:18.311+0000 7fd1aea00640 1 --2- 172.21.15.22:0/3492965851 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55dc416e90f0 0x55dc416ec6c0 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7fd1a0005ab0 tx=0x7fd1a00b9930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ff0e406e851b44d9 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:13:18.315 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:13:18.311+0000 7fd198c00640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-03-28T18:13:18.315 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:13:18.311+0000 7fd198c00640 1 -- 172.21.15.22:0/3492965851 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1a00bd0f0 con 0x55dc416e90f0 2024-03-28T18:13:18.315 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:13:18.311+0000 7fd198c00640 1 -- 172.21.15.22:0/3492965851 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(104..104 src has 1..104) v4 ==== 9357+0+0 (secure 0 0 0) 0x7fd1a002cad0 con 0x55dc416e90f0 2024-03-28T18:13:18.315 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:13:18.311+0000 7fd1b8879740 1 -- 172.21.15.22:0/3492965851 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fd1a00bdcb0 con 0x55dc416e90f0 2024-03-28T18:13:18.315 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:13:18.311+0000 7fd198c00640 1 -- 172.21.15.22:0/3492965851 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=102) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd1a00bd5b0 con 0x55dc416e90f0 2024-03-28T18:13:18.315 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:13:18.311+0000 7fd1b8879740 10 client.16623 fetch_fsmap learned FSMap version 102 2024-03-28T18:13:18.315 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:13:18.311+0000 7fd1b8879740 1 -- 172.21.15.22:0/3492965851 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=102,monmap=4+}) v3 -- 0x55dc4175db80 con 0x55dc416e90f0 2024-03-28T18:13:18.315 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:13:18.312+0000 7fd198c00640 1 -- 172.21.15.22:0/3492965851 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7fd1a00bdcb0 con 0x55dc416e90f0 2024-03-28T18:13:18.315 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:13:18.312+0000 7fd198c00640 1 -- 172.21.15.22:0/3492965851 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 102) v1 ==== 7363+0+0 (secure 0 0 0) 0x7fd1a00eece0 con 0x55dc416e90f0 2024-03-28T18:13:18.315 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:13:18.312+0000 7fd1b8879740 10 client.16623 fetch_fsmap finished waiting for FSMap version 102 2024-03-28T18:13:18.315 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:13:18.312+0000 7fd1b8879740 10 client.16623 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-28T18:13:18.315 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:13:18.312+0000 7fd1b8879740 20 client.16623 populate_metadata read hostname 'smithi022' 2024-03-28T18:13:18.316 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:13:18.312+0000 7fd1b8879740 1 --2- 172.21.15.22:0/3492965851 >> [v2:172.21.15.79:6838/2441542443,v1:172.21.15.79:6839/2441542443] conn(0x55dc4175e2b0 0x55dc417606d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:13:18.316 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:13:18.312+0000 7fd1b8879740 4 client.16623 mds_command: new command op to 34553 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-28T18:13:18.316 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:13:18.312+0000 7fd1b8879740 1 -- 172.21.15.22:0/3492965851 --> [v2:172.21.15.79:6838/2441542443,v1:172.21.15.79:6839/2441542443] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55dc41654ae0 con 0x55dc4175e2b0 2024-03-28T18:13:18.316 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:13:18.312+0000 7fd1ae000640 1 --2- 172.21.15.22:0/3492965851 >> [v2:172.21.15.79:6838/2441542443,v1:172.21.15.79:6839/2441542443] conn(0x55dc4175e2b0 0x55dc417606d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:13:18.316 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:13:18.313+0000 7fd1ae000640 1 --2- 172.21.15.22:0/3492965851 >> [v2:172.21.15.79:6838/2441542443,v1:172.21.15.79:6839/2441542443] conn(0x55dc4175e2b0 0x55dc417606d0 crc :-1 s=READY pgs=175 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3a1c8f52e999a5e6 server_cookie=bb4d85d8cfe9890b in_seq=0 out_seq=0 2024-03-28T18:13:18.316 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:13:18.313+0000 7fd198c00640 10 client.16623 ms_handle_connect on v2:172.21.15.79:6838/2441542443 2024-03-28T18:13:18.316 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:13:18.313+0000 7fd198c00640 1 -- 172.21.15.22:0/3492965851 <== mds.0 v2:172.21.15.79:6838/2441542443 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x55dc41654ae0 con 0x55dc4175e2b0 2024-03-28T18:13:18.316 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:13:18.313+0000 7fd198c00640 10 client.16623 handle_command_reply: tid=0 2024-03-28T18:13:18.316 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:13:18.313+0000 7fd1b8879740 1 client.16623 shutdown 2024-03-28T18:13:18.316 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:13:18.314+0000 7fd1b8879740 1 -- 172.21.15.22:0/3492965851 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55dc416e90f0 msgr2=0x55dc416ec6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:13:18.316 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:13:18.314+0000 7fd1b8879740 1 --2- 172.21.15.22:0/3492965851 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55dc416e90f0 0x55dc416ec6c0 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7fd1a0005ab0 tx=0x7fd1a00b9930 comp rx=0 tx=0).stop 2024-03-28T18:13:18.316 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:13:18.314+0000 7fd1b8879740 1 -- 172.21.15.22:0/3492965851 shutdown_connections 2024-03-28T18:13:18.316 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:13:18.314+0000 7fd1b8879740 1 --2- 172.21.15.22:0/3492965851 >> [v2:172.21.15.79:6838/2441542443,v1:172.21.15.79:6839/2441542443] conn(0x55dc4175e2b0 0x55dc417606d0 crc :-1 s=READY pgs=175 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:13:18.316 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:13:18.314+0000 7fd1b8879740 1 --2- 172.21.15.22:0/3492965851 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55dc416f2460 0x55dc416eb5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:13:18.316 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:13:18.314+0000 7fd1b8879740 1 --2- 172.21.15.22:0/3492965851 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55dc416e9dd0 0x55dc416eb090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:13:18.316 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:13:18.314+0000 7fd1b8879740 1 --2- 172.21.15.22:0/3492965851 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55dc416e90f0 0x55dc416ec6c0 unknown :-1 s=CLOSED pgs=726 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:13:18.317 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:13:18.314+0000 7fd198c00640 0 client.16623 ms_handle_reset on v2:172.21.15.79:6838/2441542443 2024-03-28T18:13:18.317 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:13:18.314+0000 7fd1b8879740 1 -- 172.21.15.22:0/3492965851 >> 172.21.15.22:0/3492965851 conn(0x55dc416b0200 msgr2=0x55dc416ed6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:13:18.317 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:13:18.314+0000 7fd1b8879740 1 -- 172.21.15.22:0/3492965851 shutdown_connections 2024-03-28T18:13:18.317 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:13:18.314+0000 7fd1b8879740 1 -- 172.21.15.22:0/3492965851 wait complete. 2024-03-28T18:13:18.317 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:13:18.314+0000 7fd1b8879740 20 client.16623 trim_cache size 0 max 16384 2024-03-28T18:13:18.317 INFO:teuthology.orchestra.run.smithi022.stdout:{ 2024-03-28T18:13:18.317 INFO:teuthology.orchestra.run.smithi022.stdout: "status": "IDLE (waiting for more scrubs)", 2024-03-28T18:13:18.317 INFO:teuthology.orchestra.run.smithi022.stdout: "scrubs": { 2024-03-28T18:13:18.317 INFO:teuthology.orchestra.run.smithi022.stdout: "4bbbd61d-29a3-4bf6-a4f2-d5783d585de3": { 2024-03-28T18:13:18.317 INFO:teuthology.orchestra.run.smithi022.stdout: "path": "/", 2024-03-28T18:13:18.317 INFO:teuthology.orchestra.run.smithi022.stdout: "tag": "4bbbd61d-29a3-4bf6-a4f2-d5783d585de3", 2024-03-28T18:13:18.318 INFO:teuthology.orchestra.run.smithi022.stdout: "options": "recursive,force" 2024-03-28T18:13:18.318 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-03-28T18:13:18.318 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-03-28T18:13:18.318 INFO:teuthology.orchestra.run.smithi022.stdout:} 2024-03-28T18:13:18.318 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:13:18.315+0000 7fd1afe00640 1 -- 172.21.15.22:0/4255290993 >> [v2:172.21.15.22:6800/1357365082,v1:172.21.15.22:6801/1357365082] conn(0x7fd1940a2970 msgr2=0x7fd1940a4e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:13:18.319 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:13:18.315+0000 7fd1afe00640 1 --2- 172.21.15.22:0/4255290993 >> [v2:172.21.15.22:6800/1357365082,v1:172.21.15.22:6801/1357365082] conn(0x7fd1940a2970 0x7fd1940a4e30 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fd1a8010010 tx=0x7fd1a800ff00 comp rx=0 tx=0).stop 2024-03-28T18:13:18.319 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:13:18.315+0000 7fd1afe00640 1 -- 172.21.15.22:0/4255290993 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd1b0100400 msgr2=0x7fd1b019a030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:13:18.319 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:13:18.315+0000 7fd1afe00640 1 --2- 172.21.15.22:0/4255290993 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd1b0100400 0x7fd1b019a030 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7fd1a0000c00 tx=0x7fd1a0004430 comp rx=0 tx=0).stop 2024-03-28T18:13:18.319 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:13:18.315+0000 7fd1afe00640 1 -- 172.21.15.22:0/4255290993 shutdown_connections 2024-03-28T18:13:18.319 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:13:18.315+0000 7fd1afe00640 1 --2- 172.21.15.22:0/4255290993 >> [v2:172.21.15.22:6800/1357365082,v1:172.21.15.22:6801/1357365082] conn(0x7fd1940a2970 0x7fd1940a4e30 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:13:18.319 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:13:18.315+0000 7fd1afe00640 1 --2- 172.21.15.22:0/4255290993 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd1b01064a0 0x7fd1b019e8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:13:18.319 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:13:18.315+0000 7fd1afe00640 1 --2- 172.21.15.22:0/4255290993 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd1b0100dc0 0x7fd1b019a570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:13:18.319 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:13:18.315+0000 7fd1afe00640 1 --2- 172.21.15.22:0/4255290993 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd1b0100400 0x7fd1b019a030 unknown :-1 s=CLOSED pgs=723 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:13:18.319 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:13:18.315+0000 7fd1afe00640 1 -- 172.21.15.22:0/4255290993 >> 172.21.15.22:0/4255290993 conn(0x7fd1b00fbef0 msgr2=0x7fd1b0068800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:13:18.319 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:13:18.315+0000 7fd1afe00640 1 -- 172.21.15.22:0/4255290993 shutdown_connections 2024-03-28T18:13:18.319 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:13:18.315+0000 7fd1afe00640 1 -- 172.21.15.22:0/4255290993 wait complete. 2024-03-28T18:13:18.331 INFO:tasks.cephfs.filesystem:scrub status for tag:4bbbd61d-29a3-4bf6-a4f2-d5783d585de3 - {'path': '/', 'tag': '4bbbd61d-29a3-4bf6-a4f2-d5783d585de3', 'options': 'recursive,force'} 2024-03-28T18:13:18.785 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:13:18 smithi022 ceph-mon[31985]: pgmap v185: 97 pgs: 97 active+clean; 117 MiB data, 786 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 85 B/s wr, 8 op/s 2024-03-28T18:13:18.785 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:13:18 smithi022 ceph-mon[31985]: Replacing daemon mds.h as rank 1 with standby daemon mds.f 2024-03-28T18:13:18.785 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:13:18 smithi022 ceph-mon[31985]: MDS daemon mds.h is removed because it is dead or otherwise unavailable. 2024-03-28T18:13:18.786 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:13:18 smithi022 ceph-mon[31985]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-03-28T18:13:18.786 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:13:18 smithi022 ceph-mon[31985]: osdmap e104: 12 total, 12 up, 12 in 2024-03-28T18:13:18.786 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:13:18 smithi022 ceph-mon[31985]: fsmap cephfs:5/5 4 up:standby-replay 2 up:standby 1 up:replay 4 up:active 2024-03-28T18:13:18.925 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:13:18 smithi134 ceph-mon[35037]: pgmap v185: 97 pgs: 97 active+clean; 117 MiB data, 786 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 85 B/s wr, 8 op/s 2024-03-28T18:13:18.925 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:13:18 smithi134 ceph-mon[35037]: Replacing daemon mds.h as rank 1 with standby daemon mds.f 2024-03-28T18:13:18.925 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:13:18 smithi134 ceph-mon[35037]: MDS daemon mds.h is removed because it is dead or otherwise unavailable. 2024-03-28T18:13:18.925 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:13:18 smithi134 ceph-mon[35037]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-03-28T18:13:18.925 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:13:18 smithi134 ceph-mon[35037]: osdmap e104: 12 total, 12 up, 12 in 2024-03-28T18:13:18.925 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:13:18 smithi134 ceph-mon[35037]: fsmap cephfs:5/5 4 up:standby-replay 2 up:standby 1 up:replay 4 up:active 2024-03-28T18:13:18.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:13:18 smithi079 ceph-mon[35312]: pgmap v185: 97 pgs: 97 active+clean; 117 MiB data, 786 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 85 B/s wr, 8 op/s 2024-03-28T18:13:18.933 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:13:18 smithi079 ceph-mon[35312]: Replacing daemon mds.h as rank 1 with standby daemon mds.f 2024-03-28T18:13:18.933 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:13:18 smithi079 ceph-mon[35312]: MDS daemon mds.h is removed because it is dead or otherwise unavailable. 2024-03-28T18:13:18.933 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:13:18 smithi079 ceph-mon[35312]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-03-28T18:13:18.933 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:13:18 smithi079 ceph-mon[35312]: osdmap e104: 12 total, 12 up, 12 in 2024-03-28T18:13:18.933 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:13:18 smithi079 ceph-mon[35312]: fsmap cephfs:5/5 4 up:standby-replay 2 up:standby 1 up:replay 4 up:active 2024-03-28T18:13:20.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:13:20 smithi022 ceph-mon[31985]: pgmap v187: 97 pgs: 97 active+clean; 117 MiB data, 785 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 10 op/s 2024-03-28T18:13:20.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:13:20 smithi022 ceph-mon[31985]: mds.? [v2:172.21.15.134:6834/2867168769,v1:172.21.15.134:6835/2867168769] up:resolve 2024-03-28T18:13:20.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:13:20 smithi022 ceph-mon[31985]: fsmap cephfs:5/5 4 up:standby-replay 2 up:standby 1 up:resolve 4 up:active 2024-03-28T18:13:20.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:13:20 smithi134 ceph-mon[35037]: pgmap v187: 97 pgs: 97 active+clean; 117 MiB data, 785 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 10 op/s 2024-03-28T18:13:20.925 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:13:20 smithi134 ceph-mon[35037]: mds.? [v2:172.21.15.134:6834/2867168769,v1:172.21.15.134:6835/2867168769] up:resolve 2024-03-28T18:13:20.925 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:13:20 smithi134 ceph-mon[35037]: fsmap cephfs:5/5 4 up:standby-replay 2 up:standby 1 up:resolve 4 up:active 2024-03-28T18:13:20.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:13:20 smithi079 ceph-mon[35312]: pgmap v187: 97 pgs: 97 active+clean; 117 MiB data, 785 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 10 op/s 2024-03-28T18:13:20.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:13:20 smithi079 ceph-mon[35312]: mds.? [v2:172.21.15.134:6834/2867168769,v1:172.21.15.134:6835/2867168769] up:resolve 2024-03-28T18:13:20.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:13:20 smithi079 ceph-mon[35312]: fsmap cephfs:5/5 4 up:standby-replay 2 up:standby 1 up:resolve 4 up:active 2024-03-28T18:13:21.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:13:21 smithi022 ceph-mon[31985]: mds.? [v2:172.21.15.134:6834/2867168769,v1:172.21.15.134:6835/2867168769] up:reconnect 2024-03-28T18:13:21.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:13:21 smithi022 ceph-mon[31985]: fsmap cephfs:5/5 4 up:standby-replay 2 up:standby 1 up:reconnect 4 up:active 2024-03-28T18:13:21.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:13:21 smithi022 ceph-mon[31985]: reconnect by client.35222 172.21.15.22:0/3729116781 after 0.00499988 2024-03-28T18:13:21.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:13:21 smithi134 ceph-mon[35037]: mds.? [v2:172.21.15.134:6834/2867168769,v1:172.21.15.134:6835/2867168769] up:reconnect 2024-03-28T18:13:21.925 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:13:21 smithi134 ceph-mon[35037]: fsmap cephfs:5/5 4 up:standby-replay 2 up:standby 1 up:reconnect 4 up:active 2024-03-28T18:13:21.925 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:13:21 smithi134 ceph-mon[35037]: reconnect by client.35222 172.21.15.22:0/3729116781 after 0.00499988 2024-03-28T18:13:21.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:13:21 smithi079 ceph-mon[35312]: mds.? [v2:172.21.15.134:6834/2867168769,v1:172.21.15.134:6835/2867168769] up:reconnect 2024-03-28T18:13:21.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:13:21 smithi079 ceph-mon[35312]: fsmap cephfs:5/5 4 up:standby-replay 2 up:standby 1 up:reconnect 4 up:active 2024-03-28T18:13:21.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:13:21 smithi079 ceph-mon[35312]: reconnect by client.35222 172.21.15.22:0/3729116781 after 0.00499988 2024-03-28T18:13:22.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:13:22 smithi022 ceph-mon[31985]: pgmap v188: 97 pgs: 97 active+clean; 117 MiB data, 785 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 10 op/s 2024-03-28T18:13:22.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:13:22 smithi022 ceph-mon[31985]: mds.? [v2:172.21.15.134:6834/2867168769,v1:172.21.15.134:6835/2867168769] up:rejoin 2024-03-28T18:13:22.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:13:22 smithi022 ceph-mon[31985]: fsmap cephfs:5/5 4 up:standby-replay 2 up:standby 1 up:rejoin 4 up:active 2024-03-28T18:13:22.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:13:22 smithi022 ceph-mon[31985]: daemon mds.f is now active in filesystem cephfs as rank 1 2024-03-28T18:13:22.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:13:22 smithi134 ceph-mon[35037]: pgmap v188: 97 pgs: 97 active+clean; 117 MiB data, 785 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 10 op/s 2024-03-28T18:13:22.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:13:22 smithi134 ceph-mon[35037]: mds.? [v2:172.21.15.134:6834/2867168769,v1:172.21.15.134:6835/2867168769] up:rejoin 2024-03-28T18:13:22.925 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:13:22 smithi134 ceph-mon[35037]: fsmap cephfs:5/5 4 up:standby-replay 2 up:standby 1 up:rejoin 4 up:active 2024-03-28T18:13:22.925 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:13:22 smithi134 ceph-mon[35037]: daemon mds.f is now active in filesystem cephfs as rank 1 2024-03-28T18:13:22.926 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:13:22 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[70779]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-2145-g3be9a5c9/rpm/el8/BUILD/ceph-19.0.0-2145-g3be9a5c9/src/mds/Locker.cc: In function 'void Locker::set_xlocks_done(MutationImpl*, bool)' thread 7fe4d9c00700 time 2024-03-28T18:13:22.595743+0000 2024-03-28T18:13:22.926 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:13:22 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[70779]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-2145-g3be9a5c9/rpm/el8/BUILD/ceph-19.0.0-2145-g3be9a5c9/src/mds/Locker.cc: 626: FAILED ceph_assert(obj->is_auth()) 2024-03-28T18:13:22.926 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:13:22 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[70779]: ceph version 19.0.0-2145-g3be9a5c9 (3be9a5c9bc793e11e2800a8c0c696e8b46742033) squid (dev) 2024-03-28T18:13:22.926 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:13:22 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[70779]: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x135) [0x7fe4e55b04a1] 2024-03-28T18:13:22.926 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:13:22 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[70779]: 2: /usr/lib64/ceph/libceph-common.so.2(+0x2d2667) [0x7fe4e55b0667] 2024-03-28T18:13:22.926 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:13:22 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[70779]: 3: (Locker::set_xlocks_done(MutationImpl*, bool)+0x493) [0x55839175f223] 2024-03-28T18:13:22.926 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:13:22 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[70779]: 4: (Server::early_reply(boost::intrusive_ptr const&, CInode*, CDentry*)+0x3e6) [0x5583915d95d6] 2024-03-28T18:13:22.926 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:13:22 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[70779]: 5: (Server::journal_and_reply(boost::intrusive_ptr const&, CInode*, CDentry*, LogEvent*, MDSLogContextBase*)+0xc1) [0x5583915d9da1] 2024-03-28T18:13:22.926 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:13:22 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[70779]: 6: (Server::handle_client_mkdir(boost::intrusive_ptr const&)+0x5ba) [0x55839160f6aa] 2024-03-28T18:13:22.927 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:13:22 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[70779]: 7: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0x733) [0x55839162c233] 2024-03-28T18:13:22.927 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:13:22 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[70779]: 8: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x55839162e197] 2024-03-28T18:13:22.927 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:13:22 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[70779]: 9: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x55839152d11b] 2024-03-28T18:13:22.927 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:13:22 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[70779]: 10: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x55839152d6bc] 2024-03-28T18:13:22.927 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:13:22 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[70779]: 11: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bf) [0x5583915171ff] 2024-03-28T18:13:22.927 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:13:22 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[70779]: 12: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x478) [0x7fe4e5842488] 2024-03-28T18:13:22.927 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:13:22 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[70779]: 13: (DispatchQueue::entry()+0x51f) [0x7fe4e583f60f] 2024-03-28T18:13:22.927 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:13:22 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[70779]: 14: (DispatchQueue::DispatchThread::entry()+0x11) [0x7fe4e5912e81] 2024-03-28T18:13:22.927 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:13:22 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[70779]: 15: /lib64/libpthread.so.0(+0x81ca) [0x7fe4e432d1ca] 2024-03-28T18:13:22.927 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:13:22 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[70779]: 16: clone() 2024-03-28T18:13:22.929 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:13:22 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[70779]: 2024-03-28T18:13:22.596+0000 7fe4d9c00700 -1 /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-2145-g3be9a5c9/rpm/el8/BUILD/ceph-19.0.0-2145-g3be9a5c9/src/mds/Locker.cc: In function 'void Locker::set_xlocks_done(MutationImpl*, bool)' thread 7fe4d9c00700 time 2024-03-28T18:13:22.595743+0000 2024-03-28T18:13:22.929 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:13:22 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[70779]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-2145-g3be9a5c9/rpm/el8/BUILD/ceph-19.0.0-2145-g3be9a5c9/src/mds/Locker.cc: 626: FAILED ceph_assert(obj->is_auth()) 2024-03-28T18:13:22.929 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:13:22 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[70779]: 2024-03-28T18:13:22.929 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:13:22 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[70779]: ceph version 19.0.0-2145-g3be9a5c9 (3be9a5c9bc793e11e2800a8c0c696e8b46742033) squid (dev) 2024-03-28T18:13:22.929 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:13:22 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[70779]: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x135) [0x7fe4e55b04a1] 2024-03-28T18:13:22.929 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:13:22 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[70779]: 2: /usr/lib64/ceph/libceph-common.so.2(+0x2d2667) [0x7fe4e55b0667] 2024-03-28T18:13:22.930 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:13:22 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[70779]: 3: (Locker::set_xlocks_done(MutationImpl*, bool)+0x493) [0x55839175f223] 2024-03-28T18:13:22.930 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:13:22 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[70779]: 4: (Server::early_reply(boost::intrusive_ptr const&, CInode*, CDentry*)+0x3e6) [0x5583915d95d6] 2024-03-28T18:13:22.930 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:13:22 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[70779]: 5: (Server::journal_and_reply(boost::intrusive_ptr const&, CInode*, CDentry*, LogEvent*, MDSLogContextBase*)+0xc1) [0x5583915d9da1] 2024-03-28T18:13:22.930 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:13:22 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[70779]: 6: (Server::handle_client_mkdir(boost::intrusive_ptr const&)+0x5ba) [0x55839160f6aa] 2024-03-28T18:13:22.930 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:13:22 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[70779]: 7: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0x733) [0x55839162c233] 2024-03-28T18:13:22.930 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:13:22 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[70779]: 8: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x55839162e197] 2024-03-28T18:13:22.930 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:13:22 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[70779]: 9: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x55839152d11b] 2024-03-28T18:13:22.930 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:13:22 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[70779]: 10: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x55839152d6bc] 2024-03-28T18:13:22.930 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:13:22 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[70779]: 11: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bf) [0x5583915171ff] 2024-03-28T18:13:22.930 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:13:22 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[70779]: 12: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x478) [0x7fe4e5842488] 2024-03-28T18:13:22.930 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:13:22 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[70779]: 13: (DispatchQueue::entry()+0x51f) [0x7fe4e583f60f] 2024-03-28T18:13:22.930 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:13:22 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[70779]: 14: (DispatchQueue::DispatchThread::entry()+0x11) [0x7fe4e5912e81] 2024-03-28T18:13:22.930 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:13:22 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[70779]: 15: /lib64/libpthread.so.0(+0x81ca) [0x7fe4e432d1ca] 2024-03-28T18:13:22.930 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:13:22 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[70779]: 16: clone() 2024-03-28T18:13:22.931 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:13:22 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[70779]: 2024-03-28T18:13:22.931 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:13:22 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[70779]: *** Caught signal (Aborted) ** 2024-03-28T18:13:22.931 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:13:22 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[70779]: in thread 7fe4d9c00700 thread_name:ms_dispatch 2024-03-28T18:13:22.931 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:13:22 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[70779]: ceph version 19.0.0-2145-g3be9a5c9 (3be9a5c9bc793e11e2800a8c0c696e8b46742033) squid (dev) 2024-03-28T18:13:22.931 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:13:22 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[70779]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7fe4e4337d20] 2024-03-28T18:13:22.932 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:13:22 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[70779]: 2: gsignal() 2024-03-28T18:13:22.932 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:13:22 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[70779]: 3: abort() 2024-03-28T18:13:22.932 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:13:22 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[70779]: 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x18f) [0x7fe4e55b04fb] 2024-03-28T18:13:22.932 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:13:22 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[70779]: 5: /usr/lib64/ceph/libceph-common.so.2(+0x2d2667) [0x7fe4e55b0667] 2024-03-28T18:13:22.932 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:13:22 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[70779]: 6: (Locker::set_xlocks_done(MutationImpl*, bool)+0x493) [0x55839175f223] 2024-03-28T18:13:22.932 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:13:22 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[70779]: 7: (Server::early_reply(boost::intrusive_ptr const&, CInode*, CDentry*)+0x3e6) [0x5583915d95d6] 2024-03-28T18:13:22.932 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:13:22 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[70779]: 8: (Server::journal_and_reply(boost::intrusive_ptr const&, CInode*, CDentry*, LogEvent*, MDSLogContextBase*)+0xc1) [0x5583915d9da1] 2024-03-28T18:13:22.932 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:13:22 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[70779]: 9: (Server::handle_client_mkdir(boost::intrusive_ptr const&)+0x5ba) [0x55839160f6aa] 2024-03-28T18:13:22.932 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:13:22 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[70779]: 10: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0x733) [0x55839162c233] 2024-03-28T18:13:22.932 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:13:22 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[70779]: 11: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x55839162e197] 2024-03-28T18:13:22.932 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:13:22 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[70779]: 12: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x55839152d11b] 2024-03-28T18:13:22.932 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:13:22 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[70779]: 13: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x55839152d6bc] 2024-03-28T18:13:22.933 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:13:22 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[70779]: 14: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bf) [0x5583915171ff] 2024-03-28T18:13:22.933 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:13:22 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[70779]: 15: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x478) [0x7fe4e5842488] 2024-03-28T18:13:22.933 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:13:22 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[70779]: 16: (DispatchQueue::entry()+0x51f) [0x7fe4e583f60f] 2024-03-28T18:13:22.933 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:13:22 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[70779]: 17: (DispatchQueue::DispatchThread::entry()+0x11) [0x7fe4e5912e81] 2024-03-28T18:13:22.933 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:13:22 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[70779]: 18: /lib64/libpthread.so.0(+0x81ca) [0x7fe4e432d1ca] 2024-03-28T18:13:22.933 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:13:22 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[70779]: 19: clone() 2024-03-28T18:13:22.933 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:13:22 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[70779]: 2024-03-28T18:13:22.599+0000 7fe4d9c00700 -1 *** Caught signal (Aborted) ** 2024-03-28T18:13:22.933 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:13:22 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[70779]: in thread 7fe4d9c00700 thread_name:ms_dispatch 2024-03-28T18:13:22.933 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:13:22 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[70779]: 2024-03-28T18:13:22.933 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:13:22 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[70779]: ceph version 19.0.0-2145-g3be9a5c9 (3be9a5c9bc793e11e2800a8c0c696e8b46742033) squid (dev) 2024-03-28T18:13:22.933 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:13:22 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[70779]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7fe4e4337d20] 2024-03-28T18:13:22.933 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:13:22 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[70779]: 2: gsignal() 2024-03-28T18:13:22.933 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:13:22 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[70779]: 3: abort() 2024-03-28T18:13:22.933 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:13:22 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[70779]: 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x18f) [0x7fe4e55b04fb] 2024-03-28T18:13:22.933 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:13:22 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[70779]: 5: /usr/lib64/ceph/libceph-common.so.2(+0x2d2667) [0x7fe4e55b0667] 2024-03-28T18:13:22.934 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:13:22 smithi079 ceph-mon[35312]: pgmap v188: 97 pgs: 97 active+clean; 117 MiB data, 785 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 10 op/s 2024-03-28T18:13:22.934 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:13:22 smithi079 ceph-mon[35312]: mds.? [v2:172.21.15.134:6834/2867168769,v1:172.21.15.134:6835/2867168769] up:rejoin 2024-03-28T18:13:22.934 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:13:22 smithi079 ceph-mon[35312]: fsmap cephfs:5/5 4 up:standby-replay 2 up:standby 1 up:rejoin 4 up:active 2024-03-28T18:13:22.934 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:13:22 smithi079 ceph-mon[35312]: daemon mds.f is now active in filesystem cephfs as rank 1 2024-03-28T18:13:22.934 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:13:22 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[70779]: 6: (Locker::set_xlocks_done(MutationImpl*, bool)+0x493) [0x55839175f223] 2024-03-28T18:13:22.934 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:13:22 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[70779]: 7: (Server::early_reply(boost::intrusive_ptr const&, CInode*, CDentry*)+0x3e6) [0x5583915d95d6] 2024-03-28T18:13:22.934 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:13:22 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[70779]: 8: (Server::journal_and_reply(boost::intrusive_ptr const&, CInode*, CDentry*, LogEvent*, MDSLogContextBase*)+0xc1) [0x5583915d9da1] 2024-03-28T18:13:22.935 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:13:22 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[70779]: 9: (Server::handle_client_mkdir(boost::intrusive_ptr const&)+0x5ba) [0x55839160f6aa] 2024-03-28T18:13:22.935 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:13:22 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[70779]: 10: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0x733) [0x55839162c233] 2024-03-28T18:13:22.935 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:13:22 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[70779]: 11: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x55839162e197] 2024-03-28T18:13:22.935 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:13:22 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[70779]: 12: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x55839152d11b] 2024-03-28T18:13:22.935 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:13:22 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[70779]: 13: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x55839152d6bc] 2024-03-28T18:13:22.935 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:13:22 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[70779]: 14: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bf) [0x5583915171ff] 2024-03-28T18:13:22.935 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:13:22 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[70779]: 15: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x478) [0x7fe4e5842488] 2024-03-28T18:13:22.935 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:13:22 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[70779]: 16: (DispatchQueue::entry()+0x51f) [0x7fe4e583f60f] 2024-03-28T18:13:22.935 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:13:22 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[70779]: 17: (DispatchQueue::DispatchThread::entry()+0x11) [0x7fe4e5912e81] 2024-03-28T18:13:22.935 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:13:22 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[70779]: 18: /lib64/libpthread.so.0(+0x81ca) [0x7fe4e432d1ca] 2024-03-28T18:13:22.935 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:13:22 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[70779]: 19: clone() 2024-03-28T18:13:22.935 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:13:22 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[70779]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-03-28T18:13:22.935 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:13:22 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[70779]: 2024-03-28T18:13:22.935 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:13:22 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[70779]: -1> 2024-03-28T18:13:22.596+0000 7fe4d9c00700 -1 /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-2145-g3be9a5c9/rpm/el8/BUILD/ceph-19.0.0-2145-g3be9a5c9/src/mds/Locker.cc: In function 'void Locker::set_xlocks_done(MutationImpl*, bool)' thread 7fe4d9c00700 time 2024-03-28T18:13:22.595743+0000 2024-03-28T18:13:22.935 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:13:22 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[70779]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-2145-g3be9a5c9/rpm/el8/BUILD/ceph-19.0.0-2145-g3be9a5c9/src/mds/Locker.cc: 626: FAILED ceph_assert(obj->is_auth()) 2024-03-28T18:13:22.935 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:13:22 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[70779]: 2024-03-28T18:13:22.936 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:13:22 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[70779]: ceph version 19.0.0-2145-g3be9a5c9 (3be9a5c9bc793e11e2800a8c0c696e8b46742033) squid (dev) 2024-03-28T18:13:22.936 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:13:22 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[70779]: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x135) [0x7fe4e55b04a1] 2024-03-28T18:13:22.936 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:13:22 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[70779]: 2: /usr/lib64/ceph/libceph-common.so.2(+0x2d2667) [0x7fe4e55b0667] 2024-03-28T18:13:22.936 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:13:22 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[70779]: 3: (Locker::set_xlocks_done(MutationImpl*, bool)+0x493) [0x55839175f223] 2024-03-28T18:13:22.936 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:13:22 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[70779]: 4: (Server::early_reply(boost::intrusive_ptr const&, CInode*, CDentry*)+0x3e6) [0x5583915d95d6] 2024-03-28T18:13:22.936 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:13:22 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[70779]: 5: (Server::journal_and_reply(boost::intrusive_ptr const&, CInode*, CDentry*, LogEvent*, MDSLogContextBase*)+0xc1) [0x5583915d9da1] 2024-03-28T18:13:22.936 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:13:22 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[70779]: 6: (Server::handle_client_mkdir(boost::intrusive_ptr const&)+0x5ba) [0x55839160f6aa] 2024-03-28T18:13:22.936 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:13:22 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[70779]: 7: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0x733) [0x55839162c233] 2024-03-28T18:13:22.936 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:13:22 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[70779]: 8: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x55839162e197] 2024-03-28T18:13:22.936 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:13:22 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[70779]: 9: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x55839152d11b] 2024-03-28T18:13:22.936 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:13:22 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[70779]: 10: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x55839152d6bc] 2024-03-28T18:13:22.936 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:13:22 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[70779]: 11: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bf) [0x5583915171ff] 2024-03-28T18:13:22.937 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:13:22 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[70779]: 12: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x478) [0x7fe4e5842488] 2024-03-28T18:13:22.937 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:13:22 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[70779]: 13: (DispatchQueue::entry()+0x51f) [0x7fe4e583f60f] 2024-03-28T18:13:22.937 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:13:22 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[70779]: 14: (DispatchQueue::DispatchThread::entry()+0x11) [0x7fe4e5912e81] 2024-03-28T18:13:22.937 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:13:22 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[70779]: 15: /lib64/libpthread.so.0(+0x81ca) [0x7fe4e432d1ca] 2024-03-28T18:13:22.937 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:13:22 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[70779]: 16: clone() 2024-03-28T18:13:22.937 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:13:22 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[70779]: 2024-03-28T18:13:22.937 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:13:22 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[70779]: 0> 2024-03-28T18:13:22.599+0000 7fe4d9c00700 -1 *** Caught signal (Aborted) ** 2024-03-28T18:13:22.937 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:13:22 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[70779]: in thread 7fe4d9c00700 thread_name:ms_dispatch 2024-03-28T18:13:22.937 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:13:22 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[70779]: 2024-03-28T18:13:22.937 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:13:22 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[70779]: ceph version 19.0.0-2145-g3be9a5c9 (3be9a5c9bc793e11e2800a8c0c696e8b46742033) squid (dev) 2024-03-28T18:13:22.937 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:13:22 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[70779]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7fe4e4337d20] 2024-03-28T18:13:22.937 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:13:22 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[70779]: 2: gsignal() 2024-03-28T18:13:22.937 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:13:22 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[70779]: 3: abort() 2024-03-28T18:13:22.937 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:13:22 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[70779]: 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x18f) [0x7fe4e55b04fb] 2024-03-28T18:13:22.937 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:13:22 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[70779]: 5: /usr/lib64/ceph/libceph-common.so.2(+0x2d2667) [0x7fe4e55b0667] 2024-03-28T18:13:22.937 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:13:22 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[70779]: 6: (Locker::set_xlocks_done(MutationImpl*, bool)+0x493) [0x55839175f223] 2024-03-28T18:13:22.937 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:13:22 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[70779]: 7: (Server::early_reply(boost::intrusive_ptr const&, CInode*, CDentry*)+0x3e6) [0x5583915d95d6] 2024-03-28T18:13:22.938 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:13:22 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[70779]: 8: (Server::journal_and_reply(boost::intrusive_ptr const&, CInode*, CDentry*, LogEvent*, MDSLogContextBase*)+0xc1) [0x5583915d9da1] 2024-03-28T18:13:22.938 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:13:22 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[70779]: 9: (Server::handle_client_mkdir(boost::intrusive_ptr const&)+0x5ba) [0x55839160f6aa] 2024-03-28T18:13:22.938 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:13:22 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[70779]: 10: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0x733) [0x55839162c233] 2024-03-28T18:13:22.938 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:13:22 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[70779]: 11: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x55839162e197] 2024-03-28T18:13:22.938 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:13:22 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[70779]: 12: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x55839152d11b] 2024-03-28T18:13:22.939 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:13:22 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[70779]: 13: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x55839152d6bc] 2024-03-28T18:13:22.939 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:13:22 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[70779]: 14: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bf) [0x5583915171ff] 2024-03-28T18:13:22.939 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:13:22 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[70779]: 15: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x478) [0x7fe4e5842488] 2024-03-28T18:13:22.939 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:13:22 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[70779]: 16: (DispatchQueue::entry()+0x51f) [0x7fe4e583f60f] 2024-03-28T18:13:22.939 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:13:22 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[70779]: 17: (DispatchQueue::DispatchThread::entry()+0x11) [0x7fe4e5912e81] 2024-03-28T18:13:22.939 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:13:22 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[70779]: 18: /lib64/libpthread.so.0(+0x81ca) [0x7fe4e432d1ca] 2024-03-28T18:13:22.939 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:13:22 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[70779]: 19: clone() 2024-03-28T18:13:22.939 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:13:22 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[70779]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-03-28T18:13:22.939 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:13:22 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[70779]: 2024-03-28T18:13:22.939 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:13:22 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[70779]: -1> 2024-03-28T18:13:22.596+0000 7fe4d9c00700 -1 /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-2145-g3be9a5c9/rpm/el8/BUILD/ceph-19.0.0-2145-g3be9a5c9/src/mds/Locker.cc: In function 'void Locker::set_xlocks_done(MutationImpl*, bool)' thread 7fe4d9c00700 time 2024-03-28T18:13:22.595743+0000 2024-03-28T18:13:22.939 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:13:22 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[70779]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-2145-g3be9a5c9/rpm/el8/BUILD/ceph-19.0.0-2145-g3be9a5c9/src/mds/Locker.cc: 626: FAILED ceph_assert(obj->is_auth()) 2024-03-28T18:13:22.939 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:13:22 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[70779]: 2024-03-28T18:13:22.939 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:13:22 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[70779]: ceph version 19.0.0-2145-g3be9a5c9 (3be9a5c9bc793e11e2800a8c0c696e8b46742033) squid (dev) 2024-03-28T18:13:22.939 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:13:22 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[70779]: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x135) [0x7fe4e55b04a1] 2024-03-28T18:13:22.940 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:13:22 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[70779]: 2: /usr/lib64/ceph/libceph-common.so.2(+0x2d2667) [0x7fe4e55b0667] 2024-03-28T18:13:22.940 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:13:22 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[70779]: 3: (Locker::set_xlocks_done(MutationImpl*, bool)+0x493) [0x55839175f223] 2024-03-28T18:13:22.940 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:13:22 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[70779]: 4: (Server::early_reply(boost::intrusive_ptr const&, CInode*, CDentry*)+0x3e6) [0x5583915d95d6] 2024-03-28T18:13:22.940 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:13:22 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[70779]: 5: (Server::journal_and_reply(boost::intrusive_ptr const&, CInode*, CDentry*, LogEvent*, MDSLogContextBase*)+0xc1) [0x5583915d9da1] 2024-03-28T18:13:22.940 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:13:22 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[70779]: 6: (Server::handle_client_mkdir(boost::intrusive_ptr const&)+0x5ba) [0x55839160f6aa] 2024-03-28T18:13:22.940 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:13:22 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[70779]: 7: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0x733) [0x55839162c233] 2024-03-28T18:13:22.940 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:13:22 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[70779]: 8: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x55839162e197] 2024-03-28T18:13:22.940 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:13:22 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[70779]: 9: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x55839152d11b] 2024-03-28T18:13:22.940 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:13:22 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[70779]: 10: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x55839152d6bc] 2024-03-28T18:13:22.940 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:13:22 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[70779]: 11: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bf) [0x5583915171ff] 2024-03-28T18:13:22.940 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:13:22 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[70779]: 12: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x478) [0x7fe4e5842488] 2024-03-28T18:13:22.940 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:13:22 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[70779]: 13: (DispatchQueue::entry()+0x51f) [0x7fe4e583f60f] 2024-03-28T18:13:22.940 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:13:22 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[70779]: 14: (DispatchQueue::DispatchThread::entry()+0x11) [0x7fe4e5912e81] 2024-03-28T18:13:22.940 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:13:22 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[70779]: 15: /lib64/libpthread.so.0(+0x81ca) [0x7fe4e432d1ca] 2024-03-28T18:13:22.940 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:13:22 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[70779]: 16: clone() 2024-03-28T18:13:22.940 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:13:22 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[70779]: 2024-03-28T18:13:22.941 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:13:22 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[70779]: 0> 2024-03-28T18:13:22.599+0000 7fe4d9c00700 -1 *** Caught signal (Aborted) ** 2024-03-28T18:13:22.941 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:13:22 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[70779]: in thread 7fe4d9c00700 thread_name:ms_dispatch 2024-03-28T18:13:22.941 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:13:22 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[70779]: 2024-03-28T18:13:22.941 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:13:22 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[70779]: ceph version 19.0.0-2145-g3be9a5c9 (3be9a5c9bc793e11e2800a8c0c696e8b46742033) squid (dev) 2024-03-28T18:13:22.941 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:13:22 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[70779]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7fe4e4337d20] 2024-03-28T18:13:22.941 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:13:22 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[70779]: 2: gsignal() 2024-03-28T18:13:22.941 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:13:22 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[70779]: 3: abort() 2024-03-28T18:13:22.941 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:13:22 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[70779]: 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x18f) [0x7fe4e55b04fb] 2024-03-28T18:13:22.941 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:13:22 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[70779]: 5: /usr/lib64/ceph/libceph-common.so.2(+0x2d2667) [0x7fe4e55b0667] 2024-03-28T18:13:22.941 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:13:22 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[70779]: 6: (Locker::set_xlocks_done(MutationImpl*, bool)+0x493) [0x55839175f223] 2024-03-28T18:13:22.941 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:13:22 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[70779]: 7: (Server::early_reply(boost::intrusive_ptr const&, CInode*, CDentry*)+0x3e6) [0x5583915d95d6] 2024-03-28T18:13:22.941 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:13:22 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[70779]: 8: (Server::journal_and_reply(boost::intrusive_ptr const&, CInode*, CDentry*, LogEvent*, MDSLogContextBase*)+0xc1) [0x5583915d9da1] 2024-03-28T18:13:22.941 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:13:22 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[70779]: 9: (Server::handle_client_mkdir(boost::intrusive_ptr const&)+0x5ba) [0x55839160f6aa] 2024-03-28T18:13:22.941 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:13:22 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[70779]: 10: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0x733) [0x55839162c233] 2024-03-28T18:13:22.942 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:13:22 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[70779]: 11: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x55839162e197] 2024-03-28T18:13:22.942 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:13:22 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[70779]: 12: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x55839152d11b] 2024-03-28T18:13:22.942 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:13:22 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[70779]: 13: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x55839152d6bc] 2024-03-28T18:13:22.942 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:13:22 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[70779]: 14: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bf) [0x5583915171ff] 2024-03-28T18:13:22.942 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:13:22 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[70779]: 15: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x478) [0x7fe4e5842488] 2024-03-28T18:13:22.942 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:13:22 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[70779]: 16: (DispatchQueue::entry()+0x51f) [0x7fe4e583f60f] 2024-03-28T18:13:22.942 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:13:22 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[70779]: 17: (DispatchQueue::DispatchThread::entry()+0x11) [0x7fe4e5912e81] 2024-03-28T18:13:22.942 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:13:22 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[70779]: 18: /lib64/libpthread.so.0(+0x81ca) [0x7fe4e432d1ca] 2024-03-28T18:13:22.942 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:13:22 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[70779]: 19: clone() 2024-03-28T18:13:22.942 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:13:22 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[70779]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-03-28T18:13:22.942 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:13:22 smithi134 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f[70779]: 2024-03-28T18:13:22.942 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:13:22 smithi134 podman[70820]: 2024-03-28 18:13:22.721690958 +0000 UTC m=+0.034030714 container died 647fd1aee601c2a626810f4519c48ece29371a7bbd06a71a03284376fc837cb6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, ceph=True, org.label-schema.license=GPLv2, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240319) 2024-03-28T18:13:23.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:13:23 smithi022 ceph-mon[31985]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-03-28T18:13:23.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:13:23 smithi022 ceph-mon[31985]: Cluster is now healthy 2024-03-28T18:13:23.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:13:23 smithi022 ceph-mon[31985]: mds.? [v2:172.21.15.134:6834/2867168769,v1:172.21.15.134:6835/2867168769] up:active 2024-03-28T18:13:23.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:13:23 smithi022 ceph-mon[31985]: fsmap cephfs:5 4 up:standby-replay 2 up:standby 5 up:active 2024-03-28T18:13:23.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:13:23 smithi022 ceph-mon[31985]: fsmap cephfs:5 5 up:standby-replay 1 up:standby 5 up:active 2024-03-28T18:13:23.925 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:13:23 smithi134 ceph-mon[35037]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-03-28T18:13:23.925 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:13:23 smithi134 ceph-mon[35037]: Cluster is now healthy 2024-03-28T18:13:23.925 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:13:23 smithi134 ceph-mon[35037]: mds.? [v2:172.21.15.134:6834/2867168769,v1:172.21.15.134:6835/2867168769] up:active 2024-03-28T18:13:23.925 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:13:23 smithi134 ceph-mon[35037]: fsmap cephfs:5 4 up:standby-replay 2 up:standby 5 up:active 2024-03-28T18:13:23.925 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:13:23 smithi134 ceph-mon[35037]: fsmap cephfs:5 5 up:standby-replay 1 up:standby 5 up:active 2024-03-28T18:13:23.925 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:13:23 smithi134 podman[70820]: 2024-03-28 18:13:23.535663047 +0000 UTC m=+0.848002847 container remove 647fd1aee601c2a626810f4519c48ece29371a7bbd06a71a03284376fc837cb6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-f, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_CLEAN=True, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.build-date=20240319, org.label-schema.vendor=CentOS, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30) 2024-03-28T18:13:23.925 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:13:23 smithi134 systemd[1]: ceph-fd220748-ed2b-11ee-95d0-87774f69a715@mds.f.service: Main process exited, code=exited, status=134/n/a 2024-03-28T18:13:23.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:13:23 smithi079 ceph-mon[35312]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-03-28T18:13:23.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:13:23 smithi079 ceph-mon[35312]: Cluster is now healthy 2024-03-28T18:13:23.933 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:13:23 smithi079 ceph-mon[35312]: mds.? [v2:172.21.15.134:6834/2867168769,v1:172.21.15.134:6835/2867168769] up:active 2024-03-28T18:13:23.933 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:13:23 smithi079 ceph-mon[35312]: fsmap cephfs:5 4 up:standby-replay 2 up:standby 5 up:active 2024-03-28T18:13:23.933 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:13:23 smithi079 ceph-mon[35312]: fsmap cephfs:5 5 up:standby-replay 1 up:standby 5 up:active 2024-03-28T18:13:24.385 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:13:24 smithi134 systemd[1]: ceph-fd220748-ed2b-11ee-95d0-87774f69a715@mds.f.service: Failed with result 'exit-code'. 2024-03-28T18:13:24.385 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:13:24 smithi134 systemd[1]: ceph-fd220748-ed2b-11ee-95d0-87774f69a715@mds.f.service: Consumed 1.878s CPU time. 2024-03-28T18:13:24.433 INFO:journalctl@ceph.mds.b.smithi079.stdout:Mar 28 18:13:24 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-b[65245]: 2024-03-28T18:13:24.091+0000 7f845f400700 -1 mds.pinger is_rank_lagging: rank=1 was never sent ping request. 2024-03-28T18:13:24.674 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:13:24 smithi134 ceph-mon[35037]: pgmap v189: 97 pgs: 97 active+clean; 117 MiB data, 785 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 10 op/s 2024-03-28T18:13:24.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:13:24 smithi022 ceph-mon[31985]: pgmap v189: 97 pgs: 97 active+clean; 117 MiB data, 785 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 10 op/s 2024-03-28T18:13:24.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:13:24 smithi079 ceph-mon[35312]: pgmap v189: 97 pgs: 97 active+clean; 117 MiB data, 785 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 10 op/s 2024-03-28T18:13:26.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:13:26 smithi134 ceph-mon[35037]: pgmap v190: 97 pgs: 97 active+clean; 117 MiB data, 786 MiB used, 1.0 TiB / 1.0 TiB avail; 108 KiB/s rd, 3.8 KiB/s wr, 25 op/s 2024-03-28T18:13:26.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:13:26 smithi079 ceph-mon[35312]: pgmap v190: 97 pgs: 97 active+clean; 117 MiB data, 786 MiB used, 1.0 TiB / 1.0 TiB avail; 108 KiB/s rd, 3.8 KiB/s wr, 25 op/s 2024-03-28T18:13:27.031 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:13:26 smithi022 ceph-mon[31985]: pgmap v190: 97 pgs: 97 active+clean; 117 MiB data, 786 MiB used, 1.0 TiB / 1.0 TiB avail; 108 KiB/s rd, 3.8 KiB/s wr, 25 op/s 2024-03-28T18:13:28.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:13:28 smithi134 ceph-mon[35037]: pgmap v191: 97 pgs: 97 active+clean; 117 MiB data, 786 MiB used, 1.0 TiB / 1.0 TiB avail; 108 KiB/s rd, 3.8 KiB/s wr, 25 op/s 2024-03-28T18:13:28.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:13:28 smithi079 ceph-mon[35312]: pgmap v191: 97 pgs: 97 active+clean; 117 MiB data, 786 MiB used, 1.0 TiB / 1.0 TiB avail; 108 KiB/s rd, 3.8 KiB/s wr, 25 op/s 2024-03-28T18:13:29.032 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:13:28 smithi022 ceph-mon[31985]: pgmap v191: 97 pgs: 97 active+clean; 117 MiB data, 786 MiB used, 1.0 TiB / 1.0 TiB avail; 108 KiB/s rd, 3.8 KiB/s wr, 25 op/s 2024-03-28T18:13:30.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:13:30 smithi134 ceph-mon[35037]: pgmap v192: 97 pgs: 97 active+clean; 117 MiB data, 786 MiB used, 1.0 TiB / 1.0 TiB avail; 98 KiB/s rd, 3.4 KiB/s wr, 27 op/s 2024-03-28T18:13:30.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:13:30 smithi079 ceph-mon[35312]: pgmap v192: 97 pgs: 97 active+clean; 117 MiB data, 786 MiB used, 1.0 TiB / 1.0 TiB avail; 98 KiB/s rd, 3.4 KiB/s wr, 27 op/s 2024-03-28T18:13:31.034 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:13:30 smithi022 ceph-mon[31985]: pgmap v192: 97 pgs: 97 active+clean; 117 MiB data, 786 MiB used, 1.0 TiB / 1.0 TiB avail; 98 KiB/s rd, 3.4 KiB/s wr, 27 op/s 2024-03-28T18:13:32.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:13:32 smithi134 ceph-mon[35037]: pgmap v193: 97 pgs: 97 active+clean; 117 MiB data, 786 MiB used, 1.0 TiB / 1.0 TiB avail; 89 KiB/s rd, 3.2 KiB/s wr, 21 op/s 2024-03-28T18:13:32.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:13:32 smithi079 ceph-mon[35312]: pgmap v193: 97 pgs: 97 active+clean; 117 MiB data, 786 MiB used, 1.0 TiB / 1.0 TiB avail; 89 KiB/s rd, 3.2 KiB/s wr, 21 op/s 2024-03-28T18:13:33.032 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:13:32 smithi022 ceph-mon[31985]: pgmap v193: 97 pgs: 97 active+clean; 117 MiB data, 786 MiB used, 1.0 TiB / 1.0 TiB avail; 89 KiB/s rd, 3.2 KiB/s wr, 21 op/s 2024-03-28T18:13:34.561 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:13:34 smithi134 systemd[1]: ceph-fd220748-ed2b-11ee-95d0-87774f69a715@mds.f.service: Scheduled restart job, restart counter is at 6. 2024-03-28T18:13:34.561 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:13:34 smithi134 systemd[1]: Stopped Ceph mds.f for fd220748-ed2b-11ee-95d0-87774f69a715. 2024-03-28T18:13:34.561 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:13:34 smithi134 systemd[1]: ceph-fd220748-ed2b-11ee-95d0-87774f69a715@mds.f.service: Consumed 1.878s CPU time. 2024-03-28T18:13:34.561 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:13:34 smithi134 systemd[1]: ceph-fd220748-ed2b-11ee-95d0-87774f69a715@mds.f.service: Start request repeated too quickly. 2024-03-28T18:13:34.561 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:13:34 smithi134 systemd[1]: ceph-fd220748-ed2b-11ee-95d0-87774f69a715@mds.f.service: Failed with result 'exit-code'. 2024-03-28T18:13:34.561 INFO:journalctl@ceph.mds.f.smithi134.stdout:Mar 28 18:13:34 smithi134 systemd[1]: Failed to start Ceph mds.f for fd220748-ed2b-11ee-95d0-87774f69a715. 2024-03-28T18:13:34.925 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:13:34 smithi134 ceph-mon[35037]: pgmap v194: 97 pgs: 97 active+clean; 117 MiB data, 786 MiB used, 1.0 TiB / 1.0 TiB avail; 89 KiB/s rd, 3.2 KiB/s wr, 21 op/s 2024-03-28T18:13:34.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:13:34 smithi079 ceph-mon[35312]: pgmap v194: 97 pgs: 97 active+clean; 117 MiB data, 786 MiB used, 1.0 TiB / 1.0 TiB avail; 89 KiB/s rd, 3.2 KiB/s wr, 21 op/s 2024-03-28T18:13:35.032 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:13:34 smithi022 ceph-mon[31985]: pgmap v194: 97 pgs: 97 active+clean; 117 MiB data, 786 MiB used, 1.0 TiB / 1.0 TiB avail; 89 KiB/s rd, 3.2 KiB/s wr, 21 op/s 2024-03-28T18:13:36.925 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:13:36 smithi134 ceph-mon[35037]: pgmap v195: 97 pgs: 97 active+clean; 117 MiB data, 786 MiB used, 1.0 TiB / 1.0 TiB avail; 91 KiB/s rd, 3.2 KiB/s wr, 25 op/s 2024-03-28T18:13:36.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:13:36 smithi079 ceph-mon[35312]: pgmap v195: 97 pgs: 97 active+clean; 117 MiB data, 786 MiB used, 1.0 TiB / 1.0 TiB avail; 91 KiB/s rd, 3.2 KiB/s wr, 25 op/s 2024-03-28T18:13:37.032 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:13:36 smithi022 ceph-mon[31985]: pgmap v195: 97 pgs: 97 active+clean; 117 MiB data, 786 MiB used, 1.0 TiB / 1.0 TiB avail; 91 KiB/s rd, 3.2 KiB/s wr, 25 op/s 2024-03-28T18:13:38.925 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:13:38 smithi134 ceph-mon[35037]: pgmap v196: 97 pgs: 97 active+clean; 117 MiB data, 786 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-03-28T18:13:38.925 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:13:38 smithi134 ceph-mon[35037]: Replacing daemon mds.f as rank 1 with standby daemon mds.d 2024-03-28T18:13:38.925 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:13:38 smithi134 ceph-mon[35037]: MDS daemon mds.f is removed because it is dead or otherwise unavailable. 2024-03-28T18:13:38.925 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:13:38 smithi134 ceph-mon[35037]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-03-28T18:13:38.925 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:13:38 smithi134 ceph-mon[35037]: osdmap e105: 12 total, 12 up, 12 in 2024-03-28T18:13:38.925 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:13:38 smithi134 ceph-mon[35037]: fsmap cephfs:5/5 4 up:standby-replay 1 up:standby 1 up:replay 4 up:active 2024-03-28T18:13:38.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:13:38 smithi079 ceph-mon[35312]: pgmap v196: 97 pgs: 97 active+clean; 117 MiB data, 786 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-03-28T18:13:38.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:13:38 smithi079 ceph-mon[35312]: Replacing daemon mds.f as rank 1 with standby daemon mds.d 2024-03-28T18:13:38.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:13:38 smithi079 ceph-mon[35312]: MDS daemon mds.f is removed because it is dead or otherwise unavailable. 2024-03-28T18:13:38.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:13:38 smithi079 ceph-mon[35312]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-03-28T18:13:38.933 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:13:38 smithi079 ceph-mon[35312]: osdmap e105: 12 total, 12 up, 12 in 2024-03-28T18:13:38.933 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:13:38 smithi079 ceph-mon[35312]: fsmap cephfs:5/5 4 up:standby-replay 1 up:standby 1 up:replay 4 up:active 2024-03-28T18:13:39.032 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:13:38 smithi022 ceph-mon[31985]: pgmap v196: 97 pgs: 97 active+clean; 117 MiB data, 786 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-03-28T18:13:39.032 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:13:38 smithi022 ceph-mon[31985]: Replacing daemon mds.f as rank 1 with standby daemon mds.d 2024-03-28T18:13:39.032 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:13:38 smithi022 ceph-mon[31985]: MDS daemon mds.f is removed because it is dead or otherwise unavailable. 2024-03-28T18:13:39.032 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:13:38 smithi022 ceph-mon[31985]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-03-28T18:13:39.032 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:13:38 smithi022 ceph-mon[31985]: osdmap e105: 12 total, 12 up, 12 in 2024-03-28T18:13:39.032 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:13:38 smithi022 ceph-mon[31985]: fsmap cephfs:5/5 4 up:standby-replay 1 up:standby 1 up:replay 4 up:active 2024-03-28T18:13:40.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:13:40 smithi134 ceph-mon[35037]: pgmap v198: 97 pgs: 97 active+clean; 117 MiB data, 786 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 10 op/s 2024-03-28T18:13:40.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:13:40 smithi134 ceph-mon[35037]: mds.? [v2:172.21.15.22:6836/3676939386,v1:172.21.15.22:6837/3676939386] up:resolve 2024-03-28T18:13:40.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:13:40 smithi134 ceph-mon[35037]: fsmap cephfs:5/5 4 up:standby-replay 1 up:standby 1 up:resolve 4 up:active 2024-03-28T18:13:40.933 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:13:40 smithi079 ceph-mon[35312]: pgmap v198: 97 pgs: 97 active+clean; 117 MiB data, 786 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 10 op/s 2024-03-28T18:13:40.933 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:13:40 smithi079 ceph-mon[35312]: mds.? [v2:172.21.15.22:6836/3676939386,v1:172.21.15.22:6837/3676939386] up:resolve 2024-03-28T18:13:40.933 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:13:40 smithi079 ceph-mon[35312]: fsmap cephfs:5/5 4 up:standby-replay 1 up:standby 1 up:resolve 4 up:active 2024-03-28T18:13:41.031 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:13:40 smithi022 ceph-mon[31985]: pgmap v198: 97 pgs: 97 active+clean; 117 MiB data, 786 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 10 op/s 2024-03-28T18:13:41.032 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:13:40 smithi022 ceph-mon[31985]: mds.? [v2:172.21.15.22:6836/3676939386,v1:172.21.15.22:6837/3676939386] up:resolve 2024-03-28T18:13:41.032 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:13:40 smithi022 ceph-mon[31985]: fsmap cephfs:5/5 4 up:standby-replay 1 up:standby 1 up:resolve 4 up:active 2024-03-28T18:13:41.925 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:13:41 smithi134 ceph-mon[35037]: reconnect by client.35222 172.21.15.22:0/3729116781 after 0.00499993 2024-03-28T18:13:41.925 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:13:41 smithi134 ceph-mon[35037]: mds.? [v2:172.21.15.22:6836/3676939386,v1:172.21.15.22:6837/3676939386] up:reconnect 2024-03-28T18:13:41.925 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:13:41 smithi134 ceph-mon[35037]: fsmap cephfs:5/5 4 up:standby-replay 1 up:standby 1 up:reconnect 4 up:active 2024-03-28T18:13:41.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:13:41 smithi079 ceph-mon[35312]: reconnect by client.35222 172.21.15.22:0/3729116781 after 0.00499993 2024-03-28T18:13:41.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:13:41 smithi079 ceph-mon[35312]: mds.? [v2:172.21.15.22:6836/3676939386,v1:172.21.15.22:6837/3676939386] up:reconnect 2024-03-28T18:13:41.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:13:41 smithi079 ceph-mon[35312]: fsmap cephfs:5/5 4 up:standby-replay 1 up:standby 1 up:reconnect 4 up:active 2024-03-28T18:13:42.032 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:13:41 smithi022 ceph-mon[31985]: reconnect by client.35222 172.21.15.22:0/3729116781 after 0.00499993 2024-03-28T18:13:42.032 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:13:41 smithi022 ceph-mon[31985]: mds.? [v2:172.21.15.22:6836/3676939386,v1:172.21.15.22:6837/3676939386] up:reconnect 2024-03-28T18:13:42.032 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:13:41 smithi022 ceph-mon[31985]: fsmap cephfs:5/5 4 up:standby-replay 1 up:standby 1 up:reconnect 4 up:active 2024-03-28T18:13:42.925 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:13:42 smithi134 ceph-mon[35037]: pgmap v199: 97 pgs: 97 active+clean; 117 MiB data, 786 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 10 op/s 2024-03-28T18:13:42.925 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:13:42 smithi134 ceph-mon[35037]: mds.? [v2:172.21.15.22:6836/3676939386,v1:172.21.15.22:6837/3676939386] up:rejoin 2024-03-28T18:13:42.925 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:13:42 smithi134 ceph-mon[35037]: fsmap cephfs:5/5 4 up:standby-replay 1 up:standby 1 up:rejoin 4 up:active 2024-03-28T18:13:42.925 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:13:42 smithi134 ceph-mon[35037]: daemon mds.d is now active in filesystem cephfs as rank 1 2024-03-28T18:13:42.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:13:42 smithi079 ceph-mon[35312]: pgmap v199: 97 pgs: 97 active+clean; 117 MiB data, 786 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 10 op/s 2024-03-28T18:13:42.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:13:42 smithi079 ceph-mon[35312]: mds.? [v2:172.21.15.22:6836/3676939386,v1:172.21.15.22:6837/3676939386] up:rejoin 2024-03-28T18:13:42.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:13:42 smithi079 ceph-mon[35312]: fsmap cephfs:5/5 4 up:standby-replay 1 up:standby 1 up:rejoin 4 up:active 2024-03-28T18:13:42.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:13:42 smithi079 ceph-mon[35312]: daemon mds.d is now active in filesystem cephfs as rank 1 2024-03-28T18:13:43.032 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:13:42 smithi022 ceph-mon[31985]: pgmap v199: 97 pgs: 97 active+clean; 117 MiB data, 786 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 10 op/s 2024-03-28T18:13:43.032 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:13:42 smithi022 ceph-mon[31985]: mds.? [v2:172.21.15.22:6836/3676939386,v1:172.21.15.22:6837/3676939386] up:rejoin 2024-03-28T18:13:43.032 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:13:42 smithi022 ceph-mon[31985]: fsmap cephfs:5/5 4 up:standby-replay 1 up:standby 1 up:rejoin 4 up:active 2024-03-28T18:13:43.032 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:13:42 smithi022 ceph-mon[31985]: daemon mds.d is now active in filesystem cephfs as rank 1 2024-03-28T18:13:43.033 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:13:42 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[73373]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-2145-g3be9a5c9/rpm/el8/BUILD/ceph-19.0.0-2145-g3be9a5c9/src/mds/Locker.cc: In function 'void Locker::set_xlocks_done(MutationImpl*, bool)' thread 7fa164c00700 time 2024-03-28T18:13:42.698406+0000 2024-03-28T18:13:43.033 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:13:42 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[73373]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-2145-g3be9a5c9/rpm/el8/BUILD/ceph-19.0.0-2145-g3be9a5c9/src/mds/Locker.cc: 626: FAILED ceph_assert(obj->is_auth()) 2024-03-28T18:13:43.033 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:13:42 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[73373]: ceph version 19.0.0-2145-g3be9a5c9 (3be9a5c9bc793e11e2800a8c0c696e8b46742033) squid (dev) 2024-03-28T18:13:43.034 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:13:42 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[73373]: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x135) [0x7fa1705aa4a1] 2024-03-28T18:13:43.034 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:13:42 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[73373]: 2: /usr/lib64/ceph/libceph-common.so.2(+0x2d2667) [0x7fa1705aa667] 2024-03-28T18:13:43.034 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:13:42 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[73373]: 3: (Locker::set_xlocks_done(MutationImpl*, bool)+0x493) [0x55a7259cd223] 2024-03-28T18:13:43.034 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:13:42 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[73373]: 4: (Server::early_reply(boost::intrusive_ptr const&, CInode*, CDentry*)+0x3e6) [0x55a7258475d6] 2024-03-28T18:13:43.034 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:13:42 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[73373]: 5: (Server::journal_and_reply(boost::intrusive_ptr const&, CInode*, CDentry*, LogEvent*, MDSLogContextBase*)+0xc1) [0x55a725847da1] 2024-03-28T18:13:43.034 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:13:42 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[73373]: 6: (Server::handle_client_mkdir(boost::intrusive_ptr const&)+0x5ba) [0x55a72587d6aa] 2024-03-28T18:13:43.034 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:13:42 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[73373]: 7: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0x733) [0x55a72589a233] 2024-03-28T18:13:43.034 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:13:42 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[73373]: 8: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x55a72589c197] 2024-03-28T18:13:43.034 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:13:42 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[73373]: 9: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x55a72579b11b] 2024-03-28T18:13:43.034 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:13:42 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[73373]: 10: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x55a72579b6bc] 2024-03-28T18:13:43.034 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:13:42 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[73373]: 11: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bf) [0x55a7257851ff] 2024-03-28T18:13:43.034 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:13:42 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[73373]: 12: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x478) [0x7fa17083c488] 2024-03-28T18:13:43.034 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:13:42 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[73373]: 13: (DispatchQueue::entry()+0x51f) [0x7fa17083960f] 2024-03-28T18:13:43.034 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:13:42 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[73373]: 14: (DispatchQueue::DispatchThread::entry()+0x11) [0x7fa17090ce81] 2024-03-28T18:13:43.035 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:13:42 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[73373]: 15: /lib64/libpthread.so.0(+0x81ca) [0x7fa16f3271ca] 2024-03-28T18:13:43.035 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:13:42 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[73373]: 16: clone() 2024-03-28T18:13:43.035 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:13:42 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[73373]: 2024-03-28T18:13:42.700+0000 7fa164c00700 -1 /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-2145-g3be9a5c9/rpm/el8/BUILD/ceph-19.0.0-2145-g3be9a5c9/src/mds/Locker.cc: In function 'void Locker::set_xlocks_done(MutationImpl*, bool)' thread 7fa164c00700 time 2024-03-28T18:13:42.698406+0000 2024-03-28T18:13:43.035 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:13:42 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[73373]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-2145-g3be9a5c9/rpm/el8/BUILD/ceph-19.0.0-2145-g3be9a5c9/src/mds/Locker.cc: 626: FAILED ceph_assert(obj->is_auth()) 2024-03-28T18:13:43.035 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:13:42 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[73373]: 2024-03-28T18:13:43.035 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:13:42 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[73373]: ceph version 19.0.0-2145-g3be9a5c9 (3be9a5c9bc793e11e2800a8c0c696e8b46742033) squid (dev) 2024-03-28T18:13:43.035 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:13:42 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[73373]: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x135) [0x7fa1705aa4a1] 2024-03-28T18:13:43.035 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:13:42 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[73373]: 2: /usr/lib64/ceph/libceph-common.so.2(+0x2d2667) [0x7fa1705aa667] 2024-03-28T18:13:43.035 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:13:42 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[73373]: 3: (Locker::set_xlocks_done(MutationImpl*, bool)+0x493) [0x55a7259cd223] 2024-03-28T18:13:43.035 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:13:42 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[73373]: 4: (Server::early_reply(boost::intrusive_ptr const&, CInode*, CDentry*)+0x3e6) [0x55a7258475d6] 2024-03-28T18:13:43.036 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:13:42 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[73373]: 5: (Server::journal_and_reply(boost::intrusive_ptr const&, CInode*, CDentry*, LogEvent*, MDSLogContextBase*)+0xc1) [0x55a725847da1] 2024-03-28T18:13:43.036 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:13:42 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[73373]: 6: (Server::handle_client_mkdir(boost::intrusive_ptr const&)+0x5ba) [0x55a72587d6aa] 2024-03-28T18:13:43.036 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:13:42 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[73373]: 7: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0x733) [0x55a72589a233] 2024-03-28T18:13:43.036 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:13:42 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[73373]: 8: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x55a72589c197] 2024-03-28T18:13:43.036 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:13:42 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[73373]: 9: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x55a72579b11b] 2024-03-28T18:13:43.036 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:13:42 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[73373]: 10: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x55a72579b6bc] 2024-03-28T18:13:43.036 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:13:42 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[73373]: 11: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bf) [0x55a7257851ff] 2024-03-28T18:13:43.036 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:13:42 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[73373]: 12: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x478) [0x7fa17083c488] 2024-03-28T18:13:43.036 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:13:42 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[73373]: 13: (DispatchQueue::entry()+0x51f) [0x7fa17083960f] 2024-03-28T18:13:43.036 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:13:42 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[73373]: 14: (DispatchQueue::DispatchThread::entry()+0x11) [0x7fa17090ce81] 2024-03-28T18:13:43.036 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:13:42 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[73373]: 15: /lib64/libpthread.so.0(+0x81ca) [0x7fa16f3271ca] 2024-03-28T18:13:43.036 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:13:42 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[73373]: 16: clone() 2024-03-28T18:13:43.036 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:13:42 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[73373]: 2024-03-28T18:13:43.036 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:13:42 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[73373]: *** Caught signal (Aborted) ** 2024-03-28T18:13:43.036 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:13:42 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[73373]: in thread 7fa164c00700 thread_name:ms_dispatch 2024-03-28T18:13:43.037 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:13:42 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[73373]: ceph version 19.0.0-2145-g3be9a5c9 (3be9a5c9bc793e11e2800a8c0c696e8b46742033) squid (dev) 2024-03-28T18:13:43.037 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:13:42 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[73373]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7fa16f331d20] 2024-03-28T18:13:43.037 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:13:42 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[73373]: 2: gsignal() 2024-03-28T18:13:43.037 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:13:42 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[73373]: 3: abort() 2024-03-28T18:13:43.037 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:13:42 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[73373]: 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x18f) [0x7fa1705aa4fb] 2024-03-28T18:13:43.037 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:13:42 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[73373]: 5: /usr/lib64/ceph/libceph-common.so.2(+0x2d2667) [0x7fa1705aa667] 2024-03-28T18:13:43.037 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:13:42 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[73373]: 6: (Locker::set_xlocks_done(MutationImpl*, bool)+0x493) [0x55a7259cd223] 2024-03-28T18:13:43.037 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:13:42 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[73373]: 7: (Server::early_reply(boost::intrusive_ptr const&, CInode*, CDentry*)+0x3e6) [0x55a7258475d6] 2024-03-28T18:13:43.037 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:13:42 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[73373]: 8: (Server::journal_and_reply(boost::intrusive_ptr const&, CInode*, CDentry*, LogEvent*, MDSLogContextBase*)+0xc1) [0x55a725847da1] 2024-03-28T18:13:43.037 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:13:42 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[73373]: 9: (Server::handle_client_mkdir(boost::intrusive_ptr const&)+0x5ba) [0x55a72587d6aa] 2024-03-28T18:13:43.038 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:13:42 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[73373]: 10: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0x733) [0x55a72589a233] 2024-03-28T18:13:43.038 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:13:42 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[73373]: 11: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x55a72589c197] 2024-03-28T18:13:43.038 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:13:42 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[73373]: 12: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x55a72579b11b] 2024-03-28T18:13:43.038 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:13:42 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[73373]: 13: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x55a72579b6bc] 2024-03-28T18:13:43.038 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:13:42 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[73373]: 14: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bf) [0x55a7257851ff] 2024-03-28T18:13:43.038 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:13:42 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[73373]: 15: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x478) [0x7fa17083c488] 2024-03-28T18:13:43.038 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:13:42 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[73373]: 16: (DispatchQueue::entry()+0x51f) [0x7fa17083960f] 2024-03-28T18:13:43.038 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:13:42 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[73373]: 17: (DispatchQueue::DispatchThread::entry()+0x11) [0x7fa17090ce81] 2024-03-28T18:13:43.038 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:13:42 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[73373]: 18: /lib64/libpthread.so.0(+0x81ca) [0x7fa16f3271ca] 2024-03-28T18:13:43.038 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:13:42 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[73373]: 19: clone() 2024-03-28T18:13:43.038 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:13:42 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[73373]: 2024-03-28T18:13:42.703+0000 7fa164c00700 -1 *** Caught signal (Aborted) ** 2024-03-28T18:13:43.038 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:13:42 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[73373]: in thread 7fa164c00700 thread_name:ms_dispatch 2024-03-28T18:13:43.038 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:13:42 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[73373]: 2024-03-28T18:13:43.038 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:13:42 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[73373]: ceph version 19.0.0-2145-g3be9a5c9 (3be9a5c9bc793e11e2800a8c0c696e8b46742033) squid (dev) 2024-03-28T18:13:43.038 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:13:42 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[73373]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7fa16f331d20] 2024-03-28T18:13:43.039 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:13:42 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[73373]: 2: gsignal() 2024-03-28T18:13:43.039 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:13:42 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[73373]: 3: abort() 2024-03-28T18:13:43.039 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:13:42 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[73373]: 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x18f) [0x7fa1705aa4fb] 2024-03-28T18:13:43.039 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:13:42 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[73373]: 5: /usr/lib64/ceph/libceph-common.so.2(+0x2d2667) [0x7fa1705aa667] 2024-03-28T18:13:43.039 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:13:42 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[73373]: 6: (Locker::set_xlocks_done(MutationImpl*, bool)+0x493) [0x55a7259cd223] 2024-03-28T18:13:43.039 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:13:42 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[73373]: 7: (Server::early_reply(boost::intrusive_ptr const&, CInode*, CDentry*)+0x3e6) [0x55a7258475d6] 2024-03-28T18:13:43.039 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:13:42 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[73373]: 8: (Server::journal_and_reply(boost::intrusive_ptr const&, CInode*, CDentry*, LogEvent*, MDSLogContextBase*)+0xc1) [0x55a725847da1] 2024-03-28T18:13:43.039 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:13:42 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[73373]: 9: (Server::handle_client_mkdir(boost::intrusive_ptr const&)+0x5ba) [0x55a72587d6aa] 2024-03-28T18:13:43.039 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:13:42 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[73373]: 10: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0x733) [0x55a72589a233] 2024-03-28T18:13:43.040 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:13:42 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[73373]: 11: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x55a72589c197] 2024-03-28T18:13:43.040 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:13:42 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[73373]: 12: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x55a72579b11b] 2024-03-28T18:13:43.040 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:13:42 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[73373]: 13: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x55a72579b6bc] 2024-03-28T18:13:43.040 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:13:42 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[73373]: 14: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bf) [0x55a7257851ff] 2024-03-28T18:13:43.040 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:13:42 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[73373]: 15: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x478) [0x7fa17083c488] 2024-03-28T18:13:43.040 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:13:42 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[73373]: 16: (DispatchQueue::entry()+0x51f) [0x7fa17083960f] 2024-03-28T18:13:43.040 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:13:42 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[73373]: 17: (DispatchQueue::DispatchThread::entry()+0x11) [0x7fa17090ce81] 2024-03-28T18:13:43.040 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:13:42 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[73373]: 18: /lib64/libpthread.so.0(+0x81ca) [0x7fa16f3271ca] 2024-03-28T18:13:43.040 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:13:42 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[73373]: 19: clone() 2024-03-28T18:13:43.040 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:13:42 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[73373]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-03-28T18:13:43.040 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:13:42 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[73373]: 2024-03-28T18:13:43.040 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:13:42 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[73373]: -1> 2024-03-28T18:13:42.700+0000 7fa164c00700 -1 /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-2145-g3be9a5c9/rpm/el8/BUILD/ceph-19.0.0-2145-g3be9a5c9/src/mds/Locker.cc: In function 'void Locker::set_xlocks_done(MutationImpl*, bool)' thread 7fa164c00700 time 2024-03-28T18:13:42.698406+0000 2024-03-28T18:13:43.040 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:13:42 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[73373]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-2145-g3be9a5c9/rpm/el8/BUILD/ceph-19.0.0-2145-g3be9a5c9/src/mds/Locker.cc: 626: FAILED ceph_assert(obj->is_auth()) 2024-03-28T18:13:43.040 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:13:42 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[73373]: 2024-03-28T18:13:43.040 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:13:42 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[73373]: ceph version 19.0.0-2145-g3be9a5c9 (3be9a5c9bc793e11e2800a8c0c696e8b46742033) squid (dev) 2024-03-28T18:13:43.041 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:13:42 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[73373]: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x135) [0x7fa1705aa4a1] 2024-03-28T18:13:43.041 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:13:42 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[73373]: 2: /usr/lib64/ceph/libceph-common.so.2(+0x2d2667) [0x7fa1705aa667] 2024-03-28T18:13:43.041 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:13:42 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[73373]: 3: (Locker::set_xlocks_done(MutationImpl*, bool)+0x493) [0x55a7259cd223] 2024-03-28T18:13:43.041 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:13:42 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[73373]: 4: (Server::early_reply(boost::intrusive_ptr const&, CInode*, CDentry*)+0x3e6) [0x55a7258475d6] 2024-03-28T18:13:43.041 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:13:42 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[73373]: 5: (Server::journal_and_reply(boost::intrusive_ptr const&, CInode*, CDentry*, LogEvent*, MDSLogContextBase*)+0xc1) [0x55a725847da1] 2024-03-28T18:13:43.041 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:13:42 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[73373]: 6: (Server::handle_client_mkdir(boost::intrusive_ptr const&)+0x5ba) [0x55a72587d6aa] 2024-03-28T18:13:43.041 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:13:42 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[73373]: 7: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0x733) [0x55a72589a233] 2024-03-28T18:13:43.041 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:13:42 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[73373]: 8: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x55a72589c197] 2024-03-28T18:13:43.041 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:13:42 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[73373]: 9: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x55a72579b11b] 2024-03-28T18:13:43.041 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:13:42 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[73373]: 10: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x55a72579b6bc] 2024-03-28T18:13:43.042 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:13:42 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[73373]: 11: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bf) [0x55a7257851ff] 2024-03-28T18:13:43.042 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:13:42 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[73373]: 12: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x478) [0x7fa17083c488] 2024-03-28T18:13:43.042 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:13:42 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[73373]: 13: (DispatchQueue::entry()+0x51f) [0x7fa17083960f] 2024-03-28T18:13:43.042 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:13:42 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[73373]: 14: (DispatchQueue::DispatchThread::entry()+0x11) [0x7fa17090ce81] 2024-03-28T18:13:43.042 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:13:42 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[73373]: 15: /lib64/libpthread.so.0(+0x81ca) [0x7fa16f3271ca] 2024-03-28T18:13:43.042 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:13:42 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[73373]: 16: clone() 2024-03-28T18:13:43.042 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:13:42 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[73373]: 2024-03-28T18:13:43.042 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:13:42 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[73373]: 0> 2024-03-28T18:13:42.703+0000 7fa164c00700 -1 *** Caught signal (Aborted) ** 2024-03-28T18:13:43.042 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:13:42 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[73373]: in thread 7fa164c00700 thread_name:ms_dispatch 2024-03-28T18:13:43.042 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:13:42 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[73373]: 2024-03-28T18:13:43.042 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:13:42 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[73373]: ceph version 19.0.0-2145-g3be9a5c9 (3be9a5c9bc793e11e2800a8c0c696e8b46742033) squid (dev) 2024-03-28T18:13:43.042 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:13:42 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[73373]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7fa16f331d20] 2024-03-28T18:13:43.042 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:13:42 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[73373]: 2: gsignal() 2024-03-28T18:13:43.042 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:13:42 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[73373]: 3: abort() 2024-03-28T18:13:43.043 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:13:42 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[73373]: 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x18f) [0x7fa1705aa4fb] 2024-03-28T18:13:43.043 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:13:42 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[73373]: 5: /usr/lib64/ceph/libceph-common.so.2(+0x2d2667) [0x7fa1705aa667] 2024-03-28T18:13:43.043 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:13:42 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[73373]: 6: (Locker::set_xlocks_done(MutationImpl*, bool)+0x493) [0x55a7259cd223] 2024-03-28T18:13:43.043 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:13:42 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[73373]: 7: (Server::early_reply(boost::intrusive_ptr const&, CInode*, CDentry*)+0x3e6) [0x55a7258475d6] 2024-03-28T18:13:43.043 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:13:42 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[73373]: 8: (Server::journal_and_reply(boost::intrusive_ptr const&, CInode*, CDentry*, LogEvent*, MDSLogContextBase*)+0xc1) [0x55a725847da1] 2024-03-28T18:13:43.043 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:13:42 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[73373]: 9: (Server::handle_client_mkdir(boost::intrusive_ptr const&)+0x5ba) [0x55a72587d6aa] 2024-03-28T18:13:43.043 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:13:42 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[73373]: 10: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0x733) [0x55a72589a233] 2024-03-28T18:13:43.043 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:13:42 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[73373]: 11: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x55a72589c197] 2024-03-28T18:13:43.043 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:13:42 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[73373]: 12: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x55a72579b11b] 2024-03-28T18:13:43.043 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:13:42 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[73373]: 13: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x55a72579b6bc] 2024-03-28T18:13:43.044 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:13:42 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[73373]: 14: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bf) [0x55a7257851ff] 2024-03-28T18:13:43.044 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:13:42 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[73373]: 15: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x478) [0x7fa17083c488] 2024-03-28T18:13:43.044 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:13:42 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[73373]: 16: (DispatchQueue::entry()+0x51f) [0x7fa17083960f] 2024-03-28T18:13:43.044 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:13:42 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[73373]: 17: (DispatchQueue::DispatchThread::entry()+0x11) [0x7fa17090ce81] 2024-03-28T18:13:43.044 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:13:42 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[73373]: 18: /lib64/libpthread.so.0(+0x81ca) [0x7fa16f3271ca] 2024-03-28T18:13:43.044 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:13:42 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[73373]: 19: clone() 2024-03-28T18:13:43.044 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:13:42 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[73373]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-03-28T18:13:43.044 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:13:42 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[73373]: 2024-03-28T18:13:43.044 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:13:42 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[73373]: -1> 2024-03-28T18:13:42.700+0000 7fa164c00700 -1 /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-2145-g3be9a5c9/rpm/el8/BUILD/ceph-19.0.0-2145-g3be9a5c9/src/mds/Locker.cc: In function 'void Locker::set_xlocks_done(MutationImpl*, bool)' thread 7fa164c00700 time 2024-03-28T18:13:42.698406+0000 2024-03-28T18:13:43.044 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:13:42 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[73373]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-2145-g3be9a5c9/rpm/el8/BUILD/ceph-19.0.0-2145-g3be9a5c9/src/mds/Locker.cc: 626: FAILED ceph_assert(obj->is_auth()) 2024-03-28T18:13:43.044 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:13:42 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[73373]: 2024-03-28T18:13:43.044 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:13:42 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[73373]: ceph version 19.0.0-2145-g3be9a5c9 (3be9a5c9bc793e11e2800a8c0c696e8b46742033) squid (dev) 2024-03-28T18:13:43.044 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:13:42 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[73373]: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x135) [0x7fa1705aa4a1] 2024-03-28T18:13:43.044 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:13:42 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[73373]: 2: /usr/lib64/ceph/libceph-common.so.2(+0x2d2667) [0x7fa1705aa667] 2024-03-28T18:13:43.044 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:13:42 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[73373]: 3: (Locker::set_xlocks_done(MutationImpl*, bool)+0x493) [0x55a7259cd223] 2024-03-28T18:13:43.045 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:13:42 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[73373]: 4: (Server::early_reply(boost::intrusive_ptr const&, CInode*, CDentry*)+0x3e6) [0x55a7258475d6] 2024-03-28T18:13:43.045 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:13:42 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[73373]: 5: (Server::journal_and_reply(boost::intrusive_ptr const&, CInode*, CDentry*, LogEvent*, MDSLogContextBase*)+0xc1) [0x55a725847da1] 2024-03-28T18:13:43.045 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:13:42 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[73373]: 6: (Server::handle_client_mkdir(boost::intrusive_ptr const&)+0x5ba) [0x55a72587d6aa] 2024-03-28T18:13:43.045 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:13:42 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[73373]: 7: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0x733) [0x55a72589a233] 2024-03-28T18:13:43.045 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:13:42 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[73373]: 8: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x55a72589c197] 2024-03-28T18:13:43.045 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:13:42 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[73373]: 9: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x55a72579b11b] 2024-03-28T18:13:43.045 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:13:42 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[73373]: 10: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x55a72579b6bc] 2024-03-28T18:13:43.046 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:13:42 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[73373]: 11: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bf) [0x55a7257851ff] 2024-03-28T18:13:43.046 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:13:42 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[73373]: 12: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x478) [0x7fa17083c488] 2024-03-28T18:13:43.046 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:13:42 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[73373]: 13: (DispatchQueue::entry()+0x51f) [0x7fa17083960f] 2024-03-28T18:13:43.046 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:13:42 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[73373]: 14: (DispatchQueue::DispatchThread::entry()+0x11) [0x7fa17090ce81] 2024-03-28T18:13:43.046 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:13:42 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[73373]: 15: /lib64/libpthread.so.0(+0x81ca) [0x7fa16f3271ca] 2024-03-28T18:13:43.046 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:13:42 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[73373]: 16: clone() 2024-03-28T18:13:43.046 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:13:42 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[73373]: 2024-03-28T18:13:43.046 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:13:42 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[73373]: 0> 2024-03-28T18:13:42.703+0000 7fa164c00700 -1 *** Caught signal (Aborted) ** 2024-03-28T18:13:43.046 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:13:42 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[73373]: in thread 7fa164c00700 thread_name:ms_dispatch 2024-03-28T18:13:43.046 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:13:42 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[73373]: 2024-03-28T18:13:43.046 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:13:42 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[73373]: ceph version 19.0.0-2145-g3be9a5c9 (3be9a5c9bc793e11e2800a8c0c696e8b46742033) squid (dev) 2024-03-28T18:13:43.046 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:13:42 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[73373]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7fa16f331d20] 2024-03-28T18:13:43.046 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:13:42 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[73373]: 2: gsignal() 2024-03-28T18:13:43.046 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:13:42 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[73373]: 3: abort() 2024-03-28T18:13:43.047 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:13:42 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[73373]: 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x18f) [0x7fa1705aa4fb] 2024-03-28T18:13:43.047 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:13:42 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[73373]: 5: /usr/lib64/ceph/libceph-common.so.2(+0x2d2667) [0x7fa1705aa667] 2024-03-28T18:13:43.047 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:13:42 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[73373]: 6: (Locker::set_xlocks_done(MutationImpl*, bool)+0x493) [0x55a7259cd223] 2024-03-28T18:13:43.047 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:13:42 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[73373]: 7: (Server::early_reply(boost::intrusive_ptr const&, CInode*, CDentry*)+0x3e6) [0x55a7258475d6] 2024-03-28T18:13:43.047 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:13:42 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[73373]: 8: (Server::journal_and_reply(boost::intrusive_ptr const&, CInode*, CDentry*, LogEvent*, MDSLogContextBase*)+0xc1) [0x55a725847da1] 2024-03-28T18:13:43.047 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:13:42 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[73373]: 9: (Server::handle_client_mkdir(boost::intrusive_ptr const&)+0x5ba) [0x55a72587d6aa] 2024-03-28T18:13:43.047 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:13:42 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[73373]: 10: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0x733) [0x55a72589a233] 2024-03-28T18:13:43.047 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:13:42 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[73373]: 11: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x55a72589c197] 2024-03-28T18:13:43.047 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:13:42 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[73373]: 12: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x55a72579b11b] 2024-03-28T18:13:43.047 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:13:42 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[73373]: 13: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x55a72579b6bc] 2024-03-28T18:13:43.047 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:13:42 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[73373]: 14: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bf) [0x55a7257851ff] 2024-03-28T18:13:43.047 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:13:42 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[73373]: 15: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x478) [0x7fa17083c488] 2024-03-28T18:13:43.047 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:13:42 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[73373]: 16: (DispatchQueue::entry()+0x51f) [0x7fa17083960f] 2024-03-28T18:13:43.047 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:13:42 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[73373]: 17: (DispatchQueue::DispatchThread::entry()+0x11) [0x7fa17090ce81] 2024-03-28T18:13:43.047 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:13:42 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[73373]: 18: /lib64/libpthread.so.0(+0x81ca) [0x7fa16f3271ca] 2024-03-28T18:13:43.048 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:13:42 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[73373]: 19: clone() 2024-03-28T18:13:43.048 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:13:42 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[73373]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-03-28T18:13:43.048 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:13:42 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[73373]: 2024-03-28T18:13:43.048 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:13:42 smithi022 podman[93141]: 2024-03-28 18:13:42.851585609 +0000 UTC m=+0.041695919 container died 9c7b84e0d12419cc7a24a5188bb8d58749794e79dd750596877364c5da92b74c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240319, io.buildah.version=1.33.5, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, ceph=True, GIT_CLEAN=True, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30) 2024-03-28T18:13:43.925 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:13:43 smithi134 ceph-mon[35037]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-03-28T18:13:43.925 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:13:43 smithi134 ceph-mon[35037]: Cluster is now healthy 2024-03-28T18:13:43.925 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:13:43 smithi134 ceph-mon[35037]: mds.? [v2:172.21.15.22:6836/3676939386,v1:172.21.15.22:6837/3676939386] up:active 2024-03-28T18:13:43.925 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:13:43 smithi134 ceph-mon[35037]: fsmap cephfs:5 4 up:standby-replay 1 up:standby 5 up:active 2024-03-28T18:13:43.925 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:13:43 smithi134 ceph-mon[35037]: fsmap cephfs:5 5 up:standby-replay 5 up:active 2024-03-28T18:13:43.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:13:43 smithi079 ceph-mon[35312]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-03-28T18:13:43.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:13:43 smithi079 ceph-mon[35312]: Cluster is now healthy 2024-03-28T18:13:43.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:13:43 smithi079 ceph-mon[35312]: mds.? [v2:172.21.15.22:6836/3676939386,v1:172.21.15.22:6837/3676939386] up:active 2024-03-28T18:13:43.933 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:13:43 smithi079 ceph-mon[35312]: fsmap cephfs:5 4 up:standby-replay 1 up:standby 5 up:active 2024-03-28T18:13:43.933 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:13:43 smithi079 ceph-mon[35312]: fsmap cephfs:5 5 up:standby-replay 5 up:active 2024-03-28T18:13:44.032 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:13:43 smithi022 podman[93141]: 2024-03-28 18:13:43.722629946 +0000 UTC m=+0.912740287 container remove 9c7b84e0d12419cc7a24a5188bb8d58749794e79dd750596877364c5da92b74c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.build-date=20240319, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, ceph=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS) 2024-03-28T18:13:44.032 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:13:43 smithi022 systemd[1]: ceph-fd220748-ed2b-11ee-95d0-87774f69a715@mds.d.service: Main process exited, code=exited, status=134/n/a 2024-03-28T18:13:44.032 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:13:43 smithi022 ceph-mon[31985]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-03-28T18:13:44.032 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:13:43 smithi022 ceph-mon[31985]: Cluster is now healthy 2024-03-28T18:13:44.032 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:13:43 smithi022 ceph-mon[31985]: mds.? [v2:172.21.15.22:6836/3676939386,v1:172.21.15.22:6837/3676939386] up:active 2024-03-28T18:13:44.032 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:13:43 smithi022 ceph-mon[31985]: fsmap cephfs:5 4 up:standby-replay 1 up:standby 5 up:active 2024-03-28T18:13:44.032 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:13:43 smithi022 ceph-mon[31985]: fsmap cephfs:5 5 up:standby-replay 5 up:active 2024-03-28T18:13:44.433 INFO:journalctl@ceph.mds.b.smithi079.stdout:Mar 28 18:13:44 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-b[65245]: 2024-03-28T18:13:44.091+0000 7f845f400700 -1 mds.pinger is_rank_lagging: rank=1 was never sent ping request. 2024-03-28T18:13:44.531 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:13:44 smithi022 systemd[1]: ceph-fd220748-ed2b-11ee-95d0-87774f69a715@mds.d.service: Failed with result 'exit-code'. 2024-03-28T18:13:44.532 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:13:44 smithi022 systemd[1]: ceph-fd220748-ed2b-11ee-95d0-87774f69a715@mds.d.service: Consumed 2.299s CPU time. 2024-03-28T18:13:44.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:13:44 smithi134 ceph-mon[35037]: pgmap v200: 97 pgs: 97 active+clean; 117 MiB data, 786 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 10 op/s 2024-03-28T18:13:44.933 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:13:44 smithi079 ceph-mon[35312]: pgmap v200: 97 pgs: 97 active+clean; 117 MiB data, 786 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 10 op/s 2024-03-28T18:13:45.032 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:13:44 smithi022 ceph-mon[31985]: pgmap v200: 97 pgs: 97 active+clean; 117 MiB data, 786 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 10 op/s 2024-03-28T18:13:46.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:13:46 smithi134 ceph-mon[35037]: pgmap v201: 97 pgs: 97 active+clean; 117 MiB data, 790 MiB used, 1.0 TiB / 1.0 TiB avail; 108 KiB/s rd, 3.8 KiB/s wr, 25 op/s 2024-03-28T18:13:46.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:13:46 smithi079 ceph-mon[35312]: pgmap v201: 97 pgs: 97 active+clean; 117 MiB data, 790 MiB used, 1.0 TiB / 1.0 TiB avail; 108 KiB/s rd, 3.8 KiB/s wr, 25 op/s 2024-03-28T18:13:47.031 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:13:46 smithi022 ceph-mon[31985]: pgmap v201: 97 pgs: 97 active+clean; 117 MiB data, 790 MiB used, 1.0 TiB / 1.0 TiB avail; 108 KiB/s rd, 3.8 KiB/s wr, 25 op/s 2024-03-28T18:13:48.332 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-28T18:13:48.333 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-28T18:13:48.573 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:13:48.573+0000 7f00fd800640 1 -- 172.21.15.22:0/3497424925 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f00f8109850 msgr2=0x7f00f8109c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:13:48.573 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:13:48.573+0000 7f00fd800640 1 --2- 172.21.15.22:0/3497424925 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f00f8109850 0x7f00f8109c30 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7f00e0009930 tx=0x7f00e002bc30 comp rx=0 tx=0).stop 2024-03-28T18:13:48.573 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:13:48.573+0000 7f00fd800640 1 -- 172.21.15.22:0/3497424925 shutdown_connections 2024-03-28T18:13:48.573 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:13:48.573+0000 7f00fd800640 1 --2- 172.21.15.22:0/3497424925 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f00f8104200 0x7f00f8110710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:13:48.573 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:13:48.573+0000 7f00fd800640 1 --2- 172.21.15.22:0/3497424925 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f00f8103840 0x7f00f8103cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:13:48.573 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:13:48.573+0000 7f00fd800640 1 --2- 172.21.15.22:0/3497424925 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f00f8109850 0x7f00f8109c30 unknown :-1 s=CLOSED pgs=727 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:13:48.573 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:13:48.573+0000 7f00fd800640 1 -- 172.21.15.22:0/3497424925 >> 172.21.15.22:0/3497424925 conn(0x7f00f80ff2f0 msgr2=0x7f00f8101710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:13:48.574 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:13:48.573+0000 7f00fd800640 1 -- 172.21.15.22:0/3497424925 shutdown_connections 2024-03-28T18:13:48.574 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:13:48.573+0000 7f00fd800640 1 -- 172.21.15.22:0/3497424925 wait complete. 2024-03-28T18:13:48.575 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:13:48.574+0000 7f00fd800640 1 Processor -- start 2024-03-28T18:13:48.575 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:13:48.574+0000 7f00fd800640 1 -- start start 2024-03-28T18:13:48.575 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:13:48.574+0000 7f00fd800640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f00f8103840 0x7f00f819a1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:13:48.575 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:13:48.574+0000 7f00fd800640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f00f8104200 0x7f00f819a720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:13:48.575 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:13:48.575+0000 7f00fd800640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f00f8109850 0x7f00f819eaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:13:48.575 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:13:48.575+0000 7f00fd800640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00f8113c80 con 0x7f00f8109850 2024-03-28T18:13:48.576 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:13:48.575+0000 7f00fd800640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00f8113b00 con 0x7f00f8103840 2024-03-28T18:13:48.576 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:13:48.575+0000 7f00fd800640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00f8113e00 con 0x7f00f8104200 2024-03-28T18:13:48.576 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:13:48.575+0000 7f00f7400640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f00f8109850 0x7f00f819eaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:13:48.576 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:13:48.575+0000 7f00f7400640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f00f8109850 0x7f00f819eaa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:47242/0 (socket says 172.21.15.22:47242) 2024-03-28T18:13:48.576 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:13:48.575+0000 7f00f7400640 1 -- 172.21.15.22:0/39358369 learned_addr learned my addr 172.21.15.22:0/39358369 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-28T18:13:48.576 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:13:48.575+0000 7f00f6a00640 1 --2- 172.21.15.22:0/39358369 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f00f8103840 0x7f00f819a1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:13:48.576 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:13:48.575+0000 7f00f6000640 1 --2- 172.21.15.22:0/39358369 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f00f8104200 0x7f00f819a720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:13:48.576 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:13:48.575+0000 7f00f7400640 1 -- 172.21.15.22:0/39358369 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f00f8104200 msgr2=0x7f00f819a720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:13:48.576 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:13:48.575+0000 7f00f7400640 1 --2- 172.21.15.22:0/39358369 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f00f8104200 0x7f00f819a720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:13:48.576 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:13:48.575+0000 7f00f7400640 1 -- 172.21.15.22:0/39358369 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f00f8103840 msgr2=0x7f00f819a1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:13:48.576 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:13:48.575+0000 7f00f7400640 1 --2- 172.21.15.22:0/39358369 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f00f8103840 0x7f00f819a1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:13:48.576 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:13:48.575+0000 7f00f7400640 1 -- 172.21.15.22:0/39358369 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f00f819f1d0 con 0x7f00f8109850 2024-03-28T18:13:48.576 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:13:48.576+0000 7f00f7400640 1 --2- 172.21.15.22:0/39358369 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f00f8109850 0x7f00f819eaa0 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7f00f000eb10 tx=0x7f00f000c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:13:48.577 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:13:48.576+0000 7f00ef400640 1 -- 172.21.15.22:0/39358369 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00f0004b20 con 0x7f00f8109850 2024-03-28T18:13:48.577 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:13:48.576+0000 7f00ef400640 1 -- 172.21.15.22:0/39358369 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f00f0004cc0 con 0x7f00f8109850 2024-03-28T18:13:48.577 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:13:48.576+0000 7f00ef400640 1 -- 172.21.15.22:0/39358369 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00f0010700 con 0x7f00f8109850 2024-03-28T18:13:48.577 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:13:48.576+0000 7f00fd800640 1 -- 172.21.15.22:0/39358369 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f00f819f4c0 con 0x7f00f8109850 2024-03-28T18:13:48.577 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:13:48.576+0000 7f00fd800640 1 -- 172.21.15.22:0/39358369 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f00f81a6d50 con 0x7f00f8109850 2024-03-28T18:13:48.579 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:13:48.579+0000 7f0100404740 1 Processor -- start 2024-03-28T18:13:48.579 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:13:48.579+0000 7f00ef400640 1 -- 172.21.15.22:0/39358369 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 26) v1 ==== 137421+0+0 (secure 0 0 0) 0x7f00f0004e70 con 0x7f00f8109850 2024-03-28T18:13:48.579 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:13:48.580+0000 7f0100404740 1 -- start start 2024-03-28T18:13:48.580 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:13:48.580+0000 7f0100404740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55927fdbbc00 0x55927fdba950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:13:48.580 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:13:48.580+0000 7f0100404740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55927fdbaf20 0x55927fdbb3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:13:48.581 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:13:48.580+0000 7f0100404740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55927fdc4290 0x55927fdc6680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:13:48.581 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:13:48.580+0000 7f0100404740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x55927fd1f250 con 0x55927fdc4290 2024-03-28T18:13:48.581 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:13:48.580+0000 7f0100404740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x55927fcc7a30 con 0x55927fdbaf20 2024-03-28T18:13:48.581 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:13:48.580+0000 7f0100404740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55927fd1b2f0 con 0x55927fdbbc00 2024-03-28T18:13:48.581 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:13:48.580+0000 7f00f6000640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55927fdc4290 0x55927fdc6680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:13:48.581 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:13:48.580+0000 7f00f6a00640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55927fdbaf20 0x55927fdbb3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:13:48.581 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:13:48.580+0000 7f00f6000640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55927fdc4290 0x55927fdc6680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:47258/0 (socket says 172.21.15.22:47258) 2024-03-28T18:13:48.581 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:13:48.580+0000 7f00f6000640 1 -- 172.21.15.22:0/4033733724 learned_addr learned my addr 172.21.15.22:0/4033733724 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-28T18:13:48.581 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:13:48.581+0000 7f00f6000640 1 --2- 172.21.15.22:0/4033733724 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55927fdbbc00 0x55927fdba950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:13:48.581 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:13:48.581+0000 7f00f6a00640 1 -- 172.21.15.22:0/4033733724 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55927fdbbc00 msgr2=0x55927fdba950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T18:13:48.581 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:13:48.581+0000 7f00f6a00640 1 --2- 172.21.15.22:0/4033733724 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55927fdbbc00 0x55927fdba950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:13:48.582 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:13:48.581+0000 7f00f6a00640 1 -- 172.21.15.22:0/4033733724 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55927fdc4290 msgr2=0x55927fdc6680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T18:13:48.582 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:13:48.581+0000 7f00f6a00640 1 --2- 172.21.15.22:0/4033733724 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55927fdc4290 0x55927fdc6680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:13:48.582 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:13:48.581+0000 7f00f6a00640 1 -- 172.21.15.22:0/4033733724 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55927fcb2690 con 0x55927fdbaf20 2024-03-28T18:13:48.585 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:13:48.581+0000 7f00ef400640 1 --2- 172.21.15.22:0/39358369 >> [v2:172.21.15.22:6800/1357365082,v1:172.21.15.22:6801/1357365082] conn(0x7f00dc0a29f0 0x7f00dc0a4eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:13:48.585 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:13:48.581+0000 7f00f6a00640 1 --2- 172.21.15.22:0/4033733724 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55927fdbaf20 0x55927fdbb3a0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f00e002c170 tx=0x7f00e0005c00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=65bb833ddc5e18df server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:13:48.585 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:13:48.581+0000 7f00ef400640 1 -- 172.21.15.22:0/39358369 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(105..105 src has 1..105) v4 ==== 9443+0+0 (secure 0 0 0) 0x7f00f00cf790 con 0x7f00f8109850 2024-03-28T18:13:48.585 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:13:48.582+0000 7f00f6000640 1 --2- 172.21.15.22:0/39358369 >> [v2:172.21.15.22:6800/1357365082,v1:172.21.15.22:6801/1357365082] conn(0x7f00dc0a29f0 0x7f00dc0a4eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:13:48.585 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:13:48.582+0000 7f00e7e00640 1 -- 172.21.15.22:0/4033733724 <== mon.2 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00e0045020 con 0x55927fdbaf20 2024-03-28T18:13:48.585 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:13:48.582+0000 7f00e7e00640 1 -- 172.21.15.22:0/4033733724 <== mon.2 v2:172.21.15.79:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f00e0005510 con 0x55927fdbaf20 2024-03-28T18:13:48.585 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:13:48.582+0000 7f00e7e00640 1 -- 172.21.15.22:0/4033733724 <== mon.2 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00e002cb70 con 0x55927fdbaf20 2024-03-28T18:13:48.585 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:13:48.583+0000 7f0100404740 1 -- 172.21.15.22:0/4033733724 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55927fdbaf20 msgr2=0x55927fdbb3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:13:48.585 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:13:48.583+0000 7f00f6000640 1 --2- 172.21.15.22:0/39358369 >> [v2:172.21.15.22:6800/1357365082,v1:172.21.15.22:6801/1357365082] conn(0x7f00dc0a29f0 0x7f00dc0a4eb0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f00e80058c0 tx=0x7f00e8016040 comp rx=0 tx=0).ready entity=mgr.15522 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:13:48.585 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:13:48.583+0000 7f0100404740 1 --2- 172.21.15.22:0/4033733724 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55927fdbaf20 0x55927fdbb3a0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f00e002c170 tx=0x7f00e0005c00 comp rx=0 tx=0).stop 2024-03-28T18:13:48.585 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:13:48.583+0000 7f0100404740 1 -- 172.21.15.22:0/4033733724 shutdown_connections 2024-03-28T18:13:48.586 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:13:48.583+0000 7f0100404740 1 --2- 172.21.15.22:0/4033733724 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55927fdc4290 0x55927fdc6680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:13:48.586 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:13:48.583+0000 7f0100404740 1 --2- 172.21.15.22:0/4033733724 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55927fdbaf20 0x55927fdbb3a0 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:13:48.586 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:13:48.583+0000 7f0100404740 1 --2- 172.21.15.22:0/4033733724 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55927fdbbc00 0x55927fdba950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:13:48.586 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:13:48.583+0000 7f0100404740 1 -- 172.21.15.22:0/4033733724 >> 172.21.15.22:0/4033733724 conn(0x55927fd82200 msgr2=0x55927fdb5970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:13:48.586 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:13:48.583+0000 7f0100404740 1 -- 172.21.15.22:0/4033733724 shutdown_connections 2024-03-28T18:13:48.586 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:13:48.583+0000 7f0100404740 1 -- 172.21.15.22:0/4033733724 wait complete. 2024-03-28T18:13:48.586 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:13:48.583+0000 7f0100404740 1 -- start start 2024-03-28T18:13:48.586 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:13:48.584+0000 7f0100404740 1 Processor -- start 2024-03-28T18:13:48.586 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:13:48.584+0000 7f0100404740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55927fdbaf20 0x55927fdc3920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:13:48.586 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:13:48.584+0000 7f0100404740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55927fdbbc00 0x55927fdc0cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:13:48.586 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:13:48.584+0000 7f0100404740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55927fdc4290 0x55927fdc11f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:13:48.586 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:13:48.584+0000 7f0100404740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x55927fdc70a0 con 0x55927fdbaf20 2024-03-28T18:13:48.586 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:13:48.584+0000 7f0100404740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x55927fcdae80 con 0x55927fdc4290 2024-03-28T18:13:48.586 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:13:48.584+0000 7f0100404740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55927fdc7220 con 0x55927fdbbc00 2024-03-28T18:13:48.587 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:13:48.584+0000 7f0100404740 10 client.0 _ll_register_callbacks cb 0x55927fced130 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-28T18:13:48.587 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:13:48.584+0000 7f00f6a00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55927fdbaf20 0x55927fdc3920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:13:48.587 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:13:48.584+0000 7f00f6a00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55927fdbaf20 0x55927fdc3920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:47266/0 (socket says 172.21.15.22:47266) 2024-03-28T18:13:48.587 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:13:48.584+0000 7f00f6a00640 1 -- 172.21.15.22:0/1799401038 learned_addr learned my addr 172.21.15.22:0/1799401038 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-28T18:13:48.587 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:13:48.584+0000 7f00f7400640 1 --2- 172.21.15.22:0/1799401038 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55927fdc4290 0x55927fdc11f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:13:48.587 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:13:48.585+0000 7f00f6000640 1 --2- 172.21.15.22:0/1799401038 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55927fdbbc00 0x55927fdc0cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:13:48.587 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:13:48.585+0000 7f00f6a00640 1 -- 172.21.15.22:0/1799401038 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55927fdbbc00 msgr2=0x55927fdc0cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T18:13:48.587 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:13:48.585+0000 7f00f6a00640 1 --2- 172.21.15.22:0/1799401038 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55927fdbbc00 0x55927fdc0cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:13:48.587 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:13:48.585+0000 7f00f6a00640 1 -- 172.21.15.22:0/1799401038 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55927fdc4290 msgr2=0x55927fdc11f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T18:13:48.587 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:13:48.585+0000 7f00f6a00640 1 --2- 172.21.15.22:0/1799401038 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55927fdc4290 0x55927fdc11f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:13:48.587 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:13:48.585+0000 7f00f6a00640 1 -- 172.21.15.22:0/1799401038 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55927fafd7d0 con 0x55927fdbaf20 2024-03-28T18:13:48.587 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:13:48.585+0000 7f00f6a00640 1 --2- 172.21.15.22:0/1799401038 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55927fdbaf20 0x55927fdc3920 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7f00e0036040 tx=0x7f00e0005fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1209816213f4e07b server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:13:48.587 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:13:48.585+0000 7f00e4c00640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-03-28T18:13:48.589 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:13:48.586+0000 7f00e4c00640 1 -- 172.21.15.22:0/1799401038 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00e0051070 con 0x55927fdbaf20 2024-03-28T18:13:48.589 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:13:48.586+0000 7f00e4c00640 1 -- 172.21.15.22:0/1799401038 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(105..105 src has 1..105) v4 ==== 9443+0+0 (secure 0 0 0) 0x7f00e0003a00 con 0x55927fdbaf20 2024-03-28T18:13:48.589 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:13:48.586+0000 7f0100404740 1 -- 172.21.15.22:0/1799401038 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f00e0005510 con 0x55927fdbaf20 2024-03-28T18:13:48.589 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:13:48.586+0000 7f00e4c00640 1 -- 172.21.15.22:0/1799401038 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=113) v2 ==== 24+0+0 (secure 0 0 0) 0x7f00e005c050 con 0x55927fdbaf20 2024-03-28T18:13:48.589 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:13:48.586+0000 7f0100404740 10 client.16635 fetch_fsmap learned FSMap version 113 2024-03-28T18:13:48.589 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:13:48.586+0000 7f0100404740 1 -- 172.21.15.22:0/1799401038 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=113,monmap=4+}) v3 -- 0x55927fa333f0 con 0x55927fdbaf20 2024-03-28T18:13:48.589 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:13:48.586+0000 7f00e4c00640 1 -- 172.21.15.22:0/1799401038 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f00e0005510 con 0x55927fdbaf20 2024-03-28T18:13:48.590 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:13:48.586+0000 7f00e4c00640 1 -- 172.21.15.22:0/1799401038 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 113) v1 ==== 6784+0+0 (secure 0 0 0) 0x7f00e0041a70 con 0x55927fdbaf20 2024-03-28T18:13:48.590 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:13:48.587+0000 7f0100404740 10 client.16635 fetch_fsmap finished waiting for FSMap version 113 2024-03-28T18:13:48.590 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:13:48.587+0000 7f0100404740 10 client.16635 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-28T18:13:48.590 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:13:48.587+0000 7f0100404740 20 client.16635 populate_metadata read hostname 'smithi022' 2024-03-28T18:13:48.590 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:13:48.587+0000 7f0100404740 1 --2- 172.21.15.22:0/1799401038 >> [v2:172.21.15.79:6838/2441542443,v1:172.21.15.79:6839/2441542443] conn(0x55927fdd3f80 0x55927fdd6370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:13:48.590 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:13:48.587+0000 7f0100404740 4 client.16635 mds_command: new command op to 34553 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-28T18:13:48.590 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:13:48.587+0000 7f0100404740 1 -- 172.21.15.22:0/1799401038 --> [v2:172.21.15.79:6838/2441542443,v1:172.21.15.79:6839/2441542443] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55927fd2c790 con 0x55927fdd3f80 2024-03-28T18:13:48.590 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:13:48.587+0000 7f00f6000640 1 --2- 172.21.15.22:0/1799401038 >> [v2:172.21.15.79:6838/2441542443,v1:172.21.15.79:6839/2441542443] conn(0x55927fdd3f80 0x55927fdd6370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:13:48.590 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:13:48.588+0000 7f00f6000640 1 --2- 172.21.15.22:0/1799401038 >> [v2:172.21.15.79:6838/2441542443,v1:172.21.15.79:6839/2441542443] conn(0x55927fdd3f80 0x55927fdd6370 crc :-1 s=READY pgs=191 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5b6fd1b026a152e9 server_cookie=60bc232c3c635a30 in_seq=0 out_seq=0 2024-03-28T18:13:48.590 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:13:48.588+0000 7f00e4c00640 10 client.16635 ms_handle_connect on v2:172.21.15.79:6838/2441542443 2024-03-28T18:13:48.590 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:13:48.590+0000 7f00e4c00640 1 -- 172.21.15.22:0/1799401038 <== mds.0 v2:172.21.15.79:6838/2441542443 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x55927fd2c790 con 0x55927fdd3f80 2024-03-28T18:13:48.590 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:13:48.590+0000 7f00e4c00640 10 client.16635 handle_command_reply: tid=0 2024-03-28T18:13:48.590 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:13:48.590+0000 7f0100404740 1 client.16635 shutdown 2024-03-28T18:13:48.591 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:13:48.590+0000 7f0100404740 1 -- 172.21.15.22:0/1799401038 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55927fdbaf20 msgr2=0x55927fdc3920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:13:48.591 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:13:48.590+0000 7f0100404740 1 --2- 172.21.15.22:0/1799401038 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55927fdbaf20 0x55927fdc3920 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7f00e0036040 tx=0x7f00e0005fd0 comp rx=0 tx=0).stop 2024-03-28T18:13:48.591 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:13:48.590+0000 7f0100404740 1 -- 172.21.15.22:0/1799401038 shutdown_connections 2024-03-28T18:13:48.591 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:13:48.590+0000 7f0100404740 1 --2- 172.21.15.22:0/1799401038 >> [v2:172.21.15.79:6838/2441542443,v1:172.21.15.79:6839/2441542443] conn(0x55927fdd3f80 0x55927fdd6370 crc :-1 s=READY pgs=191 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:13:48.592 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:13:48.590+0000 7f0100404740 1 --2- 172.21.15.22:0/1799401038 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55927fdc4290 0x55927fdc11f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:13:48.592 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:13:48.590+0000 7f0100404740 1 --2- 172.21.15.22:0/1799401038 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55927fdbbc00 0x55927fdc0cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:13:48.592 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:13:48.590+0000 7f0100404740 1 --2- 172.21.15.22:0/1799401038 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55927fdbaf20 0x55927fdc3920 unknown :-1 s=CLOSED pgs=729 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:13:48.592 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:13:48.590+0000 7f0100404740 1 -- 172.21.15.22:0/1799401038 >> 172.21.15.22:0/1799401038 conn(0x55927fd82200 msgr2=0x55927fdb76c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:13:48.592 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:13:48.590+0000 7f00e4c00640 0 client.16635 ms_handle_reset on v2:172.21.15.79:6838/2441542443 2024-03-28T18:13:48.592 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:13:48.590+0000 7f0100404740 1 -- 172.21.15.22:0/1799401038 shutdown_connections 2024-03-28T18:13:48.592 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:13:48.590+0000 7f0100404740 1 -- 172.21.15.22:0/1799401038 wait complete. 2024-03-28T18:13:48.592 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:13:48.590+0000 7f0100404740 20 client.16635 trim_cache size 0 max 16384 2024-03-28T18:13:48.609 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:13:48.608+0000 7f0100404740 1 Processor -- start 2024-03-28T18:13:48.609 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:13:48.608+0000 7f0100404740 1 -- start start 2024-03-28T18:13:48.609 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:13:48.609+0000 7f0100404740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55927fdbaf20 0x55927fdbfc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:13:48.609 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:13:48.609+0000 7f0100404740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55927fdbbc00 0x55927fdc33e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:13:48.609 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:13:48.609+0000 7f0100404740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55927fdc4290 0x55927fdc3920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:13:48.609 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:13:48.609+0000 7f0100404740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00f81ae780 con 0x55927fdbaf20 2024-03-28T18:13:48.609 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:13:48.609+0000 7f0100404740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00f81b0dc0 con 0x55927fdc4290 2024-03-28T18:13:48.609 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:13:48.609+0000 7f0100404740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00f81aede0 con 0x55927fdbbc00 2024-03-28T18:13:48.609 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:13:48.609+0000 7f00f6a00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55927fdbaf20 0x55927fdbfc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:13:48.610 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:13:48.609+0000 7f00f6a00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55927fdbaf20 0x55927fdbfc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:47272/0 (socket says 172.21.15.22:47272) 2024-03-28T18:13:48.610 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:13:48.609+0000 7f00f6a00640 1 -- 172.21.15.22:0/3837188159 learned_addr learned my addr 172.21.15.22:0/3837188159 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-28T18:13:48.610 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:13:48.609+0000 7f00f6000640 1 --2- 172.21.15.22:0/3837188159 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55927fdbbc00 0x55927fdc33e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:13:48.610 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:13:48.609+0000 7f00f7400640 1 --2- 172.21.15.22:0/3837188159 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55927fdc4290 0x55927fdc3920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:13:48.611 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:13:48.610+0000 7f00f6a00640 1 -- 172.21.15.22:0/3837188159 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55927fdbbc00 msgr2=0x55927fdc33e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T18:13:48.611 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:13:48.610+0000 7f00f6a00640 1 --2- 172.21.15.22:0/3837188159 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55927fdbbc00 0x55927fdc33e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:13:48.611 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:13:48.610+0000 7f00f6a00640 1 -- 172.21.15.22:0/3837188159 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55927fdc4290 msgr2=0x55927fdc3920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T18:13:48.611 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:13:48.610+0000 7f00f6a00640 1 --2- 172.21.15.22:0/3837188159 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55927fdc4290 0x55927fdc3920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:13:48.611 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:13:48.610+0000 7f00f6a00640 1 -- 172.21.15.22:0/3837188159 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55927f9f30f0 con 0x55927fdbaf20 2024-03-28T18:13:48.612 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:13:48.610+0000 7f00f7400640 1 --2- 172.21.15.22:0/3837188159 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55927fdc4290 0x55927fdc3920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-28T18:13:48.612 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:13:48.610+0000 7f00f6a00640 1 --2- 172.21.15.22:0/3837188159 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55927fdbaf20 0x55927fdbfc90 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7f00e002c170 tx=0x7f00e0005c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d05767ac7354a38f server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:13:48.612 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:13:48.610+0000 7f00e7400640 1 -- 172.21.15.22:0/3837188159 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00e0051040 con 0x55927fdbaf20 2024-03-28T18:13:48.612 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:13:48.610+0000 7f00e7400640 1 -- 172.21.15.22:0/3837188159 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f00e0009520 con 0x55927fdbaf20 2024-03-28T18:13:48.612 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:13:48.610+0000 7f00e7400640 1 -- 172.21.15.22:0/3837188159 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00e0045040 con 0x55927fdbaf20 2024-03-28T18:13:48.612 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:13:48.611+0000 7f0100404740 1 -- 172.21.15.22:0/3837188159 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55927fdbaf20 msgr2=0x55927fdbfc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:13:48.612 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:13:48.611+0000 7f0100404740 1 --2- 172.21.15.22:0/3837188159 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55927fdbaf20 0x55927fdbfc90 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7f00e002c170 tx=0x7f00e0005c00 comp rx=0 tx=0).stop 2024-03-28T18:13:48.612 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:13:48.611+0000 7f0100404740 1 -- 172.21.15.22:0/3837188159 shutdown_connections 2024-03-28T18:13:48.612 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:13:48.611+0000 7f0100404740 1 --2- 172.21.15.22:0/3837188159 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55927fdc4290 0x55927fdc3920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:13:48.612 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:13:48.611+0000 7f0100404740 1 --2- 172.21.15.22:0/3837188159 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55927fdbbc00 0x55927fdc33e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:13:48.612 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:13:48.611+0000 7f0100404740 1 --2- 172.21.15.22:0/3837188159 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55927fdbaf20 0x55927fdbfc90 unknown :-1 s=CLOSED pgs=730 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:13:48.612 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:13:48.611+0000 7f0100404740 1 -- 172.21.15.22:0/3837188159 >> 172.21.15.22:0/3837188159 conn(0x55927fd82200 msgr2=0x55927fdbf580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:13:48.612 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:13:48.611+0000 7f0100404740 1 -- 172.21.15.22:0/3837188159 shutdown_connections 2024-03-28T18:13:48.612 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:13:48.611+0000 7f0100404740 1 -- 172.21.15.22:0/3837188159 wait complete. 2024-03-28T18:13:48.612 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:13:48.611+0000 7f0100404740 1 -- start start 2024-03-28T18:13:48.613 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:13:48.612+0000 7f0100404740 1 Processor -- start 2024-03-28T18:13:48.615 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:13:48.612+0000 7f0100404740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55927fdbaf20 0x55927fdbe790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:13:48.615 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:13:48.612+0000 7f0100404740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55927fdbbc00 0x55927fdbcf20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:13:48.615 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:13:48.612+0000 7f0100404740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55927fdc4290 0x55927fdbd460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:13:48.615 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:13:48.612+0000 7f0100404740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00f81af440 con 0x55927fdbbc00 2024-03-28T18:13:48.615 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:13:48.612+0000 7f0100404740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00f81b0760 con 0x55927fdc4290 2024-03-28T18:13:48.615 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:13:48.612+0000 7f0100404740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00f81afaa0 con 0x55927fdbaf20 2024-03-28T18:13:48.615 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:13:48.612+0000 7f0100404740 10 client.0 _ll_register_callbacks cb 0x7f00e800cab0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-28T18:13:48.616 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:13:48.612+0000 7f00f6000640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55927fdbbc00 0x55927fdbcf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:13:48.616 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:13:48.613+0000 7f00f7400640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55927fdc4290 0x55927fdbd460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:13:48.616 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:13:48.613+0000 7f00f6000640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55927fdbbc00 0x55927fdbcf20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:47274/0 (socket says 172.21.15.22:47274) 2024-03-28T18:13:48.616 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:13:48.613+0000 7f00f6000640 1 -- 172.21.15.22:0/3097479338 learned_addr learned my addr 172.21.15.22:0/3097479338 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-28T18:13:48.616 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:13:48.613+0000 7f00f7400640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55927fdc4290 0x55927fdbd460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.22:42470/0 (socket says 172.21.15.22:42470) 2024-03-28T18:13:48.616 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:13:48.613+0000 7f00f6a00640 1 --2- 172.21.15.22:0/3097479338 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55927fdbaf20 0x55927fdbe790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:13:48.616 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:13:48.613+0000 7f00f7400640 1 -- 172.21.15.22:0/3097479338 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55927fdbaf20 msgr2=0x55927fdbe790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T18:13:48.616 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:13:48.613+0000 7f00f7400640 1 --2- 172.21.15.22:0/3097479338 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55927fdbaf20 0x55927fdbe790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:13:48.616 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:13:48.613+0000 7f00f7400640 1 -- 172.21.15.22:0/3097479338 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55927fdbbc00 msgr2=0x55927fdbcf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T18:13:48.616 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:13:48.613+0000 7f00f7400640 1 --2- 172.21.15.22:0/3097479338 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55927fdbbc00 0x55927fdbcf20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:13:48.616 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:13:48.613+0000 7f00f7400640 1 -- 172.21.15.22:0/3097479338 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55927fdbd9f0 con 0x55927fdc4290 2024-03-28T18:13:48.616 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:13:48.613+0000 7f00f7400640 1 --2- 172.21.15.22:0/3097479338 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55927fdc4290 0x55927fdbd460 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f00f00ccae0 tx=0x7f00f009b480 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7e7fe25a01cd4604 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:13:48.616 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:13:48.613+0000 7f00e4c00640 10 client.0 ms_handle_connect on v2:172.21.15.79:3300/0 2024-03-28T18:13:48.616 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:13:48.613+0000 7f00e4c00640 1 -- 172.21.15.22:0/3097479338 <== mon.2 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00f009be50 con 0x55927fdc4290 2024-03-28T18:13:48.617 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:13:48.614+0000 7f0100404740 1 -- 172.21.15.22:0/3097479338 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f00e0009520 con 0x55927fdc4290 2024-03-28T18:13:48.617 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:13:48.614+0000 7f00e4c00640 1 -- 172.21.15.22:0/3097479338 <== mon.2 v2:172.21.15.79:3300/0 2 ==== osd_map(105..105 src has 1..105) v4 ==== 9443+0+0 (secure 0 0 0) 0x7f00f00a3d00 con 0x55927fdc4290 2024-03-28T18:13:48.617 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:13:48.614+0000 7f00e4c00640 1 -- 172.21.15.22:0/3097479338 <== mon.2 v2:172.21.15.79:3300/0 3 ==== mon_get_version_reply(handle=1 version=113) v2 ==== 24+0+0 (secure 0 0 0) 0x7f00f00ba2b0 con 0x55927fdc4290 2024-03-28T18:13:48.617 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:13:48.614+0000 7f0100404740 10 client.36476 fetch_fsmap learned FSMap version 113 2024-03-28T18:13:48.617 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:13:48.614+0000 7f0100404740 1 -- 172.21.15.22:0/3097479338 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,fsmap=113,monmap=4+}) v3 -- 0x55927fe2f4a0 con 0x55927fdc4290 2024-03-28T18:13:48.617 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:13:48.614+0000 7f00e4c00640 1 -- 172.21.15.22:0/3097479338 <== mon.2 v2:172.21.15.79:3300/0 4 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f00e0009520 con 0x55927fdc4290 2024-03-28T18:13:48.617 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:13:48.614+0000 7f00e4c00640 1 -- 172.21.15.22:0/3097479338 <== mon.2 v2:172.21.15.79:3300/0 5 ==== fsmap(e 113) v1 ==== 6784+0+0 (secure 0 0 0) 0x7f00f00cf3d0 con 0x55927fdc4290 2024-03-28T18:13:48.617 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:13:48.614+0000 7f0100404740 10 client.36476 fetch_fsmap finished waiting for FSMap version 113 2024-03-28T18:13:48.617 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:13:48.615+0000 7f0100404740 10 client.36476 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-28T18:13:48.617 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:13:48.615+0000 7f0100404740 20 client.36476 populate_metadata read hostname 'smithi022' 2024-03-28T18:13:48.617 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:13:48.615+0000 7f0100404740 1 --2- 172.21.15.22:0/3097479338 >> [v2:172.21.15.79:6838/2441542443,v1:172.21.15.79:6839/2441542443] conn(0x55927fe2fb80 0x55927fe31fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:13:48.617 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:13:48.615+0000 7f0100404740 4 client.36476 mds_command: new command op to 34553 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-28T18:13:48.617 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:13:48.615+0000 7f0100404740 1 -- 172.21.15.22:0/3097479338 --> [v2:172.21.15.79:6838/2441542443,v1:172.21.15.79:6839/2441542443] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55927fd26ae0 con 0x55927fe2fb80 2024-03-28T18:13:48.617 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:13:48.615+0000 7f00f6a00640 1 --2- 172.21.15.22:0/3097479338 >> [v2:172.21.15.79:6838/2441542443,v1:172.21.15.79:6839/2441542443] conn(0x55927fe2fb80 0x55927fe31fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:13:48.619 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:13:48.616+0000 7f00f6a00640 1 --2- 172.21.15.22:0/3097479338 >> [v2:172.21.15.79:6838/2441542443,v1:172.21.15.79:6839/2441542443] conn(0x55927fe2fb80 0x55927fe31fa0 crc :-1 s=READY pgs=192 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=61112b85d83d90e6 server_cookie=46727a6671bb23fd in_seq=0 out_seq=0 2024-03-28T18:13:48.619 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:13:48.616+0000 7f00e4c00640 10 client.36476 ms_handle_connect on v2:172.21.15.79:6838/2441542443 2024-03-28T18:13:48.619 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:13:48.616+0000 7f00e4c00640 1 -- 172.21.15.22:0/3097479338 <== mds.0 v2:172.21.15.79:6838/2441542443 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x55927fd26ae0 con 0x55927fe2fb80 2024-03-28T18:13:48.619 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:13:48.616+0000 7f00e4c00640 10 client.36476 handle_command_reply: tid=0 2024-03-28T18:13:48.619 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:13:48.617+0000 7f0100404740 1 client.36476 shutdown 2024-03-28T18:13:48.619 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:13:48.617+0000 7f0100404740 1 -- 172.21.15.22:0/3097479338 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55927fdc4290 msgr2=0x55927fdbd460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:13:48.619 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:13:48.617+0000 7f0100404740 1 --2- 172.21.15.22:0/3097479338 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55927fdc4290 0x55927fdbd460 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f00f00ccae0 tx=0x7f00f009b480 comp rx=0 tx=0).stop 2024-03-28T18:13:48.619 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:13:48.617+0000 7f0100404740 1 -- 172.21.15.22:0/3097479338 shutdown_connections 2024-03-28T18:13:48.619 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:13:48.617+0000 7f0100404740 1 --2- 172.21.15.22:0/3097479338 >> [v2:172.21.15.79:6838/2441542443,v1:172.21.15.79:6839/2441542443] conn(0x55927fe2fb80 0x55927fe31fa0 crc :-1 s=READY pgs=192 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:13:48.619 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:13:48.617+0000 7f0100404740 1 --2- 172.21.15.22:0/3097479338 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55927fdc4290 0x55927fdbd460 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:13:48.619 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:13:48.617+0000 7f0100404740 1 --2- 172.21.15.22:0/3097479338 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55927fdbbc00 0x55927fdbcf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:13:48.619 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:13:48.617+0000 7f0100404740 1 --2- 172.21.15.22:0/3097479338 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55927fdbaf20 0x55927fdbe790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:13:48.620 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:13:48.617+0000 7f0100404740 1 -- 172.21.15.22:0/3097479338 >> 172.21.15.22:0/3097479338 conn(0x55927fd82200 msgr2=0x55927fdb6d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:13:48.620 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:13:48.617+0000 7f00e4c00640 0 client.36476 ms_handle_reset on v2:172.21.15.79:6838/2441542443 2024-03-28T18:13:48.620 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:13:48.617+0000 7f0100404740 1 -- 172.21.15.22:0/3097479338 shutdown_connections 2024-03-28T18:13:48.620 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:13:48.617+0000 7f0100404740 1 -- 172.21.15.22:0/3097479338 wait complete. 2024-03-28T18:13:48.620 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:13:48.617+0000 7f0100404740 20 client.36476 trim_cache size 0 max 16384 2024-03-28T18:13:48.620 INFO:teuthology.orchestra.run.smithi022.stdout:{ 2024-03-28T18:13:48.620 INFO:teuthology.orchestra.run.smithi022.stdout: "status": "IDLE (waiting for more scrubs)", 2024-03-28T18:13:48.620 INFO:teuthology.orchestra.run.smithi022.stdout: "scrubs": { 2024-03-28T18:13:48.620 INFO:teuthology.orchestra.run.smithi022.stdout: "4bbbd61d-29a3-4bf6-a4f2-d5783d585de3": { 2024-03-28T18:13:48.620 INFO:teuthology.orchestra.run.smithi022.stdout: "path": "/", 2024-03-28T18:13:48.621 INFO:teuthology.orchestra.run.smithi022.stdout: "tag": "4bbbd61d-29a3-4bf6-a4f2-d5783d585de3", 2024-03-28T18:13:48.621 INFO:teuthology.orchestra.run.smithi022.stdout: "options": "recursive,force" 2024-03-28T18:13:48.621 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-03-28T18:13:48.621 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-03-28T18:13:48.621 INFO:teuthology.orchestra.run.smithi022.stdout:} 2024-03-28T18:13:48.621 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:13:48.618+0000 7f00fd800640 1 -- 172.21.15.22:0/39358369 >> [v2:172.21.15.22:6800/1357365082,v1:172.21.15.22:6801/1357365082] conn(0x7f00dc0a29f0 msgr2=0x7f00dc0a4eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:13:48.622 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:13:48.618+0000 7f00fd800640 1 --2- 172.21.15.22:0/39358369 >> [v2:172.21.15.22:6800/1357365082,v1:172.21.15.22:6801/1357365082] conn(0x7f00dc0a29f0 0x7f00dc0a4eb0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f00e80058c0 tx=0x7f00e8016040 comp rx=0 tx=0).stop 2024-03-28T18:13:48.622 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:13:48.618+0000 7f00fd800640 1 -- 172.21.15.22:0/39358369 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f00f8109850 msgr2=0x7f00f819eaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:13:48.622 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:13:48.618+0000 7f00fd800640 1 --2- 172.21.15.22:0/39358369 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f00f8109850 0x7f00f819eaa0 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7f00f000eb10 tx=0x7f00f000c2d0 comp rx=0 tx=0).stop 2024-03-28T18:13:48.622 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:13:48.618+0000 7f00fd800640 1 -- 172.21.15.22:0/39358369 shutdown_connections 2024-03-28T18:13:48.622 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:13:48.618+0000 7f00fd800640 1 --2- 172.21.15.22:0/39358369 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f00f8109850 0x7f00f819eaa0 unknown :-1 s=CLOSED pgs=728 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:13:48.622 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:13:48.618+0000 7f00fd800640 1 --2- 172.21.15.22:0/39358369 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f00f8104200 0x7f00f819a720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:13:48.622 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:13:48.618+0000 7f00fd800640 1 --2- 172.21.15.22:0/39358369 >> [v2:172.21.15.22:6800/1357365082,v1:172.21.15.22:6801/1357365082] conn(0x7f00dc0a29f0 0x7f00dc0a4eb0 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:13:48.622 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:13:48.618+0000 7f00fd800640 1 --2- 172.21.15.22:0/39358369 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f00f8103840 0x7f00f819a1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:13:48.622 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:13:48.618+0000 7f00fd800640 1 -- 172.21.15.22:0/39358369 >> 172.21.15.22:0/39358369 conn(0x7f00f80ff2f0 msgr2=0x7f00f80ffb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:13:48.622 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:13:48.618+0000 7f00fd800640 1 -- 172.21.15.22:0/39358369 shutdown_connections 2024-03-28T18:13:48.622 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:13:48.618+0000 7f00fd800640 1 -- 172.21.15.22:0/39358369 wait complete. 2024-03-28T18:13:48.637 INFO:tasks.cephfs.filesystem:scrub status for tag:4bbbd61d-29a3-4bf6-a4f2-d5783d585de3 - {'path': '/', 'tag': '4bbbd61d-29a3-4bf6-a4f2-d5783d585de3', 'options': 'recursive,force'} 2024-03-28T18:13:48.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:13:48 smithi022 ceph-mon[31985]: pgmap v202: 97 pgs: 97 active+clean; 117 MiB data, 790 MiB used, 1.0 TiB / 1.0 TiB avail; 108 KiB/s rd, 3.8 KiB/s wr, 25 op/s 2024-03-28T18:13:48.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:13:48 smithi134 ceph-mon[35037]: pgmap v202: 97 pgs: 97 active+clean; 117 MiB data, 790 MiB used, 1.0 TiB / 1.0 TiB avail; 108 KiB/s rd, 3.8 KiB/s wr, 25 op/s 2024-03-28T18:13:48.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:13:48 smithi079 ceph-mon[35312]: pgmap v202: 97 pgs: 97 active+clean; 117 MiB data, 790 MiB used, 1.0 TiB / 1.0 TiB avail; 108 KiB/s rd, 3.8 KiB/s wr, 25 op/s 2024-03-28T18:13:50.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:13:50 smithi134 ceph-mon[35037]: pgmap v203: 97 pgs: 97 active+clean; 117 MiB data, 790 MiB used, 1.0 TiB / 1.0 TiB avail; 98 KiB/s rd, 3.4 KiB/s wr, 27 op/s 2024-03-28T18:13:50.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:13:50 smithi079 ceph-mon[35312]: pgmap v203: 97 pgs: 97 active+clean; 117 MiB data, 790 MiB used, 1.0 TiB / 1.0 TiB avail; 98 KiB/s rd, 3.4 KiB/s wr, 27 op/s 2024-03-28T18:13:51.031 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:13:50 smithi022 ceph-mon[31985]: pgmap v203: 97 pgs: 97 active+clean; 117 MiB data, 790 MiB used, 1.0 TiB / 1.0 TiB avail; 98 KiB/s rd, 3.4 KiB/s wr, 27 op/s 2024-03-28T18:13:52.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:13:52 smithi134 ceph-mon[35037]: pgmap v204: 97 pgs: 97 active+clean; 117 MiB data, 790 MiB used, 1.0 TiB / 1.0 TiB avail; 89 KiB/s rd, 3.2 KiB/s wr, 21 op/s 2024-03-28T18:13:52.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:13:52 smithi134 ceph-mon[35037]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T18:13:52.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:13:52 smithi079 ceph-mon[35312]: pgmap v204: 97 pgs: 97 active+clean; 117 MiB data, 790 MiB used, 1.0 TiB / 1.0 TiB avail; 89 KiB/s rd, 3.2 KiB/s wr, 21 op/s 2024-03-28T18:13:52.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:13:52 smithi079 ceph-mon[35312]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T18:13:53.031 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:13:52 smithi022 ceph-mon[31985]: pgmap v204: 97 pgs: 97 active+clean; 117 MiB data, 790 MiB used, 1.0 TiB / 1.0 TiB avail; 89 KiB/s rd, 3.2 KiB/s wr, 21 op/s 2024-03-28T18:13:53.032 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:13:52 smithi022 ceph-mon[31985]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T18:13:54.174 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:13:53 smithi134 ceph-mon[35037]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T18:13:54.175 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:13:53 smithi134 ceph-mon[35037]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T18:13:54.175 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:13:53 smithi134 ceph-mon[35037]: from='mgr.15522 ' entity='mgr.x' 2024-03-28T18:13:54.182 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:13:53 smithi079 ceph-mon[35312]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T18:13:54.182 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:13:53 smithi079 ceph-mon[35312]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T18:13:54.182 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:13:53 smithi079 ceph-mon[35312]: from='mgr.15522 ' entity='mgr.x' 2024-03-28T18:13:54.282 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:13:53 smithi022 ceph-mon[31985]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T18:13:54.282 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:13:53 smithi022 ceph-mon[31985]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T18:13:54.282 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:13:53 smithi022 ceph-mon[31985]: from='mgr.15522 ' entity='mgr.x' 2024-03-28T18:13:54.678 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:13:54 smithi022 systemd[1]: ceph-fd220748-ed2b-11ee-95d0-87774f69a715@mds.d.service: Scheduled restart job, restart counter is at 1. 2024-03-28T18:13:54.678 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:13:54 smithi022 systemd[1]: Stopped Ceph mds.d for fd220748-ed2b-11ee-95d0-87774f69a715. 2024-03-28T18:13:54.678 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:13:54 smithi022 systemd[1]: ceph-fd220748-ed2b-11ee-95d0-87774f69a715@mds.d.service: Consumed 2.299s CPU time. 2024-03-28T18:13:54.678 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:13:54 smithi022 systemd[1]: Starting Ceph mds.d for fd220748-ed2b-11ee-95d0-87774f69a715... 2024-03-28T18:13:54.940 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:13:54 smithi022 ceph-mon[31985]: pgmap v205: 97 pgs: 97 active+clean; 117 MiB data, 790 MiB used, 1.0 TiB / 1.0 TiB avail; 89 KiB/s rd, 3.2 KiB/s wr, 21 op/s 2024-03-28T18:13:54.940 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:13:54 smithi022 podman[93430]: 2024-03-28 18:13:54.584115159 +0000 UTC m=+0.033731202 image pull c664afe56bb01db4f9afa9532c20598e201e664f2e9ba2be50db0d52674a26d2 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc 2024-03-28T18:13:54.940 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:13:54 smithi022 podman[93430]: 2024-03-28 18:13:54.748976096 +0000 UTC m=+0.198592192 container create e68ebf0f6af9552391b17dde9ecb9b888cf364645d2ba0279c36335c3c5fd194 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , ceph=True, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.build-date=20240319, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0) 2024-03-28T18:13:55.174 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:13:54 smithi134 ceph-mon[35037]: pgmap v205: 97 pgs: 97 active+clean; 117 MiB data, 790 MiB used, 1.0 TiB / 1.0 TiB avail; 89 KiB/s rd, 3.2 KiB/s wr, 21 op/s 2024-03-28T18:13:55.182 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:13:54 smithi079 ceph-mon[35312]: pgmap v205: 97 pgs: 97 active+clean; 117 MiB data, 790 MiB used, 1.0 TiB / 1.0 TiB avail; 89 KiB/s rd, 3.2 KiB/s wr, 21 op/s 2024-03-28T18:13:55.532 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:13:55 smithi022 podman[93430]: 2024-03-28 18:13:55.036091963 +0000 UTC m=+0.485708033 container init e68ebf0f6af9552391b17dde9ecb9b888cf364645d2ba0279c36335c3c5fd194 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, ceph=True, io.buildah.version=1.33.5, org.label-schema.build-date=20240319, maintainer=Guillaume Abrioux , GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True) 2024-03-28T18:13:55.533 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:13:55 smithi022 podman[93430]: 2024-03-28 18:13:55.047343993 +0000 UTC m=+0.496960063 container start e68ebf0f6af9552391b17dde9ecb9b888cf364645d2ba0279c36335c3c5fd194 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.build-date=20240319, GIT_CLEAN=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-03-28T18:13:55.533 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:13:55 smithi022 ceph-mds[93448]: -- 172.21.15.22:0/3235722013 <== mon.0 v2:172.21.15.22:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x5623d002ba40 con 0x5623cfffec00 2024-03-28T18:13:55.533 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:13:55 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[93444]: starting mds.d at 2024-03-28T18:13:55.533 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:13:55 smithi022 bash[93430]: e68ebf0f6af9552391b17dde9ecb9b888cf364645d2ba0279c36335c3c5fd194 2024-03-28T18:13:55.533 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:13:55 smithi022 systemd[1]: Started Ceph mds.d for fd220748-ed2b-11ee-95d0-87774f69a715. 2024-03-28T18:13:56.175 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:13:55 smithi134 ceph-mon[35037]: daemon mds.d restarted 2024-03-28T18:13:56.175 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:13:55 smithi134 ceph-mon[35037]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-03-28T18:13:56.175 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:13:55 smithi134 ceph-mon[35037]: osdmap e106: 12 total, 12 up, 12 in 2024-03-28T18:13:56.175 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:13:55 smithi134 ceph-mon[35037]: mds.? [v2:172.21.15.22:6836/3983083841,v1:172.21.15.22:6837/3983083841] up:boot 2024-03-28T18:13:56.175 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:13:55 smithi134 ceph-mon[35037]: Standby daemon mds.j assigned to filesystem cephfs as rank 1 2024-03-28T18:13:56.175 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:13:55 smithi134 ceph-mon[35037]: fsmap cephfs:4/5 {0=b=up:active,2=e=up:active,3=k=up:active,4=i=up:active} 5 up:standby-replay 1 up:standby, 1 failed 2024-03-28T18:13:56.175 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:13:55 smithi134 ceph-mon[35037]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-03-28T18:13:56.175 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:13:55 smithi134 ceph-mon[35037]: fsmap cephfs:5/5 4 up:standby-replay 1 up:standby 1 up:replay 4 up:active 2024-03-28T18:13:56.182 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:13:55 smithi079 ceph-mon[35312]: daemon mds.d restarted 2024-03-28T18:13:56.182 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:13:55 smithi079 ceph-mon[35312]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-03-28T18:13:56.182 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:13:55 smithi079 ceph-mon[35312]: osdmap e106: 12 total, 12 up, 12 in 2024-03-28T18:13:56.182 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:13:55 smithi079 ceph-mon[35312]: mds.? [v2:172.21.15.22:6836/3983083841,v1:172.21.15.22:6837/3983083841] up:boot 2024-03-28T18:13:56.182 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:13:55 smithi079 ceph-mon[35312]: Standby daemon mds.j assigned to filesystem cephfs as rank 1 2024-03-28T18:13:56.183 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:13:55 smithi079 ceph-mon[35312]: fsmap cephfs:4/5 {0=b=up:active,2=e=up:active,3=k=up:active,4=i=up:active} 5 up:standby-replay 1 up:standby, 1 failed 2024-03-28T18:13:56.183 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:13:55 smithi079 ceph-mon[35312]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-03-28T18:13:56.183 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:13:55 smithi079 ceph-mon[35312]: fsmap cephfs:5/5 4 up:standby-replay 1 up:standby 1 up:replay 4 up:active 2024-03-28T18:13:56.282 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:13:55 smithi022 ceph-mon[31985]: daemon mds.d restarted 2024-03-28T18:13:56.282 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:13:55 smithi022 ceph-mon[31985]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-03-28T18:13:56.282 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:13:55 smithi022 ceph-mon[31985]: osdmap e106: 12 total, 12 up, 12 in 2024-03-28T18:13:56.282 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:13:55 smithi022 ceph-mon[31985]: mds.? [v2:172.21.15.22:6836/3983083841,v1:172.21.15.22:6837/3983083841] up:boot 2024-03-28T18:13:56.282 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:13:55 smithi022 ceph-mon[31985]: Standby daemon mds.j assigned to filesystem cephfs as rank 1 2024-03-28T18:13:56.282 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:13:55 smithi022 ceph-mon[31985]: fsmap cephfs:4/5 {0=b=up:active,2=e=up:active,3=k=up:active,4=i=up:active} 5 up:standby-replay 1 up:standby, 1 failed 2024-03-28T18:13:56.283 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:13:55 smithi022 ceph-mon[31985]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-03-28T18:13:56.283 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:13:55 smithi022 ceph-mon[31985]: fsmap cephfs:5/5 4 up:standby-replay 1 up:standby 1 up:replay 4 up:active 2024-03-28T18:13:57.175 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:13:56 smithi134 ceph-mon[35037]: pgmap v207: 97 pgs: 97 active+clean; 117 MiB data, 790 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:13:57.175 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:13:56 smithi134 ceph-mon[35037]: mds.? [v2:172.21.15.22:6834/3718238950,v1:172.21.15.22:6835/3718238950] up:resolve 2024-03-28T18:13:57.175 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:13:56 smithi134 ceph-mon[35037]: fsmap cephfs:5/5 4 up:standby-replay 1 up:standby 1 up:resolve 4 up:active 2024-03-28T18:13:57.182 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:13:56 smithi079 ceph-mon[35312]: pgmap v207: 97 pgs: 97 active+clean; 117 MiB data, 790 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:13:57.182 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:13:56 smithi079 ceph-mon[35312]: mds.? [v2:172.21.15.22:6834/3718238950,v1:172.21.15.22:6835/3718238950] up:resolve 2024-03-28T18:13:57.182 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:13:56 smithi079 ceph-mon[35312]: fsmap cephfs:5/5 4 up:standby-replay 1 up:standby 1 up:resolve 4 up:active 2024-03-28T18:13:57.282 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:13:56 smithi022 ceph-mon[31985]: pgmap v207: 97 pgs: 97 active+clean; 117 MiB data, 790 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:13:57.282 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:13:56 smithi022 ceph-mon[31985]: mds.? [v2:172.21.15.22:6834/3718238950,v1:172.21.15.22:6835/3718238950] up:resolve 2024-03-28T18:13:57.282 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:13:56 smithi022 ceph-mon[31985]: fsmap cephfs:5/5 4 up:standby-replay 1 up:standby 1 up:resolve 4 up:active 2024-03-28T18:13:58.282 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:13:58 smithi022 ceph-mon[31985]: reconnect by client.35222 172.21.15.22:0/3729116781 after 0.00499993 2024-03-28T18:13:58.282 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:13:58 smithi022 ceph-mon[31985]: mds.? [v2:172.21.15.22:6834/3718238950,v1:172.21.15.22:6835/3718238950] up:reconnect 2024-03-28T18:13:58.282 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:13:58 smithi022 ceph-mon[31985]: fsmap cephfs:5/5 4 up:standby-replay 1 up:standby 1 up:reconnect 4 up:active 2024-03-28T18:13:58.282 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:13:58 smithi022 ceph-mon[31985]: pgmap v208: 97 pgs: 97 active+clean; 117 MiB data, 790 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:13:58.425 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:13:58 smithi134 ceph-mon[35037]: reconnect by client.35222 172.21.15.22:0/3729116781 after 0.00499993 2024-03-28T18:13:58.425 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:13:58 smithi134 ceph-mon[35037]: mds.? [v2:172.21.15.22:6834/3718238950,v1:172.21.15.22:6835/3718238950] up:reconnect 2024-03-28T18:13:58.425 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:13:58 smithi134 ceph-mon[35037]: fsmap cephfs:5/5 4 up:standby-replay 1 up:standby 1 up:reconnect 4 up:active 2024-03-28T18:13:58.425 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:13:58 smithi134 ceph-mon[35037]: pgmap v208: 97 pgs: 97 active+clean; 117 MiB data, 790 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:13:58.433 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:13:58 smithi079 ceph-mon[35312]: reconnect by client.35222 172.21.15.22:0/3729116781 after 0.00499993 2024-03-28T18:13:58.433 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:13:58 smithi079 ceph-mon[35312]: mds.? [v2:172.21.15.22:6834/3718238950,v1:172.21.15.22:6835/3718238950] up:reconnect 2024-03-28T18:13:58.433 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:13:58 smithi079 ceph-mon[35312]: fsmap cephfs:5/5 4 up:standby-replay 1 up:standby 1 up:reconnect 4 up:active 2024-03-28T18:13:58.433 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:13:58 smithi079 ceph-mon[35312]: pgmap v208: 97 pgs: 97 active+clean; 117 MiB data, 790 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:13:59.424 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:13:59 smithi134 ceph-mon[35037]: mds.? [v2:172.21.15.22:6834/3718238950,v1:172.21.15.22:6835/3718238950] up:rejoin 2024-03-28T18:13:59.425 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:13:59 smithi134 ceph-mon[35037]: fsmap cephfs:5/5 4 up:standby-replay 1 up:standby 1 up:rejoin 4 up:active 2024-03-28T18:13:59.425 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:13:59 smithi134 ceph-mon[35037]: daemon mds.j is now active in filesystem cephfs as rank 1 2024-03-28T18:13:59.433 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:13:59 smithi079 ceph-mon[35312]: mds.? [v2:172.21.15.22:6834/3718238950,v1:172.21.15.22:6835/3718238950] up:rejoin 2024-03-28T18:13:59.433 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:13:59 smithi079 ceph-mon[35312]: fsmap cephfs:5/5 4 up:standby-replay 1 up:standby 1 up:rejoin 4 up:active 2024-03-28T18:13:59.433 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:13:59 smithi079 ceph-mon[35312]: daemon mds.j is now active in filesystem cephfs as rank 1 2024-03-28T18:13:59.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:13:59 smithi022 ceph-mon[31985]: mds.? [v2:172.21.15.22:6834/3718238950,v1:172.21.15.22:6835/3718238950] up:rejoin 2024-03-28T18:13:59.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:13:59 smithi022 ceph-mon[31985]: fsmap cephfs:5/5 4 up:standby-replay 1 up:standby 1 up:rejoin 4 up:active 2024-03-28T18:13:59.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:13:59 smithi022 ceph-mon[31985]: daemon mds.j is now active in filesystem cephfs as rank 1 2024-03-28T18:13:59.533 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:13:59 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[72988]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-2145-g3be9a5c9/rpm/el8/BUILD/ceph-19.0.0-2145-g3be9a5c9/src/mds/Locker.cc: In function 'void Locker::set_xlocks_done(MutationImpl*, bool)' thread 7f4257800700 time 2024-03-28T18:13:59.242226+0000 2024-03-28T18:13:59.533 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:13:59 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[72988]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-2145-g3be9a5c9/rpm/el8/BUILD/ceph-19.0.0-2145-g3be9a5c9/src/mds/Locker.cc: 626: FAILED ceph_assert(obj->is_auth()) 2024-03-28T18:13:59.533 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:13:59 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[72988]: ceph version 19.0.0-2145-g3be9a5c9 (3be9a5c9bc793e11e2800a8c0c696e8b46742033) squid (dev) 2024-03-28T18:13:59.533 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:13:59 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[72988]: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x135) [0x7f42631954a1] 2024-03-28T18:13:59.533 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:13:59 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[72988]: 2: /usr/lib64/ceph/libceph-common.so.2(+0x2d2667) [0x7f4263195667] 2024-03-28T18:13:59.533 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:13:59 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[72988]: 3: (Locker::set_xlocks_done(MutationImpl*, bool)+0x493) [0x555783b37223] 2024-03-28T18:13:59.534 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:13:59 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[72988]: 4: (Server::early_reply(boost::intrusive_ptr const&, CInode*, CDentry*)+0x3e6) [0x5557839b15d6] 2024-03-28T18:13:59.534 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:13:59 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[72988]: 5: (Server::journal_and_reply(boost::intrusive_ptr const&, CInode*, CDentry*, LogEvent*, MDSLogContextBase*)+0xc1) [0x5557839b1da1] 2024-03-28T18:13:59.534 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:13:59 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[72988]: 6: (Server::handle_client_mkdir(boost::intrusive_ptr const&)+0x5ba) [0x5557839e76aa] 2024-03-28T18:13:59.534 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:13:59 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[72988]: 7: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0x733) [0x555783a04233] 2024-03-28T18:13:59.534 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:13:59 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[72988]: 8: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x555783a06197] 2024-03-28T18:13:59.534 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:13:59 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[72988]: 9: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x55578390511b] 2024-03-28T18:13:59.534 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:13:59 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[72988]: 10: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x5557839056bc] 2024-03-28T18:13:59.534 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:13:59 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[72988]: 11: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bf) [0x5557838ef1ff] 2024-03-28T18:13:59.534 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:13:59 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[72988]: 12: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x478) [0x7f4263427488] 2024-03-28T18:13:59.534 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:13:59 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[72988]: 13: (DispatchQueue::entry()+0x51f) [0x7f426342460f] 2024-03-28T18:13:59.534 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:13:59 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[72988]: 14: (DispatchQueue::DispatchThread::entry()+0x11) [0x7f42634f7e81] 2024-03-28T18:13:59.534 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:13:59 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[72988]: 15: /lib64/libpthread.so.0(+0x81ca) [0x7f4261f121ca] 2024-03-28T18:13:59.534 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:13:59 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[72988]: 16: clone() 2024-03-28T18:13:59.535 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:13:59 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[72988]: 2024-03-28T18:13:59.243+0000 7f4257800700 -1 /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-2145-g3be9a5c9/rpm/el8/BUILD/ceph-19.0.0-2145-g3be9a5c9/src/mds/Locker.cc: In function 'void Locker::set_xlocks_done(MutationImpl*, bool)' thread 7f4257800700 time 2024-03-28T18:13:59.242226+0000 2024-03-28T18:13:59.535 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:13:59 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[72988]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-2145-g3be9a5c9/rpm/el8/BUILD/ceph-19.0.0-2145-g3be9a5c9/src/mds/Locker.cc: 626: FAILED ceph_assert(obj->is_auth()) 2024-03-28T18:13:59.535 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:13:59 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[72988]: 2024-03-28T18:13:59.535 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:13:59 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[72988]: ceph version 19.0.0-2145-g3be9a5c9 (3be9a5c9bc793e11e2800a8c0c696e8b46742033) squid (dev) 2024-03-28T18:13:59.535 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:13:59 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[72988]: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x135) [0x7f42631954a1] 2024-03-28T18:13:59.535 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:13:59 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[72988]: 2: /usr/lib64/ceph/libceph-common.so.2(+0x2d2667) [0x7f4263195667] 2024-03-28T18:13:59.535 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:13:59 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[72988]: 3: (Locker::set_xlocks_done(MutationImpl*, bool)+0x493) [0x555783b37223] 2024-03-28T18:13:59.535 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:13:59 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[72988]: 4: (Server::early_reply(boost::intrusive_ptr const&, CInode*, CDentry*)+0x3e6) [0x5557839b15d6] 2024-03-28T18:13:59.535 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:13:59 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[72988]: 5: (Server::journal_and_reply(boost::intrusive_ptr const&, CInode*, CDentry*, LogEvent*, MDSLogContextBase*)+0xc1) [0x5557839b1da1] 2024-03-28T18:13:59.536 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:13:59 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[72988]: 6: (Server::handle_client_mkdir(boost::intrusive_ptr const&)+0x5ba) [0x5557839e76aa] 2024-03-28T18:13:59.536 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:13:59 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[72988]: 7: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0x733) [0x555783a04233] 2024-03-28T18:13:59.536 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:13:59 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[72988]: 8: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x555783a06197] 2024-03-28T18:13:59.536 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:13:59 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[72988]: 9: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x55578390511b] 2024-03-28T18:13:59.536 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:13:59 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[72988]: 10: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x5557839056bc] 2024-03-28T18:13:59.536 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:13:59 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[72988]: 11: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bf) [0x5557838ef1ff] 2024-03-28T18:13:59.536 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:13:59 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[72988]: 12: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x478) [0x7f4263427488] 2024-03-28T18:13:59.536 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:13:59 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[72988]: 13: (DispatchQueue::entry()+0x51f) [0x7f426342460f] 2024-03-28T18:13:59.536 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:13:59 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[72988]: 14: (DispatchQueue::DispatchThread::entry()+0x11) [0x7f42634f7e81] 2024-03-28T18:13:59.536 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:13:59 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[72988]: 15: /lib64/libpthread.so.0(+0x81ca) [0x7f4261f121ca] 2024-03-28T18:13:59.536 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:13:59 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[72988]: 16: clone() 2024-03-28T18:13:59.536 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:13:59 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[72988]: 2024-03-28T18:13:59.536 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:13:59 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[72988]: *** Caught signal (Aborted) ** 2024-03-28T18:13:59.536 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:13:59 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[72988]: in thread 7f4257800700 thread_name:ms_dispatch 2024-03-28T18:13:59.536 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:13:59 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[72988]: ceph version 19.0.0-2145-g3be9a5c9 (3be9a5c9bc793e11e2800a8c0c696e8b46742033) squid (dev) 2024-03-28T18:13:59.537 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:13:59 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[72988]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7f4261f1cd20] 2024-03-28T18:13:59.537 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:13:59 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[72988]: 2: gsignal() 2024-03-28T18:13:59.537 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:13:59 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[72988]: 3: abort() 2024-03-28T18:13:59.537 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:13:59 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[72988]: 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x18f) [0x7f42631954fb] 2024-03-28T18:13:59.537 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:13:59 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[72988]: 5: /usr/lib64/ceph/libceph-common.so.2(+0x2d2667) [0x7f4263195667] 2024-03-28T18:13:59.537 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:13:59 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[72988]: 6: (Locker::set_xlocks_done(MutationImpl*, bool)+0x493) [0x555783b37223] 2024-03-28T18:13:59.537 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:13:59 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[72988]: 7: (Server::early_reply(boost::intrusive_ptr const&, CInode*, CDentry*)+0x3e6) [0x5557839b15d6] 2024-03-28T18:13:59.537 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:13:59 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[72988]: 8: (Server::journal_and_reply(boost::intrusive_ptr const&, CInode*, CDentry*, LogEvent*, MDSLogContextBase*)+0xc1) [0x5557839b1da1] 2024-03-28T18:13:59.537 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:13:59 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[72988]: 9: (Server::handle_client_mkdir(boost::intrusive_ptr const&)+0x5ba) [0x5557839e76aa] 2024-03-28T18:13:59.537 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:13:59 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[72988]: 10: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0x733) [0x555783a04233] 2024-03-28T18:13:59.538 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:13:59 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[72988]: 11: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x555783a06197] 2024-03-28T18:13:59.538 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:13:59 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[72988]: 12: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x55578390511b] 2024-03-28T18:13:59.538 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:13:59 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[72988]: 13: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x5557839056bc] 2024-03-28T18:13:59.538 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:13:59 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[72988]: 14: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bf) [0x5557838ef1ff] 2024-03-28T18:13:59.538 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:13:59 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[72988]: 15: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x478) [0x7f4263427488] 2024-03-28T18:13:59.538 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:13:59 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[72988]: 16: (DispatchQueue::entry()+0x51f) [0x7f426342460f] 2024-03-28T18:13:59.538 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:13:59 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[72988]: 17: (DispatchQueue::DispatchThread::entry()+0x11) [0x7f42634f7e81] 2024-03-28T18:13:59.538 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:13:59 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[72988]: 18: /lib64/libpthread.so.0(+0x81ca) [0x7f4261f121ca] 2024-03-28T18:13:59.538 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:13:59 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[72988]: 19: clone() 2024-03-28T18:13:59.538 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:13:59 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[72988]: 2024-03-28T18:13:59.245+0000 7f4257800700 -1 *** Caught signal (Aborted) ** 2024-03-28T18:13:59.538 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:13:59 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[72988]: in thread 7f4257800700 thread_name:ms_dispatch 2024-03-28T18:13:59.538 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:13:59 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[72988]: 2024-03-28T18:13:59.538 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:13:59 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[72988]: ceph version 19.0.0-2145-g3be9a5c9 (3be9a5c9bc793e11e2800a8c0c696e8b46742033) squid (dev) 2024-03-28T18:13:59.538 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:13:59 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[72988]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7f4261f1cd20] 2024-03-28T18:13:59.538 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:13:59 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[72988]: 2: gsignal() 2024-03-28T18:13:59.539 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:13:59 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[72988]: 3: abort() 2024-03-28T18:13:59.539 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:13:59 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[72988]: 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x18f) [0x7f42631954fb] 2024-03-28T18:13:59.539 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:13:59 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[72988]: 5: /usr/lib64/ceph/libceph-common.so.2(+0x2d2667) [0x7f4263195667] 2024-03-28T18:13:59.539 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:13:59 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[72988]: 6: (Locker::set_xlocks_done(MutationImpl*, bool)+0x493) [0x555783b37223] 2024-03-28T18:13:59.539 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:13:59 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[72988]: 7: (Server::early_reply(boost::intrusive_ptr const&, CInode*, CDentry*)+0x3e6) [0x5557839b15d6] 2024-03-28T18:13:59.539 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:13:59 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[72988]: 8: (Server::journal_and_reply(boost::intrusive_ptr const&, CInode*, CDentry*, LogEvent*, MDSLogContextBase*)+0xc1) [0x5557839b1da1] 2024-03-28T18:13:59.539 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:13:59 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[72988]: 9: (Server::handle_client_mkdir(boost::intrusive_ptr const&)+0x5ba) [0x5557839e76aa] 2024-03-28T18:13:59.539 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:13:59 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[72988]: 10: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0x733) [0x555783a04233] 2024-03-28T18:13:59.539 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:13:59 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[72988]: 11: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x555783a06197] 2024-03-28T18:13:59.539 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:13:59 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[72988]: 12: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x55578390511b] 2024-03-28T18:13:59.540 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:13:59 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[72988]: 13: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x5557839056bc] 2024-03-28T18:13:59.540 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:13:59 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[72988]: 14: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bf) [0x5557838ef1ff] 2024-03-28T18:13:59.540 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:13:59 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[72988]: 15: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x478) [0x7f4263427488] 2024-03-28T18:13:59.540 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:13:59 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[72988]: 16: (DispatchQueue::entry()+0x51f) [0x7f426342460f] 2024-03-28T18:13:59.540 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:13:59 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[72988]: 17: (DispatchQueue::DispatchThread::entry()+0x11) [0x7f42634f7e81] 2024-03-28T18:13:59.540 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:13:59 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[72988]: 18: /lib64/libpthread.so.0(+0x81ca) [0x7f4261f121ca] 2024-03-28T18:13:59.540 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:13:59 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[72988]: 19: clone() 2024-03-28T18:13:59.540 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:13:59 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[72988]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-03-28T18:13:59.540 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:13:59 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[72988]: 2024-03-28T18:13:59.540 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:13:59 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[72988]: -1> 2024-03-28T18:13:59.243+0000 7f4257800700 -1 /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-2145-g3be9a5c9/rpm/el8/BUILD/ceph-19.0.0-2145-g3be9a5c9/src/mds/Locker.cc: In function 'void Locker::set_xlocks_done(MutationImpl*, bool)' thread 7f4257800700 time 2024-03-28T18:13:59.242226+0000 2024-03-28T18:13:59.540 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:13:59 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[72988]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-2145-g3be9a5c9/rpm/el8/BUILD/ceph-19.0.0-2145-g3be9a5c9/src/mds/Locker.cc: 626: FAILED ceph_assert(obj->is_auth()) 2024-03-28T18:13:59.540 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:13:59 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[72988]: 2024-03-28T18:13:59.541 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:13:59 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[72988]: ceph version 19.0.0-2145-g3be9a5c9 (3be9a5c9bc793e11e2800a8c0c696e8b46742033) squid (dev) 2024-03-28T18:13:59.541 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:13:59 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[72988]: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x135) [0x7f42631954a1] 2024-03-28T18:13:59.541 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:13:59 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[72988]: 2: /usr/lib64/ceph/libceph-common.so.2(+0x2d2667) [0x7f4263195667] 2024-03-28T18:13:59.541 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:13:59 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[72988]: 3: (Locker::set_xlocks_done(MutationImpl*, bool)+0x493) [0x555783b37223] 2024-03-28T18:13:59.541 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:13:59 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[72988]: 4: (Server::early_reply(boost::intrusive_ptr const&, CInode*, CDentry*)+0x3e6) [0x5557839b15d6] 2024-03-28T18:13:59.541 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:13:59 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[72988]: 5: (Server::journal_and_reply(boost::intrusive_ptr const&, CInode*, CDentry*, LogEvent*, MDSLogContextBase*)+0xc1) [0x5557839b1da1] 2024-03-28T18:13:59.541 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:13:59 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[72988]: 6: (Server::handle_client_mkdir(boost::intrusive_ptr const&)+0x5ba) [0x5557839e76aa] 2024-03-28T18:13:59.542 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:13:59 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[72988]: 7: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0x733) [0x555783a04233] 2024-03-28T18:13:59.542 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:13:59 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[72988]: 8: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x555783a06197] 2024-03-28T18:13:59.542 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:13:59 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[72988]: 9: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x55578390511b] 2024-03-28T18:13:59.542 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:13:59 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[72988]: 10: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x5557839056bc] 2024-03-28T18:13:59.542 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:13:59 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[72988]: 11: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bf) [0x5557838ef1ff] 2024-03-28T18:13:59.542 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:13:59 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[72988]: 12: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x478) [0x7f4263427488] 2024-03-28T18:13:59.542 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:13:59 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[72988]: 13: (DispatchQueue::entry()+0x51f) [0x7f426342460f] 2024-03-28T18:13:59.542 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:13:59 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[72988]: 14: (DispatchQueue::DispatchThread::entry()+0x11) [0x7f42634f7e81] 2024-03-28T18:13:59.542 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:13:59 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[72988]: 15: /lib64/libpthread.so.0(+0x81ca) [0x7f4261f121ca] 2024-03-28T18:13:59.542 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:13:59 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[72988]: 16: clone() 2024-03-28T18:13:59.542 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:13:59 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[72988]: 2024-03-28T18:13:59.542 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:13:59 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[72988]: 0> 2024-03-28T18:13:59.245+0000 7f4257800700 -1 *** Caught signal (Aborted) ** 2024-03-28T18:13:59.542 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:13:59 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[72988]: in thread 7f4257800700 thread_name:ms_dispatch 2024-03-28T18:13:59.543 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:13:59 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[72988]: 2024-03-28T18:13:59.543 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:13:59 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[72988]: ceph version 19.0.0-2145-g3be9a5c9 (3be9a5c9bc793e11e2800a8c0c696e8b46742033) squid (dev) 2024-03-28T18:13:59.543 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:13:59 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[72988]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7f4261f1cd20] 2024-03-28T18:13:59.543 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:13:59 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[72988]: 2: gsignal() 2024-03-28T18:13:59.543 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:13:59 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[72988]: 3: abort() 2024-03-28T18:13:59.543 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:13:59 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[72988]: 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x18f) [0x7f42631954fb] 2024-03-28T18:13:59.543 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:13:59 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[72988]: 5: /usr/lib64/ceph/libceph-common.so.2(+0x2d2667) [0x7f4263195667] 2024-03-28T18:13:59.543 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:13:59 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[72988]: 6: (Locker::set_xlocks_done(MutationImpl*, bool)+0x493) [0x555783b37223] 2024-03-28T18:13:59.543 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:13:59 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[72988]: 7: (Server::early_reply(boost::intrusive_ptr const&, CInode*, CDentry*)+0x3e6) [0x5557839b15d6] 2024-03-28T18:13:59.543 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:13:59 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[72988]: 8: (Server::journal_and_reply(boost::intrusive_ptr const&, CInode*, CDentry*, LogEvent*, MDSLogContextBase*)+0xc1) [0x5557839b1da1] 2024-03-28T18:13:59.543 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:13:59 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[72988]: 9: (Server::handle_client_mkdir(boost::intrusive_ptr const&)+0x5ba) [0x5557839e76aa] 2024-03-28T18:13:59.543 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:13:59 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[72988]: 10: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0x733) [0x555783a04233] 2024-03-28T18:13:59.543 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:13:59 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[72988]: 11: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x555783a06197] 2024-03-28T18:13:59.543 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:13:59 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[72988]: 12: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x55578390511b] 2024-03-28T18:13:59.543 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:13:59 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[72988]: 13: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x5557839056bc] 2024-03-28T18:13:59.543 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:13:59 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[72988]: 14: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bf) [0x5557838ef1ff] 2024-03-28T18:13:59.544 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:13:59 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[72988]: 15: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x478) [0x7f4263427488] 2024-03-28T18:13:59.544 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:13:59 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[72988]: 16: (DispatchQueue::entry()+0x51f) [0x7f426342460f] 2024-03-28T18:13:59.544 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:13:59 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[72988]: 17: (DispatchQueue::DispatchThread::entry()+0x11) [0x7f42634f7e81] 2024-03-28T18:13:59.544 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:13:59 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[72988]: 18: /lib64/libpthread.so.0(+0x81ca) [0x7f4261f121ca] 2024-03-28T18:13:59.544 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:13:59 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[72988]: 19: clone() 2024-03-28T18:13:59.544 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:13:59 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[72988]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-03-28T18:13:59.544 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:13:59 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[72988]: 2024-03-28T18:13:59.544 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:13:59 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[72988]: -1> 2024-03-28T18:13:59.243+0000 7f4257800700 -1 /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-2145-g3be9a5c9/rpm/el8/BUILD/ceph-19.0.0-2145-g3be9a5c9/src/mds/Locker.cc: In function 'void Locker::set_xlocks_done(MutationImpl*, bool)' thread 7f4257800700 time 2024-03-28T18:13:59.242226+0000 2024-03-28T18:13:59.544 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:13:59 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[72988]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-2145-g3be9a5c9/rpm/el8/BUILD/ceph-19.0.0-2145-g3be9a5c9/src/mds/Locker.cc: 626: FAILED ceph_assert(obj->is_auth()) 2024-03-28T18:13:59.544 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:13:59 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[72988]: 2024-03-28T18:13:59.544 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:13:59 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[72988]: ceph version 19.0.0-2145-g3be9a5c9 (3be9a5c9bc793e11e2800a8c0c696e8b46742033) squid (dev) 2024-03-28T18:13:59.544 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:13:59 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[72988]: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x135) [0x7f42631954a1] 2024-03-28T18:13:59.544 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:13:59 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[72988]: 2: /usr/lib64/ceph/libceph-common.so.2(+0x2d2667) [0x7f4263195667] 2024-03-28T18:13:59.544 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:13:59 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[72988]: 3: (Locker::set_xlocks_done(MutationImpl*, bool)+0x493) [0x555783b37223] 2024-03-28T18:13:59.544 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:13:59 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[72988]: 4: (Server::early_reply(boost::intrusive_ptr const&, CInode*, CDentry*)+0x3e6) [0x5557839b15d6] 2024-03-28T18:13:59.545 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:13:59 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[72988]: 5: (Server::journal_and_reply(boost::intrusive_ptr const&, CInode*, CDentry*, LogEvent*, MDSLogContextBase*)+0xc1) [0x5557839b1da1] 2024-03-28T18:13:59.545 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:13:59 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[72988]: 6: (Server::handle_client_mkdir(boost::intrusive_ptr const&)+0x5ba) [0x5557839e76aa] 2024-03-28T18:13:59.545 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:13:59 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[72988]: 7: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0x733) [0x555783a04233] 2024-03-28T18:13:59.545 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:13:59 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[72988]: 8: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x555783a06197] 2024-03-28T18:13:59.545 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:13:59 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[72988]: 9: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x55578390511b] 2024-03-28T18:13:59.545 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:13:59 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[72988]: 10: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x5557839056bc] 2024-03-28T18:13:59.545 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:13:59 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[72988]: 11: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bf) [0x5557838ef1ff] 2024-03-28T18:13:59.545 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:13:59 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[72988]: 12: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x478) [0x7f4263427488] 2024-03-28T18:13:59.546 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:13:59 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[72988]: 13: (DispatchQueue::entry()+0x51f) [0x7f426342460f] 2024-03-28T18:13:59.546 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:13:59 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[72988]: 14: (DispatchQueue::DispatchThread::entry()+0x11) [0x7f42634f7e81] 2024-03-28T18:13:59.546 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:13:59 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[72988]: 15: /lib64/libpthread.so.0(+0x81ca) [0x7f4261f121ca] 2024-03-28T18:13:59.546 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:13:59 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[72988]: 16: clone() 2024-03-28T18:13:59.546 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:13:59 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[72988]: 2024-03-28T18:13:59.546 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:13:59 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[72988]: 0> 2024-03-28T18:13:59.245+0000 7f4257800700 -1 *** Caught signal (Aborted) ** 2024-03-28T18:13:59.546 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:13:59 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[72988]: in thread 7f4257800700 thread_name:ms_dispatch 2024-03-28T18:13:59.546 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:13:59 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[72988]: 2024-03-28T18:13:59.546 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:13:59 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[72988]: ceph version 19.0.0-2145-g3be9a5c9 (3be9a5c9bc793e11e2800a8c0c696e8b46742033) squid (dev) 2024-03-28T18:13:59.546 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:13:59 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[72988]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7f4261f1cd20] 2024-03-28T18:13:59.546 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:13:59 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[72988]: 2: gsignal() 2024-03-28T18:13:59.546 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:13:59 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[72988]: 3: abort() 2024-03-28T18:13:59.546 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:13:59 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[72988]: 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x18f) [0x7f42631954fb] 2024-03-28T18:13:59.546 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:13:59 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[72988]: 5: /usr/lib64/ceph/libceph-common.so.2(+0x2d2667) [0x7f4263195667] 2024-03-28T18:13:59.546 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:13:59 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[72988]: 6: (Locker::set_xlocks_done(MutationImpl*, bool)+0x493) [0x555783b37223] 2024-03-28T18:13:59.547 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:13:59 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[72988]: 7: (Server::early_reply(boost::intrusive_ptr const&, CInode*, CDentry*)+0x3e6) [0x5557839b15d6] 2024-03-28T18:13:59.547 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:13:59 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[72988]: 8: (Server::journal_and_reply(boost::intrusive_ptr const&, CInode*, CDentry*, LogEvent*, MDSLogContextBase*)+0xc1) [0x5557839b1da1] 2024-03-28T18:13:59.547 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:13:59 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[72988]: 9: (Server::handle_client_mkdir(boost::intrusive_ptr const&)+0x5ba) [0x5557839e76aa] 2024-03-28T18:13:59.547 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:13:59 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[72988]: 10: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0x733) [0x555783a04233] 2024-03-28T18:13:59.547 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:13:59 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[72988]: 11: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x555783a06197] 2024-03-28T18:13:59.547 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:13:59 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[72988]: 12: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x55578390511b] 2024-03-28T18:13:59.547 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:13:59 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[72988]: 13: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x5557839056bc] 2024-03-28T18:13:59.547 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:13:59 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[72988]: 14: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bf) [0x5557838ef1ff] 2024-03-28T18:13:59.547 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:13:59 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[72988]: 15: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x478) [0x7f4263427488] 2024-03-28T18:13:59.547 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:13:59 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[72988]: 16: (DispatchQueue::entry()+0x51f) [0x7f426342460f] 2024-03-28T18:13:59.547 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:13:59 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[72988]: 17: (DispatchQueue::DispatchThread::entry()+0x11) [0x7f42634f7e81] 2024-03-28T18:13:59.547 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:13:59 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[72988]: 18: /lib64/libpthread.so.0(+0x81ca) [0x7f4261f121ca] 2024-03-28T18:13:59.547 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:13:59 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[72988]: 19: clone() 2024-03-28T18:13:59.547 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:13:59 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[72988]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-03-28T18:13:59.547 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:13:59 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[72988]: 2024-03-28T18:13:59.548 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:13:59 smithi022 podman[93495]: 2024-03-28 18:13:59.399707939 +0000 UTC m=+0.039046855 container died 43ead9efefabdd2ee634528651fdc921807a64f9b114b59fe1378b9d713e6a84 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.build-date=20240319, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.vendor=CentOS, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-28T18:14:00.425 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:14:00 smithi134 ceph-mon[35037]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-03-28T18:14:00.425 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:14:00 smithi134 ceph-mon[35037]: Cluster is now healthy 2024-03-28T18:14:00.425 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:14:00 smithi134 ceph-mon[35037]: pgmap v209: 97 pgs: 97 active+clean; 117 MiB data, 794 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 409 B/s wr, 10 op/s 2024-03-28T18:14:00.425 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:14:00 smithi134 ceph-mon[35037]: mds.? [v2:172.21.15.22:6834/3718238950,v1:172.21.15.22:6835/3718238950] up:active 2024-03-28T18:14:00.425 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:14:00 smithi134 ceph-mon[35037]: fsmap cephfs:5 4 up:standby-replay 1 up:standby 5 up:active 2024-03-28T18:14:00.425 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:14:00 smithi134 ceph-mon[35037]: fsmap cephfs:5 5 up:standby-replay 5 up:active 2024-03-28T18:14:00.432 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:14:00 smithi079 ceph-mon[35312]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-03-28T18:14:00.432 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:14:00 smithi079 ceph-mon[35312]: Cluster is now healthy 2024-03-28T18:14:00.433 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:14:00 smithi079 ceph-mon[35312]: pgmap v209: 97 pgs: 97 active+clean; 117 MiB data, 794 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 409 B/s wr, 10 op/s 2024-03-28T18:14:00.433 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:14:00 smithi079 ceph-mon[35312]: mds.? [v2:172.21.15.22:6834/3718238950,v1:172.21.15.22:6835/3718238950] up:active 2024-03-28T18:14:00.433 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:14:00 smithi079 ceph-mon[35312]: fsmap cephfs:5 4 up:standby-replay 1 up:standby 5 up:active 2024-03-28T18:14:00.433 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:14:00 smithi079 ceph-mon[35312]: fsmap cephfs:5 5 up:standby-replay 5 up:active 2024-03-28T18:14:00.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:14:00 smithi022 ceph-mon[31985]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-03-28T18:14:00.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:14:00 smithi022 ceph-mon[31985]: Cluster is now healthy 2024-03-28T18:14:00.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:14:00 smithi022 ceph-mon[31985]: pgmap v209: 97 pgs: 97 active+clean; 117 MiB data, 794 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 409 B/s wr, 10 op/s 2024-03-28T18:14:00.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:14:00 smithi022 ceph-mon[31985]: mds.? [v2:172.21.15.22:6834/3718238950,v1:172.21.15.22:6835/3718238950] up:active 2024-03-28T18:14:00.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:14:00 smithi022 ceph-mon[31985]: fsmap cephfs:5 4 up:standby-replay 1 up:standby 5 up:active 2024-03-28T18:14:00.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:14:00 smithi022 ceph-mon[31985]: fsmap cephfs:5 5 up:standby-replay 5 up:active 2024-03-28T18:14:00.533 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:14:00 smithi022 podman[93495]: 2024-03-28 18:14:00.139432319 +0000 UTC m=+0.778771238 container remove 43ead9efefabdd2ee634528651fdc921807a64f9b114b59fe1378b9d713e6a84 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j, org.label-schema.build-date=20240319, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_BRANCH=HEAD, ceph=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=) 2024-03-28T18:14:00.533 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:14:00 smithi022 systemd[1]: ceph-fd220748-ed2b-11ee-95d0-87774f69a715@mds.j.service: Main process exited, code=exited, status=134/n/a 2024-03-28T18:14:01.032 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:14:00 smithi022 systemd[1]: ceph-fd220748-ed2b-11ee-95d0-87774f69a715@mds.j.service: Failed with result 'exit-code'. 2024-03-28T18:14:01.032 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:14:00 smithi022 systemd[1]: ceph-fd220748-ed2b-11ee-95d0-87774f69a715@mds.j.service: Consumed 2.351s CPU time. 2024-03-28T18:14:02.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:14:02 smithi022 ceph-mon[31985]: pgmap v210: 97 pgs: 97 active+clean; 117 MiB data, 794 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 409 B/s wr, 10 op/s 2024-03-28T18:14:02.674 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:14:02 smithi134 ceph-mon[35037]: pgmap v210: 97 pgs: 97 active+clean; 117 MiB data, 794 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 409 B/s wr, 10 op/s 2024-03-28T18:14:02.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:14:02 smithi079 ceph-mon[35312]: pgmap v210: 97 pgs: 97 active+clean; 117 MiB data, 794 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 409 B/s wr, 10 op/s 2024-03-28T18:14:05.174 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:14:04 smithi134 ceph-mon[35037]: pgmap v211: 97 pgs: 97 active+clean; 117 MiB data, 794 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 409 B/s wr, 10 op/s 2024-03-28T18:14:05.182 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:14:04 smithi079 ceph-mon[35312]: pgmap v211: 97 pgs: 97 active+clean; 117 MiB data, 794 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 409 B/s wr, 10 op/s 2024-03-28T18:14:05.282 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:14:04 smithi022 ceph-mon[31985]: pgmap v211: 97 pgs: 97 active+clean; 117 MiB data, 794 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 409 B/s wr, 10 op/s 2024-03-28T18:14:07.174 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:14:06 smithi134 ceph-mon[35037]: pgmap v212: 97 pgs: 97 active+clean; 117 MiB data, 774 MiB used, 1.0 TiB / 1.0 TiB avail; 109 KiB/s rd, 3.8 KiB/s wr, 27 op/s 2024-03-28T18:14:07.182 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:14:06 smithi079 ceph-mon[35312]: pgmap v212: 97 pgs: 97 active+clean; 117 MiB data, 774 MiB used, 1.0 TiB / 1.0 TiB avail; 109 KiB/s rd, 3.8 KiB/s wr, 27 op/s 2024-03-28T18:14:07.282 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:14:06 smithi022 ceph-mon[31985]: pgmap v212: 97 pgs: 97 active+clean; 117 MiB data, 774 MiB used, 1.0 TiB / 1.0 TiB avail; 109 KiB/s rd, 3.8 KiB/s wr, 27 op/s 2024-03-28T18:14:09.185 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:14:08 smithi079 ceph-mon[35312]: pgmap v213: 97 pgs: 97 active+clean; 117 MiB data, 774 MiB used, 1.0 TiB / 1.0 TiB avail; 91 KiB/s rd, 3.2 KiB/s wr, 22 op/s 2024-03-28T18:14:09.282 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:14:08 smithi022 ceph-mon[31985]: pgmap v213: 97 pgs: 97 active+clean; 117 MiB data, 774 MiB used, 1.0 TiB / 1.0 TiB avail; 91 KiB/s rd, 3.2 KiB/s wr, 22 op/s 2024-03-28T18:14:09.424 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:14:08 smithi134 ceph-mon[35037]: pgmap v213: 97 pgs: 97 active+clean; 117 MiB data, 774 MiB used, 1.0 TiB / 1.0 TiB avail; 91 KiB/s rd, 3.2 KiB/s wr, 22 op/s 2024-03-28T18:14:11.180 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:14:10 smithi022 ceph-mon[31985]: pgmap v214: 97 pgs: 97 active+clean; 117 MiB data, 774 MiB used, 1.0 TiB / 1.0 TiB avail; 93 KiB/s rd, 3.2 KiB/s wr, 27 op/s 2024-03-28T18:14:11.181 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:14:10 smithi022 systemd[1]: ceph-fd220748-ed2b-11ee-95d0-87774f69a715@mds.j.service: Scheduled restart job, restart counter is at 1. 2024-03-28T18:14:11.181 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:14:10 smithi022 systemd[1]: Stopped Ceph mds.j for fd220748-ed2b-11ee-95d0-87774f69a715. 2024-03-28T18:14:11.181 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:14:10 smithi022 systemd[1]: ceph-fd220748-ed2b-11ee-95d0-87774f69a715@mds.j.service: Consumed 2.351s CPU time. 2024-03-28T18:14:11.181 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:14:10 smithi022 systemd[1]: Starting Ceph mds.j for fd220748-ed2b-11ee-95d0-87774f69a715... 2024-03-28T18:14:11.182 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:14:10 smithi079 ceph-mon[35312]: pgmap v214: 97 pgs: 97 active+clean; 117 MiB data, 774 MiB used, 1.0 TiB / 1.0 TiB avail; 93 KiB/s rd, 3.2 KiB/s wr, 27 op/s 2024-03-28T18:14:11.424 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:14:10 smithi134 ceph-mon[35037]: pgmap v214: 97 pgs: 97 active+clean; 117 MiB data, 774 MiB used, 1.0 TiB / 1.0 TiB avail; 93 KiB/s rd, 3.2 KiB/s wr, 27 op/s 2024-03-28T18:14:11.512 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:14:11 smithi022 podman[93589]: 2024-03-28 18:14:11.087452295 +0000 UTC m=+0.036663358 image pull c664afe56bb01db4f9afa9532c20598e201e664f2e9ba2be50db0d52674a26d2 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc 2024-03-28T18:14:11.537 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:14:11 smithi022 podman[93589]: 2024-03-28 18:14:11.207344742 +0000 UTC m=+0.156555835 container create ad8b1f516a881f2fb3d6cc6bc82e170b606fae9a1e44359bbfd6d3cdb8d5e15b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j, org.label-schema.build-date=20240319, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_CLEAN=True) 2024-03-28T18:14:11.782 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:14:11 smithi022 podman[93589]: 2024-03-28 18:14:11.511981364 +0000 UTC m=+0.461192464 container init ad8b1f516a881f2fb3d6cc6bc82e170b606fae9a1e44359bbfd6d3cdb8d5e15b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, ceph=True, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.build-date=20240319) 2024-03-28T18:14:11.783 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:14:11 smithi022 podman[93589]: 2024-03-28 18:14:11.521923593 +0000 UTC m=+0.471134692 container start ad8b1f516a881f2fb3d6cc6bc82e170b606fae9a1e44359bbfd6d3cdb8d5e15b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, org.label-schema.build-date=20240319, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, ceph=True, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux ) 2024-03-28T18:14:11.783 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:14:11 smithi022 ceph-mds[93607]: -- 172.21.15.22:0/2824801662 <== mon.2 v2:172.21.15.79:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x561ef9457a40 con 0x561ef942b000 2024-03-28T18:14:11.783 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:14:11 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[93603]: starting mds.j at 2024-03-28T18:14:11.783 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:14:11 smithi022 bash[93589]: ad8b1f516a881f2fb3d6cc6bc82e170b606fae9a1e44359bbfd6d3cdb8d5e15b 2024-03-28T18:14:11.783 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:14:11 smithi022 systemd[1]: Started Ceph mds.j for fd220748-ed2b-11ee-95d0-87774f69a715. 2024-03-28T18:14:12.183 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:14:11 smithi079 ceph-mon[35312]: daemon mds.j restarted 2024-03-28T18:14:12.183 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:14:11 smithi079 ceph-mon[35312]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-03-28T18:14:12.183 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:14:11 smithi079 ceph-mon[35312]: osdmap e107: 12 total, 12 up, 12 in 2024-03-28T18:14:12.183 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:14:11 smithi079 ceph-mon[35312]: mds.? [v2:172.21.15.22:6834/803375445,v1:172.21.15.22:6835/803375445] up:boot 2024-03-28T18:14:12.183 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:14:11 smithi079 ceph-mon[35312]: Standby daemon mds.d assigned to filesystem cephfs as rank 1 2024-03-28T18:14:12.183 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:14:11 smithi079 ceph-mon[35312]: fsmap cephfs:4/5 {0=b=up:active,2=e=up:active,3=k=up:active,4=i=up:active} 5 up:standby-replay 1 up:standby, 1 failed 2024-03-28T18:14:12.183 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:14:11 smithi079 ceph-mon[35312]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-03-28T18:14:12.183 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:14:11 smithi079 ceph-mon[35312]: fsmap cephfs:5/5 4 up:standby-replay 1 up:standby 1 up:replay 4 up:active 2024-03-28T18:14:12.282 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:14:11 smithi022 ceph-mon[31985]: daemon mds.j restarted 2024-03-28T18:14:12.282 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:14:11 smithi022 ceph-mon[31985]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-03-28T18:14:12.283 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:14:11 smithi022 ceph-mon[31985]: osdmap e107: 12 total, 12 up, 12 in 2024-03-28T18:14:12.283 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:14:11 smithi022 ceph-mon[31985]: mds.? [v2:172.21.15.22:6834/803375445,v1:172.21.15.22:6835/803375445] up:boot 2024-03-28T18:14:12.283 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:14:11 smithi022 ceph-mon[31985]: Standby daemon mds.d assigned to filesystem cephfs as rank 1 2024-03-28T18:14:12.283 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:14:11 smithi022 ceph-mon[31985]: fsmap cephfs:4/5 {0=b=up:active,2=e=up:active,3=k=up:active,4=i=up:active} 5 up:standby-replay 1 up:standby, 1 failed 2024-03-28T18:14:12.283 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:14:11 smithi022 ceph-mon[31985]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-03-28T18:14:12.283 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:14:11 smithi022 ceph-mon[31985]: fsmap cephfs:5/5 4 up:standby-replay 1 up:standby 1 up:replay 4 up:active 2024-03-28T18:14:12.425 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:14:11 smithi134 ceph-mon[35037]: daemon mds.j restarted 2024-03-28T18:14:12.425 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:14:11 smithi134 ceph-mon[35037]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-03-28T18:14:12.425 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:14:11 smithi134 ceph-mon[35037]: osdmap e107: 12 total, 12 up, 12 in 2024-03-28T18:14:12.425 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:14:11 smithi134 ceph-mon[35037]: mds.? [v2:172.21.15.22:6834/803375445,v1:172.21.15.22:6835/803375445] up:boot 2024-03-28T18:14:12.425 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:14:11 smithi134 ceph-mon[35037]: Standby daemon mds.d assigned to filesystem cephfs as rank 1 2024-03-28T18:14:12.425 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:14:11 smithi134 ceph-mon[35037]: fsmap cephfs:4/5 {0=b=up:active,2=e=up:active,3=k=up:active,4=i=up:active} 5 up:standby-replay 1 up:standby, 1 failed 2024-03-28T18:14:12.425 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:14:11 smithi134 ceph-mon[35037]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-03-28T18:14:12.425 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:14:11 smithi134 ceph-mon[35037]: fsmap cephfs:5/5 4 up:standby-replay 1 up:standby 1 up:replay 4 up:active 2024-03-28T18:14:13.282 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:14:12 smithi022 ceph-mon[31985]: pgmap v215: 97 pgs: 97 active+clean; 117 MiB data, 774 MiB used, 1.0 TiB / 1.0 TiB avail; 90 KiB/s rd, 2.8 KiB/s wr, 22 op/s 2024-03-28T18:14:13.424 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:14:12 smithi134 ceph-mon[35037]: pgmap v215: 97 pgs: 97 active+clean; 117 MiB data, 774 MiB used, 1.0 TiB / 1.0 TiB avail; 90 KiB/s rd, 2.8 KiB/s wr, 22 op/s 2024-03-28T18:14:13.432 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:14:12 smithi079 ceph-mon[35312]: pgmap v215: 97 pgs: 97 active+clean; 117 MiB data, 774 MiB used, 1.0 TiB / 1.0 TiB avail; 90 KiB/s rd, 2.8 KiB/s wr, 22 op/s 2024-03-28T18:14:14.282 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:14:13 smithi022 ceph-mon[31985]: mds.? [v2:172.21.15.22:6836/3983083841,v1:172.21.15.22:6837/3983083841] up:resolve 2024-03-28T18:14:14.282 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:14:13 smithi022 ceph-mon[31985]: fsmap cephfs:5/5 4 up:standby-replay 1 up:standby 1 up:resolve 4 up:active 2024-03-28T18:14:14.424 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:14:13 smithi134 ceph-mon[35037]: mds.? [v2:172.21.15.22:6836/3983083841,v1:172.21.15.22:6837/3983083841] up:resolve 2024-03-28T18:14:14.425 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:14:13 smithi134 ceph-mon[35037]: fsmap cephfs:5/5 4 up:standby-replay 1 up:standby 1 up:resolve 4 up:active 2024-03-28T18:14:14.432 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:14:13 smithi079 ceph-mon[35312]: mds.? [v2:172.21.15.22:6836/3983083841,v1:172.21.15.22:6837/3983083841] up:resolve 2024-03-28T18:14:14.432 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:14:13 smithi079 ceph-mon[35312]: fsmap cephfs:5/5 4 up:standby-replay 1 up:standby 1 up:resolve 4 up:active 2024-03-28T18:14:15.282 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:14:14 smithi022 ceph-mon[31985]: pgmap v217: 97 pgs: 97 active+clean; 117 MiB data, 774 MiB used, 1.0 TiB / 1.0 TiB avail; 108 KiB/s rd, 3.4 KiB/s wr, 27 op/s 2024-03-28T18:14:15.282 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:14:14 smithi022 ceph-mon[31985]: reconnect by client.35222 172.21.15.22:0/3729116781 after 0.00299996 2024-03-28T18:14:15.282 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:14:14 smithi022 ceph-mon[31985]: mds.? [v2:172.21.15.22:6836/3983083841,v1:172.21.15.22:6837/3983083841] up:reconnect 2024-03-28T18:14:15.282 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:14:14 smithi022 ceph-mon[31985]: fsmap cephfs:5/5 4 up:standby-replay 1 up:standby 1 up:reconnect 4 up:active 2024-03-28T18:14:15.425 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:14:14 smithi134 ceph-mon[35037]: pgmap v217: 97 pgs: 97 active+clean; 117 MiB data, 774 MiB used, 1.0 TiB / 1.0 TiB avail; 108 KiB/s rd, 3.4 KiB/s wr, 27 op/s 2024-03-28T18:14:15.425 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:14:14 smithi134 ceph-mon[35037]: reconnect by client.35222 172.21.15.22:0/3729116781 after 0.00299996 2024-03-28T18:14:15.425 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:14:14 smithi134 ceph-mon[35037]: mds.? [v2:172.21.15.22:6836/3983083841,v1:172.21.15.22:6837/3983083841] up:reconnect 2024-03-28T18:14:15.425 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:14:14 smithi134 ceph-mon[35037]: fsmap cephfs:5/5 4 up:standby-replay 1 up:standby 1 up:reconnect 4 up:active 2024-03-28T18:14:15.432 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:14:14 smithi079 ceph-mon[35312]: pgmap v217: 97 pgs: 97 active+clean; 117 MiB data, 774 MiB used, 1.0 TiB / 1.0 TiB avail; 108 KiB/s rd, 3.4 KiB/s wr, 27 op/s 2024-03-28T18:14:15.432 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:14:14 smithi079 ceph-mon[35312]: reconnect by client.35222 172.21.15.22:0/3729116781 after 0.00299996 2024-03-28T18:14:15.432 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:14:14 smithi079 ceph-mon[35312]: mds.? [v2:172.21.15.22:6836/3983083841,v1:172.21.15.22:6837/3983083841] up:reconnect 2024-03-28T18:14:15.432 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:14:14 smithi079 ceph-mon[35312]: fsmap cephfs:5/5 4 up:standby-replay 1 up:standby 1 up:reconnect 4 up:active 2024-03-28T18:14:16.282 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:14:15 smithi022 ceph-mon[31985]: mds.? [v2:172.21.15.22:6836/3983083841,v1:172.21.15.22:6837/3983083841] up:rejoin 2024-03-28T18:14:16.282 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:14:15 smithi022 ceph-mon[31985]: fsmap cephfs:5/5 4 up:standby-replay 1 up:standby 1 up:rejoin 4 up:active 2024-03-28T18:14:16.282 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:14:15 smithi022 ceph-mon[31985]: daemon mds.d is now active in filesystem cephfs as rank 1 2024-03-28T18:14:16.282 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:14:15 smithi022 ceph-mon[31985]: pgmap v218: 97 pgs: 97 active+clean; 117 MiB data, 774 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 204 B/s wr, 10 op/s 2024-03-28T18:14:16.283 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:14:16 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[93444]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-2145-g3be9a5c9/rpm/el8/BUILD/ceph-19.0.0-2145-g3be9a5c9/src/mds/Locker.cc: In function 'void Locker::set_xlocks_done(MutationImpl*, bool)' thread 7fddc1000700 time 2024-03-28T18:14:16.078982+0000 2024-03-28T18:14:16.283 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:14:16 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[93444]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-2145-g3be9a5c9/rpm/el8/BUILD/ceph-19.0.0-2145-g3be9a5c9/src/mds/Locker.cc: 626: FAILED ceph_assert(obj->is_auth()) 2024-03-28T18:14:16.284 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:14:16 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[93444]: ceph version 19.0.0-2145-g3be9a5c9 (3be9a5c9bc793e11e2800a8c0c696e8b46742033) squid (dev) 2024-03-28T18:14:16.284 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:14:16 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[93444]: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x135) [0x7fddcc8df4a1] 2024-03-28T18:14:16.284 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:14:16 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[93444]: 2: /usr/lib64/ceph/libceph-common.so.2(+0x2d2667) [0x7fddcc8df667] 2024-03-28T18:14:16.284 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:14:16 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[93444]: 3: (Locker::set_xlocks_done(MutationImpl*, bool)+0x493) [0x5623cd93f223] 2024-03-28T18:14:16.284 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:14:16 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[93444]: 4: (Server::early_reply(boost::intrusive_ptr const&, CInode*, CDentry*)+0x3e6) [0x5623cd7b95d6] 2024-03-28T18:14:16.284 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:14:16 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[93444]: 5: (Server::journal_and_reply(boost::intrusive_ptr const&, CInode*, CDentry*, LogEvent*, MDSLogContextBase*)+0xc1) [0x5623cd7b9da1] 2024-03-28T18:14:16.284 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:14:16 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[93444]: 6: (Server::handle_client_mkdir(boost::intrusive_ptr const&)+0x5ba) [0x5623cd7ef6aa] 2024-03-28T18:14:16.285 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:14:16 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[93444]: 7: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0x733) [0x5623cd80c233] 2024-03-28T18:14:16.285 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:14:16 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[93444]: 8: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x5623cd80e197] 2024-03-28T18:14:16.285 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:14:16 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[93444]: 9: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x5623cd70d11b] 2024-03-28T18:14:16.285 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:14:16 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[93444]: 10: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x5623cd70d6bc] 2024-03-28T18:14:16.285 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:14:16 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[93444]: 11: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bf) [0x5623cd6f71ff] 2024-03-28T18:14:16.285 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:14:16 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[93444]: 12: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x478) [0x7fddccb71488] 2024-03-28T18:14:16.285 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:14:16 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[93444]: 13: (DispatchQueue::entry()+0x51f) [0x7fddccb6e60f] 2024-03-28T18:14:16.286 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:14:16 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[93444]: 14: (DispatchQueue::DispatchThread::entry()+0x11) [0x7fddccc41e81] 2024-03-28T18:14:16.286 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:14:16 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[93444]: 15: /lib64/libpthread.so.0(+0x81ca) [0x7fddcb65c1ca] 2024-03-28T18:14:16.286 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:14:16 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[93444]: 16: clone() 2024-03-28T18:14:16.287 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:14:16 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[93444]: 2024-03-28T18:14:16.079+0000 7fddc1000700 -1 /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-2145-g3be9a5c9/rpm/el8/BUILD/ceph-19.0.0-2145-g3be9a5c9/src/mds/Locker.cc: In function 'void Locker::set_xlocks_done(MutationImpl*, bool)' thread 7fddc1000700 time 2024-03-28T18:14:16.078982+0000 2024-03-28T18:14:16.287 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:14:16 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[93444]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-2145-g3be9a5c9/rpm/el8/BUILD/ceph-19.0.0-2145-g3be9a5c9/src/mds/Locker.cc: 626: FAILED ceph_assert(obj->is_auth()) 2024-03-28T18:14:16.287 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:14:16 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[93444]: 2024-03-28T18:14:16.287 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:14:16 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[93444]: ceph version 19.0.0-2145-g3be9a5c9 (3be9a5c9bc793e11e2800a8c0c696e8b46742033) squid (dev) 2024-03-28T18:14:16.288 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:14:16 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[93444]: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x135) [0x7fddcc8df4a1] 2024-03-28T18:14:16.288 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:14:16 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[93444]: 2: /usr/lib64/ceph/libceph-common.so.2(+0x2d2667) [0x7fddcc8df667] 2024-03-28T18:14:16.288 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:14:16 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[93444]: 3: (Locker::set_xlocks_done(MutationImpl*, bool)+0x493) [0x5623cd93f223] 2024-03-28T18:14:16.288 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:14:16 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[93444]: 4: (Server::early_reply(boost::intrusive_ptr const&, CInode*, CDentry*)+0x3e6) [0x5623cd7b95d6] 2024-03-28T18:14:16.288 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:14:16 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[93444]: 5: (Server::journal_and_reply(boost::intrusive_ptr const&, CInode*, CDentry*, LogEvent*, MDSLogContextBase*)+0xc1) [0x5623cd7b9da1] 2024-03-28T18:14:16.288 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:14:16 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[93444]: 6: (Server::handle_client_mkdir(boost::intrusive_ptr const&)+0x5ba) [0x5623cd7ef6aa] 2024-03-28T18:14:16.288 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:14:16 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[93444]: 7: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0x733) [0x5623cd80c233] 2024-03-28T18:14:16.288 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:14:16 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[93444]: 8: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x5623cd80e197] 2024-03-28T18:14:16.289 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:14:16 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[93444]: 9: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x5623cd70d11b] 2024-03-28T18:14:16.289 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:14:16 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[93444]: 10: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x5623cd70d6bc] 2024-03-28T18:14:16.289 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:14:16 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[93444]: 11: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bf) [0x5623cd6f71ff] 2024-03-28T18:14:16.289 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:14:16 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[93444]: 12: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x478) [0x7fddccb71488] 2024-03-28T18:14:16.289 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:14:16 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[93444]: 13: (DispatchQueue::entry()+0x51f) [0x7fddccb6e60f] 2024-03-28T18:14:16.289 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:14:16 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[93444]: 14: (DispatchQueue::DispatchThread::entry()+0x11) [0x7fddccc41e81] 2024-03-28T18:14:16.290 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:14:16 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[93444]: 15: /lib64/libpthread.so.0(+0x81ca) [0x7fddcb65c1ca] 2024-03-28T18:14:16.290 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:14:16 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[93444]: 16: clone() 2024-03-28T18:14:16.290 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:14:16 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[93444]: 2024-03-28T18:14:16.290 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:14:16 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[93444]: *** Caught signal (Aborted) ** 2024-03-28T18:14:16.290 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:14:16 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[93444]: in thread 7fddc1000700 thread_name:ms_dispatch 2024-03-28T18:14:16.290 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:14:16 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[93444]: ceph version 19.0.0-2145-g3be9a5c9 (3be9a5c9bc793e11e2800a8c0c696e8b46742033) squid (dev) 2024-03-28T18:14:16.290 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:14:16 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[93444]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7fddcb666d20] 2024-03-28T18:14:16.290 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:14:16 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[93444]: 2: gsignal() 2024-03-28T18:14:16.290 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:14:16 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[93444]: 3: abort() 2024-03-28T18:14:16.290 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:14:16 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[93444]: 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x18f) [0x7fddcc8df4fb] 2024-03-28T18:14:16.290 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:14:16 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[93444]: 5: /usr/lib64/ceph/libceph-common.so.2(+0x2d2667) [0x7fddcc8df667] 2024-03-28T18:14:16.291 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:14:16 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[93444]: 6: (Locker::set_xlocks_done(MutationImpl*, bool)+0x493) [0x5623cd93f223] 2024-03-28T18:14:16.291 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:14:16 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[93444]: 7: (Server::early_reply(boost::intrusive_ptr const&, CInode*, CDentry*)+0x3e6) [0x5623cd7b95d6] 2024-03-28T18:14:16.291 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:14:16 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[93444]: 8: (Server::journal_and_reply(boost::intrusive_ptr const&, CInode*, CDentry*, LogEvent*, MDSLogContextBase*)+0xc1) [0x5623cd7b9da1] 2024-03-28T18:14:16.291 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:14:16 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[93444]: 9: (Server::handle_client_mkdir(boost::intrusive_ptr const&)+0x5ba) [0x5623cd7ef6aa] 2024-03-28T18:14:16.291 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:14:16 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[93444]: 10: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0x733) [0x5623cd80c233] 2024-03-28T18:14:16.291 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:14:16 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[93444]: 11: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x5623cd80e197] 2024-03-28T18:14:16.291 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:14:16 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[93444]: 12: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x5623cd70d11b] 2024-03-28T18:14:16.291 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:14:16 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[93444]: 13: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x5623cd70d6bc] 2024-03-28T18:14:16.291 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:14:16 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[93444]: 14: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bf) [0x5623cd6f71ff] 2024-03-28T18:14:16.291 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:14:16 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[93444]: 15: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x478) [0x7fddccb71488] 2024-03-28T18:14:16.291 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:14:16 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[93444]: 16: (DispatchQueue::entry()+0x51f) [0x7fddccb6e60f] 2024-03-28T18:14:16.291 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:14:16 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[93444]: 17: (DispatchQueue::DispatchThread::entry()+0x11) [0x7fddccc41e81] 2024-03-28T18:14:16.291 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:14:16 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[93444]: 18: /lib64/libpthread.so.0(+0x81ca) [0x7fddcb65c1ca] 2024-03-28T18:14:16.291 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:14:16 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[93444]: 19: clone() 2024-03-28T18:14:16.292 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:14:16 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[93444]: 2024-03-28T18:14:16.082+0000 7fddc1000700 -1 *** Caught signal (Aborted) ** 2024-03-28T18:14:16.292 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:14:16 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[93444]: in thread 7fddc1000700 thread_name:ms_dispatch 2024-03-28T18:14:16.292 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:14:16 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[93444]: 2024-03-28T18:14:16.292 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:14:16 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[93444]: ceph version 19.0.0-2145-g3be9a5c9 (3be9a5c9bc793e11e2800a8c0c696e8b46742033) squid (dev) 2024-03-28T18:14:16.292 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:14:16 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[93444]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7fddcb666d20] 2024-03-28T18:14:16.292 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:14:16 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[93444]: 2: gsignal() 2024-03-28T18:14:16.292 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:14:16 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[93444]: 3: abort() 2024-03-28T18:14:16.292 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:14:16 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[93444]: 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x18f) [0x7fddcc8df4fb] 2024-03-28T18:14:16.292 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:14:16 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[93444]: 5: /usr/lib64/ceph/libceph-common.so.2(+0x2d2667) [0x7fddcc8df667] 2024-03-28T18:14:16.293 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:14:16 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[93444]: 6: (Locker::set_xlocks_done(MutationImpl*, bool)+0x493) [0x5623cd93f223] 2024-03-28T18:14:16.293 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:14:16 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[93444]: 7: (Server::early_reply(boost::intrusive_ptr const&, CInode*, CDentry*)+0x3e6) [0x5623cd7b95d6] 2024-03-28T18:14:16.293 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:14:16 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[93444]: 8: (Server::journal_and_reply(boost::intrusive_ptr const&, CInode*, CDentry*, LogEvent*, MDSLogContextBase*)+0xc1) [0x5623cd7b9da1] 2024-03-28T18:14:16.294 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:14:16 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[93444]: 9: (Server::handle_client_mkdir(boost::intrusive_ptr const&)+0x5ba) [0x5623cd7ef6aa] 2024-03-28T18:14:16.294 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:14:16 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[93444]: 10: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0x733) [0x5623cd80c233] 2024-03-28T18:14:16.294 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:14:16 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[93444]: 11: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x5623cd80e197] 2024-03-28T18:14:16.294 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:14:16 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[93444]: 12: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x5623cd70d11b] 2024-03-28T18:14:16.294 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:14:16 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[93444]: 13: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x5623cd70d6bc] 2024-03-28T18:14:16.294 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:14:16 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[93444]: 14: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bf) [0x5623cd6f71ff] 2024-03-28T18:14:16.294 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:14:16 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[93444]: 15: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x478) [0x7fddccb71488] 2024-03-28T18:14:16.294 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:14:16 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[93444]: 16: (DispatchQueue::entry()+0x51f) [0x7fddccb6e60f] 2024-03-28T18:14:16.294 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:14:16 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[93444]: 17: (DispatchQueue::DispatchThread::entry()+0x11) [0x7fddccc41e81] 2024-03-28T18:14:16.294 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:14:16 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[93444]: 18: /lib64/libpthread.so.0(+0x81ca) [0x7fddcb65c1ca] 2024-03-28T18:14:16.294 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:14:16 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[93444]: 19: clone() 2024-03-28T18:14:16.295 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:14:16 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[93444]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-03-28T18:14:16.295 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:14:16 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[93444]: 2024-03-28T18:14:16.295 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:14:16 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[93444]: -1> 2024-03-28T18:14:16.079+0000 7fddc1000700 -1 /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-2145-g3be9a5c9/rpm/el8/BUILD/ceph-19.0.0-2145-g3be9a5c9/src/mds/Locker.cc: In function 'void Locker::set_xlocks_done(MutationImpl*, bool)' thread 7fddc1000700 time 2024-03-28T18:14:16.078982+0000 2024-03-28T18:14:16.295 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:14:16 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[93444]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-2145-g3be9a5c9/rpm/el8/BUILD/ceph-19.0.0-2145-g3be9a5c9/src/mds/Locker.cc: 626: FAILED ceph_assert(obj->is_auth()) 2024-03-28T18:14:16.295 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:14:16 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[93444]: 2024-03-28T18:14:16.295 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:14:16 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[93444]: ceph version 19.0.0-2145-g3be9a5c9 (3be9a5c9bc793e11e2800a8c0c696e8b46742033) squid (dev) 2024-03-28T18:14:16.295 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:14:16 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[93444]: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x135) [0x7fddcc8df4a1] 2024-03-28T18:14:16.295 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:14:16 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[93444]: 2: /usr/lib64/ceph/libceph-common.so.2(+0x2d2667) [0x7fddcc8df667] 2024-03-28T18:14:16.295 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:14:16 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[93444]: 3: (Locker::set_xlocks_done(MutationImpl*, bool)+0x493) [0x5623cd93f223] 2024-03-28T18:14:16.295 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:14:16 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[93444]: 4: (Server::early_reply(boost::intrusive_ptr const&, CInode*, CDentry*)+0x3e6) [0x5623cd7b95d6] 2024-03-28T18:14:16.295 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:14:16 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[93444]: 5: (Server::journal_and_reply(boost::intrusive_ptr const&, CInode*, CDentry*, LogEvent*, MDSLogContextBase*)+0xc1) [0x5623cd7b9da1] 2024-03-28T18:14:16.295 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:14:16 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[93444]: 6: (Server::handle_client_mkdir(boost::intrusive_ptr const&)+0x5ba) [0x5623cd7ef6aa] 2024-03-28T18:14:16.295 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:14:16 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[93444]: 7: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0x733) [0x5623cd80c233] 2024-03-28T18:14:16.296 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:14:16 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[93444]: 8: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x5623cd80e197] 2024-03-28T18:14:16.296 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:14:16 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[93444]: 9: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x5623cd70d11b] 2024-03-28T18:14:16.296 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:14:16 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[93444]: 10: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x5623cd70d6bc] 2024-03-28T18:14:16.296 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:14:16 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[93444]: 11: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bf) [0x5623cd6f71ff] 2024-03-28T18:14:16.296 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:14:16 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[93444]: 12: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x478) [0x7fddccb71488] 2024-03-28T18:14:16.296 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:14:16 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[93444]: 13: (DispatchQueue::entry()+0x51f) [0x7fddccb6e60f] 2024-03-28T18:14:16.296 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:14:16 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[93444]: 14: (DispatchQueue::DispatchThread::entry()+0x11) [0x7fddccc41e81] 2024-03-28T18:14:16.296 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:14:16 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[93444]: 15: /lib64/libpthread.so.0(+0x81ca) [0x7fddcb65c1ca] 2024-03-28T18:14:16.296 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:14:16 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[93444]: 16: clone() 2024-03-28T18:14:16.296 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:14:16 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[93444]: 2024-03-28T18:14:16.296 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:14:16 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[93444]: 0> 2024-03-28T18:14:16.082+0000 7fddc1000700 -1 *** Caught signal (Aborted) ** 2024-03-28T18:14:16.296 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:14:16 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[93444]: in thread 7fddc1000700 thread_name:ms_dispatch 2024-03-28T18:14:16.296 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:14:16 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[93444]: 2024-03-28T18:14:16.296 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:14:16 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[93444]: ceph version 19.0.0-2145-g3be9a5c9 (3be9a5c9bc793e11e2800a8c0c696e8b46742033) squid (dev) 2024-03-28T18:14:16.296 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:14:16 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[93444]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7fddcb666d20] 2024-03-28T18:14:16.297 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:14:16 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[93444]: 2: gsignal() 2024-03-28T18:14:16.297 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:14:16 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[93444]: 3: abort() 2024-03-28T18:14:16.297 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:14:16 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[93444]: 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x18f) [0x7fddcc8df4fb] 2024-03-28T18:14:16.297 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:14:16 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[93444]: 5: /usr/lib64/ceph/libceph-common.so.2(+0x2d2667) [0x7fddcc8df667] 2024-03-28T18:14:16.297 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:14:16 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[93444]: 6: (Locker::set_xlocks_done(MutationImpl*, bool)+0x493) [0x5623cd93f223] 2024-03-28T18:14:16.297 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:14:16 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[93444]: 7: (Server::early_reply(boost::intrusive_ptr const&, CInode*, CDentry*)+0x3e6) [0x5623cd7b95d6] 2024-03-28T18:14:16.297 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:14:16 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[93444]: 8: (Server::journal_and_reply(boost::intrusive_ptr const&, CInode*, CDentry*, LogEvent*, MDSLogContextBase*)+0xc1) [0x5623cd7b9da1] 2024-03-28T18:14:16.297 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:14:16 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[93444]: 9: (Server::handle_client_mkdir(boost::intrusive_ptr const&)+0x5ba) [0x5623cd7ef6aa] 2024-03-28T18:14:16.297 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:14:16 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[93444]: 10: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0x733) [0x5623cd80c233] 2024-03-28T18:14:16.297 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:14:16 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[93444]: 11: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x5623cd80e197] 2024-03-28T18:14:16.297 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:14:16 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[93444]: 12: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x5623cd70d11b] 2024-03-28T18:14:16.297 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:14:16 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[93444]: 13: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x5623cd70d6bc] 2024-03-28T18:14:16.297 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:14:16 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[93444]: 14: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bf) [0x5623cd6f71ff] 2024-03-28T18:14:16.298 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:14:16 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[93444]: 15: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x478) [0x7fddccb71488] 2024-03-28T18:14:16.298 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:14:16 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[93444]: 16: (DispatchQueue::entry()+0x51f) [0x7fddccb6e60f] 2024-03-28T18:14:16.298 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:14:16 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[93444]: 17: (DispatchQueue::DispatchThread::entry()+0x11) [0x7fddccc41e81] 2024-03-28T18:14:16.298 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:14:16 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[93444]: 18: /lib64/libpthread.so.0(+0x81ca) [0x7fddcb65c1ca] 2024-03-28T18:14:16.298 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:14:16 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[93444]: 19: clone() 2024-03-28T18:14:16.298 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:14:16 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[93444]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-03-28T18:14:16.298 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:14:16 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[93444]: 2024-03-28T18:14:16.298 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:14:16 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[93444]: -4> 2024-03-28T18:14:16.079+0000 7fddc1000700 -1 /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-2145-g3be9a5c9/rpm/el8/BUILD/ceph-19.0.0-2145-g3be9a5c9/src/mds/Locker.cc: In function 'void Locker::set_xlocks_done(MutationImpl*, bool)' thread 7fddc1000700 time 2024-03-28T18:14:16.078982+0000 2024-03-28T18:14:16.298 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:14:16 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[93444]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-2145-g3be9a5c9/rpm/el8/BUILD/ceph-19.0.0-2145-g3be9a5c9/src/mds/Locker.cc: 626: FAILED ceph_assert(obj->is_auth()) 2024-03-28T18:14:16.298 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:14:16 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[93444]: 2024-03-28T18:14:16.298 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:14:16 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[93444]: ceph version 19.0.0-2145-g3be9a5c9 (3be9a5c9bc793e11e2800a8c0c696e8b46742033) squid (dev) 2024-03-28T18:14:16.299 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:14:16 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[93444]: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x135) [0x7fddcc8df4a1] 2024-03-28T18:14:16.299 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:14:16 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[93444]: 2: /usr/lib64/ceph/libceph-common.so.2(+0x2d2667) [0x7fddcc8df667] 2024-03-28T18:14:16.299 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:14:16 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[93444]: 3: (Locker::set_xlocks_done(MutationImpl*, bool)+0x493) [0x5623cd93f223] 2024-03-28T18:14:16.299 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:14:16 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[93444]: 4: (Server::early_reply(boost::intrusive_ptr const&, CInode*, CDentry*)+0x3e6) [0x5623cd7b95d6] 2024-03-28T18:14:16.299 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:14:16 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[93444]: 5: (Server::journal_and_reply(boost::intrusive_ptr const&, CInode*, CDentry*, LogEvent*, MDSLogContextBase*)+0xc1) [0x5623cd7b9da1] 2024-03-28T18:14:16.299 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:14:16 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[93444]: 6: (Server::handle_client_mkdir(boost::intrusive_ptr const&)+0x5ba) [0x5623cd7ef6aa] 2024-03-28T18:14:16.299 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:14:16 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[93444]: 7: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0x733) [0x5623cd80c233] 2024-03-28T18:14:16.299 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:14:16 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[93444]: 8: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x5623cd80e197] 2024-03-28T18:14:16.299 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:14:16 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[93444]: 9: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x5623cd70d11b] 2024-03-28T18:14:16.299 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:14:16 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[93444]: 10: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x5623cd70d6bc] 2024-03-28T18:14:16.299 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:14:16 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[93444]: 11: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bf) [0x5623cd6f71ff] 2024-03-28T18:14:16.299 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:14:16 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[93444]: 12: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x478) [0x7fddccb71488] 2024-03-28T18:14:16.299 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:14:16 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[93444]: 13: (DispatchQueue::entry()+0x51f) [0x7fddccb6e60f] 2024-03-28T18:14:16.299 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:14:16 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[93444]: 14: (DispatchQueue::DispatchThread::entry()+0x11) [0x7fddccc41e81] 2024-03-28T18:14:16.299 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:14:16 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[93444]: 15: /lib64/libpthread.so.0(+0x81ca) [0x7fddcb65c1ca] 2024-03-28T18:14:16.300 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:14:16 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[93444]: 16: clone() 2024-03-28T18:14:16.300 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:14:16 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[93444]: 2024-03-28T18:14:16.300 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:14:16 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[93444]: -3> 2024-03-28T18:14:16.082+0000 7fddc1000700 -1 *** Caught signal (Aborted) ** 2024-03-28T18:14:16.300 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:14:16 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[93444]: in thread 7fddc1000700 thread_name:ms_dispatch 2024-03-28T18:14:16.300 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:14:16 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[93444]: 2024-03-28T18:14:16.300 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:14:16 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[93444]: ceph version 19.0.0-2145-g3be9a5c9 (3be9a5c9bc793e11e2800a8c0c696e8b46742033) squid (dev) 2024-03-28T18:14:16.300 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:14:16 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[93444]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7fddcb666d20] 2024-03-28T18:14:16.300 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:14:16 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[93444]: 2: gsignal() 2024-03-28T18:14:16.300 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:14:16 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[93444]: 3: abort() 2024-03-28T18:14:16.300 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:14:16 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[93444]: 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x18f) [0x7fddcc8df4fb] 2024-03-28T18:14:16.300 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:14:16 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[93444]: 5: /usr/lib64/ceph/libceph-common.so.2(+0x2d2667) [0x7fddcc8df667] 2024-03-28T18:14:16.300 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:14:16 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[93444]: 6: (Locker::set_xlocks_done(MutationImpl*, bool)+0x493) [0x5623cd93f223] 2024-03-28T18:14:16.300 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:14:16 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[93444]: 7: (Server::early_reply(boost::intrusive_ptr const&, CInode*, CDentry*)+0x3e6) [0x5623cd7b95d6] 2024-03-28T18:14:16.300 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:14:16 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[93444]: 8: (Server::journal_and_reply(boost::intrusive_ptr const&, CInode*, CDentry*, LogEvent*, MDSLogContextBase*)+0xc1) [0x5623cd7b9da1] 2024-03-28T18:14:16.300 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:14:16 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[93444]: 9: (Server::handle_client_mkdir(boost::intrusive_ptr const&)+0x5ba) [0x5623cd7ef6aa] 2024-03-28T18:14:16.301 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:14:16 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[93444]: 10: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0x733) [0x5623cd80c233] 2024-03-28T18:14:16.301 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:14:16 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[93444]: 11: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x5623cd80e197] 2024-03-28T18:14:16.301 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:14:16 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[93444]: 12: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x5623cd70d11b] 2024-03-28T18:14:16.301 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:14:16 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[93444]: 13: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x5623cd70d6bc] 2024-03-28T18:14:16.301 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:14:16 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[93444]: 14: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bf) [0x5623cd6f71ff] 2024-03-28T18:14:16.301 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:14:16 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[93444]: 15: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x478) [0x7fddccb71488] 2024-03-28T18:14:16.301 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:14:16 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[93444]: 16: (DispatchQueue::entry()+0x51f) [0x7fddccb6e60f] 2024-03-28T18:14:16.301 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:14:16 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[93444]: 17: (DispatchQueue::DispatchThread::entry()+0x11) [0x7fddccc41e81] 2024-03-28T18:14:16.301 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:14:16 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[93444]: 18: /lib64/libpthread.so.0(+0x81ca) [0x7fddcb65c1ca] 2024-03-28T18:14:16.301 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:14:16 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[93444]: 19: clone() 2024-03-28T18:14:16.301 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:14:16 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[93444]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-03-28T18:14:16.301 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:14:16 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[93444]: 2024-03-28T18:14:16.301 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:14:16 smithi022 podman[93643]: 2024-03-28 18:14:16.206388749 +0000 UTC m=+0.032907741 container died e68ebf0f6af9552391b17dde9ecb9b888cf364645d2ba0279c36335c3c5fd194 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d, ceph=True, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240319, CEPH_POINT_RELEASE=, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, org.label-schema.license=GPLv2, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-28T18:14:16.425 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:14:15 smithi134 ceph-mon[35037]: mds.? [v2:172.21.15.22:6836/3983083841,v1:172.21.15.22:6837/3983083841] up:rejoin 2024-03-28T18:14:16.425 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:14:15 smithi134 ceph-mon[35037]: fsmap cephfs:5/5 4 up:standby-replay 1 up:standby 1 up:rejoin 4 up:active 2024-03-28T18:14:16.425 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:14:15 smithi134 ceph-mon[35037]: daemon mds.d is now active in filesystem cephfs as rank 1 2024-03-28T18:14:16.425 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:14:15 smithi134 ceph-mon[35037]: pgmap v218: 97 pgs: 97 active+clean; 117 MiB data, 774 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 204 B/s wr, 10 op/s 2024-03-28T18:14:16.432 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:14:15 smithi079 ceph-mon[35312]: mds.? [v2:172.21.15.22:6836/3983083841,v1:172.21.15.22:6837/3983083841] up:rejoin 2024-03-28T18:14:16.432 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:14:15 smithi079 ceph-mon[35312]: fsmap cephfs:5/5 4 up:standby-replay 1 up:standby 1 up:rejoin 4 up:active 2024-03-28T18:14:16.432 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:14:15 smithi079 ceph-mon[35312]: daemon mds.d is now active in filesystem cephfs as rank 1 2024-03-28T18:14:16.432 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:14:15 smithi079 ceph-mon[35312]: pgmap v218: 97 pgs: 97 active+clean; 117 MiB data, 774 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 204 B/s wr, 10 op/s 2024-03-28T18:14:17.282 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:14:16 smithi022 ceph-mon[31985]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-03-28T18:14:17.282 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:14:16 smithi022 ceph-mon[31985]: Cluster is now healthy 2024-03-28T18:14:17.282 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:14:16 smithi022 ceph-mon[31985]: mds.? [v2:172.21.15.22:6836/3983083841,v1:172.21.15.22:6837/3983083841] up:active 2024-03-28T18:14:17.282 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:14:16 smithi022 ceph-mon[31985]: fsmap cephfs:5 4 up:standby-replay 1 up:standby 5 up:active 2024-03-28T18:14:17.282 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:14:16 smithi022 ceph-mon[31985]: fsmap cephfs:5 5 up:standby-replay 5 up:active 2024-03-28T18:14:17.282 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:14:16 smithi022 podman[93643]: 2024-03-28 18:14:16.964164524 +0000 UTC m=+0.790683557 container remove e68ebf0f6af9552391b17dde9ecb9b888cf364645d2ba0279c36335c3c5fd194 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_CLEAN=True, ceph=True, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240319, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30) 2024-03-28T18:14:17.282 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:14:16 smithi022 systemd[1]: ceph-fd220748-ed2b-11ee-95d0-87774f69a715@mds.d.service: Main process exited, code=exited, status=134/n/a 2024-03-28T18:14:17.425 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:14:16 smithi134 ceph-mon[35037]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-03-28T18:14:17.425 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:14:16 smithi134 ceph-mon[35037]: Cluster is now healthy 2024-03-28T18:14:17.425 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:14:16 smithi134 ceph-mon[35037]: mds.? [v2:172.21.15.22:6836/3983083841,v1:172.21.15.22:6837/3983083841] up:active 2024-03-28T18:14:17.425 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:14:16 smithi134 ceph-mon[35037]: fsmap cephfs:5 4 up:standby-replay 1 up:standby 5 up:active 2024-03-28T18:14:17.425 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:14:16 smithi134 ceph-mon[35037]: fsmap cephfs:5 5 up:standby-replay 5 up:active 2024-03-28T18:14:17.432 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:14:16 smithi079 ceph-mon[35312]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-03-28T18:14:17.432 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:14:16 smithi079 ceph-mon[35312]: Cluster is now healthy 2024-03-28T18:14:17.433 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:14:16 smithi079 ceph-mon[35312]: mds.? [v2:172.21.15.22:6836/3983083841,v1:172.21.15.22:6837/3983083841] up:active 2024-03-28T18:14:17.433 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:14:16 smithi079 ceph-mon[35312]: fsmap cephfs:5 4 up:standby-replay 1 up:standby 5 up:active 2024-03-28T18:14:17.433 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:14:16 smithi079 ceph-mon[35312]: fsmap cephfs:5 5 up:standby-replay 5 up:active 2024-03-28T18:14:17.782 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:14:17 smithi022 systemd[1]: ceph-fd220748-ed2b-11ee-95d0-87774f69a715@mds.d.service: Failed with result 'exit-code'. 2024-03-28T18:14:17.782 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:14:17 smithi022 systemd[1]: ceph-fd220748-ed2b-11ee-95d0-87774f69a715@mds.d.service: Consumed 1.958s CPU time. 2024-03-28T18:14:18.282 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:14:18 smithi022 ceph-mon[31985]: pgmap v219: 97 pgs: 97 active+clean; 117 MiB data, 774 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 204 B/s wr, 10 op/s 2024-03-28T18:14:18.424 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:14:18 smithi134 ceph-mon[35037]: pgmap v219: 97 pgs: 97 active+clean; 117 MiB data, 774 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 204 B/s wr, 10 op/s 2024-03-28T18:14:18.432 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:14:18 smithi079 ceph-mon[35312]: pgmap v219: 97 pgs: 97 active+clean; 117 MiB data, 774 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 204 B/s wr, 10 op/s 2024-03-28T18:14:18.639 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-28T18:14:18.639 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-28T18:14:18.890 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:14:18.889+0000 7f4ae0a00640 1 -- 172.21.15.22:0/2675146467 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4adc107e50 msgr2=0x7f4adc108230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:14:18.891 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:14:18.889+0000 7f4ae0a00640 1 --2- 172.21.15.22:0/2675146467 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4adc107e50 0x7f4adc108230 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7f4acc009900 tx=0x7f4acc02bc50 comp rx=0 tx=0).stop 2024-03-28T18:14:18.891 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:14:18.889+0000 7f4ae0a00640 1 -- 172.21.15.22:0/2675146467 shutdown_connections 2024-03-28T18:14:18.891 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:14:18.889+0000 7f4ae0a00640 1 --2- 172.21.15.22:0/2675146467 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4adc10cea0 0x7f4adc114a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:14:18.891 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:14:18.889+0000 7f4ae0a00640 1 --2- 172.21.15.22:0/2675146467 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f4adc108800 0x7f4adc10c7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:14:18.891 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:14:18.889+0000 7f4ae0a00640 1 --2- 172.21.15.22:0/2675146467 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4adc107e50 0x7f4adc108230 unknown :-1 s=CLOSED pgs=733 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:14:18.891 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:14:18.889+0000 7f4ae0a00640 1 -- 172.21.15.22:0/2675146467 >> 172.21.15.22:0/2675146467 conn(0x7f4adc0774c0 msgr2=0x7f4adc1064c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:14:18.891 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:14:18.890+0000 7f4ae0a00640 1 -- 172.21.15.22:0/2675146467 shutdown_connections 2024-03-28T18:14:18.891 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:14:18.890+0000 7f4ae0a00640 1 -- 172.21.15.22:0/2675146467 wait complete. 2024-03-28T18:14:18.894 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:14:18.891+0000 7f4ae0a00640 1 Processor -- start 2024-03-28T18:14:18.894 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:14:18.891+0000 7f4ae0a00640 1 -- start start 2024-03-28T18:14:18.894 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:14:18.891+0000 7f4ae0a00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4adc107e50 0x7f4adc19e490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:14:18.894 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:14:18.891+0000 7f4ae0a00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4adc10cea0 0x7f4adc19e9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:14:18.894 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:14:18.891+0000 7f4ae0a00640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f4adc1a2d00 0x7f4adc1a31b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:14:18.894 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:14:18.891+0000 7f4ae0a00640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4adc117f90 con 0x7f4adc107e50 2024-03-28T18:14:18.895 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:14:18.891+0000 7f4ae0a00640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4adc117e10 con 0x7f4adc1a2d00 2024-03-28T18:14:18.895 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:14:18.891+0000 7f4ae0a00640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4adc118110 con 0x7f4adc10cea0 2024-03-28T18:14:18.895 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:14:18.891+0000 7f4adb400640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4adc107e50 0x7f4adc19e490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:14:18.895 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:14:18.892+0000 7f4adb400640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4adc107e50 0x7f4adc19e490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:41822/0 (socket says 172.21.15.22:41822) 2024-03-28T18:14:18.895 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:14:18.892+0000 7f4adb400640 1 -- 172.21.15.22:0/3923887644 learned_addr learned my addr 172.21.15.22:0/3923887644 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-28T18:14:18.895 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:14:18.892+0000 7f4adbe00640 1 --2- 172.21.15.22:0/3923887644 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f4adc1a2d00 0x7f4adc1a31b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:14:18.895 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:14:18.892+0000 7f4ad3e00640 1 --2- 172.21.15.22:0/3923887644 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4adc10cea0 0x7f4adc19e9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:14:18.895 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:14:18.892+0000 7f4adb400640 1 -- 172.21.15.22:0/3923887644 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4adc10cea0 msgr2=0x7f4adc19e9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:14:18.895 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:14:18.892+0000 7f4adb400640 1 --2- 172.21.15.22:0/3923887644 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4adc10cea0 0x7f4adc19e9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:14:18.895 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:14:18.892+0000 7f4adb400640 1 -- 172.21.15.22:0/3923887644 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f4adc1a2d00 msgr2=0x7f4adc1a31b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:14:18.895 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:14:18.892+0000 7f4adb400640 1 --2- 172.21.15.22:0/3923887644 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f4adc1a2d00 0x7f4adc1a31b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:14:18.895 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:14:18.892+0000 7f4adb400640 1 -- 172.21.15.22:0/3923887644 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4adc1a37f0 con 0x7f4adc107e50 2024-03-28T18:14:18.896 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:14:18.892+0000 7f4adb400640 1 --2- 172.21.15.22:0/3923887644 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4adc107e50 0x7f4adc19e490 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7f4acc000c00 tx=0x7f4acc0045e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:14:18.896 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:14:18.893+0000 7f4ad8c00640 1 -- 172.21.15.22:0/3923887644 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4acc043020 con 0x7f4adc107e50 2024-03-28T18:14:18.896 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:14:18.893+0000 7f4ad8c00640 1 -- 172.21.15.22:0/3923887644 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f4acc02cac0 con 0x7f4adc107e50 2024-03-28T18:14:18.896 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:14:18.893+0000 7f4ad8c00640 1 -- 172.21.15.22:0/3923887644 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4acc003ae0 con 0x7f4adc107e50 2024-03-28T18:14:18.896 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:14:18.893+0000 7f4ae0a00640 1 -- 172.21.15.22:0/3923887644 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4adc1a3a80 con 0x7f4adc107e50 2024-03-28T18:14:18.896 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:14:18.893+0000 7f4ae0a00640 1 -- 172.21.15.22:0/3923887644 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4adc1ab1f0 con 0x7f4adc107e50 2024-03-28T18:14:18.898 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:14:18.897+0000 7f4ae4d72740 1 Processor -- start 2024-03-28T18:14:18.898 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:14:18.897+0000 7f4ae4d72740 1 -- start start 2024-03-28T18:14:18.900 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:14:18.898+0000 7f4ad8c00640 1 -- 172.21.15.22:0/3923887644 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 26) v1 ==== 137421+0+0 (secure 0 0 0) 0x7f4acc046020 con 0x7f4adc107e50 2024-03-28T18:14:18.900 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:14:18.898+0000 7f4ae4d72740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5598ab318dd0 0x5598ab317b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:14:18.900 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:14:18.898+0000 7f4ae4d72740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5598ab3180f0 0x5598ab318570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:14:18.900 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:14:18.898+0000 7f4ae4d72740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5598ab321460 0x5598ab323850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:14:18.901 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:14:18.898+0000 7f4ae4d72740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x5598ab27c250 con 0x5598ab321460 2024-03-28T18:14:18.901 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:14:18.898+0000 7f4ae4d72740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x5598ab224a30 con 0x5598ab3180f0 2024-03-28T18:14:18.901 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:14:18.898+0000 7f4ae4d72740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x5598ab2782f0 con 0x5598ab318dd0 2024-03-28T18:14:18.901 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:14:18.898+0000 7f4ad3e00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5598ab321460 0x5598ab323850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:14:18.901 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:14:18.898+0000 7f4ad3e00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5598ab321460 0x5598ab323850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:41838/0 (socket says 172.21.15.22:41838) 2024-03-28T18:14:18.901 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:14:18.898+0000 7f4ad3e00640 1 -- 172.21.15.22:0/2170523909 learned_addr learned my addr 172.21.15.22:0/2170523909 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-28T18:14:18.901 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:14:18.898+0000 7f4adbe00640 1 --2- 172.21.15.22:0/2170523909 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5598ab318dd0 0x5598ab317b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:14:18.901 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:14:18.898+0000 7f4adb400640 1 --2- 172.21.15.22:0/2170523909 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5598ab3180f0 0x5598ab318570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:14:18.901 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:14:18.899+0000 7f4ad3e00640 1 -- 172.21.15.22:0/2170523909 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5598ab318dd0 msgr2=0x5598ab317b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T18:14:18.901 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:14:18.899+0000 7f4ad3e00640 1 --2- 172.21.15.22:0/2170523909 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5598ab318dd0 0x5598ab317b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:14:18.901 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:14:18.899+0000 7f4ad3e00640 1 -- 172.21.15.22:0/2170523909 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5598ab3180f0 msgr2=0x5598ab318570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T18:14:18.902 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:14:18.899+0000 7f4ad3e00640 1 --2- 172.21.15.22:0/2170523909 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5598ab3180f0 0x5598ab318570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:14:18.902 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:14:18.899+0000 7f4ad3e00640 1 -- 172.21.15.22:0/2170523909 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5598ab20f690 con 0x5598ab321460 2024-03-28T18:14:18.902 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:14:18.899+0000 7f4ad3e00640 1 --2- 172.21.15.22:0/2170523909 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5598ab321460 0x5598ab323850 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7f4ac400bc10 tx=0x7f4ac400f730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=54c56a458673a624 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:14:18.902 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:14:18.899+0000 7f4ad8c00640 1 --2- 172.21.15.22:0/3923887644 >> [v2:172.21.15.22:6800/1357365082,v1:172.21.15.22:6801/1357365082] conn(0x7f4ac00a29f0 0x7f4ac00a4eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:14:18.902 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:14:18.899+0000 7f4acbe00640 1 -- 172.21.15.22:0/2170523909 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ac4015070 con 0x5598ab321460 2024-03-28T18:14:18.902 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:14:18.899+0000 7f4acbe00640 1 -- 172.21.15.22:0/2170523909 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f4ac401a930 con 0x5598ab321460 2024-03-28T18:14:18.902 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:14:18.899+0000 7f4acbe00640 1 -- 172.21.15.22:0/2170523909 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ac40119a0 con 0x5598ab321460 2024-03-28T18:14:18.902 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:14:18.900+0000 7f4ad8c00640 1 -- 172.21.15.22:0/3923887644 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(107..107 src has 1..107) v4 ==== 9615+0+0 (secure 0 0 0) 0x7f4acc0f1310 con 0x7f4adc107e50 2024-03-28T18:14:18.902 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:14:18.900+0000 7f4adbe00640 1 --2- 172.21.15.22:0/3923887644 >> [v2:172.21.15.22:6800/1357365082,v1:172.21.15.22:6801/1357365082] conn(0x7f4ac00a29f0 0x7f4ac00a4eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:14:18.906 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:14:18.900+0000 7f4ae4d72740 1 -- 172.21.15.22:0/2170523909 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5598ab321460 msgr2=0x5598ab323850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:14:18.906 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:14:18.900+0000 7f4ae4d72740 1 --2- 172.21.15.22:0/2170523909 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5598ab321460 0x5598ab323850 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7f4ac400bc10 tx=0x7f4ac400f730 comp rx=0 tx=0).stop 2024-03-28T18:14:18.906 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:14:18.900+0000 7f4ae4d72740 1 -- 172.21.15.22:0/2170523909 shutdown_connections 2024-03-28T18:14:18.906 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:14:18.900+0000 7f4ae4d72740 1 --2- 172.21.15.22:0/2170523909 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5598ab321460 0x5598ab323850 unknown :-1 s=CLOSED pgs=735 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:14:18.906 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:14:18.900+0000 7f4ae4d72740 1 --2- 172.21.15.22:0/2170523909 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5598ab3180f0 0x5598ab318570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:14:18.907 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:14:18.900+0000 7f4ae4d72740 1 --2- 172.21.15.22:0/2170523909 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5598ab318dd0 0x5598ab317b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:14:18.907 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:14:18.900+0000 7f4ae4d72740 1 -- 172.21.15.22:0/2170523909 >> 172.21.15.22:0/2170523909 conn(0x5598ab2df200 msgr2=0x5598ab312b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:14:18.907 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:14:18.900+0000 7f4adbe00640 1 --2- 172.21.15.22:0/3923887644 >> [v2:172.21.15.22:6800/1357365082,v1:172.21.15.22:6801/1357365082] conn(0x7f4ac00a29f0 0x7f4ac00a4eb0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f4ad4005c60 tx=0x7f4ad4005bd0 comp rx=0 tx=0).ready entity=mgr.15522 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:14:18.907 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:14:18.901+0000 7f4ae4d72740 1 -- 172.21.15.22:0/2170523909 shutdown_connections 2024-03-28T18:14:18.907 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:14:18.901+0000 7f4ae4d72740 1 -- 172.21.15.22:0/2170523909 wait complete. 2024-03-28T18:14:18.907 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:14:18.901+0000 7f4ae4d72740 1 -- start start 2024-03-28T18:14:18.907 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:14:18.901+0000 7f4ae4d72740 1 Processor -- start 2024-03-28T18:14:18.907 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:14:18.902+0000 7f4ae4d72740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5598ab3180f0 0x5598ab320a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:14:18.907 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:14:18.902+0000 7f4ae4d72740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5598ab318dd0 0x5598ab31de90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:14:18.907 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:14:18.902+0000 7f4ae4d72740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5598ab321460 0x5598ab31e3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:14:18.907 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:14:18.902+0000 7f4ae4d72740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x5598ab324320 con 0x5598ab318dd0 2024-03-28T18:14:18.907 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:14:18.902+0000 7f4ae4d72740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x5598ab237e80 con 0x5598ab3180f0 2024-03-28T18:14:18.907 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:14:18.902+0000 7f4ae4d72740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x5598ab3244a0 con 0x5598ab321460 2024-03-28T18:14:18.908 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:14:18.902+0000 7f4ae4d72740 10 client.0 _ll_register_callbacks cb 0x5598ab24a130 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-28T18:14:18.908 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:14:18.902+0000 7f4ad3e00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5598ab318dd0 0x5598ab31de90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:14:18.908 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:14:18.902+0000 7f4ad3e00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5598ab318dd0 0x5598ab31de90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:41840/0 (socket says 172.21.15.22:41840) 2024-03-28T18:14:18.908 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:14:18.902+0000 7f4ad3e00640 1 -- 172.21.15.22:0/1156181914 learned_addr learned my addr 172.21.15.22:0/1156181914 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-28T18:14:18.908 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:14:18.902+0000 7f4adb400640 1 --2- 172.21.15.22:0/1156181914 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5598ab3180f0 0x5598ab320a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:14:18.908 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:14:18.902+0000 7f4ad3e00640 1 -- 172.21.15.22:0/1156181914 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5598ab321460 msgr2=0x5598ab31e3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T18:14:18.908 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:14:18.902+0000 7f4adbe00640 1 --2- 172.21.15.22:0/1156181914 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5598ab321460 0x5598ab31e3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:14:18.908 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:14:18.902+0000 7f4ad3e00640 1 --2- 172.21.15.22:0/1156181914 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5598ab321460 0x5598ab31e3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:14:18.908 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:14:18.902+0000 7f4ad3e00640 1 -- 172.21.15.22:0/1156181914 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5598ab3180f0 msgr2=0x5598ab320a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T18:14:18.908 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:14:18.902+0000 7f4ad3e00640 1 --2- 172.21.15.22:0/1156181914 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5598ab3180f0 0x5598ab320a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:14:18.908 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:14:18.902+0000 7f4ad3e00640 1 -- 172.21.15.22:0/1156181914 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5598ab05a7d0 con 0x5598ab318dd0 2024-03-28T18:14:18.908 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:14:18.903+0000 7f4ad3e00640 1 --2- 172.21.15.22:0/1156181914 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5598ab318dd0 0x5598ab31de90 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7f4ac4000c00 tx=0x7f4ac400b8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f8f8a439b3ecf849 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:14:18.909 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:14:18.903+0000 7f4ac8c00640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-03-28T18:14:18.909 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:14:18.903+0000 7f4ac8c00640 1 -- 172.21.15.22:0/1156181914 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ac4028060 con 0x5598ab318dd0 2024-03-28T18:14:18.909 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:14:18.903+0000 7f4ac8c00640 1 -- 172.21.15.22:0/1156181914 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(107..107 src has 1..107) v4 ==== 9615+0+0 (secure 0 0 0) 0x7f4ac401aae0 con 0x5598ab318dd0 2024-03-28T18:14:18.909 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:14:18.903+0000 7f4ae4d72740 1 -- 172.21.15.22:0/1156181914 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f4ac401a930 con 0x5598ab318dd0 2024-03-28T18:14:18.909 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:14:18.903+0000 7f4ac8c00640 1 -- 172.21.15.22:0/1156181914 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=127) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4ac402f050 con 0x5598ab318dd0 2024-03-28T18:14:18.909 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:14:18.903+0000 7f4ae4d72740 10 client.16662 fetch_fsmap learned FSMap version 127 2024-03-28T18:14:18.909 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:14:18.903+0000 7f4ae4d72740 1 -- 172.21.15.22:0/1156181914 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=127,monmap=4+}) v3 -- 0x5598aaf903f0 con 0x5598ab318dd0 2024-03-28T18:14:18.909 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:14:18.904+0000 7f4ac8c00640 1 -- 172.21.15.22:0/1156181914 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f4ac401a930 con 0x5598ab318dd0 2024-03-28T18:14:18.909 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:14:18.904+0000 7f4ac8c00640 1 -- 172.21.15.22:0/1156181914 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 127) v1 ==== 6784+0+0 (secure 0 0 0) 0x7f4ac4011a40 con 0x5598ab318dd0 2024-03-28T18:14:18.909 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:14:18.904+0000 7f4ae4d72740 10 client.16662 fetch_fsmap finished waiting for FSMap version 127 2024-03-28T18:14:18.909 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:14:18.904+0000 7f4ae4d72740 10 client.16662 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-28T18:14:18.909 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:14:18.904+0000 7f4ae4d72740 20 client.16662 populate_metadata read hostname 'smithi022' 2024-03-28T18:14:18.909 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:14:18.904+0000 7f4ae4d72740 1 --2- 172.21.15.22:0/1156181914 >> [v2:172.21.15.79:6838/2441542443,v1:172.21.15.79:6839/2441542443] conn(0x5598ab3312b0 0x5598ab3336a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:14:18.910 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:14:18.904+0000 7f4ae4d72740 4 client.16662 mds_command: new command op to 34553 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-28T18:14:18.910 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:14:18.904+0000 7f4ae4d72740 1 -- 172.21.15.22:0/1156181914 --> [v2:172.21.15.79:6838/2441542443,v1:172.21.15.79:6839/2441542443] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5598ab289790 con 0x5598ab3312b0 2024-03-28T18:14:18.910 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:14:18.905+0000 7f4adb400640 1 --2- 172.21.15.22:0/1156181914 >> [v2:172.21.15.79:6838/2441542443,v1:172.21.15.79:6839/2441542443] conn(0x5598ab3312b0 0x5598ab3336a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:14:18.910 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:14:18.906+0000 7f4adb400640 1 --2- 172.21.15.22:0/1156181914 >> [v2:172.21.15.79:6838/2441542443,v1:172.21.15.79:6839/2441542443] conn(0x5598ab3312b0 0x5598ab3336a0 crc :-1 s=READY pgs=208 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=92a7df7ecbe6f9b0 server_cookie=e18c6f2a8f7e9b47 in_seq=0 out_seq=0 2024-03-28T18:14:18.910 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:14:18.906+0000 7f4ac8c00640 10 client.16662 ms_handle_connect on v2:172.21.15.79:6838/2441542443 2024-03-28T18:14:18.911 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:14:18.907+0000 7f4ac8c00640 1 -- 172.21.15.22:0/1156181914 <== mds.0 v2:172.21.15.79:6838/2441542443 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x5598ab289790 con 0x5598ab3312b0 2024-03-28T18:14:18.911 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:14:18.907+0000 7f4ac8c00640 10 client.16662 handle_command_reply: tid=0 2024-03-28T18:14:18.912 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:14:18.907+0000 7f4ae4d72740 1 client.16662 shutdown 2024-03-28T18:14:18.912 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:14:18.908+0000 7f4ae4d72740 1 -- 172.21.15.22:0/1156181914 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5598ab318dd0 msgr2=0x5598ab31de90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:14:18.912 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:14:18.908+0000 7f4ae4d72740 1 --2- 172.21.15.22:0/1156181914 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5598ab318dd0 0x5598ab31de90 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7f4ac4000c00 tx=0x7f4ac400b8e0 comp rx=0 tx=0).stop 2024-03-28T18:14:18.912 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:14:18.908+0000 7f4ae4d72740 1 -- 172.21.15.22:0/1156181914 shutdown_connections 2024-03-28T18:14:18.912 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:14:18.908+0000 7f4ae4d72740 1 --2- 172.21.15.22:0/1156181914 >> [v2:172.21.15.79:6838/2441542443,v1:172.21.15.79:6839/2441542443] conn(0x5598ab3312b0 0x5598ab3336a0 crc :-1 s=READY pgs=208 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:14:18.912 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:14:18.908+0000 7f4ae4d72740 1 --2- 172.21.15.22:0/1156181914 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5598ab321460 0x5598ab31e3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:14:18.912 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:14:18.908+0000 7f4ae4d72740 1 --2- 172.21.15.22:0/1156181914 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5598ab318dd0 0x5598ab31de90 unknown :-1 s=CLOSED pgs=736 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:14:18.912 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:14:18.908+0000 7f4ae4d72740 1 --2- 172.21.15.22:0/1156181914 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5598ab3180f0 0x5598ab320a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:14:18.912 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:14:18.908+0000 7f4ac8c00640 0 client.16662 ms_handle_reset on v2:172.21.15.79:6838/2441542443 2024-03-28T18:14:18.912 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:14:18.908+0000 7f4ae4d72740 1 -- 172.21.15.22:0/1156181914 >> 172.21.15.22:0/1156181914 conn(0x5598ab2df200 msgr2=0x5598ab314890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:14:18.912 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:14:18.908+0000 7f4ae4d72740 1 -- 172.21.15.22:0/1156181914 shutdown_connections 2024-03-28T18:14:18.912 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:14:18.908+0000 7f4ae4d72740 1 -- 172.21.15.22:0/1156181914 wait complete. 2024-03-28T18:14:18.912 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:14:18.908+0000 7f4ae4d72740 20 client.16662 trim_cache size 0 max 16384 2024-03-28T18:14:18.927 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:14:18.926+0000 7f4ae4d72740 1 Processor -- start 2024-03-28T18:14:18.927 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:14:18.926+0000 7f4ae4d72740 1 -- start start 2024-03-28T18:14:18.927 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:14:18.926+0000 7f4ae4d72740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5598ab3180f0 0x5598ab322850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:14:18.927 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:14:18.926+0000 7f4ae4d72740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5598ab318dd0 0x5598ab322d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:14:18.927 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:14:18.926+0000 7f4ae4d72740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5598ab321460 0x5598ab31b130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:14:18.927 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:14:18.926+0000 7f4ae4d72740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4adc1b2a00 con 0x5598ab321460 2024-03-28T18:14:18.927 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:14:18.926+0000 7f4ae4d72740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4adc1b5040 con 0x5598ab3180f0 2024-03-28T18:14:18.927 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:14:18.926+0000 7f4ae4d72740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4adc1b3060 con 0x5598ab318dd0 2024-03-28T18:14:18.928 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:14:18.926+0000 7f4adbe00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5598ab321460 0x5598ab31b130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:14:18.928 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:14:18.926+0000 7f4adbe00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5598ab321460 0x5598ab31b130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:41856/0 (socket says 172.21.15.22:41856) 2024-03-28T18:14:18.929 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:14:18.927+0000 7f4adbe00640 1 -- 172.21.15.22:0/1296775250 learned_addr learned my addr 172.21.15.22:0/1296775250 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-28T18:14:18.929 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:14:18.927+0000 7f4adb400640 1 --2- 172.21.15.22:0/1296775250 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5598ab3180f0 0x5598ab322850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:14:18.929 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:14:18.927+0000 7f4ad3e00640 1 --2- 172.21.15.22:0/1296775250 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5598ab318dd0 0x5598ab322d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:14:18.929 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:14:18.927+0000 7f4adbe00640 1 -- 172.21.15.22:0/1296775250 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5598ab318dd0 msgr2=0x5598ab322d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T18:14:18.929 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:14:18.927+0000 7f4adbe00640 1 --2- 172.21.15.22:0/1296775250 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5598ab318dd0 0x5598ab322d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:14:18.929 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:14:18.927+0000 7f4adbe00640 1 -- 172.21.15.22:0/1296775250 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5598ab3180f0 msgr2=0x5598ab322850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T18:14:18.929 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:14:18.927+0000 7f4adbe00640 1 --2- 172.21.15.22:0/1296775250 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5598ab3180f0 0x5598ab322850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:14:18.929 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:14:18.927+0000 7f4adbe00640 1 -- 172.21.15.22:0/1296775250 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5598aaf500f0 con 0x5598ab321460 2024-03-28T18:14:18.929 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:14:18.927+0000 7f4adb400640 1 --2- 172.21.15.22:0/1296775250 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5598ab3180f0 0x5598ab322850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-28T18:14:18.929 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:14:18.927+0000 7f4ad3e00640 1 --2- 172.21.15.22:0/1296775250 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5598ab318dd0 0x5598ab322d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T18:14:18.929 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:14:18.927+0000 7f4adbe00640 1 --2- 172.21.15.22:0/1296775250 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5598ab321460 0x5598ab31b130 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7f4ad4011790 tx=0x7f4ad4013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=603933d896127107 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:14:18.929 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:14:18.928+0000 7f4acb400640 1 -- 172.21.15.22:0/1296775250 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ad401db30 con 0x5598ab321460 2024-03-28T18:14:18.930 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:14:18.928+0000 7f4acb400640 1 -- 172.21.15.22:0/1296775250 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f4ad401dcd0 con 0x5598ab321460 2024-03-28T18:14:18.930 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:14:18.928+0000 7f4acb400640 1 -- 172.21.15.22:0/1296775250 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ad401b670 con 0x5598ab321460 2024-03-28T18:14:18.933 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:14:18.928+0000 7f4ae4d72740 1 -- 172.21.15.22:0/1296775250 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5598ab321460 msgr2=0x5598ab31b130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:14:18.933 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:14:18.928+0000 7f4ae4d72740 1 --2- 172.21.15.22:0/1296775250 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5598ab321460 0x5598ab31b130 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7f4ad4011790 tx=0x7f4ad4013000 comp rx=0 tx=0).stop 2024-03-28T18:14:18.933 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:14:18.928+0000 7f4ae4d72740 1 -- 172.21.15.22:0/1296775250 shutdown_connections 2024-03-28T18:14:18.933 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:14:18.928+0000 7f4ae4d72740 1 --2- 172.21.15.22:0/1296775250 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5598ab321460 0x5598ab31b130 unknown :-1 s=CLOSED pgs=737 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:14:18.933 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:14:18.928+0000 7f4ae4d72740 1 --2- 172.21.15.22:0/1296775250 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5598ab318dd0 0x5598ab322d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:14:18.933 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:14:18.928+0000 7f4ae4d72740 1 --2- 172.21.15.22:0/1296775250 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5598ab3180f0 0x5598ab322850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:14:18.933 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:14:18.928+0000 7f4ae4d72740 1 -- 172.21.15.22:0/1296775250 >> 172.21.15.22:0/1296775250 conn(0x5598ab2df200 msgr2=0x5598ab31c780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:14:18.933 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:14:18.928+0000 7f4ae4d72740 1 -- 172.21.15.22:0/1296775250 shutdown_connections 2024-03-28T18:14:18.934 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:14:18.929+0000 7f4ae4d72740 1 -- 172.21.15.22:0/1296775250 wait complete. 2024-03-28T18:14:18.934 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:14:18.929+0000 7f4ae4d72740 1 -- start start 2024-03-28T18:14:18.934 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:14:18.929+0000 7f4ae4d72740 1 Processor -- start 2024-03-28T18:14:18.934 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:14:18.929+0000 7f4ae4d72740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5598ab3180f0 0x5598ab31a8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:14:18.934 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:14:18.929+0000 7f4ae4d72740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5598ab318dd0 0x5598ab31adf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:14:18.934 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:14:18.929+0000 7f4ae4d72740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5598ab321460 0x5598ab38ba70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:14:18.934 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:14:18.929+0000 7f4ae4d72740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4adc1b36c0 con 0x5598ab321460 2024-03-28T18:14:18.934 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:14:18.930+0000 7f4ae4d72740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4adc1b49e0 con 0x5598ab318dd0 2024-03-28T18:14:18.934 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:14:18.930+0000 7f4ae4d72740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4adc1b3d20 con 0x5598ab3180f0 2024-03-28T18:14:18.934 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:14:18.930+0000 7f4ae4d72740 10 client.0 _ll_register_callbacks cb 0x7f4ac400ba90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-28T18:14:18.934 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:14:18.930+0000 7f4adbe00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5598ab321460 0x5598ab38ba70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:14:18.935 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:14:18.930+0000 7f4adbe00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5598ab321460 0x5598ab38ba70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:41862/0 (socket says 172.21.15.22:41862) 2024-03-28T18:14:18.935 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:14:18.930+0000 7f4adbe00640 1 -- 172.21.15.22:0/2795073476 learned_addr learned my addr 172.21.15.22:0/2795073476 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-28T18:14:18.935 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:14:18.930+0000 7f4ad3e00640 1 --2- 172.21.15.22:0/2795073476 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5598ab318dd0 0x5598ab31adf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:14:18.935 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:14:18.930+0000 7f4adb400640 1 --2- 172.21.15.22:0/2795073476 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5598ab3180f0 0x5598ab31a8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:14:18.935 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:14:18.930+0000 7f4adbe00640 1 -- 172.21.15.22:0/2795073476 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5598ab3180f0 msgr2=0x5598ab31a8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T18:14:18.935 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:14:18.930+0000 7f4adbe00640 1 --2- 172.21.15.22:0/2795073476 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5598ab3180f0 0x5598ab31a8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:14:18.935 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:14:18.930+0000 7f4adbe00640 1 -- 172.21.15.22:0/2795073476 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5598ab318dd0 msgr2=0x5598ab31adf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T18:14:18.935 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:14:18.930+0000 7f4adbe00640 1 --2- 172.21.15.22:0/2795073476 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5598ab318dd0 0x5598ab31adf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:14:18.935 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:14:18.930+0000 7f4adbe00640 1 -- 172.21.15.22:0/2795073476 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5598ab38bfb0 con 0x5598ab321460 2024-03-28T18:14:18.935 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:14:18.931+0000 7f4adbe00640 1 --2- 172.21.15.22:0/2795073476 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5598ab321460 0x5598ab38ba70 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7f4ad4011760 tx=0x7f4ad4003c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=630cab931d777f56 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:14:18.935 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:14:18.931+0000 7f4ac8c00640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-03-28T18:14:18.935 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:14:18.931+0000 7f4ac8c00640 1 -- 172.21.15.22:0/2795073476 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ad4035020 con 0x5598ab321460 2024-03-28T18:14:18.935 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:14:18.931+0000 7f4ac8c00640 1 -- 172.21.15.22:0/2795073476 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(107..107 src has 1..107) v4 ==== 9615+0+0 (secure 0 0 0) 0x7f4ad401e630 con 0x5598ab321460 2024-03-28T18:14:18.936 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:14:18.931+0000 7f4ae4d72740 1 -- 172.21.15.22:0/2795073476 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f4ad401dcd0 con 0x5598ab321460 2024-03-28T18:14:18.936 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:14:18.931+0000 7f4ac8c00640 1 -- 172.21.15.22:0/2795073476 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=127) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4ad401de80 con 0x5598ab321460 2024-03-28T18:14:18.936 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:14:18.931+0000 7f4ae4d72740 10 client.16668 fetch_fsmap learned FSMap version 127 2024-03-28T18:14:18.936 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:14:18.931+0000 7f4ae4d72740 1 -- 172.21.15.22:0/2795073476 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=127,monmap=4+}) v3 -- 0x5598ab38d0d0 con 0x5598ab321460 2024-03-28T18:14:18.936 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:14:18.932+0000 7f4ac8c00640 1 -- 172.21.15.22:0/2795073476 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f4ad401dcd0 con 0x5598ab321460 2024-03-28T18:14:18.936 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:14:18.932+0000 7f4ac8c00640 1 -- 172.21.15.22:0/2795073476 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 127) v1 ==== 6784+0+0 (secure 0 0 0) 0x7f4ad40045a0 con 0x5598ab321460 2024-03-28T18:14:18.936 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:14:18.932+0000 7f4ae4d72740 10 client.16668 fetch_fsmap finished waiting for FSMap version 127 2024-03-28T18:14:18.936 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:14:18.932+0000 7f4ae4d72740 10 client.16668 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-28T18:14:18.936 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:14:18.932+0000 7f4ae4d72740 20 client.16668 populate_metadata read hostname 'smithi022' 2024-03-28T18:14:18.936 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:14:18.932+0000 7f4ae4d72740 1 --2- 172.21.15.22:0/2795073476 >> [v2:172.21.15.79:6838/2441542443,v1:172.21.15.79:6839/2441542443] conn(0x5598ab38d740 0x5598ab38fb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:14:18.936 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:14:18.932+0000 7f4ae4d72740 4 client.16668 mds_command: new command op to 34553 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-28T18:14:18.936 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:14:18.932+0000 7f4ae4d72740 1 -- 172.21.15.22:0/2795073476 --> [v2:172.21.15.79:6838/2441542443,v1:172.21.15.79:6839/2441542443] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x5598ab283ae0 con 0x5598ab38d740 2024-03-28T18:14:18.936 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:14:18.932+0000 7f4adb400640 1 --2- 172.21.15.22:0/2795073476 >> [v2:172.21.15.79:6838/2441542443,v1:172.21.15.79:6839/2441542443] conn(0x5598ab38d740 0x5598ab38fb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:14:18.938 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:14:18.933+0000 7f4adb400640 1 --2- 172.21.15.22:0/2795073476 >> [v2:172.21.15.79:6838/2441542443,v1:172.21.15.79:6839/2441542443] conn(0x5598ab38d740 0x5598ab38fb60 crc :-1 s=READY pgs=209 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ff91fd4514d19ee1 server_cookie=4ab6d0e4a5bea2c9 in_seq=0 out_seq=0 2024-03-28T18:14:18.938 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:14:18.933+0000 7f4ac8c00640 10 client.16668 ms_handle_connect on v2:172.21.15.79:6838/2441542443 2024-03-28T18:14:18.938 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:14:18.933+0000 7f4ac8c00640 1 -- 172.21.15.22:0/2795073476 <== mds.0 v2:172.21.15.79:6838/2441542443 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x5598ab283ae0 con 0x5598ab38d740 2024-03-28T18:14:18.938 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:14:18.933+0000 7f4ac8c00640 10 client.16668 handle_command_reply: tid=0 2024-03-28T18:14:18.938 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:14:18.933+0000 7f4ae4d72740 1 client.16668 shutdown 2024-03-28T18:14:18.938 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:14:18.933+0000 7f4ae4d72740 1 -- 172.21.15.22:0/2795073476 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5598ab321460 msgr2=0x5598ab38ba70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:14:18.938 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:14:18.934+0000 7f4ae4d72740 1 --2- 172.21.15.22:0/2795073476 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5598ab321460 0x5598ab38ba70 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7f4ad4011760 tx=0x7f4ad4003c20 comp rx=0 tx=0).stop 2024-03-28T18:14:18.938 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:14:18.934+0000 7f4ae4d72740 1 -- 172.21.15.22:0/2795073476 shutdown_connections 2024-03-28T18:14:18.938 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:14:18.934+0000 7f4ae4d72740 1 --2- 172.21.15.22:0/2795073476 >> [v2:172.21.15.79:6838/2441542443,v1:172.21.15.79:6839/2441542443] conn(0x5598ab38d740 0x5598ab38fb60 crc :-1 s=READY pgs=209 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:14:18.938 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:14:18.934+0000 7f4ae4d72740 1 --2- 172.21.15.22:0/2795073476 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5598ab321460 0x5598ab38ba70 unknown :-1 s=CLOSED pgs=738 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:14:18.939 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:14:18.934+0000 7f4ae4d72740 1 --2- 172.21.15.22:0/2795073476 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5598ab318dd0 0x5598ab31adf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:14:18.939 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:14:18.934+0000 7f4ae4d72740 1 --2- 172.21.15.22:0/2795073476 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5598ab3180f0 0x5598ab31a8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:14:18.939 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:14:18.934+0000 7f4ac8c00640 0 client.16668 ms_handle_reset on v2:172.21.15.79:6838/2441542443 2024-03-28T18:14:18.939 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:14:18.934+0000 7f4ae4d72740 1 -- 172.21.15.22:0/2795073476 >> 172.21.15.22:0/2795073476 conn(0x5598ab2df200 msgr2=0x5598ab311330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:14:18.939 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:14:18.934+0000 7f4ae4d72740 1 -- 172.21.15.22:0/2795073476 shutdown_connections 2024-03-28T18:14:18.939 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:14:18.934+0000 7f4ae4d72740 1 -- 172.21.15.22:0/2795073476 wait complete. 2024-03-28T18:14:18.939 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:14:18.934+0000 7f4ae4d72740 20 client.16668 trim_cache size 0 max 16384 2024-03-28T18:14:18.939 INFO:teuthology.orchestra.run.smithi022.stdout:{ 2024-03-28T18:14:18.939 INFO:teuthology.orchestra.run.smithi022.stdout: "status": "IDLE (waiting for more scrubs)", 2024-03-28T18:14:18.940 INFO:teuthology.orchestra.run.smithi022.stdout: "scrubs": { 2024-03-28T18:14:18.940 INFO:teuthology.orchestra.run.smithi022.stdout: "4bbbd61d-29a3-4bf6-a4f2-d5783d585de3": { 2024-03-28T18:14:18.940 INFO:teuthology.orchestra.run.smithi022.stdout: "path": "/", 2024-03-28T18:14:18.940 INFO:teuthology.orchestra.run.smithi022.stdout: "tag": "4bbbd61d-29a3-4bf6-a4f2-d5783d585de3", 2024-03-28T18:14:18.940 INFO:teuthology.orchestra.run.smithi022.stdout: "options": "recursive,force" 2024-03-28T18:14:18.940 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-03-28T18:14:18.940 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-03-28T18:14:18.940 INFO:teuthology.orchestra.run.smithi022.stdout:} 2024-03-28T18:14:18.941 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:14:18.934+0000 7f4ae0a00640 1 -- 172.21.15.22:0/3923887644 >> [v2:172.21.15.22:6800/1357365082,v1:172.21.15.22:6801/1357365082] conn(0x7f4ac00a29f0 msgr2=0x7f4ac00a4eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:14:18.941 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:14:18.934+0000 7f4ae0a00640 1 --2- 172.21.15.22:0/3923887644 >> [v2:172.21.15.22:6800/1357365082,v1:172.21.15.22:6801/1357365082] conn(0x7f4ac00a29f0 0x7f4ac00a4eb0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f4ad4005c60 tx=0x7f4ad4005bd0 comp rx=0 tx=0).stop 2024-03-28T18:14:18.941 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:14:18.934+0000 7f4ae0a00640 1 -- 172.21.15.22:0/3923887644 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4adc107e50 msgr2=0x7f4adc19e490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:14:18.941 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:14:18.934+0000 7f4ae0a00640 1 --2- 172.21.15.22:0/3923887644 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4adc107e50 0x7f4adc19e490 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7f4acc000c00 tx=0x7f4acc0045e0 comp rx=0 tx=0).stop 2024-03-28T18:14:18.941 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:14:18.935+0000 7f4ae0a00640 1 -- 172.21.15.22:0/3923887644 shutdown_connections 2024-03-28T18:14:18.941 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:14:18.935+0000 7f4ae0a00640 1 --2- 172.21.15.22:0/3923887644 >> [v2:172.21.15.22:6800/1357365082,v1:172.21.15.22:6801/1357365082] conn(0x7f4ac00a29f0 0x7f4ac00a4eb0 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:14:18.941 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:14:18.935+0000 7f4ae0a00640 1 --2- 172.21.15.22:0/3923887644 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f4adc1a2d00 0x7f4adc1a31b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:14:18.941 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:14:18.935+0000 7f4ae0a00640 1 --2- 172.21.15.22:0/3923887644 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4adc10cea0 0x7f4adc19e9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:14:18.941 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:14:18.935+0000 7f4ae0a00640 1 --2- 172.21.15.22:0/3923887644 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4adc107e50 0x7f4adc19e490 unknown :-1 s=CLOSED pgs=734 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:14:18.941 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:14:18.935+0000 7f4ae0a00640 1 -- 172.21.15.22:0/3923887644 >> 172.21.15.22:0/3923887644 conn(0x7f4adc0774c0 msgr2=0x7f4adc077b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:14:18.942 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:14:18.935+0000 7f4ae0a00640 1 -- 172.21.15.22:0/3923887644 shutdown_connections 2024-03-28T18:14:18.942 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:14:18.935+0000 7f4ae0a00640 1 -- 172.21.15.22:0/3923887644 wait complete. 2024-03-28T18:14:18.953 INFO:tasks.cephfs.filesystem:scrub status for tag:4bbbd61d-29a3-4bf6-a4f2-d5783d585de3 - {'path': '/', 'tag': '4bbbd61d-29a3-4bf6-a4f2-d5783d585de3', 'options': 'recursive,force'} 2024-03-28T18:14:19.432 INFO:journalctl@ceph.mds.b.smithi079.stdout:Mar 28 18:14:19 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-b[65245]: 2024-03-28T18:14:19.095+0000 7f845f400700 -1 mds.pinger is_rank_lagging: rank=1 was never sent ping request. 2024-03-28T18:14:20.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:14:20 smithi022 ceph-mon[31985]: pgmap v220: 97 pgs: 97 active+clean; 117 MiB data, 775 MiB used, 1.0 TiB / 1.0 TiB avail; 119 KiB/s rd, 31 KiB/s wr, 42 op/s 2024-03-28T18:14:20.674 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:14:20 smithi134 ceph-mon[35037]: pgmap v220: 97 pgs: 97 active+clean; 117 MiB data, 775 MiB used, 1.0 TiB / 1.0 TiB avail; 119 KiB/s rd, 31 KiB/s wr, 42 op/s 2024-03-28T18:14:20.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:14:20 smithi079 ceph-mon[35312]: pgmap v220: 97 pgs: 97 active+clean; 117 MiB data, 775 MiB used, 1.0 TiB / 1.0 TiB avail; 119 KiB/s rd, 31 KiB/s wr, 42 op/s 2024-03-28T18:14:22.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:14:22 smithi022 ceph-mon[31985]: pgmap v221: 97 pgs: 97 active+clean; 117 MiB data, 775 MiB used, 1.0 TiB / 1.0 TiB avail; 119 KiB/s rd, 31 KiB/s wr, 42 op/s 2024-03-28T18:14:22.674 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:14:22 smithi134 ceph-mon[35037]: pgmap v221: 97 pgs: 97 active+clean; 117 MiB data, 775 MiB used, 1.0 TiB / 1.0 TiB avail; 119 KiB/s rd, 31 KiB/s wr, 42 op/s 2024-03-28T18:14:22.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:14:22 smithi079 ceph-mon[35312]: pgmap v221: 97 pgs: 97 active+clean; 117 MiB data, 775 MiB used, 1.0 TiB / 1.0 TiB avail; 119 KiB/s rd, 31 KiB/s wr, 42 op/s 2024-03-28T18:14:24.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:14:24 smithi022 ceph-mon[31985]: pgmap v222: 97 pgs: 97 active+clean; 117 MiB data, 775 MiB used, 1.0 TiB / 1.0 TiB avail; 103 KiB/s rd, 27 KiB/s wr, 36 op/s 2024-03-28T18:14:24.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:14:24 smithi134 ceph-mon[35037]: pgmap v222: 97 pgs: 97 active+clean; 117 MiB data, 775 MiB used, 1.0 TiB / 1.0 TiB avail; 103 KiB/s rd, 27 KiB/s wr, 36 op/s 2024-03-28T18:14:24.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:14:24 smithi079 ceph-mon[35312]: pgmap v222: 97 pgs: 97 active+clean; 117 MiB data, 775 MiB used, 1.0 TiB / 1.0 TiB avail; 103 KiB/s rd, 27 KiB/s wr, 36 op/s 2024-03-28T18:14:26.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:14:26 smithi022 ceph-mon[31985]: pgmap v223: 97 pgs: 97 active+clean; 117 MiB data, 775 MiB used, 1.0 TiB / 1.0 TiB avail; 101 KiB/s rd, 26 KiB/s wr, 39 op/s 2024-03-28T18:14:26.674 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:14:26 smithi134 ceph-mon[35037]: pgmap v223: 97 pgs: 97 active+clean; 117 MiB data, 775 MiB used, 1.0 TiB / 1.0 TiB avail; 101 KiB/s rd, 26 KiB/s wr, 39 op/s 2024-03-28T18:14:26.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:14:26 smithi079 ceph-mon[35312]: pgmap v223: 97 pgs: 97 active+clean; 117 MiB data, 775 MiB used, 1.0 TiB / 1.0 TiB avail; 101 KiB/s rd, 26 KiB/s wr, 39 op/s 2024-03-28T18:14:27.747 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:14:27 smithi022 systemd[1]: ceph-fd220748-ed2b-11ee-95d0-87774f69a715@mds.d.service: Scheduled restart job, restart counter is at 2. 2024-03-28T18:14:27.748 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:14:27 smithi022 systemd[1]: Stopped Ceph mds.d for fd220748-ed2b-11ee-95d0-87774f69a715. 2024-03-28T18:14:27.748 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:14:27 smithi022 systemd[1]: ceph-fd220748-ed2b-11ee-95d0-87774f69a715@mds.d.service: Consumed 1.958s CPU time. 2024-03-28T18:14:27.748 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:14:27 smithi022 systemd[1]: Starting Ceph mds.d for fd220748-ed2b-11ee-95d0-87774f69a715... 2024-03-28T18:14:28.032 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:14:27 smithi022 podman[93798]: 2024-03-28 18:14:27.840156011 +0000 UTC m=+0.034148043 image pull c664afe56bb01db4f9afa9532c20598e201e664f2e9ba2be50db0d52674a26d2 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc 2024-03-28T18:14:28.032 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:14:27 smithi022 podman[93798]: 2024-03-28 18:14:27.948486 +0000 UTC m=+0.142478059 container create 1ef25f2e549c69a5228280f441b068b1bef43e7dde03ef23025a8fcfde8af15b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_CLEAN=True, ceph=True, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_BRANCH=HEAD, org.label-schema.build-date=20240319, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2) 2024-03-28T18:14:28.334 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:14:28 smithi022 ceph-mon[31985]: pgmap v224: 97 pgs: 97 active+clean; 117 MiB data, 775 MiB used, 1.0 TiB / 1.0 TiB avail; 98 KiB/s rd, 26 KiB/s wr, 34 op/s 2024-03-28T18:14:28.334 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:14:28 smithi022 podman[93798]: 2024-03-28 18:14:28.277561132 +0000 UTC m=+0.471553198 container init 1ef25f2e549c69a5228280f441b068b1bef43e7dde03ef23025a8fcfde8af15b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d, org.label-schema.build-date=20240319, GIT_CLEAN=True, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-28T18:14:28.334 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:14:28 smithi022 podman[93798]: 2024-03-28 18:14:28.290852257 +0000 UTC m=+0.484844286 container start 1ef25f2e549c69a5228280f441b068b1bef43e7dde03ef23025a8fcfde8af15b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240319, org.label-schema.license=GPLv2, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.vendor=CentOS) 2024-03-28T18:14:28.674 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:14:28 smithi134 ceph-mon[35037]: pgmap v224: 97 pgs: 97 active+clean; 117 MiB data, 775 MiB used, 1.0 TiB / 1.0 TiB avail; 98 KiB/s rd, 26 KiB/s wr, 34 op/s 2024-03-28T18:14:28.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:14:28 smithi079 ceph-mon[35312]: pgmap v224: 97 pgs: 97 active+clean; 117 MiB data, 775 MiB used, 1.0 TiB / 1.0 TiB avail; 98 KiB/s rd, 26 KiB/s wr, 34 op/s 2024-03-28T18:14:28.782 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:14:28 smithi022 ceph-mds[93825]: -- 172.21.15.22:0/291026323 <== mon.0 v2:172.21.15.22:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55fa945c5a40 con 0x55fa94599000 2024-03-28T18:14:28.782 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:14:28 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[93820]: starting mds.d at 2024-03-28T18:14:28.782 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:14:28 smithi022 bash[93798]: 1ef25f2e549c69a5228280f441b068b1bef43e7dde03ef23025a8fcfde8af15b 2024-03-28T18:14:28.782 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:14:28 smithi022 systemd[1]: Started Ceph mds.d for fd220748-ed2b-11ee-95d0-87774f69a715. 2024-03-28T18:14:29.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:14:29 smithi022 ceph-mon[31985]: daemon mds.d restarted 2024-03-28T18:14:29.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:14:29 smithi022 ceph-mon[31985]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-03-28T18:14:29.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:14:29 smithi022 ceph-mon[31985]: osdmap e108: 12 total, 12 up, 12 in 2024-03-28T18:14:29.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:14:29 smithi022 ceph-mon[31985]: mds.? [v2:172.21.15.22:6836/2984059496,v1:172.21.15.22:6837/2984059496] up:boot 2024-03-28T18:14:29.533 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:14:29 smithi022 ceph-mon[31985]: Standby daemon mds.j assigned to filesystem cephfs as rank 1 2024-03-28T18:14:29.533 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:14:29 smithi022 ceph-mon[31985]: fsmap cephfs:4/5 {0=b=up:active,2=e=up:active,3=k=up:active,4=i=up:active} 5 up:standby-replay 1 up:standby, 1 failed 2024-03-28T18:14:29.533 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:14:29 smithi022 ceph-mon[31985]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-03-28T18:14:29.533 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:14:29 smithi022 ceph-mon[31985]: fsmap cephfs:5/5 4 up:standby-replay 1 up:standby 1 up:replay 4 up:active 2024-03-28T18:14:29.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:14:29 smithi134 ceph-mon[35037]: daemon mds.d restarted 2024-03-28T18:14:29.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:14:29 smithi134 ceph-mon[35037]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-03-28T18:14:29.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:14:29 smithi134 ceph-mon[35037]: osdmap e108: 12 total, 12 up, 12 in 2024-03-28T18:14:29.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:14:29 smithi134 ceph-mon[35037]: mds.? [v2:172.21.15.22:6836/2984059496,v1:172.21.15.22:6837/2984059496] up:boot 2024-03-28T18:14:29.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:14:29 smithi134 ceph-mon[35037]: Standby daemon mds.j assigned to filesystem cephfs as rank 1 2024-03-28T18:14:29.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:14:29 smithi134 ceph-mon[35037]: fsmap cephfs:4/5 {0=b=up:active,2=e=up:active,3=k=up:active,4=i=up:active} 5 up:standby-replay 1 up:standby, 1 failed 2024-03-28T18:14:29.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:14:29 smithi134 ceph-mon[35037]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-03-28T18:14:29.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:14:29 smithi134 ceph-mon[35037]: fsmap cephfs:5/5 4 up:standby-replay 1 up:standby 1 up:replay 4 up:active 2024-03-28T18:14:29.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:14:29 smithi079 ceph-mon[35312]: daemon mds.d restarted 2024-03-28T18:14:29.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:14:29 smithi079 ceph-mon[35312]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-03-28T18:14:29.683 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:14:29 smithi079 ceph-mon[35312]: osdmap e108: 12 total, 12 up, 12 in 2024-03-28T18:14:29.683 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:14:29 smithi079 ceph-mon[35312]: mds.? [v2:172.21.15.22:6836/2984059496,v1:172.21.15.22:6837/2984059496] up:boot 2024-03-28T18:14:29.683 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:14:29 smithi079 ceph-mon[35312]: Standby daemon mds.j assigned to filesystem cephfs as rank 1 2024-03-28T18:14:29.683 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:14:29 smithi079 ceph-mon[35312]: fsmap cephfs:4/5 {0=b=up:active,2=e=up:active,3=k=up:active,4=i=up:active} 5 up:standby-replay 1 up:standby, 1 failed 2024-03-28T18:14:29.683 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:14:29 smithi079 ceph-mon[35312]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-03-28T18:14:29.683 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:14:29 smithi079 ceph-mon[35312]: fsmap cephfs:5/5 4 up:standby-replay 1 up:standby 1 up:replay 4 up:active 2024-03-28T18:14:30.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:14:30 smithi134 ceph-mon[35037]: pgmap v226: 97 pgs: 97 active+clean; 117 MiB data, 775 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 102 B/s wr, 9 op/s 2024-03-28T18:14:30.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:14:30 smithi134 ceph-mon[35037]: mds.? [v2:172.21.15.22:6834/803375445,v1:172.21.15.22:6835/803375445] up:resolve 2024-03-28T18:14:30.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:14:30 smithi134 ceph-mon[35037]: fsmap cephfs:5/5 4 up:standby-replay 1 up:standby 1 up:resolve 4 up:active 2024-03-28T18:14:30.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:14:30 smithi079 ceph-mon[35312]: pgmap v226: 97 pgs: 97 active+clean; 117 MiB data, 775 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 102 B/s wr, 9 op/s 2024-03-28T18:14:30.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:14:30 smithi079 ceph-mon[35312]: mds.? [v2:172.21.15.22:6834/803375445,v1:172.21.15.22:6835/803375445] up:resolve 2024-03-28T18:14:30.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:14:30 smithi079 ceph-mon[35312]: fsmap cephfs:5/5 4 up:standby-replay 1 up:standby 1 up:resolve 4 up:active 2024-03-28T18:14:30.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:14:30 smithi022 ceph-mon[31985]: pgmap v226: 97 pgs: 97 active+clean; 117 MiB data, 775 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 102 B/s wr, 9 op/s 2024-03-28T18:14:30.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:14:30 smithi022 ceph-mon[31985]: mds.? [v2:172.21.15.22:6834/803375445,v1:172.21.15.22:6835/803375445] up:resolve 2024-03-28T18:14:30.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:14:30 smithi022 ceph-mon[31985]: fsmap cephfs:5/5 4 up:standby-replay 1 up:standby 1 up:resolve 4 up:active 2024-03-28T18:14:31.674 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:14:31 smithi134 ceph-mon[35037]: reconnect by client.35222 172.21.15.22:0/3729116781 after 0.00399995 2024-03-28T18:14:31.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:14:31 smithi134 ceph-mon[35037]: mds.? [v2:172.21.15.22:6834/803375445,v1:172.21.15.22:6835/803375445] up:reconnect 2024-03-28T18:14:31.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:14:31 smithi134 ceph-mon[35037]: fsmap cephfs:5/5 4 up:standby-replay 1 up:standby 1 up:reconnect 4 up:active 2024-03-28T18:14:31.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:14:31 smithi079 ceph-mon[35312]: reconnect by client.35222 172.21.15.22:0/3729116781 after 0.00399995 2024-03-28T18:14:31.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:14:31 smithi079 ceph-mon[35312]: mds.? [v2:172.21.15.22:6834/803375445,v1:172.21.15.22:6835/803375445] up:reconnect 2024-03-28T18:14:31.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:14:31 smithi079 ceph-mon[35312]: fsmap cephfs:5/5 4 up:standby-replay 1 up:standby 1 up:reconnect 4 up:active 2024-03-28T18:14:31.781 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:14:31 smithi022 ceph-mon[31985]: reconnect by client.35222 172.21.15.22:0/3729116781 after 0.00399995 2024-03-28T18:14:31.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:14:31 smithi022 ceph-mon[31985]: mds.? [v2:172.21.15.22:6834/803375445,v1:172.21.15.22:6835/803375445] up:reconnect 2024-03-28T18:14:31.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:14:31 smithi022 ceph-mon[31985]: fsmap cephfs:5/5 4 up:standby-replay 1 up:standby 1 up:reconnect 4 up:active 2024-03-28T18:14:32.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:14:32 smithi134 ceph-mon[35037]: pgmap v227: 97 pgs: 97 active+clean; 117 MiB data, 775 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 102 B/s wr, 9 op/s 2024-03-28T18:14:32.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:14:32 smithi134 ceph-mon[35037]: mds.? [v2:172.21.15.22:6834/803375445,v1:172.21.15.22:6835/803375445] up:rejoin 2024-03-28T18:14:32.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:14:32 smithi134 ceph-mon[35037]: fsmap cephfs:5/5 4 up:standby-replay 1 up:standby 1 up:rejoin 4 up:active 2024-03-28T18:14:32.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:14:32 smithi134 ceph-mon[35037]: daemon mds.j is now active in filesystem cephfs as rank 1 2024-03-28T18:14:32.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:14:32 smithi079 ceph-mon[35312]: pgmap v227: 97 pgs: 97 active+clean; 117 MiB data, 775 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 102 B/s wr, 9 op/s 2024-03-28T18:14:32.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:14:32 smithi079 ceph-mon[35312]: mds.? [v2:172.21.15.22:6834/803375445,v1:172.21.15.22:6835/803375445] up:rejoin 2024-03-28T18:14:32.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:14:32 smithi079 ceph-mon[35312]: fsmap cephfs:5/5 4 up:standby-replay 1 up:standby 1 up:rejoin 4 up:active 2024-03-28T18:14:32.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:14:32 smithi079 ceph-mon[35312]: daemon mds.j is now active in filesystem cephfs as rank 1 2024-03-28T18:14:32.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:14:32 smithi022 ceph-mon[31985]: pgmap v227: 97 pgs: 97 active+clean; 117 MiB data, 775 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 102 B/s wr, 9 op/s 2024-03-28T18:14:32.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:14:32 smithi022 ceph-mon[31985]: mds.? [v2:172.21.15.22:6834/803375445,v1:172.21.15.22:6835/803375445] up:rejoin 2024-03-28T18:14:32.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:14:32 smithi022 ceph-mon[31985]: fsmap cephfs:5/5 4 up:standby-replay 1 up:standby 1 up:rejoin 4 up:active 2024-03-28T18:14:32.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:14:32 smithi022 ceph-mon[31985]: daemon mds.j is now active in filesystem cephfs as rank 1 2024-03-28T18:14:32.783 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:14:32 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[93603]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-2145-g3be9a5c9/rpm/el8/BUILD/ceph-19.0.0-2145-g3be9a5c9/src/mds/Locker.cc: In function 'void Locker::set_xlocks_done(MutationImpl*, bool)' thread 7f5be8000700 time 2024-03-28T18:14:32.488879+0000 2024-03-28T18:14:32.783 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:14:32 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[93603]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-2145-g3be9a5c9/rpm/el8/BUILD/ceph-19.0.0-2145-g3be9a5c9/src/mds/Locker.cc: 626: FAILED ceph_assert(obj->is_auth()) 2024-03-28T18:14:32.783 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:14:32 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[93603]: ceph version 19.0.0-2145-g3be9a5c9 (3be9a5c9bc793e11e2800a8c0c696e8b46742033) squid (dev) 2024-03-28T18:14:32.783 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:14:32 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[93603]: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x135) [0x7f5bf38284a1] 2024-03-28T18:14:32.783 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:14:32 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[93603]: 2: /usr/lib64/ceph/libceph-common.so.2(+0x2d2667) [0x7f5bf3828667] 2024-03-28T18:14:32.783 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:14:32 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[93603]: 3: (Locker::set_xlocks_done(MutationImpl*, bool)+0x493) [0x561ef716b223] 2024-03-28T18:14:32.783 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:14:32 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[93603]: 4: (Server::early_reply(boost::intrusive_ptr const&, CInode*, CDentry*)+0x3e6) [0x561ef6fe55d6] 2024-03-28T18:14:32.783 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:14:32 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[93603]: 5: (Server::journal_and_reply(boost::intrusive_ptr const&, CInode*, CDentry*, LogEvent*, MDSLogContextBase*)+0xc1) [0x561ef6fe5da1] 2024-03-28T18:14:32.783 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:14:32 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[93603]: 6: (Server::handle_client_mkdir(boost::intrusive_ptr const&)+0x5ba) [0x561ef701b6aa] 2024-03-28T18:14:32.783 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:14:32 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[93603]: 7: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0x733) [0x561ef7038233] 2024-03-28T18:14:32.784 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:14:32 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[93603]: 8: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x561ef703a197] 2024-03-28T18:14:32.784 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:14:32 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[93603]: 9: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x561ef6f3911b] 2024-03-28T18:14:32.784 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:14:32 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[93603]: 10: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x561ef6f396bc] 2024-03-28T18:14:32.784 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:14:32 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[93603]: 11: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bf) [0x561ef6f231ff] 2024-03-28T18:14:32.784 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:14:32 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[93603]: 12: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x478) [0x7f5bf3aba488] 2024-03-28T18:14:32.784 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:14:32 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[93603]: 13: (DispatchQueue::entry()+0x51f) [0x7f5bf3ab760f] 2024-03-28T18:14:32.784 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:14:32 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[93603]: 14: (DispatchQueue::DispatchThread::entry()+0x11) [0x7f5bf3b8ae81] 2024-03-28T18:14:32.784 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:14:32 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[93603]: 15: /lib64/libpthread.so.0(+0x81ca) [0x7f5bf25a51ca] 2024-03-28T18:14:32.784 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:14:32 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[93603]: 16: clone() 2024-03-28T18:14:32.785 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:14:32 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[93603]: 2024-03-28T18:14:32.490+0000 7f5be8000700 -1 /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-2145-g3be9a5c9/rpm/el8/BUILD/ceph-19.0.0-2145-g3be9a5c9/src/mds/Locker.cc: In function 'void Locker::set_xlocks_done(MutationImpl*, bool)' thread 7f5be8000700 time 2024-03-28T18:14:32.488879+0000 2024-03-28T18:14:32.785 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:14:32 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[93603]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-2145-g3be9a5c9/rpm/el8/BUILD/ceph-19.0.0-2145-g3be9a5c9/src/mds/Locker.cc: 626: FAILED ceph_assert(obj->is_auth()) 2024-03-28T18:14:32.785 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:14:32 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[93603]: 2024-03-28T18:14:32.785 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:14:32 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[93603]: ceph version 19.0.0-2145-g3be9a5c9 (3be9a5c9bc793e11e2800a8c0c696e8b46742033) squid (dev) 2024-03-28T18:14:32.785 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:14:32 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[93603]: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x135) [0x7f5bf38284a1] 2024-03-28T18:14:32.785 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:14:32 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[93603]: 2: /usr/lib64/ceph/libceph-common.so.2(+0x2d2667) [0x7f5bf3828667] 2024-03-28T18:14:32.785 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:14:32 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[93603]: 3: (Locker::set_xlocks_done(MutationImpl*, bool)+0x493) [0x561ef716b223] 2024-03-28T18:14:32.785 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:14:32 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[93603]: 4: (Server::early_reply(boost::intrusive_ptr const&, CInode*, CDentry*)+0x3e6) [0x561ef6fe55d6] 2024-03-28T18:14:32.785 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:14:32 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[93603]: 5: (Server::journal_and_reply(boost::intrusive_ptr const&, CInode*, CDentry*, LogEvent*, MDSLogContextBase*)+0xc1) [0x561ef6fe5da1] 2024-03-28T18:14:32.785 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:14:32 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[93603]: 6: (Server::handle_client_mkdir(boost::intrusive_ptr const&)+0x5ba) [0x561ef701b6aa] 2024-03-28T18:14:32.785 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:14:32 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[93603]: 7: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0x733) [0x561ef7038233] 2024-03-28T18:14:32.785 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:14:32 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[93603]: 8: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x561ef703a197] 2024-03-28T18:14:32.785 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:14:32 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[93603]: 9: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x561ef6f3911b] 2024-03-28T18:14:32.785 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:14:32 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[93603]: 10: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x561ef6f396bc] 2024-03-28T18:14:32.785 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:14:32 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[93603]: 11: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bf) [0x561ef6f231ff] 2024-03-28T18:14:32.786 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:14:32 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[93603]: 12: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x478) [0x7f5bf3aba488] 2024-03-28T18:14:32.786 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:14:32 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[93603]: 13: (DispatchQueue::entry()+0x51f) [0x7f5bf3ab760f] 2024-03-28T18:14:32.786 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:14:32 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[93603]: 14: (DispatchQueue::DispatchThread::entry()+0x11) [0x7f5bf3b8ae81] 2024-03-28T18:14:32.786 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:14:32 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[93603]: 15: /lib64/libpthread.so.0(+0x81ca) [0x7f5bf25a51ca] 2024-03-28T18:14:32.786 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:14:32 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[93603]: 16: clone() 2024-03-28T18:14:32.786 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:14:32 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[93603]: 2024-03-28T18:14:32.786 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:14:32 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[93603]: *** Caught signal (Aborted) ** 2024-03-28T18:14:32.786 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:14:32 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[93603]: in thread 7f5be8000700 thread_name:ms_dispatch 2024-03-28T18:14:32.786 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:14:32 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[93603]: ceph version 19.0.0-2145-g3be9a5c9 (3be9a5c9bc793e11e2800a8c0c696e8b46742033) squid (dev) 2024-03-28T18:14:32.786 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:14:32 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[93603]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7f5bf25afd20] 2024-03-28T18:14:32.787 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:14:32 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[93603]: 2: gsignal() 2024-03-28T18:14:32.787 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:14:32 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[93603]: 3: abort() 2024-03-28T18:14:32.787 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:14:32 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[93603]: 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x18f) [0x7f5bf38284fb] 2024-03-28T18:14:32.787 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:14:32 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[93603]: 5: /usr/lib64/ceph/libceph-common.so.2(+0x2d2667) [0x7f5bf3828667] 2024-03-28T18:14:32.787 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:14:32 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[93603]: 6: (Locker::set_xlocks_done(MutationImpl*, bool)+0x493) [0x561ef716b223] 2024-03-28T18:14:32.787 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:14:32 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[93603]: 7: (Server::early_reply(boost::intrusive_ptr const&, CInode*, CDentry*)+0x3e6) [0x561ef6fe55d6] 2024-03-28T18:14:32.787 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:14:32 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[93603]: 8: (Server::journal_and_reply(boost::intrusive_ptr const&, CInode*, CDentry*, LogEvent*, MDSLogContextBase*)+0xc1) [0x561ef6fe5da1] 2024-03-28T18:14:32.787 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:14:32 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[93603]: 9: (Server::handle_client_mkdir(boost::intrusive_ptr const&)+0x5ba) [0x561ef701b6aa] 2024-03-28T18:14:32.788 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:14:32 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[93603]: 10: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0x733) [0x561ef7038233] 2024-03-28T18:14:32.788 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:14:32 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[93603]: 11: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x561ef703a197] 2024-03-28T18:14:32.788 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:14:32 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[93603]: 12: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x561ef6f3911b] 2024-03-28T18:14:32.788 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:14:32 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[93603]: 13: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x561ef6f396bc] 2024-03-28T18:14:32.788 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:14:32 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[93603]: 14: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bf) [0x561ef6f231ff] 2024-03-28T18:14:32.788 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:14:32 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[93603]: 15: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x478) [0x7f5bf3aba488] 2024-03-28T18:14:32.788 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:14:32 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[93603]: 16: (DispatchQueue::entry()+0x51f) [0x7f5bf3ab760f] 2024-03-28T18:14:32.788 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:14:32 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[93603]: 17: (DispatchQueue::DispatchThread::entry()+0x11) [0x7f5bf3b8ae81] 2024-03-28T18:14:32.788 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:14:32 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[93603]: 18: /lib64/libpthread.so.0(+0x81ca) [0x7f5bf25a51ca] 2024-03-28T18:14:32.788 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:14:32 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[93603]: 19: clone() 2024-03-28T18:14:32.788 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:14:32 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[93603]: 2024-03-28T18:14:32.493+0000 7f5be8000700 -1 *** Caught signal (Aborted) ** 2024-03-28T18:14:32.788 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:14:32 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[93603]: in thread 7f5be8000700 thread_name:ms_dispatch 2024-03-28T18:14:32.788 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:14:32 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[93603]: 2024-03-28T18:14:32.788 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:14:32 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[93603]: ceph version 19.0.0-2145-g3be9a5c9 (3be9a5c9bc793e11e2800a8c0c696e8b46742033) squid (dev) 2024-03-28T18:14:32.788 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:14:32 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[93603]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7f5bf25afd20] 2024-03-28T18:14:32.788 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:14:32 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[93603]: 2: gsignal() 2024-03-28T18:14:32.789 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:14:32 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[93603]: 3: abort() 2024-03-28T18:14:32.789 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:14:32 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[93603]: 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x18f) [0x7f5bf38284fb] 2024-03-28T18:14:32.789 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:14:32 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[93603]: 5: /usr/lib64/ceph/libceph-common.so.2(+0x2d2667) [0x7f5bf3828667] 2024-03-28T18:14:32.789 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:14:32 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[93603]: 6: (Locker::set_xlocks_done(MutationImpl*, bool)+0x493) [0x561ef716b223] 2024-03-28T18:14:32.789 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:14:32 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[93603]: 7: (Server::early_reply(boost::intrusive_ptr const&, CInode*, CDentry*)+0x3e6) [0x561ef6fe55d6] 2024-03-28T18:14:32.789 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:14:32 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[93603]: 8: (Server::journal_and_reply(boost::intrusive_ptr const&, CInode*, CDentry*, LogEvent*, MDSLogContextBase*)+0xc1) [0x561ef6fe5da1] 2024-03-28T18:14:32.789 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:14:32 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[93603]: 9: (Server::handle_client_mkdir(boost::intrusive_ptr const&)+0x5ba) [0x561ef701b6aa] 2024-03-28T18:14:32.789 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:14:32 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[93603]: 10: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0x733) [0x561ef7038233] 2024-03-28T18:14:32.789 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:14:32 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[93603]: 11: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x561ef703a197] 2024-03-28T18:14:32.789 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:14:32 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[93603]: 12: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x561ef6f3911b] 2024-03-28T18:14:32.789 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:14:32 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[93603]: 13: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x561ef6f396bc] 2024-03-28T18:14:32.789 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:14:32 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[93603]: 14: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bf) [0x561ef6f231ff] 2024-03-28T18:14:32.790 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:14:32 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[93603]: 15: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x478) [0x7f5bf3aba488] 2024-03-28T18:14:32.790 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:14:32 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[93603]: 16: (DispatchQueue::entry()+0x51f) [0x7f5bf3ab760f] 2024-03-28T18:14:32.790 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:14:32 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[93603]: 17: (DispatchQueue::DispatchThread::entry()+0x11) [0x7f5bf3b8ae81] 2024-03-28T18:14:32.790 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:14:32 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[93603]: 18: /lib64/libpthread.so.0(+0x81ca) [0x7f5bf25a51ca] 2024-03-28T18:14:32.790 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:14:32 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[93603]: 19: clone() 2024-03-28T18:14:32.790 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:14:32 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[93603]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-03-28T18:14:32.790 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:14:32 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[93603]: 2024-03-28T18:14:32.790 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:14:32 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[93603]: -1> 2024-03-28T18:14:32.490+0000 7f5be8000700 -1 /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-2145-g3be9a5c9/rpm/el8/BUILD/ceph-19.0.0-2145-g3be9a5c9/src/mds/Locker.cc: In function 'void Locker::set_xlocks_done(MutationImpl*, bool)' thread 7f5be8000700 time 2024-03-28T18:14:32.488879+0000 2024-03-28T18:14:32.790 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:14:32 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[93603]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-2145-g3be9a5c9/rpm/el8/BUILD/ceph-19.0.0-2145-g3be9a5c9/src/mds/Locker.cc: 626: FAILED ceph_assert(obj->is_auth()) 2024-03-28T18:14:32.790 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:14:32 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[93603]: 2024-03-28T18:14:32.790 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:14:32 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[93603]: ceph version 19.0.0-2145-g3be9a5c9 (3be9a5c9bc793e11e2800a8c0c696e8b46742033) squid (dev) 2024-03-28T18:14:32.790 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:14:32 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[93603]: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x135) [0x7f5bf38284a1] 2024-03-28T18:14:32.790 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:14:32 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[93603]: 2: /usr/lib64/ceph/libceph-common.so.2(+0x2d2667) [0x7f5bf3828667] 2024-03-28T18:14:32.791 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:14:32 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[93603]: 3: (Locker::set_xlocks_done(MutationImpl*, bool)+0x493) [0x561ef716b223] 2024-03-28T18:14:32.791 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:14:32 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[93603]: 4: (Server::early_reply(boost::intrusive_ptr const&, CInode*, CDentry*)+0x3e6) [0x561ef6fe55d6] 2024-03-28T18:14:32.791 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:14:32 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[93603]: 5: (Server::journal_and_reply(boost::intrusive_ptr const&, CInode*, CDentry*, LogEvent*, MDSLogContextBase*)+0xc1) [0x561ef6fe5da1] 2024-03-28T18:14:32.791 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:14:32 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[93603]: 6: (Server::handle_client_mkdir(boost::intrusive_ptr const&)+0x5ba) [0x561ef701b6aa] 2024-03-28T18:14:32.791 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:14:32 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[93603]: 7: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0x733) [0x561ef7038233] 2024-03-28T18:14:32.791 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:14:32 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[93603]: 8: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x561ef703a197] 2024-03-28T18:14:32.791 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:14:32 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[93603]: 9: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x561ef6f3911b] 2024-03-28T18:14:32.791 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:14:32 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[93603]: 10: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x561ef6f396bc] 2024-03-28T18:14:32.791 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:14:32 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[93603]: 11: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bf) [0x561ef6f231ff] 2024-03-28T18:14:32.791 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:14:32 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[93603]: 12: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x478) [0x7f5bf3aba488] 2024-03-28T18:14:32.791 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:14:32 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[93603]: 13: (DispatchQueue::entry()+0x51f) [0x7f5bf3ab760f] 2024-03-28T18:14:32.791 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:14:32 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[93603]: 14: (DispatchQueue::DispatchThread::entry()+0x11) [0x7f5bf3b8ae81] 2024-03-28T18:14:32.792 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:14:32 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[93603]: 15: /lib64/libpthread.so.0(+0x81ca) [0x7f5bf25a51ca] 2024-03-28T18:14:32.792 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:14:32 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[93603]: 16: clone() 2024-03-28T18:14:32.792 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:14:32 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[93603]: 2024-03-28T18:14:32.792 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:14:32 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[93603]: 0> 2024-03-28T18:14:32.493+0000 7f5be8000700 -1 *** Caught signal (Aborted) ** 2024-03-28T18:14:32.792 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:14:32 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[93603]: in thread 7f5be8000700 thread_name:ms_dispatch 2024-03-28T18:14:32.792 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:14:32 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[93603]: 2024-03-28T18:14:32.792 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:14:32 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[93603]: ceph version 19.0.0-2145-g3be9a5c9 (3be9a5c9bc793e11e2800a8c0c696e8b46742033) squid (dev) 2024-03-28T18:14:32.792 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:14:32 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[93603]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7f5bf25afd20] 2024-03-28T18:14:32.792 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:14:32 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[93603]: 2: gsignal() 2024-03-28T18:14:32.792 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:14:32 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[93603]: 3: abort() 2024-03-28T18:14:32.792 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:14:32 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[93603]: 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x18f) [0x7f5bf38284fb] 2024-03-28T18:14:32.792 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:14:32 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[93603]: 5: /usr/lib64/ceph/libceph-common.so.2(+0x2d2667) [0x7f5bf3828667] 2024-03-28T18:14:32.792 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:14:32 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[93603]: 6: (Locker::set_xlocks_done(MutationImpl*, bool)+0x493) [0x561ef716b223] 2024-03-28T18:14:32.792 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:14:32 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[93603]: 7: (Server::early_reply(boost::intrusive_ptr const&, CInode*, CDentry*)+0x3e6) [0x561ef6fe55d6] 2024-03-28T18:14:32.793 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:14:32 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[93603]: 8: (Server::journal_and_reply(boost::intrusive_ptr const&, CInode*, CDentry*, LogEvent*, MDSLogContextBase*)+0xc1) [0x561ef6fe5da1] 2024-03-28T18:14:32.793 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:14:32 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[93603]: 9: (Server::handle_client_mkdir(boost::intrusive_ptr const&)+0x5ba) [0x561ef701b6aa] 2024-03-28T18:14:32.793 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:14:32 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[93603]: 10: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0x733) [0x561ef7038233] 2024-03-28T18:14:32.793 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:14:32 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[93603]: 11: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x561ef703a197] 2024-03-28T18:14:32.793 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:14:32 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[93603]: 12: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x561ef6f3911b] 2024-03-28T18:14:32.793 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:14:32 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[93603]: 13: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x561ef6f396bc] 2024-03-28T18:14:32.793 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:14:32 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[93603]: 14: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bf) [0x561ef6f231ff] 2024-03-28T18:14:32.793 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:14:32 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[93603]: 15: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x478) [0x7f5bf3aba488] 2024-03-28T18:14:32.793 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:14:32 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[93603]: 16: (DispatchQueue::entry()+0x51f) [0x7f5bf3ab760f] 2024-03-28T18:14:32.793 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:14:32 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[93603]: 17: (DispatchQueue::DispatchThread::entry()+0x11) [0x7f5bf3b8ae81] 2024-03-28T18:14:32.793 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:14:32 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[93603]: 18: /lib64/libpthread.so.0(+0x81ca) [0x7f5bf25a51ca] 2024-03-28T18:14:32.793 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:14:32 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[93603]: 19: clone() 2024-03-28T18:14:32.794 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:14:32 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[93603]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-03-28T18:14:32.794 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:14:32 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[93603]: 2024-03-28T18:14:32.794 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:14:32 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[93603]: -1> 2024-03-28T18:14:32.490+0000 7f5be8000700 -1 /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-2145-g3be9a5c9/rpm/el8/BUILD/ceph-19.0.0-2145-g3be9a5c9/src/mds/Locker.cc: In function 'void Locker::set_xlocks_done(MutationImpl*, bool)' thread 7f5be8000700 time 2024-03-28T18:14:32.488879+0000 2024-03-28T18:14:32.794 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:14:32 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[93603]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-2145-g3be9a5c9/rpm/el8/BUILD/ceph-19.0.0-2145-g3be9a5c9/src/mds/Locker.cc: 626: FAILED ceph_assert(obj->is_auth()) 2024-03-28T18:14:32.794 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:14:32 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[93603]: 2024-03-28T18:14:32.794 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:14:32 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[93603]: ceph version 19.0.0-2145-g3be9a5c9 (3be9a5c9bc793e11e2800a8c0c696e8b46742033) squid (dev) 2024-03-28T18:14:32.794 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:14:32 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[93603]: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x135) [0x7f5bf38284a1] 2024-03-28T18:14:32.794 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:14:32 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[93603]: 2: /usr/lib64/ceph/libceph-common.so.2(+0x2d2667) [0x7f5bf3828667] 2024-03-28T18:14:32.794 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:14:32 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[93603]: 3: (Locker::set_xlocks_done(MutationImpl*, bool)+0x493) [0x561ef716b223] 2024-03-28T18:14:32.794 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:14:32 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[93603]: 4: (Server::early_reply(boost::intrusive_ptr const&, CInode*, CDentry*)+0x3e6) [0x561ef6fe55d6] 2024-03-28T18:14:32.794 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:14:32 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[93603]: 5: (Server::journal_and_reply(boost::intrusive_ptr const&, CInode*, CDentry*, LogEvent*, MDSLogContextBase*)+0xc1) [0x561ef6fe5da1] 2024-03-28T18:14:32.795 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:14:32 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[93603]: 6: (Server::handle_client_mkdir(boost::intrusive_ptr const&)+0x5ba) [0x561ef701b6aa] 2024-03-28T18:14:32.795 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:14:32 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[93603]: 7: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0x733) [0x561ef7038233] 2024-03-28T18:14:32.795 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:14:32 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[93603]: 8: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x561ef703a197] 2024-03-28T18:14:32.795 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:14:32 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[93603]: 9: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x561ef6f3911b] 2024-03-28T18:14:32.795 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:14:32 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[93603]: 10: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x561ef6f396bc] 2024-03-28T18:14:32.795 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:14:32 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[93603]: 11: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bf) [0x561ef6f231ff] 2024-03-28T18:14:32.795 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:14:32 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[93603]: 12: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x478) [0x7f5bf3aba488] 2024-03-28T18:14:32.795 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:14:32 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[93603]: 13: (DispatchQueue::entry()+0x51f) [0x7f5bf3ab760f] 2024-03-28T18:14:32.795 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:14:32 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[93603]: 14: (DispatchQueue::DispatchThread::entry()+0x11) [0x7f5bf3b8ae81] 2024-03-28T18:14:32.795 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:14:32 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[93603]: 15: /lib64/libpthread.so.0(+0x81ca) [0x7f5bf25a51ca] 2024-03-28T18:14:32.796 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:14:32 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[93603]: 16: clone() 2024-03-28T18:14:32.796 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:14:32 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[93603]: 2024-03-28T18:14:32.796 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:14:32 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[93603]: 0> 2024-03-28T18:14:32.493+0000 7f5be8000700 -1 *** Caught signal (Aborted) ** 2024-03-28T18:14:32.796 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:14:32 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[93603]: in thread 7f5be8000700 thread_name:ms_dispatch 2024-03-28T18:14:32.796 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:14:32 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[93603]: 2024-03-28T18:14:32.796 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:14:32 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[93603]: ceph version 19.0.0-2145-g3be9a5c9 (3be9a5c9bc793e11e2800a8c0c696e8b46742033) squid (dev) 2024-03-28T18:14:32.796 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:14:32 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[93603]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7f5bf25afd20] 2024-03-28T18:14:32.796 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:14:32 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[93603]: 2: gsignal() 2024-03-28T18:14:32.796 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:14:32 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[93603]: 3: abort() 2024-03-28T18:14:32.796 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:14:32 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[93603]: 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x18f) [0x7f5bf38284fb] 2024-03-28T18:14:32.796 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:14:32 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[93603]: 5: /usr/lib64/ceph/libceph-common.so.2(+0x2d2667) [0x7f5bf3828667] 2024-03-28T18:14:32.796 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:14:32 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[93603]: 6: (Locker::set_xlocks_done(MutationImpl*, bool)+0x493) [0x561ef716b223] 2024-03-28T18:14:32.796 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:14:32 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[93603]: 7: (Server::early_reply(boost::intrusive_ptr const&, CInode*, CDentry*)+0x3e6) [0x561ef6fe55d6] 2024-03-28T18:14:32.796 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:14:32 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[93603]: 8: (Server::journal_and_reply(boost::intrusive_ptr const&, CInode*, CDentry*, LogEvent*, MDSLogContextBase*)+0xc1) [0x561ef6fe5da1] 2024-03-28T18:14:32.796 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:14:32 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[93603]: 9: (Server::handle_client_mkdir(boost::intrusive_ptr const&)+0x5ba) [0x561ef701b6aa] 2024-03-28T18:14:32.797 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:14:32 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[93603]: 10: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0x733) [0x561ef7038233] 2024-03-28T18:14:32.797 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:14:32 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[93603]: 11: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x561ef703a197] 2024-03-28T18:14:32.797 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:14:32 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[93603]: 12: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x561ef6f3911b] 2024-03-28T18:14:32.797 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:14:32 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[93603]: 13: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x561ef6f396bc] 2024-03-28T18:14:32.797 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:14:32 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[93603]: 14: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bf) [0x561ef6f231ff] 2024-03-28T18:14:32.797 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:14:32 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[93603]: 15: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x478) [0x7f5bf3aba488] 2024-03-28T18:14:32.797 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:14:32 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[93603]: 16: (DispatchQueue::entry()+0x51f) [0x7f5bf3ab760f] 2024-03-28T18:14:32.797 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:14:32 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[93603]: 17: (DispatchQueue::DispatchThread::entry()+0x11) [0x7f5bf3b8ae81] 2024-03-28T18:14:32.797 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:14:32 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[93603]: 18: /lib64/libpthread.so.0(+0x81ca) [0x7f5bf25a51ca] 2024-03-28T18:14:32.797 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:14:32 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[93603]: 19: clone() 2024-03-28T18:14:32.797 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:14:32 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[93603]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-03-28T18:14:32.797 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:14:32 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[93603]: 2024-03-28T18:14:32.797 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:14:32 smithi022 podman[93863]: 2024-03-28 18:14:32.622751753 +0000 UTC m=+0.039482655 container died ad8b1f516a881f2fb3d6cc6bc82e170b606fae9a1e44359bbfd6d3cdb8d5e15b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240319, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=) 2024-03-28T18:14:33.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:14:33 smithi079 ceph-mon[35312]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-03-28T18:14:33.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:14:33 smithi079 ceph-mon[35312]: Cluster is now healthy 2024-03-28T18:14:33.683 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:14:33 smithi079 ceph-mon[35312]: mds.? [v2:172.21.15.22:6834/803375445,v1:172.21.15.22:6835/803375445] up:active 2024-03-28T18:14:33.683 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:14:33 smithi079 ceph-mon[35312]: fsmap cephfs:5 4 up:standby-replay 1 up:standby 5 up:active 2024-03-28T18:14:33.683 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:14:33 smithi079 ceph-mon[35312]: fsmap cephfs:5 5 up:standby-replay 5 up:active 2024-03-28T18:14:33.782 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:14:33 smithi022 podman[93863]: 2024-03-28 18:14:33.493682508 +0000 UTC m=+0.910413442 container remove ad8b1f516a881f2fb3d6cc6bc82e170b606fae9a1e44359bbfd6d3cdb8d5e15b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_CLEAN=True, ceph=True, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.build-date=20240319, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-28T18:14:33.782 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:14:33 smithi022 systemd[1]: ceph-fd220748-ed2b-11ee-95d0-87774f69a715@mds.j.service: Main process exited, code=exited, status=134/n/a 2024-03-28T18:14:33.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:14:33 smithi022 ceph-mon[31985]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-03-28T18:14:33.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:14:33 smithi022 ceph-mon[31985]: Cluster is now healthy 2024-03-28T18:14:33.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:14:33 smithi022 ceph-mon[31985]: mds.? [v2:172.21.15.22:6834/803375445,v1:172.21.15.22:6835/803375445] up:active 2024-03-28T18:14:33.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:14:33 smithi022 ceph-mon[31985]: fsmap cephfs:5 4 up:standby-replay 1 up:standby 5 up:active 2024-03-28T18:14:33.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:14:33 smithi022 ceph-mon[31985]: fsmap cephfs:5 5 up:standby-replay 5 up:active 2024-03-28T18:14:33.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:14:33 smithi134 ceph-mon[35037]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-03-28T18:14:33.925 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:14:33 smithi134 ceph-mon[35037]: Cluster is now healthy 2024-03-28T18:14:33.925 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:14:33 smithi134 ceph-mon[35037]: mds.? [v2:172.21.15.22:6834/803375445,v1:172.21.15.22:6835/803375445] up:active 2024-03-28T18:14:33.925 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:14:33 smithi134 ceph-mon[35037]: fsmap cephfs:5 4 up:standby-replay 1 up:standby 5 up:active 2024-03-28T18:14:33.925 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:14:33 smithi134 ceph-mon[35037]: fsmap cephfs:5 5 up:standby-replay 5 up:active 2024-03-28T18:14:34.432 INFO:journalctl@ceph.mds.b.smithi079.stdout:Mar 28 18:14:34 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-b[65245]: 2024-03-28T18:14:34.095+0000 7f845f400700 -1 mds.pinger is_rank_lagging: rank=1 was never sent ping request. 2024-03-28T18:14:34.448 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:14:34 smithi022 systemd[1]: ceph-fd220748-ed2b-11ee-95d0-87774f69a715@mds.j.service: Failed with result 'exit-code'. 2024-03-28T18:14:34.449 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:14:34 smithi022 systemd[1]: ceph-fd220748-ed2b-11ee-95d0-87774f69a715@mds.j.service: Consumed 2.039s CPU time. 2024-03-28T18:14:34.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:14:34 smithi022 ceph-mon[31985]: pgmap v228: 97 pgs: 97 active+clean; 117 MiB data, 775 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 102 B/s wr, 9 op/s 2024-03-28T18:14:34.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:14:34 smithi134 ceph-mon[35037]: pgmap v228: 97 pgs: 97 active+clean; 117 MiB data, 775 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 102 B/s wr, 9 op/s 2024-03-28T18:14:34.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:14:34 smithi079 ceph-mon[35312]: pgmap v228: 97 pgs: 97 active+clean; 117 MiB data, 775 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 102 B/s wr, 9 op/s 2024-03-28T18:14:35.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:14:35 smithi022 ceph-mon[31985]: osdmap e109: 12 total, 12 up, 12 in 2024-03-28T18:14:35.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:14:35 smithi134 ceph-mon[35037]: osdmap e109: 12 total, 12 up, 12 in 2024-03-28T18:14:35.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:14:35 smithi079 ceph-mon[35312]: osdmap e109: 12 total, 12 up, 12 in 2024-03-28T18:14:36.781 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:14:36 smithi022 ceph-mon[31985]: pgmap v230: 97 pgs: 97 active+clean; 117 MiB data, 775 MiB used, 1.0 TiB / 1.0 TiB avail; 138 KiB/s rd, 39 KiB/s wr, 35 op/s 2024-03-28T18:14:36.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:14:36 smithi134 ceph-mon[35037]: pgmap v230: 97 pgs: 97 active+clean; 117 MiB data, 775 MiB used, 1.0 TiB / 1.0 TiB avail; 138 KiB/s rd, 39 KiB/s wr, 35 op/s 2024-03-28T18:14:36.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:14:36 smithi079 ceph-mon[35312]: pgmap v230: 97 pgs: 97 active+clean; 117 MiB data, 775 MiB used, 1.0 TiB / 1.0 TiB avail; 138 KiB/s rd, 39 KiB/s wr, 35 op/s 2024-03-28T18:14:38.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:14:38 smithi022 ceph-mon[31985]: pgmap v231: 97 pgs: 97 active+clean; 117 MiB data, 775 MiB used, 1.0 TiB / 1.0 TiB avail; 126 KiB/s rd, 36 KiB/s wr, 32 op/s 2024-03-28T18:14:38.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:14:38 smithi134 ceph-mon[35037]: pgmap v231: 97 pgs: 97 active+clean; 117 MiB data, 775 MiB used, 1.0 TiB / 1.0 TiB avail; 126 KiB/s rd, 36 KiB/s wr, 32 op/s 2024-03-28T18:14:38.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:14:38 smithi079 ceph-mon[35312]: pgmap v231: 97 pgs: 97 active+clean; 117 MiB data, 775 MiB used, 1.0 TiB / 1.0 TiB avail; 126 KiB/s rd, 36 KiB/s wr, 32 op/s 2024-03-28T18:14:40.734 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:14:40 smithi079 ceph-mon[35312]: pgmap v232: 97 pgs: 97 active+clean; 117 MiB data, 775 MiB used, 1.0 TiB / 1.0 TiB avail; 111 KiB/s rd, 32 KiB/s wr, 29 op/s 2024-03-28T18:14:40.781 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:14:40 smithi022 ceph-mon[31985]: pgmap v232: 97 pgs: 97 active+clean; 117 MiB data, 775 MiB used, 1.0 TiB / 1.0 TiB avail; 111 KiB/s rd, 32 KiB/s wr, 29 op/s 2024-03-28T18:14:40.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:14:40 smithi134 ceph-mon[35037]: pgmap v232: 97 pgs: 97 active+clean; 117 MiB data, 775 MiB used, 1.0 TiB / 1.0 TiB avail; 111 KiB/s rd, 32 KiB/s wr, 29 op/s 2024-03-28T18:14:42.781 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:14:42 smithi022 ceph-mon[31985]: pgmap v233: 97 pgs: 97 active+clean; 117 MiB data, 775 MiB used, 1.0 TiB / 1.0 TiB avail; 111 KiB/s rd, 32 KiB/s wr, 29 op/s 2024-03-28T18:14:42.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:14:42 smithi134 ceph-mon[35037]: pgmap v233: 97 pgs: 97 active+clean; 117 MiB data, 775 MiB used, 1.0 TiB / 1.0 TiB avail; 111 KiB/s rd, 32 KiB/s wr, 29 op/s 2024-03-28T18:14:42.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:14:42 smithi079 ceph-mon[35312]: pgmap v233: 97 pgs: 97 active+clean; 117 MiB data, 775 MiB used, 1.0 TiB / 1.0 TiB avail; 111 KiB/s rd, 32 KiB/s wr, 29 op/s 2024-03-28T18:14:44.282 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:14:43 smithi022 ceph-mon[31985]: osdmap e110: 12 total, 12 up, 12 in 2024-03-28T18:14:44.424 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:14:43 smithi134 ceph-mon[35037]: osdmap e110: 12 total, 12 up, 12 in 2024-03-28T18:14:44.432 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:14:43 smithi079 ceph-mon[35312]: osdmap e110: 12 total, 12 up, 12 in 2024-03-28T18:14:44.689 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:14:44 smithi022 systemd[1]: ceph-fd220748-ed2b-11ee-95d0-87774f69a715@mds.j.service: Scheduled restart job, restart counter is at 2. 2024-03-28T18:14:44.689 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:14:44 smithi022 systemd[1]: Stopped Ceph mds.j for fd220748-ed2b-11ee-95d0-87774f69a715. 2024-03-28T18:14:44.689 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:14:44 smithi022 systemd[1]: ceph-fd220748-ed2b-11ee-95d0-87774f69a715@mds.j.service: Consumed 2.039s CPU time. 2024-03-28T18:14:44.689 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:14:44 smithi022 systemd[1]: Starting Ceph mds.j for fd220748-ed2b-11ee-95d0-87774f69a715... 2024-03-28T18:14:44.944 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:14:44 smithi022 podman[93955]: 2024-03-28 18:14:44.594754375 +0000 UTC m=+0.035340588 image pull c664afe56bb01db4f9afa9532c20598e201e664f2e9ba2be50db0d52674a26d2 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc 2024-03-28T18:14:44.944 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:14:44 smithi022 podman[93955]: 2024-03-28 18:14:44.841052693 +0000 UTC m=+0.281638926 container create f713b193cb5c3ff019609676b1f8f7ee10ae6b0c45e402a6ab968463799c5836 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.build-date=20240319, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, ceph=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.license=GPLv2) 2024-03-28T18:14:45.221 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:14:44 smithi022 ceph-mon[31985]: pgmap v235: 97 pgs: 97 active+clean; 117 MiB data, 775 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 117 B/s wr, 5 op/s 2024-03-28T18:14:45.222 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:14:45 smithi022 podman[93955]: 2024-03-28 18:14:45.162189389 +0000 UTC m=+0.602775745 container init f713b193cb5c3ff019609676b1f8f7ee10ae6b0c45e402a6ab968463799c5836 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.license=GPLv2, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.build-date=20240319, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD) 2024-03-28T18:14:45.222 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:14:45 smithi022 podman[93955]: 2024-03-28 18:14:45.174822977 +0000 UTC m=+0.615409194 container start f713b193cb5c3ff019609676b1f8f7ee10ae6b0c45e402a6ab968463799c5836 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j, org.label-schema.build-date=20240319, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, ceph=True, GIT_BRANCH=HEAD, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2) 2024-03-28T18:14:45.424 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:14:44 smithi134 ceph-mon[35037]: pgmap v235: 97 pgs: 97 active+clean; 117 MiB data, 775 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 117 B/s wr, 5 op/s 2024-03-28T18:14:45.432 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:14:44 smithi079 ceph-mon[35312]: pgmap v235: 97 pgs: 97 active+clean; 117 MiB data, 775 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 117 B/s wr, 5 op/s 2024-03-28T18:14:45.532 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:14:45 smithi022 ceph-mds[93973]: -- 172.21.15.22:0/413510052 <== mon.2 v2:172.21.15.79:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55b8226a1a40 con 0x55b822674800 2024-03-28T18:14:45.532 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:14:45 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[93969]: starting mds.j at 2024-03-28T18:14:45.532 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:14:45 smithi022 bash[93955]: f713b193cb5c3ff019609676b1f8f7ee10ae6b0c45e402a6ab968463799c5836 2024-03-28T18:14:45.532 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:14:45 smithi022 systemd[1]: Started Ceph mds.j for fd220748-ed2b-11ee-95d0-87774f69a715. 2024-03-28T18:14:46.282 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:14:45 smithi022 ceph-mon[31985]: daemon mds.j restarted 2024-03-28T18:14:46.282 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:14:45 smithi022 ceph-mon[31985]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-03-28T18:14:46.283 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:14:45 smithi022 ceph-mon[31985]: osdmap e111: 12 total, 12 up, 12 in 2024-03-28T18:14:46.283 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:14:45 smithi022 ceph-mon[31985]: mds.? [v2:172.21.15.22:6834/535745119,v1:172.21.15.22:6835/535745119] up:boot 2024-03-28T18:14:46.283 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:14:45 smithi022 ceph-mon[31985]: Standby daemon mds.d assigned to filesystem cephfs as rank 1 2024-03-28T18:14:46.283 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:14:45 smithi022 ceph-mon[31985]: fsmap cephfs:4/5 {0=b=up:active,2=e=up:active,3=k=up:active,4=i=up:active} 5 up:standby-replay 1 up:standby, 1 failed 2024-03-28T18:14:46.283 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:14:45 smithi022 ceph-mon[31985]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-03-28T18:14:46.283 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:14:45 smithi022 ceph-mon[31985]: fsmap cephfs:5/5 4 up:standby-replay 1 up:standby 1 up:replay 4 up:active 2024-03-28T18:14:46.425 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:14:45 smithi134 ceph-mon[35037]: daemon mds.j restarted 2024-03-28T18:14:46.425 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:14:45 smithi134 ceph-mon[35037]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-03-28T18:14:46.425 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:14:45 smithi134 ceph-mon[35037]: osdmap e111: 12 total, 12 up, 12 in 2024-03-28T18:14:46.425 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:14:45 smithi134 ceph-mon[35037]: mds.? [v2:172.21.15.22:6834/535745119,v1:172.21.15.22:6835/535745119] up:boot 2024-03-28T18:14:46.425 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:14:45 smithi134 ceph-mon[35037]: Standby daemon mds.d assigned to filesystem cephfs as rank 1 2024-03-28T18:14:46.425 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:14:45 smithi134 ceph-mon[35037]: fsmap cephfs:4/5 {0=b=up:active,2=e=up:active,3=k=up:active,4=i=up:active} 5 up:standby-replay 1 up:standby, 1 failed 2024-03-28T18:14:46.425 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:14:45 smithi134 ceph-mon[35037]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-03-28T18:14:46.425 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:14:45 smithi134 ceph-mon[35037]: fsmap cephfs:5/5 4 up:standby-replay 1 up:standby 1 up:replay 4 up:active 2024-03-28T18:14:46.432 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:14:45 smithi079 ceph-mon[35312]: daemon mds.j restarted 2024-03-28T18:14:46.432 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:14:45 smithi079 ceph-mon[35312]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-03-28T18:14:46.433 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:14:45 smithi079 ceph-mon[35312]: osdmap e111: 12 total, 12 up, 12 in 2024-03-28T18:14:46.433 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:14:45 smithi079 ceph-mon[35312]: mds.? [v2:172.21.15.22:6834/535745119,v1:172.21.15.22:6835/535745119] up:boot 2024-03-28T18:14:46.433 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:14:45 smithi079 ceph-mon[35312]: Standby daemon mds.d assigned to filesystem cephfs as rank 1 2024-03-28T18:14:46.433 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:14:45 smithi079 ceph-mon[35312]: fsmap cephfs:4/5 {0=b=up:active,2=e=up:active,3=k=up:active,4=i=up:active} 5 up:standby-replay 1 up:standby, 1 failed 2024-03-28T18:14:46.433 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:14:45 smithi079 ceph-mon[35312]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-03-28T18:14:46.433 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:14:45 smithi079 ceph-mon[35312]: fsmap cephfs:5/5 4 up:standby-replay 1 up:standby 1 up:replay 4 up:active 2024-03-28T18:14:47.282 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:14:46 smithi022 ceph-mon[31985]: pgmap v236: 97 pgs: 97 active+clean; 117 MiB data, 793 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2024-03-28T18:14:47.282 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:14:46 smithi022 ceph-mon[31985]: mds.? [v2:172.21.15.22:6836/2984059496,v1:172.21.15.22:6837/2984059496] up:resolve 2024-03-28T18:14:47.282 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:14:46 smithi022 ceph-mon[31985]: fsmap cephfs:5/5 4 up:standby-replay 1 up:standby 1 up:resolve 4 up:active 2024-03-28T18:14:47.424 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:14:46 smithi134 ceph-mon[35037]: pgmap v236: 97 pgs: 97 active+clean; 117 MiB data, 793 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2024-03-28T18:14:47.424 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:14:46 smithi134 ceph-mon[35037]: mds.? [v2:172.21.15.22:6836/2984059496,v1:172.21.15.22:6837/2984059496] up:resolve 2024-03-28T18:14:47.425 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:14:46 smithi134 ceph-mon[35037]: fsmap cephfs:5/5 4 up:standby-replay 1 up:standby 1 up:resolve 4 up:active 2024-03-28T18:14:47.433 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:14:46 smithi079 ceph-mon[35312]: pgmap v236: 97 pgs: 97 active+clean; 117 MiB data, 793 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2024-03-28T18:14:47.433 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:14:46 smithi079 ceph-mon[35312]: mds.? [v2:172.21.15.22:6836/2984059496,v1:172.21.15.22:6837/2984059496] up:resolve 2024-03-28T18:14:47.434 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:14:46 smithi079 ceph-mon[35312]: fsmap cephfs:5/5 4 up:standby-replay 1 up:standby 1 up:resolve 4 up:active 2024-03-28T18:14:48.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:14:48 smithi022 ceph-mon[31985]: pgmap v238: 97 pgs: 97 active+clean; 117 MiB data, 793 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 127 B/s wr, 6 op/s 2024-03-28T18:14:48.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:14:48 smithi022 ceph-mon[31985]: reconnect by client.35222 172.21.15.22:0/3729116781 after 0.00399995 2024-03-28T18:14:48.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:14:48 smithi022 ceph-mon[31985]: mds.? [v2:172.21.15.22:6836/2984059496,v1:172.21.15.22:6837/2984059496] up:reconnect 2024-03-28T18:14:48.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:14:48 smithi022 ceph-mon[31985]: fsmap cephfs:5/5 4 up:standby-replay 1 up:standby 1 up:reconnect 4 up:active 2024-03-28T18:14:48.674 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:14:48 smithi134 ceph-mon[35037]: pgmap v238: 97 pgs: 97 active+clean; 117 MiB data, 793 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 127 B/s wr, 6 op/s 2024-03-28T18:14:48.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:14:48 smithi134 ceph-mon[35037]: reconnect by client.35222 172.21.15.22:0/3729116781 after 0.00399995 2024-03-28T18:14:48.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:14:48 smithi134 ceph-mon[35037]: mds.? [v2:172.21.15.22:6836/2984059496,v1:172.21.15.22:6837/2984059496] up:reconnect 2024-03-28T18:14:48.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:14:48 smithi134 ceph-mon[35037]: fsmap cephfs:5/5 4 up:standby-replay 1 up:standby 1 up:reconnect 4 up:active 2024-03-28T18:14:48.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:14:48 smithi079 ceph-mon[35312]: pgmap v238: 97 pgs: 97 active+clean; 117 MiB data, 793 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 127 B/s wr, 6 op/s 2024-03-28T18:14:48.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:14:48 smithi079 ceph-mon[35312]: reconnect by client.35222 172.21.15.22:0/3729116781 after 0.00399995 2024-03-28T18:14:48.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:14:48 smithi079 ceph-mon[35312]: mds.? [v2:172.21.15.22:6836/2984059496,v1:172.21.15.22:6837/2984059496] up:reconnect 2024-03-28T18:14:48.683 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:14:48 smithi079 ceph-mon[35312]: fsmap cephfs:5/5 4 up:standby-replay 1 up:standby 1 up:reconnect 4 up:active 2024-03-28T18:14:48.954 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-28T18:14:48.954 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-28T18:14:49.198 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:14:49.198+0000 7ff045a00640 1 -- 172.21.15.22:0/366454605 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff040109850 msgr2=0x7ff040109c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:14:49.198 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:14:49.198+0000 7ff045a00640 1 --2- 172.21.15.22:0/366454605 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff040109850 0x7ff040109c30 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7ff030009930 tx=0x7ff03002bc20 comp rx=0 tx=0).stop 2024-03-28T18:14:49.198 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:14:49.198+0000 7ff045a00640 1 -- 172.21.15.22:0/366454605 shutdown_connections 2024-03-28T18:14:49.198 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:14:49.198+0000 7ff045a00640 1 --2- 172.21.15.22:0/366454605 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ff040104200 0x7ff040110710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:14:49.198 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:14:49.198+0000 7ff045a00640 1 --2- 172.21.15.22:0/366454605 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff040103840 0x7ff040103cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:14:49.198 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:14:49.198+0000 7ff045a00640 1 --2- 172.21.15.22:0/366454605 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff040109850 0x7ff040109c30 unknown :-1 s=CLOSED pgs=742 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:14:49.198 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:14:49.198+0000 7ff045a00640 1 -- 172.21.15.22:0/366454605 >> 172.21.15.22:0/366454605 conn(0x7ff0400ff2f0 msgr2=0x7ff040101710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:14:49.199 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:14:49.198+0000 7ff045a00640 1 -- 172.21.15.22:0/366454605 shutdown_connections 2024-03-28T18:14:49.199 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:14:49.198+0000 7ff045a00640 1 -- 172.21.15.22:0/366454605 wait complete. 2024-03-28T18:14:49.201 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:14:49.199+0000 7ff045a00640 1 Processor -- start 2024-03-28T18:14:49.201 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:14:49.199+0000 7ff045a00640 1 -- start start 2024-03-28T18:14:49.201 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:14:49.199+0000 7ff045a00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff040103840 0x7ff04019a1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:14:49.201 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:14:49.200+0000 7ff045a00640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ff040104200 0x7ff04019a720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:14:49.201 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:14:49.200+0000 7ff045a00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff040109850 0x7ff04019eaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:14:49.201 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:14:49.200+0000 7ff045a00640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff040113c80 con 0x7ff040103840 2024-03-28T18:14:49.201 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:14:49.200+0000 7ff045a00640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff040113b00 con 0x7ff040104200 2024-03-28T18:14:49.201 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:14:49.200+0000 7ff045a00640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff040113e00 con 0x7ff040109850 2024-03-28T18:14:49.201 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:14:49.200+0000 7ff03ea00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff040103840 0x7ff04019a1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:14:49.202 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:14:49.200+0000 7ff03ea00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff040103840 0x7ff04019a1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:50424/0 (socket says 172.21.15.22:50424) 2024-03-28T18:14:49.202 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:14:49.200+0000 7ff03ea00640 1 -- 172.21.15.22:0/2272057994 learned_addr learned my addr 172.21.15.22:0/2272057994 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-28T18:14:49.202 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:14:49.200+0000 7ff03e000640 1 --2- 172.21.15.22:0/2272057994 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ff040104200 0x7ff04019a720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:14:49.202 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:14:49.200+0000 7ff03f400640 1 --2- 172.21.15.22:0/2272057994 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff040109850 0x7ff04019eaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:14:49.202 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:14:49.200+0000 7ff03ea00640 1 -- 172.21.15.22:0/2272057994 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff040109850 msgr2=0x7ff04019eaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:14:49.202 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:14:49.200+0000 7ff03ea00640 1 --2- 172.21.15.22:0/2272057994 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff040109850 0x7ff04019eaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:14:49.202 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:14:49.200+0000 7ff03ea00640 1 -- 172.21.15.22:0/2272057994 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ff040104200 msgr2=0x7ff04019a720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:14:49.202 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:14:49.200+0000 7ff03ea00640 1 --2- 172.21.15.22:0/2272057994 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ff040104200 0x7ff04019a720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:14:49.202 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:14:49.200+0000 7ff03ea00640 1 -- 172.21.15.22:0/2272057994 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff04019f1d0 con 0x7ff040103840 2024-03-28T18:14:49.202 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:14:49.201+0000 7ff03ea00640 1 --2- 172.21.15.22:0/2272057994 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff040103840 0x7ff04019a1e0 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7ff030000c00 tx=0x7ff0300045e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:14:49.202 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:14:49.201+0000 7ff037400640 1 -- 172.21.15.22:0/2272057994 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff030043020 con 0x7ff040103840 2024-03-28T18:14:49.202 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:14:49.201+0000 7ff037400640 1 -- 172.21.15.22:0/2272057994 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7ff03002ca90 con 0x7ff040103840 2024-03-28T18:14:49.202 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:14:49.201+0000 7ff045a00640 1 -- 172.21.15.22:0/2272057994 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff04019f460 con 0x7ff040103840 2024-03-28T18:14:49.202 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:14:49.201+0000 7ff037400640 1 -- 172.21.15.22:0/2272057994 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff030003ae0 con 0x7ff040103840 2024-03-28T18:14:49.203 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:14:49.201+0000 7ff045a00640 1 -- 172.21.15.22:0/2272057994 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff0401a6d50 con 0x7ff040103840 2024-03-28T18:14:49.205 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:14:49.205+0000 7ff037400640 1 -- 172.21.15.22:0/2272057994 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 26) v1 ==== 137421+0+0 (secure 0 0 0) 0x7ff030046020 con 0x7ff040103840 2024-03-28T18:14:49.205 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:14:49.205+0000 7ff04878b740 1 Processor -- start 2024-03-28T18:14:49.205 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:14:49.205+0000 7ff04878b740 1 -- start start 2024-03-28T18:14:49.207 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:14:49.206+0000 7ff04878b740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564e374c6c00 0x564e374c5950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:14:49.207 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:14:49.206+0000 7ff04878b740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x564e374c5f20 0x564e374c63a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:14:49.207 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:14:49.206+0000 7ff04878b740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x564e374cf290 0x564e374d1680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:14:49.207 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:14:49.206+0000 7ff04878b740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x564e3742a250 con 0x564e374cf290 2024-03-28T18:14:49.208 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:14:49.206+0000 7ff04878b740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x564e373d2a30 con 0x564e374c5f20 2024-03-28T18:14:49.208 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:14:49.206+0000 7ff04878b740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x564e374262f0 con 0x564e374c6c00 2024-03-28T18:14:49.208 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:14:49.206+0000 7ff03e000640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x564e374cf290 0x564e374d1680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:14:49.208 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:14:49.206+0000 7ff03e000640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x564e374cf290 0x564e374d1680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:50426/0 (socket says 172.21.15.22:50426) 2024-03-28T18:14:49.208 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:14:49.206+0000 7ff03e000640 1 -- 172.21.15.22:0/2342250406 learned_addr learned my addr 172.21.15.22:0/2342250406 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-28T18:14:49.208 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:14:49.206+0000 7ff03ea00640 1 --2- 172.21.15.22:0/2342250406 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x564e374c5f20 0x564e374c63a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:14:49.208 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:14:49.206+0000 7ff03f400640 1 --2- 172.21.15.22:0/2342250406 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564e374c6c00 0x564e374c5950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:14:49.208 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:14:49.206+0000 7ff03e000640 1 -- 172.21.15.22:0/2342250406 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564e374c6c00 msgr2=0x564e374c5950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T18:14:49.208 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:14:49.206+0000 7ff03e000640 1 --2- 172.21.15.22:0/2342250406 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564e374c6c00 0x564e374c5950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:14:49.208 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:14:49.206+0000 7ff03e000640 1 -- 172.21.15.22:0/2342250406 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x564e374c5f20 msgr2=0x564e374c63a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T18:14:49.208 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:14:49.207+0000 7ff03e000640 1 --2- 172.21.15.22:0/2342250406 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x564e374c5f20 0x564e374c63a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:14:49.208 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:14:49.207+0000 7ff03e000640 1 -- 172.21.15.22:0/2342250406 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564e373bd690 con 0x564e374cf290 2024-03-28T18:14:49.208 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:14:49.207+0000 7ff03f400640 1 --2- 172.21.15.22:0/2342250406 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564e374c6c00 0x564e374c5950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-28T18:14:49.208 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:14:49.207+0000 7ff037400640 1 --2- 172.21.15.22:0/2272057994 >> [v2:172.21.15.22:6800/1357365082,v1:172.21.15.22:6801/1357365082] conn(0x7ff0240a2970 0x7ff0240a4e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:14:49.209 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:14:49.207+0000 7ff03e000640 1 --2- 172.21.15.22:0/2342250406 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x564e374cf290 0x564e374d1680 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7ff02c00b740 tx=0x7ff02c00bc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=59369557b834958c server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:14:49.209 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:14:49.207+0000 7ff037400640 1 -- 172.21.15.22:0/2272057994 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(111..111 src has 1..111) v4 ==== 9787+0+0 (secure 0 0 0) 0x7ff0300f1380 con 0x7ff040103840 2024-03-28T18:14:49.209 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:14:49.207+0000 7ff03f400640 1 --2- 172.21.15.22:0/2272057994 >> [v2:172.21.15.22:6800/1357365082,v1:172.21.15.22:6801/1357365082] conn(0x7ff0240a2970 0x7ff0240a4e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:14:49.209 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:14:49.207+0000 7ff02be00640 1 -- 172.21.15.22:0/2342250406 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff02c003dd0 con 0x564e374cf290 2024-03-28T18:14:49.209 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:14:49.207+0000 7ff02be00640 1 -- 172.21.15.22:0/2342250406 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7ff02c015070 con 0x564e374cf290 2024-03-28T18:14:49.209 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:14:49.207+0000 7ff02be00640 1 -- 172.21.15.22:0/2342250406 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff02c004e10 con 0x564e374cf290 2024-03-28T18:14:49.213 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:14:49.208+0000 7ff04878b740 1 -- 172.21.15.22:0/2342250406 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x564e374cf290 msgr2=0x564e374d1680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:14:49.213 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:14:49.208+0000 7ff03f400640 1 --2- 172.21.15.22:0/2272057994 >> [v2:172.21.15.22:6800/1357365082,v1:172.21.15.22:6801/1357365082] conn(0x7ff0240a2970 0x7ff0240a4e30 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7ff038005fd0 tx=0x7ff038009500 comp rx=0 tx=0).ready entity=mgr.15522 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:14:49.213 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:14:49.208+0000 7ff04878b740 1 --2- 172.21.15.22:0/2342250406 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x564e374cf290 0x564e374d1680 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7ff02c00b740 tx=0x7ff02c00bc00 comp rx=0 tx=0).stop 2024-03-28T18:14:49.213 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:14:49.208+0000 7ff04878b740 1 -- 172.21.15.22:0/2342250406 shutdown_connections 2024-03-28T18:14:49.213 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:14:49.208+0000 7ff04878b740 1 --2- 172.21.15.22:0/2342250406 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x564e374cf290 0x564e374d1680 unknown :-1 s=CLOSED pgs=744 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:14:49.213 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:14:49.208+0000 7ff04878b740 1 --2- 172.21.15.22:0/2342250406 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x564e374c5f20 0x564e374c63a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:14:49.213 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:14:49.208+0000 7ff04878b740 1 --2- 172.21.15.22:0/2342250406 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564e374c6c00 0x564e374c5950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:14:49.213 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:14:49.208+0000 7ff04878b740 1 -- 172.21.15.22:0/2342250406 >> 172.21.15.22:0/2342250406 conn(0x564e3748d200 msgr2=0x564e374c0970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:14:49.213 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:14:49.208+0000 7ff04878b740 1 -- 172.21.15.22:0/2342250406 shutdown_connections 2024-03-28T18:14:49.213 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:14:49.209+0000 7ff04878b740 1 -- 172.21.15.22:0/2342250406 wait complete. 2024-03-28T18:14:49.214 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:14:49.209+0000 7ff04878b740 1 -- start start 2024-03-28T18:14:49.214 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:14:49.209+0000 7ff04878b740 1 Processor -- start 2024-03-28T18:14:49.214 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:14:49.210+0000 7ff04878b740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x564e374c5f20 0x564e374ce910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:14:49.214 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:14:49.210+0000 7ff04878b740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564e374c6c00 0x564e374cbca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:14:49.214 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:14:49.210+0000 7ff04878b740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x564e374cf290 0x564e374cc1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:14:49.214 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:14:49.210+0000 7ff04878b740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x564e374d2100 con 0x564e374c5f20 2024-03-28T18:14:49.214 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:14:49.210+0000 7ff04878b740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x564e373e5e80 con 0x564e374cf290 2024-03-28T18:14:49.214 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:14:49.210+0000 7ff04878b740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x564e374d2280 con 0x564e374c6c00 2024-03-28T18:14:49.214 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:14:49.210+0000 7ff04878b740 10 client.0 _ll_register_callbacks cb 0x564e373f8130 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-28T18:14:49.214 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:14:49.210+0000 7ff03ea00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x564e374c5f20 0x564e374ce910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:14:49.214 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:14:49.210+0000 7ff03ea00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x564e374c5f20 0x564e374ce910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:50430/0 (socket says 172.21.15.22:50430) 2024-03-28T18:14:49.214 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:14:49.210+0000 7ff03ea00640 1 -- 172.21.15.22:0/1446137880 learned_addr learned my addr 172.21.15.22:0/1446137880 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-28T18:14:49.214 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:14:49.210+0000 7ff03f400640 1 --2- 172.21.15.22:0/1446137880 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x564e374cf290 0x564e374cc1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:14:49.214 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:14:49.210+0000 7ff03e000640 1 --2- 172.21.15.22:0/1446137880 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564e374c6c00 0x564e374cbca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:14:49.215 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:14:49.211+0000 7ff03ea00640 1 -- 172.21.15.22:0/1446137880 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564e374c6c00 msgr2=0x564e374cbca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T18:14:49.215 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:14:49.211+0000 7ff03ea00640 1 --2- 172.21.15.22:0/1446137880 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564e374c6c00 0x564e374cbca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:14:49.215 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:14:49.211+0000 7ff03ea00640 1 -- 172.21.15.22:0/1446137880 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x564e374cf290 msgr2=0x564e374cc1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T18:14:49.215 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:14:49.211+0000 7ff03ea00640 1 --2- 172.21.15.22:0/1446137880 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x564e374cf290 0x564e374cc1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:14:49.215 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:14:49.211+0000 7ff03ea00640 1 -- 172.21.15.22:0/1446137880 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x564e372087d0 con 0x564e374c5f20 2024-03-28T18:14:49.215 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:14:49.211+0000 7ff03ea00640 1 --2- 172.21.15.22:0/1446137880 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x564e374c5f20 0x564e374ce910 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7ff030005ec0 tx=0x7ff0300b9240 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bdbf228fa7f48ab server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:14:49.215 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:14:49.211+0000 7ff028c00640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-03-28T18:14:49.215 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:14:49.211+0000 7ff028c00640 1 -- 172.21.15.22:0/1446137880 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0300bd0e0 con 0x564e374c5f20 2024-03-28T18:14:49.215 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:14:49.211+0000 7ff028c00640 1 -- 172.21.15.22:0/1446137880 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(111..111 src has 1..111) v4 ==== 9787+0+0 (secure 0 0 0) 0x7ff030003c80 con 0x564e374c5f20 2024-03-28T18:14:49.215 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:14:49.211+0000 7ff04878b740 1 -- 172.21.15.22:0/1446137880 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7ff02c015070 con 0x564e374c5f20 2024-03-28T18:14:49.215 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:14:49.211+0000 7ff028c00640 1 -- 172.21.15.22:0/1446137880 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=139) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff0300bcf00 con 0x564e374c5f20 2024-03-28T18:14:49.215 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:14:49.211+0000 7ff04878b740 10 client.16692 fetch_fsmap learned FSMap version 139 2024-03-28T18:14:49.215 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:14:49.211+0000 7ff04878b740 1 -- 172.21.15.22:0/1446137880 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=139,monmap=4+}) v3 -- 0x564e3713e3f0 con 0x564e374c5f20 2024-03-28T18:14:49.215 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:14:49.212+0000 7ff028c00640 1 -- 172.21.15.22:0/1446137880 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7ff02c015070 con 0x564e374c5f20 2024-03-28T18:14:49.215 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:14:49.212+0000 7ff028c00640 1 -- 172.21.15.22:0/1446137880 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 139) v1 ==== 6788+0+0 (secure 0 0 0) 0x7ff0300ee800 con 0x564e374c5f20 2024-03-28T18:14:49.216 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:14:49.212+0000 7ff04878b740 10 client.16692 fetch_fsmap finished waiting for FSMap version 139 2024-03-28T18:14:49.216 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:14:49.212+0000 7ff04878b740 10 client.16692 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-28T18:14:49.216 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:14:49.212+0000 7ff04878b740 20 client.16692 populate_metadata read hostname 'smithi022' 2024-03-28T18:14:49.216 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:14:49.212+0000 7ff04878b740 1 --2- 172.21.15.22:0/1446137880 >> [v2:172.21.15.79:6838/2441542443,v1:172.21.15.79:6839/2441542443] conn(0x564e374df090 0x564e374e1480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:14:49.216 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:14:49.212+0000 7ff04878b740 4 client.16692 mds_command: new command op to 34553 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-28T18:14:49.216 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:14:49.212+0000 7ff04878b740 1 -- 172.21.15.22:0/1446137880 --> [v2:172.21.15.79:6838/2441542443,v1:172.21.15.79:6839/2441542443] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x564e37437790 con 0x564e374df090 2024-03-28T18:14:49.216 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:14:49.212+0000 7ff03e000640 1 --2- 172.21.15.22:0/1446137880 >> [v2:172.21.15.79:6838/2441542443,v1:172.21.15.79:6839/2441542443] conn(0x564e374df090 0x564e374e1480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:14:49.216 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:14:49.213+0000 7ff03e000640 1 --2- 172.21.15.22:0/1446137880 >> [v2:172.21.15.79:6838/2441542443,v1:172.21.15.79:6839/2441542443] conn(0x564e374df090 0x564e374e1480 crc :-1 s=READY pgs=220 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=925c05cb659330ff server_cookie=578b114f78100fe0 in_seq=0 out_seq=0 2024-03-28T18:14:49.216 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:14:49.213+0000 7ff028c00640 10 client.16692 ms_handle_connect on v2:172.21.15.79:6838/2441542443 2024-03-28T18:14:49.217 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:14:49.215+0000 7ff028c00640 1 -- 172.21.15.22:0/1446137880 <== mds.0 v2:172.21.15.79:6838/2441542443 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x564e37437790 con 0x564e374df090 2024-03-28T18:14:49.217 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:14:49.215+0000 7ff028c00640 10 client.16692 handle_command_reply: tid=0 2024-03-28T18:14:49.217 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:14:49.215+0000 7ff04878b740 1 client.16692 shutdown 2024-03-28T18:14:49.217 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:14:49.215+0000 7ff04878b740 1 -- 172.21.15.22:0/1446137880 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x564e374c5f20 msgr2=0x564e374ce910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:14:49.218 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:14:49.215+0000 7ff04878b740 1 --2- 172.21.15.22:0/1446137880 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x564e374c5f20 0x564e374ce910 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7ff030005ec0 tx=0x7ff0300b9240 comp rx=0 tx=0).stop 2024-03-28T18:14:49.218 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:14:49.215+0000 7ff04878b740 1 -- 172.21.15.22:0/1446137880 shutdown_connections 2024-03-28T18:14:49.218 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:14:49.215+0000 7ff04878b740 1 --2- 172.21.15.22:0/1446137880 >> [v2:172.21.15.79:6838/2441542443,v1:172.21.15.79:6839/2441542443] conn(0x564e374df090 0x564e374e1480 crc :-1 s=READY pgs=220 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:14:49.218 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:14:49.215+0000 7ff04878b740 1 --2- 172.21.15.22:0/1446137880 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x564e374cf290 0x564e374cc1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:14:49.218 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:14:49.215+0000 7ff04878b740 1 --2- 172.21.15.22:0/1446137880 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564e374c6c00 0x564e374cbca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:14:49.218 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:14:49.215+0000 7ff04878b740 1 --2- 172.21.15.22:0/1446137880 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x564e374c5f20 0x564e374ce910 unknown :-1 s=CLOSED pgs=745 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:14:49.218 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:14:49.215+0000 7ff028c00640 0 client.16692 ms_handle_reset on v2:172.21.15.79:6838/2441542443 2024-03-28T18:14:49.218 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:14:49.215+0000 7ff04878b740 1 -- 172.21.15.22:0/1446137880 >> 172.21.15.22:0/1446137880 conn(0x564e3748d200 msgr2=0x564e374c26c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:14:49.218 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:14:49.215+0000 7ff04878b740 1 -- 172.21.15.22:0/1446137880 shutdown_connections 2024-03-28T18:14:49.218 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:14:49.215+0000 7ff04878b740 1 -- 172.21.15.22:0/1446137880 wait complete. 2024-03-28T18:14:49.218 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:14:49.215+0000 7ff04878b740 20 client.16692 trim_cache size 0 max 16384 2024-03-28T18:14:49.234 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:14:49.233+0000 7ff04878b740 1 Processor -- start 2024-03-28T18:14:49.234 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:14:49.233+0000 7ff04878b740 1 -- start start 2024-03-28T18:14:49.234 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:14:49.233+0000 7ff04878b740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x564e374c5f20 0x564e374ce360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:14:49.234 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:14:49.233+0000 7ff04878b740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564e374c6c00 0x564e374ce8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:14:49.234 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:14:49.233+0000 7ff04878b740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x564e374cf290 0x564e374d0680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:14:49.234 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:14:49.233+0000 7ff04878b740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0401af0b0 con 0x564e374c5f20 2024-03-28T18:14:49.234 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:14:49.233+0000 7ff04878b740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0401b1090 con 0x564e374cf290 2024-03-28T18:14:49.234 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:14:49.233+0000 7ff04878b740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0401af710 con 0x564e374c6c00 2024-03-28T18:14:49.234 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:14:49.234+0000 7ff03f400640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x564e374cf290 0x564e374d0680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:14:49.234 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:14:49.234+0000 7ff03f400640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x564e374cf290 0x564e374d0680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.22:55884/0 (socket says 172.21.15.22:55884) 2024-03-28T18:14:49.234 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:14:49.234+0000 7ff03ea00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x564e374c5f20 0x564e374ce360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:14:49.235 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:14:49.234+0000 7ff03f400640 1 -- 172.21.15.22:0/4116766335 learned_addr learned my addr 172.21.15.22:0/4116766335 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-28T18:14:49.235 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:14:49.234+0000 7ff03e000640 1 --2- 172.21.15.22:0/4116766335 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564e374c6c00 0x564e374ce8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:14:49.236 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:14:49.234+0000 7ff03ea00640 1 -- 172.21.15.22:0/4116766335 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564e374c6c00 msgr2=0x564e374ce8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T18:14:49.236 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:14:49.234+0000 7ff03ea00640 1 --2- 172.21.15.22:0/4116766335 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564e374c6c00 0x564e374ce8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:14:49.236 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:14:49.234+0000 7ff03ea00640 1 -- 172.21.15.22:0/4116766335 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x564e374cf290 msgr2=0x564e374d0680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T18:14:49.236 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:14:49.234+0000 7ff03ea00640 1 --2- 172.21.15.22:0/4116766335 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x564e374cf290 0x564e374d0680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:14:49.236 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:14:49.235+0000 7ff03ea00640 1 -- 172.21.15.22:0/4116766335 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564e370fe0f0 con 0x564e374c5f20 2024-03-28T18:14:49.237 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:14:49.235+0000 7ff03f400640 1 --2- 172.21.15.22:0/4116766335 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x564e374cf290 0x564e374d0680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-28T18:14:49.237 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:14:49.235+0000 7ff03e000640 1 --2- 172.21.15.22:0/4116766335 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564e374c6c00 0x564e374ce8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-28T18:14:49.237 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:14:49.235+0000 7ff03ea00640 1 --2- 172.21.15.22:0/4116766335 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x564e374c5f20 0x564e374ce360 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7ff0300b9f70 tx=0x7ff0300bbab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=899e46f546123a86 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:14:49.237 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:14:49.235+0000 7ff02b400640 1 -- 172.21.15.22:0/4116766335 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0300d5070 con 0x564e374c5f20 2024-03-28T18:14:49.237 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:14:49.235+0000 7ff02b400640 1 -- 172.21.15.22:0/4116766335 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7ff0300f1170 con 0x564e374c5f20 2024-03-28T18:14:49.237 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:14:49.235+0000 7ff02b400640 1 -- 172.21.15.22:0/4116766335 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0300ee770 con 0x564e374c5f20 2024-03-28T18:14:49.237 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:14:49.236+0000 7ff04878b740 1 -- 172.21.15.22:0/4116766335 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x564e374c5f20 msgr2=0x564e374ce360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:14:49.237 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:14:49.236+0000 7ff04878b740 1 --2- 172.21.15.22:0/4116766335 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x564e374c5f20 0x564e374ce360 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7ff0300b9f70 tx=0x7ff0300bbab0 comp rx=0 tx=0).stop 2024-03-28T18:14:49.237 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:14:49.236+0000 7ff04878b740 1 -- 172.21.15.22:0/4116766335 shutdown_connections 2024-03-28T18:14:49.237 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:14:49.236+0000 7ff04878b740 1 --2- 172.21.15.22:0/4116766335 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x564e374cf290 0x564e374d0680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:14:49.237 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:14:49.236+0000 7ff04878b740 1 --2- 172.21.15.22:0/4116766335 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564e374c6c00 0x564e374ce8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:14:49.237 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:14:49.236+0000 7ff04878b740 1 --2- 172.21.15.22:0/4116766335 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x564e374c5f20 0x564e374ce360 unknown :-1 s=CLOSED pgs=746 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:14:49.237 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:14:49.236+0000 7ff04878b740 1 -- 172.21.15.22:0/4116766335 >> 172.21.15.22:0/4116766335 conn(0x564e3748d200 msgr2=0x564e374c04a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:14:49.237 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:14:49.236+0000 7ff04878b740 1 -- 172.21.15.22:0/4116766335 shutdown_connections 2024-03-28T18:14:49.238 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:14:49.236+0000 7ff04878b740 1 -- 172.21.15.22:0/4116766335 wait complete. 2024-03-28T18:14:49.238 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:14:49.236+0000 7ff04878b740 1 -- start start 2024-03-28T18:14:49.238 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:14:49.237+0000 7ff04878b740 1 Processor -- start 2024-03-28T18:14:49.241 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:14:49.237+0000 7ff04878b740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x564e374c5f20 0x564e374d31b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:14:49.242 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:14:49.237+0000 7ff04878b740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564e374c6c00 0x564e374d36f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:14:49.242 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:14:49.237+0000 7ff04878b740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x564e374cf290 0x564e37533920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:14:49.242 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:14:49.237+0000 7ff04878b740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0401afd70 con 0x564e374c5f20 2024-03-28T18:14:49.242 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:14:49.237+0000 7ff04878b740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0401b0a30 con 0x564e374cf290 2024-03-28T18:14:49.242 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:14:49.237+0000 7ff04878b740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0401b03d0 con 0x564e374c6c00 2024-03-28T18:14:49.242 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:14:49.237+0000 7ff04878b740 10 client.0 _ll_register_callbacks cb 0x7ff030005b50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-28T18:14:49.242 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:14:49.237+0000 7ff03ea00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x564e374c5f20 0x564e374d31b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:14:49.242 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:14:49.237+0000 7ff03ea00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x564e374c5f20 0x564e374d31b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:50454/0 (socket says 172.21.15.22:50454) 2024-03-28T18:14:49.242 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:14:49.237+0000 7ff03ea00640 1 -- 172.21.15.22:0/3586520689 learned_addr learned my addr 172.21.15.22:0/3586520689 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-28T18:14:49.242 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:14:49.237+0000 7ff03f400640 1 --2- 172.21.15.22:0/3586520689 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x564e374cf290 0x564e37533920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:14:49.242 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:14:49.238+0000 7ff03e000640 1 --2- 172.21.15.22:0/3586520689 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564e374c6c00 0x564e374d36f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:14:49.243 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:14:49.238+0000 7ff03ea00640 1 -- 172.21.15.22:0/3586520689 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564e374c6c00 msgr2=0x564e374d36f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T18:14:49.243 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:14:49.238+0000 7ff03ea00640 1 --2- 172.21.15.22:0/3586520689 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564e374c6c00 0x564e374d36f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:14:49.243 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:14:49.238+0000 7ff03ea00640 1 -- 172.21.15.22:0/3586520689 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x564e374cf290 msgr2=0x564e37533920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T18:14:49.243 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:14:49.238+0000 7ff03ea00640 1 --2- 172.21.15.22:0/3586520689 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x564e374cf290 0x564e37533920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:14:49.243 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:14:49.238+0000 7ff03ea00640 1 -- 172.21.15.22:0/3586520689 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x564e37533f10 con 0x564e374c5f20 2024-03-28T18:14:49.243 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:14:49.238+0000 7ff03f400640 1 --2- 172.21.15.22:0/3586520689 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x564e374cf290 0x564e37533920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-28T18:14:49.243 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:14:49.238+0000 7ff03ea00640 1 --2- 172.21.15.22:0/3586520689 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x564e374c5f20 0x564e374d31b0 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7ff030004e10 tx=0x7ff0300b9960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c20c0d48424b0658 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:14:49.243 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:14:49.238+0000 7ff028c00640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-03-28T18:14:49.243 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:14:49.238+0000 7ff028c00640 1 -- 172.21.15.22:0/3586520689 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0300d5030 con 0x564e374c5f20 2024-03-28T18:14:49.243 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:14:49.238+0000 7ff028c00640 1 -- 172.21.15.22:0/3586520689 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(111..111 src has 1..111) v4 ==== 9787+0+0 (secure 0 0 0) 0x7ff0300c5ab0 con 0x564e374c5f20 2024-03-28T18:14:49.243 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:14:49.238+0000 7ff04878b740 1 -- 172.21.15.22:0/3586520689 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7ff0300f1170 con 0x564e374c5f20 2024-03-28T18:14:49.243 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:14:49.239+0000 7ff028c00640 1 -- 172.21.15.22:0/3586520689 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=139) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff0300bd5a0 con 0x564e374c5f20 2024-03-28T18:14:49.243 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:14:49.239+0000 7ff04878b740 10 client.16698 fetch_fsmap learned FSMap version 139 2024-03-28T18:14:49.243 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:14:49.239+0000 7ff04878b740 1 -- 172.21.15.22:0/3586520689 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=139,monmap=4+}) v3 -- 0x564e37533f10 con 0x564e374c5f20 2024-03-28T18:14:49.244 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:14:49.239+0000 7ff028c00640 1 -- 172.21.15.22:0/3586520689 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7ff0300f1170 con 0x564e374c5f20 2024-03-28T18:14:49.244 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:14:49.239+0000 7ff028c00640 1 -- 172.21.15.22:0/3586520689 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 139) v1 ==== 6788+0+0 (secure 0 0 0) 0x7ff0300d60f0 con 0x564e374c5f20 2024-03-28T18:14:49.244 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:14:49.239+0000 7ff04878b740 10 client.16698 fetch_fsmap finished waiting for FSMap version 139 2024-03-28T18:14:49.244 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:14:49.239+0000 7ff04878b740 10 client.16698 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-28T18:14:49.244 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:14:49.239+0000 7ff04878b740 20 client.16698 populate_metadata read hostname 'smithi022' 2024-03-28T18:14:49.244 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:14:49.239+0000 7ff04878b740 1 --2- 172.21.15.22:0/3586520689 >> [v2:172.21.15.79:6838/2441542443,v1:172.21.15.79:6839/2441542443] conn(0x564e3753b4b0 0x564e3753d8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:14:49.244 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:14:49.239+0000 7ff04878b740 4 client.16698 mds_command: new command op to 34553 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-28T18:14:49.244 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:14:49.239+0000 7ff04878b740 1 -- 172.21.15.22:0/3586520689 --> [v2:172.21.15.79:6838/2441542443,v1:172.21.15.79:6839/2441542443] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x564e37431ae0 con 0x564e3753b4b0 2024-03-28T18:14:49.244 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:14:49.240+0000 7ff03e000640 1 --2- 172.21.15.22:0/3586520689 >> [v2:172.21.15.79:6838/2441542443,v1:172.21.15.79:6839/2441542443] conn(0x564e3753b4b0 0x564e3753d8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:14:49.244 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:14:49.240+0000 7ff03e000640 1 --2- 172.21.15.22:0/3586520689 >> [v2:172.21.15.79:6838/2441542443,v1:172.21.15.79:6839/2441542443] conn(0x564e3753b4b0 0x564e3753d8a0 crc :-1 s=READY pgs=221 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=19780a0649d49528 server_cookie=17ecdc1c9f1a8fee in_seq=0 out_seq=0 2024-03-28T18:14:49.244 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:14:49.240+0000 7ff028c00640 10 client.16698 ms_handle_connect on v2:172.21.15.79:6838/2441542443 2024-03-28T18:14:49.244 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:14:49.241+0000 7ff028c00640 1 -- 172.21.15.22:0/3586520689 <== mds.0 v2:172.21.15.79:6838/2441542443 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x564e37431ae0 con 0x564e3753b4b0 2024-03-28T18:14:49.244 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:14:49.241+0000 7ff028c00640 10 client.16698 handle_command_reply: tid=0 2024-03-28T18:14:49.244 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:14:49.241+0000 7ff04878b740 1 client.16698 shutdown 2024-03-28T18:14:49.245 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:14:49.241+0000 7ff04878b740 1 -- 172.21.15.22:0/3586520689 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x564e374c5f20 msgr2=0x564e374d31b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:14:49.245 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:14:49.241+0000 7ff04878b740 1 --2- 172.21.15.22:0/3586520689 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x564e374c5f20 0x564e374d31b0 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7ff030004e10 tx=0x7ff0300b9960 comp rx=0 tx=0).stop 2024-03-28T18:14:49.245 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:14:49.241+0000 7ff04878b740 1 -- 172.21.15.22:0/3586520689 shutdown_connections 2024-03-28T18:14:49.245 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:14:49.241+0000 7ff04878b740 1 --2- 172.21.15.22:0/3586520689 >> [v2:172.21.15.79:6838/2441542443,v1:172.21.15.79:6839/2441542443] conn(0x564e3753b4b0 0x564e3753d8a0 crc :-1 s=READY pgs=221 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:14:49.245 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:14:49.241+0000 7ff04878b740 1 --2- 172.21.15.22:0/3586520689 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x564e374cf290 0x564e37533920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:14:49.245 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:14:49.241+0000 7ff04878b740 1 --2- 172.21.15.22:0/3586520689 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564e374c6c00 0x564e374d36f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:14:49.245 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:14:49.241+0000 7ff04878b740 1 --2- 172.21.15.22:0/3586520689 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x564e374c5f20 0x564e374d31b0 unknown :-1 s=CLOSED pgs=747 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:14:49.245 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:14:49.241+0000 7ff028c00640 0 client.16698 ms_handle_reset on v2:172.21.15.79:6838/2441542443 2024-03-28T18:14:49.245 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:14:49.241+0000 7ff04878b740 1 -- 172.21.15.22:0/3586520689 >> 172.21.15.22:0/3586520689 conn(0x564e3748d200 msgr2=0x564e374c1cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:14:49.245 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:14:49.241+0000 7ff04878b740 1 -- 172.21.15.22:0/3586520689 shutdown_connections 2024-03-28T18:14:49.245 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:14:49.241+0000 7ff04878b740 1 -- 172.21.15.22:0/3586520689 wait complete. 2024-03-28T18:14:49.245 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:14:49.241+0000 7ff04878b740 20 client.16698 trim_cache size 0 max 16384 2024-03-28T18:14:49.246 INFO:teuthology.orchestra.run.smithi022.stdout:{ 2024-03-28T18:14:49.246 INFO:teuthology.orchestra.run.smithi022.stdout: "status": "IDLE (waiting for more scrubs)", 2024-03-28T18:14:49.246 INFO:teuthology.orchestra.run.smithi022.stdout: "scrubs": { 2024-03-28T18:14:49.246 INFO:teuthology.orchestra.run.smithi022.stdout: "4bbbd61d-29a3-4bf6-a4f2-d5783d585de3": { 2024-03-28T18:14:49.246 INFO:teuthology.orchestra.run.smithi022.stdout: "path": "/", 2024-03-28T18:14:49.246 INFO:teuthology.orchestra.run.smithi022.stdout: "tag": "4bbbd61d-29a3-4bf6-a4f2-d5783d585de3", 2024-03-28T18:14:49.246 INFO:teuthology.orchestra.run.smithi022.stdout: "options": "recursive,force" 2024-03-28T18:14:49.246 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-03-28T18:14:49.246 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-03-28T18:14:49.246 INFO:teuthology.orchestra.run.smithi022.stdout:} 2024-03-28T18:14:49.247 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:14:49.242+0000 7ff045a00640 1 -- 172.21.15.22:0/2272057994 >> [v2:172.21.15.22:6800/1357365082,v1:172.21.15.22:6801/1357365082] conn(0x7ff0240a2970 msgr2=0x7ff0240a4e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:14:49.247 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:14:49.242+0000 7ff045a00640 1 --2- 172.21.15.22:0/2272057994 >> [v2:172.21.15.22:6800/1357365082,v1:172.21.15.22:6801/1357365082] conn(0x7ff0240a2970 0x7ff0240a4e30 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7ff038005fd0 tx=0x7ff038009500 comp rx=0 tx=0).stop 2024-03-28T18:14:49.247 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:14:49.242+0000 7ff045a00640 1 -- 172.21.15.22:0/2272057994 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff040103840 msgr2=0x7ff04019a1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:14:49.247 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:14:49.242+0000 7ff045a00640 1 --2- 172.21.15.22:0/2272057994 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff040103840 0x7ff04019a1e0 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7ff030000c00 tx=0x7ff0300045e0 comp rx=0 tx=0).stop 2024-03-28T18:14:49.247 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:14:49.242+0000 7ff045a00640 1 -- 172.21.15.22:0/2272057994 shutdown_connections 2024-03-28T18:14:49.247 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:14:49.242+0000 7ff045a00640 1 --2- 172.21.15.22:0/2272057994 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff040109850 0x7ff04019eaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:14:49.247 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:14:49.242+0000 7ff045a00640 1 --2- 172.21.15.22:0/2272057994 >> [v2:172.21.15.22:6800/1357365082,v1:172.21.15.22:6801/1357365082] conn(0x7ff0240a2970 0x7ff0240a4e30 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:14:49.247 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:14:49.242+0000 7ff045a00640 1 --2- 172.21.15.22:0/2272057994 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ff040104200 0x7ff04019a720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:14:49.248 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:14:49.242+0000 7ff045a00640 1 --2- 172.21.15.22:0/2272057994 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff040103840 0x7ff04019a1e0 unknown :-1 s=CLOSED pgs=743 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:14:49.248 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:14:49.242+0000 7ff045a00640 1 -- 172.21.15.22:0/2272057994 >> 172.21.15.22:0/2272057994 conn(0x7ff0400ff2f0 msgr2=0x7ff0400ffb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:14:49.248 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:14:49.242+0000 7ff045a00640 1 -- 172.21.15.22:0/2272057994 shutdown_connections 2024-03-28T18:14:49.248 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:14:49.242+0000 7ff045a00640 1 -- 172.21.15.22:0/2272057994 wait complete. 2024-03-28T18:14:49.259 INFO:tasks.cephfs.filesystem:scrub status for tag:4bbbd61d-29a3-4bf6-a4f2-d5783d585de3 - {'path': '/', 'tag': '4bbbd61d-29a3-4bf6-a4f2-d5783d585de3', 'options': 'recursive,force'} 2024-03-28T18:14:49.359 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:14:49 smithi022 ceph-mon[31985]: mds.? [v2:172.21.15.22:6836/2984059496,v1:172.21.15.22:6837/2984059496] up:rejoin 2024-03-28T18:14:49.360 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:14:49 smithi022 ceph-mon[31985]: fsmap cephfs:5/5 4 up:standby-replay 1 up:standby 1 up:rejoin 4 up:active 2024-03-28T18:14:49.360 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:14:49 smithi022 ceph-mon[31985]: daemon mds.d is now active in filesystem cephfs as rank 1 2024-03-28T18:14:49.674 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:14:49 smithi134 ceph-mon[35037]: mds.? [v2:172.21.15.22:6836/2984059496,v1:172.21.15.22:6837/2984059496] up:rejoin 2024-03-28T18:14:49.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:14:49 smithi134 ceph-mon[35037]: fsmap cephfs:5/5 4 up:standby-replay 1 up:standby 1 up:rejoin 4 up:active 2024-03-28T18:14:49.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:14:49 smithi134 ceph-mon[35037]: daemon mds.d is now active in filesystem cephfs as rank 1 2024-03-28T18:14:49.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:14:49 smithi079 ceph-mon[35312]: mds.? [v2:172.21.15.22:6836/2984059496,v1:172.21.15.22:6837/2984059496] up:rejoin 2024-03-28T18:14:49.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:14:49 smithi079 ceph-mon[35312]: fsmap cephfs:5/5 4 up:standby-replay 1 up:standby 1 up:rejoin 4 up:active 2024-03-28T18:14:49.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:14:49 smithi079 ceph-mon[35312]: daemon mds.d is now active in filesystem cephfs as rank 1 2024-03-28T18:14:49.784 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:14:49 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[93820]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-2145-g3be9a5c9/rpm/el8/BUILD/ceph-19.0.0-2145-g3be9a5c9/src/mds/Locker.cc: In function 'void Locker::set_xlocks_done(MutationImpl*, bool)' thread 7fcf0b400700 time 2024-03-28T18:14:49.360578+0000 2024-03-28T18:14:49.784 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:14:49 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[93820]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-2145-g3be9a5c9/rpm/el8/BUILD/ceph-19.0.0-2145-g3be9a5c9/src/mds/Locker.cc: 626: FAILED ceph_assert(obj->is_auth()) 2024-03-28T18:14:49.784 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:14:49 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[93820]: ceph version 19.0.0-2145-g3be9a5c9 (3be9a5c9bc793e11e2800a8c0c696e8b46742033) squid (dev) 2024-03-28T18:14:49.784 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:14:49 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[93820]: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x135) [0x7fcf16d0c4a1] 2024-03-28T18:14:49.785 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:14:49 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[93820]: 2: /usr/lib64/ceph/libceph-common.so.2(+0x2d2667) [0x7fcf16d0c667] 2024-03-28T18:14:49.785 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:14:49 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[93820]: 3: (Locker::set_xlocks_done(MutationImpl*, bool)+0x493) [0x55fa93613223] 2024-03-28T18:14:49.785 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:14:49 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[93820]: 4: (Server::early_reply(boost::intrusive_ptr const&, CInode*, CDentry*)+0x3e6) [0x55fa9348d5d6] 2024-03-28T18:14:49.785 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:14:49 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[93820]: 5: (Server::journal_and_reply(boost::intrusive_ptr const&, CInode*, CDentry*, LogEvent*, MDSLogContextBase*)+0xc1) [0x55fa9348dda1] 2024-03-28T18:14:49.785 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:14:49 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[93820]: 6: (Server::handle_client_mkdir(boost::intrusive_ptr const&)+0x5ba) [0x55fa934c36aa] 2024-03-28T18:14:49.785 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:14:49 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[93820]: 7: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0x733) [0x55fa934e0233] 2024-03-28T18:14:49.785 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:14:49 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[93820]: 8: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x55fa934e2197] 2024-03-28T18:14:49.785 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:14:49 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[93820]: 9: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x55fa933e111b] 2024-03-28T18:14:49.785 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:14:49 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[93820]: 10: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x55fa933e16bc] 2024-03-28T18:14:49.785 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:14:49 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[93820]: 11: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bf) [0x55fa933cb1ff] 2024-03-28T18:14:49.785 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:14:49 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[93820]: 12: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x478) [0x7fcf16f9e488] 2024-03-28T18:14:49.785 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:14:49 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[93820]: 13: (DispatchQueue::entry()+0x51f) [0x7fcf16f9b60f] 2024-03-28T18:14:49.785 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:14:49 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[93820]: 14: (DispatchQueue::DispatchThread::entry()+0x11) [0x7fcf1706ee81] 2024-03-28T18:14:49.785 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:14:49 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[93820]: 15: /lib64/libpthread.so.0(+0x81ca) [0x7fcf15a891ca] 2024-03-28T18:14:49.785 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:14:49 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[93820]: 16: clone() 2024-03-28T18:14:49.787 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:14:49 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[93820]: 2024-03-28T18:14:49.361+0000 7fcf0b400700 -1 /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-2145-g3be9a5c9/rpm/el8/BUILD/ceph-19.0.0-2145-g3be9a5c9/src/mds/Locker.cc: In function 'void Locker::set_xlocks_done(MutationImpl*, bool)' thread 7fcf0b400700 time 2024-03-28T18:14:49.360578+0000 2024-03-28T18:14:49.787 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:14:49 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[93820]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-2145-g3be9a5c9/rpm/el8/BUILD/ceph-19.0.0-2145-g3be9a5c9/src/mds/Locker.cc: 626: FAILED ceph_assert(obj->is_auth()) 2024-03-28T18:14:49.787 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:14:49 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[93820]: 2024-03-28T18:14:49.787 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:14:49 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[93820]: ceph version 19.0.0-2145-g3be9a5c9 (3be9a5c9bc793e11e2800a8c0c696e8b46742033) squid (dev) 2024-03-28T18:14:49.787 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:14:49 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[93820]: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x135) [0x7fcf16d0c4a1] 2024-03-28T18:14:49.787 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:14:49 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[93820]: 2: /usr/lib64/ceph/libceph-common.so.2(+0x2d2667) [0x7fcf16d0c667] 2024-03-28T18:14:49.788 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:14:49 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[93820]: 3: (Locker::set_xlocks_done(MutationImpl*, bool)+0x493) [0x55fa93613223] 2024-03-28T18:14:49.788 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:14:49 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[93820]: 4: (Server::early_reply(boost::intrusive_ptr const&, CInode*, CDentry*)+0x3e6) [0x55fa9348d5d6] 2024-03-28T18:14:49.788 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:14:49 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[93820]: 5: (Server::journal_and_reply(boost::intrusive_ptr const&, CInode*, CDentry*, LogEvent*, MDSLogContextBase*)+0xc1) [0x55fa9348dda1] 2024-03-28T18:14:49.788 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:14:49 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[93820]: 6: (Server::handle_client_mkdir(boost::intrusive_ptr const&)+0x5ba) [0x55fa934c36aa] 2024-03-28T18:14:49.788 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:14:49 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[93820]: 7: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0x733) [0x55fa934e0233] 2024-03-28T18:14:49.788 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:14:49 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[93820]: 8: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x55fa934e2197] 2024-03-28T18:14:49.788 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:14:49 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[93820]: 9: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x55fa933e111b] 2024-03-28T18:14:49.788 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:14:49 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[93820]: 10: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x55fa933e16bc] 2024-03-28T18:14:49.788 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:14:49 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[93820]: 11: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bf) [0x55fa933cb1ff] 2024-03-28T18:14:49.788 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:14:49 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[93820]: 12: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x478) [0x7fcf16f9e488] 2024-03-28T18:14:49.788 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:14:49 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[93820]: 13: (DispatchQueue::entry()+0x51f) [0x7fcf16f9b60f] 2024-03-28T18:14:49.788 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:14:49 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[93820]: 14: (DispatchQueue::DispatchThread::entry()+0x11) [0x7fcf1706ee81] 2024-03-28T18:14:49.788 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:14:49 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[93820]: 15: /lib64/libpthread.so.0(+0x81ca) [0x7fcf15a891ca] 2024-03-28T18:14:49.788 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:14:49 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[93820]: 16: clone() 2024-03-28T18:14:49.788 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:14:49 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[93820]: 2024-03-28T18:14:49.789 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:14:49 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[93820]: *** Caught signal (Aborted) ** 2024-03-28T18:14:49.789 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:14:49 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[93820]: in thread 7fcf0b400700 thread_name:ms_dispatch 2024-03-28T18:14:49.789 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:14:49 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[93820]: ceph version 19.0.0-2145-g3be9a5c9 (3be9a5c9bc793e11e2800a8c0c696e8b46742033) squid (dev) 2024-03-28T18:14:49.789 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:14:49 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[93820]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7fcf15a93d20] 2024-03-28T18:14:49.790 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:14:49 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[93820]: 2: gsignal() 2024-03-28T18:14:49.790 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:14:49 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[93820]: 3: abort() 2024-03-28T18:14:49.790 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:14:49 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[93820]: 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x18f) [0x7fcf16d0c4fb] 2024-03-28T18:14:49.790 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:14:49 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[93820]: 5: /usr/lib64/ceph/libceph-common.so.2(+0x2d2667) [0x7fcf16d0c667] 2024-03-28T18:14:49.790 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:14:49 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[93820]: 6: (Locker::set_xlocks_done(MutationImpl*, bool)+0x493) [0x55fa93613223] 2024-03-28T18:14:49.791 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:14:49 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[93820]: 7: (Server::early_reply(boost::intrusive_ptr const&, CInode*, CDentry*)+0x3e6) [0x55fa9348d5d6] 2024-03-28T18:14:49.791 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:14:49 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[93820]: 8: (Server::journal_and_reply(boost::intrusive_ptr const&, CInode*, CDentry*, LogEvent*, MDSLogContextBase*)+0xc1) [0x55fa9348dda1] 2024-03-28T18:14:49.791 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:14:49 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[93820]: 9: (Server::handle_client_mkdir(boost::intrusive_ptr const&)+0x5ba) [0x55fa934c36aa] 2024-03-28T18:14:49.791 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:14:49 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[93820]: 10: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0x733) [0x55fa934e0233] 2024-03-28T18:14:49.791 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:14:49 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[93820]: 11: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x55fa934e2197] 2024-03-28T18:14:49.791 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:14:49 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[93820]: 12: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x55fa933e111b] 2024-03-28T18:14:49.791 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:14:49 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[93820]: 13: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x55fa933e16bc] 2024-03-28T18:14:49.791 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:14:49 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[93820]: 14: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bf) [0x55fa933cb1ff] 2024-03-28T18:14:49.791 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:14:49 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[93820]: 15: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x478) [0x7fcf16f9e488] 2024-03-28T18:14:49.791 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:14:49 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[93820]: 16: (DispatchQueue::entry()+0x51f) [0x7fcf16f9b60f] 2024-03-28T18:14:49.791 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:14:49 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[93820]: 17: (DispatchQueue::DispatchThread::entry()+0x11) [0x7fcf1706ee81] 2024-03-28T18:14:49.791 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:14:49 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[93820]: 18: /lib64/libpthread.so.0(+0x81ca) [0x7fcf15a891ca] 2024-03-28T18:14:49.791 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:14:49 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[93820]: 19: clone() 2024-03-28T18:14:49.791 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:14:49 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[93820]: 2024-03-28T18:14:49.364+0000 7fcf0b400700 -1 *** Caught signal (Aborted) ** 2024-03-28T18:14:49.791 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:14:49 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[93820]: in thread 7fcf0b400700 thread_name:ms_dispatch 2024-03-28T18:14:49.792 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:14:49 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[93820]: 2024-03-28T18:14:49.792 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:14:49 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[93820]: ceph version 19.0.0-2145-g3be9a5c9 (3be9a5c9bc793e11e2800a8c0c696e8b46742033) squid (dev) 2024-03-28T18:14:49.792 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:14:49 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[93820]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7fcf15a93d20] 2024-03-28T18:14:49.792 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:14:49 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[93820]: 2: gsignal() 2024-03-28T18:14:49.792 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:14:49 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[93820]: 3: abort() 2024-03-28T18:14:49.792 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:14:49 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[93820]: 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x18f) [0x7fcf16d0c4fb] 2024-03-28T18:14:49.792 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:14:49 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[93820]: 5: /usr/lib64/ceph/libceph-common.so.2(+0x2d2667) [0x7fcf16d0c667] 2024-03-28T18:14:49.792 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:14:49 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[93820]: 6: (Locker::set_xlocks_done(MutationImpl*, bool)+0x493) [0x55fa93613223] 2024-03-28T18:14:49.792 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:14:49 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[93820]: 7: (Server::early_reply(boost::intrusive_ptr const&, CInode*, CDentry*)+0x3e6) [0x55fa9348d5d6] 2024-03-28T18:14:49.792 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:14:49 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[93820]: 8: (Server::journal_and_reply(boost::intrusive_ptr const&, CInode*, CDentry*, LogEvent*, MDSLogContextBase*)+0xc1) [0x55fa9348dda1] 2024-03-28T18:14:49.792 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:14:49 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[93820]: 9: (Server::handle_client_mkdir(boost::intrusive_ptr const&)+0x5ba) [0x55fa934c36aa] 2024-03-28T18:14:49.793 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:14:49 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[93820]: 10: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0x733) [0x55fa934e0233] 2024-03-28T18:14:49.793 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:14:49 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[93820]: 11: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x55fa934e2197] 2024-03-28T18:14:49.793 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:14:49 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[93820]: 12: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x55fa933e111b] 2024-03-28T18:14:49.793 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:14:49 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[93820]: 13: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x55fa933e16bc] 2024-03-28T18:14:49.793 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:14:49 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[93820]: 14: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bf) [0x55fa933cb1ff] 2024-03-28T18:14:49.793 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:14:49 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[93820]: 15: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x478) [0x7fcf16f9e488] 2024-03-28T18:14:49.793 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:14:49 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[93820]: 16: (DispatchQueue::entry()+0x51f) [0x7fcf16f9b60f] 2024-03-28T18:14:49.793 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:14:49 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[93820]: 17: (DispatchQueue::DispatchThread::entry()+0x11) [0x7fcf1706ee81] 2024-03-28T18:14:49.793 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:14:49 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[93820]: 18: /lib64/libpthread.so.0(+0x81ca) [0x7fcf15a891ca] 2024-03-28T18:14:49.793 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:14:49 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[93820]: 19: clone() 2024-03-28T18:14:49.793 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:14:49 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[93820]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-03-28T18:14:49.793 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:14:49 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[93820]: 2024-03-28T18:14:49.793 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:14:49 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[93820]: -1> 2024-03-28T18:14:49.361+0000 7fcf0b400700 -1 /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-2145-g3be9a5c9/rpm/el8/BUILD/ceph-19.0.0-2145-g3be9a5c9/src/mds/Locker.cc: In function 'void Locker::set_xlocks_done(MutationImpl*, bool)' thread 7fcf0b400700 time 2024-03-28T18:14:49.360578+0000 2024-03-28T18:14:49.793 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:14:49 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[93820]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-2145-g3be9a5c9/rpm/el8/BUILD/ceph-19.0.0-2145-g3be9a5c9/src/mds/Locker.cc: 626: FAILED ceph_assert(obj->is_auth()) 2024-03-28T18:14:49.794 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:14:49 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[93820]: 2024-03-28T18:14:49.794 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:14:49 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[93820]: ceph version 19.0.0-2145-g3be9a5c9 (3be9a5c9bc793e11e2800a8c0c696e8b46742033) squid (dev) 2024-03-28T18:14:49.794 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:14:49 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[93820]: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x135) [0x7fcf16d0c4a1] 2024-03-28T18:14:49.794 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:14:49 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[93820]: 2: /usr/lib64/ceph/libceph-common.so.2(+0x2d2667) [0x7fcf16d0c667] 2024-03-28T18:14:49.794 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:14:49 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[93820]: 3: (Locker::set_xlocks_done(MutationImpl*, bool)+0x493) [0x55fa93613223] 2024-03-28T18:14:49.794 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:14:49 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[93820]: 4: (Server::early_reply(boost::intrusive_ptr const&, CInode*, CDentry*)+0x3e6) [0x55fa9348d5d6] 2024-03-28T18:14:49.794 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:14:49 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[93820]: 5: (Server::journal_and_reply(boost::intrusive_ptr const&, CInode*, CDentry*, LogEvent*, MDSLogContextBase*)+0xc1) [0x55fa9348dda1] 2024-03-28T18:14:49.794 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:14:49 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[93820]: 6: (Server::handle_client_mkdir(boost::intrusive_ptr const&)+0x5ba) [0x55fa934c36aa] 2024-03-28T18:14:49.794 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:14:49 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[93820]: 7: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0x733) [0x55fa934e0233] 2024-03-28T18:14:49.795 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:14:49 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[93820]: 8: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x55fa934e2197] 2024-03-28T18:14:49.795 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:14:49 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[93820]: 9: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x55fa933e111b] 2024-03-28T18:14:49.795 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:14:49 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[93820]: 10: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x55fa933e16bc] 2024-03-28T18:14:49.795 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:14:49 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[93820]: 11: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bf) [0x55fa933cb1ff] 2024-03-28T18:14:49.795 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:14:49 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[93820]: 12: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x478) [0x7fcf16f9e488] 2024-03-28T18:14:49.795 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:14:49 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[93820]: 13: (DispatchQueue::entry()+0x51f) [0x7fcf16f9b60f] 2024-03-28T18:14:49.795 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:14:49 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[93820]: 14: (DispatchQueue::DispatchThread::entry()+0x11) [0x7fcf1706ee81] 2024-03-28T18:14:49.795 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:14:49 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[93820]: 15: /lib64/libpthread.so.0(+0x81ca) [0x7fcf15a891ca] 2024-03-28T18:14:49.795 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:14:49 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[93820]: 16: clone() 2024-03-28T18:14:49.795 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:14:49 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[93820]: 2024-03-28T18:14:49.795 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:14:49 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[93820]: 0> 2024-03-28T18:14:49.364+0000 7fcf0b400700 -1 *** Caught signal (Aborted) ** 2024-03-28T18:14:49.795 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:14:49 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[93820]: in thread 7fcf0b400700 thread_name:ms_dispatch 2024-03-28T18:14:49.795 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:14:49 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[93820]: 2024-03-28T18:14:49.795 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:14:49 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[93820]: ceph version 19.0.0-2145-g3be9a5c9 (3be9a5c9bc793e11e2800a8c0c696e8b46742033) squid (dev) 2024-03-28T18:14:49.795 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:14:49 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[93820]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7fcf15a93d20] 2024-03-28T18:14:49.796 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:14:49 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[93820]: 2: gsignal() 2024-03-28T18:14:49.796 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:14:49 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[93820]: 3: abort() 2024-03-28T18:14:49.796 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:14:49 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[93820]: 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x18f) [0x7fcf16d0c4fb] 2024-03-28T18:14:49.796 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:14:49 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[93820]: 5: /usr/lib64/ceph/libceph-common.so.2(+0x2d2667) [0x7fcf16d0c667] 2024-03-28T18:14:49.796 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:14:49 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[93820]: 6: (Locker::set_xlocks_done(MutationImpl*, bool)+0x493) [0x55fa93613223] 2024-03-28T18:14:49.796 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:14:49 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[93820]: 7: (Server::early_reply(boost::intrusive_ptr const&, CInode*, CDentry*)+0x3e6) [0x55fa9348d5d6] 2024-03-28T18:14:49.796 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:14:49 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[93820]: 8: (Server::journal_and_reply(boost::intrusive_ptr const&, CInode*, CDentry*, LogEvent*, MDSLogContextBase*)+0xc1) [0x55fa9348dda1] 2024-03-28T18:14:49.796 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:14:49 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[93820]: 9: (Server::handle_client_mkdir(boost::intrusive_ptr const&)+0x5ba) [0x55fa934c36aa] 2024-03-28T18:14:49.797 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:14:49 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[93820]: 10: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0x733) [0x55fa934e0233] 2024-03-28T18:14:49.797 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:14:49 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[93820]: 11: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x55fa934e2197] 2024-03-28T18:14:49.797 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:14:49 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[93820]: 12: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x55fa933e111b] 2024-03-28T18:14:49.797 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:14:49 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[93820]: 13: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x55fa933e16bc] 2024-03-28T18:14:49.797 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:14:49 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[93820]: 14: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bf) [0x55fa933cb1ff] 2024-03-28T18:14:49.797 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:14:49 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[93820]: 15: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x478) [0x7fcf16f9e488] 2024-03-28T18:14:49.797 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:14:49 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[93820]: 16: (DispatchQueue::entry()+0x51f) [0x7fcf16f9b60f] 2024-03-28T18:14:49.797 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:14:49 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[93820]: 17: (DispatchQueue::DispatchThread::entry()+0x11) [0x7fcf1706ee81] 2024-03-28T18:14:49.797 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:14:49 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[93820]: 18: /lib64/libpthread.so.0(+0x81ca) [0x7fcf15a891ca] 2024-03-28T18:14:49.797 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:14:49 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[93820]: 19: clone() 2024-03-28T18:14:49.797 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:14:49 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[93820]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-03-28T18:14:49.797 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:14:49 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[93820]: 2024-03-28T18:14:49.797 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:14:49 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[93820]: -1> 2024-03-28T18:14:49.361+0000 7fcf0b400700 -1 /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-2145-g3be9a5c9/rpm/el8/BUILD/ceph-19.0.0-2145-g3be9a5c9/src/mds/Locker.cc: In function 'void Locker::set_xlocks_done(MutationImpl*, bool)' thread 7fcf0b400700 time 2024-03-28T18:14:49.360578+0000 2024-03-28T18:14:49.798 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:14:49 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[93820]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-2145-g3be9a5c9/rpm/el8/BUILD/ceph-19.0.0-2145-g3be9a5c9/src/mds/Locker.cc: 626: FAILED ceph_assert(obj->is_auth()) 2024-03-28T18:14:49.798 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:14:49 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[93820]: 2024-03-28T18:14:49.798 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:14:49 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[93820]: ceph version 19.0.0-2145-g3be9a5c9 (3be9a5c9bc793e11e2800a8c0c696e8b46742033) squid (dev) 2024-03-28T18:14:49.798 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:14:49 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[93820]: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x135) [0x7fcf16d0c4a1] 2024-03-28T18:14:49.798 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:14:49 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[93820]: 2: /usr/lib64/ceph/libceph-common.so.2(+0x2d2667) [0x7fcf16d0c667] 2024-03-28T18:14:49.798 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:14:49 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[93820]: 3: (Locker::set_xlocks_done(MutationImpl*, bool)+0x493) [0x55fa93613223] 2024-03-28T18:14:49.798 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:14:49 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[93820]: 4: (Server::early_reply(boost::intrusive_ptr const&, CInode*, CDentry*)+0x3e6) [0x55fa9348d5d6] 2024-03-28T18:14:49.798 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:14:49 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[93820]: 5: (Server::journal_and_reply(boost::intrusive_ptr const&, CInode*, CDentry*, LogEvent*, MDSLogContextBase*)+0xc1) [0x55fa9348dda1] 2024-03-28T18:14:49.798 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:14:49 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[93820]: 6: (Server::handle_client_mkdir(boost::intrusive_ptr const&)+0x5ba) [0x55fa934c36aa] 2024-03-28T18:14:49.798 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:14:49 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[93820]: 7: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0x733) [0x55fa934e0233] 2024-03-28T18:14:49.798 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:14:49 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[93820]: 8: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x55fa934e2197] 2024-03-28T18:14:49.798 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:14:49 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[93820]: 9: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x55fa933e111b] 2024-03-28T18:14:49.798 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:14:49 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[93820]: 10: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x55fa933e16bc] 2024-03-28T18:14:49.798 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:14:49 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[93820]: 11: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bf) [0x55fa933cb1ff] 2024-03-28T18:14:49.799 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:14:49 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[93820]: 12: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x478) [0x7fcf16f9e488] 2024-03-28T18:14:49.799 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:14:49 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[93820]: 13: (DispatchQueue::entry()+0x51f) [0x7fcf16f9b60f] 2024-03-28T18:14:49.799 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:14:49 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[93820]: 14: (DispatchQueue::DispatchThread::entry()+0x11) [0x7fcf1706ee81] 2024-03-28T18:14:49.799 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:14:49 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[93820]: 15: /lib64/libpthread.so.0(+0x81ca) [0x7fcf15a891ca] 2024-03-28T18:14:49.799 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:14:49 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[93820]: 16: clone() 2024-03-28T18:14:49.799 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:14:49 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[93820]: 2024-03-28T18:14:49.799 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:14:49 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[93820]: 0> 2024-03-28T18:14:49.364+0000 7fcf0b400700 -1 *** Caught signal (Aborted) ** 2024-03-28T18:14:49.799 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:14:49 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[93820]: in thread 7fcf0b400700 thread_name:ms_dispatch 2024-03-28T18:14:49.799 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:14:49 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[93820]: 2024-03-28T18:14:49.799 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:14:49 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[93820]: ceph version 19.0.0-2145-g3be9a5c9 (3be9a5c9bc793e11e2800a8c0c696e8b46742033) squid (dev) 2024-03-28T18:14:49.799 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:14:49 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[93820]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7fcf15a93d20] 2024-03-28T18:14:49.799 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:14:49 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[93820]: 2: gsignal() 2024-03-28T18:14:49.799 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:14:49 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[93820]: 3: abort() 2024-03-28T18:14:49.799 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:14:49 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[93820]: 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x18f) [0x7fcf16d0c4fb] 2024-03-28T18:14:49.799 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:14:49 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[93820]: 5: /usr/lib64/ceph/libceph-common.so.2(+0x2d2667) [0x7fcf16d0c667] 2024-03-28T18:14:49.800 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:14:49 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[93820]: 6: (Locker::set_xlocks_done(MutationImpl*, bool)+0x493) [0x55fa93613223] 2024-03-28T18:14:49.800 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:14:49 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[93820]: 7: (Server::early_reply(boost::intrusive_ptr const&, CInode*, CDentry*)+0x3e6) [0x55fa9348d5d6] 2024-03-28T18:14:49.800 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:14:49 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[93820]: 8: (Server::journal_and_reply(boost::intrusive_ptr const&, CInode*, CDentry*, LogEvent*, MDSLogContextBase*)+0xc1) [0x55fa9348dda1] 2024-03-28T18:14:49.800 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:14:49 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[93820]: 9: (Server::handle_client_mkdir(boost::intrusive_ptr const&)+0x5ba) [0x55fa934c36aa] 2024-03-28T18:14:49.800 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:14:49 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[93820]: 10: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0x733) [0x55fa934e0233] 2024-03-28T18:14:49.800 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:14:49 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[93820]: 11: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x55fa934e2197] 2024-03-28T18:14:49.800 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:14:49 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[93820]: 12: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x55fa933e111b] 2024-03-28T18:14:49.800 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:14:49 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[93820]: 13: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x55fa933e16bc] 2024-03-28T18:14:49.800 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:14:49 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[93820]: 14: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bf) [0x55fa933cb1ff] 2024-03-28T18:14:49.800 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:14:49 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[93820]: 15: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x478) [0x7fcf16f9e488] 2024-03-28T18:14:49.801 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:14:49 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[93820]: 16: (DispatchQueue::entry()+0x51f) [0x7fcf16f9b60f] 2024-03-28T18:14:49.801 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:14:49 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[93820]: 17: (DispatchQueue::DispatchThread::entry()+0x11) [0x7fcf1706ee81] 2024-03-28T18:14:49.801 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:14:49 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[93820]: 18: /lib64/libpthread.so.0(+0x81ca) [0x7fcf15a891ca] 2024-03-28T18:14:49.801 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:14:49 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[93820]: 19: clone() 2024-03-28T18:14:49.801 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:14:49 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[93820]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-03-28T18:14:49.801 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:14:49 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[93820]: 2024-03-28T18:14:49.801 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:14:49 smithi022 podman[94073]: 2024-03-28 18:14:49.495962287 +0000 UTC m=+0.039105134 container died 1ef25f2e549c69a5228280f441b068b1bef43e7dde03ef23025a8fcfde8af15b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.build-date=20240319, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-03-28T18:14:50.532 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:14:50 smithi022 podman[94073]: 2024-03-28 18:14:50.173166601 +0000 UTC m=+0.716309463 container remove 1ef25f2e549c69a5228280f441b068b1bef43e7dde03ef23025a8fcfde8af15b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d, ceph=True, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.build-date=20240319, org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-28T18:14:50.532 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:14:50 smithi022 systemd[1]: ceph-fd220748-ed2b-11ee-95d0-87774f69a715@mds.d.service: Main process exited, code=exited, status=134/n/a 2024-03-28T18:14:50.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:14:50 smithi022 ceph-mon[31985]: pgmap v239: 97 pgs: 97 active+clean; 117 MiB data, 793 MiB used, 1.0 TiB / 1.0 TiB avail; 7.6 KiB/s rd, 639 B/s wr, 13 op/s 2024-03-28T18:14:50.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:14:50 smithi022 ceph-mon[31985]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-03-28T18:14:50.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:14:50 smithi022 ceph-mon[31985]: Cluster is now healthy 2024-03-28T18:14:50.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:14:50 smithi022 ceph-mon[31985]: mds.? [v2:172.21.15.22:6836/2984059496,v1:172.21.15.22:6837/2984059496] up:active 2024-03-28T18:14:50.533 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:14:50 smithi022 ceph-mon[31985]: fsmap cephfs:5 4 up:standby-replay 1 up:standby 5 up:active 2024-03-28T18:14:50.533 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:14:50 smithi022 ceph-mon[31985]: fsmap cephfs:5 5 up:standby-replay 5 up:active 2024-03-28T18:14:50.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:14:50 smithi134 ceph-mon[35037]: pgmap v239: 97 pgs: 97 active+clean; 117 MiB data, 793 MiB used, 1.0 TiB / 1.0 TiB avail; 7.6 KiB/s rd, 639 B/s wr, 13 op/s 2024-03-28T18:14:50.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:14:50 smithi134 ceph-mon[35037]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-03-28T18:14:50.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:14:50 smithi134 ceph-mon[35037]: Cluster is now healthy 2024-03-28T18:14:50.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:14:50 smithi134 ceph-mon[35037]: mds.? [v2:172.21.15.22:6836/2984059496,v1:172.21.15.22:6837/2984059496] up:active 2024-03-28T18:14:50.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:14:50 smithi134 ceph-mon[35037]: fsmap cephfs:5 4 up:standby-replay 1 up:standby 5 up:active 2024-03-28T18:14:50.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:14:50 smithi134 ceph-mon[35037]: fsmap cephfs:5 5 up:standby-replay 5 up:active 2024-03-28T18:14:50.683 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:14:50 smithi079 ceph-mon[35312]: pgmap v239: 97 pgs: 97 active+clean; 117 MiB data, 793 MiB used, 1.0 TiB / 1.0 TiB avail; 7.6 KiB/s rd, 639 B/s wr, 13 op/s 2024-03-28T18:14:50.683 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:14:50 smithi079 ceph-mon[35312]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-03-28T18:14:50.683 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:14:50 smithi079 ceph-mon[35312]: Cluster is now healthy 2024-03-28T18:14:50.683 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:14:50 smithi079 ceph-mon[35312]: mds.? [v2:172.21.15.22:6836/2984059496,v1:172.21.15.22:6837/2984059496] up:active 2024-03-28T18:14:50.683 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:14:50 smithi079 ceph-mon[35312]: fsmap cephfs:5 4 up:standby-replay 1 up:standby 5 up:active 2024-03-28T18:14:50.683 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:14:50 smithi079 ceph-mon[35312]: fsmap cephfs:5 5 up:standby-replay 5 up:active 2024-03-28T18:14:51.031 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:14:50 smithi022 systemd[1]: ceph-fd220748-ed2b-11ee-95d0-87774f69a715@mds.d.service: Failed with result 'exit-code'. 2024-03-28T18:14:51.032 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:14:50 smithi022 systemd[1]: ceph-fd220748-ed2b-11ee-95d0-87774f69a715@mds.d.service: Consumed 1.960s CPU time. 2024-03-28T18:14:52.674 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:14:52 smithi134 ceph-mon[35037]: pgmap v240: 97 pgs: 97 active+clean; 117 MiB data, 793 MiB used, 1.0 TiB / 1.0 TiB avail; 7.4 KiB/s rd, 620 B/s wr, 12 op/s 2024-03-28T18:14:52.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:14:52 smithi079 ceph-mon[35312]: pgmap v240: 97 pgs: 97 active+clean; 117 MiB data, 793 MiB used, 1.0 TiB / 1.0 TiB avail; 7.4 KiB/s rd, 620 B/s wr, 12 op/s 2024-03-28T18:14:52.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:14:52 smithi022 ceph-mon[31985]: pgmap v240: 97 pgs: 97 active+clean; 117 MiB data, 793 MiB used, 1.0 TiB / 1.0 TiB avail; 7.4 KiB/s rd, 620 B/s wr, 12 op/s 2024-03-28T18:14:54.674 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:14:54 smithi134 ceph-mon[35037]: pgmap v241: 97 pgs: 97 active+clean; 117 MiB data, 793 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 511 B/s wr, 10 op/s 2024-03-28T18:14:54.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:14:54 smithi134 ceph-mon[35037]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T18:14:54.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:14:54 smithi079 ceph-mon[35312]: pgmap v241: 97 pgs: 97 active+clean; 117 MiB data, 793 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 511 B/s wr, 10 op/s 2024-03-28T18:14:54.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:14:54 smithi079 ceph-mon[35312]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T18:14:54.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:14:54 smithi022 ceph-mon[31985]: pgmap v241: 97 pgs: 97 active+clean; 117 MiB data, 793 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 511 B/s wr, 10 op/s 2024-03-28T18:14:54.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:14:54 smithi022 ceph-mon[31985]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T18:14:55.674 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:14:55 smithi134 ceph-mon[35037]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T18:14:55.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:14:55 smithi134 ceph-mon[35037]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T18:14:55.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:14:55 smithi134 ceph-mon[35037]: from='mgr.15522 ' entity='mgr.x' 2024-03-28T18:14:55.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:14:55 smithi079 ceph-mon[35312]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T18:14:55.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:14:55 smithi079 ceph-mon[35312]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T18:14:55.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:14:55 smithi079 ceph-mon[35312]: from='mgr.15522 ' entity='mgr.x' 2024-03-28T18:14:55.781 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:14:55 smithi022 ceph-mon[31985]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T18:14:55.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:14:55 smithi022 ceph-mon[31985]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T18:14:55.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:14:55 smithi022 ceph-mon[31985]: from='mgr.15522 ' entity='mgr.x' 2024-03-28T18:14:56.674 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:14:56 smithi134 ceph-mon[35037]: pgmap v242: 97 pgs: 97 active+clean; 117 MiB data, 793 MiB used, 1.0 TiB / 1.0 TiB avail; 109 KiB/s rd, 4.7 KiB/s wr, 25 op/s 2024-03-28T18:14:56.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:14:56 smithi079 ceph-mon[35312]: pgmap v242: 97 pgs: 97 active+clean; 117 MiB data, 793 MiB used, 1.0 TiB / 1.0 TiB avail; 109 KiB/s rd, 4.7 KiB/s wr, 25 op/s 2024-03-28T18:14:56.684 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:14:56 smithi022 ceph-mon[31985]: pgmap v242: 97 pgs: 97 active+clean; 117 MiB data, 793 MiB used, 1.0 TiB / 1.0 TiB avail; 109 KiB/s rd, 4.7 KiB/s wr, 25 op/s 2024-03-28T18:14:58.674 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:14:58 smithi134 ceph-mon[35037]: pgmap v243: 97 pgs: 97 active+clean; 117 MiB data, 793 MiB used, 1.0 TiB / 1.0 TiB avail; 92 KiB/s rd, 3.9 KiB/s wr, 21 op/s 2024-03-28T18:14:58.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:14:58 smithi079 ceph-mon[35312]: pgmap v243: 97 pgs: 97 active+clean; 117 MiB data, 793 MiB used, 1.0 TiB / 1.0 TiB avail; 92 KiB/s rd, 3.9 KiB/s wr, 21 op/s 2024-03-28T18:14:58.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:14:58 smithi022 ceph-mon[31985]: pgmap v243: 97 pgs: 97 active+clean; 117 MiB data, 793 MiB used, 1.0 TiB / 1.0 TiB avail; 92 KiB/s rd, 3.9 KiB/s wr, 21 op/s 2024-03-28T18:15:00.674 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:15:00 smithi134 ceph-mon[35037]: pgmap v244: 97 pgs: 97 active+clean; 117 MiB data, 793 MiB used, 1.0 TiB / 1.0 TiB avail; 93 KiB/s rd, 3.9 KiB/s wr, 25 op/s 2024-03-28T18:15:00.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:15:00 smithi079 ceph-mon[35312]: pgmap v244: 97 pgs: 97 active+clean; 117 MiB data, 793 MiB used, 1.0 TiB / 1.0 TiB avail; 93 KiB/s rd, 3.9 KiB/s wr, 25 op/s 2024-03-28T18:15:00.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:15:00 smithi022 ceph-mon[31985]: pgmap v244: 97 pgs: 97 active+clean; 117 MiB data, 793 MiB used, 1.0 TiB / 1.0 TiB avail; 93 KiB/s rd, 3.9 KiB/s wr, 25 op/s 2024-03-28T18:15:01.196 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:15:00 smithi022 systemd[1]: ceph-fd220748-ed2b-11ee-95d0-87774f69a715@mds.d.service: Scheduled restart job, restart counter is at 3. 2024-03-28T18:15:01.196 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:15:00 smithi022 systemd[1]: Stopped Ceph mds.d for fd220748-ed2b-11ee-95d0-87774f69a715. 2024-03-28T18:15:01.196 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:15:00 smithi022 systemd[1]: ceph-fd220748-ed2b-11ee-95d0-87774f69a715@mds.d.service: Consumed 1.960s CPU time. 2024-03-28T18:15:01.196 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:15:00 smithi022 systemd[1]: Starting Ceph mds.d for fd220748-ed2b-11ee-95d0-87774f69a715... 2024-03-28T18:15:01.459 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:15:01 smithi022 podman[94309]: 2024-03-28 18:15:01.102908102 +0000 UTC m=+0.038922232 image pull c664afe56bb01db4f9afa9532c20598e201e664f2e9ba2be50db0d52674a26d2 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc 2024-03-28T18:15:01.459 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:15:01 smithi022 podman[94309]: 2024-03-28 18:15:01.275699356 +0000 UTC m=+0.211713536 container create 60c352fd33798abe41594e3037af250e16cd824685e11b276d9966c3c6ee9fb8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d, org.label-schema.vendor=CentOS, ceph=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, org.label-schema.build-date=20240319, CEPH_POINT_RELEASE=) 2024-03-28T18:15:02.032 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:15:01 smithi022 podman[94309]: 2024-03-28 18:15:01.57845751 +0000 UTC m=+0.514471684 container init 60c352fd33798abe41594e3037af250e16cd824685e11b276d9966c3c6ee9fb8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, ceph=True, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.build-date=20240319, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2) 2024-03-28T18:15:02.033 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:15:01 smithi022 podman[94309]: 2024-03-28 18:15:01.58975524 +0000 UTC m=+0.525769375 container start 60c352fd33798abe41594e3037af250e16cd824685e11b276d9966c3c6ee9fb8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d, org.label-schema.build-date=20240319, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.vendor=CentOS, ceph=True, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD) 2024-03-28T18:15:02.033 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:15:01 smithi022 ceph-mds[94327]: -- 172.21.15.22:0/1755843897 <== mon.0 v2:172.21.15.22:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x56304636fa40 con 0x563046342800 2024-03-28T18:15:02.033 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:15:01 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[94323]: starting mds.d at 2024-03-28T18:15:02.033 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:15:01 smithi022 bash[94309]: 60c352fd33798abe41594e3037af250e16cd824685e11b276d9966c3c6ee9fb8 2024-03-28T18:15:02.033 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:15:01 smithi022 systemd[1]: Started Ceph mds.d for fd220748-ed2b-11ee-95d0-87774f69a715. 2024-03-28T18:15:02.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:15:02 smithi022 ceph-mon[31985]: pgmap v245: 97 pgs: 97 active+clean; 117 MiB data, 793 MiB used, 1.0 TiB / 1.0 TiB avail; 90 KiB/s rd, 3.6 KiB/s wr, 21 op/s 2024-03-28T18:15:02.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:15:02 smithi022 ceph-mon[31985]: daemon mds.d restarted 2024-03-28T18:15:02.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:15:02 smithi022 ceph-mon[31985]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-03-28T18:15:02.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:15:02 smithi022 ceph-mon[31985]: osdmap e112: 12 total, 12 up, 12 in 2024-03-28T18:15:02.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:15:02 smithi022 ceph-mon[31985]: mds.? [v2:172.21.15.22:6836/488832747,v1:172.21.15.22:6837/488832747] up:boot 2024-03-28T18:15:02.533 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:15:02 smithi022 ceph-mon[31985]: Standby daemon mds.j assigned to filesystem cephfs as rank 1 2024-03-28T18:15:02.533 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:15:02 smithi022 ceph-mon[31985]: fsmap cephfs:4/5 {0=b=up:active,2=e=up:active,3=k=up:active,4=i=up:active} 5 up:standby-replay 1 up:standby, 1 failed 2024-03-28T18:15:02.533 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:15:02 smithi022 ceph-mon[31985]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-03-28T18:15:02.533 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:15:02 smithi022 ceph-mon[31985]: fsmap cephfs:5/5 4 up:standby-replay 1 up:standby 1 up:replay 4 up:active 2024-03-28T18:15:02.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:15:02 smithi134 ceph-mon[35037]: pgmap v245: 97 pgs: 97 active+clean; 117 MiB data, 793 MiB used, 1.0 TiB / 1.0 TiB avail; 90 KiB/s rd, 3.6 KiB/s wr, 21 op/s 2024-03-28T18:15:02.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:15:02 smithi134 ceph-mon[35037]: daemon mds.d restarted 2024-03-28T18:15:02.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:15:02 smithi134 ceph-mon[35037]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-03-28T18:15:02.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:15:02 smithi134 ceph-mon[35037]: osdmap e112: 12 total, 12 up, 12 in 2024-03-28T18:15:02.676 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:15:02 smithi134 ceph-mon[35037]: mds.? [v2:172.21.15.22:6836/488832747,v1:172.21.15.22:6837/488832747] up:boot 2024-03-28T18:15:02.676 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:15:02 smithi134 ceph-mon[35037]: Standby daemon mds.j assigned to filesystem cephfs as rank 1 2024-03-28T18:15:02.676 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:15:02 smithi134 ceph-mon[35037]: fsmap cephfs:4/5 {0=b=up:active,2=e=up:active,3=k=up:active,4=i=up:active} 5 up:standby-replay 1 up:standby, 1 failed 2024-03-28T18:15:02.676 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:15:02 smithi134 ceph-mon[35037]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-03-28T18:15:02.676 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:15:02 smithi134 ceph-mon[35037]: fsmap cephfs:5/5 4 up:standby-replay 1 up:standby 1 up:replay 4 up:active 2024-03-28T18:15:02.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:15:02 smithi079 ceph-mon[35312]: pgmap v245: 97 pgs: 97 active+clean; 117 MiB data, 793 MiB used, 1.0 TiB / 1.0 TiB avail; 90 KiB/s rd, 3.6 KiB/s wr, 21 op/s 2024-03-28T18:15:02.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:15:02 smithi079 ceph-mon[35312]: daemon mds.d restarted 2024-03-28T18:15:02.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:15:02 smithi079 ceph-mon[35312]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-03-28T18:15:02.683 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:15:02 smithi079 ceph-mon[35312]: osdmap e112: 12 total, 12 up, 12 in 2024-03-28T18:15:02.683 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:15:02 smithi079 ceph-mon[35312]: mds.? [v2:172.21.15.22:6836/488832747,v1:172.21.15.22:6837/488832747] up:boot 2024-03-28T18:15:02.683 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:15:02 smithi079 ceph-mon[35312]: Standby daemon mds.j assigned to filesystem cephfs as rank 1 2024-03-28T18:15:02.683 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:15:02 smithi079 ceph-mon[35312]: fsmap cephfs:4/5 {0=b=up:active,2=e=up:active,3=k=up:active,4=i=up:active} 5 up:standby-replay 1 up:standby, 1 failed 2024-03-28T18:15:02.683 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:15:02 smithi079 ceph-mon[35312]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-03-28T18:15:02.683 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:15:02 smithi079 ceph-mon[35312]: fsmap cephfs:5/5 4 up:standby-replay 1 up:standby 1 up:replay 4 up:active 2024-03-28T18:15:04.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:15:04 smithi134 ceph-mon[35037]: pgmap v247: 97 pgs: 97 active+clean; 117 MiB data, 793 MiB used, 1.0 TiB / 1.0 TiB avail; 108 KiB/s rd, 4.3 KiB/s wr, 25 op/s 2024-03-28T18:15:04.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:15:04 smithi134 ceph-mon[35037]: mds.? [v2:172.21.15.22:6834/535745119,v1:172.21.15.22:6835/535745119] up:resolve 2024-03-28T18:15:04.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:15:04 smithi134 ceph-mon[35037]: fsmap cephfs:5/5 4 up:standby-replay 1 up:standby 1 up:resolve 4 up:active 2024-03-28T18:15:04.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:15:04 smithi079 ceph-mon[35312]: pgmap v247: 97 pgs: 97 active+clean; 117 MiB data, 793 MiB used, 1.0 TiB / 1.0 TiB avail; 108 KiB/s rd, 4.3 KiB/s wr, 25 op/s 2024-03-28T18:15:04.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:15:04 smithi079 ceph-mon[35312]: mds.? [v2:172.21.15.22:6834/535745119,v1:172.21.15.22:6835/535745119] up:resolve 2024-03-28T18:15:04.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:15:04 smithi079 ceph-mon[35312]: fsmap cephfs:5/5 4 up:standby-replay 1 up:standby 1 up:resolve 4 up:active 2024-03-28T18:15:04.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:15:04 smithi022 ceph-mon[31985]: pgmap v247: 97 pgs: 97 active+clean; 117 MiB data, 793 MiB used, 1.0 TiB / 1.0 TiB avail; 108 KiB/s rd, 4.3 KiB/s wr, 25 op/s 2024-03-28T18:15:04.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:15:04 smithi022 ceph-mon[31985]: mds.? [v2:172.21.15.22:6834/535745119,v1:172.21.15.22:6835/535745119] up:resolve 2024-03-28T18:15:04.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:15:04 smithi022 ceph-mon[31985]: fsmap cephfs:5/5 4 up:standby-replay 1 up:standby 1 up:resolve 4 up:active 2024-03-28T18:15:05.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:15:05 smithi134 ceph-mon[35037]: mds.? [v2:172.21.15.22:6834/535745119,v1:172.21.15.22:6835/535745119] up:reconnect 2024-03-28T18:15:05.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:15:05 smithi134 ceph-mon[35037]: fsmap cephfs:5/5 4 up:standby-replay 1 up:standby 1 up:reconnect 4 up:active 2024-03-28T18:15:05.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:15:05 smithi134 ceph-mon[35037]: reconnect by client.35222 172.21.15.22:0/3729116781 after 0.00699991 2024-03-28T18:15:05.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:15:05 smithi079 ceph-mon[35312]: mds.? [v2:172.21.15.22:6834/535745119,v1:172.21.15.22:6835/535745119] up:reconnect 2024-03-28T18:15:05.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:15:05 smithi079 ceph-mon[35312]: fsmap cephfs:5/5 4 up:standby-replay 1 up:standby 1 up:reconnect 4 up:active 2024-03-28T18:15:05.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:15:05 smithi079 ceph-mon[35312]: reconnect by client.35222 172.21.15.22:0/3729116781 after 0.00699991 2024-03-28T18:15:05.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:15:05 smithi022 ceph-mon[31985]: mds.? [v2:172.21.15.22:6834/535745119,v1:172.21.15.22:6835/535745119] up:reconnect 2024-03-28T18:15:05.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:15:05 smithi022 ceph-mon[31985]: fsmap cephfs:5/5 4 up:standby-replay 1 up:standby 1 up:reconnect 4 up:active 2024-03-28T18:15:05.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:15:05 smithi022 ceph-mon[31985]: reconnect by client.35222 172.21.15.22:0/3729116781 after 0.00699991 2024-03-28T18:15:06.674 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:15:06 smithi134 ceph-mon[35037]: pgmap v248: 97 pgs: 97 active+clean; 117 MiB data, 793 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 204 B/s wr, 12 op/s 2024-03-28T18:15:06.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:15:06 smithi134 ceph-mon[35037]: mds.? [v2:172.21.15.22:6834/535745119,v1:172.21.15.22:6835/535745119] up:rejoin 2024-03-28T18:15:06.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:15:06 smithi134 ceph-mon[35037]: fsmap cephfs:5/5 4 up:standby-replay 1 up:standby 1 up:rejoin 4 up:active 2024-03-28T18:15:06.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:15:06 smithi134 ceph-mon[35037]: daemon mds.j is now active in filesystem cephfs as rank 1 2024-03-28T18:15:06.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:15:06 smithi079 ceph-mon[35312]: pgmap v248: 97 pgs: 97 active+clean; 117 MiB data, 793 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 204 B/s wr, 12 op/s 2024-03-28T18:15:06.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:15:06 smithi079 ceph-mon[35312]: mds.? [v2:172.21.15.22:6834/535745119,v1:172.21.15.22:6835/535745119] up:rejoin 2024-03-28T18:15:06.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:15:06 smithi079 ceph-mon[35312]: fsmap cephfs:5/5 4 up:standby-replay 1 up:standby 1 up:rejoin 4 up:active 2024-03-28T18:15:06.683 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:15:06 smithi079 ceph-mon[35312]: daemon mds.j is now active in filesystem cephfs as rank 1 2024-03-28T18:15:06.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:15:06 smithi022 ceph-mon[31985]: pgmap v248: 97 pgs: 97 active+clean; 117 MiB data, 793 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 204 B/s wr, 12 op/s 2024-03-28T18:15:06.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:15:06 smithi022 ceph-mon[31985]: mds.? [v2:172.21.15.22:6834/535745119,v1:172.21.15.22:6835/535745119] up:rejoin 2024-03-28T18:15:06.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:15:06 smithi022 ceph-mon[31985]: fsmap cephfs:5/5 4 up:standby-replay 1 up:standby 1 up:rejoin 4 up:active 2024-03-28T18:15:06.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:15:06 smithi022 ceph-mon[31985]: daemon mds.j is now active in filesystem cephfs as rank 1 2024-03-28T18:15:06.783 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:15:06 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[93969]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-2145-g3be9a5c9/rpm/el8/BUILD/ceph-19.0.0-2145-g3be9a5c9/src/mds/Locker.cc: In function 'void Locker::set_xlocks_done(MutationImpl*, bool)' thread 7f070a800700 time 2024-03-28T18:15:06.458732+0000 2024-03-28T18:15:06.783 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:15:06 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[93969]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-2145-g3be9a5c9/rpm/el8/BUILD/ceph-19.0.0-2145-g3be9a5c9/src/mds/Locker.cc: 626: FAILED ceph_assert(obj->is_auth()) 2024-03-28T18:15:06.783 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:15:06 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[93969]: ceph version 19.0.0-2145-g3be9a5c9 (3be9a5c9bc793e11e2800a8c0c696e8b46742033) squid (dev) 2024-03-28T18:15:06.783 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:15:06 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[93969]: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x135) [0x7f071604d4a1] 2024-03-28T18:15:06.784 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:15:06 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[93969]: 2: /usr/lib64/ceph/libceph-common.so.2(+0x2d2667) [0x7f071604d667] 2024-03-28T18:15:06.784 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:15:06 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[93969]: 3: (Locker::set_xlocks_done(MutationImpl*, bool)+0x493) [0x55b821600223] 2024-03-28T18:15:06.784 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:15:06 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[93969]: 4: (Server::early_reply(boost::intrusive_ptr const&, CInode*, CDentry*)+0x3e6) [0x55b82147a5d6] 2024-03-28T18:15:06.784 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:15:06 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[93969]: 5: (Server::journal_and_reply(boost::intrusive_ptr const&, CInode*, CDentry*, LogEvent*, MDSLogContextBase*)+0xc1) [0x55b82147ada1] 2024-03-28T18:15:06.784 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:15:06 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[93969]: 6: (Server::handle_client_mkdir(boost::intrusive_ptr const&)+0x5ba) [0x55b8214b06aa] 2024-03-28T18:15:06.784 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:15:06 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[93969]: 7: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0x733) [0x55b8214cd233] 2024-03-28T18:15:06.784 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:15:06 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[93969]: 8: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x55b8214cf197] 2024-03-28T18:15:06.784 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:15:06 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[93969]: 9: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x55b8213ce11b] 2024-03-28T18:15:06.784 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:15:06 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[93969]: 10: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x55b8213ce6bc] 2024-03-28T18:15:06.784 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:15:06 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[93969]: 11: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bf) [0x55b8213b81ff] 2024-03-28T18:15:06.784 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:15:06 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[93969]: 12: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x478) [0x7f07162df488] 2024-03-28T18:15:06.784 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:15:06 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[93969]: 13: (DispatchQueue::entry()+0x51f) [0x7f07162dc60f] 2024-03-28T18:15:06.784 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:15:06 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[93969]: 14: (DispatchQueue::DispatchThread::entry()+0x11) [0x7f07163afe81] 2024-03-28T18:15:06.784 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:15:06 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[93969]: 15: /lib64/libpthread.so.0(+0x81ca) [0x7f0714dca1ca] 2024-03-28T18:15:06.784 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:15:06 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[93969]: 16: clone() 2024-03-28T18:15:06.785 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:15:06 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[93969]: 2024-03-28T18:15:06.459+0000 7f070a800700 -1 /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-2145-g3be9a5c9/rpm/el8/BUILD/ceph-19.0.0-2145-g3be9a5c9/src/mds/Locker.cc: In function 'void Locker::set_xlocks_done(MutationImpl*, bool)' thread 7f070a800700 time 2024-03-28T18:15:06.458732+0000 2024-03-28T18:15:06.785 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:15:06 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[93969]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-2145-g3be9a5c9/rpm/el8/BUILD/ceph-19.0.0-2145-g3be9a5c9/src/mds/Locker.cc: 626: FAILED ceph_assert(obj->is_auth()) 2024-03-28T18:15:06.785 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:15:06 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[93969]: 2024-03-28T18:15:06.785 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:15:06 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[93969]: ceph version 19.0.0-2145-g3be9a5c9 (3be9a5c9bc793e11e2800a8c0c696e8b46742033) squid (dev) 2024-03-28T18:15:06.785 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:15:06 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[93969]: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x135) [0x7f071604d4a1] 2024-03-28T18:15:06.785 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:15:06 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[93969]: 2: /usr/lib64/ceph/libceph-common.so.2(+0x2d2667) [0x7f071604d667] 2024-03-28T18:15:06.785 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:15:06 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[93969]: 3: (Locker::set_xlocks_done(MutationImpl*, bool)+0x493) [0x55b821600223] 2024-03-28T18:15:06.785 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:15:06 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[93969]: 4: (Server::early_reply(boost::intrusive_ptr const&, CInode*, CDentry*)+0x3e6) [0x55b82147a5d6] 2024-03-28T18:15:06.785 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:15:06 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[93969]: 5: (Server::journal_and_reply(boost::intrusive_ptr const&, CInode*, CDentry*, LogEvent*, MDSLogContextBase*)+0xc1) [0x55b82147ada1] 2024-03-28T18:15:06.785 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:15:06 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[93969]: 6: (Server::handle_client_mkdir(boost::intrusive_ptr const&)+0x5ba) [0x55b8214b06aa] 2024-03-28T18:15:06.786 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:15:06 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[93969]: 7: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0x733) [0x55b8214cd233] 2024-03-28T18:15:06.786 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:15:06 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[93969]: 8: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x55b8214cf197] 2024-03-28T18:15:06.786 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:15:06 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[93969]: 9: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x55b8213ce11b] 2024-03-28T18:15:06.786 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:15:06 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[93969]: 10: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x55b8213ce6bc] 2024-03-28T18:15:06.786 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:15:06 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[93969]: 11: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bf) [0x55b8213b81ff] 2024-03-28T18:15:06.786 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:15:06 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[93969]: 12: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x478) [0x7f07162df488] 2024-03-28T18:15:06.786 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:15:06 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[93969]: 13: (DispatchQueue::entry()+0x51f) [0x7f07162dc60f] 2024-03-28T18:15:06.786 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:15:06 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[93969]: 14: (DispatchQueue::DispatchThread::entry()+0x11) [0x7f07163afe81] 2024-03-28T18:15:06.786 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:15:06 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[93969]: 15: /lib64/libpthread.so.0(+0x81ca) [0x7f0714dca1ca] 2024-03-28T18:15:06.786 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:15:06 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[93969]: 16: clone() 2024-03-28T18:15:06.786 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:15:06 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[93969]: 2024-03-28T18:15:06.786 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:15:06 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[93969]: *** Caught signal (Aborted) ** 2024-03-28T18:15:06.786 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:15:06 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[93969]: in thread 7f070a800700 thread_name:ms_dispatch 2024-03-28T18:15:06.786 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:15:06 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[93969]: ceph version 19.0.0-2145-g3be9a5c9 (3be9a5c9bc793e11e2800a8c0c696e8b46742033) squid (dev) 2024-03-28T18:15:06.787 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:15:06 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[93969]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7f0714dd4d20] 2024-03-28T18:15:06.787 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:15:06 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[93969]: 2: gsignal() 2024-03-28T18:15:06.787 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:15:06 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[93969]: 3: abort() 2024-03-28T18:15:06.787 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:15:06 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[93969]: 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x18f) [0x7f071604d4fb] 2024-03-28T18:15:06.787 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:15:06 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[93969]: 5: /usr/lib64/ceph/libceph-common.so.2(+0x2d2667) [0x7f071604d667] 2024-03-28T18:15:06.787 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:15:06 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[93969]: 6: (Locker::set_xlocks_done(MutationImpl*, bool)+0x493) [0x55b821600223] 2024-03-28T18:15:06.787 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:15:06 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[93969]: 7: (Server::early_reply(boost::intrusive_ptr const&, CInode*, CDentry*)+0x3e6) [0x55b82147a5d6] 2024-03-28T18:15:06.787 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:15:06 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[93969]: 8: (Server::journal_and_reply(boost::intrusive_ptr const&, CInode*, CDentry*, LogEvent*, MDSLogContextBase*)+0xc1) [0x55b82147ada1] 2024-03-28T18:15:06.787 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:15:06 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[93969]: 9: (Server::handle_client_mkdir(boost::intrusive_ptr const&)+0x5ba) [0x55b8214b06aa] 2024-03-28T18:15:06.787 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:15:06 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[93969]: 10: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0x733) [0x55b8214cd233] 2024-03-28T18:15:06.788 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:15:06 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[93969]: 11: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x55b8214cf197] 2024-03-28T18:15:06.788 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:15:06 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[93969]: 12: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x55b8213ce11b] 2024-03-28T18:15:06.788 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:15:06 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[93969]: 13: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x55b8213ce6bc] 2024-03-28T18:15:06.788 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:15:06 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[93969]: 14: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bf) [0x55b8213b81ff] 2024-03-28T18:15:06.788 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:15:06 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[93969]: 15: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x478) [0x7f07162df488] 2024-03-28T18:15:06.788 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:15:06 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[93969]: 16: (DispatchQueue::entry()+0x51f) [0x7f07162dc60f] 2024-03-28T18:15:06.788 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:15:06 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[93969]: 17: (DispatchQueue::DispatchThread::entry()+0x11) [0x7f07163afe81] 2024-03-28T18:15:06.788 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:15:06 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[93969]: 18: /lib64/libpthread.so.0(+0x81ca) [0x7f0714dca1ca] 2024-03-28T18:15:06.788 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:15:06 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[93969]: 19: clone() 2024-03-28T18:15:06.788 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:15:06 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[93969]: 2024-03-28T18:15:06.462+0000 7f070a800700 -1 *** Caught signal (Aborted) ** 2024-03-28T18:15:06.788 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:15:06 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[93969]: in thread 7f070a800700 thread_name:ms_dispatch 2024-03-28T18:15:06.788 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:15:06 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[93969]: 2024-03-28T18:15:06.788 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:15:06 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[93969]: ceph version 19.0.0-2145-g3be9a5c9 (3be9a5c9bc793e11e2800a8c0c696e8b46742033) squid (dev) 2024-03-28T18:15:06.788 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:15:06 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[93969]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7f0714dd4d20] 2024-03-28T18:15:06.788 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:15:06 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[93969]: 2: gsignal() 2024-03-28T18:15:06.789 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:15:06 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[93969]: 3: abort() 2024-03-28T18:15:06.789 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:15:06 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[93969]: 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x18f) [0x7f071604d4fb] 2024-03-28T18:15:06.789 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:15:06 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[93969]: 5: /usr/lib64/ceph/libceph-common.so.2(+0x2d2667) [0x7f071604d667] 2024-03-28T18:15:06.789 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:15:06 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[93969]: 6: (Locker::set_xlocks_done(MutationImpl*, bool)+0x493) [0x55b821600223] 2024-03-28T18:15:06.789 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:15:06 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[93969]: 7: (Server::early_reply(boost::intrusive_ptr const&, CInode*, CDentry*)+0x3e6) [0x55b82147a5d6] 2024-03-28T18:15:06.789 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:15:06 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[93969]: 8: (Server::journal_and_reply(boost::intrusive_ptr const&, CInode*, CDentry*, LogEvent*, MDSLogContextBase*)+0xc1) [0x55b82147ada1] 2024-03-28T18:15:06.789 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:15:06 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[93969]: 9: (Server::handle_client_mkdir(boost::intrusive_ptr const&)+0x5ba) [0x55b8214b06aa] 2024-03-28T18:15:06.789 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:15:06 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[93969]: 10: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0x733) [0x55b8214cd233] 2024-03-28T18:15:06.789 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:15:06 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[93969]: 11: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x55b8214cf197] 2024-03-28T18:15:06.789 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:15:06 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[93969]: 12: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x55b8213ce11b] 2024-03-28T18:15:06.789 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:15:06 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[93969]: 13: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x55b8213ce6bc] 2024-03-28T18:15:06.790 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:15:06 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[93969]: 14: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bf) [0x55b8213b81ff] 2024-03-28T18:15:06.790 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:15:06 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[93969]: 15: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x478) [0x7f07162df488] 2024-03-28T18:15:06.790 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:15:06 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[93969]: 16: (DispatchQueue::entry()+0x51f) [0x7f07162dc60f] 2024-03-28T18:15:06.790 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:15:06 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[93969]: 17: (DispatchQueue::DispatchThread::entry()+0x11) [0x7f07163afe81] 2024-03-28T18:15:06.790 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:15:06 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[93969]: 18: /lib64/libpthread.so.0(+0x81ca) [0x7f0714dca1ca] 2024-03-28T18:15:06.790 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:15:06 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[93969]: 19: clone() 2024-03-28T18:15:06.790 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:15:06 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[93969]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-03-28T18:15:06.790 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:15:06 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[93969]: 2024-03-28T18:15:06.790 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:15:06 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[93969]: -1> 2024-03-28T18:15:06.459+0000 7f070a800700 -1 /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-2145-g3be9a5c9/rpm/el8/BUILD/ceph-19.0.0-2145-g3be9a5c9/src/mds/Locker.cc: In function 'void Locker::set_xlocks_done(MutationImpl*, bool)' thread 7f070a800700 time 2024-03-28T18:15:06.458732+0000 2024-03-28T18:15:06.790 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:15:06 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[93969]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-2145-g3be9a5c9/rpm/el8/BUILD/ceph-19.0.0-2145-g3be9a5c9/src/mds/Locker.cc: 626: FAILED ceph_assert(obj->is_auth()) 2024-03-28T18:15:06.790 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:15:06 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[93969]: 2024-03-28T18:15:06.790 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:15:06 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[93969]: ceph version 19.0.0-2145-g3be9a5c9 (3be9a5c9bc793e11e2800a8c0c696e8b46742033) squid (dev) 2024-03-28T18:15:06.790 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:15:06 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[93969]: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x135) [0x7f071604d4a1] 2024-03-28T18:15:06.790 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:15:06 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[93969]: 2: /usr/lib64/ceph/libceph-common.so.2(+0x2d2667) [0x7f071604d667] 2024-03-28T18:15:06.791 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:15:06 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[93969]: 3: (Locker::set_xlocks_done(MutationImpl*, bool)+0x493) [0x55b821600223] 2024-03-28T18:15:06.791 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:15:06 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[93969]: 4: (Server::early_reply(boost::intrusive_ptr const&, CInode*, CDentry*)+0x3e6) [0x55b82147a5d6] 2024-03-28T18:15:06.791 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:15:06 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[93969]: 5: (Server::journal_and_reply(boost::intrusive_ptr const&, CInode*, CDentry*, LogEvent*, MDSLogContextBase*)+0xc1) [0x55b82147ada1] 2024-03-28T18:15:06.791 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:15:06 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[93969]: 6: (Server::handle_client_mkdir(boost::intrusive_ptr const&)+0x5ba) [0x55b8214b06aa] 2024-03-28T18:15:06.791 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:15:06 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[93969]: 7: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0x733) [0x55b8214cd233] 2024-03-28T18:15:06.791 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:15:06 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[93969]: 8: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x55b8214cf197] 2024-03-28T18:15:06.791 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:15:06 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[93969]: 9: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x55b8213ce11b] 2024-03-28T18:15:06.791 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:15:06 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[93969]: 10: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x55b8213ce6bc] 2024-03-28T18:15:06.791 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:15:06 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[93969]: 11: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bf) [0x55b8213b81ff] 2024-03-28T18:15:06.791 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:15:06 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[93969]: 12: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x478) [0x7f07162df488] 2024-03-28T18:15:06.791 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:15:06 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[93969]: 13: (DispatchQueue::entry()+0x51f) [0x7f07162dc60f] 2024-03-28T18:15:06.791 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:15:06 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[93969]: 14: (DispatchQueue::DispatchThread::entry()+0x11) [0x7f07163afe81] 2024-03-28T18:15:06.792 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:15:06 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[93969]: 15: /lib64/libpthread.so.0(+0x81ca) [0x7f0714dca1ca] 2024-03-28T18:15:06.792 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:15:06 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[93969]: 16: clone() 2024-03-28T18:15:06.792 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:15:06 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[93969]: 2024-03-28T18:15:06.792 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:15:06 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[93969]: 0> 2024-03-28T18:15:06.462+0000 7f070a800700 -1 *** Caught signal (Aborted) ** 2024-03-28T18:15:06.792 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:15:06 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[93969]: in thread 7f070a800700 thread_name:ms_dispatch 2024-03-28T18:15:06.792 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:15:06 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[93969]: 2024-03-28T18:15:06.792 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:15:06 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[93969]: ceph version 19.0.0-2145-g3be9a5c9 (3be9a5c9bc793e11e2800a8c0c696e8b46742033) squid (dev) 2024-03-28T18:15:06.792 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:15:06 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[93969]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7f0714dd4d20] 2024-03-28T18:15:06.792 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:15:06 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[93969]: 2: gsignal() 2024-03-28T18:15:06.792 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:15:06 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[93969]: 3: abort() 2024-03-28T18:15:06.792 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:15:06 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[93969]: 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x18f) [0x7f071604d4fb] 2024-03-28T18:15:06.792 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:15:06 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[93969]: 5: /usr/lib64/ceph/libceph-common.so.2(+0x2d2667) [0x7f071604d667] 2024-03-28T18:15:06.792 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:15:06 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[93969]: 6: (Locker::set_xlocks_done(MutationImpl*, bool)+0x493) [0x55b821600223] 2024-03-28T18:15:06.792 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:15:06 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[93969]: 7: (Server::early_reply(boost::intrusive_ptr const&, CInode*, CDentry*)+0x3e6) [0x55b82147a5d6] 2024-03-28T18:15:06.793 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:15:06 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[93969]: 8: (Server::journal_and_reply(boost::intrusive_ptr const&, CInode*, CDentry*, LogEvent*, MDSLogContextBase*)+0xc1) [0x55b82147ada1] 2024-03-28T18:15:06.793 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:15:06 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[93969]: 9: (Server::handle_client_mkdir(boost::intrusive_ptr const&)+0x5ba) [0x55b8214b06aa] 2024-03-28T18:15:06.793 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:15:06 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[93969]: 10: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0x733) [0x55b8214cd233] 2024-03-28T18:15:06.793 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:15:06 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[93969]: 11: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x55b8214cf197] 2024-03-28T18:15:06.793 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:15:06 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[93969]: 12: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x55b8213ce11b] 2024-03-28T18:15:06.793 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:15:06 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[93969]: 13: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x55b8213ce6bc] 2024-03-28T18:15:06.793 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:15:06 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[93969]: 14: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bf) [0x55b8213b81ff] 2024-03-28T18:15:06.793 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:15:06 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[93969]: 15: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x478) [0x7f07162df488] 2024-03-28T18:15:06.793 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:15:06 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[93969]: 16: (DispatchQueue::entry()+0x51f) [0x7f07162dc60f] 2024-03-28T18:15:06.793 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:15:06 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[93969]: 17: (DispatchQueue::DispatchThread::entry()+0x11) [0x7f07163afe81] 2024-03-28T18:15:06.793 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:15:06 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[93969]: 18: /lib64/libpthread.so.0(+0x81ca) [0x7f0714dca1ca] 2024-03-28T18:15:06.793 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:15:06 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[93969]: 19: clone() 2024-03-28T18:15:06.793 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:15:06 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[93969]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-03-28T18:15:06.794 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:15:06 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[93969]: 2024-03-28T18:15:06.794 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:15:06 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[93969]: -1> 2024-03-28T18:15:06.459+0000 7f070a800700 -1 /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-2145-g3be9a5c9/rpm/el8/BUILD/ceph-19.0.0-2145-g3be9a5c9/src/mds/Locker.cc: In function 'void Locker::set_xlocks_done(MutationImpl*, bool)' thread 7f070a800700 time 2024-03-28T18:15:06.458732+0000 2024-03-28T18:15:06.794 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:15:06 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[93969]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-2145-g3be9a5c9/rpm/el8/BUILD/ceph-19.0.0-2145-g3be9a5c9/src/mds/Locker.cc: 626: FAILED ceph_assert(obj->is_auth()) 2024-03-28T18:15:06.794 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:15:06 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[93969]: 2024-03-28T18:15:06.794 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:15:06 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[93969]: ceph version 19.0.0-2145-g3be9a5c9 (3be9a5c9bc793e11e2800a8c0c696e8b46742033) squid (dev) 2024-03-28T18:15:06.794 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:15:06 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[93969]: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x135) [0x7f071604d4a1] 2024-03-28T18:15:06.794 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:15:06 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[93969]: 2: /usr/lib64/ceph/libceph-common.so.2(+0x2d2667) [0x7f071604d667] 2024-03-28T18:15:06.794 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:15:06 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[93969]: 3: (Locker::set_xlocks_done(MutationImpl*, bool)+0x493) [0x55b821600223] 2024-03-28T18:15:06.794 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:15:06 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[93969]: 4: (Server::early_reply(boost::intrusive_ptr const&, CInode*, CDentry*)+0x3e6) [0x55b82147a5d6] 2024-03-28T18:15:06.794 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:15:06 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[93969]: 5: (Server::journal_and_reply(boost::intrusive_ptr const&, CInode*, CDentry*, LogEvent*, MDSLogContextBase*)+0xc1) [0x55b82147ada1] 2024-03-28T18:15:06.795 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:15:06 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[93969]: 6: (Server::handle_client_mkdir(boost::intrusive_ptr const&)+0x5ba) [0x55b8214b06aa] 2024-03-28T18:15:06.795 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:15:06 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[93969]: 7: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0x733) [0x55b8214cd233] 2024-03-28T18:15:06.795 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:15:06 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[93969]: 8: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x55b8214cf197] 2024-03-28T18:15:06.795 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:15:06 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[93969]: 9: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x55b8213ce11b] 2024-03-28T18:15:06.795 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:15:06 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[93969]: 10: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x55b8213ce6bc] 2024-03-28T18:15:06.795 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:15:06 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[93969]: 11: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bf) [0x55b8213b81ff] 2024-03-28T18:15:06.795 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:15:06 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[93969]: 12: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x478) [0x7f07162df488] 2024-03-28T18:15:06.795 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:15:06 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[93969]: 13: (DispatchQueue::entry()+0x51f) [0x7f07162dc60f] 2024-03-28T18:15:06.795 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:15:06 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[93969]: 14: (DispatchQueue::DispatchThread::entry()+0x11) [0x7f07163afe81] 2024-03-28T18:15:06.795 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:15:06 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[93969]: 15: /lib64/libpthread.so.0(+0x81ca) [0x7f0714dca1ca] 2024-03-28T18:15:06.795 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:15:06 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[93969]: 16: clone() 2024-03-28T18:15:06.795 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:15:06 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[93969]: 2024-03-28T18:15:06.795 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:15:06 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[93969]: 0> 2024-03-28T18:15:06.462+0000 7f070a800700 -1 *** Caught signal (Aborted) ** 2024-03-28T18:15:06.796 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:15:06 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[93969]: in thread 7f070a800700 thread_name:ms_dispatch 2024-03-28T18:15:06.796 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:15:06 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[93969]: 2024-03-28T18:15:06.796 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:15:06 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[93969]: ceph version 19.0.0-2145-g3be9a5c9 (3be9a5c9bc793e11e2800a8c0c696e8b46742033) squid (dev) 2024-03-28T18:15:06.796 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:15:06 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[93969]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7f0714dd4d20] 2024-03-28T18:15:06.796 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:15:06 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[93969]: 2: gsignal() 2024-03-28T18:15:06.796 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:15:06 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[93969]: 3: abort() 2024-03-28T18:15:06.796 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:15:06 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[93969]: 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x18f) [0x7f071604d4fb] 2024-03-28T18:15:06.796 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:15:06 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[93969]: 5: /usr/lib64/ceph/libceph-common.so.2(+0x2d2667) [0x7f071604d667] 2024-03-28T18:15:06.796 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:15:06 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[93969]: 6: (Locker::set_xlocks_done(MutationImpl*, bool)+0x493) [0x55b821600223] 2024-03-28T18:15:06.796 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:15:06 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[93969]: 7: (Server::early_reply(boost::intrusive_ptr const&, CInode*, CDentry*)+0x3e6) [0x55b82147a5d6] 2024-03-28T18:15:06.796 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:15:06 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[93969]: 8: (Server::journal_and_reply(boost::intrusive_ptr const&, CInode*, CDentry*, LogEvent*, MDSLogContextBase*)+0xc1) [0x55b82147ada1] 2024-03-28T18:15:06.796 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:15:06 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[93969]: 9: (Server::handle_client_mkdir(boost::intrusive_ptr const&)+0x5ba) [0x55b8214b06aa] 2024-03-28T18:15:06.796 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:15:06 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[93969]: 10: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0x733) [0x55b8214cd233] 2024-03-28T18:15:06.797 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:15:06 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[93969]: 11: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x55b8214cf197] 2024-03-28T18:15:06.797 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:15:06 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[93969]: 12: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x55b8213ce11b] 2024-03-28T18:15:06.797 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:15:06 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[93969]: 13: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x55b8213ce6bc] 2024-03-28T18:15:06.797 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:15:06 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[93969]: 14: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bf) [0x55b8213b81ff] 2024-03-28T18:15:06.797 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:15:06 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[93969]: 15: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x478) [0x7f07162df488] 2024-03-28T18:15:06.797 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:15:06 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[93969]: 16: (DispatchQueue::entry()+0x51f) [0x7f07162dc60f] 2024-03-28T18:15:06.797 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:15:06 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[93969]: 17: (DispatchQueue::DispatchThread::entry()+0x11) [0x7f07163afe81] 2024-03-28T18:15:06.797 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:15:06 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[93969]: 18: /lib64/libpthread.so.0(+0x81ca) [0x7f0714dca1ca] 2024-03-28T18:15:06.797 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:15:06 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[93969]: 19: clone() 2024-03-28T18:15:06.797 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:15:06 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[93969]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-03-28T18:15:06.797 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:15:06 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[93969]: 2024-03-28T18:15:06.797 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:15:06 smithi022 podman[94365]: 2024-03-28 18:15:06.580593437 +0000 UTC m=+0.031888412 container died f713b193cb5c3ff019609676b1f8f7ee10ae6b0c45e402a6ab968463799c5836 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, ceph=True, org.label-schema.schema-version=1.0, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, org.label-schema.build-date=20240319, CEPH_POINT_RELEASE=) 2024-03-28T18:15:07.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:15:07 smithi134 ceph-mon[35037]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-03-28T18:15:07.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:15:07 smithi134 ceph-mon[35037]: Cluster is now healthy 2024-03-28T18:15:07.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:15:07 smithi134 ceph-mon[35037]: mds.? [v2:172.21.15.22:6834/535745119,v1:172.21.15.22:6835/535745119] up:active 2024-03-28T18:15:07.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:15:07 smithi134 ceph-mon[35037]: fsmap cephfs:5 4 up:standby-replay 1 up:standby 5 up:active 2024-03-28T18:15:07.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:15:07 smithi134 ceph-mon[35037]: fsmap cephfs:5 5 up:standby-replay 5 up:active 2024-03-28T18:15:07.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:15:07 smithi079 ceph-mon[35312]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-03-28T18:15:07.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:15:07 smithi079 ceph-mon[35312]: Cluster is now healthy 2024-03-28T18:15:07.683 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:15:07 smithi079 ceph-mon[35312]: mds.? [v2:172.21.15.22:6834/535745119,v1:172.21.15.22:6835/535745119] up:active 2024-03-28T18:15:07.683 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:15:07 smithi079 ceph-mon[35312]: fsmap cephfs:5 4 up:standby-replay 1 up:standby 5 up:active 2024-03-28T18:15:07.683 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:15:07 smithi079 ceph-mon[35312]: fsmap cephfs:5 5 up:standby-replay 5 up:active 2024-03-28T18:15:07.782 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:15:07 smithi022 podman[94365]: 2024-03-28 18:15:07.323545964 +0000 UTC m=+0.774840949 container remove f713b193cb5c3ff019609676b1f8f7ee10ae6b0c45e402a6ab968463799c5836 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240319, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, ceph=True, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-28T18:15:07.782 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:15:07 smithi022 systemd[1]: ceph-fd220748-ed2b-11ee-95d0-87774f69a715@mds.j.service: Main process exited, code=exited, status=134/n/a 2024-03-28T18:15:07.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:15:07 smithi022 ceph-mon[31985]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-03-28T18:15:07.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:15:07 smithi022 ceph-mon[31985]: Cluster is now healthy 2024-03-28T18:15:07.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:15:07 smithi022 ceph-mon[31985]: mds.? [v2:172.21.15.22:6834/535745119,v1:172.21.15.22:6835/535745119] up:active 2024-03-28T18:15:07.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:15:07 smithi022 ceph-mon[31985]: fsmap cephfs:5 4 up:standby-replay 1 up:standby 5 up:active 2024-03-28T18:15:07.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:15:07 smithi022 ceph-mon[31985]: fsmap cephfs:5 5 up:standby-replay 5 up:active 2024-03-28T18:15:08.281 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:15:07 smithi022 systemd[1]: ceph-fd220748-ed2b-11ee-95d0-87774f69a715@mds.j.service: Failed with result 'exit-code'. 2024-03-28T18:15:08.282 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:15:07 smithi022 systemd[1]: ceph-fd220748-ed2b-11ee-95d0-87774f69a715@mds.j.service: Consumed 1.934s CPU time. 2024-03-28T18:15:08.674 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:15:08 smithi134 ceph-mon[35037]: pgmap v249: 97 pgs: 97 active+clean; 117 MiB data, 793 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 204 B/s wr, 12 op/s 2024-03-28T18:15:08.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:15:08 smithi079 ceph-mon[35312]: pgmap v249: 97 pgs: 97 active+clean; 117 MiB data, 793 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 204 B/s wr, 12 op/s 2024-03-28T18:15:08.781 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:15:08 smithi022 ceph-mon[31985]: pgmap v249: 97 pgs: 97 active+clean; 117 MiB data, 793 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 204 B/s wr, 12 op/s 2024-03-28T18:15:09.432 INFO:journalctl@ceph.mds.b.smithi079.stdout:Mar 28 18:15:09 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-b[65245]: 2024-03-28T18:15:09.099+0000 7f845f400700 -1 mds.pinger is_rank_lagging: rank=1 was never sent ping request. 2024-03-28T18:15:10.674 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:15:10 smithi134 ceph-mon[35037]: pgmap v250: 97 pgs: 97 active+clean; 117 MiB data, 793 MiB used, 1.0 TiB / 1.0 TiB avail; 121 KiB/s rd, 32 KiB/s wr, 43 op/s 2024-03-28T18:15:10.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:15:10 smithi079 ceph-mon[35312]: pgmap v250: 97 pgs: 97 active+clean; 117 MiB data, 793 MiB used, 1.0 TiB / 1.0 TiB avail; 121 KiB/s rd, 32 KiB/s wr, 43 op/s 2024-03-28T18:15:10.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:15:10 smithi022 ceph-mon[31985]: pgmap v250: 97 pgs: 97 active+clean; 117 MiB data, 793 MiB used, 1.0 TiB / 1.0 TiB avail; 121 KiB/s rd, 32 KiB/s wr, 43 op/s 2024-03-28T18:15:12.674 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:15:12 smithi134 ceph-mon[35037]: pgmap v251: 97 pgs: 97 active+clean; 117 MiB data, 793 MiB used, 1.0 TiB / 1.0 TiB avail; 121 KiB/s rd, 32 KiB/s wr, 43 op/s 2024-03-28T18:15:12.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:15:12 smithi079 ceph-mon[35312]: pgmap v251: 97 pgs: 97 active+clean; 117 MiB data, 793 MiB used, 1.0 TiB / 1.0 TiB avail; 121 KiB/s rd, 32 KiB/s wr, 43 op/s 2024-03-28T18:15:12.781 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:15:12 smithi022 ceph-mon[31985]: pgmap v251: 97 pgs: 97 active+clean; 117 MiB data, 793 MiB used, 1.0 TiB / 1.0 TiB avail; 121 KiB/s rd, 32 KiB/s wr, 43 op/s 2024-03-28T18:15:14.674 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:15:14 smithi134 ceph-mon[35037]: pgmap v252: 97 pgs: 97 active+clean; 117 MiB data, 793 MiB used, 1.0 TiB / 1.0 TiB avail; 105 KiB/s rd, 28 KiB/s wr, 37 op/s 2024-03-28T18:15:14.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:15:14 smithi079 ceph-mon[35312]: pgmap v252: 97 pgs: 97 active+clean; 117 MiB data, 793 MiB used, 1.0 TiB / 1.0 TiB avail; 105 KiB/s rd, 28 KiB/s wr, 37 op/s 2024-03-28T18:15:14.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:15:14 smithi022 ceph-mon[31985]: pgmap v252: 97 pgs: 97 active+clean; 117 MiB data, 793 MiB used, 1.0 TiB / 1.0 TiB avail; 105 KiB/s rd, 28 KiB/s wr, 37 op/s 2024-03-28T18:15:16.674 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:15:16 smithi134 ceph-mon[35037]: pgmap v253: 97 pgs: 97 active+clean; 117 MiB data, 793 MiB used, 1.0 TiB / 1.0 TiB avail; 103 KiB/s rd, 27 KiB/s wr, 40 op/s 2024-03-28T18:15:16.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:15:16 smithi079 ceph-mon[35312]: pgmap v253: 97 pgs: 97 active+clean; 117 MiB data, 793 MiB used, 1.0 TiB / 1.0 TiB avail; 103 KiB/s rd, 27 KiB/s wr, 40 op/s 2024-03-28T18:15:16.781 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:15:16 smithi022 ceph-mon[31985]: pgmap v253: 97 pgs: 97 active+clean; 117 MiB data, 793 MiB used, 1.0 TiB / 1.0 TiB avail; 103 KiB/s rd, 27 KiB/s wr, 40 op/s 2024-03-28T18:15:18.430 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:15:18 smithi022 systemd[1]: ceph-fd220748-ed2b-11ee-95d0-87774f69a715@mds.j.service: Scheduled restart job, restart counter is at 3. 2024-03-28T18:15:18.430 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:15:18 smithi022 systemd[1]: Stopped Ceph mds.j for fd220748-ed2b-11ee-95d0-87774f69a715. 2024-03-28T18:15:18.430 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:15:18 smithi022 systemd[1]: ceph-fd220748-ed2b-11ee-95d0-87774f69a715@mds.j.service: Consumed 1.934s CPU time. 2024-03-28T18:15:18.430 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:15:18 smithi022 systemd[1]: Starting Ceph mds.j for fd220748-ed2b-11ee-95d0-87774f69a715... 2024-03-28T18:15:18.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:15:18 smithi079 ceph-mon[35312]: pgmap v254: 97 pgs: 97 active+clean; 117 MiB data, 793 MiB used, 1.0 TiB / 1.0 TiB avail; 99 KiB/s rd, 27 KiB/s wr, 34 op/s 2024-03-28T18:15:18.779 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:15:18 smithi022 ceph-mon[31985]: pgmap v254: 97 pgs: 97 active+clean; 117 MiB data, 793 MiB used, 1.0 TiB / 1.0 TiB avail; 99 KiB/s rd, 27 KiB/s wr, 34 op/s 2024-03-28T18:15:18.779 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:15:18 smithi022 podman[94455]: 2024-03-28 18:15:18.338978601 +0000 UTC m=+0.035974603 image pull c664afe56bb01db4f9afa9532c20598e201e664f2e9ba2be50db0d52674a26d2 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc 2024-03-28T18:15:18.779 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:15:18 smithi022 podman[94455]: 2024-03-28 18:15:18.508094133 +0000 UTC m=+0.205090164 container create 7a3646bdb6cd85559385486f0aed5a2ec7a1ee9cf6f5208081a36ce4ea450d2e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.build-date=20240319, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_CLEAN=True, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c) 2024-03-28T18:15:18.779 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:15:18 smithi022 podman[94455]: 2024-03-28 18:15:18.778896204 +0000 UTC m=+0.475892236 container init 7a3646bdb6cd85559385486f0aed5a2ec7a1ee9cf6f5208081a36ce4ea450d2e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.build-date=20240319, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD) 2024-03-28T18:15:18.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:15:18 smithi134 ceph-mon[35037]: pgmap v254: 97 pgs: 97 active+clean; 117 MiB data, 793 MiB used, 1.0 TiB / 1.0 TiB avail; 99 KiB/s rd, 27 KiB/s wr, 34 op/s 2024-03-28T18:15:19.032 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:15:18 smithi022 podman[94455]: 2024-03-28 18:15:18.789893431 +0000 UTC m=+0.486889489 container start 7a3646bdb6cd85559385486f0aed5a2ec7a1ee9cf6f5208081a36ce4ea450d2e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j, io.buildah.version=1.33.5, org.label-schema.build-date=20240319, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, maintainer=Guillaume Abrioux , RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.license=GPLv2) 2024-03-28T18:15:19.032 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:15:18 smithi022 ceph-mds[94475]: -- 172.21.15.22:0/2561630894 <== mon.0 v2:172.21.15.22:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x560815635a40 con 0x560815609000 2024-03-28T18:15:19.033 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:15:18 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[94471]: starting mds.j at 2024-03-28T18:15:19.033 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:15:18 smithi022 bash[94455]: 7a3646bdb6cd85559385486f0aed5a2ec7a1ee9cf6f5208081a36ce4ea450d2e 2024-03-28T18:15:19.033 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:15:18 smithi022 systemd[1]: Started Ceph mds.j for fd220748-ed2b-11ee-95d0-87774f69a715. 2024-03-28T18:15:19.260 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-28T18:15:19.261 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-28T18:15:19.506 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:15:19.506+0000 7f4171600640 1 -- 172.21.15.22:0/199867575 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f416c1124e0 msgr2=0x7f416c1148d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:15:19.506 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:15:19.506+0000 7f4171600640 1 --2- 172.21.15.22:0/199867575 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f416c1124e0 0x7f416c1148d0 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7f4164009ca0 tx=0x7f416402c0f0 comp rx=0 tx=0).stop 2024-03-28T18:15:19.507 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:15:19.506+0000 7f4171600640 1 -- 172.21.15.22:0/199867575 shutdown_connections 2024-03-28T18:15:19.507 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:15:19.506+0000 7f4171600640 1 --2- 172.21.15.22:0/199867575 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f416c1124e0 0x7f416c1148d0 unknown :-1 s=CLOSED pgs=752 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:15:19.507 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:15:19.506+0000 7f4171600640 1 --2- 172.21.15.22:0/199867575 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f416c0764f0 0x7f416c111fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:15:19.507 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:15:19.506+0000 7f4171600640 1 --2- 172.21.15.22:0/199867575 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f416c075bd0 0x7f416c075fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:15:19.507 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:15:19.506+0000 7f4171600640 1 -- 172.21.15.22:0/199867575 >> 172.21.15.22:0/199867575 conn(0x7f416c0ff2f0 msgr2=0x7f416c101710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:15:19.507 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:15:19.506+0000 7f4171600640 1 -- 172.21.15.22:0/199867575 shutdown_connections 2024-03-28T18:15:19.507 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:15:19.506+0000 7f4171600640 1 -- 172.21.15.22:0/199867575 wait complete. 2024-03-28T18:15:19.510 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:15:19.507+0000 7f4171600640 1 Processor -- start 2024-03-28T18:15:19.510 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:15:19.508+0000 7f4171600640 1 -- start start 2024-03-28T18:15:19.510 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:15:19.508+0000 7f4171600640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f416c075bd0 0x7f416c19e640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:15:19.510 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:15:19.508+0000 7f4171600640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f416c0764f0 0x7f416c19eb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:15:19.510 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:15:19.508+0000 7f4171600640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f416c1124e0 0x7f416c1a2eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:15:19.510 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:15:19.508+0000 7f4171600640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f416c118010 con 0x7f416c0764f0 2024-03-28T18:15:19.511 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:15:19.508+0000 7f4171600640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f416c117e90 con 0x7f416c075bd0 2024-03-28T18:15:19.511 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:15:19.508+0000 7f4171600640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f416c118190 con 0x7f416c1124e0 2024-03-28T18:15:19.511 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:15:19.508+0000 7f416a000640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f416c0764f0 0x7f416c19eb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:15:19.511 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:15:19.508+0000 7f416a000640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f416c0764f0 0x7f416c19eb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:36008/0 (socket says 172.21.15.22:36008) 2024-03-28T18:15:19.511 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:15:19.508+0000 7f416a000640 1 -- 172.21.15.22:0/3968897138 learned_addr learned my addr 172.21.15.22:0/3968897138 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-28T18:15:19.511 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:15:19.509+0000 7f416aa00640 1 --2- 172.21.15.22:0/3968897138 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f416c075bd0 0x7f416c19e640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:15:19.511 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:15:19.509+0000 7f416b400640 1 --2- 172.21.15.22:0/3968897138 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f416c1124e0 0x7f416c1a2eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:15:19.511 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:15:19.509+0000 7f416a000640 1 -- 172.21.15.22:0/3968897138 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f416c1124e0 msgr2=0x7f416c1a2eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:15:19.511 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:15:19.509+0000 7f416a000640 1 --2- 172.21.15.22:0/3968897138 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f416c1124e0 0x7f416c1a2eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:15:19.511 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:15:19.509+0000 7f416a000640 1 -- 172.21.15.22:0/3968897138 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f416c075bd0 msgr2=0x7f416c19e640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:15:19.511 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:15:19.509+0000 7f416a000640 1 --2- 172.21.15.22:0/3968897138 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f416c075bd0 0x7f416c19e640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:15:19.511 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:15:19.509+0000 7f416a000640 1 -- 172.21.15.22:0/3968897138 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f416c1a35e0 con 0x7f416c0764f0 2024-03-28T18:15:19.511 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:15:19.509+0000 7f416aa00640 1 --2- 172.21.15.22:0/3968897138 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f416c075bd0 0x7f416c19e640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-28T18:15:19.511 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:15:19.509+0000 7f416a000640 1 --2- 172.21.15.22:0/3968897138 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f416c0764f0 0x7f416c19eb80 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7f415400e9f0 tx=0x7f415400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:15:19.511 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:15:19.509+0000 7f4163400640 1 -- 172.21.15.22:0/3968897138 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f415400cd80 con 0x7f416c0764f0 2024-03-28T18:15:19.512 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:15:19.509+0000 7f4163400640 1 -- 172.21.15.22:0/3968897138 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f4154010900 con 0x7f416c0764f0 2024-03-28T18:15:19.512 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:15:19.509+0000 7f4163400640 1 -- 172.21.15.22:0/3968897138 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41540189e0 con 0x7f416c0764f0 2024-03-28T18:15:19.512 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:15:19.509+0000 7f4171600640 1 -- 172.21.15.22:0/3968897138 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f416c1a38d0 con 0x7f416c0764f0 2024-03-28T18:15:19.512 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:15:19.510+0000 7f4171600640 1 -- 172.21.15.22:0/3968897138 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f416c1ab160 con 0x7f416c0764f0 2024-03-28T18:15:19.512 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:15:19.513+0000 7f41741fd740 1 Processor -- start 2024-03-28T18:15:19.512 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:15:19.513+0000 7f41741fd740 1 -- start start 2024-03-28T18:15:19.513 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:15:19.513+0000 7f4163400640 1 -- 172.21.15.22:0/3968897138 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 26) v1 ==== 137421+0+0 (secure 0 0 0) 0x7f4154010ab0 con 0x7f416c0764f0 2024-03-28T18:15:19.514 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:15:19.513+0000 7f41741fd740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55ef1cedac00 0x55ef1ced9950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:15:19.514 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:15:19.513+0000 7f41741fd740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ef1ced9f20 0x55ef1ceda3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:15:19.514 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:15:19.513+0000 7f41741fd740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ef1cee3290 0x55ef1cee5680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:15:19.514 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:15:19.513+0000 7f41741fd740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ef1ce3e250 con 0x55ef1cee3290 2024-03-28T18:15:19.514 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:15:19.513+0000 7f41741fd740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ef1cde6a30 con 0x55ef1cedac00 2024-03-28T18:15:19.514 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:15:19.513+0000 7f41741fd740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ef1ce3a2f0 con 0x55ef1ced9f20 2024-03-28T18:15:19.514 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:15:19.514+0000 7f416a000640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ef1cee3290 0x55ef1cee5680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:15:19.514 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:15:19.514+0000 7f416a000640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ef1cee3290 0x55ef1cee5680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:36018/0 (socket says 172.21.15.22:36018) 2024-03-28T18:15:19.515 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:15:19.514+0000 7f416a000640 1 -- 172.21.15.22:0/2138781736 learned_addr learned my addr 172.21.15.22:0/2138781736 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-28T18:15:19.515 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:15:19.514+0000 7f416b400640 1 --2- 172.21.15.22:0/2138781736 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55ef1cedac00 0x55ef1ced9950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:15:19.515 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:15:19.514+0000 7f416aa00640 1 --2- 172.21.15.22:0/2138781736 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ef1ced9f20 0x55ef1ceda3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:15:19.515 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:15:19.514+0000 7f416a000640 1 -- 172.21.15.22:0/2138781736 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ef1ced9f20 msgr2=0x55ef1ceda3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T18:15:19.515 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:15:19.514+0000 7f416a000640 1 --2- 172.21.15.22:0/2138781736 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ef1ced9f20 0x55ef1ceda3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:15:19.515 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:15:19.514+0000 7f416a000640 1 -- 172.21.15.22:0/2138781736 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55ef1cedac00 msgr2=0x55ef1ced9950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T18:15:19.515 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:15:19.514+0000 7f416a000640 1 --2- 172.21.15.22:0/2138781736 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55ef1cedac00 0x55ef1ced9950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:15:19.515 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:15:19.514+0000 7f416a000640 1 -- 172.21.15.22:0/2138781736 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ef1cdd1690 con 0x55ef1cee3290 2024-03-28T18:15:19.520 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:15:19.515+0000 7f416a000640 1 --2- 172.21.15.22:0/2138781736 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ef1cee3290 0x55ef1cee5680 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7f41540cc910 tx=0x7f41540d7670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e860d7c9ba6e3d3e server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:15:19.520 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:15:19.515+0000 7f4163400640 1 --2- 172.21.15.22:0/3968897138 >> [v2:172.21.15.22:6800/1357365082,v1:172.21.15.22:6801/1357365082] conn(0x7f41500a2970 0x7f41500a4e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:15:19.520 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:15:19.515+0000 7f4163400640 1 -- 172.21.15.22:0/3968897138 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(113..113 src has 1..113) v4 ==== 9959+0+0 (secure 0 0 0) 0x7f41540cf880 con 0x7f416c0764f0 2024-03-28T18:15:19.520 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:15:19.515+0000 7f415b400640 1 -- 172.21.15.22:0/2138781736 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41540de070 con 0x55ef1cee3290 2024-03-28T18:15:19.520 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:15:19.515+0000 7f415b400640 1 -- 172.21.15.22:0/2138781736 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f415409bfe0 con 0x55ef1cee3290 2024-03-28T18:15:19.520 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:15:19.515+0000 7f415b400640 1 -- 172.21.15.22:0/2138781736 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41540d2e60 con 0x55ef1cee3290 2024-03-28T18:15:19.520 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:15:19.515+0000 7f416b400640 1 --2- 172.21.15.22:0/3968897138 >> [v2:172.21.15.22:6800/1357365082,v1:172.21.15.22:6801/1357365082] conn(0x7f41500a2970 0x7f41500a4e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:15:19.520 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:15:19.516+0000 7f41741fd740 1 -- 172.21.15.22:0/2138781736 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ef1cee3290 msgr2=0x55ef1cee5680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:15:19.520 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:15:19.516+0000 7f41741fd740 1 --2- 172.21.15.22:0/2138781736 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ef1cee3290 0x55ef1cee5680 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7f41540cc910 tx=0x7f41540d7670 comp rx=0 tx=0).stop 2024-03-28T18:15:19.520 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:15:19.516+0000 7f416b400640 1 --2- 172.21.15.22:0/3968897138 >> [v2:172.21.15.22:6800/1357365082,v1:172.21.15.22:6801/1357365082] conn(0x7f41500a2970 0x7f41500a4e30 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f416402c700 tx=0x7f4164004a40 comp rx=0 tx=0).ready entity=mgr.15522 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:15:19.520 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:15:19.516+0000 7f41741fd740 1 -- 172.21.15.22:0/2138781736 shutdown_connections 2024-03-28T18:15:19.520 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:15:19.516+0000 7f41741fd740 1 --2- 172.21.15.22:0/2138781736 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ef1cee3290 0x55ef1cee5680 unknown :-1 s=CLOSED pgs=754 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:15:19.521 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:15:19.516+0000 7f41741fd740 1 --2- 172.21.15.22:0/2138781736 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ef1ced9f20 0x55ef1ceda3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:15:19.521 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:15:19.516+0000 7f41741fd740 1 --2- 172.21.15.22:0/2138781736 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55ef1cedac00 0x55ef1ced9950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:15:19.521 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:15:19.516+0000 7f41741fd740 1 -- 172.21.15.22:0/2138781736 >> 172.21.15.22:0/2138781736 conn(0x55ef1cea1200 msgr2=0x55ef1ced4970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:15:19.521 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:15:19.517+0000 7f41741fd740 1 -- 172.21.15.22:0/2138781736 shutdown_connections 2024-03-28T18:15:19.521 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:15:19.517+0000 7f41741fd740 1 -- 172.21.15.22:0/2138781736 wait complete. 2024-03-28T18:15:19.521 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:15:19.517+0000 7f41741fd740 1 -- start start 2024-03-28T18:15:19.521 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:15:19.517+0000 7f41741fd740 1 Processor -- start 2024-03-28T18:15:19.521 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:15:19.518+0000 7f41741fd740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ef1ced9f20 0x55ef1cee2970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:15:19.521 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:15:19.518+0000 7f41741fd740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ef1cedac00 0x55ef1cedfd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:15:19.521 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:15:19.518+0000 7f41741fd740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55ef1cee3290 0x55ef1cee0240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:15:19.521 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:15:19.518+0000 7f41741fd740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ef1cee6100 con 0x55ef1cedac00 2024-03-28T18:15:19.521 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:15:19.518+0000 7f41741fd740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ef1cdf9e80 con 0x55ef1cee3290 2024-03-28T18:15:19.521 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:15:19.518+0000 7f41741fd740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ef1cee6280 con 0x55ef1ced9f20 2024-03-28T18:15:19.521 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:15:19.518+0000 7f41741fd740 10 client.0 _ll_register_callbacks cb 0x55ef1ce0c130 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-28T18:15:19.521 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:15:19.518+0000 7f416a000640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ef1cedac00 0x55ef1cedfd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:15:19.522 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:15:19.518+0000 7f416a000640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ef1cedac00 0x55ef1cedfd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:36020/0 (socket says 172.21.15.22:36020) 2024-03-28T18:15:19.522 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:15:19.518+0000 7f416a000640 1 -- 172.21.15.22:0/682946451 learned_addr learned my addr 172.21.15.22:0/682946451 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-28T18:15:19.522 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:15:19.518+0000 7f416aa00640 1 --2- 172.21.15.22:0/682946451 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ef1ced9f20 0x55ef1cee2970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:15:19.522 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:15:19.519+0000 7f416b400640 1 --2- 172.21.15.22:0/682946451 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55ef1cee3290 0x55ef1cee0240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:15:19.522 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:15:19.519+0000 7f416a000640 1 -- 172.21.15.22:0/682946451 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ef1ced9f20 msgr2=0x55ef1cee2970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T18:15:19.522 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:15:19.519+0000 7f416a000640 1 --2- 172.21.15.22:0/682946451 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ef1ced9f20 0x55ef1cee2970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:15:19.523 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:15:19.519+0000 7f416a000640 1 -- 172.21.15.22:0/682946451 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55ef1cee3290 msgr2=0x55ef1cee0240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T18:15:19.523 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:15:19.519+0000 7f416a000640 1 --2- 172.21.15.22:0/682946451 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55ef1cee3290 0x55ef1cee0240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:15:19.523 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:15:19.519+0000 7f416a000640 1 -- 172.21.15.22:0/682946451 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55ef1cc1c7d0 con 0x55ef1cedac00 2024-03-28T18:15:19.523 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:15:19.519+0000 7f416a000640 1 --2- 172.21.15.22:0/682946451 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ef1cedac00 0x55ef1cedfd00 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7f41540d7b70 tx=0x7f4154009700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=292a80a6a61f1127 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:15:19.523 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:15:19.519+0000 7f414fe00640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-03-28T18:15:19.523 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:15:19.519+0000 7f414fe00640 1 -- 172.21.15.22:0/682946451 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41540cf030 con 0x55ef1cedac00 2024-03-28T18:15:19.523 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:15:19.519+0000 7f414fe00640 1 -- 172.21.15.22:0/682946451 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(113..113 src has 1..113) v4 ==== 9959+0+0 (secure 0 0 0) 0x7f415409f760 con 0x55ef1cedac00 2024-03-28T18:15:19.523 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:15:19.519+0000 7f41741fd740 1 -- 172.21.15.22:0/682946451 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f415409bfe0 con 0x55ef1cedac00 2024-03-28T18:15:19.523 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:15:19.520+0000 7f414fe00640 1 -- 172.21.15.22:0/682946451 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=150) v2 ==== 24+0+0 (secure 0 0 0) 0x7f41540de070 con 0x55ef1cedac00 2024-03-28T18:15:19.523 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:15:19.520+0000 7f41741fd740 10 client.16722 fetch_fsmap learned FSMap version 150 2024-03-28T18:15:19.523 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:15:19.520+0000 7f41741fd740 1 -- 172.21.15.22:0/682946451 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=150,monmap=4+}) v3 -- 0x55ef1cb523f0 con 0x55ef1cedac00 2024-03-28T18:15:19.523 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:15:19.520+0000 7f414fe00640 1 -- 172.21.15.22:0/682946451 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f415409bfe0 con 0x55ef1cedac00 2024-03-28T18:15:19.523 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:15:19.520+0000 7f414fe00640 1 -- 172.21.15.22:0/682946451 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 150) v1 ==== 6788+0+0 (secure 0 0 0) 0x7f41540ccc50 con 0x55ef1cedac00 2024-03-28T18:15:19.525 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:15:19.520+0000 7f41741fd740 10 client.16722 fetch_fsmap finished waiting for FSMap version 150 2024-03-28T18:15:19.525 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:15:19.520+0000 7f41741fd740 10 client.16722 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-28T18:15:19.525 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:15:19.521+0000 7f41741fd740 20 client.16722 populate_metadata read hostname 'smithi022' 2024-03-28T18:15:19.525 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:15:19.521+0000 7f41741fd740 1 --2- 172.21.15.22:0/682946451 >> [v2:172.21.15.79:6838/2441542443,v1:172.21.15.79:6839/2441542443] conn(0x55ef1cef3030 0x55ef1cef5420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:15:19.525 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:15:19.521+0000 7f41741fd740 4 client.16722 mds_command: new command op to 34553 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-28T18:15:19.525 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:15:19.521+0000 7f41741fd740 1 -- 172.21.15.22:0/682946451 --> [v2:172.21.15.79:6838/2441542443,v1:172.21.15.79:6839/2441542443] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55ef1ce4b790 con 0x55ef1cef3030 2024-03-28T18:15:19.525 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:15:19.521+0000 7f416aa00640 1 --2- 172.21.15.22:0/682946451 >> [v2:172.21.15.79:6838/2441542443,v1:172.21.15.79:6839/2441542443] conn(0x55ef1cef3030 0x55ef1cef5420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:15:19.525 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:15:19.522+0000 7f414fe00640 10 client.16722 ms_handle_connect on v2:172.21.15.79:6838/2441542443 2024-03-28T18:15:19.525 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:15:19.522+0000 7f416aa00640 1 --2- 172.21.15.22:0/682946451 >> [v2:172.21.15.79:6838/2441542443,v1:172.21.15.79:6839/2441542443] conn(0x55ef1cef3030 0x55ef1cef5420 crc :-1 s=READY pgs=235 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=561f1cf5d0d1a00c server_cookie=be67c0c7a1a6e250 in_seq=0 out_seq=0 2024-03-28T18:15:19.526 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:15:19.523+0000 7f414fe00640 1 -- 172.21.15.22:0/682946451 <== mds.0 v2:172.21.15.79:6838/2441542443 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x55ef1ce4b790 con 0x55ef1cef3030 2024-03-28T18:15:19.526 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:15:19.523+0000 7f414fe00640 10 client.16722 handle_command_reply: tid=0 2024-03-28T18:15:19.526 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:15:19.523+0000 7f41741fd740 1 client.16722 shutdown 2024-03-28T18:15:19.526 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:15:19.524+0000 7f41741fd740 1 -- 172.21.15.22:0/682946451 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ef1cedac00 msgr2=0x55ef1cedfd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:15:19.526 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:15:19.524+0000 7f41741fd740 1 --2- 172.21.15.22:0/682946451 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ef1cedac00 0x55ef1cedfd00 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7f41540d7b70 tx=0x7f4154009700 comp rx=0 tx=0).stop 2024-03-28T18:15:19.526 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:15:19.524+0000 7f41741fd740 1 -- 172.21.15.22:0/682946451 shutdown_connections 2024-03-28T18:15:19.526 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:15:19.524+0000 7f41741fd740 1 --2- 172.21.15.22:0/682946451 >> [v2:172.21.15.79:6838/2441542443,v1:172.21.15.79:6839/2441542443] conn(0x55ef1cef3030 0x55ef1cef5420 crc :-1 s=READY pgs=235 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:15:19.526 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:15:19.524+0000 7f41741fd740 1 --2- 172.21.15.22:0/682946451 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55ef1cee3290 0x55ef1cee0240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:15:19.526 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:15:19.524+0000 7f41741fd740 1 --2- 172.21.15.22:0/682946451 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ef1cedac00 0x55ef1cedfd00 unknown :-1 s=CLOSED pgs=755 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:15:19.526 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:15:19.524+0000 7f41741fd740 1 --2- 172.21.15.22:0/682946451 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ef1ced9f20 0x55ef1cee2970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:15:19.526 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:15:19.524+0000 7f41741fd740 1 -- 172.21.15.22:0/682946451 >> 172.21.15.22:0/682946451 conn(0x55ef1cea1200 msgr2=0x55ef1ced66c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:15:19.527 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:15:19.524+0000 7f414fe00640 0 client.16722 ms_handle_reset on v2:172.21.15.79:6838/2441542443 2024-03-28T18:15:19.527 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:15:19.525+0000 7f41741fd740 1 -- 172.21.15.22:0/682946451 shutdown_connections 2024-03-28T18:15:19.527 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:15:19.525+0000 7f41741fd740 1 -- 172.21.15.22:0/682946451 wait complete. 2024-03-28T18:15:19.527 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:15:19.525+0000 7f41741fd740 20 client.16722 trim_cache size 0 max 16384 2024-03-28T18:15:19.543 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:15:19.543+0000 7f41741fd740 1 Processor -- start 2024-03-28T18:15:19.543 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:15:19.543+0000 7f41741fd740 1 -- start start 2024-03-28T18:15:19.543 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:15:19.543+0000 7f41741fd740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ef1ced9f20 0x55ef1cee2430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:15:19.543 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:15:19.543+0000 7f41741fd740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ef1cedac00 0x55ef1cee2970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:15:19.543 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:15:19.543+0000 7f41741fd740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55ef1cee3290 0x55ef1cee4680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:15:19.543 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:15:19.543+0000 7f41741fd740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f416c1b3680 con 0x55ef1cedac00 2024-03-28T18:15:19.543 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:15:19.543+0000 7f41741fd740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f416c1b5660 con 0x55ef1cee3290 2024-03-28T18:15:19.543 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:15:19.543+0000 7f41741fd740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f416c1b3ce0 con 0x55ef1ced9f20 2024-03-28T18:15:19.545 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:15:19.543+0000 7f416a000640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ef1cedac00 0x55ef1cee2970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:15:19.545 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:15:19.543+0000 7f416a000640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ef1cedac00 0x55ef1cee2970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:36022/0 (socket says 172.21.15.22:36022) 2024-03-28T18:15:19.545 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:15:19.543+0000 7f416a000640 1 -- 172.21.15.22:0/1323231202 learned_addr learned my addr 172.21.15.22:0/1323231202 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-28T18:15:19.545 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:15:19.544+0000 7f416b400640 1 --2- 172.21.15.22:0/1323231202 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55ef1cee3290 0x55ef1cee4680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:15:19.545 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:15:19.544+0000 7f416aa00640 1 --2- 172.21.15.22:0/1323231202 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ef1ced9f20 0x55ef1cee2430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:15:19.545 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:15:19.544+0000 7f416a000640 1 -- 172.21.15.22:0/1323231202 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ef1ced9f20 msgr2=0x55ef1cee2430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T18:15:19.545 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:15:19.544+0000 7f416a000640 1 --2- 172.21.15.22:0/1323231202 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ef1ced9f20 0x55ef1cee2430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:15:19.546 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:15:19.544+0000 7f416a000640 1 -- 172.21.15.22:0/1323231202 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55ef1cee3290 msgr2=0x55ef1cee4680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T18:15:19.546 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:15:19.544+0000 7f416a000640 1 --2- 172.21.15.22:0/1323231202 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55ef1cee3290 0x55ef1cee4680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:15:19.546 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:15:19.544+0000 7f416a000640 1 -- 172.21.15.22:0/1323231202 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ef1cb120f0 con 0x55ef1cedac00 2024-03-28T18:15:19.546 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:15:19.544+0000 7f416b400640 1 --2- 172.21.15.22:0/1323231202 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55ef1cee3290 0x55ef1cee4680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-28T18:15:19.546 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:15:19.544+0000 7f416a000640 1 --2- 172.21.15.22:0/1323231202 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ef1cedac00 0x55ef1cee2970 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7f415400c6a0 tx=0x7f4154097cb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d794161f1825bcd server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:15:19.546 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:15:19.544+0000 7f415aa00640 1 -- 172.21.15.22:0/1323231202 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41540cf070 con 0x55ef1cedac00 2024-03-28T18:15:19.546 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:15:19.545+0000 7f415aa00640 1 -- 172.21.15.22:0/1323231202 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f415409bfe0 con 0x55ef1cedac00 2024-03-28T18:15:19.546 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:15:19.545+0000 7f415aa00640 1 -- 172.21.15.22:0/1323231202 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41540d2e60 con 0x55ef1cedac00 2024-03-28T18:15:19.546 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:15:19.545+0000 7f41741fd740 1 -- 172.21.15.22:0/1323231202 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ef1cedac00 msgr2=0x55ef1cee2970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:15:19.546 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:15:19.545+0000 7f41741fd740 1 --2- 172.21.15.22:0/1323231202 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ef1cedac00 0x55ef1cee2970 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7f415400c6a0 tx=0x7f4154097cb0 comp rx=0 tx=0).stop 2024-03-28T18:15:19.546 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:15:19.545+0000 7f41741fd740 1 -- 172.21.15.22:0/1323231202 shutdown_connections 2024-03-28T18:15:19.546 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:15:19.545+0000 7f41741fd740 1 --2- 172.21.15.22:0/1323231202 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55ef1cee3290 0x55ef1cee4680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:15:19.546 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:15:19.545+0000 7f41741fd740 1 --2- 172.21.15.22:0/1323231202 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ef1cedac00 0x55ef1cee2970 unknown :-1 s=CLOSED pgs=756 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:15:19.546 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:15:19.545+0000 7f41741fd740 1 --2- 172.21.15.22:0/1323231202 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ef1ced9f20 0x55ef1cee2430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:15:19.546 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:15:19.545+0000 7f41741fd740 1 -- 172.21.15.22:0/1323231202 >> 172.21.15.22:0/1323231202 conn(0x55ef1cea1200 msgr2=0x55ef1ced44a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:15:19.547 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:15:19.546+0000 7f41741fd740 1 -- 172.21.15.22:0/1323231202 shutdown_connections 2024-03-28T18:15:19.547 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:15:19.546+0000 7f41741fd740 1 -- 172.21.15.22:0/1323231202 wait complete. 2024-03-28T18:15:19.549 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:15:19.546+0000 7f41741fd740 1 -- start start 2024-03-28T18:15:19.549 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:15:19.546+0000 7f41741fd740 1 Processor -- start 2024-03-28T18:15:19.549 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:15:19.547+0000 7f41741fd740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ef1ced9f20 0x55ef1cedc530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:15:19.550 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:15:19.547+0000 7f41741fd740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ef1cedac00 0x55ef1cedca70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:15:19.550 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:15:19.547+0000 7f41741fd740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55ef1cee3290 0x55ef1cedcfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:15:19.550 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:15:19.547+0000 7f41741fd740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f416c1b4340 con 0x55ef1ced9f20 2024-03-28T18:15:19.550 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:15:19.547+0000 7f41741fd740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f416c1b5000 con 0x55ef1cee3290 2024-03-28T18:15:19.550 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:15:19.547+0000 7f41741fd740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f416c1b49a0 con 0x55ef1cedac00 2024-03-28T18:15:19.550 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:15:19.547+0000 7f41741fd740 10 client.0 _ll_register_callbacks cb 0x7f4164037070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-28T18:15:19.550 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:15:19.547+0000 7f416aa00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ef1ced9f20 0x55ef1cedc530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:15:19.550 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:15:19.547+0000 7f416aa00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ef1ced9f20 0x55ef1cedc530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:36026/0 (socket says 172.21.15.22:36026) 2024-03-28T18:15:19.550 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:15:19.547+0000 7f416aa00640 1 -- 172.21.15.22:0/3112870233 learned_addr learned my addr 172.21.15.22:0/3112870233 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-28T18:15:19.550 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:15:19.547+0000 7f416b400640 1 --2- 172.21.15.22:0/3112870233 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55ef1cee3290 0x55ef1cedcfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:15:19.550 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:15:19.547+0000 7f416a000640 1 --2- 172.21.15.22:0/3112870233 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ef1cedac00 0x55ef1cedca70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:15:19.550 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:15:19.547+0000 7f416aa00640 1 -- 172.21.15.22:0/3112870233 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ef1cedac00 msgr2=0x55ef1cedca70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T18:15:19.550 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:15:19.547+0000 7f416aa00640 1 --2- 172.21.15.22:0/3112870233 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ef1cedac00 0x55ef1cedca70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:15:19.550 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:15:19.547+0000 7f416aa00640 1 -- 172.21.15.22:0/3112870233 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55ef1cee3290 msgr2=0x55ef1cedcfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T18:15:19.551 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:15:19.547+0000 7f416aa00640 1 --2- 172.21.15.22:0/3112870233 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55ef1cee3290 0x55ef1cedcfb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:15:19.551 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:15:19.547+0000 7f416aa00640 1 -- 172.21.15.22:0/3112870233 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55ef1cf4d6e0 con 0x55ef1ced9f20 2024-03-28T18:15:19.551 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:15:19.547+0000 7f416a000640 1 --2- 172.21.15.22:0/3112870233 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ef1cedac00 0x55ef1cedca70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T18:15:19.551 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:15:19.548+0000 7f416aa00640 1 --2- 172.21.15.22:0/3112870233 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ef1ced9f20 0x55ef1cedc530 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x55ef1cef5960 tx=0x7f415c020b40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=67a53cbc03d974e7 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:15:19.551 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:15:19.548+0000 7f414fe00640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-03-28T18:15:19.551 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:15:19.548+0000 7f414fe00640 1 -- 172.21.15.22:0/3112870233 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f415c0202f0 con 0x55ef1ced9f20 2024-03-28T18:15:19.551 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:15:19.548+0000 7f414fe00640 1 -- 172.21.15.22:0/3112870233 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(113..113 src has 1..113) v4 ==== 9959+0+0 (secure 0 0 0) 0x7f415c012980 con 0x55ef1ced9f20 2024-03-28T18:15:19.551 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:15:19.548+0000 7f41741fd740 1 -- 172.21.15.22:0/3112870233 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f415409bfe0 con 0x55ef1ced9f20 2024-03-28T18:15:19.551 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:15:19.548+0000 7f414fe00640 1 -- 172.21.15.22:0/3112870233 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=150) v2 ==== 24+0+0 (secure 0 0 0) 0x7f415c031050 con 0x55ef1ced9f20 2024-03-28T18:15:19.551 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:15:19.548+0000 7f41741fd740 10 client.16728 fetch_fsmap learned FSMap version 150 2024-03-28T18:15:19.551 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:15:19.548+0000 7f41741fd740 1 -- 172.21.15.22:0/3112870233 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=150,monmap=4+}) v3 -- 0x55ef1cf4ea10 con 0x55ef1ced9f20 2024-03-28T18:15:19.551 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:15:19.549+0000 7f414fe00640 1 -- 172.21.15.22:0/3112870233 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f415409bfe0 con 0x55ef1ced9f20 2024-03-28T18:15:19.551 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:15:19.549+0000 7f414fe00640 1 -- 172.21.15.22:0/3112870233 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 150) v1 ==== 6788+0+0 (secure 0 0 0) 0x7f415c011440 con 0x55ef1ced9f20 2024-03-28T18:15:19.551 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:15:19.549+0000 7f41741fd740 10 client.16728 fetch_fsmap finished waiting for FSMap version 150 2024-03-28T18:15:19.551 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:15:19.549+0000 7f41741fd740 10 client.16728 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-28T18:15:19.551 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:15:19.549+0000 7f41741fd740 20 client.16728 populate_metadata read hostname 'smithi022' 2024-03-28T18:15:19.551 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:15:19.549+0000 7f41741fd740 1 --2- 172.21.15.22:0/3112870233 >> [v2:172.21.15.79:6838/2441542443,v1:172.21.15.79:6839/2441542443] conn(0x55ef1cf4f190 0x55ef1cf515b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:15:19.552 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:15:19.549+0000 7f41741fd740 4 client.16728 mds_command: new command op to 34553 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-28T18:15:19.552 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:15:19.549+0000 7f41741fd740 1 -- 172.21.15.22:0/3112870233 --> [v2:172.21.15.79:6838/2441542443,v1:172.21.15.79:6839/2441542443] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55ef1ce45ae0 con 0x55ef1cf4f190 2024-03-28T18:15:19.552 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:15:19.550+0000 7f416a000640 1 --2- 172.21.15.22:0/3112870233 >> [v2:172.21.15.79:6838/2441542443,v1:172.21.15.79:6839/2441542443] conn(0x55ef1cf4f190 0x55ef1cf515b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:15:19.553 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:15:19.550+0000 7f416a000640 1 --2- 172.21.15.22:0/3112870233 >> [v2:172.21.15.79:6838/2441542443,v1:172.21.15.79:6839/2441542443] conn(0x55ef1cf4f190 0x55ef1cf515b0 crc :-1 s=READY pgs=236 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b72292ed2130888c server_cookie=3d2e86f87e84ef63 in_seq=0 out_seq=0 2024-03-28T18:15:19.553 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:15:19.550+0000 7f414fe00640 10 client.16728 ms_handle_connect on v2:172.21.15.79:6838/2441542443 2024-03-28T18:15:19.553 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:15:19.550+0000 7f414fe00640 1 -- 172.21.15.22:0/3112870233 <== mds.0 v2:172.21.15.79:6838/2441542443 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x55ef1ce45ae0 con 0x55ef1cf4f190 2024-03-28T18:15:19.553 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:15:19.550+0000 7f414fe00640 10 client.16728 handle_command_reply: tid=0 2024-03-28T18:15:19.553 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:15:19.551+0000 7f41741fd740 1 client.16728 shutdown 2024-03-28T18:15:19.553 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:15:19.551+0000 7f41741fd740 1 -- 172.21.15.22:0/3112870233 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ef1ced9f20 msgr2=0x55ef1cedc530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:15:19.553 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:15:19.551+0000 7f41741fd740 1 --2- 172.21.15.22:0/3112870233 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ef1ced9f20 0x55ef1cedc530 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x55ef1cef5960 tx=0x7f415c020b40 comp rx=0 tx=0).stop 2024-03-28T18:15:19.553 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:15:19.551+0000 7f41741fd740 1 -- 172.21.15.22:0/3112870233 shutdown_connections 2024-03-28T18:15:19.553 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:15:19.551+0000 7f41741fd740 1 --2- 172.21.15.22:0/3112870233 >> [v2:172.21.15.79:6838/2441542443,v1:172.21.15.79:6839/2441542443] conn(0x55ef1cf4f190 0x55ef1cf515b0 crc :-1 s=READY pgs=236 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:15:19.553 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:15:19.551+0000 7f41741fd740 1 --2- 172.21.15.22:0/3112870233 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55ef1cee3290 0x55ef1cedcfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:15:19.553 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:15:19.551+0000 7f41741fd740 1 --2- 172.21.15.22:0/3112870233 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ef1cedac00 0x55ef1cedca70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:15:19.553 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:15:19.551+0000 7f41741fd740 1 --2- 172.21.15.22:0/3112870233 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ef1ced9f20 0x55ef1cedc530 unknown :-1 s=CLOSED pgs=757 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:15:19.554 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:15:19.551+0000 7f414fe00640 0 client.16728 ms_handle_reset on v2:172.21.15.79:6838/2441542443 2024-03-28T18:15:19.554 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:15:19.551+0000 7f41741fd740 1 -- 172.21.15.22:0/3112870233 >> 172.21.15.22:0/3112870233 conn(0x55ef1cea1200 msgr2=0x55ef1ced3200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:15:19.554 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:15:19.551+0000 7f41741fd740 1 -- 172.21.15.22:0/3112870233 shutdown_connections 2024-03-28T18:15:19.554 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:15:19.551+0000 7f41741fd740 1 -- 172.21.15.22:0/3112870233 wait complete. 2024-03-28T18:15:19.554 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:15:19.551+0000 7f41741fd740 20 client.16728 trim_cache size 0 max 16384 2024-03-28T18:15:19.554 INFO:teuthology.orchestra.run.smithi022.stdout:{ 2024-03-28T18:15:19.554 INFO:teuthology.orchestra.run.smithi022.stdout: "status": "IDLE (waiting for more scrubs)", 2024-03-28T18:15:19.554 INFO:teuthology.orchestra.run.smithi022.stdout: "scrubs": { 2024-03-28T18:15:19.554 INFO:teuthology.orchestra.run.smithi022.stdout: "4bbbd61d-29a3-4bf6-a4f2-d5783d585de3": { 2024-03-28T18:15:19.555 INFO:teuthology.orchestra.run.smithi022.stdout: "path": "/", 2024-03-28T18:15:19.555 INFO:teuthology.orchestra.run.smithi022.stdout: "tag": "4bbbd61d-29a3-4bf6-a4f2-d5783d585de3", 2024-03-28T18:15:19.555 INFO:teuthology.orchestra.run.smithi022.stdout: "options": "recursive,force" 2024-03-28T18:15:19.555 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-03-28T18:15:19.555 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-03-28T18:15:19.555 INFO:teuthology.orchestra.run.smithi022.stdout:} 2024-03-28T18:15:19.555 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:15:19.552+0000 7f4171600640 1 -- 172.21.15.22:0/3968897138 >> [v2:172.21.15.22:6800/1357365082,v1:172.21.15.22:6801/1357365082] conn(0x7f41500a2970 msgr2=0x7f41500a4e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:15:19.555 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:15:19.552+0000 7f4171600640 1 --2- 172.21.15.22:0/3968897138 >> [v2:172.21.15.22:6800/1357365082,v1:172.21.15.22:6801/1357365082] conn(0x7f41500a2970 0x7f41500a4e30 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f416402c700 tx=0x7f4164004a40 comp rx=0 tx=0).stop 2024-03-28T18:15:19.556 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:15:19.552+0000 7f4171600640 1 -- 172.21.15.22:0/3968897138 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f416c0764f0 msgr2=0x7f416c19eb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:15:19.556 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:15:19.552+0000 7f4171600640 1 --2- 172.21.15.22:0/3968897138 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f416c0764f0 0x7f416c19eb80 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7f415400e9f0 tx=0x7f415400eeb0 comp rx=0 tx=0).stop 2024-03-28T18:15:19.556 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:15:19.552+0000 7f4171600640 1 -- 172.21.15.22:0/3968897138 shutdown_connections 2024-03-28T18:15:19.556 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:15:19.552+0000 7f4171600640 1 --2- 172.21.15.22:0/3968897138 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f416c1124e0 0x7f416c1a2eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:15:19.556 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:15:19.552+0000 7f4171600640 1 --2- 172.21.15.22:0/3968897138 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f416c0764f0 0x7f416c19eb80 unknown :-1 s=CLOSED pgs=753 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:15:19.556 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:15:19.552+0000 7f4171600640 1 --2- 172.21.15.22:0/3968897138 >> [v2:172.21.15.22:6800/1357365082,v1:172.21.15.22:6801/1357365082] conn(0x7f41500a2970 0x7f41500a4e30 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:15:19.556 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:15:19.552+0000 7f4171600640 1 --2- 172.21.15.22:0/3968897138 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f416c075bd0 0x7f416c19e640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:15:19.556 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:15:19.552+0000 7f4171600640 1 -- 172.21.15.22:0/3968897138 >> 172.21.15.22:0/3968897138 conn(0x7f416c0ff2f0 msgr2=0x7f416c1016e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:15:19.556 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:15:19.552+0000 7f4171600640 1 -- 172.21.15.22:0/3968897138 shutdown_connections 2024-03-28T18:15:19.556 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:15:19.552+0000 7f4171600640 1 -- 172.21.15.22:0/3968897138 wait complete. 2024-03-28T18:15:19.569 INFO:tasks.cephfs.filesystem:scrub status for tag:4bbbd61d-29a3-4bf6-a4f2-d5783d585de3 - {'path': '/', 'tag': '4bbbd61d-29a3-4bf6-a4f2-d5783d585de3', 'options': 'recursive,force'} 2024-03-28T18:15:19.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:15:19 smithi022 ceph-mon[31985]: daemon mds.j restarted 2024-03-28T18:15:19.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:15:19 smithi022 ceph-mon[31985]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-03-28T18:15:19.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:15:19 smithi022 ceph-mon[31985]: osdmap e113: 12 total, 12 up, 12 in 2024-03-28T18:15:19.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:15:19 smithi022 ceph-mon[31985]: mds.? [v2:172.21.15.22:6834/3637756693,v1:172.21.15.22:6835/3637756693] up:boot 2024-03-28T18:15:19.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:15:19 smithi022 ceph-mon[31985]: Standby daemon mds.d assigned to filesystem cephfs as rank 1 2024-03-28T18:15:19.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:15:19 smithi022 ceph-mon[31985]: fsmap cephfs:4/5 {0=b=up:active,2=e=up:active,3=k=up:active,4=i=up:active} 5 up:standby-replay 1 up:standby, 1 failed 2024-03-28T18:15:19.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:15:19 smithi022 ceph-mon[31985]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-03-28T18:15:19.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:15:19 smithi022 ceph-mon[31985]: fsmap cephfs:5/5 4 up:standby-replay 1 up:standby 1 up:replay 4 up:active 2024-03-28T18:15:19.925 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:15:19 smithi134 ceph-mon[35037]: daemon mds.j restarted 2024-03-28T18:15:19.925 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:15:19 smithi134 ceph-mon[35037]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-03-28T18:15:19.925 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:15:19 smithi134 ceph-mon[35037]: osdmap e113: 12 total, 12 up, 12 in 2024-03-28T18:15:19.925 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:15:19 smithi134 ceph-mon[35037]: mds.? [v2:172.21.15.22:6834/3637756693,v1:172.21.15.22:6835/3637756693] up:boot 2024-03-28T18:15:19.925 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:15:19 smithi134 ceph-mon[35037]: Standby daemon mds.d assigned to filesystem cephfs as rank 1 2024-03-28T18:15:19.926 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:15:19 smithi134 ceph-mon[35037]: fsmap cephfs:4/5 {0=b=up:active,2=e=up:active,3=k=up:active,4=i=up:active} 5 up:standby-replay 1 up:standby, 1 failed 2024-03-28T18:15:19.926 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:15:19 smithi134 ceph-mon[35037]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-03-28T18:15:19.926 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:15:19 smithi134 ceph-mon[35037]: fsmap cephfs:5/5 4 up:standby-replay 1 up:standby 1 up:replay 4 up:active 2024-03-28T18:15:19.933 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:15:19 smithi079 ceph-mon[35312]: daemon mds.j restarted 2024-03-28T18:15:19.933 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:15:19 smithi079 ceph-mon[35312]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-03-28T18:15:19.933 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:15:19 smithi079 ceph-mon[35312]: osdmap e113: 12 total, 12 up, 12 in 2024-03-28T18:15:19.933 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:15:19 smithi079 ceph-mon[35312]: mds.? [v2:172.21.15.22:6834/3637756693,v1:172.21.15.22:6835/3637756693] up:boot 2024-03-28T18:15:19.933 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:15:19 smithi079 ceph-mon[35312]: Standby daemon mds.d assigned to filesystem cephfs as rank 1 2024-03-28T18:15:19.934 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:15:19 smithi079 ceph-mon[35312]: fsmap cephfs:4/5 {0=b=up:active,2=e=up:active,3=k=up:active,4=i=up:active} 5 up:standby-replay 1 up:standby, 1 failed 2024-03-28T18:15:19.934 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:15:19 smithi079 ceph-mon[35312]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-03-28T18:15:19.934 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:15:19 smithi079 ceph-mon[35312]: fsmap cephfs:5/5 4 up:standby-replay 1 up:standby 1 up:replay 4 up:active 2024-03-28T18:15:21.175 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:15:20 smithi134 ceph-mon[35037]: pgmap v256: 97 pgs: 97 active+clean; 117 MiB data, 794 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 10 op/s 2024-03-28T18:15:21.175 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:15:20 smithi134 ceph-mon[35037]: mds.? [v2:172.21.15.22:6836/488832747,v1:172.21.15.22:6837/488832747] up:resolve 2024-03-28T18:15:21.175 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:15:20 smithi134 ceph-mon[35037]: fsmap cephfs:5/5 4 up:standby-replay 1 up:standby 1 up:resolve 4 up:active 2024-03-28T18:15:21.182 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:15:20 smithi079 ceph-mon[35312]: pgmap v256: 97 pgs: 97 active+clean; 117 MiB data, 794 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 10 op/s 2024-03-28T18:15:21.182 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:15:20 smithi079 ceph-mon[35312]: mds.? [v2:172.21.15.22:6836/488832747,v1:172.21.15.22:6837/488832747] up:resolve 2024-03-28T18:15:21.182 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:15:20 smithi079 ceph-mon[35312]: fsmap cephfs:5/5 4 up:standby-replay 1 up:standby 1 up:resolve 4 up:active 2024-03-28T18:15:21.282 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:15:20 smithi022 ceph-mon[31985]: pgmap v256: 97 pgs: 97 active+clean; 117 MiB data, 794 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 10 op/s 2024-03-28T18:15:21.282 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:15:20 smithi022 ceph-mon[31985]: mds.? [v2:172.21.15.22:6836/488832747,v1:172.21.15.22:6837/488832747] up:resolve 2024-03-28T18:15:21.282 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:15:20 smithi022 ceph-mon[31985]: fsmap cephfs:5/5 4 up:standby-replay 1 up:standby 1 up:resolve 4 up:active 2024-03-28T18:15:22.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:15:22 smithi022 ceph-mon[31985]: pgmap v257: 97 pgs: 97 active+clean; 117 MiB data, 794 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 10 op/s 2024-03-28T18:15:22.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:15:22 smithi022 ceph-mon[31985]: reconnect by client.35222 172.21.15.22:0/3729116781 after 0.00499993 2024-03-28T18:15:22.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:15:22 smithi022 ceph-mon[31985]: mds.? [v2:172.21.15.22:6836/488832747,v1:172.21.15.22:6837/488832747] up:reconnect 2024-03-28T18:15:22.783 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:15:22 smithi022 ceph-mon[31985]: fsmap cephfs:5/5 4 up:standby-replay 1 up:standby 1 up:reconnect 4 up:active 2024-03-28T18:15:22.925 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:15:22 smithi134 ceph-mon[35037]: pgmap v257: 97 pgs: 97 active+clean; 117 MiB data, 794 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 10 op/s 2024-03-28T18:15:22.925 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:15:22 smithi134 ceph-mon[35037]: reconnect by client.35222 172.21.15.22:0/3729116781 after 0.00499993 2024-03-28T18:15:22.925 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:15:22 smithi134 ceph-mon[35037]: mds.? [v2:172.21.15.22:6836/488832747,v1:172.21.15.22:6837/488832747] up:reconnect 2024-03-28T18:15:22.925 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:15:22 smithi134 ceph-mon[35037]: fsmap cephfs:5/5 4 up:standby-replay 1 up:standby 1 up:reconnect 4 up:active 2024-03-28T18:15:22.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:15:22 smithi079 ceph-mon[35312]: pgmap v257: 97 pgs: 97 active+clean; 117 MiB data, 794 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 10 op/s 2024-03-28T18:15:22.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:15:22 smithi079 ceph-mon[35312]: reconnect by client.35222 172.21.15.22:0/3729116781 after 0.00499993 2024-03-28T18:15:22.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:15:22 smithi079 ceph-mon[35312]: mds.? [v2:172.21.15.22:6836/488832747,v1:172.21.15.22:6837/488832747] up:reconnect 2024-03-28T18:15:22.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:15:22 smithi079 ceph-mon[35312]: fsmap cephfs:5/5 4 up:standby-replay 1 up:standby 1 up:reconnect 4 up:active 2024-03-28T18:15:23.283 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:15:23 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[94323]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-2145-g3be9a5c9/rpm/el8/BUILD/ceph-19.0.0-2145-g3be9a5c9/src/mds/Locker.cc: In function 'void Locker::set_xlocks_done(MutationImpl*, bool)' thread 7f0b77600700 time 2024-03-28T18:15:23.028351+0000 2024-03-28T18:15:23.283 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:15:23 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[94323]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-2145-g3be9a5c9/rpm/el8/BUILD/ceph-19.0.0-2145-g3be9a5c9/src/mds/Locker.cc: 626: FAILED ceph_assert(obj->is_auth()) 2024-03-28T18:15:23.283 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:15:23 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[94323]: ceph version 19.0.0-2145-g3be9a5c9 (3be9a5c9bc793e11e2800a8c0c696e8b46742033) squid (dev) 2024-03-28T18:15:23.283 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:15:23 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[94323]: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x135) [0x7f0b82fb54a1] 2024-03-28T18:15:23.283 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:15:23 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[94323]: 2: /usr/lib64/ceph/libceph-common.so.2(+0x2d2667) [0x7f0b82fb5667] 2024-03-28T18:15:23.283 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:15:23 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[94323]: 3: (Locker::set_xlocks_done(MutationImpl*, bool)+0x493) [0x563045130223] 2024-03-28T18:15:23.283 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:15:23 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[94323]: 4: (Server::early_reply(boost::intrusive_ptr const&, CInode*, CDentry*)+0x3e6) [0x563044faa5d6] 2024-03-28T18:15:23.283 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:15:23 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[94323]: 5: (Server::journal_and_reply(boost::intrusive_ptr const&, CInode*, CDentry*, LogEvent*, MDSLogContextBase*)+0xc1) [0x563044faada1] 2024-03-28T18:15:23.284 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:15:23 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[94323]: 6: (Server::handle_client_mkdir(boost::intrusive_ptr const&)+0x5ba) [0x563044fe06aa] 2024-03-28T18:15:23.284 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:15:23 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[94323]: 7: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0x733) [0x563044ffd233] 2024-03-28T18:15:23.284 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:15:23 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[94323]: 8: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x563044fff197] 2024-03-28T18:15:23.284 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:15:23 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[94323]: 9: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x563044efe11b] 2024-03-28T18:15:23.284 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:15:23 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[94323]: 10: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x563044efe6bc] 2024-03-28T18:15:23.284 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:15:23 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[94323]: 11: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bf) [0x563044ee81ff] 2024-03-28T18:15:23.284 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:15:23 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[94323]: 12: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x478) [0x7f0b83247488] 2024-03-28T18:15:23.284 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:15:23 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[94323]: 13: (DispatchQueue::entry()+0x51f) [0x7f0b8324460f] 2024-03-28T18:15:23.284 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:15:23 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[94323]: 14: (DispatchQueue::DispatchThread::entry()+0x11) [0x7f0b83317e81] 2024-03-28T18:15:23.284 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:15:23 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[94323]: 15: /lib64/libpthread.so.0(+0x81ca) [0x7f0b81d321ca] 2024-03-28T18:15:23.284 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:15:23 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[94323]: 16: clone() 2024-03-28T18:15:23.285 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:15:23 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[94323]: 2024-03-28T18:15:23.029+0000 7f0b77600700 -1 /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-2145-g3be9a5c9/rpm/el8/BUILD/ceph-19.0.0-2145-g3be9a5c9/src/mds/Locker.cc: In function 'void Locker::set_xlocks_done(MutationImpl*, bool)' thread 7f0b77600700 time 2024-03-28T18:15:23.028351+0000 2024-03-28T18:15:23.285 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:15:23 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[94323]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-2145-g3be9a5c9/rpm/el8/BUILD/ceph-19.0.0-2145-g3be9a5c9/src/mds/Locker.cc: 626: FAILED ceph_assert(obj->is_auth()) 2024-03-28T18:15:23.285 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:15:23 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[94323]: 2024-03-28T18:15:23.285 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:15:23 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[94323]: ceph version 19.0.0-2145-g3be9a5c9 (3be9a5c9bc793e11e2800a8c0c696e8b46742033) squid (dev) 2024-03-28T18:15:23.285 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:15:23 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[94323]: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x135) [0x7f0b82fb54a1] 2024-03-28T18:15:23.285 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:15:23 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[94323]: 2: /usr/lib64/ceph/libceph-common.so.2(+0x2d2667) [0x7f0b82fb5667] 2024-03-28T18:15:23.285 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:15:23 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[94323]: 3: (Locker::set_xlocks_done(MutationImpl*, bool)+0x493) [0x563045130223] 2024-03-28T18:15:23.285 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:15:23 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[94323]: 4: (Server::early_reply(boost::intrusive_ptr const&, CInode*, CDentry*)+0x3e6) [0x563044faa5d6] 2024-03-28T18:15:23.285 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:15:23 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[94323]: 5: (Server::journal_and_reply(boost::intrusive_ptr const&, CInode*, CDentry*, LogEvent*, MDSLogContextBase*)+0xc1) [0x563044faada1] 2024-03-28T18:15:23.285 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:15:23 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[94323]: 6: (Server::handle_client_mkdir(boost::intrusive_ptr const&)+0x5ba) [0x563044fe06aa] 2024-03-28T18:15:23.285 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:15:23 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[94323]: 7: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0x733) [0x563044ffd233] 2024-03-28T18:15:23.285 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:15:23 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[94323]: 8: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x563044fff197] 2024-03-28T18:15:23.285 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:15:23 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[94323]: 9: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x563044efe11b] 2024-03-28T18:15:23.285 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:15:23 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[94323]: 10: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x563044efe6bc] 2024-03-28T18:15:23.286 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:15:23 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[94323]: 11: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bf) [0x563044ee81ff] 2024-03-28T18:15:23.286 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:15:23 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[94323]: 12: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x478) [0x7f0b83247488] 2024-03-28T18:15:23.286 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:15:23 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[94323]: 13: (DispatchQueue::entry()+0x51f) [0x7f0b8324460f] 2024-03-28T18:15:23.286 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:15:23 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[94323]: 14: (DispatchQueue::DispatchThread::entry()+0x11) [0x7f0b83317e81] 2024-03-28T18:15:23.286 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:15:23 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[94323]: 15: /lib64/libpthread.so.0(+0x81ca) [0x7f0b81d321ca] 2024-03-28T18:15:23.286 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:15:23 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[94323]: 16: clone() 2024-03-28T18:15:23.286 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:15:23 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[94323]: 2024-03-28T18:15:23.286 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:15:23 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[94323]: *** Caught signal (Aborted) ** 2024-03-28T18:15:23.286 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:15:23 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[94323]: in thread 7f0b77600700 thread_name:ms_dispatch 2024-03-28T18:15:23.286 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:15:23 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[94323]: ceph version 19.0.0-2145-g3be9a5c9 (3be9a5c9bc793e11e2800a8c0c696e8b46742033) squid (dev) 2024-03-28T18:15:23.286 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:15:23 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[94323]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7f0b81d3cd20] 2024-03-28T18:15:23.287 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:15:23 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[94323]: 2: gsignal() 2024-03-28T18:15:23.287 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:15:23 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[94323]: 3: abort() 2024-03-28T18:15:23.287 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:15:23 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[94323]: 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x18f) [0x7f0b82fb54fb] 2024-03-28T18:15:23.287 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:15:23 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[94323]: 5: /usr/lib64/ceph/libceph-common.so.2(+0x2d2667) [0x7f0b82fb5667] 2024-03-28T18:15:23.287 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:15:23 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[94323]: 6: (Locker::set_xlocks_done(MutationImpl*, bool)+0x493) [0x563045130223] 2024-03-28T18:15:23.287 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:15:23 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[94323]: 7: (Server::early_reply(boost::intrusive_ptr const&, CInode*, CDentry*)+0x3e6) [0x563044faa5d6] 2024-03-28T18:15:23.287 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:15:23 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[94323]: 8: (Server::journal_and_reply(boost::intrusive_ptr const&, CInode*, CDentry*, LogEvent*, MDSLogContextBase*)+0xc1) [0x563044faada1] 2024-03-28T18:15:23.287 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:15:23 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[94323]: 9: (Server::handle_client_mkdir(boost::intrusive_ptr const&)+0x5ba) [0x563044fe06aa] 2024-03-28T18:15:23.287 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:15:23 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[94323]: 10: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0x733) [0x563044ffd233] 2024-03-28T18:15:23.287 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:15:23 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[94323]: 11: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x563044fff197] 2024-03-28T18:15:23.287 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:15:23 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[94323]: 12: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x563044efe11b] 2024-03-28T18:15:23.287 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:15:23 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[94323]: 13: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x563044efe6bc] 2024-03-28T18:15:23.287 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:15:23 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[94323]: 14: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bf) [0x563044ee81ff] 2024-03-28T18:15:23.288 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:15:23 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[94323]: 15: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x478) [0x7f0b83247488] 2024-03-28T18:15:23.288 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:15:23 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[94323]: 16: (DispatchQueue::entry()+0x51f) [0x7f0b8324460f] 2024-03-28T18:15:23.288 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:15:23 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[94323]: 17: (DispatchQueue::DispatchThread::entry()+0x11) [0x7f0b83317e81] 2024-03-28T18:15:23.288 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:15:23 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[94323]: 18: /lib64/libpthread.so.0(+0x81ca) [0x7f0b81d321ca] 2024-03-28T18:15:23.288 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:15:23 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[94323]: 19: clone() 2024-03-28T18:15:23.288 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:15:23 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[94323]: 2024-03-28T18:15:23.031+0000 7f0b77600700 -1 *** Caught signal (Aborted) ** 2024-03-28T18:15:23.288 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:15:23 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[94323]: in thread 7f0b77600700 thread_name:ms_dispatch 2024-03-28T18:15:23.288 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:15:23 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[94323]: 2024-03-28T18:15:23.288 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:15:23 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[94323]: ceph version 19.0.0-2145-g3be9a5c9 (3be9a5c9bc793e11e2800a8c0c696e8b46742033) squid (dev) 2024-03-28T18:15:23.288 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:15:23 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[94323]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7f0b81d3cd20] 2024-03-28T18:15:23.288 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:15:23 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[94323]: 2: gsignal() 2024-03-28T18:15:23.288 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:15:23 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[94323]: 3: abort() 2024-03-28T18:15:23.288 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:15:23 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[94323]: 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x18f) [0x7f0b82fb54fb] 2024-03-28T18:15:23.288 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:15:23 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[94323]: 5: /usr/lib64/ceph/libceph-common.so.2(+0x2d2667) [0x7f0b82fb5667] 2024-03-28T18:15:23.289 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:15:23 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[94323]: 6: (Locker::set_xlocks_done(MutationImpl*, bool)+0x493) [0x563045130223] 2024-03-28T18:15:23.289 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:15:23 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[94323]: 7: (Server::early_reply(boost::intrusive_ptr const&, CInode*, CDentry*)+0x3e6) [0x563044faa5d6] 2024-03-28T18:15:23.289 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:15:23 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[94323]: 8: (Server::journal_and_reply(boost::intrusive_ptr const&, CInode*, CDentry*, LogEvent*, MDSLogContextBase*)+0xc1) [0x563044faada1] 2024-03-28T18:15:23.289 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:15:23 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[94323]: 9: (Server::handle_client_mkdir(boost::intrusive_ptr const&)+0x5ba) [0x563044fe06aa] 2024-03-28T18:15:23.289 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:15:23 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[94323]: 10: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0x733) [0x563044ffd233] 2024-03-28T18:15:23.289 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:15:23 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[94323]: 11: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x563044fff197] 2024-03-28T18:15:23.289 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:15:23 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[94323]: 12: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x563044efe11b] 2024-03-28T18:15:23.289 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:15:23 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[94323]: 13: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x563044efe6bc] 2024-03-28T18:15:23.289 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:15:23 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[94323]: 14: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bf) [0x563044ee81ff] 2024-03-28T18:15:23.289 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:15:23 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[94323]: 15: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x478) [0x7f0b83247488] 2024-03-28T18:15:23.290 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:15:23 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[94323]: 16: (DispatchQueue::entry()+0x51f) [0x7f0b8324460f] 2024-03-28T18:15:23.290 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:15:23 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[94323]: 17: (DispatchQueue::DispatchThread::entry()+0x11) [0x7f0b83317e81] 2024-03-28T18:15:23.290 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:15:23 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[94323]: 18: /lib64/libpthread.so.0(+0x81ca) [0x7f0b81d321ca] 2024-03-28T18:15:23.290 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:15:23 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[94323]: 19: clone() 2024-03-28T18:15:23.290 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:15:23 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[94323]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-03-28T18:15:23.290 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:15:23 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[94323]: 2024-03-28T18:15:23.290 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:15:23 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[94323]: -1> 2024-03-28T18:15:23.029+0000 7f0b77600700 -1 /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-2145-g3be9a5c9/rpm/el8/BUILD/ceph-19.0.0-2145-g3be9a5c9/src/mds/Locker.cc: In function 'void Locker::set_xlocks_done(MutationImpl*, bool)' thread 7f0b77600700 time 2024-03-28T18:15:23.028351+0000 2024-03-28T18:15:23.290 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:15:23 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[94323]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-2145-g3be9a5c9/rpm/el8/BUILD/ceph-19.0.0-2145-g3be9a5c9/src/mds/Locker.cc: 626: FAILED ceph_assert(obj->is_auth()) 2024-03-28T18:15:23.290 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:15:23 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[94323]: 2024-03-28T18:15:23.290 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:15:23 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[94323]: ceph version 19.0.0-2145-g3be9a5c9 (3be9a5c9bc793e11e2800a8c0c696e8b46742033) squid (dev) 2024-03-28T18:15:23.290 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:15:23 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[94323]: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x135) [0x7f0b82fb54a1] 2024-03-28T18:15:23.290 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:15:23 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[94323]: 2: /usr/lib64/ceph/libceph-common.so.2(+0x2d2667) [0x7f0b82fb5667] 2024-03-28T18:15:23.291 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:15:23 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[94323]: 3: (Locker::set_xlocks_done(MutationImpl*, bool)+0x493) [0x563045130223] 2024-03-28T18:15:23.291 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:15:23 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[94323]: 4: (Server::early_reply(boost::intrusive_ptr const&, CInode*, CDentry*)+0x3e6) [0x563044faa5d6] 2024-03-28T18:15:23.291 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:15:23 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[94323]: 5: (Server::journal_and_reply(boost::intrusive_ptr const&, CInode*, CDentry*, LogEvent*, MDSLogContextBase*)+0xc1) [0x563044faada1] 2024-03-28T18:15:23.291 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:15:23 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[94323]: 6: (Server::handle_client_mkdir(boost::intrusive_ptr const&)+0x5ba) [0x563044fe06aa] 2024-03-28T18:15:23.291 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:15:23 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[94323]: 7: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0x733) [0x563044ffd233] 2024-03-28T18:15:23.291 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:15:23 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[94323]: 8: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x563044fff197] 2024-03-28T18:15:23.291 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:15:23 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[94323]: 9: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x563044efe11b] 2024-03-28T18:15:23.291 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:15:23 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[94323]: 10: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x563044efe6bc] 2024-03-28T18:15:23.291 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:15:23 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[94323]: 11: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bf) [0x563044ee81ff] 2024-03-28T18:15:23.291 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:15:23 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[94323]: 12: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x478) [0x7f0b83247488] 2024-03-28T18:15:23.292 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:15:23 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[94323]: 13: (DispatchQueue::entry()+0x51f) [0x7f0b8324460f] 2024-03-28T18:15:23.292 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:15:23 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[94323]: 14: (DispatchQueue::DispatchThread::entry()+0x11) [0x7f0b83317e81] 2024-03-28T18:15:23.292 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:15:23 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[94323]: 15: /lib64/libpthread.so.0(+0x81ca) [0x7f0b81d321ca] 2024-03-28T18:15:23.292 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:15:23 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[94323]: 16: clone() 2024-03-28T18:15:23.292 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:15:23 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[94323]: 2024-03-28T18:15:23.292 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:15:23 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[94323]: 0> 2024-03-28T18:15:23.031+0000 7f0b77600700 -1 *** Caught signal (Aborted) ** 2024-03-28T18:15:23.292 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:15:23 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[94323]: in thread 7f0b77600700 thread_name:ms_dispatch 2024-03-28T18:15:23.292 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:15:23 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[94323]: 2024-03-28T18:15:23.292 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:15:23 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[94323]: ceph version 19.0.0-2145-g3be9a5c9 (3be9a5c9bc793e11e2800a8c0c696e8b46742033) squid (dev) 2024-03-28T18:15:23.292 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:15:23 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[94323]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7f0b81d3cd20] 2024-03-28T18:15:23.292 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:15:23 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[94323]: 2: gsignal() 2024-03-28T18:15:23.292 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:15:23 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[94323]: 3: abort() 2024-03-28T18:15:23.292 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:15:23 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[94323]: 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x18f) [0x7f0b82fb54fb] 2024-03-28T18:15:23.292 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:15:23 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[94323]: 5: /usr/lib64/ceph/libceph-common.so.2(+0x2d2667) [0x7f0b82fb5667] 2024-03-28T18:15:23.293 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:15:23 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[94323]: 6: (Locker::set_xlocks_done(MutationImpl*, bool)+0x493) [0x563045130223] 2024-03-28T18:15:23.293 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:15:23 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[94323]: 7: (Server::early_reply(boost::intrusive_ptr const&, CInode*, CDentry*)+0x3e6) [0x563044faa5d6] 2024-03-28T18:15:23.293 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:15:23 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[94323]: 8: (Server::journal_and_reply(boost::intrusive_ptr const&, CInode*, CDentry*, LogEvent*, MDSLogContextBase*)+0xc1) [0x563044faada1] 2024-03-28T18:15:23.293 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:15:23 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[94323]: 9: (Server::handle_client_mkdir(boost::intrusive_ptr const&)+0x5ba) [0x563044fe06aa] 2024-03-28T18:15:23.293 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:15:23 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[94323]: 10: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0x733) [0x563044ffd233] 2024-03-28T18:15:23.293 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:15:23 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[94323]: 11: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x563044fff197] 2024-03-28T18:15:23.293 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:15:23 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[94323]: 12: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x563044efe11b] 2024-03-28T18:15:23.293 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:15:23 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[94323]: 13: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x563044efe6bc] 2024-03-28T18:15:23.293 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:15:23 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[94323]: 14: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bf) [0x563044ee81ff] 2024-03-28T18:15:23.293 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:15:23 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[94323]: 15: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x478) [0x7f0b83247488] 2024-03-28T18:15:23.293 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:15:23 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[94323]: 16: (DispatchQueue::entry()+0x51f) [0x7f0b8324460f] 2024-03-28T18:15:23.293 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:15:23 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[94323]: 17: (DispatchQueue::DispatchThread::entry()+0x11) [0x7f0b83317e81] 2024-03-28T18:15:23.293 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:15:23 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[94323]: 18: /lib64/libpthread.so.0(+0x81ca) [0x7f0b81d321ca] 2024-03-28T18:15:23.293 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:15:23 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[94323]: 19: clone() 2024-03-28T18:15:23.293 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:15:23 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[94323]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-03-28T18:15:23.294 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:15:23 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[94323]: 2024-03-28T18:15:23.294 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:15:23 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[94323]: -1> 2024-03-28T18:15:23.029+0000 7f0b77600700 -1 /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-2145-g3be9a5c9/rpm/el8/BUILD/ceph-19.0.0-2145-g3be9a5c9/src/mds/Locker.cc: In function 'void Locker::set_xlocks_done(MutationImpl*, bool)' thread 7f0b77600700 time 2024-03-28T18:15:23.028351+0000 2024-03-28T18:15:23.294 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:15:23 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[94323]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-2145-g3be9a5c9/rpm/el8/BUILD/ceph-19.0.0-2145-g3be9a5c9/src/mds/Locker.cc: 626: FAILED ceph_assert(obj->is_auth()) 2024-03-28T18:15:23.294 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:15:23 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[94323]: 2024-03-28T18:15:23.294 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:15:23 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[94323]: ceph version 19.0.0-2145-g3be9a5c9 (3be9a5c9bc793e11e2800a8c0c696e8b46742033) squid (dev) 2024-03-28T18:15:23.294 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:15:23 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[94323]: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x135) [0x7f0b82fb54a1] 2024-03-28T18:15:23.294 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:15:23 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[94323]: 2: /usr/lib64/ceph/libceph-common.so.2(+0x2d2667) [0x7f0b82fb5667] 2024-03-28T18:15:23.294 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:15:23 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[94323]: 3: (Locker::set_xlocks_done(MutationImpl*, bool)+0x493) [0x563045130223] 2024-03-28T18:15:23.294 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:15:23 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[94323]: 4: (Server::early_reply(boost::intrusive_ptr const&, CInode*, CDentry*)+0x3e6) [0x563044faa5d6] 2024-03-28T18:15:23.294 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:15:23 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[94323]: 5: (Server::journal_and_reply(boost::intrusive_ptr const&, CInode*, CDentry*, LogEvent*, MDSLogContextBase*)+0xc1) [0x563044faada1] 2024-03-28T18:15:23.295 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:15:23 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[94323]: 6: (Server::handle_client_mkdir(boost::intrusive_ptr const&)+0x5ba) [0x563044fe06aa] 2024-03-28T18:15:23.295 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:15:23 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[94323]: 7: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0x733) [0x563044ffd233] 2024-03-28T18:15:23.295 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:15:23 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[94323]: 8: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x563044fff197] 2024-03-28T18:15:23.295 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:15:23 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[94323]: 9: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x563044efe11b] 2024-03-28T18:15:23.295 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:15:23 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[94323]: 10: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x563044efe6bc] 2024-03-28T18:15:23.295 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:15:23 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[94323]: 11: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bf) [0x563044ee81ff] 2024-03-28T18:15:23.295 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:15:23 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[94323]: 12: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x478) [0x7f0b83247488] 2024-03-28T18:15:23.295 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:15:23 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[94323]: 13: (DispatchQueue::entry()+0x51f) [0x7f0b8324460f] 2024-03-28T18:15:23.295 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:15:23 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[94323]: 14: (DispatchQueue::DispatchThread::entry()+0x11) [0x7f0b83317e81] 2024-03-28T18:15:23.295 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:15:23 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[94323]: 15: /lib64/libpthread.so.0(+0x81ca) [0x7f0b81d321ca] 2024-03-28T18:15:23.295 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:15:23 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[94323]: 16: clone() 2024-03-28T18:15:23.295 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:15:23 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[94323]: 2024-03-28T18:15:23.295 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:15:23 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[94323]: 0> 2024-03-28T18:15:23.031+0000 7f0b77600700 -1 *** Caught signal (Aborted) ** 2024-03-28T18:15:23.296 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:15:23 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[94323]: in thread 7f0b77600700 thread_name:ms_dispatch 2024-03-28T18:15:23.296 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:15:23 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[94323]: 2024-03-28T18:15:23.296 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:15:23 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[94323]: ceph version 19.0.0-2145-g3be9a5c9 (3be9a5c9bc793e11e2800a8c0c696e8b46742033) squid (dev) 2024-03-28T18:15:23.296 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:15:23 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[94323]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7f0b81d3cd20] 2024-03-28T18:15:23.296 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:15:23 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[94323]: 2: gsignal() 2024-03-28T18:15:23.296 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:15:23 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[94323]: 3: abort() 2024-03-28T18:15:23.296 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:15:23 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[94323]: 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x18f) [0x7f0b82fb54fb] 2024-03-28T18:15:23.296 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:15:23 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[94323]: 5: /usr/lib64/ceph/libceph-common.so.2(+0x2d2667) [0x7f0b82fb5667] 2024-03-28T18:15:23.296 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:15:23 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[94323]: 6: (Locker::set_xlocks_done(MutationImpl*, bool)+0x493) [0x563045130223] 2024-03-28T18:15:23.296 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:15:23 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[94323]: 7: (Server::early_reply(boost::intrusive_ptr const&, CInode*, CDentry*)+0x3e6) [0x563044faa5d6] 2024-03-28T18:15:23.296 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:15:23 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[94323]: 8: (Server::journal_and_reply(boost::intrusive_ptr const&, CInode*, CDentry*, LogEvent*, MDSLogContextBase*)+0xc1) [0x563044faada1] 2024-03-28T18:15:23.296 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:15:23 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[94323]: 9: (Server::handle_client_mkdir(boost::intrusive_ptr const&)+0x5ba) [0x563044fe06aa] 2024-03-28T18:15:23.296 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:15:23 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[94323]: 10: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0x733) [0x563044ffd233] 2024-03-28T18:15:23.296 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:15:23 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[94323]: 11: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x563044fff197] 2024-03-28T18:15:23.296 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:15:23 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[94323]: 12: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x563044efe11b] 2024-03-28T18:15:23.296 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:15:23 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[94323]: 13: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x563044efe6bc] 2024-03-28T18:15:23.297 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:15:23 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[94323]: 14: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bf) [0x563044ee81ff] 2024-03-28T18:15:23.297 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:15:23 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[94323]: 15: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x478) [0x7f0b83247488] 2024-03-28T18:15:23.297 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:15:23 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[94323]: 16: (DispatchQueue::entry()+0x51f) [0x7f0b8324460f] 2024-03-28T18:15:23.297 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:15:23 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[94323]: 17: (DispatchQueue::DispatchThread::entry()+0x11) [0x7f0b83317e81] 2024-03-28T18:15:23.297 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:15:23 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[94323]: 18: /lib64/libpthread.so.0(+0x81ca) [0x7f0b81d321ca] 2024-03-28T18:15:23.297 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:15:23 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[94323]: 19: clone() 2024-03-28T18:15:23.297 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:15:23 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[94323]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-03-28T18:15:23.297 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:15:23 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[94323]: 2024-03-28T18:15:23.297 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:15:23 smithi022 podman[94575]: 2024-03-28 18:15:23.167840482 +0000 UTC m=+0.037542443 container died 60c352fd33798abe41594e3037af250e16cd824685e11b276d9966c3c6ee9fb8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240319, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c) 2024-03-28T18:15:23.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:15:23 smithi022 ceph-mon[31985]: mds.? [v2:172.21.15.22:6836/488832747,v1:172.21.15.22:6837/488832747] up:rejoin 2024-03-28T18:15:23.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:15:23 smithi022 ceph-mon[31985]: fsmap cephfs:5/5 4 up:standby-replay 1 up:standby 1 up:rejoin 4 up:active 2024-03-28T18:15:23.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:15:23 smithi022 ceph-mon[31985]: daemon mds.d is now active in filesystem cephfs as rank 1 2024-03-28T18:15:23.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:15:23 smithi022 ceph-mon[31985]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-03-28T18:15:23.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:15:23 smithi022 ceph-mon[31985]: Cluster is now healthy 2024-03-28T18:15:23.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:15:23 smithi022 ceph-mon[31985]: mds.? [v2:172.21.15.22:6836/488832747,v1:172.21.15.22:6837/488832747] up:active 2024-03-28T18:15:23.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:15:23 smithi022 ceph-mon[31985]: fsmap cephfs:5 5 up:standby-replay 5 up:active 2024-03-28T18:15:23.925 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:15:23 smithi134 ceph-mon[35037]: mds.? [v2:172.21.15.22:6836/488832747,v1:172.21.15.22:6837/488832747] up:rejoin 2024-03-28T18:15:23.925 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:15:23 smithi134 ceph-mon[35037]: fsmap cephfs:5/5 4 up:standby-replay 1 up:standby 1 up:rejoin 4 up:active 2024-03-28T18:15:23.925 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:15:23 smithi134 ceph-mon[35037]: daemon mds.d is now active in filesystem cephfs as rank 1 2024-03-28T18:15:23.925 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:15:23 smithi134 ceph-mon[35037]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-03-28T18:15:23.925 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:15:23 smithi134 ceph-mon[35037]: Cluster is now healthy 2024-03-28T18:15:23.925 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:15:23 smithi134 ceph-mon[35037]: mds.? [v2:172.21.15.22:6836/488832747,v1:172.21.15.22:6837/488832747] up:active 2024-03-28T18:15:23.925 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:15:23 smithi134 ceph-mon[35037]: fsmap cephfs:5 5 up:standby-replay 5 up:active 2024-03-28T18:15:23.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:15:23 smithi079 ceph-mon[35312]: mds.? [v2:172.21.15.22:6836/488832747,v1:172.21.15.22:6837/488832747] up:rejoin 2024-03-28T18:15:23.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:15:23 smithi079 ceph-mon[35312]: fsmap cephfs:5/5 4 up:standby-replay 1 up:standby 1 up:rejoin 4 up:active 2024-03-28T18:15:23.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:15:23 smithi079 ceph-mon[35312]: daemon mds.d is now active in filesystem cephfs as rank 1 2024-03-28T18:15:23.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:15:23 smithi079 ceph-mon[35312]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-03-28T18:15:23.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:15:23 smithi079 ceph-mon[35312]: Cluster is now healthy 2024-03-28T18:15:23.933 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:15:23 smithi079 ceph-mon[35312]: mds.? [v2:172.21.15.22:6836/488832747,v1:172.21.15.22:6837/488832747] up:active 2024-03-28T18:15:23.933 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:15:23 smithi079 ceph-mon[35312]: fsmap cephfs:5 5 up:standby-replay 5 up:active 2024-03-28T18:15:24.282 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:15:23 smithi022 podman[94575]: 2024-03-28 18:15:23.999138638 +0000 UTC m=+0.868840627 container remove 60c352fd33798abe41594e3037af250e16cd824685e11b276d9966c3c6ee9fb8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_CLEAN=True, org.label-schema.build-date=20240319, ceph=True) 2024-03-28T18:15:24.282 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:15:24 smithi022 systemd[1]: ceph-fd220748-ed2b-11ee-95d0-87774f69a715@mds.d.service: Main process exited, code=exited, status=134/n/a 2024-03-28T18:15:24.433 INFO:journalctl@ceph.mds.b.smithi079.stdout:Mar 28 18:15:24 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-b[65245]: 2024-03-28T18:15:24.099+0000 7f845f400700 -1 mds.pinger is_rank_lagging: rank=1 was never sent ping request. 2024-03-28T18:15:24.957 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:15:24 smithi022 systemd[1]: ceph-fd220748-ed2b-11ee-95d0-87774f69a715@mds.d.service: Failed with result 'exit-code'. 2024-03-28T18:15:24.957 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:15:24 smithi022 systemd[1]: ceph-fd220748-ed2b-11ee-95d0-87774f69a715@mds.d.service: Consumed 2.012s CPU time. 2024-03-28T18:15:25.284 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:15:24 smithi022 ceph-mon[31985]: pgmap v258: 97 pgs: 97 active+clean; 117 MiB data, 794 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 10 op/s 2024-03-28T18:15:25.424 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:15:24 smithi134 ceph-mon[35037]: pgmap v258: 97 pgs: 97 active+clean; 117 MiB data, 794 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 10 op/s 2024-03-28T18:15:25.432 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:15:24 smithi079 ceph-mon[35312]: pgmap v258: 97 pgs: 97 active+clean; 117 MiB data, 794 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 10 op/s 2024-03-28T18:15:26.281 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:15:25 smithi022 ceph-mon[31985]: osdmap e114: 12 total, 12 up, 12 in 2024-03-28T18:15:26.424 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:15:25 smithi134 ceph-mon[35037]: osdmap e114: 12 total, 12 up, 12 in 2024-03-28T18:15:26.432 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:15:25 smithi079 ceph-mon[35312]: osdmap e114: 12 total, 12 up, 12 in 2024-03-28T18:15:27.243 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:15:26 smithi022 ceph-mon[31985]: pgmap v260: 97 pgs: 97 active+clean; 117 MiB data, 794 MiB used, 1.0 TiB / 1.0 TiB avail; 140 KiB/s rd, 40 KiB/s wr, 36 op/s 2024-03-28T18:15:27.424 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:15:26 smithi134 ceph-mon[35037]: pgmap v260: 97 pgs: 97 active+clean; 117 MiB data, 794 MiB used, 1.0 TiB / 1.0 TiB avail; 140 KiB/s rd, 40 KiB/s wr, 36 op/s 2024-03-28T18:15:27.432 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:15:26 smithi079 ceph-mon[35312]: pgmap v260: 97 pgs: 97 active+clean; 117 MiB data, 794 MiB used, 1.0 TiB / 1.0 TiB avail; 140 KiB/s rd, 40 KiB/s wr, 36 op/s 2024-03-28T18:15:28.282 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:15:27 smithi022 ceph-mon[31985]: pgmap v261: 97 pgs: 97 active+clean; 117 MiB data, 794 MiB used, 1.0 TiB / 1.0 TiB avail; 134 KiB/s rd, 39 KiB/s wr, 34 op/s 2024-03-28T18:15:28.424 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:15:27 smithi134 ceph-mon[35037]: pgmap v261: 97 pgs: 97 active+clean; 117 MiB data, 794 MiB used, 1.0 TiB / 1.0 TiB avail; 134 KiB/s rd, 39 KiB/s wr, 34 op/s 2024-03-28T18:15:28.432 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:15:27 smithi079 ceph-mon[35312]: pgmap v261: 97 pgs: 97 active+clean; 117 MiB data, 794 MiB used, 1.0 TiB / 1.0 TiB avail; 134 KiB/s rd, 39 KiB/s wr, 34 op/s 2024-03-28T18:15:30.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:15:30 smithi022 ceph-mon[31985]: pgmap v262: 97 pgs: 97 active+clean; 117 MiB data, 794 MiB used, 1.0 TiB / 1.0 TiB avail; 112 KiB/s rd, 32 KiB/s wr, 29 op/s 2024-03-28T18:15:30.674 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:15:30 smithi134 ceph-mon[35037]: pgmap v262: 97 pgs: 97 active+clean; 117 MiB data, 794 MiB used, 1.0 TiB / 1.0 TiB avail; 112 KiB/s rd, 32 KiB/s wr, 29 op/s 2024-03-28T18:15:30.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:15:30 smithi079 ceph-mon[35312]: pgmap v262: 97 pgs: 97 active+clean; 117 MiB data, 794 MiB used, 1.0 TiB / 1.0 TiB avail; 112 KiB/s rd, 32 KiB/s wr, 29 op/s 2024-03-28T18:15:32.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:15:32 smithi022 ceph-mon[31985]: pgmap v263: 97 pgs: 97 active+clean; 117 MiB data, 794 MiB used, 1.0 TiB / 1.0 TiB avail; 112 KiB/s rd, 32 KiB/s wr, 29 op/s 2024-03-28T18:15:32.674 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:15:32 smithi134 ceph-mon[35037]: pgmap v263: 97 pgs: 97 active+clean; 117 MiB data, 794 MiB used, 1.0 TiB / 1.0 TiB avail; 112 KiB/s rd, 32 KiB/s wr, 29 op/s 2024-03-28T18:15:32.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:15:32 smithi079 ceph-mon[35312]: pgmap v263: 97 pgs: 97 active+clean; 117 MiB data, 794 MiB used, 1.0 TiB / 1.0 TiB avail; 112 KiB/s rd, 32 KiB/s wr, 29 op/s 2024-03-28T18:15:34.282 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:15:33 smithi022 ceph-mon[31985]: osdmap e115: 12 total, 12 up, 12 in 2024-03-28T18:15:34.424 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:15:33 smithi134 ceph-mon[35037]: osdmap e115: 12 total, 12 up, 12 in 2024-03-28T18:15:34.432 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:15:33 smithi079 ceph-mon[35312]: osdmap e115: 12 total, 12 up, 12 in 2024-03-28T18:15:35.189 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:15:34 smithi022 ceph-mon[31985]: pgmap v265: 97 pgs: 97 active+clean; 117 MiB data, 794 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 124 B/s wr, 6 op/s 2024-03-28T18:15:35.189 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:15:34 smithi022 systemd[1]: ceph-fd220748-ed2b-11ee-95d0-87774f69a715@mds.d.service: Scheduled restart job, restart counter is at 4. 2024-03-28T18:15:35.189 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:15:34 smithi022 systemd[1]: Stopped Ceph mds.d for fd220748-ed2b-11ee-95d0-87774f69a715. 2024-03-28T18:15:35.189 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:15:34 smithi022 systemd[1]: ceph-fd220748-ed2b-11ee-95d0-87774f69a715@mds.d.service: Consumed 2.012s CPU time. 2024-03-28T18:15:35.189 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:15:34 smithi022 systemd[1]: Starting Ceph mds.d for fd220748-ed2b-11ee-95d0-87774f69a715... 2024-03-28T18:15:35.424 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:15:34 smithi134 ceph-mon[35037]: pgmap v265: 97 pgs: 97 active+clean; 117 MiB data, 794 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 124 B/s wr, 6 op/s 2024-03-28T18:15:35.432 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:15:34 smithi079 ceph-mon[35312]: pgmap v265: 97 pgs: 97 active+clean; 117 MiB data, 794 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 124 B/s wr, 6 op/s 2024-03-28T18:15:35.443 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:15:35 smithi022 podman[94679]: 2024-03-28 18:15:35.100008533 +0000 UTC m=+0.040542344 image pull c664afe56bb01db4f9afa9532c20598e201e664f2e9ba2be50db0d52674a26d2 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc 2024-03-28T18:15:35.443 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:15:35 smithi022 podman[94679]: 2024-03-28 18:15:35.224895607 +0000 UTC m=+0.165429425 container create 0729d79e3074da6b17522efd06268f831c76c8bcce697e0e349186eab75d4cb8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.build-date=20240319, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.license=GPLv2, ceph=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c) 2024-03-28T18:15:35.977 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:15:35 smithi022 podman[94679]: 2024-03-28 18:15:35.554214602 +0000 UTC m=+0.494748426 container init 0729d79e3074da6b17522efd06268f831c76c8bcce697e0e349186eab75d4cb8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_CLEAN=True, io.buildah.version=1.33.5, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.build-date=20240319, CEPH_POINT_RELEASE=) 2024-03-28T18:15:35.977 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:15:35 smithi022 podman[94679]: 2024-03-28 18:15:35.568133424 +0000 UTC m=+0.508667212 container start 0729d79e3074da6b17522efd06268f831c76c8bcce697e0e349186eab75d4cb8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d, CEPH_POINT_RELEASE=, ceph=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.build-date=20240319, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-03-28T18:15:35.977 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:15:35 smithi022 ceph-mds[94697]: -- 172.21.15.22:0/231099875 <== mon.0 v2:172.21.15.22:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x5646efbeda40 con 0x5646efbc1000 2024-03-28T18:15:35.977 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:15:35 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[94693]: starting mds.d at 2024-03-28T18:15:35.978 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:15:35 smithi022 bash[94679]: 0729d79e3074da6b17522efd06268f831c76c8bcce697e0e349186eab75d4cb8 2024-03-28T18:15:35.978 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:15:35 smithi022 systemd[1]: Started Ceph mds.d for fd220748-ed2b-11ee-95d0-87774f69a715. 2024-03-28T18:15:36.282 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:15:35 smithi022 ceph-mon[31985]: daemon mds.d restarted 2024-03-28T18:15:36.282 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:15:35 smithi022 ceph-mon[31985]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-03-28T18:15:36.282 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:15:35 smithi022 ceph-mon[31985]: osdmap e116: 12 total, 12 up, 12 in 2024-03-28T18:15:36.282 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:15:35 smithi022 ceph-mon[31985]: mds.? [v2:172.21.15.22:6836/2141892169,v1:172.21.15.22:6837/2141892169] up:boot 2024-03-28T18:15:36.282 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:15:35 smithi022 ceph-mon[31985]: Standby daemon mds.j assigned to filesystem cephfs as rank 1 2024-03-28T18:15:36.282 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:15:35 smithi022 ceph-mon[31985]: fsmap cephfs:4/5 {0=b=up:active,2=e=up:active,3=k=up:active,4=i=up:active} 5 up:standby-replay 1 up:standby, 1 failed 2024-03-28T18:15:36.282 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:15:35 smithi022 ceph-mon[31985]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-03-28T18:15:36.283 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:15:35 smithi022 ceph-mon[31985]: fsmap cephfs:5/5 4 up:standby-replay 1 up:standby 1 up:replay 4 up:active 2024-03-28T18:15:36.424 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:15:35 smithi134 ceph-mon[35037]: daemon mds.d restarted 2024-03-28T18:15:36.425 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:15:35 smithi134 ceph-mon[35037]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-03-28T18:15:36.425 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:15:35 smithi134 ceph-mon[35037]: osdmap e116: 12 total, 12 up, 12 in 2024-03-28T18:15:36.425 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:15:35 smithi134 ceph-mon[35037]: mds.? [v2:172.21.15.22:6836/2141892169,v1:172.21.15.22:6837/2141892169] up:boot 2024-03-28T18:15:36.425 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:15:35 smithi134 ceph-mon[35037]: Standby daemon mds.j assigned to filesystem cephfs as rank 1 2024-03-28T18:15:36.425 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:15:35 smithi134 ceph-mon[35037]: fsmap cephfs:4/5 {0=b=up:active,2=e=up:active,3=k=up:active,4=i=up:active} 5 up:standby-replay 1 up:standby, 1 failed 2024-03-28T18:15:36.425 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:15:35 smithi134 ceph-mon[35037]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-03-28T18:15:36.425 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:15:35 smithi134 ceph-mon[35037]: fsmap cephfs:5/5 4 up:standby-replay 1 up:standby 1 up:replay 4 up:active 2024-03-28T18:15:36.433 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:15:35 smithi079 ceph-mon[35312]: daemon mds.d restarted 2024-03-28T18:15:36.433 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:15:35 smithi079 ceph-mon[35312]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-03-28T18:15:36.433 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:15:35 smithi079 ceph-mon[35312]: osdmap e116: 12 total, 12 up, 12 in 2024-03-28T18:15:36.433 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:15:35 smithi079 ceph-mon[35312]: mds.? [v2:172.21.15.22:6836/2141892169,v1:172.21.15.22:6837/2141892169] up:boot 2024-03-28T18:15:36.433 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:15:35 smithi079 ceph-mon[35312]: Standby daemon mds.j assigned to filesystem cephfs as rank 1 2024-03-28T18:15:36.433 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:15:35 smithi079 ceph-mon[35312]: fsmap cephfs:4/5 {0=b=up:active,2=e=up:active,3=k=up:active,4=i=up:active} 5 up:standby-replay 1 up:standby, 1 failed 2024-03-28T18:15:36.433 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:15:35 smithi079 ceph-mon[35312]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-03-28T18:15:36.433 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:15:35 smithi079 ceph-mon[35312]: fsmap cephfs:5/5 4 up:standby-replay 1 up:standby 1 up:replay 4 up:active 2024-03-28T18:15:37.282 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:15:36 smithi022 ceph-mon[31985]: pgmap v266: 97 pgs: 97 active+clean; 117 MiB data, 794 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2024-03-28T18:15:37.424 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:15:36 smithi134 ceph-mon[35037]: pgmap v266: 97 pgs: 97 active+clean; 117 MiB data, 794 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2024-03-28T18:15:37.432 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:15:36 smithi079 ceph-mon[35312]: pgmap v266: 97 pgs: 97 active+clean; 117 MiB data, 794 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2024-03-28T18:15:38.282 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:15:38 smithi022 ceph-mon[31985]: mds.? [v2:172.21.15.22:6834/3637756693,v1:172.21.15.22:6835/3637756693] up:resolve 2024-03-28T18:15:38.282 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:15:38 smithi022 ceph-mon[31985]: fsmap cephfs:5/5 4 up:standby-replay 1 up:standby 1 up:resolve 4 up:active 2024-03-28T18:15:38.282 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:15:38 smithi022 ceph-mon[31985]: pgmap v268: 97 pgs: 97 active+clean; 117 MiB data, 794 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 127 B/s wr, 6 op/s 2024-03-28T18:15:38.424 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:15:38 smithi134 ceph-mon[35037]: mds.? [v2:172.21.15.22:6834/3637756693,v1:172.21.15.22:6835/3637756693] up:resolve 2024-03-28T18:15:38.425 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:15:38 smithi134 ceph-mon[35037]: fsmap cephfs:5/5 4 up:standby-replay 1 up:standby 1 up:resolve 4 up:active 2024-03-28T18:15:38.425 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:15:38 smithi134 ceph-mon[35037]: pgmap v268: 97 pgs: 97 active+clean; 117 MiB data, 794 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 127 B/s wr, 6 op/s 2024-03-28T18:15:38.432 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:15:38 smithi079 ceph-mon[35312]: mds.? [v2:172.21.15.22:6834/3637756693,v1:172.21.15.22:6835/3637756693] up:resolve 2024-03-28T18:15:38.432 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:15:38 smithi079 ceph-mon[35312]: fsmap cephfs:5/5 4 up:standby-replay 1 up:standby 1 up:resolve 4 up:active 2024-03-28T18:15:38.432 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:15:38 smithi079 ceph-mon[35312]: pgmap v268: 97 pgs: 97 active+clean; 117 MiB data, 794 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 127 B/s wr, 6 op/s 2024-03-28T18:15:39.282 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:15:39 smithi022 ceph-mon[31985]: reconnect by client.35222 172.21.15.22:0/3729116781 after 0.00499993 2024-03-28T18:15:39.282 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:15:39 smithi022 ceph-mon[31985]: mds.? [v2:172.21.15.22:6834/3637756693,v1:172.21.15.22:6835/3637756693] up:reconnect 2024-03-28T18:15:39.282 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:15:39 smithi022 ceph-mon[31985]: fsmap cephfs:5/5 4 up:standby-replay 1 up:standby 1 up:reconnect 4 up:active 2024-03-28T18:15:39.425 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:15:39 smithi134 ceph-mon[35037]: reconnect by client.35222 172.21.15.22:0/3729116781 after 0.00499993 2024-03-28T18:15:39.425 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:15:39 smithi134 ceph-mon[35037]: mds.? [v2:172.21.15.22:6834/3637756693,v1:172.21.15.22:6835/3637756693] up:reconnect 2024-03-28T18:15:39.425 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:15:39 smithi134 ceph-mon[35037]: fsmap cephfs:5/5 4 up:standby-replay 1 up:standby 1 up:reconnect 4 up:active 2024-03-28T18:15:39.432 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:15:39 smithi079 ceph-mon[35312]: reconnect by client.35222 172.21.15.22:0/3729116781 after 0.00499993 2024-03-28T18:15:39.432 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:15:39 smithi079 ceph-mon[35312]: mds.? [v2:172.21.15.22:6834/3637756693,v1:172.21.15.22:6835/3637756693] up:reconnect 2024-03-28T18:15:39.432 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:15:39 smithi079 ceph-mon[35312]: fsmap cephfs:5/5 4 up:standby-replay 1 up:standby 1 up:reconnect 4 up:active 2024-03-28T18:15:40.334 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:15:40 smithi079 ceph-mon[35312]: mds.? [v2:172.21.15.22:6834/3637756693,v1:172.21.15.22:6835/3637756693] up:rejoin 2024-03-28T18:15:40.334 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:15:40 smithi079 ceph-mon[35312]: fsmap cephfs:5/5 4 up:standby-replay 1 up:standby 1 up:rejoin 4 up:active 2024-03-28T18:15:40.334 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:15:40 smithi079 ceph-mon[35312]: daemon mds.j is now active in filesystem cephfs as rank 1 2024-03-28T18:15:40.334 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:15:40 smithi079 ceph-mon[35312]: pgmap v269: 97 pgs: 97 active+clean; 117 MiB data, 794 MiB used, 1.0 TiB / 1.0 TiB avail; 7.6 KiB/s rd, 639 B/s wr, 13 op/s 2024-03-28T18:15:40.424 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:15:40 smithi134 ceph-mon[35037]: mds.? [v2:172.21.15.22:6834/3637756693,v1:172.21.15.22:6835/3637756693] up:rejoin 2024-03-28T18:15:40.425 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:15:40 smithi134 ceph-mon[35037]: fsmap cephfs:5/5 4 up:standby-replay 1 up:standby 1 up:rejoin 4 up:active 2024-03-28T18:15:40.425 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:15:40 smithi134 ceph-mon[35037]: daemon mds.j is now active in filesystem cephfs as rank 1 2024-03-28T18:15:40.425 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:15:40 smithi134 ceph-mon[35037]: pgmap v269: 97 pgs: 97 active+clean; 117 MiB data, 794 MiB used, 1.0 TiB / 1.0 TiB avail; 7.6 KiB/s rd, 639 B/s wr, 13 op/s 2024-03-28T18:15:40.519 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:15:40 smithi022 ceph-mon[31985]: mds.? [v2:172.21.15.22:6834/3637756693,v1:172.21.15.22:6835/3637756693] up:rejoin 2024-03-28T18:15:40.519 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:15:40 smithi022 ceph-mon[31985]: fsmap cephfs:5/5 4 up:standby-replay 1 up:standby 1 up:rejoin 4 up:active 2024-03-28T18:15:40.519 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:15:40 smithi022 ceph-mon[31985]: daemon mds.j is now active in filesystem cephfs as rank 1 2024-03-28T18:15:40.519 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:15:40 smithi022 ceph-mon[31985]: pgmap v269: 97 pgs: 97 active+clean; 117 MiB data, 794 MiB used, 1.0 TiB / 1.0 TiB avail; 7.6 KiB/s rd, 639 B/s wr, 13 op/s 2024-03-28T18:15:40.522 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:15:40 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[94471]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-2145-g3be9a5c9/rpm/el8/BUILD/ceph-19.0.0-2145-g3be9a5c9/src/mds/Locker.cc: In function 'void Locker::set_xlocks_done(MutationImpl*, bool)' thread 7fae72c00700 time 2024-03-28T18:15:40.115263+0000 2024-03-28T18:15:40.522 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:15:40 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[94471]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-2145-g3be9a5c9/rpm/el8/BUILD/ceph-19.0.0-2145-g3be9a5c9/src/mds/Locker.cc: 626: FAILED ceph_assert(obj->is_auth()) 2024-03-28T18:15:40.522 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:15:40 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[94471]: ceph version 19.0.0-2145-g3be9a5c9 (3be9a5c9bc793e11e2800a8c0c696e8b46742033) squid (dev) 2024-03-28T18:15:40.522 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:15:40 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[94471]: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x135) [0x7fae7e5604a1] 2024-03-28T18:15:40.522 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:15:40 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[94471]: 2: /usr/lib64/ceph/libceph-common.so.2(+0x2d2667) [0x7fae7e560667] 2024-03-28T18:15:40.522 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:15:40 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[94471]: 3: (Locker::set_xlocks_done(MutationImpl*, bool)+0x493) [0x560813202223] 2024-03-28T18:15:40.522 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:15:40 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[94471]: 4: (Server::early_reply(boost::intrusive_ptr const&, CInode*, CDentry*)+0x3e6) [0x56081307c5d6] 2024-03-28T18:15:40.522 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:15:40 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[94471]: 5: (Server::journal_and_reply(boost::intrusive_ptr const&, CInode*, CDentry*, LogEvent*, MDSLogContextBase*)+0xc1) [0x56081307cda1] 2024-03-28T18:15:40.522 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:15:40 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[94471]: 6: (Server::handle_client_mkdir(boost::intrusive_ptr const&)+0x5ba) [0x5608130b26aa] 2024-03-28T18:15:40.522 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:15:40 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[94471]: 7: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0x733) [0x5608130cf233] 2024-03-28T18:15:40.522 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:15:40 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[94471]: 8: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x5608130d1197] 2024-03-28T18:15:40.522 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:15:40 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[94471]: 9: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x560812fd011b] 2024-03-28T18:15:40.523 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:15:40 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[94471]: 10: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x560812fd06bc] 2024-03-28T18:15:40.523 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:15:40 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[94471]: 11: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bf) [0x560812fba1ff] 2024-03-28T18:15:40.523 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:15:40 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[94471]: 12: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x478) [0x7fae7e7f2488] 2024-03-28T18:15:40.523 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:15:40 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[94471]: 13: (DispatchQueue::entry()+0x51f) [0x7fae7e7ef60f] 2024-03-28T18:15:40.523 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:15:40 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[94471]: 14: (DispatchQueue::DispatchThread::entry()+0x11) [0x7fae7e8c2e81] 2024-03-28T18:15:40.523 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:15:40 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[94471]: 15: /lib64/libpthread.so.0(+0x81ca) [0x7fae7d2dd1ca] 2024-03-28T18:15:40.523 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:15:40 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[94471]: 16: clone() 2024-03-28T18:15:40.523 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:15:40 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[94471]: 2024-03-28T18:15:40.115+0000 7fae72c00700 -1 /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-2145-g3be9a5c9/rpm/el8/BUILD/ceph-19.0.0-2145-g3be9a5c9/src/mds/Locker.cc: In function 'void Locker::set_xlocks_done(MutationImpl*, bool)' thread 7fae72c00700 time 2024-03-28T18:15:40.115263+0000 2024-03-28T18:15:40.523 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:15:40 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[94471]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-2145-g3be9a5c9/rpm/el8/BUILD/ceph-19.0.0-2145-g3be9a5c9/src/mds/Locker.cc: 626: FAILED ceph_assert(obj->is_auth()) 2024-03-28T18:15:40.523 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:15:40 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[94471]: 2024-03-28T18:15:40.524 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:15:40 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[94471]: ceph version 19.0.0-2145-g3be9a5c9 (3be9a5c9bc793e11e2800a8c0c696e8b46742033) squid (dev) 2024-03-28T18:15:40.524 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:15:40 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[94471]: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x135) [0x7fae7e5604a1] 2024-03-28T18:15:40.524 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:15:40 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[94471]: 2: /usr/lib64/ceph/libceph-common.so.2(+0x2d2667) [0x7fae7e560667] 2024-03-28T18:15:40.524 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:15:40 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[94471]: 3: (Locker::set_xlocks_done(MutationImpl*, bool)+0x493) [0x560813202223] 2024-03-28T18:15:40.524 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:15:40 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[94471]: 4: (Server::early_reply(boost::intrusive_ptr const&, CInode*, CDentry*)+0x3e6) [0x56081307c5d6] 2024-03-28T18:15:40.524 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:15:40 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[94471]: 5: (Server::journal_and_reply(boost::intrusive_ptr const&, CInode*, CDentry*, LogEvent*, MDSLogContextBase*)+0xc1) [0x56081307cda1] 2024-03-28T18:15:40.524 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:15:40 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[94471]: 6: (Server::handle_client_mkdir(boost::intrusive_ptr const&)+0x5ba) [0x5608130b26aa] 2024-03-28T18:15:40.524 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:15:40 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[94471]: 7: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0x733) [0x5608130cf233] 2024-03-28T18:15:40.524 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:15:40 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[94471]: 8: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x5608130d1197] 2024-03-28T18:15:40.524 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:15:40 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[94471]: 9: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x560812fd011b] 2024-03-28T18:15:40.524 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:15:40 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[94471]: 10: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x560812fd06bc] 2024-03-28T18:15:40.524 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:15:40 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[94471]: 11: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bf) [0x560812fba1ff] 2024-03-28T18:15:40.524 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:15:40 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[94471]: 12: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x478) [0x7fae7e7f2488] 2024-03-28T18:15:40.524 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:15:40 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[94471]: 13: (DispatchQueue::entry()+0x51f) [0x7fae7e7ef60f] 2024-03-28T18:15:40.524 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:15:40 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[94471]: 14: (DispatchQueue::DispatchThread::entry()+0x11) [0x7fae7e8c2e81] 2024-03-28T18:15:40.524 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:15:40 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[94471]: 15: /lib64/libpthread.so.0(+0x81ca) [0x7fae7d2dd1ca] 2024-03-28T18:15:40.524 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:15:40 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[94471]: 16: clone() 2024-03-28T18:15:40.525 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:15:40 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[94471]: 2024-03-28T18:15:40.525 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:15:40 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[94471]: *** Caught signal (Aborted) ** 2024-03-28T18:15:40.525 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:15:40 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[94471]: in thread 7fae72c00700 thread_name:ms_dispatch 2024-03-28T18:15:40.525 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:15:40 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[94471]: ceph version 19.0.0-2145-g3be9a5c9 (3be9a5c9bc793e11e2800a8c0c696e8b46742033) squid (dev) 2024-03-28T18:15:40.525 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:15:40 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[94471]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7fae7d2e7d20] 2024-03-28T18:15:40.525 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:15:40 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[94471]: 2: gsignal() 2024-03-28T18:15:40.525 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:15:40 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[94471]: 3: abort() 2024-03-28T18:15:40.526 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:15:40 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[94471]: 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x18f) [0x7fae7e5604fb] 2024-03-28T18:15:40.526 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:15:40 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[94471]: 5: /usr/lib64/ceph/libceph-common.so.2(+0x2d2667) [0x7fae7e560667] 2024-03-28T18:15:40.526 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:15:40 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[94471]: 6: (Locker::set_xlocks_done(MutationImpl*, bool)+0x493) [0x560813202223] 2024-03-28T18:15:40.526 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:15:40 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[94471]: 7: (Server::early_reply(boost::intrusive_ptr const&, CInode*, CDentry*)+0x3e6) [0x56081307c5d6] 2024-03-28T18:15:40.526 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:15:40 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[94471]: 8: (Server::journal_and_reply(boost::intrusive_ptr const&, CInode*, CDentry*, LogEvent*, MDSLogContextBase*)+0xc1) [0x56081307cda1] 2024-03-28T18:15:40.526 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:15:40 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[94471]: 9: (Server::handle_client_mkdir(boost::intrusive_ptr const&)+0x5ba) [0x5608130b26aa] 2024-03-28T18:15:40.526 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:15:40 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[94471]: 10: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0x733) [0x5608130cf233] 2024-03-28T18:15:40.526 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:15:40 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[94471]: 11: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x5608130d1197] 2024-03-28T18:15:40.526 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:15:40 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[94471]: 12: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x560812fd011b] 2024-03-28T18:15:40.526 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:15:40 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[94471]: 13: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x560812fd06bc] 2024-03-28T18:15:40.526 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:15:40 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[94471]: 14: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bf) [0x560812fba1ff] 2024-03-28T18:15:40.526 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:15:40 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[94471]: 15: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x478) [0x7fae7e7f2488] 2024-03-28T18:15:40.526 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:15:40 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[94471]: 16: (DispatchQueue::entry()+0x51f) [0x7fae7e7ef60f] 2024-03-28T18:15:40.526 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:15:40 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[94471]: 17: (DispatchQueue::DispatchThread::entry()+0x11) [0x7fae7e8c2e81] 2024-03-28T18:15:40.526 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:15:40 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[94471]: 18: /lib64/libpthread.so.0(+0x81ca) [0x7fae7d2dd1ca] 2024-03-28T18:15:40.526 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:15:40 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[94471]: 19: clone() 2024-03-28T18:15:40.527 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:15:40 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[94471]: 2024-03-28T18:15:40.118+0000 7fae72c00700 -1 *** Caught signal (Aborted) ** 2024-03-28T18:15:40.527 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:15:40 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[94471]: in thread 7fae72c00700 thread_name:ms_dispatch 2024-03-28T18:15:40.527 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:15:40 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[94471]: 2024-03-28T18:15:40.527 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:15:40 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[94471]: ceph version 19.0.0-2145-g3be9a5c9 (3be9a5c9bc793e11e2800a8c0c696e8b46742033) squid (dev) 2024-03-28T18:15:40.527 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:15:40 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[94471]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7fae7d2e7d20] 2024-03-28T18:15:40.527 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:15:40 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[94471]: 2: gsignal() 2024-03-28T18:15:40.527 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:15:40 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[94471]: 3: abort() 2024-03-28T18:15:40.527 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:15:40 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[94471]: 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x18f) [0x7fae7e5604fb] 2024-03-28T18:15:40.527 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:15:40 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[94471]: 5: /usr/lib64/ceph/libceph-common.so.2(+0x2d2667) [0x7fae7e560667] 2024-03-28T18:15:40.527 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:15:40 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[94471]: 6: (Locker::set_xlocks_done(MutationImpl*, bool)+0x493) [0x560813202223] 2024-03-28T18:15:40.527 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:15:40 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[94471]: 7: (Server::early_reply(boost::intrusive_ptr const&, CInode*, CDentry*)+0x3e6) [0x56081307c5d6] 2024-03-28T18:15:40.528 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:15:40 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[94471]: 8: (Server::journal_and_reply(boost::intrusive_ptr const&, CInode*, CDentry*, LogEvent*, MDSLogContextBase*)+0xc1) [0x56081307cda1] 2024-03-28T18:15:40.528 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:15:40 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[94471]: 9: (Server::handle_client_mkdir(boost::intrusive_ptr const&)+0x5ba) [0x5608130b26aa] 2024-03-28T18:15:40.528 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:15:40 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[94471]: 10: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0x733) [0x5608130cf233] 2024-03-28T18:15:40.528 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:15:40 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[94471]: 11: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x5608130d1197] 2024-03-28T18:15:40.528 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:15:40 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[94471]: 12: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x560812fd011b] 2024-03-28T18:15:40.528 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:15:40 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[94471]: 13: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x560812fd06bc] 2024-03-28T18:15:40.528 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:15:40 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[94471]: 14: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bf) [0x560812fba1ff] 2024-03-28T18:15:40.528 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:15:40 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[94471]: 15: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x478) [0x7fae7e7f2488] 2024-03-28T18:15:40.528 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:15:40 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[94471]: 16: (DispatchQueue::entry()+0x51f) [0x7fae7e7ef60f] 2024-03-28T18:15:40.528 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:15:40 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[94471]: 17: (DispatchQueue::DispatchThread::entry()+0x11) [0x7fae7e8c2e81] 2024-03-28T18:15:40.528 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:15:40 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[94471]: 18: /lib64/libpthread.so.0(+0x81ca) [0x7fae7d2dd1ca] 2024-03-28T18:15:40.528 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:15:40 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[94471]: 19: clone() 2024-03-28T18:15:40.528 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:15:40 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[94471]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-03-28T18:15:40.529 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:15:40 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[94471]: 2024-03-28T18:15:40.529 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:15:40 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[94471]: -1> 2024-03-28T18:15:40.115+0000 7fae72c00700 -1 /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-2145-g3be9a5c9/rpm/el8/BUILD/ceph-19.0.0-2145-g3be9a5c9/src/mds/Locker.cc: In function 'void Locker::set_xlocks_done(MutationImpl*, bool)' thread 7fae72c00700 time 2024-03-28T18:15:40.115263+0000 2024-03-28T18:15:40.529 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:15:40 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[94471]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-2145-g3be9a5c9/rpm/el8/BUILD/ceph-19.0.0-2145-g3be9a5c9/src/mds/Locker.cc: 626: FAILED ceph_assert(obj->is_auth()) 2024-03-28T18:15:40.529 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:15:40 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[94471]: 2024-03-28T18:15:40.529 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:15:40 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[94471]: ceph version 19.0.0-2145-g3be9a5c9 (3be9a5c9bc793e11e2800a8c0c696e8b46742033) squid (dev) 2024-03-28T18:15:40.529 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:15:40 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[94471]: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x135) [0x7fae7e5604a1] 2024-03-28T18:15:40.529 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:15:40 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[94471]: 2: /usr/lib64/ceph/libceph-common.so.2(+0x2d2667) [0x7fae7e560667] 2024-03-28T18:15:40.529 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:15:40 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[94471]: 3: (Locker::set_xlocks_done(MutationImpl*, bool)+0x493) [0x560813202223] 2024-03-28T18:15:40.529 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:15:40 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[94471]: 4: (Server::early_reply(boost::intrusive_ptr const&, CInode*, CDentry*)+0x3e6) [0x56081307c5d6] 2024-03-28T18:15:40.529 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:15:40 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[94471]: 5: (Server::journal_and_reply(boost::intrusive_ptr const&, CInode*, CDentry*, LogEvent*, MDSLogContextBase*)+0xc1) [0x56081307cda1] 2024-03-28T18:15:40.530 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:15:40 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[94471]: 6: (Server::handle_client_mkdir(boost::intrusive_ptr const&)+0x5ba) [0x5608130b26aa] 2024-03-28T18:15:40.530 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:15:40 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[94471]: 7: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0x733) [0x5608130cf233] 2024-03-28T18:15:40.530 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:15:40 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[94471]: 8: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x5608130d1197] 2024-03-28T18:15:40.530 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:15:40 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[94471]: 9: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x560812fd011b] 2024-03-28T18:15:40.530 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:15:40 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[94471]: 10: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x560812fd06bc] 2024-03-28T18:15:40.530 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:15:40 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[94471]: 11: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bf) [0x560812fba1ff] 2024-03-28T18:15:40.530 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:15:40 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[94471]: 12: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x478) [0x7fae7e7f2488] 2024-03-28T18:15:40.530 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:15:40 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[94471]: 13: (DispatchQueue::entry()+0x51f) [0x7fae7e7ef60f] 2024-03-28T18:15:40.530 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:15:40 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[94471]: 14: (DispatchQueue::DispatchThread::entry()+0x11) [0x7fae7e8c2e81] 2024-03-28T18:15:40.530 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:15:40 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[94471]: 15: /lib64/libpthread.so.0(+0x81ca) [0x7fae7d2dd1ca] 2024-03-28T18:15:40.530 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:15:40 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[94471]: 16: clone() 2024-03-28T18:15:40.530 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:15:40 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[94471]: 2024-03-28T18:15:40.530 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:15:40 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[94471]: 0> 2024-03-28T18:15:40.118+0000 7fae72c00700 -1 *** Caught signal (Aborted) ** 2024-03-28T18:15:40.530 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:15:40 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[94471]: in thread 7fae72c00700 thread_name:ms_dispatch 2024-03-28T18:15:40.531 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:15:40 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[94471]: 2024-03-28T18:15:40.531 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:15:40 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[94471]: ceph version 19.0.0-2145-g3be9a5c9 (3be9a5c9bc793e11e2800a8c0c696e8b46742033) squid (dev) 2024-03-28T18:15:40.531 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:15:40 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[94471]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7fae7d2e7d20] 2024-03-28T18:15:40.531 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:15:40 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[94471]: 2: gsignal() 2024-03-28T18:15:40.531 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:15:40 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[94471]: 3: abort() 2024-03-28T18:15:40.531 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:15:40 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[94471]: 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x18f) [0x7fae7e5604fb] 2024-03-28T18:15:40.531 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:15:40 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[94471]: 5: /usr/lib64/ceph/libceph-common.so.2(+0x2d2667) [0x7fae7e560667] 2024-03-28T18:15:40.531 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:15:40 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[94471]: 6: (Locker::set_xlocks_done(MutationImpl*, bool)+0x493) [0x560813202223] 2024-03-28T18:15:40.531 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:15:40 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[94471]: 7: (Server::early_reply(boost::intrusive_ptr const&, CInode*, CDentry*)+0x3e6) [0x56081307c5d6] 2024-03-28T18:15:40.531 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:15:40 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[94471]: 8: (Server::journal_and_reply(boost::intrusive_ptr const&, CInode*, CDentry*, LogEvent*, MDSLogContextBase*)+0xc1) [0x56081307cda1] 2024-03-28T18:15:40.532 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:15:40 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[94471]: 9: (Server::handle_client_mkdir(boost::intrusive_ptr const&)+0x5ba) [0x5608130b26aa] 2024-03-28T18:15:40.532 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:15:40 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[94471]: 10: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0x733) [0x5608130cf233] 2024-03-28T18:15:40.532 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:15:40 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[94471]: 11: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x5608130d1197] 2024-03-28T18:15:40.532 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:15:40 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[94471]: 12: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x560812fd011b] 2024-03-28T18:15:40.532 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:15:40 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[94471]: 13: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x560812fd06bc] 2024-03-28T18:15:40.532 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:15:40 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[94471]: 14: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bf) [0x560812fba1ff] 2024-03-28T18:15:40.532 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:15:40 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[94471]: 15: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x478) [0x7fae7e7f2488] 2024-03-28T18:15:40.532 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:15:40 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[94471]: 16: (DispatchQueue::entry()+0x51f) [0x7fae7e7ef60f] 2024-03-28T18:15:40.532 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:15:40 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[94471]: 17: (DispatchQueue::DispatchThread::entry()+0x11) [0x7fae7e8c2e81] 2024-03-28T18:15:40.532 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:15:40 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[94471]: 18: /lib64/libpthread.so.0(+0x81ca) [0x7fae7d2dd1ca] 2024-03-28T18:15:40.532 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:15:40 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[94471]: 19: clone() 2024-03-28T18:15:40.532 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:15:40 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[94471]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-03-28T18:15:40.532 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:15:40 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[94471]: 2024-03-28T18:15:40.532 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:15:40 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[94471]: -1> 2024-03-28T18:15:40.115+0000 7fae72c00700 -1 /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-2145-g3be9a5c9/rpm/el8/BUILD/ceph-19.0.0-2145-g3be9a5c9/src/mds/Locker.cc: In function 'void Locker::set_xlocks_done(MutationImpl*, bool)' thread 7fae72c00700 time 2024-03-28T18:15:40.115263+0000 2024-03-28T18:15:40.532 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:15:40 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[94471]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-2145-g3be9a5c9/rpm/el8/BUILD/ceph-19.0.0-2145-g3be9a5c9/src/mds/Locker.cc: 626: FAILED ceph_assert(obj->is_auth()) 2024-03-28T18:15:40.532 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:15:40 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[94471]: 2024-03-28T18:15:40.533 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:15:40 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[94471]: ceph version 19.0.0-2145-g3be9a5c9 (3be9a5c9bc793e11e2800a8c0c696e8b46742033) squid (dev) 2024-03-28T18:15:40.533 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:15:40 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[94471]: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x135) [0x7fae7e5604a1] 2024-03-28T18:15:40.533 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:15:40 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[94471]: 2: /usr/lib64/ceph/libceph-common.so.2(+0x2d2667) [0x7fae7e560667] 2024-03-28T18:15:40.533 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:15:40 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[94471]: 3: (Locker::set_xlocks_done(MutationImpl*, bool)+0x493) [0x560813202223] 2024-03-28T18:15:40.533 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:15:40 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[94471]: 4: (Server::early_reply(boost::intrusive_ptr const&, CInode*, CDentry*)+0x3e6) [0x56081307c5d6] 2024-03-28T18:15:40.533 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:15:40 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[94471]: 5: (Server::journal_and_reply(boost::intrusive_ptr const&, CInode*, CDentry*, LogEvent*, MDSLogContextBase*)+0xc1) [0x56081307cda1] 2024-03-28T18:15:40.533 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:15:40 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[94471]: 6: (Server::handle_client_mkdir(boost::intrusive_ptr const&)+0x5ba) [0x5608130b26aa] 2024-03-28T18:15:40.533 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:15:40 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[94471]: 7: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0x733) [0x5608130cf233] 2024-03-28T18:15:40.533 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:15:40 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[94471]: 8: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x5608130d1197] 2024-03-28T18:15:40.534 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:15:40 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[94471]: 9: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x560812fd011b] 2024-03-28T18:15:40.534 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:15:40 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[94471]: 10: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x560812fd06bc] 2024-03-28T18:15:40.534 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:15:40 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[94471]: 11: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bf) [0x560812fba1ff] 2024-03-28T18:15:40.534 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:15:40 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[94471]: 12: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x478) [0x7fae7e7f2488] 2024-03-28T18:15:40.534 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:15:40 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[94471]: 13: (DispatchQueue::entry()+0x51f) [0x7fae7e7ef60f] 2024-03-28T18:15:40.534 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:15:40 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[94471]: 14: (DispatchQueue::DispatchThread::entry()+0x11) [0x7fae7e8c2e81] 2024-03-28T18:15:40.534 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:15:40 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[94471]: 15: /lib64/libpthread.so.0(+0x81ca) [0x7fae7d2dd1ca] 2024-03-28T18:15:40.534 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:15:40 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[94471]: 16: clone() 2024-03-28T18:15:40.534 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:15:40 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[94471]: 2024-03-28T18:15:40.534 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:15:40 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[94471]: 0> 2024-03-28T18:15:40.118+0000 7fae72c00700 -1 *** Caught signal (Aborted) ** 2024-03-28T18:15:40.535 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:15:40 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[94471]: in thread 7fae72c00700 thread_name:ms_dispatch 2024-03-28T18:15:40.535 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:15:40 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[94471]: 2024-03-28T18:15:40.535 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:15:40 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[94471]: ceph version 19.0.0-2145-g3be9a5c9 (3be9a5c9bc793e11e2800a8c0c696e8b46742033) squid (dev) 2024-03-28T18:15:40.535 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:15:40 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[94471]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7fae7d2e7d20] 2024-03-28T18:15:40.535 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:15:40 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[94471]: 2: gsignal() 2024-03-28T18:15:40.535 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:15:40 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[94471]: 3: abort() 2024-03-28T18:15:40.535 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:15:40 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[94471]: 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x18f) [0x7fae7e5604fb] 2024-03-28T18:15:40.535 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:15:40 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[94471]: 5: /usr/lib64/ceph/libceph-common.so.2(+0x2d2667) [0x7fae7e560667] 2024-03-28T18:15:40.535 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:15:40 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[94471]: 6: (Locker::set_xlocks_done(MutationImpl*, bool)+0x493) [0x560813202223] 2024-03-28T18:15:40.535 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:15:40 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[94471]: 7: (Server::early_reply(boost::intrusive_ptr const&, CInode*, CDentry*)+0x3e6) [0x56081307c5d6] 2024-03-28T18:15:40.535 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:15:40 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[94471]: 8: (Server::journal_and_reply(boost::intrusive_ptr const&, CInode*, CDentry*, LogEvent*, MDSLogContextBase*)+0xc1) [0x56081307cda1] 2024-03-28T18:15:40.535 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:15:40 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[94471]: 9: (Server::handle_client_mkdir(boost::intrusive_ptr const&)+0x5ba) [0x5608130b26aa] 2024-03-28T18:15:40.535 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:15:40 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[94471]: 10: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0x733) [0x5608130cf233] 2024-03-28T18:15:40.535 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:15:40 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[94471]: 11: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x5608130d1197] 2024-03-28T18:15:40.535 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:15:40 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[94471]: 12: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x560812fd011b] 2024-03-28T18:15:40.535 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:15:40 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[94471]: 13: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x560812fd06bc] 2024-03-28T18:15:40.536 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:15:40 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[94471]: 14: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bf) [0x560812fba1ff] 2024-03-28T18:15:40.536 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:15:40 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[94471]: 15: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x478) [0x7fae7e7f2488] 2024-03-28T18:15:40.536 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:15:40 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[94471]: 16: (DispatchQueue::entry()+0x51f) [0x7fae7e7ef60f] 2024-03-28T18:15:40.536 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:15:40 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[94471]: 17: (DispatchQueue::DispatchThread::entry()+0x11) [0x7fae7e8c2e81] 2024-03-28T18:15:40.536 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:15:40 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[94471]: 18: /lib64/libpthread.so.0(+0x81ca) [0x7fae7d2dd1ca] 2024-03-28T18:15:40.536 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:15:40 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[94471]: 19: clone() 2024-03-28T18:15:40.536 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:15:40 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[94471]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-03-28T18:15:40.536 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:15:40 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[94471]: 2024-03-28T18:15:40.536 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:15:40 smithi022 podman[94732]: 2024-03-28 18:15:40.261334041 +0000 UTC m=+0.039119757 container died 7a3646bdb6cd85559385486f0aed5a2ec7a1ee9cf6f5208081a36ce4ea450d2e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.build-date=20240319, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0) 2024-03-28T18:15:41.425 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:15:41 smithi134 ceph-mon[35037]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-03-28T18:15:41.425 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:15:41 smithi134 ceph-mon[35037]: Cluster is now healthy 2024-03-28T18:15:41.425 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:15:41 smithi134 ceph-mon[35037]: mds.? [v2:172.21.15.22:6834/3637756693,v1:172.21.15.22:6835/3637756693] up:active 2024-03-28T18:15:41.425 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:15:41 smithi134 ceph-mon[35037]: fsmap cephfs:5 4 up:standby-replay 1 up:standby 5 up:active 2024-03-28T18:15:41.425 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:15:41 smithi134 ceph-mon[35037]: fsmap cephfs:5 5 up:standby-replay 5 up:active 2024-03-28T18:15:41.432 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:15:41 smithi079 ceph-mon[35312]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-03-28T18:15:41.432 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:15:41 smithi079 ceph-mon[35312]: Cluster is now healthy 2024-03-28T18:15:41.433 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:15:41 smithi079 ceph-mon[35312]: mds.? [v2:172.21.15.22:6834/3637756693,v1:172.21.15.22:6835/3637756693] up:active 2024-03-28T18:15:41.433 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:15:41 smithi079 ceph-mon[35312]: fsmap cephfs:5 4 up:standby-replay 1 up:standby 5 up:active 2024-03-28T18:15:41.433 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:15:41 smithi079 ceph-mon[35312]: fsmap cephfs:5 5 up:standby-replay 5 up:active 2024-03-28T18:15:41.532 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:15:41 smithi022 podman[94732]: 2024-03-28 18:15:41.065660079 +0000 UTC m=+0.843445799 container remove 7a3646bdb6cd85559385486f0aed5a2ec7a1ee9cf6f5208081a36ce4ea450d2e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j, org.label-schema.schema-version=1.0, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.build-date=20240319, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS) 2024-03-28T18:15:41.532 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:15:41 smithi022 systemd[1]: ceph-fd220748-ed2b-11ee-95d0-87774f69a715@mds.j.service: Main process exited, code=exited, status=134/n/a 2024-03-28T18:15:41.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:15:41 smithi022 ceph-mon[31985]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-03-28T18:15:41.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:15:41 smithi022 ceph-mon[31985]: Cluster is now healthy 2024-03-28T18:15:41.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:15:41 smithi022 ceph-mon[31985]: mds.? [v2:172.21.15.22:6834/3637756693,v1:172.21.15.22:6835/3637756693] up:active 2024-03-28T18:15:41.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:15:41 smithi022 ceph-mon[31985]: fsmap cephfs:5 4 up:standby-replay 1 up:standby 5 up:active 2024-03-28T18:15:41.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:15:41 smithi022 ceph-mon[31985]: fsmap cephfs:5 5 up:standby-replay 5 up:active 2024-03-28T18:15:42.032 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:15:41 smithi022 systemd[1]: ceph-fd220748-ed2b-11ee-95d0-87774f69a715@mds.j.service: Failed with result 'exit-code'. 2024-03-28T18:15:42.032 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:15:41 smithi022 systemd[1]: ceph-fd220748-ed2b-11ee-95d0-87774f69a715@mds.j.service: Consumed 2.030s CPU time. 2024-03-28T18:15:42.424 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:15:42 smithi134 ceph-mon[35037]: pgmap v270: 97 pgs: 97 active+clean; 117 MiB data, 794 MiB used, 1.0 TiB / 1.0 TiB avail; 7.4 KiB/s rd, 621 B/s wr, 12 op/s 2024-03-28T18:15:42.432 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:15:42 smithi079 ceph-mon[35312]: pgmap v270: 97 pgs: 97 active+clean; 117 MiB data, 794 MiB used, 1.0 TiB / 1.0 TiB avail; 7.4 KiB/s rd, 621 B/s wr, 12 op/s 2024-03-28T18:15:42.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:15:42 smithi022 ceph-mon[31985]: pgmap v270: 97 pgs: 97 active+clean; 117 MiB data, 794 MiB used, 1.0 TiB / 1.0 TiB avail; 7.4 KiB/s rd, 621 B/s wr, 12 op/s 2024-03-28T18:15:44.531 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:15:44 smithi022 ceph-mon[31985]: pgmap v271: 97 pgs: 97 active+clean; 117 MiB data, 794 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 511 B/s wr, 10 op/s 2024-03-28T18:15:44.674 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:15:44 smithi134 ceph-mon[35037]: pgmap v271: 97 pgs: 97 active+clean; 117 MiB data, 794 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 511 B/s wr, 10 op/s 2024-03-28T18:15:44.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:15:44 smithi079 ceph-mon[35312]: pgmap v271: 97 pgs: 97 active+clean; 117 MiB data, 794 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 511 B/s wr, 10 op/s 2024-03-28T18:15:46.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:15:46 smithi022 ceph-mon[31985]: pgmap v272: 97 pgs: 97 active+clean; 117 MiB data, 794 MiB used, 1.0 TiB / 1.0 TiB avail; 110 KiB/s rd, 5.5 KiB/s wr, 25 op/s 2024-03-28T18:15:46.674 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:15:46 smithi134 ceph-mon[35037]: pgmap v272: 97 pgs: 97 active+clean; 117 MiB data, 794 MiB used, 1.0 TiB / 1.0 TiB avail; 110 KiB/s rd, 5.5 KiB/s wr, 25 op/s 2024-03-28T18:15:46.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:15:46 smithi079 ceph-mon[35312]: pgmap v272: 97 pgs: 97 active+clean; 117 MiB data, 794 MiB used, 1.0 TiB / 1.0 TiB avail; 110 KiB/s rd, 5.5 KiB/s wr, 25 op/s 2024-03-28T18:15:48.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:15:48 smithi022 ceph-mon[31985]: pgmap v273: 97 pgs: 97 active+clean; 117 MiB data, 794 MiB used, 1.0 TiB / 1.0 TiB avail; 95 KiB/s rd, 4.8 KiB/s wr, 22 op/s 2024-03-28T18:15:48.674 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:15:48 smithi134 ceph-mon[35037]: pgmap v273: 97 pgs: 97 active+clean; 117 MiB data, 794 MiB used, 1.0 TiB / 1.0 TiB avail; 95 KiB/s rd, 4.8 KiB/s wr, 22 op/s 2024-03-28T18:15:48.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:15:48 smithi079 ceph-mon[35312]: pgmap v273: 97 pgs: 97 active+clean; 117 MiB data, 794 MiB used, 1.0 TiB / 1.0 TiB avail; 95 KiB/s rd, 4.8 KiB/s wr, 22 op/s 2024-03-28T18:15:49.570 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-28T18:15:49.570 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-28T18:15:49.812 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:15:49.812+0000 7f8027000640 1 -- 172.21.15.22:0/2653841068 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f802010bcb0 msgr2=0x7f802010e0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:15:49.812 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:15:49.812+0000 7f8027000640 1 --2- 172.21.15.22:0/2653841068 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f802010bcb0 0x7f802010e0e0 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7f8018009ca0 tx=0x7f801802c190 comp rx=0 tx=0).stop 2024-03-28T18:15:49.812 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:15:49.812+0000 7f8027000640 1 -- 172.21.15.22:0/2653841068 shutdown_connections 2024-03-28T18:15:49.812 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:15:49.812+0000 7f8027000640 1 --2- 172.21.15.22:0/2653841068 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f802010bcb0 0x7f802010e0e0 unknown :-1 s=CLOSED pgs=760 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:15:49.813 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:15:49.812+0000 7f8027000640 1 --2- 172.21.15.22:0/2653841068 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8020103200 0x7f802010b6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:15:49.813 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:15:49.812+0000 7f8027000640 1 --2- 172.21.15.22:0/2653841068 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f80201028e0 0x7f8020102cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:15:49.813 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:15:49.812+0000 7f8027000640 1 -- 172.21.15.22:0/2653841068 >> 172.21.15.22:0/2653841068 conn(0x7f8020076920 msgr2=0x7f8020076d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:15:49.813 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:15:49.812+0000 7f8027000640 1 -- 172.21.15.22:0/2653841068 shutdown_connections 2024-03-28T18:15:49.813 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:15:49.812+0000 7f8027000640 1 -- 172.21.15.22:0/2653841068 wait complete. 2024-03-28T18:15:49.815 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:15:49.813+0000 7f8027000640 1 Processor -- start 2024-03-28T18:15:49.815 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:15:49.814+0000 7f8027000640 1 -- start start 2024-03-28T18:15:49.815 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:15:49.814+0000 7f8027000640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f80201028e0 0x7f802019a100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:15:49.815 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:15:49.814+0000 7f8027000640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f8020103200 0x7f802019a640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:15:49.815 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:15:49.814+0000 7f8027000640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f802010bcb0 0x7f802019e9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:15:49.815 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:15:49.814+0000 7f8027000640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8020111a10 con 0x7f802010bcb0 2024-03-28T18:15:49.815 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:15:49.814+0000 7f8027000640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8020111890 con 0x7f8020103200 2024-03-28T18:15:49.815 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:15:49.814+0000 7f8027000640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8020111b90 con 0x7f80201028e0 2024-03-28T18:15:49.815 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:15:49.814+0000 7f8025000640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f802010bcb0 0x7f802019e9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:15:49.815 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:15:49.814+0000 7f8025000640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f802010bcb0 0x7f802019e9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:36188/0 (socket says 172.21.15.22:36188) 2024-03-28T18:15:49.816 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:15:49.814+0000 7f8025000640 1 -- 172.21.15.22:0/372131471 learned_addr learned my addr 172.21.15.22:0/372131471 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-28T18:15:49.816 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:15:49.814+0000 7f8017e00640 1 --2- 172.21.15.22:0/372131471 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f8020103200 0x7f802019a640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:15:49.816 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:15:49.815+0000 7f801fe00640 1 --2- 172.21.15.22:0/372131471 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f80201028e0 0x7f802019a100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:15:49.816 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:15:49.815+0000 7f8025000640 1 -- 172.21.15.22:0/372131471 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f80201028e0 msgr2=0x7f802019a100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:15:49.816 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:15:49.815+0000 7f8025000640 1 --2- 172.21.15.22:0/372131471 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f80201028e0 0x7f802019a100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:15:49.816 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:15:49.815+0000 7f8025000640 1 -- 172.21.15.22:0/372131471 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f8020103200 msgr2=0x7f802019a640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:15:49.816 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:15:49.815+0000 7f8025000640 1 --2- 172.21.15.22:0/372131471 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f8020103200 0x7f802019a640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:15:49.816 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:15:49.815+0000 7f8025000640 1 -- 172.21.15.22:0/372131471 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f802019f0f0 con 0x7f802010bcb0 2024-03-28T18:15:49.816 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:15:49.815+0000 7f8017e00640 1 --2- 172.21.15.22:0/372131471 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f8020103200 0x7f802019a640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-28T18:15:49.816 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:15:49.815+0000 7f8025000640 1 --2- 172.21.15.22:0/372131471 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f802010bcb0 0x7f802019e9c0 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7f8018005e30 tx=0x7f8018034cf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:15:49.817 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:15:49.816+0000 7f801d600640 1 -- 172.21.15.22:0/372131471 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8018039040 con 0x7f802010bcb0 2024-03-28T18:15:49.817 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:15:49.816+0000 7f801d600640 1 -- 172.21.15.22:0/372131471 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f8018003b70 con 0x7f802010bcb0 2024-03-28T18:15:49.817 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:15:49.816+0000 7f8027000640 1 -- 172.21.15.22:0/372131471 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f802019f380 con 0x7f802010bcb0 2024-03-28T18:15:49.817 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:15:49.816+0000 7f801d600640 1 -- 172.21.15.22:0/372131471 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8018032870 con 0x7f802010bcb0 2024-03-28T18:15:49.817 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:15:49.816+0000 7f8027000640 1 -- 172.21.15.22:0/372131471 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f80201a6c70 con 0x7f802010bcb0 2024-03-28T18:15:49.819 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:15:49.819+0000 7f801d600640 1 -- 172.21.15.22:0/372131471 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 26) v1 ==== 137421+0+0 (secure 0 0 0) 0x7f8018032a10 con 0x7f802010bcb0 2024-03-28T18:15:49.819 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:15:49.819+0000 7f8029da2740 1 Processor -- start 2024-03-28T18:15:49.819 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:15:49.820+0000 7f8029da2740 1 -- start start 2024-03-28T18:15:49.821 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:15:49.820+0000 7f8029da2740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5620e479dc00 0x5620e479c950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:15:49.821 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:15:49.820+0000 7f8029da2740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5620e479cf20 0x5620e479d3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:15:49.821 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:15:49.820+0000 7f8029da2740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5620e47a6290 0x5620e47a8680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:15:49.821 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:15:49.820+0000 7f8029da2740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x5620e4701250 con 0x5620e479dc00 2024-03-28T18:15:49.821 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:15:49.820+0000 7f8029da2740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x5620e46a9a30 con 0x5620e47a6290 2024-03-28T18:15:49.822 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:15:49.820+0000 7f8029da2740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x5620e46fd2f0 con 0x5620e479cf20 2024-03-28T18:15:49.822 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:15:49.820+0000 7f8017e00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5620e479dc00 0x5620e479c950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:15:49.822 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:15:49.820+0000 7f8017e00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5620e479dc00 0x5620e479c950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:36200/0 (socket says 172.21.15.22:36200) 2024-03-28T18:15:49.822 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:15:49.820+0000 7f8017e00640 1 -- 172.21.15.22:0/2171951099 learned_addr learned my addr 172.21.15.22:0/2171951099 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-28T18:15:49.822 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:15:49.821+0000 7f8017e00640 1 --2- 172.21.15.22:0/2171951099 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5620e47a6290 0x5620e47a8680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:15:49.822 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:15:49.821+0000 7f801fe00640 1 --2- 172.21.15.22:0/2171951099 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5620e479cf20 0x5620e479d3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:15:49.822 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:15:49.821+0000 7f8017e00640 1 -- 172.21.15.22:0/2171951099 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5620e479cf20 msgr2=0x5620e479d3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T18:15:49.822 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:15:49.821+0000 7f8017e00640 1 --2- 172.21.15.22:0/2171951099 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5620e479cf20 0x5620e479d3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:15:49.822 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:15:49.821+0000 7f8017e00640 1 -- 172.21.15.22:0/2171951099 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5620e47a6290 msgr2=0x5620e47a8680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T18:15:49.822 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:15:49.821+0000 7f8017e00640 1 --2- 172.21.15.22:0/2171951099 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5620e47a6290 0x5620e47a8680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:15:49.822 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:15:49.821+0000 7f8017e00640 1 -- 172.21.15.22:0/2171951099 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5620e4694690 con 0x5620e479dc00 2024-03-28T18:15:49.822 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:15:49.821+0000 7f801d600640 1 --2- 172.21.15.22:0/372131471 >> [v2:172.21.15.22:6800/1357365082,v1:172.21.15.22:6801/1357365082] conn(0x7f80040a2970 0x7f80040a4e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:15:49.822 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:15:49.821+0000 7f801fe00640 1 --2- 172.21.15.22:0/2171951099 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5620e479cf20 0x5620e479d3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-28T18:15:49.822 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:15:49.821+0000 7f801d600640 1 -- 172.21.15.22:0/372131471 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(116..116 src has 1..116) v4 ==== 10045+0+0 (secure 0 0 0) 0x7f80180f1580 con 0x7f802010bcb0 2024-03-28T18:15:49.823 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:15:49.821+0000 7f8017e00640 1 --2- 172.21.15.22:0/2171951099 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5620e479dc00 0x5620e479c950 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x7f8008012b10 tx=0x7f800800bb00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a344297392ecca8c server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:15:49.823 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:15:49.821+0000 7f8025000640 1 --2- 172.21.15.22:0/372131471 >> [v2:172.21.15.22:6800/1357365082,v1:172.21.15.22:6801/1357365082] conn(0x7f80040a2970 0x7f80040a4e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:15:49.823 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:15:49.822+0000 7f8015600640 1 -- 172.21.15.22:0/2171951099 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8008004030 con 0x5620e479dc00 2024-03-28T18:15:49.823 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:15:49.822+0000 7f8015600640 1 -- 172.21.15.22:0/2171951099 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f8008010cd0 con 0x5620e479dc00 2024-03-28T18:15:49.823 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:15:49.822+0000 7f8015600640 1 -- 172.21.15.22:0/2171951099 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8008003bb0 con 0x5620e479dc00 2024-03-28T18:15:49.827 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:15:49.823+0000 7f8029da2740 1 -- 172.21.15.22:0/2171951099 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5620e479dc00 msgr2=0x5620e479c950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:15:49.827 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:15:49.823+0000 7f8025000640 1 --2- 172.21.15.22:0/372131471 >> [v2:172.21.15.22:6800/1357365082,v1:172.21.15.22:6801/1357365082] conn(0x7f80040a2970 0x7f80040a4e30 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f80180ee7f0 tx=0x7f801802c9b0 comp rx=0 tx=0).ready entity=mgr.15522 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:15:49.827 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:15:49.823+0000 7f8029da2740 1 --2- 172.21.15.22:0/2171951099 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5620e479dc00 0x5620e479c950 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x7f8008012b10 tx=0x7f800800bb00 comp rx=0 tx=0).stop 2024-03-28T18:15:49.827 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:15:49.823+0000 7f8029da2740 1 -- 172.21.15.22:0/2171951099 shutdown_connections 2024-03-28T18:15:49.827 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:15:49.823+0000 7f8029da2740 1 --2- 172.21.15.22:0/2171951099 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5620e47a6290 0x5620e47a8680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:15:49.827 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:15:49.823+0000 7f8029da2740 1 --2- 172.21.15.22:0/2171951099 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5620e479cf20 0x5620e479d3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:15:49.827 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:15:49.823+0000 7f8029da2740 1 --2- 172.21.15.22:0/2171951099 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5620e479dc00 0x5620e479c950 unknown :-1 s=CLOSED pgs=762 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:15:49.827 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:15:49.823+0000 7f8029da2740 1 -- 172.21.15.22:0/2171951099 >> 172.21.15.22:0/2171951099 conn(0x5620e4764200 msgr2=0x5620e4797970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:15:49.827 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:15:49.823+0000 7f8029da2740 1 -- 172.21.15.22:0/2171951099 shutdown_connections 2024-03-28T18:15:49.827 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:15:49.823+0000 7f8029da2740 1 -- 172.21.15.22:0/2171951099 wait complete. 2024-03-28T18:15:49.828 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:15:49.823+0000 7f8029da2740 1 -- start start 2024-03-28T18:15:49.828 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:15:49.823+0000 7f8029da2740 1 Processor -- start 2024-03-28T18:15:49.828 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:15:49.824+0000 7f8029da2740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5620e479cf20 0x5620e47a5920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:15:49.828 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:15:49.824+0000 7f8029da2740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5620e479dc00 0x5620e47a2cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:15:49.828 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:15:49.824+0000 7f8029da2740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5620e47a6290 0x5620e47a31f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:15:49.828 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:15:49.824+0000 7f8029da2740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x5620e47a90a0 con 0x5620e479dc00 2024-03-28T18:15:49.828 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:15:49.824+0000 7f8029da2740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x5620e46bce80 con 0x5620e479cf20 2024-03-28T18:15:49.828 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:15:49.824+0000 7f8029da2740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x5620e47a9220 con 0x5620e47a6290 2024-03-28T18:15:49.828 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:15:49.824+0000 7f8029da2740 10 client.0 _ll_register_callbacks cb 0x5620e46cf130 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-28T18:15:49.828 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:15:49.824+0000 7f801fe00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5620e479dc00 0x5620e47a2cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:15:49.828 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:15:49.824+0000 7f801fe00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5620e479dc00 0x5620e47a2cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:36210/0 (socket says 172.21.15.22:36210) 2024-03-28T18:15:49.828 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:15:49.824+0000 7f801fe00640 1 -- 172.21.15.22:0/1032687273 learned_addr learned my addr 172.21.15.22:0/1032687273 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-28T18:15:49.828 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:15:49.824+0000 7f8017e00640 1 --2- 172.21.15.22:0/1032687273 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5620e479cf20 0x5620e47a5920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:15:49.828 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:15:49.825+0000 7f801fe00640 1 -- 172.21.15.22:0/1032687273 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5620e47a6290 msgr2=0x5620e47a31f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T18:15:49.829 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:15:49.825+0000 7f801fe00640 1 --2- 172.21.15.22:0/1032687273 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5620e47a6290 0x5620e47a31f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:15:49.829 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:15:49.825+0000 7f801fe00640 1 -- 172.21.15.22:0/1032687273 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5620e479cf20 msgr2=0x5620e47a5920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T18:15:49.829 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:15:49.825+0000 7f801fe00640 1 --2- 172.21.15.22:0/1032687273 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5620e479cf20 0x5620e47a5920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:15:49.829 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:15:49.825+0000 7f801fe00640 1 -- 172.21.15.22:0/1032687273 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5620e44df7d0 con 0x5620e479dc00 2024-03-28T18:15:49.829 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:15:49.825+0000 7f8017e00640 1 --2- 172.21.15.22:0/1032687273 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5620e479cf20 0x5620e47a5920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-28T18:15:49.829 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:15:49.825+0000 7f801fe00640 1 --2- 172.21.15.22:0/1032687273 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5620e479dc00 0x5620e47a2cb0 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 crypto rx=0x7f80100126b0 tx=0x7f8010012b70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=71cdcb43b48208fd server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:15:49.829 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:15:49.825+0000 7f800e000640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-03-28T18:15:49.829 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:15:49.825+0000 7f800e000640 1 -- 172.21.15.22:0/1032687273 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8010004030 con 0x5620e479dc00 2024-03-28T18:15:49.829 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:15:49.825+0000 7f800e000640 1 -- 172.21.15.22:0/1032687273 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(116..116 src has 1..116) v4 ==== 10045+0+0 (secure 0 0 0) 0x7f80100041d0 con 0x5620e479dc00 2024-03-28T18:15:49.829 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:15:49.825+0000 7f8029da2740 1 -- 172.21.15.22:0/1032687273 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f8008010cd0 con 0x5620e479dc00 2024-03-28T18:15:49.829 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:15:49.826+0000 7f800e000640 1 -- 172.21.15.22:0/1032687273 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=161) v2 ==== 24+0+0 (secure 0 0 0) 0x7f80100043d0 con 0x5620e479dc00 2024-03-28T18:15:49.829 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:15:49.826+0000 7f8029da2740 10 client.16746 fetch_fsmap learned FSMap version 161 2024-03-28T18:15:49.829 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:15:49.826+0000 7f8029da2740 1 -- 172.21.15.22:0/1032687273 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=161,monmap=4+}) v3 -- 0x5620e44153f0 con 0x5620e479dc00 2024-03-28T18:15:49.829 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:15:49.826+0000 7f800e000640 1 -- 172.21.15.22:0/1032687273 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f8008010cd0 con 0x5620e479dc00 2024-03-28T18:15:49.830 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:15:49.826+0000 7f800e000640 1 -- 172.21.15.22:0/1032687273 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 161) v1 ==== 6784+0+0 (secure 0 0 0) 0x7f8010003be0 con 0x5620e479dc00 2024-03-28T18:15:49.830 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:15:49.826+0000 7f8029da2740 10 client.16746 fetch_fsmap finished waiting for FSMap version 161 2024-03-28T18:15:49.830 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:15:49.826+0000 7f8029da2740 10 client.16746 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-28T18:15:49.830 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:15:49.826+0000 7f8029da2740 20 client.16746 populate_metadata read hostname 'smithi022' 2024-03-28T18:15:49.830 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:15:49.826+0000 7f8029da2740 1 --2- 172.21.15.22:0/1032687273 >> [v2:172.21.15.79:6838/2441542443,v1:172.21.15.79:6839/2441542443] conn(0x5620e47b61d0 0x5620e47b85f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:15:49.830 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:15:49.826+0000 7f8029da2740 4 client.16746 mds_command: new command op to 34553 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-28T18:15:49.830 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:15:49.826+0000 7f8029da2740 1 -- 172.21.15.22:0/1032687273 --> [v2:172.21.15.79:6838/2441542443,v1:172.21.15.79:6839/2441542443] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5620e470e790 con 0x5620e47b61d0 2024-03-28T18:15:49.830 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:15:49.827+0000 7f8017e00640 1 --2- 172.21.15.22:0/1032687273 >> [v2:172.21.15.79:6838/2441542443,v1:172.21.15.79:6839/2441542443] conn(0x5620e47b61d0 0x5620e47b85f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:15:49.831 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:15:49.828+0000 7f8017e00640 1 --2- 172.21.15.22:0/1032687273 >> [v2:172.21.15.79:6838/2441542443,v1:172.21.15.79:6839/2441542443] conn(0x5620e47b61d0 0x5620e47b85f0 crc :-1 s=READY pgs=252 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a06d574cd34a135c server_cookie=6c885e30a79fc317 in_seq=0 out_seq=0 2024-03-28T18:15:49.831 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:15:49.828+0000 7f800e000640 10 client.16746 ms_handle_connect on v2:172.21.15.79:6838/2441542443 2024-03-28T18:15:49.831 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:15:49.829+0000 7f800e000640 1 -- 172.21.15.22:0/1032687273 <== mds.0 v2:172.21.15.79:6838/2441542443 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x5620e470e790 con 0x5620e47b61d0 2024-03-28T18:15:49.831 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:15:49.829+0000 7f800e000640 10 client.16746 handle_command_reply: tid=0 2024-03-28T18:15:49.831 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:15:49.829+0000 7f8029da2740 1 client.16746 shutdown 2024-03-28T18:15:49.832 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:15:49.830+0000 7f8029da2740 1 -- 172.21.15.22:0/1032687273 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5620e479dc00 msgr2=0x5620e47a2cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:15:49.832 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:15:49.830+0000 7f8029da2740 1 --2- 172.21.15.22:0/1032687273 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5620e479dc00 0x5620e47a2cb0 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 crypto rx=0x7f80100126b0 tx=0x7f8010012b70 comp rx=0 tx=0).stop 2024-03-28T18:15:49.832 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:15:49.830+0000 7f8029da2740 1 -- 172.21.15.22:0/1032687273 shutdown_connections 2024-03-28T18:15:49.832 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:15:49.830+0000 7f8029da2740 1 --2- 172.21.15.22:0/1032687273 >> [v2:172.21.15.79:6838/2441542443,v1:172.21.15.79:6839/2441542443] conn(0x5620e47b61d0 0x5620e47b85f0 crc :-1 s=READY pgs=252 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:15:49.832 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:15:49.830+0000 7f8029da2740 1 --2- 172.21.15.22:0/1032687273 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5620e47a6290 0x5620e47a31f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:15:49.832 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:15:49.830+0000 7f8029da2740 1 --2- 172.21.15.22:0/1032687273 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5620e479dc00 0x5620e47a2cb0 unknown :-1 s=CLOSED pgs=763 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:15:49.832 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:15:49.830+0000 7f8029da2740 1 --2- 172.21.15.22:0/1032687273 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5620e479cf20 0x5620e47a5920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:15:49.832 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:15:49.830+0000 7f800e000640 0 client.16746 ms_handle_reset on v2:172.21.15.79:6838/2441542443 2024-03-28T18:15:49.832 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:15:49.830+0000 7f8029da2740 1 -- 172.21.15.22:0/1032687273 >> 172.21.15.22:0/1032687273 conn(0x5620e4764200 msgr2=0x5620e4796200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:15:49.832 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:15:49.830+0000 7f8029da2740 1 -- 172.21.15.22:0/1032687273 shutdown_connections 2024-03-28T18:15:49.832 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:15:49.830+0000 7f8029da2740 1 -- 172.21.15.22:0/1032687273 wait complete. 2024-03-28T18:15:49.832 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:15:49.830+0000 7f8029da2740 20 client.16746 trim_cache size 0 max 16384 2024-03-28T18:15:49.848 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:15:49.848+0000 7f8029da2740 1 Processor -- start 2024-03-28T18:15:49.848 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:15:49.848+0000 7f8029da2740 1 -- start start 2024-03-28T18:15:49.848 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:15:49.848+0000 7f8029da2740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5620e479cf20 0x5620e47995e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:15:49.848 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:15:49.848+0000 7f8029da2740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5620e479dc00 0x5620e47a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:15:49.848 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:15:49.848+0000 7f8029da2740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5620e47a6290 0x5620e47a5920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:15:49.848 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:15:49.848+0000 7f8029da2740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80201aeb60 con 0x5620e479cf20 2024-03-28T18:15:49.848 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:15:49.848+0000 7f8029da2740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80201b11a0 con 0x5620e479dc00 2024-03-28T18:15:49.848 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:15:49.848+0000 7f8029da2740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80201af1c0 con 0x5620e47a6290 2024-03-28T18:15:49.848 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:15:49.848+0000 7f8017e00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5620e479cf20 0x5620e47995e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:15:49.849 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:15:49.848+0000 7f8017e00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5620e479cf20 0x5620e47995e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:36222/0 (socket says 172.21.15.22:36222) 2024-03-28T18:15:49.849 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:15:49.848+0000 7f8017e00640 1 -- 172.21.15.22:0/2331785032 learned_addr learned my addr 172.21.15.22:0/2331785032 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-28T18:15:49.849 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:15:49.848+0000 7f801fe00640 1 --2- 172.21.15.22:0/2331785032 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5620e479dc00 0x5620e47a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:15:49.850 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:15:49.849+0000 7f8017e00640 1 -- 172.21.15.22:0/2331785032 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5620e47a6290 msgr2=0x5620e47a5920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T18:15:49.850 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:15:49.849+0000 7f8017e00640 1 --2- 172.21.15.22:0/2331785032 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5620e47a6290 0x5620e47a5920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:15:49.851 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:15:49.849+0000 7f8017e00640 1 -- 172.21.15.22:0/2331785032 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5620e479dc00 msgr2=0x5620e47a53e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T18:15:49.851 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:15:49.849+0000 7f8017e00640 1 --2- 172.21.15.22:0/2331785032 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5620e479dc00 0x5620e47a53e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:15:49.851 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:15:49.849+0000 7f8017e00640 1 -- 172.21.15.22:0/2331785032 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5620e479e8b0 con 0x5620e479cf20 2024-03-28T18:15:49.851 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:15:49.849+0000 7f8017e00640 1 --2- 172.21.15.22:0/2331785032 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5620e479cf20 0x5620e47995e0 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7f800800b3d0 tx=0x7f8008017040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6eca464f8874bb5e server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:15:49.851 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:15:49.849+0000 7f8014c00640 1 -- 172.21.15.22:0/2331785032 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8008030070 con 0x5620e479cf20 2024-03-28T18:15:49.851 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:15:49.849+0000 7f8014c00640 1 -- 172.21.15.22:0/2331785032 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f8008016030 con 0x5620e479cf20 2024-03-28T18:15:49.851 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:15:49.849+0000 7f8014c00640 1 -- 172.21.15.22:0/2331785032 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f800802a690 con 0x5620e479cf20 2024-03-28T18:15:49.851 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:15:49.850+0000 7f8029da2740 1 -- 172.21.15.22:0/2331785032 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5620e479cf20 msgr2=0x5620e47995e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:15:49.851 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:15:49.850+0000 7f8029da2740 1 --2- 172.21.15.22:0/2331785032 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5620e479cf20 0x5620e47995e0 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7f800800b3d0 tx=0x7f8008017040 comp rx=0 tx=0).stop 2024-03-28T18:15:49.851 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:15:49.850+0000 7f8029da2740 1 -- 172.21.15.22:0/2331785032 shutdown_connections 2024-03-28T18:15:49.851 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:15:49.850+0000 7f8029da2740 1 --2- 172.21.15.22:0/2331785032 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5620e47a6290 0x5620e47a5920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:15:49.851 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:15:49.850+0000 7f8029da2740 1 --2- 172.21.15.22:0/2331785032 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5620e479dc00 0x5620e47a53e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:15:49.851 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:15:49.850+0000 7f8029da2740 1 --2- 172.21.15.22:0/2331785032 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5620e479cf20 0x5620e47995e0 unknown :-1 s=CLOSED pgs=764 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:15:49.851 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:15:49.850+0000 7f8029da2740 1 -- 172.21.15.22:0/2331785032 >> 172.21.15.22:0/2331785032 conn(0x5620e4764200 msgr2=0x5620e4796200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:15:49.851 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:15:49.850+0000 7f8029da2740 1 -- 172.21.15.22:0/2331785032 shutdown_connections 2024-03-28T18:15:49.852 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:15:49.850+0000 7f8029da2740 1 -- 172.21.15.22:0/2331785032 wait complete. 2024-03-28T18:15:49.852 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:15:49.851+0000 7f8029da2740 1 -- start start 2024-03-28T18:15:49.852 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:15:49.851+0000 7f8029da2740 1 Processor -- start 2024-03-28T18:15:49.854 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:15:49.851+0000 7f8029da2740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5620e479cf20 0x5620e47adb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:15:49.854 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:15:49.851+0000 7f8029da2740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5620e479dc00 0x5620e47ae0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:15:49.854 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:15:49.851+0000 7f8029da2740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5620e47a6290 0x5620e47ac110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:15:49.854 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:15:49.851+0000 7f8029da2740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80201af820 con 0x5620e479dc00 2024-03-28T18:15:49.854 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:15:49.851+0000 7f8029da2740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80201b0b40 con 0x5620e479cf20 2024-03-28T18:15:49.854 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:15:49.851+0000 7f8029da2740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80201afe80 con 0x5620e47a6290 2024-03-28T18:15:49.854 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:15:49.851+0000 7f8029da2740 10 client.0 _ll_register_callbacks cb 0x7f8010009880 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-28T18:15:49.854 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:15:49.852+0000 7f801fe00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5620e479dc00 0x5620e47ae0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:15:49.855 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:15:49.852+0000 7f801fe00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5620e479dc00 0x5620e47ae0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:36238/0 (socket says 172.21.15.22:36238) 2024-03-28T18:15:49.855 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:15:49.852+0000 7f801fe00640 1 -- 172.21.15.22:0/3027982511 learned_addr learned my addr 172.21.15.22:0/3027982511 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-28T18:15:49.855 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:15:49.852+0000 7f8017e00640 1 --2- 172.21.15.22:0/3027982511 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5620e479cf20 0x5620e47adb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:15:49.855 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:15:49.852+0000 7f801fe00640 1 -- 172.21.15.22:0/3027982511 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5620e47a6290 msgr2=0x5620e47ac110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T18:15:49.855 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:15:49.852+0000 7f801fe00640 1 --2- 172.21.15.22:0/3027982511 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5620e47a6290 0x5620e47ac110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:15:49.855 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:15:49.852+0000 7f801fe00640 1 -- 172.21.15.22:0/3027982511 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5620e479cf20 msgr2=0x5620e47adb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T18:15:49.855 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:15:49.852+0000 7f801fe00640 1 --2- 172.21.15.22:0/3027982511 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5620e479cf20 0x5620e47adb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:15:49.855 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:15:49.852+0000 7f801fe00640 1 -- 172.21.15.22:0/3027982511 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5620e47b8960 con 0x5620e479dc00 2024-03-28T18:15:49.855 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:15:49.852+0000 7f801fe00640 1 --2- 172.21.15.22:0/3027982511 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5620e479dc00 0x5620e47ae0d0 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 crypto rx=0x7f80100058c0 tx=0x7f801000f8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=55ee8897e6daa2ae server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:15:49.855 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:15:49.852+0000 7f800e000640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-03-28T18:15:49.855 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:15:49.853+0000 7f800e000640 1 -- 172.21.15.22:0/3027982511 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f801000fbb0 con 0x5620e479dc00 2024-03-28T18:15:49.855 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:15:49.853+0000 7f8029da2740 1 -- 172.21.15.22:0/3027982511 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f8008016030 con 0x5620e479dc00 2024-03-28T18:15:49.855 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:15:49.853+0000 7f800e000640 1 -- 172.21.15.22:0/3027982511 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(116..116 src has 1..116) v4 ==== 10045+0+0 (secure 0 0 0) 0x7f801000fd50 con 0x5620e479dc00 2024-03-28T18:15:49.855 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:15:49.853+0000 7f800e000640 1 -- 172.21.15.22:0/3027982511 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=161) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8010036050 con 0x5620e479dc00 2024-03-28T18:15:49.856 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:15:49.853+0000 7f8029da2740 10 client.16752 fetch_fsmap learned FSMap version 161 2024-03-28T18:15:49.856 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:15:49.853+0000 7f8029da2740 1 -- 172.21.15.22:0/3027982511 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=161,monmap=4+}) v3 -- 0x5620e43d50f0 con 0x5620e479dc00 2024-03-28T18:15:49.856 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:15:49.853+0000 7f800e000640 1 -- 172.21.15.22:0/3027982511 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f8008016030 con 0x5620e479dc00 2024-03-28T18:15:49.856 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:15:49.853+0000 7f800e000640 1 -- 172.21.15.22:0/3027982511 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 161) v1 ==== 6784+0+0 (secure 0 0 0) 0x7f8010031030 con 0x5620e479dc00 2024-03-28T18:15:49.856 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:15:49.854+0000 7f8029da2740 10 client.16752 fetch_fsmap finished waiting for FSMap version 161 2024-03-28T18:15:49.856 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:15:49.854+0000 7f8029da2740 10 client.16752 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-28T18:15:49.856 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:15:49.854+0000 7f8029da2740 20 client.16752 populate_metadata read hostname 'smithi022' 2024-03-28T18:15:49.856 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:15:49.854+0000 7f8029da2740 1 --2- 172.21.15.22:0/3027982511 >> [v2:172.21.15.79:6838/2441542443,v1:172.21.15.79:6839/2441542443] conn(0x5620e48128a0 0x5620e4814cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:15:49.856 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:15:49.854+0000 7f8029da2740 4 client.16752 mds_command: new command op to 34553 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-28T18:15:49.856 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:15:49.854+0000 7f8029da2740 1 -- 172.21.15.22:0/3027982511 --> [v2:172.21.15.79:6838/2441542443,v1:172.21.15.79:6839/2441542443] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x5620e4708ae0 con 0x5620e48128a0 2024-03-28T18:15:49.856 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:15:49.854+0000 7f8017e00640 1 --2- 172.21.15.22:0/3027982511 >> [v2:172.21.15.79:6838/2441542443,v1:172.21.15.79:6839/2441542443] conn(0x5620e48128a0 0x5620e4814cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:15:49.857 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:15:49.855+0000 7f8017e00640 1 --2- 172.21.15.22:0/3027982511 >> [v2:172.21.15.79:6838/2441542443,v1:172.21.15.79:6839/2441542443] conn(0x5620e48128a0 0x5620e4814cc0 crc :-1 s=READY pgs=253 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4416f248ec9e6ee5 server_cookie=675921647bdbc6ce in_seq=0 out_seq=0 2024-03-28T18:15:49.857 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:15:49.855+0000 7f800e000640 10 client.16752 ms_handle_connect on v2:172.21.15.79:6838/2441542443 2024-03-28T18:15:49.857 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:15:49.855+0000 7f800e000640 1 -- 172.21.15.22:0/3027982511 <== mds.0 v2:172.21.15.79:6838/2441542443 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x5620e4708ae0 con 0x5620e48128a0 2024-03-28T18:15:49.858 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:15:49.855+0000 7f800e000640 10 client.16752 handle_command_reply: tid=0 2024-03-28T18:15:49.858 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:15:49.855+0000 7f8029da2740 1 client.16752 shutdown 2024-03-28T18:15:49.858 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:15:49.856+0000 7f8029da2740 1 -- 172.21.15.22:0/3027982511 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5620e479dc00 msgr2=0x5620e47ae0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:15:49.858 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:15:49.856+0000 7f8029da2740 1 --2- 172.21.15.22:0/3027982511 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5620e479dc00 0x5620e47ae0d0 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 crypto rx=0x7f80100058c0 tx=0x7f801000f8d0 comp rx=0 tx=0).stop 2024-03-28T18:15:49.858 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:15:49.856+0000 7f8029da2740 1 -- 172.21.15.22:0/3027982511 shutdown_connections 2024-03-28T18:15:49.858 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:15:49.856+0000 7f8029da2740 1 --2- 172.21.15.22:0/3027982511 >> [v2:172.21.15.79:6838/2441542443,v1:172.21.15.79:6839/2441542443] conn(0x5620e48128a0 0x5620e4814cc0 crc :-1 s=READY pgs=253 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:15:49.858 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:15:49.856+0000 7f8029da2740 1 --2- 172.21.15.22:0/3027982511 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5620e47a6290 0x5620e47ac110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:15:49.858 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:15:49.856+0000 7f8029da2740 1 --2- 172.21.15.22:0/3027982511 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5620e479dc00 0x5620e47ae0d0 unknown :-1 s=CLOSED pgs=765 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:15:49.858 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:15:49.856+0000 7f8029da2740 1 --2- 172.21.15.22:0/3027982511 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5620e479cf20 0x5620e47adb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:15:49.858 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:15:49.856+0000 7f800e000640 0 client.16752 ms_handle_reset on v2:172.21.15.79:6838/2441542443 2024-03-28T18:15:49.858 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:15:49.856+0000 7f8029da2740 1 -- 172.21.15.22:0/3027982511 >> 172.21.15.22:0/3027982511 conn(0x5620e4764200 msgr2=0x5620e4796200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:15:49.858 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:15:49.856+0000 7f8029da2740 1 -- 172.21.15.22:0/3027982511 shutdown_connections 2024-03-28T18:15:49.858 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:15:49.856+0000 7f8029da2740 1 -- 172.21.15.22:0/3027982511 wait complete. 2024-03-28T18:15:49.859 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:15:49.856+0000 7f8029da2740 20 client.16752 trim_cache size 0 max 16384 2024-03-28T18:15:49.859 INFO:teuthology.orchestra.run.smithi022.stdout:{ 2024-03-28T18:15:49.859 INFO:teuthology.orchestra.run.smithi022.stdout: "status": "IDLE (waiting for more scrubs)", 2024-03-28T18:15:49.859 INFO:teuthology.orchestra.run.smithi022.stdout: "scrubs": { 2024-03-28T18:15:49.859 INFO:teuthology.orchestra.run.smithi022.stdout: "4bbbd61d-29a3-4bf6-a4f2-d5783d585de3": { 2024-03-28T18:15:49.859 INFO:teuthology.orchestra.run.smithi022.stdout: "path": "/", 2024-03-28T18:15:49.859 INFO:teuthology.orchestra.run.smithi022.stdout: "tag": "4bbbd61d-29a3-4bf6-a4f2-d5783d585de3", 2024-03-28T18:15:49.859 INFO:teuthology.orchestra.run.smithi022.stdout: "options": "recursive,force" 2024-03-28T18:15:49.859 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-03-28T18:15:49.859 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-03-28T18:15:49.859 INFO:teuthology.orchestra.run.smithi022.stdout:} 2024-03-28T18:15:49.860 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:15:49.857+0000 7f8027000640 1 -- 172.21.15.22:0/372131471 >> [v2:172.21.15.22:6800/1357365082,v1:172.21.15.22:6801/1357365082] conn(0x7f80040a2970 msgr2=0x7f80040a4e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:15:49.860 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:15:49.857+0000 7f8027000640 1 --2- 172.21.15.22:0/372131471 >> [v2:172.21.15.22:6800/1357365082,v1:172.21.15.22:6801/1357365082] conn(0x7f80040a2970 0x7f80040a4e30 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f80180ee7f0 tx=0x7f801802c9b0 comp rx=0 tx=0).stop 2024-03-28T18:15:49.860 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:15:49.857+0000 7f8027000640 1 -- 172.21.15.22:0/372131471 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f802010bcb0 msgr2=0x7f802019e9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:15:49.860 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:15:49.857+0000 7f8027000640 1 --2- 172.21.15.22:0/372131471 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f802010bcb0 0x7f802019e9c0 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7f8018005e30 tx=0x7f8018034cf0 comp rx=0 tx=0).stop 2024-03-28T18:15:49.860 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:15:49.857+0000 7f8027000640 1 -- 172.21.15.22:0/372131471 shutdown_connections 2024-03-28T18:15:49.860 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:15:49.857+0000 7f8027000640 1 --2- 172.21.15.22:0/372131471 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f802010bcb0 0x7f802019e9c0 unknown :-1 s=CLOSED pgs=761 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:15:49.860 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:15:49.857+0000 7f8027000640 1 --2- 172.21.15.22:0/372131471 >> [v2:172.21.15.22:6800/1357365082,v1:172.21.15.22:6801/1357365082] conn(0x7f80040a2970 0x7f80040a4e30 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:15:49.861 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:15:49.857+0000 7f8027000640 1 --2- 172.21.15.22:0/372131471 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f8020103200 0x7f802019a640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:15:49.861 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:15:49.857+0000 7f8027000640 1 --2- 172.21.15.22:0/372131471 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f80201028e0 0x7f802019a100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:15:49.861 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:15:49.857+0000 7f8027000640 1 -- 172.21.15.22:0/372131471 >> 172.21.15.22:0/372131471 conn(0x7f8020076920 msgr2=0x7f80200fe4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:15:49.861 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:15:49.857+0000 7f8027000640 1 -- 172.21.15.22:0/372131471 shutdown_connections 2024-03-28T18:15:49.861 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:15:49.857+0000 7f8027000640 1 -- 172.21.15.22:0/372131471 wait complete. 2024-03-28T18:15:49.875 INFO:tasks.cephfs.filesystem:scrub status for tag:4bbbd61d-29a3-4bf6-a4f2-d5783d585de3 - {'path': '/', 'tag': '4bbbd61d-29a3-4bf6-a4f2-d5783d585de3', 'options': 'recursive,force'} 2024-03-28T18:15:50.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:15:50 smithi022 ceph-mon[31985]: pgmap v274: 97 pgs: 97 active+clean; 117 MiB data, 794 MiB used, 1.0 TiB / 1.0 TiB avail; 94 KiB/s rd, 4.6 KiB/s wr, 25 op/s 2024-03-28T18:15:50.674 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:15:50 smithi134 ceph-mon[35037]: pgmap v274: 97 pgs: 97 active+clean; 117 MiB data, 794 MiB used, 1.0 TiB / 1.0 TiB avail; 94 KiB/s rd, 4.6 KiB/s wr, 25 op/s 2024-03-28T18:15:50.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:15:50 smithi079 ceph-mon[35312]: pgmap v274: 97 pgs: 97 active+clean; 117 MiB data, 794 MiB used, 1.0 TiB / 1.0 TiB avail; 94 KiB/s rd, 4.6 KiB/s wr, 25 op/s 2024-03-28T18:15:52.185 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:15:51 smithi022 systemd[1]: ceph-fd220748-ed2b-11ee-95d0-87774f69a715@mds.j.service: Scheduled restart job, restart counter is at 4. 2024-03-28T18:15:52.185 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:15:51 smithi022 systemd[1]: Stopped Ceph mds.j for fd220748-ed2b-11ee-95d0-87774f69a715. 2024-03-28T18:15:52.185 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:15:51 smithi022 systemd[1]: ceph-fd220748-ed2b-11ee-95d0-87774f69a715@mds.j.service: Consumed 2.030s CPU time. 2024-03-28T18:15:52.185 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:15:51 smithi022 systemd[1]: Starting Ceph mds.j for fd220748-ed2b-11ee-95d0-87774f69a715... 2024-03-28T18:15:52.497 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:15:52 smithi022 ceph-mon[31985]: pgmap v275: 97 pgs: 97 active+clean; 117 MiB data, 794 MiB used, 1.0 TiB / 1.0 TiB avail; 91 KiB/s rd, 4.2 KiB/s wr, 21 op/s 2024-03-28T18:15:52.497 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:15:52 smithi022 podman[94888]: 2024-03-28 18:15:52.092861367 +0000 UTC m=+0.039964685 image pull c664afe56bb01db4f9afa9532c20598e201e664f2e9ba2be50db0d52674a26d2 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc 2024-03-28T18:15:52.498 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:15:52 smithi022 podman[94888]: 2024-03-28 18:15:52.208379051 +0000 UTC m=+0.155482406 container create bfd01a52e140e2808593024599ff285cd4ff7d4dab980a8e8111fe3ce38a29a5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, ceph=True, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240319, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, CEPH_POINT_RELEASE=) 2024-03-28T18:15:52.498 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:15:52 smithi022 podman[94888]: 2024-03-28 18:15:52.497688994 +0000 UTC m=+0.444792333 container init bfd01a52e140e2808593024599ff285cd4ff7d4dab980a8e8111fe3ce38a29a5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240319, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, ceph=True, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5) 2024-03-28T18:15:52.674 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:15:52 smithi134 ceph-mon[35037]: pgmap v275: 97 pgs: 97 active+clean; 117 MiB data, 794 MiB used, 1.0 TiB / 1.0 TiB avail; 91 KiB/s rd, 4.2 KiB/s wr, 21 op/s 2024-03-28T18:15:52.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:15:52 smithi079 ceph-mon[35312]: pgmap v275: 97 pgs: 97 active+clean; 117 MiB data, 794 MiB used, 1.0 TiB / 1.0 TiB avail; 91 KiB/s rd, 4.2 KiB/s wr, 21 op/s 2024-03-28T18:15:52.782 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:15:52 smithi022 podman[94888]: 2024-03-28 18:15:52.50880449 +0000 UTC m=+0.455907812 container start bfd01a52e140e2808593024599ff285cd4ff7d4dab980a8e8111fe3ce38a29a5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, ceph=True, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240319) 2024-03-28T18:15:52.782 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:15:52 smithi022 ceph-mds[94906]: -- 172.21.15.22:0/1214220419 <== mon.0 v2:172.21.15.22:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x56107cee9a40 con 0x56107cebc800 2024-03-28T18:15:52.783 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:15:52 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[94902]: starting mds.j at 2024-03-28T18:15:52.783 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:15:52 smithi022 bash[94888]: bfd01a52e140e2808593024599ff285cd4ff7d4dab980a8e8111fe3ce38a29a5 2024-03-28T18:15:52.783 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:15:52 smithi022 systemd[1]: Started Ceph mds.j for fd220748-ed2b-11ee-95d0-87774f69a715. 2024-03-28T18:15:53.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:15:53 smithi134 ceph-mon[35037]: daemon mds.j restarted 2024-03-28T18:15:53.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:15:53 smithi134 ceph-mon[35037]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-03-28T18:15:53.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:15:53 smithi134 ceph-mon[35037]: osdmap e117: 12 total, 12 up, 12 in 2024-03-28T18:15:53.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:15:53 smithi134 ceph-mon[35037]: mds.? [v2:172.21.15.22:6834/817442472,v1:172.21.15.22:6835/817442472] up:boot 2024-03-28T18:15:53.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:15:53 smithi134 ceph-mon[35037]: Standby daemon mds.d assigned to filesystem cephfs as rank 1 2024-03-28T18:15:53.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:15:53 smithi134 ceph-mon[35037]: fsmap cephfs:4/5 {0=b=up:active,2=e=up:active,3=k=up:active,4=i=up:active} 5 up:standby-replay 1 up:standby, 1 failed 2024-03-28T18:15:53.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:15:53 smithi134 ceph-mon[35037]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-03-28T18:15:53.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:15:53 smithi134 ceph-mon[35037]: fsmap cephfs:5/5 4 up:standby-replay 1 up:standby 1 up:replay 4 up:active 2024-03-28T18:15:53.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:15:53 smithi079 ceph-mon[35312]: daemon mds.j restarted 2024-03-28T18:15:53.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:15:53 smithi079 ceph-mon[35312]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-03-28T18:15:53.683 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:15:53 smithi079 ceph-mon[35312]: osdmap e117: 12 total, 12 up, 12 in 2024-03-28T18:15:53.683 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:15:53 smithi079 ceph-mon[35312]: mds.? [v2:172.21.15.22:6834/817442472,v1:172.21.15.22:6835/817442472] up:boot 2024-03-28T18:15:53.683 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:15:53 smithi079 ceph-mon[35312]: Standby daemon mds.d assigned to filesystem cephfs as rank 1 2024-03-28T18:15:53.683 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:15:53 smithi079 ceph-mon[35312]: fsmap cephfs:4/5 {0=b=up:active,2=e=up:active,3=k=up:active,4=i=up:active} 5 up:standby-replay 1 up:standby, 1 failed 2024-03-28T18:15:53.683 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:15:53 smithi079 ceph-mon[35312]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-03-28T18:15:53.683 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:15:53 smithi079 ceph-mon[35312]: fsmap cephfs:5/5 4 up:standby-replay 1 up:standby 1 up:replay 4 up:active 2024-03-28T18:15:53.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:15:53 smithi022 ceph-mon[31985]: daemon mds.j restarted 2024-03-28T18:15:53.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:15:53 smithi022 ceph-mon[31985]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-03-28T18:15:53.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:15:53 smithi022 ceph-mon[31985]: osdmap e117: 12 total, 12 up, 12 in 2024-03-28T18:15:53.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:15:53 smithi022 ceph-mon[31985]: mds.? [v2:172.21.15.22:6834/817442472,v1:172.21.15.22:6835/817442472] up:boot 2024-03-28T18:15:53.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:15:53 smithi022 ceph-mon[31985]: Standby daemon mds.d assigned to filesystem cephfs as rank 1 2024-03-28T18:15:53.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:15:53 smithi022 ceph-mon[31985]: fsmap cephfs:4/5 {0=b=up:active,2=e=up:active,3=k=up:active,4=i=up:active} 5 up:standby-replay 1 up:standby, 1 failed 2024-03-28T18:15:53.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:15:53 smithi022 ceph-mon[31985]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-03-28T18:15:53.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:15:53 smithi022 ceph-mon[31985]: fsmap cephfs:5/5 4 up:standby-replay 1 up:standby 1 up:replay 4 up:active 2024-03-28T18:15:54.925 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:15:54 smithi134 ceph-mon[35037]: pgmap v277: 97 pgs: 97 active+clean; 117 MiB data, 794 MiB used, 1.0 TiB / 1.0 TiB avail; 109 KiB/s rd, 5.1 KiB/s wr, 25 op/s 2024-03-28T18:15:54.925 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:15:54 smithi134 ceph-mon[35037]: mds.? [v2:172.21.15.22:6836/2141892169,v1:172.21.15.22:6837/2141892169] up:resolve 2024-03-28T18:15:54.925 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:15:54 smithi134 ceph-mon[35037]: fsmap cephfs:5/5 4 up:standby-replay 1 up:standby 1 up:resolve 4 up:active 2024-03-28T18:15:54.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:15:54 smithi079 ceph-mon[35312]: pgmap v277: 97 pgs: 97 active+clean; 117 MiB data, 794 MiB used, 1.0 TiB / 1.0 TiB avail; 109 KiB/s rd, 5.1 KiB/s wr, 25 op/s 2024-03-28T18:15:54.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:15:54 smithi079 ceph-mon[35312]: mds.? [v2:172.21.15.22:6836/2141892169,v1:172.21.15.22:6837/2141892169] up:resolve 2024-03-28T18:15:54.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:15:54 smithi079 ceph-mon[35312]: fsmap cephfs:5/5 4 up:standby-replay 1 up:standby 1 up:resolve 4 up:active 2024-03-28T18:15:55.031 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:15:54 smithi022 ceph-mon[31985]: pgmap v277: 97 pgs: 97 active+clean; 117 MiB data, 794 MiB used, 1.0 TiB / 1.0 TiB avail; 109 KiB/s rd, 5.1 KiB/s wr, 25 op/s 2024-03-28T18:15:55.032 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:15:54 smithi022 ceph-mon[31985]: mds.? [v2:172.21.15.22:6836/2141892169,v1:172.21.15.22:6837/2141892169] up:resolve 2024-03-28T18:15:55.032 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:15:54 smithi022 ceph-mon[31985]: fsmap cephfs:5/5 4 up:standby-replay 1 up:standby 1 up:resolve 4 up:active 2024-03-28T18:15:55.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:15:55 smithi134 ceph-mon[35037]: reconnect by client.35222 172.21.15.22:0/3729116781 after 0.00499994 2024-03-28T18:15:55.925 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:15:55 smithi134 ceph-mon[35037]: mds.? [v2:172.21.15.22:6836/2141892169,v1:172.21.15.22:6837/2141892169] up:reconnect 2024-03-28T18:15:55.925 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:15:55 smithi134 ceph-mon[35037]: fsmap cephfs:5/5 4 up:standby-replay 1 up:standby 1 up:reconnect 4 up:active 2024-03-28T18:15:55.925 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:15:55 smithi134 ceph-mon[35037]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T18:15:55.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:15:55 smithi079 ceph-mon[35312]: reconnect by client.35222 172.21.15.22:0/3729116781 after 0.00499994 2024-03-28T18:15:55.933 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:15:55 smithi079 ceph-mon[35312]: mds.? [v2:172.21.15.22:6836/2141892169,v1:172.21.15.22:6837/2141892169] up:reconnect 2024-03-28T18:15:55.933 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:15:55 smithi079 ceph-mon[35312]: fsmap cephfs:5/5 4 up:standby-replay 1 up:standby 1 up:reconnect 4 up:active 2024-03-28T18:15:55.933 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:15:55 smithi079 ceph-mon[35312]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T18:15:56.031 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:15:55 smithi022 ceph-mon[31985]: reconnect by client.35222 172.21.15.22:0/3729116781 after 0.00499994 2024-03-28T18:15:56.032 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:15:55 smithi022 ceph-mon[31985]: mds.? [v2:172.21.15.22:6836/2141892169,v1:172.21.15.22:6837/2141892169] up:reconnect 2024-03-28T18:15:56.032 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:15:55 smithi022 ceph-mon[31985]: fsmap cephfs:5/5 4 up:standby-replay 1 up:standby 1 up:reconnect 4 up:active 2024-03-28T18:15:56.032 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:15:55 smithi022 ceph-mon[31985]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T18:15:56.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:15:56 smithi134 ceph-mon[35037]: pgmap v278: 97 pgs: 97 active+clean; 117 MiB data, 795 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 10 op/s 2024-03-28T18:15:56.925 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:15:56 smithi134 ceph-mon[35037]: mds.? [v2:172.21.15.22:6836/2141892169,v1:172.21.15.22:6837/2141892169] up:rejoin 2024-03-28T18:15:56.925 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:15:56 smithi134 ceph-mon[35037]: fsmap cephfs:5/5 4 up:standby-replay 1 up:standby 1 up:rejoin 4 up:active 2024-03-28T18:15:56.925 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:15:56 smithi134 ceph-mon[35037]: daemon mds.d is now active in filesystem cephfs as rank 1 2024-03-28T18:15:56.925 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:15:56 smithi134 ceph-mon[35037]: from='mgr.15522 ' entity='mgr.x' 2024-03-28T18:15:56.925 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:15:56 smithi134 ceph-mon[35037]: from='mgr.15522 ' entity='mgr.x' 2024-03-28T18:15:56.925 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:15:56 smithi134 ceph-mon[35037]: from='mgr.15522 ' entity='mgr.x' 2024-03-28T18:15:56.925 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:15:56 smithi134 ceph-mon[35037]: from='mgr.15522 ' entity='mgr.x' 2024-03-28T18:15:56.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:15:56 smithi079 ceph-mon[35312]: pgmap v278: 97 pgs: 97 active+clean; 117 MiB data, 795 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 10 op/s 2024-03-28T18:15:56.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:15:56 smithi079 ceph-mon[35312]: mds.? [v2:172.21.15.22:6836/2141892169,v1:172.21.15.22:6837/2141892169] up:rejoin 2024-03-28T18:15:56.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:15:56 smithi079 ceph-mon[35312]: fsmap cephfs:5/5 4 up:standby-replay 1 up:standby 1 up:rejoin 4 up:active 2024-03-28T18:15:56.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:15:56 smithi079 ceph-mon[35312]: daemon mds.d is now active in filesystem cephfs as rank 1 2024-03-28T18:15:56.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:15:56 smithi079 ceph-mon[35312]: from='mgr.15522 ' entity='mgr.x' 2024-03-28T18:15:56.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:15:56 smithi079 ceph-mon[35312]: from='mgr.15522 ' entity='mgr.x' 2024-03-28T18:15:56.933 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:15:56 smithi079 ceph-mon[35312]: from='mgr.15522 ' entity='mgr.x' 2024-03-28T18:15:56.933 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:15:56 smithi079 ceph-mon[35312]: from='mgr.15522 ' entity='mgr.x' 2024-03-28T18:15:56.934 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:15:56 smithi022 ceph-mon[31985]: pgmap v278: 97 pgs: 97 active+clean; 117 MiB data, 795 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 10 op/s 2024-03-28T18:15:56.934 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:15:56 smithi022 ceph-mon[31985]: mds.? [v2:172.21.15.22:6836/2141892169,v1:172.21.15.22:6837/2141892169] up:rejoin 2024-03-28T18:15:56.934 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:15:56 smithi022 ceph-mon[31985]: fsmap cephfs:5/5 4 up:standby-replay 1 up:standby 1 up:rejoin 4 up:active 2024-03-28T18:15:56.934 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:15:56 smithi022 ceph-mon[31985]: daemon mds.d is now active in filesystem cephfs as rank 1 2024-03-28T18:15:56.934 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:15:56 smithi022 ceph-mon[31985]: from='mgr.15522 ' entity='mgr.x' 2024-03-28T18:15:56.934 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:15:56 smithi022 ceph-mon[31985]: from='mgr.15522 ' entity='mgr.x' 2024-03-28T18:15:56.934 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:15:56 smithi022 ceph-mon[31985]: from='mgr.15522 ' entity='mgr.x' 2024-03-28T18:15:56.934 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:15:56 smithi022 ceph-mon[31985]: from='mgr.15522 ' entity='mgr.x' 2024-03-28T18:15:56.935 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:15:56 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[94693]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-2145-g3be9a5c9/rpm/el8/BUILD/ceph-19.0.0-2145-g3be9a5c9/src/mds/Locker.cc: In function 'void Locker::set_xlocks_done(MutationImpl*, bool)' thread 7f52da600700 time 2024-03-28T18:15:56.689727+0000 2024-03-28T18:15:56.935 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:15:56 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[94693]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-2145-g3be9a5c9/rpm/el8/BUILD/ceph-19.0.0-2145-g3be9a5c9/src/mds/Locker.cc: 626: FAILED ceph_assert(obj->is_auth()) 2024-03-28T18:15:56.935 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:15:56 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[94693]: ceph version 19.0.0-2145-g3be9a5c9 (3be9a5c9bc793e11e2800a8c0c696e8b46742033) squid (dev) 2024-03-28T18:15:56.935 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:15:56 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[94693]: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x135) [0x7f52e5e614a1] 2024-03-28T18:15:56.935 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:15:56 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[94693]: 2: /usr/lib64/ceph/libceph-common.so.2(+0x2d2667) [0x7f52e5e61667] 2024-03-28T18:15:56.936 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:15:56 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[94693]: 3: (Locker::set_xlocks_done(MutationImpl*, bool)+0x493) [0x5646ed8cf223] 2024-03-28T18:15:56.936 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:15:56 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[94693]: 4: (Server::early_reply(boost::intrusive_ptr const&, CInode*, CDentry*)+0x3e6) [0x5646ed7495d6] 2024-03-28T18:15:56.936 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:15:56 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[94693]: 5: (Server::journal_and_reply(boost::intrusive_ptr const&, CInode*, CDentry*, LogEvent*, MDSLogContextBase*)+0xc1) [0x5646ed749da1] 2024-03-28T18:15:56.936 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:15:56 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[94693]: 6: (Server::handle_client_mkdir(boost::intrusive_ptr const&)+0x5ba) [0x5646ed77f6aa] 2024-03-28T18:15:56.936 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:15:56 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[94693]: 7: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0x733) [0x5646ed79c233] 2024-03-28T18:15:56.936 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:15:56 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[94693]: 8: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x5646ed79e197] 2024-03-28T18:15:56.936 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:15:56 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[94693]: 9: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x5646ed69d11b] 2024-03-28T18:15:56.936 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:15:56 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[94693]: 10: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x5646ed69d6bc] 2024-03-28T18:15:56.936 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:15:56 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[94693]: 11: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bf) [0x5646ed6871ff] 2024-03-28T18:15:56.936 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:15:56 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[94693]: 12: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x478) [0x7f52e60f3488] 2024-03-28T18:15:56.936 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:15:56 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[94693]: 13: (DispatchQueue::entry()+0x51f) [0x7f52e60f060f] 2024-03-28T18:15:56.936 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:15:56 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[94693]: 14: (DispatchQueue::DispatchThread::entry()+0x11) [0x7f52e61c3e81] 2024-03-28T18:15:56.936 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:15:56 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[94693]: 15: /lib64/libpthread.so.0(+0x81ca) [0x7f52e4bde1ca] 2024-03-28T18:15:56.936 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:15:56 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[94693]: 16: clone() 2024-03-28T18:15:56.937 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:15:56 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[94693]: 2024-03-28T18:15:56.690+0000 7f52da600700 -1 /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-2145-g3be9a5c9/rpm/el8/BUILD/ceph-19.0.0-2145-g3be9a5c9/src/mds/Locker.cc: In function 'void Locker::set_xlocks_done(MutationImpl*, bool)' thread 7f52da600700 time 2024-03-28T18:15:56.689727+0000 2024-03-28T18:15:56.937 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:15:56 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[94693]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-2145-g3be9a5c9/rpm/el8/BUILD/ceph-19.0.0-2145-g3be9a5c9/src/mds/Locker.cc: 626: FAILED ceph_assert(obj->is_auth()) 2024-03-28T18:15:56.937 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:15:56 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[94693]: 2024-03-28T18:15:56.937 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:15:56 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[94693]: ceph version 19.0.0-2145-g3be9a5c9 (3be9a5c9bc793e11e2800a8c0c696e8b46742033) squid (dev) 2024-03-28T18:15:56.937 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:15:56 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[94693]: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x135) [0x7f52e5e614a1] 2024-03-28T18:15:56.937 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:15:56 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[94693]: 2: /usr/lib64/ceph/libceph-common.so.2(+0x2d2667) [0x7f52e5e61667] 2024-03-28T18:15:56.937 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:15:56 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[94693]: 3: (Locker::set_xlocks_done(MutationImpl*, bool)+0x493) [0x5646ed8cf223] 2024-03-28T18:15:56.937 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:15:56 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[94693]: 4: (Server::early_reply(boost::intrusive_ptr const&, CInode*, CDentry*)+0x3e6) [0x5646ed7495d6] 2024-03-28T18:15:56.937 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:15:56 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[94693]: 5: (Server::journal_and_reply(boost::intrusive_ptr const&, CInode*, CDentry*, LogEvent*, MDSLogContextBase*)+0xc1) [0x5646ed749da1] 2024-03-28T18:15:56.937 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:15:56 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[94693]: 6: (Server::handle_client_mkdir(boost::intrusive_ptr const&)+0x5ba) [0x5646ed77f6aa] 2024-03-28T18:15:56.937 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:15:56 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[94693]: 7: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0x733) [0x5646ed79c233] 2024-03-28T18:15:56.937 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:15:56 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[94693]: 8: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x5646ed79e197] 2024-03-28T18:15:56.937 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:15:56 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[94693]: 9: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x5646ed69d11b] 2024-03-28T18:15:56.938 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:15:56 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[94693]: 10: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x5646ed69d6bc] 2024-03-28T18:15:56.938 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:15:56 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[94693]: 11: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bf) [0x5646ed6871ff] 2024-03-28T18:15:56.938 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:15:56 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[94693]: 12: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x478) [0x7f52e60f3488] 2024-03-28T18:15:56.938 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:15:56 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[94693]: 13: (DispatchQueue::entry()+0x51f) [0x7f52e60f060f] 2024-03-28T18:15:56.938 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:15:56 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[94693]: 14: (DispatchQueue::DispatchThread::entry()+0x11) [0x7f52e61c3e81] 2024-03-28T18:15:56.938 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:15:56 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[94693]: 15: /lib64/libpthread.so.0(+0x81ca) [0x7f52e4bde1ca] 2024-03-28T18:15:56.938 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:15:56 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[94693]: 16: clone() 2024-03-28T18:15:56.938 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:15:56 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[94693]: 2024-03-28T18:15:56.938 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:15:56 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[94693]: *** Caught signal (Aborted) ** 2024-03-28T18:15:56.938 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:15:56 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[94693]: in thread 7f52da600700 thread_name:ms_dispatch 2024-03-28T18:15:56.938 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:15:56 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[94693]: ceph version 19.0.0-2145-g3be9a5c9 (3be9a5c9bc793e11e2800a8c0c696e8b46742033) squid (dev) 2024-03-28T18:15:56.938 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:15:56 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[94693]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7f52e4be8d20] 2024-03-28T18:15:56.939 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:15:56 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[94693]: 2: gsignal() 2024-03-28T18:15:56.939 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:15:56 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[94693]: 3: abort() 2024-03-28T18:15:56.939 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:15:56 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[94693]: 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x18f) [0x7f52e5e614fb] 2024-03-28T18:15:56.939 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:15:56 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[94693]: 5: /usr/lib64/ceph/libceph-common.so.2(+0x2d2667) [0x7f52e5e61667] 2024-03-28T18:15:56.939 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:15:56 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[94693]: 6: (Locker::set_xlocks_done(MutationImpl*, bool)+0x493) [0x5646ed8cf223] 2024-03-28T18:15:56.939 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:15:56 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[94693]: 7: (Server::early_reply(boost::intrusive_ptr const&, CInode*, CDentry*)+0x3e6) [0x5646ed7495d6] 2024-03-28T18:15:56.939 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:15:56 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[94693]: 8: (Server::journal_and_reply(boost::intrusive_ptr const&, CInode*, CDentry*, LogEvent*, MDSLogContextBase*)+0xc1) [0x5646ed749da1] 2024-03-28T18:15:56.939 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:15:56 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[94693]: 9: (Server::handle_client_mkdir(boost::intrusive_ptr const&)+0x5ba) [0x5646ed77f6aa] 2024-03-28T18:15:56.939 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:15:56 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[94693]: 10: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0x733) [0x5646ed79c233] 2024-03-28T18:15:56.939 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:15:56 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[94693]: 11: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x5646ed79e197] 2024-03-28T18:15:56.939 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:15:56 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[94693]: 12: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x5646ed69d11b] 2024-03-28T18:15:56.940 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:15:56 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[94693]: 13: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x5646ed69d6bc] 2024-03-28T18:15:56.940 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:15:56 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[94693]: 14: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bf) [0x5646ed6871ff] 2024-03-28T18:15:56.940 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:15:56 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[94693]: 15: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x478) [0x7f52e60f3488] 2024-03-28T18:15:56.940 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:15:56 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[94693]: 16: (DispatchQueue::entry()+0x51f) [0x7f52e60f060f] 2024-03-28T18:15:56.940 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:15:56 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[94693]: 17: (DispatchQueue::DispatchThread::entry()+0x11) [0x7f52e61c3e81] 2024-03-28T18:15:56.940 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:15:56 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[94693]: 18: /lib64/libpthread.so.0(+0x81ca) [0x7f52e4bde1ca] 2024-03-28T18:15:56.940 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:15:56 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[94693]: 19: clone() 2024-03-28T18:15:56.940 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:15:56 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[94693]: 2024-03-28T18:15:56.693+0000 7f52da600700 -1 *** Caught signal (Aborted) ** 2024-03-28T18:15:56.940 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:15:56 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[94693]: in thread 7f52da600700 thread_name:ms_dispatch 2024-03-28T18:15:56.940 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:15:56 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[94693]: 2024-03-28T18:15:56.940 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:15:56 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[94693]: ceph version 19.0.0-2145-g3be9a5c9 (3be9a5c9bc793e11e2800a8c0c696e8b46742033) squid (dev) 2024-03-28T18:15:56.940 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:15:56 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[94693]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7f52e4be8d20] 2024-03-28T18:15:56.940 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:15:56 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[94693]: 2: gsignal() 2024-03-28T18:15:56.940 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:15:56 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[94693]: 3: abort() 2024-03-28T18:15:56.941 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:15:56 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[94693]: 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x18f) [0x7f52e5e614fb] 2024-03-28T18:15:56.941 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:15:56 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[94693]: 5: /usr/lib64/ceph/libceph-common.so.2(+0x2d2667) [0x7f52e5e61667] 2024-03-28T18:15:56.941 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:15:56 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[94693]: 6: (Locker::set_xlocks_done(MutationImpl*, bool)+0x493) [0x5646ed8cf223] 2024-03-28T18:15:56.941 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:15:56 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[94693]: 7: (Server::early_reply(boost::intrusive_ptr const&, CInode*, CDentry*)+0x3e6) [0x5646ed7495d6] 2024-03-28T18:15:56.941 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:15:56 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[94693]: 8: (Server::journal_and_reply(boost::intrusive_ptr const&, CInode*, CDentry*, LogEvent*, MDSLogContextBase*)+0xc1) [0x5646ed749da1] 2024-03-28T18:15:56.941 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:15:56 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[94693]: 9: (Server::handle_client_mkdir(boost::intrusive_ptr const&)+0x5ba) [0x5646ed77f6aa] 2024-03-28T18:15:56.941 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:15:56 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[94693]: 10: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0x733) [0x5646ed79c233] 2024-03-28T18:15:56.941 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:15:56 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[94693]: 11: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x5646ed79e197] 2024-03-28T18:15:56.941 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:15:56 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[94693]: 12: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x5646ed69d11b] 2024-03-28T18:15:56.942 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:15:56 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[94693]: 13: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x5646ed69d6bc] 2024-03-28T18:15:56.942 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:15:56 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[94693]: 14: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bf) [0x5646ed6871ff] 2024-03-28T18:15:56.942 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:15:56 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[94693]: 15: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x478) [0x7f52e60f3488] 2024-03-28T18:15:56.942 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:15:56 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[94693]: 16: (DispatchQueue::entry()+0x51f) [0x7f52e60f060f] 2024-03-28T18:15:56.942 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:15:56 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[94693]: 17: (DispatchQueue::DispatchThread::entry()+0x11) [0x7f52e61c3e81] 2024-03-28T18:15:56.942 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:15:56 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[94693]: 18: /lib64/libpthread.so.0(+0x81ca) [0x7f52e4bde1ca] 2024-03-28T18:15:56.942 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:15:56 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[94693]: 19: clone() 2024-03-28T18:15:56.942 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:15:56 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[94693]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-03-28T18:15:56.942 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:15:56 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[94693]: 2024-03-28T18:15:56.942 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:15:56 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[94693]: -1> 2024-03-28T18:15:56.690+0000 7f52da600700 -1 /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-2145-g3be9a5c9/rpm/el8/BUILD/ceph-19.0.0-2145-g3be9a5c9/src/mds/Locker.cc: In function 'void Locker::set_xlocks_done(MutationImpl*, bool)' thread 7f52da600700 time 2024-03-28T18:15:56.689727+0000 2024-03-28T18:15:56.942 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:15:56 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[94693]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-2145-g3be9a5c9/rpm/el8/BUILD/ceph-19.0.0-2145-g3be9a5c9/src/mds/Locker.cc: 626: FAILED ceph_assert(obj->is_auth()) 2024-03-28T18:15:56.942 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:15:56 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[94693]: 2024-03-28T18:15:56.942 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:15:56 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[94693]: ceph version 19.0.0-2145-g3be9a5c9 (3be9a5c9bc793e11e2800a8c0c696e8b46742033) squid (dev) 2024-03-28T18:15:56.942 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:15:56 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[94693]: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x135) [0x7f52e5e614a1] 2024-03-28T18:15:56.942 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:15:56 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[94693]: 2: /usr/lib64/ceph/libceph-common.so.2(+0x2d2667) [0x7f52e5e61667] 2024-03-28T18:15:56.943 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:15:56 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[94693]: 3: (Locker::set_xlocks_done(MutationImpl*, bool)+0x493) [0x5646ed8cf223] 2024-03-28T18:15:56.943 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:15:56 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[94693]: 4: (Server::early_reply(boost::intrusive_ptr const&, CInode*, CDentry*)+0x3e6) [0x5646ed7495d6] 2024-03-28T18:15:56.943 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:15:56 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[94693]: 5: (Server::journal_and_reply(boost::intrusive_ptr const&, CInode*, CDentry*, LogEvent*, MDSLogContextBase*)+0xc1) [0x5646ed749da1] 2024-03-28T18:15:56.943 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:15:56 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[94693]: 6: (Server::handle_client_mkdir(boost::intrusive_ptr const&)+0x5ba) [0x5646ed77f6aa] 2024-03-28T18:15:56.943 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:15:56 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[94693]: 7: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0x733) [0x5646ed79c233] 2024-03-28T18:15:56.943 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:15:56 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[94693]: 8: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x5646ed79e197] 2024-03-28T18:15:56.943 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:15:56 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[94693]: 9: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x5646ed69d11b] 2024-03-28T18:15:56.944 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:15:56 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[94693]: 10: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x5646ed69d6bc] 2024-03-28T18:15:56.944 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:15:56 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[94693]: 11: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bf) [0x5646ed6871ff] 2024-03-28T18:15:56.944 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:15:56 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[94693]: 12: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x478) [0x7f52e60f3488] 2024-03-28T18:15:56.944 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:15:56 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[94693]: 13: (DispatchQueue::entry()+0x51f) [0x7f52e60f060f] 2024-03-28T18:15:56.944 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:15:56 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[94693]: 14: (DispatchQueue::DispatchThread::entry()+0x11) [0x7f52e61c3e81] 2024-03-28T18:15:56.944 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:15:56 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[94693]: 15: /lib64/libpthread.so.0(+0x81ca) [0x7f52e4bde1ca] 2024-03-28T18:15:56.944 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:15:56 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[94693]: 16: clone() 2024-03-28T18:15:56.944 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:15:56 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[94693]: 2024-03-28T18:15:56.944 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:15:56 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[94693]: 0> 2024-03-28T18:15:56.693+0000 7f52da600700 -1 *** Caught signal (Aborted) ** 2024-03-28T18:15:56.944 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:15:56 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[94693]: in thread 7f52da600700 thread_name:ms_dispatch 2024-03-28T18:15:56.944 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:15:56 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[94693]: 2024-03-28T18:15:56.944 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:15:56 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[94693]: ceph version 19.0.0-2145-g3be9a5c9 (3be9a5c9bc793e11e2800a8c0c696e8b46742033) squid (dev) 2024-03-28T18:15:56.944 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:15:56 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[94693]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7f52e4be8d20] 2024-03-28T18:15:56.944 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:15:56 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[94693]: 2: gsignal() 2024-03-28T18:15:56.944 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:15:56 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[94693]: 3: abort() 2024-03-28T18:15:56.945 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:15:56 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[94693]: 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x18f) [0x7f52e5e614fb] 2024-03-28T18:15:56.945 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:15:56 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[94693]: 5: /usr/lib64/ceph/libceph-common.so.2(+0x2d2667) [0x7f52e5e61667] 2024-03-28T18:15:56.945 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:15:56 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[94693]: 6: (Locker::set_xlocks_done(MutationImpl*, bool)+0x493) [0x5646ed8cf223] 2024-03-28T18:15:56.945 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:15:56 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[94693]: 7: (Server::early_reply(boost::intrusive_ptr const&, CInode*, CDentry*)+0x3e6) [0x5646ed7495d6] 2024-03-28T18:15:56.945 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:15:56 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[94693]: 8: (Server::journal_and_reply(boost::intrusive_ptr const&, CInode*, CDentry*, LogEvent*, MDSLogContextBase*)+0xc1) [0x5646ed749da1] 2024-03-28T18:15:56.945 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:15:56 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[94693]: 9: (Server::handle_client_mkdir(boost::intrusive_ptr const&)+0x5ba) [0x5646ed77f6aa] 2024-03-28T18:15:56.945 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:15:56 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[94693]: 10: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0x733) [0x5646ed79c233] 2024-03-28T18:15:56.945 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:15:56 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[94693]: 11: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x5646ed79e197] 2024-03-28T18:15:56.945 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:15:56 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[94693]: 12: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x5646ed69d11b] 2024-03-28T18:15:56.945 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:15:56 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[94693]: 13: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x5646ed69d6bc] 2024-03-28T18:15:56.945 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:15:56 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[94693]: 14: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bf) [0x5646ed6871ff] 2024-03-28T18:15:56.945 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:15:56 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[94693]: 15: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x478) [0x7f52e60f3488] 2024-03-28T18:15:56.945 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:15:56 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[94693]: 16: (DispatchQueue::entry()+0x51f) [0x7f52e60f060f] 2024-03-28T18:15:56.946 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:15:56 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[94693]: 17: (DispatchQueue::DispatchThread::entry()+0x11) [0x7f52e61c3e81] 2024-03-28T18:15:56.946 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:15:56 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[94693]: 18: /lib64/libpthread.so.0(+0x81ca) [0x7f52e4bde1ca] 2024-03-28T18:15:56.946 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:15:56 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[94693]: 19: clone() 2024-03-28T18:15:56.946 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:15:56 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[94693]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-03-28T18:15:56.946 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:15:56 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[94693]: 2024-03-28T18:15:56.946 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:15:56 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[94693]: -1> 2024-03-28T18:15:56.690+0000 7f52da600700 -1 /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-2145-g3be9a5c9/rpm/el8/BUILD/ceph-19.0.0-2145-g3be9a5c9/src/mds/Locker.cc: In function 'void Locker::set_xlocks_done(MutationImpl*, bool)' thread 7f52da600700 time 2024-03-28T18:15:56.689727+0000 2024-03-28T18:15:56.946 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:15:56 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[94693]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-2145-g3be9a5c9/rpm/el8/BUILD/ceph-19.0.0-2145-g3be9a5c9/src/mds/Locker.cc: 626: FAILED ceph_assert(obj->is_auth()) 2024-03-28T18:15:56.946 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:15:56 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[94693]: 2024-03-28T18:15:56.946 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:15:56 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[94693]: ceph version 19.0.0-2145-g3be9a5c9 (3be9a5c9bc793e11e2800a8c0c696e8b46742033) squid (dev) 2024-03-28T18:15:56.946 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:15:56 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[94693]: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x135) [0x7f52e5e614a1] 2024-03-28T18:15:56.946 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:15:56 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[94693]: 2: /usr/lib64/ceph/libceph-common.so.2(+0x2d2667) [0x7f52e5e61667] 2024-03-28T18:15:56.946 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:15:56 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[94693]: 3: (Locker::set_xlocks_done(MutationImpl*, bool)+0x493) [0x5646ed8cf223] 2024-03-28T18:15:56.946 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:15:56 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[94693]: 4: (Server::early_reply(boost::intrusive_ptr const&, CInode*, CDentry*)+0x3e6) [0x5646ed7495d6] 2024-03-28T18:15:56.946 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:15:56 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[94693]: 5: (Server::journal_and_reply(boost::intrusive_ptr const&, CInode*, CDentry*, LogEvent*, MDSLogContextBase*)+0xc1) [0x5646ed749da1] 2024-03-28T18:15:56.947 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:15:56 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[94693]: 6: (Server::handle_client_mkdir(boost::intrusive_ptr const&)+0x5ba) [0x5646ed77f6aa] 2024-03-28T18:15:56.947 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:15:56 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[94693]: 7: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0x733) [0x5646ed79c233] 2024-03-28T18:15:56.947 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:15:56 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[94693]: 8: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x5646ed79e197] 2024-03-28T18:15:56.947 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:15:56 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[94693]: 9: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x5646ed69d11b] 2024-03-28T18:15:56.947 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:15:56 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[94693]: 10: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x5646ed69d6bc] 2024-03-28T18:15:56.947 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:15:56 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[94693]: 11: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bf) [0x5646ed6871ff] 2024-03-28T18:15:56.947 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:15:56 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[94693]: 12: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x478) [0x7f52e60f3488] 2024-03-28T18:15:56.947 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:15:56 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[94693]: 13: (DispatchQueue::entry()+0x51f) [0x7f52e60f060f] 2024-03-28T18:15:56.948 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:15:56 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[94693]: 14: (DispatchQueue::DispatchThread::entry()+0x11) [0x7f52e61c3e81] 2024-03-28T18:15:56.948 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:15:56 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[94693]: 15: /lib64/libpthread.so.0(+0x81ca) [0x7f52e4bde1ca] 2024-03-28T18:15:56.948 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:15:56 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[94693]: 16: clone() 2024-03-28T18:15:56.948 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:15:56 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[94693]: 2024-03-28T18:15:56.948 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:15:56 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[94693]: 0> 2024-03-28T18:15:56.693+0000 7f52da600700 -1 *** Caught signal (Aborted) ** 2024-03-28T18:15:56.948 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:15:56 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[94693]: in thread 7f52da600700 thread_name:ms_dispatch 2024-03-28T18:15:56.948 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:15:56 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[94693]: 2024-03-28T18:15:56.948 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:15:56 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[94693]: ceph version 19.0.0-2145-g3be9a5c9 (3be9a5c9bc793e11e2800a8c0c696e8b46742033) squid (dev) 2024-03-28T18:15:56.948 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:15:56 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[94693]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7f52e4be8d20] 2024-03-28T18:15:56.948 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:15:56 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[94693]: 2: gsignal() 2024-03-28T18:15:56.948 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:15:56 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[94693]: 3: abort() 2024-03-28T18:15:56.948 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:15:56 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[94693]: 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x18f) [0x7f52e5e614fb] 2024-03-28T18:15:56.948 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:15:56 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[94693]: 5: /usr/lib64/ceph/libceph-common.so.2(+0x2d2667) [0x7f52e5e61667] 2024-03-28T18:15:56.948 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:15:56 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[94693]: 6: (Locker::set_xlocks_done(MutationImpl*, bool)+0x493) [0x5646ed8cf223] 2024-03-28T18:15:56.948 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:15:56 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[94693]: 7: (Server::early_reply(boost::intrusive_ptr const&, CInode*, CDentry*)+0x3e6) [0x5646ed7495d6] 2024-03-28T18:15:56.949 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:15:56 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[94693]: 8: (Server::journal_and_reply(boost::intrusive_ptr const&, CInode*, CDentry*, LogEvent*, MDSLogContextBase*)+0xc1) [0x5646ed749da1] 2024-03-28T18:15:56.949 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:15:56 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[94693]: 9: (Server::handle_client_mkdir(boost::intrusive_ptr const&)+0x5ba) [0x5646ed77f6aa] 2024-03-28T18:15:56.949 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:15:56 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[94693]: 10: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0x733) [0x5646ed79c233] 2024-03-28T18:15:56.949 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:15:56 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[94693]: 11: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x5646ed79e197] 2024-03-28T18:15:56.949 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:15:56 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[94693]: 12: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x5646ed69d11b] 2024-03-28T18:15:56.949 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:15:56 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[94693]: 13: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x5646ed69d6bc] 2024-03-28T18:15:56.949 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:15:56 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[94693]: 14: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bf) [0x5646ed6871ff] 2024-03-28T18:15:56.949 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:15:56 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[94693]: 15: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x478) [0x7f52e60f3488] 2024-03-28T18:15:56.949 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:15:56 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[94693]: 16: (DispatchQueue::entry()+0x51f) [0x7f52e60f060f] 2024-03-28T18:15:56.949 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:15:56 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[94693]: 17: (DispatchQueue::DispatchThread::entry()+0x11) [0x7f52e61c3e81] 2024-03-28T18:15:56.949 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:15:56 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[94693]: 18: /lib64/libpthread.so.0(+0x81ca) [0x7f52e4bde1ca] 2024-03-28T18:15:56.949 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:15:56 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[94693]: 19: clone() 2024-03-28T18:15:56.949 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:15:56 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[94693]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-03-28T18:15:56.949 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:15:56 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[94693]: 2024-03-28T18:15:56.949 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:15:56 smithi022 podman[95076]: 2024-03-28 18:15:56.803418514 +0000 UTC m=+0.033153107 container died 0729d79e3074da6b17522efd06268f831c76c8bcce697e0e349186eab75d4cb8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, io.buildah.version=1.33.5, org.label-schema.build-date=20240319, org.label-schema.schema-version=1.0) 2024-03-28T18:15:57.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:15:57 smithi134 ceph-mon[35037]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-03-28T18:15:57.925 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:15:57 smithi134 ceph-mon[35037]: Cluster is now healthy 2024-03-28T18:15:57.925 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:15:57 smithi134 ceph-mon[35037]: mds.? [v2:172.21.15.22:6836/2141892169,v1:172.21.15.22:6837/2141892169] up:active 2024-03-28T18:15:57.925 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:15:57 smithi134 ceph-mon[35037]: fsmap cephfs:5 4 up:standby-replay 1 up:standby 5 up:active 2024-03-28T18:15:57.925 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:15:57 smithi134 ceph-mon[35037]: fsmap cephfs:5 5 up:standby-replay 5 up:active 2024-03-28T18:15:57.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:15:57 smithi079 ceph-mon[35312]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-03-28T18:15:57.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:15:57 smithi079 ceph-mon[35312]: Cluster is now healthy 2024-03-28T18:15:57.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:15:57 smithi079 ceph-mon[35312]: mds.? [v2:172.21.15.22:6836/2141892169,v1:172.21.15.22:6837/2141892169] up:active 2024-03-28T18:15:57.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:15:57 smithi079 ceph-mon[35312]: fsmap cephfs:5 4 up:standby-replay 1 up:standby 5 up:active 2024-03-28T18:15:57.933 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:15:57 smithi079 ceph-mon[35312]: fsmap cephfs:5 5 up:standby-replay 5 up:active 2024-03-28T18:15:58.031 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:15:57 smithi022 podman[95076]: 2024-03-28 18:15:57.812568255 +0000 UTC m=+1.042302892 container remove 0729d79e3074da6b17522efd06268f831c76c8bcce697e0e349186eab75d4cb8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, maintainer=Guillaume Abrioux , GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.build-date=20240319, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-28T18:15:58.031 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:15:57 smithi022 systemd[1]: ceph-fd220748-ed2b-11ee-95d0-87774f69a715@mds.d.service: Main process exited, code=exited, status=134/n/a 2024-03-28T18:15:58.032 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:15:57 smithi022 ceph-mon[31985]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-03-28T18:15:58.032 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:15:57 smithi022 ceph-mon[31985]: Cluster is now healthy 2024-03-28T18:15:58.032 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:15:57 smithi022 ceph-mon[31985]: mds.? [v2:172.21.15.22:6836/2141892169,v1:172.21.15.22:6837/2141892169] up:active 2024-03-28T18:15:58.032 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:15:57 smithi022 ceph-mon[31985]: fsmap cephfs:5 4 up:standby-replay 1 up:standby 5 up:active 2024-03-28T18:15:58.032 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:15:57 smithi022 ceph-mon[31985]: fsmap cephfs:5 5 up:standby-replay 5 up:active 2024-03-28T18:15:58.391 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:15:58 smithi022 systemd[1]: ceph-fd220748-ed2b-11ee-95d0-87774f69a715@mds.d.service: Failed with result 'exit-code'. 2024-03-28T18:15:58.391 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:15:58 smithi022 systemd[1]: ceph-fd220748-ed2b-11ee-95d0-87774f69a715@mds.d.service: Consumed 1.866s CPU time. 2024-03-28T18:15:58.925 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:15:58 smithi134 ceph-mon[35037]: pgmap v279: 97 pgs: 97 active+clean; 117 MiB data, 795 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 10 op/s 2024-03-28T18:15:58.926 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:15:58 smithi134 ceph-mon[35037]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T18:15:58.926 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:15:58 smithi134 ceph-mon[35037]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T18:15:58.926 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:15:58 smithi134 ceph-mon[35037]: from='mgr.15522 ' entity='mgr.x' 2024-03-28T18:15:58.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:15:58 smithi079 ceph-mon[35312]: pgmap v279: 97 pgs: 97 active+clean; 117 MiB data, 795 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 10 op/s 2024-03-28T18:15:58.933 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:15:58 smithi079 ceph-mon[35312]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T18:15:58.933 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:15:58 smithi079 ceph-mon[35312]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T18:15:58.933 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:15:58 smithi079 ceph-mon[35312]: from='mgr.15522 ' entity='mgr.x' 2024-03-28T18:15:59.032 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:15:58 smithi022 ceph-mon[31985]: pgmap v279: 97 pgs: 97 active+clean; 117 MiB data, 795 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 10 op/s 2024-03-28T18:15:59.032 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:15:58 smithi022 ceph-mon[31985]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T18:15:59.032 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:15:58 smithi022 ceph-mon[31985]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T18:15:59.032 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:15:58 smithi022 ceph-mon[31985]: from='mgr.15522 ' entity='mgr.x' 2024-03-28T18:15:59.432 INFO:journalctl@ceph.mds.b.smithi079.stdout:Mar 28 18:15:59 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-b[65245]: 2024-03-28T18:15:59.102+0000 7f845f400700 -1 mds.pinger is_rank_lagging: rank=1 was never sent ping request. 2024-03-28T18:16:00.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:16:00 smithi134 ceph-mon[35037]: pgmap v280: 97 pgs: 97 active+clean; 117 MiB data, 795 MiB used, 1.0 TiB / 1.0 TiB avail; 110 KiB/s rd, 5.5 KiB/s wr, 26 op/s 2024-03-28T18:16:00.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:16:00 smithi079 ceph-mon[35312]: pgmap v280: 97 pgs: 97 active+clean; 117 MiB data, 795 MiB used, 1.0 TiB / 1.0 TiB avail; 110 KiB/s rd, 5.5 KiB/s wr, 26 op/s 2024-03-28T18:16:01.032 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:16:00 smithi022 ceph-mon[31985]: pgmap v280: 97 pgs: 97 active+clean; 117 MiB data, 795 MiB used, 1.0 TiB / 1.0 TiB avail; 110 KiB/s rd, 5.5 KiB/s wr, 26 op/s 2024-03-28T18:16:02.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:16:02 smithi134 ceph-mon[35037]: pgmap v281: 97 pgs: 97 active+clean; 117 MiB data, 795 MiB used, 1.0 TiB / 1.0 TiB avail; 110 KiB/s rd, 5.5 KiB/s wr, 26 op/s 2024-03-28T18:16:02.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:16:02 smithi079 ceph-mon[35312]: pgmap v281: 97 pgs: 97 active+clean; 117 MiB data, 795 MiB used, 1.0 TiB / 1.0 TiB avail; 110 KiB/s rd, 5.5 KiB/s wr, 26 op/s 2024-03-28T18:16:03.032 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:16:02 smithi022 ceph-mon[31985]: pgmap v281: 97 pgs: 97 active+clean; 117 MiB data, 795 MiB used, 1.0 TiB / 1.0 TiB avail; 110 KiB/s rd, 5.5 KiB/s wr, 26 op/s 2024-03-28T18:16:05.282 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:16:04 smithi022 ceph-mon[31985]: pgmap v282: 97 pgs: 97 active+clean; 117 MiB data, 795 MiB used, 1.0 TiB / 1.0 TiB avail; 104 KiB/s rd, 5.2 KiB/s wr, 24 op/s 2024-03-28T18:16:05.424 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:16:04 smithi134 ceph-mon[35037]: pgmap v282: 97 pgs: 97 active+clean; 117 MiB data, 795 MiB used, 1.0 TiB / 1.0 TiB avail; 104 KiB/s rd, 5.2 KiB/s wr, 24 op/s 2024-03-28T18:16:05.432 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:16:04 smithi079 ceph-mon[35312]: pgmap v282: 97 pgs: 97 active+clean; 117 MiB data, 795 MiB used, 1.0 TiB / 1.0 TiB avail; 104 KiB/s rd, 5.2 KiB/s wr, 24 op/s 2024-03-28T18:16:06.424 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:16:06 smithi134 ceph-mon[35037]: pgmap v283: 97 pgs: 97 active+clean; 117 MiB data, 795 MiB used, 1.0 TiB / 1.0 TiB avail; 95 KiB/s rd, 4.6 KiB/s wr, 27 op/s 2024-03-28T18:16:06.432 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:16:06 smithi079 ceph-mon[35312]: pgmap v283: 97 pgs: 97 active+clean; 117 MiB data, 795 MiB used, 1.0 TiB / 1.0 TiB avail; 95 KiB/s rd, 4.6 KiB/s wr, 27 op/s 2024-03-28T18:16:06.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:16:06 smithi022 ceph-mon[31985]: pgmap v283: 97 pgs: 97 active+clean; 117 MiB data, 795 MiB used, 1.0 TiB / 1.0 TiB avail; 95 KiB/s rd, 4.6 KiB/s wr, 27 op/s 2024-03-28T18:16:08.531 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:16:08 smithi022 ceph-mon[31985]: pgmap v284: 97 pgs: 97 active+clean; 117 MiB data, 795 MiB used, 1.0 TiB / 1.0 TiB avail; 92 KiB/s rd, 4.6 KiB/s wr, 22 op/s 2024-03-28T18:16:08.532 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:16:08 smithi022 systemd[1]: ceph-fd220748-ed2b-11ee-95d0-87774f69a715@mds.d.service: Scheduled restart job, restart counter is at 5. 2024-03-28T18:16:08.532 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:16:08 smithi022 systemd[1]: Stopped Ceph mds.d for fd220748-ed2b-11ee-95d0-87774f69a715. 2024-03-28T18:16:08.532 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:16:08 smithi022 systemd[1]: ceph-fd220748-ed2b-11ee-95d0-87774f69a715@mds.d.service: Consumed 1.866s CPU time. 2024-03-28T18:16:08.532 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:16:08 smithi022 systemd[1]: Starting Ceph mds.d for fd220748-ed2b-11ee-95d0-87774f69a715... 2024-03-28T18:16:08.674 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:16:08 smithi134 ceph-mon[35037]: pgmap v284: 97 pgs: 97 active+clean; 117 MiB data, 795 MiB used, 1.0 TiB / 1.0 TiB avail; 92 KiB/s rd, 4.6 KiB/s wr, 22 op/s 2024-03-28T18:16:08.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:16:08 smithi079 ceph-mon[35312]: pgmap v284: 97 pgs: 97 active+clean; 117 MiB data, 795 MiB used, 1.0 TiB / 1.0 TiB avail; 92 KiB/s rd, 4.6 KiB/s wr, 22 op/s 2024-03-28T18:16:09.031 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:16:08 smithi022 podman[95266]: 2024-03-28 18:16:08.593321358 +0000 UTC m=+0.035249273 image pull c664afe56bb01db4f9afa9532c20598e201e664f2e9ba2be50db0d52674a26d2 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc 2024-03-28T18:16:09.032 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:16:08 smithi022 podman[95266]: 2024-03-28 18:16:08.856093779 +0000 UTC m=+0.298021715 container create d01401bd73e8f8acca7b0a20d5f885c739882625be4e6a98c6e83b4d80d970fd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240319, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2) 2024-03-28T18:16:09.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:16:09 smithi022 ceph-mon[31985]: daemon mds.d restarted 2024-03-28T18:16:09.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:16:09 smithi022 ceph-mon[31985]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-03-28T18:16:09.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:16:09 smithi022 ceph-mon[31985]: osdmap e118: 12 total, 12 up, 12 in 2024-03-28T18:16:09.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:16:09 smithi022 ceph-mon[31985]: mds.? [v2:172.21.15.22:6836/3255804289,v1:172.21.15.22:6837/3255804289] up:boot 2024-03-28T18:16:09.533 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:16:09 smithi022 ceph-mon[31985]: Standby daemon mds.j assigned to filesystem cephfs as rank 1 2024-03-28T18:16:09.533 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:16:09 smithi022 ceph-mon[31985]: fsmap cephfs:4/5 {0=b=up:active,2=e=up:active,3=k=up:active,4=i=up:active} 5 up:standby-replay 1 up:standby, 1 failed 2024-03-28T18:16:09.533 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:16:09 smithi022 ceph-mon[31985]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-03-28T18:16:09.533 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:16:09 smithi022 ceph-mon[31985]: fsmap cephfs:5/5 4 up:standby-replay 1 up:standby 1 up:replay 4 up:active 2024-03-28T18:16:09.533 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:16:09 smithi022 podman[95266]: 2024-03-28 18:16:09.138137976 +0000 UTC m=+0.580065948 container init d01401bd73e8f8acca7b0a20d5f885c739882625be4e6a98c6e83b4d80d970fd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240319, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, GIT_CLEAN=True, org.label-schema.schema-version=1.0, ceph=True, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD) 2024-03-28T18:16:09.533 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:16:09 smithi022 podman[95266]: 2024-03-28 18:16:09.150415545 +0000 UTC m=+0.592343530 container start d01401bd73e8f8acca7b0a20d5f885c739882625be4e6a98c6e83b4d80d970fd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240319, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-28T18:16:09.533 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:16:09 smithi022 ceph-mds[95285]: -- 172.21.15.22:0/4004781491 <== mon.0 v2:172.21.15.22:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55d8d29ffa40 con 0x55d8d29d3000 2024-03-28T18:16:09.533 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:16:09 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[95280]: starting mds.d at 2024-03-28T18:16:09.533 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:16:09 smithi022 bash[95266]: d01401bd73e8f8acca7b0a20d5f885c739882625be4e6a98c6e83b4d80d970fd 2024-03-28T18:16:09.533 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:16:09 smithi022 systemd[1]: Started Ceph mds.d for fd220748-ed2b-11ee-95d0-87774f69a715. 2024-03-28T18:16:09.674 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:16:09 smithi134 ceph-mon[35037]: daemon mds.d restarted 2024-03-28T18:16:09.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:16:09 smithi134 ceph-mon[35037]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-03-28T18:16:09.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:16:09 smithi134 ceph-mon[35037]: osdmap e118: 12 total, 12 up, 12 in 2024-03-28T18:16:09.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:16:09 smithi134 ceph-mon[35037]: mds.? [v2:172.21.15.22:6836/3255804289,v1:172.21.15.22:6837/3255804289] up:boot 2024-03-28T18:16:09.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:16:09 smithi134 ceph-mon[35037]: Standby daemon mds.j assigned to filesystem cephfs as rank 1 2024-03-28T18:16:09.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:16:09 smithi134 ceph-mon[35037]: fsmap cephfs:4/5 {0=b=up:active,2=e=up:active,3=k=up:active,4=i=up:active} 5 up:standby-replay 1 up:standby, 1 failed 2024-03-28T18:16:09.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:16:09 smithi134 ceph-mon[35037]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-03-28T18:16:09.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:16:09 smithi134 ceph-mon[35037]: fsmap cephfs:5/5 4 up:standby-replay 1 up:standby 1 up:replay 4 up:active 2024-03-28T18:16:09.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:16:09 smithi079 ceph-mon[35312]: daemon mds.d restarted 2024-03-28T18:16:09.683 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:16:09 smithi079 ceph-mon[35312]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-03-28T18:16:09.683 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:16:09 smithi079 ceph-mon[35312]: osdmap e118: 12 total, 12 up, 12 in 2024-03-28T18:16:09.683 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:16:09 smithi079 ceph-mon[35312]: mds.? [v2:172.21.15.22:6836/3255804289,v1:172.21.15.22:6837/3255804289] up:boot 2024-03-28T18:16:09.683 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:16:09 smithi079 ceph-mon[35312]: Standby daemon mds.j assigned to filesystem cephfs as rank 1 2024-03-28T18:16:09.683 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:16:09 smithi079 ceph-mon[35312]: fsmap cephfs:4/5 {0=b=up:active,2=e=up:active,3=k=up:active,4=i=up:active} 5 up:standby-replay 1 up:standby, 1 failed 2024-03-28T18:16:09.683 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:16:09 smithi079 ceph-mon[35312]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-03-28T18:16:09.683 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:16:09 smithi079 ceph-mon[35312]: fsmap cephfs:5/5 4 up:standby-replay 1 up:standby 1 up:replay 4 up:active 2024-03-28T18:16:10.584 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:16:10 smithi079 ceph-mon[35312]: pgmap v286: 97 pgs: 97 active+clean; 117 MiB data, 795 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 0 B/s wr, 11 op/s 2024-03-28T18:16:10.674 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:16:10 smithi134 ceph-mon[35037]: pgmap v286: 97 pgs: 97 active+clean; 117 MiB data, 795 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 0 B/s wr, 11 op/s 2024-03-28T18:16:10.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:16:10 smithi022 ceph-mon[31985]: pgmap v286: 97 pgs: 97 active+clean; 117 MiB data, 795 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 0 B/s wr, 11 op/s 2024-03-28T18:16:11.674 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:16:11 smithi134 ceph-mon[35037]: mds.? [v2:172.21.15.22:6834/817442472,v1:172.21.15.22:6835/817442472] up:resolve 2024-03-28T18:16:11.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:16:11 smithi134 ceph-mon[35037]: fsmap cephfs:5/5 4 up:standby-replay 1 up:standby 1 up:resolve 4 up:active 2024-03-28T18:16:11.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:16:11 smithi079 ceph-mon[35312]: mds.? [v2:172.21.15.22:6834/817442472,v1:172.21.15.22:6835/817442472] up:resolve 2024-03-28T18:16:11.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:16:11 smithi079 ceph-mon[35312]: fsmap cephfs:5/5 4 up:standby-replay 1 up:standby 1 up:resolve 4 up:active 2024-03-28T18:16:11.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:16:11 smithi022 ceph-mon[31985]: mds.? [v2:172.21.15.22:6834/817442472,v1:172.21.15.22:6835/817442472] up:resolve 2024-03-28T18:16:11.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:16:11 smithi022 ceph-mon[31985]: fsmap cephfs:5/5 4 up:standby-replay 1 up:standby 1 up:resolve 4 up:active 2024-03-28T18:16:12.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:16:12 smithi134 ceph-mon[35037]: pgmap v287: 97 pgs: 97 active+clean; 117 MiB data, 795 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 0 B/s wr, 11 op/s 2024-03-28T18:16:12.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:16:12 smithi134 ceph-mon[35037]: reconnect by client.35222 172.21.15.22:0/3729116781 after 0.00499994 2024-03-28T18:16:12.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:16:12 smithi134 ceph-mon[35037]: mds.? [v2:172.21.15.22:6834/817442472,v1:172.21.15.22:6835/817442472] up:reconnect 2024-03-28T18:16:12.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:16:12 smithi134 ceph-mon[35037]: fsmap cephfs:5/5 4 up:standby-replay 1 up:standby 1 up:reconnect 4 up:active 2024-03-28T18:16:12.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:16:12 smithi079 ceph-mon[35312]: pgmap v287: 97 pgs: 97 active+clean; 117 MiB data, 795 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 0 B/s wr, 11 op/s 2024-03-28T18:16:12.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:16:12 smithi079 ceph-mon[35312]: reconnect by client.35222 172.21.15.22:0/3729116781 after 0.00499994 2024-03-28T18:16:12.683 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:16:12 smithi079 ceph-mon[35312]: mds.? [v2:172.21.15.22:6834/817442472,v1:172.21.15.22:6835/817442472] up:reconnect 2024-03-28T18:16:12.683 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:16:12 smithi079 ceph-mon[35312]: fsmap cephfs:5/5 4 up:standby-replay 1 up:standby 1 up:reconnect 4 up:active 2024-03-28T18:16:12.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:16:12 smithi022 ceph-mon[31985]: pgmap v287: 97 pgs: 97 active+clean; 117 MiB data, 795 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 0 B/s wr, 11 op/s 2024-03-28T18:16:12.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:16:12 smithi022 ceph-mon[31985]: reconnect by client.35222 172.21.15.22:0/3729116781 after 0.00499994 2024-03-28T18:16:12.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:16:12 smithi022 ceph-mon[31985]: mds.? [v2:172.21.15.22:6834/817442472,v1:172.21.15.22:6835/817442472] up:reconnect 2024-03-28T18:16:12.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:16:12 smithi022 ceph-mon[31985]: fsmap cephfs:5/5 4 up:standby-replay 1 up:standby 1 up:reconnect 4 up:active 2024-03-28T18:16:13.321 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:16:13 smithi022 ceph-mon[31985]: mds.? [v2:172.21.15.22:6834/817442472,v1:172.21.15.22:6835/817442472] up:rejoin 2024-03-28T18:16:13.324 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:16:13 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[94902]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-2145-g3be9a5c9/rpm/el8/BUILD/ceph-19.0.0-2145-g3be9a5c9/src/mds/Locker.cc: In function 'void Locker::set_xlocks_done(MutationImpl*, bool)' thread 7f3f59e00700 time 2024-03-28T18:16:13.071466+0000 2024-03-28T18:16:13.324 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:16:13 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[94902]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-2145-g3be9a5c9/rpm/el8/BUILD/ceph-19.0.0-2145-g3be9a5c9/src/mds/Locker.cc: 626: FAILED ceph_assert(obj->is_auth()) 2024-03-28T18:16:13.324 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:16:13 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[94902]: ceph version 19.0.0-2145-g3be9a5c9 (3be9a5c9bc793e11e2800a8c0c696e8b46742033) squid (dev) 2024-03-28T18:16:13.324 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:16:13 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[94902]: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x135) [0x7f3f657f74a1] 2024-03-28T18:16:13.324 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:16:13 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[94902]: 2: /usr/lib64/ceph/libceph-common.so.2(+0x2d2667) [0x7f3f657f7667] 2024-03-28T18:16:13.324 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:16:13 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[94902]: 3: (Locker::set_xlocks_done(MutationImpl*, bool)+0x493) [0x56107bf61223] 2024-03-28T18:16:13.324 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:16:13 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[94902]: 4: (Server::early_reply(boost::intrusive_ptr const&, CInode*, CDentry*)+0x3e6) [0x56107bddb5d6] 2024-03-28T18:16:13.324 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:16:13 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[94902]: 5: (Server::journal_and_reply(boost::intrusive_ptr const&, CInode*, CDentry*, LogEvent*, MDSLogContextBase*)+0xc1) [0x56107bddbda1] 2024-03-28T18:16:13.324 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:16:13 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[94902]: 6: (Server::handle_client_mkdir(boost::intrusive_ptr const&)+0x5ba) [0x56107be116aa] 2024-03-28T18:16:13.324 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:16:13 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[94902]: 7: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0x733) [0x56107be2e233] 2024-03-28T18:16:13.324 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:16:13 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[94902]: 8: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x56107be30197] 2024-03-28T18:16:13.324 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:16:13 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[94902]: 9: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x56107bd2f11b] 2024-03-28T18:16:13.325 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:16:13 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[94902]: 10: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x56107bd2f6bc] 2024-03-28T18:16:13.325 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:16:13 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[94902]: 11: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bf) [0x56107bd191ff] 2024-03-28T18:16:13.325 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:16:13 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[94902]: 12: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x478) [0x7f3f65a89488] 2024-03-28T18:16:13.325 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:16:13 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[94902]: 13: (DispatchQueue::entry()+0x51f) [0x7f3f65a8660f] 2024-03-28T18:16:13.325 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:16:13 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[94902]: 14: (DispatchQueue::DispatchThread::entry()+0x11) [0x7f3f65b59e81] 2024-03-28T18:16:13.325 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:16:13 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[94902]: 15: /lib64/libpthread.so.0(+0x81ca) [0x7f3f645741ca] 2024-03-28T18:16:13.325 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:16:13 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[94902]: 16: clone() 2024-03-28T18:16:13.325 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:16:13 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[94902]: 2024-03-28T18:16:13.073+0000 7f3f59e00700 -1 /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-2145-g3be9a5c9/rpm/el8/BUILD/ceph-19.0.0-2145-g3be9a5c9/src/mds/Locker.cc: In function 'void Locker::set_xlocks_done(MutationImpl*, bool)' thread 7f3f59e00700 time 2024-03-28T18:16:13.071466+0000 2024-03-28T18:16:13.326 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:16:13 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[94902]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-2145-g3be9a5c9/rpm/el8/BUILD/ceph-19.0.0-2145-g3be9a5c9/src/mds/Locker.cc: 626: FAILED ceph_assert(obj->is_auth()) 2024-03-28T18:16:13.326 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:16:13 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[94902]: 2024-03-28T18:16:13.326 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:16:13 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[94902]: ceph version 19.0.0-2145-g3be9a5c9 (3be9a5c9bc793e11e2800a8c0c696e8b46742033) squid (dev) 2024-03-28T18:16:13.326 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:16:13 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[94902]: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x135) [0x7f3f657f74a1] 2024-03-28T18:16:13.326 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:16:13 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[94902]: 2: /usr/lib64/ceph/libceph-common.so.2(+0x2d2667) [0x7f3f657f7667] 2024-03-28T18:16:13.326 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:16:13 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[94902]: 3: (Locker::set_xlocks_done(MutationImpl*, bool)+0x493) [0x56107bf61223] 2024-03-28T18:16:13.326 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:16:13 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[94902]: 4: (Server::early_reply(boost::intrusive_ptr const&, CInode*, CDentry*)+0x3e6) [0x56107bddb5d6] 2024-03-28T18:16:13.326 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:16:13 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[94902]: 5: (Server::journal_and_reply(boost::intrusive_ptr const&, CInode*, CDentry*, LogEvent*, MDSLogContextBase*)+0xc1) [0x56107bddbda1] 2024-03-28T18:16:13.326 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:16:13 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[94902]: 6: (Server::handle_client_mkdir(boost::intrusive_ptr const&)+0x5ba) [0x56107be116aa] 2024-03-28T18:16:13.326 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:16:13 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[94902]: 7: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0x733) [0x56107be2e233] 2024-03-28T18:16:13.326 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:16:13 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[94902]: 8: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x56107be30197] 2024-03-28T18:16:13.326 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:16:13 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[94902]: 9: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x56107bd2f11b] 2024-03-28T18:16:13.326 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:16:13 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[94902]: 10: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x56107bd2f6bc] 2024-03-28T18:16:13.327 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:16:13 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[94902]: 11: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bf) [0x56107bd191ff] 2024-03-28T18:16:13.327 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:16:13 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[94902]: 12: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x478) [0x7f3f65a89488] 2024-03-28T18:16:13.327 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:16:13 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[94902]: 13: (DispatchQueue::entry()+0x51f) [0x7f3f65a8660f] 2024-03-28T18:16:13.327 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:16:13 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[94902]: 14: (DispatchQueue::DispatchThread::entry()+0x11) [0x7f3f65b59e81] 2024-03-28T18:16:13.327 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:16:13 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[94902]: 15: /lib64/libpthread.so.0(+0x81ca) [0x7f3f645741ca] 2024-03-28T18:16:13.327 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:16:13 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[94902]: 16: clone() 2024-03-28T18:16:13.327 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:16:13 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[94902]: 2024-03-28T18:16:13.327 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:16:13 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[94902]: *** Caught signal (Aborted) ** 2024-03-28T18:16:13.327 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:16:13 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[94902]: in thread 7f3f59e00700 thread_name:ms_dispatch 2024-03-28T18:16:13.327 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:16:13 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[94902]: ceph version 19.0.0-2145-g3be9a5c9 (3be9a5c9bc793e11e2800a8c0c696e8b46742033) squid (dev) 2024-03-28T18:16:13.327 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:16:13 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[94902]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7f3f6457ed20] 2024-03-28T18:16:13.327 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:16:13 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[94902]: 2: gsignal() 2024-03-28T18:16:13.328 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:16:13 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[94902]: 3: abort() 2024-03-28T18:16:13.328 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:16:13 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[94902]: 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x18f) [0x7f3f657f74fb] 2024-03-28T18:16:13.328 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:16:13 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[94902]: 5: /usr/lib64/ceph/libceph-common.so.2(+0x2d2667) [0x7f3f657f7667] 2024-03-28T18:16:13.328 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:16:13 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[94902]: 6: (Locker::set_xlocks_done(MutationImpl*, bool)+0x493) [0x56107bf61223] 2024-03-28T18:16:13.328 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:16:13 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[94902]: 7: (Server::early_reply(boost::intrusive_ptr const&, CInode*, CDentry*)+0x3e6) [0x56107bddb5d6] 2024-03-28T18:16:13.328 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:16:13 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[94902]: 8: (Server::journal_and_reply(boost::intrusive_ptr const&, CInode*, CDentry*, LogEvent*, MDSLogContextBase*)+0xc1) [0x56107bddbda1] 2024-03-28T18:16:13.328 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:16:13 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[94902]: 9: (Server::handle_client_mkdir(boost::intrusive_ptr const&)+0x5ba) [0x56107be116aa] 2024-03-28T18:16:13.328 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:16:13 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[94902]: 10: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0x733) [0x56107be2e233] 2024-03-28T18:16:13.328 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:16:13 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[94902]: 11: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x56107be30197] 2024-03-28T18:16:13.328 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:16:13 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[94902]: 12: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x56107bd2f11b] 2024-03-28T18:16:13.328 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:16:13 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[94902]: 13: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x56107bd2f6bc] 2024-03-28T18:16:13.328 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:16:13 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[94902]: 14: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bf) [0x56107bd191ff] 2024-03-28T18:16:13.328 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:16:13 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[94902]: 15: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x478) [0x7f3f65a89488] 2024-03-28T18:16:13.328 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:16:13 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[94902]: 16: (DispatchQueue::entry()+0x51f) [0x7f3f65a8660f] 2024-03-28T18:16:13.328 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:16:13 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[94902]: 17: (DispatchQueue::DispatchThread::entry()+0x11) [0x7f3f65b59e81] 2024-03-28T18:16:13.329 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:16:13 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[94902]: 18: /lib64/libpthread.so.0(+0x81ca) [0x7f3f645741ca] 2024-03-28T18:16:13.329 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:16:13 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[94902]: 19: clone() 2024-03-28T18:16:13.329 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:16:13 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[94902]: 2024-03-28T18:16:13.076+0000 7f3f59e00700 -1 *** Caught signal (Aborted) ** 2024-03-28T18:16:13.329 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:16:13 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[94902]: in thread 7f3f59e00700 thread_name:ms_dispatch 2024-03-28T18:16:13.329 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:16:13 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[94902]: 2024-03-28T18:16:13.329 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:16:13 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[94902]: ceph version 19.0.0-2145-g3be9a5c9 (3be9a5c9bc793e11e2800a8c0c696e8b46742033) squid (dev) 2024-03-28T18:16:13.329 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:16:13 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[94902]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7f3f6457ed20] 2024-03-28T18:16:13.329 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:16:13 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[94902]: 2: gsignal() 2024-03-28T18:16:13.329 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:16:13 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[94902]: 3: abort() 2024-03-28T18:16:13.329 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:16:13 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[94902]: 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x18f) [0x7f3f657f74fb] 2024-03-28T18:16:13.329 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:16:13 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[94902]: 5: /usr/lib64/ceph/libceph-common.so.2(+0x2d2667) [0x7f3f657f7667] 2024-03-28T18:16:13.330 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:16:13 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[94902]: 6: (Locker::set_xlocks_done(MutationImpl*, bool)+0x493) [0x56107bf61223] 2024-03-28T18:16:13.330 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:16:13 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[94902]: 7: (Server::early_reply(boost::intrusive_ptr const&, CInode*, CDentry*)+0x3e6) [0x56107bddb5d6] 2024-03-28T18:16:13.330 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:16:13 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[94902]: 8: (Server::journal_and_reply(boost::intrusive_ptr const&, CInode*, CDentry*, LogEvent*, MDSLogContextBase*)+0xc1) [0x56107bddbda1] 2024-03-28T18:16:13.330 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:16:13 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[94902]: 9: (Server::handle_client_mkdir(boost::intrusive_ptr const&)+0x5ba) [0x56107be116aa] 2024-03-28T18:16:13.330 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:16:13 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[94902]: 10: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0x733) [0x56107be2e233] 2024-03-28T18:16:13.330 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:16:13 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[94902]: 11: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x56107be30197] 2024-03-28T18:16:13.330 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:16:13 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[94902]: 12: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x56107bd2f11b] 2024-03-28T18:16:13.330 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:16:13 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[94902]: 13: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x56107bd2f6bc] 2024-03-28T18:16:13.330 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:16:13 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[94902]: 14: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bf) [0x56107bd191ff] 2024-03-28T18:16:13.330 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:16:13 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[94902]: 15: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x478) [0x7f3f65a89488] 2024-03-28T18:16:13.330 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:16:13 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[94902]: 16: (DispatchQueue::entry()+0x51f) [0x7f3f65a8660f] 2024-03-28T18:16:13.331 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:16:13 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[94902]: 17: (DispatchQueue::DispatchThread::entry()+0x11) [0x7f3f65b59e81] 2024-03-28T18:16:13.331 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:16:13 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[94902]: 18: /lib64/libpthread.so.0(+0x81ca) [0x7f3f645741ca] 2024-03-28T18:16:13.331 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:16:13 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[94902]: 19: clone() 2024-03-28T18:16:13.331 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:16:13 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[94902]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-03-28T18:16:13.331 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:16:13 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[94902]: 2024-03-28T18:16:13.331 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:16:13 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[94902]: -1> 2024-03-28T18:16:13.073+0000 7f3f59e00700 -1 /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-2145-g3be9a5c9/rpm/el8/BUILD/ceph-19.0.0-2145-g3be9a5c9/src/mds/Locker.cc: In function 'void Locker::set_xlocks_done(MutationImpl*, bool)' thread 7f3f59e00700 time 2024-03-28T18:16:13.071466+0000 2024-03-28T18:16:13.331 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:16:13 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[94902]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-2145-g3be9a5c9/rpm/el8/BUILD/ceph-19.0.0-2145-g3be9a5c9/src/mds/Locker.cc: 626: FAILED ceph_assert(obj->is_auth()) 2024-03-28T18:16:13.331 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:16:13 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[94902]: 2024-03-28T18:16:13.331 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:16:13 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[94902]: ceph version 19.0.0-2145-g3be9a5c9 (3be9a5c9bc793e11e2800a8c0c696e8b46742033) squid (dev) 2024-03-28T18:16:13.331 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:16:13 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[94902]: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x135) [0x7f3f657f74a1] 2024-03-28T18:16:13.331 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:16:13 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[94902]: 2: /usr/lib64/ceph/libceph-common.so.2(+0x2d2667) [0x7f3f657f7667] 2024-03-28T18:16:13.331 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:16:13 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[94902]: 3: (Locker::set_xlocks_done(MutationImpl*, bool)+0x493) [0x56107bf61223] 2024-03-28T18:16:13.331 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:16:13 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[94902]: 4: (Server::early_reply(boost::intrusive_ptr const&, CInode*, CDentry*)+0x3e6) [0x56107bddb5d6] 2024-03-28T18:16:13.331 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:16:13 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[94902]: 5: (Server::journal_and_reply(boost::intrusive_ptr const&, CInode*, CDentry*, LogEvent*, MDSLogContextBase*)+0xc1) [0x56107bddbda1] 2024-03-28T18:16:13.331 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:16:13 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[94902]: 6: (Server::handle_client_mkdir(boost::intrusive_ptr const&)+0x5ba) [0x56107be116aa] 2024-03-28T18:16:13.332 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:16:13 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[94902]: 7: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0x733) [0x56107be2e233] 2024-03-28T18:16:13.332 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:16:13 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[94902]: 8: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x56107be30197] 2024-03-28T18:16:13.332 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:16:13 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[94902]: 9: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x56107bd2f11b] 2024-03-28T18:16:13.332 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:16:13 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[94902]: 10: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x56107bd2f6bc] 2024-03-28T18:16:13.332 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:16:13 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[94902]: 11: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bf) [0x56107bd191ff] 2024-03-28T18:16:13.332 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:16:13 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[94902]: 12: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x478) [0x7f3f65a89488] 2024-03-28T18:16:13.332 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:16:13 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[94902]: 13: (DispatchQueue::entry()+0x51f) [0x7f3f65a8660f] 2024-03-28T18:16:13.332 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:16:13 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[94902]: 14: (DispatchQueue::DispatchThread::entry()+0x11) [0x7f3f65b59e81] 2024-03-28T18:16:13.332 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:16:13 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[94902]: 15: /lib64/libpthread.so.0(+0x81ca) [0x7f3f645741ca] 2024-03-28T18:16:13.332 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:16:13 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[94902]: 16: clone() 2024-03-28T18:16:13.332 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:16:13 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[94902]: 2024-03-28T18:16:13.332 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:16:13 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[94902]: 0> 2024-03-28T18:16:13.076+0000 7f3f59e00700 -1 *** Caught signal (Aborted) ** 2024-03-28T18:16:13.332 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:16:13 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[94902]: in thread 7f3f59e00700 thread_name:ms_dispatch 2024-03-28T18:16:13.332 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:16:13 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[94902]: 2024-03-28T18:16:13.332 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:16:13 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[94902]: ceph version 19.0.0-2145-g3be9a5c9 (3be9a5c9bc793e11e2800a8c0c696e8b46742033) squid (dev) 2024-03-28T18:16:13.332 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:16:13 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[94902]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7f3f6457ed20] 2024-03-28T18:16:13.333 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:16:13 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[94902]: 2: gsignal() 2024-03-28T18:16:13.333 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:16:13 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[94902]: 3: abort() 2024-03-28T18:16:13.333 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:16:13 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[94902]: 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x18f) [0x7f3f657f74fb] 2024-03-28T18:16:13.333 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:16:13 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[94902]: 5: /usr/lib64/ceph/libceph-common.so.2(+0x2d2667) [0x7f3f657f7667] 2024-03-28T18:16:13.333 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:16:13 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[94902]: 6: (Locker::set_xlocks_done(MutationImpl*, bool)+0x493) [0x56107bf61223] 2024-03-28T18:16:13.333 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:16:13 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[94902]: 7: (Server::early_reply(boost::intrusive_ptr const&, CInode*, CDentry*)+0x3e6) [0x56107bddb5d6] 2024-03-28T18:16:13.333 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:16:13 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[94902]: 8: (Server::journal_and_reply(boost::intrusive_ptr const&, CInode*, CDentry*, LogEvent*, MDSLogContextBase*)+0xc1) [0x56107bddbda1] 2024-03-28T18:16:13.334 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:16:13 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[94902]: 9: (Server::handle_client_mkdir(boost::intrusive_ptr const&)+0x5ba) [0x56107be116aa] 2024-03-28T18:16:13.334 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:16:13 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[94902]: 10: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0x733) [0x56107be2e233] 2024-03-28T18:16:13.334 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:16:13 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[94902]: 11: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x56107be30197] 2024-03-28T18:16:13.334 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:16:13 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[94902]: 12: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x56107bd2f11b] 2024-03-28T18:16:13.334 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:16:13 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[94902]: 13: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x56107bd2f6bc] 2024-03-28T18:16:13.334 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:16:13 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[94902]: 14: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bf) [0x56107bd191ff] 2024-03-28T18:16:13.334 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:16:13 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[94902]: 15: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x478) [0x7f3f65a89488] 2024-03-28T18:16:13.334 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:16:13 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[94902]: 16: (DispatchQueue::entry()+0x51f) [0x7f3f65a8660f] 2024-03-28T18:16:13.334 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:16:13 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[94902]: 17: (DispatchQueue::DispatchThread::entry()+0x11) [0x7f3f65b59e81] 2024-03-28T18:16:13.334 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:16:13 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[94902]: 18: /lib64/libpthread.so.0(+0x81ca) [0x7f3f645741ca] 2024-03-28T18:16:13.334 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:16:13 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[94902]: 19: clone() 2024-03-28T18:16:13.334 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:16:13 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[94902]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-03-28T18:16:13.334 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:16:13 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[94902]: 2024-03-28T18:16:13.335 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:16:13 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[94902]: -1> 2024-03-28T18:16:13.073+0000 7f3f59e00700 -1 /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-2145-g3be9a5c9/rpm/el8/BUILD/ceph-19.0.0-2145-g3be9a5c9/src/mds/Locker.cc: In function 'void Locker::set_xlocks_done(MutationImpl*, bool)' thread 7f3f59e00700 time 2024-03-28T18:16:13.071466+0000 2024-03-28T18:16:13.335 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:16:13 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[94902]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-2145-g3be9a5c9/rpm/el8/BUILD/ceph-19.0.0-2145-g3be9a5c9/src/mds/Locker.cc: 626: FAILED ceph_assert(obj->is_auth()) 2024-03-28T18:16:13.335 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:16:13 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[94902]: 2024-03-28T18:16:13.335 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:16:13 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[94902]: ceph version 19.0.0-2145-g3be9a5c9 (3be9a5c9bc793e11e2800a8c0c696e8b46742033) squid (dev) 2024-03-28T18:16:13.335 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:16:13 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[94902]: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x135) [0x7f3f657f74a1] 2024-03-28T18:16:13.335 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:16:13 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[94902]: 2: /usr/lib64/ceph/libceph-common.so.2(+0x2d2667) [0x7f3f657f7667] 2024-03-28T18:16:13.335 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:16:13 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[94902]: 3: (Locker::set_xlocks_done(MutationImpl*, bool)+0x493) [0x56107bf61223] 2024-03-28T18:16:13.335 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:16:13 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[94902]: 4: (Server::early_reply(boost::intrusive_ptr const&, CInode*, CDentry*)+0x3e6) [0x56107bddb5d6] 2024-03-28T18:16:13.335 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:16:13 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[94902]: 5: (Server::journal_and_reply(boost::intrusive_ptr const&, CInode*, CDentry*, LogEvent*, MDSLogContextBase*)+0xc1) [0x56107bddbda1] 2024-03-28T18:16:13.335 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:16:13 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[94902]: 6: (Server::handle_client_mkdir(boost::intrusive_ptr const&)+0x5ba) [0x56107be116aa] 2024-03-28T18:16:13.335 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:16:13 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[94902]: 7: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0x733) [0x56107be2e233] 2024-03-28T18:16:13.335 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:16:13 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[94902]: 8: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x56107be30197] 2024-03-28T18:16:13.335 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:16:13 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[94902]: 9: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x56107bd2f11b] 2024-03-28T18:16:13.335 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:16:13 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[94902]: 10: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x56107bd2f6bc] 2024-03-28T18:16:13.335 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:16:13 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[94902]: 11: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bf) [0x56107bd191ff] 2024-03-28T18:16:13.335 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:16:13 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[94902]: 12: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x478) [0x7f3f65a89488] 2024-03-28T18:16:13.336 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:16:13 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[94902]: 13: (DispatchQueue::entry()+0x51f) [0x7f3f65a8660f] 2024-03-28T18:16:13.336 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:16:13 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[94902]: 14: (DispatchQueue::DispatchThread::entry()+0x11) [0x7f3f65b59e81] 2024-03-28T18:16:13.336 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:16:13 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[94902]: 15: /lib64/libpthread.so.0(+0x81ca) [0x7f3f645741ca] 2024-03-28T18:16:13.336 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:16:13 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[94902]: 16: clone() 2024-03-28T18:16:13.336 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:16:13 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[94902]: 2024-03-28T18:16:13.336 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:16:13 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[94902]: 0> 2024-03-28T18:16:13.076+0000 7f3f59e00700 -1 *** Caught signal (Aborted) ** 2024-03-28T18:16:13.336 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:16:13 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[94902]: in thread 7f3f59e00700 thread_name:ms_dispatch 2024-03-28T18:16:13.336 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:16:13 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[94902]: 2024-03-28T18:16:13.336 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:16:13 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[94902]: ceph version 19.0.0-2145-g3be9a5c9 (3be9a5c9bc793e11e2800a8c0c696e8b46742033) squid (dev) 2024-03-28T18:16:13.336 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:16:13 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[94902]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7f3f6457ed20] 2024-03-28T18:16:13.336 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:16:13 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[94902]: 2: gsignal() 2024-03-28T18:16:13.336 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:16:13 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[94902]: 3: abort() 2024-03-28T18:16:13.336 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:16:13 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[94902]: 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x18f) [0x7f3f657f74fb] 2024-03-28T18:16:13.336 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:16:13 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[94902]: 5: /usr/lib64/ceph/libceph-common.so.2(+0x2d2667) [0x7f3f657f7667] 2024-03-28T18:16:13.336 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:16:13 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[94902]: 6: (Locker::set_xlocks_done(MutationImpl*, bool)+0x493) [0x56107bf61223] 2024-03-28T18:16:13.336 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:16:13 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[94902]: 7: (Server::early_reply(boost::intrusive_ptr const&, CInode*, CDentry*)+0x3e6) [0x56107bddb5d6] 2024-03-28T18:16:13.337 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:16:13 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[94902]: 8: (Server::journal_and_reply(boost::intrusive_ptr const&, CInode*, CDentry*, LogEvent*, MDSLogContextBase*)+0xc1) [0x56107bddbda1] 2024-03-28T18:16:13.337 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:16:13 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[94902]: 9: (Server::handle_client_mkdir(boost::intrusive_ptr const&)+0x5ba) [0x56107be116aa] 2024-03-28T18:16:13.337 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:16:13 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[94902]: 10: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0x733) [0x56107be2e233] 2024-03-28T18:16:13.337 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:16:13 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[94902]: 11: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x56107be30197] 2024-03-28T18:16:13.337 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:16:13 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[94902]: 12: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x56107bd2f11b] 2024-03-28T18:16:13.337 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:16:13 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[94902]: 13: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x56107bd2f6bc] 2024-03-28T18:16:13.337 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:16:13 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[94902]: 14: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bf) [0x56107bd191ff] 2024-03-28T18:16:13.337 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:16:13 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[94902]: 15: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x478) [0x7f3f65a89488] 2024-03-28T18:16:13.337 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:16:13 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[94902]: 16: (DispatchQueue::entry()+0x51f) [0x7f3f65a8660f] 2024-03-28T18:16:13.337 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:16:13 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[94902]: 17: (DispatchQueue::DispatchThread::entry()+0x11) [0x7f3f65b59e81] 2024-03-28T18:16:13.337 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:16:13 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[94902]: 18: /lib64/libpthread.so.0(+0x81ca) [0x7f3f645741ca] 2024-03-28T18:16:13.337 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:16:13 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[94902]: 19: clone() 2024-03-28T18:16:13.337 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:16:13 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[94902]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-03-28T18:16:13.337 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:16:13 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[94902]: 2024-03-28T18:16:13.337 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:16:13 smithi022 podman[95322]: 2024-03-28 18:16:13.189969056 +0000 UTC m=+0.033189030 container died bfd01a52e140e2808593024599ff285cd4ff7d4dab980a8e8111fe3ce38a29a5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_CLEAN=True, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, org.label-schema.build-date=20240319, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , ceph=True, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=) 2024-03-28T18:16:13.596 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:16:13 smithi022 ceph-mon[31985]: fsmap cephfs:5/5 4 up:standby-replay 1 up:standby 1 up:rejoin 4 up:active 2024-03-28T18:16:13.596 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:16:13 smithi022 ceph-mon[31985]: daemon mds.j is now active in filesystem cephfs as rank 1 2024-03-28T18:16:13.596 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:16:13 smithi022 ceph-mon[31985]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-03-28T18:16:13.596 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:16:13 smithi022 ceph-mon[31985]: Cluster is now healthy 2024-03-28T18:16:13.596 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:16:13 smithi022 ceph-mon[31985]: mds.? [v2:172.21.15.22:6834/817442472,v1:172.21.15.22:6835/817442472] up:active 2024-03-28T18:16:13.596 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:16:13 smithi022 ceph-mon[31985]: fsmap cephfs:5 5 up:standby-replay 5 up:active 2024-03-28T18:16:13.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:16:13 smithi134 ceph-mon[35037]: mds.? [v2:172.21.15.22:6834/817442472,v1:172.21.15.22:6835/817442472] up:rejoin 2024-03-28T18:16:13.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:16:13 smithi134 ceph-mon[35037]: fsmap cephfs:5/5 4 up:standby-replay 1 up:standby 1 up:rejoin 4 up:active 2024-03-28T18:16:13.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:16:13 smithi134 ceph-mon[35037]: daemon mds.j is now active in filesystem cephfs as rank 1 2024-03-28T18:16:13.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:16:13 smithi134 ceph-mon[35037]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-03-28T18:16:13.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:16:13 smithi134 ceph-mon[35037]: Cluster is now healthy 2024-03-28T18:16:13.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:16:13 smithi134 ceph-mon[35037]: mds.? [v2:172.21.15.22:6834/817442472,v1:172.21.15.22:6835/817442472] up:active 2024-03-28T18:16:13.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:16:13 smithi134 ceph-mon[35037]: fsmap cephfs:5 5 up:standby-replay 5 up:active 2024-03-28T18:16:13.683 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:16:13 smithi079 ceph-mon[35312]: mds.? [v2:172.21.15.22:6834/817442472,v1:172.21.15.22:6835/817442472] up:rejoin 2024-03-28T18:16:13.683 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:16:13 smithi079 ceph-mon[35312]: fsmap cephfs:5/5 4 up:standby-replay 1 up:standby 1 up:rejoin 4 up:active 2024-03-28T18:16:13.683 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:16:13 smithi079 ceph-mon[35312]: daemon mds.j is now active in filesystem cephfs as rank 1 2024-03-28T18:16:13.683 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:16:13 smithi079 ceph-mon[35312]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-03-28T18:16:13.683 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:16:13 smithi079 ceph-mon[35312]: Cluster is now healthy 2024-03-28T18:16:13.683 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:16:13 smithi079 ceph-mon[35312]: mds.? [v2:172.21.15.22:6834/817442472,v1:172.21.15.22:6835/817442472] up:active 2024-03-28T18:16:13.683 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:16:13 smithi079 ceph-mon[35312]: fsmap cephfs:5 5 up:standby-replay 5 up:active 2024-03-28T18:16:14.282 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:16:13 smithi022 podman[95322]: 2024-03-28 18:16:13.992200044 +0000 UTC m=+0.835420054 container remove bfd01a52e140e2808593024599ff285cd4ff7d4dab980a8e8111fe3ce38a29a5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_CLEAN=True, io.buildah.version=1.33.5, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240319, GIT_BRANCH=HEAD, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c) 2024-03-28T18:16:14.282 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:16:14 smithi022 systemd[1]: ceph-fd220748-ed2b-11ee-95d0-87774f69a715@mds.j.service: Main process exited, code=exited, status=134/n/a 2024-03-28T18:16:14.432 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:16:14 smithi079 ceph-mon[35312]: pgmap v288: 97 pgs: 97 active+clean; 117 MiB data, 795 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 0 B/s wr, 11 op/s 2024-03-28T18:16:14.433 INFO:journalctl@ceph.mds.b.smithi079.stdout:Mar 28 18:16:14 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-b[65245]: 2024-03-28T18:16:14.103+0000 7f845f400700 -1 mds.pinger is_rank_lagging: rank=1 was never sent ping request. 2024-03-28T18:16:14.674 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:16:14 smithi134 ceph-mon[35037]: pgmap v288: 97 pgs: 97 active+clean; 117 MiB data, 795 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 0 B/s wr, 11 op/s 2024-03-28T18:16:14.782 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:16:14 smithi022 systemd[1]: ceph-fd220748-ed2b-11ee-95d0-87774f69a715@mds.j.service: Failed with result 'exit-code'. 2024-03-28T18:16:14.782 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:16:14 smithi022 systemd[1]: ceph-fd220748-ed2b-11ee-95d0-87774f69a715@mds.j.service: Consumed 1.893s CPU time. 2024-03-28T18:16:14.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:16:14 smithi022 ceph-mon[31985]: pgmap v288: 97 pgs: 97 active+clean; 117 MiB data, 795 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 0 B/s wr, 11 op/s 2024-03-28T18:16:16.674 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:16:16 smithi134 ceph-mon[35037]: pgmap v289: 97 pgs: 97 active+clean; 118 MiB data, 795 MiB used, 1.0 TiB / 1.0 TiB avail; 121 KiB/s rd, 33 KiB/s wr, 42 op/s 2024-03-28T18:16:16.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:16:16 smithi079 ceph-mon[35312]: pgmap v289: 97 pgs: 97 active+clean; 118 MiB data, 795 MiB used, 1.0 TiB / 1.0 TiB avail; 121 KiB/s rd, 33 KiB/s wr, 42 op/s 2024-03-28T18:16:16.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:16:16 smithi022 ceph-mon[31985]: pgmap v289: 97 pgs: 97 active+clean; 118 MiB data, 795 MiB used, 1.0 TiB / 1.0 TiB avail; 121 KiB/s rd, 33 KiB/s wr, 42 op/s 2024-03-28T18:16:18.674 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:16:18 smithi134 ceph-mon[35037]: pgmap v290: 97 pgs: 97 active+clean; 118 MiB data, 795 MiB used, 1.0 TiB / 1.0 TiB avail; 121 KiB/s rd, 33 KiB/s wr, 42 op/s 2024-03-28T18:16:18.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:16:18 smithi079 ceph-mon[35312]: pgmap v290: 97 pgs: 97 active+clean; 118 MiB data, 795 MiB used, 1.0 TiB / 1.0 TiB avail; 121 KiB/s rd, 33 KiB/s wr, 42 op/s 2024-03-28T18:16:18.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:16:18 smithi022 ceph-mon[31985]: pgmap v290: 97 pgs: 97 active+clean; 118 MiB data, 795 MiB used, 1.0 TiB / 1.0 TiB avail; 121 KiB/s rd, 33 KiB/s wr, 42 op/s 2024-03-28T18:16:19.876 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-28T18:16:19.877 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-28T18:16:20.121 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:16:20.121+0000 7f5b5ac00640 1 -- 172.21.15.22:0/3505224691 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5b540743f0 msgr2=0x7f5b5410d800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:16:20.122 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:16:20.121+0000 7f5b5ac00640 1 --2- 172.21.15.22:0/3505224691 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5b540743f0 0x7f5b5410d800 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7f5b40009900 tx=0x7f5b4002bcd0 comp rx=0 tx=0).stop 2024-03-28T18:16:20.122 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:16:20.122+0000 7f5b5ac00640 1 -- 172.21.15.22:0/3505224691 shutdown_connections 2024-03-28T18:16:20.122 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:16:20.122+0000 7f5b5ac00640 1 --2- 172.21.15.22:0/3505224691 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f5b5410dd40 0x7f5b54110190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:16:20.122 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:16:20.122+0000 7f5b5ac00640 1 --2- 172.21.15.22:0/3505224691 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5b540743f0 0x7f5b5410d800 unknown :-1 s=CLOSED pgs=770 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:16:20.122 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:16:20.122+0000 7f5b5ac00640 1 --2- 172.21.15.22:0/3505224691 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5b54073ad0 0x7f5b54073eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:16:20.122 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:16:20.122+0000 7f5b5ac00640 1 -- 172.21.15.22:0/3505224691 >> 172.21.15.22:0/3505224691 conn(0x7f5b540fd010 msgr2=0x7f5b540ff430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:16:20.122 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:16:20.122+0000 7f5b5ac00640 1 -- 172.21.15.22:0/3505224691 shutdown_connections 2024-03-28T18:16:20.122 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:16:20.122+0000 7f5b5ac00640 1 -- 172.21.15.22:0/3505224691 wait complete. 2024-03-28T18:16:20.123 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:16:20.123+0000 7f5b5ac00640 1 Processor -- start 2024-03-28T18:16:20.123 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:16:20.123+0000 7f5b5ac00640 1 -- start start 2024-03-28T18:16:20.124 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:16:20.124+0000 7f5b5ac00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5b54073ad0 0x7f5b5419a010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:16:20.124 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:16:20.124+0000 7f5b5ac00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5b540743f0 0x7f5b5419a550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:16:20.124 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:16:20.124+0000 7f5b5ac00640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f5b5410dd40 0x7f5b5419e8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:16:20.125 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:16:20.124+0000 7f5b5ac00640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b54113ab0 con 0x7f5b540743f0 2024-03-28T18:16:20.125 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:16:20.124+0000 7f5b5ac00640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b54113930 con 0x7f5b5410dd40 2024-03-28T18:16:20.125 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:16:20.124+0000 7f5b5ac00640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b54113c30 con 0x7f5b54073ad0 2024-03-28T18:16:20.125 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:16:20.124+0000 7f5b58e00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5b540743f0 0x7f5b5419a550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:16:20.125 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:16:20.124+0000 7f5b58e00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5b540743f0 0x7f5b5419a550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:43268/0 (socket says 172.21.15.22:43268) 2024-03-28T18:16:20.125 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:16:20.124+0000 7f5b58e00640 1 -- 172.21.15.22:0/3645100612 learned_addr learned my addr 172.21.15.22:0/3645100612 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-28T18:16:20.125 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:16:20.124+0000 7f5b59800640 1 --2- 172.21.15.22:0/3645100612 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5b54073ad0 0x7f5b5419a010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:16:20.125 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:16:20.124+0000 7f5b5a200640 1 --2- 172.21.15.22:0/3645100612 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f5b5410dd40 0x7f5b5419e8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:16:20.125 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:16:20.124+0000 7f5b58e00640 1 -- 172.21.15.22:0/3645100612 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5b54073ad0 msgr2=0x7f5b5419a010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:16:20.125 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:16:20.124+0000 7f5b58e00640 1 --2- 172.21.15.22:0/3645100612 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5b54073ad0 0x7f5b5419a010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:16:20.125 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:16:20.124+0000 7f5b58e00640 1 -- 172.21.15.22:0/3645100612 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f5b5410dd40 msgr2=0x7f5b5419e8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:16:20.125 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:16:20.124+0000 7f5b58e00640 1 --2- 172.21.15.22:0/3645100612 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f5b5410dd40 0x7f5b5419e8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:16:20.125 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:16:20.124+0000 7f5b58e00640 1 -- 172.21.15.22:0/3645100612 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5b5419f000 con 0x7f5b540743f0 2024-03-28T18:16:20.125 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:16:20.125+0000 7f5b58e00640 1 --2- 172.21.15.22:0/3645100612 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5b540743f0 0x7f5b5419a550 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 crypto rx=0x7f5b40009230 tx=0x7f5b400045e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:16:20.126 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:16:20.125+0000 7f5b52000640 1 -- 172.21.15.22:0/3645100612 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b40043020 con 0x7f5b540743f0 2024-03-28T18:16:20.126 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:16:20.125+0000 7f5b5ac00640 1 -- 172.21.15.22:0/3645100612 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5b5419f290 con 0x7f5b540743f0 2024-03-28T18:16:20.126 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:16:20.125+0000 7f5b52000640 1 -- 172.21.15.22:0/3645100612 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f5b4002cb40 con 0x7f5b540743f0 2024-03-28T18:16:20.127 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:16:20.125+0000 7f5b5ac00640 1 -- 172.21.15.22:0/3645100612 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5b541a6b80 con 0x7f5b540743f0 2024-03-28T18:16:20.127 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:16:20.126+0000 7f5b52000640 1 -- 172.21.15.22:0/3645100612 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b40003b60 con 0x7f5b540743f0 2024-03-28T18:16:20.129 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:16:20.129+0000 7f5b5ee15740 1 Processor -- start 2024-03-28T18:16:20.129 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:16:20.130+0000 7f5b5ee15740 1 -- start start 2024-03-28T18:16:20.130 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:16:20.130+0000 7f5b5ee15740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5628060aedd0 0x5628060adb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:16:20.130 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:16:20.130+0000 7f5b5ee15740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5628060ae0f0 0x5628060ae570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:16:20.130 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:16:20.130+0000 7f5b5ee15740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5628060b7460 0x5628060b9850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:16:20.130 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:16:20.130+0000 7f5b5ee15740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x562806012250 con 0x5628060aedd0 2024-03-28T18:16:20.130 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:16:20.130+0000 7f5b5ee15740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x562805fbaa30 con 0x5628060b7460 2024-03-28T18:16:20.130 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:16:20.130+0000 7f5b5ee15740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x56280600e2f0 con 0x5628060ae0f0 2024-03-28T18:16:20.131 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:16:20.131+0000 7f5b52000640 1 -- 172.21.15.22:0/3645100612 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 26) v1 ==== 137421+0+0 (secure 0 0 0) 0x7f5b40046020 con 0x7f5b540743f0 2024-03-28T18:16:20.132 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:16:20.131+0000 7f5b5a200640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5628060aedd0 0x5628060adb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:16:20.132 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:16:20.131+0000 7f5b5a200640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5628060aedd0 0x5628060adb20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:43278/0 (socket says 172.21.15.22:43278) 2024-03-28T18:16:20.132 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:16:20.131+0000 7f5b5a200640 1 -- 172.21.15.22:0/3952708649 learned_addr learned my addr 172.21.15.22:0/3952708649 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-28T18:16:20.132 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:16:20.131+0000 7f5b59800640 1 --2- 172.21.15.22:0/3952708649 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5628060ae0f0 0x5628060ae570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:16:20.132 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:16:20.131+0000 7f5b5a200640 1 -- 172.21.15.22:0/3952708649 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5628060ae0f0 msgr2=0x5628060ae570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T18:16:20.132 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:16:20.131+0000 7f5b5a200640 1 --2- 172.21.15.22:0/3952708649 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5628060ae0f0 0x5628060ae570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:16:20.132 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:16:20.131+0000 7f5b5a200640 1 -- 172.21.15.22:0/3952708649 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5628060b7460 msgr2=0x5628060b9850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T18:16:20.132 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:16:20.131+0000 7f5b5a200640 1 --2- 172.21.15.22:0/3952708649 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5628060b7460 0x5628060b9850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:16:20.132 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:16:20.131+0000 7f5b5a200640 1 -- 172.21.15.22:0/3952708649 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562805fa5690 con 0x5628060aedd0 2024-03-28T18:16:20.132 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:16:20.131+0000 7f5b5a200640 1 --2- 172.21.15.22:0/3952708649 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5628060aedd0 0x5628060adb20 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 crypto rx=0x7f5b4c009e60 tx=0x7f5b4c010740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=27f47deda91e607 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:16:20.132 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:16:20.132+0000 7f5b4aa00640 1 -- 172.21.15.22:0/3952708649 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b4c016070 con 0x5628060aedd0 2024-03-28T18:16:20.132 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:16:20.132+0000 7f5b4aa00640 1 -- 172.21.15.22:0/3952708649 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f5b4c004d20 con 0x5628060aedd0 2024-03-28T18:16:20.132 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:16:20.132+0000 7f5b4aa00640 1 -- 172.21.15.22:0/3952708649 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b4c0127b0 con 0x5628060aedd0 2024-03-28T18:16:20.132 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:16:20.132+0000 7f5b52000640 1 --2- 172.21.15.22:0/3645100612 >> [v2:172.21.15.22:6800/1357365082,v1:172.21.15.22:6801/1357365082] conn(0x7f5b380a2970 0x7f5b380a4e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:16:20.136 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:16:20.132+0000 7f5b52000640 1 -- 172.21.15.22:0/3645100612 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(118..118 src has 1..118) v4 ==== 10217+0+0 (secure 0 0 0) 0x7f5b400f15d0 con 0x7f5b540743f0 2024-03-28T18:16:20.136 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:16:20.132+0000 7f5b5ee15740 1 -- 172.21.15.22:0/3952708649 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5628060aedd0 msgr2=0x5628060adb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:16:20.136 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:16:20.132+0000 7f5b5ee15740 1 --2- 172.21.15.22:0/3952708649 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5628060aedd0 0x5628060adb20 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 crypto rx=0x7f5b4c009e60 tx=0x7f5b4c010740 comp rx=0 tx=0).stop 2024-03-28T18:16:20.136 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:16:20.132+0000 7f5b59800640 1 --2- 172.21.15.22:0/3645100612 >> [v2:172.21.15.22:6800/1357365082,v1:172.21.15.22:6801/1357365082] conn(0x7f5b380a2970 0x7f5b380a4e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:16:20.137 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:16:20.133+0000 7f5b5ee15740 1 -- 172.21.15.22:0/3952708649 shutdown_connections 2024-03-28T18:16:20.137 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:16:20.133+0000 7f5b5ee15740 1 --2- 172.21.15.22:0/3952708649 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5628060b7460 0x5628060b9850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:16:20.137 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:16:20.133+0000 7f5b5ee15740 1 --2- 172.21.15.22:0/3952708649 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5628060ae0f0 0x5628060ae570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:16:20.137 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:16:20.133+0000 7f5b5ee15740 1 --2- 172.21.15.22:0/3952708649 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5628060aedd0 0x5628060adb20 unknown :-1 s=CLOSED pgs=772 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:16:20.137 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:16:20.133+0000 7f5b5ee15740 1 -- 172.21.15.22:0/3952708649 >> 172.21.15.22:0/3952708649 conn(0x562806075200 msgr2=0x5628060a8b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:16:20.137 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:16:20.133+0000 7f5b5ee15740 1 -- 172.21.15.22:0/3952708649 shutdown_connections 2024-03-28T18:16:20.137 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:16:20.133+0000 7f5b5ee15740 1 -- 172.21.15.22:0/3952708649 wait complete. 2024-03-28T18:16:20.137 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:16:20.133+0000 7f5b5ee15740 1 -- start start 2024-03-28T18:16:20.137 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:16:20.133+0000 7f5b59800640 1 --2- 172.21.15.22:0/3645100612 >> [v2:172.21.15.22:6800/1357365082,v1:172.21.15.22:6801/1357365082] conn(0x7f5b380a2970 0x7f5b380a4e30 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f5b44005fd0 tx=0x7f5b4400bf90 comp rx=0 tx=0).ready entity=mgr.15522 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:16:20.137 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:16:20.133+0000 7f5b5ee15740 1 Processor -- start 2024-03-28T18:16:20.137 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:16:20.134+0000 7f5b5ee15740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5628060ae0f0 0x5628060b6be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:16:20.137 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:16:20.134+0000 7f5b5ee15740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5628060aedd0 0x5628060b3f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:16:20.137 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:16:20.134+0000 7f5b5ee15740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5628060b7460 0x5628060b44b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:16:20.137 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:16:20.134+0000 7f5b5ee15740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x5628060ba2d0 con 0x5628060b7460 2024-03-28T18:16:20.138 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:16:20.134+0000 7f5b5ee15740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x562805fcde80 con 0x5628060aedd0 2024-03-28T18:16:20.138 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:16:20.134+0000 7f5b5ee15740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x5628060ba450 con 0x5628060ae0f0 2024-03-28T18:16:20.138 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:16:20.134+0000 7f5b5ee15740 10 client.0 _ll_register_callbacks cb 0x562805fe0130 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-28T18:16:20.138 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:16:20.134+0000 7f5b5a200640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5628060b7460 0x5628060b44b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:16:20.138 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:16:20.134+0000 7f5b5a200640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5628060b7460 0x5628060b44b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:43290/0 (socket says 172.21.15.22:43290) 2024-03-28T18:16:20.138 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:16:20.134+0000 7f5b5a200640 1 -- 172.21.15.22:0/496947253 learned_addr learned my addr 172.21.15.22:0/496947253 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-28T18:16:20.138 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:16:20.134+0000 7f5b58e00640 1 --2- 172.21.15.22:0/496947253 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5628060aedd0 0x5628060b3f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:16:20.138 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:16:20.134+0000 7f5b59800640 1 --2- 172.21.15.22:0/496947253 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5628060ae0f0 0x5628060b6be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:16:20.138 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:16:20.135+0000 7f5b5a200640 1 -- 172.21.15.22:0/496947253 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5628060ae0f0 msgr2=0x5628060b6be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T18:16:20.138 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:16:20.135+0000 7f5b5a200640 1 --2- 172.21.15.22:0/496947253 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5628060ae0f0 0x5628060b6be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:16:20.138 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:16:20.135+0000 7f5b5a200640 1 -- 172.21.15.22:0/496947253 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5628060aedd0 msgr2=0x5628060b3f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T18:16:20.138 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:16:20.135+0000 7f5b5a200640 1 --2- 172.21.15.22:0/496947253 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5628060aedd0 0x5628060b3f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:16:20.138 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:16:20.135+0000 7f5b5a200640 1 -- 172.21.15.22:0/496947253 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x562805df07d0 con 0x5628060b7460 2024-03-28T18:16:20.138 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:16:20.135+0000 7f5b59800640 1 --2- 172.21.15.22:0/496947253 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5628060ae0f0 0x5628060b6be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T18:16:20.139 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:16:20.135+0000 7f5b58e00640 1 --2- 172.21.15.22:0/496947253 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5628060aedd0 0x5628060b3f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-28T18:16:20.139 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:16:20.135+0000 7f5b5a200640 1 --2- 172.21.15.22:0/496947253 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5628060b7460 0x5628060b44b0 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7f5b4c006010 tx=0x7f5b4c009a60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f399a2068855f563 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:16:20.139 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:16:20.135+0000 7f5b3f400640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-03-28T18:16:20.139 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:16:20.135+0000 7f5b3f400640 1 -- 172.21.15.22:0/496947253 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b4c028060 con 0x5628060b7460 2024-03-28T18:16:20.139 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:16:20.135+0000 7f5b3f400640 1 -- 172.21.15.22:0/496947253 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(118..118 src has 1..118) v4 ==== 10217+0+0 (secure 0 0 0) 0x7f5b4c005430 con 0x5628060b7460 2024-03-28T18:16:20.139 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:16:20.135+0000 7f5b5ee15740 1 -- 172.21.15.22:0/496947253 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f5b4c004d20 con 0x5628060b7460 2024-03-28T18:16:20.139 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:16:20.136+0000 7f5b3f400640 1 -- 172.21.15.22:0/496947253 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=174) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5b4c02f050 con 0x5628060b7460 2024-03-28T18:16:20.139 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:16:20.136+0000 7f5b5ee15740 10 client.16776 fetch_fsmap learned FSMap version 174 2024-03-28T18:16:20.139 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:16:20.136+0000 7f5b5ee15740 1 -- 172.21.15.22:0/496947253 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=174,monmap=4+}) v3 -- 0x562805d263f0 con 0x5628060b7460 2024-03-28T18:16:20.139 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:16:20.136+0000 7f5b3f400640 1 -- 172.21.15.22:0/496947253 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f5b4c004d20 con 0x5628060b7460 2024-03-28T18:16:20.139 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:16:20.136+0000 7f5b3f400640 1 -- 172.21.15.22:0/496947253 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 174) v1 ==== 6784+0+0 (secure 0 0 0) 0x7f5b4c022030 con 0x5628060b7460 2024-03-28T18:16:20.139 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:16:20.136+0000 7f5b5ee15740 10 client.16776 fetch_fsmap finished waiting for FSMap version 174 2024-03-28T18:16:20.139 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:16:20.136+0000 7f5b5ee15740 10 client.16776 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-28T18:16:20.139 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:16:20.136+0000 7f5b5ee15740 20 client.16776 populate_metadata read hostname 'smithi022' 2024-03-28T18:16:20.140 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:16:20.136+0000 7f5b5ee15740 1 --2- 172.21.15.22:0/496947253 >> [v2:172.21.15.79:6838/2441542443,v1:172.21.15.79:6839/2441542443] conn(0x5628060c7370 0x5628060c9760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:16:20.140 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:16:20.136+0000 7f5b5ee15740 4 client.16776 mds_command: new command op to 34553 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-28T18:16:20.140 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:16:20.136+0000 7f5b5ee15740 1 -- 172.21.15.22:0/496947253 --> [v2:172.21.15.79:6838/2441542443,v1:172.21.15.79:6839/2441542443] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56280601f790 con 0x5628060c7370 2024-03-28T18:16:20.140 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:16:20.137+0000 7f5b59800640 1 --2- 172.21.15.22:0/496947253 >> [v2:172.21.15.79:6838/2441542443,v1:172.21.15.79:6839/2441542443] conn(0x5628060c7370 0x5628060c9760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:16:20.141 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:16:20.138+0000 7f5b59800640 1 --2- 172.21.15.22:0/496947253 >> [v2:172.21.15.79:6838/2441542443,v1:172.21.15.79:6839/2441542443] conn(0x5628060c7370 0x5628060c9760 crc :-1 s=READY pgs=268 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ad32e24f9a86d854 server_cookie=fe6ee2bd4a185a66 in_seq=0 out_seq=0 2024-03-28T18:16:20.141 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:16:20.138+0000 7f5b3f400640 10 client.16776 ms_handle_connect on v2:172.21.15.79:6838/2441542443 2024-03-28T18:16:20.141 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:16:20.139+0000 7f5b3f400640 1 -- 172.21.15.22:0/496947253 <== mds.0 v2:172.21.15.79:6838/2441542443 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x56280601f790 con 0x5628060c7370 2024-03-28T18:16:20.141 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:16:20.139+0000 7f5b3f400640 10 client.16776 handle_command_reply: tid=0 2024-03-28T18:16:20.141 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:16:20.139+0000 7f5b5ee15740 1 client.16776 shutdown 2024-03-28T18:16:20.141 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:16:20.139+0000 7f5b5ee15740 1 -- 172.21.15.22:0/496947253 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5628060b7460 msgr2=0x5628060b44b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:16:20.141 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:16:20.139+0000 7f5b5ee15740 1 --2- 172.21.15.22:0/496947253 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5628060b7460 0x5628060b44b0 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7f5b4c006010 tx=0x7f5b4c009a60 comp rx=0 tx=0).stop 2024-03-28T18:16:20.141 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:16:20.139+0000 7f5b5ee15740 1 -- 172.21.15.22:0/496947253 shutdown_connections 2024-03-28T18:16:20.141 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:16:20.139+0000 7f5b5ee15740 1 --2- 172.21.15.22:0/496947253 >> [v2:172.21.15.79:6838/2441542443,v1:172.21.15.79:6839/2441542443] conn(0x5628060c7370 0x5628060c9760 crc :-1 s=READY pgs=268 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:16:20.141 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:16:20.139+0000 7f5b5ee15740 1 --2- 172.21.15.22:0/496947253 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5628060b7460 0x5628060b44b0 unknown :-1 s=CLOSED pgs=773 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:16:20.141 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:16:20.140+0000 7f5b5ee15740 1 --2- 172.21.15.22:0/496947253 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5628060aedd0 0x5628060b3f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:16:20.142 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:16:20.140+0000 7f5b5ee15740 1 --2- 172.21.15.22:0/496947253 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5628060ae0f0 0x5628060b6be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:16:20.142 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:16:20.140+0000 7f5b3f400640 0 client.16776 ms_handle_reset on v2:172.21.15.79:6838/2441542443 2024-03-28T18:16:20.142 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:16:20.140+0000 7f5b5ee15740 1 -- 172.21.15.22:0/496947253 >> 172.21.15.22:0/496947253 conn(0x562806075200 msgr2=0x5628060aa890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:16:20.142 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:16:20.140+0000 7f5b5ee15740 1 -- 172.21.15.22:0/496947253 shutdown_connections 2024-03-28T18:16:20.142 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:16:20.140+0000 7f5b5ee15740 1 -- 172.21.15.22:0/496947253 wait complete. 2024-03-28T18:16:20.142 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:16:20.140+0000 7f5b5ee15740 20 client.16776 trim_cache size 0 max 16384 2024-03-28T18:16:20.157 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:16:20.157+0000 7f5b5ee15740 1 Processor -- start 2024-03-28T18:16:20.158 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:16:20.157+0000 7f5b5ee15740 1 -- start start 2024-03-28T18:16:20.158 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:16:20.158+0000 7f5b5ee15740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5628060ae0f0 0x5628060b2f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:16:20.158 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:16:20.158+0000 7f5b5ee15740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5628060aedd0 0x5628060b6630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:16:20.158 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:16:20.158+0000 7f5b5ee15740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5628060b7460 0x5628060b6b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:16:20.158 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:16:20.158+0000 7f5b5ee15740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b541aeb80 con 0x5628060aedd0 2024-03-28T18:16:20.158 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:16:20.158+0000 7f5b5ee15740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b541b11c0 con 0x5628060b7460 2024-03-28T18:16:20.158 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:16:20.158+0000 7f5b5ee15740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b541af1e0 con 0x5628060ae0f0 2024-03-28T18:16:20.158 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:16:20.158+0000 7f5b58e00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5628060aedd0 0x5628060b6630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:16:20.158 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:16:20.158+0000 7f5b58e00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5628060aedd0 0x5628060b6630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:43306/0 (socket says 172.21.15.22:43306) 2024-03-28T18:16:20.158 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:16:20.158+0000 7f5b58e00640 1 -- 172.21.15.22:0/3615009037 learned_addr learned my addr 172.21.15.22:0/3615009037 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-28T18:16:20.160 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:16:20.158+0000 7f5b59800640 1 --2- 172.21.15.22:0/3615009037 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5628060ae0f0 0x5628060b2f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:16:20.160 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:16:20.158+0000 7f5b58e00640 1 -- 172.21.15.22:0/3615009037 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5628060ae0f0 msgr2=0x5628060b2f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T18:16:20.160 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:16:20.158+0000 7f5b5a200640 1 --2- 172.21.15.22:0/3615009037 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5628060b7460 0x5628060b6b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:16:20.160 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:16:20.158+0000 7f5b58e00640 1 --2- 172.21.15.22:0/3615009037 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5628060ae0f0 0x5628060b2f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:16:20.160 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:16:20.158+0000 7f5b58e00640 1 -- 172.21.15.22:0/3615009037 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5628060b7460 msgr2=0x5628060b6b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T18:16:20.160 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:16:20.158+0000 7f5b58e00640 1 --2- 172.21.15.22:0/3615009037 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5628060b7460 0x5628060b6b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:16:20.160 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:16:20.158+0000 7f5b58e00640 1 -- 172.21.15.22:0/3615009037 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562805ce60f0 con 0x5628060aedd0 2024-03-28T18:16:20.160 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:16:20.159+0000 7f5b59800640 1 --2- 172.21.15.22:0/3615009037 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5628060ae0f0 0x5628060b2f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T18:16:20.161 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:16:20.159+0000 7f5b5a200640 1 --2- 172.21.15.22:0/3615009037 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5628060b7460 0x5628060b6b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T18:16:20.161 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:16:20.159+0000 7f5b58e00640 1 --2- 172.21.15.22:0/3615009037 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5628060aedd0 0x5628060b6630 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x7f5b40000c00 tx=0x7f5b40005fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6687aa650f6aa19c server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:16:20.161 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:16:20.159+0000 7f5b4a000640 1 -- 172.21.15.22:0/3615009037 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b400bd1e0 con 0x5628060aedd0 2024-03-28T18:16:20.161 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:16:20.159+0000 7f5b4a000640 1 -- 172.21.15.22:0/3615009037 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f5b40004c70 con 0x5628060aedd0 2024-03-28T18:16:20.161 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:16:20.159+0000 7f5b4a000640 1 -- 172.21.15.22:0/3615009037 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b400bd000 con 0x5628060aedd0 2024-03-28T18:16:20.161 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:16:20.160+0000 7f5b5ee15740 1 -- 172.21.15.22:0/3615009037 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5628060aedd0 msgr2=0x5628060b6630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:16:20.161 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:16:20.160+0000 7f5b5ee15740 1 --2- 172.21.15.22:0/3615009037 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5628060aedd0 0x5628060b6630 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x7f5b40000c00 tx=0x7f5b40005fd0 comp rx=0 tx=0).stop 2024-03-28T18:16:20.161 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:16:20.160+0000 7f5b5ee15740 1 -- 172.21.15.22:0/3615009037 shutdown_connections 2024-03-28T18:16:20.161 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:16:20.160+0000 7f5b5ee15740 1 --2- 172.21.15.22:0/3615009037 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5628060b7460 0x5628060b6b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:16:20.161 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:16:20.160+0000 7f5b5ee15740 1 --2- 172.21.15.22:0/3615009037 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5628060aedd0 0x5628060b6630 unknown :-1 s=CLOSED pgs=774 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:16:20.161 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:16:20.160+0000 7f5b5ee15740 1 --2- 172.21.15.22:0/3615009037 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5628060ae0f0 0x5628060b2f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:16:20.161 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:16:20.160+0000 7f5b5ee15740 1 -- 172.21.15.22:0/3615009037 >> 172.21.15.22:0/3615009037 conn(0x562806075200 msgr2=0x5628060b2840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:16:20.162 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:16:20.160+0000 7f5b5ee15740 1 -- 172.21.15.22:0/3615009037 shutdown_connections 2024-03-28T18:16:20.162 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:16:20.160+0000 7f5b5ee15740 1 -- 172.21.15.22:0/3615009037 wait complete. 2024-03-28T18:16:20.162 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:16:20.160+0000 7f5b5ee15740 1 -- start start 2024-03-28T18:16:20.162 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:16:20.160+0000 7f5b5ee15740 1 Processor -- start 2024-03-28T18:16:20.165 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:16:20.161+0000 7f5b5ee15740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5628060ae0f0 0x5628060c2380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:16:20.165 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:16:20.161+0000 7f5b5ee15740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5628060aedd0 0x5628060c28c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:16:20.165 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:16:20.161+0000 7f5b5ee15740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5628060b7460 0x5628060bd740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:16:20.165 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:16:20.161+0000 7f5b5ee15740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b541af840 con 0x5628060b7460 2024-03-28T18:16:20.165 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:16:20.161+0000 7f5b5ee15740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b541b0b60 con 0x5628060ae0f0 2024-03-28T18:16:20.166 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:16:20.161+0000 7f5b5ee15740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b541afea0 con 0x5628060aedd0 2024-03-28T18:16:20.166 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:16:20.161+0000 7f5b5ee15740 10 client.0 _ll_register_callbacks cb 0x7f5b4c009ce0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-28T18:16:20.166 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:16:20.161+0000 7f5b5a200640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5628060b7460 0x5628060bd740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:16:20.166 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:16:20.161+0000 7f5b5a200640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5628060b7460 0x5628060bd740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:43308/0 (socket says 172.21.15.22:43308) 2024-03-28T18:16:20.166 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:16:20.161+0000 7f5b5a200640 1 -- 172.21.15.22:0/1827733085 learned_addr learned my addr 172.21.15.22:0/1827733085 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-28T18:16:20.166 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:16:20.161+0000 7f5b58e00640 1 --2- 172.21.15.22:0/1827733085 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5628060aedd0 0x5628060c28c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:16:20.166 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:16:20.161+0000 7f5b59800640 1 --2- 172.21.15.22:0/1827733085 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5628060ae0f0 0x5628060c2380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:16:20.166 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:16:20.162+0000 7f5b5a200640 1 -- 172.21.15.22:0/1827733085 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5628060aedd0 msgr2=0x5628060c28c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T18:16:20.166 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:16:20.162+0000 7f5b5a200640 1 --2- 172.21.15.22:0/1827733085 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5628060aedd0 0x5628060c28c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:16:20.166 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:16:20.162+0000 7f5b5a200640 1 -- 172.21.15.22:0/1827733085 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5628060ae0f0 msgr2=0x5628060c2380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T18:16:20.166 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:16:20.162+0000 7f5b5a200640 1 --2- 172.21.15.22:0/1827733085 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5628060ae0f0 0x5628060c2380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:16:20.166 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:16:20.162+0000 7f5b5a200640 1 -- 172.21.15.22:0/1827733085 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5628060bdcb0 con 0x5628060b7460 2024-03-28T18:16:20.166 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:16:20.162+0000 7f5b59800640 1 --2- 172.21.15.22:0/1827733085 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5628060ae0f0 0x5628060c2380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T18:16:20.167 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:16:20.162+0000 7f5b5a200640 1 --2- 172.21.15.22:0/1827733085 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5628060b7460 0x5628060bd740 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 crypto rx=0x7f5b4c018850 tx=0x7f5b4c009e10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f855452a6b0e8ea6 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:16:20.167 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:16:20.162+0000 7f5b3f400640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-03-28T18:16:20.167 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:16:20.162+0000 7f5b3f400640 1 -- 172.21.15.22:0/1827733085 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b4c028070 con 0x5628060b7460 2024-03-28T18:16:20.167 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:16:20.162+0000 7f5b3f400640 1 -- 172.21.15.22:0/1827733085 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(118..118 src has 1..118) v4 ==== 10217+0+0 (secure 0 0 0) 0x7f5b4c01ab20 con 0x5628060b7460 2024-03-28T18:16:20.167 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:16:20.162+0000 7f5b5ee15740 1 -- 172.21.15.22:0/1827733085 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f5b40004c70 con 0x5628060b7460 2024-03-28T18:16:20.167 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:16:20.163+0000 7f5b3f400640 1 -- 172.21.15.22:0/1827733085 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=174) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5b4c016040 con 0x5628060b7460 2024-03-28T18:16:20.167 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:16:20.163+0000 7f5b5ee15740 10 client.16782 fetch_fsmap learned FSMap version 174 2024-03-28T18:16:20.167 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:16:20.163+0000 7f5b5ee15740 1 -- 172.21.15.22:0/1827733085 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=174,monmap=4+}) v3 -- 0x5628061232f0 con 0x5628060b7460 2024-03-28T18:16:20.167 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:16:20.163+0000 7f5b3f400640 1 -- 172.21.15.22:0/1827733085 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f5b40004c70 con 0x5628060b7460 2024-03-28T18:16:20.167 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:16:20.163+0000 7f5b3f400640 1 -- 172.21.15.22:0/1827733085 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 174) v1 ==== 6784+0+0 (secure 0 0 0) 0x7f5b4c022040 con 0x5628060b7460 2024-03-28T18:16:20.167 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:16:20.163+0000 7f5b5ee15740 10 client.16782 fetch_fsmap finished waiting for FSMap version 174 2024-03-28T18:16:20.167 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:16:20.163+0000 7f5b5ee15740 10 client.16782 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-28T18:16:20.167 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:16:20.163+0000 7f5b5ee15740 20 client.16782 populate_metadata read hostname 'smithi022' 2024-03-28T18:16:20.167 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:16:20.163+0000 7f5b5ee15740 1 --2- 172.21.15.22:0/1827733085 >> [v2:172.21.15.79:6838/2441542443,v1:172.21.15.79:6839/2441542443] conn(0x5628061239b0 0x562806125dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:16:20.167 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:16:20.163+0000 7f5b5ee15740 4 client.16782 mds_command: new command op to 34553 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-28T18:16:20.168 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:16:20.163+0000 7f5b5ee15740 1 -- 172.21.15.22:0/1827733085 --> [v2:172.21.15.79:6838/2441542443,v1:172.21.15.79:6839/2441542443] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x562806019ae0 con 0x5628061239b0 2024-03-28T18:16:20.168 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:16:20.164+0000 7f5b59800640 1 --2- 172.21.15.22:0/1827733085 >> [v2:172.21.15.79:6838/2441542443,v1:172.21.15.79:6839/2441542443] conn(0x5628061239b0 0x562806125dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:16:20.168 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:16:20.165+0000 7f5b59800640 1 --2- 172.21.15.22:0/1827733085 >> [v2:172.21.15.79:6838/2441542443,v1:172.21.15.79:6839/2441542443] conn(0x5628061239b0 0x562806125dd0 crc :-1 s=READY pgs=269 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5cc12525a5d894ba server_cookie=25fe7f7a36ccf189 in_seq=0 out_seq=0 2024-03-28T18:16:20.168 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:16:20.165+0000 7f5b3f400640 10 client.16782 ms_handle_connect on v2:172.21.15.79:6838/2441542443 2024-03-28T18:16:20.168 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:16:20.165+0000 7f5b3f400640 1 -- 172.21.15.22:0/1827733085 <== mds.0 v2:172.21.15.79:6838/2441542443 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x562806019ae0 con 0x5628061239b0 2024-03-28T18:16:20.168 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:16:20.165+0000 7f5b3f400640 10 client.16782 handle_command_reply: tid=0 2024-03-28T18:16:20.168 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:16:20.165+0000 7f5b5ee15740 1 client.16782 shutdown 2024-03-28T18:16:20.168 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:16:20.165+0000 7f5b5ee15740 1 -- 172.21.15.22:0/1827733085 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5628060b7460 msgr2=0x5628060bd740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:16:20.168 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:16:20.165+0000 7f5b5ee15740 1 --2- 172.21.15.22:0/1827733085 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5628060b7460 0x5628060bd740 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 crypto rx=0x7f5b4c018850 tx=0x7f5b4c009e10 comp rx=0 tx=0).stop 2024-03-28T18:16:20.168 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:16:20.165+0000 7f5b5ee15740 1 -- 172.21.15.22:0/1827733085 shutdown_connections 2024-03-28T18:16:20.168 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:16:20.165+0000 7f5b5ee15740 1 --2- 172.21.15.22:0/1827733085 >> [v2:172.21.15.79:6838/2441542443,v1:172.21.15.79:6839/2441542443] conn(0x5628061239b0 0x562806125dd0 crc :-1 s=READY pgs=269 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:16:20.168 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:16:20.165+0000 7f5b5ee15740 1 --2- 172.21.15.22:0/1827733085 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5628060b7460 0x5628060bd740 unknown :-1 s=CLOSED pgs=775 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:16:20.168 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:16:20.165+0000 7f5b5ee15740 1 --2- 172.21.15.22:0/1827733085 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5628060aedd0 0x5628060c28c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:16:20.168 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:16:20.165+0000 7f5b5ee15740 1 --2- 172.21.15.22:0/1827733085 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5628060ae0f0 0x5628060c2380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:16:20.168 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:16:20.165+0000 7f5b3f400640 0 client.16782 ms_handle_reset on v2:172.21.15.79:6838/2441542443 2024-03-28T18:16:20.169 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:16:20.165+0000 7f5b5ee15740 1 -- 172.21.15.22:0/1827733085 >> 172.21.15.22:0/1827733085 conn(0x562806075200 msgr2=0x5628060a9f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:16:20.169 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:16:20.166+0000 7f5b5ee15740 1 -- 172.21.15.22:0/1827733085 shutdown_connections 2024-03-28T18:16:20.169 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:16:20.166+0000 7f5b5ee15740 1 -- 172.21.15.22:0/1827733085 wait complete. 2024-03-28T18:16:20.169 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:16:20.166+0000 7f5b5ee15740 20 client.16782 trim_cache size 0 max 16384 2024-03-28T18:16:20.169 INFO:teuthology.orchestra.run.smithi022.stdout:{ 2024-03-28T18:16:20.169 INFO:teuthology.orchestra.run.smithi022.stdout: "status": "IDLE (waiting for more scrubs)", 2024-03-28T18:16:20.169 INFO:teuthology.orchestra.run.smithi022.stdout: "scrubs": { 2024-03-28T18:16:20.169 INFO:teuthology.orchestra.run.smithi022.stdout: "4bbbd61d-29a3-4bf6-a4f2-d5783d585de3": { 2024-03-28T18:16:20.169 INFO:teuthology.orchestra.run.smithi022.stdout: "path": "/", 2024-03-28T18:16:20.169 INFO:teuthology.orchestra.run.smithi022.stdout: "tag": "4bbbd61d-29a3-4bf6-a4f2-d5783d585de3", 2024-03-28T18:16:20.169 INFO:teuthology.orchestra.run.smithi022.stdout: "options": "recursive,force" 2024-03-28T18:16:20.170 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-03-28T18:16:20.170 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-03-28T18:16:20.170 INFO:teuthology.orchestra.run.smithi022.stdout:} 2024-03-28T18:16:20.170 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:16:20.166+0000 7f5b5ac00640 1 -- 172.21.15.22:0/3645100612 >> [v2:172.21.15.22:6800/1357365082,v1:172.21.15.22:6801/1357365082] conn(0x7f5b380a2970 msgr2=0x7f5b380a4e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:16:20.171 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:16:20.166+0000 7f5b5ac00640 1 --2- 172.21.15.22:0/3645100612 >> [v2:172.21.15.22:6800/1357365082,v1:172.21.15.22:6801/1357365082] conn(0x7f5b380a2970 0x7f5b380a4e30 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f5b44005fd0 tx=0x7f5b4400bf90 comp rx=0 tx=0).stop 2024-03-28T18:16:20.171 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:16:20.166+0000 7f5b5ac00640 1 -- 172.21.15.22:0/3645100612 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5b540743f0 msgr2=0x7f5b5419a550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:16:20.171 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:16:20.166+0000 7f5b5ac00640 1 --2- 172.21.15.22:0/3645100612 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5b540743f0 0x7f5b5419a550 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 crypto rx=0x7f5b40009230 tx=0x7f5b400045e0 comp rx=0 tx=0).stop 2024-03-28T18:16:20.171 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:16:20.166+0000 7f5b5ac00640 1 -- 172.21.15.22:0/3645100612 shutdown_connections 2024-03-28T18:16:20.171 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:16:20.166+0000 7f5b5ac00640 1 --2- 172.21.15.22:0/3645100612 >> [v2:172.21.15.22:6800/1357365082,v1:172.21.15.22:6801/1357365082] conn(0x7f5b380a2970 0x7f5b380a4e30 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:16:20.171 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:16:20.166+0000 7f5b5ac00640 1 --2- 172.21.15.22:0/3645100612 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f5b5410dd40 0x7f5b5419e8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:16:20.171 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:16:20.166+0000 7f5b5ac00640 1 --2- 172.21.15.22:0/3645100612 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5b540743f0 0x7f5b5419a550 unknown :-1 s=CLOSED pgs=771 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:16:20.171 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:16:20.166+0000 7f5b5ac00640 1 --2- 172.21.15.22:0/3645100612 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5b54073ad0 0x7f5b5419a010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:16:20.171 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:16:20.166+0000 7f5b5ac00640 1 -- 172.21.15.22:0/3645100612 >> 172.21.15.22:0/3645100612 conn(0x7f5b540fd010 msgr2=0x7f5b540fdd80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:16:20.171 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:16:20.167+0000 7f5b5ac00640 1 -- 172.21.15.22:0/3645100612 shutdown_connections 2024-03-28T18:16:20.171 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:16:20.167+0000 7f5b5ac00640 1 -- 172.21.15.22:0/3645100612 wait complete. 2024-03-28T18:16:20.183 INFO:tasks.cephfs.filesystem:scrub status for tag:4bbbd61d-29a3-4bf6-a4f2-d5783d585de3 - {'path': '/', 'tag': '4bbbd61d-29a3-4bf6-a4f2-d5783d585de3', 'options': 'recursive,force'} 2024-03-28T18:16:20.674 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:16:20 smithi134 ceph-mon[35037]: pgmap v291: 97 pgs: 97 active+clean; 118 MiB data, 795 MiB used, 1.0 TiB / 1.0 TiB avail; 121 KiB/s rd, 33 KiB/s wr, 42 op/s 2024-03-28T18:16:20.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:16:20 smithi079 ceph-mon[35312]: pgmap v291: 97 pgs: 97 active+clean; 118 MiB data, 795 MiB used, 1.0 TiB / 1.0 TiB avail; 121 KiB/s rd, 33 KiB/s wr, 42 op/s 2024-03-28T18:16:20.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:16:20 smithi022 ceph-mon[31985]: pgmap v291: 97 pgs: 97 active+clean; 118 MiB data, 795 MiB used, 1.0 TiB / 1.0 TiB avail; 121 KiB/s rd, 33 KiB/s wr, 42 op/s 2024-03-28T18:16:22.674 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:16:22 smithi134 ceph-mon[35037]: pgmap v292: 97 pgs: 97 active+clean; 118 MiB data, 795 MiB used, 1.0 TiB / 1.0 TiB avail; 101 KiB/s rd, 27 KiB/s wr, 35 op/s 2024-03-28T18:16:22.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:16:22 smithi079 ceph-mon[35312]: pgmap v292: 97 pgs: 97 active+clean; 118 MiB data, 795 MiB used, 1.0 TiB / 1.0 TiB avail; 101 KiB/s rd, 27 KiB/s wr, 35 op/s 2024-03-28T18:16:22.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:16:22 smithi022 ceph-mon[31985]: pgmap v292: 97 pgs: 97 active+clean; 118 MiB data, 795 MiB used, 1.0 TiB / 1.0 TiB avail; 101 KiB/s rd, 27 KiB/s wr, 35 op/s 2024-03-28T18:16:24.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:16:24 smithi134 ceph-mon[35037]: pgmap v293: 97 pgs: 97 active+clean; 118 MiB data, 795 MiB used, 1.0 TiB / 1.0 TiB avail; 101 KiB/s rd, 27 KiB/s wr, 35 op/s 2024-03-28T18:16:24.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:16:24 smithi079 ceph-mon[35312]: pgmap v293: 97 pgs: 97 active+clean; 118 MiB data, 795 MiB used, 1.0 TiB / 1.0 TiB avail; 101 KiB/s rd, 27 KiB/s wr, 35 op/s 2024-03-28T18:16:24.781 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:16:24 smithi022 ceph-mon[31985]: pgmap v293: 97 pgs: 97 active+clean; 118 MiB data, 795 MiB used, 1.0 TiB / 1.0 TiB avail; 101 KiB/s rd, 27 KiB/s wr, 35 op/s 2024-03-28T18:16:24.782 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:16:24 smithi022 systemd[1]: ceph-fd220748-ed2b-11ee-95d0-87774f69a715@mds.j.service: Scheduled restart job, restart counter is at 5. 2024-03-28T18:16:24.782 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:16:24 smithi022 systemd[1]: Stopped Ceph mds.j for fd220748-ed2b-11ee-95d0-87774f69a715. 2024-03-28T18:16:24.782 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:16:24 smithi022 systemd[1]: ceph-fd220748-ed2b-11ee-95d0-87774f69a715@mds.j.service: Consumed 1.893s CPU time. 2024-03-28T18:16:24.782 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:16:24 smithi022 systemd[1]: Starting Ceph mds.j for fd220748-ed2b-11ee-95d0-87774f69a715... 2024-03-28T18:16:25.282 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:16:24 smithi022 podman[95479]: 2024-03-28 18:16:24.864823346 +0000 UTC m=+0.037914538 image pull c664afe56bb01db4f9afa9532c20598e201e664f2e9ba2be50db0d52674a26d2 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc 2024-03-28T18:16:25.282 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:16:24 smithi022 podman[95479]: 2024-03-28 18:16:24.993141722 +0000 UTC m=+0.166232930 container create 586c5ee5ba0672ecb1067f5e9a181cbe4defa1f65c015a63ef9f411f5cb3faea (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.build-date=20240319, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True) 2024-03-28T18:16:25.782 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:16:25 smithi022 podman[95479]: 2024-03-28 18:16:25.43859852 +0000 UTC m=+0.611689798 container init 586c5ee5ba0672ecb1067f5e9a181cbe4defa1f65c015a63ef9f411f5cb3faea (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.build-date=20240319, maintainer=Guillaume Abrioux , ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS) 2024-03-28T18:16:25.783 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:16:25 smithi022 podman[95479]: 2024-03-28 18:16:25.449168026 +0000 UTC m=+0.622259244 container start 586c5ee5ba0672ecb1067f5e9a181cbe4defa1f65c015a63ef9f411f5cb3faea (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, io.buildah.version=1.33.5, org.label-schema.build-date=20240319, maintainer=Guillaume Abrioux , ceph=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.vendor=CentOS) 2024-03-28T18:16:25.783 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:16:25 smithi022 ceph-mds[95498]: -- 172.21.15.22:0/2277677762 <== mon.0 v2:172.21.15.22:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x556fb7a89a40 con 0x556fb7a5cc00 2024-03-28T18:16:25.783 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:16:25 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[95494]: starting mds.j at 2024-03-28T18:16:25.783 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:16:25 smithi022 bash[95479]: 586c5ee5ba0672ecb1067f5e9a181cbe4defa1f65c015a63ef9f411f5cb3faea 2024-03-28T18:16:25.783 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:16:25 smithi022 systemd[1]: Started Ceph mds.j for fd220748-ed2b-11ee-95d0-87774f69a715. 2024-03-28T18:16:26.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:16:26 smithi134 ceph-mon[35037]: pgmap v294: 97 pgs: 97 active+clean; 118 MiB data, 795 MiB used, 1.0 TiB / 1.0 TiB avail; 103 KiB/s rd, 27 KiB/s wr, 39 op/s 2024-03-28T18:16:26.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:16:26 smithi134 ceph-mon[35037]: daemon mds.j restarted 2024-03-28T18:16:26.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:16:26 smithi134 ceph-mon[35037]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-03-28T18:16:26.676 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:16:26 smithi134 ceph-mon[35037]: osdmap e119: 12 total, 12 up, 12 in 2024-03-28T18:16:26.676 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:16:26 smithi134 ceph-mon[35037]: mds.? [v2:172.21.15.22:6834/810826586,v1:172.21.15.22:6835/810826586] up:boot 2024-03-28T18:16:26.676 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:16:26 smithi134 ceph-mon[35037]: Standby daemon mds.d assigned to filesystem cephfs as rank 1 2024-03-28T18:16:26.676 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:16:26 smithi134 ceph-mon[35037]: fsmap cephfs:4/5 {0=b=up:active,2=e=up:active,3=k=up:active,4=i=up:active} 5 up:standby-replay 1 up:standby, 1 failed 2024-03-28T18:16:26.676 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:16:26 smithi134 ceph-mon[35037]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-03-28T18:16:26.676 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:16:26 smithi134 ceph-mon[35037]: fsmap cephfs:5/5 4 up:standby-replay 1 up:standby 1 up:replay 4 up:active 2024-03-28T18:16:26.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:16:26 smithi079 ceph-mon[35312]: pgmap v294: 97 pgs: 97 active+clean; 118 MiB data, 795 MiB used, 1.0 TiB / 1.0 TiB avail; 103 KiB/s rd, 27 KiB/s wr, 39 op/s 2024-03-28T18:16:26.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:16:26 smithi079 ceph-mon[35312]: daemon mds.j restarted 2024-03-28T18:16:26.683 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:16:26 smithi079 ceph-mon[35312]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-03-28T18:16:26.683 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:16:26 smithi079 ceph-mon[35312]: osdmap e119: 12 total, 12 up, 12 in 2024-03-28T18:16:26.683 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:16:26 smithi079 ceph-mon[35312]: mds.? [v2:172.21.15.22:6834/810826586,v1:172.21.15.22:6835/810826586] up:boot 2024-03-28T18:16:26.683 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:16:26 smithi079 ceph-mon[35312]: Standby daemon mds.d assigned to filesystem cephfs as rank 1 2024-03-28T18:16:26.683 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:16:26 smithi079 ceph-mon[35312]: fsmap cephfs:4/5 {0=b=up:active,2=e=up:active,3=k=up:active,4=i=up:active} 5 up:standby-replay 1 up:standby, 1 failed 2024-03-28T18:16:26.683 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:16:26 smithi079 ceph-mon[35312]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-03-28T18:16:26.683 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:16:26 smithi079 ceph-mon[35312]: fsmap cephfs:5/5 4 up:standby-replay 1 up:standby 1 up:replay 4 up:active 2024-03-28T18:16:26.686 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:16:26 smithi022 ceph-mon[31985]: pgmap v294: 97 pgs: 97 active+clean; 118 MiB data, 795 MiB used, 1.0 TiB / 1.0 TiB avail; 103 KiB/s rd, 27 KiB/s wr, 39 op/s 2024-03-28T18:16:26.686 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:16:26 smithi022 ceph-mon[31985]: daemon mds.j restarted 2024-03-28T18:16:26.686 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:16:26 smithi022 ceph-mon[31985]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-03-28T18:16:26.687 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:16:26 smithi022 ceph-mon[31985]: osdmap e119: 12 total, 12 up, 12 in 2024-03-28T18:16:26.687 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:16:26 smithi022 ceph-mon[31985]: mds.? [v2:172.21.15.22:6834/810826586,v1:172.21.15.22:6835/810826586] up:boot 2024-03-28T18:16:26.687 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:16:26 smithi022 ceph-mon[31985]: Standby daemon mds.d assigned to filesystem cephfs as rank 1 2024-03-28T18:16:26.687 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:16:26 smithi022 ceph-mon[31985]: fsmap cephfs:4/5 {0=b=up:active,2=e=up:active,3=k=up:active,4=i=up:active} 5 up:standby-replay 1 up:standby, 1 failed 2024-03-28T18:16:26.687 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:16:26 smithi022 ceph-mon[31985]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-03-28T18:16:26.687 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:16:26 smithi022 ceph-mon[31985]: fsmap cephfs:5/5 4 up:standby-replay 1 up:standby 1 up:replay 4 up:active 2024-03-28T18:16:27.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:16:27 smithi022 ceph-mon[31985]: mds.? [v2:172.21.15.22:6836/3255804289,v1:172.21.15.22:6837/3255804289] up:resolve 2024-03-28T18:16:27.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:16:27 smithi022 ceph-mon[31985]: fsmap cephfs:5/5 4 up:standby-replay 1 up:standby 1 up:resolve 4 up:active 2024-03-28T18:16:27.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:16:27 smithi134 ceph-mon[35037]: mds.? [v2:172.21.15.22:6836/3255804289,v1:172.21.15.22:6837/3255804289] up:resolve 2024-03-28T18:16:27.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:16:27 smithi134 ceph-mon[35037]: fsmap cephfs:5/5 4 up:standby-replay 1 up:standby 1 up:resolve 4 up:active 2024-03-28T18:16:27.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:16:27 smithi079 ceph-mon[35312]: mds.? [v2:172.21.15.22:6836/3255804289,v1:172.21.15.22:6837/3255804289] up:resolve 2024-03-28T18:16:27.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:16:27 smithi079 ceph-mon[35312]: fsmap cephfs:5/5 4 up:standby-replay 1 up:standby 1 up:resolve 4 up:active 2024-03-28T18:16:28.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:16:28 smithi022 ceph-mon[31985]: pgmap v296: 97 pgs: 97 active+clean; 118 MiB data, 795 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 10 op/s 2024-03-28T18:16:28.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:16:28 smithi022 ceph-mon[31985]: reconnect by client.35222 172.21.15.22:0/3729116781 after 0.00399995 2024-03-28T18:16:28.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:16:28 smithi022 ceph-mon[31985]: mds.? [v2:172.21.15.22:6836/3255804289,v1:172.21.15.22:6837/3255804289] up:reconnect 2024-03-28T18:16:28.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:16:28 smithi022 ceph-mon[31985]: fsmap cephfs:5/5 4 up:standby-replay 1 up:standby 1 up:reconnect 4 up:active 2024-03-28T18:16:28.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:16:28 smithi134 ceph-mon[35037]: pgmap v296: 97 pgs: 97 active+clean; 118 MiB data, 795 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 10 op/s 2024-03-28T18:16:28.925 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:16:28 smithi134 ceph-mon[35037]: reconnect by client.35222 172.21.15.22:0/3729116781 after 0.00399995 2024-03-28T18:16:28.925 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:16:28 smithi134 ceph-mon[35037]: mds.? [v2:172.21.15.22:6836/3255804289,v1:172.21.15.22:6837/3255804289] up:reconnect 2024-03-28T18:16:28.925 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:16:28 smithi134 ceph-mon[35037]: fsmap cephfs:5/5 4 up:standby-replay 1 up:standby 1 up:reconnect 4 up:active 2024-03-28T18:16:28.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:16:28 smithi079 ceph-mon[35312]: pgmap v296: 97 pgs: 97 active+clean; 118 MiB data, 795 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 10 op/s 2024-03-28T18:16:28.933 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:16:28 smithi079 ceph-mon[35312]: reconnect by client.35222 172.21.15.22:0/3729116781 after 0.00399995 2024-03-28T18:16:28.933 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:16:28 smithi079 ceph-mon[35312]: mds.? [v2:172.21.15.22:6836/3255804289,v1:172.21.15.22:6837/3255804289] up:reconnect 2024-03-28T18:16:28.933 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:16:28 smithi079 ceph-mon[35312]: fsmap cephfs:5/5 4 up:standby-replay 1 up:standby 1 up:reconnect 4 up:active 2024-03-28T18:16:29.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:16:29 smithi134 ceph-mon[35037]: mds.? [v2:172.21.15.22:6836/3255804289,v1:172.21.15.22:6837/3255804289] up:rejoin 2024-03-28T18:16:29.925 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:16:29 smithi134 ceph-mon[35037]: fsmap cephfs:5/5 4 up:standby-replay 1 up:standby 1 up:rejoin 4 up:active 2024-03-28T18:16:29.925 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:16:29 smithi134 ceph-mon[35037]: daemon mds.d is now active in filesystem cephfs as rank 1 2024-03-28T18:16:29.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:16:29 smithi079 ceph-mon[35312]: mds.? [v2:172.21.15.22:6836/3255804289,v1:172.21.15.22:6837/3255804289] up:rejoin 2024-03-28T18:16:29.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:16:29 smithi079 ceph-mon[35312]: fsmap cephfs:5/5 4 up:standby-replay 1 up:standby 1 up:rejoin 4 up:active 2024-03-28T18:16:29.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:16:29 smithi079 ceph-mon[35312]: daemon mds.d is now active in filesystem cephfs as rank 1 2024-03-28T18:16:30.032 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:16:29 smithi022 ceph-mon[31985]: mds.? [v2:172.21.15.22:6836/3255804289,v1:172.21.15.22:6837/3255804289] up:rejoin 2024-03-28T18:16:30.032 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:16:29 smithi022 ceph-mon[31985]: fsmap cephfs:5/5 4 up:standby-replay 1 up:standby 1 up:rejoin 4 up:active 2024-03-28T18:16:30.032 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:16:29 smithi022 ceph-mon[31985]: daemon mds.d is now active in filesystem cephfs as rank 1 2024-03-28T18:16:30.033 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:16:29 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[95280]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-2145-g3be9a5c9/rpm/el8/BUILD/ceph-19.0.0-2145-g3be9a5c9/src/mds/Locker.cc: In function 'void Locker::set_xlocks_done(MutationImpl*, bool)' thread 7fcdcc600700 time 2024-03-28T18:16:29.646253+0000 2024-03-28T18:16:30.034 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:16:29 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[95280]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-2145-g3be9a5c9/rpm/el8/BUILD/ceph-19.0.0-2145-g3be9a5c9/src/mds/Locker.cc: 626: FAILED ceph_assert(obj->is_auth()) 2024-03-28T18:16:30.034 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:16:29 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[95280]: ceph version 19.0.0-2145-g3be9a5c9 (3be9a5c9bc793e11e2800a8c0c696e8b46742033) squid (dev) 2024-03-28T18:16:30.034 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:16:29 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[95280]: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x135) [0x7fcdd7f854a1] 2024-03-28T18:16:30.034 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:16:29 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[95280]: 2: /usr/lib64/ceph/libceph-common.so.2(+0x2d2667) [0x7fcdd7f85667] 2024-03-28T18:16:30.034 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:16:29 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[95280]: 3: (Locker::set_xlocks_done(MutationImpl*, bool)+0x493) [0x55d8d1685223] 2024-03-28T18:16:30.035 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:16:29 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[95280]: 4: (Server::early_reply(boost::intrusive_ptr const&, CInode*, CDentry*)+0x3e6) [0x55d8d14ff5d6] 2024-03-28T18:16:30.035 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:16:29 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[95280]: 5: (Server::journal_and_reply(boost::intrusive_ptr const&, CInode*, CDentry*, LogEvent*, MDSLogContextBase*)+0xc1) [0x55d8d14ffda1] 2024-03-28T18:16:30.035 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:16:29 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[95280]: 6: (Server::handle_client_mkdir(boost::intrusive_ptr const&)+0x5ba) [0x55d8d15356aa] 2024-03-28T18:16:30.035 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:16:29 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[95280]: 7: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0x733) [0x55d8d1552233] 2024-03-28T18:16:30.035 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:16:29 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[95280]: 8: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x55d8d1554197] 2024-03-28T18:16:30.035 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:16:29 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[95280]: 9: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x55d8d145311b] 2024-03-28T18:16:30.035 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:16:29 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[95280]: 10: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x55d8d14536bc] 2024-03-28T18:16:30.036 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:16:29 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[95280]: 11: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bf) [0x55d8d143d1ff] 2024-03-28T18:16:30.036 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:16:29 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[95280]: 12: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x478) [0x7fcdd8217488] 2024-03-28T18:16:30.036 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:16:29 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[95280]: 13: (DispatchQueue::entry()+0x51f) [0x7fcdd821460f] 2024-03-28T18:16:30.036 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:16:29 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[95280]: 14: (DispatchQueue::DispatchThread::entry()+0x11) [0x7fcdd82e7e81] 2024-03-28T18:16:30.036 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:16:29 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[95280]: 15: /lib64/libpthread.so.0(+0x81ca) [0x7fcdd6d021ca] 2024-03-28T18:16:30.036 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:16:29 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[95280]: 16: clone() 2024-03-28T18:16:30.037 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:16:29 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[95280]: 2024-03-28T18:16:29.647+0000 7fcdcc600700 -1 /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-2145-g3be9a5c9/rpm/el8/BUILD/ceph-19.0.0-2145-g3be9a5c9/src/mds/Locker.cc: In function 'void Locker::set_xlocks_done(MutationImpl*, bool)' thread 7fcdcc600700 time 2024-03-28T18:16:29.646253+0000 2024-03-28T18:16:30.037 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:16:29 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[95280]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-2145-g3be9a5c9/rpm/el8/BUILD/ceph-19.0.0-2145-g3be9a5c9/src/mds/Locker.cc: 626: FAILED ceph_assert(obj->is_auth()) 2024-03-28T18:16:30.037 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:16:29 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[95280]: 2024-03-28T18:16:30.037 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:16:29 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[95280]: ceph version 19.0.0-2145-g3be9a5c9 (3be9a5c9bc793e11e2800a8c0c696e8b46742033) squid (dev) 2024-03-28T18:16:30.037 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:16:29 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[95280]: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x135) [0x7fcdd7f854a1] 2024-03-28T18:16:30.037 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:16:29 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[95280]: 2: /usr/lib64/ceph/libceph-common.so.2(+0x2d2667) [0x7fcdd7f85667] 2024-03-28T18:16:30.037 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:16:29 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[95280]: 3: (Locker::set_xlocks_done(MutationImpl*, bool)+0x493) [0x55d8d1685223] 2024-03-28T18:16:30.037 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:16:29 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[95280]: 4: (Server::early_reply(boost::intrusive_ptr const&, CInode*, CDentry*)+0x3e6) [0x55d8d14ff5d6] 2024-03-28T18:16:30.037 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:16:29 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[95280]: 5: (Server::journal_and_reply(boost::intrusive_ptr const&, CInode*, CDentry*, LogEvent*, MDSLogContextBase*)+0xc1) [0x55d8d14ffda1] 2024-03-28T18:16:30.037 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:16:29 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[95280]: 6: (Server::handle_client_mkdir(boost::intrusive_ptr const&)+0x5ba) [0x55d8d15356aa] 2024-03-28T18:16:30.037 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:16:29 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[95280]: 7: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0x733) [0x55d8d1552233] 2024-03-28T18:16:30.037 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:16:29 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[95280]: 8: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x55d8d1554197] 2024-03-28T18:16:30.038 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:16:29 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[95280]: 9: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x55d8d145311b] 2024-03-28T18:16:30.038 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:16:29 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[95280]: 10: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x55d8d14536bc] 2024-03-28T18:16:30.038 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:16:29 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[95280]: 11: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bf) [0x55d8d143d1ff] 2024-03-28T18:16:30.038 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:16:29 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[95280]: 12: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x478) [0x7fcdd8217488] 2024-03-28T18:16:30.038 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:16:29 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[95280]: 13: (DispatchQueue::entry()+0x51f) [0x7fcdd821460f] 2024-03-28T18:16:30.038 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:16:29 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[95280]: 14: (DispatchQueue::DispatchThread::entry()+0x11) [0x7fcdd82e7e81] 2024-03-28T18:16:30.038 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:16:29 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[95280]: 15: /lib64/libpthread.so.0(+0x81ca) [0x7fcdd6d021ca] 2024-03-28T18:16:30.038 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:16:29 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[95280]: 16: clone() 2024-03-28T18:16:30.038 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:16:29 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[95280]: 2024-03-28T18:16:30.038 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:16:29 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[95280]: *** Caught signal (Aborted) ** 2024-03-28T18:16:30.038 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:16:29 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[95280]: in thread 7fcdcc600700 thread_name:ms_dispatch 2024-03-28T18:16:30.038 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:16:29 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[95280]: ceph version 19.0.0-2145-g3be9a5c9 (3be9a5c9bc793e11e2800a8c0c696e8b46742033) squid (dev) 2024-03-28T18:16:30.038 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:16:29 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[95280]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7fcdd6d0cd20] 2024-03-28T18:16:30.038 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:16:29 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[95280]: 2: gsignal() 2024-03-28T18:16:30.039 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:16:29 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[95280]: 3: abort() 2024-03-28T18:16:30.039 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:16:29 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[95280]: 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x18f) [0x7fcdd7f854fb] 2024-03-28T18:16:30.039 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:16:29 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[95280]: 5: /usr/lib64/ceph/libceph-common.so.2(+0x2d2667) [0x7fcdd7f85667] 2024-03-28T18:16:30.039 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:16:29 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[95280]: 6: (Locker::set_xlocks_done(MutationImpl*, bool)+0x493) [0x55d8d1685223] 2024-03-28T18:16:30.039 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:16:29 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[95280]: 7: (Server::early_reply(boost::intrusive_ptr const&, CInode*, CDentry*)+0x3e6) [0x55d8d14ff5d6] 2024-03-28T18:16:30.039 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:16:29 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[95280]: 8: (Server::journal_and_reply(boost::intrusive_ptr const&, CInode*, CDentry*, LogEvent*, MDSLogContextBase*)+0xc1) [0x55d8d14ffda1] 2024-03-28T18:16:30.040 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:16:29 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[95280]: 9: (Server::handle_client_mkdir(boost::intrusive_ptr const&)+0x5ba) [0x55d8d15356aa] 2024-03-28T18:16:30.040 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:16:29 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[95280]: 10: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0x733) [0x55d8d1552233] 2024-03-28T18:16:30.040 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:16:29 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[95280]: 11: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x55d8d1554197] 2024-03-28T18:16:30.040 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:16:29 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[95280]: 12: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x55d8d145311b] 2024-03-28T18:16:30.040 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:16:29 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[95280]: 13: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x55d8d14536bc] 2024-03-28T18:16:30.040 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:16:29 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[95280]: 14: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bf) [0x55d8d143d1ff] 2024-03-28T18:16:30.040 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:16:29 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[95280]: 15: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x478) [0x7fcdd8217488] 2024-03-28T18:16:30.041 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:16:29 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[95280]: 16: (DispatchQueue::entry()+0x51f) [0x7fcdd821460f] 2024-03-28T18:16:30.041 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:16:29 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[95280]: 17: (DispatchQueue::DispatchThread::entry()+0x11) [0x7fcdd82e7e81] 2024-03-28T18:16:30.041 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:16:29 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[95280]: 18: /lib64/libpthread.so.0(+0x81ca) [0x7fcdd6d021ca] 2024-03-28T18:16:30.041 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:16:29 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[95280]: 19: clone() 2024-03-28T18:16:30.041 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:16:29 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[95280]: 2024-03-28T18:16:29.650+0000 7fcdcc600700 -1 *** Caught signal (Aborted) ** 2024-03-28T18:16:30.041 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:16:29 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[95280]: in thread 7fcdcc600700 thread_name:ms_dispatch 2024-03-28T18:16:30.042 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:16:29 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[95280]: 2024-03-28T18:16:30.042 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:16:29 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[95280]: ceph version 19.0.0-2145-g3be9a5c9 (3be9a5c9bc793e11e2800a8c0c696e8b46742033) squid (dev) 2024-03-28T18:16:30.042 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:16:29 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[95280]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7fcdd6d0cd20] 2024-03-28T18:16:30.042 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:16:29 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[95280]: 2: gsignal() 2024-03-28T18:16:30.042 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:16:29 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[95280]: 3: abort() 2024-03-28T18:16:30.042 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:16:29 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[95280]: 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x18f) [0x7fcdd7f854fb] 2024-03-28T18:16:30.043 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:16:29 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[95280]: 5: /usr/lib64/ceph/libceph-common.so.2(+0x2d2667) [0x7fcdd7f85667] 2024-03-28T18:16:30.043 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:16:29 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[95280]: 6: (Locker::set_xlocks_done(MutationImpl*, bool)+0x493) [0x55d8d1685223] 2024-03-28T18:16:30.043 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:16:29 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[95280]: 7: (Server::early_reply(boost::intrusive_ptr const&, CInode*, CDentry*)+0x3e6) [0x55d8d14ff5d6] 2024-03-28T18:16:30.043 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:16:29 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[95280]: 8: (Server::journal_and_reply(boost::intrusive_ptr const&, CInode*, CDentry*, LogEvent*, MDSLogContextBase*)+0xc1) [0x55d8d14ffda1] 2024-03-28T18:16:30.043 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:16:29 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[95280]: 9: (Server::handle_client_mkdir(boost::intrusive_ptr const&)+0x5ba) [0x55d8d15356aa] 2024-03-28T18:16:30.044 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:16:29 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[95280]: 10: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0x733) [0x55d8d1552233] 2024-03-28T18:16:30.044 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:16:29 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[95280]: 11: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x55d8d1554197] 2024-03-28T18:16:30.044 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:16:29 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[95280]: 12: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x55d8d145311b] 2024-03-28T18:16:30.044 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:16:29 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[95280]: 13: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x55d8d14536bc] 2024-03-28T18:16:30.044 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:16:29 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[95280]: 14: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bf) [0x55d8d143d1ff] 2024-03-28T18:16:30.044 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:16:29 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[95280]: 15: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x478) [0x7fcdd8217488] 2024-03-28T18:16:30.045 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:16:29 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[95280]: 16: (DispatchQueue::entry()+0x51f) [0x7fcdd821460f] 2024-03-28T18:16:30.045 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:16:29 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[95280]: 17: (DispatchQueue::DispatchThread::entry()+0x11) [0x7fcdd82e7e81] 2024-03-28T18:16:30.045 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:16:29 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[95280]: 18: /lib64/libpthread.so.0(+0x81ca) [0x7fcdd6d021ca] 2024-03-28T18:16:30.045 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:16:29 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[95280]: 19: clone() 2024-03-28T18:16:30.045 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:16:29 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[95280]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-03-28T18:16:30.045 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:16:29 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[95280]: 2024-03-28T18:16:30.046 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:16:29 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[95280]: -1> 2024-03-28T18:16:29.647+0000 7fcdcc600700 -1 /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-2145-g3be9a5c9/rpm/el8/BUILD/ceph-19.0.0-2145-g3be9a5c9/src/mds/Locker.cc: In function 'void Locker::set_xlocks_done(MutationImpl*, bool)' thread 7fcdcc600700 time 2024-03-28T18:16:29.646253+0000 2024-03-28T18:16:30.046 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:16:29 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[95280]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-2145-g3be9a5c9/rpm/el8/BUILD/ceph-19.0.0-2145-g3be9a5c9/src/mds/Locker.cc: 626: FAILED ceph_assert(obj->is_auth()) 2024-03-28T18:16:30.046 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:16:29 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[95280]: 2024-03-28T18:16:30.046 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:16:29 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[95280]: ceph version 19.0.0-2145-g3be9a5c9 (3be9a5c9bc793e11e2800a8c0c696e8b46742033) squid (dev) 2024-03-28T18:16:30.046 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:16:29 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[95280]: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x135) [0x7fcdd7f854a1] 2024-03-28T18:16:30.046 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:16:29 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[95280]: 2: /usr/lib64/ceph/libceph-common.so.2(+0x2d2667) [0x7fcdd7f85667] 2024-03-28T18:16:30.048 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:16:29 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[95280]: 3: (Locker::set_xlocks_done(MutationImpl*, bool)+0x493) [0x55d8d1685223] 2024-03-28T18:16:30.048 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:16:29 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[95280]: 4: (Server::early_reply(boost::intrusive_ptr const&, CInode*, CDentry*)+0x3e6) [0x55d8d14ff5d6] 2024-03-28T18:16:30.049 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:16:29 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[95280]: 5: (Server::journal_and_reply(boost::intrusive_ptr const&, CInode*, CDentry*, LogEvent*, MDSLogContextBase*)+0xc1) [0x55d8d14ffda1] 2024-03-28T18:16:30.049 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:16:29 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[95280]: 6: (Server::handle_client_mkdir(boost::intrusive_ptr const&)+0x5ba) [0x55d8d15356aa] 2024-03-28T18:16:30.049 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:16:29 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[95280]: 7: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0x733) [0x55d8d1552233] 2024-03-28T18:16:30.049 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:16:29 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[95280]: 8: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x55d8d1554197] 2024-03-28T18:16:30.049 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:16:29 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[95280]: 9: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x55d8d145311b] 2024-03-28T18:16:30.050 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:16:29 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[95280]: 10: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x55d8d14536bc] 2024-03-28T18:16:30.050 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:16:29 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[95280]: 11: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bf) [0x55d8d143d1ff] 2024-03-28T18:16:30.050 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:16:29 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[95280]: 12: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x478) [0x7fcdd8217488] 2024-03-28T18:16:30.050 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:16:29 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[95280]: 13: (DispatchQueue::entry()+0x51f) [0x7fcdd821460f] 2024-03-28T18:16:30.050 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:16:29 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[95280]: 14: (DispatchQueue::DispatchThread::entry()+0x11) [0x7fcdd82e7e81] 2024-03-28T18:16:30.050 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:16:29 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[95280]: 15: /lib64/libpthread.so.0(+0x81ca) [0x7fcdd6d021ca] 2024-03-28T18:16:30.051 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:16:29 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[95280]: 16: clone() 2024-03-28T18:16:30.051 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:16:29 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[95280]: 2024-03-28T18:16:30.051 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:16:29 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[95280]: 0> 2024-03-28T18:16:29.650+0000 7fcdcc600700 -1 *** Caught signal (Aborted) ** 2024-03-28T18:16:30.051 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:16:29 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[95280]: in thread 7fcdcc600700 thread_name:ms_dispatch 2024-03-28T18:16:30.051 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:16:29 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[95280]: 2024-03-28T18:16:30.052 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:16:29 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[95280]: ceph version 19.0.0-2145-g3be9a5c9 (3be9a5c9bc793e11e2800a8c0c696e8b46742033) squid (dev) 2024-03-28T18:16:30.052 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:16:29 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[95280]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7fcdd6d0cd20] 2024-03-28T18:16:30.052 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:16:29 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[95280]: 2: gsignal() 2024-03-28T18:16:30.052 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:16:29 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[95280]: 3: abort() 2024-03-28T18:16:30.052 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:16:29 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[95280]: 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x18f) [0x7fcdd7f854fb] 2024-03-28T18:16:30.052 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:16:29 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[95280]: 5: /usr/lib64/ceph/libceph-common.so.2(+0x2d2667) [0x7fcdd7f85667] 2024-03-28T18:16:30.053 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:16:29 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[95280]: 6: (Locker::set_xlocks_done(MutationImpl*, bool)+0x493) [0x55d8d1685223] 2024-03-28T18:16:30.053 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:16:29 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[95280]: 7: (Server::early_reply(boost::intrusive_ptr const&, CInode*, CDentry*)+0x3e6) [0x55d8d14ff5d6] 2024-03-28T18:16:30.053 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:16:29 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[95280]: 8: (Server::journal_and_reply(boost::intrusive_ptr const&, CInode*, CDentry*, LogEvent*, MDSLogContextBase*)+0xc1) [0x55d8d14ffda1] 2024-03-28T18:16:30.053 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:16:29 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[95280]: 9: (Server::handle_client_mkdir(boost::intrusive_ptr const&)+0x5ba) [0x55d8d15356aa] 2024-03-28T18:16:30.053 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:16:29 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[95280]: 10: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0x733) [0x55d8d1552233] 2024-03-28T18:16:30.053 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:16:29 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[95280]: 11: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x55d8d1554197] 2024-03-28T18:16:30.054 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:16:29 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[95280]: 12: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x55d8d145311b] 2024-03-28T18:16:30.054 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:16:29 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[95280]: 13: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x55d8d14536bc] 2024-03-28T18:16:30.054 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:16:29 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[95280]: 14: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bf) [0x55d8d143d1ff] 2024-03-28T18:16:30.054 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:16:29 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[95280]: 15: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x478) [0x7fcdd8217488] 2024-03-28T18:16:30.054 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:16:29 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[95280]: 16: (DispatchQueue::entry()+0x51f) [0x7fcdd821460f] 2024-03-28T18:16:30.054 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:16:29 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[95280]: 17: (DispatchQueue::DispatchThread::entry()+0x11) [0x7fcdd82e7e81] 2024-03-28T18:16:30.055 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:16:29 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[95280]: 18: /lib64/libpthread.so.0(+0x81ca) [0x7fcdd6d021ca] 2024-03-28T18:16:30.055 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:16:29 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[95280]: 19: clone() 2024-03-28T18:16:30.055 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:16:29 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[95280]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-03-28T18:16:30.055 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:16:29 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[95280]: 2024-03-28T18:16:30.056 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:16:29 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[95280]: -1> 2024-03-28T18:16:29.647+0000 7fcdcc600700 -1 /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-2145-g3be9a5c9/rpm/el8/BUILD/ceph-19.0.0-2145-g3be9a5c9/src/mds/Locker.cc: In function 'void Locker::set_xlocks_done(MutationImpl*, bool)' thread 7fcdcc600700 time 2024-03-28T18:16:29.646253+0000 2024-03-28T18:16:30.056 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:16:29 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[95280]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-2145-g3be9a5c9/rpm/el8/BUILD/ceph-19.0.0-2145-g3be9a5c9/src/mds/Locker.cc: 626: FAILED ceph_assert(obj->is_auth()) 2024-03-28T18:16:30.056 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:16:29 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[95280]: 2024-03-28T18:16:30.056 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:16:29 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[95280]: ceph version 19.0.0-2145-g3be9a5c9 (3be9a5c9bc793e11e2800a8c0c696e8b46742033) squid (dev) 2024-03-28T18:16:30.056 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:16:29 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[95280]: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x135) [0x7fcdd7f854a1] 2024-03-28T18:16:30.056 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:16:29 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[95280]: 2: /usr/lib64/ceph/libceph-common.so.2(+0x2d2667) [0x7fcdd7f85667] 2024-03-28T18:16:30.057 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:16:29 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[95280]: 3: (Locker::set_xlocks_done(MutationImpl*, bool)+0x493) [0x55d8d1685223] 2024-03-28T18:16:30.057 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:16:29 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[95280]: 4: (Server::early_reply(boost::intrusive_ptr const&, CInode*, CDentry*)+0x3e6) [0x55d8d14ff5d6] 2024-03-28T18:16:30.057 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:16:29 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[95280]: 5: (Server::journal_and_reply(boost::intrusive_ptr const&, CInode*, CDentry*, LogEvent*, MDSLogContextBase*)+0xc1) [0x55d8d14ffda1] 2024-03-28T18:16:30.058 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:16:29 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[95280]: 6: (Server::handle_client_mkdir(boost::intrusive_ptr const&)+0x5ba) [0x55d8d15356aa] 2024-03-28T18:16:30.058 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:16:29 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[95280]: 7: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0x733) [0x55d8d1552233] 2024-03-28T18:16:30.058 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:16:29 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[95280]: 8: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x55d8d1554197] 2024-03-28T18:16:30.058 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:16:29 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[95280]: 9: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x55d8d145311b] 2024-03-28T18:16:30.058 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:16:29 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[95280]: 10: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x55d8d14536bc] 2024-03-28T18:16:30.059 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:16:29 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[95280]: 11: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bf) [0x55d8d143d1ff] 2024-03-28T18:16:30.059 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:16:29 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[95280]: 12: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x478) [0x7fcdd8217488] 2024-03-28T18:16:30.059 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:16:29 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[95280]: 13: (DispatchQueue::entry()+0x51f) [0x7fcdd821460f] 2024-03-28T18:16:30.059 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:16:29 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[95280]: 14: (DispatchQueue::DispatchThread::entry()+0x11) [0x7fcdd82e7e81] 2024-03-28T18:16:30.059 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:16:29 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[95280]: 15: /lib64/libpthread.so.0(+0x81ca) [0x7fcdd6d021ca] 2024-03-28T18:16:30.059 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:16:29 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[95280]: 16: clone() 2024-03-28T18:16:30.060 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:16:29 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[95280]: 2024-03-28T18:16:30.060 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:16:29 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[95280]: 0> 2024-03-28T18:16:29.650+0000 7fcdcc600700 -1 *** Caught signal (Aborted) ** 2024-03-28T18:16:30.060 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:16:29 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[95280]: in thread 7fcdcc600700 thread_name:ms_dispatch 2024-03-28T18:16:30.060 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:16:29 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[95280]: 2024-03-28T18:16:30.060 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:16:29 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[95280]: ceph version 19.0.0-2145-g3be9a5c9 (3be9a5c9bc793e11e2800a8c0c696e8b46742033) squid (dev) 2024-03-28T18:16:30.061 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:16:29 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[95280]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7fcdd6d0cd20] 2024-03-28T18:16:30.061 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:16:29 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[95280]: 2: gsignal() 2024-03-28T18:16:30.061 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:16:29 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[95280]: 3: abort() 2024-03-28T18:16:30.061 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:16:29 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[95280]: 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x18f) [0x7fcdd7f854fb] 2024-03-28T18:16:30.061 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:16:29 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[95280]: 5: /usr/lib64/ceph/libceph-common.so.2(+0x2d2667) [0x7fcdd7f85667] 2024-03-28T18:16:30.061 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:16:29 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[95280]: 6: (Locker::set_xlocks_done(MutationImpl*, bool)+0x493) [0x55d8d1685223] 2024-03-28T18:16:30.062 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:16:29 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[95280]: 7: (Server::early_reply(boost::intrusive_ptr const&, CInode*, CDentry*)+0x3e6) [0x55d8d14ff5d6] 2024-03-28T18:16:30.062 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:16:29 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[95280]: 8: (Server::journal_and_reply(boost::intrusive_ptr const&, CInode*, CDentry*, LogEvent*, MDSLogContextBase*)+0xc1) [0x55d8d14ffda1] 2024-03-28T18:16:30.062 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:16:29 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[95280]: 9: (Server::handle_client_mkdir(boost::intrusive_ptr const&)+0x5ba) [0x55d8d15356aa] 2024-03-28T18:16:30.062 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:16:29 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[95280]: 10: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0x733) [0x55d8d1552233] 2024-03-28T18:16:30.063 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:16:29 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[95280]: 11: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x55d8d1554197] 2024-03-28T18:16:30.063 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:16:29 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[95280]: 12: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x55d8d145311b] 2024-03-28T18:16:30.063 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:16:29 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[95280]: 13: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x55d8d14536bc] 2024-03-28T18:16:30.063 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:16:29 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[95280]: 14: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bf) [0x55d8d143d1ff] 2024-03-28T18:16:30.063 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:16:29 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[95280]: 15: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x478) [0x7fcdd8217488] 2024-03-28T18:16:30.063 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:16:29 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[95280]: 16: (DispatchQueue::entry()+0x51f) [0x7fcdd821460f] 2024-03-28T18:16:30.063 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:16:29 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[95280]: 17: (DispatchQueue::DispatchThread::entry()+0x11) [0x7fcdd82e7e81] 2024-03-28T18:16:30.063 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:16:29 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[95280]: 18: /lib64/libpthread.so.0(+0x81ca) [0x7fcdd6d021ca] 2024-03-28T18:16:30.063 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:16:29 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[95280]: 19: clone() 2024-03-28T18:16:30.063 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:16:29 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[95280]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-03-28T18:16:30.063 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:16:29 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d[95280]: 2024-03-28T18:16:30.063 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:16:29 smithi022 podman[95545]: 2024-03-28 18:16:29.784388809 +0000 UTC m=+0.040231766 container died d01401bd73e8f8acca7b0a20d5f885c739882625be4e6a98c6e83b4d80d970fd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_CLEAN=True, ceph=True, org.label-schema.build-date=20240319, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, io.buildah.version=1.33.5) 2024-03-28T18:16:30.782 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:16:30 smithi022 podman[95545]: 2024-03-28 18:16:30.408658442 +0000 UTC m=+0.664501432 container remove d01401bd73e8f8acca7b0a20d5f885c739882625be4e6a98c6e83b4d80d970fd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-d, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.build-date=20240319, org.label-schema.license=GPLv2, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, CEPH_POINT_RELEASE=, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux ) 2024-03-28T18:16:30.782 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:16:30 smithi022 systemd[1]: ceph-fd220748-ed2b-11ee-95d0-87774f69a715@mds.d.service: Main process exited, code=exited, status=134/n/a 2024-03-28T18:16:30.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:16:30 smithi022 ceph-mon[31985]: pgmap v297: 97 pgs: 97 active+clean; 118 MiB data, 795 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 409 B/s wr, 10 op/s 2024-03-28T18:16:30.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:16:30 smithi022 ceph-mon[31985]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-03-28T18:16:30.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:16:30 smithi022 ceph-mon[31985]: Cluster is now healthy 2024-03-28T18:16:30.783 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:16:30 smithi022 ceph-mon[31985]: mds.? [v2:172.21.15.22:6836/3255804289,v1:172.21.15.22:6837/3255804289] up:active 2024-03-28T18:16:30.783 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:16:30 smithi022 ceph-mon[31985]: fsmap cephfs:5 4 up:standby-replay 1 up:standby 5 up:active 2024-03-28T18:16:30.783 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:16:30 smithi022 ceph-mon[31985]: fsmap cephfs:5 5 up:standby-replay 5 up:active 2024-03-28T18:16:30.827 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:16:30 smithi134 ceph-mon[35037]: pgmap v297: 97 pgs: 97 active+clean; 118 MiB data, 795 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 409 B/s wr, 10 op/s 2024-03-28T18:16:30.827 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:16:30 smithi134 ceph-mon[35037]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-03-28T18:16:30.827 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:16:30 smithi134 ceph-mon[35037]: Cluster is now healthy 2024-03-28T18:16:30.827 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:16:30 smithi134 ceph-mon[35037]: mds.? [v2:172.21.15.22:6836/3255804289,v1:172.21.15.22:6837/3255804289] up:active 2024-03-28T18:16:30.827 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:16:30 smithi134 ceph-mon[35037]: fsmap cephfs:5 4 up:standby-replay 1 up:standby 5 up:active 2024-03-28T18:16:30.827 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:16:30 smithi134 ceph-mon[35037]: fsmap cephfs:5 5 up:standby-replay 5 up:active 2024-03-28T18:16:30.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:16:30 smithi079 ceph-mon[35312]: pgmap v297: 97 pgs: 97 active+clean; 118 MiB data, 795 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 409 B/s wr, 10 op/s 2024-03-28T18:16:30.933 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:16:30 smithi079 ceph-mon[35312]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-03-28T18:16:30.933 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:16:30 smithi079 ceph-mon[35312]: Cluster is now healthy 2024-03-28T18:16:30.933 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:16:30 smithi079 ceph-mon[35312]: mds.? [v2:172.21.15.22:6836/3255804289,v1:172.21.15.22:6837/3255804289] up:active 2024-03-28T18:16:30.933 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:16:30 smithi079 ceph-mon[35312]: fsmap cephfs:5 4 up:standby-replay 1 up:standby 5 up:active 2024-03-28T18:16:30.933 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:16:30 smithi079 ceph-mon[35312]: fsmap cephfs:5 5 up:standby-replay 5 up:active 2024-03-28T18:16:31.282 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:16:30 smithi022 systemd[1]: ceph-fd220748-ed2b-11ee-95d0-87774f69a715@mds.d.service: Failed with result 'exit-code'. 2024-03-28T18:16:31.282 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:16:30 smithi022 systemd[1]: ceph-fd220748-ed2b-11ee-95d0-87774f69a715@mds.d.service: Consumed 2.004s CPU time. 2024-03-28T18:16:32.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:16:32 smithi134 ceph-mon[35037]: pgmap v298: 97 pgs: 97 active+clean; 118 MiB data, 795 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 409 B/s wr, 10 op/s 2024-03-28T18:16:32.933 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:16:32 smithi079 ceph-mon[35312]: pgmap v298: 97 pgs: 97 active+clean; 118 MiB data, 795 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 409 B/s wr, 10 op/s 2024-03-28T18:16:33.031 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:16:32 smithi022 ceph-mon[31985]: pgmap v298: 97 pgs: 97 active+clean; 118 MiB data, 795 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 409 B/s wr, 10 op/s 2024-03-28T18:16:34.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:16:34 smithi134 ceph-mon[35037]: pgmap v299: 97 pgs: 97 active+clean; 118 MiB data, 795 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 409 B/s wr, 10 op/s 2024-03-28T18:16:34.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:16:34 smithi079 ceph-mon[35312]: pgmap v299: 97 pgs: 97 active+clean; 118 MiB data, 795 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 409 B/s wr, 10 op/s 2024-03-28T18:16:35.032 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:16:34 smithi022 ceph-mon[31985]: pgmap v299: 97 pgs: 97 active+clean; 118 MiB data, 795 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 409 B/s wr, 10 op/s 2024-03-28T18:16:35.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:16:35 smithi134 ceph-mon[35037]: osdmap e120: 12 total, 12 up, 12 in 2024-03-28T18:16:35.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:16:35 smithi079 ceph-mon[35312]: osdmap e120: 12 total, 12 up, 12 in 2024-03-28T18:16:36.031 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:16:35 smithi022 ceph-mon[31985]: osdmap e120: 12 total, 12 up, 12 in 2024-03-28T18:16:36.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:16:36 smithi134 ceph-mon[35037]: pgmap v301: 97 pgs: 97 active+clean; 118 MiB data, 800 MiB used, 1.0 TiB / 1.0 TiB avail; 116 KiB/s rd, 34 KiB/s wr, 29 op/s 2024-03-28T18:16:36.933 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:16:36 smithi079 ceph-mon[35312]: pgmap v301: 97 pgs: 97 active+clean; 118 MiB data, 800 MiB used, 1.0 TiB / 1.0 TiB avail; 116 KiB/s rd, 34 KiB/s wr, 29 op/s 2024-03-28T18:16:37.032 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:16:36 smithi022 ceph-mon[31985]: pgmap v301: 97 pgs: 97 active+clean; 118 MiB data, 800 MiB used, 1.0 TiB / 1.0 TiB avail; 116 KiB/s rd, 34 KiB/s wr, 29 op/s 2024-03-28T18:16:38.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:16:38 smithi022 ceph-mon[31985]: pgmap v302: 97 pgs: 97 active+clean; 118 MiB data, 800 MiB used, 1.0 TiB / 1.0 TiB avail; 113 KiB/s rd, 33 KiB/s wr, 28 op/s 2024-03-28T18:16:38.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:16:38 smithi134 ceph-mon[35037]: pgmap v302: 97 pgs: 97 active+clean; 118 MiB data, 800 MiB used, 1.0 TiB / 1.0 TiB avail; 113 KiB/s rd, 33 KiB/s wr, 28 op/s 2024-03-28T18:16:38.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:16:38 smithi079 ceph-mon[35312]: pgmap v302: 97 pgs: 97 active+clean; 118 MiB data, 800 MiB used, 1.0 TiB / 1.0 TiB avail; 113 KiB/s rd, 33 KiB/s wr, 28 op/s 2024-03-28T18:16:40.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:16:40 smithi134 ceph-mon[35037]: pgmap v303: 97 pgs: 97 active+clean; 118 MiB data, 800 MiB used, 1.0 TiB / 1.0 TiB avail; 112 KiB/s rd, 33 KiB/s wr, 28 op/s 2024-03-28T18:16:40.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:16:40 smithi079 ceph-mon[35312]: pgmap v303: 97 pgs: 97 active+clean; 118 MiB data, 800 MiB used, 1.0 TiB / 1.0 TiB avail; 112 KiB/s rd, 33 KiB/s wr, 28 op/s 2024-03-28T18:16:41.032 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:16:40 smithi022 ceph-mon[31985]: pgmap v303: 97 pgs: 97 active+clean; 118 MiB data, 800 MiB used, 1.0 TiB / 1.0 TiB avail; 112 KiB/s rd, 33 KiB/s wr, 28 op/s 2024-03-28T18:16:41.532 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:16:41 smithi022 systemd[1]: ceph-fd220748-ed2b-11ee-95d0-87774f69a715@mds.d.service: Scheduled restart job, restart counter is at 6. 2024-03-28T18:16:41.532 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:16:41 smithi022 systemd[1]: Stopped Ceph mds.d for fd220748-ed2b-11ee-95d0-87774f69a715. 2024-03-28T18:16:41.532 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:16:41 smithi022 systemd[1]: ceph-fd220748-ed2b-11ee-95d0-87774f69a715@mds.d.service: Consumed 2.004s CPU time. 2024-03-28T18:16:41.532 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:16:41 smithi022 systemd[1]: ceph-fd220748-ed2b-11ee-95d0-87774f69a715@mds.d.service: Start request repeated too quickly. 2024-03-28T18:16:41.532 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:16:41 smithi022 systemd[1]: ceph-fd220748-ed2b-11ee-95d0-87774f69a715@mds.d.service: Failed with result 'exit-code'. 2024-03-28T18:16:41.532 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 28 18:16:41 smithi022 systemd[1]: Failed to start Ceph mds.d for fd220748-ed2b-11ee-95d0-87774f69a715. 2024-03-28T18:16:42.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:16:42 smithi134 ceph-mon[35037]: pgmap v304: 97 pgs: 97 active+clean; 118 MiB data, 800 MiB used, 1.0 TiB / 1.0 TiB avail; 112 KiB/s rd, 33 KiB/s wr, 28 op/s 2024-03-28T18:16:42.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:16:42 smithi079 ceph-mon[35312]: pgmap v304: 97 pgs: 97 active+clean; 118 MiB data, 800 MiB used, 1.0 TiB / 1.0 TiB avail; 112 KiB/s rd, 33 KiB/s wr, 28 op/s 2024-03-28T18:16:43.031 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:16:42 smithi022 ceph-mon[31985]: pgmap v304: 97 pgs: 97 active+clean; 118 MiB data, 800 MiB used, 1.0 TiB / 1.0 TiB avail; 112 KiB/s rd, 33 KiB/s wr, 28 op/s 2024-03-28T18:16:44.282 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:16:44 smithi022 ceph-mon[31985]: osdmap e121: 12 total, 12 up, 12 in 2024-03-28T18:16:44.282 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:16:44 smithi022 ceph-mon[31985]: pgmap v306: 97 pgs: 97 active+clean; 118 MiB data, 800 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 118 B/s wr, 6 op/s 2024-03-28T18:16:44.425 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:16:44 smithi134 ceph-mon[35037]: osdmap e121: 12 total, 12 up, 12 in 2024-03-28T18:16:44.426 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:16:44 smithi134 ceph-mon[35037]: pgmap v306: 97 pgs: 97 active+clean; 118 MiB data, 800 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 118 B/s wr, 6 op/s 2024-03-28T18:16:44.432 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:16:44 smithi079 ceph-mon[35312]: osdmap e121: 12 total, 12 up, 12 in 2024-03-28T18:16:44.433 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:16:44 smithi079 ceph-mon[35312]: pgmap v306: 97 pgs: 97 active+clean; 118 MiB data, 800 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 118 B/s wr, 6 op/s 2024-03-28T18:16:46.674 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:16:46 smithi134 ceph-mon[35037]: pgmap v307: 97 pgs: 97 active+clean; 118 MiB data, 800 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2024-03-28T18:16:46.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:16:46 smithi079 ceph-mon[35312]: pgmap v307: 97 pgs: 97 active+clean; 118 MiB data, 800 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2024-03-28T18:16:46.781 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:16:46 smithi022 ceph-mon[31985]: pgmap v307: 97 pgs: 97 active+clean; 118 MiB data, 800 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2024-03-28T18:16:48.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:16:48 smithi134 ceph-mon[35037]: pgmap v308: 97 pgs: 97 active+clean; 118 MiB data, 800 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2024-03-28T18:16:48.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:16:48 smithi134 ceph-mon[35037]: Replacing daemon mds.d as rank 1 with standby daemon mds.j 2024-03-28T18:16:48.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:16:48 smithi134 ceph-mon[35037]: MDS daemon mds.d is removed because it is dead or otherwise unavailable. 2024-03-28T18:16:48.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:16:48 smithi134 ceph-mon[35037]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-03-28T18:16:48.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:16:48 smithi134 ceph-mon[35037]: osdmap e122: 12 total, 12 up, 12 in 2024-03-28T18:16:48.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:16:48 smithi134 ceph-mon[35037]: fsmap cephfs:5/5 4 up:standby-replay 1 up:replay 4 up:active 2024-03-28T18:16:48.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:16:48 smithi079 ceph-mon[35312]: pgmap v308: 97 pgs: 97 active+clean; 118 MiB data, 800 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2024-03-28T18:16:48.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:16:48 smithi079 ceph-mon[35312]: Replacing daemon mds.d as rank 1 with standby daemon mds.j 2024-03-28T18:16:48.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:16:48 smithi079 ceph-mon[35312]: MDS daemon mds.d is removed because it is dead or otherwise unavailable. 2024-03-28T18:16:48.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:16:48 smithi079 ceph-mon[35312]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-03-28T18:16:48.683 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:16:48 smithi079 ceph-mon[35312]: osdmap e122: 12 total, 12 up, 12 in 2024-03-28T18:16:48.683 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:16:48 smithi079 ceph-mon[35312]: fsmap cephfs:5/5 4 up:standby-replay 1 up:replay 4 up:active 2024-03-28T18:16:48.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:16:48 smithi022 ceph-mon[31985]: pgmap v308: 97 pgs: 97 active+clean; 118 MiB data, 800 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2024-03-28T18:16:48.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:16:48 smithi022 ceph-mon[31985]: Replacing daemon mds.d as rank 1 with standby daemon mds.j 2024-03-28T18:16:48.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:16:48 smithi022 ceph-mon[31985]: MDS daemon mds.d is removed because it is dead or otherwise unavailable. 2024-03-28T18:16:48.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:16:48 smithi022 ceph-mon[31985]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-03-28T18:16:48.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:16:48 smithi022 ceph-mon[31985]: osdmap e122: 12 total, 12 up, 12 in 2024-03-28T18:16:48.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:16:48 smithi022 ceph-mon[31985]: fsmap cephfs:5/5 4 up:standby-replay 1 up:replay 4 up:active 2024-03-28T18:16:50.184 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-28T18:16:50.184 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-28T18:16:50.434 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:16:50.433+0000 7f8f63200640 1 -- 172.21.15.22:0/4124232835 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8f64073a40 msgr2=0x7f8f64073e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:16:50.434 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:16:50.433+0000 7f8f63200640 1 --2- 172.21.15.22:0/4124232835 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8f64073a40 0x7f8f64073e20 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7f8f54009900 tx=0x7f8f5402bcd0 comp rx=0 tx=0).stop 2024-03-28T18:16:50.434 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:16:50.433+0000 7f8f63200640 1 -- 172.21.15.22:0/4124232835 shutdown_connections 2024-03-28T18:16:50.434 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:16:50.433+0000 7f8f63200640 1 --2- 172.21.15.22:0/4124232835 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8f64112510 0x7f8f64114900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:16:50.434 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:16:50.433+0000 7f8f63200640 1 --2- 172.21.15.22:0/4124232835 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f8f64074360 0x7f8f64111fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:16:50.434 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:16:50.433+0000 7f8f63200640 1 --2- 172.21.15.22:0/4124232835 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8f64073a40 0x7f8f64073e20 unknown :-1 s=CLOSED pgs=778 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:16:50.434 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:16:50.433+0000 7f8f63200640 1 -- 172.21.15.22:0/4124232835 >> 172.21.15.22:0/4124232835 conn(0x7f8f640fd010 msgr2=0x7f8f640ff430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:16:50.434 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:16:50.433+0000 7f8f63200640 1 -- 172.21.15.22:0/4124232835 shutdown_connections 2024-03-28T18:16:50.435 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:16:50.433+0000 7f8f63200640 1 -- 172.21.15.22:0/4124232835 wait complete. 2024-03-28T18:16:50.436 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:16:50.434+0000 7f8f63200640 1 Processor -- start 2024-03-28T18:16:50.436 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:16:50.434+0000 7f8f63200640 1 -- start start 2024-03-28T18:16:50.437 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:16:50.434+0000 7f8f63200640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8f64073a40 0x7f8f6419e440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:16:50.437 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:16:50.434+0000 7f8f63200640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8f64074360 0x7f8f6419e980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:16:50.437 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:16:50.434+0000 7f8f63200640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f8f64112510 0x7f8f641a2cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:16:50.437 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:16:50.434+0000 7f8f63200640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f64117ee0 con 0x7f8f64073a40 2024-03-28T18:16:50.437 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:16:50.434+0000 7f8f63200640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f64117d60 con 0x7f8f64112510 2024-03-28T18:16:50.437 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:16:50.434+0000 7f8f63200640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f64118060 con 0x7f8f64074360 2024-03-28T18:16:50.437 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:16:50.435+0000 7f8f62800640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f8f64112510 0x7f8f641a2cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:16:50.437 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:16:50.435+0000 7f8f62800640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f8f64112510 0x7f8f641a2cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.22:40580/0 (socket says 172.21.15.22:40580) 2024-03-28T18:16:50.437 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:16:50.435+0000 7f8f62800640 1 -- 172.21.15.22:0/113103098 learned_addr learned my addr 172.21.15.22:0/113103098 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-28T18:16:50.437 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:16:50.435+0000 7f8f61e00640 1 --2- 172.21.15.22:0/113103098 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8f64073a40 0x7f8f6419e440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:16:50.437 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:16:50.435+0000 7f8f59400640 1 --2- 172.21.15.22:0/113103098 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8f64074360 0x7f8f6419e980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:16:50.437 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:16:50.436+0000 7f8f62800640 1 -- 172.21.15.22:0/113103098 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8f64074360 msgr2=0x7f8f6419e980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:16:50.437 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:16:50.436+0000 7f8f62800640 1 --2- 172.21.15.22:0/113103098 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8f64074360 0x7f8f6419e980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:16:50.437 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:16:50.436+0000 7f8f62800640 1 -- 172.21.15.22:0/113103098 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8f64073a40 msgr2=0x7f8f6419e440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:16:50.438 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:16:50.436+0000 7f8f62800640 1 --2- 172.21.15.22:0/113103098 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8f64073a40 0x7f8f6419e440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:16:50.438 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:16:50.436+0000 7f8f62800640 1 -- 172.21.15.22:0/113103098 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8f641a33e0 con 0x7f8f64112510 2024-03-28T18:16:50.438 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:16:50.436+0000 7f8f61e00640 1 --2- 172.21.15.22:0/113103098 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8f64073a40 0x7f8f6419e440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-28T18:16:50.438 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:16:50.436+0000 7f8f62800640 1 --2- 172.21.15.22:0/113103098 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f8f64112510 0x7f8f641a2cb0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f8f5c00e9d0 tx=0x7f8f5c00ee90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:16:50.438 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:16:50.436+0000 7f8f5b400640 1 -- 172.21.15.22:0/113103098 <== mon.2 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f5c00cd80 con 0x7f8f64112510 2024-03-28T18:16:50.439 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:16:50.436+0000 7f8f63200640 1 -- 172.21.15.22:0/113103098 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8f641a36d0 con 0x7f8f64112510 2024-03-28T18:16:50.439 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:16:50.437+0000 7f8f63200640 1 -- 172.21.15.22:0/113103098 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8f641aaf60 con 0x7f8f64112510 2024-03-28T18:16:50.439 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:16:50.437+0000 7f8f5b400640 1 -- 172.21.15.22:0/113103098 <== mon.2 v2:172.21.15.79:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f8f5c005020 con 0x7f8f64112510 2024-03-28T18:16:50.439 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:16:50.437+0000 7f8f5b400640 1 -- 172.21.15.22:0/113103098 <== mon.2 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f5c01f060 con 0x7f8f64112510 2024-03-28T18:16:50.440 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:16:50.440+0000 7f8f6b60e740 1 Processor -- start 2024-03-28T18:16:50.441 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:16:50.441+0000 7f8f5b400640 1 -- 172.21.15.22:0/113103098 <== mon.2 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 26) v1 ==== 137421+0+0 (secure 0 0 0) 0x7f8f5c005590 con 0x7f8f64112510 2024-03-28T18:16:50.441 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:16:50.441+0000 7f8f6b60e740 1 -- start start 2024-03-28T18:16:50.443 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:16:50.441+0000 7f8f6b60e740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56364b513dd0 0x56364b512b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:16:50.443 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:16:50.441+0000 7f8f6b60e740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56364b5130f0 0x56364b513570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:16:50.443 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:16:50.441+0000 7f8f6b60e740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56364b51c460 0x56364b51e850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:16:50.443 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:16:50.441+0000 7f8f6b60e740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x56364b477250 con 0x56364b5130f0 2024-03-28T18:16:50.443 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:16:50.441+0000 7f8f6b60e740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x56364b41fa30 con 0x56364b513dd0 2024-03-28T18:16:50.443 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:16:50.441+0000 7f8f6b60e740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x56364b4732f0 con 0x56364b51c460 2024-03-28T18:16:50.443 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:16:50.442+0000 7f8f61e00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56364b5130f0 0x56364b513570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:16:50.444 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:16:50.442+0000 7f8f61e00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56364b5130f0 0x56364b513570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:41774/0 (socket says 172.21.15.22:41774) 2024-03-28T18:16:50.444 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:16:50.442+0000 7f8f61e00640 1 -- 172.21.15.22:0/923553663 learned_addr learned my addr 172.21.15.22:0/923553663 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-28T18:16:50.444 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:16:50.442+0000 7f8f59400640 1 --2- 172.21.15.22:0/923553663 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56364b513dd0 0x56364b512b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:16:50.444 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:16:50.442+0000 7f8f59400640 1 --2- 172.21.15.22:0/923553663 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56364b51c460 0x56364b51e850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:16:50.444 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:16:50.442+0000 7f8f61e00640 1 -- 172.21.15.22:0/923553663 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56364b51c460 msgr2=0x56364b51e850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T18:16:50.444 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:16:50.442+0000 7f8f61e00640 1 --2- 172.21.15.22:0/923553663 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56364b51c460 0x56364b51e850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:16:50.444 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:16:50.442+0000 7f8f61e00640 1 -- 172.21.15.22:0/923553663 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56364b513dd0 msgr2=0x56364b512b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T18:16:50.444 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:16:50.442+0000 7f8f61e00640 1 --2- 172.21.15.22:0/923553663 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56364b513dd0 0x56364b512b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:16:50.444 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:16:50.442+0000 7f8f61e00640 1 -- 172.21.15.22:0/923553663 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56364b40a690 con 0x56364b5130f0 2024-03-28T18:16:50.444 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:16:50.442+0000 7f8f59400640 1 --2- 172.21.15.22:0/923553663 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56364b51c460 0x56364b51e850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T18:16:50.444 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:16:50.442+0000 7f8f5b400640 1 --2- 172.21.15.22:0/113103098 >> [v2:172.21.15.22:6800/1357365082,v1:172.21.15.22:6801/1357365082] conn(0x7f8f480a2970 0x7f8f480a4e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:16:50.444 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:16:50.443+0000 7f8f5b400640 1 -- 172.21.15.22:0/113103098 <== mon.2 v2:172.21.15.79:3300/0 5 ==== osd_map(122..122 src has 1..122) v4 ==== 10389+0+0 (secure 0 0 0) 0x7f8f5c0cf5f0 con 0x7f8f64112510 2024-03-28T18:16:50.444 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:16:50.443+0000 7f8f61e00640 1 --2- 172.21.15.22:0/923553663 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56364b5130f0 0x56364b513570 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 crypto rx=0x7f8f54036040 tx=0x7f8f54004a70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5d34861a51cfb95d server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:16:50.444 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:16:50.443+0000 7f8f59400640 1 --2- 172.21.15.22:0/113103098 >> [v2:172.21.15.22:6800/1357365082,v1:172.21.15.22:6801/1357365082] conn(0x7f8f480a2970 0x7f8f480a4e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:16:50.448 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:16:50.443+0000 7f8f53000640 1 -- 172.21.15.22:0/923553663 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f54045070 con 0x56364b5130f0 2024-03-28T18:16:50.448 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:16:50.443+0000 7f8f53000640 1 -- 172.21.15.22:0/923553663 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f8f5402c670 con 0x56364b5130f0 2024-03-28T18:16:50.448 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:16:50.443+0000 7f8f53000640 1 -- 172.21.15.22:0/923553663 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f540038e0 con 0x56364b5130f0 2024-03-28T18:16:50.449 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:16:50.444+0000 7f8f6b60e740 1 -- 172.21.15.22:0/923553663 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56364b5130f0 msgr2=0x56364b513570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:16:50.449 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:16:50.444+0000 7f8f59400640 1 --2- 172.21.15.22:0/113103098 >> [v2:172.21.15.22:6800/1357365082,v1:172.21.15.22:6801/1357365082] conn(0x7f8f480a2970 0x7f8f480a4e30 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f8f4c009730 tx=0x7f8f4c012040 comp rx=0 tx=0).ready entity=mgr.15522 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:16:50.449 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:16:50.444+0000 7f8f6b60e740 1 --2- 172.21.15.22:0/923553663 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56364b5130f0 0x56364b513570 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 crypto rx=0x7f8f54036040 tx=0x7f8f54004a70 comp rx=0 tx=0).stop 2024-03-28T18:16:50.449 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:16:50.444+0000 7f8f6b60e740 1 -- 172.21.15.22:0/923553663 shutdown_connections 2024-03-28T18:16:50.449 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:16:50.444+0000 7f8f6b60e740 1 --2- 172.21.15.22:0/923553663 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56364b51c460 0x56364b51e850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:16:50.449 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:16:50.444+0000 7f8f6b60e740 1 --2- 172.21.15.22:0/923553663 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56364b5130f0 0x56364b513570 unknown :-1 s=CLOSED pgs=779 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:16:50.449 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:16:50.444+0000 7f8f6b60e740 1 --2- 172.21.15.22:0/923553663 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56364b513dd0 0x56364b512b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:16:50.449 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:16:50.444+0000 7f8f6b60e740 1 -- 172.21.15.22:0/923553663 >> 172.21.15.22:0/923553663 conn(0x56364b4da200 msgr2=0x56364b50db40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:16:50.449 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:16:50.444+0000 7f8f6b60e740 1 -- 172.21.15.22:0/923553663 shutdown_connections 2024-03-28T18:16:50.449 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:16:50.444+0000 7f8f6b60e740 1 -- 172.21.15.22:0/923553663 wait complete. 2024-03-28T18:16:50.449 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:16:50.444+0000 7f8f6b60e740 1 -- start start 2024-03-28T18:16:50.449 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:16:50.445+0000 7f8f6b60e740 1 Processor -- start 2024-03-28T18:16:50.449 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:16:50.445+0000 7f8f6b60e740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56364b5130f0 0x56364b51bb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:16:50.449 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:16:50.445+0000 7f8f6b60e740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56364b513dd0 0x56364b518ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:16:50.449 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:16:50.445+0000 7f8f6b60e740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56364b51c460 0x56364b5193e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:16:50.450 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:16:50.445+0000 7f8f6b60e740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x56364b51f270 con 0x56364b513dd0 2024-03-28T18:16:50.450 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:16:50.445+0000 7f8f6b60e740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x56364b432e80 con 0x56364b51c460 2024-03-28T18:16:50.450 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:16:50.445+0000 7f8f6b60e740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x56364b51f3f0 con 0x56364b5130f0 2024-03-28T18:16:50.450 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:16:50.445+0000 7f8f6b60e740 10 client.0 _ll_register_callbacks cb 0x56364b445130 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-28T18:16:50.450 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:16:50.445+0000 7f8f59400640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56364b513dd0 0x56364b518ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:16:50.450 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:16:50.445+0000 7f8f59400640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56364b513dd0 0x56364b518ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:41788/0 (socket says 172.21.15.22:41788) 2024-03-28T18:16:50.450 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:16:50.445+0000 7f8f59400640 1 -- 172.21.15.22:0/4099608275 learned_addr learned my addr 172.21.15.22:0/4099608275 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-28T18:16:50.450 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:16:50.446+0000 7f8f62800640 1 --2- 172.21.15.22:0/4099608275 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56364b51c460 0x56364b5193e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:16:50.450 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:16:50.446+0000 7f8f61e00640 1 --2- 172.21.15.22:0/4099608275 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56364b5130f0 0x56364b51bb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:16:50.450 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:16:50.446+0000 7f8f59400640 1 -- 172.21.15.22:0/4099608275 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56364b5130f0 msgr2=0x56364b51bb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T18:16:50.450 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:16:50.446+0000 7f8f59400640 1 --2- 172.21.15.22:0/4099608275 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56364b5130f0 0x56364b51bb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:16:50.450 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:16:50.446+0000 7f8f59400640 1 -- 172.21.15.22:0/4099608275 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56364b51c460 msgr2=0x56364b5193e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T18:16:50.450 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:16:50.446+0000 7f8f59400640 1 --2- 172.21.15.22:0/4099608275 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56364b51c460 0x56364b5193e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:16:50.450 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:16:50.446+0000 7f8f59400640 1 -- 172.21.15.22:0/4099608275 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56364b2557d0 con 0x56364b513dd0 2024-03-28T18:16:50.450 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:16:50.446+0000 7f8f61e00640 1 --2- 172.21.15.22:0/4099608275 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56364b5130f0 0x56364b51bb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T18:16:50.450 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:16:50.446+0000 7f8f62800640 1 --2- 172.21.15.22:0/4099608275 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56364b51c460 0x56364b5193e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-28T18:16:50.451 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:16:50.446+0000 7f8f59400640 1 --2- 172.21.15.22:0/4099608275 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56364b513dd0 0x56364b518ea0 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 crypto rx=0x7f8f4c004600 tx=0x7f8f4c004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=975de9a14fb593fe server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:16:50.451 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:16:50.446+0000 7f8f47400640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-03-28T18:16:50.451 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:16:50.447+0000 7f8f47400640 1 -- 172.21.15.22:0/4099608275 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f4c005220 con 0x56364b513dd0 2024-03-28T18:16:50.451 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:16:50.447+0000 7f8f47400640 1 -- 172.21.15.22:0/4099608275 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(122..122 src has 1..122) v4 ==== 10389+0+0 (secure 0 0 0) 0x7f8f4c0053c0 con 0x56364b513dd0 2024-03-28T18:16:50.451 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:16:50.447+0000 7f8f6b60e740 1 -- 172.21.15.22:0/4099608275 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f8f5402c670 con 0x56364b513dd0 2024-03-28T18:16:50.451 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:16:50.447+0000 7f8f47400640 1 -- 172.21.15.22:0/4099608275 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=184) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8f4c0055c0 con 0x56364b513dd0 2024-03-28T18:16:50.451 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:16:50.447+0000 7f8f6b60e740 10 client.16800 fetch_fsmap learned FSMap version 184 2024-03-28T18:16:50.451 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:16:50.447+0000 7f8f6b60e740 1 -- 172.21.15.22:0/4099608275 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=184,monmap=4+}) v3 -- 0x56364b18b3f0 con 0x56364b513dd0 2024-03-28T18:16:50.451 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:16:50.447+0000 7f8f47400640 1 -- 172.21.15.22:0/4099608275 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f8f5402c670 con 0x56364b513dd0 2024-03-28T18:16:50.451 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:16:50.447+0000 7f8f47400640 1 -- 172.21.15.22:0/4099608275 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 184) v1 ==== 6213+0+0 (secure 0 0 0) 0x7f8f4c01f440 con 0x56364b513dd0 2024-03-28T18:16:50.451 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:16:50.448+0000 7f8f6b60e740 10 client.16800 fetch_fsmap finished waiting for FSMap version 184 2024-03-28T18:16:50.451 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:16:50.448+0000 7f8f6b60e740 10 client.16800 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-28T18:16:50.451 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:16:50.448+0000 7f8f6b60e740 20 client.16800 populate_metadata read hostname 'smithi022' 2024-03-28T18:16:50.451 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:16:50.448+0000 7f8f6b60e740 1 --2- 172.21.15.22:0/4099608275 >> [v2:172.21.15.79:6838/2441542443,v1:172.21.15.79:6839/2441542443] conn(0x56364b52c470 0x56364b52e8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:16:50.451 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:16:50.448+0000 7f8f6b60e740 4 client.16800 mds_command: new command op to 34553 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-28T18:16:50.452 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:16:50.448+0000 7f8f6b60e740 1 -- 172.21.15.22:0/4099608275 --> [v2:172.21.15.79:6838/2441542443,v1:172.21.15.79:6839/2441542443] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56364b484790 con 0x56364b52c470 2024-03-28T18:16:50.453 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:16:50.448+0000 7f8f61e00640 1 --2- 172.21.15.22:0/4099608275 >> [v2:172.21.15.79:6838/2441542443,v1:172.21.15.79:6839/2441542443] conn(0x56364b52c470 0x56364b52e8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:16:50.453 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:16:50.449+0000 7f8f61e00640 1 --2- 172.21.15.22:0/4099608275 >> [v2:172.21.15.79:6838/2441542443,v1:172.21.15.79:6839/2441542443] conn(0x56364b52c470 0x56364b52e8f0 crc :-1 s=READY pgs=279 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=606d77afe3e51262 server_cookie=965bf3fc8471225 in_seq=0 out_seq=0 2024-03-28T18:16:50.453 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:16:50.449+0000 7f8f47400640 10 client.16800 ms_handle_connect on v2:172.21.15.79:6838/2441542443 2024-03-28T18:16:50.453 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:16:50.450+0000 7f8f47400640 1 -- 172.21.15.22:0/4099608275 <== mds.0 v2:172.21.15.79:6838/2441542443 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x56364b484790 con 0x56364b52c470 2024-03-28T18:16:50.453 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:16:50.450+0000 7f8f47400640 10 client.16800 handle_command_reply: tid=0 2024-03-28T18:16:50.453 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:16:50.450+0000 7f8f6b60e740 1 client.16800 shutdown 2024-03-28T18:16:50.453 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:16:50.451+0000 7f8f6b60e740 1 -- 172.21.15.22:0/4099608275 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56364b513dd0 msgr2=0x56364b518ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:16:50.453 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:16:50.451+0000 7f8f6b60e740 1 --2- 172.21.15.22:0/4099608275 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56364b513dd0 0x56364b518ea0 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 crypto rx=0x7f8f4c004600 tx=0x7f8f4c004730 comp rx=0 tx=0).stop 2024-03-28T18:16:50.453 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:16:50.451+0000 7f8f6b60e740 1 -- 172.21.15.22:0/4099608275 shutdown_connections 2024-03-28T18:16:50.453 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:16:50.451+0000 7f8f6b60e740 1 --2- 172.21.15.22:0/4099608275 >> [v2:172.21.15.79:6838/2441542443,v1:172.21.15.79:6839/2441542443] conn(0x56364b52c470 0x56364b52e8f0 crc :-1 s=READY pgs=279 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:16:50.453 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:16:50.451+0000 7f8f6b60e740 1 --2- 172.21.15.22:0/4099608275 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56364b51c460 0x56364b5193e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:16:50.453 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:16:50.451+0000 7f8f6b60e740 1 --2- 172.21.15.22:0/4099608275 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56364b513dd0 0x56364b518ea0 unknown :-1 s=CLOSED pgs=780 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:16:50.453 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:16:50.451+0000 7f8f6b60e740 1 --2- 172.21.15.22:0/4099608275 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56364b5130f0 0x56364b51bb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:16:50.453 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:16:50.451+0000 7f8f6b60e740 1 -- 172.21.15.22:0/4099608275 >> 172.21.15.22:0/4099608275 conn(0x56364b4da200 msgr2=0x56364b50f890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:16:50.453 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:16:50.451+0000 7f8f47400640 0 client.16800 ms_handle_reset on v2:172.21.15.79:6838/2441542443 2024-03-28T18:16:50.453 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:16:50.451+0000 7f8f6b60e740 1 -- 172.21.15.22:0/4099608275 shutdown_connections 2024-03-28T18:16:50.454 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:16:50.451+0000 7f8f6b60e740 1 -- 172.21.15.22:0/4099608275 wait complete. 2024-03-28T18:16:50.454 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:16:50.451+0000 7f8f6b60e740 20 client.16800 trim_cache size 0 max 16384 2024-03-28T18:16:50.470 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:16:50.469+0000 7f8f6b60e740 1 Processor -- start 2024-03-28T18:16:50.470 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:16:50.469+0000 7f8f6b60e740 1 -- start start 2024-03-28T18:16:50.470 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:16:50.469+0000 7f8f6b60e740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56364b5130f0 0x56364b517ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:16:50.470 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:16:50.469+0000 7f8f6b60e740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56364b513dd0 0x56364b51b5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:16:50.470 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:16:50.469+0000 7f8f6b60e740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56364b51c460 0x56364b51bb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:16:50.470 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:16:50.469+0000 7f8f6b60e740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f641b5fe0 con 0x56364b513dd0 2024-03-28T18:16:50.470 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:16:50.469+0000 7f8f6b60e740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f641b3340 con 0x56364b51c460 2024-03-28T18:16:50.470 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:16:50.469+0000 7f8f6b60e740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f641b5320 con 0x56364b5130f0 2024-03-28T18:16:50.472 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:16:50.470+0000 7f8f59400640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56364b513dd0 0x56364b51b5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:16:50.472 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:16:50.470+0000 7f8f59400640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56364b513dd0 0x56364b51b5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:41802/0 (socket says 172.21.15.22:41802) 2024-03-28T18:16:50.472 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:16:50.470+0000 7f8f59400640 1 -- 172.21.15.22:0/2562250441 learned_addr learned my addr 172.21.15.22:0/2562250441 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-28T18:16:50.472 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:16:50.470+0000 7f8f61e00640 1 --2- 172.21.15.22:0/2562250441 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56364b5130f0 0x56364b517ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:16:50.472 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:16:50.470+0000 7f8f62800640 1 --2- 172.21.15.22:0/2562250441 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56364b51c460 0x56364b51bb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:16:50.472 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:16:50.470+0000 7f8f59400640 1 -- 172.21.15.22:0/2562250441 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56364b5130f0 msgr2=0x56364b517ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T18:16:50.472 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:16:50.470+0000 7f8f59400640 1 --2- 172.21.15.22:0/2562250441 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56364b5130f0 0x56364b517ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:16:50.472 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:16:50.470+0000 7f8f59400640 1 -- 172.21.15.22:0/2562250441 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56364b51c460 msgr2=0x56364b51bb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T18:16:50.472 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:16:50.470+0000 7f8f59400640 1 --2- 172.21.15.22:0/2562250441 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56364b51c460 0x56364b51bb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:16:50.472 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:16:50.470+0000 7f8f59400640 1 -- 172.21.15.22:0/2562250441 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56364b14b0f0 con 0x56364b513dd0 2024-03-28T18:16:50.472 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:16:50.470+0000 7f8f62800640 1 --2- 172.21.15.22:0/2562250441 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56364b51c460 0x56364b51bb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-28T18:16:50.473 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:16:50.471+0000 7f8f59400640 1 --2- 172.21.15.22:0/2562250441 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56364b513dd0 0x56364b51b5d0 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x7f8f4c012010 tx=0x7f8f4c003f50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=11bf918628d7f697 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:16:50.473 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:16:50.471+0000 7f8f52600640 1 -- 172.21.15.22:0/2562250441 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f4c039070 con 0x56364b513dd0 2024-03-28T18:16:50.473 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:16:50.471+0000 7f8f52600640 1 -- 172.21.15.22:0/2562250441 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f8f4c024070 con 0x56364b513dd0 2024-03-28T18:16:50.473 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:16:50.471+0000 7f8f52600640 1 -- 172.21.15.22:0/2562250441 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f4c020710 con 0x56364b513dd0 2024-03-28T18:16:50.473 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:16:50.471+0000 7f8f6b60e740 1 -- 172.21.15.22:0/2562250441 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56364b513dd0 msgr2=0x56364b51b5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:16:50.473 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:16:50.471+0000 7f8f6b60e740 1 --2- 172.21.15.22:0/2562250441 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56364b513dd0 0x56364b51b5d0 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x7f8f4c012010 tx=0x7f8f4c003f50 comp rx=0 tx=0).stop 2024-03-28T18:16:50.473 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:16:50.471+0000 7f8f6b60e740 1 -- 172.21.15.22:0/2562250441 shutdown_connections 2024-03-28T18:16:50.473 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:16:50.471+0000 7f8f6b60e740 1 --2- 172.21.15.22:0/2562250441 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56364b51c460 0x56364b51bb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:16:50.473 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:16:50.471+0000 7f8f6b60e740 1 --2- 172.21.15.22:0/2562250441 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56364b513dd0 0x56364b51b5d0 unknown :-1 s=CLOSED pgs=781 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:16:50.473 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:16:50.471+0000 7f8f6b60e740 1 --2- 172.21.15.22:0/2562250441 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56364b5130f0 0x56364b517ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:16:50.473 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:16:50.471+0000 7f8f6b60e740 1 -- 172.21.15.22:0/2562250441 >> 172.21.15.22:0/2562250441 conn(0x56364b4da200 msgr2=0x56364b517790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:16:50.476 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:16:50.472+0000 7f8f6b60e740 1 -- 172.21.15.22:0/2562250441 shutdown_connections 2024-03-28T18:16:50.476 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:16:50.472+0000 7f8f6b60e740 1 -- 172.21.15.22:0/2562250441 wait complete. 2024-03-28T18:16:50.476 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:16:50.472+0000 7f8f6b60e740 1 -- start start 2024-03-28T18:16:50.476 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:16:50.472+0000 7f8f6b60e740 1 Processor -- start 2024-03-28T18:16:50.476 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:16:50.473+0000 7f8f6b60e740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56364b5130f0 0x56364b515750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:16:50.476 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:16:50.473+0000 7f8f6b60e740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56364b513dd0 0x56364b515c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:16:50.476 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:16:50.473+0000 7f8f6b60e740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56364b51c460 0x56364b5161d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:16:50.477 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:16:50.473+0000 7f8f6b60e740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f641b4cc0 con 0x56364b5130f0 2024-03-28T18:16:50.477 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:16:50.473+0000 7f8f6b60e740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f641b39a0 con 0x56364b51c460 2024-03-28T18:16:50.477 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:16:50.473+0000 7f8f6b60e740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f641b4000 con 0x56364b513dd0 2024-03-28T18:16:50.477 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:16:50.473+0000 7f8f6b60e740 10 client.0 _ll_register_callbacks cb 0x7f8f4c010f50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-28T18:16:50.477 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:16:50.473+0000 7f8f61e00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56364b5130f0 0x56364b515750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:16:50.477 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:16:50.473+0000 7f8f61e00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56364b5130f0 0x56364b515750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:41806/0 (socket says 172.21.15.22:41806) 2024-03-28T18:16:50.477 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:16:50.473+0000 7f8f61e00640 1 -- 172.21.15.22:0/3756711275 learned_addr learned my addr 172.21.15.22:0/3756711275 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-28T18:16:50.477 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:16:50.473+0000 7f8f62800640 1 --2- 172.21.15.22:0/3756711275 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56364b51c460 0x56364b5161d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:16:50.477 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:16:50.473+0000 7f8f59400640 1 --2- 172.21.15.22:0/3756711275 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56364b513dd0 0x56364b515c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:16:50.477 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:16:50.473+0000 7f8f61e00640 1 -- 172.21.15.22:0/3756711275 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56364b513dd0 msgr2=0x56364b515c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T18:16:50.477 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:16:50.473+0000 7f8f61e00640 1 --2- 172.21.15.22:0/3756711275 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56364b513dd0 0x56364b515c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:16:50.477 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:16:50.473+0000 7f8f61e00640 1 -- 172.21.15.22:0/3756711275 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56364b51c460 msgr2=0x56364b5161d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T18:16:50.477 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:16:50.473+0000 7f8f61e00640 1 --2- 172.21.15.22:0/3756711275 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56364b51c460 0x56364b5161d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:16:50.477 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:16:50.473+0000 7f8f61e00640 1 -- 172.21.15.22:0/3756711275 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56364b586f60 con 0x56364b5130f0 2024-03-28T18:16:50.477 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:16:50.473+0000 7f8f59400640 1 --2- 172.21.15.22:0/3756711275 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56364b513dd0 0x56364b515c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-28T18:16:50.478 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:16:50.474+0000 7f8f61e00640 1 --2- 172.21.15.22:0/3756711275 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56364b5130f0 0x56364b515750 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 crypto rx=0x7f8f54000c00 tx=0x7f8f54005850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8116c169f58f34f3 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:16:50.478 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:16:50.474+0000 7f8f47400640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-03-28T18:16:50.478 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:16:50.474+0000 7f8f47400640 1 -- 172.21.15.22:0/3756711275 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f54056060 con 0x56364b5130f0 2024-03-28T18:16:50.478 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:16:50.474+0000 7f8f47400640 1 -- 172.21.15.22:0/3756711275 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(122..122 src has 1..122) v4 ==== 10389+0+0 (secure 0 0 0) 0x7f8f5402ccc0 con 0x56364b5130f0 2024-03-28T18:16:50.478 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:16:50.474+0000 7f8f6b60e740 1 -- 172.21.15.22:0/3756711275 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f8f4c024070 con 0x56364b5130f0 2024-03-28T18:16:50.478 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:16:50.474+0000 7f8f47400640 1 -- 172.21.15.22:0/3756711275 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=184) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8f5405d050 con 0x56364b5130f0 2024-03-28T18:16:50.478 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:16:50.474+0000 7f8f6b60e740 10 client.16806 fetch_fsmap learned FSMap version 184 2024-03-28T18:16:50.478 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:16:50.474+0000 7f8f6b60e740 1 -- 172.21.15.22:0/3756711275 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=184,monmap=4+}) v3 -- 0x56364b588250 con 0x56364b5130f0 2024-03-28T18:16:50.478 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:16:50.475+0000 7f8f47400640 1 -- 172.21.15.22:0/3756711275 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f8f4c024070 con 0x56364b5130f0 2024-03-28T18:16:50.478 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:16:50.475+0000 7f8f47400640 1 -- 172.21.15.22:0/3756711275 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 184) v1 ==== 6213+0+0 (secure 0 0 0) 0x7f8f54003ab0 con 0x56364b5130f0 2024-03-28T18:16:50.478 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:16:50.475+0000 7f8f6b60e740 10 client.16806 fetch_fsmap finished waiting for FSMap version 184 2024-03-28T18:16:50.478 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:16:50.475+0000 7f8f6b60e740 10 client.16806 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-28T18:16:50.478 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:16:50.475+0000 7f8f6b60e740 20 client.16806 populate_metadata read hostname 'smithi022' 2024-03-28T18:16:50.479 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:16:50.475+0000 7f8f6b60e740 1 --2- 172.21.15.22:0/3756711275 >> [v2:172.21.15.79:6838/2441542443,v1:172.21.15.79:6839/2441542443] conn(0x56364b588760 0x56364b58ab80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:16:50.479 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:16:50.475+0000 7f8f6b60e740 4 client.16806 mds_command: new command op to 34553 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-28T18:16:50.479 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:16:50.475+0000 7f8f6b60e740 1 -- 172.21.15.22:0/3756711275 --> [v2:172.21.15.79:6838/2441542443,v1:172.21.15.79:6839/2441542443] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x56364b47eae0 con 0x56364b588760 2024-03-28T18:16:50.479 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:16:50.476+0000 7f8f59400640 1 --2- 172.21.15.22:0/3756711275 >> [v2:172.21.15.79:6838/2441542443,v1:172.21.15.79:6839/2441542443] conn(0x56364b588760 0x56364b58ab80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:16:50.480 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:16:50.476+0000 7f8f59400640 1 --2- 172.21.15.22:0/3756711275 >> [v2:172.21.15.79:6838/2441542443,v1:172.21.15.79:6839/2441542443] conn(0x56364b588760 0x56364b58ab80 crc :-1 s=READY pgs=280 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c5b75dfa7c1bb535 server_cookie=1197e8d7520fe87e in_seq=0 out_seq=0 2024-03-28T18:16:50.480 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:16:50.476+0000 7f8f47400640 10 client.16806 ms_handle_connect on v2:172.21.15.79:6838/2441542443 2024-03-28T18:16:50.480 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:16:50.477+0000 7f8f47400640 1 -- 172.21.15.22:0/3756711275 <== mds.0 v2:172.21.15.79:6838/2441542443 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x56364b47eae0 con 0x56364b588760 2024-03-28T18:16:50.480 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:16:50.477+0000 7f8f47400640 10 client.16806 handle_command_reply: tid=0 2024-03-28T18:16:50.480 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:16:50.477+0000 7f8f6b60e740 1 client.16806 shutdown 2024-03-28T18:16:50.480 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:16:50.477+0000 7f8f6b60e740 1 -- 172.21.15.22:0/3756711275 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56364b5130f0 msgr2=0x56364b515750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:16:50.481 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:16:50.477+0000 7f8f6b60e740 1 --2- 172.21.15.22:0/3756711275 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56364b5130f0 0x56364b515750 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 crypto rx=0x7f8f54000c00 tx=0x7f8f54005850 comp rx=0 tx=0).stop 2024-03-28T18:16:50.481 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:16:50.477+0000 7f8f6b60e740 1 -- 172.21.15.22:0/3756711275 shutdown_connections 2024-03-28T18:16:50.481 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:16:50.477+0000 7f8f6b60e740 1 --2- 172.21.15.22:0/3756711275 >> [v2:172.21.15.79:6838/2441542443,v1:172.21.15.79:6839/2441542443] conn(0x56364b588760 0x56364b58ab80 crc :-1 s=READY pgs=280 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:16:50.481 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:16:50.477+0000 7f8f6b60e740 1 --2- 172.21.15.22:0/3756711275 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56364b51c460 0x56364b5161d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:16:50.481 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:16:50.477+0000 7f8f6b60e740 1 --2- 172.21.15.22:0/3756711275 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56364b513dd0 0x56364b515c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:16:50.481 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:16:50.477+0000 7f8f6b60e740 1 --2- 172.21.15.22:0/3756711275 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56364b5130f0 0x56364b515750 unknown :-1 s=CLOSED pgs=782 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:16:50.481 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:16:50.477+0000 7f8f47400640 0 client.16806 ms_handle_reset on v2:172.21.15.79:6838/2441542443 2024-03-28T18:16:50.481 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:16:50.477+0000 7f8f6b60e740 1 -- 172.21.15.22:0/3756711275 >> 172.21.15.22:0/3756711275 conn(0x56364b4da200 msgr2=0x56364b518020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:16:50.481 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:16:50.477+0000 7f8f6b60e740 1 -- 172.21.15.22:0/3756711275 shutdown_connections 2024-03-28T18:16:50.481 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:16:50.477+0000 7f8f6b60e740 1 -- 172.21.15.22:0/3756711275 wait complete. 2024-03-28T18:16:50.481 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:16:50.478+0000 7f8f6b60e740 20 client.16806 trim_cache size 0 max 16384 2024-03-28T18:16:50.481 INFO:teuthology.orchestra.run.smithi022.stdout:{ 2024-03-28T18:16:50.482 INFO:teuthology.orchestra.run.smithi022.stdout: "status": "IDLE (waiting for more scrubs)", 2024-03-28T18:16:50.482 INFO:teuthology.orchestra.run.smithi022.stdout: "scrubs": { 2024-03-28T18:16:50.482 INFO:teuthology.orchestra.run.smithi022.stdout: "4bbbd61d-29a3-4bf6-a4f2-d5783d585de3": { 2024-03-28T18:16:50.482 INFO:teuthology.orchestra.run.smithi022.stdout: "path": "/", 2024-03-28T18:16:50.482 INFO:teuthology.orchestra.run.smithi022.stdout: "tag": "4bbbd61d-29a3-4bf6-a4f2-d5783d585de3", 2024-03-28T18:16:50.482 INFO:teuthology.orchestra.run.smithi022.stdout: "options": "recursive,force" 2024-03-28T18:16:50.482 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-03-28T18:16:50.482 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-03-28T18:16:50.482 INFO:teuthology.orchestra.run.smithi022.stdout:} 2024-03-28T18:16:50.483 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:16:50.478+0000 7f8f63200640 1 -- 172.21.15.22:0/113103098 >> [v2:172.21.15.22:6800/1357365082,v1:172.21.15.22:6801/1357365082] conn(0x7f8f480a2970 msgr2=0x7f8f480a4e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:16:50.483 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:16:50.478+0000 7f8f63200640 1 --2- 172.21.15.22:0/113103098 >> [v2:172.21.15.22:6800/1357365082,v1:172.21.15.22:6801/1357365082] conn(0x7f8f480a2970 0x7f8f480a4e30 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f8f4c009730 tx=0x7f8f4c012040 comp rx=0 tx=0).stop 2024-03-28T18:16:50.483 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:16:50.478+0000 7f8f63200640 1 -- 172.21.15.22:0/113103098 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f8f64112510 msgr2=0x7f8f641a2cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:16:50.483 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:16:50.478+0000 7f8f63200640 1 --2- 172.21.15.22:0/113103098 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f8f64112510 0x7f8f641a2cb0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f8f5c00e9d0 tx=0x7f8f5c00ee90 comp rx=0 tx=0).stop 2024-03-28T18:16:50.483 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:16:50.478+0000 7f8f63200640 1 -- 172.21.15.22:0/113103098 shutdown_connections 2024-03-28T18:16:50.483 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:16:50.478+0000 7f8f63200640 1 --2- 172.21.15.22:0/113103098 >> [v2:172.21.15.22:6800/1357365082,v1:172.21.15.22:6801/1357365082] conn(0x7f8f480a2970 0x7f8f480a4e30 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:16:50.483 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:16:50.478+0000 7f8f63200640 1 --2- 172.21.15.22:0/113103098 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f8f64112510 0x7f8f641a2cb0 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:16:50.483 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:16:50.478+0000 7f8f63200640 1 --2- 172.21.15.22:0/113103098 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8f64074360 0x7f8f6419e980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:16:50.483 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:16:50.478+0000 7f8f63200640 1 --2- 172.21.15.22:0/113103098 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8f64073a40 0x7f8f6419e440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:16:50.483 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:16:50.478+0000 7f8f63200640 1 -- 172.21.15.22:0/113103098 >> 172.21.15.22:0/113103098 conn(0x7f8f640fd010 msgr2=0x7f8f640fe910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:16:50.483 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:16:50.478+0000 7f8f63200640 1 -- 172.21.15.22:0/113103098 shutdown_connections 2024-03-28T18:16:50.483 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:16:50.479+0000 7f8f63200640 1 -- 172.21.15.22:0/113103098 wait complete. 2024-03-28T18:16:50.498 INFO:tasks.cephfs.filesystem:scrub status for tag:4bbbd61d-29a3-4bf6-a4f2-d5783d585de3 - {'path': '/', 'tag': '4bbbd61d-29a3-4bf6-a4f2-d5783d585de3', 'options': 'recursive,force'} 2024-03-28T18:16:50.674 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:16:50 smithi134 ceph-mon[35037]: pgmap v310: 97 pgs: 97 active+clean; 118 MiB data, 800 MiB used, 1.0 TiB / 1.0 TiB avail; 7.9 KiB/s rd, 127 B/s wr, 13 op/s 2024-03-28T18:16:50.674 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:16:50 smithi134 ceph-mon[35037]: mds.? [v2:172.21.15.22:6834/810826586,v1:172.21.15.22:6835/810826586] up:resolve 2024-03-28T18:16:50.674 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:16:50 smithi134 ceph-mon[35037]: fsmap cephfs:5/5 4 up:standby-replay 1 up:resolve 4 up:active 2024-03-28T18:16:50.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:16:50 smithi079 ceph-mon[35312]: pgmap v310: 97 pgs: 97 active+clean; 118 MiB data, 800 MiB used, 1.0 TiB / 1.0 TiB avail; 7.9 KiB/s rd, 127 B/s wr, 13 op/s 2024-03-28T18:16:50.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:16:50 smithi079 ceph-mon[35312]: mds.? [v2:172.21.15.22:6834/810826586,v1:172.21.15.22:6835/810826586] up:resolve 2024-03-28T18:16:50.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:16:50 smithi079 ceph-mon[35312]: fsmap cephfs:5/5 4 up:standby-replay 1 up:resolve 4 up:active 2024-03-28T18:16:50.781 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:16:50 smithi022 ceph-mon[31985]: pgmap v310: 97 pgs: 97 active+clean; 118 MiB data, 800 MiB used, 1.0 TiB / 1.0 TiB avail; 7.9 KiB/s rd, 127 B/s wr, 13 op/s 2024-03-28T18:16:50.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:16:50 smithi022 ceph-mon[31985]: mds.? [v2:172.21.15.22:6834/810826586,v1:172.21.15.22:6835/810826586] up:resolve 2024-03-28T18:16:50.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:16:50 smithi022 ceph-mon[31985]: fsmap cephfs:5/5 4 up:standby-replay 1 up:resolve 4 up:active 2024-03-28T18:16:51.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:16:51 smithi134 ceph-mon[35037]: mds.? [v2:172.21.15.22:6834/810826586,v1:172.21.15.22:6835/810826586] up:reconnect 2024-03-28T18:16:51.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:16:51 smithi134 ceph-mon[35037]: fsmap cephfs:5/5 4 up:standby-replay 1 up:reconnect 4 up:active 2024-03-28T18:16:51.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:16:51 smithi134 ceph-mon[35037]: reconnect by client.35222 172.21.15.22:0/3729116781 after 0.00699991 2024-03-28T18:16:51.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:16:51 smithi079 ceph-mon[35312]: mds.? [v2:172.21.15.22:6834/810826586,v1:172.21.15.22:6835/810826586] up:reconnect 2024-03-28T18:16:51.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:16:51 smithi079 ceph-mon[35312]: fsmap cephfs:5/5 4 up:standby-replay 1 up:reconnect 4 up:active 2024-03-28T18:16:51.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:16:51 smithi079 ceph-mon[35312]: reconnect by client.35222 172.21.15.22:0/3729116781 after 0.00699991 2024-03-28T18:16:51.781 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:16:51 smithi022 ceph-mon[31985]: mds.? [v2:172.21.15.22:6834/810826586,v1:172.21.15.22:6835/810826586] up:reconnect 2024-03-28T18:16:51.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:16:51 smithi022 ceph-mon[31985]: fsmap cephfs:5/5 4 up:standby-replay 1 up:reconnect 4 up:active 2024-03-28T18:16:51.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:16:51 smithi022 ceph-mon[31985]: reconnect by client.35222 172.21.15.22:0/3729116781 after 0.00699991 2024-03-28T18:16:52.648 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:16:52 smithi022 ceph-mon[31985]: pgmap v311: 97 pgs: 97 active+clean; 118 MiB data, 800 MiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 124 B/s wr, 13 op/s 2024-03-28T18:16:52.649 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:16:52 smithi022 ceph-mon[31985]: mds.? [v2:172.21.15.22:6834/810826586,v1:172.21.15.22:6835/810826586] up:rejoin 2024-03-28T18:16:52.649 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:16:52 smithi022 ceph-mon[31985]: fsmap cephfs:5/5 4 up:standby-replay 1 up:rejoin 4 up:active 2024-03-28T18:16:52.649 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:16:52 smithi022 ceph-mon[31985]: daemon mds.j is now active in filesystem cephfs as rank 1 2024-03-28T18:16:52.650 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:16:52 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[95494]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-2145-g3be9a5c9/rpm/el8/BUILD/ceph-19.0.0-2145-g3be9a5c9/src/mds/Locker.cc: In function 'void Locker::set_xlocks_done(MutationImpl*, bool)' thread 7fbd9a800700 time 2024-03-28T18:16:52.472090+0000 2024-03-28T18:16:52.650 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:16:52 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[95494]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-2145-g3be9a5c9/rpm/el8/BUILD/ceph-19.0.0-2145-g3be9a5c9/src/mds/Locker.cc: 626: FAILED ceph_assert(obj->is_auth()) 2024-03-28T18:16:52.650 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:16:52 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[95494]: ceph version 19.0.0-2145-g3be9a5c9 (3be9a5c9bc793e11e2800a8c0c696e8b46742033) squid (dev) 2024-03-28T18:16:52.650 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:16:52 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[95494]: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x135) [0x7fbda60264a1] 2024-03-28T18:16:52.650 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:16:52 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[95494]: 2: /usr/lib64/ceph/libceph-common.so.2(+0x2d2667) [0x7fbda6026667] 2024-03-28T18:16:52.650 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:16:52 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[95494]: 3: (Locker::set_xlocks_done(MutationImpl*, bool)+0x493) [0x556fb6a3c223] 2024-03-28T18:16:52.650 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:16:52 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[95494]: 4: (Server::early_reply(boost::intrusive_ptr const&, CInode*, CDentry*)+0x3e6) [0x556fb68b65d6] 2024-03-28T18:16:52.650 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:16:52 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[95494]: 5: (Server::journal_and_reply(boost::intrusive_ptr const&, CInode*, CDentry*, LogEvent*, MDSLogContextBase*)+0xc1) [0x556fb68b6da1] 2024-03-28T18:16:52.650 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:16:52 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[95494]: 6: (Server::handle_client_mkdir(boost::intrusive_ptr const&)+0x5ba) [0x556fb68ec6aa] 2024-03-28T18:16:52.651 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:16:52 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[95494]: 7: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0x733) [0x556fb6909233] 2024-03-28T18:16:52.651 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:16:52 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[95494]: 8: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x556fb690b197] 2024-03-28T18:16:52.651 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:16:52 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[95494]: 9: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x556fb680a11b] 2024-03-28T18:16:52.651 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:16:52 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[95494]: 10: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x556fb680a6bc] 2024-03-28T18:16:52.651 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:16:52 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[95494]: 11: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bf) [0x556fb67f41ff] 2024-03-28T18:16:52.651 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:16:52 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[95494]: 12: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x478) [0x7fbda62b8488] 2024-03-28T18:16:52.651 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:16:52 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[95494]: 13: (DispatchQueue::entry()+0x51f) [0x7fbda62b560f] 2024-03-28T18:16:52.651 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:16:52 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[95494]: 14: (DispatchQueue::DispatchThread::entry()+0x11) [0x7fbda6388e81] 2024-03-28T18:16:52.651 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:16:52 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[95494]: 15: /lib64/libpthread.so.0(+0x81ca) [0x7fbda4da31ca] 2024-03-28T18:16:52.651 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:16:52 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[95494]: 16: clone() 2024-03-28T18:16:52.652 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:16:52 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[95494]: 2024-03-28T18:16:52.474+0000 7fbd9a800700 -1 /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-2145-g3be9a5c9/rpm/el8/BUILD/ceph-19.0.0-2145-g3be9a5c9/src/mds/Locker.cc: In function 'void Locker::set_xlocks_done(MutationImpl*, bool)' thread 7fbd9a800700 time 2024-03-28T18:16:52.472090+0000 2024-03-28T18:16:52.652 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:16:52 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[95494]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-2145-g3be9a5c9/rpm/el8/BUILD/ceph-19.0.0-2145-g3be9a5c9/src/mds/Locker.cc: 626: FAILED ceph_assert(obj->is_auth()) 2024-03-28T18:16:52.652 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:16:52 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[95494]: 2024-03-28T18:16:52.653 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:16:52 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[95494]: ceph version 19.0.0-2145-g3be9a5c9 (3be9a5c9bc793e11e2800a8c0c696e8b46742033) squid (dev) 2024-03-28T18:16:52.653 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:16:52 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[95494]: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x135) [0x7fbda60264a1] 2024-03-28T18:16:52.653 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:16:52 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[95494]: 2: /usr/lib64/ceph/libceph-common.so.2(+0x2d2667) [0x7fbda6026667] 2024-03-28T18:16:52.653 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:16:52 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[95494]: 3: (Locker::set_xlocks_done(MutationImpl*, bool)+0x493) [0x556fb6a3c223] 2024-03-28T18:16:52.653 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:16:52 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[95494]: 4: (Server::early_reply(boost::intrusive_ptr const&, CInode*, CDentry*)+0x3e6) [0x556fb68b65d6] 2024-03-28T18:16:52.653 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:16:52 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[95494]: 5: (Server::journal_and_reply(boost::intrusive_ptr const&, CInode*, CDentry*, LogEvent*, MDSLogContextBase*)+0xc1) [0x556fb68b6da1] 2024-03-28T18:16:52.653 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:16:52 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[95494]: 6: (Server::handle_client_mkdir(boost::intrusive_ptr const&)+0x5ba) [0x556fb68ec6aa] 2024-03-28T18:16:52.653 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:16:52 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[95494]: 7: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0x733) [0x556fb6909233] 2024-03-28T18:16:52.653 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:16:52 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[95494]: 8: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x556fb690b197] 2024-03-28T18:16:52.653 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:16:52 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[95494]: 9: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x556fb680a11b] 2024-03-28T18:16:52.653 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:16:52 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[95494]: 10: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x556fb680a6bc] 2024-03-28T18:16:52.653 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:16:52 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[95494]: 11: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bf) [0x556fb67f41ff] 2024-03-28T18:16:52.653 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:16:52 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[95494]: 12: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x478) [0x7fbda62b8488] 2024-03-28T18:16:52.653 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:16:52 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[95494]: 13: (DispatchQueue::entry()+0x51f) [0x7fbda62b560f] 2024-03-28T18:16:52.653 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:16:52 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[95494]: 14: (DispatchQueue::DispatchThread::entry()+0x11) [0x7fbda6388e81] 2024-03-28T18:16:52.653 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:16:52 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[95494]: 15: /lib64/libpthread.so.0(+0x81ca) [0x7fbda4da31ca] 2024-03-28T18:16:52.654 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:16:52 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[95494]: 16: clone() 2024-03-28T18:16:52.654 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:16:52 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[95494]: 2024-03-28T18:16:52.654 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:16:52 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[95494]: *** Caught signal (Aborted) ** 2024-03-28T18:16:52.654 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:16:52 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[95494]: in thread 7fbd9a800700 thread_name:ms_dispatch 2024-03-28T18:16:52.654 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:16:52 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[95494]: ceph version 19.0.0-2145-g3be9a5c9 (3be9a5c9bc793e11e2800a8c0c696e8b46742033) squid (dev) 2024-03-28T18:16:52.654 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:16:52 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[95494]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7fbda4dadd20] 2024-03-28T18:16:52.655 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:16:52 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[95494]: 2: gsignal() 2024-03-28T18:16:52.655 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:16:52 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[95494]: 3: abort() 2024-03-28T18:16:52.655 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:16:52 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[95494]: 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x18f) [0x7fbda60264fb] 2024-03-28T18:16:52.655 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:16:52 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[95494]: 5: /usr/lib64/ceph/libceph-common.so.2(+0x2d2667) [0x7fbda6026667] 2024-03-28T18:16:52.655 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:16:52 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[95494]: 6: (Locker::set_xlocks_done(MutationImpl*, bool)+0x493) [0x556fb6a3c223] 2024-03-28T18:16:52.655 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:16:52 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[95494]: 7: (Server::early_reply(boost::intrusive_ptr const&, CInode*, CDentry*)+0x3e6) [0x556fb68b65d6] 2024-03-28T18:16:52.655 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:16:52 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[95494]: 8: (Server::journal_and_reply(boost::intrusive_ptr const&, CInode*, CDentry*, LogEvent*, MDSLogContextBase*)+0xc1) [0x556fb68b6da1] 2024-03-28T18:16:52.655 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:16:52 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[95494]: 9: (Server::handle_client_mkdir(boost::intrusive_ptr const&)+0x5ba) [0x556fb68ec6aa] 2024-03-28T18:16:52.656 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:16:52 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[95494]: 10: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0x733) [0x556fb6909233] 2024-03-28T18:16:52.656 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:16:52 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[95494]: 11: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x556fb690b197] 2024-03-28T18:16:52.656 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:16:52 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[95494]: 12: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x556fb680a11b] 2024-03-28T18:16:52.656 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:16:52 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[95494]: 13: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x556fb680a6bc] 2024-03-28T18:16:52.656 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:16:52 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[95494]: 14: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bf) [0x556fb67f41ff] 2024-03-28T18:16:52.656 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:16:52 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[95494]: 15: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x478) [0x7fbda62b8488] 2024-03-28T18:16:52.656 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:16:52 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[95494]: 16: (DispatchQueue::entry()+0x51f) [0x7fbda62b560f] 2024-03-28T18:16:52.656 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:16:52 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[95494]: 17: (DispatchQueue::DispatchThread::entry()+0x11) [0x7fbda6388e81] 2024-03-28T18:16:52.656 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:16:52 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[95494]: 18: /lib64/libpthread.so.0(+0x81ca) [0x7fbda4da31ca] 2024-03-28T18:16:52.656 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:16:52 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[95494]: 19: clone() 2024-03-28T18:16:52.656 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:16:52 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[95494]: 2024-03-28T18:16:52.482+0000 7fbd9a800700 -1 *** Caught signal (Aborted) ** 2024-03-28T18:16:52.656 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:16:52 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[95494]: in thread 7fbd9a800700 thread_name:ms_dispatch 2024-03-28T18:16:52.656 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:16:52 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[95494]: 2024-03-28T18:16:52.656 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:16:52 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[95494]: ceph version 19.0.0-2145-g3be9a5c9 (3be9a5c9bc793e11e2800a8c0c696e8b46742033) squid (dev) 2024-03-28T18:16:52.656 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:16:52 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[95494]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7fbda4dadd20] 2024-03-28T18:16:52.656 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:16:52 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[95494]: 2: gsignal() 2024-03-28T18:16:52.656 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:16:52 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[95494]: 3: abort() 2024-03-28T18:16:52.657 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:16:52 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[95494]: 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x18f) [0x7fbda60264fb] 2024-03-28T18:16:52.657 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:16:52 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[95494]: 5: /usr/lib64/ceph/libceph-common.so.2(+0x2d2667) [0x7fbda6026667] 2024-03-28T18:16:52.657 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:16:52 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[95494]: 6: (Locker::set_xlocks_done(MutationImpl*, bool)+0x493) [0x556fb6a3c223] 2024-03-28T18:16:52.657 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:16:52 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[95494]: 7: (Server::early_reply(boost::intrusive_ptr const&, CInode*, CDentry*)+0x3e6) [0x556fb68b65d6] 2024-03-28T18:16:52.657 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:16:52 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[95494]: 8: (Server::journal_and_reply(boost::intrusive_ptr const&, CInode*, CDentry*, LogEvent*, MDSLogContextBase*)+0xc1) [0x556fb68b6da1] 2024-03-28T18:16:52.657 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:16:52 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[95494]: 9: (Server::handle_client_mkdir(boost::intrusive_ptr const&)+0x5ba) [0x556fb68ec6aa] 2024-03-28T18:16:52.657 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:16:52 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[95494]: 10: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0x733) [0x556fb6909233] 2024-03-28T18:16:52.657 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:16:52 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[95494]: 11: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x556fb690b197] 2024-03-28T18:16:52.657 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:16:52 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[95494]: 12: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x556fb680a11b] 2024-03-28T18:16:52.657 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:16:52 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[95494]: 13: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x556fb680a6bc] 2024-03-28T18:16:52.657 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:16:52 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[95494]: 14: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bf) [0x556fb67f41ff] 2024-03-28T18:16:52.657 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:16:52 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[95494]: 15: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x478) [0x7fbda62b8488] 2024-03-28T18:16:52.658 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:16:52 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[95494]: 16: (DispatchQueue::entry()+0x51f) [0x7fbda62b560f] 2024-03-28T18:16:52.658 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:16:52 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[95494]: 17: (DispatchQueue::DispatchThread::entry()+0x11) [0x7fbda6388e81] 2024-03-28T18:16:52.658 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:16:52 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[95494]: 18: /lib64/libpthread.so.0(+0x81ca) [0x7fbda4da31ca] 2024-03-28T18:16:52.658 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:16:52 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[95494]: 19: clone() 2024-03-28T18:16:52.658 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:16:52 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[95494]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-03-28T18:16:52.658 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:16:52 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[95494]: 2024-03-28T18:16:52.658 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:16:52 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[95494]: -1> 2024-03-28T18:16:52.474+0000 7fbd9a800700 -1 /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-2145-g3be9a5c9/rpm/el8/BUILD/ceph-19.0.0-2145-g3be9a5c9/src/mds/Locker.cc: In function 'void Locker::set_xlocks_done(MutationImpl*, bool)' thread 7fbd9a800700 time 2024-03-28T18:16:52.472090+0000 2024-03-28T18:16:52.658 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:16:52 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[95494]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-2145-g3be9a5c9/rpm/el8/BUILD/ceph-19.0.0-2145-g3be9a5c9/src/mds/Locker.cc: 626: FAILED ceph_assert(obj->is_auth()) 2024-03-28T18:16:52.658 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:16:52 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[95494]: 2024-03-28T18:16:52.658 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:16:52 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[95494]: ceph version 19.0.0-2145-g3be9a5c9 (3be9a5c9bc793e11e2800a8c0c696e8b46742033) squid (dev) 2024-03-28T18:16:52.658 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:16:52 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[95494]: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x135) [0x7fbda60264a1] 2024-03-28T18:16:52.658 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:16:52 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[95494]: 2: /usr/lib64/ceph/libceph-common.so.2(+0x2d2667) [0x7fbda6026667] 2024-03-28T18:16:52.659 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:16:52 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[95494]: 3: (Locker::set_xlocks_done(MutationImpl*, bool)+0x493) [0x556fb6a3c223] 2024-03-28T18:16:52.659 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:16:52 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[95494]: 4: (Server::early_reply(boost::intrusive_ptr const&, CInode*, CDentry*)+0x3e6) [0x556fb68b65d6] 2024-03-28T18:16:52.659 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:16:52 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[95494]: 5: (Server::journal_and_reply(boost::intrusive_ptr const&, CInode*, CDentry*, LogEvent*, MDSLogContextBase*)+0xc1) [0x556fb68b6da1] 2024-03-28T18:16:52.659 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:16:52 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[95494]: 6: (Server::handle_client_mkdir(boost::intrusive_ptr const&)+0x5ba) [0x556fb68ec6aa] 2024-03-28T18:16:52.659 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:16:52 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[95494]: 7: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0x733) [0x556fb6909233] 2024-03-28T18:16:52.659 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:16:52 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[95494]: 8: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x556fb690b197] 2024-03-28T18:16:52.659 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:16:52 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[95494]: 9: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x556fb680a11b] 2024-03-28T18:16:52.659 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:16:52 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[95494]: 10: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x556fb680a6bc] 2024-03-28T18:16:52.659 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:16:52 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[95494]: 11: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bf) [0x556fb67f41ff] 2024-03-28T18:16:52.659 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:16:52 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[95494]: 12: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x478) [0x7fbda62b8488] 2024-03-28T18:16:52.659 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:16:52 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[95494]: 13: (DispatchQueue::entry()+0x51f) [0x7fbda62b560f] 2024-03-28T18:16:52.660 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:16:52 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[95494]: 14: (DispatchQueue::DispatchThread::entry()+0x11) [0x7fbda6388e81] 2024-03-28T18:16:52.660 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:16:52 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[95494]: 15: /lib64/libpthread.so.0(+0x81ca) [0x7fbda4da31ca] 2024-03-28T18:16:52.660 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:16:52 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[95494]: 16: clone() 2024-03-28T18:16:52.660 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:16:52 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[95494]: 2024-03-28T18:16:52.660 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:16:52 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[95494]: 0> 2024-03-28T18:16:52.482+0000 7fbd9a800700 -1 *** Caught signal (Aborted) ** 2024-03-28T18:16:52.660 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:16:52 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[95494]: in thread 7fbd9a800700 thread_name:ms_dispatch 2024-03-28T18:16:52.660 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:16:52 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[95494]: 2024-03-28T18:16:52.660 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:16:52 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[95494]: ceph version 19.0.0-2145-g3be9a5c9 (3be9a5c9bc793e11e2800a8c0c696e8b46742033) squid (dev) 2024-03-28T18:16:52.660 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:16:52 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[95494]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7fbda4dadd20] 2024-03-28T18:16:52.660 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:16:52 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[95494]: 2: gsignal() 2024-03-28T18:16:52.660 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:16:52 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[95494]: 3: abort() 2024-03-28T18:16:52.660 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:16:52 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[95494]: 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x18f) [0x7fbda60264fb] 2024-03-28T18:16:52.660 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:16:52 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[95494]: 5: /usr/lib64/ceph/libceph-common.so.2(+0x2d2667) [0x7fbda6026667] 2024-03-28T18:16:52.661 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:16:52 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[95494]: 6: (Locker::set_xlocks_done(MutationImpl*, bool)+0x493) [0x556fb6a3c223] 2024-03-28T18:16:52.661 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:16:52 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[95494]: 7: (Server::early_reply(boost::intrusive_ptr const&, CInode*, CDentry*)+0x3e6) [0x556fb68b65d6] 2024-03-28T18:16:52.661 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:16:52 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[95494]: 8: (Server::journal_and_reply(boost::intrusive_ptr const&, CInode*, CDentry*, LogEvent*, MDSLogContextBase*)+0xc1) [0x556fb68b6da1] 2024-03-28T18:16:52.661 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:16:52 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[95494]: 9: (Server::handle_client_mkdir(boost::intrusive_ptr const&)+0x5ba) [0x556fb68ec6aa] 2024-03-28T18:16:52.661 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:16:52 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[95494]: 10: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0x733) [0x556fb6909233] 2024-03-28T18:16:52.661 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:16:52 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[95494]: 11: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x556fb690b197] 2024-03-28T18:16:52.661 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:16:52 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[95494]: 12: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x556fb680a11b] 2024-03-28T18:16:52.661 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:16:52 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[95494]: 13: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x556fb680a6bc] 2024-03-28T18:16:52.661 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:16:52 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[95494]: 14: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bf) [0x556fb67f41ff] 2024-03-28T18:16:52.661 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:16:52 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[95494]: 15: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x478) [0x7fbda62b8488] 2024-03-28T18:16:52.661 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:16:52 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[95494]: 16: (DispatchQueue::entry()+0x51f) [0x7fbda62b560f] 2024-03-28T18:16:52.661 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:16:52 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[95494]: 17: (DispatchQueue::DispatchThread::entry()+0x11) [0x7fbda6388e81] 2024-03-28T18:16:52.661 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:16:52 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[95494]: 18: /lib64/libpthread.so.0(+0x81ca) [0x7fbda4da31ca] 2024-03-28T18:16:52.661 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:16:52 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[95494]: 19: clone() 2024-03-28T18:16:52.661 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:16:52 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[95494]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-03-28T18:16:52.661 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:16:52 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[95494]: 2024-03-28T18:16:52.662 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:16:52 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[95494]: -4> 2024-03-28T18:16:52.474+0000 7fbd9a800700 -1 /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-2145-g3be9a5c9/rpm/el8/BUILD/ceph-19.0.0-2145-g3be9a5c9/src/mds/Locker.cc: In function 'void Locker::set_xlocks_done(MutationImpl*, bool)' thread 7fbd9a800700 time 2024-03-28T18:16:52.472090+0000 2024-03-28T18:16:52.662 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:16:52 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[95494]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-2145-g3be9a5c9/rpm/el8/BUILD/ceph-19.0.0-2145-g3be9a5c9/src/mds/Locker.cc: 626: FAILED ceph_assert(obj->is_auth()) 2024-03-28T18:16:52.662 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:16:52 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[95494]: 2024-03-28T18:16:52.662 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:16:52 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[95494]: ceph version 19.0.0-2145-g3be9a5c9 (3be9a5c9bc793e11e2800a8c0c696e8b46742033) squid (dev) 2024-03-28T18:16:52.662 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:16:52 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[95494]: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x135) [0x7fbda60264a1] 2024-03-28T18:16:52.662 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:16:52 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[95494]: 2: /usr/lib64/ceph/libceph-common.so.2(+0x2d2667) [0x7fbda6026667] 2024-03-28T18:16:52.662 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:16:52 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[95494]: 3: (Locker::set_xlocks_done(MutationImpl*, bool)+0x493) [0x556fb6a3c223] 2024-03-28T18:16:52.662 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:16:52 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[95494]: 4: (Server::early_reply(boost::intrusive_ptr const&, CInode*, CDentry*)+0x3e6) [0x556fb68b65d6] 2024-03-28T18:16:52.662 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:16:52 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[95494]: 5: (Server::journal_and_reply(boost::intrusive_ptr const&, CInode*, CDentry*, LogEvent*, MDSLogContextBase*)+0xc1) [0x556fb68b6da1] 2024-03-28T18:16:52.665 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:16:52 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[95494]: 6: (Server::handle_client_mkdir(boost::intrusive_ptr const&)+0x5ba) [0x556fb68ec6aa] 2024-03-28T18:16:52.665 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:16:52 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[95494]: 7: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0x733) [0x556fb6909233] 2024-03-28T18:16:52.666 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:16:52 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[95494]: 8: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x556fb690b197] 2024-03-28T18:16:52.666 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:16:52 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[95494]: 9: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x556fb680a11b] 2024-03-28T18:16:52.666 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:16:52 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[95494]: 10: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x556fb680a6bc] 2024-03-28T18:16:52.666 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:16:52 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[95494]: 11: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bf) [0x556fb67f41ff] 2024-03-28T18:16:52.666 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:16:52 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[95494]: 12: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x478) [0x7fbda62b8488] 2024-03-28T18:16:52.666 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:16:52 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[95494]: 13: (DispatchQueue::entry()+0x51f) [0x7fbda62b560f] 2024-03-28T18:16:52.666 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:16:52 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[95494]: 14: (DispatchQueue::DispatchThread::entry()+0x11) [0x7fbda6388e81] 2024-03-28T18:16:52.666 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:16:52 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[95494]: 15: /lib64/libpthread.so.0(+0x81ca) [0x7fbda4da31ca] 2024-03-28T18:16:52.666 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:16:52 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[95494]: 16: clone() 2024-03-28T18:16:52.666 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:16:52 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[95494]: 2024-03-28T18:16:52.666 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:16:52 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[95494]: -3> 2024-03-28T18:16:52.482+0000 7fbd9a800700 -1 *** Caught signal (Aborted) ** 2024-03-28T18:16:52.666 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:16:52 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[95494]: in thread 7fbd9a800700 thread_name:ms_dispatch 2024-03-28T18:16:52.666 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:16:52 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[95494]: 2024-03-28T18:16:52.667 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:16:52 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[95494]: ceph version 19.0.0-2145-g3be9a5c9 (3be9a5c9bc793e11e2800a8c0c696e8b46742033) squid (dev) 2024-03-28T18:16:52.667 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:16:52 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[95494]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7fbda4dadd20] 2024-03-28T18:16:52.667 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:16:52 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[95494]: 2: gsignal() 2024-03-28T18:16:52.667 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:16:52 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[95494]: 3: abort() 2024-03-28T18:16:52.667 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:16:52 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[95494]: 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x18f) [0x7fbda60264fb] 2024-03-28T18:16:52.667 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:16:52 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[95494]: 5: /usr/lib64/ceph/libceph-common.so.2(+0x2d2667) [0x7fbda6026667] 2024-03-28T18:16:52.667 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:16:52 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[95494]: 6: (Locker::set_xlocks_done(MutationImpl*, bool)+0x493) [0x556fb6a3c223] 2024-03-28T18:16:52.667 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:16:52 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[95494]: 7: (Server::early_reply(boost::intrusive_ptr const&, CInode*, CDentry*)+0x3e6) [0x556fb68b65d6] 2024-03-28T18:16:52.667 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:16:52 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[95494]: 8: (Server::journal_and_reply(boost::intrusive_ptr const&, CInode*, CDentry*, LogEvent*, MDSLogContextBase*)+0xc1) [0x556fb68b6da1] 2024-03-28T18:16:52.667 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:16:52 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[95494]: 9: (Server::handle_client_mkdir(boost::intrusive_ptr const&)+0x5ba) [0x556fb68ec6aa] 2024-03-28T18:16:52.667 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:16:52 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[95494]: 10: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0x733) [0x556fb6909233] 2024-03-28T18:16:52.667 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:16:52 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[95494]: 11: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x556fb690b197] 2024-03-28T18:16:52.667 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:16:52 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[95494]: 12: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x556fb680a11b] 2024-03-28T18:16:52.667 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:16:52 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[95494]: 13: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x556fb680a6bc] 2024-03-28T18:16:52.667 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:16:52 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[95494]: 14: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bf) [0x556fb67f41ff] 2024-03-28T18:16:52.668 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:16:52 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[95494]: 15: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x478) [0x7fbda62b8488] 2024-03-28T18:16:52.668 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:16:52 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[95494]: 16: (DispatchQueue::entry()+0x51f) [0x7fbda62b560f] 2024-03-28T18:16:52.668 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:16:52 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[95494]: 17: (DispatchQueue::DispatchThread::entry()+0x11) [0x7fbda6388e81] 2024-03-28T18:16:52.668 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:16:52 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[95494]: 18: /lib64/libpthread.so.0(+0x81ca) [0x7fbda4da31ca] 2024-03-28T18:16:52.668 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:16:52 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[95494]: 19: clone() 2024-03-28T18:16:52.668 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:16:52 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[95494]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-03-28T18:16:52.668 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:16:52 smithi022 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j[95494]: 2024-03-28T18:16:52.668 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:16:52 smithi022 podman[95657]: 2024-03-28 18:16:52.649449624 +0000 UTC m=+0.040760199 container died 586c5ee5ba0672ecb1067f5e9a181cbe4defa1f65c015a63ef9f411f5cb3faea (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.build-date=20240319, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, ceph=True) 2024-03-28T18:16:52.674 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:16:52 smithi134 ceph-mon[35037]: pgmap v311: 97 pgs: 97 active+clean; 118 MiB data, 800 MiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 124 B/s wr, 13 op/s 2024-03-28T18:16:52.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:16:52 smithi134 ceph-mon[35037]: mds.? [v2:172.21.15.22:6834/810826586,v1:172.21.15.22:6835/810826586] up:rejoin 2024-03-28T18:16:52.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:16:52 smithi134 ceph-mon[35037]: fsmap cephfs:5/5 4 up:standby-replay 1 up:rejoin 4 up:active 2024-03-28T18:16:52.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:16:52 smithi134 ceph-mon[35037]: daemon mds.j is now active in filesystem cephfs as rank 1 2024-03-28T18:16:52.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:16:52 smithi079 ceph-mon[35312]: pgmap v311: 97 pgs: 97 active+clean; 118 MiB data, 800 MiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 124 B/s wr, 13 op/s 2024-03-28T18:16:52.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:16:52 smithi079 ceph-mon[35312]: mds.? [v2:172.21.15.22:6834/810826586,v1:172.21.15.22:6835/810826586] up:rejoin 2024-03-28T18:16:52.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:16:52 smithi079 ceph-mon[35312]: fsmap cephfs:5/5 4 up:standby-replay 1 up:rejoin 4 up:active 2024-03-28T18:16:52.683 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:16:52 smithi079 ceph-mon[35312]: daemon mds.j is now active in filesystem cephfs as rank 1 2024-03-28T18:16:53.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:16:53 smithi134 ceph-mon[35037]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-03-28T18:16:53.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:16:53 smithi134 ceph-mon[35037]: Cluster is now healthy 2024-03-28T18:16:53.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:16:53 smithi134 ceph-mon[35037]: mds.? [v2:172.21.15.22:6834/810826586,v1:172.21.15.22:6835/810826586] up:active 2024-03-28T18:16:53.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:16:53 smithi134 ceph-mon[35037]: fsmap cephfs:5 4 up:standby-replay 5 up:active 2024-03-28T18:16:53.685 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:16:53 smithi079 ceph-mon[35312]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-03-28T18:16:53.685 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:16:53 smithi079 ceph-mon[35312]: Cluster is now healthy 2024-03-28T18:16:53.685 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:16:53 smithi079 ceph-mon[35312]: mds.? [v2:172.21.15.22:6834/810826586,v1:172.21.15.22:6835/810826586] up:active 2024-03-28T18:16:53.685 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:16:53 smithi079 ceph-mon[35312]: fsmap cephfs:5 4 up:standby-replay 5 up:active 2024-03-28T18:16:53.782 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:16:53 smithi022 podman[95657]: 2024-03-28 18:16:53.321759323 +0000 UTC m=+0.713069927 container remove 586c5ee5ba0672ecb1067f5e9a181cbe4defa1f65c015a63ef9f411f5cb3faea (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-j, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, ceph=True, org.label-schema.build-date=20240319, org.label-schema.vendor=CentOS, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux ) 2024-03-28T18:16:53.782 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:16:53 smithi022 systemd[1]: ceph-fd220748-ed2b-11ee-95d0-87774f69a715@mds.j.service: Main process exited, code=exited, status=134/n/a 2024-03-28T18:16:53.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:16:53 smithi022 ceph-mon[31985]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-03-28T18:16:53.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:16:53 smithi022 ceph-mon[31985]: Cluster is now healthy 2024-03-28T18:16:53.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:16:53 smithi022 ceph-mon[31985]: mds.? [v2:172.21.15.22:6834/810826586,v1:172.21.15.22:6835/810826586] up:active 2024-03-28T18:16:53.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:16:53 smithi022 ceph-mon[31985]: fsmap cephfs:5 4 up:standby-replay 5 up:active 2024-03-28T18:16:54.282 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:16:53 smithi022 systemd[1]: ceph-fd220748-ed2b-11ee-95d0-87774f69a715@mds.j.service: Failed with result 'exit-code'. 2024-03-28T18:16:54.282 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:16:53 smithi022 systemd[1]: ceph-fd220748-ed2b-11ee-95d0-87774f69a715@mds.j.service: Consumed 2.217s CPU time. 2024-03-28T18:16:54.432 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:16:54 smithi079 ceph-mon[35312]: pgmap v312: 97 pgs: 97 active+clean; 118 MiB data, 800 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 102 B/s wr, 10 op/s 2024-03-28T18:16:54.433 INFO:journalctl@ceph.mds.b.smithi079.stdout:Mar 28 18:16:54 smithi079 ceph-fd220748-ed2b-11ee-95d0-87774f69a715-mds-b[65245]: 2024-03-28T18:16:54.107+0000 7f845f400700 -1 mds.pinger is_rank_lagging: rank=1 was never sent ping request. 2024-03-28T18:16:54.674 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:16:54 smithi134 ceph-mon[35037]: pgmap v312: 97 pgs: 97 active+clean; 118 MiB data, 800 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 102 B/s wr, 10 op/s 2024-03-28T18:16:54.781 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:16:54 smithi022 ceph-mon[31985]: pgmap v312: 97 pgs: 97 active+clean; 118 MiB data, 800 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 102 B/s wr, 10 op/s 2024-03-28T18:16:56.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:16:56 smithi134 ceph-mon[35037]: pgmap v313: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 6.3 KiB/s wr, 24 op/s 2024-03-28T18:16:56.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:16:56 smithi079 ceph-mon[35312]: pgmap v313: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 6.3 KiB/s wr, 24 op/s 2024-03-28T18:16:56.781 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:16:56 smithi022 ceph-mon[31985]: pgmap v313: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 6.3 KiB/s wr, 24 op/s 2024-03-28T18:16:58.674 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:16:58 smithi134 ceph-mon[35037]: pgmap v314: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 6.3 KiB/s wr, 24 op/s 2024-03-28T18:16:58.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:16:58 smithi079 ceph-mon[35312]: pgmap v314: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 6.3 KiB/s wr, 24 op/s 2024-03-28T18:16:58.781 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:16:58 smithi022 ceph-mon[31985]: pgmap v314: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 6.3 KiB/s wr, 24 op/s 2024-03-28T18:16:59.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:16:59 smithi134 ceph-mon[35037]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T18:16:59.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:16:59 smithi079 ceph-mon[35312]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T18:16:59.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:16:59 smithi022 ceph-mon[31985]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T18:17:00.674 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:17:00 smithi134 ceph-mon[35037]: pgmap v315: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 5.6 KiB/s wr, 25 op/s 2024-03-28T18:17:00.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:17:00 smithi134 ceph-mon[35037]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T18:17:00.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:17:00 smithi134 ceph-mon[35037]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T18:17:00.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:17:00 smithi134 ceph-mon[35037]: from='mgr.15522 ' entity='mgr.x' 2024-03-28T18:17:00.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:17:00 smithi079 ceph-mon[35312]: pgmap v315: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 5.6 KiB/s wr, 25 op/s 2024-03-28T18:17:00.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:17:00 smithi079 ceph-mon[35312]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T18:17:00.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:17:00 smithi079 ceph-mon[35312]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T18:17:00.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:17:00 smithi079 ceph-mon[35312]: from='mgr.15522 ' entity='mgr.x' 2024-03-28T18:17:00.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:17:00 smithi022 ceph-mon[31985]: pgmap v315: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 5.6 KiB/s wr, 25 op/s 2024-03-28T18:17:00.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:17:00 smithi022 ceph-mon[31985]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T18:17:00.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:17:00 smithi022 ceph-mon[31985]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T18:17:00.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:17:00 smithi022 ceph-mon[31985]: from='mgr.15522 ' entity='mgr.x' 2024-03-28T18:17:02.674 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:17:02 smithi134 ceph-mon[35037]: pgmap v316: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 5.2 KiB/s wr, 19 op/s 2024-03-28T18:17:02.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:17:02 smithi079 ceph-mon[35312]: pgmap v316: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 5.2 KiB/s wr, 19 op/s 2024-03-28T18:17:02.781 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:17:02 smithi022 ceph-mon[31985]: pgmap v316: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 5.2 KiB/s wr, 19 op/s 2024-03-28T18:17:04.393 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:17:04 smithi022 systemd[1]: ceph-fd220748-ed2b-11ee-95d0-87774f69a715@mds.j.service: Scheduled restart job, restart counter is at 6. 2024-03-28T18:17:04.394 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:17:04 smithi022 systemd[1]: Stopped Ceph mds.j for fd220748-ed2b-11ee-95d0-87774f69a715. 2024-03-28T18:17:04.394 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:17:04 smithi022 systemd[1]: ceph-fd220748-ed2b-11ee-95d0-87774f69a715@mds.j.service: Consumed 2.217s CPU time. 2024-03-28T18:17:04.394 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:17:04 smithi022 systemd[1]: ceph-fd220748-ed2b-11ee-95d0-87774f69a715@mds.j.service: Start request repeated too quickly. 2024-03-28T18:17:04.394 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:17:04 smithi022 systemd[1]: ceph-fd220748-ed2b-11ee-95d0-87774f69a715@mds.j.service: Failed with result 'exit-code'. 2024-03-28T18:17:04.394 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 28 18:17:04 smithi022 systemd[1]: Failed to start Ceph mds.j for fd220748-ed2b-11ee-95d0-87774f69a715. 2024-03-28T18:17:04.674 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:17:04 smithi134 ceph-mon[35037]: pgmap v317: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 5.2 KiB/s wr, 19 op/s 2024-03-28T18:17:04.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:17:04 smithi079 ceph-mon[35312]: pgmap v317: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 5.2 KiB/s wr, 19 op/s 2024-03-28T18:17:04.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:17:04 smithi022 ceph-mon[31985]: pgmap v317: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 5.2 KiB/s wr, 19 op/s 2024-03-28T18:17:06.674 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:17:06 smithi134 ceph-mon[35037]: pgmap v318: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 5.2 KiB/s wr, 24 op/s 2024-03-28T18:17:06.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:17:06 smithi079 ceph-mon[35312]: pgmap v318: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 5.2 KiB/s wr, 24 op/s 2024-03-28T18:17:06.781 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:17:06 smithi022 ceph-mon[31985]: pgmap v318: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 5.2 KiB/s wr, 24 op/s 2024-03-28T18:17:08.674 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:17:08 smithi134 ceph-mon[35037]: pgmap v319: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 0 B/s wr, 7 op/s 2024-03-28T18:17:08.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:17:08 smithi134 ceph-mon[35037]: fsmap cephfs:5 4 up:standby-replay 5 up:active 2024-03-28T18:17:08.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:17:08 smithi079 ceph-mon[35312]: pgmap v319: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 0 B/s wr, 7 op/s 2024-03-28T18:17:08.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:17:08 smithi079 ceph-mon[35312]: fsmap cephfs:5 4 up:standby-replay 5 up:active 2024-03-28T18:17:08.781 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:17:08 smithi022 ceph-mon[31985]: pgmap v319: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 0 B/s wr, 7 op/s 2024-03-28T18:17:08.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:17:08 smithi022 ceph-mon[31985]: fsmap cephfs:5 4 up:standby-replay 5 up:active 2024-03-28T18:17:10.674 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:17:10 smithi134 ceph-mon[35037]: pgmap v320: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 0 B/s wr, 11 op/s 2024-03-28T18:17:10.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:17:10 smithi079 ceph-mon[35312]: pgmap v320: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 0 B/s wr, 11 op/s 2024-03-28T18:17:10.781 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:17:10 smithi022 ceph-mon[31985]: pgmap v320: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 0 B/s wr, 11 op/s 2024-03-28T18:17:12.674 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:17:12 smithi134 ceph-mon[35037]: pgmap v321: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 0 B/s wr, 7 op/s 2024-03-28T18:17:12.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:17:12 smithi079 ceph-mon[35312]: pgmap v321: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 0 B/s wr, 7 op/s 2024-03-28T18:17:12.781 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:17:12 smithi022 ceph-mon[31985]: pgmap v321: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 0 B/s wr, 7 op/s 2024-03-28T18:17:14.674 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:17:14 smithi134 ceph-mon[35037]: pgmap v322: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 0 B/s wr, 7 op/s 2024-03-28T18:17:14.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:17:14 smithi079 ceph-mon[35312]: pgmap v322: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 0 B/s wr, 7 op/s 2024-03-28T18:17:14.781 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:17:14 smithi022 ceph-mon[31985]: pgmap v322: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 0 B/s wr, 7 op/s 2024-03-28T18:17:16.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:17:16 smithi079 ceph-mon[35312]: pgmap v323: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 0 B/s wr, 11 op/s 2024-03-28T18:17:16.781 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:17:16 smithi022 ceph-mon[31985]: pgmap v323: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 0 B/s wr, 11 op/s 2024-03-28T18:17:16.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:17:16 smithi134 ceph-mon[35037]: pgmap v323: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 0 B/s wr, 11 op/s 2024-03-28T18:17:18.781 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:17:18 smithi022 ceph-mon[31985]: pgmap v324: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:17:18.925 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:17:18 smithi134 ceph-mon[35037]: pgmap v324: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:17:18.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:17:18 smithi079 ceph-mon[35312]: pgmap v324: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:17:20.499 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-28T18:17:20.499 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-28T18:17:20.735 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:17:20.735+0000 7f3d26400640 1 -- 172.21.15.22:0/2591195525 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3d201026d0 msgr2=0x7f3d20102ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:17:20.735 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:17:20.735+0000 7f3d26400640 1 --2- 172.21.15.22:0/2591195525 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3d201026d0 0x7f3d20102ab0 secure :-1 s=READY pgs=783 cs=0 l=1 rev1=1 crypto rx=0x7f3d10009ca0 tx=0x7f3d1002c190 comp rx=0 tx=0).stop 2024-03-28T18:17:20.735 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:17:20.735+0000 7f3d26400640 1 -- 172.21.15.22:0/2591195525 shutdown_connections 2024-03-28T18:17:20.735 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:17:20.735+0000 7f3d26400640 1 --2- 172.21.15.22:0/2591195525 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3d2010baf0 0x7f3d2010df90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:17:20.735 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:17:20.735+0000 7f3d26400640 1 --2- 172.21.15.22:0/2591195525 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f3d20102ff0 0x7f3d2010b4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:17:20.735 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:17:20.735+0000 7f3d26400640 1 --2- 172.21.15.22:0/2591195525 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3d201026d0 0x7f3d20102ab0 unknown :-1 s=CLOSED pgs=783 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:17:20.735 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:17:20.735+0000 7f3d26400640 1 -- 172.21.15.22:0/2591195525 >> 172.21.15.22:0/2591195525 conn(0x7f3d200fbef0 msgr2=0x7f3d200fe310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:17:20.735 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:17:20.735+0000 7f3d26400640 1 -- 172.21.15.22:0/2591195525 shutdown_connections 2024-03-28T18:17:20.735 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:17:20.735+0000 7f3d26400640 1 -- 172.21.15.22:0/2591195525 wait complete. 2024-03-28T18:17:20.736 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:17:20.736+0000 7f3d26400640 1 Processor -- start 2024-03-28T18:17:20.736 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:17:20.736+0000 7f3d26400640 1 -- start start 2024-03-28T18:17:20.738 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:17:20.737+0000 7f3d26400640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3d201026d0 0x7f3d20197e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:17:20.738 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:17:20.737+0000 7f3d26400640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f3d20102ff0 0x7f3d20198340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:17:20.738 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:17:20.737+0000 7f3d26400640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3d2010baf0 0x7f3d2019c6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:17:20.738 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:17:20.737+0000 7f3d26400640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d201118a0 con 0x7f3d201026d0 2024-03-28T18:17:20.738 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:17:20.737+0000 7f3d26400640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d20111720 con 0x7f3d20102ff0 2024-03-28T18:17:20.738 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:17:20.737+0000 7f3d26400640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d20111a20 con 0x7f3d2010baf0 2024-03-28T18:17:20.738 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:17:20.737+0000 7f3d25000640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3d201026d0 0x7f3d20197e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:17:20.738 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:17:20.737+0000 7f3d25000640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3d201026d0 0x7f3d20197e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:44772/0 (socket says 172.21.15.22:44772) 2024-03-28T18:17:20.738 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:17:20.737+0000 7f3d25000640 1 -- 172.21.15.22:0/1826915353 learned_addr learned my addr 172.21.15.22:0/1826915353 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-28T18:17:20.738 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:17:20.738+0000 7f3d1fe00640 1 --2- 172.21.15.22:0/1826915353 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f3d20102ff0 0x7f3d20198340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:17:20.738 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:17:20.738+0000 7f3d25a00640 1 --2- 172.21.15.22:0/1826915353 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3d2010baf0 0x7f3d2019c6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:17:20.739 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:17:20.738+0000 7f3d25000640 1 -- 172.21.15.22:0/1826915353 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3d2010baf0 msgr2=0x7f3d2019c6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:17:20.739 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:17:20.738+0000 7f3d25000640 1 --2- 172.21.15.22:0/1826915353 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3d2010baf0 0x7f3d2019c6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:17:20.739 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:17:20.738+0000 7f3d25000640 1 -- 172.21.15.22:0/1826915353 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f3d20102ff0 msgr2=0x7f3d20198340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:17:20.739 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:17:20.738+0000 7f3d25000640 1 --2- 172.21.15.22:0/1826915353 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f3d20102ff0 0x7f3d20198340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:17:20.739 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:17:20.738+0000 7f3d25000640 1 -- 172.21.15.22:0/1826915353 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3d2019cdf0 con 0x7f3d201026d0 2024-03-28T18:17:20.739 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:17:20.738+0000 7f3d1fe00640 1 --2- 172.21.15.22:0/1826915353 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f3d20102ff0 0x7f3d20198340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-28T18:17:20.739 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:17:20.738+0000 7f3d25000640 1 --2- 172.21.15.22:0/1826915353 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3d201026d0 0x7f3d20197e00 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 crypto rx=0x7f3d1002c7a0 tx=0x7f3d10034cf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:17:20.740 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:17:20.739+0000 7f3d1d600640 1 -- 172.21.15.22:0/1826915353 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d10039040 con 0x7f3d201026d0 2024-03-28T18:17:20.740 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:17:20.739+0000 7f3d1d600640 1 -- 172.21.15.22:0/1826915353 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f3d10003b70 con 0x7f3d201026d0 2024-03-28T18:17:20.740 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:17:20.739+0000 7f3d26400640 1 -- 172.21.15.22:0/1826915353 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3d2019d080 con 0x7f3d201026d0 2024-03-28T18:17:20.740 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:17:20.739+0000 7f3d1d600640 1 -- 172.21.15.22:0/1826915353 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d10032a30 con 0x7f3d201026d0 2024-03-28T18:17:20.740 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:17:20.739+0000 7f3d26400640 1 -- 172.21.15.22:0/1826915353 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3d201a4970 con 0x7f3d201026d0 2024-03-28T18:17:20.742 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:17:20.742+0000 7f3d1d600640 1 -- 172.21.15.22:0/1826915353 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 26) v1 ==== 137421+0+0 (secure 0 0 0) 0x7f3d100045a0 con 0x7f3d201026d0 2024-03-28T18:17:20.742 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:17:20.742+0000 7f3d2a7d3740 1 Processor -- start 2024-03-28T18:17:20.742 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:17:20.742+0000 7f3d2a7d3740 1 -- start start 2024-03-28T18:17:20.743 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:17:20.743+0000 7f3d2a7d3740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e5318da1e0 0x55e5318d8f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:17:20.744 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:17:20.743+0000 7f3d2a7d3740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55e5318d9500 0x55e5318d9980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:17:20.744 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:17:20.743+0000 7f3d2a7d3740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e5318e2870 0x55e5318e4c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:17:20.744 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:17:20.743+0000 7f3d2a7d3740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e531839250 con 0x55e5318da1e0 2024-03-28T18:17:20.744 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:17:20.743+0000 7f3d2a7d3740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e5317e1a30 con 0x55e5318d9500 2024-03-28T18:17:20.744 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:17:20.743+0000 7f3d2a7d3740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e5318352f0 con 0x55e5318e2870 2024-03-28T18:17:20.744 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:17:20.743+0000 7f3d25a00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e5318da1e0 0x55e5318d8f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:17:20.744 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:17:20.743+0000 7f3d25a00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e5318da1e0 0x55e5318d8f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:44782/0 (socket says 172.21.15.22:44782) 2024-03-28T18:17:20.744 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:17:20.743+0000 7f3d25a00640 1 -- 172.21.15.22:0/3037175194 learned_addr learned my addr 172.21.15.22:0/3037175194 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-28T18:17:20.744 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:17:20.743+0000 7f3d25000640 1 --2- 172.21.15.22:0/3037175194 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55e5318d9500 0x55e5318d9980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:17:20.744 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:17:20.743+0000 7f3d1fe00640 1 --2- 172.21.15.22:0/3037175194 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e5318e2870 0x55e5318e4c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:17:20.744 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:17:20.744+0000 7f3d25a00640 1 -- 172.21.15.22:0/3037175194 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e5318e2870 msgr2=0x55e5318e4c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T18:17:20.744 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:17:20.744+0000 7f3d25a00640 1 --2- 172.21.15.22:0/3037175194 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e5318e2870 0x55e5318e4c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:17:20.744 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:17:20.744+0000 7f3d25a00640 1 -- 172.21.15.22:0/3037175194 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55e5318d9500 msgr2=0x55e5318d9980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T18:17:20.744 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:17:20.744+0000 7f3d25a00640 1 --2- 172.21.15.22:0/3037175194 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55e5318d9500 0x55e5318d9980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:17:20.744 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:17:20.744+0000 7f3d25a00640 1 -- 172.21.15.22:0/3037175194 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e5317cc690 con 0x55e5318da1e0 2024-03-28T18:17:20.745 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:17:20.744+0000 7f3d1d600640 1 --2- 172.21.15.22:0/1826915353 >> [v2:172.21.15.22:6800/1357365082,v1:172.21.15.22:6801/1357365082] conn(0x7f3d040a29f0 0x7f3d040a4eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:17:20.745 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:17:20.744+0000 7f3d25a00640 1 --2- 172.21.15.22:0/3037175194 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e5318da1e0 0x55e5318d8f30 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 crypto rx=0x7f3d1800bba0 tx=0x7f3d1800f730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3f6070bcab9211f1 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:17:20.748 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:17:20.744+0000 7f3d25000640 1 --2- 172.21.15.22:0/1826915353 >> [v2:172.21.15.22:6800/1357365082,v1:172.21.15.22:6801/1357365082] conn(0x7f3d040a29f0 0x7f3d040a4eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:17:20.748 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:17:20.744+0000 7f3d16000640 1 -- 172.21.15.22:0/3037175194 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d18015070 con 0x55e5318da1e0 2024-03-28T18:17:20.748 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:17:20.744+0000 7f3d16000640 1 -- 172.21.15.22:0/3037175194 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f3d180054f0 con 0x55e5318da1e0 2024-03-28T18:17:20.748 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:17:20.744+0000 7f3d1d600640 1 -- 172.21.15.22:0/1826915353 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(122..122 src has 1..122) v4 ==== 10389+0+0 (secure 0 0 0) 0x7f3d100f1800 con 0x7f3d201026d0 2024-03-28T18:17:20.748 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:17:20.744+0000 7f3d16000640 1 -- 172.21.15.22:0/3037175194 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d180116a0 con 0x55e5318da1e0 2024-03-28T18:17:20.748 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:17:20.745+0000 7f3d2a7d3740 1 -- 172.21.15.22:0/3037175194 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e5318da1e0 msgr2=0x55e5318d8f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:17:20.748 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:17:20.745+0000 7f3d2a7d3740 1 --2- 172.21.15.22:0/3037175194 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e5318da1e0 0x55e5318d8f30 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 crypto rx=0x7f3d1800bba0 tx=0x7f3d1800f730 comp rx=0 tx=0).stop 2024-03-28T18:17:20.748 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:17:20.745+0000 7f3d25000640 1 --2- 172.21.15.22:0/1826915353 >> [v2:172.21.15.22:6800/1357365082,v1:172.21.15.22:6801/1357365082] conn(0x7f3d040a29f0 0x7f3d040a4eb0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x55e5318da730 tx=0x7f3d100f7530 comp rx=0 tx=0).ready entity=mgr.15522 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:17:20.749 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:17:20.745+0000 7f3d2a7d3740 1 -- 172.21.15.22:0/3037175194 shutdown_connections 2024-03-28T18:17:20.749 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:17:20.745+0000 7f3d2a7d3740 1 --2- 172.21.15.22:0/3037175194 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e5318e2870 0x55e5318e4c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:17:20.749 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:17:20.745+0000 7f3d2a7d3740 1 --2- 172.21.15.22:0/3037175194 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55e5318d9500 0x55e5318d9980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:17:20.749 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:17:20.745+0000 7f3d2a7d3740 1 --2- 172.21.15.22:0/3037175194 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e5318da1e0 0x55e5318d8f30 unknown :-1 s=CLOSED pgs=785 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:17:20.749 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:17:20.745+0000 7f3d2a7d3740 1 -- 172.21.15.22:0/3037175194 >> 172.21.15.22:0/3037175194 conn(0x55e53189c200 msgr2=0x55e5318d3f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:17:20.749 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:17:20.745+0000 7f3d2a7d3740 1 -- 172.21.15.22:0/3037175194 shutdown_connections 2024-03-28T18:17:20.749 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:17:20.745+0000 7f3d2a7d3740 1 -- 172.21.15.22:0/3037175194 wait complete. 2024-03-28T18:17:20.749 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:17:20.746+0000 7f3d2a7d3740 1 -- start start 2024-03-28T18:17:20.749 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:17:20.746+0000 7f3d2a7d3740 1 Processor -- start 2024-03-28T18:17:20.749 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:17:20.746+0000 7f3d2a7d3740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e5318d9500 0x55e5318e1fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:17:20.749 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:17:20.746+0000 7f3d2a7d3740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55e5318da1e0 0x55e5318df330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:17:20.749 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:17:20.746+0000 7f3d2a7d3740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e5318e2870 0x55e5318df870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:17:20.749 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:17:20.746+0000 7f3d2a7d3740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e5318e56e0 con 0x55e5318d9500 2024-03-28T18:17:20.750 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:17:20.746+0000 7f3d2a7d3740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e5317f4e80 con 0x55e5318da1e0 2024-03-28T18:17:20.750 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:17:20.746+0000 7f3d2a7d3740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e5318e5860 con 0x55e5318e2870 2024-03-28T18:17:20.750 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:17:20.747+0000 7f3d2a7d3740 10 client.0 _ll_register_callbacks cb 0x55e531807130 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-28T18:17:20.750 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:17:20.747+0000 7f3d25a00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e5318d9500 0x55e5318e1fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:17:20.750 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:17:20.747+0000 7f3d25a00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e5318d9500 0x55e5318e1fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:44792/0 (socket says 172.21.15.22:44792) 2024-03-28T18:17:20.750 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:17:20.747+0000 7f3d25a00640 1 -- 172.21.15.22:0/3106674480 learned_addr learned my addr 172.21.15.22:0/3106674480 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-28T18:17:20.750 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:17:20.747+0000 7f3d1fe00640 1 --2- 172.21.15.22:0/3106674480 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55e5318da1e0 0x55e5318df330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:17:20.750 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:17:20.747+0000 7f3d25a00640 1 -- 172.21.15.22:0/3106674480 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e5318e2870 msgr2=0x55e5318df870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T18:17:20.750 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:17:20.747+0000 7f3d25a00640 1 --2- 172.21.15.22:0/3106674480 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e5318e2870 0x55e5318df870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:17:20.750 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:17:20.747+0000 7f3d25a00640 1 -- 172.21.15.22:0/3106674480 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55e5318da1e0 msgr2=0x55e5318df330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T18:17:20.750 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:17:20.747+0000 7f3d25a00640 1 --2- 172.21.15.22:0/3106674480 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55e5318da1e0 0x55e5318df330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:17:20.750 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:17:20.747+0000 7f3d25a00640 1 -- 172.21.15.22:0/3106674480 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55e5316177d0 con 0x55e5318d9500 2024-03-28T18:17:20.750 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:17:20.748+0000 7f3d25a00640 1 --2- 172.21.15.22:0/3106674480 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e5318d9500 0x55e5318e1fa0 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 crypto rx=0x7f3d18004d20 tx=0x7f3d18004aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5dae5def4e7a1c6c server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:17:20.750 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:17:20.748+0000 7f3d0aa00640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-03-28T18:17:20.751 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:17:20.748+0000 7f3d0aa00640 1 -- 172.21.15.22:0/3106674480 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d18029060 con 0x55e5318d9500 2024-03-28T18:17:20.751 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:17:20.748+0000 7f3d2a7d3740 1 -- 172.21.15.22:0/3106674480 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f3d180054f0 con 0x55e5318d9500 2024-03-28T18:17:20.751 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:17:20.748+0000 7f3d0aa00640 1 -- 172.21.15.22:0/3106674480 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(122..122 src has 1..122) v4 ==== 10389+0+0 (secure 0 0 0) 0x7f3d1800bc90 con 0x55e5318d9500 2024-03-28T18:17:20.751 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:17:20.748+0000 7f3d0aa00640 1 -- 172.21.15.22:0/3106674480 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=187) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3d18015070 con 0x55e5318d9500 2024-03-28T18:17:20.751 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:17:20.748+0000 7f3d2a7d3740 10 client.16818 fetch_fsmap learned FSMap version 187 2024-03-28T18:17:20.751 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:17:20.748+0000 7f3d2a7d3740 1 -- 172.21.15.22:0/3106674480 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=187,monmap=4+}) v3 -- 0x55e53154d3f0 con 0x55e5318d9500 2024-03-28T18:17:20.751 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:17:20.749+0000 7f3d0aa00640 1 -- 172.21.15.22:0/3106674480 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f3d180054f0 con 0x55e5318d9500 2024-03-28T18:17:20.753 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:17:20.749+0000 7f3d0aa00640 1 -- 172.21.15.22:0/3106674480 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 187) v1 ==== 6221+0+0 (secure 0 0 0) 0x7f3d18025cc0 con 0x55e5318d9500 2024-03-28T18:17:20.753 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:17:20.749+0000 7f3d2a7d3740 10 client.16818 fetch_fsmap finished waiting for FSMap version 187 2024-03-28T18:17:20.753 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:17:20.749+0000 7f3d2a7d3740 10 client.16818 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-28T18:17:20.753 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:17:20.749+0000 7f3d2a7d3740 20 client.16818 populate_metadata read hostname 'smithi022' 2024-03-28T18:17:20.753 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:17:20.749+0000 7f3d2a7d3740 1 --2- 172.21.15.22:0/3106674480 >> [v2:172.21.15.79:6838/2441542443,v1:172.21.15.79:6839/2441542443] conn(0x55e5318f28a0 0x55e5318f4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:17:20.753 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:17:20.749+0000 7f3d2a7d3740 4 client.16818 mds_command: new command op to 34553 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-28T18:17:20.753 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:17:20.749+0000 7f3d2a7d3740 1 -- 172.21.15.22:0/3106674480 --> [v2:172.21.15.79:6838/2441542443,v1:172.21.15.79:6839/2441542443] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55e531846790 con 0x55e5318f28a0 2024-03-28T18:17:20.753 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:17:20.750+0000 7f3d1fe00640 1 --2- 172.21.15.22:0/3106674480 >> [v2:172.21.15.79:6838/2441542443,v1:172.21.15.79:6839/2441542443] conn(0x55e5318f28a0 0x55e5318f4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:17:20.753 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:17:20.750+0000 7f3d1fe00640 1 --2- 172.21.15.22:0/3106674480 >> [v2:172.21.15.79:6838/2441542443,v1:172.21.15.79:6839/2441542443] conn(0x55e5318f28a0 0x55e5318f4cc0 crc :-1 s=READY pgs=289 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=edd8df05bad40e32 server_cookie=b9e8aca54d9d643a in_seq=0 out_seq=0 2024-03-28T18:17:20.753 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:17:20.750+0000 7f3d0aa00640 10 client.16818 ms_handle_connect on v2:172.21.15.79:6838/2441542443 2024-03-28T18:17:20.754 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:17:20.752+0000 7f3d0aa00640 1 -- 172.21.15.22:0/3106674480 <== mds.0 v2:172.21.15.79:6838/2441542443 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x55e531846790 con 0x55e5318f28a0 2024-03-28T18:17:20.754 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:17:20.752+0000 7f3d0aa00640 10 client.16818 handle_command_reply: tid=0 2024-03-28T18:17:20.754 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:17:20.752+0000 7f3d2a7d3740 1 client.16818 shutdown 2024-03-28T18:17:20.754 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:17:20.753+0000 7f3d2a7d3740 1 -- 172.21.15.22:0/3106674480 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e5318d9500 msgr2=0x55e5318e1fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:17:20.754 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:17:20.753+0000 7f3d2a7d3740 1 --2- 172.21.15.22:0/3106674480 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e5318d9500 0x55e5318e1fa0 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 crypto rx=0x7f3d18004d20 tx=0x7f3d18004aa0 comp rx=0 tx=0).stop 2024-03-28T18:17:20.754 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:17:20.753+0000 7f3d2a7d3740 1 -- 172.21.15.22:0/3106674480 shutdown_connections 2024-03-28T18:17:20.754 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:17:20.753+0000 7f3d2a7d3740 1 --2- 172.21.15.22:0/3106674480 >> [v2:172.21.15.79:6838/2441542443,v1:172.21.15.79:6839/2441542443] conn(0x55e5318f28a0 0x55e5318f4cc0 crc :-1 s=READY pgs=289 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:17:20.754 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:17:20.753+0000 7f3d2a7d3740 1 --2- 172.21.15.22:0/3106674480 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e5318e2870 0x55e5318df870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:17:20.754 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:17:20.753+0000 7f3d2a7d3740 1 --2- 172.21.15.22:0/3106674480 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55e5318da1e0 0x55e5318df330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:17:20.754 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:17:20.753+0000 7f3d2a7d3740 1 --2- 172.21.15.22:0/3106674480 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e5318d9500 0x55e5318e1fa0 unknown :-1 s=CLOSED pgs=786 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:17:20.754 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:17:20.753+0000 7f3d0aa00640 0 client.16818 ms_handle_reset on v2:172.21.15.79:6838/2441542443 2024-03-28T18:17:20.754 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:17:20.753+0000 7f3d2a7d3740 1 -- 172.21.15.22:0/3106674480 >> 172.21.15.22:0/3106674480 conn(0x55e53189c200 msgr2=0x55e5318d5ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:17:20.754 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:17:20.753+0000 7f3d2a7d3740 1 -- 172.21.15.22:0/3106674480 shutdown_connections 2024-03-28T18:17:20.754 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:17:20.753+0000 7f3d2a7d3740 1 -- 172.21.15.22:0/3106674480 wait complete. 2024-03-28T18:17:20.754 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:17:20.753+0000 7f3d2a7d3740 20 client.16818 trim_cache size 0 max 16384 2024-03-28T18:17:20.770 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:17:20.771+0000 7f3d2a7d3740 1 Processor -- start 2024-03-28T18:17:20.770 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:17:20.771+0000 7f3d2a7d3740 1 -- start start 2024-03-28T18:17:20.771 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:17:20.771+0000 7f3d2a7d3740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55e5318d9500 0x55e5318e1a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:17:20.771 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:17:20.771+0000 7f3d2a7d3740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e5318da1e0 0x55e5318e1fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:17:20.772 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:17:20.771+0000 7f3d2a7d3740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e5318e2870 0x55e5318e3c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:17:20.772 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:17:20.771+0000 7f3d2a7d3740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d201ad8c0 con 0x55e5318e2870 2024-03-28T18:17:20.772 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:17:20.771+0000 7f3d2a7d3740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d201aac20 con 0x55e5318d9500 2024-03-28T18:17:20.772 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:17:20.771+0000 7f3d2a7d3740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d201acc00 con 0x55e5318da1e0 2024-03-28T18:17:20.772 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:17:20.772+0000 7f3d25a00640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55e5318d9500 0x55e5318e1a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:17:20.772 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:17:20.772+0000 7f3d25a00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e5318e2870 0x55e5318e3c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:17:20.772 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:17:20.772+0000 7f3d25a00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e5318e2870 0x55e5318e3c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:44808/0 (socket says 172.21.15.22:44808) 2024-03-28T18:17:20.772 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:17:20.772+0000 7f3d25a00640 1 -- 172.21.15.22:0/319269284 learned_addr learned my addr 172.21.15.22:0/319269284 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-28T18:17:20.772 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:17:20.772+0000 7f3d1fe00640 1 --2- 172.21.15.22:0/319269284 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e5318da1e0 0x55e5318e1fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:17:20.774 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:17:20.772+0000 7f3d25a00640 1 -- 172.21.15.22:0/319269284 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e5318da1e0 msgr2=0x55e5318e1fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T18:17:20.774 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:17:20.772+0000 7f3d25a00640 1 --2- 172.21.15.22:0/319269284 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e5318da1e0 0x55e5318e1fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:17:20.774 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:17:20.772+0000 7f3d25a00640 1 -- 172.21.15.22:0/319269284 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55e5318d9500 msgr2=0x55e5318e1a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T18:17:20.774 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:17:20.772+0000 7f3d25a00640 1 --2- 172.21.15.22:0/319269284 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55e5318d9500 0x55e5318e1a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:17:20.774 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:17:20.772+0000 7f3d25a00640 1 -- 172.21.15.22:0/319269284 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e53150d0f0 con 0x55e5318e2870 2024-03-28T18:17:20.774 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:17:20.773+0000 7f3d25a00640 1 --2- 172.21.15.22:0/319269284 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e5318e2870 0x55e5318e3c60 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 crypto rx=0x7f3d18004a70 tx=0x7f3d1800ffd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d77d74fa7d236d70 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:17:20.774 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:17:20.773+0000 7f3d15600640 1 -- 172.21.15.22:0/319269284 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d18029040 con 0x55e5318e2870 2024-03-28T18:17:20.774 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:17:20.773+0000 7f3d15600640 1 -- 172.21.15.22:0/319269284 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f3d1800fc30 con 0x55e5318e2870 2024-03-28T18:17:20.774 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:17:20.773+0000 7f3d15600640 1 -- 172.21.15.22:0/319269284 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d1800bc90 con 0x55e5318e2870 2024-03-28T18:17:20.774 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:17:20.773+0000 7f3d2a7d3740 1 -- 172.21.15.22:0/319269284 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e5318e2870 msgr2=0x55e5318e3c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:17:20.774 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:17:20.773+0000 7f3d2a7d3740 1 --2- 172.21.15.22:0/319269284 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e5318e2870 0x55e5318e3c60 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 crypto rx=0x7f3d18004a70 tx=0x7f3d1800ffd0 comp rx=0 tx=0).stop 2024-03-28T18:17:20.774 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:17:20.773+0000 7f3d2a7d3740 1 -- 172.21.15.22:0/319269284 shutdown_connections 2024-03-28T18:17:20.774 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:17:20.774+0000 7f3d2a7d3740 1 --2- 172.21.15.22:0/319269284 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e5318e2870 0x55e5318e3c60 unknown :-1 s=CLOSED pgs=787 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:17:20.775 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:17:20.774+0000 7f3d2a7d3740 1 --2- 172.21.15.22:0/319269284 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e5318da1e0 0x55e5318e1fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:17:20.775 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:17:20.774+0000 7f3d2a7d3740 1 --2- 172.21.15.22:0/319269284 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55e5318d9500 0x55e5318e1a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:17:20.775 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:17:20.774+0000 7f3d2a7d3740 1 -- 172.21.15.22:0/319269284 >> 172.21.15.22:0/319269284 conn(0x55e53189c200 msgr2=0x55e5318d3a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:17:20.775 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:17:20.774+0000 7f3d2a7d3740 1 -- 172.21.15.22:0/319269284 shutdown_connections 2024-03-28T18:17:20.775 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:17:20.774+0000 7f3d2a7d3740 1 -- 172.21.15.22:0/319269284 wait complete. 2024-03-28T18:17:20.775 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:17:20.774+0000 7f3d2a7d3740 1 -- start start 2024-03-28T18:17:20.775 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:17:20.774+0000 7f3d2a7d3740 1 Processor -- start 2024-03-28T18:17:20.777 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:17:20.775+0000 7f3d2a7d3740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55e5318d9500 0x55e5318ee560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:17:20.778 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:17:20.775+0000 7f3d2a7d3740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e5318da1e0 0x55e5318dc5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:17:20.778 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:17:20.775+0000 7f3d2a7d3740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e5318e2870 0x55e5318dcb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:17:20.778 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:17:20.775+0000 7f3d2a7d3740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d201ac5a0 con 0x55e5318e2870 2024-03-28T18:17:20.778 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:17:20.775+0000 7f3d2a7d3740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d201ab280 con 0x55e5318d9500 2024-03-28T18:17:20.778 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:17:20.775+0000 7f3d2a7d3740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d201ab8e0 con 0x55e5318da1e0 2024-03-28T18:17:20.778 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:17:20.775+0000 7f3d2a7d3740 10 client.0 _ll_register_callbacks cb 0x7f3d100f7400 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-28T18:17:20.778 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:17:20.775+0000 7f3d25a00640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55e5318d9500 0x55e5318ee560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:17:20.778 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:17:20.775+0000 7f3d25a00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e5318e2870 0x55e5318dcb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:17:20.778 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:17:20.775+0000 7f3d25a00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e5318e2870 0x55e5318dcb10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:44810/0 (socket says 172.21.15.22:44810) 2024-03-28T18:17:20.778 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:17:20.775+0000 7f3d25a00640 1 -- 172.21.15.22:0/1439169750 learned_addr learned my addr 172.21.15.22:0/1439169750 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-28T18:17:20.778 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:17:20.775+0000 7f3d1fe00640 1 --2- 172.21.15.22:0/1439169750 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e5318da1e0 0x55e5318dc5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:17:20.778 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:17:20.776+0000 7f3d25a00640 1 -- 172.21.15.22:0/1439169750 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e5318da1e0 msgr2=0x55e5318dc5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T18:17:20.778 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:17:20.776+0000 7f3d25a00640 1 --2- 172.21.15.22:0/1439169750 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e5318da1e0 0x55e5318dc5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:17:20.779 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:17:20.776+0000 7f3d25a00640 1 -- 172.21.15.22:0/1439169750 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55e5318d9500 msgr2=0x55e5318ee560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T18:17:20.779 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:17:20.776+0000 7f3d25a00640 1 --2- 172.21.15.22:0/1439169750 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55e5318d9500 0x55e5318ee560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:17:20.779 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:17:20.776+0000 7f3d25a00640 1 -- 172.21.15.22:0/1439169750 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55e5318dd250 con 0x55e5318e2870 2024-03-28T18:17:20.779 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:17:20.776+0000 7f3d1fe00640 1 --2- 172.21.15.22:0/1439169750 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e5318da1e0 0x55e5318dc5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-28T18:17:20.779 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:17:20.776+0000 7f3d25a00640 1 --2- 172.21.15.22:0/1439169750 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e5318e2870 0x55e5318dcb10 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 crypto rx=0x7f3d18017ed0 tx=0x7f3d18005d40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2438b9b6ff6efc5a server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:17:20.779 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:17:20.776+0000 7f3d0aa00640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-03-28T18:17:20.779 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:17:20.776+0000 7f3d0aa00640 1 -- 172.21.15.22:0/1439169750 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d1803a070 con 0x55e5318e2870 2024-03-28T18:17:20.779 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:17:20.776+0000 7f3d0aa00640 1 -- 172.21.15.22:0/1439169750 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(122..122 src has 1..122) v4 ==== 10389+0+0 (secure 0 0 0) 0x7f3d18011440 con 0x55e5318e2870 2024-03-28T18:17:20.779 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:17:20.776+0000 7f3d2a7d3740 1 -- 172.21.15.22:0/1439169750 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f3d1800fc30 con 0x55e5318e2870 2024-03-28T18:17:20.779 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:17:20.776+0000 7f3d0aa00640 1 -- 172.21.15.22:0/1439169750 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=187) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3d18049050 con 0x55e5318e2870 2024-03-28T18:17:20.779 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:17:20.776+0000 7f3d2a7d3740 10 client.16824 fetch_fsmap learned FSMap version 187 2024-03-28T18:17:20.779 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:17:20.777+0000 7f3d2a7d3740 1 -- 172.21.15.22:0/1439169750 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=187,monmap=4+}) v3 -- 0x55e53194e150 con 0x55e5318e2870 2024-03-28T18:17:20.779 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:17:20.777+0000 7f3d0aa00640 1 -- 172.21.15.22:0/1439169750 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f3d1800fc30 con 0x55e5318e2870 2024-03-28T18:17:20.779 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:17:20.777+0000 7f3d0aa00640 1 -- 172.21.15.22:0/1439169750 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 187) v1 ==== 6221+0+0 (secure 0 0 0) 0x7f3d180053d0 con 0x55e5318e2870 2024-03-28T18:17:20.780 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:17:20.777+0000 7f3d2a7d3740 10 client.16824 fetch_fsmap finished waiting for FSMap version 187 2024-03-28T18:17:20.780 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:17:20.777+0000 7f3d2a7d3740 10 client.16824 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-28T18:17:20.780 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:17:20.777+0000 7f3d2a7d3740 20 client.16824 populate_metadata read hostname 'smithi022' 2024-03-28T18:17:20.780 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:17:20.777+0000 7f3d2a7d3740 1 --2- 172.21.15.22:0/1439169750 >> [v2:172.21.15.79:6838/2441542443,v1:172.21.15.79:6839/2441542443] conn(0x55e53194e550 0x55e531950940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:17:20.780 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:17:20.777+0000 7f3d2a7d3740 4 client.16824 mds_command: new command op to 34553 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-28T18:17:20.780 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:17:20.777+0000 7f3d2a7d3740 1 -- 172.21.15.22:0/1439169750 --> [v2:172.21.15.79:6838/2441542443,v1:172.21.15.79:6839/2441542443] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55e531840ae0 con 0x55e53194e550 2024-03-28T18:17:20.780 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:17:20.777+0000 7f3d1fe00640 1 --2- 172.21.15.22:0/1439169750 >> [v2:172.21.15.79:6838/2441542443,v1:172.21.15.79:6839/2441542443] conn(0x55e53194e550 0x55e531950940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:17:20.780 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:17:20.778+0000 7f3d1fe00640 1 --2- 172.21.15.22:0/1439169750 >> [v2:172.21.15.79:6838/2441542443,v1:172.21.15.79:6839/2441542443] conn(0x55e53194e550 0x55e531950940 crc :-1 s=READY pgs=290 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e1b3cc0f3f6862fc server_cookie=33e004df21403df8 in_seq=0 out_seq=0 2024-03-28T18:17:20.780 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:17:20.778+0000 7f3d0aa00640 10 client.16824 ms_handle_connect on v2:172.21.15.79:6838/2441542443 2024-03-28T18:17:20.781 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:17:20.779+0000 7f3d0aa00640 1 -- 172.21.15.22:0/1439169750 <== mds.0 v2:172.21.15.79:6838/2441542443 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x55e531840ae0 con 0x55e53194e550 2024-03-28T18:17:20.781 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:17:20.779+0000 7f3d0aa00640 10 client.16824 handle_command_reply: tid=0 2024-03-28T18:17:20.781 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:17:20.779+0000 7f3d2a7d3740 1 client.16824 shutdown 2024-03-28T18:17:20.781 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:17:20.779+0000 7f3d2a7d3740 1 -- 172.21.15.22:0/1439169750 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e5318e2870 msgr2=0x55e5318dcb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:17:20.781 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:17:20.779+0000 7f3d2a7d3740 1 --2- 172.21.15.22:0/1439169750 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e5318e2870 0x55e5318dcb10 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 crypto rx=0x7f3d18017ed0 tx=0x7f3d18005d40 comp rx=0 tx=0).stop 2024-03-28T18:17:20.781 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:17:20.779+0000 7f3d2a7d3740 1 -- 172.21.15.22:0/1439169750 shutdown_connections 2024-03-28T18:17:20.782 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:17:20.779+0000 7f3d2a7d3740 1 --2- 172.21.15.22:0/1439169750 >> [v2:172.21.15.79:6838/2441542443,v1:172.21.15.79:6839/2441542443] conn(0x55e53194e550 0x55e531950940 crc :-1 s=READY pgs=290 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:17:20.782 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:17:20.779+0000 7f3d2a7d3740 1 --2- 172.21.15.22:0/1439169750 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e5318e2870 0x55e5318dcb10 unknown :-1 s=CLOSED pgs=788 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:17:20.782 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:17:20.779+0000 7f3d2a7d3740 1 --2- 172.21.15.22:0/1439169750 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e5318da1e0 0x55e5318dc5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:17:20.782 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:17:20.779+0000 7f3d2a7d3740 1 --2- 172.21.15.22:0/1439169750 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55e5318d9500 0x55e5318ee560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:17:20.782 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:17:20.779+0000 7f3d0aa00640 0 client.16824 ms_handle_reset on v2:172.21.15.79:6838/2441542443 2024-03-28T18:17:20.782 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:17:20.779+0000 7f3d2a7d3740 1 -- 172.21.15.22:0/1439169750 >> 172.21.15.22:0/1439169750 conn(0x55e53189c200 msgr2=0x55e5318ddc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:17:20.782 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:17:20.779+0000 7f3d2a7d3740 1 -- 172.21.15.22:0/1439169750 shutdown_connections 2024-03-28T18:17:20.782 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:17:20.779+0000 7f3d2a7d3740 1 -- 172.21.15.22:0/1439169750 wait complete. 2024-03-28T18:17:20.782 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:17:20.779+0000 7f3d2a7d3740 20 client.16824 trim_cache size 0 max 16384 2024-03-28T18:17:20.782 INFO:teuthology.orchestra.run.smithi022.stdout:{ 2024-03-28T18:17:20.783 INFO:teuthology.orchestra.run.smithi022.stdout: "status": "IDLE (waiting for more scrubs)", 2024-03-28T18:17:20.783 INFO:teuthology.orchestra.run.smithi022.stdout: "scrubs": { 2024-03-28T18:17:20.783 INFO:teuthology.orchestra.run.smithi022.stdout: "4bbbd61d-29a3-4bf6-a4f2-d5783d585de3": { 2024-03-28T18:17:20.783 INFO:teuthology.orchestra.run.smithi022.stdout: "path": "/", 2024-03-28T18:17:20.783 INFO:teuthology.orchestra.run.smithi022.stdout: "tag": "4bbbd61d-29a3-4bf6-a4f2-d5783d585de3", 2024-03-28T18:17:20.783 INFO:teuthology.orchestra.run.smithi022.stdout: "options": "recursive,force" 2024-03-28T18:17:20.783 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-03-28T18:17:20.783 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-03-28T18:17:20.783 INFO:teuthology.orchestra.run.smithi022.stdout:} 2024-03-28T18:17:20.784 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:17:20.780+0000 7f3d26400640 1 -- 172.21.15.22:0/1826915353 >> [v2:172.21.15.22:6800/1357365082,v1:172.21.15.22:6801/1357365082] conn(0x7f3d040a29f0 msgr2=0x7f3d040a4eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:17:20.784 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:17:20.780+0000 7f3d26400640 1 --2- 172.21.15.22:0/1826915353 >> [v2:172.21.15.22:6800/1357365082,v1:172.21.15.22:6801/1357365082] conn(0x7f3d040a29f0 0x7f3d040a4eb0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x55e5318da730 tx=0x7f3d100f7530 comp rx=0 tx=0).stop 2024-03-28T18:17:20.784 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:17:20.780+0000 7f3d26400640 1 -- 172.21.15.22:0/1826915353 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3d201026d0 msgr2=0x7f3d20197e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:17:20.784 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:17:20.780+0000 7f3d26400640 1 --2- 172.21.15.22:0/1826915353 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3d201026d0 0x7f3d20197e00 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 crypto rx=0x7f3d1002c7a0 tx=0x7f3d10034cf0 comp rx=0 tx=0).stop 2024-03-28T18:17:20.784 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:17:20.780+0000 7f3d26400640 1 -- 172.21.15.22:0/1826915353 shutdown_connections 2024-03-28T18:17:20.784 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:17:20.780+0000 7f3d26400640 1 --2- 172.21.15.22:0/1826915353 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3d2010baf0 0x7f3d2019c6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:17:20.784 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:17:20.780+0000 7f3d26400640 1 --2- 172.21.15.22:0/1826915353 >> [v2:172.21.15.22:6800/1357365082,v1:172.21.15.22:6801/1357365082] conn(0x7f3d040a29f0 0x7f3d040a4eb0 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:17:20.784 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:17:20.780+0000 7f3d26400640 1 --2- 172.21.15.22:0/1826915353 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f3d20102ff0 0x7f3d20198340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:17:20.784 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:17:20.780+0000 7f3d26400640 1 --2- 172.21.15.22:0/1826915353 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3d201026d0 0x7f3d20197e00 unknown :-1 s=CLOSED pgs=784 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:17:20.784 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:17:20.780+0000 7f3d26400640 1 -- 172.21.15.22:0/1826915353 >> 172.21.15.22:0/1826915353 conn(0x7f3d200fbef0 msgr2=0x7f3d200fcd00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:17:20.784 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:17:20.780+0000 7f3d26400640 1 -- 172.21.15.22:0/1826915353 shutdown_connections 2024-03-28T18:17:20.785 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:17:20.780+0000 7f3d26400640 1 -- 172.21.15.22:0/1826915353 wait complete. 2024-03-28T18:17:20.785 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:17:20 smithi022 ceph-mon[31985]: pgmap v325: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:17:20.799 INFO:tasks.cephfs.filesystem:scrub status for tag:4bbbd61d-29a3-4bf6-a4f2-d5783d585de3 - {'path': '/', 'tag': '4bbbd61d-29a3-4bf6-a4f2-d5783d585de3', 'options': 'recursive,force'} 2024-03-28T18:17:20.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:17:20 smithi134 ceph-mon[35037]: pgmap v325: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:17:20.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:17:20 smithi079 ceph-mon[35312]: pgmap v325: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:17:22.781 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:17:22 smithi022 ceph-mon[31985]: pgmap v326: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:17:22.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:17:22 smithi134 ceph-mon[35037]: pgmap v326: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:17:22.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:17:22 smithi079 ceph-mon[35312]: pgmap v326: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:17:24.781 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:17:24 smithi022 ceph-mon[31985]: pgmap v327: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:17:24.925 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:17:24 smithi134 ceph-mon[35037]: pgmap v327: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:17:24.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:17:24 smithi079 ceph-mon[35312]: pgmap v327: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:17:26.781 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:17:26 smithi022 ceph-mon[31985]: pgmap v328: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:17:26.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:17:26 smithi134 ceph-mon[35037]: pgmap v328: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:17:26.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:17:26 smithi079 ceph-mon[35312]: pgmap v328: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:17:28.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:17:28 smithi022 ceph-mon[31985]: pgmap v329: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:17:28.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:17:28 smithi134 ceph-mon[35037]: pgmap v329: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:17:28.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:17:28 smithi079 ceph-mon[35312]: pgmap v329: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:17:30.727 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:17:30 smithi134 ceph-mon[35037]: pgmap v330: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:17:30.781 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:17:30 smithi022 ceph-mon[31985]: pgmap v330: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:17:30.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:17:30 smithi079 ceph-mon[35312]: pgmap v330: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:17:32.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:17:32 smithi022 ceph-mon[31985]: pgmap v331: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:17:32.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:17:32 smithi134 ceph-mon[35037]: pgmap v331: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:17:32.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:17:32 smithi079 ceph-mon[35312]: pgmap v331: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:17:34.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:17:34 smithi022 ceph-mon[31985]: pgmap v332: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:17:34.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:17:34 smithi134 ceph-mon[35037]: pgmap v332: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:17:34.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:17:34 smithi079 ceph-mon[35312]: pgmap v332: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:17:36.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:17:36 smithi022 ceph-mon[31985]: pgmap v333: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:17:36.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:17:36 smithi134 ceph-mon[35037]: pgmap v333: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:17:36.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:17:36 smithi079 ceph-mon[35312]: pgmap v333: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:17:38.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:17:38 smithi022 ceph-mon[31985]: pgmap v334: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:17:38.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:17:38 smithi134 ceph-mon[35037]: pgmap v334: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:17:38.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:17:38 smithi079 ceph-mon[35312]: pgmap v334: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:17:40.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:17:40 smithi022 ceph-mon[31985]: pgmap v335: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:17:40.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:17:40 smithi134 ceph-mon[35037]: pgmap v335: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:17:40.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:17:40 smithi079 ceph-mon[35312]: pgmap v335: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:17:42.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:17:42 smithi022 ceph-mon[31985]: pgmap v336: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:17:42.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:17:42 smithi134 ceph-mon[35037]: pgmap v336: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:17:42.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:17:42 smithi079 ceph-mon[35312]: pgmap v336: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:17:44.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:17:44 smithi022 ceph-mon[31985]: pgmap v337: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:17:44.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:17:44 smithi134 ceph-mon[35037]: pgmap v337: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:17:44.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:17:44 smithi079 ceph-mon[35312]: pgmap v337: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:17:46.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:17:46 smithi022 ceph-mon[31985]: pgmap v338: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:17:46.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:17:46 smithi134 ceph-mon[35037]: pgmap v338: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:17:46.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:17:46 smithi079 ceph-mon[35312]: pgmap v338: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:17:48.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:17:48 smithi022 ceph-mon[31985]: pgmap v339: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:17:48.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:17:48 smithi134 ceph-mon[35037]: pgmap v339: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:17:48.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:17:48 smithi079 ceph-mon[35312]: pgmap v339: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:17:50.799 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-28T18:17:50.800 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-28T18:17:50.894 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:17:50 smithi022 ceph-mon[31985]: pgmap v340: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:17:50.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:17:50 smithi134 ceph-mon[35037]: pgmap v340: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:17:50.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:17:50 smithi079 ceph-mon[35312]: pgmap v340: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:17:51.042 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:17:51.042+0000 7fd669e00640 1 -- 172.21.15.22:0/3450661343 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd664108800 msgr2=0x7fd66410c7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:17:51.042 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:17:51.042+0000 7fd669e00640 1 --2- 172.21.15.22:0/3450661343 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd664108800 0x7fd66410c7c0 secure :-1 s=READY pgs=789 cs=0 l=1 rev1=1 crypto rx=0x7fd64c009900 tx=0x7fd64c02bc40 comp rx=0 tx=0).stop 2024-03-28T18:17:51.042 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:17:51.042+0000 7fd669e00640 1 -- 172.21.15.22:0/3450661343 shutdown_connections 2024-03-28T18:17:51.042 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:17:51.042+0000 7fd669e00640 1 --2- 172.21.15.22:0/3450661343 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd66410cea0 0x7fd664114a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:17:51.042 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:17:51.042+0000 7fd669e00640 1 --2- 172.21.15.22:0/3450661343 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd664108800 0x7fd66410c7c0 unknown :-1 s=CLOSED pgs=789 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:17:51.042 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:17:51.042+0000 7fd669e00640 1 --2- 172.21.15.22:0/3450661343 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd664107e50 0x7fd664108230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:17:51.042 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:17:51.042+0000 7fd669e00640 1 -- 172.21.15.22:0/3450661343 >> 172.21.15.22:0/3450661343 conn(0x7fd6640774c0 msgr2=0x7fd6641064c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:17:51.042 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:17:51.042+0000 7fd669e00640 1 -- 172.21.15.22:0/3450661343 shutdown_connections 2024-03-28T18:17:51.042 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:17:51.042+0000 7fd669e00640 1 -- 172.21.15.22:0/3450661343 wait complete. 2024-03-28T18:17:51.043 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:17:51.043+0000 7fd669e00640 1 Processor -- start 2024-03-28T18:17:51.043 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:17:51.043+0000 7fd669e00640 1 -- start start 2024-03-28T18:17:51.045 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:17:51.044+0000 7fd669e00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd664107e50 0x7fd66419e470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:17:51.045 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:17:51.044+0000 7fd669e00640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd66410cea0 0x7fd66419e9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:17:51.045 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:17:51.044+0000 7fd669e00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd6641a2d30 0x7fd6641a31e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:17:51.045 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:17:51.044+0000 7fd669e00640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd664117f70 con 0x7fd664107e50 2024-03-28T18:17:51.046 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:17:51.044+0000 7fd669e00640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd664117df0 con 0x7fd66410cea0 2024-03-28T18:17:51.046 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:17:51.044+0000 7fd669e00640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6641180f0 con 0x7fd6641a2d30 2024-03-28T18:17:51.046 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:17:51.044+0000 7fd668a00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd664107e50 0x7fd66419e470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:17:51.046 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:17:51.044+0000 7fd668a00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd664107e50 0x7fd66419e470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:59160/0 (socket says 172.21.15.22:59160) 2024-03-28T18:17:51.046 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:17:51.044+0000 7fd668a00640 1 -- 172.21.15.22:0/1233136251 learned_addr learned my addr 172.21.15.22:0/1233136251 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-28T18:17:51.046 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:17:51.045+0000 7fd65be00640 1 --2- 172.21.15.22:0/1233136251 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd66410cea0 0x7fd66419e9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:17:51.046 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:17:51.045+0000 7fd669400640 1 --2- 172.21.15.22:0/1233136251 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd6641a2d30 0x7fd6641a31e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:17:51.046 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:17:51.045+0000 7fd668a00640 1 -- 172.21.15.22:0/1233136251 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd6641a2d30 msgr2=0x7fd6641a31e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:17:51.046 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:17:51.045+0000 7fd668a00640 1 --2- 172.21.15.22:0/1233136251 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd6641a2d30 0x7fd6641a31e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:17:51.046 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:17:51.045+0000 7fd668a00640 1 -- 172.21.15.22:0/1233136251 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd66410cea0 msgr2=0x7fd66419e9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:17:51.046 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:17:51.045+0000 7fd668a00640 1 --2- 172.21.15.22:0/1233136251 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd66410cea0 0x7fd66419e9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:17:51.046 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:17:51.045+0000 7fd668a00640 1 -- 172.21.15.22:0/1233136251 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd6641a3820 con 0x7fd664107e50 2024-03-28T18:17:51.046 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:17:51.045+0000 7fd669400640 1 --2- 172.21.15.22:0/1233136251 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd6641a2d30 0x7fd6641a31e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T18:17:51.046 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:17:51.045+0000 7fd65be00640 1 --2- 172.21.15.22:0/1233136251 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd66410cea0 0x7fd66419e9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-28T18:17:51.047 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:17:51.045+0000 7fd668a00640 1 --2- 172.21.15.22:0/1233136251 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd664107e50 0x7fd66419e470 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 crypto rx=0x7fd65400d770 tx=0x7fd65400dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:17:51.047 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:17:51.045+0000 7fd662000640 1 -- 172.21.15.22:0/1233136251 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd654017070 con 0x7fd664107e50 2024-03-28T18:17:51.047 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:17:51.045+0000 7fd662000640 1 -- 172.21.15.22:0/1233136251 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7fd6540054f0 con 0x7fd664107e50 2024-03-28T18:17:51.047 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:17:51.045+0000 7fd669e00640 1 -- 172.21.15.22:0/1233136251 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd6641a3b10 con 0x7fd664107e50 2024-03-28T18:17:51.047 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:17:51.046+0000 7fd662000640 1 -- 172.21.15.22:0/1233136251 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd654004d60 con 0x7fd664107e50 2024-03-28T18:17:51.047 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:17:51.046+0000 7fd669e00640 1 -- 172.21.15.22:0/1233136251 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd6641ab270 con 0x7fd664107e50 2024-03-28T18:17:51.050 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:17:51.050+0000 7fd66e188740 1 Processor -- start 2024-03-28T18:17:51.050 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:17:51.050+0000 7fd662000640 1 -- 172.21.15.22:0/1233136251 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 26) v1 ==== 137421+0+0 (secure 0 0 0) 0x7fd654004f00 con 0x7fd664107e50 2024-03-28T18:17:51.050 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:17:51.050+0000 7fd66e188740 1 -- start start 2024-03-28T18:17:51.051 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:17:51.051+0000 7fd66e188740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x555eca6d3dd0 0x555eca6d2b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:17:51.051 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:17:51.051+0000 7fd66e188740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555eca6d30f0 0x555eca6d3570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:17:51.051 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:17:51.051+0000 7fd66e188740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x555eca6dc460 0x555eca6de850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:17:51.052 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:17:51.051+0000 7fd66e188740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x555eca637250 con 0x555eca6dc460 2024-03-28T18:17:51.052 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:17:51.051+0000 7fd66e188740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x555eca5dfa30 con 0x555eca6d3dd0 2024-03-28T18:17:51.052 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:17:51.051+0000 7fd66e188740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x555eca6332f0 con 0x555eca6d30f0 2024-03-28T18:17:51.052 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:17:51.051+0000 7fd65be00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x555eca6dc460 0x555eca6de850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:17:51.052 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:17:51.051+0000 7fd65be00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x555eca6dc460 0x555eca6de850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:59174/0 (socket says 172.21.15.22:59174) 2024-03-28T18:17:51.052 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:17:51.051+0000 7fd65be00640 1 -- 172.21.15.22:0/1454527873 learned_addr learned my addr 172.21.15.22:0/1454527873 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-28T18:17:51.052 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:17:51.051+0000 7fd669400640 1 --2- 172.21.15.22:0/1454527873 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x555eca6d3dd0 0x555eca6d2b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:17:51.052 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:17:51.051+0000 7fd668a00640 1 --2- 172.21.15.22:0/1454527873 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555eca6d30f0 0x555eca6d3570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:17:51.052 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:17:51.052+0000 7fd65be00640 1 -- 172.21.15.22:0/1454527873 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555eca6d30f0 msgr2=0x555eca6d3570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T18:17:51.052 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:17:51.052+0000 7fd65be00640 1 --2- 172.21.15.22:0/1454527873 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555eca6d30f0 0x555eca6d3570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:17:51.052 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:17:51.052+0000 7fd65be00640 1 -- 172.21.15.22:0/1454527873 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x555eca6d3dd0 msgr2=0x555eca6d2b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T18:17:51.052 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:17:51.052+0000 7fd65be00640 1 --2- 172.21.15.22:0/1454527873 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x555eca6d3dd0 0x555eca6d2b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:17:51.052 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:17:51.052+0000 7fd65be00640 1 -- 172.21.15.22:0/1454527873 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555eca5ca690 con 0x555eca6dc460 2024-03-28T18:17:51.054 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:17:51.052+0000 7fd65be00640 1 --2- 172.21.15.22:0/1454527873 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x555eca6dc460 0x555eca6de850 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 crypto rx=0x7fd664108dc0 tx=0x7fd64c005e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=227eac08caa6baa3 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:17:51.054 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:17:51.052+0000 7fd662000640 1 --2- 172.21.15.22:0/1233136251 >> [v2:172.21.15.22:6800/1357365082,v1:172.21.15.22:6801/1357365082] conn(0x7fd6480a29f0 0x7fd6480a4eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:17:51.054 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:17:51.052+0000 7fd659c00640 1 -- 172.21.15.22:0/1454527873 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd64c004440 con 0x555eca6dc460 2024-03-28T18:17:51.054 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:17:51.052+0000 7fd662000640 1 -- 172.21.15.22:0/1233136251 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(122..122 src has 1..122) v4 ==== 10389+0+0 (secure 0 0 0) 0x7fd6540cf060 con 0x7fd664107e50 2024-03-28T18:17:51.054 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:17:51.053+0000 7fd659c00640 1 -- 172.21.15.22:0/1454527873 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7fd64c005500 con 0x555eca6dc460 2024-03-28T18:17:51.054 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:17:51.053+0000 7fd659c00640 1 -- 172.21.15.22:0/1454527873 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd64c0048c0 con 0x555eca6dc460 2024-03-28T18:17:51.054 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:17:51.053+0000 7fd669400640 1 --2- 172.21.15.22:0/1233136251 >> [v2:172.21.15.22:6800/1357365082,v1:172.21.15.22:6801/1357365082] conn(0x7fd6480a29f0 0x7fd6480a4eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:17:51.054 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:17:51.053+0000 7fd66e188740 1 -- 172.21.15.22:0/1454527873 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x555eca6dc460 msgr2=0x555eca6de850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:17:51.054 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:17:51.053+0000 7fd669400640 1 --2- 172.21.15.22:0/1233136251 >> [v2:172.21.15.22:6800/1357365082,v1:172.21.15.22:6801/1357365082] conn(0x7fd6480a29f0 0x7fd6480a4eb0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fd65c005fd0 tx=0x7fd65c0095b0 comp rx=0 tx=0).ready entity=mgr.15522 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:17:51.054 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:17:51.053+0000 7fd66e188740 1 --2- 172.21.15.22:0/1454527873 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x555eca6dc460 0x555eca6de850 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 crypto rx=0x7fd664108dc0 tx=0x7fd64c005e80 comp rx=0 tx=0).stop 2024-03-28T18:17:51.055 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:17:51.053+0000 7fd66e188740 1 -- 172.21.15.22:0/1454527873 shutdown_connections 2024-03-28T18:17:51.055 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:17:51.053+0000 7fd66e188740 1 --2- 172.21.15.22:0/1454527873 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x555eca6dc460 0x555eca6de850 unknown :-1 s=CLOSED pgs=791 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:17:51.055 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:17:51.053+0000 7fd66e188740 1 --2- 172.21.15.22:0/1454527873 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555eca6d30f0 0x555eca6d3570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:17:51.055 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:17:51.053+0000 7fd66e188740 1 --2- 172.21.15.22:0/1454527873 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x555eca6d3dd0 0x555eca6d2b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:17:51.055 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:17:51.053+0000 7fd66e188740 1 -- 172.21.15.22:0/1454527873 >> 172.21.15.22:0/1454527873 conn(0x555eca69a200 msgr2=0x555eca6cdb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:17:51.055 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:17:51.054+0000 7fd66e188740 1 -- 172.21.15.22:0/1454527873 shutdown_connections 2024-03-28T18:17:51.055 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:17:51.054+0000 7fd66e188740 1 -- 172.21.15.22:0/1454527873 wait complete. 2024-03-28T18:17:51.055 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:17:51.054+0000 7fd66e188740 1 -- start start 2024-03-28T18:17:51.055 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:17:51.054+0000 7fd66e188740 1 Processor -- start 2024-03-28T18:17:51.057 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:17:51.055+0000 7fd66e188740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x555eca6d30f0 0x555eca6dbb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:17:51.058 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:17:51.055+0000 7fd66e188740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x555eca6d3dd0 0x555eca6d8ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:17:51.058 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:17:51.055+0000 7fd66e188740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555eca6dc460 0x555eca6d9420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:17:51.058 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:17:51.055+0000 7fd66e188740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x555eca6df270 con 0x555eca6d3dd0 2024-03-28T18:17:51.058 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:17:51.055+0000 7fd66e188740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x555eca5f2e80 con 0x555eca6d30f0 2024-03-28T18:17:51.058 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:17:51.055+0000 7fd66e188740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x555eca6df3f0 con 0x555eca6dc460 2024-03-28T18:17:51.058 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:17:51.055+0000 7fd66e188740 10 client.0 _ll_register_callbacks cb 0x555eca605130 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-28T18:17:51.058 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:17:51.055+0000 7fd65be00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x555eca6d3dd0 0x555eca6d8ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:17:51.058 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:17:51.055+0000 7fd65be00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x555eca6d3dd0 0x555eca6d8ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:59188/0 (socket says 172.21.15.22:59188) 2024-03-28T18:17:51.058 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:17:51.055+0000 7fd65be00640 1 -- 172.21.15.22:0/3883000114 learned_addr learned my addr 172.21.15.22:0/3883000114 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-28T18:17:51.058 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:17:51.055+0000 7fd668a00640 1 --2- 172.21.15.22:0/3883000114 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x555eca6d30f0 0x555eca6dbb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:17:51.058 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:17:51.055+0000 7fd669400640 1 --2- 172.21.15.22:0/3883000114 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555eca6dc460 0x555eca6d9420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:17:51.058 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:17:51.055+0000 7fd65be00640 1 -- 172.21.15.22:0/3883000114 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555eca6dc460 msgr2=0x555eca6d9420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T18:17:51.059 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:17:51.055+0000 7fd65be00640 1 --2- 172.21.15.22:0/3883000114 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555eca6dc460 0x555eca6d9420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:17:51.059 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:17:51.055+0000 7fd65be00640 1 -- 172.21.15.22:0/3883000114 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x555eca6d30f0 msgr2=0x555eca6dbb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T18:17:51.059 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:17:51.055+0000 7fd65be00640 1 --2- 172.21.15.22:0/3883000114 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x555eca6d30f0 0x555eca6dbb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:17:51.059 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:17:51.055+0000 7fd65be00640 1 -- 172.21.15.22:0/3883000114 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x555eca4157d0 con 0x555eca6d3dd0 2024-03-28T18:17:51.059 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:17:51.055+0000 7fd669400640 1 --2- 172.21.15.22:0/3883000114 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555eca6dc460 0x555eca6d9420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T18:17:51.059 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:17:51.056+0000 7fd65be00640 1 --2- 172.21.15.22:0/3883000114 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x555eca6d3dd0 0x555eca6d8ee0 secure :-1 s=READY pgs=792 cs=0 l=1 rev1=1 crypto rx=0x7fd64c02c180 tx=0x7fd64c02c890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6b3d0cf0bd468543 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:17:51.059 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:17:51.056+0000 7fd652000640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-03-28T18:17:51.059 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:17:51.056+0000 7fd652000640 1 -- 172.21.15.22:0/3883000114 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd64c003800 con 0x555eca6d3dd0 2024-03-28T18:17:51.059 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:17:51.056+0000 7fd652000640 1 -- 172.21.15.22:0/3883000114 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(122..122 src has 1..122) v4 ==== 10389+0+0 (secure 0 0 0) 0x7fd64c0039a0 con 0x555eca6d3dd0 2024-03-28T18:17:51.059 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:17:51.056+0000 7fd66e188740 1 -- 172.21.15.22:0/3883000114 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fd64c005500 con 0x555eca6d3dd0 2024-03-28T18:17:51.059 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:17:51.057+0000 7fd652000640 1 -- 172.21.15.22:0/3883000114 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=187) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd64c05a050 con 0x555eca6d3dd0 2024-03-28T18:17:51.059 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:17:51.057+0000 7fd66e188740 10 client.16836 fetch_fsmap learned FSMap version 187 2024-03-28T18:17:51.059 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:17:51.057+0000 7fd66e188740 1 -- 172.21.15.22:0/3883000114 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=187,monmap=4+}) v3 -- 0x555eca34b3f0 con 0x555eca6d3dd0 2024-03-28T18:17:51.059 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:17:51.057+0000 7fd652000640 1 -- 172.21.15.22:0/3883000114 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7fd64c005500 con 0x555eca6d3dd0 2024-03-28T18:17:51.059 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:17:51.057+0000 7fd652000640 1 -- 172.21.15.22:0/3883000114 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 187) v1 ==== 6221+0+0 (secure 0 0 0) 0x7fd64c034440 con 0x555eca6d3dd0 2024-03-28T18:17:51.060 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:17:51.057+0000 7fd66e188740 10 client.16836 fetch_fsmap finished waiting for FSMap version 187 2024-03-28T18:17:51.060 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:17:51.057+0000 7fd66e188740 10 client.16836 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-28T18:17:51.060 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:17:51.057+0000 7fd66e188740 20 client.16836 populate_metadata read hostname 'smithi022' 2024-03-28T18:17:51.060 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:17:51.057+0000 7fd66e188740 1 --2- 172.21.15.22:0/3883000114 >> [v2:172.21.15.79:6838/2441542443,v1:172.21.15.79:6839/2441542443] conn(0x555eca6ec1a0 0x555eca6ee590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:17:51.060 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:17:51.057+0000 7fd66e188740 4 client.16836 mds_command: new command op to 34553 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-28T18:17:51.060 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:17:51.057+0000 7fd66e188740 1 -- 172.21.15.22:0/3883000114 --> [v2:172.21.15.79:6838/2441542443,v1:172.21.15.79:6839/2441542443] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x555eca644790 con 0x555eca6ec1a0 2024-03-28T18:17:51.060 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:17:51.058+0000 7fd668a00640 1 --2- 172.21.15.22:0/3883000114 >> [v2:172.21.15.79:6838/2441542443,v1:172.21.15.79:6839/2441542443] conn(0x555eca6ec1a0 0x555eca6ee590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:17:51.061 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:17:51.059+0000 7fd668a00640 1 --2- 172.21.15.22:0/3883000114 >> [v2:172.21.15.79:6838/2441542443,v1:172.21.15.79:6839/2441542443] conn(0x555eca6ec1a0 0x555eca6ee590 crc :-1 s=READY pgs=293 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6412ccd60c4266c1 server_cookie=df7d15d51c5805f3 in_seq=0 out_seq=0 2024-03-28T18:17:51.061 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:17:51.059+0000 7fd652000640 10 client.16836 ms_handle_connect on v2:172.21.15.79:6838/2441542443 2024-03-28T18:17:51.061 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:17:51.060+0000 7fd652000640 1 -- 172.21.15.22:0/3883000114 <== mds.0 v2:172.21.15.79:6838/2441542443 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x555eca644790 con 0x555eca6ec1a0 2024-03-28T18:17:51.061 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:17:51.060+0000 7fd652000640 10 client.16836 handle_command_reply: tid=0 2024-03-28T18:17:51.061 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:17:51.060+0000 7fd66e188740 1 client.16836 shutdown 2024-03-28T18:17:51.061 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:17:51.060+0000 7fd66e188740 1 -- 172.21.15.22:0/3883000114 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x555eca6d3dd0 msgr2=0x555eca6d8ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:17:51.061 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:17:51.060+0000 7fd66e188740 1 --2- 172.21.15.22:0/3883000114 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x555eca6d3dd0 0x555eca6d8ee0 secure :-1 s=READY pgs=792 cs=0 l=1 rev1=1 crypto rx=0x7fd64c02c180 tx=0x7fd64c02c890 comp rx=0 tx=0).stop 2024-03-28T18:17:51.061 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:17:51.060+0000 7fd66e188740 1 -- 172.21.15.22:0/3883000114 shutdown_connections 2024-03-28T18:17:51.061 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:17:51.060+0000 7fd66e188740 1 --2- 172.21.15.22:0/3883000114 >> [v2:172.21.15.79:6838/2441542443,v1:172.21.15.79:6839/2441542443] conn(0x555eca6ec1a0 0x555eca6ee590 crc :-1 s=READY pgs=293 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:17:51.062 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:17:51.060+0000 7fd66e188740 1 --2- 172.21.15.22:0/3883000114 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555eca6dc460 0x555eca6d9420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:17:51.062 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:17:51.060+0000 7fd66e188740 1 --2- 172.21.15.22:0/3883000114 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x555eca6d3dd0 0x555eca6d8ee0 unknown :-1 s=CLOSED pgs=792 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:17:51.062 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:17:51.060+0000 7fd66e188740 1 --2- 172.21.15.22:0/3883000114 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x555eca6d30f0 0x555eca6dbb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:17:51.062 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:17:51.060+0000 7fd66e188740 1 -- 172.21.15.22:0/3883000114 >> 172.21.15.22:0/3883000114 conn(0x555eca69a200 msgr2=0x555eca6cf890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:17:51.062 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:17:51.060+0000 7fd652000640 0 client.16836 ms_handle_reset on v2:172.21.15.79:6838/2441542443 2024-03-28T18:17:51.062 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:17:51.060+0000 7fd66e188740 1 -- 172.21.15.22:0/3883000114 shutdown_connections 2024-03-28T18:17:51.062 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:17:51.060+0000 7fd66e188740 1 -- 172.21.15.22:0/3883000114 wait complete. 2024-03-28T18:17:51.062 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:17:51.060+0000 7fd66e188740 20 client.16836 trim_cache size 0 max 16384 2024-03-28T18:17:51.078 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:17:51.078+0000 7fd66e188740 1 Processor -- start 2024-03-28T18:17:51.078 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:17:51.078+0000 7fd66e188740 1 -- start start 2024-03-28T18:17:51.079 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:17:51.078+0000 7fd66e188740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x555eca6d30f0 0x555eca6db610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:17:51.079 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:17:51.078+0000 7fd66e188740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555eca6d3dd0 0x555eca6dbb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:17:51.079 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:17:51.079+0000 7fd66e188740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x555eca6dc460 0x555eca6dd850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:17:51.080 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:17:51.079+0000 7fd66e188740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6641b6000 con 0x555eca6d30f0 2024-03-28T18:17:51.080 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:17:51.079+0000 7fd66e188740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6641b3360 con 0x555eca6dc460 2024-03-28T18:17:51.080 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:17:51.079+0000 7fd66e188740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6641b5340 con 0x555eca6d3dd0 2024-03-28T18:17:51.080 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:17:51.079+0000 7fd668a00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x555eca6d30f0 0x555eca6db610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:17:51.080 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:17:51.079+0000 7fd668a00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x555eca6d30f0 0x555eca6db610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:59200/0 (socket says 172.21.15.22:59200) 2024-03-28T18:17:51.080 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:17:51.079+0000 7fd668a00640 1 -- 172.21.15.22:0/2480465786 learned_addr learned my addr 172.21.15.22:0/2480465786 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-28T18:17:51.080 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:17:51.079+0000 7fd669400640 1 --2- 172.21.15.22:0/2480465786 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x555eca6dc460 0x555eca6dd850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:17:51.080 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:17:51.079+0000 7fd65be00640 1 --2- 172.21.15.22:0/2480465786 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555eca6d3dd0 0x555eca6dbb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:17:51.080 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:17:51.079+0000 7fd668a00640 1 -- 172.21.15.22:0/2480465786 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555eca6d3dd0 msgr2=0x555eca6dbb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T18:17:51.080 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:17:51.079+0000 7fd668a00640 1 --2- 172.21.15.22:0/2480465786 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555eca6d3dd0 0x555eca6dbb50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:17:51.080 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:17:51.079+0000 7fd668a00640 1 -- 172.21.15.22:0/2480465786 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x555eca6dc460 msgr2=0x555eca6dd850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T18:17:51.080 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:17:51.079+0000 7fd668a00640 1 --2- 172.21.15.22:0/2480465786 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x555eca6dc460 0x555eca6dd850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:17:51.080 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:17:51.079+0000 7fd668a00640 1 -- 172.21.15.22:0/2480465786 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555eca30b0f0 con 0x555eca6d30f0 2024-03-28T18:17:51.080 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:17:51.080+0000 7fd668a00640 1 --2- 172.21.15.22:0/2480465786 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x555eca6d30f0 0x555eca6db610 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 crypto rx=0x7fd65409ac70 tx=0x7fd6540bb040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c21cdedb6265d3e7 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:17:51.080 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:17:51.080+0000 7fd659200640 1 -- 172.21.15.22:0/2480465786 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd654020040 con 0x555eca6d30f0 2024-03-28T18:17:51.081 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:17:51.080+0000 7fd659200640 1 -- 172.21.15.22:0/2480465786 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7fd6540046c0 con 0x555eca6d30f0 2024-03-28T18:17:51.081 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:17:51.080+0000 7fd659200640 1 -- 172.21.15.22:0/2480465786 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd654099300 con 0x555eca6d30f0 2024-03-28T18:17:51.083 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:17:51.080+0000 7fd66e188740 1 -- 172.21.15.22:0/2480465786 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x555eca6d30f0 msgr2=0x555eca6db610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:17:51.083 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:17:51.080+0000 7fd66e188740 1 --2- 172.21.15.22:0/2480465786 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x555eca6d30f0 0x555eca6db610 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 crypto rx=0x7fd65409ac70 tx=0x7fd6540bb040 comp rx=0 tx=0).stop 2024-03-28T18:17:51.083 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:17:51.081+0000 7fd66e188740 1 -- 172.21.15.22:0/2480465786 shutdown_connections 2024-03-28T18:17:51.083 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:17:51.081+0000 7fd66e188740 1 --2- 172.21.15.22:0/2480465786 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x555eca6dc460 0x555eca6dd850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:17:51.083 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:17:51.081+0000 7fd66e188740 1 --2- 172.21.15.22:0/2480465786 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555eca6d3dd0 0x555eca6dbb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:17:51.083 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:17:51.081+0000 7fd66e188740 1 --2- 172.21.15.22:0/2480465786 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x555eca6d30f0 0x555eca6db610 unknown :-1 s=CLOSED pgs=793 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:17:51.083 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:17:51.081+0000 7fd66e188740 1 -- 172.21.15.22:0/2480465786 >> 172.21.15.22:0/2480465786 conn(0x555eca69a200 msgr2=0x555eca6d77b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:17:51.083 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:17:51.081+0000 7fd66e188740 1 -- 172.21.15.22:0/2480465786 shutdown_connections 2024-03-28T18:17:51.083 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:17:51.081+0000 7fd66e188740 1 -- 172.21.15.22:0/2480465786 wait complete. 2024-03-28T18:17:51.084 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:17:51.081+0000 7fd66e188740 1 -- start start 2024-03-28T18:17:51.084 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:17:51.081+0000 7fd66e188740 1 Processor -- start 2024-03-28T18:17:51.084 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:17:51.082+0000 7fd66e188740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x555eca6d30f0 0x555eca6e0210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:17:51.084 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:17:51.082+0000 7fd66e188740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555eca6d3dd0 0x555eca6e0750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:17:51.084 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:17:51.082+0000 7fd66e188740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x555eca6dc460 0x555eca746950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:17:51.084 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:17:51.082+0000 7fd66e188740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6641b4ce0 con 0x555eca6dc460 2024-03-28T18:17:51.084 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:17:51.082+0000 7fd66e188740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6641b39c0 con 0x555eca6d30f0 2024-03-28T18:17:51.084 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:17:51.082+0000 7fd66e188740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6641b4020 con 0x555eca6d3dd0 2024-03-28T18:17:51.084 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:17:51.082+0000 7fd66e188740 10 client.0 _ll_register_callbacks cb 0x7fd64c005d50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-28T18:17:51.084 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:17:51.082+0000 7fd668a00640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x555eca6d30f0 0x555eca6e0210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:17:51.084 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:17:51.082+0000 7fd669400640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x555eca6dc460 0x555eca746950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:17:51.084 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:17:51.082+0000 7fd668a00640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x555eca6d30f0 0x555eca6e0210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.22:35956/0 (socket says 172.21.15.22:35956) 2024-03-28T18:17:51.084 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:17:51.082+0000 7fd668a00640 1 -- 172.21.15.22:0/3959709911 learned_addr learned my addr 172.21.15.22:0/3959709911 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-28T18:17:51.084 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:17:51.082+0000 7fd669400640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x555eca6dc460 0x555eca746950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:59204/0 (socket says 172.21.15.22:59204) 2024-03-28T18:17:51.085 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:17:51.082+0000 7fd65be00640 1 --2- 172.21.15.22:0/3959709911 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555eca6d3dd0 0x555eca6e0750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:17:51.085 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:17:51.083+0000 7fd669400640 1 -- 172.21.15.22:0/3959709911 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555eca6d3dd0 msgr2=0x555eca6e0750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T18:17:51.085 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:17:51.083+0000 7fd669400640 1 --2- 172.21.15.22:0/3959709911 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555eca6d3dd0 0x555eca6e0750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:17:51.085 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:17:51.083+0000 7fd669400640 1 -- 172.21.15.22:0/3959709911 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x555eca6d30f0 msgr2=0x555eca6e0210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T18:17:51.085 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:17:51.083+0000 7fd669400640 1 --2- 172.21.15.22:0/3959709911 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x555eca6d30f0 0x555eca6e0210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:17:51.085 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:17:51.083+0000 7fd669400640 1 -- 172.21.15.22:0/3959709911 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x555eca746ec0 con 0x555eca6dc460 2024-03-28T18:17:51.085 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:17:51.083+0000 7fd669400640 1 --2- 172.21.15.22:0/3959709911 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x555eca6dc460 0x555eca746950 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 crypto rx=0x555eca6df1f0 tx=0x7fd65c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=838aa79e79dce1ca server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:17:51.085 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:17:51.083+0000 7fd652000640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-03-28T18:17:51.085 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:17:51.083+0000 7fd652000640 1 -- 172.21.15.22:0/3959709911 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd65c020b80 con 0x555eca6dc460 2024-03-28T18:17:51.085 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:17:51.083+0000 7fd652000640 1 -- 172.21.15.22:0/3959709911 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(122..122 src has 1..122) v4 ==== 10389+0+0 (secure 0 0 0) 0x7fd65c020d20 con 0x555eca6dc460 2024-03-28T18:17:51.085 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:17:51.083+0000 7fd66e188740 1 -- 172.21.15.22:0/3959709911 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x555eca63eae0 con 0x555eca6dc460 2024-03-28T18:17:51.087 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:17:51.084+0000 7fd652000640 1 -- 172.21.15.22:0/3959709911 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=187) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd65c029440 con 0x555eca6dc460 2024-03-28T18:17:51.088 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:17:51.084+0000 7fd66e188740 10 client.16842 fetch_fsmap learned FSMap version 187 2024-03-28T18:17:51.088 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:17:51.084+0000 7fd66e188740 1 -- 172.21.15.22:0/3959709911 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=187,monmap=4+}) v3 -- 0x555eca748210 con 0x555eca6dc460 2024-03-28T18:17:51.088 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:17:51.084+0000 7fd652000640 1 -- 172.21.15.22:0/3959709911 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x555eca63eae0 con 0x555eca6dc460 2024-03-28T18:17:51.088 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:17:51.084+0000 7fd652000640 1 -- 172.21.15.22:0/3959709911 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 187) v1 ==== 6221+0+0 (secure 0 0 0) 0x7fd65c027070 con 0x555eca6dc460 2024-03-28T18:17:51.088 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:17:51.084+0000 7fd66e188740 10 client.16842 fetch_fsmap finished waiting for FSMap version 187 2024-03-28T18:17:51.088 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:17:51.084+0000 7fd66e188740 10 client.16842 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-28T18:17:51.088 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:17:51.084+0000 7fd66e188740 20 client.16842 populate_metadata read hostname 'smithi022' 2024-03-28T18:17:51.088 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:17:51.084+0000 7fd66e188740 1 --2- 172.21.15.22:0/3959709911 >> [v2:172.21.15.79:6838/2441542443,v1:172.21.15.79:6839/2441542443] conn(0x555eca7489c0 0x555eca74ade0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:17:51.088 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:17:51.084+0000 7fd66e188740 4 client.16842 mds_command: new command op to 34553 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-28T18:17:51.088 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:17:51.084+0000 7fd66e188740 1 -- 172.21.15.22:0/3959709911 --> [v2:172.21.15.79:6838/2441542443,v1:172.21.15.79:6839/2441542443] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x555eca5bc3a0 con 0x555eca7489c0 2024-03-28T18:17:51.088 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:17:51.085+0000 7fd668a00640 1 --2- 172.21.15.22:0/3959709911 >> [v2:172.21.15.79:6838/2441542443,v1:172.21.15.79:6839/2441542443] conn(0x555eca7489c0 0x555eca74ade0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:17:51.088 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:17:51.086+0000 7fd668a00640 1 --2- 172.21.15.22:0/3959709911 >> [v2:172.21.15.79:6838/2441542443,v1:172.21.15.79:6839/2441542443] conn(0x555eca7489c0 0x555eca74ade0 crc :-1 s=READY pgs=294 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=201994e35e95d6d7 server_cookie=33c6573a75f8ded1 in_seq=0 out_seq=0 2024-03-28T18:17:51.088 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:17:51.086+0000 7fd652000640 10 client.16842 ms_handle_connect on v2:172.21.15.79:6838/2441542443 2024-03-28T18:17:51.088 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:17:51.086+0000 7fd652000640 1 -- 172.21.15.22:0/3959709911 <== mds.0 v2:172.21.15.79:6838/2441542443 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x555eca5bc3a0 con 0x555eca7489c0 2024-03-28T18:17:51.089 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:17:51.086+0000 7fd652000640 10 client.16842 handle_command_reply: tid=0 2024-03-28T18:17:51.089 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:17:51.086+0000 7fd66e188740 1 client.16842 shutdown 2024-03-28T18:17:51.089 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:17:51.086+0000 7fd66e188740 1 -- 172.21.15.22:0/3959709911 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x555eca6dc460 msgr2=0x555eca746950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:17:51.089 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:17:51.086+0000 7fd66e188740 1 --2- 172.21.15.22:0/3959709911 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x555eca6dc460 0x555eca746950 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 crypto rx=0x555eca6df1f0 tx=0x7fd65c004a40 comp rx=0 tx=0).stop 2024-03-28T18:17:51.089 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:17:51.086+0000 7fd66e188740 1 -- 172.21.15.22:0/3959709911 shutdown_connections 2024-03-28T18:17:51.089 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:17:51.086+0000 7fd66e188740 1 --2- 172.21.15.22:0/3959709911 >> [v2:172.21.15.79:6838/2441542443,v1:172.21.15.79:6839/2441542443] conn(0x555eca7489c0 0x555eca74ade0 crc :-1 s=READY pgs=294 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:17:51.089 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:17:51.086+0000 7fd66e188740 1 --2- 172.21.15.22:0/3959709911 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x555eca6dc460 0x555eca746950 unknown :-1 s=CLOSED pgs=794 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:17:51.089 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:17:51.086+0000 7fd66e188740 1 --2- 172.21.15.22:0/3959709911 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555eca6d3dd0 0x555eca6e0750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:17:51.089 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:17:51.086+0000 7fd66e188740 1 --2- 172.21.15.22:0/3959709911 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x555eca6d30f0 0x555eca6e0210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:17:51.089 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:17:51.086+0000 7fd652000640 0 client.16842 ms_handle_reset on v2:172.21.15.79:6838/2441542443 2024-03-28T18:17:51.089 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:17:51.086+0000 7fd66e188740 1 -- 172.21.15.22:0/3959709911 >> 172.21.15.22:0/3959709911 conn(0x555eca69a200 msgr2=0x555eca6cee90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:17:51.089 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:17:51.086+0000 7fd66e188740 1 -- 172.21.15.22:0/3959709911 shutdown_connections 2024-03-28T18:17:51.089 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:17:51.086+0000 7fd66e188740 1 -- 172.21.15.22:0/3959709911 wait complete. 2024-03-28T18:17:51.090 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:17:51.087+0000 7fd66e188740 20 client.16842 trim_cache size 0 max 16384 2024-03-28T18:17:51.090 INFO:teuthology.orchestra.run.smithi022.stdout:{ 2024-03-28T18:17:51.090 INFO:teuthology.orchestra.run.smithi022.stdout: "status": "IDLE (waiting for more scrubs)", 2024-03-28T18:17:51.091 INFO:teuthology.orchestra.run.smithi022.stdout: "scrubs": { 2024-03-28T18:17:51.091 INFO:teuthology.orchestra.run.smithi022.stdout: "4bbbd61d-29a3-4bf6-a4f2-d5783d585de3": { 2024-03-28T18:17:51.091 INFO:teuthology.orchestra.run.smithi022.stdout: "path": "/", 2024-03-28T18:17:51.091 INFO:teuthology.orchestra.run.smithi022.stdout: "tag": "4bbbd61d-29a3-4bf6-a4f2-d5783d585de3", 2024-03-28T18:17:51.091 INFO:teuthology.orchestra.run.smithi022.stdout: "options": "recursive,force" 2024-03-28T18:17:51.091 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-03-28T18:17:51.091 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-03-28T18:17:51.092 INFO:teuthology.orchestra.run.smithi022.stdout:} 2024-03-28T18:17:51.092 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:17:51.087+0000 7fd669e00640 1 -- 172.21.15.22:0/1233136251 >> [v2:172.21.15.22:6800/1357365082,v1:172.21.15.22:6801/1357365082] conn(0x7fd6480a29f0 msgr2=0x7fd6480a4eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:17:51.092 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:17:51.087+0000 7fd669e00640 1 --2- 172.21.15.22:0/1233136251 >> [v2:172.21.15.22:6800/1357365082,v1:172.21.15.22:6801/1357365082] conn(0x7fd6480a29f0 0x7fd6480a4eb0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fd65c005fd0 tx=0x7fd65c0095b0 comp rx=0 tx=0).stop 2024-03-28T18:17:51.093 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:17:51.087+0000 7fd669e00640 1 -- 172.21.15.22:0/1233136251 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd664107e50 msgr2=0x7fd66419e470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:17:51.093 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:17:51.087+0000 7fd669e00640 1 --2- 172.21.15.22:0/1233136251 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd664107e50 0x7fd66419e470 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 crypto rx=0x7fd65400d770 tx=0x7fd65400dc30 comp rx=0 tx=0).stop 2024-03-28T18:17:51.093 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:17:51.087+0000 7fd669e00640 1 -- 172.21.15.22:0/1233136251 shutdown_connections 2024-03-28T18:17:51.093 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:17:51.087+0000 7fd669e00640 1 --2- 172.21.15.22:0/1233136251 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd6641a2d30 0x7fd6641a31e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:17:51.093 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:17:51.087+0000 7fd669e00640 1 --2- 172.21.15.22:0/1233136251 >> [v2:172.21.15.22:6800/1357365082,v1:172.21.15.22:6801/1357365082] conn(0x7fd6480a29f0 0x7fd6480a4eb0 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:17:51.093 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:17:51.087+0000 7fd669e00640 1 --2- 172.21.15.22:0/1233136251 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd66410cea0 0x7fd66419e9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:17:51.093 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:17:51.087+0000 7fd669e00640 1 --2- 172.21.15.22:0/1233136251 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd664107e50 0x7fd66419e470 unknown :-1 s=CLOSED pgs=790 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:17:51.093 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:17:51.088+0000 7fd669e00640 1 -- 172.21.15.22:0/1233136251 >> 172.21.15.22:0/1233136251 conn(0x7fd6640774c0 msgr2=0x7fd6640790b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:17:51.094 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:17:51.088+0000 7fd669e00640 1 -- 172.21.15.22:0/1233136251 shutdown_connections 2024-03-28T18:17:51.094 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:17:51.088+0000 7fd669e00640 1 -- 172.21.15.22:0/1233136251 wait complete. 2024-03-28T18:17:51.104 INFO:tasks.cephfs.filesystem:scrub status for tag:4bbbd61d-29a3-4bf6-a4f2-d5783d585de3 - {'path': '/', 'tag': '4bbbd61d-29a3-4bf6-a4f2-d5783d585de3', 'options': 'recursive,force'} 2024-03-28T18:17:52.925 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:17:52 smithi134 ceph-mon[35037]: pgmap v341: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:17:52.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:17:52 smithi079 ceph-mon[35312]: pgmap v341: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:17:53.031 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:17:52 smithi022 ceph-mon[31985]: pgmap v341: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:17:54.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:17:54 smithi134 ceph-mon[35037]: pgmap v342: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:17:54.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:17:54 smithi079 ceph-mon[35312]: pgmap v342: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:17:55.031 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:17:54 smithi022 ceph-mon[31985]: pgmap v342: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:17:56.835 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:17:56 smithi022 ceph-mon[31985]: pgmap v343: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:17:56.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:17:56 smithi134 ceph-mon[35037]: pgmap v343: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:17:56.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:17:56 smithi079 ceph-mon[35312]: pgmap v343: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:17:58.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:17:58 smithi134 ceph-mon[35037]: pgmap v344: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:17:58.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:17:58 smithi079 ceph-mon[35312]: pgmap v344: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:17:59.031 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:17:58 smithi022 ceph-mon[31985]: pgmap v344: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:18:00.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:18:00 smithi134 ceph-mon[35037]: pgmap v345: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:18:00.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:18:00 smithi134 ceph-mon[35037]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T18:18:00.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:18:00 smithi079 ceph-mon[35312]: pgmap v345: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:18:00.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:18:00 smithi079 ceph-mon[35312]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T18:18:01.031 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:18:00 smithi022 ceph-mon[31985]: pgmap v345: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:18:01.031 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:18:00 smithi022 ceph-mon[31985]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T18:18:02.362 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:18:02 smithi134 ceph-mon[35037]: from='mgr.15522 ' entity='mgr.x' 2024-03-28T18:18:02.363 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:18:02 smithi134 ceph-mon[35037]: from='mgr.15522 ' entity='mgr.x' 2024-03-28T18:18:02.363 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:18:02 smithi134 ceph-mon[35037]: pgmap v346: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:18:02.432 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:18:02 smithi079 ceph-mon[35312]: from='mgr.15522 ' entity='mgr.x' 2024-03-28T18:18:02.432 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:18:02 smithi079 ceph-mon[35312]: from='mgr.15522 ' entity='mgr.x' 2024-03-28T18:18:02.432 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:18:02 smithi079 ceph-mon[35312]: pgmap v346: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:18:02.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:18:02 smithi022 ceph-mon[31985]: from='mgr.15522 ' entity='mgr.x' 2024-03-28T18:18:02.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:18:02 smithi022 ceph-mon[31985]: from='mgr.15522 ' entity='mgr.x' 2024-03-28T18:18:02.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:18:02 smithi022 ceph-mon[31985]: pgmap v346: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:18:04.674 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:18:04 smithi134 ceph-mon[35037]: pgmap v347: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:18:04.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:18:04 smithi134 ceph-mon[35037]: from='mgr.15522 ' entity='mgr.x' 2024-03-28T18:18:04.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:18:04 smithi134 ceph-mon[35037]: from='mgr.15522 ' entity='mgr.x' 2024-03-28T18:18:04.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:18:04 smithi134 ceph-mon[35037]: from='mgr.15522 ' entity='mgr.x' 2024-03-28T18:18:04.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:18:04 smithi134 ceph-mon[35037]: from='mgr.15522 ' entity='mgr.x' 2024-03-28T18:18:04.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:18:04 smithi134 ceph-mon[35037]: from='mgr.15522 ' entity='mgr.x' 2024-03-28T18:18:04.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:18:04 smithi134 ceph-mon[35037]: from='mgr.15522 ' entity='mgr.x' 2024-03-28T18:18:04.683 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:18:04 smithi079 ceph-mon[35312]: pgmap v347: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:18:04.683 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:18:04 smithi079 ceph-mon[35312]: from='mgr.15522 ' entity='mgr.x' 2024-03-28T18:18:04.683 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:18:04 smithi079 ceph-mon[35312]: from='mgr.15522 ' entity='mgr.x' 2024-03-28T18:18:04.683 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:18:04 smithi079 ceph-mon[35312]: from='mgr.15522 ' entity='mgr.x' 2024-03-28T18:18:04.683 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:18:04 smithi079 ceph-mon[35312]: from='mgr.15522 ' entity='mgr.x' 2024-03-28T18:18:04.683 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:18:04 smithi079 ceph-mon[35312]: from='mgr.15522 ' entity='mgr.x' 2024-03-28T18:18:04.683 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:18:04 smithi079 ceph-mon[35312]: from='mgr.15522 ' entity='mgr.x' 2024-03-28T18:18:04.781 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:18:04 smithi022 ceph-mon[31985]: pgmap v347: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:18:04.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:18:04 smithi022 ceph-mon[31985]: from='mgr.15522 ' entity='mgr.x' 2024-03-28T18:18:04.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:18:04 smithi022 ceph-mon[31985]: from='mgr.15522 ' entity='mgr.x' 2024-03-28T18:18:04.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:18:04 smithi022 ceph-mon[31985]: from='mgr.15522 ' entity='mgr.x' 2024-03-28T18:18:04.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:18:04 smithi022 ceph-mon[31985]: from='mgr.15522 ' entity='mgr.x' 2024-03-28T18:18:04.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:18:04 smithi022 ceph-mon[31985]: from='mgr.15522 ' entity='mgr.x' 2024-03-28T18:18:04.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:18:04 smithi022 ceph-mon[31985]: from='mgr.15522 ' entity='mgr.x' 2024-03-28T18:18:05.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:18:05 smithi134 ceph-mon[35037]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-28T18:18:05.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:18:05 smithi134 ceph-mon[35037]: from='mgr.15522 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-28T18:18:05.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:18:05 smithi134 ceph-mon[35037]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-28T18:18:05.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:18:05 smithi134 ceph-mon[35037]: from='mgr.15522 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-28T18:18:05.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:18:05 smithi134 ceph-mon[35037]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-03-28T18:18:05.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:18:05 smithi134 ceph-mon[35037]: from='mgr.15522 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-03-28T18:18:05.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:18:05 smithi134 ceph-mon[35037]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-03-28T18:18:05.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:18:05 smithi134 ceph-mon[35037]: from='mgr.15522 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-03-28T18:18:05.676 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:18:05 smithi134 ceph-mon[35037]: Adjusting osd_memory_target on smithi079 to 176.7M 2024-03-28T18:18:05.676 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:18:05 smithi134 ceph-mon[35037]: Unable to set osd_memory_target on smithi079 to 185386700: error parsing value: Value '185386700' is below minimum 939524096 2024-03-28T18:18:05.683 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:18:05 smithi079 ceph-mon[35312]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-28T18:18:05.683 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:18:05 smithi079 ceph-mon[35312]: from='mgr.15522 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-28T18:18:05.683 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:18:05 smithi079 ceph-mon[35312]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-28T18:18:05.683 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:18:05 smithi079 ceph-mon[35312]: from='mgr.15522 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-28T18:18:05.683 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:18:05 smithi079 ceph-mon[35312]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-03-28T18:18:05.683 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:18:05 smithi079 ceph-mon[35312]: from='mgr.15522 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-03-28T18:18:05.683 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:18:05 smithi079 ceph-mon[35312]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-03-28T18:18:05.683 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:18:05 smithi079 ceph-mon[35312]: from='mgr.15522 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-03-28T18:18:05.683 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:18:05 smithi079 ceph-mon[35312]: Adjusting osd_memory_target on smithi079 to 176.7M 2024-03-28T18:18:05.683 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:18:05 smithi079 ceph-mon[35312]: Unable to set osd_memory_target on smithi079 to 185386700: error parsing value: Value '185386700' is below minimum 939524096 2024-03-28T18:18:05.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:18:05 smithi022 ceph-mon[31985]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-28T18:18:05.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:18:05 smithi022 ceph-mon[31985]: from='mgr.15522 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-28T18:18:05.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:18:05 smithi022 ceph-mon[31985]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-28T18:18:05.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:18:05 smithi022 ceph-mon[31985]: from='mgr.15522 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-28T18:18:05.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:18:05 smithi022 ceph-mon[31985]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-03-28T18:18:05.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:18:05 smithi022 ceph-mon[31985]: from='mgr.15522 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-03-28T18:18:05.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:18:05 smithi022 ceph-mon[31985]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-03-28T18:18:05.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:18:05 smithi022 ceph-mon[31985]: from='mgr.15522 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-03-28T18:18:05.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:18:05 smithi022 ceph-mon[31985]: Adjusting osd_memory_target on smithi079 to 176.7M 2024-03-28T18:18:05.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:18:05 smithi022 ceph-mon[31985]: Unable to set osd_memory_target on smithi079 to 185386700: error parsing value: Value '185386700' is below minimum 939524096 2024-03-28T18:18:06.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:18:06 smithi134 ceph-mon[35037]: pgmap v348: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:18:06.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:18:06 smithi134 ceph-mon[35037]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-28T18:18:06.676 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:18:06 smithi134 ceph-mon[35037]: from='mgr.15522 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-28T18:18:06.676 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:18:06 smithi134 ceph-mon[35037]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-28T18:18:06.676 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:18:06 smithi134 ceph-mon[35037]: from='mgr.15522 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-28T18:18:06.676 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:18:06 smithi134 ceph-mon[35037]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-03-28T18:18:06.676 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:18:06 smithi134 ceph-mon[35037]: from='mgr.15522 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-03-28T18:18:06.676 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:18:06 smithi134 ceph-mon[35037]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-03-28T18:18:06.676 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:18:06 smithi134 ceph-mon[35037]: from='mgr.15522 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-03-28T18:18:06.676 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:18:06 smithi134 ceph-mon[35037]: Adjusting osd_memory_target on smithi134 to 176.8M 2024-03-28T18:18:06.676 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:18:06 smithi134 ceph-mon[35037]: Unable to set osd_memory_target on smithi134 to 185393868: error parsing value: Value '185393868' is below minimum 939524096 2024-03-28T18:18:06.676 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:18:06 smithi134 ceph-mon[35037]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-28T18:18:06.676 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:18:06 smithi134 ceph-mon[35037]: from='mgr.15522 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-28T18:18:06.677 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:18:06 smithi134 ceph-mon[35037]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-28T18:18:06.677 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:18:06 smithi134 ceph-mon[35037]: from='mgr.15522 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-28T18:18:06.677 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:18:06 smithi134 ceph-mon[35037]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-03-28T18:18:06.677 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:18:06 smithi134 ceph-mon[35037]: from='mgr.15522 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-03-28T18:18:06.677 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:18:06 smithi134 ceph-mon[35037]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-03-28T18:18:06.677 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:18:06 smithi134 ceph-mon[35037]: from='mgr.15522 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-03-28T18:18:06.677 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:18:06 smithi134 ceph-mon[35037]: Adjusting osd_memory_target on smithi022 to 177.8M 2024-03-28T18:18:06.677 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:18:06 smithi134 ceph-mon[35037]: Unable to set osd_memory_target on smithi022 to 186497740: error parsing value: Value '186497740' is below minimum 939524096 2024-03-28T18:18:06.677 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:18:06 smithi134 ceph-mon[35037]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T18:18:06.677 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:18:06 smithi134 ceph-mon[35037]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T18:18:06.677 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:18:06 smithi134 ceph-mon[35037]: pgmap v349: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 7 op/s 2024-03-28T18:18:06.677 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:18:06 smithi134 ceph-mon[35037]: from='mgr.15522 ' entity='mgr.x' 2024-03-28T18:18:06.683 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:18:06 smithi079 ceph-mon[35312]: pgmap v348: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:18:06.683 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:18:06 smithi079 ceph-mon[35312]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-28T18:18:06.683 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:18:06 smithi079 ceph-mon[35312]: from='mgr.15522 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-28T18:18:06.683 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:18:06 smithi079 ceph-mon[35312]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-28T18:18:06.684 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:18:06 smithi079 ceph-mon[35312]: from='mgr.15522 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-28T18:18:06.684 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:18:06 smithi079 ceph-mon[35312]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-03-28T18:18:06.684 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:18:06 smithi079 ceph-mon[35312]: from='mgr.15522 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-03-28T18:18:06.684 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:18:06 smithi079 ceph-mon[35312]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-03-28T18:18:06.684 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:18:06 smithi079 ceph-mon[35312]: from='mgr.15522 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-03-28T18:18:06.684 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:18:06 smithi079 ceph-mon[35312]: Adjusting osd_memory_target on smithi134 to 176.8M 2024-03-28T18:18:06.684 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:18:06 smithi079 ceph-mon[35312]: Unable to set osd_memory_target on smithi134 to 185393868: error parsing value: Value '185393868' is below minimum 939524096 2024-03-28T18:18:06.684 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:18:06 smithi079 ceph-mon[35312]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-28T18:18:06.684 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:18:06 smithi079 ceph-mon[35312]: from='mgr.15522 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-28T18:18:06.684 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:18:06 smithi079 ceph-mon[35312]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-28T18:18:06.684 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:18:06 smithi079 ceph-mon[35312]: from='mgr.15522 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-28T18:18:06.684 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:18:06 smithi079 ceph-mon[35312]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-03-28T18:18:06.684 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:18:06 smithi079 ceph-mon[35312]: from='mgr.15522 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-03-28T18:18:06.684 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:18:06 smithi079 ceph-mon[35312]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-03-28T18:18:06.684 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:18:06 smithi079 ceph-mon[35312]: from='mgr.15522 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-03-28T18:18:06.685 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:18:06 smithi079 ceph-mon[35312]: Adjusting osd_memory_target on smithi022 to 177.8M 2024-03-28T18:18:06.685 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:18:06 smithi079 ceph-mon[35312]: Unable to set osd_memory_target on smithi022 to 186497740: error parsing value: Value '186497740' is below minimum 939524096 2024-03-28T18:18:06.685 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:18:06 smithi079 ceph-mon[35312]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T18:18:06.685 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:18:06 smithi079 ceph-mon[35312]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T18:18:06.685 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:18:06 smithi079 ceph-mon[35312]: pgmap v349: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 7 op/s 2024-03-28T18:18:06.685 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:18:06 smithi079 ceph-mon[35312]: from='mgr.15522 ' entity='mgr.x' 2024-03-28T18:18:06.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:18:06 smithi022 ceph-mon[31985]: pgmap v348: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:18:06.783 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:18:06 smithi022 ceph-mon[31985]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-28T18:18:06.783 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:18:06 smithi022 ceph-mon[31985]: from='mgr.15522 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-28T18:18:06.783 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:18:06 smithi022 ceph-mon[31985]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-28T18:18:06.783 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:18:06 smithi022 ceph-mon[31985]: from='mgr.15522 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-28T18:18:06.783 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:18:06 smithi022 ceph-mon[31985]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-03-28T18:18:06.783 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:18:06 smithi022 ceph-mon[31985]: from='mgr.15522 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-03-28T18:18:06.783 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:18:06 smithi022 ceph-mon[31985]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-03-28T18:18:06.783 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:18:06 smithi022 ceph-mon[31985]: from='mgr.15522 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-03-28T18:18:06.783 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:18:06 smithi022 ceph-mon[31985]: Adjusting osd_memory_target on smithi134 to 176.8M 2024-03-28T18:18:06.783 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:18:06 smithi022 ceph-mon[31985]: Unable to set osd_memory_target on smithi134 to 185393868: error parsing value: Value '185393868' is below minimum 939524096 2024-03-28T18:18:06.783 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:18:06 smithi022 ceph-mon[31985]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-28T18:18:06.783 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:18:06 smithi022 ceph-mon[31985]: from='mgr.15522 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-28T18:18:06.783 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:18:06 smithi022 ceph-mon[31985]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-28T18:18:06.783 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:18:06 smithi022 ceph-mon[31985]: from='mgr.15522 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-28T18:18:06.784 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:18:06 smithi022 ceph-mon[31985]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-03-28T18:18:06.784 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:18:06 smithi022 ceph-mon[31985]: from='mgr.15522 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-03-28T18:18:06.784 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:18:06 smithi022 ceph-mon[31985]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-03-28T18:18:06.784 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:18:06 smithi022 ceph-mon[31985]: from='mgr.15522 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-03-28T18:18:06.784 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:18:06 smithi022 ceph-mon[31985]: Adjusting osd_memory_target on smithi022 to 177.8M 2024-03-28T18:18:06.784 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:18:06 smithi022 ceph-mon[31985]: Unable to set osd_memory_target on smithi022 to 186497740: error parsing value: Value '186497740' is below minimum 939524096 2024-03-28T18:18:06.784 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:18:06 smithi022 ceph-mon[31985]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T18:18:06.784 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:18:06 smithi022 ceph-mon[31985]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T18:18:06.784 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:18:06 smithi022 ceph-mon[31985]: pgmap v349: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 7 op/s 2024-03-28T18:18:06.784 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:18:06 smithi022 ceph-mon[31985]: from='mgr.15522 ' entity='mgr.x' 2024-03-28T18:18:07.674 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:18:07 smithi134 ceph-mon[35037]: Health check failed: 4 failed cephadm daemon(s) (CEPHADM_FAILED_DAEMON) 2024-03-28T18:18:07.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:18:07 smithi079 ceph-mon[35312]: Health check failed: 4 failed cephadm daemon(s) (CEPHADM_FAILED_DAEMON) 2024-03-28T18:18:07.781 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:18:07 smithi022 ceph-mon[31985]: Health check failed: 4 failed cephadm daemon(s) (CEPHADM_FAILED_DAEMON) 2024-03-28T18:18:08.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:18:08 smithi079 ceph-mon[35312]: pgmap v350: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 7 op/s 2024-03-28T18:18:08.781 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:18:08 smithi022 ceph-mon[31985]: pgmap v350: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 7 op/s 2024-03-28T18:18:08.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:18:08 smithi134 ceph-mon[35037]: pgmap v350: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 7 op/s 2024-03-28T18:18:11.032 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:18:10 smithi022 ceph-mon[31985]: pgmap v351: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 7 op/s 2024-03-28T18:18:11.175 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:18:10 smithi134 ceph-mon[35037]: pgmap v351: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 7 op/s 2024-03-28T18:18:11.182 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:18:10 smithi079 ceph-mon[35312]: pgmap v351: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 7 op/s 2024-03-28T18:18:13.032 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:18:12 smithi022 ceph-mon[31985]: pgmap v352: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 7 op/s 2024-03-28T18:18:13.174 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:18:12 smithi134 ceph-mon[35037]: pgmap v352: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 7 op/s 2024-03-28T18:18:13.182 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:18:12 smithi079 ceph-mon[35312]: pgmap v352: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 7 op/s 2024-03-28T18:18:15.032 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:18:14 smithi022 ceph-mon[31985]: pgmap v353: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 7 op/s 2024-03-28T18:18:15.174 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:18:14 smithi134 ceph-mon[35037]: pgmap v353: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 7 op/s 2024-03-28T18:18:15.182 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:18:14 smithi079 ceph-mon[35312]: pgmap v353: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 7 op/s 2024-03-28T18:18:17.032 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:18:16 smithi022 ceph-mon[31985]: pgmap v354: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 7 op/s 2024-03-28T18:18:17.174 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:18:16 smithi134 ceph-mon[35037]: pgmap v354: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 7 op/s 2024-03-28T18:18:17.182 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:18:16 smithi079 ceph-mon[35312]: pgmap v354: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 7 op/s 2024-03-28T18:18:19.174 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:18:18 smithi134 ceph-mon[35037]: pgmap v355: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:18:19.182 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:18:18 smithi079 ceph-mon[35312]: pgmap v355: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:18:19.282 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:18:18 smithi022 ceph-mon[31985]: pgmap v355: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:18:21.105 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-28T18:18:21.105 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-28T18:18:21.174 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:18:20 smithi134 ceph-mon[35037]: pgmap v356: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:18:21.182 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:18:20 smithi079 ceph-mon[35312]: pgmap v356: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:18:21.200 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:18:20 smithi022 ceph-mon[31985]: pgmap v356: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:18:21.345 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:18:21.345+0000 7f31f5800640 1 -- 172.21.15.22:0/1871767041 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f31f01076b0 msgr2=0x7f31f0107a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:18:21.345 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:18:21.345+0000 7f31f5800640 1 --2- 172.21.15.22:0/1871767041 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f31f01076b0 0x7f31f0107a90 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 crypto rx=0x7f31e0009930 tx=0x7f31e002bc30 comp rx=0 tx=0).stop 2024-03-28T18:18:21.345 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:18:21.345+0000 7f31f5800640 1 -- 172.21.15.22:0/1871767041 shutdown_connections 2024-03-28T18:18:21.345 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:18:21.345+0000 7f31f5800640 1 --2- 172.21.15.22:0/1871767041 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f31f0101fd0 0x7f31f010e4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:18:21.346 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:18:21.345+0000 7f31f5800640 1 --2- 172.21.15.22:0/1871767041 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f31f0101610 0x7f31f0101a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:18:21.346 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:18:21.345+0000 7f31f5800640 1 --2- 172.21.15.22:0/1871767041 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f31f01076b0 0x7f31f0107a90 unknown :-1 s=CLOSED pgs=795 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:18:21.346 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:18:21.345+0000 7f31f5800640 1 -- 172.21.15.22:0/1871767041 >> 172.21.15.22:0/1871767041 conn(0x7f31f00fd100 msgr2=0x7f31f00ff520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:18:21.346 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:18:21.345+0000 7f31f5800640 1 -- 172.21.15.22:0/1871767041 shutdown_connections 2024-03-28T18:18:21.346 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:18:21.345+0000 7f31f5800640 1 -- 172.21.15.22:0/1871767041 wait complete. 2024-03-28T18:18:21.347 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:18:21.346+0000 7f31f5800640 1 Processor -- start 2024-03-28T18:18:21.347 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:18:21.346+0000 7f31f5800640 1 -- start start 2024-03-28T18:18:21.347 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:18:21.347+0000 7f31f5800640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f31f0101610 0x7f31f00739a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:18:21.347 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:18:21.347+0000 7f31f5800640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f31f0101fd0 0x7f31f0073ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:18:21.348 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:18:21.347+0000 7f31f5800640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f31f01076b0 0x7f31f0074420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:18:21.348 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:18:21.347+0000 7f31f5800640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31f0111a50 con 0x7f31f0101fd0 2024-03-28T18:18:21.348 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:18:21.347+0000 7f31f5800640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31f01118d0 con 0x7f31f0101610 2024-03-28T18:18:21.348 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:18:21.347+0000 7f31f5800640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31f0111bd0 con 0x7f31f01076b0 2024-03-28T18:18:21.348 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:18:21.347+0000 7f31e6000640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f31f0101fd0 0x7f31f0073ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:18:21.348 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:18:21.347+0000 7f31e6000640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f31f0101fd0 0x7f31f0073ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:33522/0 (socket says 172.21.15.22:33522) 2024-03-28T18:18:21.348 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:18:21.347+0000 7f31e6000640 1 -- 172.21.15.22:0/1144170155 learned_addr learned my addr 172.21.15.22:0/1144170155 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-28T18:18:21.348 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:18:21.347+0000 7f31eea00640 1 --2- 172.21.15.22:0/1144170155 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f31f0101610 0x7f31f00739a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:18:21.348 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:18:21.347+0000 7f31ef400640 1 --2- 172.21.15.22:0/1144170155 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f31f01076b0 0x7f31f0074420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:18:21.348 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:18:21.348+0000 7f31e6000640 1 -- 172.21.15.22:0/1144170155 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f31f01076b0 msgr2=0x7f31f0074420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:18:21.348 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:18:21.348+0000 7f31e6000640 1 --2- 172.21.15.22:0/1144170155 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f31f01076b0 0x7f31f0074420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:18:21.348 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:18:21.348+0000 7f31e6000640 1 -- 172.21.15.22:0/1144170155 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f31f0101610 msgr2=0x7f31f00739a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:18:21.348 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:18:21.348+0000 7f31e6000640 1 --2- 172.21.15.22:0/1144170155 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f31f0101610 0x7f31f00739a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:18:21.348 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:18:21.348+0000 7f31e6000640 1 -- 172.21.15.22:0/1144170155 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f31f01a6490 con 0x7f31f0101fd0 2024-03-28T18:18:21.349 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:18:21.348+0000 7f31eea00640 1 --2- 172.21.15.22:0/1144170155 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f31f0101610 0x7f31f00739a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-28T18:18:21.349 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:18:21.348+0000 7f31e6000640 1 --2- 172.21.15.22:0/1144170155 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f31f0101fd0 0x7f31f0073ee0 secure :-1 s=READY pgs=796 cs=0 l=1 rev1=1 crypto rx=0x7f31d800c970 tx=0x7f31d800ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:18:21.349 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:18:21.348+0000 7f31e7e00640 1 -- 172.21.15.22:0/1144170155 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31d8007bb0 con 0x7f31f0101fd0 2024-03-28T18:18:21.349 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:18:21.348+0000 7f31e7e00640 1 -- 172.21.15.22:0/1144170155 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f31d8007d50 con 0x7f31f0101fd0 2024-03-28T18:18:21.350 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:18:21.348+0000 7f31e7e00640 1 -- 172.21.15.22:0/1144170155 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31d800f450 con 0x7f31f0101fd0 2024-03-28T18:18:21.350 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:18:21.348+0000 7f31f5800640 1 -- 172.21.15.22:0/1144170155 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f31f01a6780 con 0x7f31f0101fd0 2024-03-28T18:18:21.350 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:18:21.348+0000 7f31f5800640 1 -- 172.21.15.22:0/1144170155 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f31f01a6c80 con 0x7f31f0101fd0 2024-03-28T18:18:21.352 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:18:21.352+0000 7f31f8571740 1 Processor -- start 2024-03-28T18:18:21.353 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:18:21.352+0000 7f31e7e00640 1 -- 172.21.15.22:0/1144170155 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 26) v1 ==== 137421+0+0 (secure 0 0 0) 0x7f31d8004fb0 con 0x7f31f0101fd0 2024-03-28T18:18:21.353 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:18:21.352+0000 7f31f8571740 1 -- start start 2024-03-28T18:18:21.353 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:18:21.352+0000 7f31f8571740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e662649c00 0x55e662648950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:18:21.353 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:18:21.352+0000 7f31f8571740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55e662648f20 0x55e6626493a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:18:21.353 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:18:21.352+0000 7f31f8571740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e662652290 0x55e662654680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:18:21.353 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:18:21.353+0000 7f31f8571740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e6625ad250 con 0x55e662652290 2024-03-28T18:18:21.353 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:18:21.353+0000 7f31f8571740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e662555a30 con 0x55e662648f20 2024-03-28T18:18:21.353 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:18:21.353+0000 7f31f8571740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e6625a92f0 con 0x55e662649c00 2024-03-28T18:18:21.353 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:18:21.353+0000 7f31e6000640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e662652290 0x55e662654680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:18:21.353 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:18:21.353+0000 7f31e6000640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e662652290 0x55e662654680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:33528/0 (socket says 172.21.15.22:33528) 2024-03-28T18:18:21.353 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:18:21.353+0000 7f31e6000640 1 -- 172.21.15.22:0/3927086276 learned_addr learned my addr 172.21.15.22:0/3927086276 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-28T18:18:21.353 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:18:21.353+0000 7f31eea00640 1 --2- 172.21.15.22:0/3927086276 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55e662648f20 0x55e6626493a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:18:21.353 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:18:21.353+0000 7f31ef400640 1 --2- 172.21.15.22:0/3927086276 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e662649c00 0x55e662648950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:18:21.354 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:18:21.353+0000 7f31e6000640 1 -- 172.21.15.22:0/3927086276 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e662649c00 msgr2=0x55e662648950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T18:18:21.355 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:18:21.353+0000 7f31e6000640 1 --2- 172.21.15.22:0/3927086276 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e662649c00 0x55e662648950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:18:21.355 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:18:21.353+0000 7f31e6000640 1 -- 172.21.15.22:0/3927086276 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55e662648f20 msgr2=0x55e6626493a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T18:18:21.355 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:18:21.354+0000 7f31e6000640 1 --2- 172.21.15.22:0/3927086276 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55e662648f20 0x55e6626493a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:18:21.355 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:18:21.354+0000 7f31e6000640 1 -- 172.21.15.22:0/3927086276 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e662540690 con 0x55e662652290 2024-03-28T18:18:21.355 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:18:21.354+0000 7f31e6000640 1 --2- 172.21.15.22:0/3927086276 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e662652290 0x55e662654680 secure :-1 s=READY pgs=797 cs=0 l=1 rev1=1 crypto rx=0x7f31d80cc200 tx=0x7f31d80d7970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c3a9e05b7ced0624 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:18:21.355 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:18:21.354+0000 7f31e7e00640 1 --2- 172.21.15.22:0/1144170155 >> [v2:172.21.15.22:6800/1357365082,v1:172.21.15.22:6801/1357365082] conn(0x7f31d40a29f0 0x7f31d40a4eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:18:21.355 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:18:21.354+0000 7f31dfe00640 1 -- 172.21.15.22:0/3927086276 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31d80e3070 con 0x55e662652290 2024-03-28T18:18:21.355 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:18:21.354+0000 7f31dfe00640 1 -- 172.21.15.22:0/3927086276 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f31d8070a00 con 0x55e662652290 2024-03-28T18:18:21.355 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:18:21.354+0000 7f31dfe00640 1 -- 172.21.15.22:0/3927086276 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31d8084740 con 0x55e662652290 2024-03-28T18:18:21.355 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:18:21.354+0000 7f31e7e00640 1 -- 172.21.15.22:0/1144170155 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(122..122 src has 1..122) v4 ==== 10389+0+0 (secure 0 0 0) 0x7f31d80cf320 con 0x7f31f0101fd0 2024-03-28T18:18:21.355 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:18:21.355+0000 7f31ef400640 1 --2- 172.21.15.22:0/1144170155 >> [v2:172.21.15.22:6800/1357365082,v1:172.21.15.22:6801/1357365082] conn(0x7f31d40a29f0 0x7f31d40a4eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:18:21.359 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:18:21.355+0000 7f31f8571740 1 -- 172.21.15.22:0/3927086276 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e662652290 msgr2=0x55e662654680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:18:21.359 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:18:21.355+0000 7f31f8571740 1 --2- 172.21.15.22:0/3927086276 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e662652290 0x55e662654680 secure :-1 s=READY pgs=797 cs=0 l=1 rev1=1 crypto rx=0x7f31d80cc200 tx=0x7f31d80d7970 comp rx=0 tx=0).stop 2024-03-28T18:18:21.359 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:18:21.355+0000 7f31ef400640 1 --2- 172.21.15.22:0/1144170155 >> [v2:172.21.15.22:6800/1357365082,v1:172.21.15.22:6801/1357365082] conn(0x7f31d40a29f0 0x7f31d40a4eb0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f31e8005fd0 tx=0x7f31e8009760 comp rx=0 tx=0).ready entity=mgr.15522 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:18:21.359 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:18:21.355+0000 7f31f8571740 1 -- 172.21.15.22:0/3927086276 shutdown_connections 2024-03-28T18:18:21.359 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:18:21.355+0000 7f31f8571740 1 --2- 172.21.15.22:0/3927086276 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e662652290 0x55e662654680 unknown :-1 s=CLOSED pgs=797 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:18:21.359 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:18:21.355+0000 7f31f8571740 1 --2- 172.21.15.22:0/3927086276 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55e662648f20 0x55e6626493a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:18:21.360 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:18:21.355+0000 7f31f8571740 1 --2- 172.21.15.22:0/3927086276 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e662649c00 0x55e662648950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:18:21.360 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:18:21.355+0000 7f31f8571740 1 -- 172.21.15.22:0/3927086276 >> 172.21.15.22:0/3927086276 conn(0x55e662610200 msgr2=0x55e662643970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:18:21.360 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:18:21.356+0000 7f31f8571740 1 -- 172.21.15.22:0/3927086276 shutdown_connections 2024-03-28T18:18:21.360 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:18:21.356+0000 7f31f8571740 1 -- 172.21.15.22:0/3927086276 wait complete. 2024-03-28T18:18:21.360 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:18:21.356+0000 7f31f8571740 1 -- start start 2024-03-28T18:18:21.360 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:18:21.356+0000 7f31f8571740 1 Processor -- start 2024-03-28T18:18:21.360 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:18:21.357+0000 7f31f8571740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55e662648f20 0x55e662651930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:18:21.360 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:18:21.357+0000 7f31f8571740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e662649c00 0x55e66264ecc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:18:21.360 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:18:21.357+0000 7f31f8571740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e662652290 0x55e66264f200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:18:21.360 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:18:21.357+0000 7f31f8571740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e662655100 con 0x55e662652290 2024-03-28T18:18:21.360 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:18:21.357+0000 7f31f8571740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e662568e80 con 0x55e662648f20 2024-03-28T18:18:21.360 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:18:21.357+0000 7f31f8571740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e662655280 con 0x55e662649c00 2024-03-28T18:18:21.360 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:18:21.357+0000 7f31f8571740 10 client.0 _ll_register_callbacks cb 0x55e66257b130 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-28T18:18:21.360 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:18:21.357+0000 7f31ef400640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e662652290 0x55e66264f200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:18:21.360 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:18:21.357+0000 7f31ef400640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e662652290 0x55e66264f200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:33540/0 (socket says 172.21.15.22:33540) 2024-03-28T18:18:21.361 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:18:21.357+0000 7f31ef400640 1 -- 172.21.15.22:0/1235739244 learned_addr learned my addr 172.21.15.22:0/1235739244 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-28T18:18:21.361 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:18:21.357+0000 7f31eea00640 1 --2- 172.21.15.22:0/1235739244 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55e662648f20 0x55e662651930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:18:21.361 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:18:21.357+0000 7f31e6000640 1 --2- 172.21.15.22:0/1235739244 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e662649c00 0x55e66264ecc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:18:21.361 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:18:21.358+0000 7f31ef400640 1 -- 172.21.15.22:0/1235739244 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e662649c00 msgr2=0x55e66264ecc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T18:18:21.361 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:18:21.358+0000 7f31ef400640 1 --2- 172.21.15.22:0/1235739244 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e662649c00 0x55e66264ecc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:18:21.361 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:18:21.358+0000 7f31ef400640 1 -- 172.21.15.22:0/1235739244 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55e662648f20 msgr2=0x55e662651930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T18:18:21.361 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:18:21.358+0000 7f31ef400640 1 --2- 172.21.15.22:0/1235739244 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55e662648f20 0x55e662651930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:18:21.361 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:18:21.358+0000 7f31ef400640 1 -- 172.21.15.22:0/1235739244 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55e66238b7d0 con 0x55e662652290 2024-03-28T18:18:21.361 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:18:21.358+0000 7f31eea00640 1 --2- 172.21.15.22:0/1235739244 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55e662648f20 0x55e662651930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-28T18:18:21.361 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:18:21.358+0000 7f31ef400640 1 --2- 172.21.15.22:0/1235739244 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e662652290 0x55e66264f200 secure :-1 s=READY pgs=798 cs=0 l=1 rev1=1 crypto rx=0x7f31e8010950 tx=0x7f31e8004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=357ea36dc43d41f8 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:18:21.361 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:18:21.358+0000 7f31dcc00640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-03-28T18:18:21.361 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:18:21.358+0000 7f31dcc00640 1 -- 172.21.15.22:0/1235739244 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31e802a060 con 0x55e662652290 2024-03-28T18:18:21.361 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:18:21.358+0000 7f31f8571740 1 -- 172.21.15.22:0/1235739244 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f31d8070a00 con 0x55e662652290 2024-03-28T18:18:21.361 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:18:21.358+0000 7f31dcc00640 1 -- 172.21.15.22:0/1235739244 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(122..122 src has 1..122) v4 ==== 10389+0+0 (secure 0 0 0) 0x7f31e801db90 con 0x55e662652290 2024-03-28T18:18:21.362 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:18:21.359+0000 7f31dcc00640 1 -- 172.21.15.22:0/1235739244 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=187) v2 ==== 24+0+0 (secure 0 0 0) 0x7f31e8031050 con 0x55e662652290 2024-03-28T18:18:21.362 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:18:21.359+0000 7f31f8571740 10 client.16854 fetch_fsmap learned FSMap version 187 2024-03-28T18:18:21.362 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:18:21.359+0000 7f31f8571740 1 -- 172.21.15.22:0/1235739244 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=187,monmap=4+}) v3 -- 0x55e6622c13f0 con 0x55e662652290 2024-03-28T18:18:21.362 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:18:21.359+0000 7f31dcc00640 1 -- 172.21.15.22:0/1235739244 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f31d8070a00 con 0x55e662652290 2024-03-28T18:18:21.362 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:18:21.359+0000 7f31dcc00640 1 -- 172.21.15.22:0/1235739244 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 187) v1 ==== 6221+0+0 (secure 0 0 0) 0x7f31e8019040 con 0x55e662652290 2024-03-28T18:18:21.362 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:18:21.359+0000 7f31f8571740 10 client.16854 fetch_fsmap finished waiting for FSMap version 187 2024-03-28T18:18:21.362 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:18:21.359+0000 7f31f8571740 10 client.16854 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-28T18:18:21.362 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:18:21.359+0000 7f31f8571740 20 client.16854 populate_metadata read hostname 'smithi022' 2024-03-28T18:18:21.362 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:18:21.359+0000 7f31f8571740 1 --2- 172.21.15.22:0/1235739244 >> [v2:172.21.15.79:6838/2441542443,v1:172.21.15.79:6839/2441542443] conn(0x55e662662430 0x55e662664880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:18:21.362 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:18:21.359+0000 7f31f8571740 4 client.16854 mds_command: new command op to 34553 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-28T18:18:21.362 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:18:21.359+0000 7f31f8571740 1 -- 172.21.15.22:0/1235739244 --> [v2:172.21.15.79:6838/2441542443,v1:172.21.15.79:6839/2441542443] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55e6625ba790 con 0x55e662662430 2024-03-28T18:18:21.364 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:18:21.360+0000 7f31eea00640 1 --2- 172.21.15.22:0/1235739244 >> [v2:172.21.15.79:6838/2441542443,v1:172.21.15.79:6839/2441542443] conn(0x55e662662430 0x55e662664880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:18:21.364 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:18:21.361+0000 7f31eea00640 1 --2- 172.21.15.22:0/1235739244 >> [v2:172.21.15.79:6838/2441542443,v1:172.21.15.79:6839/2441542443] conn(0x55e662662430 0x55e662664880 crc :-1 s=READY pgs=297 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4aceb864e81e4f87 server_cookie=78637853b3558804 in_seq=0 out_seq=0 2024-03-28T18:18:21.364 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:18:21.361+0000 7f31dcc00640 10 client.16854 ms_handle_connect on v2:172.21.15.79:6838/2441542443 2024-03-28T18:18:21.364 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:18:21.362+0000 7f31dcc00640 1 -- 172.21.15.22:0/1235739244 <== mds.0 v2:172.21.15.79:6838/2441542443 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x55e6625ba790 con 0x55e662662430 2024-03-28T18:18:21.364 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:18:21.363+0000 7f31dcc00640 10 client.16854 handle_command_reply: tid=0 2024-03-28T18:18:21.364 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:18:21.363+0000 7f31f8571740 1 client.16854 shutdown 2024-03-28T18:18:21.364 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:18:21.363+0000 7f31f8571740 1 -- 172.21.15.22:0/1235739244 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e662652290 msgr2=0x55e66264f200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:18:21.364 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:18:21.363+0000 7f31f8571740 1 --2- 172.21.15.22:0/1235739244 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e662652290 0x55e66264f200 secure :-1 s=READY pgs=798 cs=0 l=1 rev1=1 crypto rx=0x7f31e8010950 tx=0x7f31e8004a40 comp rx=0 tx=0).stop 2024-03-28T18:18:21.364 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:18:21.363+0000 7f31f8571740 1 -- 172.21.15.22:0/1235739244 shutdown_connections 2024-03-28T18:18:21.364 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:18:21.363+0000 7f31f8571740 1 --2- 172.21.15.22:0/1235739244 >> [v2:172.21.15.79:6838/2441542443,v1:172.21.15.79:6839/2441542443] conn(0x55e662662430 0x55e662664880 crc :-1 s=READY pgs=297 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:18:21.364 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:18:21.363+0000 7f31f8571740 1 --2- 172.21.15.22:0/1235739244 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e662652290 0x55e66264f200 unknown :-1 s=CLOSED pgs=798 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:18:21.364 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:18:21.363+0000 7f31f8571740 1 --2- 172.21.15.22:0/1235739244 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e662649c00 0x55e66264ecc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:18:21.364 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:18:21.363+0000 7f31f8571740 1 --2- 172.21.15.22:0/1235739244 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55e662648f20 0x55e662651930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:18:21.365 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:18:21.363+0000 7f31dcc00640 0 client.16854 ms_handle_reset on v2:172.21.15.79:6838/2441542443 2024-03-28T18:18:21.365 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:18:21.363+0000 7f31f8571740 1 -- 172.21.15.22:0/1235739244 >> 172.21.15.22:0/1235739244 conn(0x55e662610200 msgr2=0x55e6626456c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:18:21.365 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:18:21.363+0000 7f31f8571740 1 -- 172.21.15.22:0/1235739244 shutdown_connections 2024-03-28T18:18:21.365 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:18:21.363+0000 7f31f8571740 1 -- 172.21.15.22:0/1235739244 wait complete. 2024-03-28T18:18:21.365 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:18:21.363+0000 7f31f8571740 20 client.16854 trim_cache size 0 max 16384 2024-03-28T18:18:21.381 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:18:21.381+0000 7f31f8571740 1 Processor -- start 2024-03-28T18:18:21.381 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:18:21.381+0000 7f31f8571740 1 -- start start 2024-03-28T18:18:21.381 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:18:21.381+0000 7f31f8571740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e662648f20 0x55e662653680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:18:21.381 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:18:21.381+0000 7f31f8571740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55e662649c00 0x55e662653bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:18:21.381 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:18:21.381+0000 7f31f8571740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e662652290 0x55e66264d5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:18:21.381 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:18:21.381+0000 7f31f8571740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31f01b1f60 con 0x55e662648f20 2024-03-28T18:18:21.381 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:18:21.381+0000 7f31f8571740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31f01af2c0 con 0x55e662649c00 2024-03-28T18:18:21.381 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:18:21.381+0000 7f31f8571740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31f01b12a0 con 0x55e662652290 2024-03-28T18:18:21.383 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:18:21.382+0000 7f31e6000640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55e662649c00 0x55e662653bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:18:21.383 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:18:21.382+0000 7f31eea00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e662648f20 0x55e662653680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:18:21.383 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:18:21.382+0000 7f31eea00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e662648f20 0x55e662653680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:33546/0 (socket says 172.21.15.22:33546) 2024-03-28T18:18:21.384 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:18:21.382+0000 7f31ef400640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e662652290 0x55e66264d5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:18:21.384 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:18:21.382+0000 7f31e6000640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55e662649c00 0x55e662653bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.22:52364/0 (socket says 172.21.15.22:52364) 2024-03-28T18:18:21.384 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:18:21.382+0000 7f31eea00640 1 -- 172.21.15.22:0/215727915 learned_addr learned my addr 172.21.15.22:0/215727915 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-28T18:18:21.384 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:18:21.382+0000 7f31eea00640 1 -- 172.21.15.22:0/215727915 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e662652290 msgr2=0x55e66264d5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T18:18:21.384 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:18:21.382+0000 7f31eea00640 1 --2- 172.21.15.22:0/215727915 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e662652290 0x55e66264d5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:18:21.384 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:18:21.382+0000 7f31eea00640 1 -- 172.21.15.22:0/215727915 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55e662649c00 msgr2=0x55e662653bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T18:18:21.384 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:18:21.382+0000 7f31eea00640 1 --2- 172.21.15.22:0/215727915 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55e662649c00 0x55e662653bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:18:21.384 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:18:21.382+0000 7f31eea00640 1 -- 172.21.15.22:0/215727915 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e6622810f0 con 0x55e662648f20 2024-03-28T18:18:21.384 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:18:21.383+0000 7f31eea00640 1 --2- 172.21.15.22:0/215727915 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e662648f20 0x55e662653680 secure :-1 s=READY pgs=799 cs=0 l=1 rev1=1 crypto rx=0x7f31e0009930 tx=0x7f31e0005a60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=615300591b2d50cd server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:18:21.384 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:18:21.383+0000 7f31df400640 1 -- 172.21.15.22:0/215727915 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31e002c650 con 0x55e662648f20 2024-03-28T18:18:21.384 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:18:21.383+0000 7f31df400640 1 -- 172.21.15.22:0/215727915 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f31e002ccc0 con 0x55e662648f20 2024-03-28T18:18:21.384 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:18:21.383+0000 7f31df400640 1 -- 172.21.15.22:0/215727915 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31e0003850 con 0x55e662648f20 2024-03-28T18:18:21.384 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:18:21.384+0000 7f31f8571740 1 -- 172.21.15.22:0/215727915 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e662648f20 msgr2=0x55e662653680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:18:21.385 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:18:21.384+0000 7f31f8571740 1 --2- 172.21.15.22:0/215727915 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e662648f20 0x55e662653680 secure :-1 s=READY pgs=799 cs=0 l=1 rev1=1 crypto rx=0x7f31e0009930 tx=0x7f31e0005a60 comp rx=0 tx=0).stop 2024-03-28T18:18:21.385 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:18:21.384+0000 7f31f8571740 1 -- 172.21.15.22:0/215727915 shutdown_connections 2024-03-28T18:18:21.385 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:18:21.384+0000 7f31f8571740 1 --2- 172.21.15.22:0/215727915 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e662652290 0x55e66264d5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:18:21.385 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:18:21.384+0000 7f31f8571740 1 --2- 172.21.15.22:0/215727915 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55e662649c00 0x55e662653bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:18:21.385 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:18:21.384+0000 7f31f8571740 1 --2- 172.21.15.22:0/215727915 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e662648f20 0x55e662653680 unknown :-1 s=CLOSED pgs=799 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:18:21.385 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:18:21.384+0000 7f31f8571740 1 -- 172.21.15.22:0/215727915 >> 172.21.15.22:0/215727915 conn(0x55e662610200 msgr2=0x55e6626434a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:18:21.385 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:18:21.384+0000 7f31f8571740 1 -- 172.21.15.22:0/215727915 shutdown_connections 2024-03-28T18:18:21.385 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:18:21.384+0000 7f31f8571740 1 -- 172.21.15.22:0/215727915 wait complete. 2024-03-28T18:18:21.388 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:18:21.384+0000 7f31f8571740 1 -- start start 2024-03-28T18:18:21.388 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:18:21.384+0000 7f31f8571740 1 Processor -- start 2024-03-28T18:18:21.388 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:18:21.385+0000 7f31f8571740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55e662648f20 0x55e66264b4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:18:21.388 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:18:21.385+0000 7f31f8571740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e662649c00 0x55e66264ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:18:21.388 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:18:21.385+0000 7f31f8571740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e662652290 0x55e66264bf50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:18:21.388 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:18:21.385+0000 7f31f8571740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31f01b0c40 con 0x55e662652290 2024-03-28T18:18:21.388 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:18:21.385+0000 7f31f8571740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31f01af920 con 0x55e662648f20 2024-03-28T18:18:21.389 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:18:21.385+0000 7f31f8571740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31f01aff80 con 0x55e662649c00 2024-03-28T18:18:21.389 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:18:21.385+0000 7f31f8571740 10 client.0 _ll_register_callbacks cb 0x7f31e8009c40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-28T18:18:21.389 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:18:21.385+0000 7f31ef400640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e662652290 0x55e66264bf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:18:21.389 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:18:21.385+0000 7f31ef400640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e662652290 0x55e66264bf50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:33554/0 (socket says 172.21.15.22:33554) 2024-03-28T18:18:21.389 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:18:21.385+0000 7f31ef400640 1 -- 172.21.15.22:0/2022953513 learned_addr learned my addr 172.21.15.22:0/2022953513 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-28T18:18:21.389 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:18:21.385+0000 7f31eea00640 1 --2- 172.21.15.22:0/2022953513 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55e662648f20 0x55e66264b4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:18:21.389 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:18:21.385+0000 7f31e6000640 1 --2- 172.21.15.22:0/2022953513 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e662649c00 0x55e66264ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:18:21.389 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:18:21.386+0000 7f31ef400640 1 -- 172.21.15.22:0/2022953513 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e662649c00 msgr2=0x55e66264ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T18:18:21.389 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:18:21.386+0000 7f31ef400640 1 --2- 172.21.15.22:0/2022953513 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e662649c00 0x55e66264ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:18:21.389 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:18:21.386+0000 7f31ef400640 1 -- 172.21.15.22:0/2022953513 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55e662648f20 msgr2=0x55e66264b4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T18:18:21.389 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:18:21.386+0000 7f31ef400640 1 --2- 172.21.15.22:0/2022953513 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55e662648f20 0x55e66264b4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:18:21.389 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:18:21.386+0000 7f31ef400640 1 -- 172.21.15.22:0/2022953513 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55e6626c55a0 con 0x55e662652290 2024-03-28T18:18:21.389 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:18:21.386+0000 7f31e6000640 1 --2- 172.21.15.22:0/2022953513 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e662649c00 0x55e66264ba10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T18:18:21.389 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:18:21.386+0000 7f31ef400640 1 --2- 172.21.15.22:0/2022953513 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e662652290 0x55e66264bf50 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 crypto rx=0x7f31e8023db0 tx=0x7f31e8009cf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d0d9deffdc1adcb8 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:18:21.390 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:18:21.386+0000 7f31dcc00640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-03-28T18:18:21.390 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:18:21.386+0000 7f31dcc00640 1 -- 172.21.15.22:0/2022953513 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31e801dd90 con 0x55e662652290 2024-03-28T18:18:21.390 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:18:21.386+0000 7f31f8571740 1 -- 172.21.15.22:0/2022953513 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55e6625b4ae0 con 0x55e662652290 2024-03-28T18:18:21.390 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:18:21.386+0000 7f31dcc00640 1 -- 172.21.15.22:0/2022953513 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(122..122 src has 1..122) v4 ==== 10389+0+0 (secure 0 0 0) 0x7f31e8004ab0 con 0x55e662652290 2024-03-28T18:18:21.390 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:18:21.386+0000 7f31dcc00640 1 -- 172.21.15.22:0/2022953513 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=187) v2 ==== 24+0+0 (secure 0 0 0) 0x7f31e8023e40 con 0x55e662652290 2024-03-28T18:18:21.390 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:18:21.386+0000 7f31f8571740 10 client.16860 fetch_fsmap learned FSMap version 187 2024-03-28T18:18:21.390 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:18:21.386+0000 7f31f8571740 1 -- 172.21.15.22:0/2022953513 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=187,monmap=4+}) v3 -- 0x55e6626c68b0 con 0x55e662652290 2024-03-28T18:18:21.390 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:18:21.387+0000 7f31dcc00640 1 -- 172.21.15.22:0/2022953513 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x55e6625b4ae0 con 0x55e662652290 2024-03-28T18:18:21.390 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:18:21.387+0000 7f31dcc00640 1 -- 172.21.15.22:0/2022953513 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 187) v1 ==== 6221+0+0 (secure 0 0 0) 0x7f31e8019040 con 0x55e662652290 2024-03-28T18:18:21.390 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:18:21.387+0000 7f31f8571740 10 client.16860 fetch_fsmap finished waiting for FSMap version 187 2024-03-28T18:18:21.390 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:18:21.387+0000 7f31f8571740 10 client.16860 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-28T18:18:21.390 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:18:21.387+0000 7f31f8571740 20 client.16860 populate_metadata read hostname 'smithi022' 2024-03-28T18:18:21.390 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:18:21.387+0000 7f31f8571740 1 --2- 172.21.15.22:0/2022953513 >> [v2:172.21.15.79:6838/2441542443,v1:172.21.15.79:6839/2441542443] conn(0x55e6626c7000 0x55e6626c9420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:18:21.390 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:18:21.387+0000 7f31f8571740 4 client.16860 mds_command: new command op to 34553 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-28T18:18:21.390 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:18:21.387+0000 7f31f8571740 1 -- 172.21.15.22:0/2022953513 --> [v2:172.21.15.79:6838/2441542443,v1:172.21.15.79:6839/2441542443] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55e6625323a0 con 0x55e6626c7000 2024-03-28T18:18:21.391 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:18:21.388+0000 7f31eea00640 1 --2- 172.21.15.22:0/2022953513 >> [v2:172.21.15.79:6838/2441542443,v1:172.21.15.79:6839/2441542443] conn(0x55e6626c7000 0x55e6626c9420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:18:21.391 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:18:21.389+0000 7f31eea00640 1 --2- 172.21.15.22:0/2022953513 >> [v2:172.21.15.79:6838/2441542443,v1:172.21.15.79:6839/2441542443] conn(0x55e6626c7000 0x55e6626c9420 crc :-1 s=READY pgs=298 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=14c4fc74ce212fc7 server_cookie=f1f7fa5456cb6443 in_seq=0 out_seq=0 2024-03-28T18:18:21.391 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:18:21.389+0000 7f31dcc00640 10 client.16860 ms_handle_connect on v2:172.21.15.79:6838/2441542443 2024-03-28T18:18:21.392 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:18:21.389+0000 7f31dcc00640 1 -- 172.21.15.22:0/2022953513 <== mds.0 v2:172.21.15.79:6838/2441542443 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x55e6625323a0 con 0x55e6626c7000 2024-03-28T18:18:21.392 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:18:21.389+0000 7f31dcc00640 10 client.16860 handle_command_reply: tid=0 2024-03-28T18:18:21.392 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:18:21.389+0000 7f31f8571740 1 client.16860 shutdown 2024-03-28T18:18:21.392 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:18:21.389+0000 7f31f8571740 1 -- 172.21.15.22:0/2022953513 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e662652290 msgr2=0x55e66264bf50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:18:21.392 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:18:21.389+0000 7f31f8571740 1 --2- 172.21.15.22:0/2022953513 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e662652290 0x55e66264bf50 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 crypto rx=0x7f31e8023db0 tx=0x7f31e8009cf0 comp rx=0 tx=0).stop 2024-03-28T18:18:21.392 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:18:21.389+0000 7f31f8571740 1 -- 172.21.15.22:0/2022953513 shutdown_connections 2024-03-28T18:18:21.392 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:18:21.389+0000 7f31f8571740 1 --2- 172.21.15.22:0/2022953513 >> [v2:172.21.15.79:6838/2441542443,v1:172.21.15.79:6839/2441542443] conn(0x55e6626c7000 0x55e6626c9420 crc :-1 s=READY pgs=298 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:18:21.392 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:18:21.389+0000 7f31f8571740 1 --2- 172.21.15.22:0/2022953513 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e662652290 0x55e66264bf50 unknown :-1 s=CLOSED pgs=800 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:18:21.392 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:18:21.389+0000 7f31f8571740 1 --2- 172.21.15.22:0/2022953513 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e662649c00 0x55e66264ba10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:18:21.392 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:18:21.389+0000 7f31f8571740 1 --2- 172.21.15.22:0/2022953513 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55e662648f20 0x55e66264b4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:18:21.392 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:18:21.389+0000 7f31f8571740 1 -- 172.21.15.22:0/2022953513 >> 172.21.15.22:0/2022953513 conn(0x55e662610200 msgr2=0x55e66264d570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:18:21.392 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:18:21.389+0000 7f31dcc00640 0 client.16860 ms_handle_reset on v2:172.21.15.79:6838/2441542443 2024-03-28T18:18:21.393 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:18:21.390+0000 7f31f8571740 1 -- 172.21.15.22:0/2022953513 shutdown_connections 2024-03-28T18:18:21.393 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:18:21.390+0000 7f31f8571740 1 -- 172.21.15.22:0/2022953513 wait complete. 2024-03-28T18:18:21.393 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:18:21.390+0000 7f31f8571740 20 client.16860 trim_cache size 0 max 16384 2024-03-28T18:18:21.393 INFO:teuthology.orchestra.run.smithi022.stdout:{ 2024-03-28T18:18:21.393 INFO:teuthology.orchestra.run.smithi022.stdout: "status": "IDLE (waiting for more scrubs)", 2024-03-28T18:18:21.393 INFO:teuthology.orchestra.run.smithi022.stdout: "scrubs": { 2024-03-28T18:18:21.393 INFO:teuthology.orchestra.run.smithi022.stdout: "4bbbd61d-29a3-4bf6-a4f2-d5783d585de3": { 2024-03-28T18:18:21.393 INFO:teuthology.orchestra.run.smithi022.stdout: "path": "/", 2024-03-28T18:18:21.393 INFO:teuthology.orchestra.run.smithi022.stdout: "tag": "4bbbd61d-29a3-4bf6-a4f2-d5783d585de3", 2024-03-28T18:18:21.393 INFO:teuthology.orchestra.run.smithi022.stdout: "options": "recursive,force" 2024-03-28T18:18:21.393 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-03-28T18:18:21.394 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-03-28T18:18:21.394 INFO:teuthology.orchestra.run.smithi022.stdout:} 2024-03-28T18:18:21.394 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:18:21.390+0000 7f31f5800640 1 -- 172.21.15.22:0/1144170155 >> [v2:172.21.15.22:6800/1357365082,v1:172.21.15.22:6801/1357365082] conn(0x7f31d40a29f0 msgr2=0x7f31d40a4eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:18:21.394 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:18:21.390+0000 7f31f5800640 1 --2- 172.21.15.22:0/1144170155 >> [v2:172.21.15.22:6800/1357365082,v1:172.21.15.22:6801/1357365082] conn(0x7f31d40a29f0 0x7f31d40a4eb0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f31e8005fd0 tx=0x7f31e8009760 comp rx=0 tx=0).stop 2024-03-28T18:18:21.394 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:18:21.390+0000 7f31f5800640 1 -- 172.21.15.22:0/1144170155 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f31f0101fd0 msgr2=0x7f31f0073ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:18:21.394 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:18:21.390+0000 7f31f5800640 1 --2- 172.21.15.22:0/1144170155 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f31f0101fd0 0x7f31f0073ee0 secure :-1 s=READY pgs=796 cs=0 l=1 rev1=1 crypto rx=0x7f31d800c970 tx=0x7f31d800ce30 comp rx=0 tx=0).stop 2024-03-28T18:18:21.395 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:18:21.390+0000 7f31f5800640 1 -- 172.21.15.22:0/1144170155 shutdown_connections 2024-03-28T18:18:21.395 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:18:21.390+0000 7f31f5800640 1 --2- 172.21.15.22:0/1144170155 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f31f01076b0 0x7f31f0074420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:18:21.395 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:18:21.390+0000 7f31f5800640 1 --2- 172.21.15.22:0/1144170155 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f31f0101fd0 0x7f31f0073ee0 unknown :-1 s=CLOSED pgs=796 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:18:21.395 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:18:21.390+0000 7f31f5800640 1 --2- 172.21.15.22:0/1144170155 >> [v2:172.21.15.22:6800/1357365082,v1:172.21.15.22:6801/1357365082] conn(0x7f31d40a29f0 0x7f31d40a4eb0 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:18:21.395 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:18:21.390+0000 7f31f5800640 1 --2- 172.21.15.22:0/1144170155 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f31f0101610 0x7f31f00739a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:18:21.395 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:18:21.390+0000 7f31f5800640 1 -- 172.21.15.22:0/1144170155 >> 172.21.15.22:0/1144170155 conn(0x7f31f00fd100 msgr2=0x7f31f00fe890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:18:21.395 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:18:21.391+0000 7f31f5800640 1 -- 172.21.15.22:0/1144170155 shutdown_connections 2024-03-28T18:18:21.395 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:18:21.391+0000 7f31f5800640 1 -- 172.21.15.22:0/1144170155 wait complete. 2024-03-28T18:18:21.407 INFO:tasks.cephfs.filesystem:scrub status for tag:4bbbd61d-29a3-4bf6-a4f2-d5783d585de3 - {'path': '/', 'tag': '4bbbd61d-29a3-4bf6-a4f2-d5783d585de3', 'options': 'recursive,force'} 2024-03-28T18:18:23.174 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:18:22 smithi134 ceph-mon[35037]: pgmap v357: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:18:23.182 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:18:22 smithi079 ceph-mon[35312]: pgmap v357: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:18:23.281 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:18:22 smithi022 ceph-mon[31985]: pgmap v357: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:18:25.174 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:18:24 smithi134 ceph-mon[35037]: pgmap v358: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:18:25.182 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:18:24 smithi079 ceph-mon[35312]: pgmap v358: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:18:25.281 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:18:24 smithi022 ceph-mon[31985]: pgmap v358: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:18:27.084 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:18:26 smithi022 ceph-mon[31985]: pgmap v359: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:18:27.174 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:18:26 smithi134 ceph-mon[35037]: pgmap v359: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:18:27.182 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:18:26 smithi079 ceph-mon[35312]: pgmap v359: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:18:29.174 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:18:28 smithi134 ceph-mon[35037]: pgmap v360: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:18:29.182 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:18:28 smithi079 ceph-mon[35312]: pgmap v360: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:18:29.281 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:18:28 smithi022 ceph-mon[31985]: pgmap v360: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:18:31.175 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:18:30 smithi134 ceph-mon[35037]: pgmap v361: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 9 op/s 2024-03-28T18:18:31.182 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:18:30 smithi079 ceph-mon[35312]: pgmap v361: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 9 op/s 2024-03-28T18:18:31.282 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:18:30 smithi022 ceph-mon[31985]: pgmap v361: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 9 op/s 2024-03-28T18:18:33.174 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:18:32 smithi134 ceph-mon[35037]: pgmap v362: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T18:18:33.182 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:18:32 smithi079 ceph-mon[35312]: pgmap v362: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T18:18:33.281 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:18:32 smithi022 ceph-mon[31985]: pgmap v362: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T18:18:35.175 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:18:34 smithi134 ceph-mon[35037]: pgmap v363: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T18:18:35.182 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:18:34 smithi079 ceph-mon[35312]: pgmap v363: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T18:18:35.282 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:18:34 smithi022 ceph-mon[31985]: pgmap v363: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T18:18:37.175 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:18:36 smithi134 ceph-mon[35037]: pgmap v364: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 9 op/s 2024-03-28T18:18:37.183 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:18:36 smithi079 ceph-mon[35312]: pgmap v364: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 9 op/s 2024-03-28T18:18:37.282 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:18:36 smithi022 ceph-mon[31985]: pgmap v364: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 9 op/s 2024-03-28T18:18:39.174 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:18:38 smithi134 ceph-mon[35037]: pgmap v365: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T18:18:39.183 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:18:38 smithi079 ceph-mon[35312]: pgmap v365: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T18:18:39.190 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:18:38 smithi022 ceph-mon[31985]: pgmap v365: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T18:18:41.175 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:18:40 smithi134 ceph-mon[35037]: pgmap v366: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 9 op/s 2024-03-28T18:18:41.182 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:18:40 smithi079 ceph-mon[35312]: pgmap v366: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 9 op/s 2024-03-28T18:18:41.281 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:18:40 smithi022 ceph-mon[31985]: pgmap v366: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 9 op/s 2024-03-28T18:18:43.175 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:18:42 smithi134 ceph-mon[35037]: pgmap v367: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:18:43.183 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:18:42 smithi079 ceph-mon[35312]: pgmap v367: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:18:43.282 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:18:42 smithi022 ceph-mon[31985]: pgmap v367: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:18:45.175 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:18:44 smithi134 ceph-mon[35037]: pgmap v368: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:18:45.182 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:18:44 smithi079 ceph-mon[35312]: pgmap v368: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:18:45.282 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:18:44 smithi022 ceph-mon[31985]: pgmap v368: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:18:47.174 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:18:46 smithi134 ceph-mon[35037]: pgmap v369: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:18:47.182 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:18:46 smithi079 ceph-mon[35312]: pgmap v369: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:18:47.282 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:18:46 smithi022 ceph-mon[31985]: pgmap v369: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:18:49.175 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:18:48 smithi134 ceph-mon[35037]: pgmap v370: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:18:49.182 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:18:48 smithi079 ceph-mon[35312]: pgmap v370: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:18:49.281 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:18:48 smithi022 ceph-mon[31985]: pgmap v370: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:18:51.175 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:18:50 smithi134 ceph-mon[35037]: pgmap v371: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:18:51.182 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:18:50 smithi079 ceph-mon[35312]: pgmap v371: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:18:51.282 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:18:50 smithi022 ceph-mon[31985]: pgmap v371: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:18:51.409 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-28T18:18:51.409 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-28T18:18:51.652 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:18:51.651+0000 7fddcf200640 1 -- 172.21.15.22:0/2931245229 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fddd0102ff0 msgr2=0x7fddd010b4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:18:51.652 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:18:51.651+0000 7fddcf200640 1 --2- 172.21.15.22:0/2931245229 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fddd0102ff0 0x7fddd010b4d0 secure :-1 s=READY pgs=801 cs=0 l=1 rev1=1 crypto rx=0x7fddc0009ca0 tx=0x7fddc002c180 comp rx=0 tx=0).stop 2024-03-28T18:18:51.653 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:18:51.652+0000 7fddcf200640 1 -- 172.21.15.22:0/2931245229 shutdown_connections 2024-03-28T18:18:51.653 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:18:51.652+0000 7fddcf200640 1 --2- 172.21.15.22:0/2931245229 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fddd010baf0 0x7fddd010df90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:18:51.653 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:18:51.652+0000 7fddcf200640 1 --2- 172.21.15.22:0/2931245229 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fddd0102ff0 0x7fddd010b4d0 unknown :-1 s=CLOSED pgs=801 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:18:51.653 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:18:51.652+0000 7fddcf200640 1 --2- 172.21.15.22:0/2931245229 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fddd01026d0 0x7fddd0102ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:18:51.653 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:18:51.652+0000 7fddcf200640 1 -- 172.21.15.22:0/2931245229 >> 172.21.15.22:0/2931245229 conn(0x7fddd00fbef0 msgr2=0x7fddd00fe310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:18:51.653 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:18:51.652+0000 7fddcf200640 1 -- 172.21.15.22:0/2931245229 shutdown_connections 2024-03-28T18:18:51.653 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:18:51.652+0000 7fddcf200640 1 -- 172.21.15.22:0/2931245229 wait complete. 2024-03-28T18:18:51.657 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:18:51.653+0000 7fddcf200640 1 Processor -- start 2024-03-28T18:18:51.657 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:18:51.653+0000 7fddcf200640 1 -- start start 2024-03-28T18:18:51.657 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:18:51.653+0000 7fddcf200640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fddd01026d0 0x7fddd0100320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:18:51.657 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:18:51.653+0000 7fddcf200640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fddd0102ff0 0x7fddd0100860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:18:51.657 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:18:51.653+0000 7fddcf200640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fddd010baf0 0x7fddd0100da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:18:51.657 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:18:51.653+0000 7fddcf200640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fddd01118c0 con 0x7fddd0102ff0 2024-03-28T18:18:51.657 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:18:51.653+0000 7fddcf200640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fddd0111740 con 0x7fddd010baf0 2024-03-28T18:18:51.657 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:18:51.653+0000 7fddcf200640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fddd0111a40 con 0x7fddd01026d0 2024-03-28T18:18:51.658 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:18:51.654+0000 7fddcd400640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fddd0102ff0 0x7fddd0100860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:18:51.658 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:18:51.654+0000 7fddcd400640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fddd0102ff0 0x7fddd0100860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:48440/0 (socket says 172.21.15.22:48440) 2024-03-28T18:18:51.658 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:18:51.654+0000 7fddcd400640 1 -- 172.21.15.22:0/911355541 learned_addr learned my addr 172.21.15.22:0/911355541 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-28T18:18:51.658 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:18:51.654+0000 7fddce800640 1 --2- 172.21.15.22:0/911355541 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fddd010baf0 0x7fddd0100da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:18:51.658 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:18:51.654+0000 7fddcde00640 1 --2- 172.21.15.22:0/911355541 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fddd01026d0 0x7fddd0100320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:18:51.658 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:18:51.654+0000 7fddcd400640 1 -- 172.21.15.22:0/911355541 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fddd01026d0 msgr2=0x7fddd0100320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:18:51.658 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:18:51.654+0000 7fddcd400640 1 --2- 172.21.15.22:0/911355541 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fddd01026d0 0x7fddd0100320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:18:51.658 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:18:51.654+0000 7fddcd400640 1 -- 172.21.15.22:0/911355541 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fddd010baf0 msgr2=0x7fddd0100da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:18:51.659 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:18:51.654+0000 7fddcd400640 1 --2- 172.21.15.22:0/911355541 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fddd010baf0 0x7fddd0100da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:18:51.659 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:18:51.654+0000 7fddcd400640 1 -- 172.21.15.22:0/911355541 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fddd01a6320 con 0x7fddd0102ff0 2024-03-28T18:18:51.659 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:18:51.655+0000 7fddcd400640 1 --2- 172.21.15.22:0/911355541 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fddd0102ff0 0x7fddd0100860 secure :-1 s=READY pgs=802 cs=0 l=1 rev1=1 crypto rx=0x7fddc002c790 tx=0x7fddc0034d60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:18:51.659 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:18:51.655+0000 7fddc6a00640 1 -- 172.21.15.22:0/911355541 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fddc0039050 con 0x7fddd0102ff0 2024-03-28T18:18:51.659 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:18:51.655+0000 7fddc6a00640 1 -- 172.21.15.22:0/911355541 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7fddc003a040 con 0x7fddd0102ff0 2024-03-28T18:18:51.659 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:18:51.655+0000 7fddcf200640 1 -- 172.21.15.22:0/911355541 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fddd01a65b0 con 0x7fddd0102ff0 2024-03-28T18:18:51.659 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:18:51.655+0000 7fddc6a00640 1 -- 172.21.15.22:0/911355541 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fddc0032b00 con 0x7fddd0102ff0 2024-03-28T18:18:51.659 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:18:51.655+0000 7fddcf200640 1 -- 172.21.15.22:0/911355541 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fddd01a6aa0 con 0x7fddd0102ff0 2024-03-28T18:18:51.667 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:18:51.658+0000 7fddc6a00640 1 -- 172.21.15.22:0/911355541 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 26) v1 ==== 137421+0+0 (secure 0 0 0) 0x7fddc00046a0 con 0x7fddd0102ff0 2024-03-28T18:18:51.668 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:18:51.658+0000 7fddd801f740 1 Processor -- start 2024-03-28T18:18:51.668 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:18:51.658+0000 7fddd801f740 1 -- start start 2024-03-28T18:18:51.668 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:18:51.659+0000 7fddd801f740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55ae75e6bdd0 0x55ae75e6ab20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:18:51.668 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:18:51.659+0000 7fddd801f740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ae75e6b0f0 0x55ae75e6b570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:18:51.668 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:18:51.659+0000 7fddd801f740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ae75e74460 0x55ae75e76850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:18:51.668 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:18:51.659+0000 7fddd801f740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ae75dcf250 con 0x55ae75e74460 2024-03-28T18:18:51.668 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:18:51.659+0000 7fddd801f740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ae75d77a30 con 0x55ae75e6bdd0 2024-03-28T18:18:51.668 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:18:51.659+0000 7fddd801f740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ae75dcb2f0 con 0x55ae75e6b0f0 2024-03-28T18:18:51.668 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:18:51.659+0000 7fddcd400640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ae75e74460 0x55ae75e76850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:18:51.668 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:18:51.659+0000 7fddcd400640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ae75e74460 0x55ae75e76850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:48450/0 (socket says 172.21.15.22:48450) 2024-03-28T18:18:51.669 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:18:51.659+0000 7fddcd400640 1 -- 172.21.15.22:0/1254093406 learned_addr learned my addr 172.21.15.22:0/1254093406 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-28T18:18:51.669 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:18:51.659+0000 7fddce800640 1 --2- 172.21.15.22:0/1254093406 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55ae75e6bdd0 0x55ae75e6ab20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:18:51.669 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:18:51.660+0000 7fddcde00640 1 --2- 172.21.15.22:0/1254093406 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ae75e6b0f0 0x55ae75e6b570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:18:51.669 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:18:51.660+0000 7fddcd400640 1 -- 172.21.15.22:0/1254093406 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ae75e6b0f0 msgr2=0x55ae75e6b570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T18:18:51.669 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:18:51.660+0000 7fddcd400640 1 --2- 172.21.15.22:0/1254093406 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ae75e6b0f0 0x55ae75e6b570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:18:51.669 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:18:51.660+0000 7fddcd400640 1 -- 172.21.15.22:0/1254093406 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55ae75e6bdd0 msgr2=0x55ae75e6ab20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T18:18:51.669 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:18:51.660+0000 7fddcd400640 1 --2- 172.21.15.22:0/1254093406 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55ae75e6bdd0 0x55ae75e6ab20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:18:51.669 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:18:51.660+0000 7fddcd400640 1 -- 172.21.15.22:0/1254093406 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ae75d62690 con 0x55ae75e74460 2024-03-28T18:18:51.669 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:18:51.660+0000 7fddc6a00640 1 --2- 172.21.15.22:0/911355541 >> [v2:172.21.15.22:6800/1357365082,v1:172.21.15.22:6801/1357365082] conn(0x7fddb40a2970 0x7fddb40a4e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:18:51.669 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:18:51.660+0000 7fddc6a00640 1 -- 172.21.15.22:0/911355541 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(122..122 src has 1..122) v4 ==== 10389+0+0 (secure 0 0 0) 0x7fddc00f1ae0 con 0x7fddd0102ff0 2024-03-28T18:18:51.670 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:18:51.660+0000 7fddce800640 1 --2- 172.21.15.22:0/911355541 >> [v2:172.21.15.22:6800/1357365082,v1:172.21.15.22:6801/1357365082] conn(0x7fddb40a2970 0x7fddb40a4e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:18:51.670 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:18:51.660+0000 7fddcd400640 1 --2- 172.21.15.22:0/1254093406 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ae75e74460 0x55ae75e76850 secure :-1 s=READY pgs=803 cs=0 l=1 rev1=1 crypto rx=0x7fddc00eead0 tx=0x7fddc00f5b80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1671b59aaccd287b server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:18:51.670 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:18:51.661+0000 7fddbf400640 1 -- 172.21.15.22:0/1254093406 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fddc00b9d40 con 0x55ae75e74460 2024-03-28T18:18:51.670 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:18:51.661+0000 7fddbf400640 1 -- 172.21.15.22:0/1254093406 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7fddc00be1a0 con 0x55ae75e74460 2024-03-28T18:18:51.670 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:18:51.661+0000 7fddbf400640 1 -- 172.21.15.22:0/1254093406 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fddc00bd400 con 0x55ae75e74460 2024-03-28T18:18:51.670 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:18:51.662+0000 7fddd801f740 1 -- 172.21.15.22:0/1254093406 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ae75e74460 msgr2=0x55ae75e76850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:18:51.670 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:18:51.662+0000 7fddd801f740 1 --2- 172.21.15.22:0/1254093406 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ae75e74460 0x55ae75e76850 secure :-1 s=READY pgs=803 cs=0 l=1 rev1=1 crypto rx=0x7fddc00eead0 tx=0x7fddc00f5b80 comp rx=0 tx=0).stop 2024-03-28T18:18:51.670 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:18:51.662+0000 7fddce800640 1 --2- 172.21.15.22:0/911355541 >> [v2:172.21.15.22:6800/1357365082,v1:172.21.15.22:6801/1357365082] conn(0x7fddb40a2970 0x7fddb40a4e30 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fddc8009c70 tx=0x7fddc8005f90 comp rx=0 tx=0).ready entity=mgr.15522 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:18:51.670 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:18:51.662+0000 7fddd801f740 1 -- 172.21.15.22:0/1254093406 shutdown_connections 2024-03-28T18:18:51.670 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:18:51.662+0000 7fddd801f740 1 --2- 172.21.15.22:0/1254093406 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ae75e74460 0x55ae75e76850 unknown :-1 s=CLOSED pgs=803 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:18:51.671 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:18:51.662+0000 7fddd801f740 1 --2- 172.21.15.22:0/1254093406 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ae75e6b0f0 0x55ae75e6b570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:18:51.671 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:18:51.662+0000 7fddd801f740 1 --2- 172.21.15.22:0/1254093406 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55ae75e6bdd0 0x55ae75e6ab20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:18:51.671 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:18:51.662+0000 7fddd801f740 1 -- 172.21.15.22:0/1254093406 >> 172.21.15.22:0/1254093406 conn(0x55ae75e32200 msgr2=0x55ae75e65b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:18:51.671 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:18:51.662+0000 7fddd801f740 1 -- 172.21.15.22:0/1254093406 shutdown_connections 2024-03-28T18:18:51.671 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:18:51.662+0000 7fddd801f740 1 -- 172.21.15.22:0/1254093406 wait complete. 2024-03-28T18:18:51.671 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:18:51.662+0000 7fddd801f740 1 -- start start 2024-03-28T18:18:51.671 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:18:51.663+0000 7fddd801f740 1 Processor -- start 2024-03-28T18:18:51.671 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:18:51.663+0000 7fddd801f740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ae75e6b0f0 0x55ae75e73bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:18:51.671 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:18:51.663+0000 7fddd801f740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55ae75e6bdd0 0x55ae75e70f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:18:51.671 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:18:51.663+0000 7fddd801f740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ae75e74460 0x55ae75e71490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:18:51.672 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:18:51.663+0000 7fddd801f740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ae75dcf250 con 0x55ae75e6b0f0 2024-03-28T18:18:51.672 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:18:51.663+0000 7fddd801f740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ae75d8ae80 con 0x55ae75e6bdd0 2024-03-28T18:18:51.672 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:18:51.663+0000 7fddd801f740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ae75e77090 con 0x55ae75e74460 2024-03-28T18:18:51.672 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:18:51.663+0000 7fddd801f740 10 client.0 _ll_register_callbacks cb 0x55ae75d9d130 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-28T18:18:51.672 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:18:51.663+0000 7fddcde00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ae75e6b0f0 0x55ae75e73bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:18:51.672 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:18:51.663+0000 7fddcde00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ae75e6b0f0 0x55ae75e73bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:48462/0 (socket says 172.21.15.22:48462) 2024-03-28T18:18:51.672 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:18:51.663+0000 7fddcde00640 1 -- 172.21.15.22:0/1223345597 learned_addr learned my addr 172.21.15.22:0/1223345597 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-28T18:18:51.672 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:18:51.663+0000 7fddcd400640 1 --2- 172.21.15.22:0/1223345597 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55ae75e6bdd0 0x55ae75e70f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:18:51.672 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:18:51.664+0000 7fddce800640 1 --2- 172.21.15.22:0/1223345597 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ae75e74460 0x55ae75e71490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:18:51.672 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:18:51.664+0000 7fddcde00640 1 -- 172.21.15.22:0/1223345597 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ae75e74460 msgr2=0x55ae75e71490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T18:18:51.673 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:18:51.664+0000 7fddcde00640 1 --2- 172.21.15.22:0/1223345597 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ae75e74460 0x55ae75e71490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:18:51.673 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:18:51.664+0000 7fddcde00640 1 -- 172.21.15.22:0/1223345597 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55ae75e6bdd0 msgr2=0x55ae75e70f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T18:18:51.673 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:18:51.664+0000 7fddcde00640 1 --2- 172.21.15.22:0/1223345597 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55ae75e6bdd0 0x55ae75e70f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:18:51.673 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:18:51.664+0000 7fddcde00640 1 -- 172.21.15.22:0/1223345597 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55ae75bad7d0 con 0x55ae75e6b0f0 2024-03-28T18:18:51.673 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:18:51.664+0000 7fddcde00640 1 --2- 172.21.15.22:0/1223345597 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ae75e6b0f0 0x55ae75e73bc0 secure :-1 s=READY pgs=804 cs=0 l=1 rev1=1 crypto rx=0x7fddb800bf80 tx=0x7fddb80128e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e31cc953615b6616 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:18:51.673 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:18:51.664+0000 7fddb3e00640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-03-28T18:18:51.673 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:18:51.664+0000 7fddb3e00640 1 -- 172.21.15.22:0/1223345597 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fddb801f060 con 0x55ae75e6b0f0 2024-03-28T18:18:51.673 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:18:51.664+0000 7fddb3e00640 1 -- 172.21.15.22:0/1223345597 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(122..122 src has 1..122) v4 ==== 10389+0+0 (secure 0 0 0) 0x7fddb8010900 con 0x55ae75e6b0f0 2024-03-28T18:18:51.673 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:18:51.665+0000 7fddd801f740 1 -- 172.21.15.22:0/1223345597 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fddc00be1a0 con 0x55ae75e6b0f0 2024-03-28T18:18:51.673 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:18:51.665+0000 7fddb3e00640 1 -- 172.21.15.22:0/1223345597 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=187) v2 ==== 24+0+0 (secure 0 0 0) 0x7fddb8026050 con 0x55ae75e6b0f0 2024-03-28T18:18:51.673 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:18:51.665+0000 7fddd801f740 10 client.16872 fetch_fsmap learned FSMap version 187 2024-03-28T18:18:51.674 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:18:51.665+0000 7fddd801f740 1 -- 172.21.15.22:0/1223345597 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=187,monmap=4+}) v3 -- 0x55ae75ae33f0 con 0x55ae75e6b0f0 2024-03-28T18:18:51.674 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:18:51.665+0000 7fddb3e00640 1 -- 172.21.15.22:0/1223345597 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7fddc00be1a0 con 0x55ae75e6b0f0 2024-03-28T18:18:51.674 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:18:51.665+0000 7fddb3e00640 1 -- 172.21.15.22:0/1223345597 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 187) v1 ==== 6221+0+0 (secure 0 0 0) 0x7fddb8019030 con 0x55ae75e6b0f0 2024-03-28T18:18:51.674 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:18:51.665+0000 7fddd801f740 10 client.16872 fetch_fsmap finished waiting for FSMap version 187 2024-03-28T18:18:51.674 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:18:51.665+0000 7fddd801f740 10 client.16872 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-28T18:18:51.674 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:18:51.665+0000 7fddd801f740 20 client.16872 populate_metadata read hostname 'smithi022' 2024-03-28T18:18:51.674 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:18:51.665+0000 7fddd801f740 1 --2- 172.21.15.22:0/1223345597 >> [v2:172.21.15.79:6838/2441542443,v1:172.21.15.79:6839/2441542443] conn(0x55ae75e84220 0x55ae75e86660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:18:51.674 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:18:51.665+0000 7fddd801f740 4 client.16872 mds_command: new command op to 34553 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-28T18:18:51.674 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:18:51.665+0000 7fddd801f740 1 -- 172.21.15.22:0/1223345597 --> [v2:172.21.15.79:6838/2441542443,v1:172.21.15.79:6839/2441542443] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55ae75ddc790 con 0x55ae75e84220 2024-03-28T18:18:51.674 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:18:51.666+0000 7fddcd400640 1 --2- 172.21.15.22:0/1223345597 >> [v2:172.21.15.79:6838/2441542443,v1:172.21.15.79:6839/2441542443] conn(0x55ae75e84220 0x55ae75e86660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:18:51.674 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:18:51.667+0000 7fddcd400640 1 --2- 172.21.15.22:0/1223345597 >> [v2:172.21.15.79:6838/2441542443,v1:172.21.15.79:6839/2441542443] conn(0x55ae75e84220 0x55ae75e86660 crc :-1 s=READY pgs=301 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=260b5fe6515d14ce server_cookie=2492e1553bc44a62 in_seq=0 out_seq=0 2024-03-28T18:18:51.675 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:18:51.667+0000 7fddb3e00640 10 client.16872 ms_handle_connect on v2:172.21.15.79:6838/2441542443 2024-03-28T18:18:51.676 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:18:51.668+0000 7fddb3e00640 1 -- 172.21.15.22:0/1223345597 <== mds.0 v2:172.21.15.79:6838/2441542443 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x55ae75ddc790 con 0x55ae75e84220 2024-03-28T18:18:51.676 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:18:51.668+0000 7fddb3e00640 10 client.16872 handle_command_reply: tid=0 2024-03-28T18:18:51.676 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:18:51.669+0000 7fddd801f740 1 client.16872 shutdown 2024-03-28T18:18:51.676 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:18:51.669+0000 7fddd801f740 1 -- 172.21.15.22:0/1223345597 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ae75e6b0f0 msgr2=0x55ae75e73bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:18:51.676 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:18:51.669+0000 7fddd801f740 1 --2- 172.21.15.22:0/1223345597 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ae75e6b0f0 0x55ae75e73bc0 secure :-1 s=READY pgs=804 cs=0 l=1 rev1=1 crypto rx=0x7fddb800bf80 tx=0x7fddb80128e0 comp rx=0 tx=0).stop 2024-03-28T18:18:51.676 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:18:51.669+0000 7fddd801f740 1 -- 172.21.15.22:0/1223345597 shutdown_connections 2024-03-28T18:18:51.677 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:18:51.669+0000 7fddd801f740 1 --2- 172.21.15.22:0/1223345597 >> [v2:172.21.15.79:6838/2441542443,v1:172.21.15.79:6839/2441542443] conn(0x55ae75e84220 0x55ae75e86660 crc :-1 s=READY pgs=301 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:18:51.677 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:18:51.669+0000 7fddd801f740 1 --2- 172.21.15.22:0/1223345597 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ae75e74460 0x55ae75e71490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:18:51.677 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:18:51.669+0000 7fddd801f740 1 --2- 172.21.15.22:0/1223345597 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55ae75e6bdd0 0x55ae75e70f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:18:51.677 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:18:51.669+0000 7fddd801f740 1 --2- 172.21.15.22:0/1223345597 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ae75e6b0f0 0x55ae75e73bc0 unknown :-1 s=CLOSED pgs=804 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:18:51.677 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:18:51.669+0000 7fddb3e00640 0 client.16872 ms_handle_reset on v2:172.21.15.79:6838/2441542443 2024-03-28T18:18:51.677 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:18:51.669+0000 7fddd801f740 1 -- 172.21.15.22:0/1223345597 >> 172.21.15.22:0/1223345597 conn(0x55ae75e32200 msgr2=0x55ae75e67890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:18:51.677 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:18:51.669+0000 7fddd801f740 1 -- 172.21.15.22:0/1223345597 shutdown_connections 2024-03-28T18:18:51.677 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:18:51.669+0000 7fddd801f740 1 -- 172.21.15.22:0/1223345597 wait complete. 2024-03-28T18:18:51.677 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:18:51.669+0000 7fddd801f740 20 client.16872 trim_cache size 0 max 16384 2024-03-28T18:18:51.693 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:18:51.687+0000 7fddd801f740 1 Processor -- start 2024-03-28T18:18:51.693 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:18:51.687+0000 7fddd801f740 1 -- start start 2024-03-28T18:18:51.693 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:18:51.687+0000 7fddd801f740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ae75e6b0f0 0x55ae75e73680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:18:51.693 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:18:51.687+0000 7fddd801f740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55ae75e6bdd0 0x55ae75e73bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:18:51.694 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:18:51.687+0000 7fddd801f740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ae75e74460 0x55ae75e7f7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:18:51.694 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:18:51.687+0000 7fddd801f740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fddd01b1c70 con 0x55ae75e6b0f0 2024-03-28T18:18:51.694 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:18:51.687+0000 7fddd801f740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fddd01aefd0 con 0x55ae75e6bdd0 2024-03-28T18:18:51.694 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:18:51.687+0000 7fddd801f740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fddd01b0fb0 con 0x55ae75e74460 2024-03-28T18:18:51.694 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:18:51.688+0000 7fddcde00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ae75e6b0f0 0x55ae75e73680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:18:51.694 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:18:51.688+0000 7fddcde00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ae75e6b0f0 0x55ae75e73680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:48468/0 (socket says 172.21.15.22:48468) 2024-03-28T18:18:51.694 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:18:51.688+0000 7fddcde00640 1 -- 172.21.15.22:0/952255112 learned_addr learned my addr 172.21.15.22:0/952255112 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-28T18:18:51.694 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:18:51.688+0000 7fddcd400640 1 --2- 172.21.15.22:0/952255112 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55ae75e6bdd0 0x55ae75e73bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:18:51.694 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:18:51.688+0000 7fddce800640 1 --2- 172.21.15.22:0/952255112 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ae75e74460 0x55ae75e7f7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:18:51.695 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:18:51.688+0000 7fddcde00640 1 -- 172.21.15.22:0/952255112 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ae75e74460 msgr2=0x55ae75e7f7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T18:18:51.695 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:18:51.688+0000 7fddcde00640 1 --2- 172.21.15.22:0/952255112 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ae75e74460 0x55ae75e7f7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:18:51.695 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:18:51.688+0000 7fddcde00640 1 -- 172.21.15.22:0/952255112 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55ae75e6bdd0 msgr2=0x55ae75e73bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T18:18:51.695 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:18:51.688+0000 7fddcde00640 1 --2- 172.21.15.22:0/952255112 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55ae75e6bdd0 0x55ae75e73bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:18:51.695 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:18:51.688+0000 7fddcde00640 1 -- 172.21.15.22:0/952255112 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ae75aa30f0 con 0x55ae75e6b0f0 2024-03-28T18:18:51.695 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:18:51.688+0000 7fddcd400640 1 --2- 172.21.15.22:0/952255112 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55ae75e6bdd0 0x55ae75e73bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-28T18:18:51.695 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:18:51.688+0000 7fddce800640 1 --2- 172.21.15.22:0/952255112 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ae75e74460 0x55ae75e7f7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T18:18:51.695 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:18:51.689+0000 7fddcde00640 1 --2- 172.21.15.22:0/952255112 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ae75e6b0f0 0x55ae75e73680 secure :-1 s=READY pgs=805 cs=0 l=1 rev1=1 crypto rx=0x7fddb800c010 tx=0x7fddb8005c50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c9c01fb2a1c4e088 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:18:51.695 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:18:51.689+0000 7fddbea00640 1 -- 172.21.15.22:0/952255112 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fddb802e070 con 0x55ae75e6b0f0 2024-03-28T18:18:51.695 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:18:51.689+0000 7fddbea00640 1 -- 172.21.15.22:0/952255112 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7fddb8019dc0 con 0x55ae75e6b0f0 2024-03-28T18:18:51.695 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:18:51.689+0000 7fddbea00640 1 -- 172.21.15.22:0/952255112 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fddb8010e90 con 0x55ae75e6b0f0 2024-03-28T18:18:51.696 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:18:51.690+0000 7fddd801f740 1 -- 172.21.15.22:0/952255112 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ae75e6b0f0 msgr2=0x55ae75e73680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:18:51.696 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:18:51.690+0000 7fddd801f740 1 --2- 172.21.15.22:0/952255112 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ae75e6b0f0 0x55ae75e73680 secure :-1 s=READY pgs=805 cs=0 l=1 rev1=1 crypto rx=0x7fddb800c010 tx=0x7fddb8005c50 comp rx=0 tx=0).stop 2024-03-28T18:18:51.696 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:18:51.690+0000 7fddd801f740 1 -- 172.21.15.22:0/952255112 shutdown_connections 2024-03-28T18:18:51.696 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:18:51.690+0000 7fddd801f740 1 --2- 172.21.15.22:0/952255112 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ae75e74460 0x55ae75e7f7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:18:51.696 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:18:51.690+0000 7fddd801f740 1 --2- 172.21.15.22:0/952255112 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55ae75e6bdd0 0x55ae75e73bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:18:51.696 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:18:51.690+0000 7fddd801f740 1 --2- 172.21.15.22:0/952255112 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ae75e6b0f0 0x55ae75e73680 unknown :-1 s=CLOSED pgs=805 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:18:51.696 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:18:51.690+0000 7fddd801f740 1 -- 172.21.15.22:0/952255112 >> 172.21.15.22:0/952255112 conn(0x55ae75e32200 msgr2=0x55ae75e65670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:18:51.696 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:18:51.690+0000 7fddd801f740 1 -- 172.21.15.22:0/952255112 shutdown_connections 2024-03-28T18:18:51.696 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:18:51.690+0000 7fddd801f740 1 -- 172.21.15.22:0/952255112 wait complete. 2024-03-28T18:18:51.696 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:18:51.690+0000 7fddd801f740 1 -- start start 2024-03-28T18:18:51.697 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:18:51.690+0000 7fddd801f740 1 Processor -- start 2024-03-28T18:18:51.697 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:18:51.691+0000 7fddd801f740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ae75e6b0f0 0x55ae75e73ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:18:51.697 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:18:51.691+0000 7fddd801f740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55ae75e6bdd0 0x55ae75e6e1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:18:51.697 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:18:51.691+0000 7fddd801f740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ae75e74460 0x55ae75e6e730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:18:51.697 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:18:51.691+0000 7fddd801f740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fddd01b0950 con 0x55ae75e6b0f0 2024-03-28T18:18:51.697 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:18:51.691+0000 7fddd801f740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fddd01af630 con 0x55ae75e6bdd0 2024-03-28T18:18:51.697 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:18:51.691+0000 7fddd801f740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fddd01afc90 con 0x55ae75e74460 2024-03-28T18:18:51.697 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:18:51.691+0000 7fddd801f740 10 client.0 _ll_register_callbacks cb 0x7fddb800b7b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-28T18:18:51.697 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:18:51.691+0000 7fddcde00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ae75e6b0f0 0x55ae75e73ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:18:51.697 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:18:51.691+0000 7fddcde00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ae75e6b0f0 0x55ae75e73ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:48472/0 (socket says 172.21.15.22:48472) 2024-03-28T18:18:51.697 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:18:51.691+0000 7fddcde00640 1 -- 172.21.15.22:0/436834268 learned_addr learned my addr 172.21.15.22:0/436834268 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-28T18:18:51.698 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:18:51.691+0000 7fddcd400640 1 --2- 172.21.15.22:0/436834268 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55ae75e6bdd0 0x55ae75e6e1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:18:51.698 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:18:51.691+0000 7fddce800640 1 --2- 172.21.15.22:0/436834268 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ae75e74460 0x55ae75e6e730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:18:51.698 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:18:51.691+0000 7fddcde00640 1 -- 172.21.15.22:0/436834268 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ae75e74460 msgr2=0x55ae75e6e730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T18:18:51.698 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:18:51.692+0000 7fddcde00640 1 --2- 172.21.15.22:0/436834268 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ae75e74460 0x55ae75e6e730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:18:51.698 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:18:51.692+0000 7fddcde00640 1 -- 172.21.15.22:0/436834268 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55ae75e6bdd0 msgr2=0x55ae75e6e1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T18:18:51.698 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:18:51.692+0000 7fddcde00640 1 --2- 172.21.15.22:0/436834268 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55ae75e6bdd0 0x55ae75e6e1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:18:51.698 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:18:51.692+0000 7fddcde00640 1 -- 172.21.15.22:0/436834268 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55ae75e6ef60 con 0x55ae75e6b0f0 2024-03-28T18:18:51.698 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:18:51.692+0000 7fddcd400640 1 --2- 172.21.15.22:0/436834268 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55ae75e6bdd0 0x55ae75e6e1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-28T18:18:51.698 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:18:51.692+0000 7fddce800640 1 --2- 172.21.15.22:0/436834268 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ae75e74460 0x55ae75e6e730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T18:18:51.698 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:18:51.692+0000 7fddcde00640 1 --2- 172.21.15.22:0/436834268 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ae75e6b0f0 0x55ae75e73ba0 secure :-1 s=READY pgs=806 cs=0 l=1 rev1=1 crypto rx=0x7fddb8010e40 tx=0x7fddb8010e70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3594df22ab382361 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:18:51.698 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:18:51.692+0000 7fddb3e00640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-03-28T18:18:51.699 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:18:51.692+0000 7fddb3e00640 1 -- 172.21.15.22:0/436834268 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fddb802e040 con 0x55ae75e6b0f0 2024-03-28T18:18:51.699 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:18:51.692+0000 7fddd801f740 1 -- 172.21.15.22:0/436834268 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fddb8019dc0 con 0x55ae75e6b0f0 2024-03-28T18:18:51.699 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:18:51.692+0000 7fddb3e00640 1 -- 172.21.15.22:0/436834268 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(122..122 src has 1..122) v4 ==== 10389+0+0 (secure 0 0 0) 0x7fddb80194a0 con 0x55ae75e6b0f0 2024-03-28T18:18:51.699 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:18:51.693+0000 7fddb3e00640 1 -- 172.21.15.22:0/436834268 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=187) v2 ==== 24+0+0 (secure 0 0 0) 0x7fddb8045050 con 0x55ae75e6b0f0 2024-03-28T18:18:51.699 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:18:51.693+0000 7fddd801f740 10 client.16878 fetch_fsmap learned FSMap version 187 2024-03-28T18:18:51.699 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:18:51.693+0000 7fddd801f740 1 -- 172.21.15.22:0/436834268 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=187,monmap=4+}) v3 -- 0x55ae75edfb50 con 0x55ae75e6b0f0 2024-03-28T18:18:51.699 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:18:51.693+0000 7fddb3e00640 1 -- 172.21.15.22:0/436834268 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7fddb8019dc0 con 0x55ae75e6b0f0 2024-03-28T18:18:51.699 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:18:51.693+0000 7fddb3e00640 1 -- 172.21.15.22:0/436834268 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 187) v1 ==== 6221+0+0 (secure 0 0 0) 0x7fddb803c070 con 0x55ae75e6b0f0 2024-03-28T18:18:51.700 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:18:51.693+0000 7fddd801f740 10 client.16878 fetch_fsmap finished waiting for FSMap version 187 2024-03-28T18:18:51.700 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:18:51.693+0000 7fddd801f740 10 client.16878 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-28T18:18:51.700 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:18:51.693+0000 7fddd801f740 20 client.16878 populate_metadata read hostname 'smithi022' 2024-03-28T18:18:51.700 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:18:51.693+0000 7fddd801f740 1 --2- 172.21.15.22:0/436834268 >> [v2:172.21.15.79:6838/2441542443,v1:172.21.15.79:6839/2441542443] conn(0x55ae75ee00b0 0x55ae75ee24a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:18:51.700 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:18:51.693+0000 7fddd801f740 4 client.16878 mds_command: new command op to 34553 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-28T18:18:51.700 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:18:51.693+0000 7fddd801f740 1 -- 172.21.15.22:0/436834268 --> [v2:172.21.15.79:6838/2441542443,v1:172.21.15.79:6839/2441542443] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55ae75dd6ae0 con 0x55ae75ee00b0 2024-03-28T18:18:51.700 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:18:51.694+0000 7fddcd400640 1 --2- 172.21.15.22:0/436834268 >> [v2:172.21.15.79:6838/2441542443,v1:172.21.15.79:6839/2441542443] conn(0x55ae75ee00b0 0x55ae75ee24a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:18:51.701 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:18:51.694+0000 7fddcd400640 1 --2- 172.21.15.22:0/436834268 >> [v2:172.21.15.79:6838/2441542443,v1:172.21.15.79:6839/2441542443] conn(0x55ae75ee00b0 0x55ae75ee24a0 crc :-1 s=READY pgs=302 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2ef39e5e3c7694a1 server_cookie=1e9d240a42de2b97 in_seq=0 out_seq=0 2024-03-28T18:18:51.702 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:18:51.694+0000 7fddb3e00640 10 client.16878 ms_handle_connect on v2:172.21.15.79:6838/2441542443 2024-03-28T18:18:51.702 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:18:51.695+0000 7fddb3e00640 1 -- 172.21.15.22:0/436834268 <== mds.0 v2:172.21.15.79:6838/2441542443 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x55ae75dd6ae0 con 0x55ae75ee00b0 2024-03-28T18:18:51.702 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:18:51.695+0000 7fddb3e00640 10 client.16878 handle_command_reply: tid=0 2024-03-28T18:18:51.702 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:18:51.695+0000 7fddd801f740 1 client.16878 shutdown 2024-03-28T18:18:51.702 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:18:51.695+0000 7fddd801f740 1 -- 172.21.15.22:0/436834268 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ae75e6b0f0 msgr2=0x55ae75e73ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:18:51.702 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:18:51.695+0000 7fddd801f740 1 --2- 172.21.15.22:0/436834268 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ae75e6b0f0 0x55ae75e73ba0 secure :-1 s=READY pgs=806 cs=0 l=1 rev1=1 crypto rx=0x7fddb8010e40 tx=0x7fddb8010e70 comp rx=0 tx=0).stop 2024-03-28T18:18:51.702 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:18:51.695+0000 7fddd801f740 1 -- 172.21.15.22:0/436834268 shutdown_connections 2024-03-28T18:18:51.702 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:18:51.695+0000 7fddd801f740 1 --2- 172.21.15.22:0/436834268 >> [v2:172.21.15.79:6838/2441542443,v1:172.21.15.79:6839/2441542443] conn(0x55ae75ee00b0 0x55ae75ee24a0 crc :-1 s=READY pgs=302 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:18:51.702 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:18:51.695+0000 7fddd801f740 1 --2- 172.21.15.22:0/436834268 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ae75e74460 0x55ae75e6e730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:18:51.702 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:18:51.695+0000 7fddd801f740 1 --2- 172.21.15.22:0/436834268 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55ae75e6bdd0 0x55ae75e6e1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:18:51.702 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:18:51.695+0000 7fddd801f740 1 --2- 172.21.15.22:0/436834268 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ae75e6b0f0 0x55ae75e73ba0 unknown :-1 s=CLOSED pgs=806 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:18:51.703 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:18:51.695+0000 7fddb3e00640 0 client.16878 ms_handle_reset on v2:172.21.15.79:6838/2441542443 2024-03-28T18:18:51.703 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:18:51.695+0000 7fddd801f740 1 -- 172.21.15.22:0/436834268 >> 172.21.15.22:0/436834268 conn(0x55ae75e32200 msgr2=0x55ae75e6f3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:18:51.703 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:18:51.695+0000 7fddd801f740 1 -- 172.21.15.22:0/436834268 shutdown_connections 2024-03-28T18:18:51.703 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:18:51.695+0000 7fddd801f740 1 -- 172.21.15.22:0/436834268 wait complete. 2024-03-28T18:18:51.703 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:18:51.696+0000 7fddd801f740 20 client.16878 trim_cache size 0 max 16384 2024-03-28T18:18:51.703 INFO:teuthology.orchestra.run.smithi022.stdout:{ 2024-03-28T18:18:51.703 INFO:teuthology.orchestra.run.smithi022.stdout: "status": "IDLE (waiting for more scrubs)", 2024-03-28T18:18:51.703 INFO:teuthology.orchestra.run.smithi022.stdout: "scrubs": { 2024-03-28T18:18:51.704 INFO:teuthology.orchestra.run.smithi022.stdout: "4bbbd61d-29a3-4bf6-a4f2-d5783d585de3": { 2024-03-28T18:18:51.704 INFO:teuthology.orchestra.run.smithi022.stdout: "path": "/", 2024-03-28T18:18:51.704 INFO:teuthology.orchestra.run.smithi022.stdout: "tag": "4bbbd61d-29a3-4bf6-a4f2-d5783d585de3", 2024-03-28T18:18:51.704 INFO:teuthology.orchestra.run.smithi022.stdout: "options": "recursive,force" 2024-03-28T18:18:51.704 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-03-28T18:18:51.704 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-03-28T18:18:51.704 INFO:teuthology.orchestra.run.smithi022.stdout:} 2024-03-28T18:18:51.705 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:18:51.696+0000 7fddcf200640 1 -- 172.21.15.22:0/911355541 >> [v2:172.21.15.22:6800/1357365082,v1:172.21.15.22:6801/1357365082] conn(0x7fddb40a2970 msgr2=0x7fddb40a4e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:18:51.705 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:18:51.696+0000 7fddcf200640 1 --2- 172.21.15.22:0/911355541 >> [v2:172.21.15.22:6800/1357365082,v1:172.21.15.22:6801/1357365082] conn(0x7fddb40a2970 0x7fddb40a4e30 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fddc8009c70 tx=0x7fddc8005f90 comp rx=0 tx=0).stop 2024-03-28T18:18:51.705 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:18:51.696+0000 7fddcf200640 1 -- 172.21.15.22:0/911355541 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fddd0102ff0 msgr2=0x7fddd0100860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:18:51.705 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:18:51.696+0000 7fddcf200640 1 --2- 172.21.15.22:0/911355541 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fddd0102ff0 0x7fddd0100860 secure :-1 s=READY pgs=802 cs=0 l=1 rev1=1 crypto rx=0x7fddc002c790 tx=0x7fddc0034d60 comp rx=0 tx=0).stop 2024-03-28T18:18:51.705 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:18:51.696+0000 7fddcf200640 1 -- 172.21.15.22:0/911355541 shutdown_connections 2024-03-28T18:18:51.705 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:18:51.696+0000 7fddcf200640 1 --2- 172.21.15.22:0/911355541 >> [v2:172.21.15.22:6800/1357365082,v1:172.21.15.22:6801/1357365082] conn(0x7fddb40a2970 0x7fddb40a4e30 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:18:51.705 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:18:51.696+0000 7fddcf200640 1 --2- 172.21.15.22:0/911355541 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fddd010baf0 0x7fddd0100da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:18:51.705 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:18:51.696+0000 7fddcf200640 1 --2- 172.21.15.22:0/911355541 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fddd0102ff0 0x7fddd0100860 unknown :-1 s=CLOSED pgs=802 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:18:51.706 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:18:51.696+0000 7fddcf200640 1 --2- 172.21.15.22:0/911355541 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fddd01026d0 0x7fddd0100320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:18:51.706 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:18:51.696+0000 7fddcf200640 1 -- 172.21.15.22:0/911355541 >> 172.21.15.22:0/911355541 conn(0x7fddd00fbef0 msgr2=0x7fddd00fcc00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:18:51.706 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:18:51.696+0000 7fddcf200640 1 -- 172.21.15.22:0/911355541 shutdown_connections 2024-03-28T18:18:51.706 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:18:51.697+0000 7fddcf200640 1 -- 172.21.15.22:0/911355541 wait complete. 2024-03-28T18:18:51.715 INFO:tasks.cephfs.filesystem:scrub status for tag:4bbbd61d-29a3-4bf6-a4f2-d5783d585de3 - {'path': '/', 'tag': '4bbbd61d-29a3-4bf6-a4f2-d5783d585de3', 'options': 'recursive,force'} 2024-03-28T18:18:53.175 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:18:52 smithi134 ceph-mon[35037]: pgmap v372: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:18:53.182 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:18:52 smithi079 ceph-mon[35312]: pgmap v372: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:18:53.281 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:18:52 smithi022 ceph-mon[31985]: pgmap v372: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:18:55.175 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:18:54 smithi134 ceph-mon[35037]: pgmap v373: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:18:55.182 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:18:54 smithi079 ceph-mon[35312]: pgmap v373: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:18:55.282 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:18:54 smithi022 ceph-mon[31985]: pgmap v373: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:18:57.175 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:18:56 smithi134 ceph-mon[35037]: pgmap v374: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 9 op/s 2024-03-28T18:18:57.182 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:18:56 smithi079 ceph-mon[35312]: pgmap v374: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 9 op/s 2024-03-28T18:18:57.282 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:18:56 smithi022 ceph-mon[31985]: pgmap v374: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 9 op/s 2024-03-28T18:18:59.175 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:18:58 smithi134 ceph-mon[35037]: pgmap v375: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T18:18:59.182 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:18:58 smithi079 ceph-mon[35312]: pgmap v375: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T18:18:59.282 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:18:58 smithi022 ceph-mon[31985]: pgmap v375: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T18:19:01.175 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:19:00 smithi134 ceph-mon[35037]: pgmap v376: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 9 op/s 2024-03-28T18:19:01.182 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:19:00 smithi079 ceph-mon[35312]: pgmap v376: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 9 op/s 2024-03-28T18:19:01.282 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:19:00 smithi022 ceph-mon[31985]: pgmap v376: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 9 op/s 2024-03-28T18:19:03.175 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:19:02 smithi134 ceph-mon[35037]: pgmap v377: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T18:19:03.182 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:19:02 smithi079 ceph-mon[35312]: pgmap v377: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T18:19:03.282 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:19:02 smithi022 ceph-mon[31985]: pgmap v377: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T18:19:05.175 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:19:04 smithi134 ceph-mon[35037]: pgmap v378: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T18:19:05.182 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:19:04 smithi079 ceph-mon[35312]: pgmap v378: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T18:19:05.282 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:19:04 smithi022 ceph-mon[31985]: pgmap v378: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T18:19:07.183 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:19:06 smithi079 ceph-mon[35312]: pgmap v379: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 9 op/s 2024-03-28T18:19:07.183 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:19:06 smithi079 ceph-mon[35312]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T18:19:07.282 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:19:06 smithi022 ceph-mon[31985]: pgmap v379: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 9 op/s 2024-03-28T18:19:07.282 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:19:06 smithi022 ceph-mon[31985]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T18:19:07.424 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:19:06 smithi134 ceph-mon[35037]: pgmap v379: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 9 op/s 2024-03-28T18:19:07.425 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:19:06 smithi134 ceph-mon[35037]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T18:19:08.282 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:19:07 smithi022 ceph-mon[31985]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T18:19:08.282 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:19:07 smithi022 ceph-mon[31985]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T18:19:08.282 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:19:07 smithi022 ceph-mon[31985]: from='mgr.15522 ' entity='mgr.x' 2024-03-28T18:19:08.425 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:19:07 smithi134 ceph-mon[35037]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T18:19:08.425 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:19:07 smithi134 ceph-mon[35037]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T18:19:08.425 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:19:07 smithi134 ceph-mon[35037]: from='mgr.15522 ' entity='mgr.x' 2024-03-28T18:19:08.433 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:19:07 smithi079 ceph-mon[35312]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T18:19:08.433 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:19:07 smithi079 ceph-mon[35312]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T18:19:08.433 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:19:07 smithi079 ceph-mon[35312]: from='mgr.15522 ' entity='mgr.x' 2024-03-28T18:19:09.282 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:19:08 smithi022 ceph-mon[31985]: pgmap v380: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 6 op/s 2024-03-28T18:19:09.282 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:19:08 smithi022 ceph-mon[31985]: pgmap v381: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-03-28T18:19:09.425 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:19:08 smithi134 ceph-mon[35037]: pgmap v380: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 6 op/s 2024-03-28T18:19:09.425 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:19:08 smithi134 ceph-mon[35037]: pgmap v381: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-03-28T18:19:09.433 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:19:08 smithi079 ceph-mon[35312]: pgmap v380: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 6 op/s 2024-03-28T18:19:09.433 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:19:08 smithi079 ceph-mon[35312]: pgmap v381: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-03-28T18:19:11.283 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:19:10 smithi022 ceph-mon[31985]: pgmap v382: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4 KiB/s rd, 8 op/s 2024-03-28T18:19:11.424 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:19:10 smithi134 ceph-mon[35037]: pgmap v382: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4 KiB/s rd, 8 op/s 2024-03-28T18:19:11.433 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:19:10 smithi079 ceph-mon[35312]: pgmap v382: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4 KiB/s rd, 8 op/s 2024-03-28T18:19:13.236 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:19:12 smithi079 ceph-mon[35312]: pgmap v383: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4 KiB/s rd, 8 op/s 2024-03-28T18:19:13.282 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:19:12 smithi022 ceph-mon[31985]: pgmap v383: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4 KiB/s rd, 8 op/s 2024-03-28T18:19:13.425 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:19:12 smithi134 ceph-mon[35037]: pgmap v383: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4 KiB/s rd, 8 op/s 2024-03-28T18:19:15.282 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:19:14 smithi022 ceph-mon[31985]: pgmap v384: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4 KiB/s rd, 8 op/s 2024-03-28T18:19:15.424 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:19:14 smithi134 ceph-mon[35037]: pgmap v384: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4 KiB/s rd, 8 op/s 2024-03-28T18:19:15.432 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:19:14 smithi079 ceph-mon[35312]: pgmap v384: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4 KiB/s rd, 8 op/s 2024-03-28T18:19:17.282 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:19:16 smithi022 ceph-mon[31985]: pgmap v385: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4 KiB/s rd, 8 op/s 2024-03-28T18:19:17.425 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:19:16 smithi134 ceph-mon[35037]: pgmap v385: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4 KiB/s rd, 8 op/s 2024-03-28T18:19:17.432 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:19:16 smithi079 ceph-mon[35312]: pgmap v385: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4 KiB/s rd, 8 op/s 2024-03-28T18:19:19.282 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:19:18 smithi022 ceph-mon[31985]: pgmap v386: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 7 op/s 2024-03-28T18:19:19.424 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:19:18 smithi134 ceph-mon[35037]: pgmap v386: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 7 op/s 2024-03-28T18:19:19.432 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:19:18 smithi079 ceph-mon[35312]: pgmap v386: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 7 op/s 2024-03-28T18:19:21.282 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:19:20 smithi022 ceph-mon[31985]: pgmap v387: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 9 op/s 2024-03-28T18:19:21.425 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:19:20 smithi134 ceph-mon[35037]: pgmap v387: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 9 op/s 2024-03-28T18:19:21.432 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:19:20 smithi079 ceph-mon[35312]: pgmap v387: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 9 op/s 2024-03-28T18:19:21.716 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-28T18:19:21.717 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-28T18:19:21.964 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:19:21.963+0000 7f03cda00640 1 -- 172.21.15.22:0/1260339257 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f03c810cea0 msgr2=0x7f03c8114a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:19:21.964 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:19:21.963+0000 7f03cda00640 1 --2- 172.21.15.22:0/1260339257 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f03c810cea0 0x7f03c8114a70 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f03c0009be0 tx=0x7f03c002bef0 comp rx=0 tx=0).stop 2024-03-28T18:19:21.964 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:19:21.964+0000 7f03cda00640 1 -- 172.21.15.22:0/1260339257 shutdown_connections 2024-03-28T18:19:21.964 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:19:21.964+0000 7f03cda00640 1 --2- 172.21.15.22:0/1260339257 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f03c810cea0 0x7f03c8114a70 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:19:21.964 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:19:21.964+0000 7f03cda00640 1 --2- 172.21.15.22:0/1260339257 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f03c8108800 0x7f03c810c7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:19:21.964 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:19:21.964+0000 7f03cda00640 1 --2- 172.21.15.22:0/1260339257 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f03c8107e50 0x7f03c8108230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:19:21.965 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:19:21.964+0000 7f03cda00640 1 -- 172.21.15.22:0/1260339257 >> 172.21.15.22:0/1260339257 conn(0x7f03c80774c0 msgr2=0x7f03c81064c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:19:21.965 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:19:21.964+0000 7f03cda00640 1 -- 172.21.15.22:0/1260339257 shutdown_connections 2024-03-28T18:19:21.965 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:19:21.964+0000 7f03cda00640 1 -- 172.21.15.22:0/1260339257 wait complete. 2024-03-28T18:19:21.967 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:19:21.965+0000 7f03cda00640 1 Processor -- start 2024-03-28T18:19:21.967 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:19:21.965+0000 7f03cda00640 1 -- start start 2024-03-28T18:19:21.968 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:19:21.965+0000 7f03cda00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f03c8107e50 0x7f03c819e3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:19:21.968 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:19:21.965+0000 7f03cda00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f03c810cea0 0x7f03c819e930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:19:21.968 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:19:21.965+0000 7f03cda00640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f03c81a2cb0 0x7f03c81a3160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:19:21.968 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:19:21.965+0000 7f03cda00640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03c8117fb0 con 0x7f03c810cea0 2024-03-28T18:19:21.968 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:19:21.965+0000 7f03cda00640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03c8117e30 con 0x7f03c81a2cb0 2024-03-28T18:19:21.968 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:19:21.965+0000 7f03cda00640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03c8118130 con 0x7f03c8107e50 2024-03-28T18:19:21.968 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:19:21.965+0000 7f03bfe00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f03c810cea0 0x7f03c819e930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:19:21.968 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:19:21.966+0000 7f03bfe00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f03c810cea0 0x7f03c819e930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:35638/0 (socket says 172.21.15.22:35638) 2024-03-28T18:19:21.969 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:19:21.966+0000 7f03bfe00640 1 -- 172.21.15.22:0/1823907570 learned_addr learned my addr 172.21.15.22:0/1823907570 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-28T18:19:21.969 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:19:21.966+0000 7f03c7e00640 1 --2- 172.21.15.22:0/1823907570 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f03c8107e50 0x7f03c819e3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:19:21.969 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:19:21.966+0000 7f03cd000640 1 --2- 172.21.15.22:0/1823907570 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f03c81a2cb0 0x7f03c81a3160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:19:21.969 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:19:21.966+0000 7f03bfe00640 1 -- 172.21.15.22:0/1823907570 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f03c8107e50 msgr2=0x7f03c819e3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:19:21.969 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:19:21.966+0000 7f03bfe00640 1 --2- 172.21.15.22:0/1823907570 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f03c8107e50 0x7f03c819e3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:19:21.969 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:19:21.966+0000 7f03bfe00640 1 -- 172.21.15.22:0/1823907570 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f03c81a2cb0 msgr2=0x7f03c81a3160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:19:21.969 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:19:21.966+0000 7f03bfe00640 1 --2- 172.21.15.22:0/1823907570 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f03c81a2cb0 0x7f03c81a3160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:19:21.969 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:19:21.966+0000 7f03bfe00640 1 -- 172.21.15.22:0/1823907570 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f03c81a37a0 con 0x7f03c810cea0 2024-03-28T18:19:21.969 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:19:21.966+0000 7f03bfe00640 1 --2- 172.21.15.22:0/1823907570 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f03c810cea0 0x7f03c819e930 secure :-1 s=READY pgs=807 cs=0 l=1 rev1=1 crypto rx=0x7f03b000e9c0 tx=0x7f03b000ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:19:21.969 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:19:21.967+0000 7f03c5600640 1 -- 172.21.15.22:0/1823907570 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03b000ccc0 con 0x7f03c810cea0 2024-03-28T18:19:21.969 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:19:21.967+0000 7f03cda00640 1 -- 172.21.15.22:0/1823907570 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f03c81a3a90 con 0x7f03c810cea0 2024-03-28T18:19:21.969 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:19:21.967+0000 7f03c5600640 1 -- 172.21.15.22:0/1823907570 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f03b00054f0 con 0x7f03c810cea0 2024-03-28T18:19:21.969 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:19:21.967+0000 7f03c5600640 1 -- 172.21.15.22:0/1823907570 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03b0018630 con 0x7f03c810cea0 2024-03-28T18:19:21.970 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:19:21.967+0000 7f03cda00640 1 -- 172.21.15.22:0/1823907570 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f03c81ab220 con 0x7f03c810cea0 2024-03-28T18:19:21.972 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:19:21.971+0000 7f03d1c49740 1 Processor -- start 2024-03-28T18:19:21.972 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:19:21.971+0000 7f03d1c49740 1 -- start start 2024-03-28T18:19:21.972 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:19:21.972+0000 7f03c5600640 1 -- 172.21.15.22:0/1823907570 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 26) v1 ==== 137421+0+0 (secure 0 0 0) 0x7f03b0004e20 con 0x7f03c810cea0 2024-03-28T18:19:21.973 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:19:21.972+0000 7f03d1c49740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x555e9fd14dd0 0x555e9fd13b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:19:21.973 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:19:21.972+0000 7f03d1c49740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555e9fd140f0 0x555e9fd14570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:19:21.973 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:19:21.972+0000 7f03d1c49740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x555e9fd1d460 0x555e9fd1f850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:19:21.973 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:19:21.972+0000 7f03d1c49740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x555e9fc78250 con 0x555e9fd1d460 2024-03-28T18:19:21.973 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:19:21.972+0000 7f03d1c49740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x555e9fc20a30 con 0x555e9fd14dd0 2024-03-28T18:19:21.973 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:19:21.972+0000 7f03d1c49740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x555e9fc742f0 con 0x555e9fd140f0 2024-03-28T18:19:21.973 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:19:21.972+0000 7f03bfe00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x555e9fd1d460 0x555e9fd1f850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:19:21.973 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:19:21.972+0000 7f03cd000640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x555e9fd14dd0 0x555e9fd13b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:19:21.973 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:19:21.972+0000 7f03bfe00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x555e9fd1d460 0x555e9fd1f850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:35640/0 (socket says 172.21.15.22:35640) 2024-03-28T18:19:21.974 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:19:21.972+0000 7f03bfe00640 1 -- 172.21.15.22:0/2289617463 learned_addr learned my addr 172.21.15.22:0/2289617463 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-28T18:19:21.974 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:19:21.973+0000 7f03c7e00640 1 --2- 172.21.15.22:0/2289617463 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555e9fd140f0 0x555e9fd14570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:19:21.974 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:19:21.973+0000 7f03cd000640 1 -- 172.21.15.22:0/2289617463 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555e9fd140f0 msgr2=0x555e9fd14570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T18:19:21.974 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:19:21.973+0000 7f03cd000640 1 --2- 172.21.15.22:0/2289617463 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555e9fd140f0 0x555e9fd14570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:19:21.974 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:19:21.973+0000 7f03cd000640 1 -- 172.21.15.22:0/2289617463 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x555e9fd1d460 msgr2=0x555e9fd1f850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T18:19:21.974 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:19:21.973+0000 7f03cd000640 1 --2- 172.21.15.22:0/2289617463 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x555e9fd1d460 0x555e9fd1f850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:19:21.974 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:19:21.973+0000 7f03cd000640 1 -- 172.21.15.22:0/2289617463 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555e9fc0b690 con 0x555e9fd14dd0 2024-03-28T18:19:21.978 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:19:21.973+0000 7f03cd000640 1 --2- 172.21.15.22:0/2289617463 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x555e9fd14dd0 0x555e9fd13b20 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f03c0036040 tx=0x7f03c0009910 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ee19a1cbf137ef56 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:19:21.978 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:19:21.973+0000 7f03bd600640 1 -- 172.21.15.22:0/2289617463 <== mon.2 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03c0043030 con 0x555e9fd14dd0 2024-03-28T18:19:21.978 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:19:21.974+0000 7f03c5600640 1 --2- 172.21.15.22:0/1823907570 >> [v2:172.21.15.22:6800/1357365082,v1:172.21.15.22:6801/1357365082] conn(0x7f03ac0a2970 0x7f03ac0a4e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:19:21.978 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:19:21.974+0000 7f03bd600640 1 -- 172.21.15.22:0/2289617463 <== mon.2 v2:172.21.15.79:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f03c0033be0 con 0x555e9fd14dd0 2024-03-28T18:19:21.978 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:19:21.974+0000 7f03bd600640 1 -- 172.21.15.22:0/2289617463 <== mon.2 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03c002c930 con 0x555e9fd14dd0 2024-03-28T18:19:21.978 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:19:21.974+0000 7f03c5600640 1 -- 172.21.15.22:0/1823907570 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(122..122 src has 1..122) v4 ==== 10389+0+0 (secure 0 0 0) 0x7f03b00cfaa0 con 0x7f03c810cea0 2024-03-28T18:19:21.978 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:19:21.974+0000 7f03c7e00640 1 --2- 172.21.15.22:0/1823907570 >> [v2:172.21.15.22:6800/1357365082,v1:172.21.15.22:6801/1357365082] conn(0x7f03ac0a2970 0x7f03ac0a4e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:19:21.978 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:19:21.974+0000 7f03d1c49740 1 -- 172.21.15.22:0/2289617463 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x555e9fd14dd0 msgr2=0x555e9fd13b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:19:21.979 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:19:21.974+0000 7f03d1c49740 1 --2- 172.21.15.22:0/2289617463 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x555e9fd14dd0 0x555e9fd13b20 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f03c0036040 tx=0x7f03c0009910 comp rx=0 tx=0).stop 2024-03-28T18:19:21.979 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:19:21.974+0000 7f03d1c49740 1 -- 172.21.15.22:0/2289617463 shutdown_connections 2024-03-28T18:19:21.979 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:19:21.974+0000 7f03d1c49740 1 --2- 172.21.15.22:0/2289617463 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x555e9fd1d460 0x555e9fd1f850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:19:21.979 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:19:21.974+0000 7f03d1c49740 1 --2- 172.21.15.22:0/2289617463 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555e9fd140f0 0x555e9fd14570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:19:21.979 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:19:21.974+0000 7f03d1c49740 1 --2- 172.21.15.22:0/2289617463 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x555e9fd14dd0 0x555e9fd13b20 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:19:21.979 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:19:21.974+0000 7f03d1c49740 1 -- 172.21.15.22:0/2289617463 >> 172.21.15.22:0/2289617463 conn(0x555e9fcdb200 msgr2=0x555e9fd0eb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:19:21.979 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:19:21.975+0000 7f03d1c49740 1 -- 172.21.15.22:0/2289617463 shutdown_connections 2024-03-28T18:19:21.979 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:19:21.975+0000 7f03d1c49740 1 -- 172.21.15.22:0/2289617463 wait complete. 2024-03-28T18:19:21.979 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:19:21.975+0000 7f03c7e00640 1 --2- 172.21.15.22:0/1823907570 >> [v2:172.21.15.22:6800/1357365082,v1:172.21.15.22:6801/1357365082] conn(0x7f03ac0a2970 0x7f03ac0a4e30 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x555e9fd15320 tx=0x7f03b8009260 comp rx=0 tx=0).ready entity=mgr.15522 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:19:21.979 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:19:21.975+0000 7f03d1c49740 1 -- start start 2024-03-28T18:19:21.979 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:19:21.975+0000 7f03d1c49740 1 Processor -- start 2024-03-28T18:19:21.979 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:19:21.976+0000 7f03d1c49740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x555e9fd140f0 0x555e9fd1cbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:19:21.980 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:19:21.976+0000 7f03d1c49740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555e9fd14dd0 0x555e9fd19f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:19:21.980 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:19:21.976+0000 7f03d1c49740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x555e9fd1d460 0x555e9fd1a480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:19:21.980 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:19:21.976+0000 7f03d1c49740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x555e9fd20270 con 0x555e9fd140f0 2024-03-28T18:19:21.980 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:19:21.976+0000 7f03d1c49740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x555e9fc33e80 con 0x555e9fd1d460 2024-03-28T18:19:21.980 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:19:21.976+0000 7f03d1c49740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x555e9fd203f0 con 0x555e9fd14dd0 2024-03-28T18:19:21.980 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:19:21.976+0000 7f03d1c49740 10 client.0 _ll_register_callbacks cb 0x555e9fc46130 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-28T18:19:21.980 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:19:21.976+0000 7f03c7e00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x555e9fd140f0 0x555e9fd1cbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:19:21.980 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:19:21.976+0000 7f03c7e00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x555e9fd140f0 0x555e9fd1cbb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:35650/0 (socket says 172.21.15.22:35650) 2024-03-28T18:19:21.980 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:19:21.976+0000 7f03c7e00640 1 -- 172.21.15.22:0/2743273673 learned_addr learned my addr 172.21.15.22:0/2743273673 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-28T18:19:21.981 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:19:21.976+0000 7f03cd000640 1 --2- 172.21.15.22:0/2743273673 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x555e9fd1d460 0x555e9fd1a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:19:21.981 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:19:21.976+0000 7f03bfe00640 1 --2- 172.21.15.22:0/2743273673 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555e9fd14dd0 0x555e9fd19f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:19:21.981 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:19:21.977+0000 7f03c7e00640 1 -- 172.21.15.22:0/2743273673 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555e9fd14dd0 msgr2=0x555e9fd19f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T18:19:21.981 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:19:21.977+0000 7f03c7e00640 1 --2- 172.21.15.22:0/2743273673 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555e9fd14dd0 0x555e9fd19f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:19:21.981 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:19:21.977+0000 7f03c7e00640 1 -- 172.21.15.22:0/2743273673 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x555e9fd1d460 msgr2=0x555e9fd1a480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T18:19:21.981 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:19:21.977+0000 7f03c7e00640 1 --2- 172.21.15.22:0/2743273673 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x555e9fd1d460 0x555e9fd1a480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:19:21.981 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:19:21.977+0000 7f03c7e00640 1 -- 172.21.15.22:0/2743273673 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x555e9fa567d0 con 0x555e9fd140f0 2024-03-28T18:19:21.981 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:19:21.977+0000 7f03cd000640 1 --2- 172.21.15.22:0/2743273673 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x555e9fd1d460 0x555e9fd1a480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-28T18:19:21.981 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:19:21.977+0000 7f03c7e00640 1 --2- 172.21.15.22:0/2743273673 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x555e9fd140f0 0x555e9fd1cbb0 secure :-1 s=READY pgs=808 cs=0 l=1 rev1=1 crypto rx=0x7f03b8003870 tx=0x7f03b80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cfb236773b6608ee server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:19:21.981 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:19:21.977+0000 7f03b6000640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-03-28T18:19:21.984 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:19:21.980+0000 7f03b6000640 1 -- 172.21.15.22:0/2743273673 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03b8024070 con 0x555e9fd140f0 2024-03-28T18:19:21.984 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:19:21.980+0000 7f03b6000640 1 -- 172.21.15.22:0/2743273673 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(122..122 src has 1..122) v4 ==== 10389+0+0 (secure 0 0 0) 0x7f03b80119d0 con 0x555e9fd140f0 2024-03-28T18:19:21.984 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:19:21.980+0000 7f03d1c49740 1 -- 172.21.15.22:0/2743273673 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f03c0033be0 con 0x555e9fd140f0 2024-03-28T18:19:21.984 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:19:21.980+0000 7f03b6000640 1 -- 172.21.15.22:0/2743273673 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=187) v2 ==== 24+0+0 (secure 0 0 0) 0x7f03b8030050 con 0x555e9fd140f0 2024-03-28T18:19:21.984 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:19:21.980+0000 7f03d1c49740 10 client.16890 fetch_fsmap learned FSMap version 187 2024-03-28T18:19:21.984 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:19:21.980+0000 7f03d1c49740 1 -- 172.21.15.22:0/2743273673 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=187,monmap=4+}) v3 -- 0x555e9f98c3f0 con 0x555e9fd140f0 2024-03-28T18:19:21.984 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:19:21.981+0000 7f03b6000640 1 -- 172.21.15.22:0/2743273673 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f03c0033be0 con 0x555e9fd140f0 2024-03-28T18:19:21.984 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:19:21.981+0000 7f03b6000640 1 -- 172.21.15.22:0/2743273673 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 187) v1 ==== 6221+0+0 (secure 0 0 0) 0x7f03b802acf0 con 0x555e9fd140f0 2024-03-28T18:19:21.984 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:19:21.981+0000 7f03d1c49740 10 client.16890 fetch_fsmap finished waiting for FSMap version 187 2024-03-28T18:19:21.984 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:19:21.981+0000 7f03d1c49740 10 client.16890 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-28T18:19:21.984 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:19:21.981+0000 7f03d1c49740 20 client.16890 populate_metadata read hostname 'smithi022' 2024-03-28T18:19:21.985 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:19:21.981+0000 7f03d1c49740 1 --2- 172.21.15.22:0/2743273673 >> [v2:172.21.15.79:6838/2441542443,v1:172.21.15.79:6839/2441542443] conn(0x555e9fd2d4f0 0x555e9fd2f970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:19:21.985 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:19:21.981+0000 7f03d1c49740 4 client.16890 mds_command: new command op to 34553 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-28T18:19:21.985 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:19:21.981+0000 7f03d1c49740 1 -- 172.21.15.22:0/2743273673 --> [v2:172.21.15.79:6838/2441542443,v1:172.21.15.79:6839/2441542443] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x555e9fc85790 con 0x555e9fd2d4f0 2024-03-28T18:19:21.985 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:19:21.981+0000 7f03bfe00640 1 --2- 172.21.15.22:0/2743273673 >> [v2:172.21.15.79:6838/2441542443,v1:172.21.15.79:6839/2441542443] conn(0x555e9fd2d4f0 0x555e9fd2f970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:19:21.985 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:19:21.982+0000 7f03bfe00640 1 --2- 172.21.15.22:0/2743273673 >> [v2:172.21.15.79:6838/2441542443,v1:172.21.15.79:6839/2441542443] conn(0x555e9fd2d4f0 0x555e9fd2f970 crc :-1 s=READY pgs=305 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=75c6d97cccc79630 server_cookie=269ba5bccecf513 in_seq=0 out_seq=0 2024-03-28T18:19:21.985 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:19:21.982+0000 7f03b6000640 10 client.16890 ms_handle_connect on v2:172.21.15.79:6838/2441542443 2024-03-28T18:19:21.985 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:19:21.984+0000 7f03b6000640 1 -- 172.21.15.22:0/2743273673 <== mds.0 v2:172.21.15.79:6838/2441542443 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x555e9fc85790 con 0x555e9fd2d4f0 2024-03-28T18:19:21.985 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:19:21.984+0000 7f03b6000640 10 client.16890 handle_command_reply: tid=0 2024-03-28T18:19:21.985 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:19:21.984+0000 7f03d1c49740 1 client.16890 shutdown 2024-03-28T18:19:21.986 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:19:21.984+0000 7f03d1c49740 1 -- 172.21.15.22:0/2743273673 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x555e9fd140f0 msgr2=0x555e9fd1cbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:19:21.986 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:19:21.984+0000 7f03d1c49740 1 --2- 172.21.15.22:0/2743273673 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x555e9fd140f0 0x555e9fd1cbb0 secure :-1 s=READY pgs=808 cs=0 l=1 rev1=1 crypto rx=0x7f03b8003870 tx=0x7f03b80046c0 comp rx=0 tx=0).stop 2024-03-28T18:19:21.986 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:19:21.984+0000 7f03d1c49740 1 -- 172.21.15.22:0/2743273673 shutdown_connections 2024-03-28T18:19:21.986 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:19:21.984+0000 7f03d1c49740 1 --2- 172.21.15.22:0/2743273673 >> [v2:172.21.15.79:6838/2441542443,v1:172.21.15.79:6839/2441542443] conn(0x555e9fd2d4f0 0x555e9fd2f970 crc :-1 s=READY pgs=305 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:19:21.986 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:19:21.984+0000 7f03d1c49740 1 --2- 172.21.15.22:0/2743273673 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x555e9fd1d460 0x555e9fd1a480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:19:21.986 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:19:21.984+0000 7f03d1c49740 1 --2- 172.21.15.22:0/2743273673 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555e9fd14dd0 0x555e9fd19f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:19:21.986 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:19:21.984+0000 7f03d1c49740 1 --2- 172.21.15.22:0/2743273673 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x555e9fd140f0 0x555e9fd1cbb0 unknown :-1 s=CLOSED pgs=808 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:19:21.987 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:19:21.984+0000 7f03b6000640 0 client.16890 ms_handle_reset on v2:172.21.15.79:6838/2441542443 2024-03-28T18:19:21.987 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:19:21.985+0000 7f03d1c49740 1 -- 172.21.15.22:0/2743273673 >> 172.21.15.22:0/2743273673 conn(0x555e9fcdb200 msgr2=0x555e9fd10890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:19:21.987 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:19:21.985+0000 7f03d1c49740 1 -- 172.21.15.22:0/2743273673 shutdown_connections 2024-03-28T18:19:21.987 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:19:21.985+0000 7f03d1c49740 1 -- 172.21.15.22:0/2743273673 wait complete. 2024-03-28T18:19:21.987 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:19:21.985+0000 7f03d1c49740 20 client.16890 trim_cache size 0 max 16384 2024-03-28T18:19:22.004 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:19:22.002+0000 7f03d1c49740 1 Processor -- start 2024-03-28T18:19:22.004 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:19:22.003+0000 7f03d1c49740 1 -- start start 2024-03-28T18:19:22.004 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:19:22.003+0000 7f03d1c49740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x555e9fd140f0 0x555e9fd1c670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:19:22.004 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:19:22.003+0000 7f03d1c49740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x555e9fd14dd0 0x555e9fd1cbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:19:22.004 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:19:22.003+0000 7f03d1c49740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555e9fd1d460 0x555e9fd1e850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:19:22.004 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:19:22.003+0000 7f03d1c49740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03c81b5ff0 con 0x555e9fd140f0 2024-03-28T18:19:22.004 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:19:22.003+0000 7f03d1c49740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03c81b3350 con 0x555e9fd14dd0 2024-03-28T18:19:22.004 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:19:22.003+0000 7f03d1c49740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03c81b5330 con 0x555e9fd1d460 2024-03-28T18:19:22.005 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:19:22.003+0000 7f03c7e00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x555e9fd140f0 0x555e9fd1c670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:19:22.005 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:19:22.003+0000 7f03c7e00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x555e9fd140f0 0x555e9fd1c670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:35656/0 (socket says 172.21.15.22:35656) 2024-03-28T18:19:22.005 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:19:22.003+0000 7f03c7e00640 1 -- 172.21.15.22:0/3245867092 learned_addr learned my addr 172.21.15.22:0/3245867092 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-28T18:19:22.005 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:19:22.003+0000 7f03bfe00640 1 --2- 172.21.15.22:0/3245867092 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x555e9fd14dd0 0x555e9fd1cbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:19:22.005 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:19:22.004+0000 7f03cd000640 1 --2- 172.21.15.22:0/3245867092 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555e9fd1d460 0x555e9fd1e850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:19:22.005 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:19:22.004+0000 7f03c7e00640 1 -- 172.21.15.22:0/3245867092 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555e9fd1d460 msgr2=0x555e9fd1e850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T18:19:22.005 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:19:22.004+0000 7f03c7e00640 1 --2- 172.21.15.22:0/3245867092 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555e9fd1d460 0x555e9fd1e850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:19:22.005 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:19:22.004+0000 7f03c7e00640 1 -- 172.21.15.22:0/3245867092 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x555e9fd14dd0 msgr2=0x555e9fd1cbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T18:19:22.005 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:19:22.004+0000 7f03c7e00640 1 --2- 172.21.15.22:0/3245867092 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x555e9fd14dd0 0x555e9fd1cbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:19:22.005 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:19:22.004+0000 7f03c7e00640 1 -- 172.21.15.22:0/3245867092 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555e9f94c0f0 con 0x555e9fd140f0 2024-03-28T18:19:22.005 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:19:22.004+0000 7f03c7e00640 1 --2- 172.21.15.22:0/3245867092 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x555e9fd140f0 0x555e9fd1c670 secure :-1 s=READY pgs=809 cs=0 l=1 rev1=1 crypto rx=0x7f03b8027360 tx=0x7f03b8000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ca02441f8d872f4e server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:19:22.005 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:19:22.004+0000 7f03bcc00640 1 -- 172.21.15.22:0/3245867092 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03b8024070 con 0x555e9fd140f0 2024-03-28T18:19:22.005 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:19:22.005+0000 7f03bcc00640 1 -- 172.21.15.22:0/3245867092 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f03b8010ea0 con 0x555e9fd140f0 2024-03-28T18:19:22.005 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:19:22.005+0000 7f03bcc00640 1 -- 172.21.15.22:0/3245867092 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03b8005030 con 0x555e9fd140f0 2024-03-28T18:19:22.008 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:19:22.005+0000 7f03d1c49740 1 -- 172.21.15.22:0/3245867092 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x555e9fd140f0 msgr2=0x555e9fd1c670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:19:22.008 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:19:22.005+0000 7f03d1c49740 1 --2- 172.21.15.22:0/3245867092 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x555e9fd140f0 0x555e9fd1c670 secure :-1 s=READY pgs=809 cs=0 l=1 rev1=1 crypto rx=0x7f03b8027360 tx=0x7f03b8000bc0 comp rx=0 tx=0).stop 2024-03-28T18:19:22.009 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:19:22.005+0000 7f03d1c49740 1 -- 172.21.15.22:0/3245867092 shutdown_connections 2024-03-28T18:19:22.009 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:19:22.005+0000 7f03d1c49740 1 --2- 172.21.15.22:0/3245867092 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555e9fd1d460 0x555e9fd1e850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:19:22.009 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:19:22.005+0000 7f03d1c49740 1 --2- 172.21.15.22:0/3245867092 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x555e9fd14dd0 0x555e9fd1cbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:19:22.009 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:19:22.005+0000 7f03d1c49740 1 --2- 172.21.15.22:0/3245867092 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x555e9fd140f0 0x555e9fd1c670 unknown :-1 s=CLOSED pgs=809 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:19:22.009 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:19:22.005+0000 7f03d1c49740 1 -- 172.21.15.22:0/3245867092 >> 172.21.15.22:0/3245867092 conn(0x555e9fcdb200 msgr2=0x555e9fd0e670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:19:22.009 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:19:22.005+0000 7f03d1c49740 1 -- 172.21.15.22:0/3245867092 shutdown_connections 2024-03-28T18:19:22.009 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:19:22.005+0000 7f03d1c49740 1 -- 172.21.15.22:0/3245867092 wait complete. 2024-03-28T18:19:22.009 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:19:22.006+0000 7f03d1c49740 1 -- start start 2024-03-28T18:19:22.009 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:19:22.006+0000 7f03d1c49740 1 Processor -- start 2024-03-28T18:19:22.009 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:19:22.006+0000 7f03d1c49740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x555e9fd140f0 0x555e9fd171c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:19:22.009 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:19:22.006+0000 7f03d1c49740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x555e9fd14dd0 0x555e9fd17700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:19:22.009 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:19:22.006+0000 7f03d1c49740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555e9fd1d460 0x555e9fd17c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:19:22.010 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:19:22.006+0000 7f03d1c49740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03c81b4cd0 con 0x555e9fd14dd0 2024-03-28T18:19:22.010 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:19:22.006+0000 7f03d1c49740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03c81b39b0 con 0x555e9fd140f0 2024-03-28T18:19:22.010 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:19:22.006+0000 7f03d1c49740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03c81b4010 con 0x555e9fd1d460 2024-03-28T18:19:22.010 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:19:22.007+0000 7f03d1c49740 10 client.0 _ll_register_callbacks cb 0x7f03b8009d30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-28T18:19:22.010 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:19:22.007+0000 7f03bfe00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x555e9fd14dd0 0x555e9fd17700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:19:22.010 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:19:22.007+0000 7f03bfe00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x555e9fd14dd0 0x555e9fd17700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:35660/0 (socket says 172.21.15.22:35660) 2024-03-28T18:19:22.010 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:19:22.007+0000 7f03bfe00640 1 -- 172.21.15.22:0/1895241631 learned_addr learned my addr 172.21.15.22:0/1895241631 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-28T18:19:22.010 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:19:22.007+0000 7f03c7e00640 1 --2- 172.21.15.22:0/1895241631 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x555e9fd140f0 0x555e9fd171c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:19:22.010 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:19:22.007+0000 7f03cd000640 1 --2- 172.21.15.22:0/1895241631 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555e9fd1d460 0x555e9fd17c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:19:22.010 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:19:22.007+0000 7f03bfe00640 1 -- 172.21.15.22:0/1895241631 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555e9fd1d460 msgr2=0x555e9fd17c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T18:19:22.010 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:19:22.007+0000 7f03bfe00640 1 --2- 172.21.15.22:0/1895241631 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555e9fd1d460 0x555e9fd17c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:19:22.010 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:19:22.007+0000 7f03bfe00640 1 -- 172.21.15.22:0/1895241631 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x555e9fd140f0 msgr2=0x555e9fd171c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T18:19:22.010 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:19:22.007+0000 7f03bfe00640 1 --2- 172.21.15.22:0/1895241631 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x555e9fd140f0 0x555e9fd171c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:19:22.010 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:19:22.007+0000 7f03bfe00640 1 -- 172.21.15.22:0/1895241631 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x555e9fd1d840 con 0x555e9fd14dd0 2024-03-28T18:19:22.011 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:19:22.007+0000 7f03cd000640 1 --2- 172.21.15.22:0/1895241631 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555e9fd1d460 0x555e9fd17c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T18:19:22.011 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:19:22.007+0000 7f03bfe00640 1 --2- 172.21.15.22:0/1895241631 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x555e9fd14dd0 0x555e9fd17700 secure :-1 s=READY pgs=810 cs=0 l=1 rev1=1 crypto rx=0x7f03b00048f0 tx=0x7f03b0005f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6e0131fae8ed9ca3 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:19:22.011 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:19:22.007+0000 7f03b6000640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-03-28T18:19:22.011 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:19:22.008+0000 7f03b6000640 1 -- 172.21.15.22:0/1895241631 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03b000ce60 con 0x555e9fd14dd0 2024-03-28T18:19:22.011 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:19:22.008+0000 7f03b6000640 1 -- 172.21.15.22:0/1895241631 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(122..122 src has 1..122) v4 ==== 10389+0+0 (secure 0 0 0) 0x7f03b00ce070 con 0x555e9fd14dd0 2024-03-28T18:19:22.011 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:19:22.008+0000 7f03d1c49740 1 -- 172.21.15.22:0/1895241631 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f03b8010ea0 con 0x555e9fd14dd0 2024-03-28T18:19:22.011 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:19:22.008+0000 7f03b6000640 1 -- 172.21.15.22:0/1895241631 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=187) v2 ==== 24+0+0 (secure 0 0 0) 0x7f03b00056a0 con 0x555e9fd14dd0 2024-03-28T18:19:22.011 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:19:22.008+0000 7f03d1c49740 10 client.16896 fetch_fsmap learned FSMap version 187 2024-03-28T18:19:22.011 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:19:22.008+0000 7f03d1c49740 1 -- 172.21.15.22:0/1895241631 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=187,monmap=4+}) v3 -- 0x555e9fd890f0 con 0x555e9fd14dd0 2024-03-28T18:19:22.011 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:19:22.008+0000 7f03b6000640 1 -- 172.21.15.22:0/1895241631 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f03b8010ea0 con 0x555e9fd14dd0 2024-03-28T18:19:22.011 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:19:22.009+0000 7f03b6000640 1 -- 172.21.15.22:0/1895241631 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 187) v1 ==== 6221+0+0 (secure 0 0 0) 0x7f03b0098000 con 0x555e9fd14dd0 2024-03-28T18:19:22.011 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:19:22.009+0000 7f03d1c49740 10 client.16896 fetch_fsmap finished waiting for FSMap version 187 2024-03-28T18:19:22.011 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:19:22.009+0000 7f03d1c49740 10 client.16896 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-28T18:19:22.011 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:19:22.009+0000 7f03d1c49740 20 client.16896 populate_metadata read hostname 'smithi022' 2024-03-28T18:19:22.011 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:19:22.009+0000 7f03d1c49740 1 --2- 172.21.15.22:0/1895241631 >> [v2:172.21.15.79:6838/2441542443,v1:172.21.15.79:6839/2441542443] conn(0x555e9fd897b0 0x555e9fd8bbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:19:22.012 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:19:22.009+0000 7f03d1c49740 4 client.16896 mds_command: new command op to 34553 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-28T18:19:22.012 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:19:22.009+0000 7f03d1c49740 1 -- 172.21.15.22:0/1895241631 --> [v2:172.21.15.79:6838/2441542443,v1:172.21.15.79:6839/2441542443] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x555e9fc7fae0 con 0x555e9fd897b0 2024-03-28T18:19:22.013 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:19:22.009+0000 7f03c7e00640 1 --2- 172.21.15.22:0/1895241631 >> [v2:172.21.15.79:6838/2441542443,v1:172.21.15.79:6839/2441542443] conn(0x555e9fd897b0 0x555e9fd8bbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:19:22.013 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:19:22.010+0000 7f03c7e00640 1 --2- 172.21.15.22:0/1895241631 >> [v2:172.21.15.79:6838/2441542443,v1:172.21.15.79:6839/2441542443] conn(0x555e9fd897b0 0x555e9fd8bbd0 crc :-1 s=READY pgs=306 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=82e2a1cace4fb5d5 server_cookie=e6b8a2b32ca41df in_seq=0 out_seq=0 2024-03-28T18:19:22.013 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:19:22.010+0000 7f03b6000640 10 client.16896 ms_handle_connect on v2:172.21.15.79:6838/2441542443 2024-03-28T18:19:22.013 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:19:22.011+0000 7f03b6000640 1 -- 172.21.15.22:0/1895241631 <== mds.0 v2:172.21.15.79:6838/2441542443 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x555e9fc7fae0 con 0x555e9fd897b0 2024-03-28T18:19:22.013 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:19:22.011+0000 7f03b6000640 10 client.16896 handle_command_reply: tid=0 2024-03-28T18:19:22.013 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:19:22.011+0000 7f03d1c49740 1 client.16896 shutdown 2024-03-28T18:19:22.013 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:19:22.011+0000 7f03d1c49740 1 -- 172.21.15.22:0/1895241631 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x555e9fd14dd0 msgr2=0x555e9fd17700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:19:22.014 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:19:22.011+0000 7f03d1c49740 1 --2- 172.21.15.22:0/1895241631 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x555e9fd14dd0 0x555e9fd17700 secure :-1 s=READY pgs=810 cs=0 l=1 rev1=1 crypto rx=0x7f03b00048f0 tx=0x7f03b0005f10 comp rx=0 tx=0).stop 2024-03-28T18:19:22.014 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:19:22.011+0000 7f03d1c49740 1 -- 172.21.15.22:0/1895241631 shutdown_connections 2024-03-28T18:19:22.014 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:19:22.011+0000 7f03d1c49740 1 --2- 172.21.15.22:0/1895241631 >> [v2:172.21.15.79:6838/2441542443,v1:172.21.15.79:6839/2441542443] conn(0x555e9fd897b0 0x555e9fd8bbd0 crc :-1 s=READY pgs=306 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:19:22.014 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:19:22.011+0000 7f03d1c49740 1 --2- 172.21.15.22:0/1895241631 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555e9fd1d460 0x555e9fd17c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:19:22.014 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:19:22.011+0000 7f03d1c49740 1 --2- 172.21.15.22:0/1895241631 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x555e9fd14dd0 0x555e9fd17700 unknown :-1 s=CLOSED pgs=810 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:19:22.014 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:19:22.011+0000 7f03d1c49740 1 --2- 172.21.15.22:0/1895241631 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x555e9fd140f0 0x555e9fd171c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:19:22.014 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:19:22.011+0000 7f03d1c49740 1 -- 172.21.15.22:0/1895241631 >> 172.21.15.22:0/1895241631 conn(0x555e9fcdb200 msgr2=0x555e9fd18810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:19:22.014 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:19:22.011+0000 7f03b6000640 0 client.16896 ms_handle_reset on v2:172.21.15.79:6838/2441542443 2024-03-28T18:19:22.014 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:19:22.011+0000 7f03d1c49740 1 -- 172.21.15.22:0/1895241631 shutdown_connections 2024-03-28T18:19:22.014 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:19:22.011+0000 7f03d1c49740 1 -- 172.21.15.22:0/1895241631 wait complete. 2024-03-28T18:19:22.014 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:19:22.011+0000 7f03d1c49740 20 client.16896 trim_cache size 0 max 16384 2024-03-28T18:19:22.015 INFO:teuthology.orchestra.run.smithi022.stdout:{ 2024-03-28T18:19:22.015 INFO:teuthology.orchestra.run.smithi022.stdout: "status": "IDLE (waiting for more scrubs)", 2024-03-28T18:19:22.015 INFO:teuthology.orchestra.run.smithi022.stdout: "scrubs": { 2024-03-28T18:19:22.015 INFO:teuthology.orchestra.run.smithi022.stdout: "4bbbd61d-29a3-4bf6-a4f2-d5783d585de3": { 2024-03-28T18:19:22.015 INFO:teuthology.orchestra.run.smithi022.stdout: "path": "/", 2024-03-28T18:19:22.015 INFO:teuthology.orchestra.run.smithi022.stdout: "tag": "4bbbd61d-29a3-4bf6-a4f2-d5783d585de3", 2024-03-28T18:19:22.015 INFO:teuthology.orchestra.run.smithi022.stdout: "options": "recursive,force" 2024-03-28T18:19:22.015 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-03-28T18:19:22.015 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-03-28T18:19:22.015 INFO:teuthology.orchestra.run.smithi022.stdout:} 2024-03-28T18:19:22.016 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:19:22.012+0000 7f03cda00640 1 -- 172.21.15.22:0/1823907570 >> [v2:172.21.15.22:6800/1357365082,v1:172.21.15.22:6801/1357365082] conn(0x7f03ac0a2970 msgr2=0x7f03ac0a4e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:19:22.016 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:19:22.012+0000 7f03cda00640 1 --2- 172.21.15.22:0/1823907570 >> [v2:172.21.15.22:6800/1357365082,v1:172.21.15.22:6801/1357365082] conn(0x7f03ac0a2970 0x7f03ac0a4e30 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x555e9fd15320 tx=0x7f03b8009260 comp rx=0 tx=0).stop 2024-03-28T18:19:22.016 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:19:22.012+0000 7f03cda00640 1 -- 172.21.15.22:0/1823907570 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f03c810cea0 msgr2=0x7f03c819e930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:19:22.016 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:19:22.012+0000 7f03cda00640 1 --2- 172.21.15.22:0/1823907570 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f03c810cea0 0x7f03c819e930 secure :-1 s=READY pgs=807 cs=0 l=1 rev1=1 crypto rx=0x7f03b000e9c0 tx=0x7f03b000ee80 comp rx=0 tx=0).stop 2024-03-28T18:19:22.016 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:19:22.012+0000 7f03cda00640 1 -- 172.21.15.22:0/1823907570 shutdown_connections 2024-03-28T18:19:22.016 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:19:22.012+0000 7f03cda00640 1 --2- 172.21.15.22:0/1823907570 >> [v2:172.21.15.22:6800/1357365082,v1:172.21.15.22:6801/1357365082] conn(0x7f03ac0a2970 0x7f03ac0a4e30 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:19:22.016 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:19:22.012+0000 7f03cda00640 1 --2- 172.21.15.22:0/1823907570 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f03c81a2cb0 0x7f03c81a3160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:19:22.016 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:19:22.012+0000 7f03cda00640 1 --2- 172.21.15.22:0/1823907570 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f03c810cea0 0x7f03c819e930 unknown :-1 s=CLOSED pgs=807 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:19:22.016 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:19:22.012+0000 7f03cda00640 1 --2- 172.21.15.22:0/1823907570 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f03c8107e50 0x7f03c819e3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:19:22.017 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:19:22.012+0000 7f03cda00640 1 -- 172.21.15.22:0/1823907570 >> 172.21.15.22:0/1823907570 conn(0x7f03c80774c0 msgr2=0x7f03c8077a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:19:22.017 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:19:22.012+0000 7f03cda00640 1 -- 172.21.15.22:0/1823907570 shutdown_connections 2024-03-28T18:19:22.017 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:19:22.012+0000 7f03cda00640 1 -- 172.21.15.22:0/1823907570 wait complete. 2024-03-28T18:19:22.032 INFO:tasks.cephfs.filesystem:scrub status for tag:4bbbd61d-29a3-4bf6-a4f2-d5783d585de3 - {'path': '/', 'tag': '4bbbd61d-29a3-4bf6-a4f2-d5783d585de3', 'options': 'recursive,force'} 2024-03-28T18:19:23.282 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:19:22 smithi022 ceph-mon[31985]: pgmap v388: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:19:23.424 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:19:22 smithi134 ceph-mon[35037]: pgmap v388: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:19:23.432 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:19:22 smithi079 ceph-mon[35312]: pgmap v388: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:19:25.282 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:19:24 smithi022 ceph-mon[31985]: pgmap v389: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:19:25.425 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:19:24 smithi134 ceph-mon[35037]: pgmap v389: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:19:25.432 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:19:24 smithi079 ceph-mon[35312]: pgmap v389: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:19:27.282 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:19:26 smithi022 ceph-mon[31985]: pgmap v390: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:19:27.424 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:19:26 smithi134 ceph-mon[35037]: pgmap v390: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:19:27.433 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:19:26 smithi079 ceph-mon[35312]: pgmap v390: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:19:29.282 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:19:28 smithi022 ceph-mon[31985]: pgmap v391: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:19:29.424 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:19:28 smithi134 ceph-mon[35037]: pgmap v391: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:19:29.432 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:19:28 smithi079 ceph-mon[35312]: pgmap v391: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:19:31.282 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:19:31 smithi022 ceph-mon[31985]: pgmap v392: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:19:31.424 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:19:31 smithi134 ceph-mon[35037]: pgmap v392: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:19:31.432 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:19:31 smithi079 ceph-mon[35312]: pgmap v392: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:19:33.282 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:19:33 smithi022 ceph-mon[31985]: pgmap v393: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:19:33.424 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:19:33 smithi134 ceph-mon[35037]: pgmap v393: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:19:33.432 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:19:33 smithi079 ceph-mon[35312]: pgmap v393: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:19:35.282 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:19:35 smithi022 ceph-mon[31985]: pgmap v394: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:19:35.424 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:19:35 smithi134 ceph-mon[35037]: pgmap v394: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:19:35.432 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:19:35 smithi079 ceph-mon[35312]: pgmap v394: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:19:37.283 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:19:37 smithi022 ceph-mon[31985]: pgmap v395: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:19:37.425 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:19:37 smithi134 ceph-mon[35037]: pgmap v395: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:19:37.432 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:19:37 smithi079 ceph-mon[35312]: pgmap v395: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:19:39.282 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:19:39 smithi022 ceph-mon[31985]: pgmap v396: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:19:39.424 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:19:39 smithi134 ceph-mon[35037]: pgmap v396: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:19:39.432 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:19:39 smithi079 ceph-mon[35312]: pgmap v396: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:19:40.424 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:19:40 smithi134 ceph-mon[35037]: pgmap v397: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:19:40.432 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:19:40 smithi079 ceph-mon[35312]: pgmap v397: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:19:40.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:19:40 smithi022 ceph-mon[31985]: pgmap v397: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:19:42.925 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:19:42 smithi134 ceph-mon[35037]: pgmap v398: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:19:42.933 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:19:42 smithi079 ceph-mon[35312]: pgmap v398: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:19:43.031 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:19:42 smithi022 ceph-mon[31985]: pgmap v398: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:19:44.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:19:44 smithi134 ceph-mon[35037]: pgmap v399: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:19:44.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:19:44 smithi079 ceph-mon[35312]: pgmap v399: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:19:45.032 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:19:44 smithi022 ceph-mon[31985]: pgmap v399: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:19:46.925 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:19:46 smithi134 ceph-mon[35037]: pgmap v400: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:19:46.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:19:46 smithi079 ceph-mon[35312]: pgmap v400: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:19:47.032 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:19:46 smithi022 ceph-mon[31985]: pgmap v400: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:19:48.925 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:19:48 smithi134 ceph-mon[35037]: pgmap v401: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:19:48.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:19:48 smithi079 ceph-mon[35312]: pgmap v401: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:19:49.032 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:19:48 smithi022 ceph-mon[31985]: pgmap v401: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:19:50.925 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:19:50 smithi134 ceph-mon[35037]: pgmap v402: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:19:50.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:19:50 smithi079 ceph-mon[35312]: pgmap v402: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:19:51.032 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:19:50 smithi022 ceph-mon[31985]: pgmap v402: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:19:52.033 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-28T18:19:52.033 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-28T18:19:52.268 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:19:52.267+0000 7f8a07600640 1 -- 172.21.15.22:0/2144743674 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8a000764f0 msgr2=0x7f8a00111fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:19:52.268 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:19:52.267+0000 7f8a07600640 1 --2- 172.21.15.22:0/2144743674 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8a000764f0 0x7f8a00111fa0 secure :-1 s=READY pgs=811 cs=0 l=1 rev1=1 crypto rx=0x7f89e8009900 tx=0x7f89e802bc30 comp rx=0 tx=0).stop 2024-03-28T18:19:52.268 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:19:52.268+0000 7f8a07600640 1 -- 172.21.15.22:0/2144743674 shutdown_connections 2024-03-28T18:19:52.268 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:19:52.268+0000 7f8a07600640 1 --2- 172.21.15.22:0/2144743674 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f8a001124e0 0x7f8a001148d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:19:52.268 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:19:52.268+0000 7f8a07600640 1 --2- 172.21.15.22:0/2144743674 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8a000764f0 0x7f8a00111fa0 unknown :-1 s=CLOSED pgs=811 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:19:52.268 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:19:52.268+0000 7f8a07600640 1 --2- 172.21.15.22:0/2144743674 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8a00075bd0 0x7f8a00075fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:19:52.268 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:19:52.268+0000 7f8a07600640 1 -- 172.21.15.22:0/2144743674 >> 172.21.15.22:0/2144743674 conn(0x7f8a000ff310 msgr2=0x7f8a00101730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:19:52.268 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:19:52.268+0000 7f8a07600640 1 -- 172.21.15.22:0/2144743674 shutdown_connections 2024-03-28T18:19:52.268 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:19:52.268+0000 7f8a07600640 1 -- 172.21.15.22:0/2144743674 wait complete. 2024-03-28T18:19:52.270 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:19:52.269+0000 7f8a07600640 1 Processor -- start 2024-03-28T18:19:52.270 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:19:52.269+0000 7f8a07600640 1 -- start start 2024-03-28T18:19:52.270 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:19:52.269+0000 7f8a07600640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8a00075bd0 0x7f8a0019e590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:19:52.270 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:19:52.269+0000 7f8a07600640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f8a000764f0 0x7f8a0019ead0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:19:52.270 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:19:52.269+0000 7f8a07600640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8a001124e0 0x7f8a001a2e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:19:52.270 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:19:52.269+0000 7f8a07600640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a00118030 con 0x7f8a001124e0 2024-03-28T18:19:52.270 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:19:52.269+0000 7f8a07600640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a00117eb0 con 0x7f8a000764f0 2024-03-28T18:19:52.270 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:19:52.269+0000 7f8a07600640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a001181b0 con 0x7f8a00075bd0 2024-03-28T18:19:52.270 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:19:52.270+0000 7f8a05600640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8a001124e0 0x7f8a001a2e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:19:52.270 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:19:52.270+0000 7f8a05600640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8a001124e0 0x7f8a001a2e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:50536/0 (socket says 172.21.15.22:50536) 2024-03-28T18:19:52.271 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:19:52.270+0000 7f8a05600640 1 -- 172.21.15.22:0/1088496679 learned_addr learned my addr 172.21.15.22:0/1088496679 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-28T18:19:52.271 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:19:52.270+0000 7f89f7e00640 1 --2- 172.21.15.22:0/1088496679 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f8a000764f0 0x7f8a0019ead0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:19:52.271 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:19:52.270+0000 7f8a04c00640 1 --2- 172.21.15.22:0/1088496679 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8a00075bd0 0x7f8a0019e590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:19:52.271 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:19:52.270+0000 7f8a05600640 1 -- 172.21.15.22:0/1088496679 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8a00075bd0 msgr2=0x7f8a0019e590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:19:52.271 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:19:52.270+0000 7f8a05600640 1 --2- 172.21.15.22:0/1088496679 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8a00075bd0 0x7f8a0019e590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:19:52.271 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:19:52.270+0000 7f8a05600640 1 -- 172.21.15.22:0/1088496679 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f8a000764f0 msgr2=0x7f8a0019ead0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:19:52.271 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:19:52.270+0000 7f8a05600640 1 --2- 172.21.15.22:0/1088496679 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f8a000764f0 0x7f8a0019ead0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:19:52.271 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:19:52.270+0000 7f8a05600640 1 -- 172.21.15.22:0/1088496679 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8a001a3580 con 0x7f8a001124e0 2024-03-28T18:19:52.272 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:19:52.271+0000 7f8a05600640 1 --2- 172.21.15.22:0/1088496679 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8a001124e0 0x7f8a001a2e50 secure :-1 s=READY pgs=812 cs=0 l=1 rev1=1 crypto rx=0x7f89f800e9d0 tx=0x7f89f800ee90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:19:52.272 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:19:52.271+0000 7f89fe000640 1 -- 172.21.15.22:0/1088496679 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89f800cd80 con 0x7f8a001124e0 2024-03-28T18:19:52.272 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:19:52.271+0000 7f89fe000640 1 -- 172.21.15.22:0/1088496679 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f89f8005020 con 0x7f8a001124e0 2024-03-28T18:19:52.272 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:19:52.271+0000 7f89fe000640 1 -- 172.21.15.22:0/1088496679 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89f80116d0 con 0x7f8a001124e0 2024-03-28T18:19:52.272 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:19:52.271+0000 7f8a07600640 1 -- 172.21.15.22:0/1088496679 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8a001a3870 con 0x7f8a001124e0 2024-03-28T18:19:52.272 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:19:52.271+0000 7f8a07600640 1 -- 172.21.15.22:0/1088496679 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8a001ab100 con 0x7f8a001124e0 2024-03-28T18:19:52.274 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:19:52.274+0000 7f8a0a418740 1 Processor -- start 2024-03-28T18:19:52.274 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:19:52.274+0000 7f89fe000640 1 -- 172.21.15.22:0/1088496679 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 26) v1 ==== 137421+0+0 (secure 0 0 0) 0x7f89f80051d0 con 0x7f8a001124e0 2024-03-28T18:19:52.275 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:19:52.275+0000 7f8a0a418740 1 -- start start 2024-03-28T18:19:52.275 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:19:52.275+0000 7f8a0a418740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x562845d47c00 0x562845d46950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:19:52.275 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:19:52.275+0000 7f8a0a418740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562845d46f20 0x562845d473a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:19:52.275 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:19:52.275+0000 7f8a0a418740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562845d50290 0x562845d52680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:19:52.275 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:19:52.275+0000 7f8a0a418740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x562845cab250 con 0x562845d46f20 2024-03-28T18:19:52.275 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:19:52.275+0000 7f8a0a418740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x562845c53a30 con 0x562845d47c00 2024-03-28T18:19:52.275 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:19:52.275+0000 7f8a0a418740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x562845ca72f0 con 0x562845d50290 2024-03-28T18:19:52.276 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:19:52.275+0000 7f8a04c00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562845d46f20 0x562845d473a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:19:52.276 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:19:52.275+0000 7f8a04c00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562845d46f20 0x562845d473a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:50540/0 (socket says 172.21.15.22:50540) 2024-03-28T18:19:52.276 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:19:52.276+0000 7f8a04c00640 1 -- 172.21.15.22:0/366554400 learned_addr learned my addr 172.21.15.22:0/366554400 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-28T18:19:52.276 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:19:52.276+0000 7f89f7e00640 1 --2- 172.21.15.22:0/366554400 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x562845d47c00 0x562845d46950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:19:52.278 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:19:52.276+0000 7f89f7e00640 1 --2- 172.21.15.22:0/366554400 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562845d50290 0x562845d52680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:19:52.278 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:19:52.276+0000 7f8a04c00640 1 -- 172.21.15.22:0/366554400 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562845d50290 msgr2=0x562845d52680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T18:19:52.278 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:19:52.276+0000 7f8a04c00640 1 --2- 172.21.15.22:0/366554400 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562845d50290 0x562845d52680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:19:52.278 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:19:52.276+0000 7f8a04c00640 1 -- 172.21.15.22:0/366554400 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x562845d47c00 msgr2=0x562845d46950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T18:19:52.278 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:19:52.276+0000 7f8a04c00640 1 --2- 172.21.15.22:0/366554400 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x562845d47c00 0x562845d46950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:19:52.278 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:19:52.276+0000 7f8a04c00640 1 -- 172.21.15.22:0/366554400 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562845c3e690 con 0x562845d46f20 2024-03-28T18:19:52.278 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:19:52.276+0000 7f89f7e00640 1 --2- 172.21.15.22:0/366554400 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x562845d47c00 0x562845d46950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-28T18:19:52.278 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:19:52.276+0000 7f89fe000640 1 --2- 172.21.15.22:0/1088496679 >> [v2:172.21.15.22:6800/1357365082,v1:172.21.15.22:6801/1357365082] conn(0x7f89e40a29f0 0x7f89e40a4eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:19:52.278 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:19:52.276+0000 7f8a04c00640 1 --2- 172.21.15.22:0/366554400 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562845d46f20 0x562845d473a0 secure :-1 s=READY pgs=813 cs=0 l=1 rev1=1 crypto rx=0x7f89f000b760 tx=0x7f89f000bc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d65fffe097b344c7 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:19:52.278 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:19:52.276+0000 7f89fe000640 1 -- 172.21.15.22:0/1088496679 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(122..122 src has 1..122) v4 ==== 10389+0+0 (secure 0 0 0) 0x7f89f80cf6a0 con 0x7f8a001124e0 2024-03-28T18:19:52.279 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:19:52.277+0000 7f89f7e00640 1 --2- 172.21.15.22:0/1088496679 >> [v2:172.21.15.22:6800/1357365082,v1:172.21.15.22:6801/1357365082] conn(0x7f89e40a29f0 0x7f89e40a4eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:19:52.279 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:19:52.277+0000 7f89f5c00640 1 -- 172.21.15.22:0/366554400 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89f0003dd0 con 0x562845d46f20 2024-03-28T18:19:52.279 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:19:52.277+0000 7f89f5c00640 1 -- 172.21.15.22:0/366554400 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f89f0015070 con 0x562845d46f20 2024-03-28T18:19:52.279 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:19:52.277+0000 7f89f5c00640 1 -- 172.21.15.22:0/366554400 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89f0011440 con 0x562845d46f20 2024-03-28T18:19:52.279 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:19:52.278+0000 7f8a0a418740 1 -- 172.21.15.22:0/366554400 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562845d46f20 msgr2=0x562845d473a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:19:52.279 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:19:52.278+0000 7f89f7e00640 1 --2- 172.21.15.22:0/1088496679 >> [v2:172.21.15.22:6800/1357365082,v1:172.21.15.22:6801/1357365082] conn(0x7f89e40a29f0 0x7f89e40a4eb0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f89e8006010 tx=0x7f89e8005e40 comp rx=0 tx=0).ready entity=mgr.15522 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:19:52.279 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:19:52.278+0000 7f8a0a418740 1 --2- 172.21.15.22:0/366554400 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562845d46f20 0x562845d473a0 secure :-1 s=READY pgs=813 cs=0 l=1 rev1=1 crypto rx=0x7f89f000b760 tx=0x7f89f000bc20 comp rx=0 tx=0).stop 2024-03-28T18:19:52.279 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:19:52.278+0000 7f8a0a418740 1 -- 172.21.15.22:0/366554400 shutdown_connections 2024-03-28T18:19:52.279 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:19:52.278+0000 7f8a0a418740 1 --2- 172.21.15.22:0/366554400 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562845d50290 0x562845d52680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:19:52.279 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:19:52.278+0000 7f8a0a418740 1 --2- 172.21.15.22:0/366554400 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562845d46f20 0x562845d473a0 unknown :-1 s=CLOSED pgs=813 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:19:52.279 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:19:52.278+0000 7f8a0a418740 1 --2- 172.21.15.22:0/366554400 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x562845d47c00 0x562845d46950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:19:52.279 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:19:52.278+0000 7f8a0a418740 1 -- 172.21.15.22:0/366554400 >> 172.21.15.22:0/366554400 conn(0x562845d0e200 msgr2=0x562845d41970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:19:52.279 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:19:52.278+0000 7f8a0a418740 1 -- 172.21.15.22:0/366554400 shutdown_connections 2024-03-28T18:19:52.279 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:19:52.278+0000 7f8a0a418740 1 -- 172.21.15.22:0/366554400 wait complete. 2024-03-28T18:19:52.279 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:19:52.278+0000 7f8a0a418740 1 -- start start 2024-03-28T18:19:52.282 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:19:52.278+0000 7f8a0a418740 1 Processor -- start 2024-03-28T18:19:52.283 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:19:52.279+0000 7f8a0a418740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562845d46f20 0x562845d4f8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:19:52.283 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:19:52.279+0000 7f8a0a418740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562845d47c00 0x562845d4cc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:19:52.283 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:19:52.279+0000 7f8a0a418740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x562845d50290 0x562845d4d1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:19:52.283 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:19:52.279+0000 7f8a0a418740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x562845d53100 con 0x562845d46f20 2024-03-28T18:19:52.283 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:19:52.279+0000 7f8a0a418740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x562845c66e80 con 0x562845d50290 2024-03-28T18:19:52.283 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:19:52.279+0000 7f8a0a418740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x562845d53280 con 0x562845d47c00 2024-03-28T18:19:52.283 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:19:52.279+0000 7f8a0a418740 10 client.0 _ll_register_callbacks cb 0x562845c79130 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-28T18:19:52.283 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:19:52.279+0000 7f8a04c00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562845d46f20 0x562845d4f8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:19:52.283 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:19:52.279+0000 7f8a04c00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562845d46f20 0x562845d4f8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:50544/0 (socket says 172.21.15.22:50544) 2024-03-28T18:19:52.283 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:19:52.279+0000 7f8a04c00640 1 -- 172.21.15.22:0/2105339627 learned_addr learned my addr 172.21.15.22:0/2105339627 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-28T18:19:52.283 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:19:52.279+0000 7f8a05600640 1 --2- 172.21.15.22:0/2105339627 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x562845d50290 0x562845d4d1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:19:52.283 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:19:52.280+0000 7f89f7e00640 1 --2- 172.21.15.22:0/2105339627 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562845d47c00 0x562845d4cc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:19:52.283 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:19:52.280+0000 7f8a04c00640 1 -- 172.21.15.22:0/2105339627 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562845d47c00 msgr2=0x562845d4cc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T18:19:52.283 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:19:52.280+0000 7f8a04c00640 1 --2- 172.21.15.22:0/2105339627 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562845d47c00 0x562845d4cc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:19:52.283 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:19:52.280+0000 7f8a04c00640 1 -- 172.21.15.22:0/2105339627 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x562845d50290 msgr2=0x562845d4d1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T18:19:52.284 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:19:52.280+0000 7f8a04c00640 1 --2- 172.21.15.22:0/2105339627 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x562845d50290 0x562845d4d1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:19:52.284 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:19:52.280+0000 7f8a04c00640 1 -- 172.21.15.22:0/2105339627 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x562845a897d0 con 0x562845d46f20 2024-03-28T18:19:52.284 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:19:52.280+0000 7f8a04c00640 1 --2- 172.21.15.22:0/2105339627 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562845d46f20 0x562845d4f8e0 secure :-1 s=READY pgs=814 cs=0 l=1 rev1=1 crypto rx=0x7f89f0004310 tx=0x7f89f0004e20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=833ad52c8a19b684 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:19:52.284 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:19:52.280+0000 7f89ee000640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-03-28T18:19:52.284 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:19:52.280+0000 7f89ee000640 1 -- 172.21.15.22:0/2105339627 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89f000be70 con 0x562845d46f20 2024-03-28T18:19:52.284 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:19:52.280+0000 7f89ee000640 1 -- 172.21.15.22:0/2105339627 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(122..122 src has 1..122) v4 ==== 10389+0+0 (secure 0 0 0) 0x7f89f0003de0 con 0x562845d46f20 2024-03-28T18:19:52.284 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:19:52.280+0000 7f8a0a418740 1 -- 172.21.15.22:0/2105339627 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f89f0015070 con 0x562845d46f20 2024-03-28T18:19:52.284 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:19:52.281+0000 7f89ee000640 1 -- 172.21.15.22:0/2105339627 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=187) v2 ==== 24+0+0 (secure 0 0 0) 0x7f89f0030050 con 0x562845d46f20 2024-03-28T18:19:52.284 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:19:52.281+0000 7f8a0a418740 10 client.16908 fetch_fsmap learned FSMap version 187 2024-03-28T18:19:52.284 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:19:52.281+0000 7f8a0a418740 1 -- 172.21.15.22:0/2105339627 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=187,monmap=4+}) v3 -- 0x5628459bf3f0 con 0x562845d46f20 2024-03-28T18:19:52.284 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:19:52.281+0000 7f89ee000640 1 -- 172.21.15.22:0/2105339627 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f89f0015070 con 0x562845d46f20 2024-03-28T18:19:52.284 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:19:52.281+0000 7f89ee000640 1 -- 172.21.15.22:0/2105339627 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 187) v1 ==== 6221+0+0 (secure 0 0 0) 0x7f89f0011970 con 0x562845d46f20 2024-03-28T18:19:52.284 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:19:52.281+0000 7f8a0a418740 10 client.16908 fetch_fsmap finished waiting for FSMap version 187 2024-03-28T18:19:52.284 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:19:52.281+0000 7f8a0a418740 10 client.16908 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-28T18:19:52.284 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:19:52.281+0000 7f8a0a418740 20 client.16908 populate_metadata read hostname 'smithi022' 2024-03-28T18:19:52.285 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:19:52.281+0000 7f8a0a418740 1 --2- 172.21.15.22:0/2105339627 >> [v2:172.21.15.79:6838/2441542443,v1:172.21.15.79:6839/2441542443] conn(0x562845d602a0 0x562845d62720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:19:52.285 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:19:52.281+0000 7f8a0a418740 4 client.16908 mds_command: new command op to 34553 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-28T18:19:52.285 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:19:52.281+0000 7f8a0a418740 1 -- 172.21.15.22:0/2105339627 --> [v2:172.21.15.79:6838/2441542443,v1:172.21.15.79:6839/2441542443] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x562845cb8790 con 0x562845d602a0 2024-03-28T18:19:52.285 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:19:52.282+0000 7f89f7e00640 1 --2- 172.21.15.22:0/2105339627 >> [v2:172.21.15.79:6838/2441542443,v1:172.21.15.79:6839/2441542443] conn(0x562845d602a0 0x562845d62720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:19:52.285 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:19:52.283+0000 7f89f7e00640 1 --2- 172.21.15.22:0/2105339627 >> [v2:172.21.15.79:6838/2441542443,v1:172.21.15.79:6839/2441542443] conn(0x562845d602a0 0x562845d62720 crc :-1 s=READY pgs=309 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5d9b2ed3f7d36be5 server_cookie=d9fb31b20d0de389 in_seq=0 out_seq=0 2024-03-28T18:19:52.285 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:19:52.283+0000 7f89ee000640 10 client.16908 ms_handle_connect on v2:172.21.15.79:6838/2441542443 2024-03-28T18:19:52.286 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:19:52.285+0000 7f89ee000640 1 -- 172.21.15.22:0/2105339627 <== mds.0 v2:172.21.15.79:6838/2441542443 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x562845cb8790 con 0x562845d602a0 2024-03-28T18:19:52.286 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:19:52.285+0000 7f89ee000640 10 client.16908 handle_command_reply: tid=0 2024-03-28T18:19:52.286 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:19:52.285+0000 7f8a0a418740 1 client.16908 shutdown 2024-03-28T18:19:52.286 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:19:52.285+0000 7f8a0a418740 1 -- 172.21.15.22:0/2105339627 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562845d46f20 msgr2=0x562845d4f8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:19:52.286 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:19:52.285+0000 7f8a0a418740 1 --2- 172.21.15.22:0/2105339627 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562845d46f20 0x562845d4f8e0 secure :-1 s=READY pgs=814 cs=0 l=1 rev1=1 crypto rx=0x7f89f0004310 tx=0x7f89f0004e20 comp rx=0 tx=0).stop 2024-03-28T18:19:52.286 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:19:52.285+0000 7f8a0a418740 1 -- 172.21.15.22:0/2105339627 shutdown_connections 2024-03-28T18:19:52.286 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:19:52.285+0000 7f8a0a418740 1 --2- 172.21.15.22:0/2105339627 >> [v2:172.21.15.79:6838/2441542443,v1:172.21.15.79:6839/2441542443] conn(0x562845d602a0 0x562845d62720 crc :-1 s=READY pgs=309 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:19:52.287 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:19:52.285+0000 7f8a0a418740 1 --2- 172.21.15.22:0/2105339627 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x562845d50290 0x562845d4d1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:19:52.287 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:19:52.285+0000 7f8a0a418740 1 --2- 172.21.15.22:0/2105339627 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562845d47c00 0x562845d4cc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:19:52.287 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:19:52.285+0000 7f8a0a418740 1 --2- 172.21.15.22:0/2105339627 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562845d46f20 0x562845d4f8e0 unknown :-1 s=CLOSED pgs=814 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:19:52.287 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:19:52.285+0000 7f89ee000640 0 client.16908 ms_handle_reset on v2:172.21.15.79:6838/2441542443 2024-03-28T18:19:52.287 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:19:52.285+0000 7f8a0a418740 1 -- 172.21.15.22:0/2105339627 >> 172.21.15.22:0/2105339627 conn(0x562845d0e200 msgr2=0x562845d436c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:19:52.287 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:19:52.285+0000 7f8a0a418740 1 -- 172.21.15.22:0/2105339627 shutdown_connections 2024-03-28T18:19:52.287 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:19:52.285+0000 7f8a0a418740 1 -- 172.21.15.22:0/2105339627 wait complete. 2024-03-28T18:19:52.287 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:19:52.286+0000 7f8a0a418740 20 client.16908 trim_cache size 0 max 16384 2024-03-28T18:19:52.303 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:19:52.303+0000 7f8a0a418740 1 Processor -- start 2024-03-28T18:19:52.303 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:19:52.303+0000 7f8a0a418740 1 -- start start 2024-03-28T18:19:52.304 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:19:52.304+0000 7f8a0a418740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562845d46f20 0x562845d51680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:19:52.304 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:19:52.304+0000 7f8a0a418740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562845d47c00 0x562845d51bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:19:52.304 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:19:52.304+0000 7f8a0a418740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x562845d50290 0x562845d49ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:19:52.304 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:19:52.304+0000 7f8a0a418740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a001b6230 con 0x562845d46f20 2024-03-28T18:19:52.304 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:19:52.304+0000 7f8a0a418740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a001b3590 con 0x562845d50290 2024-03-28T18:19:52.304 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:19:52.304+0000 7f8a0a418740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a001b5570 con 0x562845d47c00 2024-03-28T18:19:52.304 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:19:52.304+0000 7f8a04c00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562845d46f20 0x562845d51680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:19:52.304 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:19:52.304+0000 7f8a04c00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562845d46f20 0x562845d51680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:50552/0 (socket says 172.21.15.22:50552) 2024-03-28T18:19:52.305 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:19:52.304+0000 7f8a04c00640 1 -- 172.21.15.22:0/3984068030 learned_addr learned my addr 172.21.15.22:0/3984068030 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-28T18:19:52.305 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:19:52.304+0000 7f89f7e00640 1 --2- 172.21.15.22:0/3984068030 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562845d47c00 0x562845d51bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:19:52.305 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:19:52.304+0000 7f8a04c00640 1 -- 172.21.15.22:0/3984068030 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562845d47c00 msgr2=0x562845d51bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T18:19:52.305 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:19:52.304+0000 7f8a04c00640 1 --2- 172.21.15.22:0/3984068030 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562845d47c00 0x562845d51bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:19:52.305 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:19:52.304+0000 7f8a04c00640 1 -- 172.21.15.22:0/3984068030 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x562845d50290 msgr2=0x562845d49ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T18:19:52.305 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:19:52.304+0000 7f8a04c00640 1 --2- 172.21.15.22:0/3984068030 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x562845d50290 0x562845d49ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:19:52.305 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:19:52.304+0000 7f8a04c00640 1 -- 172.21.15.22:0/3984068030 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56284597f0f0 con 0x562845d46f20 2024-03-28T18:19:52.306 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:19:52.305+0000 7f8a04c00640 1 --2- 172.21.15.22:0/3984068030 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562845d46f20 0x562845d51680 secure :-1 s=READY pgs=815 cs=0 l=1 rev1=1 crypto rx=0x7f89f0006010 tx=0x7f89f000b390 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3b5d53306a8c9db6 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:19:52.306 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:19:52.305+0000 7f89f5200640 1 -- 172.21.15.22:0/3984068030 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89f0000e60 con 0x562845d46f20 2024-03-28T18:19:52.306 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:19:52.305+0000 7f89f5200640 1 -- 172.21.15.22:0/3984068030 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f89f0015070 con 0x562845d46f20 2024-03-28T18:19:52.306 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:19:52.305+0000 7f89f5200640 1 -- 172.21.15.22:0/3984068030 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89f00047e0 con 0x562845d46f20 2024-03-28T18:19:52.306 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:19:52.306+0000 7f8a0a418740 1 -- 172.21.15.22:0/3984068030 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562845d46f20 msgr2=0x562845d51680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:19:52.306 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:19:52.306+0000 7f8a0a418740 1 --2- 172.21.15.22:0/3984068030 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562845d46f20 0x562845d51680 secure :-1 s=READY pgs=815 cs=0 l=1 rev1=1 crypto rx=0x7f89f0006010 tx=0x7f89f000b390 comp rx=0 tx=0).stop 2024-03-28T18:19:52.307 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:19:52.306+0000 7f8a0a418740 1 -- 172.21.15.22:0/3984068030 shutdown_connections 2024-03-28T18:19:52.307 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:19:52.306+0000 7f8a0a418740 1 --2- 172.21.15.22:0/3984068030 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x562845d50290 0x562845d49ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:19:52.307 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:19:52.306+0000 7f8a0a418740 1 --2- 172.21.15.22:0/3984068030 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562845d47c00 0x562845d51bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:19:52.307 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:19:52.306+0000 7f8a0a418740 1 --2- 172.21.15.22:0/3984068030 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562845d46f20 0x562845d51680 unknown :-1 s=CLOSED pgs=815 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:19:52.307 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:19:52.306+0000 7f8a0a418740 1 -- 172.21.15.22:0/3984068030 >> 172.21.15.22:0/3984068030 conn(0x562845d0e200 msgr2=0x562845d4b540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:19:52.307 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:19:52.306+0000 7f8a0a418740 1 -- 172.21.15.22:0/3984068030 shutdown_connections 2024-03-28T18:19:52.307 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:19:52.306+0000 7f8a0a418740 1 -- 172.21.15.22:0/3984068030 wait complete. 2024-03-28T18:19:52.307 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:19:52.306+0000 7f8a0a418740 1 -- start start 2024-03-28T18:19:52.307 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:19:52.307+0000 7f8a0a418740 1 Processor -- start 2024-03-28T18:19:52.309 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:19:52.307+0000 7f8a0a418740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562845d46f20 0x562845d53960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:19:52.309 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:19:52.307+0000 7f8a0a418740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562845d47c00 0x562845d53ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:19:52.309 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:19:52.307+0000 7f8a0a418740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x562845d50290 0x562845d543e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:19:52.309 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:19:52.307+0000 7f8a0a418740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a001b4f10 con 0x562845d47c00 2024-03-28T18:19:52.309 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:19:52.307+0000 7f8a0a418740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a001b3bf0 con 0x562845d50290 2024-03-28T18:19:52.310 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:19:52.307+0000 7f8a0a418740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a001b4250 con 0x562845d46f20 2024-03-28T18:19:52.310 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:19:52.307+0000 7f8a0a418740 10 client.0 _ll_register_callbacks cb 0x7f89f0004590 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-28T18:19:52.310 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:19:52.307+0000 7f89f7e00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562845d47c00 0x562845d53ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:19:52.310 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:19:52.308+0000 7f89f7e00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562845d47c00 0x562845d53ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:50556/0 (socket says 172.21.15.22:50556) 2024-03-28T18:19:52.310 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:19:52.308+0000 7f89f7e00640 1 -- 172.21.15.22:0/832831609 learned_addr learned my addr 172.21.15.22:0/832831609 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-28T18:19:52.310 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:19:52.308+0000 7f8a05600640 1 --2- 172.21.15.22:0/832831609 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x562845d50290 0x562845d543e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:19:52.310 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:19:52.308+0000 7f8a04c00640 1 --2- 172.21.15.22:0/832831609 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562845d46f20 0x562845d53960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:19:52.310 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:19:52.308+0000 7f89f7e00640 1 -- 172.21.15.22:0/832831609 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562845d46f20 msgr2=0x562845d53960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T18:19:52.310 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:19:52.308+0000 7f89f7e00640 1 --2- 172.21.15.22:0/832831609 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562845d46f20 0x562845d53960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:19:52.310 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:19:52.308+0000 7f89f7e00640 1 -- 172.21.15.22:0/832831609 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x562845d50290 msgr2=0x562845d543e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T18:19:52.310 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:19:52.308+0000 7f89f7e00640 1 --2- 172.21.15.22:0/832831609 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x562845d50290 0x562845d543e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:19:52.310 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:19:52.308+0000 7f89f7e00640 1 -- 172.21.15.22:0/832831609 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x562845dbabb0 con 0x562845d47c00 2024-03-28T18:19:52.310 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:19:52.308+0000 7f8a05600640 1 --2- 172.21.15.22:0/832831609 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x562845d50290 0x562845d543e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-28T18:19:52.310 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:19:52.308+0000 7f8a04c00640 1 --2- 172.21.15.22:0/832831609 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562845d46f20 0x562845d53960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T18:19:52.310 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:19:52.308+0000 7f89f7e00640 1 --2- 172.21.15.22:0/832831609 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562845d47c00 0x562845d53ea0 secure :-1 s=READY pgs=816 cs=0 l=1 rev1=1 crypto rx=0x7f89e8036040 tx=0x7f89e8005dd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1d97c0bbb57e4a5a server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:19:52.311 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:19:52.308+0000 7f89ee000640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-03-28T18:19:52.311 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:19:52.309+0000 7f89ee000640 1 -- 172.21.15.22:0/832831609 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89e80042c0 con 0x562845d47c00 2024-03-28T18:19:52.311 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:19:52.309+0000 7f89ee000640 1 -- 172.21.15.22:0/832831609 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(122..122 src has 1..122) v4 ==== 10389+0+0 (secure 0 0 0) 0x7f89e803a040 con 0x562845d47c00 2024-03-28T18:19:52.311 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:19:52.309+0000 7f8a0a418740 1 -- 172.21.15.22:0/832831609 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f89f0015070 con 0x562845d47c00 2024-03-28T18:19:52.311 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:19:52.309+0000 7f89ee000640 1 -- 172.21.15.22:0/832831609 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=187) v2 ==== 24+0+0 (secure 0 0 0) 0x7f89e8004460 con 0x562845d47c00 2024-03-28T18:19:52.311 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:19:52.309+0000 7f8a0a418740 10 client.16914 fetch_fsmap learned FSMap version 187 2024-03-28T18:19:52.311 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:19:52.309+0000 7f8a0a418740 1 -- 172.21.15.22:0/832831609 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=187,monmap=4+}) v3 -- 0x562845dbbe50 con 0x562845d47c00 2024-03-28T18:19:52.311 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:19:52.309+0000 7f89ee000640 1 -- 172.21.15.22:0/832831609 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f89f0015070 con 0x562845d47c00 2024-03-28T18:19:52.311 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:19:52.309+0000 7f89ee000640 1 -- 172.21.15.22:0/832831609 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 187) v1 ==== 6221+0+0 (secure 0 0 0) 0x7f89e8004a40 con 0x562845d47c00 2024-03-28T18:19:52.311 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:19:52.309+0000 7f8a0a418740 10 client.16914 fetch_fsmap finished waiting for FSMap version 187 2024-03-28T18:19:52.311 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:19:52.309+0000 7f8a0a418740 10 client.16914 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-28T18:19:52.311 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:19:52.309+0000 7f8a0a418740 20 client.16914 populate_metadata read hostname 'smithi022' 2024-03-28T18:19:52.311 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:19:52.309+0000 7f8a0a418740 1 --2- 172.21.15.22:0/832831609 >> [v2:172.21.15.79:6838/2441542443,v1:172.21.15.79:6839/2441542443] conn(0x562845dbc560 0x562845dbe980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:19:52.311 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:19:52.309+0000 7f8a0a418740 4 client.16914 mds_command: new command op to 34553 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-28T18:19:52.311 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:19:52.309+0000 7f8a0a418740 1 -- 172.21.15.22:0/832831609 --> [v2:172.21.15.79:6838/2441542443,v1:172.21.15.79:6839/2441542443] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x562845cb2ae0 con 0x562845dbc560 2024-03-28T18:19:52.313 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:19:52.310+0000 7f8a04c00640 1 --2- 172.21.15.22:0/832831609 >> [v2:172.21.15.79:6838/2441542443,v1:172.21.15.79:6839/2441542443] conn(0x562845dbc560 0x562845dbe980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:19:52.313 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:19:52.311+0000 7f8a04c00640 1 --2- 172.21.15.22:0/832831609 >> [v2:172.21.15.79:6838/2441542443,v1:172.21.15.79:6839/2441542443] conn(0x562845dbc560 0x562845dbe980 crc :-1 s=READY pgs=310 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fb3e74ea15e59f23 server_cookie=16422d02cc29e45a in_seq=0 out_seq=0 2024-03-28T18:19:52.313 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:19:52.311+0000 7f89ee000640 10 client.16914 ms_handle_connect on v2:172.21.15.79:6838/2441542443 2024-03-28T18:19:52.313 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:19:52.311+0000 7f89ee000640 1 -- 172.21.15.22:0/832831609 <== mds.0 v2:172.21.15.79:6838/2441542443 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x562845cb2ae0 con 0x562845dbc560 2024-03-28T18:19:52.313 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:19:52.311+0000 7f89ee000640 10 client.16914 handle_command_reply: tid=0 2024-03-28T18:19:52.313 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:19:52.311+0000 7f8a0a418740 1 client.16914 shutdown 2024-03-28T18:19:52.313 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:19:52.311+0000 7f8a0a418740 1 -- 172.21.15.22:0/832831609 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562845d47c00 msgr2=0x562845d53ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:19:52.313 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:19:52.311+0000 7f8a0a418740 1 --2- 172.21.15.22:0/832831609 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562845d47c00 0x562845d53ea0 secure :-1 s=READY pgs=816 cs=0 l=1 rev1=1 crypto rx=0x7f89e8036040 tx=0x7f89e8005dd0 comp rx=0 tx=0).stop 2024-03-28T18:19:52.313 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:19:52.311+0000 7f8a0a418740 1 -- 172.21.15.22:0/832831609 shutdown_connections 2024-03-28T18:19:52.313 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:19:52.311+0000 7f8a0a418740 1 --2- 172.21.15.22:0/832831609 >> [v2:172.21.15.79:6838/2441542443,v1:172.21.15.79:6839/2441542443] conn(0x562845dbc560 0x562845dbe980 crc :-1 s=READY pgs=310 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:19:52.313 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:19:52.312+0000 7f8a0a418740 1 --2- 172.21.15.22:0/832831609 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x562845d50290 0x562845d543e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:19:52.313 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:19:52.312+0000 7f8a0a418740 1 --2- 172.21.15.22:0/832831609 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562845d47c00 0x562845d53ea0 unknown :-1 s=CLOSED pgs=816 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:19:52.313 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:19:52.312+0000 7f8a0a418740 1 --2- 172.21.15.22:0/832831609 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562845d46f20 0x562845d53960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:19:52.313 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:19:52.312+0000 7f89ee000640 0 client.16914 ms_handle_reset on v2:172.21.15.79:6838/2441542443 2024-03-28T18:19:52.313 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:19:52.312+0000 7f8a0a418740 1 -- 172.21.15.22:0/832831609 >> 172.21.15.22:0/832831609 conn(0x562845d0e200 msgr2=0x562845d40200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:19:52.314 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:19:52.312+0000 7f8a0a418740 1 -- 172.21.15.22:0/832831609 shutdown_connections 2024-03-28T18:19:52.314 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:19:52.312+0000 7f8a0a418740 1 -- 172.21.15.22:0/832831609 wait complete. 2024-03-28T18:19:52.314 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:19:52.312+0000 7f8a0a418740 20 client.16914 trim_cache size 0 max 16384 2024-03-28T18:19:52.314 INFO:teuthology.orchestra.run.smithi022.stdout:{ 2024-03-28T18:19:52.314 INFO:teuthology.orchestra.run.smithi022.stdout: "status": "IDLE (waiting for more scrubs)", 2024-03-28T18:19:52.314 INFO:teuthology.orchestra.run.smithi022.stdout: "scrubs": { 2024-03-28T18:19:52.314 INFO:teuthology.orchestra.run.smithi022.stdout: "4bbbd61d-29a3-4bf6-a4f2-d5783d585de3": { 2024-03-28T18:19:52.314 INFO:teuthology.orchestra.run.smithi022.stdout: "path": "/", 2024-03-28T18:19:52.314 INFO:teuthology.orchestra.run.smithi022.stdout: "tag": "4bbbd61d-29a3-4bf6-a4f2-d5783d585de3", 2024-03-28T18:19:52.314 INFO:teuthology.orchestra.run.smithi022.stdout: "options": "recursive,force" 2024-03-28T18:19:52.314 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-03-28T18:19:52.314 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-03-28T18:19:52.315 INFO:teuthology.orchestra.run.smithi022.stdout:} 2024-03-28T18:19:52.315 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:19:52.312+0000 7f8a07600640 1 -- 172.21.15.22:0/1088496679 >> [v2:172.21.15.22:6800/1357365082,v1:172.21.15.22:6801/1357365082] conn(0x7f89e40a29f0 msgr2=0x7f89e40a4eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:19:52.315 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:19:52.312+0000 7f8a07600640 1 --2- 172.21.15.22:0/1088496679 >> [v2:172.21.15.22:6800/1357365082,v1:172.21.15.22:6801/1357365082] conn(0x7f89e40a29f0 0x7f89e40a4eb0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f89e8006010 tx=0x7f89e8005e40 comp rx=0 tx=0).stop 2024-03-28T18:19:52.315 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:19:52.312+0000 7f8a07600640 1 -- 172.21.15.22:0/1088496679 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8a001124e0 msgr2=0x7f8a001a2e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:19:52.315 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:19:52.312+0000 7f8a07600640 1 --2- 172.21.15.22:0/1088496679 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8a001124e0 0x7f8a001a2e50 secure :-1 s=READY pgs=812 cs=0 l=1 rev1=1 crypto rx=0x7f89f800e9d0 tx=0x7f89f800ee90 comp rx=0 tx=0).stop 2024-03-28T18:19:52.316 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:19:52.313+0000 7f8a07600640 1 -- 172.21.15.22:0/1088496679 shutdown_connections 2024-03-28T18:19:52.316 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:19:52.313+0000 7f8a07600640 1 --2- 172.21.15.22:0/1088496679 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8a001124e0 0x7f8a001a2e50 unknown :-1 s=CLOSED pgs=812 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:19:52.316 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:19:52.313+0000 7f8a07600640 1 --2- 172.21.15.22:0/1088496679 >> [v2:172.21.15.22:6800/1357365082,v1:172.21.15.22:6801/1357365082] conn(0x7f89e40a29f0 0x7f89e40a4eb0 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:19:52.316 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:19:52.313+0000 7f8a07600640 1 --2- 172.21.15.22:0/1088496679 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f8a000764f0 0x7f8a0019ead0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:19:52.316 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:19:52.313+0000 7f8a07600640 1 --2- 172.21.15.22:0/1088496679 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8a00075bd0 0x7f8a0019e590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:19:52.316 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:19:52.313+0000 7f8a07600640 1 -- 172.21.15.22:0/1088496679 >> 172.21.15.22:0/1088496679 conn(0x7f8a000ff310 msgr2=0x7f8a00113c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:19:52.316 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:19:52.313+0000 7f8a07600640 1 -- 172.21.15.22:0/1088496679 shutdown_connections 2024-03-28T18:19:52.316 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:19:52.313+0000 7f8a07600640 1 -- 172.21.15.22:0/1088496679 wait complete. 2024-03-28T18:19:52.329 INFO:tasks.cephfs.filesystem:scrub status for tag:4bbbd61d-29a3-4bf6-a4f2-d5783d585de3 - {'path': '/', 'tag': '4bbbd61d-29a3-4bf6-a4f2-d5783d585de3', 'options': 'recursive,force'} 2024-03-28T18:19:52.925 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:19:52 smithi134 ceph-mon[35037]: pgmap v403: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:19:52.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:19:52 smithi079 ceph-mon[35312]: pgmap v403: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:19:53.032 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:19:52 smithi022 ceph-mon[31985]: pgmap v403: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:19:54.925 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:19:54 smithi134 ceph-mon[35037]: pgmap v404: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:19:54.925 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:19:54 smithi134 ceph-mon[35037]: 1 slow requests, 1 included below; oldest blocked for > 181.841450 secs 2024-03-28T18:19:54.925 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:19:54 smithi134 ceph-mon[35037]: slow request 181.841450 seconds old, received at 2024-03-28T18:16:52.457506+0000: peer_request:client.35222:10732 currently acquired locks 2024-03-28T18:19:54.933 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:19:54 smithi079 ceph-mon[35312]: pgmap v404: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:19:54.933 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:19:54 smithi079 ceph-mon[35312]: 1 slow requests, 1 included below; oldest blocked for > 181.841450 secs 2024-03-28T18:19:54.933 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:19:54 smithi079 ceph-mon[35312]: slow request 181.841450 seconds old, received at 2024-03-28T18:16:52.457506+0000: peer_request:client.35222:10732 currently acquired locks 2024-03-28T18:19:55.032 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:19:54 smithi022 ceph-mon[31985]: pgmap v404: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:19:55.032 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:19:54 smithi022 ceph-mon[31985]: 1 slow requests, 1 included below; oldest blocked for > 181.841450 secs 2024-03-28T18:19:55.032 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:19:54 smithi022 ceph-mon[31985]: slow request 181.841450 seconds old, received at 2024-03-28T18:16:52.457506+0000: peer_request:client.35222:10732 currently acquired locks 2024-03-28T18:19:56.925 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:19:56 smithi134 ceph-mon[35037]: pgmap v405: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:19:56.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:19:56 smithi079 ceph-mon[35312]: pgmap v405: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:19:57.032 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:19:56 smithi022 ceph-mon[31985]: pgmap v405: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:19:58.835 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:19:58 smithi022 ceph-mon[31985]: pgmap v406: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:19:58.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:19:58 smithi134 ceph-mon[35037]: pgmap v406: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:19:58.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:19:58 smithi079 ceph-mon[35312]: pgmap v406: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:19:59.925 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:19:59 smithi134 ceph-mon[35037]: Health check failed: 1 MDSs report slow requests (MDS_SLOW_REQUEST) 2024-03-28T18:19:59.925 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:19:59 smithi134 ceph-mon[35037]: mds.? [v2:172.21.15.79:6838/2441542443,v1:172.21.15.79:6839/2441542443] up:active 2024-03-28T18:19:59.925 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:19:59 smithi134 ceph-mon[35037]: fsmap cephfs:5 4 up:standby-replay 5 up:active 2024-03-28T18:19:59.925 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:19:59 smithi134 ceph-mon[35037]: 1 slow requests, 0 included below; oldest blocked for > 186.841517 secs 2024-03-28T18:19:59.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:19:59 smithi079 ceph-mon[35312]: Health check failed: 1 MDSs report slow requests (MDS_SLOW_REQUEST) 2024-03-28T18:19:59.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:19:59 smithi079 ceph-mon[35312]: mds.? [v2:172.21.15.79:6838/2441542443,v1:172.21.15.79:6839/2441542443] up:active 2024-03-28T18:19:59.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:19:59 smithi079 ceph-mon[35312]: fsmap cephfs:5 4 up:standby-replay 5 up:active 2024-03-28T18:19:59.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:19:59 smithi079 ceph-mon[35312]: 1 slow requests, 0 included below; oldest blocked for > 186.841517 secs 2024-03-28T18:20:00.032 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:19:59 smithi022 ceph-mon[31985]: Health check failed: 1 MDSs report slow requests (MDS_SLOW_REQUEST) 2024-03-28T18:20:00.032 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:19:59 smithi022 ceph-mon[31985]: mds.? [v2:172.21.15.79:6838/2441542443,v1:172.21.15.79:6839/2441542443] up:active 2024-03-28T18:20:00.032 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:19:59 smithi022 ceph-mon[31985]: fsmap cephfs:5 4 up:standby-replay 5 up:active 2024-03-28T18:20:00.032 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:19:59 smithi022 ceph-mon[31985]: 1 slow requests, 0 included below; oldest blocked for > 186.841517 secs 2024-03-28T18:20:00.925 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:20:00 smithi134 ceph-mon[35037]: pgmap v407: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:20:00.925 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:20:00 smithi134 ceph-mon[35037]: Health detail: HEALTH_WARN 4 failed cephadm daemon(s); 1 MDSs report slow requests 2024-03-28T18:20:00.926 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:20:00 smithi134 ceph-mon[35037]: [WRN] CEPHADM_FAILED_DAEMON: 4 failed cephadm daemon(s) 2024-03-28T18:20:00.926 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:20:00 smithi134 ceph-mon[35037]: daemon mds.j on smithi022 is in error state 2024-03-28T18:20:00.926 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:20:00 smithi134 ceph-mon[35037]: daemon mds.d on smithi022 is in error state 2024-03-28T18:20:00.926 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:20:00 smithi134 ceph-mon[35037]: daemon mds.h on smithi079 is in error state 2024-03-28T18:20:00.926 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:20:00 smithi134 ceph-mon[35037]: daemon mds.f on smithi134 is in error state 2024-03-28T18:20:00.926 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:20:00 smithi134 ceph-mon[35037]: [WRN] MDS_SLOW_REQUEST: 1 MDSs report slow requests 2024-03-28T18:20:00.926 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:20:00 smithi134 ceph-mon[35037]: mds.b(mds.0): 1 slow requests are blocked > 180 secs 2024-03-28T18:20:00.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:20:00 smithi079 ceph-mon[35312]: pgmap v407: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:20:00.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:20:00 smithi079 ceph-mon[35312]: Health detail: HEALTH_WARN 4 failed cephadm daemon(s); 1 MDSs report slow requests 2024-03-28T18:20:00.933 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:20:00 smithi079 ceph-mon[35312]: [WRN] CEPHADM_FAILED_DAEMON: 4 failed cephadm daemon(s) 2024-03-28T18:20:00.933 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:20:00 smithi079 ceph-mon[35312]: daemon mds.j on smithi022 is in error state 2024-03-28T18:20:00.933 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:20:00 smithi079 ceph-mon[35312]: daemon mds.d on smithi022 is in error state 2024-03-28T18:20:00.933 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:20:00 smithi079 ceph-mon[35312]: daemon mds.h on smithi079 is in error state 2024-03-28T18:20:00.933 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:20:00 smithi079 ceph-mon[35312]: daemon mds.f on smithi134 is in error state 2024-03-28T18:20:00.933 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:20:00 smithi079 ceph-mon[35312]: [WRN] MDS_SLOW_REQUEST: 1 MDSs report slow requests 2024-03-28T18:20:00.933 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:20:00 smithi079 ceph-mon[35312]: mds.b(mds.0): 1 slow requests are blocked > 180 secs 2024-03-28T18:20:01.032 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:20:00 smithi022 ceph-mon[31985]: pgmap v407: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:20:01.032 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:20:00 smithi022 ceph-mon[31985]: Health detail: HEALTH_WARN 4 failed cephadm daemon(s); 1 MDSs report slow requests 2024-03-28T18:20:01.032 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:20:00 smithi022 ceph-mon[31985]: [WRN] CEPHADM_FAILED_DAEMON: 4 failed cephadm daemon(s) 2024-03-28T18:20:01.032 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:20:00 smithi022 ceph-mon[31985]: daemon mds.j on smithi022 is in error state 2024-03-28T18:20:01.032 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:20:00 smithi022 ceph-mon[31985]: daemon mds.d on smithi022 is in error state 2024-03-28T18:20:01.032 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:20:00 smithi022 ceph-mon[31985]: daemon mds.h on smithi079 is in error state 2024-03-28T18:20:01.032 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:20:00 smithi022 ceph-mon[31985]: daemon mds.f on smithi134 is in error state 2024-03-28T18:20:01.032 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:20:00 smithi022 ceph-mon[31985]: [WRN] MDS_SLOW_REQUEST: 1 MDSs report slow requests 2024-03-28T18:20:01.032 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:20:00 smithi022 ceph-mon[31985]: mds.b(mds.0): 1 slow requests are blocked > 180 secs 2024-03-28T18:20:02.925 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:20:02 smithi134 ceph-mon[35037]: pgmap v408: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:20:02.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:20:02 smithi079 ceph-mon[35312]: pgmap v408: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:20:03.032 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:20:02 smithi022 ceph-mon[31985]: pgmap v408: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:20:04.925 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:20:04 smithi134 ceph-mon[35037]: pgmap v409: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:20:04.925 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:20:04 smithi134 ceph-mon[35037]: 2 slow requests, 1 included below; oldest blocked for > 191.841582 secs 2024-03-28T18:20:04.925 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:20:04 smithi134 ceph-mon[35037]: slow request 184.143958 seconds old, received at 2024-03-28T18:17:00.155130+0000: client_request(client.35612:96 mksnap #0x1//scheduled-2024-03-28-18_17_00_UTC 2024-03-28T18:17:00.154942+0000 caller_uid=0, caller_gid=0{}) currently failed to xlock, waiting 2024-03-28T18:20:04.933 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:20:04 smithi079 ceph-mon[35312]: pgmap v409: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:20:04.933 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:20:04 smithi079 ceph-mon[35312]: 2 slow requests, 1 included below; oldest blocked for > 191.841582 secs 2024-03-28T18:20:04.933 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:20:04 smithi079 ceph-mon[35312]: slow request 184.143958 seconds old, received at 2024-03-28T18:17:00.155130+0000: client_request(client.35612:96 mksnap #0x1//scheduled-2024-03-28-18_17_00_UTC 2024-03-28T18:17:00.154942+0000 caller_uid=0, caller_gid=0{}) currently failed to xlock, waiting 2024-03-28T18:20:05.032 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:20:04 smithi022 ceph-mon[31985]: pgmap v409: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:20:05.032 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:20:04 smithi022 ceph-mon[31985]: 2 slow requests, 1 included below; oldest blocked for > 191.841582 secs 2024-03-28T18:20:05.032 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:20:04 smithi022 ceph-mon[31985]: slow request 184.143958 seconds old, received at 2024-03-28T18:17:00.155130+0000: client_request(client.35612:96 mksnap #0x1//scheduled-2024-03-28-18_17_00_UTC 2024-03-28T18:17:00.154942+0000 caller_uid=0, caller_gid=0{}) currently failed to xlock, waiting 2024-03-28T18:20:06.925 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:20:06 smithi134 ceph-mon[35037]: pgmap v410: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:20:06.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:20:06 smithi079 ceph-mon[35312]: pgmap v410: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:20:07.032 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:20:06 smithi022 ceph-mon[31985]: pgmap v410: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:20:07.925 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:20:07 smithi134 ceph-mon[35037]: mds.? [v2:172.21.15.79:6838/2441542443,v1:172.21.15.79:6839/2441542443] up:active 2024-03-28T18:20:07.925 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:20:07 smithi134 ceph-mon[35037]: fsmap cephfs:5 4 up:standby-replay 5 up:active 2024-03-28T18:20:07.933 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:20:07 smithi079 ceph-mon[35312]: mds.? [v2:172.21.15.79:6838/2441542443,v1:172.21.15.79:6839/2441542443] up:active 2024-03-28T18:20:07.933 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:20:07 smithi079 ceph-mon[35312]: fsmap cephfs:5 4 up:standby-replay 5 up:active 2024-03-28T18:20:08.032 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:20:07 smithi022 ceph-mon[31985]: mds.? [v2:172.21.15.79:6838/2441542443,v1:172.21.15.79:6839/2441542443] up:active 2024-03-28T18:20:08.032 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:20:07 smithi022 ceph-mon[31985]: fsmap cephfs:5 4 up:standby-replay 5 up:active 2024-03-28T18:20:08.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:20:08 smithi134 ceph-mon[35037]: pgmap v411: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:20:08.925 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:20:08 smithi134 ceph-mon[35037]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T18:20:08.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:20:08 smithi079 ceph-mon[35312]: pgmap v411: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:20:08.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:20:08 smithi079 ceph-mon[35312]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T18:20:09.032 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:20:08 smithi022 ceph-mon[31985]: pgmap v411: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:20:09.032 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:20:08 smithi022 ceph-mon[31985]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T18:20:09.677 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:20:09 smithi022 ceph-mon[31985]: 2 slow requests, 0 included below; oldest blocked for > 196.841658 secs 2024-03-28T18:20:09.677 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:20:09 smithi022 ceph-mon[31985]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T18:20:09.925 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:20:09 smithi134 ceph-mon[35037]: 2 slow requests, 0 included below; oldest blocked for > 196.841658 secs 2024-03-28T18:20:09.946 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:20:09 smithi134 ceph-mon[35037]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T18:20:09.946 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:20:09 smithi134 ceph-mon[35037]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T18:20:09.946 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:20:09 smithi134 ceph-mon[35037]: from='mgr.15522 ' entity='mgr.x' 2024-03-28T18:20:09.947 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:20:09 smithi079 ceph-mon[35312]: 2 slow requests, 0 included below; oldest blocked for > 196.841658 secs 2024-03-28T18:20:09.947 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:20:09 smithi079 ceph-mon[35312]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T18:20:09.947 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:20:09 smithi079 ceph-mon[35312]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T18:20:09.947 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:20:09 smithi079 ceph-mon[35312]: from='mgr.15522 ' entity='mgr.x' 2024-03-28T18:20:10.032 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:20:09 smithi022 ceph-mon[31985]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T18:20:10.032 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:20:09 smithi022 ceph-mon[31985]: from='mgr.15522 ' entity='mgr.x' 2024-03-28T18:20:10.925 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:20:10 smithi134 ceph-mon[35037]: pgmap v412: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 10 op/s 2024-03-28T18:20:10.925 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:20:10 smithi134 ceph-mon[35037]: pgmap v413: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 8 op/s 2024-03-28T18:20:10.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:20:10 smithi079 ceph-mon[35312]: pgmap v412: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 10 op/s 2024-03-28T18:20:10.933 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:20:10 smithi079 ceph-mon[35312]: pgmap v413: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 8 op/s 2024-03-28T18:20:11.032 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:20:10 smithi022 ceph-mon[31985]: pgmap v412: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 10 op/s 2024-03-28T18:20:11.032 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:20:10 smithi022 ceph-mon[31985]: pgmap v413: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 8 op/s 2024-03-28T18:20:12.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:20:12 smithi134 ceph-mon[35037]: pgmap v414: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 8 op/s 2024-03-28T18:20:12.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:20:12 smithi079 ceph-mon[35312]: pgmap v414: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 8 op/s 2024-03-28T18:20:13.032 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:20:12 smithi022 ceph-mon[31985]: pgmap v414: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 8 op/s 2024-03-28T18:20:14.925 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:20:14 smithi134 ceph-mon[35037]: pgmap v415: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 8 op/s 2024-03-28T18:20:14.925 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:20:14 smithi134 ceph-mon[35037]: 2 slow requests, 0 included below; oldest blocked for > 201.841694 secs 2024-03-28T18:20:14.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:20:14 smithi079 ceph-mon[35312]: pgmap v415: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 8 op/s 2024-03-28T18:20:14.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:20:14 smithi079 ceph-mon[35312]: 2 slow requests, 0 included below; oldest blocked for > 201.841694 secs 2024-03-28T18:20:15.032 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:20:14 smithi022 ceph-mon[31985]: pgmap v415: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 8 op/s 2024-03-28T18:20:15.032 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:20:14 smithi022 ceph-mon[31985]: 2 slow requests, 0 included below; oldest blocked for > 201.841694 secs 2024-03-28T18:20:16.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:20:16 smithi079 ceph-mon[35312]: pgmap v416: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 8 op/s 2024-03-28T18:20:17.032 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:20:16 smithi022 ceph-mon[31985]: pgmap v416: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 8 op/s 2024-03-28T18:20:17.175 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:20:16 smithi134 ceph-mon[35037]: pgmap v416: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 8 op/s 2024-03-28T18:20:19.032 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:20:18 smithi022 ceph-mon[31985]: pgmap v417: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 8 op/s 2024-03-28T18:20:19.175 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:20:18 smithi134 ceph-mon[35037]: pgmap v417: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 8 op/s 2024-03-28T18:20:19.182 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:20:18 smithi079 ceph-mon[35312]: pgmap v417: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 8 op/s 2024-03-28T18:20:20.032 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:20:19 smithi022 ceph-mon[31985]: 2 slow requests, 0 included below; oldest blocked for > 206.841814 secs 2024-03-28T18:20:20.175 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:20:19 smithi134 ceph-mon[35037]: 2 slow requests, 0 included below; oldest blocked for > 206.841814 secs 2024-03-28T18:20:20.182 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:20:19 smithi079 ceph-mon[35312]: 2 slow requests, 0 included below; oldest blocked for > 206.841814 secs 2024-03-28T18:20:21.032 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:20:20 smithi022 ceph-mon[31985]: pgmap v418: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 7 op/s 2024-03-28T18:20:21.174 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:20:20 smithi134 ceph-mon[35037]: pgmap v418: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 7 op/s 2024-03-28T18:20:21.182 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:20:20 smithi079 ceph-mon[35312]: pgmap v418: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 7 op/s 2024-03-28T18:20:22.331 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-28T18:20:22.331 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-28T18:20:22.569 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:20:22.569+0000 7f689c200640 1 -- 172.21.15.22:0/3245081448 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6894075bd0 msgr2=0x7f6894075fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:20:22.570 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:20:22.569+0000 7f689c200640 1 --2- 172.21.15.22:0/3245081448 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6894075bd0 0x7f6894075fb0 secure :-1 s=READY pgs=817 cs=0 l=1 rev1=1 crypto rx=0x7f6888009900 tx=0x7f688802bc50 comp rx=0 tx=0).stop 2024-03-28T18:20:22.570 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:20:22.570+0000 7f689c200640 1 -- 172.21.15.22:0/3245081448 shutdown_connections 2024-03-28T18:20:22.570 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:20:22.570+0000 7f689c200640 1 --2- 172.21.15.22:0/3245081448 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f68941124e0 0x7f68941148d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:20:22.570 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:20:22.570+0000 7f689c200640 1 --2- 172.21.15.22:0/3245081448 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f68940764f0 0x7f6894111fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:20:22.570 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:20:22.570+0000 7f689c200640 1 --2- 172.21.15.22:0/3245081448 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6894075bd0 0x7f6894075fb0 unknown :-1 s=CLOSED pgs=817 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:20:22.570 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:20:22.570+0000 7f689c200640 1 -- 172.21.15.22:0/3245081448 >> 172.21.15.22:0/3245081448 conn(0x7f68940ff2f0 msgr2=0x7f6894101710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:20:22.570 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:20:22.570+0000 7f689c200640 1 -- 172.21.15.22:0/3245081448 shutdown_connections 2024-03-28T18:20:22.570 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:20:22.570+0000 7f689c200640 1 -- 172.21.15.22:0/3245081448 wait complete. 2024-03-28T18:20:22.571 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:20:22.571+0000 7f689c200640 1 Processor -- start 2024-03-28T18:20:22.571 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:20:22.571+0000 7f689c200640 1 -- start start 2024-03-28T18:20:22.572 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:20:22.571+0000 7f689c200640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6894075bd0 0x7f689419e540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:20:22.572 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:20:22.571+0000 7f689c200640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f68940764f0 0x7f689419ea80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:20:22.572 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:20:22.571+0000 7f689c200640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f68941124e0 0x7f68941a2e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:20:22.572 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:20:22.571+0000 7f689c200640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6894118060 con 0x7f6894075bd0 2024-03-28T18:20:22.572 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:20:22.571+0000 7f689c200640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6894117ee0 con 0x7f68941124e0 2024-03-28T18:20:22.572 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:20:22.571+0000 7f689c200640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68941181e0 con 0x7f68940764f0 2024-03-28T18:20:22.572 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:20:22.571+0000 7f6899800640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6894075bd0 0x7f689419e540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:20:22.572 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:20:22.571+0000 7f6899800640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6894075bd0 0x7f689419e540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:46082/0 (socket says 172.21.15.22:46082) 2024-03-28T18:20:22.572 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:20:22.571+0000 7f6899800640 1 -- 172.21.15.22:0/3765626376 learned_addr learned my addr 172.21.15.22:0/3765626376 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-28T18:20:22.572 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:20:22.572+0000 7f689a200640 1 --2- 172.21.15.22:0/3765626376 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f68941124e0 0x7f68941a2e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:20:22.572 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:20:22.572+0000 7f6899800640 1 -- 172.21.15.22:0/3765626376 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f68940764f0 msgr2=0x7f689419ea80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:20:22.572 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:20:22.572+0000 7f6899800640 1 --2- 172.21.15.22:0/3765626376 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f68940764f0 0x7f689419ea80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:20:22.572 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:20:22.572+0000 7f6899800640 1 -- 172.21.15.22:0/3765626376 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f68941124e0 msgr2=0x7f68941a2e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:20:22.572 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:20:22.572+0000 7f6899800640 1 --2- 172.21.15.22:0/3765626376 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f68941124e0 0x7f68941a2e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:20:22.572 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:20:22.572+0000 7f6899800640 1 -- 172.21.15.22:0/3765626376 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f68941a3530 con 0x7f6894075bd0 2024-03-28T18:20:22.573 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:20:22.572+0000 7f6899800640 1 --2- 172.21.15.22:0/3765626376 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6894075bd0 0x7f689419e540 secure :-1 s=READY pgs=818 cs=0 l=1 rev1=1 crypto rx=0x7f6888000c00 tx=0x7f6888004430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:20:22.573 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:20:22.572+0000 7f6892000640 1 -- 172.21.15.22:0/3765626376 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6888043020 con 0x7f6894075bd0 2024-03-28T18:20:22.573 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:20:22.572+0000 7f6892000640 1 -- 172.21.15.22:0/3765626376 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f6888005490 con 0x7f6894075bd0 2024-03-28T18:20:22.573 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:20:22.572+0000 7f6892000640 1 -- 172.21.15.22:0/3765626376 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6888003bb0 con 0x7f6894075bd0 2024-03-28T18:20:22.573 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:20:22.572+0000 7f689c200640 1 -- 172.21.15.22:0/3765626376 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f68941a37c0 con 0x7f6894075bd0 2024-03-28T18:20:22.574 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:20:22.573+0000 7f689c200640 1 -- 172.21.15.22:0/3765626376 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f68941ab0b0 con 0x7f6894075bd0 2024-03-28T18:20:22.576 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:20:22.576+0000 7f6892000640 1 -- 172.21.15.22:0/3765626376 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 26) v1 ==== 137421+0+0 (secure 0 0 0) 0x7f6888046020 con 0x7f6894075bd0 2024-03-28T18:20:22.576 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:20:22.576+0000 7f689ee6c740 1 Processor -- start 2024-03-28T18:20:22.576 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:20:22.576+0000 7f689ee6c740 1 -- start start 2024-03-28T18:20:22.577 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:20:22.577+0000 7f689ee6c740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559cba753c00 0x559cba752950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:20:22.577 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:20:22.577+0000 7f689ee6c740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559cba752f20 0x559cba7533a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:20:22.577 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:20:22.577+0000 7f689ee6c740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x559cba75c290 0x559cba75e680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:20:22.577 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:20:22.577+0000 7f689ee6c740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x559cba6b7250 con 0x559cba752f20 2024-03-28T18:20:22.577 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:20:22.577+0000 7f689ee6c740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x559cba65fa30 con 0x559cba75c290 2024-03-28T18:20:22.577 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:20:22.577+0000 7f689ee6c740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x559cba6b32f0 con 0x559cba753c00 2024-03-28T18:20:22.577 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:20:22.577+0000 7f6899800640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559cba752f20 0x559cba7533a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:20:22.577 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:20:22.577+0000 7f6899800640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559cba752f20 0x559cba7533a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:46094/0 (socket says 172.21.15.22:46094) 2024-03-28T18:20:22.578 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:20:22.577+0000 7f6899800640 1 -- 172.21.15.22:0/420829563 learned_addr learned my addr 172.21.15.22:0/420829563 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-28T18:20:22.578 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:20:22.577+0000 7f6898e00640 1 --2- 172.21.15.22:0/420829563 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x559cba75c290 0x559cba75e680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:20:22.578 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:20:22.577+0000 7f689a200640 1 --2- 172.21.15.22:0/420829563 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559cba753c00 0x559cba752950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:20:22.579 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:20:22.578+0000 7f6899800640 1 -- 172.21.15.22:0/420829563 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559cba753c00 msgr2=0x559cba752950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T18:20:22.579 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:20:22.578+0000 7f6899800640 1 --2- 172.21.15.22:0/420829563 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559cba753c00 0x559cba752950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:20:22.579 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:20:22.578+0000 7f6899800640 1 -- 172.21.15.22:0/420829563 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x559cba75c290 msgr2=0x559cba75e680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T18:20:22.579 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:20:22.578+0000 7f6899800640 1 --2- 172.21.15.22:0/420829563 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x559cba75c290 0x559cba75e680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:20:22.579 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:20:22.578+0000 7f6899800640 1 -- 172.21.15.22:0/420829563 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559cba64a690 con 0x559cba752f20 2024-03-28T18:20:22.579 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:20:22.578+0000 7f6898e00640 1 --2- 172.21.15.22:0/420829563 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x559cba75c290 0x559cba75e680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-28T18:20:22.579 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:20:22.578+0000 7f6892000640 1 --2- 172.21.15.22:0/3765626376 >> [v2:172.21.15.22:6800/1357365082,v1:172.21.15.22:6801/1357365082] conn(0x7f687c0a29f0 0x7f687c0a4eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:20:22.579 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:20:22.578+0000 7f6892000640 1 -- 172.21.15.22:0/3765626376 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(122..122 src has 1..122) v4 ==== 10389+0+0 (secure 0 0 0) 0x7f68880f15e0 con 0x7f6894075bd0 2024-03-28T18:20:22.579 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:20:22.578+0000 7f689a200640 1 --2- 172.21.15.22:0/3765626376 >> [v2:172.21.15.22:6800/1357365082,v1:172.21.15.22:6801/1357365082] conn(0x7f687c0a29f0 0x7f687c0a4eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:20:22.579 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:20:22.578+0000 7f6899800640 1 --2- 172.21.15.22:0/420829563 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559cba752f20 0x559cba7533a0 secure :-1 s=READY pgs=819 cs=0 l=1 rev1=1 crypto rx=0x7f68880ee5d0 tx=0x7f68880f7650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a149aebbd3b92754 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:20:22.580 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:20:22.579+0000 7f6886a00640 1 -- 172.21.15.22:0/420829563 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68880b9840 con 0x559cba752f20 2024-03-28T18:20:22.580 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:20:22.579+0000 7f6886a00640 1 -- 172.21.15.22:0/420829563 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f6888004c70 con 0x559cba752f20 2024-03-28T18:20:22.580 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:20:22.579+0000 7f6886a00640 1 -- 172.21.15.22:0/420829563 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f688802cbb0 con 0x559cba752f20 2024-03-28T18:20:22.580 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:20:22.579+0000 7f689ee6c740 1 -- 172.21.15.22:0/420829563 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559cba752f20 msgr2=0x559cba7533a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:20:22.580 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:20:22.579+0000 7f689ee6c740 1 --2- 172.21.15.22:0/420829563 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559cba752f20 0x559cba7533a0 secure :-1 s=READY pgs=819 cs=0 l=1 rev1=1 crypto rx=0x7f68880ee5d0 tx=0x7f68880f7650 comp rx=0 tx=0).stop 2024-03-28T18:20:22.580 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:20:22.579+0000 7f689a200640 1 --2- 172.21.15.22:0/3765626376 >> [v2:172.21.15.22:6800/1357365082,v1:172.21.15.22:6801/1357365082] conn(0x7f687c0a29f0 0x7f687c0a4eb0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f689406aa60 tx=0x7f688c007520 comp rx=0 tx=0).ready entity=mgr.15522 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:20:22.583 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:20:22.580+0000 7f689ee6c740 1 -- 172.21.15.22:0/420829563 shutdown_connections 2024-03-28T18:20:22.583 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:20:22.580+0000 7f689ee6c740 1 --2- 172.21.15.22:0/420829563 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x559cba75c290 0x559cba75e680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:20:22.583 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:20:22.580+0000 7f689ee6c740 1 --2- 172.21.15.22:0/420829563 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559cba752f20 0x559cba7533a0 unknown :-1 s=CLOSED pgs=819 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:20:22.583 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:20:22.580+0000 7f689ee6c740 1 --2- 172.21.15.22:0/420829563 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559cba753c00 0x559cba752950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:20:22.583 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:20:22.580+0000 7f689ee6c740 1 -- 172.21.15.22:0/420829563 >> 172.21.15.22:0/420829563 conn(0x559cba71a200 msgr2=0x559cba74d970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:20:22.583 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:20:22.580+0000 7f689ee6c740 1 -- 172.21.15.22:0/420829563 shutdown_connections 2024-03-28T18:20:22.584 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:20:22.580+0000 7f689ee6c740 1 -- 172.21.15.22:0/420829563 wait complete. 2024-03-28T18:20:22.584 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:20:22.580+0000 7f689ee6c740 1 -- start start 2024-03-28T18:20:22.584 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:20:22.580+0000 7f689ee6c740 1 Processor -- start 2024-03-28T18:20:22.584 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:20:22.581+0000 7f689ee6c740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x559cba752f20 0x559cba75b910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:20:22.584 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:20:22.581+0000 7f689ee6c740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559cba753c00 0x559cba758ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:20:22.584 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:20:22.581+0000 7f689ee6c740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559cba75c290 0x559cba7591e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:20:22.584 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:20:22.581+0000 7f689ee6c740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x559cba75eec0 con 0x559cba753c00 2024-03-28T18:20:22.584 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:20:22.581+0000 7f689ee6c740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x559cba672e80 con 0x559cba752f20 2024-03-28T18:20:22.584 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:20:22.581+0000 7f689ee6c740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x559cba75f040 con 0x559cba75c290 2024-03-28T18:20:22.584 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:20:22.581+0000 7f689ee6c740 10 client.0 _ll_register_callbacks cb 0x559cba685130 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-28T18:20:22.584 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:20:22.581+0000 7f6898e00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559cba753c00 0x559cba758ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:20:22.584 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:20:22.581+0000 7f6898e00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559cba753c00 0x559cba758ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:46102/0 (socket says 172.21.15.22:46102) 2024-03-28T18:20:22.584 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:20:22.581+0000 7f6898e00640 1 -- 172.21.15.22:0/2064452326 learned_addr learned my addr 172.21.15.22:0/2064452326 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-28T18:20:22.584 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:20:22.581+0000 7f6899800640 1 --2- 172.21.15.22:0/2064452326 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x559cba752f20 0x559cba75b910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:20:22.584 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:20:22.581+0000 7f689a200640 1 --2- 172.21.15.22:0/2064452326 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559cba75c290 0x559cba7591e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:20:22.585 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:20:22.582+0000 7f6898e00640 1 -- 172.21.15.22:0/2064452326 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559cba75c290 msgr2=0x559cba7591e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T18:20:22.585 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:20:22.582+0000 7f6898e00640 1 --2- 172.21.15.22:0/2064452326 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559cba75c290 0x559cba7591e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:20:22.585 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:20:22.582+0000 7f6898e00640 1 -- 172.21.15.22:0/2064452326 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x559cba752f20 msgr2=0x559cba75b910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T18:20:22.585 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:20:22.582+0000 7f6898e00640 1 --2- 172.21.15.22:0/2064452326 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x559cba752f20 0x559cba75b910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:20:22.585 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:20:22.582+0000 7f6898e00640 1 -- 172.21.15.22:0/2064452326 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x559cba4957d0 con 0x559cba753c00 2024-03-28T18:20:22.585 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:20:22.582+0000 7f6898e00640 1 --2- 172.21.15.22:0/2064452326 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559cba753c00 0x559cba758ca0 secure :-1 s=READY pgs=820 cs=0 l=1 rev1=1 crypto rx=0x7f68800038e0 tx=0x7f6880003da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=69ae863edb26bfdc server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:20:22.585 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:20:22.582+0000 7f687b400640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-03-28T18:20:22.585 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:20:22.582+0000 7f687b400640 1 -- 172.21.15.22:0/2064452326 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f688001a670 con 0x559cba753c00 2024-03-28T18:20:22.585 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:20:22.582+0000 7f687b400640 1 -- 172.21.15.22:0/2064452326 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(122..122 src has 1..122) v4 ==== 10389+0+0 (secure 0 0 0) 0x7f6880011440 con 0x559cba753c00 2024-03-28T18:20:22.585 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:20:22.582+0000 7f689ee6c740 1 -- 172.21.15.22:0/2064452326 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f6888004c70 con 0x559cba753c00 2024-03-28T18:20:22.585 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:20:22.583+0000 7f687b400640 1 -- 172.21.15.22:0/2064452326 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=189) v2 ==== 24+0+0 (secure 0 0 0) 0x7f688000be20 con 0x559cba753c00 2024-03-28T18:20:22.585 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:20:22.583+0000 7f689ee6c740 10 client.16926 fetch_fsmap learned FSMap version 189 2024-03-28T18:20:22.585 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:20:22.583+0000 7f689ee6c740 1 -- 172.21.15.22:0/2064452326 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=189,monmap=4+}) v3 -- 0x559cba3cb3f0 con 0x559cba753c00 2024-03-28T18:20:22.585 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:20:22.583+0000 7f687b400640 1 -- 172.21.15.22:0/2064452326 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f6888004c70 con 0x559cba753c00 2024-03-28T18:20:22.585 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:20:22.583+0000 7f687b400640 1 -- 172.21.15.22:0/2064452326 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 189) v1 ==== 6221+0+0 (secure 0 0 0) 0x7f6880010440 con 0x559cba753c00 2024-03-28T18:20:22.585 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:20:22.583+0000 7f689ee6c740 10 client.16926 fetch_fsmap finished waiting for FSMap version 189 2024-03-28T18:20:22.585 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:20:22.583+0000 7f689ee6c740 10 client.16926 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-28T18:20:22.586 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:20:22.583+0000 7f689ee6c740 20 client.16926 populate_metadata read hostname 'smithi022' 2024-03-28T18:20:22.586 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:20:22.583+0000 7f689ee6c740 1 --2- 172.21.15.22:0/2064452326 >> [v2:172.21.15.79:6838/2441542443,v1:172.21.15.79:6839/2441542443] conn(0x559cba76bfb0 0x559cba76e3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:20:22.586 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:20:22.583+0000 7f689ee6c740 4 client.16926 mds_command: new command op to 34553 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-28T18:20:22.586 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:20:22.583+0000 7f689ee6c740 1 -- 172.21.15.22:0/2064452326 --> [v2:172.21.15.79:6838/2441542443,v1:172.21.15.79:6839/2441542443] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x559cba6c4790 con 0x559cba76bfb0 2024-03-28T18:20:22.586 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:20:22.584+0000 7f6899800640 1 --2- 172.21.15.22:0/2064452326 >> [v2:172.21.15.79:6838/2441542443,v1:172.21.15.79:6839/2441542443] conn(0x559cba76bfb0 0x559cba76e3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:20:22.587 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:20:22.584+0000 7f6899800640 1 --2- 172.21.15.22:0/2064452326 >> [v2:172.21.15.79:6838/2441542443,v1:172.21.15.79:6839/2441542443] conn(0x559cba76bfb0 0x559cba76e3f0 crc :-1 s=READY pgs=313 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b20bb94ef739c1a8 server_cookie=62a8946248aab7c5 in_seq=0 out_seq=0 2024-03-28T18:20:22.587 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:20:22.584+0000 7f687b400640 10 client.16926 ms_handle_connect on v2:172.21.15.79:6838/2441542443 2024-03-28T18:20:22.587 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:20:22.586+0000 7f687b400640 1 -- 172.21.15.22:0/2064452326 <== mds.0 v2:172.21.15.79:6838/2441542443 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x559cba6c4790 con 0x559cba76bfb0 2024-03-28T18:20:22.587 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:20:22.586+0000 7f687b400640 10 client.16926 handle_command_reply: tid=0 2024-03-28T18:20:22.587 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:20:22.586+0000 7f689ee6c740 1 client.16926 shutdown 2024-03-28T18:20:22.587 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:20:22.586+0000 7f689ee6c740 1 -- 172.21.15.22:0/2064452326 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559cba753c00 msgr2=0x559cba758ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:20:22.587 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:20:22.586+0000 7f689ee6c740 1 --2- 172.21.15.22:0/2064452326 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559cba753c00 0x559cba758ca0 secure :-1 s=READY pgs=820 cs=0 l=1 rev1=1 crypto rx=0x7f68800038e0 tx=0x7f6880003da0 comp rx=0 tx=0).stop 2024-03-28T18:20:22.588 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:20:22.586+0000 7f689ee6c740 1 -- 172.21.15.22:0/2064452326 shutdown_connections 2024-03-28T18:20:22.588 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:20:22.586+0000 7f689ee6c740 1 --2- 172.21.15.22:0/2064452326 >> [v2:172.21.15.79:6838/2441542443,v1:172.21.15.79:6839/2441542443] conn(0x559cba76bfb0 0x559cba76e3f0 crc :-1 s=READY pgs=313 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:20:22.588 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:20:22.586+0000 7f689ee6c740 1 --2- 172.21.15.22:0/2064452326 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559cba75c290 0x559cba7591e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:20:22.588 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:20:22.586+0000 7f689ee6c740 1 --2- 172.21.15.22:0/2064452326 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559cba753c00 0x559cba758ca0 unknown :-1 s=CLOSED pgs=820 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:20:22.588 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:20:22.586+0000 7f689ee6c740 1 --2- 172.21.15.22:0/2064452326 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x559cba752f20 0x559cba75b910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:20:22.588 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:20:22.586+0000 7f689ee6c740 1 -- 172.21.15.22:0/2064452326 >> 172.21.15.22:0/2064452326 conn(0x559cba71a200 msgr2=0x559cba74f6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:20:22.588 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:20:22.586+0000 7f687b400640 0 client.16926 ms_handle_reset on v2:172.21.15.79:6838/2441542443 2024-03-28T18:20:22.588 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:20:22.587+0000 7f689ee6c740 1 -- 172.21.15.22:0/2064452326 shutdown_connections 2024-03-28T18:20:22.588 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:20:22.587+0000 7f689ee6c740 1 -- 172.21.15.22:0/2064452326 wait complete. 2024-03-28T18:20:22.588 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:20:22.587+0000 7f689ee6c740 20 client.16926 trim_cache size 0 max 16384 2024-03-28T18:20:22.604 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:20:22.605+0000 7f689ee6c740 1 Processor -- start 2024-03-28T18:20:22.604 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:20:22.605+0000 7f689ee6c740 1 -- start start 2024-03-28T18:20:22.605 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:20:22.605+0000 7f689ee6c740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559cba752f20 0x559cba75b3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:20:22.605 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:20:22.605+0000 7f689ee6c740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x559cba753c00 0x559cba75b910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:20:22.606 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:20:22.605+0000 7f689ee6c740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559cba75c290 0x559cba75d680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:20:22.606 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:20:22.605+0000 7f689ee6c740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68941b6250 con 0x559cba75c290 2024-03-28T18:20:22.606 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:20:22.605+0000 7f689ee6c740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68941b35b0 con 0x559cba753c00 2024-03-28T18:20:22.606 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:20:22.605+0000 7f689ee6c740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68941b5590 con 0x559cba752f20 2024-03-28T18:20:22.606 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:20:22.606+0000 7f689a200640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559cba75c290 0x559cba75d680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:20:22.606 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:20:22.606+0000 7f6898e00640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x559cba753c00 0x559cba75b910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:20:22.606 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:20:22.606+0000 7f6898e00640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x559cba753c00 0x559cba75b910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.22:57210/0 (socket says 172.21.15.22:57210) 2024-03-28T18:20:22.606 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:20:22.606+0000 7f689a200640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559cba75c290 0x559cba75d680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:46118/0 (socket says 172.21.15.22:46118) 2024-03-28T18:20:22.606 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:20:22.606+0000 7f6898e00640 1 -- 172.21.15.22:0/3504064869 learned_addr learned my addr 172.21.15.22:0/3504064869 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-28T18:20:22.606 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:20:22.606+0000 7f6899800640 1 --2- 172.21.15.22:0/3504064869 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559cba752f20 0x559cba75b3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:20:22.608 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:20:22.606+0000 7f689a200640 1 -- 172.21.15.22:0/3504064869 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559cba752f20 msgr2=0x559cba75b3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T18:20:22.608 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:20:22.606+0000 7f689a200640 1 --2- 172.21.15.22:0/3504064869 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559cba752f20 0x559cba75b3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:20:22.608 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:20:22.606+0000 7f689a200640 1 -- 172.21.15.22:0/3504064869 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x559cba753c00 msgr2=0x559cba75b910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T18:20:22.608 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:20:22.606+0000 7f689a200640 1 --2- 172.21.15.22:0/3504064869 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x559cba753c00 0x559cba75b910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:20:22.608 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:20:22.606+0000 7f689a200640 1 -- 172.21.15.22:0/3504064869 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559cba38b0f0 con 0x559cba75c290 2024-03-28T18:20:22.608 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:20:22.606+0000 7f6898e00640 1 --2- 172.21.15.22:0/3504064869 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x559cba753c00 0x559cba75b910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-28T18:20:22.608 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:20:22.607+0000 7f689a200640 1 --2- 172.21.15.22:0/3504064869 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559cba75c290 0x559cba75d680 secure :-1 s=READY pgs=821 cs=0 l=1 rev1=1 crypto rx=0x7f688c007fd0 tx=0x7f688c014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d2e5b45481b0c6be server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:20:22.608 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:20:22.607+0000 7f6886000640 1 -- 172.21.15.22:0/3504064869 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f688c004b20 con 0x559cba75c290 2024-03-28T18:20:22.608 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:20:22.607+0000 7f6886000640 1 -- 172.21.15.22:0/3504064869 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f688c003e40 con 0x559cba75c290 2024-03-28T18:20:22.608 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:20:22.607+0000 7f6886000640 1 -- 172.21.15.22:0/3504064869 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f688c005020 con 0x559cba75c290 2024-03-28T18:20:22.608 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:20:22.608+0000 7f689ee6c740 1 -- 172.21.15.22:0/3504064869 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559cba75c290 msgr2=0x559cba75d680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:20:22.608 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:20:22.608+0000 7f689ee6c740 1 --2- 172.21.15.22:0/3504064869 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559cba75c290 0x559cba75d680 secure :-1 s=READY pgs=821 cs=0 l=1 rev1=1 crypto rx=0x7f688c007fd0 tx=0x7f688c014000 comp rx=0 tx=0).stop 2024-03-28T18:20:22.608 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:20:22.608+0000 7f689ee6c740 1 -- 172.21.15.22:0/3504064869 shutdown_connections 2024-03-28T18:20:22.608 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:20:22.608+0000 7f689ee6c740 1 --2- 172.21.15.22:0/3504064869 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559cba75c290 0x559cba75d680 unknown :-1 s=CLOSED pgs=821 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:20:22.608 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:20:22.608+0000 7f689ee6c740 1 --2- 172.21.15.22:0/3504064869 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x559cba753c00 0x559cba75b910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:20:22.609 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:20:22.608+0000 7f689ee6c740 1 --2- 172.21.15.22:0/3504064869 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559cba752f20 0x559cba75b3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:20:22.609 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:20:22.608+0000 7f689ee6c740 1 -- 172.21.15.22:0/3504064869 >> 172.21.15.22:0/3504064869 conn(0x559cba71a200 msgr2=0x559cba7575b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:20:22.609 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:20:22.608+0000 7f689ee6c740 1 -- 172.21.15.22:0/3504064869 shutdown_connections 2024-03-28T18:20:22.609 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:20:22.608+0000 7f689ee6c740 1 -- 172.21.15.22:0/3504064869 wait complete. 2024-03-28T18:20:22.611 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:20:22.608+0000 7f689ee6c740 1 -- start start 2024-03-28T18:20:22.612 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:20:22.609+0000 7f689ee6c740 1 Processor -- start 2024-03-28T18:20:22.612 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:20:22.609+0000 7f689ee6c740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559cba752f20 0x559cba755be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:20:22.612 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:20:22.609+0000 7f689ee6c740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x559cba753c00 0x559cba756120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:20:22.612 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:20:22.609+0000 7f689ee6c740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559cba75c290 0x559cba7c0890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:20:22.612 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:20:22.609+0000 7f689ee6c740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68941b4f30 con 0x559cba752f20 2024-03-28T18:20:22.612 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:20:22.609+0000 7f689ee6c740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68941b3c10 con 0x559cba753c00 2024-03-28T18:20:22.612 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:20:22.609+0000 7f689ee6c740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68941b4270 con 0x559cba75c290 2024-03-28T18:20:22.612 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:20:22.609+0000 7f689ee6c740 10 client.0 _ll_register_callbacks cb 0x7f68800041a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-28T18:20:22.612 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:20:22.609+0000 7f6899800640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559cba752f20 0x559cba755be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:20:22.612 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:20:22.610+0000 7f6899800640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559cba752f20 0x559cba755be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:46120/0 (socket says 172.21.15.22:46120) 2024-03-28T18:20:22.612 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:20:22.610+0000 7f6899800640 1 -- 172.21.15.22:0/2814536336 learned_addr learned my addr 172.21.15.22:0/2814536336 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-28T18:20:22.612 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:20:22.610+0000 7f6898e00640 1 --2- 172.21.15.22:0/2814536336 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x559cba753c00 0x559cba756120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:20:22.612 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:20:22.610+0000 7f689a200640 1 --2- 172.21.15.22:0/2814536336 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559cba75c290 0x559cba7c0890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:20:22.612 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:20:22.610+0000 7f6899800640 1 -- 172.21.15.22:0/2814536336 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559cba75c290 msgr2=0x559cba7c0890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T18:20:22.612 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:20:22.610+0000 7f6899800640 1 --2- 172.21.15.22:0/2814536336 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559cba75c290 0x559cba7c0890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:20:22.612 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:20:22.610+0000 7f6899800640 1 -- 172.21.15.22:0/2814536336 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x559cba753c00 msgr2=0x559cba756120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T18:20:22.613 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:20:22.610+0000 7f6899800640 1 --2- 172.21.15.22:0/2814536336 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x559cba753c00 0x559cba756120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:20:22.613 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:20:22.610+0000 7f6899800640 1 -- 172.21.15.22:0/2814536336 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x559cba7c0ed0 con 0x559cba752f20 2024-03-28T18:20:22.613 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:20:22.610+0000 7f689a200640 1 --2- 172.21.15.22:0/2814536336 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559cba75c290 0x559cba7c0890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T18:20:22.613 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:20:22.610+0000 7f6899800640 1 --2- 172.21.15.22:0/2814536336 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559cba752f20 0x559cba755be0 secure :-1 s=READY pgs=822 cs=0 l=1 rev1=1 crypto rx=0x7f688804a040 tx=0x7f68880f7110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c1693c794260bcdb server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:20:22.613 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:20:22.610+0000 7f687b400640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-03-28T18:20:22.613 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:20:22.610+0000 7f687b400640 1 -- 172.21.15.22:0/2814536336 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68880da020 con 0x559cba752f20 2024-03-28T18:20:22.613 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:20:22.611+0000 7f687b400640 1 -- 172.21.15.22:0/2814536336 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(122..122 src has 1..122) v4 ==== 10389+0+0 (secure 0 0 0) 0x7f68880bb8a0 con 0x559cba752f20 2024-03-28T18:20:22.613 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:20:22.611+0000 7f689ee6c740 1 -- 172.21.15.22:0/2814536336 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x559cba6beae0 con 0x559cba752f20 2024-03-28T18:20:22.613 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:20:22.611+0000 7f687b400640 1 -- 172.21.15.22:0/2814536336 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=189) v2 ==== 24+0+0 (secure 0 0 0) 0x7f68880f4e60 con 0x559cba752f20 2024-03-28T18:20:22.613 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:20:22.611+0000 7f689ee6c740 10 client.16932 fetch_fsmap learned FSMap version 189 2024-03-28T18:20:22.613 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:20:22.611+0000 7f689ee6c740 1 -- 172.21.15.22:0/2814536336 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=189,monmap=4+}) v3 -- 0x559cba7c8250 con 0x559cba752f20 2024-03-28T18:20:22.613 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:20:22.611+0000 7f687b400640 1 -- 172.21.15.22:0/2814536336 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x559cba6beae0 con 0x559cba752f20 2024-03-28T18:20:22.613 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:20:22.611+0000 7f687b400640 1 -- 172.21.15.22:0/2814536336 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 189) v1 ==== 6221+0+0 (secure 0 0 0) 0x7f68880b9c10 con 0x559cba752f20 2024-03-28T18:20:22.613 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:20:22.611+0000 7f689ee6c740 10 client.16932 fetch_fsmap finished waiting for FSMap version 189 2024-03-28T18:20:22.613 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:20:22.611+0000 7f689ee6c740 10 client.16932 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-28T18:20:22.613 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:20:22.611+0000 7f689ee6c740 20 client.16932 populate_metadata read hostname 'smithi022' 2024-03-28T18:20:22.613 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:20:22.611+0000 7f689ee6c740 1 --2- 172.21.15.22:0/2814536336 >> [v2:172.21.15.79:6838/2441542443,v1:172.21.15.79:6839/2441542443] conn(0x559cba7c85a0 0x559cba7ca990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:20:22.614 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:20:22.611+0000 7f689ee6c740 4 client.16932 mds_command: new command op to 34553 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-28T18:20:22.614 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:20:22.611+0000 7f689ee6c740 1 -- 172.21.15.22:0/2814536336 --> [v2:172.21.15.79:6838/2441542443,v1:172.21.15.79:6839/2441542443] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x559cba63c3a0 con 0x559cba7c85a0 2024-03-28T18:20:22.614 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:20:22.612+0000 7f6898e00640 1 --2- 172.21.15.22:0/2814536336 >> [v2:172.21.15.79:6838/2441542443,v1:172.21.15.79:6839/2441542443] conn(0x559cba7c85a0 0x559cba7ca990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:20:22.615 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:20:22.612+0000 7f6898e00640 1 --2- 172.21.15.22:0/2814536336 >> [v2:172.21.15.79:6838/2441542443,v1:172.21.15.79:6839/2441542443] conn(0x559cba7c85a0 0x559cba7ca990 crc :-1 s=READY pgs=314 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9aff12405414b2ad server_cookie=c627a369901794be in_seq=0 out_seq=0 2024-03-28T18:20:22.615 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:20:22.612+0000 7f687b400640 10 client.16932 ms_handle_connect on v2:172.21.15.79:6838/2441542443 2024-03-28T18:20:22.615 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:20:22.613+0000 7f687b400640 1 -- 172.21.15.22:0/2814536336 <== mds.0 v2:172.21.15.79:6838/2441542443 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x559cba63c3a0 con 0x559cba7c85a0 2024-03-28T18:20:22.615 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:20:22.613+0000 7f687b400640 10 client.16932 handle_command_reply: tid=0 2024-03-28T18:20:22.615 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:20:22.613+0000 7f689ee6c740 1 client.16932 shutdown 2024-03-28T18:20:22.615 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:20:22.613+0000 7f689ee6c740 1 -- 172.21.15.22:0/2814536336 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559cba752f20 msgr2=0x559cba755be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:20:22.615 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:20:22.613+0000 7f689ee6c740 1 --2- 172.21.15.22:0/2814536336 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559cba752f20 0x559cba755be0 secure :-1 s=READY pgs=822 cs=0 l=1 rev1=1 crypto rx=0x7f688804a040 tx=0x7f68880f7110 comp rx=0 tx=0).stop 2024-03-28T18:20:22.615 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:20:22.613+0000 7f689ee6c740 1 -- 172.21.15.22:0/2814536336 shutdown_connections 2024-03-28T18:20:22.615 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:20:22.613+0000 7f689ee6c740 1 --2- 172.21.15.22:0/2814536336 >> [v2:172.21.15.79:6838/2441542443,v1:172.21.15.79:6839/2441542443] conn(0x559cba7c85a0 0x559cba7ca990 crc :-1 s=READY pgs=314 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:20:22.615 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:20:22.613+0000 7f689ee6c740 1 --2- 172.21.15.22:0/2814536336 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559cba75c290 0x559cba7c0890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:20:22.615 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:20:22.613+0000 7f689ee6c740 1 --2- 172.21.15.22:0/2814536336 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x559cba753c00 0x559cba756120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:20:22.615 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:20:22.613+0000 7f689ee6c740 1 --2- 172.21.15.22:0/2814536336 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559cba752f20 0x559cba755be0 unknown :-1 s=CLOSED pgs=822 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:20:22.616 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:20:22.613+0000 7f687b400640 0 client.16932 ms_handle_reset on v2:172.21.15.79:6838/2441542443 2024-03-28T18:20:22.616 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:20:22.613+0000 7f689ee6c740 1 -- 172.21.15.22:0/2814536336 >> 172.21.15.22:0/2814536336 conn(0x559cba71a200 msgr2=0x559cba74c200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:20:22.616 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:20:22.613+0000 7f689ee6c740 1 -- 172.21.15.22:0/2814536336 shutdown_connections 2024-03-28T18:20:22.616 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:20:22.613+0000 7f689ee6c740 1 -- 172.21.15.22:0/2814536336 wait complete. 2024-03-28T18:20:22.616 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:20:22.613+0000 7f689ee6c740 20 client.16932 trim_cache size 0 max 16384 2024-03-28T18:20:22.616 INFO:teuthology.orchestra.run.smithi022.stdout:{ 2024-03-28T18:20:22.616 INFO:teuthology.orchestra.run.smithi022.stdout: "status": "IDLE (waiting for more scrubs)", 2024-03-28T18:20:22.616 INFO:teuthology.orchestra.run.smithi022.stdout: "scrubs": { 2024-03-28T18:20:22.616 INFO:teuthology.orchestra.run.smithi022.stdout: "4bbbd61d-29a3-4bf6-a4f2-d5783d585de3": { 2024-03-28T18:20:22.616 INFO:teuthology.orchestra.run.smithi022.stdout: "path": "/", 2024-03-28T18:20:22.616 INFO:teuthology.orchestra.run.smithi022.stdout: "tag": "4bbbd61d-29a3-4bf6-a4f2-d5783d585de3", 2024-03-28T18:20:22.616 INFO:teuthology.orchestra.run.smithi022.stdout: "options": "recursive,force" 2024-03-28T18:20:22.616 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-03-28T18:20:22.617 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-03-28T18:20:22.617 INFO:teuthology.orchestra.run.smithi022.stdout:} 2024-03-28T18:20:22.617 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:20:22.614+0000 7f689c200640 1 -- 172.21.15.22:0/3765626376 >> [v2:172.21.15.22:6800/1357365082,v1:172.21.15.22:6801/1357365082] conn(0x7f687c0a29f0 msgr2=0x7f687c0a4eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:20:22.617 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:20:22.614+0000 7f689c200640 1 --2- 172.21.15.22:0/3765626376 >> [v2:172.21.15.22:6800/1357365082,v1:172.21.15.22:6801/1357365082] conn(0x7f687c0a29f0 0x7f687c0a4eb0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f689406aa60 tx=0x7f688c007520 comp rx=0 tx=0).stop 2024-03-28T18:20:22.617 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:20:22.614+0000 7f689c200640 1 -- 172.21.15.22:0/3765626376 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6894075bd0 msgr2=0x7f689419e540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:20:22.617 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:20:22.614+0000 7f689c200640 1 --2- 172.21.15.22:0/3765626376 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6894075bd0 0x7f689419e540 secure :-1 s=READY pgs=818 cs=0 l=1 rev1=1 crypto rx=0x7f6888000c00 tx=0x7f6888004430 comp rx=0 tx=0).stop 2024-03-28T18:20:22.618 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:20:22.614+0000 7f689c200640 1 -- 172.21.15.22:0/3765626376 shutdown_connections 2024-03-28T18:20:22.618 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:20:22.614+0000 7f689c200640 1 --2- 172.21.15.22:0/3765626376 >> [v2:172.21.15.22:6800/1357365082,v1:172.21.15.22:6801/1357365082] conn(0x7f687c0a29f0 0x7f687c0a4eb0 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:20:22.618 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:20:22.614+0000 7f689c200640 1 --2- 172.21.15.22:0/3765626376 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f68941124e0 0x7f68941a2e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:20:22.618 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:20:22.614+0000 7f689c200640 1 --2- 172.21.15.22:0/3765626376 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f68940764f0 0x7f689419ea80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:20:22.618 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:20:22.614+0000 7f689c200640 1 --2- 172.21.15.22:0/3765626376 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6894075bd0 0x7f689419e540 unknown :-1 s=CLOSED pgs=818 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:20:22.618 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:20:22.614+0000 7f689c200640 1 -- 172.21.15.22:0/3765626376 >> 172.21.15.22:0/3765626376 conn(0x7f68940ff2f0 msgr2=0x7f68941016e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:20:22.618 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:20:22.614+0000 7f689c200640 1 -- 172.21.15.22:0/3765626376 shutdown_connections 2024-03-28T18:20:22.618 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:20:22.614+0000 7f689c200640 1 -- 172.21.15.22:0/3765626376 wait complete. 2024-03-28T18:20:22.634 INFO:tasks.cephfs.filesystem:scrub status for tag:4bbbd61d-29a3-4bf6-a4f2-d5783d585de3 - {'path': '/', 'tag': '4bbbd61d-29a3-4bf6-a4f2-d5783d585de3', 'options': 'recursive,force'} 2024-03-28T18:20:23.032 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:20:22 smithi022 ceph-mon[31985]: pgmap v419: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:20:23.174 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:20:22 smithi134 ceph-mon[35037]: pgmap v419: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:20:23.182 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:20:22 smithi079 ceph-mon[35312]: pgmap v419: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:20:25.032 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:20:24 smithi022 ceph-mon[31985]: pgmap v420: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:20:25.032 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:20:24 smithi022 ceph-mon[31985]: 2 slow requests, 0 included below; oldest blocked for > 211.841937 secs 2024-03-28T18:20:25.175 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:20:24 smithi134 ceph-mon[35037]: pgmap v420: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:20:25.175 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:20:24 smithi134 ceph-mon[35037]: 2 slow requests, 0 included below; oldest blocked for > 211.841937 secs 2024-03-28T18:20:25.182 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:20:24 smithi079 ceph-mon[35312]: pgmap v420: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:20:25.182 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:20:24 smithi079 ceph-mon[35312]: 2 slow requests, 0 included below; oldest blocked for > 211.841937 secs 2024-03-28T18:20:27.032 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:20:26 smithi022 ceph-mon[31985]: pgmap v421: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:20:27.175 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:20:26 smithi134 ceph-mon[35037]: pgmap v421: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:20:27.182 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:20:26 smithi079 ceph-mon[35312]: pgmap v421: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:20:29.032 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:20:28 smithi022 ceph-mon[31985]: pgmap v422: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:20:29.175 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:20:28 smithi134 ceph-mon[35037]: pgmap v422: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:20:29.182 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:20:28 smithi079 ceph-mon[35312]: pgmap v422: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:20:30.032 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:20:29 smithi022 ceph-mon[31985]: 2 slow requests, 0 included below; oldest blocked for > 216.842027 secs 2024-03-28T18:20:30.174 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:20:29 smithi134 ceph-mon[35037]: 2 slow requests, 0 included below; oldest blocked for > 216.842027 secs 2024-03-28T18:20:30.182 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:20:29 smithi079 ceph-mon[35312]: 2 slow requests, 0 included below; oldest blocked for > 216.842027 secs 2024-03-28T18:20:31.032 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:20:30 smithi022 ceph-mon[31985]: pgmap v423: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:20:31.174 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:20:30 smithi134 ceph-mon[35037]: pgmap v423: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:20:31.182 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:20:30 smithi079 ceph-mon[35312]: pgmap v423: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:20:33.000 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:20:32 smithi134 ceph-mon[35037]: pgmap v424: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:20:33.032 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:20:32 smithi022 ceph-mon[31985]: pgmap v424: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:20:33.183 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:20:32 smithi079 ceph-mon[35312]: pgmap v424: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:20:35.032 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:20:34 smithi022 ceph-mon[31985]: pgmap v425: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:20:35.032 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:20:34 smithi022 ceph-mon[31985]: 2 slow requests, 0 included below; oldest blocked for > 221.842125 secs 2024-03-28T18:20:35.175 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:20:34 smithi134 ceph-mon[35037]: pgmap v425: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:20:35.175 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:20:34 smithi134 ceph-mon[35037]: 2 slow requests, 0 included below; oldest blocked for > 221.842125 secs 2024-03-28T18:20:35.183 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:20:34 smithi079 ceph-mon[35312]: pgmap v425: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:20:35.183 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:20:34 smithi079 ceph-mon[35312]: 2 slow requests, 0 included below; oldest blocked for > 221.842125 secs 2024-03-28T18:20:37.032 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:20:36 smithi022 ceph-mon[31985]: pgmap v426: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:20:37.174 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:20:36 smithi134 ceph-mon[35037]: pgmap v426: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:20:37.182 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:20:36 smithi079 ceph-mon[35312]: pgmap v426: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:20:39.032 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:20:38 smithi022 ceph-mon[31985]: pgmap v427: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:20:39.174 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:20:38 smithi134 ceph-mon[35037]: pgmap v427: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:20:39.183 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:20:38 smithi079 ceph-mon[35312]: pgmap v427: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:20:40.032 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:20:39 smithi022 ceph-mon[31985]: 2 slow requests, 0 included below; oldest blocked for > 226.842173 secs 2024-03-28T18:20:40.174 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:20:39 smithi134 ceph-mon[35037]: 2 slow requests, 0 included below; oldest blocked for > 226.842173 secs 2024-03-28T18:20:40.182 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:20:39 smithi079 ceph-mon[35312]: 2 slow requests, 0 included below; oldest blocked for > 226.842173 secs 2024-03-28T18:20:41.032 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:20:40 smithi022 ceph-mon[31985]: pgmap v428: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:20:41.174 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:20:40 smithi134 ceph-mon[35037]: pgmap v428: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:20:41.182 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:20:40 smithi079 ceph-mon[35312]: pgmap v428: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:20:43.175 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:20:42 smithi134 ceph-mon[35037]: pgmap v429: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:20:43.182 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:20:42 smithi079 ceph-mon[35312]: pgmap v429: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:20:43.282 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:20:42 smithi022 ceph-mon[31985]: pgmap v429: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:20:45.174 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:20:44 smithi134 ceph-mon[35037]: pgmap v430: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:20:45.175 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:20:44 smithi134 ceph-mon[35037]: 2 slow requests, 0 included below; oldest blocked for > 231.842238 secs 2024-03-28T18:20:45.182 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:20:44 smithi079 ceph-mon[35312]: pgmap v430: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:20:45.182 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:20:44 smithi079 ceph-mon[35312]: 2 slow requests, 0 included below; oldest blocked for > 231.842238 secs 2024-03-28T18:20:45.282 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:20:44 smithi022 ceph-mon[31985]: pgmap v430: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:20:45.282 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:20:44 smithi022 ceph-mon[31985]: 2 slow requests, 0 included below; oldest blocked for > 231.842238 secs 2024-03-28T18:20:47.175 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:20:46 smithi134 ceph-mon[35037]: pgmap v431: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:20:47.182 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:20:46 smithi079 ceph-mon[35312]: pgmap v431: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:20:47.282 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:20:46 smithi022 ceph-mon[31985]: pgmap v431: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:20:49.174 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:20:48 smithi134 ceph-mon[35037]: pgmap v432: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:20:49.182 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:20:48 smithi079 ceph-mon[35312]: pgmap v432: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:20:49.282 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:20:48 smithi022 ceph-mon[31985]: pgmap v432: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:20:50.175 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:20:49 smithi134 ceph-mon[35037]: 2 slow requests, 0 included below; oldest blocked for > 236.842338 secs 2024-03-28T18:20:50.182 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:20:49 smithi079 ceph-mon[35312]: 2 slow requests, 0 included below; oldest blocked for > 236.842338 secs 2024-03-28T18:20:50.282 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:20:49 smithi022 ceph-mon[31985]: 2 slow requests, 0 included below; oldest blocked for > 236.842338 secs 2024-03-28T18:20:51.175 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:20:50 smithi134 ceph-mon[35037]: pgmap v433: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:20:51.182 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:20:50 smithi079 ceph-mon[35312]: pgmap v433: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:20:51.282 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:20:50 smithi022 ceph-mon[31985]: pgmap v433: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:20:52.634 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-28T18:20:52.635 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-28T18:20:52.872 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:20:52.872+0000 7f297de00640 1 -- 172.21.15.22:0/96015445 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f29781076b0 msgr2=0x7f2978107a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:20:52.872 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:20:52.872+0000 7f297de00640 1 --2- 172.21.15.22:0/96015445 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f29781076b0 0x7f2978107a90 secure :-1 s=READY pgs=823 cs=0 l=1 rev1=1 crypto rx=0x7f2968009930 tx=0x7f296802bc20 comp rx=0 tx=0).stop 2024-03-28T18:20:52.872 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:20:52.872+0000 7f297de00640 1 -- 172.21.15.22:0/96015445 shutdown_connections 2024-03-28T18:20:52.873 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:20:52.872+0000 7f297de00640 1 --2- 172.21.15.22:0/96015445 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f2978101fd0 0x7f297810e4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:20:52.873 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:20:52.872+0000 7f297de00640 1 --2- 172.21.15.22:0/96015445 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2978101610 0x7f2978101a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:20:52.873 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:20:52.872+0000 7f297de00640 1 --2- 172.21.15.22:0/96015445 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f29781076b0 0x7f2978107a90 unknown :-1 s=CLOSED pgs=823 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:20:52.873 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:20:52.872+0000 7f297de00640 1 -- 172.21.15.22:0/96015445 >> 172.21.15.22:0/96015445 conn(0x7f29780fd100 msgr2=0x7f29780ff520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:20:52.873 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:20:52.872+0000 7f297de00640 1 -- 172.21.15.22:0/96015445 shutdown_connections 2024-03-28T18:20:52.873 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:20:52.872+0000 7f297de00640 1 -- 172.21.15.22:0/96015445 wait complete. 2024-03-28T18:20:52.875 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:20:52.872+0000 7f297de00640 1 Processor -- start 2024-03-28T18:20:52.875 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:20:52.872+0000 7f297de00640 1 -- start start 2024-03-28T18:20:52.875 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:20:52.873+0000 7f297de00640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f2978101610 0x7f297819a1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:20:52.875 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:20:52.873+0000 7f297de00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2978101fd0 0x7f297819a730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:20:52.875 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:20:52.873+0000 7f297de00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f29781076b0 0x7f297819eab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:20:52.875 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:20:52.873+0000 7f297de00640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2978111a50 con 0x7f29781076b0 2024-03-28T18:20:52.876 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:20:52.873+0000 7f297de00640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29781118d0 con 0x7f2978101610 2024-03-28T18:20:52.876 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:20:52.873+0000 7f297de00640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2978111bd0 con 0x7f2978101fd0 2024-03-28T18:20:52.876 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:20:52.873+0000 7f2977400640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f29781076b0 0x7f297819eab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:20:52.876 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:20:52.873+0000 7f2976a00640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f2978101610 0x7f297819a1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:20:52.876 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:20:52.873+0000 7f2976a00640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f2978101610 0x7f297819a1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.22:48030/0 (socket says 172.21.15.22:48030) 2024-03-28T18:20:52.876 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:20:52.873+0000 7f2977400640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f29781076b0 0x7f297819eab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:35782/0 (socket says 172.21.15.22:35782) 2024-03-28T18:20:52.876 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:20:52.873+0000 7f2976a00640 1 -- 172.21.15.22:0/678638346 learned_addr learned my addr 172.21.15.22:0/678638346 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-28T18:20:52.876 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:20:52.873+0000 7f2976000640 1 --2- 172.21.15.22:0/678638346 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2978101fd0 0x7f297819a730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:20:52.876 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:20:52.874+0000 7f2976a00640 1 -- 172.21.15.22:0/678638346 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2978101fd0 msgr2=0x7f297819a730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:20:52.876 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:20:52.874+0000 7f2976a00640 1 --2- 172.21.15.22:0/678638346 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2978101fd0 0x7f297819a730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:20:52.876 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:20:52.874+0000 7f2976a00640 1 -- 172.21.15.22:0/678638346 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f29781076b0 msgr2=0x7f297819eab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:20:52.876 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:20:52.874+0000 7f2976a00640 1 --2- 172.21.15.22:0/678638346 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f29781076b0 0x7f297819eab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:20:52.876 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:20:52.874+0000 7f2976a00640 1 -- 172.21.15.22:0/678638346 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f297819f1e0 con 0x7f2978101610 2024-03-28T18:20:52.876 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:20:52.874+0000 7f2976000640 1 --2- 172.21.15.22:0/678638346 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2978101fd0 0x7f297819a730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-28T18:20:52.877 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:20:52.874+0000 7f2977400640 1 --2- 172.21.15.22:0/678638346 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f29781076b0 0x7f297819eab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-28T18:20:52.877 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:20:52.874+0000 7f2976a00640 1 --2- 172.21.15.22:0/678638346 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f2978101610 0x7f297819a1f0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f2968005ab0 tx=0x7f2968005020 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:20:52.877 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:20:52.874+0000 7f297ca00640 1 -- 172.21.15.22:0/678638346 <== mon.2 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2968043020 con 0x7f2978101610 2024-03-28T18:20:52.877 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:20:52.874+0000 7f297de00640 1 -- 172.21.15.22:0/678638346 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f297819f470 con 0x7f2978101610 2024-03-28T18:20:52.877 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:20:52.874+0000 7f297ca00640 1 -- 172.21.15.22:0/678638346 <== mon.2 v2:172.21.15.79:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f2968005420 con 0x7f2978101610 2024-03-28T18:20:52.877 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:20:52.876+0000 7f297ca00640 1 -- 172.21.15.22:0/678638346 <== mon.2 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2968003b60 con 0x7f2978101610 2024-03-28T18:20:52.877 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:20:52.876+0000 7f297de00640 1 -- 172.21.15.22:0/678638346 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f29781a6d60 con 0x7f2978101610 2024-03-28T18:20:52.878 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:20:52.878+0000 7f2980a4f740 1 Processor -- start 2024-03-28T18:20:52.879 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:20:52.878+0000 7f297ca00640 1 -- 172.21.15.22:0/678638346 <== mon.2 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 26) v1 ==== 137421+0+0 (secure 0 0 0) 0x7f2968046020 con 0x7f2978101610 2024-03-28T18:20:52.879 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:20:52.878+0000 7f2980a4f740 1 -- start start 2024-03-28T18:20:52.880 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:20:52.878+0000 7f2980a4f740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x555f1cf8ac00 0x555f1cf89950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:20:52.881 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:20:52.878+0000 7f2980a4f740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x555f1cf89f20 0x555f1cf8a3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:20:52.881 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:20:52.878+0000 7f2980a4f740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555f1cf93290 0x555f1cf95680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:20:52.881 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:20:52.878+0000 7f2980a4f740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x555f1ceee250 con 0x555f1cf89f20 2024-03-28T18:20:52.881 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:20:52.878+0000 7f2980a4f740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x555f1ce96a30 con 0x555f1cf8ac00 2024-03-28T18:20:52.881 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:20:52.878+0000 7f2980a4f740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x555f1ceea2f0 con 0x555f1cf93290 2024-03-28T18:20:52.881 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:20:52.879+0000 7f2976a00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x555f1cf89f20 0x555f1cf8a3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:20:52.881 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:20:52.879+0000 7f2977400640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x555f1cf8ac00 0x555f1cf89950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:20:52.881 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:20:52.879+0000 7f2976a00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x555f1cf89f20 0x555f1cf8a3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:35798/0 (socket says 172.21.15.22:35798) 2024-03-28T18:20:52.881 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:20:52.879+0000 7f2976a00640 1 -- 172.21.15.22:0/2587130540 learned_addr learned my addr 172.21.15.22:0/2587130540 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-28T18:20:52.881 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:20:52.879+0000 7f2976000640 1 --2- 172.21.15.22:0/2587130540 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555f1cf93290 0x555f1cf95680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:20:52.881 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:20:52.879+0000 7f2976a00640 1 -- 172.21.15.22:0/2587130540 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555f1cf93290 msgr2=0x555f1cf95680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T18:20:52.881 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:20:52.879+0000 7f2976a00640 1 --2- 172.21.15.22:0/2587130540 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555f1cf93290 0x555f1cf95680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:20:52.881 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:20:52.879+0000 7f2976a00640 1 -- 172.21.15.22:0/2587130540 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x555f1cf8ac00 msgr2=0x555f1cf89950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T18:20:52.882 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:20:52.879+0000 7f2976a00640 1 --2- 172.21.15.22:0/2587130540 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x555f1cf8ac00 0x555f1cf89950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:20:52.882 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:20:52.879+0000 7f2976a00640 1 -- 172.21.15.22:0/2587130540 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555f1ce81690 con 0x555f1cf89f20 2024-03-28T18:20:52.882 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:20:52.879+0000 7f2976000640 1 --2- 172.21.15.22:0/2587130540 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555f1cf93290 0x555f1cf95680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T18:20:52.882 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:20:52.879+0000 7f2977400640 1 --2- 172.21.15.22:0/2587130540 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x555f1cf8ac00 0x555f1cf89950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-28T18:20:52.882 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:20:52.880+0000 7f297ca00640 1 --2- 172.21.15.22:0/678638346 >> [v2:172.21.15.22:6800/1357365082,v1:172.21.15.22:6801/1357365082] conn(0x7f295c0a2970 0x7f295c0a4e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:20:52.882 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:20:52.880+0000 7f297ca00640 1 -- 172.21.15.22:0/678638346 <== mon.2 v2:172.21.15.79:3300/0 5 ==== osd_map(122..122 src has 1..122) v4 ==== 10389+0+0 (secure 0 0 0) 0x7f29680f1740 con 0x7f2978101610 2024-03-28T18:20:52.882 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:20:52.880+0000 7f2976a00640 1 --2- 172.21.15.22:0/2587130540 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x555f1cf89f20 0x555f1cf8a3a0 secure :-1 s=READY pgs=824 cs=0 l=1 rev1=1 crypto rx=0x7f29680ee730 tx=0x7f29680f7b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=af737f6b519998bc server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:20:52.882 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:20:52.880+0000 7f2977400640 1 --2- 172.21.15.22:0/678638346 >> [v2:172.21.15.22:6800/1357365082,v1:172.21.15.22:6801/1357365082] conn(0x7f295c0a2970 0x7f295c0a4e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:20:52.886 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:20:52.880+0000 7f2963e00640 1 -- 172.21.15.22:0/2587130540 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29680b99a0 con 0x555f1cf89f20 2024-03-28T18:20:52.886 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:20:52.880+0000 7f2963e00640 1 -- 172.21.15.22:0/2587130540 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f29680055d0 con 0x555f1cf89f20 2024-03-28T18:20:52.886 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:20:52.880+0000 7f2963e00640 1 -- 172.21.15.22:0/2587130540 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29680bd060 con 0x555f1cf89f20 2024-03-28T18:20:52.886 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:20:52.882+0000 7f2980a4f740 1 -- 172.21.15.22:0/2587130540 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x555f1cf89f20 msgr2=0x555f1cf8a3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:20:52.886 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:20:52.882+0000 7f2977400640 1 --2- 172.21.15.22:0/678638346 >> [v2:172.21.15.22:6800/1357365082,v1:172.21.15.22:6801/1357365082] conn(0x7f295c0a2970 0x7f295c0a4e30 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f2970005fd0 tx=0x7f2970009600 comp rx=0 tx=0).ready entity=mgr.15522 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:20:52.886 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:20:52.882+0000 7f2980a4f740 1 --2- 172.21.15.22:0/2587130540 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x555f1cf89f20 0x555f1cf8a3a0 secure :-1 s=READY pgs=824 cs=0 l=1 rev1=1 crypto rx=0x7f29680ee730 tx=0x7f29680f7b20 comp rx=0 tx=0).stop 2024-03-28T18:20:52.886 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:20:52.882+0000 7f2980a4f740 1 -- 172.21.15.22:0/2587130540 shutdown_connections 2024-03-28T18:20:52.886 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:20:52.882+0000 7f2980a4f740 1 --2- 172.21.15.22:0/2587130540 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555f1cf93290 0x555f1cf95680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:20:52.886 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:20:52.882+0000 7f2980a4f740 1 --2- 172.21.15.22:0/2587130540 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x555f1cf89f20 0x555f1cf8a3a0 unknown :-1 s=CLOSED pgs=824 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:20:52.886 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:20:52.882+0000 7f2980a4f740 1 --2- 172.21.15.22:0/2587130540 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x555f1cf8ac00 0x555f1cf89950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:20:52.886 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:20:52.882+0000 7f2980a4f740 1 -- 172.21.15.22:0/2587130540 >> 172.21.15.22:0/2587130540 conn(0x555f1cf51200 msgr2=0x555f1cf84970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:20:52.886 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:20:52.882+0000 7f2980a4f740 1 -- 172.21.15.22:0/2587130540 shutdown_connections 2024-03-28T18:20:52.887 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:20:52.882+0000 7f2980a4f740 1 -- 172.21.15.22:0/2587130540 wait complete. 2024-03-28T18:20:52.887 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:20:52.883+0000 7f2980a4f740 1 -- start start 2024-03-28T18:20:52.887 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:20:52.883+0000 7f2980a4f740 1 Processor -- start 2024-03-28T18:20:52.887 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:20:52.883+0000 7f2980a4f740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x555f1cf89f20 0x555f1cf92920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:20:52.887 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:20:52.883+0000 7f2980a4f740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555f1cf8ac00 0x555f1cf8fcb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:20:52.887 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:20:52.883+0000 7f2980a4f740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x555f1cf93290 0x555f1cf901f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:20:52.887 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:20:52.883+0000 7f2980a4f740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x555f1cf95ec0 con 0x555f1cf93290 2024-03-28T18:20:52.887 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:20:52.883+0000 7f2980a4f740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x555f1cea9e80 con 0x555f1cf89f20 2024-03-28T18:20:52.887 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:20:52.883+0000 7f2980a4f740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x555f1cf96040 con 0x555f1cf8ac00 2024-03-28T18:20:52.887 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:20:52.883+0000 7f2980a4f740 10 client.0 _ll_register_callbacks cb 0x555f1cebc130 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-28T18:20:52.887 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:20:52.883+0000 7f2977400640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x555f1cf93290 0x555f1cf901f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:20:52.887 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:20:52.883+0000 7f2976a00640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x555f1cf89f20 0x555f1cf92920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:20:52.887 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:20:52.883+0000 7f2977400640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x555f1cf93290 0x555f1cf901f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:35808/0 (socket says 172.21.15.22:35808) 2024-03-28T18:20:52.887 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:20:52.883+0000 7f2977400640 1 -- 172.21.15.22:0/2084120710 learned_addr learned my addr 172.21.15.22:0/2084120710 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-28T18:20:52.887 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:20:52.883+0000 7f2976000640 1 --2- 172.21.15.22:0/2084120710 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555f1cf8ac00 0x555f1cf8fcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:20:52.887 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:20:52.883+0000 7f2977400640 1 -- 172.21.15.22:0/2084120710 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555f1cf8ac00 msgr2=0x555f1cf8fcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T18:20:52.888 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:20:52.883+0000 7f2977400640 1 --2- 172.21.15.22:0/2084120710 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555f1cf8ac00 0x555f1cf8fcb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:20:52.888 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:20:52.883+0000 7f2977400640 1 -- 172.21.15.22:0/2084120710 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x555f1cf89f20 msgr2=0x555f1cf92920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T18:20:52.888 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:20:52.883+0000 7f2977400640 1 --2- 172.21.15.22:0/2084120710 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x555f1cf89f20 0x555f1cf92920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:20:52.888 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:20:52.883+0000 7f2977400640 1 -- 172.21.15.22:0/2084120710 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x555f1cccc7d0 con 0x555f1cf93290 2024-03-28T18:20:52.888 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:20:52.885+0000 7f2977400640 1 --2- 172.21.15.22:0/2084120710 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x555f1cf93290 0x555f1cf901f0 secure :-1 s=READY pgs=825 cs=0 l=1 rev1=1 crypto rx=0x7f2970013d60 tx=0x7f2970004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=668bdcbc3663b3c6 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:20:52.888 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:20:52.885+0000 7f2960c00640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-03-28T18:20:52.888 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:20:52.885+0000 7f2960c00640 1 -- 172.21.15.22:0/2084120710 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f297002b060 con 0x555f1cf93290 2024-03-28T18:20:52.888 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:20:52.885+0000 7f2960c00640 1 -- 172.21.15.22:0/2084120710 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(122..122 src has 1..122) v4 ==== 10389+0+0 (secure 0 0 0) 0x7f297001c460 con 0x555f1cf93290 2024-03-28T18:20:52.888 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:20:52.885+0000 7f2980a4f740 1 -- 172.21.15.22:0/2084120710 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x555f1cefb790 con 0x555f1cf93290 2024-03-28T18:20:52.888 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:20:52.885+0000 7f2960c00640 1 -- 172.21.15.22:0/2084120710 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=189) v2 ==== 24+0+0 (secure 0 0 0) 0x7f297001c660 con 0x555f1cf93290 2024-03-28T18:20:52.888 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:20:52.885+0000 7f2980a4f740 10 client.16944 fetch_fsmap learned FSMap version 189 2024-03-28T18:20:52.888 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:20:52.885+0000 7f2980a4f740 1 -- 172.21.15.22:0/2084120710 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=189,monmap=4+}) v3 -- 0x555f1cc023f0 con 0x555f1cf93290 2024-03-28T18:20:52.888 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:20:52.885+0000 7f2960c00640 1 -- 172.21.15.22:0/2084120710 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x555f1cefb790 con 0x555f1cf93290 2024-03-28T18:20:52.888 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:20:52.885+0000 7f2960c00640 1 -- 172.21.15.22:0/2084120710 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 189) v1 ==== 6221+0+0 (secure 0 0 0) 0x7f297001a040 con 0x555f1cf93290 2024-03-28T18:20:52.888 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:20:52.885+0000 7f2980a4f740 10 client.16944 fetch_fsmap finished waiting for FSMap version 189 2024-03-28T18:20:52.888 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:20:52.885+0000 7f2980a4f740 10 client.16944 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-28T18:20:52.888 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:20:52.885+0000 7f2980a4f740 20 client.16944 populate_metadata read hostname 'smithi022' 2024-03-28T18:20:52.889 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:20:52.885+0000 7f2980a4f740 1 --2- 172.21.15.22:0/2084120710 >> [v2:172.21.15.79:6838/2441542443,v1:172.21.15.79:6839/2441542443] conn(0x555f1cfa3060 0x555f1cfa54e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:20:52.889 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:20:52.885+0000 7f2980a4f740 4 client.16944 mds_command: new command op to 34553 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-28T18:20:52.889 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:20:52.885+0000 7f2980a4f740 1 -- 172.21.15.22:0/2084120710 --> [v2:172.21.15.79:6838/2441542443,v1:172.21.15.79:6839/2441542443] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x555f1cef5ae0 con 0x555f1cfa3060 2024-03-28T18:20:52.890 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:20:52.885+0000 7f2976a00640 1 --2- 172.21.15.22:0/2084120710 >> [v2:172.21.15.79:6838/2441542443,v1:172.21.15.79:6839/2441542443] conn(0x555f1cfa3060 0x555f1cfa54e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:20:52.890 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:20:52.887+0000 7f2976a00640 1 --2- 172.21.15.22:0/2084120710 >> [v2:172.21.15.79:6838/2441542443,v1:172.21.15.79:6839/2441542443] conn(0x555f1cfa3060 0x555f1cfa54e0 crc :-1 s=READY pgs=317 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1c82b4972032194 server_cookie=f31f5977d1dc1035 in_seq=0 out_seq=0 2024-03-28T18:20:52.890 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:20:52.887+0000 7f2960c00640 10 client.16944 ms_handle_connect on v2:172.21.15.79:6838/2441542443 2024-03-28T18:20:52.890 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:20:52.888+0000 7f2960c00640 1 -- 172.21.15.22:0/2084120710 <== mds.0 v2:172.21.15.79:6838/2441542443 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x555f1cef5ae0 con 0x555f1cfa3060 2024-03-28T18:20:52.890 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:20:52.888+0000 7f2960c00640 10 client.16944 handle_command_reply: tid=0 2024-03-28T18:20:52.890 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:20:52.889+0000 7f2980a4f740 1 client.16944 shutdown 2024-03-28T18:20:52.890 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:20:52.889+0000 7f2980a4f740 1 -- 172.21.15.22:0/2084120710 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x555f1cf93290 msgr2=0x555f1cf901f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:20:52.891 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:20:52.889+0000 7f2980a4f740 1 --2- 172.21.15.22:0/2084120710 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x555f1cf93290 0x555f1cf901f0 secure :-1 s=READY pgs=825 cs=0 l=1 rev1=1 crypto rx=0x7f2970013d60 tx=0x7f2970004a40 comp rx=0 tx=0).stop 2024-03-28T18:20:52.891 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:20:52.889+0000 7f2980a4f740 1 -- 172.21.15.22:0/2084120710 shutdown_connections 2024-03-28T18:20:52.891 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:20:52.889+0000 7f2980a4f740 1 --2- 172.21.15.22:0/2084120710 >> [v2:172.21.15.79:6838/2441542443,v1:172.21.15.79:6839/2441542443] conn(0x555f1cfa3060 0x555f1cfa54e0 crc :-1 s=READY pgs=317 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:20:52.891 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:20:52.889+0000 7f2980a4f740 1 --2- 172.21.15.22:0/2084120710 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x555f1cf93290 0x555f1cf901f0 unknown :-1 s=CLOSED pgs=825 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:20:52.891 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:20:52.889+0000 7f2980a4f740 1 --2- 172.21.15.22:0/2084120710 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555f1cf8ac00 0x555f1cf8fcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:20:52.891 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:20:52.889+0000 7f2980a4f740 1 --2- 172.21.15.22:0/2084120710 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x555f1cf89f20 0x555f1cf92920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:20:52.891 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:20:52.889+0000 7f2980a4f740 1 -- 172.21.15.22:0/2084120710 >> 172.21.15.22:0/2084120710 conn(0x555f1cf51200 msgr2=0x555f1cf866c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:20:52.891 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:20:52.889+0000 7f2960c00640 0 client.16944 ms_handle_reset on v2:172.21.15.79:6838/2441542443 2024-03-28T18:20:52.891 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:20:52.889+0000 7f2980a4f740 1 -- 172.21.15.22:0/2084120710 shutdown_connections 2024-03-28T18:20:52.891 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:20:52.889+0000 7f2980a4f740 1 -- 172.21.15.22:0/2084120710 wait complete. 2024-03-28T18:20:52.891 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:20:52.889+0000 7f2980a4f740 20 client.16944 trim_cache size 0 max 16384 2024-03-28T18:20:52.907 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:20:52.906+0000 7f2980a4f740 1 Processor -- start 2024-03-28T18:20:52.907 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:20:52.906+0000 7f2980a4f740 1 -- start start 2024-03-28T18:20:52.907 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:20:52.906+0000 7f2980a4f740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x555f1cf89f20 0x555f1cf923e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:20:52.907 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:20:52.906+0000 7f2980a4f740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x555f1cf8ac00 0x555f1cf92920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:20:52.907 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:20:52.906+0000 7f2980a4f740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555f1cf93290 0x555f1cf94680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:20:52.907 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:20:52.906+0000 7f2980a4f740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29781b1f50 con 0x555f1cf8ac00 2024-03-28T18:20:52.907 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:20:52.906+0000 7f2980a4f740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29781af2b0 con 0x555f1cf89f20 2024-03-28T18:20:52.908 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:20:52.906+0000 7f2980a4f740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29781b1290 con 0x555f1cf93290 2024-03-28T18:20:52.909 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:20:52.907+0000 7f2976a00640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x555f1cf89f20 0x555f1cf923e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:20:52.909 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:20:52.907+0000 7f2976a00640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x555f1cf89f20 0x555f1cf923e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.22:48062/0 (socket says 172.21.15.22:48062) 2024-03-28T18:20:52.910 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:20:52.907+0000 7f2976a00640 1 -- 172.21.15.22:0/2804775659 learned_addr learned my addr 172.21.15.22:0/2804775659 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-28T18:20:52.910 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:20:52.907+0000 7f2976000640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x555f1cf8ac00 0x555f1cf92920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:20:52.910 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:20:52.907+0000 7f2976a00640 1 -- 172.21.15.22:0/2804775659 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555f1cf93290 msgr2=0x555f1cf94680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T18:20:52.910 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:20:52.907+0000 7f2977400640 1 --2- 172.21.15.22:0/2804775659 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555f1cf93290 0x555f1cf94680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:20:52.910 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:20:52.907+0000 7f2976a00640 1 --2- 172.21.15.22:0/2804775659 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555f1cf93290 0x555f1cf94680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:20:52.910 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:20:52.907+0000 7f2976a00640 1 -- 172.21.15.22:0/2804775659 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x555f1cf8ac00 msgr2=0x555f1cf92920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T18:20:52.910 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:20:52.907+0000 7f2976a00640 1 --2- 172.21.15.22:0/2804775659 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x555f1cf8ac00 0x555f1cf92920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:20:52.910 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:20:52.907+0000 7f2976a00640 1 -- 172.21.15.22:0/2804775659 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555f1cbc20f0 con 0x555f1cf89f20 2024-03-28T18:20:52.910 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:20:52.907+0000 7f2977400640 1 --2- 172.21.15.22:0/2804775659 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555f1cf93290 0x555f1cf94680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T18:20:52.910 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:20:52.909+0000 7f2976a00640 1 --2- 172.21.15.22:0/2804775659 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x555f1cf89f20 0x555f1cf923e0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f29680bd690 tx=0x7f29680f7b20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=675b4a4e976b5874 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:20:52.910 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:20:52.909+0000 7f2963400640 1 -- 172.21.15.22:0/2804775659 <== mon.2 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29680bd240 con 0x555f1cf89f20 2024-03-28T18:20:52.910 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:20:52.909+0000 7f2963400640 1 -- 172.21.15.22:0/2804775659 <== mon.2 v2:172.21.15.79:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f29680bde00 con 0x555f1cf89f20 2024-03-28T18:20:52.910 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:20:52.909+0000 7f2963400640 1 -- 172.21.15.22:0/2804775659 <== mon.2 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29680bd4d0 con 0x555f1cf89f20 2024-03-28T18:20:52.910 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:20:52.910+0000 7f2980a4f740 1 -- 172.21.15.22:0/2804775659 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x555f1cf89f20 msgr2=0x555f1cf923e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:20:52.911 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:20:52.910+0000 7f2980a4f740 1 --2- 172.21.15.22:0/2804775659 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x555f1cf89f20 0x555f1cf923e0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f29680bd690 tx=0x7f29680f7b20 comp rx=0 tx=0).stop 2024-03-28T18:20:52.911 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:20:52.910+0000 7f2980a4f740 1 -- 172.21.15.22:0/2804775659 shutdown_connections 2024-03-28T18:20:52.911 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:20:52.910+0000 7f2980a4f740 1 --2- 172.21.15.22:0/2804775659 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555f1cf93290 0x555f1cf94680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:20:52.911 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:20:52.910+0000 7f2980a4f740 1 --2- 172.21.15.22:0/2804775659 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x555f1cf8ac00 0x555f1cf92920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:20:52.911 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:20:52.910+0000 7f2980a4f740 1 --2- 172.21.15.22:0/2804775659 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x555f1cf89f20 0x555f1cf923e0 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:20:52.911 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:20:52.910+0000 7f2980a4f740 1 -- 172.21.15.22:0/2804775659 >> 172.21.15.22:0/2804775659 conn(0x555f1cf51200 msgr2=0x555f1cf8e5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:20:52.911 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:20:52.910+0000 7f2980a4f740 1 -- 172.21.15.22:0/2804775659 shutdown_connections 2024-03-28T18:20:52.911 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:20:52.910+0000 7f2980a4f740 1 -- 172.21.15.22:0/2804775659 wait complete. 2024-03-28T18:20:52.914 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:20:52.910+0000 7f2980a4f740 1 -- start start 2024-03-28T18:20:52.914 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:20:52.910+0000 7f2980a4f740 1 Processor -- start 2024-03-28T18:20:52.914 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:20:52.911+0000 7f2980a4f740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x555f1cf89f20 0x555f1cf8cb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:20:52.914 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:20:52.911+0000 7f2980a4f740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x555f1cf8ac00 0x555f1cf8d060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:20:52.914 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:20:52.911+0000 7f2980a4f740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555f1cf93290 0x555f1cffd970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:20:52.914 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:20:52.911+0000 7f2980a4f740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29781b0c30 con 0x555f1cf8ac00 2024-03-28T18:20:52.914 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:20:52.911+0000 7f2980a4f740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29781af910 con 0x555f1cf89f20 2024-03-28T18:20:52.914 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:20:52.911+0000 7f2980a4f740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29781aff70 con 0x555f1cf93290 2024-03-28T18:20:52.914 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:20:52.911+0000 7f2980a4f740 10 client.0 _ll_register_callbacks cb 0x7f2970009ec0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-28T18:20:52.914 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:20:52.911+0000 7f2976000640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x555f1cf8ac00 0x555f1cf8d060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:20:52.914 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:20:52.911+0000 7f2976a00640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x555f1cf89f20 0x555f1cf8cb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:20:52.914 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:20:52.911+0000 7f2976000640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x555f1cf8ac00 0x555f1cf8d060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:35830/0 (socket says 172.21.15.22:35830) 2024-03-28T18:20:52.915 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:20:52.911+0000 7f2976000640 1 -- 172.21.15.22:0/3601234996 learned_addr learned my addr 172.21.15.22:0/3601234996 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-28T18:20:52.915 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:20:52.912+0000 7f2977400640 1 --2- 172.21.15.22:0/3601234996 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555f1cf93290 0x555f1cffd970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:20:52.915 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:20:52.912+0000 7f2976000640 1 -- 172.21.15.22:0/3601234996 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555f1cf93290 msgr2=0x555f1cffd970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T18:20:52.915 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:20:52.912+0000 7f2976000640 1 --2- 172.21.15.22:0/3601234996 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555f1cf93290 0x555f1cffd970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:20:52.915 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:20:52.912+0000 7f2976000640 1 -- 172.21.15.22:0/3601234996 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x555f1cf89f20 msgr2=0x555f1cf8cb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T18:20:52.915 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:20:52.912+0000 7f2976000640 1 --2- 172.21.15.22:0/3601234996 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x555f1cf89f20 0x555f1cf8cb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:20:52.915 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:20:52.912+0000 7f2976000640 1 -- 172.21.15.22:0/3601234996 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x555f1cffdf30 con 0x555f1cf8ac00 2024-03-28T18:20:52.915 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:20:52.912+0000 7f2976a00640 1 --2- 172.21.15.22:0/3601234996 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x555f1cf89f20 0x555f1cf8cb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-28T18:20:52.915 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:20:52.912+0000 7f2976000640 1 --2- 172.21.15.22:0/3601234996 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x555f1cf8ac00 0x555f1cf8d060 secure :-1 s=READY pgs=826 cs=0 l=1 rev1=1 crypto rx=0x7f297810e2a0 tx=0x7f2964012bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3c7793c5ab2fca8f server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:20:52.915 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:20:52.912+0000 7f2960c00640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-03-28T18:20:52.915 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:20:52.912+0000 7f2960c00640 1 -- 172.21.15.22:0/3601234996 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2964011b50 con 0x555f1cf8ac00 2024-03-28T18:20:52.915 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:20:52.912+0000 7f2960c00640 1 -- 172.21.15.22:0/3601234996 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(122..122 src has 1..122) v4 ==== 10389+0+0 (secure 0 0 0) 0x7f2964011cf0 con 0x555f1cf8ac00 2024-03-28T18:20:52.915 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:20:52.912+0000 7f2980a4f740 1 -- 172.21.15.22:0/3601234996 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f29680bde00 con 0x555f1cf8ac00 2024-03-28T18:20:52.915 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:20:52.913+0000 7f2960c00640 1 -- 172.21.15.22:0/3601234996 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=189) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2964005020 con 0x555f1cf8ac00 2024-03-28T18:20:52.915 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:20:52.913+0000 7f2980a4f740 10 client.16950 fetch_fsmap learned FSMap version 189 2024-03-28T18:20:52.916 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:20:52.913+0000 7f2980a4f740 1 -- 172.21.15.22:0/3601234996 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=189,monmap=4+}) v3 -- 0x555f1cfff1f0 con 0x555f1cf8ac00 2024-03-28T18:20:52.916 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:20:52.913+0000 7f2960c00640 1 -- 172.21.15.22:0/3601234996 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f29680bde00 con 0x555f1cf8ac00 2024-03-28T18:20:52.916 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:20:52.913+0000 7f2960c00640 1 -- 172.21.15.22:0/3601234996 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 189) v1 ==== 6221+0+0 (secure 0 0 0) 0x7f296401c470 con 0x555f1cf8ac00 2024-03-28T18:20:52.916 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:20:52.913+0000 7f2980a4f740 10 client.16950 fetch_fsmap finished waiting for FSMap version 189 2024-03-28T18:20:52.916 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:20:52.913+0000 7f2980a4f740 10 client.16950 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-28T18:20:52.916 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:20:52.913+0000 7f2980a4f740 20 client.16950 populate_metadata read hostname 'smithi022' 2024-03-28T18:20:52.916 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:20:52.913+0000 7f2980a4f740 1 --2- 172.21.15.22:0/3601234996 >> [v2:172.21.15.79:6838/2441542443,v1:172.21.15.79:6839/2441542443] conn(0x555f1cfff980 0x555f1d001da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:20:52.916 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:20:52.913+0000 7f2980a4f740 4 client.16950 mds_command: new command op to 34553 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-28T18:20:52.916 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:20:52.913+0000 7f2980a4f740 1 -- 172.21.15.22:0/3601234996 --> [v2:172.21.15.79:6838/2441542443,v1:172.21.15.79:6839/2441542443] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x555f1ce733a0 con 0x555f1cfff980 2024-03-28T18:20:52.916 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:20:52.914+0000 7f2976a00640 1 --2- 172.21.15.22:0/3601234996 >> [v2:172.21.15.79:6838/2441542443,v1:172.21.15.79:6839/2441542443] conn(0x555f1cfff980 0x555f1d001da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:20:52.916 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:20:52.914+0000 7f2976a00640 1 --2- 172.21.15.22:0/3601234996 >> [v2:172.21.15.79:6838/2441542443,v1:172.21.15.79:6839/2441542443] conn(0x555f1cfff980 0x555f1d001da0 crc :-1 s=READY pgs=318 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=671dede21f631b58 server_cookie=261e0fa066b00ea5 in_seq=0 out_seq=0 2024-03-28T18:20:52.916 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:20:52.914+0000 7f2960c00640 10 client.16950 ms_handle_connect on v2:172.21.15.79:6838/2441542443 2024-03-28T18:20:52.917 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:20:52.915+0000 7f2960c00640 1 -- 172.21.15.22:0/3601234996 <== mds.0 v2:172.21.15.79:6838/2441542443 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x555f1ce733a0 con 0x555f1cfff980 2024-03-28T18:20:52.917 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:20:52.915+0000 7f2960c00640 10 client.16950 handle_command_reply: tid=0 2024-03-28T18:20:52.918 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:20:52.915+0000 7f2980a4f740 1 client.16950 shutdown 2024-03-28T18:20:52.918 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:20:52.915+0000 7f2980a4f740 1 -- 172.21.15.22:0/3601234996 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x555f1cf8ac00 msgr2=0x555f1cf8d060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:20:52.918 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:20:52.915+0000 7f2980a4f740 1 --2- 172.21.15.22:0/3601234996 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x555f1cf8ac00 0x555f1cf8d060 secure :-1 s=READY pgs=826 cs=0 l=1 rev1=1 crypto rx=0x7f297810e2a0 tx=0x7f2964012bd0 comp rx=0 tx=0).stop 2024-03-28T18:20:52.918 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:20:52.915+0000 7f2980a4f740 1 -- 172.21.15.22:0/3601234996 shutdown_connections 2024-03-28T18:20:52.918 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:20:52.915+0000 7f2980a4f740 1 --2- 172.21.15.22:0/3601234996 >> [v2:172.21.15.79:6838/2441542443,v1:172.21.15.79:6839/2441542443] conn(0x555f1cfff980 0x555f1d001da0 crc :-1 s=READY pgs=318 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:20:52.918 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:20:52.915+0000 7f2980a4f740 1 --2- 172.21.15.22:0/3601234996 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555f1cf93290 0x555f1cffd970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:20:52.918 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:20:52.915+0000 7f2980a4f740 1 --2- 172.21.15.22:0/3601234996 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x555f1cf8ac00 0x555f1cf8d060 unknown :-1 s=CLOSED pgs=826 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:20:52.918 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:20:52.915+0000 7f2980a4f740 1 --2- 172.21.15.22:0/3601234996 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x555f1cf89f20 0x555f1cf8cb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:20:52.918 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:20:52.915+0000 7f2980a4f740 1 -- 172.21.15.22:0/3601234996 >> 172.21.15.22:0/3601234996 conn(0x555f1cf51200 msgr2=0x555f1cf83200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:20:52.918 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:20:52.915+0000 7f2960c00640 0 client.16950 ms_handle_reset on v2:172.21.15.79:6838/2441542443 2024-03-28T18:20:52.918 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:20:52.915+0000 7f2980a4f740 1 -- 172.21.15.22:0/3601234996 shutdown_connections 2024-03-28T18:20:52.918 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:20:52.915+0000 7f2980a4f740 1 -- 172.21.15.22:0/3601234996 wait complete. 2024-03-28T18:20:52.918 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:20:52.915+0000 7f2980a4f740 20 client.16950 trim_cache size 0 max 16384 2024-03-28T18:20:52.919 INFO:teuthology.orchestra.run.smithi022.stdout:{ 2024-03-28T18:20:52.919 INFO:teuthology.orchestra.run.smithi022.stdout: "status": "IDLE (waiting for more scrubs)", 2024-03-28T18:20:52.919 INFO:teuthology.orchestra.run.smithi022.stdout: "scrubs": { 2024-03-28T18:20:52.919 INFO:teuthology.orchestra.run.smithi022.stdout: "4bbbd61d-29a3-4bf6-a4f2-d5783d585de3": { 2024-03-28T18:20:52.919 INFO:teuthology.orchestra.run.smithi022.stdout: "path": "/", 2024-03-28T18:20:52.919 INFO:teuthology.orchestra.run.smithi022.stdout: "tag": "4bbbd61d-29a3-4bf6-a4f2-d5783d585de3", 2024-03-28T18:20:52.919 INFO:teuthology.orchestra.run.smithi022.stdout: "options": "recursive,force" 2024-03-28T18:20:52.919 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-03-28T18:20:52.919 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-03-28T18:20:52.919 INFO:teuthology.orchestra.run.smithi022.stdout:} 2024-03-28T18:20:52.920 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:20:52.916+0000 7f297de00640 1 -- 172.21.15.22:0/678638346 >> [v2:172.21.15.22:6800/1357365082,v1:172.21.15.22:6801/1357365082] conn(0x7f295c0a2970 msgr2=0x7f295c0a4e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:20:52.920 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:20:52.916+0000 7f297de00640 1 --2- 172.21.15.22:0/678638346 >> [v2:172.21.15.22:6800/1357365082,v1:172.21.15.22:6801/1357365082] conn(0x7f295c0a2970 0x7f295c0a4e30 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f2970005fd0 tx=0x7f2970009600 comp rx=0 tx=0).stop 2024-03-28T18:20:52.920 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:20:52.916+0000 7f297de00640 1 -- 172.21.15.22:0/678638346 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f2978101610 msgr2=0x7f297819a1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:20:52.920 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:20:52.916+0000 7f297de00640 1 --2- 172.21.15.22:0/678638346 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f2978101610 0x7f297819a1f0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f2968005ab0 tx=0x7f2968005020 comp rx=0 tx=0).stop 2024-03-28T18:20:52.920 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:20:52.916+0000 7f297de00640 1 -- 172.21.15.22:0/678638346 shutdown_connections 2024-03-28T18:20:52.920 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:20:52.916+0000 7f297de00640 1 --2- 172.21.15.22:0/678638346 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f29781076b0 0x7f297819eab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:20:52.920 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:20:52.916+0000 7f297de00640 1 --2- 172.21.15.22:0/678638346 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2978101fd0 0x7f297819a730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:20:52.920 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:20:52.916+0000 7f297de00640 1 --2- 172.21.15.22:0/678638346 >> [v2:172.21.15.22:6800/1357365082,v1:172.21.15.22:6801/1357365082] conn(0x7f295c0a2970 0x7f295c0a4e30 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:20:52.921 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:20:52.916+0000 7f297de00640 1 --2- 172.21.15.22:0/678638346 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f2978101610 0x7f297819a1f0 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:20:52.921 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:20:52.916+0000 7f297de00640 1 -- 172.21.15.22:0/678638346 >> 172.21.15.22:0/678638346 conn(0x7f29780fd100 msgr2=0x7f29780fe5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:20:52.921 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:20:52.916+0000 7f297de00640 1 -- 172.21.15.22:0/678638346 shutdown_connections 2024-03-28T18:20:52.921 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:20:52.916+0000 7f297de00640 1 -- 172.21.15.22:0/678638346 wait complete. 2024-03-28T18:20:52.935 INFO:tasks.cephfs.filesystem:scrub status for tag:4bbbd61d-29a3-4bf6-a4f2-d5783d585de3 - {'path': '/', 'tag': '4bbbd61d-29a3-4bf6-a4f2-d5783d585de3', 'options': 'recursive,force'} 2024-03-28T18:20:53.032 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:20:52 smithi022 ceph-mon[31985]: pgmap v434: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:20:53.175 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:20:52 smithi134 ceph-mon[35037]: pgmap v434: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:20:53.182 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:20:52 smithi079 ceph-mon[35312]: pgmap v434: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:20:54.425 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:20:54 smithi134 ceph-mon[35037]: pgmap v435: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:20:54.432 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:20:54 smithi079 ceph-mon[35312]: pgmap v435: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:20:54.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:20:54 smithi022 ceph-mon[31985]: pgmap v435: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:20:55.433 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:20:55 smithi079 ceph-mon[35312]: 2 slow requests, 0 included below; oldest blocked for > 241.842423 secs 2024-03-28T18:20:55.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:20:55 smithi022 ceph-mon[31985]: 2 slow requests, 0 included below; oldest blocked for > 241.842423 secs 2024-03-28T18:20:55.674 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:20:55 smithi134 ceph-mon[35037]: 2 slow requests, 0 included below; oldest blocked for > 241.842423 secs 2024-03-28T18:20:56.433 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:20:56 smithi079 ceph-mon[35312]: pgmap v436: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:20:56.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:20:56 smithi022 ceph-mon[31985]: pgmap v436: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:20:56.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:20:56 smithi134 ceph-mon[35037]: pgmap v436: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:20:58.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:20:58 smithi022 ceph-mon[31985]: pgmap v437: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:20:58.925 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:20:58 smithi134 ceph-mon[35037]: pgmap v437: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:20:58.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:20:58 smithi079 ceph-mon[35312]: pgmap v437: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:20:59.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:20:59 smithi022 ceph-mon[31985]: 2 slow requests, 0 included below; oldest blocked for > 246.842486 secs 2024-03-28T18:20:59.925 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:20:59 smithi134 ceph-mon[35037]: 2 slow requests, 0 included below; oldest blocked for > 246.842486 secs 2024-03-28T18:20:59.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:20:59 smithi079 ceph-mon[35312]: 2 slow requests, 0 included below; oldest blocked for > 246.842486 secs 2024-03-28T18:21:00.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:21:00 smithi022 ceph-mon[31985]: pgmap v438: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:21:00.925 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:21:00 smithi134 ceph-mon[35037]: pgmap v438: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:21:00.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:21:00 smithi079 ceph-mon[35312]: pgmap v438: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:21:02.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:21:02 smithi022 ceph-mon[31985]: pgmap v439: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:21:02.925 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:21:02 smithi134 ceph-mon[35037]: pgmap v439: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:21:02.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:21:02 smithi079 ceph-mon[35312]: pgmap v439: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:21:04.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:21:04 smithi022 ceph-mon[31985]: pgmap v440: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:21:04.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:21:04 smithi022 ceph-mon[31985]: 2 slow requests, 0 included below; oldest blocked for > 251.842562 secs 2024-03-28T18:21:04.925 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:21:04 smithi134 ceph-mon[35037]: pgmap v440: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:21:04.925 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:21:04 smithi134 ceph-mon[35037]: 2 slow requests, 0 included below; oldest blocked for > 251.842562 secs 2024-03-28T18:21:04.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:21:04 smithi079 ceph-mon[35312]: pgmap v440: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:21:04.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:21:04 smithi079 ceph-mon[35312]: 2 slow requests, 0 included below; oldest blocked for > 251.842562 secs 2024-03-28T18:21:06.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:21:06 smithi022 ceph-mon[31985]: pgmap v441: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:21:06.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:21:06 smithi134 ceph-mon[35037]: pgmap v441: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:21:06.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:21:06 smithi079 ceph-mon[35312]: pgmap v441: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:21:08.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:21:08 smithi022 ceph-mon[31985]: pgmap v442: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:21:08.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:21:08 smithi134 ceph-mon[35037]: pgmap v442: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:21:08.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:21:08 smithi079 ceph-mon[35312]: pgmap v442: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:21:09.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:21:09 smithi022 ceph-mon[31985]: 2 slow requests, 0 included below; oldest blocked for > 256.842687 secs 2024-03-28T18:21:09.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:21:09 smithi134 ceph-mon[35037]: 2 slow requests, 0 included below; oldest blocked for > 256.842687 secs 2024-03-28T18:21:09.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:21:09 smithi079 ceph-mon[35312]: 2 slow requests, 0 included below; oldest blocked for > 256.842687 secs 2024-03-28T18:21:10.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:21:10 smithi022 ceph-mon[31985]: pgmap v443: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:21:10.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:21:10 smithi022 ceph-mon[31985]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T18:21:10.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:21:10 smithi134 ceph-mon[35037]: pgmap v443: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:21:10.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:21:10 smithi134 ceph-mon[35037]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T18:21:10.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:21:10 smithi079 ceph-mon[35312]: pgmap v443: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:21:10.933 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:21:10 smithi079 ceph-mon[35312]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T18:21:11.925 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:21:11 smithi134 ceph-mon[35037]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T18:21:11.925 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:21:11 smithi134 ceph-mon[35037]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T18:21:11.925 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:21:11 smithi134 ceph-mon[35037]: from='mgr.15522 ' entity='mgr.x' 2024-03-28T18:21:11.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:21:11 smithi079 ceph-mon[35312]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T18:21:11.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:21:11 smithi079 ceph-mon[35312]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T18:21:11.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:21:11 smithi079 ceph-mon[35312]: from='mgr.15522 ' entity='mgr.x' 2024-03-28T18:21:12.032 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:21:11 smithi022 ceph-mon[31985]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T18:21:12.032 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:21:11 smithi022 ceph-mon[31985]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T18:21:12.032 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:21:11 smithi022 ceph-mon[31985]: from='mgr.15522 ' entity='mgr.x' 2024-03-28T18:21:12.925 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:21:12 smithi134 ceph-mon[35037]: pgmap v444: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 6 op/s 2024-03-28T18:21:12.925 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:21:12 smithi134 ceph-mon[35037]: pgmap v445: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-03-28T18:21:12.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:21:12 smithi079 ceph-mon[35312]: pgmap v444: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 6 op/s 2024-03-28T18:21:12.933 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:21:12 smithi079 ceph-mon[35312]: pgmap v445: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-03-28T18:21:13.032 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:21:12 smithi022 ceph-mon[31985]: pgmap v444: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 6 op/s 2024-03-28T18:21:13.032 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:21:12 smithi022 ceph-mon[31985]: pgmap v445: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-03-28T18:21:14.925 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:21:14 smithi134 ceph-mon[35037]: pgmap v446: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-03-28T18:21:14.925 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:21:14 smithi134 ceph-mon[35037]: 2 slow requests, 0 included below; oldest blocked for > 261.842805 secs 2024-03-28T18:21:14.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:21:14 smithi079 ceph-mon[35312]: pgmap v446: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-03-28T18:21:14.933 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:21:14 smithi079 ceph-mon[35312]: 2 slow requests, 0 included below; oldest blocked for > 261.842805 secs 2024-03-28T18:21:15.032 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:21:14 smithi022 ceph-mon[31985]: pgmap v446: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-03-28T18:21:15.032 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:21:14 smithi022 ceph-mon[31985]: 2 slow requests, 0 included below; oldest blocked for > 261.842805 secs 2024-03-28T18:21:16.925 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:21:16 smithi134 ceph-mon[35037]: pgmap v447: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-03-28T18:21:16.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:21:16 smithi079 ceph-mon[35312]: pgmap v447: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-03-28T18:21:17.032 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:21:16 smithi022 ceph-mon[31985]: pgmap v447: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-03-28T18:21:18.925 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:21:18 smithi134 ceph-mon[35037]: pgmap v448: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-03-28T18:21:18.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:21:18 smithi079 ceph-mon[35312]: pgmap v448: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-03-28T18:21:19.032 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:21:18 smithi022 ceph-mon[31985]: pgmap v448: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-03-28T18:21:19.925 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:21:19 smithi134 ceph-mon[35037]: 2 slow requests, 0 included below; oldest blocked for > 266.842870 secs 2024-03-28T18:21:19.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:21:19 smithi079 ceph-mon[35312]: 2 slow requests, 0 included below; oldest blocked for > 266.842870 secs 2024-03-28T18:21:20.032 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:21:19 smithi022 ceph-mon[31985]: 2 slow requests, 0 included below; oldest blocked for > 266.842870 secs 2024-03-28T18:21:20.925 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:21:20 smithi134 ceph-mon[35037]: pgmap v449: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T18:21:20.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:21:20 smithi079 ceph-mon[35312]: pgmap v449: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T18:21:21.031 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:21:20 smithi022 ceph-mon[31985]: pgmap v449: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T18:21:22.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:21:22 smithi134 ceph-mon[35037]: pgmap v450: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 7 op/s 2024-03-28T18:21:22.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:21:22 smithi079 ceph-mon[35312]: pgmap v450: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 7 op/s 2024-03-28T18:21:22.936 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-28T18:21:22.936 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-28T18:21:23.028 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:21:22 smithi022 ceph-mon[31985]: pgmap v450: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 7 op/s 2024-03-28T18:21:23.176 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:21:23.177+0000 7f842cc00640 1 -- 172.21.15.22:0/1474289744 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f842810baf0 msgr2=0x7f842810df90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:21:23.177 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:21:23.177+0000 7f842cc00640 1 --2- 172.21.15.22:0/1474289744 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f842810baf0 0x7f842810df90 secure :-1 s=READY pgs=827 cs=0 l=1 rev1=1 crypto rx=0x7f8420009900 tx=0x7f842002bcd0 comp rx=0 tx=0).stop 2024-03-28T18:21:23.177 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:21:23.177+0000 7f842cc00640 1 -- 172.21.15.22:0/1474289744 shutdown_connections 2024-03-28T18:21:23.177 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:21:23.177+0000 7f842cc00640 1 --2- 172.21.15.22:0/1474289744 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f842810baf0 0x7f842810df90 unknown :-1 s=CLOSED pgs=827 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:21:23.177 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:21:23.177+0000 7f842cc00640 1 --2- 172.21.15.22:0/1474289744 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f8428102ff0 0x7f842810b4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:21:23.177 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:21:23.177+0000 7f842cc00640 1 --2- 172.21.15.22:0/1474289744 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f84281026d0 0x7f8428102ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:21:23.177 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:21:23.177+0000 7f842cc00640 1 -- 172.21.15.22:0/1474289744 >> 172.21.15.22:0/1474289744 conn(0x7f84280fbef0 msgr2=0x7f84280fe310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:21:23.177 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:21:23.177+0000 7f842cc00640 1 -- 172.21.15.22:0/1474289744 shutdown_connections 2024-03-28T18:21:23.177 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:21:23.177+0000 7f842cc00640 1 -- 172.21.15.22:0/1474289744 wait complete. 2024-03-28T18:21:23.178 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:21:23.178+0000 7f842cc00640 1 Processor -- start 2024-03-28T18:21:23.178 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:21:23.178+0000 7f842cc00640 1 -- start start 2024-03-28T18:21:23.180 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:21:23.178+0000 7f842cc00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f84281026d0 0x7f8428199f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:21:23.180 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:21:23.178+0000 7f842cc00640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f8428102ff0 0x7f842819a4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:21:23.180 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:21:23.178+0000 7f842cc00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f842810baf0 0x7f842819e830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:21:23.180 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:21:23.178+0000 7f842cc00640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84281118f0 con 0x7f842810baf0 2024-03-28T18:21:23.180 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:21:23.178+0000 7f842cc00640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8428111770 con 0x7f8428102ff0 2024-03-28T18:21:23.180 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:21:23.178+0000 7f842cc00640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8428111a70 con 0x7f84281026d0 2024-03-28T18:21:23.180 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:21:23.179+0000 7f8427e00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f842810baf0 0x7f842819e830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:21:23.180 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:21:23.179+0000 7f8427e00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f842810baf0 0x7f842819e830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:34228/0 (socket says 172.21.15.22:34228) 2024-03-28T18:21:23.180 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:21:23.179+0000 7f8427e00640 1 -- 172.21.15.22:0/479973671 learned_addr learned my addr 172.21.15.22:0/479973671 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-28T18:21:23.180 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:21:23.179+0000 7f8426a00640 1 --2- 172.21.15.22:0/479973671 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f8428102ff0 0x7f842819a4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:21:23.180 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:21:23.179+0000 7f8427400640 1 --2- 172.21.15.22:0/479973671 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f84281026d0 0x7f8428199f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:21:23.181 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:21:23.179+0000 7f8427e00640 1 -- 172.21.15.22:0/479973671 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f84281026d0 msgr2=0x7f8428199f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:21:23.181 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:21:23.179+0000 7f8427e00640 1 --2- 172.21.15.22:0/479973671 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f84281026d0 0x7f8428199f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:21:23.181 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:21:23.179+0000 7f8427e00640 1 -- 172.21.15.22:0/479973671 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f8428102ff0 msgr2=0x7f842819a4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:21:23.181 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:21:23.179+0000 7f8427e00640 1 --2- 172.21.15.22:0/479973671 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f8428102ff0 0x7f842819a4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:21:23.181 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:21:23.179+0000 7f8427e00640 1 -- 172.21.15.22:0/479973671 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f842819ef60 con 0x7f842810baf0 2024-03-28T18:21:23.181 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:21:23.179+0000 7f8426a00640 1 --2- 172.21.15.22:0/479973671 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f8428102ff0 0x7f842819a4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-28T18:21:23.181 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:21:23.180+0000 7f8427e00640 1 --2- 172.21.15.22:0/479973671 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f842810baf0 0x7f842819e830 secure :-1 s=READY pgs=828 cs=0 l=1 rev1=1 crypto rx=0x7f8420006010 tx=0x7f8420004600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:21:23.181 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:21:23.180+0000 7f841fe00640 1 -- 172.21.15.22:0/479973671 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8420042020 con 0x7f842810baf0 2024-03-28T18:21:23.181 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:21:23.180+0000 7f841fe00640 1 -- 172.21.15.22:0/479973671 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f84200053f0 con 0x7f842810baf0 2024-03-28T18:21:23.181 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:21:23.180+0000 7f842cc00640 1 -- 172.21.15.22:0/479973671 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f842819f1f0 con 0x7f842810baf0 2024-03-28T18:21:23.181 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:21:23.180+0000 7f841fe00640 1 -- 172.21.15.22:0/479973671 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8420003a10 con 0x7f842810baf0 2024-03-28T18:21:23.181 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:21:23.180+0000 7f842cc00640 1 -- 172.21.15.22:0/479973671 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f84281a6ae0 con 0x7f842810baf0 2024-03-28T18:21:23.183 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:21:23.183+0000 7f8430e69740 1 Processor -- start 2024-03-28T18:21:23.183 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:21:23.183+0000 7f841fe00640 1 -- 172.21.15.22:0/479973671 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 26) v1 ==== 137421+0+0 (secure 0 0 0) 0x7f8420045020 con 0x7f842810baf0 2024-03-28T18:21:23.183 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:21:23.183+0000 7f8430e69740 1 -- start start 2024-03-28T18:21:23.185 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:21:23.184+0000 7f8430e69740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5624f7619dd0 0x5624f7618b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:21:23.185 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:21:23.184+0000 7f8430e69740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5624f76190f0 0x5624f7619570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:21:23.185 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:21:23.184+0000 7f8430e69740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5624f7622460 0x5624f7624850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:21:23.185 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:21:23.184+0000 7f8430e69740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x5624f757d250 con 0x5624f7619dd0 2024-03-28T18:21:23.185 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:21:23.184+0000 7f8430e69740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x5624f7525a30 con 0x5624f7622460 2024-03-28T18:21:23.186 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:21:23.184+0000 7f8430e69740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x5624f75792f0 con 0x5624f76190f0 2024-03-28T18:21:23.186 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:21:23.184+0000 7f8426a00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5624f7619dd0 0x5624f7618b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:21:23.186 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:21:23.184+0000 7f8426a00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5624f7619dd0 0x5624f7618b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:34236/0 (socket says 172.21.15.22:34236) 2024-03-28T18:21:23.186 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:21:23.184+0000 7f8426a00640 1 -- 172.21.15.22:0/4154785355 learned_addr learned my addr 172.21.15.22:0/4154785355 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-28T18:21:23.186 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:21:23.184+0000 7f8427400640 1 --2- 172.21.15.22:0/4154785355 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5624f76190f0 0x5624f7619570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:21:23.186 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:21:23.185+0000 7f8426a00640 1 --2- 172.21.15.22:0/4154785355 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5624f7622460 0x5624f7624850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:21:23.186 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:21:23.185+0000 7f841fe00640 1 --2- 172.21.15.22:0/479973671 >> [v2:172.21.15.22:6800/1357365082,v1:172.21.15.22:6801/1357365082] conn(0x7f840c0a2970 0x7f840c0a4e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:21:23.186 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:21:23.185+0000 7f8426a00640 1 -- 172.21.15.22:0/4154785355 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5624f76190f0 msgr2=0x5624f7619570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T18:21:23.186 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:21:23.185+0000 7f8426a00640 1 --2- 172.21.15.22:0/4154785355 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5624f76190f0 0x5624f7619570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:21:23.186 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:21:23.185+0000 7f8426a00640 1 -- 172.21.15.22:0/4154785355 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5624f7622460 msgr2=0x5624f7624850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T18:21:23.186 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:21:23.185+0000 7f8426a00640 1 --2- 172.21.15.22:0/4154785355 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5624f7622460 0x5624f7624850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:21:23.186 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:21:23.185+0000 7f8426a00640 1 -- 172.21.15.22:0/4154785355 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5624f7510690 con 0x5624f7619dd0 2024-03-28T18:21:23.186 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:21:23.185+0000 7f841fe00640 1 -- 172.21.15.22:0/479973671 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(122..122 src has 1..122) v4 ==== 10389+0+0 (secure 0 0 0) 0x7f84200f13b0 con 0x7f842810baf0 2024-03-28T18:21:23.186 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:21:23.185+0000 7f8427e00640 1 --2- 172.21.15.22:0/479973671 >> [v2:172.21.15.22:6800/1357365082,v1:172.21.15.22:6801/1357365082] conn(0x7f840c0a2970 0x7f840c0a4e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:21:23.187 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:21:23.185+0000 7f8426a00640 1 --2- 172.21.15.22:0/4154785355 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5624f7619dd0 0x5624f7618b20 secure :-1 s=READY pgs=829 cs=0 l=1 rev1=1 crypto rx=0x7f84100135d0 tx=0x7f8410009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cf6474f95ae8e1d8 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:21:23.187 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:21:23.185+0000 7f8417e00640 1 -- 172.21.15.22:0/4154785355 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8410011470 con 0x5624f7619dd0 2024-03-28T18:21:23.187 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:21:23.186+0000 7f8417e00640 1 -- 172.21.15.22:0/4154785355 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f8410011ae0 con 0x5624f7619dd0 2024-03-28T18:21:23.187 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:21:23.186+0000 7f8417e00640 1 -- 172.21.15.22:0/4154785355 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f841000f990 con 0x5624f7619dd0 2024-03-28T18:21:23.190 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:21:23.186+0000 7f8430e69740 1 -- 172.21.15.22:0/4154785355 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5624f7619dd0 msgr2=0x5624f7618b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:21:23.191 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:21:23.186+0000 7f8427e00640 1 --2- 172.21.15.22:0/479973671 >> [v2:172.21.15.22:6800/1357365082,v1:172.21.15.22:6801/1357365082] conn(0x7f840c0a2970 0x7f840c0a4e30 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f84200b9d40 tx=0x7f84200dd040 comp rx=0 tx=0).ready entity=mgr.15522 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:21:23.191 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:21:23.186+0000 7f8430e69740 1 --2- 172.21.15.22:0/4154785355 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5624f7619dd0 0x5624f7618b20 secure :-1 s=READY pgs=829 cs=0 l=1 rev1=1 crypto rx=0x7f84100135d0 tx=0x7f8410009750 comp rx=0 tx=0).stop 2024-03-28T18:21:23.191 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:21:23.186+0000 7f8430e69740 1 -- 172.21.15.22:0/4154785355 shutdown_connections 2024-03-28T18:21:23.191 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:21:23.186+0000 7f8430e69740 1 --2- 172.21.15.22:0/4154785355 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5624f7622460 0x5624f7624850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:21:23.191 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:21:23.186+0000 7f8430e69740 1 --2- 172.21.15.22:0/4154785355 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5624f76190f0 0x5624f7619570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:21:23.191 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:21:23.186+0000 7f8430e69740 1 --2- 172.21.15.22:0/4154785355 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5624f7619dd0 0x5624f7618b20 unknown :-1 s=CLOSED pgs=829 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:21:23.191 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:21:23.186+0000 7f8430e69740 1 -- 172.21.15.22:0/4154785355 >> 172.21.15.22:0/4154785355 conn(0x5624f75e0200 msgr2=0x5624f7613b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:21:23.191 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:21:23.186+0000 7f8430e69740 1 -- 172.21.15.22:0/4154785355 shutdown_connections 2024-03-28T18:21:23.191 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:21:23.187+0000 7f8430e69740 1 -- 172.21.15.22:0/4154785355 wait complete. 2024-03-28T18:21:23.191 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:21:23.187+0000 7f8430e69740 1 -- start start 2024-03-28T18:21:23.191 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:21:23.187+0000 7f8430e69740 1 Processor -- start 2024-03-28T18:21:23.191 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:21:23.188+0000 7f8430e69740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5624f76190f0 0x5624f7621ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:21:23.191 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:21:23.188+0000 7f8430e69740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5624f7619dd0 0x5624f761ee60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:21:23.191 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:21:23.188+0000 7f8430e69740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5624f7622460 0x5624f761f3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:21:23.192 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:21:23.188+0000 7f8430e69740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x5624f76252d0 con 0x5624f7619dd0 2024-03-28T18:21:23.192 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:21:23.188+0000 7f8430e69740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x5624f7538e80 con 0x5624f76190f0 2024-03-28T18:21:23.192 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:21:23.188+0000 7f8430e69740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x5624f7625450 con 0x5624f7622460 2024-03-28T18:21:23.192 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:21:23.188+0000 7f8430e69740 10 client.0 _ll_register_callbacks cb 0x5624f754b130 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-28T18:21:23.192 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:21:23.188+0000 7f8427400640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5624f7619dd0 0x5624f761ee60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:21:23.192 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:21:23.188+0000 7f8427400640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5624f7619dd0 0x5624f761ee60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:34248/0 (socket says 172.21.15.22:34248) 2024-03-28T18:21:23.192 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:21:23.188+0000 7f8427400640 1 -- 172.21.15.22:0/3851639100 learned_addr learned my addr 172.21.15.22:0/3851639100 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-28T18:21:23.192 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:21:23.188+0000 7f8426a00640 1 --2- 172.21.15.22:0/3851639100 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5624f76190f0 0x5624f7621ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:21:23.192 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:21:23.189+0000 7f8426a00640 1 --2- 172.21.15.22:0/3851639100 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5624f7622460 0x5624f761f3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:21:23.192 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:21:23.189+0000 7f8427400640 1 -- 172.21.15.22:0/3851639100 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5624f7622460 msgr2=0x5624f761f3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T18:21:23.192 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:21:23.189+0000 7f8427400640 1 --2- 172.21.15.22:0/3851639100 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5624f7622460 0x5624f761f3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:21:23.192 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:21:23.189+0000 7f8427400640 1 -- 172.21.15.22:0/3851639100 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5624f76190f0 msgr2=0x5624f7621ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T18:21:23.192 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:21:23.189+0000 7f8427400640 1 --2- 172.21.15.22:0/3851639100 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5624f76190f0 0x5624f7621ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:21:23.192 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:21:23.189+0000 7f8427400640 1 -- 172.21.15.22:0/3851639100 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5624f735b7d0 con 0x5624f7619dd0 2024-03-28T18:21:23.193 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:21:23.189+0000 7f8426a00640 1 --2- 172.21.15.22:0/3851639100 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5624f7622460 0x5624f761f3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T18:21:23.193 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:21:23.189+0000 7f8427400640 1 --2- 172.21.15.22:0/3851639100 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5624f7619dd0 0x5624f761ee60 secure :-1 s=READY pgs=830 cs=0 l=1 rev1=1 crypto rx=0x7f8418011910 tx=0x7f8418011dd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f1ec59ffe063ce4a server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:21:23.193 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:21:23.189+0000 7f8414c00640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-03-28T18:21:23.193 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:21:23.189+0000 7f8414c00640 1 -- 172.21.15.22:0/3851639100 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f841800fc00 con 0x5624f7619dd0 2024-03-28T18:21:23.193 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:21:23.189+0000 7f8414c00640 1 -- 172.21.15.22:0/3851639100 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(122..122 src has 1..122) v4 ==== 10389+0+0 (secure 0 0 0) 0x7f841800fda0 con 0x5624f7619dd0 2024-03-28T18:21:23.193 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:21:23.189+0000 7f8430e69740 1 -- 172.21.15.22:0/3851639100 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f8410011ae0 con 0x5624f7619dd0 2024-03-28T18:21:23.193 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:21:23.190+0000 7f8414c00640 1 -- 172.21.15.22:0/3851639100 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=189) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8418012450 con 0x5624f7619dd0 2024-03-28T18:21:23.193 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:21:23.190+0000 7f8430e69740 10 client.16962 fetch_fsmap learned FSMap version 189 2024-03-28T18:21:23.193 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:21:23.190+0000 7f8430e69740 1 -- 172.21.15.22:0/3851639100 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=189,monmap=4+}) v3 -- 0x5624f72913f0 con 0x5624f7619dd0 2024-03-28T18:21:23.193 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:21:23.190+0000 7f8414c00640 1 -- 172.21.15.22:0/3851639100 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f8410011ae0 con 0x5624f7619dd0 2024-03-28T18:21:23.193 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:21:23.190+0000 7f8414c00640 1 -- 172.21.15.22:0/3851639100 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 189) v1 ==== 6221+0+0 (secure 0 0 0) 0x7f841800f040 con 0x5624f7619dd0 2024-03-28T18:21:23.193 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:21:23.190+0000 7f8430e69740 10 client.16962 fetch_fsmap finished waiting for FSMap version 189 2024-03-28T18:21:23.193 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:21:23.190+0000 7f8430e69740 10 client.16962 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-28T18:21:23.193 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:21:23.190+0000 7f8430e69740 20 client.16962 populate_metadata read hostname 'smithi022' 2024-03-28T18:21:23.193 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:21:23.190+0000 7f8430e69740 1 --2- 172.21.15.22:0/3851639100 >> [v2:172.21.15.79:6838/2441542443,v1:172.21.15.79:6839/2441542443] conn(0x5624f7632690 0x5624f7634af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:21:23.194 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:21:23.190+0000 7f8430e69740 4 client.16962 mds_command: new command op to 34553 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-28T18:21:23.194 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:21:23.190+0000 7f8430e69740 1 -- 172.21.15.22:0/3851639100 --> [v2:172.21.15.79:6838/2441542443,v1:172.21.15.79:6839/2441542443] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5624f758a790 con 0x5624f7632690 2024-03-28T18:21:23.194 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:21:23.191+0000 7f8426a00640 1 --2- 172.21.15.22:0/3851639100 >> [v2:172.21.15.79:6838/2441542443,v1:172.21.15.79:6839/2441542443] conn(0x5624f7632690 0x5624f7634af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:21:23.195 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:21:23.192+0000 7f8426a00640 1 --2- 172.21.15.22:0/3851639100 >> [v2:172.21.15.79:6838/2441542443,v1:172.21.15.79:6839/2441542443] conn(0x5624f7632690 0x5624f7634af0 crc :-1 s=READY pgs=321 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=79649b25127a8c75 server_cookie=6adfa9a1d876a4b3 in_seq=0 out_seq=0 2024-03-28T18:21:23.195 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:21:23.192+0000 7f8414c00640 10 client.16962 ms_handle_connect on v2:172.21.15.79:6838/2441542443 2024-03-28T18:21:23.195 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:21:23.194+0000 7f8414c00640 1 -- 172.21.15.22:0/3851639100 <== mds.0 v2:172.21.15.79:6838/2441542443 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x5624f758a790 con 0x5624f7632690 2024-03-28T18:21:23.195 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:21:23.194+0000 7f8414c00640 10 client.16962 handle_command_reply: tid=0 2024-03-28T18:21:23.195 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:21:23.194+0000 7f8430e69740 1 client.16962 shutdown 2024-03-28T18:21:23.195 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:21:23.194+0000 7f8430e69740 1 -- 172.21.15.22:0/3851639100 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5624f7619dd0 msgr2=0x5624f761ee60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:21:23.196 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:21:23.194+0000 7f8430e69740 1 --2- 172.21.15.22:0/3851639100 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5624f7619dd0 0x5624f761ee60 secure :-1 s=READY pgs=830 cs=0 l=1 rev1=1 crypto rx=0x7f8418011910 tx=0x7f8418011dd0 comp rx=0 tx=0).stop 2024-03-28T18:21:23.196 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:21:23.194+0000 7f8430e69740 1 -- 172.21.15.22:0/3851639100 shutdown_connections 2024-03-28T18:21:23.196 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:21:23.194+0000 7f8430e69740 1 --2- 172.21.15.22:0/3851639100 >> [v2:172.21.15.79:6838/2441542443,v1:172.21.15.79:6839/2441542443] conn(0x5624f7632690 0x5624f7634af0 crc :-1 s=READY pgs=321 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:21:23.196 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:21:23.194+0000 7f8430e69740 1 --2- 172.21.15.22:0/3851639100 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5624f7622460 0x5624f761f3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:21:23.196 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:21:23.194+0000 7f8430e69740 1 --2- 172.21.15.22:0/3851639100 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5624f7619dd0 0x5624f761ee60 unknown :-1 s=CLOSED pgs=830 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:21:23.196 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:21:23.194+0000 7f8414c00640 0 client.16962 ms_handle_reset on v2:172.21.15.79:6838/2441542443 2024-03-28T18:21:23.196 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:21:23.194+0000 7f8430e69740 1 --2- 172.21.15.22:0/3851639100 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5624f76190f0 0x5624f7621ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:21:23.196 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:21:23.194+0000 7f8430e69740 1 -- 172.21.15.22:0/3851639100 >> 172.21.15.22:0/3851639100 conn(0x5624f75e0200 msgr2=0x5624f7615000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:21:23.196 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:21:23.194+0000 7f8430e69740 1 -- 172.21.15.22:0/3851639100 shutdown_connections 2024-03-28T18:21:23.196 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:21:23.194+0000 7f8430e69740 1 -- 172.21.15.22:0/3851639100 wait complete. 2024-03-28T18:21:23.196 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:21:23.194+0000 7f8430e69740 20 client.16962 trim_cache size 0 max 16384 2024-03-28T18:21:23.212 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:21:23.212+0000 7f8430e69740 1 Processor -- start 2024-03-28T18:21:23.212 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:21:23.212+0000 7f8430e69740 1 -- start start 2024-03-28T18:21:23.212 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:21:23.212+0000 7f8430e69740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5624f76190f0 0x5624f760e920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:21:23.212 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:21:23.212+0000 7f8430e69740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5624f7619dd0 0x5624f7623850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:21:23.212 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:21:23.212+0000 7f8430e69740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5624f7622460 0x5624f7621590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:21:23.212 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:21:23.212+0000 7f8430e69740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84281b1c20 con 0x5624f7622460 2024-03-28T18:21:23.212 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:21:23.212+0000 7f8430e69740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84281aef80 con 0x5624f7619dd0 2024-03-28T18:21:23.213 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:21:23.212+0000 7f8430e69740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84281b0f60 con 0x5624f76190f0 2024-03-28T18:21:23.214 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:21:23.213+0000 7f8427400640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5624f7619dd0 0x5624f7623850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:21:23.214 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:21:23.213+0000 7f8427400640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5624f7619dd0 0x5624f7623850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.22:56420/0 (socket says 172.21.15.22:56420) 2024-03-28T18:21:23.214 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:21:23.213+0000 7f8427400640 1 -- 172.21.15.22:0/3914496032 learned_addr learned my addr 172.21.15.22:0/3914496032 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-28T18:21:23.214 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:21:23.213+0000 7f8426a00640 1 --2- 172.21.15.22:0/3914496032 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5624f7622460 0x5624f7621590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:21:23.214 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:21:23.213+0000 7f8426a00640 1 --2- 172.21.15.22:0/3914496032 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5624f76190f0 0x5624f760e920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:21:23.214 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:21:23.213+0000 7f8427400640 1 -- 172.21.15.22:0/3914496032 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5624f76190f0 msgr2=0x5624f760e920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T18:21:23.214 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:21:23.213+0000 7f8427400640 1 --2- 172.21.15.22:0/3914496032 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5624f76190f0 0x5624f760e920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:21:23.215 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:21:23.213+0000 7f8427400640 1 -- 172.21.15.22:0/3914496032 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5624f7622460 msgr2=0x5624f7621590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T18:21:23.215 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:21:23.213+0000 7f8427400640 1 --2- 172.21.15.22:0/3914496032 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5624f7622460 0x5624f7621590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:21:23.215 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:21:23.213+0000 7f8427400640 1 -- 172.21.15.22:0/3914496032 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5624f72510f0 con 0x5624f7619dd0 2024-03-28T18:21:23.215 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:21:23.213+0000 7f8426a00640 1 --2- 172.21.15.22:0/3914496032 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5624f7622460 0x5624f7621590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-28T18:21:23.215 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:21:23.214+0000 7f8427400640 1 --2- 172.21.15.22:0/3914496032 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5624f7619dd0 0x5624f7623850 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f841801b860 tx=0x7f84180099b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3068ab910e0d7fde server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:21:23.215 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:21:23.214+0000 7f8417400640 1 -- 172.21.15.22:0/3914496032 <== mon.2 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8418012ce0 con 0x5624f7619dd0 2024-03-28T18:21:23.215 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:21:23.214+0000 7f8417400640 1 -- 172.21.15.22:0/3914496032 <== mon.2 v2:172.21.15.79:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f841801b430 con 0x5624f7619dd0 2024-03-28T18:21:23.215 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:21:23.214+0000 7f8417400640 1 -- 172.21.15.22:0/3914496032 <== mon.2 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8418003970 con 0x5624f7619dd0 2024-03-28T18:21:23.217 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:21:23.215+0000 7f8430e69740 1 -- 172.21.15.22:0/3914496032 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5624f7619dd0 msgr2=0x5624f7623850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:21:23.218 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:21:23.215+0000 7f8430e69740 1 --2- 172.21.15.22:0/3914496032 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5624f7619dd0 0x5624f7623850 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f841801b860 tx=0x7f84180099b0 comp rx=0 tx=0).stop 2024-03-28T18:21:23.218 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:21:23.215+0000 7f8430e69740 1 -- 172.21.15.22:0/3914496032 shutdown_connections 2024-03-28T18:21:23.218 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:21:23.215+0000 7f8430e69740 1 --2- 172.21.15.22:0/3914496032 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5624f7622460 0x5624f7621590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:21:23.218 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:21:23.215+0000 7f8430e69740 1 --2- 172.21.15.22:0/3914496032 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5624f7619dd0 0x5624f7623850 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:21:23.218 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:21:23.215+0000 7f8430e69740 1 --2- 172.21.15.22:0/3914496032 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5624f76190f0 0x5624f760e920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:21:23.218 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:21:23.215+0000 7f8430e69740 1 -- 172.21.15.22:0/3914496032 >> 172.21.15.22:0/3914496032 conn(0x5624f75e0200 msgr2=0x5624f761d730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:21:23.218 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:21:23.215+0000 7f8430e69740 1 -- 172.21.15.22:0/3914496032 shutdown_connections 2024-03-28T18:21:23.218 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:21:23.215+0000 7f8430e69740 1 -- 172.21.15.22:0/3914496032 wait complete. 2024-03-28T18:21:23.218 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:21:23.216+0000 7f8430e69740 1 -- start start 2024-03-28T18:21:23.218 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:21:23.216+0000 7f8430e69740 1 Processor -- start 2024-03-28T18:21:23.218 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:21:23.216+0000 7f8430e69740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5624f76190f0 0x5624f76261e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:21:23.218 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:21:23.216+0000 7f8430e69740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5624f7619dd0 0x5624f7626720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:21:23.218 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:21:23.216+0000 7f8430e69740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5624f7622460 0x5624f76957d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:21:23.219 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:21:23.216+0000 7f8430e69740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84281b0900 con 0x5624f76190f0 2024-03-28T18:21:23.219 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:21:23.216+0000 7f8430e69740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84281af5e0 con 0x5624f7619dd0 2024-03-28T18:21:23.219 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:21:23.216+0000 7f8430e69740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84281afc40 con 0x5624f7622460 2024-03-28T18:21:23.219 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:21:23.216+0000 7f8430e69740 10 client.0 _ll_register_callbacks cb 0x7f8418009740 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-28T18:21:23.219 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:21:23.217+0000 7f8426a00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5624f76190f0 0x5624f76261e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:21:23.219 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:21:23.217+0000 7f8426a00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5624f76190f0 0x5624f76261e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:34270/0 (socket says 172.21.15.22:34270) 2024-03-28T18:21:23.219 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:21:23.217+0000 7f8426a00640 1 -- 172.21.15.22:0/4202426142 learned_addr learned my addr 172.21.15.22:0/4202426142 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-28T18:21:23.219 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:21:23.217+0000 7f8427400640 1 --2- 172.21.15.22:0/4202426142 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5624f7619dd0 0x5624f7626720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:21:23.219 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:21:23.217+0000 7f8426a00640 1 -- 172.21.15.22:0/4202426142 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5624f7622460 msgr2=0x5624f76957d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T18:21:23.219 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:21:23.217+0000 7f8426a00640 1 --2- 172.21.15.22:0/4202426142 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5624f7622460 0x5624f76957d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:21:23.219 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:21:23.217+0000 7f8426a00640 1 -- 172.21.15.22:0/4202426142 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5624f7619dd0 msgr2=0x5624f7626720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T18:21:23.219 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:21:23.217+0000 7f8426a00640 1 --2- 172.21.15.22:0/4202426142 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5624f7619dd0 0x5624f7626720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:21:23.219 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:21:23.217+0000 7f8426a00640 1 -- 172.21.15.22:0/4202426142 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5624f7695d10 con 0x5624f76190f0 2024-03-28T18:21:23.219 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:21:23.217+0000 7f8426a00640 1 --2- 172.21.15.22:0/4202426142 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5624f76190f0 0x5624f76261e0 secure :-1 s=READY pgs=831 cs=0 l=1 rev1=1 crypto rx=0x7f8410009cc0 tx=0x7f84100058f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b7f478c6c67b8868 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:21:23.219 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:21:23.217+0000 7f8414c00640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-03-28T18:21:23.220 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:21:23.218+0000 7f8414c00640 1 -- 172.21.15.22:0/4202426142 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8410005240 con 0x5624f76190f0 2024-03-28T18:21:23.220 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:21:23.218+0000 7f8414c00640 1 -- 172.21.15.22:0/4202426142 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(122..122 src has 1..122) v4 ==== 10389+0+0 (secure 0 0 0) 0x7f84100053e0 con 0x5624f76190f0 2024-03-28T18:21:23.220 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:21:23.218+0000 7f8430e69740 1 -- 172.21.15.22:0/4202426142 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f841801b430 con 0x5624f76190f0 2024-03-28T18:21:23.220 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:21:23.218+0000 7f8414c00640 1 -- 172.21.15.22:0/4202426142 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=189) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8410038050 con 0x5624f76190f0 2024-03-28T18:21:23.220 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:21:23.218+0000 7f8430e69740 10 client.16968 fetch_fsmap learned FSMap version 189 2024-03-28T18:21:23.220 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:21:23.218+0000 7f8430e69740 1 -- 172.21.15.22:0/4202426142 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=189,monmap=4+}) v3 -- 0x5624f7696f90 con 0x5624f76190f0 2024-03-28T18:21:23.222 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:21:23.218+0000 7f8414c00640 1 -- 172.21.15.22:0/4202426142 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f841801b430 con 0x5624f76190f0 2024-03-28T18:21:23.222 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:21:23.218+0000 7f8414c00640 1 -- 172.21.15.22:0/4202426142 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 189) v1 ==== 6221+0+0 (secure 0 0 0) 0x7f841002d920 con 0x5624f76190f0 2024-03-28T18:21:23.222 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:21:23.219+0000 7f8430e69740 10 client.16968 fetch_fsmap finished waiting for FSMap version 189 2024-03-28T18:21:23.222 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:21:23.219+0000 7f8430e69740 10 client.16968 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-28T18:21:23.222 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:21:23.219+0000 7f8430e69740 20 client.16968 populate_metadata read hostname 'smithi022' 2024-03-28T18:21:23.222 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:21:23.219+0000 7f8430e69740 1 --2- 172.21.15.22:0/4202426142 >> [v2:172.21.15.79:6838/2441542443,v1:172.21.15.79:6839/2441542443] conn(0x5624f7697600 0x5624f7699a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:21:23.222 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:21:23.219+0000 7f8430e69740 4 client.16968 mds_command: new command op to 34553 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-28T18:21:23.222 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:21:23.219+0000 7f8430e69740 1 -- 172.21.15.22:0/4202426142 --> [v2:172.21.15.79:6838/2441542443,v1:172.21.15.79:6839/2441542443] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x5624f7584ae0 con 0x5624f7697600 2024-03-28T18:21:23.222 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:21:23.219+0000 7f8427400640 1 --2- 172.21.15.22:0/4202426142 >> [v2:172.21.15.79:6838/2441542443,v1:172.21.15.79:6839/2441542443] conn(0x5624f7697600 0x5624f7699a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:21:23.222 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:21:23.220+0000 7f8427400640 1 --2- 172.21.15.22:0/4202426142 >> [v2:172.21.15.79:6838/2441542443,v1:172.21.15.79:6839/2441542443] conn(0x5624f7697600 0x5624f7699a20 crc :-1 s=READY pgs=322 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bbe5db386fa215c2 server_cookie=e2e9edbbcc496597 in_seq=0 out_seq=0 2024-03-28T18:21:23.222 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:21:23.220+0000 7f8414c00640 10 client.16968 ms_handle_connect on v2:172.21.15.79:6838/2441542443 2024-03-28T18:21:23.222 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:21:23.220+0000 7f8414c00640 1 -- 172.21.15.22:0/4202426142 <== mds.0 v2:172.21.15.79:6838/2441542443 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x5624f7584ae0 con 0x5624f7697600 2024-03-28T18:21:23.223 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:21:23.220+0000 7f8414c00640 10 client.16968 handle_command_reply: tid=0 2024-03-28T18:21:23.223 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:21:23.220+0000 7f8430e69740 1 client.16968 shutdown 2024-03-28T18:21:23.223 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:21:23.220+0000 7f8430e69740 1 -- 172.21.15.22:0/4202426142 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5624f76190f0 msgr2=0x5624f76261e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:21:23.223 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:21:23.220+0000 7f8430e69740 1 --2- 172.21.15.22:0/4202426142 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5624f76190f0 0x5624f76261e0 secure :-1 s=READY pgs=831 cs=0 l=1 rev1=1 crypto rx=0x7f8410009cc0 tx=0x7f84100058f0 comp rx=0 tx=0).stop 2024-03-28T18:21:23.223 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:21:23.221+0000 7f8430e69740 1 -- 172.21.15.22:0/4202426142 shutdown_connections 2024-03-28T18:21:23.223 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:21:23.221+0000 7f8430e69740 1 --2- 172.21.15.22:0/4202426142 >> [v2:172.21.15.79:6838/2441542443,v1:172.21.15.79:6839/2441542443] conn(0x5624f7697600 0x5624f7699a20 crc :-1 s=READY pgs=322 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:21:23.223 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:21:23.221+0000 7f8430e69740 1 --2- 172.21.15.22:0/4202426142 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5624f7622460 0x5624f76957d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:21:23.223 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:21:23.221+0000 7f8430e69740 1 --2- 172.21.15.22:0/4202426142 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5624f7619dd0 0x5624f7626720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:21:23.223 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:21:23.221+0000 7f8430e69740 1 --2- 172.21.15.22:0/4202426142 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5624f76190f0 0x5624f76261e0 unknown :-1 s=CLOSED pgs=831 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:21:23.223 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:21:23.221+0000 7f8414c00640 0 client.16968 ms_handle_reset on v2:172.21.15.79:6838/2441542443 2024-03-28T18:21:23.223 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:21:23.221+0000 7f8430e69740 1 -- 172.21.15.22:0/4202426142 >> 172.21.15.22:0/4202426142 conn(0x5624f75e0200 msgr2=0x5624f7612330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:21:23.223 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:21:23.221+0000 7f8430e69740 1 -- 172.21.15.22:0/4202426142 shutdown_connections 2024-03-28T18:21:23.223 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:21:23.221+0000 7f8430e69740 1 -- 172.21.15.22:0/4202426142 wait complete. 2024-03-28T18:21:23.223 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:21:23.221+0000 7f8430e69740 20 client.16968 trim_cache size 0 max 16384 2024-03-28T18:21:23.224 INFO:teuthology.orchestra.run.smithi022.stdout:{ 2024-03-28T18:21:23.224 INFO:teuthology.orchestra.run.smithi022.stdout: "status": "IDLE (waiting for more scrubs)", 2024-03-28T18:21:23.224 INFO:teuthology.orchestra.run.smithi022.stdout: "scrubs": { 2024-03-28T18:21:23.224 INFO:teuthology.orchestra.run.smithi022.stdout: "4bbbd61d-29a3-4bf6-a4f2-d5783d585de3": { 2024-03-28T18:21:23.224 INFO:teuthology.orchestra.run.smithi022.stdout: "path": "/", 2024-03-28T18:21:23.224 INFO:teuthology.orchestra.run.smithi022.stdout: "tag": "4bbbd61d-29a3-4bf6-a4f2-d5783d585de3", 2024-03-28T18:21:23.224 INFO:teuthology.orchestra.run.smithi022.stdout: "options": "recursive,force" 2024-03-28T18:21:23.224 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-03-28T18:21:23.224 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-03-28T18:21:23.224 INFO:teuthology.orchestra.run.smithi022.stdout:} 2024-03-28T18:21:23.225 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:21:23.221+0000 7f842cc00640 1 -- 172.21.15.22:0/479973671 >> [v2:172.21.15.22:6800/1357365082,v1:172.21.15.22:6801/1357365082] conn(0x7f840c0a2970 msgr2=0x7f840c0a4e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:21:23.225 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:21:23.221+0000 7f842cc00640 1 --2- 172.21.15.22:0/479973671 >> [v2:172.21.15.22:6800/1357365082,v1:172.21.15.22:6801/1357365082] conn(0x7f840c0a2970 0x7f840c0a4e30 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f84200b9d40 tx=0x7f84200dd040 comp rx=0 tx=0).stop 2024-03-28T18:21:23.225 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:21:23.221+0000 7f842cc00640 1 -- 172.21.15.22:0/479973671 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f842810baf0 msgr2=0x7f842819e830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:21:23.225 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:21:23.221+0000 7f842cc00640 1 --2- 172.21.15.22:0/479973671 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f842810baf0 0x7f842819e830 secure :-1 s=READY pgs=828 cs=0 l=1 rev1=1 crypto rx=0x7f8420006010 tx=0x7f8420004600 comp rx=0 tx=0).stop 2024-03-28T18:21:23.225 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:21:23.222+0000 7f842cc00640 1 -- 172.21.15.22:0/479973671 shutdown_connections 2024-03-28T18:21:23.225 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:21:23.222+0000 7f842cc00640 1 --2- 172.21.15.22:0/479973671 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f842810baf0 0x7f842819e830 unknown :-1 s=CLOSED pgs=828 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:21:23.225 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:21:23.222+0000 7f842cc00640 1 --2- 172.21.15.22:0/479973671 >> [v2:172.21.15.22:6800/1357365082,v1:172.21.15.22:6801/1357365082] conn(0x7f840c0a2970 0x7f840c0a4e30 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:21:23.226 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:21:23.222+0000 7f842cc00640 1 --2- 172.21.15.22:0/479973671 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f8428102ff0 0x7f842819a4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:21:23.226 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:21:23.222+0000 7f842cc00640 1 --2- 172.21.15.22:0/479973671 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f84281026d0 0x7f8428199f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:21:23.226 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:21:23.222+0000 7f842cc00640 1 -- 172.21.15.22:0/479973671 >> 172.21.15.22:0/479973671 conn(0x7f84280fbef0 msgr2=0x7f84280fcb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:21:23.226 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:21:23.222+0000 7f842cc00640 1 -- 172.21.15.22:0/479973671 shutdown_connections 2024-03-28T18:21:23.226 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:21:23.222+0000 7f842cc00640 1 -- 172.21.15.22:0/479973671 wait complete. 2024-03-28T18:21:23.239 INFO:tasks.cephfs.filesystem:scrub status for tag:4bbbd61d-29a3-4bf6-a4f2-d5783d585de3 - {'path': '/', 'tag': '4bbbd61d-29a3-4bf6-a4f2-d5783d585de3', 'options': 'recursive,force'} 2024-03-28T18:21:24.925 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:21:24 smithi134 ceph-mon[35037]: pgmap v451: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:21:24.925 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:21:24 smithi134 ceph-mon[35037]: 2 slow requests, 0 included below; oldest blocked for > 271.842939 secs 2024-03-28T18:21:24.933 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:21:24 smithi079 ceph-mon[35312]: pgmap v451: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:21:24.933 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:21:24 smithi079 ceph-mon[35312]: 2 slow requests, 0 included below; oldest blocked for > 271.842939 secs 2024-03-28T18:21:25.033 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:21:24 smithi022 ceph-mon[31985]: pgmap v451: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:21:25.033 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:21:24 smithi022 ceph-mon[31985]: 2 slow requests, 0 included below; oldest blocked for > 271.842939 secs 2024-03-28T18:21:26.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:21:26 smithi134 ceph-mon[35037]: pgmap v452: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:21:26.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:21:26 smithi079 ceph-mon[35312]: pgmap v452: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:21:27.032 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:21:26 smithi022 ceph-mon[31985]: pgmap v452: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:21:28.925 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:21:28 smithi134 ceph-mon[35037]: pgmap v453: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:21:28.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:21:28 smithi079 ceph-mon[35312]: pgmap v453: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:21:29.032 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:21:28 smithi022 ceph-mon[31985]: pgmap v453: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:21:29.925 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:21:29 smithi134 ceph-mon[35037]: 2 slow requests, 0 included below; oldest blocked for > 276.843005 secs 2024-03-28T18:21:29.933 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:21:29 smithi079 ceph-mon[35312]: 2 slow requests, 0 included below; oldest blocked for > 276.843005 secs 2024-03-28T18:21:30.032 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:21:29 smithi022 ceph-mon[31985]: 2 slow requests, 0 included below; oldest blocked for > 276.843005 secs 2024-03-28T18:21:30.925 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:21:30 smithi134 ceph-mon[35037]: pgmap v454: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-28T18:21:30.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:21:30 smithi079 ceph-mon[35312]: pgmap v454: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-28T18:21:31.032 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:21:30 smithi022 ceph-mon[31985]: pgmap v454: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-28T18:21:32.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:21:32 smithi134 ceph-mon[35037]: pgmap v455: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 7 op/s 2024-03-28T18:21:32.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:21:32 smithi079 ceph-mon[35312]: pgmap v455: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 7 op/s 2024-03-28T18:21:33.032 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:21:32 smithi022 ceph-mon[31985]: pgmap v455: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 7 op/s 2024-03-28T18:21:34.925 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:21:34 smithi134 ceph-mon[35037]: pgmap v456: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:21:34.925 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:21:34 smithi134 ceph-mon[35037]: 2 slow requests, 0 included below; oldest blocked for > 281.843081 secs 2024-03-28T18:21:34.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:21:34 smithi079 ceph-mon[35312]: pgmap v456: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:21:34.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:21:34 smithi079 ceph-mon[35312]: 2 slow requests, 0 included below; oldest blocked for > 281.843081 secs 2024-03-28T18:21:35.032 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:21:34 smithi022 ceph-mon[31985]: pgmap v456: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:21:35.032 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:21:34 smithi022 ceph-mon[31985]: 2 slow requests, 0 included below; oldest blocked for > 281.843081 secs 2024-03-28T18:21:36.925 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:21:36 smithi134 ceph-mon[35037]: pgmap v457: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:21:36.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:21:36 smithi079 ceph-mon[35312]: pgmap v457: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:21:37.031 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:21:36 smithi022 ceph-mon[31985]: pgmap v457: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:21:38.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:21:38 smithi134 ceph-mon[35037]: pgmap v458: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:21:38.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:21:38 smithi079 ceph-mon[35312]: pgmap v458: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:21:39.032 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:21:38 smithi022 ceph-mon[31985]: pgmap v458: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:21:39.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:21:39 smithi134 ceph-mon[35037]: 2 slow requests, 0 included below; oldest blocked for > 286.843161 secs 2024-03-28T18:21:39.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:21:39 smithi079 ceph-mon[35312]: 2 slow requests, 0 included below; oldest blocked for > 286.843161 secs 2024-03-28T18:21:40.032 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:21:39 smithi022 ceph-mon[31985]: 2 slow requests, 0 included below; oldest blocked for > 286.843161 secs 2024-03-28T18:21:40.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:21:40 smithi134 ceph-mon[35037]: pgmap v459: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-28T18:21:40.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:21:40 smithi079 ceph-mon[35312]: pgmap v459: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-28T18:21:41.032 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:21:40 smithi022 ceph-mon[31985]: pgmap v459: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-28T18:21:42.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:21:42 smithi134 ceph-mon[35037]: pgmap v460: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 7 op/s 2024-03-28T18:21:42.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:21:42 smithi079 ceph-mon[35312]: pgmap v460: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 7 op/s 2024-03-28T18:21:43.032 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:21:42 smithi022 ceph-mon[31985]: pgmap v460: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 7 op/s 2024-03-28T18:21:44.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:21:44 smithi134 ceph-mon[35037]: pgmap v461: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:21:44.925 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:21:44 smithi134 ceph-mon[35037]: 2 slow requests, 0 included below; oldest blocked for > 291.843190 secs 2024-03-28T18:21:44.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:21:44 smithi079 ceph-mon[35312]: pgmap v461: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:21:44.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:21:44 smithi079 ceph-mon[35312]: 2 slow requests, 0 included below; oldest blocked for > 291.843190 secs 2024-03-28T18:21:45.032 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:21:44 smithi022 ceph-mon[31985]: pgmap v461: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:21:45.032 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:21:44 smithi022 ceph-mon[31985]: 2 slow requests, 0 included below; oldest blocked for > 291.843190 secs 2024-03-28T18:21:46.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:21:46 smithi079 ceph-mon[35312]: pgmap v462: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:21:47.032 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:21:46 smithi022 ceph-mon[31985]: pgmap v462: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:21:47.175 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:21:46 smithi134 ceph-mon[35037]: pgmap v462: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:21:48.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:21:48 smithi079 ceph-mon[35312]: pgmap v463: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:21:49.032 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:21:48 smithi022 ceph-mon[31985]: pgmap v463: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:21:49.174 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:21:48 smithi134 ceph-mon[35037]: pgmap v463: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:21:50.032 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:21:49 smithi022 ceph-mon[31985]: 2 slow requests, 0 included below; oldest blocked for > 296.843311 secs 2024-03-28T18:21:50.174 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:21:49 smithi134 ceph-mon[35037]: 2 slow requests, 0 included below; oldest blocked for > 296.843311 secs 2024-03-28T18:21:50.182 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:21:49 smithi079 ceph-mon[35312]: 2 slow requests, 0 included below; oldest blocked for > 296.843311 secs 2024-03-28T18:21:51.032 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:21:50 smithi022 ceph-mon[31985]: pgmap v464: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-28T18:21:51.174 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:21:50 smithi134 ceph-mon[35037]: pgmap v464: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-28T18:21:51.182 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:21:50 smithi079 ceph-mon[35312]: pgmap v464: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-28T18:21:53.032 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:21:52 smithi022 ceph-mon[31985]: pgmap v465: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 7 op/s 2024-03-28T18:21:53.175 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:21:52 smithi134 ceph-mon[35037]: pgmap v465: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 7 op/s 2024-03-28T18:21:53.182 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:21:52 smithi079 ceph-mon[35312]: pgmap v465: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 7 op/s 2024-03-28T18:21:53.240 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-28T18:21:53.241 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-28T18:21:53.477 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:21:53.476+0000 7f4757400640 1 -- 172.21.15.22:0/2553331113 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f475010cf80 msgr2=0x7f4750114b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:21:53.477 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:21:53.476+0000 7f4757400640 1 --2- 172.21.15.22:0/2553331113 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f475010cf80 0x7f4750114b50 secure :-1 s=READY pgs=832 cs=0 l=1 rev1=1 crypto rx=0x7f4748009ca0 tx=0x7f474802c0f0 comp rx=0 tx=0).stop 2024-03-28T18:21:53.477 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:21:53.476+0000 7f4757400640 1 -- 172.21.15.22:0/2553331113 shutdown_connections 2024-03-28T18:21:53.477 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:21:53.476+0000 7f4757400640 1 --2- 172.21.15.22:0/2553331113 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f475010cf80 0x7f4750114b50 unknown :-1 s=CLOSED pgs=832 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:21:53.477 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:21:53.476+0000 7f4757400640 1 --2- 172.21.15.22:0/2553331113 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f4750108860 0x7f475010c8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:21:53.477 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:21:53.476+0000 7f4757400640 1 --2- 172.21.15.22:0/2553331113 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4750107eb0 0x7f4750108290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:21:53.477 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:21:53.476+0000 7f4757400640 1 -- 172.21.15.22:0/2553331113 >> 172.21.15.22:0/2553331113 conn(0x7f4750075be0 msgr2=0x7f4750078000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:21:53.477 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:21:53.476+0000 7f4757400640 1 -- 172.21.15.22:0/2553331113 shutdown_connections 2024-03-28T18:21:53.477 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:21:53.477+0000 7f4757400640 1 -- 172.21.15.22:0/2553331113 wait complete. 2024-03-28T18:21:53.480 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:21:53.477+0000 7f4757400640 1 Processor -- start 2024-03-28T18:21:53.480 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:21:53.478+0000 7f4757400640 1 -- start start 2024-03-28T18:21:53.480 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:21:53.478+0000 7f4757400640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f4750107eb0 0x7f475019e540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:21:53.480 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:21:53.478+0000 7f4757400640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f475010cf80 0x7f475019ea80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:21:53.480 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:21:53.478+0000 7f4757400640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f47501a2e00 0x7f47501a32b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:21:53.480 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:21:53.478+0000 7f4757400640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4750118020 con 0x7f475010cf80 2024-03-28T18:21:53.480 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:21:53.478+0000 7f4757400640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4750117ea0 con 0x7f4750107eb0 2024-03-28T18:21:53.480 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:21:53.478+0000 7f4757400640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47501181a0 con 0x7f47501a2e00 2024-03-28T18:21:53.480 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:21:53.478+0000 7f474fe00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f475010cf80 0x7f475019ea80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:21:53.480 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:21:53.478+0000 7f474fe00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f475010cf80 0x7f475019ea80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:40914/0 (socket says 172.21.15.22:40914) 2024-03-28T18:21:53.481 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:21:53.478+0000 7f474fe00640 1 -- 172.21.15.22:0/90520373 learned_addr learned my addr 172.21.15.22:0/90520373 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-28T18:21:53.481 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:21:53.479+0000 7f4754a00640 1 --2- 172.21.15.22:0/90520373 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f4750107eb0 0x7f475019e540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:21:53.481 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:21:53.479+0000 7f4755400640 1 --2- 172.21.15.22:0/90520373 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f47501a2e00 0x7f47501a32b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:21:53.481 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:21:53.479+0000 7f474fe00640 1 -- 172.21.15.22:0/90520373 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f47501a2e00 msgr2=0x7f47501a32b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:21:53.481 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:21:53.479+0000 7f474fe00640 1 --2- 172.21.15.22:0/90520373 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f47501a2e00 0x7f47501a32b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:21:53.481 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:21:53.479+0000 7f474fe00640 1 -- 172.21.15.22:0/90520373 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f4750107eb0 msgr2=0x7f475019e540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:21:53.481 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:21:53.479+0000 7f474fe00640 1 --2- 172.21.15.22:0/90520373 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f4750107eb0 0x7f475019e540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:21:53.481 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:21:53.479+0000 7f474fe00640 1 -- 172.21.15.22:0/90520373 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f47501a38f0 con 0x7f475010cf80 2024-03-28T18:21:53.481 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:21:53.479+0000 7f4755400640 1 --2- 172.21.15.22:0/90520373 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f47501a2e00 0x7f47501a32b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T18:21:53.481 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:21:53.479+0000 7f474fe00640 1 --2- 172.21.15.22:0/90520373 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f475010cf80 0x7f475019ea80 secure :-1 s=READY pgs=833 cs=0 l=1 rev1=1 crypto rx=0x7f473c00e9c0 tx=0x7f473c00ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:21:53.481 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:21:53.479+0000 7f474d600640 1 -- 172.21.15.22:0/90520373 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f473c00cd80 con 0x7f475010cf80 2024-03-28T18:21:53.481 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:21:53.479+0000 7f474d600640 1 -- 172.21.15.22:0/90520373 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f473c0054f0 con 0x7f475010cf80 2024-03-28T18:21:53.481 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:21:53.479+0000 7f474d600640 1 -- 172.21.15.22:0/90520373 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f473c010690 con 0x7f475010cf80 2024-03-28T18:21:53.481 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:21:53.479+0000 7f4757400640 1 -- 172.21.15.22:0/90520373 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f47501a3be0 con 0x7f475010cf80 2024-03-28T18:21:53.481 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:21:53.480+0000 7f4757400640 1 -- 172.21.15.22:0/90520373 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f47501ab380 con 0x7f475010cf80 2024-03-28T18:21:53.483 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:21:53.483+0000 7f474d600640 1 -- 172.21.15.22:0/90520373 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 26) v1 ==== 137421+0+0 (secure 0 0 0) 0x7f473c004e10 con 0x7f475010cf80 2024-03-28T18:21:53.483 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:21:53.483+0000 7f475a00b740 1 Processor -- start 2024-03-28T18:21:53.483 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:21:53.483+0000 7f475a00b740 1 -- start start 2024-03-28T18:21:53.484 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:21:53.483+0000 7f475a00b740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x564249790c00 0x56424978f950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:21:53.484 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:21:53.483+0000 7f475a00b740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56424978ff20 0x5642497903a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:21:53.485 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:21:53.484+0000 7f475a00b740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x564249799290 0x56424979b680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:21:53.485 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:21:53.484+0000 7f475a00b740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x5642496f4250 con 0x564249799290 2024-03-28T18:21:53.485 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:21:53.484+0000 7f475a00b740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x56424969ca30 con 0x564249790c00 2024-03-28T18:21:53.485 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:21:53.484+0000 7f475a00b740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x5642496f02f0 con 0x56424978ff20 2024-03-28T18:21:53.485 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:21:53.484+0000 7f474fe00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x564249799290 0x56424979b680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:21:53.485 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:21:53.484+0000 7f474fe00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x564249799290 0x56424979b680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:40918/0 (socket says 172.21.15.22:40918) 2024-03-28T18:21:53.485 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:21:53.484+0000 7f474fe00640 1 -- 172.21.15.22:0/2280408195 learned_addr learned my addr 172.21.15.22:0/2280408195 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-28T18:21:53.485 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:21:53.484+0000 7f4755400640 1 --2- 172.21.15.22:0/2280408195 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x564249790c00 0x56424978f950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:21:53.485 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:21:53.484+0000 7f4754a00640 1 --2- 172.21.15.22:0/2280408195 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56424978ff20 0x5642497903a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:21:53.485 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:21:53.484+0000 7f474fe00640 1 -- 172.21.15.22:0/2280408195 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56424978ff20 msgr2=0x5642497903a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T18:21:53.485 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:21:53.484+0000 7f474fe00640 1 --2- 172.21.15.22:0/2280408195 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56424978ff20 0x5642497903a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:21:53.485 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:21:53.484+0000 7f474fe00640 1 -- 172.21.15.22:0/2280408195 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x564249790c00 msgr2=0x56424978f950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T18:21:53.485 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:21:53.484+0000 7f474fe00640 1 --2- 172.21.15.22:0/2280408195 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x564249790c00 0x56424978f950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:21:53.486 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:21:53.484+0000 7f474fe00640 1 -- 172.21.15.22:0/2280408195 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564249687690 con 0x564249799290 2024-03-28T18:21:53.486 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:21:53.484+0000 7f474d600640 1 --2- 172.21.15.22:0/90520373 >> [v2:172.21.15.22:6800/1357365082,v1:172.21.15.22:6801/1357365082] conn(0x7f47340a2970 0x7f47340a4e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:21:53.486 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:21:53.485+0000 7f474d600640 1 -- 172.21.15.22:0/90520373 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(122..122 src has 1..122) v4 ==== 10389+0+0 (secure 0 0 0) 0x7f473c0cfa20 con 0x7f475010cf80 2024-03-28T18:21:53.486 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:21:53.485+0000 7f4754a00640 1 --2- 172.21.15.22:0/90520373 >> [v2:172.21.15.22:6800/1357365082,v1:172.21.15.22:6801/1357365082] conn(0x7f47340a2970 0x7f47340a4e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:21:53.486 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:21:53.485+0000 7f474fe00640 1 --2- 172.21.15.22:0/2280408195 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x564249799290 0x56424979b680 secure :-1 s=READY pgs=834 cs=0 l=1 rev1=1 crypto rx=0x7f473c0cc990 tx=0x7f473c0d3b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=911f622f0252cefd server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:21:53.490 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:21:53.485+0000 7f4745c00640 1 -- 172.21.15.22:0/2280408195 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f473c09b700 con 0x564249799290 2024-03-28T18:21:53.490 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:21:53.485+0000 7f4745c00640 1 -- 172.21.15.22:0/2280408195 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f473c0d6020 con 0x564249799290 2024-03-28T18:21:53.490 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:21:53.485+0000 7f4745c00640 1 -- 172.21.15.22:0/2280408195 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f473c09b290 con 0x564249799290 2024-03-28T18:21:53.491 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:21:53.486+0000 7f475a00b740 1 -- 172.21.15.22:0/2280408195 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x564249799290 msgr2=0x56424979b680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:21:53.491 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:21:53.486+0000 7f4754a00640 1 --2- 172.21.15.22:0/90520373 >> [v2:172.21.15.22:6800/1357365082,v1:172.21.15.22:6801/1357365082] conn(0x7f47340a2970 0x7f47340a4e30 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x564249791150 tx=0x7f4740005f90 comp rx=0 tx=0).ready entity=mgr.15522 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:21:53.491 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:21:53.486+0000 7f475a00b740 1 --2- 172.21.15.22:0/2280408195 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x564249799290 0x56424979b680 secure :-1 s=READY pgs=834 cs=0 l=1 rev1=1 crypto rx=0x7f473c0cc990 tx=0x7f473c0d3b50 comp rx=0 tx=0).stop 2024-03-28T18:21:53.491 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:21:53.486+0000 7f475a00b740 1 -- 172.21.15.22:0/2280408195 shutdown_connections 2024-03-28T18:21:53.491 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:21:53.486+0000 7f475a00b740 1 --2- 172.21.15.22:0/2280408195 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x564249799290 0x56424979b680 unknown :-1 s=CLOSED pgs=834 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:21:53.491 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:21:53.486+0000 7f475a00b740 1 --2- 172.21.15.22:0/2280408195 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56424978ff20 0x5642497903a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:21:53.491 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:21:53.486+0000 7f475a00b740 1 --2- 172.21.15.22:0/2280408195 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x564249790c00 0x56424978f950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:21:53.491 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:21:53.486+0000 7f475a00b740 1 -- 172.21.15.22:0/2280408195 >> 172.21.15.22:0/2280408195 conn(0x564249757200 msgr2=0x56424978a970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:21:53.491 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:21:53.486+0000 7f475a00b740 1 -- 172.21.15.22:0/2280408195 shutdown_connections 2024-03-28T18:21:53.491 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:21:53.486+0000 7f475a00b740 1 -- 172.21.15.22:0/2280408195 wait complete. 2024-03-28T18:21:53.491 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:21:53.487+0000 7f475a00b740 1 -- start start 2024-03-28T18:21:53.491 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:21:53.487+0000 7f475a00b740 1 Processor -- start 2024-03-28T18:21:53.491 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:21:53.487+0000 7f475a00b740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56424978ff20 0x564249798980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:21:53.492 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:21:53.487+0000 7f475a00b740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x564249790c00 0x564249795d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:21:53.492 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:21:53.488+0000 7f475a00b740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564249799290 0x564249796250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:21:53.492 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:21:53.488+0000 7f475a00b740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x56424979c100 con 0x56424978ff20 2024-03-28T18:21:53.492 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:21:53.488+0000 7f475a00b740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x5642496afe80 con 0x564249790c00 2024-03-28T18:21:53.492 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:21:53.488+0000 7f475a00b740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x56424979c280 con 0x564249799290 2024-03-28T18:21:53.492 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:21:53.488+0000 7f475a00b740 10 client.0 _ll_register_callbacks cb 0x5642496c2130 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-28T18:21:53.492 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:21:53.488+0000 7f4754a00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56424978ff20 0x564249798980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:21:53.492 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:21:53.488+0000 7f4754a00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56424978ff20 0x564249798980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:40930/0 (socket says 172.21.15.22:40930) 2024-03-28T18:21:53.492 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:21:53.488+0000 7f4754a00640 1 -- 172.21.15.22:0/1304619127 learned_addr learned my addr 172.21.15.22:0/1304619127 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-28T18:21:53.492 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:21:53.488+0000 7f474fe00640 1 --2- 172.21.15.22:0/1304619127 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x564249790c00 0x564249795d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:21:53.492 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:21:53.488+0000 7f4755400640 1 --2- 172.21.15.22:0/1304619127 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564249799290 0x564249796250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:21:53.492 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:21:53.488+0000 7f4754a00640 1 -- 172.21.15.22:0/1304619127 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564249799290 msgr2=0x564249796250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T18:21:53.492 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:21:53.488+0000 7f4754a00640 1 --2- 172.21.15.22:0/1304619127 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564249799290 0x564249796250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:21:53.492 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:21:53.488+0000 7f4754a00640 1 -- 172.21.15.22:0/1304619127 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x564249790c00 msgr2=0x564249795d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T18:21:53.493 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:21:53.488+0000 7f4754a00640 1 --2- 172.21.15.22:0/1304619127 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x564249790c00 0x564249795d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:21:53.493 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:21:53.488+0000 7f4754a00640 1 -- 172.21.15.22:0/1304619127 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5642494d27d0 con 0x56424978ff20 2024-03-28T18:21:53.493 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:21:53.489+0000 7f4754a00640 1 --2- 172.21.15.22:0/1304619127 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56424978ff20 0x564249798980 secure :-1 s=READY pgs=835 cs=0 l=1 rev1=1 crypto rx=0x7f4740009770 tx=0x7f4740004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c296920d4c4636d8 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:21:53.493 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:21:53.489+0000 7f473a000640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-03-28T18:21:53.493 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:21:53.489+0000 7f473a000640 1 -- 172.21.15.22:0/1304619127 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4740003d00 con 0x56424978ff20 2024-03-28T18:21:53.493 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:21:53.489+0000 7f473a000640 1 -- 172.21.15.22:0/1304619127 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(122..122 src has 1..122) v4 ==== 10389+0+0 (secure 0 0 0) 0x7f474002a420 con 0x56424978ff20 2024-03-28T18:21:53.493 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:21:53.489+0000 7f475a00b740 1 -- 172.21.15.22:0/1304619127 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f473c0d6020 con 0x56424978ff20 2024-03-28T18:21:53.493 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:21:53.489+0000 7f473a000640 1 -- 172.21.15.22:0/1304619127 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=189) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4740032050 con 0x56424978ff20 2024-03-28T18:21:53.493 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:21:53.489+0000 7f475a00b740 10 client.16980 fetch_fsmap learned FSMap version 189 2024-03-28T18:21:53.493 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:21:53.489+0000 7f475a00b740 1 -- 172.21.15.22:0/1304619127 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=189,monmap=4+}) v3 -- 0x5642494083f0 con 0x56424978ff20 2024-03-28T18:21:53.493 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:21:53.489+0000 7f473a000640 1 -- 172.21.15.22:0/1304619127 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f473c0d6020 con 0x56424978ff20 2024-03-28T18:21:53.493 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:21:53.490+0000 7f473a000640 1 -- 172.21.15.22:0/1304619127 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 189) v1 ==== 6221+0+0 (secure 0 0 0) 0x7f4740020670 con 0x56424978ff20 2024-03-28T18:21:53.493 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:21:53.490+0000 7f475a00b740 10 client.16980 fetch_fsmap finished waiting for FSMap version 189 2024-03-28T18:21:53.493 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:21:53.490+0000 7f475a00b740 10 client.16980 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-28T18:21:53.494 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:21:53.490+0000 7f475a00b740 20 client.16980 populate_metadata read hostname 'smithi022' 2024-03-28T18:21:53.494 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:21:53.490+0000 7f475a00b740 1 --2- 172.21.15.22:0/1304619127 >> [v2:172.21.15.79:6838/2441542443,v1:172.21.15.79:6839/2441542443] conn(0x5642497a9360 0x5642497ab7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:21:53.494 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:21:53.490+0000 7f475a00b740 4 client.16980 mds_command: new command op to 34553 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-28T18:21:53.494 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:21:53.490+0000 7f475a00b740 1 -- 172.21.15.22:0/1304619127 --> [v2:172.21.15.79:6838/2441542443,v1:172.21.15.79:6839/2441542443] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x564249701790 con 0x5642497a9360 2024-03-28T18:21:53.494 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:21:53.490+0000 7f474fe00640 1 --2- 172.21.15.22:0/1304619127 >> [v2:172.21.15.79:6838/2441542443,v1:172.21.15.79:6839/2441542443] conn(0x5642497a9360 0x5642497ab7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:21:53.495 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:21:53.491+0000 7f474fe00640 1 --2- 172.21.15.22:0/1304619127 >> [v2:172.21.15.79:6838/2441542443,v1:172.21.15.79:6839/2441542443] conn(0x5642497a9360 0x5642497ab7c0 crc :-1 s=READY pgs=325 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2955734aacb320bb server_cookie=47443c7c84af463 in_seq=0 out_seq=0 2024-03-28T18:21:53.495 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:21:53.491+0000 7f473a000640 10 client.16980 ms_handle_connect on v2:172.21.15.79:6838/2441542443 2024-03-28T18:21:53.495 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:21:53.493+0000 7f473a000640 1 -- 172.21.15.22:0/1304619127 <== mds.0 v2:172.21.15.79:6838/2441542443 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x564249701790 con 0x5642497a9360 2024-03-28T18:21:53.495 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:21:53.493+0000 7f473a000640 10 client.16980 handle_command_reply: tid=0 2024-03-28T18:21:53.495 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:21:53.493+0000 7f475a00b740 1 client.16980 shutdown 2024-03-28T18:21:53.495 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:21:53.493+0000 7f475a00b740 1 -- 172.21.15.22:0/1304619127 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56424978ff20 msgr2=0x564249798980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:21:53.495 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:21:53.493+0000 7f475a00b740 1 --2- 172.21.15.22:0/1304619127 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56424978ff20 0x564249798980 secure :-1 s=READY pgs=835 cs=0 l=1 rev1=1 crypto rx=0x7f4740009770 tx=0x7f4740004730 comp rx=0 tx=0).stop 2024-03-28T18:21:53.495 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:21:53.493+0000 7f475a00b740 1 -- 172.21.15.22:0/1304619127 shutdown_connections 2024-03-28T18:21:53.495 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:21:53.493+0000 7f475a00b740 1 --2- 172.21.15.22:0/1304619127 >> [v2:172.21.15.79:6838/2441542443,v1:172.21.15.79:6839/2441542443] conn(0x5642497a9360 0x5642497ab7c0 crc :-1 s=READY pgs=325 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:21:53.495 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:21:53.493+0000 7f475a00b740 1 --2- 172.21.15.22:0/1304619127 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564249799290 0x564249796250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:21:53.496 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:21:53.493+0000 7f475a00b740 1 --2- 172.21.15.22:0/1304619127 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x564249790c00 0x564249795d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:21:53.496 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:21:53.493+0000 7f475a00b740 1 --2- 172.21.15.22:0/1304619127 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56424978ff20 0x564249798980 unknown :-1 s=CLOSED pgs=835 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:21:53.496 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:21:53.493+0000 7f475a00b740 1 -- 172.21.15.22:0/1304619127 >> 172.21.15.22:0/1304619127 conn(0x564249757200 msgr2=0x56424978c6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:21:53.496 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:21:53.493+0000 7f473a000640 0 client.16980 ms_handle_reset on v2:172.21.15.79:6838/2441542443 2024-03-28T18:21:53.496 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:21:53.494+0000 7f475a00b740 1 -- 172.21.15.22:0/1304619127 shutdown_connections 2024-03-28T18:21:53.496 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:21:53.494+0000 7f475a00b740 1 -- 172.21.15.22:0/1304619127 wait complete. 2024-03-28T18:21:53.496 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:21:53.494+0000 7f475a00b740 20 client.16980 trim_cache size 0 max 16384 2024-03-28T18:21:53.511 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:21:53.511+0000 7f475a00b740 1 Processor -- start 2024-03-28T18:21:53.512 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:21:53.511+0000 7f475a00b740 1 -- start start 2024-03-28T18:21:53.512 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:21:53.512+0000 7f475a00b740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56424978ff20 0x564249794cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:21:53.512 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:21:53.512+0000 7f475a00b740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x564249790c00 0x564249798440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:21:53.512 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:21:53.512+0000 7f475a00b740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x564249799290 0x564249798980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:21:53.512 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:21:53.512+0000 7f475a00b740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47501b6370 con 0x564249799290 2024-03-28T18:21:53.512 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:21:53.512+0000 7f475a00b740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47501b36d0 con 0x564249790c00 2024-03-28T18:21:53.512 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:21:53.512+0000 7f475a00b740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47501b56b0 con 0x56424978ff20 2024-03-28T18:21:53.513 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:21:53.512+0000 7f4755400640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x564249799290 0x564249798980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:21:53.513 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:21:53.512+0000 7f4755400640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x564249799290 0x564249798980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:40944/0 (socket says 172.21.15.22:40944) 2024-03-28T18:21:53.513 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:21:53.512+0000 7f4755400640 1 -- 172.21.15.22:0/3145114670 learned_addr learned my addr 172.21.15.22:0/3145114670 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-28T18:21:53.513 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:21:53.513+0000 7f474fe00640 1 --2- 172.21.15.22:0/3145114670 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x564249790c00 0x564249798440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:21:53.513 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:21:53.513+0000 7f4755400640 1 -- 172.21.15.22:0/3145114670 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56424978ff20 msgr2=0x564249794cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T18:21:53.513 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:21:53.513+0000 7f4755400640 1 --2- 172.21.15.22:0/3145114670 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56424978ff20 0x564249794cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:21:53.514 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:21:53.513+0000 7f4755400640 1 -- 172.21.15.22:0/3145114670 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x564249790c00 msgr2=0x564249798440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T18:21:53.514 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:21:53.513+0000 7f4755400640 1 --2- 172.21.15.22:0/3145114670 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x564249790c00 0x564249798440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:21:53.514 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:21:53.513+0000 7f4755400640 1 -- 172.21.15.22:0/3145114670 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5642493c80f0 con 0x564249799290 2024-03-28T18:21:53.514 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:21:53.513+0000 7f474fe00640 1 --2- 172.21.15.22:0/3145114670 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x564249790c00 0x564249798440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T18:21:53.514 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:21:53.513+0000 7f4755400640 1 --2- 172.21.15.22:0/3145114670 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x564249799290 0x564249798980 secure :-1 s=READY pgs=836 cs=0 l=1 rev1=1 crypto rx=0x7f4748036000 tx=0x7f4748032340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3228a38acb701dee server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:21:53.514 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:21:53.513+0000 7f4745200640 1 -- 172.21.15.22:0/3145114670 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4748044070 con 0x564249799290 2024-03-28T18:21:53.514 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:21:53.513+0000 7f4745200640 1 -- 172.21.15.22:0/3145114670 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f4748032c90 con 0x564249799290 2024-03-28T18:21:53.514 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:21:53.513+0000 7f4745200640 1 -- 172.21.15.22:0/3145114670 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4748039040 con 0x564249799290 2024-03-28T18:21:53.517 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:21:53.514+0000 7f475a00b740 1 -- 172.21.15.22:0/3145114670 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x564249799290 msgr2=0x564249798980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:21:53.517 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:21:53.514+0000 7f475a00b740 1 --2- 172.21.15.22:0/3145114670 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x564249799290 0x564249798980 secure :-1 s=READY pgs=836 cs=0 l=1 rev1=1 crypto rx=0x7f4748036000 tx=0x7f4748032340 comp rx=0 tx=0).stop 2024-03-28T18:21:53.517 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:21:53.514+0000 7f475a00b740 1 -- 172.21.15.22:0/3145114670 shutdown_connections 2024-03-28T18:21:53.517 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:21:53.514+0000 7f475a00b740 1 --2- 172.21.15.22:0/3145114670 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x564249799290 0x564249798980 unknown :-1 s=CLOSED pgs=836 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:21:53.518 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:21:53.514+0000 7f475a00b740 1 --2- 172.21.15.22:0/3145114670 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x564249790c00 0x564249798440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:21:53.518 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:21:53.514+0000 7f475a00b740 1 --2- 172.21.15.22:0/3145114670 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56424978ff20 0x564249794cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:21:53.518 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:21:53.514+0000 7f475a00b740 1 -- 172.21.15.22:0/3145114670 >> 172.21.15.22:0/3145114670 conn(0x564249757200 msgr2=0x5642497945e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:21:53.518 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:21:53.514+0000 7f475a00b740 1 -- 172.21.15.22:0/3145114670 shutdown_connections 2024-03-28T18:21:53.518 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:21:53.514+0000 7f475a00b740 1 -- 172.21.15.22:0/3145114670 wait complete. 2024-03-28T18:21:53.518 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:21:53.515+0000 7f475a00b740 1 -- start start 2024-03-28T18:21:53.518 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:21:53.515+0000 7f475a00b740 1 Processor -- start 2024-03-28T18:21:53.518 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:21:53.515+0000 7f475a00b740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56424978ff20 0x56424979c8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:21:53.518 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:21:53.515+0000 7f475a00b740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x564249790c00 0x56424979ce20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:21:53.518 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:21:53.515+0000 7f475a00b740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x564249799290 0x56424979d360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:21:53.518 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:21:53.515+0000 7f475a00b740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47501b5050 con 0x564249790c00 2024-03-28T18:21:53.519 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:21:53.515+0000 7f475a00b740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47501b3d30 con 0x564249799290 2024-03-28T18:21:53.519 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:21:53.515+0000 7f475a00b740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47501b4390 con 0x56424978ff20 2024-03-28T18:21:53.519 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:21:53.515+0000 7f475a00b740 10 client.0 _ll_register_callbacks cb 0x7f47400099a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-28T18:21:53.519 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:21:53.515+0000 7f474fe00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x564249790c00 0x56424979ce20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:21:53.519 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:21:53.516+0000 7f474fe00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x564249790c00 0x56424979ce20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:40956/0 (socket says 172.21.15.22:40956) 2024-03-28T18:21:53.519 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:21:53.516+0000 7f474fe00640 1 -- 172.21.15.22:0/2644831780 learned_addr learned my addr 172.21.15.22:0/2644831780 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-28T18:21:53.519 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:21:53.516+0000 7f4755400640 1 --2- 172.21.15.22:0/2644831780 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x564249799290 0x56424979d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:21:53.519 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:21:53.516+0000 7f4754a00640 1 --2- 172.21.15.22:0/2644831780 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56424978ff20 0x56424979c8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:21:53.519 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:21:53.516+0000 7f474fe00640 1 -- 172.21.15.22:0/2644831780 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56424978ff20 msgr2=0x56424979c8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T18:21:53.519 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:21:53.516+0000 7f474fe00640 1 --2- 172.21.15.22:0/2644831780 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56424978ff20 0x56424979c8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:21:53.519 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:21:53.516+0000 7f474fe00640 1 -- 172.21.15.22:0/2644831780 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x564249799290 msgr2=0x56424979d360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T18:21:53.519 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:21:53.516+0000 7f474fe00640 1 --2- 172.21.15.22:0/2644831780 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x564249799290 0x56424979d360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:21:53.519 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:21:53.516+0000 7f474fe00640 1 -- 172.21.15.22:0/2644831780 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x564249803c50 con 0x564249790c00 2024-03-28T18:21:53.519 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:21:53.516+0000 7f474fe00640 1 --2- 172.21.15.22:0/2644831780 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x564249790c00 0x56424979ce20 secure :-1 s=READY pgs=837 cs=0 l=1 rev1=1 crypto rx=0x7f473c00c6a0 tx=0x7f473c097bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dc245db63e7d3b7 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:21:53.519 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:21:53.516+0000 7f473a000640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-03-28T18:21:53.520 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:21:53.516+0000 7f473a000640 1 -- 172.21.15.22:0/2644831780 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f473c0ce070 con 0x564249790c00 2024-03-28T18:21:53.520 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:21:53.516+0000 7f473a000640 1 -- 172.21.15.22:0/2644831780 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(122..122 src has 1..122) v4 ==== 10389+0+0 (secure 0 0 0) 0x7f473c0d3070 con 0x564249790c00 2024-03-28T18:21:53.520 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:21:53.516+0000 7f475a00b740 1 -- 172.21.15.22:0/2644831780 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f4748032c90 con 0x564249790c00 2024-03-28T18:21:53.520 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:21:53.517+0000 7f473a000640 1 -- 172.21.15.22:0/2644831780 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=189) v2 ==== 24+0+0 (secure 0 0 0) 0x7f473c09b470 con 0x564249790c00 2024-03-28T18:21:53.520 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:21:53.517+0000 7f475a00b740 10 client.16986 fetch_fsmap learned FSMap version 189 2024-03-28T18:21:53.520 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:21:53.517+0000 7f475a00b740 1 -- 172.21.15.22:0/2644831780 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=189,monmap=4+}) v3 -- 0x564249804ea0 con 0x564249790c00 2024-03-28T18:21:53.520 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:21:53.517+0000 7f473a000640 1 -- 172.21.15.22:0/2644831780 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f4748032c90 con 0x564249790c00 2024-03-28T18:21:53.520 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:21:53.517+0000 7f473a000640 1 -- 172.21.15.22:0/2644831780 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 189) v1 ==== 6221+0+0 (secure 0 0 0) 0x7f473c097fa0 con 0x564249790c00 2024-03-28T18:21:53.520 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:21:53.517+0000 7f475a00b740 10 client.16986 fetch_fsmap finished waiting for FSMap version 189 2024-03-28T18:21:53.520 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:21:53.517+0000 7f475a00b740 10 client.16986 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-28T18:21:53.520 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:21:53.517+0000 7f475a00b740 20 client.16986 populate_metadata read hostname 'smithi022' 2024-03-28T18:21:53.520 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:21:53.517+0000 7f475a00b740 1 --2- 172.21.15.22:0/2644831780 >> [v2:172.21.15.79:6838/2441542443,v1:172.21.15.79:6839/2441542443] conn(0x5642498053d0 0x5642498077f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:21:53.520 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:21:53.517+0000 7f475a00b740 4 client.16986 mds_command: new command op to 34553 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-28T18:21:53.521 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:21:53.517+0000 7f475a00b740 1 -- 172.21.15.22:0/2644831780 --> [v2:172.21.15.79:6838/2441542443,v1:172.21.15.79:6839/2441542443] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x5642496fbae0 con 0x5642498053d0 2024-03-28T18:21:53.521 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:21:53.518+0000 7f4754a00640 1 --2- 172.21.15.22:0/2644831780 >> [v2:172.21.15.79:6838/2441542443,v1:172.21.15.79:6839/2441542443] conn(0x5642498053d0 0x5642498077f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:21:53.522 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:21:53.518+0000 7f4754a00640 1 --2- 172.21.15.22:0/2644831780 >> [v2:172.21.15.79:6838/2441542443,v1:172.21.15.79:6839/2441542443] conn(0x5642498053d0 0x5642498077f0 crc :-1 s=READY pgs=326 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=99ac10d4f85c45be server_cookie=db6d6b93ebb14659 in_seq=0 out_seq=0 2024-03-28T18:21:53.522 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:21:53.518+0000 7f473a000640 10 client.16986 ms_handle_connect on v2:172.21.15.79:6838/2441542443 2024-03-28T18:21:53.522 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:21:53.519+0000 7f473a000640 1 -- 172.21.15.22:0/2644831780 <== mds.0 v2:172.21.15.79:6838/2441542443 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x5642496fbae0 con 0x5642498053d0 2024-03-28T18:21:53.522 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:21:53.519+0000 7f473a000640 10 client.16986 handle_command_reply: tid=0 2024-03-28T18:21:53.522 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:21:53.519+0000 7f475a00b740 1 client.16986 shutdown 2024-03-28T18:21:53.522 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:21:53.519+0000 7f475a00b740 1 -- 172.21.15.22:0/2644831780 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x564249790c00 msgr2=0x56424979ce20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:21:53.522 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:21:53.519+0000 7f475a00b740 1 --2- 172.21.15.22:0/2644831780 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x564249790c00 0x56424979ce20 secure :-1 s=READY pgs=837 cs=0 l=1 rev1=1 crypto rx=0x7f473c00c6a0 tx=0x7f473c097bd0 comp rx=0 tx=0).stop 2024-03-28T18:21:53.522 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:21:53.519+0000 7f475a00b740 1 -- 172.21.15.22:0/2644831780 shutdown_connections 2024-03-28T18:21:53.522 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:21:53.519+0000 7f475a00b740 1 --2- 172.21.15.22:0/2644831780 >> [v2:172.21.15.79:6838/2441542443,v1:172.21.15.79:6839/2441542443] conn(0x5642498053d0 0x5642498077f0 crc :-1 s=READY pgs=326 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:21:53.522 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:21:53.519+0000 7f475a00b740 1 --2- 172.21.15.22:0/2644831780 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x564249799290 0x56424979d360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:21:53.523 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:21:53.519+0000 7f475a00b740 1 --2- 172.21.15.22:0/2644831780 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x564249790c00 0x56424979ce20 unknown :-1 s=CLOSED pgs=837 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:21:53.523 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:21:53.519+0000 7f475a00b740 1 --2- 172.21.15.22:0/2644831780 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56424978ff20 0x56424979c8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:21:53.523 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:21:53.519+0000 7f475a00b740 1 -- 172.21.15.22:0/2644831780 >> 172.21.15.22:0/2644831780 conn(0x564249757200 msgr2=0x564249794e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:21:53.523 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:21:53.519+0000 7f473a000640 0 client.16986 ms_handle_reset on v2:172.21.15.79:6838/2441542443 2024-03-28T18:21:53.523 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:21:53.519+0000 7f475a00b740 1 -- 172.21.15.22:0/2644831780 shutdown_connections 2024-03-28T18:21:53.523 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:21:53.519+0000 7f475a00b740 1 -- 172.21.15.22:0/2644831780 wait complete. 2024-03-28T18:21:53.523 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:21:53.519+0000 7f475a00b740 20 client.16986 trim_cache size 0 max 16384 2024-03-28T18:21:53.523 INFO:teuthology.orchestra.run.smithi022.stdout:{ 2024-03-28T18:21:53.523 INFO:teuthology.orchestra.run.smithi022.stdout: "status": "IDLE (waiting for more scrubs)", 2024-03-28T18:21:53.523 INFO:teuthology.orchestra.run.smithi022.stdout: "scrubs": { 2024-03-28T18:21:53.524 INFO:teuthology.orchestra.run.smithi022.stdout: "4bbbd61d-29a3-4bf6-a4f2-d5783d585de3": { 2024-03-28T18:21:53.524 INFO:teuthology.orchestra.run.smithi022.stdout: "path": "/", 2024-03-28T18:21:53.524 INFO:teuthology.orchestra.run.smithi022.stdout: "tag": "4bbbd61d-29a3-4bf6-a4f2-d5783d585de3", 2024-03-28T18:21:53.524 INFO:teuthology.orchestra.run.smithi022.stdout: "options": "recursive,force" 2024-03-28T18:21:53.524 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-03-28T18:21:53.524 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-03-28T18:21:53.524 INFO:teuthology.orchestra.run.smithi022.stdout:} 2024-03-28T18:21:53.525 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:21:53.520+0000 7f4757400640 1 -- 172.21.15.22:0/90520373 >> [v2:172.21.15.22:6800/1357365082,v1:172.21.15.22:6801/1357365082] conn(0x7f47340a2970 msgr2=0x7f47340a4e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:21:53.525 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:21:53.520+0000 7f4757400640 1 --2- 172.21.15.22:0/90520373 >> [v2:172.21.15.22:6800/1357365082,v1:172.21.15.22:6801/1357365082] conn(0x7f47340a2970 0x7f47340a4e30 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x564249791150 tx=0x7f4740005f90 comp rx=0 tx=0).stop 2024-03-28T18:21:53.525 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:21:53.520+0000 7f4757400640 1 -- 172.21.15.22:0/90520373 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f475010cf80 msgr2=0x7f475019ea80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:21:53.525 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:21:53.520+0000 7f4757400640 1 --2- 172.21.15.22:0/90520373 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f475010cf80 0x7f475019ea80 secure :-1 s=READY pgs=833 cs=0 l=1 rev1=1 crypto rx=0x7f473c00e9c0 tx=0x7f473c00ee80 comp rx=0 tx=0).stop 2024-03-28T18:21:53.525 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:21:53.520+0000 7f4757400640 1 -- 172.21.15.22:0/90520373 shutdown_connections 2024-03-28T18:21:53.525 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:21:53.520+0000 7f4757400640 1 --2- 172.21.15.22:0/90520373 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f47501a2e00 0x7f47501a32b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:21:53.525 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:21:53.520+0000 7f4757400640 1 --2- 172.21.15.22:0/90520373 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f475010cf80 0x7f475019ea80 unknown :-1 s=CLOSED pgs=833 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:21:53.525 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:21:53.520+0000 7f4757400640 1 --2- 172.21.15.22:0/90520373 >> [v2:172.21.15.22:6800/1357365082,v1:172.21.15.22:6801/1357365082] conn(0x7f47340a2970 0x7f47340a4e30 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:21:53.525 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:21:53.520+0000 7f4757400640 1 --2- 172.21.15.22:0/90520373 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f4750107eb0 0x7f475019e540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:21:53.525 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:21:53.520+0000 7f4757400640 1 -- 172.21.15.22:0/90520373 >> 172.21.15.22:0/90520373 conn(0x7f4750075be0 msgr2=0x7f4750077ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:21:53.525 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:21:53.520+0000 7f4757400640 1 -- 172.21.15.22:0/90520373 shutdown_connections 2024-03-28T18:21:53.526 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:21:53.520+0000 7f4757400640 1 -- 172.21.15.22:0/90520373 wait complete. 2024-03-28T18:21:53.537 INFO:tasks.cephfs.filesystem:scrub status for tag:4bbbd61d-29a3-4bf6-a4f2-d5783d585de3 - {'path': '/', 'tag': '4bbbd61d-29a3-4bf6-a4f2-d5783d585de3', 'options': 'recursive,force'} 2024-03-28T18:21:55.032 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:21:54 smithi022 ceph-mon[31985]: pgmap v466: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:21:55.032 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:21:54 smithi022 ceph-mon[31985]: 2 slow requests, 0 included below; oldest blocked for > 301.843387 secs 2024-03-28T18:21:55.175 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:21:54 smithi134 ceph-mon[35037]: pgmap v466: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:21:55.175 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:21:54 smithi134 ceph-mon[35037]: 2 slow requests, 0 included below; oldest blocked for > 301.843387 secs 2024-03-28T18:21:55.182 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:21:54 smithi079 ceph-mon[35312]: pgmap v466: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:21:55.182 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:21:54 smithi079 ceph-mon[35312]: 2 slow requests, 0 included below; oldest blocked for > 301.843387 secs 2024-03-28T18:21:57.032 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:21:56 smithi022 ceph-mon[31985]: pgmap v467: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:21:57.174 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:21:56 smithi134 ceph-mon[35037]: pgmap v467: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:21:57.182 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:21:56 smithi079 ceph-mon[35312]: pgmap v467: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:21:59.032 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:21:58 smithi022 ceph-mon[31985]: pgmap v468: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:21:59.174 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:21:58 smithi134 ceph-mon[35037]: pgmap v468: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:21:59.182 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:21:58 smithi079 ceph-mon[35312]: pgmap v468: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:22:00.032 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:21:59 smithi022 ceph-mon[31985]: 2 slow requests, 0 included below; oldest blocked for > 306.843449 secs 2024-03-28T18:22:00.174 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:21:59 smithi134 ceph-mon[35037]: 2 slow requests, 0 included below; oldest blocked for > 306.843449 secs 2024-03-28T18:22:00.182 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:21:59 smithi079 ceph-mon[35312]: 2 slow requests, 0 included below; oldest blocked for > 306.843449 secs 2024-03-28T18:22:01.032 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:22:00 smithi022 ceph-mon[31985]: pgmap v469: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-28T18:22:01.175 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:22:00 smithi134 ceph-mon[35037]: pgmap v469: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-28T18:22:01.182 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:22:00 smithi079 ceph-mon[35312]: pgmap v469: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-28T18:22:03.032 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:22:02 smithi022 ceph-mon[31985]: pgmap v470: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 7 op/s 2024-03-28T18:22:03.175 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:22:02 smithi134 ceph-mon[35037]: pgmap v470: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 7 op/s 2024-03-28T18:22:03.182 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:22:02 smithi079 ceph-mon[35312]: pgmap v470: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 7 op/s 2024-03-28T18:22:05.032 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:22:04 smithi022 ceph-mon[31985]: pgmap v471: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:22:05.032 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:22:04 smithi022 ceph-mon[31985]: 2 slow requests, 0 included below; oldest blocked for > 311.843571 secs 2024-03-28T18:22:05.175 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:22:04 smithi134 ceph-mon[35037]: pgmap v471: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:22:05.175 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:22:04 smithi134 ceph-mon[35037]: 2 slow requests, 0 included below; oldest blocked for > 311.843571 secs 2024-03-28T18:22:05.182 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:22:04 smithi079 ceph-mon[35312]: pgmap v471: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:22:05.183 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:22:04 smithi079 ceph-mon[35312]: 2 slow requests, 0 included below; oldest blocked for > 311.843571 secs 2024-03-28T18:22:07.032 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:22:06 smithi022 ceph-mon[31985]: pgmap v472: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:22:07.174 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:22:06 smithi134 ceph-mon[35037]: pgmap v472: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:22:07.185 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:22:06 smithi079 ceph-mon[35312]: pgmap v472: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:22:09.032 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:22:08 smithi022 ceph-mon[31985]: pgmap v473: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:22:09.174 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:22:08 smithi134 ceph-mon[35037]: pgmap v473: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:22:09.183 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:22:08 smithi079 ceph-mon[35312]: pgmap v473: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:22:10.032 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:22:09 smithi022 ceph-mon[31985]: 2 slow requests, 0 included below; oldest blocked for > 316.843693 secs 2024-03-28T18:22:10.174 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:22:09 smithi134 ceph-mon[35037]: 2 slow requests, 0 included below; oldest blocked for > 316.843693 secs 2024-03-28T18:22:10.182 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:22:09 smithi079 ceph-mon[35312]: 2 slow requests, 0 included below; oldest blocked for > 316.843693 secs 2024-03-28T18:22:11.174 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:22:10 smithi134 ceph-mon[35037]: pgmap v474: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-28T18:22:11.182 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:22:10 smithi079 ceph-mon[35312]: pgmap v474: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-28T18:22:11.282 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:22:10 smithi022 ceph-mon[31985]: pgmap v474: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-28T18:22:12.175 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:22:11 smithi134 ceph-mon[35037]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T18:22:12.182 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:22:11 smithi079 ceph-mon[35312]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T18:22:12.281 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:22:11 smithi022 ceph-mon[31985]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T18:22:13.174 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:22:12 smithi134 ceph-mon[35037]: pgmap v475: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 7 op/s 2024-03-28T18:22:13.182 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:22:12 smithi079 ceph-mon[35312]: pgmap v475: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 7 op/s 2024-03-28T18:22:13.282 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:22:12 smithi022 ceph-mon[31985]: pgmap v475: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 7 op/s 2024-03-28T18:22:14.084 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:22:13 smithi079 ceph-mon[35312]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T18:22:14.084 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:22:13 smithi079 ceph-mon[35312]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T18:22:14.084 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:22:13 smithi079 ceph-mon[35312]: pgmap v476: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 6 op/s 2024-03-28T18:22:14.084 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:22:13 smithi079 ceph-mon[35312]: pgmap v477: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-03-28T18:22:14.084 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:22:13 smithi079 ceph-mon[35312]: from='mgr.15522 ' entity='mgr.x' 2024-03-28T18:22:14.175 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:22:13 smithi134 ceph-mon[35037]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T18:22:14.175 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:22:13 smithi134 ceph-mon[35037]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T18:22:14.175 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:22:13 smithi134 ceph-mon[35037]: pgmap v476: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 6 op/s 2024-03-28T18:22:14.175 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:22:13 smithi134 ceph-mon[35037]: pgmap v477: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-03-28T18:22:14.175 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:22:13 smithi134 ceph-mon[35037]: from='mgr.15522 ' entity='mgr.x' 2024-03-28T18:22:14.282 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:22:13 smithi022 ceph-mon[31985]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T18:22:14.282 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:22:13 smithi022 ceph-mon[31985]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T18:22:14.282 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:22:13 smithi022 ceph-mon[31985]: pgmap v476: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 6 op/s 2024-03-28T18:22:14.282 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:22:13 smithi022 ceph-mon[31985]: pgmap v477: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-03-28T18:22:14.282 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:22:13 smithi022 ceph-mon[31985]: from='mgr.15522 ' entity='mgr.x' 2024-03-28T18:22:15.175 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:22:14 smithi134 ceph-mon[35037]: 2 slow requests, 0 included below; oldest blocked for > 321.843764 secs 2024-03-28T18:22:15.182 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:22:14 smithi079 ceph-mon[35312]: 2 slow requests, 0 included below; oldest blocked for > 321.843764 secs 2024-03-28T18:22:15.282 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:22:14 smithi022 ceph-mon[31985]: 2 slow requests, 0 included below; oldest blocked for > 321.843764 secs 2024-03-28T18:22:16.174 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:22:15 smithi134 ceph-mon[35037]: pgmap v478: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-03-28T18:22:16.185 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:22:15 smithi079 ceph-mon[35312]: pgmap v478: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-03-28T18:22:16.282 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:22:15 smithi022 ceph-mon[31985]: pgmap v478: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-03-28T18:22:18.282 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:22:18 smithi022 ceph-mon[31985]: pgmap v479: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-03-28T18:22:18.424 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:22:18 smithi134 ceph-mon[35037]: pgmap v479: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-03-28T18:22:18.432 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:22:18 smithi079 ceph-mon[35312]: pgmap v479: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-03-28T18:22:20.282 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:22:20 smithi022 ceph-mon[31985]: pgmap v480: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 5 op/s 2024-03-28T18:22:20.283 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:22:20 smithi022 ceph-mon[31985]: 2 slow requests, 0 included below; oldest blocked for > 326.843884 secs 2024-03-28T18:22:20.424 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:22:20 smithi134 ceph-mon[35037]: pgmap v480: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 5 op/s 2024-03-28T18:22:20.425 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:22:20 smithi134 ceph-mon[35037]: 2 slow requests, 0 included below; oldest blocked for > 326.843884 secs 2024-03-28T18:22:20.432 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:22:20 smithi079 ceph-mon[35312]: pgmap v480: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 5 op/s 2024-03-28T18:22:20.432 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:22:20 smithi079 ceph-mon[35312]: 2 slow requests, 0 included below; oldest blocked for > 326.843884 secs 2024-03-28T18:22:22.282 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:22:22 smithi022 ceph-mon[31985]: pgmap v481: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-03-28T18:22:22.424 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:22:22 smithi134 ceph-mon[35037]: pgmap v481: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-03-28T18:22:22.432 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:22:22 smithi079 ceph-mon[35312]: pgmap v481: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-03-28T18:22:23.538 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-28T18:22:23.539 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-28T18:22:23.782 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:22:23.781+0000 7f9eb6400640 1 -- 172.21.15.22:0/3401654001 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9eb0109850 msgr2=0x7f9eb0109c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:22:23.782 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:22:23.781+0000 7f9eb6400640 1 --2- 172.21.15.22:0/3401654001 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9eb0109850 0x7f9eb0109c30 secure :-1 s=READY pgs=838 cs=0 l=1 rev1=1 crypto rx=0x7f9ea0009930 tx=0x7f9ea002bc30 comp rx=0 tx=0).stop 2024-03-28T18:22:23.782 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:22:23.781+0000 7f9eb6400640 1 -- 172.21.15.22:0/3401654001 shutdown_connections 2024-03-28T18:22:23.782 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:22:23.781+0000 7f9eb6400640 1 --2- 172.21.15.22:0/3401654001 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f9eb0104200 0x7f9eb0110710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:22:23.782 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:22:23.781+0000 7f9eb6400640 1 --2- 172.21.15.22:0/3401654001 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9eb0103840 0x7f9eb0103cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:22:23.782 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:22:23.781+0000 7f9eb6400640 1 --2- 172.21.15.22:0/3401654001 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9eb0109850 0x7f9eb0109c30 unknown :-1 s=CLOSED pgs=838 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:22:23.783 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:22:23.781+0000 7f9eb6400640 1 -- 172.21.15.22:0/3401654001 >> 172.21.15.22:0/3401654001 conn(0x7f9eb00ff2f0 msgr2=0x7f9eb0101710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:22:23.783 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:22:23.781+0000 7f9eb6400640 1 -- 172.21.15.22:0/3401654001 shutdown_connections 2024-03-28T18:22:23.783 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:22:23.781+0000 7f9eb6400640 1 -- 172.21.15.22:0/3401654001 wait complete. 2024-03-28T18:22:23.783 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:22:23.782+0000 7f9eb6400640 1 Processor -- start 2024-03-28T18:22:23.783 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:22:23.782+0000 7f9eb6400640 1 -- start start 2024-03-28T18:22:23.786 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:22:23.782+0000 7f9eb6400640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f9eb0103840 0x7f9eb019a220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:22:23.786 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:22:23.782+0000 7f9eb6400640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9eb0104200 0x7f9eb019a760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:22:23.786 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:22:23.782+0000 7f9eb6400640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9eb0109850 0x7f9eb019eae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:22:23.786 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:22:23.782+0000 7f9eb6400640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9eb0113c80 con 0x7f9eb0109850 2024-03-28T18:22:23.786 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:22:23.782+0000 7f9eb6400640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9eb0113b00 con 0x7f9eb0103840 2024-03-28T18:22:23.786 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:22:23.782+0000 7f9eb6400640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9eb0113e00 con 0x7f9eb0104200 2024-03-28T18:22:23.786 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:22:23.783+0000 7f9eafe00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9eb0109850 0x7f9eb019eae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:22:23.786 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:22:23.783+0000 7f9eafe00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9eb0109850 0x7f9eb019eae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:46142/0 (socket says 172.21.15.22:46142) 2024-03-28T18:22:23.786 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:22:23.783+0000 7f9eafe00640 1 -- 172.21.15.22:0/857291786 learned_addr learned my addr 172.21.15.22:0/857291786 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-28T18:22:23.786 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:22:23.783+0000 7f9eaf400640 1 --2- 172.21.15.22:0/857291786 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f9eb0103840 0x7f9eb019a220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:22:23.786 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:22:23.783+0000 7f9eaea00640 1 --2- 172.21.15.22:0/857291786 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9eb0104200 0x7f9eb019a760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:22:23.787 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:22:23.783+0000 7f9eafe00640 1 -- 172.21.15.22:0/857291786 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9eb0104200 msgr2=0x7f9eb019a760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:22:23.787 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:22:23.783+0000 7f9eafe00640 1 --2- 172.21.15.22:0/857291786 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9eb0104200 0x7f9eb019a760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:22:23.787 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:22:23.783+0000 7f9eafe00640 1 -- 172.21.15.22:0/857291786 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f9eb0103840 msgr2=0x7f9eb019a220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:22:23.787 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:22:23.783+0000 7f9eafe00640 1 --2- 172.21.15.22:0/857291786 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f9eb0103840 0x7f9eb019a220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:22:23.787 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:22:23.783+0000 7f9eafe00640 1 -- 172.21.15.22:0/857291786 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9eb019f210 con 0x7f9eb0109850 2024-03-28T18:22:23.787 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:22:23.783+0000 7f9eaea00640 1 --2- 172.21.15.22:0/857291786 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9eb0104200 0x7f9eb019a760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T18:22:23.787 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:22:23.784+0000 7f9eafe00640 1 --2- 172.21.15.22:0/857291786 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9eb0109850 0x7f9eb019eae0 secure :-1 s=READY pgs=839 cs=0 l=1 rev1=1 crypto rx=0x7f9ea8009fd0 tx=0x7f9ea800b260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:22:23.787 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:22:23.784+0000 7f9ea7e00640 1 -- 172.21.15.22:0/857291786 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ea8017070 con 0x7f9eb0109850 2024-03-28T18:22:23.787 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:22:23.784+0000 7f9eb6400640 1 -- 172.21.15.22:0/857291786 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9eb019f500 con 0x7f9eb0109850 2024-03-28T18:22:23.787 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:22:23.784+0000 7f9ea7e00640 1 -- 172.21.15.22:0/857291786 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f9ea8003e40 con 0x7f9eb0109850 2024-03-28T18:22:23.787 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:22:23.784+0000 7f9ea7e00640 1 -- 172.21.15.22:0/857291786 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ea80106c0 con 0x7f9eb0109850 2024-03-28T18:22:23.787 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:22:23.784+0000 7f9eb6400640 1 -- 172.21.15.22:0/857291786 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9eb01a6d90 con 0x7f9eb0109850 2024-03-28T18:22:23.790 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:22:23.787+0000 7f9eb9060740 1 Processor -- start 2024-03-28T18:22:23.790 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:22:23.787+0000 7f9ea7e00640 1 -- 172.21.15.22:0/857291786 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 26) v1 ==== 137421+0+0 (secure 0 0 0) 0x7f9ea8003970 con 0x7f9eb0109850 2024-03-28T18:22:23.790 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:22:23.788+0000 7f9eb9060740 1 -- start start 2024-03-28T18:22:23.790 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:22:23.788+0000 7f9eb9060740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e8b1004c00 0x55e8b1003950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:22:23.790 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:22:23.788+0000 7f9eb9060740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e8b1003f20 0x55e8b10043a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:22:23.790 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:22:23.788+0000 7f9eb9060740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55e8b100d290 0x55e8b100f680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:22:23.790 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:22:23.788+0000 7f9eb9060740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e8b0f68250 con 0x55e8b1003f20 2024-03-28T18:22:23.790 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:22:23.788+0000 7f9eb9060740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e8b0f10a30 con 0x55e8b100d290 2024-03-28T18:22:23.790 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:22:23.788+0000 7f9eb9060740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e8b0f642f0 con 0x55e8b1004c00 2024-03-28T18:22:23.790 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:22:23.788+0000 7f9eaf400640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e8b1003f20 0x55e8b10043a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:22:23.791 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:22:23.788+0000 7f9eaf400640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e8b1003f20 0x55e8b10043a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:46146/0 (socket says 172.21.15.22:46146) 2024-03-28T18:22:23.791 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:22:23.789+0000 7f9eaf400640 1 -- 172.21.15.22:0/2611795568 learned_addr learned my addr 172.21.15.22:0/2611795568 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-28T18:22:23.791 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:22:23.789+0000 7f9eaea00640 1 --2- 172.21.15.22:0/2611795568 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55e8b100d290 0x55e8b100f680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:22:23.791 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:22:23.789+0000 7f9eaf400640 1 -- 172.21.15.22:0/2611795568 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e8b1004c00 msgr2=0x55e8b1003950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T18:22:23.791 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:22:23.789+0000 7f9eaf400640 1 --2- 172.21.15.22:0/2611795568 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e8b1004c00 0x55e8b1003950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:22:23.791 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:22:23.789+0000 7f9eaf400640 1 -- 172.21.15.22:0/2611795568 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55e8b100d290 msgr2=0x55e8b100f680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T18:22:23.791 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:22:23.789+0000 7f9eaf400640 1 --2- 172.21.15.22:0/2611795568 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55e8b100d290 0x55e8b100f680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:22:23.791 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:22:23.789+0000 7f9eaf400640 1 -- 172.21.15.22:0/2611795568 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e8b0efb690 con 0x55e8b1003f20 2024-03-28T18:22:23.791 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:22:23.789+0000 7f9ea7e00640 1 --2- 172.21.15.22:0/857291786 >> [v2:172.21.15.22:6800/1357365082,v1:172.21.15.22:6801/1357365082] conn(0x7f9e940a2970 0x7f9e940a4e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:22:23.791 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:22:23.789+0000 7f9eaf400640 1 --2- 172.21.15.22:0/2611795568 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e8b1003f20 0x55e8b10043a0 secure :-1 s=READY pgs=840 cs=0 l=1 rev1=1 crypto rx=0x7f9ea0009900 tx=0x7f9ea00040c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e9f4dad56cc64455 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:22:23.791 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:22:23.789+0000 7f9ea7e00640 1 -- 172.21.15.22:0/857291786 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(122..122 src has 1..122) v4 ==== 10389+0+0 (secure 0 0 0) 0x7f9ea80cf320 con 0x7f9eb0109850 2024-03-28T18:22:23.792 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:22:23.789+0000 7f9eaea00640 1 --2- 172.21.15.22:0/857291786 >> [v2:172.21.15.22:6800/1357365082,v1:172.21.15.22:6801/1357365082] conn(0x7f9e940a2970 0x7f9e940a4e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:22:23.792 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:22:23.790+0000 7f9ea4c00640 1 -- 172.21.15.22:0/2611795568 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ea0045020 con 0x55e8b1003f20 2024-03-28T18:22:23.792 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:22:23.790+0000 7f9ea4c00640 1 -- 172.21.15.22:0/2611795568 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f9ea0005510 con 0x55e8b1003f20 2024-03-28T18:22:23.792 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:22:23.790+0000 7f9ea4c00640 1 -- 172.21.15.22:0/2611795568 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ea00046c0 con 0x55e8b1003f20 2024-03-28T18:22:23.795 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:22:23.790+0000 7f9eb9060740 1 -- 172.21.15.22:0/2611795568 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e8b1003f20 msgr2=0x55e8b10043a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:22:23.795 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:22:23.790+0000 7f9eaea00640 1 --2- 172.21.15.22:0/857291786 >> [v2:172.21.15.22:6800/1357365082,v1:172.21.15.22:6801/1357365082] conn(0x7f9e940a2970 0x7f9e940a4e30 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f9e9c005d00 tx=0x7f9e9c00b690 comp rx=0 tx=0).ready entity=mgr.15522 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:22:23.796 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:22:23.790+0000 7f9eb9060740 1 --2- 172.21.15.22:0/2611795568 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e8b1003f20 0x55e8b10043a0 secure :-1 s=READY pgs=840 cs=0 l=1 rev1=1 crypto rx=0x7f9ea0009900 tx=0x7f9ea00040c0 comp rx=0 tx=0).stop 2024-03-28T18:22:23.796 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:22:23.791+0000 7f9eb9060740 1 -- 172.21.15.22:0/2611795568 shutdown_connections 2024-03-28T18:22:23.796 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:22:23.791+0000 7f9eb9060740 1 --2- 172.21.15.22:0/2611795568 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55e8b100d290 0x55e8b100f680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:22:23.796 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:22:23.791+0000 7f9eb9060740 1 --2- 172.21.15.22:0/2611795568 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e8b1003f20 0x55e8b10043a0 unknown :-1 s=CLOSED pgs=840 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:22:23.796 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:22:23.791+0000 7f9eb9060740 1 --2- 172.21.15.22:0/2611795568 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e8b1004c00 0x55e8b1003950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:22:23.796 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:22:23.791+0000 7f9eb9060740 1 -- 172.21.15.22:0/2611795568 >> 172.21.15.22:0/2611795568 conn(0x55e8b0fcb200 msgr2=0x55e8b0ffe970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:22:23.796 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:22:23.791+0000 7f9eb9060740 1 -- 172.21.15.22:0/2611795568 shutdown_connections 2024-03-28T18:22:23.796 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:22:23.791+0000 7f9eb9060740 1 -- 172.21.15.22:0/2611795568 wait complete. 2024-03-28T18:22:23.796 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:22:23.791+0000 7f9eb9060740 1 -- start start 2024-03-28T18:22:23.796 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:22:23.791+0000 7f9eb9060740 1 Processor -- start 2024-03-28T18:22:23.796 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:22:23.792+0000 7f9eb9060740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e8b1003f20 0x55e8b100c900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:22:23.797 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:22:23.792+0000 7f9eb9060740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55e8b1004c00 0x55e8b1009c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:22:23.797 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:22:23.792+0000 7f9eb9060740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e8b100d290 0x55e8b100a1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:22:23.797 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:22:23.792+0000 7f9eb9060740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e8b10100a0 con 0x55e8b1003f20 2024-03-28T18:22:23.797 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:22:23.792+0000 7f9eb9060740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e8b0f23e80 con 0x55e8b1004c00 2024-03-28T18:22:23.797 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:22:23.792+0000 7f9eb9060740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e8b1010220 con 0x55e8b100d290 2024-03-28T18:22:23.797 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:22:23.792+0000 7f9eb9060740 10 client.0 _ll_register_callbacks cb 0x55e8b0f36130 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-28T18:22:23.797 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:22:23.792+0000 7f9eaf400640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e8b1003f20 0x55e8b100c900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:22:23.797 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:22:23.792+0000 7f9eaf400640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e8b1003f20 0x55e8b100c900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:46148/0 (socket says 172.21.15.22:46148) 2024-03-28T18:22:23.797 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:22:23.792+0000 7f9eaf400640 1 -- 172.21.15.22:0/3385539452 learned_addr learned my addr 172.21.15.22:0/3385539452 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-28T18:22:23.797 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:22:23.792+0000 7f9eaea00640 1 --2- 172.21.15.22:0/3385539452 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55e8b1004c00 0x55e8b1009c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:22:23.797 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:22:23.793+0000 7f9eafe00640 1 --2- 172.21.15.22:0/3385539452 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e8b100d290 0x55e8b100a1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:22:23.797 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:22:23.793+0000 7f9eaf400640 1 -- 172.21.15.22:0/3385539452 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e8b100d290 msgr2=0x55e8b100a1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T18:22:23.798 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:22:23.793+0000 7f9eaf400640 1 --2- 172.21.15.22:0/3385539452 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e8b100d290 0x55e8b100a1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:22:23.798 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:22:23.793+0000 7f9eaf400640 1 -- 172.21.15.22:0/3385539452 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55e8b1004c00 msgr2=0x55e8b1009c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T18:22:23.798 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:22:23.793+0000 7f9eaf400640 1 --2- 172.21.15.22:0/3385539452 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55e8b1004c00 0x55e8b1009c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:22:23.798 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:22:23.793+0000 7f9eaf400640 1 -- 172.21.15.22:0/3385539452 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55e8b0d467d0 con 0x55e8b1003f20 2024-03-28T18:22:23.798 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:22:23.793+0000 7f9eaf400640 1 --2- 172.21.15.22:0/3385539452 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e8b1003f20 0x55e8b100c900 secure :-1 s=READY pgs=841 cs=0 l=1 rev1=1 crypto rx=0x7f9ea0005790 tx=0x7f9ea002c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f35cfda1eeaf0298 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:22:23.798 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:22:23.793+0000 7f9e99600640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-03-28T18:22:23.798 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:22:23.793+0000 7f9e99600640 1 -- 172.21.15.22:0/3385539452 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ea004e070 con 0x55e8b1003f20 2024-03-28T18:22:23.798 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:22:23.793+0000 7f9e99600640 1 -- 172.21.15.22:0/3385539452 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(122..122 src has 1..122) v4 ==== 10389+0+0 (secure 0 0 0) 0x7f9ea0003a00 con 0x55e8b1003f20 2024-03-28T18:22:23.798 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:22:23.793+0000 7f9eb9060740 1 -- 172.21.15.22:0/3385539452 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f9ea0005510 con 0x55e8b1003f20 2024-03-28T18:22:23.798 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:22:23.794+0000 7f9e99600640 1 -- 172.21.15.22:0/3385539452 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=189) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9ea005a050 con 0x55e8b1003f20 2024-03-28T18:22:23.798 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:22:23.794+0000 7f9eb9060740 10 client.16998 fetch_fsmap learned FSMap version 189 2024-03-28T18:22:23.798 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:22:23.794+0000 7f9eb9060740 1 -- 172.21.15.22:0/3385539452 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=189,monmap=4+}) v3 -- 0x55e8b0c7c3f0 con 0x55e8b1003f20 2024-03-28T18:22:23.798 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:22:23.794+0000 7f9e99600640 1 -- 172.21.15.22:0/3385539452 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f9ea0005510 con 0x55e8b1003f20 2024-03-28T18:22:23.799 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:22:23.794+0000 7f9e99600640 1 -- 172.21.15.22:0/3385539452 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 189) v1 ==== 6221+0+0 (secure 0 0 0) 0x7f9ea0054cf0 con 0x55e8b1003f20 2024-03-28T18:22:23.799 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:22:23.794+0000 7f9eb9060740 10 client.16998 fetch_fsmap finished waiting for FSMap version 189 2024-03-28T18:22:23.799 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:22:23.794+0000 7f9eb9060740 10 client.16998 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-28T18:22:23.799 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:22:23.794+0000 7f9eb9060740 20 client.16998 populate_metadata read hostname 'smithi022' 2024-03-28T18:22:23.799 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:22:23.794+0000 7f9eb9060740 1 --2- 172.21.15.22:0/3385539452 >> [v2:172.21.15.79:6838/2441542443,v1:172.21.15.79:6839/2441542443] conn(0x55e8b101cfb0 0x55e8b101f3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:22:23.799 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:22:23.794+0000 7f9eb9060740 4 client.16998 mds_command: new command op to 34553 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-28T18:22:23.799 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:22:23.794+0000 7f9eb9060740 1 -- 172.21.15.22:0/3385539452 --> [v2:172.21.15.79:6838/2441542443,v1:172.21.15.79:6839/2441542443] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55e8b0f75790 con 0x55e8b101cfb0 2024-03-28T18:22:23.799 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:22:23.795+0000 7f9eaea00640 1 --2- 172.21.15.22:0/3385539452 >> [v2:172.21.15.79:6838/2441542443,v1:172.21.15.79:6839/2441542443] conn(0x55e8b101cfb0 0x55e8b101f3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:22:23.799 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:22:23.796+0000 7f9eaea00640 1 --2- 172.21.15.22:0/3385539452 >> [v2:172.21.15.79:6838/2441542443,v1:172.21.15.79:6839/2441542443] conn(0x55e8b101cfb0 0x55e8b101f3a0 crc :-1 s=READY pgs=329 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=142b9c57a59c4e22 server_cookie=16ab5e53bece9930 in_seq=0 out_seq=0 2024-03-28T18:22:23.799 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:22:23.796+0000 7f9e99600640 10 client.16998 ms_handle_connect on v2:172.21.15.79:6838/2441542443 2024-03-28T18:22:23.800 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:22:23.798+0000 7f9e99600640 1 -- 172.21.15.22:0/3385539452 <== mds.0 v2:172.21.15.79:6838/2441542443 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x55e8b0f75790 con 0x55e8b101cfb0 2024-03-28T18:22:23.800 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:22:23.798+0000 7f9e99600640 10 client.16998 handle_command_reply: tid=0 2024-03-28T18:22:23.801 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:22:23.798+0000 7f9eb9060740 1 client.16998 shutdown 2024-03-28T18:22:23.801 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:22:23.798+0000 7f9eb9060740 1 -- 172.21.15.22:0/3385539452 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e8b1003f20 msgr2=0x55e8b100c900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:22:23.801 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:22:23.798+0000 7f9eb9060740 1 --2- 172.21.15.22:0/3385539452 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e8b1003f20 0x55e8b100c900 secure :-1 s=READY pgs=841 cs=0 l=1 rev1=1 crypto rx=0x7f9ea0005790 tx=0x7f9ea002c880 comp rx=0 tx=0).stop 2024-03-28T18:22:23.801 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:22:23.798+0000 7f9eb9060740 1 -- 172.21.15.22:0/3385539452 shutdown_connections 2024-03-28T18:22:23.801 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:22:23.798+0000 7f9eb9060740 1 --2- 172.21.15.22:0/3385539452 >> [v2:172.21.15.79:6838/2441542443,v1:172.21.15.79:6839/2441542443] conn(0x55e8b101cfb0 0x55e8b101f3a0 crc :-1 s=READY pgs=329 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:22:23.801 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:22:23.798+0000 7f9eb9060740 1 --2- 172.21.15.22:0/3385539452 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e8b100d290 0x55e8b100a1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:22:23.801 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:22:23.798+0000 7f9eb9060740 1 --2- 172.21.15.22:0/3385539452 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55e8b1004c00 0x55e8b1009c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:22:23.801 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:22:23.798+0000 7f9eb9060740 1 --2- 172.21.15.22:0/3385539452 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e8b1003f20 0x55e8b100c900 unknown :-1 s=CLOSED pgs=841 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:22:23.801 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:22:23.798+0000 7f9eb9060740 1 -- 172.21.15.22:0/3385539452 >> 172.21.15.22:0/3385539452 conn(0x55e8b0fcb200 msgr2=0x55e8b10006c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:22:23.801 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:22:23.798+0000 7f9e99600640 0 client.16998 ms_handle_reset on v2:172.21.15.79:6838/2441542443 2024-03-28T18:22:23.801 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:22:23.798+0000 7f9eb9060740 1 -- 172.21.15.22:0/3385539452 shutdown_connections 2024-03-28T18:22:23.801 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:22:23.798+0000 7f9eb9060740 1 -- 172.21.15.22:0/3385539452 wait complete. 2024-03-28T18:22:23.801 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:22:23.799+0000 7f9eb9060740 20 client.16998 trim_cache size 0 max 16384 2024-03-28T18:22:23.817 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:22:23.816+0000 7f9eb9060740 1 Processor -- start 2024-03-28T18:22:23.817 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:22:23.816+0000 7f9eb9060740 1 -- start start 2024-03-28T18:22:23.817 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:22:23.817+0000 7f9eb9060740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55e8b1003f20 0x55e8b100c3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:22:23.817 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:22:23.817+0000 7f9eb9060740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e8b1004c00 0x55e8b100c900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:22:23.817 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:22:23.817+0000 7f9eb9060740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e8b100d290 0x55e8b100e680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:22:23.817 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:22:23.817+0000 7f9eb9060740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9eb01b1e50 con 0x55e8b1004c00 2024-03-28T18:22:23.817 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:22:23.817+0000 7f9eb9060740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9eb01af1b0 con 0x55e8b1003f20 2024-03-28T18:22:23.817 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:22:23.817+0000 7f9eb9060740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9eb01b1190 con 0x55e8b100d290 2024-03-28T18:22:23.820 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:22:23.817+0000 7f9eaea00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e8b1004c00 0x55e8b100c900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:22:23.820 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:22:23.817+0000 7f9eaf400640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55e8b1003f20 0x55e8b100c3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:22:23.820 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:22:23.817+0000 7f9eaea00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e8b1004c00 0x55e8b100c900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:46152/0 (socket says 172.21.15.22:46152) 2024-03-28T18:22:23.820 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:22:23.817+0000 7f9eaf400640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55e8b1003f20 0x55e8b100c3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.22:38146/0 (socket says 172.21.15.22:38146) 2024-03-28T18:22:23.820 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:22:23.817+0000 7f9eaea00640 1 -- 172.21.15.22:0/1978359842 learned_addr learned my addr 172.21.15.22:0/1978359842 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-28T18:22:23.820 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:22:23.818+0000 7f9eafe00640 1 --2- 172.21.15.22:0/1978359842 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e8b100d290 0x55e8b100e680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:22:23.820 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:22:23.818+0000 7f9eaf400640 1 -- 172.21.15.22:0/1978359842 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e8b100d290 msgr2=0x55e8b100e680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T18:22:23.820 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:22:23.818+0000 7f9eaf400640 1 --2- 172.21.15.22:0/1978359842 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e8b100d290 0x55e8b100e680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:22:23.820 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:22:23.818+0000 7f9eaf400640 1 -- 172.21.15.22:0/1978359842 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e8b1004c00 msgr2=0x55e8b100c900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T18:22:23.820 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:22:23.818+0000 7f9eaf400640 1 --2- 172.21.15.22:0/1978359842 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e8b1004c00 0x55e8b100c900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:22:23.820 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:22:23.818+0000 7f9eaf400640 1 -- 172.21.15.22:0/1978359842 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e8b0c3c0f0 con 0x55e8b1003f20 2024-03-28T18:22:23.821 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:22:23.818+0000 7f9eaea00640 1 --2- 172.21.15.22:0/1978359842 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e8b1004c00 0x55e8b100c900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-28T18:22:23.821 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:22:23.818+0000 7f9eaf400640 1 --2- 172.21.15.22:0/1978359842 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55e8b1003f20 0x55e8b100c3c0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f9ea0049b20 tx=0x7f9ea0000bc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=89dd2da7084e003b server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:22:23.821 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:22:23.819+0000 7f9e9be00640 1 -- 172.21.15.22:0/1978359842 <== mon.2 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ea004e040 con 0x55e8b1003f20 2024-03-28T18:22:23.821 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:22:23.819+0000 7f9e9be00640 1 -- 172.21.15.22:0/1978359842 <== mon.2 v2:172.21.15.79:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f9ea0005850 con 0x55e8b1003f20 2024-03-28T18:22:23.821 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:22:23.819+0000 7f9e9be00640 1 -- 172.21.15.22:0/1978359842 <== mon.2 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ea0045040 con 0x55e8b1003f20 2024-03-28T18:22:23.821 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:22:23.819+0000 7f9eb9060740 1 -- 172.21.15.22:0/1978359842 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55e8b1003f20 msgr2=0x55e8b100c3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:22:23.821 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:22:23.819+0000 7f9eb9060740 1 --2- 172.21.15.22:0/1978359842 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55e8b1003f20 0x55e8b100c3c0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f9ea0049b20 tx=0x7f9ea0000bc0 comp rx=0 tx=0).stop 2024-03-28T18:22:23.821 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:22:23.819+0000 7f9eb9060740 1 -- 172.21.15.22:0/1978359842 shutdown_connections 2024-03-28T18:22:23.821 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:22:23.819+0000 7f9eb9060740 1 --2- 172.21.15.22:0/1978359842 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e8b100d290 0x55e8b100e680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:22:23.821 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:22:23.819+0000 7f9eb9060740 1 --2- 172.21.15.22:0/1978359842 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e8b1004c00 0x55e8b100c900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:22:23.821 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:22:23.819+0000 7f9eb9060740 1 --2- 172.21.15.22:0/1978359842 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55e8b1003f20 0x55e8b100c3c0 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:22:23.821 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:22:23.819+0000 7f9eb9060740 1 -- 172.21.15.22:0/1978359842 >> 172.21.15.22:0/1978359842 conn(0x55e8b0fcb200 msgr2=0x55e8b0ffe4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:22:23.821 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:22:23.820+0000 7f9eb9060740 1 -- 172.21.15.22:0/1978359842 shutdown_connections 2024-03-28T18:22:23.822 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:22:23.820+0000 7f9eb9060740 1 -- 172.21.15.22:0/1978359842 wait complete. 2024-03-28T18:22:23.822 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:22:23.820+0000 7f9eb9060740 1 -- start start 2024-03-28T18:22:23.822 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:22:23.820+0000 7f9eb9060740 1 Processor -- start 2024-03-28T18:22:23.826 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:22:23.821+0000 7f9eb9060740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e8b1003f20 0x55e8b1015000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:22:23.826 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:22:23.821+0000 7f9eb9060740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e8b1004c00 0x55e8b10104d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:22:23.826 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:22:23.821+0000 7f9eb9060740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55e8b100d290 0x55e8b1010a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:22:23.826 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:22:23.821+0000 7f9eb9060740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9eb01b0b30 con 0x55e8b1004c00 2024-03-28T18:22:23.826 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:22:23.821+0000 7f9eb9060740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9eb01af810 con 0x55e8b100d290 2024-03-28T18:22:23.826 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:22:23.821+0000 7f9eb9060740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9eb01afe70 con 0x55e8b1003f20 2024-03-28T18:22:23.826 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:22:23.821+0000 7f9eb9060740 10 client.0 _ll_register_callbacks cb 0x7f9ea00095f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-28T18:22:23.826 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:22:23.821+0000 7f9eaea00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e8b1004c00 0x55e8b10104d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:22:23.826 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:22:23.821+0000 7f9eaea00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e8b1004c00 0x55e8b10104d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:46156/0 (socket says 172.21.15.22:46156) 2024-03-28T18:22:23.826 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:22:23.821+0000 7f9eaea00640 1 -- 172.21.15.22:0/544710342 learned_addr learned my addr 172.21.15.22:0/544710342 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-28T18:22:23.827 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:22:23.821+0000 7f9eafe00640 1 --2- 172.21.15.22:0/544710342 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55e8b100d290 0x55e8b1010a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:22:23.827 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:22:23.821+0000 7f9eaf400640 1 --2- 172.21.15.22:0/544710342 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e8b1003f20 0x55e8b1015000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:22:23.827 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:22:23.821+0000 7f9eaea00640 1 -- 172.21.15.22:0/544710342 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e8b1003f20 msgr2=0x55e8b1015000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T18:22:23.827 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:22:23.821+0000 7f9eaea00640 1 --2- 172.21.15.22:0/544710342 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e8b1003f20 0x55e8b1015000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:22:23.827 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:22:23.821+0000 7f9eaea00640 1 -- 172.21.15.22:0/544710342 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55e8b100d290 msgr2=0x55e8b1010a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T18:22:23.827 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:22:23.821+0000 7f9eaea00640 1 --2- 172.21.15.22:0/544710342 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55e8b100d290 0x55e8b1010a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:22:23.827 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:22:23.821+0000 7f9eaea00640 1 -- 172.21.15.22:0/544710342 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55e8b1010fa0 con 0x55e8b1004c00 2024-03-28T18:22:23.827 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:22:23.821+0000 7f9eafe00640 1 --2- 172.21.15.22:0/544710342 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55e8b100d290 0x55e8b1010a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-28T18:22:23.827 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:22:23.822+0000 7f9eaea00640 1 --2- 172.21.15.22:0/544710342 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e8b1004c00 0x55e8b10104d0 secure :-1 s=READY pgs=842 cs=0 l=1 rev1=1 crypto rx=0x7f9eb01019a0 tx=0x7f9e9c004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e59b2500c2ccb7f server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:22:23.827 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:22:23.822+0000 7f9e99600640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-03-28T18:22:23.827 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:22:23.822+0000 7f9e99600640 1 -- 172.21.15.22:0/544710342 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e9c012860 con 0x55e8b1004c00 2024-03-28T18:22:23.827 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:22:23.822+0000 7f9e99600640 1 -- 172.21.15.22:0/544710342 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(122..122 src has 1..122) v4 ==== 10389+0+0 (secure 0 0 0) 0x7f9e9c012a00 con 0x55e8b1004c00 2024-03-28T18:22:23.828 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:22:23.822+0000 7f9eb9060740 1 -- 172.21.15.22:0/544710342 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f9ea0005850 con 0x55e8b1004c00 2024-03-28T18:22:23.828 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:22:23.822+0000 7f9e99600640 1 -- 172.21.15.22:0/544710342 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=189) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9e9c03b050 con 0x55e8b1004c00 2024-03-28T18:22:23.828 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:22:23.822+0000 7f9eb9060740 10 client.17004 fetch_fsmap learned FSMap version 189 2024-03-28T18:22:23.828 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:22:23.822+0000 7f9eb9060740 1 -- 172.21.15.22:0/544710342 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=189,monmap=4+}) v3 -- 0x55e8b10785b0 con 0x55e8b1004c00 2024-03-28T18:22:23.828 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:22:23.823+0000 7f9e99600640 1 -- 172.21.15.22:0/544710342 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f9ea0005850 con 0x55e8b1004c00 2024-03-28T18:22:23.828 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:22:23.823+0000 7f9e99600640 1 -- 172.21.15.22:0/544710342 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 189) v1 ==== 6221+0+0 (secure 0 0 0) 0x7f9e9c00f040 con 0x55e8b1004c00 2024-03-28T18:22:23.828 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:22:23.823+0000 7f9eb9060740 10 client.17004 fetch_fsmap finished waiting for FSMap version 189 2024-03-28T18:22:23.828 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:22:23.823+0000 7f9eb9060740 10 client.17004 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-28T18:22:23.828 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:22:23.823+0000 7f9eb9060740 20 client.17004 populate_metadata read hostname 'smithi022' 2024-03-28T18:22:23.828 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:22:23.823+0000 7f9eb9060740 1 --2- 172.21.15.22:0/544710342 >> [v2:172.21.15.79:6838/2441542443,v1:172.21.15.79:6839/2441542443] conn(0x55e8b1078cf0 0x55e8b107b110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:22:23.828 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:22:23.823+0000 7f9eb9060740 4 client.17004 mds_command: new command op to 34553 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-28T18:22:23.828 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:22:23.823+0000 7f9eb9060740 1 -- 172.21.15.22:0/544710342 --> [v2:172.21.15.79:6838/2441542443,v1:172.21.15.79:6839/2441542443] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55e8b0f6fae0 con 0x55e8b1078cf0 2024-03-28T18:22:23.828 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:22:23.823+0000 7f9eaf400640 1 --2- 172.21.15.22:0/544710342 >> [v2:172.21.15.79:6838/2441542443,v1:172.21.15.79:6839/2441542443] conn(0x55e8b1078cf0 0x55e8b107b110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:22:23.829 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:22:23.825+0000 7f9e99600640 10 client.17004 ms_handle_connect on v2:172.21.15.79:6838/2441542443 2024-03-28T18:22:23.829 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:22:23.824+0000 7f9eaf400640 1 --2- 172.21.15.22:0/544710342 >> [v2:172.21.15.79:6838/2441542443,v1:172.21.15.79:6839/2441542443] conn(0x55e8b1078cf0 0x55e8b107b110 crc :-1 s=READY pgs=330 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=34377a27fb8c806c server_cookie=4a2c49fb1aaa263c in_seq=0 out_seq=0 2024-03-28T18:22:23.829 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:22:23.825+0000 7f9e99600640 1 -- 172.21.15.22:0/544710342 <== mds.0 v2:172.21.15.79:6838/2441542443 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x55e8b0f6fae0 con 0x55e8b1078cf0 2024-03-28T18:22:23.829 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:22:23.825+0000 7f9e99600640 10 client.17004 handle_command_reply: tid=0 2024-03-28T18:22:23.829 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:22:23.825+0000 7f9eb9060740 1 client.17004 shutdown 2024-03-28T18:22:23.829 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:22:23.825+0000 7f9eb9060740 1 -- 172.21.15.22:0/544710342 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e8b1004c00 msgr2=0x55e8b10104d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:22:23.829 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:22:23.825+0000 7f9eb9060740 1 --2- 172.21.15.22:0/544710342 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e8b1004c00 0x55e8b10104d0 secure :-1 s=READY pgs=842 cs=0 l=1 rev1=1 crypto rx=0x7f9eb01019a0 tx=0x7f9e9c004730 comp rx=0 tx=0).stop 2024-03-28T18:22:23.829 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:22:23.826+0000 7f9eb9060740 1 -- 172.21.15.22:0/544710342 shutdown_connections 2024-03-28T18:22:23.829 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:22:23.826+0000 7f9eb9060740 1 --2- 172.21.15.22:0/544710342 >> [v2:172.21.15.79:6838/2441542443,v1:172.21.15.79:6839/2441542443] conn(0x55e8b1078cf0 0x55e8b107b110 crc :-1 s=READY pgs=330 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:22:23.829 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:22:23.826+0000 7f9eb9060740 1 --2- 172.21.15.22:0/544710342 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55e8b100d290 0x55e8b1010a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:22:23.829 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:22:23.826+0000 7f9eb9060740 1 --2- 172.21.15.22:0/544710342 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e8b1004c00 0x55e8b10104d0 unknown :-1 s=CLOSED pgs=842 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:22:23.829 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:22:23.826+0000 7f9eb9060740 1 --2- 172.21.15.22:0/544710342 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e8b1003f20 0x55e8b1015000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:22:23.830 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:22:23.826+0000 7f9e99600640 0 client.17004 ms_handle_reset on v2:172.21.15.79:6838/2441542443 2024-03-28T18:22:23.830 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:22:23.826+0000 7f9eb9060740 1 -- 172.21.15.22:0/544710342 >> 172.21.15.22:0/544710342 conn(0x55e8b0fcb200 msgr2=0x55e8b1009a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:22:23.830 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:22:23.826+0000 7f9eb9060740 1 -- 172.21.15.22:0/544710342 shutdown_connections 2024-03-28T18:22:23.830 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:22:23.826+0000 7f9eb9060740 1 -- 172.21.15.22:0/544710342 wait complete. 2024-03-28T18:22:23.830 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:22:23.826+0000 7f9eb9060740 20 client.17004 trim_cache size 0 max 16384 2024-03-28T18:22:23.830 INFO:teuthology.orchestra.run.smithi022.stdout:{ 2024-03-28T18:22:23.830 INFO:teuthology.orchestra.run.smithi022.stdout: "status": "IDLE (waiting for more scrubs)", 2024-03-28T18:22:23.830 INFO:teuthology.orchestra.run.smithi022.stdout: "scrubs": { 2024-03-28T18:22:23.831 INFO:teuthology.orchestra.run.smithi022.stdout: "4bbbd61d-29a3-4bf6-a4f2-d5783d585de3": { 2024-03-28T18:22:23.831 INFO:teuthology.orchestra.run.smithi022.stdout: "path": "/", 2024-03-28T18:22:23.831 INFO:teuthology.orchestra.run.smithi022.stdout: "tag": "4bbbd61d-29a3-4bf6-a4f2-d5783d585de3", 2024-03-28T18:22:23.831 INFO:teuthology.orchestra.run.smithi022.stdout: "options": "recursive,force" 2024-03-28T18:22:23.831 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-03-28T18:22:23.831 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-03-28T18:22:23.831 INFO:teuthology.orchestra.run.smithi022.stdout:} 2024-03-28T18:22:23.832 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:22:23.826+0000 7f9eb6400640 1 -- 172.21.15.22:0/857291786 >> [v2:172.21.15.22:6800/1357365082,v1:172.21.15.22:6801/1357365082] conn(0x7f9e940a2970 msgr2=0x7f9e940a4e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:22:23.832 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:22:23.826+0000 7f9eb6400640 1 --2- 172.21.15.22:0/857291786 >> [v2:172.21.15.22:6800/1357365082,v1:172.21.15.22:6801/1357365082] conn(0x7f9e940a2970 0x7f9e940a4e30 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f9e9c005d00 tx=0x7f9e9c00b690 comp rx=0 tx=0).stop 2024-03-28T18:22:23.832 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:22:23.827+0000 7f9eb6400640 1 -- 172.21.15.22:0/857291786 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9eb0109850 msgr2=0x7f9eb019eae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:22:23.832 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:22:23.827+0000 7f9eb6400640 1 --2- 172.21.15.22:0/857291786 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9eb0109850 0x7f9eb019eae0 secure :-1 s=READY pgs=839 cs=0 l=1 rev1=1 crypto rx=0x7f9ea8009fd0 tx=0x7f9ea800b260 comp rx=0 tx=0).stop 2024-03-28T18:22:23.832 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:22:23.827+0000 7f9eb6400640 1 -- 172.21.15.22:0/857291786 shutdown_connections 2024-03-28T18:22:23.832 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:22:23.827+0000 7f9eb6400640 1 --2- 172.21.15.22:0/857291786 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9eb0109850 0x7f9eb019eae0 unknown :-1 s=CLOSED pgs=839 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:22:23.832 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:22:23.827+0000 7f9eb6400640 1 --2- 172.21.15.22:0/857291786 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9eb0104200 0x7f9eb019a760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:22:23.832 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:22:23.827+0000 7f9eb6400640 1 --2- 172.21.15.22:0/857291786 >> [v2:172.21.15.22:6800/1357365082,v1:172.21.15.22:6801/1357365082] conn(0x7f9e940a2970 0x7f9e940a4e30 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:22:23.833 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:22:23.827+0000 7f9eb6400640 1 --2- 172.21.15.22:0/857291786 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f9eb0103840 0x7f9eb019a220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:22:23.833 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:22:23.827+0000 7f9eb6400640 1 -- 172.21.15.22:0/857291786 >> 172.21.15.22:0/857291786 conn(0x7f9eb00ff2f0 msgr2=0x7f9eb00ffb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:22:23.833 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:22:23.827+0000 7f9eb6400640 1 -- 172.21.15.22:0/857291786 shutdown_connections 2024-03-28T18:22:23.833 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:22:23.827+0000 7f9eb6400640 1 -- 172.21.15.22:0/857291786 wait complete. 2024-03-28T18:22:23.843 INFO:tasks.cephfs.filesystem:scrub status for tag:4bbbd61d-29a3-4bf6-a4f2-d5783d585de3 - {'path': '/', 'tag': '4bbbd61d-29a3-4bf6-a4f2-d5783d585de3', 'options': 'recursive,force'} 2024-03-28T18:22:24.282 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:22:24 smithi022 ceph-mon[31985]: pgmap v482: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 7 op/s 2024-03-28T18:22:24.425 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:22:24 smithi134 ceph-mon[35037]: pgmap v482: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 7 op/s 2024-03-28T18:22:24.432 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:22:24 smithi079 ceph-mon[35312]: pgmap v482: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 7 op/s 2024-03-28T18:22:25.425 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:22:25 smithi134 ceph-mon[35037]: 2 slow requests, 0 included below; oldest blocked for > 331.844011 secs 2024-03-28T18:22:25.432 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:22:25 smithi079 ceph-mon[35312]: 2 slow requests, 0 included below; oldest blocked for > 331.844011 secs 2024-03-28T18:22:25.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:22:25 smithi022 ceph-mon[31985]: 2 slow requests, 0 included below; oldest blocked for > 331.844011 secs 2024-03-28T18:22:26.424 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:22:26 smithi134 ceph-mon[35037]: pgmap v483: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:22:26.432 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:22:26 smithi079 ceph-mon[35312]: pgmap v483: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:22:26.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:22:26 smithi022 ceph-mon[31985]: pgmap v483: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:22:28.335 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:22:28 smithi022 ceph-mon[31985]: pgmap v484: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:22:28.424 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:22:28 smithi134 ceph-mon[35037]: pgmap v484: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:22:28.432 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:22:28 smithi079 ceph-mon[35312]: pgmap v484: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:22:30.425 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:22:30 smithi134 ceph-mon[35037]: pgmap v485: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:22:30.425 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:22:30 smithi134 ceph-mon[35037]: 2 slow requests, 0 included below; oldest blocked for > 336.844034 secs 2024-03-28T18:22:30.432 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:22:30 smithi079 ceph-mon[35312]: pgmap v485: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:22:30.432 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:22:30 smithi079 ceph-mon[35312]: 2 slow requests, 0 included below; oldest blocked for > 336.844034 secs 2024-03-28T18:22:30.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:22:30 smithi022 ceph-mon[31985]: pgmap v485: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:22:30.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:22:30 smithi022 ceph-mon[31985]: 2 slow requests, 0 included below; oldest blocked for > 336.844034 secs 2024-03-28T18:22:32.425 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:22:32 smithi134 ceph-mon[35037]: pgmap v486: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:22:32.433 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:22:32 smithi079 ceph-mon[35312]: pgmap v486: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:22:32.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:22:32 smithi022 ceph-mon[31985]: pgmap v486: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:22:34.425 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:22:34 smithi134 ceph-mon[35037]: pgmap v487: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:22:34.432 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:22:34 smithi079 ceph-mon[35312]: pgmap v487: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:22:34.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:22:34 smithi022 ceph-mon[31985]: pgmap v487: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:22:35.424 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:22:35 smithi134 ceph-mon[35037]: 2 slow requests, 0 included below; oldest blocked for > 341.844057 secs 2024-03-28T18:22:35.432 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:22:35 smithi079 ceph-mon[35312]: 2 slow requests, 0 included below; oldest blocked for > 341.844057 secs 2024-03-28T18:22:35.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:22:35 smithi022 ceph-mon[31985]: 2 slow requests, 0 included below; oldest blocked for > 341.844057 secs 2024-03-28T18:22:36.424 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:22:36 smithi134 ceph-mon[35037]: pgmap v488: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:22:36.433 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:22:36 smithi079 ceph-mon[35312]: pgmap v488: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:22:36.531 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:22:36 smithi022 ceph-mon[31985]: pgmap v488: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:22:37.424 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:22:37 smithi134 ceph-mon[35037]: pgmap v489: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:22:37.432 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:22:37 smithi079 ceph-mon[35312]: pgmap v489: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:22:37.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:22:37 smithi022 ceph-mon[31985]: pgmap v489: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:22:40.282 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:22:40 smithi022 ceph-mon[31985]: pgmap v490: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:22:40.282 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:22:40 smithi022 ceph-mon[31985]: 2 slow requests, 0 included below; oldest blocked for > 346.844097 secs 2024-03-28T18:22:40.425 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:22:40 smithi134 ceph-mon[35037]: pgmap v490: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:22:40.425 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:22:40 smithi134 ceph-mon[35037]: 2 slow requests, 0 included below; oldest blocked for > 346.844097 secs 2024-03-28T18:22:40.432 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:22:40 smithi079 ceph-mon[35312]: pgmap v490: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:22:40.433 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:22:40 smithi079 ceph-mon[35312]: 2 slow requests, 0 included below; oldest blocked for > 346.844097 secs 2024-03-28T18:22:42.282 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:22:42 smithi022 ceph-mon[31985]: pgmap v491: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:22:42.425 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:22:42 smithi134 ceph-mon[35037]: pgmap v491: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:22:42.432 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:22:42 smithi079 ceph-mon[35312]: pgmap v491: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:22:44.282 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:22:44 smithi022 ceph-mon[31985]: pgmap v492: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:22:44.424 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:22:44 smithi134 ceph-mon[35037]: pgmap v492: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:22:44.432 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:22:44 smithi079 ceph-mon[35312]: pgmap v492: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:22:45.425 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:22:45 smithi134 ceph-mon[35037]: 2 slow requests, 0 included below; oldest blocked for > 351.844189 secs 2024-03-28T18:22:45.432 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:22:45 smithi079 ceph-mon[35312]: 2 slow requests, 0 included below; oldest blocked for > 351.844189 secs 2024-03-28T18:22:45.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:22:45 smithi022 ceph-mon[31985]: 2 slow requests, 0 included below; oldest blocked for > 351.844189 secs 2024-03-28T18:22:46.425 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:22:46 smithi134 ceph-mon[35037]: pgmap v493: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:22:46.433 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:22:46 smithi079 ceph-mon[35312]: pgmap v493: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:22:46.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:22:46 smithi022 ceph-mon[31985]: pgmap v493: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:22:48.425 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:22:48 smithi134 ceph-mon[35037]: pgmap v494: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:22:48.432 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:22:48 smithi079 ceph-mon[35312]: pgmap v494: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:22:48.531 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:22:48 smithi022 ceph-mon[31985]: pgmap v494: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:22:50.425 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:22:50 smithi134 ceph-mon[35037]: pgmap v495: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:22:50.425 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:22:50 smithi134 ceph-mon[35037]: 2 slow requests, 0 included below; oldest blocked for > 356.844269 secs 2024-03-28T18:22:50.432 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:22:50 smithi079 ceph-mon[35312]: pgmap v495: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:22:50.433 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:22:50 smithi079 ceph-mon[35312]: 2 slow requests, 0 included below; oldest blocked for > 356.844269 secs 2024-03-28T18:22:50.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:22:50 smithi022 ceph-mon[31985]: pgmap v495: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:22:50.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:22:50 smithi022 ceph-mon[31985]: 2 slow requests, 0 included below; oldest blocked for > 356.844269 secs 2024-03-28T18:22:52.425 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:22:52 smithi134 ceph-mon[35037]: pgmap v496: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:22:52.432 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:22:52 smithi079 ceph-mon[35312]: pgmap v496: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:22:52.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:22:52 smithi022 ceph-mon[31985]: pgmap v496: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:22:53.844 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-28T18:22:53.845 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-28T18:22:54.076 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:22:54.075+0000 7f6d37000640 1 -- 172.21.15.22:0/590450212 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6d3010baf0 msgr2=0x7f6d3010df90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:22:54.076 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:22:54.075+0000 7f6d37000640 1 --2- 172.21.15.22:0/590450212 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6d3010baf0 0x7f6d3010df90 secure :-1 s=READY pgs=843 cs=0 l=1 rev1=1 crypto rx=0x7f6d28009900 tx=0x7f6d2802bcd0 comp rx=0 tx=0).stop 2024-03-28T18:22:54.076 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:22:54.076+0000 7f6d37000640 1 -- 172.21.15.22:0/590450212 shutdown_connections 2024-03-28T18:22:54.076 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:22:54.076+0000 7f6d37000640 1 --2- 172.21.15.22:0/590450212 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6d3010baf0 0x7f6d3010df90 unknown :-1 s=CLOSED pgs=843 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:22:54.076 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:22:54.076+0000 7f6d37000640 1 --2- 172.21.15.22:0/590450212 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6d30102ff0 0x7f6d3010b4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:22:54.076 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:22:54.076+0000 7f6d37000640 1 --2- 172.21.15.22:0/590450212 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f6d301026d0 0x7f6d30102ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:22:54.076 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:22:54.076+0000 7f6d37000640 1 -- 172.21.15.22:0/590450212 >> 172.21.15.22:0/590450212 conn(0x7f6d300fbef0 msgr2=0x7f6d300fe310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:22:54.076 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:22:54.076+0000 7f6d37000640 1 -- 172.21.15.22:0/590450212 shutdown_connections 2024-03-28T18:22:54.076 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:22:54.076+0000 7f6d37000640 1 -- 172.21.15.22:0/590450212 wait complete. 2024-03-28T18:22:54.077 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:22:54.077+0000 7f6d37000640 1 Processor -- start 2024-03-28T18:22:54.077 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:22:54.077+0000 7f6d37000640 1 -- start start 2024-03-28T18:22:54.078 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:22:54.078+0000 7f6d37000640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6d301026d0 0x7f6d30197e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:22:54.079 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:22:54.078+0000 7f6d37000640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f6d30102ff0 0x7f6d30198350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:22:54.079 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:22:54.078+0000 7f6d37000640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6d3010baf0 0x7f6d3019c6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:22:54.079 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:22:54.078+0000 7f6d37000640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d30111910 con 0x7f6d301026d0 2024-03-28T18:22:54.079 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:22:54.078+0000 7f6d37000640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d30111790 con 0x7f6d30102ff0 2024-03-28T18:22:54.079 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:22:54.078+0000 7f6d37000640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d30111a90 con 0x7f6d3010baf0 2024-03-28T18:22:54.079 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:22:54.078+0000 7f6d35c00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6d301026d0 0x7f6d30197e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:22:54.079 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:22:54.078+0000 7f6d35c00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6d301026d0 0x7f6d30197e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:51104/0 (socket says 172.21.15.22:51104) 2024-03-28T18:22:54.079 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:22:54.078+0000 7f6d35c00640 1 -- 172.21.15.22:0/310856298 learned_addr learned my addr 172.21.15.22:0/310856298 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-28T18:22:54.079 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:22:54.078+0000 7f6d35200640 1 --2- 172.21.15.22:0/310856298 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f6d30102ff0 0x7f6d30198350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:22:54.079 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:22:54.079+0000 7f6d36600640 1 --2- 172.21.15.22:0/310856298 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6d3010baf0 0x7f6d3019c6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:22:54.079 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:22:54.079+0000 7f6d35c00640 1 -- 172.21.15.22:0/310856298 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6d3010baf0 msgr2=0x7f6d3019c6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:22:54.079 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:22:54.079+0000 7f6d35c00640 1 --2- 172.21.15.22:0/310856298 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6d3010baf0 0x7f6d3019c6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:22:54.079 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:22:54.079+0000 7f6d35c00640 1 -- 172.21.15.22:0/310856298 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f6d30102ff0 msgr2=0x7f6d30198350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:22:54.080 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:22:54.079+0000 7f6d35c00640 1 --2- 172.21.15.22:0/310856298 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f6d30102ff0 0x7f6d30198350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:22:54.080 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:22:54.079+0000 7f6d35c00640 1 -- 172.21.15.22:0/310856298 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6d3019ce00 con 0x7f6d301026d0 2024-03-28T18:22:54.080 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:22:54.079+0000 7f6d35c00640 1 --2- 172.21.15.22:0/310856298 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6d301026d0 0x7f6d30197e10 secure :-1 s=READY pgs=844 cs=0 l=1 rev1=1 crypto rx=0x7f6d24009d50 tx=0x7f6d2400c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:22:54.080 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:22:54.079+0000 7f6d2e000640 1 -- 172.21.15.22:0/310856298 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d24003b70 con 0x7f6d301026d0 2024-03-28T18:22:54.080 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:22:54.079+0000 7f6d2e000640 1 -- 172.21.15.22:0/310856298 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f6d24003d10 con 0x7f6d301026d0 2024-03-28T18:22:54.080 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:22:54.079+0000 7f6d37000640 1 -- 172.21.15.22:0/310856298 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6d3019d0f0 con 0x7f6d301026d0 2024-03-28T18:22:54.081 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:22:54.079+0000 7f6d2e000640 1 -- 172.21.15.22:0/310856298 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d24010980 con 0x7f6d301026d0 2024-03-28T18:22:54.081 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:22:54.080+0000 7f6d37000640 1 -- 172.21.15.22:0/310856298 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6d301a4980 con 0x7f6d301026d0 2024-03-28T18:22:54.083 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:22:54.083+0000 7f6d3b386740 1 Processor -- start 2024-03-28T18:22:54.083 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:22:54.083+0000 7f6d3b386740 1 -- start start 2024-03-28T18:22:54.083 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:22:54.083+0000 7f6d3b386740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a1025e3dd0 0x55a1025e2b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:22:54.083 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:22:54.083+0000 7f6d3b386740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a1025e30f0 0x55a1025e3570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:22:54.083 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:22:54.083+0000 7f6d3b386740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55a1025ec460 0x55a1025ee850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:22:54.083 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:22:54.083+0000 7f6d3b386740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a102547250 con 0x55a1025e30f0 2024-03-28T18:22:54.083 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:22:54.083+0000 7f6d3b386740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a1024efa30 con 0x55a1025ec460 2024-03-28T18:22:54.083 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:22:54.083+0000 7f6d3b386740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a1025432f0 con 0x55a1025e3dd0 2024-03-28T18:22:54.084 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:22:54.084+0000 7f6d35200640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55a1025ec460 0x55a1025ee850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:22:54.084 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:22:54.084+0000 7f6d35200640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55a1025ec460 0x55a1025ee850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.22:57222/0 (socket says 172.21.15.22:57222) 2024-03-28T18:22:54.084 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:22:54.084+0000 7f6d35200640 1 -- 172.21.15.22:0/4256217976 learned_addr learned my addr 172.21.15.22:0/4256217976 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-28T18:22:54.084 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:22:54.084+0000 7f6d36600640 1 --2- 172.21.15.22:0/4256217976 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a1025e3dd0 0x55a1025e2b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:22:54.084 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:22:54.084+0000 7f6d35200640 1 -- 172.21.15.22:0/4256217976 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a1025e3dd0 msgr2=0x55a1025e2b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T18:22:54.085 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:22:54.084+0000 7f6d35200640 1 --2- 172.21.15.22:0/4256217976 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a1025e3dd0 0x55a1025e2b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:22:54.085 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:22:54.084+0000 7f6d35200640 1 -- 172.21.15.22:0/4256217976 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a1025e30f0 msgr2=0x55a1025e3570 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-28T18:22:54.085 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:22:54.084+0000 7f6d35200640 1 --2- 172.21.15.22:0/4256217976 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a1025e30f0 0x55a1025e3570 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:22:54.085 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:22:54.084+0000 7f6d35200640 1 -- 172.21.15.22:0/4256217976 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a1024da690 con 0x55a1025ec460 2024-03-28T18:22:54.085 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:22:54.084+0000 7f6d2e000640 1 -- 172.21.15.22:0/310856298 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 26) v1 ==== 137421+0+0 (secure 0 0 0) 0x7f6d24003fc0 con 0x7f6d301026d0 2024-03-28T18:22:54.085 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:22:54.084+0000 7f6d35200640 1 --2- 172.21.15.22:0/4256217976 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55a1025ec460 0x55a1025ee850 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f6d1c00b680 tx=0x7f6d1c00bb40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a21277bbcc9e4222 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:22:54.086 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:22:54.085+0000 7f6d22a00640 1 -- 172.21.15.22:0/4256217976 <== mon.2 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d1c003bd0 con 0x55a1025ec460 2024-03-28T18:22:54.086 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:22:54.085+0000 7f6d22a00640 1 -- 172.21.15.22:0/4256217976 <== mon.2 v2:172.21.15.79:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f6d1c015070 con 0x55a1025ec460 2024-03-28T18:22:54.087 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:22:54.085+0000 7f6d22a00640 1 -- 172.21.15.22:0/4256217976 <== mon.2 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d1c004d20 con 0x55a1025ec460 2024-03-28T18:22:54.087 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:22:54.085+0000 7f6d3b386740 1 -- 172.21.15.22:0/4256217976 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55a1025ec460 msgr2=0x55a1025ee850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:22:54.087 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:22:54.085+0000 7f6d3b386740 1 --2- 172.21.15.22:0/4256217976 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55a1025ec460 0x55a1025ee850 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f6d1c00b680 tx=0x7f6d1c00bb40 comp rx=0 tx=0).stop 2024-03-28T18:22:54.087 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:22:54.086+0000 7f6d3b386740 1 -- 172.21.15.22:0/4256217976 shutdown_connections 2024-03-28T18:22:54.087 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:22:54.086+0000 7f6d3b386740 1 --2- 172.21.15.22:0/4256217976 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55a1025ec460 0x55a1025ee850 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:22:54.087 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:22:54.086+0000 7f6d3b386740 1 --2- 172.21.15.22:0/4256217976 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a1025e30f0 0x55a1025e3570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:22:54.087 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:22:54.086+0000 7f6d3b386740 1 --2- 172.21.15.22:0/4256217976 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a1025e3dd0 0x55a1025e2b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:22:54.087 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:22:54.086+0000 7f6d3b386740 1 -- 172.21.15.22:0/4256217976 >> 172.21.15.22:0/4256217976 conn(0x55a1025aa200 msgr2=0x55a1025ddb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:22:54.087 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:22:54.086+0000 7f6d2e000640 1 --2- 172.21.15.22:0/310856298 >> [v2:172.21.15.22:6800/1357365082,v1:172.21.15.22:6801/1357365082] conn(0x7f6d180a2970 0x7f6d180a4e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:22:54.087 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:22:54.086+0000 7f6d3b386740 1 -- 172.21.15.22:0/4256217976 shutdown_connections 2024-03-28T18:22:54.087 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:22:54.086+0000 7f6d3b386740 1 -- 172.21.15.22:0/4256217976 wait complete. 2024-03-28T18:22:54.087 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:22:54.086+0000 7f6d2e000640 1 -- 172.21.15.22:0/310856298 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(122..122 src has 1..122) v4 ==== 10389+0+0 (secure 0 0 0) 0x7f6d240cfd50 con 0x7f6d301026d0 2024-03-28T18:22:54.087 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:22:54.086+0000 7f6d35200640 1 --2- 172.21.15.22:0/310856298 >> [v2:172.21.15.22:6800/1357365082,v1:172.21.15.22:6801/1357365082] conn(0x7f6d180a2970 0x7f6d180a4e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:22:54.087 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:22:54.086+0000 7f6d3b386740 1 -- start start 2024-03-28T18:22:54.088 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:22:54.087+0000 7f6d3b386740 1 Processor -- start 2024-03-28T18:22:54.088 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:22:54.087+0000 7f6d35200640 1 --2- 172.21.15.22:0/310856298 >> [v2:172.21.15.22:6800/1357365082,v1:172.21.15.22:6801/1357365082] conn(0x7f6d180a2970 0x7f6d180a4e30 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f6d1c0043f0 tx=0x7f6d1c004940 comp rx=0 tx=0).ready entity=mgr.15522 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:22:54.091 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:22:54.087+0000 7f6d3b386740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55a1025e30f0 0x55a1025ebbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:22:54.091 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:22:54.087+0000 7f6d3b386740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a1025e3dd0 0x55a1025e8f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:22:54.091 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:22:54.087+0000 7f6d3b386740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a1025ec460 0x55a1025e9480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:22:54.091 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:22:54.087+0000 7f6d3b386740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a1025ef2d0 con 0x55a1025e3dd0 2024-03-28T18:22:54.091 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:22:54.087+0000 7f6d3b386740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a102502e80 con 0x55a1025e30f0 2024-03-28T18:22:54.091 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:22:54.087+0000 7f6d3b386740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a1025ef450 con 0x55a1025ec460 2024-03-28T18:22:54.091 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:22:54.087+0000 7f6d3b386740 10 client.0 _ll_register_callbacks cb 0x55a102515130 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-28T18:22:54.091 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:22:54.088+0000 7f6d35200640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a1025e3dd0 0x55a1025e8f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:22:54.092 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:22:54.088+0000 7f6d35c00640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55a1025e30f0 0x55a1025ebbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:22:54.092 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:22:54.088+0000 7f6d35c00640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55a1025e30f0 0x55a1025ebbb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.22:57230/0 (socket says 172.21.15.22:57230) 2024-03-28T18:22:54.092 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:22:54.088+0000 7f6d35200640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a1025e3dd0 0x55a1025e8f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:51114/0 (socket says 172.21.15.22:51114) 2024-03-28T18:22:54.092 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:22:54.088+0000 7f6d35c00640 1 -- 172.21.15.22:0/1618542042 learned_addr learned my addr 172.21.15.22:0/1618542042 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-28T18:22:54.092 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:22:54.088+0000 7f6d36600640 1 --2- 172.21.15.22:0/1618542042 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a1025ec460 0x55a1025e9480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:22:54.092 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:22:54.088+0000 7f6d35c00640 1 -- 172.21.15.22:0/1618542042 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a1025ec460 msgr2=0x55a1025e9480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T18:22:54.092 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:22:54.088+0000 7f6d35c00640 1 --2- 172.21.15.22:0/1618542042 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a1025ec460 0x55a1025e9480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:22:54.092 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:22:54.088+0000 7f6d35c00640 1 -- 172.21.15.22:0/1618542042 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a1025e3dd0 msgr2=0x55a1025e8f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T18:22:54.092 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:22:54.088+0000 7f6d35c00640 1 --2- 172.21.15.22:0/1618542042 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a1025e3dd0 0x55a1025e8f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:22:54.092 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:22:54.088+0000 7f6d35c00640 1 -- 172.21.15.22:0/1618542042 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55a1023257d0 con 0x55a1025e30f0 2024-03-28T18:22:54.092 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:22:54.088+0000 7f6d35c00640 1 --2- 172.21.15.22:0/1618542042 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55a1025e30f0 0x55a1025ebbb0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f6d2409b900 tx=0x7f6d2409b830 comp rx=0 tx=0).ready entity=mon.2 client_cookie=722c85f9496f23fb server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:22:54.092 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:22:54.088+0000 7f6d17400640 10 client.0 ms_handle_connect on v2:172.21.15.79:3300/0 2024-03-28T18:22:54.092 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:22:54.089+0000 7f6d17400640 1 -- 172.21.15.22:0/1618542042 <== mon.2 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d240b3060 con 0x55a1025e30f0 2024-03-28T18:22:54.092 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:22:54.089+0000 7f6d3b386740 1 -- 172.21.15.22:0/1618542042 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f6d1c015070 con 0x55a1025e30f0 2024-03-28T18:22:54.092 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:22:54.089+0000 7f6d17400640 1 -- 172.21.15.22:0/1618542042 <== mon.2 v2:172.21.15.79:3300/0 2 ==== osd_map(122..122 src has 1..122) v4 ==== 10389+0+0 (secure 0 0 0) 0x7f6d24098270 con 0x55a1025e30f0 2024-03-28T18:22:54.093 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:22:54.089+0000 7f6d17400640 1 -- 172.21.15.22:0/1618542042 <== mon.2 v2:172.21.15.79:3300/0 3 ==== mon_get_version_reply(handle=1 version=189) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6d2409bc00 con 0x55a1025e30f0 2024-03-28T18:22:54.093 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:22:54.089+0000 7f6d3b386740 10 client.36839 fetch_fsmap learned FSMap version 189 2024-03-28T18:22:54.093 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:22:54.089+0000 7f6d3b386740 1 -- 172.21.15.22:0/1618542042 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,fsmap=189,monmap=4+}) v3 -- 0x55a10225b3f0 con 0x55a1025e30f0 2024-03-28T18:22:54.093 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:22:54.089+0000 7f6d17400640 1 -- 172.21.15.22:0/1618542042 <== mon.2 v2:172.21.15.79:3300/0 4 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f6d1c015070 con 0x55a1025e30f0 2024-03-28T18:22:54.093 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:22:54.090+0000 7f6d17400640 1 -- 172.21.15.22:0/1618542042 <== mon.2 v2:172.21.15.79:3300/0 5 ==== fsmap(e 189) v1 ==== 6221+0+0 (secure 0 0 0) 0x7f6d24010b20 con 0x55a1025e30f0 2024-03-28T18:22:54.093 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:22:54.090+0000 7f6d3b386740 10 client.36839 fetch_fsmap finished waiting for FSMap version 189 2024-03-28T18:22:54.093 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:22:54.090+0000 7f6d3b386740 10 client.36839 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-28T18:22:54.093 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:22:54.090+0000 7f6d3b386740 20 client.36839 populate_metadata read hostname 'smithi022' 2024-03-28T18:22:54.093 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:22:54.090+0000 7f6d3b386740 1 --2- 172.21.15.22:0/1618542042 >> [v2:172.21.15.79:6838/2441542443,v1:172.21.15.79:6839/2441542443] conn(0x55a102600820 0x55a102602c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:22:54.093 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:22:54.090+0000 7f6d3b386740 4 client.36839 mds_command: new command op to 34553 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-28T18:22:54.093 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:22:54.090+0000 7f6d3b386740 1 -- 172.21.15.22:0/1618542042 --> [v2:172.21.15.79:6838/2441542443,v1:172.21.15.79:6839/2441542443] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55a102554790 con 0x55a102600820 2024-03-28T18:22:54.093 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:22:54.090+0000 7f6d35200640 1 --2- 172.21.15.22:0/1618542042 >> [v2:172.21.15.79:6838/2441542443,v1:172.21.15.79:6839/2441542443] conn(0x55a102600820 0x55a102602c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:22:54.093 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:22:54.091+0000 7f6d35200640 1 --2- 172.21.15.22:0/1618542042 >> [v2:172.21.15.79:6838/2441542443,v1:172.21.15.79:6839/2441542443] conn(0x55a102600820 0x55a102602c70 crc :-1 s=READY pgs=333 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bc6058aac13bf0d9 server_cookie=5d275a6b10a68bba in_seq=0 out_seq=0 2024-03-28T18:22:54.094 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:22:54.091+0000 7f6d17400640 10 client.36839 ms_handle_connect on v2:172.21.15.79:6838/2441542443 2024-03-28T18:22:54.095 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:22:54.092+0000 7f6d17400640 1 -- 172.21.15.22:0/1618542042 <== mds.0 v2:172.21.15.79:6838/2441542443 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x55a102554790 con 0x55a102600820 2024-03-28T18:22:54.095 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:22:54.092+0000 7f6d17400640 10 client.36839 handle_command_reply: tid=0 2024-03-28T18:22:54.095 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:22:54.092+0000 7f6d3b386740 1 client.36839 shutdown 2024-03-28T18:22:54.095 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:22:54.092+0000 7f6d3b386740 1 -- 172.21.15.22:0/1618542042 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55a1025e30f0 msgr2=0x55a1025ebbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:22:54.095 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:22:54.092+0000 7f6d3b386740 1 --2- 172.21.15.22:0/1618542042 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55a1025e30f0 0x55a1025ebbb0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f6d2409b900 tx=0x7f6d2409b830 comp rx=0 tx=0).stop 2024-03-28T18:22:54.095 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:22:54.092+0000 7f6d3b386740 1 -- 172.21.15.22:0/1618542042 shutdown_connections 2024-03-28T18:22:54.095 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:22:54.092+0000 7f6d3b386740 1 --2- 172.21.15.22:0/1618542042 >> [v2:172.21.15.79:6838/2441542443,v1:172.21.15.79:6839/2441542443] conn(0x55a102600820 0x55a102602c70 crc :-1 s=READY pgs=333 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:22:54.095 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:22:54.092+0000 7f6d3b386740 1 --2- 172.21.15.22:0/1618542042 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a1025ec460 0x55a1025e9480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:22:54.095 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:22:54.092+0000 7f6d3b386740 1 --2- 172.21.15.22:0/1618542042 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a1025e3dd0 0x55a1025e8f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:22:54.095 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:22:54.092+0000 7f6d17400640 0 client.36839 ms_handle_reset on v2:172.21.15.79:6838/2441542443 2024-03-28T18:22:54.095 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:22:54.092+0000 7f6d3b386740 1 --2- 172.21.15.22:0/1618542042 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55a1025e30f0 0x55a1025ebbb0 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:22:54.096 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:22:54.092+0000 7f6d3b386740 1 -- 172.21.15.22:0/1618542042 >> 172.21.15.22:0/1618542042 conn(0x55a1025aa200 msgr2=0x55a1025df890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:22:54.096 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:22:54.092+0000 7f6d3b386740 1 -- 172.21.15.22:0/1618542042 shutdown_connections 2024-03-28T18:22:54.096 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:22:54.092+0000 7f6d3b386740 1 -- 172.21.15.22:0/1618542042 wait complete. 2024-03-28T18:22:54.096 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:22:54.092+0000 7f6d3b386740 20 client.36839 trim_cache size 0 max 16384 2024-03-28T18:22:54.110 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:22:54.110+0000 7f6d3b386740 1 Processor -- start 2024-03-28T18:22:54.110 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:22:54.110+0000 7f6d3b386740 1 -- start start 2024-03-28T18:22:54.110 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:22:54.110+0000 7f6d3b386740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a1025e30f0 0x55a1025eb670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:22:54.110 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:22:54.110+0000 7f6d3b386740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55a1025e3dd0 0x55a1025ebbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:22:54.110 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:22:54.110+0000 7f6d3b386740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a1025ec460 0x55a1025fc060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:22:54.110 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:22:54.110+0000 7f6d3b386740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d301ad820 con 0x55a1025ec460 2024-03-28T18:22:54.110 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:22:54.110+0000 7f6d3b386740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d301aab80 con 0x55a1025e3dd0 2024-03-28T18:22:54.110 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:22:54.110+0000 7f6d3b386740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d301acb60 con 0x55a1025e30f0 2024-03-28T18:22:54.112 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:22:54.111+0000 7f6d36600640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a1025ec460 0x55a1025fc060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:22:54.112 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:22:54.111+0000 7f6d36600640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a1025ec460 0x55a1025fc060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:51128/0 (socket says 172.21.15.22:51128) 2024-03-28T18:22:54.112 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:22:54.111+0000 7f6d36600640 1 -- 172.21.15.22:0/3698942930 learned_addr learned my addr 172.21.15.22:0/3698942930 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-28T18:22:54.112 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:22:54.111+0000 7f6d35200640 1 --2- 172.21.15.22:0/3698942930 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55a1025e3dd0 0x55a1025ebbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:22:54.112 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:22:54.111+0000 7f6d35c00640 1 --2- 172.21.15.22:0/3698942930 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a1025e30f0 0x55a1025eb670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:22:54.112 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:22:54.111+0000 7f6d35200640 1 -- 172.21.15.22:0/3698942930 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a1025e30f0 msgr2=0x55a1025eb670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T18:22:54.112 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:22:54.111+0000 7f6d35200640 1 --2- 172.21.15.22:0/3698942930 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a1025e30f0 0x55a1025eb670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:22:54.112 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:22:54.111+0000 7f6d35200640 1 -- 172.21.15.22:0/3698942930 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a1025ec460 msgr2=0x55a1025fc060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T18:22:54.112 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:22:54.111+0000 7f6d35200640 1 --2- 172.21.15.22:0/3698942930 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a1025ec460 0x55a1025fc060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:22:54.112 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:22:54.111+0000 7f6d35200640 1 -- 172.21.15.22:0/3698942930 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a10221b0f0 con 0x55a1025e3dd0 2024-03-28T18:22:54.112 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:22:54.112+0000 7f6d35200640 1 --2- 172.21.15.22:0/3698942930 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55a1025e3dd0 0x55a1025ebbb0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f6d1c005fd0 tx=0x7f6d1c023000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9823789820f94d2f server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:22:54.113 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:22:54.112+0000 7f6d22000640 1 -- 172.21.15.22:0/3698942930 <== mon.2 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d1c035120 con 0x55a1025e3dd0 2024-03-28T18:22:54.113 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:22:54.112+0000 7f6d22000640 1 -- 172.21.15.22:0/3698942930 <== mon.2 v2:172.21.15.79:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f6d1c015040 con 0x55a1025e3dd0 2024-03-28T18:22:54.113 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:22:54.112+0000 7f6d22000640 1 -- 172.21.15.22:0/3698942930 <== mon.2 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d1c032b70 con 0x55a1025e3dd0 2024-03-28T18:22:54.115 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:22:54.113+0000 7f6d3b386740 1 -- 172.21.15.22:0/3698942930 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55a1025e3dd0 msgr2=0x55a1025ebbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:22:54.115 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:22:54.113+0000 7f6d3b386740 1 --2- 172.21.15.22:0/3698942930 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55a1025e3dd0 0x55a1025ebbb0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f6d1c005fd0 tx=0x7f6d1c023000 comp rx=0 tx=0).stop 2024-03-28T18:22:54.115 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:22:54.113+0000 7f6d3b386740 1 -- 172.21.15.22:0/3698942930 shutdown_connections 2024-03-28T18:22:54.115 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:22:54.113+0000 7f6d3b386740 1 --2- 172.21.15.22:0/3698942930 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a1025ec460 0x55a1025fc060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:22:54.115 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:22:54.113+0000 7f6d3b386740 1 --2- 172.21.15.22:0/3698942930 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55a1025e3dd0 0x55a1025ebbb0 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:22:54.115 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:22:54.113+0000 7f6d3b386740 1 --2- 172.21.15.22:0/3698942930 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a1025e30f0 0x55a1025eb670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:22:54.115 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:22:54.113+0000 7f6d3b386740 1 -- 172.21.15.22:0/3698942930 >> 172.21.15.22:0/3698942930 conn(0x55a1025aa200 msgr2=0x55a1025dd670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:22:54.115 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:22:54.113+0000 7f6d3b386740 1 -- 172.21.15.22:0/3698942930 shutdown_connections 2024-03-28T18:22:54.115 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:22:54.113+0000 7f6d3b386740 1 -- 172.21.15.22:0/3698942930 wait complete. 2024-03-28T18:22:54.115 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:22:54.114+0000 7f6d3b386740 1 -- start start 2024-03-28T18:22:54.115 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:22:54.114+0000 7f6d3b386740 1 Processor -- start 2024-03-28T18:22:54.115 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:22:54.114+0000 7f6d3b386740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55a1025e30f0 0x55a1025ed110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:22:54.116 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:22:54.114+0000 7f6d3b386740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a1025e3dd0 0x55a1025ed650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:22:54.116 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:22:54.114+0000 7f6d3b386740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a1025ec460 0x55a1025f6dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:22:54.116 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:22:54.114+0000 7f6d3b386740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d301ac500 con 0x55a1025ec460 2024-03-28T18:22:54.116 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:22:54.114+0000 7f6d3b386740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d301ab1e0 con 0x55a1025e30f0 2024-03-28T18:22:54.116 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:22:54.114+0000 7f6d3b386740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d301ab840 con 0x55a1025e3dd0 2024-03-28T18:22:54.116 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:22:54.114+0000 7f6d3b386740 10 client.0 _ll_register_callbacks cb 0x7f6d24009ca0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-28T18:22:54.116 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:22:54.114+0000 7f6d36600640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a1025ec460 0x55a1025f6dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:22:54.116 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:22:54.114+0000 7f6d36600640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a1025ec460 0x55a1025f6dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:51138/0 (socket says 172.21.15.22:51138) 2024-03-28T18:22:54.116 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:22:54.114+0000 7f6d36600640 1 -- 172.21.15.22:0/2345120212 learned_addr learned my addr 172.21.15.22:0/2345120212 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-28T18:22:54.116 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:22:54.115+0000 7f6d35c00640 1 --2- 172.21.15.22:0/2345120212 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55a1025e30f0 0x55a1025ed110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:22:54.116 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:22:54.115+0000 7f6d35200640 1 --2- 172.21.15.22:0/2345120212 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a1025e3dd0 0x55a1025ed650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:22:54.116 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:22:54.115+0000 7f6d36600640 1 -- 172.21.15.22:0/2345120212 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a1025e3dd0 msgr2=0x55a1025ed650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T18:22:54.116 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:22:54.115+0000 7f6d36600640 1 --2- 172.21.15.22:0/2345120212 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a1025e3dd0 0x55a1025ed650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:22:54.116 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:22:54.115+0000 7f6d36600640 1 -- 172.21.15.22:0/2345120212 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55a1025e30f0 msgr2=0x55a1025ed110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T18:22:54.117 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:22:54.115+0000 7f6d36600640 1 --2- 172.21.15.22:0/2345120212 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55a1025e30f0 0x55a1025ed110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:22:54.117 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:22:54.115+0000 7f6d36600640 1 -- 172.21.15.22:0/2345120212 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55a1025f7400 con 0x55a1025ec460 2024-03-28T18:22:54.119 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:22:54.115+0000 7f6d36600640 1 --2- 172.21.15.22:0/2345120212 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a1025ec460 0x55a1025f6dc0 secure :-1 s=READY pgs=845 cs=0 l=1 rev1=1 crypto rx=0x7f6d28036040 tx=0x7f6d28005270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=78b35572a892504c server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:22:54.119 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:22:54.115+0000 7f6d17400640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-03-28T18:22:54.119 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:22:54.116+0000 7f6d17400640 1 -- 172.21.15.22:0/2345120212 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d28044070 con 0x55a1025ec460 2024-03-28T18:22:54.119 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:22:54.116+0000 7f6d3b386740 1 -- 172.21.15.22:0/2345120212 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f6d1c015040 con 0x55a1025ec460 2024-03-28T18:22:54.119 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:22:54.116+0000 7f6d17400640 1 -- 172.21.15.22:0/2345120212 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(122..122 src has 1..122) v4 ==== 10389+0+0 (secure 0 0 0) 0x7f6d280046c0 con 0x55a1025ec460 2024-03-28T18:22:54.119 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:22:54.116+0000 7f6d17400640 1 -- 172.21.15.22:0/2345120212 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=189) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6d28005620 con 0x55a1025ec460 2024-03-28T18:22:54.119 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:22:54.116+0000 7f6d3b386740 10 client.17019 fetch_fsmap learned FSMap version 189 2024-03-28T18:22:54.119 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:22:54.116+0000 7f6d3b386740 1 -- 172.21.15.22:0/2345120212 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=189,monmap=4+}) v3 -- 0x55a10265cb10 con 0x55a1025ec460 2024-03-28T18:22:54.119 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:22:54.116+0000 7f6d17400640 1 -- 172.21.15.22:0/2345120212 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f6d1c015040 con 0x55a1025ec460 2024-03-28T18:22:54.120 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:22:54.116+0000 7f6d17400640 1 -- 172.21.15.22:0/2345120212 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 189) v1 ==== 6221+0+0 (secure 0 0 0) 0x7f6d2802cce0 con 0x55a1025ec460 2024-03-28T18:22:54.120 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:22:54.116+0000 7f6d3b386740 10 client.17019 fetch_fsmap finished waiting for FSMap version 189 2024-03-28T18:22:54.120 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:22:54.116+0000 7f6d3b386740 10 client.17019 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-28T18:22:54.120 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:22:54.116+0000 7f6d3b386740 20 client.17019 populate_metadata read hostname 'smithi022' 2024-03-28T18:22:54.120 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:22:54.117+0000 7f6d3b386740 1 --2- 172.21.15.22:0/2345120212 >> [v2:172.21.15.79:6838/2441542443,v1:172.21.15.79:6839/2441542443] conn(0x55a10265d1f0 0x55a10265f610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:22:54.120 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:22:54.117+0000 7f6d3b386740 4 client.17019 mds_command: new command op to 34553 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-28T18:22:54.120 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:22:54.117+0000 7f6d3b386740 1 -- 172.21.15.22:0/2345120212 --> [v2:172.21.15.79:6838/2441542443,v1:172.21.15.79:6839/2441542443] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55a10254eae0 con 0x55a10265d1f0 2024-03-28T18:22:54.120 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:22:54.117+0000 7f6d35c00640 1 --2- 172.21.15.22:0/2345120212 >> [v2:172.21.15.79:6838/2441542443,v1:172.21.15.79:6839/2441542443] conn(0x55a10265d1f0 0x55a10265f610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:22:54.120 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:22:54.118+0000 7f6d35c00640 1 --2- 172.21.15.22:0/2345120212 >> [v2:172.21.15.79:6838/2441542443,v1:172.21.15.79:6839/2441542443] conn(0x55a10265d1f0 0x55a10265f610 crc :-1 s=READY pgs=334 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a9c91f667f37427a server_cookie=61c6e8af0c927c20 in_seq=0 out_seq=0 2024-03-28T18:22:54.120 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:22:54.118+0000 7f6d17400640 10 client.17019 ms_handle_connect on v2:172.21.15.79:6838/2441542443 2024-03-28T18:22:54.120 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:22:54.118+0000 7f6d17400640 1 -- 172.21.15.22:0/2345120212 <== mds.0 v2:172.21.15.79:6838/2441542443 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x55a10254eae0 con 0x55a10265d1f0 2024-03-28T18:22:54.120 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:22:54.118+0000 7f6d17400640 10 client.17019 handle_command_reply: tid=0 2024-03-28T18:22:54.120 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:22:54.118+0000 7f6d3b386740 1 client.17019 shutdown 2024-03-28T18:22:54.120 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:22:54.118+0000 7f6d3b386740 1 -- 172.21.15.22:0/2345120212 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a1025ec460 msgr2=0x55a1025f6dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:22:54.120 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:22:54.118+0000 7f6d3b386740 1 --2- 172.21.15.22:0/2345120212 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a1025ec460 0x55a1025f6dc0 secure :-1 s=READY pgs=845 cs=0 l=1 rev1=1 crypto rx=0x7f6d28036040 tx=0x7f6d28005270 comp rx=0 tx=0).stop 2024-03-28T18:22:54.120 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:22:54.118+0000 7f6d3b386740 1 -- 172.21.15.22:0/2345120212 shutdown_connections 2024-03-28T18:22:54.121 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:22:54.118+0000 7f6d3b386740 1 --2- 172.21.15.22:0/2345120212 >> [v2:172.21.15.79:6838/2441542443,v1:172.21.15.79:6839/2441542443] conn(0x55a10265d1f0 0x55a10265f610 crc :-1 s=READY pgs=334 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:22:54.121 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:22:54.118+0000 7f6d3b386740 1 --2- 172.21.15.22:0/2345120212 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a1025ec460 0x55a1025f6dc0 unknown :-1 s=CLOSED pgs=845 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:22:54.121 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:22:54.118+0000 7f6d3b386740 1 --2- 172.21.15.22:0/2345120212 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a1025e3dd0 0x55a1025ed650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:22:54.121 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:22:54.118+0000 7f6d3b386740 1 --2- 172.21.15.22:0/2345120212 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55a1025e30f0 0x55a1025ed110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:22:54.121 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:22:54.118+0000 7f6d17400640 0 client.17019 ms_handle_reset on v2:172.21.15.79:6838/2441542443 2024-03-28T18:22:54.121 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:22:54.118+0000 7f6d3b386740 1 -- 172.21.15.22:0/2345120212 >> 172.21.15.22:0/2345120212 conn(0x55a1025aa200 msgr2=0x55a1025e77d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:22:54.121 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:22:54.119+0000 7f6d3b386740 1 -- 172.21.15.22:0/2345120212 shutdown_connections 2024-03-28T18:22:54.121 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:22:54.119+0000 7f6d3b386740 1 -- 172.21.15.22:0/2345120212 wait complete. 2024-03-28T18:22:54.121 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:22:54.119+0000 7f6d3b386740 20 client.17019 trim_cache size 0 max 16384 2024-03-28T18:22:54.121 INFO:teuthology.orchestra.run.smithi022.stdout:{ 2024-03-28T18:22:54.121 INFO:teuthology.orchestra.run.smithi022.stdout: "status": "IDLE (waiting for more scrubs)", 2024-03-28T18:22:54.122 INFO:teuthology.orchestra.run.smithi022.stdout: "scrubs": { 2024-03-28T18:22:54.122 INFO:teuthology.orchestra.run.smithi022.stdout: "4bbbd61d-29a3-4bf6-a4f2-d5783d585de3": { 2024-03-28T18:22:54.122 INFO:teuthology.orchestra.run.smithi022.stdout: "path": "/", 2024-03-28T18:22:54.122 INFO:teuthology.orchestra.run.smithi022.stdout: "tag": "4bbbd61d-29a3-4bf6-a4f2-d5783d585de3", 2024-03-28T18:22:54.122 INFO:teuthology.orchestra.run.smithi022.stdout: "options": "recursive,force" 2024-03-28T18:22:54.122 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-03-28T18:22:54.122 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-03-28T18:22:54.122 INFO:teuthology.orchestra.run.smithi022.stdout:} 2024-03-28T18:22:54.123 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:22:54.119+0000 7f6d37000640 1 -- 172.21.15.22:0/310856298 >> [v2:172.21.15.22:6800/1357365082,v1:172.21.15.22:6801/1357365082] conn(0x7f6d180a2970 msgr2=0x7f6d180a4e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:22:54.123 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:22:54.119+0000 7f6d37000640 1 --2- 172.21.15.22:0/310856298 >> [v2:172.21.15.22:6800/1357365082,v1:172.21.15.22:6801/1357365082] conn(0x7f6d180a2970 0x7f6d180a4e30 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f6d1c0043f0 tx=0x7f6d1c004940 comp rx=0 tx=0).stop 2024-03-28T18:22:54.123 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:22:54.119+0000 7f6d37000640 1 -- 172.21.15.22:0/310856298 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6d301026d0 msgr2=0x7f6d30197e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:22:54.123 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:22:54.119+0000 7f6d37000640 1 --2- 172.21.15.22:0/310856298 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6d301026d0 0x7f6d30197e10 secure :-1 s=READY pgs=844 cs=0 l=1 rev1=1 crypto rx=0x7f6d24009d50 tx=0x7f6d2400c6a0 comp rx=0 tx=0).stop 2024-03-28T18:22:54.123 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:22:54.119+0000 7f6d37000640 1 -- 172.21.15.22:0/310856298 shutdown_connections 2024-03-28T18:22:54.123 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:22:54.119+0000 7f6d37000640 1 --2- 172.21.15.22:0/310856298 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6d3010baf0 0x7f6d3019c6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:22:54.123 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:22:54.119+0000 7f6d37000640 1 --2- 172.21.15.22:0/310856298 >> [v2:172.21.15.22:6800/1357365082,v1:172.21.15.22:6801/1357365082] conn(0x7f6d180a2970 0x7f6d180a4e30 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:22:54.123 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:22:54.119+0000 7f6d37000640 1 --2- 172.21.15.22:0/310856298 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f6d30102ff0 0x7f6d30198350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:22:54.123 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:22:54.119+0000 7f6d37000640 1 --2- 172.21.15.22:0/310856298 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6d301026d0 0x7f6d30197e10 unknown :-1 s=CLOSED pgs=844 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:22:54.123 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:22:54.119+0000 7f6d37000640 1 -- 172.21.15.22:0/310856298 >> 172.21.15.22:0/310856298 conn(0x7f6d300fbef0 msgr2=0x7f6d300fcbc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:22:54.123 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:22:54.120+0000 7f6d37000640 1 -- 172.21.15.22:0/310856298 shutdown_connections 2024-03-28T18:22:54.123 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:22:54.120+0000 7f6d37000640 1 -- 172.21.15.22:0/310856298 wait complete. 2024-03-28T18:22:54.136 INFO:tasks.cephfs.filesystem:scrub status for tag:4bbbd61d-29a3-4bf6-a4f2-d5783d585de3 - {'path': '/', 'tag': '4bbbd61d-29a3-4bf6-a4f2-d5783d585de3', 'options': 'recursive,force'} 2024-03-28T18:22:54.282 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:22:54 smithi022 ceph-mon[31985]: pgmap v497: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:22:54.424 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:22:54 smithi134 ceph-mon[35037]: pgmap v497: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:22:54.432 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:22:54 smithi079 ceph-mon[35312]: pgmap v497: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:22:55.425 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:22:55 smithi134 ceph-mon[35037]: 2 slow requests, 1 included below; oldest blocked for > 361.844354 secs 2024-03-28T18:22:55.425 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:22:55 smithi134 ceph-mon[35037]: slow request 361.844354 seconds old, received at 2024-03-28T18:16:52.457506+0000: peer_request:client.35222:10732 currently acquired locks 2024-03-28T18:22:55.433 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:22:55 smithi079 ceph-mon[35312]: 2 slow requests, 1 included below; oldest blocked for > 361.844354 secs 2024-03-28T18:22:55.433 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:22:55 smithi079 ceph-mon[35312]: slow request 361.844354 seconds old, received at 2024-03-28T18:16:52.457506+0000: peer_request:client.35222:10732 currently acquired locks 2024-03-28T18:22:55.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:22:55 smithi022 ceph-mon[31985]: 2 slow requests, 1 included below; oldest blocked for > 361.844354 secs 2024-03-28T18:22:55.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:22:55 smithi022 ceph-mon[31985]: slow request 361.844354 seconds old, received at 2024-03-28T18:16:52.457506+0000: peer_request:client.35222:10732 currently acquired locks 2024-03-28T18:22:56.425 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:22:56 smithi134 ceph-mon[35037]: pgmap v498: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:22:56.432 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:22:56 smithi079 ceph-mon[35312]: pgmap v498: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:22:56.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:22:56 smithi022 ceph-mon[31985]: pgmap v498: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:22:57.424 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:22:57 smithi134 ceph-mon[35037]: pgmap v499: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:22:57.432 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:22:57 smithi079 ceph-mon[35312]: pgmap v499: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:22:57.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:22:57 smithi022 ceph-mon[31985]: pgmap v499: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:23:00.282 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:23:00 smithi022 ceph-mon[31985]: pgmap v500: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:23:00.282 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:23:00 smithi022 ceph-mon[31985]: 2 slow requests, 0 included below; oldest blocked for > 366.844477 secs 2024-03-28T18:23:00.425 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:23:00 smithi134 ceph-mon[35037]: pgmap v500: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:23:00.425 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:23:00 smithi134 ceph-mon[35037]: 2 slow requests, 0 included below; oldest blocked for > 366.844477 secs 2024-03-28T18:23:00.433 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:23:00 smithi079 ceph-mon[35312]: pgmap v500: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:23:00.433 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:23:00 smithi079 ceph-mon[35312]: 2 slow requests, 0 included below; oldest blocked for > 366.844477 secs 2024-03-28T18:23:02.425 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:23:02 smithi134 ceph-mon[35037]: pgmap v501: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:23:02.432 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:23:02 smithi079 ceph-mon[35312]: pgmap v501: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:23:02.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:23:02 smithi022 ceph-mon[31985]: pgmap v501: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:23:04.424 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:23:04 smithi134 ceph-mon[35037]: pgmap v502: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:23:04.433 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:23:04 smithi079 ceph-mon[35312]: pgmap v502: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:23:04.531 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:23:04 smithi022 ceph-mon[31985]: pgmap v502: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:23:05.425 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:23:05 smithi134 ceph-mon[35037]: 2 slow requests, 1 included below; oldest blocked for > 371.844556 secs 2024-03-28T18:23:05.425 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:23:05 smithi134 ceph-mon[35037]: slow request 364.146932 seconds old, received at 2024-03-28T18:17:00.155130+0000: client_request(client.35612:96 mksnap #0x1//scheduled-2024-03-28-18_17_00_UTC 2024-03-28T18:17:00.154942+0000 caller_uid=0, caller_gid=0{}) currently failed to xlock, waiting 2024-03-28T18:23:05.433 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:23:05 smithi079 ceph-mon[35312]: 2 slow requests, 1 included below; oldest blocked for > 371.844556 secs 2024-03-28T18:23:05.433 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:23:05 smithi079 ceph-mon[35312]: slow request 364.146932 seconds old, received at 2024-03-28T18:17:00.155130+0000: client_request(client.35612:96 mksnap #0x1//scheduled-2024-03-28-18_17_00_UTC 2024-03-28T18:17:00.154942+0000 caller_uid=0, caller_gid=0{}) currently failed to xlock, waiting 2024-03-28T18:23:05.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:23:05 smithi022 ceph-mon[31985]: 2 slow requests, 1 included below; oldest blocked for > 371.844556 secs 2024-03-28T18:23:05.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:23:05 smithi022 ceph-mon[31985]: slow request 364.146932 seconds old, received at 2024-03-28T18:17:00.155130+0000: client_request(client.35612:96 mksnap #0x1//scheduled-2024-03-28-18_17_00_UTC 2024-03-28T18:17:00.154942+0000 caller_uid=0, caller_gid=0{}) currently failed to xlock, waiting 2024-03-28T18:23:06.425 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:23:06 smithi134 ceph-mon[35037]: pgmap v503: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:23:06.432 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:23:06 smithi079 ceph-mon[35312]: pgmap v503: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:23:06.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:23:06 smithi022 ceph-mon[31985]: pgmap v503: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:23:08.425 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:23:08 smithi134 ceph-mon[35037]: pgmap v504: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:23:08.432 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:23:08 smithi079 ceph-mon[35312]: pgmap v504: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:23:08.531 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:23:08 smithi022 ceph-mon[31985]: pgmap v504: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:23:10.425 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:23:10 smithi134 ceph-mon[35037]: pgmap v505: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:23:10.425 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:23:10 smithi134 ceph-mon[35037]: 2 slow requests, 0 included below; oldest blocked for > 376.844629 secs 2024-03-28T18:23:10.432 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:23:10 smithi079 ceph-mon[35312]: pgmap v505: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:23:10.432 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:23:10 smithi079 ceph-mon[35312]: 2 slow requests, 0 included below; oldest blocked for > 376.844629 secs 2024-03-28T18:23:10.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:23:10 smithi022 ceph-mon[31985]: pgmap v505: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:23:10.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:23:10 smithi022 ceph-mon[31985]: 2 slow requests, 0 included below; oldest blocked for > 376.844629 secs 2024-03-28T18:23:12.424 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:23:12 smithi134 ceph-mon[35037]: pgmap v506: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:23:12.432 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:23:12 smithi079 ceph-mon[35312]: pgmap v506: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:23:12.531 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:23:12 smithi022 ceph-mon[31985]: pgmap v506: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:23:13.425 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:23:13 smithi134 ceph-mon[35037]: pgmap v507: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:23:13.432 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:23:13 smithi079 ceph-mon[35312]: pgmap v507: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:23:13.531 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:23:13 smithi022 ceph-mon[31985]: pgmap v507: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:23:14.424 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:23:14 smithi134 ceph-mon[35037]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T18:23:14.432 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:23:14 smithi079 ceph-mon[35312]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T18:23:14.531 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:23:14 smithi022 ceph-mon[31985]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T18:23:15.425 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:23:15 smithi134 ceph-mon[35037]: 2 slow requests, 0 included below; oldest blocked for > 381.844719 secs 2024-03-28T18:23:15.425 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:23:15 smithi134 ceph-mon[35037]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T18:23:15.425 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:23:15 smithi134 ceph-mon[35037]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T18:23:15.425 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:23:15 smithi134 ceph-mon[35037]: pgmap v508: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 10 op/s 2024-03-28T18:23:15.425 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:23:15 smithi134 ceph-mon[35037]: pgmap v509: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-03-28T18:23:15.425 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:23:15 smithi134 ceph-mon[35037]: from='mgr.15522 ' entity='mgr.x' 2024-03-28T18:23:15.432 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:23:15 smithi079 ceph-mon[35312]: 2 slow requests, 0 included below; oldest blocked for > 381.844719 secs 2024-03-28T18:23:15.433 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:23:15 smithi079 ceph-mon[35312]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T18:23:15.433 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:23:15 smithi079 ceph-mon[35312]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T18:23:15.433 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:23:15 smithi079 ceph-mon[35312]: pgmap v508: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 10 op/s 2024-03-28T18:23:15.433 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:23:15 smithi079 ceph-mon[35312]: pgmap v509: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-03-28T18:23:15.433 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:23:15 smithi079 ceph-mon[35312]: from='mgr.15522 ' entity='mgr.x' 2024-03-28T18:23:15.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:23:15 smithi022 ceph-mon[31985]: 2 slow requests, 0 included below; oldest blocked for > 381.844719 secs 2024-03-28T18:23:15.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:23:15 smithi022 ceph-mon[31985]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T18:23:15.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:23:15 smithi022 ceph-mon[31985]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T18:23:15.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:23:15 smithi022 ceph-mon[31985]: pgmap v508: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 10 op/s 2024-03-28T18:23:15.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:23:15 smithi022 ceph-mon[31985]: pgmap v509: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-03-28T18:23:15.533 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:23:15 smithi022 ceph-mon[31985]: from='mgr.15522 ' entity='mgr.x' 2024-03-28T18:23:18.032 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:23:17 smithi022 ceph-mon[31985]: pgmap v510: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-03-28T18:23:18.175 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:23:17 smithi134 ceph-mon[35037]: pgmap v510: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-03-28T18:23:18.182 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:23:17 smithi079 ceph-mon[35312]: pgmap v510: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-03-28T18:23:20.175 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:23:19 smithi134 ceph-mon[35037]: pgmap v511: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-03-28T18:23:20.175 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:23:19 smithi134 ceph-mon[35037]: 2 slow requests, 0 included below; oldest blocked for > 386.844838 secs 2024-03-28T18:23:20.182 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:23:19 smithi079 ceph-mon[35312]: pgmap v511: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-03-28T18:23:20.182 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:23:19 smithi079 ceph-mon[35312]: 2 slow requests, 0 included below; oldest blocked for > 386.844838 secs 2024-03-28T18:23:20.282 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:23:19 smithi022 ceph-mon[31985]: pgmap v511: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-03-28T18:23:20.282 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:23:19 smithi022 ceph-mon[31985]: 2 slow requests, 0 included below; oldest blocked for > 386.844838 secs 2024-03-28T18:23:22.174 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:23:21 smithi134 ceph-mon[35037]: pgmap v512: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-03-28T18:23:22.182 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:23:21 smithi079 ceph-mon[35312]: pgmap v512: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-03-28T18:23:22.282 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:23:21 smithi022 ceph-mon[31985]: pgmap v512: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-03-28T18:23:24.137 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-28T18:23:24.137 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-28T18:23:24.174 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:23:23 smithi134 ceph-mon[35037]: pgmap v513: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-03-28T18:23:24.182 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:23:23 smithi079 ceph-mon[35312]: pgmap v513: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-03-28T18:23:24.227 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:23:23 smithi022 ceph-mon[31985]: pgmap v513: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-03-28T18:23:24.368 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:23:24.368+0000 7f8a1d400640 1 -- 172.21.15.22:0/3957795245 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8a180743f0 msgr2=0x7f8a1810d7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:23:24.368 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:23:24.368+0000 7f8a1d400640 1 --2- 172.21.15.22:0/3957795245 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8a180743f0 0x7f8a1810d7e0 secure :-1 s=READY pgs=846 cs=0 l=1 rev1=1 crypto rx=0x7f8a00009900 tx=0x7f8a0002bcc0 comp rx=0 tx=0).stop 2024-03-28T18:23:24.369 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:23:24.368+0000 7f8a1d400640 1 -- 172.21.15.22:0/3957795245 shutdown_connections 2024-03-28T18:23:24.369 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:23:24.368+0000 7f8a1d400640 1 --2- 172.21.15.22:0/3957795245 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f8a1810dd20 0x7f8a18110170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:23:24.369 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:23:24.368+0000 7f8a1d400640 1 --2- 172.21.15.22:0/3957795245 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8a180743f0 0x7f8a1810d7e0 unknown :-1 s=CLOSED pgs=846 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:23:24.369 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:23:24.368+0000 7f8a1d400640 1 --2- 172.21.15.22:0/3957795245 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8a18073ad0 0x7f8a18073eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:23:24.369 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:23:24.368+0000 7f8a1d400640 1 -- 172.21.15.22:0/3957795245 >> 172.21.15.22:0/3957795245 conn(0x7f8a180fd010 msgr2=0x7f8a180ff430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:23:24.369 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:23:24.368+0000 7f8a1d400640 1 -- 172.21.15.22:0/3957795245 shutdown_connections 2024-03-28T18:23:24.369 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:23:24.368+0000 7f8a1d400640 1 -- 172.21.15.22:0/3957795245 wait complete. 2024-03-28T18:23:24.371 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:23:24.369+0000 7f8a1d400640 1 Processor -- start 2024-03-28T18:23:24.371 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:23:24.370+0000 7f8a1d400640 1 -- start start 2024-03-28T18:23:24.371 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:23:24.370+0000 7f8a1d400640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8a18073ad0 0x7f8a1819a030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:23:24.372 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:23:24.370+0000 7f8a1d400640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f8a180743f0 0x7f8a1819a570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:23:24.372 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:23:24.370+0000 7f8a1d400640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8a1810dd20 0x7f8a1819e8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:23:24.372 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:23:24.370+0000 7f8a1d400640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a18113a90 con 0x7f8a18073ad0 2024-03-28T18:23:24.372 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:23:24.370+0000 7f8a1d400640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a18113910 con 0x7f8a180743f0 2024-03-28T18:23:24.372 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:23:24.370+0000 7f8a1d400640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a18113c10 con 0x7f8a1810dd20 2024-03-28T18:23:24.372 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:23:24.370+0000 7f8a17e00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8a18073ad0 0x7f8a1819a030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:23:24.372 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:23:24.370+0000 7f8a17e00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8a18073ad0 0x7f8a1819a030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:34364/0 (socket says 172.21.15.22:34364) 2024-03-28T18:23:24.372 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:23:24.370+0000 7f8a17e00640 1 -- 172.21.15.22:0/28732272 learned_addr learned my addr 172.21.15.22:0/28732272 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-28T18:23:24.372 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:23:24.371+0000 7f8a0fe00640 1 --2- 172.21.15.22:0/28732272 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f8a180743f0 0x7f8a1819a570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:23:24.372 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:23:24.371+0000 7f8a1ca00640 1 --2- 172.21.15.22:0/28732272 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8a1810dd20 0x7f8a1819e8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:23:24.372 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:23:24.371+0000 7f8a17e00640 1 -- 172.21.15.22:0/28732272 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8a1810dd20 msgr2=0x7f8a1819e8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:23:24.372 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:23:24.371+0000 7f8a17e00640 1 --2- 172.21.15.22:0/28732272 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8a1810dd20 0x7f8a1819e8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:23:24.372 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:23:24.371+0000 7f8a17e00640 1 -- 172.21.15.22:0/28732272 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f8a180743f0 msgr2=0x7f8a1819a570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:23:24.372 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:23:24.371+0000 7f8a17e00640 1 --2- 172.21.15.22:0/28732272 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f8a180743f0 0x7f8a1819a570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:23:24.373 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:23:24.371+0000 7f8a17e00640 1 -- 172.21.15.22:0/28732272 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8a1819f020 con 0x7f8a18073ad0 2024-03-28T18:23:24.373 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:23:24.371+0000 7f8a17e00640 1 --2- 172.21.15.22:0/28732272 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8a18073ad0 0x7f8a1819a030 secure :-1 s=READY pgs=847 cs=0 l=1 rev1=1 crypto rx=0x7f8a0800d770 tx=0x7f8a0800dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:23:24.373 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:23:24.371+0000 7f8a15600640 1 -- 172.21.15.22:0/28732272 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a08017070 con 0x7f8a18073ad0 2024-03-28T18:23:24.373 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:23:24.371+0000 7f8a1d400640 1 -- 172.21.15.22:0/28732272 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8a1819f310 con 0x7f8a18073ad0 2024-03-28T18:23:24.373 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:23:24.371+0000 7f8a15600640 1 -- 172.21.15.22:0/28732272 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f8a080054f0 con 0x7f8a18073ad0 2024-03-28T18:23:24.373 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:23:24.372+0000 7f8a15600640 1 -- 172.21.15.22:0/28732272 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a08004d60 con 0x7f8a18073ad0 2024-03-28T18:23:24.373 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:23:24.372+0000 7f8a1d400640 1 -- 172.21.15.22:0/28732272 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8a181a6ba0 con 0x7f8a18073ad0 2024-03-28T18:23:24.375 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:23:24.375+0000 7f8a15600640 1 -- 172.21.15.22:0/28732272 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 26) v1 ==== 137421+0+0 (secure 0 0 0) 0x7f8a08004f00 con 0x7f8a18073ad0 2024-03-28T18:23:24.375 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:23:24.375+0000 7f8a2176f740 1 Processor -- start 2024-03-28T18:23:24.375 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:23:24.375+0000 7f8a2176f740 1 -- start start 2024-03-28T18:23:24.377 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:23:24.375+0000 7f8a2176f740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555d5c37ddd0 0x555d5c37cb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:23:24.377 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:23:24.375+0000 7f8a2176f740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x555d5c37d0f0 0x555d5c37d570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:23:24.377 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:23:24.375+0000 7f8a2176f740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x555d5c386460 0x555d5c388850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:23:24.377 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:23:24.375+0000 7f8a2176f740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x555d5c2e1250 con 0x555d5c37d0f0 2024-03-28T18:23:24.377 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:23:24.375+0000 7f8a2176f740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x555d5c289a30 con 0x555d5c386460 2024-03-28T18:23:24.377 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:23:24.375+0000 7f8a2176f740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x555d5c2dd2f0 con 0x555d5c37ddd0 2024-03-28T18:23:24.377 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:23:24.376+0000 7f8a17e00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x555d5c37d0f0 0x555d5c37d570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:23:24.377 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:23:24.376+0000 7f8a17e00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x555d5c37d0f0 0x555d5c37d570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:34376/0 (socket says 172.21.15.22:34376) 2024-03-28T18:23:24.378 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:23:24.376+0000 7f8a17e00640 1 -- 172.21.15.22:0/3150735341 learned_addr learned my addr 172.21.15.22:0/3150735341 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-28T18:23:24.378 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:23:24.376+0000 7f8a0fe00640 1 --2- 172.21.15.22:0/3150735341 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x555d5c386460 0x555d5c388850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:23:24.378 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:23:24.376+0000 7f8a1ca00640 1 --2- 172.21.15.22:0/3150735341 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555d5c37ddd0 0x555d5c37cb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:23:24.378 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:23:24.376+0000 7f8a17e00640 1 -- 172.21.15.22:0/3150735341 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555d5c37ddd0 msgr2=0x555d5c37cb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T18:23:24.378 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:23:24.376+0000 7f8a17e00640 1 --2- 172.21.15.22:0/3150735341 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555d5c37ddd0 0x555d5c37cb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:23:24.378 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:23:24.376+0000 7f8a15600640 1 --2- 172.21.15.22:0/28732272 >> [v2:172.21.15.22:6800/1357365082,v1:172.21.15.22:6801/1357365082] conn(0x7f89fc0a2970 0x7f89fc0a4e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:23:24.378 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:23:24.376+0000 7f8a17e00640 1 -- 172.21.15.22:0/3150735341 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x555d5c386460 msgr2=0x555d5c388850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T18:23:24.378 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:23:24.376+0000 7f8a17e00640 1 --2- 172.21.15.22:0/3150735341 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x555d5c386460 0x555d5c388850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:23:24.378 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:23:24.377+0000 7f8a17e00640 1 -- 172.21.15.22:0/3150735341 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555d5c274690 con 0x555d5c37d0f0 2024-03-28T18:23:24.378 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:23:24.377+0000 7f8a0fe00640 1 --2- 172.21.15.22:0/3150735341 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x555d5c386460 0x555d5c388850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-28T18:23:24.378 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:23:24.377+0000 7f8a1ca00640 1 --2- 172.21.15.22:0/28732272 >> [v2:172.21.15.22:6800/1357365082,v1:172.21.15.22:6801/1357365082] conn(0x7f89fc0a2970 0x7f89fc0a4e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:23:24.378 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:23:24.377+0000 7f8a15600640 1 -- 172.21.15.22:0/28732272 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(122..122 src has 1..122) v4 ==== 10389+0+0 (secure 0 0 0) 0x7f8a080cf090 con 0x7f8a18073ad0 2024-03-28T18:23:24.378 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:23:24.377+0000 7f8a17e00640 1 --2- 172.21.15.22:0/3150735341 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x555d5c37d0f0 0x555d5c37d570 secure :-1 s=READY pgs=848 cs=0 l=1 rev1=1 crypto rx=0x7f8a080cbf70 tx=0x7f8a08065cb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=168cdb9a9917e2f1 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:23:24.378 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:23:24.377+0000 7f8a0d200640 1 -- 172.21.15.22:0/3150735341 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a0809ad10 con 0x555d5c37d0f0 2024-03-28T18:23:24.379 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:23:24.377+0000 7f8a0d200640 1 -- 172.21.15.22:0/3150735341 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f8a080056a0 con 0x555d5c37d0f0 2024-03-28T18:23:24.379 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:23:24.377+0000 7f8a0d200640 1 -- 172.21.15.22:0/3150735341 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a080d2c30 con 0x555d5c37d0f0 2024-03-28T18:23:24.383 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:23:24.378+0000 7f8a2176f740 1 -- 172.21.15.22:0/3150735341 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x555d5c37d0f0 msgr2=0x555d5c37d570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:23:24.383 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:23:24.378+0000 7f8a1ca00640 1 --2- 172.21.15.22:0/28732272 >> [v2:172.21.15.22:6800/1357365082,v1:172.21.15.22:6801/1357365082] conn(0x7f89fc0a2970 0x7f89fc0a4e30 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f8a10005fd0 tx=0x7f8a10009500 comp rx=0 tx=0).ready entity=mgr.15522 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:23:24.383 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:23:24.378+0000 7f8a2176f740 1 --2- 172.21.15.22:0/3150735341 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x555d5c37d0f0 0x555d5c37d570 secure :-1 s=READY pgs=848 cs=0 l=1 rev1=1 crypto rx=0x7f8a080cbf70 tx=0x7f8a08065cb0 comp rx=0 tx=0).stop 2024-03-28T18:23:24.383 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:23:24.378+0000 7f8a2176f740 1 -- 172.21.15.22:0/3150735341 shutdown_connections 2024-03-28T18:23:24.383 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:23:24.378+0000 7f8a2176f740 1 --2- 172.21.15.22:0/3150735341 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x555d5c386460 0x555d5c388850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:23:24.383 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:23:24.378+0000 7f8a2176f740 1 --2- 172.21.15.22:0/3150735341 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x555d5c37d0f0 0x555d5c37d570 unknown :-1 s=CLOSED pgs=848 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:23:24.383 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:23:24.378+0000 7f8a2176f740 1 --2- 172.21.15.22:0/3150735341 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555d5c37ddd0 0x555d5c37cb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:23:24.383 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:23:24.378+0000 7f8a2176f740 1 -- 172.21.15.22:0/3150735341 >> 172.21.15.22:0/3150735341 conn(0x555d5c344200 msgr2=0x555d5c377b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:23:24.383 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:23:24.378+0000 7f8a2176f740 1 -- 172.21.15.22:0/3150735341 shutdown_connections 2024-03-28T18:23:24.383 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:23:24.378+0000 7f8a2176f740 1 -- 172.21.15.22:0/3150735341 wait complete. 2024-03-28T18:23:24.383 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:23:24.379+0000 7f8a2176f740 1 -- start start 2024-03-28T18:23:24.384 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:23:24.379+0000 7f8a2176f740 1 Processor -- start 2024-03-28T18:23:24.384 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:23:24.379+0000 7f8a2176f740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x555d5c37d0f0 0x555d5c385b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:23:24.384 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:23:24.380+0000 7f8a2176f740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555d5c37ddd0 0x555d5c382f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:23:24.384 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:23:24.380+0000 7f8a2176f740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x555d5c386460 0x555d5c383450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:23:24.384 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:23:24.380+0000 7f8a2176f740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x555d5c389270 con 0x555d5c386460 2024-03-28T18:23:24.384 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:23:24.380+0000 7f8a2176f740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x555d5c29ce80 con 0x555d5c37d0f0 2024-03-28T18:23:24.384 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:23:24.380+0000 7f8a2176f740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x555d5c3893f0 con 0x555d5c37ddd0 2024-03-28T18:23:24.384 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:23:24.380+0000 7f8a2176f740 10 client.0 _ll_register_callbacks cb 0x555d5c2af130 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-28T18:23:24.384 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:23:24.380+0000 7f8a1ca00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x555d5c386460 0x555d5c383450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:23:24.384 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:23:24.380+0000 7f8a17e00640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x555d5c37d0f0 0x555d5c385b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:23:24.384 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:23:24.380+0000 7f8a1ca00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x555d5c386460 0x555d5c383450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:34382/0 (socket says 172.21.15.22:34382) 2024-03-28T18:23:24.384 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:23:24.380+0000 7f8a17e00640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x555d5c37d0f0 0x555d5c385b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.22:37914/0 (socket says 172.21.15.22:37914) 2024-03-28T18:23:24.384 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:23:24.380+0000 7f8a1ca00640 1 -- 172.21.15.22:0/1952295378 learned_addr learned my addr 172.21.15.22:0/1952295378 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-28T18:23:24.384 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:23:24.380+0000 7f8a0fe00640 1 --2- 172.21.15.22:0/1952295378 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555d5c37ddd0 0x555d5c382f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:23:24.385 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:23:24.380+0000 7f8a1ca00640 1 -- 172.21.15.22:0/1952295378 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555d5c37ddd0 msgr2=0x555d5c382f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T18:23:24.385 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:23:24.380+0000 7f8a1ca00640 1 --2- 172.21.15.22:0/1952295378 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555d5c37ddd0 0x555d5c382f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:23:24.385 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:23:24.380+0000 7f8a1ca00640 1 -- 172.21.15.22:0/1952295378 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x555d5c37d0f0 msgr2=0x555d5c385b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T18:23:24.385 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:23:24.380+0000 7f8a1ca00640 1 --2- 172.21.15.22:0/1952295378 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x555d5c37d0f0 0x555d5c385b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:23:24.385 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:23:24.380+0000 7f8a1ca00640 1 -- 172.21.15.22:0/1952295378 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x555d5c0bf7d0 con 0x555d5c386460 2024-03-28T18:23:24.385 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:23:24.381+0000 7f8a1ca00640 1 --2- 172.21.15.22:0/1952295378 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x555d5c386460 0x555d5c383450 secure :-1 s=READY pgs=849 cs=0 l=1 rev1=1 crypto rx=0x7f8a10012700 tx=0x7f8a10004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1e3c2f6609a787cc server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:23:24.385 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:23:24.381+0000 7f8a05600640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-03-28T18:23:24.385 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:23:24.381+0000 7f8a05600640 1 -- 172.21.15.22:0/1952295378 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a1001d900 con 0x555d5c386460 2024-03-28T18:23:24.385 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:23:24.381+0000 7f8a05600640 1 -- 172.21.15.22:0/1952295378 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(122..122 src has 1..122) v4 ==== 10389+0+0 (secure 0 0 0) 0x7f8a1001daa0 con 0x555d5c386460 2024-03-28T18:23:24.385 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:23:24.381+0000 7f8a2176f740 1 -- 172.21.15.22:0/1952295378 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f8a080056a0 con 0x555d5c386460 2024-03-28T18:23:24.385 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:23:24.381+0000 7f8a05600640 1 -- 172.21.15.22:0/1952295378 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=189) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8a1001dca0 con 0x555d5c386460 2024-03-28T18:23:24.385 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:23:24.381+0000 7f8a2176f740 10 client.17031 fetch_fsmap learned FSMap version 189 2024-03-28T18:23:24.385 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:23:24.381+0000 7f8a2176f740 1 -- 172.21.15.22:0/1952295378 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=189,monmap=4+}) v3 -- 0x555d5bff53f0 con 0x555d5c386460 2024-03-28T18:23:24.385 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:23:24.382+0000 7f8a05600640 1 -- 172.21.15.22:0/1952295378 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f8a080056a0 con 0x555d5c386460 2024-03-28T18:23:24.385 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:23:24.382+0000 7f8a05600640 1 -- 172.21.15.22:0/1952295378 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 189) v1 ==== 6221+0+0 (secure 0 0 0) 0x7f8a1001e540 con 0x555d5c386460 2024-03-28T18:23:24.386 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:23:24.382+0000 7f8a2176f740 10 client.17031 fetch_fsmap finished waiting for FSMap version 189 2024-03-28T18:23:24.386 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:23:24.382+0000 7f8a2176f740 10 client.17031 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-28T18:23:24.386 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:23:24.382+0000 7f8a2176f740 20 client.17031 populate_metadata read hostname 'smithi022' 2024-03-28T18:23:24.386 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:23:24.382+0000 7f8a2176f740 1 --2- 172.21.15.22:0/1952295378 >> [v2:172.21.15.79:6838/2441542443,v1:172.21.15.79:6839/2441542443] conn(0x555d5c3964e0 0x555d5c3988d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:23:24.386 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:23:24.382+0000 7f8a2176f740 4 client.17031 mds_command: new command op to 34553 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-28T18:23:24.386 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:23:24.382+0000 7f8a2176f740 1 -- 172.21.15.22:0/1952295378 --> [v2:172.21.15.79:6838/2441542443,v1:172.21.15.79:6839/2441542443] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x555d5c2ee790 con 0x555d5c3964e0 2024-03-28T18:23:24.386 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:23:24.383+0000 7f8a17e00640 1 --2- 172.21.15.22:0/1952295378 >> [v2:172.21.15.79:6838/2441542443,v1:172.21.15.79:6839/2441542443] conn(0x555d5c3964e0 0x555d5c3988d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:23:24.386 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:23:24.383+0000 7f8a17e00640 1 --2- 172.21.15.22:0/1952295378 >> [v2:172.21.15.79:6838/2441542443,v1:172.21.15.79:6839/2441542443] conn(0x555d5c3964e0 0x555d5c3988d0 crc :-1 s=READY pgs=337 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=af9f9441ed01626a server_cookie=36b67d6332eae4af in_seq=0 out_seq=0 2024-03-28T18:23:24.386 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:23:24.383+0000 7f8a05600640 10 client.17031 ms_handle_connect on v2:172.21.15.79:6838/2441542443 2024-03-28T18:23:24.387 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:23:24.384+0000 7f8a05600640 1 -- 172.21.15.22:0/1952295378 <== mds.0 v2:172.21.15.79:6838/2441542443 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x555d5c2ee790 con 0x555d5c3964e0 2024-03-28T18:23:24.387 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:23:24.384+0000 7f8a05600640 10 client.17031 handle_command_reply: tid=0 2024-03-28T18:23:24.387 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:23:24.385+0000 7f8a2176f740 1 client.17031 shutdown 2024-03-28T18:23:24.388 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:23:24.385+0000 7f8a2176f740 1 -- 172.21.15.22:0/1952295378 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x555d5c386460 msgr2=0x555d5c383450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:23:24.388 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:23:24.385+0000 7f8a2176f740 1 --2- 172.21.15.22:0/1952295378 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x555d5c386460 0x555d5c383450 secure :-1 s=READY pgs=849 cs=0 l=1 rev1=1 crypto rx=0x7f8a10012700 tx=0x7f8a10004a40 comp rx=0 tx=0).stop 2024-03-28T18:23:24.388 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:23:24.385+0000 7f8a2176f740 1 -- 172.21.15.22:0/1952295378 shutdown_connections 2024-03-28T18:23:24.388 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:23:24.385+0000 7f8a2176f740 1 --2- 172.21.15.22:0/1952295378 >> [v2:172.21.15.79:6838/2441542443,v1:172.21.15.79:6839/2441542443] conn(0x555d5c3964e0 0x555d5c3988d0 crc :-1 s=READY pgs=337 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:23:24.388 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:23:24.385+0000 7f8a2176f740 1 --2- 172.21.15.22:0/1952295378 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x555d5c386460 0x555d5c383450 unknown :-1 s=CLOSED pgs=849 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:23:24.388 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:23:24.385+0000 7f8a2176f740 1 --2- 172.21.15.22:0/1952295378 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555d5c37ddd0 0x555d5c382f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:23:24.388 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:23:24.385+0000 7f8a2176f740 1 --2- 172.21.15.22:0/1952295378 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x555d5c37d0f0 0x555d5c385b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:23:24.388 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:23:24.385+0000 7f8a05600640 0 client.17031 ms_handle_reset on v2:172.21.15.79:6838/2441542443 2024-03-28T18:23:24.388 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:23:24.385+0000 7f8a2176f740 1 -- 172.21.15.22:0/1952295378 >> 172.21.15.22:0/1952295378 conn(0x555d5c344200 msgr2=0x555d5c379890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:23:24.388 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:23:24.385+0000 7f8a2176f740 1 -- 172.21.15.22:0/1952295378 shutdown_connections 2024-03-28T18:23:24.388 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:23:24.385+0000 7f8a2176f740 1 -- 172.21.15.22:0/1952295378 wait complete. 2024-03-28T18:23:24.388 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:23:24.385+0000 7f8a2176f740 20 client.17031 trim_cache size 0 max 16384 2024-03-28T18:23:24.403 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:23:24.403+0000 7f8a2176f740 1 Processor -- start 2024-03-28T18:23:24.403 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:23:24.403+0000 7f8a2176f740 1 -- start start 2024-03-28T18:23:24.404 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:23:24.403+0000 7f8a2176f740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x555d5c37d0f0 0x555d5c385640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:23:24.404 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:23:24.403+0000 7f8a2176f740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555d5c37ddd0 0x555d5c385b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:23:24.404 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:23:24.404+0000 7f8a2176f740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x555d5c386460 0x555d5c387850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:23:24.405 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:23:24.404+0000 7f8a2176f740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a181b1c70 con 0x555d5c386460 2024-03-28T18:23:24.405 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:23:24.404+0000 7f8a2176f740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a181aefd0 con 0x555d5c37d0f0 2024-03-28T18:23:24.405 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:23:24.404+0000 7f8a2176f740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a181b0fb0 con 0x555d5c37ddd0 2024-03-28T18:23:24.405 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:23:24.404+0000 7f8a1ca00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x555d5c386460 0x555d5c387850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:23:24.405 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:23:24.404+0000 7f8a1ca00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x555d5c386460 0x555d5c387850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:34396/0 (socket says 172.21.15.22:34396) 2024-03-28T18:23:24.405 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:23:24.404+0000 7f8a1ca00640 1 -- 172.21.15.22:0/1149096325 learned_addr learned my addr 172.21.15.22:0/1149096325 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-28T18:23:24.405 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:23:24.404+0000 7f8a17e00640 1 --2- 172.21.15.22:0/1149096325 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x555d5c37d0f0 0x555d5c385640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:23:24.405 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:23:24.404+0000 7f8a0fe00640 1 --2- 172.21.15.22:0/1149096325 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555d5c37ddd0 0x555d5c385b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:23:24.405 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:23:24.404+0000 7f8a1ca00640 1 -- 172.21.15.22:0/1149096325 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555d5c37ddd0 msgr2=0x555d5c385b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T18:23:24.405 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:23:24.404+0000 7f8a1ca00640 1 --2- 172.21.15.22:0/1149096325 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555d5c37ddd0 0x555d5c385b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:23:24.405 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:23:24.404+0000 7f8a1ca00640 1 -- 172.21.15.22:0/1149096325 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x555d5c37d0f0 msgr2=0x555d5c385640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T18:23:24.405 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:23:24.404+0000 7f8a1ca00640 1 --2- 172.21.15.22:0/1149096325 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x555d5c37d0f0 0x555d5c385640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:23:24.405 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:23:24.404+0000 7f8a1ca00640 1 -- 172.21.15.22:0/1149096325 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555d5bfb50f0 con 0x555d5c386460 2024-03-28T18:23:24.405 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:23:24.405+0000 7f8a1ca00640 1 --2- 172.21.15.22:0/1149096325 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x555d5c386460 0x555d5c387850 secure :-1 s=READY pgs=850 cs=0 l=1 rev1=1 crypto rx=0x7f8a100126d0 tx=0x7f8a10009ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=866b875261566920 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:23:24.409 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:23:24.405+0000 7f8a07e00640 1 -- 172.21.15.22:0/1149096325 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a100046d0 con 0x555d5c386460 2024-03-28T18:23:24.409 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:23:24.405+0000 7f8a07e00640 1 -- 172.21.15.22:0/1149096325 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f8a10022070 con 0x555d5c386460 2024-03-28T18:23:24.409 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:23:24.405+0000 7f8a07e00640 1 -- 172.21.15.22:0/1149096325 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a10004d10 con 0x555d5c386460 2024-03-28T18:23:24.409 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:23:24.406+0000 7f8a2176f740 1 -- 172.21.15.22:0/1149096325 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x555d5c386460 msgr2=0x555d5c387850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:23:24.409 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:23:24.406+0000 7f8a2176f740 1 --2- 172.21.15.22:0/1149096325 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x555d5c386460 0x555d5c387850 secure :-1 s=READY pgs=850 cs=0 l=1 rev1=1 crypto rx=0x7f8a100126d0 tx=0x7f8a10009ed0 comp rx=0 tx=0).stop 2024-03-28T18:23:24.409 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:23:24.406+0000 7f8a2176f740 1 -- 172.21.15.22:0/1149096325 shutdown_connections 2024-03-28T18:23:24.409 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:23:24.406+0000 7f8a2176f740 1 --2- 172.21.15.22:0/1149096325 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x555d5c386460 0x555d5c387850 unknown :-1 s=CLOSED pgs=850 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:23:24.409 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:23:24.406+0000 7f8a2176f740 1 --2- 172.21.15.22:0/1149096325 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555d5c37ddd0 0x555d5c385b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:23:24.409 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:23:24.406+0000 7f8a2176f740 1 --2- 172.21.15.22:0/1149096325 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x555d5c37d0f0 0x555d5c385640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:23:24.409 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:23:24.406+0000 7f8a2176f740 1 -- 172.21.15.22:0/1149096325 >> 172.21.15.22:0/1149096325 conn(0x555d5c344200 msgr2=0x555d5c377670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:23:24.409 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:23:24.406+0000 7f8a2176f740 1 -- 172.21.15.22:0/1149096325 shutdown_connections 2024-03-28T18:23:24.410 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:23:24.406+0000 7f8a2176f740 1 -- 172.21.15.22:0/1149096325 wait complete. 2024-03-28T18:23:24.410 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:23:24.406+0000 7f8a2176f740 1 -- start start 2024-03-28T18:23:24.410 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:23:24.406+0000 7f8a2176f740 1 Processor -- start 2024-03-28T18:23:24.410 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:23:24.407+0000 7f8a2176f740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555d5c37d0f0 0x555d5c380190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:23:24.410 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:23:24.407+0000 7f8a2176f740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x555d5c37ddd0 0x555d5c3806d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:23:24.410 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:23:24.407+0000 7f8a2176f740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x555d5c386460 0x555d5c380c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:23:24.410 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:23:24.407+0000 7f8a2176f740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a181b0950 con 0x555d5c386460 2024-03-28T18:23:24.410 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:23:24.407+0000 7f8a2176f740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a181af630 con 0x555d5c37ddd0 2024-03-28T18:23:24.410 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:23:24.407+0000 7f8a2176f740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a181afc90 con 0x555d5c37d0f0 2024-03-28T18:23:24.410 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:23:24.407+0000 7f8a2176f740 10 client.0 _ll_register_callbacks cb 0x7f8a10009cc0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-28T18:23:24.410 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:23:24.407+0000 7f8a1ca00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x555d5c386460 0x555d5c380c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:23:24.410 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:23:24.407+0000 7f8a1ca00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x555d5c386460 0x555d5c380c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:34406/0 (socket says 172.21.15.22:34406) 2024-03-28T18:23:24.410 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:23:24.407+0000 7f8a1ca00640 1 -- 172.21.15.22:0/802177433 learned_addr learned my addr 172.21.15.22:0/802177433 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-28T18:23:24.411 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:23:24.407+0000 7f8a0fe00640 1 --2- 172.21.15.22:0/802177433 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x555d5c37ddd0 0x555d5c3806d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:23:24.411 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:23:24.408+0000 7f8a17e00640 1 --2- 172.21.15.22:0/802177433 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555d5c37d0f0 0x555d5c380190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:23:24.411 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:23:24.408+0000 7f8a1ca00640 1 -- 172.21.15.22:0/802177433 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555d5c37d0f0 msgr2=0x555d5c380190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T18:23:24.411 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:23:24.408+0000 7f8a1ca00640 1 --2- 172.21.15.22:0/802177433 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555d5c37d0f0 0x555d5c380190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:23:24.411 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:23:24.408+0000 7f8a1ca00640 1 -- 172.21.15.22:0/802177433 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x555d5c37ddd0 msgr2=0x555d5c3806d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T18:23:24.411 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:23:24.408+0000 7f8a1ca00640 1 --2- 172.21.15.22:0/802177433 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x555d5c37ddd0 0x555d5c3806d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:23:24.411 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:23:24.408+0000 7f8a1ca00640 1 -- 172.21.15.22:0/802177433 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x555d5c384880 con 0x555d5c386460 2024-03-28T18:23:24.411 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:23:24.408+0000 7f8a17e00640 1 --2- 172.21.15.22:0/802177433 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555d5c37d0f0 0x555d5c380190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T18:23:24.411 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:23:24.408+0000 7f8a1ca00640 1 --2- 172.21.15.22:0/802177433 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x555d5c386460 0x555d5c380c10 secure :-1 s=READY pgs=851 cs=0 l=1 rev1=1 crypto rx=0x7f8a10004010 tx=0x7f8a1001e6d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=38b0423114d2f29c server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:23:24.411 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:23:24.408+0000 7f8a05600640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-03-28T18:23:24.411 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:23:24.408+0000 7f8a05600640 1 -- 172.21.15.22:0/802177433 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a1001d700 con 0x555d5c386460 2024-03-28T18:23:24.411 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:23:24.408+0000 7f8a05600640 1 -- 172.21.15.22:0/802177433 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(122..122 src has 1..122) v4 ==== 10389+0+0 (secure 0 0 0) 0x7f8a1001d8a0 con 0x555d5c386460 2024-03-28T18:23:24.411 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:23:24.408+0000 7f8a2176f740 1 -- 172.21.15.22:0/802177433 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f8a10022070 con 0x555d5c386460 2024-03-28T18:23:24.411 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:23:24.409+0000 7f8a05600640 1 -- 172.21.15.22:0/802177433 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=189) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8a1001daa0 con 0x555d5c386460 2024-03-28T18:23:24.411 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:23:24.409+0000 7f8a2176f740 10 client.17037 fetch_fsmap learned FSMap version 189 2024-03-28T18:23:24.412 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:23:24.409+0000 7f8a2176f740 1 -- 172.21.15.22:0/802177433 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=189,monmap=4+}) v3 -- 0x555d5c3fa9b0 con 0x555d5c386460 2024-03-28T18:23:24.412 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:23:24.409+0000 7f8a05600640 1 -- 172.21.15.22:0/802177433 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f8a10022070 con 0x555d5c386460 2024-03-28T18:23:24.412 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:23:24.409+0000 7f8a05600640 1 -- 172.21.15.22:0/802177433 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 189) v1 ==== 6221+0+0 (secure 0 0 0) 0x7f8a10005070 con 0x555d5c386460 2024-03-28T18:23:24.412 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:23:24.409+0000 7f8a2176f740 10 client.17037 fetch_fsmap finished waiting for FSMap version 189 2024-03-28T18:23:24.412 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:23:24.409+0000 7f8a2176f740 10 client.17037 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-28T18:23:24.412 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:23:24.409+0000 7f8a2176f740 20 client.17037 populate_metadata read hostname 'smithi022' 2024-03-28T18:23:24.412 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:23:24.409+0000 7f8a2176f740 1 --2- 172.21.15.22:0/802177433 >> [v2:172.21.15.79:6838/2441542443,v1:172.21.15.79:6839/2441542443] conn(0x555d5c3fafa0 0x555d5c3fd3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:23:24.412 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:23:24.409+0000 7f8a2176f740 4 client.17037 mds_command: new command op to 34553 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-28T18:23:24.412 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:23:24.409+0000 7f8a2176f740 1 -- 172.21.15.22:0/802177433 --> [v2:172.21.15.79:6838/2441542443,v1:172.21.15.79:6839/2441542443] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x555d5c2e8ae0 con 0x555d5c3fafa0 2024-03-28T18:23:24.413 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:23:24.410+0000 7f8a17e00640 1 --2- 172.21.15.22:0/802177433 >> [v2:172.21.15.79:6838/2441542443,v1:172.21.15.79:6839/2441542443] conn(0x555d5c3fafa0 0x555d5c3fd3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:23:24.413 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:23:24.410+0000 7f8a17e00640 1 --2- 172.21.15.22:0/802177433 >> [v2:172.21.15.79:6838/2441542443,v1:172.21.15.79:6839/2441542443] conn(0x555d5c3fafa0 0x555d5c3fd3c0 crc :-1 s=READY pgs=338 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6107c2c584eb095d server_cookie=4bb137709099ec42 in_seq=0 out_seq=0 2024-03-28T18:23:24.413 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:23:24.410+0000 7f8a05600640 10 client.17037 ms_handle_connect on v2:172.21.15.79:6838/2441542443 2024-03-28T18:23:24.413 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:23:24.411+0000 7f8a05600640 1 -- 172.21.15.22:0/802177433 <== mds.0 v2:172.21.15.79:6838/2441542443 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x555d5c2e8ae0 con 0x555d5c3fafa0 2024-03-28T18:23:24.414 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:23:24.411+0000 7f8a05600640 10 client.17037 handle_command_reply: tid=0 2024-03-28T18:23:24.414 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:23:24.411+0000 7f8a2176f740 1 client.17037 shutdown 2024-03-28T18:23:24.414 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:23:24.411+0000 7f8a2176f740 1 -- 172.21.15.22:0/802177433 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x555d5c386460 msgr2=0x555d5c380c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:23:24.414 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:23:24.411+0000 7f8a2176f740 1 --2- 172.21.15.22:0/802177433 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x555d5c386460 0x555d5c380c10 secure :-1 s=READY pgs=851 cs=0 l=1 rev1=1 crypto rx=0x7f8a10004010 tx=0x7f8a1001e6d0 comp rx=0 tx=0).stop 2024-03-28T18:23:24.414 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:23:24.411+0000 7f8a2176f740 1 -- 172.21.15.22:0/802177433 shutdown_connections 2024-03-28T18:23:24.414 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:23:24.411+0000 7f8a2176f740 1 --2- 172.21.15.22:0/802177433 >> [v2:172.21.15.79:6838/2441542443,v1:172.21.15.79:6839/2441542443] conn(0x555d5c3fafa0 0x555d5c3fd3c0 crc :-1 s=READY pgs=338 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:23:24.414 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:23:24.411+0000 7f8a2176f740 1 --2- 172.21.15.22:0/802177433 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x555d5c386460 0x555d5c380c10 unknown :-1 s=CLOSED pgs=851 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:23:24.414 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:23:24.411+0000 7f8a2176f740 1 --2- 172.21.15.22:0/802177433 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x555d5c37ddd0 0x555d5c3806d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:23:24.414 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:23:24.411+0000 7f8a2176f740 1 --2- 172.21.15.22:0/802177433 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555d5c37d0f0 0x555d5c380190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:23:24.414 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:23:24.411+0000 7f8a05600640 0 client.17037 ms_handle_reset on v2:172.21.15.79:6838/2441542443 2024-03-28T18:23:24.414 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:23:24.411+0000 7f8a2176f740 1 -- 172.21.15.22:0/802177433 >> 172.21.15.22:0/802177433 conn(0x555d5c344200 msgr2=0x555d5c3817e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:23:24.414 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:23:24.411+0000 7f8a2176f740 1 -- 172.21.15.22:0/802177433 shutdown_connections 2024-03-28T18:23:24.414 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:23:24.411+0000 7f8a2176f740 1 -- 172.21.15.22:0/802177433 wait complete. 2024-03-28T18:23:24.414 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:23:24.411+0000 7f8a2176f740 20 client.17037 trim_cache size 0 max 16384 2024-03-28T18:23:24.415 INFO:teuthology.orchestra.run.smithi022.stdout:{ 2024-03-28T18:23:24.415 INFO:teuthology.orchestra.run.smithi022.stdout: "status": "IDLE (waiting for more scrubs)", 2024-03-28T18:23:24.415 INFO:teuthology.orchestra.run.smithi022.stdout: "scrubs": { 2024-03-28T18:23:24.415 INFO:teuthology.orchestra.run.smithi022.stdout: "4bbbd61d-29a3-4bf6-a4f2-d5783d585de3": { 2024-03-28T18:23:24.415 INFO:teuthology.orchestra.run.smithi022.stdout: "path": "/", 2024-03-28T18:23:24.415 INFO:teuthology.orchestra.run.smithi022.stdout: "tag": "4bbbd61d-29a3-4bf6-a4f2-d5783d585de3", 2024-03-28T18:23:24.415 INFO:teuthology.orchestra.run.smithi022.stdout: "options": "recursive,force" 2024-03-28T18:23:24.415 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-03-28T18:23:24.415 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-03-28T18:23:24.415 INFO:teuthology.orchestra.run.smithi022.stdout:} 2024-03-28T18:23:24.416 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:23:24.412+0000 7f8a1d400640 1 -- 172.21.15.22:0/28732272 >> [v2:172.21.15.22:6800/1357365082,v1:172.21.15.22:6801/1357365082] conn(0x7f89fc0a2970 msgr2=0x7f89fc0a4e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:23:24.416 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:23:24.412+0000 7f8a1d400640 1 --2- 172.21.15.22:0/28732272 >> [v2:172.21.15.22:6800/1357365082,v1:172.21.15.22:6801/1357365082] conn(0x7f89fc0a2970 0x7f89fc0a4e30 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f8a10005fd0 tx=0x7f8a10009500 comp rx=0 tx=0).stop 2024-03-28T18:23:24.416 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:23:24.412+0000 7f8a1d400640 1 -- 172.21.15.22:0/28732272 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8a18073ad0 msgr2=0x7f8a1819a030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:23:24.416 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:23:24.412+0000 7f8a1d400640 1 --2- 172.21.15.22:0/28732272 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8a18073ad0 0x7f8a1819a030 secure :-1 s=READY pgs=847 cs=0 l=1 rev1=1 crypto rx=0x7f8a0800d770 tx=0x7f8a0800dc30 comp rx=0 tx=0).stop 2024-03-28T18:23:24.416 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:23:24.412+0000 7f8a1d400640 1 -- 172.21.15.22:0/28732272 shutdown_connections 2024-03-28T18:23:24.416 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:23:24.412+0000 7f8a1d400640 1 --2- 172.21.15.22:0/28732272 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8a1810dd20 0x7f8a1819e8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:23:24.416 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:23:24.412+0000 7f8a1d400640 1 --2- 172.21.15.22:0/28732272 >> [v2:172.21.15.22:6800/1357365082,v1:172.21.15.22:6801/1357365082] conn(0x7f89fc0a2970 0x7f89fc0a4e30 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:23:24.416 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:23:24.412+0000 7f8a1d400640 1 --2- 172.21.15.22:0/28732272 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f8a180743f0 0x7f8a1819a570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:23:24.417 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:23:24.412+0000 7f8a1d400640 1 --2- 172.21.15.22:0/28732272 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8a18073ad0 0x7f8a1819a030 unknown :-1 s=CLOSED pgs=847 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:23:24.417 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:23:24.412+0000 7f8a1d400640 1 -- 172.21.15.22:0/28732272 >> 172.21.15.22:0/28732272 conn(0x7f8a180fd010 msgr2=0x7f8a180fe870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:23:24.417 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:23:24.412+0000 7f8a1d400640 1 -- 172.21.15.22:0/28732272 shutdown_connections 2024-03-28T18:23:24.417 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:23:24.412+0000 7f8a1d400640 1 -- 172.21.15.22:0/28732272 wait complete. 2024-03-28T18:23:24.432 INFO:tasks.cephfs.filesystem:scrub status for tag:4bbbd61d-29a3-4bf6-a4f2-d5783d585de3 - {'path': '/', 'tag': '4bbbd61d-29a3-4bf6-a4f2-d5783d585de3', 'options': 'recursive,force'} 2024-03-28T18:23:25.174 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:23:24 smithi134 ceph-mon[35037]: 2 slow requests, 0 included below; oldest blocked for > 391.844908 secs 2024-03-28T18:23:25.182 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:23:24 smithi079 ceph-mon[35312]: 2 slow requests, 0 included below; oldest blocked for > 391.844908 secs 2024-03-28T18:23:25.282 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:23:24 smithi022 ceph-mon[31985]: 2 slow requests, 0 included below; oldest blocked for > 391.844908 secs 2024-03-28T18:23:26.175 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:23:25 smithi134 ceph-mon[35037]: pgmap v514: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 7 op/s 2024-03-28T18:23:26.182 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:23:25 smithi079 ceph-mon[35312]: pgmap v514: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 7 op/s 2024-03-28T18:23:26.282 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:23:25 smithi022 ceph-mon[31985]: pgmap v514: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 7 op/s 2024-03-28T18:23:28.174 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:23:27 smithi134 ceph-mon[35037]: pgmap v515: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:23:28.182 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:23:27 smithi079 ceph-mon[35312]: pgmap v515: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:23:28.282 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:23:27 smithi022 ceph-mon[31985]: pgmap v515: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:23:30.086 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:23:29 smithi022 ceph-mon[31985]: pgmap v516: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:23:30.086 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:23:29 smithi022 ceph-mon[31985]: 2 slow requests, 0 included below; oldest blocked for > 396.845022 secs 2024-03-28T18:23:30.175 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:23:29 smithi134 ceph-mon[35037]: pgmap v516: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:23:30.175 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:23:29 smithi134 ceph-mon[35037]: 2 slow requests, 0 included below; oldest blocked for > 396.845022 secs 2024-03-28T18:23:30.182 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:23:29 smithi079 ceph-mon[35312]: pgmap v516: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:23:30.183 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:23:29 smithi079 ceph-mon[35312]: 2 slow requests, 0 included below; oldest blocked for > 396.845022 secs 2024-03-28T18:23:32.174 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:23:31 smithi134 ceph-mon[35037]: pgmap v517: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:23:32.182 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:23:31 smithi079 ceph-mon[35312]: pgmap v517: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:23:32.282 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:23:31 smithi022 ceph-mon[31985]: pgmap v517: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:23:34.175 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:23:33 smithi134 ceph-mon[35037]: pgmap v518: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:23:34.182 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:23:33 smithi079 ceph-mon[35312]: pgmap v518: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:23:34.282 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:23:33 smithi022 ceph-mon[31985]: pgmap v518: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:23:35.175 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:23:34 smithi134 ceph-mon[35037]: 2 slow requests, 0 included below; oldest blocked for > 401.845141 secs 2024-03-28T18:23:35.182 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:23:34 smithi079 ceph-mon[35312]: 2 slow requests, 0 included below; oldest blocked for > 401.845141 secs 2024-03-28T18:23:35.282 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:23:34 smithi022 ceph-mon[31985]: 2 slow requests, 0 included below; oldest blocked for > 401.845141 secs 2024-03-28T18:23:36.174 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:23:35 smithi134 ceph-mon[35037]: pgmap v519: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:23:36.182 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:23:35 smithi079 ceph-mon[35312]: pgmap v519: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:23:36.282 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:23:35 smithi022 ceph-mon[31985]: pgmap v519: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:23:38.175 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:23:37 smithi134 ceph-mon[35037]: pgmap v520: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:23:38.182 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:23:37 smithi079 ceph-mon[35312]: pgmap v520: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:23:38.282 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:23:37 smithi022 ceph-mon[31985]: pgmap v520: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:23:40.175 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:23:39 smithi134 ceph-mon[35037]: pgmap v521: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:23:40.175 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:23:39 smithi134 ceph-mon[35037]: 2 slow requests, 0 included below; oldest blocked for > 406.845225 secs 2024-03-28T18:23:40.182 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:23:39 smithi079 ceph-mon[35312]: pgmap v521: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:23:40.182 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:23:39 smithi079 ceph-mon[35312]: 2 slow requests, 0 included below; oldest blocked for > 406.845225 secs 2024-03-28T18:23:40.282 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:23:39 smithi022 ceph-mon[31985]: pgmap v521: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:23:40.282 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:23:39 smithi022 ceph-mon[31985]: 2 slow requests, 0 included below; oldest blocked for > 406.845225 secs 2024-03-28T18:23:42.175 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:23:41 smithi134 ceph-mon[35037]: pgmap v522: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:23:42.182 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:23:41 smithi079 ceph-mon[35312]: pgmap v522: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:23:42.282 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:23:41 smithi022 ceph-mon[31985]: pgmap v522: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:23:44.175 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:23:43 smithi134 ceph-mon[35037]: pgmap v523: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:23:44.182 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:23:43 smithi079 ceph-mon[35312]: pgmap v523: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:23:44.282 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:23:43 smithi022 ceph-mon[31985]: pgmap v523: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:23:45.175 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:23:44 smithi134 ceph-mon[35037]: 2 slow requests, 0 included below; oldest blocked for > 411.845286 secs 2024-03-28T18:23:45.182 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:23:44 smithi079 ceph-mon[35312]: 2 slow requests, 0 included below; oldest blocked for > 411.845286 secs 2024-03-28T18:23:45.282 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:23:44 smithi022 ceph-mon[31985]: 2 slow requests, 0 included below; oldest blocked for > 411.845286 secs 2024-03-28T18:23:46.175 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:23:45 smithi134 ceph-mon[35037]: pgmap v524: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:23:46.183 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:23:45 smithi079 ceph-mon[35312]: pgmap v524: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:23:46.282 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:23:45 smithi022 ceph-mon[31985]: pgmap v524: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:23:48.175 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:23:47 smithi134 ceph-mon[35037]: pgmap v525: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:23:48.182 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:23:47 smithi079 ceph-mon[35312]: pgmap v525: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:23:48.282 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:23:47 smithi022 ceph-mon[31985]: pgmap v525: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:23:50.175 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:23:49 smithi134 ceph-mon[35037]: pgmap v526: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:23:50.175 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:23:49 smithi134 ceph-mon[35037]: 2 slow requests, 0 included below; oldest blocked for > 416.845364 secs 2024-03-28T18:23:50.182 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:23:49 smithi079 ceph-mon[35312]: pgmap v526: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:23:50.182 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:23:49 smithi079 ceph-mon[35312]: 2 slow requests, 0 included below; oldest blocked for > 416.845364 secs 2024-03-28T18:23:50.282 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:23:49 smithi022 ceph-mon[31985]: pgmap v526: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:23:50.282 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:23:49 smithi022 ceph-mon[31985]: 2 slow requests, 0 included below; oldest blocked for > 416.845364 secs 2024-03-28T18:23:52.175 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:23:51 smithi134 ceph-mon[35037]: pgmap v527: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:23:52.182 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:23:51 smithi079 ceph-mon[35312]: pgmap v527: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:23:52.282 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:23:51 smithi022 ceph-mon[31985]: pgmap v527: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:23:54.174 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:23:53 smithi134 ceph-mon[35037]: pgmap v528: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:23:54.182 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:23:53 smithi079 ceph-mon[35312]: pgmap v528: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:23:54.282 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:23:53 smithi022 ceph-mon[31985]: pgmap v528: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:23:54.433 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-28T18:23:54.434 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-28T18:23:54.670 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:23:54.670+0000 7fd0f3000640 1 -- 172.21.15.22:0/1053353854 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd0ec109730 msgr2=0x7fd0ec109b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:23:54.670 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:23:54.670+0000 7fd0f3000640 1 --2- 172.21.15.22:0/1053353854 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd0ec109730 0x7fd0ec109b10 secure :-1 s=READY pgs=852 cs=0 l=1 rev1=1 crypto rx=0x7fd0dc009930 tx=0x7fd0dc02bc20 comp rx=0 tx=0).stop 2024-03-28T18:23:54.671 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:23:54.670+0000 7fd0f3000640 1 -- 172.21.15.22:0/1053353854 shutdown_connections 2024-03-28T18:23:54.671 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:23:54.670+0000 7fd0f3000640 1 --2- 172.21.15.22:0/1053353854 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd0ec1040e0 0x7fd0ec1105f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:23:54.671 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:23:54.670+0000 7fd0f3000640 1 --2- 172.21.15.22:0/1053353854 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd0ec103720 0x7fd0ec103ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:23:54.671 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:23:54.670+0000 7fd0f3000640 1 --2- 172.21.15.22:0/1053353854 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd0ec109730 0x7fd0ec109b10 unknown :-1 s=CLOSED pgs=852 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:23:54.671 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:23:54.670+0000 7fd0f3000640 1 -- 172.21.15.22:0/1053353854 >> 172.21.15.22:0/1053353854 conn(0x7fd0ec0ff1d0 msgr2=0x7fd0ec1015f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:23:54.671 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:23:54.670+0000 7fd0f3000640 1 -- 172.21.15.22:0/1053353854 shutdown_connections 2024-03-28T18:23:54.671 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:23:54.670+0000 7fd0f3000640 1 -- 172.21.15.22:0/1053353854 wait complete. 2024-03-28T18:23:54.673 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:23:54.671+0000 7fd0f3000640 1 Processor -- start 2024-03-28T18:23:54.673 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:23:54.671+0000 7fd0f3000640 1 -- start start 2024-03-28T18:23:54.673 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:23:54.672+0000 7fd0f3000640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd0ec103720 0x7fd0ec199fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:23:54.673 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:23:54.672+0000 7fd0f3000640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd0ec1040e0 0x7fd0ec19a510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:23:54.673 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:23:54.672+0000 7fd0f3000640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd0ec109730 0x7fd0ec19e890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:23:54.673 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:23:54.672+0000 7fd0f3000640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0ec113ad0 con 0x7fd0ec103720 2024-03-28T18:23:54.674 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:23:54.672+0000 7fd0f3000640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0ec113950 con 0x7fd0ec1040e0 2024-03-28T18:23:54.674 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:23:54.672+0000 7fd0f3000640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0ec113c50 con 0x7fd0ec109730 2024-03-28T18:23:54.674 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:23:54.672+0000 7fd0f1c00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd0ec103720 0x7fd0ec199fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:23:54.674 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:23:54.672+0000 7fd0f1200640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd0ec1040e0 0x7fd0ec19a510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:23:54.674 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:23:54.672+0000 7fd0f1c00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd0ec103720 0x7fd0ec199fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:54264/0 (socket says 172.21.15.22:54264) 2024-03-28T18:23:54.674 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:23:54.672+0000 7fd0f1c00640 1 -- 172.21.15.22:0/3085079497 learned_addr learned my addr 172.21.15.22:0/3085079497 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-28T18:23:54.674 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:23:54.672+0000 7fd0f2600640 1 --2- 172.21.15.22:0/3085079497 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd0ec109730 0x7fd0ec19e890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:23:54.674 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:23:54.672+0000 7fd0f1c00640 1 -- 172.21.15.22:0/3085079497 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd0ec109730 msgr2=0x7fd0ec19e890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:23:54.674 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:23:54.672+0000 7fd0f1c00640 1 --2- 172.21.15.22:0/3085079497 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd0ec109730 0x7fd0ec19e890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:23:54.674 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:23:54.672+0000 7fd0f1c00640 1 -- 172.21.15.22:0/3085079497 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd0ec1040e0 msgr2=0x7fd0ec19a510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:23:54.674 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:23:54.672+0000 7fd0f1c00640 1 --2- 172.21.15.22:0/3085079497 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd0ec1040e0 0x7fd0ec19a510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:23:54.674 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:23:54.672+0000 7fd0f1c00640 1 -- 172.21.15.22:0/3085079497 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd0ec19efc0 con 0x7fd0ec103720 2024-03-28T18:23:54.674 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:23:54.673+0000 7fd0f1c00640 1 --2- 172.21.15.22:0/3085079497 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd0ec103720 0x7fd0ec199fd0 secure :-1 s=READY pgs=853 cs=0 l=1 rev1=1 crypto rx=0x7fd0dc009900 tx=0x7fd0dc0045e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:23:54.674 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:23:54.673+0000 7fd0ea000640 1 -- 172.21.15.22:0/3085079497 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0dc043020 con 0x7fd0ec103720 2024-03-28T18:23:54.675 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:23:54.673+0000 7fd0ea000640 1 -- 172.21.15.22:0/3085079497 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7fd0dc02c5c0 con 0x7fd0ec103720 2024-03-28T18:23:54.675 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:23:54.673+0000 7fd0ea000640 1 -- 172.21.15.22:0/3085079497 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0dc003970 con 0x7fd0ec103720 2024-03-28T18:23:54.675 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:23:54.673+0000 7fd0f3000640 1 -- 172.21.15.22:0/3085079497 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd0ec19f250 con 0x7fd0ec103720 2024-03-28T18:23:54.675 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:23:54.673+0000 7fd0f3000640 1 -- 172.21.15.22:0/3085079497 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd0ec1a6b40 con 0x7fd0ec103720 2024-03-28T18:23:54.677 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:23:54.676+0000 7fd0f739a740 1 Processor -- start 2024-03-28T18:23:54.677 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:23:54.677+0000 7fd0f739a740 1 -- start start 2024-03-28T18:23:54.677 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:23:54.677+0000 7fd0ea000640 1 -- 172.21.15.22:0/3085079497 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 26) v1 ==== 137421+0+0 (secure 0 0 0) 0x7fd0dc046020 con 0x7fd0ec103720 2024-03-28T18:23:54.677 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:23:54.677+0000 7fd0f739a740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558da20a2dd0 0x558da20a1b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:23:54.677 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:23:54.677+0000 7fd0f739a740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558da20a20f0 0x558da20a2570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:23:54.677 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:23:54.677+0000 7fd0f739a740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x558da20ab460 0x558da20ad850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:23:54.677 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:23:54.677+0000 7fd0f739a740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x558da2006250 con 0x558da20a20f0 2024-03-28T18:23:54.677 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:23:54.677+0000 7fd0f739a740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x558da1faea30 con 0x558da20ab460 2024-03-28T18:23:54.677 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:23:54.677+0000 7fd0f739a740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x558da20022f0 con 0x558da20a2dd0 2024-03-28T18:23:54.679 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:23:54.677+0000 7fd0f1c00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558da20a20f0 0x558da20a2570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:23:54.679 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:23:54.678+0000 7fd0f1c00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558da20a20f0 0x558da20a2570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:54280/0 (socket says 172.21.15.22:54280) 2024-03-28T18:23:54.679 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:23:54.678+0000 7fd0f1c00640 1 -- 172.21.15.22:0/2352740961 learned_addr learned my addr 172.21.15.22:0/2352740961 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-28T18:23:54.679 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:23:54.678+0000 7fd0f1200640 1 --2- 172.21.15.22:0/2352740961 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x558da20ab460 0x558da20ad850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:23:54.679 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:23:54.678+0000 7fd0f2600640 1 --2- 172.21.15.22:0/2352740961 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558da20a2dd0 0x558da20a1b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:23:54.679 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:23:54.678+0000 7fd0f1c00640 1 -- 172.21.15.22:0/2352740961 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558da20a2dd0 msgr2=0x558da20a1b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T18:23:54.679 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:23:54.678+0000 7fd0f1c00640 1 --2- 172.21.15.22:0/2352740961 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558da20a2dd0 0x558da20a1b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:23:54.679 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:23:54.678+0000 7fd0f1c00640 1 -- 172.21.15.22:0/2352740961 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x558da20ab460 msgr2=0x558da20ad850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T18:23:54.679 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:23:54.678+0000 7fd0f1c00640 1 --2- 172.21.15.22:0/2352740961 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x558da20ab460 0x558da20ad850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:23:54.679 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:23:54.678+0000 7fd0f1c00640 1 -- 172.21.15.22:0/2352740961 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558da1f99690 con 0x558da20a20f0 2024-03-28T18:23:54.679 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:23:54.678+0000 7fd0f1c00640 1 --2- 172.21.15.22:0/2352740961 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558da20a20f0 0x558da20a2570 secure :-1 s=READY pgs=854 cs=0 l=1 rev1=1 crypto rx=0x7fd0dc0edfb0 tx=0x7fd0dc0f90e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=15fb80508d5ca0f8 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:23:54.679 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:23:54.678+0000 7fd0ea000640 1 --2- 172.21.15.22:0/3085079497 >> [v2:172.21.15.22:6800/1357365082,v1:172.21.15.22:6801/1357365082] conn(0x7fd0d00a29f0 0x7fd0d00a4eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:23:54.680 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:23:54.679+0000 7fd0ea000640 1 -- 172.21.15.22:0/3085079497 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(122..122 src has 1..122) v4 ==== 10389+0+0 (secure 0 0 0) 0x7fd0dc0f0fc0 con 0x7fd0ec103720 2024-03-28T18:23:54.680 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:23:54.679+0000 7fd0f2600640 1 --2- 172.21.15.22:0/3085079497 >> [v2:172.21.15.22:6800/1357365082,v1:172.21.15.22:6801/1357365082] conn(0x7fd0d00a29f0 0x7fd0d00a4eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:23:54.680 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:23:54.679+0000 7fd0e2a00640 1 -- 172.21.15.22:0/2352740961 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0dc0b9220 con 0x558da20a20f0 2024-03-28T18:23:54.680 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:23:54.679+0000 7fd0e2a00640 1 -- 172.21.15.22:0/2352740961 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7fd0dc0fe070 con 0x558da20a20f0 2024-03-28T18:23:54.680 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:23:54.679+0000 7fd0e2a00640 1 -- 172.21.15.22:0/2352740961 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0dc0bc8e0 con 0x558da20a20f0 2024-03-28T18:23:54.683 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:23:54.680+0000 7fd0f739a740 1 -- 172.21.15.22:0/2352740961 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558da20a20f0 msgr2=0x558da20a2570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:23:54.683 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:23:54.680+0000 7fd0f739a740 1 --2- 172.21.15.22:0/2352740961 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558da20a20f0 0x558da20a2570 secure :-1 s=READY pgs=854 cs=0 l=1 rev1=1 crypto rx=0x7fd0dc0edfb0 tx=0x7fd0dc0f90e0 comp rx=0 tx=0).stop 2024-03-28T18:23:54.684 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:23:54.680+0000 7fd0f2600640 1 --2- 172.21.15.22:0/3085079497 >> [v2:172.21.15.22:6800/1357365082,v1:172.21.15.22:6801/1357365082] conn(0x7fd0d00a29f0 0x7fd0d00a4eb0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fd0e4005fd0 tx=0x7fd0e4009500 comp rx=0 tx=0).ready entity=mgr.15522 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:23:54.684 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:23:54.680+0000 7fd0f739a740 1 -- 172.21.15.22:0/2352740961 shutdown_connections 2024-03-28T18:23:54.684 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:23:54.680+0000 7fd0f739a740 1 --2- 172.21.15.22:0/2352740961 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x558da20ab460 0x558da20ad850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:23:54.684 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:23:54.680+0000 7fd0f739a740 1 --2- 172.21.15.22:0/2352740961 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558da20a20f0 0x558da20a2570 unknown :-1 s=CLOSED pgs=854 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:23:54.684 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:23:54.680+0000 7fd0f739a740 1 --2- 172.21.15.22:0/2352740961 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558da20a2dd0 0x558da20a1b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:23:54.684 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:23:54.680+0000 7fd0f739a740 1 -- 172.21.15.22:0/2352740961 >> 172.21.15.22:0/2352740961 conn(0x558da2069200 msgr2=0x558da209cb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:23:54.684 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:23:54.680+0000 7fd0f739a740 1 -- 172.21.15.22:0/2352740961 shutdown_connections 2024-03-28T18:23:54.684 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:23:54.680+0000 7fd0f739a740 1 -- 172.21.15.22:0/2352740961 wait complete. 2024-03-28T18:23:54.684 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:23:54.680+0000 7fd0f739a740 1 -- start start 2024-03-28T18:23:54.684 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:23:54.681+0000 7fd0f739a740 1 Processor -- start 2024-03-28T18:23:54.684 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:23:54.681+0000 7fd0f739a740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x558da20a20f0 0x558da20aaad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:23:54.684 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:23:54.681+0000 7fd0f739a740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558da20a2dd0 0x558da20a7e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:23:54.684 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:23:54.681+0000 7fd0f739a740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558da20ab460 0x558da20a83a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:23:54.684 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:23:54.681+0000 7fd0f739a740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x558da2006250 con 0x558da20a2dd0 2024-03-28T18:23:54.685 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:23:54.681+0000 7fd0f739a740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x558da1fc1e80 con 0x558da20a20f0 2024-03-28T18:23:54.685 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:23:54.681+0000 7fd0f739a740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x558da20ae090 con 0x558da20ab460 2024-03-28T18:23:54.685 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:23:54.681+0000 7fd0f739a740 10 client.0 _ll_register_callbacks cb 0x558da1fd4130 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-28T18:23:54.685 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:23:54.681+0000 7fd0f1200640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558da20a2dd0 0x558da20a7e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:23:54.685 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:23:54.681+0000 7fd0f1c00640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x558da20a20f0 0x558da20aaad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:23:54.685 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:23:54.681+0000 7fd0f1200640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558da20a2dd0 0x558da20a7e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:54284/0 (socket says 172.21.15.22:54284) 2024-03-28T18:23:54.685 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:23:54.681+0000 7fd0f1200640 1 -- 172.21.15.22:0/691068588 learned_addr learned my addr 172.21.15.22:0/691068588 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-28T18:23:54.685 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:23:54.682+0000 7fd0f2600640 1 --2- 172.21.15.22:0/691068588 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558da20ab460 0x558da20a83a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:23:54.685 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:23:54.682+0000 7fd0f1200640 1 -- 172.21.15.22:0/691068588 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558da20ab460 msgr2=0x558da20a83a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T18:23:54.685 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:23:54.682+0000 7fd0f1200640 1 --2- 172.21.15.22:0/691068588 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558da20ab460 0x558da20a83a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:23:54.685 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:23:54.682+0000 7fd0f1200640 1 -- 172.21.15.22:0/691068588 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x558da20a20f0 msgr2=0x558da20aaad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T18:23:54.685 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:23:54.682+0000 7fd0f1200640 1 --2- 172.21.15.22:0/691068588 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x558da20a20f0 0x558da20aaad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:23:54.685 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:23:54.682+0000 7fd0f1200640 1 -- 172.21.15.22:0/691068588 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x558da1de47d0 con 0x558da20a2dd0 2024-03-28T18:23:54.685 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:23:54.682+0000 7fd0f1200640 1 --2- 172.21.15.22:0/691068588 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558da20a2dd0 0x558da20a7e60 secure :-1 s=READY pgs=855 cs=0 l=1 rev1=1 crypto rx=0x7fd0d8010a50 tx=0x7fd0d8010f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a0606e683461d3c2 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:23:54.685 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:23:54.682+0000 7fd0d7400640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-03-28T18:23:54.686 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:23:54.682+0000 7fd0d7400640 1 -- 172.21.15.22:0/691068588 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0d801e060 con 0x558da20a2dd0 2024-03-28T18:23:54.686 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:23:54.682+0000 7fd0d7400640 1 -- 172.21.15.22:0/691068588 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(122..122 src has 1..122) v4 ==== 10389+0+0 (secure 0 0 0) 0x7fd0d800fbd0 con 0x558da20a2dd0 2024-03-28T18:23:54.686 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:23:54.682+0000 7fd0f739a740 1 -- 172.21.15.22:0/691068588 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fd0dc0fe070 con 0x558da20a2dd0 2024-03-28T18:23:54.686 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:23:54.683+0000 7fd0d7400640 1 -- 172.21.15.22:0/691068588 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=189) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd0d8025050 con 0x558da20a2dd0 2024-03-28T18:23:54.686 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:23:54.683+0000 7fd0f739a740 10 client.17049 fetch_fsmap learned FSMap version 189 2024-03-28T18:23:54.686 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:23:54.683+0000 7fd0f739a740 1 -- 172.21.15.22:0/691068588 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=189,monmap=4+}) v3 -- 0x558da1d1a3f0 con 0x558da20a2dd0 2024-03-28T18:23:54.686 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:23:54.683+0000 7fd0d7400640 1 -- 172.21.15.22:0/691068588 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7fd0dc0fe070 con 0x558da20a2dd0 2024-03-28T18:23:54.686 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:23:54.683+0000 7fd0d7400640 1 -- 172.21.15.22:0/691068588 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 189) v1 ==== 6221+0+0 (secure 0 0 0) 0x7fd0d801acc0 con 0x558da20a2dd0 2024-03-28T18:23:54.686 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:23:54.683+0000 7fd0f739a740 10 client.17049 fetch_fsmap finished waiting for FSMap version 189 2024-03-28T18:23:54.686 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:23:54.683+0000 7fd0f739a740 10 client.17049 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-28T18:23:54.686 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:23:54.683+0000 7fd0f739a740 20 client.17049 populate_metadata read hostname 'smithi022' 2024-03-28T18:23:54.686 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:23:54.683+0000 7fd0f739a740 1 --2- 172.21.15.22:0/691068588 >> [v2:172.21.15.79:6838/2441542443,v1:172.21.15.79:6839/2441542443] conn(0x558da20bb090 0x558da20bd500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:23:54.686 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:23:54.683+0000 7fd0f739a740 4 client.17049 mds_command: new command op to 34553 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-28T18:23:54.686 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:23:54.683+0000 7fd0f739a740 1 -- 172.21.15.22:0/691068588 --> [v2:172.21.15.79:6838/2441542443,v1:172.21.15.79:6839/2441542443] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x558da2013790 con 0x558da20bb090 2024-03-28T18:23:54.687 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:23:54.684+0000 7fd0f1c00640 1 --2- 172.21.15.22:0/691068588 >> [v2:172.21.15.79:6838/2441542443,v1:172.21.15.79:6839/2441542443] conn(0x558da20bb090 0x558da20bd500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:23:54.688 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:23:54.685+0000 7fd0d7400640 10 client.17049 ms_handle_connect on v2:172.21.15.79:6838/2441542443 2024-03-28T18:23:54.688 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:23:54.685+0000 7fd0f1c00640 1 --2- 172.21.15.22:0/691068588 >> [v2:172.21.15.79:6838/2441542443,v1:172.21.15.79:6839/2441542443] conn(0x558da20bb090 0x558da20bd500 crc :-1 s=READY pgs=341 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7a776ab30220369b server_cookie=ddf0d75265d12b3 in_seq=0 out_seq=0 2024-03-28T18:23:54.688 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:23:54.686+0000 7fd0d7400640 1 -- 172.21.15.22:0/691068588 <== mds.0 v2:172.21.15.79:6838/2441542443 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x558da2013790 con 0x558da20bb090 2024-03-28T18:23:54.688 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:23:54.686+0000 7fd0d7400640 10 client.17049 handle_command_reply: tid=0 2024-03-28T18:23:54.688 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:23:54.686+0000 7fd0f739a740 1 client.17049 shutdown 2024-03-28T18:23:54.688 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:23:54.687+0000 7fd0f739a740 1 -- 172.21.15.22:0/691068588 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558da20a2dd0 msgr2=0x558da20a7e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:23:54.688 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:23:54.687+0000 7fd0f739a740 1 --2- 172.21.15.22:0/691068588 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558da20a2dd0 0x558da20a7e60 secure :-1 s=READY pgs=855 cs=0 l=1 rev1=1 crypto rx=0x7fd0d8010a50 tx=0x7fd0d8010f10 comp rx=0 tx=0).stop 2024-03-28T18:23:54.688 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:23:54.687+0000 7fd0f739a740 1 -- 172.21.15.22:0/691068588 shutdown_connections 2024-03-28T18:23:54.688 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:23:54.687+0000 7fd0f739a740 1 --2- 172.21.15.22:0/691068588 >> [v2:172.21.15.79:6838/2441542443,v1:172.21.15.79:6839/2441542443] conn(0x558da20bb090 0x558da20bd500 crc :-1 s=READY pgs=341 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:23:54.688 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:23:54.687+0000 7fd0f739a740 1 --2- 172.21.15.22:0/691068588 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558da20ab460 0x558da20a83a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:23:54.689 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:23:54.687+0000 7fd0f739a740 1 --2- 172.21.15.22:0/691068588 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558da20a2dd0 0x558da20a7e60 unknown :-1 s=CLOSED pgs=855 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:23:54.689 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:23:54.687+0000 7fd0f739a740 1 --2- 172.21.15.22:0/691068588 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x558da20a20f0 0x558da20aaad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:23:54.689 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:23:54.687+0000 7fd0f739a740 1 -- 172.21.15.22:0/691068588 >> 172.21.15.22:0/691068588 conn(0x558da2069200 msgr2=0x558da209e890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:23:54.689 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:23:54.687+0000 7fd0d7400640 0 client.17049 ms_handle_reset on v2:172.21.15.79:6838/2441542443 2024-03-28T18:23:54.689 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:23:54.687+0000 7fd0f739a740 1 -- 172.21.15.22:0/691068588 shutdown_connections 2024-03-28T18:23:54.689 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:23:54.687+0000 7fd0f739a740 1 -- 172.21.15.22:0/691068588 wait complete. 2024-03-28T18:23:54.689 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:23:54.687+0000 7fd0f739a740 20 client.17049 trim_cache size 0 max 16384 2024-03-28T18:23:54.705 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:23:54.705+0000 7fd0f739a740 1 Processor -- start 2024-03-28T18:23:54.705 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:23:54.705+0000 7fd0f739a740 1 -- start start 2024-03-28T18:23:54.705 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:23:54.705+0000 7fd0f739a740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x558da20a20f0 0x558da20a6e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:23:54.705 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:23:54.705+0000 7fd0f739a740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558da20a2dd0 0x558da20aa590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:23:54.705 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:23:54.705+0000 7fd0f739a740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558da20ab460 0x558da20aaad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:23:54.705 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:23:54.705+0000 7fd0f739a740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0ec1b1bd0 con 0x558da20ab460 2024-03-28T18:23:54.705 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:23:54.705+0000 7fd0f739a740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0ec1aef30 con 0x558da20a20f0 2024-03-28T18:23:54.705 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:23:54.705+0000 7fd0f739a740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0ec1b0f10 con 0x558da20a2dd0 2024-03-28T18:23:54.707 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:23:54.705+0000 7fd0f2600640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558da20ab460 0x558da20aaad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:23:54.707 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:23:54.705+0000 7fd0f1c00640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x558da20a20f0 0x558da20a6e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:23:54.708 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:23:54.706+0000 7fd0f2600640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558da20ab460 0x558da20aaad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:54300/0 (socket says 172.21.15.22:54300) 2024-03-28T18:23:54.708 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:23:54.706+0000 7fd0f2600640 1 -- 172.21.15.22:0/2731035458 learned_addr learned my addr 172.21.15.22:0/2731035458 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-28T18:23:54.708 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:23:54.706+0000 7fd0f1c00640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x558da20a20f0 0x558da20a6e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.22:39062/0 (socket says 172.21.15.22:39062) 2024-03-28T18:23:54.708 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:23:54.706+0000 7fd0f1200640 1 --2- 172.21.15.22:0/2731035458 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558da20a2dd0 0x558da20aa590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:23:54.708 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:23:54.706+0000 7fd0f2600640 1 -- 172.21.15.22:0/2731035458 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558da20a2dd0 msgr2=0x558da20aa590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T18:23:54.708 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:23:54.706+0000 7fd0f2600640 1 --2- 172.21.15.22:0/2731035458 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558da20a2dd0 0x558da20aa590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:23:54.708 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:23:54.706+0000 7fd0f2600640 1 -- 172.21.15.22:0/2731035458 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x558da20a20f0 msgr2=0x558da20a6e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T18:23:54.708 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:23:54.706+0000 7fd0f2600640 1 --2- 172.21.15.22:0/2731035458 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x558da20a20f0 0x558da20a6e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:23:54.708 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:23:54.706+0000 7fd0f2600640 1 -- 172.21.15.22:0/2731035458 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558da1cda0f0 con 0x558da20ab460 2024-03-28T18:23:54.708 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:23:54.706+0000 7fd0f2600640 1 --2- 172.21.15.22:0/2731035458 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558da20ab460 0x558da20aaad0 secure :-1 s=READY pgs=856 cs=0 l=1 rev1=1 crypto rx=0x7fd0e4009b50 tx=0x7fd0e4015000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8fd0c005e49cf2cc server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:23:54.708 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:23:54.706+0000 7fd0e2000640 1 -- 172.21.15.22:0/2731035458 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0e4004b20 con 0x558da20ab460 2024-03-28T18:23:54.708 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:23:54.707+0000 7fd0e2000640 1 -- 172.21.15.22:0/2731035458 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7fd0e4004cc0 con 0x558da20ab460 2024-03-28T18:23:54.708 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:23:54.707+0000 7fd0e2000640 1 -- 172.21.15.22:0/2731035458 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0e401b690 con 0x558da20ab460 2024-03-28T18:23:54.708 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:23:54.707+0000 7fd0f739a740 1 -- 172.21.15.22:0/2731035458 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558da20ab460 msgr2=0x558da20aaad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:23:54.709 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:23:54.707+0000 7fd0f739a740 1 --2- 172.21.15.22:0/2731035458 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558da20ab460 0x558da20aaad0 secure :-1 s=READY pgs=856 cs=0 l=1 rev1=1 crypto rx=0x7fd0e4009b50 tx=0x7fd0e4015000 comp rx=0 tx=0).stop 2024-03-28T18:23:54.709 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:23:54.707+0000 7fd0f739a740 1 -- 172.21.15.22:0/2731035458 shutdown_connections 2024-03-28T18:23:54.709 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:23:54.707+0000 7fd0f739a740 1 --2- 172.21.15.22:0/2731035458 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558da20ab460 0x558da20aaad0 unknown :-1 s=CLOSED pgs=856 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:23:54.709 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:23:54.707+0000 7fd0f739a740 1 --2- 172.21.15.22:0/2731035458 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558da20a2dd0 0x558da20aa590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:23:54.709 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:23:54.707+0000 7fd0f739a740 1 --2- 172.21.15.22:0/2731035458 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x558da20a20f0 0x558da20a6e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:23:54.709 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:23:54.707+0000 7fd0f739a740 1 -- 172.21.15.22:0/2731035458 >> 172.21.15.22:0/2731035458 conn(0x558da2069200 msgr2=0x558da20a6750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:23:54.709 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:23:54.707+0000 7fd0f739a740 1 -- 172.21.15.22:0/2731035458 shutdown_connections 2024-03-28T18:23:54.709 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:23:54.707+0000 7fd0f739a740 1 -- 172.21.15.22:0/2731035458 wait complete. 2024-03-28T18:23:54.709 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:23:54.708+0000 7fd0f739a740 1 -- start start 2024-03-28T18:23:54.709 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:23:54.708+0000 7fd0f739a740 1 Processor -- start 2024-03-28T18:23:54.713 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:23:54.708+0000 7fd0f739a740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558da20a20f0 0x558da20a4690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:23:54.713 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:23:54.708+0000 7fd0f739a740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x558da20a2dd0 0x558da20a4bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:23:54.713 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:23:54.708+0000 7fd0f739a740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558da20ab460 0x558da20a5110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:23:54.713 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:23:54.708+0000 7fd0f739a740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0ec1b08b0 con 0x558da20ab460 2024-03-28T18:23:54.713 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:23:54.708+0000 7fd0f739a740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0ec1af590 con 0x558da20a2dd0 2024-03-28T18:23:54.713 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:23:54.708+0000 7fd0f739a740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0ec1afbf0 con 0x558da20a20f0 2024-03-28T18:23:54.713 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:23:54.708+0000 7fd0f739a740 10 client.0 _ll_register_callbacks cb 0x7fd0d800a820 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-28T18:23:54.713 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:23:54.709+0000 7fd0f2600640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558da20ab460 0x558da20a5110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:23:54.714 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:23:54.709+0000 7fd0f2600640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558da20ab460 0x558da20a5110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:54308/0 (socket says 172.21.15.22:54308) 2024-03-28T18:23:54.714 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:23:54.709+0000 7fd0f2600640 1 -- 172.21.15.22:0/4034934695 learned_addr learned my addr 172.21.15.22:0/4034934695 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-28T18:23:54.714 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:23:54.709+0000 7fd0f1200640 1 --2- 172.21.15.22:0/4034934695 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x558da20a2dd0 0x558da20a4bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:23:54.714 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:23:54.709+0000 7fd0f1c00640 1 --2- 172.21.15.22:0/4034934695 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558da20a20f0 0x558da20a4690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:23:54.714 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:23:54.709+0000 7fd0f2600640 1 -- 172.21.15.22:0/4034934695 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558da20a20f0 msgr2=0x558da20a4690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T18:23:54.714 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:23:54.709+0000 7fd0f2600640 1 --2- 172.21.15.22:0/4034934695 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558da20a20f0 0x558da20a4690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:23:54.714 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:23:54.709+0000 7fd0f2600640 1 -- 172.21.15.22:0/4034934695 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x558da20a2dd0 msgr2=0x558da20a4bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T18:23:54.714 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:23:54.709+0000 7fd0f2600640 1 --2- 172.21.15.22:0/4034934695 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x558da20a2dd0 0x558da20a4bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:23:54.714 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:23:54.709+0000 7fd0f2600640 1 -- 172.21.15.22:0/4034934695 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x558da2115a50 con 0x558da20ab460 2024-03-28T18:23:54.714 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:23:54.709+0000 7fd0f1200640 1 --2- 172.21.15.22:0/4034934695 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x558da20a2dd0 0x558da20a4bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-28T18:23:54.714 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:23:54.709+0000 7fd0f2600640 1 --2- 172.21.15.22:0/4034934695 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558da20ab460 0x558da20a5110 secure :-1 s=READY pgs=857 cs=0 l=1 rev1=1 crypto rx=0x7fd0e4005820 tx=0x7fd0e4005850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=48547c0523e3b93c server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:23:54.714 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:23:54.709+0000 7fd0d7400640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-03-28T18:23:54.714 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:23:54.710+0000 7fd0d7400640 1 -- 172.21.15.22:0/4034934695 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0e4033060 con 0x558da20ab460 2024-03-28T18:23:54.714 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:23:54.710+0000 7fd0d7400640 1 -- 172.21.15.22:0/4034934695 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(122..122 src has 1..122) v4 ==== 10389+0+0 (secure 0 0 0) 0x7fd0e4004a40 con 0x558da20ab460 2024-03-28T18:23:54.715 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:23:54.710+0000 7fd0f739a740 1 -- 172.21.15.22:0/4034934695 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fd0e4004cc0 con 0x558da20ab460 2024-03-28T18:23:54.715 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:23:54.710+0000 7fd0d7400640 1 -- 172.21.15.22:0/4034934695 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=189) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd0e402a040 con 0x558da20ab460 2024-03-28T18:23:54.715 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:23:54.710+0000 7fd0f739a740 10 client.17055 fetch_fsmap learned FSMap version 189 2024-03-28T18:23:54.715 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:23:54.710+0000 7fd0f739a740 1 -- 172.21.15.22:0/4034934695 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=189,monmap=4+}) v3 -- 0x558da2116d20 con 0x558da20ab460 2024-03-28T18:23:54.715 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:23:54.710+0000 7fd0d7400640 1 -- 172.21.15.22:0/4034934695 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7fd0e4004cc0 con 0x558da20ab460 2024-03-28T18:23:54.715 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:23:54.710+0000 7fd0d7400640 1 -- 172.21.15.22:0/4034934695 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 189) v1 ==== 6221+0+0 (secure 0 0 0) 0x7fd0e402d030 con 0x558da20ab460 2024-03-28T18:23:54.715 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:23:54.710+0000 7fd0f739a740 10 client.17055 fetch_fsmap finished waiting for FSMap version 189 2024-03-28T18:23:54.715 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:23:54.710+0000 7fd0f739a740 10 client.17055 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-28T18:23:54.715 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:23:54.710+0000 7fd0f739a740 20 client.17055 populate_metadata read hostname 'smithi022' 2024-03-28T18:23:54.715 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:23:54.710+0000 7fd0f739a740 1 --2- 172.21.15.22:0/4034934695 >> [v2:172.21.15.79:6838/2441542443,v1:172.21.15.79:6839/2441542443] conn(0x558da21173c0 0x558da21197e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:23:54.715 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:23:54.710+0000 7fd0f739a740 4 client.17055 mds_command: new command op to 34553 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-28T18:23:54.715 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:23:54.710+0000 7fd0f739a740 1 -- 172.21.15.22:0/4034934695 --> [v2:172.21.15.79:6838/2441542443,v1:172.21.15.79:6839/2441542443] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x558da200dae0 con 0x558da21173c0 2024-03-28T18:23:54.715 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:23:54.711+0000 7fd0f1c00640 1 --2- 172.21.15.22:0/4034934695 >> [v2:172.21.15.79:6838/2441542443,v1:172.21.15.79:6839/2441542443] conn(0x558da21173c0 0x558da21197e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:23:54.715 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:23:54.712+0000 7fd0f1c00640 1 --2- 172.21.15.22:0/4034934695 >> [v2:172.21.15.79:6838/2441542443,v1:172.21.15.79:6839/2441542443] conn(0x558da21173c0 0x558da21197e0 crc :-1 s=READY pgs=342 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=60d3e0e8695766a0 server_cookie=d8970c4b66f79efb in_seq=0 out_seq=0 2024-03-28T18:23:54.715 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:23:54.712+0000 7fd0d7400640 10 client.17055 ms_handle_connect on v2:172.21.15.79:6838/2441542443 2024-03-28T18:23:54.715 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:23:54.712+0000 7fd0d7400640 1 -- 172.21.15.22:0/4034934695 <== mds.0 v2:172.21.15.79:6838/2441542443 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x558da200dae0 con 0x558da21173c0 2024-03-28T18:23:54.716 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:23:54.712+0000 7fd0d7400640 10 client.17055 handle_command_reply: tid=0 2024-03-28T18:23:54.716 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:23:54.712+0000 7fd0f739a740 1 client.17055 shutdown 2024-03-28T18:23:54.716 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:23:54.712+0000 7fd0f739a740 1 -- 172.21.15.22:0/4034934695 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558da20ab460 msgr2=0x558da20a5110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:23:54.716 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:23:54.712+0000 7fd0f739a740 1 --2- 172.21.15.22:0/4034934695 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558da20ab460 0x558da20a5110 secure :-1 s=READY pgs=857 cs=0 l=1 rev1=1 crypto rx=0x7fd0e4005820 tx=0x7fd0e4005850 comp rx=0 tx=0).stop 2024-03-28T18:23:54.716 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:23:54.712+0000 7fd0f739a740 1 -- 172.21.15.22:0/4034934695 shutdown_connections 2024-03-28T18:23:54.716 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:23:54.712+0000 7fd0f739a740 1 --2- 172.21.15.22:0/4034934695 >> [v2:172.21.15.79:6838/2441542443,v1:172.21.15.79:6839/2441542443] conn(0x558da21173c0 0x558da21197e0 crc :-1 s=READY pgs=342 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:23:54.716 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:23:54.712+0000 7fd0f739a740 1 --2- 172.21.15.22:0/4034934695 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558da20ab460 0x558da20a5110 unknown :-1 s=CLOSED pgs=857 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:23:54.716 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:23:54.712+0000 7fd0f739a740 1 --2- 172.21.15.22:0/4034934695 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x558da20a2dd0 0x558da20a4bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:23:54.716 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:23:54.712+0000 7fd0f739a740 1 --2- 172.21.15.22:0/4034934695 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558da20a20f0 0x558da20a4690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:23:54.716 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:23:54.712+0000 7fd0f739a740 1 -- 172.21.15.22:0/4034934695 >> 172.21.15.22:0/4034934695 conn(0x558da2069200 msgr2=0x558da209b330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:23:54.716 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:23:54.712+0000 7fd0d7400640 0 client.17055 ms_handle_reset on v2:172.21.15.79:6838/2441542443 2024-03-28T18:23:54.716 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:23:54.712+0000 7fd0f739a740 1 -- 172.21.15.22:0/4034934695 shutdown_connections 2024-03-28T18:23:54.716 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:23:54.712+0000 7fd0f739a740 1 -- 172.21.15.22:0/4034934695 wait complete. 2024-03-28T18:23:54.716 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:23:54.713+0000 7fd0f739a740 20 client.17055 trim_cache size 0 max 16384 2024-03-28T18:23:54.717 INFO:teuthology.orchestra.run.smithi022.stdout:{ 2024-03-28T18:23:54.717 INFO:teuthology.orchestra.run.smithi022.stdout: "status": "IDLE (waiting for more scrubs)", 2024-03-28T18:23:54.717 INFO:teuthology.orchestra.run.smithi022.stdout: "scrubs": { 2024-03-28T18:23:54.717 INFO:teuthology.orchestra.run.smithi022.stdout: "4bbbd61d-29a3-4bf6-a4f2-d5783d585de3": { 2024-03-28T18:23:54.717 INFO:teuthology.orchestra.run.smithi022.stdout: "path": "/", 2024-03-28T18:23:54.717 INFO:teuthology.orchestra.run.smithi022.stdout: "tag": "4bbbd61d-29a3-4bf6-a4f2-d5783d585de3", 2024-03-28T18:23:54.717 INFO:teuthology.orchestra.run.smithi022.stdout: "options": "recursive,force" 2024-03-28T18:23:54.717 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-03-28T18:23:54.717 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-03-28T18:23:54.717 INFO:teuthology.orchestra.run.smithi022.stdout:} 2024-03-28T18:23:54.718 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:23:54.713+0000 7fd0f3000640 1 -- 172.21.15.22:0/3085079497 >> [v2:172.21.15.22:6800/1357365082,v1:172.21.15.22:6801/1357365082] conn(0x7fd0d00a29f0 msgr2=0x7fd0d00a4eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:23:54.718 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:23:54.713+0000 7fd0f3000640 1 --2- 172.21.15.22:0/3085079497 >> [v2:172.21.15.22:6800/1357365082,v1:172.21.15.22:6801/1357365082] conn(0x7fd0d00a29f0 0x7fd0d00a4eb0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fd0e4005fd0 tx=0x7fd0e4009500 comp rx=0 tx=0).stop 2024-03-28T18:23:54.718 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:23:54.713+0000 7fd0f3000640 1 -- 172.21.15.22:0/3085079497 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd0ec103720 msgr2=0x7fd0ec199fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:23:54.718 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:23:54.713+0000 7fd0f3000640 1 --2- 172.21.15.22:0/3085079497 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd0ec103720 0x7fd0ec199fd0 secure :-1 s=READY pgs=853 cs=0 l=1 rev1=1 crypto rx=0x7fd0dc009900 tx=0x7fd0dc0045e0 comp rx=0 tx=0).stop 2024-03-28T18:23:54.718 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:23:54.713+0000 7fd0f3000640 1 -- 172.21.15.22:0/3085079497 shutdown_connections 2024-03-28T18:23:54.718 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:23:54.713+0000 7fd0f3000640 1 --2- 172.21.15.22:0/3085079497 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd0ec109730 0x7fd0ec19e890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:23:54.718 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:23:54.713+0000 7fd0f3000640 1 --2- 172.21.15.22:0/3085079497 >> [v2:172.21.15.22:6800/1357365082,v1:172.21.15.22:6801/1357365082] conn(0x7fd0d00a29f0 0x7fd0d00a4eb0 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:23:54.719 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:23:54.713+0000 7fd0f3000640 1 --2- 172.21.15.22:0/3085079497 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd0ec1040e0 0x7fd0ec19a510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:23:54.719 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:23:54.713+0000 7fd0f3000640 1 --2- 172.21.15.22:0/3085079497 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd0ec103720 0x7fd0ec199fd0 unknown :-1 s=CLOSED pgs=853 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:23:54.719 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:23:54.713+0000 7fd0f3000640 1 -- 172.21.15.22:0/3085079497 >> 172.21.15.22:0/3085079497 conn(0x7fd0ec0ff1d0 msgr2=0x7fd0ec0ff880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:23:54.719 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:23:54.713+0000 7fd0f3000640 1 -- 172.21.15.22:0/3085079497 shutdown_connections 2024-03-28T18:23:54.719 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:23:54.713+0000 7fd0f3000640 1 -- 172.21.15.22:0/3085079497 wait complete. 2024-03-28T18:23:54.731 INFO:tasks.cephfs.filesystem:scrub status for tag:4bbbd61d-29a3-4bf6-a4f2-d5783d585de3 - {'path': '/', 'tag': '4bbbd61d-29a3-4bf6-a4f2-d5783d585de3', 'options': 'recursive,force'} 2024-03-28T18:23:55.175 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:23:54 smithi134 ceph-mon[35037]: 2 slow requests, 0 included below; oldest blocked for > 421.845487 secs 2024-03-28T18:23:55.182 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:23:54 smithi079 ceph-mon[35312]: 2 slow requests, 0 included below; oldest blocked for > 421.845487 secs 2024-03-28T18:23:55.282 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:23:54 smithi022 ceph-mon[31985]: 2 slow requests, 0 included below; oldest blocked for > 421.845487 secs 2024-03-28T18:23:56.183 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:23:55 smithi079 ceph-mon[35312]: pgmap v529: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:23:56.282 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:23:55 smithi022 ceph-mon[31985]: pgmap v529: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:23:56.424 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:23:55 smithi134 ceph-mon[35037]: pgmap v529: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:23:58.183 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:23:57 smithi079 ceph-mon[35312]: pgmap v530: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:23:58.282 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:23:57 smithi022 ceph-mon[31985]: pgmap v530: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:23:58.425 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:23:57 smithi134 ceph-mon[35037]: pgmap v530: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:24:00.282 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:23:59 smithi022 ceph-mon[31985]: pgmap v531: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:24:00.282 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:23:59 smithi022 ceph-mon[31985]: 2 slow requests, 0 included below; oldest blocked for > 426.845604 secs 2024-03-28T18:24:00.425 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:23:59 smithi134 ceph-mon[35037]: pgmap v531: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:24:00.425 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:23:59 smithi134 ceph-mon[35037]: 2 slow requests, 0 included below; oldest blocked for > 426.845604 secs 2024-03-28T18:24:00.432 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:23:59 smithi079 ceph-mon[35312]: pgmap v531: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:24:00.433 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:23:59 smithi079 ceph-mon[35312]: 2 slow requests, 0 included below; oldest blocked for > 426.845604 secs 2024-03-28T18:24:02.282 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:24:01 smithi022 ceph-mon[31985]: pgmap v532: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:24:02.425 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:24:01 smithi134 ceph-mon[35037]: pgmap v532: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:24:02.432 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:24:01 smithi079 ceph-mon[35312]: pgmap v532: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:24:04.227 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:24:03 smithi134 ceph-mon[35037]: pgmap v533: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:24:04.282 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:24:03 smithi022 ceph-mon[31985]: pgmap v533: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:24:04.433 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:24:03 smithi079 ceph-mon[35312]: pgmap v533: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:24:05.282 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:24:04 smithi022 ceph-mon[31985]: 2 slow requests, 0 included below; oldest blocked for > 431.845667 secs 2024-03-28T18:24:05.424 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:24:04 smithi134 ceph-mon[35037]: 2 slow requests, 0 included below; oldest blocked for > 431.845667 secs 2024-03-28T18:24:05.432 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:24:04 smithi079 ceph-mon[35312]: 2 slow requests, 0 included below; oldest blocked for > 431.845667 secs 2024-03-28T18:24:06.282 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:24:05 smithi022 ceph-mon[31985]: pgmap v534: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:24:06.424 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:24:05 smithi134 ceph-mon[35037]: pgmap v534: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:24:06.432 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:24:05 smithi079 ceph-mon[35312]: pgmap v534: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:24:08.282 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:24:07 smithi022 ceph-mon[31985]: pgmap v535: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:24:08.424 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:24:07 smithi134 ceph-mon[35037]: pgmap v535: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:24:08.432 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:24:07 smithi079 ceph-mon[35312]: pgmap v535: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:24:10.283 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:24:09 smithi022 ceph-mon[31985]: pgmap v536: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:24:10.283 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:24:09 smithi022 ceph-mon[31985]: 2 slow requests, 0 included below; oldest blocked for > 436.845733 secs 2024-03-28T18:24:10.425 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:24:09 smithi134 ceph-mon[35037]: pgmap v536: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:24:10.425 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:24:09 smithi134 ceph-mon[35037]: 2 slow requests, 0 included below; oldest blocked for > 436.845733 secs 2024-03-28T18:24:10.432 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:24:09 smithi079 ceph-mon[35312]: pgmap v536: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:24:10.432 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:24:09 smithi079 ceph-mon[35312]: 2 slow requests, 0 included below; oldest blocked for > 436.845733 secs 2024-03-28T18:24:12.282 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:24:11 smithi022 ceph-mon[31985]: pgmap v537: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:24:12.424 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:24:11 smithi134 ceph-mon[35037]: pgmap v537: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:24:12.432 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:24:11 smithi079 ceph-mon[35312]: pgmap v537: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:24:14.282 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:24:13 smithi022 ceph-mon[31985]: pgmap v538: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:24:14.425 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:24:13 smithi134 ceph-mon[35037]: pgmap v538: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:24:14.432 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:24:13 smithi079 ceph-mon[35312]: pgmap v538: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:24:15.282 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:24:14 smithi022 ceph-mon[31985]: 2 slow requests, 0 included below; oldest blocked for > 441.845814 secs 2024-03-28T18:24:15.334 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:24:14 smithi079 ceph-mon[35312]: 2 slow requests, 0 included below; oldest blocked for > 441.845814 secs 2024-03-28T18:24:15.424 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:24:14 smithi134 ceph-mon[35037]: 2 slow requests, 0 included below; oldest blocked for > 441.845814 secs 2024-03-28T18:24:16.282 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:24:16 smithi022 ceph-mon[31985]: pgmap v539: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:24:16.282 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:24:16 smithi022 ceph-mon[31985]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T18:24:16.425 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:24:16 smithi134 ceph-mon[35037]: pgmap v539: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:24:16.425 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:24:16 smithi134 ceph-mon[35037]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T18:24:16.432 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:24:16 smithi079 ceph-mon[35312]: pgmap v539: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:24:16.433 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:24:16 smithi079 ceph-mon[35312]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T18:24:17.282 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:24:17 smithi022 ceph-mon[31985]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T18:24:17.282 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:24:17 smithi022 ceph-mon[31985]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T18:24:17.282 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:24:17 smithi022 ceph-mon[31985]: from='mgr.15522 ' entity='mgr.x' 2024-03-28T18:24:17.425 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:24:17 smithi134 ceph-mon[35037]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T18:24:17.425 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:24:17 smithi134 ceph-mon[35037]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T18:24:17.425 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:24:17 smithi134 ceph-mon[35037]: from='mgr.15522 ' entity='mgr.x' 2024-03-28T18:24:17.433 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:24:17 smithi079 ceph-mon[35312]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T18:24:17.433 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:24:17 smithi079 ceph-mon[35312]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T18:24:17.433 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:24:17 smithi079 ceph-mon[35312]: from='mgr.15522 ' entity='mgr.x' 2024-03-28T18:24:18.282 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:24:18 smithi022 ceph-mon[31985]: pgmap v540: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 6 op/s 2024-03-28T18:24:18.282 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:24:18 smithi022 ceph-mon[31985]: pgmap v541: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-03-28T18:24:18.424 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:24:18 smithi134 ceph-mon[35037]: pgmap v540: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 6 op/s 2024-03-28T18:24:18.425 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:24:18 smithi134 ceph-mon[35037]: pgmap v541: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-03-28T18:24:18.432 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:24:18 smithi079 ceph-mon[35312]: pgmap v540: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 6 op/s 2024-03-28T18:24:18.432 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:24:18 smithi079 ceph-mon[35312]: pgmap v541: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-03-28T18:24:20.282 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:24:20 smithi022 ceph-mon[31985]: pgmap v542: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-03-28T18:24:20.282 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:24:20 smithi022 ceph-mon[31985]: 2 slow requests, 0 included below; oldest blocked for > 446.845880 secs 2024-03-28T18:24:20.425 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:24:20 smithi134 ceph-mon[35037]: pgmap v542: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-03-28T18:24:20.425 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:24:20 smithi134 ceph-mon[35037]: 2 slow requests, 0 included below; oldest blocked for > 446.845880 secs 2024-03-28T18:24:20.432 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:24:20 smithi079 ceph-mon[35312]: pgmap v542: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-03-28T18:24:20.432 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:24:20 smithi079 ceph-mon[35312]: 2 slow requests, 0 included below; oldest blocked for > 446.845880 secs 2024-03-28T18:24:22.282 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:24:22 smithi022 ceph-mon[31985]: pgmap v543: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-03-28T18:24:22.425 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:24:22 smithi134 ceph-mon[35037]: pgmap v543: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-03-28T18:24:22.432 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:24:22 smithi079 ceph-mon[35312]: pgmap v543: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-03-28T18:24:24.282 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:24:24 smithi022 ceph-mon[31985]: pgmap v544: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-03-28T18:24:24.425 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:24:24 smithi134 ceph-mon[35037]: pgmap v544: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-03-28T18:24:24.432 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:24:24 smithi079 ceph-mon[35312]: pgmap v544: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-03-28T18:24:24.732 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-28T18:24:24.732 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-28T18:24:24.967 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:24:24.966+0000 7fb30fe00640 1 -- 172.21.15.22:0/1211586458 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb31010ddd0 msgr2=0x7fb3101101c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:24:24.967 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:24:24.966+0000 7fb30fe00640 1 --2- 172.21.15.22:0/1211586458 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb31010ddd0 0x7fb3101101c0 secure :-1 s=READY pgs=858 cs=0 l=1 rev1=1 crypto rx=0x7fb308009900 tx=0x7fb30802bcd0 comp rx=0 tx=0).stop 2024-03-28T18:24:24.967 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:24:24.966+0000 7fb30fe00640 1 -- 172.21.15.22:0/1211586458 shutdown_connections 2024-03-28T18:24:24.967 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:24:24.966+0000 7fb30fe00640 1 --2- 172.21.15.22:0/1211586458 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb31010ddd0 0x7fb3101101c0 unknown :-1 s=CLOSED pgs=858 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:24:24.968 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:24:24.966+0000 7fb30fe00640 1 --2- 172.21.15.22:0/1211586458 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb310074370 0x7fb31010d890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:24:24.968 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:24:24.966+0000 7fb30fe00640 1 --2- 172.21.15.22:0/1211586458 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fb310073a50 0x7fb310073e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:24:24.968 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:24:24.966+0000 7fb30fe00640 1 -- 172.21.15.22:0/1211586458 >> 172.21.15.22:0/1211586458 conn(0x7fb3100fd000 msgr2=0x7fb3100ff420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:24:24.968 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:24:24.966+0000 7fb30fe00640 1 -- 172.21.15.22:0/1211586458 shutdown_connections 2024-03-28T18:24:24.968 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:24:24.966+0000 7fb30fe00640 1 -- 172.21.15.22:0/1211586458 wait complete. 2024-03-28T18:24:24.970 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:24:24.967+0000 7fb30fe00640 1 Processor -- start 2024-03-28T18:24:24.970 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:24:24.967+0000 7fb30fe00640 1 -- start start 2024-03-28T18:24:24.970 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:24:24.968+0000 7fb30fe00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb310073a50 0x7fb310199fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:24:24.970 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:24:24.968+0000 7fb30fe00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb310074370 0x7fb31019a4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:24:24.970 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:24:24.968+0000 7fb30fe00640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fb31010ddd0 0x7fb31019e820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:24:24.971 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:24:24.968+0000 7fb30fe00640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb310113b10 con 0x7fb310073a50 2024-03-28T18:24:24.971 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:24:24.968+0000 7fb30fe00640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb310113990 con 0x7fb31010ddd0 2024-03-28T18:24:24.971 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:24:24.968+0000 7fb30fe00640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb310113c90 con 0x7fb310074370 2024-03-28T18:24:24.971 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:24:24.968+0000 7fb30ea00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb310073a50 0x7fb310199fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:24:24.971 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:24:24.968+0000 7fb30ea00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb310073a50 0x7fb310199fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:55332/0 (socket says 172.21.15.22:55332) 2024-03-28T18:24:24.971 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:24:24.968+0000 7fb30ea00640 1 -- 172.21.15.22:0/1353235237 learned_addr learned my addr 172.21.15.22:0/1353235237 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-28T18:24:24.971 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:24:24.968+0000 7fb30f400640 1 --2- 172.21.15.22:0/1353235237 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fb31010ddd0 0x7fb31019e820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:24:24.971 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:24:24.968+0000 7fb30e000640 1 --2- 172.21.15.22:0/1353235237 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb310074370 0x7fb31019a4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:24:24.971 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:24:24.968+0000 7fb30ea00640 1 -- 172.21.15.22:0/1353235237 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb310074370 msgr2=0x7fb31019a4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:24:24.971 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:24:24.968+0000 7fb30ea00640 1 --2- 172.21.15.22:0/1353235237 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb310074370 0x7fb31019a4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:24:24.971 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:24:24.968+0000 7fb30ea00640 1 -- 172.21.15.22:0/1353235237 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fb31010ddd0 msgr2=0x7fb31019e820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:24:24.971 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:24:24.968+0000 7fb30ea00640 1 --2- 172.21.15.22:0/1353235237 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fb31010ddd0 0x7fb31019e820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:24:24.971 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:24:24.968+0000 7fb30ea00640 1 -- 172.21.15.22:0/1353235237 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb31019ef50 con 0x7fb310073a50 2024-03-28T18:24:24.971 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:24:24.968+0000 7fb30f400640 1 --2- 172.21.15.22:0/1353235237 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fb31010ddd0 0x7fb31019e820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-28T18:24:24.971 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:24:24.969+0000 7fb30ea00640 1 --2- 172.21.15.22:0/1353235237 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb310073a50 0x7fb310199fb0 secure :-1 s=READY pgs=859 cs=0 l=1 rev1=1 crypto rx=0x7fb30000eab0 tx=0x7fb30000ef70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:24:24.972 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:24:24.969+0000 7fb307400640 1 -- 172.21.15.22:0/1353235237 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb30000ce40 con 0x7fb310073a50 2024-03-28T18:24:24.972 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:24:24.969+0000 7fb30fe00640 1 -- 172.21.15.22:0/1353235237 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb31019f240 con 0x7fb310073a50 2024-03-28T18:24:24.972 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:24:24.969+0000 7fb307400640 1 -- 172.21.15.22:0/1353235237 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7fb300011940 con 0x7fb310073a50 2024-03-28T18:24:24.972 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:24:24.969+0000 7fb307400640 1 -- 172.21.15.22:0/1353235237 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb300010a80 con 0x7fb310073a50 2024-03-28T18:24:24.972 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:24:24.969+0000 7fb30fe00640 1 -- 172.21.15.22:0/1353235237 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb3101a6ad0 con 0x7fb310073a50 2024-03-28T18:24:24.973 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:24:24.972+0000 7fb318a1b740 1 Processor -- start 2024-03-28T18:24:24.973 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:24:24.972+0000 7fb318a1b740 1 -- start start 2024-03-28T18:24:24.973 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:24:24.972+0000 7fb307400640 1 -- 172.21.15.22:0/1353235237 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 26) v1 ==== 137421+0+0 (secure 0 0 0) 0x7fb300011af0 con 0x7fb310073a50 2024-03-28T18:24:24.975 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:24:24.973+0000 7fb318a1b740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x561d403e0dd0 0x561d403dfb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:24:24.975 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:24:24.973+0000 7fb318a1b740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561d403e00f0 0x561d403e0570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:24:24.975 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:24:24.973+0000 7fb318a1b740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561d403e9460 0x561d403eb850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:24:24.975 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:24:24.973+0000 7fb318a1b740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x561d40344250 con 0x561d403e00f0 2024-03-28T18:24:24.975 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:24:24.973+0000 7fb318a1b740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x561d402eca30 con 0x561d403e0dd0 2024-03-28T18:24:24.975 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:24:24.973+0000 7fb318a1b740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x561d403402f0 con 0x561d403e9460 2024-03-28T18:24:24.975 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:24:24.973+0000 7fb30ea00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561d403e00f0 0x561d403e0570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:24:24.975 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:24:24.973+0000 7fb30ea00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561d403e00f0 0x561d403e0570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:55338/0 (socket says 172.21.15.22:55338) 2024-03-28T18:24:24.975 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:24:24.973+0000 7fb30ea00640 1 -- 172.21.15.22:0/4241382867 learned_addr learned my addr 172.21.15.22:0/4241382867 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-28T18:24:24.975 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:24:24.973+0000 7fb30f400640 1 --2- 172.21.15.22:0/4241382867 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x561d403e0dd0 0x561d403dfb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:24:24.975 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:24:24.973+0000 7fb30e000640 1 --2- 172.21.15.22:0/4241382867 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561d403e9460 0x561d403eb850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:24:24.975 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:24:24.974+0000 7fb30ea00640 1 -- 172.21.15.22:0/4241382867 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561d403e9460 msgr2=0x561d403eb850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T18:24:24.976 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:24:24.974+0000 7fb30ea00640 1 --2- 172.21.15.22:0/4241382867 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561d403e9460 0x561d403eb850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:24:24.976 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:24:24.974+0000 7fb30ea00640 1 -- 172.21.15.22:0/4241382867 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x561d403e0dd0 msgr2=0x561d403dfb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T18:24:24.976 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:24:24.974+0000 7fb30ea00640 1 --2- 172.21.15.22:0/4241382867 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x561d403e0dd0 0x561d403dfb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:24:24.976 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:24:24.974+0000 7fb30ea00640 1 -- 172.21.15.22:0/4241382867 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561d402d7690 con 0x561d403e00f0 2024-03-28T18:24:24.976 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:24:24.974+0000 7fb307400640 1 --2- 172.21.15.22:0/1353235237 >> [v2:172.21.15.22:6800/1357365082,v1:172.21.15.22:6801/1357365082] conn(0x7fb2f40a29f0 0x7fb2f40a4eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:24:24.976 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:24:24.974+0000 7fb30ea00640 1 --2- 172.21.15.22:0/4241382867 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561d403e00f0 0x561d403e0570 secure :-1 s=READY pgs=860 cs=0 l=1 rev1=1 crypto rx=0x7fb3000ccb90 tx=0x7fb3000d3ee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=48c3fed8541183ac server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:24:24.978 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:24:24.974+0000 7fb307400640 1 -- 172.21.15.22:0/1353235237 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(122..122 src has 1..122) v4 ==== 10389+0+0 (secure 0 0 0) 0x7fb3000cfc20 con 0x7fb310073a50 2024-03-28T18:24:24.978 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:24:24.975+0000 7fb30f400640 1 --2- 172.21.15.22:0/1353235237 >> [v2:172.21.15.22:6800/1357365082,v1:172.21.15.22:6801/1357365082] conn(0x7fb2f40a29f0 0x7fb2f40a4eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:24:24.978 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:24:24.974+0000 7fb2ffe00640 1 -- 172.21.15.22:0/4241382867 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb30009ce60 con 0x561d403e00f0 2024-03-28T18:24:24.978 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:24:24.975+0000 7fb2ffe00640 1 -- 172.21.15.22:0/4241382867 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7fb3000dd070 con 0x561d403e00f0 2024-03-28T18:24:24.978 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:24:24.975+0000 7fb2ffe00640 1 -- 172.21.15.22:0/4241382867 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb30009b670 con 0x561d403e00f0 2024-03-28T18:24:24.979 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:24:24.975+0000 7fb318a1b740 1 -- 172.21.15.22:0/4241382867 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561d403e00f0 msgr2=0x561d403e0570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:24:24.979 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:24:24.975+0000 7fb318a1b740 1 --2- 172.21.15.22:0/4241382867 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561d403e00f0 0x561d403e0570 secure :-1 s=READY pgs=860 cs=0 l=1 rev1=1 crypto rx=0x7fb3000ccb90 tx=0x7fb3000d3ee0 comp rx=0 tx=0).stop 2024-03-28T18:24:24.979 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:24:24.975+0000 7fb30f400640 1 --2- 172.21.15.22:0/1353235237 >> [v2:172.21.15.22:6800/1357365082,v1:172.21.15.22:6801/1357365082] conn(0x7fb2f40a29f0 0x7fb2f40a4eb0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fb3080096c0 tx=0x7fb308009560 comp rx=0 tx=0).ready entity=mgr.15522 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:24:24.979 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:24:24.975+0000 7fb318a1b740 1 -- 172.21.15.22:0/4241382867 shutdown_connections 2024-03-28T18:24:24.979 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:24:24.975+0000 7fb318a1b740 1 --2- 172.21.15.22:0/4241382867 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561d403e9460 0x561d403eb850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:24:24.979 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:24:24.975+0000 7fb318a1b740 1 --2- 172.21.15.22:0/4241382867 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561d403e00f0 0x561d403e0570 unknown :-1 s=CLOSED pgs=860 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:24:24.979 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:24:24.975+0000 7fb318a1b740 1 --2- 172.21.15.22:0/4241382867 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x561d403e0dd0 0x561d403dfb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:24:24.979 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:24:24.975+0000 7fb318a1b740 1 -- 172.21.15.22:0/4241382867 >> 172.21.15.22:0/4241382867 conn(0x561d403a7200 msgr2=0x561d403dab40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:24:24.979 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:24:24.976+0000 7fb318a1b740 1 -- 172.21.15.22:0/4241382867 shutdown_connections 2024-03-28T18:24:24.979 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:24:24.976+0000 7fb318a1b740 1 -- 172.21.15.22:0/4241382867 wait complete. 2024-03-28T18:24:24.979 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:24:24.976+0000 7fb318a1b740 1 -- start start 2024-03-28T18:24:24.979 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:24:24.976+0000 7fb318a1b740 1 Processor -- start 2024-03-28T18:24:24.979 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:24:24.977+0000 7fb318a1b740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x561d403e00f0 0x561d403e8a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:24:24.979 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:24:24.977+0000 7fb318a1b740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561d403e0dd0 0x561d403e5e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:24:24.980 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:24:24.977+0000 7fb318a1b740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561d403e9460 0x561d403e6360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:24:24.980 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:24:24.977+0000 7fb318a1b740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x561d403ec320 con 0x561d403e0dd0 2024-03-28T18:24:24.980 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:24:24.977+0000 7fb318a1b740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x561d402ffe80 con 0x561d403e00f0 2024-03-28T18:24:24.980 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:24:24.977+0000 7fb318a1b740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x561d403ec4a0 con 0x561d403e9460 2024-03-28T18:24:24.980 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:24:24.977+0000 7fb318a1b740 10 client.0 _ll_register_callbacks cb 0x561d40312130 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-28T18:24:24.980 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:24:24.977+0000 7fb30e000640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561d403e0dd0 0x561d403e5e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:24:24.980 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:24:24.977+0000 7fb30e000640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561d403e0dd0 0x561d403e5e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:55340/0 (socket says 172.21.15.22:55340) 2024-03-28T18:24:24.980 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:24:24.977+0000 7fb30e000640 1 -- 172.21.15.22:0/988637518 learned_addr learned my addr 172.21.15.22:0/988637518 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-28T18:24:24.980 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:24:24.977+0000 7fb30ea00640 1 --2- 172.21.15.22:0/988637518 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x561d403e00f0 0x561d403e8a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:24:24.980 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:24:24.977+0000 7fb30f400640 1 --2- 172.21.15.22:0/988637518 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561d403e9460 0x561d403e6360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:24:24.980 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:24:24.977+0000 7fb30e000640 1 -- 172.21.15.22:0/988637518 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561d403e9460 msgr2=0x561d403e6360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T18:24:24.980 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:24:24.977+0000 7fb30e000640 1 --2- 172.21.15.22:0/988637518 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561d403e9460 0x561d403e6360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:24:24.980 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:24:24.977+0000 7fb30e000640 1 -- 172.21.15.22:0/988637518 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x561d403e00f0 msgr2=0x561d403e8a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T18:24:24.980 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:24:24.977+0000 7fb30e000640 1 --2- 172.21.15.22:0/988637518 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x561d403e00f0 0x561d403e8a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:24:24.980 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:24:24.978+0000 7fb30e000640 1 -- 172.21.15.22:0/988637518 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x561d401227d0 con 0x561d403e0dd0 2024-03-28T18:24:24.983 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:24:24.978+0000 7fb30e000640 1 --2- 172.21.15.22:0/988637518 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561d403e0dd0 0x561d403e5e20 secure :-1 s=READY pgs=861 cs=0 l=1 rev1=1 crypto rx=0x7fb2f8010870 tx=0x7fb2f8010d30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7b24536f2f37072 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:24:24.983 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:24:24.978+0000 7fb2fcc00640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-03-28T18:24:24.983 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:24:24.978+0000 7fb2fcc00640 1 -- 172.21.15.22:0/988637518 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2f800cd20 con 0x561d403e0dd0 2024-03-28T18:24:24.983 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:24:24.978+0000 7fb2fcc00640 1 -- 172.21.15.22:0/988637518 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(122..122 src has 1..122) v4 ==== 10389+0+0 (secure 0 0 0) 0x7fb2f801c480 con 0x561d403e0dd0 2024-03-28T18:24:24.983 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:24:24.978+0000 7fb318a1b740 1 -- 172.21.15.22:0/988637518 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fb3000dd070 con 0x561d403e0dd0 2024-03-28T18:24:24.983 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:24:24.978+0000 7fb2fcc00640 1 -- 172.21.15.22:0/988637518 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=189) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb2f8025050 con 0x561d403e0dd0 2024-03-28T18:24:24.983 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:24:24.978+0000 7fb318a1b740 10 client.17067 fetch_fsmap learned FSMap version 189 2024-03-28T18:24:24.983 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:24:24.978+0000 7fb318a1b740 1 -- 172.21.15.22:0/988637518 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=189,monmap=4+}) v3 -- 0x561d400583f0 con 0x561d403e0dd0 2024-03-28T18:24:24.984 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:24:24.979+0000 7fb2fcc00640 1 -- 172.21.15.22:0/988637518 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7fb3000dd070 con 0x561d403e0dd0 2024-03-28T18:24:24.984 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:24:24.979+0000 7fb2fcc00640 1 -- 172.21.15.22:0/988637518 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 189) v1 ==== 6221+0+0 (secure 0 0 0) 0x7fb2f80117a0 con 0x561d403e0dd0 2024-03-28T18:24:24.984 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:24:24.979+0000 7fb318a1b740 10 client.17067 fetch_fsmap finished waiting for FSMap version 189 2024-03-28T18:24:24.984 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:24:24.979+0000 7fb318a1b740 10 client.17067 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-28T18:24:24.984 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:24:24.979+0000 7fb318a1b740 20 client.17067 populate_metadata read hostname 'smithi022' 2024-03-28T18:24:24.984 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:24:24.979+0000 7fb318a1b740 1 --2- 172.21.15.22:0/988637518 >> [v2:172.21.15.79:6838/2441542443,v1:172.21.15.79:6839/2441542443] conn(0x561d403f9530 0x561d403fb990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:24:24.984 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:24:24.979+0000 7fb318a1b740 4 client.17067 mds_command: new command op to 34553 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-28T18:24:24.984 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:24:24.979+0000 7fb318a1b740 1 -- 172.21.15.22:0/988637518 --> [v2:172.21.15.79:6838/2441542443,v1:172.21.15.79:6839/2441542443] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x561d40351790 con 0x561d403f9530 2024-03-28T18:24:24.984 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:24:24.980+0000 7fb30ea00640 1 --2- 172.21.15.22:0/988637518 >> [v2:172.21.15.79:6838/2441542443,v1:172.21.15.79:6839/2441542443] conn(0x561d403f9530 0x561d403fb990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:24:24.984 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:24:24.980+0000 7fb30ea00640 1 --2- 172.21.15.22:0/988637518 >> [v2:172.21.15.79:6838/2441542443,v1:172.21.15.79:6839/2441542443] conn(0x561d403f9530 0x561d403fb990 crc :-1 s=READY pgs=345 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a12f839a68168b0a server_cookie=75831be48f2dbe55 in_seq=0 out_seq=0 2024-03-28T18:24:24.984 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:24:24.980+0000 7fb2fcc00640 10 client.17067 ms_handle_connect on v2:172.21.15.79:6838/2441542443 2024-03-28T18:24:24.984 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:24:24.982+0000 7fb2fcc00640 1 -- 172.21.15.22:0/988637518 <== mds.0 v2:172.21.15.79:6838/2441542443 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x561d40351790 con 0x561d403f9530 2024-03-28T18:24:24.984 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:24:24.982+0000 7fb2fcc00640 10 client.17067 handle_command_reply: tid=0 2024-03-28T18:24:24.984 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:24:24.982+0000 7fb318a1b740 1 client.17067 shutdown 2024-03-28T18:24:24.984 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:24:24.982+0000 7fb318a1b740 1 -- 172.21.15.22:0/988637518 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561d403e0dd0 msgr2=0x561d403e5e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:24:24.984 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:24:24.982+0000 7fb318a1b740 1 --2- 172.21.15.22:0/988637518 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561d403e0dd0 0x561d403e5e20 secure :-1 s=READY pgs=861 cs=0 l=1 rev1=1 crypto rx=0x7fb2f8010870 tx=0x7fb2f8010d30 comp rx=0 tx=0).stop 2024-03-28T18:24:24.985 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:24:24.982+0000 7fb318a1b740 1 -- 172.21.15.22:0/988637518 shutdown_connections 2024-03-28T18:24:24.985 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:24:24.982+0000 7fb318a1b740 1 --2- 172.21.15.22:0/988637518 >> [v2:172.21.15.79:6838/2441542443,v1:172.21.15.79:6839/2441542443] conn(0x561d403f9530 0x561d403fb990 crc :-1 s=READY pgs=345 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:24:24.985 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:24:24.982+0000 7fb318a1b740 1 --2- 172.21.15.22:0/988637518 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561d403e9460 0x561d403e6360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:24:24.985 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:24:24.982+0000 7fb318a1b740 1 --2- 172.21.15.22:0/988637518 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561d403e0dd0 0x561d403e5e20 unknown :-1 s=CLOSED pgs=861 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:24:24.985 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:24:24.982+0000 7fb318a1b740 1 --2- 172.21.15.22:0/988637518 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x561d403e00f0 0x561d403e8a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:24:24.985 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:24:24.982+0000 7fb2fcc00640 0 client.17067 ms_handle_reset on v2:172.21.15.79:6838/2441542443 2024-03-28T18:24:24.985 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:24:24.982+0000 7fb318a1b740 1 -- 172.21.15.22:0/988637518 >> 172.21.15.22:0/988637518 conn(0x561d403a7200 msgr2=0x561d403dc890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:24:24.985 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:24:24.982+0000 7fb318a1b740 1 -- 172.21.15.22:0/988637518 shutdown_connections 2024-03-28T18:24:24.985 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:24:24.982+0000 7fb318a1b740 1 -- 172.21.15.22:0/988637518 wait complete. 2024-03-28T18:24:24.985 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:24:24.982+0000 7fb318a1b740 20 client.17067 trim_cache size 0 max 16384 2024-03-28T18:24:25.001 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:24:25.000+0000 7fb318a1b740 1 Processor -- start 2024-03-28T18:24:25.002 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:24:25.000+0000 7fb318a1b740 1 -- start start 2024-03-28T18:24:25.002 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:24:25.001+0000 7fb318a1b740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561d403e00f0 0x561d403e8550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:24:25.002 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:24:25.001+0000 7fb318a1b740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561d403e0dd0 0x561d403e8a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:24:25.002 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:24:25.001+0000 7fb318a1b740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x561d403e9460 0x561d403ea850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:24:25.002 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:24:25.001+0000 7fb318a1b740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3101b1c80 con 0x561d403e0dd0 2024-03-28T18:24:25.002 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:24:25.001+0000 7fb318a1b740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3101aefe0 con 0x561d403e9460 2024-03-28T18:24:25.002 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:24:25.001+0000 7fb318a1b740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3101b0fc0 con 0x561d403e00f0 2024-03-28T18:24:25.002 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:24:25.001+0000 7fb30e000640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561d403e0dd0 0x561d403e8a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:24:25.002 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:24:25.001+0000 7fb30e000640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561d403e0dd0 0x561d403e8a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:55342/0 (socket says 172.21.15.22:55342) 2024-03-28T18:24:25.002 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:24:25.001+0000 7fb30e000640 1 -- 172.21.15.22:0/2267038356 learned_addr learned my addr 172.21.15.22:0/2267038356 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-28T18:24:25.004 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:24:25.001+0000 7fb30f400640 1 --2- 172.21.15.22:0/2267038356 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x561d403e9460 0x561d403ea850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:24:25.004 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:24:25.001+0000 7fb30e000640 1 -- 172.21.15.22:0/2267038356 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561d403e00f0 msgr2=0x561d403e8550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T18:24:25.004 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:24:25.001+0000 7fb30e000640 1 --2- 172.21.15.22:0/2267038356 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561d403e00f0 0x561d403e8550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:24:25.004 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:24:25.001+0000 7fb30e000640 1 -- 172.21.15.22:0/2267038356 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x561d403e9460 msgr2=0x561d403ea850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T18:24:25.004 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:24:25.001+0000 7fb30e000640 1 --2- 172.21.15.22:0/2267038356 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x561d403e9460 0x561d403ea850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:24:25.004 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:24:25.001+0000 7fb30e000640 1 -- 172.21.15.22:0/2267038356 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561d400180f0 con 0x561d403e0dd0 2024-03-28T18:24:25.004 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:24:25.002+0000 7fb30e000640 1 --2- 172.21.15.22:0/2267038356 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561d403e0dd0 0x561d403e8a90 secure :-1 s=READY pgs=862 cs=0 l=1 rev1=1 crypto rx=0x7fb2f8011bf0 tx=0x7fb2f8005f20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3c2eac9a47d1b531 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:24:25.004 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:24:25.002+0000 7fb2ff400640 1 -- 172.21.15.22:0/2267038356 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2f802d070 con 0x561d403e0dd0 2024-03-28T18:24:25.004 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:24:25.002+0000 7fb2ff400640 1 -- 172.21.15.22:0/2267038356 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7fb2f800c700 con 0x561d403e0dd0 2024-03-28T18:24:25.004 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:24:25.002+0000 7fb2ff400640 1 -- 172.21.15.22:0/2267038356 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2f8004480 con 0x561d403e0dd0 2024-03-28T18:24:25.004 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:24:25.003+0000 7fb318a1b740 1 -- 172.21.15.22:0/2267038356 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561d403e0dd0 msgr2=0x561d403e8a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:24:25.004 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:24:25.003+0000 7fb318a1b740 1 --2- 172.21.15.22:0/2267038356 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561d403e0dd0 0x561d403e8a90 secure :-1 s=READY pgs=862 cs=0 l=1 rev1=1 crypto rx=0x7fb2f8011bf0 tx=0x7fb2f8005f20 comp rx=0 tx=0).stop 2024-03-28T18:24:25.004 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:24:25.003+0000 7fb318a1b740 1 -- 172.21.15.22:0/2267038356 shutdown_connections 2024-03-28T18:24:25.004 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:24:25.003+0000 7fb318a1b740 1 --2- 172.21.15.22:0/2267038356 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x561d403e9460 0x561d403ea850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:24:25.004 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:24:25.003+0000 7fb318a1b740 1 --2- 172.21.15.22:0/2267038356 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561d403e0dd0 0x561d403e8a90 unknown :-1 s=CLOSED pgs=862 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:24:25.005 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:24:25.003+0000 7fb318a1b740 1 --2- 172.21.15.22:0/2267038356 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561d403e00f0 0x561d403e8550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:24:25.005 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:24:25.003+0000 7fb318a1b740 1 -- 172.21.15.22:0/2267038356 >> 172.21.15.22:0/2267038356 conn(0x561d403a7200 msgr2=0x561d403da670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:24:25.005 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:24:25.003+0000 7fb318a1b740 1 -- 172.21.15.22:0/2267038356 shutdown_connections 2024-03-28T18:24:25.005 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:24:25.003+0000 7fb318a1b740 1 -- 172.21.15.22:0/2267038356 wait complete. 2024-03-28T18:24:25.007 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:24:25.003+0000 7fb318a1b740 1 -- start start 2024-03-28T18:24:25.007 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:24:25.004+0000 7fb318a1b740 1 Processor -- start 2024-03-28T18:24:25.007 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:24:25.004+0000 7fb318a1b740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561d403e00f0 0x561d403e26f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:24:25.008 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:24:25.004+0000 7fb318a1b740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x561d403e0dd0 0x561d403e2c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:24:25.008 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:24:25.004+0000 7fb318a1b740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561d403e9460 0x561d403e3170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:24:25.008 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:24:25.004+0000 7fb318a1b740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3101b0960 con 0x561d403e9460 2024-03-28T18:24:25.008 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:24:25.004+0000 7fb318a1b740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3101af640 con 0x561d403e0dd0 2024-03-28T18:24:25.008 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:24:25.004+0000 7fb318a1b740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3101afca0 con 0x561d403e00f0 2024-03-28T18:24:25.008 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:24:25.004+0000 7fb318a1b740 10 client.0 _ll_register_callbacks cb 0x7fb2f800e430 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-28T18:24:25.008 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:24:25.004+0000 7fb30f400640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561d403e9460 0x561d403e3170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:24:25.008 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:24:25.004+0000 7fb30f400640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561d403e9460 0x561d403e3170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:55350/0 (socket says 172.21.15.22:55350) 2024-03-28T18:24:25.008 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:24:25.004+0000 7fb30f400640 1 -- 172.21.15.22:0/3663335309 learned_addr learned my addr 172.21.15.22:0/3663335309 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-28T18:24:25.008 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:24:25.005+0000 7fb30ea00640 1 --2- 172.21.15.22:0/3663335309 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561d403e00f0 0x561d403e26f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:24:25.008 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:24:25.005+0000 7fb30e000640 1 --2- 172.21.15.22:0/3663335309 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x561d403e0dd0 0x561d403e2c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:24:25.008 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:24:25.005+0000 7fb30f400640 1 -- 172.21.15.22:0/3663335309 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561d403e00f0 msgr2=0x561d403e26f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T18:24:25.008 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:24:25.005+0000 7fb30f400640 1 --2- 172.21.15.22:0/3663335309 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561d403e00f0 0x561d403e26f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:24:25.008 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:24:25.005+0000 7fb30f400640 1 -- 172.21.15.22:0/3663335309 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x561d403e0dd0 msgr2=0x561d403e2c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T18:24:25.009 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:24:25.005+0000 7fb30f400640 1 --2- 172.21.15.22:0/3663335309 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x561d403e0dd0 0x561d403e2c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:24:25.009 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:24:25.005+0000 7fb30f400640 1 -- 172.21.15.22:0/3663335309 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x561d40454080 con 0x561d403e9460 2024-03-28T18:24:25.009 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:24:25.005+0000 7fb30f400640 1 --2- 172.21.15.22:0/3663335309 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561d403e9460 0x561d403e3170 secure :-1 s=READY pgs=863 cs=0 l=1 rev1=1 crypto rx=0x7fb30802c210 tx=0x7fb30802bcd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8dd6793740088bee server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:24:25.009 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:24:25.005+0000 7fb2fcc00640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-03-28T18:24:25.009 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:24:25.006+0000 7fb2fcc00640 1 -- 172.21.15.22:0/3663335309 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb308052070 con 0x561d403e9460 2024-03-28T18:24:25.009 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:24:25.006+0000 7fb2fcc00640 1 -- 172.21.15.22:0/3663335309 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(122..122 src has 1..122) v4 ==== 10389+0+0 (secure 0 0 0) 0x7fb308004d90 con 0x561d403e9460 2024-03-28T18:24:25.009 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:24:25.006+0000 7fb318a1b740 1 -- 172.21.15.22:0/3663335309 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fb2f800c700 con 0x561d403e9460 2024-03-28T18:24:25.009 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:24:25.006+0000 7fb2fcc00640 1 -- 172.21.15.22:0/3663335309 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=189) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb30805e050 con 0x561d403e9460 2024-03-28T18:24:25.009 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:24:25.006+0000 7fb318a1b740 10 client.17073 fetch_fsmap learned FSMap version 189 2024-03-28T18:24:25.009 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:24:25.006+0000 7fb318a1b740 1 -- 172.21.15.22:0/3663335309 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=189,monmap=4+}) v3 -- 0x561d40455390 con 0x561d403e9460 2024-03-28T18:24:25.009 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:24:25.006+0000 7fb2fcc00640 1 -- 172.21.15.22:0/3663335309 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7fb2f800c700 con 0x561d403e9460 2024-03-28T18:24:25.009 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:24:25.007+0000 7fb2fcc00640 1 -- 172.21.15.22:0/3663335309 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 189) v1 ==== 6221+0+0 (secure 0 0 0) 0x7fb308058cf0 con 0x561d403e9460 2024-03-28T18:24:25.009 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:24:25.007+0000 7fb318a1b740 10 client.17073 fetch_fsmap finished waiting for FSMap version 189 2024-03-28T18:24:25.009 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:24:25.007+0000 7fb318a1b740 10 client.17073 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-28T18:24:25.009 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:24:25.007+0000 7fb318a1b740 20 client.17073 populate_metadata read hostname 'smithi022' 2024-03-28T18:24:25.009 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:24:25.007+0000 7fb318a1b740 1 --2- 172.21.15.22:0/3663335309 >> [v2:172.21.15.79:6838/2441542443,v1:172.21.15.79:6839/2441542443] conn(0x561d40455ac0 0x561d40457ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:24:25.010 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:24:25.007+0000 7fb318a1b740 4 client.17073 mds_command: new command op to 34553 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-28T18:24:25.010 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:24:25.007+0000 7fb318a1b740 1 -- 172.21.15.22:0/3663335309 --> [v2:172.21.15.79:6838/2441542443,v1:172.21.15.79:6839/2441542443] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x561d4034bae0 con 0x561d40455ac0 2024-03-28T18:24:25.011 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:24:25.007+0000 7fb30ea00640 1 --2- 172.21.15.22:0/3663335309 >> [v2:172.21.15.79:6838/2441542443,v1:172.21.15.79:6839/2441542443] conn(0x561d40455ac0 0x561d40457ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:24:25.011 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:24:25.008+0000 7fb30ea00640 1 --2- 172.21.15.22:0/3663335309 >> [v2:172.21.15.79:6838/2441542443,v1:172.21.15.79:6839/2441542443] conn(0x561d40455ac0 0x561d40457ee0 crc :-1 s=READY pgs=346 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d6877cc2d80545e9 server_cookie=ddcdcfd76edccecc in_seq=0 out_seq=0 2024-03-28T18:24:25.011 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:24:25.008+0000 7fb2fcc00640 10 client.17073 ms_handle_connect on v2:172.21.15.79:6838/2441542443 2024-03-28T18:24:25.011 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:24:25.009+0000 7fb2fcc00640 1 -- 172.21.15.22:0/3663335309 <== mds.0 v2:172.21.15.79:6838/2441542443 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x561d4034bae0 con 0x561d40455ac0 2024-03-28T18:24:25.011 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:24:25.009+0000 7fb2fcc00640 10 client.17073 handle_command_reply: tid=0 2024-03-28T18:24:25.011 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:24:25.009+0000 7fb318a1b740 1 client.17073 shutdown 2024-03-28T18:24:25.011 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:24:25.009+0000 7fb318a1b740 1 -- 172.21.15.22:0/3663335309 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561d403e9460 msgr2=0x561d403e3170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:24:25.011 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:24:25.009+0000 7fb318a1b740 1 --2- 172.21.15.22:0/3663335309 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561d403e9460 0x561d403e3170 secure :-1 s=READY pgs=863 cs=0 l=1 rev1=1 crypto rx=0x7fb30802c210 tx=0x7fb30802bcd0 comp rx=0 tx=0).stop 2024-03-28T18:24:25.011 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:24:25.009+0000 7fb318a1b740 1 -- 172.21.15.22:0/3663335309 shutdown_connections 2024-03-28T18:24:25.012 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:24:25.009+0000 7fb318a1b740 1 --2- 172.21.15.22:0/3663335309 >> [v2:172.21.15.79:6838/2441542443,v1:172.21.15.79:6839/2441542443] conn(0x561d40455ac0 0x561d40457ee0 crc :-1 s=READY pgs=346 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:24:25.012 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:24:25.009+0000 7fb318a1b740 1 --2- 172.21.15.22:0/3663335309 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561d403e9460 0x561d403e3170 unknown :-1 s=CLOSED pgs=863 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:24:25.012 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:24:25.009+0000 7fb318a1b740 1 --2- 172.21.15.22:0/3663335309 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x561d403e0dd0 0x561d403e2c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:24:25.012 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:24:25.009+0000 7fb318a1b740 1 --2- 172.21.15.22:0/3663335309 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561d403e00f0 0x561d403e26f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:24:25.012 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:24:25.009+0000 7fb2fcc00640 0 client.17073 ms_handle_reset on v2:172.21.15.79:6838/2441542443 2024-03-28T18:24:25.012 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:24:25.009+0000 7fb318a1b740 1 -- 172.21.15.22:0/3663335309 >> 172.21.15.22:0/3663335309 conn(0x561d403a7200 msgr2=0x561d403e4580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:24:25.012 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:24:25.009+0000 7fb318a1b740 1 -- 172.21.15.22:0/3663335309 shutdown_connections 2024-03-28T18:24:25.012 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:24:25.009+0000 7fb318a1b740 1 -- 172.21.15.22:0/3663335309 wait complete. 2024-03-28T18:24:25.012 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:24:25.009+0000 7fb318a1b740 20 client.17073 trim_cache size 0 max 16384 2024-03-28T18:24:25.012 INFO:teuthology.orchestra.run.smithi022.stdout:{ 2024-03-28T18:24:25.012 INFO:teuthology.orchestra.run.smithi022.stdout: "status": "IDLE (waiting for more scrubs)", 2024-03-28T18:24:25.012 INFO:teuthology.orchestra.run.smithi022.stdout: "scrubs": { 2024-03-28T18:24:25.013 INFO:teuthology.orchestra.run.smithi022.stdout: "4bbbd61d-29a3-4bf6-a4f2-d5783d585de3": { 2024-03-28T18:24:25.013 INFO:teuthology.orchestra.run.smithi022.stdout: "path": "/", 2024-03-28T18:24:25.013 INFO:teuthology.orchestra.run.smithi022.stdout: "tag": "4bbbd61d-29a3-4bf6-a4f2-d5783d585de3", 2024-03-28T18:24:25.013 INFO:teuthology.orchestra.run.smithi022.stdout: "options": "recursive,force" 2024-03-28T18:24:25.013 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-03-28T18:24:25.013 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-03-28T18:24:25.013 INFO:teuthology.orchestra.run.smithi022.stdout:} 2024-03-28T18:24:25.014 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:24:25.010+0000 7fb30fe00640 1 -- 172.21.15.22:0/1353235237 >> [v2:172.21.15.22:6800/1357365082,v1:172.21.15.22:6801/1357365082] conn(0x7fb2f40a29f0 msgr2=0x7fb2f40a4eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:24:25.014 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:24:25.010+0000 7fb30fe00640 1 --2- 172.21.15.22:0/1353235237 >> [v2:172.21.15.22:6800/1357365082,v1:172.21.15.22:6801/1357365082] conn(0x7fb2f40a29f0 0x7fb2f40a4eb0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fb3080096c0 tx=0x7fb308009560 comp rx=0 tx=0).stop 2024-03-28T18:24:25.014 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:24:25.010+0000 7fb30fe00640 1 -- 172.21.15.22:0/1353235237 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb310073a50 msgr2=0x7fb310199fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:24:25.014 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:24:25.010+0000 7fb30fe00640 1 --2- 172.21.15.22:0/1353235237 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb310073a50 0x7fb310199fb0 secure :-1 s=READY pgs=859 cs=0 l=1 rev1=1 crypto rx=0x7fb30000eab0 tx=0x7fb30000ef70 comp rx=0 tx=0).stop 2024-03-28T18:24:25.014 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:24:25.010+0000 7fb30fe00640 1 -- 172.21.15.22:0/1353235237 shutdown_connections 2024-03-28T18:24:25.014 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:24:25.010+0000 7fb30fe00640 1 --2- 172.21.15.22:0/1353235237 >> [v2:172.21.15.22:6800/1357365082,v1:172.21.15.22:6801/1357365082] conn(0x7fb2f40a29f0 0x7fb2f40a4eb0 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:24:25.014 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:24:25.010+0000 7fb30fe00640 1 --2- 172.21.15.22:0/1353235237 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fb31010ddd0 0x7fb31019e820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:24:25.014 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:24:25.010+0000 7fb30fe00640 1 --2- 172.21.15.22:0/1353235237 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb310074370 0x7fb31019a4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:24:25.014 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:24:25.010+0000 7fb30fe00640 1 --2- 172.21.15.22:0/1353235237 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb310073a50 0x7fb310199fb0 unknown :-1 s=CLOSED pgs=859 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:24:25.014 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:24:25.010+0000 7fb30fe00640 1 -- 172.21.15.22:0/1353235237 >> 172.21.15.22:0/1353235237 conn(0x7fb3100fd000 msgr2=0x7fb3100fe720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:24:25.014 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:24:25.010+0000 7fb30fe00640 1 -- 172.21.15.22:0/1353235237 shutdown_connections 2024-03-28T18:24:25.014 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:24:25.010+0000 7fb30fe00640 1 -- 172.21.15.22:0/1353235237 wait complete. 2024-03-28T18:24:25.028 INFO:tasks.cephfs.filesystem:scrub status for tag:4bbbd61d-29a3-4bf6-a4f2-d5783d585de3 - {'path': '/', 'tag': '4bbbd61d-29a3-4bf6-a4f2-d5783d585de3', 'options': 'recursive,force'} 2024-03-28T18:24:25.282 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:24:25 smithi022 ceph-mon[31985]: 2 slow requests, 0 included below; oldest blocked for > 451.845936 secs 2024-03-28T18:24:25.425 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:24:25 smithi134 ceph-mon[35037]: 2 slow requests, 0 included below; oldest blocked for > 451.845936 secs 2024-03-28T18:24:25.432 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:24:25 smithi079 ceph-mon[35312]: 2 slow requests, 0 included below; oldest blocked for > 451.845936 secs 2024-03-28T18:24:26.424 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:24:26 smithi134 ceph-mon[35037]: pgmap v545: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-03-28T18:24:26.432 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:24:26 smithi079 ceph-mon[35312]: pgmap v545: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-03-28T18:24:26.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:24:26 smithi022 ceph-mon[31985]: pgmap v545: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-03-28T18:24:28.425 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:24:28 smithi134 ceph-mon[35037]: pgmap v546: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 7 op/s 2024-03-28T18:24:28.432 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:24:28 smithi079 ceph-mon[35312]: pgmap v546: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 7 op/s 2024-03-28T18:24:28.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:24:28 smithi022 ceph-mon[31985]: pgmap v546: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 7 op/s 2024-03-28T18:24:30.335 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:24:30 smithi022 ceph-mon[31985]: pgmap v547: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:24:30.335 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:24:30 smithi022 ceph-mon[31985]: 2 slow requests, 0 included below; oldest blocked for > 456.846001 secs 2024-03-28T18:24:30.424 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:24:30 smithi134 ceph-mon[35037]: pgmap v547: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:24:30.425 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:24:30 smithi134 ceph-mon[35037]: 2 slow requests, 0 included below; oldest blocked for > 456.846001 secs 2024-03-28T18:24:30.432 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:24:30 smithi079 ceph-mon[35312]: pgmap v547: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:24:30.432 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:24:30 smithi079 ceph-mon[35312]: 2 slow requests, 0 included below; oldest blocked for > 456.846001 secs 2024-03-28T18:24:32.425 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:24:32 smithi134 ceph-mon[35037]: pgmap v548: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:24:32.432 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:24:32 smithi079 ceph-mon[35312]: pgmap v548: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:24:32.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:24:32 smithi022 ceph-mon[31985]: pgmap v548: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:24:34.425 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:24:34 smithi134 ceph-mon[35037]: pgmap v549: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:24:34.432 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:24:34 smithi079 ceph-mon[35312]: pgmap v549: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:24:34.531 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:24:34 smithi022 ceph-mon[31985]: pgmap v549: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:24:35.425 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:24:35 smithi134 ceph-mon[35037]: 2 slow requests, 0 included below; oldest blocked for > 461.846077 secs 2024-03-28T18:24:35.432 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:24:35 smithi079 ceph-mon[35312]: 2 slow requests, 0 included below; oldest blocked for > 461.846077 secs 2024-03-28T18:24:35.531 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:24:35 smithi022 ceph-mon[31985]: 2 slow requests, 0 included below; oldest blocked for > 461.846077 secs 2024-03-28T18:24:36.425 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:24:36 smithi134 ceph-mon[35037]: pgmap v550: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:24:36.432 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:24:36 smithi079 ceph-mon[35312]: pgmap v550: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:24:36.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:24:36 smithi022 ceph-mon[31985]: pgmap v550: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:24:38.425 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:24:38 smithi134 ceph-mon[35037]: pgmap v551: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:24:38.432 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:24:38 smithi079 ceph-mon[35312]: pgmap v551: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:24:38.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:24:38 smithi022 ceph-mon[31985]: pgmap v551: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:24:39.425 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:24:39 smithi134 ceph-mon[35037]: pgmap v552: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:24:39.432 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:24:39 smithi079 ceph-mon[35312]: pgmap v552: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:24:39.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:24:39 smithi022 ceph-mon[31985]: pgmap v552: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:24:40.425 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:24:40 smithi134 ceph-mon[35037]: 2 slow requests, 0 included below; oldest blocked for > 466.846152 secs 2024-03-28T18:24:40.432 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:24:40 smithi079 ceph-mon[35312]: 2 slow requests, 0 included below; oldest blocked for > 466.846152 secs 2024-03-28T18:24:40.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:24:40 smithi022 ceph-mon[31985]: 2 slow requests, 0 included below; oldest blocked for > 466.846152 secs 2024-03-28T18:24:41.424 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:24:41 smithi134 ceph-mon[35037]: pgmap v553: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:24:41.432 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:24:41 smithi079 ceph-mon[35312]: pgmap v553: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:24:41.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:24:41 smithi022 ceph-mon[31985]: pgmap v553: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:24:43.925 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:24:43 smithi134 ceph-mon[35037]: pgmap v554: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:24:43.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:24:43 smithi079 ceph-mon[35312]: pgmap v554: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:24:44.032 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:24:43 smithi022 ceph-mon[31985]: pgmap v554: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:24:44.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:24:44 smithi134 ceph-mon[35037]: 2 slow requests, 0 included below; oldest blocked for > 471.846224 secs 2024-03-28T18:24:44.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:24:44 smithi079 ceph-mon[35312]: 2 slow requests, 0 included below; oldest blocked for > 471.846224 secs 2024-03-28T18:24:45.032 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:24:44 smithi022 ceph-mon[31985]: 2 slow requests, 0 included below; oldest blocked for > 471.846224 secs 2024-03-28T18:24:45.834 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:24:45 smithi079 ceph-mon[35312]: pgmap v555: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:24:45.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:24:45 smithi134 ceph-mon[35037]: pgmap v555: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:24:46.032 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:24:45 smithi022 ceph-mon[31985]: pgmap v555: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:24:47.925 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:24:47 smithi134 ceph-mon[35037]: pgmap v556: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:24:47.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:24:47 smithi079 ceph-mon[35312]: pgmap v556: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:24:48.032 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:24:47 smithi022 ceph-mon[31985]: pgmap v556: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:24:49.925 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:24:49 smithi134 ceph-mon[35037]: pgmap v557: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:24:49.925 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:24:49 smithi134 ceph-mon[35037]: 2 slow requests, 0 included below; oldest blocked for > 476.846348 secs 2024-03-28T18:24:49.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:24:49 smithi079 ceph-mon[35312]: pgmap v557: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:24:49.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:24:49 smithi079 ceph-mon[35312]: 2 slow requests, 0 included below; oldest blocked for > 476.846348 secs 2024-03-28T18:24:50.032 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:24:49 smithi022 ceph-mon[31985]: pgmap v557: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:24:50.032 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:24:49 smithi022 ceph-mon[31985]: 2 slow requests, 0 included below; oldest blocked for > 476.846348 secs 2024-03-28T18:24:51.925 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:24:51 smithi134 ceph-mon[35037]: pgmap v558: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:24:51.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:24:51 smithi079 ceph-mon[35312]: pgmap v558: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:24:52.032 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:24:51 smithi022 ceph-mon[31985]: pgmap v558: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:24:53.925 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:24:53 smithi134 ceph-mon[35037]: pgmap v559: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:24:53.933 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:24:53 smithi079 ceph-mon[35312]: pgmap v559: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:24:54.032 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:24:53 smithi022 ceph-mon[31985]: pgmap v559: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:24:54.925 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:24:54 smithi134 ceph-mon[35037]: 2 slow requests, 0 included below; oldest blocked for > 481.846464 secs 2024-03-28T18:24:54.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:24:54 smithi079 ceph-mon[35312]: 2 slow requests, 0 included below; oldest blocked for > 481.846464 secs 2024-03-28T18:24:55.028 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-28T18:24:55.029 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-28T18:24:55.037 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:24:54 smithi022 ceph-mon[31985]: 2 slow requests, 0 included below; oldest blocked for > 481.846464 secs 2024-03-28T18:24:55.267 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:24:55.267+0000 7f687a600640 1 -- 172.21.15.22:0/668225270 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6874108800 msgr2=0x7f687410c7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:24:55.267 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:24:55.267+0000 7f687a600640 1 --2- 172.21.15.22:0/668225270 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6874108800 0x7f687410c7c0 secure :-1 s=READY pgs=864 cs=0 l=1 rev1=1 crypto rx=0x7f6864009900 tx=0x7f686402bc30 comp rx=0 tx=0).stop 2024-03-28T18:24:55.268 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:24:55.267+0000 7f687a600640 1 -- 172.21.15.22:0/668225270 shutdown_connections 2024-03-28T18:24:55.268 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:24:55.267+0000 7f687a600640 1 --2- 172.21.15.22:0/668225270 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f687410cea0 0x7f6874114a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:24:55.268 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:24:55.267+0000 7f687a600640 1 --2- 172.21.15.22:0/668225270 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6874108800 0x7f687410c7c0 unknown :-1 s=CLOSED pgs=864 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:24:55.268 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:24:55.267+0000 7f687a600640 1 --2- 172.21.15.22:0/668225270 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6874107e50 0x7f6874108230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:24:55.268 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:24:55.267+0000 7f687a600640 1 -- 172.21.15.22:0/668225270 >> 172.21.15.22:0/668225270 conn(0x7f68740774c0 msgr2=0x7f68741064c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:24:55.268 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:24:55.267+0000 7f687a600640 1 -- 172.21.15.22:0/668225270 shutdown_connections 2024-03-28T18:24:55.268 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:24:55.267+0000 7f687a600640 1 -- 172.21.15.22:0/668225270 wait complete. 2024-03-28T18:24:55.270 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:24:55.268+0000 7f687a600640 1 Processor -- start 2024-03-28T18:24:55.271 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:24:55.268+0000 7f687a600640 1 -- start start 2024-03-28T18:24:55.271 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:24:55.269+0000 7f687a600640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6874107e50 0x7f687419e470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:24:55.271 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:24:55.269+0000 7f687a600640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f687410cea0 0x7f687419e9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:24:55.271 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:24:55.269+0000 7f687a600640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f68741a2d30 0x7f68741a31e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:24:55.271 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:24:55.269+0000 7f687a600640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6874117f70 con 0x7f6874107e50 2024-03-28T18:24:55.271 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:24:55.269+0000 7f687a600640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6874117df0 con 0x7f687410cea0 2024-03-28T18:24:55.271 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:24:55.269+0000 7f687a600640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68741180f0 con 0x7f68741a2d30 2024-03-28T18:24:55.271 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:24:55.269+0000 7f6879200640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6874107e50 0x7f687419e470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:24:55.271 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:24:55.269+0000 7f6879200640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6874107e50 0x7f687419e470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:41634/0 (socket says 172.21.15.22:41634) 2024-03-28T18:24:55.271 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:24:55.269+0000 7f6879200640 1 -- 172.21.15.22:0/1006309048 learned_addr learned my addr 172.21.15.22:0/1006309048 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-28T18:24:55.271 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:24:55.269+0000 7f6873e00640 1 --2- 172.21.15.22:0/1006309048 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f687410cea0 0x7f687419e9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:24:55.271 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:24:55.269+0000 7f6879200640 1 -- 172.21.15.22:0/1006309048 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f68741a2d30 msgr2=0x7f68741a31e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:24:55.271 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:24:55.269+0000 7f6879200640 1 --2- 172.21.15.22:0/1006309048 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f68741a2d30 0x7f68741a31e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:24:55.271 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:24:55.269+0000 7f6879200640 1 -- 172.21.15.22:0/1006309048 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f687410cea0 msgr2=0x7f687419e9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:24:55.271 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:24:55.269+0000 7f6879200640 1 --2- 172.21.15.22:0/1006309048 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f687410cea0 0x7f687419e9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:24:55.272 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:24:55.269+0000 7f6879200640 1 -- 172.21.15.22:0/1006309048 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f68741a3820 con 0x7f6874107e50 2024-03-28T18:24:55.272 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:24:55.270+0000 7f6879200640 1 --2- 172.21.15.22:0/1006309048 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6874107e50 0x7f687419e470 secure :-1 s=READY pgs=865 cs=0 l=1 rev1=1 crypto rx=0x7f686800d770 tx=0x7f686800dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:24:55.272 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:24:55.270+0000 7f6871600640 1 -- 172.21.15.22:0/1006309048 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6868017070 con 0x7f6874107e50 2024-03-28T18:24:55.272 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:24:55.270+0000 7f6871600640 1 -- 172.21.15.22:0/1006309048 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f68680054f0 con 0x7f6874107e50 2024-03-28T18:24:55.272 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:24:55.270+0000 7f687a600640 1 -- 172.21.15.22:0/1006309048 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f68741a3b10 con 0x7f6874107e50 2024-03-28T18:24:55.272 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:24:55.270+0000 7f6871600640 1 -- 172.21.15.22:0/1006309048 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6868004d60 con 0x7f6874107e50 2024-03-28T18:24:55.272 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:24:55.270+0000 7f687a600640 1 -- 172.21.15.22:0/1006309048 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f68741ab220 con 0x7f6874107e50 2024-03-28T18:24:55.273 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:24:55.273+0000 7f6871600640 1 -- 172.21.15.22:0/1006309048 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 26) v1 ==== 137421+0+0 (secure 0 0 0) 0x7f6868004f00 con 0x7f6874107e50 2024-03-28T18:24:55.273 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:24:55.273+0000 7f687ea1e740 1 Processor -- start 2024-03-28T18:24:55.273 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:24:55.274+0000 7f687ea1e740 1 -- start start 2024-03-28T18:24:55.276 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:24:55.274+0000 7f687ea1e740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562f94eb2dd0 0x562f94eb1b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:24:55.276 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:24:55.274+0000 7f687ea1e740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562f94eb20f0 0x562f94eb2570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:24:55.276 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:24:55.274+0000 7f687ea1e740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x562f94ebb460 0x562f94ebd850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:24:55.276 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:24:55.274+0000 7f687ea1e740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x562f94e16250 con 0x562f94eb2dd0 2024-03-28T18:24:55.276 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:24:55.274+0000 7f687ea1e740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x562f94dbea30 con 0x562f94ebb460 2024-03-28T18:24:55.276 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:24:55.274+0000 7f687ea1e740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x562f94e122f0 con 0x562f94eb20f0 2024-03-28T18:24:55.276 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:24:55.274+0000 7f6879c00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562f94eb2dd0 0x562f94eb1b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:24:55.276 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:24:55.274+0000 7f6879c00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562f94eb2dd0 0x562f94eb1b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:41642/0 (socket says 172.21.15.22:41642) 2024-03-28T18:24:55.276 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:24:55.275+0000 7f6879c00640 1 -- 172.21.15.22:0/2907877580 learned_addr learned my addr 172.21.15.22:0/2907877580 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-28T18:24:55.276 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:24:55.275+0000 7f6873e00640 1 --2- 172.21.15.22:0/2907877580 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x562f94ebb460 0x562f94ebd850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:24:55.277 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:24:55.275+0000 7f6879200640 1 --2- 172.21.15.22:0/2907877580 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562f94eb20f0 0x562f94eb2570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:24:55.277 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:24:55.275+0000 7f6879c00640 1 -- 172.21.15.22:0/2907877580 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562f94eb20f0 msgr2=0x562f94eb2570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T18:24:55.277 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:24:55.275+0000 7f6879c00640 1 --2- 172.21.15.22:0/2907877580 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562f94eb20f0 0x562f94eb2570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:24:55.277 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:24:55.275+0000 7f6879c00640 1 -- 172.21.15.22:0/2907877580 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x562f94ebb460 msgr2=0x562f94ebd850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T18:24:55.277 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:24:55.275+0000 7f6879c00640 1 --2- 172.21.15.22:0/2907877580 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x562f94ebb460 0x562f94ebd850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:24:55.277 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:24:55.275+0000 7f6879c00640 1 -- 172.21.15.22:0/2907877580 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562f94da9690 con 0x562f94eb2dd0 2024-03-28T18:24:55.277 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:24:55.275+0000 7f6879200640 1 --2- 172.21.15.22:0/2907877580 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562f94eb20f0 0x562f94eb2570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T18:24:55.277 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:24:55.275+0000 7f6871600640 1 --2- 172.21.15.22:0/1006309048 >> [v2:172.21.15.22:6800/1357365082,v1:172.21.15.22:6801/1357365082] conn(0x7f685c0a2970 0x7f685c0a4e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:24:55.277 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:24:55.275+0000 7f6871600640 1 -- 172.21.15.22:0/1006309048 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(122..122 src has 1..122) v4 ==== 10389+0+0 (secure 0 0 0) 0x7f68680cf140 con 0x7f6874107e50 2024-03-28T18:24:55.277 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:24:55.275+0000 7f6879c00640 1 --2- 172.21.15.22:0/2907877580 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562f94eb2dd0 0x562f94eb1b20 secure :-1 s=READY pgs=866 cs=0 l=1 rev1=1 crypto rx=0x7f686c009d90 tx=0x7f686c00e6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8ef3eabaa3a1a796 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:24:55.277 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:24:55.276+0000 7f6879200640 1 --2- 172.21.15.22:0/1006309048 >> [v2:172.21.15.22:6800/1357365082,v1:172.21.15.22:6801/1357365082] conn(0x7f685c0a2970 0x7f685c0a4e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:24:55.277 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:24:55.276+0000 7f6862000640 1 -- 172.21.15.22:0/2907877580 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f686c016070 con 0x562f94eb2dd0 2024-03-28T18:24:55.277 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:24:55.276+0000 7f6862000640 1 -- 172.21.15.22:0/2907877580 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f686c005020 con 0x562f94eb2dd0 2024-03-28T18:24:55.277 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:24:55.276+0000 7f6862000640 1 -- 172.21.15.22:0/2907877580 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f686c011690 con 0x562f94eb2dd0 2024-03-28T18:24:55.281 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:24:55.276+0000 7f687ea1e740 1 -- 172.21.15.22:0/2907877580 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562f94eb2dd0 msgr2=0x562f94eb1b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:24:55.281 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:24:55.276+0000 7f687ea1e740 1 --2- 172.21.15.22:0/2907877580 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562f94eb2dd0 0x562f94eb1b20 secure :-1 s=READY pgs=866 cs=0 l=1 rev1=1 crypto rx=0x7f686c009d90 tx=0x7f686c00e6c0 comp rx=0 tx=0).stop 2024-03-28T18:24:55.281 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:24:55.276+0000 7f6879200640 1 --2- 172.21.15.22:0/1006309048 >> [v2:172.21.15.22:6800/1357365082,v1:172.21.15.22:6801/1357365082] conn(0x7f685c0a2970 0x7f685c0a4e30 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x562f94eb3320 tx=0x7f68680bb040 comp rx=0 tx=0).ready entity=mgr.15522 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:24:55.281 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:24:55.277+0000 7f687ea1e740 1 -- 172.21.15.22:0/2907877580 shutdown_connections 2024-03-28T18:24:55.282 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:24:55.277+0000 7f687ea1e740 1 --2- 172.21.15.22:0/2907877580 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x562f94ebb460 0x562f94ebd850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:24:55.282 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:24:55.277+0000 7f687ea1e740 1 --2- 172.21.15.22:0/2907877580 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562f94eb20f0 0x562f94eb2570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:24:55.282 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:24:55.277+0000 7f687ea1e740 1 --2- 172.21.15.22:0/2907877580 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562f94eb2dd0 0x562f94eb1b20 unknown :-1 s=CLOSED pgs=866 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:24:55.282 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:24:55.277+0000 7f687ea1e740 1 -- 172.21.15.22:0/2907877580 >> 172.21.15.22:0/2907877580 conn(0x562f94e79200 msgr2=0x562f94eacb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:24:55.282 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:24:55.277+0000 7f687ea1e740 1 -- 172.21.15.22:0/2907877580 shutdown_connections 2024-03-28T18:24:55.282 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:24:55.277+0000 7f687ea1e740 1 -- 172.21.15.22:0/2907877580 wait complete. 2024-03-28T18:24:55.282 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:24:55.277+0000 7f687ea1e740 1 -- start start 2024-03-28T18:24:55.282 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:24:55.277+0000 7f687ea1e740 1 Processor -- start 2024-03-28T18:24:55.282 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:24:55.278+0000 7f687ea1e740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x562f94eb20f0 0x562f94ebab70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:24:55.282 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:24:55.278+0000 7f687ea1e740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562f94eb2dd0 0x562f94eb7f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:24:55.282 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:24:55.278+0000 7f687ea1e740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562f94ebb460 0x562f94eb8440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:24:55.282 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:24:55.278+0000 7f687ea1e740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x562f94ebe2d0 con 0x562f94eb2dd0 2024-03-28T18:24:55.282 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:24:55.278+0000 7f687ea1e740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x562f94dd1e80 con 0x562f94eb20f0 2024-03-28T18:24:55.283 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:24:55.278+0000 7f687ea1e740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x562f94ebe450 con 0x562f94ebb460 2024-03-28T18:24:55.286 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:24:55.278+0000 7f687ea1e740 10 client.0 _ll_register_callbacks cb 0x562f94de4130 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-28T18:24:55.286 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:24:55.278+0000 7f6873e00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562f94eb2dd0 0x562f94eb7f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:24:55.286 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:24:55.278+0000 7f6873e00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562f94eb2dd0 0x562f94eb7f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:41656/0 (socket says 172.21.15.22:41656) 2024-03-28T18:24:55.286 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:24:55.278+0000 7f6873e00640 1 -- 172.21.15.22:0/1492009215 learned_addr learned my addr 172.21.15.22:0/1492009215 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-28T18:24:55.286 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:24:55.278+0000 7f6879c00640 1 --2- 172.21.15.22:0/1492009215 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x562f94eb20f0 0x562f94ebab70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:24:55.286 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:24:55.279+0000 7f6879c00640 1 --2- 172.21.15.22:0/1492009215 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562f94ebb460 0x562f94eb8440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:24:55.286 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:24:55.279+0000 7f6873e00640 1 -- 172.21.15.22:0/1492009215 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562f94ebb460 msgr2=0x562f94eb8440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T18:24:55.286 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:24:55.279+0000 7f6873e00640 1 --2- 172.21.15.22:0/1492009215 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562f94ebb460 0x562f94eb8440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:24:55.286 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:24:55.279+0000 7f6873e00640 1 -- 172.21.15.22:0/1492009215 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x562f94eb20f0 msgr2=0x562f94ebab70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T18:24:55.286 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:24:55.279+0000 7f6873e00640 1 --2- 172.21.15.22:0/1492009215 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x562f94eb20f0 0x562f94ebab70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:24:55.286 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:24:55.279+0000 7f6873e00640 1 -- 172.21.15.22:0/1492009215 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x562f94bf47d0 con 0x562f94eb2dd0 2024-03-28T18:24:55.286 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:24:55.279+0000 7f6879c00640 1 --2- 172.21.15.22:0/1492009215 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562f94ebb460 0x562f94eb8440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T18:24:55.286 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:24:55.279+0000 7f6873e00640 1 --2- 172.21.15.22:0/1492009215 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562f94eb2dd0 0x562f94eb7f00 secure :-1 s=READY pgs=867 cs=0 l=1 rev1=1 crypto rx=0x7f686402c240 tx=0x7f6864005270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=22acee1f83d79a79 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:24:55.286 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:24:55.279+0000 7f6856a00640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-03-28T18:24:55.286 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:24:55.279+0000 7f6856a00640 1 -- 172.21.15.22:0/1492009215 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6864042030 con 0x562f94eb2dd0 2024-03-28T18:24:55.287 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:24:55.280+0000 7f6856a00640 1 -- 172.21.15.22:0/1492009215 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(122..122 src has 1..122) v4 ==== 10389+0+0 (secure 0 0 0) 0x7f686402c5d0 con 0x562f94eb2dd0 2024-03-28T18:24:55.287 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:24:55.280+0000 7f687ea1e740 1 -- 172.21.15.22:0/1492009215 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f686c005020 con 0x562f94eb2dd0 2024-03-28T18:24:55.287 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:24:55.280+0000 7f6856a00640 1 -- 172.21.15.22:0/1492009215 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=189) v2 ==== 24+0+0 (secure 0 0 0) 0x7f686404f050 con 0x562f94eb2dd0 2024-03-28T18:24:55.287 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:24:55.280+0000 7f687ea1e740 10 client.17085 fetch_fsmap learned FSMap version 189 2024-03-28T18:24:55.287 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:24:55.280+0000 7f687ea1e740 1 -- 172.21.15.22:0/1492009215 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=189,monmap=4+}) v3 -- 0x562f94b2a3f0 con 0x562f94eb2dd0 2024-03-28T18:24:55.287 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:24:55.280+0000 7f6856a00640 1 -- 172.21.15.22:0/1492009215 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f686c005020 con 0x562f94eb2dd0 2024-03-28T18:24:55.287 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:24:55.280+0000 7f6856a00640 1 -- 172.21.15.22:0/1492009215 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 189) v1 ==== 6221+0+0 (secure 0 0 0) 0x7f6864048cc0 con 0x562f94eb2dd0 2024-03-28T18:24:55.287 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:24:55.281+0000 7f687ea1e740 10 client.17085 fetch_fsmap finished waiting for FSMap version 189 2024-03-28T18:24:55.287 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:24:55.281+0000 7f687ea1e740 10 client.17085 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-28T18:24:55.287 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:24:55.281+0000 7f687ea1e740 20 client.17085 populate_metadata read hostname 'smithi022' 2024-03-28T18:24:55.287 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:24:55.281+0000 7f687ea1e740 1 --2- 172.21.15.22:0/1492009215 >> [v2:172.21.15.79:6838/2441542443,v1:172.21.15.79:6839/2441542443] conn(0x562f94ecb420 0x562f94ecd8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:24:55.287 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:24:55.281+0000 7f687ea1e740 4 client.17085 mds_command: new command op to 34553 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-28T18:24:55.287 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:24:55.281+0000 7f687ea1e740 1 -- 172.21.15.22:0/1492009215 --> [v2:172.21.15.79:6838/2441542443,v1:172.21.15.79:6839/2441542443] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x562f94e23790 con 0x562f94ecb420 2024-03-28T18:24:55.287 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:24:55.281+0000 7f6879c00640 1 --2- 172.21.15.22:0/1492009215 >> [v2:172.21.15.79:6838/2441542443,v1:172.21.15.79:6839/2441542443] conn(0x562f94ecb420 0x562f94ecd8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:24:55.289 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:24:55.282+0000 7f6879c00640 1 --2- 172.21.15.22:0/1492009215 >> [v2:172.21.15.79:6838/2441542443,v1:172.21.15.79:6839/2441542443] conn(0x562f94ecb420 0x562f94ecd8c0 crc :-1 s=READY pgs=349 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=84fb4e0b0997492 server_cookie=79557b8f50f1eaaf in_seq=0 out_seq=0 2024-03-28T18:24:55.289 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:24:55.282+0000 7f6856a00640 10 client.17085 ms_handle_connect on v2:172.21.15.79:6838/2441542443 2024-03-28T18:24:55.289 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:24:55.283+0000 7f6856a00640 1 -- 172.21.15.22:0/1492009215 <== mds.0 v2:172.21.15.79:6838/2441542443 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x562f94e23790 con 0x562f94ecb420 2024-03-28T18:24:55.289 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:24:55.283+0000 7f6856a00640 10 client.17085 handle_command_reply: tid=0 2024-03-28T18:24:55.289 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:24:55.283+0000 7f687ea1e740 1 client.17085 shutdown 2024-03-28T18:24:55.289 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:24:55.283+0000 7f687ea1e740 1 -- 172.21.15.22:0/1492009215 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562f94eb2dd0 msgr2=0x562f94eb7f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:24:55.289 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:24:55.283+0000 7f687ea1e740 1 --2- 172.21.15.22:0/1492009215 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562f94eb2dd0 0x562f94eb7f00 secure :-1 s=READY pgs=867 cs=0 l=1 rev1=1 crypto rx=0x7f686402c240 tx=0x7f6864005270 comp rx=0 tx=0).stop 2024-03-28T18:24:55.289 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:24:55.283+0000 7f687ea1e740 1 -- 172.21.15.22:0/1492009215 shutdown_connections 2024-03-28T18:24:55.289 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:24:55.283+0000 7f687ea1e740 1 --2- 172.21.15.22:0/1492009215 >> [v2:172.21.15.79:6838/2441542443,v1:172.21.15.79:6839/2441542443] conn(0x562f94ecb420 0x562f94ecd8c0 crc :-1 s=READY pgs=349 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:24:55.289 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:24:55.283+0000 7f687ea1e740 1 --2- 172.21.15.22:0/1492009215 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562f94ebb460 0x562f94eb8440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:24:55.289 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:24:55.283+0000 7f687ea1e740 1 --2- 172.21.15.22:0/1492009215 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562f94eb2dd0 0x562f94eb7f00 unknown :-1 s=CLOSED pgs=867 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:24:55.289 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:24:55.283+0000 7f687ea1e740 1 --2- 172.21.15.22:0/1492009215 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x562f94eb20f0 0x562f94ebab70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:24:55.289 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:24:55.283+0000 7f6856a00640 0 client.17085 ms_handle_reset on v2:172.21.15.79:6838/2441542443 2024-03-28T18:24:55.289 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:24:55.283+0000 7f687ea1e740 1 -- 172.21.15.22:0/1492009215 >> 172.21.15.22:0/1492009215 conn(0x562f94e79200 msgr2=0x562f94eae890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:24:55.289 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:24:55.284+0000 7f687ea1e740 1 -- 172.21.15.22:0/1492009215 shutdown_connections 2024-03-28T18:24:55.290 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:24:55.284+0000 7f687ea1e740 1 -- 172.21.15.22:0/1492009215 wait complete. 2024-03-28T18:24:55.290 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:24:55.284+0000 7f687ea1e740 20 client.17085 trim_cache size 0 max 16384 2024-03-28T18:24:55.301 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:24:55.301+0000 7f687ea1e740 1 Processor -- start 2024-03-28T18:24:55.302 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:24:55.301+0000 7f687ea1e740 1 -- start start 2024-03-28T18:24:55.302 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:24:55.302+0000 7f687ea1e740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562f94eb20f0 0x562f94eba630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:24:55.302 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:24:55.302+0000 7f687ea1e740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562f94eb2dd0 0x562f94ebab70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:24:55.302 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:24:55.302+0000 7f687ea1e740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x562f94ebb460 0x562f94ebc850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:24:55.302 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:24:55.302+0000 7f687ea1e740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68741b60f0 con 0x562f94eb2dd0 2024-03-28T18:24:55.302 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:24:55.302+0000 7f687ea1e740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68741b3450 con 0x562f94ebb460 2024-03-28T18:24:55.302 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:24:55.302+0000 7f687ea1e740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68741b5430 con 0x562f94eb20f0 2024-03-28T18:24:55.303 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:24:55.302+0000 7f6873e00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562f94eb2dd0 0x562f94ebab70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:24:55.304 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:24:55.302+0000 7f6873e00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562f94eb2dd0 0x562f94ebab70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:41658/0 (socket says 172.21.15.22:41658) 2024-03-28T18:24:55.304 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:24:55.302+0000 7f6873e00640 1 -- 172.21.15.22:0/3443521726 learned_addr learned my addr 172.21.15.22:0/3443521726 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-28T18:24:55.304 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:24:55.302+0000 7f6879c00640 1 --2- 172.21.15.22:0/3443521726 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562f94eb20f0 0x562f94eba630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:24:55.304 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:24:55.303+0000 7f6879c00640 1 --2- 172.21.15.22:0/3443521726 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x562f94ebb460 0x562f94ebc850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:24:55.304 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:24:55.303+0000 7f6873e00640 1 -- 172.21.15.22:0/3443521726 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562f94eb20f0 msgr2=0x562f94eba630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T18:24:55.304 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:24:55.303+0000 7f6873e00640 1 --2- 172.21.15.22:0/3443521726 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562f94eb20f0 0x562f94eba630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:24:55.304 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:24:55.303+0000 7f6873e00640 1 -- 172.21.15.22:0/3443521726 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x562f94ebb460 msgr2=0x562f94ebc850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T18:24:55.304 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:24:55.303+0000 7f6873e00640 1 --2- 172.21.15.22:0/3443521726 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x562f94ebb460 0x562f94ebc850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:24:55.304 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:24:55.303+0000 7f6873e00640 1 -- 172.21.15.22:0/3443521726 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562f94aea0f0 con 0x562f94eb2dd0 2024-03-28T18:24:55.304 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:24:55.303+0000 7f6879c00640 1 --2- 172.21.15.22:0/3443521726 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x562f94ebb460 0x562f94ebc850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-28T18:24:55.304 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:24:55.303+0000 7f6873e00640 1 --2- 172.21.15.22:0/3443521726 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562f94eb2dd0 0x562f94ebab70 secure :-1 s=READY pgs=868 cs=0 l=1 rev1=1 crypto rx=0x7f6864004b90 tx=0x7f6864000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=43c65daa19486a8 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:24:55.304 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:24:55.304+0000 7f6861600640 1 -- 172.21.15.22:0/3443521726 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6864042070 con 0x562f94eb2dd0 2024-03-28T18:24:55.305 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:24:55.304+0000 7f6861600640 1 -- 172.21.15.22:0/3443521726 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f6864004e80 con 0x562f94eb2dd0 2024-03-28T18:24:55.305 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:24:55.304+0000 7f6861600640 1 -- 172.21.15.22:0/3443521726 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68640046c0 con 0x562f94eb2dd0 2024-03-28T18:24:55.308 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:24:55.304+0000 7f687ea1e740 1 -- 172.21.15.22:0/3443521726 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562f94eb2dd0 msgr2=0x562f94ebab70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:24:55.308 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:24:55.304+0000 7f687ea1e740 1 --2- 172.21.15.22:0/3443521726 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562f94eb2dd0 0x562f94ebab70 secure :-1 s=READY pgs=868 cs=0 l=1 rev1=1 crypto rx=0x7f6864004b90 tx=0x7f6864000bc0 comp rx=0 tx=0).stop 2024-03-28T18:24:55.308 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:24:55.304+0000 7f687ea1e740 1 -- 172.21.15.22:0/3443521726 shutdown_connections 2024-03-28T18:24:55.308 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:24:55.304+0000 7f687ea1e740 1 --2- 172.21.15.22:0/3443521726 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x562f94ebb460 0x562f94ebc850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:24:55.308 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:24:55.304+0000 7f687ea1e740 1 --2- 172.21.15.22:0/3443521726 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562f94eb2dd0 0x562f94ebab70 unknown :-1 s=CLOSED pgs=868 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:24:55.308 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:24:55.304+0000 7f687ea1e740 1 --2- 172.21.15.22:0/3443521726 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562f94eb20f0 0x562f94eba630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:24:55.308 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:24:55.304+0000 7f687ea1e740 1 -- 172.21.15.22:0/3443521726 >> 172.21.15.22:0/3443521726 conn(0x562f94e79200 msgr2=0x562f94eac670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:24:55.308 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:24:55.304+0000 7f687ea1e740 1 -- 172.21.15.22:0/3443521726 shutdown_connections 2024-03-28T18:24:55.308 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:24:55.304+0000 7f687ea1e740 1 -- 172.21.15.22:0/3443521726 wait complete. 2024-03-28T18:24:55.308 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:24:55.305+0000 7f687ea1e740 1 -- start start 2024-03-28T18:24:55.309 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:24:55.305+0000 7f687ea1e740 1 Processor -- start 2024-03-28T18:24:55.309 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:24:55.305+0000 7f687ea1e740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562f94eb20f0 0x562f94eb5230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:24:55.309 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:24:55.305+0000 7f687ea1e740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562f94ebb460 0x562f94eb5770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:24:55.309 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:24:55.305+0000 7f687ea1e740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x562f94eb5cb0 0x562f94ebbbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:24:55.309 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:24:55.305+0000 7f687ea1e740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68741b4dd0 con 0x562f94ebb460 2024-03-28T18:24:55.309 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:24:55.305+0000 7f687ea1e740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68741b3ab0 con 0x562f94eb5cb0 2024-03-28T18:24:55.309 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:24:55.305+0000 7f687ea1e740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68741b4110 con 0x562f94eb20f0 2024-03-28T18:24:55.309 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:24:55.305+0000 7f687ea1e740 10 client.0 _ll_register_callbacks cb 0x7f6864009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-28T18:24:55.309 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:24:55.306+0000 7f6873e00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562f94ebb460 0x562f94eb5770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:24:55.309 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:24:55.306+0000 7f6873e00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562f94ebb460 0x562f94eb5770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:41662/0 (socket says 172.21.15.22:41662) 2024-03-28T18:24:55.309 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:24:55.306+0000 7f6873e00640 1 -- 172.21.15.22:0/2036176108 learned_addr learned my addr 172.21.15.22:0/2036176108 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-28T18:24:55.309 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:24:55.306+0000 7f6879c00640 1 --2- 172.21.15.22:0/2036176108 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x562f94eb5cb0 0x562f94ebbbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:24:55.310 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:24:55.306+0000 7f6873e00640 1 -- 172.21.15.22:0/2036176108 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562f94eb20f0 msgr2=0x562f94eb5230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T18:24:55.310 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:24:55.306+0000 7f6873e00640 1 --2- 172.21.15.22:0/2036176108 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562f94eb20f0 0x562f94eb5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:24:55.310 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:24:55.306+0000 7f6873e00640 1 -- 172.21.15.22:0/2036176108 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x562f94eb5cb0 msgr2=0x562f94ebbbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T18:24:55.310 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:24:55.306+0000 7f6873e00640 1 --2- 172.21.15.22:0/2036176108 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x562f94eb5cb0 0x562f94ebbbb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:24:55.310 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:24:55.306+0000 7f6873e00640 1 -- 172.21.15.22:0/2036176108 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x562f94ebc270 con 0x562f94ebb460 2024-03-28T18:24:55.310 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:24:55.306+0000 7f6873e00640 1 --2- 172.21.15.22:0/2036176108 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562f94ebb460 0x562f94eb5770 secure :-1 s=READY pgs=869 cs=0 l=1 rev1=1 crypto rx=0x7f6864004c80 tx=0x7f6864004490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=15fb81718df0d782 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:24:55.310 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:24:55.306+0000 7f6856a00640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-03-28T18:24:55.310 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:24:55.307+0000 7f6856a00640 1 -- 172.21.15.22:0/2036176108 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6864042040 con 0x562f94ebb460 2024-03-28T18:24:55.310 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:24:55.307+0000 7f687ea1e740 1 -- 172.21.15.22:0/2036176108 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x562f94e1dae0 con 0x562f94ebb460 2024-03-28T18:24:55.310 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:24:55.307+0000 7f6856a00640 1 -- 172.21.15.22:0/2036176108 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(122..122 src has 1..122) v4 ==== 10389+0+0 (secure 0 0 0) 0x7f68640046c0 con 0x562f94ebb460 2024-03-28T18:24:55.310 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:24:55.307+0000 7f6856a00640 1 -- 172.21.15.22:0/2036176108 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=189) v2 ==== 24+0+0 (secure 0 0 0) 0x7f68640048c0 con 0x562f94ebb460 2024-03-28T18:24:55.310 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:24:55.307+0000 7f687ea1e740 10 client.17091 fetch_fsmap learned FSMap version 189 2024-03-28T18:24:55.310 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:24:55.307+0000 7f687ea1e740 1 -- 172.21.15.22:0/2036176108 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=189,monmap=4+}) v3 -- 0x562f94f27720 con 0x562f94ebb460 2024-03-28T18:24:55.311 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:24:55.307+0000 7f6856a00640 1 -- 172.21.15.22:0/2036176108 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x562f94e1dae0 con 0x562f94ebb460 2024-03-28T18:24:55.311 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:24:55.307+0000 7f6856a00640 1 -- 172.21.15.22:0/2036176108 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 189) v1 ==== 6221+0+0 (secure 0 0 0) 0x7f686403fce0 con 0x562f94ebb460 2024-03-28T18:24:55.311 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:24:55.307+0000 7f687ea1e740 10 client.17091 fetch_fsmap finished waiting for FSMap version 189 2024-03-28T18:24:55.311 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:24:55.307+0000 7f687ea1e740 10 client.17091 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-28T18:24:55.311 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:24:55.307+0000 7f687ea1e740 20 client.17091 populate_metadata read hostname 'smithi022' 2024-03-28T18:24:55.311 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:24:55.307+0000 7f687ea1e740 1 --2- 172.21.15.22:0/2036176108 >> [v2:172.21.15.79:6838/2441542443,v1:172.21.15.79:6839/2441542443] conn(0x562f94f27c80 0x562f94f2a070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:24:55.311 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:24:55.307+0000 7f687ea1e740 4 client.17091 mds_command: new command op to 34553 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-28T18:24:55.311 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:24:55.307+0000 7f687ea1e740 1 -- 172.21.15.22:0/2036176108 --> [v2:172.21.15.79:6838/2441542443,v1:172.21.15.79:6839/2441542443] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x562f94d9b3a0 con 0x562f94f27c80 2024-03-28T18:24:55.311 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:24:55.308+0000 7f6879c00640 1 --2- 172.21.15.22:0/2036176108 >> [v2:172.21.15.79:6838/2441542443,v1:172.21.15.79:6839/2441542443] conn(0x562f94f27c80 0x562f94f2a070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:24:55.312 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:24:55.309+0000 7f6879c00640 1 --2- 172.21.15.22:0/2036176108 >> [v2:172.21.15.79:6838/2441542443,v1:172.21.15.79:6839/2441542443] conn(0x562f94f27c80 0x562f94f2a070 crc :-1 s=READY pgs=350 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b964cb44ab29dc4e server_cookie=79880f7bc142b2c0 in_seq=0 out_seq=0 2024-03-28T18:24:55.312 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:24:55.309+0000 7f6856a00640 10 client.17091 ms_handle_connect on v2:172.21.15.79:6838/2441542443 2024-03-28T18:24:55.312 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:24:55.309+0000 7f6856a00640 1 -- 172.21.15.22:0/2036176108 <== mds.0 v2:172.21.15.79:6838/2441542443 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x562f94d9b3a0 con 0x562f94f27c80 2024-03-28T18:24:55.312 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:24:55.309+0000 7f6856a00640 10 client.17091 handle_command_reply: tid=0 2024-03-28T18:24:55.313 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:24:55.309+0000 7f687ea1e740 1 client.17091 shutdown 2024-03-28T18:24:55.313 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:24:55.309+0000 7f687ea1e740 1 -- 172.21.15.22:0/2036176108 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562f94ebb460 msgr2=0x562f94eb5770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:24:55.313 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:24:55.309+0000 7f687ea1e740 1 --2- 172.21.15.22:0/2036176108 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562f94ebb460 0x562f94eb5770 secure :-1 s=READY pgs=869 cs=0 l=1 rev1=1 crypto rx=0x7f6864004c80 tx=0x7f6864004490 comp rx=0 tx=0).stop 2024-03-28T18:24:55.313 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:24:55.309+0000 7f687ea1e740 1 -- 172.21.15.22:0/2036176108 shutdown_connections 2024-03-28T18:24:55.313 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:24:55.309+0000 7f687ea1e740 1 --2- 172.21.15.22:0/2036176108 >> [v2:172.21.15.79:6838/2441542443,v1:172.21.15.79:6839/2441542443] conn(0x562f94f27c80 0x562f94f2a070 crc :-1 s=READY pgs=350 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:24:55.313 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:24:55.309+0000 7f687ea1e740 1 --2- 172.21.15.22:0/2036176108 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x562f94eb5cb0 0x562f94ebbbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:24:55.313 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:24:55.309+0000 7f687ea1e740 1 --2- 172.21.15.22:0/2036176108 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562f94ebb460 0x562f94eb5770 unknown :-1 s=CLOSED pgs=869 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:24:55.313 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:24:55.309+0000 7f687ea1e740 1 --2- 172.21.15.22:0/2036176108 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562f94eb20f0 0x562f94eb5230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:24:55.313 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:24:55.309+0000 7f687ea1e740 1 -- 172.21.15.22:0/2036176108 >> 172.21.15.22:0/2036176108 conn(0x562f94e79200 msgr2=0x562f94eb6770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:24:55.313 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:24:55.309+0000 7f6856a00640 0 client.17091 ms_handle_reset on v2:172.21.15.79:6838/2441542443 2024-03-28T18:24:55.313 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:24:55.309+0000 7f687ea1e740 1 -- 172.21.15.22:0/2036176108 shutdown_connections 2024-03-28T18:24:55.313 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:24:55.309+0000 7f687ea1e740 1 -- 172.21.15.22:0/2036176108 wait complete. 2024-03-28T18:24:55.313 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:24:55.310+0000 7f687ea1e740 20 client.17091 trim_cache size 0 max 16384 2024-03-28T18:24:55.314 INFO:teuthology.orchestra.run.smithi022.stdout:{ 2024-03-28T18:24:55.314 INFO:teuthology.orchestra.run.smithi022.stdout: "status": "IDLE (waiting for more scrubs)", 2024-03-28T18:24:55.314 INFO:teuthology.orchestra.run.smithi022.stdout: "scrubs": { 2024-03-28T18:24:55.314 INFO:teuthology.orchestra.run.smithi022.stdout: "4bbbd61d-29a3-4bf6-a4f2-d5783d585de3": { 2024-03-28T18:24:55.314 INFO:teuthology.orchestra.run.smithi022.stdout: "path": "/", 2024-03-28T18:24:55.314 INFO:teuthology.orchestra.run.smithi022.stdout: "tag": "4bbbd61d-29a3-4bf6-a4f2-d5783d585de3", 2024-03-28T18:24:55.314 INFO:teuthology.orchestra.run.smithi022.stdout: "options": "recursive,force" 2024-03-28T18:24:55.314 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-03-28T18:24:55.314 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-03-28T18:24:55.314 INFO:teuthology.orchestra.run.smithi022.stdout:} 2024-03-28T18:24:55.315 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:24:55.310+0000 7f687a600640 1 -- 172.21.15.22:0/1006309048 >> [v2:172.21.15.22:6800/1357365082,v1:172.21.15.22:6801/1357365082] conn(0x7f685c0a2970 msgr2=0x7f685c0a4e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:24:55.315 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:24:55.310+0000 7f687a600640 1 --2- 172.21.15.22:0/1006309048 >> [v2:172.21.15.22:6800/1357365082,v1:172.21.15.22:6801/1357365082] conn(0x7f685c0a2970 0x7f685c0a4e30 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x562f94eb3320 tx=0x7f68680bb040 comp rx=0 tx=0).stop 2024-03-28T18:24:55.315 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:24:55.310+0000 7f687a600640 1 -- 172.21.15.22:0/1006309048 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6874107e50 msgr2=0x7f687419e470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:24:55.315 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:24:55.310+0000 7f687a600640 1 --2- 172.21.15.22:0/1006309048 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6874107e50 0x7f687419e470 secure :-1 s=READY pgs=865 cs=0 l=1 rev1=1 crypto rx=0x7f686800d770 tx=0x7f686800dc30 comp rx=0 tx=0).stop 2024-03-28T18:24:55.315 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:24:55.310+0000 7f687a600640 1 -- 172.21.15.22:0/1006309048 shutdown_connections 2024-03-28T18:24:55.315 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:24:55.310+0000 7f687a600640 1 --2- 172.21.15.22:0/1006309048 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f68741a2d30 0x7f68741a31e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:24:55.315 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:24:55.310+0000 7f687a600640 1 --2- 172.21.15.22:0/1006309048 >> [v2:172.21.15.22:6800/1357365082,v1:172.21.15.22:6801/1357365082] conn(0x7f685c0a2970 0x7f685c0a4e30 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:24:55.315 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:24:55.310+0000 7f687a600640 1 --2- 172.21.15.22:0/1006309048 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f687410cea0 0x7f687419e9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:24:55.315 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:24:55.310+0000 7f687a600640 1 --2- 172.21.15.22:0/1006309048 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6874107e50 0x7f687419e470 unknown :-1 s=CLOSED pgs=865 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:24:55.315 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:24:55.310+0000 7f687a600640 1 -- 172.21.15.22:0/1006309048 >> 172.21.15.22:0/1006309048 conn(0x7f68740774c0 msgr2=0x7f68740790b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:24:55.315 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:24:55.311+0000 7f687a600640 1 -- 172.21.15.22:0/1006309048 shutdown_connections 2024-03-28T18:24:55.316 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:24:55.311+0000 7f687a600640 1 -- 172.21.15.22:0/1006309048 wait complete. 2024-03-28T18:24:55.330 INFO:tasks.cephfs.filesystem:scrub status for tag:4bbbd61d-29a3-4bf6-a4f2-d5783d585de3 - {'path': '/', 'tag': '4bbbd61d-29a3-4bf6-a4f2-d5783d585de3', 'options': 'recursive,force'} 2024-03-28T18:24:55.925 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:24:55 smithi134 ceph-mon[35037]: pgmap v560: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:24:55.933 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:24:55 smithi079 ceph-mon[35312]: pgmap v560: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:24:56.032 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:24:55 smithi022 ceph-mon[31985]: pgmap v560: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:24:57.925 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:24:57 smithi134 ceph-mon[35037]: pgmap v561: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:24:57.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:24:57 smithi079 ceph-mon[35312]: pgmap v561: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:24:58.032 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:24:57 smithi022 ceph-mon[31985]: pgmap v561: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:24:59.925 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:24:59 smithi134 ceph-mon[35037]: pgmap v562: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:24:59.925 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:24:59 smithi134 ceph-mon[35037]: 2 slow requests, 0 included below; oldest blocked for > 486.846536 secs 2024-03-28T18:24:59.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:24:59 smithi079 ceph-mon[35312]: pgmap v562: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:24:59.933 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:24:59 smithi079 ceph-mon[35312]: 2 slow requests, 0 included below; oldest blocked for > 486.846536 secs 2024-03-28T18:25:00.032 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:24:59 smithi022 ceph-mon[31985]: pgmap v562: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:25:00.032 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:24:59 smithi022 ceph-mon[31985]: 2 slow requests, 0 included below; oldest blocked for > 486.846536 secs 2024-03-28T18:25:01.925 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:25:01 smithi134 ceph-mon[35037]: pgmap v563: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 9 op/s 2024-03-28T18:25:01.933 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:25:01 smithi079 ceph-mon[35312]: pgmap v563: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 9 op/s 2024-03-28T18:25:02.032 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:25:01 smithi022 ceph-mon[31985]: pgmap v563: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 9 op/s 2024-03-28T18:25:03.925 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:25:03 smithi134 ceph-mon[35037]: pgmap v564: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T18:25:03.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:25:03 smithi079 ceph-mon[35312]: pgmap v564: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T18:25:04.032 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:25:03 smithi022 ceph-mon[31985]: pgmap v564: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T18:25:04.925 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:25:04 smithi134 ceph-mon[35037]: 2 slow requests, 0 included below; oldest blocked for > 491.846608 secs 2024-03-28T18:25:04.933 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:25:04 smithi079 ceph-mon[35312]: 2 slow requests, 0 included below; oldest blocked for > 491.846608 secs 2024-03-28T18:25:05.032 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:25:04 smithi022 ceph-mon[31985]: 2 slow requests, 0 included below; oldest blocked for > 491.846608 secs 2024-03-28T18:25:05.925 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:25:05 smithi134 ceph-mon[35037]: pgmap v565: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 9 op/s 2024-03-28T18:25:05.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:25:05 smithi079 ceph-mon[35312]: pgmap v565: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 9 op/s 2024-03-28T18:25:06.032 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:25:05 smithi022 ceph-mon[31985]: pgmap v565: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 9 op/s 2024-03-28T18:25:07.925 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:25:07 smithi134 ceph-mon[35037]: pgmap v566: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T18:25:07.933 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:25:07 smithi079 ceph-mon[35312]: pgmap v566: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T18:25:08.032 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:25:07 smithi022 ceph-mon[31985]: pgmap v566: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T18:25:09.933 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:25:09 smithi079 ceph-mon[35312]: pgmap v567: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T18:25:09.933 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:25:09 smithi079 ceph-mon[35312]: 2 slow requests, 0 included below; oldest blocked for > 496.846688 secs 2024-03-28T18:25:10.032 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:25:09 smithi022 ceph-mon[31985]: pgmap v567: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T18:25:10.032 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:25:09 smithi022 ceph-mon[31985]: 2 slow requests, 0 included below; oldest blocked for > 496.846688 secs 2024-03-28T18:25:10.175 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:25:09 smithi134 ceph-mon[35037]: pgmap v567: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T18:25:10.175 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:25:09 smithi134 ceph-mon[35037]: 2 slow requests, 0 included below; oldest blocked for > 496.846688 secs 2024-03-28T18:25:11.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:25:11 smithi079 ceph-mon[35312]: pgmap v568: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 9 op/s 2024-03-28T18:25:12.032 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:25:11 smithi022 ceph-mon[31985]: pgmap v568: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 9 op/s 2024-03-28T18:25:12.174 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:25:11 smithi134 ceph-mon[35037]: pgmap v568: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 9 op/s 2024-03-28T18:25:14.032 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:25:13 smithi022 ceph-mon[31985]: pgmap v569: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T18:25:14.174 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:25:13 smithi134 ceph-mon[35037]: pgmap v569: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T18:25:14.182 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:25:13 smithi079 ceph-mon[35312]: pgmap v569: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T18:25:15.032 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:25:14 smithi022 ceph-mon[31985]: 2 slow requests, 0 included below; oldest blocked for > 501.846753 secs 2024-03-28T18:25:15.084 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:25:14 smithi079 ceph-mon[35312]: 2 slow requests, 0 included below; oldest blocked for > 501.846753 secs 2024-03-28T18:25:15.174 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:25:14 smithi134 ceph-mon[35037]: 2 slow requests, 0 included below; oldest blocked for > 501.846753 secs 2024-03-28T18:25:16.032 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:25:15 smithi022 ceph-mon[31985]: pgmap v570: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 9 op/s 2024-03-28T18:25:16.174 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:25:15 smithi134 ceph-mon[35037]: pgmap v570: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 9 op/s 2024-03-28T18:25:16.183 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:25:15 smithi079 ceph-mon[35312]: pgmap v570: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 9 op/s 2024-03-28T18:25:18.032 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:25:17 smithi022 ceph-mon[31985]: pgmap v571: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T18:25:18.032 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:25:17 smithi022 ceph-mon[31985]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T18:25:18.174 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:25:17 smithi134 ceph-mon[35037]: pgmap v571: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T18:25:18.175 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:25:17 smithi134 ceph-mon[35037]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T18:25:18.182 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:25:17 smithi079 ceph-mon[35312]: pgmap v571: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T18:25:18.182 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:25:17 smithi079 ceph-mon[35312]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T18:25:19.032 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:25:18 smithi022 ceph-mon[31985]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T18:25:19.032 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:25:18 smithi022 ceph-mon[31985]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T18:25:19.032 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:25:18 smithi022 ceph-mon[31985]: from='mgr.15522 ' entity='mgr.x' 2024-03-28T18:25:19.174 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:25:18 smithi134 ceph-mon[35037]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T18:25:19.175 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:25:18 smithi134 ceph-mon[35037]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T18:25:19.175 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:25:18 smithi134 ceph-mon[35037]: from='mgr.15522 ' entity='mgr.x' 2024-03-28T18:25:19.182 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:25:18 smithi079 ceph-mon[35312]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T18:25:19.183 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:25:18 smithi079 ceph-mon[35312]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T18:25:19.183 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:25:18 smithi079 ceph-mon[35312]: from='mgr.15522 ' entity='mgr.x' 2024-03-28T18:25:20.032 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:25:19 smithi022 ceph-mon[31985]: pgmap v572: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T18:25:20.032 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:25:19 smithi022 ceph-mon[31985]: pgmap v573: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 7 op/s 2024-03-28T18:25:20.032 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:25:19 smithi022 ceph-mon[31985]: 2 slow requests, 0 included below; oldest blocked for > 506.846832 secs 2024-03-28T18:25:20.175 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:25:19 smithi134 ceph-mon[35037]: pgmap v572: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T18:25:20.175 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:25:19 smithi134 ceph-mon[35037]: pgmap v573: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 7 op/s 2024-03-28T18:25:20.175 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:25:19 smithi134 ceph-mon[35037]: 2 slow requests, 0 included below; oldest blocked for > 506.846832 secs 2024-03-28T18:25:20.182 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:25:19 smithi079 ceph-mon[35312]: pgmap v572: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T18:25:20.183 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:25:19 smithi079 ceph-mon[35312]: pgmap v573: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 7 op/s 2024-03-28T18:25:20.183 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:25:19 smithi079 ceph-mon[35312]: 2 slow requests, 0 included below; oldest blocked for > 506.846832 secs 2024-03-28T18:25:22.032 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:25:21 smithi022 ceph-mon[31985]: pgmap v574: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 8 op/s 2024-03-28T18:25:22.174 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:25:21 smithi134 ceph-mon[35037]: pgmap v574: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 8 op/s 2024-03-28T18:25:22.183 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:25:21 smithi079 ceph-mon[35312]: pgmap v574: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 8 op/s 2024-03-28T18:25:24.032 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:25:23 smithi022 ceph-mon[31985]: pgmap v575: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 8 op/s 2024-03-28T18:25:24.174 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:25:23 smithi134 ceph-mon[35037]: pgmap v575: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 8 op/s 2024-03-28T18:25:24.182 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:25:23 smithi079 ceph-mon[35312]: pgmap v575: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 8 op/s 2024-03-28T18:25:25.032 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:25:24 smithi022 ceph-mon[31985]: 2 slow requests, 0 included below; oldest blocked for > 511.846858 secs 2024-03-28T18:25:25.174 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:25:24 smithi134 ceph-mon[35037]: 2 slow requests, 0 included below; oldest blocked for > 511.846858 secs 2024-03-28T18:25:25.182 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:25:24 smithi079 ceph-mon[35312]: 2 slow requests, 0 included below; oldest blocked for > 511.846858 secs 2024-03-28T18:25:25.331 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-28T18:25:25.331 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-28T18:25:25.569 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:25:25.568+0000 7f7b19a00640 1 -- 172.21.15.22:0/2864630156 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7b140743f0 msgr2=0x7f7b1410d930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:25:25.569 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:25:25.568+0000 7f7b19a00640 1 --2- 172.21.15.22:0/2864630156 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7b140743f0 0x7f7b1410d930 secure :-1 s=READY pgs=870 cs=0 l=1 rev1=1 crypto rx=0x7f7afc009900 tx=0x7f7afc02bcc0 comp rx=0 tx=0).stop 2024-03-28T18:25:25.569 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:25:25.568+0000 7f7b19a00640 1 -- 172.21.15.22:0/2864630156 shutdown_connections 2024-03-28T18:25:25.569 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:25:25.568+0000 7f7b19a00640 1 --2- 172.21.15.22:0/2864630156 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7b1410de70 0x7f7b141102c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:25:25.569 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:25:25.568+0000 7f7b19a00640 1 --2- 172.21.15.22:0/2864630156 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7b140743f0 0x7f7b1410d930 unknown :-1 s=CLOSED pgs=870 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:25:25.569 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:25:25.568+0000 7f7b19a00640 1 --2- 172.21.15.22:0/2864630156 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f7b14073ad0 0x7f7b14073eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:25:25.569 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:25:25.568+0000 7f7b19a00640 1 -- 172.21.15.22:0/2864630156 >> 172.21.15.22:0/2864630156 conn(0x7f7b14078aa0 msgr2=0x7f7b14078eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:25:25.569 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:25:25.568+0000 7f7b19a00640 1 -- 172.21.15.22:0/2864630156 shutdown_connections 2024-03-28T18:25:25.569 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:25:25.569+0000 7f7b19a00640 1 -- 172.21.15.22:0/2864630156 wait complete. 2024-03-28T18:25:25.572 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:25:25.569+0000 7f7b19a00640 1 Processor -- start 2024-03-28T18:25:25.572 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:25:25.570+0000 7f7b19a00640 1 -- start start 2024-03-28T18:25:25.572 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:25:25.570+0000 7f7b19a00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7b14073ad0 0x7f7b1419a150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:25:25.572 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:25:25.570+0000 7f7b19a00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7b140743f0 0x7f7b1419a690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:25:25.572 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:25:25.570+0000 7f7b19a00640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f7b1410de70 0x7f7b1419e9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:25:25.572 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:25:25.570+0000 7f7b19a00640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b14113c30 con 0x7f7b14073ad0 2024-03-28T18:25:25.572 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:25:25.570+0000 7f7b19a00640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b14113ab0 con 0x7f7b1410de70 2024-03-28T18:25:25.572 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:25:25.570+0000 7f7b19a00640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b14113db0 con 0x7f7b140743f0 2024-03-28T18:25:25.572 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:25:25.570+0000 7f7b12a00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7b14073ad0 0x7f7b1419a150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:25:25.572 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:25:25.570+0000 7f7b12a00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7b14073ad0 0x7f7b1419a150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:38372/0 (socket says 172.21.15.22:38372) 2024-03-28T18:25:25.572 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:25:25.570+0000 7f7b12a00640 1 -- 172.21.15.22:0/3136091420 learned_addr learned my addr 172.21.15.22:0/3136091420 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-28T18:25:25.573 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:25:25.571+0000 7f7b13400640 1 --2- 172.21.15.22:0/3136091420 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f7b1410de70 0x7f7b1419e9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:25:25.573 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:25:25.571+0000 7f7b0be00640 1 --2- 172.21.15.22:0/3136091420 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7b140743f0 0x7f7b1419a690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:25:25.573 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:25:25.571+0000 7f7b12a00640 1 -- 172.21.15.22:0/3136091420 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7b140743f0 msgr2=0x7f7b1419a690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:25:25.573 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:25:25.571+0000 7f7b12a00640 1 --2- 172.21.15.22:0/3136091420 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7b140743f0 0x7f7b1419a690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:25:25.573 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:25:25.571+0000 7f7b12a00640 1 -- 172.21.15.22:0/3136091420 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f7b1410de70 msgr2=0x7f7b1419e9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:25:25.573 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:25:25.571+0000 7f7b12a00640 1 --2- 172.21.15.22:0/3136091420 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f7b1410de70 0x7f7b1419e9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:25:25.573 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:25:25.571+0000 7f7b12a00640 1 -- 172.21.15.22:0/3136091420 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7b1419f0f0 con 0x7f7b14073ad0 2024-03-28T18:25:25.573 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:25:25.571+0000 7f7b12a00640 1 --2- 172.21.15.22:0/3136091420 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7b14073ad0 0x7f7b1419a150 secure :-1 s=READY pgs=871 cs=0 l=1 rev1=1 crypto rx=0x7f7b0400d950 tx=0x7f7b0400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:25:25.573 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:25:25.571+0000 7f7b0b400640 1 -- 172.21.15.22:0/3136091420 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b04017070 con 0x7f7b14073ad0 2024-03-28T18:25:25.573 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:25:25.571+0000 7f7b0b400640 1 -- 172.21.15.22:0/3136091420 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f7b04009c30 con 0x7f7b14073ad0 2024-03-28T18:25:25.573 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:25:25.571+0000 7f7b0b400640 1 -- 172.21.15.22:0/3136091420 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b0400f6b0 con 0x7f7b14073ad0 2024-03-28T18:25:25.573 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:25:25.572+0000 7f7b19a00640 1 -- 172.21.15.22:0/3136091420 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7b1419f3e0 con 0x7f7b14073ad0 2024-03-28T18:25:25.573 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:25:25.572+0000 7f7b19a00640 1 -- 172.21.15.22:0/3136091420 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7b141a6c70 con 0x7f7b14073ad0 2024-03-28T18:25:25.576 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:25:25.575+0000 7f7b1c78f740 1 Processor -- start 2024-03-28T18:25:25.576 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:25:25.575+0000 7f7b1c78f740 1 -- start start 2024-03-28T18:25:25.576 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:25:25.575+0000 7f7b0b400640 1 -- 172.21.15.22:0/3136091420 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 26) v1 ==== 137421+0+0 (secure 0 0 0) 0x7f7b04010a50 con 0x7f7b14073ad0 2024-03-28T18:25:25.576 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:25:25.576+0000 7f7b1c78f740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55f15948ec00 0x55f15948d950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:25:25.576 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:25:25.576+0000 7f7b1c78f740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f15948df20 0x55f15948e3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:25:25.576 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:25:25.576+0000 7f7b1c78f740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f159497290 0x55f159499680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:25:25.576 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:25:25.576+0000 7f7b1c78f740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f1593f2250 con 0x55f159497290 2024-03-28T18:25:25.577 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:25:25.576+0000 7f7b1c78f740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f15939aa30 con 0x55f15948ec00 2024-03-28T18:25:25.577 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:25:25.576+0000 7f7b1c78f740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f1593ee2f0 con 0x55f15948df20 2024-03-28T18:25:25.577 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:25:25.576+0000 7f7b0be00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f159497290 0x55f159499680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:25:25.577 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:25:25.576+0000 7f7b0be00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f159497290 0x55f159499680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:38382/0 (socket says 172.21.15.22:38382) 2024-03-28T18:25:25.577 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:25:25.576+0000 7f7b0be00640 1 -- 172.21.15.22:0/3826520490 learned_addr learned my addr 172.21.15.22:0/3826520490 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-28T18:25:25.577 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:25:25.576+0000 7f7b13400640 1 --2- 172.21.15.22:0/3826520490 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55f15948ec00 0x55f15948d950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:25:25.577 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:25:25.576+0000 7f7b12a00640 1 --2- 172.21.15.22:0/3826520490 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f15948df20 0x55f15948e3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:25:25.582 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:25:25.577+0000 7f7b0be00640 1 -- 172.21.15.22:0/3826520490 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f15948df20 msgr2=0x55f15948e3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T18:25:25.582 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:25:25.577+0000 7f7b0be00640 1 --2- 172.21.15.22:0/3826520490 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f15948df20 0x55f15948e3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:25:25.582 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:25:25.577+0000 7f7b0be00640 1 -- 172.21.15.22:0/3826520490 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55f15948ec00 msgr2=0x55f15948d950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T18:25:25.582 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:25:25.577+0000 7f7b0be00640 1 --2- 172.21.15.22:0/3826520490 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55f15948ec00 0x55f15948d950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:25:25.582 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:25:25.577+0000 7f7b0be00640 1 -- 172.21.15.22:0/3826520490 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f159385690 con 0x55f159497290 2024-03-28T18:25:25.583 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:25:25.577+0000 7f7b0b400640 1 --2- 172.21.15.22:0/3136091420 >> [v2:172.21.15.22:6800/1357365082,v1:172.21.15.22:6801/1357365082] conn(0x7f7af80a29f0 0x7f7af80a4eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:25:25.583 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:25:25.577+0000 7f7b0b400640 1 -- 172.21.15.22:0/3136091420 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(122..122 src has 1..122) v4 ==== 10389+0+0 (secure 0 0 0) 0x7f7b040cf290 con 0x7f7b14073ad0 2024-03-28T18:25:25.583 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:25:25.577+0000 7f7b0be00640 1 --2- 172.21.15.22:0/3826520490 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f159497290 0x55f159499680 secure :-1 s=READY pgs=872 cs=0 l=1 rev1=1 crypto rx=0x7f7afc009900 tx=0x7f7afc004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eb383adfc4652684 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:25:25.583 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:25:25.577+0000 7f7b13400640 1 --2- 172.21.15.22:0/3136091420 >> [v2:172.21.15.22:6800/1357365082,v1:172.21.15.22:6801/1357365082] conn(0x7f7af80a29f0 0x7f7af80a4eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:25:25.583 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:25:25.577+0000 7f7b03e00640 1 -- 172.21.15.22:0/3826520490 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7afc004430 con 0x55f159497290 2024-03-28T18:25:25.583 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:25:25.578+0000 7f7b03e00640 1 -- 172.21.15.22:0/3826520490 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f7afc005510 con 0x55f159497290 2024-03-28T18:25:25.583 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:25:25.578+0000 7f7b03e00640 1 -- 172.21.15.22:0/3826520490 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7afc02c860 con 0x55f159497290 2024-03-28T18:25:25.583 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:25:25.578+0000 7f7b1c78f740 1 -- 172.21.15.22:0/3826520490 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f159497290 msgr2=0x55f159499680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:25:25.583 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:25:25.578+0000 7f7b1c78f740 1 --2- 172.21.15.22:0/3826520490 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f159497290 0x55f159499680 secure :-1 s=READY pgs=872 cs=0 l=1 rev1=1 crypto rx=0x7f7afc009900 tx=0x7f7afc004040 comp rx=0 tx=0).stop 2024-03-28T18:25:25.583 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:25:25.578+0000 7f7b13400640 1 --2- 172.21.15.22:0/3136091420 >> [v2:172.21.15.22:6800/1357365082,v1:172.21.15.22:6801/1357365082] conn(0x7f7af80a29f0 0x7f7af80a4eb0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f7b0c005fd0 tx=0x7f7b0c0096e0 comp rx=0 tx=0).ready entity=mgr.15522 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:25:25.583 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:25:25.578+0000 7f7b1c78f740 1 -- 172.21.15.22:0/3826520490 shutdown_connections 2024-03-28T18:25:25.583 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:25:25.578+0000 7f7b1c78f740 1 --2- 172.21.15.22:0/3826520490 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f159497290 0x55f159499680 unknown :-1 s=CLOSED pgs=872 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:25:25.584 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:25:25.578+0000 7f7b1c78f740 1 --2- 172.21.15.22:0/3826520490 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f15948df20 0x55f15948e3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:25:25.584 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:25:25.578+0000 7f7b1c78f740 1 --2- 172.21.15.22:0/3826520490 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55f15948ec00 0x55f15948d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:25:25.584 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:25:25.578+0000 7f7b1c78f740 1 -- 172.21.15.22:0/3826520490 >> 172.21.15.22:0/3826520490 conn(0x55f159455200 msgr2=0x55f159488970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:25:25.584 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:25:25.579+0000 7f7b1c78f740 1 -- 172.21.15.22:0/3826520490 shutdown_connections 2024-03-28T18:25:25.584 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:25:25.579+0000 7f7b1c78f740 1 -- 172.21.15.22:0/3826520490 wait complete. 2024-03-28T18:25:25.584 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:25:25.579+0000 7f7b1c78f740 1 -- start start 2024-03-28T18:25:25.584 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:25:25.579+0000 7f7b1c78f740 1 Processor -- start 2024-03-28T18:25:25.584 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:25:25.580+0000 7f7b1c78f740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55f15948df20 0x55f159496840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:25:25.584 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:25:25.580+0000 7f7b1c78f740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f15948ec00 0x55f159493bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:25:25.584 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:25:25.580+0000 7f7b1c78f740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f159497290 0x55f159494110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:25:25.584 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:25:25.580+0000 7f7b1c78f740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f15949a0f0 con 0x55f15948ec00 2024-03-28T18:25:25.584 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:25:25.580+0000 7f7b1c78f740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f1593ade80 con 0x55f15948df20 2024-03-28T18:25:25.584 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:25:25.580+0000 7f7b1c78f740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f15949a270 con 0x55f159497290 2024-03-28T18:25:25.585 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:25:25.580+0000 7f7b1c78f740 10 client.0 _ll_register_callbacks cb 0x55f1593c0130 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-28T18:25:25.585 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:25:25.580+0000 7f7b0be00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f15948ec00 0x55f159493bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:25:25.585 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:25:25.580+0000 7f7b0be00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f15948ec00 0x55f159493bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:38384/0 (socket says 172.21.15.22:38384) 2024-03-28T18:25:25.585 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:25:25.580+0000 7f7b0be00640 1 -- 172.21.15.22:0/903420335 learned_addr learned my addr 172.21.15.22:0/903420335 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-28T18:25:25.585 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:25:25.580+0000 7f7b12a00640 1 --2- 172.21.15.22:0/903420335 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55f15948df20 0x55f159496840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:25:25.585 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:25:25.580+0000 7f7b13400640 1 --2- 172.21.15.22:0/903420335 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f159497290 0x55f159494110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:25:25.585 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:25:25.580+0000 7f7b0be00640 1 -- 172.21.15.22:0/903420335 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f159497290 msgr2=0x55f159494110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T18:25:25.585 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:25:25.580+0000 7f7b0be00640 1 --2- 172.21.15.22:0/903420335 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f159497290 0x55f159494110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:25:25.585 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:25:25.580+0000 7f7b0be00640 1 -- 172.21.15.22:0/903420335 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55f15948df20 msgr2=0x55f159496840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T18:25:25.585 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:25:25.580+0000 7f7b0be00640 1 --2- 172.21.15.22:0/903420335 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55f15948df20 0x55f159496840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:25:25.585 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:25:25.580+0000 7f7b0be00640 1 -- 172.21.15.22:0/903420335 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55f1591d07d0 con 0x55f15948ec00 2024-03-28T18:25:25.585 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:25:25.581+0000 7f7b0be00640 1 --2- 172.21.15.22:0/903420335 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f15948ec00 0x55f159493bd0 secure :-1 s=READY pgs=873 cs=0 l=1 rev1=1 crypto rx=0x7f7afc02c2d0 tx=0x7f7afc0048f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6a64f992ece812df server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:25:25.585 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:25:25.581+0000 7f7b00c00640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-03-28T18:25:25.585 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:25:25.581+0000 7f7b00c00640 1 -- 172.21.15.22:0/903420335 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7afc004a90 con 0x55f15948ec00 2024-03-28T18:25:25.586 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:25:25.581+0000 7f7b00c00640 1 -- 172.21.15.22:0/903420335 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(122..122 src has 1..122) v4 ==== 10389+0+0 (secure 0 0 0) 0x7f7afc005040 con 0x55f15948ec00 2024-03-28T18:25:25.586 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:25:25.581+0000 7f7b1c78f740 1 -- 172.21.15.22:0/903420335 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f7afc005510 con 0x55f15948ec00 2024-03-28T18:25:25.586 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:25:25.581+0000 7f7b00c00640 1 -- 172.21.15.22:0/903420335 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=189) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7afc005240 con 0x55f15948ec00 2024-03-28T18:25:25.586 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:25:25.581+0000 7f7b1c78f740 10 client.17103 fetch_fsmap learned FSMap version 189 2024-03-28T18:25:25.586 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:25:25.581+0000 7f7b1c78f740 1 -- 172.21.15.22:0/903420335 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=189,monmap=4+}) v3 -- 0x55f1591063f0 con 0x55f15948ec00 2024-03-28T18:25:25.586 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:25:25.582+0000 7f7b00c00640 1 -- 172.21.15.22:0/903420335 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f7afc005510 con 0x55f15948ec00 2024-03-28T18:25:25.586 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:25:25.582+0000 7f7b00c00640 1 -- 172.21.15.22:0/903420335 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 189) v1 ==== 6221+0+0 (secure 0 0 0) 0x7f7afc054a00 con 0x55f15948ec00 2024-03-28T18:25:25.586 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:25:25.582+0000 7f7b1c78f740 10 client.17103 fetch_fsmap finished waiting for FSMap version 189 2024-03-28T18:25:25.586 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:25:25.582+0000 7f7b1c78f740 10 client.17103 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-28T18:25:25.586 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:25:25.582+0000 7f7b1c78f740 20 client.17103 populate_metadata read hostname 'smithi022' 2024-03-28T18:25:25.586 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:25:25.582+0000 7f7b1c78f740 1 --2- 172.21.15.22:0/903420335 >> [v2:172.21.15.79:6838/2441542443,v1:172.21.15.79:6839/2441542443] conn(0x55f1594a6f50 0x55f1594a9340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:25:25.586 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:25:25.582+0000 7f7b1c78f740 4 client.17103 mds_command: new command op to 34553 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-28T18:25:25.586 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:25:25.582+0000 7f7b1c78f740 1 -- 172.21.15.22:0/903420335 --> [v2:172.21.15.79:6838/2441542443,v1:172.21.15.79:6839/2441542443] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55f1593ff790 con 0x55f1594a6f50 2024-03-28T18:25:25.588 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:25:25.583+0000 7f7b12a00640 1 --2- 172.21.15.22:0/903420335 >> [v2:172.21.15.79:6838/2441542443,v1:172.21.15.79:6839/2441542443] conn(0x55f1594a6f50 0x55f1594a9340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:25:25.588 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:25:25.584+0000 7f7b12a00640 1 --2- 172.21.15.22:0/903420335 >> [v2:172.21.15.79:6838/2441542443,v1:172.21.15.79:6839/2441542443] conn(0x55f1594a6f50 0x55f1594a9340 crc :-1 s=READY pgs=353 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=76194a66146374de server_cookie=ce0010769474c98c in_seq=0 out_seq=0 2024-03-28T18:25:25.588 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:25:25.584+0000 7f7b00c00640 10 client.17103 ms_handle_connect on v2:172.21.15.79:6838/2441542443 2024-03-28T18:25:25.588 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:25:25.586+0000 7f7b00c00640 1 -- 172.21.15.22:0/903420335 <== mds.0 v2:172.21.15.79:6838/2441542443 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x55f1593ff790 con 0x55f1594a6f50 2024-03-28T18:25:25.588 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:25:25.586+0000 7f7b00c00640 10 client.17103 handle_command_reply: tid=0 2024-03-28T18:25:25.588 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:25:25.587+0000 7f7b1c78f740 1 client.17103 shutdown 2024-03-28T18:25:25.588 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:25:25.587+0000 7f7b1c78f740 1 -- 172.21.15.22:0/903420335 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f15948ec00 msgr2=0x55f159493bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:25:25.588 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:25:25.587+0000 7f7b1c78f740 1 --2- 172.21.15.22:0/903420335 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f15948ec00 0x55f159493bd0 secure :-1 s=READY pgs=873 cs=0 l=1 rev1=1 crypto rx=0x7f7afc02c2d0 tx=0x7f7afc0048f0 comp rx=0 tx=0).stop 2024-03-28T18:25:25.589 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:25:25.587+0000 7f7b1c78f740 1 -- 172.21.15.22:0/903420335 shutdown_connections 2024-03-28T18:25:25.589 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:25:25.587+0000 7f7b1c78f740 1 --2- 172.21.15.22:0/903420335 >> [v2:172.21.15.79:6838/2441542443,v1:172.21.15.79:6839/2441542443] conn(0x55f1594a6f50 0x55f1594a9340 crc :-1 s=READY pgs=353 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:25:25.589 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:25:25.587+0000 7f7b1c78f740 1 --2- 172.21.15.22:0/903420335 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f159497290 0x55f159494110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:25:25.589 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:25:25.587+0000 7f7b1c78f740 1 --2- 172.21.15.22:0/903420335 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f15948ec00 0x55f159493bd0 unknown :-1 s=CLOSED pgs=873 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:25:25.589 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:25:25.587+0000 7f7b00c00640 0 client.17103 ms_handle_reset on v2:172.21.15.79:6838/2441542443 2024-03-28T18:25:25.589 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:25:25.587+0000 7f7b1c78f740 1 --2- 172.21.15.22:0/903420335 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55f15948df20 0x55f159496840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:25:25.589 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:25:25.587+0000 7f7b1c78f740 1 -- 172.21.15.22:0/903420335 >> 172.21.15.22:0/903420335 conn(0x55f159455200 msgr2=0x55f15948a6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:25:25.589 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:25:25.587+0000 7f7b1c78f740 1 -- 172.21.15.22:0/903420335 shutdown_connections 2024-03-28T18:25:25.589 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:25:25.587+0000 7f7b1c78f740 1 -- 172.21.15.22:0/903420335 wait complete. 2024-03-28T18:25:25.589 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:25:25.587+0000 7f7b1c78f740 20 client.17103 trim_cache size 0 max 16384 2024-03-28T18:25:25.606 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:25:25.605+0000 7f7b1c78f740 1 Processor -- start 2024-03-28T18:25:25.606 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:25:25.605+0000 7f7b1c78f740 1 -- start start 2024-03-28T18:25:25.606 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:25:25.605+0000 7f7b1c78f740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55f15948df20 0x55f159498680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:25:25.606 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:25:25.605+0000 7f7b1c78f740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f15948ec00 0x55f159498bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:25:25.606 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:25:25.605+0000 7f7b1c78f740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f159497290 0x55f159496300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:25:25.606 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:25:25.605+0000 7f7b1c78f740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b141b1ea0 con 0x55f15948ec00 2024-03-28T18:25:25.606 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:25:25.605+0000 7f7b1c78f740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b141af200 con 0x55f15948df20 2024-03-28T18:25:25.607 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:25:25.605+0000 7f7b1c78f740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b141b11e0 con 0x55f159497290 2024-03-28T18:25:25.607 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:25:25.605+0000 7f7b0be00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f15948ec00 0x55f159498bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:25:25.607 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:25:25.605+0000 7f7b0be00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f15948ec00 0x55f159498bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:38396/0 (socket says 172.21.15.22:38396) 2024-03-28T18:25:25.607 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:25:25.605+0000 7f7b0be00640 1 -- 172.21.15.22:0/1180288220 learned_addr learned my addr 172.21.15.22:0/1180288220 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-28T18:25:25.607 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:25:25.606+0000 7f7b12a00640 1 --2- 172.21.15.22:0/1180288220 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55f15948df20 0x55f159498680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:25:25.607 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:25:25.606+0000 7f7b13400640 1 --2- 172.21.15.22:0/1180288220 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f159497290 0x55f159496300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:25:25.607 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:25:25.606+0000 7f7b0be00640 1 -- 172.21.15.22:0/1180288220 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f159497290 msgr2=0x55f159496300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T18:25:25.607 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:25:25.606+0000 7f7b0be00640 1 --2- 172.21.15.22:0/1180288220 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f159497290 0x55f159496300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:25:25.607 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:25:25.606+0000 7f7b0be00640 1 -- 172.21.15.22:0/1180288220 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55f15948df20 msgr2=0x55f159498680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T18:25:25.607 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:25:25.606+0000 7f7b0be00640 1 --2- 172.21.15.22:0/1180288220 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55f15948df20 0x55f159498680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:25:25.607 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:25:25.606+0000 7f7b0be00640 1 -- 172.21.15.22:0/1180288220 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f1590c60f0 con 0x55f15948ec00 2024-03-28T18:25:25.608 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:25:25.606+0000 7f7b13400640 1 --2- 172.21.15.22:0/1180288220 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f159497290 0x55f159496300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T18:25:25.608 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:25:25.606+0000 7f7b0be00640 1 --2- 172.21.15.22:0/1180288220 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f15948ec00 0x55f159498bc0 secure :-1 s=READY pgs=874 cs=0 l=1 rev1=1 crypto rx=0x7f7afc056040 tx=0x7f7afc004dc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e63914f3c23c672c server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:25:25.611 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:25:25.606+0000 7f7b03400640 1 -- 172.21.15.22:0/1180288220 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7afc005040 con 0x55f15948ec00 2024-03-28T18:25:25.611 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:25:25.606+0000 7f7b03400640 1 -- 172.21.15.22:0/1180288220 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f7afc02cb30 con 0x55f15948ec00 2024-03-28T18:25:25.611 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:25:25.606+0000 7f7b03400640 1 -- 172.21.15.22:0/1180288220 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7afc0541f0 con 0x55f15948ec00 2024-03-28T18:25:25.611 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:25:25.607+0000 7f7b1c78f740 1 -- 172.21.15.22:0/1180288220 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f15948ec00 msgr2=0x55f159498bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:25:25.611 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:25:25.607+0000 7f7b1c78f740 1 --2- 172.21.15.22:0/1180288220 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f15948ec00 0x55f159498bc0 secure :-1 s=READY pgs=874 cs=0 l=1 rev1=1 crypto rx=0x7f7afc056040 tx=0x7f7afc004dc0 comp rx=0 tx=0).stop 2024-03-28T18:25:25.612 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:25:25.607+0000 7f7b1c78f740 1 -- 172.21.15.22:0/1180288220 shutdown_connections 2024-03-28T18:25:25.612 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:25:25.607+0000 7f7b1c78f740 1 --2- 172.21.15.22:0/1180288220 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f159497290 0x55f159496300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:25:25.612 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:25:25.607+0000 7f7b1c78f740 1 --2- 172.21.15.22:0/1180288220 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f15948ec00 0x55f159498bc0 unknown :-1 s=CLOSED pgs=874 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:25:25.612 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:25:25.607+0000 7f7b1c78f740 1 --2- 172.21.15.22:0/1180288220 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55f15948df20 0x55f159498680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:25:25.612 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:25:25.607+0000 7f7b1c78f740 1 -- 172.21.15.22:0/1180288220 >> 172.21.15.22:0/1180288220 conn(0x55f159455200 msgr2=0x55f1594884a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:25:25.612 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:25:25.607+0000 7f7b1c78f740 1 -- 172.21.15.22:0/1180288220 shutdown_connections 2024-03-28T18:25:25.612 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:25:25.607+0000 7f7b1c78f740 1 -- 172.21.15.22:0/1180288220 wait complete. 2024-03-28T18:25:25.612 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:25:25.608+0000 7f7b1c78f740 1 -- start start 2024-03-28T18:25:25.612 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:25:25.608+0000 7f7b1c78f740 1 Processor -- start 2024-03-28T18:25:25.612 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:25:25.608+0000 7f7b1c78f740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f15948df20 0x55f159490460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:25:25.612 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:25:25.608+0000 7f7b1c78f740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f15948ec00 0x55f1594909a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:25:25.613 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:25:25.608+0000 7f7b1c78f740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55f159497290 0x55f159490ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:25:25.613 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:25:25.608+0000 7f7b1c78f740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b141b0b80 con 0x55f15948df20 2024-03-28T18:25:25.613 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:25:25.608+0000 7f7b1c78f740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b141af860 con 0x55f159497290 2024-03-28T18:25:25.613 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:25:25.608+0000 7f7b1c78f740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b141afec0 con 0x55f15948ec00 2024-03-28T18:25:25.613 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:25:25.608+0000 7f7b1c78f740 10 client.0 _ll_register_callbacks cb 0x7f7b0c009580 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-28T18:25:25.613 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:25:25.609+0000 7f7b12a00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f15948df20 0x55f159490460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:25:25.613 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:25:25.609+0000 7f7b12a00640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f15948df20 0x55f159490460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:38412/0 (socket says 172.21.15.22:38412) 2024-03-28T18:25:25.613 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:25:25.609+0000 7f7b12a00640 1 -- 172.21.15.22:0/525900751 learned_addr learned my addr 172.21.15.22:0/525900751 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-28T18:25:25.613 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:25:25.609+0000 7f7b13400640 1 --2- 172.21.15.22:0/525900751 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55f159497290 0x55f159490ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:25:25.613 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:25:25.609+0000 7f7b0be00640 1 --2- 172.21.15.22:0/525900751 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f15948ec00 0x55f1594909a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:25:25.613 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:25:25.609+0000 7f7b12a00640 1 -- 172.21.15.22:0/525900751 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f15948ec00 msgr2=0x55f1594909a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T18:25:25.613 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:25:25.609+0000 7f7b12a00640 1 --2- 172.21.15.22:0/525900751 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f15948ec00 0x55f1594909a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:25:25.613 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:25:25.609+0000 7f7b12a00640 1 -- 172.21.15.22:0/525900751 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55f159497290 msgr2=0x55f159490ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T18:25:25.613 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:25:25.609+0000 7f7b12a00640 1 --2- 172.21.15.22:0/525900751 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55f159497290 0x55f159490ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:25:25.614 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:25:25.609+0000 7f7b12a00640 1 -- 172.21.15.22:0/525900751 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55f159501720 con 0x55f15948df20 2024-03-28T18:25:25.614 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:25:25.609+0000 7f7b12a00640 1 --2- 172.21.15.22:0/525900751 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f15948df20 0x55f159490460 secure :-1 s=READY pgs=875 cs=0 l=1 rev1=1 crypto rx=0x7f7b0409b0d0 tx=0x7f7b040bb040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=75f7c1a7e1b8b3d0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T18:25:25.614 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:25:25.609+0000 7f7b00c00640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-03-28T18:25:25.614 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:25:25.610+0000 7f7b00c00640 1 -- 172.21.15.22:0/525900751 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b04020040 con 0x55f15948df20 2024-03-28T18:25:25.614 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:25:25.610+0000 7f7b00c00640 1 -- 172.21.15.22:0/525900751 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(122..122 src has 1..122) v4 ==== 10389+0+0 (secure 0 0 0) 0x7f7b040cc4b0 con 0x55f15948df20 2024-03-28T18:25:25.614 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:25:25.610+0000 7f7b1c78f740 1 -- 172.21.15.22:0/525900751 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f7afc02cb30 con 0x55f15948df20 2024-03-28T18:25:25.614 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:25:25.610+0000 7f7b00c00640 1 -- 172.21.15.22:0/525900751 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=189) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7b040cc6b0 con 0x55f15948df20 2024-03-28T18:25:25.614 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:25:25.610+0000 7f7b1c78f740 10 client.17109 fetch_fsmap learned FSMap version 189 2024-03-28T18:25:25.614 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:25:25.610+0000 7f7b1c78f740 1 -- 172.21.15.22:0/525900751 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=189,monmap=4+}) v3 -- 0x55f1595029a0 con 0x55f15948df20 2024-03-28T18:25:25.614 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:25:25.610+0000 7f7b00c00640 1 -- 172.21.15.22:0/525900751 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f7afc02cb30 con 0x55f15948df20 2024-03-28T18:25:25.614 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:25:25.610+0000 7f7b00c00640 1 -- 172.21.15.22:0/525900751 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 189) v1 ==== 6221+0+0 (secure 0 0 0) 0x7f7b04096e90 con 0x55f15948df20 2024-03-28T18:25:25.614 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:25:25.610+0000 7f7b1c78f740 10 client.17109 fetch_fsmap finished waiting for FSMap version 189 2024-03-28T18:25:25.614 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:25:25.610+0000 7f7b1c78f740 10 client.17109 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-28T18:25:25.614 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:25:25.610+0000 7f7b1c78f740 20 client.17109 populate_metadata read hostname 'smithi022' 2024-03-28T18:25:25.615 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:25:25.610+0000 7f7b1c78f740 1 --2- 172.21.15.22:0/525900751 >> [v2:172.21.15.79:6838/2441542443,v1:172.21.15.79:6839/2441542443] conn(0x55f1595030f0 0x55f159505510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T18:25:25.615 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:25:25.610+0000 7f7b1c78f740 4 client.17109 mds_command: new command op to 34553 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-28T18:25:25.615 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:25:25.610+0000 7f7b1c78f740 1 -- 172.21.15.22:0/525900751 --> [v2:172.21.15.79:6838/2441542443,v1:172.21.15.79:6839/2441542443] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55f1593f9ae0 con 0x55f1595030f0 2024-03-28T18:25:25.615 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:25:25.611+0000 7f7b0be00640 1 --2- 172.21.15.22:0/525900751 >> [v2:172.21.15.79:6838/2441542443,v1:172.21.15.79:6839/2441542443] conn(0x55f1595030f0 0x55f159505510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T18:25:25.616 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:25:25.612+0000 7f7b0be00640 1 --2- 172.21.15.22:0/525900751 >> [v2:172.21.15.79:6838/2441542443,v1:172.21.15.79:6839/2441542443] conn(0x55f1595030f0 0x55f159505510 crc :-1 s=READY pgs=354 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8c6e3f4782423cc2 server_cookie=1d11195126e66587 in_seq=0 out_seq=0 2024-03-28T18:25:25.616 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:25:25.612+0000 7f7b00c00640 10 client.17109 ms_handle_connect on v2:172.21.15.79:6838/2441542443 2024-03-28T18:25:25.616 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:25:25.612+0000 7f7b00c00640 1 -- 172.21.15.22:0/525900751 <== mds.0 v2:172.21.15.79:6838/2441542443 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x55f1593f9ae0 con 0x55f1595030f0 2024-03-28T18:25:25.616 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:25:25.612+0000 7f7b00c00640 10 client.17109 handle_command_reply: tid=0 2024-03-28T18:25:25.616 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:25:25.612+0000 7f7b1c78f740 1 client.17109 shutdown 2024-03-28T18:25:25.616 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:25:25.613+0000 7f7b1c78f740 1 -- 172.21.15.22:0/525900751 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f15948df20 msgr2=0x55f159490460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:25:25.616 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:25:25.613+0000 7f7b1c78f740 1 --2- 172.21.15.22:0/525900751 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f15948df20 0x55f159490460 secure :-1 s=READY pgs=875 cs=0 l=1 rev1=1 crypto rx=0x7f7b0409b0d0 tx=0x7f7b040bb040 comp rx=0 tx=0).stop 2024-03-28T18:25:25.616 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:25:25.613+0000 7f7b1c78f740 1 -- 172.21.15.22:0/525900751 shutdown_connections 2024-03-28T18:25:25.617 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:25:25.613+0000 7f7b1c78f740 1 --2- 172.21.15.22:0/525900751 >> [v2:172.21.15.79:6838/2441542443,v1:172.21.15.79:6839/2441542443] conn(0x55f1595030f0 0x55f159505510 crc :-1 s=READY pgs=354 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:25:25.617 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:25:25.613+0000 7f7b1c78f740 1 --2- 172.21.15.22:0/525900751 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55f159497290 0x55f159490ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:25:25.617 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:25:25.613+0000 7f7b1c78f740 1 --2- 172.21.15.22:0/525900751 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f15948ec00 0x55f1594909a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:25:25.617 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:25:25.613+0000 7f7b1c78f740 1 --2- 172.21.15.22:0/525900751 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f15948df20 0x55f159490460 unknown :-1 s=CLOSED pgs=875 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:25:25.617 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:25:25.613+0000 7f7b1c78f740 1 -- 172.21.15.22:0/525900751 >> 172.21.15.22:0/525900751 conn(0x55f159455200 msgr2=0x55f1594923d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:25:25.617 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:25:25.613+0000 7f7b00c00640 0 client.17109 ms_handle_reset on v2:172.21.15.79:6838/2441542443 2024-03-28T18:25:25.617 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:25:25.613+0000 7f7b1c78f740 1 -- 172.21.15.22:0/525900751 shutdown_connections 2024-03-28T18:25:25.617 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:25:25.613+0000 7f7b1c78f740 1 -- 172.21.15.22:0/525900751 wait complete. 2024-03-28T18:25:25.617 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:25:25.613+0000 7f7b1c78f740 20 client.17109 trim_cache size 0 max 16384 2024-03-28T18:25:25.617 INFO:teuthology.orchestra.run.smithi022.stdout:{ 2024-03-28T18:25:25.617 INFO:teuthology.orchestra.run.smithi022.stdout: "status": "IDLE (waiting for more scrubs)", 2024-03-28T18:25:25.617 INFO:teuthology.orchestra.run.smithi022.stdout: "scrubs": { 2024-03-28T18:25:25.618 INFO:teuthology.orchestra.run.smithi022.stdout: "4bbbd61d-29a3-4bf6-a4f2-d5783d585de3": { 2024-03-28T18:25:25.618 INFO:teuthology.orchestra.run.smithi022.stdout: "path": "/", 2024-03-28T18:25:25.618 INFO:teuthology.orchestra.run.smithi022.stdout: "tag": "4bbbd61d-29a3-4bf6-a4f2-d5783d585de3", 2024-03-28T18:25:25.618 INFO:teuthology.orchestra.run.smithi022.stdout: "options": "recursive,force" 2024-03-28T18:25:25.618 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-03-28T18:25:25.618 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-03-28T18:25:25.618 INFO:teuthology.orchestra.run.smithi022.stdout:} 2024-03-28T18:25:25.619 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:25:25.614+0000 7f7b19a00640 1 -- 172.21.15.22:0/3136091420 >> [v2:172.21.15.22:6800/1357365082,v1:172.21.15.22:6801/1357365082] conn(0x7f7af80a29f0 msgr2=0x7f7af80a4eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:25:25.619 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:25:25.614+0000 7f7b19a00640 1 --2- 172.21.15.22:0/3136091420 >> [v2:172.21.15.22:6800/1357365082,v1:172.21.15.22:6801/1357365082] conn(0x7f7af80a29f0 0x7f7af80a4eb0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f7b0c005fd0 tx=0x7f7b0c0096e0 comp rx=0 tx=0).stop 2024-03-28T18:25:25.619 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:25:25.614+0000 7f7b19a00640 1 -- 172.21.15.22:0/3136091420 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7b14073ad0 msgr2=0x7f7b1419a150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T18:25:25.619 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:25:25.614+0000 7f7b19a00640 1 --2- 172.21.15.22:0/3136091420 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7b14073ad0 0x7f7b1419a150 secure :-1 s=READY pgs=871 cs=0 l=1 rev1=1 crypto rx=0x7f7b0400d950 tx=0x7f7b0400de10 comp rx=0 tx=0).stop 2024-03-28T18:25:25.619 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:25:25.614+0000 7f7b19a00640 1 -- 172.21.15.22:0/3136091420 shutdown_connections 2024-03-28T18:25:25.619 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:25:25.614+0000 7f7b19a00640 1 --2- 172.21.15.22:0/3136091420 >> [v2:172.21.15.22:6800/1357365082,v1:172.21.15.22:6801/1357365082] conn(0x7f7af80a29f0 0x7f7af80a4eb0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:25:25.619 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:25:25.614+0000 7f7b19a00640 1 --2- 172.21.15.22:0/3136091420 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f7b1410de70 0x7f7b1419e9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:25:25.619 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:25:25.614+0000 7f7b19a00640 1 --2- 172.21.15.22:0/3136091420 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7b140743f0 0x7f7b1419a690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:25:25.619 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:25:25.614+0000 7f7b19a00640 1 --2- 172.21.15.22:0/3136091420 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7b14073ad0 0x7f7b1419a150 unknown :-1 s=CLOSED pgs=871 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T18:25:25.619 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:25:25.614+0000 7f7b19a00640 1 -- 172.21.15.22:0/3136091420 >> 172.21.15.22:0/3136091420 conn(0x7f7b14078aa0 msgr2=0x7f7b140fe560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T18:25:25.619 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:25:25.614+0000 7f7b19a00640 1 -- 172.21.15.22:0/3136091420 shutdown_connections 2024-03-28T18:25:25.619 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-28T18:25:25.614+0000 7f7b19a00640 1 -- 172.21.15.22:0/3136091420 wait complete. 2024-03-28T18:25:25.631 INFO:tasks.cephfs.filesystem:scrub status for tag:4bbbd61d-29a3-4bf6-a4f2-d5783d585de3 - {'path': '/', 'tag': '4bbbd61d-29a3-4bf6-a4f2-d5783d585de3', 'options': 'recursive,force'} 2024-03-28T18:25:25.631 ERROR:tasks.fwd_scrub.fs.[cephfs]:exception: Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_ceph-c_3be9a5c9bc793e11e2800a8c0c696e8b46742033/qa/tasks/fwd_scrub.py", line 38, in _run self.do_scrub() File "/home/teuthworker/src/git.ceph.com_ceph-c_3be9a5c9bc793e11e2800a8c0c696e8b46742033/qa/tasks/fwd_scrub.py", line 55, in do_scrub self._scrub() File "/home/teuthworker/src/git.ceph.com_ceph-c_3be9a5c9bc793e11e2800a8c0c696e8b46742033/qa/tasks/fwd_scrub.py", line 76, in _scrub done = self.fs.wait_until_scrub_complete(tag=tag, sleep=30, timeout=self.scrub_timeout) File "/home/teuthworker/src/git.ceph.com_ceph-c_3be9a5c9bc793e11e2800a8c0c696e8b46742033/qa/tasks/cephfs/filesystem.py", line 1749, in wait_until_scrub_complete while proceed(): File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/contextutil.py", line 134, in __call__ raise MaxWhileTries(error_msg) teuthology.exceptions.MaxWhileTries: reached maximum tries (31) after waiting for 900 seconds 2024-03-28T18:25:26.032 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:25:25 smithi022 ceph-mon[31985]: pgmap v576: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-03-28T18:25:26.174 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:25:25 smithi134 ceph-mon[35037]: pgmap v576: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-03-28T18:25:26.182 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:25:25 smithi079 ceph-mon[35312]: pgmap v576: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-03-28T18:25:28.032 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:25:27 smithi022 ceph-mon[31985]: pgmap v577: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-03-28T18:25:28.175 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:25:27 smithi134 ceph-mon[35037]: pgmap v577: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-03-28T18:25:28.182 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:25:27 smithi079 ceph-mon[35312]: pgmap v577: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-03-28T18:25:30.032 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:25:29 smithi022 ceph-mon[31985]: pgmap v578: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 7 op/s 2024-03-28T18:25:30.032 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:25:29 smithi022 ceph-mon[31985]: 2 slow requests, 0 included below; oldest blocked for > 516.846933 secs 2024-03-28T18:25:30.175 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:25:29 smithi134 ceph-mon[35037]: pgmap v578: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 7 op/s 2024-03-28T18:25:30.175 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:25:29 smithi134 ceph-mon[35037]: 2 slow requests, 0 included below; oldest blocked for > 516.846933 secs 2024-03-28T18:25:30.182 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:25:29 smithi079 ceph-mon[35312]: pgmap v578: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 7 op/s 2024-03-28T18:25:30.182 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:25:29 smithi079 ceph-mon[35312]: 2 slow requests, 0 included below; oldest blocked for > 516.846933 secs 2024-03-28T18:25:32.032 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:25:31 smithi022 ceph-mon[31985]: pgmap v579: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:25:32.174 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:25:31 smithi134 ceph-mon[35037]: pgmap v579: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:25:32.183 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:25:31 smithi079 ceph-mon[35312]: pgmap v579: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:25:34.032 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:25:33 smithi022 ceph-mon[31985]: pgmap v580: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:25:34.174 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:25:33 smithi134 ceph-mon[35037]: pgmap v580: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:25:34.182 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:25:33 smithi079 ceph-mon[35312]: pgmap v580: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:25:35.076 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:25:34 smithi134 ceph-mon[35037]: 2 slow requests, 0 included below; oldest blocked for > 521.847054 secs 2024-03-28T18:25:35.183 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:25:34 smithi079 ceph-mon[35312]: 2 slow requests, 0 included below; oldest blocked for > 521.847054 secs 2024-03-28T18:25:35.282 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:25:34 smithi022 ceph-mon[31985]: 2 slow requests, 0 included below; oldest blocked for > 521.847054 secs 2024-03-28T18:25:36.175 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:25:35 smithi134 ceph-mon[35037]: pgmap v581: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:25:36.183 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:25:35 smithi079 ceph-mon[35312]: pgmap v581: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:25:36.282 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:25:35 smithi022 ceph-mon[31985]: pgmap v581: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:25:38.175 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:25:37 smithi134 ceph-mon[35037]: pgmap v582: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:25:38.182 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:25:37 smithi079 ceph-mon[35312]: pgmap v582: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:25:38.282 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:25:37 smithi022 ceph-mon[31985]: pgmap v582: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:25:40.175 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:25:39 smithi134 ceph-mon[35037]: pgmap v583: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:25:40.175 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:25:39 smithi134 ceph-mon[35037]: 2 slow requests, 0 included below; oldest blocked for > 526.847174 secs 2024-03-28T18:25:40.182 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:25:39 smithi079 ceph-mon[35312]: pgmap v583: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:25:40.182 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:25:39 smithi079 ceph-mon[35312]: 2 slow requests, 0 included below; oldest blocked for > 526.847174 secs 2024-03-28T18:25:40.282 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:25:39 smithi022 ceph-mon[31985]: pgmap v583: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:25:40.282 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:25:39 smithi022 ceph-mon[31985]: 2 slow requests, 0 included below; oldest blocked for > 526.847174 secs 2024-03-28T18:25:42.175 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:25:41 smithi134 ceph-mon[35037]: pgmap v584: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:25:42.182 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:25:41 smithi079 ceph-mon[35312]: pgmap v584: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:25:42.282 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:25:41 smithi022 ceph-mon[31985]: pgmap v584: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:25:44.175 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:25:43 smithi134 ceph-mon[35037]: pgmap v585: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:25:44.182 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:25:43 smithi079 ceph-mon[35312]: pgmap v585: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:25:44.282 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:25:43 smithi022 ceph-mon[31985]: pgmap v585: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:25:45.084 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:25:44 smithi079 ceph-mon[35312]: 2 slow requests, 0 included below; oldest blocked for > 531.847213 secs 2024-03-28T18:25:45.174 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:25:44 smithi134 ceph-mon[35037]: 2 slow requests, 0 included below; oldest blocked for > 531.847213 secs 2024-03-28T18:25:45.282 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:25:44 smithi022 ceph-mon[31985]: 2 slow requests, 0 included below; oldest blocked for > 531.847213 secs 2024-03-28T18:25:46.175 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:25:45 smithi134 ceph-mon[35037]: pgmap v586: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:25:46.183 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:25:45 smithi079 ceph-mon[35312]: pgmap v586: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:25:46.282 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:25:45 smithi022 ceph-mon[31985]: pgmap v586: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:25:48.175 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:25:47 smithi134 ceph-mon[35037]: pgmap v587: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:25:48.183 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:25:47 smithi079 ceph-mon[35312]: pgmap v587: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:25:48.282 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:25:47 smithi022 ceph-mon[31985]: pgmap v587: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:25:50.174 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:25:49 smithi134 ceph-mon[35037]: pgmap v588: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:25:50.175 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:25:49 smithi134 ceph-mon[35037]: 2 slow requests, 0 included below; oldest blocked for > 536.847285 secs 2024-03-28T18:25:50.182 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:25:49 smithi079 ceph-mon[35312]: pgmap v588: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:25:50.183 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:25:49 smithi079 ceph-mon[35312]: 2 slow requests, 0 included below; oldest blocked for > 536.847285 secs 2024-03-28T18:25:50.282 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:25:49 smithi022 ceph-mon[31985]: pgmap v588: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:25:50.282 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:25:49 smithi022 ceph-mon[31985]: 2 slow requests, 0 included below; oldest blocked for > 536.847285 secs 2024-03-28T18:25:52.175 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:25:51 smithi134 ceph-mon[35037]: pgmap v589: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:25:52.183 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:25:51 smithi079 ceph-mon[35312]: pgmap v589: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:25:52.282 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:25:51 smithi022 ceph-mon[31985]: pgmap v589: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:25:54.174 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:25:53 smithi134 ceph-mon[35037]: pgmap v590: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:25:54.182 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:25:53 smithi079 ceph-mon[35312]: pgmap v590: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:25:54.282 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:25:53 smithi022 ceph-mon[31985]: pgmap v590: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:25:55.175 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:25:54 smithi134 ceph-mon[35037]: 2 slow requests, 0 included below; oldest blocked for > 541.847349 secs 2024-03-28T18:25:55.182 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:25:54 smithi079 ceph-mon[35312]: 2 slow requests, 0 included below; oldest blocked for > 541.847349 secs 2024-03-28T18:25:55.282 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:25:54 smithi022 ceph-mon[31985]: 2 slow requests, 0 included below; oldest blocked for > 541.847349 secs 2024-03-28T18:25:56.174 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:25:55 smithi134 ceph-mon[35037]: pgmap v591: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:25:56.182 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:25:55 smithi079 ceph-mon[35312]: pgmap v591: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:25:56.282 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:25:55 smithi022 ceph-mon[31985]: pgmap v591: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:25:58.175 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:25:57 smithi134 ceph-mon[35037]: pgmap v592: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:25:58.182 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:25:57 smithi079 ceph-mon[35312]: pgmap v592: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:25:58.282 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:25:57 smithi022 ceph-mon[31985]: pgmap v592: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:26:00.175 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:25:59 smithi134 ceph-mon[35037]: pgmap v593: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:26:00.175 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:25:59 smithi134 ceph-mon[35037]: 2 slow requests, 0 included below; oldest blocked for > 546.847382 secs 2024-03-28T18:26:00.182 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:25:59 smithi079 ceph-mon[35312]: pgmap v593: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:26:00.182 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:25:59 smithi079 ceph-mon[35312]: 2 slow requests, 0 included below; oldest blocked for > 546.847382 secs 2024-03-28T18:26:00.282 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:25:59 smithi022 ceph-mon[31985]: pgmap v593: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:26:00.282 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:25:59 smithi022 ceph-mon[31985]: 2 slow requests, 0 included below; oldest blocked for > 546.847382 secs 2024-03-28T18:26:02.175 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:26:01 smithi134 ceph-mon[35037]: pgmap v594: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:26:02.182 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:26:01 smithi079 ceph-mon[35312]: pgmap v594: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:26:02.282 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:26:01 smithi022 ceph-mon[31985]: pgmap v594: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:26:04.175 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:26:03 smithi134 ceph-mon[35037]: pgmap v595: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:26:04.183 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:26:03 smithi079 ceph-mon[35312]: pgmap v595: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:26:04.282 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:26:03 smithi022 ceph-mon[31985]: pgmap v595: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:26:05.175 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:26:04 smithi134 ceph-mon[35037]: 2 slow requests, 0 included below; oldest blocked for > 551.847504 secs 2024-03-28T18:26:05.182 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:26:04 smithi079 ceph-mon[35312]: 2 slow requests, 0 included below; oldest blocked for > 551.847504 secs 2024-03-28T18:26:05.282 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:26:04 smithi022 ceph-mon[31985]: 2 slow requests, 0 included below; oldest blocked for > 551.847504 secs 2024-03-28T18:26:06.175 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:26:05 smithi134 ceph-mon[35037]: pgmap v596: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:26:06.182 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:26:05 smithi079 ceph-mon[35312]: pgmap v596: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:26:06.282 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:26:05 smithi022 ceph-mon[31985]: pgmap v596: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:26:08.175 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:26:07 smithi134 ceph-mon[35037]: pgmap v597: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:26:08.183 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:26:07 smithi079 ceph-mon[35312]: pgmap v597: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:26:08.282 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:26:07 smithi022 ceph-mon[31985]: pgmap v597: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:26:10.183 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:26:09 smithi079 ceph-mon[35312]: pgmap v598: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:26:10.183 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:26:09 smithi079 ceph-mon[35312]: 2 slow requests, 0 included below; oldest blocked for > 556.847605 secs 2024-03-28T18:26:10.282 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:26:09 smithi022 ceph-mon[31985]: pgmap v598: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:26:10.282 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:26:09 smithi022 ceph-mon[31985]: 2 slow requests, 0 included below; oldest blocked for > 556.847605 secs 2024-03-28T18:26:10.425 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:26:09 smithi134 ceph-mon[35037]: pgmap v598: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:26:10.425 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:26:09 smithi134 ceph-mon[35037]: 2 slow requests, 0 included below; oldest blocked for > 556.847605 secs 2024-03-28T18:26:12.282 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:26:11 smithi022 ceph-mon[31985]: pgmap v599: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:26:12.424 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:26:11 smithi134 ceph-mon[35037]: pgmap v599: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:26:12.433 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:26:11 smithi079 ceph-mon[35312]: pgmap v599: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:26:14.282 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:26:13 smithi022 ceph-mon[31985]: pgmap v600: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:26:14.424 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:26:13 smithi134 ceph-mon[35037]: pgmap v600: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:26:14.432 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:26:13 smithi079 ceph-mon[35312]: pgmap v600: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:26:15.282 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:26:14 smithi022 ceph-mon[31985]: 2 slow requests, 0 included below; oldest blocked for > 561.847668 secs 2024-03-28T18:26:15.334 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:26:14 smithi079 ceph-mon[35312]: 2 slow requests, 0 included below; oldest blocked for > 561.847668 secs 2024-03-28T18:26:15.424 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:26:14 smithi134 ceph-mon[35037]: 2 slow requests, 0 included below; oldest blocked for > 561.847668 secs 2024-03-28T18:26:16.282 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:26:15 smithi022 ceph-mon[31985]: pgmap v601: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:26:16.424 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:26:15 smithi134 ceph-mon[35037]: pgmap v601: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:26:16.433 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:26:15 smithi079 ceph-mon[35312]: pgmap v601: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:26:18.282 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:26:17 smithi022 ceph-mon[31985]: pgmap v602: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:26:18.425 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:26:17 smithi134 ceph-mon[35037]: pgmap v602: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:26:18.432 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:26:17 smithi079 ceph-mon[35312]: pgmap v602: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:26:19.282 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:26:18 smithi022 ceph-mon[31985]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T18:26:19.424 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:26:18 smithi134 ceph-mon[35037]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T18:26:19.432 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:26:18 smithi079 ceph-mon[35312]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T18:26:20.282 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:26:19 smithi022 ceph-mon[31985]: pgmap v603: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:26:20.282 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:26:19 smithi022 ceph-mon[31985]: 2 slow requests, 0 included below; oldest blocked for > 566.847736 secs 2024-03-28T18:26:20.283 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:26:19 smithi022 ceph-mon[31985]: from='mgr.15522 ' entity='mgr.x' 2024-03-28T18:26:20.283 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:26:19 smithi022 ceph-mon[31985]: from='mgr.15522 ' entity='mgr.x' 2024-03-28T18:26:20.283 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:26:19 smithi022 ceph-mon[31985]: from='mgr.15522 ' entity='mgr.x' 2024-03-28T18:26:20.283 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:26:19 smithi022 ceph-mon[31985]: from='mgr.15522 ' entity='mgr.x' 2024-03-28T18:26:20.425 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:26:19 smithi134 ceph-mon[35037]: pgmap v603: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:26:20.425 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:26:19 smithi134 ceph-mon[35037]: 2 slow requests, 0 included below; oldest blocked for > 566.847736 secs 2024-03-28T18:26:20.425 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:26:19 smithi134 ceph-mon[35037]: from='mgr.15522 ' entity='mgr.x' 2024-03-28T18:26:20.425 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:26:19 smithi134 ceph-mon[35037]: from='mgr.15522 ' entity='mgr.x' 2024-03-28T18:26:20.425 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:26:19 smithi134 ceph-mon[35037]: from='mgr.15522 ' entity='mgr.x' 2024-03-28T18:26:20.425 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:26:19 smithi134 ceph-mon[35037]: from='mgr.15522 ' entity='mgr.x' 2024-03-28T18:26:20.432 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:26:19 smithi079 ceph-mon[35312]: pgmap v603: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:26:20.433 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:26:19 smithi079 ceph-mon[35312]: 2 slow requests, 0 included below; oldest blocked for > 566.847736 secs 2024-03-28T18:26:20.433 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:26:19 smithi079 ceph-mon[35312]: from='mgr.15522 ' entity='mgr.x' 2024-03-28T18:26:20.433 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:26:19 smithi079 ceph-mon[35312]: from='mgr.15522 ' entity='mgr.x' 2024-03-28T18:26:20.433 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:26:19 smithi079 ceph-mon[35312]: from='mgr.15522 ' entity='mgr.x' 2024-03-28T18:26:20.433 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:26:19 smithi079 ceph-mon[35312]: from='mgr.15522 ' entity='mgr.x' 2024-03-28T18:26:22.282 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:26:21 smithi022 ceph-mon[31985]: pgmap v604: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:26:22.282 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:26:21 smithi022 ceph-mon[31985]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T18:26:22.282 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:26:21 smithi022 ceph-mon[31985]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T18:26:22.282 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:26:21 smithi022 ceph-mon[31985]: pgmap v605: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T18:26:22.283 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:26:21 smithi022 ceph-mon[31985]: pgmap v606: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-03-28T18:26:22.283 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:26:21 smithi022 ceph-mon[31985]: from='mgr.15522 ' entity='mgr.x' 2024-03-28T18:26:22.425 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:26:21 smithi134 ceph-mon[35037]: pgmap v604: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:26:22.425 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:26:21 smithi134 ceph-mon[35037]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T18:26:22.425 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:26:21 smithi134 ceph-mon[35037]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T18:26:22.425 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:26:21 smithi134 ceph-mon[35037]: pgmap v605: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T18:26:22.425 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:26:21 smithi134 ceph-mon[35037]: pgmap v606: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-03-28T18:26:22.425 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:26:21 smithi134 ceph-mon[35037]: from='mgr.15522 ' entity='mgr.x' 2024-03-28T18:26:22.432 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:26:21 smithi079 ceph-mon[35312]: pgmap v604: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:26:22.433 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:26:21 smithi079 ceph-mon[35312]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T18:26:22.433 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:26:21 smithi079 ceph-mon[35312]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T18:26:22.433 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:26:21 smithi079 ceph-mon[35312]: pgmap v605: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T18:26:22.433 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:26:21 smithi079 ceph-mon[35312]: pgmap v606: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-03-28T18:26:22.433 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:26:21 smithi079 ceph-mon[35312]: from='mgr.15522 ' entity='mgr.x' 2024-03-28T18:26:24.425 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:26:24 smithi134 ceph-mon[35037]: pgmap v607: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s rd, 4 op/s 2024-03-28T18:26:24.433 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:26:24 smithi079 ceph-mon[35312]: pgmap v607: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s rd, 4 op/s 2024-03-28T18:26:24.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:26:24 smithi022 ceph-mon[31985]: pgmap v607: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s rd, 4 op/s 2024-03-28T18:26:25.425 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:26:25 smithi134 ceph-mon[35037]: 2 slow requests, 0 included below; oldest blocked for > 571.847810 secs 2024-03-28T18:26:25.433 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:26:25 smithi079 ceph-mon[35312]: 2 slow requests, 0 included below; oldest blocked for > 571.847810 secs 2024-03-28T18:26:25.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:26:25 smithi022 ceph-mon[31985]: 2 slow requests, 0 included below; oldest blocked for > 571.847810 secs 2024-03-28T18:26:26.425 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:26:26 smithi134 ceph-mon[35037]: pgmap v608: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-03-28T18:26:26.432 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:26:26 smithi079 ceph-mon[35312]: pgmap v608: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-03-28T18:26:26.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:26:26 smithi022 ceph-mon[31985]: pgmap v608: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-03-28T18:26:27.425 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:26:27 smithi134 ceph-mon[35037]: pgmap v609: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-03-28T18:26:27.432 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:26:27 smithi079 ceph-mon[35312]: pgmap v609: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-03-28T18:26:27.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:26:27 smithi022 ceph-mon[31985]: pgmap v609: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-03-28T18:26:30.425 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:26:30 smithi134 ceph-mon[35037]: pgmap v610: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s rd, 4 op/s 2024-03-28T18:26:30.425 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:26:30 smithi134 ceph-mon[35037]: 2 slow requests, 0 included below; oldest blocked for > 576.847830 secs 2024-03-28T18:26:30.432 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:26:30 smithi079 ceph-mon[35312]: pgmap v610: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s rd, 4 op/s 2024-03-28T18:26:30.433 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:26:30 smithi079 ceph-mon[35312]: 2 slow requests, 0 included below; oldest blocked for > 576.847830 secs 2024-03-28T18:26:30.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:26:30 smithi022 ceph-mon[31985]: pgmap v610: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s rd, 4 op/s 2024-03-28T18:26:30.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:26:30 smithi022 ceph-mon[31985]: 2 slow requests, 0 included below; oldest blocked for > 576.847830 secs 2024-03-28T18:26:32.425 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:26:32 smithi134 ceph-mon[35037]: pgmap v611: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 7 op/s 2024-03-28T18:26:32.433 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:26:32 smithi079 ceph-mon[35312]: pgmap v611: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 7 op/s 2024-03-28T18:26:32.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:26:32 smithi022 ceph-mon[31985]: pgmap v611: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 7 op/s 2024-03-28T18:26:34.425 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:26:34 smithi134 ceph-mon[35037]: pgmap v612: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:26:34.432 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:26:34 smithi079 ceph-mon[35312]: pgmap v612: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:26:34.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:26:34 smithi022 ceph-mon[31985]: pgmap v612: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:26:35.425 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:26:35 smithi134 ceph-mon[35037]: 2 slow requests, 0 included below; oldest blocked for > 581.847933 secs 2024-03-28T18:26:35.432 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:26:35 smithi079 ceph-mon[35312]: 2 slow requests, 0 included below; oldest blocked for > 581.847933 secs 2024-03-28T18:26:35.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:26:35 smithi022 ceph-mon[31985]: 2 slow requests, 0 included below; oldest blocked for > 581.847933 secs 2024-03-28T18:26:36.425 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:26:36 smithi134 ceph-mon[35037]: pgmap v613: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:26:36.432 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:26:36 smithi079 ceph-mon[35312]: pgmap v613: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:26:36.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:26:36 smithi022 ceph-mon[31985]: pgmap v613: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:26:37.424 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:26:37 smithi134 ceph-mon[35037]: pgmap v614: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:26:37.432 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:26:37 smithi079 ceph-mon[35312]: pgmap v614: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:26:37.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:26:37 smithi022 ceph-mon[31985]: pgmap v614: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:26:40.425 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:26:40 smithi134 ceph-mon[35037]: pgmap v615: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:26:40.425 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:26:40 smithi134 ceph-mon[35037]: 2 slow requests, 0 included below; oldest blocked for > 586.847984 secs 2024-03-28T18:26:40.432 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:26:40 smithi079 ceph-mon[35312]: pgmap v615: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:26:40.432 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:26:40 smithi079 ceph-mon[35312]: 2 slow requests, 0 included below; oldest blocked for > 586.847984 secs 2024-03-28T18:26:40.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:26:40 smithi022 ceph-mon[31985]: pgmap v615: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:26:40.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:26:40 smithi022 ceph-mon[31985]: 2 slow requests, 0 included below; oldest blocked for > 586.847984 secs 2024-03-28T18:26:42.425 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:26:42 smithi134 ceph-mon[35037]: pgmap v616: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:26:42.432 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:26:42 smithi079 ceph-mon[35312]: pgmap v616: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:26:42.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:26:42 smithi022 ceph-mon[31985]: pgmap v616: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:26:44.424 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:26:44 smithi134 ceph-mon[35037]: pgmap v617: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:26:44.432 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:26:44 smithi079 ceph-mon[35312]: pgmap v617: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:26:44.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:26:44 smithi022 ceph-mon[31985]: pgmap v617: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:26:45.425 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:26:45 smithi134 ceph-mon[35037]: 2 slow requests, 0 included below; oldest blocked for > 591.848049 secs 2024-03-28T18:26:45.425 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:26:45 smithi134 ceph-mon[35037]: pgmap v618: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:26:45.432 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:26:45 smithi079 ceph-mon[35312]: 2 slow requests, 0 included below; oldest blocked for > 591.848049 secs 2024-03-28T18:26:45.432 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:26:45 smithi079 ceph-mon[35312]: pgmap v618: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:26:45.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:26:45 smithi022 ceph-mon[31985]: 2 slow requests, 0 included below; oldest blocked for > 591.848049 secs 2024-03-28T18:26:45.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:26:45 smithi022 ceph-mon[31985]: pgmap v618: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:26:48.424 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:26:48 smithi134 ceph-mon[35037]: pgmap v619: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:26:48.432 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:26:48 smithi079 ceph-mon[35312]: pgmap v619: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:26:48.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:26:48 smithi022 ceph-mon[31985]: pgmap v619: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:26:50.425 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:26:50 smithi134 ceph-mon[35037]: pgmap v620: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:26:50.425 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:26:50 smithi134 ceph-mon[35037]: 2 slow requests, 0 included below; oldest blocked for > 596.848113 secs 2024-03-28T18:26:50.433 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:26:50 smithi079 ceph-mon[35312]: pgmap v620: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:26:50.433 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:26:50 smithi079 ceph-mon[35312]: 2 slow requests, 0 included below; oldest blocked for > 596.848113 secs 2024-03-28T18:26:50.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:26:50 smithi022 ceph-mon[31985]: pgmap v620: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:26:50.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:26:50 smithi022 ceph-mon[31985]: 2 slow requests, 0 included below; oldest blocked for > 596.848113 secs 2024-03-28T18:26:51.424 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:26:51 smithi134 ceph-mon[35037]: pgmap v621: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:26:51.432 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:26:51 smithi079 ceph-mon[35312]: pgmap v621: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:26:51.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:26:51 smithi022 ceph-mon[31985]: pgmap v621: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:26:54.425 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:26:54 smithi134 ceph-mon[35037]: pgmap v622: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:26:54.432 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:26:54 smithi079 ceph-mon[35312]: pgmap v622: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:26:54.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:26:54 smithi022 ceph-mon[31985]: pgmap v622: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:26:55.425 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:26:55 smithi134 ceph-mon[35037]: 2 slow requests, 0 included below; oldest blocked for > 601.848158 secs 2024-03-28T18:26:55.432 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:26:55 smithi079 ceph-mon[35312]: 2 slow requests, 0 included below; oldest blocked for > 601.848158 secs 2024-03-28T18:26:55.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:26:55 smithi022 ceph-mon[31985]: 2 slow requests, 0 included below; oldest blocked for > 601.848158 secs 2024-03-28T18:26:56.425 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:26:56 smithi134 ceph-mon[35037]: pgmap v623: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:26:56.432 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:26:56 smithi079 ceph-mon[35312]: pgmap v623: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:26:56.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:26:56 smithi022 ceph-mon[31985]: pgmap v623: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:26:57.424 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:26:57 smithi134 ceph-mon[35037]: pgmap v624: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:26:57.432 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:26:57 smithi079 ceph-mon[35312]: pgmap v624: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:26:57.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:26:57 smithi022 ceph-mon[31985]: pgmap v624: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:27:00.425 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:27:00 smithi134 ceph-mon[35037]: pgmap v625: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:27:00.425 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:27:00 smithi134 ceph-mon[35037]: 2 slow requests, 0 included below; oldest blocked for > 606.848277 secs 2024-03-28T18:27:00.432 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:27:00 smithi079 ceph-mon[35312]: pgmap v625: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:27:00.433 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:27:00 smithi079 ceph-mon[35312]: 2 slow requests, 0 included below; oldest blocked for > 606.848277 secs 2024-03-28T18:27:00.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:27:00 smithi022 ceph-mon[31985]: pgmap v625: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:27:00.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:27:00 smithi022 ceph-mon[31985]: 2 slow requests, 0 included below; oldest blocked for > 606.848277 secs 2024-03-28T18:27:02.424 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:27:02 smithi134 ceph-mon[35037]: pgmap v626: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:27:02.432 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:27:02 smithi079 ceph-mon[35312]: pgmap v626: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:27:02.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:27:02 smithi022 ceph-mon[31985]: pgmap v626: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:27:03.424 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:27:03 smithi134 ceph-mon[35037]: pgmap v627: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:27:03.432 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:27:03 smithi079 ceph-mon[35312]: pgmap v627: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:27:03.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:27:03 smithi022 ceph-mon[31985]: pgmap v627: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:27:04.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:27:04 smithi022 ceph-mon[31985]: 2 slow requests, 0 included below; oldest blocked for > 611.848350 secs 2024-03-28T18:27:04.925 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:27:04 smithi134 ceph-mon[35037]: 2 slow requests, 0 included below; oldest blocked for > 611.848350 secs 2024-03-28T18:27:04.933 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:27:04 smithi079 ceph-mon[35312]: 2 slow requests, 0 included below; oldest blocked for > 611.848350 secs 2024-03-28T18:27:05.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:27:05 smithi022 ceph-mon[31985]: pgmap v628: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:27:05.827 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:27:05 smithi134 ceph-mon[35037]: pgmap v628: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:27:05.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:27:05 smithi079 ceph-mon[35312]: pgmap v628: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:27:08.425 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:27:08 smithi134 ceph-mon[35037]: pgmap v629: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:27:08.432 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:27:08 smithi079 ceph-mon[35312]: pgmap v629: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:27:08.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:27:08 smithi022 ceph-mon[31985]: pgmap v629: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:27:09.433 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:27:09 smithi079 ceph-mon[35312]: pgmap v630: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:27:09.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:27:09 smithi022 ceph-mon[31985]: pgmap v630: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:27:09.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:27:09 smithi134 ceph-mon[35037]: pgmap v630: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:27:10.433 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:27:10 smithi079 ceph-mon[35312]: 2 slow requests, 0 included below; oldest blocked for > 616.848414 secs 2024-03-28T18:27:10.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:27:10 smithi022 ceph-mon[31985]: 2 slow requests, 0 included below; oldest blocked for > 616.848414 secs 2024-03-28T18:27:10.674 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:27:10 smithi134 ceph-mon[35037]: 2 slow requests, 0 included below; oldest blocked for > 616.848414 secs 2024-03-28T18:27:11.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:27:11 smithi022 ceph-mon[31985]: pgmap v631: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:27:11.674 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:27:11 smithi134 ceph-mon[35037]: pgmap v631: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:27:11.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:27:11 smithi079 ceph-mon[35312]: pgmap v631: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:27:14.425 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:27:14 smithi134 ceph-mon[35037]: pgmap v632: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:27:14.433 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:27:14 smithi079 ceph-mon[35312]: pgmap v632: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:27:14.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:27:14 smithi022 ceph-mon[31985]: pgmap v632: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:27:15.427 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:27:15 smithi134 ceph-mon[35037]: 2 slow requests, 0 included below; oldest blocked for > 621.848440 secs 2024-03-28T18:27:15.432 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:27:15 smithi079 ceph-mon[35312]: 2 slow requests, 0 included below; oldest blocked for > 621.848440 secs 2024-03-28T18:27:15.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:27:15 smithi022 ceph-mon[31985]: 2 slow requests, 0 included below; oldest blocked for > 621.848440 secs 2024-03-28T18:27:16.425 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:27:16 smithi134 ceph-mon[35037]: pgmap v633: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:27:16.432 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:27:16 smithi079 ceph-mon[35312]: pgmap v633: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:27:16.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:27:16 smithi022 ceph-mon[31985]: pgmap v633: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:27:17.425 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:27:17 smithi134 ceph-mon[35037]: pgmap v634: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:27:17.432 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:27:17 smithi079 ceph-mon[35312]: pgmap v634: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:27:17.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:27:17 smithi022 ceph-mon[31985]: pgmap v634: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:27:20.424 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:27:20 smithi134 ceph-mon[35037]: pgmap v635: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:27:20.425 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:27:20 smithi134 ceph-mon[35037]: 2 slow requests, 0 included below; oldest blocked for > 626.848475 secs 2024-03-28T18:27:20.432 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:27:20 smithi079 ceph-mon[35312]: pgmap v635: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:27:20.432 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:27:20 smithi079 ceph-mon[35312]: 2 slow requests, 0 included below; oldest blocked for > 626.848475 secs 2024-03-28T18:27:20.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:27:20 smithi022 ceph-mon[31985]: pgmap v635: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:27:20.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:27:20 smithi022 ceph-mon[31985]: 2 slow requests, 0 included below; oldest blocked for > 626.848475 secs 2024-03-28T18:27:21.424 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:27:21 smithi134 ceph-mon[35037]: pgmap v636: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:27:21.432 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:27:21 smithi079 ceph-mon[35312]: pgmap v636: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:27:21.531 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:27:21 smithi022 ceph-mon[31985]: pgmap v636: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:27:22.424 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:27:22 smithi134 ceph-mon[35037]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T18:27:22.432 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:27:22 smithi079 ceph-mon[35312]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T18:27:22.531 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:27:22 smithi022 ceph-mon[31985]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T18:27:23.424 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:27:23 smithi134 ceph-mon[35037]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T18:27:23.425 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:27:23 smithi134 ceph-mon[35037]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T18:27:23.425 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:27:23 smithi134 ceph-mon[35037]: pgmap v637: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 6 op/s 2024-03-28T18:27:23.425 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:27:23 smithi134 ceph-mon[35037]: pgmap v638: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-03-28T18:27:23.425 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:27:23 smithi134 ceph-mon[35037]: from='mgr.15522 ' entity='mgr.x' 2024-03-28T18:27:23.432 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:27:23 smithi079 ceph-mon[35312]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T18:27:23.432 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:27:23 smithi079 ceph-mon[35312]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T18:27:23.432 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:27:23 smithi079 ceph-mon[35312]: pgmap v637: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 6 op/s 2024-03-28T18:27:23.433 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:27:23 smithi079 ceph-mon[35312]: pgmap v638: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-03-28T18:27:23.433 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:27:23 smithi079 ceph-mon[35312]: from='mgr.15522 ' entity='mgr.x' 2024-03-28T18:27:23.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:27:23 smithi022 ceph-mon[31985]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T18:27:23.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:27:23 smithi022 ceph-mon[31985]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T18:27:23.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:27:23 smithi022 ceph-mon[31985]: pgmap v637: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 6 op/s 2024-03-28T18:27:23.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:27:23 smithi022 ceph-mon[31985]: pgmap v638: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-03-28T18:27:23.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:27:23 smithi022 ceph-mon[31985]: from='mgr.15522 ' entity='mgr.x' 2024-03-28T18:27:24.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:27:24 smithi022 ceph-mon[31985]: 2 slow requests, 0 included below; oldest blocked for > 631.848500 secs 2024-03-28T18:27:24.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:27:24 smithi134 ceph-mon[35037]: 2 slow requests, 0 included below; oldest blocked for > 631.848500 secs 2024-03-28T18:27:24.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:27:24 smithi079 ceph-mon[35312]: 2 slow requests, 0 included below; oldest blocked for > 631.848500 secs 2024-03-28T18:27:25.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:27:25 smithi022 ceph-mon[31985]: pgmap v639: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-03-28T18:27:25.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:27:25 smithi134 ceph-mon[35037]: pgmap v639: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-03-28T18:27:25.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:27:25 smithi079 ceph-mon[35312]: pgmap v639: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-03-28T18:27:28.174 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:27:27 smithi134 ceph-mon[35037]: pgmap v640: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-03-28T18:27:28.182 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:27:27 smithi079 ceph-mon[35312]: pgmap v640: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-03-28T18:27:28.282 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:27:27 smithi022 ceph-mon[31985]: pgmap v640: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-03-28T18:27:29.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:27:29 smithi134 ceph-mon[35037]: pgmap v641: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-03-28T18:27:29.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:27:29 smithi079 ceph-mon[35312]: pgmap v641: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-03-28T18:27:29.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:27:29 smithi022 ceph-mon[31985]: pgmap v641: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-03-28T18:27:30.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:27:30 smithi134 ceph-mon[35037]: 2 slow requests, 0 included below; oldest blocked for > 636.848532 secs 2024-03-28T18:27:30.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:27:30 smithi079 ceph-mon[35312]: 2 slow requests, 0 included below; oldest blocked for > 636.848532 secs 2024-03-28T18:27:30.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:27:30 smithi022 ceph-mon[31985]: 2 slow requests, 0 included below; oldest blocked for > 636.848532 secs 2024-03-28T18:27:31.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:27:31 smithi134 ceph-mon[35037]: pgmap v642: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-03-28T18:27:31.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:27:31 smithi079 ceph-mon[35312]: pgmap v642: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-03-28T18:27:31.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:27:31 smithi022 ceph-mon[31985]: pgmap v642: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-03-28T18:27:34.174 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:27:33 smithi134 ceph-mon[35037]: pgmap v643: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 7 op/s 2024-03-28T18:27:34.183 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:27:33 smithi079 ceph-mon[35312]: pgmap v643: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 7 op/s 2024-03-28T18:27:34.282 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:27:33 smithi022 ceph-mon[31985]: pgmap v643: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 7 op/s 2024-03-28T18:27:35.175 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:27:34 smithi134 ceph-mon[35037]: 2 slow requests, 0 included below; oldest blocked for > 641.848600 secs 2024-03-28T18:27:35.182 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:27:34 smithi079 ceph-mon[35312]: 2 slow requests, 0 included below; oldest blocked for > 641.848600 secs 2024-03-28T18:27:35.282 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:27:34 smithi022 ceph-mon[31985]: 2 slow requests, 0 included below; oldest blocked for > 641.848600 secs 2024-03-28T18:27:36.174 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:27:35 smithi134 ceph-mon[35037]: pgmap v644: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:27:36.182 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:27:35 smithi079 ceph-mon[35312]: pgmap v644: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:27:36.282 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:27:35 smithi022 ceph-mon[31985]: pgmap v644: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:27:38.174 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:27:37 smithi134 ceph-mon[35037]: pgmap v645: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:27:38.182 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:27:37 smithi079 ceph-mon[35312]: pgmap v645: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:27:38.282 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:27:37 smithi022 ceph-mon[31985]: pgmap v645: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:27:40.174 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:27:39 smithi134 ceph-mon[35037]: pgmap v646: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:27:40.174 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:27:39 smithi134 ceph-mon[35037]: 2 slow requests, 0 included below; oldest blocked for > 646.848675 secs 2024-03-28T18:27:40.182 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:27:39 smithi079 ceph-mon[35312]: pgmap v646: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:27:40.182 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:27:39 smithi079 ceph-mon[35312]: 2 slow requests, 0 included below; oldest blocked for > 646.848675 secs 2024-03-28T18:27:40.282 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:27:39 smithi022 ceph-mon[31985]: pgmap v646: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:27:40.282 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:27:39 smithi022 ceph-mon[31985]: 2 slow requests, 0 included below; oldest blocked for > 646.848675 secs 2024-03-28T18:27:42.174 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:27:41 smithi134 ceph-mon[35037]: pgmap v647: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:27:42.182 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:27:41 smithi079 ceph-mon[35312]: pgmap v647: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:27:42.282 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:27:41 smithi022 ceph-mon[31985]: pgmap v647: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:27:44.175 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:27:43 smithi134 ceph-mon[35037]: pgmap v648: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:27:44.182 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:27:43 smithi079 ceph-mon[35312]: pgmap v648: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:27:44.282 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:27:43 smithi022 ceph-mon[31985]: pgmap v648: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:27:45.174 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:27:44 smithi134 ceph-mon[35037]: 2 slow requests, 0 included below; oldest blocked for > 651.848694 secs 2024-03-28T18:27:45.182 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:27:44 smithi079 ceph-mon[35312]: 2 slow requests, 0 included below; oldest blocked for > 651.848694 secs 2024-03-28T18:27:45.282 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:27:44 smithi022 ceph-mon[31985]: 2 slow requests, 0 included below; oldest blocked for > 651.848694 secs 2024-03-28T18:27:46.174 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:27:45 smithi134 ceph-mon[35037]: pgmap v649: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:27:46.182 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:27:45 smithi079 ceph-mon[35312]: pgmap v649: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:27:46.281 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:27:45 smithi022 ceph-mon[31985]: pgmap v649: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:27:48.174 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:27:47 smithi134 ceph-mon[35037]: pgmap v650: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:27:48.182 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:27:47 smithi079 ceph-mon[35312]: pgmap v650: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:27:48.282 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:27:47 smithi022 ceph-mon[31985]: pgmap v650: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:27:50.174 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:27:49 smithi134 ceph-mon[35037]: pgmap v651: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:27:50.175 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:27:49 smithi134 ceph-mon[35037]: 2 slow requests, 0 included below; oldest blocked for > 656.848781 secs 2024-03-28T18:27:50.182 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:27:49 smithi079 ceph-mon[35312]: pgmap v651: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:27:50.182 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:27:49 smithi079 ceph-mon[35312]: 2 slow requests, 0 included below; oldest blocked for > 656.848781 secs 2024-03-28T18:27:50.282 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:27:49 smithi022 ceph-mon[31985]: pgmap v651: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:27:50.282 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:27:49 smithi022 ceph-mon[31985]: 2 slow requests, 0 included below; oldest blocked for > 656.848781 secs 2024-03-28T18:27:52.175 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:27:51 smithi134 ceph-mon[35037]: pgmap v652: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:27:52.182 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:27:51 smithi079 ceph-mon[35312]: pgmap v652: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:27:52.282 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:27:51 smithi022 ceph-mon[31985]: pgmap v652: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:27:54.174 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:27:53 smithi134 ceph-mon[35037]: pgmap v653: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:27:54.183 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:27:53 smithi079 ceph-mon[35312]: pgmap v653: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:27:54.282 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:27:53 smithi022 ceph-mon[31985]: pgmap v653: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:27:55.174 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:27:54 smithi134 ceph-mon[35037]: 2 slow requests, 0 included below; oldest blocked for > 661.848815 secs 2024-03-28T18:27:55.182 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:27:54 smithi079 ceph-mon[35312]: 2 slow requests, 0 included below; oldest blocked for > 661.848815 secs 2024-03-28T18:27:55.282 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:27:54 smithi022 ceph-mon[31985]: 2 slow requests, 0 included below; oldest blocked for > 661.848815 secs 2024-03-28T18:27:56.174 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:27:55 smithi134 ceph-mon[35037]: pgmap v654: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:27:56.182 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:27:55 smithi079 ceph-mon[35312]: pgmap v654: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:27:56.281 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:27:55 smithi022 ceph-mon[31985]: pgmap v654: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:27:58.182 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:27:57 smithi079 ceph-mon[35312]: pgmap v655: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:27:58.282 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:27:57 smithi022 ceph-mon[31985]: pgmap v655: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:27:58.424 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:27:57 smithi134 ceph-mon[35037]: pgmap v655: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:28:00.282 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:27:59 smithi022 ceph-mon[31985]: pgmap v656: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:28:00.282 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:27:59 smithi022 ceph-mon[31985]: 2 slow requests, 0 included below; oldest blocked for > 666.848885 secs 2024-03-28T18:28:00.424 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:27:59 smithi134 ceph-mon[35037]: pgmap v656: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:28:00.424 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:27:59 smithi134 ceph-mon[35037]: 2 slow requests, 0 included below; oldest blocked for > 666.848885 secs 2024-03-28T18:28:00.432 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:27:59 smithi079 ceph-mon[35312]: pgmap v656: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:28:00.432 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:27:59 smithi079 ceph-mon[35312]: 2 slow requests, 0 included below; oldest blocked for > 666.848885 secs 2024-03-28T18:28:02.282 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:28:01 smithi022 ceph-mon[31985]: pgmap v657: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:28:02.424 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:28:01 smithi134 ceph-mon[35037]: pgmap v657: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:28:02.432 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:28:01 smithi079 ceph-mon[35312]: pgmap v657: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:28:04.282 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:28:03 smithi022 ceph-mon[31985]: pgmap v658: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:28:04.424 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:28:03 smithi134 ceph-mon[35037]: pgmap v658: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:28:04.432 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:28:03 smithi079 ceph-mon[35312]: pgmap v658: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:28:05.282 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:28:04 smithi022 ceph-mon[31985]: 2 slow requests, 0 included below; oldest blocked for > 671.848957 secs 2024-03-28T18:28:05.326 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:28:04 smithi134 ceph-mon[35037]: 2 slow requests, 0 included below; oldest blocked for > 671.848957 secs 2024-03-28T18:28:05.432 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:28:04 smithi079 ceph-mon[35312]: 2 slow requests, 0 included below; oldest blocked for > 671.848957 secs 2024-03-28T18:28:06.282 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:28:05 smithi022 ceph-mon[31985]: pgmap v659: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:28:06.424 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:28:05 smithi134 ceph-mon[35037]: pgmap v659: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:28:06.432 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:28:05 smithi079 ceph-mon[35312]: pgmap v659: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:28:08.282 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:28:07 smithi022 ceph-mon[31985]: pgmap v660: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:28:08.424 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:28:07 smithi134 ceph-mon[35037]: pgmap v660: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:28:08.432 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:28:07 smithi079 ceph-mon[35312]: pgmap v660: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:28:10.282 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:28:09 smithi022 ceph-mon[31985]: pgmap v661: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:28:10.282 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:28:09 smithi022 ceph-mon[31985]: 2 slow requests, 0 included below; oldest blocked for > 676.848978 secs 2024-03-28T18:28:10.425 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:28:09 smithi134 ceph-mon[35037]: pgmap v661: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:28:10.425 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:28:09 smithi134 ceph-mon[35037]: 2 slow requests, 0 included below; oldest blocked for > 676.848978 secs 2024-03-28T18:28:10.433 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:28:09 smithi079 ceph-mon[35312]: pgmap v661: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:28:10.433 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:28:09 smithi079 ceph-mon[35312]: 2 slow requests, 0 included below; oldest blocked for > 676.848978 secs 2024-03-28T18:28:12.282 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:28:11 smithi022 ceph-mon[31985]: pgmap v662: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:28:12.424 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:28:11 smithi134 ceph-mon[35037]: pgmap v662: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:28:12.432 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:28:11 smithi079 ceph-mon[35312]: pgmap v662: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:28:14.282 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:28:13 smithi022 ceph-mon[31985]: pgmap v663: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:28:14.424 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:28:13 smithi134 ceph-mon[35037]: pgmap v663: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:28:14.433 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:28:13 smithi079 ceph-mon[35312]: pgmap v663: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:28:15.282 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:28:14 smithi022 ceph-mon[31985]: 2 slow requests, 0 included below; oldest blocked for > 681.849066 secs 2024-03-28T18:28:15.334 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:28:15 smithi079 ceph-mon[35312]: 2 slow requests, 0 included below; oldest blocked for > 681.849066 secs 2024-03-28T18:28:15.424 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:28:15 smithi134 ceph-mon[35037]: 2 slow requests, 0 included below; oldest blocked for > 681.849066 secs 2024-03-28T18:28:16.282 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:28:16 smithi022 ceph-mon[31985]: pgmap v664: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:28:16.424 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:28:16 smithi134 ceph-mon[35037]: pgmap v664: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:28:16.432 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:28:16 smithi079 ceph-mon[35312]: pgmap v664: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:28:18.282 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:28:18 smithi022 ceph-mon[31985]: pgmap v665: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:28:18.424 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:28:18 smithi134 ceph-mon[35037]: pgmap v665: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:28:18.432 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:28:18 smithi079 ceph-mon[35312]: pgmap v665: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:28:20.282 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:28:20 smithi022 ceph-mon[31985]: pgmap v666: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:28:20.282 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:28:20 smithi022 ceph-mon[31985]: 2 slow requests, 0 included below; oldest blocked for > 686.849141 secs 2024-03-28T18:28:20.425 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:28:20 smithi134 ceph-mon[35037]: pgmap v666: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:28:20.425 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:28:20 smithi134 ceph-mon[35037]: 2 slow requests, 0 included below; oldest blocked for > 686.849141 secs 2024-03-28T18:28:20.432 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:28:20 smithi079 ceph-mon[35312]: pgmap v666: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:28:20.432 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:28:20 smithi079 ceph-mon[35312]: 2 slow requests, 0 included below; oldest blocked for > 686.849141 secs 2024-03-28T18:28:22.282 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:28:22 smithi022 ceph-mon[31985]: pgmap v667: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:28:22.425 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:28:22 smithi134 ceph-mon[35037]: pgmap v667: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:28:22.433 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:28:22 smithi079 ceph-mon[35312]: pgmap v667: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:28:24.281 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:28:24 smithi022 ceph-mon[31985]: pgmap v668: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:28:24.281 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:28:24 smithi022 ceph-mon[31985]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T18:28:24.425 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:28:24 smithi134 ceph-mon[35037]: pgmap v668: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:28:24.425 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:28:24 smithi134 ceph-mon[35037]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T18:28:24.432 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:28:24 smithi079 ceph-mon[35312]: pgmap v668: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:28:24.432 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:28:24 smithi079 ceph-mon[35312]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T18:28:25.425 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:28:25 smithi134 ceph-mon[35037]: from='mgr.15522 ' entity='mgr.x' 2024-03-28T18:28:25.425 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:28:25 smithi134 ceph-mon[35037]: from='mgr.15522 ' entity='mgr.x' 2024-03-28T18:28:25.425 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:28:25 smithi134 ceph-mon[35037]: 2 slow requests, 0 included below; oldest blocked for > 691.849206 secs 2024-03-28T18:28:25.432 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:28:25 smithi079 ceph-mon[35312]: from='mgr.15522 ' entity='mgr.x' 2024-03-28T18:28:25.432 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:28:25 smithi079 ceph-mon[35312]: from='mgr.15522 ' entity='mgr.x' 2024-03-28T18:28:25.432 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:28:25 smithi079 ceph-mon[35312]: 2 slow requests, 0 included below; oldest blocked for > 691.849206 secs 2024-03-28T18:28:25.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:28:25 smithi022 ceph-mon[31985]: from='mgr.15522 ' entity='mgr.x' 2024-03-28T18:28:25.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:28:25 smithi022 ceph-mon[31985]: from='mgr.15522 ' entity='mgr.x' 2024-03-28T18:28:25.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:28:25 smithi022 ceph-mon[31985]: 2 slow requests, 0 included below; oldest blocked for > 691.849206 secs 2024-03-28T18:28:26.424 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:28:26 smithi134 ceph-mon[35037]: pgmap v669: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:28:26.432 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:28:26 smithi079 ceph-mon[35312]: pgmap v669: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:28:26.531 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:28:26 smithi022 ceph-mon[31985]: pgmap v669: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:28:27.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:28:27 smithi134 ceph-mon[35037]: from='mgr.15522 ' entity='mgr.x' 2024-03-28T18:28:27.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:28:27 smithi134 ceph-mon[35037]: from='mgr.15522 ' entity='mgr.x' 2024-03-28T18:28:27.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:28:27 smithi134 ceph-mon[35037]: pgmap v670: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:28:27.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:28:27 smithi134 ceph-mon[35037]: from='mgr.15522 ' entity='mgr.x' 2024-03-28T18:28:27.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:28:27 smithi134 ceph-mon[35037]: from='mgr.15522 ' entity='mgr.x' 2024-03-28T18:28:27.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:28:27 smithi134 ceph-mon[35037]: from='mgr.15522 ' entity='mgr.x' 2024-03-28T18:28:27.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:28:27 smithi134 ceph-mon[35037]: from='mgr.15522 ' entity='mgr.x' 2024-03-28T18:28:27.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:28:27 smithi079 ceph-mon[35312]: from='mgr.15522 ' entity='mgr.x' 2024-03-28T18:28:27.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:28:27 smithi079 ceph-mon[35312]: from='mgr.15522 ' entity='mgr.x' 2024-03-28T18:28:27.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:28:27 smithi079 ceph-mon[35312]: pgmap v670: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:28:27.683 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:28:27 smithi079 ceph-mon[35312]: from='mgr.15522 ' entity='mgr.x' 2024-03-28T18:28:27.683 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:28:27 smithi079 ceph-mon[35312]: from='mgr.15522 ' entity='mgr.x' 2024-03-28T18:28:27.683 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:28:27 smithi079 ceph-mon[35312]: from='mgr.15522 ' entity='mgr.x' 2024-03-28T18:28:27.683 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:28:27 smithi079 ceph-mon[35312]: from='mgr.15522 ' entity='mgr.x' 2024-03-28T18:28:27.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:28:27 smithi022 ceph-mon[31985]: from='mgr.15522 ' entity='mgr.x' 2024-03-28T18:28:27.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:28:27 smithi022 ceph-mon[31985]: from='mgr.15522 ' entity='mgr.x' 2024-03-28T18:28:27.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:28:27 smithi022 ceph-mon[31985]: pgmap v670: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:28:27.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:28:27 smithi022 ceph-mon[31985]: from='mgr.15522 ' entity='mgr.x' 2024-03-28T18:28:27.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:28:27 smithi022 ceph-mon[31985]: from='mgr.15522 ' entity='mgr.x' 2024-03-28T18:28:27.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:28:27 smithi022 ceph-mon[31985]: from='mgr.15522 ' entity='mgr.x' 2024-03-28T18:28:27.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:28:27 smithi022 ceph-mon[31985]: from='mgr.15522 ' entity='mgr.x' 2024-03-28T18:28:28.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:28:28 smithi134 ceph-mon[35037]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-28T18:28:28.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:28:28 smithi134 ceph-mon[35037]: from='mgr.15522 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-28T18:28:28.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:28:28 smithi134 ceph-mon[35037]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-28T18:28:28.676 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:28:28 smithi134 ceph-mon[35037]: from='mgr.15522 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-28T18:28:28.676 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:28:28 smithi134 ceph-mon[35037]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-03-28T18:28:28.676 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:28:28 smithi134 ceph-mon[35037]: from='mgr.15522 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-03-28T18:28:28.676 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:28:28 smithi134 ceph-mon[35037]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-03-28T18:28:28.676 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:28:28 smithi134 ceph-mon[35037]: from='mgr.15522 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-03-28T18:28:28.676 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:28:28 smithi134 ceph-mon[35037]: Adjusting osd_memory_target on smithi079 to 176.7M 2024-03-28T18:28:28.676 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:28:28 smithi134 ceph-mon[35037]: Unable to set osd_memory_target on smithi079 to 185386700: error parsing value: Value '185386700' is below minimum 939524096 2024-03-28T18:28:28.676 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:28:28 smithi134 ceph-mon[35037]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-28T18:28:28.676 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:28:28 smithi134 ceph-mon[35037]: from='mgr.15522 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-28T18:28:28.676 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:28:28 smithi134 ceph-mon[35037]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-28T18:28:28.676 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:28:28 smithi134 ceph-mon[35037]: from='mgr.15522 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-28T18:28:28.676 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:28:28 smithi134 ceph-mon[35037]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-03-28T18:28:28.676 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:28:28 smithi134 ceph-mon[35037]: from='mgr.15522 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-03-28T18:28:28.676 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:28:28 smithi134 ceph-mon[35037]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-03-28T18:28:28.676 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:28:28 smithi134 ceph-mon[35037]: from='mgr.15522 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-03-28T18:28:28.683 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:28:28 smithi079 ceph-mon[35312]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-28T18:28:28.683 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:28:28 smithi079 ceph-mon[35312]: from='mgr.15522 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-28T18:28:28.683 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:28:28 smithi079 ceph-mon[35312]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-28T18:28:28.683 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:28:28 smithi079 ceph-mon[35312]: from='mgr.15522 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-28T18:28:28.683 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:28:28 smithi079 ceph-mon[35312]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-03-28T18:28:28.683 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:28:28 smithi079 ceph-mon[35312]: from='mgr.15522 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-03-28T18:28:28.683 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:28:28 smithi079 ceph-mon[35312]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-03-28T18:28:28.683 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:28:28 smithi079 ceph-mon[35312]: from='mgr.15522 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-03-28T18:28:28.683 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:28:28 smithi079 ceph-mon[35312]: Adjusting osd_memory_target on smithi079 to 176.7M 2024-03-28T18:28:28.683 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:28:28 smithi079 ceph-mon[35312]: Unable to set osd_memory_target on smithi079 to 185386700: error parsing value: Value '185386700' is below minimum 939524096 2024-03-28T18:28:28.683 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:28:28 smithi079 ceph-mon[35312]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-28T18:28:28.683 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:28:28 smithi079 ceph-mon[35312]: from='mgr.15522 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-28T18:28:28.684 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:28:28 smithi079 ceph-mon[35312]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-28T18:28:28.684 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:28:28 smithi079 ceph-mon[35312]: from='mgr.15522 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-28T18:28:28.684 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:28:28 smithi079 ceph-mon[35312]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-03-28T18:28:28.684 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:28:28 smithi079 ceph-mon[35312]: from='mgr.15522 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-03-28T18:28:28.684 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:28:28 smithi079 ceph-mon[35312]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-03-28T18:28:28.684 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:28:28 smithi079 ceph-mon[35312]: from='mgr.15522 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-03-28T18:28:28.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:28:28 smithi022 ceph-mon[31985]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-28T18:28:28.783 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:28:28 smithi022 ceph-mon[31985]: from='mgr.15522 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-28T18:28:28.783 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:28:28 smithi022 ceph-mon[31985]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-28T18:28:28.783 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:28:28 smithi022 ceph-mon[31985]: from='mgr.15522 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-28T18:28:28.783 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:28:28 smithi022 ceph-mon[31985]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-03-28T18:28:28.783 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:28:28 smithi022 ceph-mon[31985]: from='mgr.15522 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-03-28T18:28:28.783 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:28:28 smithi022 ceph-mon[31985]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-03-28T18:28:28.783 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:28:28 smithi022 ceph-mon[31985]: from='mgr.15522 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-03-28T18:28:28.783 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:28:28 smithi022 ceph-mon[31985]: Adjusting osd_memory_target on smithi079 to 176.7M 2024-03-28T18:28:28.783 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:28:28 smithi022 ceph-mon[31985]: Unable to set osd_memory_target on smithi079 to 185386700: error parsing value: Value '185386700' is below minimum 939524096 2024-03-28T18:28:28.783 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:28:28 smithi022 ceph-mon[31985]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-28T18:28:28.783 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:28:28 smithi022 ceph-mon[31985]: from='mgr.15522 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-28T18:28:28.783 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:28:28 smithi022 ceph-mon[31985]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-28T18:28:28.783 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:28:28 smithi022 ceph-mon[31985]: from='mgr.15522 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-28T18:28:28.783 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:28:28 smithi022 ceph-mon[31985]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-03-28T18:28:28.784 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:28:28 smithi022 ceph-mon[31985]: from='mgr.15522 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-03-28T18:28:28.784 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:28:28 smithi022 ceph-mon[31985]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-03-28T18:28:28.784 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:28:28 smithi022 ceph-mon[31985]: from='mgr.15522 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-03-28T18:28:29.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:28:29 smithi134 ceph-mon[35037]: Adjusting osd_memory_target on smithi134 to 176.8M 2024-03-28T18:28:29.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:28:29 smithi134 ceph-mon[35037]: Unable to set osd_memory_target on smithi134 to 185393868: error parsing value: Value '185393868' is below minimum 939524096 2024-03-28T18:28:29.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:28:29 smithi134 ceph-mon[35037]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-28T18:28:29.676 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:28:29 smithi134 ceph-mon[35037]: from='mgr.15522 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-28T18:28:29.676 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:28:29 smithi134 ceph-mon[35037]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-28T18:28:29.676 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:28:29 smithi134 ceph-mon[35037]: from='mgr.15522 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-28T18:28:29.676 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:28:29 smithi134 ceph-mon[35037]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-03-28T18:28:29.676 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:28:29 smithi134 ceph-mon[35037]: from='mgr.15522 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-03-28T18:28:29.676 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:28:29 smithi134 ceph-mon[35037]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-03-28T18:28:29.676 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:28:29 smithi134 ceph-mon[35037]: from='mgr.15522 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-03-28T18:28:29.676 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:28:29 smithi134 ceph-mon[35037]: Adjusting osd_memory_target on smithi022 to 177.8M 2024-03-28T18:28:29.676 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:28:29 smithi134 ceph-mon[35037]: Unable to set osd_memory_target on smithi022 to 186497740: error parsing value: Value '186497740' is below minimum 939524096 2024-03-28T18:28:29.676 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:28:29 smithi134 ceph-mon[35037]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T18:28:29.676 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:28:29 smithi134 ceph-mon[35037]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T18:28:29.676 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:28:29 smithi134 ceph-mon[35037]: pgmap v671: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 6 op/s 2024-03-28T18:28:29.676 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:28:29 smithi134 ceph-mon[35037]: pgmap v672: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-03-28T18:28:29.676 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:28:29 smithi134 ceph-mon[35037]: from='mgr.15522 ' entity='mgr.x' 2024-03-28T18:28:29.683 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:28:29 smithi079 ceph-mon[35312]: Adjusting osd_memory_target on smithi134 to 176.8M 2024-03-28T18:28:29.683 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:28:29 smithi079 ceph-mon[35312]: Unable to set osd_memory_target on smithi134 to 185393868: error parsing value: Value '185393868' is below minimum 939524096 2024-03-28T18:28:29.683 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:28:29 smithi079 ceph-mon[35312]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-28T18:28:29.683 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:28:29 smithi079 ceph-mon[35312]: from='mgr.15522 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-28T18:28:29.683 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:28:29 smithi079 ceph-mon[35312]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-28T18:28:29.683 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:28:29 smithi079 ceph-mon[35312]: from='mgr.15522 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-28T18:28:29.683 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:28:29 smithi079 ceph-mon[35312]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-03-28T18:28:29.683 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:28:29 smithi079 ceph-mon[35312]: from='mgr.15522 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-03-28T18:28:29.683 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:28:29 smithi079 ceph-mon[35312]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-03-28T18:28:29.683 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:28:29 smithi079 ceph-mon[35312]: from='mgr.15522 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-03-28T18:28:29.683 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:28:29 smithi079 ceph-mon[35312]: Adjusting osd_memory_target on smithi022 to 177.8M 2024-03-28T18:28:29.684 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:28:29 smithi079 ceph-mon[35312]: Unable to set osd_memory_target on smithi022 to 186497740: error parsing value: Value '186497740' is below minimum 939524096 2024-03-28T18:28:29.684 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:28:29 smithi079 ceph-mon[35312]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T18:28:29.684 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:28:29 smithi079 ceph-mon[35312]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T18:28:29.684 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:28:29 smithi079 ceph-mon[35312]: pgmap v671: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 6 op/s 2024-03-28T18:28:29.684 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:28:29 smithi079 ceph-mon[35312]: pgmap v672: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-03-28T18:28:29.684 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:28:29 smithi079 ceph-mon[35312]: from='mgr.15522 ' entity='mgr.x' 2024-03-28T18:28:29.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:28:29 smithi022 ceph-mon[31985]: Adjusting osd_memory_target on smithi134 to 176.8M 2024-03-28T18:28:29.783 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:28:29 smithi022 ceph-mon[31985]: Unable to set osd_memory_target on smithi134 to 185393868: error parsing value: Value '185393868' is below minimum 939524096 2024-03-28T18:28:29.783 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:28:29 smithi022 ceph-mon[31985]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-28T18:28:29.783 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:28:29 smithi022 ceph-mon[31985]: from='mgr.15522 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-28T18:28:29.783 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:28:29 smithi022 ceph-mon[31985]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-28T18:28:29.783 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:28:29 smithi022 ceph-mon[31985]: from='mgr.15522 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-28T18:28:29.783 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:28:29 smithi022 ceph-mon[31985]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-03-28T18:28:29.783 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:28:29 smithi022 ceph-mon[31985]: from='mgr.15522 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-03-28T18:28:29.783 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:28:29 smithi022 ceph-mon[31985]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-03-28T18:28:29.783 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:28:29 smithi022 ceph-mon[31985]: from='mgr.15522 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-03-28T18:28:29.783 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:28:29 smithi022 ceph-mon[31985]: Adjusting osd_memory_target on smithi022 to 177.8M 2024-03-28T18:28:29.783 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:28:29 smithi022 ceph-mon[31985]: Unable to set osd_memory_target on smithi022 to 186497740: error parsing value: Value '186497740' is below minimum 939524096 2024-03-28T18:28:29.783 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:28:29 smithi022 ceph-mon[31985]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T18:28:29.783 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:28:29 smithi022 ceph-mon[31985]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T18:28:29.783 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:28:29 smithi022 ceph-mon[31985]: pgmap v671: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 6 op/s 2024-03-28T18:28:29.783 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:28:29 smithi022 ceph-mon[31985]: pgmap v672: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-03-28T18:28:29.783 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:28:29 smithi022 ceph-mon[31985]: from='mgr.15522 ' entity='mgr.x' 2024-03-28T18:28:30.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:28:30 smithi134 ceph-mon[35037]: 2 slow requests, 0 included below; oldest blocked for > 696.849322 secs 2024-03-28T18:28:30.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:28:30 smithi079 ceph-mon[35312]: 2 slow requests, 0 included below; oldest blocked for > 696.849322 secs 2024-03-28T18:28:30.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:28:30 smithi022 ceph-mon[31985]: 2 slow requests, 0 included below; oldest blocked for > 696.849322 secs 2024-03-28T18:28:31.674 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:28:31 smithi134 ceph-mon[35037]: pgmap v673: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-03-28T18:28:31.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:28:31 smithi079 ceph-mon[35312]: pgmap v673: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-03-28T18:28:31.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:28:31 smithi022 ceph-mon[31985]: pgmap v673: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-03-28T18:28:33.925 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:28:33 smithi134 ceph-mon[35037]: pgmap v674: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-03-28T18:28:33.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:28:33 smithi079 ceph-mon[35312]: pgmap v674: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-03-28T18:28:34.032 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:28:33 smithi022 ceph-mon[31985]: pgmap v674: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-03-28T18:28:34.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:28:34 smithi134 ceph-mon[35037]: 2 slow requests, 0 included below; oldest blocked for > 701.849388 secs 2024-03-28T18:28:34.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:28:34 smithi079 ceph-mon[35312]: 2 slow requests, 0 included below; oldest blocked for > 701.849388 secs 2024-03-28T18:28:34.934 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:28:34 smithi022 ceph-mon[31985]: 2 slow requests, 0 included below; oldest blocked for > 701.849388 secs 2024-03-28T18:28:35.826 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:28:35 smithi134 ceph-mon[35037]: pgmap v675: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-03-28T18:28:35.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:28:35 smithi079 ceph-mon[35312]: pgmap v675: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-03-28T18:28:35.964 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:28:35 smithi022 ceph-mon[31985]: pgmap v675: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-03-28T18:28:37.925 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:28:37 smithi134 ceph-mon[35037]: pgmap v676: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-03-28T18:28:37.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:28:37 smithi079 ceph-mon[35312]: pgmap v676: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-03-28T18:28:38.032 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:28:37 smithi022 ceph-mon[31985]: pgmap v676: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-03-28T18:28:39.925 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:28:39 smithi134 ceph-mon[35037]: pgmap v677: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 7 op/s 2024-03-28T18:28:39.925 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:28:39 smithi134 ceph-mon[35037]: 2 slow requests, 0 included below; oldest blocked for > 706.849469 secs 2024-03-28T18:28:39.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:28:39 smithi079 ceph-mon[35312]: pgmap v677: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 7 op/s 2024-03-28T18:28:39.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:28:39 smithi079 ceph-mon[35312]: 2 slow requests, 0 included below; oldest blocked for > 706.849469 secs 2024-03-28T18:28:40.032 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:28:39 smithi022 ceph-mon[31985]: pgmap v677: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 7 op/s 2024-03-28T18:28:40.032 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:28:39 smithi022 ceph-mon[31985]: 2 slow requests, 0 included below; oldest blocked for > 706.849469 secs 2024-03-28T18:28:41.925 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:28:41 smithi134 ceph-mon[35037]: pgmap v678: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:28:41.933 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:28:41 smithi079 ceph-mon[35312]: pgmap v678: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:28:42.032 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:28:41 smithi022 ceph-mon[31985]: pgmap v678: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:28:43.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:28:43 smithi134 ceph-mon[35037]: pgmap v679: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:28:43.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:28:43 smithi079 ceph-mon[35312]: pgmap v679: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:28:44.032 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:28:43 smithi022 ceph-mon[31985]: pgmap v679: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:28:44.925 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:28:44 smithi134 ceph-mon[35037]: 2 slow requests, 0 included below; oldest blocked for > 711.849588 secs 2024-03-28T18:28:44.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:28:44 smithi079 ceph-mon[35312]: 2 slow requests, 0 included below; oldest blocked for > 711.849588 secs 2024-03-28T18:28:45.032 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:28:44 smithi022 ceph-mon[31985]: 2 slow requests, 0 included below; oldest blocked for > 711.849588 secs 2024-03-28T18:28:45.857 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:28:45 smithi079 ceph-mon[35312]: pgmap v680: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:28:45.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:28:45 smithi134 ceph-mon[35037]: pgmap v680: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:28:46.032 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:28:45 smithi022 ceph-mon[31985]: pgmap v680: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:28:47.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:28:47 smithi134 ceph-mon[35037]: pgmap v681: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:28:47.933 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:28:47 smithi079 ceph-mon[35312]: pgmap v681: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:28:48.032 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:28:47 smithi022 ceph-mon[31985]: pgmap v681: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:28:49.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:28:49 smithi134 ceph-mon[35037]: pgmap v682: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:28:49.925 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:28:49 smithi134 ceph-mon[35037]: 2 slow requests, 0 included below; oldest blocked for > 716.849710 secs 2024-03-28T18:28:49.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:28:49 smithi079 ceph-mon[35312]: pgmap v682: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:28:49.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:28:49 smithi079 ceph-mon[35312]: 2 slow requests, 0 included below; oldest blocked for > 716.849710 secs 2024-03-28T18:28:50.032 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:28:49 smithi022 ceph-mon[31985]: pgmap v682: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:28:50.032 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:28:49 smithi022 ceph-mon[31985]: 2 slow requests, 0 included below; oldest blocked for > 716.849710 secs 2024-03-28T18:28:51.925 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:28:51 smithi134 ceph-mon[35037]: pgmap v683: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:28:51.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:28:51 smithi079 ceph-mon[35312]: pgmap v683: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:28:52.032 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:28:51 smithi022 ceph-mon[31985]: pgmap v683: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:28:53.925 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:28:53 smithi134 ceph-mon[35037]: pgmap v684: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:28:53.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:28:53 smithi079 ceph-mon[35312]: pgmap v684: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:28:54.032 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:28:53 smithi022 ceph-mon[31985]: pgmap v684: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:28:54.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:28:54 smithi134 ceph-mon[35037]: 2 slow requests, 1 included below; oldest blocked for > 721.849832 secs 2024-03-28T18:28:54.925 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:28:54 smithi134 ceph-mon[35037]: slow request 721.849831 seconds old, received at 2024-03-28T18:16:52.457506+0000: peer_request:client.35222:10732 currently acquired locks 2024-03-28T18:28:54.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:28:54 smithi079 ceph-mon[35312]: 2 slow requests, 1 included below; oldest blocked for > 721.849832 secs 2024-03-28T18:28:54.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:28:54 smithi079 ceph-mon[35312]: slow request 721.849831 seconds old, received at 2024-03-28T18:16:52.457506+0000: peer_request:client.35222:10732 currently acquired locks 2024-03-28T18:28:55.032 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:28:54 smithi022 ceph-mon[31985]: 2 slow requests, 1 included below; oldest blocked for > 721.849832 secs 2024-03-28T18:28:55.032 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:28:54 smithi022 ceph-mon[31985]: slow request 721.849831 seconds old, received at 2024-03-28T18:16:52.457506+0000: peer_request:client.35222:10732 currently acquired locks 2024-03-28T18:28:55.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:28:55 smithi134 ceph-mon[35037]: pgmap v685: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:28:55.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:28:55 smithi079 ceph-mon[35312]: pgmap v685: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:28:56.031 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:28:55 smithi022 ceph-mon[31985]: pgmap v685: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:28:57.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:28:57 smithi134 ceph-mon[35037]: pgmap v686: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:28:57.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:28:57 smithi079 ceph-mon[35312]: pgmap v686: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:28:58.032 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:28:57 smithi022 ceph-mon[31985]: pgmap v686: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:28:59.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:28:59 smithi134 ceph-mon[35037]: pgmap v687: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:28:59.925 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:28:59 smithi134 ceph-mon[35037]: 2 slow requests, 0 included below; oldest blocked for > 726.849861 secs 2024-03-28T18:28:59.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:28:59 smithi079 ceph-mon[35312]: pgmap v687: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:28:59.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:28:59 smithi079 ceph-mon[35312]: 2 slow requests, 0 included below; oldest blocked for > 726.849861 secs 2024-03-28T18:29:00.032 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:28:59 smithi022 ceph-mon[31985]: pgmap v687: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:29:00.032 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:28:59 smithi022 ceph-mon[31985]: 2 slow requests, 0 included below; oldest blocked for > 726.849861 secs 2024-03-28T18:29:01.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:29:01 smithi134 ceph-mon[35037]: pgmap v688: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:29:01.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:29:01 smithi079 ceph-mon[35312]: pgmap v688: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:29:02.032 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:29:01 smithi022 ceph-mon[31985]: pgmap v688: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:29:03.925 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:29:03 smithi134 ceph-mon[35037]: pgmap v689: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:29:03.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:29:03 smithi079 ceph-mon[35312]: pgmap v689: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:29:04.032 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:29:03 smithi022 ceph-mon[31985]: pgmap v689: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:29:04.935 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:29:04 smithi022 ceph-mon[31985]: 2 slow requests, 1 included below; oldest blocked for > 731.849947 secs 2024-03-28T18:29:04.935 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:29:04 smithi022 ceph-mon[31985]: slow request 724.152323 seconds old, received at 2024-03-28T18:17:00.155130+0000: client_request(client.35612:96 mksnap #0x1//scheduled-2024-03-28-18_17_00_UTC 2024-03-28T18:17:00.154942+0000 caller_uid=0, caller_gid=0{}) currently failed to xlock, waiting 2024-03-28T18:29:05.175 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:29:04 smithi134 ceph-mon[35037]: 2 slow requests, 1 included below; oldest blocked for > 731.849947 secs 2024-03-28T18:29:05.175 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:29:04 smithi134 ceph-mon[35037]: slow request 724.152323 seconds old, received at 2024-03-28T18:17:00.155130+0000: client_request(client.35612:96 mksnap #0x1//scheduled-2024-03-28-18_17_00_UTC 2024-03-28T18:17:00.154942+0000 caller_uid=0, caller_gid=0{}) currently failed to xlock, waiting 2024-03-28T18:29:05.182 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:29:04 smithi079 ceph-mon[35312]: 2 slow requests, 1 included below; oldest blocked for > 731.849947 secs 2024-03-28T18:29:05.182 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:29:04 smithi079 ceph-mon[35312]: slow request 724.152323 seconds old, received at 2024-03-28T18:17:00.155130+0000: client_request(client.35612:96 mksnap #0x1//scheduled-2024-03-28-18_17_00_UTC 2024-03-28T18:17:00.154942+0000 caller_uid=0, caller_gid=0{}) currently failed to xlock, waiting 2024-03-28T18:29:06.032 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:29:05 smithi022 ceph-mon[31985]: pgmap v690: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:29:06.077 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:29:05 smithi134 ceph-mon[35037]: pgmap v690: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:29:06.182 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:29:05 smithi079 ceph-mon[35312]: pgmap v690: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:29:08.032 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:29:07 smithi022 ceph-mon[31985]: pgmap v691: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:29:08.175 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:29:07 smithi134 ceph-mon[35037]: pgmap v691: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:29:08.182 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:29:07 smithi079 ceph-mon[35312]: pgmap v691: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:29:10.032 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:29:09 smithi022 ceph-mon[31985]: pgmap v692: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:29:10.032 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:29:09 smithi022 ceph-mon[31985]: 2 slow requests, 0 included below; oldest blocked for > 736.849971 secs 2024-03-28T18:29:10.175 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:29:09 smithi134 ceph-mon[35037]: pgmap v692: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:29:10.175 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:29:09 smithi134 ceph-mon[35037]: 2 slow requests, 0 included below; oldest blocked for > 736.849971 secs 2024-03-28T18:29:10.182 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:29:09 smithi079 ceph-mon[35312]: pgmap v692: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:29:10.182 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:29:09 smithi079 ceph-mon[35312]: 2 slow requests, 0 included below; oldest blocked for > 736.849971 secs 2024-03-28T18:29:12.032 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:29:11 smithi022 ceph-mon[31985]: pgmap v693: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:29:12.175 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:29:11 smithi134 ceph-mon[35037]: pgmap v693: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:29:12.182 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:29:11 smithi079 ceph-mon[35312]: pgmap v693: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:29:14.032 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:29:13 smithi022 ceph-mon[31985]: pgmap v694: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:29:14.174 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:29:13 smithi134 ceph-mon[35037]: pgmap v694: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:29:14.182 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:29:13 smithi079 ceph-mon[35312]: pgmap v694: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:29:15.032 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:29:14 smithi022 ceph-mon[31985]: 2 slow requests, 0 included below; oldest blocked for > 741.850065 secs 2024-03-28T18:29:15.084 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:29:14 smithi079 ceph-mon[35312]: 2 slow requests, 0 included below; oldest blocked for > 741.850065 secs 2024-03-28T18:29:15.174 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:29:14 smithi134 ceph-mon[35037]: 2 slow requests, 0 included below; oldest blocked for > 741.850065 secs 2024-03-28T18:29:16.032 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:29:15 smithi022 ceph-mon[31985]: pgmap v695: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:29:16.174 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:29:15 smithi134 ceph-mon[35037]: pgmap v695: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:29:16.182 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:29:15 smithi079 ceph-mon[35312]: pgmap v695: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:29:18.032 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:29:17 smithi022 ceph-mon[31985]: pgmap v696: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:29:18.174 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:29:17 smithi134 ceph-mon[35037]: pgmap v696: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:29:18.182 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:29:17 smithi079 ceph-mon[35312]: pgmap v696: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:29:20.032 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:29:19 smithi022 ceph-mon[31985]: pgmap v697: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:29:20.032 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:29:19 smithi022 ceph-mon[31985]: 2 slow requests, 0 included below; oldest blocked for > 746.850086 secs 2024-03-28T18:29:20.175 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:29:19 smithi134 ceph-mon[35037]: pgmap v697: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:29:20.175 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:29:19 smithi134 ceph-mon[35037]: 2 slow requests, 0 included below; oldest blocked for > 746.850086 secs 2024-03-28T18:29:20.182 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:29:19 smithi079 ceph-mon[35312]: pgmap v697: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:29:20.182 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:29:19 smithi079 ceph-mon[35312]: 2 slow requests, 0 included below; oldest blocked for > 746.850086 secs 2024-03-28T18:29:22.032 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:29:21 smithi022 ceph-mon[31985]: pgmap v698: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:29:22.174 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:29:21 smithi134 ceph-mon[35037]: pgmap v698: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:29:22.182 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:29:21 smithi079 ceph-mon[35312]: pgmap v698: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:29:24.032 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:29:23 smithi022 ceph-mon[31985]: pgmap v699: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:29:24.174 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:29:23 smithi134 ceph-mon[35037]: pgmap v699: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:29:24.183 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:29:23 smithi079 ceph-mon[35312]: pgmap v699: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:29:25.032 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:29:24 smithi022 ceph-mon[31985]: 2 slow requests, 0 included below; oldest blocked for > 751.850152 secs 2024-03-28T18:29:25.174 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:29:24 smithi134 ceph-mon[35037]: 2 slow requests, 0 included below; oldest blocked for > 751.850152 secs 2024-03-28T18:29:25.182 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:29:24 smithi079 ceph-mon[35312]: 2 slow requests, 0 included below; oldest blocked for > 751.850152 secs 2024-03-28T18:29:26.032 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:29:25 smithi022 ceph-mon[31985]: pgmap v700: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:29:26.174 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:29:25 smithi134 ceph-mon[35037]: pgmap v700: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:29:26.182 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:29:25 smithi079 ceph-mon[35312]: pgmap v700: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:29:28.032 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:29:27 smithi022 ceph-mon[31985]: pgmap v701: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:29:28.174 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:29:27 smithi134 ceph-mon[35037]: pgmap v701: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:29:28.182 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:29:27 smithi079 ceph-mon[35312]: pgmap v701: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:29:30.032 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:29:29 smithi022 ceph-mon[31985]: pgmap v702: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:29:30.032 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:29:29 smithi022 ceph-mon[31985]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T18:29:30.032 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:29:29 smithi022 ceph-mon[31985]: 2 slow requests, 0 included below; oldest blocked for > 756.850278 secs 2024-03-28T18:29:30.174 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:29:29 smithi134 ceph-mon[35037]: pgmap v702: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:29:30.175 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:29:29 smithi134 ceph-mon[35037]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T18:29:30.175 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:29:29 smithi134 ceph-mon[35037]: 2 slow requests, 0 included below; oldest blocked for > 756.850278 secs 2024-03-28T18:29:30.182 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:29:29 smithi079 ceph-mon[35312]: pgmap v702: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:29:30.182 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:29:29 smithi079 ceph-mon[35312]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T18:29:30.183 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:29:29 smithi079 ceph-mon[35312]: 2 slow requests, 0 included below; oldest blocked for > 756.850278 secs 2024-03-28T18:29:31.175 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:29:30 smithi134 ceph-mon[35037]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T18:29:31.175 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:29:30 smithi134 ceph-mon[35037]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T18:29:31.175 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:29:30 smithi134 ceph-mon[35037]: from='mgr.15522 ' entity='mgr.x' 2024-03-28T18:29:31.182 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:29:30 smithi079 ceph-mon[35312]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T18:29:31.182 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:29:30 smithi079 ceph-mon[35312]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T18:29:31.182 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:29:30 smithi079 ceph-mon[35312]: from='mgr.15522 ' entity='mgr.x' 2024-03-28T18:29:31.282 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:29:30 smithi022 ceph-mon[31985]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T18:29:31.282 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:29:30 smithi022 ceph-mon[31985]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T18:29:31.282 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:29:30 smithi022 ceph-mon[31985]: from='mgr.15522 ' entity='mgr.x' 2024-03-28T18:29:32.175 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:29:31 smithi134 ceph-mon[35037]: pgmap v703: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 10 op/s 2024-03-28T18:29:32.182 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:29:31 smithi079 ceph-mon[35312]: pgmap v703: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 10 op/s 2024-03-28T18:29:32.282 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:29:31 smithi022 ceph-mon[31985]: pgmap v703: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 10 op/s 2024-03-28T18:29:34.085 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:29:33 smithi022 ceph-mon[31985]: pgmap v704: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 6 op/s 2024-03-28T18:29:34.174 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:29:33 smithi134 ceph-mon[35037]: pgmap v704: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 6 op/s 2024-03-28T18:29:34.182 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:29:33 smithi079 ceph-mon[35312]: pgmap v704: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 6 op/s 2024-03-28T18:29:35.174 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:29:34 smithi134 ceph-mon[35037]: 2 slow requests, 0 included below; oldest blocked for > 761.850363 secs 2024-03-28T18:29:35.182 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:29:34 smithi079 ceph-mon[35312]: 2 slow requests, 0 included below; oldest blocked for > 761.850363 secs 2024-03-28T18:29:35.282 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:29:34 smithi022 ceph-mon[31985]: 2 slow requests, 0 included below; oldest blocked for > 761.850363 secs 2024-03-28T18:29:36.076 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:29:35 smithi134 ceph-mon[35037]: pgmap v705: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 10 op/s 2024-03-28T18:29:36.182 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:29:35 smithi079 ceph-mon[35312]: pgmap v705: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 10 op/s 2024-03-28T18:29:36.282 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:29:35 smithi022 ceph-mon[31985]: pgmap v705: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 10 op/s 2024-03-28T18:29:38.175 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:29:37 smithi134 ceph-mon[35037]: pgmap v706: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 6 op/s 2024-03-28T18:29:38.182 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:29:37 smithi079 ceph-mon[35312]: pgmap v706: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 6 op/s 2024-03-28T18:29:38.282 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:29:37 smithi022 ceph-mon[31985]: pgmap v706: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 6 op/s 2024-03-28T18:29:40.175 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:29:39 smithi134 ceph-mon[35037]: pgmap v707: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 6 op/s 2024-03-28T18:29:40.175 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:29:39 smithi134 ceph-mon[35037]: 2 slow requests, 0 included below; oldest blocked for > 766.850440 secs 2024-03-28T18:29:40.182 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:29:39 smithi079 ceph-mon[35312]: pgmap v707: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 6 op/s 2024-03-28T18:29:40.182 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:29:39 smithi079 ceph-mon[35312]: 2 slow requests, 0 included below; oldest blocked for > 766.850440 secs 2024-03-28T18:29:40.282 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:29:39 smithi022 ceph-mon[31985]: pgmap v707: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 6 op/s 2024-03-28T18:29:40.282 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:29:39 smithi022 ceph-mon[31985]: 2 slow requests, 0 included below; oldest blocked for > 766.850440 secs 2024-03-28T18:29:42.175 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:29:41 smithi134 ceph-mon[35037]: pgmap v708: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 10 op/s 2024-03-28T18:29:42.182 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:29:41 smithi079 ceph-mon[35312]: pgmap v708: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 10 op/s 2024-03-28T18:29:42.282 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:29:41 smithi022 ceph-mon[31985]: pgmap v708: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 10 op/s 2024-03-28T18:29:44.175 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:29:43 smithi134 ceph-mon[35037]: pgmap v709: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:29:44.182 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:29:43 smithi079 ceph-mon[35312]: pgmap v709: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:29:44.282 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:29:43 smithi022 ceph-mon[31985]: pgmap v709: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:29:45.175 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:29:44 smithi134 ceph-mon[35037]: 2 slow requests, 0 included below; oldest blocked for > 771.850526 secs 2024-03-28T18:29:45.182 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:29:44 smithi079 ceph-mon[35312]: 2 slow requests, 0 included below; oldest blocked for > 771.850526 secs 2024-03-28T18:29:45.282 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:29:44 smithi022 ceph-mon[31985]: 2 slow requests, 0 included below; oldest blocked for > 771.850526 secs 2024-03-28T18:29:46.175 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:29:45 smithi134 ceph-mon[35037]: pgmap v710: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:29:46.182 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:29:45 smithi079 ceph-mon[35312]: pgmap v710: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:29:46.282 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:29:45 smithi022 ceph-mon[31985]: pgmap v710: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:29:48.175 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:29:47 smithi134 ceph-mon[35037]: pgmap v711: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:29:48.182 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:29:47 smithi079 ceph-mon[35312]: pgmap v711: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:29:48.282 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:29:47 smithi022 ceph-mon[31985]: pgmap v711: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:29:50.175 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:29:49 smithi134 ceph-mon[35037]: pgmap v712: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:29:50.175 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:29:49 smithi134 ceph-mon[35037]: 2 slow requests, 0 included below; oldest blocked for > 776.850645 secs 2024-03-28T18:29:50.182 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:29:49 smithi079 ceph-mon[35312]: pgmap v712: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:29:50.183 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:29:49 smithi079 ceph-mon[35312]: 2 slow requests, 0 included below; oldest blocked for > 776.850645 secs 2024-03-28T18:29:50.282 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:29:49 smithi022 ceph-mon[31985]: pgmap v712: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:29:50.282 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:29:49 smithi022 ceph-mon[31985]: 2 slow requests, 0 included below; oldest blocked for > 776.850645 secs 2024-03-28T18:29:52.175 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:29:51 smithi134 ceph-mon[35037]: pgmap v713: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:29:52.182 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:29:51 smithi079 ceph-mon[35312]: pgmap v713: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:29:52.282 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:29:51 smithi022 ceph-mon[31985]: pgmap v713: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:29:54.174 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:29:53 smithi134 ceph-mon[35037]: pgmap v714: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:29:54.182 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:29:53 smithi079 ceph-mon[35312]: pgmap v714: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:29:54.281 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:29:53 smithi022 ceph-mon[31985]: pgmap v714: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:29:55.174 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:29:54 smithi134 ceph-mon[35037]: 2 slow requests, 0 included below; oldest blocked for > 781.850766 secs 2024-03-28T18:29:55.182 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:29:54 smithi079 ceph-mon[35312]: 2 slow requests, 0 included below; oldest blocked for > 781.850766 secs 2024-03-28T18:29:55.282 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:29:54 smithi022 ceph-mon[31985]: 2 slow requests, 0 included below; oldest blocked for > 781.850766 secs 2024-03-28T18:29:56.174 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:29:55 smithi134 ceph-mon[35037]: pgmap v715: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:29:56.182 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:29:55 smithi079 ceph-mon[35312]: pgmap v715: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:29:56.282 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:29:55 smithi022 ceph-mon[31985]: pgmap v715: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:29:58.175 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:29:57 smithi134 ceph-mon[35037]: pgmap v716: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:29:58.182 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:29:57 smithi079 ceph-mon[35312]: pgmap v716: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:29:58.282 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:29:57 smithi022 ceph-mon[31985]: pgmap v716: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:30:00.175 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:29:59 smithi134 ceph-mon[35037]: pgmap v717: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:30:00.175 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:29:59 smithi134 ceph-mon[35037]: 2 slow requests, 0 included below; oldest blocked for > 786.850834 secs 2024-03-28T18:30:00.182 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:29:59 smithi079 ceph-mon[35312]: pgmap v717: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:30:00.182 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:29:59 smithi079 ceph-mon[35312]: 2 slow requests, 0 included below; oldest blocked for > 786.850834 secs 2024-03-28T18:30:00.282 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:29:59 smithi022 ceph-mon[31985]: pgmap v717: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:30:00.282 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:29:59 smithi022 ceph-mon[31985]: 2 slow requests, 0 included below; oldest blocked for > 786.850834 secs 2024-03-28T18:30:01.174 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:30:00 smithi134 ceph-mon[35037]: overall HEALTH_WARN 4 failed cephadm daemon(s); 1 MDSs report slow requests 2024-03-28T18:30:01.182 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:30:00 smithi079 ceph-mon[35312]: overall HEALTH_WARN 4 failed cephadm daemon(s); 1 MDSs report slow requests 2024-03-28T18:30:01.282 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:30:00 smithi022 ceph-mon[31985]: overall HEALTH_WARN 4 failed cephadm daemon(s); 1 MDSs report slow requests 2024-03-28T18:30:02.174 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:30:01 smithi134 ceph-mon[35037]: pgmap v718: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:30:02.182 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:30:01 smithi079 ceph-mon[35312]: pgmap v718: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:30:02.281 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:30:01 smithi022 ceph-mon[31985]: pgmap v718: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:30:04.175 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:30:03 smithi134 ceph-mon[35037]: pgmap v719: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:30:04.182 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:30:03 smithi079 ceph-mon[35312]: pgmap v719: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:30:04.282 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:30:03 smithi022 ceph-mon[31985]: pgmap v719: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:30:05.182 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:30:04 smithi079 ceph-mon[35312]: 2 slow requests, 0 included below; oldest blocked for > 791.850942 secs 2024-03-28T18:30:05.282 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:30:04 smithi022 ceph-mon[31985]: 2 slow requests, 0 included below; oldest blocked for > 791.850942 secs 2024-03-28T18:30:05.425 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:30:04 smithi134 ceph-mon[35037]: 2 slow requests, 0 included below; oldest blocked for > 791.850942 secs 2024-03-28T18:30:06.182 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:30:05 smithi079 ceph-mon[35312]: pgmap v720: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:30:06.282 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:30:05 smithi022 ceph-mon[31985]: pgmap v720: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:30:06.326 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:30:05 smithi134 ceph-mon[35037]: pgmap v720: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:30:08.282 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:30:07 smithi022 ceph-mon[31985]: pgmap v721: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:30:08.424 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:30:07 smithi134 ceph-mon[35037]: pgmap v721: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:30:08.432 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:30:07 smithi079 ceph-mon[35312]: pgmap v721: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:30:10.282 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:30:09 smithi022 ceph-mon[31985]: pgmap v722: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:30:10.282 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:30:09 smithi022 ceph-mon[31985]: 2 slow requests, 0 included below; oldest blocked for > 796.851015 secs 2024-03-28T18:30:10.424 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:30:09 smithi134 ceph-mon[35037]: pgmap v722: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:30:10.425 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:30:09 smithi134 ceph-mon[35037]: 2 slow requests, 0 included below; oldest blocked for > 796.851015 secs 2024-03-28T18:30:10.432 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:30:09 smithi079 ceph-mon[35312]: pgmap v722: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:30:10.433 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:30:09 smithi079 ceph-mon[35312]: 2 slow requests, 0 included below; oldest blocked for > 796.851015 secs 2024-03-28T18:30:12.282 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:30:11 smithi022 ceph-mon[31985]: pgmap v723: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:30:12.424 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:30:11 smithi134 ceph-mon[35037]: pgmap v723: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:30:12.432 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:30:11 smithi079 ceph-mon[35312]: pgmap v723: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:30:14.282 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:30:13 smithi022 ceph-mon[31985]: pgmap v724: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:30:14.424 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:30:13 smithi134 ceph-mon[35037]: pgmap v724: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:30:14.432 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:30:13 smithi079 ceph-mon[35312]: pgmap v724: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:30:15.282 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:30:14 smithi022 ceph-mon[31985]: 2 slow requests, 0 included below; oldest blocked for > 801.851102 secs 2024-03-28T18:30:15.334 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:30:14 smithi079 ceph-mon[35312]: 2 slow requests, 0 included below; oldest blocked for > 801.851102 secs 2024-03-28T18:30:15.424 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:30:14 smithi134 ceph-mon[35037]: 2 slow requests, 0 included below; oldest blocked for > 801.851102 secs 2024-03-28T18:30:16.282 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:30:15 smithi022 ceph-mon[31985]: pgmap v725: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:30:16.424 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:30:15 smithi134 ceph-mon[35037]: pgmap v725: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:30:16.432 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:30:15 smithi079 ceph-mon[35312]: pgmap v725: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:30:18.282 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:30:17 smithi022 ceph-mon[31985]: pgmap v726: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:30:18.424 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:30:17 smithi134 ceph-mon[35037]: pgmap v726: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:30:18.432 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:30:17 smithi079 ceph-mon[35312]: pgmap v726: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:30:20.282 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:30:19 smithi022 ceph-mon[31985]: pgmap v727: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:30:20.282 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:30:19 smithi022 ceph-mon[31985]: 2 slow requests, 0 included below; oldest blocked for > 806.851124 secs 2024-03-28T18:30:20.424 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:30:19 smithi134 ceph-mon[35037]: pgmap v727: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:30:20.425 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:30:19 smithi134 ceph-mon[35037]: 2 slow requests, 0 included below; oldest blocked for > 806.851124 secs 2024-03-28T18:30:20.432 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:30:19 smithi079 ceph-mon[35312]: pgmap v727: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:30:20.433 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:30:19 smithi079 ceph-mon[35312]: 2 slow requests, 0 included below; oldest blocked for > 806.851124 secs 2024-03-28T18:30:22.282 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:30:21 smithi022 ceph-mon[31985]: pgmap v728: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:30:22.424 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:30:21 smithi134 ceph-mon[35037]: pgmap v728: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:30:22.433 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:30:21 smithi079 ceph-mon[35312]: pgmap v728: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:30:24.282 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:30:23 smithi022 ceph-mon[31985]: pgmap v729: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:30:24.424 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:30:23 smithi134 ceph-mon[35037]: pgmap v729: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:30:24.432 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:30:23 smithi079 ceph-mon[35312]: pgmap v729: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:30:25.282 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:30:24 smithi022 ceph-mon[31985]: 2 slow requests, 0 included below; oldest blocked for > 811.851217 secs 2024-03-28T18:30:25.425 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:30:24 smithi134 ceph-mon[35037]: 2 slow requests, 0 included below; oldest blocked for > 811.851217 secs 2024-03-28T18:30:25.432 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:30:24 smithi079 ceph-mon[35312]: 2 slow requests, 0 included below; oldest blocked for > 811.851217 secs 2024-03-28T18:30:26.282 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:30:26 smithi022 ceph-mon[31985]: pgmap v730: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:30:26.424 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:30:26 smithi134 ceph-mon[35037]: pgmap v730: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:30:26.432 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:30:26 smithi079 ceph-mon[35312]: pgmap v730: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:30:28.282 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:30:28 smithi022 ceph-mon[31985]: pgmap v731: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:30:28.424 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:30:28 smithi134 ceph-mon[35037]: pgmap v731: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:30:28.432 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:30:28 smithi079 ceph-mon[35312]: pgmap v731: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:30:30.282 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:30:30 smithi022 ceph-mon[31985]: pgmap v732: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:30:30.282 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:30:30 smithi022 ceph-mon[31985]: 2 slow requests, 0 included below; oldest blocked for > 816.851335 secs 2024-03-28T18:30:30.425 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:30:30 smithi134 ceph-mon[35037]: pgmap v732: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:30:30.425 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:30:30 smithi134 ceph-mon[35037]: 2 slow requests, 0 included below; oldest blocked for > 816.851335 secs 2024-03-28T18:30:30.432 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:30:30 smithi079 ceph-mon[35312]: pgmap v732: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:30:30.432 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:30:30 smithi079 ceph-mon[35312]: 2 slow requests, 0 included below; oldest blocked for > 816.851335 secs 2024-03-28T18:30:31.282 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:30:31 smithi022 ceph-mon[31985]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T18:30:31.424 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:30:31 smithi134 ceph-mon[35037]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T18:30:31.432 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:30:31 smithi079 ceph-mon[35312]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T18:30:32.282 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:30:32 smithi022 ceph-mon[31985]: pgmap v733: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:30:32.424 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:30:32 smithi134 ceph-mon[35037]: pgmap v733: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:30:32.432 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:30:32 smithi079 ceph-mon[35312]: pgmap v733: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:30:33.425 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:30:33 smithi134 ceph-mon[35037]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T18:30:33.425 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:30:33 smithi134 ceph-mon[35037]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T18:30:33.425 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:30:33 smithi134 ceph-mon[35037]: pgmap v734: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 6 op/s 2024-03-28T18:30:33.425 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:30:33 smithi134 ceph-mon[35037]: pgmap v735: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-03-28T18:30:33.425 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:30:33 smithi134 ceph-mon[35037]: from='mgr.15522 ' entity='mgr.x' 2024-03-28T18:30:33.433 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:30:33 smithi079 ceph-mon[35312]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T18:30:33.433 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:30:33 smithi079 ceph-mon[35312]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T18:30:33.433 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:30:33 smithi079 ceph-mon[35312]: pgmap v734: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 6 op/s 2024-03-28T18:30:33.433 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:30:33 smithi079 ceph-mon[35312]: pgmap v735: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-03-28T18:30:33.433 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:30:33 smithi079 ceph-mon[35312]: from='mgr.15522 ' entity='mgr.x' 2024-03-28T18:30:33.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:30:33 smithi022 ceph-mon[31985]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T18:30:33.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:30:33 smithi022 ceph-mon[31985]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T18:30:33.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:30:33 smithi022 ceph-mon[31985]: pgmap v734: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 6 op/s 2024-03-28T18:30:33.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:30:33 smithi022 ceph-mon[31985]: pgmap v735: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-03-28T18:30:33.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:30:33 smithi022 ceph-mon[31985]: from='mgr.15522 ' entity='mgr.x' 2024-03-28T18:30:35.425 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:30:35 smithi134 ceph-mon[35037]: pgmap v736: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 KiB/s rd, 4 op/s 2024-03-28T18:30:35.425 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:30:35 smithi134 ceph-mon[35037]: 2 slow requests, 0 included below; oldest blocked for > 821.851416 secs 2024-03-28T18:30:35.432 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:30:35 smithi079 ceph-mon[35312]: pgmap v736: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 KiB/s rd, 4 op/s 2024-03-28T18:30:35.433 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:30:35 smithi079 ceph-mon[35312]: 2 slow requests, 0 included below; oldest blocked for > 821.851416 secs 2024-03-28T18:30:35.531 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:30:35 smithi022 ceph-mon[31985]: pgmap v736: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 KiB/s rd, 4 op/s 2024-03-28T18:30:35.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:30:35 smithi022 ceph-mon[31985]: 2 slow requests, 0 included below; oldest blocked for > 821.851416 secs 2024-03-28T18:30:37.424 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:30:37 smithi134 ceph-mon[35037]: pgmap v737: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-03-28T18:30:37.432 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:30:37 smithi079 ceph-mon[35312]: pgmap v737: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-03-28T18:30:37.531 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:30:37 smithi022 ceph-mon[31985]: pgmap v737: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-03-28T18:30:39.425 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:30:39 smithi134 ceph-mon[35037]: pgmap v738: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-03-28T18:30:39.432 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:30:39 smithi079 ceph-mon[35312]: pgmap v738: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-03-28T18:30:39.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:30:39 smithi022 ceph-mon[31985]: pgmap v738: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-03-28T18:30:40.432 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:30:40 smithi079 ceph-mon[35312]: 2 slow requests, 0 included below; oldest blocked for > 826.851492 secs 2024-03-28T18:30:40.531 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:30:40 smithi022 ceph-mon[31985]: 2 slow requests, 0 included below; oldest blocked for > 826.851492 secs 2024-03-28T18:30:40.674 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:30:40 smithi134 ceph-mon[35037]: 2 slow requests, 0 included below; oldest blocked for > 826.851492 secs 2024-03-28T18:30:41.433 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:30:41 smithi079 ceph-mon[35312]: pgmap v739: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-03-28T18:30:41.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:30:41 smithi022 ceph-mon[31985]: pgmap v739: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-03-28T18:30:41.674 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:30:41 smithi134 ceph-mon[35037]: pgmap v739: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-03-28T18:30:42.433 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:30:42 smithi079 ceph-mon[35312]: pgmap v740: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 7 op/s 2024-03-28T18:30:42.531 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:30:42 smithi022 ceph-mon[31985]: pgmap v740: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 7 op/s 2024-03-28T18:30:42.674 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:30:42 smithi134 ceph-mon[35037]: pgmap v740: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 7 op/s 2024-03-28T18:30:45.425 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:30:45 smithi134 ceph-mon[35037]: pgmap v741: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:30:45.426 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:30:45 smithi134 ceph-mon[35037]: 2 slow requests, 0 included below; oldest blocked for > 831.851644 secs 2024-03-28T18:30:45.432 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:30:45 smithi079 ceph-mon[35312]: pgmap v741: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:30:45.432 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:30:45 smithi079 ceph-mon[35312]: 2 slow requests, 0 included below; oldest blocked for > 831.851644 secs 2024-03-28T18:30:45.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:30:45 smithi022 ceph-mon[31985]: pgmap v741: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:30:45.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:30:45 smithi022 ceph-mon[31985]: 2 slow requests, 0 included below; oldest blocked for > 831.851644 secs 2024-03-28T18:30:47.424 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:30:47 smithi134 ceph-mon[35037]: pgmap v742: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:30:47.433 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:30:47 smithi079 ceph-mon[35312]: pgmap v742: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:30:47.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:30:47 smithi022 ceph-mon[31985]: pgmap v742: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:30:49.424 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:30:49 smithi134 ceph-mon[35037]: pgmap v743: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:30:49.432 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:30:49 smithi079 ceph-mon[35312]: pgmap v743: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:30:49.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:30:49 smithi022 ceph-mon[31985]: pgmap v743: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:30:50.433 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:30:50 smithi079 ceph-mon[35312]: 2 slow requests, 0 included below; oldest blocked for > 836.851720 secs 2024-03-28T18:30:50.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:30:50 smithi022 ceph-mon[31985]: 2 slow requests, 0 included below; oldest blocked for > 836.851720 secs 2024-03-28T18:30:50.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:30:50 smithi134 ceph-mon[35037]: 2 slow requests, 0 included below; oldest blocked for > 836.851720 secs 2024-03-28T18:30:51.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:30:51 smithi022 ceph-mon[31985]: pgmap v744: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:30:51.674 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:30:51 smithi134 ceph-mon[35037]: pgmap v744: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:30:51.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:30:51 smithi079 ceph-mon[35312]: pgmap v744: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:30:52.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:30:52 smithi022 ceph-mon[31985]: pgmap v745: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:30:52.674 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:30:52 smithi134 ceph-mon[35037]: pgmap v745: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:30:52.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:30:52 smithi079 ceph-mon[35312]: pgmap v745: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:30:55.425 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:30:55 smithi134 ceph-mon[35037]: pgmap v746: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:30:55.425 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:30:55 smithi134 ceph-mon[35037]: 2 slow requests, 0 included below; oldest blocked for > 841.851836 secs 2024-03-28T18:30:55.432 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:30:55 smithi079 ceph-mon[35312]: pgmap v746: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:30:55.432 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:30:55 smithi079 ceph-mon[35312]: 2 slow requests, 0 included below; oldest blocked for > 841.851836 secs 2024-03-28T18:30:55.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:30:55 smithi022 ceph-mon[31985]: pgmap v746: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:30:55.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:30:55 smithi022 ceph-mon[31985]: 2 slow requests, 0 included below; oldest blocked for > 841.851836 secs 2024-03-28T18:30:57.425 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:30:57 smithi134 ceph-mon[35037]: pgmap v747: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:30:57.432 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:30:57 smithi079 ceph-mon[35312]: pgmap v747: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:30:57.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:30:57 smithi022 ceph-mon[31985]: pgmap v747: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:30:59.425 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:30:59 smithi134 ceph-mon[35037]: pgmap v748: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:30:59.432 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:30:59 smithi079 ceph-mon[35312]: pgmap v748: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:30:59.531 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:30:59 smithi022 ceph-mon[31985]: pgmap v748: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:31:00.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:31:00 smithi022 ceph-mon[31985]: 2 slow requests, 0 included below; oldest blocked for > 846.851889 secs 2024-03-28T18:31:00.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:31:00 smithi022 ceph-mon[31985]: pgmap v749: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:31:00.674 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:31:00 smithi134 ceph-mon[35037]: 2 slow requests, 0 included below; oldest blocked for > 846.851889 secs 2024-03-28T18:31:00.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:31:00 smithi134 ceph-mon[35037]: pgmap v749: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:31:00.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:31:00 smithi079 ceph-mon[35312]: 2 slow requests, 0 included below; oldest blocked for > 846.851889 secs 2024-03-28T18:31:00.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:31:00 smithi079 ceph-mon[35312]: pgmap v749: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:31:03.425 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:31:03 smithi134 ceph-mon[35037]: pgmap v750: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:31:03.432 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:31:03 smithi079 ceph-mon[35312]: pgmap v750: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:31:03.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:31:03 smithi022 ceph-mon[31985]: pgmap v750: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:31:05.425 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:31:05 smithi134 ceph-mon[35037]: pgmap v751: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:31:05.425 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:31:05 smithi134 ceph-mon[35037]: 2 slow requests, 0 included below; oldest blocked for > 851.851988 secs 2024-03-28T18:31:05.433 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:31:05 smithi079 ceph-mon[35312]: pgmap v751: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:31:05.433 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:31:05 smithi079 ceph-mon[35312]: 2 slow requests, 0 included below; oldest blocked for > 851.851988 secs 2024-03-28T18:31:05.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:31:05 smithi022 ceph-mon[31985]: pgmap v751: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:31:05.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:31:05 smithi022 ceph-mon[31985]: 2 slow requests, 0 included below; oldest blocked for > 851.851988 secs 2024-03-28T18:31:07.433 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:31:07 smithi079 ceph-mon[35312]: pgmap v752: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 9 op/s 2024-03-28T18:31:07.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:31:07 smithi022 ceph-mon[31985]: pgmap v752: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 9 op/s 2024-03-28T18:31:07.674 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:31:07 smithi134 ceph-mon[35037]: pgmap v752: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 9 op/s 2024-03-28T18:31:08.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:31:08 smithi022 ceph-mon[31985]: pgmap v753: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T18:31:08.674 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:31:08 smithi134 ceph-mon[35037]: pgmap v753: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T18:31:08.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:31:08 smithi079 ceph-mon[35312]: pgmap v753: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T18:31:09.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:31:09 smithi022 ceph-mon[31985]: 2 slow requests, 0 included below; oldest blocked for > 856.852075 secs 2024-03-28T18:31:09.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:31:09 smithi134 ceph-mon[35037]: 2 slow requests, 0 included below; oldest blocked for > 856.852075 secs 2024-03-28T18:31:09.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:31:09 smithi079 ceph-mon[35312]: 2 slow requests, 0 included below; oldest blocked for > 856.852075 secs 2024-03-28T18:31:10.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:31:10 smithi022 ceph-mon[31985]: pgmap v754: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 9 op/s 2024-03-28T18:31:10.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:31:10 smithi134 ceph-mon[35037]: pgmap v754: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 9 op/s 2024-03-28T18:31:10.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:31:10 smithi079 ceph-mon[35312]: pgmap v754: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 9 op/s 2024-03-28T18:31:13.425 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:31:13 smithi134 ceph-mon[35037]: pgmap v755: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T18:31:13.432 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:31:13 smithi079 ceph-mon[35312]: pgmap v755: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T18:31:13.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:31:13 smithi022 ceph-mon[31985]: pgmap v755: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T18:31:15.433 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:31:15 smithi079 ceph-mon[35312]: pgmap v756: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T18:31:15.433 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:31:15 smithi079 ceph-mon[35312]: 2 slow requests, 0 included below; oldest blocked for > 861.852160 secs 2024-03-28T18:31:15.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:31:15 smithi022 ceph-mon[31985]: pgmap v756: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T18:31:15.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:31:15 smithi022 ceph-mon[31985]: 2 slow requests, 0 included below; oldest blocked for > 861.852160 secs 2024-03-28T18:31:15.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:31:15 smithi134 ceph-mon[35037]: pgmap v756: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T18:31:15.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:31:15 smithi134 ceph-mon[35037]: 2 slow requests, 0 included below; oldest blocked for > 861.852160 secs 2024-03-28T18:31:16.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:31:16 smithi022 ceph-mon[31985]: pgmap v757: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 9 op/s 2024-03-28T18:31:16.674 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:31:16 smithi134 ceph-mon[35037]: pgmap v757: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 9 op/s 2024-03-28T18:31:16.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:31:16 smithi079 ceph-mon[35312]: pgmap v757: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 9 op/s 2024-03-28T18:31:19.424 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:31:19 smithi134 ceph-mon[35037]: pgmap v758: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:31:19.432 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:31:19 smithi079 ceph-mon[35312]: pgmap v758: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:31:19.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:31:19 smithi022 ceph-mon[31985]: pgmap v758: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:31:20.433 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:31:20 smithi079 ceph-mon[35312]: 2 slow requests, 0 included below; oldest blocked for > 866.852226 secs 2024-03-28T18:31:20.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:31:20 smithi022 ceph-mon[31985]: 2 slow requests, 0 included below; oldest blocked for > 866.852226 secs 2024-03-28T18:31:20.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:31:20 smithi134 ceph-mon[35037]: 2 slow requests, 0 included below; oldest blocked for > 866.852226 secs 2024-03-28T18:31:21.433 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:31:21 smithi079 ceph-mon[35312]: pgmap v759: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:31:21.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:31:21 smithi022 ceph-mon[31985]: pgmap v759: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:31:21.674 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:31:21 smithi134 ceph-mon[35037]: pgmap v759: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:31:22.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:31:22 smithi022 ceph-mon[31985]: pgmap v760: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:31:22.674 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:31:22 smithi134 ceph-mon[35037]: pgmap v760: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:31:22.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:31:22 smithi079 ceph-mon[35312]: pgmap v760: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:31:25.433 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:31:25 smithi079 ceph-mon[35312]: pgmap v761: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:31:25.433 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:31:25 smithi079 ceph-mon[35312]: 2 slow requests, 0 included below; oldest blocked for > 871.852307 secs 2024-03-28T18:31:25.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:31:25 smithi022 ceph-mon[31985]: pgmap v761: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:31:25.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:31:25 smithi022 ceph-mon[31985]: 2 slow requests, 0 included below; oldest blocked for > 871.852307 secs 2024-03-28T18:31:25.674 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:31:25 smithi134 ceph-mon[35037]: pgmap v761: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:31:25.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:31:25 smithi134 ceph-mon[35037]: 2 slow requests, 0 included below; oldest blocked for > 871.852307 secs 2024-03-28T18:31:26.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:31:26 smithi022 ceph-mon[31985]: pgmap v762: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 9 op/s 2024-03-28T18:31:26.674 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:31:26 smithi134 ceph-mon[35037]: pgmap v762: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 9 op/s 2024-03-28T18:31:26.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:31:26 smithi079 ceph-mon[35312]: pgmap v762: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 9 op/s 2024-03-28T18:31:29.432 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:31:29 smithi079 ceph-mon[35312]: pgmap v763: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T18:31:29.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:31:29 smithi022 ceph-mon[31985]: pgmap v763: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T18:31:29.674 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:31:29 smithi134 ceph-mon[35037]: pgmap v763: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T18:31:30.433 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:31:30 smithi079 ceph-mon[35312]: 2 slow requests, 0 included below; oldest blocked for > 876.852431 secs 2024-03-28T18:31:30.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:31:30 smithi022 ceph-mon[31985]: 2 slow requests, 0 included below; oldest blocked for > 876.852431 secs 2024-03-28T18:31:30.674 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:31:30 smithi134 ceph-mon[35037]: 2 slow requests, 0 included below; oldest blocked for > 876.852431 secs 2024-03-28T18:31:31.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:31:31 smithi022 ceph-mon[31985]: pgmap v764: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 9 op/s 2024-03-28T18:31:31.674 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:31:31 smithi134 ceph-mon[35037]: pgmap v764: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 9 op/s 2024-03-28T18:31:31.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:31:31 smithi079 ceph-mon[35312]: pgmap v764: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 9 op/s 2024-03-28T18:31:32.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:31:32 smithi022 ceph-mon[31985]: pgmap v765: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T18:31:32.674 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:31:32 smithi134 ceph-mon[35037]: pgmap v765: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T18:31:32.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:31:32 smithi079 ceph-mon[35312]: pgmap v765: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T18:31:33.531 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:31:33 smithi022 ceph-mon[31985]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T18:31:33.674 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:31:33 smithi134 ceph-mon[35037]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T18:31:33.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:31:33 smithi079 ceph-mon[35312]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T18:31:34.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:31:34 smithi022 ceph-mon[31985]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T18:31:34.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:31:34 smithi022 ceph-mon[31985]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T18:31:34.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:31:34 smithi022 ceph-mon[31985]: pgmap v766: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T18:31:34.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:31:34 smithi022 ceph-mon[31985]: pgmap v767: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 7 op/s 2024-03-28T18:31:34.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:31:34 smithi022 ceph-mon[31985]: from='mgr.15522 ' entity='mgr.x' 2024-03-28T18:31:34.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:31:34 smithi134 ceph-mon[35037]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T18:31:34.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:31:34 smithi134 ceph-mon[35037]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T18:31:34.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:31:34 smithi134 ceph-mon[35037]: pgmap v766: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T18:31:34.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:31:34 smithi134 ceph-mon[35037]: pgmap v767: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 7 op/s 2024-03-28T18:31:34.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:31:34 smithi134 ceph-mon[35037]: from='mgr.15522 ' entity='mgr.x' 2024-03-28T18:31:34.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:31:34 smithi079 ceph-mon[35312]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T18:31:34.683 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:31:34 smithi079 ceph-mon[35312]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T18:31:34.683 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:31:34 smithi079 ceph-mon[35312]: pgmap v766: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T18:31:34.683 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:31:34 smithi079 ceph-mon[35312]: pgmap v767: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 7 op/s 2024-03-28T18:31:34.683 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:31:34 smithi079 ceph-mon[35312]: from='mgr.15522 ' entity='mgr.x' 2024-03-28T18:31:35.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:31:35 smithi022 ceph-mon[31985]: 2 slow requests, 0 included below; oldest blocked for > 881.852460 secs 2024-03-28T18:31:35.674 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:31:35 smithi134 ceph-mon[35037]: 2 slow requests, 0 included below; oldest blocked for > 881.852460 secs 2024-03-28T18:31:35.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:31:35 smithi079 ceph-mon[35312]: 2 slow requests, 0 included below; oldest blocked for > 881.852460 secs 2024-03-28T18:31:36.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:31:36 smithi022 ceph-mon[31985]: pgmap v768: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 7 op/s 2024-03-28T18:31:36.674 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:31:36 smithi134 ceph-mon[35037]: pgmap v768: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 7 op/s 2024-03-28T18:31:36.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:31:36 smithi079 ceph-mon[35312]: pgmap v768: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 7 op/s 2024-03-28T18:31:39.378 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:31:39 smithi134 ceph-mon[35037]: pgmap v769: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 7 op/s 2024-03-28T18:31:39.432 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:31:39 smithi079 ceph-mon[35312]: pgmap v769: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 7 op/s 2024-03-28T18:31:39.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:31:39 smithi022 ceph-mon[31985]: pgmap v769: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 7 op/s 2024-03-28T18:31:40.425 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:31:40 smithi134 ceph-mon[35037]: 2 slow requests, 0 included below; oldest blocked for > 886.852527 secs 2024-03-28T18:31:40.432 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:31:40 smithi079 ceph-mon[35312]: 2 slow requests, 0 included below; oldest blocked for > 886.852527 secs 2024-03-28T18:31:40.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:31:40 smithi022 ceph-mon[31985]: 2 slow requests, 0 included below; oldest blocked for > 886.852527 secs 2024-03-28T18:31:41.424 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:31:41 smithi134 ceph-mon[35037]: pgmap v770: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 7 op/s 2024-03-28T18:31:41.432 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:31:41 smithi079 ceph-mon[35312]: pgmap v770: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 7 op/s 2024-03-28T18:31:41.531 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:31:41 smithi022 ceph-mon[31985]: pgmap v770: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 7 op/s 2024-03-28T18:31:43.425 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:31:43 smithi134 ceph-mon[35037]: pgmap v771: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 7 op/s 2024-03-28T18:31:43.432 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:31:43 smithi079 ceph-mon[35312]: pgmap v771: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 7 op/s 2024-03-28T18:31:43.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:31:43 smithi022 ceph-mon[31985]: pgmap v771: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 7 op/s 2024-03-28T18:31:45.425 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:31:45 smithi134 ceph-mon[35037]: pgmap v772: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 7 op/s 2024-03-28T18:31:45.425 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:31:45 smithi134 ceph-mon[35037]: 2 slow requests, 0 included below; oldest blocked for > 891.852606 secs 2024-03-28T18:31:45.433 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:31:45 smithi079 ceph-mon[35312]: pgmap v772: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 7 op/s 2024-03-28T18:31:45.433 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:31:45 smithi079 ceph-mon[35312]: 2 slow requests, 0 included below; oldest blocked for > 891.852606 secs 2024-03-28T18:31:45.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:31:45 smithi022 ceph-mon[31985]: pgmap v772: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 7 op/s 2024-03-28T18:31:45.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:31:45 smithi022 ceph-mon[31985]: 2 slow requests, 0 included below; oldest blocked for > 891.852606 secs 2024-03-28T18:31:47.424 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:31:47 smithi134 ceph-mon[35037]: pgmap v773: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 9 op/s 2024-03-28T18:31:47.432 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:31:47 smithi079 ceph-mon[35312]: pgmap v773: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 9 op/s 2024-03-28T18:31:47.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:31:47 smithi022 ceph-mon[31985]: pgmap v773: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 9 op/s 2024-03-28T18:31:49.425 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:31:49 smithi134 ceph-mon[35037]: pgmap v774: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:31:49.432 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:31:49 smithi079 ceph-mon[35312]: pgmap v774: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:31:49.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:31:49 smithi022 ceph-mon[31985]: pgmap v774: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:31:50.425 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:31:50 smithi134 ceph-mon[35037]: 2 slow requests, 0 included below; oldest blocked for > 896.852679 secs 2024-03-28T18:31:50.432 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:31:50 smithi079 ceph-mon[35312]: 2 slow requests, 0 included below; oldest blocked for > 896.852679 secs 2024-03-28T18:31:50.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:31:50 smithi022 ceph-mon[31985]: 2 slow requests, 0 included below; oldest blocked for > 896.852679 secs 2024-03-28T18:31:51.424 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:31:51 smithi134 ceph-mon[35037]: pgmap v775: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:31:51.432 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:31:51 smithi079 ceph-mon[35312]: pgmap v775: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:31:51.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:31:51 smithi022 ceph-mon[31985]: pgmap v775: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:31:53.425 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:31:53 smithi134 ceph-mon[35037]: pgmap v776: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:31:53.432 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:31:53 smithi079 ceph-mon[35312]: pgmap v776: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:31:53.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:31:53 smithi022 ceph-mon[31985]: pgmap v776: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:31:55.425 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:31:55 smithi134 ceph-mon[35037]: pgmap v777: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:31:55.425 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:31:55 smithi134 ceph-mon[35037]: 2 slow requests, 0 included below; oldest blocked for > 901.852749 secs 2024-03-28T18:31:55.432 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:31:55 smithi079 ceph-mon[35312]: pgmap v777: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:31:55.432 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:31:55 smithi079 ceph-mon[35312]: 2 slow requests, 0 included below; oldest blocked for > 901.852749 secs 2024-03-28T18:31:55.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:31:55 smithi022 ceph-mon[31985]: pgmap v777: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:31:55.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:31:55 smithi022 ceph-mon[31985]: 2 slow requests, 0 included below; oldest blocked for > 901.852749 secs 2024-03-28T18:31:57.424 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:31:57 smithi134 ceph-mon[35037]: pgmap v778: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:31:57.432 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:31:57 smithi079 ceph-mon[35312]: pgmap v778: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:31:57.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:31:57 smithi022 ceph-mon[31985]: pgmap v778: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:31:59.425 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:31:59 smithi134 ceph-mon[35037]: pgmap v779: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:31:59.432 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:31:59 smithi079 ceph-mon[35312]: pgmap v779: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:31:59.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:31:59 smithi022 ceph-mon[31985]: pgmap v779: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:32:00.424 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:32:00 smithi134 ceph-mon[35037]: 2 slow requests, 0 included below; oldest blocked for > 906.852819 secs 2024-03-28T18:32:00.432 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:32:00 smithi079 ceph-mon[35312]: 2 slow requests, 0 included below; oldest blocked for > 906.852819 secs 2024-03-28T18:32:00.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:32:00 smithi022 ceph-mon[31985]: 2 slow requests, 0 included below; oldest blocked for > 906.852819 secs 2024-03-28T18:32:01.425 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:32:01 smithi134 ceph-mon[35037]: pgmap v780: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:32:01.433 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:32:01 smithi079 ceph-mon[35312]: pgmap v780: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:32:01.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:32:01 smithi022 ceph-mon[31985]: pgmap v780: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:32:03.424 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:32:03 smithi134 ceph-mon[35037]: pgmap v781: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:32:03.432 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:32:03 smithi079 ceph-mon[35312]: pgmap v781: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:32:03.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:32:03 smithi022 ceph-mon[31985]: pgmap v781: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:32:05.425 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:32:05 smithi134 ceph-mon[35037]: pgmap v782: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:32:05.425 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:32:05 smithi134 ceph-mon[35037]: 2 slow requests, 0 included below; oldest blocked for > 911.852921 secs 2024-03-28T18:32:05.432 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:32:05 smithi079 ceph-mon[35312]: pgmap v782: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:32:05.433 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:32:05 smithi079 ceph-mon[35312]: 2 slow requests, 0 included below; oldest blocked for > 911.852921 secs 2024-03-28T18:32:05.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:32:05 smithi022 ceph-mon[31985]: pgmap v782: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:32:05.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:32:05 smithi022 ceph-mon[31985]: 2 slow requests, 0 included below; oldest blocked for > 911.852921 secs 2024-03-28T18:32:07.425 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:32:07 smithi134 ceph-mon[35037]: pgmap v783: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:32:07.432 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:32:07 smithi079 ceph-mon[35312]: pgmap v783: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:32:07.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:32:07 smithi022 ceph-mon[31985]: pgmap v783: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:32:09.424 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:32:09 smithi134 ceph-mon[35037]: pgmap v784: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:32:09.432 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:32:09 smithi079 ceph-mon[35312]: pgmap v784: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:32:09.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:32:09 smithi022 ceph-mon[31985]: pgmap v784: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:32:10.425 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:32:10 smithi134 ceph-mon[35037]: 2 slow requests, 0 included below; oldest blocked for > 916.852959 secs 2024-03-28T18:32:10.432 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:32:10 smithi079 ceph-mon[35312]: 2 slow requests, 0 included below; oldest blocked for > 916.852959 secs 2024-03-28T18:32:10.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:32:10 smithi022 ceph-mon[31985]: 2 slow requests, 0 included below; oldest blocked for > 916.852959 secs 2024-03-28T18:32:11.432 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:32:11 smithi079 ceph-mon[35312]: pgmap v785: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:32:11.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:32:11 smithi022 ceph-mon[31985]: pgmap v785: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:32:11.674 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:32:11 smithi134 ceph-mon[35037]: pgmap v785: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:32:13.433 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:32:13 smithi079 ceph-mon[35312]: pgmap v786: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:32:13.533 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:32:13 smithi022 ceph-mon[31985]: pgmap v786: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:32:13.674 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:32:13 smithi134 ceph-mon[35037]: pgmap v786: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:32:15.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:32:15 smithi022 ceph-mon[31985]: pgmap v787: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:32:15.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:32:15 smithi022 ceph-mon[31985]: 2 slow requests, 0 included below; oldest blocked for > 921.853041 secs 2024-03-28T18:32:15.674 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:32:15 smithi134 ceph-mon[35037]: pgmap v787: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:32:15.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:32:15 smithi134 ceph-mon[35037]: 2 slow requests, 0 included below; oldest blocked for > 921.853041 secs 2024-03-28T18:32:15.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:32:15 smithi079 ceph-mon[35312]: pgmap v787: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:32:15.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:32:15 smithi079 ceph-mon[35312]: 2 slow requests, 0 included below; oldest blocked for > 921.853041 secs 2024-03-28T18:32:16.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:32:16 smithi022 ceph-mon[31985]: pgmap v788: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:32:16.674 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:32:16 smithi134 ceph-mon[35037]: pgmap v788: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:32:16.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:32:16 smithi079 ceph-mon[35312]: pgmap v788: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:32:19.424 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:32:19 smithi134 ceph-mon[35037]: pgmap v789: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:32:19.432 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:32:19 smithi079 ceph-mon[35312]: pgmap v789: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:32:19.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:32:19 smithi022 ceph-mon[31985]: pgmap v789: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:32:20.424 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:32:20 smithi134 ceph-mon[35037]: 2 slow requests, 0 included below; oldest blocked for > 926.853098 secs 2024-03-28T18:32:20.432 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:32:20 smithi079 ceph-mon[35312]: 2 slow requests, 0 included below; oldest blocked for > 926.853098 secs 2024-03-28T18:32:20.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:32:20 smithi022 ceph-mon[31985]: 2 slow requests, 0 included below; oldest blocked for > 926.853098 secs 2024-03-28T18:32:21.424 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:32:21 smithi134 ceph-mon[35037]: pgmap v790: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:32:21.432 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:32:21 smithi079 ceph-mon[35312]: pgmap v790: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:32:21.531 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:32:21 smithi022 ceph-mon[31985]: pgmap v790: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:32:23.425 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:32:23 smithi134 ceph-mon[35037]: pgmap v791: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:32:23.432 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:32:23 smithi079 ceph-mon[35312]: pgmap v791: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:32:23.533 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:32:23 smithi022 ceph-mon[31985]: pgmap v791: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:32:25.425 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:32:25 smithi134 ceph-mon[35037]: pgmap v792: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:32:25.425 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:32:25 smithi134 ceph-mon[35037]: 2 slow requests, 0 included below; oldest blocked for > 931.853122 secs 2024-03-28T18:32:25.432 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:32:25 smithi079 ceph-mon[35312]: pgmap v792: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:32:25.433 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:32:25 smithi079 ceph-mon[35312]: 2 slow requests, 0 included below; oldest blocked for > 931.853122 secs 2024-03-28T18:32:25.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:32:25 smithi022 ceph-mon[31985]: pgmap v792: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:32:25.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:32:25 smithi022 ceph-mon[31985]: 2 slow requests, 0 included below; oldest blocked for > 931.853122 secs 2024-03-28T18:32:27.425 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:32:27 smithi134 ceph-mon[35037]: pgmap v793: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:32:27.432 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:32:27 smithi079 ceph-mon[35312]: pgmap v793: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:32:27.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:32:27 smithi022 ceph-mon[31985]: pgmap v793: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:32:29.424 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:32:29 smithi134 ceph-mon[35037]: pgmap v794: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:32:29.433 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:32:29 smithi079 ceph-mon[35312]: pgmap v794: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:32:29.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:32:29 smithi022 ceph-mon[31985]: pgmap v794: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:32:30.425 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:32:30 smithi134 ceph-mon[35037]: 2 slow requests, 0 included below; oldest blocked for > 936.853244 secs 2024-03-28T18:32:30.432 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:32:30 smithi079 ceph-mon[35312]: 2 slow requests, 0 included below; oldest blocked for > 936.853244 secs 2024-03-28T18:32:30.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:32:30 smithi022 ceph-mon[31985]: 2 slow requests, 0 included below; oldest blocked for > 936.853244 secs 2024-03-28T18:32:31.425 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:32:31 smithi134 ceph-mon[35037]: pgmap v795: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:32:31.433 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:32:31 smithi079 ceph-mon[35312]: pgmap v795: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:32:31.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:32:31 smithi022 ceph-mon[31985]: pgmap v795: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:32:33.425 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:32:33 smithi134 ceph-mon[35037]: pgmap v796: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:32:33.432 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:32:33 smithi079 ceph-mon[35312]: pgmap v796: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:32:33.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:32:33 smithi022 ceph-mon[31985]: pgmap v796: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:32:35.425 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:32:35 smithi134 ceph-mon[35037]: pgmap v797: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:32:35.425 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:32:35 smithi134 ceph-mon[35037]: 2 slow requests, 0 included below; oldest blocked for > 941.853309 secs 2024-03-28T18:32:35.425 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:32:35 smithi134 ceph-mon[35037]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T18:32:35.432 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:32:35 smithi079 ceph-mon[35312]: pgmap v797: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:32:35.433 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:32:35 smithi079 ceph-mon[35312]: 2 slow requests, 0 included below; oldest blocked for > 941.853309 secs 2024-03-28T18:32:35.433 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:32:35 smithi079 ceph-mon[35312]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T18:32:35.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:32:35 smithi022 ceph-mon[31985]: pgmap v797: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:32:35.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:32:35 smithi022 ceph-mon[31985]: 2 slow requests, 0 included below; oldest blocked for > 941.853309 secs 2024-03-28T18:32:35.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:32:35 smithi022 ceph-mon[31985]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T18:32:36.425 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:32:36 smithi134 ceph-mon[35037]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T18:32:36.425 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:32:36 smithi134 ceph-mon[35037]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T18:32:36.425 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:32:36 smithi134 ceph-mon[35037]: from='mgr.15522 ' entity='mgr.x' 2024-03-28T18:32:36.432 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:32:36 smithi079 ceph-mon[35312]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T18:32:36.433 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:32:36 smithi079 ceph-mon[35312]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T18:32:36.433 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:32:36 smithi079 ceph-mon[35312]: from='mgr.15522 ' entity='mgr.x' 2024-03-28T18:32:36.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:32:36 smithi022 ceph-mon[31985]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T18:32:36.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:32:36 smithi022 ceph-mon[31985]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T18:32:36.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:32:36 smithi022 ceph-mon[31985]: from='mgr.15522 ' entity='mgr.x' 2024-03-28T18:32:37.425 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:32:37 smithi134 ceph-mon[35037]: pgmap v798: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 10 op/s 2024-03-28T18:32:37.425 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:32:37 smithi134 ceph-mon[35037]: pgmap v799: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-03-28T18:32:37.432 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:32:37 smithi079 ceph-mon[35312]: pgmap v798: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 10 op/s 2024-03-28T18:32:37.432 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:32:37 smithi079 ceph-mon[35312]: pgmap v799: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-03-28T18:32:37.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:32:37 smithi022 ceph-mon[31985]: pgmap v798: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 10 op/s 2024-03-28T18:32:37.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:32:37 smithi022 ceph-mon[31985]: pgmap v799: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-03-28T18:32:39.425 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:32:39 smithi134 ceph-mon[35037]: pgmap v800: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-03-28T18:32:39.432 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:32:39 smithi079 ceph-mon[35312]: pgmap v800: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-03-28T18:32:39.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:32:39 smithi022 ceph-mon[31985]: pgmap v800: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-03-28T18:32:40.425 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:32:40 smithi134 ceph-mon[35037]: 2 slow requests, 0 included below; oldest blocked for > 946.853385 secs 2024-03-28T18:32:40.432 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:32:40 smithi079 ceph-mon[35312]: 2 slow requests, 0 included below; oldest blocked for > 946.853385 secs 2024-03-28T18:32:40.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:32:40 smithi022 ceph-mon[31985]: 2 slow requests, 0 included below; oldest blocked for > 946.853385 secs 2024-03-28T18:32:41.425 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:32:41 smithi134 ceph-mon[35037]: pgmap v801: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-03-28T18:32:41.432 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:32:41 smithi079 ceph-mon[35312]: pgmap v801: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-03-28T18:32:41.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:32:41 smithi022 ceph-mon[31985]: pgmap v801: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-03-28T18:32:43.424 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:32:43 smithi134 ceph-mon[35037]: pgmap v802: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-03-28T18:32:43.432 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:32:43 smithi079 ceph-mon[35312]: pgmap v802: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-03-28T18:32:43.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:32:43 smithi022 ceph-mon[31985]: pgmap v802: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-03-28T18:32:45.433 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:32:45 smithi079 ceph-mon[35312]: pgmap v803: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-03-28T18:32:45.433 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:32:45 smithi079 ceph-mon[35312]: 2 slow requests, 0 included below; oldest blocked for > 951.853464 secs 2024-03-28T18:32:45.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:32:45 smithi022 ceph-mon[31985]: pgmap v803: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-03-28T18:32:45.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:32:45 smithi022 ceph-mon[31985]: 2 slow requests, 0 included below; oldest blocked for > 951.853464 secs 2024-03-28T18:32:45.674 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:32:45 smithi134 ceph-mon[35037]: pgmap v803: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-03-28T18:32:45.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:32:45 smithi134 ceph-mon[35037]: 2 slow requests, 0 included below; oldest blocked for > 951.853464 secs 2024-03-28T18:32:47.433 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:32:47 smithi079 ceph-mon[35312]: pgmap v804: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 7 op/s 2024-03-28T18:32:47.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:32:47 smithi022 ceph-mon[31985]: pgmap v804: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 7 op/s 2024-03-28T18:32:47.674 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:32:47 smithi134 ceph-mon[35037]: pgmap v804: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 7 op/s 2024-03-28T18:32:49.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:32:49 smithi022 ceph-mon[31985]: pgmap v805: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:32:49.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:32:49 smithi134 ceph-mon[35037]: pgmap v805: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:32:49.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:32:49 smithi079 ceph-mon[35312]: pgmap v805: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:32:50.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:32:50 smithi022 ceph-mon[31985]: 2 slow requests, 0 included below; oldest blocked for > 956.853581 secs 2024-03-28T18:32:50.674 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:32:50 smithi134 ceph-mon[35037]: 2 slow requests, 0 included below; oldest blocked for > 956.853581 secs 2024-03-28T18:32:50.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:32:50 smithi079 ceph-mon[35312]: 2 slow requests, 0 included below; oldest blocked for > 956.853581 secs 2024-03-28T18:32:51.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:32:51 smithi022 ceph-mon[31985]: pgmap v806: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:32:51.674 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:32:51 smithi134 ceph-mon[35037]: pgmap v806: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:32:51.685 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:32:51 smithi079 ceph-mon[35312]: pgmap v806: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:32:52.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:32:52 smithi022 ceph-mon[31985]: pgmap v807: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:32:52.674 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:32:52 smithi134 ceph-mon[35037]: pgmap v807: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:32:52.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:32:52 smithi079 ceph-mon[35312]: pgmap v807: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:32:55.175 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:32:54 smithi134 ceph-mon[35037]: pgmap v808: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:32:55.175 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:32:54 smithi134 ceph-mon[35037]: 2 slow requests, 0 included below; oldest blocked for > 961.853649 secs 2024-03-28T18:32:55.183 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:32:54 smithi079 ceph-mon[35312]: pgmap v808: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:32:55.183 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:32:54 smithi079 ceph-mon[35312]: 2 slow requests, 0 included below; oldest blocked for > 961.853649 secs 2024-03-28T18:32:55.282 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:32:54 smithi022 ceph-mon[31985]: pgmap v808: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:32:55.282 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:32:54 smithi022 ceph-mon[31985]: 2 slow requests, 0 included below; oldest blocked for > 961.853649 secs 2024-03-28T18:32:57.174 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:32:56 smithi134 ceph-mon[35037]: pgmap v809: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:32:57.182 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:32:56 smithi079 ceph-mon[35312]: pgmap v809: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:32:57.282 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:32:56 smithi022 ceph-mon[31985]: pgmap v809: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:32:59.175 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:32:58 smithi134 ceph-mon[35037]: pgmap v810: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:32:59.182 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:32:58 smithi079 ceph-mon[35312]: pgmap v810: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:32:59.282 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:32:58 smithi022 ceph-mon[31985]: pgmap v810: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:33:00.174 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:32:59 smithi134 ceph-mon[35037]: 2 slow requests, 0 included below; oldest blocked for > 966.853670 secs 2024-03-28T18:33:00.182 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:32:59 smithi079 ceph-mon[35312]: 2 slow requests, 0 included below; oldest blocked for > 966.853670 secs 2024-03-28T18:33:00.282 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:32:59 smithi022 ceph-mon[31985]: 2 slow requests, 0 included below; oldest blocked for > 966.853670 secs 2024-03-28T18:33:01.174 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:33:00 smithi134 ceph-mon[35037]: pgmap v811: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:33:01.182 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:33:00 smithi079 ceph-mon[35312]: pgmap v811: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:33:01.282 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:33:00 smithi022 ceph-mon[31985]: pgmap v811: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:33:03.174 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:33:02 smithi134 ceph-mon[35037]: pgmap v812: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:33:03.182 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:33:02 smithi079 ceph-mon[35312]: pgmap v812: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:33:03.281 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:33:02 smithi022 ceph-mon[31985]: pgmap v812: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:33:05.174 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:33:04 smithi134 ceph-mon[35037]: pgmap v813: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:33:05.175 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:33:04 smithi134 ceph-mon[35037]: 2 slow requests, 0 included below; oldest blocked for > 971.853745 secs 2024-03-28T18:33:05.182 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:33:04 smithi079 ceph-mon[35312]: pgmap v813: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:33:05.183 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:33:04 smithi079 ceph-mon[35312]: 2 slow requests, 0 included below; oldest blocked for > 971.853745 secs 2024-03-28T18:33:05.282 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:33:04 smithi022 ceph-mon[31985]: pgmap v813: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:33:05.282 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:33:04 smithi022 ceph-mon[31985]: 2 slow requests, 0 included below; oldest blocked for > 971.853745 secs 2024-03-28T18:33:07.174 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:33:06 smithi134 ceph-mon[35037]: pgmap v814: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:33:07.182 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:33:06 smithi079 ceph-mon[35312]: pgmap v814: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:33:07.282 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:33:06 smithi022 ceph-mon[31985]: pgmap v814: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:33:09.175 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:33:08 smithi134 ceph-mon[35037]: pgmap v815: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:33:09.182 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:33:08 smithi079 ceph-mon[35312]: pgmap v815: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:33:09.281 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:33:08 smithi022 ceph-mon[31985]: pgmap v815: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:33:10.174 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:33:09 smithi134 ceph-mon[35037]: 2 slow requests, 0 included below; oldest blocked for > 976.853810 secs 2024-03-28T18:33:10.182 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:33:09 smithi079 ceph-mon[35312]: 2 slow requests, 0 included below; oldest blocked for > 976.853810 secs 2024-03-28T18:33:10.282 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:33:09 smithi022 ceph-mon[31985]: 2 slow requests, 0 included below; oldest blocked for > 976.853810 secs 2024-03-28T18:33:11.175 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:33:10 smithi134 ceph-mon[35037]: pgmap v816: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:33:11.182 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:33:10 smithi079 ceph-mon[35312]: pgmap v816: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:33:11.282 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:33:10 smithi022 ceph-mon[31985]: pgmap v816: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:33:13.175 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:33:12 smithi134 ceph-mon[35037]: pgmap v817: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:33:13.182 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:33:12 smithi079 ceph-mon[35312]: pgmap v817: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:33:13.282 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:33:12 smithi022 ceph-mon[31985]: pgmap v817: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:33:15.177 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:33:14 smithi134 ceph-mon[35037]: pgmap v818: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:33:15.178 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:33:14 smithi134 ceph-mon[35037]: 2 slow requests, 0 included below; oldest blocked for > 981.853883 secs 2024-03-28T18:33:15.182 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:33:14 smithi079 ceph-mon[35312]: pgmap v818: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:33:15.183 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:33:14 smithi079 ceph-mon[35312]: 2 slow requests, 0 included below; oldest blocked for > 981.853883 secs 2024-03-28T18:33:15.282 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:33:14 smithi022 ceph-mon[31985]: pgmap v818: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:33:15.282 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:33:14 smithi022 ceph-mon[31985]: 2 slow requests, 0 included below; oldest blocked for > 981.853883 secs 2024-03-28T18:33:17.175 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:33:16 smithi134 ceph-mon[35037]: pgmap v819: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:33:17.182 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:33:16 smithi079 ceph-mon[35312]: pgmap v819: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:33:17.282 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:33:16 smithi022 ceph-mon[31985]: pgmap v819: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:33:19.174 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:33:18 smithi134 ceph-mon[35037]: pgmap v820: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:33:19.182 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:33:18 smithi079 ceph-mon[35312]: pgmap v820: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:33:19.281 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:33:18 smithi022 ceph-mon[31985]: pgmap v820: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:33:20.175 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:33:19 smithi134 ceph-mon[35037]: 2 slow requests, 0 included below; oldest blocked for > 986.853947 secs 2024-03-28T18:33:20.182 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:33:19 smithi079 ceph-mon[35312]: 2 slow requests, 0 included below; oldest blocked for > 986.853947 secs 2024-03-28T18:33:20.282 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:33:19 smithi022 ceph-mon[31985]: 2 slow requests, 0 included below; oldest blocked for > 986.853947 secs 2024-03-28T18:33:21.174 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:33:20 smithi134 ceph-mon[35037]: pgmap v821: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:33:21.182 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:33:20 smithi079 ceph-mon[35312]: pgmap v821: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:33:21.281 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:33:20 smithi022 ceph-mon[31985]: pgmap v821: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:33:23.183 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:33:22 smithi079 ceph-mon[35312]: pgmap v822: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:33:23.282 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:33:22 smithi022 ceph-mon[31985]: pgmap v822: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:33:23.425 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:33:22 smithi134 ceph-mon[35037]: pgmap v822: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:33:25.183 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:33:24 smithi079 ceph-mon[35312]: pgmap v823: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:33:25.183 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:33:24 smithi079 ceph-mon[35312]: 2 slow requests, 0 included below; oldest blocked for > 991.854067 secs 2024-03-28T18:33:25.282 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:33:24 smithi022 ceph-mon[31985]: pgmap v823: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:33:25.282 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:33:24 smithi022 ceph-mon[31985]: 2 slow requests, 0 included below; oldest blocked for > 991.854067 secs 2024-03-28T18:33:25.424 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:33:24 smithi134 ceph-mon[35037]: pgmap v823: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:33:25.425 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:33:24 smithi134 ceph-mon[35037]: 2 slow requests, 0 included below; oldest blocked for > 991.854067 secs 2024-03-28T18:33:27.282 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:33:26 smithi022 ceph-mon[31985]: pgmap v824: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:33:27.425 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:33:26 smithi134 ceph-mon[35037]: pgmap v824: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:33:27.434 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:33:26 smithi079 ceph-mon[35312]: pgmap v824: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:33:29.282 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:33:28 smithi022 ceph-mon[31985]: pgmap v825: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:33:29.425 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:33:28 smithi134 ceph-mon[35037]: pgmap v825: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:33:29.432 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:33:28 smithi079 ceph-mon[35312]: pgmap v825: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:33:30.282 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:33:29 smithi022 ceph-mon[31985]: 2 slow requests, 0 included below; oldest blocked for > 996.854091 secs 2024-03-28T18:33:30.424 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:33:29 smithi134 ceph-mon[35037]: 2 slow requests, 0 included below; oldest blocked for > 996.854091 secs 2024-03-28T18:33:30.432 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:33:29 smithi079 ceph-mon[35312]: 2 slow requests, 0 included below; oldest blocked for > 996.854091 secs 2024-03-28T18:33:31.282 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:33:30 smithi022 ceph-mon[31985]: pgmap v826: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:33:31.424 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:33:30 smithi134 ceph-mon[35037]: pgmap v826: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:33:31.432 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:33:30 smithi079 ceph-mon[35312]: pgmap v826: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:33:33.282 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:33:32 smithi022 ceph-mon[31985]: pgmap v827: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:33:33.424 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:33:32 smithi134 ceph-mon[35037]: pgmap v827: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:33:33.432 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:33:32 smithi079 ceph-mon[35312]: pgmap v827: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:33:35.282 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:33:34 smithi022 ceph-mon[31985]: pgmap v828: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:33:35.282 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:33:34 smithi022 ceph-mon[31985]: 2 slow requests, 0 included below; oldest blocked for > 1001.854168 secs 2024-03-28T18:33:35.425 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:33:34 smithi134 ceph-mon[35037]: pgmap v828: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:33:35.425 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:33:34 smithi134 ceph-mon[35037]: 2 slow requests, 0 included below; oldest blocked for > 1001.854168 secs 2024-03-28T18:33:35.432 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:33:34 smithi079 ceph-mon[35312]: pgmap v828: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:33:35.432 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:33:34 smithi079 ceph-mon[35312]: 2 slow requests, 0 included below; oldest blocked for > 1001.854168 secs 2024-03-28T18:33:37.282 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:33:36 smithi022 ceph-mon[31985]: pgmap v829: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:33:37.282 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:33:36 smithi022 ceph-mon[31985]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T18:33:37.424 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:33:36 smithi134 ceph-mon[35037]: pgmap v829: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:33:37.425 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:33:36 smithi134 ceph-mon[35037]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T18:33:37.432 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:33:36 smithi079 ceph-mon[35312]: pgmap v829: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:33:37.432 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:33:36 smithi079 ceph-mon[35312]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T18:33:39.182 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:33:38 smithi079 ceph-mon[35312]: pgmap v830: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:33:39.282 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:33:38 smithi022 ceph-mon[31985]: pgmap v830: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:33:39.424 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:33:38 smithi134 ceph-mon[35037]: pgmap v830: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:33:40.272 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:33:39 smithi079 ceph-mon[35312]: 2 slow requests, 0 included below; oldest blocked for > 1006.854262 secs 2024-03-28T18:33:40.282 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:33:39 smithi022 ceph-mon[31985]: 2 slow requests, 0 included below; oldest blocked for > 1006.854262 secs 2024-03-28T18:33:40.326 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:33:39 smithi134 ceph-mon[35037]: 2 slow requests, 0 included below; oldest blocked for > 1006.854262 secs 2024-03-28T18:33:41.282 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:33:40 smithi022 ceph-mon[31985]: pgmap v831: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:33:41.424 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:33:40 smithi134 ceph-mon[35037]: pgmap v831: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:33:41.432 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:33:40 smithi079 ceph-mon[35312]: pgmap v831: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:33:43.282 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:33:43 smithi022 ceph-mon[31985]: pgmap v832: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:33:43.282 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:33:43 smithi022 ceph-mon[31985]: from='mgr.15522 ' entity='mgr.x' 2024-03-28T18:33:43.282 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:33:43 smithi022 ceph-mon[31985]: from='mgr.15522 ' entity='mgr.x' 2024-03-28T18:33:43.425 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:33:43 smithi134 ceph-mon[35037]: pgmap v832: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:33:43.425 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:33:43 smithi134 ceph-mon[35037]: from='mgr.15522 ' entity='mgr.x' 2024-03-28T18:33:43.425 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:33:43 smithi134 ceph-mon[35037]: from='mgr.15522 ' entity='mgr.x' 2024-03-28T18:33:43.433 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:33:43 smithi079 ceph-mon[35312]: pgmap v832: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:33:43.433 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:33:43 smithi079 ceph-mon[35312]: from='mgr.15522 ' entity='mgr.x' 2024-03-28T18:33:43.433 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:33:43 smithi079 ceph-mon[35312]: from='mgr.15522 ' entity='mgr.x' 2024-03-28T18:33:44.925 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:33:44 smithi134 ceph-mon[35037]: from='mgr.15522 ' entity='mgr.x' 2024-03-28T18:33:44.925 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:33:44 smithi134 ceph-mon[35037]: from='mgr.15522 ' entity='mgr.x' 2024-03-28T18:33:44.925 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:33:44 smithi134 ceph-mon[35037]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T18:33:44.925 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:33:44 smithi134 ceph-mon[35037]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T18:33:44.925 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:33:44 smithi134 ceph-mon[35037]: pgmap v833: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 6 op/s 2024-03-28T18:33:44.925 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:33:44 smithi134 ceph-mon[35037]: pgmap v834: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-03-28T18:33:44.926 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:33:44 smithi134 ceph-mon[35037]: from='mgr.15522 ' entity='mgr.x' 2024-03-28T18:33:44.926 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:33:44 smithi134 ceph-mon[35037]: 2 slow requests, 0 included below; oldest blocked for > 1011.854382 secs 2024-03-28T18:33:44.933 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:33:44 smithi079 ceph-mon[35312]: from='mgr.15522 ' entity='mgr.x' 2024-03-28T18:33:44.933 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:33:44 smithi079 ceph-mon[35312]: from='mgr.15522 ' entity='mgr.x' 2024-03-28T18:33:44.933 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:33:44 smithi079 ceph-mon[35312]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T18:33:44.933 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:33:44 smithi079 ceph-mon[35312]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T18:33:44.933 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:33:44 smithi079 ceph-mon[35312]: pgmap v833: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 6 op/s 2024-03-28T18:33:44.933 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:33:44 smithi079 ceph-mon[35312]: pgmap v834: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-03-28T18:33:44.933 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:33:44 smithi079 ceph-mon[35312]: from='mgr.15522 ' entity='mgr.x' 2024-03-28T18:33:44.933 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:33:44 smithi079 ceph-mon[35312]: 2 slow requests, 0 included below; oldest blocked for > 1011.854382 secs 2024-03-28T18:33:45.032 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:33:44 smithi022 ceph-mon[31985]: from='mgr.15522 ' entity='mgr.x' 2024-03-28T18:33:45.032 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:33:44 smithi022 ceph-mon[31985]: from='mgr.15522 ' entity='mgr.x' 2024-03-28T18:33:45.032 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:33:44 smithi022 ceph-mon[31985]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T18:33:45.032 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:33:44 smithi022 ceph-mon[31985]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T18:33:45.032 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:33:44 smithi022 ceph-mon[31985]: pgmap v833: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 6 op/s 2024-03-28T18:33:45.032 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:33:44 smithi022 ceph-mon[31985]: pgmap v834: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-03-28T18:33:45.032 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:33:44 smithi022 ceph-mon[31985]: from='mgr.15522 ' entity='mgr.x' 2024-03-28T18:33:45.033 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:33:44 smithi022 ceph-mon[31985]: 2 slow requests, 0 included below; oldest blocked for > 1011.854382 secs 2024-03-28T18:33:47.032 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:33:46 smithi022 ceph-mon[31985]: pgmap v835: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-03-28T18:33:47.175 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:33:46 smithi134 ceph-mon[35037]: pgmap v835: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-03-28T18:33:47.182 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:33:46 smithi079 ceph-mon[35312]: pgmap v835: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-03-28T18:33:49.032 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:33:48 smithi022 ceph-mon[31985]: pgmap v836: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-03-28T18:33:49.174 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:33:48 smithi134 ceph-mon[35037]: pgmap v836: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-03-28T18:33:49.183 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:33:48 smithi079 ceph-mon[35312]: pgmap v836: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-03-28T18:33:50.032 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:33:49 smithi022 ceph-mon[31985]: 2 slow requests, 0 included below; oldest blocked for > 1016.854499 secs 2024-03-28T18:33:50.174 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:33:49 smithi134 ceph-mon[35037]: 2 slow requests, 0 included below; oldest blocked for > 1016.854499 secs 2024-03-28T18:33:50.182 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:33:49 smithi079 ceph-mon[35312]: 2 slow requests, 0 included below; oldest blocked for > 1016.854499 secs 2024-03-28T18:33:51.032 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:33:50 smithi022 ceph-mon[31985]: pgmap v837: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-03-28T18:33:51.175 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:33:50 smithi134 ceph-mon[35037]: pgmap v837: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-03-28T18:33:51.182 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:33:50 smithi079 ceph-mon[35312]: pgmap v837: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-03-28T18:33:53.032 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:33:52 smithi022 ceph-mon[31985]: pgmap v838: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-03-28T18:33:53.174 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:33:52 smithi134 ceph-mon[35037]: pgmap v838: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-03-28T18:33:53.182 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:33:52 smithi079 ceph-mon[35312]: pgmap v838: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-03-28T18:33:55.032 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:33:54 smithi022 ceph-mon[31985]: pgmap v839: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 7 op/s 2024-03-28T18:33:55.032 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:33:54 smithi022 ceph-mon[31985]: 2 slow requests, 0 included below; oldest blocked for > 1021.854569 secs 2024-03-28T18:33:55.174 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:33:54 smithi134 ceph-mon[35037]: pgmap v839: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 7 op/s 2024-03-28T18:33:55.174 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:33:54 smithi134 ceph-mon[35037]: 2 slow requests, 0 included below; oldest blocked for > 1021.854569 secs 2024-03-28T18:33:55.182 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:33:54 smithi079 ceph-mon[35312]: pgmap v839: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 7 op/s 2024-03-28T18:33:55.183 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:33:54 smithi079 ceph-mon[35312]: 2 slow requests, 0 included below; oldest blocked for > 1021.854569 secs 2024-03-28T18:33:57.032 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:33:56 smithi022 ceph-mon[31985]: pgmap v840: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:33:57.174 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:33:56 smithi134 ceph-mon[35037]: pgmap v840: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:33:57.182 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:33:56 smithi079 ceph-mon[35312]: pgmap v840: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:33:59.032 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:33:58 smithi022 ceph-mon[31985]: pgmap v841: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:33:59.174 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:33:58 smithi134 ceph-mon[35037]: pgmap v841: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:33:59.182 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:33:58 smithi079 ceph-mon[35312]: pgmap v841: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:34:00.032 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:33:59 smithi022 ceph-mon[31985]: 2 slow requests, 0 included below; oldest blocked for > 1026.854639 secs 2024-03-28T18:34:00.175 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:33:59 smithi134 ceph-mon[35037]: 2 slow requests, 0 included below; oldest blocked for > 1026.854639 secs 2024-03-28T18:34:00.182 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:33:59 smithi079 ceph-mon[35312]: 2 slow requests, 0 included below; oldest blocked for > 1026.854639 secs 2024-03-28T18:34:01.032 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:34:00 smithi022 ceph-mon[31985]: pgmap v842: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:34:01.174 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:34:00 smithi134 ceph-mon[35037]: pgmap v842: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:34:01.182 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:34:00 smithi079 ceph-mon[35312]: pgmap v842: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:34:03.032 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:34:02 smithi022 ceph-mon[31985]: pgmap v843: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:34:03.174 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:34:02 smithi134 ceph-mon[35037]: pgmap v843: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:34:03.182 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:34:02 smithi079 ceph-mon[35312]: pgmap v843: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:34:04.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:34:04 smithi134 ceph-mon[35037]: pgmap v844: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:34:04.925 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:34:04 smithi134 ceph-mon[35037]: 2 slow requests, 0 included below; oldest blocked for > 1031.854756 secs 2024-03-28T18:34:04.933 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:34:04 smithi079 ceph-mon[35312]: pgmap v844: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:34:04.933 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:34:04 smithi079 ceph-mon[35312]: 2 slow requests, 0 included below; oldest blocked for > 1031.854756 secs 2024-03-28T18:34:05.032 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:34:04 smithi022 ceph-mon[31985]: pgmap v844: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:34:05.032 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:34:04 smithi022 ceph-mon[31985]: 2 slow requests, 0 included below; oldest blocked for > 1031.854756 secs 2024-03-28T18:34:07.032 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:34:06 smithi022 ceph-mon[31985]: pgmap v845: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:34:07.174 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:34:06 smithi134 ceph-mon[35037]: pgmap v845: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:34:07.183 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:34:06 smithi079 ceph-mon[35312]: pgmap v845: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:34:09.032 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:34:08 smithi022 ceph-mon[31985]: pgmap v846: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:34:09.174 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:34:08 smithi134 ceph-mon[35037]: pgmap v846: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:34:09.182 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:34:08 smithi079 ceph-mon[35312]: pgmap v846: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:34:09.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:34:09 smithi022 ceph-mon[31985]: 2 slow requests, 0 included below; oldest blocked for > 1036.854831 secs 2024-03-28T18:34:10.077 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:34:09 smithi134 ceph-mon[35037]: 2 slow requests, 0 included below; oldest blocked for > 1036.854831 secs 2024-03-28T18:34:10.183 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:34:09 smithi079 ceph-mon[35312]: 2 slow requests, 0 included below; oldest blocked for > 1036.854831 secs 2024-03-28T18:34:11.032 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:34:10 smithi022 ceph-mon[31985]: pgmap v847: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:34:11.174 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:34:10 smithi134 ceph-mon[35037]: pgmap v847: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:34:11.182 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:34:10 smithi079 ceph-mon[35312]: pgmap v847: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:34:13.032 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:34:12 smithi022 ceph-mon[31985]: pgmap v848: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:34:13.174 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:34:12 smithi134 ceph-mon[35037]: pgmap v848: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:34:13.182 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:34:12 smithi079 ceph-mon[35312]: pgmap v848: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:34:14.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:34:14 smithi134 ceph-mon[35037]: pgmap v849: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:34:14.925 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:34:14 smithi134 ceph-mon[35037]: 2 slow requests, 0 included below; oldest blocked for > 1041.854913 secs 2024-03-28T18:34:14.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:34:14 smithi079 ceph-mon[35312]: pgmap v849: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:34:14.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:34:14 smithi079 ceph-mon[35312]: 2 slow requests, 0 included below; oldest blocked for > 1041.854913 secs 2024-03-28T18:34:15.032 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:34:14 smithi022 ceph-mon[31985]: pgmap v849: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:34:15.032 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:34:14 smithi022 ceph-mon[31985]: 2 slow requests, 0 included below; oldest blocked for > 1041.854913 secs 2024-03-28T18:34:17.032 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:34:16 smithi022 ceph-mon[31985]: pgmap v850: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:34:17.174 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:34:16 smithi134 ceph-mon[35037]: pgmap v850: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:34:17.182 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:34:16 smithi079 ceph-mon[35312]: pgmap v850: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:34:19.032 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:34:18 smithi022 ceph-mon[31985]: pgmap v851: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:34:19.174 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:34:18 smithi134 ceph-mon[35037]: pgmap v851: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:34:19.182 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:34:18 smithi079 ceph-mon[35312]: pgmap v851: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:34:20.032 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:34:19 smithi022 ceph-mon[31985]: 2 slow requests, 0 included below; oldest blocked for > 1046.854991 secs 2024-03-28T18:34:20.175 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:34:19 smithi134 ceph-mon[35037]: 2 slow requests, 0 included below; oldest blocked for > 1046.854991 secs 2024-03-28T18:34:20.182 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:34:19 smithi079 ceph-mon[35312]: 2 slow requests, 0 included below; oldest blocked for > 1046.854991 secs 2024-03-28T18:34:21.032 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:34:20 smithi022 ceph-mon[31985]: pgmap v852: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:34:21.174 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:34:20 smithi134 ceph-mon[35037]: pgmap v852: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:34:21.183 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:34:20 smithi079 ceph-mon[35312]: pgmap v852: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:34:23.032 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:34:22 smithi022 ceph-mon[31985]: pgmap v853: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:34:23.175 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:34:22 smithi134 ceph-mon[35037]: pgmap v853: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:34:23.182 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:34:22 smithi079 ceph-mon[35312]: pgmap v853: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:34:25.032 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:34:24 smithi022 ceph-mon[31985]: pgmap v854: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:34:25.032 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:34:24 smithi022 ceph-mon[31985]: 2 slow requests, 0 included below; oldest blocked for > 1051.855068 secs 2024-03-28T18:34:25.174 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:34:24 smithi134 ceph-mon[35037]: pgmap v854: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:34:25.175 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:34:24 smithi134 ceph-mon[35037]: 2 slow requests, 0 included below; oldest blocked for > 1051.855068 secs 2024-03-28T18:34:25.182 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:34:24 smithi079 ceph-mon[35312]: pgmap v854: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:34:25.182 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:34:24 smithi079 ceph-mon[35312]: 2 slow requests, 0 included below; oldest blocked for > 1051.855068 secs 2024-03-28T18:34:27.032 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:34:26 smithi022 ceph-mon[31985]: pgmap v855: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:34:27.174 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:34:26 smithi134 ceph-mon[35037]: pgmap v855: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:34:27.183 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:34:26 smithi079 ceph-mon[35312]: pgmap v855: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:34:29.032 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:34:28 smithi022 ceph-mon[31985]: pgmap v856: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:34:29.175 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:34:28 smithi134 ceph-mon[35037]: pgmap v856: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:34:29.182 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:34:28 smithi079 ceph-mon[35312]: pgmap v856: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:34:30.032 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:34:29 smithi022 ceph-mon[31985]: 2 slow requests, 0 included below; oldest blocked for > 1056.855137 secs 2024-03-28T18:34:30.174 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:34:29 smithi134 ceph-mon[35037]: 2 slow requests, 0 included below; oldest blocked for > 1056.855137 secs 2024-03-28T18:34:30.182 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:34:29 smithi079 ceph-mon[35312]: 2 slow requests, 0 included below; oldest blocked for > 1056.855137 secs 2024-03-28T18:34:31.175 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:34:30 smithi134 ceph-mon[35037]: pgmap v857: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:34:31.182 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:34:30 smithi079 ceph-mon[35312]: pgmap v857: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:34:31.282 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:34:30 smithi022 ceph-mon[31985]: pgmap v857: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:34:33.175 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:34:32 smithi134 ceph-mon[35037]: pgmap v858: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:34:33.182 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:34:32 smithi079 ceph-mon[35312]: pgmap v858: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:34:33.282 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:34:32 smithi022 ceph-mon[31985]: pgmap v858: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:34:35.175 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:34:34 smithi134 ceph-mon[35037]: pgmap v859: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:34:35.176 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:34:34 smithi134 ceph-mon[35037]: 2 slow requests, 0 included below; oldest blocked for > 1061.855236 secs 2024-03-28T18:34:35.182 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:34:34 smithi079 ceph-mon[35312]: pgmap v859: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:34:35.183 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:34:34 smithi079 ceph-mon[35312]: 2 slow requests, 0 included below; oldest blocked for > 1061.855236 secs 2024-03-28T18:34:35.282 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:34:34 smithi022 ceph-mon[31985]: pgmap v859: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:34:35.282 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:34:34 smithi022 ceph-mon[31985]: 2 slow requests, 0 included below; oldest blocked for > 1061.855236 secs 2024-03-28T18:34:37.085 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:34:36 smithi022 ceph-mon[31985]: pgmap v860: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:34:37.174 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:34:36 smithi134 ceph-mon[35037]: pgmap v860: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:34:37.182 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:34:36 smithi079 ceph-mon[35312]: pgmap v860: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:34:39.175 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:34:38 smithi134 ceph-mon[35037]: pgmap v861: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:34:39.182 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:34:38 smithi079 ceph-mon[35312]: pgmap v861: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:34:39.282 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:34:38 smithi022 ceph-mon[31985]: pgmap v861: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:34:40.076 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:34:39 smithi134 ceph-mon[35037]: 2 slow requests, 0 included below; oldest blocked for > 1066.855355 secs 2024-03-28T18:34:40.183 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:34:39 smithi079 ceph-mon[35312]: 2 slow requests, 0 included below; oldest blocked for > 1066.855355 secs 2024-03-28T18:34:40.282 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:34:39 smithi022 ceph-mon[31985]: 2 slow requests, 0 included below; oldest blocked for > 1066.855355 secs 2024-03-28T18:34:41.175 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:34:40 smithi134 ceph-mon[35037]: pgmap v862: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:34:41.182 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:34:40 smithi079 ceph-mon[35312]: pgmap v862: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:34:41.282 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:34:40 smithi022 ceph-mon[31985]: pgmap v862: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:34:43.177 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:34:42 smithi134 ceph-mon[35037]: pgmap v863: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:34:43.182 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:34:42 smithi079 ceph-mon[35312]: pgmap v863: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:34:43.282 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:34:42 smithi022 ceph-mon[31985]: pgmap v863: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:34:45.174 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:34:44 smithi134 ceph-mon[35037]: pgmap v864: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:34:45.175 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:34:44 smithi134 ceph-mon[35037]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T18:34:45.175 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:34:44 smithi134 ceph-mon[35037]: 2 slow requests, 0 included below; oldest blocked for > 1071.855452 secs 2024-03-28T18:34:45.182 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:34:44 smithi079 ceph-mon[35312]: pgmap v864: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:34:45.182 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:34:44 smithi079 ceph-mon[35312]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T18:34:45.182 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:34:44 smithi079 ceph-mon[35312]: 2 slow requests, 0 included below; oldest blocked for > 1071.855452 secs 2024-03-28T18:34:45.282 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:34:44 smithi022 ceph-mon[31985]: pgmap v864: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:34:45.282 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:34:44 smithi022 ceph-mon[31985]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T18:34:45.282 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:34:44 smithi022 ceph-mon[31985]: 2 slow requests, 0 included below; oldest blocked for > 1071.855452 secs 2024-03-28T18:34:46.925 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:34:46 smithi134 ceph-mon[35037]: pgmap v865: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:34:47.182 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:34:46 smithi079 ceph-mon[35312]: pgmap v865: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:34:47.282 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:34:46 smithi022 ceph-mon[31985]: pgmap v865: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:34:49.174 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:34:48 smithi134 ceph-mon[35037]: pgmap v866: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:34:49.182 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:34:48 smithi079 ceph-mon[35312]: pgmap v866: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:34:49.282 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:34:48 smithi022 ceph-mon[31985]: pgmap v866: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:34:50.175 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:34:49 smithi134 ceph-mon[35037]: 2 slow requests, 0 included below; oldest blocked for > 1076.855523 secs 2024-03-28T18:34:50.182 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:34:49 smithi079 ceph-mon[35312]: 2 slow requests, 0 included below; oldest blocked for > 1076.855523 secs 2024-03-28T18:34:50.282 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:34:49 smithi022 ceph-mon[31985]: 2 slow requests, 0 included below; oldest blocked for > 1076.855523 secs 2024-03-28T18:34:51.175 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:34:50 smithi134 ceph-mon[35037]: pgmap v867: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:34:51.175 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:34:50 smithi134 ceph-mon[35037]: from='mgr.15522 ' entity='mgr.x' 2024-03-28T18:34:51.175 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:34:50 smithi134 ceph-mon[35037]: from='mgr.15522 ' entity='mgr.x' 2024-03-28T18:34:51.175 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:34:50 smithi134 ceph-mon[35037]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T18:34:51.175 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:34:50 smithi134 ceph-mon[35037]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T18:34:51.175 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:34:50 smithi134 ceph-mon[35037]: from='mgr.15522 ' entity='mgr.x' 2024-03-28T18:34:51.182 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:34:50 smithi079 ceph-mon[35312]: pgmap v867: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:34:51.182 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:34:50 smithi079 ceph-mon[35312]: from='mgr.15522 ' entity='mgr.x' 2024-03-28T18:34:51.183 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:34:50 smithi079 ceph-mon[35312]: from='mgr.15522 ' entity='mgr.x' 2024-03-28T18:34:51.183 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:34:50 smithi079 ceph-mon[35312]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T18:34:51.183 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:34:50 smithi079 ceph-mon[35312]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T18:34:51.183 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:34:50 smithi079 ceph-mon[35312]: from='mgr.15522 ' entity='mgr.x' 2024-03-28T18:34:51.282 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:34:50 smithi022 ceph-mon[31985]: pgmap v867: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:34:51.282 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:34:50 smithi022 ceph-mon[31985]: from='mgr.15522 ' entity='mgr.x' 2024-03-28T18:34:51.282 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:34:50 smithi022 ceph-mon[31985]: from='mgr.15522 ' entity='mgr.x' 2024-03-28T18:34:51.282 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:34:50 smithi022 ceph-mon[31985]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T18:34:51.282 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:34:50 smithi022 ceph-mon[31985]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T18:34:51.282 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:34:50 smithi022 ceph-mon[31985]: from='mgr.15522 ' entity='mgr.x' 2024-03-28T18:34:52.175 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:34:51 smithi134 ceph-mon[35037]: pgmap v868: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T18:34:52.175 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:34:51 smithi134 ceph-mon[35037]: pgmap v869: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-03-28T18:34:52.182 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:34:51 smithi079 ceph-mon[35312]: pgmap v868: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T18:34:52.182 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:34:51 smithi079 ceph-mon[35312]: pgmap v869: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-03-28T18:34:52.282 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:34:51 smithi022 ceph-mon[31985]: pgmap v868: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T18:34:52.282 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:34:51 smithi022 ceph-mon[31985]: pgmap v869: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-03-28T18:34:54.175 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:34:53 smithi134 ceph-mon[35037]: pgmap v870: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-03-28T18:34:54.182 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:34:53 smithi079 ceph-mon[35312]: pgmap v870: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-03-28T18:34:54.282 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:34:53 smithi022 ceph-mon[31985]: pgmap v870: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-03-28T18:34:55.175 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:34:54 smithi134 ceph-mon[35037]: 2 slow requests, 0 included below; oldest blocked for > 1081.855644 secs 2024-03-28T18:34:55.182 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:34:54 smithi079 ceph-mon[35312]: 2 slow requests, 0 included below; oldest blocked for > 1081.855644 secs 2024-03-28T18:34:55.282 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:34:54 smithi022 ceph-mon[31985]: 2 slow requests, 0 included below; oldest blocked for > 1081.855644 secs 2024-03-28T18:34:56.175 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:34:55 smithi134 ceph-mon[35037]: pgmap v871: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-03-28T18:34:56.182 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:34:55 smithi079 ceph-mon[35312]: pgmap v871: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-03-28T18:34:56.282 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:34:55 smithi022 ceph-mon[31985]: pgmap v871: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-03-28T18:34:58.175 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:34:57 smithi134 ceph-mon[35037]: pgmap v872: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-03-28T18:34:58.182 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:34:57 smithi079 ceph-mon[35312]: pgmap v872: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-03-28T18:34:58.282 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:34:57 smithi022 ceph-mon[31985]: pgmap v872: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-03-28T18:35:00.175 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:34:59 smithi134 ceph-mon[35037]: pgmap v873: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s rd, 4 op/s 2024-03-28T18:35:00.175 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:34:59 smithi134 ceph-mon[35037]: 2 slow requests, 0 included below; oldest blocked for > 1086.855725 secs 2024-03-28T18:35:00.182 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:34:59 smithi079 ceph-mon[35312]: pgmap v873: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s rd, 4 op/s 2024-03-28T18:35:00.182 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:34:59 smithi079 ceph-mon[35312]: 2 slow requests, 0 included below; oldest blocked for > 1086.855725 secs 2024-03-28T18:35:00.282 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:34:59 smithi022 ceph-mon[31985]: pgmap v873: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s rd, 4 op/s 2024-03-28T18:35:00.282 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:34:59 smithi022 ceph-mon[31985]: 2 slow requests, 0 included below; oldest blocked for > 1086.855725 secs 2024-03-28T18:35:02.174 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:35:01 smithi134 ceph-mon[35037]: pgmap v874: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 7 op/s 2024-03-28T18:35:02.183 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:35:01 smithi079 ceph-mon[35312]: pgmap v874: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 7 op/s 2024-03-28T18:35:02.282 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:35:01 smithi022 ceph-mon[31985]: pgmap v874: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 7 op/s 2024-03-28T18:35:04.175 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:35:03 smithi134 ceph-mon[35037]: pgmap v875: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:35:04.182 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:35:03 smithi079 ceph-mon[35312]: pgmap v875: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:35:04.282 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:35:03 smithi022 ceph-mon[31985]: pgmap v875: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:35:05.175 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:35:04 smithi134 ceph-mon[35037]: 2 slow requests, 0 included below; oldest blocked for > 1091.855819 secs 2024-03-28T18:35:05.183 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:35:04 smithi079 ceph-mon[35312]: 2 slow requests, 0 included below; oldest blocked for > 1091.855819 secs 2024-03-28T18:35:05.282 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:35:04 smithi022 ceph-mon[31985]: 2 slow requests, 0 included below; oldest blocked for > 1091.855819 secs 2024-03-28T18:35:06.182 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:35:05 smithi079 ceph-mon[35312]: pgmap v876: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:35:06.282 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:35:05 smithi022 ceph-mon[31985]: pgmap v876: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:35:06.424 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:35:05 smithi134 ceph-mon[35037]: pgmap v876: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:35:08.183 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:35:07 smithi079 ceph-mon[35312]: pgmap v877: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:35:08.282 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:35:07 smithi022 ceph-mon[31985]: pgmap v877: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:35:08.425 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:35:07 smithi134 ceph-mon[35037]: pgmap v877: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:35:10.227 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:35:09 smithi134 ceph-mon[35037]: pgmap v878: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:35:10.227 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:35:09 smithi134 ceph-mon[35037]: 2 slow requests, 0 included below; oldest blocked for > 1096.855891 secs 2024-03-28T18:35:10.282 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:35:09 smithi022 ceph-mon[31985]: pgmap v878: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:35:10.282 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:35:09 smithi022 ceph-mon[31985]: 2 slow requests, 0 included below; oldest blocked for > 1096.855891 secs 2024-03-28T18:35:10.432 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:35:09 smithi079 ceph-mon[35312]: pgmap v878: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:35:10.433 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:35:09 smithi079 ceph-mon[35312]: 2 slow requests, 0 included below; oldest blocked for > 1096.855891 secs 2024-03-28T18:35:12.175 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:35:11 smithi134 ceph-mon[35037]: pgmap v879: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:35:12.282 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:35:11 smithi022 ceph-mon[31985]: pgmap v879: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:35:12.432 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:35:11 smithi079 ceph-mon[35312]: pgmap v879: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:35:14.282 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:35:13 smithi022 ceph-mon[31985]: pgmap v880: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:35:14.425 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:35:13 smithi134 ceph-mon[35037]: pgmap v880: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:35:14.432 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:35:13 smithi079 ceph-mon[35312]: pgmap v880: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:35:15.282 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:35:14 smithi022 ceph-mon[31985]: 2 slow requests, 0 included below; oldest blocked for > 1101.856000 secs 2024-03-28T18:35:15.424 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:35:14 smithi134 ceph-mon[35037]: 2 slow requests, 0 included below; oldest blocked for > 1101.856000 secs 2024-03-28T18:35:15.432 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:35:14 smithi079 ceph-mon[35312]: 2 slow requests, 0 included below; oldest blocked for > 1101.856000 secs 2024-03-28T18:35:16.282 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:35:15 smithi022 ceph-mon[31985]: pgmap v881: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:35:16.334 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:35:15 smithi079 ceph-mon[35312]: pgmap v881: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:35:16.424 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:35:15 smithi134 ceph-mon[35037]: pgmap v881: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:35:18.282 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:35:17 smithi022 ceph-mon[31985]: pgmap v882: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:35:18.424 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:35:17 smithi134 ceph-mon[35037]: pgmap v882: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:35:18.433 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:35:17 smithi079 ceph-mon[35312]: pgmap v882: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:35:20.282 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:35:19 smithi022 ceph-mon[31985]: pgmap v883: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:35:20.282 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:35:19 smithi022 ceph-mon[31985]: 2 slow requests, 0 included below; oldest blocked for > 1106.856086 secs 2024-03-28T18:35:20.425 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:35:19 smithi134 ceph-mon[35037]: pgmap v883: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:35:20.425 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:35:19 smithi134 ceph-mon[35037]: 2 slow requests, 0 included below; oldest blocked for > 1106.856086 secs 2024-03-28T18:35:20.433 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:35:19 smithi079 ceph-mon[35312]: pgmap v883: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:35:20.433 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:35:19 smithi079 ceph-mon[35312]: 2 slow requests, 0 included below; oldest blocked for > 1106.856086 secs 2024-03-28T18:35:22.282 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:35:21 smithi022 ceph-mon[31985]: pgmap v884: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:35:22.284 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:35:21 smithi134 ceph-mon[35037]: pgmap v884: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:35:22.433 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:35:21 smithi079 ceph-mon[35312]: pgmap v884: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:35:24.282 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:35:23 smithi022 ceph-mon[31985]: pgmap v885: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:35:24.425 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:35:23 smithi134 ceph-mon[35037]: pgmap v885: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:35:24.432 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:35:23 smithi079 ceph-mon[35312]: pgmap v885: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:35:25.282 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:35:24 smithi022 ceph-mon[31985]: 2 slow requests, 0 included below; oldest blocked for > 1111.856150 secs 2024-03-28T18:35:25.425 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:35:25 smithi134 ceph-mon[35037]: 2 slow requests, 0 included below; oldest blocked for > 1111.856150 secs 2024-03-28T18:35:25.432 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:35:25 smithi079 ceph-mon[35312]: 2 slow requests, 0 included below; oldest blocked for > 1111.856150 secs 2024-03-28T18:35:26.282 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:35:26 smithi022 ceph-mon[31985]: pgmap v886: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:35:26.425 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:35:26 smithi134 ceph-mon[35037]: pgmap v886: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:35:26.432 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:35:26 smithi079 ceph-mon[35312]: pgmap v886: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:35:28.282 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:35:28 smithi022 ceph-mon[31985]: pgmap v887: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:35:28.425 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:35:28 smithi134 ceph-mon[35037]: pgmap v887: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:35:28.432 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:35:28 smithi079 ceph-mon[35312]: pgmap v887: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:35:30.282 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:35:30 smithi022 ceph-mon[31985]: pgmap v888: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:35:30.282 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:35:30 smithi022 ceph-mon[31985]: 2 slow requests, 0 included below; oldest blocked for > 1116.856215 secs 2024-03-28T18:35:30.425 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:35:30 smithi134 ceph-mon[35037]: pgmap v888: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:35:30.425 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:35:30 smithi134 ceph-mon[35037]: 2 slow requests, 0 included below; oldest blocked for > 1116.856215 secs 2024-03-28T18:35:30.432 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:35:30 smithi079 ceph-mon[35312]: pgmap v888: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:35:30.432 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:35:30 smithi079 ceph-mon[35312]: 2 slow requests, 0 included below; oldest blocked for > 1116.856215 secs 2024-03-28T18:35:32.282 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:35:32 smithi022 ceph-mon[31985]: pgmap v889: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:35:32.424 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:35:32 smithi134 ceph-mon[35037]: pgmap v889: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:35:32.432 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:35:32 smithi079 ceph-mon[35312]: pgmap v889: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:35:34.282 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:35:34 smithi022 ceph-mon[31985]: pgmap v890: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:35:34.425 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:35:34 smithi134 ceph-mon[35037]: pgmap v890: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:35:34.432 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:35:34 smithi079 ceph-mon[35312]: pgmap v890: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:35:35.425 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:35:35 smithi134 ceph-mon[35037]: 2 slow requests, 0 included below; oldest blocked for > 1121.856239 secs 2024-03-28T18:35:35.432 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:35:35 smithi079 ceph-mon[35312]: 2 slow requests, 0 included below; oldest blocked for > 1121.856239 secs 2024-03-28T18:35:35.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:35:35 smithi022 ceph-mon[31985]: 2 slow requests, 0 included below; oldest blocked for > 1121.856239 secs 2024-03-28T18:35:36.425 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:35:36 smithi134 ceph-mon[35037]: pgmap v891: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:35:36.432 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:35:36 smithi079 ceph-mon[35312]: pgmap v891: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:35:36.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:35:36 smithi022 ceph-mon[31985]: pgmap v891: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:35:38.335 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:35:38 smithi022 ceph-mon[31985]: pgmap v892: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:35:38.425 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:35:38 smithi134 ceph-mon[35037]: pgmap v892: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:35:38.432 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:35:38 smithi079 ceph-mon[35312]: pgmap v892: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:35:40.425 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:35:40 smithi134 ceph-mon[35037]: pgmap v893: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:35:40.425 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:35:40 smithi134 ceph-mon[35037]: 2 slow requests, 0 included below; oldest blocked for > 1126.856305 secs 2024-03-28T18:35:40.433 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:35:40 smithi079 ceph-mon[35312]: pgmap v893: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:35:40.433 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:35:40 smithi079 ceph-mon[35312]: 2 slow requests, 0 included below; oldest blocked for > 1126.856305 secs 2024-03-28T18:35:40.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:35:40 smithi022 ceph-mon[31985]: pgmap v893: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:35:40.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:35:40 smithi022 ceph-mon[31985]: 2 slow requests, 0 included below; oldest blocked for > 1126.856305 secs 2024-03-28T18:35:42.425 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:35:42 smithi134 ceph-mon[35037]: pgmap v894: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:35:42.432 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:35:42 smithi079 ceph-mon[35312]: pgmap v894: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:35:42.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:35:42 smithi022 ceph-mon[31985]: pgmap v894: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:35:44.424 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:35:44 smithi134 ceph-mon[35037]: pgmap v895: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:35:44.432 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:35:44 smithi079 ceph-mon[35312]: pgmap v895: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:35:44.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:35:44 smithi022 ceph-mon[31985]: pgmap v895: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:35:45.425 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:35:45 smithi134 ceph-mon[35037]: 2 slow requests, 0 included below; oldest blocked for > 1131.856432 secs 2024-03-28T18:35:45.432 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:35:45 smithi079 ceph-mon[35312]: 2 slow requests, 0 included below; oldest blocked for > 1131.856432 secs 2024-03-28T18:35:45.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:35:45 smithi022 ceph-mon[31985]: 2 slow requests, 0 included below; oldest blocked for > 1131.856432 secs 2024-03-28T18:35:46.334 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:35:46 smithi079 ceph-mon[35312]: pgmap v896: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:35:46.425 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:35:46 smithi134 ceph-mon[35037]: pgmap v896: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:35:46.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:35:46 smithi022 ceph-mon[31985]: pgmap v896: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:35:48.425 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:35:48 smithi134 ceph-mon[35037]: pgmap v897: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:35:48.432 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:35:48 smithi079 ceph-mon[35312]: pgmap v897: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:35:48.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:35:48 smithi022 ceph-mon[31985]: pgmap v897: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:35:50.426 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:35:50 smithi134 ceph-mon[35037]: pgmap v898: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:35:50.426 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:35:50 smithi134 ceph-mon[35037]: 2 slow requests, 0 included below; oldest blocked for > 1136.856550 secs 2024-03-28T18:35:50.432 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:35:50 smithi079 ceph-mon[35312]: pgmap v898: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:35:50.433 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:35:50 smithi079 ceph-mon[35312]: 2 slow requests, 0 included below; oldest blocked for > 1136.856550 secs 2024-03-28T18:35:50.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:35:50 smithi022 ceph-mon[31985]: pgmap v898: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:35:50.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:35:50 smithi022 ceph-mon[31985]: 2 slow requests, 0 included below; oldest blocked for > 1136.856550 secs 2024-03-28T18:35:51.424 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:35:51 smithi134 ceph-mon[35037]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T18:35:51.432 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:35:51 smithi079 ceph-mon[35312]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T18:35:51.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:35:51 smithi022 ceph-mon[31985]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T18:35:52.425 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:35:52 smithi134 ceph-mon[35037]: pgmap v899: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:35:52.432 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:35:52 smithi079 ceph-mon[35312]: pgmap v899: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:35:52.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:35:52 smithi022 ceph-mon[31985]: pgmap v899: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:35:53.424 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:35:53 smithi134 ceph-mon[35037]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T18:35:53.425 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:35:53 smithi134 ceph-mon[35037]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T18:35:53.425 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:35:53 smithi134 ceph-mon[35037]: from='mgr.15522 ' entity='mgr.x' 2024-03-28T18:35:53.432 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:35:53 smithi079 ceph-mon[35312]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T18:35:53.432 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:35:53 smithi079 ceph-mon[35312]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T18:35:53.432 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:35:53 smithi079 ceph-mon[35312]: from='mgr.15522 ' entity='mgr.x' 2024-03-28T18:35:53.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:35:53 smithi022 ceph-mon[31985]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T18:35:53.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:35:53 smithi022 ceph-mon[31985]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T18:35:53.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:35:53 smithi022 ceph-mon[31985]: from='mgr.15522 ' entity='mgr.x' 2024-03-28T18:35:54.425 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:35:54 smithi134 ceph-mon[35037]: pgmap v900: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 6 op/s 2024-03-28T18:35:54.425 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:35:54 smithi134 ceph-mon[35037]: pgmap v901: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-03-28T18:35:54.432 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:35:54 smithi079 ceph-mon[35312]: pgmap v900: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 6 op/s 2024-03-28T18:35:54.432 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:35:54 smithi079 ceph-mon[35312]: pgmap v901: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-03-28T18:35:54.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:35:54 smithi022 ceph-mon[31985]: pgmap v900: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 6 op/s 2024-03-28T18:35:54.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:35:54 smithi022 ceph-mon[31985]: pgmap v901: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-03-28T18:35:55.424 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:35:55 smithi134 ceph-mon[35037]: 2 slow requests, 0 included below; oldest blocked for > 1141.856571 secs 2024-03-28T18:35:55.432 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:35:55 smithi079 ceph-mon[35312]: 2 slow requests, 0 included below; oldest blocked for > 1141.856571 secs 2024-03-28T18:35:55.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:35:55 smithi022 ceph-mon[31985]: 2 slow requests, 0 included below; oldest blocked for > 1141.856571 secs 2024-03-28T18:35:56.425 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:35:56 smithi134 ceph-mon[35037]: pgmap v902: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 6 op/s 2024-03-28T18:35:56.432 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:35:56 smithi079 ceph-mon[35312]: pgmap v902: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 6 op/s 2024-03-28T18:35:56.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:35:56 smithi022 ceph-mon[31985]: pgmap v902: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 6 op/s 2024-03-28T18:35:58.425 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:35:58 smithi134 ceph-mon[35037]: pgmap v903: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-03-28T18:35:58.432 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:35:58 smithi079 ceph-mon[35312]: pgmap v903: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-03-28T18:35:58.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:35:58 smithi022 ceph-mon[31985]: pgmap v903: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-03-28T18:36:00.425 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:36:00 smithi134 ceph-mon[35037]: pgmap v904: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-03-28T18:36:00.425 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:36:00 smithi134 ceph-mon[35037]: 2 slow requests, 0 included below; oldest blocked for > 1146.856692 secs 2024-03-28T18:36:00.432 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:36:00 smithi079 ceph-mon[35312]: pgmap v904: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-03-28T18:36:00.433 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:36:00 smithi079 ceph-mon[35312]: 2 slow requests, 0 included below; oldest blocked for > 1146.856692 secs 2024-03-28T18:36:00.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:36:00 smithi022 ceph-mon[31985]: pgmap v904: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-03-28T18:36:00.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:36:00 smithi022 ceph-mon[31985]: 2 slow requests, 0 included below; oldest blocked for > 1146.856692 secs 2024-03-28T18:36:02.424 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:36:02 smithi134 ceph-mon[35037]: pgmap v905: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-03-28T18:36:02.432 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:36:02 smithi079 ceph-mon[35312]: pgmap v905: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-03-28T18:36:02.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:36:02 smithi022 ceph-mon[31985]: pgmap v905: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-03-28T18:36:04.425 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:36:04 smithi134 ceph-mon[35037]: pgmap v906: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 7 op/s 2024-03-28T18:36:04.433 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:36:04 smithi079 ceph-mon[35312]: pgmap v906: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 7 op/s 2024-03-28T18:36:04.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:36:04 smithi022 ceph-mon[31985]: pgmap v906: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 7 op/s 2024-03-28T18:36:05.424 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:36:05 smithi134 ceph-mon[35037]: 2 slow requests, 0 included below; oldest blocked for > 1151.856759 secs 2024-03-28T18:36:05.432 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:36:05 smithi079 ceph-mon[35312]: 2 slow requests, 0 included below; oldest blocked for > 1151.856759 secs 2024-03-28T18:36:05.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:36:05 smithi022 ceph-mon[31985]: 2 slow requests, 0 included below; oldest blocked for > 1151.856759 secs 2024-03-28T18:36:06.425 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:36:06 smithi134 ceph-mon[35037]: pgmap v907: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-28T18:36:06.432 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:36:06 smithi079 ceph-mon[35312]: pgmap v907: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-28T18:36:06.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:36:06 smithi022 ceph-mon[31985]: pgmap v907: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-28T18:36:08.432 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:36:08 smithi079 ceph-mon[35312]: pgmap v908: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 7 op/s 2024-03-28T18:36:08.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:36:08 smithi022 ceph-mon[31985]: pgmap v908: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 7 op/s 2024-03-28T18:36:08.674 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:36:08 smithi134 ceph-mon[35037]: pgmap v908: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 7 op/s 2024-03-28T18:36:10.478 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:36:10 smithi134 ceph-mon[35037]: pgmap v909: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:36:10.478 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:36:10 smithi134 ceph-mon[35037]: 2 slow requests, 0 included below; oldest blocked for > 1156.856881 secs 2024-03-28T18:36:10.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:36:10 smithi022 ceph-mon[31985]: pgmap v909: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:36:10.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:36:10 smithi022 ceph-mon[31985]: 2 slow requests, 0 included below; oldest blocked for > 1156.856881 secs 2024-03-28T18:36:10.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:36:10 smithi079 ceph-mon[35312]: pgmap v909: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:36:10.683 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:36:10 smithi079 ceph-mon[35312]: 2 slow requests, 0 included below; oldest blocked for > 1156.856881 secs 2024-03-28T18:36:12.424 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:36:12 smithi134 ceph-mon[35037]: pgmap v910: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:36:12.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:36:12 smithi022 ceph-mon[31985]: pgmap v910: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:36:12.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:36:12 smithi079 ceph-mon[35312]: pgmap v910: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:36:14.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:36:14 smithi022 ceph-mon[31985]: pgmap v911: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:36:14.674 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:36:14 smithi134 ceph-mon[35037]: pgmap v911: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:36:14.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:36:14 smithi079 ceph-mon[35312]: pgmap v911: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:36:15.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:36:15 smithi022 ceph-mon[31985]: 2 slow requests, 0 included below; oldest blocked for > 1161.856940 secs 2024-03-28T18:36:15.674 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:36:15 smithi134 ceph-mon[35037]: 2 slow requests, 0 included below; oldest blocked for > 1161.856940 secs 2024-03-28T18:36:15.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:36:15 smithi079 ceph-mon[35312]: 2 slow requests, 0 included below; oldest blocked for > 1161.856940 secs 2024-03-28T18:36:16.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:36:16 smithi022 ceph-mon[31985]: pgmap v912: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-03-28T18:36:16.584 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:36:16 smithi079 ceph-mon[35312]: pgmap v912: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-03-28T18:36:16.674 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:36:16 smithi134 ceph-mon[35037]: pgmap v912: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-03-28T18:36:18.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:36:18 smithi022 ceph-mon[31985]: pgmap v913: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 7 op/s 2024-03-28T18:36:18.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:36:18 smithi134 ceph-mon[35037]: pgmap v913: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 7 op/s 2024-03-28T18:36:18.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:36:18 smithi079 ceph-mon[35312]: pgmap v913: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 7 op/s 2024-03-28T18:36:20.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:36:20 smithi022 ceph-mon[31985]: pgmap v914: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:36:20.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:36:20 smithi022 ceph-mon[31985]: 2 slow requests, 0 included below; oldest blocked for > 1166.857023 secs 2024-03-28T18:36:20.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:36:20 smithi134 ceph-mon[35037]: pgmap v914: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:36:20.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:36:20 smithi134 ceph-mon[35037]: 2 slow requests, 0 included below; oldest blocked for > 1166.857023 secs 2024-03-28T18:36:20.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:36:20 smithi079 ceph-mon[35312]: pgmap v914: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:36:20.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:36:20 smithi079 ceph-mon[35312]: 2 slow requests, 0 included below; oldest blocked for > 1166.857023 secs 2024-03-28T18:36:21.674 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:36:21 smithi134 ceph-mon[35037]: pgmap v915: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:36:21.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:36:21 smithi079 ceph-mon[35312]: pgmap v915: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:36:21.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:36:21 smithi022 ceph-mon[31985]: pgmap v915: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:36:23.674 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:36:23 smithi134 ceph-mon[35037]: pgmap v916: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:36:23.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:36:23 smithi079 ceph-mon[35312]: pgmap v916: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:36:23.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:36:23 smithi022 ceph-mon[31985]: pgmap v916: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:36:25.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:36:25 smithi134 ceph-mon[35037]: 2 slow requests, 0 included below; oldest blocked for > 1171.857086 secs 2024-03-28T18:36:25.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:36:25 smithi134 ceph-mon[35037]: pgmap v917: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-03-28T18:36:25.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:36:25 smithi079 ceph-mon[35312]: 2 slow requests, 0 included below; oldest blocked for > 1171.857086 secs 2024-03-28T18:36:25.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:36:25 smithi079 ceph-mon[35312]: pgmap v917: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-03-28T18:36:25.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:36:25 smithi022 ceph-mon[31985]: 2 slow requests, 0 included below; oldest blocked for > 1171.857086 secs 2024-03-28T18:36:25.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:36:25 smithi022 ceph-mon[31985]: pgmap v917: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-03-28T18:36:27.674 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:36:27 smithi134 ceph-mon[35037]: pgmap v918: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 7 op/s 2024-03-28T18:36:27.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:36:27 smithi079 ceph-mon[35312]: pgmap v918: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 7 op/s 2024-03-28T18:36:27.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:36:27 smithi022 ceph-mon[31985]: pgmap v918: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 7 op/s 2024-03-28T18:36:29.674 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:36:29 smithi134 ceph-mon[35037]: pgmap v919: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:36:29.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:36:29 smithi079 ceph-mon[35312]: pgmap v919: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:36:29.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:36:29 smithi022 ceph-mon[31985]: pgmap v919: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:36:30.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:36:30 smithi079 ceph-mon[35312]: 2 slow requests, 0 included below; oldest blocked for > 1176.857206 secs 2024-03-28T18:36:30.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:36:30 smithi022 ceph-mon[31985]: 2 slow requests, 0 included below; oldest blocked for > 1176.857206 secs 2024-03-28T18:36:30.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:36:30 smithi134 ceph-mon[35037]: 2 slow requests, 0 included below; oldest blocked for > 1176.857206 secs 2024-03-28T18:36:31.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:36:31 smithi022 ceph-mon[31985]: pgmap v920: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:36:31.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:36:31 smithi134 ceph-mon[35037]: pgmap v920: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:36:31.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:36:31 smithi079 ceph-mon[35312]: pgmap v920: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:36:33.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:36:33 smithi022 ceph-mon[31985]: pgmap v921: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:36:33.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:36:33 smithi134 ceph-mon[35037]: pgmap v921: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:36:33.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:36:33 smithi079 ceph-mon[35312]: pgmap v921: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:36:35.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:36:35 smithi022 ceph-mon[31985]: 2 slow requests, 0 included below; oldest blocked for > 1181.857270 secs 2024-03-28T18:36:35.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:36:35 smithi022 ceph-mon[31985]: pgmap v922: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-03-28T18:36:35.925 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:36:35 smithi134 ceph-mon[35037]: 2 slow requests, 0 included below; oldest blocked for > 1181.857270 secs 2024-03-28T18:36:35.925 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:36:35 smithi134 ceph-mon[35037]: pgmap v922: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-03-28T18:36:35.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:36:35 smithi079 ceph-mon[35312]: 2 slow requests, 0 included below; oldest blocked for > 1181.857270 secs 2024-03-28T18:36:35.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:36:35 smithi079 ceph-mon[35312]: pgmap v922: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-03-28T18:36:37.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:36:37 smithi022 ceph-mon[31985]: pgmap v923: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 7 op/s 2024-03-28T18:36:37.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:36:37 smithi134 ceph-mon[35037]: pgmap v923: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 7 op/s 2024-03-28T18:36:37.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:36:37 smithi079 ceph-mon[35312]: pgmap v923: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 7 op/s 2024-03-28T18:36:39.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:36:39 smithi022 ceph-mon[31985]: pgmap v924: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:36:39.925 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:36:39 smithi134 ceph-mon[35037]: pgmap v924: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:36:39.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:36:39 smithi079 ceph-mon[35312]: pgmap v924: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:36:40.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:36:40 smithi022 ceph-mon[31985]: 2 slow requests, 0 included below; oldest blocked for > 1186.857341 secs 2024-03-28T18:36:40.826 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:36:40 smithi134 ceph-mon[35037]: 2 slow requests, 0 included below; oldest blocked for > 1186.857341 secs 2024-03-28T18:36:40.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:36:40 smithi079 ceph-mon[35312]: 2 slow requests, 0 included below; oldest blocked for > 1186.857341 secs 2024-03-28T18:36:41.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:36:41 smithi022 ceph-mon[31985]: pgmap v925: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:36:41.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:36:41 smithi134 ceph-mon[35037]: pgmap v925: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:36:41.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:36:41 smithi079 ceph-mon[35312]: pgmap v925: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:36:43.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:36:43 smithi022 ceph-mon[31985]: pgmap v926: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:36:43.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:36:43 smithi134 ceph-mon[35037]: pgmap v926: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:36:43.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:36:43 smithi079 ceph-mon[35312]: pgmap v926: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:36:45.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:36:45 smithi022 ceph-mon[31985]: 2 slow requests, 0 included below; oldest blocked for > 1191.857421 secs 2024-03-28T18:36:45.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:36:45 smithi022 ceph-mon[31985]: pgmap v927: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:36:45.925 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:36:45 smithi134 ceph-mon[35037]: 2 slow requests, 0 included below; oldest blocked for > 1191.857421 secs 2024-03-28T18:36:45.925 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:36:45 smithi134 ceph-mon[35037]: pgmap v927: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:36:45.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:36:45 smithi079 ceph-mon[35312]: 2 slow requests, 0 included below; oldest blocked for > 1191.857421 secs 2024-03-28T18:36:45.933 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:36:45 smithi079 ceph-mon[35312]: pgmap v927: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:36:47.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:36:47 smithi022 ceph-mon[31985]: pgmap v928: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T18:36:47.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:36:47 smithi134 ceph-mon[35037]: pgmap v928: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T18:36:47.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:36:47 smithi079 ceph-mon[35312]: pgmap v928: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T18:36:49.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:36:49 smithi022 ceph-mon[31985]: pgmap v929: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:36:49.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:36:49 smithi134 ceph-mon[35037]: pgmap v929: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:36:49.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:36:49 smithi079 ceph-mon[35312]: pgmap v929: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:36:50.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:36:50 smithi022 ceph-mon[31985]: 2 slow requests, 0 included below; oldest blocked for > 1196.857488 secs 2024-03-28T18:36:50.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:36:50 smithi134 ceph-mon[35037]: 2 slow requests, 0 included below; oldest blocked for > 1196.857488 secs 2024-03-28T18:36:50.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:36:50 smithi079 ceph-mon[35312]: 2 slow requests, 0 included below; oldest blocked for > 1196.857488 secs 2024-03-28T18:36:51.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:36:51 smithi022 ceph-mon[31985]: pgmap v930: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:36:51.925 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:36:51 smithi134 ceph-mon[35037]: pgmap v930: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:36:51.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:36:51 smithi079 ceph-mon[35312]: pgmap v930: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:36:53.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:36:53 smithi022 ceph-mon[31985]: pgmap v931: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:36:53.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:36:53 smithi022 ceph-mon[31985]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T18:36:53.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:36:53 smithi134 ceph-mon[35037]: pgmap v931: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:36:53.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:36:53 smithi134 ceph-mon[35037]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T18:36:53.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:36:53 smithi079 ceph-mon[35312]: pgmap v931: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:36:53.933 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:36:53 smithi079 ceph-mon[35312]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T18:36:54.925 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:36:54 smithi134 ceph-mon[35037]: from='mgr.15522 ' entity='mgr.x' 2024-03-28T18:36:54.925 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:36:54 smithi134 ceph-mon[35037]: from='mgr.15522 ' entity='mgr.x' 2024-03-28T18:36:54.925 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:36:54 smithi134 ceph-mon[35037]: from='mgr.15522 ' entity='mgr.x' 2024-03-28T18:36:54.925 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:36:54 smithi134 ceph-mon[35037]: from='mgr.15522 ' entity='mgr.x' 2024-03-28T18:36:54.925 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:36:54 smithi134 ceph-mon[35037]: 2 slow requests, 0 included below; oldest blocked for > 1201.857584 secs 2024-03-28T18:36:54.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:36:54 smithi079 ceph-mon[35312]: from='mgr.15522 ' entity='mgr.x' 2024-03-28T18:36:54.933 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:36:54 smithi079 ceph-mon[35312]: from='mgr.15522 ' entity='mgr.x' 2024-03-28T18:36:54.933 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:36:54 smithi079 ceph-mon[35312]: from='mgr.15522 ' entity='mgr.x' 2024-03-28T18:36:54.933 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:36:54 smithi079 ceph-mon[35312]: from='mgr.15522 ' entity='mgr.x' 2024-03-28T18:36:54.933 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:36:54 smithi079 ceph-mon[35312]: 2 slow requests, 0 included below; oldest blocked for > 1201.857584 secs 2024-03-28T18:36:55.032 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:36:54 smithi022 ceph-mon[31985]: from='mgr.15522 ' entity='mgr.x' 2024-03-28T18:36:55.032 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:36:54 smithi022 ceph-mon[31985]: from='mgr.15522 ' entity='mgr.x' 2024-03-28T18:36:55.032 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:36:54 smithi022 ceph-mon[31985]: from='mgr.15522 ' entity='mgr.x' 2024-03-28T18:36:55.032 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:36:54 smithi022 ceph-mon[31985]: from='mgr.15522 ' entity='mgr.x' 2024-03-28T18:36:55.032 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:36:54 smithi022 ceph-mon[31985]: 2 slow requests, 0 included below; oldest blocked for > 1201.857584 secs 2024-03-28T18:36:56.032 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:36:55 smithi022 ceph-mon[31985]: pgmap v932: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:36:56.032 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:36:55 smithi022 ceph-mon[31985]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T18:36:56.032 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:36:55 smithi022 ceph-mon[31985]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T18:36:56.032 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:36:55 smithi022 ceph-mon[31985]: pgmap v933: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T18:36:56.033 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:36:55 smithi022 ceph-mon[31985]: pgmap v934: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-03-28T18:36:56.033 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:36:55 smithi022 ceph-mon[31985]: from='mgr.15522 ' entity='mgr.x' 2024-03-28T18:36:56.175 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:36:55 smithi134 ceph-mon[35037]: pgmap v932: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:36:56.175 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:36:55 smithi134 ceph-mon[35037]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T18:36:56.175 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:36:55 smithi134 ceph-mon[35037]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T18:36:56.175 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:36:55 smithi134 ceph-mon[35037]: pgmap v933: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T18:36:56.175 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:36:55 smithi134 ceph-mon[35037]: pgmap v934: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-03-28T18:36:56.175 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:36:55 smithi134 ceph-mon[35037]: from='mgr.15522 ' entity='mgr.x' 2024-03-28T18:36:56.183 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:36:55 smithi079 ceph-mon[35312]: pgmap v932: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:36:56.183 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:36:55 smithi079 ceph-mon[35312]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T18:36:56.183 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:36:55 smithi079 ceph-mon[35312]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T18:36:56.183 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:36:55 smithi079 ceph-mon[35312]: pgmap v933: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T18:36:56.183 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:36:55 smithi079 ceph-mon[35312]: pgmap v934: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-03-28T18:36:56.183 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:36:55 smithi079 ceph-mon[35312]: from='mgr.15522 ' entity='mgr.x' 2024-03-28T18:36:58.425 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:36:58 smithi134 ceph-mon[35037]: pgmap v935: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-03-28T18:36:58.432 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:36:58 smithi079 ceph-mon[35312]: pgmap v935: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-03-28T18:36:58.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:36:58 smithi022 ceph-mon[31985]: pgmap v935: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-03-28T18:37:00.424 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:37:00 smithi134 ceph-mon[35037]: pgmap v936: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s rd, 4 op/s 2024-03-28T18:37:00.425 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:37:00 smithi134 ceph-mon[35037]: 2 slow requests, 0 included below; oldest blocked for > 1206.857647 secs 2024-03-28T18:37:00.432 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:37:00 smithi079 ceph-mon[35312]: pgmap v936: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s rd, 4 op/s 2024-03-28T18:37:00.432 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:37:00 smithi079 ceph-mon[35312]: 2 slow requests, 0 included below; oldest blocked for > 1206.857647 secs 2024-03-28T18:37:00.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:37:00 smithi022 ceph-mon[31985]: pgmap v936: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s rd, 4 op/s 2024-03-28T18:37:00.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:37:00 smithi022 ceph-mon[31985]: 2 slow requests, 0 included below; oldest blocked for > 1206.857647 secs 2024-03-28T18:37:02.424 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:37:02 smithi134 ceph-mon[35037]: pgmap v937: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 9 op/s 2024-03-28T18:37:02.432 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:37:02 smithi079 ceph-mon[35312]: pgmap v937: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 9 op/s 2024-03-28T18:37:02.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:37:02 smithi022 ceph-mon[31985]: pgmap v937: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 9 op/s 2024-03-28T18:37:04.424 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:37:04 smithi134 ceph-mon[35037]: pgmap v938: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 KiB/s rd, 4 op/s 2024-03-28T18:37:04.432 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:37:04 smithi079 ceph-mon[35312]: pgmap v938: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 KiB/s rd, 4 op/s 2024-03-28T18:37:04.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:37:04 smithi022 ceph-mon[31985]: pgmap v938: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 KiB/s rd, 4 op/s 2024-03-28T18:37:05.432 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:37:05 smithi079 ceph-mon[35312]: 2 slow requests, 0 included below; oldest blocked for > 1211.857720 secs 2024-03-28T18:37:05.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:37:05 smithi022 ceph-mon[31985]: 2 slow requests, 0 included below; oldest blocked for > 1211.857720 secs 2024-03-28T18:37:05.674 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:37:05 smithi134 ceph-mon[35037]: 2 slow requests, 0 included below; oldest blocked for > 1211.857720 secs 2024-03-28T18:37:06.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:37:06 smithi022 ceph-mon[31985]: pgmap v939: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 7 op/s 2024-03-28T18:37:06.674 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:37:06 smithi134 ceph-mon[35037]: pgmap v939: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 7 op/s 2024-03-28T18:37:06.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:37:06 smithi079 ceph-mon[35312]: pgmap v939: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 7 op/s 2024-03-28T18:37:08.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:37:08 smithi022 ceph-mon[31985]: pgmap v940: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T18:37:08.674 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:37:08 smithi134 ceph-mon[35037]: pgmap v940: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T18:37:08.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:37:08 smithi079 ceph-mon[35312]: pgmap v940: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T18:37:10.477 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:37:10 smithi134 ceph-mon[35037]: pgmap v941: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T18:37:10.477 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:37:10 smithi134 ceph-mon[35037]: 2 slow requests, 0 included below; oldest blocked for > 1216.857783 secs 2024-03-28T18:37:10.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:37:10 smithi022 ceph-mon[31985]: pgmap v941: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T18:37:10.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:37:10 smithi022 ceph-mon[31985]: 2 slow requests, 0 included below; oldest blocked for > 1216.857783 secs 2024-03-28T18:37:10.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:37:10 smithi079 ceph-mon[35312]: pgmap v941: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T18:37:10.683 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:37:10 smithi079 ceph-mon[35312]: 2 slow requests, 0 included below; oldest blocked for > 1216.857783 secs 2024-03-28T18:37:12.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:37:12 smithi022 ceph-mon[31985]: pgmap v942: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 9 op/s 2024-03-28T18:37:12.674 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:37:12 smithi134 ceph-mon[35037]: pgmap v942: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 9 op/s 2024-03-28T18:37:12.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:37:12 smithi079 ceph-mon[35312]: pgmap v942: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 9 op/s 2024-03-28T18:37:14.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:37:14 smithi022 ceph-mon[31985]: pgmap v943: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:37:14.674 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:37:14 smithi134 ceph-mon[35037]: pgmap v943: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:37:14.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:37:14 smithi079 ceph-mon[35312]: pgmap v943: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:37:15.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:37:15 smithi022 ceph-mon[31985]: 2 slow requests, 0 included below; oldest blocked for > 1221.857858 secs 2024-03-28T18:37:15.674 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:37:15 smithi134 ceph-mon[35037]: 2 slow requests, 0 included below; oldest blocked for > 1221.857858 secs 2024-03-28T18:37:15.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:37:15 smithi079 ceph-mon[35312]: 2 slow requests, 0 included below; oldest blocked for > 1221.857858 secs 2024-03-28T18:37:16.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:37:16 smithi022 ceph-mon[31985]: pgmap v944: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:37:16.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:37:16 smithi134 ceph-mon[35037]: pgmap v944: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:37:16.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:37:16 smithi079 ceph-mon[35312]: pgmap v944: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:37:18.484 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:37:18 smithi079 ceph-mon[35312]: pgmap v945: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:37:18.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:37:18 smithi022 ceph-mon[31985]: pgmap v945: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:37:18.674 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:37:18 smithi134 ceph-mon[35037]: pgmap v945: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:37:20.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:37:20 smithi022 ceph-mon[31985]: pgmap v946: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:37:20.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:37:20 smithi022 ceph-mon[31985]: 2 slow requests, 0 included below; oldest blocked for > 1226.857977 secs 2024-03-28T18:37:20.674 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:37:20 smithi134 ceph-mon[35037]: pgmap v946: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:37:20.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:37:20 smithi134 ceph-mon[35037]: 2 slow requests, 0 included below; oldest blocked for > 1226.857977 secs 2024-03-28T18:37:20.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:37:20 smithi079 ceph-mon[35312]: pgmap v946: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:37:20.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:37:20 smithi079 ceph-mon[35312]: 2 slow requests, 0 included below; oldest blocked for > 1226.857977 secs 2024-03-28T18:37:22.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:37:22 smithi022 ceph-mon[31985]: pgmap v947: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:37:22.674 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:37:22 smithi134 ceph-mon[35037]: pgmap v947: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:37:22.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:37:22 smithi079 ceph-mon[35312]: pgmap v947: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:37:24.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:37:24 smithi022 ceph-mon[31985]: pgmap v948: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:37:24.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:37:24 smithi134 ceph-mon[35037]: pgmap v948: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:37:24.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:37:24 smithi079 ceph-mon[35312]: pgmap v948: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:37:25.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:37:25 smithi022 ceph-mon[31985]: 2 slow requests, 0 included below; oldest blocked for > 1231.858062 secs 2024-03-28T18:37:25.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:37:25 smithi134 ceph-mon[35037]: 2 slow requests, 0 included below; oldest blocked for > 1231.858062 secs 2024-03-28T18:37:25.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:37:25 smithi079 ceph-mon[35312]: 2 slow requests, 0 included below; oldest blocked for > 1231.858062 secs 2024-03-28T18:37:26.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:37:26 smithi022 ceph-mon[31985]: pgmap v949: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:37:26.674 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:37:26 smithi134 ceph-mon[35037]: pgmap v949: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:37:26.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:37:26 smithi079 ceph-mon[35312]: pgmap v949: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:37:27.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:37:27 smithi022 ceph-mon[31985]: pgmap v950: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:37:27.674 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:37:27 smithi134 ceph-mon[35037]: pgmap v950: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:37:27.683 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:37:27 smithi079 ceph-mon[35312]: pgmap v950: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:37:30.425 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:37:30 smithi134 ceph-mon[35037]: pgmap v951: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:37:30.425 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:37:30 smithi134 ceph-mon[35037]: 2 slow requests, 0 included below; oldest blocked for > 1236.858135 secs 2024-03-28T18:37:30.432 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:37:30 smithi079 ceph-mon[35312]: pgmap v951: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:37:30.433 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:37:30 smithi079 ceph-mon[35312]: 2 slow requests, 0 included below; oldest blocked for > 1236.858135 secs 2024-03-28T18:37:30.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:37:30 smithi022 ceph-mon[31985]: pgmap v951: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:37:30.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:37:30 smithi022 ceph-mon[31985]: 2 slow requests, 0 included below; oldest blocked for > 1236.858135 secs 2024-03-28T18:37:32.424 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:37:32 smithi134 ceph-mon[35037]: pgmap v952: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:37:32.432 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:37:32 smithi079 ceph-mon[35312]: pgmap v952: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:37:32.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:37:32 smithi022 ceph-mon[31985]: pgmap v952: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:37:34.432 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:37:34 smithi079 ceph-mon[35312]: pgmap v953: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:37:34.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:37:34 smithi022 ceph-mon[31985]: pgmap v953: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:37:34.674 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:37:34 smithi134 ceph-mon[35037]: pgmap v953: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:37:35.432 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:37:35 smithi079 ceph-mon[35312]: 2 slow requests, 0 included below; oldest blocked for > 1241.858181 secs 2024-03-28T18:37:35.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:37:35 smithi022 ceph-mon[31985]: 2 slow requests, 0 included below; oldest blocked for > 1241.858181 secs 2024-03-28T18:37:35.674 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:37:35 smithi134 ceph-mon[35037]: 2 slow requests, 0 included below; oldest blocked for > 1241.858181 secs 2024-03-28T18:37:36.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:37:36 smithi022 ceph-mon[31985]: pgmap v954: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 9 op/s 2024-03-28T18:37:36.674 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:37:36 smithi134 ceph-mon[35037]: pgmap v954: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 9 op/s 2024-03-28T18:37:36.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:37:36 smithi079 ceph-mon[35312]: pgmap v954: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 9 op/s 2024-03-28T18:37:38.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:37:38 smithi022 ceph-mon[31985]: pgmap v955: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T18:37:38.674 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:37:38 smithi134 ceph-mon[35037]: pgmap v955: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T18:37:38.683 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:37:38 smithi079 ceph-mon[35312]: pgmap v955: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T18:37:40.477 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:37:40 smithi134 ceph-mon[35037]: pgmap v956: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 9 op/s 2024-03-28T18:37:40.477 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:37:40 smithi134 ceph-mon[35037]: 2 slow requests, 0 included below; oldest blocked for > 1246.858257 secs 2024-03-28T18:37:40.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:37:40 smithi022 ceph-mon[31985]: pgmap v956: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 9 op/s 2024-03-28T18:37:40.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:37:40 smithi022 ceph-mon[31985]: 2 slow requests, 0 included below; oldest blocked for > 1246.858257 secs 2024-03-28T18:37:40.683 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:37:40 smithi079 ceph-mon[35312]: pgmap v956: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 9 op/s 2024-03-28T18:37:40.683 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:37:40 smithi079 ceph-mon[35312]: 2 slow requests, 0 included below; oldest blocked for > 1246.858257 secs 2024-03-28T18:37:42.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:37:42 smithi022 ceph-mon[31985]: pgmap v957: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 9 op/s 2024-03-28T18:37:42.674 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:37:42 smithi134 ceph-mon[35037]: pgmap v957: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 9 op/s 2024-03-28T18:37:42.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:37:42 smithi079 ceph-mon[35312]: pgmap v957: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 9 op/s 2024-03-28T18:37:44.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:37:44 smithi022 ceph-mon[31985]: pgmap v958: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T18:37:44.674 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:37:44 smithi134 ceph-mon[35037]: pgmap v958: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T18:37:44.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:37:44 smithi079 ceph-mon[35312]: pgmap v958: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T18:37:45.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:37:45 smithi022 ceph-mon[31985]: 2 slow requests, 0 included below; oldest blocked for > 1251.858335 secs 2024-03-28T18:37:45.674 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:37:45 smithi134 ceph-mon[35037]: 2 slow requests, 0 included below; oldest blocked for > 1251.858335 secs 2024-03-28T18:37:45.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:37:45 smithi079 ceph-mon[35312]: 2 slow requests, 0 included below; oldest blocked for > 1251.858335 secs 2024-03-28T18:37:46.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:37:46 smithi022 ceph-mon[31985]: pgmap v959: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 9 op/s 2024-03-28T18:37:46.674 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:37:46 smithi134 ceph-mon[35037]: pgmap v959: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 9 op/s 2024-03-28T18:37:46.683 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:37:46 smithi079 ceph-mon[35312]: pgmap v959: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 9 op/s 2024-03-28T18:37:48.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:37:48 smithi022 ceph-mon[31985]: pgmap v960: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:37:48.674 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:37:48 smithi134 ceph-mon[35037]: pgmap v960: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:37:48.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:37:48 smithi079 ceph-mon[35312]: pgmap v960: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:37:50.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:37:50 smithi022 ceph-mon[31985]: pgmap v961: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:37:50.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:37:50 smithi022 ceph-mon[31985]: 2 slow requests, 0 included below; oldest blocked for > 1256.858399 secs 2024-03-28T18:37:50.674 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:37:50 smithi134 ceph-mon[35037]: pgmap v961: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:37:50.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:37:50 smithi134 ceph-mon[35037]: 2 slow requests, 0 included below; oldest blocked for > 1256.858399 secs 2024-03-28T18:37:50.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:37:50 smithi079 ceph-mon[35312]: pgmap v961: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:37:50.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:37:50 smithi079 ceph-mon[35312]: 2 slow requests, 0 included below; oldest blocked for > 1256.858399 secs 2024-03-28T18:37:52.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:37:52 smithi022 ceph-mon[31985]: pgmap v962: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:37:52.674 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:37:52 smithi134 ceph-mon[35037]: pgmap v962: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:37:52.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:37:52 smithi079 ceph-mon[35312]: pgmap v962: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:37:54.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:37:54 smithi022 ceph-mon[31985]: pgmap v963: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:37:54.674 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:37:54 smithi134 ceph-mon[35037]: pgmap v963: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:37:54.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:37:54 smithi079 ceph-mon[35312]: pgmap v963: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:37:55.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:37:55 smithi022 ceph-mon[31985]: 2 slow requests, 0 included below; oldest blocked for > 1261.858476 secs 2024-03-28T18:37:55.674 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:37:55 smithi134 ceph-mon[35037]: 2 slow requests, 0 included below; oldest blocked for > 1261.858476 secs 2024-03-28T18:37:55.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:37:55 smithi079 ceph-mon[35312]: 2 slow requests, 0 included below; oldest blocked for > 1261.858476 secs 2024-03-28T18:37:56.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:37:56 smithi022 ceph-mon[31985]: pgmap v964: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:37:56.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:37:56 smithi022 ceph-mon[31985]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T18:37:56.674 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:37:56 smithi134 ceph-mon[35037]: pgmap v964: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:37:56.674 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:37:56 smithi134 ceph-mon[35037]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T18:37:56.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:37:56 smithi079 ceph-mon[35312]: pgmap v964: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:37:56.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:37:56 smithi079 ceph-mon[35312]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T18:37:57.674 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:37:57 smithi134 ceph-mon[35037]: pgmap v965: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:37:57.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:37:57 smithi079 ceph-mon[35312]: pgmap v965: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:37:57.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:37:57 smithi022 ceph-mon[31985]: pgmap v965: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:37:59.033 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:37:58 smithi022 ceph-mon[31985]: from='mgr.15522 ' entity='mgr.x' 2024-03-28T18:37:59.033 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:37:58 smithi022 ceph-mon[31985]: from='mgr.15522 ' entity='mgr.x' 2024-03-28T18:37:59.033 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:37:58 smithi022 ceph-mon[31985]: from='mgr.15522 ' entity='mgr.x' 2024-03-28T18:37:59.033 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:37:58 smithi022 ceph-mon[31985]: from='mgr.15522 ' entity='mgr.x' 2024-03-28T18:37:59.033 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:37:58 smithi022 ceph-mon[31985]: from='mgr.15522 ' entity='mgr.x' 2024-03-28T18:37:59.033 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:37:58 smithi022 ceph-mon[31985]: from='mgr.15522 ' entity='mgr.x' 2024-03-28T18:37:59.033 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:37:58 smithi022 ceph-mon[31985]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T18:37:59.033 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:37:58 smithi022 ceph-mon[31985]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T18:37:59.033 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:37:58 smithi022 ceph-mon[31985]: pgmap v966: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T18:37:59.033 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:37:58 smithi022 ceph-mon[31985]: pgmap v967: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s rd, 4 op/s 2024-03-28T18:37:59.033 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:37:58 smithi022 ceph-mon[31985]: from='mgr.15522 ' entity='mgr.x' 2024-03-28T18:37:59.175 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:37:58 smithi134 ceph-mon[35037]: from='mgr.15522 ' entity='mgr.x' 2024-03-28T18:37:59.175 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:37:58 smithi134 ceph-mon[35037]: from='mgr.15522 ' entity='mgr.x' 2024-03-28T18:37:59.175 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:37:58 smithi134 ceph-mon[35037]: from='mgr.15522 ' entity='mgr.x' 2024-03-28T18:37:59.175 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:37:58 smithi134 ceph-mon[35037]: from='mgr.15522 ' entity='mgr.x' 2024-03-28T18:37:59.175 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:37:58 smithi134 ceph-mon[35037]: from='mgr.15522 ' entity='mgr.x' 2024-03-28T18:37:59.175 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:37:58 smithi134 ceph-mon[35037]: from='mgr.15522 ' entity='mgr.x' 2024-03-28T18:37:59.175 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:37:58 smithi134 ceph-mon[35037]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T18:37:59.175 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:37:58 smithi134 ceph-mon[35037]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T18:37:59.175 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:37:58 smithi134 ceph-mon[35037]: pgmap v966: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T18:37:59.175 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:37:58 smithi134 ceph-mon[35037]: pgmap v967: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s rd, 4 op/s 2024-03-28T18:37:59.176 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:37:58 smithi134 ceph-mon[35037]: from='mgr.15522 ' entity='mgr.x' 2024-03-28T18:37:59.183 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:37:58 smithi079 ceph-mon[35312]: from='mgr.15522 ' entity='mgr.x' 2024-03-28T18:37:59.183 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:37:58 smithi079 ceph-mon[35312]: from='mgr.15522 ' entity='mgr.x' 2024-03-28T18:37:59.183 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:37:58 smithi079 ceph-mon[35312]: from='mgr.15522 ' entity='mgr.x' 2024-03-28T18:37:59.183 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:37:58 smithi079 ceph-mon[35312]: from='mgr.15522 ' entity='mgr.x' 2024-03-28T18:37:59.183 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:37:58 smithi079 ceph-mon[35312]: from='mgr.15522 ' entity='mgr.x' 2024-03-28T18:37:59.183 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:37:58 smithi079 ceph-mon[35312]: from='mgr.15522 ' entity='mgr.x' 2024-03-28T18:37:59.183 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:37:58 smithi079 ceph-mon[35312]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T18:37:59.183 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:37:58 smithi079 ceph-mon[35312]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T18:37:59.184 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:37:58 smithi079 ceph-mon[35312]: pgmap v966: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T18:37:59.184 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:37:58 smithi079 ceph-mon[35312]: pgmap v967: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s rd, 4 op/s 2024-03-28T18:37:59.184 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:37:58 smithi079 ceph-mon[35312]: from='mgr.15522 ' entity='mgr.x' 2024-03-28T18:38:00.174 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:37:59 smithi134 ceph-mon[35037]: 2 slow requests, 0 included below; oldest blocked for > 1266.858594 secs 2024-03-28T18:38:00.182 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:37:59 smithi079 ceph-mon[35312]: 2 slow requests, 0 included below; oldest blocked for > 1266.858594 secs 2024-03-28T18:38:00.282 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:37:59 smithi022 ceph-mon[31985]: 2 slow requests, 0 included below; oldest blocked for > 1266.858594 secs 2024-03-28T18:38:01.175 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:38:00 smithi134 ceph-mon[35037]: pgmap v968: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-03-28T18:38:01.182 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:38:00 smithi079 ceph-mon[35312]: pgmap v968: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-03-28T18:38:01.282 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:38:00 smithi022 ceph-mon[31985]: pgmap v968: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-03-28T18:38:03.174 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:38:02 smithi134 ceph-mon[35037]: pgmap v969: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-03-28T18:38:03.182 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:38:02 smithi079 ceph-mon[35312]: pgmap v969: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-03-28T18:38:03.282 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:38:02 smithi022 ceph-mon[31985]: pgmap v969: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-03-28T18:38:05.175 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:38:04 smithi134 ceph-mon[35037]: pgmap v970: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s rd, 4 op/s 2024-03-28T18:38:05.327 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:38:04 smithi134 ceph-mon[35037]: 2 slow requests, 0 included below; oldest blocked for > 1271.858658 secs 2024-03-28T18:38:05.328 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:38:04 smithi022 ceph-mon[31985]: pgmap v970: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s rd, 4 op/s 2024-03-28T18:38:05.329 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:38:04 smithi022 ceph-mon[31985]: 2 slow requests, 0 included below; oldest blocked for > 1271.858658 secs 2024-03-28T18:38:05.329 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:38:04 smithi079 ceph-mon[35312]: pgmap v970: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s rd, 4 op/s 2024-03-28T18:38:05.329 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:38:04 smithi079 ceph-mon[35312]: 2 slow requests, 0 included below; oldest blocked for > 1271.858658 secs 2024-03-28T18:38:07.174 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:38:06 smithi134 ceph-mon[35037]: pgmap v971: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-03-28T18:38:07.182 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:38:06 smithi079 ceph-mon[35312]: pgmap v971: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-03-28T18:38:07.281 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:38:06 smithi022 ceph-mon[31985]: pgmap v971: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-03-28T18:38:09.175 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:38:08 smithi134 ceph-mon[35037]: pgmap v972: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 7 op/s 2024-03-28T18:38:09.182 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:38:08 smithi079 ceph-mon[35312]: pgmap v972: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 7 op/s 2024-03-28T18:38:09.282 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:38:08 smithi022 ceph-mon[31985]: pgmap v972: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 7 op/s 2024-03-28T18:38:10.175 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:38:09 smithi134 ceph-mon[35037]: 2 slow requests, 0 included below; oldest blocked for > 1276.858737 secs 2024-03-28T18:38:10.182 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:38:09 smithi079 ceph-mon[35312]: 2 slow requests, 0 included below; oldest blocked for > 1276.858737 secs 2024-03-28T18:38:10.184 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:38:09 smithi022 ceph-mon[31985]: 2 slow requests, 0 included below; oldest blocked for > 1276.858737 secs 2024-03-28T18:38:11.174 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:38:10 smithi134 ceph-mon[35037]: pgmap v973: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:38:11.182 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:38:10 smithi079 ceph-mon[35312]: pgmap v973: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:38:11.282 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:38:10 smithi022 ceph-mon[31985]: pgmap v973: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:38:13.175 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:38:12 smithi134 ceph-mon[35037]: pgmap v974: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:38:13.182 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:38:12 smithi079 ceph-mon[35312]: pgmap v974: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:38:13.282 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:38:12 smithi022 ceph-mon[31985]: pgmap v974: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:38:15.175 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:38:14 smithi134 ceph-mon[35037]: pgmap v975: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:38:15.175 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:38:14 smithi134 ceph-mon[35037]: 2 slow requests, 0 included below; oldest blocked for > 1281.858861 secs 2024-03-28T18:38:15.182 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:38:14 smithi079 ceph-mon[35312]: pgmap v975: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:38:15.183 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:38:14 smithi079 ceph-mon[35312]: 2 slow requests, 0 included below; oldest blocked for > 1281.858861 secs 2024-03-28T18:38:15.282 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:38:14 smithi022 ceph-mon[31985]: pgmap v975: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:38:15.282 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:38:14 smithi022 ceph-mon[31985]: 2 slow requests, 0 included below; oldest blocked for > 1281.858861 secs 2024-03-28T18:38:17.183 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:38:16 smithi079 ceph-mon[35312]: pgmap v976: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:38:17.282 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:38:16 smithi022 ceph-mon[31985]: pgmap v976: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:38:17.424 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:38:16 smithi134 ceph-mon[35037]: pgmap v976: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:38:19.182 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:38:18 smithi079 ceph-mon[35312]: pgmap v977: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:38:19.282 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:38:18 smithi022 ceph-mon[31985]: pgmap v977: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:38:19.424 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:38:18 smithi134 ceph-mon[35037]: pgmap v977: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:38:20.282 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:38:19 smithi022 ceph-mon[31985]: 2 slow requests, 0 included below; oldest blocked for > 1286.858934 secs 2024-03-28T18:38:20.424 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:38:19 smithi134 ceph-mon[35037]: 2 slow requests, 0 included below; oldest blocked for > 1286.858934 secs 2024-03-28T18:38:20.432 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:38:19 smithi079 ceph-mon[35312]: 2 slow requests, 0 included below; oldest blocked for > 1286.858934 secs 2024-03-28T18:38:21.282 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:38:20 smithi022 ceph-mon[31985]: pgmap v978: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:38:21.424 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:38:20 smithi134 ceph-mon[35037]: pgmap v978: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:38:21.433 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:38:20 smithi079 ceph-mon[35312]: pgmap v978: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:38:23.282 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:38:22 smithi022 ceph-mon[31985]: pgmap v979: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:38:23.425 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:38:22 smithi134 ceph-mon[35037]: pgmap v979: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:38:23.432 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:38:22 smithi079 ceph-mon[35312]: pgmap v979: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:38:25.282 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:38:24 smithi022 ceph-mon[31985]: pgmap v980: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:38:25.282 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:38:24 smithi022 ceph-mon[31985]: 2 slow requests, 0 included below; oldest blocked for > 1291.859013 secs 2024-03-28T18:38:25.425 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:38:24 smithi134 ceph-mon[35037]: pgmap v980: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:38:25.425 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:38:24 smithi134 ceph-mon[35037]: 2 slow requests, 0 included below; oldest blocked for > 1291.859013 secs 2024-03-28T18:38:25.433 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:38:24 smithi079 ceph-mon[35312]: pgmap v980: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:38:25.433 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:38:24 smithi079 ceph-mon[35312]: 2 slow requests, 0 included below; oldest blocked for > 1291.859013 secs 2024-03-28T18:38:27.282 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:38:26 smithi022 ceph-mon[31985]: pgmap v981: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:38:27.424 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:38:26 smithi134 ceph-mon[35037]: pgmap v981: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:38:27.432 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:38:26 smithi079 ceph-mon[35312]: pgmap v981: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:38:29.282 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:38:28 smithi022 ceph-mon[31985]: pgmap v982: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:38:29.424 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:38:28 smithi134 ceph-mon[35037]: pgmap v982: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:38:29.432 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:38:28 smithi079 ceph-mon[35312]: pgmap v982: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:38:30.282 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:38:29 smithi022 ceph-mon[31985]: 2 slow requests, 0 included below; oldest blocked for > 1296.859091 secs 2024-03-28T18:38:30.425 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:38:29 smithi134 ceph-mon[35037]: 2 slow requests, 0 included below; oldest blocked for > 1296.859091 secs 2024-03-28T18:38:30.432 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:38:29 smithi079 ceph-mon[35312]: 2 slow requests, 0 included below; oldest blocked for > 1296.859091 secs 2024-03-28T18:38:31.282 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:38:30 smithi022 ceph-mon[31985]: pgmap v983: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:38:31.424 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:38:30 smithi134 ceph-mon[35037]: pgmap v983: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:38:31.432 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:38:30 smithi079 ceph-mon[35312]: pgmap v983: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:38:33.282 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:38:32 smithi022 ceph-mon[31985]: pgmap v984: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:38:33.424 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:38:32 smithi134 ceph-mon[35037]: pgmap v984: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:38:33.432 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:38:32 smithi079 ceph-mon[35312]: pgmap v984: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:38:35.282 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:38:34 smithi022 ceph-mon[31985]: pgmap v985: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:38:35.282 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:38:34 smithi022 ceph-mon[31985]: 2 slow requests, 0 included below; oldest blocked for > 1301.859177 secs 2024-03-28T18:38:35.424 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:38:34 smithi134 ceph-mon[35037]: pgmap v985: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:38:35.424 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:38:34 smithi134 ceph-mon[35037]: 2 slow requests, 0 included below; oldest blocked for > 1301.859177 secs 2024-03-28T18:38:35.433 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:38:34 smithi079 ceph-mon[35312]: pgmap v985: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:38:35.433 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:38:34 smithi079 ceph-mon[35312]: 2 slow requests, 0 included below; oldest blocked for > 1301.859177 secs 2024-03-28T18:38:37.282 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:38:37 smithi022 ceph-mon[31985]: pgmap v986: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:38:37.424 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:38:37 smithi134 ceph-mon[35037]: pgmap v986: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:38:37.432 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:38:37 smithi079 ceph-mon[35312]: pgmap v986: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:38:39.282 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:38:39 smithi022 ceph-mon[31985]: pgmap v987: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:38:39.425 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:38:39 smithi134 ceph-mon[35037]: pgmap v987: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:38:39.432 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:38:39 smithi079 ceph-mon[35312]: pgmap v987: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:38:40.282 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:38:40 smithi022 ceph-mon[31985]: 2 slow requests, 0 included below; oldest blocked for > 1306.859251 secs 2024-03-28T18:38:40.326 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:38:40 smithi134 ceph-mon[35037]: 2 slow requests, 0 included below; oldest blocked for > 1306.859251 secs 2024-03-28T18:38:40.433 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:38:40 smithi079 ceph-mon[35312]: 2 slow requests, 0 included below; oldest blocked for > 1306.859251 secs 2024-03-28T18:38:41.282 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:38:41 smithi022 ceph-mon[31985]: pgmap v988: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:38:41.424 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:38:41 smithi134 ceph-mon[35037]: pgmap v988: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:38:41.432 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:38:41 smithi079 ceph-mon[35312]: pgmap v988: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:38:43.282 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:38:43 smithi022 ceph-mon[31985]: pgmap v989: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:38:43.425 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:38:43 smithi134 ceph-mon[35037]: pgmap v989: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:38:43.432 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:38:43 smithi079 ceph-mon[35312]: pgmap v989: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:38:45.425 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:38:45 smithi134 ceph-mon[35037]: pgmap v990: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:38:45.425 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:38:45 smithi134 ceph-mon[35037]: 2 slow requests, 0 included below; oldest blocked for > 1311.859268 secs 2024-03-28T18:38:45.432 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:38:45 smithi079 ceph-mon[35312]: pgmap v990: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:38:45.432 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:38:45 smithi079 ceph-mon[35312]: 2 slow requests, 0 included below; oldest blocked for > 1311.859268 secs 2024-03-28T18:38:45.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:38:45 smithi022 ceph-mon[31985]: pgmap v990: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:38:45.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:38:45 smithi022 ceph-mon[31985]: 2 slow requests, 0 included below; oldest blocked for > 1311.859268 secs 2024-03-28T18:38:47.425 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:38:47 smithi134 ceph-mon[35037]: pgmap v991: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:38:47.433 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:38:47 smithi079 ceph-mon[35312]: pgmap v991: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:38:47.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:38:47 smithi022 ceph-mon[31985]: pgmap v991: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:38:49.425 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:38:49 smithi134 ceph-mon[35037]: pgmap v992: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:38:49.432 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:38:49 smithi079 ceph-mon[35312]: pgmap v992: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:38:49.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:38:49 smithi022 ceph-mon[31985]: pgmap v992: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:38:50.425 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:38:50 smithi134 ceph-mon[35037]: 2 slow requests, 0 included below; oldest blocked for > 1316.859392 secs 2024-03-28T18:38:50.432 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:38:50 smithi079 ceph-mon[35312]: 2 slow requests, 0 included below; oldest blocked for > 1316.859392 secs 2024-03-28T18:38:50.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:38:50 smithi022 ceph-mon[31985]: 2 slow requests, 0 included below; oldest blocked for > 1316.859392 secs 2024-03-28T18:38:51.425 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:38:51 smithi134 ceph-mon[35037]: pgmap v993: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:38:51.432 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:38:51 smithi079 ceph-mon[35312]: pgmap v993: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:38:51.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:38:51 smithi022 ceph-mon[31985]: pgmap v993: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:38:53.425 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:38:53 smithi134 ceph-mon[35037]: pgmap v994: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:38:53.432 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:38:53 smithi079 ceph-mon[35312]: pgmap v994: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:38:53.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:38:53 smithi022 ceph-mon[31985]: pgmap v994: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:38:55.424 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:38:55 smithi134 ceph-mon[35037]: pgmap v995: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:38:55.425 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:38:55 smithi134 ceph-mon[35037]: 2 slow requests, 0 included below; oldest blocked for > 1321.859459 secs 2024-03-28T18:38:55.432 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:38:55 smithi079 ceph-mon[35312]: pgmap v995: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:38:55.432 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:38:55 smithi079 ceph-mon[35312]: 2 slow requests, 0 included below; oldest blocked for > 1321.859459 secs 2024-03-28T18:38:55.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:38:55 smithi022 ceph-mon[31985]: pgmap v995: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:38:55.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:38:55 smithi022 ceph-mon[31985]: 2 slow requests, 0 included below; oldest blocked for > 1321.859459 secs 2024-03-28T18:38:57.424 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:38:57 smithi134 ceph-mon[35037]: pgmap v996: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:38:57.432 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:38:57 smithi079 ceph-mon[35312]: pgmap v996: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:38:57.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:38:57 smithi022 ceph-mon[31985]: pgmap v996: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:38:59.425 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:38:59 smithi134 ceph-mon[35037]: pgmap v997: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:38:59.425 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:38:59 smithi134 ceph-mon[35037]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T18:38:59.433 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:38:59 smithi079 ceph-mon[35312]: pgmap v997: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:38:59.433 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:38:59 smithi079 ceph-mon[35312]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T18:38:59.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:38:59 smithi022 ceph-mon[31985]: pgmap v997: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:38:59.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:38:59 smithi022 ceph-mon[31985]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T18:39:00.424 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:39:00 smithi134 ceph-mon[35037]: from='mgr.15522 ' entity='mgr.x' 2024-03-28T18:39:00.425 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:39:00 smithi134 ceph-mon[35037]: from='mgr.15522 ' entity='mgr.x' 2024-03-28T18:39:00.425 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:39:00 smithi134 ceph-mon[35037]: 2 slow requests, 0 included below; oldest blocked for > 1326.859524 secs 2024-03-28T18:39:00.432 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:39:00 smithi079 ceph-mon[35312]: from='mgr.15522 ' entity='mgr.x' 2024-03-28T18:39:00.433 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:39:00 smithi079 ceph-mon[35312]: from='mgr.15522 ' entity='mgr.x' 2024-03-28T18:39:00.433 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:39:00 smithi079 ceph-mon[35312]: 2 slow requests, 0 included below; oldest blocked for > 1326.859524 secs 2024-03-28T18:39:00.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:39:00 smithi022 ceph-mon[31985]: from='mgr.15522 ' entity='mgr.x' 2024-03-28T18:39:00.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:39:00 smithi022 ceph-mon[31985]: from='mgr.15522 ' entity='mgr.x' 2024-03-28T18:39:00.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:39:00 smithi022 ceph-mon[31985]: 2 slow requests, 0 included below; oldest blocked for > 1326.859524 secs 2024-03-28T18:39:01.424 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:39:01 smithi134 ceph-mon[35037]: pgmap v998: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:39:01.432 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:39:01 smithi079 ceph-mon[35312]: pgmap v998: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:39:01.531 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:39:01 smithi022 ceph-mon[31985]: pgmap v998: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:39:02.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:39:02 smithi134 ceph-mon[35037]: from='mgr.15522 ' entity='mgr.x' 2024-03-28T18:39:02.925 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:39:02 smithi134 ceph-mon[35037]: from='mgr.15522 ' entity='mgr.x' 2024-03-28T18:39:02.925 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:39:02 smithi134 ceph-mon[35037]: pgmap v999: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:39:02.925 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:39:02 smithi134 ceph-mon[35037]: from='mgr.15522 ' entity='mgr.x' 2024-03-28T18:39:02.925 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:39:02 smithi134 ceph-mon[35037]: from='mgr.15522 ' entity='mgr.x' 2024-03-28T18:39:02.925 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:39:02 smithi134 ceph-mon[35037]: from='mgr.15522 ' entity='mgr.x' 2024-03-28T18:39:02.925 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:39:02 smithi134 ceph-mon[35037]: from='mgr.15522 ' entity='mgr.x' 2024-03-28T18:39:02.933 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:39:02 smithi079 ceph-mon[35312]: from='mgr.15522 ' entity='mgr.x' 2024-03-28T18:39:02.933 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:39:02 smithi079 ceph-mon[35312]: from='mgr.15522 ' entity='mgr.x' 2024-03-28T18:39:02.933 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:39:02 smithi079 ceph-mon[35312]: pgmap v999: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:39:02.933 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:39:02 smithi079 ceph-mon[35312]: from='mgr.15522 ' entity='mgr.x' 2024-03-28T18:39:02.933 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:39:02 smithi079 ceph-mon[35312]: from='mgr.15522 ' entity='mgr.x' 2024-03-28T18:39:02.933 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:39:02 smithi079 ceph-mon[35312]: from='mgr.15522 ' entity='mgr.x' 2024-03-28T18:39:02.933 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:39:02 smithi079 ceph-mon[35312]: from='mgr.15522 ' entity='mgr.x' 2024-03-28T18:39:03.032 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:39:02 smithi022 ceph-mon[31985]: from='mgr.15522 ' entity='mgr.x' 2024-03-28T18:39:03.032 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:39:02 smithi022 ceph-mon[31985]: from='mgr.15522 ' entity='mgr.x' 2024-03-28T18:39:03.032 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:39:02 smithi022 ceph-mon[31985]: pgmap v999: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:39:03.032 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:39:02 smithi022 ceph-mon[31985]: from='mgr.15522 ' entity='mgr.x' 2024-03-28T18:39:03.032 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:39:02 smithi022 ceph-mon[31985]: from='mgr.15522 ' entity='mgr.x' 2024-03-28T18:39:03.032 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:39:02 smithi022 ceph-mon[31985]: from='mgr.15522 ' entity='mgr.x' 2024-03-28T18:39:03.032 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:39:02 smithi022 ceph-mon[31985]: from='mgr.15522 ' entity='mgr.x' 2024-03-28T18:39:03.926 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:39:03 smithi134 ceph-mon[35037]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-28T18:39:03.926 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:39:03 smithi134 ceph-mon[35037]: from='mgr.15522 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-28T18:39:03.926 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:39:03 smithi134 ceph-mon[35037]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-28T18:39:03.927 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:39:03 smithi134 ceph-mon[35037]: from='mgr.15522 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-28T18:39:03.927 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:39:03 smithi134 ceph-mon[35037]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-03-28T18:39:03.927 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:39:03 smithi134 ceph-mon[35037]: from='mgr.15522 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-03-28T18:39:03.927 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:39:03 smithi134 ceph-mon[35037]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-03-28T18:39:03.927 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:39:03 smithi134 ceph-mon[35037]: from='mgr.15522 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-03-28T18:39:03.927 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:39:03 smithi134 ceph-mon[35037]: Adjusting osd_memory_target on smithi079 to 176.7M 2024-03-28T18:39:03.927 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:39:03 smithi134 ceph-mon[35037]: Unable to set osd_memory_target on smithi079 to 185386700: error parsing value: Value '185386700' is below minimum 939524096 2024-03-28T18:39:03.927 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:39:03 smithi134 ceph-mon[35037]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-28T18:39:03.927 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:39:03 smithi134 ceph-mon[35037]: from='mgr.15522 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-28T18:39:03.928 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:39:03 smithi134 ceph-mon[35037]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-28T18:39:03.928 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:39:03 smithi134 ceph-mon[35037]: from='mgr.15522 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-28T18:39:03.928 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:39:03 smithi134 ceph-mon[35037]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-03-28T18:39:03.928 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:39:03 smithi134 ceph-mon[35037]: from='mgr.15522 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-03-28T18:39:03.928 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:39:03 smithi134 ceph-mon[35037]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-03-28T18:39:03.928 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:39:03 smithi134 ceph-mon[35037]: from='mgr.15522 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-03-28T18:39:03.928 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:39:03 smithi134 ceph-mon[35037]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-28T18:39:03.928 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:39:03 smithi134 ceph-mon[35037]: from='mgr.15522 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-28T18:39:03.929 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:39:03 smithi134 ceph-mon[35037]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-28T18:39:03.929 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:39:03 smithi134 ceph-mon[35037]: from='mgr.15522 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-28T18:39:03.929 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:39:03 smithi134 ceph-mon[35037]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-03-28T18:39:03.929 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:39:03 smithi134 ceph-mon[35037]: from='mgr.15522 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-03-28T18:39:03.929 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:39:03 smithi134 ceph-mon[35037]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-03-28T18:39:03.929 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:39:03 smithi134 ceph-mon[35037]: from='mgr.15522 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-03-28T18:39:03.929 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:39:03 smithi134 ceph-mon[35037]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T18:39:03.929 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:39:03 smithi134 ceph-mon[35037]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T18:39:03.929 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:39:03 smithi134 ceph-mon[35037]: from='mgr.15522 ' entity='mgr.x' 2024-03-28T18:39:03.933 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:39:03 smithi079 ceph-mon[35312]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-28T18:39:03.933 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:39:03 smithi079 ceph-mon[35312]: from='mgr.15522 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-28T18:39:03.933 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:39:03 smithi079 ceph-mon[35312]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-28T18:39:03.933 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:39:03 smithi079 ceph-mon[35312]: from='mgr.15522 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-28T18:39:03.933 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:39:03 smithi079 ceph-mon[35312]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-03-28T18:39:03.934 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:39:03 smithi079 ceph-mon[35312]: from='mgr.15522 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-03-28T18:39:03.934 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:39:03 smithi079 ceph-mon[35312]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-03-28T18:39:03.934 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:39:03 smithi079 ceph-mon[35312]: from='mgr.15522 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-03-28T18:39:03.934 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:39:03 smithi079 ceph-mon[35312]: Adjusting osd_memory_target on smithi079 to 176.7M 2024-03-28T18:39:03.934 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:39:03 smithi079 ceph-mon[35312]: Unable to set osd_memory_target on smithi079 to 185386700: error parsing value: Value '185386700' is below minimum 939524096 2024-03-28T18:39:03.934 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:39:03 smithi079 ceph-mon[35312]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-28T18:39:03.934 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:39:03 smithi079 ceph-mon[35312]: from='mgr.15522 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-28T18:39:03.934 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:39:03 smithi079 ceph-mon[35312]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-28T18:39:03.935 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:39:03 smithi079 ceph-mon[35312]: from='mgr.15522 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-28T18:39:03.935 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:39:03 smithi079 ceph-mon[35312]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-03-28T18:39:03.935 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:39:03 smithi079 ceph-mon[35312]: from='mgr.15522 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-03-28T18:39:03.935 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:39:03 smithi079 ceph-mon[35312]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-03-28T18:39:03.935 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:39:03 smithi079 ceph-mon[35312]: from='mgr.15522 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-03-28T18:39:03.935 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:39:03 smithi079 ceph-mon[35312]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-28T18:39:03.935 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:39:03 smithi079 ceph-mon[35312]: from='mgr.15522 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-28T18:39:03.935 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:39:03 smithi079 ceph-mon[35312]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-28T18:39:03.935 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:39:03 smithi079 ceph-mon[35312]: from='mgr.15522 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-28T18:39:03.935 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:39:03 smithi079 ceph-mon[35312]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-03-28T18:39:03.936 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:39:03 smithi079 ceph-mon[35312]: from='mgr.15522 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-03-28T18:39:03.936 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:39:03 smithi079 ceph-mon[35312]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-03-28T18:39:03.936 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:39:03 smithi079 ceph-mon[35312]: from='mgr.15522 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-03-28T18:39:03.936 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:39:03 smithi079 ceph-mon[35312]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T18:39:03.936 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:39:03 smithi079 ceph-mon[35312]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T18:39:03.936 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:39:03 smithi079 ceph-mon[35312]: from='mgr.15522 ' entity='mgr.x' 2024-03-28T18:39:04.032 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:39:03 smithi022 ceph-mon[31985]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-28T18:39:04.033 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:39:03 smithi022 ceph-mon[31985]: from='mgr.15522 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-28T18:39:04.033 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:39:03 smithi022 ceph-mon[31985]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-28T18:39:04.033 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:39:03 smithi022 ceph-mon[31985]: from='mgr.15522 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-28T18:39:04.033 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:39:03 smithi022 ceph-mon[31985]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-03-28T18:39:04.033 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:39:03 smithi022 ceph-mon[31985]: from='mgr.15522 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-03-28T18:39:04.033 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:39:03 smithi022 ceph-mon[31985]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-03-28T18:39:04.033 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:39:03 smithi022 ceph-mon[31985]: from='mgr.15522 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-03-28T18:39:04.033 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:39:03 smithi022 ceph-mon[31985]: Adjusting osd_memory_target on smithi079 to 176.7M 2024-03-28T18:39:04.033 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:39:03 smithi022 ceph-mon[31985]: Unable to set osd_memory_target on smithi079 to 185386700: error parsing value: Value '185386700' is below minimum 939524096 2024-03-28T18:39:04.033 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:39:03 smithi022 ceph-mon[31985]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-28T18:39:04.033 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:39:03 smithi022 ceph-mon[31985]: from='mgr.15522 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-28T18:39:04.033 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:39:03 smithi022 ceph-mon[31985]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-28T18:39:04.033 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:39:03 smithi022 ceph-mon[31985]: from='mgr.15522 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-28T18:39:04.034 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:39:03 smithi022 ceph-mon[31985]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-03-28T18:39:04.034 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:39:03 smithi022 ceph-mon[31985]: from='mgr.15522 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-03-28T18:39:04.034 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:39:03 smithi022 ceph-mon[31985]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-03-28T18:39:04.034 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:39:03 smithi022 ceph-mon[31985]: from='mgr.15522 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-03-28T18:39:04.034 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:39:03 smithi022 ceph-mon[31985]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-28T18:39:04.034 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:39:03 smithi022 ceph-mon[31985]: from='mgr.15522 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-28T18:39:04.034 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:39:03 smithi022 ceph-mon[31985]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-28T18:39:04.034 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:39:03 smithi022 ceph-mon[31985]: from='mgr.15522 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-28T18:39:04.034 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:39:03 smithi022 ceph-mon[31985]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-03-28T18:39:04.034 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:39:03 smithi022 ceph-mon[31985]: from='mgr.15522 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-03-28T18:39:04.034 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:39:03 smithi022 ceph-mon[31985]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-03-28T18:39:04.034 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:39:03 smithi022 ceph-mon[31985]: from='mgr.15522 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-03-28T18:39:04.034 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:39:03 smithi022 ceph-mon[31985]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T18:39:04.034 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:39:03 smithi022 ceph-mon[31985]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T18:39:04.034 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:39:03 smithi022 ceph-mon[31985]: from='mgr.15522 ' entity='mgr.x' 2024-03-28T18:39:04.926 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:39:04 smithi134 ceph-mon[35037]: Adjusting osd_memory_target on smithi022 to 177.8M 2024-03-28T18:39:04.926 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:39:04 smithi134 ceph-mon[35037]: Unable to set osd_memory_target on smithi022 to 186497740: error parsing value: Value '186497740' is below minimum 939524096 2024-03-28T18:39:04.926 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:39:04 smithi134 ceph-mon[35037]: Adjusting osd_memory_target on smithi134 to 176.8M 2024-03-28T18:39:04.926 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:39:04 smithi134 ceph-mon[35037]: Unable to set osd_memory_target on smithi134 to 185393868: error parsing value: Value '185393868' is below minimum 939524096 2024-03-28T18:39:04.926 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:39:04 smithi134 ceph-mon[35037]: pgmap v1000: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 6 op/s 2024-03-28T18:39:04.926 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:39:04 smithi134 ceph-mon[35037]: pgmap v1001: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-28T18:39:04.926 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:39:04 smithi134 ceph-mon[35037]: 2 slow requests, 0 included below; oldest blocked for > 1331.859648 secs 2024-03-28T18:39:04.933 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:39:04 smithi079 ceph-mon[35312]: Adjusting osd_memory_target on smithi022 to 177.8M 2024-03-28T18:39:04.933 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:39:04 smithi079 ceph-mon[35312]: Unable to set osd_memory_target on smithi022 to 186497740: error parsing value: Value '186497740' is below minimum 939524096 2024-03-28T18:39:04.933 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:39:04 smithi079 ceph-mon[35312]: Adjusting osd_memory_target on smithi134 to 176.8M 2024-03-28T18:39:04.933 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:39:04 smithi079 ceph-mon[35312]: Unable to set osd_memory_target on smithi134 to 185393868: error parsing value: Value '185393868' is below minimum 939524096 2024-03-28T18:39:04.933 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:39:04 smithi079 ceph-mon[35312]: pgmap v1000: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 6 op/s 2024-03-28T18:39:04.933 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:39:04 smithi079 ceph-mon[35312]: pgmap v1001: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-28T18:39:04.933 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:39:04 smithi079 ceph-mon[35312]: 2 slow requests, 0 included below; oldest blocked for > 1331.859648 secs 2024-03-28T18:39:05.032 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:39:04 smithi022 ceph-mon[31985]: Adjusting osd_memory_target on smithi022 to 177.8M 2024-03-28T18:39:05.032 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:39:04 smithi022 ceph-mon[31985]: Unable to set osd_memory_target on smithi022 to 186497740: error parsing value: Value '186497740' is below minimum 939524096 2024-03-28T18:39:05.032 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:39:04 smithi022 ceph-mon[31985]: Adjusting osd_memory_target on smithi134 to 176.8M 2024-03-28T18:39:05.032 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:39:04 smithi022 ceph-mon[31985]: Unable to set osd_memory_target on smithi134 to 185393868: error parsing value: Value '185393868' is below minimum 939524096 2024-03-28T18:39:05.032 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:39:04 smithi022 ceph-mon[31985]: pgmap v1000: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 6 op/s 2024-03-28T18:39:05.032 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:39:04 smithi022 ceph-mon[31985]: pgmap v1001: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-28T18:39:05.032 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:39:04 smithi022 ceph-mon[31985]: 2 slow requests, 0 included below; oldest blocked for > 1331.859648 secs 2024-03-28T18:39:06.925 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:39:06 smithi134 ceph-mon[35037]: pgmap v1002: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-28T18:39:06.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:39:06 smithi079 ceph-mon[35312]: pgmap v1002: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-28T18:39:07.032 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:39:06 smithi022 ceph-mon[31985]: pgmap v1002: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-28T18:39:08.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:39:08 smithi134 ceph-mon[35037]: pgmap v1003: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-28T18:39:08.933 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:39:08 smithi079 ceph-mon[35312]: pgmap v1003: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-28T18:39:09.032 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:39:08 smithi022 ceph-mon[31985]: pgmap v1003: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-28T18:39:09.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:39:09 smithi134 ceph-mon[35037]: 2 slow requests, 0 included below; oldest blocked for > 1336.859771 secs 2024-03-28T18:39:09.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:39:09 smithi079 ceph-mon[35312]: 2 slow requests, 0 included below; oldest blocked for > 1336.859771 secs 2024-03-28T18:39:10.032 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:39:09 smithi022 ceph-mon[31985]: 2 slow requests, 0 included below; oldest blocked for > 1336.859771 secs 2024-03-28T18:39:10.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:39:10 smithi134 ceph-mon[35037]: pgmap v1004: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-28T18:39:10.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:39:10 smithi079 ceph-mon[35312]: pgmap v1004: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-28T18:39:10.934 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:39:10 smithi022 ceph-mon[31985]: pgmap v1004: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-28T18:39:12.925 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:39:12 smithi134 ceph-mon[35037]: pgmap v1005: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-28T18:39:12.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:39:12 smithi079 ceph-mon[35312]: pgmap v1005: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-28T18:39:13.032 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:39:12 smithi022 ceph-mon[31985]: pgmap v1005: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-28T18:39:14.925 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:39:14 smithi134 ceph-mon[35037]: pgmap v1006: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 7 op/s 2024-03-28T18:39:14.925 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:39:14 smithi134 ceph-mon[35037]: 2 slow requests, 0 included below; oldest blocked for > 1341.859891 secs 2024-03-28T18:39:14.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:39:14 smithi079 ceph-mon[35312]: pgmap v1006: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 7 op/s 2024-03-28T18:39:14.933 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:39:14 smithi079 ceph-mon[35312]: 2 slow requests, 0 included below; oldest blocked for > 1341.859891 secs 2024-03-28T18:39:15.032 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:39:14 smithi022 ceph-mon[31985]: pgmap v1006: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 7 op/s 2024-03-28T18:39:15.032 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:39:14 smithi022 ceph-mon[31985]: 2 slow requests, 0 included below; oldest blocked for > 1341.859891 secs 2024-03-28T18:39:16.925 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:39:16 smithi134 ceph-mon[35037]: pgmap v1007: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:39:16.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:39:16 smithi079 ceph-mon[35312]: pgmap v1007: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:39:17.032 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:39:16 smithi022 ceph-mon[31985]: pgmap v1007: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:39:18.925 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:39:18 smithi134 ceph-mon[35037]: pgmap v1008: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:39:18.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:39:18 smithi079 ceph-mon[35312]: pgmap v1008: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:39:19.032 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:39:18 smithi022 ceph-mon[31985]: pgmap v1008: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:39:20.032 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:39:19 smithi022 ceph-mon[31985]: 2 slow requests, 0 included below; oldest blocked for > 1346.859917 secs 2024-03-28T18:39:20.174 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:39:19 smithi134 ceph-mon[35037]: 2 slow requests, 0 included below; oldest blocked for > 1346.859917 secs 2024-03-28T18:39:20.183 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:39:19 smithi079 ceph-mon[35312]: 2 slow requests, 0 included below; oldest blocked for > 1346.859917 secs 2024-03-28T18:39:21.032 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:39:20 smithi022 ceph-mon[31985]: pgmap v1009: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:39:21.174 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:39:20 smithi134 ceph-mon[35037]: pgmap v1009: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:39:21.183 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:39:20 smithi079 ceph-mon[35312]: pgmap v1009: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:39:23.029 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:39:22 smithi022 ceph-mon[31985]: pgmap v1010: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:39:23.174 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:39:22 smithi134 ceph-mon[35037]: pgmap v1010: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:39:23.182 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:39:22 smithi079 ceph-mon[35312]: pgmap v1010: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:39:25.032 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:39:24 smithi022 ceph-mon[31985]: pgmap v1011: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:39:25.032 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:39:24 smithi022 ceph-mon[31985]: 2 slow requests, 0 included below; oldest blocked for > 1351.860005 secs 2024-03-28T18:39:25.174 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:39:24 smithi134 ceph-mon[35037]: pgmap v1011: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:39:25.175 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:39:24 smithi134 ceph-mon[35037]: 2 slow requests, 0 included below; oldest blocked for > 1351.860005 secs 2024-03-28T18:39:25.182 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:39:24 smithi079 ceph-mon[35312]: pgmap v1011: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:39:25.182 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:39:24 smithi079 ceph-mon[35312]: 2 slow requests, 0 included below; oldest blocked for > 1351.860005 secs 2024-03-28T18:39:27.032 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:39:26 smithi022 ceph-mon[31985]: pgmap v1012: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:39:27.174 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:39:26 smithi134 ceph-mon[35037]: pgmap v1012: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:39:27.182 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:39:26 smithi079 ceph-mon[35312]: pgmap v1012: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:39:29.032 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:39:28 smithi022 ceph-mon[31985]: pgmap v1013: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:39:29.174 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:39:28 smithi134 ceph-mon[35037]: pgmap v1013: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:39:29.182 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:39:28 smithi079 ceph-mon[35312]: pgmap v1013: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:39:30.032 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:39:29 smithi022 ceph-mon[31985]: 2 slow requests, 0 included below; oldest blocked for > 1356.860061 secs 2024-03-28T18:39:30.174 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:39:29 smithi134 ceph-mon[35037]: 2 slow requests, 0 included below; oldest blocked for > 1356.860061 secs 2024-03-28T18:39:30.182 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:39:29 smithi079 ceph-mon[35312]: 2 slow requests, 0 included below; oldest blocked for > 1356.860061 secs 2024-03-28T18:39:31.032 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:39:30 smithi022 ceph-mon[31985]: pgmap v1014: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:39:31.174 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:39:30 smithi134 ceph-mon[35037]: pgmap v1014: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:39:31.182 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:39:30 smithi079 ceph-mon[35312]: pgmap v1014: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:39:33.032 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:39:32 smithi022 ceph-mon[31985]: pgmap v1015: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:39:33.174 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:39:32 smithi134 ceph-mon[35037]: pgmap v1015: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:39:33.183 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:39:32 smithi079 ceph-mon[35312]: pgmap v1015: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:39:35.032 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:39:34 smithi022 ceph-mon[31985]: pgmap v1016: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:39:35.032 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:39:34 smithi022 ceph-mon[31985]: 2 slow requests, 0 included below; oldest blocked for > 1361.860132 secs 2024-03-28T18:39:35.174 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:39:34 smithi134 ceph-mon[35037]: pgmap v1016: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:39:35.175 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:39:34 smithi134 ceph-mon[35037]: 2 slow requests, 0 included below; oldest blocked for > 1361.860132 secs 2024-03-28T18:39:35.183 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:39:34 smithi079 ceph-mon[35312]: pgmap v1016: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:39:35.183 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:39:34 smithi079 ceph-mon[35312]: 2 slow requests, 0 included below; oldest blocked for > 1361.860132 secs 2024-03-28T18:39:37.032 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:39:36 smithi022 ceph-mon[31985]: pgmap v1017: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:39:37.174 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:39:36 smithi134 ceph-mon[35037]: pgmap v1017: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:39:37.182 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:39:36 smithi079 ceph-mon[35312]: pgmap v1017: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:39:39.032 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:39:38 smithi022 ceph-mon[31985]: pgmap v1018: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:39:39.174 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:39:38 smithi134 ceph-mon[35037]: pgmap v1018: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:39:39.182 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:39:38 smithi079 ceph-mon[35312]: pgmap v1018: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:39:40.032 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:39:39 smithi022 ceph-mon[31985]: 2 slow requests, 0 included below; oldest blocked for > 1366.860245 secs 2024-03-28T18:39:40.076 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:39:39 smithi134 ceph-mon[35037]: 2 slow requests, 0 included below; oldest blocked for > 1366.860245 secs 2024-03-28T18:39:40.182 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:39:39 smithi079 ceph-mon[35312]: 2 slow requests, 0 included below; oldest blocked for > 1366.860245 secs 2024-03-28T18:39:41.032 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:39:40 smithi022 ceph-mon[31985]: pgmap v1019: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:39:41.175 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:39:40 smithi134 ceph-mon[35037]: pgmap v1019: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:39:41.182 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:39:40 smithi079 ceph-mon[35312]: pgmap v1019: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:39:43.032 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:39:42 smithi022 ceph-mon[31985]: pgmap v1020: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:39:43.174 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:39:42 smithi134 ceph-mon[35037]: pgmap v1020: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:39:43.182 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:39:42 smithi079 ceph-mon[35312]: pgmap v1020: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:39:45.032 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:39:44 smithi022 ceph-mon[31985]: pgmap v1021: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:39:45.032 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:39:44 smithi022 ceph-mon[31985]: 2 slow requests, 0 included below; oldest blocked for > 1371.860307 secs 2024-03-28T18:39:45.175 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:39:44 smithi134 ceph-mon[35037]: pgmap v1021: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:39:45.175 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:39:44 smithi134 ceph-mon[35037]: 2 slow requests, 0 included below; oldest blocked for > 1371.860307 secs 2024-03-28T18:39:45.182 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:39:44 smithi079 ceph-mon[35312]: pgmap v1021: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:39:45.183 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:39:44 smithi079 ceph-mon[35312]: 2 slow requests, 0 included below; oldest blocked for > 1371.860307 secs 2024-03-28T18:39:47.174 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:39:46 smithi134 ceph-mon[35037]: pgmap v1022: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:39:47.182 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:39:46 smithi079 ceph-mon[35312]: pgmap v1022: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:39:47.282 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:39:46 smithi022 ceph-mon[31985]: pgmap v1022: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:39:49.175 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:39:48 smithi134 ceph-mon[35037]: pgmap v1023: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:39:49.182 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:39:48 smithi079 ceph-mon[35312]: pgmap v1023: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:39:49.282 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:39:48 smithi022 ceph-mon[31985]: pgmap v1023: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:39:50.084 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:39:49 smithi079 ceph-mon[35312]: 2 slow requests, 0 included below; oldest blocked for > 1376.860373 secs 2024-03-28T18:39:50.174 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:39:49 smithi134 ceph-mon[35037]: 2 slow requests, 0 included below; oldest blocked for > 1376.860373 secs 2024-03-28T18:39:50.282 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:39:49 smithi022 ceph-mon[31985]: 2 slow requests, 0 included below; oldest blocked for > 1376.860373 secs 2024-03-28T18:39:51.175 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:39:50 smithi134 ceph-mon[35037]: pgmap v1024: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:39:51.182 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:39:50 smithi079 ceph-mon[35312]: pgmap v1024: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:39:51.282 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:39:50 smithi022 ceph-mon[31985]: pgmap v1024: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:39:53.175 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:39:52 smithi134 ceph-mon[35037]: pgmap v1025: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:39:53.182 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:39:52 smithi079 ceph-mon[35312]: pgmap v1025: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:39:53.282 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:39:52 smithi022 ceph-mon[31985]: pgmap v1025: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:39:55.175 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:39:54 smithi134 ceph-mon[35037]: pgmap v1026: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:39:55.175 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:39:54 smithi134 ceph-mon[35037]: 2 slow requests, 0 included below; oldest blocked for > 1381.860438 secs 2024-03-28T18:39:55.182 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:39:54 smithi079 ceph-mon[35312]: pgmap v1026: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:39:55.182 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:39:54 smithi079 ceph-mon[35312]: 2 slow requests, 0 included below; oldest blocked for > 1381.860438 secs 2024-03-28T18:39:55.282 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:39:54 smithi022 ceph-mon[31985]: pgmap v1026: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:39:55.282 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:39:54 smithi022 ceph-mon[31985]: 2 slow requests, 0 included below; oldest blocked for > 1381.860438 secs 2024-03-28T18:39:57.174 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:39:56 smithi134 ceph-mon[35037]: pgmap v1027: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:39:57.182 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:39:56 smithi079 ceph-mon[35312]: pgmap v1027: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:39:57.282 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:39:56 smithi022 ceph-mon[31985]: pgmap v1027: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:39:59.175 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:39:58 smithi134 ceph-mon[35037]: pgmap v1028: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:39:59.182 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:39:58 smithi079 ceph-mon[35312]: pgmap v1028: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:39:59.282 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:39:58 smithi022 ceph-mon[31985]: pgmap v1028: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:40:00.174 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:39:59 smithi134 ceph-mon[35037]: 2 slow requests, 0 included below; oldest blocked for > 1386.860506 secs 2024-03-28T18:40:00.182 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:39:59 smithi079 ceph-mon[35312]: 2 slow requests, 0 included below; oldest blocked for > 1386.860506 secs 2024-03-28T18:40:00.282 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:39:59 smithi022 ceph-mon[31985]: 2 slow requests, 0 included below; oldest blocked for > 1386.860506 secs 2024-03-28T18:40:01.175 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:40:00 smithi134 ceph-mon[35037]: pgmap v1029: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:40:01.175 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:40:00 smithi134 ceph-mon[35037]: overall HEALTH_WARN 4 failed cephadm daemon(s); 1 MDSs report slow requests 2024-03-28T18:40:01.182 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:40:00 smithi079 ceph-mon[35312]: pgmap v1029: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:40:01.183 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:40:00 smithi079 ceph-mon[35312]: overall HEALTH_WARN 4 failed cephadm daemon(s); 1 MDSs report slow requests 2024-03-28T18:40:01.282 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:40:00 smithi022 ceph-mon[31985]: pgmap v1029: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:40:01.282 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:40:00 smithi022 ceph-mon[31985]: overall HEALTH_WARN 4 failed cephadm daemon(s); 1 MDSs report slow requests 2024-03-28T18:40:03.174 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:40:02 smithi134 ceph-mon[35037]: pgmap v1030: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:40:03.182 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:40:02 smithi079 ceph-mon[35312]: pgmap v1030: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:40:03.282 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:40:02 smithi022 ceph-mon[31985]: pgmap v1030: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:40:05.174 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:40:04 smithi134 ceph-mon[35037]: pgmap v1031: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:40:05.175 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:40:04 smithi134 ceph-mon[35037]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T18:40:05.175 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:40:04 smithi134 ceph-mon[35037]: 2 slow requests, 0 included below; oldest blocked for > 1391.860578 secs 2024-03-28T18:40:05.183 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:40:04 smithi079 ceph-mon[35312]: pgmap v1031: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:40:05.183 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:40:04 smithi079 ceph-mon[35312]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T18:40:05.183 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:40:04 smithi079 ceph-mon[35312]: 2 slow requests, 0 included below; oldest blocked for > 1391.860578 secs 2024-03-28T18:40:05.281 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:40:04 smithi022 ceph-mon[31985]: pgmap v1031: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:40:05.281 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:40:04 smithi022 ceph-mon[31985]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T18:40:05.282 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:40:04 smithi022 ceph-mon[31985]: 2 slow requests, 0 included below; oldest blocked for > 1391.860578 secs 2024-03-28T18:40:06.174 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:40:05 smithi134 ceph-mon[35037]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T18:40:06.175 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:40:05 smithi134 ceph-mon[35037]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T18:40:06.175 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:40:05 smithi134 ceph-mon[35037]: pgmap v1032: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 10 op/s 2024-03-28T18:40:06.175 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:40:05 smithi134 ceph-mon[35037]: pgmap v1033: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-03-28T18:40:06.175 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:40:05 smithi134 ceph-mon[35037]: from='mgr.15522 ' entity='mgr.x' 2024-03-28T18:40:06.182 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:40:05 smithi079 ceph-mon[35312]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T18:40:06.183 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:40:05 smithi079 ceph-mon[35312]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T18:40:06.183 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:40:05 smithi079 ceph-mon[35312]: pgmap v1032: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 10 op/s 2024-03-28T18:40:06.183 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:40:05 smithi079 ceph-mon[35312]: pgmap v1033: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-03-28T18:40:06.183 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:40:05 smithi079 ceph-mon[35312]: from='mgr.15522 ' entity='mgr.x' 2024-03-28T18:40:06.282 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:40:05 smithi022 ceph-mon[31985]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T18:40:06.283 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:40:05 smithi022 ceph-mon[31985]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T18:40:06.283 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:40:05 smithi022 ceph-mon[31985]: pgmap v1032: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 10 op/s 2024-03-28T18:40:06.283 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:40:05 smithi022 ceph-mon[31985]: pgmap v1033: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-03-28T18:40:06.283 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:40:05 smithi022 ceph-mon[31985]: from='mgr.15522 ' entity='mgr.x' 2024-03-28T18:40:08.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:40:08 smithi022 ceph-mon[31985]: pgmap v1034: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-03-28T18:40:08.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:40:08 smithi134 ceph-mon[35037]: pgmap v1034: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-03-28T18:40:08.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:40:08 smithi079 ceph-mon[35312]: pgmap v1034: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-03-28T18:40:10.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:40:10 smithi022 ceph-mon[31985]: pgmap v1035: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-03-28T18:40:10.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:40:10 smithi022 ceph-mon[31985]: 2 slow requests, 0 included below; oldest blocked for > 1396.860663 secs 2024-03-28T18:40:10.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:40:10 smithi134 ceph-mon[35037]: pgmap v1035: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-03-28T18:40:10.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:40:10 smithi134 ceph-mon[35037]: 2 slow requests, 0 included below; oldest blocked for > 1396.860663 secs 2024-03-28T18:40:10.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:40:10 smithi079 ceph-mon[35312]: pgmap v1035: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-03-28T18:40:10.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:40:10 smithi079 ceph-mon[35312]: 2 slow requests, 0 included below; oldest blocked for > 1396.860663 secs 2024-03-28T18:40:12.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:40:12 smithi134 ceph-mon[35037]: pgmap v1036: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-03-28T18:40:12.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:40:12 smithi079 ceph-mon[35312]: pgmap v1036: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-03-28T18:40:12.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:40:12 smithi022 ceph-mon[31985]: pgmap v1036: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-03-28T18:40:13.674 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:40:13 smithi134 ceph-mon[35037]: pgmap v1037: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-03-28T18:40:13.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:40:13 smithi079 ceph-mon[35312]: pgmap v1037: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-03-28T18:40:13.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:40:13 smithi022 ceph-mon[31985]: pgmap v1037: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-03-28T18:40:14.925 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:40:14 smithi134 ceph-mon[35037]: 2 slow requests, 0 included below; oldest blocked for > 1401.860773 secs 2024-03-28T18:40:14.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:40:14 smithi079 ceph-mon[35312]: 2 slow requests, 0 included below; oldest blocked for > 1401.860773 secs 2024-03-28T18:40:15.032 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:40:14 smithi022 ceph-mon[31985]: 2 slow requests, 0 included below; oldest blocked for > 1401.860773 secs 2024-03-28T18:40:15.925 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:40:15 smithi134 ceph-mon[35037]: pgmap v1038: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 7 op/s 2024-03-28T18:40:15.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:40:15 smithi079 ceph-mon[35312]: pgmap v1038: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 7 op/s 2024-03-28T18:40:16.032 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:40:15 smithi022 ceph-mon[31985]: pgmap v1038: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 7 op/s 2024-03-28T18:40:18.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:40:18 smithi022 ceph-mon[31985]: pgmap v1039: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:40:18.674 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:40:18 smithi134 ceph-mon[35037]: pgmap v1039: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:40:18.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:40:18 smithi079 ceph-mon[35312]: pgmap v1039: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:40:20.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:40:20 smithi134 ceph-mon[35037]: pgmap v1040: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:40:20.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:40:20 smithi134 ceph-mon[35037]: 2 slow requests, 0 included below; oldest blocked for > 1406.860917 secs 2024-03-28T18:40:20.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:40:20 smithi079 ceph-mon[35312]: pgmap v1040: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:40:20.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:40:20 smithi079 ceph-mon[35312]: 2 slow requests, 0 included below; oldest blocked for > 1406.860917 secs 2024-03-28T18:40:20.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:40:20 smithi022 ceph-mon[31985]: pgmap v1040: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:40:20.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:40:20 smithi022 ceph-mon[31985]: 2 slow requests, 0 included below; oldest blocked for > 1406.860917 secs 2024-03-28T18:40:22.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:40:22 smithi134 ceph-mon[35037]: pgmap v1041: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:40:22.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:40:22 smithi079 ceph-mon[35312]: pgmap v1041: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:40:22.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:40:22 smithi022 ceph-mon[31985]: pgmap v1041: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:40:23.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:40:23 smithi134 ceph-mon[35037]: pgmap v1042: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:40:23.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:40:23 smithi079 ceph-mon[35312]: pgmap v1042: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:40:23.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:40:23 smithi022 ceph-mon[31985]: pgmap v1042: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:40:24.926 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:40:24 smithi134 ceph-mon[35037]: 2 slow requests, 0 included below; oldest blocked for > 1411.860940 secs 2024-03-28T18:40:24.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:40:24 smithi079 ceph-mon[35312]: 2 slow requests, 0 included below; oldest blocked for > 1411.860940 secs 2024-03-28T18:40:25.032 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:40:24 smithi022 ceph-mon[31985]: 2 slow requests, 0 included below; oldest blocked for > 1411.860940 secs 2024-03-28T18:40:25.925 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:40:25 smithi134 ceph-mon[35037]: pgmap v1043: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:40:25.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:40:25 smithi079 ceph-mon[35312]: pgmap v1043: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:40:26.032 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:40:25 smithi022 ceph-mon[31985]: pgmap v1043: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:40:28.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:40:28 smithi134 ceph-mon[35037]: pgmap v1044: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:40:28.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:40:28 smithi079 ceph-mon[35312]: pgmap v1044: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:40:28.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:40:28 smithi022 ceph-mon[31985]: pgmap v1044: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:40:30.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:40:30 smithi134 ceph-mon[35037]: pgmap v1045: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:40:30.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:40:30 smithi134 ceph-mon[35037]: 2 slow requests, 0 included below; oldest blocked for > 1416.861013 secs 2024-03-28T18:40:30.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:40:30 smithi079 ceph-mon[35312]: pgmap v1045: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:40:30.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:40:30 smithi079 ceph-mon[35312]: 2 slow requests, 0 included below; oldest blocked for > 1416.861013 secs 2024-03-28T18:40:30.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:40:30 smithi022 ceph-mon[31985]: pgmap v1045: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:40:30.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:40:30 smithi022 ceph-mon[31985]: 2 slow requests, 0 included below; oldest blocked for > 1416.861013 secs 2024-03-28T18:40:31.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:40:31 smithi134 ceph-mon[35037]: pgmap v1046: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:40:31.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:40:31 smithi079 ceph-mon[35312]: pgmap v1046: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:40:31.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:40:31 smithi022 ceph-mon[31985]: pgmap v1046: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:40:34.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:40:34 smithi134 ceph-mon[35037]: pgmap v1047: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:40:34.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:40:34 smithi079 ceph-mon[35312]: pgmap v1047: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:40:34.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:40:34 smithi022 ceph-mon[31985]: pgmap v1047: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:40:35.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:40:35 smithi134 ceph-mon[35037]: 2 slow requests, 0 included below; oldest blocked for > 1421.861093 secs 2024-03-28T18:40:35.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:40:35 smithi079 ceph-mon[35312]: 2 slow requests, 0 included below; oldest blocked for > 1421.861093 secs 2024-03-28T18:40:35.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:40:35 smithi022 ceph-mon[31985]: 2 slow requests, 0 included below; oldest blocked for > 1421.861093 secs 2024-03-28T18:40:36.674 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:40:36 smithi134 ceph-mon[35037]: pgmap v1048: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:40:36.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:40:36 smithi079 ceph-mon[35312]: pgmap v1048: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:40:36.781 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:40:36 smithi022 ceph-mon[31985]: pgmap v1048: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:40:37.674 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:40:37 smithi134 ceph-mon[35037]: pgmap v1049: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:40:37.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:40:37 smithi079 ceph-mon[35312]: pgmap v1049: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:40:37.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:40:37 smithi022 ceph-mon[31985]: pgmap v1049: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:40:40.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:40:40 smithi134 ceph-mon[35037]: pgmap v1050: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:40:40.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:40:40 smithi134 ceph-mon[35037]: 2 slow requests, 0 included below; oldest blocked for > 1426.861214 secs 2024-03-28T18:40:40.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:40:40 smithi079 ceph-mon[35312]: pgmap v1050: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:40:40.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:40:40 smithi079 ceph-mon[35312]: 2 slow requests, 0 included below; oldest blocked for > 1426.861214 secs 2024-03-28T18:40:40.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:40:40 smithi022 ceph-mon[31985]: pgmap v1050: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:40:40.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:40:40 smithi022 ceph-mon[31985]: 2 slow requests, 0 included below; oldest blocked for > 1426.861214 secs 2024-03-28T18:40:42.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:40:42 smithi134 ceph-mon[35037]: pgmap v1051: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:40:42.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:40:42 smithi079 ceph-mon[35312]: pgmap v1051: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:40:42.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:40:42 smithi022 ceph-mon[31985]: pgmap v1051: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:40:43.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:40:43 smithi134 ceph-mon[35037]: pgmap v1052: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:40:43.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:40:43 smithi079 ceph-mon[35312]: pgmap v1052: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:40:43.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:40:43 smithi022 ceph-mon[31985]: pgmap v1052: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:40:44.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:40:44 smithi134 ceph-mon[35037]: 2 slow requests, 0 included below; oldest blocked for > 1431.861332 secs 2024-03-28T18:40:44.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:40:44 smithi079 ceph-mon[35312]: 2 slow requests, 0 included below; oldest blocked for > 1431.861332 secs 2024-03-28T18:40:45.032 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:40:44 smithi022 ceph-mon[31985]: 2 slow requests, 0 included below; oldest blocked for > 1431.861332 secs 2024-03-28T18:40:45.925 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:40:45 smithi134 ceph-mon[35037]: pgmap v1053: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:40:45.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:40:45 smithi079 ceph-mon[35312]: pgmap v1053: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:40:46.032 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:40:45 smithi022 ceph-mon[31985]: pgmap v1053: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:40:48.674 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:40:48 smithi134 ceph-mon[35037]: pgmap v1054: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:40:48.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:40:48 smithi079 ceph-mon[35312]: pgmap v1054: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:40:48.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:40:48 smithi022 ceph-mon[31985]: pgmap v1054: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:40:50.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:40:50 smithi134 ceph-mon[35037]: pgmap v1055: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:40:50.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:40:50 smithi134 ceph-mon[35037]: 2 slow requests, 0 included below; oldest blocked for > 1436.861452 secs 2024-03-28T18:40:50.683 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:40:50 smithi079 ceph-mon[35312]: pgmap v1055: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:40:50.683 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:40:50 smithi079 ceph-mon[35312]: 2 slow requests, 0 included below; oldest blocked for > 1436.861452 secs 2024-03-28T18:40:50.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:40:50 smithi022 ceph-mon[31985]: pgmap v1055: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:40:50.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:40:50 smithi022 ceph-mon[31985]: 2 slow requests, 0 included below; oldest blocked for > 1436.861452 secs 2024-03-28T18:40:51.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:40:51 smithi134 ceph-mon[35037]: pgmap v1056: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:40:51.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:40:51 smithi079 ceph-mon[35312]: pgmap v1056: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:40:51.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:40:51 smithi022 ceph-mon[31985]: pgmap v1056: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:40:54.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:40:54 smithi134 ceph-mon[35037]: pgmap v1057: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:40:54.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:40:54 smithi079 ceph-mon[35312]: pgmap v1057: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:40:54.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:40:54 smithi022 ceph-mon[31985]: pgmap v1057: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:40:55.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:40:55 smithi134 ceph-mon[35037]: 2 slow requests, 1 included below; oldest blocked for > 1441.861575 secs 2024-03-28T18:40:55.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:40:55 smithi134 ceph-mon[35037]: slow request 1441.861574 seconds old, received at 2024-03-28T18:16:52.457506+0000: peer_request:client.35222:10732 currently acquired locks 2024-03-28T18:40:55.683 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:40:55 smithi079 ceph-mon[35312]: 2 slow requests, 1 included below; oldest blocked for > 1441.861575 secs 2024-03-28T18:40:55.683 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:40:55 smithi079 ceph-mon[35312]: slow request 1441.861574 seconds old, received at 2024-03-28T18:16:52.457506+0000: peer_request:client.35222:10732 currently acquired locks 2024-03-28T18:40:55.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:40:55 smithi022 ceph-mon[31985]: 2 slow requests, 1 included below; oldest blocked for > 1441.861575 secs 2024-03-28T18:40:55.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:40:55 smithi022 ceph-mon[31985]: slow request 1441.861574 seconds old, received at 2024-03-28T18:16:52.457506+0000: peer_request:client.35222:10732 currently acquired locks 2024-03-28T18:40:56.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:40:56 smithi134 ceph-mon[35037]: pgmap v1058: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:40:56.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:40:56 smithi079 ceph-mon[35312]: pgmap v1058: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:40:56.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:40:56 smithi022 ceph-mon[31985]: pgmap v1058: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:40:57.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:40:57 smithi134 ceph-mon[35037]: pgmap v1059: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:40:57.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:40:57 smithi079 ceph-mon[35312]: pgmap v1059: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:40:57.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:40:57 smithi022 ceph-mon[31985]: pgmap v1059: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:41:00.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:41:00 smithi134 ceph-mon[35037]: pgmap v1060: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:41:00.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:41:00 smithi134 ceph-mon[35037]: 2 slow requests, 0 included below; oldest blocked for > 1446.861661 secs 2024-03-28T18:41:00.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:41:00 smithi079 ceph-mon[35312]: pgmap v1060: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:41:00.683 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:41:00 smithi079 ceph-mon[35312]: 2 slow requests, 0 included below; oldest blocked for > 1446.861661 secs 2024-03-28T18:41:00.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:41:00 smithi022 ceph-mon[31985]: pgmap v1060: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:41:00.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:41:00 smithi022 ceph-mon[31985]: 2 slow requests, 0 included below; oldest blocked for > 1446.861661 secs 2024-03-28T18:41:01.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:41:01 smithi134 ceph-mon[35037]: pgmap v1061: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:41:01.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:41:01 smithi079 ceph-mon[35312]: pgmap v1061: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:41:01.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:41:01 smithi022 ceph-mon[31985]: pgmap v1061: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:41:04.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:41:04 smithi134 ceph-mon[35037]: pgmap v1062: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:41:04.683 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:41:04 smithi079 ceph-mon[35312]: pgmap v1062: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:41:04.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:41:04 smithi022 ceph-mon[31985]: pgmap v1062: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:41:05.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:41:05 smithi134 ceph-mon[35037]: 2 slow requests, 1 included below; oldest blocked for > 1451.861694 secs 2024-03-28T18:41:05.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:41:05 smithi134 ceph-mon[35037]: slow request 1444.164070 seconds old, received at 2024-03-28T18:17:00.155130+0000: client_request(client.35612:96 mksnap #0x1//scheduled-2024-03-28-18_17_00_UTC 2024-03-28T18:17:00.154942+0000 caller_uid=0, caller_gid=0{}) currently failed to xlock, waiting 2024-03-28T18:41:05.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:41:05 smithi134 ceph-mon[35037]: pgmap v1063: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:41:05.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:41:05 smithi079 ceph-mon[35312]: 2 slow requests, 1 included below; oldest blocked for > 1451.861694 secs 2024-03-28T18:41:05.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:41:05 smithi079 ceph-mon[35312]: slow request 1444.164070 seconds old, received at 2024-03-28T18:17:00.155130+0000: client_request(client.35612:96 mksnap #0x1//scheduled-2024-03-28-18_17_00_UTC 2024-03-28T18:17:00.154942+0000 caller_uid=0, caller_gid=0{}) currently failed to xlock, waiting 2024-03-28T18:41:05.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:41:05 smithi079 ceph-mon[35312]: pgmap v1063: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:41:05.781 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:41:05 smithi022 ceph-mon[31985]: 2 slow requests, 1 included below; oldest blocked for > 1451.861694 secs 2024-03-28T18:41:05.781 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:41:05 smithi022 ceph-mon[31985]: slow request 1444.164070 seconds old, received at 2024-03-28T18:17:00.155130+0000: client_request(client.35612:96 mksnap #0x1//scheduled-2024-03-28-18_17_00_UTC 2024-03-28T18:17:00.154942+0000 caller_uid=0, caller_gid=0{}) currently failed to xlock, waiting 2024-03-28T18:41:05.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:41:05 smithi022 ceph-mon[31985]: pgmap v1063: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:41:06.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:41:06 smithi134 ceph-mon[35037]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T18:41:06.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:41:06 smithi079 ceph-mon[35312]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T18:41:06.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:41:06 smithi022 ceph-mon[31985]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T18:41:07.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:41:07 smithi134 ceph-mon[35037]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T18:41:07.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:41:07 smithi134 ceph-mon[35037]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T18:41:07.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:41:07 smithi134 ceph-mon[35037]: pgmap v1064: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 6 op/s 2024-03-28T18:41:07.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:41:07 smithi134 ceph-mon[35037]: from='mgr.15522 ' entity='mgr.x' 2024-03-28T18:41:07.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:41:07 smithi079 ceph-mon[35312]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T18:41:07.683 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:41:07 smithi079 ceph-mon[35312]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T18:41:07.683 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:41:07 smithi079 ceph-mon[35312]: pgmap v1064: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 6 op/s 2024-03-28T18:41:07.683 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:41:07 smithi079 ceph-mon[35312]: from='mgr.15522 ' entity='mgr.x' 2024-03-28T18:41:07.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:41:07 smithi022 ceph-mon[31985]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T18:41:07.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:41:07 smithi022 ceph-mon[31985]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T18:41:07.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:41:07 smithi022 ceph-mon[31985]: pgmap v1064: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 6 op/s 2024-03-28T18:41:07.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:41:07 smithi022 ceph-mon[31985]: from='mgr.15522 ' entity='mgr.x' 2024-03-28T18:41:10.378 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:41:10 smithi134 ceph-mon[35037]: pgmap v1065: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 6 op/s 2024-03-28T18:41:10.378 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:41:10 smithi134 ceph-mon[35037]: 2 slow requests, 0 included below; oldest blocked for > 1456.861771 secs 2024-03-28T18:41:10.433 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:41:10 smithi079 ceph-mon[35312]: pgmap v1065: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 6 op/s 2024-03-28T18:41:10.433 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:41:10 smithi079 ceph-mon[35312]: 2 slow requests, 0 included below; oldest blocked for > 1456.861771 secs 2024-03-28T18:41:10.486 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:41:10 smithi022 ceph-mon[31985]: pgmap v1065: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 6 op/s 2024-03-28T18:41:10.486 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:41:10 smithi022 ceph-mon[31985]: 2 slow requests, 0 included below; oldest blocked for > 1456.861771 secs 2024-03-28T18:41:12.424 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:41:12 smithi134 ceph-mon[35037]: pgmap v1066: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 6 op/s 2024-03-28T18:41:12.432 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:41:12 smithi079 ceph-mon[35312]: pgmap v1066: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 6 op/s 2024-03-28T18:41:12.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:41:12 smithi022 ceph-mon[31985]: pgmap v1066: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 6 op/s 2024-03-28T18:41:14.425 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:41:14 smithi134 ceph-mon[35037]: pgmap v1067: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 6 op/s 2024-03-28T18:41:14.432 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:41:14 smithi079 ceph-mon[35312]: pgmap v1067: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 6 op/s 2024-03-28T18:41:14.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:41:14 smithi022 ceph-mon[31985]: pgmap v1067: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 6 op/s 2024-03-28T18:41:15.425 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:41:15 smithi134 ceph-mon[35037]: 2 slow requests, 0 included below; oldest blocked for > 1461.861848 secs 2024-03-28T18:41:15.433 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:41:15 smithi079 ceph-mon[35312]: 2 slow requests, 0 included below; oldest blocked for > 1461.861848 secs 2024-03-28T18:41:15.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:41:15 smithi022 ceph-mon[31985]: 2 slow requests, 0 included below; oldest blocked for > 1461.861848 secs 2024-03-28T18:41:16.425 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:41:16 smithi134 ceph-mon[35037]: pgmap v1068: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 10 op/s 2024-03-28T18:41:16.432 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:41:16 smithi079 ceph-mon[35312]: pgmap v1068: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 10 op/s 2024-03-28T18:41:16.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:41:16 smithi022 ceph-mon[31985]: pgmap v1068: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 10 op/s 2024-03-28T18:41:18.425 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:41:18 smithi134 ceph-mon[35037]: pgmap v1069: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:41:18.432 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:41:18 smithi079 ceph-mon[35312]: pgmap v1069: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:41:18.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:41:18 smithi022 ceph-mon[31985]: pgmap v1069: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:41:20.425 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:41:20 smithi134 ceph-mon[35037]: pgmap v1070: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T18:41:20.425 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:41:20 smithi134 ceph-mon[35037]: 2 slow requests, 0 included below; oldest blocked for > 1466.861960 secs 2024-03-28T18:41:20.432 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:41:20 smithi079 ceph-mon[35312]: pgmap v1070: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T18:41:20.433 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:41:20 smithi079 ceph-mon[35312]: 2 slow requests, 0 included below; oldest blocked for > 1466.861960 secs 2024-03-28T18:41:20.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:41:20 smithi022 ceph-mon[31985]: pgmap v1070: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T18:41:20.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:41:20 smithi022 ceph-mon[31985]: 2 slow requests, 0 included below; oldest blocked for > 1466.861960 secs 2024-03-28T18:41:22.425 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:41:22 smithi134 ceph-mon[35037]: pgmap v1071: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 9 op/s 2024-03-28T18:41:22.432 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:41:22 smithi079 ceph-mon[35312]: pgmap v1071: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 9 op/s 2024-03-28T18:41:22.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:41:22 smithi022 ceph-mon[31985]: pgmap v1071: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 9 op/s 2024-03-28T18:41:24.425 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:41:24 smithi134 ceph-mon[35037]: pgmap v1072: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T18:41:24.432 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:41:24 smithi079 ceph-mon[35312]: pgmap v1072: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T18:41:24.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:41:24 smithi022 ceph-mon[31985]: pgmap v1072: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T18:41:25.424 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:41:25 smithi134 ceph-mon[35037]: 2 slow requests, 0 included below; oldest blocked for > 1471.862026 secs 2024-03-28T18:41:25.432 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:41:25 smithi079 ceph-mon[35312]: 2 slow requests, 0 included below; oldest blocked for > 1471.862026 secs 2024-03-28T18:41:25.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:41:25 smithi022 ceph-mon[31985]: 2 slow requests, 0 included below; oldest blocked for > 1471.862026 secs 2024-03-28T18:41:26.425 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:41:26 smithi134 ceph-mon[35037]: pgmap v1073: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 9 op/s 2024-03-28T18:41:26.432 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:41:26 smithi079 ceph-mon[35312]: pgmap v1073: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 9 op/s 2024-03-28T18:41:26.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:41:26 smithi022 ceph-mon[31985]: pgmap v1073: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 9 op/s 2024-03-28T18:41:28.425 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:41:28 smithi134 ceph-mon[35037]: pgmap v1074: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:41:28.432 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:41:28 smithi079 ceph-mon[35312]: pgmap v1074: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:41:28.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:41:28 smithi022 ceph-mon[31985]: pgmap v1074: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:41:30.425 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:41:30 smithi134 ceph-mon[35037]: pgmap v1075: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:41:30.425 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:41:30 smithi134 ceph-mon[35037]: 2 slow requests, 0 included below; oldest blocked for > 1476.862095 secs 2024-03-28T18:41:30.434 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:41:30 smithi079 ceph-mon[35312]: pgmap v1075: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:41:30.434 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:41:30 smithi079 ceph-mon[35312]: 2 slow requests, 0 included below; oldest blocked for > 1476.862095 secs 2024-03-28T18:41:30.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:41:30 smithi022 ceph-mon[31985]: pgmap v1075: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:41:30.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:41:30 smithi022 ceph-mon[31985]: 2 slow requests, 0 included below; oldest blocked for > 1476.862095 secs 2024-03-28T18:41:32.433 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:41:32 smithi079 ceph-mon[35312]: pgmap v1076: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:41:32.533 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:41:32 smithi022 ceph-mon[31985]: pgmap v1076: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:41:32.678 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:41:32 smithi134 ceph-mon[35037]: pgmap v1076: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:41:34.433 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:41:34 smithi079 ceph-mon[35312]: pgmap v1077: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:41:34.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:41:34 smithi022 ceph-mon[31985]: pgmap v1077: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:41:34.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:41:34 smithi134 ceph-mon[35037]: pgmap v1077: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:41:35.534 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:41:35 smithi022 ceph-mon[31985]: 2 slow requests, 0 included below; oldest blocked for > 1481.862158 secs 2024-03-28T18:41:35.674 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:41:35 smithi134 ceph-mon[35037]: 2 slow requests, 0 included below; oldest blocked for > 1481.862158 secs 2024-03-28T18:41:35.683 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:41:35 smithi079 ceph-mon[35312]: 2 slow requests, 0 included below; oldest blocked for > 1481.862158 secs 2024-03-28T18:41:36.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:41:36 smithi022 ceph-mon[31985]: pgmap v1078: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:41:36.674 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:41:36 smithi134 ceph-mon[35037]: pgmap v1078: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:41:36.683 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:41:36 smithi079 ceph-mon[35312]: pgmap v1078: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:41:38.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:41:38 smithi022 ceph-mon[31985]: pgmap v1079: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:41:38.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:41:38 smithi134 ceph-mon[35037]: pgmap v1079: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:41:38.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:41:38 smithi079 ceph-mon[35312]: pgmap v1079: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:41:40.478 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:41:40 smithi134 ceph-mon[35037]: pgmap v1080: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:41:40.478 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:41:40 smithi134 ceph-mon[35037]: 2 slow requests, 0 included below; oldest blocked for > 1486.862236 secs 2024-03-28T18:41:40.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:41:40 smithi022 ceph-mon[31985]: pgmap v1080: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:41:40.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:41:40 smithi022 ceph-mon[31985]: 2 slow requests, 0 included below; oldest blocked for > 1486.862236 secs 2024-03-28T18:41:40.683 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:41:40 smithi079 ceph-mon[35312]: pgmap v1080: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:41:40.683 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:41:40 smithi079 ceph-mon[35312]: 2 slow requests, 0 included below; oldest blocked for > 1486.862236 secs 2024-03-28T18:41:42.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:41:42 smithi022 ceph-mon[31985]: pgmap v1081: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:41:42.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:41:42 smithi134 ceph-mon[35037]: pgmap v1081: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:41:42.683 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:41:42 smithi079 ceph-mon[35312]: pgmap v1081: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:41:44.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:41:44 smithi022 ceph-mon[31985]: pgmap v1082: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:41:44.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:41:44 smithi134 ceph-mon[35037]: pgmap v1082: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:41:44.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:41:44 smithi079 ceph-mon[35312]: pgmap v1082: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:41:45.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:41:45 smithi022 ceph-mon[31985]: 2 slow requests, 0 included below; oldest blocked for > 1491.862318 secs 2024-03-28T18:41:45.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:41:45 smithi134 ceph-mon[35037]: 2 slow requests, 0 included below; oldest blocked for > 1491.862318 secs 2024-03-28T18:41:45.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:41:45 smithi079 ceph-mon[35312]: 2 slow requests, 0 included below; oldest blocked for > 1491.862318 secs 2024-03-28T18:41:46.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:41:46 smithi022 ceph-mon[31985]: pgmap v1083: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:41:46.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:41:46 smithi134 ceph-mon[35037]: pgmap v1083: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:41:46.683 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:41:46 smithi079 ceph-mon[35312]: pgmap v1083: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:41:48.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:41:48 smithi022 ceph-mon[31985]: pgmap v1084: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:41:48.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:41:48 smithi134 ceph-mon[35037]: pgmap v1084: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:41:48.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:41:48 smithi079 ceph-mon[35312]: pgmap v1084: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:41:50.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:41:50 smithi022 ceph-mon[31985]: pgmap v1085: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:41:50.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:41:50 smithi022 ceph-mon[31985]: 2 slow requests, 0 included below; oldest blocked for > 1496.862403 secs 2024-03-28T18:41:50.674 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:41:50 smithi134 ceph-mon[35037]: pgmap v1085: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:41:50.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:41:50 smithi134 ceph-mon[35037]: 2 slow requests, 0 included below; oldest blocked for > 1496.862403 secs 2024-03-28T18:41:50.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:41:50 smithi079 ceph-mon[35312]: pgmap v1085: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:41:50.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:41:50 smithi079 ceph-mon[35312]: 2 slow requests, 0 included below; oldest blocked for > 1496.862403 secs 2024-03-28T18:41:52.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:41:52 smithi022 ceph-mon[31985]: pgmap v1086: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:41:52.678 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:41:52 smithi134 ceph-mon[35037]: pgmap v1086: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:41:52.683 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:41:52 smithi079 ceph-mon[35312]: pgmap v1086: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:41:54.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:41:54 smithi022 ceph-mon[31985]: pgmap v1087: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:41:54.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:41:54 smithi134 ceph-mon[35037]: pgmap v1087: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:41:54.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:41:54 smithi079 ceph-mon[35312]: pgmap v1087: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:41:55.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:41:55 smithi022 ceph-mon[31985]: 2 slow requests, 0 included below; oldest blocked for > 1501.862499 secs 2024-03-28T18:41:55.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:41:55 smithi134 ceph-mon[35037]: 2 slow requests, 0 included below; oldest blocked for > 1501.862499 secs 2024-03-28T18:41:55.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:41:55 smithi079 ceph-mon[35312]: 2 slow requests, 0 included below; oldest blocked for > 1501.862499 secs 2024-03-28T18:41:56.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:41:56 smithi022 ceph-mon[31985]: pgmap v1088: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:41:56.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:41:56 smithi134 ceph-mon[35037]: pgmap v1088: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:41:56.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:41:56 smithi079 ceph-mon[35312]: pgmap v1088: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:41:58.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:41:58 smithi134 ceph-mon[35037]: pgmap v1089: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:41:58.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:41:58 smithi079 ceph-mon[35312]: pgmap v1089: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:41:58.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:41:58 smithi022 ceph-mon[31985]: pgmap v1089: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:42:00.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:42:00 smithi134 ceph-mon[35037]: pgmap v1090: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:42:00.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:42:00 smithi134 ceph-mon[35037]: 2 slow requests, 0 included below; oldest blocked for > 1506.862564 secs 2024-03-28T18:42:00.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:42:00 smithi079 ceph-mon[35312]: pgmap v1090: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:42:00.683 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:42:00 smithi079 ceph-mon[35312]: 2 slow requests, 0 included below; oldest blocked for > 1506.862564 secs 2024-03-28T18:42:00.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:42:00 smithi022 ceph-mon[31985]: pgmap v1090: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:42:00.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:42:00 smithi022 ceph-mon[31985]: 2 slow requests, 0 included below; oldest blocked for > 1506.862564 secs 2024-03-28T18:42:02.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:42:02 smithi134 ceph-mon[35037]: pgmap v1091: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 9 op/s 2024-03-28T18:42:02.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:42:02 smithi079 ceph-mon[35312]: pgmap v1091: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 9 op/s 2024-03-28T18:42:02.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:42:02 smithi022 ceph-mon[31985]: pgmap v1091: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 9 op/s 2024-03-28T18:42:04.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:42:04 smithi134 ceph-mon[35037]: pgmap v1092: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T18:42:04.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:42:04 smithi079 ceph-mon[35312]: pgmap v1092: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T18:42:04.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:42:04 smithi022 ceph-mon[31985]: pgmap v1092: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T18:42:05.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:42:05 smithi134 ceph-mon[35037]: 2 slow requests, 0 included below; oldest blocked for > 1511.862632 secs 2024-03-28T18:42:05.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:42:05 smithi079 ceph-mon[35312]: 2 slow requests, 0 included below; oldest blocked for > 1511.862632 secs 2024-03-28T18:42:05.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:42:05 smithi022 ceph-mon[31985]: 2 slow requests, 0 included below; oldest blocked for > 1511.862632 secs 2024-03-28T18:42:06.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:42:06 smithi134 ceph-mon[35037]: pgmap v1093: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 9 op/s 2024-03-28T18:42:06.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:42:06 smithi079 ceph-mon[35312]: pgmap v1093: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 9 op/s 2024-03-28T18:42:06.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:42:06 smithi022 ceph-mon[31985]: pgmap v1093: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 9 op/s 2024-03-28T18:42:07.674 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:42:07 smithi134 ceph-mon[35037]: pgmap v1094: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T18:42:07.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:42:07 smithi079 ceph-mon[35312]: pgmap v1094: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T18:42:07.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:42:07 smithi022 ceph-mon[31985]: pgmap v1094: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T18:42:08.674 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:42:08 smithi134 ceph-mon[35037]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T18:42:08.683 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:42:08 smithi079 ceph-mon[35312]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T18:42:08.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:42:08 smithi022 ceph-mon[31985]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T18:42:09.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:42:09 smithi134 ceph-mon[35037]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T18:42:09.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:42:09 smithi134 ceph-mon[35037]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T18:42:09.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:42:09 smithi134 ceph-mon[35037]: pgmap v1095: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:42:09.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:42:09 smithi134 ceph-mon[35037]: from='mgr.15522 ' entity='mgr.x' 2024-03-28T18:42:09.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:42:09 smithi079 ceph-mon[35312]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T18:42:09.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:42:09 smithi079 ceph-mon[35312]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T18:42:09.683 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:42:09 smithi079 ceph-mon[35312]: pgmap v1095: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:42:09.683 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:42:09 smithi079 ceph-mon[35312]: from='mgr.15522 ' entity='mgr.x' 2024-03-28T18:42:09.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:42:09 smithi022 ceph-mon[31985]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T18:42:09.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:42:09 smithi022 ceph-mon[31985]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T18:42:09.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:42:09 smithi022 ceph-mon[31985]: pgmap v1095: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:42:09.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:42:09 smithi022 ceph-mon[31985]: from='mgr.15522 ' entity='mgr.x' 2024-03-28T18:42:10.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:42:10 smithi134 ceph-mon[35037]: 2 slow requests, 0 included below; oldest blocked for > 1516.862752 secs 2024-03-28T18:42:10.683 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:42:10 smithi079 ceph-mon[35312]: 2 slow requests, 0 included below; oldest blocked for > 1516.862752 secs 2024-03-28T18:42:10.683 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:42:10 smithi022 ceph-mon[31985]: 2 slow requests, 0 included below; oldest blocked for > 1516.862752 secs 2024-03-28T18:42:11.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:42:11 smithi134 ceph-mon[35037]: pgmap v1096: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 10 op/s 2024-03-28T18:42:11.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:42:11 smithi079 ceph-mon[35312]: pgmap v1096: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 10 op/s 2024-03-28T18:42:11.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:42:11 smithi022 ceph-mon[31985]: pgmap v1096: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 10 op/s 2024-03-28T18:42:14.175 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:42:13 smithi134 ceph-mon[35037]: pgmap v1097: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 6 op/s 2024-03-28T18:42:14.182 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:42:13 smithi079 ceph-mon[35312]: pgmap v1097: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 6 op/s 2024-03-28T18:42:14.282 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:42:13 smithi022 ceph-mon[31985]: pgmap v1097: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 6 op/s 2024-03-28T18:42:15.174 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:42:14 smithi134 ceph-mon[35037]: 2 slow requests, 0 included below; oldest blocked for > 1521.862874 secs 2024-03-28T18:42:15.183 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:42:14 smithi079 ceph-mon[35312]: 2 slow requests, 0 included below; oldest blocked for > 1521.862874 secs 2024-03-28T18:42:15.282 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:42:14 smithi022 ceph-mon[31985]: 2 slow requests, 0 included below; oldest blocked for > 1521.862874 secs 2024-03-28T18:42:16.175 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:42:15 smithi134 ceph-mon[35037]: pgmap v1098: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 10 op/s 2024-03-28T18:42:16.182 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:42:15 smithi079 ceph-mon[35312]: pgmap v1098: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 10 op/s 2024-03-28T18:42:16.282 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:42:15 smithi022 ceph-mon[31985]: pgmap v1098: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 10 op/s 2024-03-28T18:42:18.174 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:42:17 smithi134 ceph-mon[35037]: pgmap v1099: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 6 op/s 2024-03-28T18:42:18.182 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:42:17 smithi079 ceph-mon[35312]: pgmap v1099: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 6 op/s 2024-03-28T18:42:18.281 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:42:17 smithi022 ceph-mon[31985]: pgmap v1099: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 6 op/s 2024-03-28T18:42:20.175 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:42:19 smithi134 ceph-mon[35037]: pgmap v1100: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 6 op/s 2024-03-28T18:42:20.175 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:42:19 smithi134 ceph-mon[35037]: 2 slow requests, 0 included below; oldest blocked for > 1526.862917 secs 2024-03-28T18:42:20.182 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:42:19 smithi079 ceph-mon[35312]: pgmap v1100: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 6 op/s 2024-03-28T18:42:20.182 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:42:19 smithi079 ceph-mon[35312]: 2 slow requests, 0 included below; oldest blocked for > 1526.862917 secs 2024-03-28T18:42:20.282 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:42:19 smithi022 ceph-mon[31985]: pgmap v1100: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 6 op/s 2024-03-28T18:42:20.282 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:42:19 smithi022 ceph-mon[31985]: 2 slow requests, 0 included below; oldest blocked for > 1526.862917 secs 2024-03-28T18:42:22.175 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:42:21 smithi134 ceph-mon[35037]: pgmap v1101: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 9 op/s 2024-03-28T18:42:22.182 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:42:21 smithi079 ceph-mon[35312]: pgmap v1101: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 9 op/s 2024-03-28T18:42:22.282 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:42:21 smithi022 ceph-mon[31985]: pgmap v1101: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 9 op/s 2024-03-28T18:42:24.175 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:42:23 smithi134 ceph-mon[35037]: pgmap v1102: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T18:42:24.182 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:42:23 smithi079 ceph-mon[35312]: pgmap v1102: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T18:42:24.282 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:42:23 smithi022 ceph-mon[31985]: pgmap v1102: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T18:42:25.183 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:42:24 smithi079 ceph-mon[35312]: 2 slow requests, 0 included below; oldest blocked for > 1531.862995 secs 2024-03-28T18:42:25.282 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:42:24 smithi022 ceph-mon[31985]: 2 slow requests, 0 included below; oldest blocked for > 1531.862995 secs 2024-03-28T18:42:25.424 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:42:24 smithi134 ceph-mon[35037]: 2 slow requests, 0 included below; oldest blocked for > 1531.862995 secs 2024-03-28T18:42:26.183 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:42:25 smithi079 ceph-mon[35312]: pgmap v1103: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 9 op/s 2024-03-28T18:42:26.282 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:42:25 smithi022 ceph-mon[31985]: pgmap v1103: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 9 op/s 2024-03-28T18:42:26.424 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:42:25 smithi134 ceph-mon[35037]: pgmap v1103: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 9 op/s 2024-03-28T18:42:28.282 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:42:27 smithi022 ceph-mon[31985]: pgmap v1104: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T18:42:28.424 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:42:27 smithi134 ceph-mon[35037]: pgmap v1104: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T18:42:28.433 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:42:27 smithi079 ceph-mon[35312]: pgmap v1104: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T18:42:30.282 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:42:29 smithi022 ceph-mon[31985]: pgmap v1105: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T18:42:30.282 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:42:29 smithi022 ceph-mon[31985]: 2 slow requests, 0 included below; oldest blocked for > 1536.863110 secs 2024-03-28T18:42:30.425 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:42:29 smithi134 ceph-mon[35037]: pgmap v1105: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T18:42:30.425 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:42:29 smithi134 ceph-mon[35037]: 2 slow requests, 0 included below; oldest blocked for > 1536.863110 secs 2024-03-28T18:42:30.432 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:42:29 smithi079 ceph-mon[35312]: pgmap v1105: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T18:42:30.432 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:42:29 smithi079 ceph-mon[35312]: 2 slow requests, 0 included below; oldest blocked for > 1536.863110 secs 2024-03-28T18:42:32.282 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:42:31 smithi022 ceph-mon[31985]: pgmap v1106: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 9 op/s 2024-03-28T18:42:32.424 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:42:31 smithi134 ceph-mon[35037]: pgmap v1106: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 9 op/s 2024-03-28T18:42:32.432 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:42:31 smithi079 ceph-mon[35312]: pgmap v1106: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 9 op/s 2024-03-28T18:42:34.282 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:42:33 smithi022 ceph-mon[31985]: pgmap v1107: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:42:34.425 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:42:33 smithi134 ceph-mon[35037]: pgmap v1107: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:42:34.432 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:42:33 smithi079 ceph-mon[35312]: pgmap v1107: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:42:35.282 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:42:34 smithi022 ceph-mon[31985]: 2 slow requests, 0 included below; oldest blocked for > 1541.863208 secs 2024-03-28T18:42:35.425 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:42:34 smithi134 ceph-mon[35037]: 2 slow requests, 0 included below; oldest blocked for > 1541.863208 secs 2024-03-28T18:42:35.432 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:42:34 smithi079 ceph-mon[35312]: 2 slow requests, 0 included below; oldest blocked for > 1541.863208 secs 2024-03-28T18:42:36.282 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:42:35 smithi022 ceph-mon[31985]: pgmap v1108: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:42:36.425 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:42:35 smithi134 ceph-mon[35037]: pgmap v1108: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:42:36.432 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:42:35 smithi079 ceph-mon[35312]: pgmap v1108: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:42:38.282 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:42:37 smithi022 ceph-mon[31985]: pgmap v1109: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:42:38.425 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:42:37 smithi134 ceph-mon[35037]: pgmap v1109: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:42:38.433 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:42:37 smithi079 ceph-mon[35312]: pgmap v1109: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:42:40.282 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:42:39 smithi022 ceph-mon[31985]: pgmap v1110: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:42:40.282 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:42:39 smithi022 ceph-mon[31985]: 2 slow requests, 0 included below; oldest blocked for > 1546.863292 secs 2024-03-28T18:42:40.424 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:42:39 smithi134 ceph-mon[35037]: pgmap v1110: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:42:40.425 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:42:39 smithi134 ceph-mon[35037]: 2 slow requests, 0 included below; oldest blocked for > 1546.863292 secs 2024-03-28T18:42:40.432 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:42:39 smithi079 ceph-mon[35312]: pgmap v1110: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:42:40.432 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:42:39 smithi079 ceph-mon[35312]: 2 slow requests, 0 included below; oldest blocked for > 1546.863292 secs 2024-03-28T18:42:42.282 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:42:41 smithi022 ceph-mon[31985]: pgmap v1111: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:42:42.424 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:42:41 smithi134 ceph-mon[35037]: pgmap v1111: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:42:42.432 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:42:41 smithi079 ceph-mon[35312]: pgmap v1111: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:42:44.282 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:42:43 smithi022 ceph-mon[31985]: pgmap v1112: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:42:44.425 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:42:43 smithi134 ceph-mon[35037]: pgmap v1112: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:42:44.432 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:42:43 smithi079 ceph-mon[35312]: pgmap v1112: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:42:45.282 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:42:45 smithi022 ceph-mon[31985]: 2 slow requests, 0 included below; oldest blocked for > 1551.863410 secs 2024-03-28T18:42:45.424 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:42:45 smithi134 ceph-mon[35037]: 2 slow requests, 0 included below; oldest blocked for > 1551.863410 secs 2024-03-28T18:42:45.432 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:42:45 smithi079 ceph-mon[35312]: 2 slow requests, 0 included below; oldest blocked for > 1551.863410 secs 2024-03-28T18:42:46.282 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:42:46 smithi022 ceph-mon[31985]: pgmap v1113: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:42:46.425 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:42:46 smithi134 ceph-mon[35037]: pgmap v1113: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:42:46.432 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:42:46 smithi079 ceph-mon[35312]: pgmap v1113: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:42:48.283 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:42:48 smithi022 ceph-mon[31985]: pgmap v1114: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:42:48.425 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:42:48 smithi134 ceph-mon[35037]: pgmap v1114: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:42:48.433 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:42:48 smithi079 ceph-mon[35312]: pgmap v1114: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:42:50.282 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:42:50 smithi022 ceph-mon[31985]: pgmap v1115: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:42:50.282 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:42:50 smithi022 ceph-mon[31985]: 2 slow requests, 0 included below; oldest blocked for > 1556.863491 secs 2024-03-28T18:42:50.425 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:42:50 smithi134 ceph-mon[35037]: pgmap v1115: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:42:50.425 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:42:50 smithi134 ceph-mon[35037]: 2 slow requests, 0 included below; oldest blocked for > 1556.863491 secs 2024-03-28T18:42:50.432 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:42:50 smithi079 ceph-mon[35312]: pgmap v1115: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:42:50.432 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:42:50 smithi079 ceph-mon[35312]: 2 slow requests, 0 included below; oldest blocked for > 1556.863491 secs 2024-03-28T18:42:52.282 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:42:52 smithi022 ceph-mon[31985]: pgmap v1116: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:42:52.424 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:42:52 smithi134 ceph-mon[35037]: pgmap v1116: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:42:52.433 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:42:52 smithi079 ceph-mon[35312]: pgmap v1116: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:42:54.425 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:42:54 smithi134 ceph-mon[35037]: pgmap v1117: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:42:54.432 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:42:54 smithi079 ceph-mon[35312]: pgmap v1117: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:42:54.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:42:54 smithi022 ceph-mon[31985]: pgmap v1117: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:42:55.425 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:42:55 smithi134 ceph-mon[35037]: 2 slow requests, 0 included below; oldest blocked for > 1561.863563 secs 2024-03-28T18:42:55.433 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:42:55 smithi079 ceph-mon[35312]: 2 slow requests, 0 included below; oldest blocked for > 1561.863563 secs 2024-03-28T18:42:55.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:42:55 smithi022 ceph-mon[31985]: 2 slow requests, 0 included below; oldest blocked for > 1561.863563 secs 2024-03-28T18:42:56.425 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:42:56 smithi134 ceph-mon[35037]: pgmap v1118: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:42:56.432 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:42:56 smithi079 ceph-mon[35312]: pgmap v1118: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:42:56.531 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:42:56 smithi022 ceph-mon[31985]: pgmap v1118: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:42:58.425 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:42:58 smithi134 ceph-mon[35037]: pgmap v1119: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:42:58.433 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:42:58 smithi079 ceph-mon[35312]: pgmap v1119: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:42:58.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:42:58 smithi022 ceph-mon[31985]: pgmap v1119: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:43:00.425 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:43:00 smithi134 ceph-mon[35037]: pgmap v1120: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:43:00.425 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:43:00 smithi134 ceph-mon[35037]: 2 slow requests, 0 included below; oldest blocked for > 1566.863683 secs 2024-03-28T18:43:00.432 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:43:00 smithi079 ceph-mon[35312]: pgmap v1120: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:43:00.432 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:43:00 smithi079 ceph-mon[35312]: 2 slow requests, 0 included below; oldest blocked for > 1566.863683 secs 2024-03-28T18:43:00.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:43:00 smithi022 ceph-mon[31985]: pgmap v1120: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:43:00.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:43:00 smithi022 ceph-mon[31985]: 2 slow requests, 0 included below; oldest blocked for > 1566.863683 secs 2024-03-28T18:43:02.425 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:43:02 smithi134 ceph-mon[35037]: pgmap v1121: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:43:02.433 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:43:02 smithi079 ceph-mon[35312]: pgmap v1121: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:43:02.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:43:02 smithi022 ceph-mon[31985]: pgmap v1121: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:43:04.425 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:43:04 smithi134 ceph-mon[35037]: pgmap v1122: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:43:04.432 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:43:04 smithi079 ceph-mon[35312]: pgmap v1122: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:43:04.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:43:04 smithi022 ceph-mon[31985]: pgmap v1122: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:43:05.425 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:43:05 smithi134 ceph-mon[35037]: 2 slow requests, 0 included below; oldest blocked for > 1571.863753 secs 2024-03-28T18:43:05.432 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:43:05 smithi079 ceph-mon[35312]: 2 slow requests, 0 included below; oldest blocked for > 1571.863753 secs 2024-03-28T18:43:05.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:43:05 smithi022 ceph-mon[31985]: 2 slow requests, 0 included below; oldest blocked for > 1571.863753 secs 2024-03-28T18:43:06.424 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:43:06 smithi134 ceph-mon[35037]: pgmap v1123: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:43:06.432 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:43:06 smithi079 ceph-mon[35312]: pgmap v1123: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:43:06.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:43:06 smithi022 ceph-mon[31985]: pgmap v1123: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:43:08.425 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:43:08 smithi134 ceph-mon[35037]: pgmap v1124: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:43:08.433 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:43:08 smithi079 ceph-mon[35312]: pgmap v1124: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:43:08.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:43:08 smithi022 ceph-mon[31985]: pgmap v1124: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:43:10.425 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:43:10 smithi134 ceph-mon[35037]: pgmap v1125: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:43:10.425 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:43:10 smithi134 ceph-mon[35037]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T18:43:10.425 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:43:10 smithi134 ceph-mon[35037]: 2 slow requests, 0 included below; oldest blocked for > 1576.863842 secs 2024-03-28T18:43:10.432 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:43:10 smithi079 ceph-mon[35312]: pgmap v1125: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:43:10.433 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:43:10 smithi079 ceph-mon[35312]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T18:43:10.433 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:43:10 smithi079 ceph-mon[35312]: 2 slow requests, 0 included below; oldest blocked for > 1576.863842 secs 2024-03-28T18:43:10.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:43:10 smithi022 ceph-mon[31985]: pgmap v1125: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:43:10.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:43:10 smithi022 ceph-mon[31985]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T18:43:10.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:43:10 smithi022 ceph-mon[31985]: 2 slow requests, 0 included below; oldest blocked for > 1576.863842 secs 2024-03-28T18:43:11.424 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:43:11 smithi134 ceph-mon[35037]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T18:43:11.425 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:43:11 smithi134 ceph-mon[35037]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T18:43:11.425 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:43:11 smithi134 ceph-mon[35037]: from='mgr.15522 ' entity='mgr.x' 2024-03-28T18:43:11.432 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:43:11 smithi079 ceph-mon[35312]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T18:43:11.432 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:43:11 smithi079 ceph-mon[35312]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T18:43:11.432 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:43:11 smithi079 ceph-mon[35312]: from='mgr.15522 ' entity='mgr.x' 2024-03-28T18:43:11.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:43:11 smithi022 ceph-mon[31985]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T18:43:11.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:43:11 smithi022 ceph-mon[31985]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T18:43:11.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:43:11 smithi022 ceph-mon[31985]: from='mgr.15522 ' entity='mgr.x' 2024-03-28T18:43:12.425 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:43:12 smithi134 ceph-mon[35037]: pgmap v1126: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 10 op/s 2024-03-28T18:43:12.425 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:43:12 smithi134 ceph-mon[35037]: pgmap v1127: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-03-28T18:43:12.432 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:43:12 smithi079 ceph-mon[35312]: pgmap v1126: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 10 op/s 2024-03-28T18:43:12.432 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:43:12 smithi079 ceph-mon[35312]: pgmap v1127: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-03-28T18:43:12.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:43:12 smithi022 ceph-mon[31985]: pgmap v1126: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 10 op/s 2024-03-28T18:43:12.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:43:12 smithi022 ceph-mon[31985]: pgmap v1127: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-03-28T18:43:14.425 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:43:14 smithi134 ceph-mon[35037]: pgmap v1128: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-03-28T18:43:14.432 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:43:14 smithi079 ceph-mon[35312]: pgmap v1128: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-03-28T18:43:14.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:43:14 smithi022 ceph-mon[31985]: pgmap v1128: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-03-28T18:43:15.424 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:43:15 smithi134 ceph-mon[35037]: 2 slow requests, 0 included below; oldest blocked for > 1581.863925 secs 2024-03-28T18:43:15.432 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:43:15 smithi079 ceph-mon[35312]: 2 slow requests, 0 included below; oldest blocked for > 1581.863925 secs 2024-03-28T18:43:15.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:43:15 smithi022 ceph-mon[31985]: 2 slow requests, 0 included below; oldest blocked for > 1581.863925 secs 2024-03-28T18:43:16.424 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:43:16 smithi134 ceph-mon[35037]: pgmap v1129: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 8 op/s 2024-03-28T18:43:16.433 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:43:16 smithi079 ceph-mon[35312]: pgmap v1129: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 8 op/s 2024-03-28T18:43:16.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:43:16 smithi022 ceph-mon[31985]: pgmap v1129: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 8 op/s 2024-03-28T18:43:18.425 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:43:18 smithi134 ceph-mon[35037]: pgmap v1130: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 8 op/s 2024-03-28T18:43:18.432 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:43:18 smithi079 ceph-mon[35312]: pgmap v1130: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 8 op/s 2024-03-28T18:43:18.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:43:18 smithi022 ceph-mon[31985]: pgmap v1130: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 8 op/s 2024-03-28T18:43:20.425 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:43:20 smithi134 ceph-mon[35037]: pgmap v1131: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 8 op/s 2024-03-28T18:43:20.425 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:43:20 smithi134 ceph-mon[35037]: 2 slow requests, 0 included below; oldest blocked for > 1586.864043 secs 2024-03-28T18:43:20.432 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:43:20 smithi079 ceph-mon[35312]: pgmap v1131: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 8 op/s 2024-03-28T18:43:20.432 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:43:20 smithi079 ceph-mon[35312]: 2 slow requests, 0 included below; oldest blocked for > 1586.864043 secs 2024-03-28T18:43:20.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:43:20 smithi022 ceph-mon[31985]: pgmap v1131: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 8 op/s 2024-03-28T18:43:20.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:43:20 smithi022 ceph-mon[31985]: 2 slow requests, 0 included below; oldest blocked for > 1586.864043 secs 2024-03-28T18:43:22.425 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:43:22 smithi134 ceph-mon[35037]: pgmap v1132: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 7 op/s 2024-03-28T18:43:22.432 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:43:22 smithi079 ceph-mon[35312]: pgmap v1132: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 7 op/s 2024-03-28T18:43:22.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:43:22 smithi022 ceph-mon[31985]: pgmap v1132: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 7 op/s 2024-03-28T18:43:24.433 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:43:24 smithi079 ceph-mon[35312]: pgmap v1133: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T18:43:24.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:43:24 smithi022 ceph-mon[31985]: pgmap v1133: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T18:43:24.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:43:24 smithi134 ceph-mon[35037]: pgmap v1133: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T18:43:25.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:43:25 smithi022 ceph-mon[31985]: 2 slow requests, 0 included below; oldest blocked for > 1591.864115 secs 2024-03-28T18:43:25.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:43:25 smithi134 ceph-mon[35037]: 2 slow requests, 0 included below; oldest blocked for > 1591.864115 secs 2024-03-28T18:43:25.683 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:43:25 smithi079 ceph-mon[35312]: 2 slow requests, 0 included below; oldest blocked for > 1591.864115 secs 2024-03-28T18:43:26.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:43:26 smithi022 ceph-mon[31985]: pgmap v1134: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 9 op/s 2024-03-28T18:43:26.674 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:43:26 smithi134 ceph-mon[35037]: pgmap v1134: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 9 op/s 2024-03-28T18:43:26.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:43:26 smithi079 ceph-mon[35312]: pgmap v1134: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 9 op/s 2024-03-28T18:43:28.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:43:28 smithi022 ceph-mon[31985]: pgmap v1135: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:43:28.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:43:28 smithi134 ceph-mon[35037]: pgmap v1135: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:43:28.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:43:28 smithi079 ceph-mon[35312]: pgmap v1135: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:43:30.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:43:30 smithi022 ceph-mon[31985]: pgmap v1136: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:43:30.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:43:30 smithi022 ceph-mon[31985]: 2 slow requests, 0 included below; oldest blocked for > 1596.864202 secs 2024-03-28T18:43:30.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:43:30 smithi134 ceph-mon[35037]: pgmap v1136: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:43:30.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:43:30 smithi134 ceph-mon[35037]: 2 slow requests, 0 included below; oldest blocked for > 1596.864202 secs 2024-03-28T18:43:30.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:43:30 smithi079 ceph-mon[35312]: pgmap v1136: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:43:30.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:43:30 smithi079 ceph-mon[35312]: 2 slow requests, 0 included below; oldest blocked for > 1596.864202 secs 2024-03-28T18:43:32.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:43:32 smithi022 ceph-mon[31985]: pgmap v1137: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:43:32.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:43:32 smithi134 ceph-mon[35037]: pgmap v1137: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:43:32.683 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:43:32 smithi079 ceph-mon[35312]: pgmap v1137: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:43:34.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:43:34 smithi022 ceph-mon[31985]: pgmap v1138: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:43:34.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:43:34 smithi134 ceph-mon[35037]: pgmap v1138: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:43:34.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:43:34 smithi079 ceph-mon[35312]: pgmap v1138: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:43:35.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:43:35 smithi022 ceph-mon[31985]: 2 slow requests, 0 included below; oldest blocked for > 1601.864267 secs 2024-03-28T18:43:35.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:43:35 smithi134 ceph-mon[35037]: 2 slow requests, 0 included below; oldest blocked for > 1601.864267 secs 2024-03-28T18:43:35.683 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:43:35 smithi079 ceph-mon[35312]: 2 slow requests, 0 included below; oldest blocked for > 1601.864267 secs 2024-03-28T18:43:36.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:43:36 smithi022 ceph-mon[31985]: pgmap v1139: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:43:36.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:43:36 smithi134 ceph-mon[35037]: pgmap v1139: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:43:36.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:43:36 smithi079 ceph-mon[35312]: pgmap v1139: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:43:38.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:43:38 smithi022 ceph-mon[31985]: pgmap v1140: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:43:38.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:43:38 smithi134 ceph-mon[35037]: pgmap v1140: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:43:38.683 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:43:38 smithi079 ceph-mon[35312]: pgmap v1140: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:43:40.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:43:40 smithi022 ceph-mon[31985]: pgmap v1141: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:43:40.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:43:40 smithi022 ceph-mon[31985]: 2 slow requests, 0 included below; oldest blocked for > 1606.864333 secs 2024-03-28T18:43:40.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:43:40 smithi134 ceph-mon[35037]: pgmap v1141: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:43:40.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:43:40 smithi134 ceph-mon[35037]: 2 slow requests, 0 included below; oldest blocked for > 1606.864333 secs 2024-03-28T18:43:40.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:43:40 smithi079 ceph-mon[35312]: pgmap v1141: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:43:40.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:43:40 smithi079 ceph-mon[35312]: 2 slow requests, 0 included below; oldest blocked for > 1606.864333 secs 2024-03-28T18:43:42.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:43:42 smithi022 ceph-mon[31985]: pgmap v1142: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:43:42.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:43:42 smithi134 ceph-mon[35037]: pgmap v1142: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:43:42.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:43:42 smithi079 ceph-mon[35312]: pgmap v1142: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:43:44.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:43:44 smithi022 ceph-mon[31985]: pgmap v1143: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:43:44.674 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:43:44 smithi134 ceph-mon[35037]: pgmap v1143: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:43:44.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:43:44 smithi079 ceph-mon[35312]: pgmap v1143: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:43:45.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:43:45 smithi022 ceph-mon[31985]: 2 slow requests, 0 included below; oldest blocked for > 1611.864399 secs 2024-03-28T18:43:45.674 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:43:45 smithi134 ceph-mon[35037]: 2 slow requests, 0 included below; oldest blocked for > 1611.864399 secs 2024-03-28T18:43:45.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:43:45 smithi079 ceph-mon[35312]: 2 slow requests, 0 included below; oldest blocked for > 1611.864399 secs 2024-03-28T18:43:46.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:43:46 smithi022 ceph-mon[31985]: pgmap v1144: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:43:46.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:43:46 smithi134 ceph-mon[35037]: pgmap v1144: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:43:46.683 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:43:46 smithi079 ceph-mon[35312]: pgmap v1144: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:43:48.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:43:48 smithi022 ceph-mon[31985]: pgmap v1145: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:43:48.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:43:48 smithi134 ceph-mon[35037]: pgmap v1145: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:43:48.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:43:48 smithi079 ceph-mon[35312]: pgmap v1145: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:43:50.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:43:50 smithi022 ceph-mon[31985]: pgmap v1146: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:43:50.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:43:50 smithi022 ceph-mon[31985]: 2 slow requests, 0 included below; oldest blocked for > 1616.864465 secs 2024-03-28T18:43:50.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:43:50 smithi134 ceph-mon[35037]: pgmap v1146: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:43:50.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:43:50 smithi134 ceph-mon[35037]: 2 slow requests, 0 included below; oldest blocked for > 1616.864465 secs 2024-03-28T18:43:50.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:43:50 smithi079 ceph-mon[35312]: pgmap v1146: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:43:50.683 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:43:50 smithi079 ceph-mon[35312]: 2 slow requests, 0 included below; oldest blocked for > 1616.864465 secs 2024-03-28T18:43:52.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:43:52 smithi134 ceph-mon[35037]: pgmap v1147: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:43:52.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:43:52 smithi079 ceph-mon[35312]: pgmap v1147: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:43:52.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:43:52 smithi022 ceph-mon[31985]: pgmap v1147: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:43:54.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:43:54 smithi134 ceph-mon[35037]: pgmap v1148: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:43:54.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:43:54 smithi079 ceph-mon[35312]: pgmap v1148: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:43:54.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:43:54 smithi022 ceph-mon[31985]: pgmap v1148: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:43:55.674 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:43:55 smithi134 ceph-mon[35037]: 2 slow requests, 0 included below; oldest blocked for > 1621.864498 secs 2024-03-28T18:43:55.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:43:55 smithi079 ceph-mon[35312]: 2 slow requests, 0 included below; oldest blocked for > 1621.864498 secs 2024-03-28T18:43:55.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:43:55 smithi022 ceph-mon[31985]: 2 slow requests, 0 included below; oldest blocked for > 1621.864498 secs 2024-03-28T18:43:56.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:43:56 smithi134 ceph-mon[35037]: pgmap v1149: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:43:56.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:43:56 smithi079 ceph-mon[35312]: pgmap v1149: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:43:56.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:43:56 smithi022 ceph-mon[31985]: pgmap v1149: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:43:58.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:43:58 smithi134 ceph-mon[35037]: pgmap v1150: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:43:58.683 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:43:58 smithi079 ceph-mon[35312]: pgmap v1150: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:43:58.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:43:58 smithi022 ceph-mon[31985]: pgmap v1150: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:44:00.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:44:00 smithi134 ceph-mon[35037]: pgmap v1151: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:44:00.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:44:00 smithi134 ceph-mon[35037]: 2 slow requests, 0 included below; oldest blocked for > 1626.864588 secs 2024-03-28T18:44:00.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:44:00 smithi079 ceph-mon[35312]: pgmap v1151: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:44:00.683 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:44:00 smithi079 ceph-mon[35312]: 2 slow requests, 0 included below; oldest blocked for > 1626.864588 secs 2024-03-28T18:44:00.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:44:00 smithi022 ceph-mon[31985]: pgmap v1151: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:44:00.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:44:00 smithi022 ceph-mon[31985]: 2 slow requests, 0 included below; oldest blocked for > 1626.864588 secs 2024-03-28T18:44:02.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:44:02 smithi134 ceph-mon[35037]: pgmap v1152: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:44:02.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:44:02 smithi079 ceph-mon[35312]: pgmap v1152: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:44:02.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:44:02 smithi022 ceph-mon[31985]: pgmap v1152: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:44:04.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:44:04 smithi134 ceph-mon[35037]: pgmap v1153: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:44:04.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:44:04 smithi079 ceph-mon[35312]: pgmap v1153: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:44:04.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:44:04 smithi022 ceph-mon[31985]: pgmap v1153: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:44:05.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:44:05 smithi134 ceph-mon[35037]: 2 slow requests, 0 included below; oldest blocked for > 1631.864709 secs 2024-03-28T18:44:05.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:44:05 smithi079 ceph-mon[35312]: 2 slow requests, 0 included below; oldest blocked for > 1631.864709 secs 2024-03-28T18:44:05.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:44:05 smithi022 ceph-mon[31985]: 2 slow requests, 0 included below; oldest blocked for > 1631.864709 secs 2024-03-28T18:44:06.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:44:06 smithi134 ceph-mon[35037]: pgmap v1154: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:44:06.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:44:06 smithi079 ceph-mon[35312]: pgmap v1154: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:44:06.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:44:06 smithi022 ceph-mon[31985]: pgmap v1154: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:44:07.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:44:07 smithi134 ceph-mon[35037]: pgmap v1155: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:44:07.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:44:07 smithi079 ceph-mon[35312]: pgmap v1155: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:44:07.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:44:07 smithi022 ceph-mon[31985]: pgmap v1155: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:44:09.925 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:44:09 smithi134 ceph-mon[35037]: pgmap v1156: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:44:09.925 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:44:09 smithi134 ceph-mon[35037]: 2 slow requests, 0 included below; oldest blocked for > 1636.864832 secs 2024-03-28T18:44:09.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:44:09 smithi079 ceph-mon[35312]: pgmap v1156: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:44:09.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:44:09 smithi079 ceph-mon[35312]: 2 slow requests, 0 included below; oldest blocked for > 1636.864832 secs 2024-03-28T18:44:10.032 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:44:09 smithi022 ceph-mon[31985]: pgmap v1156: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:44:10.032 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:44:09 smithi022 ceph-mon[31985]: 2 slow requests, 0 included below; oldest blocked for > 1636.864832 secs 2024-03-28T18:44:11.927 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:44:11 smithi134 ceph-mon[35037]: pgmap v1157: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:44:11.927 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:44:11 smithi134 ceph-mon[35037]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T18:44:11.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:44:11 smithi079 ceph-mon[35312]: pgmap v1157: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:44:11.933 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:44:11 smithi079 ceph-mon[35312]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T18:44:12.031 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:44:11 smithi022 ceph-mon[31985]: pgmap v1157: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:44:12.032 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:44:11 smithi022 ceph-mon[31985]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T18:44:12.925 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:44:12 smithi134 ceph-mon[35037]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T18:44:12.925 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:44:12 smithi134 ceph-mon[35037]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T18:44:12.925 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:44:12 smithi134 ceph-mon[35037]: from='mgr.15522 ' entity='mgr.x' 2024-03-28T18:44:12.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:44:12 smithi079 ceph-mon[35312]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T18:44:12.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:44:12 smithi079 ceph-mon[35312]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T18:44:12.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:44:12 smithi079 ceph-mon[35312]: from='mgr.15522 ' entity='mgr.x' 2024-03-28T18:44:13.032 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:44:12 smithi022 ceph-mon[31985]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T18:44:13.032 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:44:12 smithi022 ceph-mon[31985]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T18:44:13.032 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:44:12 smithi022 ceph-mon[31985]: from='mgr.15522 ' entity='mgr.x' 2024-03-28T18:44:13.925 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:44:13 smithi134 ceph-mon[35037]: pgmap v1158: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 6 op/s 2024-03-28T18:44:13.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:44:13 smithi079 ceph-mon[35312]: pgmap v1158: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 6 op/s 2024-03-28T18:44:14.032 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:44:13 smithi022 ceph-mon[31985]: pgmap v1158: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 6 op/s 2024-03-28T18:44:14.925 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:44:14 smithi134 ceph-mon[35037]: 2 slow requests, 0 included below; oldest blocked for > 1641.864957 secs 2024-03-28T18:44:14.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:44:14 smithi079 ceph-mon[35312]: 2 slow requests, 0 included below; oldest blocked for > 1641.864957 secs 2024-03-28T18:44:15.033 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:44:14 smithi022 ceph-mon[31985]: 2 slow requests, 0 included below; oldest blocked for > 1641.864957 secs 2024-03-28T18:44:15.925 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:44:15 smithi134 ceph-mon[35037]: pgmap v1159: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 10 op/s 2024-03-28T18:44:15.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:44:15 smithi079 ceph-mon[35312]: pgmap v1159: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 10 op/s 2024-03-28T18:44:16.032 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:44:15 smithi022 ceph-mon[31985]: pgmap v1159: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 10 op/s 2024-03-28T18:44:17.925 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:44:17 smithi134 ceph-mon[35037]: pgmap v1160: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 6 op/s 2024-03-28T18:44:17.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:44:17 smithi079 ceph-mon[35312]: pgmap v1160: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 6 op/s 2024-03-28T18:44:18.032 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:44:17 smithi022 ceph-mon[31985]: pgmap v1160: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 6 op/s 2024-03-28T18:44:19.925 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:44:19 smithi134 ceph-mon[35037]: pgmap v1161: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 6 op/s 2024-03-28T18:44:19.925 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:44:19 smithi134 ceph-mon[35037]: 2 slow requests, 0 included below; oldest blocked for > 1646.865022 secs 2024-03-28T18:44:19.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:44:19 smithi079 ceph-mon[35312]: pgmap v1161: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 6 op/s 2024-03-28T18:44:19.933 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:44:19 smithi079 ceph-mon[35312]: 2 slow requests, 0 included below; oldest blocked for > 1646.865022 secs 2024-03-28T18:44:20.032 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:44:19 smithi022 ceph-mon[31985]: pgmap v1161: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 6 op/s 2024-03-28T18:44:20.032 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:44:19 smithi022 ceph-mon[31985]: 2 slow requests, 0 included below; oldest blocked for > 1646.865022 secs 2024-03-28T18:44:21.933 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:44:21 smithi079 ceph-mon[35312]: pgmap v1162: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 10 op/s 2024-03-28T18:44:22.032 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:44:21 smithi022 ceph-mon[31985]: pgmap v1162: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 10 op/s 2024-03-28T18:44:22.175 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:44:21 smithi134 ceph-mon[35037]: pgmap v1162: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 10 op/s 2024-03-28T18:44:24.032 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:44:23 smithi022 ceph-mon[31985]: pgmap v1163: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 6 op/s 2024-03-28T18:44:24.175 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:44:23 smithi134 ceph-mon[35037]: pgmap v1163: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 6 op/s 2024-03-28T18:44:24.182 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:44:23 smithi079 ceph-mon[35312]: pgmap v1163: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 6 op/s 2024-03-28T18:44:25.032 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:44:24 smithi022 ceph-mon[31985]: 2 slow requests, 0 included below; oldest blocked for > 1651.865103 secs 2024-03-28T18:44:25.175 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:44:24 smithi134 ceph-mon[35037]: 2 slow requests, 0 included below; oldest blocked for > 1651.865103 secs 2024-03-28T18:44:25.182 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:44:24 smithi079 ceph-mon[35312]: 2 slow requests, 0 included below; oldest blocked for > 1651.865103 secs 2024-03-28T18:44:26.032 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:44:25 smithi022 ceph-mon[31985]: pgmap v1164: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:44:26.175 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:44:25 smithi134 ceph-mon[35037]: pgmap v1164: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:44:26.182 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:44:25 smithi079 ceph-mon[35312]: pgmap v1164: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:44:28.032 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:44:27 smithi022 ceph-mon[31985]: pgmap v1165: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:44:28.175 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:44:27 smithi134 ceph-mon[35037]: pgmap v1165: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:44:28.182 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:44:27 smithi079 ceph-mon[35312]: pgmap v1165: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:44:30.032 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:44:29 smithi022 ceph-mon[31985]: pgmap v1166: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:44:30.032 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:44:29 smithi022 ceph-mon[31985]: 2 slow requests, 0 included below; oldest blocked for > 1656.865174 secs 2024-03-28T18:44:30.175 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:44:29 smithi134 ceph-mon[35037]: pgmap v1166: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:44:30.175 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:44:29 smithi134 ceph-mon[35037]: 2 slow requests, 0 included below; oldest blocked for > 1656.865174 secs 2024-03-28T18:44:30.182 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:44:29 smithi079 ceph-mon[35312]: pgmap v1166: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:44:30.182 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:44:29 smithi079 ceph-mon[35312]: 2 slow requests, 0 included below; oldest blocked for > 1656.865174 secs 2024-03-28T18:44:32.032 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:44:31 smithi022 ceph-mon[31985]: pgmap v1167: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:44:32.174 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:44:31 smithi134 ceph-mon[35037]: pgmap v1167: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:44:32.182 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:44:31 smithi079 ceph-mon[35312]: pgmap v1167: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:44:34.032 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:44:33 smithi022 ceph-mon[31985]: pgmap v1168: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:44:34.174 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:44:33 smithi134 ceph-mon[35037]: pgmap v1168: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:44:34.182 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:44:33 smithi079 ceph-mon[35312]: pgmap v1168: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:44:35.032 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:44:34 smithi022 ceph-mon[31985]: 2 slow requests, 0 included below; oldest blocked for > 1661.865243 secs 2024-03-28T18:44:35.174 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:44:34 smithi134 ceph-mon[35037]: 2 slow requests, 0 included below; oldest blocked for > 1661.865243 secs 2024-03-28T18:44:35.182 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:44:34 smithi079 ceph-mon[35312]: 2 slow requests, 0 included below; oldest blocked for > 1661.865243 secs 2024-03-28T18:44:36.032 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:44:35 smithi022 ceph-mon[31985]: pgmap v1169: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:44:36.175 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:44:35 smithi134 ceph-mon[35037]: pgmap v1169: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:44:36.182 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:44:35 smithi079 ceph-mon[35312]: pgmap v1169: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:44:38.032 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:44:37 smithi022 ceph-mon[31985]: pgmap v1170: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:44:38.175 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:44:37 smithi134 ceph-mon[35037]: pgmap v1170: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:44:38.182 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:44:37 smithi079 ceph-mon[35312]: pgmap v1170: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:44:40.032 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:44:39 smithi022 ceph-mon[31985]: pgmap v1171: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:44:40.032 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:44:39 smithi022 ceph-mon[31985]: 2 slow requests, 0 included below; oldest blocked for > 1666.865308 secs 2024-03-28T18:44:40.175 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:44:39 smithi134 ceph-mon[35037]: pgmap v1171: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:44:40.175 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:44:39 smithi134 ceph-mon[35037]: 2 slow requests, 0 included below; oldest blocked for > 1666.865308 secs 2024-03-28T18:44:40.183 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:44:39 smithi079 ceph-mon[35312]: pgmap v1171: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:44:40.183 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:44:39 smithi079 ceph-mon[35312]: 2 slow requests, 0 included below; oldest blocked for > 1666.865308 secs 2024-03-28T18:44:42.032 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:44:41 smithi022 ceph-mon[31985]: pgmap v1172: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:44:42.174 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:44:41 smithi134 ceph-mon[35037]: pgmap v1172: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:44:42.182 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:44:41 smithi079 ceph-mon[35312]: pgmap v1172: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:44:44.032 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:44:43 smithi022 ceph-mon[31985]: pgmap v1173: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:44:44.175 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:44:43 smithi134 ceph-mon[35037]: pgmap v1173: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:44:44.182 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:44:43 smithi079 ceph-mon[35312]: pgmap v1173: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:44:45.032 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:44:44 smithi022 ceph-mon[31985]: 2 slow requests, 0 included below; oldest blocked for > 1671.865374 secs 2024-03-28T18:44:45.174 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:44:44 smithi134 ceph-mon[35037]: 2 slow requests, 0 included below; oldest blocked for > 1671.865374 secs 2024-03-28T18:44:45.182 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:44:44 smithi079 ceph-mon[35312]: 2 slow requests, 0 included below; oldest blocked for > 1671.865374 secs 2024-03-28T18:44:46.032 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:44:45 smithi022 ceph-mon[31985]: pgmap v1174: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:44:46.174 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:44:45 smithi134 ceph-mon[35037]: pgmap v1174: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:44:46.182 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:44:45 smithi079 ceph-mon[35312]: pgmap v1174: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:44:48.174 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:44:47 smithi134 ceph-mon[35037]: pgmap v1175: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:44:48.182 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:44:47 smithi079 ceph-mon[35312]: pgmap v1175: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:44:48.282 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:44:47 smithi022 ceph-mon[31985]: pgmap v1175: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:44:50.175 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:44:49 smithi134 ceph-mon[35037]: pgmap v1176: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:44:50.175 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:44:49 smithi134 ceph-mon[35037]: 2 slow requests, 0 included below; oldest blocked for > 1676.865494 secs 2024-03-28T18:44:50.182 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:44:49 smithi079 ceph-mon[35312]: pgmap v1176: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:44:50.183 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:44:49 smithi079 ceph-mon[35312]: 2 slow requests, 0 included below; oldest blocked for > 1676.865494 secs 2024-03-28T18:44:50.282 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:44:49 smithi022 ceph-mon[31985]: pgmap v1176: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:44:50.282 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:44:49 smithi022 ceph-mon[31985]: 2 slow requests, 0 included below; oldest blocked for > 1676.865494 secs 2024-03-28T18:44:52.175 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:44:51 smithi134 ceph-mon[35037]: pgmap v1177: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:44:52.182 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:44:51 smithi079 ceph-mon[35312]: pgmap v1177: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:44:52.282 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:44:51 smithi022 ceph-mon[31985]: pgmap v1177: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:44:54.175 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:44:53 smithi134 ceph-mon[35037]: pgmap v1178: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:44:54.182 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:44:53 smithi079 ceph-mon[35312]: pgmap v1178: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:44:54.282 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:44:53 smithi022 ceph-mon[31985]: pgmap v1178: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:44:55.175 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:44:54 smithi134 ceph-mon[35037]: 2 slow requests, 0 included below; oldest blocked for > 1681.865618 secs 2024-03-28T18:44:55.182 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:44:54 smithi079 ceph-mon[35312]: 2 slow requests, 0 included below; oldest blocked for > 1681.865618 secs 2024-03-28T18:44:55.282 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:44:54 smithi022 ceph-mon[31985]: 2 slow requests, 0 included below; oldest blocked for > 1681.865618 secs 2024-03-28T18:44:56.175 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:44:55 smithi134 ceph-mon[35037]: pgmap v1179: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:44:56.182 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:44:55 smithi079 ceph-mon[35312]: pgmap v1179: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:44:56.281 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:44:55 smithi022 ceph-mon[31985]: pgmap v1179: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:44:58.175 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:44:57 smithi134 ceph-mon[35037]: pgmap v1180: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:44:58.182 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:44:57 smithi079 ceph-mon[35312]: pgmap v1180: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:44:58.282 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:44:57 smithi022 ceph-mon[31985]: pgmap v1180: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:45:00.175 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:44:59 smithi134 ceph-mon[35037]: pgmap v1181: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:45:00.175 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:44:59 smithi134 ceph-mon[35037]: 2 slow requests, 0 included below; oldest blocked for > 1686.865702 secs 2024-03-28T18:45:00.182 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:44:59 smithi079 ceph-mon[35312]: pgmap v1181: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:45:00.182 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:44:59 smithi079 ceph-mon[35312]: 2 slow requests, 0 included below; oldest blocked for > 1686.865702 secs 2024-03-28T18:45:00.282 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:44:59 smithi022 ceph-mon[31985]: pgmap v1181: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:45:00.282 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:44:59 smithi022 ceph-mon[31985]: 2 slow requests, 0 included below; oldest blocked for > 1686.865702 secs 2024-03-28T18:45:02.175 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:45:01 smithi134 ceph-mon[35037]: pgmap v1182: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:45:02.182 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:45:01 smithi079 ceph-mon[35312]: pgmap v1182: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:45:02.282 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:45:01 smithi022 ceph-mon[31985]: pgmap v1182: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:45:04.174 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:45:03 smithi134 ceph-mon[35037]: pgmap v1183: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:45:04.182 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:45:03 smithi079 ceph-mon[35312]: pgmap v1183: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:45:04.282 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:45:03 smithi022 ceph-mon[31985]: pgmap v1183: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:45:05.174 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:45:04 smithi134 ceph-mon[35037]: 2 slow requests, 0 included below; oldest blocked for > 1691.865784 secs 2024-03-28T18:45:05.182 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:45:04 smithi079 ceph-mon[35312]: 2 slow requests, 0 included below; oldest blocked for > 1691.865784 secs 2024-03-28T18:45:05.282 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:45:04 smithi022 ceph-mon[31985]: 2 slow requests, 0 included below; oldest blocked for > 1691.865784 secs 2024-03-28T18:45:06.175 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:45:05 smithi134 ceph-mon[35037]: pgmap v1184: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:45:06.182 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:45:05 smithi079 ceph-mon[35312]: pgmap v1184: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:45:06.282 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:45:05 smithi022 ceph-mon[31985]: pgmap v1184: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:45:08.174 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:45:07 smithi134 ceph-mon[35037]: pgmap v1185: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:45:08.182 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:45:07 smithi079 ceph-mon[35312]: pgmap v1185: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:45:08.282 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:45:07 smithi022 ceph-mon[31985]: pgmap v1185: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:45:10.175 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:45:09 smithi134 ceph-mon[35037]: pgmap v1186: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:45:10.175 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:45:09 smithi134 ceph-mon[35037]: 2 slow requests, 0 included below; oldest blocked for > 1696.865871 secs 2024-03-28T18:45:10.182 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:45:09 smithi079 ceph-mon[35312]: pgmap v1186: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:45:10.183 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:45:09 smithi079 ceph-mon[35312]: 2 slow requests, 0 included below; oldest blocked for > 1696.865871 secs 2024-03-28T18:45:10.282 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:45:09 smithi022 ceph-mon[31985]: pgmap v1186: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:45:10.282 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:45:09 smithi022 ceph-mon[31985]: 2 slow requests, 0 included below; oldest blocked for > 1696.865871 secs 2024-03-28T18:45:12.175 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:45:11 smithi134 ceph-mon[35037]: pgmap v1187: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:45:12.182 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:45:11 smithi079 ceph-mon[35312]: pgmap v1187: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:45:12.282 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:45:11 smithi022 ceph-mon[31985]: pgmap v1187: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:45:14.175 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:45:13 smithi134 ceph-mon[35037]: pgmap v1188: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:45:14.175 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:45:13 smithi134 ceph-mon[35037]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T18:45:14.182 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:45:13 smithi079 ceph-mon[35312]: pgmap v1188: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:45:14.183 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:45:13 smithi079 ceph-mon[35312]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T18:45:14.281 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:45:13 smithi022 ceph-mon[31985]: pgmap v1188: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:45:14.282 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:45:13 smithi022 ceph-mon[31985]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T18:45:15.175 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:45:14 smithi134 ceph-mon[35037]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T18:45:15.175 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:45:14 smithi134 ceph-mon[35037]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T18:45:15.175 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:45:14 smithi134 ceph-mon[35037]: pgmap v1189: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 10 op/s 2024-03-28T18:45:15.175 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:45:14 smithi134 ceph-mon[35037]: pgmap v1190: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-03-28T18:45:15.175 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:45:14 smithi134 ceph-mon[35037]: from='mgr.15522 ' entity='mgr.x' 2024-03-28T18:45:15.175 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:45:14 smithi134 ceph-mon[35037]: 2 slow requests, 0 included below; oldest blocked for > 1701.865941 secs 2024-03-28T18:45:15.182 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:45:14 smithi079 ceph-mon[35312]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T18:45:15.182 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:45:14 smithi079 ceph-mon[35312]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T18:45:15.183 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:45:14 smithi079 ceph-mon[35312]: pgmap v1189: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 10 op/s 2024-03-28T18:45:15.183 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:45:14 smithi079 ceph-mon[35312]: pgmap v1190: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-03-28T18:45:15.183 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:45:14 smithi079 ceph-mon[35312]: from='mgr.15522 ' entity='mgr.x' 2024-03-28T18:45:15.183 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:45:14 smithi079 ceph-mon[35312]: 2 slow requests, 0 included below; oldest blocked for > 1701.865941 secs 2024-03-28T18:45:15.282 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:45:14 smithi022 ceph-mon[31985]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T18:45:15.282 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:45:14 smithi022 ceph-mon[31985]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T18:45:15.282 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:45:14 smithi022 ceph-mon[31985]: pgmap v1189: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 10 op/s 2024-03-28T18:45:15.282 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:45:14 smithi022 ceph-mon[31985]: pgmap v1190: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-03-28T18:45:15.282 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:45:14 smithi022 ceph-mon[31985]: from='mgr.15522 ' entity='mgr.x' 2024-03-28T18:45:15.282 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:45:14 smithi022 ceph-mon[31985]: 2 slow requests, 0 included below; oldest blocked for > 1701.865941 secs 2024-03-28T18:45:17.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:45:17 smithi022 ceph-mon[31985]: pgmap v1191: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-03-28T18:45:17.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:45:17 smithi134 ceph-mon[35037]: pgmap v1191: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-03-28T18:45:17.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:45:17 smithi079 ceph-mon[35312]: pgmap v1191: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-03-28T18:45:19.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:45:19 smithi022 ceph-mon[31985]: pgmap v1192: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-03-28T18:45:19.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:45:19 smithi134 ceph-mon[35037]: pgmap v1192: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-03-28T18:45:19.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:45:19 smithi079 ceph-mon[35312]: pgmap v1192: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-03-28T18:45:20.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:45:20 smithi022 ceph-mon[31985]: 2 slow requests, 0 included below; oldest blocked for > 1706.866012 secs 2024-03-28T18:45:20.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:45:20 smithi134 ceph-mon[35037]: 2 slow requests, 0 included below; oldest blocked for > 1706.866012 secs 2024-03-28T18:45:20.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:45:20 smithi079 ceph-mon[35312]: 2 slow requests, 0 included below; oldest blocked for > 1706.866012 secs 2024-03-28T18:45:21.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:45:21 smithi022 ceph-mon[31985]: pgmap v1193: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-03-28T18:45:21.674 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:45:21 smithi134 ceph-mon[35037]: pgmap v1193: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-03-28T18:45:21.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:45:21 smithi079 ceph-mon[35312]: pgmap v1193: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-03-28T18:45:23.533 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:45:23 smithi022 ceph-mon[31985]: pgmap v1194: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-03-28T18:45:23.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:45:23 smithi134 ceph-mon[35037]: pgmap v1194: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-03-28T18:45:23.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:45:23 smithi079 ceph-mon[35312]: pgmap v1194: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-03-28T18:45:25.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:45:25 smithi022 ceph-mon[31985]: pgmap v1195: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 7 op/s 2024-03-28T18:45:25.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:45:25 smithi022 ceph-mon[31985]: 2 slow requests, 0 included below; oldest blocked for > 1711.866122 secs 2024-03-28T18:45:25.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:45:25 smithi134 ceph-mon[35037]: pgmap v1195: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 7 op/s 2024-03-28T18:45:25.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:45:25 smithi134 ceph-mon[35037]: 2 slow requests, 0 included below; oldest blocked for > 1711.866122 secs 2024-03-28T18:45:25.683 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:45:25 smithi079 ceph-mon[35312]: pgmap v1195: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 7 op/s 2024-03-28T18:45:25.683 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:45:25 smithi079 ceph-mon[35312]: 2 slow requests, 0 included below; oldest blocked for > 1711.866122 secs 2024-03-28T18:45:27.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:45:27 smithi022 ceph-mon[31985]: pgmap v1196: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:45:27.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:45:27 smithi134 ceph-mon[35037]: pgmap v1196: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:45:27.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:45:27 smithi079 ceph-mon[35312]: pgmap v1196: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:45:29.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:45:29 smithi134 ceph-mon[35037]: pgmap v1197: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:45:29.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:45:29 smithi079 ceph-mon[35312]: pgmap v1197: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:45:29.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:45:29 smithi022 ceph-mon[31985]: pgmap v1197: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:45:30.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:45:30 smithi134 ceph-mon[35037]: 2 slow requests, 0 included below; oldest blocked for > 1716.866160 secs 2024-03-28T18:45:30.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:45:30 smithi079 ceph-mon[35312]: 2 slow requests, 0 included below; oldest blocked for > 1716.866160 secs 2024-03-28T18:45:30.783 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:45:30 smithi022 ceph-mon[31985]: 2 slow requests, 0 included below; oldest blocked for > 1716.866160 secs 2024-03-28T18:45:31.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:45:31 smithi134 ceph-mon[35037]: pgmap v1198: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:45:31.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:45:31 smithi079 ceph-mon[35312]: pgmap v1198: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:45:31.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:45:31 smithi022 ceph-mon[31985]: pgmap v1198: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:45:33.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:45:33 smithi134 ceph-mon[35037]: pgmap v1199: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:45:33.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:45:33 smithi079 ceph-mon[35312]: pgmap v1199: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:45:33.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:45:33 smithi022 ceph-mon[31985]: pgmap v1199: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:45:35.679 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:45:35 smithi134 ceph-mon[35037]: pgmap v1200: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:45:35.679 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:45:35 smithi134 ceph-mon[35037]: 2 slow requests, 0 included below; oldest blocked for > 1721.866191 secs 2024-03-28T18:45:35.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:45:35 smithi079 ceph-mon[35312]: pgmap v1200: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:45:35.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:45:35 smithi079 ceph-mon[35312]: 2 slow requests, 0 included below; oldest blocked for > 1721.866191 secs 2024-03-28T18:45:35.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:45:35 smithi022 ceph-mon[31985]: pgmap v1200: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:45:35.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:45:35 smithi022 ceph-mon[31985]: 2 slow requests, 0 included below; oldest blocked for > 1721.866191 secs 2024-03-28T18:45:37.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:45:37 smithi134 ceph-mon[35037]: pgmap v1201: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:45:37.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:45:37 smithi079 ceph-mon[35312]: pgmap v1201: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:45:37.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:45:37 smithi022 ceph-mon[31985]: pgmap v1201: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:45:39.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:45:39 smithi134 ceph-mon[35037]: pgmap v1202: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:45:39.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:45:39 smithi079 ceph-mon[35312]: pgmap v1202: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:45:39.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:45:39 smithi022 ceph-mon[31985]: pgmap v1202: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:45:40.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:45:40 smithi134 ceph-mon[35037]: 2 slow requests, 0 included below; oldest blocked for > 1726.866257 secs 2024-03-28T18:45:40.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:45:40 smithi079 ceph-mon[35312]: 2 slow requests, 0 included below; oldest blocked for > 1726.866257 secs 2024-03-28T18:45:40.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:45:40 smithi022 ceph-mon[31985]: 2 slow requests, 0 included below; oldest blocked for > 1726.866257 secs 2024-03-28T18:45:41.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:45:41 smithi134 ceph-mon[35037]: pgmap v1203: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:45:41.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:45:41 smithi079 ceph-mon[35312]: pgmap v1203: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:45:41.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:45:41 smithi022 ceph-mon[31985]: pgmap v1203: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:45:43.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:45:43 smithi134 ceph-mon[35037]: pgmap v1204: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:45:43.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:45:43 smithi079 ceph-mon[35312]: pgmap v1204: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:45:43.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:45:43 smithi022 ceph-mon[31985]: pgmap v1204: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:45:45.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:45:45 smithi134 ceph-mon[35037]: pgmap v1205: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:45:45.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:45:45 smithi134 ceph-mon[35037]: 2 slow requests, 0 included below; oldest blocked for > 1731.866328 secs 2024-03-28T18:45:45.683 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:45:45 smithi079 ceph-mon[35312]: pgmap v1205: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:45:45.683 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:45:45 smithi079 ceph-mon[35312]: 2 slow requests, 0 included below; oldest blocked for > 1731.866328 secs 2024-03-28T18:45:45.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:45:45 smithi022 ceph-mon[31985]: pgmap v1205: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:45:45.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:45:45 smithi022 ceph-mon[31985]: 2 slow requests, 0 included below; oldest blocked for > 1731.866328 secs 2024-03-28T18:45:47.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:45:47 smithi134 ceph-mon[35037]: pgmap v1206: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:45:47.683 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:45:47 smithi079 ceph-mon[35312]: pgmap v1206: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:45:47.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:45:47 smithi022 ceph-mon[31985]: pgmap v1206: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:45:49.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:45:49 smithi134 ceph-mon[35037]: pgmap v1207: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:45:49.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:45:49 smithi079 ceph-mon[35312]: pgmap v1207: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:45:49.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:45:49 smithi022 ceph-mon[31985]: pgmap v1207: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:45:50.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:45:50 smithi134 ceph-mon[35037]: 2 slow requests, 0 included below; oldest blocked for > 1736.866427 secs 2024-03-28T18:45:50.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:45:50 smithi134 ceph-mon[35037]: pgmap v1208: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:45:50.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:45:50 smithi079 ceph-mon[35312]: 2 slow requests, 0 included below; oldest blocked for > 1736.866427 secs 2024-03-28T18:45:50.683 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:45:50 smithi079 ceph-mon[35312]: pgmap v1208: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:45:50.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:45:50 smithi022 ceph-mon[31985]: 2 slow requests, 0 included below; oldest blocked for > 1736.866427 secs 2024-03-28T18:45:50.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:45:50 smithi022 ceph-mon[31985]: pgmap v1208: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:45:53.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:45:53 smithi022 ceph-mon[31985]: pgmap v1209: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:45:53.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:45:53 smithi134 ceph-mon[35037]: pgmap v1209: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:45:53.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:45:53 smithi079 ceph-mon[35312]: pgmap v1209: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:45:55.425 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:45:55 smithi134 ceph-mon[35037]: pgmap v1210: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:45:55.425 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:45:55 smithi134 ceph-mon[35037]: 2 slow requests, 0 included below; oldest blocked for > 1741.866501 secs 2024-03-28T18:45:55.433 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:45:55 smithi079 ceph-mon[35312]: pgmap v1210: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:45:55.433 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:45:55 smithi079 ceph-mon[35312]: 2 slow requests, 0 included below; oldest blocked for > 1741.866501 secs 2024-03-28T18:45:55.533 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:45:55 smithi022 ceph-mon[31985]: pgmap v1210: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:45:55.533 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:45:55 smithi022 ceph-mon[31985]: 2 slow requests, 0 included below; oldest blocked for > 1741.866501 secs 2024-03-28T18:45:57.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:45:57 smithi022 ceph-mon[31985]: pgmap v1211: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:45:57.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:45:57 smithi134 ceph-mon[35037]: pgmap v1211: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:45:57.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:45:57 smithi079 ceph-mon[35312]: pgmap v1211: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:45:59.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:45:59 smithi022 ceph-mon[31985]: pgmap v1212: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:45:59.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:45:59 smithi134 ceph-mon[35037]: pgmap v1212: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:45:59.683 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:45:59 smithi079 ceph-mon[35312]: pgmap v1212: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:46:00.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:46:00 smithi022 ceph-mon[31985]: 2 slow requests, 0 included below; oldest blocked for > 1746.866629 secs 2024-03-28T18:46:00.674 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:46:00 smithi134 ceph-mon[35037]: 2 slow requests, 0 included below; oldest blocked for > 1746.866629 secs 2024-03-28T18:46:00.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:46:00 smithi079 ceph-mon[35312]: 2 slow requests, 0 included below; oldest blocked for > 1746.866629 secs 2024-03-28T18:46:01.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:46:01 smithi022 ceph-mon[31985]: pgmap v1213: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:46:01.674 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:46:01 smithi134 ceph-mon[35037]: pgmap v1213: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:46:01.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:46:01 smithi079 ceph-mon[35312]: pgmap v1213: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:46:03.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:46:03 smithi134 ceph-mon[35037]: pgmap v1214: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:46:03.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:46:03 smithi079 ceph-mon[35312]: pgmap v1214: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:46:03.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:46:03 smithi022 ceph-mon[31985]: pgmap v1214: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:46:05.425 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:46:05 smithi134 ceph-mon[35037]: pgmap v1215: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:46:05.425 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:46:05 smithi134 ceph-mon[35037]: 2 slow requests, 0 included below; oldest blocked for > 1751.866697 secs 2024-03-28T18:46:05.432 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:46:05 smithi079 ceph-mon[35312]: pgmap v1215: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:46:05.432 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:46:05 smithi079 ceph-mon[35312]: 2 slow requests, 0 included below; oldest blocked for > 1751.866697 secs 2024-03-28T18:46:05.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:46:05 smithi022 ceph-mon[31985]: pgmap v1215: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:46:05.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:46:05 smithi022 ceph-mon[31985]: 2 slow requests, 0 included below; oldest blocked for > 1751.866697 secs 2024-03-28T18:46:07.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:46:07 smithi022 ceph-mon[31985]: pgmap v1216: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:46:07.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:46:07 smithi134 ceph-mon[35037]: pgmap v1216: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:46:07.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:46:07 smithi079 ceph-mon[35312]: pgmap v1216: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:46:09.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:46:09 smithi022 ceph-mon[31985]: pgmap v1217: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:46:09.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:46:09 smithi134 ceph-mon[35037]: pgmap v1217: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:46:09.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:46:09 smithi079 ceph-mon[35312]: pgmap v1217: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:46:10.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:46:10 smithi022 ceph-mon[31985]: 2 slow requests, 0 included below; oldest blocked for > 1756.866776 secs 2024-03-28T18:46:10.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:46:10 smithi134 ceph-mon[35037]: 2 slow requests, 0 included below; oldest blocked for > 1756.866776 secs 2024-03-28T18:46:10.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:46:10 smithi079 ceph-mon[35312]: 2 slow requests, 0 included below; oldest blocked for > 1756.866776 secs 2024-03-28T18:46:11.576 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:46:11 smithi134 ceph-mon[35037]: pgmap v1218: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:46:11.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:46:11 smithi079 ceph-mon[35312]: pgmap v1218: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:46:11.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:46:11 smithi022 ceph-mon[31985]: pgmap v1218: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:46:13.585 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:46:13 smithi022 ceph-mon[31985]: pgmap v1219: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:46:13.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:46:13 smithi134 ceph-mon[35037]: pgmap v1219: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:46:13.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:46:13 smithi079 ceph-mon[35312]: pgmap v1219: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:46:15.674 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:46:15 smithi134 ceph-mon[35037]: pgmap v1220: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:46:15.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:46:15 smithi134 ceph-mon[35037]: 2 slow requests, 0 included below; oldest blocked for > 1761.866846 secs 2024-03-28T18:46:15.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:46:15 smithi134 ceph-mon[35037]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T18:46:15.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:46:15 smithi079 ceph-mon[35312]: pgmap v1220: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:46:15.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:46:15 smithi079 ceph-mon[35312]: 2 slow requests, 0 included below; oldest blocked for > 1761.866846 secs 2024-03-28T18:46:15.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:46:15 smithi079 ceph-mon[35312]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T18:46:15.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:46:15 smithi022 ceph-mon[31985]: pgmap v1220: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:46:15.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:46:15 smithi022 ceph-mon[31985]: 2 slow requests, 0 included below; oldest blocked for > 1761.866846 secs 2024-03-28T18:46:15.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:46:15 smithi022 ceph-mon[31985]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T18:46:16.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:46:16 smithi134 ceph-mon[35037]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T18:46:16.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:46:16 smithi134 ceph-mon[35037]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T18:46:16.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:46:16 smithi134 ceph-mon[35037]: from='mgr.15522 ' entity='mgr.x' 2024-03-28T18:46:16.683 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:46:16 smithi079 ceph-mon[35312]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T18:46:16.683 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:46:16 smithi079 ceph-mon[35312]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T18:46:16.683 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:46:16 smithi079 ceph-mon[35312]: from='mgr.15522 ' entity='mgr.x' 2024-03-28T18:46:16.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:46:16 smithi022 ceph-mon[31985]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T18:46:16.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:46:16 smithi022 ceph-mon[31985]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T18:46:16.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:46:16 smithi022 ceph-mon[31985]: from='mgr.15522 ' entity='mgr.x' 2024-03-28T18:46:17.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:46:17 smithi134 ceph-mon[35037]: pgmap v1221: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 6 op/s 2024-03-28T18:46:17.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:46:17 smithi134 ceph-mon[35037]: pgmap v1222: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-03-28T18:46:17.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:46:17 smithi079 ceph-mon[35312]: pgmap v1221: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 6 op/s 2024-03-28T18:46:17.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:46:17 smithi079 ceph-mon[35312]: pgmap v1222: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-03-28T18:46:17.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:46:17 smithi022 ceph-mon[31985]: pgmap v1221: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 6 op/s 2024-03-28T18:46:17.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:46:17 smithi022 ceph-mon[31985]: pgmap v1222: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-03-28T18:46:19.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:46:19 smithi134 ceph-mon[35037]: pgmap v1223: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-03-28T18:46:19.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:46:19 smithi079 ceph-mon[35312]: pgmap v1223: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-03-28T18:46:19.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:46:19 smithi022 ceph-mon[31985]: pgmap v1223: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-03-28T18:46:20.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:46:20 smithi134 ceph-mon[35037]: 2 slow requests, 0 included below; oldest blocked for > 1766.866913 secs 2024-03-28T18:46:20.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:46:20 smithi079 ceph-mon[35312]: 2 slow requests, 0 included below; oldest blocked for > 1766.866913 secs 2024-03-28T18:46:20.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:46:20 smithi022 ceph-mon[31985]: 2 slow requests, 0 included below; oldest blocked for > 1766.866913 secs 2024-03-28T18:46:21.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:46:21 smithi134 ceph-mon[35037]: pgmap v1224: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-03-28T18:46:21.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:46:21 smithi079 ceph-mon[35312]: pgmap v1224: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-03-28T18:46:21.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:46:21 smithi022 ceph-mon[31985]: pgmap v1224: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-03-28T18:46:23.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:46:23 smithi134 ceph-mon[35037]: pgmap v1225: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-03-28T18:46:23.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:46:23 smithi079 ceph-mon[35312]: pgmap v1225: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-03-28T18:46:23.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:46:23 smithi022 ceph-mon[31985]: pgmap v1225: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-03-28T18:46:25.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:46:25 smithi134 ceph-mon[35037]: pgmap v1226: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 KiB/s rd, 4 op/s 2024-03-28T18:46:25.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:46:25 smithi134 ceph-mon[35037]: 2 slow requests, 0 included below; oldest blocked for > 1771.866987 secs 2024-03-28T18:46:25.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:46:25 smithi079 ceph-mon[35312]: pgmap v1226: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 KiB/s rd, 4 op/s 2024-03-28T18:46:25.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:46:25 smithi079 ceph-mon[35312]: 2 slow requests, 0 included below; oldest blocked for > 1771.866987 secs 2024-03-28T18:46:25.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:46:25 smithi022 ceph-mon[31985]: pgmap v1226: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 KiB/s rd, 4 op/s 2024-03-28T18:46:25.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:46:25 smithi022 ceph-mon[31985]: 2 slow requests, 0 included below; oldest blocked for > 1771.866987 secs 2024-03-28T18:46:27.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:46:27 smithi134 ceph-mon[35037]: pgmap v1227: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 7 op/s 2024-03-28T18:46:27.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:46:27 smithi079 ceph-mon[35312]: pgmap v1227: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 7 op/s 2024-03-28T18:46:27.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:46:27 smithi022 ceph-mon[31985]: pgmap v1227: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 7 op/s 2024-03-28T18:46:29.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:46:29 smithi134 ceph-mon[35037]: pgmap v1228: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:46:29.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:46:29 smithi079 ceph-mon[35312]: pgmap v1228: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:46:29.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:46:29 smithi022 ceph-mon[31985]: pgmap v1228: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:46:30.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:46:30 smithi134 ceph-mon[35037]: 2 slow requests, 0 included below; oldest blocked for > 1776.867107 secs 2024-03-28T18:46:30.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:46:30 smithi079 ceph-mon[35312]: 2 slow requests, 0 included below; oldest blocked for > 1776.867107 secs 2024-03-28T18:46:30.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:46:30 smithi022 ceph-mon[31985]: 2 slow requests, 0 included below; oldest blocked for > 1776.867107 secs 2024-03-28T18:46:31.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:46:31 smithi134 ceph-mon[35037]: pgmap v1229: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:46:31.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:46:31 smithi079 ceph-mon[35312]: pgmap v1229: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:46:31.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:46:31 smithi022 ceph-mon[31985]: pgmap v1229: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:46:33.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:46:33 smithi134 ceph-mon[35037]: pgmap v1230: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:46:33.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:46:33 smithi079 ceph-mon[35312]: pgmap v1230: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:46:33.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:46:33 smithi022 ceph-mon[31985]: pgmap v1230: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:46:34.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:46:34 smithi134 ceph-mon[35037]: pgmap v1231: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:46:34.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:46:34 smithi079 ceph-mon[35312]: pgmap v1231: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:46:34.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:46:34 smithi022 ceph-mon[31985]: pgmap v1231: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:46:35.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:46:35 smithi134 ceph-mon[35037]: 2 slow requests, 0 included below; oldest blocked for > 1781.867192 secs 2024-03-28T18:46:35.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:46:35 smithi079 ceph-mon[35312]: 2 slow requests, 0 included below; oldest blocked for > 1781.867192 secs 2024-03-28T18:46:35.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:46:35 smithi022 ceph-mon[31985]: 2 slow requests, 0 included below; oldest blocked for > 1781.867192 secs 2024-03-28T18:46:36.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:46:36 smithi134 ceph-mon[35037]: pgmap v1232: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:46:36.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:46:36 smithi079 ceph-mon[35312]: pgmap v1232: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:46:36.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:46:36 smithi022 ceph-mon[31985]: pgmap v1232: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:46:39.425 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:46:39 smithi134 ceph-mon[35037]: pgmap v1233: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:46:39.432 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:46:39 smithi079 ceph-mon[35312]: pgmap v1233: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:46:39.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:46:39 smithi022 ceph-mon[31985]: pgmap v1233: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:46:40.425 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:46:40 smithi134 ceph-mon[35037]: 2 slow requests, 0 included below; oldest blocked for > 1786.867287 secs 2024-03-28T18:46:40.432 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:46:40 smithi079 ceph-mon[35312]: 2 slow requests, 0 included below; oldest blocked for > 1786.867287 secs 2024-03-28T18:46:40.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:46:40 smithi022 ceph-mon[31985]: 2 slow requests, 0 included below; oldest blocked for > 1786.867287 secs 2024-03-28T18:46:41.424 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:46:41 smithi134 ceph-mon[35037]: pgmap v1234: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:46:41.432 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:46:41 smithi079 ceph-mon[35312]: pgmap v1234: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:46:41.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:46:41 smithi022 ceph-mon[31985]: pgmap v1234: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:46:43.425 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:46:43 smithi134 ceph-mon[35037]: pgmap v1235: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:46:43.432 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:46:43 smithi079 ceph-mon[35312]: pgmap v1235: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:46:43.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:46:43 smithi022 ceph-mon[31985]: pgmap v1235: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:46:45.425 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:46:45 smithi134 ceph-mon[35037]: pgmap v1236: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:46:45.425 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:46:45 smithi134 ceph-mon[35037]: 2 slow requests, 0 included below; oldest blocked for > 1791.867359 secs 2024-03-28T18:46:45.432 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:46:45 smithi079 ceph-mon[35312]: pgmap v1236: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:46:45.432 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:46:45 smithi079 ceph-mon[35312]: 2 slow requests, 0 included below; oldest blocked for > 1791.867359 secs 2024-03-28T18:46:45.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:46:45 smithi022 ceph-mon[31985]: pgmap v1236: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:46:45.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:46:45 smithi022 ceph-mon[31985]: 2 slow requests, 0 included below; oldest blocked for > 1791.867359 secs 2024-03-28T18:46:47.425 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:46:47 smithi134 ceph-mon[35037]: pgmap v1237: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:46:47.432 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:46:47 smithi079 ceph-mon[35312]: pgmap v1237: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:46:47.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:46:47 smithi022 ceph-mon[31985]: pgmap v1237: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:46:49.425 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:46:49 smithi134 ceph-mon[35037]: pgmap v1238: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:46:49.432 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:46:49 smithi079 ceph-mon[35312]: pgmap v1238: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:46:49.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:46:49 smithi022 ceph-mon[31985]: pgmap v1238: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:46:50.425 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:46:50 smithi134 ceph-mon[35037]: 2 slow requests, 0 included below; oldest blocked for > 1796.867383 secs 2024-03-28T18:46:50.432 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:46:50 smithi079 ceph-mon[35312]: 2 slow requests, 0 included below; oldest blocked for > 1796.867383 secs 2024-03-28T18:46:50.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:46:50 smithi022 ceph-mon[31985]: 2 slow requests, 0 included below; oldest blocked for > 1796.867383 secs 2024-03-28T18:46:51.425 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:46:51 smithi134 ceph-mon[35037]: pgmap v1239: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:46:51.432 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:46:51 smithi079 ceph-mon[35312]: pgmap v1239: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:46:51.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:46:51 smithi022 ceph-mon[31985]: pgmap v1239: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:46:53.425 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:46:53 smithi134 ceph-mon[35037]: pgmap v1240: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:46:53.432 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:46:53 smithi079 ceph-mon[35312]: pgmap v1240: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:46:53.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:46:53 smithi022 ceph-mon[31985]: pgmap v1240: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:46:55.425 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:46:55 smithi134 ceph-mon[35037]: pgmap v1241: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:46:55.425 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:46:55 smithi134 ceph-mon[35037]: 2 slow requests, 0 included below; oldest blocked for > 1801.867450 secs 2024-03-28T18:46:55.432 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:46:55 smithi079 ceph-mon[35312]: pgmap v1241: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:46:55.432 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:46:55 smithi079 ceph-mon[35312]: 2 slow requests, 0 included below; oldest blocked for > 1801.867450 secs 2024-03-28T18:46:55.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:46:55 smithi022 ceph-mon[31985]: pgmap v1241: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:46:55.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:46:55 smithi022 ceph-mon[31985]: 2 slow requests, 0 included below; oldest blocked for > 1801.867450 secs 2024-03-28T18:46:57.425 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:46:57 smithi134 ceph-mon[35037]: pgmap v1242: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:46:57.432 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:46:57 smithi079 ceph-mon[35312]: pgmap v1242: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:46:57.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:46:57 smithi022 ceph-mon[31985]: pgmap v1242: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:46:59.425 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:46:59 smithi134 ceph-mon[35037]: pgmap v1243: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:46:59.432 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:46:59 smithi079 ceph-mon[35312]: pgmap v1243: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:46:59.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:46:59 smithi022 ceph-mon[31985]: pgmap v1243: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:47:00.425 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:47:00 smithi134 ceph-mon[35037]: 2 slow requests, 0 included below; oldest blocked for > 1806.867554 secs 2024-03-28T18:47:00.432 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:47:00 smithi079 ceph-mon[35312]: 2 slow requests, 0 included below; oldest blocked for > 1806.867554 secs 2024-03-28T18:47:00.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:47:00 smithi022 ceph-mon[31985]: 2 slow requests, 0 included below; oldest blocked for > 1806.867554 secs 2024-03-28T18:47:01.425 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:47:01 smithi134 ceph-mon[35037]: pgmap v1244: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:47:01.432 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:47:01 smithi079 ceph-mon[35312]: pgmap v1244: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:47:01.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:47:01 smithi022 ceph-mon[31985]: pgmap v1244: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:47:03.433 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:47:03 smithi079 ceph-mon[35312]: pgmap v1245: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:47:03.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:47:03 smithi022 ceph-mon[31985]: pgmap v1245: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:47:03.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:47:03 smithi134 ceph-mon[35037]: pgmap v1245: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:47:05.433 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:47:05 smithi079 ceph-mon[35312]: pgmap v1246: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:47:05.433 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:47:05 smithi079 ceph-mon[35312]: 2 slow requests, 0 included below; oldest blocked for > 1811.867633 secs 2024-03-28T18:47:05.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:47:05 smithi022 ceph-mon[31985]: pgmap v1246: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:47:05.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:47:05 smithi022 ceph-mon[31985]: 2 slow requests, 0 included below; oldest blocked for > 1811.867633 secs 2024-03-28T18:47:05.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:47:05 smithi134 ceph-mon[35037]: pgmap v1246: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:47:05.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:47:05 smithi134 ceph-mon[35037]: 2 slow requests, 0 included below; oldest blocked for > 1811.867633 secs 2024-03-28T18:47:07.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:47:07 smithi022 ceph-mon[31985]: pgmap v1247: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:47:07.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:47:07 smithi134 ceph-mon[35037]: pgmap v1247: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:47:07.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:47:07 smithi079 ceph-mon[35312]: pgmap v1247: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:47:09.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:47:09 smithi022 ceph-mon[31985]: pgmap v1248: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:47:09.676 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:47:09 smithi134 ceph-mon[35037]: pgmap v1248: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:47:09.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:47:09 smithi079 ceph-mon[35312]: pgmap v1248: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:47:10.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:47:10 smithi022 ceph-mon[31985]: 2 slow requests, 0 included below; oldest blocked for > 1816.867731 secs 2024-03-28T18:47:10.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:47:10 smithi134 ceph-mon[35037]: 2 slow requests, 0 included below; oldest blocked for > 1816.867731 secs 2024-03-28T18:47:10.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:47:10 smithi079 ceph-mon[35312]: 2 slow requests, 0 included below; oldest blocked for > 1816.867731 secs 2024-03-28T18:47:11.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:47:11 smithi022 ceph-mon[31985]: pgmap v1249: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:47:11.577 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:47:11 smithi134 ceph-mon[35037]: pgmap v1249: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:47:11.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:47:11 smithi079 ceph-mon[35312]: pgmap v1249: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:47:13.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:47:13 smithi022 ceph-mon[31985]: pgmap v1250: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:47:13.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:47:13 smithi134 ceph-mon[35037]: pgmap v1250: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:47:13.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:47:13 smithi079 ceph-mon[35312]: pgmap v1250: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:47:15.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:47:15 smithi022 ceph-mon[31985]: pgmap v1251: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:47:15.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:47:15 smithi022 ceph-mon[31985]: 2 slow requests, 0 included below; oldest blocked for > 1821.867810 secs 2024-03-28T18:47:15.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:47:15 smithi134 ceph-mon[35037]: pgmap v1251: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:47:15.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:47:15 smithi134 ceph-mon[35037]: 2 slow requests, 0 included below; oldest blocked for > 1821.867810 secs 2024-03-28T18:47:15.683 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:47:15 smithi079 ceph-mon[35312]: pgmap v1251: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:47:15.683 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:47:15 smithi079 ceph-mon[35312]: 2 slow requests, 0 included below; oldest blocked for > 1821.867810 secs 2024-03-28T18:47:17.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:47:17 smithi022 ceph-mon[31985]: pgmap v1252: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 9 op/s 2024-03-28T18:47:17.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:47:17 smithi022 ceph-mon[31985]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T18:47:17.674 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:47:17 smithi134 ceph-mon[35037]: pgmap v1252: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 9 op/s 2024-03-28T18:47:17.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:47:17 smithi134 ceph-mon[35037]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T18:47:17.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:47:17 smithi079 ceph-mon[35312]: pgmap v1252: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 9 op/s 2024-03-28T18:47:17.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:47:17 smithi079 ceph-mon[35312]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T18:47:18.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:47:18 smithi022 ceph-mon[31985]: from='mgr.15522 ' entity='mgr.x' 2024-03-28T18:47:18.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:47:18 smithi022 ceph-mon[31985]: from='mgr.15522 ' entity='mgr.x' 2024-03-28T18:47:18.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:47:18 smithi022 ceph-mon[31985]: from='mgr.15522 ' entity='mgr.x' 2024-03-28T18:47:18.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:47:18 smithi022 ceph-mon[31985]: from='mgr.15522 ' entity='mgr.x' 2024-03-28T18:47:18.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:47:18 smithi134 ceph-mon[35037]: from='mgr.15522 ' entity='mgr.x' 2024-03-28T18:47:18.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:47:18 smithi134 ceph-mon[35037]: from='mgr.15522 ' entity='mgr.x' 2024-03-28T18:47:18.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:47:18 smithi134 ceph-mon[35037]: from='mgr.15522 ' entity='mgr.x' 2024-03-28T18:47:18.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:47:18 smithi134 ceph-mon[35037]: from='mgr.15522 ' entity='mgr.x' 2024-03-28T18:47:18.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:47:18 smithi079 ceph-mon[35312]: from='mgr.15522 ' entity='mgr.x' 2024-03-28T18:47:18.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:47:18 smithi079 ceph-mon[35312]: from='mgr.15522 ' entity='mgr.x' 2024-03-28T18:47:18.683 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:47:18 smithi079 ceph-mon[35312]: from='mgr.15522 ' entity='mgr.x' 2024-03-28T18:47:18.683 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:47:18 smithi079 ceph-mon[35312]: from='mgr.15522 ' entity='mgr.x' 2024-03-28T18:47:19.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:47:19 smithi134 ceph-mon[35037]: pgmap v1253: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T18:47:19.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:47:19 smithi134 ceph-mon[35037]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T18:47:19.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:47:19 smithi134 ceph-mon[35037]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T18:47:19.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:47:19 smithi134 ceph-mon[35037]: from='mgr.15522 ' entity='mgr.x' 2024-03-28T18:47:19.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:47:19 smithi079 ceph-mon[35312]: pgmap v1253: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T18:47:19.683 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:47:19 smithi079 ceph-mon[35312]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T18:47:19.683 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:47:19 smithi079 ceph-mon[35312]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T18:47:19.683 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:47:19 smithi079 ceph-mon[35312]: from='mgr.15522 ' entity='mgr.x' 2024-03-28T18:47:19.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:47:19 smithi022 ceph-mon[31985]: pgmap v1253: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T18:47:19.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:47:19 smithi022 ceph-mon[31985]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T18:47:19.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:47:19 smithi022 ceph-mon[31985]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T18:47:19.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:47:19 smithi022 ceph-mon[31985]: from='mgr.15522 ' entity='mgr.x' 2024-03-28T18:47:20.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:47:20 smithi134 ceph-mon[35037]: pgmap v1254: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 7 op/s 2024-03-28T18:47:20.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:47:20 smithi134 ceph-mon[35037]: pgmap v1255: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 KiB/s rd, 4 op/s 2024-03-28T18:47:20.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:47:20 smithi134 ceph-mon[35037]: 2 slow requests, 0 included below; oldest blocked for > 1826.867884 secs 2024-03-28T18:47:20.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:47:20 smithi079 ceph-mon[35312]: pgmap v1254: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 7 op/s 2024-03-28T18:47:20.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:47:20 smithi079 ceph-mon[35312]: pgmap v1255: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 KiB/s rd, 4 op/s 2024-03-28T18:47:20.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:47:20 smithi079 ceph-mon[35312]: 2 slow requests, 0 included below; oldest blocked for > 1826.867884 secs 2024-03-28T18:47:20.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:47:20 smithi022 ceph-mon[31985]: pgmap v1254: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 7 op/s 2024-03-28T18:47:20.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:47:20 smithi022 ceph-mon[31985]: pgmap v1255: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 KiB/s rd, 4 op/s 2024-03-28T18:47:20.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:47:20 smithi022 ceph-mon[31985]: 2 slow requests, 0 included below; oldest blocked for > 1826.867884 secs 2024-03-28T18:47:22.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:47:22 smithi134 ceph-mon[35037]: pgmap v1256: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 9 op/s 2024-03-28T18:47:22.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:47:22 smithi079 ceph-mon[35312]: pgmap v1256: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 9 op/s 2024-03-28T18:47:22.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:47:22 smithi022 ceph-mon[31985]: pgmap v1256: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 9 op/s 2024-03-28T18:47:24.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:47:24 smithi134 ceph-mon[35037]: pgmap v1257: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 9 op/s 2024-03-28T18:47:24.683 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:47:24 smithi079 ceph-mon[35312]: pgmap v1257: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 9 op/s 2024-03-28T18:47:24.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:47:24 smithi022 ceph-mon[31985]: pgmap v1257: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 9 op/s 2024-03-28T18:47:25.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:47:25 smithi134 ceph-mon[35037]: 2 slow requests, 0 included below; oldest blocked for > 1831.867964 secs 2024-03-28T18:47:25.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:47:25 smithi079 ceph-mon[35312]: 2 slow requests, 0 included below; oldest blocked for > 1831.867964 secs 2024-03-28T18:47:25.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:47:25 smithi022 ceph-mon[31985]: 2 slow requests, 0 included below; oldest blocked for > 1831.867964 secs 2024-03-28T18:47:26.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:47:26 smithi134 ceph-mon[35037]: pgmap v1258: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-03-28T18:47:26.683 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:47:26 smithi079 ceph-mon[35312]: pgmap v1258: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-03-28T18:47:26.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:47:26 smithi022 ceph-mon[31985]: pgmap v1258: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-03-28T18:47:28.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:47:28 smithi134 ceph-mon[35037]: pgmap v1259: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-03-28T18:47:28.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:47:28 smithi079 ceph-mon[35312]: pgmap v1259: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-03-28T18:47:28.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:47:28 smithi022 ceph-mon[31985]: pgmap v1259: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-03-28T18:47:29.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:47:29 smithi022 ceph-mon[31985]: pgmap v1260: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 7 op/s 2024-03-28T18:47:29.925 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:47:29 smithi134 ceph-mon[35037]: pgmap v1260: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 7 op/s 2024-03-28T18:47:29.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:47:29 smithi079 ceph-mon[35312]: pgmap v1260: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 7 op/s 2024-03-28T18:47:30.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:47:30 smithi022 ceph-mon[31985]: 2 slow requests, 0 included below; oldest blocked for > 1836.868036 secs 2024-03-28T18:47:30.925 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:47:30 smithi134 ceph-mon[35037]: 2 slow requests, 0 included below; oldest blocked for > 1836.868036 secs 2024-03-28T18:47:30.933 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:47:30 smithi079 ceph-mon[35312]: 2 slow requests, 0 included below; oldest blocked for > 1836.868036 secs 2024-03-28T18:47:31.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:47:31 smithi022 ceph-mon[31985]: pgmap v1261: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:47:31.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:47:31 smithi134 ceph-mon[35037]: pgmap v1261: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:47:31.933 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:47:31 smithi079 ceph-mon[35312]: pgmap v1261: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:47:34.175 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:47:33 smithi134 ceph-mon[35037]: pgmap v1262: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:47:34.182 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:47:33 smithi079 ceph-mon[35312]: pgmap v1262: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:47:34.282 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:47:33 smithi022 ceph-mon[31985]: pgmap v1262: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:47:35.175 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:47:34 smithi134 ceph-mon[35037]: 2 slow requests, 0 included below; oldest blocked for > 1841.868102 secs 2024-03-28T18:47:35.182 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:47:34 smithi079 ceph-mon[35312]: 2 slow requests, 0 included below; oldest blocked for > 1841.868102 secs 2024-03-28T18:47:35.282 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:47:34 smithi022 ceph-mon[31985]: 2 slow requests, 0 included below; oldest blocked for > 1841.868102 secs 2024-03-28T18:47:36.175 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:47:35 smithi134 ceph-mon[35037]: pgmap v1263: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:47:36.182 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:47:35 smithi079 ceph-mon[35312]: pgmap v1263: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:47:36.282 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:47:35 smithi022 ceph-mon[31985]: pgmap v1263: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:47:38.175 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:47:37 smithi134 ceph-mon[35037]: pgmap v1264: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:47:38.182 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:47:37 smithi079 ceph-mon[35312]: pgmap v1264: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:47:38.282 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:47:37 smithi022 ceph-mon[31985]: pgmap v1264: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:47:40.175 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:47:39 smithi134 ceph-mon[35037]: pgmap v1265: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:47:40.175 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:47:39 smithi134 ceph-mon[35037]: 2 slow requests, 0 included below; oldest blocked for > 1846.868169 secs 2024-03-28T18:47:40.183 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:47:39 smithi079 ceph-mon[35312]: pgmap v1265: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:47:40.183 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:47:39 smithi079 ceph-mon[35312]: 2 slow requests, 0 included below; oldest blocked for > 1846.868169 secs 2024-03-28T18:47:40.282 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:47:39 smithi022 ceph-mon[31985]: pgmap v1265: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:47:40.282 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:47:39 smithi022 ceph-mon[31985]: 2 slow requests, 0 included below; oldest blocked for > 1846.868169 secs 2024-03-28T18:47:42.175 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:47:41 smithi134 ceph-mon[35037]: pgmap v1266: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:47:42.182 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:47:41 smithi079 ceph-mon[35312]: pgmap v1266: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:47:42.282 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:47:41 smithi022 ceph-mon[31985]: pgmap v1266: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:47:44.174 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:47:43 smithi134 ceph-mon[35037]: pgmap v1267: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:47:44.182 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:47:43 smithi079 ceph-mon[35312]: pgmap v1267: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:47:44.282 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:47:43 smithi022 ceph-mon[31985]: pgmap v1267: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:47:45.175 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:47:44 smithi134 ceph-mon[35037]: 2 slow requests, 0 included below; oldest blocked for > 1851.868243 secs 2024-03-28T18:47:45.182 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:47:44 smithi079 ceph-mon[35312]: 2 slow requests, 0 included below; oldest blocked for > 1851.868243 secs 2024-03-28T18:47:45.282 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:47:44 smithi022 ceph-mon[31985]: 2 slow requests, 0 included below; oldest blocked for > 1851.868243 secs 2024-03-28T18:47:46.175 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:47:45 smithi134 ceph-mon[35037]: pgmap v1268: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:47:46.182 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:47:45 smithi079 ceph-mon[35312]: pgmap v1268: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:47:46.282 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:47:45 smithi022 ceph-mon[31985]: pgmap v1268: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:47:48.175 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:47:47 smithi134 ceph-mon[35037]: pgmap v1269: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:47:48.182 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:47:47 smithi079 ceph-mon[35312]: pgmap v1269: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:47:48.282 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:47:47 smithi022 ceph-mon[31985]: pgmap v1269: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:47:50.424 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:47:50 smithi134 ceph-mon[35037]: pgmap v1270: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:47:50.425 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:47:50 smithi134 ceph-mon[35037]: 2 slow requests, 0 included below; oldest blocked for > 1856.868313 secs 2024-03-28T18:47:50.432 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:47:50 smithi079 ceph-mon[35312]: pgmap v1270: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:47:50.432 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:47:50 smithi079 ceph-mon[35312]: 2 slow requests, 0 included below; oldest blocked for > 1856.868313 secs 2024-03-28T18:47:50.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:47:50 smithi022 ceph-mon[31985]: pgmap v1270: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:47:50.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:47:50 smithi022 ceph-mon[31985]: 2 slow requests, 0 included below; oldest blocked for > 1856.868313 secs 2024-03-28T18:47:52.425 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:47:52 smithi134 ceph-mon[35037]: pgmap v1271: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:47:52.432 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:47:52 smithi079 ceph-mon[35312]: pgmap v1271: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:47:52.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:47:52 smithi022 ceph-mon[31985]: pgmap v1271: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:47:54.425 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:47:54 smithi134 ceph-mon[35037]: pgmap v1272: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:47:54.432 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:47:54 smithi079 ceph-mon[35312]: pgmap v1272: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:47:54.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:47:54 smithi022 ceph-mon[31985]: pgmap v1272: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:47:55.425 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:47:55 smithi134 ceph-mon[35037]: 2 slow requests, 0 included below; oldest blocked for > 1861.868396 secs 2024-03-28T18:47:55.455 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:47:55 smithi079 ceph-mon[35312]: 2 slow requests, 0 included below; oldest blocked for > 1861.868396 secs 2024-03-28T18:47:55.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:47:55 smithi022 ceph-mon[31985]: 2 slow requests, 0 included below; oldest blocked for > 1861.868396 secs 2024-03-28T18:47:56.425 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:47:56 smithi134 ceph-mon[35037]: pgmap v1273: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:47:56.432 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:47:56 smithi079 ceph-mon[35312]: pgmap v1273: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:47:56.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:47:56 smithi022 ceph-mon[31985]: pgmap v1273: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:47:58.425 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:47:58 smithi134 ceph-mon[35037]: pgmap v1274: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:47:58.432 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:47:58 smithi079 ceph-mon[35312]: pgmap v1274: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:47:58.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:47:58 smithi022 ceph-mon[31985]: pgmap v1274: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:48:00.425 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:48:00 smithi134 ceph-mon[35037]: pgmap v1275: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:48:00.425 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:48:00 smithi134 ceph-mon[35037]: 2 slow requests, 0 included below; oldest blocked for > 1866.868461 secs 2024-03-28T18:48:00.432 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:48:00 smithi079 ceph-mon[35312]: pgmap v1275: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:48:00.432 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:48:00 smithi079 ceph-mon[35312]: 2 slow requests, 0 included below; oldest blocked for > 1866.868461 secs 2024-03-28T18:48:00.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:48:00 smithi022 ceph-mon[31985]: pgmap v1275: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:48:00.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:48:00 smithi022 ceph-mon[31985]: 2 slow requests, 0 included below; oldest blocked for > 1866.868461 secs 2024-03-28T18:48:02.433 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:48:02 smithi079 ceph-mon[35312]: pgmap v1276: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:48:02.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:48:02 smithi022 ceph-mon[31985]: pgmap v1276: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:48:02.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:48:02 smithi134 ceph-mon[35037]: pgmap v1276: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:48:04.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:48:04 smithi022 ceph-mon[31985]: pgmap v1277: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:48:04.674 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:48:04 smithi134 ceph-mon[35037]: pgmap v1277: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:48:04.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:48:04 smithi079 ceph-mon[35312]: pgmap v1277: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:48:05.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:48:05 smithi022 ceph-mon[31985]: 2 slow requests, 0 included below; oldest blocked for > 1871.868549 secs 2024-03-28T18:48:05.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:48:05 smithi134 ceph-mon[35037]: 2 slow requests, 0 included below; oldest blocked for > 1871.868549 secs 2024-03-28T18:48:05.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:48:05 smithi079 ceph-mon[35312]: 2 slow requests, 0 included below; oldest blocked for > 1871.868549 secs 2024-03-28T18:48:06.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:48:06 smithi022 ceph-mon[31985]: pgmap v1278: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:48:06.674 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:48:06 smithi134 ceph-mon[35037]: pgmap v1278: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:48:06.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:48:06 smithi079 ceph-mon[35312]: pgmap v1278: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:48:08.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:48:08 smithi022 ceph-mon[31985]: pgmap v1279: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:48:08.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:48:08 smithi134 ceph-mon[35037]: pgmap v1279: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:48:08.683 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:48:08 smithi079 ceph-mon[35312]: pgmap v1279: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:48:10.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:48:10 smithi022 ceph-mon[31985]: pgmap v1280: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:48:10.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:48:10 smithi022 ceph-mon[31985]: 2 slow requests, 0 included below; oldest blocked for > 1876.868644 secs 2024-03-28T18:48:10.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:48:10 smithi134 ceph-mon[35037]: pgmap v1280: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:48:10.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:48:10 smithi134 ceph-mon[35037]: 2 slow requests, 0 included below; oldest blocked for > 1876.868644 secs 2024-03-28T18:48:10.683 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:48:10 smithi079 ceph-mon[35312]: pgmap v1280: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:48:10.683 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:48:10 smithi079 ceph-mon[35312]: 2 slow requests, 0 included below; oldest blocked for > 1876.868644 secs 2024-03-28T18:48:12.478 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:48:12 smithi134 ceph-mon[35037]: pgmap v1281: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:48:12.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:48:12 smithi022 ceph-mon[31985]: pgmap v1281: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:48:12.683 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:48:12 smithi079 ceph-mon[35312]: pgmap v1281: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:48:14.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:48:14 smithi022 ceph-mon[31985]: pgmap v1282: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:48:14.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:48:14 smithi134 ceph-mon[35037]: pgmap v1282: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:48:14.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:48:14 smithi079 ceph-mon[35312]: pgmap v1282: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:48:15.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:48:15 smithi022 ceph-mon[31985]: 2 slow requests, 0 included below; oldest blocked for > 1881.868712 secs 2024-03-28T18:48:15.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:48:15 smithi134 ceph-mon[35037]: 2 slow requests, 0 included below; oldest blocked for > 1881.868712 secs 2024-03-28T18:48:15.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:48:15 smithi079 ceph-mon[35312]: 2 slow requests, 0 included below; oldest blocked for > 1881.868712 secs 2024-03-28T18:48:16.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:48:16 smithi022 ceph-mon[31985]: pgmap v1283: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 9 op/s 2024-03-28T18:48:16.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:48:16 smithi134 ceph-mon[35037]: pgmap v1283: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 9 op/s 2024-03-28T18:48:16.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:48:16 smithi079 ceph-mon[35312]: pgmap v1283: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 9 op/s 2024-03-28T18:48:18.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:48:18 smithi022 ceph-mon[31985]: pgmap v1284: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T18:48:18.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:48:18 smithi134 ceph-mon[35037]: pgmap v1284: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T18:48:18.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:48:18 smithi079 ceph-mon[35312]: pgmap v1284: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T18:48:20.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:48:20 smithi022 ceph-mon[31985]: pgmap v1285: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T18:48:20.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:48:20 smithi022 ceph-mon[31985]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T18:48:20.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:48:20 smithi022 ceph-mon[31985]: 2 slow requests, 0 included below; oldest blocked for > 1886.868789 secs 2024-03-28T18:48:20.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:48:20 smithi134 ceph-mon[35037]: pgmap v1285: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T18:48:20.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:48:20 smithi134 ceph-mon[35037]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T18:48:20.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:48:20 smithi134 ceph-mon[35037]: 2 slow requests, 0 included below; oldest blocked for > 1886.868789 secs 2024-03-28T18:48:20.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:48:20 smithi079 ceph-mon[35312]: pgmap v1285: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T18:48:20.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:48:20 smithi079 ceph-mon[35312]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T18:48:20.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:48:20 smithi079 ceph-mon[35312]: 2 slow requests, 0 included below; oldest blocked for > 1886.868789 secs 2024-03-28T18:48:21.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:48:21 smithi022 ceph-mon[31985]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T18:48:21.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:48:21 smithi022 ceph-mon[31985]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T18:48:21.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:48:21 smithi022 ceph-mon[31985]: from='mgr.15522 ' entity='mgr.x' 2024-03-28T18:48:21.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:48:21 smithi134 ceph-mon[35037]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T18:48:21.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:48:21 smithi134 ceph-mon[35037]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T18:48:21.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:48:21 smithi134 ceph-mon[35037]: from='mgr.15522 ' entity='mgr.x' 2024-03-28T18:48:21.683 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:48:21 smithi079 ceph-mon[35312]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T18:48:21.683 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:48:21 smithi079 ceph-mon[35312]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T18:48:21.683 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:48:21 smithi079 ceph-mon[35312]: from='mgr.15522 ' entity='mgr.x' 2024-03-28T18:48:22.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:48:22 smithi022 ceph-mon[31985]: pgmap v1286: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 10 op/s 2024-03-28T18:48:22.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:48:22 smithi134 ceph-mon[35037]: pgmap v1286: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 10 op/s 2024-03-28T18:48:22.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:48:22 smithi079 ceph-mon[35312]: pgmap v1286: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 10 op/s 2024-03-28T18:48:24.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:48:24 smithi022 ceph-mon[31985]: pgmap v1287: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:48:24.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:48:24 smithi134 ceph-mon[35037]: pgmap v1287: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:48:24.683 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:48:24 smithi079 ceph-mon[35312]: pgmap v1287: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:48:25.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:48:25 smithi022 ceph-mon[31985]: 2 slow requests, 0 included below; oldest blocked for > 1891.868933 secs 2024-03-28T18:48:25.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:48:25 smithi134 ceph-mon[35037]: 2 slow requests, 0 included below; oldest blocked for > 1891.868933 secs 2024-03-28T18:48:25.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:48:25 smithi079 ceph-mon[35312]: 2 slow requests, 0 included below; oldest blocked for > 1891.868933 secs 2024-03-28T18:48:26.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:48:26 smithi134 ceph-mon[35037]: pgmap v1288: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 10 op/s 2024-03-28T18:48:26.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:48:26 smithi079 ceph-mon[35312]: pgmap v1288: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 10 op/s 2024-03-28T18:48:26.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:48:26 smithi022 ceph-mon[31985]: pgmap v1288: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 10 op/s 2024-03-28T18:48:28.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:48:28 smithi134 ceph-mon[35037]: pgmap v1289: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 6 op/s 2024-03-28T18:48:28.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:48:28 smithi079 ceph-mon[35312]: pgmap v1289: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 6 op/s 2024-03-28T18:48:28.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:48:28 smithi022 ceph-mon[31985]: pgmap v1289: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 6 op/s 2024-03-28T18:48:30.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:48:30 smithi134 ceph-mon[35037]: pgmap v1290: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 6 op/s 2024-03-28T18:48:30.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:48:30 smithi134 ceph-mon[35037]: 2 slow requests, 0 included below; oldest blocked for > 1896.869008 secs 2024-03-28T18:48:30.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:48:30 smithi079 ceph-mon[35312]: pgmap v1290: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 6 op/s 2024-03-28T18:48:30.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:48:30 smithi079 ceph-mon[35312]: 2 slow requests, 0 included below; oldest blocked for > 1896.869008 secs 2024-03-28T18:48:30.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:48:30 smithi022 ceph-mon[31985]: pgmap v1290: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 6 op/s 2024-03-28T18:48:30.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:48:30 smithi022 ceph-mon[31985]: 2 slow requests, 0 included below; oldest blocked for > 1896.869008 secs 2024-03-28T18:48:32.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:48:32 smithi134 ceph-mon[35037]: pgmap v1291: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 10 op/s 2024-03-28T18:48:32.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:48:32 smithi079 ceph-mon[35312]: pgmap v1291: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 10 op/s 2024-03-28T18:48:32.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:48:32 smithi022 ceph-mon[31985]: pgmap v1291: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 10 op/s 2024-03-28T18:48:34.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:48:34 smithi134 ceph-mon[35037]: pgmap v1292: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T18:48:34.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:48:34 smithi079 ceph-mon[35312]: pgmap v1292: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T18:48:34.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:48:34 smithi022 ceph-mon[31985]: pgmap v1292: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T18:48:35.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:48:35 smithi134 ceph-mon[35037]: 2 slow requests, 0 included below; oldest blocked for > 1901.869129 secs 2024-03-28T18:48:35.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:48:35 smithi079 ceph-mon[35312]: 2 slow requests, 0 included below; oldest blocked for > 1901.869129 secs 2024-03-28T18:48:35.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:48:35 smithi022 ceph-mon[31985]: 2 slow requests, 0 included below; oldest blocked for > 1901.869129 secs 2024-03-28T18:48:36.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:48:36 smithi134 ceph-mon[35037]: pgmap v1293: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 9 op/s 2024-03-28T18:48:36.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:48:36 smithi079 ceph-mon[35312]: pgmap v1293: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 9 op/s 2024-03-28T18:48:36.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:48:36 smithi022 ceph-mon[31985]: pgmap v1293: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 9 op/s 2024-03-28T18:48:38.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:48:38 smithi134 ceph-mon[35037]: pgmap v1294: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T18:48:38.683 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:48:38 smithi079 ceph-mon[35312]: pgmap v1294: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T18:48:38.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:48:38 smithi022 ceph-mon[31985]: pgmap v1294: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T18:48:40.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:48:40 smithi134 ceph-mon[35037]: pgmap v1295: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T18:48:40.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:48:40 smithi134 ceph-mon[35037]: 2 slow requests, 0 included below; oldest blocked for > 1906.869197 secs 2024-03-28T18:48:40.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:48:40 smithi079 ceph-mon[35312]: pgmap v1295: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T18:48:40.683 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:48:40 smithi079 ceph-mon[35312]: 2 slow requests, 0 included below; oldest blocked for > 1906.869197 secs 2024-03-28T18:48:40.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:48:40 smithi022 ceph-mon[31985]: pgmap v1295: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T18:48:40.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:48:40 smithi022 ceph-mon[31985]: 2 slow requests, 0 included below; oldest blocked for > 1906.869197 secs 2024-03-28T18:48:42.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:48:42 smithi134 ceph-mon[35037]: pgmap v1296: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 9 op/s 2024-03-28T18:48:42.683 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:48:42 smithi079 ceph-mon[35312]: pgmap v1296: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 9 op/s 2024-03-28T18:48:42.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:48:42 smithi022 ceph-mon[31985]: pgmap v1296: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 9 op/s 2024-03-28T18:48:44.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:48:44 smithi134 ceph-mon[35037]: pgmap v1297: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:48:44.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:48:44 smithi079 ceph-mon[35312]: pgmap v1297: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:48:44.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:48:44 smithi022 ceph-mon[31985]: pgmap v1297: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:48:45.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:48:45 smithi134 ceph-mon[35037]: 2 slow requests, 0 included below; oldest blocked for > 1911.869317 secs 2024-03-28T18:48:45.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:48:45 smithi079 ceph-mon[35312]: 2 slow requests, 0 included below; oldest blocked for > 1911.869317 secs 2024-03-28T18:48:45.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:48:45 smithi022 ceph-mon[31985]: 2 slow requests, 0 included below; oldest blocked for > 1911.869317 secs 2024-03-28T18:48:46.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:48:46 smithi134 ceph-mon[35037]: pgmap v1298: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:48:46.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:48:46 smithi079 ceph-mon[35312]: pgmap v1298: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:48:46.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:48:46 smithi022 ceph-mon[31985]: pgmap v1298: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:48:48.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:48:48 smithi134 ceph-mon[35037]: pgmap v1299: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:48:48.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:48:48 smithi079 ceph-mon[35312]: pgmap v1299: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:48:48.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:48:48 smithi022 ceph-mon[31985]: pgmap v1299: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:48:50.674 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:48:50 smithi134 ceph-mon[35037]: pgmap v1300: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:48:50.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:48:50 smithi134 ceph-mon[35037]: 2 slow requests, 0 included below; oldest blocked for > 1916.869440 secs 2024-03-28T18:48:50.683 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:48:50 smithi079 ceph-mon[35312]: pgmap v1300: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:48:50.683 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:48:50 smithi079 ceph-mon[35312]: 2 slow requests, 0 included below; oldest blocked for > 1916.869440 secs 2024-03-28T18:48:50.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:48:50 smithi022 ceph-mon[31985]: pgmap v1300: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:48:50.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:48:50 smithi022 ceph-mon[31985]: 2 slow requests, 0 included below; oldest blocked for > 1916.869440 secs 2024-03-28T18:48:52.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:48:52 smithi134 ceph-mon[35037]: pgmap v1301: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:48:52.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:48:52 smithi079 ceph-mon[35312]: pgmap v1301: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:48:52.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:48:52 smithi022 ceph-mon[31985]: pgmap v1301: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:48:54.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:48:54 smithi134 ceph-mon[35037]: pgmap v1302: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:48:54.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:48:54 smithi079 ceph-mon[35312]: pgmap v1302: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:48:54.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:48:54 smithi022 ceph-mon[31985]: pgmap v1302: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:48:55.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:48:55 smithi134 ceph-mon[35037]: 2 slow requests, 0 included below; oldest blocked for > 1921.869505 secs 2024-03-28T18:48:55.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:48:55 smithi079 ceph-mon[35312]: 2 slow requests, 0 included below; oldest blocked for > 1921.869505 secs 2024-03-28T18:48:55.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:48:55 smithi022 ceph-mon[31985]: 2 slow requests, 0 included below; oldest blocked for > 1921.869505 secs 2024-03-28T18:48:56.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:48:56 smithi134 ceph-mon[35037]: pgmap v1303: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:48:56.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:48:56 smithi079 ceph-mon[35312]: pgmap v1303: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:48:56.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:48:56 smithi022 ceph-mon[31985]: pgmap v1303: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:48:57.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:48:57 smithi022 ceph-mon[31985]: pgmap v1304: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:48:57.925 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:48:57 smithi134 ceph-mon[35037]: pgmap v1304: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:48:57.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:48:57 smithi079 ceph-mon[35312]: pgmap v1304: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:48:59.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:48:59 smithi079 ceph-mon[35312]: pgmap v1305: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:48:59.933 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:48:59 smithi079 ceph-mon[35312]: 2 slow requests, 0 included below; oldest blocked for > 1926.869621 secs 2024-03-28T18:49:00.032 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:48:59 smithi022 ceph-mon[31985]: pgmap v1305: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:49:00.032 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:48:59 smithi022 ceph-mon[31985]: 2 slow requests, 0 included below; oldest blocked for > 1926.869621 secs 2024-03-28T18:49:00.174 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:48:59 smithi134 ceph-mon[35037]: pgmap v1305: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:49:00.175 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:48:59 smithi134 ceph-mon[35037]: 2 slow requests, 0 included below; oldest blocked for > 1926.869621 secs 2024-03-28T18:49:02.032 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:49:01 smithi022 ceph-mon[31985]: pgmap v1306: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:49:02.175 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:49:01 smithi134 ceph-mon[35037]: pgmap v1306: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:49:02.182 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:49:01 smithi079 ceph-mon[35312]: pgmap v1306: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:49:04.032 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:49:03 smithi022 ceph-mon[31985]: pgmap v1307: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:49:04.175 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:49:03 smithi134 ceph-mon[35037]: pgmap v1307: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:49:04.182 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:49:03 smithi079 ceph-mon[35312]: pgmap v1307: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:49:05.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:49:05 smithi022 ceph-mon[31985]: 2 slow requests, 0 included below; oldest blocked for > 1931.869686 secs 2024-03-28T18:49:05.676 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:49:05 smithi134 ceph-mon[35037]: 2 slow requests, 0 included below; oldest blocked for > 1931.869686 secs 2024-03-28T18:49:05.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:49:05 smithi079 ceph-mon[35312]: 2 slow requests, 0 included below; oldest blocked for > 1931.869686 secs 2024-03-28T18:49:06.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:49:06 smithi022 ceph-mon[31985]: pgmap v1308: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:49:06.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:49:06 smithi134 ceph-mon[35037]: pgmap v1308: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:49:06.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:49:06 smithi079 ceph-mon[35312]: pgmap v1308: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:49:08.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:49:08 smithi022 ceph-mon[31985]: pgmap v1309: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:49:08.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:49:08 smithi134 ceph-mon[35037]: pgmap v1309: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:49:08.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:49:08 smithi079 ceph-mon[35312]: pgmap v1309: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:49:10.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:49:10 smithi022 ceph-mon[31985]: pgmap v1310: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:49:10.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:49:10 smithi022 ceph-mon[31985]: 2 slow requests, 0 included below; oldest blocked for > 1936.869792 secs 2024-03-28T18:49:10.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:49:10 smithi134 ceph-mon[35037]: pgmap v1310: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:49:10.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:49:10 smithi134 ceph-mon[35037]: 2 slow requests, 0 included below; oldest blocked for > 1936.869792 secs 2024-03-28T18:49:10.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:49:10 smithi079 ceph-mon[35312]: pgmap v1310: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:49:10.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:49:10 smithi079 ceph-mon[35312]: 2 slow requests, 0 included below; oldest blocked for > 1936.869792 secs 2024-03-28T18:49:12.477 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:49:12 smithi134 ceph-mon[35037]: pgmap v1311: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:49:12.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:49:12 smithi022 ceph-mon[31985]: pgmap v1311: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:49:12.683 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:49:12 smithi079 ceph-mon[35312]: pgmap v1311: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:49:14.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:49:14 smithi022 ceph-mon[31985]: pgmap v1312: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:49:14.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:49:14 smithi134 ceph-mon[35037]: pgmap v1312: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:49:14.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:49:14 smithi079 ceph-mon[35312]: pgmap v1312: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:49:15.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:49:15 smithi022 ceph-mon[31985]: 2 slow requests, 0 included below; oldest blocked for > 1941.869870 secs 2024-03-28T18:49:15.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:49:15 smithi134 ceph-mon[35037]: 2 slow requests, 0 included below; oldest blocked for > 1941.869870 secs 2024-03-28T18:49:15.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:49:15 smithi079 ceph-mon[35312]: 2 slow requests, 0 included below; oldest blocked for > 1941.869870 secs 2024-03-28T18:49:16.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:49:16 smithi022 ceph-mon[31985]: pgmap v1313: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:49:16.674 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:49:16 smithi134 ceph-mon[35037]: pgmap v1313: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:49:16.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:49:16 smithi079 ceph-mon[35312]: pgmap v1313: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:49:18.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:49:18 smithi022 ceph-mon[31985]: pgmap v1314: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:49:18.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:49:18 smithi134 ceph-mon[35037]: pgmap v1314: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:49:18.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:49:18 smithi079 ceph-mon[35312]: pgmap v1314: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:49:20.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:49:20 smithi022 ceph-mon[31985]: pgmap v1315: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:49:20.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:49:20 smithi022 ceph-mon[31985]: 2 slow requests, 0 included below; oldest blocked for > 1946.869932 secs 2024-03-28T18:49:20.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:49:20 smithi134 ceph-mon[35037]: pgmap v1315: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:49:20.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:49:20 smithi134 ceph-mon[35037]: 2 slow requests, 0 included below; oldest blocked for > 1946.869932 secs 2024-03-28T18:49:20.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:49:20 smithi079 ceph-mon[35312]: pgmap v1315: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:49:20.683 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:49:20 smithi079 ceph-mon[35312]: 2 slow requests, 0 included below; oldest blocked for > 1946.869932 secs 2024-03-28T18:49:21.531 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:49:21 smithi022 ceph-mon[31985]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T18:49:21.674 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:49:21 smithi134 ceph-mon[35037]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T18:49:21.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:49:21 smithi079 ceph-mon[35312]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T18:49:22.529 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:49:22 smithi022 ceph-mon[31985]: pgmap v1316: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:49:22.529 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:49:22 smithi022 ceph-mon[31985]: from='mgr.15522 ' entity='mgr.x' 2024-03-28T18:49:22.529 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:49:22 smithi022 ceph-mon[31985]: from='mgr.15522 ' entity='mgr.x' 2024-03-28T18:49:22.565 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:49:22 smithi079 ceph-mon[35312]: pgmap v1316: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:49:22.565 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:49:22 smithi079 ceph-mon[35312]: from='mgr.15522 ' entity='mgr.x' 2024-03-28T18:49:22.565 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:49:22 smithi079 ceph-mon[35312]: from='mgr.15522 ' entity='mgr.x' 2024-03-28T18:49:22.674 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:49:22 smithi134 ceph-mon[35037]: pgmap v1316: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:49:22.674 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:49:22 smithi134 ceph-mon[35037]: from='mgr.15522 ' entity='mgr.x' 2024-03-28T18:49:22.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:49:22 smithi134 ceph-mon[35037]: from='mgr.15522 ' entity='mgr.x' 2024-03-28T18:49:24.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:49:24 smithi022 ceph-mon[31985]: pgmap v1317: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:49:24.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:49:24 smithi022 ceph-mon[31985]: from='mgr.15522 ' entity='mgr.x' 2024-03-28T18:49:24.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:49:24 smithi022 ceph-mon[31985]: from='mgr.15522 ' entity='mgr.x' 2024-03-28T18:49:24.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:49:24 smithi134 ceph-mon[35037]: pgmap v1317: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:49:24.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:49:24 smithi134 ceph-mon[35037]: from='mgr.15522 ' entity='mgr.x' 2024-03-28T18:49:24.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:49:24 smithi134 ceph-mon[35037]: from='mgr.15522 ' entity='mgr.x' 2024-03-28T18:49:24.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:49:24 smithi079 ceph-mon[35312]: pgmap v1317: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:49:24.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:49:24 smithi079 ceph-mon[35312]: from='mgr.15522 ' entity='mgr.x' 2024-03-28T18:49:24.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:49:24 smithi079 ceph-mon[35312]: from='mgr.15522 ' entity='mgr.x' 2024-03-28T18:49:25.531 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:49:25 smithi022 ceph-mon[31985]: 2 slow requests, 0 included below; oldest blocked for > 1951.870001 secs 2024-03-28T18:49:25.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:49:25 smithi022 ceph-mon[31985]: from='mgr.15522 ' entity='mgr.x' 2024-03-28T18:49:25.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:49:25 smithi022 ceph-mon[31985]: from='mgr.15522 ' entity='mgr.x' 2024-03-28T18:49:25.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:49:25 smithi022 ceph-mon[31985]: from='mgr.15522 ' entity='mgr.x' 2024-03-28T18:49:25.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:49:25 smithi022 ceph-mon[31985]: from='mgr.15522 ' entity='mgr.x' 2024-03-28T18:49:25.557 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:49:25 smithi079 ceph-mon[35312]: 2 slow requests, 0 included below; oldest blocked for > 1951.870001 secs 2024-03-28T18:49:25.557 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:49:25 smithi079 ceph-mon[35312]: from='mgr.15522 ' entity='mgr.x' 2024-03-28T18:49:25.557 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:49:25 smithi079 ceph-mon[35312]: from='mgr.15522 ' entity='mgr.x' 2024-03-28T18:49:25.557 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:49:25 smithi079 ceph-mon[35312]: from='mgr.15522 ' entity='mgr.x' 2024-03-28T18:49:25.557 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:49:25 smithi079 ceph-mon[35312]: from='mgr.15522 ' entity='mgr.x' 2024-03-28T18:49:25.674 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:49:25 smithi134 ceph-mon[35037]: 2 slow requests, 0 included below; oldest blocked for > 1951.870001 secs 2024-03-28T18:49:25.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:49:25 smithi134 ceph-mon[35037]: from='mgr.15522 ' entity='mgr.x' 2024-03-28T18:49:25.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:49:25 smithi134 ceph-mon[35037]: from='mgr.15522 ' entity='mgr.x' 2024-03-28T18:49:25.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:49:25 smithi134 ceph-mon[35037]: from='mgr.15522 ' entity='mgr.x' 2024-03-28T18:49:25.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:49:25 smithi134 ceph-mon[35037]: from='mgr.15522 ' entity='mgr.x' 2024-03-28T18:49:26.534 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:49:26 smithi022 ceph-mon[31985]: pgmap v1318: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:49:26.534 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:49:26 smithi022 ceph-mon[31985]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-28T18:49:26.534 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:49:26 smithi022 ceph-mon[31985]: from='mgr.15522 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-28T18:49:26.535 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:49:26 smithi022 ceph-mon[31985]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-28T18:49:26.535 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:49:26 smithi022 ceph-mon[31985]: from='mgr.15522 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-28T18:49:26.535 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:49:26 smithi022 ceph-mon[31985]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-03-28T18:49:26.535 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:49:26 smithi022 ceph-mon[31985]: from='mgr.15522 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-03-28T18:49:26.535 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:49:26 smithi022 ceph-mon[31985]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-03-28T18:49:26.535 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:49:26 smithi022 ceph-mon[31985]: from='mgr.15522 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-03-28T18:49:26.535 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:49:26 smithi022 ceph-mon[31985]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-28T18:49:26.535 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:49:26 smithi022 ceph-mon[31985]: from='mgr.15522 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-28T18:49:26.535 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:49:26 smithi022 ceph-mon[31985]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-28T18:49:26.535 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:49:26 smithi022 ceph-mon[31985]: from='mgr.15522 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-28T18:49:26.535 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:49:26 smithi022 ceph-mon[31985]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-03-28T18:49:26.535 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:49:26 smithi022 ceph-mon[31985]: from='mgr.15522 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-03-28T18:49:26.536 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:49:26 smithi022 ceph-mon[31985]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-03-28T18:49:26.536 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:49:26 smithi022 ceph-mon[31985]: from='mgr.15522 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-03-28T18:49:26.536 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:49:26 smithi022 ceph-mon[31985]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-28T18:49:26.536 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:49:26 smithi022 ceph-mon[31985]: from='mgr.15522 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-28T18:49:26.536 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:49:26 smithi022 ceph-mon[31985]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-28T18:49:26.536 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:49:26 smithi022 ceph-mon[31985]: from='mgr.15522 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-28T18:49:26.536 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:49:26 smithi022 ceph-mon[31985]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-03-28T18:49:26.536 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:49:26 smithi022 ceph-mon[31985]: from='mgr.15522 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-03-28T18:49:26.536 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:49:26 smithi022 ceph-mon[31985]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-03-28T18:49:26.536 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:49:26 smithi022 ceph-mon[31985]: from='mgr.15522 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-03-28T18:49:26.536 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:49:26 smithi022 ceph-mon[31985]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T18:49:26.536 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:49:26 smithi022 ceph-mon[31985]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T18:49:26.536 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:49:26 smithi022 ceph-mon[31985]: from='mgr.15522 ' entity='mgr.x' 2024-03-28T18:49:26.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:49:26 smithi134 ceph-mon[35037]: pgmap v1318: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:49:26.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:49:26 smithi134 ceph-mon[35037]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-28T18:49:26.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:49:26 smithi134 ceph-mon[35037]: from='mgr.15522 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-28T18:49:26.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:49:26 smithi134 ceph-mon[35037]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-28T18:49:26.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:49:26 smithi134 ceph-mon[35037]: from='mgr.15522 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-28T18:49:26.676 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:49:26 smithi134 ceph-mon[35037]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-03-28T18:49:26.676 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:49:26 smithi134 ceph-mon[35037]: from='mgr.15522 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-03-28T18:49:26.676 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:49:26 smithi134 ceph-mon[35037]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-03-28T18:49:26.676 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:49:26 smithi134 ceph-mon[35037]: from='mgr.15522 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-03-28T18:49:26.676 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:49:26 smithi134 ceph-mon[35037]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-28T18:49:26.676 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:49:26 smithi134 ceph-mon[35037]: from='mgr.15522 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-28T18:49:26.676 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:49:26 smithi134 ceph-mon[35037]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-28T18:49:26.676 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:49:26 smithi134 ceph-mon[35037]: from='mgr.15522 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-28T18:49:26.676 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:49:26 smithi134 ceph-mon[35037]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-03-28T18:49:26.676 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:49:26 smithi134 ceph-mon[35037]: from='mgr.15522 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-03-28T18:49:26.676 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:49:26 smithi134 ceph-mon[35037]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-03-28T18:49:26.677 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:49:26 smithi134 ceph-mon[35037]: from='mgr.15522 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-03-28T18:49:26.677 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:49:26 smithi134 ceph-mon[35037]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-28T18:49:26.677 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:49:26 smithi134 ceph-mon[35037]: from='mgr.15522 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-28T18:49:26.677 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:49:26 smithi134 ceph-mon[35037]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-28T18:49:26.677 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:49:26 smithi134 ceph-mon[35037]: from='mgr.15522 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-28T18:49:26.677 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:49:26 smithi134 ceph-mon[35037]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-03-28T18:49:26.677 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:49:26 smithi134 ceph-mon[35037]: from='mgr.15522 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-03-28T18:49:26.677 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:49:26 smithi134 ceph-mon[35037]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-03-28T18:49:26.677 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:49:26 smithi134 ceph-mon[35037]: from='mgr.15522 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-03-28T18:49:26.677 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:49:26 smithi134 ceph-mon[35037]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T18:49:26.677 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:49:26 smithi134 ceph-mon[35037]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T18:49:26.677 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:49:26 smithi134 ceph-mon[35037]: from='mgr.15522 ' entity='mgr.x' 2024-03-28T18:49:26.684 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:49:26 smithi079 ceph-mon[35312]: pgmap v1318: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:49:26.684 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:49:26 smithi079 ceph-mon[35312]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-28T18:49:26.684 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:49:26 smithi079 ceph-mon[35312]: from='mgr.15522 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-28T18:49:26.684 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:49:26 smithi079 ceph-mon[35312]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-28T18:49:26.684 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:49:26 smithi079 ceph-mon[35312]: from='mgr.15522 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-28T18:49:26.685 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:49:26 smithi079 ceph-mon[35312]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-03-28T18:49:26.685 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:49:26 smithi079 ceph-mon[35312]: from='mgr.15522 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-03-28T18:49:26.685 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:49:26 smithi079 ceph-mon[35312]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-03-28T18:49:26.685 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:49:26 smithi079 ceph-mon[35312]: from='mgr.15522 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-03-28T18:49:26.685 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:49:26 smithi079 ceph-mon[35312]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-28T18:49:26.685 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:49:26 smithi079 ceph-mon[35312]: from='mgr.15522 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-28T18:49:26.685 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:49:26 smithi079 ceph-mon[35312]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-28T18:49:26.685 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:49:26 smithi079 ceph-mon[35312]: from='mgr.15522 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-28T18:49:26.685 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:49:26 smithi079 ceph-mon[35312]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-03-28T18:49:26.685 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:49:26 smithi079 ceph-mon[35312]: from='mgr.15522 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-03-28T18:49:26.685 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:49:26 smithi079 ceph-mon[35312]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-03-28T18:49:26.685 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:49:26 smithi079 ceph-mon[35312]: from='mgr.15522 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-03-28T18:49:26.685 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:49:26 smithi079 ceph-mon[35312]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-28T18:49:26.685 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:49:26 smithi079 ceph-mon[35312]: from='mgr.15522 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-28T18:49:26.686 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:49:26 smithi079 ceph-mon[35312]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-28T18:49:26.686 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:49:26 smithi079 ceph-mon[35312]: from='mgr.15522 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-28T18:49:26.686 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:49:26 smithi079 ceph-mon[35312]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-03-28T18:49:26.686 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:49:26 smithi079 ceph-mon[35312]: from='mgr.15522 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-03-28T18:49:26.686 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:49:26 smithi079 ceph-mon[35312]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-03-28T18:49:26.686 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:49:26 smithi079 ceph-mon[35312]: from='mgr.15522 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-03-28T18:49:26.686 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:49:26 smithi079 ceph-mon[35312]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T18:49:26.686 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:49:26 smithi079 ceph-mon[35312]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T18:49:26.686 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:49:26 smithi079 ceph-mon[35312]: from='mgr.15522 ' entity='mgr.x' 2024-03-28T18:49:27.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:49:27 smithi022 ceph-mon[31985]: Adjusting osd_memory_target on smithi079 to 176.7M 2024-03-28T18:49:27.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:49:27 smithi022 ceph-mon[31985]: Unable to set osd_memory_target on smithi079 to 185386700: error parsing value: Value '185386700' is below minimum 939524096 2024-03-28T18:49:27.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:49:27 smithi022 ceph-mon[31985]: Adjusting osd_memory_target on smithi022 to 177.8M 2024-03-28T18:49:27.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:49:27 smithi022 ceph-mon[31985]: Unable to set osd_memory_target on smithi022 to 186497740: error parsing value: Value '186497740' is below minimum 939524096 2024-03-28T18:49:27.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:49:27 smithi022 ceph-mon[31985]: Adjusting osd_memory_target on smithi134 to 176.8M 2024-03-28T18:49:27.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:49:27 smithi022 ceph-mon[31985]: Unable to set osd_memory_target on smithi134 to 185393868: error parsing value: Value '185393868' is below minimum 939524096 2024-03-28T18:49:27.533 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:49:27 smithi022 ceph-mon[31985]: pgmap v1319: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 6 op/s 2024-03-28T18:49:27.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:49:27 smithi134 ceph-mon[35037]: Adjusting osd_memory_target on smithi079 to 176.7M 2024-03-28T18:49:27.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:49:27 smithi134 ceph-mon[35037]: Unable to set osd_memory_target on smithi079 to 185386700: error parsing value: Value '185386700' is below minimum 939524096 2024-03-28T18:49:27.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:49:27 smithi134 ceph-mon[35037]: Adjusting osd_memory_target on smithi022 to 177.8M 2024-03-28T18:49:27.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:49:27 smithi134 ceph-mon[35037]: Unable to set osd_memory_target on smithi022 to 186497740: error parsing value: Value '186497740' is below minimum 939524096 2024-03-28T18:49:27.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:49:27 smithi134 ceph-mon[35037]: Adjusting osd_memory_target on smithi134 to 176.8M 2024-03-28T18:49:27.676 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:49:27 smithi134 ceph-mon[35037]: Unable to set osd_memory_target on smithi134 to 185393868: error parsing value: Value '185393868' is below minimum 939524096 2024-03-28T18:49:27.676 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:49:27 smithi134 ceph-mon[35037]: pgmap v1319: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 6 op/s 2024-03-28T18:49:27.683 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:49:27 smithi079 ceph-mon[35312]: Adjusting osd_memory_target on smithi079 to 176.7M 2024-03-28T18:49:27.683 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:49:27 smithi079 ceph-mon[35312]: Unable to set osd_memory_target on smithi079 to 185386700: error parsing value: Value '185386700' is below minimum 939524096 2024-03-28T18:49:27.683 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:49:27 smithi079 ceph-mon[35312]: Adjusting osd_memory_target on smithi022 to 177.8M 2024-03-28T18:49:27.683 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:49:27 smithi079 ceph-mon[35312]: Unable to set osd_memory_target on smithi022 to 186497740: error parsing value: Value '186497740' is below minimum 939524096 2024-03-28T18:49:27.683 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:49:27 smithi079 ceph-mon[35312]: Adjusting osd_memory_target on smithi134 to 176.8M 2024-03-28T18:49:27.683 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:49:27 smithi079 ceph-mon[35312]: Unable to set osd_memory_target on smithi134 to 185393868: error parsing value: Value '185393868' is below minimum 939524096 2024-03-28T18:49:27.683 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:49:27 smithi079 ceph-mon[35312]: pgmap v1319: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 6 op/s 2024-03-28T18:49:29.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:49:29 smithi134 ceph-mon[35037]: pgmap v1320: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 6 op/s 2024-03-28T18:49:29.683 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:49:29 smithi079 ceph-mon[35312]: pgmap v1320: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 6 op/s 2024-03-28T18:49:29.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:49:29 smithi022 ceph-mon[31985]: pgmap v1320: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 6 op/s 2024-03-28T18:49:30.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:49:30 smithi134 ceph-mon[35037]: 2 slow requests, 0 included below; oldest blocked for > 1956.870067 secs 2024-03-28T18:49:30.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:49:30 smithi079 ceph-mon[35312]: 2 slow requests, 0 included below; oldest blocked for > 1956.870067 secs 2024-03-28T18:49:30.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:49:30 smithi022 ceph-mon[31985]: 2 slow requests, 0 included below; oldest blocked for > 1956.870067 secs 2024-03-28T18:49:31.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:49:31 smithi134 ceph-mon[35037]: pgmap v1321: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 10 op/s 2024-03-28T18:49:31.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:49:31 smithi079 ceph-mon[35312]: pgmap v1321: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 10 op/s 2024-03-28T18:49:31.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:49:31 smithi022 ceph-mon[31985]: pgmap v1321: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 10 op/s 2024-03-28T18:49:33.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:49:33 smithi134 ceph-mon[35037]: pgmap v1322: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 6 op/s 2024-03-28T18:49:33.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:49:33 smithi079 ceph-mon[35312]: pgmap v1322: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 6 op/s 2024-03-28T18:49:33.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:49:33 smithi022 ceph-mon[31985]: pgmap v1322: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 6 op/s 2024-03-28T18:49:35.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:49:35 smithi134 ceph-mon[35037]: pgmap v1323: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 10 op/s 2024-03-28T18:49:35.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:49:35 smithi134 ceph-mon[35037]: 2 slow requests, 0 included below; oldest blocked for > 1961.870139 secs 2024-03-28T18:49:35.683 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:49:35 smithi079 ceph-mon[35312]: pgmap v1323: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 10 op/s 2024-03-28T18:49:35.683 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:49:35 smithi079 ceph-mon[35312]: 2 slow requests, 0 included below; oldest blocked for > 1961.870139 secs 2024-03-28T18:49:35.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:49:35 smithi022 ceph-mon[31985]: pgmap v1323: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 10 op/s 2024-03-28T18:49:35.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:49:35 smithi022 ceph-mon[31985]: 2 slow requests, 0 included below; oldest blocked for > 1961.870139 secs 2024-03-28T18:49:37.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:49:37 smithi134 ceph-mon[35037]: pgmap v1324: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 6 op/s 2024-03-28T18:49:37.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:49:37 smithi079 ceph-mon[35312]: pgmap v1324: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 6 op/s 2024-03-28T18:49:37.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:49:37 smithi022 ceph-mon[31985]: pgmap v1324: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 6 op/s 2024-03-28T18:49:39.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:49:39 smithi134 ceph-mon[35037]: pgmap v1325: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:49:39.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:49:39 smithi079 ceph-mon[35312]: pgmap v1325: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:49:39.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:49:39 smithi022 ceph-mon[31985]: pgmap v1325: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:49:40.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:49:40 smithi134 ceph-mon[35037]: 2 slow requests, 0 included below; oldest blocked for > 1966.870207 secs 2024-03-28T18:49:40.683 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:49:40 smithi079 ceph-mon[35312]: 2 slow requests, 0 included below; oldest blocked for > 1966.870207 secs 2024-03-28T18:49:40.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:49:40 smithi022 ceph-mon[31985]: 2 slow requests, 0 included below; oldest blocked for > 1966.870207 secs 2024-03-28T18:49:41.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:49:41 smithi134 ceph-mon[35037]: pgmap v1326: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:49:41.683 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:49:41 smithi079 ceph-mon[35312]: pgmap v1326: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:49:41.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:49:41 smithi022 ceph-mon[31985]: pgmap v1326: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:49:43.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:49:43 smithi134 ceph-mon[35037]: pgmap v1327: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:49:43.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:49:43 smithi079 ceph-mon[35312]: pgmap v1327: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:49:43.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:49:43 smithi022 ceph-mon[31985]: pgmap v1327: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:49:45.676 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:49:45 smithi134 ceph-mon[35037]: pgmap v1328: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 9 op/s 2024-03-28T18:49:45.676 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:49:45 smithi134 ceph-mon[35037]: 2 slow requests, 0 included below; oldest blocked for > 1971.870249 secs 2024-03-28T18:49:45.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:49:45 smithi079 ceph-mon[35312]: pgmap v1328: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 9 op/s 2024-03-28T18:49:45.683 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:49:45 smithi079 ceph-mon[35312]: 2 slow requests, 0 included below; oldest blocked for > 1971.870249 secs 2024-03-28T18:49:45.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:49:45 smithi022 ceph-mon[31985]: pgmap v1328: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 9 op/s 2024-03-28T18:49:45.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:49:45 smithi022 ceph-mon[31985]: 2 slow requests, 0 included below; oldest blocked for > 1971.870249 secs 2024-03-28T18:49:47.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:49:47 smithi134 ceph-mon[35037]: pgmap v1329: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T18:49:47.683 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:49:47 smithi079 ceph-mon[35312]: pgmap v1329: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T18:49:47.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:49:47 smithi022 ceph-mon[31985]: pgmap v1329: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T18:49:49.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:49:49 smithi134 ceph-mon[35037]: pgmap v1330: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T18:49:49.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:49:49 smithi079 ceph-mon[35312]: pgmap v1330: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T18:49:49.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:49:49 smithi022 ceph-mon[31985]: pgmap v1330: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T18:49:50.678 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:49:50 smithi134 ceph-mon[35037]: 2 slow requests, 0 included below; oldest blocked for > 1976.870314 secs 2024-03-28T18:49:50.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:49:50 smithi079 ceph-mon[35312]: 2 slow requests, 0 included below; oldest blocked for > 1976.870314 secs 2024-03-28T18:49:50.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:49:50 smithi022 ceph-mon[31985]: 2 slow requests, 0 included below; oldest blocked for > 1976.870314 secs 2024-03-28T18:49:51.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:49:51 smithi134 ceph-mon[35037]: pgmap v1331: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 9 op/s 2024-03-28T18:49:51.683 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:49:51 smithi079 ceph-mon[35312]: pgmap v1331: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 9 op/s 2024-03-28T18:49:51.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:49:51 smithi022 ceph-mon[31985]: pgmap v1331: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 9 op/s 2024-03-28T18:49:53.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:49:53 smithi134 ceph-mon[35037]: pgmap v1332: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T18:49:53.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:49:53 smithi079 ceph-mon[35312]: pgmap v1332: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T18:49:53.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:49:53 smithi022 ceph-mon[31985]: pgmap v1332: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T18:49:55.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:49:55 smithi134 ceph-mon[35037]: pgmap v1333: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 9 op/s 2024-03-28T18:49:55.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:49:55 smithi134 ceph-mon[35037]: 2 slow requests, 0 included below; oldest blocked for > 1981.870384 secs 2024-03-28T18:49:55.683 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:49:55 smithi079 ceph-mon[35312]: pgmap v1333: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 9 op/s 2024-03-28T18:49:55.683 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:49:55 smithi079 ceph-mon[35312]: 2 slow requests, 0 included below; oldest blocked for > 1981.870384 secs 2024-03-28T18:49:55.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:49:55 smithi022 ceph-mon[31985]: pgmap v1333: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 9 op/s 2024-03-28T18:49:55.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:49:55 smithi022 ceph-mon[31985]: 2 slow requests, 0 included below; oldest blocked for > 1981.870384 secs 2024-03-28T18:49:57.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:49:57 smithi134 ceph-mon[35037]: pgmap v1334: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:49:57.680 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:49:57 smithi022 ceph-mon[31985]: pgmap v1334: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:49:57.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:49:57 smithi079 ceph-mon[35312]: pgmap v1334: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:49:59.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:49:59 smithi134 ceph-mon[35037]: pgmap v1335: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:49:59.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:49:59 smithi079 ceph-mon[35312]: pgmap v1335: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:49:59.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:49:59 smithi022 ceph-mon[31985]: pgmap v1335: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:50:00.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:50:00 smithi134 ceph-mon[35037]: 2 slow requests, 0 included below; oldest blocked for > 1986.870472 secs 2024-03-28T18:50:00.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:50:00 smithi134 ceph-mon[35037]: overall HEALTH_WARN 4 failed cephadm daemon(s); 1 MDSs report slow requests 2024-03-28T18:50:00.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:50:00 smithi079 ceph-mon[35312]: 2 slow requests, 0 included below; oldest blocked for > 1986.870472 secs 2024-03-28T18:50:00.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:50:00 smithi079 ceph-mon[35312]: overall HEALTH_WARN 4 failed cephadm daemon(s); 1 MDSs report slow requests 2024-03-28T18:50:00.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:50:00 smithi022 ceph-mon[31985]: 2 slow requests, 0 included below; oldest blocked for > 1986.870472 secs 2024-03-28T18:50:00.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:50:00 smithi022 ceph-mon[31985]: overall HEALTH_WARN 4 failed cephadm daemon(s); 1 MDSs report slow requests 2024-03-28T18:50:01.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:50:01 smithi134 ceph-mon[35037]: pgmap v1336: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:50:01.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:50:01 smithi079 ceph-mon[35312]: pgmap v1336: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:50:01.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:50:01 smithi022 ceph-mon[31985]: pgmap v1336: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:50:02.683 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:50:02 smithi079 ceph-mon[35312]: pgmap v1337: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:50:02.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:50:02 smithi022 ceph-mon[31985]: pgmap v1337: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:50:02.925 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:50:02 smithi134 ceph-mon[35037]: pgmap v1337: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:50:05.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:50:05 smithi022 ceph-mon[31985]: pgmap v1338: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:50:05.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:50:05 smithi022 ceph-mon[31985]: 2 slow requests, 0 included below; oldest blocked for > 1991.870546 secs 2024-03-28T18:50:05.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:50:05 smithi134 ceph-mon[35037]: pgmap v1338: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:50:05.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:50:05 smithi134 ceph-mon[35037]: 2 slow requests, 0 included below; oldest blocked for > 1991.870546 secs 2024-03-28T18:50:05.683 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:50:05 smithi079 ceph-mon[35312]: pgmap v1338: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:50:05.683 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:50:05 smithi079 ceph-mon[35312]: 2 slow requests, 0 included below; oldest blocked for > 1991.870546 secs 2024-03-28T18:50:07.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:50:07 smithi134 ceph-mon[35037]: pgmap v1339: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:50:07.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:50:07 smithi079 ceph-mon[35312]: pgmap v1339: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:50:07.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:50:07 smithi022 ceph-mon[31985]: pgmap v1339: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:50:09.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:50:09 smithi134 ceph-mon[35037]: pgmap v1340: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:50:09.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:50:09 smithi079 ceph-mon[35312]: pgmap v1340: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:50:09.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:50:09 smithi022 ceph-mon[31985]: pgmap v1340: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:50:10.678 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:50:10 smithi134 ceph-mon[35037]: 2 slow requests, 0 included below; oldest blocked for > 1996.870621 secs 2024-03-28T18:50:10.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:50:10 smithi079 ceph-mon[35312]: 2 slow requests, 0 included below; oldest blocked for > 1996.870621 secs 2024-03-28T18:50:10.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:50:10 smithi022 ceph-mon[31985]: 2 slow requests, 0 included below; oldest blocked for > 1996.870621 secs 2024-03-28T18:50:11.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:50:11 smithi134 ceph-mon[35037]: pgmap v1341: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:50:11.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:50:11 smithi079 ceph-mon[35312]: pgmap v1341: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:50:11.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:50:11 smithi022 ceph-mon[31985]: pgmap v1341: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:50:13.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:50:13 smithi134 ceph-mon[35037]: pgmap v1342: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:50:13.683 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:50:13 smithi079 ceph-mon[35312]: pgmap v1342: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:50:13.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:50:13 smithi022 ceph-mon[31985]: pgmap v1342: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:50:15.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:50:15 smithi134 ceph-mon[35037]: pgmap v1343: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:50:15.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:50:15 smithi134 ceph-mon[35037]: 2 slow requests, 0 included below; oldest blocked for > 2001.870695 secs 2024-03-28T18:50:15.683 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:50:15 smithi079 ceph-mon[35312]: pgmap v1343: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:50:15.683 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:50:15 smithi079 ceph-mon[35312]: 2 slow requests, 0 included below; oldest blocked for > 2001.870695 secs 2024-03-28T18:50:15.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:50:15 smithi022 ceph-mon[31985]: pgmap v1343: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:50:15.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:50:15 smithi022 ceph-mon[31985]: 2 slow requests, 0 included below; oldest blocked for > 2001.870695 secs 2024-03-28T18:50:17.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:50:17 smithi134 ceph-mon[35037]: pgmap v1344: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:50:17.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:50:17 smithi079 ceph-mon[35312]: pgmap v1344: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:50:17.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:50:17 smithi022 ceph-mon[31985]: pgmap v1344: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:50:19.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:50:19 smithi134 ceph-mon[35037]: pgmap v1345: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:50:19.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:50:19 smithi079 ceph-mon[35312]: pgmap v1345: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:50:19.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:50:19 smithi022 ceph-mon[31985]: pgmap v1345: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:50:20.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:50:20 smithi134 ceph-mon[35037]: 2 slow requests, 0 included below; oldest blocked for > 2006.870772 secs 2024-03-28T18:50:20.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:50:20 smithi079 ceph-mon[35312]: 2 slow requests, 0 included below; oldest blocked for > 2006.870772 secs 2024-03-28T18:50:20.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:50:20 smithi022 ceph-mon[31985]: 2 slow requests, 0 included below; oldest blocked for > 2006.870772 secs 2024-03-28T18:50:21.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:50:21 smithi134 ceph-mon[35037]: pgmap v1346: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:50:21.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:50:21 smithi079 ceph-mon[35312]: pgmap v1346: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:50:21.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:50:21 smithi022 ceph-mon[31985]: pgmap v1346: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:50:23.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:50:23 smithi134 ceph-mon[35037]: pgmap v1347: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:50:23.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:50:23 smithi079 ceph-mon[35312]: pgmap v1347: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:50:23.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:50:23 smithi022 ceph-mon[31985]: pgmap v1347: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:50:25.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:50:25 smithi134 ceph-mon[35037]: pgmap v1348: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:50:25.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:50:25 smithi134 ceph-mon[35037]: 2 slow requests, 0 included below; oldest blocked for > 2011.870892 secs 2024-03-28T18:50:25.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:50:25 smithi079 ceph-mon[35312]: pgmap v1348: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:50:25.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:50:25 smithi079 ceph-mon[35312]: 2 slow requests, 0 included below; oldest blocked for > 2011.870892 secs 2024-03-28T18:50:25.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:50:25 smithi022 ceph-mon[31985]: pgmap v1348: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:50:25.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:50:25 smithi022 ceph-mon[31985]: 2 slow requests, 0 included below; oldest blocked for > 2011.870892 secs 2024-03-28T18:50:27.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:50:27 smithi134 ceph-mon[35037]: pgmap v1349: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:50:27.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:50:27 smithi134 ceph-mon[35037]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T18:50:27.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:50:27 smithi079 ceph-mon[35312]: pgmap v1349: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:50:27.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:50:27 smithi079 ceph-mon[35312]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T18:50:27.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:50:27 smithi022 ceph-mon[31985]: pgmap v1349: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:50:27.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:50:27 smithi022 ceph-mon[31985]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T18:50:28.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:50:28 smithi134 ceph-mon[35037]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T18:50:28.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:50:28 smithi134 ceph-mon[35037]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T18:50:28.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:50:28 smithi134 ceph-mon[35037]: from='mgr.15522 ' entity='mgr.x' 2024-03-28T18:50:28.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:50:28 smithi079 ceph-mon[35312]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T18:50:28.683 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:50:28 smithi079 ceph-mon[35312]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T18:50:28.683 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:50:28 smithi079 ceph-mon[35312]: from='mgr.15522 ' entity='mgr.x' 2024-03-28T18:50:28.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:50:28 smithi022 ceph-mon[31985]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T18:50:28.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:50:28 smithi022 ceph-mon[31985]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T18:50:28.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:50:28 smithi022 ceph-mon[31985]: from='mgr.15522 ' entity='mgr.x' 2024-03-28T18:50:29.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:50:29 smithi134 ceph-mon[35037]: pgmap v1350: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 6 op/s 2024-03-28T18:50:29.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:50:29 smithi079 ceph-mon[35312]: pgmap v1350: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 6 op/s 2024-03-28T18:50:29.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:50:29 smithi022 ceph-mon[31985]: pgmap v1350: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 6 op/s 2024-03-28T18:50:30.683 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:50:30 smithi079 ceph-mon[35312]: 2 slow requests, 0 included below; oldest blocked for > 2016.871012 secs 2024-03-28T18:50:30.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:50:30 smithi022 ceph-mon[31985]: 2 slow requests, 0 included below; oldest blocked for > 2016.871012 secs 2024-03-28T18:50:30.925 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:50:30 smithi134 ceph-mon[35037]: 2 slow requests, 0 included below; oldest blocked for > 2016.871012 secs 2024-03-28T18:50:31.683 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:50:31 smithi079 ceph-mon[35312]: pgmap v1351: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 10 op/s 2024-03-28T18:50:31.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:50:31 smithi022 ceph-mon[31985]: pgmap v1351: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 10 op/s 2024-03-28T18:50:31.925 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:50:31 smithi134 ceph-mon[35037]: pgmap v1351: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 10 op/s 2024-03-28T18:50:32.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:50:32 smithi022 ceph-mon[31985]: pgmap v1352: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 6 op/s 2024-03-28T18:50:32.925 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:50:32 smithi134 ceph-mon[35037]: pgmap v1352: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 6 op/s 2024-03-28T18:50:32.933 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:50:32 smithi079 ceph-mon[35312]: pgmap v1352: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 6 op/s 2024-03-28T18:50:35.433 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:50:35 smithi079 ceph-mon[35312]: pgmap v1353: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 6 op/s 2024-03-28T18:50:35.433 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:50:35 smithi079 ceph-mon[35312]: 2 slow requests, 0 included below; oldest blocked for > 2021.871103 secs 2024-03-28T18:50:35.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:50:35 smithi022 ceph-mon[31985]: pgmap v1353: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 6 op/s 2024-03-28T18:50:35.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:50:35 smithi022 ceph-mon[31985]: 2 slow requests, 0 included below; oldest blocked for > 2021.871103 secs 2024-03-28T18:50:35.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:50:35 smithi134 ceph-mon[35037]: pgmap v1353: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 6 op/s 2024-03-28T18:50:35.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:50:35 smithi134 ceph-mon[35037]: 2 slow requests, 0 included below; oldest blocked for > 2021.871103 secs 2024-03-28T18:50:37.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:50:37 smithi022 ceph-mon[31985]: pgmap v1354: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 6 op/s 2024-03-28T18:50:37.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:50:37 smithi134 ceph-mon[35037]: pgmap v1354: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 6 op/s 2024-03-28T18:50:37.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:50:37 smithi079 ceph-mon[35312]: pgmap v1354: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 6 op/s 2024-03-28T18:50:39.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:50:39 smithi022 ceph-mon[31985]: pgmap v1355: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 6 op/s 2024-03-28T18:50:39.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:50:39 smithi134 ceph-mon[35037]: pgmap v1355: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 6 op/s 2024-03-28T18:50:39.683 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:50:39 smithi079 ceph-mon[35312]: pgmap v1355: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 6 op/s 2024-03-28T18:50:40.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:50:40 smithi022 ceph-mon[31985]: 2 slow requests, 0 included below; oldest blocked for > 2026.871180 secs 2024-03-28T18:50:40.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:50:40 smithi134 ceph-mon[35037]: 2 slow requests, 0 included below; oldest blocked for > 2026.871180 secs 2024-03-28T18:50:40.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:50:40 smithi079 ceph-mon[35312]: 2 slow requests, 0 included below; oldest blocked for > 2026.871180 secs 2024-03-28T18:50:41.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:50:41 smithi022 ceph-mon[31985]: pgmap v1356: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:50:41.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:50:41 smithi134 ceph-mon[35037]: pgmap v1356: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:50:41.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:50:41 smithi079 ceph-mon[35312]: pgmap v1356: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:50:43.478 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:50:43 smithi134 ceph-mon[35037]: pgmap v1357: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:50:43.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:50:43 smithi022 ceph-mon[31985]: pgmap v1357: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:50:43.683 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:50:43 smithi079 ceph-mon[35312]: pgmap v1357: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:50:45.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:50:45 smithi022 ceph-mon[31985]: pgmap v1358: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:50:45.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:50:45 smithi022 ceph-mon[31985]: 2 slow requests, 0 included below; oldest blocked for > 2031.871270 secs 2024-03-28T18:50:45.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:50:45 smithi134 ceph-mon[35037]: pgmap v1358: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:50:45.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:50:45 smithi134 ceph-mon[35037]: 2 slow requests, 0 included below; oldest blocked for > 2031.871270 secs 2024-03-28T18:50:45.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:50:45 smithi079 ceph-mon[35312]: pgmap v1358: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:50:45.683 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:50:45 smithi079 ceph-mon[35312]: 2 slow requests, 0 included below; oldest blocked for > 2031.871270 secs 2024-03-28T18:50:47.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:50:47 smithi022 ceph-mon[31985]: pgmap v1359: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:50:47.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:50:47 smithi134 ceph-mon[35037]: pgmap v1359: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:50:47.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:50:47 smithi079 ceph-mon[35312]: pgmap v1359: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:50:49.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:50:49 smithi022 ceph-mon[31985]: pgmap v1360: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:50:49.676 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:50:49 smithi134 ceph-mon[35037]: pgmap v1360: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:50:49.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:50:49 smithi079 ceph-mon[35312]: pgmap v1360: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:50:50.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:50:50 smithi022 ceph-mon[31985]: 2 slow requests, 0 included below; oldest blocked for > 2036.871364 secs 2024-03-28T18:50:50.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:50:50 smithi134 ceph-mon[35037]: 2 slow requests, 0 included below; oldest blocked for > 2036.871364 secs 2024-03-28T18:50:50.683 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:50:50 smithi079 ceph-mon[35312]: 2 slow requests, 0 included below; oldest blocked for > 2036.871364 secs 2024-03-28T18:50:51.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:50:51 smithi022 ceph-mon[31985]: pgmap v1361: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:50:51.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:50:51 smithi134 ceph-mon[35037]: pgmap v1361: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:50:51.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:50:51 smithi079 ceph-mon[35312]: pgmap v1361: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:50:53.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:50:53 smithi022 ceph-mon[31985]: pgmap v1362: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:50:53.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:50:53 smithi134 ceph-mon[35037]: pgmap v1362: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:50:53.683 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:50:53 smithi079 ceph-mon[35312]: pgmap v1362: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:50:55.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:50:55 smithi022 ceph-mon[31985]: pgmap v1363: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:50:55.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:50:55 smithi022 ceph-mon[31985]: 2 slow requests, 0 included below; oldest blocked for > 2041.871487 secs 2024-03-28T18:50:55.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:50:55 smithi134 ceph-mon[35037]: pgmap v1363: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:50:55.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:50:55 smithi134 ceph-mon[35037]: 2 slow requests, 0 included below; oldest blocked for > 2041.871487 secs 2024-03-28T18:50:55.683 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:50:55 smithi079 ceph-mon[35312]: pgmap v1363: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:50:55.683 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:50:55 smithi079 ceph-mon[35312]: 2 slow requests, 0 included below; oldest blocked for > 2041.871487 secs 2024-03-28T18:50:57.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:50:57 smithi022 ceph-mon[31985]: pgmap v1364: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:50:57.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:50:57 smithi134 ceph-mon[35037]: pgmap v1364: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:50:57.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:50:57 smithi079 ceph-mon[35312]: pgmap v1364: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:50:59.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:50:59 smithi022 ceph-mon[31985]: pgmap v1365: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:50:59.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:50:59 smithi134 ceph-mon[35037]: pgmap v1365: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:50:59.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:50:59 smithi079 ceph-mon[35312]: pgmap v1365: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:51:00.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:51:00 smithi022 ceph-mon[31985]: 2 slow requests, 0 included below; oldest blocked for > 2046.871568 secs 2024-03-28T18:51:00.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:51:00 smithi134 ceph-mon[35037]: 2 slow requests, 0 included below; oldest blocked for > 2046.871568 secs 2024-03-28T18:51:00.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:51:00 smithi079 ceph-mon[35312]: 2 slow requests, 0 included below; oldest blocked for > 2046.871568 secs 2024-03-28T18:51:01.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:51:01 smithi134 ceph-mon[35037]: pgmap v1366: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:51:01.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:51:01 smithi079 ceph-mon[35312]: pgmap v1366: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:51:01.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:51:01 smithi022 ceph-mon[31985]: pgmap v1366: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:51:03.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:51:03 smithi134 ceph-mon[35037]: pgmap v1367: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:51:03.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:51:03 smithi079 ceph-mon[35312]: pgmap v1367: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:51:03.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:51:03 smithi022 ceph-mon[31985]: pgmap v1367: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:51:05.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:51:05 smithi134 ceph-mon[35037]: pgmap v1368: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:51:05.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:51:05 smithi134 ceph-mon[35037]: 2 slow requests, 0 included below; oldest blocked for > 2051.871630 secs 2024-03-28T18:51:05.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:51:05 smithi079 ceph-mon[35312]: pgmap v1368: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:51:05.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:51:05 smithi079 ceph-mon[35312]: 2 slow requests, 0 included below; oldest blocked for > 2051.871630 secs 2024-03-28T18:51:05.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:51:05 smithi022 ceph-mon[31985]: pgmap v1368: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:51:05.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:51:05 smithi022 ceph-mon[31985]: 2 slow requests, 0 included below; oldest blocked for > 2051.871630 secs 2024-03-28T18:51:07.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:51:07 smithi134 ceph-mon[35037]: pgmap v1369: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:51:07.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:51:07 smithi079 ceph-mon[35312]: pgmap v1369: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:51:07.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:51:07 smithi022 ceph-mon[31985]: pgmap v1369: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:51:09.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:51:09 smithi134 ceph-mon[35037]: pgmap v1370: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:51:09.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:51:09 smithi079 ceph-mon[35312]: pgmap v1370: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:51:09.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:51:09 smithi022 ceph-mon[31985]: pgmap v1370: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:51:10.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:51:10 smithi134 ceph-mon[35037]: 2 slow requests, 0 included below; oldest blocked for > 2056.871659 secs 2024-03-28T18:51:10.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:51:10 smithi079 ceph-mon[35312]: 2 slow requests, 0 included below; oldest blocked for > 2056.871659 secs 2024-03-28T18:51:10.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:51:10 smithi022 ceph-mon[31985]: 2 slow requests, 0 included below; oldest blocked for > 2056.871659 secs 2024-03-28T18:51:11.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:51:11 smithi134 ceph-mon[35037]: pgmap v1371: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:51:11.683 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:51:11 smithi079 ceph-mon[35312]: pgmap v1371: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:51:11.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:51:11 smithi022 ceph-mon[31985]: pgmap v1371: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:51:13.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:51:13 smithi134 ceph-mon[35037]: pgmap v1372: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:51:13.683 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:51:13 smithi079 ceph-mon[35312]: pgmap v1372: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:51:13.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:51:13 smithi022 ceph-mon[31985]: pgmap v1372: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:51:15.585 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:51:15 smithi022 ceph-mon[31985]: pgmap v1373: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:51:15.585 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:51:15 smithi022 ceph-mon[31985]: 2 slow requests, 0 included below; oldest blocked for > 2061.871747 secs 2024-03-28T18:51:15.674 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:51:15 smithi134 ceph-mon[35037]: pgmap v1373: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:51:15.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:51:15 smithi134 ceph-mon[35037]: 2 slow requests, 0 included below; oldest blocked for > 2061.871747 secs 2024-03-28T18:51:15.683 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:51:15 smithi079 ceph-mon[35312]: pgmap v1373: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:51:15.683 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:51:15 smithi079 ceph-mon[35312]: 2 slow requests, 0 included below; oldest blocked for > 2061.871747 secs 2024-03-28T18:51:17.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:51:17 smithi134 ceph-mon[35037]: pgmap v1374: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:51:17.683 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:51:17 smithi079 ceph-mon[35312]: pgmap v1374: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:51:17.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:51:17 smithi022 ceph-mon[31985]: pgmap v1374: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:51:19.677 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:51:19 smithi134 ceph-mon[35037]: pgmap v1375: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:51:19.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:51:19 smithi079 ceph-mon[35312]: pgmap v1375: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:51:19.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:51:19 smithi022 ceph-mon[31985]: pgmap v1375: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:51:20.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:51:20 smithi134 ceph-mon[35037]: 2 slow requests, 0 included below; oldest blocked for > 2066.871872 secs 2024-03-28T18:51:20.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:51:20 smithi079 ceph-mon[35312]: 2 slow requests, 0 included below; oldest blocked for > 2066.871872 secs 2024-03-28T18:51:20.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:51:20 smithi022 ceph-mon[31985]: 2 slow requests, 0 included below; oldest blocked for > 2066.871872 secs 2024-03-28T18:51:21.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:51:21 smithi134 ceph-mon[35037]: pgmap v1376: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:51:21.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:51:21 smithi079 ceph-mon[35312]: pgmap v1376: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:51:21.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:51:21 smithi022 ceph-mon[31985]: pgmap v1376: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:51:23.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:51:23 smithi134 ceph-mon[35037]: pgmap v1377: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:51:23.683 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:51:23 smithi079 ceph-mon[35312]: pgmap v1377: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:51:23.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:51:23 smithi022 ceph-mon[31985]: pgmap v1377: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:51:25.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:51:25 smithi134 ceph-mon[35037]: pgmap v1378: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:51:25.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:51:25 smithi134 ceph-mon[35037]: 2 slow requests, 0 included below; oldest blocked for > 2071.871977 secs 2024-03-28T18:51:25.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:51:25 smithi079 ceph-mon[35312]: pgmap v1378: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:51:25.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:51:25 smithi079 ceph-mon[35312]: 2 slow requests, 0 included below; oldest blocked for > 2071.871977 secs 2024-03-28T18:51:25.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:51:25 smithi022 ceph-mon[31985]: pgmap v1378: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:51:25.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:51:25 smithi022 ceph-mon[31985]: 2 slow requests, 0 included below; oldest blocked for > 2071.871977 secs 2024-03-28T18:51:27.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:51:27 smithi134 ceph-mon[35037]: pgmap v1379: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:51:27.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:51:27 smithi079 ceph-mon[35312]: pgmap v1379: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:51:27.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:51:27 smithi022 ceph-mon[31985]: pgmap v1379: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:51:29.674 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:51:29 smithi134 ceph-mon[35037]: pgmap v1380: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:51:29.674 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:51:29 smithi134 ceph-mon[35037]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T18:51:29.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:51:29 smithi079 ceph-mon[35312]: pgmap v1380: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:51:29.683 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:51:29 smithi079 ceph-mon[35312]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T18:51:29.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:51:29 smithi022 ceph-mon[31985]: pgmap v1380: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:51:29.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:51:29 smithi022 ceph-mon[31985]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T18:51:30.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:51:30 smithi134 ceph-mon[35037]: 2 slow requests, 0 included below; oldest blocked for > 2076.872071 secs 2024-03-28T18:51:30.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:51:30 smithi134 ceph-mon[35037]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T18:51:30.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:51:30 smithi134 ceph-mon[35037]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T18:51:30.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:51:30 smithi134 ceph-mon[35037]: from='mgr.15522 ' entity='mgr.x' 2024-03-28T18:51:30.683 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:51:30 smithi079 ceph-mon[35312]: 2 slow requests, 0 included below; oldest blocked for > 2076.872071 secs 2024-03-28T18:51:30.683 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:51:30 smithi079 ceph-mon[35312]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T18:51:30.683 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:51:30 smithi079 ceph-mon[35312]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T18:51:30.683 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:51:30 smithi079 ceph-mon[35312]: from='mgr.15522 ' entity='mgr.x' 2024-03-28T18:51:30.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:51:30 smithi022 ceph-mon[31985]: 2 slow requests, 0 included below; oldest blocked for > 2076.872071 secs 2024-03-28T18:51:30.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:51:30 smithi022 ceph-mon[31985]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T18:51:30.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:51:30 smithi022 ceph-mon[31985]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T18:51:30.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:51:30 smithi022 ceph-mon[31985]: from='mgr.15522 ' entity='mgr.x' 2024-03-28T18:51:31.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:51:31 smithi134 ceph-mon[35037]: pgmap v1381: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 10 op/s 2024-03-28T18:51:31.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:51:31 smithi134 ceph-mon[35037]: pgmap v1382: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-03-28T18:51:31.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:51:31 smithi079 ceph-mon[35312]: pgmap v1381: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 10 op/s 2024-03-28T18:51:31.683 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:51:31 smithi079 ceph-mon[35312]: pgmap v1382: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-03-28T18:51:31.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:51:31 smithi022 ceph-mon[31985]: pgmap v1381: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 10 op/s 2024-03-28T18:51:31.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:51:31 smithi022 ceph-mon[31985]: pgmap v1382: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-03-28T18:51:33.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:51:33 smithi134 ceph-mon[35037]: pgmap v1383: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-03-28T18:51:33.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:51:33 smithi079 ceph-mon[35312]: pgmap v1383: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-03-28T18:51:33.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:51:33 smithi022 ceph-mon[31985]: pgmap v1383: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-03-28T18:51:35.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:51:35 smithi134 ceph-mon[35037]: pgmap v1384: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-03-28T18:51:35.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:51:35 smithi134 ceph-mon[35037]: 2 slow requests, 0 included below; oldest blocked for > 2081.872152 secs 2024-03-28T18:51:35.683 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:51:35 smithi079 ceph-mon[35312]: pgmap v1384: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-03-28T18:51:35.683 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:51:35 smithi079 ceph-mon[35312]: 2 slow requests, 0 included below; oldest blocked for > 2081.872152 secs 2024-03-28T18:51:35.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:51:35 smithi022 ceph-mon[31985]: pgmap v1384: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-03-28T18:51:35.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:51:35 smithi022 ceph-mon[31985]: 2 slow requests, 0 included below; oldest blocked for > 2081.872152 secs 2024-03-28T18:51:37.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:51:37 smithi134 ceph-mon[35037]: pgmap v1385: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-03-28T18:51:37.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:51:37 smithi079 ceph-mon[35312]: pgmap v1385: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-03-28T18:51:37.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:51:37 smithi022 ceph-mon[31985]: pgmap v1385: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-03-28T18:51:39.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:51:39 smithi134 ceph-mon[35037]: pgmap v1386: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-03-28T18:51:39.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:51:39 smithi079 ceph-mon[35312]: pgmap v1386: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-03-28T18:51:39.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:51:39 smithi022 ceph-mon[31985]: pgmap v1386: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-03-28T18:51:40.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:51:40 smithi022 ceph-mon[31985]: 2 slow requests, 0 included below; oldest blocked for > 2086.872237 secs 2024-03-28T18:51:40.925 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:51:40 smithi134 ceph-mon[35037]: 2 slow requests, 0 included below; oldest blocked for > 2086.872237 secs 2024-03-28T18:51:40.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:51:40 smithi079 ceph-mon[35312]: 2 slow requests, 0 included below; oldest blocked for > 2086.872237 secs 2024-03-28T18:51:41.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:51:41 smithi022 ceph-mon[31985]: pgmap v1387: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 7 op/s 2024-03-28T18:51:41.925 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:51:41 smithi134 ceph-mon[35037]: pgmap v1387: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 7 op/s 2024-03-28T18:51:41.933 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:51:41 smithi079 ceph-mon[35312]: pgmap v1387: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 7 op/s 2024-03-28T18:51:42.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:51:42 smithi022 ceph-mon[31985]: pgmap v1388: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:51:42.925 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:51:42 smithi134 ceph-mon[35037]: pgmap v1388: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:51:42.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:51:42 smithi079 ceph-mon[35312]: pgmap v1388: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:51:45.425 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:51:45 smithi134 ceph-mon[35037]: pgmap v1389: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:51:45.425 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:51:45 smithi134 ceph-mon[35037]: 2 slow requests, 0 included below; oldest blocked for > 2091.872303 secs 2024-03-28T18:51:45.433 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:51:45 smithi079 ceph-mon[35312]: pgmap v1389: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:51:45.433 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:51:45 smithi079 ceph-mon[35312]: 2 slow requests, 0 included below; oldest blocked for > 2091.872303 secs 2024-03-28T18:51:45.486 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:51:45 smithi022 ceph-mon[31985]: pgmap v1389: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:51:45.486 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:51:45 smithi022 ceph-mon[31985]: 2 slow requests, 0 included below; oldest blocked for > 2091.872303 secs 2024-03-28T18:51:47.425 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:51:47 smithi134 ceph-mon[35037]: pgmap v1390: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:51:47.432 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:51:47 smithi079 ceph-mon[35312]: pgmap v1390: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:51:47.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:51:47 smithi022 ceph-mon[31985]: pgmap v1390: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:51:49.425 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:51:49 smithi134 ceph-mon[35037]: pgmap v1391: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:51:49.432 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:51:49 smithi079 ceph-mon[35312]: pgmap v1391: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:51:49.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:51:49 smithi022 ceph-mon[31985]: pgmap v1391: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:51:50.425 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:51:50 smithi134 ceph-mon[35037]: 2 slow requests, 0 included below; oldest blocked for > 2096.872369 secs 2024-03-28T18:51:50.432 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:51:50 smithi079 ceph-mon[35312]: 2 slow requests, 0 included below; oldest blocked for > 2096.872369 secs 2024-03-28T18:51:50.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:51:50 smithi022 ceph-mon[31985]: 2 slow requests, 0 included below; oldest blocked for > 2096.872369 secs 2024-03-28T18:51:51.425 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:51:51 smithi134 ceph-mon[35037]: pgmap v1392: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:51:51.433 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:51:51 smithi079 ceph-mon[35312]: pgmap v1392: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:51:51.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:51:51 smithi022 ceph-mon[31985]: pgmap v1392: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:51:53.425 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:51:53 smithi134 ceph-mon[35037]: pgmap v1393: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:51:53.432 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:51:53 smithi079 ceph-mon[35312]: pgmap v1393: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:51:53.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:51:53 smithi022 ceph-mon[31985]: pgmap v1393: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:51:55.425 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:51:55 smithi134 ceph-mon[35037]: pgmap v1394: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:51:55.425 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:51:55 smithi134 ceph-mon[35037]: 2 slow requests, 0 included below; oldest blocked for > 2101.872453 secs 2024-03-28T18:51:55.432 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:51:55 smithi079 ceph-mon[35312]: pgmap v1394: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:51:55.433 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:51:55 smithi079 ceph-mon[35312]: 2 slow requests, 0 included below; oldest blocked for > 2101.872453 secs 2024-03-28T18:51:55.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:51:55 smithi022 ceph-mon[31985]: pgmap v1394: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:51:55.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:51:55 smithi022 ceph-mon[31985]: 2 slow requests, 0 included below; oldest blocked for > 2101.872453 secs 2024-03-28T18:51:57.425 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:51:57 smithi134 ceph-mon[35037]: pgmap v1395: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:51:57.433 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:51:57 smithi079 ceph-mon[35312]: pgmap v1395: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:51:57.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:51:57 smithi022 ceph-mon[31985]: pgmap v1395: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:51:59.425 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:51:59 smithi134 ceph-mon[35037]: pgmap v1396: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:51:59.432 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:51:59 smithi079 ceph-mon[35312]: pgmap v1396: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:51:59.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:51:59 smithi022 ceph-mon[31985]: pgmap v1396: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:52:00.425 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:52:00 smithi134 ceph-mon[35037]: 2 slow requests, 0 included below; oldest blocked for > 2106.872490 secs 2024-03-28T18:52:00.432 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:52:00 smithi079 ceph-mon[35312]: 2 slow requests, 0 included below; oldest blocked for > 2106.872490 secs 2024-03-28T18:52:00.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:52:00 smithi022 ceph-mon[31985]: 2 slow requests, 0 included below; oldest blocked for > 2106.872490 secs 2024-03-28T18:52:01.425 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:52:01 smithi134 ceph-mon[35037]: pgmap v1397: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:52:01.433 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:52:01 smithi079 ceph-mon[35312]: pgmap v1397: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:52:01.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:52:01 smithi022 ceph-mon[31985]: pgmap v1397: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:52:03.425 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:52:03 smithi134 ceph-mon[35037]: pgmap v1398: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:52:03.432 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:52:03 smithi079 ceph-mon[35312]: pgmap v1398: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:52:03.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:52:03 smithi022 ceph-mon[31985]: pgmap v1398: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:52:05.425 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:52:05 smithi134 ceph-mon[35037]: pgmap v1399: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:52:05.425 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:52:05 smithi134 ceph-mon[35037]: 2 slow requests, 0 included below; oldest blocked for > 2111.872556 secs 2024-03-28T18:52:05.432 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:52:05 smithi079 ceph-mon[35312]: pgmap v1399: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:52:05.432 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:52:05 smithi079 ceph-mon[35312]: 2 slow requests, 0 included below; oldest blocked for > 2111.872556 secs 2024-03-28T18:52:05.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:52:05 smithi022 ceph-mon[31985]: pgmap v1399: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:52:05.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:52:05 smithi022 ceph-mon[31985]: 2 slow requests, 0 included below; oldest blocked for > 2111.872556 secs 2024-03-28T18:52:07.425 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:52:07 smithi134 ceph-mon[35037]: pgmap v1400: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:52:07.432 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:52:07 smithi079 ceph-mon[35312]: pgmap v1400: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:52:07.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:52:07 smithi022 ceph-mon[31985]: pgmap v1400: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:52:09.425 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:52:09 smithi134 ceph-mon[35037]: pgmap v1401: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:52:09.432 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:52:09 smithi079 ceph-mon[35312]: pgmap v1401: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:52:09.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:52:09 smithi022 ceph-mon[31985]: pgmap v1401: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:52:10.424 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:52:10 smithi134 ceph-mon[35037]: 2 slow requests, 0 included below; oldest blocked for > 2116.872579 secs 2024-03-28T18:52:10.432 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:52:10 smithi079 ceph-mon[35312]: 2 slow requests, 0 included below; oldest blocked for > 2116.872579 secs 2024-03-28T18:52:10.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:52:10 smithi022 ceph-mon[31985]: 2 slow requests, 0 included below; oldest blocked for > 2116.872579 secs 2024-03-28T18:52:11.424 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:52:11 smithi134 ceph-mon[35037]: pgmap v1402: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:52:11.432 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:52:11 smithi079 ceph-mon[35312]: pgmap v1402: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:52:11.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:52:11 smithi022 ceph-mon[31985]: pgmap v1402: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:52:13.425 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:52:13 smithi134 ceph-mon[35037]: pgmap v1403: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:52:13.432 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:52:13 smithi079 ceph-mon[35312]: pgmap v1403: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:52:13.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:52:13 smithi022 ceph-mon[31985]: pgmap v1403: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:52:15.425 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:52:15 smithi134 ceph-mon[35037]: pgmap v1404: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:52:15.425 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:52:15 smithi134 ceph-mon[35037]: 2 slow requests, 0 included below; oldest blocked for > 2121.872655 secs 2024-03-28T18:52:15.432 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:52:15 smithi079 ceph-mon[35312]: pgmap v1404: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:52:15.433 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:52:15 smithi079 ceph-mon[35312]: 2 slow requests, 0 included below; oldest blocked for > 2121.872655 secs 2024-03-28T18:52:15.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:52:15 smithi022 ceph-mon[31985]: pgmap v1404: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:52:15.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:52:15 smithi022 ceph-mon[31985]: 2 slow requests, 0 included below; oldest blocked for > 2121.872655 secs 2024-03-28T18:52:17.425 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:52:17 smithi134 ceph-mon[35037]: pgmap v1405: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:52:17.433 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:52:17 smithi079 ceph-mon[35312]: pgmap v1405: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:52:17.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:52:17 smithi022 ceph-mon[31985]: pgmap v1405: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:52:19.433 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:52:19 smithi079 ceph-mon[35312]: pgmap v1406: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:52:19.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:52:19 smithi022 ceph-mon[31985]: pgmap v1406: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:52:19.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:52:19 smithi134 ceph-mon[35037]: pgmap v1406: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:52:20.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:52:20 smithi022 ceph-mon[31985]: 2 slow requests, 0 included below; oldest blocked for > 2126.872720 secs 2024-03-28T18:52:20.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:52:20 smithi134 ceph-mon[35037]: 2 slow requests, 0 included below; oldest blocked for > 2126.872720 secs 2024-03-28T18:52:20.683 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:52:20 smithi079 ceph-mon[35312]: 2 slow requests, 0 included below; oldest blocked for > 2126.872720 secs 2024-03-28T18:52:21.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:52:21 smithi022 ceph-mon[31985]: pgmap v1407: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:52:21.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:52:21 smithi134 ceph-mon[35037]: pgmap v1407: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:52:21.683 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:52:21 smithi079 ceph-mon[35312]: pgmap v1407: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:52:23.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:52:23 smithi022 ceph-mon[31985]: pgmap v1408: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:52:23.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:52:23 smithi134 ceph-mon[35037]: pgmap v1408: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:52:23.683 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:52:23 smithi079 ceph-mon[35312]: pgmap v1408: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:52:25.485 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:52:25 smithi079 ceph-mon[35312]: pgmap v1409: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:52:25.486 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:52:25 smithi079 ceph-mon[35312]: 2 slow requests, 0 included below; oldest blocked for > 2131.872783 secs 2024-03-28T18:52:25.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:52:25 smithi022 ceph-mon[31985]: pgmap v1409: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:52:25.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:52:25 smithi022 ceph-mon[31985]: 2 slow requests, 0 included below; oldest blocked for > 2131.872783 secs 2024-03-28T18:52:25.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:52:25 smithi134 ceph-mon[35037]: pgmap v1409: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:52:25.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:52:25 smithi134 ceph-mon[35037]: 2 slow requests, 0 included below; oldest blocked for > 2131.872783 secs 2024-03-28T18:52:27.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:52:27 smithi022 ceph-mon[31985]: pgmap v1410: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:52:27.674 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:52:27 smithi134 ceph-mon[35037]: pgmap v1410: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:52:27.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:52:27 smithi079 ceph-mon[35312]: pgmap v1410: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:52:29.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:52:29 smithi022 ceph-mon[31985]: pgmap v1411: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:52:29.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:52:29 smithi134 ceph-mon[35037]: pgmap v1411: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:52:29.683 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:52:29 smithi079 ceph-mon[35312]: pgmap v1411: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:52:30.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:52:30 smithi022 ceph-mon[31985]: 2 slow requests, 0 included below; oldest blocked for > 2136.872858 secs 2024-03-28T18:52:30.674 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:52:30 smithi134 ceph-mon[35037]: 2 slow requests, 0 included below; oldest blocked for > 2136.872858 secs 2024-03-28T18:52:30.683 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:52:30 smithi079 ceph-mon[35312]: 2 slow requests, 0 included below; oldest blocked for > 2136.872858 secs 2024-03-28T18:52:31.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:52:31 smithi022 ceph-mon[31985]: pgmap v1412: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:52:31.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:52:31 smithi022 ceph-mon[31985]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T18:52:31.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:52:31 smithi134 ceph-mon[35037]: pgmap v1412: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:52:31.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:52:31 smithi134 ceph-mon[35037]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T18:52:31.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:52:31 smithi079 ceph-mon[35312]: pgmap v1412: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:52:31.683 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:52:31 smithi079 ceph-mon[35312]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T18:52:32.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:52:32 smithi022 ceph-mon[31985]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T18:52:32.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:52:32 smithi022 ceph-mon[31985]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T18:52:32.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:52:32 smithi022 ceph-mon[31985]: from='mgr.15522 ' entity='mgr.x' 2024-03-28T18:52:32.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:52:32 smithi134 ceph-mon[35037]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T18:52:32.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:52:32 smithi134 ceph-mon[35037]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T18:52:32.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:52:32 smithi134 ceph-mon[35037]: from='mgr.15522 ' entity='mgr.x' 2024-03-28T18:52:32.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:52:32 smithi079 ceph-mon[35312]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T18:52:32.683 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:52:32 smithi079 ceph-mon[35312]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T18:52:32.683 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:52:32 smithi079 ceph-mon[35312]: from='mgr.15522 ' entity='mgr.x' 2024-03-28T18:52:33.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:52:33 smithi022 ceph-mon[31985]: pgmap v1413: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 6 op/s 2024-03-28T18:52:33.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:52:33 smithi022 ceph-mon[31985]: pgmap v1414: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-03-28T18:52:33.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:52:33 smithi134 ceph-mon[35037]: pgmap v1413: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 6 op/s 2024-03-28T18:52:33.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:52:33 smithi134 ceph-mon[35037]: pgmap v1414: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-03-28T18:52:33.683 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:52:33 smithi079 ceph-mon[35312]: pgmap v1413: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 6 op/s 2024-03-28T18:52:33.683 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:52:33 smithi079 ceph-mon[35312]: pgmap v1414: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-03-28T18:52:35.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:52:35 smithi022 ceph-mon[31985]: pgmap v1415: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-03-28T18:52:35.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:52:35 smithi022 ceph-mon[31985]: 2 slow requests, 0 included below; oldest blocked for > 2141.872961 secs 2024-03-28T18:52:35.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:52:35 smithi134 ceph-mon[35037]: pgmap v1415: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-03-28T18:52:35.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:52:35 smithi134 ceph-mon[35037]: 2 slow requests, 0 included below; oldest blocked for > 2141.872961 secs 2024-03-28T18:52:35.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:52:35 smithi079 ceph-mon[35312]: pgmap v1415: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-03-28T18:52:35.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:52:35 smithi079 ceph-mon[35312]: 2 slow requests, 0 included below; oldest blocked for > 2141.872961 secs 2024-03-28T18:52:37.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:52:37 smithi022 ceph-mon[31985]: pgmap v1416: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-03-28T18:52:37.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:52:37 smithi134 ceph-mon[35037]: pgmap v1416: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-03-28T18:52:37.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:52:37 smithi079 ceph-mon[35312]: pgmap v1416: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-03-28T18:52:39.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:52:39 smithi022 ceph-mon[31985]: pgmap v1417: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-03-28T18:52:39.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:52:39 smithi134 ceph-mon[35037]: pgmap v1417: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-03-28T18:52:39.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:52:39 smithi079 ceph-mon[35312]: pgmap v1417: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-03-28T18:52:40.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:52:40 smithi022 ceph-mon[31985]: 2 slow requests, 0 included below; oldest blocked for > 2146.873038 secs 2024-03-28T18:52:40.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:52:40 smithi134 ceph-mon[35037]: 2 slow requests, 0 included below; oldest blocked for > 2146.873038 secs 2024-03-28T18:52:40.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:52:40 smithi079 ceph-mon[35312]: 2 slow requests, 0 included below; oldest blocked for > 2146.873038 secs 2024-03-28T18:52:41.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:52:41 smithi022 ceph-mon[31985]: pgmap v1418: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-03-28T18:52:41.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:52:41 smithi134 ceph-mon[35037]: pgmap v1418: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-03-28T18:52:41.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:52:41 smithi079 ceph-mon[35312]: pgmap v1418: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-03-28T18:52:43.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:52:43 smithi134 ceph-mon[35037]: pgmap v1419: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 7 op/s 2024-03-28T18:52:43.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:52:43 smithi079 ceph-mon[35312]: pgmap v1419: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 7 op/s 2024-03-28T18:52:43.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:52:43 smithi022 ceph-mon[31985]: pgmap v1419: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 7 op/s 2024-03-28T18:52:45.585 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:52:45 smithi022 ceph-mon[31985]: pgmap v1420: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:52:45.585 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:52:45 smithi022 ceph-mon[31985]: 2 slow requests, 0 included below; oldest blocked for > 2151.873056 secs 2024-03-28T18:52:45.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:52:45 smithi134 ceph-mon[35037]: pgmap v1420: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:52:45.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:52:45 smithi134 ceph-mon[35037]: 2 slow requests, 0 included below; oldest blocked for > 2151.873056 secs 2024-03-28T18:52:45.683 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:52:45 smithi079 ceph-mon[35312]: pgmap v1420: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:52:45.683 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:52:45 smithi079 ceph-mon[35312]: 2 slow requests, 0 included below; oldest blocked for > 2151.873056 secs 2024-03-28T18:52:47.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:52:47 smithi134 ceph-mon[35037]: pgmap v1421: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:52:47.683 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:52:47 smithi079 ceph-mon[35312]: pgmap v1421: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:52:47.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:52:47 smithi022 ceph-mon[31985]: pgmap v1421: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:52:49.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:52:49 smithi134 ceph-mon[35037]: pgmap v1422: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:52:49.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:52:49 smithi079 ceph-mon[35312]: pgmap v1422: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:52:49.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:52:49 smithi022 ceph-mon[31985]: pgmap v1422: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:52:50.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:52:50 smithi134 ceph-mon[35037]: 2 slow requests, 0 included below; oldest blocked for > 2156.873179 secs 2024-03-28T18:52:50.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:52:50 smithi079 ceph-mon[35312]: 2 slow requests, 0 included below; oldest blocked for > 2156.873179 secs 2024-03-28T18:52:50.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:52:50 smithi022 ceph-mon[31985]: 2 slow requests, 0 included below; oldest blocked for > 2156.873179 secs 2024-03-28T18:52:51.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:52:51 smithi134 ceph-mon[35037]: pgmap v1423: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:52:51.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:52:51 smithi079 ceph-mon[35312]: pgmap v1423: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:52:51.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:52:51 smithi022 ceph-mon[31985]: pgmap v1423: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:52:53.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:52:53 smithi134 ceph-mon[35037]: pgmap v1424: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:52:53.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:52:53 smithi079 ceph-mon[35312]: pgmap v1424: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:52:53.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:52:53 smithi022 ceph-mon[31985]: pgmap v1424: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:52:55.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:52:55 smithi134 ceph-mon[35037]: pgmap v1425: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:52:55.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:52:55 smithi134 ceph-mon[35037]: 2 slow requests, 0 included below; oldest blocked for > 2161.873244 secs 2024-03-28T18:52:55.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:52:55 smithi079 ceph-mon[35312]: pgmap v1425: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:52:55.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:52:55 smithi079 ceph-mon[35312]: 2 slow requests, 0 included below; oldest blocked for > 2161.873244 secs 2024-03-28T18:52:55.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:52:55 smithi022 ceph-mon[31985]: pgmap v1425: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:52:55.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:52:55 smithi022 ceph-mon[31985]: 2 slow requests, 0 included below; oldest blocked for > 2161.873244 secs 2024-03-28T18:52:57.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:52:57 smithi134 ceph-mon[35037]: pgmap v1426: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:52:57.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:52:57 smithi079 ceph-mon[35312]: pgmap v1426: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:52:57.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:52:57 smithi022 ceph-mon[31985]: pgmap v1426: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:52:59.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:52:59 smithi134 ceph-mon[35037]: pgmap v1427: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:52:59.683 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:52:59 smithi079 ceph-mon[35312]: pgmap v1427: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:52:59.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:52:59 smithi022 ceph-mon[31985]: pgmap v1427: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:53:00.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:53:00 smithi134 ceph-mon[35037]: 2 slow requests, 0 included below; oldest blocked for > 2166.873309 secs 2024-03-28T18:53:00.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:53:00 smithi079 ceph-mon[35312]: 2 slow requests, 0 included below; oldest blocked for > 2166.873309 secs 2024-03-28T18:53:00.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:53:00 smithi022 ceph-mon[31985]: 2 slow requests, 0 included below; oldest blocked for > 2166.873309 secs 2024-03-28T18:53:01.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:53:01 smithi134 ceph-mon[35037]: pgmap v1428: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:53:01.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:53:01 smithi079 ceph-mon[35312]: pgmap v1428: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:53:01.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:53:01 smithi022 ceph-mon[31985]: pgmap v1428: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:53:03.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:53:03 smithi134 ceph-mon[35037]: pgmap v1429: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:53:03.683 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:53:03 smithi079 ceph-mon[35312]: pgmap v1429: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:53:03.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:53:03 smithi022 ceph-mon[31985]: pgmap v1429: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:53:05.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:53:05 smithi134 ceph-mon[35037]: pgmap v1430: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:53:05.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:53:05 smithi134 ceph-mon[35037]: 2 slow requests, 0 included below; oldest blocked for > 2171.873416 secs 2024-03-28T18:53:05.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:53:05 smithi079 ceph-mon[35312]: pgmap v1430: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:53:05.683 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:53:05 smithi079 ceph-mon[35312]: 2 slow requests, 0 included below; oldest blocked for > 2171.873416 secs 2024-03-28T18:53:05.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:53:05 smithi022 ceph-mon[31985]: pgmap v1430: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:53:05.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:53:05 smithi022 ceph-mon[31985]: 2 slow requests, 0 included below; oldest blocked for > 2171.873416 secs 2024-03-28T18:53:07.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:53:07 smithi134 ceph-mon[35037]: pgmap v1431: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:53:07.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:53:07 smithi079 ceph-mon[35312]: pgmap v1431: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:53:07.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:53:07 smithi022 ceph-mon[31985]: pgmap v1431: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:53:09.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:53:09 smithi134 ceph-mon[35037]: pgmap v1432: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:53:09.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:53:09 smithi079 ceph-mon[35312]: pgmap v1432: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:53:09.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:53:09 smithi022 ceph-mon[31985]: pgmap v1432: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:53:10.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:53:10 smithi134 ceph-mon[35037]: 2 slow requests, 0 included below; oldest blocked for > 2176.873447 secs 2024-03-28T18:53:10.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:53:10 smithi079 ceph-mon[35312]: 2 slow requests, 0 included below; oldest blocked for > 2176.873447 secs 2024-03-28T18:53:10.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:53:10 smithi022 ceph-mon[31985]: 2 slow requests, 0 included below; oldest blocked for > 2176.873447 secs 2024-03-28T18:53:11.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:53:11 smithi134 ceph-mon[35037]: pgmap v1433: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:53:11.683 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:53:11 smithi079 ceph-mon[35312]: pgmap v1433: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:53:11.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:53:11 smithi022 ceph-mon[31985]: pgmap v1433: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:53:13.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:53:13 smithi134 ceph-mon[35037]: pgmap v1434: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:53:13.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:53:13 smithi079 ceph-mon[35312]: pgmap v1434: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:53:13.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:53:13 smithi022 ceph-mon[31985]: pgmap v1434: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:53:15.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:53:15 smithi134 ceph-mon[35037]: pgmap v1435: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:53:15.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:53:15 smithi134 ceph-mon[35037]: 2 slow requests, 0 included below; oldest blocked for > 2181.873530 secs 2024-03-28T18:53:15.683 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:53:15 smithi079 ceph-mon[35312]: pgmap v1435: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:53:15.683 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:53:15 smithi079 ceph-mon[35312]: 2 slow requests, 0 included below; oldest blocked for > 2181.873530 secs 2024-03-28T18:53:15.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:53:15 smithi022 ceph-mon[31985]: pgmap v1435: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:53:15.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:53:15 smithi022 ceph-mon[31985]: 2 slow requests, 0 included below; oldest blocked for > 2181.873530 secs 2024-03-28T18:53:17.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:53:17 smithi134 ceph-mon[35037]: pgmap v1436: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 9 op/s 2024-03-28T18:53:17.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:53:17 smithi079 ceph-mon[35312]: pgmap v1436: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 9 op/s 2024-03-28T18:53:17.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:53:17 smithi022 ceph-mon[31985]: pgmap v1436: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 9 op/s 2024-03-28T18:53:19.683 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:53:19 smithi079 ceph-mon[35312]: pgmap v1437: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T18:53:19.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:53:19 smithi022 ceph-mon[31985]: pgmap v1437: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T18:53:19.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:53:19 smithi134 ceph-mon[35037]: pgmap v1437: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T18:53:20.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:53:20 smithi022 ceph-mon[31985]: 2 slow requests, 0 included below; oldest blocked for > 2186.873651 secs 2024-03-28T18:53:20.925 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:53:20 smithi134 ceph-mon[35037]: 2 slow requests, 0 included below; oldest blocked for > 2186.873651 secs 2024-03-28T18:53:20.933 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:53:20 smithi079 ceph-mon[35312]: 2 slow requests, 0 included below; oldest blocked for > 2186.873651 secs 2024-03-28T18:53:21.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:53:21 smithi022 ceph-mon[31985]: pgmap v1438: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 9 op/s 2024-03-28T18:53:21.925 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:53:21 smithi134 ceph-mon[35037]: pgmap v1438: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 9 op/s 2024-03-28T18:53:21.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:53:21 smithi079 ceph-mon[35312]: pgmap v1438: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 9 op/s 2024-03-28T18:53:23.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:53:23 smithi022 ceph-mon[31985]: pgmap v1439: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T18:53:23.925 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:53:23 smithi134 ceph-mon[35037]: pgmap v1439: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T18:53:23.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:53:23 smithi079 ceph-mon[35312]: pgmap v1439: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T18:53:25.735 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:53:25 smithi079 ceph-mon[35312]: pgmap v1440: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T18:53:25.735 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:53:25 smithi079 ceph-mon[35312]: 2 slow requests, 0 included below; oldest blocked for > 2191.873770 secs 2024-03-28T18:53:25.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:53:25 smithi022 ceph-mon[31985]: pgmap v1440: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T18:53:25.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:53:25 smithi022 ceph-mon[31985]: 2 slow requests, 0 included below; oldest blocked for > 2191.873770 secs 2024-03-28T18:53:25.925 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:53:25 smithi134 ceph-mon[35037]: pgmap v1440: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T18:53:25.925 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:53:25 smithi134 ceph-mon[35037]: 2 slow requests, 0 included below; oldest blocked for > 2191.873770 secs 2024-03-28T18:53:27.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:53:27 smithi022 ceph-mon[31985]: pgmap v1441: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 9 op/s 2024-03-28T18:53:27.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:53:27 smithi134 ceph-mon[35037]: pgmap v1441: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 9 op/s 2024-03-28T18:53:27.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:53:27 smithi079 ceph-mon[35312]: pgmap v1441: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 9 op/s 2024-03-28T18:53:28.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:53:28 smithi022 ceph-mon[31985]: pgmap v1442: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:53:28.925 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:53:28 smithi134 ceph-mon[35037]: pgmap v1442: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:53:28.933 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:53:28 smithi079 ceph-mon[35312]: pgmap v1442: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:53:30.032 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:53:29 smithi022 ceph-mon[31985]: 2 slow requests, 0 included below; oldest blocked for > 2196.873840 secs 2024-03-28T18:53:30.174 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:53:29 smithi134 ceph-mon[35037]: 2 slow requests, 0 included below; oldest blocked for > 2196.873840 secs 2024-03-28T18:53:30.182 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:53:29 smithi079 ceph-mon[35312]: 2 slow requests, 0 included below; oldest blocked for > 2196.873840 secs 2024-03-28T18:53:31.032 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:53:30 smithi022 ceph-mon[31985]: pgmap v1443: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:53:31.175 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:53:30 smithi134 ceph-mon[35037]: pgmap v1443: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:53:31.182 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:53:30 smithi079 ceph-mon[35312]: pgmap v1443: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:53:33.282 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:53:32 smithi022 ceph-mon[31985]: pgmap v1444: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:53:33.282 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:53:32 smithi022 ceph-mon[31985]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T18:53:33.425 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:53:32 smithi134 ceph-mon[35037]: pgmap v1444: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:53:33.425 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:53:32 smithi134 ceph-mon[35037]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T18:53:33.432 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:53:32 smithi079 ceph-mon[35312]: pgmap v1444: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:53:33.432 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:53:32 smithi079 ceph-mon[35312]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T18:53:34.282 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:53:33 smithi022 ceph-mon[31985]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T18:53:34.282 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:53:33 smithi022 ceph-mon[31985]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T18:53:34.282 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:53:33 smithi022 ceph-mon[31985]: from='mgr.15522 ' entity='mgr.x' 2024-03-28T18:53:34.424 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:53:33 smithi134 ceph-mon[35037]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T18:53:34.425 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:53:33 smithi134 ceph-mon[35037]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T18:53:34.425 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:53:33 smithi134 ceph-mon[35037]: from='mgr.15522 ' entity='mgr.x' 2024-03-28T18:53:34.432 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:53:33 smithi079 ceph-mon[35312]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T18:53:34.432 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:53:33 smithi079 ceph-mon[35312]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T18:53:34.432 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:53:33 smithi079 ceph-mon[35312]: from='mgr.15522 ' entity='mgr.x' 2024-03-28T18:53:35.282 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:53:34 smithi022 ceph-mon[31985]: pgmap v1445: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 6 op/s 2024-03-28T18:53:35.282 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:53:34 smithi022 ceph-mon[31985]: pgmap v1446: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-03-28T18:53:35.282 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:53:34 smithi022 ceph-mon[31985]: 2 slow requests, 0 included below; oldest blocked for > 2201.873915 secs 2024-03-28T18:53:35.425 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:53:34 smithi134 ceph-mon[35037]: pgmap v1445: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 6 op/s 2024-03-28T18:53:35.425 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:53:34 smithi134 ceph-mon[35037]: pgmap v1446: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-03-28T18:53:35.425 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:53:34 smithi134 ceph-mon[35037]: 2 slow requests, 0 included below; oldest blocked for > 2201.873915 secs 2024-03-28T18:53:35.432 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:53:34 smithi079 ceph-mon[35312]: pgmap v1445: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 6 op/s 2024-03-28T18:53:35.432 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:53:34 smithi079 ceph-mon[35312]: pgmap v1446: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-03-28T18:53:35.432 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:53:34 smithi079 ceph-mon[35312]: 2 slow requests, 0 included below; oldest blocked for > 2201.873915 secs 2024-03-28T18:53:37.282 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:53:36 smithi022 ceph-mon[31985]: pgmap v1447: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-03-28T18:53:37.424 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:53:36 smithi134 ceph-mon[35037]: pgmap v1447: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-03-28T18:53:37.432 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:53:36 smithi079 ceph-mon[35312]: pgmap v1447: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-03-28T18:53:39.282 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:53:38 smithi022 ceph-mon[31985]: pgmap v1448: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-03-28T18:53:39.424 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:53:38 smithi134 ceph-mon[35037]: pgmap v1448: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-03-28T18:53:39.432 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:53:38 smithi079 ceph-mon[35312]: pgmap v1448: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-03-28T18:53:40.282 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:53:39 smithi022 ceph-mon[31985]: 2 slow requests, 0 included below; oldest blocked for > 2206.873969 secs 2024-03-28T18:53:40.425 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:53:39 smithi134 ceph-mon[35037]: 2 slow requests, 0 included below; oldest blocked for > 2206.873969 secs 2024-03-28T18:53:40.433 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:53:39 smithi079 ceph-mon[35312]: 2 slow requests, 0 included below; oldest blocked for > 2206.873969 secs 2024-03-28T18:53:41.282 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:53:40 smithi022 ceph-mon[31985]: pgmap v1449: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-03-28T18:53:41.425 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:53:40 smithi134 ceph-mon[35037]: pgmap v1449: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-03-28T18:53:41.433 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:53:40 smithi079 ceph-mon[35312]: pgmap v1449: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-03-28T18:53:43.282 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:53:42 smithi022 ceph-mon[31985]: pgmap v1450: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-03-28T18:53:43.424 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:53:42 smithi134 ceph-mon[35037]: pgmap v1450: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-03-28T18:53:43.432 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:53:42 smithi079 ceph-mon[35312]: pgmap v1450: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-03-28T18:53:45.282 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:53:44 smithi022 ceph-mon[31985]: pgmap v1451: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 7 op/s 2024-03-28T18:53:45.282 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:53:44 smithi022 ceph-mon[31985]: 2 slow requests, 0 included below; oldest blocked for > 2211.874008 secs 2024-03-28T18:53:45.425 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:53:44 smithi134 ceph-mon[35037]: pgmap v1451: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 7 op/s 2024-03-28T18:53:45.425 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:53:44 smithi134 ceph-mon[35037]: 2 slow requests, 0 included below; oldest blocked for > 2211.874008 secs 2024-03-28T18:53:45.433 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:53:44 smithi079 ceph-mon[35312]: pgmap v1451: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 7 op/s 2024-03-28T18:53:45.433 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:53:44 smithi079 ceph-mon[35312]: 2 slow requests, 0 included below; oldest blocked for > 2211.874008 secs 2024-03-28T18:53:47.282 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:53:47 smithi022 ceph-mon[31985]: pgmap v1452: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:53:47.425 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:53:47 smithi134 ceph-mon[35037]: pgmap v1452: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:53:47.432 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:53:47 smithi079 ceph-mon[35312]: pgmap v1452: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:53:49.282 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:53:49 smithi022 ceph-mon[31985]: pgmap v1453: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:53:49.424 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:53:49 smithi134 ceph-mon[35037]: pgmap v1453: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:53:49.432 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:53:49 smithi079 ceph-mon[35312]: pgmap v1453: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:53:50.282 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:53:50 smithi022 ceph-mon[31985]: 2 slow requests, 0 included below; oldest blocked for > 2216.874130 secs 2024-03-28T18:53:50.424 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:53:50 smithi134 ceph-mon[35037]: 2 slow requests, 0 included below; oldest blocked for > 2216.874130 secs 2024-03-28T18:53:50.433 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:53:50 smithi079 ceph-mon[35312]: 2 slow requests, 0 included below; oldest blocked for > 2216.874130 secs 2024-03-28T18:53:51.282 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:53:51 smithi022 ceph-mon[31985]: pgmap v1454: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:53:51.424 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:53:51 smithi134 ceph-mon[35037]: pgmap v1454: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:53:51.432 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:53:51 smithi079 ceph-mon[35312]: pgmap v1454: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:53:53.282 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:53:53 smithi022 ceph-mon[31985]: pgmap v1455: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:53:53.425 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:53:53 smithi134 ceph-mon[35037]: pgmap v1455: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:53:53.432 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:53:53 smithi079 ceph-mon[35312]: pgmap v1455: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:53:55.425 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:53:55 smithi134 ceph-mon[35037]: pgmap v1456: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:53:55.425 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:53:55 smithi134 ceph-mon[35037]: 2 slow requests, 0 included below; oldest blocked for > 2221.874224 secs 2024-03-28T18:53:55.433 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:53:55 smithi079 ceph-mon[35312]: pgmap v1456: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:53:55.433 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:53:55 smithi079 ceph-mon[35312]: 2 slow requests, 0 included below; oldest blocked for > 2221.874224 secs 2024-03-28T18:53:55.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:53:55 smithi022 ceph-mon[31985]: pgmap v1456: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:53:55.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:53:55 smithi022 ceph-mon[31985]: 2 slow requests, 0 included below; oldest blocked for > 2221.874224 secs 2024-03-28T18:53:57.425 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:53:57 smithi134 ceph-mon[35037]: pgmap v1457: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:53:57.432 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:53:57 smithi079 ceph-mon[35312]: pgmap v1457: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:53:57.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:53:57 smithi022 ceph-mon[31985]: pgmap v1457: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:53:59.425 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:53:59 smithi134 ceph-mon[35037]: pgmap v1458: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:53:59.432 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:53:59 smithi079 ceph-mon[35312]: pgmap v1458: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:53:59.531 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:53:59 smithi022 ceph-mon[31985]: pgmap v1458: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:54:00.425 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:54:00 smithi134 ceph-mon[35037]: 2 slow requests, 0 included below; oldest blocked for > 2226.874348 secs 2024-03-28T18:54:00.432 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:54:00 smithi079 ceph-mon[35312]: 2 slow requests, 0 included below; oldest blocked for > 2226.874348 secs 2024-03-28T18:54:00.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:54:00 smithi022 ceph-mon[31985]: 2 slow requests, 0 included below; oldest blocked for > 2226.874348 secs 2024-03-28T18:54:01.425 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:54:01 smithi134 ceph-mon[35037]: pgmap v1459: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:54:01.432 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:54:01 smithi079 ceph-mon[35312]: pgmap v1459: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:54:01.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:54:01 smithi022 ceph-mon[31985]: pgmap v1459: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:54:03.425 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:54:03 smithi134 ceph-mon[35037]: pgmap v1460: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:54:03.432 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:54:03 smithi079 ceph-mon[35312]: pgmap v1460: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:54:03.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:54:03 smithi022 ceph-mon[31985]: pgmap v1460: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:54:05.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:54:05 smithi134 ceph-mon[35037]: pgmap v1461: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:54:05.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:54:05 smithi134 ceph-mon[35037]: 2 slow requests, 0 included below; oldest blocked for > 2231.874415 secs 2024-03-28T18:54:05.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:54:05 smithi079 ceph-mon[35312]: pgmap v1461: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:54:05.683 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:54:05 smithi079 ceph-mon[35312]: 2 slow requests, 0 included below; oldest blocked for > 2231.874415 secs 2024-03-28T18:54:05.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:54:05 smithi022 ceph-mon[31985]: pgmap v1461: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:54:05.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:54:05 smithi022 ceph-mon[31985]: 2 slow requests, 0 included below; oldest blocked for > 2231.874415 secs 2024-03-28T18:54:07.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:54:07 smithi134 ceph-mon[35037]: pgmap v1462: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:54:07.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:54:07 smithi079 ceph-mon[35312]: pgmap v1462: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:54:07.781 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:54:07 smithi022 ceph-mon[31985]: pgmap v1462: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:54:09.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:54:09 smithi134 ceph-mon[35037]: pgmap v1463: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:54:09.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:54:09 smithi079 ceph-mon[35312]: pgmap v1463: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:54:09.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:54:09 smithi022 ceph-mon[31985]: pgmap v1463: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:54:10.674 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:54:10 smithi134 ceph-mon[35037]: 2 slow requests, 0 included below; oldest blocked for > 2236.874495 secs 2024-03-28T18:54:10.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:54:10 smithi079 ceph-mon[35312]: 2 slow requests, 0 included below; oldest blocked for > 2236.874495 secs 2024-03-28T18:54:10.781 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:54:10 smithi022 ceph-mon[31985]: 2 slow requests, 0 included below; oldest blocked for > 2236.874495 secs 2024-03-28T18:54:11.674 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:54:11 smithi134 ceph-mon[35037]: pgmap v1464: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:54:11.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:54:11 smithi079 ceph-mon[35312]: pgmap v1464: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:54:11.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:54:11 smithi022 ceph-mon[31985]: pgmap v1464: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:54:13.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:54:13 smithi134 ceph-mon[35037]: pgmap v1465: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:54:13.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:54:13 smithi079 ceph-mon[35312]: pgmap v1465: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:54:13.781 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:54:13 smithi022 ceph-mon[31985]: pgmap v1465: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:54:15.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:54:15 smithi134 ceph-mon[35037]: pgmap v1466: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:54:15.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:54:15 smithi134 ceph-mon[35037]: 2 slow requests, 0 included below; oldest blocked for > 2241.874570 secs 2024-03-28T18:54:15.683 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:54:15 smithi079 ceph-mon[35312]: pgmap v1466: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:54:15.683 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:54:15 smithi079 ceph-mon[35312]: 2 slow requests, 0 included below; oldest blocked for > 2241.874570 secs 2024-03-28T18:54:15.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:54:15 smithi022 ceph-mon[31985]: pgmap v1466: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:54:15.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:54:15 smithi022 ceph-mon[31985]: 2 slow requests, 0 included below; oldest blocked for > 2241.874570 secs 2024-03-28T18:54:17.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:54:17 smithi134 ceph-mon[35037]: pgmap v1467: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:54:17.683 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:54:17 smithi079 ceph-mon[35312]: pgmap v1467: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:54:17.781 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:54:17 smithi022 ceph-mon[31985]: pgmap v1467: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:54:19.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:54:19 smithi134 ceph-mon[35037]: pgmap v1468: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:54:19.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:54:19 smithi079 ceph-mon[35312]: pgmap v1468: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:54:19.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:54:19 smithi022 ceph-mon[31985]: pgmap v1468: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:54:20.674 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:54:20 smithi134 ceph-mon[35037]: 2 slow requests, 0 included below; oldest blocked for > 2246.874689 secs 2024-03-28T18:54:20.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:54:20 smithi079 ceph-mon[35312]: 2 slow requests, 0 included below; oldest blocked for > 2246.874689 secs 2024-03-28T18:54:20.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:54:20 smithi022 ceph-mon[31985]: 2 slow requests, 0 included below; oldest blocked for > 2246.874689 secs 2024-03-28T18:54:21.683 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:54:21 smithi079 ceph-mon[35312]: pgmap v1469: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:54:21.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:54:21 smithi022 ceph-mon[31985]: pgmap v1469: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:54:21.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:54:21 smithi134 ceph-mon[35037]: pgmap v1469: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:54:23.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:54:23 smithi022 ceph-mon[31985]: pgmap v1470: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:54:23.925 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:54:23 smithi134 ceph-mon[35037]: pgmap v1470: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:54:23.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:54:23 smithi079 ceph-mon[35312]: pgmap v1470: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:54:25.735 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:54:25 smithi079 ceph-mon[35312]: pgmap v1471: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:54:25.735 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:54:25 smithi079 ceph-mon[35312]: 2 slow requests, 0 included below; oldest blocked for > 2251.874810 secs 2024-03-28T18:54:25.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:54:25 smithi022 ceph-mon[31985]: pgmap v1471: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:54:25.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:54:25 smithi022 ceph-mon[31985]: 2 slow requests, 0 included below; oldest blocked for > 2251.874810 secs 2024-03-28T18:54:25.925 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:54:25 smithi134 ceph-mon[35037]: pgmap v1471: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:54:25.925 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:54:25 smithi134 ceph-mon[35037]: 2 slow requests, 0 included below; oldest blocked for > 2251.874810 secs 2024-03-28T18:54:27.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:54:27 smithi022 ceph-mon[31985]: pgmap v1472: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 9 op/s 2024-03-28T18:54:27.925 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:54:27 smithi134 ceph-mon[35037]: pgmap v1472: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 9 op/s 2024-03-28T18:54:27.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:54:27 smithi079 ceph-mon[35312]: pgmap v1472: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 9 op/s 2024-03-28T18:54:29.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:54:29 smithi022 ceph-mon[31985]: pgmap v1473: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T18:54:29.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:54:29 smithi134 ceph-mon[35037]: pgmap v1473: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T18:54:29.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:54:29 smithi079 ceph-mon[35312]: pgmap v1473: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T18:54:30.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:54:30 smithi022 ceph-mon[31985]: 2 slow requests, 0 included below; oldest blocked for > 2256.875009 secs 2024-03-28T18:54:30.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:54:30 smithi134 ceph-mon[35037]: 2 slow requests, 0 included below; oldest blocked for > 2256.875009 secs 2024-03-28T18:54:30.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:54:30 smithi079 ceph-mon[35312]: 2 slow requests, 0 included below; oldest blocked for > 2256.875009 secs 2024-03-28T18:54:31.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:54:31 smithi022 ceph-mon[31985]: pgmap v1474: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 9 op/s 2024-03-28T18:54:31.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:54:31 smithi134 ceph-mon[35037]: pgmap v1474: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 9 op/s 2024-03-28T18:54:31.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:54:31 smithi079 ceph-mon[35312]: pgmap v1474: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 9 op/s 2024-03-28T18:54:33.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:54:33 smithi022 ceph-mon[31985]: pgmap v1475: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T18:54:33.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:54:33 smithi134 ceph-mon[35037]: pgmap v1475: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T18:54:33.933 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:54:33 smithi079 ceph-mon[35312]: pgmap v1475: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T18:54:34.781 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:54:34 smithi022 ceph-mon[31985]: pgmap v1476: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T18:54:34.781 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:54:34 smithi022 ceph-mon[31985]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T18:54:34.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:54:34 smithi134 ceph-mon[35037]: pgmap v1476: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T18:54:34.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:54:34 smithi134 ceph-mon[35037]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T18:54:34.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:54:34 smithi079 ceph-mon[35312]: pgmap v1476: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T18:54:34.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:54:34 smithi079 ceph-mon[35312]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T18:54:35.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:54:35 smithi022 ceph-mon[31985]: 2 slow requests, 0 included below; oldest blocked for > 2261.875079 secs 2024-03-28T18:54:35.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:54:35 smithi022 ceph-mon[31985]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T18:54:35.783 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:54:35 smithi022 ceph-mon[31985]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T18:54:35.783 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:54:35 smithi022 ceph-mon[31985]: pgmap v1477: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 10 op/s 2024-03-28T18:54:35.783 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:54:35 smithi022 ceph-mon[31985]: pgmap v1478: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-03-28T18:54:35.783 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:54:35 smithi022 ceph-mon[31985]: from='mgr.15522 ' entity='mgr.x' 2024-03-28T18:54:35.925 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:54:35 smithi134 ceph-mon[35037]: 2 slow requests, 0 included below; oldest blocked for > 2261.875079 secs 2024-03-28T18:54:35.925 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:54:35 smithi134 ceph-mon[35037]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T18:54:35.925 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:54:35 smithi134 ceph-mon[35037]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T18:54:35.925 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:54:35 smithi134 ceph-mon[35037]: pgmap v1477: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 10 op/s 2024-03-28T18:54:35.925 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:54:35 smithi134 ceph-mon[35037]: pgmap v1478: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-03-28T18:54:35.925 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:54:35 smithi134 ceph-mon[35037]: from='mgr.15522 ' entity='mgr.x' 2024-03-28T18:54:35.933 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:54:35 smithi079 ceph-mon[35312]: 2 slow requests, 0 included below; oldest blocked for > 2261.875079 secs 2024-03-28T18:54:35.933 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:54:35 smithi079 ceph-mon[35312]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T18:54:35.933 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:54:35 smithi079 ceph-mon[35312]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T18:54:35.933 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:54:35 smithi079 ceph-mon[35312]: pgmap v1477: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 10 op/s 2024-03-28T18:54:35.933 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:54:35 smithi079 ceph-mon[35312]: pgmap v1478: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-03-28T18:54:35.933 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:54:35 smithi079 ceph-mon[35312]: from='mgr.15522 ' entity='mgr.x' 2024-03-28T18:54:38.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:54:38 smithi134 ceph-mon[35037]: pgmap v1479: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-03-28T18:54:38.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:54:38 smithi079 ceph-mon[35312]: pgmap v1479: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-03-28T18:54:38.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:54:38 smithi022 ceph-mon[31985]: pgmap v1479: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-03-28T18:54:40.674 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:54:40 smithi134 ceph-mon[35037]: 2 slow requests, 0 included below; oldest blocked for > 2266.875166 secs 2024-03-28T18:54:40.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:54:40 smithi134 ceph-mon[35037]: pgmap v1480: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 8 op/s 2024-03-28T18:54:40.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:54:40 smithi079 ceph-mon[35312]: 2 slow requests, 0 included below; oldest blocked for > 2266.875166 secs 2024-03-28T18:54:40.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:54:40 smithi079 ceph-mon[35312]: pgmap v1480: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 8 op/s 2024-03-28T18:54:40.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:54:40 smithi022 ceph-mon[31985]: 2 slow requests, 0 included below; oldest blocked for > 2266.875166 secs 2024-03-28T18:54:40.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:54:40 smithi022 ceph-mon[31985]: pgmap v1480: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 8 op/s 2024-03-28T18:54:42.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:54:42 smithi134 ceph-mon[35037]: pgmap v1481: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 8 op/s 2024-03-28T18:54:42.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:54:42 smithi079 ceph-mon[35312]: pgmap v1481: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 8 op/s 2024-03-28T18:54:42.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:54:42 smithi022 ceph-mon[31985]: pgmap v1481: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 8 op/s 2024-03-28T18:54:44.676 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:54:44 smithi134 ceph-mon[35037]: pgmap v1482: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 8 op/s 2024-03-28T18:54:44.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:54:44 smithi079 ceph-mon[35312]: pgmap v1482: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 8 op/s 2024-03-28T18:54:44.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:54:44 smithi022 ceph-mon[31985]: pgmap v1482: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 8 op/s 2024-03-28T18:54:45.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:54:45 smithi134 ceph-mon[35037]: 2 slow requests, 0 included below; oldest blocked for > 2271.875285 secs 2024-03-28T18:54:45.683 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:54:45 smithi079 ceph-mon[35312]: 2 slow requests, 0 included below; oldest blocked for > 2271.875285 secs 2024-03-28T18:54:45.684 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:54:45 smithi022 ceph-mon[31985]: 2 slow requests, 0 included below; oldest blocked for > 2271.875285 secs 2024-03-28T18:54:46.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:54:46 smithi134 ceph-mon[35037]: pgmap v1483: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 7 op/s 2024-03-28T18:54:46.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:54:46 smithi079 ceph-mon[35312]: pgmap v1483: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 7 op/s 2024-03-28T18:54:46.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:54:46 smithi022 ceph-mon[31985]: pgmap v1483: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 7 op/s 2024-03-28T18:54:48.683 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:54:48 smithi079 ceph-mon[35312]: pgmap v1484: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T18:54:48.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:54:48 smithi022 ceph-mon[31985]: pgmap v1484: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T18:54:48.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:54:48 smithi134 ceph-mon[35037]: pgmap v1484: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T18:54:50.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:54:50 smithi022 ceph-mon[31985]: 2 slow requests, 0 included below; oldest blocked for > 2276.875407 secs 2024-03-28T18:54:50.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:54:50 smithi022 ceph-mon[31985]: pgmap v1485: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 9 op/s 2024-03-28T18:54:50.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:54:50 smithi134 ceph-mon[35037]: 2 slow requests, 0 included below; oldest blocked for > 2276.875407 secs 2024-03-28T18:54:50.925 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:54:50 smithi134 ceph-mon[35037]: pgmap v1485: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 9 op/s 2024-03-28T18:54:50.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:54:50 smithi079 ceph-mon[35312]: 2 slow requests, 0 included below; oldest blocked for > 2276.875407 secs 2024-03-28T18:54:50.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:54:50 smithi079 ceph-mon[35312]: pgmap v1485: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 9 op/s 2024-03-28T18:54:52.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:54:52 smithi022 ceph-mon[31985]: pgmap v1486: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:54:52.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:54:52 smithi134 ceph-mon[35037]: pgmap v1486: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:54:52.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:54:52 smithi079 ceph-mon[35312]: pgmap v1486: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:54:54.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:54:54 smithi022 ceph-mon[31985]: pgmap v1487: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:54:54.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:54:54 smithi134 ceph-mon[35037]: pgmap v1487: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:54:54.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:54:54 smithi079 ceph-mon[35312]: pgmap v1487: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:54:55.783 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:54:55 smithi022 ceph-mon[31985]: 2 slow requests, 0 included below; oldest blocked for > 2281.875477 secs 2024-03-28T18:54:55.834 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:54:55 smithi079 ceph-mon[35312]: 2 slow requests, 0 included below; oldest blocked for > 2281.875477 secs 2024-03-28T18:54:55.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:54:55 smithi134 ceph-mon[35037]: 2 slow requests, 0 included below; oldest blocked for > 2281.875477 secs 2024-03-28T18:54:56.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:54:56 smithi022 ceph-mon[31985]: pgmap v1488: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:54:56.925 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:54:56 smithi134 ceph-mon[35037]: pgmap v1488: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:54:56.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:54:56 smithi079 ceph-mon[35312]: pgmap v1488: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:54:58.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:54:58 smithi022 ceph-mon[31985]: pgmap v1489: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:54:58.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:54:58 smithi134 ceph-mon[35037]: pgmap v1489: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:54:58.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:54:58 smithi079 ceph-mon[35312]: pgmap v1489: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:55:00.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:55:00 smithi022 ceph-mon[31985]: 2 slow requests, 0 included below; oldest blocked for > 2286.875555 secs 2024-03-28T18:55:00.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:55:00 smithi022 ceph-mon[31985]: pgmap v1490: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:55:00.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:55:00 smithi134 ceph-mon[35037]: 2 slow requests, 0 included below; oldest blocked for > 2286.875555 secs 2024-03-28T18:55:00.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:55:00 smithi134 ceph-mon[35037]: pgmap v1490: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:55:00.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:55:00 smithi079 ceph-mon[35312]: 2 slow requests, 0 included below; oldest blocked for > 2286.875555 secs 2024-03-28T18:55:00.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:55:00 smithi079 ceph-mon[35312]: pgmap v1490: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:55:02.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:55:02 smithi022 ceph-mon[31985]: pgmap v1491: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:55:02.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:55:02 smithi134 ceph-mon[35037]: pgmap v1491: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:55:02.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:55:02 smithi079 ceph-mon[35312]: pgmap v1491: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:55:03.781 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:55:03 smithi022 ceph-mon[31985]: pgmap v1492: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:55:03.925 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:55:03 smithi134 ceph-mon[35037]: pgmap v1492: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:55:03.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:55:03 smithi079 ceph-mon[35312]: pgmap v1492: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:55:05.032 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:55:04 smithi022 ceph-mon[31985]: 2 slow requests, 0 included below; oldest blocked for > 2291.875631 secs 2024-03-28T18:55:05.175 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:55:04 smithi134 ceph-mon[35037]: 2 slow requests, 0 included below; oldest blocked for > 2291.875631 secs 2024-03-28T18:55:05.182 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:55:04 smithi079 ceph-mon[35312]: 2 slow requests, 0 included below; oldest blocked for > 2291.875631 secs 2024-03-28T18:55:06.032 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:55:05 smithi022 ceph-mon[31985]: pgmap v1493: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:55:06.174 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:55:05 smithi134 ceph-mon[35037]: pgmap v1493: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:55:06.182 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:55:05 smithi079 ceph-mon[35312]: pgmap v1493: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:55:08.674 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:55:08 smithi134 ceph-mon[35037]: pgmap v1494: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:55:08.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:55:08 smithi079 ceph-mon[35312]: pgmap v1494: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:55:08.781 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:55:08 smithi022 ceph-mon[31985]: pgmap v1494: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:55:10.674 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:55:10 smithi134 ceph-mon[35037]: 2 slow requests, 0 included below; oldest blocked for > 2296.875750 secs 2024-03-28T18:55:10.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:55:10 smithi134 ceph-mon[35037]: pgmap v1495: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:55:10.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:55:10 smithi079 ceph-mon[35312]: 2 slow requests, 0 included below; oldest blocked for > 2296.875750 secs 2024-03-28T18:55:10.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:55:10 smithi079 ceph-mon[35312]: pgmap v1495: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:55:10.781 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:55:10 smithi022 ceph-mon[31985]: 2 slow requests, 0 included below; oldest blocked for > 2296.875750 secs 2024-03-28T18:55:10.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:55:10 smithi022 ceph-mon[31985]: pgmap v1495: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:55:12.674 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:55:12 smithi134 ceph-mon[35037]: pgmap v1496: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:55:12.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:55:12 smithi079 ceph-mon[35312]: pgmap v1496: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:55:12.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:55:12 smithi022 ceph-mon[31985]: pgmap v1496: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:55:14.674 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:55:14 smithi134 ceph-mon[35037]: pgmap v1497: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:55:14.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:55:14 smithi079 ceph-mon[35312]: pgmap v1497: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:55:14.781 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:55:14 smithi022 ceph-mon[31985]: pgmap v1497: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:55:15.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:55:15 smithi079 ceph-mon[35312]: 2 slow requests, 0 included below; oldest blocked for > 2301.875811 secs 2024-03-28T18:55:15.684 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:55:15 smithi022 ceph-mon[31985]: 2 slow requests, 0 included below; oldest blocked for > 2301.875811 secs 2024-03-28T18:55:15.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:55:15 smithi134 ceph-mon[35037]: 2 slow requests, 0 included below; oldest blocked for > 2301.875811 secs 2024-03-28T18:55:16.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:55:16 smithi022 ceph-mon[31985]: pgmap v1498: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:55:16.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:55:16 smithi134 ceph-mon[35037]: pgmap v1498: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:55:16.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:55:16 smithi079 ceph-mon[35312]: pgmap v1498: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:55:18.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:55:18 smithi022 ceph-mon[31985]: pgmap v1499: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:55:18.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:55:18 smithi134 ceph-mon[35037]: pgmap v1499: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:55:18.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:55:18 smithi079 ceph-mon[35312]: pgmap v1499: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:55:20.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:55:20 smithi022 ceph-mon[31985]: 2 slow requests, 0 included below; oldest blocked for > 2306.875907 secs 2024-03-28T18:55:20.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:55:20 smithi022 ceph-mon[31985]: pgmap v1500: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:55:20.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:55:20 smithi134 ceph-mon[35037]: 2 slow requests, 0 included below; oldest blocked for > 2306.875907 secs 2024-03-28T18:55:20.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:55:20 smithi134 ceph-mon[35037]: pgmap v1500: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:55:20.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:55:20 smithi079 ceph-mon[35312]: 2 slow requests, 0 included below; oldest blocked for > 2306.875907 secs 2024-03-28T18:55:20.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:55:20 smithi079 ceph-mon[35312]: pgmap v1500: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:55:22.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:55:22 smithi022 ceph-mon[31985]: pgmap v1501: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:55:22.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:55:22 smithi134 ceph-mon[35037]: pgmap v1501: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:55:22.933 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:55:22 smithi079 ceph-mon[35312]: pgmap v1501: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:55:24.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:55:24 smithi022 ceph-mon[31985]: pgmap v1502: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:55:24.925 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:55:24 smithi134 ceph-mon[35037]: pgmap v1502: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:55:24.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:55:24 smithi079 ceph-mon[35312]: pgmap v1502: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:55:25.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:55:25 smithi022 ceph-mon[31985]: 2 slow requests, 0 included below; oldest blocked for > 2311.875971 secs 2024-03-28T18:55:25.835 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:55:25 smithi079 ceph-mon[35312]: 2 slow requests, 0 included below; oldest blocked for > 2311.875971 secs 2024-03-28T18:55:25.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:55:25 smithi134 ceph-mon[35037]: 2 slow requests, 0 included below; oldest blocked for > 2311.875971 secs 2024-03-28T18:55:26.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:55:26 smithi022 ceph-mon[31985]: pgmap v1503: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:55:26.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:55:26 smithi134 ceph-mon[35037]: pgmap v1503: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:55:26.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:55:26 smithi079 ceph-mon[35312]: pgmap v1503: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:55:28.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:55:28 smithi022 ceph-mon[31985]: pgmap v1504: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:55:28.925 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:55:28 smithi134 ceph-mon[35037]: pgmap v1504: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:55:28.933 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:55:28 smithi079 ceph-mon[35312]: pgmap v1504: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:55:29.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:55:29 smithi022 ceph-mon[31985]: pgmap v1505: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:55:29.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:55:29 smithi134 ceph-mon[35037]: pgmap v1505: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:55:29.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:55:29 smithi079 ceph-mon[35312]: pgmap v1505: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:55:30.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:55:30 smithi022 ceph-mon[31985]: 2 slow requests, 0 included below; oldest blocked for > 2316.876040 secs 2024-03-28T18:55:30.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:55:30 smithi134 ceph-mon[35037]: 2 slow requests, 0 included below; oldest blocked for > 2316.876040 secs 2024-03-28T18:55:30.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:55:30 smithi079 ceph-mon[35312]: 2 slow requests, 0 included below; oldest blocked for > 2316.876040 secs 2024-03-28T18:55:31.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:55:31 smithi022 ceph-mon[31985]: pgmap v1506: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:55:31.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:55:31 smithi134 ceph-mon[35037]: pgmap v1506: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:55:31.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:55:31 smithi079 ceph-mon[35312]: pgmap v1506: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:55:34.674 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:55:34 smithi134 ceph-mon[35037]: pgmap v1507: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:55:34.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:55:34 smithi079 ceph-mon[35312]: pgmap v1507: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:55:34.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:55:34 smithi022 ceph-mon[31985]: pgmap v1507: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:55:35.683 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:55:35 smithi079 ceph-mon[35312]: 2 slow requests, 0 included below; oldest blocked for > 2321.876070 secs 2024-03-28T18:55:35.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:55:35 smithi022 ceph-mon[31985]: 2 slow requests, 0 included below; oldest blocked for > 2321.876070 secs 2024-03-28T18:55:35.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:55:35 smithi134 ceph-mon[35037]: 2 slow requests, 0 included below; oldest blocked for > 2321.876070 secs 2024-03-28T18:55:36.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:55:36 smithi079 ceph-mon[35312]: pgmap v1508: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:55:36.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:55:36 smithi079 ceph-mon[35312]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T18:55:36.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:55:36 smithi022 ceph-mon[31985]: pgmap v1508: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:55:36.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:55:36 smithi022 ceph-mon[31985]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T18:55:36.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:55:36 smithi134 ceph-mon[35037]: pgmap v1508: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:55:36.925 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:55:36 smithi134 ceph-mon[35037]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T18:55:37.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:55:37 smithi022 ceph-mon[31985]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T18:55:37.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:55:37 smithi022 ceph-mon[31985]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T18:55:37.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:55:37 smithi022 ceph-mon[31985]: from='mgr.15522 ' entity='mgr.x' 2024-03-28T18:55:37.925 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:55:37 smithi134 ceph-mon[35037]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T18:55:37.925 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:55:37 smithi134 ceph-mon[35037]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T18:55:37.925 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:55:37 smithi134 ceph-mon[35037]: from='mgr.15522 ' entity='mgr.x' 2024-03-28T18:55:37.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:55:37 smithi079 ceph-mon[35312]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T18:55:37.933 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:55:37 smithi079 ceph-mon[35312]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T18:55:37.933 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:55:37 smithi079 ceph-mon[35312]: from='mgr.15522 ' entity='mgr.x' 2024-03-28T18:55:38.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:55:38 smithi022 ceph-mon[31985]: pgmap v1509: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 6 op/s 2024-03-28T18:55:38.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:55:38 smithi022 ceph-mon[31985]: pgmap v1510: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-03-28T18:55:38.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:55:38 smithi134 ceph-mon[35037]: pgmap v1509: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 6 op/s 2024-03-28T18:55:38.925 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:55:38 smithi134 ceph-mon[35037]: pgmap v1510: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-03-28T18:55:38.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:55:38 smithi079 ceph-mon[35312]: pgmap v1509: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 6 op/s 2024-03-28T18:55:38.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:55:38 smithi079 ceph-mon[35312]: pgmap v1510: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-03-28T18:55:40.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:55:40 smithi022 ceph-mon[31985]: pgmap v1511: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s rd, 4 op/s 2024-03-28T18:55:40.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:55:40 smithi022 ceph-mon[31985]: 2 slow requests, 0 included below; oldest blocked for > 2326.876119 secs 2024-03-28T18:55:40.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:55:40 smithi134 ceph-mon[35037]: pgmap v1511: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s rd, 4 op/s 2024-03-28T18:55:40.925 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:55:40 smithi134 ceph-mon[35037]: 2 slow requests, 0 included below; oldest blocked for > 2326.876119 secs 2024-03-28T18:55:40.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:55:40 smithi079 ceph-mon[35312]: pgmap v1511: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s rd, 4 op/s 2024-03-28T18:55:40.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:55:40 smithi079 ceph-mon[35312]: 2 slow requests, 0 included below; oldest blocked for > 2326.876119 secs 2024-03-28T18:55:42.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:55:42 smithi022 ceph-mon[31985]: pgmap v1512: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-03-28T18:55:42.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:55:42 smithi134 ceph-mon[35037]: pgmap v1512: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-03-28T18:55:42.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:55:42 smithi079 ceph-mon[35312]: pgmap v1512: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-03-28T18:55:44.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:55:44 smithi022 ceph-mon[31985]: pgmap v1513: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-03-28T18:55:44.925 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:55:44 smithi134 ceph-mon[35037]: pgmap v1513: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-03-28T18:55:44.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:55:44 smithi079 ceph-mon[35312]: pgmap v1513: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-03-28T18:55:45.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:55:45 smithi022 ceph-mon[31985]: 2 slow requests, 0 included below; oldest blocked for > 2331.876235 secs 2024-03-28T18:55:45.826 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:55:45 smithi134 ceph-mon[35037]: 2 slow requests, 0 included below; oldest blocked for > 2331.876235 secs 2024-03-28T18:55:45.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:55:45 smithi079 ceph-mon[35312]: 2 slow requests, 0 included below; oldest blocked for > 2331.876235 secs 2024-03-28T18:55:46.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:55:46 smithi022 ceph-mon[31985]: pgmap v1514: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-03-28T18:55:46.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:55:46 smithi134 ceph-mon[35037]: pgmap v1514: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-03-28T18:55:46.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:55:46 smithi079 ceph-mon[35312]: pgmap v1514: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-03-28T18:55:48.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:55:48 smithi022 ceph-mon[31985]: pgmap v1515: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 7 op/s 2024-03-28T18:55:48.925 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:55:48 smithi134 ceph-mon[35037]: pgmap v1515: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 7 op/s 2024-03-28T18:55:48.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:55:48 smithi079 ceph-mon[35312]: pgmap v1515: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 7 op/s 2024-03-28T18:55:50.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:55:50 smithi022 ceph-mon[31985]: pgmap v1516: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:55:50.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:55:50 smithi022 ceph-mon[31985]: 2 slow requests, 0 included below; oldest blocked for > 2336.876308 secs 2024-03-28T18:55:50.925 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:55:50 smithi134 ceph-mon[35037]: pgmap v1516: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:55:50.925 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:55:50 smithi134 ceph-mon[35037]: 2 slow requests, 0 included below; oldest blocked for > 2336.876308 secs 2024-03-28T18:55:50.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:55:50 smithi079 ceph-mon[35312]: pgmap v1516: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:55:50.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:55:50 smithi079 ceph-mon[35312]: 2 slow requests, 0 included below; oldest blocked for > 2336.876308 secs 2024-03-28T18:55:51.781 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:55:51 smithi022 ceph-mon[31985]: pgmap v1517: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:55:51.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:55:51 smithi134 ceph-mon[35037]: pgmap v1517: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:55:51.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:55:51 smithi079 ceph-mon[35312]: pgmap v1517: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:55:54.425 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:55:54 smithi134 ceph-mon[35037]: pgmap v1518: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:55:54.432 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:55:54 smithi079 ceph-mon[35312]: pgmap v1518: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:55:54.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:55:54 smithi022 ceph-mon[31985]: pgmap v1518: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:55:55.426 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:55:55 smithi134 ceph-mon[35037]: 2 slow requests, 0 included below; oldest blocked for > 2341.876378 secs 2024-03-28T18:55:55.432 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:55:55 smithi079 ceph-mon[35312]: 2 slow requests, 0 included below; oldest blocked for > 2341.876378 secs 2024-03-28T18:55:55.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:55:55 smithi022 ceph-mon[31985]: 2 slow requests, 0 included below; oldest blocked for > 2341.876378 secs 2024-03-28T18:55:56.432 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:55:56 smithi079 ceph-mon[35312]: pgmap v1519: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:55:56.531 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:55:56 smithi022 ceph-mon[31985]: pgmap v1519: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:55:56.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:55:56 smithi134 ceph-mon[35037]: pgmap v1519: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:55:58.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:55:58 smithi022 ceph-mon[31985]: pgmap v1520: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:55:58.674 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:55:58 smithi134 ceph-mon[35037]: pgmap v1520: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:55:58.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:55:58 smithi079 ceph-mon[35312]: pgmap v1520: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:56:00.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:56:00 smithi022 ceph-mon[31985]: pgmap v1521: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:56:00.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:56:00 smithi022 ceph-mon[31985]: 2 slow requests, 0 included below; oldest blocked for > 2346.876427 secs 2024-03-28T18:56:00.674 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:56:00 smithi134 ceph-mon[35037]: pgmap v1521: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:56:00.674 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:56:00 smithi134 ceph-mon[35037]: 2 slow requests, 0 included below; oldest blocked for > 2346.876427 secs 2024-03-28T18:56:00.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:56:00 smithi079 ceph-mon[35312]: pgmap v1521: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:56:00.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:56:00 smithi079 ceph-mon[35312]: 2 slow requests, 0 included below; oldest blocked for > 2346.876427 secs 2024-03-28T18:56:02.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:56:02 smithi022 ceph-mon[31985]: pgmap v1522: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:56:02.674 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:56:02 smithi134 ceph-mon[35037]: pgmap v1522: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:56:02.683 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:56:02 smithi079 ceph-mon[35312]: pgmap v1522: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:56:04.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:56:04 smithi022 ceph-mon[31985]: pgmap v1523: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:56:04.674 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:56:04 smithi134 ceph-mon[35037]: pgmap v1523: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:56:04.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:56:04 smithi079 ceph-mon[35312]: pgmap v1523: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:56:05.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:56:05 smithi022 ceph-mon[31985]: 2 slow requests, 0 included below; oldest blocked for > 2351.876549 secs 2024-03-28T18:56:05.674 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:56:05 smithi134 ceph-mon[35037]: 2 slow requests, 0 included below; oldest blocked for > 2351.876549 secs 2024-03-28T18:56:05.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:56:05 smithi079 ceph-mon[35312]: 2 slow requests, 0 included below; oldest blocked for > 2351.876549 secs 2024-03-28T18:56:06.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:56:06 smithi022 ceph-mon[31985]: pgmap v1524: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:56:06.674 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:56:06 smithi134 ceph-mon[35037]: pgmap v1524: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:56:06.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:56:06 smithi079 ceph-mon[35312]: pgmap v1524: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:56:08.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:56:08 smithi022 ceph-mon[31985]: pgmap v1525: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:56:08.674 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:56:08 smithi134 ceph-mon[35037]: pgmap v1525: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:56:08.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:56:08 smithi079 ceph-mon[35312]: pgmap v1525: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:56:10.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:56:10 smithi022 ceph-mon[31985]: pgmap v1526: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:56:10.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:56:10 smithi022 ceph-mon[31985]: 2 slow requests, 0 included below; oldest blocked for > 2356.876634 secs 2024-03-28T18:56:10.674 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:56:10 smithi134 ceph-mon[35037]: pgmap v1526: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:56:10.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:56:10 smithi134 ceph-mon[35037]: 2 slow requests, 0 included below; oldest blocked for > 2356.876634 secs 2024-03-28T18:56:10.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:56:10 smithi079 ceph-mon[35312]: pgmap v1526: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:56:10.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:56:10 smithi079 ceph-mon[35312]: 2 slow requests, 0 included below; oldest blocked for > 2356.876634 secs 2024-03-28T18:56:12.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:56:12 smithi022 ceph-mon[31985]: pgmap v1527: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 9 op/s 2024-03-28T18:56:12.674 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:56:12 smithi134 ceph-mon[35037]: pgmap v1527: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 9 op/s 2024-03-28T18:56:12.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:56:12 smithi079 ceph-mon[35312]: pgmap v1527: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 9 op/s 2024-03-28T18:56:14.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:56:14 smithi022 ceph-mon[31985]: pgmap v1528: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T18:56:14.674 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:56:14 smithi134 ceph-mon[35037]: pgmap v1528: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T18:56:14.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:56:14 smithi079 ceph-mon[35312]: pgmap v1528: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T18:56:15.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:56:15 smithi022 ceph-mon[31985]: 2 slow requests, 0 included below; oldest blocked for > 2361.876696 secs 2024-03-28T18:56:15.576 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:56:15 smithi134 ceph-mon[35037]: 2 slow requests, 0 included below; oldest blocked for > 2361.876696 secs 2024-03-28T18:56:15.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:56:15 smithi079 ceph-mon[35312]: 2 slow requests, 0 included below; oldest blocked for > 2361.876696 secs 2024-03-28T18:56:16.531 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:56:16 smithi022 ceph-mon[31985]: pgmap v1529: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 9 op/s 2024-03-28T18:56:16.674 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:56:16 smithi134 ceph-mon[35037]: pgmap v1529: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 9 op/s 2024-03-28T18:56:16.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:56:16 smithi079 ceph-mon[35312]: pgmap v1529: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 9 op/s 2024-03-28T18:56:18.531 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:56:18 smithi022 ceph-mon[31985]: pgmap v1530: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T18:56:18.674 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:56:18 smithi134 ceph-mon[35037]: pgmap v1530: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T18:56:18.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:56:18 smithi079 ceph-mon[35312]: pgmap v1530: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T18:56:20.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:56:20 smithi022 ceph-mon[31985]: pgmap v1531: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T18:56:20.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:56:20 smithi022 ceph-mon[31985]: 2 slow requests, 0 included below; oldest blocked for > 2366.876766 secs 2024-03-28T18:56:20.674 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:56:20 smithi134 ceph-mon[35037]: pgmap v1531: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T18:56:20.674 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:56:20 smithi134 ceph-mon[35037]: 2 slow requests, 0 included below; oldest blocked for > 2366.876766 secs 2024-03-28T18:56:20.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:56:20 smithi079 ceph-mon[35312]: pgmap v1531: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T18:56:20.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:56:20 smithi079 ceph-mon[35312]: 2 slow requests, 0 included below; oldest blocked for > 2366.876766 secs 2024-03-28T18:56:22.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:56:22 smithi022 ceph-mon[31985]: pgmap v1532: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 9 op/s 2024-03-28T18:56:22.674 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:56:22 smithi134 ceph-mon[35037]: pgmap v1532: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 9 op/s 2024-03-28T18:56:22.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:56:22 smithi079 ceph-mon[35312]: pgmap v1532: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 9 op/s 2024-03-28T18:56:24.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:56:24 smithi022 ceph-mon[31985]: pgmap v1533: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:56:24.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:56:24 smithi134 ceph-mon[35037]: pgmap v1533: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:56:24.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:56:24 smithi079 ceph-mon[35312]: pgmap v1533: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:56:25.584 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:56:25 smithi079 ceph-mon[35312]: 2 slow requests, 0 included below; oldest blocked for > 2371.876886 secs 2024-03-28T18:56:25.674 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:56:25 smithi134 ceph-mon[35037]: 2 slow requests, 0 included below; oldest blocked for > 2371.876886 secs 2024-03-28T18:56:25.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:56:25 smithi022 ceph-mon[31985]: 2 slow requests, 0 included below; oldest blocked for > 2371.876886 secs 2024-03-28T18:56:26.674 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:56:26 smithi134 ceph-mon[35037]: pgmap v1534: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:56:26.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:56:26 smithi079 ceph-mon[35312]: pgmap v1534: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:56:26.781 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:56:26 smithi022 ceph-mon[31985]: pgmap v1534: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:56:28.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:56:28 smithi134 ceph-mon[35037]: pgmap v1535: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:56:28.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:56:28 smithi079 ceph-mon[35312]: pgmap v1535: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:56:28.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:56:28 smithi022 ceph-mon[31985]: pgmap v1535: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:56:30.674 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:56:30 smithi134 ceph-mon[35037]: pgmap v1536: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:56:30.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:56:30 smithi134 ceph-mon[35037]: 2 slow requests, 0 included below; oldest blocked for > 2376.876957 secs 2024-03-28T18:56:30.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:56:30 smithi079 ceph-mon[35312]: pgmap v1536: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:56:30.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:56:30 smithi079 ceph-mon[35312]: 2 slow requests, 0 included below; oldest blocked for > 2376.876957 secs 2024-03-28T18:56:30.781 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:56:30 smithi022 ceph-mon[31985]: pgmap v1536: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:56:30.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:56:30 smithi022 ceph-mon[31985]: 2 slow requests, 0 included below; oldest blocked for > 2376.876957 secs 2024-03-28T18:56:32.674 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:56:32 smithi134 ceph-mon[35037]: pgmap v1537: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:56:32.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:56:32 smithi079 ceph-mon[35312]: pgmap v1537: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:56:32.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:56:32 smithi022 ceph-mon[31985]: pgmap v1537: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:56:34.674 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:56:34 smithi134 ceph-mon[35037]: pgmap v1538: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:56:34.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:56:34 smithi079 ceph-mon[35312]: pgmap v1538: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:56:34.781 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:56:34 smithi022 ceph-mon[31985]: pgmap v1538: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:56:35.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:56:35 smithi134 ceph-mon[35037]: 2 slow requests, 0 included below; oldest blocked for > 2381.877010 secs 2024-03-28T18:56:35.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:56:35 smithi079 ceph-mon[35312]: 2 slow requests, 0 included below; oldest blocked for > 2381.877010 secs 2024-03-28T18:56:35.781 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:56:35 smithi022 ceph-mon[31985]: 2 slow requests, 0 included below; oldest blocked for > 2381.877010 secs 2024-03-28T18:56:36.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:56:36 smithi134 ceph-mon[35037]: pgmap v1539: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:56:36.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:56:36 smithi079 ceph-mon[35312]: pgmap v1539: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:56:36.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:56:36 smithi022 ceph-mon[31985]: pgmap v1539: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:56:38.674 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:56:38 smithi134 ceph-mon[35037]: pgmap v1540: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:56:38.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:56:38 smithi134 ceph-mon[35037]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T18:56:38.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:56:38 smithi079 ceph-mon[35312]: pgmap v1540: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:56:38.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:56:38 smithi079 ceph-mon[35312]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T18:56:38.781 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:56:38 smithi022 ceph-mon[31985]: pgmap v1540: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:56:38.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:56:38 smithi022 ceph-mon[31985]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T18:56:39.674 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:56:39 smithi134 ceph-mon[35037]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T18:56:39.674 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:56:39 smithi134 ceph-mon[35037]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T18:56:39.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:56:39 smithi134 ceph-mon[35037]: from='mgr.15522 ' entity='mgr.x' 2024-03-28T18:56:39.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:56:39 smithi079 ceph-mon[35312]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T18:56:39.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:56:39 smithi079 ceph-mon[35312]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T18:56:39.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:56:39 smithi079 ceph-mon[35312]: from='mgr.15522 ' entity='mgr.x' 2024-03-28T18:56:39.781 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:56:39 smithi022 ceph-mon[31985]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T18:56:39.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:56:39 smithi022 ceph-mon[31985]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T18:56:39.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:56:39 smithi022 ceph-mon[31985]: from='mgr.15522 ' entity='mgr.x' 2024-03-28T18:56:40.674 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:56:40 smithi134 ceph-mon[35037]: pgmap v1541: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 6 op/s 2024-03-28T18:56:40.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:56:40 smithi134 ceph-mon[35037]: pgmap v1542: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-03-28T18:56:40.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:56:40 smithi134 ceph-mon[35037]: 2 slow requests, 0 included below; oldest blocked for > 2386.877059 secs 2024-03-28T18:56:40.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:56:40 smithi079 ceph-mon[35312]: pgmap v1541: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 6 op/s 2024-03-28T18:56:40.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:56:40 smithi079 ceph-mon[35312]: pgmap v1542: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-03-28T18:56:40.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:56:40 smithi079 ceph-mon[35312]: 2 slow requests, 0 included below; oldest blocked for > 2386.877059 secs 2024-03-28T18:56:40.781 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:56:40 smithi022 ceph-mon[31985]: pgmap v1541: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 6 op/s 2024-03-28T18:56:40.781 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:56:40 smithi022 ceph-mon[31985]: pgmap v1542: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-03-28T18:56:40.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:56:40 smithi022 ceph-mon[31985]: 2 slow requests, 0 included below; oldest blocked for > 2386.877059 secs 2024-03-28T18:56:42.674 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:56:42 smithi134 ceph-mon[35037]: pgmap v1543: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-03-28T18:56:42.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:56:42 smithi079 ceph-mon[35312]: pgmap v1543: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-03-28T18:56:42.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:56:42 smithi022 ceph-mon[31985]: pgmap v1543: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-03-28T18:56:44.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:56:44 smithi134 ceph-mon[35037]: pgmap v1544: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-03-28T18:56:44.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:56:44 smithi079 ceph-mon[35312]: pgmap v1544: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-03-28T18:56:44.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:56:44 smithi022 ceph-mon[31985]: pgmap v1544: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-03-28T18:56:45.674 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:56:45 smithi134 ceph-mon[35037]: 2 slow requests, 0 included below; oldest blocked for > 2391.877151 secs 2024-03-28T18:56:45.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:56:45 smithi079 ceph-mon[35312]: 2 slow requests, 0 included below; oldest blocked for > 2391.877151 secs 2024-03-28T18:56:45.683 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:56:45 smithi022 ceph-mon[31985]: 2 slow requests, 0 included below; oldest blocked for > 2391.877151 secs 2024-03-28T18:56:46.674 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:56:46 smithi134 ceph-mon[35037]: pgmap v1545: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-03-28T18:56:46.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:56:46 smithi079 ceph-mon[35312]: pgmap v1545: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-03-28T18:56:46.781 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:56:46 smithi022 ceph-mon[31985]: pgmap v1545: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-03-28T18:56:48.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:56:48 smithi134 ceph-mon[35037]: pgmap v1546: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-03-28T18:56:48.683 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:56:48 smithi079 ceph-mon[35312]: pgmap v1546: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-03-28T18:56:48.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:56:48 smithi022 ceph-mon[31985]: pgmap v1546: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-03-28T18:56:50.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:56:50 smithi134 ceph-mon[35037]: pgmap v1547: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 7 op/s 2024-03-28T18:56:50.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:56:50 smithi134 ceph-mon[35037]: 2 slow requests, 0 included below; oldest blocked for > 2396.877241 secs 2024-03-28T18:56:50.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:56:50 smithi079 ceph-mon[35312]: pgmap v1547: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 7 op/s 2024-03-28T18:56:50.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:56:50 smithi079 ceph-mon[35312]: 2 slow requests, 0 included below; oldest blocked for > 2396.877241 secs 2024-03-28T18:56:50.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:56:50 smithi022 ceph-mon[31985]: pgmap v1547: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 7 op/s 2024-03-28T18:56:50.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:56:50 smithi022 ceph-mon[31985]: 2 slow requests, 0 included below; oldest blocked for > 2396.877241 secs 2024-03-28T18:56:52.674 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:56:52 smithi134 ceph-mon[35037]: pgmap v1548: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:56:52.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:56:52 smithi079 ceph-mon[35312]: pgmap v1548: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:56:52.781 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:56:52 smithi022 ceph-mon[31985]: pgmap v1548: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:56:54.674 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:56:54 smithi134 ceph-mon[35037]: pgmap v1549: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:56:54.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:56:54 smithi079 ceph-mon[35312]: pgmap v1549: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:56:54.781 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:56:54 smithi022 ceph-mon[31985]: pgmap v1549: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:56:55.674 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:56:55 smithi134 ceph-mon[35037]: 2 slow requests, 0 included below; oldest blocked for > 2401.877319 secs 2024-03-28T18:56:55.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:56:55 smithi079 ceph-mon[35312]: 2 slow requests, 0 included below; oldest blocked for > 2401.877319 secs 2024-03-28T18:56:55.781 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:56:55 smithi022 ceph-mon[31985]: 2 slow requests, 0 included below; oldest blocked for > 2401.877319 secs 2024-03-28T18:56:56.674 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:56:56 smithi134 ceph-mon[35037]: pgmap v1550: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:56:56.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:56:56 smithi079 ceph-mon[35312]: pgmap v1550: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:56:56.781 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:56:56 smithi022 ceph-mon[31985]: pgmap v1550: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:56:58.683 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:56:58 smithi079 ceph-mon[35312]: pgmap v1551: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:56:58.781 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:56:58 smithi022 ceph-mon[31985]: pgmap v1551: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:56:58.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:56:58 smithi134 ceph-mon[35037]: pgmap v1551: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:57:00.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:57:00 smithi079 ceph-mon[35312]: pgmap v1552: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:57:00.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:57:00 smithi079 ceph-mon[35312]: 2 slow requests, 0 included below; oldest blocked for > 2406.877342 secs 2024-03-28T18:57:00.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:57:00 smithi022 ceph-mon[31985]: pgmap v1552: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:57:00.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:57:00 smithi022 ceph-mon[31985]: 2 slow requests, 0 included below; oldest blocked for > 2406.877342 secs 2024-03-28T18:57:00.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:57:00 smithi134 ceph-mon[35037]: pgmap v1552: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:57:00.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:57:00 smithi134 ceph-mon[35037]: 2 slow requests, 0 included below; oldest blocked for > 2406.877342 secs 2024-03-28T18:57:02.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:57:02 smithi022 ceph-mon[31985]: pgmap v1553: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:57:02.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:57:02 smithi134 ceph-mon[35037]: pgmap v1553: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:57:02.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:57:02 smithi079 ceph-mon[35312]: pgmap v1553: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:57:04.781 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:57:04 smithi022 ceph-mon[31985]: pgmap v1554: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:57:04.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:57:04 smithi134 ceph-mon[35037]: pgmap v1554: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:57:04.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:57:04 smithi079 ceph-mon[35312]: pgmap v1554: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:57:05.781 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:57:05 smithi022 ceph-mon[31985]: 2 slow requests, 0 included below; oldest blocked for > 2411.877406 secs 2024-03-28T18:57:05.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:57:05 smithi134 ceph-mon[35037]: 2 slow requests, 0 included below; oldest blocked for > 2411.877406 secs 2024-03-28T18:57:05.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:57:05 smithi079 ceph-mon[35312]: 2 slow requests, 0 included below; oldest blocked for > 2411.877406 secs 2024-03-28T18:57:06.781 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:57:06 smithi022 ceph-mon[31985]: pgmap v1555: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:57:06.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:57:06 smithi134 ceph-mon[35037]: pgmap v1555: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:57:06.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:57:06 smithi079 ceph-mon[35312]: pgmap v1555: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:57:08.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:57:08 smithi022 ceph-mon[31985]: pgmap v1556: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:57:08.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:57:08 smithi134 ceph-mon[35037]: pgmap v1556: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:57:08.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:57:08 smithi079 ceph-mon[35312]: pgmap v1556: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:57:10.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:57:10 smithi022 ceph-mon[31985]: pgmap v1557: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:57:10.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:57:10 smithi022 ceph-mon[31985]: 2 slow requests, 0 included below; oldest blocked for > 2416.877471 secs 2024-03-28T18:57:10.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:57:10 smithi134 ceph-mon[35037]: pgmap v1557: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:57:10.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:57:10 smithi134 ceph-mon[35037]: 2 slow requests, 0 included below; oldest blocked for > 2416.877471 secs 2024-03-28T18:57:10.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:57:10 smithi079 ceph-mon[35312]: pgmap v1557: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:57:10.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:57:10 smithi079 ceph-mon[35312]: 2 slow requests, 0 included below; oldest blocked for > 2416.877471 secs 2024-03-28T18:57:12.781 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:57:12 smithi022 ceph-mon[31985]: pgmap v1558: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:57:12.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:57:12 smithi134 ceph-mon[35037]: pgmap v1558: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:57:12.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:57:12 smithi079 ceph-mon[35312]: pgmap v1558: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:57:14.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:57:14 smithi022 ceph-mon[31985]: pgmap v1559: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:57:14.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:57:14 smithi134 ceph-mon[35037]: pgmap v1559: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:57:14.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:57:14 smithi079 ceph-mon[35312]: pgmap v1559: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:57:15.781 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:57:15 smithi022 ceph-mon[31985]: 2 slow requests, 0 included below; oldest blocked for > 2421.877534 secs 2024-03-28T18:57:15.826 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:57:15 smithi134 ceph-mon[35037]: 2 slow requests, 0 included below; oldest blocked for > 2421.877534 secs 2024-03-28T18:57:15.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:57:15 smithi079 ceph-mon[35312]: 2 slow requests, 0 included below; oldest blocked for > 2421.877534 secs 2024-03-28T18:57:16.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:57:16 smithi022 ceph-mon[31985]: pgmap v1560: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:57:16.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:57:16 smithi134 ceph-mon[35037]: pgmap v1560: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:57:16.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:57:16 smithi079 ceph-mon[35312]: pgmap v1560: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:57:18.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:57:18 smithi022 ceph-mon[31985]: pgmap v1561: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:57:18.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:57:18 smithi134 ceph-mon[35037]: pgmap v1561: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:57:18.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:57:18 smithi079 ceph-mon[35312]: pgmap v1561: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:57:19.781 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:57:19 smithi022 ceph-mon[31985]: pgmap v1562: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:57:19.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:57:19 smithi134 ceph-mon[35037]: pgmap v1562: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:57:19.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:57:19 smithi079 ceph-mon[35312]: pgmap v1562: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:57:20.781 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:57:20 smithi022 ceph-mon[31985]: 2 slow requests, 0 included below; oldest blocked for > 2426.877604 secs 2024-03-28T18:57:20.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:57:20 smithi134 ceph-mon[35037]: 2 slow requests, 0 included below; oldest blocked for > 2426.877604 secs 2024-03-28T18:57:20.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:57:20 smithi079 ceph-mon[35312]: 2 slow requests, 0 included below; oldest blocked for > 2426.877604 secs 2024-03-28T18:57:21.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:57:21 smithi134 ceph-mon[35037]: pgmap v1563: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:57:21.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:57:21 smithi079 ceph-mon[35312]: pgmap v1563: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:57:22.031 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:57:21 smithi022 ceph-mon[31985]: pgmap v1563: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:57:24.174 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:57:23 smithi134 ceph-mon[35037]: pgmap v1564: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:57:24.182 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:57:23 smithi079 ceph-mon[35312]: pgmap v1564: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:57:24.281 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:57:23 smithi022 ceph-mon[31985]: pgmap v1564: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:57:25.174 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:57:24 smithi134 ceph-mon[35037]: 2 slow requests, 0 included below; oldest blocked for > 2431.877667 secs 2024-03-28T18:57:25.182 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:57:24 smithi079 ceph-mon[35312]: 2 slow requests, 0 included below; oldest blocked for > 2431.877667 secs 2024-03-28T18:57:25.281 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:57:24 smithi022 ceph-mon[31985]: 2 slow requests, 0 included below; oldest blocked for > 2431.877667 secs 2024-03-28T18:57:26.174 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:57:25 smithi134 ceph-mon[35037]: pgmap v1565: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:57:26.182 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:57:25 smithi079 ceph-mon[35312]: pgmap v1565: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:57:26.281 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:57:25 smithi022 ceph-mon[31985]: pgmap v1565: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:57:28.182 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:57:27 smithi079 ceph-mon[35312]: pgmap v1566: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:57:28.281 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:57:27 smithi022 ceph-mon[31985]: pgmap v1566: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:57:28.425 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:57:27 smithi134 ceph-mon[35037]: pgmap v1566: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:57:30.282 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:57:29 smithi022 ceph-mon[31985]: pgmap v1567: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:57:30.282 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:57:29 smithi022 ceph-mon[31985]: 2 slow requests, 0 included below; oldest blocked for > 2436.877753 secs 2024-03-28T18:57:30.424 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:57:29 smithi134 ceph-mon[35037]: pgmap v1567: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:57:30.424 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:57:29 smithi134 ceph-mon[35037]: 2 slow requests, 0 included below; oldest blocked for > 2436.877753 secs 2024-03-28T18:57:30.432 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:57:29 smithi079 ceph-mon[35312]: pgmap v1567: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:57:30.432 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:57:29 smithi079 ceph-mon[35312]: 2 slow requests, 0 included below; oldest blocked for > 2436.877753 secs 2024-03-28T18:57:32.281 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:57:31 smithi022 ceph-mon[31985]: pgmap v1568: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:57:32.424 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:57:31 smithi134 ceph-mon[35037]: pgmap v1568: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:57:32.432 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:57:31 smithi079 ceph-mon[35312]: pgmap v1568: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:57:34.281 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:57:33 smithi022 ceph-mon[31985]: pgmap v1569: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:57:34.425 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:57:33 smithi134 ceph-mon[35037]: pgmap v1569: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:57:34.432 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:57:33 smithi079 ceph-mon[35312]: pgmap v1569: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:57:35.282 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:57:34 smithi022 ceph-mon[31985]: 2 slow requests, 0 included below; oldest blocked for > 2441.877781 secs 2024-03-28T18:57:35.424 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:57:34 smithi134 ceph-mon[35037]: 2 slow requests, 0 included below; oldest blocked for > 2441.877781 secs 2024-03-28T18:57:35.432 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:57:34 smithi079 ceph-mon[35312]: 2 slow requests, 0 included below; oldest blocked for > 2441.877781 secs 2024-03-28T18:57:36.281 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:57:35 smithi022 ceph-mon[31985]: pgmap v1570: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:57:36.424 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:57:35 smithi134 ceph-mon[35037]: pgmap v1570: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:57:36.432 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:57:35 smithi079 ceph-mon[35312]: pgmap v1570: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:57:38.281 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:57:37 smithi022 ceph-mon[31985]: pgmap v1571: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:57:38.424 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:57:37 smithi134 ceph-mon[35037]: pgmap v1571: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:57:38.432 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:57:37 smithi079 ceph-mon[35312]: pgmap v1571: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:57:40.281 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:57:39 smithi022 ceph-mon[31985]: pgmap v1572: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:57:40.282 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:57:39 smithi022 ceph-mon[31985]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T18:57:40.282 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:57:39 smithi022 ceph-mon[31985]: 2 slow requests, 0 included below; oldest blocked for > 2446.877849 secs 2024-03-28T18:57:40.424 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:57:39 smithi134 ceph-mon[35037]: pgmap v1572: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:57:40.424 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:57:39 smithi134 ceph-mon[35037]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T18:57:40.424 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:57:39 smithi134 ceph-mon[35037]: 2 slow requests, 0 included below; oldest blocked for > 2446.877849 secs 2024-03-28T18:57:40.432 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:57:39 smithi079 ceph-mon[35312]: pgmap v1572: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:57:40.432 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:57:39 smithi079 ceph-mon[35312]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T18:57:40.432 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:57:39 smithi079 ceph-mon[35312]: 2 slow requests, 0 included below; oldest blocked for > 2446.877849 secs 2024-03-28T18:57:41.282 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:57:41 smithi022 ceph-mon[31985]: from='mgr.15522 ' entity='mgr.x' 2024-03-28T18:57:41.282 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:57:41 smithi022 ceph-mon[31985]: from='mgr.15522 ' entity='mgr.x' 2024-03-28T18:57:41.282 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:57:41 smithi022 ceph-mon[31985]: from='mgr.15522 ' entity='mgr.x' 2024-03-28T18:57:41.282 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:57:41 smithi022 ceph-mon[31985]: from='mgr.15522 ' entity='mgr.x' 2024-03-28T18:57:41.424 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:57:41 smithi134 ceph-mon[35037]: from='mgr.15522 ' entity='mgr.x' 2024-03-28T18:57:41.425 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:57:41 smithi134 ceph-mon[35037]: from='mgr.15522 ' entity='mgr.x' 2024-03-28T18:57:41.425 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:57:41 smithi134 ceph-mon[35037]: from='mgr.15522 ' entity='mgr.x' 2024-03-28T18:57:41.425 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:57:41 smithi134 ceph-mon[35037]: from='mgr.15522 ' entity='mgr.x' 2024-03-28T18:57:41.432 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:57:41 smithi079 ceph-mon[35312]: from='mgr.15522 ' entity='mgr.x' 2024-03-28T18:57:41.432 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:57:41 smithi079 ceph-mon[35312]: from='mgr.15522 ' entity='mgr.x' 2024-03-28T18:57:41.432 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:57:41 smithi079 ceph-mon[35312]: from='mgr.15522 ' entity='mgr.x' 2024-03-28T18:57:41.433 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:57:41 smithi079 ceph-mon[35312]: from='mgr.15522 ' entity='mgr.x' 2024-03-28T18:57:42.424 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:57:42 smithi134 ceph-mon[35037]: pgmap v1573: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:57:42.425 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:57:42 smithi134 ceph-mon[35037]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T18:57:42.425 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:57:42 smithi134 ceph-mon[35037]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T18:57:42.425 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:57:42 smithi134 ceph-mon[35037]: pgmap v1574: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 7 op/s 2024-03-28T18:57:42.425 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:57:42 smithi134 ceph-mon[35037]: from='mgr.15522 ' entity='mgr.x' 2024-03-28T18:57:42.432 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:57:42 smithi079 ceph-mon[35312]: pgmap v1573: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:57:42.432 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:57:42 smithi079 ceph-mon[35312]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T18:57:42.432 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:57:42 smithi079 ceph-mon[35312]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T18:57:42.432 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:57:42 smithi079 ceph-mon[35312]: pgmap v1574: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 7 op/s 2024-03-28T18:57:42.432 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:57:42 smithi079 ceph-mon[35312]: from='mgr.15522 ' entity='mgr.x' 2024-03-28T18:57:42.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:57:42 smithi022 ceph-mon[31985]: pgmap v1573: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:57:42.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:57:42 smithi022 ceph-mon[31985]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T18:57:42.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:57:42 smithi022 ceph-mon[31985]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T18:57:42.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:57:42 smithi022 ceph-mon[31985]: pgmap v1574: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 7 op/s 2024-03-28T18:57:42.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:57:42 smithi022 ceph-mon[31985]: from='mgr.15522 ' entity='mgr.x' 2024-03-28T18:57:43.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:57:43 smithi134 ceph-mon[35037]: pgmap v1575: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 7 op/s 2024-03-28T18:57:43.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:57:43 smithi079 ceph-mon[35312]: pgmap v1575: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 7 op/s 2024-03-28T18:57:44.031 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:57:43 smithi022 ceph-mon[31985]: pgmap v1575: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 7 op/s 2024-03-28T18:57:45.031 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:57:44 smithi022 ceph-mon[31985]: 2 slow requests, 0 included below; oldest blocked for > 2451.877924 secs 2024-03-28T18:57:45.174 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:57:44 smithi134 ceph-mon[35037]: 2 slow requests, 0 included below; oldest blocked for > 2451.877924 secs 2024-03-28T18:57:45.182 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:57:44 smithi079 ceph-mon[35312]: 2 slow requests, 0 included below; oldest blocked for > 2451.877924 secs 2024-03-28T18:57:46.032 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:57:45 smithi022 ceph-mon[31985]: pgmap v1576: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 7 op/s 2024-03-28T18:57:46.077 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:57:45 smithi134 ceph-mon[35037]: pgmap v1576: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 7 op/s 2024-03-28T18:57:46.182 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:57:45 smithi079 ceph-mon[35312]: pgmap v1576: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 7 op/s 2024-03-28T18:57:47.925 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:57:47 smithi134 ceph-mon[35037]: pgmap v1577: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 7 op/s 2024-03-28T18:57:47.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:57:47 smithi079 ceph-mon[35312]: pgmap v1577: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 7 op/s 2024-03-28T18:57:48.032 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:57:47 smithi022 ceph-mon[31985]: pgmap v1577: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 7 op/s 2024-03-28T18:57:49.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:57:49 smithi134 ceph-mon[35037]: pgmap v1578: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 11 op/s 2024-03-28T18:57:49.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:57:49 smithi079 ceph-mon[35312]: pgmap v1578: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 11 op/s 2024-03-28T18:57:50.031 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:57:49 smithi022 ceph-mon[31985]: pgmap v1578: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 11 op/s 2024-03-28T18:57:50.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:57:50 smithi134 ceph-mon[35037]: 2 slow requests, 0 included below; oldest blocked for > 2456.878237 secs 2024-03-28T18:57:50.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:57:50 smithi079 ceph-mon[35312]: 2 slow requests, 0 included below; oldest blocked for > 2456.878237 secs 2024-03-28T18:57:51.032 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:57:50 smithi022 ceph-mon[31985]: 2 slow requests, 0 included below; oldest blocked for > 2456.878237 secs 2024-03-28T18:57:51.925 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:57:51 smithi134 ceph-mon[35037]: pgmap v1579: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 7 op/s 2024-03-28T18:57:51.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:57:51 smithi079 ceph-mon[35312]: pgmap v1579: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 7 op/s 2024-03-28T18:57:52.032 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:57:51 smithi022 ceph-mon[31985]: pgmap v1579: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 7 op/s 2024-03-28T18:57:53.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:57:53 smithi134 ceph-mon[35037]: pgmap v1580: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:57:53.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:57:53 smithi079 ceph-mon[35312]: pgmap v1580: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:57:54.031 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:57:53 smithi022 ceph-mon[31985]: pgmap v1580: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:57:55.031 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:57:54 smithi022 ceph-mon[31985]: 2 slow requests, 0 included below; oldest blocked for > 2461.878298 secs 2024-03-28T18:57:55.174 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:57:54 smithi134 ceph-mon[35037]: 2 slow requests, 0 included below; oldest blocked for > 2461.878298 secs 2024-03-28T18:57:55.182 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:57:54 smithi079 ceph-mon[35312]: 2 slow requests, 0 included below; oldest blocked for > 2461.878298 secs 2024-03-28T18:57:56.034 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:57:55 smithi022 ceph-mon[31985]: pgmap v1581: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:57:56.084 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:57:55 smithi079 ceph-mon[35312]: pgmap v1581: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:57:56.174 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:57:55 smithi134 ceph-mon[35037]: pgmap v1581: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:57:57.925 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:57:57 smithi134 ceph-mon[35037]: pgmap v1582: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:57:57.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:57:57 smithi079 ceph-mon[35312]: pgmap v1582: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:57:58.031 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:57:57 smithi022 ceph-mon[31985]: pgmap v1582: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:57:59.925 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:57:59 smithi134 ceph-mon[35037]: pgmap v1583: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:57:59.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:57:59 smithi079 ceph-mon[35312]: pgmap v1583: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:58:00.031 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:57:59 smithi022 ceph-mon[31985]: pgmap v1583: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:58:00.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:58:00 smithi134 ceph-mon[35037]: 2 slow requests, 0 included below; oldest blocked for > 2466.878393 secs 2024-03-28T18:58:00.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:58:00 smithi079 ceph-mon[35312]: 2 slow requests, 0 included below; oldest blocked for > 2466.878393 secs 2024-03-28T18:58:01.031 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:58:00 smithi022 ceph-mon[31985]: 2 slow requests, 0 included below; oldest blocked for > 2466.878393 secs 2024-03-28T18:58:01.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:58:01 smithi134 ceph-mon[35037]: pgmap v1584: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:58:01.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:58:01 smithi079 ceph-mon[35312]: pgmap v1584: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:58:02.031 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:58:01 smithi022 ceph-mon[31985]: pgmap v1584: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:58:03.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:58:03 smithi134 ceph-mon[35037]: pgmap v1585: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:58:03.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:58:03 smithi079 ceph-mon[35312]: pgmap v1585: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:58:04.031 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:58:03 smithi022 ceph-mon[31985]: pgmap v1585: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:58:05.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:58:05 smithi022 ceph-mon[31985]: 2 slow requests, 0 included below; oldest blocked for > 2471.878430 secs 2024-03-28T18:58:05.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:58:05 smithi134 ceph-mon[35037]: 2 slow requests, 0 included below; oldest blocked for > 2471.878430 secs 2024-03-28T18:58:05.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:58:05 smithi079 ceph-mon[35312]: 2 slow requests, 0 included below; oldest blocked for > 2471.878430 secs 2024-03-28T18:58:06.781 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:58:06 smithi022 ceph-mon[31985]: pgmap v1586: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:58:06.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:58:06 smithi134 ceph-mon[35037]: pgmap v1586: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:58:06.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:58:06 smithi079 ceph-mon[35312]: pgmap v1586: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:58:08.781 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:58:08 smithi022 ceph-mon[31985]: pgmap v1587: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:58:08.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:58:08 smithi134 ceph-mon[35037]: pgmap v1587: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:58:08.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:58:08 smithi079 ceph-mon[35312]: pgmap v1587: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:58:09.781 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:58:09 smithi022 ceph-mon[31985]: pgmap v1588: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:58:09.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:58:09 smithi134 ceph-mon[35037]: pgmap v1588: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:58:09.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:58:09 smithi079 ceph-mon[35312]: pgmap v1588: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:58:10.781 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:58:10 smithi022 ceph-mon[31985]: 2 slow requests, 0 included below; oldest blocked for > 2476.878508 secs 2024-03-28T18:58:10.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:58:10 smithi134 ceph-mon[35037]: 2 slow requests, 0 included below; oldest blocked for > 2476.878508 secs 2024-03-28T18:58:10.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:58:10 smithi079 ceph-mon[35312]: 2 slow requests, 0 included below; oldest blocked for > 2476.878508 secs 2024-03-28T18:58:11.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:58:11 smithi134 ceph-mon[35037]: pgmap v1589: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:58:11.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:58:11 smithi079 ceph-mon[35312]: pgmap v1589: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:58:12.031 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:58:11 smithi022 ceph-mon[31985]: pgmap v1589: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:58:13.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:58:13 smithi134 ceph-mon[35037]: pgmap v1590: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:58:13.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:58:13 smithi079 ceph-mon[35312]: pgmap v1590: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:58:14.031 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:58:13 smithi022 ceph-mon[31985]: pgmap v1590: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:58:15.032 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:58:14 smithi022 ceph-mon[31985]: 2 slow requests, 0 included below; oldest blocked for > 2481.878571 secs 2024-03-28T18:58:15.175 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:58:14 smithi134 ceph-mon[35037]: 2 slow requests, 0 included below; oldest blocked for > 2481.878571 secs 2024-03-28T18:58:15.182 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:58:14 smithi079 ceph-mon[35312]: 2 slow requests, 0 included below; oldest blocked for > 2481.878571 secs 2024-03-28T18:58:16.031 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:58:15 smithi022 ceph-mon[31985]: pgmap v1591: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:58:16.076 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:58:15 smithi134 ceph-mon[35037]: pgmap v1591: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:58:16.182 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:58:15 smithi079 ceph-mon[35312]: pgmap v1591: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:58:17.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:58:17 smithi134 ceph-mon[35037]: pgmap v1592: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:58:17.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:58:17 smithi079 ceph-mon[35312]: pgmap v1592: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:58:18.031 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:58:17 smithi022 ceph-mon[31985]: pgmap v1592: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:58:19.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:58:19 smithi134 ceph-mon[35037]: pgmap v1593: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:58:19.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:58:19 smithi079 ceph-mon[35312]: pgmap v1593: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:58:20.031 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:58:19 smithi022 ceph-mon[31985]: pgmap v1593: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:58:20.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:58:20 smithi134 ceph-mon[35037]: 2 slow requests, 0 included below; oldest blocked for > 2486.878695 secs 2024-03-28T18:58:20.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:58:20 smithi079 ceph-mon[35312]: 2 slow requests, 0 included below; oldest blocked for > 2486.878695 secs 2024-03-28T18:58:21.031 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:58:20 smithi022 ceph-mon[31985]: 2 slow requests, 0 included below; oldest blocked for > 2486.878695 secs 2024-03-28T18:58:21.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:58:21 smithi134 ceph-mon[35037]: pgmap v1594: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:58:21.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:58:21 smithi079 ceph-mon[35312]: pgmap v1594: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:58:22.031 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:58:21 smithi022 ceph-mon[31985]: pgmap v1594: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:58:23.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:58:23 smithi134 ceph-mon[35037]: pgmap v1595: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:58:23.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:58:23 smithi079 ceph-mon[35312]: pgmap v1595: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:58:24.031 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:58:23 smithi022 ceph-mon[31985]: pgmap v1595: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:58:25.031 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:58:24 smithi022 ceph-mon[31985]: 2 slow requests, 0 included below; oldest blocked for > 2491.878794 secs 2024-03-28T18:58:25.174 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:58:24 smithi134 ceph-mon[35037]: 2 slow requests, 0 included below; oldest blocked for > 2491.878794 secs 2024-03-28T18:58:25.182 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:58:24 smithi079 ceph-mon[35312]: 2 slow requests, 0 included below; oldest blocked for > 2491.878794 secs 2024-03-28T18:58:26.031 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:58:25 smithi022 ceph-mon[31985]: pgmap v1596: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:58:26.083 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:58:25 smithi079 ceph-mon[35312]: pgmap v1596: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:58:26.174 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:58:25 smithi134 ceph-mon[35037]: pgmap v1596: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:58:27.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:58:27 smithi134 ceph-mon[35037]: pgmap v1597: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:58:27.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:58:27 smithi079 ceph-mon[35312]: pgmap v1597: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:58:28.031 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:58:27 smithi022 ceph-mon[31985]: pgmap v1597: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:58:29.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:58:29 smithi134 ceph-mon[35037]: pgmap v1598: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:58:29.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:58:29 smithi079 ceph-mon[35312]: pgmap v1598: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:58:30.032 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:58:29 smithi022 ceph-mon[31985]: pgmap v1598: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:58:30.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:58:30 smithi134 ceph-mon[35037]: 2 slow requests, 0 included below; oldest blocked for > 2496.878859 secs 2024-03-28T18:58:30.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:58:30 smithi079 ceph-mon[35312]: 2 slow requests, 0 included below; oldest blocked for > 2496.878859 secs 2024-03-28T18:58:31.031 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:58:30 smithi022 ceph-mon[31985]: 2 slow requests, 0 included below; oldest blocked for > 2496.878859 secs 2024-03-28T18:58:31.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:58:31 smithi134 ceph-mon[35037]: pgmap v1599: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:58:31.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:58:31 smithi079 ceph-mon[35312]: pgmap v1599: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:58:32.031 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:58:31 smithi022 ceph-mon[31985]: pgmap v1599: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:58:33.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:58:33 smithi134 ceph-mon[35037]: pgmap v1600: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:58:33.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:58:33 smithi079 ceph-mon[35312]: pgmap v1600: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:58:34.031 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:58:33 smithi022 ceph-mon[31985]: pgmap v1600: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:58:35.031 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:58:34 smithi022 ceph-mon[31985]: 2 slow requests, 0 included below; oldest blocked for > 2501.878923 secs 2024-03-28T18:58:35.174 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:58:34 smithi134 ceph-mon[35037]: 2 slow requests, 0 included below; oldest blocked for > 2501.878923 secs 2024-03-28T18:58:35.182 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:58:34 smithi079 ceph-mon[35312]: 2 slow requests, 0 included below; oldest blocked for > 2501.878923 secs 2024-03-28T18:58:36.031 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:58:35 smithi022 ceph-mon[31985]: pgmap v1601: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:58:36.174 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:58:35 smithi134 ceph-mon[35037]: pgmap v1601: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:58:36.182 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:58:35 smithi079 ceph-mon[35312]: pgmap v1601: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:58:37.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:58:37 smithi134 ceph-mon[35037]: pgmap v1602: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:58:37.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:58:37 smithi079 ceph-mon[35312]: pgmap v1602: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:58:38.031 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:58:37 smithi022 ceph-mon[31985]: pgmap v1602: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:58:40.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:58:40 smithi134 ceph-mon[35037]: 2 slow requests, 0 included below; oldest blocked for > 2506.878987 secs 2024-03-28T18:58:40.925 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:58:40 smithi134 ceph-mon[35037]: pgmap v1603: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:58:40.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:58:40 smithi079 ceph-mon[35312]: 2 slow requests, 0 included below; oldest blocked for > 2506.878987 secs 2024-03-28T18:58:40.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:58:40 smithi079 ceph-mon[35312]: pgmap v1603: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:58:41.031 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:58:40 smithi022 ceph-mon[31985]: 2 slow requests, 0 included below; oldest blocked for > 2506.878987 secs 2024-03-28T18:58:41.031 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:58:40 smithi022 ceph-mon[31985]: pgmap v1603: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:58:42.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:58:42 smithi134 ceph-mon[35037]: pgmap v1604: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:58:42.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:58:42 smithi134 ceph-mon[35037]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T18:58:42.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:58:42 smithi079 ceph-mon[35312]: pgmap v1604: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:58:42.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:58:42 smithi079 ceph-mon[35312]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T18:58:43.031 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:58:42 smithi022 ceph-mon[31985]: pgmap v1604: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:58:43.031 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:58:42 smithi022 ceph-mon[31985]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T18:58:43.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:58:43 smithi134 ceph-mon[35037]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T18:58:43.925 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:58:43 smithi134 ceph-mon[35037]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T18:58:43.925 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:58:43 smithi134 ceph-mon[35037]: pgmap v1605: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 6 op/s 2024-03-28T18:58:43.925 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:58:43 smithi134 ceph-mon[35037]: from='mgr.15522 ' entity='mgr.x' 2024-03-28T18:58:43.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:58:43 smithi079 ceph-mon[35312]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T18:58:43.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:58:43 smithi079 ceph-mon[35312]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T18:58:43.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:58:43 smithi079 ceph-mon[35312]: pgmap v1605: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 6 op/s 2024-03-28T18:58:43.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:58:43 smithi079 ceph-mon[35312]: from='mgr.15522 ' entity='mgr.x' 2024-03-28T18:58:44.031 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:58:43 smithi022 ceph-mon[31985]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T18:58:44.032 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:58:43 smithi022 ceph-mon[31985]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T18:58:44.032 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:58:43 smithi022 ceph-mon[31985]: pgmap v1605: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 6 op/s 2024-03-28T18:58:44.032 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:58:43 smithi022 ceph-mon[31985]: from='mgr.15522 ' entity='mgr.x' 2024-03-28T18:58:45.031 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:58:44 smithi022 ceph-mon[31985]: 2 slow requests, 0 included below; oldest blocked for > 2511.879105 secs 2024-03-28T18:58:45.174 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:58:44 smithi134 ceph-mon[35037]: 2 slow requests, 0 included below; oldest blocked for > 2511.879105 secs 2024-03-28T18:58:45.182 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:58:44 smithi079 ceph-mon[35312]: 2 slow requests, 0 included below; oldest blocked for > 2511.879105 secs 2024-03-28T18:58:46.031 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:58:45 smithi022 ceph-mon[31985]: pgmap v1606: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 10 op/s 2024-03-28T18:58:46.076 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:58:45 smithi134 ceph-mon[35037]: pgmap v1606: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 10 op/s 2024-03-28T18:58:46.182 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:58:45 smithi079 ceph-mon[35312]: pgmap v1606: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 10 op/s 2024-03-28T18:58:48.531 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:58:48 smithi022 ceph-mon[31985]: pgmap v1607: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 6 op/s 2024-03-28T18:58:48.674 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:58:48 smithi134 ceph-mon[35037]: pgmap v1607: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 6 op/s 2024-03-28T18:58:48.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:58:48 smithi079 ceph-mon[35312]: pgmap v1607: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 6 op/s 2024-03-28T18:58:50.531 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:58:50 smithi022 ceph-mon[31985]: pgmap v1608: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 6 op/s 2024-03-28T18:58:50.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:58:50 smithi022 ceph-mon[31985]: 2 slow requests, 0 included below; oldest blocked for > 2516.879196 secs 2024-03-28T18:58:50.674 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:58:50 smithi134 ceph-mon[35037]: pgmap v1608: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 6 op/s 2024-03-28T18:58:50.674 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:58:50 smithi134 ceph-mon[35037]: 2 slow requests, 0 included below; oldest blocked for > 2516.879196 secs 2024-03-28T18:58:50.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:58:50 smithi079 ceph-mon[35312]: pgmap v1608: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 6 op/s 2024-03-28T18:58:50.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:58:50 smithi079 ceph-mon[35312]: 2 slow requests, 0 included below; oldest blocked for > 2516.879196 secs 2024-03-28T18:58:52.531 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:58:52 smithi022 ceph-mon[31985]: pgmap v1609: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 6 op/s 2024-03-28T18:58:52.674 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:58:52 smithi134 ceph-mon[35037]: pgmap v1609: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 6 op/s 2024-03-28T18:58:52.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:58:52 smithi079 ceph-mon[35312]: pgmap v1609: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 6 op/s 2024-03-28T18:58:54.531 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:58:54 smithi022 ceph-mon[31985]: pgmap v1610: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 6 op/s 2024-03-28T18:58:54.674 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:58:54 smithi134 ceph-mon[35037]: pgmap v1610: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 6 op/s 2024-03-28T18:58:54.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:58:54 smithi079 ceph-mon[35312]: pgmap v1610: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 6 op/s 2024-03-28T18:58:55.531 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:58:55 smithi022 ceph-mon[31985]: 2 slow requests, 0 included below; oldest blocked for > 2521.879263 secs 2024-03-28T18:58:55.674 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:58:55 smithi134 ceph-mon[35037]: 2 slow requests, 0 included below; oldest blocked for > 2521.879263 secs 2024-03-28T18:58:55.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:58:55 smithi079 ceph-mon[35312]: 2 slow requests, 0 included below; oldest blocked for > 2521.879263 secs 2024-03-28T18:58:56.531 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:58:56 smithi022 ceph-mon[31985]: pgmap v1611: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:58:56.584 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:58:56 smithi079 ceph-mon[35312]: pgmap v1611: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:58:56.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:58:56 smithi134 ceph-mon[35037]: pgmap v1611: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:58:58.531 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:58:58 smithi022 ceph-mon[31985]: pgmap v1612: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:58:58.674 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:58:58 smithi134 ceph-mon[35037]: pgmap v1612: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:58:58.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:58:58 smithi079 ceph-mon[35312]: pgmap v1612: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:59:00.531 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:59:00 smithi022 ceph-mon[31985]: pgmap v1613: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:59:00.531 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:59:00 smithi022 ceph-mon[31985]: 2 slow requests, 0 included below; oldest blocked for > 2526.879383 secs 2024-03-28T18:59:00.674 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:59:00 smithi134 ceph-mon[35037]: pgmap v1613: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:59:00.674 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:59:00 smithi134 ceph-mon[35037]: 2 slow requests, 0 included below; oldest blocked for > 2526.879383 secs 2024-03-28T18:59:00.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:59:00 smithi079 ceph-mon[35312]: pgmap v1613: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:59:00.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:59:00 smithi079 ceph-mon[35312]: 2 slow requests, 0 included below; oldest blocked for > 2526.879383 secs 2024-03-28T18:59:02.531 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:59:02 smithi022 ceph-mon[31985]: pgmap v1614: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:59:02.674 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:59:02 smithi134 ceph-mon[35037]: pgmap v1614: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:59:02.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:59:02 smithi079 ceph-mon[35312]: pgmap v1614: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:59:04.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:59:04 smithi134 ceph-mon[35037]: pgmap v1615: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:59:04.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:59:04 smithi079 ceph-mon[35312]: pgmap v1615: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:59:04.781 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:59:04 smithi022 ceph-mon[31985]: pgmap v1615: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:59:05.674 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:59:05 smithi134 ceph-mon[35037]: 2 slow requests, 0 included below; oldest blocked for > 2531.879464 secs 2024-03-28T18:59:05.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:59:05 smithi079 ceph-mon[35312]: 2 slow requests, 0 included below; oldest blocked for > 2531.879464 secs 2024-03-28T18:59:05.781 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:59:05 smithi022 ceph-mon[31985]: 2 slow requests, 0 included below; oldest blocked for > 2531.879464 secs 2024-03-28T18:59:06.674 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:59:06 smithi134 ceph-mon[35037]: pgmap v1616: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:59:06.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:59:06 smithi079 ceph-mon[35312]: pgmap v1616: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:59:06.781 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:59:06 smithi022 ceph-mon[31985]: pgmap v1616: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:59:08.674 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:59:08 smithi134 ceph-mon[35037]: pgmap v1617: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:59:08.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:59:08 smithi079 ceph-mon[35312]: pgmap v1617: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:59:08.781 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:59:08 smithi022 ceph-mon[31985]: pgmap v1617: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:59:10.674 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:59:10 smithi134 ceph-mon[35037]: pgmap v1618: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:59:10.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:59:10 smithi134 ceph-mon[35037]: 2 slow requests, 0 included below; oldest blocked for > 2536.879540 secs 2024-03-28T18:59:10.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:59:10 smithi079 ceph-mon[35312]: pgmap v1618: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:59:10.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:59:10 smithi079 ceph-mon[35312]: 2 slow requests, 0 included below; oldest blocked for > 2536.879540 secs 2024-03-28T18:59:10.781 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:59:10 smithi022 ceph-mon[31985]: pgmap v1618: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:59:10.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:59:10 smithi022 ceph-mon[31985]: 2 slow requests, 0 included below; oldest blocked for > 2536.879540 secs 2024-03-28T18:59:12.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:59:12 smithi134 ceph-mon[35037]: pgmap v1619: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 9 op/s 2024-03-28T18:59:12.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:59:12 smithi079 ceph-mon[35312]: pgmap v1619: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 9 op/s 2024-03-28T18:59:12.781 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:59:12 smithi022 ceph-mon[31985]: pgmap v1619: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 9 op/s 2024-03-28T18:59:14.674 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:59:14 smithi134 ceph-mon[35037]: pgmap v1620: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T18:59:14.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:59:14 smithi079 ceph-mon[35312]: pgmap v1620: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T18:59:14.781 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:59:14 smithi022 ceph-mon[31985]: pgmap v1620: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T18:59:15.674 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:59:15 smithi134 ceph-mon[35037]: 2 slow requests, 0 included below; oldest blocked for > 2541.879604 secs 2024-03-28T18:59:15.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:59:15 smithi079 ceph-mon[35312]: 2 slow requests, 0 included below; oldest blocked for > 2541.879604 secs 2024-03-28T18:59:15.683 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:59:15 smithi022 ceph-mon[31985]: 2 slow requests, 0 included below; oldest blocked for > 2541.879604 secs 2024-03-28T18:59:16.674 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:59:16 smithi134 ceph-mon[35037]: pgmap v1621: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 9 op/s 2024-03-28T18:59:16.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:59:16 smithi079 ceph-mon[35312]: pgmap v1621: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 9 op/s 2024-03-28T18:59:16.781 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:59:16 smithi022 ceph-mon[31985]: pgmap v1621: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 9 op/s 2024-03-28T18:59:18.674 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:59:18 smithi134 ceph-mon[35037]: pgmap v1622: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T18:59:18.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:59:18 smithi079 ceph-mon[35312]: pgmap v1622: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T18:59:18.781 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:59:18 smithi022 ceph-mon[31985]: pgmap v1622: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T18:59:20.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:59:20 smithi134 ceph-mon[35037]: pgmap v1623: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T18:59:20.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:59:20 smithi134 ceph-mon[35037]: 2 slow requests, 0 included below; oldest blocked for > 2546.879667 secs 2024-03-28T18:59:20.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:59:20 smithi079 ceph-mon[35312]: pgmap v1623: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T18:59:20.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:59:20 smithi079 ceph-mon[35312]: 2 slow requests, 0 included below; oldest blocked for > 2546.879667 secs 2024-03-28T18:59:20.781 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:59:20 smithi022 ceph-mon[31985]: pgmap v1623: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T18:59:20.781 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:59:20 smithi022 ceph-mon[31985]: 2 slow requests, 0 included below; oldest blocked for > 2546.879667 secs 2024-03-28T18:59:22.674 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:59:22 smithi134 ceph-mon[35037]: pgmap v1624: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 9 op/s 2024-03-28T18:59:22.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:59:22 smithi079 ceph-mon[35312]: pgmap v1624: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 9 op/s 2024-03-28T18:59:22.781 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:59:22 smithi022 ceph-mon[31985]: pgmap v1624: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 9 op/s 2024-03-28T18:59:24.674 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:59:24 smithi134 ceph-mon[35037]: pgmap v1625: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:59:24.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:59:24 smithi079 ceph-mon[35312]: pgmap v1625: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:59:24.781 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:59:24 smithi022 ceph-mon[31985]: pgmap v1625: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:59:25.674 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:59:25 smithi134 ceph-mon[35037]: 2 slow requests, 0 included below; oldest blocked for > 2551.879738 secs 2024-03-28T18:59:25.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:59:25 smithi079 ceph-mon[35312]: 2 slow requests, 0 included below; oldest blocked for > 2551.879738 secs 2024-03-28T18:59:25.781 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:59:25 smithi022 ceph-mon[31985]: 2 slow requests, 0 included below; oldest blocked for > 2551.879738 secs 2024-03-28T18:59:26.674 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:59:26 smithi134 ceph-mon[35037]: pgmap v1626: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:59:26.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:59:26 smithi079 ceph-mon[35312]: pgmap v1626: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:59:26.781 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:59:26 smithi022 ceph-mon[31985]: pgmap v1626: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:59:28.674 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:59:28 smithi134 ceph-mon[35037]: pgmap v1627: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:59:28.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:59:28 smithi079 ceph-mon[35312]: pgmap v1627: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:59:28.781 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:59:28 smithi022 ceph-mon[31985]: pgmap v1627: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:59:30.674 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:59:30 smithi134 ceph-mon[35037]: pgmap v1628: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:59:30.674 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:59:30 smithi134 ceph-mon[35037]: 2 slow requests, 0 included below; oldest blocked for > 2556.879778 secs 2024-03-28T18:59:30.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:59:30 smithi079 ceph-mon[35312]: pgmap v1628: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:59:30.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:59:30 smithi079 ceph-mon[35312]: 2 slow requests, 0 included below; oldest blocked for > 2556.879778 secs 2024-03-28T18:59:30.781 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:59:30 smithi022 ceph-mon[31985]: pgmap v1628: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:59:30.781 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:59:30 smithi022 ceph-mon[31985]: 2 slow requests, 0 included below; oldest blocked for > 2556.879778 secs 2024-03-28T18:59:32.674 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:59:32 smithi134 ceph-mon[35037]: pgmap v1629: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:59:32.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:59:32 smithi079 ceph-mon[35312]: pgmap v1629: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:59:32.781 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:59:32 smithi022 ceph-mon[31985]: pgmap v1629: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:59:34.674 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:59:34 smithi134 ceph-mon[35037]: pgmap v1630: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:59:34.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:59:34 smithi079 ceph-mon[35312]: pgmap v1630: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:59:34.781 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:59:34 smithi022 ceph-mon[31985]: pgmap v1630: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:59:35.674 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:59:35 smithi134 ceph-mon[35037]: 2 slow requests, 0 included below; oldest blocked for > 2561.879842 secs 2024-03-28T18:59:35.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:59:35 smithi079 ceph-mon[35312]: 2 slow requests, 0 included below; oldest blocked for > 2561.879842 secs 2024-03-28T18:59:35.781 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:59:35 smithi022 ceph-mon[31985]: 2 slow requests, 0 included below; oldest blocked for > 2561.879842 secs 2024-03-28T18:59:36.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:59:36 smithi134 ceph-mon[35037]: pgmap v1631: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:59:36.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:59:36 smithi079 ceph-mon[35312]: pgmap v1631: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:59:36.781 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:59:36 smithi022 ceph-mon[31985]: pgmap v1631: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:59:38.674 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:59:38 smithi134 ceph-mon[35037]: pgmap v1632: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:59:38.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:59:38 smithi079 ceph-mon[35312]: pgmap v1632: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:59:38.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:59:38 smithi022 ceph-mon[31985]: pgmap v1632: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:59:40.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:59:40 smithi079 ceph-mon[35312]: pgmap v1633: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:59:40.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:59:40 smithi079 ceph-mon[35312]: 2 slow requests, 0 included below; oldest blocked for > 2566.879975 secs 2024-03-28T18:59:40.781 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:59:40 smithi022 ceph-mon[31985]: pgmap v1633: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:59:40.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:59:40 smithi022 ceph-mon[31985]: 2 slow requests, 0 included below; oldest blocked for > 2566.879975 secs 2024-03-28T18:59:40.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:59:40 smithi134 ceph-mon[35037]: pgmap v1633: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:59:40.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:59:40 smithi134 ceph-mon[35037]: 2 slow requests, 0 included below; oldest blocked for > 2566.879975 secs 2024-03-28T18:59:42.781 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:59:42 smithi022 ceph-mon[31985]: pgmap v1634: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:59:42.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:59:42 smithi134 ceph-mon[35037]: pgmap v1634: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:59:42.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:59:42 smithi079 ceph-mon[35312]: pgmap v1634: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:59:44.781 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:59:44 smithi022 ceph-mon[31985]: pgmap v1635: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:59:44.781 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:59:44 smithi022 ceph-mon[31985]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T18:59:44.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:59:44 smithi134 ceph-mon[35037]: pgmap v1635: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:59:44.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:59:44 smithi134 ceph-mon[35037]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T18:59:44.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:59:44 smithi079 ceph-mon[35312]: pgmap v1635: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:59:44.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:59:44 smithi079 ceph-mon[35312]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T18:59:45.781 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:59:45 smithi022 ceph-mon[31985]: 2 slow requests, 0 included below; oldest blocked for > 2571.880045 secs 2024-03-28T18:59:45.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:59:45 smithi022 ceph-mon[31985]: from='mgr.15522 ' entity='mgr.x' 2024-03-28T18:59:45.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:59:45 smithi022 ceph-mon[31985]: from='mgr.15522 ' entity='mgr.x' 2024-03-28T18:59:45.899 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:59:45 smithi134 ceph-mon[35037]: 2 slow requests, 0 included below; oldest blocked for > 2571.880045 secs 2024-03-28T18:59:45.899 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:59:45 smithi134 ceph-mon[35037]: from='mgr.15522 ' entity='mgr.x' 2024-03-28T18:59:45.899 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:59:45 smithi134 ceph-mon[35037]: from='mgr.15522 ' entity='mgr.x' 2024-03-28T18:59:45.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:59:45 smithi079 ceph-mon[35312]: 2 slow requests, 0 included below; oldest blocked for > 2571.880045 secs 2024-03-28T18:59:45.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:59:45 smithi079 ceph-mon[35312]: from='mgr.15522 ' entity='mgr.x' 2024-03-28T18:59:45.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:59:45 smithi079 ceph-mon[35312]: from='mgr.15522 ' entity='mgr.x' 2024-03-28T18:59:46.697 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:59:46 smithi022 ceph-mon[31985]: pgmap v1636: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:59:46.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:59:46 smithi134 ceph-mon[35037]: pgmap v1636: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:59:46.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:59:46 smithi079 ceph-mon[35312]: pgmap v1636: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T18:59:47.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:59:47 smithi134 ceph-mon[35037]: from='mgr.15522 ' entity='mgr.x' 2024-03-28T18:59:47.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:59:47 smithi134 ceph-mon[35037]: from='mgr.15522 ' entity='mgr.x' 2024-03-28T18:59:47.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:59:47 smithi134 ceph-mon[35037]: pgmap v1637: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:59:47.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:59:47 smithi134 ceph-mon[35037]: from='mgr.15522 ' entity='mgr.x' 2024-03-28T18:59:47.925 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:59:47 smithi134 ceph-mon[35037]: from='mgr.15522 ' entity='mgr.x' 2024-03-28T18:59:47.925 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:59:47 smithi134 ceph-mon[35037]: from='mgr.15522 ' entity='mgr.x' 2024-03-28T18:59:47.925 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:59:47 smithi134 ceph-mon[35037]: from='mgr.15522 ' entity='mgr.x' 2024-03-28T18:59:47.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:59:47 smithi079 ceph-mon[35312]: from='mgr.15522 ' entity='mgr.x' 2024-03-28T18:59:47.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:59:47 smithi079 ceph-mon[35312]: from='mgr.15522 ' entity='mgr.x' 2024-03-28T18:59:47.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:59:47 smithi079 ceph-mon[35312]: pgmap v1637: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:59:47.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:59:47 smithi079 ceph-mon[35312]: from='mgr.15522 ' entity='mgr.x' 2024-03-28T18:59:47.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:59:47 smithi079 ceph-mon[35312]: from='mgr.15522 ' entity='mgr.x' 2024-03-28T18:59:47.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:59:47 smithi079 ceph-mon[35312]: from='mgr.15522 ' entity='mgr.x' 2024-03-28T18:59:47.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:59:47 smithi079 ceph-mon[35312]: from='mgr.15522 ' entity='mgr.x' 2024-03-28T18:59:48.018 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:59:47 smithi022 ceph-mon[31985]: from='mgr.15522 ' entity='mgr.x' 2024-03-28T18:59:48.018 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:59:47 smithi022 ceph-mon[31985]: from='mgr.15522 ' entity='mgr.x' 2024-03-28T18:59:48.018 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:59:47 smithi022 ceph-mon[31985]: pgmap v1637: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T18:59:48.019 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:59:47 smithi022 ceph-mon[31985]: from='mgr.15522 ' entity='mgr.x' 2024-03-28T18:59:48.019 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:59:47 smithi022 ceph-mon[31985]: from='mgr.15522 ' entity='mgr.x' 2024-03-28T18:59:48.019 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:59:47 smithi022 ceph-mon[31985]: from='mgr.15522 ' entity='mgr.x' 2024-03-28T18:59:48.019 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:59:47 smithi022 ceph-mon[31985]: from='mgr.15522 ' entity='mgr.x' 2024-03-28T18:59:49.032 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:59:48 smithi022 ceph-mon[31985]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-28T18:59:49.032 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:59:48 smithi022 ceph-mon[31985]: from='mgr.15522 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-28T18:59:49.032 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:59:48 smithi022 ceph-mon[31985]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-28T18:59:49.032 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:59:48 smithi022 ceph-mon[31985]: from='mgr.15522 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-28T18:59:49.032 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:59:48 smithi022 ceph-mon[31985]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-03-28T18:59:49.032 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:59:48 smithi022 ceph-mon[31985]: from='mgr.15522 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-03-28T18:59:49.032 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:59:48 smithi022 ceph-mon[31985]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-03-28T18:59:49.033 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:59:48 smithi022 ceph-mon[31985]: from='mgr.15522 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-03-28T18:59:49.033 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:59:48 smithi022 ceph-mon[31985]: Adjusting osd_memory_target on smithi079 to 176.7M 2024-03-28T18:59:49.033 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:59:48 smithi022 ceph-mon[31985]: Unable to set osd_memory_target on smithi079 to 185386700: error parsing value: Value '185386700' is below minimum 939524096 2024-03-28T18:59:49.033 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:59:48 smithi022 ceph-mon[31985]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-28T18:59:49.033 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:59:48 smithi022 ceph-mon[31985]: from='mgr.15522 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-28T18:59:49.033 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:59:48 smithi022 ceph-mon[31985]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-28T18:59:49.033 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:59:48 smithi022 ceph-mon[31985]: from='mgr.15522 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-28T18:59:49.033 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:59:48 smithi022 ceph-mon[31985]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-03-28T18:59:49.033 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:59:48 smithi022 ceph-mon[31985]: from='mgr.15522 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-03-28T18:59:49.033 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:59:48 smithi022 ceph-mon[31985]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-03-28T18:59:49.033 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:59:48 smithi022 ceph-mon[31985]: from='mgr.15522 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-03-28T18:59:49.174 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:59:48 smithi134 ceph-mon[35037]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-28T18:59:49.175 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:59:48 smithi134 ceph-mon[35037]: from='mgr.15522 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-28T18:59:49.175 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:59:48 smithi134 ceph-mon[35037]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-28T18:59:49.175 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:59:48 smithi134 ceph-mon[35037]: from='mgr.15522 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-28T18:59:49.175 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:59:48 smithi134 ceph-mon[35037]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-03-28T18:59:49.175 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:59:48 smithi134 ceph-mon[35037]: from='mgr.15522 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-03-28T18:59:49.175 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:59:48 smithi134 ceph-mon[35037]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-03-28T18:59:49.175 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:59:48 smithi134 ceph-mon[35037]: from='mgr.15522 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-03-28T18:59:49.175 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:59:48 smithi134 ceph-mon[35037]: Adjusting osd_memory_target on smithi079 to 176.7M 2024-03-28T18:59:49.175 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:59:48 smithi134 ceph-mon[35037]: Unable to set osd_memory_target on smithi079 to 185386700: error parsing value: Value '185386700' is below minimum 939524096 2024-03-28T18:59:49.175 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:59:48 smithi134 ceph-mon[35037]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-28T18:59:49.175 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:59:48 smithi134 ceph-mon[35037]: from='mgr.15522 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-28T18:59:49.175 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:59:48 smithi134 ceph-mon[35037]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-28T18:59:49.175 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:59:48 smithi134 ceph-mon[35037]: from='mgr.15522 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-28T18:59:49.175 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:59:48 smithi134 ceph-mon[35037]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-03-28T18:59:49.175 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:59:48 smithi134 ceph-mon[35037]: from='mgr.15522 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-03-28T18:59:49.175 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:59:48 smithi134 ceph-mon[35037]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-03-28T18:59:49.176 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:59:48 smithi134 ceph-mon[35037]: from='mgr.15522 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-03-28T18:59:49.182 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:59:48 smithi079 ceph-mon[35312]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-28T18:59:49.182 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:59:48 smithi079 ceph-mon[35312]: from='mgr.15522 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-28T18:59:49.182 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:59:48 smithi079 ceph-mon[35312]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-28T18:59:49.182 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:59:48 smithi079 ceph-mon[35312]: from='mgr.15522 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-28T18:59:49.182 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:59:48 smithi079 ceph-mon[35312]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-03-28T18:59:49.183 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:59:48 smithi079 ceph-mon[35312]: from='mgr.15522 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-03-28T18:59:49.183 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:59:48 smithi079 ceph-mon[35312]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-03-28T18:59:49.183 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:59:48 smithi079 ceph-mon[35312]: from='mgr.15522 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-03-28T18:59:49.183 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:59:48 smithi079 ceph-mon[35312]: Adjusting osd_memory_target on smithi079 to 176.7M 2024-03-28T18:59:49.183 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:59:48 smithi079 ceph-mon[35312]: Unable to set osd_memory_target on smithi079 to 185386700: error parsing value: Value '185386700' is below minimum 939524096 2024-03-28T18:59:49.183 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:59:48 smithi079 ceph-mon[35312]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-28T18:59:49.183 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:59:48 smithi079 ceph-mon[35312]: from='mgr.15522 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-28T18:59:49.183 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:59:48 smithi079 ceph-mon[35312]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-28T18:59:49.183 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:59:48 smithi079 ceph-mon[35312]: from='mgr.15522 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-28T18:59:49.183 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:59:48 smithi079 ceph-mon[35312]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-03-28T18:59:49.183 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:59:48 smithi079 ceph-mon[35312]: from='mgr.15522 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-03-28T18:59:49.183 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:59:48 smithi079 ceph-mon[35312]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-03-28T18:59:49.183 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:59:48 smithi079 ceph-mon[35312]: from='mgr.15522 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-03-28T18:59:50.032 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:59:49 smithi022 ceph-mon[31985]: Adjusting osd_memory_target on smithi022 to 177.8M 2024-03-28T18:59:50.032 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:59:49 smithi022 ceph-mon[31985]: Unable to set osd_memory_target on smithi022 to 186497740: error parsing value: Value '186497740' is below minimum 939524096 2024-03-28T18:59:50.032 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:59:49 smithi022 ceph-mon[31985]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-28T18:59:50.032 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:59:49 smithi022 ceph-mon[31985]: from='mgr.15522 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-28T18:59:50.032 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:59:49 smithi022 ceph-mon[31985]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-28T18:59:50.032 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:59:49 smithi022 ceph-mon[31985]: from='mgr.15522 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-28T18:59:50.032 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:59:49 smithi022 ceph-mon[31985]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-03-28T18:59:50.032 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:59:49 smithi022 ceph-mon[31985]: from='mgr.15522 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-03-28T18:59:50.033 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:59:49 smithi022 ceph-mon[31985]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-03-28T18:59:50.033 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:59:49 smithi022 ceph-mon[31985]: from='mgr.15522 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-03-28T18:59:50.033 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:59:49 smithi022 ceph-mon[31985]: Adjusting osd_memory_target on smithi134 to 176.8M 2024-03-28T18:59:50.033 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:59:49 smithi022 ceph-mon[31985]: Unable to set osd_memory_target on smithi134 to 185393868: error parsing value: Value '185393868' is below minimum 939524096 2024-03-28T18:59:50.033 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:59:49 smithi022 ceph-mon[31985]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T18:59:50.033 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:59:49 smithi022 ceph-mon[31985]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T18:59:50.033 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:59:49 smithi022 ceph-mon[31985]: pgmap v1638: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 6 op/s 2024-03-28T18:59:50.033 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:59:49 smithi022 ceph-mon[31985]: from='mgr.15522 ' entity='mgr.x' 2024-03-28T18:59:50.033 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:59:49 smithi022 ceph-mon[31985]: 2 slow requests, 0 included below; oldest blocked for > 2576.880072 secs 2024-03-28T18:59:50.175 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:59:49 smithi134 ceph-mon[35037]: Adjusting osd_memory_target on smithi022 to 177.8M 2024-03-28T18:59:50.175 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:59:49 smithi134 ceph-mon[35037]: Unable to set osd_memory_target on smithi022 to 186497740: error parsing value: Value '186497740' is below minimum 939524096 2024-03-28T18:59:50.175 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:59:49 smithi134 ceph-mon[35037]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-28T18:59:50.175 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:59:49 smithi134 ceph-mon[35037]: from='mgr.15522 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-28T18:59:50.175 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:59:49 smithi134 ceph-mon[35037]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-28T18:59:50.175 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:59:49 smithi134 ceph-mon[35037]: from='mgr.15522 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-28T18:59:50.175 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:59:49 smithi134 ceph-mon[35037]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-03-28T18:59:50.176 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:59:49 smithi134 ceph-mon[35037]: from='mgr.15522 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-03-28T18:59:50.176 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:59:49 smithi134 ceph-mon[35037]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-03-28T18:59:50.176 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:59:49 smithi134 ceph-mon[35037]: from='mgr.15522 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-03-28T18:59:50.176 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:59:49 smithi134 ceph-mon[35037]: Adjusting osd_memory_target on smithi134 to 176.8M 2024-03-28T18:59:50.176 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:59:49 smithi134 ceph-mon[35037]: Unable to set osd_memory_target on smithi134 to 185393868: error parsing value: Value '185393868' is below minimum 939524096 2024-03-28T18:59:50.176 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:59:49 smithi134 ceph-mon[35037]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T18:59:50.176 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:59:49 smithi134 ceph-mon[35037]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T18:59:50.176 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:59:49 smithi134 ceph-mon[35037]: pgmap v1638: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 6 op/s 2024-03-28T18:59:50.176 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:59:49 smithi134 ceph-mon[35037]: from='mgr.15522 ' entity='mgr.x' 2024-03-28T18:59:50.176 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:59:49 smithi134 ceph-mon[35037]: 2 slow requests, 0 included below; oldest blocked for > 2576.880072 secs 2024-03-28T18:59:50.182 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:59:49 smithi079 ceph-mon[35312]: Adjusting osd_memory_target on smithi022 to 177.8M 2024-03-28T18:59:50.182 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:59:49 smithi079 ceph-mon[35312]: Unable to set osd_memory_target on smithi022 to 186497740: error parsing value: Value '186497740' is below minimum 939524096 2024-03-28T18:59:50.182 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:59:49 smithi079 ceph-mon[35312]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-28T18:59:50.183 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:59:49 smithi079 ceph-mon[35312]: from='mgr.15522 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-28T18:59:50.183 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:59:49 smithi079 ceph-mon[35312]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-28T18:59:50.183 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:59:49 smithi079 ceph-mon[35312]: from='mgr.15522 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-28T18:59:50.183 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:59:49 smithi079 ceph-mon[35312]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-03-28T18:59:50.183 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:59:49 smithi079 ceph-mon[35312]: from='mgr.15522 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-03-28T18:59:50.183 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:59:49 smithi079 ceph-mon[35312]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-03-28T18:59:50.183 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:59:49 smithi079 ceph-mon[35312]: from='mgr.15522 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-03-28T18:59:50.183 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:59:49 smithi079 ceph-mon[35312]: Adjusting osd_memory_target on smithi134 to 176.8M 2024-03-28T18:59:50.183 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:59:49 smithi079 ceph-mon[35312]: Unable to set osd_memory_target on smithi134 to 185393868: error parsing value: Value '185393868' is below minimum 939524096 2024-03-28T18:59:50.183 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:59:49 smithi079 ceph-mon[35312]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T18:59:50.183 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:59:49 smithi079 ceph-mon[35312]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T18:59:50.183 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:59:49 smithi079 ceph-mon[35312]: pgmap v1638: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 6 op/s 2024-03-28T18:59:50.183 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:59:49 smithi079 ceph-mon[35312]: from='mgr.15522 ' entity='mgr.x' 2024-03-28T18:59:50.183 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:59:49 smithi079 ceph-mon[35312]: 2 slow requests, 0 included below; oldest blocked for > 2576.880072 secs 2024-03-28T18:59:52.174 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:59:51 smithi134 ceph-mon[35037]: pgmap v1639: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 10 op/s 2024-03-28T18:59:52.182 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:59:51 smithi079 ceph-mon[35312]: pgmap v1639: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 10 op/s 2024-03-28T18:59:52.281 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:59:51 smithi022 ceph-mon[31985]: pgmap v1639: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 10 op/s 2024-03-28T18:59:54.174 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:59:53 smithi134 ceph-mon[35037]: pgmap v1640: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 6 op/s 2024-03-28T18:59:54.182 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:59:53 smithi079 ceph-mon[35312]: pgmap v1640: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 6 op/s 2024-03-28T18:59:54.281 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:59:53 smithi022 ceph-mon[31985]: pgmap v1640: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 6 op/s 2024-03-28T18:59:55.174 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:59:54 smithi134 ceph-mon[35037]: 2 slow requests, 0 included below; oldest blocked for > 2581.880149 secs 2024-03-28T18:59:55.182 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:59:54 smithi079 ceph-mon[35312]: 2 slow requests, 0 included below; oldest blocked for > 2581.880149 secs 2024-03-28T18:59:55.281 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:59:54 smithi022 ceph-mon[31985]: 2 slow requests, 0 included below; oldest blocked for > 2581.880149 secs 2024-03-28T18:59:56.174 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:59:55 smithi134 ceph-mon[35037]: pgmap v1641: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 10 op/s 2024-03-28T18:59:56.182 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:59:55 smithi079 ceph-mon[35312]: pgmap v1641: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 10 op/s 2024-03-28T18:59:56.281 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:59:55 smithi022 ceph-mon[31985]: pgmap v1641: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 10 op/s 2024-03-28T18:59:58.182 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:59:57 smithi079 ceph-mon[35312]: pgmap v1642: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 6 op/s 2024-03-28T18:59:58.281 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:59:57 smithi022 ceph-mon[31985]: pgmap v1642: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 6 op/s 2024-03-28T18:59:58.424 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:59:57 smithi134 ceph-mon[35037]: pgmap v1642: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 6 op/s 2024-03-28T19:00:00.281 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:59:59 smithi022 ceph-mon[31985]: pgmap v1643: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 6 op/s 2024-03-28T19:00:00.282 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 18:59:59 smithi022 ceph-mon[31985]: 2 slow requests, 0 included below; oldest blocked for > 2586.880222 secs 2024-03-28T19:00:00.424 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:59:59 smithi134 ceph-mon[35037]: pgmap v1643: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 6 op/s 2024-03-28T19:00:00.424 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 18:59:59 smithi134 ceph-mon[35037]: 2 slow requests, 0 included below; oldest blocked for > 2586.880222 secs 2024-03-28T19:00:00.432 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:59:59 smithi079 ceph-mon[35312]: pgmap v1643: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 6 op/s 2024-03-28T19:00:00.432 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 18:59:59 smithi079 ceph-mon[35312]: 2 slow requests, 0 included below; oldest blocked for > 2586.880222 secs 2024-03-28T19:00:01.282 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:00:00 smithi022 ceph-mon[31985]: overall HEALTH_WARN 4 failed cephadm daemon(s); 1 MDSs report slow requests 2024-03-28T19:00:01.424 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:00:00 smithi134 ceph-mon[35037]: overall HEALTH_WARN 4 failed cephadm daemon(s); 1 MDSs report slow requests 2024-03-28T19:00:01.432 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:00:00 smithi079 ceph-mon[35312]: overall HEALTH_WARN 4 failed cephadm daemon(s); 1 MDSs report slow requests 2024-03-28T19:00:02.281 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:00:01 smithi022 ceph-mon[31985]: pgmap v1644: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T19:00:02.424 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:00:01 smithi134 ceph-mon[35037]: pgmap v1644: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T19:00:02.432 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:00:01 smithi079 ceph-mon[35312]: pgmap v1644: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T19:00:04.281 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:00:03 smithi022 ceph-mon[31985]: pgmap v1645: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:00:04.424 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:00:03 smithi134 ceph-mon[35037]: pgmap v1645: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:00:04.432 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:00:03 smithi079 ceph-mon[35312]: pgmap v1645: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:00:05.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:00:05 smithi134 ceph-mon[35037]: 2 slow requests, 0 included below; oldest blocked for > 2591.880285 secs 2024-03-28T19:00:05.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:00:05 smithi134 ceph-mon[35037]: pgmap v1646: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T19:00:05.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:00:05 smithi079 ceph-mon[35312]: 2 slow requests, 0 included below; oldest blocked for > 2591.880285 secs 2024-03-28T19:00:05.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:00:05 smithi079 ceph-mon[35312]: pgmap v1646: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T19:00:06.031 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:00:05 smithi022 ceph-mon[31985]: 2 slow requests, 0 included below; oldest blocked for > 2591.880285 secs 2024-03-28T19:00:06.031 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:00:05 smithi022 ceph-mon[31985]: pgmap v1646: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T19:00:08.174 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:00:07 smithi134 ceph-mon[35037]: pgmap v1647: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:00:08.182 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:00:07 smithi079 ceph-mon[35312]: pgmap v1647: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:00:08.281 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:00:07 smithi022 ceph-mon[31985]: pgmap v1647: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:00:10.174 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:00:09 smithi134 ceph-mon[35037]: pgmap v1648: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:00:10.174 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:00:09 smithi134 ceph-mon[35037]: 2 slow requests, 0 included below; oldest blocked for > 2596.880366 secs 2024-03-28T19:00:10.182 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:00:09 smithi079 ceph-mon[35312]: pgmap v1648: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:00:10.182 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:00:09 smithi079 ceph-mon[35312]: 2 slow requests, 0 included below; oldest blocked for > 2596.880366 secs 2024-03-28T19:00:10.281 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:00:09 smithi022 ceph-mon[31985]: pgmap v1648: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:00:10.281 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:00:09 smithi022 ceph-mon[31985]: 2 slow requests, 0 included below; oldest blocked for > 2596.880366 secs 2024-03-28T19:00:12.174 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:00:11 smithi134 ceph-mon[35037]: pgmap v1649: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T19:00:12.182 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:00:11 smithi079 ceph-mon[35312]: pgmap v1649: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T19:00:12.281 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:00:11 smithi022 ceph-mon[31985]: pgmap v1649: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T19:00:14.182 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:00:13 smithi079 ceph-mon[35312]: pgmap v1650: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:00:14.281 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:00:13 smithi022 ceph-mon[31985]: pgmap v1650: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:00:14.424 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:00:13 smithi134 ceph-mon[35037]: pgmap v1650: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:00:15.281 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:00:14 smithi022 ceph-mon[31985]: 2 slow requests, 0 included below; oldest blocked for > 2601.880431 secs 2024-03-28T19:00:15.424 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:00:14 smithi134 ceph-mon[35037]: 2 slow requests, 0 included below; oldest blocked for > 2601.880431 secs 2024-03-28T19:00:15.432 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:00:14 smithi079 ceph-mon[35312]: 2 slow requests, 0 included below; oldest blocked for > 2601.880431 secs 2024-03-28T19:00:16.281 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:00:15 smithi022 ceph-mon[31985]: pgmap v1651: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T19:00:16.326 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:00:15 smithi134 ceph-mon[35037]: pgmap v1651: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T19:00:16.432 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:00:15 smithi079 ceph-mon[35312]: pgmap v1651: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T19:00:18.276 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:00:17 smithi022 ceph-mon[31985]: pgmap v1652: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:00:18.424 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:00:17 smithi134 ceph-mon[35037]: pgmap v1652: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:00:18.432 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:00:17 smithi079 ceph-mon[35312]: pgmap v1652: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:00:20.282 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:00:19 smithi022 ceph-mon[31985]: pgmap v1653: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:00:20.282 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:00:19 smithi022 ceph-mon[31985]: 2 slow requests, 0 included below; oldest blocked for > 2606.880515 secs 2024-03-28T19:00:20.424 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:00:19 smithi134 ceph-mon[35037]: pgmap v1653: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:00:20.424 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:00:19 smithi134 ceph-mon[35037]: 2 slow requests, 0 included below; oldest blocked for > 2606.880515 secs 2024-03-28T19:00:20.432 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:00:19 smithi079 ceph-mon[35312]: pgmap v1653: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:00:20.432 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:00:19 smithi079 ceph-mon[35312]: 2 slow requests, 0 included below; oldest blocked for > 2606.880515 secs 2024-03-28T19:00:22.281 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:00:21 smithi022 ceph-mon[31985]: pgmap v1654: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T19:00:22.424 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:00:21 smithi134 ceph-mon[35037]: pgmap v1654: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T19:00:22.432 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:00:21 smithi079 ceph-mon[35312]: pgmap v1654: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T19:00:24.281 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:00:23 smithi022 ceph-mon[31985]: pgmap v1655: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:00:24.424 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:00:23 smithi134 ceph-mon[35037]: pgmap v1655: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:00:24.432 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:00:23 smithi079 ceph-mon[35312]: pgmap v1655: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:00:25.281 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:00:24 smithi022 ceph-mon[31985]: 2 slow requests, 0 included below; oldest blocked for > 2611.880579 secs 2024-03-28T19:00:25.424 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:00:24 smithi134 ceph-mon[35037]: 2 slow requests, 0 included below; oldest blocked for > 2611.880579 secs 2024-03-28T19:00:25.432 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:00:24 smithi079 ceph-mon[35312]: 2 slow requests, 0 included below; oldest blocked for > 2611.880579 secs 2024-03-28T19:00:26.281 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:00:25 smithi022 ceph-mon[31985]: pgmap v1656: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T19:00:26.334 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:00:25 smithi079 ceph-mon[35312]: pgmap v1656: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T19:00:26.424 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:00:25 smithi134 ceph-mon[35037]: pgmap v1656: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T19:00:28.281 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:00:27 smithi022 ceph-mon[31985]: pgmap v1657: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:00:28.424 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:00:27 smithi134 ceph-mon[35037]: pgmap v1657: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:00:28.432 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:00:27 smithi079 ceph-mon[35312]: pgmap v1657: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:00:30.281 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:00:29 smithi022 ceph-mon[31985]: pgmap v1658: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:00:30.282 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:00:29 smithi022 ceph-mon[31985]: 2 slow requests, 0 included below; oldest blocked for > 2616.880647 secs 2024-03-28T19:00:30.424 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:00:29 smithi134 ceph-mon[35037]: pgmap v1658: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:00:30.424 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:00:29 smithi134 ceph-mon[35037]: 2 slow requests, 0 included below; oldest blocked for > 2616.880647 secs 2024-03-28T19:00:30.432 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:00:29 smithi079 ceph-mon[35312]: pgmap v1658: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:00:30.432 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:00:29 smithi079 ceph-mon[35312]: 2 slow requests, 0 included below; oldest blocked for > 2616.880647 secs 2024-03-28T19:00:32.281 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:00:31 smithi022 ceph-mon[31985]: pgmap v1659: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T19:00:32.424 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:00:31 smithi134 ceph-mon[35037]: pgmap v1659: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T19:00:32.432 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:00:31 smithi079 ceph-mon[35312]: pgmap v1659: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T19:00:34.281 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:00:34 smithi022 ceph-mon[31985]: pgmap v1660: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:00:34.424 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:00:34 smithi134 ceph-mon[35037]: pgmap v1660: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:00:34.432 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:00:34 smithi079 ceph-mon[35312]: pgmap v1660: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:00:35.281 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:00:35 smithi022 ceph-mon[31985]: 2 slow requests, 0 included below; oldest blocked for > 2621.880717 secs 2024-03-28T19:00:35.424 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:00:35 smithi134 ceph-mon[35037]: 2 slow requests, 0 included below; oldest blocked for > 2621.880717 secs 2024-03-28T19:00:35.432 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:00:35 smithi079 ceph-mon[35312]: 2 slow requests, 0 included below; oldest blocked for > 2621.880717 secs 2024-03-28T19:00:36.282 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:00:36 smithi022 ceph-mon[31985]: pgmap v1661: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 9 op/s 2024-03-28T19:00:36.424 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:00:36 smithi134 ceph-mon[35037]: pgmap v1661: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 9 op/s 2024-03-28T19:00:36.432 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:00:36 smithi079 ceph-mon[35312]: pgmap v1661: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 9 op/s 2024-03-28T19:00:38.281 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:00:38 smithi022 ceph-mon[31985]: pgmap v1662: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T19:00:38.424 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:00:38 smithi134 ceph-mon[35037]: pgmap v1662: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T19:00:38.432 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:00:38 smithi079 ceph-mon[35312]: pgmap v1662: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T19:00:40.281 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:00:40 smithi022 ceph-mon[31985]: pgmap v1663: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T19:00:40.282 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:00:40 smithi022 ceph-mon[31985]: 2 slow requests, 0 included below; oldest blocked for > 2626.880840 secs 2024-03-28T19:00:40.424 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:00:40 smithi134 ceph-mon[35037]: pgmap v1663: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T19:00:40.424 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:00:40 smithi134 ceph-mon[35037]: 2 slow requests, 0 included below; oldest blocked for > 2626.880840 secs 2024-03-28T19:00:40.432 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:00:40 smithi079 ceph-mon[35312]: pgmap v1663: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T19:00:40.432 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:00:40 smithi079 ceph-mon[35312]: 2 slow requests, 0 included below; oldest blocked for > 2626.880840 secs 2024-03-28T19:00:42.424 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:00:42 smithi134 ceph-mon[35037]: pgmap v1664: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 9 op/s 2024-03-28T19:00:42.432 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:00:42 smithi079 ceph-mon[35312]: pgmap v1664: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 9 op/s 2024-03-28T19:00:42.531 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:00:42 smithi022 ceph-mon[31985]: pgmap v1664: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 9 op/s 2024-03-28T19:00:44.314 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:00:44 smithi134 ceph-mon[35037]: pgmap v1665: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T19:00:44.432 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:00:44 smithi079 ceph-mon[35312]: pgmap v1665: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T19:00:44.531 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:00:44 smithi022 ceph-mon[31985]: pgmap v1665: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T19:00:45.424 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:00:45 smithi134 ceph-mon[35037]: 2 slow requests, 0 included below; oldest blocked for > 2631.880877 secs 2024-03-28T19:00:45.432 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:00:45 smithi079 ceph-mon[35312]: 2 slow requests, 0 included below; oldest blocked for > 2631.880877 secs 2024-03-28T19:00:45.433 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:00:45 smithi022 ceph-mon[31985]: 2 slow requests, 0 included below; oldest blocked for > 2631.880877 secs 2024-03-28T19:00:46.326 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:00:46 smithi134 ceph-mon[35037]: pgmap v1666: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 9 op/s 2024-03-28T19:00:46.432 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:00:46 smithi079 ceph-mon[35312]: pgmap v1666: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 9 op/s 2024-03-28T19:00:46.531 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:00:46 smithi022 ceph-mon[31985]: pgmap v1666: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 9 op/s 2024-03-28T19:00:48.424 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:00:48 smithi134 ceph-mon[35037]: pgmap v1667: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T19:00:48.432 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:00:48 smithi079 ceph-mon[35312]: pgmap v1667: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T19:00:48.531 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:00:48 smithi022 ceph-mon[31985]: pgmap v1667: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T19:00:50.424 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:00:50 smithi134 ceph-mon[35037]: pgmap v1668: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T19:00:50.425 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:00:50 smithi134 ceph-mon[35037]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T19:00:50.425 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:00:50 smithi134 ceph-mon[35037]: 2 slow requests, 0 included below; oldest blocked for > 2636.880972 secs 2024-03-28T19:00:50.432 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:00:50 smithi079 ceph-mon[35312]: pgmap v1668: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T19:00:50.432 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:00:50 smithi079 ceph-mon[35312]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T19:00:50.432 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:00:50 smithi079 ceph-mon[35312]: 2 slow requests, 0 included below; oldest blocked for > 2636.880972 secs 2024-03-28T19:00:50.531 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:00:50 smithi022 ceph-mon[31985]: pgmap v1668: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T19:00:50.531 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:00:50 smithi022 ceph-mon[31985]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T19:00:50.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:00:50 smithi022 ceph-mon[31985]: 2 slow requests, 0 included below; oldest blocked for > 2636.880972 secs 2024-03-28T19:00:51.426 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:00:51 smithi134 ceph-mon[35037]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T19:00:51.426 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:00:51 smithi134 ceph-mon[35037]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T19:00:51.426 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:00:51 smithi134 ceph-mon[35037]: from='mgr.15522 ' entity='mgr.x' 2024-03-28T19:00:51.432 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:00:51 smithi079 ceph-mon[35312]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T19:00:51.432 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:00:51 smithi079 ceph-mon[35312]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T19:00:51.432 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:00:51 smithi079 ceph-mon[35312]: from='mgr.15522 ' entity='mgr.x' 2024-03-28T19:00:51.531 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:00:51 smithi022 ceph-mon[31985]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T19:00:51.531 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:00:51 smithi022 ceph-mon[31985]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T19:00:51.531 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:00:51 smithi022 ceph-mon[31985]: from='mgr.15522 ' entity='mgr.x' 2024-03-28T19:00:52.424 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:00:52 smithi134 ceph-mon[35037]: pgmap v1669: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 10 op/s 2024-03-28T19:00:52.432 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:00:52 smithi079 ceph-mon[35312]: pgmap v1669: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 10 op/s 2024-03-28T19:00:52.531 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:00:52 smithi022 ceph-mon[31985]: pgmap v1669: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 10 op/s 2024-03-28T19:00:54.424 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:00:54 smithi134 ceph-mon[35037]: pgmap v1670: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:00:54.432 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:00:54 smithi079 ceph-mon[35312]: pgmap v1670: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:00:54.531 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:00:54 smithi022 ceph-mon[31985]: pgmap v1670: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:00:55.424 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:00:55 smithi134 ceph-mon[35037]: 2 slow requests, 0 included below; oldest blocked for > 2641.880997 secs 2024-03-28T19:00:55.432 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:00:55 smithi079 ceph-mon[35312]: 2 slow requests, 0 included below; oldest blocked for > 2641.880997 secs 2024-03-28T19:00:55.531 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:00:55 smithi022 ceph-mon[31985]: 2 slow requests, 0 included below; oldest blocked for > 2641.880997 secs 2024-03-28T19:00:56.424 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:00:56 smithi134 ceph-mon[35037]: pgmap v1671: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 10 op/s 2024-03-28T19:00:56.432 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:00:56 smithi079 ceph-mon[35312]: pgmap v1671: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 10 op/s 2024-03-28T19:00:56.531 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:00:56 smithi022 ceph-mon[31985]: pgmap v1671: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 10 op/s 2024-03-28T19:00:58.424 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:00:58 smithi134 ceph-mon[35037]: pgmap v1672: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 6 op/s 2024-03-28T19:00:58.432 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:00:58 smithi079 ceph-mon[35312]: pgmap v1672: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 6 op/s 2024-03-28T19:00:58.531 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:00:58 smithi022 ceph-mon[31985]: pgmap v1672: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 6 op/s 2024-03-28T19:01:00.424 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:01:00 smithi134 ceph-mon[35037]: pgmap v1673: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 6 op/s 2024-03-28T19:01:00.424 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:01:00 smithi134 ceph-mon[35037]: 2 slow requests, 0 included below; oldest blocked for > 2646.881062 secs 2024-03-28T19:01:00.432 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:01:00 smithi079 ceph-mon[35312]: pgmap v1673: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 6 op/s 2024-03-28T19:01:00.432 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:01:00 smithi079 ceph-mon[35312]: 2 slow requests, 0 included below; oldest blocked for > 2646.881062 secs 2024-03-28T19:01:00.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:01:00 smithi022 ceph-mon[31985]: pgmap v1673: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 6 op/s 2024-03-28T19:01:00.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:01:00 smithi022 ceph-mon[31985]: 2 slow requests, 0 included below; oldest blocked for > 2646.881062 secs 2024-03-28T19:01:02.424 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:01:02 smithi134 ceph-mon[35037]: pgmap v1674: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 10 op/s 2024-03-28T19:01:02.432 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:01:02 smithi079 ceph-mon[35312]: pgmap v1674: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 10 op/s 2024-03-28T19:01:02.531 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:01:02 smithi022 ceph-mon[31985]: pgmap v1674: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 10 op/s 2024-03-28T19:01:04.424 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:01:04 smithi134 ceph-mon[35037]: pgmap v1675: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:01:04.432 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:01:04 smithi079 ceph-mon[35312]: pgmap v1675: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:01:04.531 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:01:04 smithi022 ceph-mon[31985]: pgmap v1675: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:01:05.424 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:01:05 smithi134 ceph-mon[35037]: 2 slow requests, 0 included below; oldest blocked for > 2651.881148 secs 2024-03-28T19:01:05.432 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:01:05 smithi079 ceph-mon[35312]: 2 slow requests, 0 included below; oldest blocked for > 2651.881148 secs 2024-03-28T19:01:05.531 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:01:05 smithi022 ceph-mon[31985]: 2 slow requests, 0 included below; oldest blocked for > 2651.881148 secs 2024-03-28T19:01:06.424 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:01:06 smithi134 ceph-mon[35037]: pgmap v1676: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T19:01:06.432 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:01:06 smithi079 ceph-mon[35312]: pgmap v1676: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T19:01:06.531 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:01:06 smithi022 ceph-mon[31985]: pgmap v1676: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T19:01:08.424 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:01:08 smithi134 ceph-mon[35037]: pgmap v1677: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:01:08.432 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:01:08 smithi079 ceph-mon[35312]: pgmap v1677: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:01:08.531 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:01:08 smithi022 ceph-mon[31985]: pgmap v1677: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:01:10.424 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:01:10 smithi134 ceph-mon[35037]: pgmap v1678: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:01:10.424 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:01:10 smithi134 ceph-mon[35037]: 2 slow requests, 0 included below; oldest blocked for > 2656.881230 secs 2024-03-28T19:01:10.432 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:01:10 smithi079 ceph-mon[35312]: pgmap v1678: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:01:10.432 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:01:10 smithi079 ceph-mon[35312]: 2 slow requests, 0 included below; oldest blocked for > 2656.881230 secs 2024-03-28T19:01:10.531 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:01:10 smithi022 ceph-mon[31985]: pgmap v1678: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:01:10.531 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:01:10 smithi022 ceph-mon[31985]: 2 slow requests, 0 included below; oldest blocked for > 2656.881230 secs 2024-03-28T19:01:12.424 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:01:12 smithi134 ceph-mon[35037]: pgmap v1679: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T19:01:12.432 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:01:12 smithi079 ceph-mon[35312]: pgmap v1679: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T19:01:12.531 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:01:12 smithi022 ceph-mon[31985]: pgmap v1679: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T19:01:14.432 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:01:14 smithi079 ceph-mon[35312]: pgmap v1680: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:01:14.531 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:01:14 smithi022 ceph-mon[31985]: pgmap v1680: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:01:14.674 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:01:14 smithi134 ceph-mon[35037]: pgmap v1680: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:01:15.531 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:01:15 smithi022 ceph-mon[31985]: 2 slow requests, 0 included below; oldest blocked for > 2661.881306 secs 2024-03-28T19:01:15.674 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:01:15 smithi134 ceph-mon[35037]: 2 slow requests, 0 included below; oldest blocked for > 2661.881306 secs 2024-03-28T19:01:15.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:01:15 smithi079 ceph-mon[35312]: 2 slow requests, 0 included below; oldest blocked for > 2661.881306 secs 2024-03-28T19:01:16.531 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:01:16 smithi022 ceph-mon[31985]: pgmap v1681: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T19:01:16.576 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:01:16 smithi134 ceph-mon[35037]: pgmap v1681: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T19:01:16.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:01:16 smithi079 ceph-mon[35312]: pgmap v1681: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T19:01:18.269 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:01:18 smithi134 ceph-mon[35037]: pgmap v1682: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:01:18.531 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:01:18 smithi022 ceph-mon[31985]: pgmap v1682: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:01:18.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:01:18 smithi079 ceph-mon[35312]: pgmap v1682: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:01:20.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:01:20 smithi022 ceph-mon[31985]: pgmap v1683: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:01:20.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:01:20 smithi022 ceph-mon[31985]: 2 slow requests, 0 included below; oldest blocked for > 2666.881372 secs 2024-03-28T19:01:20.674 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:01:20 smithi134 ceph-mon[35037]: pgmap v1683: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:01:20.674 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:01:20 smithi134 ceph-mon[35037]: 2 slow requests, 0 included below; oldest blocked for > 2666.881372 secs 2024-03-28T19:01:20.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:01:20 smithi079 ceph-mon[35312]: pgmap v1683: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:01:20.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:01:20 smithi079 ceph-mon[35312]: 2 slow requests, 0 included below; oldest blocked for > 2666.881372 secs 2024-03-28T19:01:22.531 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:01:22 smithi022 ceph-mon[31985]: pgmap v1684: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T19:01:22.674 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:01:22 smithi134 ceph-mon[35037]: pgmap v1684: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T19:01:22.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:01:22 smithi079 ceph-mon[35312]: pgmap v1684: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T19:01:24.531 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:01:24 smithi022 ceph-mon[31985]: pgmap v1685: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:01:24.674 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:01:24 smithi134 ceph-mon[35037]: pgmap v1685: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:01:24.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:01:24 smithi079 ceph-mon[35312]: pgmap v1685: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:01:25.531 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:01:25 smithi022 ceph-mon[31985]: 2 slow requests, 0 included below; oldest blocked for > 2671.881444 secs 2024-03-28T19:01:25.674 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:01:25 smithi134 ceph-mon[35037]: 2 slow requests, 0 included below; oldest blocked for > 2671.881444 secs 2024-03-28T19:01:25.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:01:25 smithi079 ceph-mon[35312]: 2 slow requests, 0 included below; oldest blocked for > 2671.881444 secs 2024-03-28T19:01:26.531 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:01:26 smithi022 ceph-mon[31985]: pgmap v1686: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T19:01:26.584 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:01:26 smithi079 ceph-mon[35312]: pgmap v1686: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T19:01:26.674 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:01:26 smithi134 ceph-mon[35037]: pgmap v1686: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T19:01:28.531 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:01:28 smithi022 ceph-mon[31985]: pgmap v1687: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:01:28.627 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:01:28 smithi134 ceph-mon[35037]: pgmap v1687: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:01:28.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:01:28 smithi079 ceph-mon[35312]: pgmap v1687: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:01:30.531 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:01:30 smithi022 ceph-mon[31985]: pgmap v1688: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:01:30.531 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:01:30 smithi022 ceph-mon[31985]: 2 slow requests, 0 included below; oldest blocked for > 2676.881510 secs 2024-03-28T19:01:30.674 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:01:30 smithi134 ceph-mon[35037]: pgmap v1688: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:01:30.674 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:01:30 smithi134 ceph-mon[35037]: 2 slow requests, 0 included below; oldest blocked for > 2676.881510 secs 2024-03-28T19:01:30.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:01:30 smithi079 ceph-mon[35312]: pgmap v1688: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:01:30.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:01:30 smithi079 ceph-mon[35312]: 2 slow requests, 0 included below; oldest blocked for > 2676.881510 secs 2024-03-28T19:01:32.531 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:01:32 smithi022 ceph-mon[31985]: pgmap v1689: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T19:01:32.674 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:01:32 smithi134 ceph-mon[35037]: pgmap v1689: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T19:01:32.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:01:32 smithi079 ceph-mon[35312]: pgmap v1689: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T19:01:34.531 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:01:34 smithi022 ceph-mon[31985]: pgmap v1690: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:01:34.674 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:01:34 smithi134 ceph-mon[35037]: pgmap v1690: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:01:34.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:01:34 smithi079 ceph-mon[35312]: pgmap v1690: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:01:35.531 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:01:35 smithi022 ceph-mon[31985]: 2 slow requests, 0 included below; oldest blocked for > 2681.881632 secs 2024-03-28T19:01:35.674 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:01:35 smithi134 ceph-mon[35037]: 2 slow requests, 0 included below; oldest blocked for > 2681.881632 secs 2024-03-28T19:01:35.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:01:35 smithi079 ceph-mon[35312]: 2 slow requests, 0 included below; oldest blocked for > 2681.881632 secs 2024-03-28T19:01:36.531 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:01:36 smithi022 ceph-mon[31985]: pgmap v1691: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T19:01:36.674 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:01:36 smithi134 ceph-mon[35037]: pgmap v1691: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T19:01:36.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:01:36 smithi079 ceph-mon[35312]: pgmap v1691: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T19:01:38.531 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:01:38 smithi022 ceph-mon[31985]: pgmap v1692: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:01:38.674 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:01:38 smithi134 ceph-mon[35037]: pgmap v1692: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:01:38.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:01:38 smithi079 ceph-mon[35312]: pgmap v1692: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:01:40.531 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:01:40 smithi022 ceph-mon[31985]: pgmap v1693: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:01:40.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:01:40 smithi022 ceph-mon[31985]: 2 slow requests, 0 included below; oldest blocked for > 2686.881755 secs 2024-03-28T19:01:40.674 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:01:40 smithi134 ceph-mon[35037]: pgmap v1693: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:01:40.674 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:01:40 smithi134 ceph-mon[35037]: 2 slow requests, 0 included below; oldest blocked for > 2686.881755 secs 2024-03-28T19:01:40.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:01:40 smithi079 ceph-mon[35312]: pgmap v1693: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:01:40.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:01:40 smithi079 ceph-mon[35312]: 2 slow requests, 0 included below; oldest blocked for > 2686.881755 secs 2024-03-28T19:01:42.674 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:01:42 smithi134 ceph-mon[35037]: pgmap v1694: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T19:01:42.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:01:42 smithi079 ceph-mon[35312]: pgmap v1694: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T19:01:42.781 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:01:42 smithi022 ceph-mon[31985]: pgmap v1694: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T19:01:44.674 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:01:44 smithi134 ceph-mon[35037]: pgmap v1695: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:01:44.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:01:44 smithi079 ceph-mon[35312]: pgmap v1695: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:01:44.781 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:01:44 smithi022 ceph-mon[31985]: pgmap v1695: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:01:45.674 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:01:45 smithi134 ceph-mon[35037]: 2 slow requests, 0 included below; oldest blocked for > 2691.881832 secs 2024-03-28T19:01:45.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:01:45 smithi079 ceph-mon[35312]: 2 slow requests, 0 included below; oldest blocked for > 2691.881832 secs 2024-03-28T19:01:45.683 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:01:45 smithi022 ceph-mon[31985]: 2 slow requests, 0 included below; oldest blocked for > 2691.881832 secs 2024-03-28T19:01:46.575 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:01:46 smithi134 ceph-mon[35037]: pgmap v1696: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T19:01:46.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:01:46 smithi079 ceph-mon[35312]: pgmap v1696: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T19:01:46.781 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:01:46 smithi022 ceph-mon[31985]: pgmap v1696: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T19:01:48.674 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:01:48 smithi134 ceph-mon[35037]: pgmap v1697: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:01:48.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:01:48 smithi079 ceph-mon[35312]: pgmap v1697: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:01:48.781 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:01:48 smithi022 ceph-mon[31985]: pgmap v1697: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:01:50.674 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:01:50 smithi134 ceph-mon[35037]: pgmap v1698: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:01:50.674 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:01:50 smithi134 ceph-mon[35037]: 2 slow requests, 0 included below; oldest blocked for > 2696.881976 secs 2024-03-28T19:01:50.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:01:50 smithi079 ceph-mon[35312]: pgmap v1698: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:01:50.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:01:50 smithi079 ceph-mon[35312]: 2 slow requests, 0 included below; oldest blocked for > 2696.881976 secs 2024-03-28T19:01:50.781 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:01:50 smithi022 ceph-mon[31985]: pgmap v1698: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:01:50.781 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:01:50 smithi022 ceph-mon[31985]: 2 slow requests, 0 included below; oldest blocked for > 2696.881976 secs 2024-03-28T19:01:51.674 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:01:51 smithi134 ceph-mon[35037]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T19:01:51.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:01:51 smithi079 ceph-mon[35312]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T19:01:51.781 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:01:51 smithi022 ceph-mon[31985]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T19:01:52.674 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:01:52 smithi134 ceph-mon[35037]: pgmap v1699: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T19:01:52.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:01:52 smithi079 ceph-mon[35312]: pgmap v1699: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T19:01:52.781 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:01:52 smithi022 ceph-mon[31985]: pgmap v1699: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T19:01:53.674 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:01:53 smithi134 ceph-mon[35037]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T19:01:53.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:01:53 smithi134 ceph-mon[35037]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T19:01:53.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:01:53 smithi134 ceph-mon[35037]: pgmap v1700: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 6 op/s 2024-03-28T19:01:53.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:01:53 smithi134 ceph-mon[35037]: pgmap v1701: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-03-28T19:01:53.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:01:53 smithi134 ceph-mon[35037]: from='mgr.15522 ' entity='mgr.x' 2024-03-28T19:01:53.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:01:53 smithi079 ceph-mon[35312]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T19:01:53.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:01:53 smithi079 ceph-mon[35312]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T19:01:53.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:01:53 smithi079 ceph-mon[35312]: pgmap v1700: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 6 op/s 2024-03-28T19:01:53.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:01:53 smithi079 ceph-mon[35312]: pgmap v1701: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-03-28T19:01:53.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:01:53 smithi079 ceph-mon[35312]: from='mgr.15522 ' entity='mgr.x' 2024-03-28T19:01:53.781 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:01:53 smithi022 ceph-mon[31985]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T19:01:53.781 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:01:53 smithi022 ceph-mon[31985]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T19:01:53.781 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:01:53 smithi022 ceph-mon[31985]: pgmap v1700: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 6 op/s 2024-03-28T19:01:53.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:01:53 smithi022 ceph-mon[31985]: pgmap v1701: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-03-28T19:01:53.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:01:53 smithi022 ceph-mon[31985]: from='mgr.15522 ' entity='mgr.x' 2024-03-28T19:01:55.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:01:55 smithi022 ceph-mon[31985]: 2 slow requests, 0 included below; oldest blocked for > 2701.882093 secs 2024-03-28T19:01:55.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:01:55 smithi022 ceph-mon[31985]: pgmap v1702: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-03-28T19:01:55.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:01:55 smithi134 ceph-mon[35037]: 2 slow requests, 0 included below; oldest blocked for > 2701.882093 secs 2024-03-28T19:01:55.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:01:55 smithi134 ceph-mon[35037]: pgmap v1702: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-03-28T19:01:55.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:01:55 smithi079 ceph-mon[35312]: 2 slow requests, 0 included below; oldest blocked for > 2701.882093 secs 2024-03-28T19:01:55.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:01:55 smithi079 ceph-mon[35312]: pgmap v1702: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-03-28T19:01:57.734 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:01:57 smithi079 ceph-mon[35312]: pgmap v1703: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-03-28T19:01:57.781 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:01:57 smithi022 ceph-mon[31985]: pgmap v1703: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-03-28T19:01:57.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:01:57 smithi134 ceph-mon[35037]: pgmap v1703: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-03-28T19:01:59.781 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:01:59 smithi022 ceph-mon[31985]: pgmap v1704: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-03-28T19:01:59.925 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:01:59 smithi134 ceph-mon[35037]: pgmap v1704: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-03-28T19:01:59.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:01:59 smithi079 ceph-mon[35312]: pgmap v1704: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-03-28T19:02:00.781 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:02:00 smithi022 ceph-mon[31985]: 2 slow requests, 0 included below; oldest blocked for > 2706.882253 secs 2024-03-28T19:02:00.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:02:00 smithi134 ceph-mon[35037]: 2 slow requests, 0 included below; oldest blocked for > 2706.882253 secs 2024-03-28T19:02:00.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:02:00 smithi079 ceph-mon[35312]: 2 slow requests, 0 included below; oldest blocked for > 2706.882253 secs 2024-03-28T19:02:01.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:02:01 smithi022 ceph-mon[31985]: pgmap v1705: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-03-28T19:02:01.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:02:01 smithi134 ceph-mon[35037]: pgmap v1705: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-03-28T19:02:01.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:02:01 smithi079 ceph-mon[35312]: pgmap v1705: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-03-28T19:02:03.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:02:03 smithi022 ceph-mon[31985]: pgmap v1706: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 7 op/s 2024-03-28T19:02:03.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:02:03 smithi134 ceph-mon[35037]: pgmap v1706: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 7 op/s 2024-03-28T19:02:03.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:02:03 smithi079 ceph-mon[35312]: pgmap v1706: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 7 op/s 2024-03-28T19:02:05.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:02:05 smithi022 ceph-mon[31985]: 2 slow requests, 0 included below; oldest blocked for > 2711.882343 secs 2024-03-28T19:02:05.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:02:05 smithi022 ceph-mon[31985]: pgmap v1707: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T19:02:05.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:02:05 smithi134 ceph-mon[35037]: 2 slow requests, 0 included below; oldest blocked for > 2711.882343 secs 2024-03-28T19:02:05.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:02:05 smithi134 ceph-mon[35037]: pgmap v1707: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T19:02:05.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:02:05 smithi079 ceph-mon[35312]: 2 slow requests, 0 included below; oldest blocked for > 2711.882343 secs 2024-03-28T19:02:05.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:02:05 smithi079 ceph-mon[35312]: pgmap v1707: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T19:02:07.781 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:02:07 smithi022 ceph-mon[31985]: pgmap v1708: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:02:07.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:02:07 smithi134 ceph-mon[35037]: pgmap v1708: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:02:07.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:02:07 smithi079 ceph-mon[35312]: pgmap v1708: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:02:09.781 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:02:09 smithi022 ceph-mon[31985]: pgmap v1709: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:02:09.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:02:09 smithi134 ceph-mon[35037]: pgmap v1709: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:02:09.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:02:09 smithi079 ceph-mon[35312]: pgmap v1709: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:02:10.781 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:02:10 smithi022 ceph-mon[31985]: 2 slow requests, 0 included below; oldest blocked for > 2716.882413 secs 2024-03-28T19:02:10.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:02:10 smithi134 ceph-mon[35037]: 2 slow requests, 0 included below; oldest blocked for > 2716.882413 secs 2024-03-28T19:02:10.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:02:10 smithi079 ceph-mon[35312]: 2 slow requests, 0 included below; oldest blocked for > 2716.882413 secs 2024-03-28T19:02:11.781 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:02:11 smithi022 ceph-mon[31985]: pgmap v1710: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T19:02:11.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:02:11 smithi134 ceph-mon[35037]: pgmap v1710: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T19:02:11.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:02:11 smithi079 ceph-mon[35312]: pgmap v1710: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T19:02:13.781 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:02:13 smithi022 ceph-mon[31985]: pgmap v1711: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:02:13.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:02:13 smithi134 ceph-mon[35037]: pgmap v1711: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:02:13.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:02:13 smithi079 ceph-mon[35312]: pgmap v1711: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:02:15.781 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:02:15 smithi022 ceph-mon[31985]: 2 slow requests, 0 included below; oldest blocked for > 2721.882453 secs 2024-03-28T19:02:15.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:02:15 smithi022 ceph-mon[31985]: pgmap v1712: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T19:02:15.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:02:15 smithi134 ceph-mon[35037]: 2 slow requests, 0 included below; oldest blocked for > 2721.882453 secs 2024-03-28T19:02:15.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:02:15 smithi134 ceph-mon[35037]: pgmap v1712: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T19:02:15.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:02:15 smithi079 ceph-mon[35312]: 2 slow requests, 0 included below; oldest blocked for > 2721.882453 secs 2024-03-28T19:02:15.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:02:15 smithi079 ceph-mon[35312]: pgmap v1712: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T19:02:17.781 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:02:17 smithi022 ceph-mon[31985]: pgmap v1713: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:02:17.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:02:17 smithi134 ceph-mon[35037]: pgmap v1713: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:02:17.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:02:17 smithi079 ceph-mon[35312]: pgmap v1713: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:02:19.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:02:19 smithi022 ceph-mon[31985]: pgmap v1714: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:02:19.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:02:19 smithi134 ceph-mon[35037]: pgmap v1714: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:02:19.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:02:19 smithi079 ceph-mon[35312]: pgmap v1714: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:02:20.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:02:20 smithi134 ceph-mon[35037]: 2 slow requests, 0 included below; oldest blocked for > 2726.882530 secs 2024-03-28T19:02:20.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:02:20 smithi079 ceph-mon[35312]: 2 slow requests, 0 included below; oldest blocked for > 2726.882530 secs 2024-03-28T19:02:21.032 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:02:20 smithi022 ceph-mon[31985]: 2 slow requests, 0 included below; oldest blocked for > 2726.882530 secs 2024-03-28T19:02:21.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:02:21 smithi134 ceph-mon[35037]: pgmap v1715: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T19:02:21.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:02:21 smithi079 ceph-mon[35312]: pgmap v1715: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T19:02:22.031 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:02:21 smithi022 ceph-mon[31985]: pgmap v1715: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T19:02:23.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:02:23 smithi134 ceph-mon[35037]: pgmap v1716: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:02:23.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:02:23 smithi079 ceph-mon[35312]: pgmap v1716: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:02:24.031 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:02:23 smithi022 ceph-mon[31985]: pgmap v1716: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:02:25.925 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:02:25 smithi134 ceph-mon[35037]: 2 slow requests, 0 included below; oldest blocked for > 2731.882601 secs 2024-03-28T19:02:25.925 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:02:25 smithi134 ceph-mon[35037]: pgmap v1717: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T19:02:25.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:02:25 smithi079 ceph-mon[35312]: 2 slow requests, 0 included below; oldest blocked for > 2731.882601 secs 2024-03-28T19:02:25.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:02:25 smithi079 ceph-mon[35312]: pgmap v1717: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T19:02:26.031 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:02:25 smithi022 ceph-mon[31985]: 2 slow requests, 0 included below; oldest blocked for > 2731.882601 secs 2024-03-28T19:02:26.031 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:02:25 smithi022 ceph-mon[31985]: pgmap v1717: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T19:02:27.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:02:27 smithi134 ceph-mon[35037]: pgmap v1718: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:02:27.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:02:27 smithi079 ceph-mon[35312]: pgmap v1718: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:02:28.031 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:02:27 smithi022 ceph-mon[31985]: pgmap v1718: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:02:29.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:02:29 smithi134 ceph-mon[35037]: pgmap v1719: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:02:29.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:02:29 smithi079 ceph-mon[35312]: pgmap v1719: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:02:30.031 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:02:29 smithi022 ceph-mon[31985]: pgmap v1719: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:02:30.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:02:30 smithi134 ceph-mon[35037]: 2 slow requests, 0 included below; oldest blocked for > 2736.882676 secs 2024-03-28T19:02:30.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:02:30 smithi134 ceph-mon[35037]: pgmap v1720: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T19:02:30.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:02:30 smithi079 ceph-mon[35312]: 2 slow requests, 0 included below; oldest blocked for > 2736.882676 secs 2024-03-28T19:02:30.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:02:30 smithi079 ceph-mon[35312]: pgmap v1720: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T19:02:31.031 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:02:30 smithi022 ceph-mon[31985]: 2 slow requests, 0 included below; oldest blocked for > 2736.882676 secs 2024-03-28T19:02:31.031 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:02:30 smithi022 ceph-mon[31985]: pgmap v1720: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T19:02:33.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:02:33 smithi022 ceph-mon[31985]: pgmap v1721: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:02:33.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:02:33 smithi134 ceph-mon[35037]: pgmap v1721: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:02:33.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:02:33 smithi079 ceph-mon[35312]: pgmap v1721: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:02:35.781 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:02:35 smithi022 ceph-mon[31985]: 2 slow requests, 0 included below; oldest blocked for > 2741.882739 secs 2024-03-28T19:02:35.781 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:02:35 smithi022 ceph-mon[31985]: pgmap v1722: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T19:02:35.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:02:35 smithi134 ceph-mon[35037]: 2 slow requests, 0 included below; oldest blocked for > 2741.882739 secs 2024-03-28T19:02:35.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:02:35 smithi134 ceph-mon[35037]: pgmap v1722: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T19:02:35.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:02:35 smithi079 ceph-mon[35312]: 2 slow requests, 0 included below; oldest blocked for > 2741.882739 secs 2024-03-28T19:02:35.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:02:35 smithi079 ceph-mon[35312]: pgmap v1722: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T19:02:37.781 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:02:37 smithi022 ceph-mon[31985]: pgmap v1723: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:02:37.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:02:37 smithi134 ceph-mon[35037]: pgmap v1723: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:02:37.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:02:37 smithi079 ceph-mon[35312]: pgmap v1723: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:02:39.781 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:02:39 smithi022 ceph-mon[31985]: pgmap v1724: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:02:39.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:02:39 smithi134 ceph-mon[35037]: pgmap v1724: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:02:39.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:02:39 smithi079 ceph-mon[35312]: pgmap v1724: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:02:40.781 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:02:40 smithi022 ceph-mon[31985]: 2 slow requests, 0 included below; oldest blocked for > 2746.882807 secs 2024-03-28T19:02:40.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:02:40 smithi134 ceph-mon[35037]: 2 slow requests, 0 included below; oldest blocked for > 2746.882807 secs 2024-03-28T19:02:40.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:02:40 smithi079 ceph-mon[35312]: 2 slow requests, 0 included below; oldest blocked for > 2746.882807 secs 2024-03-28T19:02:41.781 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:02:41 smithi022 ceph-mon[31985]: pgmap v1725: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T19:02:41.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:02:41 smithi134 ceph-mon[35037]: pgmap v1725: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T19:02:41.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:02:41 smithi079 ceph-mon[35312]: pgmap v1725: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T19:02:43.781 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:02:43 smithi022 ceph-mon[31985]: pgmap v1726: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:02:43.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:02:43 smithi134 ceph-mon[35037]: pgmap v1726: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:02:43.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:02:43 smithi079 ceph-mon[35312]: pgmap v1726: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:02:45.781 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:02:45 smithi022 ceph-mon[31985]: 2 slow requests, 0 included below; oldest blocked for > 2751.882881 secs 2024-03-28T19:02:45.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:02:45 smithi022 ceph-mon[31985]: pgmap v1727: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T19:02:45.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:02:45 smithi134 ceph-mon[35037]: 2 slow requests, 0 included below; oldest blocked for > 2751.882881 secs 2024-03-28T19:02:45.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:02:45 smithi134 ceph-mon[35037]: pgmap v1727: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T19:02:45.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:02:45 smithi079 ceph-mon[35312]: 2 slow requests, 0 included below; oldest blocked for > 2751.882881 secs 2024-03-28T19:02:45.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:02:45 smithi079 ceph-mon[35312]: pgmap v1727: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T19:02:47.781 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:02:47 smithi022 ceph-mon[31985]: pgmap v1728: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:02:47.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:02:47 smithi134 ceph-mon[35037]: pgmap v1728: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:02:47.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:02:47 smithi079 ceph-mon[35312]: pgmap v1728: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:02:49.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:02:49 smithi022 ceph-mon[31985]: pgmap v1729: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:02:49.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:02:49 smithi134 ceph-mon[35037]: pgmap v1729: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:02:49.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:02:49 smithi079 ceph-mon[35312]: pgmap v1729: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:02:50.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:02:50 smithi022 ceph-mon[31985]: 2 slow requests, 0 included below; oldest blocked for > 2756.882917 secs 2024-03-28T19:02:50.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:02:50 smithi134 ceph-mon[35037]: 2 slow requests, 0 included below; oldest blocked for > 2756.882917 secs 2024-03-28T19:02:50.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:02:50 smithi079 ceph-mon[35312]: 2 slow requests, 0 included below; oldest blocked for > 2756.882917 secs 2024-03-28T19:02:51.781 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:02:51 smithi022 ceph-mon[31985]: pgmap v1730: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 9 op/s 2024-03-28T19:02:51.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:02:51 smithi134 ceph-mon[35037]: pgmap v1730: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 9 op/s 2024-03-28T19:02:51.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:02:51 smithi079 ceph-mon[35312]: pgmap v1730: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 9 op/s 2024-03-28T19:02:53.781 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:02:53 smithi022 ceph-mon[31985]: pgmap v1731: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T19:02:53.781 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:02:53 smithi022 ceph-mon[31985]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T19:02:53.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:02:53 smithi134 ceph-mon[35037]: pgmap v1731: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T19:02:53.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:02:53 smithi134 ceph-mon[35037]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T19:02:53.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:02:53 smithi079 ceph-mon[35312]: pgmap v1731: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T19:02:53.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:02:53 smithi079 ceph-mon[35312]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T19:02:54.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:02:54 smithi134 ceph-mon[35037]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T19:02:54.925 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:02:54 smithi134 ceph-mon[35037]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T19:02:54.925 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:02:54 smithi134 ceph-mon[35037]: from='mgr.15522 ' entity='mgr.x' 2024-03-28T19:02:54.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:02:54 smithi079 ceph-mon[35312]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T19:02:54.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:02:54 smithi079 ceph-mon[35312]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T19:02:54.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:02:54 smithi079 ceph-mon[35312]: from='mgr.15522 ' entity='mgr.x' 2024-03-28T19:02:55.031 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:02:54 smithi022 ceph-mon[31985]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T19:02:55.031 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:02:54 smithi022 ceph-mon[31985]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T19:02:55.032 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:02:54 smithi022 ceph-mon[31985]: from='mgr.15522 ' entity='mgr.x' 2024-03-28T19:02:55.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:02:55 smithi134 ceph-mon[35037]: pgmap v1732: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:02:55.925 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:02:55 smithi134 ceph-mon[35037]: pgmap v1733: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 KiB/s rd, 3 op/s 2024-03-28T19:02:55.925 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:02:55 smithi134 ceph-mon[35037]: 2 slow requests, 0 included below; oldest blocked for > 2761.883021 secs 2024-03-28T19:02:55.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:02:55 smithi079 ceph-mon[35312]: pgmap v1732: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:02:55.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:02:55 smithi079 ceph-mon[35312]: pgmap v1733: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 KiB/s rd, 3 op/s 2024-03-28T19:02:55.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:02:55 smithi079 ceph-mon[35312]: 2 slow requests, 0 included below; oldest blocked for > 2761.883021 secs 2024-03-28T19:02:56.031 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:02:55 smithi022 ceph-mon[31985]: pgmap v1732: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:02:56.031 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:02:55 smithi022 ceph-mon[31985]: pgmap v1733: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 KiB/s rd, 3 op/s 2024-03-28T19:02:56.032 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:02:55 smithi022 ceph-mon[31985]: 2 slow requests, 0 included below; oldest blocked for > 2761.883021 secs 2024-03-28T19:02:57.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:02:57 smithi134 ceph-mon[35037]: pgmap v1734: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 7 op/s 2024-03-28T19:02:57.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:02:57 smithi079 ceph-mon[35312]: pgmap v1734: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 7 op/s 2024-03-28T19:02:58.031 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:02:57 smithi022 ceph-mon[31985]: pgmap v1734: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 7 op/s 2024-03-28T19:02:59.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:02:59 smithi134 ceph-mon[35037]: pgmap v1735: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 7 op/s 2024-03-28T19:02:59.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:02:59 smithi079 ceph-mon[35312]: pgmap v1735: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 7 op/s 2024-03-28T19:03:00.031 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:02:59 smithi022 ceph-mon[31985]: pgmap v1735: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 7 op/s 2024-03-28T19:03:00.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:03:00 smithi134 ceph-mon[35037]: 2 slow requests, 0 included below; oldest blocked for > 2766.883139 secs 2024-03-28T19:03:00.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:03:00 smithi079 ceph-mon[35312]: 2 slow requests, 0 included below; oldest blocked for > 2766.883139 secs 2024-03-28T19:03:01.031 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:03:00 smithi022 ceph-mon[31985]: 2 slow requests, 0 included below; oldest blocked for > 2766.883139 secs 2024-03-28T19:03:01.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:03:01 smithi134 ceph-mon[35037]: pgmap v1736: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-03-28T19:03:01.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:03:01 smithi079 ceph-mon[35312]: pgmap v1736: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-03-28T19:03:02.031 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:03:01 smithi022 ceph-mon[31985]: pgmap v1736: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-03-28T19:03:03.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:03:03 smithi134 ceph-mon[35037]: pgmap v1737: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-03-28T19:03:03.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:03:03 smithi079 ceph-mon[35312]: pgmap v1737: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-03-28T19:03:04.031 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:03:03 smithi022 ceph-mon[31985]: pgmap v1737: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-03-28T19:03:04.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:03:04 smithi134 ceph-mon[35037]: pgmap v1738: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 7 op/s 2024-03-28T19:03:04.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:03:04 smithi079 ceph-mon[35312]: pgmap v1738: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 7 op/s 2024-03-28T19:03:05.032 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:03:04 smithi022 ceph-mon[31985]: pgmap v1738: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 7 op/s 2024-03-28T19:03:05.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:03:05 smithi134 ceph-mon[35037]: 2 slow requests, 0 included below; oldest blocked for > 2771.883208 secs 2024-03-28T19:03:05.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:03:05 smithi079 ceph-mon[35312]: 2 slow requests, 0 included below; oldest blocked for > 2771.883208 secs 2024-03-28T19:03:06.031 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:03:05 smithi022 ceph-mon[31985]: 2 slow requests, 0 included below; oldest blocked for > 2771.883208 secs 2024-03-28T19:03:06.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:03:06 smithi134 ceph-mon[35037]: pgmap v1739: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T19:03:06.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:03:06 smithi079 ceph-mon[35312]: pgmap v1739: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T19:03:07.031 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:03:06 smithi022 ceph-mon[31985]: pgmap v1739: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T19:03:09.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:03:09 smithi022 ceph-mon[31985]: pgmap v1740: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:03:09.674 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:03:09 smithi134 ceph-mon[35037]: pgmap v1740: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:03:09.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:03:09 smithi079 ceph-mon[35312]: pgmap v1740: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:03:10.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:03:10 smithi022 ceph-mon[31985]: 2 slow requests, 0 included below; oldest blocked for > 2776.883328 secs 2024-03-28T19:03:10.674 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:03:10 smithi134 ceph-mon[35037]: 2 slow requests, 0 included below; oldest blocked for > 2776.883328 secs 2024-03-28T19:03:10.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:03:10 smithi079 ceph-mon[35312]: 2 slow requests, 0 included below; oldest blocked for > 2776.883328 secs 2024-03-28T19:03:11.531 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:03:11 smithi022 ceph-mon[31985]: pgmap v1741: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T19:03:11.674 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:03:11 smithi134 ceph-mon[35037]: pgmap v1741: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T19:03:11.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:03:11 smithi079 ceph-mon[35312]: pgmap v1741: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T19:03:13.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:03:13 smithi022 ceph-mon[31985]: pgmap v1742: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:03:13.674 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:03:13 smithi134 ceph-mon[35037]: pgmap v1742: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:03:13.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:03:13 smithi079 ceph-mon[35312]: pgmap v1742: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:03:15.531 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:03:15 smithi022 ceph-mon[31985]: pgmap v1743: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:03:15.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:03:15 smithi022 ceph-mon[31985]: 2 slow requests, 0 included below; oldest blocked for > 2781.883400 secs 2024-03-28T19:03:15.674 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:03:15 smithi134 ceph-mon[35037]: pgmap v1743: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:03:15.674 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:03:15 smithi134 ceph-mon[35037]: 2 slow requests, 0 included below; oldest blocked for > 2781.883400 secs 2024-03-28T19:03:15.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:03:15 smithi079 ceph-mon[35312]: pgmap v1743: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:03:15.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:03:15 smithi079 ceph-mon[35312]: 2 slow requests, 0 included below; oldest blocked for > 2781.883400 secs 2024-03-28T19:03:17.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:03:17 smithi022 ceph-mon[31985]: pgmap v1744: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T19:03:17.674 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:03:17 smithi134 ceph-mon[35037]: pgmap v1744: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T19:03:17.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:03:17 smithi079 ceph-mon[35312]: pgmap v1744: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T19:03:19.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:03:19 smithi022 ceph-mon[31985]: pgmap v1745: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:03:19.674 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:03:19 smithi134 ceph-mon[35037]: pgmap v1745: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:03:19.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:03:19 smithi079 ceph-mon[35312]: pgmap v1745: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:03:20.531 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:03:20 smithi022 ceph-mon[31985]: 2 slow requests, 0 included below; oldest blocked for > 2786.883524 secs 2024-03-28T19:03:20.674 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:03:20 smithi134 ceph-mon[35037]: 2 slow requests, 0 included below; oldest blocked for > 2786.883524 secs 2024-03-28T19:03:20.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:03:20 smithi079 ceph-mon[35312]: 2 slow requests, 0 included below; oldest blocked for > 2786.883524 secs 2024-03-28T19:03:21.531 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:03:21 smithi022 ceph-mon[31985]: pgmap v1746: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T19:03:21.674 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:03:21 smithi134 ceph-mon[35037]: pgmap v1746: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T19:03:21.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:03:21 smithi079 ceph-mon[35312]: pgmap v1746: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T19:03:23.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:03:23 smithi134 ceph-mon[35037]: pgmap v1747: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:03:23.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:03:23 smithi079 ceph-mon[35312]: pgmap v1747: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:03:23.781 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:03:23 smithi022 ceph-mon[31985]: pgmap v1747: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:03:25.674 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:03:25 smithi134 ceph-mon[35037]: pgmap v1748: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:03:25.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:03:25 smithi134 ceph-mon[35037]: 2 slow requests, 0 included below; oldest blocked for > 2791.883645 secs 2024-03-28T19:03:25.683 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:03:25 smithi079 ceph-mon[35312]: pgmap v1748: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:03:25.683 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:03:25 smithi079 ceph-mon[35312]: 2 slow requests, 0 included below; oldest blocked for > 2791.883645 secs 2024-03-28T19:03:25.781 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:03:25 smithi022 ceph-mon[31985]: pgmap v1748: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:03:25.781 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:03:25 smithi022 ceph-mon[31985]: 2 slow requests, 0 included below; oldest blocked for > 2791.883645 secs 2024-03-28T19:03:27.674 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:03:27 smithi134 ceph-mon[35037]: pgmap v1749: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T19:03:27.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:03:27 smithi079 ceph-mon[35312]: pgmap v1749: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T19:03:27.781 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:03:27 smithi022 ceph-mon[31985]: pgmap v1749: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T19:03:29.674 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:03:29 smithi134 ceph-mon[35037]: pgmap v1750: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:03:29.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:03:29 smithi079 ceph-mon[35312]: pgmap v1750: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:03:29.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:03:29 smithi022 ceph-mon[31985]: pgmap v1750: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:03:30.674 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:03:30 smithi134 ceph-mon[35037]: 2 slow requests, 0 included below; oldest blocked for > 2796.883716 secs 2024-03-28T19:03:30.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:03:30 smithi079 ceph-mon[35312]: 2 slow requests, 0 included below; oldest blocked for > 2796.883716 secs 2024-03-28T19:03:30.781 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:03:30 smithi022 ceph-mon[31985]: 2 slow requests, 0 included below; oldest blocked for > 2796.883716 secs 2024-03-28T19:03:31.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:03:31 smithi134 ceph-mon[35037]: pgmap v1751: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T19:03:31.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:03:31 smithi079 ceph-mon[35312]: pgmap v1751: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T19:03:31.781 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:03:31 smithi022 ceph-mon[31985]: pgmap v1751: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T19:03:33.674 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:03:33 smithi134 ceph-mon[35037]: pgmap v1752: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:03:33.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:03:33 smithi079 ceph-mon[35312]: pgmap v1752: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:03:33.781 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:03:33 smithi022 ceph-mon[31985]: pgmap v1752: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:03:35.674 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:03:35 smithi134 ceph-mon[35037]: pgmap v1753: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:03:35.674 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:03:35 smithi134 ceph-mon[35037]: 2 slow requests, 0 included below; oldest blocked for > 2801.883753 secs 2024-03-28T19:03:35.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:03:35 smithi079 ceph-mon[35312]: pgmap v1753: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:03:35.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:03:35 smithi079 ceph-mon[35312]: 2 slow requests, 0 included below; oldest blocked for > 2801.883753 secs 2024-03-28T19:03:35.781 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:03:35 smithi022 ceph-mon[31985]: pgmap v1753: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:03:35.781 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:03:35 smithi022 ceph-mon[31985]: 2 slow requests, 0 included below; oldest blocked for > 2801.883753 secs 2024-03-28T19:03:37.674 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:03:37 smithi134 ceph-mon[35037]: pgmap v1754: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T19:03:37.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:03:37 smithi079 ceph-mon[35312]: pgmap v1754: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T19:03:37.781 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:03:37 smithi022 ceph-mon[31985]: pgmap v1754: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T19:03:39.674 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:03:39 smithi134 ceph-mon[35037]: pgmap v1755: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:03:39.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:03:39 smithi079 ceph-mon[35312]: pgmap v1755: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:03:39.781 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:03:39 smithi022 ceph-mon[31985]: pgmap v1755: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:03:40.674 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:03:40 smithi134 ceph-mon[35037]: 2 slow requests, 0 included below; oldest blocked for > 2806.883844 secs 2024-03-28T19:03:40.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:03:40 smithi079 ceph-mon[35312]: 2 slow requests, 0 included below; oldest blocked for > 2806.883844 secs 2024-03-28T19:03:40.781 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:03:40 smithi022 ceph-mon[31985]: 2 slow requests, 0 included below; oldest blocked for > 2806.883844 secs 2024-03-28T19:03:41.674 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:03:41 smithi134 ceph-mon[35037]: pgmap v1756: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T19:03:41.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:03:41 smithi079 ceph-mon[35312]: pgmap v1756: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T19:03:41.781 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:03:41 smithi022 ceph-mon[31985]: pgmap v1756: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T19:03:43.674 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:03:43 smithi134 ceph-mon[35037]: pgmap v1757: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:03:43.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:03:43 smithi079 ceph-mon[35312]: pgmap v1757: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:03:43.781 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:03:43 smithi022 ceph-mon[31985]: pgmap v1757: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:03:45.674 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:03:45 smithi134 ceph-mon[35037]: pgmap v1758: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:03:45.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:03:45 smithi134 ceph-mon[35037]: 2 slow requests, 0 included below; oldest blocked for > 2811.883919 secs 2024-03-28T19:03:45.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:03:45 smithi079 ceph-mon[35312]: pgmap v1758: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:03:45.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:03:45 smithi079 ceph-mon[35312]: 2 slow requests, 0 included below; oldest blocked for > 2811.883919 secs 2024-03-28T19:03:45.781 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:03:45 smithi022 ceph-mon[31985]: pgmap v1758: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:03:45.781 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:03:45 smithi022 ceph-mon[31985]: 2 slow requests, 0 included below; oldest blocked for > 2811.883919 secs 2024-03-28T19:03:47.674 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:03:47 smithi134 ceph-mon[35037]: pgmap v1759: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T19:03:47.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:03:47 smithi079 ceph-mon[35312]: pgmap v1759: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T19:03:47.781 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:03:47 smithi022 ceph-mon[31985]: pgmap v1759: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T19:03:49.674 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:03:49 smithi134 ceph-mon[35037]: pgmap v1760: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:03:49.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:03:49 smithi079 ceph-mon[35312]: pgmap v1760: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:03:49.781 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:03:49 smithi022 ceph-mon[31985]: pgmap v1760: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:03:50.674 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:03:50 smithi134 ceph-mon[35037]: 2 slow requests, 0 included below; oldest blocked for > 2816.884040 secs 2024-03-28T19:03:50.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:03:50 smithi079 ceph-mon[35312]: 2 slow requests, 0 included below; oldest blocked for > 2816.884040 secs 2024-03-28T19:03:50.781 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:03:50 smithi022 ceph-mon[31985]: 2 slow requests, 0 included below; oldest blocked for > 2816.884040 secs 2024-03-28T19:03:51.674 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:03:51 smithi134 ceph-mon[35037]: pgmap v1761: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T19:03:51.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:03:51 smithi079 ceph-mon[35312]: pgmap v1761: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T19:03:51.781 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:03:51 smithi022 ceph-mon[31985]: pgmap v1761: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T19:03:53.674 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:03:53 smithi134 ceph-mon[35037]: pgmap v1762: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:03:53.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:03:53 smithi079 ceph-mon[35312]: pgmap v1762: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:03:53.781 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:03:53 smithi022 ceph-mon[31985]: pgmap v1762: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:03:55.674 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:03:55 smithi134 ceph-mon[35037]: pgmap v1763: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:03:55.674 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:03:55 smithi134 ceph-mon[35037]: 2 slow requests, 0 included below; oldest blocked for > 2821.884123 secs 2024-03-28T19:03:55.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:03:55 smithi134 ceph-mon[35037]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T19:03:55.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:03:55 smithi079 ceph-mon[35312]: pgmap v1763: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:03:55.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:03:55 smithi079 ceph-mon[35312]: 2 slow requests, 0 included below; oldest blocked for > 2821.884123 secs 2024-03-28T19:03:55.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:03:55 smithi079 ceph-mon[35312]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T19:03:55.781 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:03:55 smithi022 ceph-mon[31985]: pgmap v1763: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:03:55.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:03:55 smithi022 ceph-mon[31985]: 2 slow requests, 0 included below; oldest blocked for > 2821.884123 secs 2024-03-28T19:03:55.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:03:55 smithi022 ceph-mon[31985]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T19:03:57.674 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:03:57 smithi134 ceph-mon[35037]: pgmap v1764: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T19:03:57.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:03:57 smithi079 ceph-mon[35312]: pgmap v1764: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T19:03:57.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:03:57 smithi022 ceph-mon[31985]: pgmap v1764: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T19:03:59.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:03:59 smithi022 ceph-mon[31985]: pgmap v1765: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:03:59.674 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:03:59 smithi134 ceph-mon[35037]: pgmap v1765: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:03:59.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:03:59 smithi079 ceph-mon[35312]: pgmap v1765: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:04:00.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:04:00 smithi079 ceph-mon[35312]: 2 slow requests, 0 included below; oldest blocked for > 2826.884190 secs 2024-03-28T19:04:00.781 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:04:00 smithi022 ceph-mon[31985]: 2 slow requests, 0 included below; oldest blocked for > 2826.884190 secs 2024-03-28T19:04:00.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:04:00 smithi134 ceph-mon[35037]: 2 slow requests, 0 included below; oldest blocked for > 2826.884190 secs 2024-03-28T19:04:01.531 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:04:01 smithi022 ceph-mon[31985]: pgmap v1766: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T19:04:01.531 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:04:01 smithi022 ceph-mon[31985]: from='mgr.15522 ' entity='mgr.x' 2024-03-28T19:04:01.531 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:04:01 smithi022 ceph-mon[31985]: from='mgr.15522 ' entity='mgr.x' 2024-03-28T19:04:01.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:04:01 smithi079 ceph-mon[35312]: pgmap v1766: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T19:04:01.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:04:01 smithi079 ceph-mon[35312]: from='mgr.15522 ' entity='mgr.x' 2024-03-28T19:04:01.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:04:01 smithi079 ceph-mon[35312]: from='mgr.15522 ' entity='mgr.x' 2024-03-28T19:04:01.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:04:01 smithi134 ceph-mon[35037]: pgmap v1766: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T19:04:01.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:04:01 smithi134 ceph-mon[35037]: from='mgr.15522 ' entity='mgr.x' 2024-03-28T19:04:01.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:04:01 smithi134 ceph-mon[35037]: from='mgr.15522 ' entity='mgr.x' 2024-03-28T19:04:03.424 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:04:03 smithi134 ceph-mon[35037]: from='mgr.15522 ' entity='mgr.x' 2024-03-28T19:04:03.424 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:04:03 smithi134 ceph-mon[35037]: from='mgr.15522 ' entity='mgr.x' 2024-03-28T19:04:03.424 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:04:03 smithi134 ceph-mon[35037]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T19:04:03.425 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:04:03 smithi134 ceph-mon[35037]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T19:04:03.425 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:04:03 smithi134 ceph-mon[35037]: pgmap v1767: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:04:03.425 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:04:03 smithi134 ceph-mon[35037]: pgmap v1768: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 8 op/s 2024-03-28T19:04:03.425 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:04:03 smithi134 ceph-mon[35037]: from='mgr.15522 ' entity='mgr.x' 2024-03-28T19:04:03.432 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:04:03 smithi079 ceph-mon[35312]: from='mgr.15522 ' entity='mgr.x' 2024-03-28T19:04:03.432 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:04:03 smithi079 ceph-mon[35312]: from='mgr.15522 ' entity='mgr.x' 2024-03-28T19:04:03.432 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:04:03 smithi079 ceph-mon[35312]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T19:04:03.432 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:04:03 smithi079 ceph-mon[35312]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T19:04:03.432 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:04:03 smithi079 ceph-mon[35312]: pgmap v1767: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:04:03.432 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:04:03 smithi079 ceph-mon[35312]: pgmap v1768: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 8 op/s 2024-03-28T19:04:03.432 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:04:03 smithi079 ceph-mon[35312]: from='mgr.15522 ' entity='mgr.x' 2024-03-28T19:04:03.531 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:04:03 smithi022 ceph-mon[31985]: from='mgr.15522 ' entity='mgr.x' 2024-03-28T19:04:03.531 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:04:03 smithi022 ceph-mon[31985]: from='mgr.15522 ' entity='mgr.x' 2024-03-28T19:04:03.531 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:04:03 smithi022 ceph-mon[31985]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T19:04:03.531 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:04:03 smithi022 ceph-mon[31985]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T19:04:03.531 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:04:03 smithi022 ceph-mon[31985]: pgmap v1767: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:04:03.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:04:03 smithi022 ceph-mon[31985]: pgmap v1768: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 8 op/s 2024-03-28T19:04:03.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:04:03 smithi022 ceph-mon[31985]: from='mgr.15522 ' entity='mgr.x' 2024-03-28T19:04:05.531 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:04:05 smithi022 ceph-mon[31985]: pgmap v1769: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 8 op/s 2024-03-28T19:04:05.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:04:05 smithi022 ceph-mon[31985]: 2 slow requests, 0 included below; oldest blocked for > 2831.884267 secs 2024-03-28T19:04:05.674 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:04:05 smithi134 ceph-mon[35037]: pgmap v1769: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 8 op/s 2024-03-28T19:04:05.674 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:04:05 smithi134 ceph-mon[35037]: 2 slow requests, 0 included below; oldest blocked for > 2831.884267 secs 2024-03-28T19:04:05.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:04:05 smithi079 ceph-mon[35312]: pgmap v1769: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 8 op/s 2024-03-28T19:04:05.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:04:05 smithi079 ceph-mon[35312]: 2 slow requests, 0 included below; oldest blocked for > 2831.884267 secs 2024-03-28T19:04:07.531 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:04:07 smithi022 ceph-mon[31985]: pgmap v1770: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 8 op/s 2024-03-28T19:04:07.674 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:04:07 smithi134 ceph-mon[35037]: pgmap v1770: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 8 op/s 2024-03-28T19:04:07.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:04:07 smithi079 ceph-mon[35312]: pgmap v1770: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 8 op/s 2024-03-28T19:04:09.531 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:04:09 smithi022 ceph-mon[31985]: pgmap v1771: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 8 op/s 2024-03-28T19:04:09.674 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:04:09 smithi134 ceph-mon[35037]: pgmap v1771: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 8 op/s 2024-03-28T19:04:09.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:04:09 smithi079 ceph-mon[35312]: pgmap v1771: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 8 op/s 2024-03-28T19:04:10.531 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:04:10 smithi022 ceph-mon[31985]: 2 slow requests, 0 included below; oldest blocked for > 2836.884292 secs 2024-03-28T19:04:10.674 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:04:10 smithi134 ceph-mon[35037]: 2 slow requests, 0 included below; oldest blocked for > 2836.884292 secs 2024-03-28T19:04:10.681 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:04:10 smithi079 ceph-mon[35312]: 2 slow requests, 0 included below; oldest blocked for > 2836.884292 secs 2024-03-28T19:04:11.432 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:04:11 smithi079 ceph-mon[35312]: pgmap v1772: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 8 op/s 2024-03-28T19:04:11.531 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:04:11 smithi022 ceph-mon[31985]: pgmap v1772: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 8 op/s 2024-03-28T19:04:11.674 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:04:11 smithi134 ceph-mon[35037]: pgmap v1772: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 8 op/s 2024-03-28T19:04:13.531 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:04:13 smithi022 ceph-mon[31985]: pgmap v1773: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 7 op/s 2024-03-28T19:04:13.674 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:04:13 smithi134 ceph-mon[35037]: pgmap v1773: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 7 op/s 2024-03-28T19:04:13.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:04:13 smithi079 ceph-mon[35312]: pgmap v1773: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 7 op/s 2024-03-28T19:04:15.531 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:04:15 smithi022 ceph-mon[31985]: pgmap v1774: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:04:15.531 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:04:15 smithi022 ceph-mon[31985]: 2 slow requests, 0 included below; oldest blocked for > 2841.884414 secs 2024-03-28T19:04:15.674 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:04:15 smithi134 ceph-mon[35037]: pgmap v1774: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:04:15.674 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:04:15 smithi134 ceph-mon[35037]: 2 slow requests, 0 included below; oldest blocked for > 2841.884414 secs 2024-03-28T19:04:15.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:04:15 smithi079 ceph-mon[35312]: pgmap v1774: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:04:15.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:04:15 smithi079 ceph-mon[35312]: 2 slow requests, 0 included below; oldest blocked for > 2841.884414 secs 2024-03-28T19:04:17.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:04:17 smithi022 ceph-mon[31985]: pgmap v1775: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T19:04:17.674 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:04:17 smithi134 ceph-mon[35037]: pgmap v1775: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T19:04:17.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:04:17 smithi079 ceph-mon[35312]: pgmap v1775: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T19:04:19.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:04:19 smithi022 ceph-mon[31985]: pgmap v1776: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:04:19.672 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:04:19 smithi079 ceph-mon[35312]: pgmap v1776: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:04:19.674 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:04:19 smithi134 ceph-mon[35037]: pgmap v1776: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:04:20.531 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:04:20 smithi022 ceph-mon[31985]: 2 slow requests, 0 included below; oldest blocked for > 2846.884536 secs 2024-03-28T19:04:20.674 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:04:20 smithi134 ceph-mon[35037]: 2 slow requests, 0 included below; oldest blocked for > 2846.884536 secs 2024-03-28T19:04:20.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:04:20 smithi079 ceph-mon[35312]: 2 slow requests, 0 included below; oldest blocked for > 2846.884536 secs 2024-03-28T19:04:21.531 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:04:21 smithi022 ceph-mon[31985]: pgmap v1777: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T19:04:21.674 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:04:21 smithi134 ceph-mon[35037]: pgmap v1777: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T19:04:21.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:04:21 smithi079 ceph-mon[35312]: pgmap v1777: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T19:04:23.531 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:04:23 smithi022 ceph-mon[31985]: pgmap v1778: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:04:23.674 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:04:23 smithi134 ceph-mon[35037]: pgmap v1778: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:04:23.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:04:23 smithi079 ceph-mon[35312]: pgmap v1778: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:04:25.531 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:04:25 smithi022 ceph-mon[31985]: pgmap v1779: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:04:25.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:04:25 smithi022 ceph-mon[31985]: 2 slow requests, 0 included below; oldest blocked for > 2851.884571 secs 2024-03-28T19:04:25.674 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:04:25 smithi134 ceph-mon[35037]: pgmap v1779: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:04:25.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:04:25 smithi134 ceph-mon[35037]: 2 slow requests, 0 included below; oldest blocked for > 2851.884571 secs 2024-03-28T19:04:25.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:04:25 smithi079 ceph-mon[35312]: pgmap v1779: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:04:25.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:04:25 smithi079 ceph-mon[35312]: 2 slow requests, 0 included below; oldest blocked for > 2851.884571 secs 2024-03-28T19:04:27.674 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:04:27 smithi134 ceph-mon[35037]: pgmap v1780: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T19:04:27.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:04:27 smithi079 ceph-mon[35312]: pgmap v1780: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T19:04:27.781 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:04:27 smithi022 ceph-mon[31985]: pgmap v1780: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T19:04:29.674 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:04:29 smithi134 ceph-mon[35037]: pgmap v1781: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:04:29.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:04:29 smithi079 ceph-mon[35312]: pgmap v1781: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:04:29.781 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:04:29 smithi022 ceph-mon[31985]: pgmap v1781: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:04:30.674 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:04:30 smithi134 ceph-mon[35037]: 2 slow requests, 0 included below; oldest blocked for > 2856.884691 secs 2024-03-28T19:04:30.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:04:30 smithi079 ceph-mon[35312]: 2 slow requests, 0 included below; oldest blocked for > 2856.884691 secs 2024-03-28T19:04:30.781 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:04:30 smithi022 ceph-mon[31985]: 2 slow requests, 0 included below; oldest blocked for > 2856.884691 secs 2024-03-28T19:04:31.674 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:04:31 smithi134 ceph-mon[35037]: pgmap v1782: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T19:04:31.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:04:31 smithi079 ceph-mon[35312]: pgmap v1782: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T19:04:31.781 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:04:31 smithi022 ceph-mon[31985]: pgmap v1782: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T19:04:33.674 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:04:33 smithi134 ceph-mon[35037]: pgmap v1783: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:04:33.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:04:33 smithi079 ceph-mon[35312]: pgmap v1783: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:04:33.781 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:04:33 smithi022 ceph-mon[31985]: pgmap v1783: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:04:35.674 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:04:35 smithi134 ceph-mon[35037]: pgmap v1784: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:04:35.674 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:04:35 smithi134 ceph-mon[35037]: 2 slow requests, 0 included below; oldest blocked for > 2861.884759 secs 2024-03-28T19:04:35.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:04:35 smithi079 ceph-mon[35312]: pgmap v1784: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:04:35.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:04:35 smithi079 ceph-mon[35312]: 2 slow requests, 0 included below; oldest blocked for > 2861.884759 secs 2024-03-28T19:04:35.781 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:04:35 smithi022 ceph-mon[31985]: pgmap v1784: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:04:35.781 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:04:35 smithi022 ceph-mon[31985]: 2 slow requests, 0 included below; oldest blocked for > 2861.884759 secs 2024-03-28T19:04:37.674 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:04:37 smithi134 ceph-mon[35037]: pgmap v1785: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T19:04:37.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:04:37 smithi079 ceph-mon[35312]: pgmap v1785: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T19:04:37.781 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:04:37 smithi022 ceph-mon[31985]: pgmap v1785: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T19:04:39.674 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:04:39 smithi134 ceph-mon[35037]: pgmap v1786: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:04:39.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:04:39 smithi079 ceph-mon[35312]: pgmap v1786: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:04:39.781 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:04:39 smithi022 ceph-mon[31985]: pgmap v1786: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:04:40.674 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:04:40 smithi134 ceph-mon[35037]: 2 slow requests, 0 included below; oldest blocked for > 2866.884787 secs 2024-03-28T19:04:40.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:04:40 smithi079 ceph-mon[35312]: 2 slow requests, 0 included below; oldest blocked for > 2866.884787 secs 2024-03-28T19:04:40.781 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:04:40 smithi022 ceph-mon[31985]: 2 slow requests, 0 included below; oldest blocked for > 2866.884787 secs 2024-03-28T19:04:41.674 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:04:41 smithi134 ceph-mon[35037]: pgmap v1787: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T19:04:41.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:04:41 smithi079 ceph-mon[35312]: pgmap v1787: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T19:04:41.781 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:04:41 smithi022 ceph-mon[31985]: pgmap v1787: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T19:04:43.674 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:04:43 smithi134 ceph-mon[35037]: pgmap v1788: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:04:43.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:04:43 smithi079 ceph-mon[35312]: pgmap v1788: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:04:43.781 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:04:43 smithi022 ceph-mon[31985]: pgmap v1788: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:04:45.674 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:04:45 smithi134 ceph-mon[35037]: pgmap v1789: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:04:45.674 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:04:45 smithi134 ceph-mon[35037]: 2 slow requests, 0 included below; oldest blocked for > 2871.884863 secs 2024-03-28T19:04:45.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:04:45 smithi079 ceph-mon[35312]: pgmap v1789: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:04:45.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:04:45 smithi079 ceph-mon[35312]: 2 slow requests, 0 included below; oldest blocked for > 2871.884863 secs 2024-03-28T19:04:45.781 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:04:45 smithi022 ceph-mon[31985]: pgmap v1789: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:04:45.781 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:04:45 smithi022 ceph-mon[31985]: 2 slow requests, 0 included below; oldest blocked for > 2871.884863 secs 2024-03-28T19:04:47.674 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:04:47 smithi134 ceph-mon[35037]: pgmap v1790: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T19:04:47.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:04:47 smithi079 ceph-mon[35312]: pgmap v1790: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T19:04:47.781 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:04:47 smithi022 ceph-mon[31985]: pgmap v1790: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T19:04:49.674 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:04:49 smithi134 ceph-mon[35037]: pgmap v1791: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:04:49.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:04:49 smithi079 ceph-mon[35312]: pgmap v1791: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:04:49.781 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:04:49 smithi022 ceph-mon[31985]: pgmap v1791: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:04:50.674 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:04:50 smithi134 ceph-mon[35037]: 2 slow requests, 0 included below; oldest blocked for > 2876.884970 secs 2024-03-28T19:04:50.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:04:50 smithi079 ceph-mon[35312]: 2 slow requests, 0 included below; oldest blocked for > 2876.884970 secs 2024-03-28T19:04:50.781 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:04:50 smithi022 ceph-mon[31985]: 2 slow requests, 0 included below; oldest blocked for > 2876.884970 secs 2024-03-28T19:04:51.674 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:04:51 smithi134 ceph-mon[35037]: pgmap v1792: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T19:04:51.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:04:51 smithi079 ceph-mon[35312]: pgmap v1792: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T19:04:51.781 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:04:51 smithi022 ceph-mon[31985]: pgmap v1792: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T19:04:53.674 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:04:53 smithi134 ceph-mon[35037]: pgmap v1793: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:04:53.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:04:53 smithi079 ceph-mon[35312]: pgmap v1793: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:04:53.781 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:04:53 smithi022 ceph-mon[31985]: pgmap v1793: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:04:55.674 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:04:55 smithi134 ceph-mon[35037]: pgmap v1794: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:04:55.674 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:04:55 smithi134 ceph-mon[35037]: 2 slow requests, 1 included below; oldest blocked for > 2881.885055 secs 2024-03-28T19:04:55.674 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:04:55 smithi134 ceph-mon[35037]: slow request 2881.885054 seconds old, received at 2024-03-28T18:16:52.457506+0000: peer_request:client.35222:10732 currently acquired locks 2024-03-28T19:04:55.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:04:55 smithi079 ceph-mon[35312]: pgmap v1794: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:04:55.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:04:55 smithi079 ceph-mon[35312]: 2 slow requests, 1 included below; oldest blocked for > 2881.885055 secs 2024-03-28T19:04:55.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:04:55 smithi079 ceph-mon[35312]: slow request 2881.885054 seconds old, received at 2024-03-28T18:16:52.457506+0000: peer_request:client.35222:10732 currently acquired locks 2024-03-28T19:04:55.781 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:04:55 smithi022 ceph-mon[31985]: pgmap v1794: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:04:55.781 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:04:55 smithi022 ceph-mon[31985]: 2 slow requests, 1 included below; oldest blocked for > 2881.885055 secs 2024-03-28T19:04:55.781 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:04:55 smithi022 ceph-mon[31985]: slow request 2881.885054 seconds old, received at 2024-03-28T18:16:52.457506+0000: peer_request:client.35222:10732 currently acquired locks 2024-03-28T19:04:57.674 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:04:57 smithi134 ceph-mon[35037]: pgmap v1795: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T19:04:57.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:04:57 smithi079 ceph-mon[35312]: pgmap v1795: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T19:04:57.781 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:04:57 smithi022 ceph-mon[31985]: pgmap v1795: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T19:04:59.674 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:04:59 smithi134 ceph-mon[35037]: pgmap v1796: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:04:59.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:04:59 smithi079 ceph-mon[35312]: pgmap v1796: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:04:59.781 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:04:59 smithi022 ceph-mon[31985]: pgmap v1796: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:05:00.781 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:05:00 smithi022 ceph-mon[31985]: 2 slow requests, 0 included below; oldest blocked for > 2886.885131 secs 2024-03-28T19:05:00.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:05:00 smithi134 ceph-mon[35037]: 2 slow requests, 0 included below; oldest blocked for > 2886.885131 secs 2024-03-28T19:05:00.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:05:00 smithi079 ceph-mon[35312]: 2 slow requests, 0 included below; oldest blocked for > 2886.885131 secs 2024-03-28T19:05:01.781 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:05:01 smithi022 ceph-mon[31985]: pgmap v1797: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T19:05:01.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:05:01 smithi134 ceph-mon[35037]: pgmap v1797: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T19:05:01.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:05:01 smithi079 ceph-mon[35312]: pgmap v1797: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T19:05:03.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:05:03 smithi022 ceph-mon[31985]: pgmap v1798: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:05:03.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:05:03 smithi022 ceph-mon[31985]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T19:05:03.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:05:03 smithi134 ceph-mon[35037]: pgmap v1798: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:05:03.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:05:03 smithi134 ceph-mon[35037]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T19:05:03.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:05:03 smithi079 ceph-mon[35312]: pgmap v1798: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:05:03.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:05:03 smithi079 ceph-mon[35312]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T19:05:05.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:05:05 smithi022 ceph-mon[31985]: pgmap v1799: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:05:05.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:05:05 smithi022 ceph-mon[31985]: 2 slow requests, 1 included below; oldest blocked for > 2891.885204 secs 2024-03-28T19:05:05.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:05:05 smithi022 ceph-mon[31985]: slow request 2884.187580 seconds old, received at 2024-03-28T18:17:00.155130+0000: client_request(client.35612:96 mksnap #0x1//scheduled-2024-03-28-18_17_00_UTC 2024-03-28T18:17:00.154942+0000 caller_uid=0, caller_gid=0{}) currently failed to xlock, waiting 2024-03-28T19:05:05.925 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:05:05 smithi134 ceph-mon[35037]: pgmap v1799: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:05:05.925 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:05:05 smithi134 ceph-mon[35037]: 2 slow requests, 1 included below; oldest blocked for > 2891.885204 secs 2024-03-28T19:05:05.925 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:05:05 smithi134 ceph-mon[35037]: slow request 2884.187580 seconds old, received at 2024-03-28T18:17:00.155130+0000: client_request(client.35612:96 mksnap #0x1//scheduled-2024-03-28-18_17_00_UTC 2024-03-28T18:17:00.154942+0000 caller_uid=0, caller_gid=0{}) currently failed to xlock, waiting 2024-03-28T19:05:05.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:05:05 smithi079 ceph-mon[35312]: pgmap v1799: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:05:05.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:05:05 smithi079 ceph-mon[35312]: 2 slow requests, 1 included below; oldest blocked for > 2891.885204 secs 2024-03-28T19:05:05.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:05:05 smithi079 ceph-mon[35312]: slow request 2884.187580 seconds old, received at 2024-03-28T18:17:00.155130+0000: client_request(client.35612:96 mksnap #0x1//scheduled-2024-03-28-18_17_00_UTC 2024-03-28T18:17:00.154942+0000 caller_uid=0, caller_gid=0{}) currently failed to xlock, waiting 2024-03-28T19:05:07.781 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:05:07 smithi022 ceph-mon[31985]: pgmap v1800: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 9 op/s 2024-03-28T19:05:07.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:05:07 smithi134 ceph-mon[35037]: pgmap v1800: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 9 op/s 2024-03-28T19:05:07.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:05:07 smithi079 ceph-mon[35312]: pgmap v1800: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 9 op/s 2024-03-28T19:05:09.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:05:09 smithi022 ceph-mon[31985]: pgmap v1801: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T19:05:09.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:05:09 smithi022 ceph-mon[31985]: from='mgr.15522 ' entity='mgr.x' 2024-03-28T19:05:09.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:05:09 smithi022 ceph-mon[31985]: from='mgr.15522 ' entity='mgr.x' 2024-03-28T19:05:09.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:05:09 smithi022 ceph-mon[31985]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T19:05:09.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:05:09 smithi022 ceph-mon[31985]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T19:05:09.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:05:09 smithi022 ceph-mon[31985]: from='mgr.15522 ' entity='mgr.x' 2024-03-28T19:05:09.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:05:09 smithi134 ceph-mon[35037]: pgmap v1801: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T19:05:09.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:05:09 smithi134 ceph-mon[35037]: from='mgr.15522 ' entity='mgr.x' 2024-03-28T19:05:09.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:05:09 smithi134 ceph-mon[35037]: from='mgr.15522 ' entity='mgr.x' 2024-03-28T19:05:09.925 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:05:09 smithi134 ceph-mon[35037]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T19:05:09.925 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:05:09 smithi134 ceph-mon[35037]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T19:05:09.925 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:05:09 smithi134 ceph-mon[35037]: from='mgr.15522 ' entity='mgr.x' 2024-03-28T19:05:09.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:05:09 smithi079 ceph-mon[35312]: pgmap v1801: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T19:05:09.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:05:09 smithi079 ceph-mon[35312]: from='mgr.15522 ' entity='mgr.x' 2024-03-28T19:05:09.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:05:09 smithi079 ceph-mon[35312]: from='mgr.15522 ' entity='mgr.x' 2024-03-28T19:05:09.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:05:09 smithi079 ceph-mon[35312]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T19:05:09.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:05:09 smithi079 ceph-mon[35312]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T19:05:09.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:05:09 smithi079 ceph-mon[35312]: from='mgr.15522 ' entity='mgr.x' 2024-03-28T19:05:10.781 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:05:10 smithi022 ceph-mon[31985]: pgmap v1802: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 7 op/s 2024-03-28T19:05:10.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:05:10 smithi022 ceph-mon[31985]: pgmap v1803: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s rd, 4 op/s 2024-03-28T19:05:10.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:05:10 smithi022 ceph-mon[31985]: 2 slow requests, 0 included below; oldest blocked for > 2896.885272 secs 2024-03-28T19:05:10.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:05:10 smithi134 ceph-mon[35037]: pgmap v1802: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 7 op/s 2024-03-28T19:05:10.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:05:10 smithi134 ceph-mon[35037]: pgmap v1803: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s rd, 4 op/s 2024-03-28T19:05:10.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:05:10 smithi134 ceph-mon[35037]: 2 slow requests, 0 included below; oldest blocked for > 2896.885272 secs 2024-03-28T19:05:10.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:05:10 smithi079 ceph-mon[35312]: pgmap v1802: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 7 op/s 2024-03-28T19:05:10.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:05:10 smithi079 ceph-mon[35312]: pgmap v1803: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s rd, 4 op/s 2024-03-28T19:05:10.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:05:10 smithi079 ceph-mon[35312]: 2 slow requests, 0 included below; oldest blocked for > 2896.885272 secs 2024-03-28T19:05:12.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:05:12 smithi022 ceph-mon[31985]: pgmap v1804: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 8 op/s 2024-03-28T19:05:12.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:05:12 smithi134 ceph-mon[35037]: pgmap v1804: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 8 op/s 2024-03-28T19:05:12.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:05:12 smithi079 ceph-mon[35312]: pgmap v1804: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 8 op/s 2024-03-28T19:05:14.781 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:05:14 smithi022 ceph-mon[31985]: pgmap v1805: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 8 op/s 2024-03-28T19:05:14.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:05:14 smithi134 ceph-mon[35037]: pgmap v1805: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 8 op/s 2024-03-28T19:05:14.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:05:14 smithi079 ceph-mon[35312]: pgmap v1805: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 8 op/s 2024-03-28T19:05:15.781 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:05:15 smithi022 ceph-mon[31985]: 2 slow requests, 0 included below; oldest blocked for > 2901.885340 secs 2024-03-28T19:05:15.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:05:15 smithi134 ceph-mon[35037]: 2 slow requests, 0 included below; oldest blocked for > 2901.885340 secs 2024-03-28T19:05:15.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:05:15 smithi079 ceph-mon[35312]: 2 slow requests, 0 included below; oldest blocked for > 2901.885340 secs 2024-03-28T19:05:16.781 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:05:16 smithi022 ceph-mon[31985]: pgmap v1806: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 8 op/s 2024-03-28T19:05:16.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:05:16 smithi134 ceph-mon[35037]: pgmap v1806: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 8 op/s 2024-03-28T19:05:16.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:05:16 smithi079 ceph-mon[35312]: pgmap v1806: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 8 op/s 2024-03-28T19:05:18.749 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:05:18 smithi134 ceph-mon[35037]: pgmap v1807: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 8 op/s 2024-03-28T19:05:18.781 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:05:18 smithi022 ceph-mon[31985]: pgmap v1807: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 8 op/s 2024-03-28T19:05:18.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:05:18 smithi079 ceph-mon[35312]: pgmap v1807: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 8 op/s 2024-03-28T19:05:20.674 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:05:20 smithi134 ceph-mon[35037]: pgmap v1808: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 7 op/s 2024-03-28T19:05:20.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:05:20 smithi134 ceph-mon[35037]: 2 slow requests, 0 included below; oldest blocked for > 2906.885462 secs 2024-03-28T19:05:20.781 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:05:20 smithi022 ceph-mon[31985]: pgmap v1808: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 7 op/s 2024-03-28T19:05:20.781 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:05:20 smithi022 ceph-mon[31985]: 2 slow requests, 0 included below; oldest blocked for > 2906.885462 secs 2024-03-28T19:05:20.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:05:20 smithi079 ceph-mon[35312]: pgmap v1808: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 7 op/s 2024-03-28T19:05:20.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:05:20 smithi079 ceph-mon[35312]: 2 slow requests, 0 included below; oldest blocked for > 2906.885462 secs 2024-03-28T19:05:22.781 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:05:22 smithi022 ceph-mon[31985]: pgmap v1809: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T19:05:22.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:05:22 smithi134 ceph-mon[35037]: pgmap v1809: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T19:05:22.931 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:05:22 smithi079 ceph-mon[35312]: pgmap v1809: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T19:05:24.781 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:05:24 smithi022 ceph-mon[31985]: pgmap v1810: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:05:24.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:05:24 smithi134 ceph-mon[35037]: pgmap v1810: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:05:24.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:05:24 smithi079 ceph-mon[35312]: pgmap v1810: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:05:25.781 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:05:25 smithi022 ceph-mon[31985]: 2 slow requests, 0 included below; oldest blocked for > 2911.885530 secs 2024-03-28T19:05:25.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:05:25 smithi134 ceph-mon[35037]: 2 slow requests, 0 included below; oldest blocked for > 2911.885530 secs 2024-03-28T19:05:25.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:05:25 smithi079 ceph-mon[35312]: 2 slow requests, 0 included below; oldest blocked for > 2911.885530 secs 2024-03-28T19:05:26.781 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:05:26 smithi022 ceph-mon[31985]: pgmap v1811: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T19:05:26.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:05:26 smithi134 ceph-mon[35037]: pgmap v1811: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T19:05:26.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:05:26 smithi079 ceph-mon[35312]: pgmap v1811: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T19:05:28.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:05:28 smithi134 ceph-mon[35037]: pgmap v1812: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:05:28.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:05:28 smithi079 ceph-mon[35312]: pgmap v1812: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:05:29.031 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:05:28 smithi022 ceph-mon[31985]: pgmap v1812: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:05:30.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:05:30 smithi134 ceph-mon[35037]: pgmap v1813: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:05:30.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:05:30 smithi134 ceph-mon[35037]: 2 slow requests, 0 included below; oldest blocked for > 2916.885602 secs 2024-03-28T19:05:30.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:05:30 smithi079 ceph-mon[35312]: pgmap v1813: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:05:30.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:05:30 smithi079 ceph-mon[35312]: 2 slow requests, 0 included below; oldest blocked for > 2916.885602 secs 2024-03-28T19:05:31.031 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:05:30 smithi022 ceph-mon[31985]: pgmap v1813: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:05:31.031 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:05:30 smithi022 ceph-mon[31985]: 2 slow requests, 0 included below; oldest blocked for > 2916.885602 secs 2024-03-28T19:05:32.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:05:32 smithi134 ceph-mon[35037]: pgmap v1814: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T19:05:32.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:05:32 smithi079 ceph-mon[35312]: pgmap v1814: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T19:05:33.031 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:05:32 smithi022 ceph-mon[31985]: pgmap v1814: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T19:05:34.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:05:34 smithi134 ceph-mon[35037]: pgmap v1815: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:05:34.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:05:34 smithi079 ceph-mon[35312]: pgmap v1815: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:05:35.031 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:05:34 smithi022 ceph-mon[31985]: pgmap v1815: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:05:35.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:05:35 smithi134 ceph-mon[35037]: 2 slow requests, 0 included below; oldest blocked for > 2921.885673 secs 2024-03-28T19:05:35.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:05:35 smithi079 ceph-mon[35312]: 2 slow requests, 0 included below; oldest blocked for > 2921.885673 secs 2024-03-28T19:05:36.031 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:05:35 smithi022 ceph-mon[31985]: 2 slow requests, 0 included below; oldest blocked for > 2921.885673 secs 2024-03-28T19:05:36.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:05:36 smithi134 ceph-mon[35037]: pgmap v1816: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T19:05:36.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:05:36 smithi079 ceph-mon[35312]: pgmap v1816: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T19:05:37.031 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:05:36 smithi022 ceph-mon[31985]: pgmap v1816: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T19:05:38.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:05:38 smithi134 ceph-mon[35037]: pgmap v1817: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:05:38.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:05:38 smithi079 ceph-mon[35312]: pgmap v1817: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:05:39.031 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:05:38 smithi022 ceph-mon[31985]: pgmap v1817: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:05:40.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:05:40 smithi134 ceph-mon[35037]: pgmap v1818: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:05:40.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:05:40 smithi134 ceph-mon[35037]: 2 slow requests, 0 included below; oldest blocked for > 2926.885716 secs 2024-03-28T19:05:40.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:05:40 smithi079 ceph-mon[35312]: pgmap v1818: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:05:40.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:05:40 smithi079 ceph-mon[35312]: 2 slow requests, 0 included below; oldest blocked for > 2926.885716 secs 2024-03-28T19:05:41.031 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:05:40 smithi022 ceph-mon[31985]: pgmap v1818: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:05:41.031 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:05:40 smithi022 ceph-mon[31985]: 2 slow requests, 0 included below; oldest blocked for > 2926.885716 secs 2024-03-28T19:05:42.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:05:42 smithi134 ceph-mon[35037]: pgmap v1819: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T19:05:42.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:05:42 smithi079 ceph-mon[35312]: pgmap v1819: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T19:05:43.031 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:05:42 smithi022 ceph-mon[31985]: pgmap v1819: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T19:05:44.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:05:44 smithi134 ceph-mon[35037]: pgmap v1820: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:05:44.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:05:44 smithi079 ceph-mon[35312]: pgmap v1820: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:05:45.031 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:05:44 smithi022 ceph-mon[31985]: pgmap v1820: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:05:45.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:05:45 smithi134 ceph-mon[35037]: 2 slow requests, 0 included below; oldest blocked for > 2931.885780 secs 2024-03-28T19:05:45.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:05:45 smithi079 ceph-mon[35312]: 2 slow requests, 0 included below; oldest blocked for > 2931.885780 secs 2024-03-28T19:05:45.933 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:05:45 smithi022 ceph-mon[31985]: 2 slow requests, 0 included below; oldest blocked for > 2931.885780 secs 2024-03-28T19:05:46.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:05:46 smithi134 ceph-mon[35037]: pgmap v1821: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T19:05:46.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:05:46 smithi079 ceph-mon[35312]: pgmap v1821: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T19:05:47.031 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:05:46 smithi022 ceph-mon[31985]: pgmap v1821: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T19:05:48.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:05:48 smithi134 ceph-mon[35037]: pgmap v1822: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:05:48.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:05:48 smithi079 ceph-mon[35312]: pgmap v1822: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:05:49.031 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:05:48 smithi022 ceph-mon[31985]: pgmap v1822: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:05:50.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:05:50 smithi134 ceph-mon[35037]: pgmap v1823: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:05:50.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:05:50 smithi134 ceph-mon[35037]: 2 slow requests, 0 included below; oldest blocked for > 2936.885916 secs 2024-03-28T19:05:50.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:05:50 smithi079 ceph-mon[35312]: pgmap v1823: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:05:50.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:05:50 smithi079 ceph-mon[35312]: 2 slow requests, 0 included below; oldest blocked for > 2936.885916 secs 2024-03-28T19:05:51.031 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:05:50 smithi022 ceph-mon[31985]: pgmap v1823: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:05:51.031 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:05:50 smithi022 ceph-mon[31985]: 2 slow requests, 0 included below; oldest blocked for > 2936.885916 secs 2024-03-28T19:05:51.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:05:51 smithi134 ceph-mon[35037]: pgmap v1824: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T19:05:51.931 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:05:51 smithi079 ceph-mon[35312]: pgmap v1824: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T19:05:52.031 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:05:51 smithi022 ceph-mon[31985]: pgmap v1824: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T19:05:54.531 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:05:54 smithi022 ceph-mon[31985]: pgmap v1825: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:05:54.674 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:05:54 smithi134 ceph-mon[35037]: pgmap v1825: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:05:54.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:05:54 smithi079 ceph-mon[35312]: pgmap v1825: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:05:55.531 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:05:55 smithi022 ceph-mon[31985]: 2 slow requests, 0 included below; oldest blocked for > 2941.886009 secs 2024-03-28T19:05:55.674 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:05:55 smithi134 ceph-mon[35037]: 2 slow requests, 0 included below; oldest blocked for > 2941.886009 secs 2024-03-28T19:05:55.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:05:55 smithi079 ceph-mon[35312]: 2 slow requests, 0 included below; oldest blocked for > 2941.886009 secs 2024-03-28T19:05:56.531 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:05:56 smithi022 ceph-mon[31985]: pgmap v1826: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T19:05:56.674 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:05:56 smithi134 ceph-mon[35037]: pgmap v1826: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T19:05:56.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:05:56 smithi079 ceph-mon[35312]: pgmap v1826: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T19:05:58.485 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:05:58 smithi079 ceph-mon[35312]: pgmap v1827: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:05:58.531 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:05:58 smithi022 ceph-mon[31985]: pgmap v1827: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:05:58.674 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:05:58 smithi134 ceph-mon[35037]: pgmap v1827: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:06:00.531 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:06:00 smithi022 ceph-mon[31985]: pgmap v1828: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:06:00.531 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:06:00 smithi022 ceph-mon[31985]: 2 slow requests, 0 included below; oldest blocked for > 2946.886075 secs 2024-03-28T19:06:00.674 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:06:00 smithi134 ceph-mon[35037]: pgmap v1828: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:06:00.674 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:06:00 smithi134 ceph-mon[35037]: 2 slow requests, 0 included below; oldest blocked for > 2946.886075 secs 2024-03-28T19:06:00.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:06:00 smithi079 ceph-mon[35312]: pgmap v1828: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:06:00.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:06:00 smithi079 ceph-mon[35312]: 2 slow requests, 0 included below; oldest blocked for > 2946.886075 secs 2024-03-28T19:06:02.531 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:06:02 smithi022 ceph-mon[31985]: pgmap v1829: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T19:06:02.674 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:06:02 smithi134 ceph-mon[35037]: pgmap v1829: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T19:06:02.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:06:02 smithi079 ceph-mon[35312]: pgmap v1829: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T19:06:04.531 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:06:04 smithi022 ceph-mon[31985]: pgmap v1830: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:06:04.674 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:06:04 smithi134 ceph-mon[35037]: pgmap v1830: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:06:04.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:06:04 smithi079 ceph-mon[35312]: pgmap v1830: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:06:05.531 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:06:05 smithi022 ceph-mon[31985]: 2 slow requests, 0 included below; oldest blocked for > 2951.886160 secs 2024-03-28T19:06:05.674 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:06:05 smithi134 ceph-mon[35037]: 2 slow requests, 0 included below; oldest blocked for > 2951.886160 secs 2024-03-28T19:06:05.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:06:05 smithi079 ceph-mon[35312]: 2 slow requests, 0 included below; oldest blocked for > 2951.886160 secs 2024-03-28T19:06:06.531 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:06:06 smithi022 ceph-mon[31985]: pgmap v1831: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T19:06:06.674 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:06:06 smithi134 ceph-mon[35037]: pgmap v1831: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T19:06:06.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:06:06 smithi079 ceph-mon[35312]: pgmap v1831: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T19:06:08.533 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:06:08 smithi022 ceph-mon[31985]: pgmap v1832: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:06:08.674 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:06:08 smithi134 ceph-mon[35037]: pgmap v1832: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:06:08.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:06:08 smithi079 ceph-mon[35312]: pgmap v1832: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:06:10.531 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:06:10 smithi022 ceph-mon[31985]: pgmap v1833: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:06:10.531 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:06:10 smithi022 ceph-mon[31985]: 2 slow requests, 0 included below; oldest blocked for > 2956.886299 secs 2024-03-28T19:06:10.531 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:06:10 smithi022 ceph-mon[31985]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T19:06:10.674 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:06:10 smithi134 ceph-mon[35037]: pgmap v1833: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:06:10.674 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:06:10 smithi134 ceph-mon[35037]: 2 slow requests, 0 included below; oldest blocked for > 2956.886299 secs 2024-03-28T19:06:10.674 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:06:10 smithi134 ceph-mon[35037]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T19:06:10.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:06:10 smithi079 ceph-mon[35312]: pgmap v1833: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:06:10.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:06:10 smithi079 ceph-mon[35312]: 2 slow requests, 0 included below; oldest blocked for > 2956.886299 secs 2024-03-28T19:06:10.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:06:10 smithi079 ceph-mon[35312]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T19:06:11.531 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:06:11 smithi022 ceph-mon[31985]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T19:06:11.531 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:06:11 smithi022 ceph-mon[31985]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T19:06:11.531 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:06:11 smithi022 ceph-mon[31985]: from='mgr.15522 ' entity='mgr.x' 2024-03-28T19:06:11.674 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:06:11 smithi134 ceph-mon[35037]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T19:06:11.674 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:06:11 smithi134 ceph-mon[35037]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T19:06:11.674 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:06:11 smithi134 ceph-mon[35037]: from='mgr.15522 ' entity='mgr.x' 2024-03-28T19:06:11.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:06:11 smithi079 ceph-mon[35312]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T19:06:11.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:06:11 smithi079 ceph-mon[35312]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T19:06:11.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:06:11 smithi079 ceph-mon[35312]: from='mgr.15522 ' entity='mgr.x' 2024-03-28T19:06:12.531 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:06:12 smithi022 ceph-mon[31985]: pgmap v1834: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 10 op/s 2024-03-28T19:06:12.531 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:06:12 smithi022 ceph-mon[31985]: pgmap v1835: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-03-28T19:06:12.674 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:06:12 smithi134 ceph-mon[35037]: pgmap v1834: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 10 op/s 2024-03-28T19:06:12.674 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:06:12 smithi134 ceph-mon[35037]: pgmap v1835: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-03-28T19:06:12.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:06:12 smithi079 ceph-mon[35312]: pgmap v1834: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 10 op/s 2024-03-28T19:06:12.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:06:12 smithi079 ceph-mon[35312]: pgmap v1835: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-03-28T19:06:14.674 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:06:14 smithi134 ceph-mon[35037]: pgmap v1836: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-03-28T19:06:14.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:06:14 smithi079 ceph-mon[35312]: pgmap v1836: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-03-28T19:06:14.781 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:06:14 smithi022 ceph-mon[31985]: pgmap v1836: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-03-28T19:06:15.674 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:06:15 smithi134 ceph-mon[35037]: 2 slow requests, 0 included below; oldest blocked for > 2961.886391 secs 2024-03-28T19:06:15.681 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:06:15 smithi079 ceph-mon[35312]: 2 slow requests, 0 included below; oldest blocked for > 2961.886391 secs 2024-03-28T19:06:15.683 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:06:15 smithi022 ceph-mon[31985]: 2 slow requests, 0 included below; oldest blocked for > 2961.886391 secs 2024-03-28T19:06:16.674 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:06:16 smithi134 ceph-mon[35037]: pgmap v1837: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-03-28T19:06:16.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:06:16 smithi079 ceph-mon[35312]: pgmap v1837: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-03-28T19:06:16.781 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:06:16 smithi022 ceph-mon[31985]: pgmap v1837: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-03-28T19:06:18.674 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:06:18 smithi134 ceph-mon[35037]: pgmap v1838: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-03-28T19:06:18.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:06:18 smithi079 ceph-mon[35312]: pgmap v1838: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-03-28T19:06:18.781 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:06:18 smithi022 ceph-mon[31985]: pgmap v1838: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-03-28T19:06:20.674 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:06:20 smithi134 ceph-mon[35037]: pgmap v1839: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-03-28T19:06:20.674 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:06:20 smithi134 ceph-mon[35037]: 2 slow requests, 0 included below; oldest blocked for > 2966.886511 secs 2024-03-28T19:06:20.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:06:20 smithi079 ceph-mon[35312]: pgmap v1839: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-03-28T19:06:20.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:06:20 smithi079 ceph-mon[35312]: 2 slow requests, 0 included below; oldest blocked for > 2966.886511 secs 2024-03-28T19:06:20.781 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:06:20 smithi022 ceph-mon[31985]: pgmap v1839: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-03-28T19:06:20.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:06:20 smithi022 ceph-mon[31985]: 2 slow requests, 0 included below; oldest blocked for > 2966.886511 secs 2024-03-28T19:06:22.674 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:06:22 smithi134 ceph-mon[35037]: pgmap v1840: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 7 op/s 2024-03-28T19:06:22.681 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:06:22 smithi079 ceph-mon[35312]: pgmap v1840: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 7 op/s 2024-03-28T19:06:22.781 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:06:22 smithi022 ceph-mon[31985]: pgmap v1840: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 7 op/s 2024-03-28T19:06:24.674 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:06:24 smithi134 ceph-mon[35037]: pgmap v1841: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:06:24.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:06:24 smithi079 ceph-mon[35312]: pgmap v1841: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:06:24.781 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:06:24 smithi022 ceph-mon[31985]: pgmap v1841: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:06:25.674 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:06:25 smithi134 ceph-mon[35037]: 2 slow requests, 0 included below; oldest blocked for > 2971.886630 secs 2024-03-28T19:06:25.681 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:06:25 smithi079 ceph-mon[35312]: 2 slow requests, 0 included below; oldest blocked for > 2971.886630 secs 2024-03-28T19:06:25.781 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:06:25 smithi022 ceph-mon[31985]: 2 slow requests, 0 included below; oldest blocked for > 2971.886630 secs 2024-03-28T19:06:26.674 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:06:26 smithi134 ceph-mon[35037]: pgmap v1842: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T19:06:26.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:06:26 smithi079 ceph-mon[35312]: pgmap v1842: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T19:06:26.781 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:06:26 smithi022 ceph-mon[31985]: pgmap v1842: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T19:06:28.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:06:28 smithi134 ceph-mon[35037]: pgmap v1843: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:06:28.683 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:06:28 smithi079 ceph-mon[35312]: pgmap v1843: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:06:28.781 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:06:28 smithi022 ceph-mon[31985]: pgmap v1843: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:06:30.674 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:06:30 smithi134 ceph-mon[35037]: pgmap v1844: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:06:30.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:06:30 smithi134 ceph-mon[35037]: 2 slow requests, 0 included below; oldest blocked for > 2976.886712 secs 2024-03-28T19:06:30.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:06:30 smithi079 ceph-mon[35312]: pgmap v1844: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:06:30.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:06:30 smithi079 ceph-mon[35312]: 2 slow requests, 0 included below; oldest blocked for > 2976.886712 secs 2024-03-28T19:06:30.781 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:06:30 smithi022 ceph-mon[31985]: pgmap v1844: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:06:30.781 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:06:30 smithi022 ceph-mon[31985]: 2 slow requests, 0 included below; oldest blocked for > 2976.886712 secs 2024-03-28T19:06:32.674 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:06:32 smithi134 ceph-mon[35037]: pgmap v1845: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T19:06:32.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:06:32 smithi079 ceph-mon[35312]: pgmap v1845: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T19:06:32.781 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:06:32 smithi022 ceph-mon[31985]: pgmap v1845: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T19:06:34.676 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:06:34 smithi134 ceph-mon[35037]: pgmap v1846: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:06:34.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:06:34 smithi079 ceph-mon[35312]: pgmap v1846: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:06:34.781 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:06:34 smithi022 ceph-mon[31985]: pgmap v1846: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:06:35.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:06:35 smithi134 ceph-mon[35037]: 2 slow requests, 0 included below; oldest blocked for > 2981.886801 secs 2024-03-28T19:06:35.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:06:35 smithi079 ceph-mon[35312]: 2 slow requests, 0 included below; oldest blocked for > 2981.886801 secs 2024-03-28T19:06:35.781 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:06:35 smithi022 ceph-mon[31985]: 2 slow requests, 0 included below; oldest blocked for > 2981.886801 secs 2024-03-28T19:06:36.674 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:06:36 smithi134 ceph-mon[35037]: pgmap v1847: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T19:06:36.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:06:36 smithi079 ceph-mon[35312]: pgmap v1847: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T19:06:36.781 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:06:36 smithi022 ceph-mon[31985]: pgmap v1847: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T19:06:38.674 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:06:38 smithi134 ceph-mon[35037]: pgmap v1848: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:06:38.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:06:38 smithi079 ceph-mon[35312]: pgmap v1848: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:06:38.781 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:06:38 smithi022 ceph-mon[31985]: pgmap v1848: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:06:40.674 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:06:40 smithi134 ceph-mon[35037]: pgmap v1849: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:06:40.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:06:40 smithi134 ceph-mon[35037]: 2 slow requests, 0 included below; oldest blocked for > 2986.886918 secs 2024-03-28T19:06:40.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:06:40 smithi079 ceph-mon[35312]: pgmap v1849: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:06:40.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:06:40 smithi079 ceph-mon[35312]: 2 slow requests, 0 included below; oldest blocked for > 2986.886918 secs 2024-03-28T19:06:40.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:06:40 smithi022 ceph-mon[31985]: pgmap v1849: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:06:40.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:06:40 smithi022 ceph-mon[31985]: 2 slow requests, 0 included below; oldest blocked for > 2986.886918 secs 2024-03-28T19:06:42.674 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:06:42 smithi134 ceph-mon[35037]: pgmap v1850: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T19:06:42.681 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:06:42 smithi079 ceph-mon[35312]: pgmap v1850: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T19:06:42.781 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:06:42 smithi022 ceph-mon[31985]: pgmap v1850: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T19:06:44.674 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:06:44 smithi134 ceph-mon[35037]: pgmap v1851: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:06:44.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:06:44 smithi079 ceph-mon[35312]: pgmap v1851: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:06:44.781 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:06:44 smithi022 ceph-mon[31985]: pgmap v1851: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:06:45.674 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:06:45 smithi134 ceph-mon[35037]: 2 slow requests, 0 included below; oldest blocked for > 2991.886997 secs 2024-03-28T19:06:45.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:06:45 smithi079 ceph-mon[35312]: 2 slow requests, 0 included below; oldest blocked for > 2991.886997 secs 2024-03-28T19:06:45.683 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:06:45 smithi022 ceph-mon[31985]: 2 slow requests, 0 included below; oldest blocked for > 2991.886997 secs 2024-03-28T19:06:46.674 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:06:46 smithi134 ceph-mon[35037]: pgmap v1852: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T19:06:46.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:06:46 smithi079 ceph-mon[35312]: pgmap v1852: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T19:06:46.781 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:06:46 smithi022 ceph-mon[31985]: pgmap v1852: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T19:06:48.674 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:06:48 smithi134 ceph-mon[35037]: pgmap v1853: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:06:48.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:06:48 smithi079 ceph-mon[35312]: pgmap v1853: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:06:48.781 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:06:48 smithi022 ceph-mon[31985]: pgmap v1853: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:06:50.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:06:50 smithi079 ceph-mon[35312]: pgmap v1854: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:06:50.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:06:50 smithi079 ceph-mon[35312]: 2 slow requests, 0 included below; oldest blocked for > 2996.887089 secs 2024-03-28T19:06:50.781 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:06:50 smithi022 ceph-mon[31985]: pgmap v1854: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:06:50.781 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:06:50 smithi022 ceph-mon[31985]: 2 slow requests, 0 included below; oldest blocked for > 2996.887089 secs 2024-03-28T19:06:50.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:06:50 smithi134 ceph-mon[35037]: pgmap v1854: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:06:50.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:06:50 smithi134 ceph-mon[35037]: 2 slow requests, 0 included below; oldest blocked for > 2996.887089 secs 2024-03-28T19:06:52.781 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:06:52 smithi022 ceph-mon[31985]: pgmap v1855: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 9 op/s 2024-03-28T19:06:52.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:06:52 smithi134 ceph-mon[35037]: pgmap v1855: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 9 op/s 2024-03-28T19:06:52.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:06:52 smithi079 ceph-mon[35312]: pgmap v1855: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 9 op/s 2024-03-28T19:06:54.781 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:06:54 smithi022 ceph-mon[31985]: pgmap v1856: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T19:06:54.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:06:54 smithi134 ceph-mon[35037]: pgmap v1856: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T19:06:54.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:06:54 smithi079 ceph-mon[35312]: pgmap v1856: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T19:06:55.781 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:06:55 smithi022 ceph-mon[31985]: 2 slow requests, 0 included below; oldest blocked for > 3001.887164 secs 2024-03-28T19:06:55.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:06:55 smithi134 ceph-mon[35037]: 2 slow requests, 0 included below; oldest blocked for > 3001.887164 secs 2024-03-28T19:06:55.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:06:55 smithi079 ceph-mon[35312]: 2 slow requests, 0 included below; oldest blocked for > 3001.887164 secs 2024-03-28T19:06:56.781 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:06:56 smithi022 ceph-mon[31985]: pgmap v1857: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 9 op/s 2024-03-28T19:06:56.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:06:56 smithi134 ceph-mon[35037]: pgmap v1857: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 9 op/s 2024-03-28T19:06:56.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:06:56 smithi079 ceph-mon[35312]: pgmap v1857: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 9 op/s 2024-03-28T19:06:58.733 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:06:58 smithi079 ceph-mon[35312]: pgmap v1858: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T19:06:58.781 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:06:58 smithi022 ceph-mon[31985]: pgmap v1858: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T19:06:58.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:06:58 smithi134 ceph-mon[35037]: pgmap v1858: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T19:07:00.781 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:07:00 smithi022 ceph-mon[31985]: pgmap v1859: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T19:07:00.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:07:00 smithi022 ceph-mon[31985]: 2 slow requests, 0 included below; oldest blocked for > 3006.887219 secs 2024-03-28T19:07:00.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:07:00 smithi134 ceph-mon[35037]: pgmap v1859: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T19:07:00.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:07:00 smithi134 ceph-mon[35037]: 2 slow requests, 0 included below; oldest blocked for > 3006.887219 secs 2024-03-28T19:07:00.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:07:00 smithi079 ceph-mon[35312]: pgmap v1859: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T19:07:00.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:07:00 smithi079 ceph-mon[35312]: 2 slow requests, 0 included below; oldest blocked for > 3006.887219 secs 2024-03-28T19:07:02.781 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:07:02 smithi022 ceph-mon[31985]: pgmap v1860: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 9 op/s 2024-03-28T19:07:02.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:07:02 smithi134 ceph-mon[35037]: pgmap v1860: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 9 op/s 2024-03-28T19:07:02.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:07:02 smithi079 ceph-mon[35312]: pgmap v1860: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 9 op/s 2024-03-28T19:07:04.781 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:07:04 smithi022 ceph-mon[31985]: pgmap v1861: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T19:07:04.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:07:04 smithi134 ceph-mon[35037]: pgmap v1861: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T19:07:04.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:07:04 smithi079 ceph-mon[35312]: pgmap v1861: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T19:07:05.781 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:07:05 smithi022 ceph-mon[31985]: 2 slow requests, 0 included below; oldest blocked for > 3011.887294 secs 2024-03-28T19:07:05.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:07:05 smithi134 ceph-mon[35037]: 2 slow requests, 0 included below; oldest blocked for > 3011.887294 secs 2024-03-28T19:07:05.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:07:05 smithi079 ceph-mon[35312]: 2 slow requests, 0 included below; oldest blocked for > 3011.887294 secs 2024-03-28T19:07:06.781 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:07:06 smithi022 ceph-mon[31985]: pgmap v1862: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 9 op/s 2024-03-28T19:07:06.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:07:06 smithi134 ceph-mon[35037]: pgmap v1862: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 9 op/s 2024-03-28T19:07:06.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:07:06 smithi079 ceph-mon[35312]: pgmap v1862: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 9 op/s 2024-03-28T19:07:08.781 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:07:08 smithi022 ceph-mon[31985]: pgmap v1863: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:07:08.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:07:08 smithi134 ceph-mon[35037]: pgmap v1863: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:07:08.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:07:08 smithi079 ceph-mon[35312]: pgmap v1863: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:07:10.781 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:07:10 smithi022 ceph-mon[31985]: pgmap v1864: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:07:10.781 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:07:10 smithi022 ceph-mon[31985]: 2 slow requests, 0 included below; oldest blocked for > 3016.887379 secs 2024-03-28T19:07:10.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:07:10 smithi134 ceph-mon[35037]: pgmap v1864: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:07:10.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:07:10 smithi134 ceph-mon[35037]: 2 slow requests, 0 included below; oldest blocked for > 3016.887379 secs 2024-03-28T19:07:10.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:07:10 smithi079 ceph-mon[35312]: pgmap v1864: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:07:10.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:07:10 smithi079 ceph-mon[35312]: 2 slow requests, 0 included below; oldest blocked for > 3016.887379 secs 2024-03-28T19:07:11.781 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:07:11 smithi022 ceph-mon[31985]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T19:07:11.923 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:07:11 smithi134 ceph-mon[35037]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T19:07:11.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:07:11 smithi079 ceph-mon[35312]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T19:07:12.781 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:07:12 smithi022 ceph-mon[31985]: pgmap v1865: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T19:07:12.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:07:12 smithi134 ceph-mon[35037]: pgmap v1865: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T19:07:12.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:07:12 smithi079 ceph-mon[35312]: pgmap v1865: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T19:07:13.781 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:07:13 smithi022 ceph-mon[31985]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T19:07:13.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:07:13 smithi022 ceph-mon[31985]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T19:07:13.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:07:13 smithi022 ceph-mon[31985]: pgmap v1866: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 6 op/s 2024-03-28T19:07:13.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:07:13 smithi022 ceph-mon[31985]: pgmap v1867: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-03-28T19:07:13.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:07:13 smithi022 ceph-mon[31985]: from='mgr.15522 ' entity='mgr.x' 2024-03-28T19:07:13.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:07:13 smithi134 ceph-mon[35037]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T19:07:13.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:07:13 smithi134 ceph-mon[35037]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T19:07:13.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:07:13 smithi134 ceph-mon[35037]: pgmap v1866: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 6 op/s 2024-03-28T19:07:13.925 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:07:13 smithi134 ceph-mon[35037]: pgmap v1867: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-03-28T19:07:13.925 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:07:13 smithi134 ceph-mon[35037]: from='mgr.15522 ' entity='mgr.x' 2024-03-28T19:07:13.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:07:13 smithi079 ceph-mon[35312]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T19:07:13.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:07:13 smithi079 ceph-mon[35312]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T19:07:13.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:07:13 smithi079 ceph-mon[35312]: pgmap v1866: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 6 op/s 2024-03-28T19:07:13.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:07:13 smithi079 ceph-mon[35312]: pgmap v1867: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-03-28T19:07:13.933 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:07:13 smithi079 ceph-mon[35312]: from='mgr.15522 ' entity='mgr.x' 2024-03-28T19:07:15.031 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:07:14 smithi022 ceph-mon[31985]: pgmap v1868: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-03-28T19:07:15.174 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:07:14 smithi134 ceph-mon[35037]: pgmap v1868: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-03-28T19:07:15.182 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:07:14 smithi079 ceph-mon[35312]: pgmap v1868: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-03-28T19:07:16.031 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:07:15 smithi022 ceph-mon[31985]: 2 slow requests, 0 included below; oldest blocked for > 3021.887440 secs 2024-03-28T19:07:16.174 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:07:15 smithi134 ceph-mon[35037]: 2 slow requests, 0 included below; oldest blocked for > 3021.887440 secs 2024-03-28T19:07:16.182 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:07:15 smithi079 ceph-mon[35312]: 2 slow requests, 0 included below; oldest blocked for > 3021.887440 secs 2024-03-28T19:07:17.031 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:07:16 smithi022 ceph-mon[31985]: pgmap v1869: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-03-28T19:07:17.174 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:07:16 smithi134 ceph-mon[35037]: pgmap v1869: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-03-28T19:07:17.182 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:07:16 smithi079 ceph-mon[35312]: pgmap v1869: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-03-28T19:07:18.977 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:07:18 smithi134 ceph-mon[35037]: pgmap v1870: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-03-28T19:07:19.031 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:07:18 smithi022 ceph-mon[31985]: pgmap v1870: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-03-28T19:07:19.182 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:07:18 smithi079 ceph-mon[35312]: pgmap v1870: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-03-28T19:07:20.174 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:07:19 smithi134 ceph-mon[35037]: 2 slow requests, 0 included below; oldest blocked for > 3026.887563 secs 2024-03-28T19:07:20.182 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:07:19 smithi079 ceph-mon[35312]: 2 slow requests, 0 included below; oldest blocked for > 3026.887563 secs 2024-03-28T19:07:20.281 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:07:19 smithi022 ceph-mon[31985]: 2 slow requests, 0 included below; oldest blocked for > 3026.887563 secs 2024-03-28T19:07:21.174 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:07:20 smithi134 ceph-mon[35037]: pgmap v1871: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-03-28T19:07:21.182 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:07:20 smithi079 ceph-mon[35312]: pgmap v1871: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-03-28T19:07:21.281 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:07:20 smithi022 ceph-mon[31985]: pgmap v1871: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-03-28T19:07:23.031 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:07:22 smithi022 ceph-mon[31985]: pgmap v1872: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 7 op/s 2024-03-28T19:07:23.174 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:07:22 smithi134 ceph-mon[35037]: pgmap v1872: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 7 op/s 2024-03-28T19:07:23.182 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:07:22 smithi079 ceph-mon[35312]: pgmap v1872: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 7 op/s 2024-03-28T19:07:25.031 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:07:24 smithi022 ceph-mon[31985]: pgmap v1873: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T19:07:25.174 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:07:24 smithi134 ceph-mon[35037]: pgmap v1873: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T19:07:25.182 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:07:24 smithi079 ceph-mon[35312]: pgmap v1873: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T19:07:26.031 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:07:25 smithi022 ceph-mon[31985]: 2 slow requests, 0 included below; oldest blocked for > 3031.887624 secs 2024-03-28T19:07:26.174 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:07:25 smithi134 ceph-mon[35037]: 2 slow requests, 0 included below; oldest blocked for > 3031.887624 secs 2024-03-28T19:07:26.182 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:07:25 smithi079 ceph-mon[35312]: 2 slow requests, 0 included below; oldest blocked for > 3031.887624 secs 2024-03-28T19:07:27.031 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:07:26 smithi022 ceph-mon[31985]: pgmap v1874: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:07:27.174 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:07:26 smithi134 ceph-mon[35037]: pgmap v1874: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:07:27.181 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:07:26 smithi079 ceph-mon[35312]: pgmap v1874: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:07:28.984 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:07:28 smithi079 ceph-mon[35312]: pgmap v1875: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:07:29.031 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:07:28 smithi022 ceph-mon[31985]: pgmap v1875: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:07:29.174 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:07:28 smithi134 ceph-mon[35037]: pgmap v1875: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:07:30.174 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:07:29 smithi134 ceph-mon[35037]: 2 slow requests, 0 included below; oldest blocked for > 3036.887690 secs 2024-03-28T19:07:30.182 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:07:29 smithi079 ceph-mon[35312]: 2 slow requests, 0 included below; oldest blocked for > 3036.887690 secs 2024-03-28T19:07:30.281 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:07:29 smithi022 ceph-mon[31985]: 2 slow requests, 0 included below; oldest blocked for > 3036.887690 secs 2024-03-28T19:07:31.142 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:07:30 smithi134 ceph-mon[35037]: pgmap v1876: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T19:07:31.182 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:07:30 smithi079 ceph-mon[35312]: pgmap v1876: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T19:07:31.281 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:07:30 smithi022 ceph-mon[31985]: pgmap v1876: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T19:07:33.031 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:07:32 smithi022 ceph-mon[31985]: pgmap v1877: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:07:33.174 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:07:32 smithi134 ceph-mon[35037]: pgmap v1877: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:07:33.182 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:07:32 smithi079 ceph-mon[35312]: pgmap v1877: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:07:35.031 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:07:34 smithi022 ceph-mon[31985]: pgmap v1878: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T19:07:35.174 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:07:34 smithi134 ceph-mon[35037]: pgmap v1878: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T19:07:35.182 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:07:34 smithi079 ceph-mon[35312]: pgmap v1878: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T19:07:36.031 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:07:35 smithi022 ceph-mon[31985]: 2 slow requests, 0 included below; oldest blocked for > 3041.887751 secs 2024-03-28T19:07:36.174 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:07:35 smithi134 ceph-mon[35037]: 2 slow requests, 0 included below; oldest blocked for > 3041.887751 secs 2024-03-28T19:07:36.182 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:07:35 smithi079 ceph-mon[35312]: 2 slow requests, 0 included below; oldest blocked for > 3041.887751 secs 2024-03-28T19:07:37.031 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:07:36 smithi022 ceph-mon[31985]: pgmap v1879: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:07:37.174 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:07:36 smithi134 ceph-mon[35037]: pgmap v1879: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:07:37.182 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:07:36 smithi079 ceph-mon[35312]: pgmap v1879: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:07:39.031 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:07:38 smithi022 ceph-mon[31985]: pgmap v1880: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:07:39.174 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:07:38 smithi134 ceph-mon[35037]: pgmap v1880: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:07:39.182 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:07:38 smithi079 ceph-mon[35312]: pgmap v1880: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:07:40.174 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:07:39 smithi134 ceph-mon[35037]: 2 slow requests, 0 included below; oldest blocked for > 3046.887822 secs 2024-03-28T19:07:40.182 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:07:39 smithi079 ceph-mon[35312]: 2 slow requests, 0 included below; oldest blocked for > 3046.887822 secs 2024-03-28T19:07:40.281 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:07:39 smithi022 ceph-mon[31985]: 2 slow requests, 0 included below; oldest blocked for > 3046.887822 secs 2024-03-28T19:07:41.174 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:07:40 smithi134 ceph-mon[35037]: pgmap v1881: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T19:07:41.182 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:07:40 smithi079 ceph-mon[35312]: pgmap v1881: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T19:07:41.281 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:07:40 smithi022 ceph-mon[31985]: pgmap v1881: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T19:07:43.031 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:07:42 smithi022 ceph-mon[31985]: pgmap v1882: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:07:43.174 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:07:42 smithi134 ceph-mon[35037]: pgmap v1882: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:07:43.182 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:07:42 smithi079 ceph-mon[35312]: pgmap v1882: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:07:45.031 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:07:44 smithi022 ceph-mon[31985]: pgmap v1883: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T19:07:45.174 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:07:44 smithi134 ceph-mon[35037]: pgmap v1883: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T19:07:45.182 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:07:44 smithi079 ceph-mon[35312]: pgmap v1883: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T19:07:46.031 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:07:45 smithi022 ceph-mon[31985]: 2 slow requests, 0 included below; oldest blocked for > 3051.887883 secs 2024-03-28T19:07:46.174 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:07:45 smithi134 ceph-mon[35037]: 2 slow requests, 0 included below; oldest blocked for > 3051.887883 secs 2024-03-28T19:07:46.182 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:07:45 smithi079 ceph-mon[35312]: 2 slow requests, 0 included below; oldest blocked for > 3051.887883 secs 2024-03-28T19:07:47.031 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:07:46 smithi022 ceph-mon[31985]: pgmap v1884: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:07:47.174 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:07:46 smithi134 ceph-mon[35037]: pgmap v1884: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:07:47.182 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:07:46 smithi079 ceph-mon[35312]: pgmap v1884: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:07:48.976 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:07:48 smithi134 ceph-mon[35037]: pgmap v1885: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:07:49.031 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:07:48 smithi022 ceph-mon[31985]: pgmap v1885: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:07:49.181 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:07:48 smithi079 ceph-mon[35312]: pgmap v1885: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:07:50.174 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:07:49 smithi134 ceph-mon[35037]: 2 slow requests, 0 included below; oldest blocked for > 3056.887926 secs 2024-03-28T19:07:50.182 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:07:49 smithi079 ceph-mon[35312]: 2 slow requests, 0 included below; oldest blocked for > 3056.887926 secs 2024-03-28T19:07:50.281 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:07:49 smithi022 ceph-mon[31985]: 2 slow requests, 0 included below; oldest blocked for > 3056.887926 secs 2024-03-28T19:07:51.174 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:07:50 smithi134 ceph-mon[35037]: pgmap v1886: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T19:07:51.182 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:07:50 smithi079 ceph-mon[35312]: pgmap v1886: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T19:07:51.281 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:07:50 smithi022 ceph-mon[31985]: pgmap v1886: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T19:07:53.031 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:07:52 smithi022 ceph-mon[31985]: pgmap v1887: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:07:53.174 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:07:52 smithi134 ceph-mon[35037]: pgmap v1887: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:07:53.182 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:07:52 smithi079 ceph-mon[35312]: pgmap v1887: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:07:55.031 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:07:54 smithi022 ceph-mon[31985]: pgmap v1888: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 9 op/s 2024-03-28T19:07:55.174 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:07:54 smithi134 ceph-mon[35037]: pgmap v1888: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 9 op/s 2024-03-28T19:07:55.181 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:07:54 smithi079 ceph-mon[35312]: pgmap v1888: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 9 op/s 2024-03-28T19:07:56.031 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:07:55 smithi022 ceph-mon[31985]: 2 slow requests, 0 included below; oldest blocked for > 3061.888001 secs 2024-03-28T19:07:56.174 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:07:55 smithi134 ceph-mon[35037]: 2 slow requests, 0 included below; oldest blocked for > 3061.888001 secs 2024-03-28T19:07:56.181 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:07:55 smithi079 ceph-mon[35312]: 2 slow requests, 0 included below; oldest blocked for > 3061.888001 secs 2024-03-28T19:07:57.031 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:07:56 smithi022 ceph-mon[31985]: pgmap v1889: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T19:07:57.174 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:07:56 smithi134 ceph-mon[35037]: pgmap v1889: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T19:07:57.182 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:07:56 smithi079 ceph-mon[35312]: pgmap v1889: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T19:07:58.985 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:07:58 smithi079 ceph-mon[35312]: pgmap v1890: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T19:07:59.031 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:07:58 smithi022 ceph-mon[31985]: pgmap v1890: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T19:07:59.174 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:07:58 smithi134 ceph-mon[35037]: pgmap v1890: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T19:08:00.174 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:07:59 smithi134 ceph-mon[35037]: 2 slow requests, 0 included below; oldest blocked for > 3066.888074 secs 2024-03-28T19:08:00.182 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:07:59 smithi079 ceph-mon[35312]: 2 slow requests, 0 included below; oldest blocked for > 3066.888074 secs 2024-03-28T19:08:00.281 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:07:59 smithi022 ceph-mon[31985]: 2 slow requests, 0 included below; oldest blocked for > 3066.888074 secs 2024-03-28T19:08:01.174 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:08:00 smithi134 ceph-mon[35037]: pgmap v1891: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 9 op/s 2024-03-28T19:08:01.182 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:08:00 smithi079 ceph-mon[35312]: pgmap v1891: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 9 op/s 2024-03-28T19:08:01.281 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:08:00 smithi022 ceph-mon[31985]: pgmap v1891: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 9 op/s 2024-03-28T19:08:03.031 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:08:02 smithi022 ceph-mon[31985]: pgmap v1892: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T19:08:03.174 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:08:02 smithi134 ceph-mon[35037]: pgmap v1892: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T19:08:03.182 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:08:02 smithi079 ceph-mon[35312]: pgmap v1892: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T19:08:05.031 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:08:04 smithi022 ceph-mon[31985]: pgmap v1893: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 9 op/s 2024-03-28T19:08:05.174 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:08:04 smithi134 ceph-mon[35037]: pgmap v1893: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 9 op/s 2024-03-28T19:08:05.181 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:08:04 smithi079 ceph-mon[35312]: pgmap v1893: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 9 op/s 2024-03-28T19:08:06.031 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:08:05 smithi022 ceph-mon[31985]: 2 slow requests, 0 included below; oldest blocked for > 3071.888150 secs 2024-03-28T19:08:06.174 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:08:05 smithi134 ceph-mon[35037]: 2 slow requests, 0 included below; oldest blocked for > 3071.888150 secs 2024-03-28T19:08:06.182 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:08:05 smithi079 ceph-mon[35312]: 2 slow requests, 0 included below; oldest blocked for > 3071.888150 secs 2024-03-28T19:08:07.031 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:08:06 smithi022 ceph-mon[31985]: pgmap v1894: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:08:07.174 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:08:06 smithi134 ceph-mon[35037]: pgmap v1894: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:08:07.181 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:08:06 smithi079 ceph-mon[35312]: pgmap v1894: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:08:09.031 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:08:08 smithi022 ceph-mon[31985]: pgmap v1895: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:08:09.174 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:08:08 smithi134 ceph-mon[35037]: pgmap v1895: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:08:09.182 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:08:08 smithi079 ceph-mon[35312]: pgmap v1895: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:08:10.174 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:08:09 smithi134 ceph-mon[35037]: 2 slow requests, 0 included below; oldest blocked for > 3076.888229 secs 2024-03-28T19:08:10.182 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:08:09 smithi079 ceph-mon[35312]: 2 slow requests, 0 included below; oldest blocked for > 3076.888229 secs 2024-03-28T19:08:10.281 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:08:09 smithi022 ceph-mon[31985]: 2 slow requests, 0 included below; oldest blocked for > 3076.888229 secs 2024-03-28T19:08:11.174 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:08:10 smithi134 ceph-mon[35037]: pgmap v1896: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T19:08:11.181 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:08:10 smithi079 ceph-mon[35312]: pgmap v1896: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T19:08:11.281 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:08:10 smithi022 ceph-mon[31985]: pgmap v1896: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T19:08:13.031 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:08:12 smithi022 ceph-mon[31985]: pgmap v1897: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:08:13.174 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:08:12 smithi134 ceph-mon[35037]: pgmap v1897: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:08:13.181 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:08:12 smithi079 ceph-mon[35312]: pgmap v1897: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:08:14.031 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:08:13 smithi022 ceph-mon[31985]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T19:08:14.174 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:08:13 smithi134 ceph-mon[35037]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T19:08:14.181 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:08:13 smithi079 ceph-mon[35312]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T19:08:15.174 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:08:14 smithi134 ceph-mon[35037]: from='mgr.15522 ' entity='mgr.x' 2024-03-28T19:08:15.174 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:08:14 smithi134 ceph-mon[35037]: from='mgr.15522 ' entity='mgr.x' 2024-03-28T19:08:15.174 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:08:14 smithi134 ceph-mon[35037]: from='mgr.15522 ' entity='mgr.x' 2024-03-28T19:08:15.174 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:08:14 smithi134 ceph-mon[35037]: from='mgr.15522 ' entity='mgr.x' 2024-03-28T19:08:15.174 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:08:14 smithi134 ceph-mon[35037]: 2 slow requests, 0 included below; oldest blocked for > 3081.888294 secs 2024-03-28T19:08:15.174 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:08:14 smithi134 ceph-mon[35037]: pgmap v1898: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T19:08:15.182 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:08:14 smithi079 ceph-mon[35312]: from='mgr.15522 ' entity='mgr.x' 2024-03-28T19:08:15.182 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:08:14 smithi079 ceph-mon[35312]: from='mgr.15522 ' entity='mgr.x' 2024-03-28T19:08:15.182 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:08:14 smithi079 ceph-mon[35312]: from='mgr.15522 ' entity='mgr.x' 2024-03-28T19:08:15.182 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:08:14 smithi079 ceph-mon[35312]: from='mgr.15522 ' entity='mgr.x' 2024-03-28T19:08:15.182 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:08:14 smithi079 ceph-mon[35312]: 2 slow requests, 0 included below; oldest blocked for > 3081.888294 secs 2024-03-28T19:08:15.182 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:08:14 smithi079 ceph-mon[35312]: pgmap v1898: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T19:08:15.281 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:08:14 smithi022 ceph-mon[31985]: from='mgr.15522 ' entity='mgr.x' 2024-03-28T19:08:15.281 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:08:14 smithi022 ceph-mon[31985]: from='mgr.15522 ' entity='mgr.x' 2024-03-28T19:08:15.281 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:08:14 smithi022 ceph-mon[31985]: from='mgr.15522 ' entity='mgr.x' 2024-03-28T19:08:15.281 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:08:14 smithi022 ceph-mon[31985]: from='mgr.15522 ' entity='mgr.x' 2024-03-28T19:08:15.282 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:08:14 smithi022 ceph-mon[31985]: 2 slow requests, 0 included below; oldest blocked for > 3081.888294 secs 2024-03-28T19:08:15.282 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:08:14 smithi022 ceph-mon[31985]: pgmap v1898: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T19:08:16.424 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:08:16 smithi134 ceph-mon[35037]: from='mgr.15522 ' entity='mgr.x' 2024-03-28T19:08:16.424 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:08:16 smithi134 ceph-mon[35037]: from='mgr.15522 ' entity='mgr.x' 2024-03-28T19:08:16.425 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:08:16 smithi134 ceph-mon[35037]: from='mgr.15522 ' entity='mgr.x' 2024-03-28T19:08:16.425 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:08:16 smithi134 ceph-mon[35037]: from='mgr.15522 ' entity='mgr.x' 2024-03-28T19:08:16.432 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:08:16 smithi079 ceph-mon[35312]: from='mgr.15522 ' entity='mgr.x' 2024-03-28T19:08:16.432 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:08:16 smithi079 ceph-mon[35312]: from='mgr.15522 ' entity='mgr.x' 2024-03-28T19:08:16.432 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:08:16 smithi079 ceph-mon[35312]: from='mgr.15522 ' entity='mgr.x' 2024-03-28T19:08:16.432 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:08:16 smithi079 ceph-mon[35312]: from='mgr.15522 ' entity='mgr.x' 2024-03-28T19:08:16.531 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:08:16 smithi022 ceph-mon[31985]: from='mgr.15522 ' entity='mgr.x' 2024-03-28T19:08:16.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:08:16 smithi022 ceph-mon[31985]: from='mgr.15522 ' entity='mgr.x' 2024-03-28T19:08:16.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:08:16 smithi022 ceph-mon[31985]: from='mgr.15522 ' entity='mgr.x' 2024-03-28T19:08:16.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:08:16 smithi022 ceph-mon[31985]: from='mgr.15522 ' entity='mgr.x' 2024-03-28T19:08:17.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:08:17 smithi022 ceph-mon[31985]: from='mgr.15522 ' entity='mgr.x' 2024-03-28T19:08:17.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:08:17 smithi022 ceph-mon[31985]: from='mgr.15522 ' entity='mgr.x' 2024-03-28T19:08:17.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:08:17 smithi022 ceph-mon[31985]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T19:08:17.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:08:17 smithi022 ceph-mon[31985]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T19:08:17.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:08:17 smithi022 ceph-mon[31985]: pgmap v1899: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 6 op/s 2024-03-28T19:08:17.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:08:17 smithi022 ceph-mon[31985]: pgmap v1900: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-28T19:08:17.533 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:08:17 smithi022 ceph-mon[31985]: from='mgr.15522 ' entity='mgr.x' 2024-03-28T19:08:17.674 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:08:17 smithi134 ceph-mon[35037]: from='mgr.15522 ' entity='mgr.x' 2024-03-28T19:08:17.674 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:08:17 smithi134 ceph-mon[35037]: from='mgr.15522 ' entity='mgr.x' 2024-03-28T19:08:17.674 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:08:17 smithi134 ceph-mon[35037]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T19:08:17.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:08:17 smithi134 ceph-mon[35037]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T19:08:17.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:08:17 smithi134 ceph-mon[35037]: pgmap v1899: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 6 op/s 2024-03-28T19:08:17.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:08:17 smithi134 ceph-mon[35037]: pgmap v1900: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-28T19:08:17.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:08:17 smithi134 ceph-mon[35037]: from='mgr.15522 ' entity='mgr.x' 2024-03-28T19:08:17.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:08:17 smithi079 ceph-mon[35312]: from='mgr.15522 ' entity='mgr.x' 2024-03-28T19:08:17.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:08:17 smithi079 ceph-mon[35312]: from='mgr.15522 ' entity='mgr.x' 2024-03-28T19:08:17.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:08:17 smithi079 ceph-mon[35312]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T19:08:17.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:08:17 smithi079 ceph-mon[35312]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T19:08:17.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:08:17 smithi079 ceph-mon[35312]: pgmap v1899: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 6 op/s 2024-03-28T19:08:17.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:08:17 smithi079 ceph-mon[35312]: pgmap v1900: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-28T19:08:17.683 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:08:17 smithi079 ceph-mon[35312]: from='mgr.15522 ' entity='mgr.x' 2024-03-28T19:08:19.531 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:08:19 smithi022 ceph-mon[31985]: pgmap v1901: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-28T19:08:19.674 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:08:19 smithi134 ceph-mon[35037]: pgmap v1901: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-28T19:08:19.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:08:19 smithi079 ceph-mon[35312]: pgmap v1901: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-28T19:08:20.674 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:08:20 smithi134 ceph-mon[35037]: 2 slow requests, 0 included below; oldest blocked for > 3086.888360 secs 2024-03-28T19:08:20.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:08:20 smithi079 ceph-mon[35312]: 2 slow requests, 0 included below; oldest blocked for > 3086.888360 secs 2024-03-28T19:08:20.781 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:08:20 smithi022 ceph-mon[31985]: 2 slow requests, 0 included below; oldest blocked for > 3086.888360 secs 2024-03-28T19:08:21.674 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:08:21 smithi134 ceph-mon[35037]: pgmap v1902: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-28T19:08:21.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:08:21 smithi079 ceph-mon[35312]: pgmap v1902: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-28T19:08:21.781 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:08:21 smithi022 ceph-mon[31985]: pgmap v1902: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-28T19:08:23.674 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:08:23 smithi134 ceph-mon[35037]: pgmap v1903: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-28T19:08:23.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:08:23 smithi079 ceph-mon[35312]: pgmap v1903: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-28T19:08:23.781 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:08:23 smithi022 ceph-mon[31985]: pgmap v1903: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-28T19:08:25.674 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:08:25 smithi134 ceph-mon[35037]: pgmap v1904: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-28T19:08:25.674 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:08:25 smithi134 ceph-mon[35037]: 2 slow requests, 0 included below; oldest blocked for > 3091.888424 secs 2024-03-28T19:08:25.681 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:08:25 smithi079 ceph-mon[35312]: pgmap v1904: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-28T19:08:25.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:08:25 smithi079 ceph-mon[35312]: 2 slow requests, 0 included below; oldest blocked for > 3091.888424 secs 2024-03-28T19:08:25.781 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:08:25 smithi022 ceph-mon[31985]: pgmap v1904: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-28T19:08:25.781 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:08:25 smithi022 ceph-mon[31985]: 2 slow requests, 0 included below; oldest blocked for > 3091.888424 secs 2024-03-28T19:08:27.674 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:08:27 smithi134 ceph-mon[35037]: pgmap v1905: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 7 op/s 2024-03-28T19:08:27.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:08:27 smithi079 ceph-mon[35312]: pgmap v1905: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 7 op/s 2024-03-28T19:08:27.781 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:08:27 smithi022 ceph-mon[31985]: pgmap v1905: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 7 op/s 2024-03-28T19:08:29.674 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:08:29 smithi134 ceph-mon[35037]: pgmap v1906: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:08:29.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:08:29 smithi079 ceph-mon[35312]: pgmap v1906: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:08:29.781 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:08:29 smithi022 ceph-mon[31985]: pgmap v1906: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:08:30.674 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:08:30 smithi134 ceph-mon[35037]: 2 slow requests, 0 included below; oldest blocked for > 3096.888501 secs 2024-03-28T19:08:30.681 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:08:30 smithi079 ceph-mon[35312]: 2 slow requests, 0 included below; oldest blocked for > 3096.888501 secs 2024-03-28T19:08:30.781 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:08:30 smithi022 ceph-mon[31985]: 2 slow requests, 0 included below; oldest blocked for > 3096.888501 secs 2024-03-28T19:08:31.674 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:08:31 smithi134 ceph-mon[35037]: pgmap v1907: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T19:08:31.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:08:31 smithi079 ceph-mon[35312]: pgmap v1907: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T19:08:31.781 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:08:31 smithi022 ceph-mon[31985]: pgmap v1907: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T19:08:33.674 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:08:33 smithi134 ceph-mon[35037]: pgmap v1908: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:08:33.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:08:33 smithi079 ceph-mon[35312]: pgmap v1908: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:08:33.781 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:08:33 smithi022 ceph-mon[31985]: pgmap v1908: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:08:35.674 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:08:35 smithi134 ceph-mon[35037]: pgmap v1909: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T19:08:35.674 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:08:35 smithi134 ceph-mon[35037]: 2 slow requests, 0 included below; oldest blocked for > 3101.888578 secs 2024-03-28T19:08:35.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:08:35 smithi079 ceph-mon[35312]: pgmap v1909: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T19:08:35.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:08:35 smithi079 ceph-mon[35312]: 2 slow requests, 0 included below; oldest blocked for > 3101.888578 secs 2024-03-28T19:08:35.781 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:08:35 smithi022 ceph-mon[31985]: pgmap v1909: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T19:08:35.781 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:08:35 smithi022 ceph-mon[31985]: 2 slow requests, 0 included below; oldest blocked for > 3101.888578 secs 2024-03-28T19:08:37.674 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:08:37 smithi134 ceph-mon[35037]: pgmap v1910: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:08:37.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:08:37 smithi079 ceph-mon[35312]: pgmap v1910: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:08:37.781 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:08:37 smithi022 ceph-mon[31985]: pgmap v1910: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:08:39.674 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:08:39 smithi134 ceph-mon[35037]: pgmap v1911: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:08:39.681 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:08:39 smithi079 ceph-mon[35312]: pgmap v1911: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:08:39.781 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:08:39 smithi022 ceph-mon[31985]: pgmap v1911: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:08:40.674 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:08:40 smithi134 ceph-mon[35037]: 2 slow requests, 0 included below; oldest blocked for > 3106.888646 secs 2024-03-28T19:08:40.681 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:08:40 smithi079 ceph-mon[35312]: 2 slow requests, 0 included below; oldest blocked for > 3106.888646 secs 2024-03-28T19:08:40.781 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:08:40 smithi022 ceph-mon[31985]: 2 slow requests, 0 included below; oldest blocked for > 3106.888646 secs 2024-03-28T19:08:41.674 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:08:41 smithi134 ceph-mon[35037]: pgmap v1912: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T19:08:41.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:08:41 smithi079 ceph-mon[35312]: pgmap v1912: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T19:08:41.781 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:08:41 smithi022 ceph-mon[31985]: pgmap v1912: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T19:08:43.674 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:08:43 smithi134 ceph-mon[35037]: pgmap v1913: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:08:43.681 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:08:43 smithi079 ceph-mon[35312]: pgmap v1913: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:08:43.781 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:08:43 smithi022 ceph-mon[31985]: pgmap v1913: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:08:45.674 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:08:45 smithi134 ceph-mon[35037]: pgmap v1914: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T19:08:45.674 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:08:45 smithi134 ceph-mon[35037]: 2 slow requests, 0 included below; oldest blocked for > 3111.888711 secs 2024-03-28T19:08:45.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:08:45 smithi079 ceph-mon[35312]: pgmap v1914: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T19:08:45.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:08:45 smithi079 ceph-mon[35312]: 2 slow requests, 0 included below; oldest blocked for > 3111.888711 secs 2024-03-28T19:08:45.781 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:08:45 smithi022 ceph-mon[31985]: pgmap v1914: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T19:08:45.781 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:08:45 smithi022 ceph-mon[31985]: 2 slow requests, 0 included below; oldest blocked for > 3111.888711 secs 2024-03-28T19:08:47.674 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:08:47 smithi134 ceph-mon[35037]: pgmap v1915: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:08:47.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:08:47 smithi079 ceph-mon[35312]: pgmap v1915: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:08:47.781 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:08:47 smithi022 ceph-mon[31985]: pgmap v1915: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:08:49.674 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:08:49 smithi134 ceph-mon[35037]: pgmap v1916: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:08:49.681 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:08:49 smithi079 ceph-mon[35312]: pgmap v1916: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:08:49.781 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:08:49 smithi022 ceph-mon[31985]: pgmap v1916: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:08:50.674 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:08:50 smithi134 ceph-mon[35037]: 2 slow requests, 0 included below; oldest blocked for > 3116.888777 secs 2024-03-28T19:08:50.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:08:50 smithi079 ceph-mon[35312]: 2 slow requests, 0 included below; oldest blocked for > 3116.888777 secs 2024-03-28T19:08:50.781 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:08:50 smithi022 ceph-mon[31985]: 2 slow requests, 0 included below; oldest blocked for > 3116.888777 secs 2024-03-28T19:08:51.674 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:08:51 smithi134 ceph-mon[35037]: pgmap v1917: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T19:08:51.681 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:08:51 smithi079 ceph-mon[35312]: pgmap v1917: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T19:08:51.781 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:08:51 smithi022 ceph-mon[31985]: pgmap v1917: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T19:08:53.674 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:08:53 smithi134 ceph-mon[35037]: pgmap v1918: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:08:53.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:08:53 smithi079 ceph-mon[35312]: pgmap v1918: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:08:53.781 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:08:53 smithi022 ceph-mon[31985]: pgmap v1918: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:08:55.674 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:08:55 smithi134 ceph-mon[35037]: pgmap v1919: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T19:08:55.674 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:08:55 smithi134 ceph-mon[35037]: 2 slow requests, 0 included below; oldest blocked for > 3121.888914 secs 2024-03-28T19:08:55.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:08:55 smithi079 ceph-mon[35312]: pgmap v1919: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T19:08:55.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:08:55 smithi079 ceph-mon[35312]: 2 slow requests, 0 included below; oldest blocked for > 3121.888914 secs 2024-03-28T19:08:55.781 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:08:55 smithi022 ceph-mon[31985]: pgmap v1919: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T19:08:55.781 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:08:55 smithi022 ceph-mon[31985]: 2 slow requests, 0 included below; oldest blocked for > 3121.888914 secs 2024-03-28T19:08:57.674 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:08:57 smithi134 ceph-mon[35037]: pgmap v1920: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:08:57.681 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:08:57 smithi079 ceph-mon[35312]: pgmap v1920: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:08:57.781 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:08:57 smithi022 ceph-mon[31985]: pgmap v1920: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:08:59.674 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:08:59 smithi134 ceph-mon[35037]: pgmap v1921: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:08:59.681 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:08:59 smithi079 ceph-mon[35312]: pgmap v1921: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:08:59.781 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:08:59 smithi022 ceph-mon[31985]: pgmap v1921: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:09:00.781 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:09:00 smithi022 ceph-mon[31985]: 2 slow requests, 0 included below; oldest blocked for > 3126.888977 secs 2024-03-28T19:09:00.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:09:00 smithi134 ceph-mon[35037]: 2 slow requests, 0 included below; oldest blocked for > 3126.888977 secs 2024-03-28T19:09:00.931 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:09:00 smithi079 ceph-mon[35312]: 2 slow requests, 0 included below; oldest blocked for > 3126.888977 secs 2024-03-28T19:09:01.781 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:09:01 smithi022 ceph-mon[31985]: pgmap v1922: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T19:09:01.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:09:01 smithi134 ceph-mon[35037]: pgmap v1922: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T19:09:01.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:09:01 smithi079 ceph-mon[35312]: pgmap v1922: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T19:09:03.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:09:03 smithi022 ceph-mon[31985]: pgmap v1923: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:09:03.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:09:03 smithi134 ceph-mon[35037]: pgmap v1923: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:09:03.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:09:03 smithi079 ceph-mon[35312]: pgmap v1923: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:09:05.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:09:05 smithi022 ceph-mon[31985]: pgmap v1924: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T19:09:05.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:09:05 smithi022 ceph-mon[31985]: 2 slow requests, 0 included below; oldest blocked for > 3131.889071 secs 2024-03-28T19:09:05.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:09:05 smithi134 ceph-mon[35037]: pgmap v1924: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T19:09:05.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:09:05 smithi134 ceph-mon[35037]: 2 slow requests, 0 included below; oldest blocked for > 3131.889071 secs 2024-03-28T19:09:05.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:09:05 smithi079 ceph-mon[35312]: pgmap v1924: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T19:09:05.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:09:05 smithi079 ceph-mon[35312]: 2 slow requests, 0 included below; oldest blocked for > 3131.889071 secs 2024-03-28T19:09:07.781 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:09:07 smithi022 ceph-mon[31985]: pgmap v1925: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:09:07.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:09:07 smithi134 ceph-mon[35037]: pgmap v1925: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:09:07.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:09:07 smithi079 ceph-mon[35312]: pgmap v1925: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:09:09.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:09:09 smithi022 ceph-mon[31985]: pgmap v1926: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:09:09.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:09:09 smithi134 ceph-mon[35037]: pgmap v1926: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:09:09.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:09:09 smithi079 ceph-mon[35312]: pgmap v1926: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:09:10.781 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:09:10 smithi022 ceph-mon[31985]: 2 slow requests, 0 included below; oldest blocked for > 3136.889145 secs 2024-03-28T19:09:10.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:09:10 smithi134 ceph-mon[35037]: 2 slow requests, 0 included below; oldest blocked for > 3136.889145 secs 2024-03-28T19:09:10.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:09:10 smithi079 ceph-mon[35312]: 2 slow requests, 0 included below; oldest blocked for > 3136.889145 secs 2024-03-28T19:09:11.781 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:09:11 smithi022 ceph-mon[31985]: pgmap v1927: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T19:09:11.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:09:11 smithi134 ceph-mon[35037]: pgmap v1927: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T19:09:11.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:09:11 smithi079 ceph-mon[35312]: pgmap v1927: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T19:09:13.781 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:09:13 smithi022 ceph-mon[31985]: pgmap v1928: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:09:13.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:09:13 smithi134 ceph-mon[35037]: pgmap v1928: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:09:13.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:09:13 smithi079 ceph-mon[35312]: pgmap v1928: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:09:15.781 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:09:15 smithi022 ceph-mon[31985]: pgmap v1929: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T19:09:15.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:09:15 smithi022 ceph-mon[31985]: 2 slow requests, 0 included below; oldest blocked for > 3141.889265 secs 2024-03-28T19:09:15.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:09:15 smithi134 ceph-mon[35037]: pgmap v1929: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T19:09:15.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:09:15 smithi134 ceph-mon[35037]: 2 slow requests, 0 included below; oldest blocked for > 3141.889265 secs 2024-03-28T19:09:15.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:09:15 smithi079 ceph-mon[35312]: pgmap v1929: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T19:09:15.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:09:15 smithi079 ceph-mon[35312]: 2 slow requests, 0 included below; oldest blocked for > 3141.889265 secs 2024-03-28T19:09:17.781 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:09:17 smithi022 ceph-mon[31985]: pgmap v1930: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:09:17.781 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:09:17 smithi022 ceph-mon[31985]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T19:09:17.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:09:17 smithi134 ceph-mon[35037]: pgmap v1930: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:09:17.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:09:17 smithi134 ceph-mon[35037]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T19:09:17.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:09:17 smithi079 ceph-mon[35312]: pgmap v1930: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:09:17.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:09:17 smithi079 ceph-mon[35312]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T19:09:18.781 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:09:18 smithi022 ceph-mon[31985]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T19:09:18.781 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:09:18 smithi022 ceph-mon[31985]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T19:09:18.781 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:09:18 smithi022 ceph-mon[31985]: from='mgr.15522 ' entity='mgr.x' 2024-03-28T19:09:18.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:09:18 smithi134 ceph-mon[35037]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T19:09:18.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:09:18 smithi134 ceph-mon[35037]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T19:09:18.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:09:18 smithi134 ceph-mon[35037]: from='mgr.15522 ' entity='mgr.x' 2024-03-28T19:09:18.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:09:18 smithi079 ceph-mon[35312]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T19:09:18.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:09:18 smithi079 ceph-mon[35312]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T19:09:18.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:09:18 smithi079 ceph-mon[35312]: from='mgr.15522 ' entity='mgr.x' 2024-03-28T19:09:19.781 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:09:19 smithi022 ceph-mon[31985]: pgmap v1931: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 6 op/s 2024-03-28T19:09:19.781 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:09:19 smithi022 ceph-mon[31985]: pgmap v1932: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-03-28T19:09:19.826 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:09:19 smithi134 ceph-mon[35037]: pgmap v1931: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 6 op/s 2024-03-28T19:09:19.826 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:09:19 smithi134 ceph-mon[35037]: pgmap v1932: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-03-28T19:09:19.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:09:19 smithi079 ceph-mon[35312]: pgmap v1931: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 6 op/s 2024-03-28T19:09:19.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:09:19 smithi079 ceph-mon[35312]: pgmap v1932: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-03-28T19:09:20.781 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:09:20 smithi022 ceph-mon[31985]: 2 slow requests, 0 included below; oldest blocked for > 3146.889342 secs 2024-03-28T19:09:20.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:09:20 smithi134 ceph-mon[35037]: 2 slow requests, 0 included below; oldest blocked for > 3146.889342 secs 2024-03-28T19:09:20.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:09:20 smithi079 ceph-mon[35312]: 2 slow requests, 0 included below; oldest blocked for > 3146.889342 secs 2024-03-28T19:09:21.781 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:09:21 smithi022 ceph-mon[31985]: pgmap v1933: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 8 op/s 2024-03-28T19:09:21.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:09:21 smithi134 ceph-mon[35037]: pgmap v1933: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 8 op/s 2024-03-28T19:09:21.931 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:09:21 smithi079 ceph-mon[35312]: pgmap v1933: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 8 op/s 2024-03-28T19:09:23.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:09:23 smithi134 ceph-mon[35037]: pgmap v1934: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 8 op/s 2024-03-28T19:09:23.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:09:23 smithi079 ceph-mon[35312]: pgmap v1934: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 8 op/s 2024-03-28T19:09:24.031 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:09:23 smithi022 ceph-mon[31985]: pgmap v1934: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 8 op/s 2024-03-28T19:09:25.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:09:25 smithi134 ceph-mon[35037]: pgmap v1935: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 KiB/s rd, 4 op/s 2024-03-28T19:09:25.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:09:25 smithi134 ceph-mon[35037]: 2 slow requests, 0 included below; oldest blocked for > 3151.889419 secs 2024-03-28T19:09:25.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:09:25 smithi079 ceph-mon[35312]: pgmap v1935: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 KiB/s rd, 4 op/s 2024-03-28T19:09:25.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:09:25 smithi079 ceph-mon[35312]: 2 slow requests, 0 included below; oldest blocked for > 3151.889419 secs 2024-03-28T19:09:26.031 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:09:25 smithi022 ceph-mon[31985]: pgmap v1935: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 KiB/s rd, 4 op/s 2024-03-28T19:09:26.031 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:09:25 smithi022 ceph-mon[31985]: 2 slow requests, 0 included below; oldest blocked for > 3151.889419 secs 2024-03-28T19:09:27.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:09:27 smithi134 ceph-mon[35037]: pgmap v1936: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 8 op/s 2024-03-28T19:09:27.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:09:27 smithi079 ceph-mon[35312]: pgmap v1936: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 8 op/s 2024-03-28T19:09:28.031 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:09:27 smithi022 ceph-mon[31985]: pgmap v1936: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 8 op/s 2024-03-28T19:09:29.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:09:29 smithi134 ceph-mon[35037]: pgmap v1937: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 7 op/s 2024-03-28T19:09:29.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:09:29 smithi079 ceph-mon[35312]: pgmap v1937: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 7 op/s 2024-03-28T19:09:30.031 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:09:29 smithi022 ceph-mon[31985]: pgmap v1937: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 7 op/s 2024-03-28T19:09:30.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:09:30 smithi134 ceph-mon[35037]: 2 slow requests, 0 included below; oldest blocked for > 3156.889507 secs 2024-03-28T19:09:30.931 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:09:30 smithi079 ceph-mon[35312]: 2 slow requests, 0 included below; oldest blocked for > 3156.889507 secs 2024-03-28T19:09:31.031 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:09:30 smithi022 ceph-mon[31985]: 2 slow requests, 0 included below; oldest blocked for > 3156.889507 secs 2024-03-28T19:09:31.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:09:31 smithi134 ceph-mon[35037]: pgmap v1938: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 9 op/s 2024-03-28T19:09:31.931 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:09:31 smithi079 ceph-mon[35312]: pgmap v1938: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 9 op/s 2024-03-28T19:09:32.031 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:09:31 smithi022 ceph-mon[31985]: pgmap v1938: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 9 op/s 2024-03-28T19:09:33.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:09:33 smithi134 ceph-mon[35037]: pgmap v1939: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:09:33.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:09:33 smithi079 ceph-mon[35312]: pgmap v1939: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:09:34.031 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:09:33 smithi022 ceph-mon[31985]: pgmap v1939: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:09:35.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:09:35 smithi134 ceph-mon[35037]: pgmap v1940: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:09:35.925 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:09:35 smithi134 ceph-mon[35037]: 2 slow requests, 0 included below; oldest blocked for > 3161.889535 secs 2024-03-28T19:09:35.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:09:35 smithi079 ceph-mon[35312]: pgmap v1940: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:09:35.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:09:35 smithi079 ceph-mon[35312]: 2 slow requests, 0 included below; oldest blocked for > 3161.889535 secs 2024-03-28T19:09:36.031 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:09:35 smithi022 ceph-mon[31985]: pgmap v1940: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:09:36.031 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:09:35 smithi022 ceph-mon[31985]: 2 slow requests, 0 included below; oldest blocked for > 3161.889535 secs 2024-03-28T19:09:37.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:09:37 smithi134 ceph-mon[35037]: pgmap v1941: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T19:09:37.931 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:09:37 smithi079 ceph-mon[35312]: pgmap v1941: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T19:09:38.031 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:09:37 smithi022 ceph-mon[31985]: pgmap v1941: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T19:09:39.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:09:39 smithi134 ceph-mon[35037]: pgmap v1942: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:09:39.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:09:39 smithi079 ceph-mon[35312]: pgmap v1942: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:09:40.031 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:09:39 smithi022 ceph-mon[31985]: pgmap v1942: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:09:40.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:09:40 smithi134 ceph-mon[35037]: 2 slow requests, 0 included below; oldest blocked for > 3166.889614 secs 2024-03-28T19:09:40.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:09:40 smithi079 ceph-mon[35312]: 2 slow requests, 0 included below; oldest blocked for > 3166.889614 secs 2024-03-28T19:09:41.031 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:09:40 smithi022 ceph-mon[31985]: 2 slow requests, 0 included below; oldest blocked for > 3166.889614 secs 2024-03-28T19:09:41.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:09:41 smithi134 ceph-mon[35037]: pgmap v1943: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T19:09:41.931 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:09:41 smithi079 ceph-mon[35312]: pgmap v1943: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T19:09:42.031 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:09:41 smithi022 ceph-mon[31985]: pgmap v1943: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T19:09:43.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:09:43 smithi134 ceph-mon[35037]: pgmap v1944: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:09:43.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:09:43 smithi079 ceph-mon[35312]: pgmap v1944: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:09:44.031 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:09:43 smithi022 ceph-mon[31985]: pgmap v1944: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:09:45.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:09:45 smithi134 ceph-mon[35037]: pgmap v1945: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:09:45.925 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:09:45 smithi134 ceph-mon[35037]: 2 slow requests, 0 included below; oldest blocked for > 3171.889685 secs 2024-03-28T19:09:45.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:09:45 smithi079 ceph-mon[35312]: pgmap v1945: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:09:45.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:09:45 smithi079 ceph-mon[35312]: 2 slow requests, 0 included below; oldest blocked for > 3171.889685 secs 2024-03-28T19:09:46.031 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:09:45 smithi022 ceph-mon[31985]: pgmap v1945: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:09:46.031 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:09:45 smithi022 ceph-mon[31985]: 2 slow requests, 0 included below; oldest blocked for > 3171.889685 secs 2024-03-28T19:09:47.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:09:47 smithi134 ceph-mon[35037]: pgmap v1946: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T19:09:47.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:09:47 smithi079 ceph-mon[35312]: pgmap v1946: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T19:09:48.031 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:09:47 smithi022 ceph-mon[31985]: pgmap v1946: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T19:09:49.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:09:49 smithi134 ceph-mon[35037]: pgmap v1947: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:09:49.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:09:49 smithi079 ceph-mon[35312]: pgmap v1947: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:09:50.031 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:09:49 smithi022 ceph-mon[31985]: pgmap v1947: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:09:50.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:09:50 smithi134 ceph-mon[35037]: 2 slow requests, 0 included below; oldest blocked for > 3176.889807 secs 2024-03-28T19:09:50.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:09:50 smithi079 ceph-mon[35312]: 2 slow requests, 0 included below; oldest blocked for > 3176.889807 secs 2024-03-28T19:09:51.031 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:09:50 smithi022 ceph-mon[31985]: 2 slow requests, 0 included below; oldest blocked for > 3176.889807 secs 2024-03-28T19:09:51.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:09:51 smithi134 ceph-mon[35037]: pgmap v1948: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T19:09:51.931 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:09:51 smithi079 ceph-mon[35312]: pgmap v1948: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T19:09:52.031 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:09:51 smithi022 ceph-mon[31985]: pgmap v1948: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T19:09:53.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:09:53 smithi134 ceph-mon[35037]: pgmap v1949: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:09:53.931 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:09:53 smithi079 ceph-mon[35312]: pgmap v1949: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:09:54.031 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:09:53 smithi022 ceph-mon[31985]: pgmap v1949: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:09:55.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:09:55 smithi134 ceph-mon[35037]: pgmap v1950: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:09:55.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:09:55 smithi134 ceph-mon[35037]: 2 slow requests, 0 included below; oldest blocked for > 3181.889863 secs 2024-03-28T19:09:55.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:09:55 smithi079 ceph-mon[35312]: pgmap v1950: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:09:55.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:09:55 smithi079 ceph-mon[35312]: 2 slow requests, 0 included below; oldest blocked for > 3181.889863 secs 2024-03-28T19:09:56.031 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:09:55 smithi022 ceph-mon[31985]: pgmap v1950: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:09:56.031 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:09:55 smithi022 ceph-mon[31985]: 2 slow requests, 0 included below; oldest blocked for > 3181.889863 secs 2024-03-28T19:09:57.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:09:57 smithi134 ceph-mon[35037]: pgmap v1951: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T19:09:57.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:09:57 smithi079 ceph-mon[35312]: pgmap v1951: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T19:09:58.032 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:09:57 smithi022 ceph-mon[31985]: pgmap v1951: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T19:09:58.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:09:58 smithi134 ceph-mon[35037]: pgmap v1952: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:09:58.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:09:58 smithi079 ceph-mon[35312]: pgmap v1952: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:09:59.031 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:09:58 smithi022 ceph-mon[31985]: pgmap v1952: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:10:00.174 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:09:59 smithi134 ceph-mon[35037]: 2 slow requests, 0 included below; oldest blocked for > 3186.889983 secs 2024-03-28T19:10:00.181 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:09:59 smithi079 ceph-mon[35312]: 2 slow requests, 0 included below; oldest blocked for > 3186.889983 secs 2024-03-28T19:10:00.281 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:09:59 smithi022 ceph-mon[31985]: 2 slow requests, 0 included below; oldest blocked for > 3186.889983 secs 2024-03-28T19:10:01.174 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:10:00 smithi134 ceph-mon[35037]: overall HEALTH_WARN 4 failed cephadm daemon(s); 1 MDSs report slow requests 2024-03-28T19:10:01.174 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:10:00 smithi134 ceph-mon[35037]: pgmap v1953: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T19:10:01.182 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:10:00 smithi079 ceph-mon[35312]: overall HEALTH_WARN 4 failed cephadm daemon(s); 1 MDSs report slow requests 2024-03-28T19:10:01.182 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:10:00 smithi079 ceph-mon[35312]: pgmap v1953: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T19:10:01.281 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:10:00 smithi022 ceph-mon[31985]: overall HEALTH_WARN 4 failed cephadm daemon(s); 1 MDSs report slow requests 2024-03-28T19:10:01.281 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:10:00 smithi022 ceph-mon[31985]: pgmap v1953: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T19:10:03.424 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:10:03 smithi134 ceph-mon[35037]: pgmap v1954: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:10:03.432 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:10:03 smithi079 ceph-mon[35312]: pgmap v1954: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:10:03.531 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:10:03 smithi022 ceph-mon[31985]: pgmap v1954: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:10:05.424 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:10:05 smithi134 ceph-mon[35037]: pgmap v1955: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:10:05.424 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:10:05 smithi134 ceph-mon[35037]: 2 slow requests, 0 included below; oldest blocked for > 3191.890053 secs 2024-03-28T19:10:05.432 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:10:05 smithi079 ceph-mon[35312]: pgmap v1955: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:10:05.432 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:10:05 smithi079 ceph-mon[35312]: 2 slow requests, 0 included below; oldest blocked for > 3191.890053 secs 2024-03-28T19:10:05.531 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:10:05 smithi022 ceph-mon[31985]: pgmap v1955: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:10:05.531 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:10:05 smithi022 ceph-mon[31985]: 2 slow requests, 0 included below; oldest blocked for > 3191.890053 secs 2024-03-28T19:10:07.424 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:10:07 smithi134 ceph-mon[35037]: pgmap v1956: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T19:10:07.432 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:10:07 smithi079 ceph-mon[35312]: pgmap v1956: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T19:10:07.531 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:10:07 smithi022 ceph-mon[31985]: pgmap v1956: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T19:10:09.424 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:10:09 smithi134 ceph-mon[35037]: pgmap v1957: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:10:09.432 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:10:09 smithi079 ceph-mon[35312]: pgmap v1957: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:10:09.531 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:10:09 smithi022 ceph-mon[31985]: pgmap v1957: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:10:10.424 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:10:10 smithi134 ceph-mon[35037]: 2 slow requests, 0 included below; oldest blocked for > 3196.890122 secs 2024-03-28T19:10:10.432 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:10:10 smithi079 ceph-mon[35312]: 2 slow requests, 0 included below; oldest blocked for > 3196.890122 secs 2024-03-28T19:10:10.531 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:10:10 smithi022 ceph-mon[31985]: 2 slow requests, 0 included below; oldest blocked for > 3196.890122 secs 2024-03-28T19:10:11.424 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:10:11 smithi134 ceph-mon[35037]: pgmap v1958: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T19:10:11.432 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:10:11 smithi079 ceph-mon[35312]: pgmap v1958: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T19:10:11.531 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:10:11 smithi022 ceph-mon[31985]: pgmap v1958: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T19:10:13.424 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:10:13 smithi134 ceph-mon[35037]: pgmap v1959: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:10:13.432 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:10:13 smithi079 ceph-mon[35312]: pgmap v1959: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:10:13.531 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:10:13 smithi022 ceph-mon[31985]: pgmap v1959: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:10:15.424 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:10:15 smithi134 ceph-mon[35037]: pgmap v1960: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:10:15.424 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:10:15 smithi134 ceph-mon[35037]: 2 slow requests, 0 included below; oldest blocked for > 3201.890213 secs 2024-03-28T19:10:15.432 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:10:15 smithi079 ceph-mon[35312]: pgmap v1960: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:10:15.432 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:10:15 smithi079 ceph-mon[35312]: 2 slow requests, 0 included below; oldest blocked for > 3201.890213 secs 2024-03-28T19:10:15.531 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:10:15 smithi022 ceph-mon[31985]: pgmap v1960: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:10:15.531 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:10:15 smithi022 ceph-mon[31985]: 2 slow requests, 0 included below; oldest blocked for > 3201.890213 secs 2024-03-28T19:10:17.424 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:10:17 smithi134 ceph-mon[35037]: pgmap v1961: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T19:10:17.431 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:10:17 smithi079 ceph-mon[35312]: pgmap v1961: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T19:10:17.531 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:10:17 smithi022 ceph-mon[31985]: pgmap v1961: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T19:10:19.424 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:10:19 smithi134 ceph-mon[35037]: pgmap v1962: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:10:19.424 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:10:19 smithi134 ceph-mon[35037]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T19:10:19.432 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:10:19 smithi079 ceph-mon[35312]: pgmap v1962: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:10:19.432 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:10:19 smithi079 ceph-mon[35312]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T19:10:19.531 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:10:19 smithi022 ceph-mon[31985]: pgmap v1962: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:10:19.531 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:10:19 smithi022 ceph-mon[31985]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T19:10:20.531 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:10:20 smithi022 ceph-mon[31985]: from='mgr.15522 ' entity='mgr.x' 2024-03-28T19:10:20.531 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:10:20 smithi022 ceph-mon[31985]: from='mgr.15522 ' entity='mgr.x' 2024-03-28T19:10:20.531 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:10:20 smithi022 ceph-mon[31985]: 2 slow requests, 0 included below; oldest blocked for > 3206.890280 secs 2024-03-28T19:10:20.595 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:10:20 smithi134 ceph-mon[35037]: from='mgr.15522 ' entity='mgr.x' 2024-03-28T19:10:20.595 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:10:20 smithi134 ceph-mon[35037]: from='mgr.15522 ' entity='mgr.x' 2024-03-28T19:10:20.595 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:10:20 smithi134 ceph-mon[35037]: 2 slow requests, 0 included below; oldest blocked for > 3206.890280 secs 2024-03-28T19:10:20.681 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:10:20 smithi079 ceph-mon[35312]: from='mgr.15522 ' entity='mgr.x' 2024-03-28T19:10:20.681 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:10:20 smithi079 ceph-mon[35312]: from='mgr.15522 ' entity='mgr.x' 2024-03-28T19:10:20.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:10:20 smithi079 ceph-mon[35312]: 2 slow requests, 0 included below; oldest blocked for > 3206.890280 secs 2024-03-28T19:10:21.531 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:10:21 smithi022 ceph-mon[31985]: pgmap v1963: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T19:10:21.674 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:10:21 smithi134 ceph-mon[35037]: pgmap v1963: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T19:10:21.681 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:10:21 smithi079 ceph-mon[35312]: pgmap v1963: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T19:10:22.781 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:10:22 smithi022 ceph-mon[31985]: from='mgr.15522 ' entity='mgr.x' 2024-03-28T19:10:22.796 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:10:22 smithi022 ceph-mon[31985]: from='mgr.15522 ' entity='mgr.x' 2024-03-28T19:10:22.796 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:10:22 smithi022 ceph-mon[31985]: from='mgr.15522 ' entity='mgr.x' 2024-03-28T19:10:22.796 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:10:22 smithi022 ceph-mon[31985]: from='mgr.15522 ' entity='mgr.x' 2024-03-28T19:10:22.796 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:10:22 smithi022 ceph-mon[31985]: from='mgr.15522 ' entity='mgr.x' 2024-03-28T19:10:22.796 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:10:22 smithi022 ceph-mon[31985]: from='mgr.15522 ' entity='mgr.x' 2024-03-28T19:10:22.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:10:22 smithi134 ceph-mon[35037]: from='mgr.15522 ' entity='mgr.x' 2024-03-28T19:10:22.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:10:22 smithi134 ceph-mon[35037]: from='mgr.15522 ' entity='mgr.x' 2024-03-28T19:10:22.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:10:22 smithi134 ceph-mon[35037]: from='mgr.15522 ' entity='mgr.x' 2024-03-28T19:10:22.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:10:22 smithi134 ceph-mon[35037]: from='mgr.15522 ' entity='mgr.x' 2024-03-28T19:10:22.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:10:22 smithi134 ceph-mon[35037]: from='mgr.15522 ' entity='mgr.x' 2024-03-28T19:10:22.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:10:22 smithi134 ceph-mon[35037]: from='mgr.15522 ' entity='mgr.x' 2024-03-28T19:10:22.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:10:22 smithi079 ceph-mon[35312]: from='mgr.15522 ' entity='mgr.x' 2024-03-28T19:10:22.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:10:22 smithi079 ceph-mon[35312]: from='mgr.15522 ' entity='mgr.x' 2024-03-28T19:10:22.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:10:22 smithi079 ceph-mon[35312]: from='mgr.15522 ' entity='mgr.x' 2024-03-28T19:10:22.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:10:22 smithi079 ceph-mon[35312]: from='mgr.15522 ' entity='mgr.x' 2024-03-28T19:10:22.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:10:22 smithi079 ceph-mon[35312]: from='mgr.15522 ' entity='mgr.x' 2024-03-28T19:10:22.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:10:22 smithi079 ceph-mon[35312]: from='mgr.15522 ' entity='mgr.x' 2024-03-28T19:10:23.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:10:23 smithi022 ceph-mon[31985]: pgmap v1964: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:10:23.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:10:23 smithi022 ceph-mon[31985]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-28T19:10:23.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:10:23 smithi022 ceph-mon[31985]: from='mgr.15522 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-28T19:10:23.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:10:23 smithi022 ceph-mon[31985]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-28T19:10:23.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:10:23 smithi022 ceph-mon[31985]: from='mgr.15522 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-28T19:10:23.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:10:23 smithi022 ceph-mon[31985]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-03-28T19:10:23.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:10:23 smithi022 ceph-mon[31985]: from='mgr.15522 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-03-28T19:10:23.783 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:10:23 smithi022 ceph-mon[31985]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-03-28T19:10:23.783 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:10:23 smithi022 ceph-mon[31985]: from='mgr.15522 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-03-28T19:10:23.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:10:23 smithi134 ceph-mon[35037]: pgmap v1964: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:10:23.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:10:23 smithi134 ceph-mon[35037]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-28T19:10:23.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:10:23 smithi134 ceph-mon[35037]: from='mgr.15522 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-28T19:10:23.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:10:23 smithi134 ceph-mon[35037]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-28T19:10:23.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:10:23 smithi134 ceph-mon[35037]: from='mgr.15522 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-28T19:10:23.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:10:23 smithi134 ceph-mon[35037]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-03-28T19:10:23.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:10:23 smithi134 ceph-mon[35037]: from='mgr.15522 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-03-28T19:10:23.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:10:23 smithi134 ceph-mon[35037]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-03-28T19:10:23.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:10:23 smithi134 ceph-mon[35037]: from='mgr.15522 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-03-28T19:10:23.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:10:23 smithi079 ceph-mon[35312]: pgmap v1964: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:10:23.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:10:23 smithi079 ceph-mon[35312]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-28T19:10:23.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:10:23 smithi079 ceph-mon[35312]: from='mgr.15522 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-28T19:10:23.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:10:23 smithi079 ceph-mon[35312]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-28T19:10:23.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:10:23 smithi079 ceph-mon[35312]: from='mgr.15522 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-28T19:10:23.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:10:23 smithi079 ceph-mon[35312]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-03-28T19:10:23.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:10:23 smithi079 ceph-mon[35312]: from='mgr.15522 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-03-28T19:10:23.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:10:23 smithi079 ceph-mon[35312]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-03-28T19:10:23.933 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:10:23 smithi079 ceph-mon[35312]: from='mgr.15522 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-03-28T19:10:24.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:10:24 smithi022 ceph-mon[31985]: Adjusting osd_memory_target on smithi079 to 176.7M 2024-03-28T19:10:24.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:10:24 smithi022 ceph-mon[31985]: Unable to set osd_memory_target on smithi079 to 185386700: error parsing value: Value '185386700' is below minimum 939524096 2024-03-28T19:10:24.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:10:24 smithi022 ceph-mon[31985]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-28T19:10:24.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:10:24 smithi022 ceph-mon[31985]: from='mgr.15522 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-28T19:10:24.783 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:10:24 smithi022 ceph-mon[31985]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-28T19:10:24.783 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:10:24 smithi022 ceph-mon[31985]: from='mgr.15522 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-28T19:10:24.783 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:10:24 smithi022 ceph-mon[31985]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-03-28T19:10:24.783 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:10:24 smithi022 ceph-mon[31985]: from='mgr.15522 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-03-28T19:10:24.783 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:10:24 smithi022 ceph-mon[31985]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-03-28T19:10:24.783 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:10:24 smithi022 ceph-mon[31985]: from='mgr.15522 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-03-28T19:10:24.783 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:10:24 smithi022 ceph-mon[31985]: Adjusting osd_memory_target on smithi134 to 176.8M 2024-03-28T19:10:24.783 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:10:24 smithi022 ceph-mon[31985]: Unable to set osd_memory_target on smithi134 to 185393868: error parsing value: Value '185393868' is below minimum 939524096 2024-03-28T19:10:24.783 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:10:24 smithi022 ceph-mon[31985]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-28T19:10:24.783 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:10:24 smithi022 ceph-mon[31985]: from='mgr.15522 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-28T19:10:24.783 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:10:24 smithi022 ceph-mon[31985]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-28T19:10:24.783 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:10:24 smithi022 ceph-mon[31985]: from='mgr.15522 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-28T19:10:24.783 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:10:24 smithi022 ceph-mon[31985]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-03-28T19:10:24.783 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:10:24 smithi022 ceph-mon[31985]: from='mgr.15522 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-03-28T19:10:24.784 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:10:24 smithi022 ceph-mon[31985]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-03-28T19:10:24.784 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:10:24 smithi022 ceph-mon[31985]: from='mgr.15522 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-03-28T19:10:24.784 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:10:24 smithi022 ceph-mon[31985]: Adjusting osd_memory_target on smithi022 to 177.8M 2024-03-28T19:10:24.784 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:10:24 smithi022 ceph-mon[31985]: Unable to set osd_memory_target on smithi022 to 186497740: error parsing value: Value '186497740' is below minimum 939524096 2024-03-28T19:10:24.784 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:10:24 smithi022 ceph-mon[31985]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T19:10:24.784 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:10:24 smithi022 ceph-mon[31985]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T19:10:24.784 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:10:24 smithi022 ceph-mon[31985]: pgmap v1965: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 6 op/s 2024-03-28T19:10:24.784 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:10:24 smithi022 ceph-mon[31985]: pgmap v1966: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-28T19:10:24.784 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:10:24 smithi022 ceph-mon[31985]: from='mgr.15522 ' entity='mgr.x' 2024-03-28T19:10:24.925 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:10:24 smithi134 ceph-mon[35037]: Adjusting osd_memory_target on smithi079 to 176.7M 2024-03-28T19:10:24.925 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:10:24 smithi134 ceph-mon[35037]: Unable to set osd_memory_target on smithi079 to 185386700: error parsing value: Value '185386700' is below minimum 939524096 2024-03-28T19:10:24.925 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:10:24 smithi134 ceph-mon[35037]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-28T19:10:24.925 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:10:24 smithi134 ceph-mon[35037]: from='mgr.15522 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-28T19:10:24.925 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:10:24 smithi134 ceph-mon[35037]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-28T19:10:24.925 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:10:24 smithi134 ceph-mon[35037]: from='mgr.15522 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-28T19:10:24.925 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:10:24 smithi134 ceph-mon[35037]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-03-28T19:10:24.925 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:10:24 smithi134 ceph-mon[35037]: from='mgr.15522 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-03-28T19:10:24.925 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:10:24 smithi134 ceph-mon[35037]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-03-28T19:10:24.926 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:10:24 smithi134 ceph-mon[35037]: from='mgr.15522 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-03-28T19:10:24.926 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:10:24 smithi134 ceph-mon[35037]: Adjusting osd_memory_target on smithi134 to 176.8M 2024-03-28T19:10:24.926 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:10:24 smithi134 ceph-mon[35037]: Unable to set osd_memory_target on smithi134 to 185393868: error parsing value: Value '185393868' is below minimum 939524096 2024-03-28T19:10:24.926 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:10:24 smithi134 ceph-mon[35037]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-28T19:10:24.926 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:10:24 smithi134 ceph-mon[35037]: from='mgr.15522 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-28T19:10:24.926 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:10:24 smithi134 ceph-mon[35037]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-28T19:10:24.926 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:10:24 smithi134 ceph-mon[35037]: from='mgr.15522 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-28T19:10:24.926 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:10:24 smithi134 ceph-mon[35037]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-03-28T19:10:24.926 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:10:24 smithi134 ceph-mon[35037]: from='mgr.15522 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-03-28T19:10:24.926 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:10:24 smithi134 ceph-mon[35037]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-03-28T19:10:24.926 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:10:24 smithi134 ceph-mon[35037]: from='mgr.15522 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-03-28T19:10:24.927 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:10:24 smithi134 ceph-mon[35037]: Adjusting osd_memory_target on smithi022 to 177.8M 2024-03-28T19:10:24.927 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:10:24 smithi134 ceph-mon[35037]: Unable to set osd_memory_target on smithi022 to 186497740: error parsing value: Value '186497740' is below minimum 939524096 2024-03-28T19:10:24.927 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:10:24 smithi134 ceph-mon[35037]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T19:10:24.927 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:10:24 smithi134 ceph-mon[35037]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T19:10:24.927 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:10:24 smithi134 ceph-mon[35037]: pgmap v1965: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 6 op/s 2024-03-28T19:10:24.927 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:10:24 smithi134 ceph-mon[35037]: pgmap v1966: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-28T19:10:24.927 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:10:24 smithi134 ceph-mon[35037]: from='mgr.15522 ' entity='mgr.x' 2024-03-28T19:10:24.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:10:24 smithi079 ceph-mon[35312]: Adjusting osd_memory_target on smithi079 to 176.7M 2024-03-28T19:10:24.933 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:10:24 smithi079 ceph-mon[35312]: Unable to set osd_memory_target on smithi079 to 185386700: error parsing value: Value '185386700' is below minimum 939524096 2024-03-28T19:10:24.933 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:10:24 smithi079 ceph-mon[35312]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-28T19:10:24.933 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:10:24 smithi079 ceph-mon[35312]: from='mgr.15522 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-28T19:10:24.933 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:10:24 smithi079 ceph-mon[35312]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-28T19:10:24.933 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:10:24 smithi079 ceph-mon[35312]: from='mgr.15522 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-28T19:10:24.933 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:10:24 smithi079 ceph-mon[35312]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-03-28T19:10:24.933 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:10:24 smithi079 ceph-mon[35312]: from='mgr.15522 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-03-28T19:10:24.933 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:10:24 smithi079 ceph-mon[35312]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-03-28T19:10:24.933 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:10:24 smithi079 ceph-mon[35312]: from='mgr.15522 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-03-28T19:10:24.933 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:10:24 smithi079 ceph-mon[35312]: Adjusting osd_memory_target on smithi134 to 176.8M 2024-03-28T19:10:24.933 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:10:24 smithi079 ceph-mon[35312]: Unable to set osd_memory_target on smithi134 to 185393868: error parsing value: Value '185393868' is below minimum 939524096 2024-03-28T19:10:24.933 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:10:24 smithi079 ceph-mon[35312]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-28T19:10:24.933 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:10:24 smithi079 ceph-mon[35312]: from='mgr.15522 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-28T19:10:24.933 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:10:24 smithi079 ceph-mon[35312]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-28T19:10:24.933 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:10:24 smithi079 ceph-mon[35312]: from='mgr.15522 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-28T19:10:24.933 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:10:24 smithi079 ceph-mon[35312]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-03-28T19:10:24.934 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:10:24 smithi079 ceph-mon[35312]: from='mgr.15522 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-03-28T19:10:24.934 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:10:24 smithi079 ceph-mon[35312]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-03-28T19:10:24.934 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:10:24 smithi079 ceph-mon[35312]: from='mgr.15522 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-03-28T19:10:24.934 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:10:24 smithi079 ceph-mon[35312]: Adjusting osd_memory_target on smithi022 to 177.8M 2024-03-28T19:10:24.934 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:10:24 smithi079 ceph-mon[35312]: Unable to set osd_memory_target on smithi022 to 186497740: error parsing value: Value '186497740' is below minimum 939524096 2024-03-28T19:10:24.934 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:10:24 smithi079 ceph-mon[35312]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T19:10:24.934 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:10:24 smithi079 ceph-mon[35312]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T19:10:24.934 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:10:24 smithi079 ceph-mon[35312]: pgmap v1965: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 6 op/s 2024-03-28T19:10:24.934 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:10:24 smithi079 ceph-mon[35312]: pgmap v1966: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-28T19:10:24.934 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:10:24 smithi079 ceph-mon[35312]: from='mgr.15522 ' entity='mgr.x' 2024-03-28T19:10:25.781 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:10:25 smithi022 ceph-mon[31985]: 2 slow requests, 0 included below; oldest blocked for > 3211.890342 secs 2024-03-28T19:10:25.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:10:25 smithi134 ceph-mon[35037]: 2 slow requests, 0 included below; oldest blocked for > 3211.890342 secs 2024-03-28T19:10:25.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:10:25 smithi079 ceph-mon[35312]: 2 slow requests, 0 included below; oldest blocked for > 3211.890342 secs 2024-03-28T19:10:26.781 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:10:26 smithi022 ceph-mon[31985]: pgmap v1967: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-28T19:10:26.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:10:26 smithi134 ceph-mon[35037]: pgmap v1967: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-28T19:10:26.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:10:26 smithi079 ceph-mon[35312]: pgmap v1967: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-28T19:10:28.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:10:28 smithi134 ceph-mon[35037]: pgmap v1968: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-28T19:10:28.931 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:10:28 smithi079 ceph-mon[35312]: pgmap v1968: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-28T19:10:29.031 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:10:28 smithi022 ceph-mon[31985]: pgmap v1968: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-28T19:10:30.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:10:30 smithi134 ceph-mon[35037]: 2 slow requests, 0 included below; oldest blocked for > 3216.890419 secs 2024-03-28T19:10:30.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:10:30 smithi134 ceph-mon[35037]: pgmap v1969: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-28T19:10:30.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:10:30 smithi079 ceph-mon[35312]: 2 slow requests, 0 included below; oldest blocked for > 3216.890419 secs 2024-03-28T19:10:30.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:10:30 smithi079 ceph-mon[35312]: pgmap v1969: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-28T19:10:31.031 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:10:30 smithi022 ceph-mon[31985]: 2 slow requests, 0 included below; oldest blocked for > 3216.890419 secs 2024-03-28T19:10:31.031 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:10:30 smithi022 ceph-mon[31985]: pgmap v1969: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-28T19:10:32.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:10:32 smithi134 ceph-mon[35037]: pgmap v1970: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-28T19:10:32.931 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:10:32 smithi079 ceph-mon[35312]: pgmap v1970: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-28T19:10:33.031 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:10:32 smithi022 ceph-mon[31985]: pgmap v1970: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-28T19:10:34.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:10:34 smithi134 ceph-mon[35037]: pgmap v1971: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 7 op/s 2024-03-28T19:10:34.931 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:10:34 smithi079 ceph-mon[35312]: pgmap v1971: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 7 op/s 2024-03-28T19:10:35.031 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:10:34 smithi022 ceph-mon[31985]: pgmap v1971: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 7 op/s 2024-03-28T19:10:36.031 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:10:35 smithi022 ceph-mon[31985]: 2 slow requests, 0 included below; oldest blocked for > 3221.890537 secs 2024-03-28T19:10:36.174 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:10:35 smithi134 ceph-mon[35037]: 2 slow requests, 0 included below; oldest blocked for > 3221.890537 secs 2024-03-28T19:10:36.181 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:10:35 smithi079 ceph-mon[35312]: 2 slow requests, 0 included below; oldest blocked for > 3221.890537 secs 2024-03-28T19:10:37.031 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:10:36 smithi022 ceph-mon[31985]: pgmap v1972: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T19:10:37.174 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:10:36 smithi134 ceph-mon[35037]: pgmap v1972: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T19:10:37.182 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:10:36 smithi079 ceph-mon[35312]: pgmap v1972: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T19:10:38.031 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:10:37 smithi022 ceph-mon[31985]: pgmap v1973: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:10:38.174 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:10:37 smithi134 ceph-mon[35037]: pgmap v1973: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:10:38.181 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:10:37 smithi079 ceph-mon[35312]: pgmap v1973: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:10:40.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:10:40 smithi134 ceph-mon[35037]: 2 slow requests, 0 included below; oldest blocked for > 3226.890637 secs 2024-03-28T19:10:40.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:10:40 smithi134 ceph-mon[35037]: pgmap v1974: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T19:10:40.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:10:40 smithi079 ceph-mon[35312]: 2 slow requests, 0 included below; oldest blocked for > 3226.890637 secs 2024-03-28T19:10:40.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:10:40 smithi079 ceph-mon[35312]: pgmap v1974: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T19:10:41.031 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:10:40 smithi022 ceph-mon[31985]: 2 slow requests, 0 included below; oldest blocked for > 3226.890637 secs 2024-03-28T19:10:41.031 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:10:40 smithi022 ceph-mon[31985]: pgmap v1974: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T19:10:42.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:10:42 smithi134 ceph-mon[35037]: pgmap v1975: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:10:42.931 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:10:42 smithi079 ceph-mon[35312]: pgmap v1975: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:10:43.031 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:10:42 smithi022 ceph-mon[31985]: pgmap v1975: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:10:44.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:10:44 smithi134 ceph-mon[35037]: pgmap v1976: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:10:44.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:10:44 smithi079 ceph-mon[35312]: pgmap v1976: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:10:45.031 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:10:44 smithi022 ceph-mon[31985]: pgmap v1976: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:10:45.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:10:45 smithi134 ceph-mon[35037]: 2 slow requests, 0 included below; oldest blocked for > 3231.890707 secs 2024-03-28T19:10:45.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:10:45 smithi079 ceph-mon[35312]: 2 slow requests, 0 included below; oldest blocked for > 3231.890707 secs 2024-03-28T19:10:46.031 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:10:45 smithi022 ceph-mon[31985]: 2 slow requests, 0 included below; oldest blocked for > 3231.890707 secs 2024-03-28T19:10:46.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:10:46 smithi079 ceph-mon[35312]: pgmap v1977: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T19:10:47.031 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:10:46 smithi022 ceph-mon[31985]: pgmap v1977: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T19:10:47.174 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:10:46 smithi134 ceph-mon[35037]: pgmap v1977: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T19:10:48.031 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:10:47 smithi022 ceph-mon[31985]: pgmap v1978: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:10:48.174 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:10:47 smithi134 ceph-mon[35037]: pgmap v1978: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:10:48.182 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:10:47 smithi079 ceph-mon[35312]: pgmap v1978: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:10:50.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:10:50 smithi134 ceph-mon[35037]: 2 slow requests, 0 included below; oldest blocked for > 3236.890772 secs 2024-03-28T19:10:50.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:10:50 smithi134 ceph-mon[35037]: pgmap v1979: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T19:10:50.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:10:50 smithi079 ceph-mon[35312]: 2 slow requests, 0 included below; oldest blocked for > 3236.890772 secs 2024-03-28T19:10:50.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:10:50 smithi079 ceph-mon[35312]: pgmap v1979: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T19:10:51.031 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:10:50 smithi022 ceph-mon[31985]: 2 slow requests, 0 included below; oldest blocked for > 3236.890772 secs 2024-03-28T19:10:51.031 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:10:50 smithi022 ceph-mon[31985]: pgmap v1979: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T19:10:52.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:10:52 smithi134 ceph-mon[35037]: pgmap v1980: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:10:52.931 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:10:52 smithi079 ceph-mon[35312]: pgmap v1980: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:10:53.031 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:10:52 smithi022 ceph-mon[31985]: pgmap v1980: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:10:54.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:10:54 smithi079 ceph-mon[35312]: pgmap v1981: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:10:55.031 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:10:54 smithi022 ceph-mon[31985]: pgmap v1981: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:10:55.174 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:10:54 smithi134 ceph-mon[35037]: pgmap v1981: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:10:55.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:10:55 smithi079 ceph-mon[35312]: 2 slow requests, 0 included below; oldest blocked for > 3241.890844 secs 2024-03-28T19:10:56.031 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:10:55 smithi022 ceph-mon[31985]: 2 slow requests, 0 included below; oldest blocked for > 3241.890844 secs 2024-03-28T19:10:56.174 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:10:55 smithi134 ceph-mon[35037]: 2 slow requests, 0 included below; oldest blocked for > 3241.890844 secs 2024-03-28T19:10:56.931 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:10:56 smithi079 ceph-mon[35312]: pgmap v1982: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T19:10:57.031 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:10:56 smithi022 ceph-mon[31985]: pgmap v1982: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T19:10:57.174 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:10:56 smithi134 ceph-mon[35037]: pgmap v1982: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T19:10:58.031 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:10:57 smithi022 ceph-mon[31985]: pgmap v1983: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:10:58.174 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:10:57 smithi134 ceph-mon[35037]: pgmap v1983: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:10:58.182 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:10:57 smithi079 ceph-mon[35312]: pgmap v1983: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:11:00.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:11:00 smithi134 ceph-mon[35037]: 2 slow requests, 0 included below; oldest blocked for > 3246.890978 secs 2024-03-28T19:11:00.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:11:00 smithi134 ceph-mon[35037]: pgmap v1984: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T19:11:00.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:11:00 smithi079 ceph-mon[35312]: 2 slow requests, 0 included below; oldest blocked for > 3246.890978 secs 2024-03-28T19:11:00.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:11:00 smithi079 ceph-mon[35312]: pgmap v1984: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T19:11:01.031 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:11:00 smithi022 ceph-mon[31985]: 2 slow requests, 0 included below; oldest blocked for > 3246.890978 secs 2024-03-28T19:11:01.031 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:11:00 smithi022 ceph-mon[31985]: pgmap v1984: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T19:11:02.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:11:02 smithi134 ceph-mon[35037]: pgmap v1985: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:11:02.931 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:11:02 smithi079 ceph-mon[35312]: pgmap v1985: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:11:03.031 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:11:02 smithi022 ceph-mon[31985]: pgmap v1985: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:11:04.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:11:04 smithi079 ceph-mon[35312]: pgmap v1986: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:11:05.031 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:11:04 smithi022 ceph-mon[31985]: pgmap v1986: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:11:05.174 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:11:04 smithi134 ceph-mon[35037]: pgmap v1986: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:11:05.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:11:05 smithi079 ceph-mon[35312]: 2 slow requests, 0 included below; oldest blocked for > 3251.891043 secs 2024-03-28T19:11:05.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:11:05 smithi079 ceph-mon[35312]: pgmap v1987: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T19:11:06.031 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:11:05 smithi022 ceph-mon[31985]: 2 slow requests, 0 included below; oldest blocked for > 3251.891043 secs 2024-03-28T19:11:06.031 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:11:05 smithi022 ceph-mon[31985]: pgmap v1987: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T19:11:06.174 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:11:05 smithi134 ceph-mon[35037]: 2 slow requests, 0 included below; oldest blocked for > 3251.891043 secs 2024-03-28T19:11:06.174 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:11:05 smithi134 ceph-mon[35037]: pgmap v1987: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T19:11:08.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:11:08 smithi134 ceph-mon[35037]: pgmap v1988: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:11:08.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:11:08 smithi079 ceph-mon[35312]: pgmap v1988: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:11:09.031 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:11:08 smithi022 ceph-mon[31985]: pgmap v1988: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:11:10.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:11:10 smithi079 ceph-mon[35312]: 2 slow requests, 0 included below; oldest blocked for > 3256.891129 secs 2024-03-28T19:11:10.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:11:10 smithi079 ceph-mon[35312]: pgmap v1989: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T19:11:11.031 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:11:10 smithi022 ceph-mon[31985]: 2 slow requests, 0 included below; oldest blocked for > 3256.891129 secs 2024-03-28T19:11:11.031 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:11:10 smithi022 ceph-mon[31985]: pgmap v1989: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T19:11:11.174 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:11:10 smithi134 ceph-mon[35037]: 2 slow requests, 0 included below; oldest blocked for > 3256.891129 secs 2024-03-28T19:11:11.174 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:11:10 smithi134 ceph-mon[35037]: pgmap v1989: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T19:11:13.031 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:11:12 smithi022 ceph-mon[31985]: pgmap v1990: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:11:13.174 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:11:12 smithi134 ceph-mon[35037]: pgmap v1990: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:11:13.182 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:11:12 smithi079 ceph-mon[35312]: pgmap v1990: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:11:14.035 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:11:13 smithi022 ceph-mon[31985]: pgmap v1991: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:11:14.174 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:11:13 smithi134 ceph-mon[35037]: pgmap v1991: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:11:14.181 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:11:13 smithi079 ceph-mon[35312]: pgmap v1991: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:11:15.174 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:11:14 smithi134 ceph-mon[35037]: 2 slow requests, 0 included below; oldest blocked for > 3261.891169 secs 2024-03-28T19:11:15.181 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:11:14 smithi079 ceph-mon[35312]: 2 slow requests, 0 included below; oldest blocked for > 3261.891169 secs 2024-03-28T19:11:15.281 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:11:14 smithi022 ceph-mon[31985]: 2 slow requests, 0 included below; oldest blocked for > 3261.891169 secs 2024-03-28T19:11:16.174 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:11:15 smithi134 ceph-mon[35037]: pgmap v1992: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 9 op/s 2024-03-28T19:11:16.182 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:11:15 smithi079 ceph-mon[35312]: pgmap v1992: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 9 op/s 2024-03-28T19:11:16.281 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:11:15 smithi022 ceph-mon[31985]: pgmap v1992: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 9 op/s 2024-03-28T19:11:18.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:11:18 smithi079 ceph-mon[35312]: pgmap v1993: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T19:11:18.985 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:11:18 smithi022 ceph-mon[31985]: pgmap v1993: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T19:11:19.174 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:11:18 smithi134 ceph-mon[35037]: pgmap v1993: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T19:11:20.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:11:20 smithi079 ceph-mon[35312]: 2 slow requests, 0 included below; oldest blocked for > 3266.891231 secs 2024-03-28T19:11:20.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:11:20 smithi079 ceph-mon[35312]: pgmap v1994: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 9 op/s 2024-03-28T19:11:20.976 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:11:20 smithi134 ceph-mon[35037]: 2 slow requests, 0 included below; oldest blocked for > 3266.891231 secs 2024-03-28T19:11:20.976 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:11:20 smithi134 ceph-mon[35037]: pgmap v1994: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 9 op/s 2024-03-28T19:11:21.033 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:11:20 smithi022 ceph-mon[31985]: 2 slow requests, 0 included below; oldest blocked for > 3266.891231 secs 2024-03-28T19:11:21.034 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:11:20 smithi022 ceph-mon[31985]: pgmap v1994: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 9 op/s 2024-03-28T19:11:22.031 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:11:21 smithi022 ceph-mon[31985]: pgmap v1995: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T19:11:22.174 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:11:21 smithi134 ceph-mon[35037]: pgmap v1995: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T19:11:22.181 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:11:21 smithi079 ceph-mon[35312]: pgmap v1995: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T19:11:24.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:11:24 smithi079 ceph-mon[35312]: pgmap v1996: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T19:11:24.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:11:24 smithi079 ceph-mon[35312]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T19:11:25.031 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:11:24 smithi022 ceph-mon[31985]: pgmap v1996: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T19:11:25.031 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:11:24 smithi022 ceph-mon[31985]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T19:11:25.174 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:11:24 smithi134 ceph-mon[35037]: pgmap v1996: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T19:11:25.174 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:11:24 smithi134 ceph-mon[35037]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T19:11:26.031 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:11:25 smithi022 ceph-mon[31985]: 2 slow requests, 0 included below; oldest blocked for > 3271.891304 secs 2024-03-28T19:11:26.031 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:11:25 smithi022 ceph-mon[31985]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T19:11:26.031 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:11:25 smithi022 ceph-mon[31985]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T19:11:26.031 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:11:25 smithi022 ceph-mon[31985]: from='mgr.15522 ' entity='mgr.x' 2024-03-28T19:11:26.174 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:11:25 smithi134 ceph-mon[35037]: 2 slow requests, 0 included below; oldest blocked for > 3271.891304 secs 2024-03-28T19:11:26.174 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:11:25 smithi134 ceph-mon[35037]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T19:11:26.174 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:11:25 smithi134 ceph-mon[35037]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T19:11:26.174 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:11:25 smithi134 ceph-mon[35037]: from='mgr.15522 ' entity='mgr.x' 2024-03-28T19:11:26.181 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:11:25 smithi079 ceph-mon[35312]: 2 slow requests, 0 included below; oldest blocked for > 3271.891304 secs 2024-03-28T19:11:26.182 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:11:25 smithi079 ceph-mon[35312]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T19:11:26.182 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:11:25 smithi079 ceph-mon[35312]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T19:11:26.182 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:11:25 smithi079 ceph-mon[35312]: from='mgr.15522 ' entity='mgr.x' 2024-03-28T19:11:27.031 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:11:26 smithi022 ceph-mon[31985]: pgmap v1997: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 10 op/s 2024-03-28T19:11:27.031 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:11:26 smithi022 ceph-mon[31985]: pgmap v1998: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-03-28T19:11:27.174 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:11:26 smithi134 ceph-mon[35037]: pgmap v1997: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 10 op/s 2024-03-28T19:11:27.174 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:11:26 smithi134 ceph-mon[35037]: pgmap v1998: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-03-28T19:11:27.182 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:11:26 smithi079 ceph-mon[35312]: pgmap v1997: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 10 op/s 2024-03-28T19:11:27.182 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:11:26 smithi079 ceph-mon[35312]: pgmap v1998: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-03-28T19:11:28.031 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:11:27 smithi022 ceph-mon[31985]: pgmap v1999: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-03-28T19:11:28.174 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:11:27 smithi134 ceph-mon[35037]: pgmap v1999: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-03-28T19:11:28.182 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:11:27 smithi079 ceph-mon[35312]: pgmap v1999: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-03-28T19:11:30.781 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:11:30 smithi022 ceph-mon[31985]: 2 slow requests, 0 included below; oldest blocked for > 3276.891371 secs 2024-03-28T19:11:30.781 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:11:30 smithi022 ceph-mon[31985]: pgmap v2000: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 8 op/s 2024-03-28T19:11:30.884 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:11:30 smithi079 ceph-mon[35312]: 2 slow requests, 0 included below; oldest blocked for > 3276.891371 secs 2024-03-28T19:11:30.884 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:11:30 smithi079 ceph-mon[35312]: pgmap v2000: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 8 op/s 2024-03-28T19:11:30.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:11:30 smithi134 ceph-mon[35037]: 2 slow requests, 0 included below; oldest blocked for > 3276.891371 secs 2024-03-28T19:11:30.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:11:30 smithi134 ceph-mon[35037]: pgmap v2000: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 8 op/s 2024-03-28T19:11:32.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:11:32 smithi134 ceph-mon[35037]: pgmap v2001: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 8 op/s 2024-03-28T19:11:32.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:11:32 smithi079 ceph-mon[35312]: pgmap v2001: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 8 op/s 2024-03-28T19:11:33.032 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:11:32 smithi022 ceph-mon[31985]: pgmap v2001: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 8 op/s 2024-03-28T19:11:34.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:11:34 smithi134 ceph-mon[35037]: pgmap v2002: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 8 op/s 2024-03-28T19:11:34.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:11:34 smithi079 ceph-mon[35312]: pgmap v2002: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 8 op/s 2024-03-28T19:11:35.031 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:11:34 smithi022 ceph-mon[31985]: pgmap v2002: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 8 op/s 2024-03-28T19:11:35.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:11:35 smithi134 ceph-mon[35037]: 2 slow requests, 0 included below; oldest blocked for > 3281.891483 secs 2024-03-28T19:11:35.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:11:35 smithi079 ceph-mon[35312]: 2 slow requests, 0 included below; oldest blocked for > 3281.891483 secs 2024-03-28T19:11:36.031 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:11:35 smithi022 ceph-mon[31985]: 2 slow requests, 0 included below; oldest blocked for > 3281.891483 secs 2024-03-28T19:11:36.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:11:36 smithi134 ceph-mon[35037]: pgmap v2003: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 7 op/s 2024-03-28T19:11:36.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:11:36 smithi079 ceph-mon[35312]: pgmap v2003: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 7 op/s 2024-03-28T19:11:37.031 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:11:36 smithi022 ceph-mon[31985]: pgmap v2003: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 7 op/s 2024-03-28T19:11:38.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:11:38 smithi134 ceph-mon[35037]: pgmap v2004: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T19:11:38.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:11:38 smithi079 ceph-mon[35312]: pgmap v2004: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T19:11:39.031 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:11:38 smithi022 ceph-mon[31985]: pgmap v2004: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T19:11:40.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:11:40 smithi134 ceph-mon[35037]: 2 slow requests, 0 included below; oldest blocked for > 3286.891552 secs 2024-03-28T19:11:40.925 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:11:40 smithi134 ceph-mon[35037]: pgmap v2005: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 9 op/s 2024-03-28T19:11:40.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:11:40 smithi079 ceph-mon[35312]: 2 slow requests, 0 included below; oldest blocked for > 3286.891552 secs 2024-03-28T19:11:40.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:11:40 smithi079 ceph-mon[35312]: pgmap v2005: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 9 op/s 2024-03-28T19:11:41.031 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:11:40 smithi022 ceph-mon[31985]: 2 slow requests, 0 included below; oldest blocked for > 3286.891552 secs 2024-03-28T19:11:41.031 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:11:40 smithi022 ceph-mon[31985]: pgmap v2005: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 9 op/s 2024-03-28T19:11:42.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:11:42 smithi134 ceph-mon[35037]: pgmap v2006: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:11:42.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:11:42 smithi079 ceph-mon[35312]: pgmap v2006: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:11:43.031 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:11:42 smithi022 ceph-mon[31985]: pgmap v2006: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:11:44.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:11:44 smithi134 ceph-mon[35037]: pgmap v2007: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:11:44.931 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:11:44 smithi079 ceph-mon[35312]: pgmap v2007: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:11:45.031 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:11:44 smithi022 ceph-mon[31985]: pgmap v2007: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:11:45.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:11:45 smithi134 ceph-mon[35037]: 2 slow requests, 0 included below; oldest blocked for > 3291.891624 secs 2024-03-28T19:11:45.931 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:11:45 smithi079 ceph-mon[35312]: 2 slow requests, 0 included below; oldest blocked for > 3291.891624 secs 2024-03-28T19:11:46.031 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:11:45 smithi022 ceph-mon[31985]: 2 slow requests, 0 included below; oldest blocked for > 3291.891624 secs 2024-03-28T19:11:46.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:11:46 smithi134 ceph-mon[35037]: pgmap v2008: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T19:11:46.931 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:11:46 smithi079 ceph-mon[35312]: pgmap v2008: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T19:11:47.031 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:11:46 smithi022 ceph-mon[31985]: pgmap v2008: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T19:11:48.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:11:48 smithi134 ceph-mon[35037]: pgmap v2009: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:11:48.931 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:11:48 smithi079 ceph-mon[35312]: pgmap v2009: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:11:49.031 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:11:48 smithi022 ceph-mon[31985]: pgmap v2009: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:11:50.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:11:50 smithi134 ceph-mon[35037]: 2 slow requests, 0 included below; oldest blocked for > 3296.891687 secs 2024-03-28T19:11:50.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:11:50 smithi134 ceph-mon[35037]: pgmap v2010: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T19:11:50.931 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:11:50 smithi079 ceph-mon[35312]: 2 slow requests, 0 included below; oldest blocked for > 3296.891687 secs 2024-03-28T19:11:50.931 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:11:50 smithi079 ceph-mon[35312]: pgmap v2010: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T19:11:51.031 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:11:50 smithi022 ceph-mon[31985]: 2 slow requests, 0 included below; oldest blocked for > 3296.891687 secs 2024-03-28T19:11:51.031 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:11:50 smithi022 ceph-mon[31985]: pgmap v2010: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T19:11:52.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:11:52 smithi134 ceph-mon[35037]: pgmap v2011: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:11:52.931 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:11:52 smithi079 ceph-mon[35312]: pgmap v2011: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:11:53.031 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:11:52 smithi022 ceph-mon[31985]: pgmap v2011: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:11:54.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:11:54 smithi134 ceph-mon[35037]: pgmap v2012: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:11:54.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:11:54 smithi079 ceph-mon[35312]: pgmap v2012: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:11:55.031 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:11:54 smithi022 ceph-mon[31985]: pgmap v2012: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:11:55.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:11:55 smithi134 ceph-mon[35037]: 2 slow requests, 0 included below; oldest blocked for > 3301.891751 secs 2024-03-28T19:11:55.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:11:55 smithi079 ceph-mon[35312]: 2 slow requests, 0 included below; oldest blocked for > 3301.891751 secs 2024-03-28T19:11:56.031 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:11:55 smithi022 ceph-mon[31985]: 2 slow requests, 0 included below; oldest blocked for > 3301.891751 secs 2024-03-28T19:11:56.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:11:56 smithi134 ceph-mon[35037]: pgmap v2013: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T19:11:56.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:11:56 smithi079 ceph-mon[35312]: pgmap v2013: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T19:11:57.031 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:11:56 smithi022 ceph-mon[31985]: pgmap v2013: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T19:11:58.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:11:58 smithi134 ceph-mon[35037]: pgmap v2014: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:11:58.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:11:58 smithi079 ceph-mon[35312]: pgmap v2014: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:11:59.031 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:11:58 smithi022 ceph-mon[31985]: pgmap v2014: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:12:00.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:12:00 smithi134 ceph-mon[35037]: 2 slow requests, 0 included below; oldest blocked for > 3306.891818 secs 2024-03-28T19:12:00.925 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:12:00 smithi134 ceph-mon[35037]: pgmap v2015: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T19:12:00.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:12:00 smithi079 ceph-mon[35312]: 2 slow requests, 0 included below; oldest blocked for > 3306.891818 secs 2024-03-28T19:12:00.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:12:00 smithi079 ceph-mon[35312]: pgmap v2015: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T19:12:01.031 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:12:00 smithi022 ceph-mon[31985]: 2 slow requests, 0 included below; oldest blocked for > 3306.891818 secs 2024-03-28T19:12:01.031 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:12:00 smithi022 ceph-mon[31985]: pgmap v2015: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T19:12:02.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:12:02 smithi134 ceph-mon[35037]: pgmap v2016: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:12:02.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:12:02 smithi079 ceph-mon[35312]: pgmap v2016: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:12:03.031 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:12:02 smithi022 ceph-mon[31985]: pgmap v2016: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:12:04.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:12:04 smithi134 ceph-mon[35037]: pgmap v2017: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:12:04.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:12:04 smithi079 ceph-mon[35312]: pgmap v2017: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:12:05.031 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:12:04 smithi022 ceph-mon[31985]: pgmap v2017: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:12:05.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:12:05 smithi079 ceph-mon[35312]: 2 slow requests, 0 included below; oldest blocked for > 3311.891917 secs 2024-03-28T19:12:06.031 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:12:05 smithi022 ceph-mon[31985]: 2 slow requests, 0 included below; oldest blocked for > 3311.891917 secs 2024-03-28T19:12:06.174 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:12:05 smithi134 ceph-mon[35037]: 2 slow requests, 0 included below; oldest blocked for > 3311.891917 secs 2024-03-28T19:12:07.031 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:12:06 smithi022 ceph-mon[31985]: pgmap v2018: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T19:12:07.174 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:12:06 smithi134 ceph-mon[35037]: pgmap v2018: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T19:12:07.181 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:12:06 smithi079 ceph-mon[35312]: pgmap v2018: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T19:12:09.031 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:12:08 smithi022 ceph-mon[31985]: pgmap v2019: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:12:09.174 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:12:08 smithi134 ceph-mon[35037]: pgmap v2019: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:12:09.181 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:12:08 smithi079 ceph-mon[35312]: pgmap v2019: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:12:11.031 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:12:10 smithi022 ceph-mon[31985]: 2 slow requests, 0 included below; oldest blocked for > 3316.891988 secs 2024-03-28T19:12:11.031 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:12:10 smithi022 ceph-mon[31985]: pgmap v2020: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T19:12:11.174 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:12:10 smithi134 ceph-mon[35037]: 2 slow requests, 0 included below; oldest blocked for > 3316.891988 secs 2024-03-28T19:12:11.174 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:12:10 smithi134 ceph-mon[35037]: pgmap v2020: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T19:12:11.182 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:12:10 smithi079 ceph-mon[35312]: 2 slow requests, 0 included below; oldest blocked for > 3316.891988 secs 2024-03-28T19:12:11.182 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:12:10 smithi079 ceph-mon[35312]: pgmap v2020: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T19:12:12.031 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:12:11 smithi022 ceph-mon[31985]: pgmap v2021: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:12:12.174 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:12:11 smithi134 ceph-mon[35037]: pgmap v2021: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:12:12.181 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:12:11 smithi079 ceph-mon[35312]: pgmap v2021: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:12:14.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:12:14 smithi134 ceph-mon[35037]: pgmap v2022: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:12:14.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:12:14 smithi079 ceph-mon[35312]: pgmap v2022: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:12:15.031 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:12:14 smithi022 ceph-mon[31985]: pgmap v2022: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:12:15.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:12:15 smithi134 ceph-mon[35037]: 2 slow requests, 0 included below; oldest blocked for > 3321.892011 secs 2024-03-28T19:12:15.931 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:12:15 smithi079 ceph-mon[35312]: 2 slow requests, 0 included below; oldest blocked for > 3321.892011 secs 2024-03-28T19:12:16.031 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:12:15 smithi022 ceph-mon[31985]: 2 slow requests, 0 included below; oldest blocked for > 3321.892011 secs 2024-03-28T19:12:16.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:12:16 smithi134 ceph-mon[35037]: pgmap v2023: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T19:12:16.931 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:12:16 smithi079 ceph-mon[35312]: pgmap v2023: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T19:12:17.031 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:12:16 smithi022 ceph-mon[31985]: pgmap v2023: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T19:12:18.834 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:12:18 smithi022 ceph-mon[31985]: pgmap v2024: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:12:18.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:12:18 smithi134 ceph-mon[35037]: pgmap v2024: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:12:18.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:12:18 smithi079 ceph-mon[35312]: pgmap v2024: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:12:20.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:12:20 smithi134 ceph-mon[35037]: 2 slow requests, 0 included below; oldest blocked for > 3326.892087 secs 2024-03-28T19:12:20.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:12:20 smithi134 ceph-mon[35037]: pgmap v2025: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T19:12:20.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:12:20 smithi079 ceph-mon[35312]: 2 slow requests, 0 included below; oldest blocked for > 3326.892087 secs 2024-03-28T19:12:20.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:12:20 smithi079 ceph-mon[35312]: pgmap v2025: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T19:12:21.031 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:12:20 smithi022 ceph-mon[31985]: 2 slow requests, 0 included below; oldest blocked for > 3326.892087 secs 2024-03-28T19:12:21.031 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:12:20 smithi022 ceph-mon[31985]: pgmap v2025: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T19:12:22.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:12:22 smithi134 ceph-mon[35037]: pgmap v2026: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:12:22.931 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:12:22 smithi079 ceph-mon[35312]: pgmap v2026: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:12:23.031 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:12:22 smithi022 ceph-mon[31985]: pgmap v2026: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:12:24.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:12:24 smithi134 ceph-mon[35037]: pgmap v2027: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:12:24.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:12:24 smithi079 ceph-mon[35312]: pgmap v2027: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:12:25.031 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:12:24 smithi022 ceph-mon[31985]: pgmap v2027: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:12:25.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:12:25 smithi134 ceph-mon[35037]: 2 slow requests, 0 included below; oldest blocked for > 3331.892170 secs 2024-03-28T19:12:25.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:12:25 smithi079 ceph-mon[35312]: 2 slow requests, 0 included below; oldest blocked for > 3331.892170 secs 2024-03-28T19:12:26.031 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:12:25 smithi022 ceph-mon[31985]: 2 slow requests, 0 included below; oldest blocked for > 3331.892170 secs 2024-03-28T19:12:26.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:12:26 smithi134 ceph-mon[35037]: pgmap v2028: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T19:12:26.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:12:26 smithi134 ceph-mon[35037]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T19:12:26.931 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:12:26 smithi079 ceph-mon[35312]: pgmap v2028: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T19:12:26.931 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:12:26 smithi079 ceph-mon[35312]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T19:12:27.031 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:12:26 smithi022 ceph-mon[31985]: pgmap v2028: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T19:12:27.031 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:12:26 smithi022 ceph-mon[31985]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T19:12:27.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:12:27 smithi134 ceph-mon[35037]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T19:12:27.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:12:27 smithi134 ceph-mon[35037]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T19:12:27.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:12:27 smithi134 ceph-mon[35037]: from='mgr.15522 ' entity='mgr.x' 2024-03-28T19:12:27.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:12:27 smithi079 ceph-mon[35312]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T19:12:27.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:12:27 smithi079 ceph-mon[35312]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T19:12:27.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:12:27 smithi079 ceph-mon[35312]: from='mgr.15522 ' entity='mgr.x' 2024-03-28T19:12:28.031 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:12:27 smithi022 ceph-mon[31985]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T19:12:28.031 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:12:27 smithi022 ceph-mon[31985]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T19:12:28.031 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:12:27 smithi022 ceph-mon[31985]: from='mgr.15522 ' entity='mgr.x' 2024-03-28T19:12:28.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:12:28 smithi134 ceph-mon[35037]: pgmap v2029: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 6 op/s 2024-03-28T19:12:28.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:12:28 smithi134 ceph-mon[35037]: pgmap v2030: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-03-28T19:12:28.931 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:12:28 smithi079 ceph-mon[35312]: pgmap v2029: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 6 op/s 2024-03-28T19:12:28.931 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:12:28 smithi079 ceph-mon[35312]: pgmap v2030: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-03-28T19:12:29.031 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:12:28 smithi022 ceph-mon[31985]: pgmap v2029: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 6 op/s 2024-03-28T19:12:29.031 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:12:28 smithi022 ceph-mon[31985]: pgmap v2030: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-03-28T19:12:30.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:12:30 smithi134 ceph-mon[35037]: pgmap v2031: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-03-28T19:12:30.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:12:30 smithi134 ceph-mon[35037]: 2 slow requests, 0 included below; oldest blocked for > 3336.892253 secs 2024-03-28T19:12:30.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:12:30 smithi079 ceph-mon[35312]: pgmap v2031: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-03-28T19:12:30.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:12:30 smithi079 ceph-mon[35312]: 2 slow requests, 0 included below; oldest blocked for > 3336.892253 secs 2024-03-28T19:12:31.031 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:12:30 smithi022 ceph-mon[31985]: pgmap v2031: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-03-28T19:12:31.031 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:12:30 smithi022 ceph-mon[31985]: 2 slow requests, 0 included below; oldest blocked for > 3336.892253 secs 2024-03-28T19:12:32.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:12:32 smithi134 ceph-mon[35037]: pgmap v2032: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-03-28T19:12:32.931 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:12:32 smithi079 ceph-mon[35312]: pgmap v2032: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-03-28T19:12:33.031 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:12:32 smithi022 ceph-mon[31985]: pgmap v2032: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-03-28T19:12:34.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:12:34 smithi134 ceph-mon[35037]: pgmap v2033: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-03-28T19:12:34.931 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:12:34 smithi079 ceph-mon[35312]: pgmap v2033: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-03-28T19:12:35.031 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:12:34 smithi022 ceph-mon[31985]: pgmap v2033: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-03-28T19:12:35.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:12:35 smithi134 ceph-mon[35037]: 2 slow requests, 0 included below; oldest blocked for > 3341.892283 secs 2024-03-28T19:12:35.931 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:12:35 smithi079 ceph-mon[35312]: 2 slow requests, 0 included below; oldest blocked for > 3341.892283 secs 2024-03-28T19:12:36.031 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:12:35 smithi022 ceph-mon[31985]: 2 slow requests, 0 included below; oldest blocked for > 3341.892283 secs 2024-03-28T19:12:36.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:12:36 smithi134 ceph-mon[35037]: pgmap v2034: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-03-28T19:12:36.931 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:12:36 smithi079 ceph-mon[35312]: pgmap v2034: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-03-28T19:12:37.031 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:12:36 smithi022 ceph-mon[31985]: pgmap v2034: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-03-28T19:12:38.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:12:38 smithi134 ceph-mon[35037]: pgmap v2035: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 7 op/s 2024-03-28T19:12:38.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:12:38 smithi079 ceph-mon[35312]: pgmap v2035: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 7 op/s 2024-03-28T19:12:39.031 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:12:38 smithi022 ceph-mon[31985]: pgmap v2035: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 7 op/s 2024-03-28T19:12:40.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:12:40 smithi134 ceph-mon[35037]: 2 slow requests, 0 included below; oldest blocked for > 3346.892391 secs 2024-03-28T19:12:40.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:12:40 smithi134 ceph-mon[35037]: pgmap v2036: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T19:12:40.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:12:40 smithi079 ceph-mon[35312]: 2 slow requests, 0 included below; oldest blocked for > 3346.892391 secs 2024-03-28T19:12:40.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:12:40 smithi079 ceph-mon[35312]: pgmap v2036: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T19:12:41.031 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:12:40 smithi022 ceph-mon[31985]: 2 slow requests, 0 included below; oldest blocked for > 3346.892391 secs 2024-03-28T19:12:41.031 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:12:40 smithi022 ceph-mon[31985]: pgmap v2036: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T19:12:42.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:12:42 smithi134 ceph-mon[35037]: pgmap v2037: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:12:42.931 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:12:42 smithi079 ceph-mon[35312]: pgmap v2037: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:12:43.031 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:12:42 smithi022 ceph-mon[31985]: pgmap v2037: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:12:44.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:12:44 smithi079 ceph-mon[35312]: pgmap v2038: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:12:45.031 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:12:44 smithi022 ceph-mon[31985]: pgmap v2038: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:12:45.174 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:12:44 smithi134 ceph-mon[35037]: pgmap v2038: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:12:46.031 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:12:45 smithi022 ceph-mon[31985]: 2 slow requests, 0 included below; oldest blocked for > 3351.892508 secs 2024-03-28T19:12:46.174 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:12:45 smithi134 ceph-mon[35037]: 2 slow requests, 0 included below; oldest blocked for > 3351.892508 secs 2024-03-28T19:12:46.182 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:12:45 smithi079 ceph-mon[35312]: 2 slow requests, 0 included below; oldest blocked for > 3351.892508 secs 2024-03-28T19:12:47.031 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:12:46 smithi022 ceph-mon[31985]: pgmap v2039: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T19:12:47.174 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:12:46 smithi134 ceph-mon[35037]: pgmap v2039: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T19:12:47.181 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:12:46 smithi079 ceph-mon[35312]: pgmap v2039: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T19:12:49.031 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:12:48 smithi022 ceph-mon[31985]: pgmap v2040: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:12:49.174 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:12:48 smithi134 ceph-mon[35037]: pgmap v2040: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:12:49.181 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:12:48 smithi079 ceph-mon[35312]: pgmap v2040: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:12:50.031 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:12:49 smithi022 ceph-mon[31985]: pgmap v2041: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T19:12:50.075 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:12:49 smithi134 ceph-mon[35037]: pgmap v2041: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T19:12:50.181 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:12:49 smithi079 ceph-mon[35312]: pgmap v2041: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T19:12:51.031 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:12:50 smithi022 ceph-mon[31985]: 2 slow requests, 0 included below; oldest blocked for > 3356.892627 secs 2024-03-28T19:12:51.174 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:12:50 smithi134 ceph-mon[35037]: 2 slow requests, 0 included below; oldest blocked for > 3356.892627 secs 2024-03-28T19:12:51.181 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:12:50 smithi079 ceph-mon[35312]: 2 slow requests, 0 included below; oldest blocked for > 3356.892627 secs 2024-03-28T19:12:52.031 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:12:51 smithi022 ceph-mon[31985]: pgmap v2042: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:12:52.174 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:12:51 smithi134 ceph-mon[35037]: pgmap v2042: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:12:52.181 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:12:51 smithi079 ceph-mon[35312]: pgmap v2042: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:12:54.674 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:12:54 smithi134 ceph-mon[35037]: pgmap v2043: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:12:54.681 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:12:54 smithi079 ceph-mon[35312]: pgmap v2043: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:12:54.781 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:12:54 smithi022 ceph-mon[31985]: pgmap v2043: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:12:55.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:12:55 smithi079 ceph-mon[35312]: 2 slow requests, 0 included below; oldest blocked for > 3361.892691 secs 2024-03-28T19:12:55.781 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:12:55 smithi022 ceph-mon[31985]: 2 slow requests, 0 included below; oldest blocked for > 3361.892691 secs 2024-03-28T19:12:55.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:12:55 smithi134 ceph-mon[35037]: 2 slow requests, 0 included below; oldest blocked for > 3361.892691 secs 2024-03-28T19:12:56.681 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:12:56 smithi079 ceph-mon[35312]: pgmap v2044: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T19:12:56.781 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:12:56 smithi022 ceph-mon[31985]: pgmap v2044: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T19:12:56.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:12:56 smithi134 ceph-mon[35037]: pgmap v2044: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T19:12:58.781 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:12:58 smithi022 ceph-mon[31985]: pgmap v2045: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:12:58.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:12:58 smithi134 ceph-mon[35037]: pgmap v2045: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:12:58.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:12:58 smithi079 ceph-mon[35312]: pgmap v2045: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:13:00.735 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:13:00 smithi079 ceph-mon[35312]: 2 slow requests, 0 included below; oldest blocked for > 3366.892815 secs 2024-03-28T19:13:00.735 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:13:00 smithi079 ceph-mon[35312]: pgmap v2046: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T19:13:00.781 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:13:00 smithi022 ceph-mon[31985]: 2 slow requests, 0 included below; oldest blocked for > 3366.892815 secs 2024-03-28T19:13:00.781 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:13:00 smithi022 ceph-mon[31985]: pgmap v2046: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T19:13:00.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:13:00 smithi134 ceph-mon[35037]: 2 slow requests, 0 included below; oldest blocked for > 3366.892815 secs 2024-03-28T19:13:00.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:13:00 smithi134 ceph-mon[35037]: pgmap v2046: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T19:13:02.781 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:13:02 smithi022 ceph-mon[31985]: pgmap v2047: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:13:02.923 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:13:02 smithi134 ceph-mon[35037]: pgmap v2047: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:13:02.931 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:13:02 smithi079 ceph-mon[35312]: pgmap v2047: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:13:04.781 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:13:04 smithi022 ceph-mon[31985]: pgmap v2048: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:13:04.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:13:04 smithi134 ceph-mon[35037]: pgmap v2048: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:13:04.931 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:13:04 smithi079 ceph-mon[35312]: pgmap v2048: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:13:05.781 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:13:05 smithi022 ceph-mon[31985]: 2 slow requests, 0 included below; oldest blocked for > 3371.892945 secs 2024-03-28T19:13:05.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:13:05 smithi134 ceph-mon[35037]: 2 slow requests, 0 included below; oldest blocked for > 3371.892945 secs 2024-03-28T19:13:05.931 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:13:05 smithi079 ceph-mon[35312]: 2 slow requests, 0 included below; oldest blocked for > 3371.892945 secs 2024-03-28T19:13:06.781 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:13:06 smithi022 ceph-mon[31985]: pgmap v2049: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T19:13:06.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:13:06 smithi134 ceph-mon[35037]: pgmap v2049: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T19:13:06.931 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:13:06 smithi079 ceph-mon[35312]: pgmap v2049: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T19:13:08.781 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:13:08 smithi022 ceph-mon[31985]: pgmap v2050: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:13:08.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:13:08 smithi134 ceph-mon[35037]: pgmap v2050: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:13:08.931 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:13:08 smithi079 ceph-mon[35312]: pgmap v2050: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:13:10.781 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:13:10 smithi022 ceph-mon[31985]: 2 slow requests, 0 included below; oldest blocked for > 3376.893012 secs 2024-03-28T19:13:10.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:13:10 smithi022 ceph-mon[31985]: pgmap v2051: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T19:13:10.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:13:10 smithi134 ceph-mon[35037]: 2 slow requests, 0 included below; oldest blocked for > 3376.893012 secs 2024-03-28T19:13:10.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:13:10 smithi134 ceph-mon[35037]: pgmap v2051: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T19:13:10.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:13:10 smithi079 ceph-mon[35312]: 2 slow requests, 0 included below; oldest blocked for > 3376.893012 secs 2024-03-28T19:13:10.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:13:10 smithi079 ceph-mon[35312]: pgmap v2051: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T19:13:12.781 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:13:12 smithi022 ceph-mon[31985]: pgmap v2052: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:13:12.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:13:12 smithi134 ceph-mon[35037]: pgmap v2052: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:13:12.931 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:13:12 smithi079 ceph-mon[35312]: pgmap v2052: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:13:14.781 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:13:14 smithi022 ceph-mon[31985]: pgmap v2053: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:13:14.923 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:13:14 smithi134 ceph-mon[35037]: pgmap v2053: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:13:14.931 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:13:14 smithi079 ceph-mon[35312]: pgmap v2053: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:13:15.781 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:13:15 smithi022 ceph-mon[31985]: 2 slow requests, 0 included below; oldest blocked for > 3381.893075 secs 2024-03-28T19:13:15.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:13:15 smithi134 ceph-mon[35037]: 2 slow requests, 0 included below; oldest blocked for > 3381.893075 secs 2024-03-28T19:13:15.931 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:13:15 smithi079 ceph-mon[35312]: 2 slow requests, 0 included below; oldest blocked for > 3381.893075 secs 2024-03-28T19:13:16.781 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:13:16 smithi022 ceph-mon[31985]: pgmap v2054: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T19:13:16.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:13:16 smithi134 ceph-mon[35037]: pgmap v2054: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T19:13:16.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:13:16 smithi079 ceph-mon[35312]: pgmap v2054: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T19:13:18.781 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:13:18 smithi022 ceph-mon[31985]: pgmap v2055: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:13:18.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:13:18 smithi134 ceph-mon[35037]: pgmap v2055: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:13:18.931 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:13:18 smithi079 ceph-mon[35312]: pgmap v2055: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:13:20.781 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:13:20 smithi022 ceph-mon[31985]: 2 slow requests, 0 included below; oldest blocked for > 3386.893145 secs 2024-03-28T19:13:20.781 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:13:20 smithi022 ceph-mon[31985]: pgmap v2056: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T19:13:20.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:13:20 smithi134 ceph-mon[35037]: 2 slow requests, 0 included below; oldest blocked for > 3386.893145 secs 2024-03-28T19:13:20.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:13:20 smithi134 ceph-mon[35037]: pgmap v2056: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T19:13:20.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:13:20 smithi079 ceph-mon[35312]: 2 slow requests, 0 included below; oldest blocked for > 3386.893145 secs 2024-03-28T19:13:20.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:13:20 smithi079 ceph-mon[35312]: pgmap v2056: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T19:13:22.781 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:13:22 smithi022 ceph-mon[31985]: pgmap v2057: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:13:22.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:13:22 smithi134 ceph-mon[35037]: pgmap v2057: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:13:22.931 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:13:22 smithi079 ceph-mon[35312]: pgmap v2057: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:13:24.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:13:24 smithi134 ceph-mon[35037]: pgmap v2058: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:13:24.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:13:24 smithi079 ceph-mon[35312]: pgmap v2058: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:13:25.031 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:13:24 smithi022 ceph-mon[31985]: pgmap v2058: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:13:25.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:13:25 smithi134 ceph-mon[35037]: 2 slow requests, 0 included below; oldest blocked for > 3391.893208 secs 2024-03-28T19:13:25.931 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:13:25 smithi079 ceph-mon[35312]: 2 slow requests, 0 included below; oldest blocked for > 3391.893208 secs 2024-03-28T19:13:26.031 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:13:25 smithi022 ceph-mon[31985]: 2 slow requests, 0 included below; oldest blocked for > 3391.893208 secs 2024-03-28T19:13:26.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:13:26 smithi134 ceph-mon[35037]: pgmap v2059: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T19:13:26.931 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:13:26 smithi079 ceph-mon[35312]: pgmap v2059: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T19:13:27.031 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:13:26 smithi022 ceph-mon[31985]: pgmap v2059: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T19:13:28.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:13:28 smithi134 ceph-mon[35037]: pgmap v2060: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:13:28.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:13:28 smithi134 ceph-mon[35037]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T19:13:28.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:13:28 smithi079 ceph-mon[35312]: pgmap v2060: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:13:28.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:13:28 smithi079 ceph-mon[35312]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T19:13:29.031 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:13:28 smithi022 ceph-mon[31985]: pgmap v2060: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:13:29.031 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:13:28 smithi022 ceph-mon[31985]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T19:13:29.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:13:29 smithi134 ceph-mon[35037]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T19:13:29.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:13:29 smithi134 ceph-mon[35037]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T19:13:29.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:13:29 smithi134 ceph-mon[35037]: from='mgr.15522 ' entity='mgr.x' 2024-03-28T19:13:29.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:13:29 smithi079 ceph-mon[35312]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T19:13:29.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:13:29 smithi079 ceph-mon[35312]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T19:13:29.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:13:29 smithi079 ceph-mon[35312]: from='mgr.15522 ' entity='mgr.x' 2024-03-28T19:13:30.031 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:13:29 smithi022 ceph-mon[31985]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T19:13:30.031 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:13:29 smithi022 ceph-mon[31985]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T19:13:30.031 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:13:29 smithi022 ceph-mon[31985]: from='mgr.15522 ' entity='mgr.x' 2024-03-28T19:13:30.834 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:13:30 smithi079 ceph-mon[35312]: pgmap v2061: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 6 op/s 2024-03-28T19:13:30.834 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:13:30 smithi079 ceph-mon[35312]: pgmap v2062: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s rd, 4 op/s 2024-03-28T19:13:30.834 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:13:30 smithi079 ceph-mon[35312]: 2 slow requests, 0 included below; oldest blocked for > 3396.893275 secs 2024-03-28T19:13:30.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:13:30 smithi134 ceph-mon[35037]: pgmap v2061: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 6 op/s 2024-03-28T19:13:30.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:13:30 smithi134 ceph-mon[35037]: pgmap v2062: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s rd, 4 op/s 2024-03-28T19:13:30.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:13:30 smithi134 ceph-mon[35037]: 2 slow requests, 0 included below; oldest blocked for > 3396.893275 secs 2024-03-28T19:13:31.031 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:13:30 smithi022 ceph-mon[31985]: pgmap v2061: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 6 op/s 2024-03-28T19:13:31.031 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:13:30 smithi022 ceph-mon[31985]: pgmap v2062: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s rd, 4 op/s 2024-03-28T19:13:31.031 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:13:30 smithi022 ceph-mon[31985]: 2 slow requests, 0 included below; oldest blocked for > 3396.893275 secs 2024-03-28T19:13:32.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:13:32 smithi134 ceph-mon[35037]: pgmap v2063: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-03-28T19:13:32.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:13:32 smithi079 ceph-mon[35312]: pgmap v2063: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-03-28T19:13:33.031 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:13:32 smithi022 ceph-mon[31985]: pgmap v2063: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-03-28T19:13:34.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:13:34 smithi134 ceph-mon[35037]: pgmap v2064: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-03-28T19:13:34.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:13:34 smithi079 ceph-mon[35312]: pgmap v2064: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-03-28T19:13:35.031 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:13:34 smithi022 ceph-mon[31985]: pgmap v2064: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-03-28T19:13:35.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:13:35 smithi134 ceph-mon[35037]: 2 slow requests, 0 included below; oldest blocked for > 3401.893295 secs 2024-03-28T19:13:35.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:13:35 smithi079 ceph-mon[35312]: 2 slow requests, 0 included below; oldest blocked for > 3401.893295 secs 2024-03-28T19:13:36.031 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:13:35 smithi022 ceph-mon[31985]: 2 slow requests, 0 included below; oldest blocked for > 3401.893295 secs 2024-03-28T19:13:36.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:13:36 smithi134 ceph-mon[35037]: pgmap v2065: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-03-28T19:13:36.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:13:36 smithi079 ceph-mon[35312]: pgmap v2065: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-03-28T19:13:37.031 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:13:36 smithi022 ceph-mon[31985]: pgmap v2065: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-03-28T19:13:38.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:13:38 smithi134 ceph-mon[35037]: pgmap v2066: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-03-28T19:13:38.931 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:13:38 smithi079 ceph-mon[35312]: pgmap v2066: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-03-28T19:13:39.031 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:13:38 smithi022 ceph-mon[31985]: pgmap v2066: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-03-28T19:13:40.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:13:40 smithi134 ceph-mon[35037]: pgmap v2067: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 7 op/s 2024-03-28T19:13:40.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:13:40 smithi134 ceph-mon[35037]: 2 slow requests, 0 included below; oldest blocked for > 3406.893369 secs 2024-03-28T19:13:40.931 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:13:40 smithi079 ceph-mon[35312]: pgmap v2067: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 7 op/s 2024-03-28T19:13:40.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:13:40 smithi079 ceph-mon[35312]: 2 slow requests, 0 included below; oldest blocked for > 3406.893369 secs 2024-03-28T19:13:41.031 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:13:40 smithi022 ceph-mon[31985]: pgmap v2067: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 7 op/s 2024-03-28T19:13:41.031 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:13:40 smithi022 ceph-mon[31985]: 2 slow requests, 0 included below; oldest blocked for > 3406.893369 secs 2024-03-28T19:13:42.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:13:42 smithi134 ceph-mon[35037]: pgmap v2068: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T19:13:42.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:13:42 smithi079 ceph-mon[35312]: pgmap v2068: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T19:13:43.031 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:13:42 smithi022 ceph-mon[31985]: pgmap v2068: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T19:13:44.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:13:44 smithi134 ceph-mon[35037]: pgmap v2069: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:13:44.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:13:44 smithi079 ceph-mon[35312]: pgmap v2069: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:13:45.031 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:13:44 smithi022 ceph-mon[31985]: pgmap v2069: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:13:45.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:13:45 smithi134 ceph-mon[35037]: 2 slow requests, 0 included below; oldest blocked for > 3411.893487 secs 2024-03-28T19:13:45.931 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:13:45 smithi079 ceph-mon[35312]: 2 slow requests, 0 included below; oldest blocked for > 3411.893487 secs 2024-03-28T19:13:46.031 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:13:45 smithi022 ceph-mon[31985]: 2 slow requests, 0 included below; oldest blocked for > 3411.893487 secs 2024-03-28T19:13:46.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:13:46 smithi134 ceph-mon[35037]: pgmap v2070: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T19:13:46.931 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:13:46 smithi079 ceph-mon[35312]: pgmap v2070: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T19:13:47.031 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:13:46 smithi022 ceph-mon[31985]: pgmap v2070: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T19:13:48.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:13:48 smithi134 ceph-mon[35037]: pgmap v2071: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:13:48.931 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:13:48 smithi079 ceph-mon[35312]: pgmap v2071: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:13:49.031 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:13:48 smithi022 ceph-mon[31985]: pgmap v2071: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:13:50.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:13:50 smithi134 ceph-mon[35037]: pgmap v2072: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:13:50.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:13:50 smithi134 ceph-mon[35037]: 2 slow requests, 0 included below; oldest blocked for > 3416.893551 secs 2024-03-28T19:13:50.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:13:50 smithi079 ceph-mon[35312]: pgmap v2072: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:13:50.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:13:50 smithi079 ceph-mon[35312]: 2 slow requests, 0 included below; oldest blocked for > 3416.893551 secs 2024-03-28T19:13:51.031 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:13:50 smithi022 ceph-mon[31985]: pgmap v2072: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:13:51.031 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:13:50 smithi022 ceph-mon[31985]: 2 slow requests, 0 included below; oldest blocked for > 3416.893551 secs 2024-03-28T19:13:52.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:13:52 smithi134 ceph-mon[35037]: pgmap v2073: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T19:13:52.931 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:13:52 smithi079 ceph-mon[35312]: pgmap v2073: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T19:13:53.031 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:13:52 smithi022 ceph-mon[31985]: pgmap v2073: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T19:13:54.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:13:54 smithi134 ceph-mon[35037]: pgmap v2074: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:13:54.931 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:13:54 smithi079 ceph-mon[35312]: pgmap v2074: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:13:55.031 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:13:54 smithi022 ceph-mon[31985]: pgmap v2074: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:13:55.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:13:55 smithi134 ceph-mon[35037]: 2 slow requests, 0 included below; oldest blocked for > 3421.893674 secs 2024-03-28T19:13:55.931 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:13:55 smithi079 ceph-mon[35312]: 2 slow requests, 0 included below; oldest blocked for > 3421.893674 secs 2024-03-28T19:13:56.031 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:13:55 smithi022 ceph-mon[31985]: 2 slow requests, 0 included below; oldest blocked for > 3421.893674 secs 2024-03-28T19:13:56.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:13:56 smithi079 ceph-mon[35312]: pgmap v2075: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T19:13:57.031 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:13:56 smithi022 ceph-mon[31985]: pgmap v2075: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T19:13:57.174 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:13:56 smithi134 ceph-mon[35037]: pgmap v2075: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T19:13:58.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:13:58 smithi079 ceph-mon[35312]: pgmap v2076: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:13:59.031 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:13:58 smithi022 ceph-mon[31985]: pgmap v2076: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:13:59.174 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:13:58 smithi134 ceph-mon[35037]: pgmap v2076: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:14:01.031 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:14:00 smithi022 ceph-mon[31985]: pgmap v2077: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:14:01.031 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:14:00 smithi022 ceph-mon[31985]: 2 slow requests, 0 included below; oldest blocked for > 3426.893698 secs 2024-03-28T19:14:01.174 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:14:00 smithi134 ceph-mon[35037]: pgmap v2077: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:14:01.174 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:14:00 smithi134 ceph-mon[35037]: 2 slow requests, 0 included below; oldest blocked for > 3426.893698 secs 2024-03-28T19:14:01.182 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:14:00 smithi079 ceph-mon[35312]: pgmap v2077: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:14:01.182 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:14:00 smithi079 ceph-mon[35312]: 2 slow requests, 0 included below; oldest blocked for > 3426.893698 secs 2024-03-28T19:14:03.031 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:14:02 smithi022 ceph-mon[31985]: pgmap v2078: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T19:14:03.174 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:14:02 smithi134 ceph-mon[35037]: pgmap v2078: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T19:14:03.181 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:14:02 smithi079 ceph-mon[35312]: pgmap v2078: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T19:14:05.031 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:14:04 smithi022 ceph-mon[31985]: pgmap v2079: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:14:05.174 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:14:04 smithi134 ceph-mon[35037]: pgmap v2079: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:14:05.181 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:14:04 smithi079 ceph-mon[35312]: pgmap v2079: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:14:06.031 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:14:05 smithi022 ceph-mon[31985]: 2 slow requests, 0 included below; oldest blocked for > 3431.893774 secs 2024-03-28T19:14:06.174 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:14:05 smithi134 ceph-mon[35037]: 2 slow requests, 0 included below; oldest blocked for > 3431.893774 secs 2024-03-28T19:14:06.182 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:14:05 smithi079 ceph-mon[35312]: 2 slow requests, 0 included below; oldest blocked for > 3431.893774 secs 2024-03-28T19:14:07.031 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:14:06 smithi022 ceph-mon[31985]: pgmap v2080: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T19:14:07.174 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:14:06 smithi134 ceph-mon[35037]: pgmap v2080: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T19:14:07.182 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:14:06 smithi079 ceph-mon[35312]: pgmap v2080: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T19:14:09.031 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:14:08 smithi022 ceph-mon[31985]: pgmap v2081: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:14:09.174 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:14:08 smithi134 ceph-mon[35037]: pgmap v2081: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:14:09.182 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:14:08 smithi079 ceph-mon[35312]: pgmap v2081: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:14:11.031 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:14:10 smithi022 ceph-mon[31985]: pgmap v2082: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:14:11.032 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:14:10 smithi022 ceph-mon[31985]: 2 slow requests, 0 included below; oldest blocked for > 3436.893801 secs 2024-03-28T19:14:11.174 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:14:10 smithi134 ceph-mon[35037]: pgmap v2082: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:14:11.174 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:14:10 smithi134 ceph-mon[35037]: 2 slow requests, 0 included below; oldest blocked for > 3436.893801 secs 2024-03-28T19:14:11.181 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:14:10 smithi079 ceph-mon[35312]: pgmap v2082: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:14:11.182 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:14:10 smithi079 ceph-mon[35312]: 2 slow requests, 0 included below; oldest blocked for > 3436.893801 secs 2024-03-28T19:14:12.031 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:14:11 smithi022 ceph-mon[31985]: pgmap v2083: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T19:14:12.174 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:14:11 smithi134 ceph-mon[35037]: pgmap v2083: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T19:14:12.182 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:14:11 smithi079 ceph-mon[35312]: pgmap v2083: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T19:14:14.424 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:14:14 smithi134 ceph-mon[35037]: pgmap v2084: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:14:14.432 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:14:14 smithi079 ceph-mon[35312]: pgmap v2084: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:14:14.531 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:14:14 smithi022 ceph-mon[31985]: pgmap v2084: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:14:15.424 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:14:15 smithi134 ceph-mon[35037]: 2 slow requests, 0 included below; oldest blocked for > 3441.893868 secs 2024-03-28T19:14:15.431 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:14:15 smithi079 ceph-mon[35312]: 2 slow requests, 0 included below; oldest blocked for > 3441.893868 secs 2024-03-28T19:14:15.531 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:14:15 smithi022 ceph-mon[31985]: 2 slow requests, 0 included below; oldest blocked for > 3441.893868 secs 2024-03-28T19:14:16.424 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:14:16 smithi134 ceph-mon[35037]: pgmap v2085: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T19:14:16.431 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:14:16 smithi079 ceph-mon[35312]: pgmap v2085: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T19:14:16.531 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:14:16 smithi022 ceph-mon[31985]: pgmap v2085: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T19:14:18.424 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:14:18 smithi134 ceph-mon[35037]: pgmap v2086: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:14:18.431 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:14:18 smithi079 ceph-mon[35312]: pgmap v2086: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:14:18.531 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:14:18 smithi022 ceph-mon[31985]: pgmap v2086: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:14:20.432 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:14:20 smithi079 ceph-mon[35312]: pgmap v2087: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:14:20.432 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:14:20 smithi079 ceph-mon[35312]: 2 slow requests, 0 included below; oldest blocked for > 3446.893991 secs 2024-03-28T19:14:20.531 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:14:20 smithi022 ceph-mon[31985]: pgmap v2087: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:14:20.531 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:14:20 smithi022 ceph-mon[31985]: 2 slow requests, 0 included below; oldest blocked for > 3446.893991 secs 2024-03-28T19:14:20.674 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:14:20 smithi134 ceph-mon[35037]: pgmap v2087: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:14:20.674 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:14:20 smithi134 ceph-mon[35037]: 2 slow requests, 0 included below; oldest blocked for > 3446.893991 secs 2024-03-28T19:14:22.431 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:14:22 smithi079 ceph-mon[35312]: pgmap v2088: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T19:14:22.531 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:14:22 smithi022 ceph-mon[31985]: pgmap v2088: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T19:14:22.674 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:14:22 smithi134 ceph-mon[35037]: pgmap v2088: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T19:14:24.531 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:14:24 smithi022 ceph-mon[31985]: pgmap v2089: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:14:24.674 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:14:24 smithi134 ceph-mon[35037]: pgmap v2089: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:14:24.681 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:14:24 smithi079 ceph-mon[35312]: pgmap v2089: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:14:25.531 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:14:25 smithi022 ceph-mon[31985]: 2 slow requests, 0 included below; oldest blocked for > 3451.894057 secs 2024-03-28T19:14:25.674 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:14:25 smithi134 ceph-mon[35037]: 2 slow requests, 0 included below; oldest blocked for > 3451.894057 secs 2024-03-28T19:14:25.681 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:14:25 smithi079 ceph-mon[35312]: 2 slow requests, 0 included below; oldest blocked for > 3451.894057 secs 2024-03-28T19:14:26.531 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:14:26 smithi022 ceph-mon[31985]: pgmap v2090: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T19:14:26.674 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:14:26 smithi134 ceph-mon[35037]: pgmap v2090: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T19:14:26.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:14:26 smithi079 ceph-mon[35312]: pgmap v2090: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T19:14:28.531 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:14:28 smithi022 ceph-mon[31985]: pgmap v2091: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:14:28.674 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:14:28 smithi134 ceph-mon[35037]: pgmap v2091: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:14:28.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:14:28 smithi079 ceph-mon[35312]: pgmap v2091: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:14:30.484 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:14:30 smithi079 ceph-mon[35312]: pgmap v2092: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:14:30.484 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:14:30 smithi079 ceph-mon[35312]: 2 slow requests, 0 included below; oldest blocked for > 3456.894139 secs 2024-03-28T19:14:30.484 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:14:30 smithi079 ceph-mon[35312]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T19:14:30.531 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:14:30 smithi022 ceph-mon[31985]: pgmap v2092: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:14:30.531 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:14:30 smithi022 ceph-mon[31985]: 2 slow requests, 0 included below; oldest blocked for > 3456.894139 secs 2024-03-28T19:14:30.531 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:14:30 smithi022 ceph-mon[31985]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T19:14:30.674 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:14:30 smithi134 ceph-mon[35037]: pgmap v2092: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:14:30.674 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:14:30 smithi134 ceph-mon[35037]: 2 slow requests, 0 included below; oldest blocked for > 3456.894139 secs 2024-03-28T19:14:30.674 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:14:30 smithi134 ceph-mon[35037]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T19:14:31.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:14:31 smithi022 ceph-mon[31985]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T19:14:31.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:14:31 smithi022 ceph-mon[31985]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T19:14:31.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:14:31 smithi022 ceph-mon[31985]: from='mgr.15522 ' entity='mgr.x' 2024-03-28T19:14:31.674 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:14:31 smithi134 ceph-mon[35037]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T19:14:31.674 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:14:31 smithi134 ceph-mon[35037]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T19:14:31.674 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:14:31 smithi134 ceph-mon[35037]: from='mgr.15522 ' entity='mgr.x' 2024-03-28T19:14:31.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:14:31 smithi079 ceph-mon[35312]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T19:14:31.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:14:31 smithi079 ceph-mon[35312]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T19:14:31.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:14:31 smithi079 ceph-mon[35312]: from='mgr.15522 ' entity='mgr.x' 2024-03-28T19:14:32.531 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:14:32 smithi022 ceph-mon[31985]: pgmap v2093: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 10 op/s 2024-03-28T19:14:32.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:14:32 smithi022 ceph-mon[31985]: pgmap v2094: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-03-28T19:14:32.674 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:14:32 smithi134 ceph-mon[35037]: pgmap v2093: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 10 op/s 2024-03-28T19:14:32.674 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:14:32 smithi134 ceph-mon[35037]: pgmap v2094: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-03-28T19:14:32.681 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:14:32 smithi079 ceph-mon[35312]: pgmap v2093: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 10 op/s 2024-03-28T19:14:32.681 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:14:32 smithi079 ceph-mon[35312]: pgmap v2094: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-03-28T19:14:34.531 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:14:34 smithi022 ceph-mon[31985]: pgmap v2095: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-03-28T19:14:34.674 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:14:34 smithi134 ceph-mon[35037]: pgmap v2095: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-03-28T19:14:34.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:14:34 smithi079 ceph-mon[35312]: pgmap v2095: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-03-28T19:14:35.531 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:14:35 smithi022 ceph-mon[31985]: 2 slow requests, 0 included below; oldest blocked for > 3461.894233 secs 2024-03-28T19:14:35.674 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:14:35 smithi134 ceph-mon[35037]: 2 slow requests, 0 included below; oldest blocked for > 3461.894233 secs 2024-03-28T19:14:35.681 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:14:35 smithi079 ceph-mon[35312]: 2 slow requests, 0 included below; oldest blocked for > 3461.894233 secs 2024-03-28T19:14:36.531 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:14:36 smithi022 ceph-mon[31985]: pgmap v2096: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 7 op/s 2024-03-28T19:14:36.674 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:14:36 smithi134 ceph-mon[35037]: pgmap v2096: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 7 op/s 2024-03-28T19:14:36.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:14:36 smithi079 ceph-mon[35312]: pgmap v2096: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 7 op/s 2024-03-28T19:14:38.531 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:14:38 smithi022 ceph-mon[31985]: pgmap v2097: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 7 op/s 2024-03-28T19:14:38.674 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:14:38 smithi134 ceph-mon[35037]: pgmap v2097: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 7 op/s 2024-03-28T19:14:38.681 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:14:38 smithi079 ceph-mon[35312]: pgmap v2097: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 7 op/s 2024-03-28T19:14:40.531 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:14:40 smithi022 ceph-mon[31985]: pgmap v2098: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 7 op/s 2024-03-28T19:14:40.531 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:14:40 smithi022 ceph-mon[31985]: 2 slow requests, 0 included below; oldest blocked for > 3466.894321 secs 2024-03-28T19:14:40.674 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:14:40 smithi134 ceph-mon[35037]: pgmap v2098: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 7 op/s 2024-03-28T19:14:40.674 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:14:40 smithi134 ceph-mon[35037]: 2 slow requests, 0 included below; oldest blocked for > 3466.894321 secs 2024-03-28T19:14:40.681 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:14:40 smithi079 ceph-mon[35312]: pgmap v2098: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 7 op/s 2024-03-28T19:14:40.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:14:40 smithi079 ceph-mon[35312]: 2 slow requests, 0 included below; oldest blocked for > 3466.894321 secs 2024-03-28T19:14:42.531 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:14:42 smithi022 ceph-mon[31985]: pgmap v2099: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 7 op/s 2024-03-28T19:14:42.674 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:14:42 smithi134 ceph-mon[35037]: pgmap v2099: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 7 op/s 2024-03-28T19:14:42.681 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:14:42 smithi079 ceph-mon[35312]: pgmap v2099: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 7 op/s 2024-03-28T19:14:44.531 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:14:44 smithi022 ceph-mon[31985]: pgmap v2100: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T19:14:44.674 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:14:44 smithi134 ceph-mon[35037]: pgmap v2100: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T19:14:44.681 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:14:44 smithi079 ceph-mon[35312]: pgmap v2100: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T19:14:45.674 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:14:45 smithi134 ceph-mon[35037]: 2 slow requests, 0 included below; oldest blocked for > 3471.894388 secs 2024-03-28T19:14:45.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:14:45 smithi079 ceph-mon[35312]: 2 slow requests, 0 included below; oldest blocked for > 3471.894388 secs 2024-03-28T19:14:45.781 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:14:45 smithi022 ceph-mon[31985]: 2 slow requests, 0 included below; oldest blocked for > 3471.894388 secs 2024-03-28T19:14:46.674 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:14:46 smithi134 ceph-mon[35037]: pgmap v2101: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 9 op/s 2024-03-28T19:14:46.681 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:14:46 smithi079 ceph-mon[35312]: pgmap v2101: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 9 op/s 2024-03-28T19:14:46.781 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:14:46 smithi022 ceph-mon[31985]: pgmap v2101: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 9 op/s 2024-03-28T19:14:48.584 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:14:48 smithi022 ceph-mon[31985]: pgmap v2102: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:14:48.674 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:14:48 smithi134 ceph-mon[35037]: pgmap v2102: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:14:48.681 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:14:48 smithi079 ceph-mon[35312]: pgmap v2102: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:14:50.674 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:14:50 smithi134 ceph-mon[35037]: pgmap v2103: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:14:50.674 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:14:50 smithi134 ceph-mon[35037]: 2 slow requests, 0 included below; oldest blocked for > 3476.894453 secs 2024-03-28T19:14:50.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:14:50 smithi079 ceph-mon[35312]: pgmap v2103: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:14:50.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:14:50 smithi079 ceph-mon[35312]: 2 slow requests, 0 included below; oldest blocked for > 3476.894453 secs 2024-03-28T19:14:50.781 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:14:50 smithi022 ceph-mon[31985]: pgmap v2103: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:14:50.781 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:14:50 smithi022 ceph-mon[31985]: 2 slow requests, 0 included below; oldest blocked for > 3476.894453 secs 2024-03-28T19:14:52.674 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:14:52 smithi134 ceph-mon[35037]: pgmap v2104: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T19:14:52.681 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:14:52 smithi079 ceph-mon[35312]: pgmap v2104: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T19:14:52.781 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:14:52 smithi022 ceph-mon[31985]: pgmap v2104: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T19:14:54.674 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:14:54 smithi134 ceph-mon[35037]: pgmap v2105: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:14:54.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:14:54 smithi079 ceph-mon[35312]: pgmap v2105: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:14:54.781 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:14:54 smithi022 ceph-mon[31985]: pgmap v2105: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:14:55.674 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:14:55 smithi134 ceph-mon[35037]: 2 slow requests, 0 included below; oldest blocked for > 3481.894531 secs 2024-03-28T19:14:55.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:14:55 smithi079 ceph-mon[35312]: 2 slow requests, 0 included below; oldest blocked for > 3481.894531 secs 2024-03-28T19:14:55.781 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:14:55 smithi022 ceph-mon[31985]: 2 slow requests, 0 included below; oldest blocked for > 3481.894531 secs 2024-03-28T19:14:56.674 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:14:56 smithi134 ceph-mon[35037]: pgmap v2106: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T19:14:56.681 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:14:56 smithi079 ceph-mon[35312]: pgmap v2106: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T19:14:56.781 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:14:56 smithi022 ceph-mon[31985]: pgmap v2106: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T19:14:58.674 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:14:58 smithi134 ceph-mon[35037]: pgmap v2107: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:14:58.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:14:58 smithi079 ceph-mon[35312]: pgmap v2107: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:14:58.781 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:14:58 smithi022 ceph-mon[31985]: pgmap v2107: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:15:00.674 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:15:00 smithi134 ceph-mon[35037]: pgmap v2108: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:15:00.674 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:15:00 smithi134 ceph-mon[35037]: 2 slow requests, 0 included below; oldest blocked for > 3486.894654 secs 2024-03-28T19:15:00.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:15:00 smithi079 ceph-mon[35312]: pgmap v2108: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:15:00.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:15:00 smithi079 ceph-mon[35312]: 2 slow requests, 0 included below; oldest blocked for > 3486.894654 secs 2024-03-28T19:15:00.781 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:15:00 smithi022 ceph-mon[31985]: pgmap v2108: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:15:00.781 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:15:00 smithi022 ceph-mon[31985]: 2 slow requests, 0 included below; oldest blocked for > 3486.894654 secs 2024-03-28T19:15:02.674 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:15:02 smithi134 ceph-mon[35037]: pgmap v2109: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T19:15:02.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:15:02 smithi079 ceph-mon[35312]: pgmap v2109: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T19:15:02.781 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:15:02 smithi022 ceph-mon[31985]: pgmap v2109: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T19:15:04.674 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:15:04 smithi134 ceph-mon[35037]: pgmap v2110: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:15:04.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:15:04 smithi079 ceph-mon[35312]: pgmap v2110: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:15:04.781 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:15:04 smithi022 ceph-mon[31985]: pgmap v2110: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:15:05.674 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:15:05 smithi134 ceph-mon[35037]: 2 slow requests, 0 included below; oldest blocked for > 3491.894679 secs 2024-03-28T19:15:05.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:15:05 smithi079 ceph-mon[35312]: 2 slow requests, 0 included below; oldest blocked for > 3491.894679 secs 2024-03-28T19:15:05.781 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:15:05 smithi022 ceph-mon[31985]: 2 slow requests, 0 included below; oldest blocked for > 3491.894679 secs 2024-03-28T19:15:06.674 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:15:06 smithi134 ceph-mon[35037]: pgmap v2111: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T19:15:06.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:15:06 smithi079 ceph-mon[35312]: pgmap v2111: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T19:15:06.781 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:15:06 smithi022 ceph-mon[31985]: pgmap v2111: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T19:15:08.674 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:15:08 smithi134 ceph-mon[35037]: pgmap v2112: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:15:08.681 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:15:08 smithi079 ceph-mon[35312]: pgmap v2112: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:15:08.781 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:15:08 smithi022 ceph-mon[31985]: pgmap v2112: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:15:10.674 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:15:10 smithi134 ceph-mon[35037]: pgmap v2113: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:15:10.674 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:15:10 smithi134 ceph-mon[35037]: 2 slow requests, 0 included below; oldest blocked for > 3496.894758 secs 2024-03-28T19:15:10.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:15:10 smithi079 ceph-mon[35312]: pgmap v2113: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:15:10.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:15:10 smithi079 ceph-mon[35312]: 2 slow requests, 0 included below; oldest blocked for > 3496.894758 secs 2024-03-28T19:15:10.781 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:15:10 smithi022 ceph-mon[31985]: pgmap v2113: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:15:10.781 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:15:10 smithi022 ceph-mon[31985]: 2 slow requests, 0 included below; oldest blocked for > 3496.894758 secs 2024-03-28T19:15:12.674 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:15:12 smithi134 ceph-mon[35037]: pgmap v2114: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T19:15:12.681 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:15:12 smithi079 ceph-mon[35312]: pgmap v2114: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T19:15:12.781 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:15:12 smithi022 ceph-mon[31985]: pgmap v2114: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T19:15:14.674 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:15:14 smithi134 ceph-mon[35037]: pgmap v2115: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:15:14.681 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:15:14 smithi079 ceph-mon[35312]: pgmap v2115: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:15:14.781 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:15:14 smithi022 ceph-mon[31985]: pgmap v2115: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:15:15.674 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:15:15 smithi134 ceph-mon[35037]: 2 slow requests, 0 included below; oldest blocked for > 3501.894805 secs 2024-03-28T19:15:15.681 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:15:15 smithi079 ceph-mon[35312]: 2 slow requests, 0 included below; oldest blocked for > 3501.894805 secs 2024-03-28T19:15:15.781 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:15:15 smithi022 ceph-mon[31985]: 2 slow requests, 0 included below; oldest blocked for > 3501.894805 secs 2024-03-28T19:15:16.674 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:15:16 smithi134 ceph-mon[35037]: pgmap v2116: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 9 op/s 2024-03-28T19:15:16.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:15:16 smithi079 ceph-mon[35312]: pgmap v2116: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 9 op/s 2024-03-28T19:15:16.781 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:15:16 smithi022 ceph-mon[31985]: pgmap v2116: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 9 op/s 2024-03-28T19:15:18.674 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:15:18 smithi134 ceph-mon[35037]: pgmap v2117: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T19:15:18.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:15:18 smithi079 ceph-mon[35312]: pgmap v2117: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T19:15:18.781 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:15:18 smithi022 ceph-mon[31985]: pgmap v2117: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T19:15:20.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:15:20 smithi079 ceph-mon[35312]: pgmap v2118: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T19:15:20.683 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:15:20 smithi079 ceph-mon[35312]: 2 slow requests, 0 included below; oldest blocked for > 3506.894843 secs 2024-03-28T19:15:20.781 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:15:20 smithi022 ceph-mon[31985]: pgmap v2118: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T19:15:20.781 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:15:20 smithi022 ceph-mon[31985]: 2 slow requests, 0 included below; oldest blocked for > 3506.894843 secs 2024-03-28T19:15:20.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:15:20 smithi134 ceph-mon[35037]: pgmap v2118: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T19:15:20.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:15:20 smithi134 ceph-mon[35037]: 2 slow requests, 0 included below; oldest blocked for > 3506.894843 secs 2024-03-28T19:15:22.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:15:22 smithi079 ceph-mon[35312]: pgmap v2119: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 9 op/s 2024-03-28T19:15:22.781 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:15:22 smithi022 ceph-mon[31985]: pgmap v2119: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 9 op/s 2024-03-28T19:15:22.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:15:22 smithi134 ceph-mon[35037]: pgmap v2119: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 9 op/s 2024-03-28T19:15:24.781 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:15:24 smithi022 ceph-mon[31985]: pgmap v2120: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T19:15:24.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:15:24 smithi134 ceph-mon[35037]: pgmap v2120: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T19:15:24.931 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:15:24 smithi079 ceph-mon[35312]: pgmap v2120: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T19:15:25.781 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:15:25 smithi022 ceph-mon[31985]: 2 slow requests, 0 included below; oldest blocked for > 3511.894966 secs 2024-03-28T19:15:25.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:15:25 smithi134 ceph-mon[35037]: 2 slow requests, 0 included below; oldest blocked for > 3511.894966 secs 2024-03-28T19:15:25.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:15:25 smithi079 ceph-mon[35312]: 2 slow requests, 0 included below; oldest blocked for > 3511.894966 secs 2024-03-28T19:15:26.781 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:15:26 smithi022 ceph-mon[31985]: pgmap v2121: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 9 op/s 2024-03-28T19:15:26.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:15:26 smithi134 ceph-mon[35037]: pgmap v2121: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 9 op/s 2024-03-28T19:15:26.931 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:15:26 smithi079 ceph-mon[35312]: pgmap v2121: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 9 op/s 2024-03-28T19:15:28.781 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:15:28 smithi022 ceph-mon[31985]: pgmap v2122: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:15:28.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:15:28 smithi134 ceph-mon[35037]: pgmap v2122: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:15:28.931 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:15:28 smithi079 ceph-mon[35312]: pgmap v2122: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:15:30.734 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:15:30 smithi079 ceph-mon[35312]: pgmap v2123: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:15:30.734 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:15:30 smithi079 ceph-mon[35312]: 2 slow requests, 0 included below; oldest blocked for > 3516.895056 secs 2024-03-28T19:15:30.781 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:15:30 smithi022 ceph-mon[31985]: pgmap v2123: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:15:30.781 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:15:30 smithi022 ceph-mon[31985]: 2 slow requests, 0 included below; oldest blocked for > 3516.895056 secs 2024-03-28T19:15:30.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:15:30 smithi134 ceph-mon[35037]: pgmap v2123: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:15:30.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:15:30 smithi134 ceph-mon[35037]: 2 slow requests, 0 included below; oldest blocked for > 3516.895056 secs 2024-03-28T19:15:31.781 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:15:31 smithi022 ceph-mon[31985]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T19:15:31.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:15:31 smithi134 ceph-mon[35037]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T19:15:31.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:15:31 smithi079 ceph-mon[35312]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T19:15:32.781 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:15:32 smithi022 ceph-mon[31985]: pgmap v2124: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T19:15:32.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:15:32 smithi134 ceph-mon[35037]: pgmap v2124: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T19:15:32.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:15:32 smithi079 ceph-mon[35312]: pgmap v2124: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T19:15:33.781 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:15:33 smithi022 ceph-mon[31985]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T19:15:33.781 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:15:33 smithi022 ceph-mon[31985]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T19:15:33.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:15:33 smithi022 ceph-mon[31985]: from='mgr.15522 ' entity='mgr.x' 2024-03-28T19:15:33.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:15:33 smithi134 ceph-mon[35037]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T19:15:33.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:15:33 smithi134 ceph-mon[35037]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T19:15:33.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:15:33 smithi134 ceph-mon[35037]: from='mgr.15522 ' entity='mgr.x' 2024-03-28T19:15:33.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:15:33 smithi079 ceph-mon[35312]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T19:15:33.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:15:33 smithi079 ceph-mon[35312]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T19:15:33.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:15:33 smithi079 ceph-mon[35312]: from='mgr.15522 ' entity='mgr.x' 2024-03-28T19:15:34.781 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:15:34 smithi022 ceph-mon[31985]: pgmap v2125: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 6 op/s 2024-03-28T19:15:34.781 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:15:34 smithi022 ceph-mon[31985]: pgmap v2126: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-03-28T19:15:34.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:15:34 smithi134 ceph-mon[35037]: pgmap v2125: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 6 op/s 2024-03-28T19:15:34.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:15:34 smithi134 ceph-mon[35037]: pgmap v2126: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-03-28T19:15:34.931 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:15:34 smithi079 ceph-mon[35312]: pgmap v2125: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 6 op/s 2024-03-28T19:15:34.931 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:15:34 smithi079 ceph-mon[35312]: pgmap v2126: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-03-28T19:15:35.781 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:15:35 smithi022 ceph-mon[31985]: 2 slow requests, 0 included below; oldest blocked for > 3521.895139 secs 2024-03-28T19:15:35.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:15:35 smithi134 ceph-mon[35037]: 2 slow requests, 0 included below; oldest blocked for > 3521.895139 secs 2024-03-28T19:15:35.931 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:15:35 smithi079 ceph-mon[35312]: 2 slow requests, 0 included below; oldest blocked for > 3521.895139 secs 2024-03-28T19:15:36.781 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:15:36 smithi022 ceph-mon[31985]: pgmap v2127: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-03-28T19:15:36.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:15:36 smithi134 ceph-mon[35037]: pgmap v2127: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-03-28T19:15:36.931 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:15:36 smithi079 ceph-mon[35312]: pgmap v2127: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-03-28T19:15:38.781 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:15:38 smithi022 ceph-mon[31985]: pgmap v2128: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-03-28T19:15:38.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:15:38 smithi134 ceph-mon[35037]: pgmap v2128: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-03-28T19:15:38.931 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:15:38 smithi079 ceph-mon[35312]: pgmap v2128: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-03-28T19:15:40.781 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:15:40 smithi022 ceph-mon[31985]: pgmap v2129: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-03-28T19:15:40.781 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:15:40 smithi022 ceph-mon[31985]: 2 slow requests, 0 included below; oldest blocked for > 3526.895227 secs 2024-03-28T19:15:40.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:15:40 smithi134 ceph-mon[35037]: pgmap v2129: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-03-28T19:15:40.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:15:40 smithi134 ceph-mon[35037]: 2 slow requests, 0 included below; oldest blocked for > 3526.895227 secs 2024-03-28T19:15:40.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:15:40 smithi079 ceph-mon[35312]: pgmap v2129: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-03-28T19:15:40.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:15:40 smithi079 ceph-mon[35312]: 2 slow requests, 0 included below; oldest blocked for > 3526.895227 secs 2024-03-28T19:15:42.781 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:15:42 smithi022 ceph-mon[31985]: pgmap v2130: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-03-28T19:15:42.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:15:42 smithi134 ceph-mon[35037]: pgmap v2130: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-03-28T19:15:42.931 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:15:42 smithi079 ceph-mon[35312]: pgmap v2130: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-03-28T19:15:44.781 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:15:44 smithi022 ceph-mon[31985]: pgmap v2131: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 7 op/s 2024-03-28T19:15:44.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:15:44 smithi134 ceph-mon[35037]: pgmap v2131: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 7 op/s 2024-03-28T19:15:44.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:15:44 smithi079 ceph-mon[35312]: pgmap v2131: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 7 op/s 2024-03-28T19:15:45.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:15:45 smithi134 ceph-mon[35037]: 2 slow requests, 0 included below; oldest blocked for > 3531.895299 secs 2024-03-28T19:15:45.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:15:45 smithi079 ceph-mon[35312]: 2 slow requests, 0 included below; oldest blocked for > 3531.895299 secs 2024-03-28T19:15:46.031 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:15:45 smithi022 ceph-mon[31985]: 2 slow requests, 0 included below; oldest blocked for > 3531.895299 secs 2024-03-28T19:15:46.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:15:46 smithi134 ceph-mon[35037]: pgmap v2132: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T19:15:46.931 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:15:46 smithi079 ceph-mon[35312]: pgmap v2132: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T19:15:47.031 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:15:46 smithi022 ceph-mon[31985]: pgmap v2132: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T19:15:48.834 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:15:48 smithi022 ceph-mon[31985]: pgmap v2133: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:15:48.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:15:48 smithi134 ceph-mon[35037]: pgmap v2133: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:15:48.931 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:15:48 smithi079 ceph-mon[35312]: pgmap v2133: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:15:50.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:15:50 smithi134 ceph-mon[35037]: pgmap v2134: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:15:50.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:15:50 smithi134 ceph-mon[35037]: 2 slow requests, 0 included below; oldest blocked for > 3536.895327 secs 2024-03-28T19:15:50.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:15:50 smithi079 ceph-mon[35312]: pgmap v2134: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:15:50.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:15:50 smithi079 ceph-mon[35312]: 2 slow requests, 0 included below; oldest blocked for > 3536.895327 secs 2024-03-28T19:15:51.031 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:15:50 smithi022 ceph-mon[31985]: pgmap v2134: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:15:51.031 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:15:50 smithi022 ceph-mon[31985]: 2 slow requests, 0 included below; oldest blocked for > 3536.895327 secs 2024-03-28T19:15:52.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:15:52 smithi134 ceph-mon[35037]: pgmap v2135: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T19:15:52.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:15:52 smithi079 ceph-mon[35312]: pgmap v2135: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T19:15:53.032 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:15:52 smithi022 ceph-mon[31985]: pgmap v2135: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T19:15:54.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:15:54 smithi134 ceph-mon[35037]: pgmap v2136: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:15:54.931 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:15:54 smithi079 ceph-mon[35312]: pgmap v2136: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:15:55.031 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:15:54 smithi022 ceph-mon[31985]: pgmap v2136: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:15:55.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:15:55 smithi134 ceph-mon[35037]: 2 slow requests, 0 included below; oldest blocked for > 3541.895393 secs 2024-03-28T19:15:55.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:15:55 smithi079 ceph-mon[35312]: 2 slow requests, 0 included below; oldest blocked for > 3541.895393 secs 2024-03-28T19:15:56.031 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:15:55 smithi022 ceph-mon[31985]: 2 slow requests, 0 included below; oldest blocked for > 3541.895393 secs 2024-03-28T19:15:56.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:15:56 smithi134 ceph-mon[35037]: pgmap v2137: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T19:15:56.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:15:56 smithi079 ceph-mon[35312]: pgmap v2137: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T19:15:57.031 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:15:56 smithi022 ceph-mon[31985]: pgmap v2137: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T19:15:58.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:15:58 smithi134 ceph-mon[35037]: pgmap v2138: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:15:58.931 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:15:58 smithi079 ceph-mon[35312]: pgmap v2138: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:15:59.031 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:15:58 smithi022 ceph-mon[31985]: pgmap v2138: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:16:00.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:16:00 smithi134 ceph-mon[35037]: pgmap v2139: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:16:00.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:16:00 smithi134 ceph-mon[35037]: 2 slow requests, 0 included below; oldest blocked for > 3546.895481 secs 2024-03-28T19:16:00.931 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:16:00 smithi079 ceph-mon[35312]: pgmap v2139: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:16:00.931 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:16:00 smithi079 ceph-mon[35312]: 2 slow requests, 0 included below; oldest blocked for > 3546.895481 secs 2024-03-28T19:16:01.031 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:16:00 smithi022 ceph-mon[31985]: pgmap v2139: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:16:01.031 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:16:00 smithi022 ceph-mon[31985]: 2 slow requests, 0 included below; oldest blocked for > 3546.895481 secs 2024-03-28T19:16:02.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:16:02 smithi134 ceph-mon[35037]: pgmap v2140: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T19:16:02.931 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:16:02 smithi079 ceph-mon[35312]: pgmap v2140: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T19:16:03.031 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:16:02 smithi022 ceph-mon[31985]: pgmap v2140: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T19:16:04.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:16:04 smithi134 ceph-mon[35037]: pgmap v2141: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:16:04.931 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:16:04 smithi079 ceph-mon[35312]: pgmap v2141: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:16:05.031 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:16:04 smithi022 ceph-mon[31985]: pgmap v2141: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:16:05.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:16:05 smithi134 ceph-mon[35037]: 2 slow requests, 0 included below; oldest blocked for > 3551.895572 secs 2024-03-28T19:16:05.931 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:16:05 smithi079 ceph-mon[35312]: 2 slow requests, 0 included below; oldest blocked for > 3551.895572 secs 2024-03-28T19:16:06.031 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:16:05 smithi022 ceph-mon[31985]: 2 slow requests, 0 included below; oldest blocked for > 3551.895572 secs 2024-03-28T19:16:06.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:16:06 smithi134 ceph-mon[35037]: pgmap v2142: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T19:16:06.931 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:16:06 smithi079 ceph-mon[35312]: pgmap v2142: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T19:16:07.031 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:16:06 smithi022 ceph-mon[31985]: pgmap v2142: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T19:16:08.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:16:08 smithi134 ceph-mon[35037]: pgmap v2143: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:16:08.931 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:16:08 smithi079 ceph-mon[35312]: pgmap v2143: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:16:09.031 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:16:08 smithi022 ceph-mon[31985]: pgmap v2143: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:16:10.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:16:10 smithi134 ceph-mon[35037]: pgmap v2144: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:16:10.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:16:10 smithi134 ceph-mon[35037]: 2 slow requests, 0 included below; oldest blocked for > 3556.895641 secs 2024-03-28T19:16:10.931 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:16:10 smithi079 ceph-mon[35312]: pgmap v2144: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:16:10.931 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:16:10 smithi079 ceph-mon[35312]: 2 slow requests, 0 included below; oldest blocked for > 3556.895641 secs 2024-03-28T19:16:11.031 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:16:10 smithi022 ceph-mon[31985]: pgmap v2144: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:16:11.031 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:16:10 smithi022 ceph-mon[31985]: 2 slow requests, 0 included below; oldest blocked for > 3556.895641 secs 2024-03-28T19:16:12.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:16:12 smithi134 ceph-mon[35037]: pgmap v2145: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T19:16:12.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:16:12 smithi079 ceph-mon[35312]: pgmap v2145: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T19:16:13.031 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:16:12 smithi022 ceph-mon[31985]: pgmap v2145: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T19:16:14.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:16:14 smithi134 ceph-mon[35037]: pgmap v2146: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:16:14.931 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:16:14 smithi079 ceph-mon[35312]: pgmap v2146: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:16:15.031 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:16:14 smithi022 ceph-mon[31985]: pgmap v2146: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:16:15.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:16:15 smithi134 ceph-mon[35037]: 2 slow requests, 0 included below; oldest blocked for > 3561.895713 secs 2024-03-28T19:16:15.931 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:16:15 smithi079 ceph-mon[35312]: 2 slow requests, 0 included below; oldest blocked for > 3561.895713 secs 2024-03-28T19:16:16.031 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:16:15 smithi022 ceph-mon[31985]: 2 slow requests, 0 included below; oldest blocked for > 3561.895713 secs 2024-03-28T19:16:16.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:16:16 smithi134 ceph-mon[35037]: pgmap v2147: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T19:16:16.931 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:16:16 smithi079 ceph-mon[35312]: pgmap v2147: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T19:16:17.031 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:16:16 smithi022 ceph-mon[31985]: pgmap v2147: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T19:16:18.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:16:18 smithi134 ceph-mon[35037]: pgmap v2148: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:16:18.931 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:16:18 smithi079 ceph-mon[35312]: pgmap v2148: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:16:19.031 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:16:18 smithi022 ceph-mon[31985]: pgmap v2148: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:16:20.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:16:20 smithi079 ceph-mon[35312]: pgmap v2149: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:16:20.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:16:20 smithi079 ceph-mon[35312]: 2 slow requests, 0 included below; oldest blocked for > 3566.895828 secs 2024-03-28T19:16:21.031 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:16:20 smithi022 ceph-mon[31985]: pgmap v2149: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:16:21.031 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:16:20 smithi022 ceph-mon[31985]: 2 slow requests, 0 included below; oldest blocked for > 3566.895828 secs 2024-03-28T19:16:21.174 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:16:20 smithi134 ceph-mon[35037]: pgmap v2149: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:16:21.174 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:16:20 smithi134 ceph-mon[35037]: 2 slow requests, 0 included below; oldest blocked for > 3566.895828 secs 2024-03-28T19:16:22.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:16:22 smithi079 ceph-mon[35312]: pgmap v2150: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T19:16:23.031 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:16:22 smithi022 ceph-mon[31985]: pgmap v2150: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T19:16:23.174 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:16:22 smithi134 ceph-mon[35037]: pgmap v2150: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T19:16:25.031 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:16:24 smithi022 ceph-mon[31985]: pgmap v2151: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:16:25.174 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:16:24 smithi134 ceph-mon[35037]: pgmap v2151: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:16:25.182 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:16:24 smithi079 ceph-mon[35312]: pgmap v2151: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:16:26.031 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:16:25 smithi022 ceph-mon[31985]: 2 slow requests, 0 included below; oldest blocked for > 3571.895889 secs 2024-03-28T19:16:26.174 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:16:25 smithi134 ceph-mon[35037]: 2 slow requests, 0 included below; oldest blocked for > 3571.895889 secs 2024-03-28T19:16:26.181 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:16:25 smithi079 ceph-mon[35312]: 2 slow requests, 0 included below; oldest blocked for > 3571.895889 secs 2024-03-28T19:16:27.031 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:16:26 smithi022 ceph-mon[31985]: pgmap v2152: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T19:16:27.174 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:16:26 smithi134 ceph-mon[35037]: pgmap v2152: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T19:16:27.181 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:16:26 smithi079 ceph-mon[35312]: pgmap v2152: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T19:16:29.031 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:16:28 smithi022 ceph-mon[31985]: pgmap v2153: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:16:29.174 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:16:28 smithi134 ceph-mon[35037]: pgmap v2153: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:16:29.181 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:16:28 smithi079 ceph-mon[35312]: pgmap v2153: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:16:30.983 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:16:30 smithi079 ceph-mon[35312]: pgmap v2154: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:16:30.983 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:16:30 smithi079 ceph-mon[35312]: 2 slow requests, 0 included below; oldest blocked for > 3576.895917 secs 2024-03-28T19:16:31.031 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:16:30 smithi022 ceph-mon[31985]: pgmap v2154: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:16:31.031 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:16:30 smithi022 ceph-mon[31985]: 2 slow requests, 0 included below; oldest blocked for > 3576.895917 secs 2024-03-28T19:16:31.173 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:16:30 smithi134 ceph-mon[35037]: pgmap v2154: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:16:31.174 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:16:30 smithi134 ceph-mon[35037]: 2 slow requests, 0 included below; oldest blocked for > 3576.895917 secs 2024-03-28T19:16:33.031 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:16:32 smithi022 ceph-mon[31985]: pgmap v2155: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T19:16:33.174 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:16:32 smithi134 ceph-mon[35037]: pgmap v2155: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T19:16:33.181 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:16:32 smithi079 ceph-mon[35312]: pgmap v2155: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T19:16:34.031 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:16:33 smithi022 ceph-mon[31985]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T19:16:34.174 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:16:33 smithi134 ceph-mon[35037]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T19:16:34.181 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:16:33 smithi079 ceph-mon[35312]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T19:16:35.031 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:16:34 smithi022 ceph-mon[31985]: pgmap v2156: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:16:35.031 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:16:34 smithi022 ceph-mon[31985]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T19:16:35.031 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:16:34 smithi022 ceph-mon[31985]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T19:16:35.031 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:16:34 smithi022 ceph-mon[31985]: from='mgr.15522 ' entity='mgr.x' 2024-03-28T19:16:35.174 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:16:34 smithi134 ceph-mon[35037]: pgmap v2156: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:16:35.174 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:16:34 smithi134 ceph-mon[35037]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T19:16:35.174 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:16:34 smithi134 ceph-mon[35037]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T19:16:35.174 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:16:34 smithi134 ceph-mon[35037]: from='mgr.15522 ' entity='mgr.x' 2024-03-28T19:16:35.181 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:16:34 smithi079 ceph-mon[35312]: pgmap v2156: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:16:35.181 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:16:34 smithi079 ceph-mon[35312]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T19:16:35.181 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:16:34 smithi079 ceph-mon[35312]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T19:16:35.181 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:16:34 smithi079 ceph-mon[35312]: from='mgr.15522 ' entity='mgr.x' 2024-03-28T19:16:36.031 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:16:35 smithi022 ceph-mon[31985]: 2 slow requests, 0 included below; oldest blocked for > 3581.895982 secs 2024-03-28T19:16:36.031 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:16:35 smithi022 ceph-mon[31985]: pgmap v2157: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 10 op/s 2024-03-28T19:16:36.031 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:16:35 smithi022 ceph-mon[31985]: pgmap v2158: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-03-28T19:16:36.174 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:16:35 smithi134 ceph-mon[35037]: 2 slow requests, 0 included below; oldest blocked for > 3581.895982 secs 2024-03-28T19:16:36.174 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:16:35 smithi134 ceph-mon[35037]: pgmap v2157: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 10 op/s 2024-03-28T19:16:36.174 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:16:35 smithi134 ceph-mon[35037]: pgmap v2158: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-03-28T19:16:36.182 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:16:35 smithi079 ceph-mon[35312]: 2 slow requests, 0 included below; oldest blocked for > 3581.895982 secs 2024-03-28T19:16:36.182 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:16:35 smithi079 ceph-mon[35312]: pgmap v2157: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 10 op/s 2024-03-28T19:16:36.182 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:16:35 smithi079 ceph-mon[35312]: pgmap v2158: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-03-28T19:16:38.031 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:16:37 smithi022 ceph-mon[31985]: pgmap v2159: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-03-28T19:16:38.173 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:16:37 smithi134 ceph-mon[35037]: pgmap v2159: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-03-28T19:16:38.181 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:16:37 smithi079 ceph-mon[35312]: pgmap v2159: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-03-28T19:16:39.174 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:16:38 smithi134 ceph-mon[35037]: pgmap v2160: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-03-28T19:16:39.181 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:16:38 smithi079 ceph-mon[35312]: pgmap v2160: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-03-28T19:16:39.281 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:16:38 smithi022 ceph-mon[31985]: pgmap v2160: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-03-28T19:16:40.174 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:16:39 smithi134 ceph-mon[35037]: 2 slow requests, 0 included below; oldest blocked for > 3586.896096 secs 2024-03-28T19:16:40.182 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:16:39 smithi079 ceph-mon[35312]: 2 slow requests, 0 included below; oldest blocked for > 3586.896096 secs 2024-03-28T19:16:40.281 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:16:39 smithi022 ceph-mon[31985]: 2 slow requests, 0 included below; oldest blocked for > 3586.896096 secs 2024-03-28T19:16:41.174 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:16:40 smithi134 ceph-mon[35037]: pgmap v2161: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-03-28T19:16:41.182 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:16:40 smithi079 ceph-mon[35312]: pgmap v2161: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-03-28T19:16:41.281 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:16:40 smithi022 ceph-mon[31985]: pgmap v2161: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-03-28T19:16:44.031 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:16:43 smithi022 ceph-mon[31985]: pgmap v2162: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-03-28T19:16:44.174 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:16:43 smithi134 ceph-mon[35037]: pgmap v2162: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-03-28T19:16:44.181 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:16:43 smithi079 ceph-mon[35312]: pgmap v2162: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-03-28T19:16:46.031 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:16:45 smithi022 ceph-mon[31985]: 2 slow requests, 0 included below; oldest blocked for > 3591.896158 secs 2024-03-28T19:16:46.031 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:16:45 smithi022 ceph-mon[31985]: pgmap v2163: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 7 op/s 2024-03-28T19:16:46.174 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:16:45 smithi134 ceph-mon[35037]: 2 slow requests, 0 included below; oldest blocked for > 3591.896158 secs 2024-03-28T19:16:46.174 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:16:45 smithi134 ceph-mon[35037]: pgmap v2163: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 7 op/s 2024-03-28T19:16:46.181 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:16:45 smithi079 ceph-mon[35312]: 2 slow requests, 0 included below; oldest blocked for > 3591.896158 secs 2024-03-28T19:16:46.182 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:16:45 smithi079 ceph-mon[35312]: pgmap v2163: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 7 op/s 2024-03-28T19:16:48.031 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:16:47 smithi022 ceph-mon[31985]: pgmap v2164: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:16:48.174 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:16:47 smithi134 ceph-mon[35037]: pgmap v2164: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:16:48.182 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:16:47 smithi079 ceph-mon[35312]: pgmap v2164: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:16:50.031 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:16:49 smithi022 ceph-mon[31985]: pgmap v2165: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:16:50.174 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:16:49 smithi134 ceph-mon[35037]: pgmap v2165: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:16:50.182 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:16:49 smithi079 ceph-mon[35312]: pgmap v2165: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:16:51.031 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:16:50 smithi022 ceph-mon[31985]: 2 slow requests, 0 included below; oldest blocked for > 3596.896227 secs 2024-03-28T19:16:51.076 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:16:50 smithi134 ceph-mon[35037]: 2 slow requests, 0 included below; oldest blocked for > 3596.896227 secs 2024-03-28T19:16:51.181 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:16:50 smithi079 ceph-mon[35312]: 2 slow requests, 0 included below; oldest blocked for > 3596.896227 secs 2024-03-28T19:16:52.031 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:16:51 smithi022 ceph-mon[31985]: pgmap v2166: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T19:16:52.174 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:16:51 smithi134 ceph-mon[35037]: pgmap v2166: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T19:16:52.181 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:16:51 smithi079 ceph-mon[35312]: pgmap v2166: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T19:16:54.031 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:16:53 smithi022 ceph-mon[31985]: pgmap v2167: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:16:54.174 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:16:53 smithi134 ceph-mon[35037]: pgmap v2167: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:16:54.182 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:16:53 smithi079 ceph-mon[35312]: pgmap v2167: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:16:56.031 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:16:55 smithi022 ceph-mon[31985]: 2 slow requests, 0 included below; oldest blocked for > 3601.896301 secs 2024-03-28T19:16:56.031 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:16:55 smithi022 ceph-mon[31985]: pgmap v2168: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T19:16:56.174 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:16:55 smithi134 ceph-mon[35037]: 2 slow requests, 0 included below; oldest blocked for > 3601.896301 secs 2024-03-28T19:16:56.174 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:16:55 smithi134 ceph-mon[35037]: pgmap v2168: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T19:16:56.181 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:16:55 smithi079 ceph-mon[35312]: 2 slow requests, 0 included below; oldest blocked for > 3601.896301 secs 2024-03-28T19:16:56.181 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:16:55 smithi079 ceph-mon[35312]: pgmap v2168: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T19:16:58.031 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:16:57 smithi022 ceph-mon[31985]: pgmap v2169: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:16:58.174 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:16:57 smithi134 ceph-mon[35037]: pgmap v2169: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:16:58.181 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:16:57 smithi079 ceph-mon[35312]: pgmap v2169: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:17:00.031 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:16:59 smithi022 ceph-mon[31985]: pgmap v2170: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:17:00.174 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:16:59 smithi134 ceph-mon[35037]: pgmap v2170: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:17:00.182 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:16:59 smithi079 ceph-mon[35312]: pgmap v2170: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:17:01.031 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:17:00 smithi022 ceph-mon[31985]: 2 slow requests, 0 included below; oldest blocked for > 3606.896364 secs 2024-03-28T19:17:01.031 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:17:00 smithi022 ceph-mon[31985]: pgmap v2171: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T19:17:01.083 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:17:00 smithi079 ceph-mon[35312]: 2 slow requests, 0 included below; oldest blocked for > 3606.896364 secs 2024-03-28T19:17:01.083 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:17:00 smithi079 ceph-mon[35312]: pgmap v2171: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T19:17:01.174 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:17:00 smithi134 ceph-mon[35037]: 2 slow requests, 0 included below; oldest blocked for > 3606.896364 secs 2024-03-28T19:17:01.174 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:17:00 smithi134 ceph-mon[35037]: pgmap v2171: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T19:17:04.031 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:17:03 smithi022 ceph-mon[31985]: pgmap v2172: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:17:04.173 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:17:03 smithi134 ceph-mon[35037]: pgmap v2172: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:17:04.181 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:17:03 smithi079 ceph-mon[35312]: pgmap v2172: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:17:06.031 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:17:05 smithi022 ceph-mon[31985]: 2 slow requests, 0 included below; oldest blocked for > 3611.896431 secs 2024-03-28T19:17:06.031 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:17:05 smithi022 ceph-mon[31985]: pgmap v2173: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T19:17:06.174 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:17:05 smithi134 ceph-mon[35037]: 2 slow requests, 0 included below; oldest blocked for > 3611.896431 secs 2024-03-28T19:17:06.174 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:17:05 smithi134 ceph-mon[35037]: pgmap v2173: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T19:17:06.182 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:17:05 smithi079 ceph-mon[35312]: 2 slow requests, 0 included below; oldest blocked for > 3611.896431 secs 2024-03-28T19:17:06.182 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:17:05 smithi079 ceph-mon[35312]: pgmap v2173: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T19:17:08.031 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:17:07 smithi022 ceph-mon[31985]: pgmap v2174: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:17:08.174 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:17:07 smithi134 ceph-mon[35037]: pgmap v2174: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:17:08.181 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:17:07 smithi079 ceph-mon[35312]: pgmap v2174: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:17:10.031 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:17:09 smithi022 ceph-mon[31985]: pgmap v2175: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:17:10.174 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:17:09 smithi134 ceph-mon[35037]: pgmap v2175: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:17:10.181 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:17:09 smithi079 ceph-mon[35312]: pgmap v2175: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:17:11.031 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:17:10 smithi022 ceph-mon[31985]: 2 slow requests, 0 included below; oldest blocked for > 3616.896514 secs 2024-03-28T19:17:11.174 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:17:10 smithi134 ceph-mon[35037]: 2 slow requests, 0 included below; oldest blocked for > 3616.896514 secs 2024-03-28T19:17:11.182 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:17:10 smithi079 ceph-mon[35312]: 2 slow requests, 0 included below; oldest blocked for > 3616.896514 secs 2024-03-28T19:17:12.031 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:17:11 smithi022 ceph-mon[31985]: pgmap v2176: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T19:17:12.174 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:17:11 smithi134 ceph-mon[35037]: pgmap v2176: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T19:17:12.182 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:17:11 smithi079 ceph-mon[35312]: pgmap v2176: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T19:17:14.031 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:17:13 smithi022 ceph-mon[31985]: pgmap v2177: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:17:14.174 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:17:13 smithi134 ceph-mon[35037]: pgmap v2177: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:17:14.181 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:17:13 smithi079 ceph-mon[35312]: pgmap v2177: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:17:16.031 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:17:15 smithi022 ceph-mon[31985]: 2 slow requests, 0 included below; oldest blocked for > 3621.896636 secs 2024-03-28T19:17:16.032 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:17:15 smithi022 ceph-mon[31985]: pgmap v2178: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T19:17:16.174 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:17:15 smithi134 ceph-mon[35037]: 2 slow requests, 0 included below; oldest blocked for > 3621.896636 secs 2024-03-28T19:17:16.174 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:17:15 smithi134 ceph-mon[35037]: pgmap v2178: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T19:17:16.181 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:17:15 smithi079 ceph-mon[35312]: 2 slow requests, 0 included below; oldest blocked for > 3621.896636 secs 2024-03-28T19:17:16.181 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:17:15 smithi079 ceph-mon[35312]: pgmap v2178: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T19:17:18.031 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:17:17 smithi022 ceph-mon[31985]: pgmap v2179: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:17:18.174 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:17:17 smithi134 ceph-mon[35037]: pgmap v2179: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:17:18.181 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:17:17 smithi079 ceph-mon[35312]: pgmap v2179: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:17:19.031 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:17:18 smithi022 ceph-mon[31985]: pgmap v2180: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:17:19.174 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:17:18 smithi134 ceph-mon[35037]: pgmap v2180: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:17:19.181 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:17:18 smithi079 ceph-mon[35312]: pgmap v2180: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:17:20.174 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:17:19 smithi134 ceph-mon[35037]: 2 slow requests, 0 included below; oldest blocked for > 3626.896672 secs 2024-03-28T19:17:20.181 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:17:19 smithi079 ceph-mon[35312]: 2 slow requests, 0 included below; oldest blocked for > 3626.896672 secs 2024-03-28T19:17:20.281 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:17:19 smithi022 ceph-mon[31985]: 2 slow requests, 0 included below; oldest blocked for > 3626.896672 secs 2024-03-28T19:17:21.174 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:17:20 smithi134 ceph-mon[35037]: pgmap v2181: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T19:17:21.181 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:17:20 smithi079 ceph-mon[35312]: pgmap v2181: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T19:17:21.282 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:17:20 smithi022 ceph-mon[31985]: pgmap v2181: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T19:17:24.031 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:17:23 smithi022 ceph-mon[31985]: pgmap v2182: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:17:24.174 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:17:23 smithi134 ceph-mon[35037]: pgmap v2182: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:17:24.182 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:17:23 smithi079 ceph-mon[35312]: pgmap v2182: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:17:26.031 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:17:25 smithi022 ceph-mon[31985]: 2 slow requests, 0 included below; oldest blocked for > 3631.896736 secs 2024-03-28T19:17:26.032 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:17:25 smithi022 ceph-mon[31985]: pgmap v2183: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T19:17:26.174 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:17:25 smithi134 ceph-mon[35037]: 2 slow requests, 0 included below; oldest blocked for > 3631.896736 secs 2024-03-28T19:17:26.175 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:17:25 smithi134 ceph-mon[35037]: pgmap v2183: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T19:17:26.181 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:17:25 smithi079 ceph-mon[35312]: 2 slow requests, 0 included below; oldest blocked for > 3631.896736 secs 2024-03-28T19:17:26.182 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:17:25 smithi079 ceph-mon[35312]: pgmap v2183: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T19:17:28.031 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:17:27 smithi022 ceph-mon[31985]: pgmap v2184: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:17:28.174 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:17:27 smithi134 ceph-mon[35037]: pgmap v2184: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:17:28.182 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:17:27 smithi079 ceph-mon[35312]: pgmap v2184: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:17:30.031 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:17:29 smithi022 ceph-mon[31985]: pgmap v2185: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:17:30.174 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:17:29 smithi134 ceph-mon[35037]: pgmap v2185: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:17:30.181 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:17:29 smithi079 ceph-mon[35312]: pgmap v2185: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:17:31.031 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:17:30 smithi022 ceph-mon[31985]: 2 slow requests, 0 included below; oldest blocked for > 3636.896806 secs 2024-03-28T19:17:31.083 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:17:30 smithi079 ceph-mon[35312]: 2 slow requests, 0 included below; oldest blocked for > 3636.896806 secs 2024-03-28T19:17:31.174 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:17:30 smithi134 ceph-mon[35037]: 2 slow requests, 0 included below; oldest blocked for > 3636.896806 secs 2024-03-28T19:17:32.031 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:17:31 smithi022 ceph-mon[31985]: pgmap v2186: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 9 op/s 2024-03-28T19:17:32.173 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:17:31 smithi134 ceph-mon[35037]: pgmap v2186: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 9 op/s 2024-03-28T19:17:32.182 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:17:31 smithi079 ceph-mon[35312]: pgmap v2186: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 9 op/s 2024-03-28T19:17:34.031 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:17:33 smithi022 ceph-mon[31985]: pgmap v2187: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T19:17:34.174 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:17:33 smithi134 ceph-mon[35037]: pgmap v2187: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T19:17:34.182 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:17:33 smithi079 ceph-mon[35312]: pgmap v2187: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T19:17:35.174 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:17:34 smithi134 ceph-mon[35037]: 2 slow requests, 0 included below; oldest blocked for > 3641.896870 secs 2024-03-28T19:17:35.174 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:17:34 smithi134 ceph-mon[35037]: pgmap v2188: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 9 op/s 2024-03-28T19:17:35.181 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:17:34 smithi079 ceph-mon[35312]: 2 slow requests, 0 included below; oldest blocked for > 3641.896870 secs 2024-03-28T19:17:35.181 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:17:34 smithi079 ceph-mon[35312]: pgmap v2188: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 9 op/s 2024-03-28T19:17:35.281 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:17:34 smithi022 ceph-mon[31985]: 2 slow requests, 0 included below; oldest blocked for > 3641.896870 secs 2024-03-28T19:17:35.281 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:17:34 smithi022 ceph-mon[31985]: pgmap v2188: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 9 op/s 2024-03-28T19:17:36.174 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:17:35 smithi134 ceph-mon[35037]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T19:17:36.181 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:17:35 smithi079 ceph-mon[35312]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T19:17:36.281 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:17:35 smithi022 ceph-mon[31985]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T19:17:37.174 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:17:36 smithi134 ceph-mon[35037]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T19:17:37.174 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:17:36 smithi134 ceph-mon[35037]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T19:17:37.174 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:17:36 smithi134 ceph-mon[35037]: pgmap v2189: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T19:17:37.174 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:17:36 smithi134 ceph-mon[35037]: pgmap v2190: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 7 op/s 2024-03-28T19:17:37.175 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:17:36 smithi134 ceph-mon[35037]: from='mgr.15522 ' entity='mgr.x' 2024-03-28T19:17:37.182 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:17:36 smithi079 ceph-mon[35312]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T19:17:37.182 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:17:36 smithi079 ceph-mon[35312]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T19:17:37.182 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:17:36 smithi079 ceph-mon[35312]: pgmap v2189: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T19:17:37.182 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:17:36 smithi079 ceph-mon[35312]: pgmap v2190: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 7 op/s 2024-03-28T19:17:37.182 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:17:36 smithi079 ceph-mon[35312]: from='mgr.15522 ' entity='mgr.x' 2024-03-28T19:17:37.281 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:17:36 smithi022 ceph-mon[31985]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T19:17:37.282 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:17:36 smithi022 ceph-mon[31985]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T19:17:37.282 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:17:36 smithi022 ceph-mon[31985]: pgmap v2189: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T19:17:37.282 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:17:36 smithi022 ceph-mon[31985]: pgmap v2190: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 7 op/s 2024-03-28T19:17:37.282 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:17:36 smithi022 ceph-mon[31985]: from='mgr.15522 ' entity='mgr.x' 2024-03-28T19:17:39.781 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:17:39 smithi022 ceph-mon[31985]: pgmap v2191: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 7 op/s 2024-03-28T19:17:39.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:17:39 smithi134 ceph-mon[35037]: pgmap v2191: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 7 op/s 2024-03-28T19:17:39.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:17:39 smithi079 ceph-mon[35312]: pgmap v2191: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 7 op/s 2024-03-28T19:17:40.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:17:40 smithi134 ceph-mon[35037]: 2 slow requests, 0 included below; oldest blocked for > 3646.896945 secs 2024-03-28T19:17:40.931 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:17:40 smithi079 ceph-mon[35312]: 2 slow requests, 0 included below; oldest blocked for > 3646.896945 secs 2024-03-28T19:17:41.031 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:17:40 smithi022 ceph-mon[31985]: 2 slow requests, 0 included below; oldest blocked for > 3646.896945 secs 2024-03-28T19:17:41.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:17:41 smithi134 ceph-mon[35037]: pgmap v2192: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 8 op/s 2024-03-28T19:17:41.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:17:41 smithi079 ceph-mon[35312]: pgmap v2192: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 8 op/s 2024-03-28T19:17:42.031 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:17:41 smithi022 ceph-mon[31985]: pgmap v2192: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 8 op/s 2024-03-28T19:17:43.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:17:43 smithi134 ceph-mon[35037]: pgmap v2193: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 8 op/s 2024-03-28T19:17:43.931 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:17:43 smithi079 ceph-mon[35312]: pgmap v2193: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 8 op/s 2024-03-28T19:17:44.031 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:17:43 smithi022 ceph-mon[31985]: pgmap v2193: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 8 op/s 2024-03-28T19:17:45.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:17:45 smithi134 ceph-mon[35037]: 2 slow requests, 0 included below; oldest blocked for > 3651.897023 secs 2024-03-28T19:17:45.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:17:45 smithi134 ceph-mon[35037]: pgmap v2194: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-03-28T19:17:45.931 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:17:45 smithi079 ceph-mon[35312]: 2 slow requests, 0 included below; oldest blocked for > 3651.897023 secs 2024-03-28T19:17:45.931 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:17:45 smithi079 ceph-mon[35312]: pgmap v2194: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-03-28T19:17:46.031 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:17:45 smithi022 ceph-mon[31985]: 2 slow requests, 0 included below; oldest blocked for > 3651.897023 secs 2024-03-28T19:17:46.031 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:17:45 smithi022 ceph-mon[31985]: pgmap v2194: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-03-28T19:17:47.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:17:47 smithi134 ceph-mon[35037]: pgmap v2195: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 7 op/s 2024-03-28T19:17:47.931 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:17:47 smithi079 ceph-mon[35312]: pgmap v2195: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 7 op/s 2024-03-28T19:17:48.031 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:17:47 smithi022 ceph-mon[31985]: pgmap v2195: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 7 op/s 2024-03-28T19:17:49.834 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:17:49 smithi022 ceph-mon[31985]: pgmap v2196: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:17:49.923 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:17:49 smithi134 ceph-mon[35037]: pgmap v2196: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:17:49.931 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:17:49 smithi079 ceph-mon[35312]: pgmap v2196: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:17:50.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:17:50 smithi134 ceph-mon[35037]: 2 slow requests, 0 included below; oldest blocked for > 3656.897115 secs 2024-03-28T19:17:50.931 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:17:50 smithi079 ceph-mon[35312]: 2 slow requests, 0 included below; oldest blocked for > 3656.897115 secs 2024-03-28T19:17:51.031 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:17:50 smithi022 ceph-mon[31985]: 2 slow requests, 0 included below; oldest blocked for > 3656.897115 secs 2024-03-28T19:17:51.828 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:17:51 smithi134 ceph-mon[35037]: pgmap v2197: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T19:17:51.931 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:17:51 smithi079 ceph-mon[35312]: pgmap v2197: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T19:17:52.031 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:17:51 smithi022 ceph-mon[31985]: pgmap v2197: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T19:17:53.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:17:53 smithi134 ceph-mon[35037]: pgmap v2198: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:17:53.931 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:17:53 smithi079 ceph-mon[35312]: pgmap v2198: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:17:54.031 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:17:53 smithi022 ceph-mon[31985]: pgmap v2198: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:17:55.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:17:55 smithi134 ceph-mon[35037]: 2 slow requests, 0 included below; oldest blocked for > 3661.897206 secs 2024-03-28T19:17:55.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:17:55 smithi134 ceph-mon[35037]: pgmap v2199: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T19:17:55.931 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:17:55 smithi079 ceph-mon[35312]: 2 slow requests, 0 included below; oldest blocked for > 3661.897206 secs 2024-03-28T19:17:55.931 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:17:55 smithi079 ceph-mon[35312]: pgmap v2199: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T19:17:56.031 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:17:55 smithi022 ceph-mon[31985]: 2 slow requests, 0 included below; oldest blocked for > 3661.897206 secs 2024-03-28T19:17:56.031 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:17:55 smithi022 ceph-mon[31985]: pgmap v2199: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T19:17:57.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:17:57 smithi134 ceph-mon[35037]: pgmap v2200: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:17:57.931 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:17:57 smithi079 ceph-mon[35312]: pgmap v2200: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:17:58.031 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:17:57 smithi022 ceph-mon[31985]: pgmap v2200: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:17:59.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:17:59 smithi134 ceph-mon[35037]: pgmap v2201: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:17:59.931 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:17:59 smithi079 ceph-mon[35312]: pgmap v2201: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:18:00.031 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:17:59 smithi022 ceph-mon[31985]: pgmap v2201: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:18:00.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:18:00 smithi134 ceph-mon[35037]: 2 slow requests, 0 included below; oldest blocked for > 3666.897285 secs 2024-03-28T19:18:00.931 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:18:00 smithi079 ceph-mon[35312]: 2 slow requests, 0 included below; oldest blocked for > 3666.897285 secs 2024-03-28T19:18:01.031 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:18:00 smithi022 ceph-mon[31985]: 2 slow requests, 0 included below; oldest blocked for > 3666.897285 secs 2024-03-28T19:18:01.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:18:01 smithi134 ceph-mon[35037]: pgmap v2202: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T19:18:01.931 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:18:01 smithi079 ceph-mon[35312]: pgmap v2202: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T19:18:02.031 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:18:01 smithi022 ceph-mon[31985]: pgmap v2202: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T19:18:03.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:18:03 smithi134 ceph-mon[35037]: pgmap v2203: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:18:03.931 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:18:03 smithi079 ceph-mon[35312]: pgmap v2203: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:18:04.031 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:18:03 smithi022 ceph-mon[31985]: pgmap v2203: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:18:05.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:18:05 smithi134 ceph-mon[35037]: 2 slow requests, 0 included below; oldest blocked for > 3671.897375 secs 2024-03-28T19:18:05.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:18:05 smithi134 ceph-mon[35037]: pgmap v2204: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T19:18:05.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:18:05 smithi079 ceph-mon[35312]: 2 slow requests, 0 included below; oldest blocked for > 3671.897375 secs 2024-03-28T19:18:05.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:18:05 smithi079 ceph-mon[35312]: pgmap v2204: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T19:18:06.031 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:18:05 smithi022 ceph-mon[31985]: 2 slow requests, 0 included below; oldest blocked for > 3671.897375 secs 2024-03-28T19:18:06.031 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:18:05 smithi022 ceph-mon[31985]: pgmap v2204: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T19:18:07.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:18:07 smithi134 ceph-mon[35037]: pgmap v2205: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:18:07.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:18:07 smithi079 ceph-mon[35312]: pgmap v2205: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:18:08.031 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:18:07 smithi022 ceph-mon[31985]: pgmap v2205: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:18:09.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:18:09 smithi134 ceph-mon[35037]: pgmap v2206: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:18:09.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:18:09 smithi079 ceph-mon[35312]: pgmap v2206: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:18:10.031 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:18:09 smithi022 ceph-mon[31985]: pgmap v2206: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:18:10.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:18:10 smithi134 ceph-mon[35037]: 2 slow requests, 0 included below; oldest blocked for > 3676.897452 secs 2024-03-28T19:18:10.931 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:18:10 smithi079 ceph-mon[35312]: 2 slow requests, 0 included below; oldest blocked for > 3676.897452 secs 2024-03-28T19:18:11.031 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:18:10 smithi022 ceph-mon[31985]: 2 slow requests, 0 included below; oldest blocked for > 3676.897452 secs 2024-03-28T19:18:11.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:18:11 smithi134 ceph-mon[35037]: pgmap v2207: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T19:18:11.931 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:18:11 smithi079 ceph-mon[35312]: pgmap v2207: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T19:18:12.031 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:18:11 smithi022 ceph-mon[31985]: pgmap v2207: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T19:18:13.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:18:13 smithi134 ceph-mon[35037]: pgmap v2208: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:18:13.931 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:18:13 smithi079 ceph-mon[35312]: pgmap v2208: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:18:14.031 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:18:13 smithi022 ceph-mon[31985]: pgmap v2208: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:18:15.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:18:15 smithi134 ceph-mon[35037]: 2 slow requests, 0 included below; oldest blocked for > 3681.897521 secs 2024-03-28T19:18:15.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:18:15 smithi134 ceph-mon[35037]: pgmap v2209: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T19:18:15.931 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:18:15 smithi079 ceph-mon[35312]: 2 slow requests, 0 included below; oldest blocked for > 3681.897521 secs 2024-03-28T19:18:15.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:18:15 smithi079 ceph-mon[35312]: pgmap v2209: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T19:18:16.031 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:18:15 smithi022 ceph-mon[31985]: 2 slow requests, 0 included below; oldest blocked for > 3681.897521 secs 2024-03-28T19:18:16.031 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:18:15 smithi022 ceph-mon[31985]: pgmap v2209: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T19:18:17.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:18:17 smithi079 ceph-mon[35312]: pgmap v2210: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:18:18.031 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:18:17 smithi022 ceph-mon[31985]: pgmap v2210: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:18:18.174 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:18:17 smithi134 ceph-mon[35037]: pgmap v2210: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:18:19.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:18:19 smithi079 ceph-mon[35312]: pgmap v2211: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:18:20.031 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:18:19 smithi022 ceph-mon[31985]: pgmap v2211: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:18:20.174 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:18:19 smithi134 ceph-mon[35037]: pgmap v2211: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:18:21.031 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:18:20 smithi022 ceph-mon[31985]: 2 slow requests, 0 included below; oldest blocked for > 3686.897636 secs 2024-03-28T19:18:21.075 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:18:20 smithi134 ceph-mon[35037]: 2 slow requests, 0 included below; oldest blocked for > 3686.897636 secs 2024-03-28T19:18:21.181 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:18:20 smithi079 ceph-mon[35312]: 2 slow requests, 0 included below; oldest blocked for > 3686.897636 secs 2024-03-28T19:18:22.031 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:18:21 smithi022 ceph-mon[31985]: pgmap v2212: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T19:18:22.174 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:18:21 smithi134 ceph-mon[35037]: pgmap v2212: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T19:18:22.182 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:18:21 smithi079 ceph-mon[35312]: pgmap v2212: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T19:18:24.031 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:18:23 smithi022 ceph-mon[31985]: pgmap v2213: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:18:24.174 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:18:23 smithi134 ceph-mon[35037]: pgmap v2213: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:18:24.181 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:18:23 smithi079 ceph-mon[35312]: pgmap v2213: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:18:26.031 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:18:25 smithi022 ceph-mon[31985]: 2 slow requests, 0 included below; oldest blocked for > 3691.897708 secs 2024-03-28T19:18:26.031 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:18:25 smithi022 ceph-mon[31985]: pgmap v2214: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T19:18:26.174 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:18:25 smithi134 ceph-mon[35037]: 2 slow requests, 0 included below; oldest blocked for > 3691.897708 secs 2024-03-28T19:18:26.174 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:18:25 smithi134 ceph-mon[35037]: pgmap v2214: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T19:18:26.182 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:18:25 smithi079 ceph-mon[35312]: 2 slow requests, 0 included below; oldest blocked for > 3691.897708 secs 2024-03-28T19:18:26.182 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:18:25 smithi079 ceph-mon[35312]: pgmap v2214: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T19:18:28.031 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:18:27 smithi022 ceph-mon[31985]: pgmap v2215: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:18:28.174 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:18:27 smithi134 ceph-mon[35037]: pgmap v2215: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:18:28.181 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:18:27 smithi079 ceph-mon[35312]: pgmap v2215: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:18:30.031 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:18:29 smithi022 ceph-mon[31985]: pgmap v2216: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:18:30.174 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:18:29 smithi134 ceph-mon[35037]: pgmap v2216: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:18:30.181 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:18:29 smithi079 ceph-mon[35312]: pgmap v2216: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:18:31.031 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:18:30 smithi022 ceph-mon[31985]: 2 slow requests, 0 included below; oldest blocked for > 3696.897826 secs 2024-03-28T19:18:31.083 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:18:30 smithi079 ceph-mon[35312]: 2 slow requests, 0 included below; oldest blocked for > 3696.897826 secs 2024-03-28T19:18:31.173 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:18:30 smithi134 ceph-mon[35037]: 2 slow requests, 0 included below; oldest blocked for > 3696.897826 secs 2024-03-28T19:18:32.031 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:18:31 smithi022 ceph-mon[31985]: pgmap v2217: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T19:18:32.174 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:18:31 smithi134 ceph-mon[35037]: pgmap v2217: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T19:18:32.181 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:18:31 smithi079 ceph-mon[35312]: pgmap v2217: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T19:18:34.031 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:18:33 smithi022 ceph-mon[31985]: pgmap v2218: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:18:34.174 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:18:33 smithi134 ceph-mon[35037]: pgmap v2218: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:18:34.181 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:18:33 smithi079 ceph-mon[35312]: pgmap v2218: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:18:36.031 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:18:35 smithi022 ceph-mon[31985]: 2 slow requests, 0 included below; oldest blocked for > 3701.897919 secs 2024-03-28T19:18:36.032 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:18:35 smithi022 ceph-mon[31985]: pgmap v2219: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T19:18:36.174 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:18:35 smithi134 ceph-mon[35037]: 2 slow requests, 0 included below; oldest blocked for > 3701.897919 secs 2024-03-28T19:18:36.174 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:18:35 smithi134 ceph-mon[35037]: pgmap v2219: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T19:18:36.181 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:18:35 smithi079 ceph-mon[35312]: 2 slow requests, 0 included below; oldest blocked for > 3701.897919 secs 2024-03-28T19:18:36.182 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:18:35 smithi079 ceph-mon[35312]: pgmap v2219: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T19:18:38.034 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:18:37 smithi022 ceph-mon[31985]: pgmap v2220: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:18:38.034 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:18:37 smithi022 ceph-mon[31985]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T19:18:38.174 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:18:37 smithi134 ceph-mon[35037]: pgmap v2220: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:18:38.174 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:18:37 smithi134 ceph-mon[35037]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T19:18:38.182 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:18:37 smithi079 ceph-mon[35312]: pgmap v2220: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:18:38.182 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:18:37 smithi079 ceph-mon[35312]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T19:18:39.174 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:18:38 smithi134 ceph-mon[35037]: from='mgr.15522 ' entity='mgr.x' 2024-03-28T19:18:39.175 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:18:38 smithi134 ceph-mon[35037]: from='mgr.15522 ' entity='mgr.x' 2024-03-28T19:18:39.175 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:18:38 smithi134 ceph-mon[35037]: from='mgr.15522 ' entity='mgr.x' 2024-03-28T19:18:39.175 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:18:38 smithi134 ceph-mon[35037]: from='mgr.15522 ' entity='mgr.x' 2024-03-28T19:18:39.175 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:18:38 smithi134 ceph-mon[35037]: pgmap v2221: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:18:39.182 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:18:38 smithi079 ceph-mon[35312]: from='mgr.15522 ' entity='mgr.x' 2024-03-28T19:18:39.182 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:18:38 smithi079 ceph-mon[35312]: from='mgr.15522 ' entity='mgr.x' 2024-03-28T19:18:39.182 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:18:38 smithi079 ceph-mon[35312]: from='mgr.15522 ' entity='mgr.x' 2024-03-28T19:18:39.182 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:18:38 smithi079 ceph-mon[35312]: from='mgr.15522 ' entity='mgr.x' 2024-03-28T19:18:39.182 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:18:38 smithi079 ceph-mon[35312]: pgmap v2221: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:18:39.281 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:18:38 smithi022 ceph-mon[31985]: from='mgr.15522 ' entity='mgr.x' 2024-03-28T19:18:39.281 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:18:38 smithi022 ceph-mon[31985]: from='mgr.15522 ' entity='mgr.x' 2024-03-28T19:18:39.282 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:18:38 smithi022 ceph-mon[31985]: from='mgr.15522 ' entity='mgr.x' 2024-03-28T19:18:39.282 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:18:38 smithi022 ceph-mon[31985]: from='mgr.15522 ' entity='mgr.x' 2024-03-28T19:18:39.282 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:18:38 smithi022 ceph-mon[31985]: pgmap v2221: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:18:40.175 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:18:39 smithi134 ceph-mon[35037]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T19:18:40.175 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:18:39 smithi134 ceph-mon[35037]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T19:18:40.175 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:18:39 smithi134 ceph-mon[35037]: pgmap v2222: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 11 op/s 2024-03-28T19:18:40.175 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:18:39 smithi134 ceph-mon[35037]: pgmap v2223: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 9 op/s 2024-03-28T19:18:40.175 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:18:39 smithi134 ceph-mon[35037]: from='mgr.15522 ' entity='mgr.x' 2024-03-28T19:18:40.175 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:18:39 smithi134 ceph-mon[35037]: 2 slow requests, 0 included below; oldest blocked for > 3706.897948 secs 2024-03-28T19:18:40.181 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:18:39 smithi079 ceph-mon[35312]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T19:18:40.182 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:18:39 smithi079 ceph-mon[35312]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T19:18:40.182 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:18:39 smithi079 ceph-mon[35312]: pgmap v2222: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 11 op/s 2024-03-28T19:18:40.182 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:18:39 smithi079 ceph-mon[35312]: pgmap v2223: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 9 op/s 2024-03-28T19:18:40.182 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:18:39 smithi079 ceph-mon[35312]: from='mgr.15522 ' entity='mgr.x' 2024-03-28T19:18:40.182 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:18:39 smithi079 ceph-mon[35312]: 2 slow requests, 0 included below; oldest blocked for > 3706.897948 secs 2024-03-28T19:18:40.281 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:18:39 smithi022 ceph-mon[31985]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T19:18:40.282 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:18:39 smithi022 ceph-mon[31985]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T19:18:40.282 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:18:39 smithi022 ceph-mon[31985]: pgmap v2222: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 11 op/s 2024-03-28T19:18:40.282 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:18:39 smithi022 ceph-mon[31985]: pgmap v2223: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 9 op/s 2024-03-28T19:18:40.282 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:18:39 smithi022 ceph-mon[31985]: from='mgr.15522 ' entity='mgr.x' 2024-03-28T19:18:40.282 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:18:39 smithi022 ceph-mon[31985]: 2 slow requests, 0 included below; oldest blocked for > 3706.897948 secs 2024-03-28T19:18:42.674 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:18:42 smithi134 ceph-mon[35037]: pgmap v2224: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 9 op/s 2024-03-28T19:18:42.681 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:18:42 smithi079 ceph-mon[35312]: pgmap v2224: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 9 op/s 2024-03-28T19:18:42.781 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:18:42 smithi022 ceph-mon[31985]: pgmap v2224: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 9 op/s 2024-03-28T19:18:44.674 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:18:44 smithi134 ceph-mon[35037]: pgmap v2225: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s rd, 4 op/s 2024-03-28T19:18:44.681 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:18:44 smithi079 ceph-mon[35312]: pgmap v2225: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s rd, 4 op/s 2024-03-28T19:18:44.781 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:18:44 smithi022 ceph-mon[31985]: pgmap v2225: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s rd, 4 op/s 2024-03-28T19:18:45.674 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:18:45 smithi134 ceph-mon[35037]: 2 slow requests, 0 included below; oldest blocked for > 3711.897974 secs 2024-03-28T19:18:45.681 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:18:45 smithi079 ceph-mon[35312]: 2 slow requests, 0 included below; oldest blocked for > 3711.897974 secs 2024-03-28T19:18:45.781 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:18:45 smithi022 ceph-mon[31985]: 2 slow requests, 0 included below; oldest blocked for > 3711.897974 secs 2024-03-28T19:18:46.674 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:18:46 smithi134 ceph-mon[35037]: pgmap v2226: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 9 op/s 2024-03-28T19:18:46.681 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:18:46 smithi079 ceph-mon[35312]: pgmap v2226: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 9 op/s 2024-03-28T19:18:46.781 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:18:46 smithi022 ceph-mon[31985]: pgmap v2226: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 9 op/s 2024-03-28T19:18:48.674 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:18:48 smithi134 ceph-mon[35037]: pgmap v2227: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 9 op/s 2024-03-28T19:18:48.681 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:18:48 smithi079 ceph-mon[35312]: pgmap v2227: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 9 op/s 2024-03-28T19:18:48.781 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:18:48 smithi022 ceph-mon[31985]: pgmap v2227: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 9 op/s 2024-03-28T19:18:50.674 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:18:50 smithi134 ceph-mon[35037]: pgmap v2228: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 7 op/s 2024-03-28T19:18:50.674 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:18:50 smithi134 ceph-mon[35037]: 2 slow requests, 0 included below; oldest blocked for > 3716.898041 secs 2024-03-28T19:18:50.681 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:18:50 smithi079 ceph-mon[35312]: pgmap v2228: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 7 op/s 2024-03-28T19:18:50.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:18:50 smithi079 ceph-mon[35312]: 2 slow requests, 0 included below; oldest blocked for > 3716.898041 secs 2024-03-28T19:18:50.781 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:18:50 smithi022 ceph-mon[31985]: pgmap v2228: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 7 op/s 2024-03-28T19:18:50.781 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:18:50 smithi022 ceph-mon[31985]: 2 slow requests, 0 included below; oldest blocked for > 3716.898041 secs 2024-03-28T19:18:52.674 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:18:52 smithi134 ceph-mon[35037]: pgmap v2229: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:18:52.681 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:18:52 smithi079 ceph-mon[35312]: pgmap v2229: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:18:52.781 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:18:52 smithi022 ceph-mon[31985]: pgmap v2229: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:18:54.674 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:18:54 smithi134 ceph-mon[35037]: pgmap v2230: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:18:54.681 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:18:54 smithi079 ceph-mon[35312]: pgmap v2230: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:18:54.781 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:18:54 smithi022 ceph-mon[31985]: pgmap v2230: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:18:55.674 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:18:55 smithi134 ceph-mon[35037]: 2 slow requests, 0 included below; oldest blocked for > 3721.898065 secs 2024-03-28T19:18:55.681 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:18:55 smithi079 ceph-mon[35312]: 2 slow requests, 0 included below; oldest blocked for > 3721.898065 secs 2024-03-28T19:18:55.781 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:18:55 smithi022 ceph-mon[31985]: 2 slow requests, 0 included below; oldest blocked for > 3721.898065 secs 2024-03-28T19:18:56.674 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:18:56 smithi134 ceph-mon[35037]: pgmap v2231: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T19:18:56.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:18:56 smithi079 ceph-mon[35312]: pgmap v2231: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T19:18:56.781 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:18:56 smithi022 ceph-mon[31985]: pgmap v2231: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T19:18:58.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:18:58 smithi134 ceph-mon[35037]: pgmap v2232: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:18:58.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:18:58 smithi079 ceph-mon[35312]: pgmap v2232: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:18:58.781 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:18:58 smithi022 ceph-mon[31985]: pgmap v2232: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:19:00.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:19:00 smithi079 ceph-mon[35312]: pgmap v2233: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T19:19:00.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:19:00 smithi079 ceph-mon[35312]: 2 slow requests, 0 included below; oldest blocked for > 3726.898128 secs 2024-03-28T19:19:00.781 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:19:00 smithi022 ceph-mon[31985]: pgmap v2233: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T19:19:00.781 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:19:00 smithi022 ceph-mon[31985]: 2 slow requests, 0 included below; oldest blocked for > 3726.898128 secs 2024-03-28T19:19:00.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:19:00 smithi134 ceph-mon[35037]: pgmap v2233: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T19:19:00.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:19:00 smithi134 ceph-mon[35037]: 2 slow requests, 0 included below; oldest blocked for > 3726.898128 secs 2024-03-28T19:19:02.781 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:19:02 smithi022 ceph-mon[31985]: pgmap v2234: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:19:02.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:19:02 smithi134 ceph-mon[35037]: pgmap v2234: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:19:02.931 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:19:02 smithi079 ceph-mon[35312]: pgmap v2234: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:19:04.781 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:19:04 smithi022 ceph-mon[31985]: pgmap v2235: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:19:04.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:19:04 smithi134 ceph-mon[35037]: pgmap v2235: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:19:04.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:19:04 smithi079 ceph-mon[35312]: pgmap v2235: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:19:05.781 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:19:05 smithi022 ceph-mon[31985]: 2 slow requests, 0 included below; oldest blocked for > 3731.898243 secs 2024-03-28T19:19:05.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:19:05 smithi134 ceph-mon[35037]: 2 slow requests, 0 included below; oldest blocked for > 3731.898243 secs 2024-03-28T19:19:05.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:19:05 smithi079 ceph-mon[35312]: 2 slow requests, 0 included below; oldest blocked for > 3731.898243 secs 2024-03-28T19:19:06.781 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:19:06 smithi022 ceph-mon[31985]: pgmap v2236: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T19:19:06.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:19:06 smithi134 ceph-mon[35037]: pgmap v2236: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T19:19:06.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:19:06 smithi079 ceph-mon[35312]: pgmap v2236: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T19:19:08.781 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:19:08 smithi022 ceph-mon[31985]: pgmap v2237: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:19:08.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:19:08 smithi134 ceph-mon[35037]: pgmap v2237: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:19:08.931 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:19:08 smithi079 ceph-mon[35312]: pgmap v2237: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:19:10.781 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:19:10 smithi022 ceph-mon[31985]: pgmap v2238: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T19:19:10.781 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:19:10 smithi022 ceph-mon[31985]: 2 slow requests, 0 included below; oldest blocked for > 3736.898306 secs 2024-03-28T19:19:10.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:19:10 smithi134 ceph-mon[35037]: pgmap v2238: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T19:19:10.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:19:10 smithi134 ceph-mon[35037]: 2 slow requests, 0 included below; oldest blocked for > 3736.898306 secs 2024-03-28T19:19:10.931 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:19:10 smithi079 ceph-mon[35312]: pgmap v2238: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T19:19:10.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:19:10 smithi079 ceph-mon[35312]: 2 slow requests, 0 included below; oldest blocked for > 3736.898306 secs 2024-03-28T19:19:12.783 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:19:12 smithi022 ceph-mon[31985]: pgmap v2239: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:19:12.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:19:12 smithi134 ceph-mon[35037]: pgmap v2239: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:19:12.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:19:12 smithi079 ceph-mon[35312]: pgmap v2239: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:19:14.781 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:19:14 smithi022 ceph-mon[31985]: pgmap v2240: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:19:14.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:19:14 smithi134 ceph-mon[35037]: pgmap v2240: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:19:14.931 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:19:14 smithi079 ceph-mon[35312]: pgmap v2240: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:19:15.781 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:19:15 smithi022 ceph-mon[31985]: 2 slow requests, 0 included below; oldest blocked for > 3741.898341 secs 2024-03-28T19:19:15.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:19:15 smithi134 ceph-mon[35037]: 2 slow requests, 0 included below; oldest blocked for > 3741.898341 secs 2024-03-28T19:19:15.931 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:19:15 smithi079 ceph-mon[35312]: 2 slow requests, 0 included below; oldest blocked for > 3741.898341 secs 2024-03-28T19:19:16.781 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:19:16 smithi022 ceph-mon[31985]: pgmap v2241: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T19:19:16.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:19:16 smithi134 ceph-mon[35037]: pgmap v2241: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T19:19:16.931 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:19:16 smithi079 ceph-mon[35312]: pgmap v2241: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T19:19:18.781 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:19:18 smithi022 ceph-mon[31985]: pgmap v2242: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:19:18.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:19:18 smithi134 ceph-mon[35037]: pgmap v2242: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:19:18.931 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:19:18 smithi079 ceph-mon[35312]: pgmap v2242: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:19:20.781 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:19:20 smithi022 ceph-mon[31985]: pgmap v2243: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T19:19:20.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:19:20 smithi022 ceph-mon[31985]: 2 slow requests, 0 included below; oldest blocked for > 3746.898404 secs 2024-03-28T19:19:20.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:19:20 smithi134 ceph-mon[35037]: pgmap v2243: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T19:19:20.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:19:20 smithi134 ceph-mon[35037]: 2 slow requests, 0 included below; oldest blocked for > 3746.898404 secs 2024-03-28T19:19:20.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:19:20 smithi079 ceph-mon[35312]: pgmap v2243: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T19:19:20.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:19:20 smithi079 ceph-mon[35312]: 2 slow requests, 0 included below; oldest blocked for > 3746.898404 secs 2024-03-28T19:19:22.781 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:19:22 smithi022 ceph-mon[31985]: pgmap v2244: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:19:22.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:19:22 smithi134 ceph-mon[35037]: pgmap v2244: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:19:22.934 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:19:22 smithi079 ceph-mon[35312]: pgmap v2244: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:19:24.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:19:24 smithi022 ceph-mon[31985]: pgmap v2245: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:19:24.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:19:24 smithi134 ceph-mon[35037]: pgmap v2245: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:19:24.931 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:19:24 smithi079 ceph-mon[35312]: pgmap v2245: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:19:25.781 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:19:25 smithi022 ceph-mon[31985]: 2 slow requests, 0 included below; oldest blocked for > 3751.898467 secs 2024-03-28T19:19:25.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:19:25 smithi134 ceph-mon[35037]: 2 slow requests, 0 included below; oldest blocked for > 3751.898467 secs 2024-03-28T19:19:25.931 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:19:25 smithi079 ceph-mon[35312]: 2 slow requests, 0 included below; oldest blocked for > 3751.898467 secs 2024-03-28T19:19:26.781 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:19:26 smithi022 ceph-mon[31985]: pgmap v2246: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T19:19:26.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:19:26 smithi134 ceph-mon[35037]: pgmap v2246: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T19:19:26.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:19:26 smithi079 ceph-mon[35312]: pgmap v2246: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T19:19:28.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:19:28 smithi134 ceph-mon[35037]: pgmap v2247: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:19:28.931 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:19:28 smithi079 ceph-mon[35312]: pgmap v2247: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:19:29.031 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:19:28 smithi022 ceph-mon[31985]: pgmap v2247: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:19:30.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:19:30 smithi134 ceph-mon[35037]: pgmap v2248: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T19:19:30.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:19:30 smithi134 ceph-mon[35037]: 2 slow requests, 0 included below; oldest blocked for > 3756.898541 secs 2024-03-28T19:19:30.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:19:30 smithi079 ceph-mon[35312]: pgmap v2248: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T19:19:30.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:19:30 smithi079 ceph-mon[35312]: 2 slow requests, 0 included below; oldest blocked for > 3756.898541 secs 2024-03-28T19:19:31.031 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:19:30 smithi022 ceph-mon[31985]: pgmap v2248: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T19:19:31.031 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:19:30 smithi022 ceph-mon[31985]: 2 slow requests, 0 included below; oldest blocked for > 3756.898541 secs 2024-03-28T19:19:32.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:19:32 smithi134 ceph-mon[35037]: pgmap v2249: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:19:32.931 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:19:32 smithi079 ceph-mon[35312]: pgmap v2249: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:19:33.031 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:19:32 smithi022 ceph-mon[31985]: pgmap v2249: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:19:34.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:19:34 smithi134 ceph-mon[35037]: pgmap v2250: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:19:34.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:19:34 smithi079 ceph-mon[35312]: pgmap v2250: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:19:35.031 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:19:34 smithi022 ceph-mon[31985]: pgmap v2250: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:19:35.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:19:35 smithi134 ceph-mon[35037]: 2 slow requests, 0 included below; oldest blocked for > 3761.898571 secs 2024-03-28T19:19:35.931 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:19:35 smithi079 ceph-mon[35312]: 2 slow requests, 0 included below; oldest blocked for > 3761.898571 secs 2024-03-28T19:19:36.031 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:19:35 smithi022 ceph-mon[31985]: 2 slow requests, 0 included below; oldest blocked for > 3761.898571 secs 2024-03-28T19:19:36.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:19:36 smithi134 ceph-mon[35037]: pgmap v2251: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T19:19:36.931 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:19:36 smithi079 ceph-mon[35312]: pgmap v2251: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T19:19:37.031 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:19:36 smithi022 ceph-mon[31985]: pgmap v2251: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T19:19:38.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:19:38 smithi134 ceph-mon[35037]: pgmap v2252: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:19:38.931 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:19:38 smithi079 ceph-mon[35312]: pgmap v2252: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:19:39.031 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:19:38 smithi022 ceph-mon[31985]: pgmap v2252: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:19:40.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:19:40 smithi134 ceph-mon[35037]: pgmap v2253: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T19:19:40.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:19:40 smithi134 ceph-mon[35037]: 2 slow requests, 0 included below; oldest blocked for > 3766.898634 secs 2024-03-28T19:19:40.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:19:40 smithi134 ceph-mon[35037]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T19:19:40.931 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:19:40 smithi079 ceph-mon[35312]: pgmap v2253: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T19:19:40.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:19:40 smithi079 ceph-mon[35312]: 2 slow requests, 0 included below; oldest blocked for > 3766.898634 secs 2024-03-28T19:19:40.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:19:40 smithi079 ceph-mon[35312]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T19:19:41.031 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:19:40 smithi022 ceph-mon[31985]: pgmap v2253: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T19:19:41.031 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:19:40 smithi022 ceph-mon[31985]: 2 slow requests, 0 included below; oldest blocked for > 3766.898634 secs 2024-03-28T19:19:41.031 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:19:40 smithi022 ceph-mon[31985]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T19:19:41.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:19:41 smithi134 ceph-mon[35037]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T19:19:41.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:19:41 smithi134 ceph-mon[35037]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T19:19:41.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:19:41 smithi134 ceph-mon[35037]: from='mgr.15522 ' entity='mgr.x' 2024-03-28T19:19:41.931 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:19:41 smithi079 ceph-mon[35312]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T19:19:41.931 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:19:41 smithi079 ceph-mon[35312]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T19:19:41.931 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:19:41 smithi079 ceph-mon[35312]: from='mgr.15522 ' entity='mgr.x' 2024-03-28T19:19:42.031 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:19:41 smithi022 ceph-mon[31985]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T19:19:42.031 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:19:41 smithi022 ceph-mon[31985]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T19:19:42.031 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:19:41 smithi022 ceph-mon[31985]: from='mgr.15522 ' entity='mgr.x' 2024-03-28T19:19:42.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:19:42 smithi134 ceph-mon[35037]: pgmap v2254: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 6 op/s 2024-03-28T19:19:42.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:19:42 smithi134 ceph-mon[35037]: pgmap v2255: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-03-28T19:19:42.931 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:19:42 smithi079 ceph-mon[35312]: pgmap v2254: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 6 op/s 2024-03-28T19:19:42.931 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:19:42 smithi079 ceph-mon[35312]: pgmap v2255: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-03-28T19:19:43.031 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:19:42 smithi022 ceph-mon[31985]: pgmap v2254: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 6 op/s 2024-03-28T19:19:43.031 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:19:42 smithi022 ceph-mon[31985]: pgmap v2255: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-03-28T19:19:44.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:19:44 smithi134 ceph-mon[35037]: pgmap v2256: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-03-28T19:19:44.931 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:19:44 smithi079 ceph-mon[35312]: pgmap v2256: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-03-28T19:19:45.031 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:19:44 smithi022 ceph-mon[31985]: pgmap v2256: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-03-28T19:19:45.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:19:45 smithi134 ceph-mon[35037]: 2 slow requests, 0 included below; oldest blocked for > 3771.898668 secs 2024-03-28T19:19:45.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:19:45 smithi079 ceph-mon[35312]: 2 slow requests, 0 included below; oldest blocked for > 3771.898668 secs 2024-03-28T19:19:46.031 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:19:45 smithi022 ceph-mon[31985]: 2 slow requests, 0 included below; oldest blocked for > 3771.898668 secs 2024-03-28T19:19:46.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:19:46 smithi134 ceph-mon[35037]: pgmap v2257: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-03-28T19:19:46.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:19:46 smithi079 ceph-mon[35312]: pgmap v2257: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-03-28T19:19:47.031 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:19:46 smithi022 ceph-mon[31985]: pgmap v2257: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-03-28T19:19:48.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:19:48 smithi134 ceph-mon[35037]: pgmap v2258: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-03-28T19:19:48.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:19:48 smithi079 ceph-mon[35312]: pgmap v2258: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-03-28T19:19:49.031 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:19:48 smithi022 ceph-mon[31985]: pgmap v2258: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-03-28T19:19:50.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:19:50 smithi134 ceph-mon[35037]: pgmap v2259: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 KiB/s rd, 4 op/s 2024-03-28T19:19:50.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:19:50 smithi134 ceph-mon[35037]: 2 slow requests, 0 included below; oldest blocked for > 3776.898759 secs 2024-03-28T19:19:50.931 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:19:50 smithi079 ceph-mon[35312]: pgmap v2259: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 KiB/s rd, 4 op/s 2024-03-28T19:19:50.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:19:50 smithi079 ceph-mon[35312]: 2 slow requests, 0 included below; oldest blocked for > 3776.898759 secs 2024-03-28T19:19:51.031 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:19:50 smithi022 ceph-mon[31985]: pgmap v2259: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 KiB/s rd, 4 op/s 2024-03-28T19:19:51.031 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:19:50 smithi022 ceph-mon[31985]: 2 slow requests, 0 included below; oldest blocked for > 3776.898759 secs 2024-03-28T19:19:52.923 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:19:52 smithi134 ceph-mon[35037]: pgmap v2260: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 7 op/s 2024-03-28T19:19:52.931 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:19:52 smithi079 ceph-mon[35312]: pgmap v2260: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 7 op/s 2024-03-28T19:19:53.031 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:19:52 smithi022 ceph-mon[31985]: pgmap v2260: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 7 op/s 2024-03-28T19:19:54.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:19:54 smithi134 ceph-mon[35037]: pgmap v2261: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:19:54.931 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:19:54 smithi079 ceph-mon[35312]: pgmap v2261: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:19:55.031 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:19:54 smithi022 ceph-mon[31985]: pgmap v2261: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:19:55.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:19:55 smithi134 ceph-mon[35037]: 2 slow requests, 0 included below; oldest blocked for > 3781.898843 secs 2024-03-28T19:19:55.931 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:19:55 smithi079 ceph-mon[35312]: 2 slow requests, 0 included below; oldest blocked for > 3781.898843 secs 2024-03-28T19:19:56.031 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:19:55 smithi022 ceph-mon[31985]: 2 slow requests, 0 included below; oldest blocked for > 3781.898843 secs 2024-03-28T19:19:56.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:19:56 smithi134 ceph-mon[35037]: pgmap v2262: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T19:19:56.931 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:19:56 smithi079 ceph-mon[35312]: pgmap v2262: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T19:19:57.031 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:19:56 smithi022 ceph-mon[31985]: pgmap v2262: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T19:19:58.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:19:58 smithi134 ceph-mon[35037]: pgmap v2263: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:19:58.931 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:19:58 smithi079 ceph-mon[35312]: pgmap v2263: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:19:59.031 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:19:58 smithi022 ceph-mon[31985]: pgmap v2263: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:20:00.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:20:00 smithi134 ceph-mon[35037]: pgmap v2264: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:20:00.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:20:00 smithi134 ceph-mon[35037]: 2 slow requests, 0 included below; oldest blocked for > 3786.898910 secs 2024-03-28T19:20:00.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:20:00 smithi134 ceph-mon[35037]: overall HEALTH_WARN 4 failed cephadm daemon(s); 1 MDSs report slow requests 2024-03-28T19:20:00.931 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:20:00 smithi079 ceph-mon[35312]: pgmap v2264: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:20:00.931 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:20:00 smithi079 ceph-mon[35312]: 2 slow requests, 0 included below; oldest blocked for > 3786.898910 secs 2024-03-28T19:20:00.931 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:20:00 smithi079 ceph-mon[35312]: overall HEALTH_WARN 4 failed cephadm daemon(s); 1 MDSs report slow requests 2024-03-28T19:20:01.031 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:20:00 smithi022 ceph-mon[31985]: pgmap v2264: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:20:01.031 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:20:00 smithi022 ceph-mon[31985]: 2 slow requests, 0 included below; oldest blocked for > 3786.898910 secs 2024-03-28T19:20:01.031 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:20:00 smithi022 ceph-mon[31985]: overall HEALTH_WARN 4 failed cephadm daemon(s); 1 MDSs report slow requests 2024-03-28T19:20:02.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:20:02 smithi079 ceph-mon[35312]: pgmap v2265: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T19:20:03.031 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:20:02 smithi022 ceph-mon[31985]: pgmap v2265: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T19:20:03.173 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:20:02 smithi134 ceph-mon[35037]: pgmap v2265: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T19:20:04.931 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:20:04 smithi079 ceph-mon[35312]: pgmap v2266: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:20:05.031 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:20:04 smithi022 ceph-mon[31985]: pgmap v2266: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:20:05.174 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:20:04 smithi134 ceph-mon[35037]: pgmap v2266: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:20:06.031 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:20:05 smithi022 ceph-mon[31985]: 2 slow requests, 0 included below; oldest blocked for > 3791.898994 secs 2024-03-28T19:20:06.173 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:20:05 smithi134 ceph-mon[35037]: 2 slow requests, 0 included below; oldest blocked for > 3791.898994 secs 2024-03-28T19:20:06.181 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:20:05 smithi079 ceph-mon[35312]: 2 slow requests, 0 included below; oldest blocked for > 3791.898994 secs 2024-03-28T19:20:07.031 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:20:06 smithi022 ceph-mon[31985]: pgmap v2267: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T19:20:07.173 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:20:06 smithi134 ceph-mon[35037]: pgmap v2267: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T19:20:07.181 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:20:06 smithi079 ceph-mon[35312]: pgmap v2267: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T19:20:09.031 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:20:08 smithi022 ceph-mon[31985]: pgmap v2268: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:20:09.174 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:20:08 smithi134 ceph-mon[35037]: pgmap v2268: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:20:09.181 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:20:08 smithi079 ceph-mon[35312]: pgmap v2268: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:20:11.031 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:20:10 smithi022 ceph-mon[31985]: pgmap v2269: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:20:11.031 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:20:10 smithi022 ceph-mon[31985]: 2 slow requests, 0 included below; oldest blocked for > 3796.899067 secs 2024-03-28T19:20:11.174 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:20:10 smithi134 ceph-mon[35037]: pgmap v2269: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:20:11.174 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:20:10 smithi134 ceph-mon[35037]: 2 slow requests, 0 included below; oldest blocked for > 3796.899067 secs 2024-03-28T19:20:11.181 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:20:10 smithi079 ceph-mon[35312]: pgmap v2269: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:20:11.181 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:20:10 smithi079 ceph-mon[35312]: 2 slow requests, 0 included below; oldest blocked for > 3796.899067 secs 2024-03-28T19:20:13.031 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:20:12 smithi022 ceph-mon[31985]: pgmap v2270: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T19:20:13.174 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:20:12 smithi134 ceph-mon[35037]: pgmap v2270: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T19:20:13.181 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:20:12 smithi079 ceph-mon[35312]: pgmap v2270: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T19:20:15.031 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:20:14 smithi022 ceph-mon[31985]: pgmap v2271: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:20:15.173 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:20:14 smithi134 ceph-mon[35037]: pgmap v2271: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:20:15.181 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:20:14 smithi079 ceph-mon[35312]: pgmap v2271: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:20:16.031 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:20:15 smithi022 ceph-mon[31985]: 2 slow requests, 0 included below; oldest blocked for > 3801.899164 secs 2024-03-28T19:20:16.173 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:20:15 smithi134 ceph-mon[35037]: 2 slow requests, 0 included below; oldest blocked for > 3801.899164 secs 2024-03-28T19:20:16.181 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:20:15 smithi079 ceph-mon[35312]: 2 slow requests, 0 included below; oldest blocked for > 3801.899164 secs 2024-03-28T19:20:17.031 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:20:16 smithi022 ceph-mon[31985]: pgmap v2272: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T19:20:17.174 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:20:16 smithi134 ceph-mon[35037]: pgmap v2272: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T19:20:17.181 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:20:16 smithi079 ceph-mon[35312]: pgmap v2272: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T19:20:19.031 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:20:18 smithi022 ceph-mon[31985]: pgmap v2273: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:20:19.174 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:20:18 smithi134 ceph-mon[35037]: pgmap v2273: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:20:19.181 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:20:18 smithi079 ceph-mon[35312]: pgmap v2273: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:20:21.031 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:20:20 smithi022 ceph-mon[31985]: pgmap v2274: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:20:21.031 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:20:20 smithi022 ceph-mon[31985]: 2 slow requests, 0 included below; oldest blocked for > 3806.899267 secs 2024-03-28T19:20:21.174 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:20:20 smithi134 ceph-mon[35037]: pgmap v2274: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:20:21.174 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:20:20 smithi134 ceph-mon[35037]: 2 slow requests, 0 included below; oldest blocked for > 3806.899267 secs 2024-03-28T19:20:21.181 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:20:20 smithi079 ceph-mon[35312]: pgmap v2274: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:20:21.181 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:20:20 smithi079 ceph-mon[35312]: 2 slow requests, 0 included below; oldest blocked for > 3806.899267 secs 2024-03-28T19:20:23.031 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:20:22 smithi022 ceph-mon[31985]: pgmap v2275: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T19:20:23.174 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:20:22 smithi134 ceph-mon[35037]: pgmap v2275: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T19:20:23.181 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:20:22 smithi079 ceph-mon[35312]: pgmap v2275: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T19:20:25.031 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:20:24 smithi022 ceph-mon[31985]: pgmap v2276: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:20:25.174 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:20:24 smithi134 ceph-mon[35037]: pgmap v2276: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:20:25.181 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:20:24 smithi079 ceph-mon[35312]: pgmap v2276: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:20:26.031 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:20:25 smithi022 ceph-mon[31985]: 2 slow requests, 0 included below; oldest blocked for > 3811.899393 secs 2024-03-28T19:20:26.174 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:20:25 smithi134 ceph-mon[35037]: 2 slow requests, 0 included below; oldest blocked for > 3811.899393 secs 2024-03-28T19:20:26.181 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:20:25 smithi079 ceph-mon[35312]: 2 slow requests, 0 included below; oldest blocked for > 3811.899393 secs 2024-03-28T19:20:27.031 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:20:26 smithi022 ceph-mon[31985]: pgmap v2277: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T19:20:27.174 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:20:26 smithi134 ceph-mon[35037]: pgmap v2277: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T19:20:27.181 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:20:26 smithi079 ceph-mon[35312]: pgmap v2277: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T19:20:29.174 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:20:28 smithi134 ceph-mon[35037]: pgmap v2278: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:20:29.181 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:20:28 smithi079 ceph-mon[35312]: pgmap v2278: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:20:29.281 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:20:28 smithi022 ceph-mon[31985]: pgmap v2278: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:20:31.174 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:20:30 smithi134 ceph-mon[35037]: pgmap v2279: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:20:31.174 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:20:30 smithi134 ceph-mon[35037]: 2 slow requests, 0 included below; oldest blocked for > 3816.899528 secs 2024-03-28T19:20:31.181 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:20:30 smithi079 ceph-mon[35312]: pgmap v2279: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:20:31.181 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:20:30 smithi079 ceph-mon[35312]: 2 slow requests, 0 included below; oldest blocked for > 3816.899528 secs 2024-03-28T19:20:31.281 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:20:30 smithi022 ceph-mon[31985]: pgmap v2279: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:20:31.281 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:20:30 smithi022 ceph-mon[31985]: 2 slow requests, 0 included below; oldest blocked for > 3816.899528 secs 2024-03-28T19:20:33.174 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:20:32 smithi134 ceph-mon[35037]: pgmap v2280: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T19:20:33.181 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:20:32 smithi079 ceph-mon[35312]: pgmap v2280: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T19:20:33.281 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:20:32 smithi022 ceph-mon[31985]: pgmap v2280: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T19:20:34.174 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:20:33 smithi134 ceph-mon[35037]: pgmap v2281: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:20:34.181 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:20:33 smithi079 ceph-mon[35312]: pgmap v2281: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:20:34.281 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:20:33 smithi022 ceph-mon[31985]: pgmap v2281: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:20:35.174 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:20:34 smithi134 ceph-mon[35037]: 2 slow requests, 0 included below; oldest blocked for > 3821.899613 secs 2024-03-28T19:20:35.181 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:20:34 smithi079 ceph-mon[35312]: 2 slow requests, 0 included below; oldest blocked for > 3821.899613 secs 2024-03-28T19:20:35.281 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:20:34 smithi022 ceph-mon[31985]: 2 slow requests, 0 included below; oldest blocked for > 3821.899613 secs 2024-03-28T19:20:36.174 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:20:35 smithi134 ceph-mon[35037]: pgmap v2282: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T19:20:36.181 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:20:35 smithi079 ceph-mon[35312]: pgmap v2282: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T19:20:36.281 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:20:35 smithi022 ceph-mon[31985]: pgmap v2282: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T19:20:38.531 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:20:38 smithi022 ceph-mon[31985]: pgmap v2283: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:20:38.674 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:20:38 smithi134 ceph-mon[35037]: pgmap v2283: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:20:38.681 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:20:38 smithi079 ceph-mon[35312]: pgmap v2283: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:20:40.531 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:20:40 smithi022 ceph-mon[31985]: pgmap v2284: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:20:40.531 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:20:40 smithi022 ceph-mon[31985]: 2 slow requests, 0 included below; oldest blocked for > 3826.899683 secs 2024-03-28T19:20:40.673 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:20:40 smithi134 ceph-mon[35037]: pgmap v2284: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:20:40.674 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:20:40 smithi134 ceph-mon[35037]: 2 slow requests, 0 included below; oldest blocked for > 3826.899683 secs 2024-03-28T19:20:40.681 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:20:40 smithi079 ceph-mon[35312]: pgmap v2284: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:20:40.681 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:20:40 smithi079 ceph-mon[35312]: 2 slow requests, 0 included below; oldest blocked for > 3826.899683 secs 2024-03-28T19:20:42.531 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:20:42 smithi022 ceph-mon[31985]: pgmap v2285: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T19:20:42.531 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:20:42 smithi022 ceph-mon[31985]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T19:20:42.674 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:20:42 smithi134 ceph-mon[35037]: pgmap v2285: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T19:20:42.674 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:20:42 smithi134 ceph-mon[35037]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T19:20:42.681 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:20:42 smithi079 ceph-mon[35312]: pgmap v2285: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T19:20:42.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:20:42 smithi079 ceph-mon[35312]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T19:20:43.673 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:20:43 smithi134 ceph-mon[35037]: from='mgr.15522 ' entity='mgr.x' 2024-03-28T19:20:43.674 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:20:43 smithi134 ceph-mon[35037]: from='mgr.15522 ' entity='mgr.x' 2024-03-28T19:20:43.681 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:20:43 smithi079 ceph-mon[35312]: from='mgr.15522 ' entity='mgr.x' 2024-03-28T19:20:43.681 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:20:43 smithi079 ceph-mon[35312]: from='mgr.15522 ' entity='mgr.x' 2024-03-28T19:20:43.780 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:20:43 smithi022 ceph-mon[31985]: from='mgr.15522 ' entity='mgr.x' 2024-03-28T19:20:43.781 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:20:43 smithi022 ceph-mon[31985]: from='mgr.15522 ' entity='mgr.x' 2024-03-28T19:20:44.681 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:20:44 smithi079 ceph-mon[35312]: pgmap v2286: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:20:44.781 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:20:44 smithi022 ceph-mon[31985]: pgmap v2286: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:20:44.923 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:20:44 smithi134 ceph-mon[35037]: pgmap v2286: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:20:45.781 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:20:45 smithi022 ceph-mon[31985]: 2 slow requests, 0 included below; oldest blocked for > 3831.899758 secs 2024-03-28T19:20:45.781 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:20:45 smithi022 ceph-mon[31985]: from='mgr.15522 ' entity='mgr.x' 2024-03-28T19:20:45.781 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:20:45 smithi022 ceph-mon[31985]: from='mgr.15522 ' entity='mgr.x' 2024-03-28T19:20:45.781 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:20:45 smithi022 ceph-mon[31985]: from='mgr.15522 ' entity='mgr.x' 2024-03-28T19:20:45.781 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:20:45 smithi022 ceph-mon[31985]: from='mgr.15522 ' entity='mgr.x' 2024-03-28T19:20:45.781 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:20:45 smithi022 ceph-mon[31985]: from='mgr.15522 ' entity='mgr.x' 2024-03-28T19:20:45.781 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:20:45 smithi022 ceph-mon[31985]: from='mgr.15522 ' entity='mgr.x' 2024-03-28T19:20:45.923 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:20:45 smithi134 ceph-mon[35037]: 2 slow requests, 0 included below; oldest blocked for > 3831.899758 secs 2024-03-28T19:20:45.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:20:45 smithi134 ceph-mon[35037]: from='mgr.15522 ' entity='mgr.x' 2024-03-28T19:20:45.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:20:45 smithi134 ceph-mon[35037]: from='mgr.15522 ' entity='mgr.x' 2024-03-28T19:20:45.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:20:45 smithi134 ceph-mon[35037]: from='mgr.15522 ' entity='mgr.x' 2024-03-28T19:20:45.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:20:45 smithi134 ceph-mon[35037]: from='mgr.15522 ' entity='mgr.x' 2024-03-28T19:20:45.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:20:45 smithi134 ceph-mon[35037]: from='mgr.15522 ' entity='mgr.x' 2024-03-28T19:20:45.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:20:45 smithi134 ceph-mon[35037]: from='mgr.15522 ' entity='mgr.x' 2024-03-28T19:20:45.931 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:20:45 smithi079 ceph-mon[35312]: 2 slow requests, 0 included below; oldest blocked for > 3831.899758 secs 2024-03-28T19:20:45.931 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:20:45 smithi079 ceph-mon[35312]: from='mgr.15522 ' entity='mgr.x' 2024-03-28T19:20:45.931 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:20:45 smithi079 ceph-mon[35312]: from='mgr.15522 ' entity='mgr.x' 2024-03-28T19:20:45.931 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:20:45 smithi079 ceph-mon[35312]: from='mgr.15522 ' entity='mgr.x' 2024-03-28T19:20:45.931 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:20:45 smithi079 ceph-mon[35312]: from='mgr.15522 ' entity='mgr.x' 2024-03-28T19:20:45.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:20:45 smithi079 ceph-mon[35312]: from='mgr.15522 ' entity='mgr.x' 2024-03-28T19:20:45.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:20:45 smithi079 ceph-mon[35312]: from='mgr.15522 ' entity='mgr.x' 2024-03-28T19:20:46.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:20:46 smithi134 ceph-mon[35037]: pgmap v2287: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T19:20:46.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:20:46 smithi134 ceph-mon[35037]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-28T19:20:46.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:20:46 smithi134 ceph-mon[35037]: from='mgr.15522 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-28T19:20:46.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:20:46 smithi134 ceph-mon[35037]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-28T19:20:46.925 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:20:46 smithi134 ceph-mon[35037]: from='mgr.15522 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-28T19:20:46.925 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:20:46 smithi134 ceph-mon[35037]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-03-28T19:20:46.925 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:20:46 smithi134 ceph-mon[35037]: from='mgr.15522 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-03-28T19:20:46.925 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:20:46 smithi134 ceph-mon[35037]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-03-28T19:20:46.925 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:20:46 smithi134 ceph-mon[35037]: from='mgr.15522 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-03-28T19:20:46.925 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:20:46 smithi134 ceph-mon[35037]: Adjusting osd_memory_target on smithi079 to 176.7M 2024-03-28T19:20:46.925 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:20:46 smithi134 ceph-mon[35037]: Unable to set osd_memory_target on smithi079 to 185386700: error parsing value: Value '185386700' is below minimum 939524096 2024-03-28T19:20:46.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:20:46 smithi079 ceph-mon[35312]: pgmap v2287: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T19:20:46.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:20:46 smithi079 ceph-mon[35312]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-28T19:20:46.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:20:46 smithi079 ceph-mon[35312]: from='mgr.15522 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-28T19:20:46.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:20:46 smithi079 ceph-mon[35312]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-28T19:20:46.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:20:46 smithi079 ceph-mon[35312]: from='mgr.15522 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-28T19:20:46.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:20:46 smithi079 ceph-mon[35312]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-03-28T19:20:46.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:20:46 smithi079 ceph-mon[35312]: from='mgr.15522 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-03-28T19:20:46.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:20:46 smithi079 ceph-mon[35312]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-03-28T19:20:46.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:20:46 smithi079 ceph-mon[35312]: from='mgr.15522 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-03-28T19:20:46.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:20:46 smithi079 ceph-mon[35312]: Adjusting osd_memory_target on smithi079 to 176.7M 2024-03-28T19:20:46.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:20:46 smithi079 ceph-mon[35312]: Unable to set osd_memory_target on smithi079 to 185386700: error parsing value: Value '185386700' is below minimum 939524096 2024-03-28T19:20:47.030 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:20:46 smithi022 ceph-mon[31985]: pgmap v2287: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T19:20:47.031 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:20:46 smithi022 ceph-mon[31985]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-28T19:20:47.031 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:20:46 smithi022 ceph-mon[31985]: from='mgr.15522 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-28T19:20:47.031 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:20:46 smithi022 ceph-mon[31985]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-28T19:20:47.031 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:20:46 smithi022 ceph-mon[31985]: from='mgr.15522 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-28T19:20:47.031 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:20:46 smithi022 ceph-mon[31985]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-03-28T19:20:47.031 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:20:46 smithi022 ceph-mon[31985]: from='mgr.15522 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-03-28T19:20:47.031 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:20:46 smithi022 ceph-mon[31985]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-03-28T19:20:47.031 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:20:46 smithi022 ceph-mon[31985]: from='mgr.15522 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-03-28T19:20:47.031 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:20:46 smithi022 ceph-mon[31985]: Adjusting osd_memory_target on smithi079 to 176.7M 2024-03-28T19:20:47.031 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:20:46 smithi022 ceph-mon[31985]: Unable to set osd_memory_target on smithi079 to 185386700: error parsing value: Value '185386700' is below minimum 939524096 2024-03-28T19:20:47.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:20:47 smithi134 ceph-mon[35037]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-28T19:20:47.925 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:20:47 smithi134 ceph-mon[35037]: from='mgr.15522 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-28T19:20:47.925 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:20:47 smithi134 ceph-mon[35037]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-28T19:20:47.925 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:20:47 smithi134 ceph-mon[35037]: from='mgr.15522 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-28T19:20:47.925 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:20:47 smithi134 ceph-mon[35037]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-03-28T19:20:47.925 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:20:47 smithi134 ceph-mon[35037]: from='mgr.15522 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-03-28T19:20:47.925 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:20:47 smithi134 ceph-mon[35037]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-03-28T19:20:47.925 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:20:47 smithi134 ceph-mon[35037]: from='mgr.15522 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-03-28T19:20:47.925 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:20:47 smithi134 ceph-mon[35037]: Adjusting osd_memory_target on smithi134 to 176.8M 2024-03-28T19:20:47.925 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:20:47 smithi134 ceph-mon[35037]: Unable to set osd_memory_target on smithi134 to 185393868: error parsing value: Value '185393868' is below minimum 939524096 2024-03-28T19:20:47.925 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:20:47 smithi134 ceph-mon[35037]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-28T19:20:47.925 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:20:47 smithi134 ceph-mon[35037]: from='mgr.15522 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-28T19:20:47.925 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:20:47 smithi134 ceph-mon[35037]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-28T19:20:47.925 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:20:47 smithi134 ceph-mon[35037]: from='mgr.15522 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-28T19:20:47.925 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:20:47 smithi134 ceph-mon[35037]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-03-28T19:20:47.926 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:20:47 smithi134 ceph-mon[35037]: from='mgr.15522 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-03-28T19:20:47.926 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:20:47 smithi134 ceph-mon[35037]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-03-28T19:20:47.926 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:20:47 smithi134 ceph-mon[35037]: from='mgr.15522 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-03-28T19:20:47.926 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:20:47 smithi134 ceph-mon[35037]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T19:20:47.926 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:20:47 smithi134 ceph-mon[35037]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T19:20:47.926 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:20:47 smithi134 ceph-mon[35037]: from='mgr.15522 ' entity='mgr.x' 2024-03-28T19:20:47.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:20:47 smithi079 ceph-mon[35312]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-28T19:20:47.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:20:47 smithi079 ceph-mon[35312]: from='mgr.15522 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-28T19:20:47.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:20:47 smithi079 ceph-mon[35312]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-28T19:20:47.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:20:47 smithi079 ceph-mon[35312]: from='mgr.15522 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-28T19:20:47.933 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:20:47 smithi079 ceph-mon[35312]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-03-28T19:20:47.933 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:20:47 smithi079 ceph-mon[35312]: from='mgr.15522 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-03-28T19:20:47.933 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:20:47 smithi079 ceph-mon[35312]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-03-28T19:20:47.933 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:20:47 smithi079 ceph-mon[35312]: from='mgr.15522 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-03-28T19:20:47.933 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:20:47 smithi079 ceph-mon[35312]: Adjusting osd_memory_target on smithi134 to 176.8M 2024-03-28T19:20:47.933 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:20:47 smithi079 ceph-mon[35312]: Unable to set osd_memory_target on smithi134 to 185393868: error parsing value: Value '185393868' is below minimum 939524096 2024-03-28T19:20:47.933 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:20:47 smithi079 ceph-mon[35312]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-28T19:20:47.933 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:20:47 smithi079 ceph-mon[35312]: from='mgr.15522 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-28T19:20:47.933 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:20:47 smithi079 ceph-mon[35312]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-28T19:20:47.933 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:20:47 smithi079 ceph-mon[35312]: from='mgr.15522 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-28T19:20:47.933 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:20:47 smithi079 ceph-mon[35312]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-03-28T19:20:47.933 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:20:47 smithi079 ceph-mon[35312]: from='mgr.15522 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-03-28T19:20:47.933 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:20:47 smithi079 ceph-mon[35312]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-03-28T19:20:47.933 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:20:47 smithi079 ceph-mon[35312]: from='mgr.15522 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-03-28T19:20:47.933 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:20:47 smithi079 ceph-mon[35312]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T19:20:47.933 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:20:47 smithi079 ceph-mon[35312]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T19:20:47.934 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:20:47 smithi079 ceph-mon[35312]: from='mgr.15522 ' entity='mgr.x' 2024-03-28T19:20:48.032 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:20:47 smithi022 ceph-mon[31985]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-28T19:20:48.032 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:20:47 smithi022 ceph-mon[31985]: from='mgr.15522 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-28T19:20:48.032 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:20:47 smithi022 ceph-mon[31985]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-28T19:20:48.032 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:20:47 smithi022 ceph-mon[31985]: from='mgr.15522 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-28T19:20:48.032 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:20:47 smithi022 ceph-mon[31985]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-03-28T19:20:48.032 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:20:47 smithi022 ceph-mon[31985]: from='mgr.15522 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-03-28T19:20:48.032 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:20:47 smithi022 ceph-mon[31985]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-03-28T19:20:48.032 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:20:47 smithi022 ceph-mon[31985]: from='mgr.15522 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-03-28T19:20:48.032 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:20:47 smithi022 ceph-mon[31985]: Adjusting osd_memory_target on smithi134 to 176.8M 2024-03-28T19:20:48.032 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:20:47 smithi022 ceph-mon[31985]: Unable to set osd_memory_target on smithi134 to 185393868: error parsing value: Value '185393868' is below minimum 939524096 2024-03-28T19:20:48.032 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:20:47 smithi022 ceph-mon[31985]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-28T19:20:48.032 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:20:47 smithi022 ceph-mon[31985]: from='mgr.15522 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-28T19:20:48.033 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:20:47 smithi022 ceph-mon[31985]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-28T19:20:48.033 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:20:47 smithi022 ceph-mon[31985]: from='mgr.15522 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-28T19:20:48.033 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:20:47 smithi022 ceph-mon[31985]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-03-28T19:20:48.033 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:20:47 smithi022 ceph-mon[31985]: from='mgr.15522 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-03-28T19:20:48.033 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:20:47 smithi022 ceph-mon[31985]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-03-28T19:20:48.033 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:20:47 smithi022 ceph-mon[31985]: from='mgr.15522 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-03-28T19:20:48.033 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:20:47 smithi022 ceph-mon[31985]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T19:20:48.033 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:20:47 smithi022 ceph-mon[31985]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T19:20:48.033 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:20:47 smithi022 ceph-mon[31985]: from='mgr.15522 ' entity='mgr.x' 2024-03-28T19:20:48.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:20:48 smithi134 ceph-mon[35037]: Adjusting osd_memory_target on smithi022 to 177.8M 2024-03-28T19:20:48.925 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:20:48 smithi134 ceph-mon[35037]: Unable to set osd_memory_target on smithi022 to 186497740: error parsing value: Value '186497740' is below minimum 939524096 2024-03-28T19:20:48.925 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:20:48 smithi134 ceph-mon[35037]: pgmap v2288: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 6 op/s 2024-03-28T19:20:48.925 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:20:48 smithi134 ceph-mon[35037]: pgmap v2289: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-03-28T19:20:48.931 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:20:48 smithi079 ceph-mon[35312]: Adjusting osd_memory_target on smithi022 to 177.8M 2024-03-28T19:20:48.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:20:48 smithi079 ceph-mon[35312]: Unable to set osd_memory_target on smithi022 to 186497740: error parsing value: Value '186497740' is below minimum 939524096 2024-03-28T19:20:48.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:20:48 smithi079 ceph-mon[35312]: pgmap v2288: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 6 op/s 2024-03-28T19:20:48.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:20:48 smithi079 ceph-mon[35312]: pgmap v2289: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-03-28T19:20:49.031 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:20:48 smithi022 ceph-mon[31985]: Adjusting osd_memory_target on smithi022 to 177.8M 2024-03-28T19:20:49.031 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:20:48 smithi022 ceph-mon[31985]: Unable to set osd_memory_target on smithi022 to 186497740: error parsing value: Value '186497740' is below minimum 939524096 2024-03-28T19:20:49.031 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:20:48 smithi022 ceph-mon[31985]: pgmap v2288: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 6 op/s 2024-03-28T19:20:49.031 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:20:48 smithi022 ceph-mon[31985]: pgmap v2289: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-03-28T19:20:50.834 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:20:50 smithi022 ceph-mon[31985]: pgmap v2290: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-03-28T19:20:50.835 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:20:50 smithi022 ceph-mon[31985]: 2 slow requests, 0 included below; oldest blocked for > 3836.899826 secs 2024-03-28T19:20:50.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:20:50 smithi134 ceph-mon[35037]: pgmap v2290: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-03-28T19:20:50.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:20:50 smithi134 ceph-mon[35037]: 2 slow requests, 0 included below; oldest blocked for > 3836.899826 secs 2024-03-28T19:20:50.931 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:20:50 smithi079 ceph-mon[35312]: pgmap v2290: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-03-28T19:20:50.931 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:20:50 smithi079 ceph-mon[35312]: 2 slow requests, 0 included below; oldest blocked for > 3836.899826 secs 2024-03-28T19:20:52.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:20:52 smithi134 ceph-mon[35037]: pgmap v2291: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-03-28T19:20:52.931 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:20:52 smithi079 ceph-mon[35312]: pgmap v2291: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-03-28T19:20:53.031 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:20:52 smithi022 ceph-mon[31985]: pgmap v2291: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-03-28T19:20:54.673 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:20:54 smithi134 ceph-mon[35037]: pgmap v2292: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-03-28T19:20:54.931 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:20:54 smithi079 ceph-mon[35312]: pgmap v2292: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-03-28T19:20:55.031 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:20:54 smithi022 ceph-mon[31985]: pgmap v2292: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-03-28T19:20:55.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:20:55 smithi134 ceph-mon[35037]: 2 slow requests, 0 included below; oldest blocked for > 3841.899906 secs 2024-03-28T19:20:55.931 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:20:55 smithi079 ceph-mon[35312]: 2 slow requests, 0 included below; oldest blocked for > 3841.899906 secs 2024-03-28T19:20:56.031 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:20:55 smithi022 ceph-mon[31985]: 2 slow requests, 0 included below; oldest blocked for > 3841.899906 secs 2024-03-28T19:20:56.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:20:56 smithi134 ceph-mon[35037]: pgmap v2293: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-03-28T19:20:56.931 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:20:56 smithi079 ceph-mon[35312]: pgmap v2293: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-03-28T19:20:57.031 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:20:56 smithi022 ceph-mon[31985]: pgmap v2293: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-03-28T19:20:58.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:20:58 smithi134 ceph-mon[35037]: pgmap v2294: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 7 op/s 2024-03-28T19:20:58.931 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:20:58 smithi079 ceph-mon[35312]: pgmap v2294: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 7 op/s 2024-03-28T19:20:59.031 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:20:58 smithi022 ceph-mon[31985]: pgmap v2294: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 7 op/s 2024-03-28T19:21:00.923 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:21:00 smithi134 ceph-mon[35037]: pgmap v2295: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:21:00.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:21:00 smithi134 ceph-mon[35037]: 2 slow requests, 0 included below; oldest blocked for > 3846.899968 secs 2024-03-28T19:21:00.931 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:21:00 smithi079 ceph-mon[35312]: pgmap v2295: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:21:00.931 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:21:00 smithi079 ceph-mon[35312]: 2 slow requests, 0 included below; oldest blocked for > 3846.899968 secs 2024-03-28T19:21:01.031 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:21:00 smithi022 ceph-mon[31985]: pgmap v2295: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:21:01.032 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:21:00 smithi022 ceph-mon[31985]: 2 slow requests, 0 included below; oldest blocked for > 3846.899968 secs 2024-03-28T19:21:02.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:21:02 smithi134 ceph-mon[35037]: pgmap v2296: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T19:21:02.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:21:02 smithi079 ceph-mon[35312]: pgmap v2296: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T19:21:02.938 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:21:02 smithi022 ceph-mon[31985]: pgmap v2296: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T19:21:04.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:21:04 smithi134 ceph-mon[35037]: pgmap v2297: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:21:04.931 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:21:04 smithi079 ceph-mon[35312]: pgmap v2297: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:21:05.031 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:21:04 smithi022 ceph-mon[31985]: pgmap v2297: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:21:05.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:21:05 smithi134 ceph-mon[35037]: 2 slow requests, 0 included below; oldest blocked for > 3851.900036 secs 2024-03-28T19:21:05.931 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:21:05 smithi079 ceph-mon[35312]: 2 slow requests, 0 included below; oldest blocked for > 3851.900036 secs 2024-03-28T19:21:06.031 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:21:05 smithi022 ceph-mon[31985]: 2 slow requests, 0 included below; oldest blocked for > 3851.900036 secs 2024-03-28T19:21:06.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:21:06 smithi134 ceph-mon[35037]: pgmap v2298: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T19:21:06.931 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:21:06 smithi079 ceph-mon[35312]: pgmap v2298: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T19:21:07.031 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:21:06 smithi022 ceph-mon[31985]: pgmap v2298: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T19:21:08.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:21:08 smithi134 ceph-mon[35037]: pgmap v2299: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:21:08.931 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:21:08 smithi079 ceph-mon[35312]: pgmap v2299: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:21:09.031 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:21:08 smithi022 ceph-mon[31985]: pgmap v2299: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:21:10.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:21:10 smithi134 ceph-mon[35037]: pgmap v2300: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:21:10.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:21:10 smithi134 ceph-mon[35037]: 2 slow requests, 0 included below; oldest blocked for > 3856.900119 secs 2024-03-28T19:21:10.931 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:21:10 smithi079 ceph-mon[35312]: pgmap v2300: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:21:10.931 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:21:10 smithi079 ceph-mon[35312]: 2 slow requests, 0 included below; oldest blocked for > 3856.900119 secs 2024-03-28T19:21:11.031 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:21:10 smithi022 ceph-mon[31985]: pgmap v2300: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:21:11.031 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:21:10 smithi022 ceph-mon[31985]: 2 slow requests, 0 included below; oldest blocked for > 3856.900119 secs 2024-03-28T19:21:12.923 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:21:12 smithi134 ceph-mon[35037]: pgmap v2301: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 9 op/s 2024-03-28T19:21:12.931 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:21:12 smithi079 ceph-mon[35312]: pgmap v2301: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 9 op/s 2024-03-28T19:21:13.031 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:21:12 smithi022 ceph-mon[31985]: pgmap v2301: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 9 op/s 2024-03-28T19:21:14.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:21:14 smithi134 ceph-mon[35037]: pgmap v2302: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T19:21:14.931 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:21:14 smithi079 ceph-mon[35312]: pgmap v2302: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T19:21:15.031 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:21:14 smithi022 ceph-mon[31985]: pgmap v2302: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T19:21:15.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:21:15 smithi134 ceph-mon[35037]: 2 slow requests, 0 included below; oldest blocked for > 3861.900212 secs 2024-03-28T19:21:15.931 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:21:15 smithi079 ceph-mon[35312]: 2 slow requests, 0 included below; oldest blocked for > 3861.900212 secs 2024-03-28T19:21:16.033 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:21:15 smithi022 ceph-mon[31985]: 2 slow requests, 0 included below; oldest blocked for > 3861.900212 secs 2024-03-28T19:21:16.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:21:16 smithi134 ceph-mon[35037]: pgmap v2303: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 9 op/s 2024-03-28T19:21:16.931 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:21:16 smithi079 ceph-mon[35312]: pgmap v2303: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 9 op/s 2024-03-28T19:21:17.031 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:21:16 smithi022 ceph-mon[31985]: pgmap v2303: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 9 op/s 2024-03-28T19:21:18.931 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:21:18 smithi079 ceph-mon[35312]: pgmap v2304: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T19:21:19.031 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:21:18 smithi022 ceph-mon[31985]: pgmap v2304: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T19:21:19.174 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:21:18 smithi134 ceph-mon[35037]: pgmap v2304: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T19:21:20.931 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:21:20 smithi079 ceph-mon[35312]: pgmap v2305: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T19:21:20.931 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:21:20 smithi079 ceph-mon[35312]: 2 slow requests, 0 included below; oldest blocked for > 3866.900276 secs 2024-03-28T19:21:21.031 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:21:20 smithi022 ceph-mon[31985]: pgmap v2305: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T19:21:21.031 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:21:20 smithi022 ceph-mon[31985]: 2 slow requests, 0 included below; oldest blocked for > 3866.900276 secs 2024-03-28T19:21:21.173 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:21:20 smithi134 ceph-mon[35037]: pgmap v2305: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T19:21:21.174 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:21:20 smithi134 ceph-mon[35037]: 2 slow requests, 0 included below; oldest blocked for > 3866.900276 secs 2024-03-28T19:21:22.977 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:21:22 smithi134 ceph-mon[35037]: pgmap v2306: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 9 op/s 2024-03-28T19:21:23.031 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:21:22 smithi022 ceph-mon[31985]: pgmap v2306: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 9 op/s 2024-03-28T19:21:23.181 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:21:22 smithi079 ceph-mon[35312]: pgmap v2306: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 9 op/s 2024-03-28T19:21:25.031 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:21:24 smithi022 ceph-mon[31985]: pgmap v2307: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:21:25.174 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:21:24 smithi134 ceph-mon[35037]: pgmap v2307: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:21:25.181 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:21:24 smithi079 ceph-mon[35312]: pgmap v2307: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:21:26.031 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:21:25 smithi022 ceph-mon[31985]: 2 slow requests, 0 included below; oldest blocked for > 3871.900370 secs 2024-03-28T19:21:26.173 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:21:25 smithi134 ceph-mon[35037]: 2 slow requests, 0 included below; oldest blocked for > 3871.900370 secs 2024-03-28T19:21:26.181 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:21:25 smithi079 ceph-mon[35312]: 2 slow requests, 0 included below; oldest blocked for > 3871.900370 secs 2024-03-28T19:21:27.031 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:21:26 smithi022 ceph-mon[31985]: pgmap v2308: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T19:21:27.173 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:21:26 smithi134 ceph-mon[35037]: pgmap v2308: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T19:21:27.181 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:21:26 smithi079 ceph-mon[35312]: pgmap v2308: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T19:21:29.031 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:21:28 smithi022 ceph-mon[31985]: pgmap v2309: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:21:29.173 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:21:28 smithi134 ceph-mon[35037]: pgmap v2309: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:21:29.181 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:21:28 smithi079 ceph-mon[35312]: pgmap v2309: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:21:31.031 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:21:30 smithi022 ceph-mon[31985]: pgmap v2310: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:21:31.031 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:21:30 smithi022 ceph-mon[31985]: 2 slow requests, 0 included below; oldest blocked for > 3876.900448 secs 2024-03-28T19:21:31.174 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:21:30 smithi134 ceph-mon[35037]: pgmap v2310: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:21:31.174 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:21:30 smithi134 ceph-mon[35037]: 2 slow requests, 0 included below; oldest blocked for > 3876.900448 secs 2024-03-28T19:21:31.181 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:21:30 smithi079 ceph-mon[35312]: pgmap v2310: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:21:31.181 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:21:30 smithi079 ceph-mon[35312]: 2 slow requests, 0 included below; oldest blocked for > 3876.900448 secs 2024-03-28T19:21:32.984 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:21:32 smithi079 ceph-mon[35312]: pgmap v2311: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T19:21:33.031 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:21:32 smithi022 ceph-mon[31985]: pgmap v2311: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T19:21:33.174 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:21:32 smithi134 ceph-mon[35037]: pgmap v2311: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T19:21:35.031 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:21:34 smithi022 ceph-mon[31985]: pgmap v2312: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:21:35.174 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:21:34 smithi134 ceph-mon[35037]: pgmap v2312: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:21:35.181 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:21:34 smithi079 ceph-mon[35312]: pgmap v2312: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:21:36.031 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:21:35 smithi022 ceph-mon[31985]: 2 slow requests, 0 included below; oldest blocked for > 3881.900519 secs 2024-03-28T19:21:36.174 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:21:35 smithi134 ceph-mon[35037]: 2 slow requests, 0 included below; oldest blocked for > 3881.900519 secs 2024-03-28T19:21:36.181 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:21:35 smithi079 ceph-mon[35312]: 2 slow requests, 0 included below; oldest blocked for > 3881.900519 secs 2024-03-28T19:21:37.031 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:21:36 smithi022 ceph-mon[31985]: pgmap v2313: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T19:21:37.174 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:21:36 smithi134 ceph-mon[35037]: pgmap v2313: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T19:21:37.181 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:21:36 smithi079 ceph-mon[35312]: pgmap v2313: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T19:21:39.031 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:21:38 smithi022 ceph-mon[31985]: pgmap v2314: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:21:39.174 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:21:38 smithi134 ceph-mon[35037]: pgmap v2314: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:21:39.181 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:21:38 smithi079 ceph-mon[35312]: pgmap v2314: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:21:41.031 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:21:40 smithi022 ceph-mon[31985]: pgmap v2315: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:21:41.031 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:21:40 smithi022 ceph-mon[31985]: 2 slow requests, 0 included below; oldest blocked for > 3886.900584 secs 2024-03-28T19:21:41.174 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:21:40 smithi134 ceph-mon[35037]: pgmap v2315: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:21:41.174 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:21:40 smithi134 ceph-mon[35037]: 2 slow requests, 0 included below; oldest blocked for > 3886.900584 secs 2024-03-28T19:21:41.181 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:21:40 smithi079 ceph-mon[35312]: pgmap v2315: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:21:41.181 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:21:40 smithi079 ceph-mon[35312]: 2 slow requests, 0 included below; oldest blocked for > 3886.900584 secs 2024-03-28T19:21:43.031 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:21:42 smithi022 ceph-mon[31985]: pgmap v2316: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T19:21:43.173 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:21:42 smithi134 ceph-mon[35037]: pgmap v2316: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T19:21:43.181 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:21:42 smithi079 ceph-mon[35312]: pgmap v2316: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T19:21:45.031 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:21:44 smithi022 ceph-mon[31985]: pgmap v2317: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:21:45.173 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:21:44 smithi134 ceph-mon[35037]: pgmap v2317: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:21:45.181 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:21:44 smithi079 ceph-mon[35312]: pgmap v2317: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:21:46.173 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:21:45 smithi134 ceph-mon[35037]: 2 slow requests, 0 included below; oldest blocked for > 3891.900659 secs 2024-03-28T19:21:46.181 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:21:45 smithi079 ceph-mon[35312]: 2 slow requests, 0 included below; oldest blocked for > 3891.900659 secs 2024-03-28T19:21:46.281 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:21:45 smithi022 ceph-mon[31985]: 2 slow requests, 0 included below; oldest blocked for > 3891.900659 secs 2024-03-28T19:21:47.174 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:21:46 smithi134 ceph-mon[35037]: pgmap v2318: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T19:21:47.181 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:21:46 smithi079 ceph-mon[35312]: pgmap v2318: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T19:21:47.281 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:21:46 smithi022 ceph-mon[31985]: pgmap v2318: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T19:21:48.174 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:21:47 smithi134 ceph-mon[35037]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T19:21:48.181 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:21:47 smithi079 ceph-mon[35312]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T19:21:48.281 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:21:47 smithi022 ceph-mon[31985]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T19:21:49.174 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:21:48 smithi134 ceph-mon[35037]: pgmap v2319: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:21:49.174 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:21:48 smithi134 ceph-mon[35037]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T19:21:49.174 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:21:48 smithi134 ceph-mon[35037]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T19:21:49.174 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:21:48 smithi134 ceph-mon[35037]: from='mgr.15522 ' entity='mgr.x' 2024-03-28T19:21:49.181 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:21:48 smithi079 ceph-mon[35312]: pgmap v2319: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:21:49.181 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:21:48 smithi079 ceph-mon[35312]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T19:21:49.181 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:21:48 smithi079 ceph-mon[35312]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T19:21:49.181 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:21:48 smithi079 ceph-mon[35312]: from='mgr.15522 ' entity='mgr.x' 2024-03-28T19:21:49.281 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:21:48 smithi022 ceph-mon[31985]: pgmap v2319: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:21:49.281 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:21:48 smithi022 ceph-mon[31985]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T19:21:49.281 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:21:48 smithi022 ceph-mon[31985]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T19:21:49.281 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:21:48 smithi022 ceph-mon[31985]: from='mgr.15522 ' entity='mgr.x' 2024-03-28T19:21:50.174 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:21:49 smithi134 ceph-mon[35037]: pgmap v2320: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 6 op/s 2024-03-28T19:21:50.174 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:21:49 smithi134 ceph-mon[35037]: pgmap v2321: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-03-28T19:21:50.181 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:21:49 smithi079 ceph-mon[35312]: pgmap v2320: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 6 op/s 2024-03-28T19:21:50.181 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:21:49 smithi079 ceph-mon[35312]: pgmap v2321: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-03-28T19:21:50.183 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:21:49 smithi022 ceph-mon[31985]: pgmap v2320: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 6 op/s 2024-03-28T19:21:50.183 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:21:49 smithi022 ceph-mon[31985]: pgmap v2321: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-03-28T19:21:51.174 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:21:50 smithi134 ceph-mon[35037]: 2 slow requests, 0 included below; oldest blocked for > 3896.900724 secs 2024-03-28T19:21:51.181 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:21:50 smithi079 ceph-mon[35312]: 2 slow requests, 0 included below; oldest blocked for > 3896.900724 secs 2024-03-28T19:21:51.281 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:21:50 smithi022 ceph-mon[31985]: 2 slow requests, 0 included below; oldest blocked for > 3896.900724 secs 2024-03-28T19:21:52.174 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:21:51 smithi134 ceph-mon[35037]: pgmap v2322: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-03-28T19:21:52.181 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:21:51 smithi079 ceph-mon[35312]: pgmap v2322: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-03-28T19:21:52.281 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:21:51 smithi022 ceph-mon[31985]: pgmap v2322: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-03-28T19:21:53.174 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:21:52 smithi134 ceph-mon[35037]: pgmap v2323: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-03-28T19:21:53.181 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:21:52 smithi079 ceph-mon[35312]: pgmap v2323: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-03-28T19:21:53.281 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:21:52 smithi022 ceph-mon[31985]: pgmap v2323: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-03-28T19:21:56.173 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:21:55 smithi134 ceph-mon[35037]: 2 slow requests, 0 included below; oldest blocked for > 3901.900799 secs 2024-03-28T19:21:56.174 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:21:55 smithi134 ceph-mon[35037]: pgmap v2324: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-03-28T19:21:56.181 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:21:55 smithi079 ceph-mon[35312]: 2 slow requests, 0 included below; oldest blocked for > 3901.900799 secs 2024-03-28T19:21:56.181 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:21:55 smithi079 ceph-mon[35312]: pgmap v2324: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-03-28T19:21:56.281 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:21:55 smithi022 ceph-mon[31985]: 2 slow requests, 0 included below; oldest blocked for > 3901.900799 secs 2024-03-28T19:21:56.281 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:21:55 smithi022 ceph-mon[31985]: pgmap v2324: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-03-28T19:21:58.173 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:21:57 smithi134 ceph-mon[35037]: pgmap v2325: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-03-28T19:21:58.181 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:21:57 smithi079 ceph-mon[35312]: pgmap v2325: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-03-28T19:21:58.281 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:21:57 smithi022 ceph-mon[31985]: pgmap v2325: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-03-28T19:22:00.174 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:21:59 smithi134 ceph-mon[35037]: pgmap v2326: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 7 op/s 2024-03-28T19:22:00.181 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:21:59 smithi079 ceph-mon[35312]: pgmap v2326: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 7 op/s 2024-03-28T19:22:00.281 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:21:59 smithi022 ceph-mon[31985]: pgmap v2326: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 7 op/s 2024-03-28T19:22:01.174 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:22:00 smithi134 ceph-mon[35037]: 2 slow requests, 0 included below; oldest blocked for > 3906.900929 secs 2024-03-28T19:22:01.174 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:22:00 smithi134 ceph-mon[35037]: pgmap v2327: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T19:22:01.181 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:22:00 smithi079 ceph-mon[35312]: 2 slow requests, 0 included below; oldest blocked for > 3906.900929 secs 2024-03-28T19:22:01.181 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:22:00 smithi079 ceph-mon[35312]: pgmap v2327: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T19:22:01.281 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:22:00 smithi022 ceph-mon[31985]: 2 slow requests, 0 included below; oldest blocked for > 3906.900929 secs 2024-03-28T19:22:01.281 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:22:00 smithi022 ceph-mon[31985]: pgmap v2327: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T19:22:04.174 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:22:03 smithi134 ceph-mon[35037]: pgmap v2328: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:22:04.181 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:22:03 smithi079 ceph-mon[35312]: pgmap v2328: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:22:04.281 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:22:03 smithi022 ceph-mon[31985]: pgmap v2328: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:22:06.174 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:22:05 smithi134 ceph-mon[35037]: 2 slow requests, 0 included below; oldest blocked for > 3911.901021 secs 2024-03-28T19:22:06.174 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:22:05 smithi134 ceph-mon[35037]: pgmap v2329: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T19:22:06.181 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:22:05 smithi079 ceph-mon[35312]: 2 slow requests, 0 included below; oldest blocked for > 3911.901021 secs 2024-03-28T19:22:06.181 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:22:05 smithi079 ceph-mon[35312]: pgmap v2329: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T19:22:06.281 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:22:05 smithi022 ceph-mon[31985]: 2 slow requests, 0 included below; oldest blocked for > 3911.901021 secs 2024-03-28T19:22:06.281 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:22:05 smithi022 ceph-mon[31985]: pgmap v2329: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T19:22:08.174 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:22:07 smithi134 ceph-mon[35037]: pgmap v2330: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:22:08.181 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:22:07 smithi079 ceph-mon[35312]: pgmap v2330: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:22:08.281 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:22:07 smithi022 ceph-mon[31985]: pgmap v2330: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:22:09.174 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:22:08 smithi134 ceph-mon[35037]: pgmap v2331: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:22:09.181 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:22:08 smithi079 ceph-mon[35312]: pgmap v2331: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:22:09.281 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:22:08 smithi022 ceph-mon[31985]: pgmap v2331: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:22:10.174 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:22:09 smithi134 ceph-mon[35037]: 2 slow requests, 0 included below; oldest blocked for > 3916.901088 secs 2024-03-28T19:22:10.181 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:22:09 smithi079 ceph-mon[35312]: 2 slow requests, 0 included below; oldest blocked for > 3916.901088 secs 2024-03-28T19:22:10.281 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:22:09 smithi022 ceph-mon[31985]: 2 slow requests, 0 included below; oldest blocked for > 3916.901088 secs 2024-03-28T19:22:11.174 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:22:10 smithi134 ceph-mon[35037]: pgmap v2332: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T19:22:11.181 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:22:10 smithi079 ceph-mon[35312]: pgmap v2332: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T19:22:11.281 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:22:10 smithi022 ceph-mon[31985]: pgmap v2332: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T19:22:14.173 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:22:13 smithi134 ceph-mon[35037]: pgmap v2333: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:22:14.181 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:22:13 smithi079 ceph-mon[35312]: pgmap v2333: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:22:14.281 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:22:13 smithi022 ceph-mon[31985]: pgmap v2333: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:22:16.174 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:22:15 smithi134 ceph-mon[35037]: 2 slow requests, 0 included below; oldest blocked for > 3921.901163 secs 2024-03-28T19:22:16.174 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:22:15 smithi134 ceph-mon[35037]: pgmap v2334: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T19:22:16.181 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:22:15 smithi079 ceph-mon[35312]: 2 slow requests, 0 included below; oldest blocked for > 3921.901163 secs 2024-03-28T19:22:16.181 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:22:15 smithi079 ceph-mon[35312]: pgmap v2334: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T19:22:16.281 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:22:15 smithi022 ceph-mon[31985]: 2 slow requests, 0 included below; oldest blocked for > 3921.901163 secs 2024-03-28T19:22:16.281 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:22:15 smithi022 ceph-mon[31985]: pgmap v2334: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T19:22:17.174 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:22:16 smithi134 ceph-mon[35037]: pgmap v2335: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:22:17.181 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:22:16 smithi079 ceph-mon[35312]: pgmap v2335: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:22:17.281 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:22:16 smithi022 ceph-mon[31985]: pgmap v2335: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:22:20.174 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:22:19 smithi134 ceph-mon[35037]: pgmap v2336: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:22:20.181 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:22:19 smithi079 ceph-mon[35312]: pgmap v2336: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:22:20.281 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:22:19 smithi022 ceph-mon[31985]: pgmap v2336: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:22:21.174 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:22:20 smithi134 ceph-mon[35037]: 2 slow requests, 0 included below; oldest blocked for > 3926.901249 secs 2024-03-28T19:22:21.181 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:22:20 smithi079 ceph-mon[35312]: 2 slow requests, 0 included below; oldest blocked for > 3926.901249 secs 2024-03-28T19:22:21.182 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:22:20 smithi022 ceph-mon[31985]: 2 slow requests, 0 included below; oldest blocked for > 3926.901249 secs 2024-03-28T19:22:22.174 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:22:21 smithi134 ceph-mon[35037]: pgmap v2337: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T19:22:22.181 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:22:21 smithi079 ceph-mon[35312]: pgmap v2337: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T19:22:22.281 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:22:21 smithi022 ceph-mon[31985]: pgmap v2337: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T19:22:23.174 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:22:22 smithi134 ceph-mon[35037]: pgmap v2338: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:22:23.181 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:22:22 smithi079 ceph-mon[35312]: pgmap v2338: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:22:23.281 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:22:22 smithi022 ceph-mon[31985]: pgmap v2338: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:22:26.173 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:22:25 smithi134 ceph-mon[35037]: 2 slow requests, 0 included below; oldest blocked for > 3931.901310 secs 2024-03-28T19:22:26.174 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:22:25 smithi134 ceph-mon[35037]: pgmap v2339: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T19:22:26.181 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:22:25 smithi079 ceph-mon[35312]: 2 slow requests, 0 included below; oldest blocked for > 3931.901310 secs 2024-03-28T19:22:26.181 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:22:25 smithi079 ceph-mon[35312]: pgmap v2339: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T19:22:26.281 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:22:25 smithi022 ceph-mon[31985]: 2 slow requests, 0 included below; oldest blocked for > 3931.901310 secs 2024-03-28T19:22:26.281 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:22:25 smithi022 ceph-mon[31985]: pgmap v2339: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T19:22:28.174 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:22:27 smithi134 ceph-mon[35037]: pgmap v2340: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:22:28.181 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:22:27 smithi079 ceph-mon[35312]: pgmap v2340: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:22:28.281 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:22:27 smithi022 ceph-mon[31985]: pgmap v2340: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:22:29.174 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:22:28 smithi134 ceph-mon[35037]: pgmap v2341: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:22:29.181 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:22:28 smithi079 ceph-mon[35312]: pgmap v2341: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:22:29.281 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:22:28 smithi022 ceph-mon[31985]: pgmap v2341: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:22:30.173 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:22:29 smithi134 ceph-mon[35037]: 2 slow requests, 0 included below; oldest blocked for > 3936.901366 secs 2024-03-28T19:22:30.181 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:22:29 smithi079 ceph-mon[35312]: 2 slow requests, 0 included below; oldest blocked for > 3936.901366 secs 2024-03-28T19:22:30.282 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:22:29 smithi022 ceph-mon[31985]: 2 slow requests, 0 included below; oldest blocked for > 3936.901366 secs 2024-03-28T19:22:31.173 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:22:30 smithi134 ceph-mon[35037]: pgmap v2342: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T19:22:31.181 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:22:30 smithi079 ceph-mon[35312]: pgmap v2342: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T19:22:31.281 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:22:30 smithi022 ceph-mon[31985]: pgmap v2342: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T19:22:34.174 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:22:33 smithi134 ceph-mon[35037]: pgmap v2343: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:22:34.181 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:22:33 smithi079 ceph-mon[35312]: pgmap v2343: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:22:34.281 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:22:33 smithi022 ceph-mon[31985]: pgmap v2343: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:22:35.174 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:22:34 smithi134 ceph-mon[35037]: 2 slow requests, 0 included below; oldest blocked for > 3941.901433 secs 2024-03-28T19:22:35.174 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:22:34 smithi134 ceph-mon[35037]: pgmap v2344: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T19:22:35.181 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:22:34 smithi079 ceph-mon[35312]: 2 slow requests, 0 included below; oldest blocked for > 3941.901433 secs 2024-03-28T19:22:35.181 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:22:34 smithi079 ceph-mon[35312]: pgmap v2344: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T19:22:35.281 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:22:34 smithi022 ceph-mon[31985]: 2 slow requests, 0 included below; oldest blocked for > 3941.901433 secs 2024-03-28T19:22:35.281 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:22:34 smithi022 ceph-mon[31985]: pgmap v2344: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T19:22:38.174 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:22:37 smithi134 ceph-mon[35037]: pgmap v2345: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:22:38.181 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:22:37 smithi079 ceph-mon[35312]: pgmap v2345: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:22:38.281 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:22:37 smithi022 ceph-mon[31985]: pgmap v2345: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:22:39.174 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:22:38 smithi134 ceph-mon[35037]: pgmap v2346: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:22:39.182 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:22:38 smithi079 ceph-mon[35312]: pgmap v2346: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:22:39.281 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:22:38 smithi022 ceph-mon[31985]: pgmap v2346: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:22:40.174 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:22:39 smithi134 ceph-mon[35037]: 2 slow requests, 0 included below; oldest blocked for > 3946.901497 secs 2024-03-28T19:22:40.181 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:22:39 smithi079 ceph-mon[35312]: 2 slow requests, 0 included below; oldest blocked for > 3946.901497 secs 2024-03-28T19:22:40.281 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:22:39 smithi022 ceph-mon[31985]: 2 slow requests, 0 included below; oldest blocked for > 3946.901497 secs 2024-03-28T19:22:41.174 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:22:40 smithi134 ceph-mon[35037]: pgmap v2347: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T19:22:41.181 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:22:40 smithi079 ceph-mon[35312]: pgmap v2347: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T19:22:41.280 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:22:40 smithi022 ceph-mon[31985]: pgmap v2347: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T19:22:44.174 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:22:43 smithi134 ceph-mon[35037]: pgmap v2348: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:22:44.181 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:22:43 smithi079 ceph-mon[35312]: pgmap v2348: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:22:44.281 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:22:43 smithi022 ceph-mon[31985]: pgmap v2348: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:22:45.174 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:22:44 smithi134 ceph-mon[35037]: pgmap v2349: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T19:22:45.181 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:22:44 smithi079 ceph-mon[35312]: pgmap v2349: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T19:22:45.281 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:22:44 smithi022 ceph-mon[31985]: pgmap v2349: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T19:22:46.173 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:22:45 smithi134 ceph-mon[35037]: 2 slow requests, 0 included below; oldest blocked for > 3951.901573 secs 2024-03-28T19:22:46.181 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:22:45 smithi079 ceph-mon[35312]: 2 slow requests, 0 included below; oldest blocked for > 3951.901573 secs 2024-03-28T19:22:46.281 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:22:45 smithi022 ceph-mon[31985]: 2 slow requests, 0 included below; oldest blocked for > 3951.901573 secs 2024-03-28T19:22:47.174 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:22:46 smithi134 ceph-mon[35037]: pgmap v2350: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:22:47.181 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:22:46 smithi079 ceph-mon[35312]: pgmap v2350: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:22:47.281 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:22:46 smithi022 ceph-mon[31985]: pgmap v2350: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:22:49.173 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:22:48 smithi134 ceph-mon[35037]: pgmap v2351: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:22:49.181 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:22:48 smithi079 ceph-mon[35312]: pgmap v2351: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:22:49.280 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:22:48 smithi022 ceph-mon[31985]: pgmap v2351: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:22:50.174 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:22:49 smithi134 ceph-mon[35037]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T19:22:50.174 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:22:49 smithi134 ceph-mon[35037]: 2 slow requests, 0 included below; oldest blocked for > 3956.901641 secs 2024-03-28T19:22:50.181 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:22:49 smithi079 ceph-mon[35312]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T19:22:50.181 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:22:49 smithi079 ceph-mon[35312]: 2 slow requests, 0 included below; oldest blocked for > 3956.901641 secs 2024-03-28T19:22:50.281 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:22:49 smithi022 ceph-mon[31985]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T19:22:50.281 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:22:49 smithi022 ceph-mon[31985]: 2 slow requests, 0 included below; oldest blocked for > 3956.901641 secs 2024-03-28T19:22:51.174 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:22:50 smithi134 ceph-mon[35037]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T19:22:51.174 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:22:50 smithi134 ceph-mon[35037]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T19:22:51.174 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:22:50 smithi134 ceph-mon[35037]: pgmap v2352: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 10 op/s 2024-03-28T19:22:51.174 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:22:50 smithi134 ceph-mon[35037]: pgmap v2353: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-03-28T19:22:51.174 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:22:50 smithi134 ceph-mon[35037]: from='mgr.15522 ' entity='mgr.x' 2024-03-28T19:22:51.181 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:22:50 smithi079 ceph-mon[35312]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T19:22:51.181 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:22:50 smithi079 ceph-mon[35312]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T19:22:51.182 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:22:50 smithi079 ceph-mon[35312]: pgmap v2352: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 10 op/s 2024-03-28T19:22:51.182 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:22:50 smithi079 ceph-mon[35312]: pgmap v2353: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-03-28T19:22:51.182 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:22:50 smithi079 ceph-mon[35312]: from='mgr.15522 ' entity='mgr.x' 2024-03-28T19:22:51.183 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:22:50 smithi022 ceph-mon[31985]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T19:22:51.183 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:22:50 smithi022 ceph-mon[31985]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T19:22:51.183 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:22:50 smithi022 ceph-mon[31985]: pgmap v2352: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 10 op/s 2024-03-28T19:22:51.183 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:22:50 smithi022 ceph-mon[31985]: pgmap v2353: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-03-28T19:22:51.183 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:22:50 smithi022 ceph-mon[31985]: from='mgr.15522 ' entity='mgr.x' 2024-03-28T19:22:53.877 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:22:53 smithi134 ceph-mon[35037]: pgmap v2354: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-03-28T19:22:53.931 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:22:53 smithi079 ceph-mon[35312]: pgmap v2354: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-03-28T19:22:54.031 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:22:53 smithi022 ceph-mon[31985]: pgmap v2354: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-03-28T19:22:55.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:22:55 smithi134 ceph-mon[35037]: 2 slow requests, 0 included below; oldest blocked for > 3961.901745 secs 2024-03-28T19:22:55.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:22:55 smithi134 ceph-mon[35037]: pgmap v2355: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-03-28T19:22:55.931 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:22:55 smithi079 ceph-mon[35312]: 2 slow requests, 0 included below; oldest blocked for > 3961.901745 secs 2024-03-28T19:22:55.931 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:22:55 smithi079 ceph-mon[35312]: pgmap v2355: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-03-28T19:22:56.031 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:22:55 smithi022 ceph-mon[31985]: 2 slow requests, 0 included below; oldest blocked for > 3961.901745 secs 2024-03-28T19:22:56.031 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:22:55 smithi022 ceph-mon[31985]: pgmap v2355: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-03-28T19:22:57.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:22:57 smithi134 ceph-mon[35037]: pgmap v2356: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-03-28T19:22:57.931 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:22:57 smithi079 ceph-mon[35312]: pgmap v2356: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-03-28T19:22:58.031 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:22:57 smithi022 ceph-mon[31985]: pgmap v2356: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-03-28T19:22:59.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:22:59 smithi134 ceph-mon[35037]: pgmap v2357: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-03-28T19:22:59.931 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:22:59 smithi079 ceph-mon[35312]: pgmap v2357: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-03-28T19:23:00.031 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:22:59 smithi022 ceph-mon[31985]: pgmap v2357: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-03-28T19:23:00.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:23:00 smithi134 ceph-mon[35037]: 2 slow requests, 0 included below; oldest blocked for > 3966.901825 secs 2024-03-28T19:23:00.931 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:23:00 smithi079 ceph-mon[35312]: 2 slow requests, 0 included below; oldest blocked for > 3966.901825 secs 2024-03-28T19:23:01.031 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:23:00 smithi022 ceph-mon[31985]: 2 slow requests, 0 included below; oldest blocked for > 3966.901825 secs 2024-03-28T19:23:01.923 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:23:01 smithi134 ceph-mon[35037]: pgmap v2358: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 7 op/s 2024-03-28T19:23:01.931 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:23:01 smithi079 ceph-mon[35312]: pgmap v2358: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 7 op/s 2024-03-28T19:23:02.031 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:23:01 smithi022 ceph-mon[31985]: pgmap v2358: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 7 op/s 2024-03-28T19:23:03.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:23:03 smithi134 ceph-mon[35037]: pgmap v2359: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:23:03.931 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:23:03 smithi079 ceph-mon[35312]: pgmap v2359: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:23:04.031 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:23:03 smithi022 ceph-mon[31985]: pgmap v2359: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:23:05.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:23:05 smithi134 ceph-mon[35037]: 2 slow requests, 0 included below; oldest blocked for > 3971.901914 secs 2024-03-28T19:23:05.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:23:05 smithi134 ceph-mon[35037]: pgmap v2360: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T19:23:05.931 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:23:05 smithi079 ceph-mon[35312]: 2 slow requests, 0 included below; oldest blocked for > 3971.901914 secs 2024-03-28T19:23:05.931 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:23:05 smithi079 ceph-mon[35312]: pgmap v2360: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T19:23:06.031 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:23:05 smithi022 ceph-mon[31985]: 2 slow requests, 0 included below; oldest blocked for > 3971.901914 secs 2024-03-28T19:23:06.031 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:23:05 smithi022 ceph-mon[31985]: pgmap v2360: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T19:23:07.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:23:07 smithi134 ceph-mon[35037]: pgmap v2361: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:23:07.931 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:23:07 smithi079 ceph-mon[35312]: pgmap v2361: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:23:08.031 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:23:07 smithi022 ceph-mon[31985]: pgmap v2361: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:23:09.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:23:09 smithi134 ceph-mon[35037]: pgmap v2362: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:23:09.931 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:23:09 smithi079 ceph-mon[35312]: pgmap v2362: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:23:10.031 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:23:09 smithi022 ceph-mon[31985]: pgmap v2362: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:23:10.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:23:10 smithi134 ceph-mon[35037]: 2 slow requests, 0 included below; oldest blocked for > 3976.901987 secs 2024-03-28T19:23:10.931 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:23:10 smithi079 ceph-mon[35312]: 2 slow requests, 0 included below; oldest blocked for > 3976.901987 secs 2024-03-28T19:23:11.031 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:23:10 smithi022 ceph-mon[31985]: 2 slow requests, 0 included below; oldest blocked for > 3976.901987 secs 2024-03-28T19:23:11.923 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:23:11 smithi134 ceph-mon[35037]: pgmap v2363: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T19:23:11.931 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:23:11 smithi079 ceph-mon[35312]: pgmap v2363: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T19:23:12.031 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:23:11 smithi022 ceph-mon[31985]: pgmap v2363: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T19:23:13.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:23:13 smithi134 ceph-mon[35037]: pgmap v2364: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:23:13.931 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:23:13 smithi079 ceph-mon[35312]: pgmap v2364: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:23:14.031 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:23:13 smithi022 ceph-mon[31985]: pgmap v2364: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:23:15.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:23:15 smithi134 ceph-mon[35037]: 2 slow requests, 0 included below; oldest blocked for > 3981.902106 secs 2024-03-28T19:23:15.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:23:15 smithi134 ceph-mon[35037]: pgmap v2365: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T19:23:15.931 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:23:15 smithi079 ceph-mon[35312]: 2 slow requests, 0 included below; oldest blocked for > 3981.902106 secs 2024-03-28T19:23:15.931 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:23:15 smithi079 ceph-mon[35312]: pgmap v2365: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T19:23:16.031 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:23:15 smithi022 ceph-mon[31985]: 2 slow requests, 0 included below; oldest blocked for > 3981.902106 secs 2024-03-28T19:23:16.031 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:23:15 smithi022 ceph-mon[31985]: pgmap v2365: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T19:23:17.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:23:17 smithi134 ceph-mon[35037]: pgmap v2366: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:23:17.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:23:17 smithi079 ceph-mon[35312]: pgmap v2366: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:23:18.031 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:23:17 smithi022 ceph-mon[31985]: pgmap v2366: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:23:19.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:23:19 smithi134 ceph-mon[35037]: pgmap v2367: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:23:19.931 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:23:19 smithi079 ceph-mon[35312]: pgmap v2367: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:23:20.031 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:23:19 smithi022 ceph-mon[31985]: pgmap v2367: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:23:21.031 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:23:20 smithi022 ceph-mon[31985]: 2 slow requests, 0 included below; oldest blocked for > 3986.902172 secs 2024-03-28T19:23:21.174 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:23:20 smithi134 ceph-mon[35037]: 2 slow requests, 0 included below; oldest blocked for > 3986.902172 secs 2024-03-28T19:23:21.181 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:23:20 smithi079 ceph-mon[35312]: 2 slow requests, 0 included below; oldest blocked for > 3986.902172 secs 2024-03-28T19:23:21.935 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:23:21 smithi022 ceph-mon[31985]: pgmap v2368: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T19:23:22.174 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:23:21 smithi134 ceph-mon[35037]: pgmap v2368: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T19:23:22.181 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:23:21 smithi079 ceph-mon[35312]: pgmap v2368: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T19:23:23.976 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:23:23 smithi134 ceph-mon[35037]: pgmap v2369: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:23:24.031 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:23:23 smithi022 ceph-mon[31985]: pgmap v2369: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:23:24.181 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:23:23 smithi079 ceph-mon[35312]: pgmap v2369: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:23:26.031 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:23:25 smithi022 ceph-mon[31985]: 2 slow requests, 0 included below; oldest blocked for > 3991.902236 secs 2024-03-28T19:23:26.031 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:23:25 smithi022 ceph-mon[31985]: pgmap v2370: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T19:23:26.174 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:23:25 smithi134 ceph-mon[35037]: 2 slow requests, 0 included below; oldest blocked for > 3991.902236 secs 2024-03-28T19:23:26.174 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:23:25 smithi134 ceph-mon[35037]: pgmap v2370: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T19:23:26.181 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:23:25 smithi079 ceph-mon[35312]: 2 slow requests, 0 included below; oldest blocked for > 3991.902236 secs 2024-03-28T19:23:26.181 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:23:25 smithi079 ceph-mon[35312]: pgmap v2370: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T19:23:28.031 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:23:27 smithi022 ceph-mon[31985]: pgmap v2371: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:23:28.173 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:23:27 smithi134 ceph-mon[35037]: pgmap v2371: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:23:28.181 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:23:27 smithi079 ceph-mon[35312]: pgmap v2371: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:23:30.031 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:23:29 smithi022 ceph-mon[31985]: pgmap v2372: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:23:30.174 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:23:29 smithi134 ceph-mon[35037]: pgmap v2372: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:23:30.181 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:23:29 smithi079 ceph-mon[35312]: pgmap v2372: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:23:31.031 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:23:30 smithi022 ceph-mon[31985]: 2 slow requests, 0 included below; oldest blocked for > 3996.902305 secs 2024-03-28T19:23:31.174 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:23:30 smithi134 ceph-mon[35037]: 2 slow requests, 0 included below; oldest blocked for > 3996.902305 secs 2024-03-28T19:23:31.181 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:23:30 smithi079 ceph-mon[35312]: 2 slow requests, 0 included below; oldest blocked for > 3996.902305 secs 2024-03-28T19:23:32.031 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:23:31 smithi022 ceph-mon[31985]: pgmap v2373: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T19:23:32.173 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:23:31 smithi134 ceph-mon[35037]: pgmap v2373: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T19:23:32.181 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:23:31 smithi079 ceph-mon[35312]: pgmap v2373: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T19:23:33.984 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:23:33 smithi079 ceph-mon[35312]: pgmap v2374: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:23:34.031 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:23:33 smithi022 ceph-mon[31985]: pgmap v2374: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:23:34.174 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:23:33 smithi134 ceph-mon[35037]: pgmap v2374: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:23:36.031 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:23:35 smithi022 ceph-mon[31985]: 2 slow requests, 0 included below; oldest blocked for > 4001.902371 secs 2024-03-28T19:23:36.031 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:23:35 smithi022 ceph-mon[31985]: pgmap v2375: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T19:23:36.174 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:23:35 smithi134 ceph-mon[35037]: 2 slow requests, 0 included below; oldest blocked for > 4001.902371 secs 2024-03-28T19:23:36.174 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:23:35 smithi134 ceph-mon[35037]: pgmap v2375: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T19:23:36.181 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:23:35 smithi079 ceph-mon[35312]: 2 slow requests, 0 included below; oldest blocked for > 4001.902371 secs 2024-03-28T19:23:36.181 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:23:35 smithi079 ceph-mon[35312]: pgmap v2375: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T19:23:38.031 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:23:37 smithi022 ceph-mon[31985]: pgmap v2376: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:23:38.173 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:23:37 smithi134 ceph-mon[35037]: pgmap v2376: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:23:38.181 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:23:37 smithi079 ceph-mon[35312]: pgmap v2376: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:23:40.031 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:23:39 smithi022 ceph-mon[31985]: pgmap v2377: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:23:40.174 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:23:39 smithi134 ceph-mon[35037]: pgmap v2377: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:23:40.181 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:23:39 smithi079 ceph-mon[35312]: pgmap v2377: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:23:41.031 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:23:40 smithi022 ceph-mon[31985]: 2 slow requests, 0 included below; oldest blocked for > 4006.902440 secs 2024-03-28T19:23:41.174 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:23:40 smithi134 ceph-mon[35037]: 2 slow requests, 0 included below; oldest blocked for > 4006.902440 secs 2024-03-28T19:23:41.181 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:23:40 smithi079 ceph-mon[35312]: 2 slow requests, 0 included below; oldest blocked for > 4006.902440 secs 2024-03-28T19:23:42.031 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:23:41 smithi022 ceph-mon[31985]: pgmap v2378: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 9 op/s 2024-03-28T19:23:42.174 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:23:41 smithi134 ceph-mon[35037]: pgmap v2378: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 9 op/s 2024-03-28T19:23:42.181 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:23:41 smithi079 ceph-mon[35312]: pgmap v2378: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 9 op/s 2024-03-28T19:23:44.031 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:23:43 smithi022 ceph-mon[31985]: pgmap v2379: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T19:23:44.174 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:23:43 smithi134 ceph-mon[35037]: pgmap v2379: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T19:23:44.181 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:23:43 smithi079 ceph-mon[35312]: pgmap v2379: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T19:23:46.031 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:23:45 smithi022 ceph-mon[31985]: 2 slow requests, 0 included below; oldest blocked for > 4011.902560 secs 2024-03-28T19:23:46.031 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:23:45 smithi022 ceph-mon[31985]: pgmap v2380: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 9 op/s 2024-03-28T19:23:46.174 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:23:45 smithi134 ceph-mon[35037]: 2 slow requests, 0 included below; oldest blocked for > 4011.902560 secs 2024-03-28T19:23:46.174 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:23:45 smithi134 ceph-mon[35037]: pgmap v2380: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 9 op/s 2024-03-28T19:23:46.181 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:23:45 smithi079 ceph-mon[35312]: 2 slow requests, 0 included below; oldest blocked for > 4011.902560 secs 2024-03-28T19:23:46.181 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:23:45 smithi079 ceph-mon[35312]: pgmap v2380: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 9 op/s 2024-03-28T19:23:48.031 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:23:47 smithi022 ceph-mon[31985]: pgmap v2381: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T19:23:48.173 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:23:47 smithi134 ceph-mon[35037]: pgmap v2381: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T19:23:48.181 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:23:47 smithi079 ceph-mon[35312]: pgmap v2381: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T19:23:50.174 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:23:49 smithi134 ceph-mon[35037]: pgmap v2382: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T19:23:50.181 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:23:49 smithi079 ceph-mon[35312]: pgmap v2382: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T19:23:50.281 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:23:49 smithi022 ceph-mon[31985]: pgmap v2382: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T19:23:51.173 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:23:50 smithi134 ceph-mon[35037]: 2 slow requests, 0 included below; oldest blocked for > 4016.902624 secs 2024-03-28T19:23:51.181 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:23:50 smithi079 ceph-mon[35312]: 2 slow requests, 0 included below; oldest blocked for > 4016.902624 secs 2024-03-28T19:23:51.182 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:23:50 smithi022 ceph-mon[31985]: 2 slow requests, 0 included below; oldest blocked for > 4016.902624 secs 2024-03-28T19:23:52.174 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:23:51 smithi134 ceph-mon[35037]: pgmap v2383: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 9 op/s 2024-03-28T19:23:52.174 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:23:51 smithi134 ceph-mon[35037]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T19:23:52.181 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:23:51 smithi079 ceph-mon[35312]: pgmap v2383: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 9 op/s 2024-03-28T19:23:52.181 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:23:51 smithi079 ceph-mon[35312]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T19:23:52.280 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:23:51 smithi022 ceph-mon[31985]: pgmap v2383: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 9 op/s 2024-03-28T19:23:52.281 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:23:51 smithi022 ceph-mon[31985]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T19:23:53.076 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:23:52 smithi134 ceph-mon[35037]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T19:23:53.076 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:23:52 smithi134 ceph-mon[35037]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T19:23:53.076 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:23:52 smithi134 ceph-mon[35037]: from='mgr.15522 ' entity='mgr.x' 2024-03-28T19:23:53.181 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:23:52 smithi079 ceph-mon[35312]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T19:23:53.181 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:23:52 smithi079 ceph-mon[35312]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T19:23:53.181 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:23:52 smithi079 ceph-mon[35312]: from='mgr.15522 ' entity='mgr.x' 2024-03-28T19:23:53.281 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:23:52 smithi022 ceph-mon[31985]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T19:23:53.281 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:23:52 smithi022 ceph-mon[31985]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T19:23:53.281 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:23:52 smithi022 ceph-mon[31985]: from='mgr.15522 ' entity='mgr.x' 2024-03-28T19:23:54.174 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:23:53 smithi134 ceph-mon[35037]: pgmap v2384: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 6 op/s 2024-03-28T19:23:54.174 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:23:53 smithi134 ceph-mon[35037]: pgmap v2385: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-03-28T19:23:54.181 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:23:53 smithi079 ceph-mon[35312]: pgmap v2384: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 6 op/s 2024-03-28T19:23:54.181 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:23:53 smithi079 ceph-mon[35312]: pgmap v2385: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-03-28T19:23:54.281 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:23:53 smithi022 ceph-mon[31985]: pgmap v2384: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 6 op/s 2024-03-28T19:23:54.281 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:23:53 smithi022 ceph-mon[31985]: pgmap v2385: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-03-28T19:23:56.174 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:23:55 smithi134 ceph-mon[35037]: pgmap v2386: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T19:23:56.174 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:23:55 smithi134 ceph-mon[35037]: 2 slow requests, 0 included below; oldest blocked for > 4021.902745 secs 2024-03-28T19:23:56.181 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:23:55 smithi079 ceph-mon[35312]: pgmap v2386: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T19:23:56.181 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:23:55 smithi079 ceph-mon[35312]: 2 slow requests, 0 included below; oldest blocked for > 4021.902745 secs 2024-03-28T19:23:56.281 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:23:55 smithi022 ceph-mon[31985]: pgmap v2386: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T19:23:56.281 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:23:55 smithi022 ceph-mon[31985]: 2 slow requests, 0 included below; oldest blocked for > 4021.902745 secs 2024-03-28T19:23:58.174 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:23:57 smithi134 ceph-mon[35037]: pgmap v2387: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-03-28T19:23:58.181 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:23:57 smithi079 ceph-mon[35312]: pgmap v2387: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-03-28T19:23:58.281 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:23:57 smithi022 ceph-mon[31985]: pgmap v2387: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-03-28T19:24:00.174 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:23:59 smithi134 ceph-mon[35037]: pgmap v2388: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-03-28T19:24:00.181 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:23:59 smithi079 ceph-mon[35312]: pgmap v2388: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-03-28T19:24:00.281 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:23:59 smithi022 ceph-mon[31985]: pgmap v2388: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-03-28T19:24:01.174 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:24:00 smithi134 ceph-mon[35037]: 2 slow requests, 0 included below; oldest blocked for > 4026.902824 secs 2024-03-28T19:24:01.181 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:24:00 smithi079 ceph-mon[35312]: 2 slow requests, 0 included below; oldest blocked for > 4026.902824 secs 2024-03-28T19:24:01.281 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:24:00 smithi022 ceph-mon[31985]: 2 slow requests, 0 included below; oldest blocked for > 4026.902824 secs 2024-03-28T19:24:02.173 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:24:01 smithi134 ceph-mon[35037]: pgmap v2389: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-03-28T19:24:02.181 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:24:01 smithi079 ceph-mon[35312]: pgmap v2389: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-03-28T19:24:02.281 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:24:01 smithi022 ceph-mon[31985]: pgmap v2389: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-03-28T19:24:03.173 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:24:02 smithi134 ceph-mon[35037]: pgmap v2390: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 7 op/s 2024-03-28T19:24:03.181 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:24:02 smithi079 ceph-mon[35312]: pgmap v2390: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 7 op/s 2024-03-28T19:24:03.281 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:24:02 smithi022 ceph-mon[31985]: pgmap v2390: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 7 op/s 2024-03-28T19:24:05.674 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:24:05 smithi134 ceph-mon[35037]: pgmap v2391: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-28T19:24:05.674 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:24:05 smithi134 ceph-mon[35037]: 2 slow requests, 0 included below; oldest blocked for > 4031.902890 secs 2024-03-28T19:24:05.681 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:24:05 smithi079 ceph-mon[35312]: pgmap v2391: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-28T19:24:05.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:24:05 smithi079 ceph-mon[35312]: 2 slow requests, 0 included below; oldest blocked for > 4031.902890 secs 2024-03-28T19:24:05.781 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:24:05 smithi022 ceph-mon[31985]: pgmap v2391: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-28T19:24:05.781 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:24:05 smithi022 ceph-mon[31985]: 2 slow requests, 0 included below; oldest blocked for > 4031.902890 secs 2024-03-28T19:24:07.673 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:24:07 smithi134 ceph-mon[35037]: pgmap v2392: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 7 op/s 2024-03-28T19:24:07.681 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:24:07 smithi079 ceph-mon[35312]: pgmap v2392: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 7 op/s 2024-03-28T19:24:07.781 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:24:07 smithi022 ceph-mon[31985]: pgmap v2392: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 7 op/s 2024-03-28T19:24:09.674 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:24:09 smithi134 ceph-mon[35037]: pgmap v2393: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:24:09.681 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:24:09 smithi079 ceph-mon[35312]: pgmap v2393: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:24:09.781 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:24:09 smithi022 ceph-mon[31985]: pgmap v2393: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:24:10.674 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:24:10 smithi134 ceph-mon[35037]: 2 slow requests, 0 included below; oldest blocked for > 4036.902969 secs 2024-03-28T19:24:10.681 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:24:10 smithi079 ceph-mon[35312]: 2 slow requests, 0 included below; oldest blocked for > 4036.902969 secs 2024-03-28T19:24:10.781 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:24:10 smithi022 ceph-mon[31985]: 2 slow requests, 0 included below; oldest blocked for > 4036.902969 secs 2024-03-28T19:24:11.673 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:24:11 smithi134 ceph-mon[35037]: pgmap v2394: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T19:24:11.681 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:24:11 smithi079 ceph-mon[35312]: pgmap v2394: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T19:24:11.781 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:24:11 smithi022 ceph-mon[31985]: pgmap v2394: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T19:24:13.674 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:24:13 smithi134 ceph-mon[35037]: pgmap v2395: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:24:13.681 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:24:13 smithi079 ceph-mon[35312]: pgmap v2395: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:24:13.781 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:24:13 smithi022 ceph-mon[31985]: pgmap v2395: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:24:15.674 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:24:15 smithi134 ceph-mon[35037]: pgmap v2396: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-03-28T19:24:15.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:24:15 smithi134 ceph-mon[35037]: 2 slow requests, 0 included below; oldest blocked for > 4041.903087 secs 2024-03-28T19:24:15.681 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:24:15 smithi079 ceph-mon[35312]: pgmap v2396: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-03-28T19:24:15.681 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:24:15 smithi079 ceph-mon[35312]: 2 slow requests, 0 included below; oldest blocked for > 4041.903087 secs 2024-03-28T19:24:15.781 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:24:15 smithi022 ceph-mon[31985]: pgmap v2396: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-03-28T19:24:15.781 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:24:15 smithi022 ceph-mon[31985]: 2 slow requests, 0 included below; oldest blocked for > 4041.903087 secs 2024-03-28T19:24:17.673 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:24:17 smithi134 ceph-mon[35037]: pgmap v2397: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 7 op/s 2024-03-28T19:24:17.681 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:24:17 smithi079 ceph-mon[35312]: pgmap v2397: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 7 op/s 2024-03-28T19:24:17.781 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:24:17 smithi022 ceph-mon[31985]: pgmap v2397: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 7 op/s 2024-03-28T19:24:19.673 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:24:19 smithi134 ceph-mon[35037]: pgmap v2398: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:24:19.681 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:24:19 smithi079 ceph-mon[35312]: pgmap v2398: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:24:19.781 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:24:19 smithi022 ceph-mon[31985]: pgmap v2398: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:24:20.673 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:24:20 smithi134 ceph-mon[35037]: 2 slow requests, 0 included below; oldest blocked for > 4046.903136 secs 2024-03-28T19:24:20.681 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:24:20 smithi079 ceph-mon[35312]: 2 slow requests, 0 included below; oldest blocked for > 4046.903136 secs 2024-03-28T19:24:20.781 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:24:20 smithi022 ceph-mon[31985]: 2 slow requests, 0 included below; oldest blocked for > 4046.903136 secs 2024-03-28T19:24:21.674 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:24:21 smithi134 ceph-mon[35037]: pgmap v2399: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T19:24:21.681 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:24:21 smithi079 ceph-mon[35312]: pgmap v2399: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T19:24:21.683 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:24:21 smithi022 ceph-mon[31985]: pgmap v2399: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T19:24:23.674 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:24:23 smithi134 ceph-mon[35037]: pgmap v2400: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:24:23.681 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:24:23 smithi079 ceph-mon[35312]: pgmap v2400: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:24:23.781 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:24:23 smithi022 ceph-mon[31985]: pgmap v2400: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:24:25.674 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:24:25 smithi134 ceph-mon[35037]: pgmap v2401: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-03-28T19:24:25.674 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:24:25 smithi134 ceph-mon[35037]: 2 slow requests, 0 included below; oldest blocked for > 4051.903256 secs 2024-03-28T19:24:25.681 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:24:25 smithi079 ceph-mon[35312]: pgmap v2401: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-03-28T19:24:25.681 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:24:25 smithi079 ceph-mon[35312]: 2 slow requests, 0 included below; oldest blocked for > 4051.903256 secs 2024-03-28T19:24:25.781 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:24:25 smithi022 ceph-mon[31985]: pgmap v2401: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-03-28T19:24:25.781 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:24:25 smithi022 ceph-mon[31985]: 2 slow requests, 0 included below; oldest blocked for > 4051.903256 secs 2024-03-28T19:24:27.673 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:24:27 smithi134 ceph-mon[35037]: pgmap v2402: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T19:24:27.681 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:24:27 smithi079 ceph-mon[35312]: pgmap v2402: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T19:24:27.781 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:24:27 smithi022 ceph-mon[31985]: pgmap v2402: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T19:24:29.674 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:24:29 smithi134 ceph-mon[35037]: pgmap v2403: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:24:29.681 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:24:29 smithi079 ceph-mon[35312]: pgmap v2403: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:24:29.781 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:24:29 smithi022 ceph-mon[31985]: pgmap v2403: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:24:30.674 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:24:30 smithi134 ceph-mon[35037]: 2 slow requests, 0 included below; oldest blocked for > 4056.903373 secs 2024-03-28T19:24:30.681 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:24:30 smithi079 ceph-mon[35312]: 2 slow requests, 0 included below; oldest blocked for > 4056.903373 secs 2024-03-28T19:24:30.781 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:24:30 smithi022 ceph-mon[31985]: 2 slow requests, 0 included below; oldest blocked for > 4056.903373 secs 2024-03-28T19:24:31.674 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:24:31 smithi134 ceph-mon[35037]: pgmap v2404: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T19:24:31.681 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:24:31 smithi079 ceph-mon[35312]: pgmap v2404: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T19:24:31.780 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:24:31 smithi022 ceph-mon[31985]: pgmap v2404: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T19:24:33.674 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:24:33 smithi134 ceph-mon[35037]: pgmap v2405: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:24:33.681 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:24:33 smithi079 ceph-mon[35312]: pgmap v2405: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:24:33.781 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:24:33 smithi022 ceph-mon[31985]: pgmap v2405: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:24:35.674 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:24:35 smithi134 ceph-mon[35037]: pgmap v2406: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T19:24:35.674 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:24:35 smithi134 ceph-mon[35037]: 2 slow requests, 0 included below; oldest blocked for > 4061.903412 secs 2024-03-28T19:24:35.681 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:24:35 smithi079 ceph-mon[35312]: pgmap v2406: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T19:24:35.681 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:24:35 smithi079 ceph-mon[35312]: 2 slow requests, 0 included below; oldest blocked for > 4061.903412 secs 2024-03-28T19:24:35.781 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:24:35 smithi022 ceph-mon[31985]: pgmap v2406: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T19:24:35.781 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:24:35 smithi022 ceph-mon[31985]: 2 slow requests, 0 included below; oldest blocked for > 4061.903412 secs 2024-03-28T19:24:37.674 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:24:37 smithi134 ceph-mon[35037]: pgmap v2407: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T19:24:37.681 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:24:37 smithi079 ceph-mon[35312]: pgmap v2407: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T19:24:37.781 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:24:37 smithi022 ceph-mon[31985]: pgmap v2407: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-28T19:24:39.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:24:39 smithi079 ceph-mon[35312]: pgmap v2408: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:24:39.781 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:24:39 smithi022 ceph-mon[31985]: pgmap v2408: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:24:39.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:24:39 smithi134 ceph-mon[35037]: pgmap v2408: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:24:40.781 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:24:40 smithi022 ceph-mon[31985]: 2 slow requests, 0 included below; oldest blocked for > 4066.903532 secs 2024-03-28T19:24:40.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:24:40 smithi134 ceph-mon[35037]: 2 slow requests, 0 included below; oldest blocked for > 4066.903532 secs 2024-03-28T19:24:40.931 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:24:40 smithi079 ceph-mon[35312]: 2 slow requests, 0 included below; oldest blocked for > 4066.903532 secs 2024-03-28T19:24:41.781 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:24:41 smithi022 ceph-mon[31985]: pgmap v2409: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T19:24:41.923 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:24:41 smithi134 ceph-mon[35037]: pgmap v2409: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T19:24:41.931 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:24:41 smithi079 ceph-mon[35312]: pgmap v2409: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T19:24:43.781 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:24:43 smithi022 ceph-mon[31985]: pgmap v2410: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:24:43.923 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:24:43 smithi134 ceph-mon[35037]: pgmap v2410: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:24:43.931 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:24:43 smithi079 ceph-mon[35312]: pgmap v2410: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:24:45.781 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:24:45 smithi022 ceph-mon[31985]: pgmap v2411: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T19:24:45.781 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:24:45 smithi022 ceph-mon[31985]: 2 slow requests, 0 included below; oldest blocked for > 4071.903609 secs 2024-03-28T19:24:45.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:24:45 smithi134 ceph-mon[35037]: pgmap v2411: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T19:24:45.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:24:45 smithi134 ceph-mon[35037]: 2 slow requests, 0 included below; oldest blocked for > 4071.903609 secs 2024-03-28T19:24:45.931 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:24:45 smithi079 ceph-mon[35312]: pgmap v2411: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T19:24:45.931 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:24:45 smithi079 ceph-mon[35312]: 2 slow requests, 0 included below; oldest blocked for > 4071.903609 secs 2024-03-28T19:24:47.781 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:24:47 smithi022 ceph-mon[31985]: pgmap v2412: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:24:47.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:24:47 smithi134 ceph-mon[35037]: pgmap v2412: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:24:47.931 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:24:47 smithi079 ceph-mon[35312]: pgmap v2412: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:24:49.781 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:24:49 smithi022 ceph-mon[31985]: pgmap v2413: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:24:49.923 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:24:49 smithi134 ceph-mon[35037]: pgmap v2413: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:24:49.931 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:24:49 smithi079 ceph-mon[35312]: pgmap v2413: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:24:50.781 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:24:50 smithi022 ceph-mon[31985]: 2 slow requests, 0 included below; oldest blocked for > 4076.903679 secs 2024-03-28T19:24:50.923 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:24:50 smithi134 ceph-mon[35037]: 2 slow requests, 0 included below; oldest blocked for > 4076.903679 secs 2024-03-28T19:24:50.931 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:24:50 smithi079 ceph-mon[35312]: 2 slow requests, 0 included below; oldest blocked for > 4076.903679 secs 2024-03-28T19:24:51.781 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:24:51 smithi022 ceph-mon[31985]: pgmap v2414: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T19:24:51.923 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:24:51 smithi134 ceph-mon[35037]: pgmap v2414: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T19:24:51.931 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:24:51 smithi079 ceph-mon[35312]: pgmap v2414: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T19:24:53.781 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:24:53 smithi022 ceph-mon[31985]: pgmap v2415: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:24:53.781 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:24:53 smithi022 ceph-mon[31985]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T19:24:53.923 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:24:53 smithi134 ceph-mon[35037]: pgmap v2415: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:24:53.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:24:53 smithi134 ceph-mon[35037]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T19:24:53.931 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:24:53 smithi079 ceph-mon[35312]: pgmap v2415: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:24:53.931 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:24:53 smithi079 ceph-mon[35312]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T19:24:54.781 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:24:54 smithi022 ceph-mon[31985]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T19:24:54.781 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:24:54 smithi022 ceph-mon[31985]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T19:24:54.781 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:24:54 smithi022 ceph-mon[31985]: from='mgr.15522 ' entity='mgr.x' 2024-03-28T19:24:54.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:24:54 smithi134 ceph-mon[35037]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T19:24:54.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:24:54 smithi134 ceph-mon[35037]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T19:24:54.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:24:54 smithi134 ceph-mon[35037]: from='mgr.15522 ' entity='mgr.x' 2024-03-28T19:24:54.931 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:24:54 smithi079 ceph-mon[35312]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T19:24:54.931 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:24:54 smithi079 ceph-mon[35312]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T19:24:54.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:24:54 smithi079 ceph-mon[35312]: from='mgr.15522 ' entity='mgr.x' 2024-03-28T19:24:55.781 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:24:55 smithi022 ceph-mon[31985]: pgmap v2416: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 6 op/s 2024-03-28T19:24:55.781 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:24:55 smithi022 ceph-mon[31985]: pgmap v2417: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 KiB/s rd, 4 op/s 2024-03-28T19:24:55.781 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:24:55 smithi022 ceph-mon[31985]: 2 slow requests, 0 included below; oldest blocked for > 4081.903794 secs 2024-03-28T19:24:55.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:24:55 smithi134 ceph-mon[35037]: pgmap v2416: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 6 op/s 2024-03-28T19:24:55.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:24:55 smithi134 ceph-mon[35037]: pgmap v2417: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 KiB/s rd, 4 op/s 2024-03-28T19:24:55.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:24:55 smithi134 ceph-mon[35037]: 2 slow requests, 0 included below; oldest blocked for > 4081.903794 secs 2024-03-28T19:24:55.931 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:24:55 smithi079 ceph-mon[35312]: pgmap v2416: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 6 op/s 2024-03-28T19:24:55.931 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:24:55 smithi079 ceph-mon[35312]: pgmap v2417: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 KiB/s rd, 4 op/s 2024-03-28T19:24:55.931 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:24:55 smithi079 ceph-mon[35312]: 2 slow requests, 0 included below; oldest blocked for > 4081.903794 secs 2024-03-28T19:24:57.781 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:24:57 smithi022 ceph-mon[31985]: pgmap v2418: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-03-28T19:24:57.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:24:57 smithi134 ceph-mon[35037]: pgmap v2418: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-03-28T19:24:57.931 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:24:57 smithi079 ceph-mon[35312]: pgmap v2418: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-03-28T19:24:59.781 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:24:59 smithi022 ceph-mon[31985]: pgmap v2419: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-03-28T19:24:59.923 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:24:59 smithi134 ceph-mon[35037]: pgmap v2419: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-03-28T19:24:59.931 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:24:59 smithi079 ceph-mon[35312]: pgmap v2419: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-03-28T19:25:00.781 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:25:00 smithi022 ceph-mon[31985]: 2 slow requests, 0 included below; oldest blocked for > 4086.903867 secs 2024-03-28T19:25:00.923 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:25:00 smithi134 ceph-mon[35037]: 2 slow requests, 0 included below; oldest blocked for > 4086.903867 secs 2024-03-28T19:25:00.931 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:25:00 smithi079 ceph-mon[35312]: 2 slow requests, 0 included below; oldest blocked for > 4086.903867 secs 2024-03-28T19:25:01.781 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:25:01 smithi022 ceph-mon[31985]: pgmap v2420: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-03-28T19:25:01.923 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:25:01 smithi134 ceph-mon[35037]: pgmap v2420: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-03-28T19:25:01.931 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:25:01 smithi079 ceph-mon[35312]: pgmap v2420: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-03-28T19:25:03.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:25:03 smithi134 ceph-mon[35037]: pgmap v2421: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-03-28T19:25:03.931 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:25:03 smithi079 ceph-mon[35312]: pgmap v2421: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-03-28T19:25:04.031 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:25:03 smithi022 ceph-mon[31985]: pgmap v2421: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-03-28T19:25:05.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:25:05 smithi134 ceph-mon[35037]: pgmap v2422: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 7 op/s 2024-03-28T19:25:05.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:25:05 smithi134 ceph-mon[35037]: 2 slow requests, 0 included below; oldest blocked for > 4091.903939 secs 2024-03-28T19:25:05.931 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:25:05 smithi079 ceph-mon[35312]: pgmap v2422: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 7 op/s 2024-03-28T19:25:05.931 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:25:05 smithi079 ceph-mon[35312]: 2 slow requests, 0 included below; oldest blocked for > 4091.903939 secs 2024-03-28T19:25:06.031 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:25:05 smithi022 ceph-mon[31985]: pgmap v2422: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 7 op/s 2024-03-28T19:25:06.031 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:25:05 smithi022 ceph-mon[31985]: 2 slow requests, 0 included below; oldest blocked for > 4091.903939 secs 2024-03-28T19:25:07.923 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:25:07 smithi134 ceph-mon[35037]: pgmap v2423: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T19:25:07.931 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:25:07 smithi079 ceph-mon[35312]: pgmap v2423: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T19:25:08.031 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:25:07 smithi022 ceph-mon[31985]: pgmap v2423: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T19:25:09.923 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:25:09 smithi134 ceph-mon[35037]: pgmap v2424: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:25:09.931 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:25:09 smithi079 ceph-mon[35312]: pgmap v2424: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:25:10.031 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:25:09 smithi022 ceph-mon[31985]: pgmap v2424: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:25:10.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:25:10 smithi134 ceph-mon[35037]: 2 slow requests, 0 included below; oldest blocked for > 4096.904012 secs 2024-03-28T19:25:10.931 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:25:10 smithi079 ceph-mon[35312]: 2 slow requests, 0 included below; oldest blocked for > 4096.904012 secs 2024-03-28T19:25:11.031 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:25:10 smithi022 ceph-mon[31985]: 2 slow requests, 0 included below; oldest blocked for > 4096.904012 secs 2024-03-28T19:25:11.923 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:25:11 smithi134 ceph-mon[35037]: pgmap v2425: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T19:25:11.931 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:25:11 smithi079 ceph-mon[35312]: pgmap v2425: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T19:25:12.030 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:25:11 smithi022 ceph-mon[31985]: pgmap v2425: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T19:25:13.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:25:13 smithi134 ceph-mon[35037]: pgmap v2426: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:25:13.931 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:25:13 smithi079 ceph-mon[35312]: pgmap v2426: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:25:14.031 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:25:13 smithi022 ceph-mon[31985]: pgmap v2426: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:25:15.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:25:15 smithi134 ceph-mon[35037]: pgmap v2427: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:25:15.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:25:15 smithi134 ceph-mon[35037]: 2 slow requests, 0 included below; oldest blocked for > 4101.904082 secs 2024-03-28T19:25:15.931 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:25:15 smithi079 ceph-mon[35312]: pgmap v2427: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:25:15.931 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:25:15 smithi079 ceph-mon[35312]: 2 slow requests, 0 included below; oldest blocked for > 4101.904082 secs 2024-03-28T19:25:16.031 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:25:15 smithi022 ceph-mon[31985]: pgmap v2427: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:25:16.031 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:25:15 smithi022 ceph-mon[31985]: 2 slow requests, 0 included below; oldest blocked for > 4101.904082 secs 2024-03-28T19:25:17.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:25:17 smithi134 ceph-mon[35037]: pgmap v2428: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T19:25:17.931 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:25:17 smithi079 ceph-mon[35312]: pgmap v2428: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T19:25:18.031 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:25:17 smithi022 ceph-mon[31985]: pgmap v2428: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T19:25:19.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:25:19 smithi134 ceph-mon[35037]: pgmap v2429: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:25:19.931 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:25:19 smithi079 ceph-mon[35312]: pgmap v2429: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:25:20.031 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:25:19 smithi022 ceph-mon[31985]: pgmap v2429: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:25:20.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:25:20 smithi134 ceph-mon[35037]: 2 slow requests, 0 included below; oldest blocked for > 4106.904177 secs 2024-03-28T19:25:20.931 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:25:20 smithi079 ceph-mon[35312]: 2 slow requests, 0 included below; oldest blocked for > 4106.904177 secs 2024-03-28T19:25:21.031 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:25:20 smithi022 ceph-mon[31985]: 2 slow requests, 0 included below; oldest blocked for > 4106.904177 secs 2024-03-28T19:25:21.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:25:21 smithi134 ceph-mon[35037]: pgmap v2430: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T19:25:21.931 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:25:21 smithi079 ceph-mon[35312]: pgmap v2430: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T19:25:21.933 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:25:21 smithi022 ceph-mon[31985]: pgmap v2430: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T19:25:23.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:25:23 smithi134 ceph-mon[35037]: pgmap v2431: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:25:23.931 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:25:23 smithi079 ceph-mon[35312]: pgmap v2431: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:25:24.031 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:25:23 smithi022 ceph-mon[31985]: pgmap v2431: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:25:25.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:25:25 smithi134 ceph-mon[35037]: pgmap v2432: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:25:25.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:25:25 smithi134 ceph-mon[35037]: 2 slow requests, 0 included below; oldest blocked for > 4111.904292 secs 2024-03-28T19:25:25.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:25:25 smithi079 ceph-mon[35312]: pgmap v2432: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:25:25.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:25:25 smithi079 ceph-mon[35312]: 2 slow requests, 0 included below; oldest blocked for > 4111.904292 secs 2024-03-28T19:25:26.031 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:25:25 smithi022 ceph-mon[31985]: pgmap v2432: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:25:26.031 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:25:25 smithi022 ceph-mon[31985]: 2 slow requests, 0 included below; oldest blocked for > 4111.904292 secs 2024-03-28T19:25:27.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:25:27 smithi134 ceph-mon[35037]: pgmap v2433: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T19:25:27.931 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:25:27 smithi079 ceph-mon[35312]: pgmap v2433: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T19:25:28.031 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:25:27 smithi022 ceph-mon[31985]: pgmap v2433: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T19:25:29.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:25:29 smithi134 ceph-mon[35037]: pgmap v2434: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:25:29.931 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:25:29 smithi079 ceph-mon[35312]: pgmap v2434: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:25:30.031 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:25:29 smithi022 ceph-mon[31985]: pgmap v2434: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:25:30.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:25:30 smithi134 ceph-mon[35037]: 2 slow requests, 0 included below; oldest blocked for > 4116.904356 secs 2024-03-28T19:25:30.931 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:25:30 smithi079 ceph-mon[35312]: 2 slow requests, 0 included below; oldest blocked for > 4116.904356 secs 2024-03-28T19:25:31.031 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:25:30 smithi022 ceph-mon[31985]: 2 slow requests, 0 included below; oldest blocked for > 4116.904356 secs 2024-03-28T19:25:31.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:25:31 smithi134 ceph-mon[35037]: pgmap v2435: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T19:25:31.931 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:25:31 smithi079 ceph-mon[35312]: pgmap v2435: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T19:25:32.031 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:25:31 smithi022 ceph-mon[31985]: pgmap v2435: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T19:25:33.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:25:33 smithi134 ceph-mon[35037]: pgmap v2436: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:25:33.931 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:25:33 smithi079 ceph-mon[35312]: pgmap v2436: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:25:34.031 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:25:33 smithi022 ceph-mon[31985]: pgmap v2436: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:25:35.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:25:35 smithi134 ceph-mon[35037]: pgmap v2437: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:25:35.925 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:25:35 smithi134 ceph-mon[35037]: 2 slow requests, 0 included below; oldest blocked for > 4121.904437 secs 2024-03-28T19:25:35.931 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:25:35 smithi079 ceph-mon[35312]: pgmap v2437: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:25:35.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:25:35 smithi079 ceph-mon[35312]: 2 slow requests, 0 included below; oldest blocked for > 4121.904437 secs 2024-03-28T19:25:36.031 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:25:35 smithi022 ceph-mon[31985]: pgmap v2437: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:25:36.031 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:25:35 smithi022 ceph-mon[31985]: 2 slow requests, 0 included below; oldest blocked for > 4121.904437 secs 2024-03-28T19:25:37.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:25:37 smithi134 ceph-mon[35037]: pgmap v2438: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T19:25:37.931 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:25:37 smithi079 ceph-mon[35312]: pgmap v2438: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T19:25:38.031 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:25:37 smithi022 ceph-mon[31985]: pgmap v2438: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T19:25:39.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:25:39 smithi134 ceph-mon[35037]: pgmap v2439: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:25:39.931 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:25:39 smithi079 ceph-mon[35312]: pgmap v2439: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:25:40.031 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:25:39 smithi022 ceph-mon[31985]: pgmap v2439: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:25:41.031 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:25:40 smithi022 ceph-mon[31985]: 2 slow requests, 0 included below; oldest blocked for > 4126.904507 secs 2024-03-28T19:25:41.174 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:25:40 smithi134 ceph-mon[35037]: 2 slow requests, 0 included below; oldest blocked for > 4126.904507 secs 2024-03-28T19:25:41.181 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:25:40 smithi079 ceph-mon[35312]: 2 slow requests, 0 included below; oldest blocked for > 4126.904507 secs 2024-03-28T19:25:42.031 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:25:41 smithi022 ceph-mon[31985]: pgmap v2440: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T19:25:42.174 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:25:41 smithi134 ceph-mon[35037]: pgmap v2440: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T19:25:42.181 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:25:41 smithi079 ceph-mon[35312]: pgmap v2440: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T19:25:44.031 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:25:43 smithi022 ceph-mon[31985]: pgmap v2441: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:25:44.174 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:25:43 smithi134 ceph-mon[35037]: pgmap v2441: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:25:44.181 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:25:43 smithi079 ceph-mon[35312]: pgmap v2441: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:25:46.031 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:25:45 smithi022 ceph-mon[31985]: pgmap v2442: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:25:46.031 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:25:45 smithi022 ceph-mon[31985]: 2 slow requests, 0 included below; oldest blocked for > 4131.904531 secs 2024-03-28T19:25:46.174 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:25:45 smithi134 ceph-mon[35037]: pgmap v2442: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:25:46.174 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:25:45 smithi134 ceph-mon[35037]: 2 slow requests, 0 included below; oldest blocked for > 4131.904531 secs 2024-03-28T19:25:46.181 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:25:45 smithi079 ceph-mon[35312]: pgmap v2442: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:25:46.182 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:25:45 smithi079 ceph-mon[35312]: 2 slow requests, 0 included below; oldest blocked for > 4131.904531 secs 2024-03-28T19:25:48.031 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:25:47 smithi022 ceph-mon[31985]: pgmap v2443: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T19:25:48.174 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:25:47 smithi134 ceph-mon[35037]: pgmap v2443: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T19:25:48.181 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:25:47 smithi079 ceph-mon[35312]: pgmap v2443: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T19:25:50.031 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:25:49 smithi022 ceph-mon[31985]: pgmap v2444: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:25:50.174 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:25:49 smithi134 ceph-mon[35037]: pgmap v2444: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:25:50.181 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:25:49 smithi079 ceph-mon[35312]: pgmap v2444: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:25:51.031 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:25:50 smithi022 ceph-mon[31985]: 2 slow requests, 0 included below; oldest blocked for > 4136.904650 secs 2024-03-28T19:25:51.174 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:25:50 smithi134 ceph-mon[35037]: 2 slow requests, 0 included below; oldest blocked for > 4136.904650 secs 2024-03-28T19:25:51.181 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:25:50 smithi079 ceph-mon[35312]: 2 slow requests, 0 included below; oldest blocked for > 4136.904650 secs 2024-03-28T19:25:52.031 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:25:51 smithi022 ceph-mon[31985]: pgmap v2445: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T19:25:52.174 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:25:51 smithi134 ceph-mon[35037]: pgmap v2445: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T19:25:52.181 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:25:51 smithi079 ceph-mon[35312]: pgmap v2445: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T19:25:54.031 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:25:53 smithi022 ceph-mon[31985]: pgmap v2446: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:25:54.174 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:25:53 smithi134 ceph-mon[35037]: pgmap v2446: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:25:54.182 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:25:53 smithi079 ceph-mon[35312]: pgmap v2446: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:25:55.031 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:25:54 smithi022 ceph-mon[31985]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T19:25:55.173 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:25:54 smithi134 ceph-mon[35037]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T19:25:55.181 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:25:54 smithi079 ceph-mon[35312]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T19:25:56.031 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:25:55 smithi022 ceph-mon[31985]: pgmap v2447: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:25:56.032 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:25:55 smithi022 ceph-mon[31985]: 2 slow requests, 0 included below; oldest blocked for > 4141.904768 secs 2024-03-28T19:25:56.032 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:25:55 smithi022 ceph-mon[31985]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T19:25:56.032 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:25:55 smithi022 ceph-mon[31985]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T19:25:56.032 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:25:55 smithi022 ceph-mon[31985]: from='mgr.15522 ' entity='mgr.x' 2024-03-28T19:25:56.174 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:25:55 smithi134 ceph-mon[35037]: pgmap v2447: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:25:56.174 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:25:55 smithi134 ceph-mon[35037]: 2 slow requests, 0 included below; oldest blocked for > 4141.904768 secs 2024-03-28T19:25:56.174 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:25:55 smithi134 ceph-mon[35037]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T19:25:56.174 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:25:55 smithi134 ceph-mon[35037]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T19:25:56.174 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:25:55 smithi134 ceph-mon[35037]: from='mgr.15522 ' entity='mgr.x' 2024-03-28T19:25:56.181 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:25:55 smithi079 ceph-mon[35312]: pgmap v2447: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:25:56.182 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:25:55 smithi079 ceph-mon[35312]: 2 slow requests, 0 included below; oldest blocked for > 4141.904768 secs 2024-03-28T19:25:56.182 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:25:55 smithi079 ceph-mon[35312]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T19:25:56.182 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:25:55 smithi079 ceph-mon[35312]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T19:25:56.182 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:25:55 smithi079 ceph-mon[35312]: from='mgr.15522 ' entity='mgr.x' 2024-03-28T19:25:57.031 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:25:56 smithi022 ceph-mon[31985]: pgmap v2448: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 10 op/s 2024-03-28T19:25:57.031 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:25:56 smithi022 ceph-mon[31985]: pgmap v2449: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-03-28T19:25:57.174 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:25:56 smithi134 ceph-mon[35037]: pgmap v2448: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 10 op/s 2024-03-28T19:25:57.174 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:25:56 smithi134 ceph-mon[35037]: pgmap v2449: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-03-28T19:25:57.182 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:25:56 smithi079 ceph-mon[35312]: pgmap v2448: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 10 op/s 2024-03-28T19:25:57.182 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:25:56 smithi079 ceph-mon[35312]: pgmap v2449: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-03-28T19:25:59.031 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:25:58 smithi022 ceph-mon[31985]: pgmap v2450: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-03-28T19:25:59.174 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:25:58 smithi134 ceph-mon[35037]: pgmap v2450: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-03-28T19:25:59.181 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:25:58 smithi079 ceph-mon[35312]: pgmap v2450: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-03-28T19:26:01.031 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:26:00 smithi022 ceph-mon[31985]: 2 slow requests, 0 included below; oldest blocked for > 4146.904793 secs 2024-03-28T19:26:01.032 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:26:00 smithi022 ceph-mon[31985]: pgmap v2451: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-03-28T19:26:01.174 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:26:00 smithi134 ceph-mon[35037]: 2 slow requests, 0 included below; oldest blocked for > 4146.904793 secs 2024-03-28T19:26:01.174 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:26:00 smithi134 ceph-mon[35037]: pgmap v2451: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-03-28T19:26:01.181 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:26:00 smithi079 ceph-mon[35312]: 2 slow requests, 0 included below; oldest blocked for > 4146.904793 secs 2024-03-28T19:26:01.181 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:26:00 smithi079 ceph-mon[35312]: pgmap v2451: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-03-28T19:26:03.174 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:26:02 smithi134 ceph-mon[35037]: pgmap v2452: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-03-28T19:26:03.181 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:26:02 smithi079 ceph-mon[35312]: pgmap v2452: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-03-28T19:26:03.281 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:26:02 smithi022 ceph-mon[31985]: pgmap v2452: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-03-28T19:26:05.174 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:26:04 smithi134 ceph-mon[35037]: pgmap v2453: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-03-28T19:26:05.181 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:26:04 smithi079 ceph-mon[35312]: pgmap v2453: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-03-28T19:26:05.281 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:26:04 smithi022 ceph-mon[31985]: pgmap v2453: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-03-28T19:26:06.174 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:26:05 smithi134 ceph-mon[35037]: 2 slow requests, 0 included below; oldest blocked for > 4151.904817 secs 2024-03-28T19:26:06.181 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:26:05 smithi079 ceph-mon[35312]: 2 slow requests, 0 included below; oldest blocked for > 4151.904817 secs 2024-03-28T19:26:06.281 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:26:05 smithi022 ceph-mon[31985]: 2 slow requests, 0 included below; oldest blocked for > 4151.904817 secs 2024-03-28T19:26:07.174 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:26:06 smithi134 ceph-mon[35037]: pgmap v2454: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 7 op/s 2024-03-28T19:26:07.182 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:26:06 smithi079 ceph-mon[35312]: pgmap v2454: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 7 op/s 2024-03-28T19:26:07.281 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:26:06 smithi022 ceph-mon[31985]: pgmap v2454: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 7 op/s 2024-03-28T19:26:09.174 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:26:08 smithi134 ceph-mon[35037]: pgmap v2455: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:26:09.181 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:26:08 smithi079 ceph-mon[35312]: pgmap v2455: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:26:09.281 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:26:08 smithi022 ceph-mon[31985]: pgmap v2455: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:26:11.174 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:26:10 smithi134 ceph-mon[35037]: 2 slow requests, 0 included below; oldest blocked for > 4156.904923 secs 2024-03-28T19:26:11.174 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:26:10 smithi134 ceph-mon[35037]: pgmap v2456: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T19:26:11.181 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:26:10 smithi079 ceph-mon[35312]: 2 slow requests, 0 included below; oldest blocked for > 4156.904923 secs 2024-03-28T19:26:11.182 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:26:10 smithi079 ceph-mon[35312]: pgmap v2456: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T19:26:11.281 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:26:10 smithi022 ceph-mon[31985]: 2 slow requests, 0 included below; oldest blocked for > 4156.904923 secs 2024-03-28T19:26:11.281 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:26:10 smithi022 ceph-mon[31985]: pgmap v2456: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T19:26:13.174 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:26:12 smithi134 ceph-mon[35037]: pgmap v2457: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:26:13.181 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:26:12 smithi079 ceph-mon[35312]: pgmap v2457: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:26:13.281 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:26:12 smithi022 ceph-mon[31985]: pgmap v2457: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:26:15.174 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:26:14 smithi134 ceph-mon[35037]: pgmap v2458: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:26:15.181 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:26:14 smithi079 ceph-mon[35312]: pgmap v2458: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:26:15.281 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:26:14 smithi022 ceph-mon[31985]: pgmap v2458: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:26:16.174 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:26:15 smithi134 ceph-mon[35037]: 2 slow requests, 0 included below; oldest blocked for > 4161.904987 secs 2024-03-28T19:26:16.182 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:26:15 smithi079 ceph-mon[35312]: 2 slow requests, 0 included below; oldest blocked for > 4161.904987 secs 2024-03-28T19:26:16.281 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:26:15 smithi022 ceph-mon[31985]: 2 slow requests, 0 included below; oldest blocked for > 4161.904987 secs 2024-03-28T19:26:17.178 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:26:16 smithi134 ceph-mon[35037]: pgmap v2459: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T19:26:17.181 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:26:16 smithi079 ceph-mon[35312]: pgmap v2459: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T19:26:17.281 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:26:16 smithi022 ceph-mon[31985]: pgmap v2459: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T19:26:19.174 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:26:18 smithi134 ceph-mon[35037]: pgmap v2460: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:26:19.181 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:26:18 smithi079 ceph-mon[35312]: pgmap v2460: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:26:19.281 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:26:18 smithi022 ceph-mon[31985]: pgmap v2460: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:26:21.174 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:26:20 smithi134 ceph-mon[35037]: 2 slow requests, 0 included below; oldest blocked for > 4166.905051 secs 2024-03-28T19:26:21.174 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:26:20 smithi134 ceph-mon[35037]: pgmap v2461: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T19:26:21.181 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:26:20 smithi079 ceph-mon[35312]: 2 slow requests, 0 included below; oldest blocked for > 4166.905051 secs 2024-03-28T19:26:21.181 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:26:20 smithi079 ceph-mon[35312]: pgmap v2461: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T19:26:21.281 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:26:20 smithi022 ceph-mon[31985]: 2 slow requests, 0 included below; oldest blocked for > 4166.905051 secs 2024-03-28T19:26:21.281 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:26:20 smithi022 ceph-mon[31985]: pgmap v2461: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T19:26:23.174 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:26:22 smithi134 ceph-mon[35037]: pgmap v2462: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:26:23.182 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:26:22 smithi079 ceph-mon[35312]: pgmap v2462: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:26:23.281 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:26:22 smithi022 ceph-mon[31985]: pgmap v2462: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:26:25.174 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:26:24 smithi134 ceph-mon[35037]: pgmap v2463: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:26:25.181 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:26:24 smithi079 ceph-mon[35312]: pgmap v2463: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:26:25.281 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:26:24 smithi022 ceph-mon[31985]: pgmap v2463: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:26:26.174 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:26:25 smithi134 ceph-mon[35037]: 2 slow requests, 0 included below; oldest blocked for > 4171.905120 secs 2024-03-28T19:26:26.181 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:26:25 smithi079 ceph-mon[35312]: 2 slow requests, 0 included below; oldest blocked for > 4171.905120 secs 2024-03-28T19:26:26.281 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:26:25 smithi022 ceph-mon[31985]: 2 slow requests, 0 included below; oldest blocked for > 4171.905120 secs 2024-03-28T19:26:27.174 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:26:26 smithi134 ceph-mon[35037]: pgmap v2464: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T19:26:27.181 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:26:26 smithi079 ceph-mon[35312]: pgmap v2464: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T19:26:27.281 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:26:26 smithi022 ceph-mon[31985]: pgmap v2464: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T19:26:28.174 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:26:27 smithi134 ceph-mon[35037]: pgmap v2465: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:26:28.181 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:26:27 smithi079 ceph-mon[35312]: pgmap v2465: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:26:28.281 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:26:27 smithi022 ceph-mon[31985]: pgmap v2465: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:26:31.174 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:26:30 smithi134 ceph-mon[35037]: 2 slow requests, 0 included below; oldest blocked for > 4176.905193 secs 2024-03-28T19:26:31.174 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:26:30 smithi134 ceph-mon[35037]: pgmap v2466: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T19:26:31.181 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:26:30 smithi079 ceph-mon[35312]: 2 slow requests, 0 included below; oldest blocked for > 4176.905193 secs 2024-03-28T19:26:31.182 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:26:30 smithi079 ceph-mon[35312]: pgmap v2466: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T19:26:31.281 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:26:30 smithi022 ceph-mon[31985]: 2 slow requests, 0 included below; oldest blocked for > 4176.905193 secs 2024-03-28T19:26:31.281 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:26:30 smithi022 ceph-mon[31985]: pgmap v2466: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T19:26:33.174 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:26:32 smithi134 ceph-mon[35037]: pgmap v2467: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:26:33.181 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:26:32 smithi079 ceph-mon[35312]: pgmap v2467: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:26:33.281 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:26:32 smithi022 ceph-mon[31985]: pgmap v2467: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:26:35.174 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:26:34 smithi134 ceph-mon[35037]: pgmap v2468: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:26:35.181 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:26:34 smithi079 ceph-mon[35312]: pgmap v2468: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:26:35.281 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:26:34 smithi022 ceph-mon[31985]: pgmap v2468: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:26:36.174 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:26:35 smithi134 ceph-mon[35037]: 2 slow requests, 0 included below; oldest blocked for > 4181.905275 secs 2024-03-28T19:26:36.181 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:26:35 smithi079 ceph-mon[35312]: 2 slow requests, 0 included below; oldest blocked for > 4181.905275 secs 2024-03-28T19:26:36.281 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:26:35 smithi022 ceph-mon[31985]: 2 slow requests, 0 included below; oldest blocked for > 4181.905275 secs 2024-03-28T19:26:37.174 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:26:36 smithi134 ceph-mon[35037]: pgmap v2469: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T19:26:37.182 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:26:36 smithi079 ceph-mon[35312]: pgmap v2469: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T19:26:37.281 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:26:36 smithi022 ceph-mon[31985]: pgmap v2469: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T19:26:39.174 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:26:38 smithi134 ceph-mon[35037]: pgmap v2470: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:26:39.181 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:26:38 smithi079 ceph-mon[35312]: pgmap v2470: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:26:39.281 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:26:38 smithi022 ceph-mon[31985]: pgmap v2470: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:26:41.174 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:26:40 smithi134 ceph-mon[35037]: 2 slow requests, 0 included below; oldest blocked for > 4186.905396 secs 2024-03-28T19:26:41.174 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:26:40 smithi134 ceph-mon[35037]: pgmap v2471: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T19:26:41.181 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:26:40 smithi079 ceph-mon[35312]: 2 slow requests, 0 included below; oldest blocked for > 4186.905396 secs 2024-03-28T19:26:41.182 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:26:40 smithi079 ceph-mon[35312]: pgmap v2471: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T19:26:41.281 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:26:40 smithi022 ceph-mon[31985]: 2 slow requests, 0 included below; oldest blocked for > 4186.905396 secs 2024-03-28T19:26:41.281 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:26:40 smithi022 ceph-mon[31985]: pgmap v2471: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T19:26:43.174 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:26:42 smithi134 ceph-mon[35037]: pgmap v2472: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:26:43.181 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:26:42 smithi079 ceph-mon[35312]: pgmap v2472: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:26:43.281 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:26:42 smithi022 ceph-mon[31985]: pgmap v2472: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:26:45.174 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:26:44 smithi134 ceph-mon[35037]: pgmap v2473: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:26:45.181 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:26:44 smithi079 ceph-mon[35312]: pgmap v2473: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:26:45.281 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:26:44 smithi022 ceph-mon[31985]: pgmap v2473: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:26:46.174 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:26:45 smithi134 ceph-mon[35037]: 2 slow requests, 0 included below; oldest blocked for > 4191.905467 secs 2024-03-28T19:26:46.181 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:26:45 smithi079 ceph-mon[35312]: 2 slow requests, 0 included below; oldest blocked for > 4191.905467 secs 2024-03-28T19:26:46.281 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:26:45 smithi022 ceph-mon[31985]: 2 slow requests, 0 included below; oldest blocked for > 4191.905467 secs 2024-03-28T19:26:47.174 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:26:46 smithi134 ceph-mon[35037]: pgmap v2474: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T19:26:47.181 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:26:46 smithi079 ceph-mon[35312]: pgmap v2474: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T19:26:47.281 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:26:46 smithi022 ceph-mon[31985]: pgmap v2474: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T19:26:49.174 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:26:48 smithi134 ceph-mon[35037]: pgmap v2475: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:26:49.181 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:26:48 smithi079 ceph-mon[35312]: pgmap v2475: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:26:49.281 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:26:48 smithi022 ceph-mon[31985]: pgmap v2475: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:26:51.174 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:26:50 smithi134 ceph-mon[35037]: 2 slow requests, 0 included below; oldest blocked for > 4196.905533 secs 2024-03-28T19:26:51.174 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:26:50 smithi134 ceph-mon[35037]: pgmap v2476: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T19:26:51.181 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:26:50 smithi079 ceph-mon[35312]: 2 slow requests, 0 included below; oldest blocked for > 4196.905533 secs 2024-03-28T19:26:51.181 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:26:50 smithi079 ceph-mon[35312]: pgmap v2476: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T19:26:51.281 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:26:50 smithi022 ceph-mon[31985]: 2 slow requests, 0 included below; oldest blocked for > 4196.905533 secs 2024-03-28T19:26:51.281 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:26:50 smithi022 ceph-mon[31985]: pgmap v2476: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T19:26:52.181 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:26:51 smithi079 ceph-mon[35312]: pgmap v2477: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:26:52.281 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:26:51 smithi022 ceph-mon[31985]: pgmap v2477: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:26:52.424 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:26:51 smithi134 ceph-mon[35037]: pgmap v2477: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:26:55.127 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:26:54 smithi134 ceph-mon[35037]: pgmap v2478: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:26:55.181 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:26:54 smithi079 ceph-mon[35312]: pgmap v2478: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:26:55.281 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:26:54 smithi022 ceph-mon[31985]: pgmap v2478: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:26:56.174 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:26:55 smithi134 ceph-mon[35037]: 2 slow requests, 0 included below; oldest blocked for > 4201.905648 secs 2024-03-28T19:26:56.181 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:26:55 smithi079 ceph-mon[35312]: 2 slow requests, 0 included below; oldest blocked for > 4201.905648 secs 2024-03-28T19:26:56.280 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:26:55 smithi022 ceph-mon[31985]: 2 slow requests, 0 included below; oldest blocked for > 4201.905648 secs 2024-03-28T19:26:57.174 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:26:56 smithi134 ceph-mon[35037]: pgmap v2479: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T19:26:57.174 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:26:56 smithi134 ceph-mon[35037]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T19:26:57.181 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:26:56 smithi079 ceph-mon[35312]: pgmap v2479: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T19:26:57.181 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:26:56 smithi079 ceph-mon[35312]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T19:26:57.280 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:26:56 smithi022 ceph-mon[31985]: pgmap v2479: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T19:26:57.281 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:26:56 smithi022 ceph-mon[31985]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T19:26:58.174 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:26:57 smithi134 ceph-mon[35037]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T19:26:58.174 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:26:57 smithi134 ceph-mon[35037]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T19:26:58.174 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:26:57 smithi134 ceph-mon[35037]: from='mgr.15522 ' entity='mgr.x' 2024-03-28T19:26:58.182 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:26:57 smithi079 ceph-mon[35312]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T19:26:58.182 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:26:57 smithi079 ceph-mon[35312]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T19:26:58.182 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:26:57 smithi079 ceph-mon[35312]: from='mgr.15522 ' entity='mgr.x' 2024-03-28T19:26:58.281 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:26:57 smithi022 ceph-mon[31985]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T19:26:58.281 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:26:57 smithi022 ceph-mon[31985]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T19:26:58.281 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:26:57 smithi022 ceph-mon[31985]: from='mgr.15522 ' entity='mgr.x' 2024-03-28T19:26:59.174 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:26:58 smithi134 ceph-mon[35037]: pgmap v2480: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 6 op/s 2024-03-28T19:26:59.174 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:26:58 smithi134 ceph-mon[35037]: pgmap v2481: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-03-28T19:26:59.181 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:26:58 smithi079 ceph-mon[35312]: pgmap v2480: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 6 op/s 2024-03-28T19:26:59.181 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:26:58 smithi079 ceph-mon[35312]: pgmap v2481: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-03-28T19:26:59.281 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:26:58 smithi022 ceph-mon[31985]: pgmap v2480: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 6 op/s 2024-03-28T19:26:59.281 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:26:58 smithi022 ceph-mon[31985]: pgmap v2481: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-03-28T19:27:01.174 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:27:00 smithi134 ceph-mon[35037]: 2 slow requests, 0 included below; oldest blocked for > 4206.905768 secs 2024-03-28T19:27:01.174 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:27:00 smithi134 ceph-mon[35037]: pgmap v2482: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-03-28T19:27:01.181 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:27:00 smithi079 ceph-mon[35312]: 2 slow requests, 0 included below; oldest blocked for > 4206.905768 secs 2024-03-28T19:27:01.181 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:27:00 smithi079 ceph-mon[35312]: pgmap v2482: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-03-28T19:27:01.281 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:27:00 smithi022 ceph-mon[31985]: 2 slow requests, 0 included below; oldest blocked for > 4206.905768 secs 2024-03-28T19:27:01.281 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:27:00 smithi022 ceph-mon[31985]: pgmap v2482: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-03-28T19:27:03.174 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:27:02 smithi134 ceph-mon[35037]: pgmap v2483: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-03-28T19:27:03.181 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:27:02 smithi079 ceph-mon[35312]: pgmap v2483: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-03-28T19:27:03.281 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:27:02 smithi022 ceph-mon[31985]: pgmap v2483: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-03-28T19:27:05.174 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:27:04 smithi134 ceph-mon[35037]: pgmap v2484: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-03-28T19:27:05.181 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:27:04 smithi079 ceph-mon[35312]: pgmap v2484: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-03-28T19:27:05.281 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:27:04 smithi022 ceph-mon[31985]: pgmap v2484: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-03-28T19:27:06.174 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:27:05 smithi134 ceph-mon[35037]: 2 slow requests, 0 included below; oldest blocked for > 4211.905804 secs 2024-03-28T19:27:06.181 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:27:05 smithi079 ceph-mon[35312]: 2 slow requests, 0 included below; oldest blocked for > 4211.905804 secs 2024-03-28T19:27:06.281 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:27:05 smithi022 ceph-mon[31985]: 2 slow requests, 0 included below; oldest blocked for > 4211.905804 secs 2024-03-28T19:27:07.174 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:27:06 smithi134 ceph-mon[35037]: pgmap v2485: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-03-28T19:27:07.181 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:27:06 smithi079 ceph-mon[35312]: pgmap v2485: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-03-28T19:27:07.281 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:27:06 smithi022 ceph-mon[31985]: pgmap v2485: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-03-28T19:27:09.174 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:27:08 smithi134 ceph-mon[35037]: pgmap v2486: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 7 op/s 2024-03-28T19:27:09.181 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:27:08 smithi079 ceph-mon[35312]: pgmap v2486: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 7 op/s 2024-03-28T19:27:09.281 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:27:08 smithi022 ceph-mon[31985]: pgmap v2486: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 7 op/s 2024-03-28T19:27:10.281 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:27:09 smithi022 ceph-mon[31985]: 2 slow requests, 0 included below; oldest blocked for > 4216.905870 secs 2024-03-28T19:27:10.281 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:27:09 smithi022 ceph-mon[31985]: pgmap v2487: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T19:27:10.424 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:27:09 smithi134 ceph-mon[35037]: 2 slow requests, 0 included below; oldest blocked for > 4216.905870 secs 2024-03-28T19:27:10.424 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:27:09 smithi134 ceph-mon[35037]: pgmap v2487: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T19:27:10.432 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:27:09 smithi079 ceph-mon[35312]: 2 slow requests, 0 included below; oldest blocked for > 4216.905870 secs 2024-03-28T19:27:10.432 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:27:09 smithi079 ceph-mon[35312]: pgmap v2487: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T19:27:12.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:27:12 smithi134 ceph-mon[35037]: pgmap v2488: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:27:12.931 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:27:12 smithi079 ceph-mon[35312]: pgmap v2488: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:27:13.031 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:27:12 smithi022 ceph-mon[31985]: pgmap v2488: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:27:14.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:27:14 smithi134 ceph-mon[35037]: pgmap v2489: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:27:14.931 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:27:14 smithi079 ceph-mon[35312]: pgmap v2489: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:27:15.031 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:27:14 smithi022 ceph-mon[31985]: pgmap v2489: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:27:15.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:27:15 smithi134 ceph-mon[35037]: 2 slow requests, 0 included below; oldest blocked for > 4221.905989 secs 2024-03-28T19:27:15.931 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:27:15 smithi079 ceph-mon[35312]: 2 slow requests, 0 included below; oldest blocked for > 4221.905989 secs 2024-03-28T19:27:16.031 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:27:15 smithi022 ceph-mon[31985]: 2 slow requests, 0 included below; oldest blocked for > 4221.905989 secs 2024-03-28T19:27:16.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:27:16 smithi134 ceph-mon[35037]: pgmap v2490: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 9 op/s 2024-03-28T19:27:16.931 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:27:16 smithi079 ceph-mon[35312]: pgmap v2490: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 9 op/s 2024-03-28T19:27:17.031 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:27:16 smithi022 ceph-mon[31985]: pgmap v2490: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 9 op/s 2024-03-28T19:27:18.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:27:18 smithi134 ceph-mon[35037]: pgmap v2491: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T19:27:18.931 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:27:18 smithi079 ceph-mon[35312]: pgmap v2491: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T19:27:19.031 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:27:18 smithi022 ceph-mon[31985]: pgmap v2491: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T19:27:20.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:27:20 smithi134 ceph-mon[35037]: 2 slow requests, 0 included below; oldest blocked for > 4226.906064 secs 2024-03-28T19:27:20.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:27:20 smithi134 ceph-mon[35037]: pgmap v2492: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 9 op/s 2024-03-28T19:27:20.931 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:27:20 smithi079 ceph-mon[35312]: 2 slow requests, 0 included below; oldest blocked for > 4226.906064 secs 2024-03-28T19:27:20.931 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:27:20 smithi079 ceph-mon[35312]: pgmap v2492: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 9 op/s 2024-03-28T19:27:21.031 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:27:20 smithi022 ceph-mon[31985]: 2 slow requests, 0 included below; oldest blocked for > 4226.906064 secs 2024-03-28T19:27:21.031 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:27:20 smithi022 ceph-mon[31985]: pgmap v2492: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 9 op/s 2024-03-28T19:27:22.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:27:22 smithi134 ceph-mon[35037]: pgmap v2493: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T19:27:22.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:27:22 smithi079 ceph-mon[35312]: pgmap v2493: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T19:27:23.031 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:27:22 smithi022 ceph-mon[31985]: pgmap v2493: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T19:27:24.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:27:24 smithi134 ceph-mon[35037]: pgmap v2494: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T19:27:24.931 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:27:24 smithi079 ceph-mon[35312]: pgmap v2494: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T19:27:25.031 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:27:24 smithi022 ceph-mon[31985]: pgmap v2494: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T19:27:25.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:27:25 smithi134 ceph-mon[35037]: 2 slow requests, 0 included below; oldest blocked for > 4231.906132 secs 2024-03-28T19:27:25.931 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:27:25 smithi079 ceph-mon[35312]: 2 slow requests, 0 included below; oldest blocked for > 4231.906132 secs 2024-03-28T19:27:26.031 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:27:25 smithi022 ceph-mon[31985]: 2 slow requests, 0 included below; oldest blocked for > 4231.906132 secs 2024-03-28T19:27:26.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:27:26 smithi134 ceph-mon[35037]: pgmap v2495: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 9 op/s 2024-03-28T19:27:26.931 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:27:26 smithi079 ceph-mon[35312]: pgmap v2495: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 9 op/s 2024-03-28T19:27:27.031 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:27:26 smithi022 ceph-mon[31985]: pgmap v2495: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 9 op/s 2024-03-28T19:27:28.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:27:28 smithi134 ceph-mon[35037]: pgmap v2496: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:27:28.931 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:27:28 smithi079 ceph-mon[35312]: pgmap v2496: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:27:29.031 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:27:28 smithi022 ceph-mon[31985]: pgmap v2496: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:27:30.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:27:30 smithi134 ceph-mon[35037]: 2 slow requests, 0 included below; oldest blocked for > 4236.906200 secs 2024-03-28T19:27:30.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:27:30 smithi134 ceph-mon[35037]: pgmap v2497: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T19:27:30.931 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:27:30 smithi079 ceph-mon[35312]: 2 slow requests, 0 included below; oldest blocked for > 4236.906200 secs 2024-03-28T19:27:30.931 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:27:30 smithi079 ceph-mon[35312]: pgmap v2497: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T19:27:31.031 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:27:30 smithi022 ceph-mon[31985]: 2 slow requests, 0 included below; oldest blocked for > 4236.906200 secs 2024-03-28T19:27:31.031 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:27:30 smithi022 ceph-mon[31985]: pgmap v2497: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T19:27:32.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:27:32 smithi134 ceph-mon[35037]: pgmap v2498: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:27:32.931 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:27:32 smithi079 ceph-mon[35312]: pgmap v2498: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:27:33.031 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:27:32 smithi022 ceph-mon[31985]: pgmap v2498: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:27:34.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:27:34 smithi134 ceph-mon[35037]: pgmap v2499: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:27:34.931 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:27:34 smithi079 ceph-mon[35312]: pgmap v2499: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:27:35.031 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:27:34 smithi022 ceph-mon[31985]: pgmap v2499: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:27:35.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:27:35 smithi134 ceph-mon[35037]: 2 slow requests, 0 included below; oldest blocked for > 4241.906250 secs 2024-03-28T19:27:35.931 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:27:35 smithi079 ceph-mon[35312]: 2 slow requests, 0 included below; oldest blocked for > 4241.906250 secs 2024-03-28T19:27:36.031 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:27:35 smithi022 ceph-mon[31985]: 2 slow requests, 0 included below; oldest blocked for > 4241.906250 secs 2024-03-28T19:27:36.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:27:36 smithi134 ceph-mon[35037]: pgmap v2500: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T19:27:36.931 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:27:36 smithi079 ceph-mon[35312]: pgmap v2500: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T19:27:37.031 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:27:36 smithi022 ceph-mon[31985]: pgmap v2500: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T19:27:38.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:27:38 smithi134 ceph-mon[35037]: pgmap v2501: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:27:38.931 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:27:38 smithi079 ceph-mon[35312]: pgmap v2501: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:27:39.031 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:27:38 smithi022 ceph-mon[31985]: pgmap v2501: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:27:40.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:27:40 smithi134 ceph-mon[35037]: 2 slow requests, 0 included below; oldest blocked for > 4246.906364 secs 2024-03-28T19:27:40.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:27:40 smithi134 ceph-mon[35037]: pgmap v2502: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 9 op/s 2024-03-28T19:27:40.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:27:40 smithi079 ceph-mon[35312]: 2 slow requests, 0 included below; oldest blocked for > 4246.906364 secs 2024-03-28T19:27:40.932 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:27:40 smithi079 ceph-mon[35312]: pgmap v2502: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 9 op/s 2024-03-28T19:27:41.031 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:27:40 smithi022 ceph-mon[31985]: 2 slow requests, 0 included below; oldest blocked for > 4246.906364 secs 2024-03-28T19:27:41.031 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:27:40 smithi022 ceph-mon[31985]: pgmap v2502: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 9 op/s 2024-03-28T19:27:42.924 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:27:42 smithi134 ceph-mon[35037]: pgmap v2503: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T19:27:42.931 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:27:42 smithi079 ceph-mon[35312]: pgmap v2503: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T19:27:43.031 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:27:42 smithi022 ceph-mon[31985]: pgmap v2503: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T19:27:44.931 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:27:44 smithi079 ceph-mon[35312]: pgmap v2504: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T19:27:45.031 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:27:44 smithi022 ceph-mon[31985]: pgmap v2504: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T19:27:45.174 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:27:44 smithi134 ceph-mon[35037]: pgmap v2504: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T19:27:46.031 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:27:45 smithi022 ceph-mon[31985]: 2 slow requests, 0 included below; oldest blocked for > 4251.906439 secs 2024-03-28T19:27:46.174 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:27:45 smithi134 ceph-mon[35037]: 2 slow requests, 0 included below; oldest blocked for > 4251.906439 secs 2024-03-28T19:27:46.181 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:27:45 smithi079 ceph-mon[35312]: 2 slow requests, 0 included below; oldest blocked for > 4251.906439 secs 2024-03-28T19:27:47.033 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:27:46 smithi022 ceph-mon[31985]: pgmap v2505: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 9 op/s 2024-03-28T19:27:47.174 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:27:46 smithi134 ceph-mon[35037]: pgmap v2505: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 9 op/s 2024-03-28T19:27:47.181 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:27:46 smithi079 ceph-mon[35312]: pgmap v2505: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 9 op/s 2024-03-28T19:27:49.031 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:27:48 smithi022 ceph-mon[31985]: pgmap v2506: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T19:27:49.174 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:27:48 smithi134 ceph-mon[35037]: pgmap v2506: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T19:27:49.181 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:27:48 smithi079 ceph-mon[35312]: pgmap v2506: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-28T19:27:51.031 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:27:50 smithi022 ceph-mon[31985]: 2 slow requests, 0 included below; oldest blocked for > 4256.906501 secs 2024-03-28T19:27:51.031 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:27:50 smithi022 ceph-mon[31985]: pgmap v2507: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 9 op/s 2024-03-28T19:27:51.174 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:27:50 smithi134 ceph-mon[35037]: 2 slow requests, 0 included below; oldest blocked for > 4256.906501 secs 2024-03-28T19:27:51.174 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:27:50 smithi134 ceph-mon[35037]: pgmap v2507: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 9 op/s 2024-03-28T19:27:51.182 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:27:50 smithi079 ceph-mon[35312]: 2 slow requests, 0 included below; oldest blocked for > 4256.906501 secs 2024-03-28T19:27:51.182 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:27:50 smithi079 ceph-mon[35312]: pgmap v2507: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 9 op/s 2024-03-28T19:27:53.031 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:27:52 smithi022 ceph-mon[31985]: pgmap v2508: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:27:53.173 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:27:52 smithi134 ceph-mon[35037]: pgmap v2508: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:27:53.181 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:27:52 smithi079 ceph-mon[35312]: pgmap v2508: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:27:55.031 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:27:54 smithi022 ceph-mon[31985]: pgmap v2509: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:27:55.174 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:27:54 smithi134 ceph-mon[35037]: pgmap v2509: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:27:55.181 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:27:54 smithi079 ceph-mon[35312]: pgmap v2509: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:27:56.031 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:27:55 smithi022 ceph-mon[31985]: 2 slow requests, 0 included below; oldest blocked for > 4261.906619 secs 2024-03-28T19:27:56.076 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:27:55 smithi134 ceph-mon[35037]: 2 slow requests, 0 included below; oldest blocked for > 4261.906619 secs 2024-03-28T19:27:56.181 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:27:55 smithi079 ceph-mon[35312]: 2 slow requests, 0 included below; oldest blocked for > 4261.906619 secs 2024-03-28T19:27:57.031 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:27:56 smithi022 ceph-mon[31985]: pgmap v2510: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T19:27:57.174 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:27:56 smithi134 ceph-mon[35037]: pgmap v2510: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T19:27:57.182 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:27:56 smithi079 ceph-mon[35312]: pgmap v2510: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T19:27:59.031 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:27:58 smithi022 ceph-mon[31985]: pgmap v2511: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:27:59.031 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:27:58 smithi022 ceph-mon[31985]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T19:27:59.174 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:27:58 smithi134 ceph-mon[35037]: pgmap v2511: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:27:59.174 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:27:58 smithi134 ceph-mon[35037]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T19:27:59.181 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:27:58 smithi079 ceph-mon[35312]: pgmap v2511: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:27:59.181 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:27:58 smithi079 ceph-mon[35312]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-28T19:28:00.031 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:27:59 smithi022 ceph-mon[31985]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T19:28:00.031 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:27:59 smithi022 ceph-mon[31985]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T19:28:00.031 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:27:59 smithi022 ceph-mon[31985]: from='mgr.15522 ' entity='mgr.x' 2024-03-28T19:28:00.174 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:27:59 smithi134 ceph-mon[35037]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T19:28:00.174 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:27:59 smithi134 ceph-mon[35037]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T19:28:00.174 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:27:59 smithi134 ceph-mon[35037]: from='mgr.15522 ' entity='mgr.x' 2024-03-28T19:28:00.182 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:27:59 smithi079 ceph-mon[35312]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-28T19:28:00.182 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:27:59 smithi079 ceph-mon[35312]: from='mgr.15522 172.21.15.22:0/10650448' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-28T19:28:00.182 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:27:59 smithi079 ceph-mon[35312]: from='mgr.15522 ' entity='mgr.x' 2024-03-28T19:28:01.031 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:28:00 smithi022 ceph-mon[31985]: pgmap v2512: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 8 op/s 2024-03-28T19:28:01.031 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:28:00 smithi022 ceph-mon[31985]: pgmap v2513: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T19:28:01.031 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:28:00 smithi022 ceph-mon[31985]: 2 slow requests, 0 included below; oldest blocked for > 4266.906684 secs 2024-03-28T19:28:01.174 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:28:00 smithi134 ceph-mon[35037]: pgmap v2512: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 8 op/s 2024-03-28T19:28:01.174 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:28:00 smithi134 ceph-mon[35037]: pgmap v2513: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T19:28:01.174 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:28:00 smithi134 ceph-mon[35037]: 2 slow requests, 0 included below; oldest blocked for > 4266.906684 secs 2024-03-28T19:28:01.181 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:28:00 smithi079 ceph-mon[35312]: pgmap v2512: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 8 op/s 2024-03-28T19:28:01.182 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:28:00 smithi079 ceph-mon[35312]: pgmap v2513: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-28T19:28:01.182 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:28:00 smithi079 ceph-mon[35312]: 2 slow requests, 0 included below; oldest blocked for > 4266.906684 secs 2024-03-28T19:28:03.031 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:28:02 smithi022 ceph-mon[31985]: pgmap v2514: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-03-28T19:28:03.173 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:28:02 smithi134 ceph-mon[35037]: pgmap v2514: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-03-28T19:28:03.181 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:28:02 smithi079 ceph-mon[35312]: pgmap v2514: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-03-28T19:28:05.031 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:28:04 smithi022 ceph-mon[31985]: pgmap v2515: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-03-28T19:28:05.174 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:28:04 smithi134 ceph-mon[35037]: pgmap v2515: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-03-28T19:28:05.182 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:28:04 smithi079 ceph-mon[35312]: pgmap v2515: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-03-28T19:28:06.031 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:28:05 smithi022 ceph-mon[31985]: 2 slow requests, 0 included below; oldest blocked for > 4271.906748 secs 2024-03-28T19:28:06.173 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:28:05 smithi134 ceph-mon[35037]: 2 slow requests, 0 included below; oldest blocked for > 4271.906748 secs 2024-03-28T19:28:06.181 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:28:05 smithi079 ceph-mon[35312]: 2 slow requests, 0 included below; oldest blocked for > 4271.906748 secs 2024-03-28T19:28:07.031 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:28:06 smithi022 ceph-mon[31985]: pgmap v2516: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-03-28T19:28:07.174 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:28:06 smithi134 ceph-mon[35037]: pgmap v2516: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-03-28T19:28:07.181 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:28:06 smithi079 ceph-mon[35312]: pgmap v2516: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-03-28T19:28:09.174 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:28:08 smithi134 ceph-mon[35037]: pgmap v2517: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-03-28T19:28:09.181 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:28:08 smithi079 ceph-mon[35312]: pgmap v2517: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-03-28T19:28:09.281 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:28:08 smithi022 ceph-mon[31985]: pgmap v2517: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-03-28T19:28:11.174 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:28:10 smithi134 ceph-mon[35037]: pgmap v2518: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 7 op/s 2024-03-28T19:28:11.174 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:28:10 smithi134 ceph-mon[35037]: 2 slow requests, 0 included below; oldest blocked for > 4276.906792 secs 2024-03-28T19:28:11.181 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:28:10 smithi079 ceph-mon[35312]: pgmap v2518: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 7 op/s 2024-03-28T19:28:11.182 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:28:10 smithi079 ceph-mon[35312]: 2 slow requests, 0 included below; oldest blocked for > 4276.906792 secs 2024-03-28T19:28:11.281 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:28:10 smithi022 ceph-mon[31985]: pgmap v2518: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 7 op/s 2024-03-28T19:28:11.281 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:28:10 smithi022 ceph-mon[31985]: 2 slow requests, 0 included below; oldest blocked for > 4276.906792 secs 2024-03-28T19:28:13.174 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:28:12 smithi134 ceph-mon[35037]: pgmap v2519: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 7 op/s 2024-03-28T19:28:13.181 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:28:12 smithi079 ceph-mon[35312]: pgmap v2519: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 7 op/s 2024-03-28T19:28:13.281 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:28:12 smithi022 ceph-mon[31985]: pgmap v2519: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 7 op/s 2024-03-28T19:28:15.174 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:28:14 smithi134 ceph-mon[35037]: pgmap v2520: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:28:15.181 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:28:14 smithi079 ceph-mon[35312]: pgmap v2520: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:28:15.281 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:28:14 smithi022 ceph-mon[31985]: pgmap v2520: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:28:16.174 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:28:15 smithi134 ceph-mon[35037]: 2 slow requests, 0 included below; oldest blocked for > 4281.906867 secs 2024-03-28T19:28:16.181 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:28:15 smithi079 ceph-mon[35312]: 2 slow requests, 0 included below; oldest blocked for > 4281.906867 secs 2024-03-28T19:28:16.281 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:28:15 smithi022 ceph-mon[31985]: 2 slow requests, 0 included below; oldest blocked for > 4281.906867 secs 2024-03-28T19:28:17.174 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:28:16 smithi134 ceph-mon[35037]: pgmap v2521: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T19:28:17.181 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:28:16 smithi079 ceph-mon[35312]: pgmap v2521: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T19:28:17.281 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:28:16 smithi022 ceph-mon[31985]: pgmap v2521: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T19:28:19.174 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:28:18 smithi134 ceph-mon[35037]: pgmap v2522: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:28:19.181 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:28:18 smithi079 ceph-mon[35312]: pgmap v2522: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:28:19.281 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:28:18 smithi022 ceph-mon[31985]: pgmap v2522: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:28:21.174 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:28:20 smithi134 ceph-mon[35037]: pgmap v2523: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-28T19:28:21.174 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:28:20 smithi134 ceph-mon[35037]: 2 slow requests, 0 included below; oldest blocked for > 4286.906934 secs 2024-03-28T19:28:21.181 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:28:20 smithi079 ceph-mon[35312]: pgmap v2523: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-28T19:28:21.182 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:28:20 smithi079 ceph-mon[35312]: 2 slow requests, 0 included below; oldest blocked for > 4286.906934 secs 2024-03-28T19:28:21.281 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:28:20 smithi022 ceph-mon[31985]: pgmap v2523: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-28T19:28:21.281 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:28:20 smithi022 ceph-mon[31985]: 2 slow requests, 0 included below; oldest blocked for > 4286.906934 secs 2024-03-28T19:28:23.084 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:28:22 smithi022 ceph-mon[31985]: pgmap v2524: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 7 op/s 2024-03-28T19:28:23.173 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:28:22 smithi134 ceph-mon[35037]: pgmap v2524: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 7 op/s 2024-03-28T19:28:23.181 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:28:22 smithi079 ceph-mon[35312]: pgmap v2524: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 7 op/s 2024-03-28T19:28:25.174 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:28:24 smithi134 ceph-mon[35037]: pgmap v2525: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:28:25.181 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:28:24 smithi079 ceph-mon[35312]: pgmap v2525: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:28:25.281 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:28:24 smithi022 ceph-mon[31985]: pgmap v2525: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:28:26.174 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:28:25 smithi134 ceph-mon[35037]: 2 slow requests, 0 included below; oldest blocked for > 4291.907013 secs 2024-03-28T19:28:26.181 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:28:25 smithi079 ceph-mon[35312]: 2 slow requests, 0 included below; oldest blocked for > 4291.907013 secs 2024-03-28T19:28:26.281 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:28:25 smithi022 ceph-mon[31985]: 2 slow requests, 0 included below; oldest blocked for > 4291.907013 secs 2024-03-28T19:28:27.174 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:28:26 smithi134 ceph-mon[35037]: pgmap v2526: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T19:28:27.181 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:28:26 smithi079 ceph-mon[35312]: pgmap v2526: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T19:28:27.281 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:28:26 smithi022 ceph-mon[31985]: pgmap v2526: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T19:28:29.174 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:28:28 smithi134 ceph-mon[35037]: pgmap v2527: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:28:29.181 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:28:28 smithi079 ceph-mon[35312]: pgmap v2527: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:28:29.281 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:28:28 smithi022 ceph-mon[31985]: pgmap v2527: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:28:31.174 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:28:30 smithi134 ceph-mon[35037]: pgmap v2528: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-28T19:28:31.174 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:28:30 smithi134 ceph-mon[35037]: 2 slow requests, 0 included below; oldest blocked for > 4296.907047 secs 2024-03-28T19:28:31.182 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:28:30 smithi079 ceph-mon[35312]: pgmap v2528: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-28T19:28:31.182 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:28:30 smithi079 ceph-mon[35312]: 2 slow requests, 0 included below; oldest blocked for > 4296.907047 secs 2024-03-28T19:28:31.281 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:28:30 smithi022 ceph-mon[31985]: pgmap v2528: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-28T19:28:31.281 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:28:30 smithi022 ceph-mon[31985]: 2 slow requests, 0 included below; oldest blocked for > 4296.907047 secs 2024-03-28T19:28:33.174 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:28:32 smithi134 ceph-mon[35037]: pgmap v2529: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 7 op/s 2024-03-28T19:28:33.181 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:28:32 smithi079 ceph-mon[35312]: pgmap v2529: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 7 op/s 2024-03-28T19:28:33.281 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:28:32 smithi022 ceph-mon[31985]: pgmap v2529: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 7 op/s 2024-03-28T19:28:35.174 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:28:34 smithi134 ceph-mon[35037]: pgmap v2530: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:28:35.181 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:28:34 smithi079 ceph-mon[35312]: pgmap v2530: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:28:35.281 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:28:34 smithi022 ceph-mon[31985]: pgmap v2530: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:28:36.174 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:28:35 smithi134 ceph-mon[35037]: 2 slow requests, 0 included below; oldest blocked for > 4301.907109 secs 2024-03-28T19:28:36.181 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:28:35 smithi079 ceph-mon[35312]: 2 slow requests, 0 included below; oldest blocked for > 4301.907109 secs 2024-03-28T19:28:36.281 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:28:35 smithi022 ceph-mon[31985]: 2 slow requests, 0 included below; oldest blocked for > 4301.907109 secs 2024-03-28T19:28:37.174 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:28:36 smithi134 ceph-mon[35037]: pgmap v2531: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T19:28:37.181 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:28:36 smithi079 ceph-mon[35312]: pgmap v2531: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T19:28:37.281 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:28:36 smithi022 ceph-mon[31985]: pgmap v2531: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T19:28:39.174 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:28:38 smithi134 ceph-mon[35037]: pgmap v2532: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:28:39.181 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:28:38 smithi079 ceph-mon[35312]: pgmap v2532: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:28:39.281 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:28:38 smithi022 ceph-mon[31985]: pgmap v2532: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:28:40.174 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:28:39 smithi134 ceph-mon[35037]: 2 slow requests, 0 included below; oldest blocked for > 4306.907234 secs 2024-03-28T19:28:40.181 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:28:39 smithi079 ceph-mon[35312]: 2 slow requests, 0 included below; oldest blocked for > 4306.907234 secs 2024-03-28T19:28:40.281 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:28:39 smithi022 ceph-mon[31985]: 2 slow requests, 0 included below; oldest blocked for > 4306.907234 secs 2024-03-28T19:28:41.174 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:28:40 smithi134 ceph-mon[35037]: pgmap v2533: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-28T19:28:41.181 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:28:40 smithi079 ceph-mon[35312]: pgmap v2533: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-28T19:28:41.281 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:28:40 smithi022 ceph-mon[31985]: pgmap v2533: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-28T19:28:42.281 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:28:41 smithi022 ceph-mon[31985]: pgmap v2534: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 7 op/s 2024-03-28T19:28:42.424 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:28:41 smithi134 ceph-mon[35037]: pgmap v2534: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 7 op/s 2024-03-28T19:28:42.431 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:28:41 smithi079 ceph-mon[35312]: pgmap v2534: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 7 op/s 2024-03-28T19:28:44.674 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:28:44 smithi134 ceph-mon[35037]: pgmap v2535: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:28:44.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:28:44 smithi079 ceph-mon[35312]: pgmap v2535: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:28:44.781 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:28:44 smithi022 ceph-mon[31985]: pgmap v2535: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:28:45.674 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:28:45 smithi134 ceph-mon[35037]: 2 slow requests, 0 included below; oldest blocked for > 4311.907310 secs 2024-03-28T19:28:45.681 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:28:45 smithi079 ceph-mon[35312]: 2 slow requests, 0 included below; oldest blocked for > 4311.907310 secs 2024-03-28T19:28:45.781 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:28:45 smithi022 ceph-mon[31985]: 2 slow requests, 0 included below; oldest blocked for > 4311.907310 secs 2024-03-28T19:28:46.674 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:28:46 smithi134 ceph-mon[35037]: pgmap v2536: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T19:28:46.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:28:46 smithi079 ceph-mon[35312]: pgmap v2536: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T19:28:46.781 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:28:46 smithi022 ceph-mon[31985]: pgmap v2536: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-28T19:28:48.674 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:28:48 smithi134 ceph-mon[35037]: pgmap v2537: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:28:48.681 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:28:48 smithi079 ceph-mon[35312]: pgmap v2537: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:28:48.781 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:28:48 smithi022 ceph-mon[31985]: pgmap v2537: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-28T19:28:50.674 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:28:50 smithi134 ceph-mon[35037]: pgmap v2538: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-28T19:28:50.674 INFO:journalctl@ceph.mon.c.smithi134.stdout:Mar 28 19:28:50 smithi134 ceph-mon[35037]: 2 slow requests, 0 included below; oldest blocked for > 4316.907386 secs 2024-03-28T19:28:50.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:28:50 smithi079 ceph-mon[35312]: pgmap v2538: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-28T19:28:50.682 INFO:journalctl@ceph.mon.b.smithi079.stdout:Mar 28 19:28:50 smithi079 ceph-mon[35312]: 2 slow requests, 0 included below; oldest blocked for > 4316.907386 secs 2024-03-28T19:28:50.781 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:28:50 smithi022 ceph-mon[31985]: pgmap v2538: 97 pgs: 97 active+clean; 118 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-28T19:28:50.781 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 28 19:28:50 smithi022 ceph-mon[31985]: 2 slow requests, 0 included below; oldest blocked for > 4316.907386 secs 2024-03-28T19:28:51.714 DEBUG:teuthology.exit:Got signal 15; running 1 handler... 2024-03-28T19:28:51.715 DEBUG:teuthology.task.console_log:Killing console logger for smithi022 2024-03-28T19:28:51.716 DEBUG:teuthology.task.console_log:Killing console logger for smithi079 2024-03-28T19:28:51.716 DEBUG:teuthology.task.console_log:Killing console logger for smithi134 2024-03-28T19:28:51.716 DEBUG:teuthology.exit:Finished running handlers